2024-04-04T05:51:48.295 INFO:root:teuthology version: 0.0.1.dev265+gd7d6e60 2024-04-04T05:51:48.295 DEBUG:teuthology.run:Teuthology command: teuthology --name vshankar-2024-04-04_05:15:36-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi --archive /home/teuthworker/archive/vshankar-2024-04-04_05:15:36-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi/7639210 --owner scheduled_vshankar@teuthology --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} -- /home/teuthworker/archive/vshankar-2024-04-04_05:15:36-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi/7639210/orig.config.yaml 2024-04-04T05:51:48.345 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-04T05:51:48.438 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-04-04_05:15:36-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi/7639210 branch: wip-vshankar-testing-20240330.172700 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} email: vshankar@redhat.com first_in_suite: false job_id: '7639210' kernel: client: branch: testing flavor: default kdb: true sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: vshankar-2024-04-04_05:15:36-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240330.172700 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: addbc488dc763e752c640619b677f10e0da02c9f ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: addbc488dc763e752c640619b677f10e0da02c9f kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240330.172700 sha1: addbc488dc763e752c640619b677f10e0da02c9f owner: scheduled_vshankar@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3016 sha1: addbc488dc763e752c640619b677f10e0da02c9f sleep_before_teardown: 0 subset: 140/1024 suite: fs suite_branch: wip-vshankar-testing-20240330.172700 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_addbc488dc763e752c640619b677f10e0da02c9f/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: addbc488dc763e752c640619b677f10e0da02c9f syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi159.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI9+sH1zuQF/EKfdRkBUOY6pUkyed9HhnOnkKrOygkBuTyZsrgsl2SJcEyeid/oNbaQwHgtR9HSF5IzgfoYOAgM= smithi178.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI75ZPTr++WrvmTssgpUy8yMeTqSpAYeKCfe+sxpb8CHITh35dNVY8L6Ku/faFCk6orumF6zBTI9iJ1DATOBKxI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: d7d6e60867cadf19fbfd7655a551ae7d58060757 timestamp: 2024-04-04_05:15:36 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-04T05:51:48.438 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_addbc488dc763e752c640619b677f10e0da02c9f/qa; will attempt to use it 2024-04-04T05:51:48.439 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_addbc488dc763e752c640619b677f10e0da02c9f/qa/tasks 2024-04-04T05:51:48.439 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-04T05:51:48.440 INFO:teuthology.task.internal:Checking packages... 2024-04-04T05:51:48.465 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'addbc488dc763e752c640619b677f10e0da02c9f' 2024-04-04T05:51:48.465 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-04T05:51:48.466 INFO:teuthology.packaging:ref: None 2024-04-04T05:51:48.466 INFO:teuthology.packaging:tag: None 2024-04-04T05:51:48.466 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240330.172700 2024-04-04T05:51:48.466 INFO:teuthology.packaging:sha1: addbc488dc763e752c640619b677f10e0da02c9f 2024-04-04T05:51:48.466 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240330.172700 2024-04-04T05:51:48.607 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2571.gaddbc488 2024-04-04T05:51:48.609 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-04T05:51:48.615 INFO:teuthology.task.internal:no buildpackages task found 2024-04-04T05:51:48.615 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-04T05:51:48.620 INFO:teuthology.task.internal:Saving configuration 2024-04-04T05:51:48.637 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-04T05:51:48.643 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-04T05:51:48.665 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-04_05:15:36-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi/7639210', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-04 05:46:31.995958', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI9+sH1zuQF/EKfdRkBUOY6pUkyed9HhnOnkKrOygkBuTyZsrgsl2SJcEyeid/oNbaQwHgtR9HSF5IzgfoYOAgM='} 2024-04-04T05:51:48.687 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-04_05:15:36-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi/7639210', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-04 05:46:31.994384', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI75ZPTr++WrvmTssgpUy8yMeTqSpAYeKCfe+sxpb8CHITh35dNVY8L6Ku/faFCk6orumF6zBTI9iJ1DATOBKxI='} 2024-04-04T05:51:48.687 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-04T05:51:48.692 INFO:teuthology.task.internal:roles: ubuntu@smithi159.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-04T05:51:48.693 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-04T05:51:48.693 INFO:teuthology.run_tasks:Running task console_log... 2024-04-04T05:51:48.757 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f31b80fb160>, signals=[15]) 2024-04-04T05:51:48.757 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-04T05:51:48.763 INFO:teuthology.task.internal:Opening connections... 2024-04-04T05:51:48.763 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T05:51:48.765 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-04T05:51:48.854 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi178.front.sepia.ceph.com 2024-04-04T05:51:48.855 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-04T05:51:48.950 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-04T05:51:48.956 DEBUG:teuthology.orchestra.run.smithi159:> uname -m 2024-04-04T05:51:48.974 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2024-04-04T05:51:48.974 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2024-04-04T05:51:49.031 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream" 2024-04-04T05:51:49.031 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="9" 2024-04-04T05:51:49.031 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2024-04-04T05:51:49.032 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2024-04-04T05:51:49.032 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="9" 2024-04-04T05:51:49.032 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el9" 2024-04-04T05:51:49.032 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-04T05:51:49.032 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2024-04-04T05:51:49.032 INFO:teuthology.orchestra.run.smithi159.stdout:LOGO="fedora-logo-icon" 2024-04-04T05:51:49.032 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-04T05:51:49.032 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2024-04-04T05:51:49.032 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-04T05:51:49.032 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-04T05:51:49.032 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-04T05:51:49.033 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2024-04-04T05:51:49.059 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2024-04-04T05:51:49.076 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2024-04-04T05:51:49.076 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2024-04-04T05:51:49.133 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="CentOS Stream" 2024-04-04T05:51:49.133 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="9" 2024-04-04T05:51:49.133 INFO:teuthology.orchestra.run.smithi178.stdout:ID="centos" 2024-04-04T05:51:49.133 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="rhel fedora" 2024-04-04T05:51:49.133 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="9" 2024-04-04T05:51:49.133 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el9" 2024-04-04T05:51:49.133 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-04T05:51:49.134 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2024-04-04T05:51:49.134 INFO:teuthology.orchestra.run.smithi178.stdout:LOGO="fedora-logo-icon" 2024-04-04T05:51:49.134 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-04T05:51:49.134 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://centos.org/" 2024-04-04T05:51:49.134 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-04T05:51:49.134 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-04T05:51:49.134 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-04T05:51:49.134 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2024-04-04T05:51:49.159 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-04T05:51:49.168 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-04T05:51:49.174 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-04T05:51:49.175 DEBUG:teuthology.orchestra.run.smithi159:> test '!' -e /home/ubuntu/cephtest 2024-04-04T05:51:49.177 DEBUG:teuthology.orchestra.run.smithi178:> test '!' -e /home/ubuntu/cephtest 2024-04-04T05:51:49.195 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-04T05:51:49.203 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-04T05:51:49.204 DEBUG:teuthology.orchestra.run.smithi159:> test -z $(ls -A /var/lib/ceph) 2024-04-04T05:51:49.237 DEBUG:teuthology.orchestra.run.smithi178:> test -z $(ls -A /var/lib/ceph) 2024-04-04T05:51:49.264 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-04T05:51:49.321 INFO:teuthology.run_tasks:Running task kernel... 2024-04-04T05:51:49.337 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'} 2024-04-04T05:51:49.337 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-04-04T05:51:49.337 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-04-04T05:51:49.337 DEBUG:teuthology.orchestra.run.smithi159:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-04T05:51:49.338 DEBUG:teuthology.orchestra.run.smithi178:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-04T05:51:49.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-04T05:51:49.354 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-04T05:51:49.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-04T05:51:49.357 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-04T05:51:49.522 DEBUG:teuthology.task.kernel:sha1 for client.0 is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-04-04T05:51:49.522 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-04-04T05:51:49.522 DEBUG:teuthology.orchestra.run.smithi159:> uname -r 2024-04-04T05:51:49.539 INFO:teuthology.orchestra.run.smithi159.stdout:5.14.0-361.el9.x86_64 2024-04-04T05:51:49.539 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-04-04T05:51:49.540 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-04T05:51:49.540 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum upgrade -y linux-firmware 2024-04-04T05:51:49.687 DEBUG:teuthology.task.kernel:sha1 for client.1 is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-04-04T05:51:49.688 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-04-04T05:51:49.688 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2024-04-04T05:51:49.705 INFO:teuthology.orchestra.run.smithi178.stdout:5.14.0-361.el9.x86_64 2024-04-04T05:51:49.705 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-04-04T05:51:49.705 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-04T05:51:49.705 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum upgrade -y linux-firmware 2024-04-04T05:51:51.746 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-04-04T05:51:51.940 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-04-04T05:51:53.405 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-9 - BaseOS 3.5 MB/s | 2.0 MB 00:00 2024-04-04T05:51:53.624 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS-9 - BaseOS 3.4 MB/s | 2.0 MB 00:00 2024-04-04T05:51:54.370 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-04T05:51:55.589 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.0 MB 00:01 2024-04-04T05:51:56.196 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-04T05:51:57.432 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-04T05:52:01.614 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 6.1 MB 00:01 2024-04-04T05:52:03.565 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 16 kB 00:00 2024-04-04T05:52:04.657 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - CRB 1.6 MB/s | 6.1 MB 00:03 2024-04-04T05:52:05.285 INFO:teuthology.orchestra.run.smithi159.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-04T05:52:06.300 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-04-04T05:52:10.440 INFO:teuthology.orchestra.run.smithi159.stdout:lab-extras 6.4 kB/s | 1.7 kB 00:00 2024-04-04T05:52:12.811 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-04-04T05:52:12.825 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-04-04T05:52:12.826 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2024-04-04T05:52:12.826 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-04-04T05:52:12.826 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrading: 2024-04-04T05:52:12.826 INFO:teuthology.orchestra.run.smithi159.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-04-04T05:52:12.826 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T05:52:12.826 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-04-04T05:52:12.826 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-04-04T05:52:12.826 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrade 1 Package 2024-04-04T05:52:12.826 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T05:52:12.826 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 375 M 2024-04-04T05:52:12.827 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-04-04T05:52:17.218 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 1.9 MB/s | 21 MB 00:10 2024-04-04T05:52:21.899 INFO:teuthology.orchestra.run.smithi159.stdout:linux-firmware-20240219-143.el9.noarch.rpm 41 MB/s | 375 MB 00:09 2024-04-04T05:52:21.900 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-04-04T05:52:21.900 INFO:teuthology.orchestra.run.smithi159.stdout:Total 41 MB/s | 375 MB 00:09 2024-04-04T05:52:21.900 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-04-04T05:52:22.006 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-04-04T05:52:22.007 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-04-04T05:52:22.231 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-04T05:52:24.381 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-04-04T05:52:24.381 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-04-04T05:52:24.614 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-04-04T05:52:24.624 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-04-04T05:52:24.624 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-04-04T05:52:24.624 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-04-04T05:52:24.624 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrading: 2024-04-04T05:52:24.624 INFO:teuthology.orchestra.run.smithi178.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-04-04T05:52:24.624 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T05:52:24.625 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-04-04T05:52:24.625 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-04-04T05:52:24.625 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrade 1 Package 2024-04-04T05:52:24.625 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T05:52:24.625 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 375 M 2024-04-04T05:52:24.625 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-04-04T05:52:30.785 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-04-04T05:52:31.076 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-04T05:52:31.166 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-04T05:52:32.417 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-04T05:52:32.417 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-04T05:52:32.824 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-04T05:52:32.824 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T05:52:32.824 INFO:teuthology.orchestra.run.smithi159.stdout:Upgraded: 2024-04-04T05:52:32.824 INFO:teuthology.orchestra.run.smithi159.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-04T05:52:32.824 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T05:52:32.824 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-04-04T05:52:33.121 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.0... 2024-04-04T05:52:33.122 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-04-04T05:52:33.281 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-04-04T05:52:33.281 DEBUG:teuthology.orchestra.run.smithi159:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-04T05:52:33.284 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi159.front.sepia.ceph.com to complete... 2024-04-04T05:52:36.198 INFO:teuthology.orchestra.run.smithi178.stdout:linux-firmware-20240219-143.el9.noarch.rpm 32 MB/s | 375 MB 00:11 2024-04-04T05:52:36.199 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-04-04T05:52:36.199 INFO:teuthology.orchestra.run.smithi178.stdout:Total 32 MB/s | 375 MB 00:11 2024-04-04T05:52:36.199 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-04-04T05:52:36.300 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-04-04T05:52:36.300 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-04-04T05:52:38.768 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-04-04T05:52:38.768 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-04-04T05:52:45.066 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-04-04T05:52:45.313 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-04T05:52:45.405 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-04T05:52:46.587 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-04T05:52:46.588 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-04T05:52:47.004 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-04T05:52:47.004 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T05:52:47.004 INFO:teuthology.orchestra.run.smithi178.stdout:Upgraded: 2024-04-04T05:52:47.004 INFO:teuthology.orchestra.run.smithi178.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-04T05:52:47.004 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T05:52:47.005 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-04-04T05:52:47.295 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.1... 2024-04-04T05:52:47.295 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-04-04T05:52:47.621 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-04-04T05:52:47.621 DEBUG:teuthology.orchestra.run.smithi178:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-04T05:52:47.623 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi178.front.sepia.ceph.com to complete... 2024-04-04T05:52:53.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04 05:52:53 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-04T05:52:53.908 INFO:teuthology.orchestra.run.smithi159.stderr:FINISHED --2024-04-04 05:52:53-- 2024-04-04T05:52:53.908 INFO:teuthology.orchestra.run.smithi159.stderr:Total wall clock time: 21s 2024-04-04T05:52:53.908 INFO:teuthology.orchestra.run.smithi159.stderr:Downloaded: 1 files, 321M in 20s (15.9 MB/s) 2024-04-04T05:52:53.911 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.0... 2024-04-04T05:52:53.912 DEBUG:teuthology.orchestra.run.smithi159:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-04T05:52:55.846 INFO:teuthology.orchestra.run.smithi159.stdout:Verifying... ######################################## 2024-04-04T05:52:55.965 INFO:teuthology.orchestra.run.smithi159.stdout:Preparing... ######################################## 2024-04-04T05:52:56.045 INFO:teuthology.orchestra.run.smithi159.stdout:Updating / installing... 2024-04-04T05:53:03.623 INFO:teuthology.orchestra.run.smithi159.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-04-04T05:53:09.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04 05:53:09 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-04T05:53:09.620 INFO:teuthology.orchestra.run.smithi178.stderr:FINISHED --2024-04-04 05:53:09-- 2024-04-04T05:53:09.620 INFO:teuthology.orchestra.run.smithi178.stderr:Total wall clock time: 22s 2024-04-04T05:53:09.620 INFO:teuthology.orchestra.run.smithi178.stderr:Downloaded: 1 files, 321M in 22s (14.8 MB/s) 2024-04-04T05:53:09.623 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.1... 2024-04-04T05:53:09.624 DEBUG:teuthology.orchestra.run.smithi178:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-04T05:53:11.565 INFO:teuthology.orchestra.run.smithi178.stdout:Verifying... ######################################## 2024-04-04T05:53:11.672 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing... ######################################## 2024-04-04T05:53:11.752 INFO:teuthology.orchestra.run.smithi178.stdout:Updating / installing... 2024-04-04T05:53:19.295 INFO:teuthology.orchestra.run.smithi178.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-04-04T05:53:19.348 INFO:teuthology.orchestra.run.smithi159.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-04T05:53:35.546 INFO:teuthology.orchestra.run.smithi178.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-04T05:53:41.802 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi159.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-04T05:53:41.803 DEBUG:teuthology.orchestra.run.smithi159:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-04T05:53:41.867 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-04-04T05:53:41.868 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-04-04T05:53:41.868 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-04-04T05:53:41.868 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-04-04T05:53:41.868 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-04-04T05:53:41.868 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-04-04T05:53:41.868 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-04-04T05:53:41.868 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-04T05:53:41.868 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-04T05:53:41.868 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-04T05:53:41.868 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-04T05:53:41.868 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-04T05:53:41.868 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-04T05:53:41.868 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-04T05:53:41.869 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-04T05:53:41.869 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-04T05:53:41.869 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-04T05:53:41.869 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-04T05:53:41.869 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-04T05:53:41.869 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-04T05:53:41.869 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-04T05:53:41.869 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-04T05:53:41.869 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-04T05:53:41.869 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-04T05:53:41.869 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-04T05:53:41.869 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-04T05:53:41.869 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-04T05:53:41.870 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-04-04T05:53:41.870 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-04-04T05:53:41.870 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-04-04T05:53:41.870 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-04T05:53:41.870 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-04-04T05:53:41.870 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-04T05:53:41.870 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-04T05:53:41.870 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-04-04T05:53:41.870 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-04-04T05:53:41.870 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-04-04T05:53:41.870 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-04-04T05:53:41.870 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-04-04T05:53:41.871 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-04-04T05:53:41.871 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-04-04T05:53:41.871 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-04-04T05:53:41.871 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-04T05:53:41.871 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-04-04T05:53:41.871 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-04-04T05:53:41.871 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-04-04T05:53:41.871 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-04-04T05:53:41.871 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-04-04T05:53:41.871 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-04-04T05:53:41.871 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-04-04T05:53:41.871 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-04-04T05:53:41.871 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-04-04T05:53:41.871 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-04-04T05:53:41.872 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-04-04T05:53:41.872 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-04-04T05:53:41.872 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-04-04T05:53:41.872 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-04-04T05:53:41.872 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-04-04T05:53:41.872 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-04-04T05:53:41.872 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-04-04T05:53:41.872 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-04-04T05:53:41.872 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-04-04T05:53:41.872 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-04-04T05:53:41.872 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-04-04T05:53:41.872 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-04-04T05:53:41.872 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-04-04T05:53:41.872 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-04-04T05:53:41.873 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-04-04T05:53:41.873 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-04-04T05:53:41.873 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-04-04T05:53:41.873 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-04-04T05:53:41.873 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-04-04T05:53:41.873 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-04-04T05:53:41.873 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-04-04T05:53:41.873 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-04-04T05:53:41.873 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-04-04T05:53:41.873 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-04-04T05:53:41.873 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-04-04T05:53:41.873 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-04-04T05:53:41.873 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-04-04T05:53:41.873 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-04-04T05:53:41.874 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-04-04T05:53:41.874 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-04-04T05:53:41.874 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-04-04T05:53:41.874 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-04-04T05:53:41.874 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-04-04T05:53:41.874 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-04-04T05:53:41.874 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-04T05:53:41.874 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-04-04T05:53:41.874 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-04-04T05:53:41.874 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-04-04T05:53:41.874 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-04-04T05:53:41.874 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-04-04T05:53:41.874 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-04-04T05:53:41.875 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-04-04T05:53:41.875 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-04-04T05:53:41.875 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-04-04T05:53:41.875 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-04-04T05:53:41.875 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-04-04T05:53:41.875 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-04-04T05:53:41.875 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-04-04T05:53:41.875 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-04-04T05:53:41.875 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-04-04T05:53:41.875 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-04-04T05:53:41.875 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-04-04T05:53:41.875 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-04-04T05:53:41.875 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-04-04T05:53:41.875 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-04-04T05:53:41.875 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-04-04T05:53:41.876 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-04-04T05:53:41.876 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-04-04T05:53:41.876 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-04-04T05:53:41.876 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-04T05:53:41.876 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-04T05:53:41.876 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-04-04T05:53:41.876 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-04-04T05:53:41.876 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-04-04T05:53:41.876 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-04-04T05:53:41.876 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-04-04T05:53:41.876 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-04-04T05:53:41.876 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-04-04T05:53:41.876 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-04-04T05:53:41.876 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-04-04T05:53:41.877 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-04-04T05:53:41.877 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-04-04T05:53:41.877 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-04-04T05:53:41.877 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-04-04T05:53:41.877 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-04T05:53:41.877 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-04T05:53:41.877 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-04-04T05:53:41.877 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-04-04T05:53:41.877 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-04-04T05:53:41.877 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-04-04T05:53:41.877 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-04-04T05:53:41.877 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-04-04T05:53:41.877 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-04-04T05:53:41.878 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-04-04T05:53:41.878 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-04-04T05:53:41.878 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-04-04T05:53:41.878 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-04-04T05:53:41.878 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-04-04T05:53:41.878 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-04-04T05:53:41.878 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-04-04T05:53:41.878 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-04-04T05:53:41.878 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-04-04T05:53:41.878 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-04-04T05:53:41.878 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-04-04T05:53:41.878 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-04-04T05:53:41.878 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-04-04T05:53:41.878 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-04-04T05:53:41.879 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-04-04T05:53:41.879 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-04-04T05:53:41.879 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-04-04T05:53:41.879 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-04-04T05:53:41.879 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-04-04T05:53:41.879 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-04-04T05:53:41.879 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-04T05:53:41.879 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-04T05:53:41.879 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-04-04T05:53:41.879 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-04-04T05:53:41.879 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-04-04T05:53:41.879 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-04-04T05:53:41.879 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-04-04T05:53:41.880 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-04-04T05:53:41.880 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-04-04T05:53:41.880 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-04-04T05:53:41.880 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-04-04T05:53:41.880 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-04-04T05:53:41.880 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-04-04T05:53:41.880 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-04T05:53:41.880 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-04-04T05:53:41.880 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-04-04T05:53:41.880 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-04-04T05:53:41.880 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-04-04T05:53:41.880 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-04-04T05:53:41.880 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-04-04T05:53:41.880 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-04-04T05:53:41.881 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-04T05:53:41.881 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-04-04T05:53:41.881 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-04-04T05:53:41.881 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-04-04T05:53:41.881 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-04-04T05:53:41.881 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-04-04T05:53:41.881 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-04-04T05:53:41.881 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-04T05:53:41.881 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-04T05:53:41.881 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-04T05:53:41.881 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-04-04T05:53:41.881 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-04T05:53:41.881 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-04-04T05:53:41.881 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-04T05:53:41.882 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-04T05:53:41.882 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-04T05:53:41.882 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-04T05:53:41.882 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-04T05:53:41.882 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-04T05:53:41.882 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-04-04T05:53:41.882 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-04-04T05:53:41.882 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-04-04T05:53:41.882 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-04-04T05:53:41.882 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-04-04T05:53:41.882 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-04-04T05:53:41.882 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-04-04T05:53:41.882 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-04T05:53:41.883 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-04T05:53:41.883 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-04T05:53:41.883 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-04T05:53:41.883 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-04T05:53:41.883 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-04-04T05:53:41.883 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-04-04T05:53:41.883 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-04-04T05:53:41.883 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-04T05:53:41.883 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-04T05:53:41.883 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-04T05:53:41.883 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-04T05:53:41.883 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-04T05:53:41.883 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-04T05:53:41.883 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-04-04T05:53:41.884 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-04-04T05:53:41.884 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-04T05:53:41.884 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-04-04T05:53:41.884 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-04-04T05:53:41.884 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-04-04T05:53:41.884 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-04-04T05:53:41.884 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-04T05:53:41.884 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-04T05:53:41.884 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-04T05:53:41.884 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-04-04T05:53:41.884 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-04-04T05:53:41.884 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-04-04T05:53:41.884 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-04T05:53:41.885 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-04-04T05:53:41.885 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-04-04T05:53:41.885 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-04-04T05:53:41.885 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-04T05:53:41.885 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-04-04T05:53:41.885 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-04-04T05:53:41.885 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-04T05:53:41.885 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-04-04T05:53:41.885 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-04T05:53:41.885 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-04-04T05:53:41.885 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-04-04T05:53:41.885 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-04-04T05:53:41.885 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-04-04T05:53:41.885 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-04-04T05:53:41.886 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-04-04T05:53:41.886 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-04-04T05:53:41.886 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-04-04T05:53:41.886 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-04-04T05:53:41.886 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-04-04T05:53:41.886 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-04-04T05:53:41.886 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-04-04T05:53:41.886 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-04-04T05:53:41.886 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-04-04T05:53:41.886 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-04-04T05:53:41.886 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-04-04T05:53:41.886 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-04T05:53:41.886 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-04-04T05:53:41.886 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-04-04T05:53:41.887 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-04T05:53:41.887 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-04-04T05:53:41.887 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-04-04T05:53:41.887 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-04-04T05:53:41.887 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-04-04T05:53:41.887 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-04-04T05:53:41.887 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-04-04T05:53:41.887 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-04-04T05:53:41.887 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-04-04T05:53:41.887 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-04-04T05:53:41.887 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-04-04T05:53:41.887 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-04-04T05:53:41.887 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-04T05:53:41.887 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-04T05:53:41.888 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-04-04T05:53:41.888 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-04-04T05:53:41.888 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-04-04T05:53:41.888 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-04-04T05:53:41.888 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-04-04T05:53:41.888 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-04-04T05:53:41.888 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-04-04T05:53:41.888 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-04-04T05:53:41.888 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-04T05:53:41.888 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-04T05:53:41.888 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-04-04T05:53:41.888 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-04-04T05:53:41.888 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-04-04T05:53:41.888 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-04-04T05:53:41.889 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-04-04T05:53:41.889 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-04-04T05:53:41.889 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-04-04T05:53:41.889 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-04-04T05:53:41.889 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-04-04T05:53:41.889 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-04T05:53:41.889 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-04-04T05:53:41.889 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-04T05:53:41.889 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-04T05:53:41.889 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-04T05:53:41.889 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-04T05:53:41.889 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-04T05:53:41.889 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-04T05:53:41.889 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-04T05:53:41.890 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-04T05:53:41.890 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-04T05:53:41.890 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-04-04T05:53:41.890 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-04-04T05:53:41.890 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-04-04T05:53:41.890 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-04T05:53:41.890 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-04-04T05:53:41.890 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-04-04T05:53:41.890 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-04-04T05:53:41.890 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-04-04T05:53:41.890 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-04-04T05:53:41.890 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-04T05:53:41.890 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-04-04T05:53:41.891 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-04-04T05:53:41.891 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-04-04T05:53:41.891 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-04-04T05:53:41.891 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-04-04T05:53:41.891 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-04-04T05:53:41.891 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-04-04T05:53:41.891 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-04-04T05:53:41.891 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-04-04T05:53:41.891 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-04-04T05:53:41.891 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-04-04T05:53:41.891 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-04-04T05:53:41.891 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-04-04T05:53:41.891 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-04-04T05:53:41.892 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-04-04T05:53:41.892 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-04-04T05:53:41.892 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-04-04T05:53:41.892 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-04-04T05:53:41.892 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-04-04T05:53:41.892 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-04T05:53:41.892 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-04-04T05:53:41.892 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-04-04T05:53:41.892 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-04-04T05:53:41.892 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-04-04T05:53:41.892 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-04-04T05:53:41.892 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-04-04T05:53:41.892 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-04-04T05:53:41.893 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-04-04T05:53:41.893 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-04-04T05:53:41.893 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-04-04T05:53:41.893 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-04-04T05:53:41.893 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-04-04T05:53:41.893 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-04-04T05:53:41.893 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-04-04T05:53:41.893 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-04-04T05:53:41.893 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-04-04T05:53:41.893 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-04-04T05:53:41.893 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-04-04T05:53:41.893 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-04-04T05:53:41.893 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-04T05:53:41.894 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-04-04T05:53:41.894 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-04-04T05:53:41.894 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-04T05:53:41.894 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-04-04T05:53:41.894 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-04-04T05:53:41.894 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-04-04T05:53:41.894 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-04-04T05:53:41.894 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-04-04T05:53:41.894 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-04-04T05:53:41.894 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-04-04T05:53:41.894 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-04-04T05:53:41.894 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-04-04T05:53:41.894 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-04-04T05:53:41.895 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-04-04T05:53:41.895 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-04-04T05:53:41.895 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-04-04T05:53:41.895 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-04-04T05:53:41.895 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-04T05:53:41.895 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-04-04T05:53:41.895 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-04-04T05:53:41.895 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-04-04T05:53:41.895 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-04-04T05:53:41.895 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-04-04T05:53:41.895 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-04-04T05:53:41.895 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-04-04T05:53:41.895 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-04-04T05:53:41.896 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-04-04T05:53:41.896 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-04-04T05:53:41.896 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-04-04T05:53:41.896 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-04-04T05:53:41.896 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-04-04T05:53:41.896 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-04-04T05:53:41.896 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-04-04T05:53:41.896 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-04-04T05:53:41.896 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-04-04T05:53:41.896 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-04-04T05:53:41.896 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-04-04T05:53:41.896 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-04-04T05:53:41.896 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-04-04T05:53:41.896 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-04-04T05:53:41.897 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-04-04T05:53:41.897 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-04-04T05:53:41.897 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-04-04T05:53:41.897 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-04-04T05:53:41.897 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-04-04T05:53:41.897 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-04-04T05:53:41.897 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-04-04T05:53:41.897 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-04-04T05:53:41.897 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-04-04T05:53:41.897 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-04-04T05:53:41.897 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-04-04T05:53:41.897 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-04-04T05:53:41.897 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-04-04T05:53:41.897 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-04-04T05:53:41.898 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-04-04T05:53:41.898 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-04-04T05:53:41.898 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-04-04T05:53:41.898 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-04-04T05:53:41.898 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-04-04T05:53:41.898 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-04-04T05:53:41.898 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-04-04T05:53:41.898 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-04-04T05:53:41.898 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-04-04T05:53:41.898 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-04-04T05:53:41.898 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-04-04T05:53:41.898 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-04-04T05:53:41.898 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-04T05:53:41.899 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-04T05:53:41.899 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-04T05:53:41.899 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-04T05:53:41.899 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-04T05:53:41.899 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-04T05:53:41.899 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-04T05:53:41.899 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-04T05:53:41.899 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-04T05:53:41.899 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-04T05:53:41.899 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-04T05:53:41.899 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-04T05:53:41.899 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-04-04T05:53:41.899 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-04-04T05:53:41.899 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-04-04T05:53:41.899 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-04T05:53:41.900 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-04-04T05:53:41.900 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-04-04T05:53:41.900 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-04-04T05:53:41.900 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-04-04T05:53:41.900 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-04-04T05:53:41.900 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-04-04T05:53:41.900 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-04-04T05:53:41.900 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-04-04T05:53:41.900 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-04-04T05:53:41.900 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-04-04T05:53:41.900 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-04-04T05:53:41.900 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-04-04T05:53:41.900 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-04-04T05:53:41.900 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-04-04T05:53:41.901 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-04-04T05:53:41.901 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-04-04T05:53:41.901 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-04-04T05:53:41.901 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-04-04T05:53:41.901 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-04-04T05:53:41.901 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-04-04T05:53:41.901 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-04-04T05:53:41.901 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-04-04T05:53:41.901 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-04-04T05:53:41.901 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-04-04T05:53:41.901 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-04-04T05:53:41.901 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-04-04T05:53:41.901 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-04-04T05:53:41.902 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-04T05:53:41.902 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-04T05:53:41.902 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-04-04T05:53:41.902 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-04T05:53:41.902 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-04T05:53:41.902 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-04T05:53:41.902 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-04T05:53:41.902 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-04T05:53:41.902 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-04T05:53:41.902 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-04T05:53:41.902 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-04T05:53:41.902 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-04T05:53:41.902 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-04T05:53:41.903 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-04T05:53:41.903 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-04T05:53:41.903 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-04T05:53:41.903 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-04T05:53:41.903 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-04T05:53:41.903 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-04T05:53:41.903 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-04T05:53:41.903 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-04T05:53:41.903 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-04T05:53:41.903 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-04T05:53:41.903 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-04T05:53:41.903 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-04T05:53:41.903 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-04T05:53:41.903 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-04-04T05:53:41.904 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-04-04T05:53:41.904 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-04T05:53:41.904 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-04-04T05:53:41.904 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-04-04T05:53:41.904 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-04-04T05:53:41.904 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-04-04T05:53:41.904 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-04T05:53:41.904 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-04-04T05:53:41.904 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-04T05:53:41.904 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-04T05:53:41.904 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-04T05:53:41.904 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-04T05:53:41.904 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-04T05:53:41.905 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-04-04T05:53:41.905 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-04-04T05:53:41.905 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-04T05:53:41.905 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-04-04T05:53:41.905 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-04T05:53:41.905 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-04-04T05:53:41.905 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-04T05:53:41.905 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-04-04T05:53:41.905 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-04T05:53:41.905 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-04-04T05:53:41.905 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-04-04T05:53:41.905 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-04T05:53:41.905 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-04-04T05:53:41.906 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-04T05:53:41.906 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-04-04T05:53:41.906 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-04T05:53:41.906 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-04-04T05:53:41.906 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-04T05:53:41.906 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-04-04T05:53:41.906 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-04T05:53:41.906 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-04-04T05:53:41.906 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-04-04T05:53:41.906 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-04T05:53:41.906 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-04T05:53:41.906 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-04-04T05:53:41.906 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-04-04T05:53:41.907 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-04-04T05:53:41.907 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-04T05:53:41.907 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-04-04T05:53:41.907 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-04-04T05:53:41.907 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-04-04T05:53:41.907 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-04-04T05:53:41.907 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-04T05:53:41.907 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-04-04T05:53:41.907 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-04T05:53:41.907 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-04T05:53:41.907 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-04T05:53:41.907 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-04-04T05:53:41.907 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-04T05:53:41.907 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-04-04T05:53:41.908 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-04-04T05:53:41.908 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-04-04T05:53:41.908 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-04T05:53:41.908 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-04-04T05:53:41.908 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-04-04T05:53:41.908 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-04-04T05:53:41.908 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-04-04T05:53:41.908 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-04-04T05:53:41.908 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-04-04T05:53:41.908 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-04-04T05:53:41.908 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-04-04T05:53:41.908 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-04T05:53:41.908 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-04T05:53:41.908 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-04T05:53:41.909 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-04T05:53:41.909 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-04T05:53:41.909 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-04T05:53:41.909 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-04T05:53:41.909 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-04T05:53:41.909 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-04T05:53:41.909 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-04-04T05:53:41.909 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-04-04T05:53:41.909 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-04-04T05:53:41.909 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-04-04T05:53:41.909 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-04-04T05:53:41.909 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-04-04T05:53:41.909 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-04-04T05:53:41.909 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-04-04T05:53:41.910 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-04-04T05:53:41.910 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-04-04T05:53:41.910 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-04-04T05:53:41.910 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-04-04T05:53:41.910 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-04-04T05:53:41.910 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-04-04T05:53:41.910 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-04-04T05:53:41.910 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-04-04T05:53:41.910 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-04-04T05:53:41.910 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-04-04T05:53:41.910 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-04-04T05:53:41.910 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-04T05:53:41.910 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-04-04T05:53:41.910 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-04-04T05:53:41.911 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-04-04T05:53:41.911 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-04-04T05:53:41.911 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-04-04T05:53:41.911 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-04-04T05:53:41.911 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-04-04T05:53:41.911 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-04-04T05:53:41.911 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-04-04T05:53:41.911 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-04-04T05:53:41.911 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-04-04T05:53:41.911 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-04-04T05:53:41.911 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-04-04T05:53:41.911 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-04-04T05:53:41.911 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-04-04T05:53:41.911 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-04-04T05:53:41.912 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-04-04T05:53:41.912 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-04-04T05:53:41.912 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-04-04T05:53:41.912 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-04-04T05:53:41.912 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-04-04T05:53:41.912 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-04-04T05:53:41.912 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-04-04T05:53:41.912 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-04T05:53:41.912 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-04-04T05:53:41.912 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-04-04T05:53:41.912 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-04-04T05:53:41.912 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-04-04T05:53:41.912 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-04-04T05:53:41.912 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-04-04T05:53:41.913 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-04-04T05:53:41.913 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-04T05:53:41.913 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-04-04T05:53:41.913 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-04-04T05:53:41.913 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-04-04T05:53:41.913 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-04-04T05:53:41.913 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-04-04T05:53:41.913 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-04-04T05:53:41.913 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-04T05:53:41.913 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-04T05:53:41.913 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-04-04T05:53:41.913 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-04-04T05:53:41.913 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-04-04T05:53:41.913 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-04-04T05:53:41.914 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-04-04T05:53:41.914 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-04T05:53:41.914 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-04-04T05:53:41.914 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-04T05:53:41.914 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-04T05:53:41.914 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-04T05:53:41.914 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-04-04T05:53:41.914 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-04-04T05:53:41.914 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-04-04T05:53:41.914 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-04-04T05:53:41.914 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-04-04T05:53:41.914 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-04T05:53:41.914 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-04-04T05:53:41.915 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-04-04T05:53:41.915 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-04-04T05:53:41.915 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-04-04T05:53:41.915 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-04-04T05:53:41.915 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-04T05:53:41.915 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-04T05:53:41.915 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-04T05:53:41.915 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-04-04T05:53:41.915 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-04T05:53:41.915 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-04-04T05:53:41.915 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-04-04T05:53:41.915 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-04-04T05:53:41.915 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-04-04T05:53:41.915 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-04T05:53:41.916 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-04T05:53:41.916 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-04T05:53:41.916 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-04T05:53:41.916 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-04-04T05:53:41.916 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-04T05:53:41.916 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-04T05:53:41.916 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-04T05:53:41.916 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-04T05:53:41.916 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-04-04T05:53:41.916 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-04T05:53:41.916 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-04-04T05:53:41.916 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-04T05:53:41.916 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-04-04T05:53:41.917 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-04T05:53:41.917 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-04-04T05:53:41.917 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-04T05:53:41.917 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-04-04T05:53:41.917 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-04T05:53:41.917 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-04T05:53:41.917 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-04T05:53:41.917 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-04-04T05:53:41.917 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-04-04T05:53:41.917 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-04T05:53:41.917 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-04T05:53:41.917 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-04T05:53:41.917 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-04T05:53:41.917 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-04T05:53:41.918 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-04-04T05:53:41.918 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-04T05:53:41.918 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-04T05:53:41.918 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-04-04T05:53:41.918 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-04T05:53:41.918 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-04T05:53:41.918 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-04T05:53:41.918 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-04T05:53:41.918 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-04T05:53:41.918 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-04-04T05:53:41.918 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-04T05:53:41.918 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-04T05:53:41.918 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-04-04T05:53:41.919 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-04-04T05:53:41.919 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-04T05:53:41.919 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-04-04T05:53:41.919 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-04T05:53:41.919 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-04T05:53:41.919 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-04T05:53:41.919 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-04T05:53:41.919 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-04T05:53:41.919 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-04T05:53:41.919 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-04T05:53:41.919 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-04T05:53:41.919 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-04T05:53:41.919 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-04T05:53:41.919 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-04-04T05:53:41.920 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-04-04T05:53:41.920 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-04T05:53:41.920 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-04-04T05:53:41.920 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-04-04T05:53:41.920 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-04T05:53:41.920 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-04T05:53:41.920 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-04T05:53:41.920 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-04T05:53:41.920 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-04T05:53:41.920 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-04T05:53:41.920 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-04-04T05:53:41.920 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-04T05:53:41.920 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-04T05:53:41.921 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-04-04T05:53:41.921 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-04-04T05:53:41.921 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-04-04T05:53:41.921 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-04T05:53:41.921 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-04-04T05:53:41.921 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-04-04T05:53:41.921 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-04-04T05:53:41.921 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-04T05:53:41.921 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-04-04T05:53:41.921 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-04T05:53:41.921 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-04-04T05:53:41.921 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-04T05:53:41.921 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-04-04T05:53:41.921 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-04T05:53:41.921 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-04-04T05:53:41.922 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-04T05:53:41.922 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-04-04T05:53:41.922 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-04T05:53:41.922 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-04-04T05:53:41.922 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-04T05:53:41.922 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-04-04T05:53:41.922 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-04T05:53:41.922 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-04T05:53:41.922 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-04T05:53:41.922 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-04T05:53:41.922 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-04T05:53:41.922 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-04-04T05:53:41.922 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-04-04T05:53:41.922 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-04T05:53:41.922 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-04T05:53:41.923 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-04T05:53:41.923 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-04-04T05:53:41.923 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-04T05:53:41.923 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-04T05:53:41.923 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-04T05:53:41.923 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-04T05:53:41.923 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-04T05:53:41.923 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-04T05:53:41.923 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-04-04T05:53:41.923 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-04T05:53:41.923 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-04-04T05:53:41.923 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-04T05:53:41.923 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-04T05:53:41.924 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-04-04T05:53:41.924 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-04T05:53:41.924 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-04T05:53:41.924 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-04-04T05:53:41.924 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-04T05:53:41.924 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-04-04T05:53:41.924 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-04T05:53:41.924 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-04-04T05:53:41.924 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-04T05:53:41.924 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-04T05:53:41.924 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-04-04T05:53:41.924 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-04T05:53:41.924 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-04T05:53:41.925 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-04T05:53:41.925 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-04T05:53:41.925 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-04T05:53:41.925 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-04-04T05:53:41.925 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-04T05:53:41.925 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-04-04T05:53:41.925 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-04T05:53:41.925 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-04T05:53:41.925 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-04T05:53:41.925 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-04T05:53:41.925 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-04-04T05:53:41.925 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-04T05:53:41.925 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-04-04T05:53:41.926 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-04T05:53:41.926 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-04-04T05:53:41.926 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-04T05:53:41.926 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-04-04T05:53:41.926 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-04T05:53:41.926 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-04T05:53:41.926 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-04-04T05:53:41.926 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-04T05:53:41.926 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-04T05:53:41.926 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-04T05:53:41.926 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-04T05:53:41.926 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-04-04T05:53:41.926 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-04T05:53:41.926 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-04T05:53:41.926 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-04T05:53:41.927 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-04-04T05:53:41.927 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-04T05:53:41.927 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-04T05:53:41.927 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-04T05:53:41.927 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-04T05:53:41.927 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-04-04T05:53:41.927 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-04T05:53:41.927 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-04-04T05:53:41.927 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-04T05:53:41.927 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-04-04T05:53:41.927 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-04T05:53:41.927 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-04T05:53:41.927 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-04-04T05:53:41.927 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-04T05:53:41.927 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-04T05:53:41.928 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-04-04T05:53:41.928 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-04T05:53:41.928 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-04-04T05:53:41.928 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-04-04T05:53:41.928 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-04T05:53:41.928 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-04-04T05:53:41.928 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-04T05:53:41.928 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-04-04T05:53:41.928 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-04-04T05:53:41.928 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-04T05:53:41.928 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-04-04T05:53:41.928 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-04-04T05:53:41.929 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-04-04T05:53:41.929 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-04-04T05:53:41.929 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-04T05:53:41.929 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-04T05:53:41.929 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-04T05:53:41.929 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-04-04T05:53:41.929 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-04-04T05:53:41.929 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-04-04T05:53:41.929 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-04-04T05:53:41.929 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-04-04T05:53:41.929 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-04T05:53:41.929 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-04T05:53:41.929 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-04-04T05:53:41.929 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-04-04T05:53:41.930 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-04T05:53:41.930 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-04T05:53:41.930 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-04T05:53:41.930 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-04-04T05:53:41.930 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-04-04T05:53:41.930 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-04-04T05:53:41.930 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-04-04T05:53:41.930 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-04-04T05:53:41.930 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-04-04T05:53:41.930 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-04-04T05:53:41.930 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-04-04T05:53:41.930 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-04-04T05:53:41.931 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-04-04T05:53:41.931 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-04-04T05:53:41.931 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-04-04T05:53:41.931 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-04-04T05:53:41.931 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-04-04T05:53:41.931 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-04-04T05:53:41.931 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-04-04T05:53:41.931 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-04-04T05:53:41.931 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-04-04T05:53:41.931 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-04T05:53:41.931 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-04-04T05:53:41.931 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-04T05:53:41.931 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-04T05:53:41.932 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-04T05:53:41.932 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-04T05:53:41.932 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-04-04T05:53:41.932 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-04-04T05:53:41.932 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-04-04T05:53:41.932 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-04-04T05:53:41.932 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-04-04T05:53:41.932 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-04-04T05:53:41.932 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-04-04T05:53:41.932 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-04-04T05:53:41.932 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-04-04T05:53:41.932 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-04-04T05:53:41.932 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-04T05:53:41.932 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-04T05:53:41.933 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-04T05:53:41.933 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-04-04T05:53:41.933 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-04T05:53:41.933 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-04-04T05:53:41.933 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-04-04T05:53:41.933 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-04-04T05:53:41.933 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-04-04T05:53:41.933 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-04T05:53:41.933 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-04-04T05:53:41.933 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-04-04T05:53:41.933 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-04-04T05:53:41.933 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-04T05:53:41.933 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-04-04T05:53:41.933 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-04-04T05:53:41.934 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-04-04T05:53:41.934 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-04-04T05:53:41.934 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-04-04T05:53:41.934 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-04-04T05:53:41.934 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-04-04T05:53:41.934 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-04-04T05:53:41.934 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-04-04T05:53:41.934 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-04-04T05:53:41.934 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-04-04T05:53:41.934 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-04-04T05:53:41.934 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-04-04T05:53:41.934 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-04-04T05:53:41.934 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-04T05:53:41.934 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-04-04T05:53:41.935 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-04T05:53:41.935 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-04-04T05:53:41.935 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-04-04T05:53:41.935 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-04-04T05:53:41.935 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-04-04T05:53:41.935 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-04T05:53:41.935 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-04-04T05:53:41.935 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-04-04T05:53:41.935 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-04T05:53:41.935 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-04-04T05:53:41.935 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-04-04T05:53:41.935 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-04T05:53:41.935 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-04-04T05:53:41.935 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-04T05:53:41.935 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-04T05:53:41.936 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-04T05:53:41.936 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-04T05:53:41.936 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-04T05:53:41.936 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-04T05:53:41.936 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-04T05:53:41.936 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-04T05:53:41.936 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-04T05:53:41.936 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-04-04T05:53:41.936 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-04-04T05:53:41.936 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-04T05:53:41.936 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-04-04T05:53:41.936 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-04T05:53:41.936 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-04T05:53:41.936 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-04T05:53:41.937 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-04T05:53:41.937 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-04T05:53:41.937 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-04T05:53:41.937 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-04T05:53:41.937 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-04T05:53:41.937 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-04T05:53:41.937 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-04-04T05:53:41.937 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-04-04T05:53:41.937 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-04T05:53:41.937 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-04T05:53:41.937 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-04-04T05:53:41.937 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-04-04T05:53:41.937 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-04T05:53:41.937 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-04T05:53:41.937 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-04-04T05:53:41.938 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-04-04T05:53:41.938 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-04-04T05:53:41.938 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-04-04T05:53:41.938 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-04-04T05:53:41.938 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-04-04T05:53:41.938 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-04T05:53:41.938 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-04T05:53:41.938 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-04T05:53:41.938 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-04T05:53:41.938 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-04T05:53:41.938 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-04T05:53:41.938 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-04T05:53:41.939 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-04T05:53:41.939 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-04T05:53:41.939 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-04T05:53:41.939 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-04-04T05:53:41.939 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-04T05:53:41.939 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-04T05:53:41.939 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-04T05:53:41.939 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-04T05:53:41.939 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-04T05:53:41.939 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-04T05:53:41.939 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-04-04T05:53:41.939 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-04T05:53:41.939 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-04T05:53:41.939 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-04T05:53:41.939 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-04T05:53:41.940 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-04T05:53:41.940 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-04T05:53:41.940 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-04T05:53:41.940 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-04T05:53:41.940 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-04T05:53:41.940 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-04T05:53:41.940 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-04T05:53:41.940 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-04T05:53:41.940 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-04-04T05:53:41.940 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-04-04T05:53:41.940 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-04-04T05:53:41.940 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-04-04T05:53:41.940 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-04-04T05:53:41.941 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-04-04T05:53:41.941 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-04-04T05:53:41.941 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-04-04T05:53:41.941 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-04T05:53:41.941 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-04-04T05:53:41.941 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-04-04T05:53:41.941 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-04T05:53:41.941 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-04T05:53:41.941 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-04-04T05:53:41.941 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-04-04T05:53:41.941 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-04T05:53:41.941 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-04-04T05:53:41.941 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-04-04T05:53:41.942 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-04T05:53:41.942 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-04-04T05:53:41.942 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-04-04T05:53:41.942 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-04-04T05:53:41.942 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-04T05:53:41.942 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-04-04T05:53:41.942 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-04T05:53:41.942 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-04-04T05:53:41.942 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-04-04T05:53:41.942 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-04T05:53:41.942 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-04-04T05:53:41.942 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-04T05:53:41.942 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-04T05:53:41.942 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-04T05:53:41.943 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-04T05:53:41.943 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-04T05:53:41.943 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-04-04T05:53:41.943 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-04-04T05:53:41.943 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-04T05:53:41.943 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-04T05:53:41.943 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-04T05:53:41.943 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-04T05:53:41.943 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-04-04T05:53:41.943 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-04-04T05:53:41.943 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-04T05:53:41.943 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-04-04T05:53:41.943 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-04-04T05:53:41.943 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-04T05:53:41.943 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-04T05:53:41.944 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-04-04T05:53:41.944 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-04-04T05:53:41.944 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-04-04T05:53:41.944 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-04-04T05:53:41.944 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-04-04T05:53:41.944 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-04-04T05:53:41.944 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-04-04T05:53:41.944 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-04-04T05:53:41.944 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-04-04T05:53:41.944 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-04-04T05:53:41.944 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-04T05:53:41.944 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-04T05:53:41.944 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-04-04T05:53:41.944 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-04-04T05:53:41.944 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-04-04T05:53:41.945 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-04T05:53:41.945 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-04-04T05:53:41.945 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-04-04T05:53:41.945 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-04-04T05:53:41.945 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-04T05:53:41.945 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-04-04T05:53:41.945 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-04-04T05:53:41.945 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-04T05:53:41.945 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-04T05:53:41.945 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-04T05:53:41.945 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-04-04T05:53:41.945 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-04T05:53:41.945 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-04-04T05:53:41.946 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-04T05:53:41.946 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-04-04T05:53:41.946 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-04-04T05:53:41.946 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-04T05:53:41.946 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-04-04T05:53:41.946 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-04-04T05:53:41.946 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-04-04T05:53:41.946 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-04T05:53:41.946 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-04T05:53:41.946 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-04-04T05:53:41.946 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-04T05:53:41.946 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-04-04T05:53:41.946 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-04-04T05:53:41.946 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-04T05:53:41.947 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-04T05:53:41.947 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-04T05:53:41.947 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-04T05:53:41.947 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-04T05:53:41.947 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-04-04T05:53:41.947 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-04-04T05:53:41.947 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-04-04T05:53:41.947 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-04-04T05:53:41.947 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-04T05:53:41.947 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-04-04T05:53:41.947 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-04T05:53:41.947 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-04-04T05:53:41.947 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-04-04T05:53:41.947 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-04-04T05:53:41.947 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-04-04T05:53:41.948 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-04-04T05:53:41.948 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-04T05:53:41.948 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-04-04T05:53:41.948 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-04T05:53:41.948 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-04-04T05:53:41.948 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-04-04T05:53:41.948 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-04-04T05:53:41.948 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-04-04T05:53:41.948 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-04-04T05:53:41.948 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-04-04T05:53:41.948 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-04T05:53:41.948 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-04-04T05:53:41.948 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-04-04T05:53:41.948 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-04-04T05:53:41.948 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-04T05:53:41.949 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-04T05:53:41.949 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-04T05:53:41.949 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-04T05:53:41.949 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-04T05:53:41.949 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-04T05:53:41.949 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-04T05:53:41.949 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-04T05:53:41.949 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-04T05:53:41.949 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-04T05:53:41.949 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-04T05:53:41.949 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-04T05:53:41.949 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-04T05:53:41.949 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-04T05:53:41.949 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-04T05:53:41.950 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-04T05:53:41.950 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-04T05:53:41.950 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-04T05:53:41.950 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-04-04T05:53:41.950 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-04-04T05:53:41.950 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-04T05:53:41.950 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-04-04T05:53:41.950 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-04-04T05:53:41.950 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-04-04T05:53:41.950 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-04-04T05:53:41.950 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-04-04T05:53:41.950 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-04T05:53:41.950 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-04-04T05:53:41.951 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-04-04T05:53:41.951 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-04T05:53:41.951 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-04-04T05:53:41.951 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-04-04T05:53:41.951 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-04T05:53:41.951 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-04-04T05:53:41.951 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-04T05:53:41.951 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-04-04T05:53:41.951 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-04-04T05:53:41.951 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-04-04T05:53:41.951 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-04-04T05:53:41.951 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-04-04T05:53:41.951 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-04-04T05:53:41.951 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-04-04T05:53:41.952 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-04-04T05:53:41.952 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-04-04T05:53:41.952 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-04T05:53:41.952 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-04-04T05:53:41.952 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-04-04T05:53:41.952 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-04-04T05:53:41.952 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-04T05:53:41.952 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-04-04T05:53:41.952 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-04-04T05:53:41.952 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-04-04T05:53:41.952 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-04-04T05:53:41.952 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-04-04T05:53:41.952 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-04-04T05:53:41.952 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-04-04T05:53:41.952 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-04-04T05:53:41.952 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-04-04T05:53:41.953 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-04-04T05:53:41.953 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-04-04T05:53:41.953 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-04T05:53:41.953 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-04-04T05:53:41.953 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-04-04T05:53:41.953 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-04-04T05:53:41.953 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-04T05:53:41.953 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-04T05:53:41.953 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-04-04T05:53:41.953 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-04-04T05:53:41.953 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-04T05:53:41.953 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-04-04T05:53:41.953 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-04-04T05:53:41.954 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-04-04T05:53:41.954 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-04-04T05:53:41.954 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-04-04T05:53:41.954 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-04-04T05:53:41.954 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-04-04T05:53:41.954 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-04T05:53:41.954 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-04-04T05:53:41.954 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-04-04T05:53:41.954 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-04-04T05:53:41.954 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-04T05:53:41.954 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-04T05:53:41.954 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-04-04T05:53:41.954 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-04T05:53:41.954 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-04T05:53:41.954 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-04T05:53:41.955 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-04-04T05:53:41.955 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-04-04T05:53:41.955 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-04-04T05:53:41.955 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-04-04T05:53:41.955 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-04T05:53:41.955 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-04-04T05:53:41.955 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-04T05:53:41.955 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-04T05:53:41.955 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-04T05:53:41.955 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-04-04T05:53:41.955 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-04-04T05:53:41.955 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-04-04T05:53:41.955 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-04-04T05:53:41.955 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-04-04T05:53:41.956 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-04-04T05:53:41.956 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-04-04T05:53:41.956 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-04-04T05:53:41.956 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-04-04T05:53:41.956 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-04-04T05:53:41.956 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-04-04T05:53:41.956 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-04-04T05:53:41.956 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-04-04T05:53:41.956 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-04-04T05:53:41.956 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-04T05:53:41.956 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-04-04T05:53:41.956 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-04-04T05:53:41.956 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-04T05:53:41.956 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-04T05:53:41.956 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-04-04T05:53:41.956 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-04T05:53:41.957 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-04-04T05:53:41.957 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-04-04T05:53:41.957 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-04T05:53:41.957 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-04-04T05:53:41.957 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-04-04T05:53:41.957 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-04T05:53:41.957 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-04T05:53:41.957 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-04T05:53:41.957 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-04T05:53:41.957 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-04T05:53:41.957 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-04T05:53:41.957 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-04T05:53:41.957 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-04-04T05:53:41.958 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-04T05:53:41.958 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-04T05:53:41.958 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-04T05:53:41.958 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-04T05:53:41.958 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-04T05:53:41.958 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-04-04T05:53:41.958 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-04-04T05:53:41.958 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-04T05:53:41.958 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-04T05:53:41.958 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-04T05:53:41.958 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-04-04T05:53:41.958 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-04-04T05:53:41.958 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-04T05:53:41.958 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-04T05:53:41.959 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-04-04T05:53:41.959 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-04T05:53:41.959 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-04-04T05:53:41.959 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-04-04T05:53:41.959 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-04T05:53:41.959 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-04-04T05:53:41.959 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-04-04T05:53:41.959 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-04-04T05:53:41.959 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-04-04T05:53:41.959 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-04-04T05:53:41.959 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-04-04T05:53:41.959 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-04T05:53:41.959 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-04-04T05:53:41.959 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-04-04T05:53:41.960 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-04-04T05:53:41.960 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-04-04T05:53:41.960 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-04-04T05:53:41.960 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-04T05:53:41.960 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-04T05:53:41.960 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-04-04T05:53:41.960 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-04-04T05:53:41.960 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-04-04T05:53:41.960 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-04-04T05:53:41.960 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-04-04T05:53:41.960 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-04-04T05:53:41.960 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-04T05:53:41.960 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-04T05:53:41.960 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-04T05:53:41.961 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-04T05:53:41.961 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-04T05:53:41.961 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-04T05:53:41.961 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-04T05:53:41.961 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-04T05:53:41.961 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-04T05:53:41.961 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-04T05:53:41.961 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-04T05:53:41.961 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-04-04T05:53:41.961 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-04-04T05:53:41.961 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-04T05:53:41.961 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-04T05:53:41.962 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-04T05:53:41.962 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-04-04T05:53:41.962 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-04-04T05:53:41.962 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-04T05:53:41.962 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-04T05:53:41.962 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-04-04T05:53:41.962 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-04-04T05:53:41.962 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-04T05:53:41.962 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-04T05:53:41.962 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-04-04T05:53:41.962 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-04-04T05:53:41.962 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-04-04T05:53:41.962 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-04T05:53:41.962 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-04T05:53:41.963 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-04T05:53:41.963 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-04T05:53:41.963 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-04-04T05:53:41.963 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-04T05:53:41.963 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-04-04T05:53:41.963 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-04T05:53:41.963 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-04T05:53:41.963 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-04T05:53:41.963 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-04-04T05:53:41.963 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-04-04T05:53:41.963 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-04-04T05:53:41.963 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-04T05:53:41.963 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-04-04T05:53:41.963 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-04-04T05:53:41.964 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-04-04T05:53:41.964 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-04-04T05:53:41.964 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-04T05:53:41.964 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-04-04T05:53:41.964 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-04-04T05:53:41.964 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-04T05:53:41.964 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-04-04T05:53:41.964 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-04-04T05:53:41.964 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-04-04T05:53:41.964 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-04-04T05:53:41.964 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-04-04T05:53:41.964 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-04-04T05:53:41.964 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-04-04T05:53:41.965 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-04-04T05:53:41.965 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-04-04T05:53:41.965 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-04-04T05:53:41.965 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-04-04T05:53:41.965 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-04-04T05:53:41.965 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-04-04T05:53:41.965 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-04-04T05:53:41.965 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-04-04T05:53:41.965 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-04-04T05:53:41.965 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-04-04T05:53:41.965 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-04-04T05:53:41.965 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-04-04T05:53:41.965 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-04-04T05:53:41.965 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-04-04T05:53:41.966 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-04T05:53:41.966 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-04-04T05:53:41.966 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-04-04T05:53:41.966 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-04-04T05:53:41.966 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-04-04T05:53:41.966 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-04-04T05:53:41.966 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-04-04T05:53:41.966 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-04-04T05:53:41.966 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-04-04T05:53:41.966 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-04-04T05:53:41.966 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-04-04T05:53:41.966 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-04-04T05:53:41.966 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-04-04T05:53:41.967 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-04-04T05:53:41.967 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-04-04T05:53:41.967 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-04-04T05:53:41.967 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-04-04T05:53:41.967 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-04-04T05:53:41.967 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-04-04T05:53:41.967 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-04-04T05:53:41.967 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-04-04T05:53:41.967 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-04-04T05:53:41.967 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-04-04T05:53:41.967 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-04-04T05:53:41.967 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-04-04T05:53:41.967 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-04-04T05:53:41.967 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-04-04T05:53:41.967 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-04-04T05:53:41.968 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-04T05:53:41.968 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-04-04T05:53:41.968 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-04T05:53:41.968 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-04-04T05:53:41.968 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-04T05:53:41.968 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-04-04T05:53:41.968 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-04-04T05:53:41.968 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-04-04T05:53:41.968 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-04-04T05:53:41.968 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-04-04T05:53:41.968 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-04-04T05:53:41.968 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-04-04T05:53:41.968 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-04-04T05:53:41.968 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-04-04T05:53:41.969 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-04-04T05:53:41.969 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-04-04T05:53:41.969 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-04-04T05:53:41.969 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-04-04T05:53:41.969 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-04-04T05:53:41.969 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-04-04T05:53:41.969 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-04-04T05:53:41.969 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-04-04T05:53:41.969 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-04-04T05:53:41.969 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-04-04T05:53:41.969 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-04-04T05:53:41.969 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-04-04T05:53:41.969 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-04-04T05:53:41.969 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-04-04T05:53:41.969 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-04-04T05:53:41.969 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-04-04T05:53:41.970 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-04-04T05:53:41.970 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-04-04T05:53:41.970 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-04-04T05:53:41.970 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-04-04T05:53:41.970 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-04-04T05:53:41.970 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-04-04T05:53:41.970 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-04-04T05:53:41.970 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-04-04T05:53:41.970 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-04-04T05:53:41.970 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-04-04T05:53:41.970 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-04-04T05:53:41.970 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-04-04T05:53:41.970 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-04-04T05:53:41.971 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-04-04T05:53:41.971 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-04-04T05:53:41.971 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-04-04T05:53:41.971 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-04-04T05:53:41.971 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-04-04T05:53:41.971 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-04-04T05:53:41.971 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-04-04T05:53:41.971 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-04-04T05:53:41.971 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-04-04T05:53:41.971 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-04-04T05:53:41.971 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-04-04T05:53:41.971 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-04-04T05:53:41.971 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-04-04T05:53:41.971 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-04-04T05:53:41.972 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-04-04T05:53:41.972 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-04-04T05:53:41.972 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-04-04T05:53:41.972 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-04-04T05:53:41.972 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-04-04T05:53:41.972 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-04-04T05:53:41.972 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-04-04T05:53:41.972 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-04T05:53:41.972 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-04-04T05:53:41.972 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-04-04T05:53:41.972 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-04-04T05:53:41.972 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-04T05:53:41.972 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-04-04T05:53:41.972 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-04T05:53:41.972 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-04-04T05:53:41.973 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-04T05:53:41.973 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-04-04T05:53:41.973 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-04T05:53:41.973 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-04T05:53:41.973 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-04T05:53:41.973 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-04-04T05:53:41.973 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-04-04T05:53:41.973 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-04-04T05:53:41.973 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-04-04T05:53:41.973 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-04-04T05:53:41.973 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-04-04T05:53:41.973 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-04-04T05:53:41.973 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-04-04T05:53:41.973 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-04-04T05:53:41.973 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-04-04T05:53:41.973 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-04-04T05:53:41.974 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-04-04T05:53:41.974 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-04-04T05:53:41.974 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-04-04T05:53:41.974 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-04-04T05:53:41.974 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-04-04T05:53:41.974 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-04-04T05:53:41.974 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-04-04T05:53:41.974 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-04-04T05:53:41.974 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-04-04T05:53:41.974 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-04-04T05:53:41.974 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-04-04T05:53:41.974 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-04-04T05:53:41.974 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-04-04T05:53:41.975 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-04-04T05:53:41.975 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-04-04T05:53:41.975 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-04-04T05:53:41.975 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-04-04T05:53:41.975 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-04-04T05:53:41.975 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-04T05:53:41.975 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-04-04T05:53:41.975 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-04-04T05:53:41.975 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-04-04T05:53:41.975 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-04-04T05:53:41.975 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-04-04T05:53:41.975 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-04-04T05:53:41.975 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-04-04T05:53:41.976 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-04-04T05:53:41.976 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-04-04T05:53:41.976 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-04-04T05:53:41.976 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-04-04T05:53:41.976 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-04-04T05:53:41.976 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-04-04T05:53:41.976 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-04-04T05:53:41.976 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-04-04T05:53:41.976 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-04-04T05:53:41.976 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-04-04T05:53:41.976 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-04T05:53:41.976 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-04-04T05:53:41.976 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-04-04T05:53:41.976 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-04-04T05:53:41.976 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-04-04T05:53:41.977 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-04-04T05:53:41.977 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-04-04T05:53:41.977 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-04-04T05:53:41.977 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-04-04T05:53:41.977 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-04-04T05:53:41.977 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-04-04T05:53:41.977 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-04-04T05:53:41.977 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-04-04T05:53:41.977 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-04-04T05:53:41.977 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-04T05:53:41.977 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-04-04T05:53:41.977 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-04-04T05:53:41.977 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-04-04T05:53:41.977 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-04-04T05:53:41.978 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-04-04T05:53:41.978 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-04-04T05:53:41.978 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-04-04T05:53:41.978 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-04-04T05:53:41.978 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-04-04T05:53:41.978 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-04-04T05:53:41.978 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-04-04T05:53:41.978 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-04-04T05:53:41.978 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-04-04T05:53:41.978 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-04-04T05:53:41.978 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-04-04T05:53:41.978 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-04-04T05:53:41.978 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-04-04T05:53:41.978 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-04-04T05:53:41.979 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-04-04T05:53:41.979 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-04-04T05:53:41.979 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-04-04T05:53:41.979 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-04-04T05:53:41.979 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-04T05:53:41.979 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-04T05:53:41.979 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-04T05:53:41.979 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-04T05:53:41.979 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-04T05:53:41.979 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-04T05:53:41.979 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-04T05:53:41.979 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-04T05:53:41.979 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-04T05:53:41.979 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-04T05:53:41.980 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-04T05:53:41.980 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-04T05:53:41.980 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-04T05:53:41.980 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-04T05:53:41.980 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-04T05:53:41.980 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-04T05:53:41.980 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-04T05:53:41.980 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-04T05:53:41.980 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-04T05:53:41.980 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-04T05:53:41.980 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-04-04T05:53:41.980 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-04T05:53:41.980 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-04T05:53:41.980 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-04T05:53:41.981 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-04-04T05:53:41.981 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-04-04T05:53:41.981 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-04-04T05:53:41.981 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-04-04T05:53:41.981 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-04-04T05:53:41.981 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-04-04T05:53:41.981 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-04-04T05:53:41.981 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-04-04T05:53:41.981 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-04-04T05:53:41.981 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-04-04T05:53:41.981 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-04-04T05:53:41.981 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-04-04T05:53:41.981 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-04-04T05:53:41.981 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-04-04T05:53:41.982 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-04-04T05:53:41.982 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-04-04T05:53:41.982 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-04-04T05:53:41.982 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-04-04T05:53:41.982 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-04-04T05:53:41.982 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-04-04T05:53:41.982 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-04T05:53:41.982 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-04-04T05:53:41.982 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-04T05:53:41.982 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-04-04T05:53:41.982 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-04-04T05:53:41.982 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-04-04T05:53:41.982 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-04-04T05:53:41.982 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-04-04T05:53:41.982 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-04-04T05:53:41.983 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-04-04T05:53:41.983 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-04-04T05:53:41.983 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-04-04T05:53:41.983 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-04-04T05:53:41.983 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-04-04T05:53:41.983 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-04-04T05:53:41.983 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-04T05:53:41.983 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-04T05:53:41.983 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-04T05:53:41.983 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-04T05:53:41.983 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-04T05:53:41.983 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-04T05:53:41.983 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-04T05:53:41.983 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-04T05:53:41.983 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-04T05:53:41.984 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-04T05:53:41.984 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-04T05:53:41.984 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-04T05:53:41.984 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-04T05:53:41.984 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-04T05:53:41.984 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-04T05:53:41.984 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-04T05:53:41.984 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-04T05:53:41.984 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-04T05:53:41.984 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-04T05:53:41.984 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-04T05:53:41.984 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-04T05:53:41.984 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-04-04T05:53:41.984 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-04-04T05:53:41.985 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-04-04T05:53:41.985 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-04-04T05:53:41.985 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-04-04T05:53:41.985 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-04-04T05:53:41.985 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-04-04T05:53:41.985 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-04-04T05:53:41.985 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-04-04T05:53:41.985 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-04-04T05:53:41.985 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-04-04T05:53:41.985 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-04-04T05:53:41.985 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-04-04T05:53:41.985 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-04-04T05:53:41.985 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-04-04T05:53:41.985 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-04-04T05:53:41.986 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-04T05:53:41.986 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-04-04T05:53:41.986 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-04-04T05:53:41.986 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-04-04T05:53:41.986 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-04-04T05:53:41.986 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-04-04T05:53:41.986 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-04T05:53:41.986 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-04-04T05:53:41.986 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-04-04T05:53:41.986 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-04-04T05:53:41.986 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-04T05:53:41.986 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-04T05:53:41.986 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-04T05:53:41.986 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-04T05:53:41.987 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-04T05:53:41.987 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-04T05:53:41.987 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-04T05:53:41.987 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-04T05:53:41.987 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-04T05:53:41.987 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-04T05:53:41.987 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-04T05:53:41.987 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-04T05:53:41.987 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-04T05:53:41.987 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-04T05:53:41.987 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-04T05:53:41.987 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-04T05:53:41.987 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-04T05:53:41.987 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-04T05:53:41.988 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-04T05:53:41.988 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-04T05:53:41.988 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-04T05:53:41.988 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-04T05:53:41.988 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-04T05:53:41.988 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-04T05:53:41.988 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-04-04T05:53:41.988 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-04-04T05:53:41.988 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-04T05:53:41.988 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-04-04T05:53:41.988 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-04-04T05:53:41.988 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-04-04T05:53:41.988 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-04-04T05:53:41.988 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-04T05:53:41.988 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-04-04T05:53:41.989 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-04-04T05:53:41.989 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-04-04T05:53:41.989 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-04-04T05:53:41.989 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-04-04T05:53:41.989 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-04-04T05:53:41.989 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-04-04T05:53:41.989 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-04-04T05:53:41.989 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-04-04T05:53:41.989 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-04-04T05:53:41.989 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-04-04T05:53:41.989 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-04-04T05:53:41.989 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-04-04T05:53:41.989 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-04-04T05:53:41.989 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-04T05:53:41.990 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-04T05:53:41.990 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-04T05:53:41.990 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-04T05:53:41.990 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-04T05:53:41.990 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-04T05:53:41.990 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-04T05:53:41.990 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-04T05:53:41.990 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-04T05:53:41.990 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-04T05:53:41.990 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-04T05:53:41.990 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-04T05:53:41.990 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-04T05:53:41.990 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-04T05:53:41.990 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-04T05:53:41.990 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-04-04T05:53:41.990 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-04T05:53:41.991 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-04T05:53:41.991 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-04T05:53:41.991 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-04T05:53:41.991 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-04T05:53:41.991 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-04T05:53:41.991 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-04T05:53:41.991 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-04T05:53:41.991 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-04T05:53:41.991 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-04T05:53:41.991 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-04T05:53:41.991 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-04T05:53:41.991 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-04T05:53:41.991 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-04T05:53:41.992 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-04-04T05:53:41.992 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-04-04T05:53:41.992 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-04T05:53:41.992 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-04T05:53:41.992 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-04T05:53:41.992 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-04T05:53:41.992 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-04T05:53:41.992 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-04T05:53:41.992 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-04T05:53:41.992 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-04T05:53:41.992 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-04T05:53:41.992 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-04T05:53:41.992 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-04T05:53:41.992 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-04T05:53:41.992 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-04T05:53:41.993 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-04-04T05:53:41.993 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-04T05:53:41.993 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-04-04T05:53:41.993 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-04-04T05:53:41.993 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-04T05:53:41.993 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-04T05:53:41.993 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-04-04T05:53:41.993 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-04-04T05:53:41.993 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-04T05:53:41.993 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-04T05:53:41.993 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-04-04T05:53:41.993 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-04-04T05:53:41.993 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-04T05:53:41.993 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-04-04T05:53:41.994 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-04T05:53:41.994 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-04T05:53:41.994 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-04-04T05:53:41.994 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-04-04T05:53:41.994 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-04-04T05:53:41.994 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-04T05:53:41.994 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-04-04T05:53:41.994 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-04T05:53:41.994 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-04-04T05:53:41.994 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-04-04T05:53:41.994 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-04-04T05:53:41.994 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-04-04T05:53:41.994 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-04-04T05:53:41.994 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-04-04T05:53:41.995 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-04-04T05:53:41.995 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-04-04T05:53:41.995 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-04-04T05:53:41.995 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-04-04T05:53:41.995 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-04-04T05:53:41.995 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-04-04T05:53:41.995 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-04-04T05:53:41.995 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-04-04T05:53:41.995 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-04T05:53:41.995 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-04T05:53:41.995 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-04T05:53:41.995 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-04-04T05:53:41.995 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-04-04T05:53:41.995 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-04-04T05:53:41.996 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-04-04T05:53:41.996 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-04T05:53:41.996 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-04-04T05:53:41.996 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-04-04T05:53:41.996 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-04T05:53:41.996 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-04-04T05:53:41.996 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-04-04T05:53:41.996 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-04T05:53:41.996 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-04-04T05:53:41.996 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-04T05:53:41.996 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-04-04T05:53:41.996 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-04T05:53:41.996 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-04T05:53:41.997 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-04-04T05:53:41.997 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-04-04T05:53:41.997 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-04-04T05:53:41.997 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-04-04T05:53:41.997 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-04-04T05:53:41.997 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-04-04T05:53:41.997 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-04-04T05:53:41.997 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-04-04T05:53:41.997 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-04-04T05:53:41.997 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-04-04T05:53:41.997 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-04-04T05:53:41.997 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-04-04T05:53:41.997 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-04-04T05:53:41.997 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-04-04T05:53:41.997 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-04-04T05:53:41.998 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-04-04T05:53:41.998 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-04-04T05:53:41.998 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-04-04T05:53:41.998 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-04-04T05:53:41.998 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-04-04T05:53:41.998 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-04-04T05:53:41.998 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-04-04T05:53:41.998 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-04-04T05:53:41.998 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-04-04T05:53:41.998 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-04-04T05:53:41.998 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-04-04T05:53:41.998 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-04-04T05:53:41.998 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-04-04T05:53:41.998 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-04-04T05:53:41.999 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-04-04T05:53:41.999 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-04-04T05:53:41.999 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-04-04T05:53:41.999 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-04-04T05:53:41.999 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-04-04T05:53:41.999 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-04-04T05:53:41.999 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-04-04T05:53:41.999 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-04-04T05:53:41.999 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-04-04T05:53:41.999 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-04-04T05:53:41.999 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-04-04T05:53:41.999 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-04-04T05:53:41.999 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-04-04T05:53:41.999 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-04-04T05:53:42.000 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-04-04T05:53:42.000 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-04-04T05:53:42.000 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-04-04T05:53:42.000 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-04-04T05:53:42.000 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-04-04T05:53:42.000 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-04-04T05:53:42.000 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-04-04T05:53:42.000 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-04-04T05:53:42.000 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-04-04T05:53:42.000 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-04-04T05:53:42.000 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-04-04T05:53:42.000 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-04-04T05:53:42.000 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-04-04T05:53:42.001 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-04-04T05:53:42.001 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-04-04T05:53:42.001 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-04-04T05:53:42.001 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-04-04T05:53:42.001 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-04-04T05:53:42.001 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-04-04T05:53:42.001 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-04-04T05:53:42.001 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-04-04T05:53:42.001 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-04-04T05:53:42.001 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-04-04T05:53:42.001 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-04-04T05:53:42.001 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-04-04T05:53:42.001 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-04-04T05:53:42.001 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-04-04T05:53:42.001 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-04-04T05:53:42.002 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-04-04T05:53:42.002 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-04-04T05:53:42.002 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-04-04T05:53:42.002 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-04-04T05:53:42.002 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-04-04T05:53:42.002 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-04-04T05:53:42.002 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-04-04T05:53:42.002 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-04-04T05:53:42.002 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-04-04T05:53:42.002 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-04-04T05:53:42.002 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-04-04T05:53:42.002 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-04-04T05:53:42.002 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-04-04T05:53:42.003 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-04-04T05:53:42.003 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-04-04T05:53:42.003 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-04-04T05:53:42.003 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-04-04T05:53:42.003 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-04-04T05:53:42.003 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-04-04T05:53:42.003 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-04-04T05:53:42.003 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-04-04T05:53:42.003 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-04-04T05:53:42.003 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-04-04T05:53:42.003 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-04-04T05:53:42.003 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-04-04T05:53:42.003 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-04-04T05:53:42.003 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-04-04T05:53:42.004 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-04-04T05:53:42.004 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-04-04T05:53:42.004 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-04-04T05:53:42.004 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-04-04T05:53:42.004 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-04-04T05:53:42.004 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-04-04T05:53:42.004 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-04-04T05:53:42.004 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-04-04T05:53:42.004 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-04-04T05:53:42.004 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-04-04T05:53:42.004 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-04-04T05:53:42.004 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-04-04T05:53:42.004 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-04-04T05:53:42.004 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-04-04T05:53:42.004 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-04-04T05:53:42.005 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-04-04T05:53:42.005 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-04-04T05:53:42.005 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-04T05:53:42.005 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-04T05:53:42.005 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-04-04T05:53:42.005 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-04-04T05:53:42.005 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-04T05:53:42.005 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-04-04T05:53:42.005 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-04-04T05:53:42.005 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-04-04T05:53:42.005 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-04-04T05:53:42.005 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-04-04T05:53:42.005 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-04-04T05:53:42.005 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-04T05:53:42.006 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-04T05:53:42.006 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-04-04T05:53:42.006 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-04T05:53:42.006 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-04T05:53:42.006 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-04-04T05:53:42.006 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-04T05:53:42.006 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-04-04T05:53:42.006 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-04-04T05:53:42.006 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-04-04T05:53:42.006 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-04T05:53:42.006 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-04-04T05:53:42.006 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-04-04T05:53:42.006 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-04-04T05:53:42.006 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-04-04T05:53:42.006 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-04-04T05:53:42.007 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-04-04T05:53:42.007 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-04-04T05:53:42.007 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-04-04T05:53:42.007 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-04-04T05:53:42.007 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-04-04T05:53:42.007 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-04-04T05:53:42.007 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-04-04T05:53:42.007 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-04-04T05:53:42.007 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-04-04T05:53:42.007 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-04-04T05:53:42.007 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-04-04T05:53:42.007 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-04-04T05:53:42.007 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-04-04T05:53:42.007 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-04-04T05:53:42.012 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-04-04T05:53:42.012 DEBUG:teuthology.orchestra.run.smithi159:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-04T05:53:42.034 INFO:teuthology.orchestra.run.smithi159.stdout:postinstall scriptlet (using /bin/sh): 2024-04-04T05:53:42.035 INFO:teuthology.orchestra.run.smithi159.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-04T05:53:42.035 INFO:teuthology.orchestra.run.smithi159.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-04-04T05:53:42.035 INFO:teuthology.orchestra.run.smithi159.stdout:fi 2024-04-04T05:53:42.035 INFO:teuthology.orchestra.run.smithi159.stdout:for file in vmlinuz System.map config; do 2024-04-04T05:53:42.035 INFO:teuthology.orchestra.run.smithi159.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-04-04T05:53:42.035 INFO:teuthology.orchestra.run.smithi159.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-04-04T05:53:42.035 INFO:teuthology.orchestra.run.smithi159.stdout: fi 2024-04-04T05:53:42.035 INFO:teuthology.orchestra.run.smithi159.stdout:done 2024-04-04T05:53:42.035 INFO:teuthology.orchestra.run.smithi159.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-04T05:53:42.035 INFO:teuthology.orchestra.run.smithi159.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-04-04T05:53:42.035 INFO:teuthology.orchestra.run.smithi159.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-04-04T05:53:42.035 INFO:teuthology.orchestra.run.smithi159.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-04-04T05:53:42.035 INFO:teuthology.orchestra.run.smithi159.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-04-04T05:53:42.035 INFO:teuthology.orchestra.run.smithi159.stdout:fi 2024-04-04T05:53:42.036 INFO:teuthology.orchestra.run.smithi159.stdout:postuninstall scriptlet (using /bin/sh): 2024-04-04T05:53:42.036 INFO:teuthology.orchestra.run.smithi159.stdout:if [ -x /sbin/update-bootloader ]; then 2024-04-04T05:53:42.036 INFO:teuthology.orchestra.run.smithi159.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-04-04T05:53:42.036 INFO:teuthology.orchestra.run.smithi159.stdout:fi 2024-04-04T05:53:42.037 DEBUG:teuthology.orchestra.run.smithi159:> sudo rpm -qi grub2-tools 2024-04-04T05:53:42.111 INFO:teuthology.orchestra.run.smithi159.stdout:Name : grub2-tools 2024-04-04T05:53:42.112 INFO:teuthology.orchestra.run.smithi159.stdout:Epoch : 1 2024-04-04T05:53:42.112 INFO:teuthology.orchestra.run.smithi159.stdout:Version : 2.06 2024-04-04T05:53:42.112 INFO:teuthology.orchestra.run.smithi159.stdout:Release : 68.el9 2024-04-04T05:53:42.112 INFO:teuthology.orchestra.run.smithi159.stdout:Architecture: x86_64 2024-04-04T05:53:42.112 INFO:teuthology.orchestra.run.smithi159.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-04T05:53:42.112 INFO:teuthology.orchestra.run.smithi159.stdout:Group : Unspecified 2024-04-04T05:53:42.112 INFO:teuthology.orchestra.run.smithi159.stdout:Size : 8264254 2024-04-04T05:53:42.112 INFO:teuthology.orchestra.run.smithi159.stdout:License : GPLv3+ 2024-04-04T05:53:42.112 INFO:teuthology.orchestra.run.smithi159.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-04T05:53:42.112 INFO:teuthology.orchestra.run.smithi159.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-04T05:53:42.112 INFO:teuthology.orchestra.run.smithi159.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-04T05:53:42.112 INFO:teuthology.orchestra.run.smithi159.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-04T05:53:42.112 INFO:teuthology.orchestra.run.smithi159.stdout:Packager : builder@centos.org 2024-04-04T05:53:42.112 INFO:teuthology.orchestra.run.smithi159.stdout:Vendor : CentOS 2024-04-04T05:53:42.113 INFO:teuthology.orchestra.run.smithi159.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-04T05:53:42.113 INFO:teuthology.orchestra.run.smithi159.stdout:Summary : Support tools for GRUB. 2024-04-04T05:53:42.113 INFO:teuthology.orchestra.run.smithi159.stdout:Description : 2024-04-04T05:53:42.113 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T05:53:42.113 INFO:teuthology.orchestra.run.smithi159.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-04T05:53:42.113 INFO:teuthology.orchestra.run.smithi159.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-04T05:53:42.113 INFO:teuthology.orchestra.run.smithi159.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-04T05:53:42.113 INFO:teuthology.orchestra.run.smithi159.stdout:hardware devices. 2024-04-04T05:53:42.113 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T05:53:42.113 INFO:teuthology.orchestra.run.smithi159.stdout:This subpackage provides tools for support of all platforms. 2024-04-04T05:53:42.114 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-04T05:53:42.114 INFO:teuthology.task.kernel:Updating grub on smithi159 to boot 6.8.0-g1df0c7e8a235 2024-04-04T05:53:42.114 DEBUG:teuthology.orchestra.run.smithi159:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-04T05:53:42.711 INFO:teuthology.orchestra.run.smithi159.stderr:Generating grub configuration file ... 2024-04-04T05:53:44.172 INFO:teuthology.orchestra.run.smithi159.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-04T05:53:44.210 INFO:teuthology.orchestra.run.smithi159.stderr:done 2024-04-04T05:53:44.212 DEBUG:teuthology.orchestra.run.smithi159:> mktemp 2024-04-04T05:53:44.227 INFO:teuthology.orchestra.run.smithi159.stdout:/tmp/tmp.HJJLXw7vGY 2024-04-04T05:53:44.227 DEBUG:teuthology.orchestra.run.smithi159:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.HJJLXw7vGY 2024-04-04T05:53:44.296 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 0666 /tmp/tmp.HJJLXw7vGY 2024-04-04T05:53:44.439 DEBUG:teuthology.orchestra.remote:smithi159:/tmp/tmp.HJJLXw7vGY is 6KB 2024-04-04T05:53:44.488 DEBUG:teuthology.orchestra.run.smithi159:> rm -fr /tmp/tmp.HJJLXw7vGY 2024-04-04T05:53:44.501 DEBUG:teuthology.orchestra.run.smithi159:> sudo /bin/ls /boot/loader/entries || true 2024-04-04T05:53:44.571 INFO:teuthology.orchestra.run.smithi159.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-04T05:53:44.571 INFO:teuthology.orchestra.run.smithi159.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-04T05:53:44.571 INFO:teuthology.orchestra.run.smithi159.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-04T05:53:44.571 INFO:teuthology.orchestra.run.smithi159.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-04-04T05:53:44.572 DEBUG:teuthology.orchestra.run.smithi159:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-04-04T05:53:44.740 DEBUG:teuthology.orchestra.run.smithi159:> sudo shutdown -r now 2024-04-04T05:53:57.903 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi178.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-04T05:53:57.903 DEBUG:teuthology.orchestra.run.smithi178:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-04T05:53:57.976 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-04-04T05:53:57.977 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-04-04T05:53:57.977 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-04-04T05:53:57.977 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-04-04T05:53:57.977 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-04-04T05:53:57.977 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-04-04T05:53:57.977 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-04-04T05:53:57.977 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-04T05:53:57.977 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-04T05:53:57.977 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-04T05:53:57.977 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-04T05:53:57.977 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-04T05:53:57.977 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-04T05:53:57.977 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-04T05:53:57.978 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-04T05:53:57.978 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-04T05:53:57.978 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-04T05:53:57.978 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-04T05:53:57.978 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-04T05:53:57.978 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-04T05:53:57.978 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-04T05:53:57.978 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-04T05:53:57.978 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-04T05:53:57.978 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-04T05:53:57.978 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-04T05:53:57.978 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-04T05:53:57.978 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-04T05:53:57.979 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-04-04T05:53:57.979 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-04-04T05:53:57.979 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-04-04T05:53:57.979 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-04T05:53:57.979 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-04-04T05:53:57.979 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-04T05:53:57.979 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-04T05:53:57.979 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-04-04T05:53:57.979 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-04-04T05:53:57.979 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-04-04T05:53:57.979 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-04-04T05:53:57.979 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-04-04T05:53:57.979 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-04-04T05:53:57.979 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-04-04T05:53:57.980 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-04-04T05:53:57.980 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-04T05:53:57.980 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-04-04T05:53:57.980 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-04-04T05:53:57.980 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-04-04T05:53:57.980 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-04-04T05:53:57.980 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-04-04T05:53:57.980 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-04-04T05:53:57.980 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-04-04T05:53:57.980 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-04-04T05:53:57.980 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-04-04T05:53:57.980 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-04-04T05:53:57.980 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-04-04T05:53:57.980 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-04-04T05:53:57.981 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-04-04T05:53:57.981 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-04-04T05:53:57.981 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-04-04T05:53:57.981 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-04-04T05:53:57.981 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-04-04T05:53:57.981 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-04-04T05:53:57.981 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-04-04T05:53:57.981 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-04-04T05:53:57.981 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-04-04T05:53:57.981 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-04-04T05:53:57.981 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-04-04T05:53:57.981 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-04-04T05:53:57.981 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-04-04T05:53:57.981 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-04-04T05:53:57.982 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-04-04T05:53:57.982 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-04-04T05:53:57.982 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-04-04T05:53:57.982 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-04-04T05:53:57.982 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-04-04T05:53:57.982 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-04-04T05:53:57.982 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-04-04T05:53:57.982 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-04-04T05:53:57.982 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-04-04T05:53:57.982 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-04-04T05:53:57.982 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-04-04T05:53:57.982 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-04-04T05:53:57.982 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-04-04T05:53:57.982 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-04-04T05:53:57.983 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-04-04T05:53:57.983 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-04-04T05:53:57.983 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-04-04T05:53:57.983 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-04-04T05:53:57.983 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-04T05:53:57.983 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-04-04T05:53:57.983 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-04-04T05:53:57.983 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-04-04T05:53:57.983 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-04-04T05:53:57.983 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-04-04T05:53:57.983 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-04-04T05:53:57.983 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-04-04T05:53:57.983 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-04-04T05:53:57.983 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-04-04T05:53:57.983 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-04-04T05:53:57.984 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-04-04T05:53:57.984 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-04-04T05:53:57.984 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-04-04T05:53:57.984 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-04-04T05:53:57.984 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-04-04T05:53:57.984 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-04-04T05:53:57.984 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-04-04T05:53:57.984 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-04-04T05:53:57.984 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-04-04T05:53:57.984 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-04-04T05:53:57.984 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-04-04T05:53:57.984 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-04-04T05:53:57.984 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-04-04T05:53:57.984 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-04-04T05:53:57.985 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-04T05:53:57.985 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-04T05:53:57.985 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-04-04T05:53:57.985 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-04-04T05:53:57.985 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-04-04T05:53:57.985 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-04-04T05:53:57.985 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-04-04T05:53:57.985 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-04-04T05:53:57.985 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-04-04T05:53:57.985 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-04-04T05:53:57.985 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-04-04T05:53:57.985 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-04-04T05:53:57.985 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-04-04T05:53:57.985 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-04-04T05:53:57.985 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-04-04T05:53:57.986 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-04T05:53:57.986 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-04T05:53:57.986 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-04-04T05:53:57.986 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-04-04T05:53:57.986 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-04-04T05:53:57.986 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-04-04T05:53:57.986 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-04-04T05:53:57.986 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-04-04T05:53:57.986 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-04-04T05:53:57.986 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-04-04T05:53:57.986 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-04-04T05:53:57.986 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-04-04T05:53:57.986 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-04-04T05:53:57.986 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-04-04T05:53:57.987 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-04-04T05:53:57.987 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-04-04T05:53:57.987 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-04-04T05:53:57.987 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-04-04T05:53:57.987 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-04-04T05:53:57.987 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-04-04T05:53:57.987 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-04-04T05:53:57.987 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-04-04T05:53:57.987 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-04-04T05:53:57.987 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-04-04T05:53:57.987 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-04-04T05:53:57.987 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-04-04T05:53:57.987 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-04-04T05:53:57.987 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-04-04T05:53:57.988 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-04-04T05:53:57.988 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-04T05:53:57.988 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-04T05:53:57.988 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-04-04T05:53:57.988 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-04-04T05:53:57.988 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-04-04T05:53:57.988 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-04-04T05:53:57.988 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-04-04T05:53:57.988 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-04-04T05:53:57.988 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-04-04T05:53:57.988 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-04-04T05:53:57.988 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-04-04T05:53:57.988 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-04-04T05:53:57.988 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-04-04T05:53:57.989 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-04T05:53:57.989 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-04-04T05:53:57.989 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-04-04T05:53:57.989 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-04-04T05:53:57.989 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-04-04T05:53:57.989 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-04-04T05:53:57.989 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-04-04T05:53:57.989 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-04-04T05:53:57.989 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-04T05:53:57.989 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-04-04T05:53:57.989 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-04-04T05:53:57.989 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-04-04T05:53:57.989 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-04-04T05:53:57.989 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-04-04T05:53:57.990 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-04-04T05:53:57.990 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-04T05:53:57.990 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-04T05:53:57.990 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-04T05:53:57.990 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-04-04T05:53:57.990 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-04T05:53:57.990 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-04-04T05:53:57.990 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-04T05:53:57.990 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-04T05:53:57.990 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-04T05:53:57.990 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-04T05:53:57.990 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-04T05:53:57.990 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-04T05:53:57.990 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-04-04T05:53:57.990 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-04-04T05:53:57.991 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-04-04T05:53:57.991 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-04-04T05:53:57.991 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-04-04T05:53:57.991 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-04-04T05:53:57.991 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-04-04T05:53:57.991 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-04T05:53:57.991 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-04T05:53:57.991 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-04T05:53:57.991 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-04T05:53:57.991 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-04T05:53:57.991 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-04-04T05:53:57.991 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-04-04T05:53:57.991 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-04-04T05:53:57.991 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-04T05:53:57.992 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-04T05:53:57.992 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-04T05:53:57.992 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-04T05:53:57.992 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-04T05:53:57.992 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-04T05:53:57.992 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-04-04T05:53:57.992 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-04-04T05:53:57.992 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-04T05:53:57.992 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-04-04T05:53:57.992 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-04-04T05:53:57.992 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-04-04T05:53:57.992 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-04-04T05:53:57.992 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-04T05:53:57.992 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-04T05:53:57.992 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-04T05:53:57.993 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-04-04T05:53:57.993 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-04-04T05:53:57.993 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-04-04T05:53:57.993 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-04T05:53:57.993 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-04-04T05:53:57.993 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-04-04T05:53:57.993 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-04-04T05:53:57.993 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-04T05:53:57.993 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-04-04T05:53:57.993 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-04-04T05:53:57.993 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-04T05:53:57.993 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-04-04T05:53:57.993 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-04T05:53:57.993 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-04-04T05:53:57.993 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-04-04T05:53:57.994 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-04-04T05:53:57.994 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-04-04T05:53:57.994 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-04-04T05:53:57.994 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-04-04T05:53:57.994 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-04-04T05:53:57.994 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-04-04T05:53:57.994 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-04-04T05:53:57.994 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-04-04T05:53:57.994 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-04-04T05:53:57.994 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-04-04T05:53:57.994 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-04-04T05:53:57.994 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-04-04T05:53:57.994 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-04-04T05:53:57.994 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-04-04T05:53:57.995 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-04T05:53:57.995 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-04-04T05:53:57.995 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-04-04T05:53:57.995 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-04T05:53:57.995 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-04-04T05:53:57.995 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-04-04T05:53:57.995 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-04-04T05:53:57.995 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-04-04T05:53:57.995 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-04-04T05:53:57.995 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-04-04T05:53:57.995 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-04-04T05:53:57.995 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-04-04T05:53:57.995 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-04-04T05:53:57.995 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-04-04T05:53:57.995 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-04-04T05:53:57.996 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-04T05:53:57.996 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-04T05:53:57.996 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-04-04T05:53:57.996 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-04-04T05:53:57.996 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-04-04T05:53:57.996 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-04-04T05:53:57.996 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-04-04T05:53:57.996 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-04-04T05:53:57.996 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-04-04T05:53:57.996 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-04-04T05:53:57.996 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-04T05:53:57.996 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-04T05:53:57.996 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-04-04T05:53:57.996 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-04-04T05:53:57.997 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-04-04T05:53:57.997 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-04-04T05:53:57.997 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-04-04T05:53:57.997 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-04-04T05:53:57.997 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-04-04T05:53:57.997 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-04-04T05:53:57.997 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-04-04T05:53:57.997 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-04T05:53:57.997 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-04-04T05:53:57.997 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-04T05:53:57.997 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-04T05:53:57.997 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-04T05:53:57.997 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-04T05:53:57.997 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-04T05:53:57.997 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-04T05:53:57.997 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-04T05:53:57.998 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-04T05:53:57.998 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-04T05:53:57.998 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-04-04T05:53:57.998 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-04-04T05:53:57.998 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-04-04T05:53:57.998 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-04T05:53:57.998 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-04-04T05:53:57.998 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-04-04T05:53:57.998 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-04-04T05:53:57.998 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-04-04T05:53:57.998 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-04-04T05:53:57.998 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-04T05:53:57.998 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-04-04T05:53:57.998 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-04-04T05:53:57.999 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-04-04T05:53:57.999 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-04-04T05:53:57.999 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-04-04T05:53:57.999 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-04-04T05:53:57.999 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-04-04T05:53:57.999 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-04-04T05:53:57.999 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-04-04T05:53:57.999 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-04-04T05:53:57.999 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-04-04T05:53:57.999 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-04-04T05:53:57.999 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-04-04T05:53:57.999 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-04-04T05:53:57.999 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-04-04T05:53:57.999 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-04-04T05:53:58.000 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-04-04T05:53:58.000 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-04-04T05:53:58.000 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-04-04T05:53:58.000 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-04T05:53:58.000 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-04-04T05:53:58.000 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-04-04T05:53:58.000 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-04-04T05:53:58.000 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-04-04T05:53:58.000 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-04-04T05:53:58.000 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-04-04T05:53:58.000 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-04-04T05:53:58.000 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-04-04T05:53:58.000 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-04-04T05:53:58.000 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-04-04T05:53:58.000 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-04-04T05:53:58.001 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-04-04T05:53:58.001 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-04-04T05:53:58.001 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-04-04T05:53:58.001 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-04-04T05:53:58.001 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-04-04T05:53:58.001 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-04-04T05:53:58.001 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-04-04T05:53:58.001 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-04-04T05:53:58.001 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-04T05:53:58.001 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-04-04T05:53:58.001 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-04-04T05:53:58.001 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-04T05:53:58.001 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-04-04T05:53:58.001 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-04-04T05:53:58.002 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-04-04T05:53:58.002 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-04-04T05:53:58.002 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-04-04T05:53:58.002 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-04-04T05:53:58.002 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-04-04T05:53:58.002 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-04-04T05:53:58.002 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-04-04T05:53:58.002 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-04-04T05:53:58.002 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-04-04T05:53:58.002 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-04-04T05:53:58.002 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-04-04T05:53:58.002 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-04-04T05:53:58.002 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-04T05:53:58.002 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-04-04T05:53:58.003 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-04-04T05:53:58.003 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-04-04T05:53:58.003 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-04-04T05:53:58.003 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-04-04T05:53:58.003 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-04-04T05:53:58.003 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-04-04T05:53:58.003 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-04-04T05:53:58.003 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-04-04T05:53:58.003 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-04-04T05:53:58.003 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-04-04T05:53:58.003 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-04-04T05:53:58.003 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-04-04T05:53:58.003 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-04-04T05:53:58.003 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-04-04T05:53:58.004 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-04-04T05:53:58.004 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-04-04T05:53:58.004 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-04-04T05:53:58.004 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-04-04T05:53:58.004 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-04-04T05:53:58.004 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-04-04T05:53:58.004 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-04-04T05:53:58.004 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-04-04T05:53:58.004 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-04-04T05:53:58.004 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-04-04T05:53:58.004 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-04-04T05:53:58.004 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-04-04T05:53:58.004 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-04-04T05:53:58.005 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-04-04T05:53:58.005 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-04-04T05:53:58.005 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-04-04T05:53:58.005 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-04-04T05:53:58.005 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-04-04T05:53:58.005 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-04-04T05:53:58.005 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-04-04T05:53:58.005 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-04-04T05:53:58.005 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-04-04T05:53:58.005 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-04-04T05:53:58.005 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-04-04T05:53:58.005 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-04-04T05:53:58.005 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-04-04T05:53:58.005 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-04-04T05:53:58.005 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-04-04T05:53:58.006 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-04-04T05:53:58.006 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-04-04T05:53:58.006 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-04-04T05:53:58.006 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-04-04T05:53:58.006 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-04-04T05:53:58.006 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-04T05:53:58.006 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-04T05:53:58.006 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-04T05:53:58.006 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-04T05:53:58.006 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-04T05:53:58.006 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-04T05:53:58.006 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-04T05:53:58.006 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-04T05:53:58.006 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-04T05:53:58.007 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-04T05:53:58.007 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-04T05:53:58.007 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-04T05:53:58.007 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-04-04T05:53:58.007 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-04-04T05:53:58.007 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-04-04T05:53:58.007 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-04T05:53:58.007 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-04-04T05:53:58.007 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-04-04T05:53:58.007 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-04-04T05:53:58.007 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-04-04T05:53:58.007 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-04-04T05:53:58.007 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-04-04T05:53:58.007 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-04-04T05:53:58.007 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-04-04T05:53:58.008 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-04-04T05:53:58.008 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-04-04T05:53:58.008 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-04-04T05:53:58.008 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-04-04T05:53:58.008 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-04-04T05:53:58.008 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-04-04T05:53:58.008 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-04-04T05:53:58.008 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-04-04T05:53:58.008 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-04-04T05:53:58.008 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-04-04T05:53:58.008 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-04-04T05:53:58.008 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-04-04T05:53:58.008 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-04-04T05:53:58.008 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-04-04T05:53:58.009 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-04-04T05:53:58.009 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-04-04T05:53:58.009 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-04-04T05:53:58.009 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-04-04T05:53:58.009 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-04-04T05:53:58.009 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-04T05:53:58.009 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-04T05:53:58.009 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-04-04T05:53:58.009 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-04T05:53:58.009 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-04T05:53:58.009 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-04T05:53:58.009 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-04T05:53:58.009 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-04T05:53:58.009 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-04T05:53:58.010 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-04T05:53:58.010 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-04T05:53:58.010 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-04T05:53:58.010 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-04T05:53:58.010 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-04T05:53:58.010 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-04T05:53:58.010 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-04T05:53:58.010 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-04T05:53:58.010 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-04T05:53:58.010 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-04T05:53:58.010 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-04T05:53:58.010 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-04T05:53:58.010 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-04T05:53:58.010 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-04T05:53:58.011 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-04T05:53:58.011 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-04T05:53:58.011 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-04T05:53:58.011 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-04-04T05:53:58.011 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-04-04T05:53:58.011 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-04T05:53:58.011 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-04-04T05:53:58.011 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-04-04T05:53:58.011 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-04-04T05:53:58.011 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-04-04T05:53:58.011 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-04T05:53:58.011 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-04-04T05:53:58.011 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-04T05:53:58.012 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-04T05:53:58.012 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-04T05:53:58.012 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-04T05:53:58.012 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-04T05:53:58.012 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-04-04T05:53:58.012 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-04-04T05:53:58.012 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-04T05:53:58.012 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-04-04T05:53:58.012 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-04T05:53:58.012 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-04-04T05:53:58.012 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-04T05:53:58.012 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-04-04T05:53:58.012 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-04T05:53:58.012 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-04-04T05:53:58.013 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-04-04T05:53:58.013 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-04T05:53:58.013 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-04-04T05:53:58.013 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-04T05:53:58.013 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-04-04T05:53:58.013 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-04T05:53:58.013 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-04-04T05:53:58.013 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-04T05:53:58.013 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-04-04T05:53:58.013 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-04T05:53:58.013 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-04-04T05:53:58.013 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-04-04T05:53:58.013 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-04T05:53:58.013 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-04T05:53:58.013 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-04-04T05:53:58.014 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-04-04T05:53:58.014 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-04-04T05:53:58.014 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-04T05:53:58.014 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-04-04T05:53:58.014 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-04-04T05:53:58.014 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-04-04T05:53:58.014 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-04-04T05:53:58.014 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-04T05:53:58.014 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-04-04T05:53:58.014 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-04T05:53:58.014 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-04T05:53:58.014 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-04T05:53:58.014 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-04-04T05:53:58.014 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-04T05:53:58.015 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-04-04T05:53:58.015 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-04-04T05:53:58.015 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-04-04T05:53:58.015 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-04T05:53:58.015 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-04-04T05:53:58.015 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-04-04T05:53:58.015 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-04-04T05:53:58.015 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-04-04T05:53:58.015 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-04-04T05:53:58.015 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-04-04T05:53:58.015 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-04-04T05:53:58.015 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-04-04T05:53:58.015 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-04T05:53:58.015 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-04T05:53:58.016 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-04T05:53:58.016 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-04T05:53:58.016 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-04T05:53:58.016 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-04T05:53:58.016 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-04T05:53:58.016 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-04T05:53:58.016 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-04T05:53:58.016 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-04-04T05:53:58.016 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-04-04T05:53:58.016 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-04-04T05:53:58.016 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-04-04T05:53:58.016 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-04-04T05:53:58.016 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-04-04T05:53:58.017 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-04-04T05:53:58.017 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-04-04T05:53:58.017 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-04-04T05:53:58.017 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-04-04T05:53:58.017 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-04-04T05:53:58.017 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-04-04T05:53:58.017 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-04-04T05:53:58.017 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-04-04T05:53:58.017 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-04-04T05:53:58.017 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-04-04T05:53:58.017 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-04-04T05:53:58.017 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-04-04T05:53:58.017 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-04-04T05:53:58.017 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-04T05:53:58.018 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-04-04T05:53:58.018 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-04-04T05:53:58.018 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-04-04T05:53:58.018 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-04-04T05:53:58.018 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-04-04T05:53:58.018 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-04-04T05:53:58.018 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-04-04T05:53:58.018 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-04-04T05:53:58.018 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-04-04T05:53:58.018 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-04-04T05:53:58.018 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-04-04T05:53:58.018 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-04-04T05:53:58.019 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-04-04T05:53:58.019 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-04-04T05:53:58.019 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-04-04T05:53:58.019 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-04-04T05:53:58.019 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-04-04T05:53:58.019 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-04-04T05:53:58.019 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-04-04T05:53:58.019 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-04-04T05:53:58.019 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-04-04T05:53:58.019 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-04-04T05:53:58.019 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-04-04T05:53:58.019 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-04T05:53:58.019 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-04-04T05:53:58.019 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-04-04T05:53:58.020 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-04-04T05:53:58.020 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-04-04T05:53:58.020 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-04-04T05:53:58.020 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-04-04T05:53:58.020 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-04-04T05:53:58.020 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-04T05:53:58.020 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-04-04T05:53:58.020 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-04-04T05:53:58.020 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-04-04T05:53:58.020 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-04-04T05:53:58.020 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-04-04T05:53:58.020 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-04-04T05:53:58.020 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-04T05:53:58.020 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-04T05:53:58.021 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-04-04T05:53:58.021 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-04-04T05:53:58.021 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-04-04T05:53:58.021 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-04-04T05:53:58.021 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-04-04T05:53:58.021 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-04T05:53:58.021 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-04-04T05:53:58.021 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-04T05:53:58.021 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-04T05:53:58.021 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-04T05:53:58.021 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-04-04T05:53:58.021 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-04-04T05:53:58.021 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-04-04T05:53:58.022 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-04-04T05:53:58.022 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-04-04T05:53:58.022 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-04T05:53:58.022 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-04-04T05:53:58.022 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-04-04T05:53:58.022 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-04-04T05:53:58.022 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-04-04T05:53:58.022 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-04-04T05:53:58.022 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-04T05:53:58.022 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-04T05:53:58.022 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-04T05:53:58.022 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-04-04T05:53:58.022 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-04T05:53:58.023 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-04-04T05:53:58.023 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-04-04T05:53:58.023 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-04-04T05:53:58.023 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-04-04T05:53:58.023 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-04T05:53:58.023 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-04T05:53:58.023 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-04T05:53:58.023 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-04T05:53:58.023 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-04-04T05:53:58.023 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-04T05:53:58.023 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-04T05:53:58.023 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-04T05:53:58.023 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-04T05:53:58.024 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-04-04T05:53:58.024 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-04T05:53:58.024 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-04-04T05:53:58.024 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-04T05:53:58.024 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-04-04T05:53:58.024 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-04T05:53:58.024 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-04-04T05:53:58.024 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-04T05:53:58.024 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-04-04T05:53:58.024 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-04T05:53:58.024 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-04T05:53:58.024 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-04T05:53:58.024 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-04-04T05:53:58.025 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-04-04T05:53:58.025 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-04T05:53:58.025 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-04T05:53:58.025 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-04T05:53:58.025 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-04T05:53:58.025 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-04T05:53:58.025 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-04-04T05:53:58.025 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-04T05:53:58.025 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-04T05:53:58.025 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-04-04T05:53:58.025 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-04T05:53:58.025 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-04T05:53:58.025 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-04T05:53:58.025 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-04T05:53:58.026 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-04T05:53:58.026 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-04-04T05:53:58.026 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-04T05:53:58.026 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-04T05:53:58.026 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-04-04T05:53:58.026 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-04-04T05:53:58.026 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-04T05:53:58.026 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-04-04T05:53:58.026 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-04T05:53:58.026 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-04T05:53:58.026 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-04T05:53:58.026 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-04T05:53:58.026 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-04T05:53:58.027 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-04T05:53:58.027 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-04T05:53:58.027 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-04T05:53:58.027 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-04T05:53:58.027 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-04T05:53:58.027 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-04-04T05:53:58.027 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-04-04T05:53:58.027 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-04T05:53:58.027 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-04-04T05:53:58.027 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-04-04T05:53:58.027 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-04T05:53:58.027 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-04T05:53:58.027 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-04T05:53:58.027 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-04T05:53:58.027 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-04T05:53:58.028 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-04T05:53:58.028 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-04-04T05:53:58.028 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-04T05:53:58.028 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-04T05:53:58.028 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-04-04T05:53:58.028 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-04-04T05:53:58.028 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-04-04T05:53:58.028 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-04T05:53:58.028 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-04-04T05:53:58.028 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-04-04T05:53:58.028 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-04-04T05:53:58.028 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-04T05:53:58.028 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-04-04T05:53:58.028 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-04T05:53:58.028 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-04-04T05:53:58.029 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-04T05:53:58.029 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-04-04T05:53:58.029 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-04T05:53:58.029 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-04-04T05:53:58.029 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-04T05:53:58.029 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-04-04T05:53:58.029 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-04T05:53:58.029 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-04-04T05:53:58.029 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-04T05:53:58.029 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-04-04T05:53:58.029 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-04T05:53:58.029 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-04T05:53:58.029 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-04T05:53:58.029 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-04T05:53:58.030 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-04T05:53:58.030 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-04-04T05:53:58.030 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-04-04T05:53:58.030 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-04T05:53:58.030 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-04T05:53:58.030 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-04T05:53:58.030 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-04-04T05:53:58.030 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-04T05:53:58.030 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-04T05:53:58.030 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-04T05:53:58.030 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-04T05:53:58.030 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-04T05:53:58.031 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-04T05:53:58.031 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-04-04T05:53:58.031 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-04T05:53:58.031 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-04-04T05:53:58.031 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-04T05:53:58.031 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-04T05:53:58.031 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-04-04T05:53:58.031 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-04T05:53:58.031 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-04T05:53:58.031 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-04-04T05:53:58.031 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-04T05:53:58.031 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-04-04T05:53:58.031 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-04T05:53:58.032 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-04-04T05:53:58.032 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-04T05:53:58.032 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-04T05:53:58.032 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-04-04T05:53:58.032 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-04T05:53:58.032 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-04T05:53:58.032 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-04T05:53:58.032 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-04T05:53:58.032 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-04T05:53:58.032 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-04-04T05:53:58.032 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-04T05:53:58.032 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-04-04T05:53:58.032 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-04T05:53:58.032 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-04T05:53:58.033 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-04T05:53:58.033 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-04T05:53:58.033 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-04-04T05:53:58.033 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-04T05:53:58.033 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-04-04T05:53:58.033 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-04T05:53:58.033 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-04-04T05:53:58.033 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-04T05:53:58.033 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-04-04T05:53:58.033 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-04T05:53:58.033 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-04T05:53:58.033 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-04-04T05:53:58.033 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-04T05:53:58.033 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-04T05:53:58.034 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-04T05:53:58.034 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-04T05:53:58.034 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-04-04T05:53:58.034 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-04T05:53:58.034 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-04T05:53:58.034 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-04T05:53:58.034 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-04-04T05:53:58.034 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-04T05:53:58.034 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-04T05:53:58.034 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-04T05:53:58.034 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-04T05:53:58.034 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-04-04T05:53:58.034 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-04T05:53:58.035 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-04-04T05:53:58.035 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-04T05:53:58.035 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-04-04T05:53:58.035 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-04T05:53:58.035 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-04T05:53:58.035 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-04-04T05:53:58.035 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-04T05:53:58.035 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-04T05:53:58.035 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-04-04T05:53:58.035 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-04T05:53:58.035 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-04-04T05:53:58.035 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-04-04T05:53:58.035 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-04T05:53:58.035 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-04-04T05:53:58.035 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-04T05:53:58.036 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-04-04T05:53:58.036 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-04-04T05:53:58.036 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-04T05:53:58.036 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-04-04T05:53:58.036 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-04-04T05:53:58.036 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-04-04T05:53:58.036 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-04-04T05:53:58.036 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-04T05:53:58.036 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-04T05:53:58.036 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-04T05:53:58.036 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-04-04T05:53:58.036 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-04-04T05:53:58.036 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-04-04T05:53:58.036 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-04-04T05:53:58.037 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-04-04T05:53:58.037 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-04T05:53:58.037 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-04T05:53:58.037 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-04-04T05:53:58.037 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-04-04T05:53:58.037 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-04T05:53:58.037 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-04T05:53:58.037 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-04T05:53:58.037 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-04-04T05:53:58.037 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-04-04T05:53:58.037 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-04-04T05:53:58.037 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-04-04T05:53:58.037 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-04-04T05:53:58.037 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-04-04T05:53:58.038 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-04-04T05:53:58.038 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-04-04T05:53:58.038 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-04-04T05:53:58.038 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-04-04T05:53:58.038 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-04-04T05:53:58.038 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-04-04T05:53:58.038 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-04-04T05:53:58.038 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-04-04T05:53:58.038 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-04-04T05:53:58.038 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-04-04T05:53:58.038 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-04-04T05:53:58.038 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-04-04T05:53:58.038 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-04T05:53:58.038 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-04-04T05:53:58.039 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-04T05:53:58.039 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-04T05:53:58.039 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-04T05:53:58.039 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-04T05:53:58.039 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-04-04T05:53:58.039 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-04-04T05:53:58.039 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-04-04T05:53:58.039 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-04-04T05:53:58.039 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-04-04T05:53:58.039 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-04-04T05:53:58.039 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-04-04T05:53:58.039 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-04-04T05:53:58.039 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-04-04T05:53:58.039 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-04-04T05:53:58.039 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-04T05:53:58.040 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-04T05:53:58.040 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-04T05:53:58.040 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-04-04T05:53:58.040 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-04T05:53:58.040 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-04-04T05:53:58.040 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-04-04T05:53:58.040 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-04-04T05:53:58.040 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-04-04T05:53:58.040 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-04T05:53:58.040 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-04-04T05:53:58.040 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-04-04T05:53:58.040 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-04-04T05:53:58.040 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-04T05:53:58.040 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-04-04T05:53:58.041 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-04-04T05:53:58.041 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-04-04T05:53:58.041 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-04-04T05:53:58.041 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-04-04T05:53:58.041 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-04-04T05:53:58.041 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-04-04T05:53:58.041 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-04-04T05:53:58.041 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-04-04T05:53:58.041 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-04-04T05:53:58.041 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-04-04T05:53:58.041 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-04-04T05:53:58.041 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-04-04T05:53:58.041 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-04-04T05:53:58.041 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-04T05:53:58.042 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-04-04T05:53:58.042 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-04T05:53:58.042 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-04-04T05:53:58.042 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-04-04T05:53:58.042 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-04-04T05:53:58.042 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-04-04T05:53:58.042 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-04T05:53:58.042 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-04-04T05:53:58.042 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-04-04T05:53:58.042 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-04T05:53:58.042 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-04-04T05:53:58.042 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-04-04T05:53:58.042 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-04T05:53:58.042 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-04-04T05:53:58.043 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-04T05:53:58.043 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-04T05:53:58.043 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-04T05:53:58.043 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-04T05:53:58.043 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-04T05:53:58.043 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-04T05:53:58.043 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-04T05:53:58.043 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-04T05:53:58.043 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-04T05:53:58.043 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-04-04T05:53:58.043 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-04-04T05:53:58.043 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-04T05:53:58.043 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-04-04T05:53:58.043 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-04T05:53:58.043 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-04T05:53:58.044 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-04T05:53:58.044 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-04T05:53:58.044 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-04T05:53:58.044 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-04T05:53:58.044 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-04T05:53:58.044 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-04T05:53:58.044 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-04T05:53:58.044 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-04-04T05:53:58.044 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-04-04T05:53:58.044 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-04T05:53:58.044 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-04T05:53:58.044 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-04-04T05:53:58.044 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-04-04T05:53:58.044 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-04T05:53:58.045 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-04T05:53:58.045 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-04-04T05:53:58.045 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-04-04T05:53:58.045 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-04-04T05:53:58.045 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-04-04T05:53:58.045 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-04-04T05:53:58.045 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-04-04T05:53:58.045 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-04T05:53:58.045 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-04T05:53:58.045 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-04T05:53:58.045 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-04T05:53:58.045 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-04T05:53:58.045 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-04T05:53:58.046 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-04T05:53:58.046 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-04T05:53:58.046 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-04T05:53:58.046 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-04T05:53:58.046 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-04-04T05:53:58.046 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-04T05:53:58.046 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-04T05:53:58.046 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-04T05:53:58.046 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-04T05:53:58.046 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-04T05:53:58.046 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-04T05:53:58.046 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-04-04T05:53:58.046 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-04T05:53:58.046 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-04T05:53:58.046 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-04T05:53:58.047 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-04T05:53:58.047 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-04T05:53:58.047 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-04T05:53:58.047 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-04T05:53:58.047 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-04T05:53:58.047 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-04T05:53:58.047 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-04T05:53:58.047 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-04T05:53:58.047 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-04T05:53:58.047 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-04-04T05:53:58.047 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-04-04T05:53:58.047 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-04-04T05:53:58.047 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-04-04T05:53:58.047 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-04-04T05:53:58.048 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-04-04T05:53:58.048 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-04-04T05:53:58.048 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-04-04T05:53:58.048 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-04T05:53:58.048 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-04-04T05:53:58.048 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-04-04T05:53:58.048 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-04T05:53:58.048 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-04T05:53:58.048 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-04-04T05:53:58.048 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-04-04T05:53:58.048 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-04T05:53:58.048 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-04-04T05:53:58.048 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-04-04T05:53:58.048 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-04T05:53:58.049 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-04-04T05:53:58.049 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-04-04T05:53:58.049 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-04-04T05:53:58.049 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-04T05:53:58.049 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-04-04T05:53:58.049 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-04T05:53:58.049 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-04-04T05:53:58.049 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-04-04T05:53:58.049 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-04T05:53:58.049 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-04-04T05:53:58.049 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-04T05:53:58.049 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-04T05:53:58.049 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-04T05:53:58.049 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-04T05:53:58.050 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-04T05:53:58.050 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-04-04T05:53:58.050 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-04-04T05:53:58.050 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-04T05:53:58.050 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-04T05:53:58.050 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-04T05:53:58.050 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-04T05:53:58.050 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-04-04T05:53:58.050 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-04-04T05:53:58.050 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-04T05:53:58.050 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-04-04T05:53:58.050 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-04-04T05:53:58.050 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-04T05:53:58.051 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-04T05:53:58.051 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-04-04T05:53:58.051 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-04-04T05:53:58.051 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-04-04T05:53:58.051 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-04-04T05:53:58.051 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-04-04T05:53:58.051 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-04-04T05:53:58.051 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-04-04T05:53:58.051 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-04-04T05:53:58.051 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-04-04T05:53:58.051 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-04-04T05:53:58.051 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-04T05:53:58.051 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-04T05:53:58.051 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-04-04T05:53:58.051 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-04-04T05:53:58.052 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-04-04T05:53:58.052 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-04T05:53:58.052 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-04-04T05:53:58.052 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-04-04T05:53:58.052 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-04-04T05:53:58.052 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-04T05:53:58.052 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-04-04T05:53:58.052 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-04-04T05:53:58.052 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-04T05:53:58.052 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-04T05:53:58.052 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-04T05:53:58.052 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-04-04T05:53:58.052 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-04T05:53:58.052 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-04-04T05:53:58.053 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-04T05:53:58.053 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-04-04T05:53:58.053 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-04-04T05:53:58.053 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-04T05:53:58.053 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-04-04T05:53:58.053 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-04-04T05:53:58.053 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-04-04T05:53:58.053 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-04T05:53:58.053 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-04T05:53:58.053 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-04-04T05:53:58.053 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-04T05:53:58.053 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-04-04T05:53:58.053 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-04-04T05:53:58.053 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-04T05:53:58.053 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-04T05:53:58.054 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-04T05:53:58.054 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-04T05:53:58.054 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-04T05:53:58.054 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-04-04T05:53:58.054 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-04-04T05:53:58.054 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-04-04T05:53:58.054 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-04-04T05:53:58.054 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-04T05:53:58.054 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-04-04T05:53:58.054 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-04T05:53:58.054 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-04-04T05:53:58.054 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-04-04T05:53:58.055 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-04-04T05:53:58.055 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-04-04T05:53:58.055 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-04-04T05:53:58.055 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-04T05:53:58.055 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-04-04T05:53:58.055 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-04T05:53:58.055 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-04-04T05:53:58.055 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-04-04T05:53:58.055 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-04-04T05:53:58.055 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-04-04T05:53:58.055 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-04-04T05:53:58.055 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-04-04T05:53:58.055 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-04T05:53:58.055 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-04-04T05:53:58.055 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-04-04T05:53:58.056 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-04-04T05:53:58.056 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-04T05:53:58.056 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-04T05:53:58.056 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-04T05:53:58.056 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-04T05:53:58.056 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-04T05:53:58.056 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-04T05:53:58.056 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-04T05:53:58.056 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-04T05:53:58.056 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-04T05:53:58.056 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-04T05:53:58.056 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-04T05:53:58.056 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-04T05:53:58.056 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-04T05:53:58.056 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-04T05:53:58.056 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-04T05:53:58.057 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-04T05:53:58.057 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-04T05:53:58.057 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-04T05:53:58.057 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-04-04T05:53:58.057 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-04-04T05:53:58.057 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-04T05:53:58.057 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-04-04T05:53:58.057 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-04-04T05:53:58.057 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-04-04T05:53:58.057 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-04-04T05:53:58.057 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-04-04T05:53:58.057 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-04T05:53:58.057 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-04-04T05:53:58.057 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-04-04T05:53:58.058 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-04T05:53:58.058 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-04-04T05:53:58.058 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-04-04T05:53:58.058 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-04T05:53:58.058 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-04-04T05:53:58.058 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-04T05:53:58.058 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-04-04T05:53:58.058 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-04-04T05:53:58.058 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-04-04T05:53:58.058 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-04-04T05:53:58.058 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-04-04T05:53:58.058 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-04-04T05:53:58.058 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-04-04T05:53:58.058 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-04-04T05:53:58.059 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-04-04T05:53:58.059 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-04T05:53:58.059 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-04-04T05:53:58.059 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-04-04T05:53:58.059 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-04-04T05:53:58.059 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-04T05:53:58.059 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-04-04T05:53:58.059 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-04-04T05:53:58.059 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-04-04T05:53:58.059 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-04-04T05:53:58.059 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-04-04T05:53:58.059 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-04-04T05:53:58.059 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-04-04T05:53:58.059 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-04-04T05:53:58.060 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-04-04T05:53:58.060 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-04-04T05:53:58.060 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-04-04T05:53:58.060 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-04T05:53:58.060 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-04-04T05:53:58.060 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-04-04T05:53:58.060 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-04-04T05:53:58.060 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-04T05:53:58.060 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-04T05:53:58.060 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-04-04T05:53:58.060 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-04-04T05:53:58.060 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-04T05:53:58.060 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-04-04T05:53:58.060 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-04-04T05:53:58.060 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-04-04T05:53:58.061 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-04-04T05:53:58.061 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-04-04T05:53:58.061 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-04-04T05:53:58.061 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-04-04T05:53:58.061 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-04T05:53:58.061 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-04-04T05:53:58.061 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-04-04T05:53:58.061 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-04-04T05:53:58.061 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-04T05:53:58.061 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-04T05:53:58.061 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-04-04T05:53:58.061 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-04T05:53:58.061 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-04T05:53:58.061 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-04T05:53:58.062 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-04-04T05:53:58.062 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-04-04T05:53:58.062 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-04-04T05:53:58.062 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-04-04T05:53:58.062 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-04T05:53:58.062 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-04-04T05:53:58.062 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-04T05:53:58.062 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-04T05:53:58.062 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-04T05:53:58.062 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-04-04T05:53:58.062 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-04-04T05:53:58.062 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-04-04T05:53:58.062 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-04-04T05:53:58.062 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-04-04T05:53:58.063 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-04-04T05:53:58.063 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-04-04T05:53:58.063 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-04-04T05:53:58.063 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-04-04T05:53:58.063 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-04-04T05:53:58.063 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-04-04T05:53:58.063 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-04-04T05:53:58.063 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-04-04T05:53:58.063 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-04-04T05:53:58.063 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-04T05:53:58.063 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-04-04T05:53:58.063 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-04-04T05:53:58.063 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-04T05:53:58.064 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-04T05:53:58.064 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-04-04T05:53:58.064 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-04T05:53:58.064 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-04-04T05:53:58.064 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-04-04T05:53:58.064 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-04T05:53:58.064 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-04-04T05:53:58.064 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-04-04T05:53:58.064 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-04T05:53:58.064 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-04T05:53:58.064 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-04T05:53:58.064 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-04T05:53:58.064 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-04T05:53:58.064 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-04T05:53:58.065 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-04T05:53:58.065 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-04-04T05:53:58.065 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-04T05:53:58.065 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-04T05:53:58.065 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-04T05:53:58.065 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-04T05:53:58.065 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-04T05:53:58.065 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-04-04T05:53:58.065 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-04-04T05:53:58.065 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-04T05:53:58.065 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-04T05:53:58.065 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-04T05:53:58.065 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-04-04T05:53:58.065 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-04-04T05:53:58.066 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-04T05:53:58.066 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-04T05:53:58.066 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-04-04T05:53:58.066 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-04T05:53:58.066 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-04-04T05:53:58.066 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-04-04T05:53:58.066 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-04T05:53:58.066 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-04-04T05:53:58.066 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-04-04T05:53:58.066 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-04-04T05:53:58.066 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-04-04T05:53:58.066 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-04-04T05:53:58.066 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-04-04T05:53:58.067 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-04T05:53:58.067 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-04-04T05:53:58.067 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-04-04T05:53:58.067 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-04-04T05:53:58.067 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-04-04T05:53:58.067 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-04-04T05:53:58.067 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-04T05:53:58.067 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-04T05:53:58.067 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-04-04T05:53:58.067 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-04-04T05:53:58.067 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-04-04T05:53:58.067 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-04-04T05:53:58.067 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-04-04T05:53:58.067 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-04-04T05:53:58.067 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-04T05:53:58.068 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-04T05:53:58.068 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-04T05:53:58.068 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-04T05:53:58.068 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-04T05:53:58.068 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-04T05:53:58.068 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-04T05:53:58.068 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-04T05:53:58.068 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-04T05:53:58.068 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-04T05:53:58.068 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-04T05:53:58.068 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-04-04T05:53:58.068 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-04-04T05:53:58.068 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-04T05:53:58.069 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-04T05:53:58.069 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-04T05:53:58.069 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-04-04T05:53:58.069 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-04-04T05:53:58.069 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-04T05:53:58.069 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-04T05:53:58.069 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-04-04T05:53:58.069 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-04-04T05:53:58.069 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-04T05:53:58.069 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-04T05:53:58.069 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-04-04T05:53:58.069 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-04-04T05:53:58.069 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-04-04T05:53:58.069 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-04T05:53:58.069 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-04T05:53:58.070 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-04T05:53:58.070 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-04T05:53:58.070 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-04-04T05:53:58.070 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-04T05:53:58.070 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-04-04T05:53:58.070 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-04T05:53:58.070 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-04T05:53:58.070 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-04T05:53:58.070 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-04-04T05:53:58.070 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-04-04T05:53:58.070 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-04-04T05:53:58.070 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-04T05:53:58.070 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-04-04T05:53:58.071 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-04-04T05:53:58.071 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-04-04T05:53:58.071 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-04-04T05:53:58.071 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-04T05:53:58.071 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-04-04T05:53:58.071 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-04-04T05:53:58.071 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-04T05:53:58.071 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-04-04T05:53:58.071 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-04-04T05:53:58.071 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-04-04T05:53:58.071 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-04-04T05:53:58.071 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-04-04T05:53:58.071 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-04-04T05:53:58.071 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-04-04T05:53:58.072 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-04-04T05:53:58.072 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-04-04T05:53:58.072 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-04-04T05:53:58.072 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-04-04T05:53:58.072 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-04-04T05:53:58.072 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-04-04T05:53:58.072 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-04-04T05:53:58.072 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-04-04T05:53:58.072 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-04-04T05:53:58.072 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-04-04T05:53:58.072 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-04-04T05:53:58.072 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-04-04T05:53:58.072 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-04-04T05:53:58.072 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-04-04T05:53:58.073 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-04T05:53:58.073 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-04-04T05:53:58.073 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-04-04T05:53:58.073 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-04-04T05:53:58.073 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-04-04T05:53:58.073 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-04-04T05:53:58.073 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-04-04T05:53:58.073 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-04-04T05:53:58.073 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-04-04T05:53:58.073 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-04-04T05:53:58.073 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-04-04T05:53:58.073 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-04-04T05:53:58.073 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-04-04T05:53:58.074 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-04-04T05:53:58.074 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-04-04T05:53:58.074 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-04-04T05:53:58.074 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-04-04T05:53:58.074 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-04-04T05:53:58.074 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-04-04T05:53:58.074 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-04-04T05:53:58.074 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-04-04T05:53:58.074 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-04-04T05:53:58.074 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-04-04T05:53:58.074 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-04-04T05:53:58.074 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-04-04T05:53:58.074 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-04-04T05:53:58.074 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-04-04T05:53:58.075 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-04-04T05:53:58.075 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-04T05:53:58.075 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-04-04T05:53:58.075 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-04T05:53:58.075 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-04-04T05:53:58.075 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-04T05:53:58.075 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-04-04T05:53:58.075 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-04-04T05:53:58.075 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-04-04T05:53:58.075 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-04-04T05:53:58.075 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-04-04T05:53:58.075 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-04-04T05:53:58.075 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-04-04T05:53:58.075 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-04-04T05:53:58.075 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-04-04T05:53:58.076 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-04-04T05:53:58.076 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-04-04T05:53:58.076 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-04-04T05:53:58.076 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-04-04T05:53:58.076 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-04-04T05:53:58.076 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-04-04T05:53:58.076 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-04-04T05:53:58.076 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-04-04T05:53:58.076 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-04-04T05:53:58.076 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-04-04T05:53:58.076 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-04-04T05:53:58.076 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-04-04T05:53:58.076 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-04-04T05:53:58.076 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-04-04T05:53:58.077 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-04-04T05:53:58.077 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-04-04T05:53:58.077 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-04-04T05:53:58.077 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-04-04T05:53:58.077 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-04-04T05:53:58.077 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-04-04T05:53:58.077 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-04-04T05:53:58.077 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-04-04T05:53:58.077 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-04-04T05:53:58.077 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-04-04T05:53:58.077 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-04-04T05:53:58.077 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-04-04T05:53:58.077 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-04-04T05:53:58.078 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-04-04T05:53:58.078 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-04-04T05:53:58.078 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-04-04T05:53:58.078 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-04-04T05:53:58.078 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-04-04T05:53:58.078 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-04-04T05:53:58.078 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-04-04T05:53:58.078 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-04-04T05:53:58.078 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-04-04T05:53:58.078 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-04-04T05:53:58.078 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-04-04T05:53:58.078 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-04-04T05:53:58.078 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-04-04T05:53:58.078 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-04-04T05:53:58.078 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-04-04T05:53:58.079 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-04-04T05:53:58.079 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-04-04T05:53:58.079 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-04-04T05:53:58.079 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-04-04T05:53:58.079 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-04-04T05:53:58.079 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-04-04T05:53:58.079 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-04-04T05:53:58.079 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-04-04T05:53:58.079 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-04T05:53:58.079 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-04-04T05:53:58.079 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-04-04T05:53:58.079 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-04-04T05:53:58.079 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-04T05:53:58.079 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-04-04T05:53:58.080 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-04T05:53:58.080 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-04-04T05:53:58.080 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-04T05:53:58.080 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-04-04T05:53:58.080 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-04T05:53:58.080 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-04T05:53:58.080 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-04T05:53:58.080 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-04-04T05:53:58.080 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-04-04T05:53:58.080 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-04-04T05:53:58.080 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-04-04T05:53:58.080 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-04-04T05:53:58.080 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-04-04T05:53:58.081 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-04-04T05:53:58.081 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-04-04T05:53:58.081 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-04-04T05:53:58.081 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-04-04T05:53:58.081 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-04-04T05:53:58.081 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-04-04T05:53:58.081 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-04-04T05:53:58.081 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-04-04T05:53:58.081 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-04-04T05:53:58.081 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-04-04T05:53:58.081 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-04-04T05:53:58.081 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-04-04T05:53:58.081 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-04-04T05:53:58.081 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-04-04T05:53:58.082 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-04-04T05:53:58.082 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-04-04T05:53:58.082 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-04-04T05:53:58.082 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-04-04T05:53:58.082 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-04-04T05:53:58.082 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-04-04T05:53:58.082 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-04-04T05:53:58.082 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-04-04T05:53:58.082 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-04-04T05:53:58.082 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-04T05:53:58.082 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-04-04T05:53:58.082 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-04-04T05:53:58.082 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-04-04T05:53:58.083 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-04-04T05:53:58.083 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-04-04T05:53:58.083 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-04-04T05:53:58.083 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-04-04T05:53:58.083 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-04-04T05:53:58.083 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-04-04T05:53:58.083 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-04-04T05:53:58.083 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-04-04T05:53:58.083 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-04-04T05:53:58.083 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-04-04T05:53:58.083 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-04-04T05:53:58.083 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-04-04T05:53:58.083 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-04-04T05:53:58.083 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-04-04T05:53:58.083 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-04T05:53:58.084 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-04-04T05:53:58.084 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-04-04T05:53:58.084 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-04-04T05:53:58.084 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-04-04T05:53:58.084 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-04-04T05:53:58.084 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-04-04T05:53:58.084 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-04-04T05:53:58.084 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-04-04T05:53:58.084 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-04-04T05:53:58.084 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-04-04T05:53:58.084 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-04-04T05:53:58.084 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-04-04T05:53:58.084 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-04-04T05:53:58.084 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-04T05:53:58.085 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-04-04T05:53:58.085 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-04-04T05:53:58.085 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-04-04T05:53:58.085 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-04-04T05:53:58.085 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-04-04T05:53:58.085 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-04-04T05:53:58.085 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-04-04T05:53:58.085 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-04-04T05:53:58.085 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-04-04T05:53:58.085 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-04-04T05:53:58.085 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-04-04T05:53:58.085 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-04-04T05:53:58.085 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-04-04T05:53:58.085 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-04-04T05:53:58.086 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-04-04T05:53:58.086 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-04-04T05:53:58.086 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-04-04T05:53:58.086 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-04-04T05:53:58.086 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-04-04T05:53:58.086 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-04-04T05:53:58.086 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-04-04T05:53:58.086 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-04-04T05:53:58.086 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-04T05:53:58.086 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-04T05:53:58.086 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-04T05:53:58.086 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-04T05:53:58.086 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-04T05:53:58.086 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-04T05:53:58.087 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-04T05:53:58.087 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-04T05:53:58.087 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-04T05:53:58.087 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-04T05:53:58.087 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-04T05:53:58.087 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-04T05:53:58.087 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-04T05:53:58.087 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-04T05:53:58.087 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-04T05:53:58.087 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-04T05:53:58.087 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-04T05:53:58.087 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-04T05:53:58.087 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-04T05:53:58.087 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-04T05:53:58.088 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-04-04T05:53:58.088 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-04T05:53:58.088 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-04T05:53:58.088 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-04T05:53:58.088 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-04-04T05:53:58.088 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-04-04T05:53:58.088 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-04-04T05:53:58.088 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-04-04T05:53:58.088 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-04-04T05:53:58.088 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-04-04T05:53:58.088 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-04-04T05:53:58.088 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-04-04T05:53:58.088 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-04-04T05:53:58.088 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-04-04T05:53:58.088 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-04-04T05:53:58.089 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-04-04T05:53:58.089 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-04-04T05:53:58.089 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-04-04T05:53:58.089 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-04-04T05:53:58.089 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-04-04T05:53:58.089 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-04-04T05:53:58.089 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-04-04T05:53:58.089 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-04-04T05:53:58.089 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-04-04T05:53:58.089 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-04T05:53:58.089 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-04-04T05:53:58.089 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-04T05:53:58.089 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-04-04T05:53:58.089 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-04-04T05:53:58.090 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-04-04T05:53:58.090 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-04-04T05:53:58.090 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-04-04T05:53:58.090 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-04-04T05:53:58.090 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-04-04T05:53:58.090 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-04-04T05:53:58.090 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-04-04T05:53:58.090 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-04-04T05:53:58.090 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-04-04T05:53:58.090 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-04-04T05:53:58.090 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-04T05:53:58.090 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-04T05:53:58.090 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-04T05:53:58.091 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-04T05:53:58.091 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-04T05:53:58.091 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-04T05:53:58.091 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-04T05:53:58.091 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-04T05:53:58.091 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-04T05:53:58.091 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-04T05:53:58.091 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-04T05:53:58.091 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-04T05:53:58.091 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-04T05:53:58.091 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-04T05:53:58.091 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-04T05:53:58.091 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-04T05:53:58.091 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-04T05:53:58.092 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-04T05:53:58.092 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-04T05:53:58.092 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-04T05:53:58.092 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-04T05:53:58.092 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-04-04T05:53:58.092 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-04-04T05:53:58.092 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-04-04T05:53:58.092 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-04-04T05:53:58.092 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-04-04T05:53:58.092 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-04-04T05:53:58.092 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-04-04T05:53:58.092 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-04-04T05:53:58.092 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-04-04T05:53:58.092 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-04-04T05:53:58.093 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-04-04T05:53:58.093 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-04-04T05:53:58.093 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-04-04T05:53:58.093 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-04-04T05:53:58.093 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-04-04T05:53:58.093 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-04-04T05:53:58.093 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-04T05:53:58.093 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-04-04T05:53:58.093 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-04-04T05:53:58.093 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-04-04T05:53:58.093 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-04-04T05:53:58.093 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-04-04T05:53:58.093 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-04T05:53:58.093 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-04-04T05:53:58.094 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-04-04T05:53:58.094 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-04-04T05:53:58.094 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-04T05:53:58.094 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-04T05:53:58.094 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-04T05:53:58.094 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-04T05:53:58.094 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-04T05:53:58.094 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-04T05:53:58.094 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-04T05:53:58.094 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-04T05:53:58.094 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-04T05:53:58.094 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-04T05:53:58.094 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-04T05:53:58.094 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-04T05:53:58.095 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-04T05:53:58.095 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-04T05:53:58.095 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-04T05:53:58.095 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-04T05:53:58.095 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-04T05:53:58.095 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-04T05:53:58.095 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-04T05:53:58.095 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-04T05:53:58.095 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-04T05:53:58.095 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-04T05:53:58.095 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-04T05:53:58.095 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-04T05:53:58.095 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-04-04T05:53:58.095 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-04-04T05:53:58.096 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-04T05:53:58.096 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-04-04T05:53:58.096 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-04-04T05:53:58.096 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-04-04T05:53:58.096 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-04-04T05:53:58.096 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-04T05:53:58.096 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-04-04T05:53:58.096 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-04-04T05:53:58.096 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-04-04T05:53:58.096 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-04-04T05:53:58.096 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-04-04T05:53:58.096 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-04-04T05:53:58.096 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-04-04T05:53:58.096 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-04-04T05:53:58.097 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-04-04T05:53:58.097 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-04-04T05:53:58.097 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-04-04T05:53:58.097 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-04-04T05:53:58.097 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-04-04T05:53:58.097 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-04-04T05:53:58.097 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-04T05:53:58.097 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-04T05:53:58.097 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-04T05:53:58.097 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-04T05:53:58.097 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-04T05:53:58.097 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-04T05:53:58.097 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-04T05:53:58.097 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-04T05:53:58.097 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-04T05:53:58.098 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-04T05:53:58.098 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-04T05:53:58.098 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-04T05:53:58.098 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-04T05:53:58.098 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-04T05:53:58.098 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-04T05:53:58.098 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-04-04T05:53:58.098 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-04T05:53:58.098 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-04T05:53:58.098 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-04T05:53:58.098 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-04T05:53:58.098 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-04T05:53:58.098 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-04T05:53:58.099 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-04T05:53:58.099 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-04T05:53:58.099 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-04T05:53:58.099 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-04T05:53:58.099 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-04T05:53:58.099 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-04T05:53:58.099 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-04T05:53:58.099 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-04T05:53:58.099 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-04-04T05:53:58.099 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-04-04T05:53:58.099 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-04T05:53:58.099 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-04T05:53:58.099 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-04T05:53:58.099 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-04T05:53:58.100 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-04T05:53:58.100 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-04T05:53:58.100 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-04T05:53:58.100 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-04T05:53:58.100 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-04T05:53:58.100 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-04T05:53:58.100 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-04T05:53:58.100 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-04T05:53:58.100 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-04T05:53:58.100 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-04-04T05:53:58.100 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-04T05:53:58.100 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-04-04T05:53:58.100 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-04-04T05:53:58.100 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-04T05:53:58.100 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-04T05:53:58.101 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-04-04T05:53:58.101 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-04-04T05:53:58.101 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-04T05:53:58.101 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-04T05:53:58.101 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-04-04T05:53:58.101 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-04-04T05:53:58.101 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-04T05:53:58.101 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-04-04T05:53:58.101 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-04T05:53:58.101 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-04T05:53:58.101 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-04-04T05:53:58.101 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-04-04T05:53:58.101 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-04-04T05:53:58.101 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-04T05:53:58.101 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-04-04T05:53:58.102 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-04T05:53:58.102 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-04-04T05:53:58.102 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-04-04T05:53:58.102 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-04-04T05:53:58.102 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-04-04T05:53:58.102 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-04-04T05:53:58.102 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-04-04T05:53:58.102 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-04-04T05:53:58.102 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-04-04T05:53:58.102 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-04-04T05:53:58.102 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-04-04T05:53:58.102 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-04-04T05:53:58.102 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-04-04T05:53:58.103 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-04-04T05:53:58.103 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-04-04T05:53:58.103 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-04T05:53:58.103 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-04T05:53:58.103 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-04T05:53:58.103 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-04-04T05:53:58.103 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-04-04T05:53:58.103 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-04-04T05:53:58.103 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-04-04T05:53:58.103 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-04T05:53:58.103 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-04-04T05:53:58.103 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-04-04T05:53:58.103 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-04T05:53:58.103 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-04-04T05:53:58.104 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-04-04T05:53:58.104 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-04T05:53:58.104 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-04-04T05:53:58.104 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-04T05:53:58.104 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-04-04T05:53:58.104 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-04T05:53:58.104 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-04T05:53:58.104 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-04-04T05:53:58.104 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-04-04T05:53:58.104 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-04-04T05:53:58.104 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-04-04T05:53:58.104 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-04-04T05:53:58.104 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-04-04T05:53:58.104 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-04-04T05:53:58.104 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-04-04T05:53:58.105 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-04-04T05:53:58.105 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-04-04T05:53:58.105 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-04-04T05:53:58.105 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-04-04T05:53:58.105 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-04-04T05:53:58.105 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-04-04T05:53:58.105 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-04-04T05:53:58.105 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-04-04T05:53:58.105 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-04-04T05:53:58.105 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-04-04T05:53:58.105 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-04-04T05:53:58.105 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-04-04T05:53:58.105 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-04-04T05:53:58.105 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-04-04T05:53:58.105 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-04-04T05:53:58.106 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-04-04T05:53:58.106 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-04-04T05:53:58.106 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-04-04T05:53:58.106 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-04-04T05:53:58.106 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-04-04T05:53:58.106 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-04-04T05:53:58.106 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-04-04T05:53:58.106 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-04-04T05:53:58.106 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-04-04T05:53:58.106 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-04-04T05:53:58.106 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-04-04T05:53:58.106 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-04-04T05:53:58.106 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-04-04T05:53:58.106 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-04-04T05:53:58.107 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-04-04T05:53:58.107 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-04-04T05:53:58.107 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-04-04T05:53:58.107 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-04-04T05:53:58.107 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-04-04T05:53:58.107 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-04-04T05:53:58.107 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-04-04T05:53:58.107 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-04-04T05:53:58.107 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-04-04T05:53:58.107 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-04-04T05:53:58.107 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-04-04T05:53:58.107 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-04-04T05:53:58.107 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-04-04T05:53:58.107 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-04-04T05:53:58.108 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-04-04T05:53:58.108 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-04-04T05:53:58.108 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-04-04T05:53:58.108 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-04-04T05:53:58.108 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-04-04T05:53:58.108 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-04-04T05:53:58.108 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-04-04T05:53:58.108 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-04-04T05:53:58.108 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-04-04T05:53:58.108 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-04-04T05:53:58.108 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-04-04T05:53:58.108 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-04-04T05:53:58.108 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-04-04T05:53:58.108 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-04-04T05:53:58.109 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-04-04T05:53:58.109 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-04-04T05:53:58.109 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-04-04T05:53:58.109 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-04-04T05:53:58.109 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-04-04T05:53:58.109 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-04-04T05:53:58.109 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-04-04T05:53:58.109 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-04-04T05:53:58.109 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-04-04T05:53:58.109 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-04-04T05:53:58.109 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-04-04T05:53:58.109 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-04-04T05:53:58.109 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-04-04T05:53:58.109 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-04-04T05:53:58.109 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-04-04T05:53:58.110 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-04-04T05:53:58.110 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-04-04T05:53:58.110 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-04-04T05:53:58.110 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-04-04T05:53:58.110 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-04-04T05:53:58.110 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-04-04T05:53:58.110 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-04-04T05:53:58.110 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-04-04T05:53:58.110 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-04-04T05:53:58.110 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-04-04T05:53:58.110 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-04-04T05:53:58.110 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-04-04T05:53:58.110 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-04-04T05:53:58.111 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-04-04T05:53:58.111 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-04-04T05:53:58.111 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-04-04T05:53:58.111 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-04-04T05:53:58.111 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-04-04T05:53:58.111 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-04-04T05:53:58.111 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-04-04T05:53:58.111 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-04-04T05:53:58.111 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-04-04T05:53:58.111 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-04-04T05:53:58.111 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-04-04T05:53:58.111 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-04-04T05:53:58.111 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-04-04T05:53:58.111 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-04-04T05:53:58.112 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-04-04T05:53:58.112 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-04-04T05:53:58.112 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-04-04T05:53:58.112 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-04-04T05:53:58.112 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-04-04T05:53:58.112 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-04-04T05:53:58.112 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-04-04T05:53:58.112 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-04-04T05:53:58.112 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-04T05:53:58.112 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-04T05:53:58.112 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-04-04T05:53:58.112 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-04-04T05:53:58.112 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-04T05:53:58.112 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-04-04T05:53:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-04-04T05:53:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-04-04T05:53:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-04-04T05:53:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-04-04T05:53:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-04-04T05:53:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-04T05:53:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-04T05:53:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-04-04T05:53:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-04T05:53:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-04T05:53:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-04-04T05:53:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-04T05:53:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-04-04T05:53:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-04-04T05:53:58.113 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-04-04T05:53:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-04T05:53:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-04-04T05:53:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-04-04T05:53:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-04-04T05:53:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-04-04T05:53:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-04-04T05:53:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-04-04T05:53:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-04-04T05:53:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-04-04T05:53:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-04-04T05:53:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-04-04T05:53:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-04-04T05:53:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-04-04T05:53:58.114 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-04-04T05:53:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-04-04T05:53:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-04-04T05:53:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-04-04T05:53:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-04-04T05:53:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-04-04T05:53:58.115 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-04-04T05:53:58.118 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-04-04T05:53:58.119 DEBUG:teuthology.orchestra.run.smithi178:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-04T05:53:58.142 INFO:teuthology.orchestra.run.smithi178.stdout:postinstall scriptlet (using /bin/sh): 2024-04-04T05:53:58.142 INFO:teuthology.orchestra.run.smithi178.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-04T05:53:58.142 INFO:teuthology.orchestra.run.smithi178.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-04-04T05:53:58.142 INFO:teuthology.orchestra.run.smithi178.stdout:fi 2024-04-04T05:53:58.143 INFO:teuthology.orchestra.run.smithi178.stdout:for file in vmlinuz System.map config; do 2024-04-04T05:53:58.143 INFO:teuthology.orchestra.run.smithi178.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-04-04T05:53:58.143 INFO:teuthology.orchestra.run.smithi178.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-04-04T05:53:58.143 INFO:teuthology.orchestra.run.smithi178.stdout: fi 2024-04-04T05:53:58.143 INFO:teuthology.orchestra.run.smithi178.stdout:done 2024-04-04T05:53:58.143 INFO:teuthology.orchestra.run.smithi178.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-04T05:53:58.143 INFO:teuthology.orchestra.run.smithi178.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-04-04T05:53:58.143 INFO:teuthology.orchestra.run.smithi178.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-04-04T05:53:58.143 INFO:teuthology.orchestra.run.smithi178.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-04-04T05:53:58.143 INFO:teuthology.orchestra.run.smithi178.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-04-04T05:53:58.143 INFO:teuthology.orchestra.run.smithi178.stdout:fi 2024-04-04T05:53:58.143 INFO:teuthology.orchestra.run.smithi178.stdout:postuninstall scriptlet (using /bin/sh): 2024-04-04T05:53:58.143 INFO:teuthology.orchestra.run.smithi178.stdout:if [ -x /sbin/update-bootloader ]; then 2024-04-04T05:53:58.143 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-04-04T05:53:58.144 INFO:teuthology.orchestra.run.smithi178.stdout:fi 2024-04-04T05:53:58.144 DEBUG:teuthology.orchestra.run.smithi178:> sudo rpm -qi grub2-tools 2024-04-04T05:53:58.225 INFO:teuthology.orchestra.run.smithi178.stdout:Name : grub2-tools 2024-04-04T05:53:58.225 INFO:teuthology.orchestra.run.smithi178.stdout:Epoch : 1 2024-04-04T05:53:58.226 INFO:teuthology.orchestra.run.smithi178.stdout:Version : 2.06 2024-04-04T05:53:58.226 INFO:teuthology.orchestra.run.smithi178.stdout:Release : 68.el9 2024-04-04T05:53:58.226 INFO:teuthology.orchestra.run.smithi178.stdout:Architecture: x86_64 2024-04-04T05:53:58.226 INFO:teuthology.orchestra.run.smithi178.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-04T05:53:58.226 INFO:teuthology.orchestra.run.smithi178.stdout:Group : Unspecified 2024-04-04T05:53:58.226 INFO:teuthology.orchestra.run.smithi178.stdout:Size : 8264254 2024-04-04T05:53:58.226 INFO:teuthology.orchestra.run.smithi178.stdout:License : GPLv3+ 2024-04-04T05:53:58.226 INFO:teuthology.orchestra.run.smithi178.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-04T05:53:58.226 INFO:teuthology.orchestra.run.smithi178.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-04T05:53:58.226 INFO:teuthology.orchestra.run.smithi178.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-04T05:53:58.226 INFO:teuthology.orchestra.run.smithi178.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-04T05:53:58.226 INFO:teuthology.orchestra.run.smithi178.stdout:Packager : builder@centos.org 2024-04-04T05:53:58.226 INFO:teuthology.orchestra.run.smithi178.stdout:Vendor : CentOS 2024-04-04T05:53:58.227 INFO:teuthology.orchestra.run.smithi178.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-04T05:53:58.227 INFO:teuthology.orchestra.run.smithi178.stdout:Summary : Support tools for GRUB. 2024-04-04T05:53:58.227 INFO:teuthology.orchestra.run.smithi178.stdout:Description : 2024-04-04T05:53:58.227 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T05:53:58.227 INFO:teuthology.orchestra.run.smithi178.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-04T05:53:58.227 INFO:teuthology.orchestra.run.smithi178.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-04T05:53:58.227 INFO:teuthology.orchestra.run.smithi178.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-04T05:53:58.227 INFO:teuthology.orchestra.run.smithi178.stdout:hardware devices. 2024-04-04T05:53:58.227 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T05:53:58.227 INFO:teuthology.orchestra.run.smithi178.stdout:This subpackage provides tools for support of all platforms. 2024-04-04T05:53:58.228 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-04T05:53:58.228 INFO:teuthology.task.kernel:Updating grub on smithi178 to boot 6.8.0-g1df0c7e8a235 2024-04-04T05:53:58.228 DEBUG:teuthology.orchestra.run.smithi178:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-04T05:53:58.797 INFO:teuthology.orchestra.run.smithi178.stderr:Generating grub configuration file ... 2024-04-04T05:54:00.226 INFO:teuthology.orchestra.run.smithi178.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-04T05:54:00.254 INFO:teuthology.orchestra.run.smithi178.stderr:done 2024-04-04T05:54:00.257 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-04-04T05:54:00.272 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.jSL59XHaxb 2024-04-04T05:54:00.272 DEBUG:teuthology.orchestra.run.smithi178:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.jSL59XHaxb 2024-04-04T05:54:00.346 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.jSL59XHaxb 2024-04-04T05:54:00.510 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.jSL59XHaxb is 6KB 2024-04-04T05:54:00.531 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.jSL59XHaxb 2024-04-04T05:54:00.545 DEBUG:teuthology.orchestra.run.smithi178:> sudo /bin/ls /boot/loader/entries || true 2024-04-04T05:54:00.611 INFO:teuthology.orchestra.run.smithi178.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-04T05:54:00.611 INFO:teuthology.orchestra.run.smithi178.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-04T05:54:00.611 INFO:teuthology.orchestra.run.smithi178.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-04T05:54:00.612 INFO:teuthology.orchestra.run.smithi178.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-04-04T05:54:00.613 DEBUG:teuthology.orchestra.run.smithi178:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-04-04T05:54:00.759 DEBUG:teuthology.orchestra.run.smithi178:> sudo shutdown -r now 2024-04-04T05:54:14.754 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-04T05:54:14.755 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-04-04T05:54:14.756 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-04T05:54:30.774 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-04T05:54:30.775 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-04-04T05:54:30.775 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-04T05:54:33.122 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-04-04T05:54:42.130 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-04-04T05:54:42.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-04T05:54:45.182 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-04-04T05:54:49.246 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-04-04T05:54:57.190 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-04-04T05:54:57.191 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-04T05:54:58.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-04-04T05:54:58.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-04T05:55:01.310 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-04-04T05:55:13.322 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-04-04T05:55:13.323 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-04T05:55:16.382 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-04-04T05:55:31.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-04-04T05:55:31.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-04T05:55:57.219 DEBUG:teuthology.orchestra.remote:timed out 2024-04-04T05:56:03.433 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-04-04T05:56:04.071 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-04-04T05:56:04.071 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.8.0-g1df0c7e8a235"... 2024-04-04T05:56:04.071 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2024-04-04T05:56:04.112 INFO:teuthology.orchestra.run.smithi178.stdout:6.8.0-g1df0c7e8a235 2024-04-04T05:56:04.112 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-04-04T05:56:04.112 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-04T05:56:04.113 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-04T05:56:05.114 DEBUG:teuthology.parallel:result is None 2024-04-04T05:56:12.220 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-04-04T05:56:12.221 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-04T05:56:12.769 DEBUG:teuthology.orchestra.run.smithi159:> true 2024-04-04T05:56:13.453 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-04-04T05:56:13.454 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-g1df0c7e8a235"... 2024-04-04T05:56:13.454 DEBUG:teuthology.orchestra.run.smithi159:> uname -r 2024-04-04T05:56:13.505 INFO:teuthology.orchestra.run.smithi159.stdout:6.8.0-g1df0c7e8a235 2024-04-04T05:56:13.506 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-04-04T05:56:13.506 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-04T05:56:13.506 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-04T05:56:14.507 DEBUG:teuthology.parallel:result is None 2024-04-04T05:56:14.507 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-04T05:56:14.523 INFO:teuthology.task.internal:Creating test directory... 2024-04-04T05:56:14.523 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-04T05:56:14.531 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-04T05:56:14.607 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-04T05:56:14.613 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-04T05:56:14.622 INFO:teuthology.task.internal:Creating archive directory... 2024-04-04T05:56:14.623 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-04T05:56:14.630 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-04T05:56:14.725 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-04T05:56:14.731 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-04T05:56:14.731 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-04T05:56:14.738 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-04T05:56:14.878 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-04T05:56:14.941 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-04T05:56:14.943 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-04T05:56:15.042 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-04T05:56:15.048 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-04T05:56:15.056 INFO:teuthology.task.internal:Configuring sudo... 2024-04-04T05:56:15.056 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-04T05:56:15.063 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-04T05:56:15.202 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-04T05:56:15.211 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-04T05:56:15.211 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-04T05:56:15.218 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-04T05:56:15.294 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-04T05:56:15.373 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-04T05:56:15.474 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-04T05:56:15.555 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-04T05:56:15.654 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T05:56:15.654 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-04T05:56:15.747 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-04T05:56:15.854 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-04T05:56:15.995 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-04T05:56:16.093 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-04T05:56:16.229 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T05:56:16.229 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-04T05:56:16.352 DEBUG:teuthology.orchestra.run.smithi159:> sudo service rsyslog restart 2024-04-04T05:56:16.359 DEBUG:teuthology.orchestra.run.smithi178:> sudo service rsyslog restart 2024-04-04T05:56:16.475 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-04T05:56:16.512 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-04T05:56:16.961 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-04T05:56:16.969 INFO:teuthology.task.internal:Starting timer... 2024-04-04T05:56:16.969 INFO:teuthology.run_tasks:Running task pcp... 2024-04-04T05:56:16.978 INFO:teuthology.run_tasks:Running task selinux... 2024-04-04T05:56:16.986 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-04T05:56:16.987 DEBUG:teuthology.orchestra.run.smithi159:> sudo service auditd rotate 2024-04-04T05:56:17.096 INFO:teuthology.orchestra.run.smithi159.stdout:Rotating logs: 2024-04-04T05:56:17.100 DEBUG:teuthology.orchestra.run.smithi178:> sudo service auditd rotate 2024-04-04T05:56:17.230 INFO:teuthology.orchestra.run.smithi178.stdout:Rotating logs: 2024-04-04T05:56:17.234 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-04T05:56:17.234 DEBUG:teuthology.orchestra.run.smithi159:> /usr/sbin/getenforce 2024-04-04T05:56:17.290 INFO:teuthology.orchestra.run.smithi159.stdout:Permissive 2024-04-04T05:56:17.291 DEBUG:teuthology.orchestra.run.smithi178:> /usr/sbin/getenforce 2024-04-04T05:56:17.349 INFO:teuthology.orchestra.run.smithi178.stdout:Permissive 2024-04-04T05:56:17.350 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi159.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi178.front.sepia.ceph.com': 'permissive'} 2024-04-04T05:56:17.350 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-04T05:56:17.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-04T05:56:17.431 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-04T05:56:17.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-04T05:56:17.511 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-04T05:56:17.511 DEBUG:teuthology.orchestra.run.smithi159:> sudo /usr/sbin/setenforce permissive 2024-04-04T05:56:17.595 DEBUG:teuthology.orchestra.run.smithi178:> sudo /usr/sbin/setenforce permissive 2024-04-04T05:56:17.687 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-04T05:56:17.696 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-04T05:56:17.734 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-04T05:56:17.735 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi159.front.sepia.ceph.com,smithi178.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-04T06:02:20.154 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi159.front.sepia.ceph.com'), Remote(name='ubuntu@smithi178.front.sepia.ceph.com')] 2024-04-04T06:02:20.155 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-04-04T06:02:20.157 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-04T06:02:20.296 DEBUG:teuthology.orchestra.run.smithi159:> true 2024-04-04T06:02:20.446 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-04-04T06:02:20.446 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-04-04T06:02:20.447 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-04T06:02:20.594 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-04-04T06:02:20.752 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-04-04T06:02:20.752 INFO:teuthology.run_tasks:Running task clock... 2024-04-04T06:02:20.765 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-04T06:02:20.765 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-04T06:02:20.766 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-04T06:02:20.773 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-04T06:02:20.773 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-04T06:02:20.864 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-04T06:02:20.867 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-04T06:02:20.912 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-04T06:02:20.918 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-04T06:02:21.005 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: ntpd: command not found 2024-04-04T06:02:21.009 INFO:teuthology.orchestra.run.smithi159.stderr:sudo: ntpd: command not found 2024-04-04T06:02:21.048 INFO:teuthology.orchestra.run.smithi178.stdout:506 Cannot talk to daemon 2024-04-04T06:02:21.050 INFO:teuthology.orchestra.run.smithi159.stdout:506 Cannot talk to daemon 2024-04-04T06:02:21.100 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-04T06:02:21.102 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-04T06:02:21.151 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-04T06:02:21.155 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-04T06:02:21.276 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-04-04T06:02:21.283 INFO:teuthology.orchestra.run.smithi159.stderr:bash: line 1: ntpq: command not found 2024-04-04T06:02:21.284 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-04T06:02:21.284 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-04-04T06:02:21.284 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-04T06:02:21.284 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-04T06:02:21.284 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-04T06:02:21.284 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-04T06:02:21.292 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-04T06:02:21.292 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2024-04-04T06:02:21.292 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-04T06:02:21.292 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-04T06:02:21.292 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-04T06:02:21.293 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-04T06:02:21.294 INFO:teuthology.run_tasks:Running task install... 2024-04-04T06:02:21.302 DEBUG:teuthology.task.install:project ceph 2024-04-04T06:02:21.303 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'addbc488dc763e752c640619b677f10e0da02c9f'}} 2024-04-04T06:02:21.303 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'addbc488dc763e752c640619b677f10e0da02c9f'} 2024-04-04T06:02:21.303 INFO:teuthology.task.install:Using flavor: default 2024-04-04T06:02:21.309 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-04T06:02:21.310 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-04T06:02:21.310 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'addbc488dc763e752c640619b677f10e0da02c9f', 'tag': None, 'wait_for_package': False} 2024-04-04T06:02:21.311 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=addbc488dc763e752c640619b677f10e0da02c9f 2024-04-04T06:02:21.313 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'addbc488dc763e752c640619b677f10e0da02c9f', 'tag': None, 'wait_for_package': False} 2024-04-04T06:02:21.313 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=addbc488dc763e752c640619b677f10e0da02c9f 2024-04-04T06:02:21.458 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240330.172700/addbc488dc763e752c640619b677f10e0da02c9f/centos/9/flavors/default/ 2024-04-04T06:02:21.458 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2571.gaddbc488 2024-04-04T06:02:21.544 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240330.172700/addbc488dc763e752c640619b677f10e0da02c9f/centos/9/flavors/default/ 2024-04-04T06:02:21.544 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2571.gaddbc488 2024-04-04T06:02:21.694 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240330.172700/addbc488dc763e752c640619b677f10e0da02c9f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240330.172700/addbc488dc763e752c640619b677f10e0da02c9f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240330.172700/addbc488dc763e752c640619b677f10e0da02c9f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-04T06:02:21.694 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:02:21.694 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-04T06:02:21.734 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240330.172700/addbc488dc763e752c640619b677f10e0da02c9f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240330.172700/addbc488dc763e752c640619b677f10e0da02c9f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240330.172700/addbc488dc763e752c640619b677f10e0da02c9f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-04T06:02:21.734 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:02:21.734 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-04T06:02:21.796 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-04T06:02:21.796 DEBUG:teuthology.orchestra.run.smithi159:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/addbc488dc763e752c640619b677f10e0da02c9f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-04T06:02:21.834 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-04T06:02:21.835 DEBUG:teuthology.orchestra.run.smithi178:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/addbc488dc763e752c640619b677f10e0da02c9f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-04T06:02:21.933 DEBUG:teuthology.orchestra.run.smithi159:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-04T06:02:21.972 DEBUG:teuthology.orchestra.run.smithi178:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-04T06:02:22.065 DEBUG:teuthology.orchestra.run.smithi159:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-04T06:02:22.100 DEBUG:teuthology.orchestra.run.smithi178:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-04T06:02:22.157 INFO:teuthology.orchestra.run.smithi159.stdout:check_obsoletes = 1 2024-04-04T06:02:22.162 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum clean all 2024-04-04T06:02:22.189 INFO:teuthology.orchestra.run.smithi178.stdout:check_obsoletes = 1 2024-04-04T06:02:22.194 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum clean all 2024-04-04T06:02:22.513 INFO:teuthology.orchestra.run.smithi159.stdout:74 files removed 2024-04-04T06:02:22.550 INFO:teuthology.orchestra.run.smithi178.stdout:74 files removed 2024-04-04T06:02:22.551 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-04T06:02:22.591 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-04T06:02:23.977 INFO:teuthology.orchestra.run.smithi159.stdout:ceph packages for x86_64 90 kB/s | 83 kB 00:00 2024-04-04T06:02:24.100 INFO:teuthology.orchestra.run.smithi178.stdout:ceph packages for x86_64 83 kB/s | 83 kB 00:01 2024-04-04T06:02:24.511 INFO:teuthology.orchestra.run.smithi159.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-04-04T06:02:24.651 INFO:teuthology.orchestra.run.smithi178.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-04-04T06:02:25.037 INFO:teuthology.orchestra.run.smithi159.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-04-04T06:02:25.268 INFO:teuthology.orchestra.run.smithi178.stdout:ceph source packages 3.5 kB/s | 1.9 kB 00:00 2024-04-04T06:02:25.631 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-04-04T06:02:25.989 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-04-04T06:02:27.508 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-9 - BaseOS 3.4 MB/s | 2.0 MB 00:00 2024-04-04T06:02:27.818 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.0 MB 00:00 2024-04-04T06:02:28.784 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - BaseOS 9.3 MB/s | 8.0 MB 00:00 2024-04-04T06:02:31.893 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - BaseOS 2.2 MB/s | 8.0 MB 00:03 2024-04-04T06:02:33.205 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 19 MB 00:03 2024-04-04T06:02:34.682 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-04-04T06:02:38.970 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.1 MB 00:00 2024-04-04T06:02:39.030 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.1 MB 00:01 2024-04-04T06:02:40.546 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 16 kB 00:00 2024-04-04T06:02:40.958 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 16 kB 00:00 2024-04-04T06:02:43.446 INFO:teuthology.orchestra.run.smithi159.stdout:Extra Packages for Enterprise Linux 9.0 MB/s | 21 MB 00:02 2024-04-04T06:02:44.355 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 5.7 MB/s | 21 MB 00:03 2024-04-04T06:02:49.391 INFO:teuthology.orchestra.run.smithi159.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-04-04T06:02:50.096 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-04-04T06:02:52.153 INFO:teuthology.orchestra.run.smithi159.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-04T06:02:52.155 INFO:teuthology.orchestra.run.smithi159.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-04T06:02:52.174 INFO:teuthology.orchestra.run.smithi159.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-04T06:02:52.175 INFO:teuthology.orchestra.run.smithi159.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-04T06:02:52.177 INFO:teuthology.orchestra.run.smithi159.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-04T06:02:52.178 INFO:teuthology.orchestra.run.smithi159.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-04T06:02:52.181 INFO:teuthology.orchestra.run.smithi159.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-04T06:02:52.185 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-04T06:02:52.186 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-04T06:02:52.274 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-04-04T06:02:52.292 INFO:teuthology.orchestra.run.smithi159.stdout:============================================================================================ 2024-04-04T06:02:52.292 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2024-04-04T06:02:52.292 INFO:teuthology.orchestra.run.smithi159.stdout:============================================================================================ 2024-04-04T06:02:52.292 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-04-04T06:02:52.292 INFO:teuthology.orchestra.run.smithi159.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-04T06:02:52.293 INFO:teuthology.orchestra.run.smithi159.stdout: ceph x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 6.6 k 2024-04-04T06:02:52.293 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 5.4 M 2024-04-04T06:02:52.293 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 890 k 2024-04-04T06:02:52.293 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 145 k 2024-04-04T06:02:52.293 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 1.5 M 2024-04-04T06:02:52.293 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2571.gaddbc488.el9 ceph-noarch 141 k 2024-04-04T06:02:52.293 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2571.gaddbc488.el9 ceph-noarch 3.5 M 2024-04-04T06:02:52.293 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2571.gaddbc488.el9 ceph-noarch 7.4 M 2024-04-04T06:02:52.293 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook noarch 2:19.0.0-2571.gaddbc488.el9 ceph-noarch 49 k 2024-04-04T06:02:52.293 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 9.8 M 2024-04-04T06:02:52.293 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 48 M 2024-04-04T06:02:52.293 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume noarch 2:19.0.0-2571.gaddbc488.el9 ceph-noarch 264 k 2024-04-04T06:02:52.293 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm noarch 2:19.0.0-2571.gaddbc488.el9 ceph-noarch 762 k 2024-04-04T06:02:52.294 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 225 k 2024-04-04T06:02:52.294 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-shell x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 27 k 2024-04-04T06:02:52.294 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top noarch 2:19.0.0-2571.gaddbc488.el9 ceph-noarch 18 k 2024-04-04T06:02:52.294 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-04T06:02:52.294 INFO:teuthology.orchestra.run.smithi159.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-04T06:02:52.294 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-04T06:02:52.294 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 32 k 2024-04-04T06:02:52.294 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2 x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 746 k 2024-04-04T06:02:52.294 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 127 k 2024-04-04T06:02:52.294 INFO:teuthology.orchestra.run.smithi159.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-04T06:02:52.294 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-04T06:02:52.294 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-04T06:02:52.295 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-04T06:02:52.295 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-04T06:02:52.295 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-04T06:02:52.295 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 161 k 2024-04-04T06:02:52.295 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 320 k 2024-04-04T06:02:52.295 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 298 k 2024-04-04T06:02:52.295 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 99 k 2024-04-04T06:02:52.295 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 85 k 2024-04-04T06:02:52.295 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 3.1 M 2024-04-04T06:02:52.295 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 170 k 2024-04-04T06:02:52.295 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-04T06:02:52.295 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrading: 2024-04-04T06:02:52.295 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-04-04T06:02:52.296 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-04-04T06:02:52.296 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-04-04T06:02:52.296 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-04-04T06:02:52.296 INFO:teuthology.orchestra.run.smithi159.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-04T06:02:52.296 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-04T06:02:52.296 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-04T06:02:52.296 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-04T06:02:52.296 INFO:teuthology.orchestra.run.smithi159.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-04T06:02:52.296 INFO:teuthology.orchestra.run.smithi159.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-04T06:02:52.296 INFO:teuthology.orchestra.run.smithi159.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-04T06:02:52.296 INFO:teuthology.orchestra.run.smithi159.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-04T06:02:52.296 INFO:teuthology.orchestra.run.smithi159.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-04T06:02:52.296 INFO:teuthology.orchestra.run.smithi159.stdout: librados2 x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 3.4 M 2024-04-04T06:02:52.296 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1 x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 3.1 M 2024-04-04T06:02:52.297 INFO:teuthology.orchestra.run.smithi159.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-04T06:02:52.297 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-04T06:02:52.297 INFO:teuthology.orchestra.run.smithi159.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-04T06:02:52.297 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-04T06:02:52.297 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-04T06:02:52.297 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-04T06:02:52.297 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-04T06:02:52.297 INFO:teuthology.orchestra.run.smithi159.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-04T06:02:52.297 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-04T06:02:52.297 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2024-04-04T06:02:52.297 INFO:teuthology.orchestra.run.smithi159.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-04T06:02:52.297 INFO:teuthology.orchestra.run.smithi159.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-04T06:02:52.297 INFO:teuthology.orchestra.run.smithi159.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-04T06:02:52.298 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 21 M 2024-04-04T06:02:52.298 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2571.gaddbc488.el9 ceph-noarch 34 k 2024-04-04T06:02:52.298 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 2.4 M 2024-04-04T06:02:52.298 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2571.gaddbc488.el9 ceph-noarch 249 k 2024-04-04T06:02:52.298 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 4.7 M 2024-04-04T06:02:52.298 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 17 M 2024-04-04T06:02:52.298 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2571.gaddbc488.el9 ceph-noarch 16 k 2024-04-04T06:02:52.298 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 25 k 2024-04-04T06:02:52.298 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-04T06:02:52.298 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-04T06:02:52.298 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-04T06:02:52.298 INFO:teuthology.orchestra.run.smithi159.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-04T06:02:52.298 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-04T06:02:52.299 INFO:teuthology.orchestra.run.smithi159.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-04T06:02:52.299 INFO:teuthology.orchestra.run.smithi159.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-04T06:02:52.299 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-04T06:02:52.299 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-04T06:02:52.299 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 163 k 2024-04-04T06:02:52.299 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-04T06:02:52.299 INFO:teuthology.orchestra.run.smithi159.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-04T06:02:52.299 INFO:teuthology.orchestra.run.smithi159.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-04T06:02:52.299 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-04T06:02:52.299 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-04T06:02:52.299 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1 x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 500 k 2024-04-04T06:02:52.299 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-04T06:02:52.299 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2 x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 5.1 M 2024-04-04T06:02:52.299 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-04T06:02:52.300 INFO:teuthology.orchestra.run.smithi159.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-04T06:02:52.300 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-04T06:02:52.300 INFO:teuthology.orchestra.run.smithi159.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-04T06:02:52.300 INFO:teuthology.orchestra.run.smithi159.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-04T06:02:52.300 INFO:teuthology.orchestra.run.smithi159.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-04T06:02:52.300 INFO:teuthology.orchestra.run.smithi159.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-04T06:02:52.300 INFO:teuthology.orchestra.run.smithi159.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-04T06:02:52.300 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-04T06:02:52.300 INFO:teuthology.orchestra.run.smithi159.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-04T06:02:52.300 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-04T06:02:52.300 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-04T06:02:52.300 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-04T06:02:52.300 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-04T06:02:52.301 INFO:teuthology.orchestra.run.smithi159.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-04T06:02:52.301 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-04T06:02:52.301 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-04T06:02:52.301 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-04T06:02:52.301 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 45 k 2024-04-04T06:02:52.301 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 134 k 2024-04-04T06:02:52.301 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-04T06:02:52.301 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-04T06:02:52.301 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-04T06:02:52.301 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-04T06:02:52.301 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-04T06:02:52.301 INFO:teuthology.orchestra.run.smithi159.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-04T06:02:52.301 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-04T06:02:52.302 INFO:teuthology.orchestra.run.smithi159.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-04T06:02:52.302 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-04T06:02:52.302 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-04T06:02:52.302 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-04T06:02:52.302 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-04T06:02:52.302 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-04T06:02:52.302 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-04T06:02:52.302 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-04T06:02:52.302 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-04T06:02:52.302 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-04T06:02:52.302 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-04T06:02:52.302 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-04T06:02:52.302 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-04T06:02:52.302 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-04T06:02:52.303 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-04T06:02:52.303 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-04T06:02:52.303 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-04T06:02:52.303 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-04T06:02:52.303 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-04T06:02:52.303 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-04T06:02:52.303 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-04T06:02:52.303 INFO:teuthology.orchestra.run.smithi159.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-04T06:02:52.303 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-04T06:02:52.303 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-04T06:02:52.303 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-04T06:02:52.303 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-04T06:02:52.303 INFO:teuthology.orchestra.run.smithi159.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-04T06:02:52.304 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-04T06:02:52.304 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-04T06:02:52.304 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-04T06:02:52.304 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-04T06:02:52.304 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-04T06:02:52.304 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-04T06:02:52.304 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-04T06:02:52.304 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-04T06:02:52.304 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-04T06:02:52.304 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-04T06:02:52.304 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-04T06:02:52.304 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-04T06:02:52.304 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-04T06:02:52.304 INFO:teuthology.orchestra.run.smithi159.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-04T06:02:52.305 INFO:teuthology.orchestra.run.smithi159.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-04T06:02:52.305 INFO:teuthology.orchestra.run.smithi159.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-04T06:02:52.305 INFO:teuthology.orchestra.run.smithi159.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-04T06:02:52.305 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-04T06:02:52.305 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-04T06:02:52.305 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-04T06:02:52.305 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-04T06:02:52.305 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-04T06:02:52.305 INFO:teuthology.orchestra.run.smithi159.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-04T06:02:52.305 INFO:teuthology.orchestra.run.smithi159.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-04T06:02:52.305 INFO:teuthology.orchestra.run.smithi159.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-04T06:02:52.305 INFO:teuthology.orchestra.run.smithi159.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-04T06:02:52.305 INFO:teuthology.orchestra.run.smithi159.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-04T06:02:52.305 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-04T06:02:52.306 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-04T06:02:52.306 INFO:teuthology.orchestra.run.smithi159.stdout:Installing weak dependencies: 2024-04-04T06:02:52.306 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-04T06:02:52.306 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:02:52.306 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-04-04T06:02:52.306 INFO:teuthology.orchestra.run.smithi159.stdout:============================================================================================ 2024-04-04T06:02:52.306 INFO:teuthology.orchestra.run.smithi159.stdout:Install 150 Packages 2024-04-04T06:02:52.306 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrade 24 Packages 2024-04-04T06:02:52.306 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:02:52.307 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 226 M 2024-04-04T06:02:52.307 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-04-04T06:02:52.903 INFO:teuthology.orchestra.run.smithi178.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-04T06:02:52.905 INFO:teuthology.orchestra.run.smithi178.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-04T06:02:52.926 INFO:teuthology.orchestra.run.smithi178.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-04T06:02:52.927 INFO:teuthology.orchestra.run.smithi178.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-04T06:02:52.929 INFO:teuthology.orchestra.run.smithi178.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-04T06:02:52.930 INFO:teuthology.orchestra.run.smithi178.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-04T06:02:52.933 INFO:teuthology.orchestra.run.smithi178.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-04T06:02:52.936 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-04T06:02:52.938 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-04T06:02:53.032 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-04-04T06:02:53.063 INFO:teuthology.orchestra.run.smithi178.stdout:============================================================================================ 2024-04-04T06:02:53.063 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-04-04T06:02:53.063 INFO:teuthology.orchestra.run.smithi178.stdout:============================================================================================ 2024-04-04T06:02:53.063 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-04-04T06:02:53.063 INFO:teuthology.orchestra.run.smithi178.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-04T06:02:53.064 INFO:teuthology.orchestra.run.smithi178.stdout: ceph x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 6.6 k 2024-04-04T06:02:53.064 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 5.4 M 2024-04-04T06:02:53.064 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 890 k 2024-04-04T06:02:53.064 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 145 k 2024-04-04T06:02:53.064 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 1.5 M 2024-04-04T06:02:53.064 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2571.gaddbc488.el9 ceph-noarch 141 k 2024-04-04T06:02:53.064 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2571.gaddbc488.el9 ceph-noarch 3.5 M 2024-04-04T06:02:53.064 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2571.gaddbc488.el9 ceph-noarch 7.4 M 2024-04-04T06:02:53.064 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook noarch 2:19.0.0-2571.gaddbc488.el9 ceph-noarch 49 k 2024-04-04T06:02:53.064 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 9.8 M 2024-04-04T06:02:53.064 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 48 M 2024-04-04T06:02:53.064 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume noarch 2:19.0.0-2571.gaddbc488.el9 ceph-noarch 264 k 2024-04-04T06:02:53.064 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm noarch 2:19.0.0-2571.gaddbc488.el9 ceph-noarch 762 k 2024-04-04T06:02:53.065 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 225 k 2024-04-04T06:02:53.065 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-shell x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 27 k 2024-04-04T06:02:53.065 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top noarch 2:19.0.0-2571.gaddbc488.el9 ceph-noarch 18 k 2024-04-04T06:02:53.065 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-04T06:02:53.065 INFO:teuthology.orchestra.run.smithi178.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-04T06:02:53.065 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-04T06:02:53.065 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 32 k 2024-04-04T06:02:53.065 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2 x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 746 k 2024-04-04T06:02:53.065 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 127 k 2024-04-04T06:02:53.065 INFO:teuthology.orchestra.run.smithi178.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-04T06:02:53.065 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-04T06:02:53.065 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-04T06:02:53.066 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-04T06:02:53.066 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-04T06:02:53.066 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-04T06:02:53.066 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 161 k 2024-04-04T06:02:53.066 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 320 k 2024-04-04T06:02:53.066 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 298 k 2024-04-04T06:02:53.066 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 99 k 2024-04-04T06:02:53.066 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 85 k 2024-04-04T06:02:53.066 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 3.1 M 2024-04-04T06:02:53.066 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 170 k 2024-04-04T06:02:53.066 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-04T06:02:53.066 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrading: 2024-04-04T06:02:53.067 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-04-04T06:02:53.067 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-04-04T06:02:53.067 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-04-04T06:02:53.067 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-04-04T06:02:53.067 INFO:teuthology.orchestra.run.smithi178.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-04T06:02:53.067 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-04T06:02:53.067 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-04T06:02:53.067 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-04T06:02:53.067 INFO:teuthology.orchestra.run.smithi178.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-04T06:02:53.067 INFO:teuthology.orchestra.run.smithi178.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-04T06:02:53.067 INFO:teuthology.orchestra.run.smithi178.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-04T06:02:53.067 INFO:teuthology.orchestra.run.smithi178.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-04T06:02:53.068 INFO:teuthology.orchestra.run.smithi178.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-04T06:02:53.068 INFO:teuthology.orchestra.run.smithi178.stdout: librados2 x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 3.4 M 2024-04-04T06:02:53.068 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1 x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 3.1 M 2024-04-04T06:02:53.068 INFO:teuthology.orchestra.run.smithi178.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-04T06:02:53.068 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-04T06:02:53.068 INFO:teuthology.orchestra.run.smithi178.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-04T06:02:53.068 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-04T06:02:53.068 INFO:teuthology.orchestra.run.smithi178.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-04T06:02:53.068 INFO:teuthology.orchestra.run.smithi178.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-04T06:02:53.069 INFO:teuthology.orchestra.run.smithi178.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-04T06:02:53.069 INFO:teuthology.orchestra.run.smithi178.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-04T06:02:53.069 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-04T06:02:53.069 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-04-04T06:02:53.069 INFO:teuthology.orchestra.run.smithi178.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-04T06:02:53.069 INFO:teuthology.orchestra.run.smithi178.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-04T06:02:53.069 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-04T06:02:53.069 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 21 M 2024-04-04T06:02:53.069 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2571.gaddbc488.el9 ceph-noarch 34 k 2024-04-04T06:02:53.069 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 2.4 M 2024-04-04T06:02:53.069 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2571.gaddbc488.el9 ceph-noarch 249 k 2024-04-04T06:02:53.070 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 4.7 M 2024-04-04T06:02:53.070 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 17 M 2024-04-04T06:02:53.070 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2571.gaddbc488.el9 ceph-noarch 16 k 2024-04-04T06:02:53.070 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 25 k 2024-04-04T06:02:53.070 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-04T06:02:53.070 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-04T06:02:53.070 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-04T06:02:53.070 INFO:teuthology.orchestra.run.smithi178.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-04T06:02:53.070 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-04T06:02:53.070 INFO:teuthology.orchestra.run.smithi178.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-04T06:02:53.070 INFO:teuthology.orchestra.run.smithi178.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-04T06:02:53.070 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-04T06:02:53.071 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-04T06:02:53.071 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 163 k 2024-04-04T06:02:53.072 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-04T06:02:53.072 INFO:teuthology.orchestra.run.smithi178.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-04T06:02:53.072 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-04T06:02:53.072 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-04T06:02:53.072 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-04T06:02:53.072 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1 x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 500 k 2024-04-04T06:02:53.072 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-04T06:02:53.072 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2 x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 5.1 M 2024-04-04T06:02:53.073 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-04T06:02:53.073 INFO:teuthology.orchestra.run.smithi178.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-04T06:02:53.073 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-04T06:02:53.073 INFO:teuthology.orchestra.run.smithi178.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-04T06:02:53.073 INFO:teuthology.orchestra.run.smithi178.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-04T06:02:53.073 INFO:teuthology.orchestra.run.smithi178.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-04T06:02:53.073 INFO:teuthology.orchestra.run.smithi178.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-04T06:02:53.073 INFO:teuthology.orchestra.run.smithi178.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-04T06:02:53.073 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-04T06:02:53.073 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-04T06:02:53.073 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-04T06:02:53.073 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-04T06:02:53.073 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-04T06:02:53.073 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-04T06:02:53.074 INFO:teuthology.orchestra.run.smithi178.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-04T06:02:53.074 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-04T06:02:53.074 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-04T06:02:53.074 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-04T06:02:53.074 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 45 k 2024-04-04T06:02:53.074 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common x86_64 2:19.0.0-2571.gaddbc488.el9 ceph 134 k 2024-04-04T06:02:53.074 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-04T06:02:53.074 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-04T06:02:53.074 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-04T06:02:53.074 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-04T06:02:53.074 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-04T06:02:53.074 INFO:teuthology.orchestra.run.smithi178.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-04T06:02:53.074 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-04T06:02:53.075 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-04T06:02:53.075 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-04T06:02:53.075 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-04T06:02:53.075 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-04T06:02:53.075 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-04T06:02:53.075 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-04T06:02:53.075 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-04T06:02:53.075 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-04T06:02:53.075 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-04T06:02:53.075 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-04T06:02:53.075 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-04T06:02:53.075 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-04T06:02:53.075 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-04T06:02:53.075 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-04T06:02:53.076 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-04T06:02:53.076 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-04T06:02:53.076 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-04T06:02:53.076 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-04T06:02:53.076 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-04T06:02:53.076 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-04T06:02:53.076 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-04T06:02:53.076 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-04T06:02:53.076 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-04T06:02:53.076 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-04T06:02:53.076 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-04T06:02:53.076 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-04T06:02:53.076 INFO:teuthology.orchestra.run.smithi178.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-04T06:02:53.076 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-04T06:02:53.077 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-04T06:02:53.077 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-04T06:02:53.077 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-04T06:02:53.077 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-04T06:02:53.077 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-04T06:02:53.077 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-04T06:02:53.077 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-04T06:02:53.077 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-04T06:02:53.077 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-04T06:02:53.077 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-04T06:02:53.077 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-04T06:02:53.077 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-04T06:02:53.077 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-04T06:02:53.077 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-04T06:02:53.078 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-04T06:02:53.078 INFO:teuthology.orchestra.run.smithi178.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-04T06:02:53.078 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-04T06:02:53.078 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-04T06:02:53.078 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-04T06:02:53.078 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-04T06:02:53.078 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-04T06:02:53.078 INFO:teuthology.orchestra.run.smithi178.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-04T06:02:53.078 INFO:teuthology.orchestra.run.smithi178.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-04T06:02:53.078 INFO:teuthology.orchestra.run.smithi178.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-04T06:02:53.078 INFO:teuthology.orchestra.run.smithi178.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-04T06:02:53.078 INFO:teuthology.orchestra.run.smithi178.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-04T06:02:53.078 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-04T06:02:53.079 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-04T06:02:53.079 INFO:teuthology.orchestra.run.smithi178.stdout:Installing weak dependencies: 2024-04-04T06:02:53.079 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-04T06:02:53.079 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:02:53.079 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-04-04T06:02:53.079 INFO:teuthology.orchestra.run.smithi178.stdout:============================================================================================ 2024-04-04T06:02:53.079 INFO:teuthology.orchestra.run.smithi178.stdout:Install 150 Packages 2024-04-04T06:02:53.079 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrade 24 Packages 2024-04-04T06:02:53.079 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:02:53.080 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 226 M 2024-04-04T06:02:53.080 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-04-04T06:02:53.254 INFO:teuthology.orchestra.run.smithi159.stdout:(1/174): ceph-19.0.0-2571.gaddbc488.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-04-04T06:02:54.009 INFO:teuthology.orchestra.run.smithi159.stdout:(2/174): ceph-fuse-19.0.0-2571.gaddbc488.el9.x8 1.1 MB/s | 890 kB 00:00 2024-04-04T06:02:54.115 INFO:teuthology.orchestra.run.smithi178.stdout:(1/174): ceph-19.0.0-2571.gaddbc488.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-04-04T06:02:54.301 INFO:teuthology.orchestra.run.smithi159.stdout:(3/174): ceph-common-19.0.0-2571.gaddbc488.el9. 15 MB/s | 21 MB 00:01 2024-04-04T06:02:54.332 INFO:teuthology.orchestra.run.smithi178.stdout:(2/174): ceph-fuse-19.0.0-2571.gaddbc488.el9.x8 4.0 MB/s | 890 kB 00:00 2024-04-04T06:02:54.334 INFO:teuthology.orchestra.run.smithi159.stdout:(4/174): ceph-immutable-object-cache-19.0.0-257 446 kB/s | 145 kB 00:00 2024-04-04T06:02:54.407 INFO:teuthology.orchestra.run.smithi178.stdout:(3/174): ceph-immutable-object-cache-19.0.0-257 1.9 MB/s | 145 kB 00:00 2024-04-04T06:02:54.568 INFO:teuthology.orchestra.run.smithi159.stdout:(5/174): ceph-mds-19.0.0-2571.gaddbc488.el9.x86 8.9 MB/s | 2.4 MB 00:00 2024-04-04T06:02:55.040 INFO:teuthology.orchestra.run.smithi178.stdout:(4/174): ceph-base-19.0.0-2571.gaddbc488.el9.x8 4.2 MB/s | 5.4 MB 00:01 2024-04-04T06:02:55.282 INFO:teuthology.orchestra.run.smithi178.stdout:(5/174): ceph-mds-19.0.0-2571.gaddbc488.el9.x86 2.7 MB/s | 2.4 MB 00:00 2024-04-04T06:02:55.341 INFO:teuthology.orchestra.run.smithi178.stdout:(6/174): ceph-mgr-19.0.0-2571.gaddbc488.el9.x86 5.0 MB/s | 1.5 MB 00:00 2024-04-04T06:02:55.352 INFO:teuthology.orchestra.run.smithi159.stdout:(6/174): ceph-mon-19.0.0-2571.gaddbc488.el9.x86 6.0 MB/s | 4.7 MB 00:00 2024-04-04T06:02:55.544 INFO:teuthology.orchestra.run.smithi159.stdout:(7/174): ceph-base-19.0.0-2571.gaddbc488.el9.x8 2.1 MB/s | 5.4 MB 00:02 2024-04-04T06:02:56.427 INFO:teuthology.orchestra.run.smithi159.stdout:(8/174): ceph-mgr-19.0.0-2571.gaddbc488.el9.x86 731 kB/s | 1.5 MB 00:02 2024-04-04T06:02:56.452 INFO:teuthology.orchestra.run.smithi178.stdout:(7/174): ceph-mon-19.0.0-2571.gaddbc488.el9.x86 4.0 MB/s | 4.7 MB 00:01 2024-04-04T06:02:56.536 INFO:teuthology.orchestra.run.smithi159.stdout:(9/174): ceph-selinux-19.0.0-2571.gaddbc488.el9 234 kB/s | 25 kB 00:00 2024-04-04T06:02:57.286 INFO:teuthology.orchestra.run.smithi159.stdout:(10/174): ceph-radosgw-19.0.0-2571.gaddbc488.el 5.7 MB/s | 9.8 MB 00:01 2024-04-04T06:02:57.353 INFO:teuthology.orchestra.run.smithi159.stdout:(11/174): cephfs-mirror-19.0.0-2571.gaddbc488.e 3.4 MB/s | 225 kB 00:00 2024-04-04T06:02:57.420 INFO:teuthology.orchestra.run.smithi159.stdout:(12/174): cephfs-shell-19.0.0-2571.gaddbc488.el 410 kB/s | 27 kB 00:00 2024-04-04T06:02:57.474 INFO:teuthology.orchestra.run.smithi178.stdout:(8/174): ceph-osd-19.0.0-2571.gaddbc488.el9.x86 7.9 MB/s | 17 MB 00:02 2024-04-04T06:02:57.487 INFO:teuthology.orchestra.run.smithi159.stdout:(13/174): libcephfs-devel-19.0.0-2571.gaddbc488 482 kB/s | 32 kB 00:00 2024-04-04T06:02:57.542 INFO:teuthology.orchestra.run.smithi178.stdout:(9/174): ceph-selinux-19.0.0-2571.gaddbc488.el9 383 kB/s | 25 kB 00:00 2024-04-04T06:02:57.646 INFO:teuthology.orchestra.run.smithi159.stdout:(14/174): libcephfs2-19.0.0-2571.gaddbc488.el9. 4.6 MB/s | 746 kB 00:00 2024-04-04T06:02:57.738 INFO:teuthology.orchestra.run.smithi159.stdout:(15/174): libcephsqlite-19.0.0-2571.gaddbc488.e 1.8 MB/s | 163 kB 00:00 2024-04-04T06:02:57.805 INFO:teuthology.orchestra.run.smithi159.stdout:(16/174): librados-devel-19.0.0-2571.gaddbc488. 1.9 MB/s | 127 kB 00:00 2024-04-04T06:02:57.888 INFO:teuthology.orchestra.run.smithi159.stdout:(17/174): libradosstriper1-19.0.0-2571.gaddbc48 5.9 MB/s | 500 kB 00:00 2024-04-04T06:02:57.933 INFO:teuthology.orchestra.run.smithi178.stdout:(10/174): ceph-common-19.0.0-2571.gaddbc488.el9 4.9 MB/s | 21 MB 00:04 2024-04-04T06:02:58.134 INFO:teuthology.orchestra.run.smithi178.stdout:(11/174): cephfs-mirror-19.0.0-2571.gaddbc488.e 1.1 MB/s | 225 kB 00:00 2024-04-04T06:02:58.193 INFO:teuthology.orchestra.run.smithi178.stdout:(12/174): cephfs-shell-19.0.0-2571.gaddbc488.el 472 kB/s | 27 kB 00:00 2024-04-04T06:02:58.260 INFO:teuthology.orchestra.run.smithi178.stdout:(13/174): libcephfs-devel-19.0.0-2571.gaddbc488 482 kB/s | 32 kB 00:00 2024-04-04T06:02:58.330 INFO:teuthology.orchestra.run.smithi159.stdout:(18/174): ceph-osd-19.0.0-2571.gaddbc488.el9.x8 5.7 MB/s | 17 MB 00:02 2024-04-04T06:02:58.414 INFO:teuthology.orchestra.run.smithi159.stdout:(19/174): python3-ceph-argparse-19.0.0-2571.gad 549 kB/s | 45 kB 00:00 2024-04-04T06:02:58.460 INFO:teuthology.orchestra.run.smithi178.stdout:(14/174): libcephfs2-19.0.0-2571.gaddbc488.el9. 3.7 MB/s | 746 kB 00:00 2024-04-04T06:02:58.489 INFO:teuthology.orchestra.run.smithi159.stdout:(20/174): python3-ceph-common-19.0.0-2571.gaddb 1.8 MB/s | 134 kB 00:00 2024-04-04T06:02:58.544 INFO:teuthology.orchestra.run.smithi178.stdout:(15/174): libcephsqlite-19.0.0-2571.gaddbc488.e 1.9 MB/s | 163 kB 00:00 2024-04-04T06:02:58.556 INFO:teuthology.orchestra.run.smithi159.stdout:(21/174): python3-cephfs-19.0.0-2571.gaddbc488. 2.4 MB/s | 161 kB 00:00 2024-04-04T06:02:58.610 INFO:teuthology.orchestra.run.smithi178.stdout:(16/174): librados-devel-19.0.0-2571.gaddbc488. 1.9 MB/s | 127 kB 00:00 2024-04-04T06:02:58.673 INFO:teuthology.orchestra.run.smithi159.stdout:(22/174): librgw2-19.0.0-2571.gaddbc488.el9.x86 6.6 MB/s | 5.1 MB 00:00 2024-04-04T06:02:58.706 INFO:teuthology.orchestra.run.smithi159.stdout:(23/174): python3-rados-19.0.0-2571.gaddbc488.e 2.1 MB/s | 320 kB 00:00 2024-04-04T06:02:58.744 INFO:teuthology.orchestra.run.smithi178.stdout:(17/174): libradosstriper1-19.0.0-2571.gaddbc48 3.7 MB/s | 500 kB 00:00 2024-04-04T06:02:58.757 INFO:teuthology.orchestra.run.smithi159.stdout:(24/174): python3-rbd-19.0.0-2571.gaddbc488.el9 3.5 MB/s | 298 kB 00:00 2024-04-04T06:02:58.790 INFO:teuthology.orchestra.run.smithi159.stdout:(25/174): python3-rgw-19.0.0-2571.gaddbc488.el9 1.2 MB/s | 99 kB 00:00 2024-04-04T06:02:58.841 INFO:teuthology.orchestra.run.smithi159.stdout:(26/174): rbd-fuse-19.0.0-2571.gaddbc488.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-04T06:02:58.919 INFO:teuthology.orchestra.run.smithi178.stdout:(18/174): ceph-radosgw-19.0.0-2571.gaddbc488.el 4.0 MB/s | 9.8 MB 00:02 2024-04-04T06:02:58.933 INFO:teuthology.orchestra.run.smithi159.stdout:(27/174): rbd-nbd-19.0.0-2571.gaddbc488.el9.x86 1.8 MB/s | 170 kB 00:00 2024-04-04T06:02:58.970 INFO:teuthology.orchestra.run.smithi178.stdout:(19/174): python3-ceph-argparse-19.0.0-2571.gad 917 kB/s | 45 kB 00:00 2024-04-04T06:02:59.028 INFO:teuthology.orchestra.run.smithi178.stdout:(20/174): python3-ceph-common-19.0.0-2571.gaddb 2.3 MB/s | 134 kB 00:00 2024-04-04T06:02:59.033 INFO:teuthology.orchestra.run.smithi159.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2571.g 342 kB/s | 34 kB 00:00 2024-04-04T06:02:59.095 INFO:teuthology.orchestra.run.smithi178.stdout:(21/174): python3-cephfs-19.0.0-2571.gaddbc488. 2.4 MB/s | 161 kB 00:00 2024-04-04T06:02:59.125 INFO:teuthology.orchestra.run.smithi159.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2571.gaddbc48 1.5 MB/s | 141 kB 00:00 2024-04-04T06:02:59.229 INFO:teuthology.orchestra.run.smithi178.stdout:(22/174): python3-rados-19.0.0-2571.gaddbc488.e 2.4 MB/s | 320 kB 00:00 2024-04-04T06:02:59.233 INFO:teuthology.orchestra.run.smithi159.stdout:(30/174): rbd-mirror-19.0.0-2571.gaddbc488.el9. 7.0 MB/s | 3.1 MB 00:00 2024-04-04T06:02:59.338 INFO:teuthology.orchestra.run.smithi178.stdout:(23/174): python3-rbd-19.0.0-2571.gaddbc488.el9 2.7 MB/s | 298 kB 00:00 2024-04-04T06:02:59.421 INFO:teuthology.orchestra.run.smithi178.stdout:(24/174): python3-rgw-19.0.0-2571.gaddbc488.el9 1.2 MB/s | 99 kB 00:00 2024-04-04T06:02:59.505 INFO:teuthology.orchestra.run.smithi178.stdout:(25/174): rbd-fuse-19.0.0-2571.gaddbc488.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-04T06:02:59.701 INFO:teuthology.orchestra.run.smithi159.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2571.gaddbc 6.2 MB/s | 3.5 MB 00:00 2024-04-04T06:02:59.780 INFO:teuthology.orchestra.run.smithi178.stdout:(26/174): librgw2-19.0.0-2571.gaddbc488.el9.x86 5.0 MB/s | 5.1 MB 00:01 2024-04-04T06:02:59.784 INFO:teuthology.orchestra.run.smithi159.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2571.gad 3.0 MB/s | 249 kB 00:00 2024-04-04T06:02:59.864 INFO:teuthology.orchestra.run.smithi178.stdout:(27/174): rbd-nbd-19.0.0-2571.gaddbc488.el9.x86 2.0 MB/s | 170 kB 00:00 2024-04-04T06:02:59.876 INFO:teuthology.orchestra.run.smithi159.stdout:(33/174): ceph-mgr-rook-19.0.0-2571.gaddbc488.e 543 kB/s | 49 kB 00:00 2024-04-04T06:02:59.931 INFO:teuthology.orchestra.run.smithi178.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2571.g 512 kB/s | 34 kB 00:00 2024-04-04T06:02:59.968 INFO:teuthology.orchestra.run.smithi159.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2571.ga 181 kB/s | 16 kB 00:00 2024-04-04T06:03:00.007 INFO:teuthology.orchestra.run.smithi178.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2571.gaddbc48 1.9 MB/s | 141 kB 00:00 2024-04-04T06:03:00.052 INFO:teuthology.orchestra.run.smithi159.stdout:(35/174): ceph-volume-19.0.0-2571.gaddbc488.el9 3.1 MB/s | 264 kB 00:00 2024-04-04T06:03:00.219 INFO:teuthology.orchestra.run.smithi159.stdout:(36/174): cephadm-19.0.0-2571.gaddbc488.el9.noa 4.5 MB/s | 762 kB 00:00 2024-04-04T06:03:00.248 INFO:teuthology.orchestra.run.smithi178.stdout:(30/174): rbd-mirror-19.0.0-2571.gaddbc488.el9. 4.2 MB/s | 3.1 MB 00:00 2024-04-04T06:03:00.286 INFO:teuthology.orchestra.run.smithi159.stdout:(37/174): cephfs-top-19.0.0-2571.gaddbc488.el9. 276 kB/s | 18 kB 00:00 2024-04-04T06:03:00.478 INFO:teuthology.orchestra.run.smithi159.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 6.0 MB/s | 7.4 MB 00:01 2024-04-04T06:03:00.570 INFO:teuthology.orchestra.run.smithi159.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 662 kB 00:00 2024-04-04T06:03:00.582 INFO:teuthology.orchestra.run.smithi178.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2571.gaddbc 6.2 MB/s | 3.5 MB 00:00 2024-04-04T06:03:00.620 INFO:teuthology.orchestra.run.smithi159.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 6.5 MB/s | 920 kB 00:00 2024-04-04T06:03:00.654 INFO:teuthology.orchestra.run.smithi159.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 106 kB 00:00 2024-04-04T06:03:00.657 INFO:teuthology.orchestra.run.smithi178.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2571.gad 3.3 MB/s | 249 kB 00:00 2024-04-04T06:03:00.687 INFO:teuthology.orchestra.run.smithi159.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 307 kB 00:00 2024-04-04T06:03:00.713 INFO:teuthology.orchestra.run.smithi159.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 572 kB/s | 32 kB 00:00 2024-04-04T06:03:00.740 INFO:teuthology.orchestra.run.smithi178.stdout:(33/174): ceph-mgr-rook-19.0.0-2571.gaddbc488.e 598 kB/s | 49 kB 00:00 2024-04-04T06:03:00.805 INFO:teuthology.orchestra.run.smithi159.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-04-04T06:03:00.816 INFO:teuthology.orchestra.run.smithi178.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2571.ga 222 kB/s | 16 kB 00:00 2024-04-04T06:03:00.830 INFO:teuthology.orchestra.run.smithi159.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 169 kB/s | 17 kB 00:00 2024-04-04T06:03:00.863 INFO:teuthology.orchestra.run.smithi159.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 313 kB/s | 10 kB 00:00 2024-04-04T06:03:00.888 INFO:teuthology.orchestra.run.smithi159.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-04-04T06:03:00.908 INFO:teuthology.orchestra.run.smithi178.stdout:(35/174): ceph-volume-19.0.0-2571.gaddbc488.el9 2.8 MB/s | 264 kB 00:00 2024-04-04T06:03:00.930 INFO:teuthology.orchestra.run.smithi159.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 44 kB 00:00 2024-04-04T06:03:00.964 INFO:teuthology.orchestra.run.smithi159.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.8 MB/s | 661 kB 00:00 2024-04-04T06:03:01.014 INFO:teuthology.orchestra.run.smithi159.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 12 MB/s | 578 kB 00:00 2024-04-04T06:03:01.040 INFO:teuthology.orchestra.run.smithi159.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 628 kB/s | 46 kB 00:00 2024-04-04T06:03:01.059 INFO:teuthology.orchestra.run.smithi178.stdout:(36/174): cephadm-19.0.0-2571.gaddbc488.el9.noa 5.0 MB/s | 762 kB 00:00 2024-04-04T06:03:01.081 INFO:teuthology.orchestra.run.smithi159.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.8 MB/s | 282 kB 00:00 2024-04-04T06:03:01.107 INFO:teuthology.orchestra.run.smithi159.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 187 kB 00:00 2024-04-04T06:03:01.133 INFO:teuthology.orchestra.run.smithi178.stdout:(37/174): cephfs-top-19.0.0-2571.gaddbc488.el9. 245 kB/s | 18 kB 00:00 2024-04-04T06:03:01.149 INFO:teuthology.orchestra.run.smithi159.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.2 MB/s | 217 kB 00:00 2024-04-04T06:03:01.174 INFO:teuthology.orchestra.run.smithi159.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 521 kB/s | 34 kB 00:00 2024-04-04T06:03:01.291 INFO:teuthology.orchestra.run.smithi159.stdout:(56/174): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.5 MB 00:00 2024-04-04T06:03:01.399 INFO:teuthology.orchestra.run.smithi159.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2024-04-04T06:03:01.417 INFO:teuthology.orchestra.run.smithi178.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 6.3 MB/s | 7.4 MB 00:01 2024-04-04T06:03:01.450 INFO:teuthology.orchestra.run.smithi159.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 5.5 MB/s | 817 kB 00:00 2024-04-04T06:03:01.475 INFO:teuthology.orchestra.run.smithi159.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 132 kB 00:00 2024-04-04T06:03:01.484 INFO:teuthology.orchestra.run.smithi178.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 662 kB 00:00 2024-04-04T06:03:01.565 INFO:teuthology.orchestra.run.smithi178.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 6.2 MB/s | 920 kB 00:00 2024-04-04T06:03:01.593 INFO:teuthology.orchestra.run.smithi178.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 106 kB 00:00 2024-04-04T06:03:01.627 INFO:teuthology.orchestra.run.smithi178.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.0 MB/s | 307 kB 00:00 2024-04-04T06:03:01.642 INFO:teuthology.orchestra.run.smithi159.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 30 MB/s | 5.7 MB 00:00 2024-04-04T06:03:01.643 INFO:teuthology.orchestra.run.smithi178.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 666 kB/s | 32 kB 00:00 2024-04-04T06:03:01.684 INFO:teuthology.orchestra.run.smithi159.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.2 MB/s | 1.0 MB 00:00 2024-04-04T06:03:01.742 INFO:teuthology.orchestra.run.smithi159.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-04-04T06:03:01.752 INFO:teuthology.orchestra.run.smithi178.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-04T06:03:01.769 INFO:teuthology.orchestra.run.smithi178.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 140 kB/s | 17 kB 00:00 2024-04-04T06:03:01.819 INFO:teuthology.orchestra.run.smithi178.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 153 kB/s | 10 kB 00:00 2024-04-04T06:03:01.845 INFO:teuthology.orchestra.run.smithi178.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-04-04T06:03:01.870 INFO:teuthology.orchestra.run.smithi178.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 918 kB/s | 44 kB 00:00 2024-04-04T06:03:01.920 INFO:teuthology.orchestra.run.smithi178.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 661 kB 00:00 2024-04-04T06:03:01.943 INFO:teuthology.orchestra.run.smithi159.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 5.8 MB 00:00 2024-04-04T06:03:01.968 INFO:teuthology.orchestra.run.smithi159.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 116 kB/s | 24 kB 00:00 2024-04-04T06:03:01.979 INFO:teuthology.orchestra.run.smithi178.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 5.5 MB/s | 578 kB 00:00 2024-04-04T06:03:02.001 INFO:teuthology.orchestra.run.smithi159.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 564 kB/s | 18 kB 00:00 2024-04-04T06:03:02.004 INFO:teuthology.orchestra.run.smithi178.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 597 kB/s | 46 kB 00:00 2024-04-04T06:03:02.027 INFO:teuthology.orchestra.run.smithi159.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 144 kB 00:00 2024-04-04T06:03:02.038 INFO:teuthology.orchestra.run.smithi178.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-04-04T06:03:02.072 INFO:teuthology.orchestra.run.smithi178.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 187 kB 00:00 2024-04-04T06:03:02.102 INFO:teuthology.orchestra.run.smithi159.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 427 kB/s | 32 kB 00:00 2024-04-04T06:03:02.105 INFO:teuthology.orchestra.run.smithi178.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2024-04-04T06:03:02.131 INFO:teuthology.orchestra.run.smithi178.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 609 kB/s | 34 kB 00:00 2024-04-04T06:03:02.202 INFO:teuthology.orchestra.run.smithi159.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 4.8 MB 00:00 2024-04-04T06:03:02.236 INFO:teuthology.orchestra.run.smithi159.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 217 kB 00:00 2024-04-04T06:03:02.270 INFO:teuthology.orchestra.run.smithi159.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.7 MB/s | 174 kB 00:00 2024-04-04T06:03:02.289 INFO:teuthology.orchestra.run.smithi178.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 26 MB/s | 4.7 MB 00:00 2024-04-04T06:03:02.295 INFO:teuthology.orchestra.run.smithi159.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 69 kB 00:00 2024-04-04T06:03:02.320 INFO:teuthology.orchestra.run.smithi159.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 838 kB/s | 41 kB 00:00 2024-04-04T06:03:02.354 INFO:teuthology.orchestra.run.smithi159.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.1 MB/s | 238 kB 00:00 2024-04-04T06:03:02.373 INFO:teuthology.orchestra.run.smithi178.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 6.7 MB/s | 1.5 MB 00:00 2024-04-04T06:03:02.379 INFO:teuthology.orchestra.run.smithi159.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2024-04-04T06:03:02.423 INFO:teuthology.orchestra.run.smithi178.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 6.0 MB/s | 817 kB 00:00 2024-04-04T06:03:02.438 INFO:teuthology.orchestra.run.smithi159.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 211 kB 00:00 2024-04-04T06:03:02.448 INFO:teuthology.orchestra.run.smithi178.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 132 kB 00:00 2024-04-04T06:03:02.463 INFO:teuthology.orchestra.run.smithi159.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 583 kB/s | 47 kB 00:00 2024-04-04T06:03:02.488 INFO:teuthology.orchestra.run.smithi159.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 872 kB/s | 43 kB 00:00 2024-04-04T06:03:02.591 INFO:teuthology.orchestra.run.smithi178.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 35 MB/s | 5.7 MB 00:00 2024-04-04T06:03:02.614 INFO:teuthology.orchestra.run.smithi159.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 315 kB/s | 39 kB 00:00 2024-04-04T06:03:02.632 INFO:teuthology.orchestra.run.smithi178.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.0 MB/s | 1.0 MB 00:00 2024-04-04T06:03:02.649 INFO:teuthology.orchestra.run.smithi178.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.4 MB/s | 78 kB 00:00 2024-04-04T06:03:02.808 INFO:teuthology.orchestra.run.smithi178.stdout:(63/174): python3-jsonpatch-1.21-16.el9.noarch. 155 kB/s | 24 kB 00:00 2024-04-04T06:03:02.925 INFO:teuthology.orchestra.run.smithi178.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2024-04-04T06:03:02.951 INFO:teuthology.orchestra.run.smithi178.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 141 kB/s | 18 kB 00:00 2024-04-04T06:03:02.976 INFO:teuthology.orchestra.run.smithi178.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2024-04-04T06:03:02.981 INFO:teuthology.orchestra.run.smithi159.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-04-04T06:03:03.001 INFO:teuthology.orchestra.run.smithi178.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 656 kB/s | 32 kB 00:00 2024-04-04T06:03:03.006 INFO:teuthology.orchestra.run.smithi159.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 107 kB/s | 41 kB 00:00 2024-04-04T06:03:03.031 INFO:teuthology.orchestra.run.smithi159.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 4.4 MB/s | 220 kB 00:00 2024-04-04T06:03:03.073 INFO:teuthology.orchestra.run.smithi159.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 543 kB/s | 35 kB 00:00 2024-04-04T06:03:03.115 INFO:teuthology.orchestra.run.smithi159.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 559 kB/s | 46 kB 00:00 2024-04-04T06:03:03.149 INFO:teuthology.orchestra.run.smithi159.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-04-04T06:03:03.174 INFO:teuthology.orchestra.run.smithi159.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2024-04-04T06:03:03.200 INFO:teuthology.orchestra.run.smithi159.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-04T06:03:03.225 INFO:teuthology.orchestra.run.smithi159.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 901 kB/s | 44 kB 00:00 2024-04-04T06:03:03.250 INFO:teuthology.orchestra.run.smithi159.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2024-04-04T06:03:03.252 INFO:teuthology.orchestra.run.smithi178.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 4.8 MB 00:00 2024-04-04T06:03:03.284 INFO:teuthology.orchestra.run.smithi159.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.1 MB/s | 241 kB 00:00 2024-04-04T06:03:03.285 INFO:teuthology.orchestra.run.smithi178.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 807 kB/s | 217 kB 00:00 2024-04-04T06:03:03.310 INFO:teuthology.orchestra.run.smithi178.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.0 MB/s | 174 kB 00:00 2024-04-04T06:03:03.335 INFO:teuthology.orchestra.run.smithi159.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-04-04T06:03:03.336 INFO:teuthology.orchestra.run.smithi178.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.4 MB/s | 69 kB 00:00 2024-04-04T06:03:03.353 INFO:teuthology.orchestra.run.smithi178.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 41 kB 00:00 2024-04-04T06:03:03.359 INFO:teuthology.orchestra.run.smithi159.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-04-04T06:03:03.385 INFO:teuthology.orchestra.run.smithi159.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2024-04-04T06:03:03.386 INFO:teuthology.orchestra.run.smithi178.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.8 MB/s | 238 kB 00:00 2024-04-04T06:03:03.412 INFO:teuthology.orchestra.run.smithi178.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2024-04-04T06:03:03.418 INFO:teuthology.orchestra.run.smithi159.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-04-04T06:03:03.446 INFO:teuthology.orchestra.run.smithi178.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2024-04-04T06:03:03.462 INFO:teuthology.orchestra.run.smithi178.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 992 kB/s | 47 kB 00:00 2024-04-04T06:03:03.488 INFO:teuthology.orchestra.run.smithi178.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 43 kB 00:00 2024-04-04T06:03:03.810 INFO:teuthology.orchestra.run.smithi159.stdout:(94/174): jq-1.6-16.el9.x86_64.rpm 443 kB/s | 187 kB 00:00 2024-04-04T06:03:03.911 INFO:teuthology.orchestra.run.smithi159.stdout:(95/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-04T06:03:03.953 INFO:teuthology.orchestra.run.smithi159.stdout:(96/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 803 kB 00:00 2024-04-04T06:03:04.020 INFO:teuthology.orchestra.run.smithi159.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-04-04T06:03:04.055 INFO:teuthology.orchestra.run.smithi178.stdout:(78/174): ceph-test-19.0.0-2571.gaddbc488.el9.x 7.3 MB/s | 48 MB 00:06 2024-04-04T06:03:04.397 INFO:teuthology.orchestra.run.smithi178.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-04-04T06:03:04.422 INFO:teuthology.orchestra.run.smithi178.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 42 kB/s | 39 kB 00:00 2024-04-04T06:03:04.447 INFO:teuthology.orchestra.run.smithi178.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 108 kB/s | 41 kB 00:00 2024-04-04T06:03:04.481 INFO:teuthology.orchestra.run.smithi178.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 220 kB 00:00 2024-04-04T06:03:04.506 INFO:teuthology.orchestra.run.smithi178.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 624 kB/s | 35 kB 00:00 2024-04-04T06:03:04.531 INFO:teuthology.orchestra.run.smithi178.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 573 kB/s | 46 kB 00:00 2024-04-04T06:03:04.565 INFO:teuthology.orchestra.run.smithi178.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 300 kB 00:00 2024-04-04T06:03:04.590 INFO:teuthology.orchestra.run.smithi178.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 63 kB 00:00 2024-04-04T06:03:04.615 INFO:teuthology.orchestra.run.smithi178.stdout:(87/174): uuid-1.6.2-55.el9.x86_64.rpm 701 kB/s | 56 kB 00:00 2024-04-04T06:03:04.640 INFO:teuthology.orchestra.run.smithi178.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 594 kB/s | 44 kB 00:00 2024-04-04T06:03:04.666 INFO:teuthology.orchestra.run.smithi178.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-04-04T06:03:04.708 INFO:teuthology.orchestra.run.smithi178.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.6 MB/s | 241 kB 00:00 2024-04-04T06:03:04.772 INFO:teuthology.orchestra.run.smithi178.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 8.9 MB/s | 1.1 MB 00:00 2024-04-04T06:03:04.800 INFO:teuthology.orchestra.run.smithi178.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 772 kB/s | 103 kB 00:00 2024-04-04T06:03:04.825 INFO:teuthology.orchestra.run.smithi178.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 124 kB 00:00 2024-04-04T06:03:04.859 INFO:teuthology.orchestra.run.smithi178.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-04-04T06:03:05.351 INFO:teuthology.orchestra.run.smithi178.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 341 kB/s | 187 kB 00:00 2024-04-04T06:03:05.385 INFO:teuthology.orchestra.run.smithi178.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 372 kB/s | 195 kB 00:00 2024-04-04T06:03:05.468 INFO:teuthology.orchestra.run.smithi178.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-04-04T06:03:05.494 INFO:teuthology.orchestra.run.smithi178.stdout:(98/174): elfutils-libelf-devel-0.190-2.el9.x86 199 kB/s | 21 kB 00:00 2024-04-04T06:03:05.519 INFO:teuthology.orchestra.run.smithi178.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 512 kB/s | 25 kB 00:00 2024-04-04T06:03:05.619 INFO:teuthology.orchestra.run.smithi178.stdout:(100/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-04-04T06:03:05.653 INFO:teuthology.orchestra.run.smithi178.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 1.8 MB/s | 245 kB 00:00 2024-04-04T06:03:05.703 INFO:teuthology.orchestra.run.smithi178.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.9 MB/s | 94 kB 00:00 2024-04-04T06:03:05.787 INFO:teuthology.orchestra.run.smithi178.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 270 kB/s | 22 kB 00:00 2024-04-04T06:03:05.820 INFO:teuthology.orchestra.run.smithi178.stdout:(104/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.2 MB/s | 248 kB 00:00 2024-04-04T06:03:05.921 INFO:teuthology.orchestra.run.smithi178.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 9.7 MB/s | 4.1 MB 00:00 2024-04-04T06:03:06.003 INFO:teuthology.orchestra.run.smithi159.stdout:(98/174): elfutils-libelf-devel-0.190-2.el9.x86 10 kB/s | 21 kB 00:02 2024-04-04T06:03:06.021 INFO:teuthology.orchestra.run.smithi178.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 3.6 MB/s | 863 kB 00:00 2024-04-04T06:03:06.028 INFO:teuthology.orchestra.run.smithi159.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 13 kB/s | 25 kB 00:02 2024-04-04T06:03:06.138 INFO:teuthology.orchestra.run.smithi178.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 169 kB/s | 19 kB 00:00 2024-04-04T06:03:06.180 INFO:teuthology.orchestra.run.smithi178.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-04-04T06:03:06.222 INFO:teuthology.orchestra.run.smithi178.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-04-04T06:03:06.239 INFO:teuthology.orchestra.run.smithi178.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 449 kB/s | 26 kB 00:00 2024-04-04T06:03:06.281 INFO:teuthology.orchestra.run.smithi178.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 840 kB/s | 49 kB 00:00 2024-04-04T06:03:06.306 INFO:teuthology.orchestra.run.smithi178.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-04T06:03:06.365 INFO:teuthology.orchestra.run.smithi178.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-04T06:03:06.473 INFO:teuthology.orchestra.run.smithi178.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.9 MB/s | 838 kB 00:00 2024-04-04T06:03:06.515 INFO:teuthology.orchestra.run.smithi178.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-04-04T06:03:06.532 INFO:teuthology.orchestra.run.smithi178.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 750 kB/s | 43 kB 00:00 2024-04-04T06:03:06.583 INFO:teuthology.orchestra.run.smithi178.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 486 kB/s | 32 kB 00:00 2024-04-04T06:03:06.608 INFO:teuthology.orchestra.run.smithi178.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 201 kB/s | 15 kB 00:00 2024-04-04T06:03:06.650 INFO:teuthology.orchestra.run.smithi178.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-04T06:03:06.767 INFO:teuthology.orchestra.run.smithi178.stdout:(120/174): libarrow-9.0.0-10.el9.x86_64.rpm 7.0 MB/s | 4.4 MB 00:00 2024-04-04T06:03:06.801 INFO:teuthology.orchestra.run.smithi178.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 1.8 MB/s | 358 kB 00:00 2024-04-04T06:03:06.826 INFO:teuthology.orchestra.run.smithi178.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 611 kB/s | 35 kB 00:00 2024-04-04T06:03:06.851 INFO:teuthology.orchestra.run.smithi178.stdout:(123/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.5 MB/s | 303 kB 00:00 2024-04-04T06:03:06.876 INFO:teuthology.orchestra.run.smithi178.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.7 MB/s | 203 kB 00:00 2024-04-04T06:03:06.918 INFO:teuthology.orchestra.run.smithi178.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-04-04T06:03:06.944 INFO:teuthology.orchestra.run.smithi178.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 161 kB/s | 11 kB 00:00 2024-04-04T06:03:06.977 INFO:teuthology.orchestra.run.smithi178.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 306 kB/s | 18 kB 00:00 2024-04-04T06:03:07.002 INFO:teuthology.orchestra.run.smithi178.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 398 kB/s | 23 kB 00:00 2024-04-04T06:03:07.036 INFO:teuthology.orchestra.run.smithi178.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 336 kB/s | 19 kB 00:00 2024-04-04T06:03:07.062 INFO:teuthology.orchestra.run.smithi178.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 336 kB/s | 20 kB 00:00 2024-04-04T06:03:07.143 INFO:teuthology.orchestra.run.smithi178.stdout:(131/174): python3-grpcio-1.46.7-10.el9.x86_64. 6.4 MB/s | 2.0 MB 00:00 2024-04-04T06:03:07.171 INFO:teuthology.orchestra.run.smithi178.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 428 kB/s | 46 kB 00:00 2024-04-04T06:03:07.204 INFO:teuthology.orchestra.run.smithi178.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-04T06:03:07.229 INFO:teuthology.orchestra.run.smithi178.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 996 kB/s | 58 kB 00:00 2024-04-04T06:03:07.288 INFO:teuthology.orchestra.run.smithi178.stdout:(135/174): python3-kubernetes-26.1.0-2.el9.noar 4.1 MB/s | 1.0 MB 00:00 2024-04-04T06:03:07.305 INFO:teuthology.orchestra.run.smithi178.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 220 kB/s | 16 kB 00:00 2024-04-04T06:03:07.339 INFO:teuthology.orchestra.run.smithi178.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-04-04T06:03:07.355 INFO:teuthology.orchestra.run.smithi178.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 568 kB/s | 28 kB 00:00 2024-04-04T06:03:07.381 INFO:teuthology.orchestra.run.smithi178.stdout:(139/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 980 kB/s | 90 kB 00:00 2024-04-04T06:03:07.406 INFO:teuthology.orchestra.run.smithi178.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 463 kB/s | 31 kB 00:00 2024-04-04T06:03:07.440 INFO:teuthology.orchestra.run.smithi178.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-04T06:03:07.465 INFO:teuthology.orchestra.run.smithi178.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 617 kB/s | 36 kB 00:00 2024-04-04T06:03:07.490 INFO:teuthology.orchestra.run.smithi178.stdout:(143/174): python3-rsa-4.9-2.el9.noarch.rpm 545 kB/s | 59 kB 00:00 2024-04-04T06:03:07.547 INFO:teuthology.orchestra.run.smithi178.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 479 kB/s | 51 kB 00:00 2024-04-04T06:03:07.583 INFO:teuthology.orchestra.run.smithi178.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 981 kB/s | 90 kB 00:00 2024-04-04T06:03:07.616 INFO:teuthology.orchestra.run.smithi178.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2024-04-04T06:03:07.667 INFO:teuthology.orchestra.run.smithi178.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-04-04T06:03:07.692 INFO:teuthology.orchestra.run.smithi178.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-04-04T06:03:07.726 INFO:teuthology.orchestra.run.smithi178.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-04T06:03:07.885 INFO:teuthology.orchestra.run.smithi178.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-04-04T06:03:08.118 INFO:teuthology.orchestra.run.smithi178.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 9.9 MB/s | 2.3 MB 00:00 2024-04-04T06:03:08.202 INFO:teuthology.orchestra.run.smithi178.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 22 MB/s | 1.8 MB 00:00 2024-04-04T06:03:08.235 INFO:teuthology.orchestra.run.smithi178.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-04T06:03:08.430 INFO:teuthology.orchestra.run.smithi178.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 1.2 MB/s | 247 kB 00:00 2024-04-04T06:03:08.518 INFO:teuthology.orchestra.run.smithi178.stdout:(155/174): librados2-19.0.0-2571.gaddbc488.el9. 4.1 MB/s | 3.4 MB 00:00 2024-04-04T06:03:08.560 INFO:teuthology.orchestra.run.smithi178.stdout:(156/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.4 MB/s | 321 kB 00:00 2024-04-04T06:03:08.744 INFO:teuthology.orchestra.run.smithi178.stdout:(157/174): librbd1-19.0.0-2571.gaddbc488.el9.x8 3.1 MB/s | 3.1 MB 00:01 2024-04-04T06:03:08.769 INFO:teuthology.orchestra.run.smithi178.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 175 kB/s | 36 kB 00:00 2024-04-04T06:03:08.823 INFO:teuthology.orchestra.run.smithi178.stdout:(159/174): acl-2.3.1-4.el9.x86_64.rpm 238 kB/s | 71 kB 00:00 2024-04-04T06:03:08.937 INFO:teuthology.orchestra.run.smithi178.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 204 kB/s | 23 kB 00:00 2024-04-04T06:03:09.054 INFO:teuthology.orchestra.run.smithi178.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 902 kB/s | 256 kB 00:00 2024-04-04T06:03:09.162 INFO:teuthology.orchestra.run.smithi159.stdout:(100/174): ceph-test-19.0.0-2571.gaddbc488.el9. 3.8 MB/s | 48 MB 00:12 2024-04-04T06:03:09.162 INFO:teuthology.orchestra.run.smithi178.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 482 kB/s | 108 kB 00:00 2024-04-04T06:03:09.225 INFO:teuthology.orchestra.run.smithi178.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 908 kB/s | 155 kB 00:00 2024-04-04T06:03:09.279 INFO:teuthology.orchestra.run.smithi178.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 136 kB 00:00 2024-04-04T06:03:09.338 INFO:teuthology.orchestra.run.smithi178.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 569 kB/s | 64 kB 00:00 2024-04-04T06:03:09.363 INFO:teuthology.orchestra.run.smithi178.stdout:(166/174): elfutils-libelf-0.190-2.el9.x86_64.r 311 kB/s | 192 kB 00:00 2024-04-04T06:03:09.389 INFO:teuthology.orchestra.run.smithi178.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 264 kB/s | 29 kB 00:00 2024-04-04T06:03:09.806 INFO:teuthology.orchestra.run.smithi178.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 2.6 MB/s | 1.2 MB 00:00 2024-04-04T06:03:09.889 INFO:teuthology.orchestra.run.smithi178.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 4.6 MB/s | 2.3 MB 00:00 2024-04-04T06:03:09.941 INFO:teuthology.orchestra.run.smithi178.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 3.4 MB/s | 465 kB 00:00 2024-04-04T06:03:10.023 INFO:teuthology.orchestra.run.smithi178.stdout:(171/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 3.3 MB/s | 2.2 MB 00:00 2024-04-04T06:03:10.049 INFO:teuthology.orchestra.run.smithi178.stdout:(172/174): util-linux-user-2.37.4-18.el9.x86_64 197 kB/s | 31 kB 00:00 2024-04-04T06:03:10.099 INFO:teuthology.orchestra.run.smithi178.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 6.8 MB/s | 1.1 MB 00:00 2024-04-04T06:03:10.124 INFO:teuthology.orchestra.run.smithi178.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 747 kB/s | 74 kB 00:00 2024-04-04T06:03:10.132 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-04-04T06:03:10.133 INFO:teuthology.orchestra.run.smithi178.stdout:Total 13 MB/s | 226 MB 00:17 2024-04-04T06:03:11.268 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-04-04T06:03:11.642 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-04-04T06:03:11.642 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-04-04T06:03:12.053 INFO:teuthology.orchestra.run.smithi159.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 41 kB/s | 245 kB 00:06 2024-04-04T06:03:12.095 INFO:teuthology.orchestra.run.smithi159.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 85 kB/s | 248 kB 00:02 2024-04-04T06:03:12.312 INFO:teuthology.orchestra.run.smithi159.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 103 kB/s | 22 kB 00:00 2024-04-04T06:03:12.462 INFO:teuthology.orchestra.run.smithi159.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 5.6 MB/s | 863 kB 00:00 2024-04-04T06:03:13.738 INFO:teuthology.orchestra.run.smithi159.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 430 kB/s | 548 kB 00:01 2024-04-04T06:03:14.021 INFO:teuthology.orchestra.run.smithi159.stdout:(106/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 48 kB/s | 94 kB 00:01 2024-04-04T06:03:14.098 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-04-04T06:03:14.098 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-04-04T06:03:14.221 INFO:teuthology.orchestra.run.smithi159.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 639 kB/s | 309 kB 00:00 2024-04-04T06:03:14.605 INFO:teuthology.orchestra.run.smithi159.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 33 kB/s | 19 kB 00:00 2024-04-04T06:03:14.714 INFO:teuthology.orchestra.run.smithi159.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 241 kB/s | 26 kB 00:00 2024-04-04T06:03:14.906 INFO:teuthology.orchestra.run.smithi159.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 254 kB/s | 49 kB 00:00 2024-04-04T06:03:15.096 INFO:teuthology.orchestra.run.smithi159.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 356 kB/s | 67 kB 00:00 2024-04-04T06:03:15.371 INFO:teuthology.orchestra.run.smithi159.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 551 kB/s | 151 kB 00:00 2024-04-04T06:03:16.505 INFO:teuthology.orchestra.run.smithi159.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 740 kB/s | 838 kB 00:01 2024-04-04T06:03:16.689 INFO:teuthology.orchestra.run.smithi159.stdout:(114/174): openssl-devel-3.0.7-27.el9.x86_64.rp 397 kB/s | 4.1 MB 00:10 2024-04-04T06:03:17.297 INFO:teuthology.orchestra.run.smithi159.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 71 kB/s | 43 kB 00:00 2024-04-04T06:03:17.330 INFO:teuthology.orchestra.run.smithi159.stdout:(116/174): python3-asyncssh-2.13.2-5.el9.noarch 664 kB/s | 548 kB 00:00 2024-04-04T06:03:17.406 INFO:teuthology.orchestra.run.smithi159.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 299 kB/s | 32 kB 00:00 2024-04-04T06:03:17.440 INFO:teuthology.orchestra.run.smithi159.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-04-04T06:03:17.682 INFO:teuthology.orchestra.run.smithi159.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 625 kB/s | 172 kB 00:00 2024-04-04T06:03:17.974 INFO:teuthology.orchestra.run.smithi159.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 673 kB/s | 358 kB 00:00 2024-04-04T06:03:18.216 INFO:teuthology.orchestra.run.smithi159.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 146 kB/s | 35 kB 00:00 2024-04-04T06:03:18.249 INFO:teuthology.orchestra.run.smithi159.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 536 kB/s | 303 kB 00:00 2024-04-04T06:03:18.583 INFO:teuthology.orchestra.run.smithi159.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 554 kB/s | 203 kB 00:00 2024-04-04T06:03:18.867 INFO:teuthology.orchestra.run.smithi159.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 511 kB/s | 144 kB 00:00 2024-04-04T06:03:18.983 INFO:teuthology.orchestra.run.smithi159.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-04-04T06:03:19.100 INFO:teuthology.orchestra.run.smithi159.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 152 kB/s | 18 kB 00:00 2024-04-04T06:03:19.209 INFO:teuthology.orchestra.run.smithi159.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 215 kB/s | 23 kB 00:00 2024-04-04T06:03:19.368 INFO:teuthology.orchestra.run.smithi159.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 123 kB/s | 19 kB 00:00 2024-04-04T06:03:19.501 INFO:teuthology.orchestra.run.smithi159.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 148 kB/s | 20 kB 00:00 2024-04-04T06:03:20.135 INFO:teuthology.orchestra.run.smithi159.stdout:(130/174): libarrow-9.0.0-10.el9.x86_64.rpm 765 kB/s | 4.4 MB 00:05 2024-04-04T06:03:20.244 INFO:teuthology.orchestra.run.smithi159.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 432 kB/s | 46 kB 00:00 2024-04-04T06:03:20.444 INFO:teuthology.orchestra.run.smithi159.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 396 kB/s | 79 kB 00:00 2024-04-04T06:03:20.644 INFO:teuthology.orchestra.run.smithi159.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 289 kB/s | 58 kB 00:00 2024-04-04T06:03:20.937 INFO:teuthology.orchestra.run.smithi159.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 729 kB/s | 1.0 MB 00:01 2024-04-04T06:03:21.053 INFO:teuthology.orchestra.run.smithi159.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 142 kB/s | 16 kB 00:00 2024-04-04T06:03:21.111 INFO:teuthology.orchestra.run.smithi159.stdout:(136/174): python3-grpcio-1.46.7-10.el9.x86_64. 729 kB/s | 2.0 MB 00:02 2024-04-04T06:03:21.137 INFO:teuthology.orchestra.run.smithi159.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 554 kB/s | 272 kB 00:00 2024-04-04T06:03:21.221 INFO:teuthology.orchestra.run.smithi159.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-04-04T06:03:21.246 INFO:teuthology.orchestra.run.smithi159.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 285 kB/s | 31 kB 00:00 2024-04-04T06:03:21.279 INFO:teuthology.orchestra.run.smithi159.stdout:(140/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 399 kB/s | 90 kB 00:00 2024-04-04T06:03:21.388 INFO:teuthology.orchestra.run.smithi159.stdout:(141/174): python3-tempora-5.0.0-2.el9.noarch.r 331 kB/s | 36 kB 00:00 2024-04-04T06:03:21.438 INFO:teuthology.orchestra.run.smithi159.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 308 kB/s | 59 kB 00:00 2024-04-04T06:03:21.497 INFO:teuthology.orchestra.run.smithi159.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 473 kB/s | 51 kB 00:00 2024-04-04T06:03:21.522 INFO:teuthology.orchestra.run.smithi159.stdout:(144/174): python3-routes-2.5.1-5.el9.noarch.rp 627 kB/s | 188 kB 00:00 2024-04-04T06:03:21.706 INFO:teuthology.orchestra.run.smithi159.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 431 kB/s | 90 kB 00:00 2024-04-04T06:03:21.725 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-04-04T06:03:21.823 INFO:teuthology.orchestra.run.smithi159.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 598 kB/s | 230 kB 00:00 2024-04-04T06:03:21.849 INFO:teuthology.orchestra.run.smithi159.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 142 kB/s | 20 kB 00:00 2024-04-04T06:03:21.868 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-04T06:03:21.914 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-04T06:03:22.037 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-04T06:03:22.112 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-04T06:03:22.132 INFO:teuthology.orchestra.run.smithi159.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 679 kB/s | 192 kB 00:00 2024-04-04T06:03:22.165 INFO:teuthology.orchestra.run.smithi159.stdout:(149/174): python3-werkzeug-2.0.3-3.el9.1.noarc 665 kB/s | 427 kB 00:00 2024-04-04T06:03:22.312 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-04T06:03:22.426 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-04T06:03:22.486 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-04T06:03:22.525 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-04T06:03:22.612 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-04T06:03:22.616 INFO:teuthology.orchestra.run.smithi159.stdout:(150/174): librbd1-19.0.0-2571.gaddbc488.el9.x8 7.0 MB/s | 3.1 MB 00:00 2024-04-04T06:03:22.716 INFO:teuthology.orchestra.run.smithi159.stdout:(151/174): librados2-19.0.0-2571.gaddbc488.el9. 5.8 MB/s | 3.4 MB 00:00 2024-04-04T06:03:22.792 INFO:teuthology.orchestra.run.smithi159.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-04-04T06:03:22.867 INFO:teuthology.orchestra.run.smithi159.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 13 MB/s | 1.8 MB 00:00 2024-04-04T06:03:22.892 INFO:teuthology.orchestra.run.smithi159.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 477 kB/s | 41 kB 00:00 2024-04-04T06:03:22.926 INFO:teuthology.orchestra.run.smithi159.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 4.2 MB/s | 247 kB 00:00 2024-04-04T06:03:22.959 INFO:teuthology.orchestra.run.smithi159.stdout:(156/174): xfsdump-3.1.12-4.el9.x86_64.rpm 4.9 MB/s | 321 kB 00:00 2024-04-04T06:03:23.040 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-04T06:03:23.115 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librados2-2:19.0.0-2571.gaddbc488.el9.x86_64 10/198 2024-04-04T06:03:23.276 INFO:teuthology.orchestra.run.smithi159.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 115 kB/s | 36 kB 00:00 2024-04-04T06:03:23.278 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:19.0.0-2571.gaddbc488.el9.x86_64 10/198 2024-04-04T06:03:23.301 INFO:teuthology.orchestra.run.smithi159.stdout:(158/174): acl-2.3.1-4.el9.x86_64.rpm 192 kB/s | 71 kB 00:00 2024-04-04T06:03:23.313 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs2-2:19.0.0-2571.gaddbc488.el9.x86_64 11/198 2024-04-04T06:03:23.394 INFO:teuthology.orchestra.run.smithi159.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.6 MB/s | 192 kB 00:00 2024-04-04T06:03:23.429 INFO:teuthology.orchestra.run.smithi159.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.0 MB/s | 256 kB 00:00 2024-04-04T06:03:23.436 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs2-2:19.0.0-2571.gaddbc488.el9.x86_64 11/198 2024-04-04T06:03:23.471 INFO:teuthology.orchestra.run.smithi159.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 302 kB/s | 23 kB 00:00 2024-04-04T06:03:23.478 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rados-2:19.0.0-2571.gaddbc488.el9.x86_64 12/198 2024-04-04T06:03:23.505 INFO:teuthology.orchestra.run.smithi159.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 108 kB 00:00 2024-04-04T06:03:23.543 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-04T06:03:23.555 INFO:teuthology.orchestra.run.smithi159.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 155 kB 00:00 2024-04-04T06:03:23.589 INFO:teuthology.orchestra.run.smithi159.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 136 kB 00:00 2024-04-04T06:03:23.639 INFO:teuthology.orchestra.run.smithi159.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 767 kB/s | 64 kB 00:00 2024-04-04T06:03:23.664 INFO:teuthology.orchestra.run.smithi159.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 385 kB/s | 29 kB 00:00 2024-04-04T06:03:23.806 INFO:teuthology.orchestra.run.smithi159.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 7.3 MB/s | 1.2 MB 00:00 2024-04-04T06:03:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-04T06:03:23.865 INFO:teuthology.orchestra.run.smithi159.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 11 MB/s | 2.2 MB 00:00 2024-04-04T06:03:23.892 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-04T06:03:23.918 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephsqlite-2:19.0.0-2571.gaddbc488.el9.x86_64 16/198 2024-04-04T06:03:23.948 INFO:teuthology.orchestra.run.smithi159.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 5.5 MB/s | 465 kB 00:00 2024-04-04T06:03:24.016 INFO:teuthology.orchestra.run.smithi159.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 472 kB/s | 31 kB 00:00 2024-04-04T06:03:24.072 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2571.gaddbc488.el9.x86_64 16/198 2024-04-04T06:03:24.082 INFO:teuthology.orchestra.run.smithi159.stdout:(171/174): thrift-0.15.0-2.el9.x86_64.rpm 725 kB/s | 1.6 MB 00:02 2024-04-04T06:03:24.105 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libradosstriper1-2:19.0.0-2571.gaddbc488.el9.x86 17/198 2024-04-04T06:03:24.158 INFO:teuthology.orchestra.run.smithi159.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 6.5 MB/s | 2.3 MB 00:00 2024-04-04T06:03:24.195 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2571.gaddbc488.el9.x86 17/198 2024-04-04T06:03:24.208 INFO:teuthology.orchestra.run.smithi159.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 599 kB/s | 74 kB 00:00 2024-04-04T06:03:24.277 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-04T06:03:24.283 INFO:teuthology.orchestra.run.smithi159.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.0 MB/s | 1.1 MB 00:00 2024-04-04T06:03:24.300 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-04-04T06:03:24.301 INFO:teuthology.orchestra.run.smithi159.stdout:Total 7.1 MB/s | 226 MB 00:31 2024-04-04T06:03:24.315 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-04T06:03:24.386 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-04T06:03:24.682 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-04T06:03:24.732 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-04T06:03:24.781 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-04T06:03:25.091 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-04T06:03:25.289 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-04T06:03:25.377 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-04T06:03:25.441 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-04-04T06:03:25.559 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-04T06:03:25.625 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-04T06:03:25.829 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-04-04T06:03:25.829 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-04-04T06:03:25.972 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-04T06:03:26.150 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-04T06:03:26.273 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-04T06:03:26.322 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-04T06:03:26.350 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-04T06:03:26.407 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-04T06:03:26.518 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-04T06:03:26.580 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-04T06:03:26.664 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-04T06:03:26.762 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-04T06:03:26.811 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-04T06:03:26.874 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-04T06:03:26.924 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-04T06:03:26.956 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-04T06:03:27.070 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-04T06:03:27.118 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-common-2:19.0.0-2571.gaddbc488.el9. 39/198 2024-04-04T06:03:27.170 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-argparse-2:19.0.0-2571.gaddbc488.el 40/198 2024-04-04T06:03:27.200 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cephfs-2:19.0.0-2571.gaddbc488.el9.x86_6 41/198 2024-04-04T06:03:27.316 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-04T06:03:27.370 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-04T06:03:27.466 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-04T06:03:27.647 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-04T06:03:27.730 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-04T06:03:28.166 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-04-04T06:03:28.167 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-04-04T06:03:28.415 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-04T06:03:28.510 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-04T06:03:28.627 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-04T06:03:28.687 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-04T06:03:28.715 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-04T06:03:28.760 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-04T06:03:28.827 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-04T06:03:28.879 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librados-devel-2:19.0.0-2571.gaddbc488.el9.x86_6 53/198 2024-04-04T06:03:28.919 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-04T06:03:29.313 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-04T06:03:29.564 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-04T06:03:29.650 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-04T06:03:29.718 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-04T06:03:29.925 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-04T06:03:29.979 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-04T06:03:30.057 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-04T06:03:30.149 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-04T06:03:30.186 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-04T06:03:30.225 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-04T06:03:30.311 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-04T06:03:30.352 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-04T06:03:30.415 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-04T06:03:30.491 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-04T06:03:30.689 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-04T06:03:30.740 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-04T06:03:30.777 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-04T06:03:30.826 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-04T06:03:30.953 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-04T06:03:30.993 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-04T06:03:31.068 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-04T06:03:31.109 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-04T06:03:31.134 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-04T06:03:31.291 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-04T06:03:31.476 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-04T06:03:31.621 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-04T06:03:31.678 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-04T06:03:31.739 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-04T06:03:31.836 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-04T06:03:32.081 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-04T06:03:32.140 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-04T06:03:32.312 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-04T06:03:32.362 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-04T06:03:32.410 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-04T06:03:32.441 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-04T06:03:32.481 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-04T06:03:32.528 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-04T06:03:32.595 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-04T06:03:32.687 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-04T06:03:32.729 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-04T06:03:32.856 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-04T06:03:32.914 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-04T06:03:32.951 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-04T06:03:33.015 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-04T06:03:33.062 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-04T06:03:33.208 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-04T06:03:33.329 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-04T06:03:33.391 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-04T06:03:33.443 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-04T06:03:33.503 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-04T06:03:33.642 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-04T06:03:33.708 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-04T06:03:34.653 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-04T06:03:34.716 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-04T06:03:34.750 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-04T06:03:34.807 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-04T06:03:34.917 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-04T06:03:34.959 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-04T06:03:35.214 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-04T06:03:35.247 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-04T06:03:35.339 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-04T06:03:35.406 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-04-04T06:03:35.498 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-04T06:03:35.538 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-04T06:03:35.568 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-04T06:03:35.612 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-04T06:03:35.707 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-04T06:03:35.804 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-04T06:03:35.859 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-04T06:03:35.909 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-04T06:03:35.999 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-04T06:03:36.175 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-04T06:03:36.309 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-04T06:03:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-04T06:03:36.440 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librados2-2:19.0.0-2571.gaddbc488.el9.x86_64 10/198 2024-04-04T06:03:36.599 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:19.0.0-2571.gaddbc488.el9.x86_64 10/198 2024-04-04T06:03:36.633 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs2-2:19.0.0-2571.gaddbc488.el9.x86_64 11/198 2024-04-04T06:03:36.738 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephfs2-2:19.0.0-2571.gaddbc488.el9.x86_64 11/198 2024-04-04T06:03:36.773 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rados-2:19.0.0-2571.gaddbc488.el9.x86_64 12/198 2024-04-04T06:03:36.842 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-04T06:03:37.138 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-04T06:03:37.196 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-04T06:03:37.222 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephsqlite-2:19.0.0-2571.gaddbc488.el9.x86_64 16/198 2024-04-04T06:03:37.395 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2571.gaddbc488.el9.x86_64 16/198 2024-04-04T06:03:37.423 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libradosstriper1-2:19.0.0-2571.gaddbc488.el9.x86 17/198 2024-04-04T06:03:37.513 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2571.gaddbc488.el9.x86 17/198 2024-04-04T06:03:37.566 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-04T06:03:37.592 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-04T06:03:37.659 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-04T06:03:37.814 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-04T06:03:37.909 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-04T06:03:37.932 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-04T06:03:37.946 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-04T06:03:37.975 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-04T06:03:37.977 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-04T06:03:38.016 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-04T06:03:38.025 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-04T06:03:38.346 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-04T06:03:38.390 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-04T06:03:38.433 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librbd1-2:19.0.0-2571.gaddbc488.el9.x86_64 124/198 2024-04-04T06:03:38.528 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-04T06:03:38.536 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:19.0.0-2571.gaddbc488.el9.x86_64 124/198 2024-04-04T06:03:38.580 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rbd-2:19.0.0-2571.gaddbc488.el9.x86_64 125/198 2024-04-04T06:03:38.613 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-04T06:03:38.637 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-04T06:03:38.805 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-04T06:03:38.862 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-04T06:03:39.169 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-04T06:03:39.213 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-04T06:03:39.368 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librgw2-2:19.0.0-2571.gaddbc488.el9.x86_64 128/198 2024-04-04T06:03:39.403 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-04T06:03:39.472 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librgw2-2:19.0.0-2571.gaddbc488.el9.x86_64 128/198 2024-04-04T06:03:39.505 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rgw-2:19.0.0-2571.gaddbc488.el9.x86_64 129/198 2024-04-04T06:03:39.518 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-04T06:03:39.560 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-04T06:03:39.597 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-04T06:03:39.671 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-04T06:03:39.796 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-04T06:03:39.862 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-04T06:03:39.911 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-04T06:03:40.013 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-04T06:03:40.057 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-04T06:03:40.118 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-04T06:03:40.162 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-04T06:03:40.210 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-04T06:03:40.326 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-04T06:03:40.364 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-common-2:19.0.0-2571.gaddbc488.el9. 39/198 2024-04-04T06:03:40.413 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-argparse-2:19.0.0-2571.gaddbc488.el 40/198 2024-04-04T06:03:40.445 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cephfs-2:19.0.0-2571.gaddbc488.el9.x86_6 41/198 2024-04-04T06:03:40.542 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-04T06:03:40.597 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-04T06:03:40.690 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-04T06:03:40.887 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-04T06:03:40.951 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-04T06:03:41.588 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-04T06:03:41.673 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-04T06:03:41.811 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-04T06:03:41.867 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-04T06:03:41.896 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-04T06:03:41.947 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-04T06:03:42.017 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-04T06:03:42.069 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librados-devel-2:19.0.0-2571.gaddbc488.el9.x86_6 53/198 2024-04-04T06:03:42.116 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-04T06:03:42.141 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:19.0.0-2571.gaddbc488.el9.x86_64 130/198 2024-04-04T06:03:42.184 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-common-2:19.0.0-2571.gaddbc488.el9.x86_64 130/198 2024-04-04T06:03:42.545 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-04T06:03:42.727 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-04T06:03:42.796 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:19.0.0-2571.gaddbc488.el9.x86_64 130/198 2024-04-04T06:03:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-04T06:03:42.837 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-base-2:19.0.0-2571.gaddbc488.el9.x86_64 131/198 2024-04-04T06:03:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-04T06:03:43.040 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-base-2:19.0.0-2571.gaddbc488.el9.x86_64 131/198 2024-04-04T06:03:43.040 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-04T06:03:43.041 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-04T06:03:43.041 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:03:43.071 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-04T06:03:43.088 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-selinux-2:19.0.0-2571.gaddbc488.el9.x86_64 132/198 2024-04-04T06:03:43.126 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-04T06:03:43.218 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-04T06:03:43.314 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-04T06:03:43.359 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-04T06:03:43.398 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-04T06:03:43.466 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-04T06:03:43.507 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-04T06:03:43.568 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-04T06:03:43.626 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-04T06:03:43.818 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-04T06:03:43.871 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-04T06:03:43.908 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-04T06:03:43.958 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-04T06:03:44.097 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-04T06:03:44.133 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-04T06:03:44.178 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-04T06:03:44.215 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-04T06:03:44.248 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-04T06:03:44.407 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-04T06:03:44.577 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-04T06:03:44.722 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-04T06:03:44.777 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-04T06:03:44.839 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-04T06:03:44.923 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-04T06:03:45.172 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-04T06:03:45.232 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-04T06:03:45.408 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-04T06:03:45.453 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-04T06:03:45.493 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-04T06:03:45.541 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-04T06:03:45.582 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-04T06:03:45.630 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-04T06:03:45.697 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-04T06:03:46.139 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-04T06:03:46.191 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-04T06:03:46.329 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-04T06:03:46.366 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-04T06:03:46.409 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-04T06:03:46.474 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-04T06:03:46.513 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-04T06:03:46.656 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-04T06:03:46.771 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-04T06:03:46.824 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-04T06:03:46.860 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-04T06:03:46.917 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-04T06:03:47.055 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-04T06:03:47.118 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-04T06:03:48.072 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-04T06:03:48.125 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-04T06:03:48.159 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-04T06:03:48.215 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-04T06:03:48.329 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-04T06:03:48.368 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-04T06:03:48.600 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-04T06:03:48.623 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-04T06:03:48.709 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-04T06:03:49.512 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-04T06:03:49.652 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-04T06:03:51.182 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-04T06:03:51.278 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-04T06:03:51.315 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-04T06:03:51.455 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-04T06:03:51.506 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-04T06:03:51.909 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-04T06:03:51.943 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librbd1-2:19.0.0-2571.gaddbc488.el9.x86_64 124/198 2024-04-04T06:03:52.065 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:19.0.0-2571.gaddbc488.el9.x86_64 124/198 2024-04-04T06:03:52.100 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rbd-2:19.0.0-2571.gaddbc488.el9.x86_64 125/198 2024-04-04T06:03:52.159 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-04T06:03:52.710 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-04T06:03:52.746 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librgw2-2:19.0.0-2571.gaddbc488.el9.x86_64 128/198 2024-04-04T06:03:52.838 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librgw2-2:19.0.0-2571.gaddbc488.el9.x86_64 128/198 2024-04-04T06:03:52.865 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rgw-2:19.0.0-2571.gaddbc488.el9.x86_64 129/198 2024-04-04T06:03:55.453 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:19.0.0-2571.gaddbc488.el9.x86_64 130/198 2024-04-04T06:03:55.505 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-common-2:19.0.0-2571.gaddbc488.el9.x86_64 130/198 2024-04-04T06:03:56.155 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:19.0.0-2571.gaddbc488.el9.x86_64 130/198 2024-04-04T06:03:56.199 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-base-2:19.0.0-2571.gaddbc488.el9.x86_64 131/198 2024-04-04T06:03:56.214 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2571.gaddbc488.el9.x86_64 132/198 2024-04-04T06:03:56.214 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /sys 2024-04-04T06:03:56.214 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /proc 2024-04-04T06:03:56.214 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /mnt 2024-04-04T06:03:56.214 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /var/tmp 2024-04-04T06:03:56.215 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /home 2024-04-04T06:03:56.215 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /root 2024-04-04T06:03:56.215 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /tmp 2024-04-04T06:03:56.215 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:03:56.350 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-base-2:19.0.0-2571.gaddbc488.el9.x86_64 131/198 2024-04-04T06:03:56.350 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-04T06:03:56.350 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-04T06:03:56.351 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:03:56.392 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-selinux-2:19.0.0-2571.gaddbc488.el9.x86_64 132/198 2024-04-04T06:03:56.483 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-volume-2:19.0.0-2571.gaddbc488.el9.noarch 133/198 2024-04-04T06:03:56.543 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-volume-2:19.0.0-2571.gaddbc488.el9.noarch 133/198 2024-04-04T06:03:56.543 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-04T06:03:56.544 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-04T06:03:56.544 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:03:58.204 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-osd-2:19.0.0-2571.gaddbc488.el9.x86_64 134/198 2024-04-04T06:03:58.277 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-osd-2:19.0.0-2571.gaddbc488.el9.x86_64 134/198 2024-04-04T06:03:58.277 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-04T06:03:58.277 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-04T06:03:58.278 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-04T06:03:58.278 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-04T06:03:58.278 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:03:58.536 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mds-2:19.0.0-2571.gaddbc488.el9.x86_64 135/198 2024-04-04T06:03:58.601 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mds-2:19.0.0-2571.gaddbc488.el9.x86_64 135/198 2024-04-04T06:03:58.602 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-04T06:03:58.602 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-04T06:03:58.602 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-04T06:03:58.602 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-04T06:03:58.602 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:03:59.087 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mon-2:19.0.0-2571.gaddbc488.el9.x86_64 136/198 2024-04-04T06:03:59.154 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mon-2:19.0.0-2571.gaddbc488.el9.x86_64 136/198 2024-04-04T06:03:59.154 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-04T06:03:59.154 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-04T06:03:59.154 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-04T06:03:59.154 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-04T06:03:59.154 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:03:59.273 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-04T06:03:59.928 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:19.0.0-2571.gaddbc488.el9.noarch 138/198 2024-04-04T06:03:59.966 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephadm-2:19.0.0-2571.gaddbc488.el9.noarch 138/198 2024-04-04T06:04:00.043 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2571.gaddbc488.e 139/198 2024-04-04T06:04:00.155 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2571.gaddbc488. 140/198 2024-04-04T06:04:00.186 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2571.gaddbc488.el9.noa 141/198 2024-04-04T06:04:01.481 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2571.gaddbc488.el9.noa 141/198 2024-04-04T06:04:01.521 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2571.gaddbc488.el9.n 142/198 2024-04-04T06:04:02.600 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2571.gaddbc488.el9.n 142/198 2024-04-04T06:04:02.629 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2571.gadd 143/198 2024-04-04T06:04:02.770 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2571.gadd 143/198 2024-04-04T06:04:02.960 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2571.gaddbc488.el 144/198 2024-04-04T06:04:02.990 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-2:19.0.0-2571.gaddbc488.el9.x86_64 145/198 2024-04-04T06:04:03.056 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2571.gaddbc488.el9.x86_64 145/198 2024-04-04T06:04:03.056 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-04T06:04:03.057 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-04T06:04:03.057 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-04T06:04:03.057 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-04T06:04:03.057 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:03.115 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-rook-2:19.0.0-2571.gaddbc488.el9.noarch 146/198 2024-04-04T06:04:03.152 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2571.gaddbc488.el9.noarch 146/198 2024-04-04T06:04:03.241 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-2:19.0.0-2571.gaddbc488.el9.x86_64 147/198 2024-04-04T06:04:04.248 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-04T06:04:04.284 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-radosgw-2:19.0.0-2571.gaddbc488.el9.x86_64 149/198 2024-04-04T06:04:04.375 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2571.gaddbc488.el9.x86_64 149/198 2024-04-04T06:04:04.375 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-04T06:04:04.376 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-04T06:04:04.376 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-04T06:04:04.376 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-04T06:04:04.376 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:04.443 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2571.gaddbc 150/198 2024-04-04T06:04:04.538 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2571.gaddbc 150/198 2024-04-04T06:04:04.538 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-04T06:04:04.538 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-04T06:04:04.538 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:04.627 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-mirror-2:19.0.0-2571.gaddbc488.el9.x86_64 151/198 2024-04-04T06:04:04.706 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2571.gaddbc488.el9.x86_64 151/198 2024-04-04T06:04:04.706 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-04T06:04:04.706 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-04T06:04:04.706 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-04T06:04:04.707 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-04T06:04:04.707 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:05.118 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-mirror-2:19.0.0-2571.gaddbc488.el9.x86_64 152/198 2024-04-04T06:04:05.193 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2571.gaddbc488.el9.x86_64 152/198 2024-04-04T06:04:05.193 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-04T06:04:05.193 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-04T06:04:05.193 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-04T06:04:05.193 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-04T06:04:05.193 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:09.371 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2571.gaddbc488.el9.x86_64 132/198 2024-04-04T06:04:09.372 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /sys 2024-04-04T06:04:09.372 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /proc 2024-04-04T06:04:09.372 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /mnt 2024-04-04T06:04:09.372 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /var/tmp 2024-04-04T06:04:09.372 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /home 2024-04-04T06:04:09.372 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /root 2024-04-04T06:04:09.372 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /tmp 2024-04-04T06:04:09.372 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:09.606 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-volume-2:19.0.0-2571.gaddbc488.el9.noarch 133/198 2024-04-04T06:04:09.654 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-volume-2:19.0.0-2571.gaddbc488.el9.noarch 133/198 2024-04-04T06:04:09.654 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-04T06:04:09.654 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-04T06:04:09.654 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:10.009 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-test-2:19.0.0-2571.gaddbc488.el9.x86_64 153/198 2024-04-04T06:04:10.063 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-fuse-2:19.0.0-2571.gaddbc488.el9.x86_64 154/198 2024-04-04T06:04:10.098 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-nbd-2:19.0.0-2571.gaddbc488.el9.x86_64 155/198 2024-04-04T06:04:10.136 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-04T06:04:10.438 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-shell-2:19.0.0-2571.gaddbc488.el9.x86_64 157/198 2024-04-04T06:04:10.568 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-04T06:04:10.649 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-fuse-2:19.0.0-2571.gaddbc488.el9.x86_64 159/198 2024-04-04T06:04:10.699 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-04T06:04:10.766 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-04T06:04:10.810 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-04T06:04:10.846 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs-devel-2:19.0.0-2571.gaddbc488.el9.x86_ 163/198 2024-04-04T06:04:10.890 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-04T06:04:10.942 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-04-04T06:04:11.033 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-04-04T06:04:11.144 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-04T06:04:11.184 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-04T06:04:11.292 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-osd-2:19.0.0-2571.gaddbc488.el9.x86_64 134/198 2024-04-04T06:04:11.371 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-osd-2:19.0.0-2571.gaddbc488.el9.x86_64 134/198 2024-04-04T06:04:11.372 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-04T06:04:11.372 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-04T06:04:11.372 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-04T06:04:11.372 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-04T06:04:11.372 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:11.625 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mds-2:19.0.0-2571.gaddbc488.el9.x86_64 135/198 2024-04-04T06:04:11.700 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mds-2:19.0.0-2571.gaddbc488.el9.x86_64 135/198 2024-04-04T06:04:11.700 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-04T06:04:11.700 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-04T06:04:11.700 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-04T06:04:11.700 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-04T06:04:11.700 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:12.120 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-04T06:04:12.165 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-04T06:04:12.184 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mon-2:19.0.0-2571.gaddbc488.el9.x86_64 136/198 2024-04-04T06:04:12.219 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-04T06:04:12.254 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mon-2:19.0.0-2571.gaddbc488.el9.x86_64 136/198 2024-04-04T06:04:12.255 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-04T06:04:12.255 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-04T06:04:12.255 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-04T06:04:12.255 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-04T06:04:12.255 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:12.369 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-04T06:04:13.005 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephadm-2:19.0.0-2571.gaddbc488.el9.noarch 138/198 2024-04-04T06:04:13.037 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephadm-2:19.0.0-2571.gaddbc488.el9.noarch 138/198 2024-04-04T06:04:13.089 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2571.gaddbc488.e 139/198 2024-04-04T06:04:13.190 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2571.gaddbc488. 140/198 2024-04-04T06:04:13.216 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2571.gaddbc488.el9.noa 141/198 2024-04-04T06:04:13.549 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-top-2:19.0.0-2571.gaddbc488.el9.noarch 170/198 2024-04-04T06:04:13.636 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-04T06:04:13.689 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-04T06:04:13.809 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-04T06:04:13.847 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-04T06:04:13.880 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-04T06:04:13.914 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-04T06:04:13.955 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-04T06:04:13.955 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-04T06:04:14.092 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-04T06:04:14.162 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-04T06:04:14.162 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-04T06:04:14.306 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-04T06:04:14.351 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-04T06:04:14.396 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-04T06:04:14.430 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-04T06:04:14.462 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-04T06:04:14.525 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2571.gaddbc488.el9.noa 141/198 2024-04-04T06:04:14.531 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-04T06:04:14.576 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2571.gaddbc488.el9.n 142/198 2024-04-04T06:04:14.609 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-04T06:04:14.650 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-04T06:04:14.683 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-04T06:04:14.717 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-04T06:04:14.751 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-04T06:04:14.784 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-04T06:04:14.820 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-04T06:04:14.820 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-04T06:04:15.159 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-04T06:04:15.159 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-04T06:04:15.290 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-04T06:04:15.342 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-04T06:04:15.387 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-04T06:04:15.423 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-04T06:04:15.463 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-04T06:04:15.584 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-04T06:04:15.663 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2571.gaddbc488.el9.n 142/198 2024-04-04T06:04:15.716 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2571.gadd 143/198 2024-04-04T06:04:15.844 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-04T06:04:15.844 INFO:teuthology.orchestra.run.smithi178.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-04T06:04:15.844 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:15.863 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2571.gadd 143/198 2024-04-04T06:04:16.031 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2571.gaddbc488.el 144/198 2024-04-04T06:04:16.061 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-2:19.0.0-2571.gaddbc488.el9.x86_64 145/198 2024-04-04T06:04:16.133 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2571.gaddbc488.el9.x86_64 145/198 2024-04-04T06:04:16.134 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-04T06:04:16.134 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-04T06:04:16.134 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-04T06:04:16.134 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-04T06:04:16.134 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:16.188 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-rook-2:19.0.0-2571.gaddbc488.el9.noarch 146/198 2024-04-04T06:04:16.223 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2571.gaddbc488.el9.noarch 146/198 2024-04-04T06:04:16.281 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-2:19.0.0-2571.gaddbc488.el9.x86_64 147/198 2024-04-04T06:04:17.237 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-04T06:04:17.273 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-radosgw-2:19.0.0-2571.gaddbc488.el9.x86_64 149/198 2024-04-04T06:04:17.354 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2571.gaddbc488.el9.x86_64 149/198 2024-04-04T06:04:17.354 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-04T06:04:17.354 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-04T06:04:17.354 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-04T06:04:17.354 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-04T06:04:17.354 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:17.406 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2571.gaddbc 150/198 2024-04-04T06:04:17.476 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2571.gaddbc 150/198 2024-04-04T06:04:17.476 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-04T06:04:17.476 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-04T06:04:17.476 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:17.550 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-mirror-2:19.0.0-2571.gaddbc488.el9.x86_64 151/198 2024-04-04T06:04:17.626 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2571.gaddbc488.el9.x86_64 151/198 2024-04-04T06:04:17.626 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-04T06:04:17.627 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-04T06:04:17.627 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-04T06:04:17.627 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-04T06:04:17.627 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:17.966 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-mirror-2:19.0.0-2571.gaddbc488.el9.x86_64 152/198 2024-04-04T06:04:18.036 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2571.gaddbc488.el9.x86_64 152/198 2024-04-04T06:04:18.037 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-04T06:04:18.037 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-04T06:04:18.037 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-04T06:04:18.037 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-04T06:04:18.037 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:21.195 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-2:19.0.0-2571.gaddbc488.el9.x86_64 1/198 2024-04-04T06:04:21.195 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-base-2:19.0.0-2571.gaddbc488.el9.x86_64 2/198 2024-04-04T06:04:21.195 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-common-2:19.0.0-2571.gaddbc488.el9.x86_64 3/198 2024-04-04T06:04:21.195 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-fuse-2:19.0.0-2571.gaddbc488.el9.x86_64 4/198 2024-04-04T06:04:21.195 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2571.gaddbc 5/198 2024-04-04T06:04:21.195 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mds-2:19.0.0-2571.gaddbc488.el9.x86_64 6/198 2024-04-04T06:04:21.195 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-2:19.0.0-2571.gaddbc488.el9.x86_64 7/198 2024-04-04T06:04:21.195 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mon-2:19.0.0-2571.gaddbc488.el9.x86_64 8/198 2024-04-04T06:04:21.195 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-osd-2:19.0.0-2571.gaddbc488.el9.x86_64 9/198 2024-04-04T06:04:21.195 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-radosgw-2:19.0.0-2571.gaddbc488.el9.x86_64 10/198 2024-04-04T06:04:21.196 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-selinux-2:19.0.0-2571.gaddbc488.el9.x86_64 11/198 2024-04-04T06:04:21.196 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-test-2:19.0.0-2571.gaddbc488.el9.x86_64 12/198 2024-04-04T06:04:21.196 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-mirror-2:19.0.0-2571.gaddbc488.el9.x86_64 13/198 2024-04-04T06:04:21.196 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-shell-2:19.0.0-2571.gaddbc488.el9.x86_64 14/198 2024-04-04T06:04:21.196 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs-devel-2:19.0.0-2571.gaddbc488.el9.x86_ 15/198 2024-04-04T06:04:21.196 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs2-2:19.0.0-2571.gaddbc488.el9.x86_64 16/198 2024-04-04T06:04:21.196 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephsqlite-2:19.0.0-2571.gaddbc488.el9.x86_64 17/198 2024-04-04T06:04:21.196 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados-devel-2:19.0.0-2571.gaddbc488.el9.x86_6 18/198 2024-04-04T06:04:21.196 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libradosstriper1-2:19.0.0-2571.gaddbc488.el9.x86 19/198 2024-04-04T06:04:21.196 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librgw2-2:19.0.0-2571.gaddbc488.el9.x86_64 20/198 2024-04-04T06:04:21.196 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2571.gaddbc488.el 21/198 2024-04-04T06:04:21.196 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-common-2:19.0.0-2571.gaddbc488.el9. 22/198 2024-04-04T06:04:21.196 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cephfs-2:19.0.0-2571.gaddbc488.el9.x86_6 23/198 2024-04-04T06:04:21.197 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rados-2:19.0.0-2571.gaddbc488.el9.x86_64 24/198 2024-04-04T06:04:21.197 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rbd-2:19.0.0-2571.gaddbc488.el9.x86_64 25/198 2024-04-04T06:04:21.197 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rgw-2:19.0.0-2571.gaddbc488.el9.x86_64 26/198 2024-04-04T06:04:21.197 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-fuse-2:19.0.0-2571.gaddbc488.el9.x86_64 27/198 2024-04-04T06:04:21.197 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-mirror-2:19.0.0-2571.gaddbc488.el9.x86_64 28/198 2024-04-04T06:04:21.197 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-nbd-2:19.0.0-2571.gaddbc488.el9.x86_64 29/198 2024-04-04T06:04:21.197 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2571.gaddbc488. 30/198 2024-04-04T06:04:21.197 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2571.gaddbc488.el9.noa 31/198 2024-04-04T06:04:21.213 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2571.gaddbc488.el9.n 32/198 2024-04-04T06:04:21.213 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2571.gadd 33/198 2024-04-04T06:04:21.213 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2571.gaddbc488.el 34/198 2024-04-04T06:04:21.213 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2571.gaddbc488.el9.noarch 35/198 2024-04-04T06:04:21.213 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2571.gaddbc488.e 36/198 2024-04-04T06:04:21.213 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-volume-2:19.0.0-2571.gaddbc488.el9.noarch 37/198 2024-04-04T06:04:21.213 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephadm-2:19.0.0-2571.gaddbc488.el9.noarch 38/198 2024-04-04T06:04:21.213 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-top-2:19.0.0-2571.gaddbc488.el9.noarch 39/198 2024-04-04T06:04:21.213 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-04T06:04:21.213 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-04T06:04:21.213 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-04T06:04:21.213 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-04T06:04:21.214 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-04T06:04:21.214 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-04T06:04:21.214 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-04T06:04:21.214 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-04T06:04:21.214 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-04T06:04:21.214 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-04T06:04:21.214 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-04T06:04:21.214 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-04T06:04:21.214 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-04T06:04:21.214 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-04T06:04:21.214 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-04T06:04:21.215 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-04T06:04:21.215 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-04T06:04:21.215 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-04T06:04:21.215 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-04T06:04:21.215 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-04T06:04:21.215 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-04T06:04:21.215 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-04T06:04:21.215 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-04T06:04:21.215 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-04T06:04:21.215 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-04T06:04:21.215 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-04T06:04:21.215 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-04T06:04:21.215 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-04T06:04:21.216 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-04T06:04:21.216 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-04T06:04:21.216 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-04T06:04:21.216 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-04T06:04:21.216 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-04T06:04:21.216 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-04T06:04:21.216 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-04T06:04:21.216 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-04T06:04:21.216 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-04T06:04:21.216 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-04T06:04:21.216 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-04T06:04:21.216 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-04T06:04:21.216 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-04T06:04:21.217 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-04T06:04:21.217 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-04T06:04:21.217 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-04T06:04:21.217 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-04T06:04:21.217 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-04T06:04:21.217 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-04T06:04:21.217 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-04T06:04:21.217 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-04T06:04:21.217 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-04T06:04:21.217 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-04T06:04:21.217 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-04T06:04:21.217 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-04T06:04:21.217 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-04T06:04:21.217 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-04T06:04:21.217 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-04T06:04:21.218 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-04T06:04:21.218 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-04T06:04:21.218 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-04T06:04:21.218 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-04T06:04:21.218 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-04T06:04:21.218 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-04T06:04:21.218 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-04T06:04:21.218 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-04T06:04:21.218 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-04T06:04:21.218 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-04T06:04:21.218 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-04T06:04:21.218 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-04T06:04:21.218 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-04T06:04:21.219 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-04T06:04:21.219 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-04T06:04:21.219 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-04T06:04:21.219 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-04T06:04:21.219 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-04T06:04:21.219 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-04T06:04:21.219 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-04T06:04:21.219 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-04T06:04:21.219 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-04T06:04:21.219 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-04T06:04:21.219 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-04T06:04:21.219 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-04T06:04:21.219 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-04T06:04:21.219 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-04T06:04:21.220 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-04T06:04:21.220 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-04T06:04:21.220 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-04T06:04:21.220 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-04T06:04:21.220 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-04T06:04:21.220 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-04T06:04:21.220 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-04T06:04:21.220 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-04T06:04:21.220 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-04T06:04:21.220 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-04T06:04:21.220 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-04T06:04:21.220 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-04T06:04:21.220 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-04T06:04:21.220 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-04T06:04:21.221 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-04T06:04:21.221 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-04T06:04:21.221 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-04T06:04:21.221 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-04T06:04:21.221 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-04T06:04:21.221 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-04T06:04:21.221 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-04T06:04:21.221 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-04T06:04:21.221 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-04T06:04:21.221 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-04T06:04:21.221 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-04T06:04:21.221 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-04T06:04:21.221 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-04T06:04:21.221 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-04T06:04:21.222 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:19.0.0-2571.gaddbc488.el9.x86_64 151/198 2024-04-04T06:04:21.222 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-04T06:04:21.222 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:19.0.0-2571.gaddbc488.el9.x86_64 153/198 2024-04-04T06:04:21.222 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-04T06:04:21.222 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-04-04T06:04:21.222 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-04T06:04:21.222 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-04-04T06:04:21.222 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-04T06:04:21.222 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-04-04T06:04:21.222 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-04T06:04:21.222 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-04-04T06:04:21.222 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-04T06:04:21.222 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-04T06:04:21.222 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-04T06:04:21.223 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-04T06:04:21.223 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-04T06:04:21.223 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-04T06:04:21.223 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-04T06:04:21.223 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-04T06:04:21.223 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-04T06:04:21.223 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-04T06:04:21.223 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-04T06:04:21.223 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-04T06:04:21.223 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-04T06:04:21.223 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-04T06:04:21.223 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-04T06:04:21.223 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-04T06:04:21.223 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-04T06:04:21.224 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-04T06:04:21.224 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-04T06:04:21.224 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-04T06:04:21.224 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-04T06:04:21.224 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-04T06:04:21.224 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-04T06:04:21.224 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-04T06:04:21.224 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-04T06:04:21.224 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-04T06:04:21.224 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-04T06:04:21.224 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-04T06:04:21.224 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-04T06:04:21.224 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-04T06:04:21.224 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-04T06:04:21.225 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-04T06:04:21.225 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-04T06:04:21.225 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-04T06:04:21.225 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-04T06:04:21.225 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-04T06:04:22.865 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-test-2:19.0.0-2571.gaddbc488.el9.x86_64 153/198 2024-04-04T06:04:22.909 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-fuse-2:19.0.0-2571.gaddbc488.el9.x86_64 154/198 2024-04-04T06:04:22.945 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-nbd-2:19.0.0-2571.gaddbc488.el9.x86_64 155/198 2024-04-04T06:04:22.983 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-04T06:04:23.244 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-shell-2:19.0.0-2571.gaddbc488.el9.x86_64 157/198 2024-04-04T06:04:23.374 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-04T06:04:23.421 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-fuse-2:19.0.0-2571.gaddbc488.el9.x86_64 159/198 2024-04-04T06:04:23.475 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-04T06:04:23.546 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-04T06:04:23.583 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-04T06:04:23.626 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs-devel-2:19.0.0-2571.gaddbc488.el9.x86_ 163/198 2024-04-04T06:04:23.671 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-04T06:04:23.723 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-04-04T06:04:23.813 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-04-04T06:04:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-04T06:04:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:23.895 INFO:teuthology.orchestra.run.smithi178.stdout:Upgraded: 2024-04-04T06:04:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-04T06:04:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-04T06:04:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-04T06:04:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-04T06:04:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: acl-2.3.1-4.el9.x86_64 2024-04-04T06:04:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-04T06:04:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-04T06:04:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-04T06:04:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-04T06:04:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-04T06:04:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-04T06:04:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-04T06:04:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-04T06:04:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: librados2-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-04T06:04:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-04T06:04:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-04T06:04:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-04T06:04:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-04T06:04:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-04T06:04:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-04T06:04:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-04T06:04:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-04T06:04:23.897 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-04-04T06:04:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-04T06:04:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: automake-1.16.2-8.el9.noarch 2024-04-04T06:04:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: bison-3.7.4-5.el9.x86_64 2024-04-04T06:04:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-04T06:04:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards-2:19.0.0-2571.gaddbc488.el9.noarch 2024-04-04T06:04:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm-2:19.0.0-2571.gaddbc488.el9.noarch 2024-04-04T06:04:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard-2:19.0.0-2571.gaddbc488.el9.noarch 2024-04-04T06:04:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2571.gaddbc488.el9.noarch 2024-04-04T06:04:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core-2:19.0.0-2571.gaddbc488.el9.noarch 2024-04-04T06:04:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook-2:19.0.0-2571.gaddbc488.el9.noarch 2024-04-04T06:04:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts-2:19.0.0-2571.gaddbc488.el9.noarch 2024-04-04T06:04:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume-2:19.0.0-2571.gaddbc488.el9.noarch 2024-04-04T06:04:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm-2:19.0.0-2571.gaddbc488.el9.noarch 2024-04-04T06:04:23.899 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.899 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-shell-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.899 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top-2:19.0.0-2571.gaddbc488.el9.noarch 2024-04-04T06:04:23.899 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-04T06:04:23.899 INFO:teuthology.orchestra.run.smithi178.stdout: flex-2.6.4-9.el9.x86_64 2024-04-04T06:04:23.899 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-04T06:04:23.899 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-04T06:04:23.899 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-04T06:04:23.899 INFO:teuthology.orchestra.run.smithi178.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-04T06:04:23.899 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-04T06:04:23.899 INFO:teuthology.orchestra.run.smithi178.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-04T06:04:23.899 INFO:teuthology.orchestra.run.smithi178.stdout: jq-1.6-16.el9.x86_64 2024-04-04T06:04:23.900 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-04T06:04:23.900 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-04T06:04:23.900 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-04T06:04:23.900 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.900 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.900 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.900 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-04T06:04:23.900 INFO:teuthology.orchestra.run.smithi178.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-04T06:04:23.900 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-04T06:04:23.900 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-04T06:04:23.900 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-04T06:04:23.900 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.900 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.900 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-04T06:04:23.901 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.901 INFO:teuthology.orchestra.run.smithi178.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-04T06:04:23.901 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-04T06:04:23.901 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-04T06:04:23.901 INFO:teuthology.orchestra.run.smithi178.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-04T06:04:23.901 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-04T06:04:23.901 INFO:teuthology.orchestra.run.smithi178.stdout: lua-5.4.4-4.el9.x86_64 2024-04-04T06:04:23.901 INFO:teuthology.orchestra.run.smithi178.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-04T06:04:23.901 INFO:teuthology.orchestra.run.smithi178.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-04T06:04:23.901 INFO:teuthology.orchestra.run.smithi178.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-04T06:04:23.901 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-04T06:04:23.901 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-04T06:04:23.902 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-04T06:04:23.902 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-04T06:04:23.902 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-13.14-1.el9.x86_64 2024-04-04T06:04:23.902 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-04T06:04:23.902 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-04T06:04:23.902 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-04T06:04:23.902 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-04T06:04:23.902 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-04T06:04:23.902 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-04T06:04:23.902 INFO:teuthology.orchestra.run.smithi178.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-04T06:04:23.902 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-04T06:04:23.902 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-04T06:04:23.902 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-04T06:04:23.902 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.903 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.903 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.903 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-04T06:04:23.903 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-04T06:04:23.903 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-04T06:04:23.903 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-04T06:04:23.903 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-04T06:04:23.903 INFO:teuthology.orchestra.run.smithi178.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-04T06:04:23.903 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-04T06:04:23.903 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-04T06:04:23.903 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-04T06:04:23.903 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-04T06:04:23.904 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-04T06:04:23.904 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-04T06:04:23.904 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-04T06:04:23.904 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-04T06:04:23.904 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-04T06:04:23.904 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-04T06:04:23.904 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-04T06:04:23.904 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-04T06:04:23.904 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-04T06:04:23.904 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-04T06:04:23.904 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-04T06:04:23.904 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-04T06:04:23.904 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-04T06:04:23.905 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-04T06:04:23.905 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-04T06:04:23.905 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-04T06:04:23.905 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-04T06:04:23.905 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-04T06:04:23.905 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-04T06:04:23.905 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-04T06:04:23.905 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply-3.11-14.el9.noarch 2024-04-04T06:04:23.905 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-04T06:04:23.905 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-04T06:04:23.905 INFO:teuthology.orchestra.run.smithi178.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-04T06:04:23.905 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-04T06:04:23.905 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-04T06:04:23.905 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-04T06:04:23.906 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-04T06:04:23.906 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-04T06:04:23.906 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-04T06:04:23.906 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.906 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.906 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-04T06:04:23.906 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-04T06:04:23.906 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-04T06:04:23.906 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.906 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-04T06:04:23.906 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-04T06:04:23.906 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-04T06:04:23.906 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-04T06:04:23.906 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-04T06:04:23.906 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-04T06:04:23.907 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-04T06:04:23.907 INFO:teuthology.orchestra.run.smithi178.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-04T06:04:23.907 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-04T06:04:23.907 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-04T06:04:23.907 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-04T06:04:23.907 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-04T06:04:23.907 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-04T06:04:23.907 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-04T06:04:23.907 INFO:teuthology.orchestra.run.smithi178.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-04T06:04:23.907 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.907 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.907 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:23.907 INFO:teuthology.orchestra.run.smithi178.stdout: re2-1:20211101-3.el9.x86_64 2024-04-04T06:04:23.907 INFO:teuthology.orchestra.run.smithi178.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-04T06:04:23.908 INFO:teuthology.orchestra.run.smithi178.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-04T06:04:23.908 INFO:teuthology.orchestra.run.smithi178.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-04T06:04:23.908 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-04T06:04:23.908 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-04T06:04:23.908 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-04T06:04:23.908 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:23.908 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-04-04T06:04:23.931 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-04T06:04:23.972 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-04T06:04:24.569 DEBUG:teuthology.parallel:result is None 2024-04-04T06:04:24.920 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-04T06:04:24.972 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-04T06:04:25.026 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-04T06:04:26.373 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-top-2:19.0.0-2571.gaddbc488.el9.noarch 170/198 2024-04-04T06:04:26.469 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-04T06:04:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-04T06:04:26.666 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-04T06:04:26.705 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-04T06:04:26.738 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-04T06:04:26.772 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-04T06:04:26.813 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-04T06:04:26.814 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-04T06:04:26.944 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-04T06:04:26.985 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-04T06:04:26.985 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-04T06:04:27.106 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-04T06:04:27.152 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-04T06:04:27.195 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-04T06:04:27.230 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-04T06:04:27.262 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-04T06:04:27.333 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-04T06:04:27.402 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-04T06:04:27.450 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-04T06:04:27.483 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-04T06:04:27.517 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-04T06:04:27.550 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-04T06:04:27.584 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-04T06:04:27.620 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-04T06:04:27.620 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-04T06:04:27.935 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-04T06:04:27.935 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-04T06:04:28.073 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-04T06:04:28.124 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-04T06:04:28.153 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-04T06:04:28.190 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-04T06:04:28.238 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-04T06:04:28.357 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-04T06:04:28.609 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-04T06:04:28.609 INFO:teuthology.orchestra.run.smithi159.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-04T06:04:28.609 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:33.850 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-2:19.0.0-2571.gaddbc488.el9.x86_64 1/198 2024-04-04T06:04:33.850 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-base-2:19.0.0-2571.gaddbc488.el9.x86_64 2/198 2024-04-04T06:04:33.850 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-common-2:19.0.0-2571.gaddbc488.el9.x86_64 3/198 2024-04-04T06:04:33.850 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-fuse-2:19.0.0-2571.gaddbc488.el9.x86_64 4/198 2024-04-04T06:04:33.850 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2571.gaddbc 5/198 2024-04-04T06:04:33.850 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mds-2:19.0.0-2571.gaddbc488.el9.x86_64 6/198 2024-04-04T06:04:33.850 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-2:19.0.0-2571.gaddbc488.el9.x86_64 7/198 2024-04-04T06:04:33.850 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mon-2:19.0.0-2571.gaddbc488.el9.x86_64 8/198 2024-04-04T06:04:33.850 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-osd-2:19.0.0-2571.gaddbc488.el9.x86_64 9/198 2024-04-04T06:04:33.850 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-radosgw-2:19.0.0-2571.gaddbc488.el9.x86_64 10/198 2024-04-04T06:04:33.850 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-selinux-2:19.0.0-2571.gaddbc488.el9.x86_64 11/198 2024-04-04T06:04:33.851 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-test-2:19.0.0-2571.gaddbc488.el9.x86_64 12/198 2024-04-04T06:04:33.851 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-mirror-2:19.0.0-2571.gaddbc488.el9.x86_64 13/198 2024-04-04T06:04:33.851 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-shell-2:19.0.0-2571.gaddbc488.el9.x86_64 14/198 2024-04-04T06:04:33.851 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs-devel-2:19.0.0-2571.gaddbc488.el9.x86_ 15/198 2024-04-04T06:04:33.851 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs2-2:19.0.0-2571.gaddbc488.el9.x86_64 16/198 2024-04-04T06:04:33.851 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephsqlite-2:19.0.0-2571.gaddbc488.el9.x86_64 17/198 2024-04-04T06:04:33.851 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados-devel-2:19.0.0-2571.gaddbc488.el9.x86_6 18/198 2024-04-04T06:04:33.851 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libradosstriper1-2:19.0.0-2571.gaddbc488.el9.x86 19/198 2024-04-04T06:04:33.851 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librgw2-2:19.0.0-2571.gaddbc488.el9.x86_64 20/198 2024-04-04T06:04:33.851 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2571.gaddbc488.el 21/198 2024-04-04T06:04:33.851 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-common-2:19.0.0-2571.gaddbc488.el9. 22/198 2024-04-04T06:04:33.851 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cephfs-2:19.0.0-2571.gaddbc488.el9.x86_6 23/198 2024-04-04T06:04:33.851 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rados-2:19.0.0-2571.gaddbc488.el9.x86_64 24/198 2024-04-04T06:04:33.851 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rbd-2:19.0.0-2571.gaddbc488.el9.x86_64 25/198 2024-04-04T06:04:33.868 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rgw-2:19.0.0-2571.gaddbc488.el9.x86_64 26/198 2024-04-04T06:04:33.868 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-fuse-2:19.0.0-2571.gaddbc488.el9.x86_64 27/198 2024-04-04T06:04:33.868 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-mirror-2:19.0.0-2571.gaddbc488.el9.x86_64 28/198 2024-04-04T06:04:33.868 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-nbd-2:19.0.0-2571.gaddbc488.el9.x86_64 29/198 2024-04-04T06:04:33.868 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2571.gaddbc488. 30/198 2024-04-04T06:04:33.868 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2571.gaddbc488.el9.noa 31/198 2024-04-04T06:04:33.868 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2571.gaddbc488.el9.n 32/198 2024-04-04T06:04:33.868 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2571.gadd 33/198 2024-04-04T06:04:33.868 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2571.gaddbc488.el 34/198 2024-04-04T06:04:33.868 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2571.gaddbc488.el9.noarch 35/198 2024-04-04T06:04:33.868 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2571.gaddbc488.e 36/198 2024-04-04T06:04:33.868 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-volume-2:19.0.0-2571.gaddbc488.el9.noarch 37/198 2024-04-04T06:04:33.869 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephadm-2:19.0.0-2571.gaddbc488.el9.noarch 38/198 2024-04-04T06:04:33.869 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-top-2:19.0.0-2571.gaddbc488.el9.noarch 39/198 2024-04-04T06:04:33.869 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-04T06:04:33.869 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-04T06:04:33.869 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-04T06:04:33.869 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-04T06:04:33.869 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-04T06:04:33.869 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-04T06:04:33.869 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-04T06:04:33.869 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-04T06:04:33.869 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-04T06:04:33.869 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-04T06:04:33.869 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-04T06:04:33.869 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-04T06:04:33.870 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-04T06:04:33.870 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-04T06:04:33.870 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-04T06:04:33.870 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-04T06:04:33.870 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-04T06:04:33.870 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-04T06:04:33.870 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-04T06:04:33.870 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-04T06:04:33.870 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-04T06:04:33.870 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-04T06:04:33.870 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-04T06:04:33.870 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-04T06:04:33.870 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-04T06:04:33.870 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-04T06:04:33.871 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-04T06:04:33.871 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-04T06:04:33.871 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-04T06:04:33.871 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-04T06:04:33.871 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-04T06:04:33.871 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-04T06:04:33.871 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-04T06:04:33.871 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-04T06:04:33.871 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-04T06:04:33.871 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-04T06:04:33.871 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-04T06:04:33.871 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-04T06:04:33.871 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-04T06:04:33.871 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-04T06:04:33.872 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-04T06:04:33.872 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-04T06:04:33.872 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-04T06:04:33.872 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-04T06:04:33.872 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-04T06:04:33.872 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-04T06:04:33.872 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-04T06:04:33.872 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-04T06:04:33.872 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-04T06:04:33.872 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-04T06:04:33.872 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-04T06:04:33.872 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-04T06:04:33.872 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-04T06:04:33.872 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-04T06:04:33.872 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-04T06:04:33.873 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-04T06:04:33.873 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-04T06:04:33.873 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-04T06:04:33.873 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-04T06:04:33.873 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-04T06:04:33.873 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-04T06:04:33.873 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-04T06:04:33.873 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-04T06:04:33.873 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-04T06:04:33.873 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-04T06:04:33.873 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-04T06:04:33.873 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-04T06:04:33.873 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-04T06:04:33.873 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-04T06:04:33.873 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-04T06:04:33.874 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-04T06:04:33.874 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-04T06:04:33.874 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-04T06:04:33.874 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-04T06:04:33.874 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-04T06:04:33.874 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-04T06:04:33.874 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-04T06:04:33.874 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-04T06:04:33.874 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-04T06:04:33.874 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-04T06:04:33.874 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-04T06:04:33.874 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-04T06:04:33.874 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-04T06:04:33.875 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-04T06:04:33.875 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-04T06:04:33.875 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-04T06:04:33.875 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-04T06:04:33.875 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-04T06:04:33.875 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-04T06:04:33.875 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-04T06:04:33.875 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-04T06:04:33.875 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-04T06:04:33.875 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-04T06:04:33.875 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-04T06:04:33.875 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-04T06:04:33.875 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-04T06:04:33.875 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-04T06:04:33.876 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-04T06:04:33.876 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-04T06:04:33.876 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-04T06:04:33.876 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-04T06:04:33.876 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-04T06:04:33.876 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-04T06:04:33.876 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-04T06:04:33.876 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-04T06:04:33.876 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-04T06:04:33.876 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-04T06:04:33.876 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-04T06:04:33.876 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-04T06:04:33.876 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-04T06:04:33.876 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-04T06:04:33.876 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:19.0.0-2571.gaddbc488.el9.x86_64 151/198 2024-04-04T06:04:33.877 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-04T06:04:33.877 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:19.0.0-2571.gaddbc488.el9.x86_64 153/198 2024-04-04T06:04:33.877 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-04T06:04:33.877 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-04-04T06:04:33.877 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-04T06:04:33.877 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-04-04T06:04:33.877 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-04T06:04:33.877 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-04-04T06:04:33.877 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-04T06:04:33.877 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-04-04T06:04:33.877 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-04T06:04:33.877 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-04T06:04:33.877 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-04T06:04:33.877 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-04T06:04:33.878 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-04T06:04:33.878 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-04T06:04:33.878 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-04T06:04:33.878 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-04T06:04:33.878 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-04T06:04:33.878 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-04T06:04:33.878 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-04T06:04:33.878 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-04T06:04:33.878 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-04T06:04:33.878 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-04T06:04:33.878 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-04T06:04:33.878 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-04T06:04:33.878 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-04T06:04:33.878 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-04T06:04:33.879 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-04T06:04:33.879 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-04T06:04:33.879 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-04T06:04:33.879 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-04T06:04:33.879 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-04T06:04:33.879 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-04T06:04:33.879 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-04T06:04:33.879 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-04T06:04:33.879 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-04T06:04:33.879 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-04T06:04:33.879 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-04T06:04:33.879 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-04T06:04:33.879 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-04T06:04:33.879 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-04T06:04:33.879 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-04T06:04:33.880 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-04T06:04:33.880 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-04T06:04:33.880 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-04T06:04:36.406 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-04T06:04:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:36.409 INFO:teuthology.orchestra.run.smithi159.stdout:Upgraded: 2024-04-04T06:04:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-04T06:04:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-04T06:04:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-04T06:04:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-04T06:04:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: acl-2.3.1-4.el9.x86_64 2024-04-04T06:04:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-04T06:04:36.409 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-04T06:04:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-04T06:04:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-04T06:04:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-04T06:04:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-04T06:04:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-04T06:04:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-04T06:04:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: librados2-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-04T06:04:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-04T06:04:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-04T06:04:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-04T06:04:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-04T06:04:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-04T06:04:36.410 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-04T06:04:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-04T06:04:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-04T06:04:36.411 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-04-04T06:04:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-04T06:04:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: automake-1.16.2-8.el9.noarch 2024-04-04T06:04:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: bison-3.7.4-5.el9.x86_64 2024-04-04T06:04:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-04T06:04:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards-2:19.0.0-2571.gaddbc488.el9.noarch 2024-04-04T06:04:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.411 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.412 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.412 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm-2:19.0.0-2571.gaddbc488.el9.noarch 2024-04-04T06:04:36.412 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard-2:19.0.0-2571.gaddbc488.el9.noarch 2024-04-04T06:04:36.412 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2571.gaddbc488.el9.noarch 2024-04-04T06:04:36.412 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core-2:19.0.0-2571.gaddbc488.el9.noarch 2024-04-04T06:04:36.412 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook-2:19.0.0-2571.gaddbc488.el9.noarch 2024-04-04T06:04:36.412 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.412 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.412 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts-2:19.0.0-2571.gaddbc488.el9.noarch 2024-04-04T06:04:36.412 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.412 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.412 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.412 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume-2:19.0.0-2571.gaddbc488.el9.noarch 2024-04-04T06:04:36.412 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm-2:19.0.0-2571.gaddbc488.el9.noarch 2024-04-04T06:04:36.412 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.413 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-shell-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.413 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top-2:19.0.0-2571.gaddbc488.el9.noarch 2024-04-04T06:04:36.413 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-04T06:04:36.413 INFO:teuthology.orchestra.run.smithi159.stdout: flex-2.6.4-9.el9.x86_64 2024-04-04T06:04:36.413 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-04T06:04:36.413 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-04T06:04:36.413 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-04T06:04:36.413 INFO:teuthology.orchestra.run.smithi159.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-04T06:04:36.413 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-04T06:04:36.413 INFO:teuthology.orchestra.run.smithi159.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-04T06:04:36.413 INFO:teuthology.orchestra.run.smithi159.stdout: jq-1.6-16.el9.x86_64 2024-04-04T06:04:36.413 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-04T06:04:36.414 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-04T06:04:36.414 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-04T06:04:36.414 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.414 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.414 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.414 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-04T06:04:36.414 INFO:teuthology.orchestra.run.smithi159.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-04T06:04:36.414 INFO:teuthology.orchestra.run.smithi159.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-04T06:04:36.414 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-04T06:04:36.414 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-04T06:04:36.414 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.414 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.414 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-04T06:04:36.414 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.415 INFO:teuthology.orchestra.run.smithi159.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-04T06:04:36.415 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-04T06:04:36.415 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-04T06:04:36.415 INFO:teuthology.orchestra.run.smithi159.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-04T06:04:36.415 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-04T06:04:36.415 INFO:teuthology.orchestra.run.smithi159.stdout: lua-5.4.4-4.el9.x86_64 2024-04-04T06:04:36.415 INFO:teuthology.orchestra.run.smithi159.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-04T06:04:36.415 INFO:teuthology.orchestra.run.smithi159.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-04T06:04:36.415 INFO:teuthology.orchestra.run.smithi159.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-04T06:04:36.415 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-04T06:04:36.415 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-04T06:04:36.415 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-04T06:04:36.415 INFO:teuthology.orchestra.run.smithi159.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-04T06:04:36.415 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-13.14-1.el9.x86_64 2024-04-04T06:04:36.416 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-04T06:04:36.416 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-04T06:04:36.416 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-04T06:04:36.416 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-04T06:04:36.416 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-04T06:04:36.416 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-04T06:04:36.416 INFO:teuthology.orchestra.run.smithi159.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-04T06:04:36.416 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-04T06:04:36.416 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-04T06:04:36.416 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-04T06:04:36.416 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.416 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.416 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.416 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-04T06:04:36.417 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-04T06:04:36.417 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-04T06:04:36.417 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-04T06:04:36.417 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-04T06:04:36.417 INFO:teuthology.orchestra.run.smithi159.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-04T06:04:36.417 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-04T06:04:36.417 INFO:teuthology.orchestra.run.smithi159.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-04T06:04:36.417 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-04T06:04:36.417 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-04T06:04:36.417 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-04T06:04:36.417 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-04T06:04:36.417 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-04T06:04:36.417 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-04T06:04:36.417 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-04T06:04:36.417 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-04T06:04:36.418 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-04T06:04:36.418 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-04T06:04:36.418 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-04T06:04:36.418 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-04T06:04:36.418 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-04T06:04:36.418 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-04T06:04:36.418 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-04T06:04:36.418 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-04T06:04:36.418 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-04T06:04:36.418 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-04T06:04:36.418 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-04T06:04:36.418 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-04T06:04:36.418 INFO:teuthology.orchestra.run.smithi159.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-04T06:04:36.418 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-04T06:04:36.418 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ply-3.11-14.el9.noarch 2024-04-04T06:04:36.419 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-04T06:04:36.419 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-04T06:04:36.419 INFO:teuthology.orchestra.run.smithi159.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-04T06:04:36.419 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-04T06:04:36.419 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-04T06:04:36.419 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-04T06:04:36.419 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-04T06:04:36.419 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-04T06:04:36.419 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-04T06:04:36.419 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.419 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.419 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-04T06:04:36.419 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-04T06:04:36.419 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-04T06:04:36.419 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.420 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-04T06:04:36.420 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-04T06:04:36.420 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-04T06:04:36.420 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-04T06:04:36.420 INFO:teuthology.orchestra.run.smithi159.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-04T06:04:36.420 INFO:teuthology.orchestra.run.smithi159.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-04T06:04:36.420 INFO:teuthology.orchestra.run.smithi159.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-04T06:04:36.420 INFO:teuthology.orchestra.run.smithi159.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-04T06:04:36.420 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-04T06:04:36.420 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-04T06:04:36.420 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-04T06:04:36.420 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-04T06:04:36.420 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-04T06:04:36.420 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-04T06:04:36.421 INFO:teuthology.orchestra.run.smithi159.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-04T06:04:36.421 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.421 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.421 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd-2:19.0.0-2571.gaddbc488.el9.x86_64 2024-04-04T06:04:36.421 INFO:teuthology.orchestra.run.smithi159.stdout: re2-1:20211101-3.el9.x86_64 2024-04-04T06:04:36.421 INFO:teuthology.orchestra.run.smithi159.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-04T06:04:36.421 INFO:teuthology.orchestra.run.smithi159.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-04T06:04:36.421 INFO:teuthology.orchestra.run.smithi159.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-04T06:04:36.421 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-04T06:04:36.421 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-04T06:04:36.421 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-04T06:04:36.422 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:36.422 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-04-04T06:04:37.127 DEBUG:teuthology.parallel:result is None 2024-04-04T06:04:37.128 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=addbc488dc763e752c640619b677f10e0da02c9f 2024-04-04T06:04:37.348 DEBUG:teuthology.orchestra.run.smithi159:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-04T06:04:37.407 INFO:teuthology.orchestra.run.smithi159.stdout:19.0.0-2571.gaddbc488.el9 2024-04-04T06:04:37.408 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2571.gaddbc488.el9 2024-04-04T06:04:37.408 INFO:teuthology.task.install:The correct ceph version 19.0.0-2571.gaddbc488 is installed. 2024-04-04T06:04:37.410 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=addbc488dc763e752c640619b677f10e0da02c9f 2024-04-04T06:04:37.625 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-04T06:04:37.696 INFO:teuthology.orchestra.run.smithi178.stdout:19.0.0-2571.gaddbc488.el9 2024-04-04T06:04:37.697 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2571.gaddbc488.el9 2024-04-04T06:04:37.697 INFO:teuthology.task.install:The correct ceph version 19.0.0-2571.gaddbc488 is installed. 2024-04-04T06:04:37.699 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-04T06:04:37.699 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:04:37.699 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-04T06:04:37.777 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:04:37.777 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-04T06:04:37.875 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-04T06:04:37.875 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:04:37.875 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/daemon-helper 2024-04-04T06:04:37.954 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-04T06:04:38.068 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:04:38.068 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/daemon-helper 2024-04-04T06:04:38.169 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-04T06:04:38.306 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-04T06:04:38.306 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:04:38.307 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-04T06:04:38.376 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-04T06:04:38.490 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:04:38.490 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-04T06:04:38.588 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-04T06:04:38.727 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-04T06:04:38.727 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:04:38.727 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/stdin-killer 2024-04-04T06:04:38.807 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-04T06:04:38.917 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:04:38.917 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/stdin-killer 2024-04-04T06:04:39.015 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-04T06:04:39.106 INFO:teuthology.run_tasks:Running task ceph... 2024-04-04T06:04:39.182 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-04T06:04:39.182 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 777 /var/log/ceph 2024-04-04T06:04:39.187 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 777 /var/log/ceph 2024-04-04T06:04:39.279 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-04T06:04:39.279 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-04T06:04:39.283 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-04T06:04:39.379 INFO:tasks.ceph:Creating extra log directories... 2024-04-04T06:04:39.379 DEBUG:teuthology.orchestra.run.smithi159:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-04T06:04:39.383 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-04T06:04:39.484 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-04T06:04:39.484 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-04T06:04:39.484 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-04T06:04:39.484 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:04:39.484 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-04T06:04:39.522 DEBUG:teuthology.orchestra.run.smithi159:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-04T06:04:39.748 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-04T06:04:39.823 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:04:39.823 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-04T06:04:39.878 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-04T06:04:40.148 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-04T06:04:40.251 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-04T06:04:40.252 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-04T06:04:40.252 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2024-04-04_05:15:36-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi/7639210', 'branch': 'wip-vshankar-testing-20240330.172700', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7639210', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'vshankar-2024-04-04_05:15:36-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20240330.172700'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'addbc488dc763e752c640619b677f10e0da02c9f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'addbc488dc763e752c640619b677f10e0da02c9f'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20240330.172700', 'sha1': 'addbc488dc763e752c640619b677f10e0da02c9f'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3016, 'sha1': 'addbc488dc763e752c640619b677f10e0da02c9f', 'sleep_before_teardown': 0, 'subset': '140/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20240330.172700', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_addbc488dc763e752c640619b677f10e0da02c9f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'addbc488dc763e752c640619b677f10e0da02c9f', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi159.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI9+sH1zuQF/EKfdRkBUOY6pUkyed9HhnOnkKrOygkBuTyZsrgsl2SJcEyeid/oNbaQwHgtR9HSF5IzgfoYOAgM=', 'smithi178.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI75ZPTr++WrvmTssgpUy8yMeTqSpAYeKCfe+sxpb8CHITh35dNVY8L6Ku/faFCk6orumF6zBTI9iJ1DATOBKxI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'addbc488dc763e752c640619b677f10e0da02c9f'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'addbc488dc763e752c640619b677f10e0da02c9f', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'd7d6e60867cadf19fbfd7655a551ae7d58060757', 'timestamp': '2024-04-04_05:15:36', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-04T06:04:40.252 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-04T06:04:40.257 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-04T06:04:40.312 DEBUG:teuthology.orchestra.run.smithi159:> sudo install -d -m0777 -- /var/run/ceph 2024-04-04T06:04:40.316 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m0777 -- /var/run/ceph 2024-04-04T06:04:40.408 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:04:40.408 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/scratch_devs of=/dev/stdout 2024-04-04T06:04:40.460 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-04T06:04:40.460 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_1 2024-04-04T06:04:40.504 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-04T06:04:40.505 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-04T06:04:40.505 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 631 Links: 1 2024-04-04T06:04:40.505 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-04T06:04:40.505 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-04-04T06:04:40.505 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-04-04 06:04:29.055458619 +0000 2024-04-04T06:04:40.505 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-04-04 06:01:42.364762069 +0000 2024-04-04T06:04:40.505 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-04-04 06:01:42.364762069 +0000 2024-04-04T06:04:40.505 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-04-04 06:01:42.364762069 +0000 2024-04-04T06:04:40.505 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-04T06:04:40.639 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-04-04T06:04:40.639 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-04-04T06:04:40.639 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000434112 s, 1.2 MB/s 2024-04-04T06:04:40.644 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-04T06:04:40.699 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_2 2024-04-04T06:04:40.741 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-04T06:04:40.741 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-04T06:04:40.741 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-04-04T06:04:40.741 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-04T06:04:40.741 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-04-04T06:04:40.741 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-04-04 06:04:29.056458600 +0000 2024-04-04T06:04:40.741 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-04-04 06:01:42.942751002 +0000 2024-04-04T06:04:40.741 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-04-04 06:01:42.942751002 +0000 2024-04-04T06:04:40.741 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-04-04 06:01:42.942751002 +0000 2024-04-04T06:04:40.742 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-04T06:04:40.867 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-04-04T06:04:40.867 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-04-04T06:04:40.868 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000358699 s, 1.4 MB/s 2024-04-04T06:04:40.871 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-04T06:04:40.924 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_3 2024-04-04T06:04:40.973 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-04T06:04:40.973 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-04T06:04:40.973 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 671 Links: 1 2024-04-04T06:04:40.973 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-04T06:04:40.973 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-04-04T06:04:40.973 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-04-04 06:04:29.057458581 +0000 2024-04-04T06:04:40.973 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-04-04 06:01:43.523739878 +0000 2024-04-04T06:04:40.973 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-04-04 06:01:43.523739878 +0000 2024-04-04T06:04:40.973 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-04-04 06:01:43.523739878 +0000 2024-04-04T06:04:40.973 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-04T06:04:41.057 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-04-04T06:04:41.057 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-04-04T06:04:41.057 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000325385 s, 1.6 MB/s 2024-04-04T06:04:41.061 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-04T06:04:41.116 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_4 2024-04-04T06:04:41.162 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-04T06:04:41.162 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-04T06:04:41.162 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 688 Links: 1 2024-04-04T06:04:41.162 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-04T06:04:41.162 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-04-04T06:04:41.162 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-04-04 06:04:29.058458561 +0000 2024-04-04T06:04:41.162 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-04-04 06:01:44.065729500 +0000 2024-04-04T06:04:41.162 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-04-04 06:01:44.065729500 +0000 2024-04-04T06:04:41.162 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-04-04 06:01:44.065729500 +0000 2024-04-04T06:04:41.163 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-04T06:04:41.252 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-04-04T06:04:41.252 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-04-04T06:04:41.252 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000343024 s, 1.5 MB/s 2024-04-04T06:04:41.256 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-04T06:04:41.312 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-04T06:04:41.312 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:04:41.312 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/scratch_devs of=/dev/stdout 2024-04-04T06:04:41.371 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-04T06:04:41.371 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_1 2024-04-04T06:04:41.423 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-04T06:04:41.423 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-04T06:04:41.424 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 631 Links: 1 2024-04-04T06:04:41.424 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-04T06:04:41.424 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-04-04T06:04:41.424 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-04-04 06:04:16.303750509 +0000 2024-04-04T06:04:41.424 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-04-04 06:01:49.439996318 +0000 2024-04-04T06:04:41.424 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-04-04 06:01:49.439996318 +0000 2024-04-04T06:04:41.424 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-04-04 06:01:49.439996318 +0000 2024-04-04T06:04:41.424 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-04T06:04:41.515 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-04-04T06:04:41.516 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-04-04T06:04:41.516 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000252377 s, 2.0 MB/s 2024-04-04T06:04:41.520 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-04T06:04:41.575 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_2 2024-04-04T06:04:41.622 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-04T06:04:41.623 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-04T06:04:41.623 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-04-04T06:04:41.623 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-04T06:04:41.623 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-04-04T06:04:41.623 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-04-04 06:04:16.303750509 +0000 2024-04-04T06:04:41.623 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-04-04 06:01:50.010985999 +0000 2024-04-04T06:04:41.623 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-04-04 06:01:50.010985999 +0000 2024-04-04T06:04:41.623 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-04-04 06:01:50.010985999 +0000 2024-04-04T06:04:41.623 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-04T06:04:41.709 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-04-04T06:04:41.710 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-04-04T06:04:41.710 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000315344 s, 1.6 MB/s 2024-04-04T06:04:41.714 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-04T06:04:41.771 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_3 2024-04-04T06:04:41.819 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-04T06:04:41.820 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-04T06:04:41.820 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-04-04T06:04:41.820 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-04T06:04:41.820 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-04-04T06:04:41.820 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-04-04 06:04:16.304750491 +0000 2024-04-04T06:04:41.820 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-04-04 06:01:50.541976403 +0000 2024-04-04T06:04:41.820 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-04-04 06:01:50.541976403 +0000 2024-04-04T06:04:41.820 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-04-04 06:01:50.541976403 +0000 2024-04-04T06:04:41.820 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-04T06:04:41.908 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-04-04T06:04:41.908 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-04-04T06:04:41.908 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000245388 s, 2.1 MB/s 2024-04-04T06:04:41.911 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-04T06:04:41.965 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_4 2024-04-04T06:04:42.012 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-04T06:04:42.012 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-04T06:04:42.012 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 688 Links: 1 2024-04-04T06:04:42.012 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-04T06:04:42.012 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-04-04T06:04:42.012 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-04-04 06:04:16.305750473 +0000 2024-04-04T06:04:42.012 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-04-04 06:01:51.111966103 +0000 2024-04-04T06:04:42.012 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-04-04 06:01:51.111966103 +0000 2024-04-04T06:04:42.012 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-04-04 06:01:51.111966103 +0000 2024-04-04T06:04:42.012 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-04T06:04:42.088 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-04-04T06:04:42.088 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-04-04T06:04:42.089 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.0002151 s, 2.4 MB/s 2024-04-04T06:04:42.092 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-04T06:04:42.144 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-04T06:04:42.144 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi159.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi178.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-04T06:04:42.144 INFO:tasks.ceph:Generating config... 2024-04-04T06:04:42.145 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-04T06:04:42.145 INFO:tasks.ceph:[client] debug client = 20 2024-04-04T06:04:42.146 INFO:tasks.ceph:[client] debug ms = 1 2024-04-04T06:04:42.146 INFO:tasks.ceph:[client] pid file = 2024-04-04T06:04:42.146 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-04T06:04:42.146 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-04T06:04:42.146 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-04T06:04:42.146 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-04T06:04:42.146 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-04T06:04:42.146 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-04T06:04:42.146 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-04T06:04:42.146 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-04T06:04:42.146 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-04T06:04:42.146 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-04T06:04:42.146 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-04T06:04:42.146 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-04T06:04:42.146 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-04T06:04:42.147 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-04T06:04:42.147 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-04T06:04:42.147 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-04T06:04:42.147 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-04T06:04:42.147 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-04T06:04:42.147 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-04T06:04:42.147 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-04T06:04:42.147 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-04T06:04:42.147 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-04T06:04:42.147 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-04T06:04:42.147 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-04T06:04:42.147 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-04T06:04:42.147 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-04T06:04:42.147 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-04T06:04:42.147 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-04T06:04:42.148 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-04T06:04:42.148 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-04T06:04:42.148 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-04T06:04:42.148 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-04T06:04:42.148 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-04T06:04:42.148 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-04T06:04:42.148 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-04T06:04:42.148 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-04T06:04:42.148 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-04T06:04:42.148 INFO:tasks.ceph:Setting up mon.a... 2024-04-04T06:04:42.148 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-04T06:04:42.292 INFO:teuthology.orchestra.run.smithi159.stdout:creating /etc/ceph/ceph.keyring 2024-04-04T06:04:42.301 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-04T06:04:42.435 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-04T06:04:42.524 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.159'), ('mon.b', '172.21.15.178'), ('mon.c', '[v2:172.21.15.178:3301,v1:172.21.15.178:6790]')] 2024-04-04T06:04:42.525 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.159,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-04T06:04:42.525 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:04:42.526 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-04T06:04:42.577 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.159 --add b 172.21.15.178 --addv c '[v2:172.21.15.178:3301,v1:172.21.15.178:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-04T06:04:42.728 INFO:teuthology.orchestra.run.smithi159.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-04T06:04:42.728 INFO:teuthology.orchestra.run.smithi159.stdout:monmaptool: generated fsid 30311045-f714-48fa-832b-4cb2ba8bf0bf 2024-04-04T06:04:42.729 INFO:teuthology.orchestra.run.smithi159.stdout:setting min_mon_release = quincy 2024-04-04T06:04:42.729 INFO:teuthology.orchestra.run.smithi159.stdout:epoch 0 2024-04-04T06:04:42.729 INFO:teuthology.orchestra.run.smithi159.stdout:fsid 30311045-f714-48fa-832b-4cb2ba8bf0bf 2024-04-04T06:04:42.729 INFO:teuthology.orchestra.run.smithi159.stdout:last_changed 2024-04-04T06:04:42.734791+0000 2024-04-04T06:04:42.729 INFO:teuthology.orchestra.run.smithi159.stdout:created 2024-04-04T06:04:42.734791+0000 2024-04-04T06:04:42.729 INFO:teuthology.orchestra.run.smithi159.stdout:min_mon_release 17 (quincy) 2024-04-04T06:04:42.729 INFO:teuthology.orchestra.run.smithi159.stdout:election_strategy: 1 2024-04-04T06:04:42.729 INFO:teuthology.orchestra.run.smithi159.stdout:0: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.a 2024-04-04T06:04:42.729 INFO:teuthology.orchestra.run.smithi159.stdout:1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.b 2024-04-04T06:04:42.729 INFO:teuthology.orchestra.run.smithi159.stdout:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-04-04T06:04:42.729 INFO:teuthology.orchestra.run.smithi159.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-04T06:04:42.732 DEBUG:teuthology.orchestra.run.smithi159:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-04T06:04:42.779 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 30311045-f714-48fa-832b-4cb2ba8bf0bf... 2024-04-04T06:04:42.780 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-04T06:04:42.787 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-04T06:04:42.946 INFO:teuthology.orchestra.run.smithi159.stdout:[global] 2024-04-04T06:04:42.947 INFO:teuthology.orchestra.run.smithi159.stdout: chdir = "" 2024-04-04T06:04:42.947 INFO:teuthology.orchestra.run.smithi159.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-04T06:04:42.947 INFO:teuthology.orchestra.run.smithi159.stdout: auth supported = cephx 2024-04-04T06:04:42.947 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.947 INFO:teuthology.orchestra.run.smithi159.stdout: filestore xattr use omap = true 2024-04-04T06:04:42.947 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.947 INFO:teuthology.orchestra.run.smithi159.stdout: mon clock drift allowed = 1.000 2024-04-04T06:04:42.947 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.947 INFO:teuthology.orchestra.run.smithi159.stdout: osd crush chooseleaf type = 0 2024-04-04T06:04:42.947 INFO:teuthology.orchestra.run.smithi159.stdout: auth debug = true 2024-04-04T06:04:42.947 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.947 INFO:teuthology.orchestra.run.smithi159.stdout: ms die on old message = true 2024-04-04T06:04:42.947 INFO:teuthology.orchestra.run.smithi159.stdout: ms die on bug = true 2024-04-04T06:04:42.948 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.948 INFO:teuthology.orchestra.run.smithi159.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-04T06:04:42.948 INFO:teuthology.orchestra.run.smithi159.stdout: mon pg warn max object skew = 0 2024-04-04T06:04:42.948 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.948 INFO:teuthology.orchestra.run.smithi159.stdout: # disable pg_autoscaler by default for new pools 2024-04-04T06:04:42.948 INFO:teuthology.orchestra.run.smithi159.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-04T06:04:42.948 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.948 INFO:teuthology.orchestra.run.smithi159.stdout: osd pool default size = 2 2024-04-04T06:04:42.948 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.948 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd allow primary affinity = true 2024-04-04T06:04:42.948 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd allow pg remap = true 2024-04-04T06:04:42.948 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on legacy crush tunables = false 2024-04-04T06:04:42.948 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on crush straw calc version zero = false 2024-04-04T06:04:42.948 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on no sortbitwise = false 2024-04-04T06:04:42.949 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on osd down out interval zero = false 2024-04-04T06:04:42.949 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on too few osds = false 2024-04-04T06:04:42.949 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-04T06:04:42.949 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-04T06:04:42.949 INFO:teuthology.orchestra.run.smithi159.stdout: mon_allow_pool_size_one = true 2024-04-04T06:04:42.949 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.949 INFO:teuthology.orchestra.run.smithi159.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-04T06:04:42.949 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.949 INFO:teuthology.orchestra.run.smithi159.stdout: osd default data pool replay window = 5 2024-04-04T06:04:42.949 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.949 INFO:teuthology.orchestra.run.smithi159.stdout: mon allow pool delete = true 2024-04-04T06:04:42.949 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.949 INFO:teuthology.orchestra.run.smithi159.stdout: mon cluster log file level = debug 2024-04-04T06:04:42.949 INFO:teuthology.orchestra.run.smithi159.stdout: debug asserts on shutdown = true 2024-04-04T06:04:42.949 INFO:teuthology.orchestra.run.smithi159.stdout: mon health detail to clog = false 2024-04-04T06:04:42.949 INFO:teuthology.orchestra.run.smithi159.stdout: mon host = "172.21.15.159,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-04-04T06:04:42.950 INFO:teuthology.orchestra.run.smithi159.stdout: ms die on skipped message = False 2024-04-04T06:04:42.950 INFO:teuthology.orchestra.run.smithi159.stdout: fsid = 30311045-f714-48fa-832b-4cb2ba8bf0bf 2024-04-04T06:04:42.950 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.950 INFO:teuthology.orchestra.run.smithi159.stdout:[osd] 2024-04-04T06:04:42.950 INFO:teuthology.orchestra.run.smithi159.stdout: osd journal size = 100 2024-04-04T06:04:42.950 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.950 INFO:teuthology.orchestra.run.smithi159.stdout: osd scrub load threshold = 5.0 2024-04-04T06:04:42.950 INFO:teuthology.orchestra.run.smithi159.stdout: osd scrub max interval = 600 2024-04-04T06:04:42.950 INFO:teuthology.orchestra.run.smithi159.stdout: osd mclock profile = high_recovery_ops 2024-04-04T06:04:42.950 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.950 INFO:teuthology.orchestra.run.smithi159.stdout: osd recover clone overlap = true 2024-04-04T06:04:42.950 INFO:teuthology.orchestra.run.smithi159.stdout: osd recovery max chunk = 1048576 2024-04-04T06:04:42.950 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.950 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug shutdown = true 2024-04-04T06:04:42.951 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug op order = true 2024-04-04T06:04:42.951 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug verify stray on activate = true 2024-04-04T06:04:42.951 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug trim objects = true 2024-04-04T06:04:42.951 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.951 INFO:teuthology.orchestra.run.smithi159.stdout: osd open classes on start = true 2024-04-04T06:04:42.951 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug pg log writeout = true 2024-04-04T06:04:42.951 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.951 INFO:teuthology.orchestra.run.smithi159.stdout: osd deep scrub update digest min age = 30 2024-04-04T06:04:42.951 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.951 INFO:teuthology.orchestra.run.smithi159.stdout: osd map max advance = 10 2024-04-04T06:04:42.951 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.951 INFO:teuthology.orchestra.run.smithi159.stdout: journal zero on create = true 2024-04-04T06:04:42.951 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.951 INFO:teuthology.orchestra.run.smithi159.stdout: filestore ondisk finisher threads = 3 2024-04-04T06:04:42.951 INFO:teuthology.orchestra.run.smithi159.stdout: filestore apply finisher threads = 3 2024-04-04T06:04:42.951 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.952 INFO:teuthology.orchestra.run.smithi159.stdout: bdev debug aio = true 2024-04-04T06:04:42.952 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug misdirected ops = true 2024-04-04T06:04:42.952 INFO:teuthology.orchestra.run.smithi159.stdout: bdev async discard = True 2024-04-04T06:04:42.952 INFO:teuthology.orchestra.run.smithi159.stdout: bdev enable discard = True 2024-04-04T06:04:42.952 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore allocator = bitmap 2024-04-04T06:04:42.952 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore block size = 96636764160 2024-04-04T06:04:42.952 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore fsck on mount = True 2024-04-04T06:04:42.952 INFO:teuthology.orchestra.run.smithi159.stdout: debug bluefs = 1/20 2024-04-04T06:04:42.952 INFO:teuthology.orchestra.run.smithi159.stdout: debug bluestore = 1/20 2024-04-04T06:04:42.952 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:04:42.952 INFO:teuthology.orchestra.run.smithi159.stdout: debug osd = 20 2024-04-04T06:04:42.952 INFO:teuthology.orchestra.run.smithi159.stdout: debug rocksdb = 4/10 2024-04-04T06:04:42.952 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd backfillfull_ratio = 0.85 2024-04-04T06:04:42.952 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd full ratio = 0.9 2024-04-04T06:04:42.952 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd nearfull ratio = 0.8 2024-04-04T06:04:42.952 INFO:teuthology.orchestra.run.smithi159.stdout: osd failsafe full ratio = 0.95 2024-04-04T06:04:42.953 INFO:teuthology.orchestra.run.smithi159.stdout: osd objectstore = bluestore 2024-04-04T06:04:42.953 INFO:teuthology.orchestra.run.smithi159.stdout: osd op complaint time = 180 2024-04-04T06:04:42.953 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.953 INFO:teuthology.orchestra.run.smithi159.stdout:[mgr] 2024-04-04T06:04:42.953 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:04:42.953 INFO:teuthology.orchestra.run.smithi159.stdout: debug mgr = 20 2024-04-04T06:04:42.953 INFO:teuthology.orchestra.run.smithi159.stdout: debug mon = 20 2024-04-04T06:04:42.953 INFO:teuthology.orchestra.run.smithi159.stdout: debug auth = 20 2024-04-04T06:04:42.953 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min pgs per osd = 4 2024-04-04T06:04:42.953 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min bytes per osd = 10 2024-04-04T06:04:42.953 INFO:teuthology.orchestra.run.smithi159.stdout: mgr/telemetry/nag = false 2024-04-04T06:04:42.953 INFO:teuthology.orchestra.run.smithi159.stdout: debug client = 20 2024-04-04T06:04:42.953 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.953 INFO:teuthology.orchestra.run.smithi159.stdout:[mon] 2024-04-04T06:04:42.953 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:04:42.953 INFO:teuthology.orchestra.run.smithi159.stdout: debug mon = 20 2024-04-04T06:04:42.953 INFO:teuthology.orchestra.run.smithi159.stdout: debug paxos = 20 2024-04-04T06:04:42.954 INFO:teuthology.orchestra.run.smithi159.stdout: debug auth = 20 2024-04-04T06:04:42.954 INFO:teuthology.orchestra.run.smithi159.stdout: mon data avail warn = 5 2024-04-04T06:04:42.954 INFO:teuthology.orchestra.run.smithi159.stdout: mon mgr mkfs grace = 240 2024-04-04T06:04:42.954 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min pgs per osd = 4 2024-04-04T06:04:42.954 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd reporter subtree level = osd 2024-04-04T06:04:42.954 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd prime pg temp = true 2024-04-04T06:04:42.954 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min bytes per osd = 10 2024-04-04T06:04:42.954 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.954 INFO:teuthology.orchestra.run.smithi159.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-04T06:04:42.954 INFO:teuthology.orchestra.run.smithi159.stdout: auth mon ticket ttl = 660 # 11m 2024-04-04T06:04:42.954 INFO:teuthology.orchestra.run.smithi159.stdout: auth service ticket ttl = 240 # 4m 2024-04-04T06:04:42.954 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.954 INFO:teuthology.orchestra.run.smithi159.stdout: # don't complain about insecure global_id in the test suite 2024-04-04T06:04:42.955 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-04T06:04:42.955 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-04T06:04:42.955 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.955 INFO:teuthology.orchestra.run.smithi159.stdout: # 1m isn't quite enough 2024-04-04T06:04:42.955 INFO:teuthology.orchestra.run.smithi159.stdout: mon_down_mkfs_grace = 2m 2024-04-04T06:04:42.955 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.955 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_filestore_osds = false 2024-04-04T06:04:42.955 INFO:teuthology.orchestra.run.smithi159.stdout: mon down mkfs grace = 300 2024-04-04T06:04:42.955 INFO:teuthology.orchestra.run.smithi159.stdout: mon op complaint time = 120 2024-04-04T06:04:42.955 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:04:42.955 INFO:teuthology.orchestra.run.smithi159.stdout:[client] 2024-04-04T06:04:42.955 INFO:teuthology.orchestra.run.smithi159.stdout: rgw cache enabled = true 2024-04-04T06:04:42.955 INFO:teuthology.orchestra.run.smithi159.stdout: rgw enable ops log = true 2024-04-04T06:04:42.956 INFO:teuthology.orchestra.run.smithi159.stdout: rgw enable usage log = true 2024-04-04T06:04:42.956 INFO:teuthology.orchestra.run.smithi159.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-04T06:04:42.956 INFO:teuthology.orchestra.run.smithi159.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-04T06:04:42.956 INFO:teuthology.orchestra.run.smithi159.stdout: client mount timeout = 600 2024-04-04T06:04:42.956 INFO:teuthology.orchestra.run.smithi159.stdout: debug client = 20 2024-04-04T06:04:42.956 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:04:42.956 INFO:teuthology.orchestra.run.smithi159.stdout: pid file = "" 2024-04-04T06:04:42.956 INFO:teuthology.orchestra.run.smithi159.stdout: rados mon op timeout = 900 2024-04-04T06:04:42.956 INFO:teuthology.orchestra.run.smithi159.stdout: rados osd op timeout = 900 2024-04-04T06:04:42.956 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.a] 2024-04-04T06:04:42.956 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.b] 2024-04-04T06:04:42.956 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.c] 2024-04-04T06:04:42.956 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.a] 2024-04-04T06:04:42.956 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.c] 2024-04-04T06:04:42.956 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.b] 2024-04-04T06:04:42.956 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.d] 2024-04-04T06:04:42.957 INFO:teuthology.orchestra.run.smithi159.stdout:[mds] 2024-04-04T06:04:42.957 INFO:teuthology.orchestra.run.smithi159.stdout: debug mds = 20 2024-04-04T06:04:42.957 INFO:teuthology.orchestra.run.smithi159.stdout: debug mds balancer = 20 2024-04-04T06:04:42.957 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:04:42.957 INFO:teuthology.orchestra.run.smithi159.stdout: mds debug frag = True 2024-04-04T06:04:42.957 INFO:teuthology.orchestra.run.smithi159.stdout: mds debug scatterstat = True 2024-04-04T06:04:42.957 INFO:teuthology.orchestra.run.smithi159.stdout: mds op complaint time = 180 2024-04-04T06:04:42.957 INFO:teuthology.orchestra.run.smithi159.stdout: mds verify scatter = True 2024-04-04T06:04:42.957 INFO:teuthology.orchestra.run.smithi159.stdout: osd op complaint time = 180 2024-04-04T06:04:42.957 INFO:teuthology.orchestra.run.smithi159.stdout: rados mon op timeout = 900 2024-04-04T06:04:42.957 INFO:teuthology.orchestra.run.smithi159.stdout: rados osd op timeout = 900 2024-04-04T06:04:42.960 INFO:teuthology.orchestra.run.smithi178.stdout:[global] 2024-04-04T06:04:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: chdir = "" 2024-04-04T06:04:42.960 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-04T06:04:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: auth supported = cephx 2024-04-04T06:04:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: filestore xattr use omap = true 2024-04-04T06:04:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: mon clock drift allowed = 1.000 2024-04-04T06:04:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: osd crush chooseleaf type = 0 2024-04-04T06:04:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: auth debug = true 2024-04-04T06:04:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on old message = true 2024-04-04T06:04:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on bug = true 2024-04-04T06:04:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-04T06:04:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: mon pg warn max object skew = 0 2024-04-04T06:04:42.961 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: # disable pg_autoscaler by default for new pools 2024-04-04T06:04:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-04T06:04:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default size = 2 2024-04-04T06:04:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow primary affinity = true 2024-04-04T06:04:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow pg remap = true 2024-04-04T06:04:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on legacy crush tunables = false 2024-04-04T06:04:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on crush straw calc version zero = false 2024-04-04T06:04:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on no sortbitwise = false 2024-04-04T06:04:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on osd down out interval zero = false 2024-04-04T06:04:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on too few osds = false 2024-04-04T06:04:42.962 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-04T06:04:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-04T06:04:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: mon_allow_pool_size_one = true 2024-04-04T06:04:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-04T06:04:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: osd default data pool replay window = 5 2024-04-04T06:04:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: mon allow pool delete = true 2024-04-04T06:04:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: mon cluster log file level = debug 2024-04-04T06:04:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: debug asserts on shutdown = true 2024-04-04T06:04:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: mon health detail to clog = false 2024-04-04T06:04:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: mon host = "172.21.15.159,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-04-04T06:04:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on skipped message = False 2024-04-04T06:04:42.963 INFO:teuthology.orchestra.run.smithi178.stdout: fsid = 30311045-f714-48fa-832b-4cb2ba8bf0bf 2024-04-04T06:04:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.964 INFO:teuthology.orchestra.run.smithi178.stdout:[osd] 2024-04-04T06:04:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: osd journal size = 100 2024-04-04T06:04:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub load threshold = 5.0 2024-04-04T06:04:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub max interval = 600 2024-04-04T06:04:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: osd mclock profile = high_recovery_ops 2024-04-04T06:04:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: osd recover clone overlap = true 2024-04-04T06:04:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: osd recovery max chunk = 1048576 2024-04-04T06:04:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug shutdown = true 2024-04-04T06:04:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug op order = true 2024-04-04T06:04:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify stray on activate = true 2024-04-04T06:04:42.964 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug trim objects = true 2024-04-04T06:04:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: osd open classes on start = true 2024-04-04T06:04:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug pg log writeout = true 2024-04-04T06:04:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: osd deep scrub update digest min age = 30 2024-04-04T06:04:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: osd map max advance = 10 2024-04-04T06:04:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: journal zero on create = true 2024-04-04T06:04:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: filestore ondisk finisher threads = 3 2024-04-04T06:04:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: filestore apply finisher threads = 3 2024-04-04T06:04:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.965 INFO:teuthology.orchestra.run.smithi178.stdout: bdev debug aio = true 2024-04-04T06:04:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug misdirected ops = true 2024-04-04T06:04:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: bdev async discard = True 2024-04-04T06:04:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: bdev enable discard = True 2024-04-04T06:04:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore allocator = bitmap 2024-04-04T06:04:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore block size = 96636764160 2024-04-04T06:04:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore fsck on mount = True 2024-04-04T06:04:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluefs = 1/20 2024-04-04T06:04:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluestore = 1/20 2024-04-04T06:04:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:04:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: debug osd = 20 2024-04-04T06:04:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: debug rocksdb = 4/10 2024-04-04T06:04:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd backfillfull_ratio = 0.85 2024-04-04T06:04:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd full ratio = 0.9 2024-04-04T06:04:42.966 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd nearfull ratio = 0.8 2024-04-04T06:04:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: osd failsafe full ratio = 0.95 2024-04-04T06:04:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: osd objectstore = bluestore 2024-04-04T06:04:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-04-04T06:04:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.967 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr] 2024-04-04T06:04:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:04:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: debug mgr = 20 2024-04-04T06:04:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-04T06:04:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-04T06:04:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-04T06:04:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-04T06:04:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: mgr/telemetry/nag = false 2024-04-04T06:04:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-04-04T06:04:42.967 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.968 INFO:teuthology.orchestra.run.smithi178.stdout:[mon] 2024-04-04T06:04:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:04:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-04T06:04:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: debug paxos = 20 2024-04-04T06:04:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-04T06:04:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: mon data avail warn = 5 2024-04-04T06:04:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: mon mgr mkfs grace = 240 2024-04-04T06:04:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-04T06:04:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd reporter subtree level = osd 2024-04-04T06:04:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd prime pg temp = true 2024-04-04T06:04:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-04T06:04:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-04T06:04:42.968 INFO:teuthology.orchestra.run.smithi178.stdout: auth mon ticket ttl = 660 # 11m 2024-04-04T06:04:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: auth service ticket ttl = 240 # 4m 2024-04-04T06:04:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: # don't complain about insecure global_id in the test suite 2024-04-04T06:04:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-04T06:04:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-04T06:04:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: # 1m isn't quite enough 2024-04-04T06:04:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: mon_down_mkfs_grace = 2m 2024-04-04T06:04:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_filestore_osds = false 2024-04-04T06:04:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: mon down mkfs grace = 300 2024-04-04T06:04:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: mon op complaint time = 120 2024-04-04T06:04:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:04:42.969 INFO:teuthology.orchestra.run.smithi178.stdout:[client] 2024-04-04T06:04:42.969 INFO:teuthology.orchestra.run.smithi178.stdout: rgw cache enabled = true 2024-04-04T06:04:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable ops log = true 2024-04-04T06:04:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable usage log = true 2024-04-04T06:04:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-04T06:04:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-04T06:04:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: client mount timeout = 600 2024-04-04T06:04:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-04-04T06:04:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:04:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = "" 2024-04-04T06:04:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-04-04T06:04:42.970 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-04-04T06:04:42.970 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.a] 2024-04-04T06:04:42.970 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.b] 2024-04-04T06:04:42.970 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.c] 2024-04-04T06:04:42.970 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.a] 2024-04-04T06:04:42.971 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.c] 2024-04-04T06:04:42.971 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.b] 2024-04-04T06:04:42.971 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.d] 2024-04-04T06:04:42.971 INFO:teuthology.orchestra.run.smithi178.stdout:[mds] 2024-04-04T06:04:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds = 20 2024-04-04T06:04:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds balancer = 20 2024-04-04T06:04:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:04:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug frag = True 2024-04-04T06:04:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug scatterstat = True 2024-04-04T06:04:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: mds op complaint time = 180 2024-04-04T06:04:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: mds verify scatter = True 2024-04-04T06:04:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-04-04T06:04:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-04-04T06:04:42.971 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-04-04T06:04:43.006 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-04T06:04:43.007 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-04T06:04:43.150 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-04T06:04:43.151 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:04:43.151 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-04T06:04:43.204 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:04:43.205 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-04T06:04:43.253 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:04:43.253 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:04:43.253 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-04T06:04:43.253 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-04T06:04:43.440 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:04:43.440 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-04T06:04:43.535 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi178.front.sepia.ceph.com 2024-04-04T06:04:43.535 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:04:43.536 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-04T06:04:43.536 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-04T06:04:43.685 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:04:43.686 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-04T06:04:43.781 INFO:tasks.ceph:Setting up mon nodes... 2024-04-04T06:04:43.781 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-04T06:04:43.781 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-04T06:04:43.966 INFO:teuthology.orchestra.run.smithi159.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-04T06:04:43.976 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-04T06:04:44.191 INFO:teuthology.orchestra.run.smithi178.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-04T06:04:44.202 INFO:tasks.ceph:Setting up mds nodes... 2024-04-04T06:04:44.203 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-04T06:04:44.387 INFO:teuthology.orchestra.run.smithi159.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-04T06:04:44.396 DEBUG:teuthology.orchestra.run.smithi159:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-04T06:04:44.491 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-04T06:04:44.667 INFO:teuthology.orchestra.run.smithi159.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-04T06:04:44.676 DEBUG:teuthology.orchestra.run.smithi159:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-04T06:04:44.768 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-04T06:04:44.952 INFO:teuthology.orchestra.run.smithi178.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-04T06:04:44.961 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-04T06:04:45.050 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-04T06:04:45.219 INFO:teuthology.orchestra.run.smithi178.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-04T06:04:45.228 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-04T06:04:45.323 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-04T06:04:45.323 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-04T06:04:45.456 INFO:teuthology.orchestra.run.smithi159.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-04T06:04:45.511 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-04T06:04:45.627 INFO:teuthology.orchestra.run.smithi159.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-04T06:04:45.676 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-04T06:04:45.804 INFO:teuthology.orchestra.run.smithi178.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-04T06:04:45.858 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-04T06:04:45.983 INFO:teuthology.orchestra.run.smithi178.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-04T06:04:46.036 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-04T06:04:46.037 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi159.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi178.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-04T06:04:46.037 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-04T06:04:46.133 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-04T06:04:46.133 INFO:tasks.ceph:role: osd.0 2024-04-04T06:04:46.133 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:04:46.133 DEBUG:teuthology.orchestra.run.smithi159:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-04T06:04:46.227 INFO:teuthology.orchestra.run.smithi159.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-04T06:04:46.227 INFO:teuthology.orchestra.run.smithi159.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-04T06:04:46.227 INFO:teuthology.orchestra.run.smithi159.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-04T06:04:46.227 INFO:teuthology.orchestra.run.smithi159.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-04T06:04:46.227 INFO:teuthology.orchestra.run.smithi159.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-04T06:04:46.227 INFO:teuthology.orchestra.run.smithi159.stdout: = sunit=0 swidth=0 blks 2024-04-04T06:04:46.227 INFO:teuthology.orchestra.run.smithi159.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-04T06:04:46.227 INFO:teuthology.orchestra.run.smithi159.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-04T06:04:46.227 INFO:teuthology.orchestra.run.smithi159.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-04T06:04:46.228 INFO:teuthology.orchestra.run.smithi159.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-04T06:04:46.754 INFO:teuthology.orchestra.run.smithi159.stdout:Discarding blocks...Done. 2024-04-04T06:04:46.761 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi159.front.sepia.ceph.com -o noatime 2024-04-04T06:04:46.761 DEBUG:teuthology.orchestra.run.smithi159:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-04T06:04:46.927 DEBUG:teuthology.orchestra.run.smithi159:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-04T06:04:47.029 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-04T06:04:47.113 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-04T06:04:47.113 INFO:tasks.ceph:role: osd.1 2024-04-04T06:04:47.113 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:04:47.113 DEBUG:teuthology.orchestra.run.smithi159:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-04T06:04:47.201 INFO:teuthology.orchestra.run.smithi159.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-04T06:04:47.201 INFO:teuthology.orchestra.run.smithi159.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-04T06:04:47.201 INFO:teuthology.orchestra.run.smithi159.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-04T06:04:47.201 INFO:teuthology.orchestra.run.smithi159.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-04T06:04:47.201 INFO:teuthology.orchestra.run.smithi159.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-04T06:04:47.201 INFO:teuthology.orchestra.run.smithi159.stdout: = sunit=0 swidth=0 blks 2024-04-04T06:04:47.202 INFO:teuthology.orchestra.run.smithi159.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-04T06:04:47.202 INFO:teuthology.orchestra.run.smithi159.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-04T06:04:47.202 INFO:teuthology.orchestra.run.smithi159.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-04T06:04:47.202 INFO:teuthology.orchestra.run.smithi159.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-04T06:04:47.676 INFO:teuthology.orchestra.run.smithi159.stdout:Discarding blocks...Done. 2024-04-04T06:04:47.682 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi159.front.sepia.ceph.com -o noatime 2024-04-04T06:04:47.682 DEBUG:teuthology.orchestra.run.smithi159:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-04T06:04:47.800 DEBUG:teuthology.orchestra.run.smithi159:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-04T06:04:47.887 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-04T06:04:47.974 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-04T06:04:47.974 INFO:tasks.ceph:role: osd.2 2024-04-04T06:04:47.974 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:04:47.974 DEBUG:teuthology.orchestra.run.smithi159:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-04T06:04:48.061 INFO:teuthology.orchestra.run.smithi159.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-04T06:04:48.062 INFO:teuthology.orchestra.run.smithi159.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-04T06:04:48.062 INFO:teuthology.orchestra.run.smithi159.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-04T06:04:48.062 INFO:teuthology.orchestra.run.smithi159.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-04T06:04:48.062 INFO:teuthology.orchestra.run.smithi159.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-04T06:04:48.062 INFO:teuthology.orchestra.run.smithi159.stdout: = sunit=0 swidth=0 blks 2024-04-04T06:04:48.062 INFO:teuthology.orchestra.run.smithi159.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-04T06:04:48.062 INFO:teuthology.orchestra.run.smithi159.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-04T06:04:48.062 INFO:teuthology.orchestra.run.smithi159.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-04T06:04:48.062 INFO:teuthology.orchestra.run.smithi159.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-04T06:04:48.574 INFO:teuthology.orchestra.run.smithi159.stdout:Discarding blocks...Done. 2024-04-04T06:04:48.580 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi159.front.sepia.ceph.com -o noatime 2024-04-04T06:04:48.580 DEBUG:teuthology.orchestra.run.smithi159:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-04T06:04:48.762 DEBUG:teuthology.orchestra.run.smithi159:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-04T06:04:48.864 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-04T06:04:48.956 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-04T06:04:48.957 INFO:tasks.ceph:role: osd.3 2024-04-04T06:04:48.957 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:04:48.957 DEBUG:teuthology.orchestra.run.smithi159:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-04T06:04:49.048 INFO:teuthology.orchestra.run.smithi159.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-04T06:04:49.048 INFO:teuthology.orchestra.run.smithi159.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-04T06:04:49.048 INFO:teuthology.orchestra.run.smithi159.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-04T06:04:49.048 INFO:teuthology.orchestra.run.smithi159.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-04T06:04:49.048 INFO:teuthology.orchestra.run.smithi159.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-04T06:04:49.048 INFO:teuthology.orchestra.run.smithi159.stdout: = sunit=0 swidth=0 blks 2024-04-04T06:04:49.048 INFO:teuthology.orchestra.run.smithi159.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-04T06:04:49.048 INFO:teuthology.orchestra.run.smithi159.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-04T06:04:49.048 INFO:teuthology.orchestra.run.smithi159.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-04T06:04:49.048 INFO:teuthology.orchestra.run.smithi159.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-04T06:04:49.517 INFO:teuthology.orchestra.run.smithi159.stdout:Discarding blocks...Done. 2024-04-04T06:04:49.523 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi159.front.sepia.ceph.com -o noatime 2024-04-04T06:04:49.524 DEBUG:teuthology.orchestra.run.smithi159:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-04T06:04:49.658 DEBUG:teuthology.orchestra.run.smithi159:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-04T06:04:49.744 DEBUG:teuthology.orchestra.run.smithi159:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-04T06:04:49.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:49.903+0000 7f6d9e716640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-04T06:04:49.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:49.904+0000 7f6d9e716640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-04T06:04:49.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:49.904+0000 7f6d9e716640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-04T06:04:49.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:49.904+0000 7f6d9e716640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-04T06:04:49.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:49.904+0000 7f6d9e716640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-04T06:04:49.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:49.912+0000 7f6d9e716640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-04T06:04:53.099 DEBUG:teuthology.orchestra.run.smithi159:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-04T06:04:53.190 DEBUG:teuthology.orchestra.run.smithi159:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-04T06:04:53.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:53.327+0000 7fe3c7e89640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-04T06:04:53.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:53.328+0000 7fe3c7e89640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-04T06:04:53.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:53.328+0000 7fe3c7e89640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-04T06:04:53.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:53.328+0000 7fe3c7e89640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-04T06:04:53.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:53.328+0000 7fe3c7e89640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-04T06:04:53.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:53.330+0000 7fe3c7e89640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-04T06:04:56.265 DEBUG:teuthology.orchestra.run.smithi159:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-04T06:04:56.359 DEBUG:teuthology.orchestra.run.smithi159:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-04T06:04:56.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:56.500+0000 7f173a36d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-04T06:04:56.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:56.500+0000 7f173a36d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-04T06:04:56.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:56.500+0000 7f173a36d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-04T06:04:56.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:56.500+0000 7f173a36d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-04T06:04:56.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:56.500+0000 7f173a36d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-04T06:04:56.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:56.503+0000 7f173a36d640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-04T06:04:59.419 DEBUG:teuthology.orchestra.run.smithi159:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-04T06:04:59.512 DEBUG:teuthology.orchestra.run.smithi159:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-04T06:04:59.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:59.651+0000 7f351f211640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-04T06:04:59.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:59.652+0000 7f351f211640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-04T06:04:59.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:59.652+0000 7f351f211640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-04T06:04:59.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:59.652+0000 7f351f211640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-04T06:04:59.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:59.652+0000 7f351f211640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-04T06:04:59.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:04:59.654+0000 7f351f211640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-04T06:05:02.576 DEBUG:teuthology.orchestra.run.smithi159:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-04T06:05:02.676 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-04T06:05:02.777 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-04T06:05:02.777 INFO:tasks.ceph:role: osd.4 2024-04-04T06:05:02.778 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-04T06:05:02.778 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-04T06:05:02.867 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-04T06:05:02.867 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-04T06:05:02.867 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-04T06:05:02.867 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-04T06:05:02.867 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-04T06:05:02.867 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-04-04T06:05:02.867 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-04T06:05:02.867 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-04T06:05:02.867 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-04T06:05:02.867 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-04T06:05:02.900 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2024-04-04T06:05:02.907 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-04-04T06:05:02.907 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-04T06:05:03.019 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-04T06:05:03.107 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-04T06:05:03.198 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-04T06:05:03.198 INFO:tasks.ceph:role: osd.5 2024-04-04T06:05:03.198 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-04T06:05:03.198 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-04T06:05:03.284 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-04T06:05:03.284 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-04T06:05:03.284 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-04T06:05:03.284 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-04T06:05:03.284 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-04T06:05:03.284 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-04-04T06:05:03.284 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-04T06:05:03.284 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-04T06:05:03.284 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-04T06:05:03.284 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-04T06:05:03.318 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2024-04-04T06:05:03.324 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-04-04T06:05:03.325 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-04T06:05:03.436 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-04T06:05:03.515 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-04T06:05:03.596 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-04T06:05:03.596 INFO:tasks.ceph:role: osd.6 2024-04-04T06:05:03.596 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-04T06:05:03.596 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-04T06:05:03.676 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-04T06:05:03.676 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-04T06:05:03.676 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-04T06:05:03.676 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-04T06:05:03.676 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-04T06:05:03.676 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-04-04T06:05:03.677 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-04T06:05:03.677 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-04T06:05:03.677 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-04T06:05:03.677 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-04T06:05:03.710 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2024-04-04T06:05:03.715 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-04-04T06:05:03.715 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-04T06:05:03.813 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-04T06:05:03.888 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-04T06:05:03.972 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-04T06:05:03.972 INFO:tasks.ceph:role: osd.7 2024-04-04T06:05:03.973 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-04T06:05:03.973 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-04T06:05:04.057 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-04T06:05:04.057 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-04T06:05:04.058 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-04T06:05:04.058 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-04T06:05:04.058 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-04T06:05:04.058 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-04-04T06:05:04.058 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-04T06:05:04.058 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-04T06:05:04.058 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-04T06:05:04.058 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-04T06:05:04.091 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2024-04-04T06:05:04.096 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-04-04T06:05:04.096 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-04T06:05:04.207 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-04T06:05:04.287 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-04T06:05:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:04.422+0000 7fbedcc8b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-04T06:05:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:04.422+0000 7fbedcc8b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-04T06:05:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:04.422+0000 7fbedcc8b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-04T06:05:04.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:04.422+0000 7fbedcc8b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-04T06:05:04.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:04.422+0000 7fbedcc8b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-04T06:05:04.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:04.425+0000 7fbedcc8b640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-04T06:05:07.584 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-04T06:05:07.679 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-04T06:05:07.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:07.819+0000 7f92304a4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-04T06:05:07.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:07.819+0000 7f92304a4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-04T06:05:07.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:07.819+0000 7f92304a4640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-04T06:05:07.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:07.819+0000 7f92304a4640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-04T06:05:07.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:07.819+0000 7f92304a4640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-04T06:05:07.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:07.821+0000 7f92304a4640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-04T06:05:10.254 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:05:10.262 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:05:10.730 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-04T06:05:10.824 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-04T06:05:10.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:10.965+0000 7efe4878c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-04T06:05:10.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:10.965+0000 7efe4878c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-04T06:05:10.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:10.965+0000 7efe4878c640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-04T06:05:10.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:10.966+0000 7efe4878c640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-04T06:05:10.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:10.966+0000 7efe4878c640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-04T06:05:10.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:10.968+0000 7efe4878c640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-04T06:05:13.890 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-04T06:05:13.978 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-04T06:05:14.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:14.117+0000 7fd91700f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-04T06:05:14.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:14.117+0000 7fd91700f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-04T06:05:14.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:14.117+0000 7fd91700f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-04T06:05:14.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:14.117+0000 7fd91700f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-04T06:05:14.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:14.117+0000 7fd91700f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-04T06:05:14.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:14.119+0000 7fd91700f640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-04T06:05:17.035 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-04T06:05:17.123 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-04T06:05:17.124 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:05:17.124 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-04T06:05:17.228 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:05:17.228 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-04T06:05:17.322 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:05:17.322 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-04T06:05:17.418 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:05:17.418 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-04T06:05:17.513 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:05:17.513 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-04T06:05:17.608 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:05:17.608 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-04T06:05:17.700 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:05:17.701 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-04T06:05:17.795 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:05:17.795 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-04T06:05:17.891 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:05:17.891 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-04T06:05:17.971 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:05:17.972 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-04T06:05:18.054 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:05:18.054 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-04T06:05:18.190 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:05:18.191 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-04T06:05:18.279 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:05:18.279 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-04T06:05:18.369 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:05:18.369 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-04T06:05:18.457 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:05:18.457 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-04T06:05:18.514 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:05:18.514 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-04T06:05:18.564 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:05:18.565 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-04T06:05:18.621 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:05:18.621 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-04T06:05:18.671 INFO:tasks.ceph:Adding keys to all mons... 2024-04-04T06:05:18.671 DEBUG:teuthology.orchestra.run.smithi159:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-04T06:05:18.679 DEBUG:teuthology.orchestra.run.smithi178:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-04T06:05:18.765 INFO:teuthology.orchestra.run.smithi159.stdout:[mgr.y] 2024-04-04T06:05:18.765 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQD7Qg5miXT0ORAAL6y306CPAfEGQKAJ0no4uw== 2024-04-04T06:05:18.765 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.a] 2024-04-04T06:05:18.765 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQD8Qg5mvoZ2FxAAsVEQrkiICIH6T7ZzaqcWbA== 2024-04-04T06:05:18.765 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.c] 2024-04-04T06:05:18.765 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQD8Qg5mjmQcKBAA8BjpQce6G7EycXOkyVECpQ== 2024-04-04T06:05:18.765 INFO:teuthology.orchestra.run.smithi159.stdout:[osd.0] 2024-04-04T06:05:18.765 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQABQw5mCe7fNRAA6ZKxWvnMLf4jhAlpjudFRA== 2024-04-04T06:05:18.765 INFO:teuthology.orchestra.run.smithi159.stdout:[osd.1] 2024-04-04T06:05:18.765 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQAFQw5mvIqTExAAU8Pm5alxFtHEL4qM9MavPw== 2024-04-04T06:05:18.765 INFO:teuthology.orchestra.run.smithi159.stdout:[osd.2] 2024-04-04T06:05:18.766 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQAIQw5mAHblHRAAqEDe5dIGTDpdy2OsQyzJXA== 2024-04-04T06:05:18.766 INFO:teuthology.orchestra.run.smithi159.stdout:[osd.3] 2024-04-04T06:05:18.766 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQALQw5mwK7rJhAADQBAqE0Eej9rn91Wxhq/KA== 2024-04-04T06:05:18.766 INFO:teuthology.orchestra.run.smithi159.stdout:[mgr.x] 2024-04-04T06:05:18.766 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQD8Qg5mhqbBCxAASBknMIj4uu7OiB3jn7WP8Q== 2024-04-04T06:05:18.766 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.b] 2024-04-04T06:05:18.766 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQD8Qg5m+rccORAA6I/lQ+22QfWYL9+aoOFJ0w== 2024-04-04T06:05:18.766 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.d] 2024-04-04T06:05:18.766 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQD9Qg5m04hoDRAAKdhRI7nosZp7d4y7dVvzvg== 2024-04-04T06:05:18.766 INFO:teuthology.orchestra.run.smithi159.stdout:[osd.4] 2024-04-04T06:05:18.766 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQAQQw5m/6I8GRAA2P7jI+AXYld2J11nDfhAUQ== 2024-04-04T06:05:18.766 INFO:teuthology.orchestra.run.smithi159.stdout:[osd.5] 2024-04-04T06:05:18.766 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQATQw5mv5HoMBAAn2mp2ys5KjS0UygOcX4vvg== 2024-04-04T06:05:18.766 INFO:teuthology.orchestra.run.smithi159.stdout:[osd.6] 2024-04-04T06:05:18.767 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQAWQw5mx12gORAAT2LhrfEB5BJSl0ypgospzg== 2024-04-04T06:05:18.767 INFO:teuthology.orchestra.run.smithi159.stdout:[osd.7] 2024-04-04T06:05:18.767 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQAaQw5m6HEOBxAAVegLH3bjVM7r1A6tK5eqHQ== 2024-04-04T06:05:18.767 INFO:teuthology.orchestra.run.smithi159.stdout:[client.0] 2024-04-04T06:05:18.767 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQD9Qg5mqNKJGxAA4jP9cadhRIyhQ3lssttWIw== 2024-04-04T06:05:18.767 INFO:teuthology.orchestra.run.smithi159.stdout:[client.2] 2024-04-04T06:05:18.767 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQD9Qg5mtW2/JRAAucxSSN9q0EQtI2a/P2yozQ== 2024-04-04T06:05:18.767 INFO:teuthology.orchestra.run.smithi159.stdout:[client.1] 2024-04-04T06:05:18.767 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQD9Qg5mRkdOMBAA1ae1roIwrlB49DFQUoFSTg== 2024-04-04T06:05:18.767 INFO:teuthology.orchestra.run.smithi159.stdout:[client.3] 2024-04-04T06:05:18.767 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQD9Qg5mgqD0OhAAov+3aDGyGEoqxep8+SBpSA== 2024-04-04T06:05:18.768 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr.y] 2024-04-04T06:05:18.768 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQD7Qg5miXT0ORAAL6y306CPAfEGQKAJ0no4uw== 2024-04-04T06:05:18.768 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.a] 2024-04-04T06:05:18.768 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQD8Qg5mvoZ2FxAAsVEQrkiICIH6T7ZzaqcWbA== 2024-04-04T06:05:18.768 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.c] 2024-04-04T06:05:18.768 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQD8Qg5mjmQcKBAA8BjpQce6G7EycXOkyVECpQ== 2024-04-04T06:05:18.768 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.0] 2024-04-04T06:05:18.768 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQABQw5mCe7fNRAA6ZKxWvnMLf4jhAlpjudFRA== 2024-04-04T06:05:18.768 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.1] 2024-04-04T06:05:18.768 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQAFQw5mvIqTExAAU8Pm5alxFtHEL4qM9MavPw== 2024-04-04T06:05:18.768 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.2] 2024-04-04T06:05:18.769 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQAIQw5mAHblHRAAqEDe5dIGTDpdy2OsQyzJXA== 2024-04-04T06:05:18.769 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.3] 2024-04-04T06:05:18.769 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQALQw5mwK7rJhAADQBAqE0Eej9rn91Wxhq/KA== 2024-04-04T06:05:18.769 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr.x] 2024-04-04T06:05:18.769 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQD8Qg5mhqbBCxAASBknMIj4uu7OiB3jn7WP8Q== 2024-04-04T06:05:18.769 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.b] 2024-04-04T06:05:18.769 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQD8Qg5m+rccORAA6I/lQ+22QfWYL9+aoOFJ0w== 2024-04-04T06:05:18.769 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.d] 2024-04-04T06:05:18.769 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQD9Qg5m04hoDRAAKdhRI7nosZp7d4y7dVvzvg== 2024-04-04T06:05:18.769 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.4] 2024-04-04T06:05:18.769 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQAQQw5m/6I8GRAA2P7jI+AXYld2J11nDfhAUQ== 2024-04-04T06:05:18.769 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.5] 2024-04-04T06:05:18.769 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQATQw5mv5HoMBAAn2mp2ys5KjS0UygOcX4vvg== 2024-04-04T06:05:18.769 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.6] 2024-04-04T06:05:18.769 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQAWQw5mx12gORAAT2LhrfEB5BJSl0ypgospzg== 2024-04-04T06:05:18.770 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.7] 2024-04-04T06:05:18.770 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQAaQw5m6HEOBxAAVegLH3bjVM7r1A6tK5eqHQ== 2024-04-04T06:05:18.770 INFO:teuthology.orchestra.run.smithi178.stdout:[client.0] 2024-04-04T06:05:18.770 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQD9Qg5mqNKJGxAA4jP9cadhRIyhQ3lssttWIw== 2024-04-04T06:05:18.770 INFO:teuthology.orchestra.run.smithi178.stdout:[client.2] 2024-04-04T06:05:18.770 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQD9Qg5mtW2/JRAAucxSSN9q0EQtI2a/P2yozQ== 2024-04-04T06:05:18.770 INFO:teuthology.orchestra.run.smithi178.stdout:[client.1] 2024-04-04T06:05:18.770 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQD9Qg5mRkdOMBAA1ae1roIwrlB49DFQUoFSTg== 2024-04-04T06:05:18.770 INFO:teuthology.orchestra.run.smithi178.stdout:[client.3] 2024-04-04T06:05:18.770 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQD9Qg5mgqD0OhAAov+3aDGyGEoqxep8+SBpSA== 2024-04-04T06:05:18.772 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-04T06:05:18.781 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-04T06:05:18.922 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-04T06:05:18.926 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-04T06:05:19.067 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-04T06:05:19.073 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-04T06:05:19.212 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-04T06:05:19.217 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-04T06:05:19.349 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-04T06:05:19.355 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-04T06:05:19.497 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-04T06:05:19.503 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-04T06:05:19.639 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-04T06:05:19.646 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-04T06:05:19.788 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-04T06:05:19.794 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-04T06:05:19.934 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-04T06:05:19.940 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-04T06:05:20.078 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-04T06:05:20.084 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-04T06:05:20.227 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-04T06:05:20.234 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-04T06:05:20.372 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-04T06:05:20.378 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-04T06:05:20.514 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-04T06:05:20.520 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-04T06:05:20.652 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-04T06:05:20.658 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-04T06:05:20.800 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-04T06:05:20.806 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-04T06:05:20.946 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-04T06:05:20.952 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-04T06:05:21.094 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-04T06:05:21.099 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-04T06:05:21.245 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-04T06:05:21.251 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-04T06:05:21.389 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-04T06:05:21.389 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-04T06:05:21.485 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-04T06:05:21.637 DEBUG:teuthology.orchestra.run.smithi159:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-04T06:05:21.727 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-04T06:05:21.820 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-04T06:05:21.971 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-04T06:05:22.061 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-04T06:05:22.154 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-04T06:05:22.306 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-04T06:05:22.385 DEBUG:teuthology.orchestra.run.smithi159:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-04T06:05:22.393 DEBUG:teuthology.orchestra.run.smithi178:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-04T06:05:22.445 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-04T06:05:22.445 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-04T06:05:22.445 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-04T06:05:22.450 INFO:tasks.ceph.mon.a:Started 2024-04-04T06:05:22.450 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-04T06:05:22.450 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-04T06:05:22.455 INFO:tasks.ceph.mon.b:Started 2024-04-04T06:05:22.456 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-04T06:05:22.456 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-04T06:05:22.461 INFO:tasks.ceph.mon.c:Started 2024-04-04T06:05:22.461 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-04T06:05:22.461 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-04T06:05:22.461 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-04T06:05:22.465 INFO:tasks.ceph.mgr.y:Started 2024-04-04T06:05:22.466 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-04T06:05:22.466 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-04T06:05:22.469 INFO:tasks.ceph.mgr.x:Started 2024-04-04T06:05:22.470 INFO:tasks.ceph:Setting crush tunables to default 2024-04-04T06:05:22.470 DEBUG:teuthology.orchestra.run.smithi159:> sudo ceph --cluster ceph osd crush tunables default 2024-04-04T06:05:22.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:22.658+0000 7f5fcb000640 1 Processor -- start 2024-04-04T06:05:22.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:22.658+0000 7f5fcb000640 1 -- start start 2024-04-04T06:05:22.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:22.659+0000 7f5fcb000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 0x7f5fc406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:22.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:22.659+0000 7f5fcb000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fc406dfa0 con 0x7f5fc406da30 2024-04-04T06:05:22.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:22.659+0000 7f5fcb000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fc406e240 con 0x7f5fc40d4ae0 2024-04-04T06:05:22.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:22.659+0000 7f5fcb000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f5fc40cf220 con 0x7f5fc415e860 2024-04-04T06:05:22.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:22.660+0000 7f5fc3400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f5fc406da30 0x7f5fc40d43d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:44760/0 (socket says 172.21.15.159:44760) 2024-04-04T06:05:22.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:22.660+0000 7f5fc3400640 1 -- 172.21.15.159:0/766868792 learned_addr learned my addr 172.21.15.159:0/766868792 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:05:22.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:22.660+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 0x7f5fc406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:22.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:22.660+0000 7f5fc3e00640 1 -- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 msgr2=0x7f5fc406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-04T06:05:22.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:22.660+0000 7f5fc3e00640 1 -- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 msgr2=0x7f5fc406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-04T06:05:22.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:22.660+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 0x7f5fc406d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-04T06:05:22.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:22.660+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 0x7f5fc406d4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-04T06:05:22.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:22.861+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 0x7f5fc406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:22.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:22.862+0000 7f5fc3e00640 1 -- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 msgr2=0x7f5fc406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-04T06:05:22.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:22.862+0000 7f5fc3e00640 1 -- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 msgr2=0x7f5fc406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-04T06:05:22.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:22.862+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 0x7f5fc406d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-04T06:05:22.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:22.862+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 0x7f5fc406d4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-04T06:05:23.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:23.263+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 0x7f5fc406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:23.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:23.264+0000 7f5fc3e00640 1 -- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 msgr2=0x7f5fc406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-04T06:05:23.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:23.264+0000 7f5fc3e00640 1 -- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 msgr2=0x7f5fc406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-04T06:05:23.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:23.264+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 0x7f5fc406d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-04T06:05:23.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:23.264+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 0x7f5fc406d4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-04T06:05:24.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:24.065+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 0x7f5fc406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:24.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:24.066+0000 7f5fc3e00640 1 -- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 msgr2=0x7f5fc406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-04T06:05:24.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:24.066+0000 7f5fc3e00640 1 -- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 msgr2=0x7f5fc406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-04T06:05:24.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:24.066+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 0x7f5fc406d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-04T06:05:24.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:24.066+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 0x7f5fc406d4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-04T06:05:25.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.660+0000 7f5fc1600640 1 -- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 msgr2=0x7f5fc406d4f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-04T06:05:25.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.660+0000 7f5fc1600640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 0x7f5fc406d4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:25.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.660+0000 7f5fc1600640 1 -- 172.21.15.159:0/766868792 >> v1:172.21.15.178:6789/0 conn(0x7f5fc40d4ae0 legacy=0x7f5fc40cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:25.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.660+0000 7f5fc1600640 1 -- 172.21.15.159:0/766868792 >> v1:172.21.15.159:6789/0 conn(0x7f5fc406da30 legacy=0x7f5fc40d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:25.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.660+0000 7f5fc1600640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 0x7f5fa4005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:25.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.660+0000 7f5fc1600640 1 --2- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fa4008640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:25.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.660+0000 7f5fc1600640 1 -- 172.21.15.159:0/766868792 --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f5fc40cf220 con 0x7f5fa40036e0 2024-04-04T06:05:25.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.660+0000 7f5fc1600640 1 -- 172.21.15.159:0/766868792 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5fa4008d60 con 0x7f5fa4006180 2024-04-04T06:05:25.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.660+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 0x7f5fa4005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:25.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.660+0000 7f5fc3400640 1 --2- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fa4008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:25.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.661+0000 7f5fc3400640 1 -- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 msgr2=0x7f5fa4008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-04T06:05:25.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.661+0000 7f5fc3400640 1 -- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 msgr2=0x7f5fa4008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-04T06:05:25.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.661+0000 7f5fc3400640 1 --2- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fa4008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-04T06:05:25.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.661+0000 7f5fc3e00640 1 -- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 msgr2=0x7f5fa4005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-04T06:05:25.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.661+0000 7f5fc3e00640 1 -- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 msgr2=0x7f5fa4005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-04T06:05:25.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.661+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 0x7f5fa4005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-04T06:05:25.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.661+0000 7f5fc3400640 1 --2- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fa4008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-04T06:05:25.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.661+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 0x7f5fa4005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-04T06:05:25.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.862+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 0x7f5fa4005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:25.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.862+0000 7f5fc3400640 1 --2- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fa4008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:25.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.862+0000 7f5fc3400640 1 -- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 msgr2=0x7f5fa4008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-04T06:05:25.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.862+0000 7f5fc3400640 1 -- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 msgr2=0x7f5fa4008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-04T06:05:25.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.862+0000 7f5fc3400640 1 --2- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fa4008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-04T06:05:25.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.862+0000 7f5fc3400640 1 --2- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fa4008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-04T06:05:25.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.862+0000 7f5fc3e00640 1 -- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 msgr2=0x7f5fa4005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-04T06:05:25.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.862+0000 7f5fc3e00640 1 -- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 msgr2=0x7f5fa4005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-04T06:05:25.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.862+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 0x7f5fa4005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-04T06:05:25.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:25.863+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 0x7f5fa4005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-04T06:05:26.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:26.264+0000 7f5fc3400640 1 --2- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fa4008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:26.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:26.264+0000 7f5fc3400640 1 -- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 msgr2=0x7f5fa4008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-04T06:05:26.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:26.264+0000 7f5fc3400640 1 -- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 msgr2=0x7f5fa4008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-04T06:05:26.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:26.264+0000 7f5fc3400640 1 --2- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fa4008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-04T06:05:26.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:26.264+0000 7f5fc3400640 1 --2- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fa4008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-04T06:05:26.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:26.264+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 0x7f5fa4005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:26.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:26.265+0000 7f5fc3e00640 1 -- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 msgr2=0x7f5fa4005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-04T06:05:26.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:26.265+0000 7f5fc3e00640 1 -- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 msgr2=0x7f5fa4005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-04T06:05:26.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:26.265+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 0x7f5fa4005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-04T06:05:26.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:26.265+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 0x7f5fa4005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-04T06:05:27.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:27.066+0000 7f5fc3400640 1 --2- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fa4008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:27.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:27.066+0000 7f5fc3400640 1 -- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 msgr2=0x7f5fa4008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-04T06:05:27.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:27.066+0000 7f5fc3400640 1 -- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 msgr2=0x7f5fa4008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-04T06:05:27.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:27.066+0000 7f5fc3400640 1 --2- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fa4008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-04T06:05:27.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:27.066+0000 7f5fc3400640 1 --2- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fa4008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-04T06:05:27.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:27.066+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 0x7f5fa4005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:27.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:27.067+0000 7f5fc3e00640 1 -- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 msgr2=0x7f5fa4005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-04T06:05:27.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:27.067+0000 7f5fc3e00640 1 -- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 msgr2=0x7f5fa4005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-04T06:05:27.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:27.067+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 0x7f5fa4005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-04T06:05:27.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:27.067+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 0x7f5fa4005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-04T06:05:28.294 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:28.299+0000 7f0a7001a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-04T06:05:28.357 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:28.361+0000 7f0a7001a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-04T06:05:28.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.661+0000 7f5fc1600640 1 -- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 msgr2=0x7f5fa4008640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-04T06:05:28.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.661+0000 7f5fc1600640 1 --2- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fa4008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:28.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.661+0000 7f5fc1600640 1 -- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 msgr2=0x7f5fa4005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-04T06:05:28.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.661+0000 7f5fc1600640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.159:3300/0 conn(0x7f5fa40036e0 0x7f5fa4005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:28.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.661+0000 7f5fc9000640 1 -- 172.21.15.159:0/766868792 reap_dead start 2024-04-04T06:05:28.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.661+0000 7f5fc1600640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 0x7f5fa400b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:28.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.661+0000 7f5fc1600640 1 --2- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fa4008640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:28.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.661+0000 7f5fc1600640 1 -- 172.21.15.159:0/766868792 --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fc406e520 con 0x7f5fc40d4ae0 2024-04-04T06:05:28.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.661+0000 7f5fc1600640 1 -- 172.21.15.159:0/766868792 --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f5fc40cf220 con 0x7f5fc415e860 2024-04-04T06:05:28.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.661+0000 7f5fc1600640 1 -- 172.21.15.159:0/766868792 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5fa4008d60 con 0x7f5fa4006180 2024-04-04T06:05:28.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.661+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> v2:172.21.15.178:3300/0 conn(0x7f5fc415e860 0x7f5fa400b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:28.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.661+0000 7f5fc9000640 1 --2- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fa4008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:28.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.661+0000 7f5fc3e00640 1 -- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 msgr2=0x7f5fa4008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:28.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.661+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fa4008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:28.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.661+0000 7f5fc3e00640 1 -- 172.21.15.159:0/766868792 >> v1:172.21.15.178:6789/0 conn(0x7f5fc40d4ae0 legacy=0x7f5fa400d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:28.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.661+0000 7f5fc3e00640 1 -- 172.21.15.159:0/766868792 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa4003b80 con 0x7f5fc415e860 2024-04-04T06:05:28.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.663+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fc415e860 0x7f5fa400b270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4006fd0 tx=0x7f5fb402db30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=accef3195a3b1052 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:28.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.664+0000 7f5fc2a00640 1 -- 172.21.15.159:0/766868792 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb402f4e0 con 0x7f5fc415e860 2024-04-04T06:05:28.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.664+0000 7f5fc2a00640 1 -- 172.21.15.159:0/766868792 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb402f680 con 0x7f5fc415e860 2024-04-04T06:05:28.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.664+0000 7f5fc2a00640 1 -- 172.21.15.159:0/766868792 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb402fa70 con 0x7f5fc415e860 2024-04-04T06:05:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.665+0000 7f5fcb000640 1 -- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fc415e860 msgr2=0x7f5fa400b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.665+0000 7f5fcb000640 1 --2- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fc415e860 0x7f5fa400b270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4006fd0 tx=0x7f5fb402db30 comp rx=0 tx=0).stop 2024-04-04T06:05:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.665+0000 7f5fcb000640 1 -- 172.21.15.159:0/766868792 shutdown_connections 2024-04-04T06:05:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.665+0000 7f5fcb000640 1 --2- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fa4008640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.665+0000 7f5fcb000640 1 --2- 172.21.15.159:0/766868792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fc415e860 0x7f5fa400b270 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.665+0000 7f5fcb000640 1 -- 172.21.15.159:0/766868792 >> 172.21.15.159:0/766868792 conn(0x7f5fc409d7d0 msgr2=0x7f5fc408bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.665+0000 7f5fcb000640 1 -- 172.21.15.159:0/766868792 shutdown_connections 2024-04-04T06:05:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.665+0000 7f5fcb000640 1 -- 172.21.15.159:0/766868792 wait complete. 2024-04-04T06:05:28.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.668+0000 7f5fcb000640 1 Processor -- start 2024-04-04T06:05:28.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.668+0000 7f5fcb000640 1 -- start start 2024-04-04T06:05:28.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.669+0000 7f5fcb000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fc41584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:28.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.669+0000 7f5fcb000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fc40d4ae0 0x7f5fc40cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:28.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.669+0000 7f5fcb000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fc415e860 0x7f5fc40b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:28.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.669+0000 7f5fcb000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc40cef30 con 0x7f5fc415e860 2024-04-04T06:05:28.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.669+0000 7f5fcb000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc40cedb0 con 0x7f5fc40d4ae0 2024-04-04T06:05:28.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.669+0000 7f5fcb000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5fc406d4f0 con 0x7f5fa4006180 2024-04-04T06:05:28.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.670+0000 7f5fc9000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fc415e860 0x7f5fc40b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:28.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.670+0000 7f5fc9000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fc415e860 0x7f5fc40b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39026/0 (socket says 172.21.15.159:39026) 2024-04-04T06:05:28.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.670+0000 7f5fc9000640 1 -- 172.21.15.159:0/4042529122 learned_addr learned my addr 172.21.15.159:0/4042529122 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:28.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.670+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/4042529122 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fc41584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:28.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.670+0000 7f5fc3400640 1 --2- 172.21.15.159:0/4042529122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fc40d4ae0 0x7f5fc40cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:28.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.671+0000 7f5fc3e00640 1 -- 172.21.15.159:0/4042529122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fc40d4ae0 msgr2=0x7f5fc40cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:28.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.671+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/4042529122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fc40d4ae0 0x7f5fc40cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:28.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.671+0000 7f5fc3e00640 1 -- 172.21.15.159:0/4042529122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fc415e860 msgr2=0x7f5fc40b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:28.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.671+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/4042529122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fc415e860 0x7f5fc40b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:28.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.671+0000 7f5fc3e00640 1 -- 172.21.15.159:0/4042529122 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc40b7370 con 0x7f5fa4006180 2024-04-04T06:05:28.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.671+0000 7f5fc3400640 1 --2- 172.21.15.159:0/4042529122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fc40d4ae0 0x7f5fc40cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:05:28.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.671+0000 7f5fc9000640 1 --2- 172.21.15.159:0/4042529122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fc415e860 0x7f5fc40b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:05:28.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.672+0000 7f5fc3e00640 1 --2- 172.21.15.159:0/4042529122 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fc41584b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4003670 tx=0x7f5fb4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:28.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.672+0000 7f5fbbe00640 1 -- 172.21.15.159:0/4042529122 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb402f4e0 con 0x7f5fa4006180 2024-04-04T06:05:28.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.672+0000 7f5fbbe00640 1 -- 172.21.15.159:0/4042529122 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb4039600 con 0x7f5fa4006180 2024-04-04T06:05:28.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.672+0000 7f5fbbe00640 1 -- 172.21.15.159:0/4042529122 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4039810 con 0x7f5fa4006180 2024-04-04T06:05:28.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.672+0000 7f5fcb000640 1 -- 172.21.15.159:0/4042529122 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc40b7650 con 0x7f5fa4006180 2024-04-04T06:05:28.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.672+0000 7f5fcb000640 1 -- 172.21.15.159:0/4042529122 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc40b7c10 con 0x7f5fa4006180 2024-04-04T06:05:28.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.673+0000 7f5fbbe00640 1 -- 172.21.15.159:0/4042529122 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5fb404a020 con 0x7f5fa4006180 2024-04-04T06:05:28.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.673+0000 7f5fbbe00640 1 -- 172.21.15.159:0/4042529122 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5fb40437a0 con 0x7f5fa4006180 2024-04-04T06:05:28.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.674+0000 7f5fcb000640 1 -- 172.21.15.159:0/4042529122 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fc40d2e60 con 0x7f5fa4006180 2024-04-04T06:05:28.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.679+0000 7f5fbbe00640 1 -- 172.21.15.159:0/4042529122 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5fc40d2e60 con 0x7f5fa4006180 2024-04-04T06:05:28.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.755+0000 7f5fcb000640 1 -- 172.21.15.159:0/4042529122 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f5fc40685c0 con 0x7f5fa4006180 2024-04-04T06:05:28.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.764+0000 7f5fbbe00640 1 -- 172.21.15.159:0/4042529122 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f5fb40399b0 con 0x7f5fa4006180 2024-04-04T06:05:28.759 INFO:teuthology.orchestra.run.smithi159.stderr:adjusted tunables profile to default 2024-04-04T06:05:28.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.766+0000 7f5fcb000640 1 -- 172.21.15.159:0/4042529122 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 msgr2=0x7f5fc41584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:28.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.766+0000 7f5fcb000640 1 --2- 172.21.15.159:0/4042529122 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fc41584b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4003670 tx=0x7f5fb4004730 comp rx=0 tx=0).stop 2024-04-04T06:05:28.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.766+0000 7f5fcb000640 1 -- 172.21.15.159:0/4042529122 shutdown_connections 2024-04-04T06:05:28.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.766+0000 7f5fcb000640 1 --2- 172.21.15.159:0/4042529122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fc415e860 0x7f5fc40b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:28.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.766+0000 7f5fcb000640 1 --2- 172.21.15.159:0/4042529122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fc40d4ae0 0x7f5fc40cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:28.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.766+0000 7f5fcb000640 1 --2- 172.21.15.159:0/4042529122 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5fa4006180 0x7f5fc41584b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:28.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.766+0000 7f5fcb000640 1 -- 172.21.15.159:0/4042529122 >> 172.21.15.159:0/4042529122 conn(0x7f5fc409d7d0 msgr2=0x7f5fc40b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:28.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.766+0000 7f5fcb000640 1 -- 172.21.15.159:0/4042529122 shutdown_connections 2024-04-04T06:05:28.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:28.766+0000 7f5fcb000640 1 -- 172.21.15.159:0/4042529122 wait complete. 2024-04-04T06:05:28.779 INFO:tasks.ceph:check_enable_crimson: False 2024-04-04T06:05:28.779 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-04T06:05:28.780 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:05:28.780 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-04T06:05:28.860 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:05:28.860 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-04T06:05:28.950 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:28.955+0000 7f0a7001a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-04T06:05:28.988 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:05:28.988 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-04T06:05:29.012 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:29.016+0000 7f0a7001a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-04T06:05:29.061 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:29.066+0000 7f0a7001a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-04T06:05:29.121 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:29.126+0000 7f0a7001a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-04T06:05:29.122 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:05:29.122 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-04T06:05:29.170 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:29.175+0000 7f0a7001a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-04T06:05:29.207 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:29.212+0000 7f88e803b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-04T06:05:29.250 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:05:29.251 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-04T06:05:29.270 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:29.275+0000 7f88e803b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-04T06:05:29.301 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:29.306+0000 7f0a7001a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-04T06:05:29.340 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:05:29.340 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-04T06:05:29.387 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:29.388+0000 7f0a7001a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-04T06:05:29.439 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:29.444+0000 7f0a7001a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-04T06:05:29.464 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:05:29.464 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-04T06:05:29.502 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:29.506+0000 7f0a7001a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-04T06:05:29.558 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:29.563+0000 7f0a7001a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-04T06:05:29.583 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:05:29.583 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-04T06:05:29.712 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 6fd6dfe9-0d71-40c2-a535-7bb5be04f685 0 2024-04-04T06:05:29.876 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:29.881+0000 7f88e803b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-04T06:05:29.939 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:29.944+0000 7f88e803b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-04T06:05:29.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.946+0000 7f9d4be00640 1 Processor -- start 2024-04-04T06:05:29.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.946+0000 7f9d4be00640 1 -- start start 2024-04-04T06:05:29.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.947+0000 7f9d4be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9d4c0d2310 0x7f9d4c0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:29.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.947+0000 7f9d4be00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d4c15e880 con 0x7f9d4c15e4a0 2024-04-04T06:05:29.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.948+0000 7f9d4be00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d4c0d28d0 con 0x7f9d4c0cf530 2024-04-04T06:05:29.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.948+0000 7f9d4be00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f9d4c0cf9a0 con 0x7f9d4c0d2310 2024-04-04T06:05:29.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.948+0000 7f9d4b400640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f9d4c0cf530 0x7f9d4c0b73e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:42906/0 (socket says 172.21.15.178:42906) 2024-04-04T06:05:29.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.948+0000 7f9d4b400640 1 -- 172.21.15.178:0/2109504125 learned_addr learned my addr 172.21.15.178:0/2109504125 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-04-04T06:05:29.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.948+0000 7f9d4a000640 1 --2- 172.21.15.178:0/2109504125 >> v2:172.21.15.159:3300/0 conn(0x7f9d4c0d2310 0x7f9d4c0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:29.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.948+0000 7f9d49600640 1 -- 172.21.15.178:0/2109504125 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2172618281 0 0) 0x7f9d4c0d28d0 con 0x7f9d4c0cf530 2024-04-04T06:05:29.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.948+0000 7f9d49600640 1 -- 172.21.15.178:0/2109504125 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d30003520 con 0x7f9d4c0cf530 2024-04-04T06:05:29.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.949+0000 7f9d49600640 1 -- 172.21.15.178:0/2109504125 <== mon.1 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 219425143 0 0) 0x7f9d30003520 con 0x7f9d4c0cf530 2024-04-04T06:05:29.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.949+0000 7f9d49600640 1 -- 172.21.15.178:0/2109504125 >> v2:172.21.15.159:3300/0 conn(0x7f9d4c0d2310 msgr2=0x7f9d4c0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:29.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.949+0000 7f9d49600640 1 --2- 172.21.15.178:0/2109504125 >> v2:172.21.15.159:3300/0 conn(0x7f9d4c0d2310 0x7f9d4c0ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:29.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.949+0000 7f9d49600640 1 -- 172.21.15.178:0/2109504125 >> v1:172.21.15.159:6789/0 conn(0x7f9d4c15e4a0 legacy=0x7f9d4c0b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:29.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.949+0000 7f9d49600640 1 -- 172.21.15.178:0/2109504125 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4c0d2bb0 con 0x7f9d4c0cf530 2024-04-04T06:05:29.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.949+0000 7f9d49600640 1 -- 172.21.15.178:0/2109504125 <== mon.1 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f9d440019f0 con 0x7f9d4c0cf530 2024-04-04T06:05:29.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.949+0000 7f9d49600640 1 -- 172.21.15.178:0/2109504125 >> v1:172.21.15.178:6789/0 conn(0x7f9d4c0cf530 legacy=0x7f9d4c0b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:29.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.949+0000 7f9d49600640 1 --2- 172.21.15.178:0/2109504125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d300039b0 0x7f9d30003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:29.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.949+0000 7f9d49600640 1 --2- 172.21.15.178:0/2109504125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d30004800 0x7f9d30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:29.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.949+0000 7f9d49600640 1 --2- 172.21.15.178:0/2109504125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d300051c0 0x7f9d3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:29.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.949+0000 7f9d49600640 1 -- 172.21.15.178:0/2109504125 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c0cf9a0 con 0x7f9d300051c0 2024-04-04T06:05:29.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.949+0000 7f9d49600640 1 -- 172.21.15.178:0/2109504125 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3000c160 con 0x7f9d300039b0 2024-04-04T06:05:29.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.949+0000 7f9d49600640 1 -- 172.21.15.178:0/2109504125 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9d3000c360 con 0x7f9d30004800 2024-04-04T06:05:29.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.949+0000 7f9d4aa00640 1 --2- 172.21.15.178:0/2109504125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d300039b0 0x7f9d30003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:29.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.949+0000 7f9d4a000640 1 --2- 172.21.15.178:0/2109504125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d30004800 0x7f9d30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:29.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.949+0000 7f9d4b400640 1 --2- 172.21.15.178:0/2109504125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d300051c0 0x7f9d3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:29.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.949+0000 7f9d4a000640 1 -- 172.21.15.178:0/2109504125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d300039b0 msgr2=0x7f9d30003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:29.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.949+0000 7f9d4a000640 1 --2- 172.21.15.178:0/2109504125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d300039b0 0x7f9d30003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:29.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.950+0000 7f9d4a000640 1 -- 172.21.15.178:0/2109504125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d300051c0 msgr2=0x7f9d3000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:29.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.950+0000 7f9d4a000640 1 --2- 172.21.15.178:0/2109504125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d300051c0 0x7f9d3000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:29.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.950+0000 7f9d4a000640 1 -- 172.21.15.178:0/2109504125 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d30003520 con 0x7f9d30004800 2024-04-04T06:05:29.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.950+0000 7f9d4b400640 1 -- 172.21.15.178:0/2109504125 reap_dead start 2024-04-04T06:05:29.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.951+0000 7f9d4a000640 1 --2- 172.21.15.178:0/2109504125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d30004800 0x7f9d30004c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009ac0 tx=0x7f9d30005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfeb91df45c0ff48 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:29.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.951+0000 7f9d49600640 1 -- 172.21.15.178:0/2109504125 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38039070 con 0x7f9d30004800 2024-04-04T06:05:29.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.951+0000 7f9d49600640 1 -- 172.21.15.178:0/2109504125 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d3802c6b0 con 0x7f9d30004800 2024-04-04T06:05:29.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.951+0000 7f9d49600640 1 -- 172.21.15.178:0/2109504125 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38009690 con 0x7f9d30004800 2024-04-04T06:05:29.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.951+0000 7f9d4be00640 1 -- 172.21.15.178:0/2109504125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d30004800 msgr2=0x7f9d30004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:29.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.951+0000 7f9d4be00640 1 --2- 172.21.15.178:0/2109504125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d30004800 0x7f9d30004c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009ac0 tx=0x7f9d30005830 comp rx=0 tx=0).stop 2024-04-04T06:05:29.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.951+0000 7f9d4be00640 1 -- 172.21.15.178:0/2109504125 shutdown_connections 2024-04-04T06:05:29.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.951+0000 7f9d4be00640 1 --2- 172.21.15.178:0/2109504125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d30004800 0x7f9d30004c80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:29.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.951+0000 7f9d4be00640 1 -- 172.21.15.178:0/2109504125 >> 172.21.15.178:0/2109504125 conn(0x7f9d4c0676d0 msgr2=0x7f9d4c067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:29.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.952+0000 7f9d4be00640 1 -- 172.21.15.178:0/2109504125 shutdown_connections 2024-04-04T06:05:29.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.952+0000 7f9d4be00640 1 -- 172.21.15.178:0/2109504125 wait complete. 2024-04-04T06:05:29.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.953+0000 7f9d4be00640 1 Processor -- start 2024-04-04T06:05:29.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.953+0000 7f9d4be00640 1 -- start start 2024-04-04T06:05:29.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.954+0000 7f9d4be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d30004800 0x7f9d4c156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:29.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.954+0000 7f9d4be00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d4c157330 0x7f9d4c15fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:29.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.954+0000 7f9d4be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d4c160170 0x7f9d4c158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:29.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.954+0000 7f9d4be00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c0d3170 con 0x7f9d30004800 2024-04-04T06:05:29.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.954+0000 7f9d4be00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c0d34e0 con 0x7f9d4c160170 2024-04-04T06:05:29.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.954+0000 7f9d4be00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9d4c0d3970 con 0x7f9d4c157330 2024-04-04T06:05:29.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.954+0000 7f9d4b400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d4c160170 0x7f9d4c158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:29.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.954+0000 7f9d4a000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d4c157330 0x7f9d4c15fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:29.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.954+0000 7f9d4b400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d4c160170 0x7f9d4c158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52720/0 (socket says 172.21.15.178:52720) 2024-04-04T06:05:29.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.955+0000 7f9d4b400640 1 -- 172.21.15.178:0/2698215795 learned_addr learned my addr 172.21.15.178:0/2698215795 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-04T06:05:29.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.955+0000 7f9d4aa00640 1 --2- 172.21.15.178:0/2698215795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d30004800 0x7f9d4c156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:29.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.955+0000 7f9d4b400640 1 -- 172.21.15.178:0/2698215795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d4c157330 msgr2=0x7f9d4c15fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:29.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.955+0000 7f9d4b400640 1 --2- 172.21.15.178:0/2698215795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d4c157330 0x7f9d4c15fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:29.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.955+0000 7f9d4b400640 1 -- 172.21.15.178:0/2698215795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d30004800 msgr2=0x7f9d4c156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:29.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.955+0000 7f9d4b400640 1 --2- 172.21.15.178:0/2698215795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d30004800 0x7f9d4c156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:29.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.955+0000 7f9d4b400640 1 -- 172.21.15.178:0/2698215795 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4c15eb90 con 0x7f9d4c160170 2024-04-04T06:05:29.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.955+0000 7f9d4b400640 1 --2- 172.21.15.178:0/2698215795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d4c160170 0x7f9d4c158170 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0d50f0 tx=0x7f9d44007bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:29.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.955+0000 7f9d42a00640 1 -- 172.21.15.178:0/2698215795 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4400ac40 con 0x7f9d4c160170 2024-04-04T06:05:29.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.955+0000 7f9d42a00640 1 -- 172.21.15.178:0/2698215795 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d44018070 con 0x7f9d4c160170 2024-04-04T06:05:29.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.955+0000 7f9d4be00640 1 -- 172.21.15.178:0/2698215795 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d4c15ee70 con 0x7f9d4c160170 2024-04-04T06:05:29.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.955+0000 7f9d42a00640 1 -- 172.21.15.178:0/2698215795 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d440133f0 con 0x7f9d4c160170 2024-04-04T06:05:29.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.956+0000 7f9d4be00640 1 -- 172.21.15.178:0/2698215795 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d4c15f430 con 0x7f9d4c160170 2024-04-04T06:05:29.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.956+0000 7f9d42a00640 1 -- 172.21.15.178:0/2698215795 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9d44013590 con 0x7f9d4c160170 2024-04-04T06:05:29.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.956+0000 7f9d42a00640 1 -- 172.21.15.178:0/2698215795 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f9d4401da20 con 0x7f9d4c160170 2024-04-04T06:05:29.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.957+0000 7f9d4be00640 1 -- 172.21.15.178:0/2698215795 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d4c075940 con 0x7f9d4c160170 2024-04-04T06:05:29.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:29.960+0000 7f9d42a00640 1 -- 172.21.15.178:0/2698215795 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9d4401d460 con 0x7f9d4c160170 2024-04-04T06:05:29.990 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:29.995+0000 7f88e803b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-04T06:05:30.028 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:30.032+0000 7f0a7001a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-04T06:05:30.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.038+0000 7f9d4be00640 1 -- 172.21.15.178:0/2698215795 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6fd6dfe9-0d71-40c2-a535-7bb5be04f685", "id": 0} v 0) -- 0x7f9d4c1561c0 con 0x7f9d4c160170 2024-04-04T06:05:30.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.047+0000 7f9d42a00640 1 -- 172.21.15.178:0/2698215795 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6fd6dfe9-0d71-40c2-a535-7bb5be04f685", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f9d4401dc20 con 0x7f9d4c160170 2024-04-04T06:05:30.042 INFO:teuthology.orchestra.run.smithi178.stdout:0 2024-04-04T06:05:30.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.048+0000 7f9d4be00640 1 -- 172.21.15.178:0/2698215795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d4c160170 msgr2=0x7f9d4c158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:30.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.048+0000 7f9d4be00640 1 --2- 172.21.15.178:0/2698215795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d4c160170 0x7f9d4c158170 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0d50f0 tx=0x7f9d44007bd0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.048+0000 7f9d4be00640 1 -- 172.21.15.178:0/2698215795 shutdown_connections 2024-04-04T06:05:30.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.048+0000 7f9d4be00640 1 --2- 172.21.15.178:0/2698215795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d4c160170 0x7f9d4c158170 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.048+0000 7f9d4be00640 1 --2- 172.21.15.178:0/2698215795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d4c157330 0x7f9d4c15fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.048+0000 7f9d4be00640 1 --2- 172.21.15.178:0/2698215795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d30004800 0x7f9d4c156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.048+0000 7f9d4be00640 1 -- 172.21.15.178:0/2698215795 >> 172.21.15.178:0/2698215795 conn(0x7f9d4c0676d0 msgr2=0x7f9d4c0d3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:30.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.048+0000 7f9d4be00640 1 -- 172.21.15.178:0/2698215795 shutdown_connections 2024-04-04T06:05:30.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.048+0000 7f9d4be00640 1 -- 172.21.15.178:0/2698215795 wait complete. 2024-04-04T06:05:30.052 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:30.057+0000 7f88e803b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-04T06:05:30.060 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 3c4ff6d4-2734-4993-a993-cb75cce59323 1 2024-04-04T06:05:30.101 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:30.106+0000 7f88e803b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-04T06:05:30.196 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:30.201+0000 7f0a7001a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-04T06:05:30.228 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:30.233+0000 7f88e803b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-04T06:05:30.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.246+0000 7f7d22000640 1 Processor -- start 2024-04-04T06:05:30.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.247+0000 7f7d22000640 1 -- start start 2024-04-04T06:05:30.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.248+0000 7f7d22000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7d1c15e860 0x7f7d1c06d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:30.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.248+0000 7f7d22000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7d1c06da30 0x7f7d1c0d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:30.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.248+0000 7f7d22000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d1c06dfa0 con 0x7f7d1c0d4910 2024-04-04T06:05:30.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.248+0000 7f7d22000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f7d1c0cf220 con 0x7f7d1c06da30 2024-04-04T06:05:30.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.248+0000 7f7d22000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7d1c0d4cf0 con 0x7f7d1c15e860 2024-04-04T06:05:30.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.248+0000 7f7d1b400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7d1c15e860 0x7f7d1c06d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:30.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.248+0000 7f7d1be00640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f7d1c0d4910 0x7f7d1c0d0e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:42914/0 (socket says 172.21.15.178:42914) 2024-04-04T06:05:30.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.248+0000 7f7d1be00640 1 -- 172.21.15.178:0/955237533 learned_addr learned my addr 172.21.15.178:0/955237533 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-04-04T06:05:30.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.249+0000 7f7d13e00640 1 --2- 172.21.15.178:0/955237533 >> v2:172.21.15.159:3300/0 conn(0x7f7d1c06da30 0x7f7d1c0d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:30.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.249+0000 7f7d1b400640 1 -- 172.21.15.178:0/955237533 >> v2:172.21.15.159:3300/0 conn(0x7f7d1c06da30 msgr2=0x7f7d1c0d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:30.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.249+0000 7f7d1b400640 1 --2- 172.21.15.178:0/955237533 >> v2:172.21.15.159:3300/0 conn(0x7f7d1c06da30 0x7f7d1c0d43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.249+0000 7f7d1b400640 1 -- 172.21.15.178:0/955237533 >> v1:172.21.15.178:6789/0 conn(0x7f7d1c0d4910 legacy=0x7f7d1c0d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:30.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.249+0000 7f7d1b400640 1 -- 172.21.15.178:0/955237533 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1c06e240 con 0x7f7d1c15e860 2024-04-04T06:05:30.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.249+0000 7f7d1aa00640 1 -- 172.21.15.178:0/955237533 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4104858525 0 0) 0x7f7d1c06dfa0 con 0x7f7d1c0d4910 2024-04-04T06:05:30.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.250+0000 7f7d1b400640 1 --2- 172.21.15.178:0/955237533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d1c15e860 0x7f7d1c06d4f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0d1fa0 tx=0x7f7d0c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b16b6ff3c35afbb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:30.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.250+0000 7f7d1aa00640 1 -- 172.21.15.178:0/955237533 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c035020 con 0x7f7d1c15e860 2024-04-04T06:05:30.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.250+0000 7f7d1aa00640 1 -- 172.21.15.178:0/955237533 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d0c031830 con 0x7f7d1c15e860 2024-04-04T06:05:30.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.250+0000 7f7d1aa00640 1 -- 172.21.15.178:0/955237533 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c031c00 con 0x7f7d1c15e860 2024-04-04T06:05:30.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.251+0000 7f7d22000640 1 -- 172.21.15.178:0/955237533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d1c15e860 msgr2=0x7f7d1c06d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:30.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.251+0000 7f7d22000640 1 --2- 172.21.15.178:0/955237533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d1c15e860 0x7f7d1c06d4f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0d1fa0 tx=0x7f7d0c02b8d0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.251+0000 7f7d22000640 1 -- 172.21.15.178:0/955237533 shutdown_connections 2024-04-04T06:05:30.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.251+0000 7f7d22000640 1 --2- 172.21.15.178:0/955237533 >> v2:172.21.15.159:3300/0 conn(0x7f7d1c06da30 0x7f7d1c0d43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.251+0000 7f7d22000640 1 --2- 172.21.15.178:0/955237533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d1c15e860 0x7f7d1c06d4f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.251+0000 7f7d22000640 1 -- 172.21.15.178:0/955237533 >> 172.21.15.178:0/955237533 conn(0x7f7d1c09d7d0 msgr2=0x7f7d1c08bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:30.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.251+0000 7f7d22000640 1 -- 172.21.15.178:0/955237533 shutdown_connections 2024-04-04T06:05:30.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.251+0000 7f7d22000640 1 -- 172.21.15.178:0/955237533 wait complete. 2024-04-04T06:05:30.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.252+0000 7f7d22000640 1 Processor -- start 2024-04-04T06:05:30.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.253+0000 7f7d22000640 1 -- start start 2024-04-04T06:05:30.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.253+0000 7f7d22000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d1c06da30 0x7f7d1c14e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:30.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.253+0000 7f7d22000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d1c0d4910 0x7f7d1c0cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:30.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.253+0000 7f7d22000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d1c15e860 0x7f7d1c0ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:30.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.253+0000 7f7d22000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c0cedb0 con 0x7f7d1c0d4910 2024-04-04T06:05:30.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.253+0000 7f7d22000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c0d2c50 con 0x7f7d1c15e860 2024-04-04T06:05:30.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.253+0000 7f7d22000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7d1c0cef30 con 0x7f7d1c06da30 2024-04-04T06:05:30.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.254+0000 7f7d1b400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d1c06da30 0x7f7d1c14e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:30.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.254+0000 7f7d1be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d1c15e860 0x7f7d1c0ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:30.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.254+0000 7f7d1be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d1c15e860 0x7f7d1c0ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52740/0 (socket says 172.21.15.178:52740) 2024-04-04T06:05:30.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.254+0000 7f7d1be00640 1 -- 172.21.15.178:0/3352595328 learned_addr learned my addr 172.21.15.178:0/3352595328 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-04T06:05:30.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.254+0000 7f7d1b400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d1c06da30 0x7f7d1c14e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:55542/0 (socket says 172.21.15.178:55542) 2024-04-04T06:05:30.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.254+0000 7f7d13e00640 1 --2- 172.21.15.178:0/3352595328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d1c0d4910 0x7f7d1c0cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:30.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.254+0000 7f7d1be00640 1 -- 172.21.15.178:0/3352595328 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d1c06da30 msgr2=0x7f7d1c14e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:30.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.254+0000 7f7d1be00640 1 --2- 172.21.15.178:0/3352595328 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d1c06da30 0x7f7d1c14e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.254+0000 7f7d1be00640 1 -- 172.21.15.178:0/3352595328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d1c0d4910 msgr2=0x7f7d1c0cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:30.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.254+0000 7f7d1be00640 1 --2- 172.21.15.178:0/3352595328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d1c0d4910 0x7f7d1c0cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.254+0000 7f7d1be00640 1 -- 172.21.15.178:0/3352595328 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1c0ba310 con 0x7f7d1c15e860 2024-04-04T06:05:30.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.254+0000 7f7d1be00640 1 --2- 172.21.15.178:0/3352595328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d1c15e860 0x7f7d1c0ca0d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400caf0 tx=0x7f7d1400a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:30.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.255+0000 7f7d13400640 1 -- 172.21.15.178:0/3352595328 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1400aca0 con 0x7f7d1c15e860 2024-04-04T06:05:30.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.255+0000 7f7d13400640 1 -- 172.21.15.178:0/3352595328 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d1400ae40 con 0x7f7d1c15e860 2024-04-04T06:05:30.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.255+0000 7f7d22000640 1 -- 172.21.15.178:0/3352595328 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1c0ce4c0 con 0x7f7d1c15e860 2024-04-04T06:05:30.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.255+0000 7f7d13400640 1 -- 172.21.15.178:0/3352595328 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14012620 con 0x7f7d1c15e860 2024-04-04T06:05:30.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.255+0000 7f7d22000640 1 -- 172.21.15.178:0/3352595328 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d1c0cba50 con 0x7f7d1c15e860 2024-04-04T06:05:30.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.255+0000 7f7d13400640 1 -- 172.21.15.178:0/3352595328 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7d14012880 con 0x7f7d1c15e860 2024-04-04T06:05:30.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.255+0000 7f7d13400640 1 -- 172.21.15.178:0/3352595328 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f7d1401bd40 con 0x7f7d1c15e860 2024-04-04T06:05:30.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.256+0000 7f7d22000640 1 -- 172.21.15.178:0/3352595328 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d1c0685c0 con 0x7f7d1c15e860 2024-04-04T06:05:30.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.260+0000 7f7d13400640 1 -- 172.21.15.178:0/3352595328 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7d1c0ba310 con 0x7f7d1c15e860 2024-04-04T06:05:30.308 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:30.313+0000 7f88e803b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-04T06:05:30.327 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:30.332+0000 7f0a7001a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-04T06:05:30.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.340+0000 7f7d22000640 1 -- 172.21.15.178:0/3352595328 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3c4ff6d4-2734-4993-a993-cb75cce59323", "id": 1} v 0) -- 0x7f7d1c0c9980 con 0x7f7d1c15e860 2024-04-04T06:05:30.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.347+0000 7f7d13400640 1 -- 172.21.15.178:0/3352595328 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3c4ff6d4-2734-4993-a993-cb75cce59323", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f7d14012d50 con 0x7f7d1c15e860 2024-04-04T06:05:30.342 INFO:teuthology.orchestra.run.smithi178.stdout:1 2024-04-04T06:05:30.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.348+0000 7f7d22000640 1 -- 172.21.15.178:0/3352595328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d1c15e860 msgr2=0x7f7d1c0ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:30.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.348+0000 7f7d22000640 1 --2- 172.21.15.178:0/3352595328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d1c15e860 0x7f7d1c0ca0d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400caf0 tx=0x7f7d1400a060 comp rx=0 tx=0).stop 2024-04-04T06:05:30.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.348+0000 7f7d22000640 1 -- 172.21.15.178:0/3352595328 shutdown_connections 2024-04-04T06:05:30.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.348+0000 7f7d22000640 1 --2- 172.21.15.178:0/3352595328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d1c15e860 0x7f7d1c0ca0d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.348+0000 7f7d22000640 1 --2- 172.21.15.178:0/3352595328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d1c0d4910 0x7f7d1c0cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.348+0000 7f7d22000640 1 --2- 172.21.15.178:0/3352595328 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d1c06da30 0x7f7d1c14e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.348+0000 7f7d22000640 1 -- 172.21.15.178:0/3352595328 >> 172.21.15.178:0/3352595328 conn(0x7f7d1c09d7d0 msgr2=0x7f7d1c0b5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:30.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.348+0000 7f7d22000640 1 -- 172.21.15.178:0/3352595328 shutdown_connections 2024-04-04T06:05:30.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.348+0000 7f7d22000640 1 -- 172.21.15.178:0/3352595328 wait complete. 2024-04-04T06:05:30.358 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new a00bdbab-b841-4e3e-88ea-c1bf744b3753 2 2024-04-04T06:05:30.362 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:30.367+0000 7f88e803b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-04T06:05:30.383 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:30.389+0000 7f0a7001a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-04T06:05:30.421 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:30.426+0000 7f88e803b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-04T06:05:30.474 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:30.479+0000 7f88e803b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-04T06:05:30.514 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:30.519+0000 7f0a7001a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-04T06:05:30.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.543+0000 7fd00d600640 1 Processor -- start 2024-04-04T06:05:30.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.544+0000 7fd00d600640 1 -- start start 2024-04-04T06:05:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.545+0000 7fd00d600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd00815e4a0 0x7fd0080b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.545+0000 7fd00d600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd00815e880 con 0x7fd0080d2310 2024-04-04T06:05:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.545+0000 7fd00d600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0080d28d0 con 0x7fd0080cf700 2024-04-04T06:05:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.545+0000 7fd00d600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd0080d34e0 con 0x7fd00815e4a0 2024-04-04T06:05:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.545+0000 7fd007e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd00815e4a0 0x7fd0080b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.545+0000 7fd00cc00640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fd0080cf700 0x7fd0080b73e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:42924/0 (socket says 172.21.15.178:42924) 2024-04-04T06:05:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.545+0000 7fd00cc00640 1 -- 172.21.15.178:0/3574929717 learned_addr learned my addr 172.21.15.178:0/3574929717 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-04-04T06:05:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.546+0000 7fd007e00640 1 -- 172.21.15.178:0/3574929717 >> v1:172.21.15.178:6789/0 conn(0x7fd0080cf700 legacy=0x7fd0080b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.546+0000 7fd007e00640 1 -- 172.21.15.178:0/3574929717 >> v1:172.21.15.159:6789/0 conn(0x7fd0080d2310 legacy=0x7fd0080ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:30.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.546+0000 7fd007e00640 1 -- 172.21.15.178:0/3574929717 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0080d2bb0 con 0x7fd00815e4a0 2024-04-04T06:05:30.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.547+0000 7fd007e00640 1 --2- 172.21.15.178:0/3574929717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd00815e4a0 0x7fd0080b9c60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd0080d20b0 tx=0x7fcff802b800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2860d29737f72ee server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:30.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.547+0000 7fd006a00640 1 -- 172.21.15.178:0/3574929717 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8031760 con 0x7fd00815e4a0 2024-04-04T06:05:30.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.547+0000 7fd006a00640 1 -- 172.21.15.178:0/3574929717 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff8037070 con 0x7fd00815e4a0 2024-04-04T06:05:30.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.547+0000 7fd006a00640 1 -- 172.21.15.178:0/3574929717 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8031b20 con 0x7fd00815e4a0 2024-04-04T06:05:30.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.548+0000 7fd00d600640 1 -- 172.21.15.178:0/3574929717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd00815e4a0 msgr2=0x7fd0080b9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:30.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.548+0000 7fd00d600640 1 --2- 172.21.15.178:0/3574929717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd00815e4a0 0x7fd0080b9c60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd0080d20b0 tx=0x7fcff802b800 comp rx=0 tx=0).stop 2024-04-04T06:05:30.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.548+0000 7fd00d600640 1 -- 172.21.15.178:0/3574929717 shutdown_connections 2024-04-04T06:05:30.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.548+0000 7fd00d600640 1 --2- 172.21.15.178:0/3574929717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd00815e4a0 0x7fd0080b9c60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.548+0000 7fd00d600640 1 -- 172.21.15.178:0/3574929717 >> 172.21.15.178:0/3574929717 conn(0x7fd0080676d0 msgr2=0x7fd008067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:30.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.548+0000 7fd00d600640 1 -- 172.21.15.178:0/3574929717 shutdown_connections 2024-04-04T06:05:30.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.548+0000 7fd00d600640 1 -- 172.21.15.178:0/3574929717 wait complete. 2024-04-04T06:05:30.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.550+0000 7fd00d600640 1 Processor -- start 2024-04-04T06:05:30.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.550+0000 7fd00d600640 1 -- start start 2024-04-04T06:05:30.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.551+0000 7fd00d600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0080cf700 0x7fd008152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:30.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.551+0000 7fd00d600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0080d2310 0x7fd0080c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:30.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.551+0000 7fd00d600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd00815e4a0 0x7fd0080cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:30.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.551+0000 7fd00d600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd0080d3970 con 0x7fd00815e4a0 2024-04-04T06:05:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.551+0000 7fd00d600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd0080d3170 con 0x7fd0080cf700 2024-04-04T06:05:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.551+0000 7fd00d600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd0080ba1a0 con 0x7fd0080d2310 2024-04-04T06:05:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.551+0000 7fd007e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0080cf700 0x7fd008152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.551+0000 7fd007400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0080d2310 0x7fd0080c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.551+0000 7fd007e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0080cf700 0x7fd008152f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52752/0 (socket says 172.21.15.178:52752) 2024-04-04T06:05:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.551+0000 7fd007e00640 1 -- 172.21.15.178:0/2897270479 learned_addr learned my addr 172.21.15.178:0/2897270479 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-04T06:05:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.551+0000 7fd00cc00640 1 --2- 172.21.15.178:0/2897270479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd00815e4a0 0x7fd0080cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.551+0000 7fd007400640 1 -- 172.21.15.178:0/2897270479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0080cf700 msgr2=0x7fd008152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.551+0000 7fd007400640 1 --2- 172.21.15.178:0/2897270479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0080cf700 0x7fd008152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.551+0000 7fd007400640 1 -- 172.21.15.178:0/2897270479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd00815e4a0 msgr2=0x7fd0080cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.551+0000 7fd007400640 1 --2- 172.21.15.178:0/2897270479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd00815e4a0 0x7fd0080cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.551+0000 7fd007400640 1 -- 172.21.15.178:0/2897270479 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd008153660 con 0x7fd0080d2310 2024-04-04T06:05:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.551+0000 7fd007e00640 1 --2- 172.21.15.178:0/2897270479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0080cf700 0x7fd008152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:05:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.552+0000 7fd007400640 1 --2- 172.21.15.178:0/2897270479 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0080d2310 0x7fd0080c9750 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcff400c410 tx=0x7fcff400c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:30.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.552+0000 7fcfffe00640 1 -- 172.21.15.178:0/2897270479 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff4017070 con 0x7fd0080d2310 2024-04-04T06:05:30.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.552+0000 7fcfffe00640 1 -- 172.21.15.178:0/2897270479 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff4005230 con 0x7fd0080d2310 2024-04-04T06:05:30.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.552+0000 7fcfffe00640 1 -- 172.21.15.178:0/2897270479 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff40045e0 con 0x7fd0080d2310 2024-04-04T06:05:30.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.552+0000 7fd00d600640 1 -- 172.21.15.178:0/2897270479 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd008157a80 con 0x7fd0080d2310 2024-04-04T06:05:30.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.552+0000 7fd00d600640 1 -- 172.21.15.178:0/2897270479 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0081538d0 con 0x7fd0080d2310 2024-04-04T06:05:30.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.552+0000 7fcfffe00640 1 -- 172.21.15.178:0/2897270479 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcff401a070 con 0x7fd0080d2310 2024-04-04T06:05:30.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.552+0000 7fcfffe00640 1 -- 172.21.15.178:0/2897270479 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fcff4007440 con 0x7fd0080d2310 2024-04-04T06:05:30.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.553+0000 7fd00d600640 1 -- 172.21.15.178:0/2897270479 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd008153ad0 con 0x7fd0080d2310 2024-04-04T06:05:30.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.555+0000 7fcfffe00640 1 -- 172.21.15.178:0/2897270479 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd008153ad0 con 0x7fd0080d2310 2024-04-04T06:05:30.629 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:30.634+0000 7f0a7001a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-04T06:05:30.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.634+0000 7fd00d600640 1 -- 172.21.15.178:0/2897270479 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a00bdbab-b841-4e3e-88ea-c1bf744b3753", "id": 2} v 0) -- 0x7fd008075940 con 0x7fd0080d2310 2024-04-04T06:05:30.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.642+0000 7fcfffe00640 1 -- 172.21.15.178:0/2897270479 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a00bdbab-b841-4e3e-88ea-c1bf744b3753", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fcff40067d0 con 0x7fd0080d2310 2024-04-04T06:05:30.637 INFO:teuthology.orchestra.run.smithi178.stdout:2 2024-04-04T06:05:30.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.643+0000 7fd00d600640 1 -- 172.21.15.178:0/2897270479 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0080d2310 msgr2=0x7fd0080c9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:30.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.643+0000 7fd00d600640 1 --2- 172.21.15.178:0/2897270479 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0080d2310 0x7fd0080c9750 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcff400c410 tx=0x7fcff400c8d0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.644+0000 7fd00d600640 1 -- 172.21.15.178:0/2897270479 shutdown_connections 2024-04-04T06:05:30.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.644+0000 7fd00d600640 1 --2- 172.21.15.178:0/2897270479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd00815e4a0 0x7fd0080cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.644+0000 7fd00d600640 1 --2- 172.21.15.178:0/2897270479 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd0080d2310 0x7fd0080c9750 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.644+0000 7fd00d600640 1 --2- 172.21.15.178:0/2897270479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0080cf700 0x7fd008152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.644+0000 7fd00d600640 1 -- 172.21.15.178:0/2897270479 >> 172.21.15.178:0/2897270479 conn(0x7fd0080676d0 msgr2=0x7fd0080b93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:30.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.644+0000 7fd00d600640 1 -- 172.21.15.178:0/2897270479 shutdown_connections 2024-04-04T06:05:30.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.644+0000 7fd00d600640 1 -- 172.21.15.178:0/2897270479 wait complete. 2024-04-04T06:05:30.662 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 34e5c351-621f-4a57-9c6a-24582faca139 3 2024-04-04T06:05:30.820 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:30.825+0000 7f0a7001a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-04T06:05:30.872 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:30.877+0000 7f0a7001a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-04T06:05:30.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.895+0000 7f35e6e00640 1 Processor -- start 2024-04-04T06:05:30.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.895+0000 7f35e6e00640 1 -- start start 2024-04-04T06:05:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.897+0000 7f35e6e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e015e440 0x7f35e00b9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.897+0000 7f35e6e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35e00d2d80 con 0x7f35e00d3e20 2024-04-04T06:05:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.897+0000 7f35e6e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35e00d3060 con 0x7f35e00cef00 2024-04-04T06:05:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.897+0000 7f35e6e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f35e00d3340 con 0x7f35e015e440 2024-04-04T06:05:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.897+0000 7f35e5a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e015e440 0x7f35e00b9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.897+0000 7f35e6400640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f35e00cef00 0x7f35e00b72f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:42932/0 (socket says 172.21.15.178:42932) 2024-04-04T06:05:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.897+0000 7f35e6400640 1 -- 172.21.15.178:0/375363913 learned_addr learned my addr 172.21.15.178:0/375363913 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-04-04T06:05:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.897+0000 7f35e5a00640 1 --2- 172.21.15.178:0/375363913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e015e440 0x7f35e00b9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:55572/0 (socket says 172.21.15.178:55572) 2024-04-04T06:05:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.898+0000 7f35e5a00640 1 -- 172.21.15.178:0/375363913 >> v1:172.21.15.178:6789/0 conn(0x7f35e00cef00 legacy=0x7f35e00b72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.898+0000 7f35dfe00640 1 -- 172.21.15.178:0/375363913 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1702741936 0 0) 0x7f35e00d3060 con 0x7f35e00cef00 2024-04-04T06:05:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.898+0000 7f35e5a00640 1 -- 172.21.15.178:0/375363913 >> v1:172.21.15.159:6789/0 conn(0x7f35e00d3e20 legacy=0x7f35e00d2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:30.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.898+0000 7f35e5a00640 1 -- 172.21.15.178:0/375363913 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e00d3620 con 0x7f35e015e440 2024-04-04T06:05:30.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.899+0000 7f35e5a00640 1 --2- 172.21.15.178:0/375363913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e015e440 0x7f35e00b9e00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f35d40097d0 tx=0x7f35d402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f891de10fed0b505 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:30.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.899+0000 7f35dfe00640 1 -- 172.21.15.178:0/375363913 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4035020 con 0x7f35e015e440 2024-04-04T06:05:30.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.899+0000 7f35dfe00640 1 -- 172.21.15.178:0/375363913 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d4031800 con 0x7f35e015e440 2024-04-04T06:05:30.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.899+0000 7f35dfe00640 1 -- 172.21.15.178:0/375363913 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4031bd0 con 0x7f35e015e440 2024-04-04T06:05:30.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.900+0000 7f35e6e00640 1 -- 172.21.15.178:0/375363913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e015e440 msgr2=0x7f35e00b9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:30.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.900+0000 7f35e6e00640 1 --2- 172.21.15.178:0/375363913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e015e440 0x7f35e00b9e00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f35d40097d0 tx=0x7f35d402b8a0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.900+0000 7f35e6e00640 1 -- 172.21.15.178:0/375363913 shutdown_connections 2024-04-04T06:05:30.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.900+0000 7f35e6e00640 1 --2- 172.21.15.178:0/375363913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e015e440 0x7f35e00b9e00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.900+0000 7f35e6e00640 1 -- 172.21.15.178:0/375363913 >> 172.21.15.178:0/375363913 conn(0x7f35e00678b0 msgr2=0x7f35e00d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:30.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.900+0000 7f35e6e00640 1 -- 172.21.15.178:0/375363913 shutdown_connections 2024-04-04T06:05:30.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.900+0000 7f35e6e00640 1 -- 172.21.15.178:0/375363913 wait complete. 2024-04-04T06:05:30.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.902+0000 7f35e6e00640 1 Processor -- start 2024-04-04T06:05:30.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.902+0000 7f35e6e00640 1 -- start start 2024-04-04T06:05:30.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.903+0000 7f35e6e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e00cef00 0x7f35e01e4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:30.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.903+0000 7f35e6e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e00d3e20 0x7f35e01e3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:30.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.903+0000 7f35e6e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e015e440 0x7f35e015d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:30.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.903+0000 7f35e6e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f35e00cf510 con 0x7f35e015e440 2024-04-04T06:05:30.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.903+0000 7f35e6e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35e00cf390 con 0x7f35e00d3e20 2024-04-04T06:05:30.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.903+0000 7f35e6e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f35e00cf690 con 0x7f35e00cef00 2024-04-04T06:05:30.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.903+0000 7f35e5000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e00d3e20 0x7f35e01e3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:30.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.903+0000 7f35e5a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e00cef00 0x7f35e01e4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:30.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.903+0000 7f35e5a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e00cef00 0x7f35e01e4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:55574/0 (socket says 172.21.15.178:55574) 2024-04-04T06:05:30.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.903+0000 7f35e5000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e00d3e20 0x7f35e01e3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52768/0 (socket says 172.21.15.178:52768) 2024-04-04T06:05:30.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.903+0000 7f35e5a00640 1 -- 172.21.15.178:0/1957523210 learned_addr learned my addr 172.21.15.178:0/1957523210 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-04T06:05:30.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.903+0000 7f35e6400640 1 --2- 172.21.15.178:0/1957523210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e015e440 0x7f35e015d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:30.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.903+0000 7f35e5a00640 1 -- 172.21.15.178:0/1957523210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e00d3e20 msgr2=0x7f35e01e3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:30.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.903+0000 7f35e5a00640 1 --2- 172.21.15.178:0/1957523210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e00d3e20 0x7f35e01e3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.904+0000 7f35e5a00640 1 -- 172.21.15.178:0/1957523210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e015e440 msgr2=0x7f35e015d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:30.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.904+0000 7f35e5a00640 1 --2- 172.21.15.178:0/1957523210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e015e440 0x7f35e015d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:30.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.904+0000 7f35e5a00640 1 -- 172.21.15.178:0/1957523210 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e01e3e30 con 0x7f35e00cef00 2024-04-04T06:05:30.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.904+0000 7f35e5a00640 1 --2- 172.21.15.178:0/1957523210 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e00cef00 0x7f35e01e4ed0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f35d4034040 tx=0x7f35d40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:30.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.904+0000 7f35dd600640 1 -- 172.21.15.178:0/1957523210 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d403bb60 con 0x7f35e00cef00 2024-04-04T06:05:30.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.904+0000 7f35dd600640 1 -- 172.21.15.178:0/1957523210 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d403bd20 con 0x7f35e00cef00 2024-04-04T06:05:30.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.904+0000 7f35dd600640 1 -- 172.21.15.178:0/1957523210 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4004270 con 0x7f35e00cef00 2024-04-04T06:05:30.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.904+0000 7f35e6e00640 1 -- 172.21.15.178:0/1957523210 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e01e4110 con 0x7f35e00cef00 2024-04-04T06:05:30.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.904+0000 7f35e6e00640 1 -- 172.21.15.178:0/1957523210 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e01e46d0 con 0x7f35e00cef00 2024-04-04T06:05:30.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.904+0000 7f35dd600640 1 -- 172.21.15.178:0/1957523210 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f35d404c060 con 0x7f35e00cef00 2024-04-04T06:05:30.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.905+0000 7f35dd600640 1 -- 172.21.15.178:0/1957523210 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f35d4045410 con 0x7f35e00cef00 2024-04-04T06:05:30.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.906+0000 7f35e6e00640 1 -- 172.21.15.178:0/1957523210 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e0075940 con 0x7f35e00cef00 2024-04-04T06:05:30.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:30.908+0000 7f35dd600640 1 -- 172.21.15.178:0/1957523210 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f35d4040070 con 0x7f35e00cef00 2024-04-04T06:05:30.921 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:30.926+0000 7f88e803b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-04T06:05:31.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.009+0000 7f35e6e00640 1 -- 172.21.15.178:0/1957523210 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd new", "uuid": "34e5c351-621f-4a57-9c6a-24582faca139", "id": 3} v 0) -- 0x7f35e00d0fc0 con 0x7f35e00cef00 2024-04-04T06:05:31.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.017+0000 7f35dd600640 1 -- 172.21.15.178:0/1957523210 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "34e5c351-621f-4a57-9c6a-24582faca139", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f35d40039a0 con 0x7f35e00cef00 2024-04-04T06:05:31.012 INFO:teuthology.orchestra.run.smithi178.stdout:3 2024-04-04T06:05:31.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.018+0000 7f35e6e00640 1 -- 172.21.15.178:0/1957523210 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e00cef00 msgr2=0x7f35e01e4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:31.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.018+0000 7f35e6e00640 1 --2- 172.21.15.178:0/1957523210 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e00cef00 0x7f35e01e4ed0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f35d4034040 tx=0x7f35d40046c0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.018+0000 7f35e6e00640 1 -- 172.21.15.178:0/1957523210 shutdown_connections 2024-04-04T06:05:31.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.018+0000 7f35e6e00640 1 --2- 172.21.15.178:0/1957523210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e015e440 0x7f35e015d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.018+0000 7f35e6e00640 1 --2- 172.21.15.178:0/1957523210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e00d3e20 0x7f35e01e3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.018+0000 7f35e6e00640 1 --2- 172.21.15.178:0/1957523210 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e00cef00 0x7f35e01e4ed0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.018+0000 7f35e6e00640 1 -- 172.21.15.178:0/1957523210 >> 172.21.15.178:0/1957523210 conn(0x7f35e00678b0 msgr2=0x7f35e00d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:31.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.018+0000 7f35e6e00640 1 -- 172.21.15.178:0/1957523210 shutdown_connections 2024-04-04T06:05:31.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.018+0000 7f35e6e00640 1 -- 172.21.15.178:0/1957523210 wait complete. 2024-04-04T06:05:31.030 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 2e340e67-8903-40e4-87aa-8e03b538ad9d 4 2024-04-04T06:05:31.074 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:31.079+0000 7f0a7001a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-04T06:05:31.080 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:31.086+0000 7f88e803b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-04T06:05:31.210 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:31.215+0000 7f88e803b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-04T06:05:31.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.226+0000 7f81a8600640 1 Processor -- start 2024-04-04T06:05:31.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.226+0000 7f81a8600640 1 -- start start 2024-04-04T06:05:31.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.228+0000 7f81a8600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f81a00b4db0 0x7f81a00cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:31.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.228+0000 7f81a8600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81a00b5ce0 con 0x7f81a0161f60 2024-04-04T06:05:31.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.228+0000 7f81a8600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81a00d27f0 con 0x7f81a00d1580 2024-04-04T06:05:31.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.228+0000 7f81a8600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f81a00d2a90 con 0x7f81a00b4db0 2024-04-04T06:05:31.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.228+0000 7f81a5200640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f81a00d1580 0x7f81a00cf0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:42934/0 (socket says 172.21.15.178:42934) 2024-04-04T06:05:31.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.228+0000 7f81a5200640 1 -- 172.21.15.178:0/1291028889 learned_addr learned my addr 172.21.15.178:0/1291028889 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-04-04T06:05:31.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.228+0000 7f81a6600640 1 --2- 172.21.15.178:0/1291028889 >> v2:172.21.15.159:3300/0 conn(0x7f81a00b4db0 0x7f81a00cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:31.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.228+0000 7f819fe00640 1 -- 172.21.15.178:0/1291028889 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1839722068 0 0) 0x7f81a00d27f0 con 0x7f81a00d1580 2024-04-04T06:05:31.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.228+0000 7f819fe00640 1 -- 172.21.15.178:0/1291028889 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8188003520 con 0x7f81a00d1580 2024-04-04T06:05:31.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.229+0000 7f819fe00640 1 -- 172.21.15.178:0/1291028889 <== mon.1 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 911926663 0 0) 0x7f8188003520 con 0x7f81a00d1580 2024-04-04T06:05:31.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.229+0000 7f819fe00640 1 -- 172.21.15.178:0/1291028889 >> v2:172.21.15.159:3300/0 conn(0x7f81a00b4db0 msgr2=0x7f81a00cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:31.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.229+0000 7f819fe00640 1 --2- 172.21.15.178:0/1291028889 >> v2:172.21.15.159:3300/0 conn(0x7f81a00b4db0 0x7f81a00cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.229+0000 7f819fe00640 1 -- 172.21.15.178:0/1291028889 >> v1:172.21.15.159:6789/0 conn(0x7f81a0161f60 legacy=0x7f81a00d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:31.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.229+0000 7f819fe00640 1 -- 172.21.15.178:0/1291028889 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a0162340 con 0x7f81a00d1580 2024-04-04T06:05:31.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.229+0000 7f819fe00640 1 -- 172.21.15.178:0/1291028889 <== mon.1 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f818c001670 con 0x7f81a00d1580 2024-04-04T06:05:31.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.229+0000 7f819fe00640 1 -- 172.21.15.178:0/1291028889 >> v1:172.21.15.178:6789/0 conn(0x7f81a00d1580 legacy=0x7f81a00cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:31.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.229+0000 7f819fe00640 1 --2- 172.21.15.178:0/1291028889 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8188003a00 0x7f8188003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:31.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.229+0000 7f819fe00640 1 --2- 172.21.15.178:0/1291028889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8188004800 0x7f8188004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:31.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.229+0000 7f819fe00640 1 --2- 172.21.15.178:0/1291028889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81880051c0 0x7f818800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:31.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.229+0000 7f819fe00640 1 -- 172.21.15.178:0/1291028889 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f81a00d2a90 con 0x7f81880051c0 2024-04-04T06:05:31.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.229+0000 7f819fe00640 1 -- 172.21.15.178:0/1291028889 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f818800c0c0 con 0x7f8188004800 2024-04-04T06:05:31.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.229+0000 7f819fe00640 1 -- 172.21.15.178:0/1291028889 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f818800c2c0 con 0x7f8188003a00 2024-04-04T06:05:31.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.229+0000 7f81a5c00640 1 --2- 172.21.15.178:0/1291028889 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8188003a00 0x7f8188003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:31.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.229+0000 7f81a5200640 1 --2- 172.21.15.178:0/1291028889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8188004800 0x7f8188004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:31.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.229+0000 7f81a6600640 1 --2- 172.21.15.178:0/1291028889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81880051c0 0x7f818800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:31.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.230+0000 7f81a5c00640 1 -- 172.21.15.178:0/1291028889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8188004800 msgr2=0x7f8188004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:31.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.230+0000 7f81a5c00640 1 --2- 172.21.15.178:0/1291028889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8188004800 0x7f8188004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.230+0000 7f81a5c00640 1 -- 172.21.15.178:0/1291028889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81880051c0 msgr2=0x7f818800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:31.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.230+0000 7f81a5c00640 1 --2- 172.21.15.178:0/1291028889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81880051c0 0x7f818800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.230+0000 7f81a5c00640 1 -- 172.21.15.178:0/1291028889 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8188003520 con 0x7f8188003a00 2024-04-04T06:05:31.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.230+0000 7f81a6600640 1 --2- 172.21.15.178:0/1291028889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81880051c0 0x7f818800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:05:31.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.230+0000 7f81a6600640 1 -- 172.21.15.178:0/1291028889 reap_dead start 2024-04-04T06:05:31.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.231+0000 7f81a5c00640 1 --2- 172.21.15.178:0/1291028889 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8188003a00 0x7f8188003e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8194001750 tx=0x7f8188005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97cc8ffb0ed176ce server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:31.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.231+0000 7f819fe00640 1 -- 172.21.15.178:0/1291028889 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194001370 con 0x7f8188003a00 2024-04-04T06:05:31.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.231+0000 7f819fe00640 1 -- 172.21.15.178:0/1291028889 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81940052d0 con 0x7f8188003a00 2024-04-04T06:05:31.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.231+0000 7f819fe00640 1 -- 172.21.15.178:0/1291028889 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194003830 con 0x7f8188003a00 2024-04-04T06:05:31.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.232+0000 7f81a8600640 1 -- 172.21.15.178:0/1291028889 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8188003a00 msgr2=0x7f8188003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:31.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.232+0000 7f81a8600640 1 --2- 172.21.15.178:0/1291028889 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8188003a00 0x7f8188003e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8194001750 tx=0x7f8188005830 comp rx=0 tx=0).stop 2024-04-04T06:05:31.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.232+0000 7f81a8600640 1 -- 172.21.15.178:0/1291028889 shutdown_connections 2024-04-04T06:05:31.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.232+0000 7f81a8600640 1 --2- 172.21.15.178:0/1291028889 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8188003a00 0x7f8188003e20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.232+0000 7f81a8600640 1 -- 172.21.15.178:0/1291028889 >> 172.21.15.178:0/1291028889 conn(0x7f81a008bfb0 msgr2=0x7f81a00d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:31.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.232+0000 7f81a8600640 1 -- 172.21.15.178:0/1291028889 shutdown_connections 2024-04-04T06:05:31.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.232+0000 7f81a8600640 1 -- 172.21.15.178:0/1291028889 wait complete. 2024-04-04T06:05:31.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.235+0000 7f81a8600640 1 Processor -- start 2024-04-04T06:05:31.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.235+0000 7f81a8600640 1 -- start start 2024-04-04T06:05:31.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.236+0000 7f81a8600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8188003a00 0x7f81a00cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:31.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.236+0000 7f81a8600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81a00b8920 0x7f81a015deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:31.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.236+0000 7f81a8600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81a01f3830 0x7f81a00cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:31.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.236+0000 7f81a8600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f81a00d3310 con 0x7f8188003a00 2024-04-04T06:05:31.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.236+0000 7f81a8600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f81a00d3030 con 0x7f81a01f3830 2024-04-04T06:05:31.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.236+0000 7f81a8600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f81a00b3170 con 0x7f81a00b8920 2024-04-04T06:05:31.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.236+0000 7f81a5200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81a00b8920 0x7f81a015deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:31.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.236+0000 7f81a6600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81a01f3830 0x7f81a00cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:31.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.236+0000 7f81a5200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81a00b8920 0x7f81a015deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:55588/0 (socket says 172.21.15.178:55588) 2024-04-04T06:05:31.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.237+0000 7f81a5200640 1 -- 172.21.15.178:0/2795725121 learned_addr learned my addr 172.21.15.178:0/2795725121 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-04T06:05:31.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.237+0000 7f81a5c00640 1 --2- 172.21.15.178:0/2795725121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8188003a00 0x7f81a00cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:31.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.237+0000 7f81a6600640 1 -- 172.21.15.178:0/2795725121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81a00b8920 msgr2=0x7f81a015deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:31.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.237+0000 7f81a6600640 1 --2- 172.21.15.178:0/2795725121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81a00b8920 0x7f81a015deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.237+0000 7f81a6600640 1 -- 172.21.15.178:0/2795725121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8188003a00 msgr2=0x7f81a00cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:31.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.237+0000 7f81a6600640 1 --2- 172.21.15.178:0/2795725121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8188003a00 0x7f81a00cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.237+0000 7f81a6600640 1 -- 172.21.15.178:0/2795725121 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a00ce3a0 con 0x7f81a01f3830 2024-04-04T06:05:31.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.237+0000 7f81a6600640 1 --2- 172.21.15.178:0/2795725121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81a01f3830 0x7f81a00cd600 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f81a008c980 tx=0x7f8198010590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:31.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.237+0000 7f819d600640 1 -- 172.21.15.178:0/2795725121 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198010da0 con 0x7f81a01f3830 2024-04-04T06:05:31.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.237+0000 7f819d600640 1 -- 172.21.15.178:0/2795725121 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8198003990 con 0x7f81a01f3830 2024-04-04T06:05:31.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.237+0000 7f81a8600640 1 -- 172.21.15.178:0/2795725121 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a00cc520 con 0x7f81a01f3830 2024-04-04T06:05:31.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.237+0000 7f819d600640 1 -- 172.21.15.178:0/2795725121 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198003d80 con 0x7f81a01f3830 2024-04-04T06:05:31.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.238+0000 7f81a8600640 1 -- 172.21.15.178:0/2795725121 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a00c8190 con 0x7f81a01f3830 2024-04-04T06:05:31.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.238+0000 7f819d600640 1 -- 172.21.15.178:0/2795725121 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f819801e020 con 0x7f81a01f3830 2024-04-04T06:05:31.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.238+0000 7f819d600640 1 -- 172.21.15.178:0/2795725121 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f81980047c0 con 0x7f81a01f3830 2024-04-04T06:05:31.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.239+0000 7f81a8600640 1 -- 172.21.15.178:0/2795725121 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a00cca20 con 0x7f81a01f3830 2024-04-04T06:05:31.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.241+0000 7f819d600640 1 -- 172.21.15.178:0/2795725121 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f81980042b0 con 0x7f81a01f3830 2024-04-04T06:05:31.263 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:31.268+0000 7f88e803b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-04T06:05:31.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.320+0000 7f81a8600640 1 -- 172.21.15.178:0/2795725121 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2e340e67-8903-40e4-87aa-8e03b538ad9d", "id": 4} v 0) -- 0x7f81a00685c0 con 0x7f81a01f3830 2024-04-04T06:05:31.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.327+0000 7f819d600640 1 -- 172.21.15.178:0/2795725121 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2e340e67-8903-40e4-87aa-8e03b538ad9d", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f8198004490 con 0x7f81a01f3830 2024-04-04T06:05:31.322 INFO:teuthology.orchestra.run.smithi178.stdout:4 2024-04-04T06:05:31.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.329+0000 7f81a8600640 1 -- 172.21.15.178:0/2795725121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81a01f3830 msgr2=0x7f81a00cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:31.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.329+0000 7f81a8600640 1 --2- 172.21.15.178:0/2795725121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81a01f3830 0x7f81a00cd600 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f81a008c980 tx=0x7f8198010590 comp rx=0 tx=0).stop 2024-04-04T06:05:31.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.329+0000 7f81a8600640 1 -- 172.21.15.178:0/2795725121 shutdown_connections 2024-04-04T06:05:31.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.329+0000 7f81a8600640 1 --2- 172.21.15.178:0/2795725121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81a01f3830 0x7f81a00cd600 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.329+0000 7f81a8600640 1 --2- 172.21.15.178:0/2795725121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81a00b8920 0x7f81a015deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.329+0000 7f81a8600640 1 --2- 172.21.15.178:0/2795725121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8188003a00 0x7f81a00cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.329+0000 7f81a8600640 1 -- 172.21.15.178:0/2795725121 >> 172.21.15.178:0/2795725121 conn(0x7f81a008bfb0 msgr2=0x7f81a006d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:31.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.329+0000 7f81a8600640 1 -- 172.21.15.178:0/2795725121 shutdown_connections 2024-04-04T06:05:31.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.329+0000 7f81a8600640 1 -- 172.21.15.178:0/2795725121 wait complete. 2024-04-04T06:05:31.340 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 9d0aead7-b517-4d97-b37e-747b7fc1c03d 5 2024-04-04T06:05:31.389 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:31.394+0000 7f88e803b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-04T06:05:31.497 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:31.502+0000 7f88e803b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-04T06:05:31.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.575+0000 7fb43b400640 1 Processor -- start 2024-04-04T06:05:31.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.575+0000 7fb43b400640 1 -- start start 2024-04-04T06:05:31.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.576+0000 7fb43b400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb4340cf050 0x7fb43415f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:31.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.576+0000 7fb43b400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4340d2110 0x7fb4340d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:31.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.576+0000 7fb43b400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4340cf6c0 con 0x7fb4340d0e70 2024-04-04T06:05:31.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.577+0000 7fb43b400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fb43415fc20 con 0x7fb4340cf050 2024-04-04T06:05:31.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.577+0000 7fb43b400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb4340cf9a0 con 0x7fb4340d2110 2024-04-04T06:05:31.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.577+0000 7fb439400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4340d2110 0x7fb4340d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:31.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.577+0000 7fb439400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4340d2110 0x7fb4340d25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:55604/0 (socket says 172.21.15.178:55604) 2024-04-04T06:05:31.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.577+0000 7fb439400640 1 -- 172.21.15.178:0/2312408173 learned_addr learned my addr 172.21.15.178:0/2312408173 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-04T06:05:31.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.577+0000 7fb433e00640 1 --2- 172.21.15.178:0/2312408173 >> v2:172.21.15.159:3300/0 conn(0x7fb4340cf050 0x7fb43415f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:31.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.577+0000 7fb439400640 1 -- 172.21.15.178:0/2312408173 >> v2:172.21.15.159:3300/0 conn(0x7fb4340cf050 msgr2=0x7fb43415f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:31.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.577+0000 7fb439400640 1 --2- 172.21.15.178:0/2312408173 >> v2:172.21.15.159:3300/0 conn(0x7fb4340cf050 0x7fb43415f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.577+0000 7fb439400640 1 -- 172.21.15.178:0/2312408173 >> v1:172.21.15.159:6789/0 conn(0x7fb4340d0e70 legacy=0x7fb4340d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:31.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.577+0000 7fb439400640 1 -- 172.21.15.178:0/2312408173 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4340cfc80 con 0x7fb4340d2110 2024-04-04T06:05:31.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.577+0000 7fb433e00640 1 --2- 172.21.15.178:0/2312408173 >> v2:172.21.15.159:3300/0 conn(0x7fb4340cf050 0x7fb43415f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:05:31.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.578+0000 7fb439400640 1 --2- 172.21.15.178:0/2312408173 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4340d2110 0x7fb4340d25a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009e50 tx=0x7fb42c02bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=336e64b9db7503d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:31.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.578+0000 7fb433400640 1 -- 172.21.15.178:0/2312408173 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c02ca80 con 0x7fb4340d2110 2024-04-04T06:05:31.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.578+0000 7fb433400640 1 -- 172.21.15.178:0/2312408173 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb42c02cc20 con 0x7fb4340d2110 2024-04-04T06:05:31.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.579+0000 7fb433400640 1 -- 172.21.15.178:0/2312408173 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c0343f0 con 0x7fb4340d2110 2024-04-04T06:05:31.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.579+0000 7fb43b400640 1 -- 172.21.15.178:0/2312408173 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4340d2110 msgr2=0x7fb4340d25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:31.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.579+0000 7fb43b400640 1 --2- 172.21.15.178:0/2312408173 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4340d2110 0x7fb4340d25a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009e50 tx=0x7fb42c02bee0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.579+0000 7fb43b400640 1 -- 172.21.15.178:0/2312408173 shutdown_connections 2024-04-04T06:05:31.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.579+0000 7fb43b400640 1 --2- 172.21.15.178:0/2312408173 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4340d2110 0x7fb4340d25a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.579+0000 7fb43b400640 1 --2- 172.21.15.178:0/2312408173 >> v2:172.21.15.159:3300/0 conn(0x7fb4340cf050 0x7fb43415f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.579+0000 7fb43b400640 1 -- 172.21.15.178:0/2312408173 >> 172.21.15.178:0/2312408173 conn(0x7fb43408bfb0 msgr2=0x7fb4340d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:31.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.579+0000 7fb43b400640 1 -- 172.21.15.178:0/2312408173 shutdown_connections 2024-04-04T06:05:31.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.579+0000 7fb43b400640 1 -- 172.21.15.178:0/2312408173 wait complete. 2024-04-04T06:05:31.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.581+0000 7fb43b400640 1 Processor -- start 2024-04-04T06:05:31.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.581+0000 7fb43b400640 1 -- start start 2024-04-04T06:05:31.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.582+0000 7fb43b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4340cf050 0x7fb43414b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:31.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.582+0000 7fb43b400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4340d0e70 0x7fb4340b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:31.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.582+0000 7fb43b400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4340d2110 0x7fb4340cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:31.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.582+0000 7fb43b400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb43415e910 con 0x7fb4340cf050 2024-04-04T06:05:31.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.582+0000 7fb43b400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb4340d0240 con 0x7fb4340d0e70 2024-04-04T06:05:31.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.582+0000 7fb43b400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb43415ea90 con 0x7fb4340d2110 2024-04-04T06:05:31.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.582+0000 7fb439400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4340d2110 0x7fb4340cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:31.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.582+0000 7fb433e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4340d0e70 0x7fb4340b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:31.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.582+0000 7fb439400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4340d2110 0x7fb4340cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:55620/0 (socket says 172.21.15.178:55620) 2024-04-04T06:05:31.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.582+0000 7fb439400640 1 -- 172.21.15.178:0/728592183 learned_addr learned my addr 172.21.15.178:0/728592183 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-04T06:05:31.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.582+0000 7fb438a00640 1 --2- 172.21.15.178:0/728592183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4340cf050 0x7fb43414b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:31.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.583+0000 7fb439400640 1 -- 172.21.15.178:0/728592183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4340d0e70 msgr2=0x7fb4340b8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:31.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.583+0000 7fb439400640 1 --2- 172.21.15.178:0/728592183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4340d0e70 0x7fb4340b8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.583+0000 7fb439400640 1 -- 172.21.15.178:0/728592183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4340cf050 msgr2=0x7fb43414b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:31.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.583+0000 7fb439400640 1 --2- 172.21.15.178:0/728592183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4340cf050 0x7fb43414b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.583+0000 7fb439400640 1 -- 172.21.15.178:0/728592183 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4340b9700 con 0x7fb4340d2110 2024-04-04T06:05:31.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.583+0000 7fb433e00640 1 --2- 172.21.15.178:0/728592183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4340d0e70 0x7fb4340b8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:05:31.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.583+0000 7fb439400640 1 --2- 172.21.15.178:0/728592183 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4340d2110 0x7fb4340cd700 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb42c036040 tx=0x7fb42c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:31.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.583+0000 7fb430c00640 1 -- 172.21.15.178:0/728592183 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c044020 con 0x7fb4340d2110 2024-04-04T06:05:31.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.583+0000 7fb430c00640 1 -- 172.21.15.178:0/728592183 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb42c040690 con 0x7fb4340d2110 2024-04-04T06:05:31.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.583+0000 7fb43b400640 1 -- 172.21.15.178:0/728592183 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4340b99e0 con 0x7fb4340d2110 2024-04-04T06:05:31.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.583+0000 7fb430c00640 1 -- 172.21.15.178:0/728592183 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c040a80 con 0x7fb4340d2110 2024-04-04T06:05:31.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.584+0000 7fb43b400640 1 -- 172.21.15.178:0/728592183 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4340c8180 con 0x7fb4340d2110 2024-04-04T06:05:31.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.584+0000 7fb430c00640 1 -- 172.21.15.178:0/728592183 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb42c047020 con 0x7fb4340d2110 2024-04-04T06:05:31.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.584+0000 7fb430c00640 1 -- 172.21.15.178:0/728592183 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fb42c02ca90 con 0x7fb4340d2110 2024-04-04T06:05:31.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.585+0000 7fb43b400640 1 -- 172.21.15.178:0/728592183 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb434068cd0 con 0x7fb4340d2110 2024-04-04T06:05:31.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.587+0000 7fb430c00640 1 -- 172.21.15.178:0/728592183 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb42c04e030 con 0x7fb4340d2110 2024-04-04T06:05:31.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.665+0000 7fb43b400640 1 -- 172.21.15.178:0/728592183 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9d0aead7-b517-4d97-b37e-747b7fc1c03d", "id": 5} v 0) -- 0x7fb4340d1b80 con 0x7fb4340d2110 2024-04-04T06:05:31.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.672+0000 7fb430c00640 1 -- 172.21.15.178:0/728592183 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9d0aead7-b517-4d97-b37e-747b7fc1c03d", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fb42c03eb50 con 0x7fb4340d2110 2024-04-04T06:05:31.668 INFO:teuthology.orchestra.run.smithi178.stdout:5 2024-04-04T06:05:31.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.674+0000 7fb43b400640 1 -- 172.21.15.178:0/728592183 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4340d2110 msgr2=0x7fb4340cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:31.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.674+0000 7fb43b400640 1 --2- 172.21.15.178:0/728592183 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4340d2110 0x7fb4340cd700 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb42c036040 tx=0x7fb42c004a40 comp rx=0 tx=0).stop 2024-04-04T06:05:31.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.674+0000 7fb43b400640 1 -- 172.21.15.178:0/728592183 shutdown_connections 2024-04-04T06:05:31.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.674+0000 7fb43b400640 1 --2- 172.21.15.178:0/728592183 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb4340d2110 0x7fb4340cd700 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.674+0000 7fb43b400640 1 --2- 172.21.15.178:0/728592183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4340d0e70 0x7fb4340b8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.674+0000 7fb43b400640 1 --2- 172.21.15.178:0/728592183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4340cf050 0x7fb43414b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.674+0000 7fb43b400640 1 -- 172.21.15.178:0/728592183 >> 172.21.15.178:0/728592183 conn(0x7fb43408bfb0 msgr2=0x7fb43406dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:31.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.674+0000 7fb43b400640 1 -- 172.21.15.178:0/728592183 shutdown_connections 2024-04-04T06:05:31.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.674+0000 7fb43b400640 1 -- 172.21.15.178:0/728592183 wait complete. 2024-04-04T06:05:31.673 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:31.678+0000 7f88e803b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-04T06:05:31.685 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 472edb50-4e29-40ab-bee7-91b081541e1c 6 2024-04-04T06:05:31.723 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:31.728+0000 7f88e803b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-04T06:05:31.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.876+0000 7f617da00640 1 Processor -- start 2024-04-04T06:05:31.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.876+0000 7f617da00640 1 -- start start 2024-04-04T06:05:31.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.877+0000 7f617da00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f617815da60 0x7f61780cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:31.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.877+0000 7f617da00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f617815fe90 0x7f61780d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:31.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.877+0000 7f617da00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f617815d060 con 0x7f617815f900 2024-04-04T06:05:31.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.877+0000 7f617da00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f617815d340 con 0x7f617815fe90 2024-04-04T06:05:31.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.877+0000 7f617da00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f617815d620 con 0x7f617815da60 2024-04-04T06:05:31.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.878+0000 7f6176000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f617815da60 0x7f61780cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:31.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.878+0000 7f6176000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f617815da60 0x7f61780cfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52814/0 (socket says 172.21.15.178:52814) 2024-04-04T06:05:31.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.878+0000 7f6176000640 1 -- 172.21.15.178:0/310427340 learned_addr learned my addr 172.21.15.178:0/310427340 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-04T06:05:31.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.878+0000 7f6177400640 1 --2- 172.21.15.178:0/310427340 >> v2:172.21.15.159:3300/0 conn(0x7f617815fe90 0x7f61780d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:31.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.878+0000 7f6176000640 1 -- 172.21.15.178:0/310427340 >> v2:172.21.15.159:3300/0 conn(0x7f617815fe90 msgr2=0x7f61780d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:31.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.878+0000 7f6176000640 1 --2- 172.21.15.178:0/310427340 >> v2:172.21.15.159:3300/0 conn(0x7f617815fe90 0x7f61780d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.878+0000 7f6176000640 1 -- 172.21.15.178:0/310427340 >> v1:172.21.15.159:6789/0 conn(0x7f617815f900 legacy=0x7f617815c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:31.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.878+0000 7f6176000640 1 -- 172.21.15.178:0/310427340 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61780c74b0 con 0x7f617815da60 2024-04-04T06:05:31.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.879+0000 7f6176000640 1 --2- 172.21.15.178:0/310427340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f617815da60 0x7f61780cfe30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f617806f390 tx=0x7f616402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30de479c5ee67e2a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:31.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.879+0000 7f6175600640 1 -- 172.21.15.178:0/310427340 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164035020 con 0x7f617815da60 2024-04-04T06:05:31.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.879+0000 7f6175600640 1 -- 172.21.15.178:0/310427340 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6164031870 con 0x7f617815da60 2024-04-04T06:05:31.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.879+0000 7f6175600640 1 -- 172.21.15.178:0/310427340 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164031c40 con 0x7f617815da60 2024-04-04T06:05:31.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.880+0000 7f617da00640 1 -- 172.21.15.178:0/310427340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f617815da60 msgr2=0x7f61780cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:31.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.880+0000 7f617da00640 1 --2- 172.21.15.178:0/310427340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f617815da60 0x7f61780cfe30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f617806f390 tx=0x7f616402b910 comp rx=0 tx=0).stop 2024-04-04T06:05:31.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.880+0000 7f617da00640 1 -- 172.21.15.178:0/310427340 shutdown_connections 2024-04-04T06:05:31.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.880+0000 7f617da00640 1 --2- 172.21.15.178:0/310427340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f617815da60 0x7f61780cfe30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.880+0000 7f617da00640 1 --2- 172.21.15.178:0/310427340 >> v2:172.21.15.159:3300/0 conn(0x7f617815fe90 0x7f61780d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.880+0000 7f617da00640 1 -- 172.21.15.178:0/310427340 >> 172.21.15.178:0/310427340 conn(0x7f61780b3e90 msgr2=0x7f61780b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:31.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.880+0000 7f617da00640 1 -- 172.21.15.178:0/310427340 shutdown_connections 2024-04-04T06:05:31.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.880+0000 7f617da00640 1 -- 172.21.15.178:0/310427340 wait complete. 2024-04-04T06:05:31.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.882+0000 7f617da00640 1 Processor -- start 2024-04-04T06:05:31.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.882+0000 7f617da00640 1 -- start start 2024-04-04T06:05:31.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.883+0000 7f617da00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f617815da60 0x7f61780b6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:31.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.883+0000 7f617da00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f617815f900 0x7f61781f85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:31.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.883+0000 7f617da00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f617815fe90 0x7f6178154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:31.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.883+0000 7f617da00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f61780cece0 con 0x7f617815fe90 2024-04-04T06:05:31.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.883+0000 7f617da00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f61780ceb60 con 0x7f617815da60 2024-04-04T06:05:31.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.883+0000 7f617da00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f61780cee60 con 0x7f617815f900 2024-04-04T06:05:31.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.883+0000 7f6176a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f617815da60 0x7f61780b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:31.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.883+0000 7f6176000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f617815f900 0x7f61781f85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:31.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.883+0000 7f6176a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f617815da60 0x7f61780b6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52822/0 (socket says 172.21.15.178:52822) 2024-04-04T06:05:31.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.883+0000 7f6176a00640 1 -- 172.21.15.178:0/1605537125 learned_addr learned my addr 172.21.15.178:0/1605537125 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-04T06:05:31.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.883+0000 7f6176000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f617815f900 0x7f61781f85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:55626/0 (socket says 172.21.15.178:55626) 2024-04-04T06:05:31.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.883+0000 7f6177400640 1 --2- 172.21.15.178:0/1605537125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f617815fe90 0x7f6178154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:31.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.884+0000 7f6176a00640 1 -- 172.21.15.178:0/1605537125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f617815f900 msgr2=0x7f61781f85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:31.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.884+0000 7f6176a00640 1 --2- 172.21.15.178:0/1605537125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f617815f900 0x7f61781f85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.884+0000 7f6176a00640 1 -- 172.21.15.178:0/1605537125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f617815fe90 msgr2=0x7f6178154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:31.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.884+0000 7f6176a00640 1 --2- 172.21.15.178:0/1605537125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f617815fe90 0x7f6178154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.884+0000 7f6176a00640 1 -- 172.21.15.178:0/1605537125 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61780b6e20 con 0x7f617815da60 2024-04-04T06:05:31.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.884+0000 7f6176000640 1 --2- 172.21.15.178:0/1605537125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f617815f900 0x7f61781f85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:05:31.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.884+0000 7f6176a00640 1 --2- 172.21.15.178:0/1605537125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f617815da60 0x7f61780b6840 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f616800c430 tx=0x7f616800c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:31.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.884+0000 7f616ea00640 1 -- 172.21.15.178:0/1605537125 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6168011790 con 0x7f617815da60 2024-04-04T06:05:31.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.884+0000 7f616ea00640 1 -- 172.21.15.178:0/1605537125 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6168017070 con 0x7f617815da60 2024-04-04T06:05:31.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.884+0000 7f616ea00640 1 -- 172.21.15.178:0/1605537125 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6168011bb0 con 0x7f617815da60 2024-04-04T06:05:31.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.884+0000 7f617da00640 1 -- 172.21.15.178:0/1605537125 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61780b7100 con 0x7f617815da60 2024-04-04T06:05:31.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.884+0000 7f617da00640 1 -- 172.21.15.178:0/1605537125 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61780b76c0 con 0x7f617815da60 2024-04-04T06:05:31.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.885+0000 7f616ea00640 1 -- 172.21.15.178:0/1605537125 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f616801c460 con 0x7f617815da60 2024-04-04T06:05:31.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.885+0000 7f616ea00640 1 -- 172.21.15.178:0/1605537125 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f6168011d50 con 0x7f617815da60 2024-04-04T06:05:31.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.886+0000 7f617da00640 1 -- 172.21.15.178:0/1605537125 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f617815a130 con 0x7f617815da60 2024-04-04T06:05:31.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.888+0000 7f616ea00640 1 -- 172.21.15.178:0/1605537125 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f617815a130 con 0x7f617815da60 2024-04-04T06:05:31.895 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:31.900+0000 7f88e803b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-04T06:05:31.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.962+0000 7f617da00640 1 -- 172.21.15.178:0/1605537125 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd new", "uuid": "472edb50-4e29-40ab-bee7-91b081541e1c", "id": 6} v 0) -- 0x7f61780759e0 con 0x7f617815da60 2024-04-04T06:05:31.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.969+0000 7f616ea00640 1 -- 172.21.15.178:0/1605537125 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "472edb50-4e29-40ab-bee7-91b081541e1c", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f61780759e0 con 0x7f617815da60 2024-04-04T06:05:31.964 INFO:teuthology.orchestra.run.smithi178.stdout:6 2024-04-04T06:05:31.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.970+0000 7f617da00640 1 -- 172.21.15.178:0/1605537125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f617815da60 msgr2=0x7f61780b6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:31.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.970+0000 7f617da00640 1 --2- 172.21.15.178:0/1605537125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f617815da60 0x7f61780b6840 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f616800c430 tx=0x7f616800c8f0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.970+0000 7f617da00640 1 -- 172.21.15.178:0/1605537125 shutdown_connections 2024-04-04T06:05:31.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.970+0000 7f617da00640 1 --2- 172.21.15.178:0/1605537125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f617815fe90 0x7f6178154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.970+0000 7f617da00640 1 --2- 172.21.15.178:0/1605537125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f617815f900 0x7f61781f85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.970+0000 7f617da00640 1 --2- 172.21.15.178:0/1605537125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f617815da60 0x7f61780b6840 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:31.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.970+0000 7f617da00640 1 -- 172.21.15.178:0/1605537125 >> 172.21.15.178:0/1605537125 conn(0x7f61780b3e90 msgr2=0x7f61780d1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:31.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.970+0000 7f617da00640 1 -- 172.21.15.178:0/1605537125 shutdown_connections 2024-04-04T06:05:31.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:31.970+0000 7f617da00640 1 -- 172.21.15.178:0/1605537125 wait complete. 2024-04-04T06:05:31.980 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 3ea70d65-8690-4b91-a011-b3897a96e780 7 2024-04-04T06:05:32.109 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:32.114+0000 7f0a7001a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-04T06:05:32.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.166+0000 7f8cdda00640 1 Processor -- start 2024-04-04T06:05:32.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.166+0000 7f8cdda00640 1 -- start start 2024-04-04T06:05:32.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.167+0000 7f8cdda00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f8cd80d0280 0x7f8cd80d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:32.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.167+0000 7f8cdda00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cd8141c10 0x7f8cd8071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:32.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.167+0000 7f8cdda00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cd815ec80 con 0x7f8cd8071eb0 2024-04-04T06:05:32.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.167+0000 7f8cdda00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f8cd80d0ba0 con 0x7f8cd80d0280 2024-04-04T06:05:32.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.167+0000 7f8cdda00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd815ef20 con 0x7f8cd8141c10 2024-04-04T06:05:32.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.168+0000 7f8cce000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cd8141c10 0x7f8cd8071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:32.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.168+0000 7f8cd7400640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f8cd8071eb0 0x7f8cd806d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:42938/0 (socket says 172.21.15.178:42938) 2024-04-04T06:05:32.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.168+0000 7f8cce000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cd8141c10 0x7f8cd8071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:55630/0 (socket says 172.21.15.178:55630) 2024-04-04T06:05:32.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.168+0000 7f8cd7400640 1 -- 172.21.15.178:0/122592724 learned_addr learned my addr 172.21.15.178:0/122592724 (peer_addr_for_me v1:172.21.15.178:0/0) 2024-04-04T06:05:32.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.168+0000 7f8cd6a00640 1 --2- 172.21.15.178:0/122592724 >> v2:172.21.15.159:3300/0 conn(0x7f8cd80d0280 0x7f8cd80d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:32.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.168+0000 7f8cce000640 1 -- 172.21.15.178:0/122592724 >> v2:172.21.15.159:3300/0 conn(0x7f8cd80d0280 msgr2=0x7f8cd80d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:32.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.168+0000 7f8cce000640 1 --2- 172.21.15.178:0/122592724 >> v2:172.21.15.159:3300/0 conn(0x7f8cd80d0280 0x7f8cd80d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:32.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.168+0000 7f8cce000640 1 -- 172.21.15.178:0/122592724 >> v1:172.21.15.178:6789/0 conn(0x7f8cd8071eb0 legacy=0x7f8cd806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:32.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.168+0000 7f8cce000640 1 -- 172.21.15.178:0/122592724 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd815f200 con 0x7f8cd8141c10 2024-04-04T06:05:32.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.169+0000 7f8cce000640 1 --2- 172.21.15.178:0/122592724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cd8141c10 0x7f8cd8071970 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00097a0 tx=0x7f8cc002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=917d4eeb2d9d71b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:32.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.169+0000 7f8cd6000640 1 -- 172.21.15.178:0/122592724 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0035020 con 0x7f8cd8141c10 2024-04-04T06:05:32.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.170+0000 7f8cd6000640 1 -- 172.21.15.178:0/122592724 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cc0031810 con 0x7f8cd8141c10 2024-04-04T06:05:32.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.170+0000 7f8cd6000640 1 -- 172.21.15.178:0/122592724 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0031be0 con 0x7f8cd8141c10 2024-04-04T06:05:32.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.170+0000 7f8cdda00640 1 -- 172.21.15.178:0/122592724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cd8141c10 msgr2=0x7f8cd8071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:32.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.170+0000 7f8cdda00640 1 --2- 172.21.15.178:0/122592724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cd8141c10 0x7f8cd8071970 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00097a0 tx=0x7f8cc002b8b0 comp rx=0 tx=0).stop 2024-04-04T06:05:32.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.170+0000 7f8cdda00640 1 -- 172.21.15.178:0/122592724 shutdown_connections 2024-04-04T06:05:32.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.170+0000 7f8cdda00640 1 --2- 172.21.15.178:0/122592724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cd8141c10 0x7f8cd8071970 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:32.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.170+0000 7f8cdda00640 1 --2- 172.21.15.178:0/122592724 >> v2:172.21.15.159:3300/0 conn(0x7f8cd80d0280 0x7f8cd80d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:32.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.170+0000 7f8cdda00640 1 -- 172.21.15.178:0/122592724 >> 172.21.15.178:0/122592724 conn(0x7f8cd806b750 msgr2=0x7f8cd806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:32.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.170+0000 7f8cdda00640 1 -- 172.21.15.178:0/122592724 shutdown_connections 2024-04-04T06:05:32.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.170+0000 7f8cdda00640 1 -- 172.21.15.178:0/122592724 wait complete. 2024-04-04T06:05:32.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.172+0000 7f8cdda00640 1 Processor -- start 2024-04-04T06:05:32.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.172+0000 7f8cdda00640 1 -- start start 2024-04-04T06:05:32.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.173+0000 7f8cdda00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cd8071eb0 0x7f8cd81e91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:32.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.173+0000 7f8cdda00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cd80d0280 0x7f8cd81e7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:32.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.173+0000 7f8cdda00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8cd8141c10 0x7f8cd80b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:32.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.173+0000 7f8cdda00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd8158e90 con 0x7f8cd8141c10 2024-04-04T06:05:32.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.173+0000 7f8cdda00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd815f7c0 con 0x7f8cd80d0280 2024-04-04T06:05:32.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.173+0000 7f8cdda00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd81e9870 con 0x7f8cd8071eb0 2024-04-04T06:05:32.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.173+0000 7f8cce000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cd80d0280 0x7f8cd81e7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:32.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.173+0000 7f8cd6a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cd8071eb0 0x7f8cd81e91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:32.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.173+0000 7f8cce000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cd80d0280 0x7f8cd81e7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52832/0 (socket says 172.21.15.178:52832) 2024-04-04T06:05:32.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.173+0000 7f8cce000640 1 -- 172.21.15.178:0/3078570745 learned_addr learned my addr 172.21.15.178:0/3078570745 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-04-04T06:05:32.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.174+0000 7f8cd7400640 1 --2- 172.21.15.178:0/3078570745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8cd8141c10 0x7f8cd80b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:32.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.174+0000 7f8cce000640 1 -- 172.21.15.178:0/3078570745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cd8071eb0 msgr2=0x7f8cd81e91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:32.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.174+0000 7f8cce000640 1 --2- 172.21.15.178:0/3078570745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cd8071eb0 0x7f8cd81e91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:32.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.174+0000 7f8cce000640 1 -- 172.21.15.178:0/3078570745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8cd8141c10 msgr2=0x7f8cd80b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:32.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.174+0000 7f8cce000640 1 --2- 172.21.15.178:0/3078570745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8cd8141c10 0x7f8cd80b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:32.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.174+0000 7f8cce000640 1 -- 172.21.15.178:0/3078570745 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd81e9b50 con 0x7f8cd80d0280 2024-04-04T06:05:32.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.174+0000 7f8cce000640 1 --2- 172.21.15.178:0/3078570745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cd80d0280 0x7f8cd81e7f30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0034040 tx=0x7f8cc00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:32.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.174+0000 7f8ccf400640 1 -- 172.21.15.178:0/3078570745 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0040070 con 0x7f8cd80d0280 2024-04-04T06:05:32.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.174+0000 7f8ccf400640 1 -- 172.21.15.178:0/3078570745 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cc0039cf0 con 0x7f8cd80d0280 2024-04-04T06:05:32.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.175+0000 7f8ccf400640 1 -- 172.21.15.178:0/3078570745 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0035040 con 0x7f8cd80d0280 2024-04-04T06:05:32.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.175+0000 7f8cdda00640 1 -- 172.21.15.178:0/3078570745 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd81e9e30 con 0x7f8cd80d0280 2024-04-04T06:05:32.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.175+0000 7f8cdda00640 1 -- 172.21.15.178:0/3078570745 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd809bbb0 con 0x7f8cd80d0280 2024-04-04T06:05:32.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.175+0000 7f8ccf400640 1 -- 172.21.15.178:0/3078570745 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8cc0038020 con 0x7f8cd80d0280 2024-04-04T06:05:32.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.175+0000 7f8ccf400640 1 -- 172.21.15.178:0/3078570745 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f8cc003ae00 con 0x7f8cd80d0280 2024-04-04T06:05:32.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.176+0000 7f8cdda00640 1 -- 172.21.15.178:0/3078570745 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd815e560 con 0x7f8cd80d0280 2024-04-04T06:05:32.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.178+0000 7f8ccf400640 1 -- 172.21.15.178:0/3078570745 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f8cd815e560 con 0x7f8cd80d0280 2024-04-04T06:05:32.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.257+0000 7f8cdda00640 1 -- 172.21.15.178:0/3078570745 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3ea70d65-8690-4b91-a011-b3897a96e780", "id": 7} v 0) -- 0x7f8cd80685c0 con 0x7f8cd80d0280 2024-04-04T06:05:32.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.266+0000 7f8ccf400640 1 -- 172.21.15.178:0/3078570745 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3ea70d65-8690-4b91-a011-b3897a96e780", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f8cc003a680 con 0x7f8cd80d0280 2024-04-04T06:05:32.262 INFO:teuthology.orchestra.run.smithi178.stdout:7 2024-04-04T06:05:32.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.268+0000 7f8cdda00640 1 -- 172.21.15.178:0/3078570745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cd80d0280 msgr2=0x7f8cd81e7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:32.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.268+0000 7f8cdda00640 1 --2- 172.21.15.178:0/3078570745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cd80d0280 0x7f8cd81e7f30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0034040 tx=0x7f8cc00046c0 comp rx=0 tx=0).stop 2024-04-04T06:05:32.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.268+0000 7f8cdda00640 1 -- 172.21.15.178:0/3078570745 shutdown_connections 2024-04-04T06:05:32.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.268+0000 7f8cdda00640 1 --2- 172.21.15.178:0/3078570745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8cd8141c10 0x7f8cd80b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:32.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.268+0000 7f8cdda00640 1 --2- 172.21.15.178:0/3078570745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8cd80d0280 0x7f8cd81e7f30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:32.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.268+0000 7f8cdda00640 1 --2- 172.21.15.178:0/3078570745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8cd8071eb0 0x7f8cd81e91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:32.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.268+0000 7f8cdda00640 1 -- 172.21.15.178:0/3078570745 >> 172.21.15.178:0/3078570745 conn(0x7f8cd806b750 msgr2=0x7f8cd815fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:32.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.268+0000 7f8cdda00640 1 -- 172.21.15.178:0/3078570745 shutdown_connections 2024-04-04T06:05:32.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:05:32.268+0000 7f8cdda00640 1 -- 172.21.15.178:0/3078570745 wait complete. 2024-04-04T06:05:32.279 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-04T06:05:32.279 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-04T06:05:32.286 INFO:tasks.ceph.osd.0:Started 2024-04-04T06:05:32.286 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-04T06:05:32.286 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-04T06:05:32.294 INFO:tasks.ceph.osd.1:Started 2024-04-04T06:05:32.294 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-04T06:05:32.294 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-04T06:05:32.300 INFO:tasks.ceph.osd.2:Started 2024-04-04T06:05:32.300 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-04T06:05:32.300 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-04T06:05:32.305 INFO:tasks.ceph.osd.3:Started 2024-04-04T06:05:32.305 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-04T06:05:32.305 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-04T06:05:32.310 INFO:tasks.ceph.osd.4:Started 2024-04-04T06:05:32.310 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-04T06:05:32.310 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-04T06:05:32.314 INFO:tasks.ceph.osd.5:Started 2024-04-04T06:05:32.314 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-04T06:05:32.314 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-04T06:05:32.318 INFO:tasks.ceph.osd.6:Started 2024-04-04T06:05:32.319 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-04T06:05:32.319 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-04T06:05:32.323 INFO:tasks.ceph.osd.7:Started 2024-04-04T06:05:32.324 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:05:32.490 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:32.495+0000 7f0a7001a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-04T06:05:32.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.545+0000 7fa917200640 1 Processor -- start 2024-04-04T06:05:32.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.546+0000 7fa917200640 1 -- start start 2024-04-04T06:05:32.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.547+0000 7fa917200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa91815e3b0 0x7fa9180d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:32.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.547+0000 7fa917200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa9180b63c0 0x7fa9180b7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:32.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.547+0000 7fa917200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9180d2cf0 con 0x7fa9180d3d90 2024-04-04T06:05:32.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.547+0000 7fa917200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fa9180d2fd0 con 0x7fa91815e3b0 2024-04-04T06:05:32.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.547+0000 7fa917200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa9180d32b0 con 0x7fa9180b63c0 2024-04-04T06:05:32.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.547+0000 7fa915e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa91815e3b0 0x7fa9180d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:32.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.547+0000 7fa915e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa91815e3b0 0x7fa9180d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39066/0 (socket says 172.21.15.159:39066) 2024-04-04T06:05:32.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.547+0000 7fa916800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa9180b63c0 0x7fa9180b7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:32.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.547+0000 7fa915e00640 1 -- 172.21.15.159:0/1066551607 learned_addr learned my addr 172.21.15.159:0/1066551607 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:32.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.548+0000 7fa915e00640 1 -- 172.21.15.159:0/1066551607 >> v2:172.21.15.178:3300/0 conn(0x7fa9180b63c0 msgr2=0x7fa9180b7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:32.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.548+0000 7fa915e00640 1 --2- 172.21.15.159:0/1066551607 >> v2:172.21.15.178:3300/0 conn(0x7fa9180b63c0 0x7fa9180b7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:32.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.548+0000 7fa915e00640 1 -- 172.21.15.159:0/1066551607 >> v1:172.21.15.178:6789/0 conn(0x7fa9180d3d90 legacy=0x7fa9180b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:32.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.548+0000 7fa915400640 1 -- 172.21.15.159:0/1066551607 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4243601784 0 0) 0x7fa9180d2cf0 con 0x7fa9180d3d90 2024-04-04T06:05:32.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.548+0000 7fa915e00640 1 -- 172.21.15.159:0/1066551607 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9180d3590 con 0x7fa91815e3b0 2024-04-04T06:05:32.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.549+0000 7fa915e00640 1 --2- 172.21.15.159:0/1066551607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa91815e3b0 0x7fa9180d22e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa908009870 tx=0x7fa90802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80864c39dfd5c47f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:32.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.549+0000 7fa915400640 1 -- 172.21.15.159:0/1066551607 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90802f840 con 0x7fa91815e3b0 2024-04-04T06:05:32.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.549+0000 7fa915400640 1 -- 172.21.15.159:0/1066551607 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa90802f9e0 con 0x7fa91815e3b0 2024-04-04T06:05:32.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.549+0000 7fa915400640 1 -- 172.21.15.159:0/1066551607 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90802fdb0 con 0x7fa91815e3b0 2024-04-04T06:05:32.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.549+0000 7fa917200640 1 -- 172.21.15.159:0/1066551607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa91815e3b0 msgr2=0x7fa9180d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:32.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.549+0000 7fa917200640 1 --2- 172.21.15.159:0/1066551607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa91815e3b0 0x7fa9180d22e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa908009870 tx=0x7fa90802ba20 comp rx=0 tx=0).stop 2024-04-04T06:05:32.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.550+0000 7fa917200640 1 -- 172.21.15.159:0/1066551607 shutdown_connections 2024-04-04T06:05:32.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.550+0000 7fa917200640 1 --2- 172.21.15.159:0/1066551607 >> v2:172.21.15.178:3300/0 conn(0x7fa9180b63c0 0x7fa9180b7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:32.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.550+0000 7fa917200640 1 --2- 172.21.15.159:0/1066551607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa91815e3b0 0x7fa9180d22e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:32.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.550+0000 7fa917200640 1 -- 172.21.15.159:0/1066551607 >> 172.21.15.159:0/1066551607 conn(0x7fa9180678b0 msgr2=0x7fa9180d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:32.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.550+0000 7fa917200640 1 -- 172.21.15.159:0/1066551607 shutdown_connections 2024-04-04T06:05:32.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.550+0000 7fa917200640 1 -- 172.21.15.159:0/1066551607 wait complete. 2024-04-04T06:05:32.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.551+0000 7fa917200640 1 Processor -- start 2024-04-04T06:05:32.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.552+0000 7fa917200640 1 -- start start 2024-04-04T06:05:32.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.553+0000 7fa917200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9180b63c0 0x7fa9180ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:32.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.553+0000 7fa917200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9180d3d90 0x7fa9180b7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:32.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.553+0000 7fa917200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa91815e3b0 0x7fa9180cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:32.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.553+0000 7fa917200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa9180ba030 con 0x7fa9180b63c0 2024-04-04T06:05:32.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.553+0000 7fa917200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa91815e790 con 0x7fa9180d3d90 2024-04-04T06:05:32.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.553+0000 7fa917200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa9180ba1b0 con 0x7fa91815e3b0 2024-04-04T06:05:32.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.553+0000 7fa915e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9180b63c0 0x7fa9180ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:32.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.553+0000 7fa915e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9180b63c0 0x7fa9180ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39080/0 (socket says 172.21.15.159:39080) 2024-04-04T06:05:32.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.553+0000 7fa915e00640 1 -- 172.21.15.159:0/142573478 learned_addr learned my addr 172.21.15.159:0/142573478 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:32.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.553+0000 7fa90fe00640 1 --2- 172.21.15.159:0/142573478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9180d3d90 0x7fa9180b7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:32.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.553+0000 7fa916800640 1 --2- 172.21.15.159:0/142573478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa91815e3b0 0x7fa9180cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:32.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.553+0000 7fa915e00640 1 -- 172.21.15.159:0/142573478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa91815e3b0 msgr2=0x7fa9180cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:32.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.553+0000 7fa915e00640 1 --2- 172.21.15.159:0/142573478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa91815e3b0 0x7fa9180cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:32.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.553+0000 7fa915e00640 1 -- 172.21.15.159:0/142573478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9180d3d90 msgr2=0x7fa9180b7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:32.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.553+0000 7fa915e00640 1 --2- 172.21.15.159:0/142573478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9180d3d90 0x7fa9180b7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:32.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.553+0000 7fa915e00640 1 -- 172.21.15.159:0/142573478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9180c91a0 con 0x7fa9180b63c0 2024-04-04T06:05:32.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.553+0000 7fa90fe00640 1 --2- 172.21.15.159:0/142573478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9180d3d90 0x7fa9180b7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:05:32.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.554+0000 7fa915e00640 1 --2- 172.21.15.159:0/142573478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9180b63c0 0x7fa9180ca240 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa9080090b0 tx=0x7fa908004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:32.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.554+0000 7fa90e000640 1 -- 172.21.15.159:0/142573478 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908047070 con 0x7fa9180b63c0 2024-04-04T06:05:32.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.554+0000 7fa90e000640 1 -- 172.21.15.159:0/142573478 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa908004230 con 0x7fa9180b63c0 2024-04-04T06:05:32.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.554+0000 7fa90e000640 1 -- 172.21.15.159:0/142573478 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908037940 con 0x7fa9180b63c0 2024-04-04T06:05:32.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.554+0000 7fa917200640 1 -- 172.21.15.159:0/142573478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9180c9480 con 0x7fa9180b63c0 2024-04-04T06:05:32.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.554+0000 7fa917200640 1 -- 172.21.15.159:0/142573478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9180c9a40 con 0x7fa9180b63c0 2024-04-04T06:05:32.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.555+0000 7fa90e000640 1 -- 172.21.15.159:0/142573478 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa90804a070 con 0x7fa9180b63c0 2024-04-04T06:05:32.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.555+0000 7fa90e000640 1 -- 172.21.15.159:0/142573478 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fa908042b00 con 0x7fa9180b63c0 2024-04-04T06:05:32.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.555+0000 7fa917200640 1 -- 172.21.15.159:0/142573478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa918075940 con 0x7fa9180b63c0 2024-04-04T06:05:32.553 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:32.559+0000 7f0a7001a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-04T06:05:32.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.559+0000 7fa90e000640 1 -- 172.21.15.159:0/142573478 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa908033030 con 0x7fa9180b63c0 2024-04-04T06:05:32.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.629+0000 7fa917200640 1 -- 172.21.15.159:0/142573478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa91815cd50 con 0x7fa9180b63c0 2024-04-04T06:05:32.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.629+0000 7fa90e000640 1 -- 172.21.15.159:0/142573478 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fa908037ae0 con 0x7fa9180b63c0 2024-04-04T06:05:32.624 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:05:32.624 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":10,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:05:32.259925+0000","last_up_change":"0.000000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:05:32.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.630+0000 7fa917200640 1 -- 172.21.15.159:0/142573478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9180b63c0 msgr2=0x7fa9180ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:32.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.630+0000 7fa917200640 1 --2- 172.21.15.159:0/142573478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9180b63c0 0x7fa9180ca240 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa9080090b0 tx=0x7fa908004730 comp rx=0 tx=0).stop 2024-04-04T06:05:32.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.630+0000 7fa917200640 1 -- 172.21.15.159:0/142573478 shutdown_connections 2024-04-04T06:05:32.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.630+0000 7fa917200640 1 --2- 172.21.15.159:0/142573478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa91815e3b0 0x7fa9180cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:32.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.630+0000 7fa917200640 1 --2- 172.21.15.159:0/142573478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9180d3d90 0x7fa9180b7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:32.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.630+0000 7fa917200640 1 --2- 172.21.15.159:0/142573478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9180b63c0 0x7fa9180ca240 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:32.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.630+0000 7fa917200640 1 -- 172.21.15.159:0/142573478 >> 172.21.15.159:0/142573478 conn(0x7fa9180678b0 msgr2=0x7fa9180b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:32.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.631+0000 7fa917200640 1 -- 172.21.15.159:0/142573478 shutdown_connections 2024-04-04T06:05:32.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:32.631+0000 7fa917200640 1 -- 172.21.15.159:0/142573478 wait complete. 2024-04-04T06:05:32.643 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-04T06:05:32.643 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-04T06:05:32.681 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:32.686+0000 7f0a7001a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-04T06:05:32.946 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:32.951+0000 7f88e803b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-04T06:05:32.989 INFO:tasks.ceph.osd.5.smithi178.stderr:2024-04-04T06:05:32.994+0000 7f31e5868640 -1 Falling back to public interface 2024-04-04T06:05:32.997 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-04-04T06:05:33.003+0000 7fac112ae640 -1 Falling back to public interface 2024-04-04T06:05:33.001 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-04T06:05:33.006+0000 7f2754837640 -1 Falling back to public interface 2024-04-04T06:05:33.002 INFO:tasks.ceph.osd.4.smithi178.stderr:2024-04-04T06:05:33.007+0000 7fd8c0acc640 -1 Falling back to public interface 2024-04-04T06:05:33.024 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-04-04T06:05:33.030+0000 7f7a13c01640 -1 Falling back to public interface 2024-04-04T06:05:33.028 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-04-04T06:05:33.034+0000 7eff36e68640 -1 Falling back to public interface 2024-04-04T06:05:33.030 INFO:tasks.ceph.osd.6.smithi178.stderr:2024-04-04T06:05:33.035+0000 7f4019813640 -1 Falling back to public interface 2024-04-04T06:05:33.033 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-04-04T06:05:33.038+0000 7f379a40c640 -1 Falling back to public interface 2024-04-04T06:05:33.262 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:33.267+0000 7f88e803b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-04T06:05:33.322 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:33.327+0000 7f88e803b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-04T06:05:33.448 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-04T06:05:33.449 INFO:tasks.ceph.mgr.y.smithi159.stderr:2024-04-04T06:05:33.454+0000 7f88e803b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-04T06:05:33.619 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.624+0000 7f4c1f200640 1 Processor -- start 2024-04-04T06:05:33.619 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.624+0000 7f4c1f200640 1 -- start start 2024-04-04T06:05:33.621 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.625+0000 7f4c1f200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4c2015e630 0x7f4c200d24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:33.621 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.625+0000 7f4c1f200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c200b6a90 0x7f4c200b74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:33.621 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.625+0000 7f4c1f200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c200d2f00 con 0x7f4c200d29e0 2024-04-04T06:05:33.621 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.625+0000 7f4c1f200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f4c200d31e0 con 0x7f4c2015e630 2024-04-04T06:05:33.621 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.625+0000 7f4c1f200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4c200d34c0 con 0x7f4c200b6a90 2024-04-04T06:05:33.621 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.626+0000 7f4c1de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4c2015e630 0x7f4c200d24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:33.621 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.626+0000 7f4c1de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4c2015e630 0x7f4c200d24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39094/0 (socket says 172.21.15.159:39094) 2024-04-04T06:05:33.621 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.626+0000 7f4c1de00640 1 -- 172.21.15.159:0/3930507427 learned_addr learned my addr 172.21.15.159:0/3930507427 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:33.622 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.626+0000 7f4c1e800640 1 --2- 172.21.15.159:0/3930507427 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c200b6a90 0x7f4c200b74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:33.622 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.626+0000 7f4c1de00640 1 -- 172.21.15.159:0/3930507427 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c200b6a90 msgr2=0x7f4c200b74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:33.622 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.626+0000 7f4c1de00640 1 --2- 172.21.15.159:0/3930507427 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c200b6a90 0x7f4c200b74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:33.622 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.626+0000 7f4c1de00640 1 -- 172.21.15.159:0/3930507427 >> v1:172.21.15.159:6789/0 conn(0x7f4c200d29e0 legacy=0x7f4c200b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:33.622 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.626+0000 7f4c1de00640 1 -- 172.21.15.159:0/3930507427 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c200d37a0 con 0x7f4c2015e630 2024-04-04T06:05:33.623 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.627+0000 7f4c1de00640 1 --2- 172.21.15.159:0/3930507427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c2015e630 0x7f4c200d24a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009870 tx=0x7f4c1002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd40379d11087b58 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:33.623 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.627+0000 7f4c1ca00640 1 -- 172.21.15.159:0/3930507427 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10034070 con 0x7f4c2015e630 2024-04-04T06:05:33.623 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.627+0000 7f4c1ca00640 1 -- 172.21.15.159:0/3930507427 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c10030840 con 0x7f4c2015e630 2024-04-04T06:05:33.623 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.627+0000 7f4c1ca00640 1 -- 172.21.15.159:0/3930507427 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10030c10 con 0x7f4c2015e630 2024-04-04T06:05:33.623 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.628+0000 7f4c1f200640 1 -- 172.21.15.159:0/3930507427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c2015e630 msgr2=0x7f4c200d24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:33.624 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.628+0000 7f4c1f200640 1 --2- 172.21.15.159:0/3930507427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c2015e630 0x7f4c200d24a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009870 tx=0x7f4c1002b980 comp rx=0 tx=0).stop 2024-04-04T06:05:33.624 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.628+0000 7f4c1f200640 1 -- 172.21.15.159:0/3930507427 shutdown_connections 2024-04-04T06:05:33.624 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.628+0000 7f4c1f200640 1 --2- 172.21.15.159:0/3930507427 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c200b6a90 0x7f4c200b74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:33.624 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.628+0000 7f4c1f200640 1 --2- 172.21.15.159:0/3930507427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c2015e630 0x7f4c200d24a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:33.624 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.628+0000 7f4c1f200640 1 -- 172.21.15.159:0/3930507427 >> 172.21.15.159:0/3930507427 conn(0x7f4c20067620 msgr2=0x7f4c20067a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:33.624 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.628+0000 7f4c1f200640 1 -- 172.21.15.159:0/3930507427 shutdown_connections 2024-04-04T06:05:33.624 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.628+0000 7f4c1f200640 1 -- 172.21.15.159:0/3930507427 wait complete. 2024-04-04T06:05:33.624 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.630+0000 7f4c1f200640 1 Processor -- start 2024-04-04T06:05:33.625 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.630+0000 7f4c1f200640 1 -- start start 2024-04-04T06:05:33.627 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.631+0000 7f4c1f200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c200b6a90 0x7f4c2014c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:33.627 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.631+0000 7f4c1f200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c200d29e0 0x7f4c200c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:33.627 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.631+0000 7f4c1f200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c2015e630 0x7f4c200b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:33.627 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.631+0000 7f4c1f200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4c200d3fa0 con 0x7f4c200d29e0 2024-04-04T06:05:33.627 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.631+0000 7f4c1f200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2015ea10 con 0x7f4c2015e630 2024-04-04T06:05:33.627 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.631+0000 7f4c1f200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4c200d4120 con 0x7f4c200b6a90 2024-04-04T06:05:33.627 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.631+0000 7f4c1d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c200d29e0 0x7f4c200c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:33.627 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.631+0000 7f4c1d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c200d29e0 0x7f4c200c96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39110/0 (socket says 172.21.15.159:39110) 2024-04-04T06:05:33.628 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.631+0000 7f4c1d400640 1 -- 172.21.15.159:0/1457327272 learned_addr learned my addr 172.21.15.159:0/1457327272 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:33.628 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.631+0000 7f4c1de00640 1 --2- 172.21.15.159:0/1457327272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c200b6a90 0x7f4c2014c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:33.628 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.631+0000 7f4c1e800640 1 --2- 172.21.15.159:0/1457327272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c2015e630 0x7f4c200b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:33.628 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.631+0000 7f4c1d400640 1 -- 172.21.15.159:0/1457327272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c200b6a90 msgr2=0x7f4c2014c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:33.628 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.631+0000 7f4c1d400640 1 --2- 172.21.15.159:0/1457327272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c200b6a90 0x7f4c2014c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:33.628 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.631+0000 7f4c1d400640 1 -- 172.21.15.159:0/1457327272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c2015e630 msgr2=0x7f4c200b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:33.628 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.631+0000 7f4c1d400640 1 --2- 172.21.15.159:0/1457327272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c2015e630 0x7f4c200b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:33.628 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.631+0000 7f4c1d400640 1 -- 172.21.15.159:0/1457327272 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c200c7fd0 con 0x7f4c200d29e0 2024-04-04T06:05:33.628 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.631+0000 7f4c1e800640 1 --2- 172.21.15.159:0/1457327272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c2015e630 0x7f4c200b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:05:33.628 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.632+0000 7f4c1d400640 1 --2- 172.21.15.159:0/1457327272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c200d29e0 0x7f4c200c96d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00c440 tx=0x7f4c0c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:33.628 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.632+0000 7f4c16000640 1 -- 172.21.15.159:0/1457327272 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c0052d0 con 0x7f4c200d29e0 2024-04-04T06:05:33.628 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.632+0000 7f4c16000640 1 -- 172.21.15.159:0/1457327272 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c0c005470 con 0x7f4c200d29e0 2024-04-04T06:05:33.628 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.632+0000 7f4c1f200640 1 -- 172.21.15.159:0/1457327272 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c200c82b0 con 0x7f4c200d29e0 2024-04-04T06:05:33.628 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.632+0000 7f4c16000640 1 -- 172.21.15.159:0/1457327272 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c007260 con 0x7f4c200d29e0 2024-04-04T06:05:33.628 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.632+0000 7f4c1f200640 1 -- 172.21.15.159:0/1457327272 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c200c8870 con 0x7f4c200d29e0 2024-04-04T06:05:33.629 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.633+0000 7f4c1f200640 1 -- 172.21.15.159:0/1457327272 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c20075a60 con 0x7f4c200d29e0 2024-04-04T06:05:33.629 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.633+0000 7f4c16000640 1 -- 172.21.15.159:0/1457327272 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 2) ==== 45545+0+0 (secure 0 0 0) 0x7f4c0c007400 con 0x7f4c200d29e0 2024-04-04T06:05:33.629 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.634+0000 7f4c16000640 1 -- 172.21.15.159:0/1457327272 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f4c0c049ff0 con 0x7f4c200d29e0 2024-04-04T06:05:33.630 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.636+0000 7f4c16000640 1 -- 172.21.15.159:0/1457327272 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4c0c012a10 con 0x7f4c200d29e0 2024-04-04T06:05:33.693 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.698+0000 7f4c1f200640 1 -- 172.21.15.159:0/1457327272 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c200b9360 con 0x7f4c200d29e0 2024-04-04T06:05:33.694 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.699+0000 7f4c16000640 1 -- 172.21.15.159:0/1457327272 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f4c0c012c70 con 0x7f4c200d29e0 2024-04-04T06:05:33.694 INFO:teuthology.misc.health.smithi159.stdout: 2024-04-04T06:05:33.695 INFO:teuthology.misc.health.smithi159.stdout:{"epoch":10,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:05:32.259925+0000","last_up_change":"0.000000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:05:33.695 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.700+0000 7f4c1f200640 1 -- 172.21.15.159:0/1457327272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c200d29e0 msgr2=0x7f4c200c96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:33.695 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.700+0000 7f4c1f200640 1 --2- 172.21.15.159:0/1457327272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c200d29e0 0x7f4c200c96d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00c440 tx=0x7f4c0c001e70 comp rx=0 tx=0).stop 2024-04-04T06:05:33.696 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.700+0000 7f4c1f200640 1 -- 172.21.15.159:0/1457327272 shutdown_connections 2024-04-04T06:05:33.696 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.700+0000 7f4c1f200640 1 --2- 172.21.15.159:0/1457327272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c2015e630 0x7f4c200b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:33.696 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.700+0000 7f4c1f200640 1 --2- 172.21.15.159:0/1457327272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c200d29e0 0x7f4c200c96d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:33.696 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.700+0000 7f4c1f200640 1 --2- 172.21.15.159:0/1457327272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c200b6a90 0x7f4c2014c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:33.696 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.700+0000 7f4c1f200640 1 -- 172.21.15.159:0/1457327272 >> 172.21.15.159:0/1457327272 conn(0x7f4c20067620 msgr2=0x7f4c200b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:33.696 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.700+0000 7f4c1f200640 1 -- 172.21.15.159:0/1457327272 shutdown_connections 2024-04-04T06:05:33.696 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:33.701+0000 7f4c1f200640 1 -- 172.21.15.159:0/1457327272 wait complete. 2024-04-04T06:05:33.709 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-04T06:05:33.740 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:33.745+0000 7f0a10400640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-04T06:05:33.741 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-04T06:05:33.746+0000 7f0a10400640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-04T06:05:36.349 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-04-04T06:05:36.354+0000 7f379a40c640 -1 osd.3 0 log_to_monitors true 2024-04-04T06:05:36.485 INFO:tasks.ceph.osd.5.smithi178.stderr:2024-04-04T06:05:36.490+0000 7f31e5868640 -1 osd.5 0 log_to_monitors true 2024-04-04T06:05:36.504 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-04-04T06:05:36.510+0000 7fac112ae640 -1 osd.0 0 log_to_monitors true 2024-04-04T06:05:36.557 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-04-04T06:05:36.562+0000 7f7a13c01640 -1 osd.2 0 log_to_monitors true 2024-04-04T06:05:36.570 INFO:tasks.ceph.osd.4.smithi178.stderr:2024-04-04T06:05:36.575+0000 7fd8c0acc640 -1 osd.4 0 log_to_monitors true 2024-04-04T06:05:36.592 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-04-04T06:05:36.597+0000 7eff36e68640 -1 osd.1 0 log_to_monitors true 2024-04-04T06:05:36.785 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-04T06:05:36.790+0000 7f2754837640 -1 osd.7 0 log_to_monitors true 2024-04-04T06:05:36.832 INFO:tasks.ceph.osd.6.smithi178.stderr:2024-04-04T06:05:36.837+0000 7f4019813640 -1 osd.6 0 log_to_monitors true 2024-04-04T06:05:39.033 INFO:tasks.ceph.osd.4.smithi178.stderr:2024-04-04T06:05:39.039+0000 7fd8bc400640 -1 osd.4 0 waiting for initial osdmap 2024-04-04T06:05:39.042 INFO:tasks.ceph.osd.4.smithi178.stderr:2024-04-04T06:05:39.048+0000 7fd8b1a00640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-04T06:05:39.051 INFO:tasks.ceph.osd.5.smithi178.stderr:2024-04-04T06:05:39.057+0000 7f31e1200640 -1 osd.5 0 waiting for initial osdmap 2024-04-04T06:05:39.058 INFO:tasks.ceph.osd.5.smithi178.stderr:2024-04-04T06:05:39.064+0000 7f31d6800640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-04T06:05:40.048 INFO:tasks.ceph.osd.6.smithi178.stderr:2024-04-04T06:05:40.054+0000 7f4014a00640 -1 osd.6 0 waiting for initial osdmap 2024-04-04T06:05:40.055 INFO:tasks.ceph.osd.6.smithi178.stderr:2024-04-04T06:05:40.062+0000 7f400a000640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-04T06:05:40.059 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-04T06:05:40.066+0000 7f274f600640 -1 osd.7 0 waiting for initial osdmap 2024-04-04T06:05:40.068 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-04T06:05:40.074+0000 7f2745600640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-04T06:05:40.091 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-04-04T06:05:40.097+0000 7fac0cc00640 -1 osd.0 0 waiting for initial osdmap 2024-04-04T06:05:40.099 INFO:tasks.ceph.osd.0.smithi159.stderr:2024-04-04T06:05:40.105+0000 7fac02200640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-04T06:05:40.118 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-04-04T06:05:40.125+0000 7f7a0ea00640 -1 osd.2 0 waiting for initial osdmap 2024-04-04T06:05:40.131 INFO:tasks.ceph.osd.2.smithi159.stderr:2024-04-04T06:05:40.137+0000 7f7a04a00640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-04T06:05:40.142 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-04-04T06:05:40.149+0000 7f3794c00640 -1 osd.3 0 waiting for initial osdmap 2024-04-04T06:05:40.151 INFO:tasks.ceph.osd.3.smithi159.stderr:2024-04-04T06:05:40.157+0000 7f378ac00640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-04T06:05:40.152 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-04-04T06:05:40.159+0000 7eff31e00640 -1 osd.1 0 waiting for initial osdmap 2024-04-04T06:05:40.159 INFO:tasks.ceph.osd.1.smithi159.stderr:2024-04-04T06:05:40.166+0000 7eff27e00640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-04T06:05:40.407 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:05:40.414 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:05:40.515 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-04T06:05:40.688 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.694+0000 7ff92b000640 1 Processor -- start 2024-04-04T06:05:40.688 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.694+0000 7ff92b000640 1 -- start start 2024-04-04T06:05:40.691 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.696+0000 7ff92b000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9240b2f00 0x7ff9240730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:40.692 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.696+0000 7ff92b000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9240adc40 con 0x7ff924158e90 2024-04-04T06:05:40.692 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.696+0000 7ff92b000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff924073710 con 0x7ff92406ed80 2024-04-04T06:05:40.692 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.696+0000 7ff92b000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff9240ae530 con 0x7ff9240b2f00 2024-04-04T06:05:40.692 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.696+0000 7ff923e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7ff924158e90 0x7ff92406e670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37610/0 (socket says 172.21.15.159:37610) 2024-04-04T06:05:40.692 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.696+0000 7ff923e00640 1 -- 172.21.15.159:0/760884515 learned_addr learned my addr 172.21.15.159:0/760884515 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:05:40.692 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.696+0000 7ff923400640 1 -- 172.21.15.159:0/760884515 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3048763862 0 0) 0x7ff9240adc40 con 0x7ff924158e90 2024-04-04T06:05:40.692 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.696+0000 7ff923400640 1 -- 172.21.15.159:0/760884515 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff908005730 con 0x7ff924158e90 2024-04-04T06:05:40.692 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.696+0000 7ff929000640 1 --2- 172.21.15.159:0/760884515 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9240b2f00 0x7ff9240730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:40.692 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.696+0000 7ff923400640 1 -- 172.21.15.159:0/760884515 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 764055355 0 0) 0x7ff908005730 con 0x7ff924158e90 2024-04-04T06:05:40.692 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.696+0000 7ff923400640 1 -- 172.21.15.159:0/760884515 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9240b2f00 msgr2=0x7ff9240730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:40.692 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.696+0000 7ff923400640 1 --2- 172.21.15.159:0/760884515 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9240b2f00 0x7ff9240730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:40.692 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.696+0000 7ff923400640 1 -- 172.21.15.159:0/760884515 >> v1:172.21.15.178:6789/0 conn(0x7ff92406ed80 legacy=0x7ff9240729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:40.693 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.696+0000 7ff923400640 1 -- 172.21.15.159:0/760884515 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9240adee0 con 0x7ff924158e90 2024-04-04T06:05:40.693 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.697+0000 7ff923400640 1 -- 172.21.15.159:0/760884515 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7ff914001690 con 0x7ff924158e90 2024-04-04T06:05:40.693 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.697+0000 7ff923400640 1 -- 172.21.15.159:0/760884515 >> v1:172.21.15.159:6789/0 conn(0x7ff924158e90 legacy=0x7ff92406e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:40.693 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.697+0000 7ff923400640 1 --2- 172.21.15.159:0/760884515 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9240b2f00 0x7ff908005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:40.693 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.697+0000 7ff923400640 1 --2- 172.21.15.159:0/760884515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff908006250 0x7ff908006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:40.693 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.697+0000 7ff923400640 1 --2- 172.21.15.159:0/760884515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff908006f50 0x7ff9080073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:40.693 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.697+0000 7ff923400640 1 -- 172.21.15.159:0/760884515 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff9240ae530 con 0x7ff908006f50 2024-04-04T06:05:40.693 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.697+0000 7ff923400640 1 -- 172.21.15.159:0/760884515 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff90800ddc0 con 0x7ff908006250 2024-04-04T06:05:40.693 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.697+0000 7ff923400640 1 -- 172.21.15.159:0/760884515 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff90800dfc0 con 0x7ff9240b2f00 2024-04-04T06:05:40.693 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.697+0000 7ff929000640 1 --2- 172.21.15.159:0/760884515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff908006f50 0x7ff9080073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:40.693 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.697+0000 7ff91b400640 1 --2- 172.21.15.159:0/760884515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff908006250 0x7ff908006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:40.693 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.697+0000 7ff923e00640 1 --2- 172.21.15.159:0/760884515 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9240b2f00 0x7ff908005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:40.693 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.697+0000 7ff929000640 1 -- 172.21.15.159:0/760884515 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9240b2f00 msgr2=0x7ff908005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:40.693 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.697+0000 7ff929000640 1 --2- 172.21.15.159:0/760884515 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9240b2f00 0x7ff908005bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:40.693 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.697+0000 7ff929000640 1 -- 172.21.15.159:0/760884515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff908006250 msgr2=0x7ff908006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:40.694 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.697+0000 7ff929000640 1 --2- 172.21.15.159:0/760884515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff908006250 0x7ff908006a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:40.694 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.697+0000 7ff929000640 1 -- 172.21.15.159:0/760884515 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff908005730 con 0x7ff908006f50 2024-04-04T06:05:40.695 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.698+0000 7ff929000640 1 --2- 172.21.15.159:0/760884515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff908006f50 0x7ff9080073d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff91c007360 tx=0x7ff908007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e71538aea8400b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:40.695 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.699+0000 7ff923400640 1 -- 172.21.15.159:0/760884515 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c039070 con 0x7ff908006f50 2024-04-04T06:05:40.695 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.699+0000 7ff923400640 1 -- 172.21.15.159:0/760884515 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff91c02ccd0 con 0x7ff908006f50 2024-04-04T06:05:40.695 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.699+0000 7ff923400640 1 -- 172.21.15.159:0/760884515 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c009be0 con 0x7ff908006f50 2024-04-04T06:05:40.695 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.699+0000 7ff92b000640 1 -- 172.21.15.159:0/760884515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff908006f50 msgr2=0x7ff9080073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:40.695 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.699+0000 7ff92b000640 1 --2- 172.21.15.159:0/760884515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff908006f50 0x7ff9080073d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff91c007360 tx=0x7ff908007a40 comp rx=0 tx=0).stop 2024-04-04T06:05:40.695 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.699+0000 7ff929000640 1 -- 172.21.15.159:0/760884515 reap_dead start 2024-04-04T06:05:40.696 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.699+0000 7ff92b000640 1 -- 172.21.15.159:0/760884515 shutdown_connections 2024-04-04T06:05:40.696 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.699+0000 7ff92b000640 1 -- 172.21.15.159:0/760884515 >> 172.21.15.159:0/760884515 conn(0x7ff92406bba0 msgr2=0x7ff9240b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:40.696 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.699+0000 7ff92b000640 1 -- 172.21.15.159:0/760884515 shutdown_connections 2024-04-04T06:05:40.696 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.699+0000 7ff92b000640 1 -- 172.21.15.159:0/760884515 wait complete. 2024-04-04T06:05:40.696 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.701+0000 7ff92b000640 1 Processor -- start 2024-04-04T06:05:40.696 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.701+0000 7ff92b000640 1 -- start start 2024-04-04T06:05:40.698 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.702+0000 7ff92b000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff908006f50 0x7ff9241574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:40.698 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.702+0000 7ff92b000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff92414bde0 0x7ff92415d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:40.698 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.702+0000 7ff92b000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9241562f0 0x7ff9241567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:40.698 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.702+0000 7ff92b000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff924073230 con 0x7ff908006f50 2024-04-04T06:05:40.698 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.702+0000 7ff92b000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9240730b0 con 0x7ff92414bde0 2024-04-04T06:05:40.699 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.702+0000 7ff92b000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff9240733b0 con 0x7ff9241562f0 2024-04-04T06:05:40.699 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.702+0000 7ff923e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff908006f50 0x7ff9241574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:40.699 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.702+0000 7ff923e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff908006f50 0x7ff9241574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58984/0 (socket says 172.21.15.159:58984) 2024-04-04T06:05:40.699 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.702+0000 7ff923e00640 1 -- 172.21.15.159:0/664299785 learned_addr learned my addr 172.21.15.159:0/664299785 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:40.699 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.702+0000 7ff91b400640 1 --2- 172.21.15.159:0/664299785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff92414bde0 0x7ff92415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:40.699 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.703+0000 7ff923e00640 1 -- 172.21.15.159:0/664299785 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9241562f0 msgr2=0x7ff9241567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:40.699 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.703+0000 7ff923e00640 1 --2- 172.21.15.159:0/664299785 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9241562f0 0x7ff9241567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:40.699 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.703+0000 7ff923e00640 1 -- 172.21.15.159:0/664299785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff92414bde0 msgr2=0x7ff92415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:40.699 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.703+0000 7ff923e00640 1 --2- 172.21.15.159:0/664299785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff92414bde0 0x7ff92415d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:40.699 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.703+0000 7ff923e00640 1 -- 172.21.15.159:0/664299785 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92415db70 con 0x7ff908006f50 2024-04-04T06:05:40.699 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.703+0000 7ff923e00640 1 --2- 172.21.15.159:0/664299785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff908006f50 0x7ff9241574c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff914001010 tx=0x7ff91400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:40.699 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.703+0000 7ff920c00640 1 -- 172.21.15.159:0/664299785 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914019070 con 0x7ff908006f50 2024-04-04T06:05:40.699 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.703+0000 7ff920c00640 1 -- 172.21.15.159:0/664299785 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff91400de50 con 0x7ff908006f50 2024-04-04T06:05:40.699 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.703+0000 7ff920c00640 1 -- 172.21.15.159:0/664299785 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9140047e0 con 0x7ff908006f50 2024-04-04T06:05:40.700 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.703+0000 7ff92b000640 1 -- 172.21.15.159:0/664299785 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92415de10 con 0x7ff908006f50 2024-04-04T06:05:40.700 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.703+0000 7ff92b000640 1 -- 172.21.15.159:0/664299785 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff92415e3d0 con 0x7ff908006f50 2024-04-04T06:05:40.700 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.705+0000 7ff92b000640 1 -- 172.21.15.159:0/664299785 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff92404f8b0 con 0x7ff908006f50 2024-04-04T06:05:40.700 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.705+0000 7ff920c00640 1 -- 172.21.15.159:0/664299785 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7ff914004980 con 0x7ff908006f50 2024-04-04T06:05:40.700 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.706+0000 7ff920c00640 1 --2- 172.21.15.159:0/664299785 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff90006c860 0x7ff90006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:40.700 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.707+0000 7ff920c00640 1 -- 172.21.15.159:0/664299785 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2791+0+0 (secure 0 0 0) 0x7ff91408a7a0 con 0x7ff908006f50 2024-04-04T06:05:40.701 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.707+0000 7ff91b400640 1 --2- 172.21.15.159:0/664299785 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff90006c860 0x7ff90006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:40.702 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.708+0000 7ff91b400640 1 --2- 172.21.15.159:0/664299785 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff90006c860 0x7ff90006ed20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff924071ba0 tx=0x7ff90c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:40.706 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.712+0000 7ff920c00640 1 -- 172.21.15.159:0/664299785 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff914004c70 con 0x7ff908006f50 2024-04-04T06:05:40.900 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.906+0000 7ff92b000640 1 -- 172.21.15.159:0/664299785 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9240b4aa0 con 0x7ff908006f50 2024-04-04T06:05:40.902 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.908+0000 7ff920c00640 1 -- 172.21.15.159:0/664299785 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+8029 (secure 0 0 0) 0x7ff9140598a0 con 0x7ff908006f50 2024-04-04T06:05:40.902 INFO:teuthology.misc.health.smithi159.stdout: 2024-04-04T06:05:40.902 INFO:teuthology.misc.health.smithi159.stdout:{"epoch":14,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:05:39.880993+0000","last_up_change":"2024-04-04T06:05:39.880993+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:05:40.904 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.910+0000 7ff92b000640 1 -- 172.21.15.159:0/664299785 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff90006c860 msgr2=0x7ff90006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:40.904 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.910+0000 7ff92b000640 1 --2- 172.21.15.159:0/664299785 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff90006c860 0x7ff90006ed20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff924071ba0 tx=0x7ff90c009040 comp rx=0 tx=0).stop 2024-04-04T06:05:40.904 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.910+0000 7ff92b000640 1 -- 172.21.15.159:0/664299785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff908006f50 msgr2=0x7ff9241574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:40.905 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.910+0000 7ff92b000640 1 --2- 172.21.15.159:0/664299785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff908006f50 0x7ff9241574c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff914001010 tx=0x7ff91400dc00 comp rx=0 tx=0).stop 2024-04-04T06:05:40.905 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.910+0000 7ff92b000640 1 -- 172.21.15.159:0/664299785 shutdown_connections 2024-04-04T06:05:40.905 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.910+0000 7ff92b000640 1 --2- 172.21.15.159:0/664299785 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9241562f0 0x7ff9241567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:40.905 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.910+0000 7ff92b000640 1 --2- 172.21.15.159:0/664299785 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff90006c860 0x7ff90006ed20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:40.905 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.910+0000 7ff92b000640 1 --2- 172.21.15.159:0/664299785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff92414bde0 0x7ff92415d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:40.905 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.910+0000 7ff92b000640 1 --2- 172.21.15.159:0/664299785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff908006f50 0x7ff9241574c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:40.905 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.910+0000 7ff92b000640 1 -- 172.21.15.159:0/664299785 >> 172.21.15.159:0/664299785 conn(0x7ff92406bba0 msgr2=0x7ff9240cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:40.905 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.911+0000 7ff92b000640 1 -- 172.21.15.159:0/664299785 shutdown_connections 2024-04-04T06:05:40.905 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:40.911+0000 7ff92b000640 1 -- 172.21.15.159:0/664299785 wait complete. 2024-04-04T06:05:40.917 DEBUG:teuthology.misc:2 of 8 OSDs are up 2024-04-04T06:05:47.724 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-04T06:05:47.897 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.902+0000 7f82fee00640 1 Processor -- start 2024-04-04T06:05:47.897 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.902+0000 7f82fee00640 1 -- start start 2024-04-04T06:05:47.899 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.904+0000 7f82fee00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f82f80b22f0 0x7f82f80b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:47.899 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.904+0000 7f82fee00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f82f815e680 0x7f82f80af530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:47.899 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.904+0000 7f82fee00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82f80aee90 con 0x7f82f815e2a0 2024-04-04T06:05:47.900 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.904+0000 7f82fee00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f82f80b3310 con 0x7f82f815e680 2024-04-04T06:05:47.900 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.904+0000 7f82fee00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f82f80732d0 con 0x7f82f80b22f0 2024-04-04T06:05:47.900 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.904+0000 7f82fe400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f82f815e680 0x7f82f80af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:47.900 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.904+0000 7f82fe400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f82f815e680 0x7f82f80af530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47918/0 (socket says 172.21.15.159:47918) 2024-04-04T06:05:47.900 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.904+0000 7f82fe400640 1 -- 172.21.15.159:0/1759742468 learned_addr learned my addr 172.21.15.159:0/1759742468 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:47.900 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.904+0000 7f82fd000640 1 --2- 172.21.15.159:0/1759742468 >> v2:172.21.15.178:3300/0 conn(0x7f82f80b22f0 0x7f82f80b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:47.900 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.905+0000 7f82fe400640 1 -- 172.21.15.159:0/1759742468 >> v2:172.21.15.178:3300/0 conn(0x7f82f80b22f0 msgr2=0x7f82f80b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:47.900 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.905+0000 7f82fe400640 1 --2- 172.21.15.159:0/1759742468 >> v2:172.21.15.178:3300/0 conn(0x7f82f80b22f0 0x7f82f80b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:47.900 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.905+0000 7f82fe400640 1 -- 172.21.15.159:0/1759742468 >> v1:172.21.15.178:6789/0 conn(0x7f82f815e2a0 legacy=0x7f82f80b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:47.900 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.905+0000 7f82fe400640 1 -- 172.21.15.159:0/1759742468 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f80735b0 con 0x7f82f815e680 2024-04-04T06:05:47.900 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.905+0000 7f82fd000640 1 --2- 172.21.15.159:0/1759742468 >> v2:172.21.15.178:3300/0 conn(0x7f82f80b22f0 0x7f82f80b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:05:47.902 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.906+0000 7f82fe400640 1 --2- 172.21.15.159:0/1759742468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82f815e680 0x7f82f80af530 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f82f80b51a0 tx=0x7f82e802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410e4a4f9e2972cf server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:47.902 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.906+0000 7f82f7e00640 1 -- 172.21.15.159:0/1759742468 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8035020 con 0x7f82f815e680 2024-04-04T06:05:47.902 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.906+0000 7f82f7e00640 1 -- 172.21.15.159:0/1759742468 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82e8031870 con 0x7f82f815e680 2024-04-04T06:05:47.902 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.906+0000 7f82f7e00640 1 -- 172.21.15.159:0/1759742468 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8031c40 con 0x7f82f815e680 2024-04-04T06:05:47.902 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.907+0000 7f82fee00640 1 -- 172.21.15.159:0/1759742468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82f815e680 msgr2=0x7f82f80af530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:47.902 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.907+0000 7f82fee00640 1 --2- 172.21.15.159:0/1759742468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82f815e680 0x7f82f80af530 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f82f80b51a0 tx=0x7f82e802b910 comp rx=0 tx=0).stop 2024-04-04T06:05:47.902 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.907+0000 7f82fee00640 1 -- 172.21.15.159:0/1759742468 shutdown_connections 2024-04-04T06:05:47.902 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.907+0000 7f82fee00640 1 --2- 172.21.15.159:0/1759742468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82f815e680 0x7f82f80af530 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:47.902 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.907+0000 7f82fee00640 1 --2- 172.21.15.159:0/1759742468 >> v2:172.21.15.178:3300/0 conn(0x7f82f80b22f0 0x7f82f80b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:47.902 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.907+0000 7f82fee00640 1 -- 172.21.15.159:0/1759742468 >> 172.21.15.159:0/1759742468 conn(0x7f82f806b570 msgr2=0x7f82f806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:47.902 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.907+0000 7f82fee00640 1 -- 172.21.15.159:0/1759742468 shutdown_connections 2024-04-04T06:05:47.903 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.907+0000 7f82fee00640 1 -- 172.21.15.159:0/1759742468 wait complete. 2024-04-04T06:05:47.903 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.909+0000 7f82fee00640 1 Processor -- start 2024-04-04T06:05:47.903 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.909+0000 7f82fee00640 1 -- start start 2024-04-04T06:05:47.906 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.910+0000 7f82fee00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f80b22f0 0x7f82f814ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:47.906 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.910+0000 7f82fee00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82f815e2a0 0x7f82f80ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:47.906 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.910+0000 7f82fee00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82f815e680 0x7f82f80cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:47.906 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.910+0000 7f82fee00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f82f8071650 con 0x7f82f815e680 2024-04-04T06:05:47.906 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.910+0000 7f82fee00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f82f80aeb40 con 0x7f82f80b22f0 2024-04-04T06:05:47.907 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.910+0000 7f82fee00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f82f80717d0 con 0x7f82f815e2a0 2024-04-04T06:05:47.907 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.910+0000 7f82fe400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82f815e680 0x7f82f80cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:47.907 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.910+0000 7f82fe400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82f815e680 0x7f82f80cd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47928/0 (socket says 172.21.15.159:47928) 2024-04-04T06:05:47.907 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.910+0000 7f82fe400640 1 -- 172.21.15.159:0/2055935843 learned_addr learned my addr 172.21.15.159:0/2055935843 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:47.907 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.910+0000 7f82fda00640 1 --2- 172.21.15.159:0/2055935843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f80b22f0 0x7f82f814ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:47.907 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.910+0000 7f82fd000640 1 --2- 172.21.15.159:0/2055935843 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82f815e2a0 0x7f82f80ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:47.907 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.910+0000 7f82fe400640 1 -- 172.21.15.159:0/2055935843 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82f815e2a0 msgr2=0x7f82f80ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:47.907 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.910+0000 7f82fe400640 1 --2- 172.21.15.159:0/2055935843 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82f815e2a0 0x7f82f80ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:47.907 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.910+0000 7f82fe400640 1 -- 172.21.15.159:0/2055935843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f80b22f0 msgr2=0x7f82f814ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:47.907 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.910+0000 7f82fe400640 1 --2- 172.21.15.159:0/2055935843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f80b22f0 0x7f82f814ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:47.907 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.910+0000 7f82fe400640 1 -- 172.21.15.159:0/2055935843 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f80cde40 con 0x7f82f815e680 2024-04-04T06:05:47.907 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.910+0000 7f82fda00640 1 --2- 172.21.15.159:0/2055935843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f80b22f0 0x7f82f814ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:05:47.907 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.911+0000 7f82fe400640 1 --2- 172.21.15.159:0/2055935843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82f815e680 0x7f82f80cd480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f82e802b950 tx=0x7f82e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:47.908 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.911+0000 7f82f5600640 1 -- 172.21.15.159:0/2055935843 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8040070 con 0x7f82f815e680 2024-04-04T06:05:47.908 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.911+0000 7f82f5600640 1 -- 172.21.15.159:0/2055935843 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82e803a420 con 0x7f82f815e680 2024-04-04T06:05:47.908 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.911+0000 7f82fee00640 1 -- 172.21.15.159:0/2055935843 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f814bf70 con 0x7f82f815e680 2024-04-04T06:05:47.908 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.911+0000 7f82f5600640 1 -- 172.21.15.159:0/2055935843 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8035040 con 0x7f82f815e680 2024-04-04T06:05:47.908 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.911+0000 7f82fee00640 1 -- 172.21.15.159:0/2055935843 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f80ce3c0 con 0x7f82f815e680 2024-04-04T06:05:47.908 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.912+0000 7f82fee00640 1 -- 172.21.15.159:0/2055935843 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f8075a60 con 0x7f82f815e680 2024-04-04T06:05:47.909 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.914+0000 7f82f5600640 1 -- 172.21.15.159:0/2055935843 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f82e8038020 con 0x7f82f815e680 2024-04-04T06:05:47.909 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.914+0000 7f82f5600640 1 --2- 172.21.15.159:0/2055935843 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f82d806c8f0 0x7f82d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:47.909 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.915+0000 7f82f5600640 1 -- 172.21.15.159:0/2055935843 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f82e80ad790 con 0x7f82f815e680 2024-04-04T06:05:47.910 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.915+0000 7f82fda00640 1 --2- 172.21.15.159:0/2055935843 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f82d806c8f0 0x7f82d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:47.911 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.916+0000 7f82fda00640 1 --2- 172.21.15.159:0/2055935843 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f82d806c8f0 0x7f82d806edb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f82f815ab50 tx=0x7f82f0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:47.915 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:47.920+0000 7f82f5600640 1 -- 172.21.15.159:0/2055935843 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f82e807bf00 con 0x7f82f815e680 2024-04-04T06:05:48.106 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:48.111+0000 7f82fee00640 1 -- 172.21.15.159:0/2055935843 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82f80ada50 con 0x7f82f815e680 2024-04-04T06:05:48.108 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:48.114+0000 7f82f5600640 1 -- 172.21.15.159:0/2055935843 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12896 (secure 0 0 0) 0x7f82e807b800 con 0x7f82f815e680 2024-04-04T06:05:48.109 INFO:teuthology.misc.health.smithi159.stdout: 2024-04-04T06:05:48.109 INFO:teuthology.misc.health.smithi159.stdout:{"epoch":17,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:05:43.073157+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:05:48.111 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:48.116+0000 7f82fee00640 1 -- 172.21.15.159:0/2055935843 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f82d806c8f0 msgr2=0x7f82d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:48.111 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:48.116+0000 7f82fee00640 1 --2- 172.21.15.159:0/2055935843 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f82d806c8f0 0x7f82d806edb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f82f815ab50 tx=0x7f82f0009040 comp rx=0 tx=0).stop 2024-04-04T06:05:48.111 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:48.116+0000 7f82fee00640 1 -- 172.21.15.159:0/2055935843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82f815e680 msgr2=0x7f82f80cd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:48.111 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:48.116+0000 7f82fee00640 1 --2- 172.21.15.159:0/2055935843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82f815e680 0x7f82f80cd480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f82e802b950 tx=0x7f82e80046c0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.111 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:48.116+0000 7f82fee00640 1 -- 172.21.15.159:0/2055935843 shutdown_connections 2024-04-04T06:05:48.111 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:48.116+0000 7f82fee00640 1 --2- 172.21.15.159:0/2055935843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82f815e680 0x7f82f80cd480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.112 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:48.116+0000 7f82fee00640 1 --2- 172.21.15.159:0/2055935843 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82f815e2a0 0x7f82f80ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.112 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:48.116+0000 7f82fee00640 1 --2- 172.21.15.159:0/2055935843 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f82d806c8f0 0x7f82d806edb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.112 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:48.116+0000 7f82fee00640 1 --2- 172.21.15.159:0/2055935843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82f80b22f0 0x7f82f814ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.112 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:48.116+0000 7f82fee00640 1 -- 172.21.15.159:0/2055935843 >> 172.21.15.159:0/2055935843 conn(0x7f82f806b570 msgr2=0x7f82f80af840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:48.112 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:48.116+0000 7f82fee00640 1 -- 172.21.15.159:0/2055935843 shutdown_connections 2024-04-04T06:05:48.112 INFO:teuthology.misc.health.smithi159.stderr:2024-04-04T06:05:48.116+0000 7f82fee00640 1 -- 172.21.15.159:0/2055935843 wait complete. 2024-04-04T06:05:48.125 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-04T06:05:48.125 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-04T06:05:48.126 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-04T06:05:48.126 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-04T06:05:48.131 INFO:tasks.ceph.mds.a:Started 2024-04-04T06:05:48.131 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-04T06:05:48.131 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-04T06:05:48.137 INFO:tasks.ceph.mds.c:Started 2024-04-04T06:05:48.137 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-04T06:05:48.137 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-04T06:05:48.145 INFO:tasks.ceph.mds.b:Started 2024-04-04T06:05:48.145 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-04T06:05:48.145 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-04T06:05:48.152 INFO:tasks.ceph.mds.d:Started 2024-04-04T06:05:48.152 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-04T06:05:48.152 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:05:48.274 INFO:tasks.ceph.mds.a.smithi159.stdout:starting mds.a at 2024-04-04T06:05:48.281 INFO:tasks.ceph.mds.c.smithi159.stdout:starting mds.c at 2024-04-04T06:05:48.309 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-04T06:05:48.310 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-04T06:05:48.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.334+0000 7fa390e00640 1 Processor -- start 2024-04-04T06:05:48.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.334+0000 7fa390e00640 1 -- start start 2024-04-04T06:05:48.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.335+0000 7fa390e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa38c15e7e0 0x7fa38c0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:48.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.335+0000 7fa390e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa38c0cfb20 0x7fa38c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:48.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.335+0000 7fa390e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa38c15ebc0 con 0x7fa38c06ca00 2024-04-04T06:05:48.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.335+0000 7fa390e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fa38c06d1c0 con 0x7fa38c15e7e0 2024-04-04T06:05:48.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.335+0000 7fa390e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa38c06d4a0 con 0x7fa38c0cfb20 2024-04-04T06:05:48.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.335+0000 7fa389e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa38c15e7e0 0x7fa38c0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:48.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.335+0000 7fa389e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa38c15e7e0 0x7fa38c0d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47972/0 (socket says 172.21.15.159:47972) 2024-04-04T06:05:48.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.335+0000 7fa389e00640 1 -- 172.21.15.159:0/56740533 learned_addr learned my addr 172.21.15.159:0/56740533 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:48.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.336+0000 7fa38a800640 1 --2- 172.21.15.159:0/56740533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa38c0cfb20 0x7fa38c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:48.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.336+0000 7fa389e00640 1 -- 172.21.15.159:0/56740533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa38c0cfb20 msgr2=0x7fa38c0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:48.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.336+0000 7fa389e00640 1 --2- 172.21.15.159:0/56740533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa38c0cfb20 0x7fa38c0cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.336+0000 7fa389e00640 1 -- 172.21.15.159:0/56740533 >> v1:172.21.15.178:6789/0 conn(0x7fa38c06ca00 legacy=0x7fa38c0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:48.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.336+0000 7fa389e00640 1 -- 172.21.15.159:0/56740533 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38c0cf320 con 0x7fa38c15e7e0 2024-04-04T06:05:48.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.337+0000 7fa389e00640 1 --2- 172.21.15.159:0/56740533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa38c15e7e0 0x7fa38c0d1950 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009870 tx=0x7fa37c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362754184767936e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:48.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.337+0000 7fa388a00640 1 -- 172.21.15.159:0/56740533 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c035020 con 0x7fa38c15e7e0 2024-04-04T06:05:48.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.337+0000 7fa388a00640 1 -- 172.21.15.159:0/56740533 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa37c031640 con 0x7fa38c15e7e0 2024-04-04T06:05:48.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.337+0000 7fa388a00640 1 -- 172.21.15.159:0/56740533 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c02d040 con 0x7fa38c15e7e0 2024-04-04T06:05:48.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.338+0000 7fa390e00640 1 -- 172.21.15.159:0/56740533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa38c15e7e0 msgr2=0x7fa38c0d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:48.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.338+0000 7fa390e00640 1 --2- 172.21.15.159:0/56740533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa38c15e7e0 0x7fa38c0d1950 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009870 tx=0x7fa37c02b8f0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.338+0000 7fa390e00640 1 -- 172.21.15.159:0/56740533 shutdown_connections 2024-04-04T06:05:48.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.338+0000 7fa390e00640 1 --2- 172.21.15.159:0/56740533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa38c0cfb20 0x7fa38c0cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.338+0000 7fa390e00640 1 --2- 172.21.15.159:0/56740533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa38c15e7e0 0x7fa38c0d1950 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.338+0000 7fa390e00640 1 -- 172.21.15.159:0/56740533 >> 172.21.15.159:0/56740533 conn(0x7fa38c09d7c0 msgr2=0x7fa38c06bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:48.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.338+0000 7fa390e00640 1 -- 172.21.15.159:0/56740533 shutdown_connections 2024-04-04T06:05:48.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.338+0000 7fa390e00640 1 -- 172.21.15.159:0/56740533 wait complete. 2024-04-04T06:05:48.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.339+0000 7fa390e00640 1 Processor -- start 2024-04-04T06:05:48.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.340+0000 7fa390e00640 1 -- start start 2024-04-04T06:05:48.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.340+0000 7fa390e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa38c06ca00 0x7fa38c0cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:48.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.340+0000 7fa390e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa38c0cfb20 0x7fa38c1e80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:48.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.340+0000 7fa390e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa38c15e7e0 0x7fa38c159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:48.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.340+0000 7fa390e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa38c06ce60 con 0x7fa38c15e7e0 2024-04-04T06:05:48.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.340+0000 7fa390e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa38c0cefd0 con 0x7fa38c0cfb20 2024-04-04T06:05:48.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.340+0000 7fa390e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa38c0d2f30 con 0x7fa38c06ca00 2024-04-04T06:05:48.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.341+0000 7fa38a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa38c15e7e0 0x7fa38c159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:48.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.341+0000 7fa389e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa38c06ca00 0x7fa38c0cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:48.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.341+0000 7fa389400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa38c0cfb20 0x7fa38c1e80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:48.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.341+0000 7fa389e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa38c06ca00 0x7fa38c0cab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:44256/0 (socket says 172.21.15.159:44256) 2024-04-04T06:05:48.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.341+0000 7fa389e00640 1 -- 172.21.15.159:0/3906488794 learned_addr learned my addr 172.21.15.159:0/3906488794 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:48.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.341+0000 7fa389e00640 1 -- 172.21.15.159:0/3906488794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa38c0cfb20 msgr2=0x7fa38c1e80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:48.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.341+0000 7fa389e00640 1 --2- 172.21.15.159:0/3906488794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa38c0cfb20 0x7fa38c1e80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.341+0000 7fa389e00640 1 -- 172.21.15.159:0/3906488794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa38c15e7e0 msgr2=0x7fa38c159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:48.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.341+0000 7fa389e00640 1 --2- 172.21.15.159:0/3906488794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa38c15e7e0 0x7fa38c159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.341+0000 7fa389e00640 1 -- 172.21.15.159:0/3906488794 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38c1e8860 con 0x7fa38c06ca00 2024-04-04T06:05:48.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.341+0000 7fa38a800640 1 --2- 172.21.15.159:0/3906488794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa38c15e7e0 0x7fa38c159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:05:48.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.341+0000 7fa389400640 1 --2- 172.21.15.159:0/3906488794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa38c0cfb20 0x7fa38c1e80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:05:48.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.341+0000 7fa389e00640 1 --2- 172.21.15.159:0/3906488794 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa38c06ca00 0x7fa38c0cab50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa37c000c00 tx=0x7fa37c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:48.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.342+0000 7fa382000640 1 -- 172.21.15.159:0/3906488794 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c040070 con 0x7fa38c06ca00 2024-04-04T06:05:48.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.342+0000 7fa382000640 1 -- 172.21.15.159:0/3906488794 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa37c0399f0 con 0x7fa38c06ca00 2024-04-04T06:05:48.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.342+0000 7fa390e00640 1 -- 172.21.15.159:0/3906488794 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa38c1e8b40 con 0x7fa38c06ca00 2024-04-04T06:05:48.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.342+0000 7fa382000640 1 -- 172.21.15.159:0/3906488794 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c035040 con 0x7fa38c06ca00 2024-04-04T06:05:48.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.342+0000 7fa390e00640 1 -- 172.21.15.159:0/3906488794 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa38c1e9100 con 0x7fa38c06ca00 2024-04-04T06:05:48.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.343+0000 7fa390e00640 1 -- 172.21.15.159:0/3906488794 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa38c0b8040 con 0x7fa38c06ca00 2024-04-04T06:05:48.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.344+0000 7fa382000640 1 -- 172.21.15.159:0/3906488794 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa37c04b060 con 0x7fa38c06ca00 2024-04-04T06:05:48.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.345+0000 7fa382000640 1 --2- 172.21.15.159:0/3906488794 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa36c06c8f0 0x7fa36c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:48.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.345+0000 7fa382000640 1 -- 172.21.15.159:0/3906488794 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fa37c0acec0 con 0x7fa38c06ca00 2024-04-04T06:05:48.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.345+0000 7fa389400640 1 --2- 172.21.15.159:0/3906488794 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa36c06c8f0 0x7fa36c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:48.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.346+0000 7fa389400640 1 --2- 172.21.15.159:0/3906488794 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa36c06c8f0 0x7fa36c06edb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa38c0b5950 tx=0x7fa378009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:48.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.351+0000 7fa382000640 1 -- 172.21.15.159:0/3906488794 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa38c0b8040 con 0x7fa38c06ca00 2024-04-04T06:05:48.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.550+0000 7fa390e00640 1 -- 172.21.15.159:0/3906488794 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa38c0685c0 con 0x7fa38c06ca00 2024-04-04T06:05:48.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.552+0000 7fa382000640 1 -- 172.21.15.159:0/3906488794 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12896 (secure 0 0 0) 0x7fa37c07afe0 con 0x7fa38c06ca00 2024-04-04T06:05:48.548 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:05:48.548 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":17,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:05:43.073157+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:05:48.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.555+0000 7fa390e00640 1 -- 172.21.15.159:0/3906488794 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa36c06c8f0 msgr2=0x7fa36c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:48.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.555+0000 7fa390e00640 1 --2- 172.21.15.159:0/3906488794 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa36c06c8f0 0x7fa36c06edb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa38c0b5950 tx=0x7fa378009040 comp rx=0 tx=0).stop 2024-04-04T06:05:48.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.555+0000 7fa390e00640 1 -- 172.21.15.159:0/3906488794 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa38c06ca00 msgr2=0x7fa38c0cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:48.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.555+0000 7fa390e00640 1 --2- 172.21.15.159:0/3906488794 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa38c06ca00 0x7fa38c0cab50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa37c000c00 tx=0x7fa37c0046c0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.555+0000 7fa390e00640 1 -- 172.21.15.159:0/3906488794 shutdown_connections 2024-04-04T06:05:48.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.555+0000 7fa390e00640 1 --2- 172.21.15.159:0/3906488794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa38c15e7e0 0x7fa38c159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.555+0000 7fa390e00640 1 --2- 172.21.15.159:0/3906488794 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa36c06c8f0 0x7fa36c06edb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.555+0000 7fa390e00640 1 --2- 172.21.15.159:0/3906488794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa38c0cfb20 0x7fa38c1e80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.555+0000 7fa390e00640 1 --2- 172.21.15.159:0/3906488794 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa38c06ca00 0x7fa38c0cab50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.555+0000 7fa390e00640 1 -- 172.21.15.159:0/3906488794 >> 172.21.15.159:0/3906488794 conn(0x7fa38c09d7c0 msgr2=0x7fa38c0b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:48.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.555+0000 7fa390e00640 1 -- 172.21.15.159:0/3906488794 shutdown_connections 2024-04-04T06:05:48.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.555+0000 7fa390e00640 1 -- 172.21.15.159:0/3906488794 wait complete. 2024-04-04T06:05:48.567 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:05:48.568 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:05:48.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.780+0000 7f8308c00640 1 Processor -- start 2024-04-04T06:05:48.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.780+0000 7f8308c00640 1 -- start start 2024-04-04T06:05:48.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.781+0000 7f8308c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f83040b4890 0x7f8304072570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:48.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.781+0000 7f8308c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83040b4c70 con 0x7f830415f6a0 2024-04-04T06:05:48.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.781+0000 7f8308c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8304072200 con 0x7f830406d620 2024-04-04T06:05:48.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.781+0000 7f8308c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f83040b3eb0 con 0x7f83040b4890 2024-04-04T06:05:48.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.782+0000 7f82fbe00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f830415f6a0 0x7f83040713e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58186/0 (socket says 172.21.15.159:58186) 2024-04-04T06:05:48.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.782+0000 7f82fbe00640 1 -- 172.21.15.159:0/3595245790 learned_addr learned my addr 172.21.15.159:0/3595245790 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:05:48.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.782+0000 7f8303400640 1 --2- 172.21.15.159:0/3595245790 >> v2:172.21.15.159:3300/0 conn(0x7f83040b4890 0x7f8304072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:48.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.782+0000 7f8302a00640 1 -- 172.21.15.159:0/3595245790 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3644708445 0 0) 0x7f83040b4c70 con 0x7f830415f6a0 2024-04-04T06:05:48.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.782+0000 7f8303400640 1 -- 172.21.15.159:0/3595245790 >> v1:172.21.15.178:6789/0 conn(0x7f830406d620 legacy=0x7f8304071af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:48.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.782+0000 7f8303400640 1 -- 172.21.15.159:0/3595245790 >> v1:172.21.15.159:6789/0 conn(0x7f830415f6a0 legacy=0x7f83040713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:48.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.782+0000 7f8303400640 1 -- 172.21.15.159:0/3595245790 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830415fb30 con 0x7f83040b4890 2024-04-04T06:05:48.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.783+0000 7f8303400640 1 --2- 172.21.15.159:0/3595245790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83040b4890 0x7f8304072570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f82f40097a0 tx=0x7f82f402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701129242d050b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:48.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.783+0000 7f8302a00640 1 -- 172.21.15.159:0/3595245790 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4035020 con 0x7f83040b4890 2024-04-04T06:05:48.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.783+0000 7f8302a00640 1 -- 172.21.15.159:0/3595245790 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f4031830 con 0x7f83040b4890 2024-04-04T06:05:48.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.783+0000 7f8302a00640 1 -- 172.21.15.159:0/3595245790 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4031c00 con 0x7f83040b4890 2024-04-04T06:05:48.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.784+0000 7f8308c00640 1 -- 172.21.15.159:0/3595245790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83040b4890 msgr2=0x7f8304072570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:48.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.784+0000 7f8308c00640 1 --2- 172.21.15.159:0/3595245790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83040b4890 0x7f8304072570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f82f40097a0 tx=0x7f82f402b8d0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.784+0000 7f8308c00640 1 -- 172.21.15.159:0/3595245790 shutdown_connections 2024-04-04T06:05:48.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.784+0000 7f8308c00640 1 --2- 172.21.15.159:0/3595245790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83040b4890 0x7f8304072570 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.784+0000 7f8308c00640 1 -- 172.21.15.159:0/3595245790 >> 172.21.15.159:0/3595245790 conn(0x7f83040b1ba0 msgr2=0x7f83040b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:48.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.784+0000 7f8308c00640 1 -- 172.21.15.159:0/3595245790 shutdown_connections 2024-04-04T06:05:48.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.784+0000 7f8308c00640 1 -- 172.21.15.159:0/3595245790 wait complete. 2024-04-04T06:05:48.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.786+0000 7f8308c00640 1 Processor -- start 2024-04-04T06:05:48.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.786+0000 7f8308c00640 1 -- start start 2024-04-04T06:05:48.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.787+0000 7f8308c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f830406d620 0x7f8304154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:48.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.787+0000 7f8308c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83040b4890 0x7f8304158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:48.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.787+0000 7f8308c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8304149dd0 0x7f83040b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:48.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.787+0000 7f8308c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f830415bd70 con 0x7f83040b4890 2024-04-04T06:05:48.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.787+0000 7f8308c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f830415ba90 con 0x7f8304149dd0 2024-04-04T06:05:48.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.787+0000 7f8308c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f830415c050 con 0x7f830406d620 2024-04-04T06:05:48.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.787+0000 7f82fbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83040b4890 0x7f8304158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:48.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.787+0000 7f82fbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83040b4890 0x7f8304158890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48002/0 (socket says 172.21.15.159:48002) 2024-04-04T06:05:48.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.787+0000 7f82fbe00640 1 -- 172.21.15.159:0/1075600433 learned_addr learned my addr 172.21.15.159:0/1075600433 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:48.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.787+0000 7f8303400640 1 --2- 172.21.15.159:0/1075600433 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f830406d620 0x7f8304154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:48.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.787+0000 7f8303e00640 1 --2- 172.21.15.159:0/1075600433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8304149dd0 0x7f83040b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:48.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.787+0000 7f82fbe00640 1 -- 172.21.15.159:0/1075600433 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f830406d620 msgr2=0x7f8304154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:48.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.787+0000 7f82fbe00640 1 --2- 172.21.15.159:0/1075600433 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f830406d620 0x7f8304154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.787+0000 7f82fbe00640 1 -- 172.21.15.159:0/1075600433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8304149dd0 msgr2=0x7f83040b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:48.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.788+0000 7f82fbe00640 1 --2- 172.21.15.159:0/1075600433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8304149dd0 0x7f83040b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.788+0000 7f82fbe00640 1 -- 172.21.15.159:0/1075600433 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8304158dd0 con 0x7f83040b4890 2024-04-04T06:05:48.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.788+0000 7f82fbe00640 1 --2- 172.21.15.159:0/1075600433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83040b4890 0x7f8304158890 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c4d0 tx=0x7f82ec00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:48.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.788+0000 7f82fb400640 1 -- 172.21.15.159:0/1075600433 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec018070 con 0x7f83040b4890 2024-04-04T06:05:48.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.788+0000 7f82fb400640 1 -- 172.21.15.159:0/1075600433 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82ec007500 con 0x7f83040b4890 2024-04-04T06:05:48.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.788+0000 7f8308c00640 1 -- 172.21.15.159:0/1075600433 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8304154950 con 0x7f83040b4890 2024-04-04T06:05:48.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.788+0000 7f82fb400640 1 -- 172.21.15.159:0/1075600433 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec004a20 con 0x7f83040b4890 2024-04-04T06:05:48.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.788+0000 7f8308c00640 1 -- 172.21.15.159:0/1075600433 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83040cec50 con 0x7f83040b4890 2024-04-04T06:05:48.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.789+0000 7f8308c00640 1 -- 172.21.15.159:0/1075600433 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8304075940 con 0x7f83040b4890 2024-04-04T06:05:48.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.790+0000 7f82fb400640 1 -- 172.21.15.159:0/1075600433 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f82ec01b070 con 0x7f83040b4890 2024-04-04T06:05:48.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.791+0000 7f82fb400640 1 --2- 172.21.15.159:0/1075600433 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f82e006c8f0 0x7f82e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:48.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.791+0000 7f82fb400640 1 -- 172.21.15.159:0/1075600433 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f82ec08ab70 con 0x7f83040b4890 2024-04-04T06:05:48.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.792+0000 7f8303400640 1 --2- 172.21.15.159:0/1075600433 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f82e006c8f0 0x7f82e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:48.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.793+0000 7f8303400640 1 --2- 172.21.15.159:0/1075600433 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f82e006c8f0 0x7f82e006edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f82f4036000 tx=0x7f82f4008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:48.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.796+0000 7f82fb400640 1 -- 172.21.15.159:0/1075600433 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f82ec0592e0 con 0x7f83040b4890 2024-04-04T06:05:48.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.995+0000 7f8308c00640 1 -- 172.21.15.159:0/1075600433 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f83040cea30 con 0x7f83040b4890 2024-04-04T06:05:48.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.996+0000 7f82fb400640 1 -- 172.21.15.159:0/1075600433 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f82ec058be0 con 0x7f83040b4890 2024-04-04T06:05:48.991 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:05:48.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.998+0000 7f8308c00640 1 -- 172.21.15.159:0/1075600433 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f82e006c8f0 msgr2=0x7f82e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:48.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.998+0000 7f8308c00640 1 --2- 172.21.15.159:0/1075600433 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f82e006c8f0 0x7f82e006edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f82f4036000 tx=0x7f82f4008030 comp rx=0 tx=0).stop 2024-04-04T06:05:48.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.998+0000 7f8308c00640 1 -- 172.21.15.159:0/1075600433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83040b4890 msgr2=0x7f8304158890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:48.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.998+0000 7f8308c00640 1 --2- 172.21.15.159:0/1075600433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83040b4890 0x7f8304158890 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c4d0 tx=0x7f82ec00c990 comp rx=0 tx=0).stop 2024-04-04T06:05:48.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.998+0000 7f8308c00640 1 -- 172.21.15.159:0/1075600433 shutdown_connections 2024-04-04T06:05:48.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.998+0000 7f8308c00640 1 --2- 172.21.15.159:0/1075600433 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f82e006c8f0 0x7f82e006edb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.998+0000 7f8308c00640 1 --2- 172.21.15.159:0/1075600433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8304149dd0 0x7f83040b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.998+0000 7f8308c00640 1 --2- 172.21.15.159:0/1075600433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83040b4890 0x7f8304158890 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.998+0000 7f8308c00640 1 --2- 172.21.15.159:0/1075600433 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f830406d620 0x7f8304154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:48.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.998+0000 7f8308c00640 1 -- 172.21.15.159:0/1075600433 >> 172.21.15.159:0/1075600433 conn(0x7f83040b1ba0 msgr2=0x7f83040b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:48.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.998+0000 7f8308c00640 1 -- 172.21.15.159:0/1075600433 shutdown_connections 2024-04-04T06:05:48.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:48.998+0000 7f8308c00640 1 -- 172.21.15.159:0/1075600433 wait complete. 2024-04-04T06:05:49.011 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:05:49.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.217+0000 7f9a81c00640 1 Processor -- start 2024-04-04T06:05:49.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.217+0000 7f9a81c00640 1 -- start start 2024-04-04T06:05:49.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.218+0000 7f9a81c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9a7c15e4a0 0x7f9a7c0b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:49.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.218+0000 7f9a81c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9a7c0cf700 0x7f9a7c0b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:49.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.218+0000 7f9a81c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a7c15e880 con 0x7f9a7c0d2310 2024-04-04T06:05:49.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.218+0000 7f9a81c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f9a7c0d34e0 con 0x7f9a7c15e4a0 2024-04-04T06:05:49.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.218+0000 7f9a81c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f9a7c0d28d0 con 0x7f9a7c0cf700 2024-04-04T06:05:49.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.219+0000 7f9a73400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f9a7c0d2310 0x7f9a7c0ceff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58190/0 (socket says 172.21.15.159:58190) 2024-04-04T06:05:49.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.219+0000 7f9a73400640 1 -- 172.21.15.159:0/434321402 learned_addr learned my addr 172.21.15.159:0/434321402 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:05:49.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.219+0000 7f9a7be00640 1 --2- 172.21.15.159:0/434321402 >> v2:172.21.15.159:3300/0 conn(0x7f9a7c15e4a0 0x7f9a7c0b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:49.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.219+0000 7f9a81200640 1 --2- 172.21.15.159:0/434321402 >> v2:172.21.15.178:3300/0 conn(0x7f9a7c0cf700 0x7f9a7c0b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:49.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.219+0000 7f9a7b400640 1 -- 172.21.15.159:0/434321402 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1025354779 0 0) 0x7f9a7c15e880 con 0x7f9a7c0d2310 2024-04-04T06:05:49.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.219+0000 7f9a7b400640 1 -- 172.21.15.159:0/434321402 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a60003520 con 0x7f9a7c0d2310 2024-04-04T06:05:49.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.219+0000 7f9a7be00640 1 -- 172.21.15.159:0/434321402 >> v2:172.21.15.178:3300/0 conn(0x7f9a7c0cf700 msgr2=0x7f9a7c0b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:49.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.219+0000 7f9a7be00640 1 --2- 172.21.15.159:0/434321402 >> v2:172.21.15.178:3300/0 conn(0x7f9a7c0cf700 0x7f9a7c0b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:49.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.219+0000 7f9a7be00640 1 -- 172.21.15.159:0/434321402 >> v1:172.21.15.159:6789/0 conn(0x7f9a7c0d2310 legacy=0x7f9a7c0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:49.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.219+0000 7f9a7be00640 1 -- 172.21.15.159:0/434321402 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7c0d2bb0 con 0x7f9a7c15e4a0 2024-04-04T06:05:49.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.220+0000 7f9a7be00640 1 --2- 172.21.15.159:0/434321402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a7c15e4a0 0x7f9a7c0b9c60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0d20b0 tx=0x7f9a6c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250e2658f3ba5de server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:49.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.221+0000 7f9a7b400640 1 -- 172.21.15.159:0/434321402 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0315a0 con 0x7f9a7c15e4a0 2024-04-04T06:05:49.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.221+0000 7f9a7b400640 1 -- 172.21.15.159:0/434321402 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a6c037070 con 0x7f9a7c15e4a0 2024-04-04T06:05:49.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.221+0000 7f9a7b400640 1 -- 172.21.15.159:0/434321402 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c031940 con 0x7f9a7c15e4a0 2024-04-04T06:05:49.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.221+0000 7f9a81c00640 1 -- 172.21.15.159:0/434321402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a7c15e4a0 msgr2=0x7f9a7c0b9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:49.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.221+0000 7f9a81c00640 1 --2- 172.21.15.159:0/434321402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a7c15e4a0 0x7f9a7c0b9c60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0d20b0 tx=0x7f9a6c02b840 comp rx=0 tx=0).stop 2024-04-04T06:05:49.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.221+0000 7f9a81c00640 1 -- 172.21.15.159:0/434321402 shutdown_connections 2024-04-04T06:05:49.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.221+0000 7f9a81c00640 1 --2- 172.21.15.159:0/434321402 >> v2:172.21.15.178:3300/0 conn(0x7f9a7c0cf700 0x7f9a7c0b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:49.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.221+0000 7f9a81c00640 1 --2- 172.21.15.159:0/434321402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a7c15e4a0 0x7f9a7c0b9c60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:49.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.221+0000 7f9a81c00640 1 -- 172.21.15.159:0/434321402 >> 172.21.15.159:0/434321402 conn(0x7f9a7c0676d0 msgr2=0x7f9a7c067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:49.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.221+0000 7f9a81c00640 1 -- 172.21.15.159:0/434321402 shutdown_connections 2024-04-04T06:05:49.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.221+0000 7f9a81c00640 1 -- 172.21.15.159:0/434321402 wait complete. 2024-04-04T06:05:49.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.223+0000 7f9a81c00640 1 Processor -- start 2024-04-04T06:05:49.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.223+0000 7f9a81c00640 1 -- start start 2024-04-04T06:05:49.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.224+0000 7f9a81c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a7c0cf700 0x7f9a7c0cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:49.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.224+0000 7f9a81c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a7c0d2310 0x7f9a7c0ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:49.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.224+0000 7f9a81c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a7c15e4a0 0x7f9a7c0cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:49.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.224+0000 7f9a81c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c0d3970 con 0x7f9a7c15e4a0 2024-04-04T06:05:49.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.224+0000 7f9a81c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c0d3170 con 0x7f9a7c0d2310 2024-04-04T06:05:49.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.224+0000 7f9a81c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9a7c0ba1a0 con 0x7f9a7c0cf700 2024-04-04T06:05:49.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.224+0000 7f9a81200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a7c15e4a0 0x7f9a7c0cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:49.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.224+0000 7f9a81200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a7c15e4a0 0x7f9a7c0cd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48024/0 (socket says 172.21.15.159:48024) 2024-04-04T06:05:49.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.224+0000 7f9a81200640 1 -- 172.21.15.159:0/72539564 learned_addr learned my addr 172.21.15.159:0/72539564 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:49.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.225+0000 7f9a73400640 1 --2- 172.21.15.159:0/72539564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a7c0d2310 0x7f9a7c0ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:49.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.225+0000 7f9a7be00640 1 --2- 172.21.15.159:0/72539564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a7c0cf700 0x7f9a7c0cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:49.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.225+0000 7f9a81200640 1 -- 172.21.15.159:0/72539564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a7c0cf700 msgr2=0x7f9a7c0cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:49.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.225+0000 7f9a81200640 1 --2- 172.21.15.159:0/72539564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a7c0cf700 0x7f9a7c0cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:49.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.225+0000 7f9a81200640 1 -- 172.21.15.159:0/72539564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a7c0d2310 msgr2=0x7f9a7c0ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:49.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.225+0000 7f9a81200640 1 --2- 172.21.15.159:0/72539564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a7c0d2310 0x7f9a7c0ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:49.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.225+0000 7f9a81200640 1 -- 172.21.15.159:0/72539564 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7c0b8270 con 0x7f9a7c15e4a0 2024-04-04T06:05:49.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.225+0000 7f9a81200640 1 --2- 172.21.15.159:0/72539564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a7c15e4a0 0x7f9a7c0cd3e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9a74007d40 tx=0x7f9a7400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:49.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.225+0000 7f9a78c00640 1 -- 172.21.15.159:0/72539564 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a74014420 con 0x7f9a7c15e4a0 2024-04-04T06:05:49.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.225+0000 7f9a78c00640 1 -- 172.21.15.159:0/72539564 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a74019070 con 0x7f9a7c15e4a0 2024-04-04T06:05:49.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.225+0000 7f9a78c00640 1 -- 172.21.15.159:0/72539564 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a74014820 con 0x7f9a7c15e4a0 2024-04-04T06:05:49.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.225+0000 7f9a81c00640 1 -- 172.21.15.159:0/72539564 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a7c0b8550 con 0x7f9a7c15e4a0 2024-04-04T06:05:49.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.226+0000 7f9a81c00640 1 -- 172.21.15.159:0/72539564 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a7c0b8b10 con 0x7f9a7c15e4a0 2024-04-04T06:05:49.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.227+0000 7f9a81c00640 1 -- 172.21.15.159:0/72539564 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a7c075940 con 0x7f9a7c15e4a0 2024-04-04T06:05:49.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.227+0000 7f9a78c00640 1 -- 172.21.15.159:0/72539564 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a740149c0 con 0x7f9a7c15e4a0 2024-04-04T06:05:49.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.229+0000 7f9a78c00640 1 --2- 172.21.15.159:0/72539564 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9a5806c8f0 0x7f9a5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:49.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.229+0000 7f9a78c00640 1 -- 172.21.15.159:0/72539564 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f9a7408bff0 con 0x7f9a7c15e4a0 2024-04-04T06:05:49.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.229+0000 7f9a7be00640 1 --2- 172.21.15.159:0/72539564 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9a5806c8f0 0x7f9a5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:49.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.230+0000 7f9a7be00640 1 --2- 172.21.15.159:0/72539564 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9a5806c8f0 0x7f9a5806edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c036000 tx=0x7f9a6c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:49.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.234+0000 7f9a78c00640 1 -- 172.21.15.159:0/72539564 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9a7405a780 con 0x7f9a7c15e4a0 2024-04-04T06:05:49.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.478+0000 7f9a81c00640 1 -- 172.21.15.159:0/72539564 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a7c0c80d0 con 0x7f9a7c15e4a0 2024-04-04T06:05:49.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.478+0000 7f9a78c00640 1 -- 172.21.15.159:0/72539564 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3873 (secure 0 0 0) 0x7f9a7405a080 con 0x7f9a7c15e4a0 2024-04-04T06:05:49.473 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:05:49.474 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/789759868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":789759868},{"type":"v1","addr":"172.21.15.178:6836","nonce":789759868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2438288045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2438288045},{"type":"v1","addr":"172.21.15.159:6833","nonce":2438288045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/1705205607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1705205607},{"type":"v1","addr":"172.21.15.159:6835","nonce":1705205607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3362141613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":3362141613},{"type":"v1","addr":"172.21.15.178:6837","nonce":3362141613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-04T06:05:49.474 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 3 2024-04-04T06:05:49.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.481+0000 7f9a81c00640 1 -- 172.21.15.159:0/72539564 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9a5806c8f0 msgr2=0x7f9a5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:49.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.481+0000 7f9a81c00640 1 --2- 172.21.15.159:0/72539564 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9a5806c8f0 0x7f9a5806edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c036000 tx=0x7f9a6c008030 comp rx=0 tx=0).stop 2024-04-04T06:05:49.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.481+0000 7f9a81c00640 1 -- 172.21.15.159:0/72539564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a7c15e4a0 msgr2=0x7f9a7c0cd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:49.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.481+0000 7f9a81c00640 1 --2- 172.21.15.159:0/72539564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a7c15e4a0 0x7f9a7c0cd3e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9a74007d40 tx=0x7f9a7400e4b0 comp rx=0 tx=0).stop 2024-04-04T06:05:49.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.481+0000 7f9a81c00640 1 -- 172.21.15.159:0/72539564 shutdown_connections 2024-04-04T06:05:49.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.481+0000 7f9a81c00640 1 --2- 172.21.15.159:0/72539564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a7c15e4a0 0x7f9a7c0cd3e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:49.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.481+0000 7f9a81c00640 1 --2- 172.21.15.159:0/72539564 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9a5806c8f0 0x7f9a5806edb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:49.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.481+0000 7f9a81c00640 1 --2- 172.21.15.159:0/72539564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a7c0d2310 0x7f9a7c0ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:49.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.481+0000 7f9a81c00640 1 --2- 172.21.15.159:0/72539564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9a7c0cf700 0x7f9a7c0cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:49.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.481+0000 7f9a81c00640 1 -- 172.21.15.159:0/72539564 >> 172.21.15.159:0/72539564 conn(0x7f9a7c0676d0 msgr2=0x7f9a7c0ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:49.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.481+0000 7f9a81c00640 1 -- 172.21.15.159:0/72539564 shutdown_connections 2024-04-04T06:05:49.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.481+0000 7f9a81c00640 1 -- 172.21.15.159:0/72539564 wait complete. 2024-04-04T06:05:49.495 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:05:49.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.706+0000 7f5eabe00640 1 Processor -- start 2024-04-04T06:05:49.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.706+0000 7f5eabe00640 1 -- start start 2024-04-04T06:05:49.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.708+0000 7f5eabe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5eac0d2310 0x7f5eac0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:49.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.708+0000 7f5eabe00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5eac0cf530 0x7f5eac0b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:49.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.708+0000 7f5eabe00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eac15e880 con 0x7f5eac15e4a0 2024-04-04T06:05:49.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.708+0000 7f5eabe00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f5eac0cf9a0 con 0x7f5eac0d2310 2024-04-04T06:05:49.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.708+0000 7f5eabe00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f5eac0d34e0 con 0x7f5eac0cf530 2024-04-04T06:05:49.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.708+0000 7f5ea2000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5eac0d2310 0x7f5eac0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:49.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.708+0000 7f5ea2000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5eac0d2310 0x7f5eac0ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48040/0 (socket says 172.21.15.159:48040) 2024-04-04T06:05:49.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.708+0000 7f5ea2000640 1 -- 172.21.15.159:0/644747760 learned_addr learned my addr 172.21.15.159:0/644747760 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:49.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.709+0000 7f5eab400640 1 --2- 172.21.15.159:0/644747760 >> v2:172.21.15.178:3300/0 conn(0x7f5eac0cf530 0x7f5eac0b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:49.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.709+0000 7f5ea2000640 1 -- 172.21.15.159:0/644747760 >> v2:172.21.15.178:3300/0 conn(0x7f5eac0cf530 msgr2=0x7f5eac0b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:49.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.709+0000 7f5ea2000640 1 --2- 172.21.15.159:0/644747760 >> v2:172.21.15.178:3300/0 conn(0x7f5eac0cf530 0x7f5eac0b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:49.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.709+0000 7f5ea2000640 1 -- 172.21.15.159:0/644747760 >> v1:172.21.15.159:6789/0 conn(0x7f5eac15e4a0 legacy=0x7f5eac0b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:49.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.709+0000 7f5ea2000640 1 -- 172.21.15.159:0/644747760 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eac0d28d0 con 0x7f5eac0d2310 2024-04-04T06:05:49.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.709+0000 7f5eab400640 1 --2- 172.21.15.159:0/644747760 >> v2:172.21.15.178:3300/0 conn(0x7f5eac0cf530 0x7f5eac0b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:05:49.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.710+0000 7f5ea2000640 1 --2- 172.21.15.159:0/644747760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5eac0d2310 0x7f5eac0ceff0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0d3660 tx=0x7f5e9402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9917a7c35bb1f197 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:49.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.710+0000 7f5eaa000640 1 -- 172.21.15.159:0/644747760 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e94035020 con 0x7f5eac0d2310 2024-04-04T06:05:49.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.710+0000 7f5eaa000640 1 -- 172.21.15.159:0/644747760 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e9402d040 con 0x7f5eac0d2310 2024-04-04T06:05:49.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.710+0000 7f5eaa000640 1 -- 172.21.15.159:0/644747760 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e94031a80 con 0x7f5eac0d2310 2024-04-04T06:05:49.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.711+0000 7f5eabe00640 1 -- 172.21.15.159:0/644747760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5eac0d2310 msgr2=0x7f5eac0ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:49.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.711+0000 7f5eabe00640 1 --2- 172.21.15.159:0/644747760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5eac0d2310 0x7f5eac0ceff0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0d3660 tx=0x7f5e9402b900 comp rx=0 tx=0).stop 2024-04-04T06:05:49.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.711+0000 7f5eabe00640 1 -- 172.21.15.159:0/644747760 shutdown_connections 2024-04-04T06:05:49.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.711+0000 7f5eabe00640 1 --2- 172.21.15.159:0/644747760 >> v2:172.21.15.178:3300/0 conn(0x7f5eac0cf530 0x7f5eac0b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:49.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.711+0000 7f5eabe00640 1 --2- 172.21.15.159:0/644747760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5eac0d2310 0x7f5eac0ceff0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:49.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.711+0000 7f5eabe00640 1 -- 172.21.15.159:0/644747760 >> 172.21.15.159:0/644747760 conn(0x7f5eac0676d0 msgr2=0x7f5eac067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:49.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.711+0000 7f5eabe00640 1 -- 172.21.15.159:0/644747760 shutdown_connections 2024-04-04T06:05:49.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.711+0000 7f5eabe00640 1 -- 172.21.15.159:0/644747760 wait complete. 2024-04-04T06:05:49.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.713+0000 7f5eabe00640 1 Processor -- start 2024-04-04T06:05:49.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.713+0000 7f5eabe00640 1 -- start start 2024-04-04T06:05:49.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.714+0000 7f5eabe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5eac0cf530 0x7f5eac153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:49.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.714+0000 7f5eabe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5eac0d2310 0x7f5eac157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:49.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.714+0000 7f5eabe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5eac15e4a0 0x7f5eac0cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:49.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.714+0000 7f5eabe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac0d3170 con 0x7f5eac0cf530 2024-04-04T06:05:49.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.714+0000 7f5eabe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac0d2e90 con 0x7f5eac0d2310 2024-04-04T06:05:49.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.714+0000 7f5eabe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5eac0d3970 con 0x7f5eac15e4a0 2024-04-04T06:05:49.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.714+0000 7f5eaaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5eac0cf530 0x7f5eac153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:49.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.714+0000 7f5eaaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5eac0cf530 0x7f5eac153e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48048/0 (socket says 172.21.15.159:48048) 2024-04-04T06:05:49.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.714+0000 7f5eaaa00640 1 -- 172.21.15.159:0/3570213026 learned_addr learned my addr 172.21.15.159:0/3570213026 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:49.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.714+0000 7f5ea2000640 1 --2- 172.21.15.159:0/3570213026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5eac0d2310 0x7f5eac157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:49.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.714+0000 7f5eaaa00640 1 -- 172.21.15.159:0/3570213026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5eac15e4a0 msgr2=0x7f5eac0cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:49.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.714+0000 7f5eaaa00640 1 --2- 172.21.15.159:0/3570213026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5eac15e4a0 0x7f5eac0cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:49.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.715+0000 7f5eaaa00640 1 -- 172.21.15.159:0/3570213026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5eac0d2310 msgr2=0x7f5eac157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:49.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.715+0000 7f5eaaa00640 1 --2- 172.21.15.159:0/3570213026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5eac0d2310 0x7f5eac157900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:49.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.715+0000 7f5eaaa00640 1 -- 172.21.15.159:0/3570213026 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eac15f000 con 0x7f5eac0cf530 2024-04-04T06:05:49.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.715+0000 7f5eaaa00640 1 --2- 172.21.15.159:0/3570213026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5eac0cf530 0x7f5eac153e00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00c3f0 tx=0x7f5e9c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:49.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.715+0000 7f5ea3400640 1 -- 172.21.15.159:0/3570213026 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c0052d0 con 0x7f5eac0cf530 2024-04-04T06:05:49.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.715+0000 7f5ea3400640 1 -- 172.21.15.159:0/3570213026 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e9c005490 con 0x7f5eac0cf530 2024-04-04T06:05:49.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.715+0000 7f5eabe00640 1 -- 172.21.15.159:0/3570213026 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eac15f2e0 con 0x7f5eac0cf530 2024-04-04T06:05:49.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.715+0000 7f5ea3400640 1 -- 172.21.15.159:0/3570213026 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c007260 con 0x7f5eac0cf530 2024-04-04T06:05:49.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.715+0000 7f5eabe00640 1 -- 172.21.15.159:0/3570213026 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eac15f8a0 con 0x7f5eac0cf530 2024-04-04T06:05:49.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.717+0000 7f5eabe00640 1 -- 172.21.15.159:0/3570213026 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eac075940 con 0x7f5eac0cf530 2024-04-04T06:05:49.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.717+0000 7f5ea3400640 1 -- 172.21.15.159:0/3570213026 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e9c007400 con 0x7f5eac0cf530 2024-04-04T06:05:49.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.718+0000 7f5ea3400640 1 --2- 172.21.15.159:0/3570213026 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5e8806c8f0 0x7f5e8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:49.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.718+0000 7f5ea3400640 1 -- 172.21.15.159:0/3570213026 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f5e9c08ac30 con 0x7f5eac0cf530 2024-04-04T06:05:49.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.719+0000 7f5ea2000640 1 --2- 172.21.15.159:0/3570213026 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5e8806c8f0 0x7f5e8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:49.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.720+0000 7f5ea2000640 1 --2- 172.21.15.159:0/3570213026 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5e8806c8f0 0x7f5e8806edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5e94036000 tx=0x7f5e94008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:49.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.723+0000 7f5ea3400640 1 -- 172.21.15.159:0/3570213026 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5e9c059440 con 0x7f5eac0cf530 2024-04-04T06:05:49.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.922+0000 7f5eabe00640 1 -- 172.21.15.159:0/3570213026 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5eac0b6aa0 con 0x7f5eac0cf530 2024-04-04T06:05:49.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.924+0000 7f5ea3400640 1 -- 172.21.15.159:0/3570213026 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12896 (secure 0 0 0) 0x7f5e9c058d40 con 0x7f5eac0cf530 2024-04-04T06:05:49.919 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:05:49.919 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":17,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:05:43.073157+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:05:49.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.926+0000 7f5eabe00640 1 -- 172.21.15.159:0/3570213026 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5e8806c8f0 msgr2=0x7f5e8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:49.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.926+0000 7f5eabe00640 1 --2- 172.21.15.159:0/3570213026 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5e8806c8f0 0x7f5e8806edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5e94036000 tx=0x7f5e94008030 comp rx=0 tx=0).stop 2024-04-04T06:05:49.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.926+0000 7f5eabe00640 1 -- 172.21.15.159:0/3570213026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5eac0cf530 msgr2=0x7f5eac153e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:49.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.926+0000 7f5eabe00640 1 --2- 172.21.15.159:0/3570213026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5eac0cf530 0x7f5eac153e00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00c3f0 tx=0x7f5e9c00c8b0 comp rx=0 tx=0).stop 2024-04-04T06:05:49.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.927+0000 7f5eabe00640 1 -- 172.21.15.159:0/3570213026 shutdown_connections 2024-04-04T06:05:49.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.927+0000 7f5eabe00640 1 --2- 172.21.15.159:0/3570213026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5eac15e4a0 0x7f5eac0cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:49.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.927+0000 7f5eabe00640 1 --2- 172.21.15.159:0/3570213026 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5e8806c8f0 0x7f5e8806edb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:49.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.927+0000 7f5eabe00640 1 --2- 172.21.15.159:0/3570213026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5eac0d2310 0x7f5eac157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:49.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.927+0000 7f5eabe00640 1 --2- 172.21.15.159:0/3570213026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5eac0cf530 0x7f5eac153e00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:49.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.927+0000 7f5eabe00640 1 -- 172.21.15.159:0/3570213026 >> 172.21.15.159:0/3570213026 conn(0x7f5eac0676d0 msgr2=0x7f5eac0d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:49.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.927+0000 7f5eabe00640 1 -- 172.21.15.159:0/3570213026 shutdown_connections 2024-04-04T06:05:49.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:49.927+0000 7f5eabe00640 1 -- 172.21.15.159:0/3570213026 wait complete. 2024-04-04T06:05:49.941 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:05:49.942 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:05:50.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.157+0000 7f0c25600640 1 Processor -- start 2024-04-04T06:05:50.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.158+0000 7f0c25600640 1 -- start start 2024-04-04T06:05:50.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.159+0000 7f0c25600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0c2015e6d0 0x7f0c200d21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:50.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.159+0000 7f0c25600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c200d0cb0 0x7f0c200d4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:50.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.159+0000 7f0c25600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c200ceef0 con 0x7f0c200d4590 2024-04-04T06:05:50.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.159+0000 7f0c25600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f0c2015eab0 con 0x7f0c2015e6d0 2024-04-04T06:05:50.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.159+0000 7f0c25600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0c200d1220 con 0x7f0c200d0cb0 2024-04-04T06:05:50.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.159+0000 7f0c1ea00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0c2015e6d0 0x7f0c200d21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:50.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.159+0000 7f0c1f400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f0c200d4590 0x7f0c201621c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58222/0 (socket says 172.21.15.159:58222) 2024-04-04T06:05:50.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.159+0000 7f0c1f400640 1 -- 172.21.15.159:0/4122071407 learned_addr learned my addr 172.21.15.159:0/4122071407 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:05:50.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.160+0000 7f0c1e000640 1 --2- 172.21.15.159:0/4122071407 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c200d0cb0 0x7f0c200d4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:50.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.160+0000 7f0c1ea00640 1 -- 172.21.15.159:0/4122071407 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c200d0cb0 msgr2=0x7f0c200d4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:50.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.160+0000 7f0c1ea00640 1 --2- 172.21.15.159:0/4122071407 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c200d0cb0 0x7f0c200d4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.160+0000 7f0c1ea00640 1 -- 172.21.15.159:0/4122071407 >> v1:172.21.15.159:6789/0 conn(0x7f0c200d4590 legacy=0x7f0c201621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:50.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.160+0000 7f0c1ea00640 1 -- 172.21.15.159:0/4122071407 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c200d1500 con 0x7f0c2015e6d0 2024-04-04T06:05:50.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.161+0000 7f0c1ea00640 1 --2- 172.21.15.159:0/4122071407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c2015e6d0 0x7f0c200d21e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0c100097a0 tx=0x7f0c1002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e648155376401f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:50.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.161+0000 7f0c1d600640 1 -- 172.21.15.159:0/4122071407 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c10035020 con 0x7f0c2015e6d0 2024-04-04T06:05:50.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.161+0000 7f0c1d600640 1 -- 172.21.15.159:0/4122071407 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c10031810 con 0x7f0c2015e6d0 2024-04-04T06:05:50.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.161+0000 7f0c1d600640 1 -- 172.21.15.159:0/4122071407 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c10031be0 con 0x7f0c2015e6d0 2024-04-04T06:05:50.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.162+0000 7f0c25600640 1 -- 172.21.15.159:0/4122071407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c2015e6d0 msgr2=0x7f0c200d21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:50.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.162+0000 7f0c25600640 1 --2- 172.21.15.159:0/4122071407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c2015e6d0 0x7f0c200d21e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0c100097a0 tx=0x7f0c1002b8b0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.162+0000 7f0c25600640 1 -- 172.21.15.159:0/4122071407 shutdown_connections 2024-04-04T06:05:50.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.162+0000 7f0c25600640 1 --2- 172.21.15.159:0/4122071407 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c200d0cb0 0x7f0c200d4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.162+0000 7f0c25600640 1 --2- 172.21.15.159:0/4122071407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c2015e6d0 0x7f0c200d21e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.162+0000 7f0c25600640 1 -- 172.21.15.159:0/4122071407 >> 172.21.15.159:0/4122071407 conn(0x7f0c2009d7b0 msgr2=0x7f0c2008bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:50.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.162+0000 7f0c25600640 1 -- 172.21.15.159:0/4122071407 shutdown_connections 2024-04-04T06:05:50.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.162+0000 7f0c25600640 1 -- 172.21.15.159:0/4122071407 wait complete. 2024-04-04T06:05:50.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.164+0000 7f0c25600640 1 Processor -- start 2024-04-04T06:05:50.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.164+0000 7f0c25600640 1 -- start start 2024-04-04T06:05:50.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.165+0000 7f0c25600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c200d0cb0 0x7f0c2015ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:50.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.165+0000 7f0c25600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c200d4590 0x7f0c200b9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:50.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.165+0000 7f0c25600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c2015e6d0 0x7f0c201e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:50.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.165+0000 7f0c25600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0c200d4a20 con 0x7f0c200d0cb0 2024-04-04T06:05:50.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.165+0000 7f0c25600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0c200d1ac0 con 0x7f0c2015e6d0 2024-04-04T06:05:50.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.165+0000 7f0c25600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0c200d2720 con 0x7f0c200d4590 2024-04-04T06:05:50.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.165+0000 7f0c1ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c200d0cb0 0x7f0c2015ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:50.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.165+0000 7f0c1ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c200d0cb0 0x7f0c2015ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48064/0 (socket says 172.21.15.159:48064) 2024-04-04T06:05:50.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.165+0000 7f0c1ea00640 1 -- 172.21.15.159:0/1089813138 learned_addr learned my addr 172.21.15.159:0/1089813138 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:50.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.165+0000 7f0c1e000640 1 --2- 172.21.15.159:0/1089813138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c200d4590 0x7f0c200b9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:50.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.165+0000 7f0c1f400640 1 --2- 172.21.15.159:0/1089813138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c2015e6d0 0x7f0c201e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:50.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.165+0000 7f0c1ea00640 1 -- 172.21.15.159:0/1089813138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c200d4590 msgr2=0x7f0c200b9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:50.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.166+0000 7f0c1ea00640 1 --2- 172.21.15.159:0/1089813138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c200d4590 0x7f0c200b9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.166+0000 7f0c1ea00640 1 -- 172.21.15.159:0/1089813138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c2015e6d0 msgr2=0x7f0c201e8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:50.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.166+0000 7f0c1ea00640 1 --2- 172.21.15.159:0/1089813138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c2015e6d0 0x7f0c201e8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.166+0000 7f0c1ea00640 1 -- 172.21.15.159:0/1089813138 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c201e7980 con 0x7f0c200d0cb0 2024-04-04T06:05:50.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.166+0000 7f0c1ea00640 1 --2- 172.21.15.159:0/1089813138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c200d0cb0 0x7f0c2015ace0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0c1002bec0 tx=0x7f0c100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:50.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.166+0000 7f0c16a00640 1 -- 172.21.15.159:0/1089813138 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c10041070 con 0x7f0c200d0cb0 2024-04-04T06:05:50.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.166+0000 7f0c16a00640 1 -- 172.21.15.159:0/1089813138 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1003ad00 con 0x7f0c200d0cb0 2024-04-04T06:05:50.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.166+0000 7f0c16a00640 1 -- 172.21.15.159:0/1089813138 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c10035040 con 0x7f0c200d0cb0 2024-04-04T06:05:50.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.166+0000 7f0c25600640 1 -- 172.21.15.159:0/1089813138 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c201e7c20 con 0x7f0c200d0cb0 2024-04-04T06:05:50.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.166+0000 7f0c25600640 1 -- 172.21.15.159:0/1089813138 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c200b73d0 con 0x7f0c200d0cb0 2024-04-04T06:05:50.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.167+0000 7f0c25600640 1 -- 172.21.15.159:0/1089813138 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c200685c0 con 0x7f0c200d0cb0 2024-04-04T06:05:50.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.168+0000 7f0c16a00640 1 -- 172.21.15.159:0/1089813138 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c10038020 con 0x7f0c200d0cb0 2024-04-04T06:05:50.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.170+0000 7f0c16a00640 1 --2- 172.21.15.159:0/1089813138 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0bfc06c8f0 0x7f0bfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:50.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.170+0000 7f0c16a00640 1 -- 172.21.15.159:0/1089813138 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f0c100ad4e0 con 0x7f0c200d0cb0 2024-04-04T06:05:50.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.170+0000 7f0c1e000640 1 --2- 172.21.15.159:0/1089813138 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0bfc06c8f0 0x7f0bfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:50.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.171+0000 7f0c1e000640 1 --2- 172.21.15.159:0/1089813138 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0bfc06c8f0 0x7f0bfc06edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0c20160390 tx=0x7f0c08008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:50.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.175+0000 7f0c16a00640 1 -- 172.21.15.159:0/1089813138 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0c1007bc50 con 0x7f0c200d0cb0 2024-04-04T06:05:50.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.379+0000 7f0c25600640 1 -- 172.21.15.159:0/1089813138 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0c2006c670 con 0x7f0c200d0cb0 2024-04-04T06:05:50.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.380+0000 7f0c16a00640 1 -- 172.21.15.159:0/1089813138 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f0c1007b550 con 0x7f0c200d0cb0 2024-04-04T06:05:50.374 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:05:50.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.382+0000 7f0c25600640 1 -- 172.21.15.159:0/1089813138 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0bfc06c8f0 msgr2=0x7f0bfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:50.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.382+0000 7f0c25600640 1 --2- 172.21.15.159:0/1089813138 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0bfc06c8f0 0x7f0bfc06edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0c20160390 tx=0x7f0c08008040 comp rx=0 tx=0).stop 2024-04-04T06:05:50.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.382+0000 7f0c25600640 1 -- 172.21.15.159:0/1089813138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c200d0cb0 msgr2=0x7f0c2015ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:50.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.382+0000 7f0c25600640 1 --2- 172.21.15.159:0/1089813138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c200d0cb0 0x7f0c2015ace0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0c1002bec0 tx=0x7f0c100046c0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.382+0000 7f0c25600640 1 -- 172.21.15.159:0/1089813138 shutdown_connections 2024-04-04T06:05:50.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.382+0000 7f0c25600640 1 --2- 172.21.15.159:0/1089813138 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0bfc06c8f0 0x7f0bfc06edb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.382+0000 7f0c25600640 1 --2- 172.21.15.159:0/1089813138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c2015e6d0 0x7f0c201e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.382+0000 7f0c25600640 1 --2- 172.21.15.159:0/1089813138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c200d4590 0x7f0c200b9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.382+0000 7f0c25600640 1 --2- 172.21.15.159:0/1089813138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c200d0cb0 0x7f0c2015ace0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.382+0000 7f0c25600640 1 -- 172.21.15.159:0/1089813138 >> 172.21.15.159:0/1089813138 conn(0x7f0c2009d7b0 msgr2=0x7f0c2006d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:50.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.383+0000 7f0c25600640 1 -- 172.21.15.159:0/1089813138 shutdown_connections 2024-04-04T06:05:50.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.383+0000 7f0c25600640 1 -- 172.21.15.159:0/1089813138 wait complete. 2024-04-04T06:05:50.396 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-04T06:05:50.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.601+0000 7f7eba400640 1 Processor -- start 2024-04-04T06:05:50.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.602+0000 7f7eba400640 1 -- start start 2024-04-04T06:05:50.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.603+0000 7f7eba400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7eb4153580 0x7f7eb40d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:50.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.603+0000 7f7eba400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7eb40d07c0 0x7f7eb40aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:50.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.603+0000 7f7eba400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eb40afab0 con 0x7f7eb40af2e0 2024-04-04T06:05:50.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.603+0000 7f7eba400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7eb40d0c00 con 0x7f7eb40d07c0 2024-04-04T06:05:50.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.603+0000 7f7eba400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb415cc80 con 0x7f7eb4153580 2024-04-04T06:05:50.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.603+0000 7f7eb3e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f7eb40af2e0 0x7f7eb406d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58232/0 (socket says 172.21.15.159:58232) 2024-04-04T06:05:50.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.603+0000 7f7eb3e00640 1 -- 172.21.15.159:0/912184021 learned_addr learned my addr 172.21.15.159:0/912184021 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:05:50.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.604+0000 7f7eb2a00640 1 -- 172.21.15.159:0/912184021 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1637145145 0 0) 0x7f7eb40afab0 con 0x7f7eb40af2e0 2024-04-04T06:05:50.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.604+0000 7f7eb2a00640 1 -- 172.21.15.159:0/912184021 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e98003520 con 0x7f7eb40af2e0 2024-04-04T06:05:50.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.604+0000 7f7eb3400640 1 --2- 172.21.15.159:0/912184021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7eb4153580 0x7f7eb40d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:50.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.604+0000 7f7eabe00640 1 --2- 172.21.15.159:0/912184021 >> v2:172.21.15.178:3300/0 conn(0x7f7eb40d07c0 0x7f7eb40aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:50.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.604+0000 7f7eb2a00640 1 -- 172.21.15.159:0/912184021 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3748047079 0 0) 0x7f7e98003520 con 0x7f7eb40af2e0 2024-04-04T06:05:50.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.604+0000 7f7eb2a00640 1 -- 172.21.15.159:0/912184021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7eb4153580 msgr2=0x7f7eb40d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:50.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.604+0000 7f7eb2a00640 1 --2- 172.21.15.159:0/912184021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7eb4153580 0x7f7eb40d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.604+0000 7f7eb2a00640 1 -- 172.21.15.159:0/912184021 >> v2:172.21.15.178:3300/0 conn(0x7f7eb40d07c0 msgr2=0x7f7eb40aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:50.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.604+0000 7f7eb2a00640 1 --2- 172.21.15.159:0/912184021 >> v2:172.21.15.178:3300/0 conn(0x7f7eb40d07c0 0x7f7eb40aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.604+0000 7f7eb2a00640 1 -- 172.21.15.159:0/912184021 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb415da00 con 0x7f7eb40af2e0 2024-04-04T06:05:50.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.604+0000 7f7eb2a00640 1 -- 172.21.15.159:0/912184021 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f7eac0019f0 con 0x7f7eb40af2e0 2024-04-04T06:05:50.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.604+0000 7f7eb2a00640 1 -- 172.21.15.159:0/912184021 >> v1:172.21.15.159:6789/0 conn(0x7f7eb40af2e0 legacy=0x7f7eb406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:50.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.604+0000 7f7eb2a00640 1 --2- 172.21.15.159:0/912184021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e980039b0 0x7f7e98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:50.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.604+0000 7f7eb2a00640 1 --2- 172.21.15.159:0/912184021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e98004800 0x7f7e98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:50.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.604+0000 7f7eb2a00640 1 --2- 172.21.15.159:0/912184021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e980051c0 0x7f7e9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:50.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.604+0000 7f7eb2a00640 1 -- 172.21.15.159:0/912184021 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb40d0c00 con 0x7f7e98004800 2024-04-04T06:05:50.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.604+0000 7f7eb2a00640 1 -- 172.21.15.159:0/912184021 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb415cc80 con 0x7f7e980051c0 2024-04-04T06:05:50.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.604+0000 7f7eb2a00640 1 -- 172.21.15.159:0/912184021 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7e9800c0e0 con 0x7f7e980039b0 2024-04-04T06:05:50.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.605+0000 7f7eb3400640 1 --2- 172.21.15.159:0/912184021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7eb4153580 0x7f7eb40d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:05:50.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.605+0000 7f7eabe00640 1 --2- 172.21.15.159:0/912184021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e98004800 0x7f7e98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:50.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.605+0000 7f7eb3e00640 1 --2- 172.21.15.159:0/912184021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e980051c0 0x7f7e9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:50.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.605+0000 7f7eb3400640 1 --2- 172.21.15.159:0/912184021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e980039b0 0x7f7e98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:50.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.605+0000 7f7eabe00640 1 -- 172.21.15.159:0/912184021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e980039b0 msgr2=0x7f7e98003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:50.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.605+0000 7f7eabe00640 1 --2- 172.21.15.159:0/912184021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e980039b0 0x7f7e98003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.605+0000 7f7eabe00640 1 -- 172.21.15.159:0/912184021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e980051c0 msgr2=0x7f7e9800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:50.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.605+0000 7f7eabe00640 1 --2- 172.21.15.159:0/912184021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e980051c0 0x7f7e9800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.605+0000 7f7eabe00640 1 -- 172.21.15.159:0/912184021 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e98003520 con 0x7f7e98004800 2024-04-04T06:05:50.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.605+0000 7f7eb3400640 1 --2- 172.21.15.159:0/912184021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e980039b0 0x7f7e98003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:05:50.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.606+0000 7f7eabe00640 1 --2- 172.21.15.159:0/912184021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e98004800 0x7f7e98004c80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00ba30 tx=0x7f7e98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be371e99b09a76d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:50.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.607+0000 7f7eb2a00640 1 -- 172.21.15.159:0/912184021 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c036020 con 0x7f7e98004800 2024-04-04T06:05:50.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.607+0000 7f7eb2a00640 1 -- 172.21.15.159:0/912184021 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9c032680 con 0x7f7e98004800 2024-04-04T06:05:50.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.607+0000 7f7eb2a00640 1 -- 172.21.15.159:0/912184021 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c00b600 con 0x7f7e98004800 2024-04-04T06:05:50.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.607+0000 7f7eba400640 1 -- 172.21.15.159:0/912184021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e98004800 msgr2=0x7f7e98004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:50.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.607+0000 7f7eba400640 1 --2- 172.21.15.159:0/912184021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e98004800 0x7f7e98004c80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00ba30 tx=0x7f7e98005830 comp rx=0 tx=0).stop 2024-04-04T06:05:50.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.607+0000 7f7eb3e00640 1 -- 172.21.15.159:0/912184021 reap_dead start 2024-04-04T06:05:50.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.607+0000 7f7eba400640 1 -- 172.21.15.159:0/912184021 shutdown_connections 2024-04-04T06:05:50.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.607+0000 7f7eba400640 1 -- 172.21.15.159:0/912184021 >> 172.21.15.159:0/912184021 conn(0x7f7eb406b750 msgr2=0x7f7eb406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:50.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.607+0000 7f7eba400640 1 -- 172.21.15.159:0/912184021 shutdown_connections 2024-04-04T06:05:50.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.607+0000 7f7eba400640 1 -- 172.21.15.159:0/912184021 wait complete. 2024-04-04T06:05:50.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.609+0000 7f7eba400640 1 Processor -- start 2024-04-04T06:05:50.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.609+0000 7f7eba400640 1 -- start start 2024-04-04T06:05:50.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.610+0000 7f7eba400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e98004800 0x7f7eb414fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:50.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.610+0000 7f7eba400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7eb4159dc0 0x7f7eb414e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:50.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.610+0000 7f7eba400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7eb40b1010 0x7f7eb40b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:50.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.610+0000 7f7eba400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb415d4e0 con 0x7f7e98004800 2024-04-04T06:05:50.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.610+0000 7f7eba400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb415d200 con 0x7f7eb40b1010 2024-04-04T06:05:50.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.610+0000 7f7eba400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb4153960 con 0x7f7eb4159dc0 2024-04-04T06:05:50.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.610+0000 7f7eb3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e98004800 0x7f7eb414fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:50.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.610+0000 7f7eb3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e98004800 0x7f7eb414fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48090/0 (socket says 172.21.15.159:48090) 2024-04-04T06:05:50.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.610+0000 7f7eb3400640 1 -- 172.21.15.159:0/667522412 learned_addr learned my addr 172.21.15.159:0/667522412 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:50.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.611+0000 7f7eabe00640 1 --2- 172.21.15.159:0/667522412 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7eb4159dc0 0x7f7eb414e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:50.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.611+0000 7f7eb3400640 1 -- 172.21.15.159:0/667522412 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7eb4159dc0 msgr2=0x7f7eb414e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:50.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.611+0000 7f7eb3400640 1 --2- 172.21.15.159:0/667522412 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7eb4159dc0 0x7f7eb414e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.611+0000 7f7eb3400640 1 -- 172.21.15.159:0/667522412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7eb40b1010 msgr2=0x7f7eb40b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:50.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.611+0000 7f7eb3400640 1 --2- 172.21.15.159:0/667522412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7eb40b1010 0x7f7eb40b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.611+0000 7f7eb3400640 1 -- 172.21.15.159:0/667522412 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb4150520 con 0x7f7e98004800 2024-04-04T06:05:50.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.611+0000 7f7eb3400640 1 --2- 172.21.15.159:0/667522412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e98004800 0x7f7eb414fdb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009fd0 tx=0x7f7ea400e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:50.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.611+0000 7f7eab400640 1 -- 172.21.15.159:0/667522412 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4017070 con 0x7f7e98004800 2024-04-04T06:05:50.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.612+0000 7f7eab400640 1 -- 172.21.15.159:0/667522412 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea4013610 con 0x7f7e98004800 2024-04-04T06:05:50.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.612+0000 7f7eba400640 1 -- 172.21.15.159:0/667522412 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb4150800 con 0x7f7e98004800 2024-04-04T06:05:50.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.612+0000 7f7eab400640 1 -- 172.21.15.159:0/667522412 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea400f040 con 0x7f7e98004800 2024-04-04T06:05:50.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.612+0000 7f7eba400640 1 -- 172.21.15.159:0/667522412 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb4150dc0 con 0x7f7e98004800 2024-04-04T06:05:50.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.613+0000 7f7eba400640 1 -- 172.21.15.159:0/667522412 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb40b19d0 con 0x7f7e98004800 2024-04-04T06:05:50.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.614+0000 7f7eab400640 1 -- 172.21.15.159:0/667522412 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ea401a070 con 0x7f7e98004800 2024-04-04T06:05:50.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.615+0000 7f7eab400640 1 --2- 172.21.15.159:0/667522412 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e9006c8f0 0x7f7e9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:50.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.615+0000 7f7eab400640 1 -- 172.21.15.159:0/667522412 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f7ea408b260 con 0x7f7e98004800 2024-04-04T06:05:50.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.616+0000 7f7eabe00640 1 --2- 172.21.15.159:0/667522412 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e9006c8f0 0x7f7e9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:50.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.617+0000 7f7eabe00640 1 --2- 172.21.15.159:0/667522412 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e9006c8f0 0x7f7e9006edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00ba00 tx=0x7f7e9c030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:50.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.620+0000 7f7eab400640 1 -- 172.21.15.159:0/667522412 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7ea40599e0 con 0x7f7e98004800 2024-04-04T06:05:50.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.823+0000 7f7eba400640 1 -- 172.21.15.159:0/667522412 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7eb415b950 con 0x7f7e98004800 2024-04-04T06:05:50.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.824+0000 7f7eab400640 1 -- 172.21.15.159:0/667522412 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f7eb415b950 con 0x7f7e98004800 2024-04-04T06:05:50.819 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:05:50.819 INFO:teuthology.orchestra.run.smithi159.stdout:[ 2024-04-04T06:05:50.819 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:05:50.819 INFO:teuthology.orchestra.run.smithi159.stdout: "poolnum": 1, 2024-04-04T06:05:50.819 INFO:teuthology.orchestra.run.smithi159.stdout: "poolname": ".mgr" 2024-04-04T06:05:50.819 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:05:50.819 INFO:teuthology.orchestra.run.smithi159.stdout:] 2024-04-04T06:05:50.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.826+0000 7f7eba400640 1 -- 172.21.15.159:0/667522412 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e9006c8f0 msgr2=0x7f7e9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:50.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.826+0000 7f7eba400640 1 --2- 172.21.15.159:0/667522412 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e9006c8f0 0x7f7e9006edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00ba00 tx=0x7f7e9c030000 comp rx=0 tx=0).stop 2024-04-04T06:05:50.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.826+0000 7f7eba400640 1 -- 172.21.15.159:0/667522412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e98004800 msgr2=0x7f7eb414fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:50.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.826+0000 7f7eba400640 1 --2- 172.21.15.159:0/667522412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e98004800 0x7f7eb414fdb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009fd0 tx=0x7f7ea400e9f0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.827+0000 7f7eba400640 1 -- 172.21.15.159:0/667522412 shutdown_connections 2024-04-04T06:05:50.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.827+0000 7f7eba400640 1 --2- 172.21.15.159:0/667522412 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e9006c8f0 0x7f7e9006edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.827+0000 7f7eba400640 1 --2- 172.21.15.159:0/667522412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7eb40b1010 0x7f7eb40b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.827+0000 7f7eba400640 1 --2- 172.21.15.159:0/667522412 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7eb4159dc0 0x7f7eb414e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.827+0000 7f7eba400640 1 --2- 172.21.15.159:0/667522412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e98004800 0x7f7eb414fdb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:50.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.827+0000 7f7eba400640 1 -- 172.21.15.159:0/667522412 >> 172.21.15.159:0/667522412 conn(0x7f7eb406b750 msgr2=0x7f7eb406d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:50.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.827+0000 7f7eba400640 1 -- 172.21.15.159:0/667522412 shutdown_connections 2024-04-04T06:05:50.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:50.827+0000 7f7eba400640 1 -- 172.21.15.159:0/667522412 wait complete. 2024-04-04T06:05:50.840 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-04T06:05:50.840 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-04T06:05:51.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.064+0000 7fda40a00640 1 Processor -- start 2024-04-04T06:05:51.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.064+0000 7fda40a00640 1 -- start start 2024-04-04T06:05:51.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.065+0000 7fda40a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fda3c09dcf0 0x7fda3c1453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:51.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.065+0000 7fda40a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda3c071710 0x7fda3c071b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:51.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.065+0000 7fda40a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda3c15cb90 con 0x7fda3c0acf10 2024-04-04T06:05:51.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.065+0000 7fda40a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fda3c145f60 con 0x7fda3c09dcf0 2024-04-04T06:05:51.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.065+0000 7fda40a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fda3c15c950 con 0x7fda3c071710 2024-04-04T06:05:51.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.066+0000 7fda39e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fda3c0acf10 0x7fda3c0ad2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58236/0 (socket says 172.21.15.159:58236) 2024-04-04T06:05:51.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.066+0000 7fda39e00640 1 -- 172.21.15.159:0/3694536469 learned_addr learned my addr 172.21.15.159:0/3694536469 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:05:51.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.066+0000 7fda3a800640 1 --2- 172.21.15.159:0/3694536469 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda3c071710 0x7fda3c071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:51.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.066+0000 7fda33e00640 1 --2- 172.21.15.159:0/3694536469 >> v2:172.21.15.178:3300/0 conn(0x7fda3c09dcf0 0x7fda3c1453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:51.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.066+0000 7fda39400640 1 -- 172.21.15.159:0/3694536469 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2621061647 0 0) 0x7fda3c15cb90 con 0x7fda3c0acf10 2024-04-04T06:05:51.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.066+0000 7fda39400640 1 -- 172.21.15.159:0/3694536469 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda20003520 con 0x7fda3c0acf10 2024-04-04T06:05:51.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.067+0000 7fda39400640 1 -- 172.21.15.159:0/3694536469 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 531084305 0 0) 0x7fda20003520 con 0x7fda3c0acf10 2024-04-04T06:05:51.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.067+0000 7fda39400640 1 -- 172.21.15.159:0/3694536469 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda3c071710 msgr2=0x7fda3c071b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:51.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.067+0000 7fda39400640 1 --2- 172.21.15.159:0/3694536469 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda3c071710 0x7fda3c071b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:51.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.067+0000 7fda39400640 1 -- 172.21.15.159:0/3694536469 >> v2:172.21.15.178:3300/0 conn(0x7fda3c09dcf0 msgr2=0x7fda3c1453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:51.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.067+0000 7fda39400640 1 --2- 172.21.15.159:0/3694536469 >> v2:172.21.15.178:3300/0 conn(0x7fda3c09dcf0 0x7fda3c1453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:51.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.067+0000 7fda39400640 1 -- 172.21.15.159:0/3694536469 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c15ce70 con 0x7fda3c0acf10 2024-04-04T06:05:51.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.067+0000 7fda39400640 1 -- 172.21.15.159:0/3694536469 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fda2c001670 con 0x7fda3c0acf10 2024-04-04T06:05:51.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.067+0000 7fda39400640 1 -- 172.21.15.159:0/3694536469 >> v1:172.21.15.159:6789/0 conn(0x7fda3c0acf10 legacy=0x7fda3c0ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:51.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.067+0000 7fda39400640 1 --2- 172.21.15.159:0/3694536469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda200039b0 0x7fda20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:51.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.067+0000 7fda39400640 1 --2- 172.21.15.159:0/3694536469 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda3c071710 0x7fda20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:51.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.067+0000 7fda39400640 1 --2- 172.21.15.159:0/3694536469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda20004d40 0x7fda200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:51.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.067+0000 7fda39400640 1 -- 172.21.15.159:0/3694536469 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c145f60 con 0x7fda20004d40 2024-04-04T06:05:51.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.067+0000 7fda39400640 1 -- 172.21.15.159:0/3694536469 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c15c950 con 0x7fda200039b0 2024-04-04T06:05:51.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.067+0000 7fda39400640 1 -- 172.21.15.159:0/3694536469 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fda2000b9c0 con 0x7fda3c071710 2024-04-04T06:05:51.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.067+0000 7fda3a800640 1 --2- 172.21.15.159:0/3694536469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda20004d40 0x7fda200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:51.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.067+0000 7fda39e00640 1 --2- 172.21.15.159:0/3694536469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda200039b0 0x7fda20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:51.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.067+0000 7fda33e00640 1 --2- 172.21.15.159:0/3694536469 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda3c071710 0x7fda20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:51.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.068+0000 7fda3a800640 1 -- 172.21.15.159:0/3694536469 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda3c071710 msgr2=0x7fda20004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:51.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.068+0000 7fda3a800640 1 --2- 172.21.15.159:0/3694536469 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda3c071710 0x7fda20004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:51.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.068+0000 7fda3a800640 1 -- 172.21.15.159:0/3694536469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda200039b0 msgr2=0x7fda20003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:51.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.068+0000 7fda3a800640 1 --2- 172.21.15.159:0/3694536469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda200039b0 0x7fda20003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:51.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.068+0000 7fda3a800640 1 -- 172.21.15.159:0/3694536469 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda20003520 con 0x7fda20004d40 2024-04-04T06:05:51.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.068+0000 7fda39e00640 1 --2- 172.21.15.159:0/3694536469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda200039b0 0x7fda20003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:05:51.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.069+0000 7fda3a800640 1 --2- 172.21.15.159:0/3694536469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda20004d40 0x7fda200051c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fda34009fc0 tx=0x7fda20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86053562e7d52afc server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:51.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.069+0000 7fda39400640 1 -- 172.21.15.159:0/3694536469 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34039070 con 0x7fda20004d40 2024-04-04T06:05:51.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.069+0000 7fda39400640 1 -- 172.21.15.159:0/3694536469 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda3402cc20 con 0x7fda20004d40 2024-04-04T06:05:51.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.069+0000 7fda39400640 1 -- 172.21.15.159:0/3694536469 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34009c10 con 0x7fda20004d40 2024-04-04T06:05:51.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.069+0000 7fda40a00640 1 -- 172.21.15.159:0/3694536469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda20004d40 msgr2=0x7fda200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:51.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.069+0000 7fda40a00640 1 --2- 172.21.15.159:0/3694536469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda20004d40 0x7fda200051c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fda34009fc0 tx=0x7fda20005830 comp rx=0 tx=0).stop 2024-04-04T06:05:51.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.070+0000 7fda3a800640 1 -- 172.21.15.159:0/3694536469 reap_dead start 2024-04-04T06:05:51.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.070+0000 7fda40a00640 1 -- 172.21.15.159:0/3694536469 shutdown_connections 2024-04-04T06:05:51.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.070+0000 7fda40a00640 1 -- 172.21.15.159:0/3694536469 >> 172.21.15.159:0/3694536469 conn(0x7fda3c06bba0 msgr2=0x7fda3c0b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:51.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.070+0000 7fda40a00640 1 -- 172.21.15.159:0/3694536469 shutdown_connections 2024-04-04T06:05:51.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.070+0000 7fda40a00640 1 -- 172.21.15.159:0/3694536469 wait complete. 2024-04-04T06:05:51.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.071+0000 7fda40a00640 1 Processor -- start 2024-04-04T06:05:51.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.072+0000 7fda40a00640 1 -- start start 2024-04-04T06:05:51.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.072+0000 7fda40a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda20004d40 0x7fda3c15bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:51.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.072+0000 7fda40a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda3c13e500 0x7fda3c13e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:51.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.072+0000 7fda40a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda3c0af480 0x7fda3c0af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:51.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.072+0000 7fda40a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c15d430 con 0x7fda3c13e500 2024-04-04T06:05:51.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.072+0000 7fda40a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c15d7a0 con 0x7fda3c0af480 2024-04-04T06:05:51.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.072+0000 7fda40a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fda3c145930 con 0x7fda20004d40 2024-04-04T06:05:51.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.073+0000 7fda33e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda3c13e500 0x7fda3c13e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:51.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.073+0000 7fda33e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda3c13e500 0x7fda3c13e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48118/0 (socket says 172.21.15.159:48118) 2024-04-04T06:05:51.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.073+0000 7fda33e00640 1 -- 172.21.15.159:0/1534234306 learned_addr learned my addr 172.21.15.159:0/1534234306 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:51.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.073+0000 7fda39e00640 1 --2- 172.21.15.159:0/1534234306 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda20004d40 0x7fda3c15bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:51.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.073+0000 7fda3a800640 1 --2- 172.21.15.159:0/1534234306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda3c0af480 0x7fda3c0af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:51.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.073+0000 7fda33e00640 1 -- 172.21.15.159:0/1534234306 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda20004d40 msgr2=0x7fda3c15bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:51.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.073+0000 7fda33e00640 1 --2- 172.21.15.159:0/1534234306 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda20004d40 0x7fda3c15bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:51.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.073+0000 7fda33e00640 1 -- 172.21.15.159:0/1534234306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda3c0af480 msgr2=0x7fda3c0af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:51.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.073+0000 7fda33e00640 1 --2- 172.21.15.159:0/1534234306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda3c0af480 0x7fda3c0af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:51.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.073+0000 7fda33e00640 1 -- 172.21.15.159:0/1534234306 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c15c280 con 0x7fda3c13e500 2024-04-04T06:05:51.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.073+0000 7fda3a800640 1 --2- 172.21.15.159:0/1534234306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda3c0af480 0x7fda3c0af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:05:51.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.073+0000 7fda33e00640 1 --2- 172.21.15.159:0/1534234306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda3c13e500 0x7fda3c13e980 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fda240116d0 tx=0x7fda24011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:51.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.074+0000 7fda32000640 1 -- 172.21.15.159:0/1534234306 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2400f9c0 con 0x7fda3c13e500 2024-04-04T06:05:51.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.074+0000 7fda32000640 1 -- 172.21.15.159:0/1534234306 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda2400fb60 con 0x7fda3c13e500 2024-04-04T06:05:51.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.074+0000 7fda32000640 1 -- 172.21.15.159:0/1534234306 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda24015460 con 0x7fda3c13e500 2024-04-04T06:05:51.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.074+0000 7fda40a00640 1 -- 172.21.15.159:0/1534234306 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3c0afe70 con 0x7fda3c13e500 2024-04-04T06:05:51.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.074+0000 7fda40a00640 1 -- 172.21.15.159:0/1534234306 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda3c1f46c0 con 0x7fda3c13e500 2024-04-04T06:05:51.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.075+0000 7fda40a00640 1 -- 172.21.15.159:0/1534234306 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda3c0685c0 con 0x7fda3c13e500 2024-04-04T06:05:51.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.076+0000 7fda32000640 1 -- 172.21.15.159:0/1534234306 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda24015600 con 0x7fda3c13e500 2024-04-04T06:05:51.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.077+0000 7fda32000640 1 --2- 172.21.15.159:0/1534234306 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fda1806c920 0x7fda1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:51.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.077+0000 7fda32000640 1 -- 172.21.15.159:0/1534234306 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fda2408c290 con 0x7fda3c13e500 2024-04-04T06:05:51.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.078+0000 7fda39e00640 1 --2- 172.21.15.159:0/1534234306 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fda1806c920 0x7fda1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:51.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.079+0000 7fda39e00640 1 --2- 172.21.15.159:0/1534234306 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fda1806c920 0x7fda1806ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fda2c001eb0 tx=0x7fda2c001e20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:51.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.081+0000 7fda32000640 1 -- 172.21.15.159:0/1534234306 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fda24019030 con 0x7fda3c13e500 2024-04-04T06:05:51.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.282+0000 7fda40a00640 1 -- 172.21.15.159:0/1534234306 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fda3c0b3030 con 0x7fda3c13e500 2024-04-04T06:05:51.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.315+0000 7fda32000640 1 -- 172.21.15.159:0/1534234306 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7fda2405a970 con 0x7fda3c13e500 2024-04-04T06:05:51.311 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_metadata' created 2024-04-04T06:05:51.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.318+0000 7fda40a00640 1 -- 172.21.15.159:0/1534234306 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fda1806c920 msgr2=0x7fda1806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:51.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.318+0000 7fda40a00640 1 --2- 172.21.15.159:0/1534234306 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fda1806c920 0x7fda1806ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fda2c001eb0 tx=0x7fda2c001e20 comp rx=0 tx=0).stop 2024-04-04T06:05:51.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.318+0000 7fda40a00640 1 -- 172.21.15.159:0/1534234306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda3c13e500 msgr2=0x7fda3c13e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:51.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.318+0000 7fda40a00640 1 --2- 172.21.15.159:0/1534234306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda3c13e500 0x7fda3c13e980 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fda240116d0 tx=0x7fda24011b90 comp rx=0 tx=0).stop 2024-04-04T06:05:51.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.318+0000 7fda40a00640 1 -- 172.21.15.159:0/1534234306 shutdown_connections 2024-04-04T06:05:51.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.319+0000 7fda40a00640 1 --2- 172.21.15.159:0/1534234306 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fda1806c920 0x7fda1806ede0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:51.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.319+0000 7fda40a00640 1 --2- 172.21.15.159:0/1534234306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda3c0af480 0x7fda3c0af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:51.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.319+0000 7fda40a00640 1 --2- 172.21.15.159:0/1534234306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda3c13e500 0x7fda3c13e980 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:51.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.319+0000 7fda40a00640 1 --2- 172.21.15.159:0/1534234306 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda20004d40 0x7fda3c15bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:51.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.319+0000 7fda40a00640 1 -- 172.21.15.159:0/1534234306 >> 172.21.15.159:0/1534234306 conn(0x7fda3c06bba0 msgr2=0x7fda3c0b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:51.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.319+0000 7fda40a00640 1 -- 172.21.15.159:0/1534234306 shutdown_connections 2024-04-04T06:05:51.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.319+0000 7fda40a00640 1 -- 172.21.15.159:0/1534234306 wait complete. 2024-04-04T06:05:51.338 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-04T06:05:51.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.536+0000 7f10b4200640 1 Processor -- start 2024-04-04T06:05:51.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.536+0000 7f10b4200640 1 -- start start 2024-04-04T06:05:51.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.538+0000 7f10b4200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10ac0d0280 0x7f10ac0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:51.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.538+0000 7f10b4200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f10ac141c10 0x7f10ac071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:51.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.538+0000 7f10b4200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f10ac071eb0 0x7f10ac06d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:51.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.538+0000 7f10b4200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f10ac15ec80 con 0x7f10ac141c10 2024-04-04T06:05:51.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.538+0000 7f10b4200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f10ac0d0ba0 con 0x7f10ac071eb0 2024-04-04T06:05:51.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.538+0000 7f10b4200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f10ac15ef20 con 0x7f10ac0d0280 2024-04-04T06:05:51.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.538+0000 7f10b0e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f10ac141c10 0x7f10ac071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:51.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.538+0000 7f10b0e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f10ac141c10 0x7f10ac071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48134/0 (socket says 172.21.15.159:48134) 2024-04-04T06:05:51.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.538+0000 7f10b0e00640 1 -- 172.21.15.159:0/4177535635 learned_addr learned my addr 172.21.15.159:0/4177535635 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:51.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.539+0000 7f10b2200640 1 --2- 172.21.15.159:0/4177535635 >> v2:172.21.15.178:3300/0 conn(0x7f10ac071eb0 0x7f10ac06d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:51.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.539+0000 7f10b1800640 1 --2- 172.21.15.159:0/4177535635 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10ac0d0280 0x7f10ac0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:51.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.539+0000 7f10b0e00640 1 -- 172.21.15.159:0/4177535635 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10ac0d0280 msgr2=0x7f10ac0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:51.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.539+0000 7f10b0e00640 1 --2- 172.21.15.159:0/4177535635 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10ac0d0280 0x7f10ac0d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:51.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.539+0000 7f10b0e00640 1 -- 172.21.15.159:0/4177535635 >> v2:172.21.15.178:3300/0 conn(0x7f10ac071eb0 msgr2=0x7f10ac06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:51.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.539+0000 7f10b0e00640 1 --2- 172.21.15.159:0/4177535635 >> v2:172.21.15.178:3300/0 conn(0x7f10ac071eb0 0x7f10ac06d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:51.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.539+0000 7f10b0e00640 1 -- 172.21.15.159:0/4177535635 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ac15f200 con 0x7f10ac141c10 2024-04-04T06:05:51.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.540+0000 7f10b0e00640 1 --2- 172.21.15.159:0/4177535635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10ac141c10 0x7f10ac071970 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f10ac09e360 tx=0x7f109c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbe89841972edf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:51.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.540+0000 7f10abe00640 1 -- 172.21.15.159:0/4177535635 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c035020 con 0x7f10ac141c10 2024-04-04T06:05:51.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.540+0000 7f10abe00640 1 -- 172.21.15.159:0/4177535635 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f109c02d040 con 0x7f10ac141c10 2024-04-04T06:05:51.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.540+0000 7f10abe00640 1 -- 172.21.15.159:0/4177535635 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c031a80 con 0x7f10ac141c10 2024-04-04T06:05:51.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.541+0000 7f10b4200640 1 -- 172.21.15.159:0/4177535635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10ac141c10 msgr2=0x7f10ac071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:51.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.541+0000 7f10b4200640 1 --2- 172.21.15.159:0/4177535635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10ac141c10 0x7f10ac071970 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f10ac09e360 tx=0x7f109c02b900 comp rx=0 tx=0).stop 2024-04-04T06:05:51.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.541+0000 7f10b4200640 1 -- 172.21.15.159:0/4177535635 shutdown_connections 2024-04-04T06:05:51.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.541+0000 7f10b4200640 1 --2- 172.21.15.159:0/4177535635 >> v2:172.21.15.178:3300/0 conn(0x7f10ac071eb0 0x7f10ac06d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:51.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.541+0000 7f10b4200640 1 --2- 172.21.15.159:0/4177535635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10ac141c10 0x7f10ac071970 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:51.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.541+0000 7f10b4200640 1 --2- 172.21.15.159:0/4177535635 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10ac0d0280 0x7f10ac0d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:51.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.541+0000 7f10b4200640 1 -- 172.21.15.159:0/4177535635 >> 172.21.15.159:0/4177535635 conn(0x7f10ac06b750 msgr2=0x7f10ac06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:51.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.541+0000 7f10b4200640 1 -- 172.21.15.159:0/4177535635 shutdown_connections 2024-04-04T06:05:51.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.541+0000 7f10b4200640 1 -- 172.21.15.159:0/4177535635 wait complete. 2024-04-04T06:05:51.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.543+0000 7f10b4200640 1 Processor -- start 2024-04-04T06:05:51.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.543+0000 7f10b4200640 1 -- start start 2024-04-04T06:05:51.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.544+0000 7f10b4200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10ac071eb0 0x7f10ac099520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:51.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.544+0000 7f10b4200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10ac0d0280 0x7f10ac148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:51.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.544+0000 7f10b4200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10ac141c10 0x7f10ac1e6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:51.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.544+0000 7f10b4200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac158e90 con 0x7f10ac071eb0 2024-04-04T06:05:51.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.544+0000 7f10b4200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac15f7c0 con 0x7f10ac141c10 2024-04-04T06:05:51.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.544+0000 7f10b4200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f10ac1e7980 con 0x7f10ac0d0280 2024-04-04T06:05:51.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.544+0000 7f10b1800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10ac071eb0 0x7f10ac099520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:51.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.544+0000 7f10b1800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10ac071eb0 0x7f10ac099520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48138/0 (socket says 172.21.15.159:48138) 2024-04-04T06:05:51.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.544+0000 7f10b1800640 1 -- 172.21.15.159:0/3098789793 learned_addr learned my addr 172.21.15.159:0/3098789793 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:51.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.544+0000 7f10b0e00640 1 --2- 172.21.15.159:0/3098789793 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10ac0d0280 0x7f10ac148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:51.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.544+0000 7f10b2200640 1 --2- 172.21.15.159:0/3098789793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10ac141c10 0x7f10ac1e6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:51.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.544+0000 7f10b1800640 1 -- 172.21.15.159:0/3098789793 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10ac0d0280 msgr2=0x7f10ac148150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:51.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.544+0000 7f10b1800640 1 --2- 172.21.15.159:0/3098789793 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10ac0d0280 0x7f10ac148150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:51.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.544+0000 7f10b1800640 1 -- 172.21.15.159:0/3098789793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10ac141c10 msgr2=0x7f10ac1e6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:51.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.544+0000 7f10b1800640 1 --2- 172.21.15.159:0/3098789793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10ac141c10 0x7f10ac1e6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:51.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.544+0000 7f10b1800640 1 -- 172.21.15.159:0/3098789793 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ac1e7c60 con 0x7f10ac071eb0 2024-04-04T06:05:51.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.545+0000 7f10b1800640 1 --2- 172.21.15.159:0/3098789793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10ac071eb0 0x7f10ac099520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f10a000d800 tx=0x7f10a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:51.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.545+0000 7f10a9600640 1 -- 172.21.15.159:0/3098789793 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a000b6c0 con 0x7f10ac071eb0 2024-04-04T06:05:51.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.545+0000 7f10a9600640 1 -- 172.21.15.159:0/3098789793 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a0019070 con 0x7f10ac071eb0 2024-04-04T06:05:51.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.545+0000 7f10a9600640 1 -- 172.21.15.159:0/3098789793 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a000ba80 con 0x7f10ac071eb0 2024-04-04T06:05:51.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.545+0000 7f10b4200640 1 -- 172.21.15.159:0/3098789793 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10ac1e7f40 con 0x7f10ac071eb0 2024-04-04T06:05:51.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.545+0000 7f10b4200640 1 -- 172.21.15.159:0/3098789793 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10ac14aeb0 con 0x7f10ac071eb0 2024-04-04T06:05:51.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.546+0000 7f10b4200640 1 -- 172.21.15.159:0/3098789793 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10ac0685c0 con 0x7f10ac071eb0 2024-04-04T06:05:51.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.548+0000 7f10a9600640 1 -- 172.21.15.159:0/3098789793 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10a000bc20 con 0x7f10ac071eb0 2024-04-04T06:05:51.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.549+0000 7f10a9600640 1 --2- 172.21.15.159:0/3098789793 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f109006c8f0 0x7f109006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:51.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.549+0000 7f10a9600640 1 -- 172.21.15.159:0/3098789793 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f10a008be90 con 0x7f10ac071eb0 2024-04-04T06:05:51.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.550+0000 7f10b0e00640 1 --2- 172.21.15.159:0/3098789793 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f109006c8f0 0x7f109006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:51.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.551+0000 7f10b0e00640 1 --2- 172.21.15.159:0/3098789793 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f109006c8f0 0x7f109006edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f109c036000 tx=0x7f109c03c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:51.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.555+0000 7f10a9600640 1 -- 172.21.15.159:0/3098789793 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f10a005a4a0 con 0x7f10ac071eb0 2024-04-04T06:05:51.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:51.757+0000 7f10b4200640 1 -- 172.21.15.159:0/3098789793 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f10ac0cfdb0 con 0x7f10ac071eb0 2024-04-04T06:05:52.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.331+0000 7f10a9600640 1 -- 172.21.15.159:0/3098789793 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7f10a0059da0 con 0x7f10ac071eb0 2024-04-04T06:05:52.326 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_data' created 2024-04-04T06:05:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.335+0000 7f10b4200640 1 -- 172.21.15.159:0/3098789793 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f109006c8f0 msgr2=0x7f109006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.335+0000 7f10b4200640 1 --2- 172.21.15.159:0/3098789793 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f109006c8f0 0x7f109006edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f109c036000 tx=0x7f109c03c040 comp rx=0 tx=0).stop 2024-04-04T06:05:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.335+0000 7f10b4200640 1 -- 172.21.15.159:0/3098789793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10ac071eb0 msgr2=0x7f10ac099520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.335+0000 7f10b4200640 1 --2- 172.21.15.159:0/3098789793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10ac071eb0 0x7f10ac099520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f10a000d800 tx=0x7f10a000dcc0 comp rx=0 tx=0).stop 2024-04-04T06:05:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.335+0000 7f10b4200640 1 -- 172.21.15.159:0/3098789793 shutdown_connections 2024-04-04T06:05:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.335+0000 7f10b4200640 1 --2- 172.21.15.159:0/3098789793 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f109006c8f0 0x7f109006edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.335+0000 7f10b4200640 1 --2- 172.21.15.159:0/3098789793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10ac141c10 0x7f10ac1e6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.335+0000 7f10b4200640 1 --2- 172.21.15.159:0/3098789793 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10ac0d0280 0x7f10ac148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.335+0000 7f10b4200640 1 --2- 172.21.15.159:0/3098789793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10ac071eb0 0x7f10ac099520 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.335+0000 7f10b4200640 1 -- 172.21.15.159:0/3098789793 >> 172.21.15.159:0/3098789793 conn(0x7f10ac06b750 msgr2=0x7f10ac0af320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.335+0000 7f10b4200640 1 -- 172.21.15.159:0/3098789793 shutdown_connections 2024-04-04T06:05:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.335+0000 7f10b4200640 1 -- 172.21.15.159:0/3098789793 wait complete. 2024-04-04T06:05:52.363 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-04T06:05:52.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.552+0000 7f44b8800640 1 Processor -- start 2024-04-04T06:05:52.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.552+0000 7f44b8800640 1 -- start start 2024-04-04T06:05:52.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.554+0000 7f44b8800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f44b00d0990 0x7f44b00aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:52.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.554+0000 7f44b8800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44b00af2e0 0x7f44b01600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:52.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.554+0000 7f44b8800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44b00afab0 con 0x7f44b0153580 2024-04-04T06:05:52.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.554+0000 7f44b8800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f44b015cc80 con 0x7f44b00d0990 2024-04-04T06:05:52.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.554+0000 7f44b8800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f44b015da00 con 0x7f44b00af2e0 2024-04-04T06:05:52.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.554+0000 7f44b5400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f44b00d0990 0x7f44b00aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:52.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.554+0000 7f44b5400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f44b00d0990 0x7f44b00aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48140/0 (socket says 172.21.15.159:48140) 2024-04-04T06:05:52.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.554+0000 7f44b5400640 1 -- 172.21.15.159:0/2012209369 learned_addr learned my addr 172.21.15.159:0/2012209369 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:52.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.554+0000 7f44b6800640 1 --2- 172.21.15.159:0/2012209369 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44b00af2e0 0x7f44b01600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:52.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.555+0000 7f44b5400640 1 -- 172.21.15.159:0/2012209369 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44b00af2e0 msgr2=0x7f44b01600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:52.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.555+0000 7f44b5400640 1 --2- 172.21.15.159:0/2012209369 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44b00af2e0 0x7f44b01600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:52.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.555+0000 7f44b5400640 1 -- 172.21.15.159:0/2012209369 >> v1:172.21.15.178:6789/0 conn(0x7f44b0153580 legacy=0x7f44b00d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:52.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.555+0000 7f44b5400640 1 -- 172.21.15.159:0/2012209369 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b015cf20 con 0x7f44b00d0990 2024-04-04T06:05:52.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.556+0000 7f44b5400640 1 --2- 172.21.15.159:0/2012209369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44b00d0990 0x7f44b00aeda0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009870 tx=0x7f44a002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4561f0112c52f0c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:52.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.556+0000 7f44b4a00640 1 -- 172.21.15.159:0/2012209369 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0035020 con 0x7f44b00d0990 2024-04-04T06:05:52.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.556+0000 7f44b4a00640 1 -- 172.21.15.159:0/2012209369 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a0031640 con 0x7f44b00d0990 2024-04-04T06:05:52.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.556+0000 7f44b4a00640 1 -- 172.21.15.159:0/2012209369 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a002d040 con 0x7f44b00d0990 2024-04-04T06:05:52.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.557+0000 7f44b8800640 1 -- 172.21.15.159:0/2012209369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44b00d0990 msgr2=0x7f44b00aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:52.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.557+0000 7f44b8800640 1 --2- 172.21.15.159:0/2012209369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44b00d0990 0x7f44b00aeda0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009870 tx=0x7f44a002b8f0 comp rx=0 tx=0).stop 2024-04-04T06:05:52.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.557+0000 7f44b8800640 1 -- 172.21.15.159:0/2012209369 shutdown_connections 2024-04-04T06:05:52.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.557+0000 7f44b8800640 1 --2- 172.21.15.159:0/2012209369 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44b00af2e0 0x7f44b01600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:52.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.557+0000 7f44b8800640 1 --2- 172.21.15.159:0/2012209369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44b00d0990 0x7f44b00aeda0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:52.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.557+0000 7f44b8800640 1 -- 172.21.15.159:0/2012209369 >> 172.21.15.159:0/2012209369 conn(0x7f44b006b750 msgr2=0x7f44b006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:52.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.557+0000 7f44b8800640 1 -- 172.21.15.159:0/2012209369 shutdown_connections 2024-04-04T06:05:52.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.557+0000 7f44b8800640 1 -- 172.21.15.159:0/2012209369 wait complete. 2024-04-04T06:05:52.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.559+0000 7f44b8800640 1 Processor -- start 2024-04-04T06:05:52.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.559+0000 7f44b8800640 1 -- start start 2024-04-04T06:05:52.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.560+0000 7f44b8800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44b00af2e0 0x7f44b009a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:52.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.560+0000 7f44b8800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44b00d0990 0x7f44b014a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:52.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.560+0000 7f44b8800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44b0153580 0x7f44b014af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:52.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.560+0000 7f44b8800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f44b0153960 con 0x7f44b00d0990 2024-04-04T06:05:52.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.560+0000 7f44b8800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f44b015d4e0 con 0x7f44b00af2e0 2024-04-04T06:05:52.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.560+0000 7f44b8800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f44b00af710 con 0x7f44b0153580 2024-04-04T06:05:52.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.560+0000 7f44b5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44b00d0990 0x7f44b014a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:52.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.560+0000 7f44b5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44b00d0990 0x7f44b014a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48144/0 (socket says 172.21.15.159:48144) 2024-04-04T06:05:52.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.560+0000 7f44b5400640 1 -- 172.21.15.159:0/2739010096 learned_addr learned my addr 172.21.15.159:0/2739010096 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:52.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.561+0000 7f44b5e00640 1 --2- 172.21.15.159:0/2739010096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44b00af2e0 0x7f44b009a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:52.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.561+0000 7f44b6800640 1 --2- 172.21.15.159:0/2739010096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44b0153580 0x7f44b014af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:52.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.561+0000 7f44b5400640 1 -- 172.21.15.159:0/2739010096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44b0153580 msgr2=0x7f44b014af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:52.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.561+0000 7f44b5400640 1 --2- 172.21.15.159:0/2739010096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44b0153580 0x7f44b014af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:52.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.561+0000 7f44b5400640 1 -- 172.21.15.159:0/2739010096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44b00af2e0 msgr2=0x7f44b009a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:52.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.561+0000 7f44b5400640 1 --2- 172.21.15.159:0/2739010096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44b00af2e0 0x7f44b009a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:52.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.561+0000 7f44b5400640 1 -- 172.21.15.159:0/2739010096 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b0145580 con 0x7f44b00d0990 2024-04-04T06:05:52.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.561+0000 7f44b5400640 1 --2- 172.21.15.159:0/2739010096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44b00d0990 0x7f44b014a9f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f44a0000c00 tx=0x7f44a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:52.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.561+0000 7f44aa000640 1 -- 172.21.15.159:0/2739010096 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0040070 con 0x7f44b00d0990 2024-04-04T06:05:52.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.561+0000 7f44aa000640 1 -- 172.21.15.159:0/2739010096 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a00399f0 con 0x7f44b00d0990 2024-04-04T06:05:52.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.562+0000 7f44aa000640 1 -- 172.21.15.159:0/2739010096 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0035040 con 0x7f44b00d0990 2024-04-04T06:05:52.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.562+0000 7f44b8800640 1 -- 172.21.15.159:0/2739010096 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b0156a00 con 0x7f44b00d0990 2024-04-04T06:05:52.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.562+0000 7f44b8800640 1 -- 172.21.15.159:0/2739010096 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b0145b00 con 0x7f44b00d0990 2024-04-04T06:05:52.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.563+0000 7f44b8800640 1 -- 172.21.15.159:0/2739010096 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b00685c0 con 0x7f44b00d0990 2024-04-04T06:05:52.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.564+0000 7f44aa000640 1 -- 172.21.15.159:0/2739010096 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f44a004b060 con 0x7f44b00d0990 2024-04-04T06:05:52.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.565+0000 7f44aa000640 1 --2- 172.21.15.159:0/2739010096 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f449406c8f0 0x7f449406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:52.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.565+0000 7f44aa000640 1 -- 172.21.15.159:0/2739010096 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f44a00ad3b0 con 0x7f44b00d0990 2024-04-04T06:05:52.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.565+0000 7f44b5e00640 1 --2- 172.21.15.159:0/2739010096 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f449406c8f0 0x7f449406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:52.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.566+0000 7f44b5e00640 1 --2- 172.21.15.159:0/2739010096 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f449406c8f0 0x7f449406edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f44a4001640 tx=0x7f44a4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:52.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.570+0000 7f44aa000640 1 -- 172.21.15.159:0/2739010096 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f44a007b7a0 con 0x7f44b00d0990 2024-04-04T06:05:52.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:52.799+0000 7f44b8800640 1 -- 172.21.15.159:0/2739010096 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f44b015eeb0 con 0x7f44b00d0990 2024-04-04T06:05:53.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.335+0000 7f44aa000640 1 -- 172.21.15.159:0/2739010096 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f44a007b0a0 con 0x7f44b00d0990 2024-04-04T06:05:53.330 INFO:teuthology.orchestra.run.smithi159.stderr:new fs with metadata pool 2 and data pool 3 2024-04-04T06:05:53.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.339+0000 7f44b8800640 1 -- 172.21.15.159:0/2739010096 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f449406c8f0 msgr2=0x7f449406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:53.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.339+0000 7f44b8800640 1 --2- 172.21.15.159:0/2739010096 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f449406c8f0 0x7f449406edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f44a4001640 tx=0x7f44a4009040 comp rx=0 tx=0).stop 2024-04-04T06:05:53.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.339+0000 7f44b8800640 1 -- 172.21.15.159:0/2739010096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44b00d0990 msgr2=0x7f44b014a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:53.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.339+0000 7f44b8800640 1 --2- 172.21.15.159:0/2739010096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44b00d0990 0x7f44b014a9f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f44a0000c00 tx=0x7f44a00046c0 comp rx=0 tx=0).stop 2024-04-04T06:05:53.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.339+0000 7f44b8800640 1 -- 172.21.15.159:0/2739010096 shutdown_connections 2024-04-04T06:05:53.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.339+0000 7f44b8800640 1 --2- 172.21.15.159:0/2739010096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44b0153580 0x7f44b014af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:53.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.340+0000 7f44b8800640 1 --2- 172.21.15.159:0/2739010096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44b00d0990 0x7f44b014a9f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:53.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.340+0000 7f44b8800640 1 --2- 172.21.15.159:0/2739010096 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f449406c8f0 0x7f449406edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:53.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.340+0000 7f44b8800640 1 --2- 172.21.15.159:0/2739010096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44b00af2e0 0x7f44b009a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:53.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.340+0000 7f44b8800640 1 -- 172.21.15.159:0/2739010096 >> 172.21.15.159:0/2739010096 conn(0x7f44b006b750 msgr2=0x7f44b00b0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:53.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.342+0000 7f44b8800640 1 -- 172.21.15.159:0/2739010096 shutdown_connections 2024-04-04T06:05:53.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.342+0000 7f44b8800640 1 -- 172.21.15.159:0/2739010096 wait complete. 2024-04-04T06:05:53.376 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:05:53.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.555+0000 7f6b39a00640 1 Processor -- start 2024-04-04T06:05:53.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.555+0000 7f6b39a00640 1 -- start start 2024-04-04T06:05:53.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.556+0000 7f6b39a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b340d02a0 0x7f6b34165c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:53.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.556+0000 7f6b39a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b34071460 con 0x7f6b340cf1b0 2024-04-04T06:05:53.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.556+0000 7f6b39a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b340721b0 con 0x7f6b341442c0 2024-04-04T06:05:53.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.556+0000 7f6b39a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6b34071d20 con 0x7f6b340d02a0 2024-04-04T06:05:53.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.557+0000 7f6b2b400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f6b340cf1b0 0x7f6b340cfb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58250/0 (socket says 172.21.15.159:58250) 2024-04-04T06:05:53.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.557+0000 7f6b2b400640 1 -- 172.21.15.159:0/44919154 learned_addr learned my addr 172.21.15.159:0/44919154 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:05:53.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.557+0000 7f6b33400640 1 -- 172.21.15.159:0/44919154 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 392208467 0 0) 0x7f6b34071460 con 0x7f6b340cf1b0 2024-04-04T06:05:53.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.557+0000 7f6b33400640 1 -- 172.21.15.159:0/44919154 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b18003520 con 0x7f6b340cf1b0 2024-04-04T06:05:53.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.557+0000 7f6b39000640 1 --2- 172.21.15.159:0/44919154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b340d02a0 0x7f6b34165c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:53.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.557+0000 7f6b33400640 1 -- 172.21.15.159:0/44919154 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 928185376 0 0) 0x7f6b18003520 con 0x7f6b340cf1b0 2024-04-04T06:05:53.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.557+0000 7f6b33400640 1 -- 172.21.15.159:0/44919154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b340d02a0 msgr2=0x7f6b34165c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:53.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.557+0000 7f6b33400640 1 --2- 172.21.15.159:0/44919154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b340d02a0 0x7f6b34165c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:53.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.557+0000 7f6b33400640 1 -- 172.21.15.159:0/44919154 >> v1:172.21.15.178:6789/0 conn(0x7f6b341442c0 legacy=0x7f6b340ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:53.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.557+0000 7f6b33400640 1 -- 172.21.15.159:0/44919154 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b340716d0 con 0x7f6b340cf1b0 2024-04-04T06:05:53.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.557+0000 7f6b33400640 1 -- 172.21.15.159:0/44919154 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f6b1c001670 con 0x7f6b340cf1b0 2024-04-04T06:05:53.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.557+0000 7f6b33400640 1 -- 172.21.15.159:0/44919154 >> v1:172.21.15.159:6789/0 conn(0x7f6b340cf1b0 legacy=0x7f6b340cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:53.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.557+0000 7f6b33400640 1 --2- 172.21.15.159:0/44919154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b180039e0 0x7f6b18003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:53.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.557+0000 7f6b33400640 1 --2- 172.21.15.159:0/44919154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b340d02a0 0x7f6b18004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:53.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.557+0000 7f6b33400640 1 --2- 172.21.15.159:0/44919154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b18004d40 0x7f6b180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:53.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.558+0000 7f6b33400640 1 -- 172.21.15.159:0/44919154 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6b34071d20 con 0x7f6b18004d40 2024-04-04T06:05:53.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.558+0000 7f6b33400640 1 -- 172.21.15.159:0/44919154 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1800bbe0 con 0x7f6b180039e0 2024-04-04T06:05:53.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.558+0000 7f6b33400640 1 -- 172.21.15.159:0/44919154 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1800bde0 con 0x7f6b340d02a0 2024-04-04T06:05:53.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.558+0000 7f6b39000640 1 --2- 172.21.15.159:0/44919154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b18004d40 0x7f6b180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:53.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.558+0000 7f6b2b400640 1 --2- 172.21.15.159:0/44919154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b340d02a0 0x7f6b18004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:53.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.558+0000 7f6b33e00640 1 --2- 172.21.15.159:0/44919154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b180039e0 0x7f6b18003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:53.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.558+0000 7f6b39000640 1 -- 172.21.15.159:0/44919154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b340d02a0 msgr2=0x7f6b18004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:53.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.558+0000 7f6b39000640 1 --2- 172.21.15.159:0/44919154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b340d02a0 0x7f6b18004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:53.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.558+0000 7f6b39000640 1 -- 172.21.15.159:0/44919154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b180039e0 msgr2=0x7f6b18003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:53.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.558+0000 7f6b39000640 1 --2- 172.21.15.159:0/44919154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b180039e0 0x7f6b18003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:53.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.558+0000 7f6b39000640 1 -- 172.21.15.159:0/44919154 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b18003520 con 0x7f6b18004d40 2024-04-04T06:05:53.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.558+0000 7f6b2b400640 1 --2- 172.21.15.159:0/44919154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b340d02a0 0x7f6b18004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:05:53.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.559+0000 7f6b39000640 1 --2- 172.21.15.159:0/44919154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b18004d40 0x7f6b180051c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c007330 tx=0x7f6b18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f349254ac15d80 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:53.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.559+0000 7f6b33400640 1 -- 172.21.15.159:0/44919154 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c039070 con 0x7f6b18004d40 2024-04-04T06:05:53.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.559+0000 7f6b33400640 1 -- 172.21.15.159:0/44919154 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b2c02cc80 con 0x7f6b18004d40 2024-04-04T06:05:53.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.559+0000 7f6b33400640 1 -- 172.21.15.159:0/44919154 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c009be0 con 0x7f6b18004d40 2024-04-04T06:05:53.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.560+0000 7f6b39a00640 1 -- 172.21.15.159:0/44919154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b18004d40 msgr2=0x7f6b180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:53.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.560+0000 7f6b39a00640 1 --2- 172.21.15.159:0/44919154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b18004d40 0x7f6b180051c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c007330 tx=0x7f6b18005830 comp rx=0 tx=0).stop 2024-04-04T06:05:53.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.560+0000 7f6b39000640 1 -- 172.21.15.159:0/44919154 reap_dead start 2024-04-04T06:05:53.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.560+0000 7f6b39a00640 1 -- 172.21.15.159:0/44919154 shutdown_connections 2024-04-04T06:05:53.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.560+0000 7f6b39a00640 1 -- 172.21.15.159:0/44919154 >> 172.21.15.159:0/44919154 conn(0x7f6b340ccad0 msgr2=0x7f6b340cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:53.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.560+0000 7f6b39a00640 1 -- 172.21.15.159:0/44919154 shutdown_connections 2024-04-04T06:05:53.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.560+0000 7f6b39a00640 1 -- 172.21.15.159:0/44919154 wait complete. 2024-04-04T06:05:53.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.562+0000 7f6b39a00640 1 Processor -- start 2024-04-04T06:05:53.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.562+0000 7f6b39a00640 1 -- start start 2024-04-04T06:05:53.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.563+0000 7f6b39a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b18004d40 0x7f6b3415a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:53.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.563+0000 7f6b39a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b3413ee10 0x7f6b3413d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:53.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.563+0000 7f6b39a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b3413f290 0x7f6b340b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:53.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.563+0000 7f6b39a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6b341446a0 con 0x7f6b3413f290 2024-04-04T06:05:53.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.563+0000 7f6b39a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6b340d0770 con 0x7f6b3413ee10 2024-04-04T06:05:53.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.563+0000 7f6b39a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6b3415a7e0 con 0x7f6b18004d40 2024-04-04T06:05:53.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.563+0000 7f6b39000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b3413f290 0x7f6b340b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:53.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.563+0000 7f6b39000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b3413f290 0x7f6b340b0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48164/0 (socket says 172.21.15.159:48164) 2024-04-04T06:05:53.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.563+0000 7f6b39000640 1 -- 172.21.15.159:0/65391780 learned_addr learned my addr 172.21.15.159:0/65391780 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:53.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.563+0000 7f6b33e00640 1 --2- 172.21.15.159:0/65391780 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b18004d40 0x7f6b3415a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:53.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.563+0000 7f6b2b400640 1 --2- 172.21.15.159:0/65391780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b3413ee10 0x7f6b3413d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:53.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.563+0000 7f6b39000640 1 -- 172.21.15.159:0/65391780 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b18004d40 msgr2=0x7f6b3415a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:53.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.563+0000 7f6b39000640 1 --2- 172.21.15.159:0/65391780 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b18004d40 0x7f6b3415a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:53.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.563+0000 7f6b39000640 1 -- 172.21.15.159:0/65391780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b3413ee10 msgr2=0x7f6b3413d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:53.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.563+0000 7f6b39000640 1 --2- 172.21.15.159:0/65391780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b3413ee10 0x7f6b3413d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:53.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.563+0000 7f6b39000640 1 -- 172.21.15.159:0/65391780 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3413dd70 con 0x7f6b3413f290 2024-04-04T06:05:53.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.563+0000 7f6b33e00640 1 --2- 172.21.15.159:0/65391780 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b18004d40 0x7f6b3415a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:05:53.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.564+0000 7f6b39000640 1 --2- 172.21.15.159:0/65391780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b3413f290 0x7f6b340b0b60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c007450 tx=0x7f6b2c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:53.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.564+0000 7f6b30c00640 1 -- 172.21.15.159:0/65391780 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c039030 con 0x7f6b3413f290 2024-04-04T06:05:53.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.564+0000 7f6b30c00640 1 -- 172.21.15.159:0/65391780 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b2c043070 con 0x7f6b3413f290 2024-04-04T06:05:53.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.564+0000 7f6b30c00640 1 -- 172.21.15.159:0/65391780 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c009be0 con 0x7f6b3413f290 2024-04-04T06:05:53.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.564+0000 7f6b39a00640 1 -- 172.21.15.159:0/65391780 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3413e050 con 0x7f6b3413f290 2024-04-04T06:05:53.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.564+0000 7f6b39a00640 1 -- 172.21.15.159:0/65391780 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b3413e610 con 0x7f6b3413f290 2024-04-04T06:05:53.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.565+0000 7f6b39a00640 1 -- 172.21.15.159:0/65391780 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b34144820 con 0x7f6b3413f290 2024-04-04T06:05:53.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.566+0000 7f6b30c00640 1 -- 172.21.15.159:0/65391780 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b2c03d7d0 con 0x7f6b3413f290 2024-04-04T06:05:53.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.567+0000 7f6b30c00640 1 --2- 172.21.15.159:0/65391780 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6b1006c8f0 0x7f6b1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:53.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.567+0000 7f6b30c00640 1 -- 172.21.15.159:0/65391780 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6b2c0ade70 con 0x7f6b3413f290 2024-04-04T06:05:53.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.567+0000 7f6b33e00640 1 --2- 172.21.15.159:0/65391780 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6b1006c8f0 0x7f6b1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:53.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.568+0000 7f6b33e00640 1 --2- 172.21.15.159:0/65391780 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6b1006c8f0 0x7f6b1006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6b240016f0 tx=0x7f6b24009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:53.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.571+0000 7f6b30c00640 1 -- 172.21.15.159:0/65391780 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6b34144820 con 0x7f6b3413f290 2024-04-04T06:05:53.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.766+0000 7f6b39a00640 1 -- 172.21.15.159:0/65391780 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b34075940 con 0x7f6b3413f290 2024-04-04T06:05:53.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.768+0000 7f6b30c00640 1 -- 172.21.15.159:0/65391780 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16630 (secure 0 0 0) 0x7f6b2c07c170 con 0x7f6b3413f290 2024-04-04T06:05:53.763 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:05:53.763 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":20,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:05:52.801141+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:05:51.297293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-04T06:05:51.770792+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:05:53.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.770+0000 7f6b39a00640 1 -- 172.21.15.159:0/65391780 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6b1006c8f0 msgr2=0x7f6b1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:53.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.770+0000 7f6b39a00640 1 --2- 172.21.15.159:0/65391780 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6b1006c8f0 0x7f6b1006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6b240016f0 tx=0x7f6b24009040 comp rx=0 tx=0).stop 2024-04-04T06:05:53.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.770+0000 7f6b39a00640 1 -- 172.21.15.159:0/65391780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b3413f290 msgr2=0x7f6b340b0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:53.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.770+0000 7f6b39a00640 1 --2- 172.21.15.159:0/65391780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b3413f290 0x7f6b340b0b60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c007450 tx=0x7f6b2c004ab0 comp rx=0 tx=0).stop 2024-04-04T06:05:53.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.770+0000 7f6b39a00640 1 -- 172.21.15.159:0/65391780 shutdown_connections 2024-04-04T06:05:53.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.770+0000 7f6b39a00640 1 --2- 172.21.15.159:0/65391780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b3413f290 0x7f6b340b0b60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:53.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.770+0000 7f6b39a00640 1 --2- 172.21.15.159:0/65391780 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6b1006c8f0 0x7f6b1006edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:53.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.770+0000 7f6b39a00640 1 --2- 172.21.15.159:0/65391780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b3413ee10 0x7f6b3413d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:53.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.770+0000 7f6b39a00640 1 --2- 172.21.15.159:0/65391780 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b18004d40 0x7f6b3415a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:53.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.770+0000 7f6b39a00640 1 -- 172.21.15.159:0/65391780 >> 172.21.15.159:0/65391780 conn(0x7f6b340ccad0 msgr2=0x7f6b3406f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:53.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.771+0000 7f6b39a00640 1 -- 172.21.15.159:0/65391780 shutdown_connections 2024-04-04T06:05:53.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.771+0000 7f6b39a00640 1 -- 172.21.15.159:0/65391780 wait complete. 2024-04-04T06:05:53.783 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:05:53.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.993+0000 7fc712600640 1 Processor -- start 2024-04-04T06:05:53.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.993+0000 7fc712600640 1 -- start start 2024-04-04T06:05:53.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.994+0000 7fc712600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc70c15c3b0 0x7fc70c15d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:53.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.994+0000 7fc712600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc70c0d0d60 0x7fc70c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:53.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.995+0000 7fc712600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc70c0afb80 con 0x7fc70c0cf980 2024-04-04T06:05:53.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.995+0000 7fc712600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fc70c0ad120 con 0x7fc70c15c3b0 2024-04-04T06:05:53.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.995+0000 7fc712600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc70c0ad370 con 0x7fc70c0d0d60 2024-04-04T06:05:53.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.995+0000 7fc70b400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc70c15c3b0 0x7fc70c15d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:53.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.995+0000 7fc70b400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc70c15c3b0 0x7fc70c15d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43186/0 (socket says 172.21.15.159:43186) 2024-04-04T06:05:53.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.995+0000 7fc70b400640 1 -- 172.21.15.159:0/4208578419 learned_addr learned my addr 172.21.15.159:0/4208578419 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:53.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.995+0000 7fc70b400640 1 -- 172.21.15.159:0/4208578419 >> v2:172.21.15.178:3300/0 conn(0x7fc70c0d0d60 msgr2=0x7fc70c06d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:53.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.995+0000 7fc70be00640 1 --2- 172.21.15.159:0/4208578419 >> v2:172.21.15.178:3300/0 conn(0x7fc70c0d0d60 0x7fc70c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:53.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.995+0000 7fc70b400640 1 --2- 172.21.15.159:0/4208578419 >> v2:172.21.15.178:3300/0 conn(0x7fc70c0d0d60 0x7fc70c06d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:53.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.995+0000 7fc70b400640 1 -- 172.21.15.159:0/4208578419 >> v1:172.21.15.178:6789/0 conn(0x7fc70c0cf980 legacy=0x7fc70c0d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:53.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.995+0000 7fc70b400640 1 -- 172.21.15.159:0/4208578419 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c0ad650 con 0x7fc70c15c3b0 2024-04-04T06:05:53.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.996+0000 7fc70b400640 1 --2- 172.21.15.159:0/4208578419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc70c15c3b0 0x7fc70c15d2c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0b2ed0 tx=0x7fc6fc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2c598f0ca87b0c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:53.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.997+0000 7fc70aa00640 1 -- 172.21.15.159:0/4208578419 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc035020 con 0x7fc70c15c3b0 2024-04-04T06:05:53.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.997+0000 7fc70aa00640 1 -- 172.21.15.159:0/4208578419 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6fc031870 con 0x7fc70c15c3b0 2024-04-04T06:05:53.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.997+0000 7fc70aa00640 1 -- 172.21.15.159:0/4208578419 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc031c40 con 0x7fc70c15c3b0 2024-04-04T06:05:53.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.997+0000 7fc712600640 1 -- 172.21.15.159:0/4208578419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc70c15c3b0 msgr2=0x7fc70c15d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:53.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.997+0000 7fc712600640 1 --2- 172.21.15.159:0/4208578419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc70c15c3b0 0x7fc70c15d2c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0b2ed0 tx=0x7fc6fc02b910 comp rx=0 tx=0).stop 2024-04-04T06:05:53.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.997+0000 7fc712600640 1 -- 172.21.15.159:0/4208578419 shutdown_connections 2024-04-04T06:05:53.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.997+0000 7fc712600640 1 --2- 172.21.15.159:0/4208578419 >> v2:172.21.15.178:3300/0 conn(0x7fc70c0d0d60 0x7fc70c06d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:53.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.997+0000 7fc712600640 1 --2- 172.21.15.159:0/4208578419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc70c15c3b0 0x7fc70c15d2c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:53.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.997+0000 7fc712600640 1 -- 172.21.15.159:0/4208578419 >> 172.21.15.159:0/4208578419 conn(0x7fc70c06b990 msgr2=0x7fc70c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:53.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.997+0000 7fc712600640 1 -- 172.21.15.159:0/4208578419 shutdown_connections 2024-04-04T06:05:53.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.997+0000 7fc712600640 1 -- 172.21.15.159:0/4208578419 wait complete. 2024-04-04T06:05:53.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.999+0000 7fc712600640 1 Processor -- start 2024-04-04T06:05:53.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:53.999+0000 7fc712600640 1 -- start start 2024-04-04T06:05:53.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.000+0000 7fc712600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc70c0cf980 0x7fc70c14a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:53.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.000+0000 7fc712600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc70c0d0d60 0x7fc70c156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:53.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.000+0000 7fc712600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc70c15c3b0 0x7fc70c157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:53.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.000+0000 7fc712600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c0ad930 con 0x7fc70c0cf980 2024-04-04T06:05:53.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.000+0000 7fc712600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c15cda0 con 0x7fc70c0d0d60 2024-04-04T06:05:53.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.000+0000 7fc712600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc70c15d800 con 0x7fc70c15c3b0 2024-04-04T06:05:53.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.000+0000 7fc70b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc70c0cf980 0x7fc70c14a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:53.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.000+0000 7fc70b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc70c0cf980 0x7fc70c14a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43200/0 (socket says 172.21.15.159:43200) 2024-04-04T06:05:53.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.000+0000 7fc70b400640 1 -- 172.21.15.159:0/2109518917 learned_addr learned my addr 172.21.15.159:0/2109518917 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:53.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.001+0000 7fc70b400640 1 -- 172.21.15.159:0/2109518917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc70c15c3b0 msgr2=0x7fc70c157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:53.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.001+0000 7fc70b400640 1 --2- 172.21.15.159:0/2109518917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc70c15c3b0 0x7fc70c157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:53.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.001+0000 7fc70b400640 1 -- 172.21.15.159:0/2109518917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc70c0d0d60 msgr2=0x7fc70c156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:53.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.001+0000 7fc70b400640 1 --2- 172.21.15.159:0/2109518917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc70c0d0d60 0x7fc70c156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:53.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.001+0000 7fc70b400640 1 -- 172.21.15.159:0/2109518917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c151c10 con 0x7fc70c0cf980 2024-04-04T06:05:53.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.001+0000 7fc70b400640 1 --2- 172.21.15.159:0/2109518917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc70c0cf980 0x7fc70c14a890 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc000c00 tx=0x7fc6fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:53.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.001+0000 7fc703e00640 1 -- 172.21.15.159:0/2109518917 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc03ad60 con 0x7fc70c0cf980 2024-04-04T06:05:53.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.001+0000 7fc703e00640 1 -- 172.21.15.159:0/2109518917 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6fc0443f0 con 0x7fc70c0cf980 2024-04-04T06:05:53.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.001+0000 7fc712600640 1 -- 172.21.15.159:0/2109518917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70c157b50 con 0x7fc70c0cf980 2024-04-04T06:05:53.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.001+0000 7fc703e00640 1 -- 172.21.15.159:0/2109518917 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc0447e0 con 0x7fc70c0cf980 2024-04-04T06:05:53.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.001+0000 7fc712600640 1 -- 172.21.15.159:0/2109518917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70c158110 con 0x7fc70c0cf980 2024-04-04T06:05:53.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.003+0000 7fc712600640 1 -- 172.21.15.159:0/2109518917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc70c0685c0 con 0x7fc70c0cf980 2024-04-04T06:05:53.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.003+0000 7fc703e00640 1 -- 172.21.15.159:0/2109518917 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc6fc04b060 con 0x7fc70c0cf980 2024-04-04T06:05:53.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.004+0000 7fc703e00640 1 --2- 172.21.15.159:0/2109518917 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc6e806c8f0 0x7fc6e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:53.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.004+0000 7fc703e00640 1 -- 172.21.15.159:0/2109518917 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc6fc0add90 con 0x7fc70c0cf980 2024-04-04T06:05:53.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.005+0000 7fc702a00640 1 --2- 172.21.15.159:0/2109518917 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc6e806c8f0 0x7fc6e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:54.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.006+0000 7fc702a00640 1 --2- 172.21.15.159:0/2109518917 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc6e806c8f0 0x7fc6e806edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4006fd0 tx=0x7fc6f4006bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:54.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.009+0000 7fc703e00640 1 -- 172.21.15.159:0/2109518917 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc6fc03f070 con 0x7fc70c0cf980 2024-04-04T06:05:54.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.194+0000 7fc712600640 1 -- 172.21.15.159:0/2109518917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc70c156dc0 con 0x7fc70c0cf980 2024-04-04T06:05:54.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.197+0000 7fc703e00640 1 -- 172.21.15.159:0/2109518917 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16630 (secure 0 0 0) 0x7fc6fc02f320 con 0x7fc70c0cf980 2024-04-04T06:05:54.193 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:05:54.193 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":20,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:05:52.801141+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:05:51.297293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-04T06:05:51.770792+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:05:54.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.200+0000 7fc712600640 1 -- 172.21.15.159:0/2109518917 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc6e806c8f0 msgr2=0x7fc6e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:54.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.200+0000 7fc712600640 1 --2- 172.21.15.159:0/2109518917 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc6e806c8f0 0x7fc6e806edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4006fd0 tx=0x7fc6f4006bc0 comp rx=0 tx=0).stop 2024-04-04T06:05:54.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.200+0000 7fc712600640 1 -- 172.21.15.159:0/2109518917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc70c0cf980 msgr2=0x7fc70c14a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:54.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.200+0000 7fc712600640 1 --2- 172.21.15.159:0/2109518917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc70c0cf980 0x7fc70c14a890 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc000c00 tx=0x7fc6fc0046c0 comp rx=0 tx=0).stop 2024-04-04T06:05:54.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.200+0000 7fc712600640 1 -- 172.21.15.159:0/2109518917 shutdown_connections 2024-04-04T06:05:54.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.200+0000 7fc712600640 1 --2- 172.21.15.159:0/2109518917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc70c15c3b0 0x7fc70c157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:54.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.200+0000 7fc712600640 1 --2- 172.21.15.159:0/2109518917 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc6e806c8f0 0x7fc6e806edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:54.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.200+0000 7fc712600640 1 --2- 172.21.15.159:0/2109518917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc70c0d0d60 0x7fc70c156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:54.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.200+0000 7fc712600640 1 --2- 172.21.15.159:0/2109518917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc70c0cf980 0x7fc70c14a890 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:54.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.200+0000 7fc712600640 1 -- 172.21.15.159:0/2109518917 >> 172.21.15.159:0/2109518917 conn(0x7fc70c06b990 msgr2=0x7fc70c1521a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:54.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.200+0000 7fc712600640 1 -- 172.21.15.159:0/2109518917 shutdown_connections 2024-04-04T06:05:54.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.200+0000 7fc712600640 1 -- 172.21.15.159:0/2109518917 wait complete. 2024-04-04T06:05:54.213 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-04T06:05:54.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.428+0000 7f2026000640 1 Processor -- start 2024-04-04T06:05:54.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.428+0000 7f2026000640 1 -- start start 2024-04-04T06:05:54.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.430+0000 7f2026000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20200d1560 0x7f202006c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:54.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.430+0000 7f2026000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f202006cbc0 0x7f20200cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:54.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.430+0000 7f2026000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2020071060 con 0x7f202015e590 2024-04-04T06:05:54.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.430+0000 7f2026000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f20200d19a0 con 0x7f202006cbc0 2024-04-04T06:05:54.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.430+0000 7f2026000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f202006cff0 con 0x7f20200d1560 2024-04-04T06:05:54.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.430+0000 7f201f400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f202015e590 0x7f20200b47d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43498/0 (socket says 172.21.15.159:43498) 2024-04-04T06:05:54.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.430+0000 7f201f400640 1 -- 172.21.15.159:0/2813206058 learned_addr learned my addr 172.21.15.159:0/2813206058 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:05:54.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.431+0000 7f201ea00640 1 --2- 172.21.15.159:0/2813206058 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20200d1560 0x7f202006c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:54.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.431+0000 7f201fe00640 1 --2- 172.21.15.159:0/2813206058 >> v2:172.21.15.178:3300/0 conn(0x7f202006cbc0 0x7f20200cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:54.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.431+0000 7f201e000640 1 -- 172.21.15.159:0/2813206058 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3274196664 0 0) 0x7f2020071060 con 0x7f202015e590 2024-04-04T06:05:54.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.431+0000 7f201e000640 1 -- 172.21.15.159:0/2813206058 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2004003520 con 0x7f202015e590 2024-04-04T06:05:54.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.431+0000 7f201e000640 1 -- 172.21.15.159:0/2813206058 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1972471694 0 0) 0x7f2004003520 con 0x7f202015e590 2024-04-04T06:05:54.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.431+0000 7f201e000640 1 -- 172.21.15.159:0/2813206058 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20200d1560 msgr2=0x7f202006c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:54.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.431+0000 7f201e000640 1 --2- 172.21.15.159:0/2813206058 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20200d1560 0x7f202006c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:54.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.431+0000 7f201e000640 1 -- 172.21.15.159:0/2813206058 >> v2:172.21.15.178:3300/0 conn(0x7f202006cbc0 msgr2=0x7f20200cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:54.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.431+0000 7f201e000640 1 --2- 172.21.15.159:0/2813206058 >> v2:172.21.15.178:3300/0 conn(0x7f202006cbc0 0x7f20200cfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:54.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.431+0000 7f201e000640 1 -- 172.21.15.159:0/2813206058 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2020071340 con 0x7f202015e590 2024-04-04T06:05:54.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.431+0000 7f201e000640 1 -- 172.21.15.159:0/2813206058 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f2008001670 con 0x7f202015e590 2024-04-04T06:05:54.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.431+0000 7f201e000640 1 -- 172.21.15.159:0/2813206058 >> v1:172.21.15.159:6789/0 conn(0x7f202015e590 legacy=0x7f20200b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:54.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.431+0000 7f201e000640 1 --2- 172.21.15.159:0/2813206058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20040039b0 0x7f2004003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:54.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.431+0000 7f201e000640 1 --2- 172.21.15.159:0/2813206058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2004004800 0x7f2004004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:54.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.431+0000 7f201e000640 1 --2- 172.21.15.159:0/2813206058 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20200d1560 0x7f20040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:54.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.431+0000 7f201e000640 1 -- 172.21.15.159:0/2813206058 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f20200d19a0 con 0x7f2004004800 2024-04-04T06:05:54.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.431+0000 7f201e000640 1 -- 172.21.15.159:0/2813206058 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f202006cff0 con 0x7f20040039b0 2024-04-04T06:05:54.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.431+0000 7f201e000640 1 -- 172.21.15.159:0/2813206058 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f200400b9c0 con 0x7f20200d1560 2024-04-04T06:05:54.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.432+0000 7f201ea00640 1 --2- 172.21.15.159:0/2813206058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2004004800 0x7f2004004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:54.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.432+0000 7f201fe00640 1 --2- 172.21.15.159:0/2813206058 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20200d1560 0x7f20040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:54.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.432+0000 7f201f400640 1 --2- 172.21.15.159:0/2813206058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20040039b0 0x7f2004003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:54.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.432+0000 7f201ea00640 1 -- 172.21.15.159:0/2813206058 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20200d1560 msgr2=0x7f20040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:54.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.432+0000 7f201ea00640 1 --2- 172.21.15.159:0/2813206058 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20200d1560 0x7f20040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:54.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.432+0000 7f201ea00640 1 -- 172.21.15.159:0/2813206058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20040039b0 msgr2=0x7f2004003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:54.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.432+0000 7f201ea00640 1 --2- 172.21.15.159:0/2813206058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20040039b0 0x7f2004003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:54.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.432+0000 7f201ea00640 1 -- 172.21.15.159:0/2813206058 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2004003520 con 0x7f2004004800 2024-04-04T06:05:54.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.432+0000 7f201fe00640 1 --2- 172.21.15.159:0/2813206058 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20200d1560 0x7f20040051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:05:54.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.433+0000 7f201ea00640 1 --2- 172.21.15.159:0/2813206058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2004004800 0x7f2004004c80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2010009fc0 tx=0x7f2004005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809927938b883a97 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:54.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.433+0000 7f201e000640 1 -- 172.21.15.159:0/2813206058 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010039070 con 0x7f2004004800 2024-04-04T06:05:54.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.433+0000 7f201e000640 1 -- 172.21.15.159:0/2813206058 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f201002cc20 con 0x7f2004004800 2024-04-04T06:05:54.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.433+0000 7f201e000640 1 -- 172.21.15.159:0/2813206058 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010009c10 con 0x7f2004004800 2024-04-04T06:05:54.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.434+0000 7f2026000640 1 -- 172.21.15.159:0/2813206058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2004004800 msgr2=0x7f2004004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:54.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.434+0000 7f2026000640 1 --2- 172.21.15.159:0/2813206058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2004004800 0x7f2004004c80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2010009fc0 tx=0x7f2004005830 comp rx=0 tx=0).stop 2024-04-04T06:05:54.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.434+0000 7f201fe00640 1 -- 172.21.15.159:0/2813206058 reap_dead start 2024-04-04T06:05:54.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.434+0000 7f2026000640 1 -- 172.21.15.159:0/2813206058 shutdown_connections 2024-04-04T06:05:54.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.434+0000 7f2026000640 1 -- 172.21.15.159:0/2813206058 >> 172.21.15.159:0/2813206058 conn(0x7f20200d2240 msgr2=0x7f20200d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:54.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.434+0000 7f2026000640 1 -- 172.21.15.159:0/2813206058 shutdown_connections 2024-04-04T06:05:54.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.434+0000 7f2026000640 1 -- 172.21.15.159:0/2813206058 wait complete. 2024-04-04T06:05:54.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.436+0000 7f2026000640 1 Processor -- start 2024-04-04T06:05:54.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.436+0000 7f2026000640 1 -- start start 2024-04-04T06:05:54.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.437+0000 7f2026000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2004004800 0x7f20200cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:54.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.437+0000 7f2026000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2020161920 0x7f20200b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:54.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.437+0000 7f2026000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20200cbdc0 0x7f20200b7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:54.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.437+0000 7f2026000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2020071be0 con 0x7f2004004800 2024-04-04T06:05:54.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.437+0000 7f2026000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2020071900 con 0x7f2020161920 2024-04-04T06:05:54.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.437+0000 7f2026000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f202015e970 con 0x7f20200cbdc0 2024-04-04T06:05:54.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.437+0000 7f201f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2004004800 0x7f20200cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:54.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.437+0000 7f201f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2004004800 0x7f20200cb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43222/0 (socket says 172.21.15.159:43222) 2024-04-04T06:05:54.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.437+0000 7f201f400640 1 -- 172.21.15.159:0/1121049820 learned_addr learned my addr 172.21.15.159:0/1121049820 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:54.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.438+0000 7f201fe00640 1 --2- 172.21.15.159:0/1121049820 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20200cbdc0 0x7f20200b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:54.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.438+0000 7f201ea00640 1 --2- 172.21.15.159:0/1121049820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2020161920 0x7f20200b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:54.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.438+0000 7f201f400640 1 -- 172.21.15.159:0/1121049820 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20200cbdc0 msgr2=0x7f20200b7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:54.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.438+0000 7f201f400640 1 --2- 172.21.15.159:0/1121049820 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20200cbdc0 0x7f20200b7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:54.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.438+0000 7f201f400640 1 -- 172.21.15.159:0/1121049820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2020161920 msgr2=0x7f20200b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:54.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.438+0000 7f201f400640 1 --2- 172.21.15.159:0/1121049820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2020161920 0x7f20200b6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:54.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.438+0000 7f201f400640 1 -- 172.21.15.159:0/1121049820 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20200b26d0 con 0x7f2004004800 2024-04-04T06:05:54.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.438+0000 7f201f400640 1 --2- 172.21.15.159:0/1121049820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2004004800 0x7f20200cb740 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f200800d700 tx=0x7f200800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:54.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.438+0000 7f2017400640 1 -- 172.21.15.159:0/1121049820 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2008019070 con 0x7f2004004800 2024-04-04T06:05:54.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.438+0000 7f2017400640 1 -- 172.21.15.159:0/1121049820 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20080074e0 con 0x7f2004004800 2024-04-04T06:05:54.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.438+0000 7f2017400640 1 -- 172.21.15.159:0/1121049820 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2008004a20 con 0x7f2004004800 2024-04-04T06:05:54.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.438+0000 7f2026000640 1 -- 172.21.15.159:0/1121049820 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20200b29b0 con 0x7f2004004800 2024-04-04T06:05:54.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.438+0000 7f2026000640 1 -- 172.21.15.159:0/1121049820 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20200b2210 con 0x7f2004004800 2024-04-04T06:05:54.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.440+0000 7f2026000640 1 -- 172.21.15.159:0/1121049820 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20200685c0 con 0x7f2004004800 2024-04-04T06:05:54.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.440+0000 7f2017400640 1 -- 172.21.15.159:0/1121049820 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2008004bc0 con 0x7f2004004800 2024-04-04T06:05:54.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.441+0000 7f2017400640 1 --2- 172.21.15.159:0/1121049820 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1ffc06c920 0x7f1ffc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:54.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.441+0000 7f2017400640 1 -- 172.21.15.159:0/1121049820 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f200808b4e0 con 0x7f2004004800 2024-04-04T06:05:54.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.442+0000 7f201ea00640 1 --2- 172.21.15.159:0/1121049820 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1ffc06c920 0x7f1ffc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:54.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.443+0000 7f201ea00640 1 --2- 172.21.15.159:0/1121049820 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1ffc06c920 0x7f1ffc06ede0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2010036000 tx=0x7f201002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:54.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.446+0000 7f2017400640 1 -- 172.21.15.159:0/1121049820 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f20080599c0 con 0x7f2004004800 2024-04-04T06:05:54.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:54.688+0000 7f2026000640 1 -- 172.21.15.159:0/1121049820 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2020162f30 con 0x7f2004004800 2024-04-04T06:05:55.304 INFO:tasks.ceph.mds.b.smithi178.stderr:2024-04-04T06:05:55.309+0000 7f36b5000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:05:55.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.367+0000 7f2017400640 1 -- 172.21.15.159:0/1121049820 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f20080592c0 con 0x7f2004004800 2024-04-04T06:05:55.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.370+0000 7f2026000640 1 -- 172.21.15.159:0/1121049820 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1ffc06c920 msgr2=0x7f1ffc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:55.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.370+0000 7f2026000640 1 --2- 172.21.15.159:0/1121049820 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1ffc06c920 0x7f1ffc06ede0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2010036000 tx=0x7f201002e000 comp rx=0 tx=0).stop 2024-04-04T06:05:55.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.370+0000 7f2026000640 1 -- 172.21.15.159:0/1121049820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2004004800 msgr2=0x7f20200cb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:55.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.370+0000 7f2026000640 1 --2- 172.21.15.159:0/1121049820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2004004800 0x7f20200cb740 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f200800d700 tx=0x7f200800dbc0 comp rx=0 tx=0).stop 2024-04-04T06:05:55.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.371+0000 7f2026000640 1 -- 172.21.15.159:0/1121049820 shutdown_connections 2024-04-04T06:05:55.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.371+0000 7f2026000640 1 --2- 172.21.15.159:0/1121049820 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20200cbdc0 0x7f20200b7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:55.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.371+0000 7f2026000640 1 --2- 172.21.15.159:0/1121049820 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1ffc06c920 0x7f1ffc06ede0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:55.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.371+0000 7f2026000640 1 --2- 172.21.15.159:0/1121049820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2020161920 0x7f20200b6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:55.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.371+0000 7f2026000640 1 --2- 172.21.15.159:0/1121049820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2004004800 0x7f20200cb740 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:55.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.371+0000 7f2026000640 1 -- 172.21.15.159:0/1121049820 >> 172.21.15.159:0/1121049820 conn(0x7f20200d2240 msgr2=0x7f20200cd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:55.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.371+0000 7f2026000640 1 -- 172.21.15.159:0/1121049820 shutdown_connections 2024-04-04T06:05:55.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.371+0000 7f2026000640 1 -- 172.21.15.159:0/1121049820 wait complete. 2024-04-04T06:05:55.391 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-04T06:05:55.391 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-04T06:05:55.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.608+0000 7f77dd000640 1 Processor -- start 2024-04-04T06:05:55.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.608+0000 7f77dd000640 1 -- start start 2024-04-04T06:05:55.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.609+0000 7f77dd000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f77d80d0e70 0x7f77d80cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:55.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.609+0000 7f77dd000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77d815f6e0 0x7f77d80d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:55.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.609+0000 7f77dd000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77d80d1790 con 0x7f77d81621d0 2024-04-04T06:05:55.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.609+0000 7f77dd000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f77d80cf6c0 con 0x7f77d80d0e70 2024-04-04T06:05:55.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.609+0000 7f77dd000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f77d80cf9a0 con 0x7f77d815f6e0 2024-04-04T06:05:55.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.609+0000 7f77d5e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f77d80d0e70 0x7f77d80cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:55.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.609+0000 7f77d6800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f77d81621d0 0x7f77d8162660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43510/0 (socket says 172.21.15.159:43510) 2024-04-04T06:05:55.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.610+0000 7f77d6800640 1 -- 172.21.15.159:0/835986788 learned_addr learned my addr 172.21.15.159:0/835986788 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:05:55.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.610+0000 7f77d5400640 1 --2- 172.21.15.159:0/835986788 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77d815f6e0 0x7f77d80d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:55.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.610+0000 7f77d5e00640 1 -- 172.21.15.159:0/835986788 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77d815f6e0 msgr2=0x7f77d80d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:55.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.610+0000 7f77d5e00640 1 --2- 172.21.15.159:0/835986788 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77d815f6e0 0x7f77d80d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:55.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.610+0000 7f77d5e00640 1 -- 172.21.15.159:0/835986788 >> v1:172.21.15.159:6789/0 conn(0x7f77d81621d0 legacy=0x7f77d8162660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:55.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.610+0000 7f77d5e00640 1 -- 172.21.15.159:0/835986788 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d80cfc80 con 0x7f77d80d0e70 2024-04-04T06:05:55.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.611+0000 7f77d5e00640 1 --2- 172.21.15.159:0/835986788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d80d0e70 0x7f77d80cf050 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f77c80097a0 tx=0x7f77c802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9bf2addc4496ea server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:55.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.611+0000 7f77d4a00640 1 -- 172.21.15.159:0/835986788 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8031780 con 0x7f77d80d0e70 2024-04-04T06:05:55.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.611+0000 7f77d4a00640 1 -- 172.21.15.159:0/835986788 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c8037070 con 0x7f77d80d0e70 2024-04-04T06:05:55.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.611+0000 7f77d4a00640 1 -- 172.21.15.159:0/835986788 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8031b40 con 0x7f77d80d0e70 2024-04-04T06:05:55.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.612+0000 7f77dd000640 1 -- 172.21.15.159:0/835986788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d80d0e70 msgr2=0x7f77d80cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:55.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.612+0000 7f77dd000640 1 --2- 172.21.15.159:0/835986788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d80d0e70 0x7f77d80cf050 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f77c80097a0 tx=0x7f77c802b820 comp rx=0 tx=0).stop 2024-04-04T06:05:55.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.612+0000 7f77dd000640 1 -- 172.21.15.159:0/835986788 shutdown_connections 2024-04-04T06:05:55.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.612+0000 7f77dd000640 1 --2- 172.21.15.159:0/835986788 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77d815f6e0 0x7f77d80d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:55.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.612+0000 7f77dd000640 1 --2- 172.21.15.159:0/835986788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d80d0e70 0x7f77d80cf050 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:55.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.612+0000 7f77dd000640 1 -- 172.21.15.159:0/835986788 >> 172.21.15.159:0/835986788 conn(0x7f77d808bfb0 msgr2=0x7f77d80d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:55.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.612+0000 7f77dd000640 1 -- 172.21.15.159:0/835986788 shutdown_connections 2024-04-04T06:05:55.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.612+0000 7f77dd000640 1 -- 172.21.15.159:0/835986788 wait complete. 2024-04-04T06:05:55.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.614+0000 7f77dd000640 1 Processor -- start 2024-04-04T06:05:55.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.614+0000 7f77dd000640 1 -- start start 2024-04-04T06:05:55.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.615+0000 7f77dd000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d80d0e70 0x7f77d809e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:55.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.615+0000 7f77dd000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77d815f6e0 0x7f77d815a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:55.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.615+0000 7f77dd000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77d81621d0 0x7f77d80b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:55.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.615+0000 7f77dd000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f77d815fb40 con 0x7f77d80d0e70 2024-04-04T06:05:55.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.615+0000 7f77dd000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f77d80d0240 con 0x7f77d81621d0 2024-04-04T06:05:55.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.615+0000 7f77dd000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f77d815e910 con 0x7f77d815f6e0 2024-04-04T06:05:55.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.615+0000 7f77d5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d80d0e70 0x7f77d809e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:55.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.615+0000 7f77d5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d80d0e70 0x7f77d809e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43248/0 (socket says 172.21.15.159:43248) 2024-04-04T06:05:55.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.615+0000 7f77d5e00640 1 -- 172.21.15.159:0/2056282310 learned_addr learned my addr 172.21.15.159:0/2056282310 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:55.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.615+0000 7f77d5400640 1 --2- 172.21.15.159:0/2056282310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77d815f6e0 0x7f77d815a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:55.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.615+0000 7f77d6800640 1 --2- 172.21.15.159:0/2056282310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77d81621d0 0x7f77d80b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:55.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.616+0000 7f77d5e00640 1 -- 172.21.15.159:0/2056282310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77d815f6e0 msgr2=0x7f77d815a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:55.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.616+0000 7f77d5e00640 1 --2- 172.21.15.159:0/2056282310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77d815f6e0 0x7f77d815a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:55.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.616+0000 7f77d5e00640 1 -- 172.21.15.159:0/2056282310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77d81621d0 msgr2=0x7f77d80b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:55.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.616+0000 7f77d5e00640 1 --2- 172.21.15.159:0/2056282310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77d81621d0 0x7f77d80b7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:55.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.616+0000 7f77d5e00640 1 -- 172.21.15.159:0/2056282310 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d80cc720 con 0x7f77d80d0e70 2024-04-04T06:05:55.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.616+0000 7f77d5400640 1 --2- 172.21.15.159:0/2056282310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77d815f6e0 0x7f77d815a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:05:55.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.616+0000 7f77d5e00640 1 --2- 172.21.15.159:0/2056282310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d80d0e70 0x7f77d809e140 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f77c80098c0 tx=0x7f77c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:55.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.616+0000 7f77ce000640 1 -- 172.21.15.159:0/2056282310 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8032820 con 0x7f77d80d0e70 2024-04-04T06:05:55.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.616+0000 7f77ce000640 1 -- 172.21.15.159:0/2056282310 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c80329e0 con 0x7f77d80d0e70 2024-04-04T06:05:55.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.616+0000 7f77ce000640 1 -- 172.21.15.159:0/2056282310 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8032dd0 con 0x7f77d80d0e70 2024-04-04T06:05:55.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.616+0000 7f77dd000640 1 -- 172.21.15.159:0/2056282310 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d80cca00 con 0x7f77d80d0e70 2024-04-04T06:05:55.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.616+0000 7f77dd000640 1 -- 172.21.15.159:0/2056282310 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d80ccfc0 con 0x7f77d80d0e70 2024-04-04T06:05:55.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.618+0000 7f77dd000640 1 -- 172.21.15.159:0/2056282310 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d8068cd0 con 0x7f77d80d0e70 2024-04-04T06:05:55.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.618+0000 7f77ce000640 1 -- 172.21.15.159:0/2056282310 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77c804e440 con 0x7f77d80d0e70 2024-04-04T06:05:55.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.619+0000 7f77ce000640 1 --2- 172.21.15.159:0/2056282310 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f77b406c920 0x7f77b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:55.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.619+0000 7f77ce000640 1 -- 172.21.15.159:0/2056282310 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f77c80ad980 con 0x7f77d80d0e70 2024-04-04T06:05:55.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.620+0000 7f77d5400640 1 --2- 172.21.15.159:0/2056282310 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f77b406c920 0x7f77b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:55.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.621+0000 7f77d5400640 1 --2- 172.21.15.159:0/2056282310 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f77b406c920 0x7f77b406ede0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f77d80ba220 tx=0x7f77c0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:55.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.624+0000 7f77ce000640 1 -- 172.21.15.159:0/2056282310 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f77c8040070 con 0x7f77d80d0e70 2024-04-04T06:05:55.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:55.864+0000 7f77dd000640 1 -- 172.21.15.159:0/2056282310 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f77d80d1b80 con 0x7f77d80d0e70 2024-04-04T06:05:56.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.370+0000 7f77ce000640 1 -- 172.21.15.159:0/2056282310 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f77c802f050 con 0x7f77d80d0e70 2024-04-04T06:05:56.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.373+0000 7f77dd000640 1 -- 172.21.15.159:0/2056282310 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f77b406c920 msgr2=0x7f77b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.373+0000 7f77dd000640 1 --2- 172.21.15.159:0/2056282310 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f77b406c920 0x7f77b406ede0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f77d80ba220 tx=0x7f77c0008040 comp rx=0 tx=0).stop 2024-04-04T06:05:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.373+0000 7f77dd000640 1 -- 172.21.15.159:0/2056282310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d80d0e70 msgr2=0x7f77d809e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.373+0000 7f77dd000640 1 --2- 172.21.15.159:0/2056282310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d80d0e70 0x7f77d809e140 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f77c80098c0 tx=0x7f77c80046c0 comp rx=0 tx=0).stop 2024-04-04T06:05:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.373+0000 7f77dd000640 1 -- 172.21.15.159:0/2056282310 shutdown_connections 2024-04-04T06:05:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.373+0000 7f77dd000640 1 --2- 172.21.15.159:0/2056282310 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f77b406c920 0x7f77b406ede0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.374+0000 7f77dd000640 1 --2- 172.21.15.159:0/2056282310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77d81621d0 0x7f77d80b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.374+0000 7f77dd000640 1 --2- 172.21.15.159:0/2056282310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77d815f6e0 0x7f77d815a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.374+0000 7f77dd000640 1 --2- 172.21.15.159:0/2056282310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d80d0e70 0x7f77d809e140 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.374+0000 7f77dd000640 1 -- 172.21.15.159:0/2056282310 >> 172.21.15.159:0/2056282310 conn(0x7f77d808bfb0 msgr2=0x7f77d815eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.374+0000 7f77dd000640 1 -- 172.21.15.159:0/2056282310 shutdown_connections 2024-04-04T06:05:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.374+0000 7f77dd000640 1 -- 172.21.15.159:0/2056282310 wait complete. 2024-04-04T06:05:56.390 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-04T06:05:56.390 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:05:56.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.604+0000 7f7ce9200640 1 Processor -- start 2024-04-04T06:05:56.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.604+0000 7f7ce9200640 1 -- start start 2024-04-04T06:05:56.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.605+0000 7f7ce9200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ce41442c0 0x7f7ce40ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:56.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.605+0000 7f7ce9200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7ce40cefe0 0x7f7ce40cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:56.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.605+0000 7f7ce9200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ce4071460 con 0x7f7ce40d00d0 2024-04-04T06:05:56.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.605+0000 7f7ce9200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f7ce4071d20 con 0x7f7ce40cefe0 2024-04-04T06:05:56.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.605+0000 7f7ce9200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7ce40cf420 con 0x7f7ce41442c0 2024-04-04T06:05:56.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.605+0000 7f7ce2a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7ce40cefe0 0x7f7ce40cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:56.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.606+0000 7f7ce2a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7ce40cefe0 0x7f7ce40cfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43250/0 (socket says 172.21.15.159:43250) 2024-04-04T06:05:56.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.606+0000 7f7ce2a00640 1 -- 172.21.15.159:0/3340597302 learned_addr learned my addr 172.21.15.159:0/3340597302 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:56.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.606+0000 7f7ce3400640 1 --2- 172.21.15.159:0/3340597302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ce41442c0 0x7f7ce40ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:56.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.606+0000 7f7ce2a00640 1 -- 172.21.15.159:0/3340597302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ce41442c0 msgr2=0x7f7ce40ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:56.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.606+0000 7f7ce2a00640 1 --2- 172.21.15.159:0/3340597302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ce41442c0 0x7f7ce40ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:56.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.606+0000 7f7ce2a00640 1 -- 172.21.15.159:0/3340597302 >> v1:172.21.15.178:6789/0 conn(0x7f7ce40d00d0 legacy=0x7f7ce406e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:56.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.606+0000 7f7ce2a00640 1 -- 172.21.15.159:0/3340597302 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce40721b0 con 0x7f7ce40cefe0 2024-04-04T06:05:56.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.607+0000 7f7ce2a00640 1 --2- 172.21.15.159:0/3340597302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ce40cefe0 0x7f7ce40cfb90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0009870 tx=0x7f7cd002db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ecf75759113b75e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:56.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.607+0000 7f7ce2000640 1 -- 172.21.15.159:0/3340597302 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd003b070 con 0x7f7ce40cefe0 2024-04-04T06:05:56.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.607+0000 7f7ce2000640 1 -- 172.21.15.159:0/3340597302 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cd002e940 con 0x7f7ce40cefe0 2024-04-04T06:05:56.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.607+0000 7f7ce2000640 1 -- 172.21.15.159:0/3340597302 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd002ed10 con 0x7f7ce40cefe0 2024-04-04T06:05:56.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.608+0000 7f7ce9200640 1 -- 172.21.15.159:0/3340597302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ce40cefe0 msgr2=0x7f7ce40cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:56.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.608+0000 7f7ce9200640 1 --2- 172.21.15.159:0/3340597302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ce40cefe0 0x7f7ce40cfb90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0009870 tx=0x7f7cd002db90 comp rx=0 tx=0).stop 2024-04-04T06:05:56.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.608+0000 7f7ce9200640 1 -- 172.21.15.159:0/3340597302 shutdown_connections 2024-04-04T06:05:56.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.608+0000 7f7ce9200640 1 --2- 172.21.15.159:0/3340597302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ce40cefe0 0x7f7ce40cfb90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:56.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.608+0000 7f7ce9200640 1 --2- 172.21.15.159:0/3340597302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ce41442c0 0x7f7ce40ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:56.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.608+0000 7f7ce9200640 1 -- 172.21.15.159:0/3340597302 >> 172.21.15.159:0/3340597302 conn(0x7f7ce40ccad0 msgr2=0x7f7ce40cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:56.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.608+0000 7f7ce9200640 1 -- 172.21.15.159:0/3340597302 shutdown_connections 2024-04-04T06:05:56.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.608+0000 7f7ce9200640 1 -- 172.21.15.159:0/3340597302 wait complete. 2024-04-04T06:05:56.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.610+0000 7f7ce9200640 1 Processor -- start 2024-04-04T06:05:56.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.610+0000 7f7ce9200640 1 -- start start 2024-04-04T06:05:56.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.611+0000 7f7ce9200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ce40cefe0 0x7f7ce413e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:56.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.611+0000 7f7ce9200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ce40d00d0 0x7f7ce415a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:56.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.611+0000 7f7ce9200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ce41442c0 0x7f7ce41401a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:56.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.611+0000 7f7ce9200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce40d0770 con 0x7f7ce41442c0 2024-04-04T06:05:56.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.611+0000 7f7ce9200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce40719b0 con 0x7f7ce40cefe0 2024-04-04T06:05:56.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.611+0000 7f7ce9200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7ce41446a0 con 0x7f7ce40d00d0 2024-04-04T06:05:56.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.611+0000 7f7ce3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ce41442c0 0x7f7ce41401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:56.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.611+0000 7f7ce3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ce41442c0 0x7f7ce41401a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43264/0 (socket says 172.21.15.159:43264) 2024-04-04T06:05:56.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.611+0000 7f7ce3e00640 1 -- 172.21.15.159:0/633044365 learned_addr learned my addr 172.21.15.159:0/633044365 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:56.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.611+0000 7f7ce2a00640 1 --2- 172.21.15.159:0/633044365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ce40d00d0 0x7f7ce415a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:56.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.611+0000 7f7ce3400640 1 --2- 172.21.15.159:0/633044365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ce40cefe0 0x7f7ce413e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:56.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.612+0000 7f7ce3e00640 1 -- 172.21.15.159:0/633044365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ce40d00d0 msgr2=0x7f7ce415a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:56.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.612+0000 7f7ce3e00640 1 --2- 172.21.15.159:0/633044365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ce40d00d0 0x7f7ce415a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:56.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.612+0000 7f7ce3e00640 1 -- 172.21.15.159:0/633044365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ce40cefe0 msgr2=0x7f7ce413e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:56.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.612+0000 7f7ce3e00640 1 --2- 172.21.15.159:0/633044365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ce40cefe0 0x7f7ce413e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:56.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.612+0000 7f7ce3e00640 1 -- 172.21.15.159:0/633044365 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce413ea70 con 0x7f7ce41442c0 2024-04-04T06:05:56.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.612+0000 7f7ce3e00640 1 --2- 172.21.15.159:0/633044365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ce41442c0 0x7f7ce41401a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00c780 tx=0x7f7cdc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:56.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.612+0000 7f7cdb400640 1 -- 172.21.15.159:0/633044365 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc004b60 con 0x7f7ce41442c0 2024-04-04T06:05:56.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.612+0000 7f7cdb400640 1 -- 172.21.15.159:0/633044365 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cdc004d00 con 0x7f7ce41442c0 2024-04-04T06:05:56.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.612+0000 7f7cdb400640 1 -- 172.21.15.159:0/633044365 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc00aa90 con 0x7f7ce41442c0 2024-04-04T06:05:56.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.612+0000 7f7ce9200640 1 -- 172.21.15.159:0/633044365 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce413d1f0 con 0x7f7ce41442c0 2024-04-04T06:05:56.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.613+0000 7f7ce9200640 1 -- 172.21.15.159:0/633044365 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce413d770 con 0x7f7ce41442c0 2024-04-04T06:05:56.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.614+0000 7f7ce9200640 1 -- 172.21.15.159:0/633044365 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ce4075940 con 0x7f7ce41442c0 2024-04-04T06:05:56.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.615+0000 7f7cdb400640 1 -- 172.21.15.159:0/633044365 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7cdc00ac30 con 0x7f7ce41442c0 2024-04-04T06:05:56.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.615+0000 7f7cdb400640 1 --2- 172.21.15.159:0/633044365 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7cc406c8f0 0x7f7cc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:56.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.616+0000 7f7cdb400640 1 -- 172.21.15.159:0/633044365 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7cdc08bca0 con 0x7f7ce41442c0 2024-04-04T06:05:56.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.616+0000 7f7ce3400640 1 --2- 172.21.15.159:0/633044365 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7cc406c8f0 0x7f7cc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:56.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.617+0000 7f7ce3400640 1 --2- 172.21.15.159:0/633044365 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7cc406c8f0 0x7f7cc406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40079a0 tx=0x7f7cd4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:56.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.620+0000 7f7cdb400640 1 -- 172.21.15.159:0/633044365 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7cdc020070 con 0x7f7ce41442c0 2024-04-04T06:05:56.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.858+0000 7f7ce9200640 1 -- 172.21.15.159:0/633044365 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ce40b2820 con 0x7f7ce41442c0 2024-04-04T06:05:56.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.859+0000 7f7cdb400640 1 -- 172.21.15.159:0/633044365 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5184 (secure 0 0 0) 0x7f7cdc059a80 con 0x7f7ce41442c0 2024-04-04T06:05:56.855 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:05:56.855 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/789759868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":789759868},{"type":"v1","addr":"172.21.15.178:6836","nonce":789759868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2438288045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2438288045},{"type":"v1","addr":"172.21.15.159:6833","nonce":2438288045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/1705205607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1705205607},{"type":"v1","addr":"172.21.15.159:6835","nonce":1705205607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:05:52.801529+0000","modified":"2024-04-04T06:05:55.866207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6837/3362141613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":3362141613},{"type":"v1","addr":"172.21.15.178:6837","nonce":3362141613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-04T06:05:56.855 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 8 2024-04-04T06:05:56.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.862+0000 7f7ce9200640 1 -- 172.21.15.159:0/633044365 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7cc406c8f0 msgr2=0x7f7cc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:56.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.862+0000 7f7ce9200640 1 --2- 172.21.15.159:0/633044365 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7cc406c8f0 0x7f7cc406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40079a0 tx=0x7f7cd4008040 comp rx=0 tx=0).stop 2024-04-04T06:05:56.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.862+0000 7f7ce9200640 1 -- 172.21.15.159:0/633044365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ce41442c0 msgr2=0x7f7ce41401a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:56.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.862+0000 7f7ce9200640 1 --2- 172.21.15.159:0/633044365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ce41442c0 0x7f7ce41401a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00c780 tx=0x7f7cdc00cc40 comp rx=0 tx=0).stop 2024-04-04T06:05:56.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.862+0000 7f7ce9200640 1 -- 172.21.15.159:0/633044365 shutdown_connections 2024-04-04T06:05:56.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.862+0000 7f7ce9200640 1 --2- 172.21.15.159:0/633044365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ce41442c0 0x7f7ce41401a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:56.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.862+0000 7f7ce9200640 1 --2- 172.21.15.159:0/633044365 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ce40d00d0 0x7f7ce415a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:56.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.862+0000 7f7ce9200640 1 --2- 172.21.15.159:0/633044365 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7cc406c8f0 0x7f7cc406edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:56.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.862+0000 7f7ce9200640 1 --2- 172.21.15.159:0/633044365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ce40cefe0 0x7f7ce413e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:56.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.862+0000 7f7ce9200640 1 -- 172.21.15.159:0/633044365 >> 172.21.15.159:0/633044365 conn(0x7f7ce40ccad0 msgr2=0x7f7ce4141d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:56.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.862+0000 7f7ce9200640 1 -- 172.21.15.159:0/633044365 shutdown_connections 2024-04-04T06:05:56.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:56.862+0000 7f7ce9200640 1 -- 172.21.15.159:0/633044365 wait complete. 2024-04-04T06:05:56.876 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:05:57.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.088+0000 7f7639a00640 1 Processor -- start 2024-04-04T06:05:57.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.088+0000 7f7639a00640 1 -- start start 2024-04-04T06:05:57.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.090+0000 7f7639a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f763415aa90 0x7f763415b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:57.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.090+0000 7f7639a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f76340717f0 0x7f76340b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:57.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.090+0000 7f7639a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7634072100 0x7f7634072570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:57.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.090+0000 7f7639a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f76340b5190 con 0x7f76340717f0 2024-04-04T06:05:57.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.090+0000 7f7639a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7634072ab0 con 0x7f7634072100 2024-04-04T06:05:57.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.090+0000 7f7639a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f76340b4240 con 0x7f763415aa90 2024-04-04T06:05:57.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.090+0000 7f762a000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f76340717f0 0x7f76340b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:57.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.090+0000 7f762a000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f76340717f0 0x7f76340b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43276/0 (socket says 172.21.15.159:43276) 2024-04-04T06:05:57.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.090+0000 7f762a000640 1 -- 172.21.15.159:0/234089061 learned_addr learned my addr 172.21.15.159:0/234089061 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:57.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.091+0000 7f7632a00640 1 --2- 172.21.15.159:0/234089061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f763415aa90 0x7f763415b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:57.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.091+0000 7f7633400640 1 --2- 172.21.15.159:0/234089061 >> v2:172.21.15.178:3300/0 conn(0x7f7634072100 0x7f7634072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:57.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.091+0000 7f762a000640 1 -- 172.21.15.159:0/234089061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f763415aa90 msgr2=0x7f763415b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:57.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.091+0000 7f762a000640 1 --2- 172.21.15.159:0/234089061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f763415aa90 0x7f763415b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.091+0000 7f762a000640 1 -- 172.21.15.159:0/234089061 >> v2:172.21.15.178:3300/0 conn(0x7f7634072100 msgr2=0x7f7634072570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:57.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.091+0000 7f762a000640 1 --2- 172.21.15.159:0/234089061 >> v2:172.21.15.178:3300/0 conn(0x7f7634072100 0x7f7634072570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.091+0000 7f762a000640 1 -- 172.21.15.159:0/234089061 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763415bee0 con 0x7f76340717f0 2024-04-04T06:05:57.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.092+0000 7f762a000640 1 --2- 172.21.15.159:0/234089061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76340717f0 0x7f76340b46d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7634072d30 tx=0x7f761c02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee8cd71106cd8ab server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:57.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.092+0000 7f7632000640 1 -- 172.21.15.159:0/234089061 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c03b070 con 0x7f76340717f0 2024-04-04T06:05:57.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.092+0000 7f7632000640 1 -- 172.21.15.159:0/234089061 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f761c036440 con 0x7f76340717f0 2024-04-04T06:05:57.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.092+0000 7f7632000640 1 -- 172.21.15.159:0/234089061 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c036810 con 0x7f76340717f0 2024-04-04T06:05:57.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.093+0000 7f7639a00640 1 -- 172.21.15.159:0/234089061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76340717f0 msgr2=0x7f76340b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:57.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.093+0000 7f7639a00640 1 --2- 172.21.15.159:0/234089061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76340717f0 0x7f76340b46d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7634072d30 tx=0x7f761c02e0e0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.093+0000 7f7639a00640 1 -- 172.21.15.159:0/234089061 shutdown_connections 2024-04-04T06:05:57.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.093+0000 7f7639a00640 1 --2- 172.21.15.159:0/234089061 >> v2:172.21.15.178:3300/0 conn(0x7f7634072100 0x7f7634072570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.093+0000 7f7639a00640 1 --2- 172.21.15.159:0/234089061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76340717f0 0x7f76340b46d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.093+0000 7f7639a00640 1 --2- 172.21.15.159:0/234089061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f763415aa90 0x7f763415b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.093+0000 7f7639a00640 1 -- 172.21.15.159:0/234089061 >> 172.21.15.159:0/234089061 conn(0x7f763406b990 msgr2=0x7f763406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:57.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.093+0000 7f7639a00640 1 -- 172.21.15.159:0/234089061 shutdown_connections 2024-04-04T06:05:57.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.093+0000 7f7639a00640 1 -- 172.21.15.159:0/234089061 wait complete. 2024-04-04T06:05:57.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.095+0000 7f7639a00640 1 Processor -- start 2024-04-04T06:05:57.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.095+0000 7f7639a00640 1 -- start start 2024-04-04T06:05:57.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.096+0000 7f7639a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76340717f0 0x7f76341573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:57.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.096+0000 7f7639a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7634072100 0x7f7634158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:57.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.096+0000 7f7639a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f763415aa90 0x7f76341f0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:57.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.096+0000 7f7639a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f76340b3ed0 con 0x7f7634072100 2024-04-04T06:05:57.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.096+0000 7f7639a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f76340b3bf0 con 0x7f76340717f0 2024-04-04T06:05:57.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.096+0000 7f7639a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f76340b4c10 con 0x7f763415aa90 2024-04-04T06:05:57.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.096+0000 7f762a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7634072100 0x7f7634158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:57.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.096+0000 7f762a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7634072100 0x7f7634158c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43280/0 (socket says 172.21.15.159:43280) 2024-04-04T06:05:57.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.096+0000 7f762a000640 1 -- 172.21.15.159:0/3968608083 learned_addr learned my addr 172.21.15.159:0/3968608083 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:57.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.097+0000 7f7632a00640 1 --2- 172.21.15.159:0/3968608083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76340717f0 0x7f76341573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:57.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.097+0000 7f7633400640 1 --2- 172.21.15.159:0/3968608083 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f763415aa90 0x7f76341f0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:57.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.097+0000 7f762a000640 1 -- 172.21.15.159:0/3968608083 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f763415aa90 msgr2=0x7f76341f0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:57.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.097+0000 7f762a000640 1 --2- 172.21.15.159:0/3968608083 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f763415aa90 0x7f76341f0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.097+0000 7f762a000640 1 -- 172.21.15.159:0/3968608083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76340717f0 msgr2=0x7f76341573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:57.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.097+0000 7f762a000640 1 --2- 172.21.15.159:0/3968608083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76340717f0 0x7f76341573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.097+0000 7f762a000640 1 -- 172.21.15.159:0/3968608083 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76340cde60 con 0x7f7634072100 2024-04-04T06:05:57.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.097+0000 7f7633400640 1 --2- 172.21.15.159:0/3968608083 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f763415aa90 0x7f76341f0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:05:57.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.097+0000 7f762a000640 1 --2- 172.21.15.159:0/3968608083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7634072100 0x7f7634158c60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f761c02e120 tx=0x7f761c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:57.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.097+0000 7f762b400640 1 -- 172.21.15.159:0/3968608083 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c03b070 con 0x7f7634072100 2024-04-04T06:05:57.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.097+0000 7f762b400640 1 -- 172.21.15.159:0/3968608083 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f761c03d050 con 0x7f7634072100 2024-04-04T06:05:57.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.097+0000 7f7639a00640 1 -- 172.21.15.159:0/3968608083 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76340ce140 con 0x7f7634072100 2024-04-04T06:05:57.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.097+0000 7f762b400640 1 -- 172.21.15.159:0/3968608083 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c034a90 con 0x7f7634072100 2024-04-04T06:05:57.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.098+0000 7f7639a00640 1 -- 172.21.15.159:0/3968608083 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76340ce700 con 0x7f7634072100 2024-04-04T06:05:57.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.099+0000 7f7639a00640 1 -- 172.21.15.159:0/3968608083 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76340685c0 con 0x7f7634072100 2024-04-04T06:05:57.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.100+0000 7f762b400640 1 -- 172.21.15.159:0/3968608083 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f761c048060 con 0x7f7634072100 2024-04-04T06:05:57.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.101+0000 7f762b400640 1 --2- 172.21.15.159:0/3968608083 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f761006c8f0 0x7f761006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:57.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.101+0000 7f762b400640 1 -- 172.21.15.159:0/3968608083 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f761c0ad6f0 con 0x7f7634072100 2024-04-04T06:05:57.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.102+0000 7f7632a00640 1 --2- 172.21.15.159:0/3968608083 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f761006c8f0 0x7f761006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:57.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.103+0000 7f7632a00640 1 --2- 172.21.15.159:0/3968608083 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f761006c8f0 0x7f761006edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7624009a30 tx=0x7f7624008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:57.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.106+0000 7f762b400640 1 -- 172.21.15.159:0/3968608083 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f763415bee0 con 0x7f7634072100 2024-04-04T06:05:57.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.292+0000 7f7639a00640 1 -- 172.21.15.159:0/3968608083 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76341591a0 con 0x7f7634072100 2024-04-04T06:05:57.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.295+0000 7f762b400640 1 -- 172.21.15.159:0/3968608083 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f761c07b560 con 0x7f7634072100 2024-04-04T06:05:57.290 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:05:57.290 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":21,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:05:54.347275+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:05:51.297293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-04T06:05:51.770792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:05:57.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.298+0000 7f7639a00640 1 -- 172.21.15.159:0/3968608083 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f761006c8f0 msgr2=0x7f761006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:57.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.298+0000 7f7639a00640 1 --2- 172.21.15.159:0/3968608083 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f761006c8f0 0x7f761006edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7624009a30 tx=0x7f7624008040 comp rx=0 tx=0).stop 2024-04-04T06:05:57.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.298+0000 7f7639a00640 1 -- 172.21.15.159:0/3968608083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7634072100 msgr2=0x7f7634158c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:57.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.298+0000 7f7639a00640 1 --2- 172.21.15.159:0/3968608083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7634072100 0x7f7634158c60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f761c02e120 tx=0x7f761c0046c0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.298+0000 7f7639a00640 1 -- 172.21.15.159:0/3968608083 shutdown_connections 2024-04-04T06:05:57.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.298+0000 7f7639a00640 1 --2- 172.21.15.159:0/3968608083 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f763415aa90 0x7f76341f0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.298+0000 7f7639a00640 1 --2- 172.21.15.159:0/3968608083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7634072100 0x7f7634158c60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.298+0000 7f7639a00640 1 --2- 172.21.15.159:0/3968608083 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f761006c8f0 0x7f761006edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.298+0000 7f7639a00640 1 --2- 172.21.15.159:0/3968608083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76340717f0 0x7f76341573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.298+0000 7f7639a00640 1 -- 172.21.15.159:0/3968608083 >> 172.21.15.159:0/3968608083 conn(0x7f763406b990 msgr2=0x7f7634154ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:57.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.298+0000 7f7639a00640 1 -- 172.21.15.159:0/3968608083 shutdown_connections 2024-04-04T06:05:57.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.298+0000 7f7639a00640 1 -- 172.21.15.159:0/3968608083 wait complete. 2024-04-04T06:05:57.309 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-04T06:05:57.310 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:05:57.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.514+0000 7f5f27e00640 1 Processor -- start 2024-04-04T06:05:57.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.514+0000 7f5f27e00640 1 -- start start 2024-04-04T06:05:57.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.516+0000 7f5f27e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5f280cc0b0 0x7f5f2806dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:57.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.516+0000 7f5f27e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5f2815fc50 0x7f5f280cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:57.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.516+0000 7f5f27e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f2806e4f0 con 0x7f5f280cc490 2024-04-04T06:05:57.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.516+0000 7f5f27e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f5f2815c610 con 0x7f5f2815fc50 2024-04-04T06:05:57.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.516+0000 7f5f27e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f5f280b3150 con 0x7f5f280cc0b0 2024-04-04T06:05:57.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.516+0000 7f5f1fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5f2815fc50 0x7f5f280cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:57.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.517+0000 7f5f1fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5f2815fc50 0x7f5f280cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43286/0 (socket says 172.21.15.159:43286) 2024-04-04T06:05:57.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.517+0000 7f5f1fe00640 1 -- 172.21.15.159:0/929469037 learned_addr learned my addr 172.21.15.159:0/929469037 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:57.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.517+0000 7f5f26a00640 1 --2- 172.21.15.159:0/929469037 >> v2:172.21.15.178:3300/0 conn(0x7f5f280cc0b0 0x7f5f2806dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:57.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.517+0000 7f5f1fe00640 1 -- 172.21.15.159:0/929469037 >> v2:172.21.15.178:3300/0 conn(0x7f5f280cc0b0 msgr2=0x7f5f2806dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:57.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.517+0000 7f5f1fe00640 1 --2- 172.21.15.159:0/929469037 >> v2:172.21.15.178:3300/0 conn(0x7f5f280cc0b0 0x7f5f2806dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.517+0000 7f5f1fe00640 1 -- 172.21.15.159:0/929469037 >> v1:172.21.15.178:6789/0 conn(0x7f5f280cc490 legacy=0x7f5f2815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:57.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.517+0000 7f5f1fe00640 1 -- 172.21.15.159:0/929469037 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f281600e0 con 0x7f5f2815fc50 2024-04-04T06:05:57.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.518+0000 7f5f1fe00640 1 --2- 172.21.15.159:0/929469037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f2815fc50 0x7f5f280cfdc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5f280cfd10 tx=0x7f5f1002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69695ab561e0cd56 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:57.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.518+0000 7f5f26000640 1 -- 172.21.15.159:0/929469037 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10035020 con 0x7f5f2815fc50 2024-04-04T06:05:57.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.518+0000 7f5f26000640 1 -- 172.21.15.159:0/929469037 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f10031870 con 0x7f5f2815fc50 2024-04-04T06:05:57.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.518+0000 7f5f26000640 1 -- 172.21.15.159:0/929469037 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10031c40 con 0x7f5f2815fc50 2024-04-04T06:05:57.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.519+0000 7f5f27e00640 1 -- 172.21.15.159:0/929469037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f2815fc50 msgr2=0x7f5f280cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:57.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.519+0000 7f5f27e00640 1 --2- 172.21.15.159:0/929469037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f2815fc50 0x7f5f280cfdc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5f280cfd10 tx=0x7f5f1002b910 comp rx=0 tx=0).stop 2024-04-04T06:05:57.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.519+0000 7f5f27e00640 1 -- 172.21.15.159:0/929469037 shutdown_connections 2024-04-04T06:05:57.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.519+0000 7f5f27e00640 1 --2- 172.21.15.159:0/929469037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f2815fc50 0x7f5f280cfdc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.519+0000 7f5f27e00640 1 --2- 172.21.15.159:0/929469037 >> v2:172.21.15.178:3300/0 conn(0x7f5f280cc0b0 0x7f5f2806dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.519+0000 7f5f27e00640 1 -- 172.21.15.159:0/929469037 >> 172.21.15.159:0/929469037 conn(0x7f5f280ccad0 msgr2=0x7f5f280cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:57.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.519+0000 7f5f27e00640 1 -- 172.21.15.159:0/929469037 shutdown_connections 2024-04-04T06:05:57.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.519+0000 7f5f27e00640 1 -- 172.21.15.159:0/929469037 wait complete. 2024-04-04T06:05:57.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.521+0000 7f5f27e00640 1 Processor -- start 2024-04-04T06:05:57.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.522+0000 7f5f27e00640 1 -- start start 2024-04-04T06:05:57.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.523+0000 7f5f27e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f280cc0b0 0x7f5f281f7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:57.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.523+0000 7f5f27e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f280cc490 0x7f5f2815b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:57.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.523+0000 7f5f27e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f281f5570 0x7f5f2815b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:57.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.523+0000 7f5f27e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28160980 con 0x7f5f280cc490 2024-04-04T06:05:57.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.523+0000 7f5f27e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5f281606a0 con 0x7f5f280cc0b0 2024-04-04T06:05:57.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.523+0000 7f5f27e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5f280d0300 con 0x7f5f281f5570 2024-04-04T06:05:57.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.523+0000 7f5f1fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f280cc490 0x7f5f2815b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:57.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.523+0000 7f5f1fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f280cc490 0x7f5f2815b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43300/0 (socket says 172.21.15.159:43300) 2024-04-04T06:05:57.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.523+0000 7f5f1fe00640 1 -- 172.21.15.159:0/895687379 learned_addr learned my addr 172.21.15.159:0/895687379 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:57.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.523+0000 7f5f26a00640 1 --2- 172.21.15.159:0/895687379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f280cc0b0 0x7f5f281f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:57.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.523+0000 7f5f1fe00640 1 -- 172.21.15.159:0/895687379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f281f5570 msgr2=0x7f5f2815b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:57.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.523+0000 7f5f1fe00640 1 --2- 172.21.15.159:0/895687379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f281f5570 0x7f5f2815b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.523+0000 7f5f1fe00640 1 -- 172.21.15.159:0/895687379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f280cc0b0 msgr2=0x7f5f281f7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:57.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.523+0000 7f5f1fe00640 1 --2- 172.21.15.159:0/895687379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f280cc0b0 0x7f5f281f7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.523+0000 7f5f1fe00640 1 -- 172.21.15.159:0/895687379 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f280b1400 con 0x7f5f280cc490 2024-04-04T06:05:57.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.524+0000 7f5f1fe00640 1 --2- 172.21.15.159:0/895687379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f280cc490 0x7f5f2815b130 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5f10000c00 tx=0x7f5f100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:57.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.524+0000 7f5f1ea00640 1 -- 172.21.15.159:0/895687379 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10040070 con 0x7f5f280cc490 2024-04-04T06:05:57.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.524+0000 7f5f1ea00640 1 -- 172.21.15.159:0/895687379 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f1003a420 con 0x7f5f280cc490 2024-04-04T06:05:57.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.524+0000 7f5f27e00640 1 -- 172.21.15.159:0/895687379 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f280b16a0 con 0x7f5f280cc490 2024-04-04T06:05:57.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.524+0000 7f5f1ea00640 1 -- 172.21.15.159:0/895687379 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10035040 con 0x7f5f280cc490 2024-04-04T06:05:57.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.524+0000 7f5f27e00640 1 -- 172.21.15.159:0/895687379 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f280b1c60 con 0x7f5f280cc490 2024-04-04T06:05:57.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.525+0000 7f5f27e00640 1 -- 172.21.15.159:0/895687379 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f28075940 con 0x7f5f280cc490 2024-04-04T06:05:57.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.526+0000 7f5f1ea00640 1 -- 172.21.15.159:0/895687379 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f1004b060 con 0x7f5f280cc490 2024-04-04T06:05:57.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.527+0000 7f5f1ea00640 1 --2- 172.21.15.159:0/895687379 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5f0406c8f0 0x7f5f0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:57.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.528+0000 7f5f1ea00640 1 -- 172.21.15.159:0/895687379 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5f100adb20 con 0x7f5f280cc490 2024-04-04T06:05:57.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.528+0000 7f5f26a00640 1 --2- 172.21.15.159:0/895687379 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5f0406c8f0 0x7f5f0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:57.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.529+0000 7f5f26a00640 1 --2- 172.21.15.159:0/895687379 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5f0406c8f0 0x7f5f0406edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800a970 tx=0x7f5f18008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:57.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.531+0000 7f5f1ea00640 1 -- 172.21.15.159:0/895687379 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5f1007bf40 con 0x7f5f280cc490 2024-04-04T06:05:57.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.712+0000 7f5f27e00640 1 -- 172.21.15.159:0/895687379 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f2806f2a0 con 0x7f5f0406c8f0 2024-04-04T06:05:57.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.737+0000 7f5f1ea00640 1 -- 172.21.15.159:0/895687379 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242990 (secure 0 0 0) 0x7f5f2806f2a0 con 0x7f5f0406c8f0 2024-04-04T06:05:57.736 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:05:57.737 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:05:57.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.743+0000 7f5f27e00640 1 -- 172.21.15.159:0/895687379 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5f0406c8f0 msgr2=0x7f5f0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:57.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.743+0000 7f5f27e00640 1 --2- 172.21.15.159:0/895687379 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5f0406c8f0 0x7f5f0406edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800a970 tx=0x7f5f18008040 comp rx=0 tx=0).stop 2024-04-04T06:05:57.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.743+0000 7f5f27e00640 1 -- 172.21.15.159:0/895687379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f280cc490 msgr2=0x7f5f2815b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:57.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.743+0000 7f5f27e00640 1 --2- 172.21.15.159:0/895687379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f280cc490 0x7f5f2815b130 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5f10000c00 tx=0x7f5f100046c0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.743+0000 7f5f27e00640 1 -- 172.21.15.159:0/895687379 shutdown_connections 2024-04-04T06:05:57.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.743+0000 7f5f27e00640 1 --2- 172.21.15.159:0/895687379 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f281f5570 0x7f5f2815b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.743+0000 7f5f27e00640 1 --2- 172.21.15.159:0/895687379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f280cc490 0x7f5f2815b130 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.743+0000 7f5f27e00640 1 --2- 172.21.15.159:0/895687379 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5f0406c8f0 0x7f5f0406edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.743+0000 7f5f27e00640 1 --2- 172.21.15.159:0/895687379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f280cc0b0 0x7f5f281f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:57.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.743+0000 7f5f27e00640 1 -- 172.21.15.159:0/895687379 >> 172.21.15.159:0/895687379 conn(0x7f5f280ccad0 msgr2=0x7f5f2815ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:57.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.743+0000 7f5f27e00640 1 -- 172.21.15.159:0/895687379 shutdown_connections 2024-04-04T06:05:57.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:57.744+0000 7f5f27e00640 1 -- 172.21.15.159:0/895687379 wait complete. 2024-04-04T06:05:57.755 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-04T06:05:56.711698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590390,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":160,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":100,"acting":100,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235380,"kb_used_data":2580,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739340,"statfs":{"total":773094113280,"available":772853084160,"internally_reserved":0,"allocated":2641920,"data_stored":1638899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.392595"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365717+0000","last_change":"2024-04-04T06:05:53.373115+0000","last_active":"2024-04-04T06:05:54.365717+0000","last_peered":"2024-04-04T06:05:54.365717+0000","last_clean":"2024-04-04T06:05:54.365717+0000","last_became_active":"2024-04-04T06:05:53.372848+0000","last_became_peered":"2024-04-04T06:05:53.372848+0000","last_unstale":"2024-04-04T06:05:54.365717+0000","last_undegraded":"2024-04-04T06:05:54.365717+0000","last_fullsized":"2024-04-04T06:05:54.365717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:45:21.548886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367929+0000","last_change":"2024-04-04T06:05:53.387263+0000","last_active":"2024-04-04T06:05:54.367929+0000","last_peered":"2024-04-04T06:05:54.367929+0000","last_clean":"2024-04-04T06:05:54.367929+0000","last_became_active":"2024-04-04T06:05:53.387012+0000","last_became_peered":"2024-04-04T06:05:53.387012+0000","last_unstale":"2024-04-04T06:05:54.367929+0000","last_undegraded":"2024-04-04T06:05:54.367929+0000","last_fullsized":"2024-04-04T06:05:54.367929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:13.472940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367238+0000","last_change":"2024-04-04T06:05:53.364063+0000","last_active":"2024-04-04T06:05:54.367238+0000","last_peered":"2024-04-04T06:05:54.367238+0000","last_clean":"2024-04-04T06:05:54.367238+0000","last_became_active":"2024-04-04T06:05:53.363715+0000","last_became_peered":"2024-04-04T06:05:53.363715+0000","last_unstale":"2024-04-04T06:05:54.367238+0000","last_undegraded":"2024-04-04T06:05:54.367238+0000","last_fullsized":"2024-04-04T06:05:54.367238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:21.889411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.364261+0000","last_change":"2024-04-04T06:05:53.387499+0000","last_active":"2024-04-04T06:05:54.364261+0000","last_peered":"2024-04-04T06:05:54.364261+0000","last_clean":"2024-04-04T06:05:54.364261+0000","last_became_active":"2024-04-04T06:05:53.387187+0000","last_became_peered":"2024-04-04T06:05:53.387187+0000","last_unstale":"2024-04-04T06:05:54.364261+0000","last_undegraded":"2024-04-04T06:05:54.364261+0000","last_fullsized":"2024-04-04T06:05:54.364261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.366558+0000","last_change":"2024-04-04T06:05:53.378705+0000","last_active":"2024-04-04T06:05:54.366558+0000","last_peered":"2024-04-04T06:05:54.366558+0000","last_clean":"2024-04-04T06:05:54.366558+0000","last_became_active":"2024-04-04T06:05:53.378428+0000","last_became_peered":"2024-04-04T06:05:53.378428+0000","last_unstale":"2024-04-04T06:05:54.366558+0000","last_undegraded":"2024-04-04T06:05:54.366558+0000","last_fullsized":"2024-04-04T06:05:54.366558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:17:42.995360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367881+0000","last_change":"2024-04-04T06:05:53.377200+0000","last_active":"2024-04-04T06:05:54.367881+0000","last_peered":"2024-04-04T06:05:54.367881+0000","last_clean":"2024-04-04T06:05:54.367881+0000","last_became_active":"2024-04-04T06:05:53.373063+0000","last_became_peered":"2024-04-04T06:05:53.373063+0000","last_unstale":"2024-04-04T06:05:54.367881+0000","last_undegraded":"2024-04-04T06:05:54.367881+0000","last_fullsized":"2024-04-04T06:05:54.367881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:28:48.569552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365536+0000","last_change":"2024-04-04T06:05:53.381591+0000","last_active":"2024-04-04T06:05:54.365536+0000","last_peered":"2024-04-04T06:05:54.365536+0000","last_clean":"2024-04-04T06:05:54.365536+0000","last_became_active":"2024-04-04T06:05:53.377261+0000","last_became_peered":"2024-04-04T06:05:53.377261+0000","last_unstale":"2024-04-04T06:05:54.365536+0000","last_undegraded":"2024-04-04T06:05:54.365536+0000","last_fullsized":"2024-04-04T06:05:54.365536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:07:53.196090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368055+0000","last_change":"2024-04-04T06:05:53.364565+0000","last_active":"2024-04-04T06:05:54.368055+0000","last_peered":"2024-04-04T06:05:54.368055+0000","last_clean":"2024-04-04T06:05:54.368055+0000","last_became_active":"2024-04-04T06:05:53.364292+0000","last_became_peered":"2024-04-04T06:05:53.364292+0000","last_unstale":"2024-04-04T06:05:54.368055+0000","last_undegraded":"2024-04-04T06:05:54.368055+0000","last_fullsized":"2024-04-04T06:05:54.368055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:37:54.850791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.364842+0000","last_change":"2024-04-04T06:05:53.361164+0000","last_active":"2024-04-04T06:05:54.364842+0000","last_peered":"2024-04-04T06:05:54.364842+0000","last_clean":"2024-04-04T06:05:54.364842+0000","last_became_active":"2024-04-04T06:05:53.357301+0000","last_became_peered":"2024-04-04T06:05:53.357301+0000","last_unstale":"2024-04-04T06:05:54.364842+0000","last_undegraded":"2024-04-04T06:05:54.364842+0000","last_fullsized":"2024-04-04T06:05:54.364842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:56:10.393146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.327776+0000","last_change":"2024-04-04T06:05:51.327776+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.327776+0000","last_undegraded":"2024-04-04T06:05:51.327776+0000","last_fullsized":"2024-04-04T06:05:51.327776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.336676+0000","last_change":"2024-04-04T06:05:51.336676+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.336676+0000","last_undegraded":"2024-04-04T06:05:51.336676+0000","last_fullsized":"2024-04-04T06:05:51.336676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.325761+0000","last_change":"2024-04-04T06:05:51.325761+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.325761+0000","last_undegraded":"2024-04-04T06:05:51.325761+0000","last_fullsized":"2024-04-04T06:05:51.325761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365617+0000","last_change":"2024-04-04T06:05:52.371090+0000","last_active":"2024-04-04T06:05:54.365617+0000","last_peered":"2024-04-04T06:05:54.365617+0000","last_clean":"2024-04-04T06:05:54.365617+0000","last_became_active":"2024-04-04T06:05:52.369650+0000","last_became_peered":"2024-04-04T06:05:52.369650+0000","last_unstale":"2024-04-04T06:05:54.365617+0000","last_undegraded":"2024-04-04T06:05:54.365617+0000","last_fullsized":"2024-04-04T06:05:54.365617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:45:43.480308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.339473+0000","last_change":"2024-04-04T06:05:51.339473+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.339473+0000","last_undegraded":"2024-04-04T06:05:51.339473+0000","last_fullsized":"2024-04-04T06:05:51.339473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368597+0000","last_change":"2024-04-04T06:05:53.377612+0000","last_active":"2024-04-04T06:05:54.368597+0000","last_peered":"2024-04-04T06:05:54.368597+0000","last_clean":"2024-04-04T06:05:54.368597+0000","last_became_active":"2024-04-04T06:05:53.374258+0000","last_became_peered":"2024-04-04T06:05:53.374258+0000","last_unstale":"2024-04-04T06:05:54.368597+0000","last_undegraded":"2024-04-04T06:05:54.368597+0000","last_fullsized":"2024-04-04T06:05:54.368597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:08:44.769578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368664+0000","last_change":"2024-04-04T06:05:52.373979+0000","last_active":"2024-04-04T06:05:54.368664+0000","last_peered":"2024-04-04T06:05:54.368664+0000","last_clean":"2024-04-04T06:05:54.368664+0000","last_became_active":"2024-04-04T06:05:52.373408+0000","last_became_peered":"2024-04-04T06:05:52.373408+0000","last_unstale":"2024-04-04T06:05:54.368664+0000","last_undegraded":"2024-04-04T06:05:54.368664+0000","last_fullsized":"2024-04-04T06:05:54.368664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:56:09.383441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.336685+0000","last_change":"2024-04-04T06:05:51.336685+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.336685+0000","last_undegraded":"2024-04-04T06:05:51.336685+0000","last_fullsized":"2024-04-04T06:05:51.336685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367547+0000","last_change":"2024-04-04T06:05:53.379475+0000","last_active":"2024-04-04T06:05:54.367547+0000","last_peered":"2024-04-04T06:05:54.367547+0000","last_clean":"2024-04-04T06:05:54.367547+0000","last_became_active":"2024-04-04T06:05:53.376075+0000","last_became_peered":"2024-04-04T06:05:53.376075+0000","last_unstale":"2024-04-04T06:05:54.367547+0000","last_undegraded":"2024-04-04T06:05:54.367547+0000","last_fullsized":"2024-04-04T06:05:54.367547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:03:00.104400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.337795+0000","last_change":"2024-04-04T06:05:51.337795+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.337795+0000","last_undegraded":"2024-04-04T06:05:51.337795+0000","last_fullsized":"2024-04-04T06:05:51.337795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.329056+0000","last_change":"2024-04-04T06:05:51.329056+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.329056+0000","last_undegraded":"2024-04-04T06:05:51.329056+0000","last_fullsized":"2024-04-04T06:05:51.329056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368191+0000","last_change":"2024-04-04T06:05:52.365334+0000","last_active":"2024-04-04T06:05:54.368191+0000","last_peered":"2024-04-04T06:05:54.368191+0000","last_clean":"2024-04-04T06:05:54.368191+0000","last_became_active":"2024-04-04T06:05:52.363154+0000","last_became_peered":"2024-04-04T06:05:52.363154+0000","last_unstale":"2024-04-04T06:05:54.368191+0000","last_undegraded":"2024-04-04T06:05:54.368191+0000","last_fullsized":"2024-04-04T06:05:54.368191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:53:42.904072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.335788+0000","last_change":"2024-04-04T06:05:51.335788+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.335788+0000","last_undegraded":"2024-04-04T06:05:51.335788+0000","last_fullsized":"2024-04-04T06:05:51.335788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.337021+0000","last_change":"2024-04-04T06:05:51.337021+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.337021+0000","last_undegraded":"2024-04-04T06:05:51.337021+0000","last_fullsized":"2024-04-04T06:05:51.337021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367992+0000","last_change":"2024-04-04T06:05:52.362010+0000","last_active":"2024-04-04T06:05:54.367992+0000","last_peered":"2024-04-04T06:05:54.367992+0000","last_clean":"2024-04-04T06:05:54.367992+0000","last_became_active":"2024-04-04T06:05:52.361424+0000","last_became_peered":"2024-04-04T06:05:52.361424+0000","last_unstale":"2024-04-04T06:05:54.367992+0000","last_undegraded":"2024-04-04T06:05:54.367992+0000","last_fullsized":"2024-04-04T06:05:54.367992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:26:03.163432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.336405+0000","last_change":"2024-04-04T06:05:51.336405+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.336405+0000","last_undegraded":"2024-04-04T06:05:51.336405+0000","last_fullsized":"2024-04-04T06:05:51.336405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.332479+0000","last_change":"2024-04-04T06:05:51.332479+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.332479+0000","last_undegraded":"2024-04-04T06:05:51.332479+0000","last_fullsized":"2024-04-04T06:05:51.332479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.366141+0000","last_change":"2024-04-04T06:05:53.381556+0000","last_active":"2024-04-04T06:05:54.366141+0000","last_peered":"2024-04-04T06:05:54.366141+0000","last_clean":"2024-04-04T06:05:54.366141+0000","last_became_active":"2024-04-04T06:05:53.373492+0000","last_became_peered":"2024-04-04T06:05:53.373492+0000","last_unstale":"2024-04-04T06:05:54.366141+0000","last_undegraded":"2024-04-04T06:05:54.366141+0000","last_fullsized":"2024-04-04T06:05:54.366141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:53:19.419468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.338625+0000","last_change":"2024-04-04T06:05:51.338625+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.338625+0000","last_undegraded":"2024-04-04T06:05:51.338625+0000","last_fullsized":"2024-04-04T06:05:51.338625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-04T06:05:51.334129+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:05:51.334129+0000","last_peered":"2024-04-04T06:05:51.334129+0000","last_clean":"2024-04-04T06:05:51.334129+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:05:51.334129+0000","last_undegraded":"2024-04-04T06:05:51.334129+0000","last_fullsized":"2024-04-04T06:05:51.334129+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365145+0000","last_change":"2024-04-04T06:05:52.356730+0000","last_active":"2024-04-04T06:05:54.365145+0000","last_peered":"2024-04-04T06:05:54.365145+0000","last_clean":"2024-04-04T06:05:54.365145+0000","last_became_active":"2024-04-04T06:05:52.352963+0000","last_became_peered":"2024-04-04T06:05:52.352963+0000","last_unstale":"2024-04-04T06:05:54.365145+0000","last_undegraded":"2024-04-04T06:05:54.365145+0000","last_fullsized":"2024-04-04T06:05:54.365145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:05:06.878796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.325331+0000","last_change":"2024-04-04T06:05:51.325331+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.325331+0000","last_undegraded":"2024-04-04T06:05:51.325331+0000","last_fullsized":"2024-04-04T06:05:51.325331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.336360+0000","last_change":"2024-04-04T06:05:51.336360+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.336360+0000","last_undegraded":"2024-04-04T06:05:51.336360+0000","last_fullsized":"2024-04-04T06:05:51.336360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368163+0000","last_change":"2024-04-04T06:05:52.370387+0000","last_active":"2024-04-04T06:05:54.368163+0000","last_peered":"2024-04-04T06:05:54.368163+0000","last_clean":"2024-04-04T06:05:54.368163+0000","last_became_active":"2024-04-04T06:05:52.369905+0000","last_became_peered":"2024-04-04T06:05:52.369905+0000","last_unstale":"2024-04-04T06:05:54.368163+0000","last_undegraded":"2024-04-04T06:05:54.368163+0000","last_fullsized":"2024-04-04T06:05:54.368163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:58:40.874784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367307+0000","last_change":"2024-04-04T06:05:53.371957+0000","last_active":"2024-04-04T06:05:54.367307+0000","last_peered":"2024-04-04T06:05:54.367307+0000","last_clean":"2024-04-04T06:05:54.367307+0000","last_became_active":"2024-04-04T06:05:53.371121+0000","last_became_peered":"2024-04-04T06:05:53.371121+0000","last_unstale":"2024-04-04T06:05:54.367307+0000","last_undegraded":"2024-04-04T06:05:54.367307+0000","last_fullsized":"2024-04-04T06:05:54.367307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:31.723861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.332576+0000","last_change":"2024-04-04T06:05:51.332576+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.332576+0000","last_undegraded":"2024-04-04T06:05:51.332576+0000","last_fullsized":"2024-04-04T06:05:51.332576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365934+0000","last_change":"2024-04-04T06:05:53.379438+0000","last_active":"2024-04-04T06:05:54.365934+0000","last_peered":"2024-04-04T06:05:54.365934+0000","last_clean":"2024-04-04T06:05:54.365934+0000","last_became_active":"2024-04-04T06:05:53.378781+0000","last_became_peered":"2024-04-04T06:05:53.378781+0000","last_unstale":"2024-04-04T06:05:54.365934+0000","last_undegraded":"2024-04-04T06:05:54.365934+0000","last_fullsized":"2024-04-04T06:05:54.365934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:18:05.337194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.325478+0000","last_change":"2024-04-04T06:05:51.325478+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.325478+0000","last_undegraded":"2024-04-04T06:05:51.325478+0000","last_fullsized":"2024-04-04T06:05:51.325478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.335106+0000","last_change":"2024-04-04T06:05:51.335106+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.335106+0000","last_undegraded":"2024-04-04T06:05:51.335106+0000","last_fullsized":"2024-04-04T06:05:51.335106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.336927+0000","last_change":"2024-04-04T06:05:51.336927+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.336927+0000","last_undegraded":"2024-04-04T06:05:51.336927+0000","last_fullsized":"2024-04-04T06:05:51.336927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.328237+0000","last_change":"2024-04-04T06:05:51.328237+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.328237+0000","last_undegraded":"2024-04-04T06:05:51.328237+0000","last_fullsized":"2024-04-04T06:05:51.328237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.364951+0000","last_change":"2024-04-04T06:05:53.364563+0000","last_active":"2024-04-04T06:05:54.364951+0000","last_peered":"2024-04-04T06:05:54.364951+0000","last_clean":"2024-04-04T06:05:54.364951+0000","last_became_active":"2024-04-04T06:05:53.362702+0000","last_became_peered":"2024-04-04T06:05:53.362702+0000","last_unstale":"2024-04-04T06:05:54.364951+0000","last_undegraded":"2024-04-04T06:05:54.364951+0000","last_fullsized":"2024-04-04T06:05:54.364951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:54:43.620559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.331520+0000","last_change":"2024-04-04T06:05:51.331520+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.331520+0000","last_undegraded":"2024-04-04T06:05:51.331520+0000","last_fullsized":"2024-04-04T06:05:51.331520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.331679+0000","last_change":"2024-04-04T06:05:51.331679+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.331679+0000","last_undegraded":"2024-04-04T06:05:51.331679+0000","last_fullsized":"2024-04-04T06:05:51.331679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.325454+0000","last_change":"2024-04-04T06:05:51.325454+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.325454+0000","last_undegraded":"2024-04-04T06:05:51.325454+0000","last_fullsized":"2024-04-04T06:05:51.325454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365635+0000","last_change":"2024-04-04T06:05:53.378971+0000","last_active":"2024-04-04T06:05:54.365635+0000","last_peered":"2024-04-04T06:05:54.365635+0000","last_clean":"2024-04-04T06:05:54.365635+0000","last_became_active":"2024-04-04T06:05:53.377287+0000","last_became_peered":"2024-04-04T06:05:53.377287+0000","last_unstale":"2024-04-04T06:05:54.365635+0000","last_undegraded":"2024-04-04T06:05:54.365635+0000","last_fullsized":"2024-04-04T06:05:54.365635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:17:05.576402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.325331+0000","last_change":"2024-04-04T06:05:51.325331+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.325331+0000","last_undegraded":"2024-04-04T06:05:51.325331+0000","last_fullsized":"2024-04-04T06:05:51.325331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365922+0000","last_change":"2024-04-04T06:05:53.381179+0000","last_active":"2024-04-04T06:05:54.365922+0000","last_peered":"2024-04-04T06:05:54.365922+0000","last_clean":"2024-04-04T06:05:54.365922+0000","last_became_active":"2024-04-04T06:05:53.374859+0000","last_became_peered":"2024-04-04T06:05:53.374859+0000","last_unstale":"2024-04-04T06:05:54.365922+0000","last_undegraded":"2024-04-04T06:05:54.365922+0000","last_fullsized":"2024-04-04T06:05:54.365922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:20:24.966139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.327946+0000","last_change":"2024-04-04T06:05:51.327946+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.327946+0000","last_undegraded":"2024-04-04T06:05:51.327946+0000","last_fullsized":"2024-04-04T06:05:51.327946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.326789+0000","last_change":"2024-04-04T06:05:51.326789+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.326789+0000","last_undegraded":"2024-04-04T06:05:51.326789+0000","last_fullsized":"2024-04-04T06:05:51.326789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.330229+0000","last_change":"2024-04-04T06:05:51.330229+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.330229+0000","last_undegraded":"2024-04-04T06:05:51.330229+0000","last_fullsized":"2024-04-04T06:05:51.330229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":34,"acting":34,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31928,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339912,"statfs":{"total":96636764160,"available":96604069888,"internally_reserved":0,"allocated":765952,"data_stored":643501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":159744,"data_stored":47316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":172032,"data_stored":53133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":172032,"data_stored":53133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":172032,"data_stored":53133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":765952,"data_stored":643501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":72573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":72609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:05:57.759 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:05:58.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.014+0000 7fe60f200640 1 Processor -- start 2024-04-04T06:05:58.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.015+0000 7fe60f200640 1 -- start start 2024-04-04T06:05:58.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.016+0000 7fe60f200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe61015e4a0 0x7fe6100b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:58.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.016+0000 7fe60f200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe6100cf700 0x7fe6100b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:58.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.016+0000 7fe60f200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe61015e880 con 0x7fe6100d2310 2024-04-04T06:05:58.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.016+0000 7fe60f200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe6100d34e0 con 0x7fe61015e4a0 2024-04-04T06:05:58.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.016+0000 7fe60f200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe6100d28d0 con 0x7fe6100cf700 2024-04-04T06:05:58.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.016+0000 7fe60de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe61015e4a0 0x7fe6100b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:58.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.017+0000 7fe60de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe61015e4a0 0x7fe6100b9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43308/0 (socket says 172.21.15.159:43308) 2024-04-04T06:05:58.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.017+0000 7fe60de00640 1 -- 172.21.15.159:0/1133174848 learned_addr learned my addr 172.21.15.159:0/1133174848 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:58.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.017+0000 7fe60e800640 1 --2- 172.21.15.159:0/1133174848 >> v2:172.21.15.178:3300/0 conn(0x7fe6100cf700 0x7fe6100b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:58.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.017+0000 7fe60de00640 1 -- 172.21.15.159:0/1133174848 >> v2:172.21.15.178:3300/0 conn(0x7fe6100cf700 msgr2=0x7fe6100b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:58.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.017+0000 7fe60de00640 1 --2- 172.21.15.159:0/1133174848 >> v2:172.21.15.178:3300/0 conn(0x7fe6100cf700 0x7fe6100b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:58.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.017+0000 7fe60de00640 1 -- 172.21.15.159:0/1133174848 >> v1:172.21.15.178:6789/0 conn(0x7fe6100d2310 legacy=0x7fe6100ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:58.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.017+0000 7fe60de00640 1 -- 172.21.15.159:0/1133174848 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6100d2bb0 con 0x7fe61015e4a0 2024-04-04T06:05:58.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.017+0000 7fe60e800640 1 --2- 172.21.15.159:0/1133174848 >> v2:172.21.15.178:3300/0 conn(0x7fe6100cf700 0x7fe6100b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:05:58.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.018+0000 7fe60de00640 1 --2- 172.21.15.159:0/1133174848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe61015e4a0 0x7fe6100b9c60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe6100cfb20 tx=0x7fe5f802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7757b9d5e375fc85 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:58.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.018+0000 7fe60ca00640 1 -- 172.21.15.159:0/1133174848 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f80317c0 con 0x7fe61015e4a0 2024-04-04T06:05:58.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.018+0000 7fe60ca00640 1 -- 172.21.15.159:0/1133174848 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f8037070 con 0x7fe61015e4a0 2024-04-04T06:05:58.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.018+0000 7fe60ca00640 1 -- 172.21.15.159:0/1133174848 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8031b80 con 0x7fe61015e4a0 2024-04-04T06:05:58.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.019+0000 7fe60f200640 1 -- 172.21.15.159:0/1133174848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe61015e4a0 msgr2=0x7fe6100b9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:58.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.019+0000 7fe60f200640 1 --2- 172.21.15.159:0/1133174848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe61015e4a0 0x7fe6100b9c60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe6100cfb20 tx=0x7fe5f802b860 comp rx=0 tx=0).stop 2024-04-04T06:05:58.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.019+0000 7fe60f200640 1 -- 172.21.15.159:0/1133174848 shutdown_connections 2024-04-04T06:05:58.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.019+0000 7fe60f200640 1 --2- 172.21.15.159:0/1133174848 >> v2:172.21.15.178:3300/0 conn(0x7fe6100cf700 0x7fe6100b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:58.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.019+0000 7fe60f200640 1 --2- 172.21.15.159:0/1133174848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe61015e4a0 0x7fe6100b9c60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:58.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.019+0000 7fe60f200640 1 -- 172.21.15.159:0/1133174848 >> 172.21.15.159:0/1133174848 conn(0x7fe6100676d0 msgr2=0x7fe610067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:58.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.019+0000 7fe60f200640 1 -- 172.21.15.159:0/1133174848 shutdown_connections 2024-04-04T06:05:58.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.019+0000 7fe60f200640 1 -- 172.21.15.159:0/1133174848 wait complete. 2024-04-04T06:05:58.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.022+0000 7fe60f200640 1 Processor -- start 2024-04-04T06:05:58.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.022+0000 7fe60f200640 1 -- start start 2024-04-04T06:05:58.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.023+0000 7fe60f200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6100cf700 0x7fe610154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:58.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.023+0000 7fe60f200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6100d2310 0x7fe61015f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:58.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.023+0000 7fe60f200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe61015e4a0 0x7fe6101f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:58.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.023+0000 7fe60f200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe6100d3970 con 0x7fe6100cf700 2024-04-04T06:05:58.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.023+0000 7fe60f200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6100d3170 con 0x7fe6100d2310 2024-04-04T06:05:58.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.023+0000 7fe60f200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe6100ba1a0 con 0x7fe61015e4a0 2024-04-04T06:05:58.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.023+0000 7fe60de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6100cf700 0x7fe610154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:58.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.023+0000 7fe60de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6100cf700 0x7fe610154300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43314/0 (socket says 172.21.15.159:43314) 2024-04-04T06:05:58.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.023+0000 7fe60de00640 1 -- 172.21.15.159:0/3420331297 learned_addr learned my addr 172.21.15.159:0/3420331297 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:58.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.024+0000 7fe60d400640 1 --2- 172.21.15.159:0/3420331297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6100d2310 0x7fe61015f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:58.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.024+0000 7fe60e800640 1 --2- 172.21.15.159:0/3420331297 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe61015e4a0 0x7fe6101f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:58.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.024+0000 7fe60de00640 1 -- 172.21.15.159:0/3420331297 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe61015e4a0 msgr2=0x7fe6101f44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:58.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.024+0000 7fe60de00640 1 --2- 172.21.15.159:0/3420331297 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe61015e4a0 0x7fe6101f44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:58.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.024+0000 7fe60de00640 1 -- 172.21.15.159:0/3420331297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6100d2310 msgr2=0x7fe61015f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:58.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.024+0000 7fe60de00640 1 --2- 172.21.15.159:0/3420331297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6100d2310 0x7fe61015f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:58.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.024+0000 7fe60de00640 1 -- 172.21.15.159:0/3420331297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6100cc0b0 con 0x7fe6100cf700 2024-04-04T06:05:58.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.024+0000 7fe60de00640 1 --2- 172.21.15.159:0/3420331297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6100cf700 0x7fe610154300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe5f802b8a0 tx=0x7fe5f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:58.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.024+0000 7fe606000640 1 -- 172.21.15.159:0/3420331297 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8032860 con 0x7fe6100cf700 2024-04-04T06:05:58.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.025+0000 7fe606000640 1 -- 172.21.15.159:0/3420331297 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f8032a00 con 0x7fe6100cf700 2024-04-04T06:05:58.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.025+0000 7fe60f200640 1 -- 172.21.15.159:0/3420331297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6100cc8d0 con 0x7fe6100cf700 2024-04-04T06:05:58.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.025+0000 7fe606000640 1 -- 172.21.15.159:0/3420331297 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8032dd0 con 0x7fe6100cf700 2024-04-04T06:05:58.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.025+0000 7fe60f200640 1 -- 172.21.15.159:0/3420331297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61015fdd0 con 0x7fe6100cf700 2024-04-04T06:05:58.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.026+0000 7fe60f200640 1 -- 172.21.15.159:0/3420331297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe610160060 con 0x7fe6100cf700 2024-04-04T06:05:58.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.027+0000 7fe606000640 1 -- 172.21.15.159:0/3420331297 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe5f80453f0 con 0x7fe6100cf700 2024-04-04T06:05:58.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.028+0000 7fe606000640 1 --2- 172.21.15.159:0/3420331297 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe5ec06c8f0 0x7fe5ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:58.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.028+0000 7fe606000640 1 -- 172.21.15.159:0/3420331297 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe5f80add60 con 0x7fe6100cf700 2024-04-04T06:05:58.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.028+0000 7fe60d400640 1 --2- 172.21.15.159:0/3420331297 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe5ec06c8f0 0x7fe5ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:58.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.029+0000 7fe60d400640 1 --2- 172.21.15.159:0/3420331297 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe5ec06c8f0 0x7fe5ec06edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe6100cae00 tx=0x7fe600009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:58.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.032+0000 7fe606000640 1 -- 172.21.15.159:0/3420331297 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe5f8040070 con 0x7fe6100cf700 2024-04-04T06:05:58.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.228+0000 7fe60f200640 1 -- 172.21.15.159:0/3420331297 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe6100c9a90 con 0x7fe5ec06c8f0 2024-04-04T06:05:58.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.245+0000 7fe606000640 1 -- 172.21.15.159:0/3420331297 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242990 (secure 0 0 0) 0x7fe60000e8d0 con 0x7fe5ec06c8f0 2024-04-04T06:05:58.244 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:05:58.245 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:05:58.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.251+0000 7fe60f200640 1 -- 172.21.15.159:0/3420331297 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe5ec06c8f0 msgr2=0x7fe5ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:58.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.251+0000 7fe60f200640 1 --2- 172.21.15.159:0/3420331297 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe5ec06c8f0 0x7fe5ec06edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe6100cae00 tx=0x7fe600009040 comp rx=0 tx=0).stop 2024-04-04T06:05:58.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.251+0000 7fe60f200640 1 -- 172.21.15.159:0/3420331297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6100cf700 msgr2=0x7fe610154300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:58.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.251+0000 7fe60f200640 1 --2- 172.21.15.159:0/3420331297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6100cf700 0x7fe610154300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe5f802b8a0 tx=0x7fe5f80046c0 comp rx=0 tx=0).stop 2024-04-04T06:05:58.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.251+0000 7fe60f200640 1 -- 172.21.15.159:0/3420331297 shutdown_connections 2024-04-04T06:05:58.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.251+0000 7fe60f200640 1 --2- 172.21.15.159:0/3420331297 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe61015e4a0 0x7fe6101f44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:58.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.251+0000 7fe60f200640 1 --2- 172.21.15.159:0/3420331297 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe5ec06c8f0 0x7fe5ec06edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:58.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.251+0000 7fe60f200640 1 --2- 172.21.15.159:0/3420331297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6100d2310 0x7fe61015f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:58.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.251+0000 7fe60f200640 1 --2- 172.21.15.159:0/3420331297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6100cf700 0x7fe610154300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:58.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.251+0000 7fe60f200640 1 -- 172.21.15.159:0/3420331297 >> 172.21.15.159:0/3420331297 conn(0x7fe6100676d0 msgr2=0x7fe6100ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:58.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.251+0000 7fe60f200640 1 -- 172.21.15.159:0/3420331297 shutdown_connections 2024-04-04T06:05:58.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.251+0000 7fe60f200640 1 -- 172.21.15.159:0/3420331297 wait complete. 2024-04-04T06:05:58.264 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-04T06:05:56.711698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590390,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":160,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":100,"acting":100,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235380,"kb_used_data":2580,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739340,"statfs":{"total":773094113280,"available":772853084160,"internally_reserved":0,"allocated":2641920,"data_stored":1638899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.392595"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365717+0000","last_change":"2024-04-04T06:05:53.373115+0000","last_active":"2024-04-04T06:05:54.365717+0000","last_peered":"2024-04-04T06:05:54.365717+0000","last_clean":"2024-04-04T06:05:54.365717+0000","last_became_active":"2024-04-04T06:05:53.372848+0000","last_became_peered":"2024-04-04T06:05:53.372848+0000","last_unstale":"2024-04-04T06:05:54.365717+0000","last_undegraded":"2024-04-04T06:05:54.365717+0000","last_fullsized":"2024-04-04T06:05:54.365717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:45:21.548886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367929+0000","last_change":"2024-04-04T06:05:53.387263+0000","last_active":"2024-04-04T06:05:54.367929+0000","last_peered":"2024-04-04T06:05:54.367929+0000","last_clean":"2024-04-04T06:05:54.367929+0000","last_became_active":"2024-04-04T06:05:53.387012+0000","last_became_peered":"2024-04-04T06:05:53.387012+0000","last_unstale":"2024-04-04T06:05:54.367929+0000","last_undegraded":"2024-04-04T06:05:54.367929+0000","last_fullsized":"2024-04-04T06:05:54.367929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:13.472940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367238+0000","last_change":"2024-04-04T06:05:53.364063+0000","last_active":"2024-04-04T06:05:54.367238+0000","last_peered":"2024-04-04T06:05:54.367238+0000","last_clean":"2024-04-04T06:05:54.367238+0000","last_became_active":"2024-04-04T06:05:53.363715+0000","last_became_peered":"2024-04-04T06:05:53.363715+0000","last_unstale":"2024-04-04T06:05:54.367238+0000","last_undegraded":"2024-04-04T06:05:54.367238+0000","last_fullsized":"2024-04-04T06:05:54.367238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:21.889411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.364261+0000","last_change":"2024-04-04T06:05:53.387499+0000","last_active":"2024-04-04T06:05:54.364261+0000","last_peered":"2024-04-04T06:05:54.364261+0000","last_clean":"2024-04-04T06:05:54.364261+0000","last_became_active":"2024-04-04T06:05:53.387187+0000","last_became_peered":"2024-04-04T06:05:53.387187+0000","last_unstale":"2024-04-04T06:05:54.364261+0000","last_undegraded":"2024-04-04T06:05:54.364261+0000","last_fullsized":"2024-04-04T06:05:54.364261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.366558+0000","last_change":"2024-04-04T06:05:53.378705+0000","last_active":"2024-04-04T06:05:54.366558+0000","last_peered":"2024-04-04T06:05:54.366558+0000","last_clean":"2024-04-04T06:05:54.366558+0000","last_became_active":"2024-04-04T06:05:53.378428+0000","last_became_peered":"2024-04-04T06:05:53.378428+0000","last_unstale":"2024-04-04T06:05:54.366558+0000","last_undegraded":"2024-04-04T06:05:54.366558+0000","last_fullsized":"2024-04-04T06:05:54.366558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:17:42.995360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367881+0000","last_change":"2024-04-04T06:05:53.377200+0000","last_active":"2024-04-04T06:05:54.367881+0000","last_peered":"2024-04-04T06:05:54.367881+0000","last_clean":"2024-04-04T06:05:54.367881+0000","last_became_active":"2024-04-04T06:05:53.373063+0000","last_became_peered":"2024-04-04T06:05:53.373063+0000","last_unstale":"2024-04-04T06:05:54.367881+0000","last_undegraded":"2024-04-04T06:05:54.367881+0000","last_fullsized":"2024-04-04T06:05:54.367881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:28:48.569552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365536+0000","last_change":"2024-04-04T06:05:53.381591+0000","last_active":"2024-04-04T06:05:54.365536+0000","last_peered":"2024-04-04T06:05:54.365536+0000","last_clean":"2024-04-04T06:05:54.365536+0000","last_became_active":"2024-04-04T06:05:53.377261+0000","last_became_peered":"2024-04-04T06:05:53.377261+0000","last_unstale":"2024-04-04T06:05:54.365536+0000","last_undegraded":"2024-04-04T06:05:54.365536+0000","last_fullsized":"2024-04-04T06:05:54.365536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:07:53.196090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368055+0000","last_change":"2024-04-04T06:05:53.364565+0000","last_active":"2024-04-04T06:05:54.368055+0000","last_peered":"2024-04-04T06:05:54.368055+0000","last_clean":"2024-04-04T06:05:54.368055+0000","last_became_active":"2024-04-04T06:05:53.364292+0000","last_became_peered":"2024-04-04T06:05:53.364292+0000","last_unstale":"2024-04-04T06:05:54.368055+0000","last_undegraded":"2024-04-04T06:05:54.368055+0000","last_fullsized":"2024-04-04T06:05:54.368055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:37:54.850791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.364842+0000","last_change":"2024-04-04T06:05:53.361164+0000","last_active":"2024-04-04T06:05:54.364842+0000","last_peered":"2024-04-04T06:05:54.364842+0000","last_clean":"2024-04-04T06:05:54.364842+0000","last_became_active":"2024-04-04T06:05:53.357301+0000","last_became_peered":"2024-04-04T06:05:53.357301+0000","last_unstale":"2024-04-04T06:05:54.364842+0000","last_undegraded":"2024-04-04T06:05:54.364842+0000","last_fullsized":"2024-04-04T06:05:54.364842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:56:10.393146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.327776+0000","last_change":"2024-04-04T06:05:51.327776+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.327776+0000","last_undegraded":"2024-04-04T06:05:51.327776+0000","last_fullsized":"2024-04-04T06:05:51.327776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.336676+0000","last_change":"2024-04-04T06:05:51.336676+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.336676+0000","last_undegraded":"2024-04-04T06:05:51.336676+0000","last_fullsized":"2024-04-04T06:05:51.336676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.325761+0000","last_change":"2024-04-04T06:05:51.325761+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.325761+0000","last_undegraded":"2024-04-04T06:05:51.325761+0000","last_fullsized":"2024-04-04T06:05:51.325761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365617+0000","last_change":"2024-04-04T06:05:52.371090+0000","last_active":"2024-04-04T06:05:54.365617+0000","last_peered":"2024-04-04T06:05:54.365617+0000","last_clean":"2024-04-04T06:05:54.365617+0000","last_became_active":"2024-04-04T06:05:52.369650+0000","last_became_peered":"2024-04-04T06:05:52.369650+0000","last_unstale":"2024-04-04T06:05:54.365617+0000","last_undegraded":"2024-04-04T06:05:54.365617+0000","last_fullsized":"2024-04-04T06:05:54.365617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:45:43.480308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.339473+0000","last_change":"2024-04-04T06:05:51.339473+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.339473+0000","last_undegraded":"2024-04-04T06:05:51.339473+0000","last_fullsized":"2024-04-04T06:05:51.339473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368597+0000","last_change":"2024-04-04T06:05:53.377612+0000","last_active":"2024-04-04T06:05:54.368597+0000","last_peered":"2024-04-04T06:05:54.368597+0000","last_clean":"2024-04-04T06:05:54.368597+0000","last_became_active":"2024-04-04T06:05:53.374258+0000","last_became_peered":"2024-04-04T06:05:53.374258+0000","last_unstale":"2024-04-04T06:05:54.368597+0000","last_undegraded":"2024-04-04T06:05:54.368597+0000","last_fullsized":"2024-04-04T06:05:54.368597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:08:44.769578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368664+0000","last_change":"2024-04-04T06:05:52.373979+0000","last_active":"2024-04-04T06:05:54.368664+0000","last_peered":"2024-04-04T06:05:54.368664+0000","last_clean":"2024-04-04T06:05:54.368664+0000","last_became_active":"2024-04-04T06:05:52.373408+0000","last_became_peered":"2024-04-04T06:05:52.373408+0000","last_unstale":"2024-04-04T06:05:54.368664+0000","last_undegraded":"2024-04-04T06:05:54.368664+0000","last_fullsized":"2024-04-04T06:05:54.368664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:56:09.383441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.336685+0000","last_change":"2024-04-04T06:05:51.336685+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.336685+0000","last_undegraded":"2024-04-04T06:05:51.336685+0000","last_fullsized":"2024-04-04T06:05:51.336685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367547+0000","last_change":"2024-04-04T06:05:53.379475+0000","last_active":"2024-04-04T06:05:54.367547+0000","last_peered":"2024-04-04T06:05:54.367547+0000","last_clean":"2024-04-04T06:05:54.367547+0000","last_became_active":"2024-04-04T06:05:53.376075+0000","last_became_peered":"2024-04-04T06:05:53.376075+0000","last_unstale":"2024-04-04T06:05:54.367547+0000","last_undegraded":"2024-04-04T06:05:54.367547+0000","last_fullsized":"2024-04-04T06:05:54.367547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:03:00.104400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.337795+0000","last_change":"2024-04-04T06:05:51.337795+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.337795+0000","last_undegraded":"2024-04-04T06:05:51.337795+0000","last_fullsized":"2024-04-04T06:05:51.337795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.329056+0000","last_change":"2024-04-04T06:05:51.329056+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.329056+0000","last_undegraded":"2024-04-04T06:05:51.329056+0000","last_fullsized":"2024-04-04T06:05:51.329056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368191+0000","last_change":"2024-04-04T06:05:52.365334+0000","last_active":"2024-04-04T06:05:54.368191+0000","last_peered":"2024-04-04T06:05:54.368191+0000","last_clean":"2024-04-04T06:05:54.368191+0000","last_became_active":"2024-04-04T06:05:52.363154+0000","last_became_peered":"2024-04-04T06:05:52.363154+0000","last_unstale":"2024-04-04T06:05:54.368191+0000","last_undegraded":"2024-04-04T06:05:54.368191+0000","last_fullsized":"2024-04-04T06:05:54.368191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:53:42.904072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.335788+0000","last_change":"2024-04-04T06:05:51.335788+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.335788+0000","last_undegraded":"2024-04-04T06:05:51.335788+0000","last_fullsized":"2024-04-04T06:05:51.335788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.337021+0000","last_change":"2024-04-04T06:05:51.337021+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.337021+0000","last_undegraded":"2024-04-04T06:05:51.337021+0000","last_fullsized":"2024-04-04T06:05:51.337021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367992+0000","last_change":"2024-04-04T06:05:52.362010+0000","last_active":"2024-04-04T06:05:54.367992+0000","last_peered":"2024-04-04T06:05:54.367992+0000","last_clean":"2024-04-04T06:05:54.367992+0000","last_became_active":"2024-04-04T06:05:52.361424+0000","last_became_peered":"2024-04-04T06:05:52.361424+0000","last_unstale":"2024-04-04T06:05:54.367992+0000","last_undegraded":"2024-04-04T06:05:54.367992+0000","last_fullsized":"2024-04-04T06:05:54.367992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:26:03.163432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.336405+0000","last_change":"2024-04-04T06:05:51.336405+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.336405+0000","last_undegraded":"2024-04-04T06:05:51.336405+0000","last_fullsized":"2024-04-04T06:05:51.336405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.332479+0000","last_change":"2024-04-04T06:05:51.332479+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.332479+0000","last_undegraded":"2024-04-04T06:05:51.332479+0000","last_fullsized":"2024-04-04T06:05:51.332479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.366141+0000","last_change":"2024-04-04T06:05:53.381556+0000","last_active":"2024-04-04T06:05:54.366141+0000","last_peered":"2024-04-04T06:05:54.366141+0000","last_clean":"2024-04-04T06:05:54.366141+0000","last_became_active":"2024-04-04T06:05:53.373492+0000","last_became_peered":"2024-04-04T06:05:53.373492+0000","last_unstale":"2024-04-04T06:05:54.366141+0000","last_undegraded":"2024-04-04T06:05:54.366141+0000","last_fullsized":"2024-04-04T06:05:54.366141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:53:19.419468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.338625+0000","last_change":"2024-04-04T06:05:51.338625+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.338625+0000","last_undegraded":"2024-04-04T06:05:51.338625+0000","last_fullsized":"2024-04-04T06:05:51.338625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-04T06:05:51.334129+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:05:51.334129+0000","last_peered":"2024-04-04T06:05:51.334129+0000","last_clean":"2024-04-04T06:05:51.334129+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:05:51.334129+0000","last_undegraded":"2024-04-04T06:05:51.334129+0000","last_fullsized":"2024-04-04T06:05:51.334129+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365145+0000","last_change":"2024-04-04T06:05:52.356730+0000","last_active":"2024-04-04T06:05:54.365145+0000","last_peered":"2024-04-04T06:05:54.365145+0000","last_clean":"2024-04-04T06:05:54.365145+0000","last_became_active":"2024-04-04T06:05:52.352963+0000","last_became_peered":"2024-04-04T06:05:52.352963+0000","last_unstale":"2024-04-04T06:05:54.365145+0000","last_undegraded":"2024-04-04T06:05:54.365145+0000","last_fullsized":"2024-04-04T06:05:54.365145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:05:06.878796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.325331+0000","last_change":"2024-04-04T06:05:51.325331+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.325331+0000","last_undegraded":"2024-04-04T06:05:51.325331+0000","last_fullsized":"2024-04-04T06:05:51.325331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.336360+0000","last_change":"2024-04-04T06:05:51.336360+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.336360+0000","last_undegraded":"2024-04-04T06:05:51.336360+0000","last_fullsized":"2024-04-04T06:05:51.336360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368163+0000","last_change":"2024-04-04T06:05:52.370387+0000","last_active":"2024-04-04T06:05:54.368163+0000","last_peered":"2024-04-04T06:05:54.368163+0000","last_clean":"2024-04-04T06:05:54.368163+0000","last_became_active":"2024-04-04T06:05:52.369905+0000","last_became_peered":"2024-04-04T06:05:52.369905+0000","last_unstale":"2024-04-04T06:05:54.368163+0000","last_undegraded":"2024-04-04T06:05:54.368163+0000","last_fullsized":"2024-04-04T06:05:54.368163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:58:40.874784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367307+0000","last_change":"2024-04-04T06:05:53.371957+0000","last_active":"2024-04-04T06:05:54.367307+0000","last_peered":"2024-04-04T06:05:54.367307+0000","last_clean":"2024-04-04T06:05:54.367307+0000","last_became_active":"2024-04-04T06:05:53.371121+0000","last_became_peered":"2024-04-04T06:05:53.371121+0000","last_unstale":"2024-04-04T06:05:54.367307+0000","last_undegraded":"2024-04-04T06:05:54.367307+0000","last_fullsized":"2024-04-04T06:05:54.367307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:31.723861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.332576+0000","last_change":"2024-04-04T06:05:51.332576+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.332576+0000","last_undegraded":"2024-04-04T06:05:51.332576+0000","last_fullsized":"2024-04-04T06:05:51.332576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365934+0000","last_change":"2024-04-04T06:05:53.379438+0000","last_active":"2024-04-04T06:05:54.365934+0000","last_peered":"2024-04-04T06:05:54.365934+0000","last_clean":"2024-04-04T06:05:54.365934+0000","last_became_active":"2024-04-04T06:05:53.378781+0000","last_became_peered":"2024-04-04T06:05:53.378781+0000","last_unstale":"2024-04-04T06:05:54.365934+0000","last_undegraded":"2024-04-04T06:05:54.365934+0000","last_fullsized":"2024-04-04T06:05:54.365934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:18:05.337194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.325478+0000","last_change":"2024-04-04T06:05:51.325478+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.325478+0000","last_undegraded":"2024-04-04T06:05:51.325478+0000","last_fullsized":"2024-04-04T06:05:51.325478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.335106+0000","last_change":"2024-04-04T06:05:51.335106+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.335106+0000","last_undegraded":"2024-04-04T06:05:51.335106+0000","last_fullsized":"2024-04-04T06:05:51.335106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.336927+0000","last_change":"2024-04-04T06:05:51.336927+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.336927+0000","last_undegraded":"2024-04-04T06:05:51.336927+0000","last_fullsized":"2024-04-04T06:05:51.336927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.328237+0000","last_change":"2024-04-04T06:05:51.328237+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.328237+0000","last_undegraded":"2024-04-04T06:05:51.328237+0000","last_fullsized":"2024-04-04T06:05:51.328237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.364951+0000","last_change":"2024-04-04T06:05:53.364563+0000","last_active":"2024-04-04T06:05:54.364951+0000","last_peered":"2024-04-04T06:05:54.364951+0000","last_clean":"2024-04-04T06:05:54.364951+0000","last_became_active":"2024-04-04T06:05:53.362702+0000","last_became_peered":"2024-04-04T06:05:53.362702+0000","last_unstale":"2024-04-04T06:05:54.364951+0000","last_undegraded":"2024-04-04T06:05:54.364951+0000","last_fullsized":"2024-04-04T06:05:54.364951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:54:43.620559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.331520+0000","last_change":"2024-04-04T06:05:51.331520+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.331520+0000","last_undegraded":"2024-04-04T06:05:51.331520+0000","last_fullsized":"2024-04-04T06:05:51.331520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.331679+0000","last_change":"2024-04-04T06:05:51.331679+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.331679+0000","last_undegraded":"2024-04-04T06:05:51.331679+0000","last_fullsized":"2024-04-04T06:05:51.331679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.325454+0000","last_change":"2024-04-04T06:05:51.325454+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.325454+0000","last_undegraded":"2024-04-04T06:05:51.325454+0000","last_fullsized":"2024-04-04T06:05:51.325454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365635+0000","last_change":"2024-04-04T06:05:53.378971+0000","last_active":"2024-04-04T06:05:54.365635+0000","last_peered":"2024-04-04T06:05:54.365635+0000","last_clean":"2024-04-04T06:05:54.365635+0000","last_became_active":"2024-04-04T06:05:53.377287+0000","last_became_peered":"2024-04-04T06:05:53.377287+0000","last_unstale":"2024-04-04T06:05:54.365635+0000","last_undegraded":"2024-04-04T06:05:54.365635+0000","last_fullsized":"2024-04-04T06:05:54.365635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:17:05.576402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.325331+0000","last_change":"2024-04-04T06:05:51.325331+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.325331+0000","last_undegraded":"2024-04-04T06:05:51.325331+0000","last_fullsized":"2024-04-04T06:05:51.325331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365922+0000","last_change":"2024-04-04T06:05:53.381179+0000","last_active":"2024-04-04T06:05:54.365922+0000","last_peered":"2024-04-04T06:05:54.365922+0000","last_clean":"2024-04-04T06:05:54.365922+0000","last_became_active":"2024-04-04T06:05:53.374859+0000","last_became_peered":"2024-04-04T06:05:53.374859+0000","last_unstale":"2024-04-04T06:05:54.365922+0000","last_undegraded":"2024-04-04T06:05:54.365922+0000","last_fullsized":"2024-04-04T06:05:54.365922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:20:24.966139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.327946+0000","last_change":"2024-04-04T06:05:51.327946+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.327946+0000","last_undegraded":"2024-04-04T06:05:51.327946+0000","last_fullsized":"2024-04-04T06:05:51.327946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.326789+0000","last_change":"2024-04-04T06:05:51.326789+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.326789+0000","last_undegraded":"2024-04-04T06:05:51.326789+0000","last_fullsized":"2024-04-04T06:05:51.326789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:05:52.317707+0000","last_change":"2024-04-04T06:05:52.317707+0000","last_active":"2024-04-04T06:05:52.317707+0000","last_peered":"2024-04-04T06:05:52.317707+0000","last_clean":"2024-04-04T06:05:52.317707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:52.317707+0000","last_undegraded":"2024-04-04T06:05:52.317707+0000","last_fullsized":"2024-04-04T06:05:52.317707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-04-04T06:05:51.330229+0000","last_change":"2024-04-04T06:05:51.330229+0000","last_active":"2024-04-04T06:05:51.308002+0000","last_peered":"2024-04-04T06:05:51.308002+0000","last_clean":"2024-04-04T06:05:51.308002+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:05:51.330229+0000","last_undegraded":"2024-04-04T06:05:51.330229+0000","last_fullsized":"2024-04-04T06:05:51.330229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":34,"acting":34,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31928,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339912,"statfs":{"total":96636764160,"available":96604069888,"internally_reserved":0,"allocated":765952,"data_stored":643501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":159744,"data_stored":47316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":172032,"data_stored":53133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":172032,"data_stored":53133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":172032,"data_stored":53133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":765952,"data_stored":643501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":72573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":72609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:05:58.268 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:05:58.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.523+0000 7fe317e00640 1 Processor -- start 2024-04-04T06:05:58.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.523+0000 7fe317e00640 1 -- start start 2024-04-04T06:05:58.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.524+0000 7fe317e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe31814ba20 0x7fe3180b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:58.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.524+0000 7fe317e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3180cfda0 0x7fe3180d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:58.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.524+0000 7fe317e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3180b33e0 con 0x7fe31814c1b0 2024-04-04T06:05:58.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.525+0000 7fe317e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe3180d06f0 con 0x7fe31814ba20 2024-04-04T06:05:58.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.525+0000 7fe317e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe3180cde20 con 0x7fe3180cfda0 2024-04-04T06:05:58.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.525+0000 7fe30e000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fe31814c1b0 0x7fe318166290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43518/0 (socket says 172.21.15.159:43518) 2024-04-04T06:05:58.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.525+0000 7fe30e000640 1 -- 172.21.15.159:0/4149386709 learned_addr learned my addr 172.21.15.159:0/4149386709 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:05:58.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.525+0000 7fe316a00640 1 --2- 172.21.15.159:0/4149386709 >> v2:172.21.15.178:3300/0 conn(0x7fe31814ba20 0x7fe3180b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:58.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.525+0000 7fe317400640 1 --2- 172.21.15.159:0/4149386709 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3180cfda0 0x7fe3180d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:58.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.525+0000 7fe316000640 1 -- 172.21.15.159:0/4149386709 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 636438182 0 0) 0x7fe3180b33e0 con 0x7fe31814c1b0 2024-04-04T06:05:58.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.525+0000 7fe316000640 1 -- 172.21.15.159:0/4149386709 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2fc003520 con 0x7fe31814c1b0 2024-04-04T06:05:58.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.526+0000 7fe316000640 1 -- 172.21.15.159:0/4149386709 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1110426293 0 0) 0x7fe2fc003520 con 0x7fe31814c1b0 2024-04-04T06:05:58.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.526+0000 7fe316000640 1 -- 172.21.15.159:0/4149386709 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3180cfda0 msgr2=0x7fe3180d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:58.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.526+0000 7fe316000640 1 --2- 172.21.15.159:0/4149386709 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3180cfda0 0x7fe3180d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:58.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.526+0000 7fe316000640 1 -- 172.21.15.159:0/4149386709 >> v2:172.21.15.178:3300/0 conn(0x7fe31814ba20 msgr2=0x7fe3180b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:58.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.526+0000 7fe316000640 1 --2- 172.21.15.159:0/4149386709 >> v2:172.21.15.178:3300/0 conn(0x7fe31814ba20 0x7fe3180b2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:58.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.526+0000 7fe316000640 1 -- 172.21.15.159:0/4149386709 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe318166ae0 con 0x7fe31814c1b0 2024-04-04T06:05:58.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.526+0000 7fe316000640 1 -- 172.21.15.159:0/4149386709 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fe300001690 con 0x7fe31814c1b0 2024-04-04T06:05:58.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.526+0000 7fe316000640 1 -- 172.21.15.159:0/4149386709 >> v1:172.21.15.159:6789/0 conn(0x7fe31814c1b0 legacy=0x7fe318166290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:58.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.526+0000 7fe316000640 1 --2- 172.21.15.159:0/4149386709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2fc0039b0 0x7fe2fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:58.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.526+0000 7fe316000640 1 --2- 172.21.15.159:0/4149386709 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3180cfda0 0x7fe2fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:58.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.526+0000 7fe316000640 1 --2- 172.21.15.159:0/4149386709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2fc004d40 0x7fe2fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:58.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.526+0000 7fe316000640 1 -- 172.21.15.159:0/4149386709 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe3180d06f0 con 0x7fe2fc0039b0 2024-04-04T06:05:58.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.526+0000 7fe316000640 1 -- 172.21.15.159:0/4149386709 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe3180cde20 con 0x7fe2fc004d40 2024-04-04T06:05:58.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.526+0000 7fe316000640 1 -- 172.21.15.159:0/4149386709 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe2fc00b9c0 con 0x7fe3180cfda0 2024-04-04T06:05:58.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.526+0000 7fe316a00640 1 --2- 172.21.15.159:0/4149386709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2fc0039b0 0x7fe2fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:58.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.526+0000 7fe30e000640 1 --2- 172.21.15.159:0/4149386709 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3180cfda0 0x7fe2fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:58.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.526+0000 7fe316a00640 1 -- 172.21.15.159:0/4149386709 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3180cfda0 msgr2=0x7fe2fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:58.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.526+0000 7fe316a00640 1 --2- 172.21.15.159:0/4149386709 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3180cfda0 0x7fe2fc004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:58.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.526+0000 7fe316a00640 1 -- 172.21.15.159:0/4149386709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2fc004d40 msgr2=0x7fe2fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:58.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.526+0000 7fe316a00640 1 --2- 172.21.15.159:0/4149386709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2fc004d40 0x7fe2fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:58.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.526+0000 7fe316a00640 1 -- 172.21.15.159:0/4149386709 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2fc003520 con 0x7fe2fc0039b0 2024-04-04T06:05:58.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.527+0000 7fe316a00640 1 --2- 172.21.15.159:0/4149386709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2fc0039b0 0x7fe2fc003dd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe308009af0 tx=0x7fe2fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16cfc321e8e926e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:58.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.528+0000 7fe316000640 1 -- 172.21.15.159:0/4149386709 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe308039070 con 0x7fe2fc0039b0 2024-04-04T06:05:58.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.528+0000 7fe316000640 1 -- 172.21.15.159:0/4149386709 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe30802c710 con 0x7fe2fc0039b0 2024-04-04T06:05:58.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.528+0000 7fe316000640 1 -- 172.21.15.159:0/4149386709 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe308009690 con 0x7fe2fc0039b0 2024-04-04T06:05:58.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.528+0000 7fe317e00640 1 -- 172.21.15.159:0/4149386709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2fc0039b0 msgr2=0x7fe2fc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:58.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.528+0000 7fe317e00640 1 --2- 172.21.15.159:0/4149386709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2fc0039b0 0x7fe2fc003dd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe308009af0 tx=0x7fe2fc005830 comp rx=0 tx=0).stop 2024-04-04T06:05:58.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.528+0000 7fe317400640 1 -- 172.21.15.159:0/4149386709 reap_dead start 2024-04-04T06:05:58.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.528+0000 7fe317e00640 1 -- 172.21.15.159:0/4149386709 shutdown_connections 2024-04-04T06:05:58.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.528+0000 7fe317e00640 1 -- 172.21.15.159:0/4149386709 >> 172.21.15.159:0/4149386709 conn(0x7fe3180ccad0 msgr2=0x7fe3180cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:58.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.528+0000 7fe317e00640 1 -- 172.21.15.159:0/4149386709 shutdown_connections 2024-04-04T06:05:58.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.528+0000 7fe317e00640 1 -- 172.21.15.159:0/4149386709 wait complete. 2024-04-04T06:05:58.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.530+0000 7fe317e00640 1 Processor -- start 2024-04-04T06:05:58.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.530+0000 7fe317e00640 1 -- start start 2024-04-04T06:05:58.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.531+0000 7fe317e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2fc0039b0 0x7fe3181fb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:58.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.531+0000 7fe317e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3181ee060 0x7fe3181486c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:58.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.531+0000 7fe317e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe318148c00 0x7fe318149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:58.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.531+0000 7fe317e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe3180ce0c0 con 0x7fe3181ee060 2024-04-04T06:05:58.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.531+0000 7fe317e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe3181670a0 con 0x7fe2fc0039b0 2024-04-04T06:05:58.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.531+0000 7fe317e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe31814be00 con 0x7fe318148c00 2024-04-04T06:05:58.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.531+0000 7fe30e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3181ee060 0x7fe3181486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:58.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.532+0000 7fe30e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3181ee060 0x7fe3181486c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43338/0 (socket says 172.21.15.159:43338) 2024-04-04T06:05:58.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.532+0000 7fe30e000640 1 -- 172.21.15.159:0/307760801 learned_addr learned my addr 172.21.15.159:0/307760801 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:58.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.532+0000 7fe317400640 1 --2- 172.21.15.159:0/307760801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe318148c00 0x7fe318149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:58.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.532+0000 7fe316a00640 1 --2- 172.21.15.159:0/307760801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2fc0039b0 0x7fe3181fb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:58.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.532+0000 7fe30e000640 1 -- 172.21.15.159:0/307760801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe318148c00 msgr2=0x7fe318149080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:58.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.532+0000 7fe30e000640 1 --2- 172.21.15.159:0/307760801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe318148c00 0x7fe318149080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:58.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.532+0000 7fe30e000640 1 -- 172.21.15.159:0/307760801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2fc0039b0 msgr2=0x7fe3181fb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:58.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.532+0000 7fe30e000640 1 --2- 172.21.15.159:0/307760801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2fc0039b0 0x7fe3181fb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:58.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.532+0000 7fe30e000640 1 -- 172.21.15.159:0/307760801 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3181ee590 con 0x7fe3181ee060 2024-04-04T06:05:58.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.532+0000 7fe30e000640 1 --2- 172.21.15.159:0/307760801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3181ee060 0x7fe3181486c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe30000d840 tx=0x7fe30000dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:58.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.533+0000 7fe30f400640 1 -- 172.21.15.159:0/307760801 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe300019070 con 0x7fe3181ee060 2024-04-04T06:05:58.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.533+0000 7fe30f400640 1 -- 172.21.15.159:0/307760801 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe30000de90 con 0x7fe3181ee060 2024-04-04T06:05:58.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.533+0000 7fe30f400640 1 -- 172.21.15.159:0/307760801 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3000047e0 con 0x7fe3181ee060 2024-04-04T06:05:58.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.533+0000 7fe317e00640 1 -- 172.21.15.159:0/307760801 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3181ee870 con 0x7fe3181ee060 2024-04-04T06:05:58.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.533+0000 7fe317e00640 1 -- 172.21.15.159:0/307760801 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3181eee30 con 0x7fe3181ee060 2024-04-04T06:05:58.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.534+0000 7fe317e00640 1 -- 172.21.15.159:0/307760801 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe318072a70 con 0x7fe3181ee060 2024-04-04T06:05:58.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.535+0000 7fe30f400640 1 -- 172.21.15.159:0/307760801 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe30001c070 con 0x7fe3181ee060 2024-04-04T06:05:58.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.535+0000 7fe30f400640 1 --2- 172.21.15.159:0/307760801 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe2f406c8f0 0x7fe2f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:58.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.536+0000 7fe30f400640 1 -- 172.21.15.159:0/307760801 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe30008b5b0 con 0x7fe3181ee060 2024-04-04T06:05:58.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.536+0000 7fe316a00640 1 --2- 172.21.15.159:0/307760801 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe2f406c8f0 0x7fe2f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:58.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.537+0000 7fe316a00640 1 --2- 172.21.15.159:0/307760801 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe2f406c8f0 0x7fe2f406edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe308036000 tx=0x7fe308030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:58.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.540+0000 7fe30f400640 1 -- 172.21.15.159:0/307760801 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe318072a70 con 0x7fe3181ee060 2024-04-04T06:05:58.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.736+0000 7fe317e00640 1 -- 172.21.15.159:0/307760801 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe318075940 con 0x7fe2f406c8f0 2024-04-04T06:05:58.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.757+0000 7fe30f400640 1 -- 172.21.15.159:0/307760801 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7fe308082840 con 0x7fe2f406c8f0 2024-04-04T06:05:58.753 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:05:58.757 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:05:58.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.763+0000 7fe317e00640 1 -- 172.21.15.159:0/307760801 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe2f406c8f0 msgr2=0x7fe2f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:58.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.763+0000 7fe317e00640 1 --2- 172.21.15.159:0/307760801 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe2f406c8f0 0x7fe2f406edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe308036000 tx=0x7fe308030000 comp rx=0 tx=0).stop 2024-04-04T06:05:58.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.763+0000 7fe317e00640 1 -- 172.21.15.159:0/307760801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3181ee060 msgr2=0x7fe3181486c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:58.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.763+0000 7fe317e00640 1 --2- 172.21.15.159:0/307760801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3181ee060 0x7fe3181486c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe30000d840 tx=0x7fe30000dc40 comp rx=0 tx=0).stop 2024-04-04T06:05:58.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.763+0000 7fe317e00640 1 -- 172.21.15.159:0/307760801 shutdown_connections 2024-04-04T06:05:58.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.763+0000 7fe317e00640 1 --2- 172.21.15.159:0/307760801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe318148c00 0x7fe318149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:58.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.763+0000 7fe317e00640 1 --2- 172.21.15.159:0/307760801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3181ee060 0x7fe3181486c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:58.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.763+0000 7fe317e00640 1 --2- 172.21.15.159:0/307760801 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe2f406c8f0 0x7fe2f406edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:58.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.763+0000 7fe317e00640 1 --2- 172.21.15.159:0/307760801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2fc0039b0 0x7fe3181fb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:58.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.763+0000 7fe317e00640 1 -- 172.21.15.159:0/307760801 >> 172.21.15.159:0/307760801 conn(0x7fe3180ccad0 msgr2=0x7fe3180cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:58.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.763+0000 7fe317e00640 1 -- 172.21.15.159:0/307760801 shutdown_connections 2024-04-04T06:05:58.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:58.763+0000 7fe317e00640 1 -- 172.21.15.159:0/307760801 wait complete. 2024-04-04T06:05:58.778 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-04T06:05:58.712390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235672,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739048,"statfs":{"total":773094113280,"available":772852785152,"internally_reserved":0,"allocated":2924544,"data_stored":1765420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.377959"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594198+0000","last_change":"2024-04-04T06:05:53.366079+0000","last_active":"2024-04-04T06:05:54.594198+0000","last_peered":"2024-04-04T06:05:54.594198+0000","last_clean":"2024-04-04T06:05:54.594198+0000","last_became_active":"2024-04-04T06:05:53.365810+0000","last_became_peered":"2024-04-04T06:05:53.365810+0000","last_unstale":"2024-04-04T06:05:54.594198+0000","last_undegraded":"2024-04-04T06:05:54.594198+0000","last_fullsized":"2024-04-04T06:05:54.594198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:46:01.152077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362281+0000","last_change":"2024-04-04T06:05:53.376777+0000","last_active":"2024-04-04T06:05:54.362281+0000","last_peered":"2024-04-04T06:05:54.362281+0000","last_clean":"2024-04-04T06:05:54.362281+0000","last_became_active":"2024-04-04T06:05:53.374947+0000","last_became_peered":"2024-04-04T06:05:53.374947+0000","last_unstale":"2024-04-04T06:05:54.362281+0000","last_undegraded":"2024-04-04T06:05:54.362281+0000","last_fullsized":"2024-04-04T06:05:54.362281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:34.766930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591301+0000","last_change":"2024-04-04T06:05:53.362328+0000","last_active":"2024-04-04T06:05:54.591301+0000","last_peered":"2024-04-04T06:05:54.591301+0000","last_clean":"2024-04-04T06:05:54.591301+0000","last_became_active":"2024-04-04T06:05:53.359587+0000","last_became_peered":"2024-04-04T06:05:53.359587+0000","last_unstale":"2024-04-04T06:05:54.591301+0000","last_undegraded":"2024-04-04T06:05:54.591301+0000","last_fullsized":"2024-04-04T06:05:54.591301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:10.411661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365717+0000","last_change":"2024-04-04T06:05:53.373115+0000","last_active":"2024-04-04T06:05:54.365717+0000","last_peered":"2024-04-04T06:05:54.365717+0000","last_clean":"2024-04-04T06:05:54.365717+0000","last_became_active":"2024-04-04T06:05:53.372848+0000","last_became_peered":"2024-04-04T06:05:53.372848+0000","last_unstale":"2024-04-04T06:05:54.365717+0000","last_undegraded":"2024-04-04T06:05:54.365717+0000","last_fullsized":"2024-04-04T06:05:54.365717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:45:21.548886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593377+0000","last_change":"2024-04-04T06:05:53.370721+0000","last_active":"2024-04-04T06:05:54.593377+0000","last_peered":"2024-04-04T06:05:54.593377+0000","last_clean":"2024-04-04T06:05:54.593377+0000","last_became_active":"2024-04-04T06:05:53.370183+0000","last_became_peered":"2024-04-04T06:05:53.370183+0000","last_unstale":"2024-04-04T06:05:54.593377+0000","last_undegraded":"2024-04-04T06:05:54.593377+0000","last_fullsized":"2024-04-04T06:05:54.593377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:30:56.836366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367929+0000","last_change":"2024-04-04T06:05:53.387263+0000","last_active":"2024-04-04T06:05:54.367929+0000","last_peered":"2024-04-04T06:05:54.367929+0000","last_clean":"2024-04-04T06:05:54.367929+0000","last_became_active":"2024-04-04T06:05:53.387012+0000","last_became_peered":"2024-04-04T06:05:53.387012+0000","last_unstale":"2024-04-04T06:05:54.367929+0000","last_undegraded":"2024-04-04T06:05:54.367929+0000","last_fullsized":"2024-04-04T06:05:54.367929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:13.472940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592309+0000","last_change":"2024-04-04T06:05:53.370203+0000","last_active":"2024-04-04T06:05:54.592309+0000","last_peered":"2024-04-04T06:05:54.592309+0000","last_clean":"2024-04-04T06:05:54.592309+0000","last_became_active":"2024-04-04T06:05:53.369800+0000","last_became_peered":"2024-04-04T06:05:53.369800+0000","last_unstale":"2024-04-04T06:05:54.592309+0000","last_undegraded":"2024-04-04T06:05:54.592309+0000","last_fullsized":"2024-04-04T06:05:54.592309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:39:30.391173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592465+0000","last_change":"2024-04-04T06:05:53.369028+0000","last_active":"2024-04-04T06:05:54.592465+0000","last_peered":"2024-04-04T06:05:54.592465+0000","last_clean":"2024-04-04T06:05:54.592465+0000","last_became_active":"2024-04-04T06:05:53.365998+0000","last_became_peered":"2024-04-04T06:05:53.365998+0000","last_unstale":"2024-04-04T06:05:54.592465+0000","last_undegraded":"2024-04-04T06:05:54.592465+0000","last_fullsized":"2024-04-04T06:05:54.592465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:34.766930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594227+0000","last_change":"2024-04-04T06:05:53.369886+0000","last_active":"2024-04-04T06:05:54.594227+0000","last_peered":"2024-04-04T06:05:54.594227+0000","last_clean":"2024-04-04T06:05:54.594227+0000","last_became_active":"2024-04-04T06:05:53.368815+0000","last_became_peered":"2024-04-04T06:05:53.368815+0000","last_unstale":"2024-04-04T06:05:54.594227+0000","last_undegraded":"2024-04-04T06:05:54.594227+0000","last_fullsized":"2024-04-04T06:05:54.594227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:37:59.044530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592641+0000","last_change":"2024-04-04T06:05:53.366948+0000","last_active":"2024-04-04T06:05:54.592641+0000","last_peered":"2024-04-04T06:05:54.592641+0000","last_clean":"2024-04-04T06:05:54.592641+0000","last_became_active":"2024-04-04T06:05:53.366550+0000","last_became_peered":"2024-04-04T06:05:53.366550+0000","last_unstale":"2024-04-04T06:05:54.592641+0000","last_undegraded":"2024-04-04T06:05:54.592641+0000","last_fullsized":"2024-04-04T06:05:54.592641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:47:43.615957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592078+0000","last_change":"2024-04-04T06:05:53.376979+0000","last_active":"2024-04-04T06:05:54.592078+0000","last_peered":"2024-04-04T06:05:54.592078+0000","last_clean":"2024-04-04T06:05:54.592078+0000","last_became_active":"2024-04-04T06:05:53.376622+0000","last_became_peered":"2024-04-04T06:05:53.376622+0000","last_unstale":"2024-04-04T06:05:54.592078+0000","last_undegraded":"2024-04-04T06:05:54.592078+0000","last_fullsized":"2024-04-04T06:05:54.592078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:42:00.014108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367238+0000","last_change":"2024-04-04T06:05:53.364063+0000","last_active":"2024-04-04T06:05:54.367238+0000","last_peered":"2024-04-04T06:05:54.367238+0000","last_clean":"2024-04-04T06:05:54.367238+0000","last_became_active":"2024-04-04T06:05:53.363715+0000","last_became_peered":"2024-04-04T06:05:53.363715+0000","last_unstale":"2024-04-04T06:05:54.367238+0000","last_undegraded":"2024-04-04T06:05:54.367238+0000","last_fullsized":"2024-04-04T06:05:54.367238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:21.889411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593467+0000","last_change":"2024-04-04T06:05:53.362035+0000","last_active":"2024-04-04T06:05:54.593467+0000","last_peered":"2024-04-04T06:05:54.593467+0000","last_clean":"2024-04-04T06:05:54.593467+0000","last_became_active":"2024-04-04T06:05:53.361614+0000","last_became_peered":"2024-04-04T06:05:53.361614+0000","last_unstale":"2024-04-04T06:05:54.593467+0000","last_undegraded":"2024-04-04T06:05:54.593467+0000","last_fullsized":"2024-04-04T06:05:54.593467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:17:05.576402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590275+0000","last_change":"2024-04-04T06:05:53.355832+0000","last_active":"2024-04-04T06:05:54.590275+0000","last_peered":"2024-04-04T06:05:54.590275+0000","last_clean":"2024-04-04T06:05:54.590275+0000","last_became_active":"2024-04-04T06:05:53.355508+0000","last_became_peered":"2024-04-04T06:05:53.355508+0000","last_unstale":"2024-04-04T06:05:54.590275+0000","last_undegraded":"2024-04-04T06:05:54.590275+0000","last_fullsized":"2024-04-04T06:05:54.590275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:11:00.324524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362799+0000","last_change":"2024-04-04T06:05:53.353630+0000","last_active":"2024-04-04T06:05:54.362799+0000","last_peered":"2024-04-04T06:05:54.362799+0000","last_clean":"2024-04-04T06:05:54.362799+0000","last_became_active":"2024-04-04T06:05:53.353315+0000","last_became_peered":"2024-04-04T06:05:53.353315+0000","last_unstale":"2024-04-04T06:05:54.362799+0000","last_undegraded":"2024-04-04T06:05:54.362799+0000","last_fullsized":"2024-04-04T06:05:54.362799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.364261+0000","last_change":"2024-04-04T06:05:53.387499+0000","last_active":"2024-04-04T06:05:54.364261+0000","last_peered":"2024-04-04T06:05:54.364261+0000","last_clean":"2024-04-04T06:05:54.364261+0000","last_became_active":"2024-04-04T06:05:53.387187+0000","last_became_peered":"2024-04-04T06:05:53.387187+0000","last_unstale":"2024-04-04T06:05:54.364261+0000","last_undegraded":"2024-04-04T06:05:54.364261+0000","last_fullsized":"2024-04-04T06:05:54.364261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.366558+0000","last_change":"2024-04-04T06:05:53.378705+0000","last_active":"2024-04-04T06:05:54.366558+0000","last_peered":"2024-04-04T06:05:54.366558+0000","last_clean":"2024-04-04T06:05:54.366558+0000","last_became_active":"2024-04-04T06:05:53.378428+0000","last_became_peered":"2024-04-04T06:05:53.378428+0000","last_unstale":"2024-04-04T06:05:54.366558+0000","last_undegraded":"2024-04-04T06:05:54.366558+0000","last_fullsized":"2024-04-04T06:05:54.366558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:17:42.995360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591477+0000","last_change":"2024-04-04T06:05:53.372718+0000","last_active":"2024-04-04T06:05:54.591477+0000","last_peered":"2024-04-04T06:05:54.591477+0000","last_clean":"2024-04-04T06:05:54.591477+0000","last_became_active":"2024-04-04T06:05:53.372458+0000","last_became_peered":"2024-04-04T06:05:53.372458+0000","last_unstale":"2024-04-04T06:05:54.591477+0000","last_undegraded":"2024-04-04T06:05:54.591477+0000","last_fullsized":"2024-04-04T06:05:54.591477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:31.723861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594485+0000","last_change":"2024-04-04T06:05:53.360929+0000","last_active":"2024-04-04T06:05:54.594485+0000","last_peered":"2024-04-04T06:05:54.594485+0000","last_clean":"2024-04-04T06:05:54.594485+0000","last_became_active":"2024-04-04T06:05:53.360661+0000","last_became_peered":"2024-04-04T06:05:53.360661+0000","last_unstale":"2024-04-04T06:05:54.594485+0000","last_undegraded":"2024-04-04T06:05:54.594485+0000","last_fullsized":"2024-04-04T06:05:54.594485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:06:23.755362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591699+0000","last_change":"2024-04-04T06:05:53.374155+0000","last_active":"2024-04-04T06:05:54.591699+0000","last_peered":"2024-04-04T06:05:54.591699+0000","last_clean":"2024-04-04T06:05:54.591699+0000","last_became_active":"2024-04-04T06:05:53.373872+0000","last_became_peered":"2024-04-04T06:05:53.373872+0000","last_unstale":"2024-04-04T06:05:54.591699+0000","last_undegraded":"2024-04-04T06:05:54.591699+0000","last_fullsized":"2024-04-04T06:05:54.591699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:19:20.840644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590730+0000","last_change":"2024-04-04T06:05:53.375944+0000","last_active":"2024-04-04T06:05:54.590730+0000","last_peered":"2024-04-04T06:05:54.590730+0000","last_clean":"2024-04-04T06:05:54.590730+0000","last_became_active":"2024-04-04T06:05:53.375551+0000","last_became_peered":"2024-04-04T06:05:53.375551+0000","last_unstale":"2024-04-04T06:05:54.590730+0000","last_undegraded":"2024-04-04T06:05:54.590730+0000","last_fullsized":"2024-04-04T06:05:54.590730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:33:17.187978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367881+0000","last_change":"2024-04-04T06:05:53.377200+0000","last_active":"2024-04-04T06:05:54.367881+0000","last_peered":"2024-04-04T06:05:54.367881+0000","last_clean":"2024-04-04T06:05:54.367881+0000","last_became_active":"2024-04-04T06:05:53.373063+0000","last_became_peered":"2024-04-04T06:05:53.373063+0000","last_unstale":"2024-04-04T06:05:54.367881+0000","last_undegraded":"2024-04-04T06:05:54.367881+0000","last_fullsized":"2024-04-04T06:05:54.367881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:28:48.569552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592389+0000","last_change":"2024-04-04T06:05:53.358659+0000","last_active":"2024-04-04T06:05:54.592389+0000","last_peered":"2024-04-04T06:05:54.592389+0000","last_clean":"2024-04-04T06:05:54.592389+0000","last_became_active":"2024-04-04T06:05:53.358372+0000","last_became_peered":"2024-04-04T06:05:53.358372+0000","last_unstale":"2024-04-04T06:05:54.592389+0000","last_undegraded":"2024-04-04T06:05:54.592389+0000","last_fullsized":"2024-04-04T06:05:54.592389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:14:41.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365536+0000","last_change":"2024-04-04T06:05:53.381591+0000","last_active":"2024-04-04T06:05:54.365536+0000","last_peered":"2024-04-04T06:05:54.365536+0000","last_clean":"2024-04-04T06:05:54.365536+0000","last_became_active":"2024-04-04T06:05:53.377261+0000","last_became_peered":"2024-04-04T06:05:53.377261+0000","last_unstale":"2024-04-04T06:05:54.365536+0000","last_undegraded":"2024-04-04T06:05:54.365536+0000","last_fullsized":"2024-04-04T06:05:54.365536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:07:53.196090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593367+0000","last_change":"2024-04-04T06:05:53.371961+0000","last_active":"2024-04-04T06:05:54.593367+0000","last_peered":"2024-04-04T06:05:54.593367+0000","last_clean":"2024-04-04T06:05:54.593367+0000","last_became_active":"2024-04-04T06:05:53.371702+0000","last_became_peered":"2024-04-04T06:05:53.371702+0000","last_unstale":"2024-04-04T06:05:54.593367+0000","last_undegraded":"2024-04-04T06:05:54.593367+0000","last_fullsized":"2024-04-04T06:05:54.593367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592985+0000","last_change":"2024-04-04T06:05:53.383384+0000","last_active":"2024-04-04T06:05:54.592985+0000","last_peered":"2024-04-04T06:05:54.592985+0000","last_clean":"2024-04-04T06:05:54.592985+0000","last_became_active":"2024-04-04T06:05:53.382809+0000","last_became_peered":"2024-04-04T06:05:53.382809+0000","last_unstale":"2024-04-04T06:05:54.592985+0000","last_undegraded":"2024-04-04T06:05:54.592985+0000","last_fullsized":"2024-04-04T06:05:54.592985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:15:27.732386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362466+0000","last_change":"2024-04-04T06:05:53.373499+0000","last_active":"2024-04-04T06:05:54.362466+0000","last_peered":"2024-04-04T06:05:54.362466+0000","last_clean":"2024-04-04T06:05:54.362466+0000","last_became_active":"2024-04-04T06:05:53.367324+0000","last_became_peered":"2024-04-04T06:05:53.367324+0000","last_unstale":"2024-04-04T06:05:54.362466+0000","last_undegraded":"2024-04-04T06:05:54.362466+0000","last_fullsized":"2024-04-04T06:05:54.362466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:21:23.535889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593436+0000","last_change":"2024-04-04T06:05:53.371688+0000","last_active":"2024-04-04T06:05:54.593436+0000","last_peered":"2024-04-04T06:05:54.593436+0000","last_clean":"2024-04-04T06:05:54.593436+0000","last_became_active":"2024-04-04T06:05:53.371350+0000","last_became_peered":"2024-04-04T06:05:53.371350+0000","last_unstale":"2024-04-04T06:05:54.593436+0000","last_undegraded":"2024-04-04T06:05:54.593436+0000","last_fullsized":"2024-04-04T06:05:54.593436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:07:53.196090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590942+0000","last_change":"2024-04-04T06:05:53.370489+0000","last_active":"2024-04-04T06:05:54.590942+0000","last_peered":"2024-04-04T06:05:54.590942+0000","last_clean":"2024-04-04T06:05:54.590942+0000","last_became_active":"2024-04-04T06:05:53.367223+0000","last_became_peered":"2024-04-04T06:05:53.367223+0000","last_unstale":"2024-04-04T06:05:54.590942+0000","last_undegraded":"2024-04-04T06:05:54.590942+0000","last_fullsized":"2024-04-04T06:05:54.590942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:28:13.818701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368055+0000","last_change":"2024-04-04T06:05:53.364565+0000","last_active":"2024-04-04T06:05:54.368055+0000","last_peered":"2024-04-04T06:05:54.368055+0000","last_clean":"2024-04-04T06:05:54.368055+0000","last_became_active":"2024-04-04T06:05:53.364292+0000","last_became_peered":"2024-04-04T06:05:53.364292+0000","last_unstale":"2024-04-04T06:05:54.368055+0000","last_undegraded":"2024-04-04T06:05:54.368055+0000","last_fullsized":"2024-04-04T06:05:54.368055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:37:54.850791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590194+0000","last_change":"2024-04-04T06:05:53.366686+0000","last_active":"2024-04-04T06:05:54.590194+0000","last_peered":"2024-04-04T06:05:54.590194+0000","last_clean":"2024-04-04T06:05:54.590194+0000","last_became_active":"2024-04-04T06:05:53.366409+0000","last_became_peered":"2024-04-04T06:05:53.366409+0000","last_unstale":"2024-04-04T06:05:54.590194+0000","last_undegraded":"2024-04-04T06:05:54.590194+0000","last_fullsized":"2024-04-04T06:05:54.590194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:21.889411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591239+0000","last_change":"2024-04-04T06:05:53.361681+0000","last_active":"2024-04-04T06:05:54.591239+0000","last_peered":"2024-04-04T06:05:54.591239+0000","last_clean":"2024-04-04T06:05:54.591239+0000","last_became_active":"2024-04-04T06:05:53.361307+0000","last_became_peered":"2024-04-04T06:05:53.361307+0000","last_unstale":"2024-04-04T06:05:54.591239+0000","last_undegraded":"2024-04-04T06:05:54.591239+0000","last_fullsized":"2024-04-04T06:05:54.591239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:16:20.634573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.364842+0000","last_change":"2024-04-04T06:05:53.361164+0000","last_active":"2024-04-04T06:05:54.364842+0000","last_peered":"2024-04-04T06:05:54.364842+0000","last_clean":"2024-04-04T06:05:54.364842+0000","last_became_active":"2024-04-04T06:05:53.357301+0000","last_became_peered":"2024-04-04T06:05:53.357301+0000","last_unstale":"2024-04-04T06:05:54.364842+0000","last_undegraded":"2024-04-04T06:05:54.364842+0000","last_fullsized":"2024-04-04T06:05:54.364842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:56:10.393146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594020+0000","last_change":"2024-04-04T06:05:52.368795+0000","last_active":"2024-04-04T06:05:54.594020+0000","last_peered":"2024-04-04T06:05:54.594020+0000","last_clean":"2024-04-04T06:05:54.594020+0000","last_became_active":"2024-04-04T06:05:52.368526+0000","last_became_peered":"2024-04-04T06:05:52.368526+0000","last_unstale":"2024-04-04T06:05:54.594020+0000","last_undegraded":"2024-04-04T06:05:54.594020+0000","last_fullsized":"2024-04-04T06:05:54.594020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:45:43.480308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591181+0000","last_change":"2024-04-04T06:05:53.377207+0000","last_active":"2024-04-04T06:05:54.591181+0000","last_peered":"2024-04-04T06:05:54.591181+0000","last_clean":"2024-04-04T06:05:54.591181+0000","last_became_active":"2024-04-04T06:05:53.376853+0000","last_became_peered":"2024-04-04T06:05:53.376853+0000","last_unstale":"2024-04-04T06:05:54.591181+0000","last_undegraded":"2024-04-04T06:05:54.591181+0000","last_fullsized":"2024-04-04T06:05:54.591181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:21:23.535889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591256+0000","last_change":"2024-04-04T06:05:52.372922+0000","last_active":"2024-04-04T06:05:54.591256+0000","last_peered":"2024-04-04T06:05:54.591256+0000","last_clean":"2024-04-04T06:05:54.591256+0000","last_became_active":"2024-04-04T06:05:52.372615+0000","last_became_peered":"2024-04-04T06:05:52.372615+0000","last_unstale":"2024-04-04T06:05:54.591256+0000","last_undegraded":"2024-04-04T06:05:54.591256+0000","last_fullsized":"2024-04-04T06:05:54.591256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:36:38.928830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592633+0000","last_change":"2024-04-04T06:05:52.371943+0000","last_active":"2024-04-04T06:05:54.592633+0000","last_peered":"2024-04-04T06:05:54.592633+0000","last_clean":"2024-04-04T06:05:54.592633+0000","last_became_active":"2024-04-04T06:05:52.371686+0000","last_became_peered":"2024-04-04T06:05:52.371686+0000","last_unstale":"2024-04-04T06:05:54.592633+0000","last_undegraded":"2024-04-04T06:05:54.592633+0000","last_fullsized":"2024-04-04T06:05:54.592633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:52:05.788594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592504+0000","last_change":"2024-04-04T06:05:53.363121+0000","last_active":"2024-04-04T06:05:54.592504+0000","last_peered":"2024-04-04T06:05:54.592504+0000","last_clean":"2024-04-04T06:05:54.592504+0000","last_became_active":"2024-04-04T06:05:53.362861+0000","last_became_peered":"2024-04-04T06:05:53.362861+0000","last_unstale":"2024-04-04T06:05:54.592504+0000","last_undegraded":"2024-04-04T06:05:54.592504+0000","last_fullsized":"2024-04-04T06:05:54.592504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:28:13.818701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365617+0000","last_change":"2024-04-04T06:05:52.371090+0000","last_active":"2024-04-04T06:05:54.365617+0000","last_peered":"2024-04-04T06:05:54.365617+0000","last_clean":"2024-04-04T06:05:54.365617+0000","last_became_active":"2024-04-04T06:05:52.369650+0000","last_became_peered":"2024-04-04T06:05:52.369650+0000","last_unstale":"2024-04-04T06:05:54.365617+0000","last_undegraded":"2024-04-04T06:05:54.365617+0000","last_fullsized":"2024-04-04T06:05:54.365617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:45:43.480308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362432+0000","last_change":"2024-04-04T06:05:53.357537+0000","last_active":"2024-04-04T06:05:54.362432+0000","last_peered":"2024-04-04T06:05:54.362432+0000","last_clean":"2024-04-04T06:05:54.362432+0000","last_became_active":"2024-04-04T06:05:53.357284+0000","last_became_peered":"2024-04-04T06:05:53.357284+0000","last_unstale":"2024-04-04T06:05:54.362432+0000","last_undegraded":"2024-04-04T06:05:54.362432+0000","last_fullsized":"2024-04-04T06:05:54.362432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:28:13.818701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592398+0000","last_change":"2024-04-04T06:05:52.374921+0000","last_active":"2024-04-04T06:05:54.592398+0000","last_peered":"2024-04-04T06:05:54.592398+0000","last_clean":"2024-04-04T06:05:54.592398+0000","last_became_active":"2024-04-04T06:05:52.374555+0000","last_became_peered":"2024-04-04T06:05:52.374555+0000","last_unstale":"2024-04-04T06:05:54.592398+0000","last_undegraded":"2024-04-04T06:05:54.592398+0000","last_fullsized":"2024-04-04T06:05:54.592398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:05.925335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368597+0000","last_change":"2024-04-04T06:05:53.377612+0000","last_active":"2024-04-04T06:05:54.368597+0000","last_peered":"2024-04-04T06:05:54.368597+0000","last_clean":"2024-04-04T06:05:54.368597+0000","last_became_active":"2024-04-04T06:05:53.374258+0000","last_became_peered":"2024-04-04T06:05:53.374258+0000","last_unstale":"2024-04-04T06:05:54.368597+0000","last_undegraded":"2024-04-04T06:05:54.368597+0000","last_fullsized":"2024-04-04T06:05:54.368597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:08:44.769578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368664+0000","last_change":"2024-04-04T06:05:52.373979+0000","last_active":"2024-04-04T06:05:54.368664+0000","last_peered":"2024-04-04T06:05:54.368664+0000","last_clean":"2024-04-04T06:05:54.368664+0000","last_became_active":"2024-04-04T06:05:52.373408+0000","last_became_peered":"2024-04-04T06:05:52.373408+0000","last_unstale":"2024-04-04T06:05:54.368664+0000","last_undegraded":"2024-04-04T06:05:54.368664+0000","last_fullsized":"2024-04-04T06:05:54.368664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:56:09.383441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591338+0000","last_change":"2024-04-04T06:05:53.368728+0000","last_active":"2024-04-04T06:05:54.591338+0000","last_peered":"2024-04-04T06:05:54.591338+0000","last_clean":"2024-04-04T06:05:54.591338+0000","last_became_active":"2024-04-04T06:05:53.367971+0000","last_became_peered":"2024-04-04T06:05:53.367971+0000","last_unstale":"2024-04-04T06:05:54.591338+0000","last_undegraded":"2024-04-04T06:05:54.591338+0000","last_fullsized":"2024-04-04T06:05:54.591338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362602+0000","last_change":"2024-04-04T06:05:52.368574+0000","last_active":"2024-04-04T06:05:54.362602+0000","last_peered":"2024-04-04T06:05:54.362602+0000","last_clean":"2024-04-04T06:05:54.362602+0000","last_became_active":"2024-04-04T06:05:52.367460+0000","last_became_peered":"2024-04-04T06:05:52.367460+0000","last_unstale":"2024-04-04T06:05:54.362602+0000","last_undegraded":"2024-04-04T06:05:54.362602+0000","last_fullsized":"2024-04-04T06:05:54.362602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:56:09.383441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367547+0000","last_change":"2024-04-04T06:05:53.379475+0000","last_active":"2024-04-04T06:05:54.367547+0000","last_peered":"2024-04-04T06:05:54.367547+0000","last_clean":"2024-04-04T06:05:54.367547+0000","last_became_active":"2024-04-04T06:05:53.376075+0000","last_became_peered":"2024-04-04T06:05:53.376075+0000","last_unstale":"2024-04-04T06:05:54.367547+0000","last_undegraded":"2024-04-04T06:05:54.367547+0000","last_fullsized":"2024-04-04T06:05:54.367547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:03:00.104400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591818+0000","last_change":"2024-04-04T06:05:52.370133+0000","last_active":"2024-04-04T06:05:54.591818+0000","last_peered":"2024-04-04T06:05:54.591818+0000","last_clean":"2024-04-04T06:05:54.591818+0000","last_became_active":"2024-04-04T06:05:52.369825+0000","last_became_peered":"2024-04-04T06:05:52.369825+0000","last_unstale":"2024-04-04T06:05:54.591818+0000","last_undegraded":"2024-04-04T06:05:54.591818+0000","last_fullsized":"2024-04-04T06:05:54.591818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:01:02.629926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594503+0000","last_change":"2024-04-04T06:05:53.371020+0000","last_active":"2024-04-04T06:05:54.594503+0000","last_peered":"2024-04-04T06:05:54.594503+0000","last_clean":"2024-04-04T06:05:54.594503+0000","last_became_active":"2024-04-04T06:05:53.370755+0000","last_became_peered":"2024-04-04T06:05:53.370755+0000","last_unstale":"2024-04-04T06:05:54.594503+0000","last_undegraded":"2024-04-04T06:05:54.594503+0000","last_fullsized":"2024-04-04T06:05:54.594503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:54:36.353079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591316+0000","last_change":"2024-04-04T06:05:52.367020+0000","last_active":"2024-04-04T06:05:54.591316+0000","last_peered":"2024-04-04T06:05:54.591316+0000","last_clean":"2024-04-04T06:05:54.591316+0000","last_became_active":"2024-04-04T06:05:52.366654+0000","last_became_peered":"2024-04-04T06:05:52.366654+0000","last_unstale":"2024-04-04T06:05:54.591316+0000","last_undegraded":"2024-04-04T06:05:54.591316+0000","last_fullsized":"2024-04-04T06:05:54.591316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:26:03.163432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590433+0000","last_change":"2024-04-04T06:05:53.373111+0000","last_active":"2024-04-04T06:05:54.590433+0000","last_peered":"2024-04-04T06:05:54.590433+0000","last_clean":"2024-04-04T06:05:54.590433+0000","last_became_active":"2024-04-04T06:05:53.372761+0000","last_became_peered":"2024-04-04T06:05:53.372761+0000","last_unstale":"2024-04-04T06:05:54.590433+0000","last_undegraded":"2024-04-04T06:05:54.590433+0000","last_fullsized":"2024-04-04T06:05:54.590433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:27:06.279930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368191+0000","last_change":"2024-04-04T06:05:52.365334+0000","last_active":"2024-04-04T06:05:54.368191+0000","last_peered":"2024-04-04T06:05:54.368191+0000","last_clean":"2024-04-04T06:05:54.368191+0000","last_became_active":"2024-04-04T06:05:52.363154+0000","last_became_peered":"2024-04-04T06:05:52.363154+0000","last_unstale":"2024-04-04T06:05:54.368191+0000","last_undegraded":"2024-04-04T06:05:54.368191+0000","last_fullsized":"2024-04-04T06:05:54.368191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:53:42.904072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593559+0000","last_change":"2024-04-04T06:05:53.380196+0000","last_active":"2024-04-04T06:05:54.593559+0000","last_peered":"2024-04-04T06:05:54.593559+0000","last_clean":"2024-04-04T06:05:54.593559+0000","last_became_active":"2024-04-04T06:05:53.379800+0000","last_became_peered":"2024-04-04T06:05:53.379800+0000","last_unstale":"2024-04-04T06:05:54.593559+0000","last_undegraded":"2024-04-04T06:05:54.593559+0000","last_fullsized":"2024-04-04T06:05:54.593559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:41:09.510239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591089+0000","last_change":"2024-04-04T06:05:52.377332+0000","last_active":"2024-04-04T06:05:54.591089+0000","last_peered":"2024-04-04T06:05:54.591089+0000","last_clean":"2024-04-04T06:05:54.591089+0000","last_became_active":"2024-04-04T06:05:52.377002+0000","last_became_peered":"2024-04-04T06:05:52.377002+0000","last_unstale":"2024-04-04T06:05:54.591089+0000","last_undegraded":"2024-04-04T06:05:54.591089+0000","last_fullsized":"2024-04-04T06:05:54.591089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:05:57.500385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591711+0000","last_change":"2024-04-04T06:05:53.378591+0000","last_active":"2024-04-04T06:05:54.591711+0000","last_peered":"2024-04-04T06:05:54.591711+0000","last_clean":"2024-04-04T06:05:54.591711+0000","last_became_active":"2024-04-04T06:05:53.378227+0000","last_became_peered":"2024-04-04T06:05:53.378227+0000","last_unstale":"2024-04-04T06:05:54.591711+0000","last_undegraded":"2024-04-04T06:05:54.591711+0000","last_fullsized":"2024-04-04T06:05:54.591711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:42:54.347331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.589595+0000","last_change":"2024-04-04T06:05:52.357745+0000","last_active":"2024-04-04T06:05:54.589595+0000","last_peered":"2024-04-04T06:05:54.589595+0000","last_clean":"2024-04-04T06:05:54.589595+0000","last_became_active":"2024-04-04T06:05:52.357280+0000","last_became_peered":"2024-04-04T06:05:52.357280+0000","last_unstale":"2024-04-04T06:05:54.589595+0000","last_undegraded":"2024-04-04T06:05:54.589595+0000","last_fullsized":"2024-04-04T06:05:54.589595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:45:43.480308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592420+0000","last_change":"2024-04-04T06:05:53.387060+0000","last_active":"2024-04-04T06:05:54.592420+0000","last_peered":"2024-04-04T06:05:54.592420+0000","last_clean":"2024-04-04T06:05:54.592420+0000","last_became_active":"2024-04-04T06:05:53.386804+0000","last_became_peered":"2024-04-04T06:05:53.386804+0000","last_unstale":"2024-04-04T06:05:54.592420+0000","last_undegraded":"2024-04-04T06:05:54.592420+0000","last_fullsized":"2024-04-04T06:05:54.592420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:37:36.681617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367992+0000","last_change":"2024-04-04T06:05:52.362010+0000","last_active":"2024-04-04T06:05:54.367992+0000","last_peered":"2024-04-04T06:05:54.367992+0000","last_clean":"2024-04-04T06:05:54.367992+0000","last_became_active":"2024-04-04T06:05:52.361424+0000","last_became_peered":"2024-04-04T06:05:52.361424+0000","last_unstale":"2024-04-04T06:05:54.367992+0000","last_undegraded":"2024-04-04T06:05:54.367992+0000","last_fullsized":"2024-04-04T06:05:54.367992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:26:03.163432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594202+0000","last_change":"2024-04-04T06:05:53.375944+0000","last_active":"2024-04-04T06:05:54.594202+0000","last_peered":"2024-04-04T06:05:54.594202+0000","last_clean":"2024-04-04T06:05:54.594202+0000","last_became_active":"2024-04-04T06:05:53.375684+0000","last_became_peered":"2024-04-04T06:05:53.375684+0000","last_unstale":"2024-04-04T06:05:54.594202+0000","last_undegraded":"2024-04-04T06:05:54.594202+0000","last_fullsized":"2024-04-04T06:05:54.594202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:02:57.074580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591274+0000","last_change":"2024-04-04T06:05:52.358735+0000","last_active":"2024-04-04T06:05:54.591274+0000","last_peered":"2024-04-04T06:05:54.591274+0000","last_clean":"2024-04-04T06:05:54.591274+0000","last_became_active":"2024-04-04T06:05:52.358203+0000","last_became_peered":"2024-04-04T06:05:52.358203+0000","last_unstale":"2024-04-04T06:05:54.591274+0000","last_undegraded":"2024-04-04T06:05:54.591274+0000","last_fullsized":"2024-04-04T06:05:54.591274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:53:42.904072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594617+0000","last_change":"2024-04-04T06:05:53.388782+0000","last_active":"2024-04-04T06:05:54.594617+0000","last_peered":"2024-04-04T06:05:54.594617+0000","last_clean":"2024-04-04T06:05:54.594617+0000","last_became_active":"2024-04-04T06:05:53.388230+0000","last_became_peered":"2024-04-04T06:05:53.388230+0000","last_unstale":"2024-04-04T06:05:54.594617+0000","last_undegraded":"2024-04-04T06:05:54.594617+0000","last_fullsized":"2024-04-04T06:05:54.594617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:31:19.282600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362887+0000","last_change":"2024-04-04T06:05:52.366026+0000","last_active":"2024-04-04T06:05:54.362887+0000","last_peered":"2024-04-04T06:05:54.362887+0000","last_clean":"2024-04-04T06:05:54.362887+0000","last_became_active":"2024-04-04T06:05:52.365663+0000","last_became_peered":"2024-04-04T06:05:52.365663+0000","last_unstale":"2024-04-04T06:05:54.362887+0000","last_undegraded":"2024-04-04T06:05:54.362887+0000","last_fullsized":"2024-04-04T06:05:54.362887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:30.714156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.366141+0000","last_change":"2024-04-04T06:05:53.381556+0000","last_active":"2024-04-04T06:05:54.366141+0000","last_peered":"2024-04-04T06:05:54.366141+0000","last_clean":"2024-04-04T06:05:54.366141+0000","last_became_active":"2024-04-04T06:05:53.373492+0000","last_became_peered":"2024-04-04T06:05:53.373492+0000","last_unstale":"2024-04-04T06:05:54.366141+0000","last_undegraded":"2024-04-04T06:05:54.366141+0000","last_fullsized":"2024-04-04T06:05:54.366141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:53:19.419468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590914+0000","last_change":"2024-04-04T06:05:52.353584+0000","last_active":"2024-04-04T06:05:54.590914+0000","last_peered":"2024-04-04T06:05:54.590914+0000","last_clean":"2024-04-04T06:05:54.590914+0000","last_became_active":"2024-04-04T06:05:52.353017+0000","last_became_peered":"2024-04-04T06:05:52.353017+0000","last_unstale":"2024-04-04T06:05:54.590914+0000","last_undegraded":"2024-04-04T06:05:54.590914+0000","last_fullsized":"2024-04-04T06:05:54.590914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:58:40.874784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593698+0000","last_change":"2024-04-04T06:05:53.368622+0000","last_active":"2024-04-04T06:05:54.593698+0000","last_peered":"2024-04-04T06:05:54.593698+0000","last_clean":"2024-04-04T06:05:54.593698+0000","last_became_active":"2024-04-04T06:05:53.368311+0000","last_became_peered":"2024-04-04T06:05:53.368311+0000","last_unstale":"2024-04-04T06:05:54.593698+0000","last_undegraded":"2024-04-04T06:05:54.593698+0000","last_fullsized":"2024-04-04T06:05:54.593698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:01:24.602038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591729+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:05:54.591729+0000","last_peered":"2024-04-04T06:05:54.591729+0000","last_clean":"2024-04-04T06:05:54.591729+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:05:54.591729+0000","last_undegraded":"2024-04-04T06:05:54.591729+0000","last_fullsized":"2024-04-04T06:05:54.591729+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365145+0000","last_change":"2024-04-04T06:05:52.356730+0000","last_active":"2024-04-04T06:05:54.365145+0000","last_peered":"2024-04-04T06:05:54.365145+0000","last_clean":"2024-04-04T06:05:54.365145+0000","last_became_active":"2024-04-04T06:05:52.352963+0000","last_became_peered":"2024-04-04T06:05:52.352963+0000","last_unstale":"2024-04-04T06:05:54.365145+0000","last_undegraded":"2024-04-04T06:05:54.365145+0000","last_fullsized":"2024-04-04T06:05:54.365145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:05:06.878796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592869+0000","last_change":"2024-04-04T06:05:53.366506+0000","last_active":"2024-04-04T06:05:54.592869+0000","last_peered":"2024-04-04T06:05:54.592869+0000","last_clean":"2024-04-04T06:05:54.592869+0000","last_became_active":"2024-04-04T06:05:53.365677+0000","last_became_peered":"2024-04-04T06:05:53.365677+0000","last_unstale":"2024-04-04T06:05:54.592869+0000","last_undegraded":"2024-04-04T06:05:54.592869+0000","last_fullsized":"2024-04-04T06:05:54.592869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:10.411661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591456+0000","last_change":"2024-04-04T06:05:52.374798+0000","last_active":"2024-04-04T06:05:54.591456+0000","last_peered":"2024-04-04T06:05:54.591456+0000","last_clean":"2024-04-04T06:05:54.591456+0000","last_became_active":"2024-04-04T06:05:52.373838+0000","last_became_peered":"2024-04-04T06:05:52.373838+0000","last_unstale":"2024-04-04T06:05:54.591456+0000","last_undegraded":"2024-04-04T06:05:54.591456+0000","last_fullsized":"2024-04-04T06:05:54.591456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:51:50.301107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590751+0000","last_change":"2024-04-04T06:05:53.385393+0000","last_active":"2024-04-04T06:05:54.590751+0000","last_peered":"2024-04-04T06:05:54.590751+0000","last_clean":"2024-04-04T06:05:54.590751+0000","last_became_active":"2024-04-04T06:05:53.385062+0000","last_became_peered":"2024-04-04T06:05:53.385062+0000","last_unstale":"2024-04-04T06:05:54.590751+0000","last_undegraded":"2024-04-04T06:05:54.590751+0000","last_fullsized":"2024-04-04T06:05:54.590751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:14:41.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590309+0000","last_change":"2024-04-04T06:05:52.360018+0000","last_active":"2024-04-04T06:05:54.590309+0000","last_peered":"2024-04-04T06:05:54.590309+0000","last_clean":"2024-04-04T06:05:54.590309+0000","last_became_active":"2024-04-04T06:05:52.359735+0000","last_became_peered":"2024-04-04T06:05:52.359735+0000","last_unstale":"2024-04-04T06:05:54.590309+0000","last_undegraded":"2024-04-04T06:05:54.590309+0000","last_fullsized":"2024-04-04T06:05:54.590309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:05:06.878796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.363467+0000","last_change":"2024-04-04T06:05:53.356771+0000","last_active":"2024-04-04T06:05:54.363467+0000","last_peered":"2024-04-04T06:05:54.363467+0000","last_clean":"2024-04-04T06:05:54.363467+0000","last_became_active":"2024-04-04T06:05:53.356431+0000","last_became_peered":"2024-04-04T06:05:53.356431+0000","last_unstale":"2024-04-04T06:05:54.363467+0000","last_undegraded":"2024-04-04T06:05:54.363467+0000","last_fullsized":"2024-04-04T06:05:54.363467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:07:53.196090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368163+0000","last_change":"2024-04-04T06:05:52.370387+0000","last_active":"2024-04-04T06:05:54.368163+0000","last_peered":"2024-04-04T06:05:54.368163+0000","last_clean":"2024-04-04T06:05:54.368163+0000","last_became_active":"2024-04-04T06:05:52.369905+0000","last_became_peered":"2024-04-04T06:05:52.369905+0000","last_unstale":"2024-04-04T06:05:54.368163+0000","last_undegraded":"2024-04-04T06:05:54.368163+0000","last_fullsized":"2024-04-04T06:05:54.368163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:58:40.874784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367307+0000","last_change":"2024-04-04T06:05:53.371957+0000","last_active":"2024-04-04T06:05:54.367307+0000","last_peered":"2024-04-04T06:05:54.367307+0000","last_clean":"2024-04-04T06:05:54.367307+0000","last_became_active":"2024-04-04T06:05:53.371121+0000","last_became_peered":"2024-04-04T06:05:53.371121+0000","last_unstale":"2024-04-04T06:05:54.367307+0000","last_undegraded":"2024-04-04T06:05:54.367307+0000","last_fullsized":"2024-04-04T06:05:54.367307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:31.723861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362920+0000","last_change":"2024-04-04T06:05:52.361392+0000","last_active":"2024-04-04T06:05:54.362920+0000","last_peered":"2024-04-04T06:05:54.362920+0000","last_clean":"2024-04-04T06:05:54.362920+0000","last_became_active":"2024-04-04T06:05:52.360932+0000","last_became_peered":"2024-04-04T06:05:52.360932+0000","last_unstale":"2024-04-04T06:05:54.362920+0000","last_undegraded":"2024-04-04T06:05:54.362920+0000","last_fullsized":"2024-04-04T06:05:54.362920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:20.879706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365934+0000","last_change":"2024-04-04T06:05:53.379438+0000","last_active":"2024-04-04T06:05:54.365934+0000","last_peered":"2024-04-04T06:05:54.365934+0000","last_clean":"2024-04-04T06:05:54.365934+0000","last_became_active":"2024-04-04T06:05:53.378781+0000","last_became_peered":"2024-04-04T06:05:53.378781+0000","last_unstale":"2024-04-04T06:05:54.365934+0000","last_undegraded":"2024-04-04T06:05:54.365934+0000","last_fullsized":"2024-04-04T06:05:54.365934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:18:05.337194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594629+0000","last_change":"2024-04-04T06:05:52.372922+0000","last_active":"2024-04-04T06:05:54.594629+0000","last_peered":"2024-04-04T06:05:54.594629+0000","last_clean":"2024-04-04T06:05:54.594629+0000","last_became_active":"2024-04-04T06:05:52.372696+0000","last_became_peered":"2024-04-04T06:05:52.372696+0000","last_unstale":"2024-04-04T06:05:54.594629+0000","last_undegraded":"2024-04-04T06:05:54.594629+0000","last_fullsized":"2024-04-04T06:05:54.594629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:17:53.847287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362796+0000","last_change":"2024-04-04T06:05:53.373066+0000","last_active":"2024-04-04T06:05:54.362796+0000","last_peered":"2024-04-04T06:05:54.362796+0000","last_clean":"2024-04-04T06:05:54.362796+0000","last_became_active":"2024-04-04T06:05:53.367063+0000","last_became_peered":"2024-04-04T06:05:53.367063+0000","last_unstale":"2024-04-04T06:05:54.362796+0000","last_undegraded":"2024-04-04T06:05:54.362796+0000","last_fullsized":"2024-04-04T06:05:54.362796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:10.411661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.363331+0000","last_change":"2024-04-04T06:05:52.365125+0000","last_active":"2024-04-04T06:05:54.363331+0000","last_peered":"2024-04-04T06:05:54.363331+0000","last_clean":"2024-04-04T06:05:54.363331+0000","last_became_active":"2024-04-04T06:05:52.364821+0000","last_became_peered":"2024-04-04T06:05:52.364821+0000","last_unstale":"2024-04-04T06:05:54.363331+0000","last_undegraded":"2024-04-04T06:05:54.363331+0000","last_fullsized":"2024-04-04T06:05:54.363331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:54:42.610854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593547+0000","last_change":"2024-04-04T06:05:53.375887+0000","last_active":"2024-04-04T06:05:54.593547+0000","last_peered":"2024-04-04T06:05:54.593547+0000","last_clean":"2024-04-04T06:05:54.593547+0000","last_became_active":"2024-04-04T06:05:53.374688+0000","last_became_peered":"2024-04-04T06:05:53.374688+0000","last_unstale":"2024-04-04T06:05:54.593547+0000","last_undegraded":"2024-04-04T06:05:54.593547+0000","last_fullsized":"2024-04-04T06:05:54.593547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:45:21.548886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592567+0000","last_change":"2024-04-04T06:05:52.374804+0000","last_active":"2024-04-04T06:05:54.592567+0000","last_peered":"2024-04-04T06:05:54.592567+0000","last_clean":"2024-04-04T06:05:54.592567+0000","last_became_active":"2024-04-04T06:05:52.374476+0000","last_became_peered":"2024-04-04T06:05:52.374476+0000","last_unstale":"2024-04-04T06:05:54.592567+0000","last_undegraded":"2024-04-04T06:05:54.592567+0000","last_fullsized":"2024-04-04T06:05:54.592567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:21:06.579458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590233+0000","last_change":"2024-04-04T06:05:53.358691+0000","last_active":"2024-04-04T06:05:54.590233+0000","last_peered":"2024-04-04T06:05:54.590233+0000","last_clean":"2024-04-04T06:05:54.590233+0000","last_became_active":"2024-04-04T06:05:53.357996+0000","last_became_peered":"2024-04-04T06:05:53.357996+0000","last_unstale":"2024-04-04T06:05:54.590233+0000","last_undegraded":"2024-04-04T06:05:54.590233+0000","last_fullsized":"2024-04-04T06:05:54.590233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:55:58.689192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592379+0000","last_change":"2024-04-04T06:05:52.370047+0000","last_active":"2024-04-04T06:05:54.592379+0000","last_peered":"2024-04-04T06:05:54.592379+0000","last_clean":"2024-04-04T06:05:54.592379+0000","last_became_active":"2024-04-04T06:05:52.369741+0000","last_became_peered":"2024-04-04T06:05:52.369741+0000","last_unstale":"2024-04-04T06:05:54.592379+0000","last_undegraded":"2024-04-04T06:05:54.592379+0000","last_fullsized":"2024-04-04T06:05:54.592379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:36:38.928830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.364951+0000","last_change":"2024-04-04T06:05:53.364563+0000","last_active":"2024-04-04T06:05:54.364951+0000","last_peered":"2024-04-04T06:05:54.364951+0000","last_clean":"2024-04-04T06:05:54.364951+0000","last_became_active":"2024-04-04T06:05:53.362702+0000","last_became_peered":"2024-04-04T06:05:53.362702+0000","last_unstale":"2024-04-04T06:05:54.364951+0000","last_undegraded":"2024-04-04T06:05:54.364951+0000","last_fullsized":"2024-04-04T06:05:54.364951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:54:43.620559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362753+0000","last_change":"2024-04-04T06:05:52.370003+0000","last_active":"2024-04-04T06:05:54.362753+0000","last_peered":"2024-04-04T06:05:54.362753+0000","last_clean":"2024-04-04T06:05:54.362753+0000","last_became_active":"2024-04-04T06:05:52.369724+0000","last_became_peered":"2024-04-04T06:05:52.369724+0000","last_unstale":"2024-04-04T06:05:54.362753+0000","last_undegraded":"2024-04-04T06:05:54.362753+0000","last_fullsized":"2024-04-04T06:05:54.362753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:58:40.874784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592885+0000","last_change":"2024-04-04T06:05:53.361642+0000","last_active":"2024-04-04T06:05:54.592885+0000","last_peered":"2024-04-04T06:05:54.592885+0000","last_clean":"2024-04-04T06:05:54.592885+0000","last_became_active":"2024-04-04T06:05:53.360951+0000","last_became_peered":"2024-04-04T06:05:53.360951+0000","last_unstale":"2024-04-04T06:05:54.592885+0000","last_undegraded":"2024-04-04T06:05:54.592885+0000","last_fullsized":"2024-04-04T06:05:54.592885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:21:23.535889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362582+0000","last_change":"2024-04-04T06:05:52.371223+0000","last_active":"2024-04-04T06:05:54.362582+0000","last_peered":"2024-04-04T06:05:54.362582+0000","last_clean":"2024-04-04T06:05:54.362582+0000","last_became_active":"2024-04-04T06:05:52.370916+0000","last_became_peered":"2024-04-04T06:05:52.370916+0000","last_unstale":"2024-04-04T06:05:54.362582+0000","last_undegraded":"2024-04-04T06:05:54.362582+0000","last_fullsized":"2024-04-04T06:05:54.362582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:53:42.904072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593889+0000","last_change":"2024-04-04T06:05:53.371605+0000","last_active":"2024-04-04T06:05:54.593889+0000","last_peered":"2024-04-04T06:05:54.593889+0000","last_clean":"2024-04-04T06:05:54.593889+0000","last_became_active":"2024-04-04T06:05:53.371328+0000","last_became_peered":"2024-04-04T06:05:53.371328+0000","last_unstale":"2024-04-04T06:05:54.593889+0000","last_undegraded":"2024-04-04T06:05:54.593889+0000","last_fullsized":"2024-04-04T06:05:54.593889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:18:05.337194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593483+0000","last_change":"2024-04-04T06:05:52.368424+0000","last_active":"2024-04-04T06:05:54.593483+0000","last_peered":"2024-04-04T06:05:54.593483+0000","last_clean":"2024-04-04T06:05:54.593483+0000","last_became_active":"2024-04-04T06:05:52.368164+0000","last_became_peered":"2024-04-04T06:05:52.368164+0000","last_unstale":"2024-04-04T06:05:54.593483+0000","last_undegraded":"2024-04-04T06:05:54.593483+0000","last_fullsized":"2024-04-04T06:05:54.593483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:54:18.736812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591138+0000","last_change":"2024-04-04T06:05:53.373894+0000","last_active":"2024-04-04T06:05:54.591138+0000","last_peered":"2024-04-04T06:05:54.591138+0000","last_clean":"2024-04-04T06:05:54.591138+0000","last_became_active":"2024-04-04T06:05:53.373615+0000","last_became_peered":"2024-04-04T06:05:53.373615+0000","last_unstale":"2024-04-04T06:05:54.591138+0000","last_undegraded":"2024-04-04T06:05:54.591138+0000","last_fullsized":"2024-04-04T06:05:54.591138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:03:39.636711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365635+0000","last_change":"2024-04-04T06:05:53.378971+0000","last_active":"2024-04-04T06:05:54.365635+0000","last_peered":"2024-04-04T06:05:54.365635+0000","last_clean":"2024-04-04T06:05:54.365635+0000","last_became_active":"2024-04-04T06:05:53.377287+0000","last_became_peered":"2024-04-04T06:05:53.377287+0000","last_unstale":"2024-04-04T06:05:54.365635+0000","last_undegraded":"2024-04-04T06:05:54.365635+0000","last_fullsized":"2024-04-04T06:05:54.365635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:17:05.576402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594399+0000","last_change":"2024-04-04T06:05:52.376155+0000","last_active":"2024-04-04T06:05:54.594399+0000","last_peered":"2024-04-04T06:05:54.594399+0000","last_clean":"2024-04-04T06:05:54.594399+0000","last_became_active":"2024-04-04T06:05:52.375772+0000","last_became_peered":"2024-04-04T06:05:52.375772+0000","last_unstale":"2024-04-04T06:05:54.594399+0000","last_undegraded":"2024-04-04T06:05:54.594399+0000","last_fullsized":"2024-04-04T06:05:54.594399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:40:09.906862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365922+0000","last_change":"2024-04-04T06:05:53.381179+0000","last_active":"2024-04-04T06:05:54.365922+0000","last_peered":"2024-04-04T06:05:54.365922+0000","last_clean":"2024-04-04T06:05:54.365922+0000","last_became_active":"2024-04-04T06:05:53.374859+0000","last_became_peered":"2024-04-04T06:05:53.374859+0000","last_unstale":"2024-04-04T06:05:54.365922+0000","last_undegraded":"2024-04-04T06:05:54.365922+0000","last_fullsized":"2024-04-04T06:05:54.365922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:20:24.966139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593848+0000","last_change":"2024-04-04T06:05:52.378067+0000","last_active":"2024-04-04T06:05:54.593848+0000","last_peered":"2024-04-04T06:05:54.593848+0000","last_clean":"2024-04-04T06:05:54.593848+0000","last_became_active":"2024-04-04T06:05:52.377777+0000","last_became_peered":"2024-04-04T06:05:52.377777+0000","last_unstale":"2024-04-04T06:05:54.593848+0000","last_undegraded":"2024-04-04T06:05:54.593848+0000","last_fullsized":"2024-04-04T06:05:54.593848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:01:02.629926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590922+0000","last_change":"2024-04-04T06:05:53.368705+0000","last_active":"2024-04-04T06:05:54.590922+0000","last_peered":"2024-04-04T06:05:54.590922+0000","last_clean":"2024-04-04T06:05:54.590922+0000","last_became_active":"2024-04-04T06:05:53.368366+0000","last_became_peered":"2024-04-04T06:05:53.368366+0000","last_unstale":"2024-04-04T06:05:54.590922+0000","last_undegraded":"2024-04-04T06:05:54.590922+0000","last_fullsized":"2024-04-04T06:05:54.590922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:34.766930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593814+0000","last_change":"2024-04-04T06:05:52.367095+0000","last_active":"2024-04-04T06:05:54.593814+0000","last_peered":"2024-04-04T06:05:54.593814+0000","last_clean":"2024-04-04T06:05:54.593814+0000","last_became_active":"2024-04-04T06:05:52.366316+0000","last_became_peered":"2024-04-04T06:05:52.366316+0000","last_unstale":"2024-04-04T06:05:54.593814+0000","last_undegraded":"2024-04-04T06:05:54.593814+0000","last_fullsized":"2024-04-04T06:05:54.593814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:05:06.878796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593646+0000","last_change":"2024-04-04T06:05:53.375226+0000","last_active":"2024-04-04T06:05:54.593646+0000","last_peered":"2024-04-04T06:05:54.593646+0000","last_clean":"2024-04-04T06:05:54.593646+0000","last_became_active":"2024-04-04T06:05:53.374921+0000","last_became_peered":"2024-04-04T06:05:53.374921+0000","last_unstale":"2024-04-04T06:05:54.593646+0000","last_undegraded":"2024-04-04T06:05:54.593646+0000","last_fullsized":"2024-04-04T06:05:54.593646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:20:24.966139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591478+0000","last_change":"2024-04-04T06:05:52.370572+0000","last_active":"2024-04-04T06:05:54.591478+0000","last_peered":"2024-04-04T06:05:54.591478+0000","last_clean":"2024-04-04T06:05:54.591478+0000","last_became_active":"2024-04-04T06:05:52.369378+0000","last_became_peered":"2024-04-04T06:05:52.369378+0000","last_unstale":"2024-04-04T06:05:54.591478+0000","last_undegraded":"2024-04-04T06:05:54.591478+0000","last_fullsized":"2024-04-04T06:05:54.591478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:45:47.508028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":806912,"data_stored":663433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":74095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":74095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":72483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":663489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":72573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":72609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:05:58.782 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-04T06:05:59.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.033+0000 7f081ca00640 1 Processor -- start 2024-04-04T06:05:59.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.034+0000 7f081ca00640 1 -- start start 2024-04-04T06:05:59.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.035+0000 7f081ca00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0818153580 0x7f08180d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:59.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.035+0000 7f081ca00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08180afab0 con 0x7f08180d07c0 2024-04-04T06:05:59.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.035+0000 7f081ca00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f081815cc80 con 0x7f08180af4b0 2024-04-04T06:05:59.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.035+0000 7f081ca00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f08180d0c00 con 0x7f0818153580 2024-04-04T06:05:59.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.035+0000 7f080d400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f08180d07c0 0x7f08180aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43528/0 (socket says 172.21.15.159:43528) 2024-04-04T06:05:59.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.035+0000 7f080d400640 1 -- 172.21.15.159:0/4164430245 learned_addr learned my addr 172.21.15.159:0/4164430245 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:05:59.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.036+0000 7f0815400640 1 -- 172.21.15.159:0/4164430245 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1406953137 0 0) 0x7f08180afab0 con 0x7f08180d07c0 2024-04-04T06:05:59.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.036+0000 7f0815400640 1 -- 172.21.15.159:0/4164430245 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07fc003520 con 0x7f08180d07c0 2024-04-04T06:05:59.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.036+0000 7f0815e00640 1 --2- 172.21.15.159:0/4164430245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0818153580 0x7f08180d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:59.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.036+0000 7f0815400640 1 -- 172.21.15.159:0/4164430245 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3400217289 0 0) 0x7f07fc003520 con 0x7f08180d07c0 2024-04-04T06:05:59.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.036+0000 7f0815400640 1 -- 172.21.15.159:0/4164430245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0818153580 msgr2=0x7f08180d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:59.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.036+0000 7f0815400640 1 --2- 172.21.15.159:0/4164430245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0818153580 0x7f08180d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:59.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.036+0000 7f0815400640 1 -- 172.21.15.159:0/4164430245 >> v1:172.21.15.178:6789/0 conn(0x7f08180af4b0 legacy=0x7f08181600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:59.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.036+0000 7f0815400640 1 -- 172.21.15.159:0/4164430245 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081815da00 con 0x7f08180d07c0 2024-04-04T06:05:59.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.036+0000 7f0815400640 1 -- 172.21.15.159:0/4164430245 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f0800001670 con 0x7f08180d07c0 2024-04-04T06:05:59.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.036+0000 7f0815400640 1 -- 172.21.15.159:0/4164430245 >> v1:172.21.15.159:6789/0 conn(0x7f08180d07c0 legacy=0x7f08180aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:59.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.036+0000 7f0815400640 1 --2- 172.21.15.159:0/4164430245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07fc0039e0 0x7f07fc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:59.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.036+0000 7f0815400640 1 --2- 172.21.15.159:0/4164430245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07fc004800 0x7f07fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:59.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.036+0000 7f0815400640 1 --2- 172.21.15.159:0/4164430245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0818153580 0x7f07fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:59.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.036+0000 7f0815400640 1 -- 172.21.15.159:0/4164430245 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f08180d0c00 con 0x7f07fc004800 2024-04-04T06:05:59.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.036+0000 7f0815400640 1 -- 172.21.15.159:0/4164430245 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc005520 con 0x7f07fc0039e0 2024-04-04T06:05:59.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.036+0000 7f0815400640 1 -- 172.21.15.159:0/4164430245 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f07fc00bf00 con 0x7f0818153580 2024-04-04T06:05:59.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.036+0000 7f080d400640 1 --2- 172.21.15.159:0/4164430245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07fc004800 0x7f07fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:59.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.037+0000 7f0815e00640 1 --2- 172.21.15.159:0/4164430245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07fc0039e0 0x7f07fc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:59.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.037+0000 7f080d400640 1 -- 172.21.15.159:0/4164430245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0818153580 msgr2=0x7f07fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:59.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.037+0000 7f080d400640 1 --2- 172.21.15.159:0/4164430245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0818153580 0x7f07fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:59.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.037+0000 7f080d400640 1 -- 172.21.15.159:0/4164430245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07fc0039e0 msgr2=0x7f07fc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:59.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.037+0000 7f080d400640 1 --2- 172.21.15.159:0/4164430245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07fc0039e0 0x7f07fc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:59.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.037+0000 7f080d400640 1 -- 172.21.15.159:0/4164430245 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07fc003520 con 0x7f07fc004800 2024-04-04T06:05:59.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.038+0000 7f080d400640 1 --2- 172.21.15.159:0/4164430245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07fc004800 0x7f07fc004c80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f080000c210 tx=0x7f07fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c29882a04b842d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:59.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.038+0000 7f0815400640 1 -- 172.21.15.159:0/4164430245 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800007d30 con 0x7f07fc004800 2024-04-04T06:05:59.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.038+0000 7f0815400640 1 -- 172.21.15.159:0/4164430245 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0800030070 con 0x7f07fc004800 2024-04-04T06:05:59.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.038+0000 7f0815400640 1 -- 172.21.15.159:0/4164430245 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080002cd60 con 0x7f07fc004800 2024-04-04T06:05:59.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.038+0000 7f081ca00640 1 -- 172.21.15.159:0/4164430245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07fc004800 msgr2=0x7f07fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:59.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.038+0000 7f081ca00640 1 --2- 172.21.15.159:0/4164430245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07fc004800 0x7f07fc004c80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f080000c210 tx=0x7f07fc005830 comp rx=0 tx=0).stop 2024-04-04T06:05:59.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.038+0000 7f0816800640 1 -- 172.21.15.159:0/4164430245 reap_dead start 2024-04-04T06:05:59.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.039+0000 7f081ca00640 1 -- 172.21.15.159:0/4164430245 shutdown_connections 2024-04-04T06:05:59.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.039+0000 7f081ca00640 1 -- 172.21.15.159:0/4164430245 >> 172.21.15.159:0/4164430245 conn(0x7f081806b750 msgr2=0x7f081806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:59.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.039+0000 7f081ca00640 1 -- 172.21.15.159:0/4164430245 shutdown_connections 2024-04-04T06:05:59.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.039+0000 7f081ca00640 1 -- 172.21.15.159:0/4164430245 wait complete. 2024-04-04T06:05:59.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.040+0000 7f081ca00640 1 Processor -- start 2024-04-04T06:05:59.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.040+0000 7f081ca00640 1 -- start start 2024-04-04T06:05:59.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.041+0000 7f081ca00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07fc004800 0x7f08181e62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:59.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.041+0000 7f081ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08181e4d00 0x7f08180b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:59.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.041+0000 7f081ca00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f081814e510 0x7f081814e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:59.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.041+0000 7f081ca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f081815d4e0 con 0x7f08181e4d00 2024-04-04T06:05:59.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.041+0000 7f081ca00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f081815d200 con 0x7f081814e510 2024-04-04T06:05:59.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.041+0000 7f081ca00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0818153960 con 0x7f07fc004800 2024-04-04T06:05:59.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.041+0000 7f080d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08181e4d00 0x7f08180b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:59.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.042+0000 7f080d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08181e4d00 0x7f08180b0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43360/0 (socket says 172.21.15.159:43360) 2024-04-04T06:05:59.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.042+0000 7f080d400640 1 -- 172.21.15.159:0/3742481835 learned_addr learned my addr 172.21.15.159:0/3742481835 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:59.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.042+0000 7f0815e00640 1 --2- 172.21.15.159:0/3742481835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07fc004800 0x7f08181e62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:59.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.042+0000 7f0816800640 1 --2- 172.21.15.159:0/3742481835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f081814e510 0x7f081814e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:59.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.042+0000 7f080d400640 1 -- 172.21.15.159:0/3742481835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07fc004800 msgr2=0x7f08181e62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:59.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.042+0000 7f080d400640 1 --2- 172.21.15.159:0/3742481835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07fc004800 0x7f08181e62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:59.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.042+0000 7f080d400640 1 -- 172.21.15.159:0/3742481835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f081814e510 msgr2=0x7f081814e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:59.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.042+0000 7f080d400640 1 --2- 172.21.15.159:0/3742481835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f081814e510 0x7f081814e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:59.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.042+0000 7f080d400640 1 -- 172.21.15.159:0/3742481835 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08181e5200 con 0x7f08181e4d00 2024-04-04T06:05:59.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.042+0000 7f0815e00640 1 --2- 172.21.15.159:0/3742481835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07fc004800 0x7f08181e62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:05:59.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.042+0000 7f0816800640 1 --2- 172.21.15.159:0/3742481835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f081814e510 0x7f081814e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:05:59.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.042+0000 7f080d400640 1 --2- 172.21.15.159:0/3742481835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08181e4d00 0x7f08180b0ae0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f080002c570 tx=0x7f0800001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:59.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.042+0000 7f080ea00640 1 -- 172.21.15.159:0/3742481835 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800004bf0 con 0x7f08181e4d00 2024-04-04T06:05:59.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.043+0000 7f080ea00640 1 -- 172.21.15.159:0/3742481835 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0800004db0 con 0x7f08181e4d00 2024-04-04T06:05:59.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.043+0000 7f080ea00640 1 -- 172.21.15.159:0/3742481835 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08000072b0 con 0x7f08181e4d00 2024-04-04T06:05:59.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.043+0000 7f081ca00640 1 -- 172.21.15.159:0/3742481835 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08181e54e0 con 0x7f08181e4d00 2024-04-04T06:05:59.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.043+0000 7f081ca00640 1 -- 172.21.15.159:0/3742481835 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08181e5aa0 con 0x7f08181e4d00 2024-04-04T06:05:59.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.044+0000 7f081ca00640 1 -- 172.21.15.159:0/3742481835 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08180685c0 con 0x7f08181e4d00 2024-04-04T06:05:59.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.044+0000 7f080ea00640 1 -- 172.21.15.159:0/3742481835 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0800048020 con 0x7f08181e4d00 2024-04-04T06:05:59.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.045+0000 7f080ea00640 1 --2- 172.21.15.159:0/3742481835 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f07f406c920 0x7f07f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:59.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.045+0000 7f080ea00640 1 -- 172.21.15.159:0/3742481835 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f08000ad100 con 0x7f08181e4d00 2024-04-04T06:05:59.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.046+0000 7f0815e00640 1 --2- 172.21.15.159:0/3742481835 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f07f406c920 0x7f07f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:59.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.047+0000 7f0815e00640 1 --2- 172.21.15.159:0/3742481835 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f07f406c920 0x7f07f406ede0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f08180cfc70 tx=0x7f0808008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:59.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.049+0000 7f080ea00640 1 -- 172.21.15.159:0/3742481835 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f080007b490 con 0x7f08181e4d00 2024-04-04T06:05:59.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.327+0000 7f081ca00640 1 -- 172.21.15.159:0/3742481835 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f081806ec40 con 0x7f08181e4d00 2024-04-04T06:05:59.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.328+0000 7f080ea00640 1 -- 172.21.15.159:0/3742481835 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1170 (secure 0 0 0) 0x7f080007ad90 con 0x7f08181e4d00 2024-04-04T06:05:59.323 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:05:59.323 INFO:teuthology.orchestra.run.smithi159.stdout:{"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":31,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":21,"num_osds":8,"num_up_osds":8,"osd_up_since":1712210741,"num_in_osds":8,"osd_in_since":1712210732,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":47},{"state_name":"creating+peering","count":26},{"state_name":"active+clean","count":24}],"num_pgs":97,"num_pools":3,"num_objects":3,"data_bytes":590390,"bytes_used":241029120,"bytes_avail":772853084160,"bytes_total":773094113280,"unknown_pgs_ratio":0.48453608155250549,"inactive_pgs_ratio":0.2680412232875824,"write_bytes_sec":189,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4293}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-04-04T06:05:27.670157+0000","services":{}},"progress_events":{}} 2024-04-04T06:05:59.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.331+0000 7f081ca00640 1 -- 172.21.15.159:0/3742481835 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f07f406c920 msgr2=0x7f07f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:59.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.331+0000 7f081ca00640 1 --2- 172.21.15.159:0/3742481835 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f07f406c920 0x7f07f406ede0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f08180cfc70 tx=0x7f0808008040 comp rx=0 tx=0).stop 2024-04-04T06:05:59.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.331+0000 7f081ca00640 1 -- 172.21.15.159:0/3742481835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08181e4d00 msgr2=0x7f08180b0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:59.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.331+0000 7f081ca00640 1 --2- 172.21.15.159:0/3742481835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08181e4d00 0x7f08180b0ae0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f080002c570 tx=0x7f0800001a40 comp rx=0 tx=0).stop 2024-04-04T06:05:59.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.331+0000 7f081ca00640 1 -- 172.21.15.159:0/3742481835 shutdown_connections 2024-04-04T06:05:59.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.331+0000 7f081ca00640 1 --2- 172.21.15.159:0/3742481835 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f07f406c920 0x7f07f406ede0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:59.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.331+0000 7f081ca00640 1 --2- 172.21.15.159:0/3742481835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f081814e510 0x7f081814e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:59.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.331+0000 7f081ca00640 1 --2- 172.21.15.159:0/3742481835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08181e4d00 0x7f08180b0ae0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:59.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.331+0000 7f081ca00640 1 --2- 172.21.15.159:0/3742481835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07fc004800 0x7f08181e62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:59.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.331+0000 7f081ca00640 1 -- 172.21.15.159:0/3742481835 >> 172.21.15.159:0/3742481835 conn(0x7f081806b750 msgr2=0x7f08180b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:59.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.331+0000 7f081ca00640 1 -- 172.21.15.159:0/3742481835 shutdown_connections 2024-04-04T06:05:59.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.331+0000 7f081ca00640 1 -- 172.21.15.159:0/3742481835 wait complete. 2024-04-04T06:05:59.345 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-04T06:05:59.345 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:05:59.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.556+0000 7fb814a00640 1 Processor -- start 2024-04-04T06:05:59.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.556+0000 7fb814a00640 1 -- start start 2024-04-04T06:05:59.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.558+0000 7fb814a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb810161b00 0x7fb810162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:59.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.558+0000 7fb814a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb8100d0d60 0x7fb8100b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:59.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.558+0000 7fb814a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb810071b60 con 0x7fb8100ad120 2024-04-04T06:05:59.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.558+0000 7fb814a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fb810161ee0 con 0x7fb810161b00 2024-04-04T06:05:59.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.558+0000 7fb814a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb8100b3450 con 0x7fb8100d0d60 2024-04-04T06:05:59.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.558+0000 7fb80de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb810161b00 0x7fb810162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:59.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.558+0000 7fb80de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb810161b00 0x7fb810162ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43362/0 (socket says 172.21.15.159:43362) 2024-04-04T06:05:59.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.558+0000 7fb80de00640 1 -- 172.21.15.159:0/2384097801 learned_addr learned my addr 172.21.15.159:0/2384097801 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:59.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.558+0000 7fb80de00640 1 -- 172.21.15.159:0/2384097801 >> v2:172.21.15.178:3300/0 conn(0x7fb8100d0d60 msgr2=0x7fb8100b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:59.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.558+0000 7fb80de00640 1 --2- 172.21.15.159:0/2384097801 >> v2:172.21.15.178:3300/0 conn(0x7fb8100d0d60 0x7fb8100b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:59.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.558+0000 7fb80de00640 1 -- 172.21.15.159:0/2384097801 >> v1:172.21.15.178:6789/0 conn(0x7fb8100ad120 legacy=0x7fb8100d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:05:59.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.558+0000 7fb80de00640 1 -- 172.21.15.159:0/2384097801 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb810071e40 con 0x7fb810161b00 2024-04-04T06:05:59.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.559+0000 7fb80de00640 1 --2- 172.21.15.159:0/2384097801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb810161b00 0x7fb810162ab0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb8100b63b0 tx=0x7fb80002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b85178d6f88920f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:59.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.560+0000 7fb80ca00640 1 -- 172.21.15.159:0/2384097801 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800035020 con 0x7fb810161b00 2024-04-04T06:05:59.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.560+0000 7fb80ca00640 1 -- 172.21.15.159:0/2384097801 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb800031640 con 0x7fb810161b00 2024-04-04T06:05:59.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.560+0000 7fb80ca00640 1 -- 172.21.15.159:0/2384097801 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80002d040 con 0x7fb810161b00 2024-04-04T06:05:59.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.560+0000 7fb814a00640 1 -- 172.21.15.159:0/2384097801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb810161b00 msgr2=0x7fb810162ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:59.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.560+0000 7fb814a00640 1 --2- 172.21.15.159:0/2384097801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb810161b00 0x7fb810162ab0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb8100b63b0 tx=0x7fb80002b8f0 comp rx=0 tx=0).stop 2024-04-04T06:05:59.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.561+0000 7fb814a00640 1 -- 172.21.15.159:0/2384097801 shutdown_connections 2024-04-04T06:05:59.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.561+0000 7fb814a00640 1 --2- 172.21.15.159:0/2384097801 >> v2:172.21.15.178:3300/0 conn(0x7fb8100d0d60 0x7fb8100b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:59.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.561+0000 7fb814a00640 1 --2- 172.21.15.159:0/2384097801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb810161b00 0x7fb810162ab0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:59.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.561+0000 7fb814a00640 1 -- 172.21.15.159:0/2384097801 >> 172.21.15.159:0/2384097801 conn(0x7fb81006b990 msgr2=0x7fb81006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:59.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.561+0000 7fb814a00640 1 -- 172.21.15.159:0/2384097801 shutdown_connections 2024-04-04T06:05:59.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.561+0000 7fb814a00640 1 -- 172.21.15.159:0/2384097801 wait complete. 2024-04-04T06:05:59.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.562+0000 7fb814a00640 1 Processor -- start 2024-04-04T06:05:59.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.562+0000 7fb814a00640 1 -- start start 2024-04-04T06:05:59.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.563+0000 7fb814a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8100ad120 0x7fb810156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:59.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.563+0000 7fb814a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8100d0d60 0x7fb81015ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:59.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.563+0000 7fb814a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb810161b00 0x7fb8100b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:59.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.563+0000 7fb814a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb810162ff0 con 0x7fb810161b00 2024-04-04T06:05:59.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.563+0000 7fb814a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8100b3730 con 0x7fb8100d0d60 2024-04-04T06:05:59.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.563+0000 7fb814a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb8100717f0 con 0x7fb8100ad120 2024-04-04T06:05:59.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.564+0000 7fb80e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb810161b00 0x7fb8100b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:59.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.564+0000 7fb80e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb810161b00 0x7fb8100b1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43374/0 (socket says 172.21.15.159:43374) 2024-04-04T06:05:59.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.564+0000 7fb80e800640 1 -- 172.21.15.159:0/1578390563 learned_addr learned my addr 172.21.15.159:0/1578390563 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:05:59.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.564+0000 7fb80de00640 1 --2- 172.21.15.159:0/1578390563 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8100ad120 0x7fb810156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:59.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.564+0000 7fb80e800640 1 -- 172.21.15.159:0/1578390563 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8100ad120 msgr2=0x7fb810156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:59.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.564+0000 7fb80e800640 1 --2- 172.21.15.159:0/1578390563 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8100ad120 0x7fb810156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:59.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.564+0000 7fb80e800640 1 -- 172.21.15.159:0/1578390563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8100d0d60 msgr2=0x7fb81015ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:59.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.564+0000 7fb80e800640 1 --2- 172.21.15.159:0/1578390563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8100d0d60 0x7fb81015ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:59.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.564+0000 7fb80e800640 1 -- 172.21.15.159:0/1578390563 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81015f720 con 0x7fb810161b00 2024-04-04T06:05:59.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.565+0000 7fb80e800640 1 --2- 172.21.15.159:0/1578390563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb810161b00 0x7fb8100b1ee0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb80800b9e0 tx=0x7fb80800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:59.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.565+0000 7fb806000640 1 -- 172.21.15.159:0/1578390563 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb808019070 con 0x7fb810161b00 2024-04-04T06:05:59.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.565+0000 7fb806000640 1 -- 172.21.15.159:0/1578390563 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb80800cb20 con 0x7fb810161b00 2024-04-04T06:05:59.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.565+0000 7fb814a00640 1 -- 172.21.15.159:0/1578390563 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb81015fa00 con 0x7fb810161b00 2024-04-04T06:05:59.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.565+0000 7fb806000640 1 -- 172.21.15.159:0/1578390563 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb808014440 con 0x7fb810161b00 2024-04-04T06:05:59.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.565+0000 7fb814a00640 1 -- 172.21.15.159:0/1578390563 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb81015ffc0 con 0x7fb810161b00 2024-04-04T06:05:59.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.566+0000 7fb814a00640 1 -- 172.21.15.159:0/1578390563 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8100685c0 con 0x7fb810161b00 2024-04-04T06:05:59.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.567+0000 7fb806000640 1 -- 172.21.15.159:0/1578390563 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb80801c070 con 0x7fb810161b00 2024-04-04T06:05:59.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.568+0000 7fb806000640 1 --2- 172.21.15.159:0/1578390563 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb7f006c920 0x7fb7f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:05:59.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.568+0000 7fb806000640 1 -- 172.21.15.159:0/1578390563 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb80808b7a0 con 0x7fb810161b00 2024-04-04T06:05:59.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.569+0000 7fb80de00640 1 --2- 172.21.15.159:0/1578390563 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb7f006c920 0x7fb7f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:05:59.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.570+0000 7fb80de00640 1 --2- 172.21.15.159:0/1578390563 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb7f006c920 0x7fb7f006ede0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb800036000 tx=0x7fb80003c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:05:59.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.573+0000 7fb806000640 1 -- 172.21.15.159:0/1578390563 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb808059c60 con 0x7fb810161b00 2024-04-04T06:05:59.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.766+0000 7fb814a00640 1 -- 172.21.15.159:0/1578390563 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb81006f3a0 con 0x7fb7f006c920 2024-04-04T06:05:59.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.796+0000 7fb806000640 1 -- 172.21.15.159:0/1578390563 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7fb81006f3a0 con 0x7fb7f006c920 2024-04-04T06:05:59.795 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:05:59.796 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:05:59.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.801+0000 7fb814a00640 1 -- 172.21.15.159:0/1578390563 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb7f006c920 msgr2=0x7fb7f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:59.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.801+0000 7fb814a00640 1 --2- 172.21.15.159:0/1578390563 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb7f006c920 0x7fb7f006ede0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb800036000 tx=0x7fb80003c040 comp rx=0 tx=0).stop 2024-04-04T06:05:59.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.801+0000 7fb814a00640 1 -- 172.21.15.159:0/1578390563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb810161b00 msgr2=0x7fb8100b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:05:59.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.801+0000 7fb814a00640 1 --2- 172.21.15.159:0/1578390563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb810161b00 0x7fb8100b1ee0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb80800b9e0 tx=0x7fb80800bea0 comp rx=0 tx=0).stop 2024-04-04T06:05:59.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.801+0000 7fb814a00640 1 -- 172.21.15.159:0/1578390563 shutdown_connections 2024-04-04T06:05:59.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.801+0000 7fb814a00640 1 --2- 172.21.15.159:0/1578390563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb810161b00 0x7fb8100b1ee0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:59.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.801+0000 7fb814a00640 1 --2- 172.21.15.159:0/1578390563 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb7f006c920 0x7fb7f006ede0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:59.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.801+0000 7fb814a00640 1 --2- 172.21.15.159:0/1578390563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8100d0d60 0x7fb81015ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:59.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.801+0000 7fb814a00640 1 --2- 172.21.15.159:0/1578390563 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8100ad120 0x7fb810156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:05:59.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.801+0000 7fb814a00640 1 -- 172.21.15.159:0/1578390563 >> 172.21.15.159:0/1578390563 conn(0x7fb81006b990 msgr2=0x7fb8100b3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:05:59.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.801+0000 7fb814a00640 1 -- 172.21.15.159:0/1578390563 shutdown_connections 2024-04-04T06:05:59.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:05:59.801+0000 7fb814a00640 1 -- 172.21.15.159:0/1578390563 wait complete. 2024-04-04T06:05:59.814 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-04T06:05:58.712390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235672,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739048,"statfs":{"total":773094113280,"available":772852785152,"internally_reserved":0,"allocated":2924544,"data_stored":1765420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.377959"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594198+0000","last_change":"2024-04-04T06:05:53.366079+0000","last_active":"2024-04-04T06:05:54.594198+0000","last_peered":"2024-04-04T06:05:54.594198+0000","last_clean":"2024-04-04T06:05:54.594198+0000","last_became_active":"2024-04-04T06:05:53.365810+0000","last_became_peered":"2024-04-04T06:05:53.365810+0000","last_unstale":"2024-04-04T06:05:54.594198+0000","last_undegraded":"2024-04-04T06:05:54.594198+0000","last_fullsized":"2024-04-04T06:05:54.594198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:46:01.152077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362281+0000","last_change":"2024-04-04T06:05:53.376777+0000","last_active":"2024-04-04T06:05:54.362281+0000","last_peered":"2024-04-04T06:05:54.362281+0000","last_clean":"2024-04-04T06:05:54.362281+0000","last_became_active":"2024-04-04T06:05:53.374947+0000","last_became_peered":"2024-04-04T06:05:53.374947+0000","last_unstale":"2024-04-04T06:05:54.362281+0000","last_undegraded":"2024-04-04T06:05:54.362281+0000","last_fullsized":"2024-04-04T06:05:54.362281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:34.766930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591301+0000","last_change":"2024-04-04T06:05:53.362328+0000","last_active":"2024-04-04T06:05:54.591301+0000","last_peered":"2024-04-04T06:05:54.591301+0000","last_clean":"2024-04-04T06:05:54.591301+0000","last_became_active":"2024-04-04T06:05:53.359587+0000","last_became_peered":"2024-04-04T06:05:53.359587+0000","last_unstale":"2024-04-04T06:05:54.591301+0000","last_undegraded":"2024-04-04T06:05:54.591301+0000","last_fullsized":"2024-04-04T06:05:54.591301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:10.411661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365717+0000","last_change":"2024-04-04T06:05:53.373115+0000","last_active":"2024-04-04T06:05:54.365717+0000","last_peered":"2024-04-04T06:05:54.365717+0000","last_clean":"2024-04-04T06:05:54.365717+0000","last_became_active":"2024-04-04T06:05:53.372848+0000","last_became_peered":"2024-04-04T06:05:53.372848+0000","last_unstale":"2024-04-04T06:05:54.365717+0000","last_undegraded":"2024-04-04T06:05:54.365717+0000","last_fullsized":"2024-04-04T06:05:54.365717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:45:21.548886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593377+0000","last_change":"2024-04-04T06:05:53.370721+0000","last_active":"2024-04-04T06:05:54.593377+0000","last_peered":"2024-04-04T06:05:54.593377+0000","last_clean":"2024-04-04T06:05:54.593377+0000","last_became_active":"2024-04-04T06:05:53.370183+0000","last_became_peered":"2024-04-04T06:05:53.370183+0000","last_unstale":"2024-04-04T06:05:54.593377+0000","last_undegraded":"2024-04-04T06:05:54.593377+0000","last_fullsized":"2024-04-04T06:05:54.593377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:30:56.836366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367929+0000","last_change":"2024-04-04T06:05:53.387263+0000","last_active":"2024-04-04T06:05:54.367929+0000","last_peered":"2024-04-04T06:05:54.367929+0000","last_clean":"2024-04-04T06:05:54.367929+0000","last_became_active":"2024-04-04T06:05:53.387012+0000","last_became_peered":"2024-04-04T06:05:53.387012+0000","last_unstale":"2024-04-04T06:05:54.367929+0000","last_undegraded":"2024-04-04T06:05:54.367929+0000","last_fullsized":"2024-04-04T06:05:54.367929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:13.472940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592309+0000","last_change":"2024-04-04T06:05:53.370203+0000","last_active":"2024-04-04T06:05:54.592309+0000","last_peered":"2024-04-04T06:05:54.592309+0000","last_clean":"2024-04-04T06:05:54.592309+0000","last_became_active":"2024-04-04T06:05:53.369800+0000","last_became_peered":"2024-04-04T06:05:53.369800+0000","last_unstale":"2024-04-04T06:05:54.592309+0000","last_undegraded":"2024-04-04T06:05:54.592309+0000","last_fullsized":"2024-04-04T06:05:54.592309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:39:30.391173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592465+0000","last_change":"2024-04-04T06:05:53.369028+0000","last_active":"2024-04-04T06:05:54.592465+0000","last_peered":"2024-04-04T06:05:54.592465+0000","last_clean":"2024-04-04T06:05:54.592465+0000","last_became_active":"2024-04-04T06:05:53.365998+0000","last_became_peered":"2024-04-04T06:05:53.365998+0000","last_unstale":"2024-04-04T06:05:54.592465+0000","last_undegraded":"2024-04-04T06:05:54.592465+0000","last_fullsized":"2024-04-04T06:05:54.592465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:34.766930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594227+0000","last_change":"2024-04-04T06:05:53.369886+0000","last_active":"2024-04-04T06:05:54.594227+0000","last_peered":"2024-04-04T06:05:54.594227+0000","last_clean":"2024-04-04T06:05:54.594227+0000","last_became_active":"2024-04-04T06:05:53.368815+0000","last_became_peered":"2024-04-04T06:05:53.368815+0000","last_unstale":"2024-04-04T06:05:54.594227+0000","last_undegraded":"2024-04-04T06:05:54.594227+0000","last_fullsized":"2024-04-04T06:05:54.594227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:37:59.044530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592641+0000","last_change":"2024-04-04T06:05:53.366948+0000","last_active":"2024-04-04T06:05:54.592641+0000","last_peered":"2024-04-04T06:05:54.592641+0000","last_clean":"2024-04-04T06:05:54.592641+0000","last_became_active":"2024-04-04T06:05:53.366550+0000","last_became_peered":"2024-04-04T06:05:53.366550+0000","last_unstale":"2024-04-04T06:05:54.592641+0000","last_undegraded":"2024-04-04T06:05:54.592641+0000","last_fullsized":"2024-04-04T06:05:54.592641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:47:43.615957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592078+0000","last_change":"2024-04-04T06:05:53.376979+0000","last_active":"2024-04-04T06:05:54.592078+0000","last_peered":"2024-04-04T06:05:54.592078+0000","last_clean":"2024-04-04T06:05:54.592078+0000","last_became_active":"2024-04-04T06:05:53.376622+0000","last_became_peered":"2024-04-04T06:05:53.376622+0000","last_unstale":"2024-04-04T06:05:54.592078+0000","last_undegraded":"2024-04-04T06:05:54.592078+0000","last_fullsized":"2024-04-04T06:05:54.592078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:42:00.014108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367238+0000","last_change":"2024-04-04T06:05:53.364063+0000","last_active":"2024-04-04T06:05:54.367238+0000","last_peered":"2024-04-04T06:05:54.367238+0000","last_clean":"2024-04-04T06:05:54.367238+0000","last_became_active":"2024-04-04T06:05:53.363715+0000","last_became_peered":"2024-04-04T06:05:53.363715+0000","last_unstale":"2024-04-04T06:05:54.367238+0000","last_undegraded":"2024-04-04T06:05:54.367238+0000","last_fullsized":"2024-04-04T06:05:54.367238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:21.889411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593467+0000","last_change":"2024-04-04T06:05:53.362035+0000","last_active":"2024-04-04T06:05:54.593467+0000","last_peered":"2024-04-04T06:05:54.593467+0000","last_clean":"2024-04-04T06:05:54.593467+0000","last_became_active":"2024-04-04T06:05:53.361614+0000","last_became_peered":"2024-04-04T06:05:53.361614+0000","last_unstale":"2024-04-04T06:05:54.593467+0000","last_undegraded":"2024-04-04T06:05:54.593467+0000","last_fullsized":"2024-04-04T06:05:54.593467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:17:05.576402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590275+0000","last_change":"2024-04-04T06:05:53.355832+0000","last_active":"2024-04-04T06:05:54.590275+0000","last_peered":"2024-04-04T06:05:54.590275+0000","last_clean":"2024-04-04T06:05:54.590275+0000","last_became_active":"2024-04-04T06:05:53.355508+0000","last_became_peered":"2024-04-04T06:05:53.355508+0000","last_unstale":"2024-04-04T06:05:54.590275+0000","last_undegraded":"2024-04-04T06:05:54.590275+0000","last_fullsized":"2024-04-04T06:05:54.590275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:11:00.324524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362799+0000","last_change":"2024-04-04T06:05:53.353630+0000","last_active":"2024-04-04T06:05:54.362799+0000","last_peered":"2024-04-04T06:05:54.362799+0000","last_clean":"2024-04-04T06:05:54.362799+0000","last_became_active":"2024-04-04T06:05:53.353315+0000","last_became_peered":"2024-04-04T06:05:53.353315+0000","last_unstale":"2024-04-04T06:05:54.362799+0000","last_undegraded":"2024-04-04T06:05:54.362799+0000","last_fullsized":"2024-04-04T06:05:54.362799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.364261+0000","last_change":"2024-04-04T06:05:53.387499+0000","last_active":"2024-04-04T06:05:54.364261+0000","last_peered":"2024-04-04T06:05:54.364261+0000","last_clean":"2024-04-04T06:05:54.364261+0000","last_became_active":"2024-04-04T06:05:53.387187+0000","last_became_peered":"2024-04-04T06:05:53.387187+0000","last_unstale":"2024-04-04T06:05:54.364261+0000","last_undegraded":"2024-04-04T06:05:54.364261+0000","last_fullsized":"2024-04-04T06:05:54.364261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.366558+0000","last_change":"2024-04-04T06:05:53.378705+0000","last_active":"2024-04-04T06:05:54.366558+0000","last_peered":"2024-04-04T06:05:54.366558+0000","last_clean":"2024-04-04T06:05:54.366558+0000","last_became_active":"2024-04-04T06:05:53.378428+0000","last_became_peered":"2024-04-04T06:05:53.378428+0000","last_unstale":"2024-04-04T06:05:54.366558+0000","last_undegraded":"2024-04-04T06:05:54.366558+0000","last_fullsized":"2024-04-04T06:05:54.366558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:17:42.995360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591477+0000","last_change":"2024-04-04T06:05:53.372718+0000","last_active":"2024-04-04T06:05:54.591477+0000","last_peered":"2024-04-04T06:05:54.591477+0000","last_clean":"2024-04-04T06:05:54.591477+0000","last_became_active":"2024-04-04T06:05:53.372458+0000","last_became_peered":"2024-04-04T06:05:53.372458+0000","last_unstale":"2024-04-04T06:05:54.591477+0000","last_undegraded":"2024-04-04T06:05:54.591477+0000","last_fullsized":"2024-04-04T06:05:54.591477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:31.723861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594485+0000","last_change":"2024-04-04T06:05:53.360929+0000","last_active":"2024-04-04T06:05:54.594485+0000","last_peered":"2024-04-04T06:05:54.594485+0000","last_clean":"2024-04-04T06:05:54.594485+0000","last_became_active":"2024-04-04T06:05:53.360661+0000","last_became_peered":"2024-04-04T06:05:53.360661+0000","last_unstale":"2024-04-04T06:05:54.594485+0000","last_undegraded":"2024-04-04T06:05:54.594485+0000","last_fullsized":"2024-04-04T06:05:54.594485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:06:23.755362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591699+0000","last_change":"2024-04-04T06:05:53.374155+0000","last_active":"2024-04-04T06:05:54.591699+0000","last_peered":"2024-04-04T06:05:54.591699+0000","last_clean":"2024-04-04T06:05:54.591699+0000","last_became_active":"2024-04-04T06:05:53.373872+0000","last_became_peered":"2024-04-04T06:05:53.373872+0000","last_unstale":"2024-04-04T06:05:54.591699+0000","last_undegraded":"2024-04-04T06:05:54.591699+0000","last_fullsized":"2024-04-04T06:05:54.591699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:19:20.840644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590730+0000","last_change":"2024-04-04T06:05:53.375944+0000","last_active":"2024-04-04T06:05:54.590730+0000","last_peered":"2024-04-04T06:05:54.590730+0000","last_clean":"2024-04-04T06:05:54.590730+0000","last_became_active":"2024-04-04T06:05:53.375551+0000","last_became_peered":"2024-04-04T06:05:53.375551+0000","last_unstale":"2024-04-04T06:05:54.590730+0000","last_undegraded":"2024-04-04T06:05:54.590730+0000","last_fullsized":"2024-04-04T06:05:54.590730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:33:17.187978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367881+0000","last_change":"2024-04-04T06:05:53.377200+0000","last_active":"2024-04-04T06:05:54.367881+0000","last_peered":"2024-04-04T06:05:54.367881+0000","last_clean":"2024-04-04T06:05:54.367881+0000","last_became_active":"2024-04-04T06:05:53.373063+0000","last_became_peered":"2024-04-04T06:05:53.373063+0000","last_unstale":"2024-04-04T06:05:54.367881+0000","last_undegraded":"2024-04-04T06:05:54.367881+0000","last_fullsized":"2024-04-04T06:05:54.367881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:28:48.569552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592389+0000","last_change":"2024-04-04T06:05:53.358659+0000","last_active":"2024-04-04T06:05:54.592389+0000","last_peered":"2024-04-04T06:05:54.592389+0000","last_clean":"2024-04-04T06:05:54.592389+0000","last_became_active":"2024-04-04T06:05:53.358372+0000","last_became_peered":"2024-04-04T06:05:53.358372+0000","last_unstale":"2024-04-04T06:05:54.592389+0000","last_undegraded":"2024-04-04T06:05:54.592389+0000","last_fullsized":"2024-04-04T06:05:54.592389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:14:41.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365536+0000","last_change":"2024-04-04T06:05:53.381591+0000","last_active":"2024-04-04T06:05:54.365536+0000","last_peered":"2024-04-04T06:05:54.365536+0000","last_clean":"2024-04-04T06:05:54.365536+0000","last_became_active":"2024-04-04T06:05:53.377261+0000","last_became_peered":"2024-04-04T06:05:53.377261+0000","last_unstale":"2024-04-04T06:05:54.365536+0000","last_undegraded":"2024-04-04T06:05:54.365536+0000","last_fullsized":"2024-04-04T06:05:54.365536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:07:53.196090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593367+0000","last_change":"2024-04-04T06:05:53.371961+0000","last_active":"2024-04-04T06:05:54.593367+0000","last_peered":"2024-04-04T06:05:54.593367+0000","last_clean":"2024-04-04T06:05:54.593367+0000","last_became_active":"2024-04-04T06:05:53.371702+0000","last_became_peered":"2024-04-04T06:05:53.371702+0000","last_unstale":"2024-04-04T06:05:54.593367+0000","last_undegraded":"2024-04-04T06:05:54.593367+0000","last_fullsized":"2024-04-04T06:05:54.593367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592985+0000","last_change":"2024-04-04T06:05:53.383384+0000","last_active":"2024-04-04T06:05:54.592985+0000","last_peered":"2024-04-04T06:05:54.592985+0000","last_clean":"2024-04-04T06:05:54.592985+0000","last_became_active":"2024-04-04T06:05:53.382809+0000","last_became_peered":"2024-04-04T06:05:53.382809+0000","last_unstale":"2024-04-04T06:05:54.592985+0000","last_undegraded":"2024-04-04T06:05:54.592985+0000","last_fullsized":"2024-04-04T06:05:54.592985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:15:27.732386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362466+0000","last_change":"2024-04-04T06:05:53.373499+0000","last_active":"2024-04-04T06:05:54.362466+0000","last_peered":"2024-04-04T06:05:54.362466+0000","last_clean":"2024-04-04T06:05:54.362466+0000","last_became_active":"2024-04-04T06:05:53.367324+0000","last_became_peered":"2024-04-04T06:05:53.367324+0000","last_unstale":"2024-04-04T06:05:54.362466+0000","last_undegraded":"2024-04-04T06:05:54.362466+0000","last_fullsized":"2024-04-04T06:05:54.362466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:21:23.535889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593436+0000","last_change":"2024-04-04T06:05:53.371688+0000","last_active":"2024-04-04T06:05:54.593436+0000","last_peered":"2024-04-04T06:05:54.593436+0000","last_clean":"2024-04-04T06:05:54.593436+0000","last_became_active":"2024-04-04T06:05:53.371350+0000","last_became_peered":"2024-04-04T06:05:53.371350+0000","last_unstale":"2024-04-04T06:05:54.593436+0000","last_undegraded":"2024-04-04T06:05:54.593436+0000","last_fullsized":"2024-04-04T06:05:54.593436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:07:53.196090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590942+0000","last_change":"2024-04-04T06:05:53.370489+0000","last_active":"2024-04-04T06:05:54.590942+0000","last_peered":"2024-04-04T06:05:54.590942+0000","last_clean":"2024-04-04T06:05:54.590942+0000","last_became_active":"2024-04-04T06:05:53.367223+0000","last_became_peered":"2024-04-04T06:05:53.367223+0000","last_unstale":"2024-04-04T06:05:54.590942+0000","last_undegraded":"2024-04-04T06:05:54.590942+0000","last_fullsized":"2024-04-04T06:05:54.590942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:28:13.818701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368055+0000","last_change":"2024-04-04T06:05:53.364565+0000","last_active":"2024-04-04T06:05:54.368055+0000","last_peered":"2024-04-04T06:05:54.368055+0000","last_clean":"2024-04-04T06:05:54.368055+0000","last_became_active":"2024-04-04T06:05:53.364292+0000","last_became_peered":"2024-04-04T06:05:53.364292+0000","last_unstale":"2024-04-04T06:05:54.368055+0000","last_undegraded":"2024-04-04T06:05:54.368055+0000","last_fullsized":"2024-04-04T06:05:54.368055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:37:54.850791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590194+0000","last_change":"2024-04-04T06:05:53.366686+0000","last_active":"2024-04-04T06:05:54.590194+0000","last_peered":"2024-04-04T06:05:54.590194+0000","last_clean":"2024-04-04T06:05:54.590194+0000","last_became_active":"2024-04-04T06:05:53.366409+0000","last_became_peered":"2024-04-04T06:05:53.366409+0000","last_unstale":"2024-04-04T06:05:54.590194+0000","last_undegraded":"2024-04-04T06:05:54.590194+0000","last_fullsized":"2024-04-04T06:05:54.590194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:21.889411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591239+0000","last_change":"2024-04-04T06:05:53.361681+0000","last_active":"2024-04-04T06:05:54.591239+0000","last_peered":"2024-04-04T06:05:54.591239+0000","last_clean":"2024-04-04T06:05:54.591239+0000","last_became_active":"2024-04-04T06:05:53.361307+0000","last_became_peered":"2024-04-04T06:05:53.361307+0000","last_unstale":"2024-04-04T06:05:54.591239+0000","last_undegraded":"2024-04-04T06:05:54.591239+0000","last_fullsized":"2024-04-04T06:05:54.591239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:16:20.634573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.364842+0000","last_change":"2024-04-04T06:05:53.361164+0000","last_active":"2024-04-04T06:05:54.364842+0000","last_peered":"2024-04-04T06:05:54.364842+0000","last_clean":"2024-04-04T06:05:54.364842+0000","last_became_active":"2024-04-04T06:05:53.357301+0000","last_became_peered":"2024-04-04T06:05:53.357301+0000","last_unstale":"2024-04-04T06:05:54.364842+0000","last_undegraded":"2024-04-04T06:05:54.364842+0000","last_fullsized":"2024-04-04T06:05:54.364842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:56:10.393146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594020+0000","last_change":"2024-04-04T06:05:52.368795+0000","last_active":"2024-04-04T06:05:54.594020+0000","last_peered":"2024-04-04T06:05:54.594020+0000","last_clean":"2024-04-04T06:05:54.594020+0000","last_became_active":"2024-04-04T06:05:52.368526+0000","last_became_peered":"2024-04-04T06:05:52.368526+0000","last_unstale":"2024-04-04T06:05:54.594020+0000","last_undegraded":"2024-04-04T06:05:54.594020+0000","last_fullsized":"2024-04-04T06:05:54.594020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:45:43.480308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591181+0000","last_change":"2024-04-04T06:05:53.377207+0000","last_active":"2024-04-04T06:05:54.591181+0000","last_peered":"2024-04-04T06:05:54.591181+0000","last_clean":"2024-04-04T06:05:54.591181+0000","last_became_active":"2024-04-04T06:05:53.376853+0000","last_became_peered":"2024-04-04T06:05:53.376853+0000","last_unstale":"2024-04-04T06:05:54.591181+0000","last_undegraded":"2024-04-04T06:05:54.591181+0000","last_fullsized":"2024-04-04T06:05:54.591181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:21:23.535889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591256+0000","last_change":"2024-04-04T06:05:52.372922+0000","last_active":"2024-04-04T06:05:54.591256+0000","last_peered":"2024-04-04T06:05:54.591256+0000","last_clean":"2024-04-04T06:05:54.591256+0000","last_became_active":"2024-04-04T06:05:52.372615+0000","last_became_peered":"2024-04-04T06:05:52.372615+0000","last_unstale":"2024-04-04T06:05:54.591256+0000","last_undegraded":"2024-04-04T06:05:54.591256+0000","last_fullsized":"2024-04-04T06:05:54.591256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:36:38.928830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592633+0000","last_change":"2024-04-04T06:05:52.371943+0000","last_active":"2024-04-04T06:05:54.592633+0000","last_peered":"2024-04-04T06:05:54.592633+0000","last_clean":"2024-04-04T06:05:54.592633+0000","last_became_active":"2024-04-04T06:05:52.371686+0000","last_became_peered":"2024-04-04T06:05:52.371686+0000","last_unstale":"2024-04-04T06:05:54.592633+0000","last_undegraded":"2024-04-04T06:05:54.592633+0000","last_fullsized":"2024-04-04T06:05:54.592633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:52:05.788594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592504+0000","last_change":"2024-04-04T06:05:53.363121+0000","last_active":"2024-04-04T06:05:54.592504+0000","last_peered":"2024-04-04T06:05:54.592504+0000","last_clean":"2024-04-04T06:05:54.592504+0000","last_became_active":"2024-04-04T06:05:53.362861+0000","last_became_peered":"2024-04-04T06:05:53.362861+0000","last_unstale":"2024-04-04T06:05:54.592504+0000","last_undegraded":"2024-04-04T06:05:54.592504+0000","last_fullsized":"2024-04-04T06:05:54.592504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:28:13.818701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365617+0000","last_change":"2024-04-04T06:05:52.371090+0000","last_active":"2024-04-04T06:05:54.365617+0000","last_peered":"2024-04-04T06:05:54.365617+0000","last_clean":"2024-04-04T06:05:54.365617+0000","last_became_active":"2024-04-04T06:05:52.369650+0000","last_became_peered":"2024-04-04T06:05:52.369650+0000","last_unstale":"2024-04-04T06:05:54.365617+0000","last_undegraded":"2024-04-04T06:05:54.365617+0000","last_fullsized":"2024-04-04T06:05:54.365617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:45:43.480308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362432+0000","last_change":"2024-04-04T06:05:53.357537+0000","last_active":"2024-04-04T06:05:54.362432+0000","last_peered":"2024-04-04T06:05:54.362432+0000","last_clean":"2024-04-04T06:05:54.362432+0000","last_became_active":"2024-04-04T06:05:53.357284+0000","last_became_peered":"2024-04-04T06:05:53.357284+0000","last_unstale":"2024-04-04T06:05:54.362432+0000","last_undegraded":"2024-04-04T06:05:54.362432+0000","last_fullsized":"2024-04-04T06:05:54.362432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:28:13.818701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592398+0000","last_change":"2024-04-04T06:05:52.374921+0000","last_active":"2024-04-04T06:05:54.592398+0000","last_peered":"2024-04-04T06:05:54.592398+0000","last_clean":"2024-04-04T06:05:54.592398+0000","last_became_active":"2024-04-04T06:05:52.374555+0000","last_became_peered":"2024-04-04T06:05:52.374555+0000","last_unstale":"2024-04-04T06:05:54.592398+0000","last_undegraded":"2024-04-04T06:05:54.592398+0000","last_fullsized":"2024-04-04T06:05:54.592398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:05.925335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368597+0000","last_change":"2024-04-04T06:05:53.377612+0000","last_active":"2024-04-04T06:05:54.368597+0000","last_peered":"2024-04-04T06:05:54.368597+0000","last_clean":"2024-04-04T06:05:54.368597+0000","last_became_active":"2024-04-04T06:05:53.374258+0000","last_became_peered":"2024-04-04T06:05:53.374258+0000","last_unstale":"2024-04-04T06:05:54.368597+0000","last_undegraded":"2024-04-04T06:05:54.368597+0000","last_fullsized":"2024-04-04T06:05:54.368597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:08:44.769578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368664+0000","last_change":"2024-04-04T06:05:52.373979+0000","last_active":"2024-04-04T06:05:54.368664+0000","last_peered":"2024-04-04T06:05:54.368664+0000","last_clean":"2024-04-04T06:05:54.368664+0000","last_became_active":"2024-04-04T06:05:52.373408+0000","last_became_peered":"2024-04-04T06:05:52.373408+0000","last_unstale":"2024-04-04T06:05:54.368664+0000","last_undegraded":"2024-04-04T06:05:54.368664+0000","last_fullsized":"2024-04-04T06:05:54.368664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:56:09.383441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591338+0000","last_change":"2024-04-04T06:05:53.368728+0000","last_active":"2024-04-04T06:05:54.591338+0000","last_peered":"2024-04-04T06:05:54.591338+0000","last_clean":"2024-04-04T06:05:54.591338+0000","last_became_active":"2024-04-04T06:05:53.367971+0000","last_became_peered":"2024-04-04T06:05:53.367971+0000","last_unstale":"2024-04-04T06:05:54.591338+0000","last_undegraded":"2024-04-04T06:05:54.591338+0000","last_fullsized":"2024-04-04T06:05:54.591338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362602+0000","last_change":"2024-04-04T06:05:52.368574+0000","last_active":"2024-04-04T06:05:54.362602+0000","last_peered":"2024-04-04T06:05:54.362602+0000","last_clean":"2024-04-04T06:05:54.362602+0000","last_became_active":"2024-04-04T06:05:52.367460+0000","last_became_peered":"2024-04-04T06:05:52.367460+0000","last_unstale":"2024-04-04T06:05:54.362602+0000","last_undegraded":"2024-04-04T06:05:54.362602+0000","last_fullsized":"2024-04-04T06:05:54.362602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:56:09.383441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367547+0000","last_change":"2024-04-04T06:05:53.379475+0000","last_active":"2024-04-04T06:05:54.367547+0000","last_peered":"2024-04-04T06:05:54.367547+0000","last_clean":"2024-04-04T06:05:54.367547+0000","last_became_active":"2024-04-04T06:05:53.376075+0000","last_became_peered":"2024-04-04T06:05:53.376075+0000","last_unstale":"2024-04-04T06:05:54.367547+0000","last_undegraded":"2024-04-04T06:05:54.367547+0000","last_fullsized":"2024-04-04T06:05:54.367547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:03:00.104400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591818+0000","last_change":"2024-04-04T06:05:52.370133+0000","last_active":"2024-04-04T06:05:54.591818+0000","last_peered":"2024-04-04T06:05:54.591818+0000","last_clean":"2024-04-04T06:05:54.591818+0000","last_became_active":"2024-04-04T06:05:52.369825+0000","last_became_peered":"2024-04-04T06:05:52.369825+0000","last_unstale":"2024-04-04T06:05:54.591818+0000","last_undegraded":"2024-04-04T06:05:54.591818+0000","last_fullsized":"2024-04-04T06:05:54.591818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:01:02.629926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594503+0000","last_change":"2024-04-04T06:05:53.371020+0000","last_active":"2024-04-04T06:05:54.594503+0000","last_peered":"2024-04-04T06:05:54.594503+0000","last_clean":"2024-04-04T06:05:54.594503+0000","last_became_active":"2024-04-04T06:05:53.370755+0000","last_became_peered":"2024-04-04T06:05:53.370755+0000","last_unstale":"2024-04-04T06:05:54.594503+0000","last_undegraded":"2024-04-04T06:05:54.594503+0000","last_fullsized":"2024-04-04T06:05:54.594503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:54:36.353079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591316+0000","last_change":"2024-04-04T06:05:52.367020+0000","last_active":"2024-04-04T06:05:54.591316+0000","last_peered":"2024-04-04T06:05:54.591316+0000","last_clean":"2024-04-04T06:05:54.591316+0000","last_became_active":"2024-04-04T06:05:52.366654+0000","last_became_peered":"2024-04-04T06:05:52.366654+0000","last_unstale":"2024-04-04T06:05:54.591316+0000","last_undegraded":"2024-04-04T06:05:54.591316+0000","last_fullsized":"2024-04-04T06:05:54.591316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:26:03.163432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590433+0000","last_change":"2024-04-04T06:05:53.373111+0000","last_active":"2024-04-04T06:05:54.590433+0000","last_peered":"2024-04-04T06:05:54.590433+0000","last_clean":"2024-04-04T06:05:54.590433+0000","last_became_active":"2024-04-04T06:05:53.372761+0000","last_became_peered":"2024-04-04T06:05:53.372761+0000","last_unstale":"2024-04-04T06:05:54.590433+0000","last_undegraded":"2024-04-04T06:05:54.590433+0000","last_fullsized":"2024-04-04T06:05:54.590433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:27:06.279930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368191+0000","last_change":"2024-04-04T06:05:52.365334+0000","last_active":"2024-04-04T06:05:54.368191+0000","last_peered":"2024-04-04T06:05:54.368191+0000","last_clean":"2024-04-04T06:05:54.368191+0000","last_became_active":"2024-04-04T06:05:52.363154+0000","last_became_peered":"2024-04-04T06:05:52.363154+0000","last_unstale":"2024-04-04T06:05:54.368191+0000","last_undegraded":"2024-04-04T06:05:54.368191+0000","last_fullsized":"2024-04-04T06:05:54.368191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:53:42.904072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593559+0000","last_change":"2024-04-04T06:05:53.380196+0000","last_active":"2024-04-04T06:05:54.593559+0000","last_peered":"2024-04-04T06:05:54.593559+0000","last_clean":"2024-04-04T06:05:54.593559+0000","last_became_active":"2024-04-04T06:05:53.379800+0000","last_became_peered":"2024-04-04T06:05:53.379800+0000","last_unstale":"2024-04-04T06:05:54.593559+0000","last_undegraded":"2024-04-04T06:05:54.593559+0000","last_fullsized":"2024-04-04T06:05:54.593559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:41:09.510239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591089+0000","last_change":"2024-04-04T06:05:52.377332+0000","last_active":"2024-04-04T06:05:54.591089+0000","last_peered":"2024-04-04T06:05:54.591089+0000","last_clean":"2024-04-04T06:05:54.591089+0000","last_became_active":"2024-04-04T06:05:52.377002+0000","last_became_peered":"2024-04-04T06:05:52.377002+0000","last_unstale":"2024-04-04T06:05:54.591089+0000","last_undegraded":"2024-04-04T06:05:54.591089+0000","last_fullsized":"2024-04-04T06:05:54.591089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:05:57.500385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591711+0000","last_change":"2024-04-04T06:05:53.378591+0000","last_active":"2024-04-04T06:05:54.591711+0000","last_peered":"2024-04-04T06:05:54.591711+0000","last_clean":"2024-04-04T06:05:54.591711+0000","last_became_active":"2024-04-04T06:05:53.378227+0000","last_became_peered":"2024-04-04T06:05:53.378227+0000","last_unstale":"2024-04-04T06:05:54.591711+0000","last_undegraded":"2024-04-04T06:05:54.591711+0000","last_fullsized":"2024-04-04T06:05:54.591711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:42:54.347331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.589595+0000","last_change":"2024-04-04T06:05:52.357745+0000","last_active":"2024-04-04T06:05:54.589595+0000","last_peered":"2024-04-04T06:05:54.589595+0000","last_clean":"2024-04-04T06:05:54.589595+0000","last_became_active":"2024-04-04T06:05:52.357280+0000","last_became_peered":"2024-04-04T06:05:52.357280+0000","last_unstale":"2024-04-04T06:05:54.589595+0000","last_undegraded":"2024-04-04T06:05:54.589595+0000","last_fullsized":"2024-04-04T06:05:54.589595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:45:43.480308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592420+0000","last_change":"2024-04-04T06:05:53.387060+0000","last_active":"2024-04-04T06:05:54.592420+0000","last_peered":"2024-04-04T06:05:54.592420+0000","last_clean":"2024-04-04T06:05:54.592420+0000","last_became_active":"2024-04-04T06:05:53.386804+0000","last_became_peered":"2024-04-04T06:05:53.386804+0000","last_unstale":"2024-04-04T06:05:54.592420+0000","last_undegraded":"2024-04-04T06:05:54.592420+0000","last_fullsized":"2024-04-04T06:05:54.592420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:37:36.681617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367992+0000","last_change":"2024-04-04T06:05:52.362010+0000","last_active":"2024-04-04T06:05:54.367992+0000","last_peered":"2024-04-04T06:05:54.367992+0000","last_clean":"2024-04-04T06:05:54.367992+0000","last_became_active":"2024-04-04T06:05:52.361424+0000","last_became_peered":"2024-04-04T06:05:52.361424+0000","last_unstale":"2024-04-04T06:05:54.367992+0000","last_undegraded":"2024-04-04T06:05:54.367992+0000","last_fullsized":"2024-04-04T06:05:54.367992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:26:03.163432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594202+0000","last_change":"2024-04-04T06:05:53.375944+0000","last_active":"2024-04-04T06:05:54.594202+0000","last_peered":"2024-04-04T06:05:54.594202+0000","last_clean":"2024-04-04T06:05:54.594202+0000","last_became_active":"2024-04-04T06:05:53.375684+0000","last_became_peered":"2024-04-04T06:05:53.375684+0000","last_unstale":"2024-04-04T06:05:54.594202+0000","last_undegraded":"2024-04-04T06:05:54.594202+0000","last_fullsized":"2024-04-04T06:05:54.594202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:02:57.074580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591274+0000","last_change":"2024-04-04T06:05:52.358735+0000","last_active":"2024-04-04T06:05:54.591274+0000","last_peered":"2024-04-04T06:05:54.591274+0000","last_clean":"2024-04-04T06:05:54.591274+0000","last_became_active":"2024-04-04T06:05:52.358203+0000","last_became_peered":"2024-04-04T06:05:52.358203+0000","last_unstale":"2024-04-04T06:05:54.591274+0000","last_undegraded":"2024-04-04T06:05:54.591274+0000","last_fullsized":"2024-04-04T06:05:54.591274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:53:42.904072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594617+0000","last_change":"2024-04-04T06:05:53.388782+0000","last_active":"2024-04-04T06:05:54.594617+0000","last_peered":"2024-04-04T06:05:54.594617+0000","last_clean":"2024-04-04T06:05:54.594617+0000","last_became_active":"2024-04-04T06:05:53.388230+0000","last_became_peered":"2024-04-04T06:05:53.388230+0000","last_unstale":"2024-04-04T06:05:54.594617+0000","last_undegraded":"2024-04-04T06:05:54.594617+0000","last_fullsized":"2024-04-04T06:05:54.594617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:31:19.282600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362887+0000","last_change":"2024-04-04T06:05:52.366026+0000","last_active":"2024-04-04T06:05:54.362887+0000","last_peered":"2024-04-04T06:05:54.362887+0000","last_clean":"2024-04-04T06:05:54.362887+0000","last_became_active":"2024-04-04T06:05:52.365663+0000","last_became_peered":"2024-04-04T06:05:52.365663+0000","last_unstale":"2024-04-04T06:05:54.362887+0000","last_undegraded":"2024-04-04T06:05:54.362887+0000","last_fullsized":"2024-04-04T06:05:54.362887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:30.714156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.366141+0000","last_change":"2024-04-04T06:05:53.381556+0000","last_active":"2024-04-04T06:05:54.366141+0000","last_peered":"2024-04-04T06:05:54.366141+0000","last_clean":"2024-04-04T06:05:54.366141+0000","last_became_active":"2024-04-04T06:05:53.373492+0000","last_became_peered":"2024-04-04T06:05:53.373492+0000","last_unstale":"2024-04-04T06:05:54.366141+0000","last_undegraded":"2024-04-04T06:05:54.366141+0000","last_fullsized":"2024-04-04T06:05:54.366141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:53:19.419468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590914+0000","last_change":"2024-04-04T06:05:52.353584+0000","last_active":"2024-04-04T06:05:54.590914+0000","last_peered":"2024-04-04T06:05:54.590914+0000","last_clean":"2024-04-04T06:05:54.590914+0000","last_became_active":"2024-04-04T06:05:52.353017+0000","last_became_peered":"2024-04-04T06:05:52.353017+0000","last_unstale":"2024-04-04T06:05:54.590914+0000","last_undegraded":"2024-04-04T06:05:54.590914+0000","last_fullsized":"2024-04-04T06:05:54.590914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:58:40.874784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593698+0000","last_change":"2024-04-04T06:05:53.368622+0000","last_active":"2024-04-04T06:05:54.593698+0000","last_peered":"2024-04-04T06:05:54.593698+0000","last_clean":"2024-04-04T06:05:54.593698+0000","last_became_active":"2024-04-04T06:05:53.368311+0000","last_became_peered":"2024-04-04T06:05:53.368311+0000","last_unstale":"2024-04-04T06:05:54.593698+0000","last_undegraded":"2024-04-04T06:05:54.593698+0000","last_fullsized":"2024-04-04T06:05:54.593698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:01:24.602038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591729+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:05:54.591729+0000","last_peered":"2024-04-04T06:05:54.591729+0000","last_clean":"2024-04-04T06:05:54.591729+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:05:54.591729+0000","last_undegraded":"2024-04-04T06:05:54.591729+0000","last_fullsized":"2024-04-04T06:05:54.591729+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365145+0000","last_change":"2024-04-04T06:05:52.356730+0000","last_active":"2024-04-04T06:05:54.365145+0000","last_peered":"2024-04-04T06:05:54.365145+0000","last_clean":"2024-04-04T06:05:54.365145+0000","last_became_active":"2024-04-04T06:05:52.352963+0000","last_became_peered":"2024-04-04T06:05:52.352963+0000","last_unstale":"2024-04-04T06:05:54.365145+0000","last_undegraded":"2024-04-04T06:05:54.365145+0000","last_fullsized":"2024-04-04T06:05:54.365145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:05:06.878796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592869+0000","last_change":"2024-04-04T06:05:53.366506+0000","last_active":"2024-04-04T06:05:54.592869+0000","last_peered":"2024-04-04T06:05:54.592869+0000","last_clean":"2024-04-04T06:05:54.592869+0000","last_became_active":"2024-04-04T06:05:53.365677+0000","last_became_peered":"2024-04-04T06:05:53.365677+0000","last_unstale":"2024-04-04T06:05:54.592869+0000","last_undegraded":"2024-04-04T06:05:54.592869+0000","last_fullsized":"2024-04-04T06:05:54.592869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:10.411661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591456+0000","last_change":"2024-04-04T06:05:52.374798+0000","last_active":"2024-04-04T06:05:54.591456+0000","last_peered":"2024-04-04T06:05:54.591456+0000","last_clean":"2024-04-04T06:05:54.591456+0000","last_became_active":"2024-04-04T06:05:52.373838+0000","last_became_peered":"2024-04-04T06:05:52.373838+0000","last_unstale":"2024-04-04T06:05:54.591456+0000","last_undegraded":"2024-04-04T06:05:54.591456+0000","last_fullsized":"2024-04-04T06:05:54.591456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:51:50.301107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590751+0000","last_change":"2024-04-04T06:05:53.385393+0000","last_active":"2024-04-04T06:05:54.590751+0000","last_peered":"2024-04-04T06:05:54.590751+0000","last_clean":"2024-04-04T06:05:54.590751+0000","last_became_active":"2024-04-04T06:05:53.385062+0000","last_became_peered":"2024-04-04T06:05:53.385062+0000","last_unstale":"2024-04-04T06:05:54.590751+0000","last_undegraded":"2024-04-04T06:05:54.590751+0000","last_fullsized":"2024-04-04T06:05:54.590751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:14:41.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590309+0000","last_change":"2024-04-04T06:05:52.360018+0000","last_active":"2024-04-04T06:05:54.590309+0000","last_peered":"2024-04-04T06:05:54.590309+0000","last_clean":"2024-04-04T06:05:54.590309+0000","last_became_active":"2024-04-04T06:05:52.359735+0000","last_became_peered":"2024-04-04T06:05:52.359735+0000","last_unstale":"2024-04-04T06:05:54.590309+0000","last_undegraded":"2024-04-04T06:05:54.590309+0000","last_fullsized":"2024-04-04T06:05:54.590309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:05:06.878796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.363467+0000","last_change":"2024-04-04T06:05:53.356771+0000","last_active":"2024-04-04T06:05:54.363467+0000","last_peered":"2024-04-04T06:05:54.363467+0000","last_clean":"2024-04-04T06:05:54.363467+0000","last_became_active":"2024-04-04T06:05:53.356431+0000","last_became_peered":"2024-04-04T06:05:53.356431+0000","last_unstale":"2024-04-04T06:05:54.363467+0000","last_undegraded":"2024-04-04T06:05:54.363467+0000","last_fullsized":"2024-04-04T06:05:54.363467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:07:53.196090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368163+0000","last_change":"2024-04-04T06:05:52.370387+0000","last_active":"2024-04-04T06:05:54.368163+0000","last_peered":"2024-04-04T06:05:54.368163+0000","last_clean":"2024-04-04T06:05:54.368163+0000","last_became_active":"2024-04-04T06:05:52.369905+0000","last_became_peered":"2024-04-04T06:05:52.369905+0000","last_unstale":"2024-04-04T06:05:54.368163+0000","last_undegraded":"2024-04-04T06:05:54.368163+0000","last_fullsized":"2024-04-04T06:05:54.368163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:58:40.874784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367307+0000","last_change":"2024-04-04T06:05:53.371957+0000","last_active":"2024-04-04T06:05:54.367307+0000","last_peered":"2024-04-04T06:05:54.367307+0000","last_clean":"2024-04-04T06:05:54.367307+0000","last_became_active":"2024-04-04T06:05:53.371121+0000","last_became_peered":"2024-04-04T06:05:53.371121+0000","last_unstale":"2024-04-04T06:05:54.367307+0000","last_undegraded":"2024-04-04T06:05:54.367307+0000","last_fullsized":"2024-04-04T06:05:54.367307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:31.723861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362920+0000","last_change":"2024-04-04T06:05:52.361392+0000","last_active":"2024-04-04T06:05:54.362920+0000","last_peered":"2024-04-04T06:05:54.362920+0000","last_clean":"2024-04-04T06:05:54.362920+0000","last_became_active":"2024-04-04T06:05:52.360932+0000","last_became_peered":"2024-04-04T06:05:52.360932+0000","last_unstale":"2024-04-04T06:05:54.362920+0000","last_undegraded":"2024-04-04T06:05:54.362920+0000","last_fullsized":"2024-04-04T06:05:54.362920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:20.879706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365934+0000","last_change":"2024-04-04T06:05:53.379438+0000","last_active":"2024-04-04T06:05:54.365934+0000","last_peered":"2024-04-04T06:05:54.365934+0000","last_clean":"2024-04-04T06:05:54.365934+0000","last_became_active":"2024-04-04T06:05:53.378781+0000","last_became_peered":"2024-04-04T06:05:53.378781+0000","last_unstale":"2024-04-04T06:05:54.365934+0000","last_undegraded":"2024-04-04T06:05:54.365934+0000","last_fullsized":"2024-04-04T06:05:54.365934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:18:05.337194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594629+0000","last_change":"2024-04-04T06:05:52.372922+0000","last_active":"2024-04-04T06:05:54.594629+0000","last_peered":"2024-04-04T06:05:54.594629+0000","last_clean":"2024-04-04T06:05:54.594629+0000","last_became_active":"2024-04-04T06:05:52.372696+0000","last_became_peered":"2024-04-04T06:05:52.372696+0000","last_unstale":"2024-04-04T06:05:54.594629+0000","last_undegraded":"2024-04-04T06:05:54.594629+0000","last_fullsized":"2024-04-04T06:05:54.594629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:17:53.847287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362796+0000","last_change":"2024-04-04T06:05:53.373066+0000","last_active":"2024-04-04T06:05:54.362796+0000","last_peered":"2024-04-04T06:05:54.362796+0000","last_clean":"2024-04-04T06:05:54.362796+0000","last_became_active":"2024-04-04T06:05:53.367063+0000","last_became_peered":"2024-04-04T06:05:53.367063+0000","last_unstale":"2024-04-04T06:05:54.362796+0000","last_undegraded":"2024-04-04T06:05:54.362796+0000","last_fullsized":"2024-04-04T06:05:54.362796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:10.411661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.363331+0000","last_change":"2024-04-04T06:05:52.365125+0000","last_active":"2024-04-04T06:05:54.363331+0000","last_peered":"2024-04-04T06:05:54.363331+0000","last_clean":"2024-04-04T06:05:54.363331+0000","last_became_active":"2024-04-04T06:05:52.364821+0000","last_became_peered":"2024-04-04T06:05:52.364821+0000","last_unstale":"2024-04-04T06:05:54.363331+0000","last_undegraded":"2024-04-04T06:05:54.363331+0000","last_fullsized":"2024-04-04T06:05:54.363331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:54:42.610854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593547+0000","last_change":"2024-04-04T06:05:53.375887+0000","last_active":"2024-04-04T06:05:54.593547+0000","last_peered":"2024-04-04T06:05:54.593547+0000","last_clean":"2024-04-04T06:05:54.593547+0000","last_became_active":"2024-04-04T06:05:53.374688+0000","last_became_peered":"2024-04-04T06:05:53.374688+0000","last_unstale":"2024-04-04T06:05:54.593547+0000","last_undegraded":"2024-04-04T06:05:54.593547+0000","last_fullsized":"2024-04-04T06:05:54.593547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:45:21.548886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592567+0000","last_change":"2024-04-04T06:05:52.374804+0000","last_active":"2024-04-04T06:05:54.592567+0000","last_peered":"2024-04-04T06:05:54.592567+0000","last_clean":"2024-04-04T06:05:54.592567+0000","last_became_active":"2024-04-04T06:05:52.374476+0000","last_became_peered":"2024-04-04T06:05:52.374476+0000","last_unstale":"2024-04-04T06:05:54.592567+0000","last_undegraded":"2024-04-04T06:05:54.592567+0000","last_fullsized":"2024-04-04T06:05:54.592567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:21:06.579458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590233+0000","last_change":"2024-04-04T06:05:53.358691+0000","last_active":"2024-04-04T06:05:54.590233+0000","last_peered":"2024-04-04T06:05:54.590233+0000","last_clean":"2024-04-04T06:05:54.590233+0000","last_became_active":"2024-04-04T06:05:53.357996+0000","last_became_peered":"2024-04-04T06:05:53.357996+0000","last_unstale":"2024-04-04T06:05:54.590233+0000","last_undegraded":"2024-04-04T06:05:54.590233+0000","last_fullsized":"2024-04-04T06:05:54.590233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:55:58.689192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592379+0000","last_change":"2024-04-04T06:05:52.370047+0000","last_active":"2024-04-04T06:05:54.592379+0000","last_peered":"2024-04-04T06:05:54.592379+0000","last_clean":"2024-04-04T06:05:54.592379+0000","last_became_active":"2024-04-04T06:05:52.369741+0000","last_became_peered":"2024-04-04T06:05:52.369741+0000","last_unstale":"2024-04-04T06:05:54.592379+0000","last_undegraded":"2024-04-04T06:05:54.592379+0000","last_fullsized":"2024-04-04T06:05:54.592379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:36:38.928830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.364951+0000","last_change":"2024-04-04T06:05:53.364563+0000","last_active":"2024-04-04T06:05:54.364951+0000","last_peered":"2024-04-04T06:05:54.364951+0000","last_clean":"2024-04-04T06:05:54.364951+0000","last_became_active":"2024-04-04T06:05:53.362702+0000","last_became_peered":"2024-04-04T06:05:53.362702+0000","last_unstale":"2024-04-04T06:05:54.364951+0000","last_undegraded":"2024-04-04T06:05:54.364951+0000","last_fullsized":"2024-04-04T06:05:54.364951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:54:43.620559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362753+0000","last_change":"2024-04-04T06:05:52.370003+0000","last_active":"2024-04-04T06:05:54.362753+0000","last_peered":"2024-04-04T06:05:54.362753+0000","last_clean":"2024-04-04T06:05:54.362753+0000","last_became_active":"2024-04-04T06:05:52.369724+0000","last_became_peered":"2024-04-04T06:05:52.369724+0000","last_unstale":"2024-04-04T06:05:54.362753+0000","last_undegraded":"2024-04-04T06:05:54.362753+0000","last_fullsized":"2024-04-04T06:05:54.362753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:58:40.874784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592885+0000","last_change":"2024-04-04T06:05:53.361642+0000","last_active":"2024-04-04T06:05:54.592885+0000","last_peered":"2024-04-04T06:05:54.592885+0000","last_clean":"2024-04-04T06:05:54.592885+0000","last_became_active":"2024-04-04T06:05:53.360951+0000","last_became_peered":"2024-04-04T06:05:53.360951+0000","last_unstale":"2024-04-04T06:05:54.592885+0000","last_undegraded":"2024-04-04T06:05:54.592885+0000","last_fullsized":"2024-04-04T06:05:54.592885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:21:23.535889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362582+0000","last_change":"2024-04-04T06:05:52.371223+0000","last_active":"2024-04-04T06:05:54.362582+0000","last_peered":"2024-04-04T06:05:54.362582+0000","last_clean":"2024-04-04T06:05:54.362582+0000","last_became_active":"2024-04-04T06:05:52.370916+0000","last_became_peered":"2024-04-04T06:05:52.370916+0000","last_unstale":"2024-04-04T06:05:54.362582+0000","last_undegraded":"2024-04-04T06:05:54.362582+0000","last_fullsized":"2024-04-04T06:05:54.362582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:53:42.904072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593889+0000","last_change":"2024-04-04T06:05:53.371605+0000","last_active":"2024-04-04T06:05:54.593889+0000","last_peered":"2024-04-04T06:05:54.593889+0000","last_clean":"2024-04-04T06:05:54.593889+0000","last_became_active":"2024-04-04T06:05:53.371328+0000","last_became_peered":"2024-04-04T06:05:53.371328+0000","last_unstale":"2024-04-04T06:05:54.593889+0000","last_undegraded":"2024-04-04T06:05:54.593889+0000","last_fullsized":"2024-04-04T06:05:54.593889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:18:05.337194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593483+0000","last_change":"2024-04-04T06:05:52.368424+0000","last_active":"2024-04-04T06:05:54.593483+0000","last_peered":"2024-04-04T06:05:54.593483+0000","last_clean":"2024-04-04T06:05:54.593483+0000","last_became_active":"2024-04-04T06:05:52.368164+0000","last_became_peered":"2024-04-04T06:05:52.368164+0000","last_unstale":"2024-04-04T06:05:54.593483+0000","last_undegraded":"2024-04-04T06:05:54.593483+0000","last_fullsized":"2024-04-04T06:05:54.593483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:54:18.736812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591138+0000","last_change":"2024-04-04T06:05:53.373894+0000","last_active":"2024-04-04T06:05:54.591138+0000","last_peered":"2024-04-04T06:05:54.591138+0000","last_clean":"2024-04-04T06:05:54.591138+0000","last_became_active":"2024-04-04T06:05:53.373615+0000","last_became_peered":"2024-04-04T06:05:53.373615+0000","last_unstale":"2024-04-04T06:05:54.591138+0000","last_undegraded":"2024-04-04T06:05:54.591138+0000","last_fullsized":"2024-04-04T06:05:54.591138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:03:39.636711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365635+0000","last_change":"2024-04-04T06:05:53.378971+0000","last_active":"2024-04-04T06:05:54.365635+0000","last_peered":"2024-04-04T06:05:54.365635+0000","last_clean":"2024-04-04T06:05:54.365635+0000","last_became_active":"2024-04-04T06:05:53.377287+0000","last_became_peered":"2024-04-04T06:05:53.377287+0000","last_unstale":"2024-04-04T06:05:54.365635+0000","last_undegraded":"2024-04-04T06:05:54.365635+0000","last_fullsized":"2024-04-04T06:05:54.365635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:17:05.576402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594399+0000","last_change":"2024-04-04T06:05:52.376155+0000","last_active":"2024-04-04T06:05:54.594399+0000","last_peered":"2024-04-04T06:05:54.594399+0000","last_clean":"2024-04-04T06:05:54.594399+0000","last_became_active":"2024-04-04T06:05:52.375772+0000","last_became_peered":"2024-04-04T06:05:52.375772+0000","last_unstale":"2024-04-04T06:05:54.594399+0000","last_undegraded":"2024-04-04T06:05:54.594399+0000","last_fullsized":"2024-04-04T06:05:54.594399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:40:09.906862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365922+0000","last_change":"2024-04-04T06:05:53.381179+0000","last_active":"2024-04-04T06:05:54.365922+0000","last_peered":"2024-04-04T06:05:54.365922+0000","last_clean":"2024-04-04T06:05:54.365922+0000","last_became_active":"2024-04-04T06:05:53.374859+0000","last_became_peered":"2024-04-04T06:05:53.374859+0000","last_unstale":"2024-04-04T06:05:54.365922+0000","last_undegraded":"2024-04-04T06:05:54.365922+0000","last_fullsized":"2024-04-04T06:05:54.365922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:20:24.966139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593848+0000","last_change":"2024-04-04T06:05:52.378067+0000","last_active":"2024-04-04T06:05:54.593848+0000","last_peered":"2024-04-04T06:05:54.593848+0000","last_clean":"2024-04-04T06:05:54.593848+0000","last_became_active":"2024-04-04T06:05:52.377777+0000","last_became_peered":"2024-04-04T06:05:52.377777+0000","last_unstale":"2024-04-04T06:05:54.593848+0000","last_undegraded":"2024-04-04T06:05:54.593848+0000","last_fullsized":"2024-04-04T06:05:54.593848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:01:02.629926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590922+0000","last_change":"2024-04-04T06:05:53.368705+0000","last_active":"2024-04-04T06:05:54.590922+0000","last_peered":"2024-04-04T06:05:54.590922+0000","last_clean":"2024-04-04T06:05:54.590922+0000","last_became_active":"2024-04-04T06:05:53.368366+0000","last_became_peered":"2024-04-04T06:05:53.368366+0000","last_unstale":"2024-04-04T06:05:54.590922+0000","last_undegraded":"2024-04-04T06:05:54.590922+0000","last_fullsized":"2024-04-04T06:05:54.590922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:34.766930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593814+0000","last_change":"2024-04-04T06:05:52.367095+0000","last_active":"2024-04-04T06:05:54.593814+0000","last_peered":"2024-04-04T06:05:54.593814+0000","last_clean":"2024-04-04T06:05:54.593814+0000","last_became_active":"2024-04-04T06:05:52.366316+0000","last_became_peered":"2024-04-04T06:05:52.366316+0000","last_unstale":"2024-04-04T06:05:54.593814+0000","last_undegraded":"2024-04-04T06:05:54.593814+0000","last_fullsized":"2024-04-04T06:05:54.593814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:05:06.878796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593646+0000","last_change":"2024-04-04T06:05:53.375226+0000","last_active":"2024-04-04T06:05:54.593646+0000","last_peered":"2024-04-04T06:05:54.593646+0000","last_clean":"2024-04-04T06:05:54.593646+0000","last_became_active":"2024-04-04T06:05:53.374921+0000","last_became_peered":"2024-04-04T06:05:53.374921+0000","last_unstale":"2024-04-04T06:05:54.593646+0000","last_undegraded":"2024-04-04T06:05:54.593646+0000","last_fullsized":"2024-04-04T06:05:54.593646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:20:24.966139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591478+0000","last_change":"2024-04-04T06:05:52.370572+0000","last_active":"2024-04-04T06:05:54.591478+0000","last_peered":"2024-04-04T06:05:54.591478+0000","last_clean":"2024-04-04T06:05:54.591478+0000","last_became_active":"2024-04-04T06:05:52.369378+0000","last_became_peered":"2024-04-04T06:05:52.369378+0000","last_unstale":"2024-04-04T06:05:54.591478+0000","last_undegraded":"2024-04-04T06:05:54.591478+0000","last_fullsized":"2024-04-04T06:05:54.591478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:45:47.508028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":806912,"data_stored":663433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":74095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":74095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":72483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":663489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":72573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":72609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:06:02.819 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:06:03.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.044+0000 7f156e000640 1 Processor -- start 2024-04-04T06:06:03.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.044+0000 7f156e000640 1 -- start start 2024-04-04T06:06:03.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.045+0000 7f156e000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f15680d0d60 0x7f15680b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:03.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.045+0000 7f156e000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1568071b60 con 0x7f15680ad120 2024-04-04T06:06:03.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.045+0000 7f156e000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1568071e40 con 0x7f1568161b00 2024-04-04T06:06:03.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.045+0000 7f156e000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f1568161ee0 con 0x7f15680d0d60 2024-04-04T06:06:03.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.045+0000 7f1566a00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f15680ad120 0x7f15680d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43534/0 (socket says 172.21.15.159:43534) 2024-04-04T06:06:03.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.046+0000 7f1566a00640 1 -- 172.21.15.159:0/3564529499 learned_addr learned my addr 172.21.15.159:0/3564529499 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:03.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.046+0000 7f1567e00640 1 --2- 172.21.15.159:0/3564529499 >> v2:172.21.15.178:3300/0 conn(0x7f15680d0d60 0x7f15680b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:03.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.046+0000 7f1566000640 1 -- 172.21.15.159:0/3564529499 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3840222095 0 0) 0x7f1568071b60 con 0x7f15680ad120 2024-04-04T06:06:03.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.046+0000 7f1566000640 1 -- 172.21.15.159:0/3564529499 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f154c003520 con 0x7f15680ad120 2024-04-04T06:06:03.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.046+0000 7f1566000640 1 -- 172.21.15.159:0/3564529499 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3993074869 0 0) 0x7f154c003520 con 0x7f15680ad120 2024-04-04T06:06:03.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.046+0000 7f1566000640 1 -- 172.21.15.159:0/3564529499 >> v2:172.21.15.178:3300/0 conn(0x7f15680d0d60 msgr2=0x7f15680b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:03.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.046+0000 7f1566000640 1 --2- 172.21.15.159:0/3564529499 >> v2:172.21.15.178:3300/0 conn(0x7f15680d0d60 0x7f15680b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:03.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.046+0000 7f1566000640 1 -- 172.21.15.159:0/3564529499 >> v1:172.21.15.178:6789/0 conn(0x7f1568161b00 legacy=0x7f1568162ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:03.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.046+0000 7f1566000640 1 -- 172.21.15.159:0/3564529499 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1568072120 con 0x7f15680ad120 2024-04-04T06:06:03.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.046+0000 7f1566000640 1 -- 172.21.15.159:0/3564529499 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f1554001690 con 0x7f15680ad120 2024-04-04T06:06:03.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.046+0000 7f1567e00640 1 --2- 172.21.15.159:0/3564529499 >> v2:172.21.15.178:3300/0 conn(0x7f15680d0d60 0x7f15680b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:06:03.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.046+0000 7f1566000640 1 -- 172.21.15.159:0/3564529499 >> v1:172.21.15.159:6789/0 conn(0x7f15680ad120 legacy=0x7f15680d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:03.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.046+0000 7f1566000640 1 --2- 172.21.15.159:0/3564529499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f154c0039b0 0x7f154c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:03.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.046+0000 7f1566000640 1 --2- 172.21.15.159:0/3564529499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f154c004800 0x7f154c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:03.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.046+0000 7f1566000640 1 --2- 172.21.15.159:0/3564529499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f154c0052e0 0x7f154c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:03.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.047+0000 7f1566000640 1 -- 172.21.15.159:0/3564529499 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1568161ee0 con 0x7f154c0039b0 2024-04-04T06:06:03.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.047+0000 7f1566000640 1 -- 172.21.15.159:0/3564529499 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f154c00c1b0 con 0x7f154c004800 2024-04-04T06:06:03.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.047+0000 7f1566000640 1 -- 172.21.15.159:0/3564529499 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f154c00c3b0 con 0x7f154c0052e0 2024-04-04T06:06:03.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.047+0000 7f1567400640 1 --2- 172.21.15.159:0/3564529499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f154c0039b0 0x7f154c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:03.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.047+0000 7f1567e00640 1 --2- 172.21.15.159:0/3564529499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f154c0052e0 0x7f154c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:03.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.047+0000 7f1566a00640 1 --2- 172.21.15.159:0/3564529499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f154c004800 0x7f154c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:03.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.047+0000 7f1567400640 1 -- 172.21.15.159:0/3564529499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f154c0052e0 msgr2=0x7f154c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:03.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.047+0000 7f1567400640 1 --2- 172.21.15.159:0/3564529499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f154c0052e0 0x7f154c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:03.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.047+0000 7f1567400640 1 -- 172.21.15.159:0/3564529499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f154c004800 msgr2=0x7f154c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:03.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.047+0000 7f1567400640 1 --2- 172.21.15.159:0/3564529499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f154c004800 0x7f154c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:03.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.047+0000 7f1567400640 1 -- 172.21.15.159:0/3564529499 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154c003520 con 0x7f154c0039b0 2024-04-04T06:06:03.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.047+0000 7f1566a00640 1 --2- 172.21.15.159:0/3564529499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f154c004800 0x7f154c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:03.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.047+0000 7f1567e00640 1 -- 172.21.15.159:0/3564529499 reap_dead start 2024-04-04T06:06:03.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.048+0000 7f1567400640 1 --2- 172.21.15.159:0/3564529499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f154c0039b0 0x7f154c003dd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1558001720 tx=0x7f154c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb748779713cd94 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:03.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.049+0000 7f1566000640 1 -- 172.21.15.159:0/3564529499 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558001370 con 0x7f154c0039b0 2024-04-04T06:06:03.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.049+0000 7f1566000640 1 -- 172.21.15.159:0/3564529499 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15580052d0 con 0x7f154c0039b0 2024-04-04T06:06:03.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.049+0000 7f1566000640 1 -- 172.21.15.159:0/3564529499 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558003830 con 0x7f154c0039b0 2024-04-04T06:06:03.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.049+0000 7f156e000640 1 -- 172.21.15.159:0/3564529499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f154c0039b0 msgr2=0x7f154c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:03.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.049+0000 7f156e000640 1 --2- 172.21.15.159:0/3564529499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f154c0039b0 0x7f154c003dd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1558001720 tx=0x7f154c005830 comp rx=0 tx=0).stop 2024-04-04T06:06:03.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.049+0000 7f156e000640 1 -- 172.21.15.159:0/3564529499 shutdown_connections 2024-04-04T06:06:03.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.049+0000 7f156e000640 1 --2- 172.21.15.159:0/3564529499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f154c0039b0 0x7f154c003dd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:03.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.049+0000 7f156e000640 1 -- 172.21.15.159:0/3564529499 >> 172.21.15.159:0/3564529499 conn(0x7f156806b990 msgr2=0x7f156806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:03.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.049+0000 7f156e000640 1 -- 172.21.15.159:0/3564529499 shutdown_connections 2024-04-04T06:06:03.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.049+0000 7f156e000640 1 -- 172.21.15.159:0/3564529499 wait complete. 2024-04-04T06:06:03.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.051+0000 7f156e000640 1 Processor -- start 2024-04-04T06:06:03.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.051+0000 7f156e000640 1 -- start start 2024-04-04T06:06:03.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.052+0000 7f156e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f154c0039b0 0x7f1568147eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:03.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.052+0000 7f156e000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f156809d740 0x7f156809dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:03.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.052+0000 7f156e000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156814aa70 0x7f156814af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:03.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.052+0000 7f156e000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f15680b3730 con 0x7f154c0039b0 2024-04-04T06:06:03.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.052+0000 7f156e000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f15680b3450 con 0x7f156809d740 2024-04-04T06:06:03.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.052+0000 7f156e000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f15680b5020 con 0x7f156814aa70 2024-04-04T06:06:03.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.052+0000 7f1567400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f154c0039b0 0x7f1568147eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:03.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.052+0000 7f1567400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f154c0039b0 0x7f1568147eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43398/0 (socket says 172.21.15.159:43398) 2024-04-04T06:06:03.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.052+0000 7f1567400640 1 -- 172.21.15.159:0/497175838 learned_addr learned my addr 172.21.15.159:0/497175838 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:03.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.053+0000 7f1566a00640 1 --2- 172.21.15.159:0/497175838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f156809d740 0x7f156809dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:03.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.053+0000 7f1567e00640 1 --2- 172.21.15.159:0/497175838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156814aa70 0x7f156814af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:03.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.053+0000 7f1567400640 1 -- 172.21.15.159:0/497175838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156814aa70 msgr2=0x7f156814af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:03.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.053+0000 7f1567400640 1 --2- 172.21.15.159:0/497175838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156814aa70 0x7f156814af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:03.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.053+0000 7f1567400640 1 -- 172.21.15.159:0/497175838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f156809d740 msgr2=0x7f156809dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:03.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.053+0000 7f1567400640 1 --2- 172.21.15.159:0/497175838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f156809d740 0x7f156809dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:03.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.053+0000 7f1567400640 1 -- 172.21.15.159:0/497175838 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156809a440 con 0x7f154c0039b0 2024-04-04T06:06:03.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.053+0000 7f1567400640 1 --2- 172.21.15.159:0/497175838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f154c0039b0 0x7f1568147eb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1558005040 tx=0x7f1558001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:03.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.053+0000 7f155f400640 1 -- 172.21.15.159:0/497175838 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558041020 con 0x7f154c0039b0 2024-04-04T06:06:03.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.053+0000 7f155f400640 1 -- 172.21.15.159:0/497175838 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1558005480 con 0x7f154c0039b0 2024-04-04T06:06:03.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.053+0000 7f155f400640 1 -- 172.21.15.159:0/497175838 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558001370 con 0x7f154c0039b0 2024-04-04T06:06:03.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.053+0000 7f156e000640 1 -- 172.21.15.159:0/497175838 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f156809a720 con 0x7f154c0039b0 2024-04-04T06:06:03.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.054+0000 7f156e000640 1 -- 172.21.15.159:0/497175838 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f156809ace0 con 0x7f154c0039b0 2024-04-04T06:06:03.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.055+0000 7f156e000640 1 -- 172.21.15.159:0/497175838 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15680685c0 con 0x7f154c0039b0 2024-04-04T06:06:03.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.055+0000 7f155f400640 1 -- 172.21.15.159:0/497175838 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1558044020 con 0x7f154c0039b0 2024-04-04T06:06:03.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.056+0000 7f155f400640 1 --2- 172.21.15.159:0/497175838 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f154806c8f0 0x7f154806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:03.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.056+0000 7f155f400640 1 -- 172.21.15.159:0/497175838 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f15580ad670 con 0x7f154c0039b0 2024-04-04T06:06:03.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.057+0000 7f1566a00640 1 --2- 172.21.15.159:0/497175838 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f154806c8f0 0x7f154806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:03.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.058+0000 7f1566a00640 1 --2- 172.21.15.159:0/497175838 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f154806c8f0 0x7f154806edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1554001010 tx=0x7f1554009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:03.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.061+0000 7f155f400640 1 -- 172.21.15.159:0/497175838 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f155804b030 con 0x7f154c0039b0 2024-04-04T06:06:03.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.256+0000 7f156e000640 1 -- 172.21.15.159:0/497175838 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f156806f3a0 con 0x7f154806c8f0 2024-04-04T06:06:03.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.274+0000 7f155f400640 1 -- 172.21.15.159:0/497175838 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f156806f3a0 con 0x7f154806c8f0 2024-04-04T06:06:03.273 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:03.274 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:06:03.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.280+0000 7f156e000640 1 -- 172.21.15.159:0/497175838 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f154806c8f0 msgr2=0x7f154806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:03.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.280+0000 7f156e000640 1 --2- 172.21.15.159:0/497175838 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f154806c8f0 0x7f154806edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1554001010 tx=0x7f1554009040 comp rx=0 tx=0).stop 2024-04-04T06:06:03.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.280+0000 7f156e000640 1 -- 172.21.15.159:0/497175838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f154c0039b0 msgr2=0x7f1568147eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:03.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.280+0000 7f156e000640 1 --2- 172.21.15.159:0/497175838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f154c0039b0 0x7f1568147eb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1558005040 tx=0x7f1558001120 comp rx=0 tx=0).stop 2024-04-04T06:06:03.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.280+0000 7f156e000640 1 -- 172.21.15.159:0/497175838 shutdown_connections 2024-04-04T06:06:03.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.280+0000 7f156e000640 1 --2- 172.21.15.159:0/497175838 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f156814aa70 0x7f156814af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:03.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.280+0000 7f156e000640 1 --2- 172.21.15.159:0/497175838 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f154806c8f0 0x7f154806edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:03.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.280+0000 7f156e000640 1 --2- 172.21.15.159:0/497175838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f156809d740 0x7f156809dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:03.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.280+0000 7f156e000640 1 --2- 172.21.15.159:0/497175838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f154c0039b0 0x7f1568147eb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:03.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.280+0000 7f156e000640 1 -- 172.21.15.159:0/497175838 >> 172.21.15.159:0/497175838 conn(0x7f156806b990 msgr2=0x7f15680d0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:03.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.280+0000 7f156e000640 1 -- 172.21.15.159:0/497175838 shutdown_connections 2024-04-04T06:06:03.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.280+0000 7f156e000640 1 -- 172.21.15.159:0/497175838 wait complete. 2024-04-04T06:06:03.294 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-04T06:06:02.713844+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235672,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739048,"statfs":{"total":773094113280,"available":772852785152,"internally_reserved":0,"allocated":2924544,"data_stored":1765420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.375065"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594198+0000","last_change":"2024-04-04T06:05:53.366079+0000","last_active":"2024-04-04T06:05:54.594198+0000","last_peered":"2024-04-04T06:05:54.594198+0000","last_clean":"2024-04-04T06:05:54.594198+0000","last_became_active":"2024-04-04T06:05:53.365810+0000","last_became_peered":"2024-04-04T06:05:53.365810+0000","last_unstale":"2024-04-04T06:05:54.594198+0000","last_undegraded":"2024-04-04T06:05:54.594198+0000","last_fullsized":"2024-04-04T06:05:54.594198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:46:01.152077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362281+0000","last_change":"2024-04-04T06:05:53.376777+0000","last_active":"2024-04-04T06:05:54.362281+0000","last_peered":"2024-04-04T06:05:54.362281+0000","last_clean":"2024-04-04T06:05:54.362281+0000","last_became_active":"2024-04-04T06:05:53.374947+0000","last_became_peered":"2024-04-04T06:05:53.374947+0000","last_unstale":"2024-04-04T06:05:54.362281+0000","last_undegraded":"2024-04-04T06:05:54.362281+0000","last_fullsized":"2024-04-04T06:05:54.362281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:34.766930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591301+0000","last_change":"2024-04-04T06:05:53.362328+0000","last_active":"2024-04-04T06:05:54.591301+0000","last_peered":"2024-04-04T06:05:54.591301+0000","last_clean":"2024-04-04T06:05:54.591301+0000","last_became_active":"2024-04-04T06:05:53.359587+0000","last_became_peered":"2024-04-04T06:05:53.359587+0000","last_unstale":"2024-04-04T06:05:54.591301+0000","last_undegraded":"2024-04-04T06:05:54.591301+0000","last_fullsized":"2024-04-04T06:05:54.591301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:10.411661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365717+0000","last_change":"2024-04-04T06:05:53.373115+0000","last_active":"2024-04-04T06:05:54.365717+0000","last_peered":"2024-04-04T06:05:54.365717+0000","last_clean":"2024-04-04T06:05:54.365717+0000","last_became_active":"2024-04-04T06:05:53.372848+0000","last_became_peered":"2024-04-04T06:05:53.372848+0000","last_unstale":"2024-04-04T06:05:54.365717+0000","last_undegraded":"2024-04-04T06:05:54.365717+0000","last_fullsized":"2024-04-04T06:05:54.365717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:45:21.548886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593377+0000","last_change":"2024-04-04T06:05:53.370721+0000","last_active":"2024-04-04T06:05:54.593377+0000","last_peered":"2024-04-04T06:05:54.593377+0000","last_clean":"2024-04-04T06:05:54.593377+0000","last_became_active":"2024-04-04T06:05:53.370183+0000","last_became_peered":"2024-04-04T06:05:53.370183+0000","last_unstale":"2024-04-04T06:05:54.593377+0000","last_undegraded":"2024-04-04T06:05:54.593377+0000","last_fullsized":"2024-04-04T06:05:54.593377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:30:56.836366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367929+0000","last_change":"2024-04-04T06:05:53.387263+0000","last_active":"2024-04-04T06:05:54.367929+0000","last_peered":"2024-04-04T06:05:54.367929+0000","last_clean":"2024-04-04T06:05:54.367929+0000","last_became_active":"2024-04-04T06:05:53.387012+0000","last_became_peered":"2024-04-04T06:05:53.387012+0000","last_unstale":"2024-04-04T06:05:54.367929+0000","last_undegraded":"2024-04-04T06:05:54.367929+0000","last_fullsized":"2024-04-04T06:05:54.367929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:13.472940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592309+0000","last_change":"2024-04-04T06:05:53.370203+0000","last_active":"2024-04-04T06:05:54.592309+0000","last_peered":"2024-04-04T06:05:54.592309+0000","last_clean":"2024-04-04T06:05:54.592309+0000","last_became_active":"2024-04-04T06:05:53.369800+0000","last_became_peered":"2024-04-04T06:05:53.369800+0000","last_unstale":"2024-04-04T06:05:54.592309+0000","last_undegraded":"2024-04-04T06:05:54.592309+0000","last_fullsized":"2024-04-04T06:05:54.592309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:39:30.391173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592465+0000","last_change":"2024-04-04T06:05:53.369028+0000","last_active":"2024-04-04T06:05:54.592465+0000","last_peered":"2024-04-04T06:05:54.592465+0000","last_clean":"2024-04-04T06:05:54.592465+0000","last_became_active":"2024-04-04T06:05:53.365998+0000","last_became_peered":"2024-04-04T06:05:53.365998+0000","last_unstale":"2024-04-04T06:05:54.592465+0000","last_undegraded":"2024-04-04T06:05:54.592465+0000","last_fullsized":"2024-04-04T06:05:54.592465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:34.766930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594227+0000","last_change":"2024-04-04T06:05:53.369886+0000","last_active":"2024-04-04T06:05:54.594227+0000","last_peered":"2024-04-04T06:05:54.594227+0000","last_clean":"2024-04-04T06:05:54.594227+0000","last_became_active":"2024-04-04T06:05:53.368815+0000","last_became_peered":"2024-04-04T06:05:53.368815+0000","last_unstale":"2024-04-04T06:05:54.594227+0000","last_undegraded":"2024-04-04T06:05:54.594227+0000","last_fullsized":"2024-04-04T06:05:54.594227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:37:59.044530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592641+0000","last_change":"2024-04-04T06:05:53.366948+0000","last_active":"2024-04-04T06:05:54.592641+0000","last_peered":"2024-04-04T06:05:54.592641+0000","last_clean":"2024-04-04T06:05:54.592641+0000","last_became_active":"2024-04-04T06:05:53.366550+0000","last_became_peered":"2024-04-04T06:05:53.366550+0000","last_unstale":"2024-04-04T06:05:54.592641+0000","last_undegraded":"2024-04-04T06:05:54.592641+0000","last_fullsized":"2024-04-04T06:05:54.592641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:47:43.615957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592078+0000","last_change":"2024-04-04T06:05:53.376979+0000","last_active":"2024-04-04T06:05:54.592078+0000","last_peered":"2024-04-04T06:05:54.592078+0000","last_clean":"2024-04-04T06:05:54.592078+0000","last_became_active":"2024-04-04T06:05:53.376622+0000","last_became_peered":"2024-04-04T06:05:53.376622+0000","last_unstale":"2024-04-04T06:05:54.592078+0000","last_undegraded":"2024-04-04T06:05:54.592078+0000","last_fullsized":"2024-04-04T06:05:54.592078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:42:00.014108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367238+0000","last_change":"2024-04-04T06:05:53.364063+0000","last_active":"2024-04-04T06:05:54.367238+0000","last_peered":"2024-04-04T06:05:54.367238+0000","last_clean":"2024-04-04T06:05:54.367238+0000","last_became_active":"2024-04-04T06:05:53.363715+0000","last_became_peered":"2024-04-04T06:05:53.363715+0000","last_unstale":"2024-04-04T06:05:54.367238+0000","last_undegraded":"2024-04-04T06:05:54.367238+0000","last_fullsized":"2024-04-04T06:05:54.367238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:21.889411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593467+0000","last_change":"2024-04-04T06:05:53.362035+0000","last_active":"2024-04-04T06:05:54.593467+0000","last_peered":"2024-04-04T06:05:54.593467+0000","last_clean":"2024-04-04T06:05:54.593467+0000","last_became_active":"2024-04-04T06:05:53.361614+0000","last_became_peered":"2024-04-04T06:05:53.361614+0000","last_unstale":"2024-04-04T06:05:54.593467+0000","last_undegraded":"2024-04-04T06:05:54.593467+0000","last_fullsized":"2024-04-04T06:05:54.593467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:17:05.576402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590275+0000","last_change":"2024-04-04T06:05:53.355832+0000","last_active":"2024-04-04T06:05:54.590275+0000","last_peered":"2024-04-04T06:05:54.590275+0000","last_clean":"2024-04-04T06:05:54.590275+0000","last_became_active":"2024-04-04T06:05:53.355508+0000","last_became_peered":"2024-04-04T06:05:53.355508+0000","last_unstale":"2024-04-04T06:05:54.590275+0000","last_undegraded":"2024-04-04T06:05:54.590275+0000","last_fullsized":"2024-04-04T06:05:54.590275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:11:00.324524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362799+0000","last_change":"2024-04-04T06:05:53.353630+0000","last_active":"2024-04-04T06:05:54.362799+0000","last_peered":"2024-04-04T06:05:54.362799+0000","last_clean":"2024-04-04T06:05:54.362799+0000","last_became_active":"2024-04-04T06:05:53.353315+0000","last_became_peered":"2024-04-04T06:05:53.353315+0000","last_unstale":"2024-04-04T06:05:54.362799+0000","last_undegraded":"2024-04-04T06:05:54.362799+0000","last_fullsized":"2024-04-04T06:05:54.362799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.364261+0000","last_change":"2024-04-04T06:05:53.387499+0000","last_active":"2024-04-04T06:05:54.364261+0000","last_peered":"2024-04-04T06:05:54.364261+0000","last_clean":"2024-04-04T06:05:54.364261+0000","last_became_active":"2024-04-04T06:05:53.387187+0000","last_became_peered":"2024-04-04T06:05:53.387187+0000","last_unstale":"2024-04-04T06:05:54.364261+0000","last_undegraded":"2024-04-04T06:05:54.364261+0000","last_fullsized":"2024-04-04T06:05:54.364261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.366558+0000","last_change":"2024-04-04T06:05:53.378705+0000","last_active":"2024-04-04T06:05:54.366558+0000","last_peered":"2024-04-04T06:05:54.366558+0000","last_clean":"2024-04-04T06:05:54.366558+0000","last_became_active":"2024-04-04T06:05:53.378428+0000","last_became_peered":"2024-04-04T06:05:53.378428+0000","last_unstale":"2024-04-04T06:05:54.366558+0000","last_undegraded":"2024-04-04T06:05:54.366558+0000","last_fullsized":"2024-04-04T06:05:54.366558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:17:42.995360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591477+0000","last_change":"2024-04-04T06:05:53.372718+0000","last_active":"2024-04-04T06:05:54.591477+0000","last_peered":"2024-04-04T06:05:54.591477+0000","last_clean":"2024-04-04T06:05:54.591477+0000","last_became_active":"2024-04-04T06:05:53.372458+0000","last_became_peered":"2024-04-04T06:05:53.372458+0000","last_unstale":"2024-04-04T06:05:54.591477+0000","last_undegraded":"2024-04-04T06:05:54.591477+0000","last_fullsized":"2024-04-04T06:05:54.591477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:31.723861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594485+0000","last_change":"2024-04-04T06:05:53.360929+0000","last_active":"2024-04-04T06:05:54.594485+0000","last_peered":"2024-04-04T06:05:54.594485+0000","last_clean":"2024-04-04T06:05:54.594485+0000","last_became_active":"2024-04-04T06:05:53.360661+0000","last_became_peered":"2024-04-04T06:05:53.360661+0000","last_unstale":"2024-04-04T06:05:54.594485+0000","last_undegraded":"2024-04-04T06:05:54.594485+0000","last_fullsized":"2024-04-04T06:05:54.594485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:06:23.755362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591699+0000","last_change":"2024-04-04T06:05:53.374155+0000","last_active":"2024-04-04T06:05:54.591699+0000","last_peered":"2024-04-04T06:05:54.591699+0000","last_clean":"2024-04-04T06:05:54.591699+0000","last_became_active":"2024-04-04T06:05:53.373872+0000","last_became_peered":"2024-04-04T06:05:53.373872+0000","last_unstale":"2024-04-04T06:05:54.591699+0000","last_undegraded":"2024-04-04T06:05:54.591699+0000","last_fullsized":"2024-04-04T06:05:54.591699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:19:20.840644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590730+0000","last_change":"2024-04-04T06:05:53.375944+0000","last_active":"2024-04-04T06:05:54.590730+0000","last_peered":"2024-04-04T06:05:54.590730+0000","last_clean":"2024-04-04T06:05:54.590730+0000","last_became_active":"2024-04-04T06:05:53.375551+0000","last_became_peered":"2024-04-04T06:05:53.375551+0000","last_unstale":"2024-04-04T06:05:54.590730+0000","last_undegraded":"2024-04-04T06:05:54.590730+0000","last_fullsized":"2024-04-04T06:05:54.590730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:33:17.187978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367881+0000","last_change":"2024-04-04T06:05:53.377200+0000","last_active":"2024-04-04T06:05:54.367881+0000","last_peered":"2024-04-04T06:05:54.367881+0000","last_clean":"2024-04-04T06:05:54.367881+0000","last_became_active":"2024-04-04T06:05:53.373063+0000","last_became_peered":"2024-04-04T06:05:53.373063+0000","last_unstale":"2024-04-04T06:05:54.367881+0000","last_undegraded":"2024-04-04T06:05:54.367881+0000","last_fullsized":"2024-04-04T06:05:54.367881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:28:48.569552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592389+0000","last_change":"2024-04-04T06:05:53.358659+0000","last_active":"2024-04-04T06:05:54.592389+0000","last_peered":"2024-04-04T06:05:54.592389+0000","last_clean":"2024-04-04T06:05:54.592389+0000","last_became_active":"2024-04-04T06:05:53.358372+0000","last_became_peered":"2024-04-04T06:05:53.358372+0000","last_unstale":"2024-04-04T06:05:54.592389+0000","last_undegraded":"2024-04-04T06:05:54.592389+0000","last_fullsized":"2024-04-04T06:05:54.592389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:14:41.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365536+0000","last_change":"2024-04-04T06:05:53.381591+0000","last_active":"2024-04-04T06:05:54.365536+0000","last_peered":"2024-04-04T06:05:54.365536+0000","last_clean":"2024-04-04T06:05:54.365536+0000","last_became_active":"2024-04-04T06:05:53.377261+0000","last_became_peered":"2024-04-04T06:05:53.377261+0000","last_unstale":"2024-04-04T06:05:54.365536+0000","last_undegraded":"2024-04-04T06:05:54.365536+0000","last_fullsized":"2024-04-04T06:05:54.365536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:07:53.196090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593367+0000","last_change":"2024-04-04T06:05:53.371961+0000","last_active":"2024-04-04T06:05:54.593367+0000","last_peered":"2024-04-04T06:05:54.593367+0000","last_clean":"2024-04-04T06:05:54.593367+0000","last_became_active":"2024-04-04T06:05:53.371702+0000","last_became_peered":"2024-04-04T06:05:53.371702+0000","last_unstale":"2024-04-04T06:05:54.593367+0000","last_undegraded":"2024-04-04T06:05:54.593367+0000","last_fullsized":"2024-04-04T06:05:54.593367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592985+0000","last_change":"2024-04-04T06:05:53.383384+0000","last_active":"2024-04-04T06:05:54.592985+0000","last_peered":"2024-04-04T06:05:54.592985+0000","last_clean":"2024-04-04T06:05:54.592985+0000","last_became_active":"2024-04-04T06:05:53.382809+0000","last_became_peered":"2024-04-04T06:05:53.382809+0000","last_unstale":"2024-04-04T06:05:54.592985+0000","last_undegraded":"2024-04-04T06:05:54.592985+0000","last_fullsized":"2024-04-04T06:05:54.592985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:15:27.732386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362466+0000","last_change":"2024-04-04T06:05:53.373499+0000","last_active":"2024-04-04T06:05:54.362466+0000","last_peered":"2024-04-04T06:05:54.362466+0000","last_clean":"2024-04-04T06:05:54.362466+0000","last_became_active":"2024-04-04T06:05:53.367324+0000","last_became_peered":"2024-04-04T06:05:53.367324+0000","last_unstale":"2024-04-04T06:05:54.362466+0000","last_undegraded":"2024-04-04T06:05:54.362466+0000","last_fullsized":"2024-04-04T06:05:54.362466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:21:23.535889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593436+0000","last_change":"2024-04-04T06:05:53.371688+0000","last_active":"2024-04-04T06:05:54.593436+0000","last_peered":"2024-04-04T06:05:54.593436+0000","last_clean":"2024-04-04T06:05:54.593436+0000","last_became_active":"2024-04-04T06:05:53.371350+0000","last_became_peered":"2024-04-04T06:05:53.371350+0000","last_unstale":"2024-04-04T06:05:54.593436+0000","last_undegraded":"2024-04-04T06:05:54.593436+0000","last_fullsized":"2024-04-04T06:05:54.593436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:07:53.196090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590942+0000","last_change":"2024-04-04T06:05:53.370489+0000","last_active":"2024-04-04T06:05:54.590942+0000","last_peered":"2024-04-04T06:05:54.590942+0000","last_clean":"2024-04-04T06:05:54.590942+0000","last_became_active":"2024-04-04T06:05:53.367223+0000","last_became_peered":"2024-04-04T06:05:53.367223+0000","last_unstale":"2024-04-04T06:05:54.590942+0000","last_undegraded":"2024-04-04T06:05:54.590942+0000","last_fullsized":"2024-04-04T06:05:54.590942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:28:13.818701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368055+0000","last_change":"2024-04-04T06:05:53.364565+0000","last_active":"2024-04-04T06:05:54.368055+0000","last_peered":"2024-04-04T06:05:54.368055+0000","last_clean":"2024-04-04T06:05:54.368055+0000","last_became_active":"2024-04-04T06:05:53.364292+0000","last_became_peered":"2024-04-04T06:05:53.364292+0000","last_unstale":"2024-04-04T06:05:54.368055+0000","last_undegraded":"2024-04-04T06:05:54.368055+0000","last_fullsized":"2024-04-04T06:05:54.368055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:37:54.850791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590194+0000","last_change":"2024-04-04T06:05:53.366686+0000","last_active":"2024-04-04T06:05:54.590194+0000","last_peered":"2024-04-04T06:05:54.590194+0000","last_clean":"2024-04-04T06:05:54.590194+0000","last_became_active":"2024-04-04T06:05:53.366409+0000","last_became_peered":"2024-04-04T06:05:53.366409+0000","last_unstale":"2024-04-04T06:05:54.590194+0000","last_undegraded":"2024-04-04T06:05:54.590194+0000","last_fullsized":"2024-04-04T06:05:54.590194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:21.889411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591239+0000","last_change":"2024-04-04T06:05:53.361681+0000","last_active":"2024-04-04T06:05:54.591239+0000","last_peered":"2024-04-04T06:05:54.591239+0000","last_clean":"2024-04-04T06:05:54.591239+0000","last_became_active":"2024-04-04T06:05:53.361307+0000","last_became_peered":"2024-04-04T06:05:53.361307+0000","last_unstale":"2024-04-04T06:05:54.591239+0000","last_undegraded":"2024-04-04T06:05:54.591239+0000","last_fullsized":"2024-04-04T06:05:54.591239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:16:20.634573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.364842+0000","last_change":"2024-04-04T06:05:53.361164+0000","last_active":"2024-04-04T06:05:54.364842+0000","last_peered":"2024-04-04T06:05:54.364842+0000","last_clean":"2024-04-04T06:05:54.364842+0000","last_became_active":"2024-04-04T06:05:53.357301+0000","last_became_peered":"2024-04-04T06:05:53.357301+0000","last_unstale":"2024-04-04T06:05:54.364842+0000","last_undegraded":"2024-04-04T06:05:54.364842+0000","last_fullsized":"2024-04-04T06:05:54.364842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:56:10.393146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594020+0000","last_change":"2024-04-04T06:05:52.368795+0000","last_active":"2024-04-04T06:05:54.594020+0000","last_peered":"2024-04-04T06:05:54.594020+0000","last_clean":"2024-04-04T06:05:54.594020+0000","last_became_active":"2024-04-04T06:05:52.368526+0000","last_became_peered":"2024-04-04T06:05:52.368526+0000","last_unstale":"2024-04-04T06:05:54.594020+0000","last_undegraded":"2024-04-04T06:05:54.594020+0000","last_fullsized":"2024-04-04T06:05:54.594020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:45:43.480308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591181+0000","last_change":"2024-04-04T06:05:53.377207+0000","last_active":"2024-04-04T06:05:54.591181+0000","last_peered":"2024-04-04T06:05:54.591181+0000","last_clean":"2024-04-04T06:05:54.591181+0000","last_became_active":"2024-04-04T06:05:53.376853+0000","last_became_peered":"2024-04-04T06:05:53.376853+0000","last_unstale":"2024-04-04T06:05:54.591181+0000","last_undegraded":"2024-04-04T06:05:54.591181+0000","last_fullsized":"2024-04-04T06:05:54.591181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:21:23.535889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591256+0000","last_change":"2024-04-04T06:05:52.372922+0000","last_active":"2024-04-04T06:05:54.591256+0000","last_peered":"2024-04-04T06:05:54.591256+0000","last_clean":"2024-04-04T06:05:54.591256+0000","last_became_active":"2024-04-04T06:05:52.372615+0000","last_became_peered":"2024-04-04T06:05:52.372615+0000","last_unstale":"2024-04-04T06:05:54.591256+0000","last_undegraded":"2024-04-04T06:05:54.591256+0000","last_fullsized":"2024-04-04T06:05:54.591256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:36:38.928830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592633+0000","last_change":"2024-04-04T06:05:52.371943+0000","last_active":"2024-04-04T06:05:54.592633+0000","last_peered":"2024-04-04T06:05:54.592633+0000","last_clean":"2024-04-04T06:05:54.592633+0000","last_became_active":"2024-04-04T06:05:52.371686+0000","last_became_peered":"2024-04-04T06:05:52.371686+0000","last_unstale":"2024-04-04T06:05:54.592633+0000","last_undegraded":"2024-04-04T06:05:54.592633+0000","last_fullsized":"2024-04-04T06:05:54.592633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:52:05.788594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592504+0000","last_change":"2024-04-04T06:05:53.363121+0000","last_active":"2024-04-04T06:05:54.592504+0000","last_peered":"2024-04-04T06:05:54.592504+0000","last_clean":"2024-04-04T06:05:54.592504+0000","last_became_active":"2024-04-04T06:05:53.362861+0000","last_became_peered":"2024-04-04T06:05:53.362861+0000","last_unstale":"2024-04-04T06:05:54.592504+0000","last_undegraded":"2024-04-04T06:05:54.592504+0000","last_fullsized":"2024-04-04T06:05:54.592504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:28:13.818701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365617+0000","last_change":"2024-04-04T06:05:52.371090+0000","last_active":"2024-04-04T06:05:54.365617+0000","last_peered":"2024-04-04T06:05:54.365617+0000","last_clean":"2024-04-04T06:05:54.365617+0000","last_became_active":"2024-04-04T06:05:52.369650+0000","last_became_peered":"2024-04-04T06:05:52.369650+0000","last_unstale":"2024-04-04T06:05:54.365617+0000","last_undegraded":"2024-04-04T06:05:54.365617+0000","last_fullsized":"2024-04-04T06:05:54.365617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:45:43.480308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362432+0000","last_change":"2024-04-04T06:05:53.357537+0000","last_active":"2024-04-04T06:05:54.362432+0000","last_peered":"2024-04-04T06:05:54.362432+0000","last_clean":"2024-04-04T06:05:54.362432+0000","last_became_active":"2024-04-04T06:05:53.357284+0000","last_became_peered":"2024-04-04T06:05:53.357284+0000","last_unstale":"2024-04-04T06:05:54.362432+0000","last_undegraded":"2024-04-04T06:05:54.362432+0000","last_fullsized":"2024-04-04T06:05:54.362432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:28:13.818701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592398+0000","last_change":"2024-04-04T06:05:52.374921+0000","last_active":"2024-04-04T06:05:54.592398+0000","last_peered":"2024-04-04T06:05:54.592398+0000","last_clean":"2024-04-04T06:05:54.592398+0000","last_became_active":"2024-04-04T06:05:52.374555+0000","last_became_peered":"2024-04-04T06:05:52.374555+0000","last_unstale":"2024-04-04T06:05:54.592398+0000","last_undegraded":"2024-04-04T06:05:54.592398+0000","last_fullsized":"2024-04-04T06:05:54.592398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:05.925335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368597+0000","last_change":"2024-04-04T06:05:53.377612+0000","last_active":"2024-04-04T06:05:54.368597+0000","last_peered":"2024-04-04T06:05:54.368597+0000","last_clean":"2024-04-04T06:05:54.368597+0000","last_became_active":"2024-04-04T06:05:53.374258+0000","last_became_peered":"2024-04-04T06:05:53.374258+0000","last_unstale":"2024-04-04T06:05:54.368597+0000","last_undegraded":"2024-04-04T06:05:54.368597+0000","last_fullsized":"2024-04-04T06:05:54.368597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:08:44.769578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368664+0000","last_change":"2024-04-04T06:05:52.373979+0000","last_active":"2024-04-04T06:05:54.368664+0000","last_peered":"2024-04-04T06:05:54.368664+0000","last_clean":"2024-04-04T06:05:54.368664+0000","last_became_active":"2024-04-04T06:05:52.373408+0000","last_became_peered":"2024-04-04T06:05:52.373408+0000","last_unstale":"2024-04-04T06:05:54.368664+0000","last_undegraded":"2024-04-04T06:05:54.368664+0000","last_fullsized":"2024-04-04T06:05:54.368664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:56:09.383441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591338+0000","last_change":"2024-04-04T06:05:53.368728+0000","last_active":"2024-04-04T06:05:54.591338+0000","last_peered":"2024-04-04T06:05:54.591338+0000","last_clean":"2024-04-04T06:05:54.591338+0000","last_became_active":"2024-04-04T06:05:53.367971+0000","last_became_peered":"2024-04-04T06:05:53.367971+0000","last_unstale":"2024-04-04T06:05:54.591338+0000","last_undegraded":"2024-04-04T06:05:54.591338+0000","last_fullsized":"2024-04-04T06:05:54.591338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362602+0000","last_change":"2024-04-04T06:05:52.368574+0000","last_active":"2024-04-04T06:05:54.362602+0000","last_peered":"2024-04-04T06:05:54.362602+0000","last_clean":"2024-04-04T06:05:54.362602+0000","last_became_active":"2024-04-04T06:05:52.367460+0000","last_became_peered":"2024-04-04T06:05:52.367460+0000","last_unstale":"2024-04-04T06:05:54.362602+0000","last_undegraded":"2024-04-04T06:05:54.362602+0000","last_fullsized":"2024-04-04T06:05:54.362602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:56:09.383441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367547+0000","last_change":"2024-04-04T06:05:53.379475+0000","last_active":"2024-04-04T06:05:54.367547+0000","last_peered":"2024-04-04T06:05:54.367547+0000","last_clean":"2024-04-04T06:05:54.367547+0000","last_became_active":"2024-04-04T06:05:53.376075+0000","last_became_peered":"2024-04-04T06:05:53.376075+0000","last_unstale":"2024-04-04T06:05:54.367547+0000","last_undegraded":"2024-04-04T06:05:54.367547+0000","last_fullsized":"2024-04-04T06:05:54.367547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:03:00.104400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591818+0000","last_change":"2024-04-04T06:05:52.370133+0000","last_active":"2024-04-04T06:05:54.591818+0000","last_peered":"2024-04-04T06:05:54.591818+0000","last_clean":"2024-04-04T06:05:54.591818+0000","last_became_active":"2024-04-04T06:05:52.369825+0000","last_became_peered":"2024-04-04T06:05:52.369825+0000","last_unstale":"2024-04-04T06:05:54.591818+0000","last_undegraded":"2024-04-04T06:05:54.591818+0000","last_fullsized":"2024-04-04T06:05:54.591818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:01:02.629926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594503+0000","last_change":"2024-04-04T06:05:53.371020+0000","last_active":"2024-04-04T06:05:54.594503+0000","last_peered":"2024-04-04T06:05:54.594503+0000","last_clean":"2024-04-04T06:05:54.594503+0000","last_became_active":"2024-04-04T06:05:53.370755+0000","last_became_peered":"2024-04-04T06:05:53.370755+0000","last_unstale":"2024-04-04T06:05:54.594503+0000","last_undegraded":"2024-04-04T06:05:54.594503+0000","last_fullsized":"2024-04-04T06:05:54.594503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:54:36.353079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591316+0000","last_change":"2024-04-04T06:05:52.367020+0000","last_active":"2024-04-04T06:05:54.591316+0000","last_peered":"2024-04-04T06:05:54.591316+0000","last_clean":"2024-04-04T06:05:54.591316+0000","last_became_active":"2024-04-04T06:05:52.366654+0000","last_became_peered":"2024-04-04T06:05:52.366654+0000","last_unstale":"2024-04-04T06:05:54.591316+0000","last_undegraded":"2024-04-04T06:05:54.591316+0000","last_fullsized":"2024-04-04T06:05:54.591316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:26:03.163432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590433+0000","last_change":"2024-04-04T06:05:53.373111+0000","last_active":"2024-04-04T06:05:54.590433+0000","last_peered":"2024-04-04T06:05:54.590433+0000","last_clean":"2024-04-04T06:05:54.590433+0000","last_became_active":"2024-04-04T06:05:53.372761+0000","last_became_peered":"2024-04-04T06:05:53.372761+0000","last_unstale":"2024-04-04T06:05:54.590433+0000","last_undegraded":"2024-04-04T06:05:54.590433+0000","last_fullsized":"2024-04-04T06:05:54.590433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:27:06.279930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368191+0000","last_change":"2024-04-04T06:05:52.365334+0000","last_active":"2024-04-04T06:05:54.368191+0000","last_peered":"2024-04-04T06:05:54.368191+0000","last_clean":"2024-04-04T06:05:54.368191+0000","last_became_active":"2024-04-04T06:05:52.363154+0000","last_became_peered":"2024-04-04T06:05:52.363154+0000","last_unstale":"2024-04-04T06:05:54.368191+0000","last_undegraded":"2024-04-04T06:05:54.368191+0000","last_fullsized":"2024-04-04T06:05:54.368191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:53:42.904072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593559+0000","last_change":"2024-04-04T06:05:53.380196+0000","last_active":"2024-04-04T06:05:54.593559+0000","last_peered":"2024-04-04T06:05:54.593559+0000","last_clean":"2024-04-04T06:05:54.593559+0000","last_became_active":"2024-04-04T06:05:53.379800+0000","last_became_peered":"2024-04-04T06:05:53.379800+0000","last_unstale":"2024-04-04T06:05:54.593559+0000","last_undegraded":"2024-04-04T06:05:54.593559+0000","last_fullsized":"2024-04-04T06:05:54.593559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:41:09.510239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591089+0000","last_change":"2024-04-04T06:05:52.377332+0000","last_active":"2024-04-04T06:05:54.591089+0000","last_peered":"2024-04-04T06:05:54.591089+0000","last_clean":"2024-04-04T06:05:54.591089+0000","last_became_active":"2024-04-04T06:05:52.377002+0000","last_became_peered":"2024-04-04T06:05:52.377002+0000","last_unstale":"2024-04-04T06:05:54.591089+0000","last_undegraded":"2024-04-04T06:05:54.591089+0000","last_fullsized":"2024-04-04T06:05:54.591089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:05:57.500385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591711+0000","last_change":"2024-04-04T06:05:53.378591+0000","last_active":"2024-04-04T06:05:54.591711+0000","last_peered":"2024-04-04T06:05:54.591711+0000","last_clean":"2024-04-04T06:05:54.591711+0000","last_became_active":"2024-04-04T06:05:53.378227+0000","last_became_peered":"2024-04-04T06:05:53.378227+0000","last_unstale":"2024-04-04T06:05:54.591711+0000","last_undegraded":"2024-04-04T06:05:54.591711+0000","last_fullsized":"2024-04-04T06:05:54.591711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:42:54.347331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.589595+0000","last_change":"2024-04-04T06:05:52.357745+0000","last_active":"2024-04-04T06:05:54.589595+0000","last_peered":"2024-04-04T06:05:54.589595+0000","last_clean":"2024-04-04T06:05:54.589595+0000","last_became_active":"2024-04-04T06:05:52.357280+0000","last_became_peered":"2024-04-04T06:05:52.357280+0000","last_unstale":"2024-04-04T06:05:54.589595+0000","last_undegraded":"2024-04-04T06:05:54.589595+0000","last_fullsized":"2024-04-04T06:05:54.589595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:45:43.480308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592420+0000","last_change":"2024-04-04T06:05:53.387060+0000","last_active":"2024-04-04T06:05:54.592420+0000","last_peered":"2024-04-04T06:05:54.592420+0000","last_clean":"2024-04-04T06:05:54.592420+0000","last_became_active":"2024-04-04T06:05:53.386804+0000","last_became_peered":"2024-04-04T06:05:53.386804+0000","last_unstale":"2024-04-04T06:05:54.592420+0000","last_undegraded":"2024-04-04T06:05:54.592420+0000","last_fullsized":"2024-04-04T06:05:54.592420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:37:36.681617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367992+0000","last_change":"2024-04-04T06:05:52.362010+0000","last_active":"2024-04-04T06:05:54.367992+0000","last_peered":"2024-04-04T06:05:54.367992+0000","last_clean":"2024-04-04T06:05:54.367992+0000","last_became_active":"2024-04-04T06:05:52.361424+0000","last_became_peered":"2024-04-04T06:05:52.361424+0000","last_unstale":"2024-04-04T06:05:54.367992+0000","last_undegraded":"2024-04-04T06:05:54.367992+0000","last_fullsized":"2024-04-04T06:05:54.367992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:26:03.163432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594202+0000","last_change":"2024-04-04T06:05:53.375944+0000","last_active":"2024-04-04T06:05:54.594202+0000","last_peered":"2024-04-04T06:05:54.594202+0000","last_clean":"2024-04-04T06:05:54.594202+0000","last_became_active":"2024-04-04T06:05:53.375684+0000","last_became_peered":"2024-04-04T06:05:53.375684+0000","last_unstale":"2024-04-04T06:05:54.594202+0000","last_undegraded":"2024-04-04T06:05:54.594202+0000","last_fullsized":"2024-04-04T06:05:54.594202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:02:57.074580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591274+0000","last_change":"2024-04-04T06:05:52.358735+0000","last_active":"2024-04-04T06:05:54.591274+0000","last_peered":"2024-04-04T06:05:54.591274+0000","last_clean":"2024-04-04T06:05:54.591274+0000","last_became_active":"2024-04-04T06:05:52.358203+0000","last_became_peered":"2024-04-04T06:05:52.358203+0000","last_unstale":"2024-04-04T06:05:54.591274+0000","last_undegraded":"2024-04-04T06:05:54.591274+0000","last_fullsized":"2024-04-04T06:05:54.591274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:53:42.904072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594617+0000","last_change":"2024-04-04T06:05:53.388782+0000","last_active":"2024-04-04T06:05:54.594617+0000","last_peered":"2024-04-04T06:05:54.594617+0000","last_clean":"2024-04-04T06:05:54.594617+0000","last_became_active":"2024-04-04T06:05:53.388230+0000","last_became_peered":"2024-04-04T06:05:53.388230+0000","last_unstale":"2024-04-04T06:05:54.594617+0000","last_undegraded":"2024-04-04T06:05:54.594617+0000","last_fullsized":"2024-04-04T06:05:54.594617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:31:19.282600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362887+0000","last_change":"2024-04-04T06:05:52.366026+0000","last_active":"2024-04-04T06:05:54.362887+0000","last_peered":"2024-04-04T06:05:54.362887+0000","last_clean":"2024-04-04T06:05:54.362887+0000","last_became_active":"2024-04-04T06:05:52.365663+0000","last_became_peered":"2024-04-04T06:05:52.365663+0000","last_unstale":"2024-04-04T06:05:54.362887+0000","last_undegraded":"2024-04-04T06:05:54.362887+0000","last_fullsized":"2024-04-04T06:05:54.362887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:30.714156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.366141+0000","last_change":"2024-04-04T06:05:53.381556+0000","last_active":"2024-04-04T06:05:54.366141+0000","last_peered":"2024-04-04T06:05:54.366141+0000","last_clean":"2024-04-04T06:05:54.366141+0000","last_became_active":"2024-04-04T06:05:53.373492+0000","last_became_peered":"2024-04-04T06:05:53.373492+0000","last_unstale":"2024-04-04T06:05:54.366141+0000","last_undegraded":"2024-04-04T06:05:54.366141+0000","last_fullsized":"2024-04-04T06:05:54.366141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:53:19.419468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590914+0000","last_change":"2024-04-04T06:05:52.353584+0000","last_active":"2024-04-04T06:05:54.590914+0000","last_peered":"2024-04-04T06:05:54.590914+0000","last_clean":"2024-04-04T06:05:54.590914+0000","last_became_active":"2024-04-04T06:05:52.353017+0000","last_became_peered":"2024-04-04T06:05:52.353017+0000","last_unstale":"2024-04-04T06:05:54.590914+0000","last_undegraded":"2024-04-04T06:05:54.590914+0000","last_fullsized":"2024-04-04T06:05:54.590914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:58:40.874784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593698+0000","last_change":"2024-04-04T06:05:53.368622+0000","last_active":"2024-04-04T06:05:54.593698+0000","last_peered":"2024-04-04T06:05:54.593698+0000","last_clean":"2024-04-04T06:05:54.593698+0000","last_became_active":"2024-04-04T06:05:53.368311+0000","last_became_peered":"2024-04-04T06:05:53.368311+0000","last_unstale":"2024-04-04T06:05:54.593698+0000","last_undegraded":"2024-04-04T06:05:54.593698+0000","last_fullsized":"2024-04-04T06:05:54.593698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:01:24.602038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591729+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:05:54.591729+0000","last_peered":"2024-04-04T06:05:54.591729+0000","last_clean":"2024-04-04T06:05:54.591729+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:05:54.591729+0000","last_undegraded":"2024-04-04T06:05:54.591729+0000","last_fullsized":"2024-04-04T06:05:54.591729+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365145+0000","last_change":"2024-04-04T06:05:52.356730+0000","last_active":"2024-04-04T06:05:54.365145+0000","last_peered":"2024-04-04T06:05:54.365145+0000","last_clean":"2024-04-04T06:05:54.365145+0000","last_became_active":"2024-04-04T06:05:52.352963+0000","last_became_peered":"2024-04-04T06:05:52.352963+0000","last_unstale":"2024-04-04T06:05:54.365145+0000","last_undegraded":"2024-04-04T06:05:54.365145+0000","last_fullsized":"2024-04-04T06:05:54.365145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:05:06.878796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592869+0000","last_change":"2024-04-04T06:05:53.366506+0000","last_active":"2024-04-04T06:05:54.592869+0000","last_peered":"2024-04-04T06:05:54.592869+0000","last_clean":"2024-04-04T06:05:54.592869+0000","last_became_active":"2024-04-04T06:05:53.365677+0000","last_became_peered":"2024-04-04T06:05:53.365677+0000","last_unstale":"2024-04-04T06:05:54.592869+0000","last_undegraded":"2024-04-04T06:05:54.592869+0000","last_fullsized":"2024-04-04T06:05:54.592869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:10.411661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591456+0000","last_change":"2024-04-04T06:05:52.374798+0000","last_active":"2024-04-04T06:05:54.591456+0000","last_peered":"2024-04-04T06:05:54.591456+0000","last_clean":"2024-04-04T06:05:54.591456+0000","last_became_active":"2024-04-04T06:05:52.373838+0000","last_became_peered":"2024-04-04T06:05:52.373838+0000","last_unstale":"2024-04-04T06:05:54.591456+0000","last_undegraded":"2024-04-04T06:05:54.591456+0000","last_fullsized":"2024-04-04T06:05:54.591456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:51:50.301107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590751+0000","last_change":"2024-04-04T06:05:53.385393+0000","last_active":"2024-04-04T06:05:54.590751+0000","last_peered":"2024-04-04T06:05:54.590751+0000","last_clean":"2024-04-04T06:05:54.590751+0000","last_became_active":"2024-04-04T06:05:53.385062+0000","last_became_peered":"2024-04-04T06:05:53.385062+0000","last_unstale":"2024-04-04T06:05:54.590751+0000","last_undegraded":"2024-04-04T06:05:54.590751+0000","last_fullsized":"2024-04-04T06:05:54.590751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:14:41.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590309+0000","last_change":"2024-04-04T06:05:52.360018+0000","last_active":"2024-04-04T06:05:54.590309+0000","last_peered":"2024-04-04T06:05:54.590309+0000","last_clean":"2024-04-04T06:05:54.590309+0000","last_became_active":"2024-04-04T06:05:52.359735+0000","last_became_peered":"2024-04-04T06:05:52.359735+0000","last_unstale":"2024-04-04T06:05:54.590309+0000","last_undegraded":"2024-04-04T06:05:54.590309+0000","last_fullsized":"2024-04-04T06:05:54.590309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:05:06.878796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.363467+0000","last_change":"2024-04-04T06:05:53.356771+0000","last_active":"2024-04-04T06:05:54.363467+0000","last_peered":"2024-04-04T06:05:54.363467+0000","last_clean":"2024-04-04T06:05:54.363467+0000","last_became_active":"2024-04-04T06:05:53.356431+0000","last_became_peered":"2024-04-04T06:05:53.356431+0000","last_unstale":"2024-04-04T06:05:54.363467+0000","last_undegraded":"2024-04-04T06:05:54.363467+0000","last_fullsized":"2024-04-04T06:05:54.363467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:07:53.196090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368163+0000","last_change":"2024-04-04T06:05:52.370387+0000","last_active":"2024-04-04T06:05:54.368163+0000","last_peered":"2024-04-04T06:05:54.368163+0000","last_clean":"2024-04-04T06:05:54.368163+0000","last_became_active":"2024-04-04T06:05:52.369905+0000","last_became_peered":"2024-04-04T06:05:52.369905+0000","last_unstale":"2024-04-04T06:05:54.368163+0000","last_undegraded":"2024-04-04T06:05:54.368163+0000","last_fullsized":"2024-04-04T06:05:54.368163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:58:40.874784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367307+0000","last_change":"2024-04-04T06:05:53.371957+0000","last_active":"2024-04-04T06:05:54.367307+0000","last_peered":"2024-04-04T06:05:54.367307+0000","last_clean":"2024-04-04T06:05:54.367307+0000","last_became_active":"2024-04-04T06:05:53.371121+0000","last_became_peered":"2024-04-04T06:05:53.371121+0000","last_unstale":"2024-04-04T06:05:54.367307+0000","last_undegraded":"2024-04-04T06:05:54.367307+0000","last_fullsized":"2024-04-04T06:05:54.367307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:31.723861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362920+0000","last_change":"2024-04-04T06:05:52.361392+0000","last_active":"2024-04-04T06:05:54.362920+0000","last_peered":"2024-04-04T06:05:54.362920+0000","last_clean":"2024-04-04T06:05:54.362920+0000","last_became_active":"2024-04-04T06:05:52.360932+0000","last_became_peered":"2024-04-04T06:05:52.360932+0000","last_unstale":"2024-04-04T06:05:54.362920+0000","last_undegraded":"2024-04-04T06:05:54.362920+0000","last_fullsized":"2024-04-04T06:05:54.362920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:20.879706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365934+0000","last_change":"2024-04-04T06:05:53.379438+0000","last_active":"2024-04-04T06:05:54.365934+0000","last_peered":"2024-04-04T06:05:54.365934+0000","last_clean":"2024-04-04T06:05:54.365934+0000","last_became_active":"2024-04-04T06:05:53.378781+0000","last_became_peered":"2024-04-04T06:05:53.378781+0000","last_unstale":"2024-04-04T06:05:54.365934+0000","last_undegraded":"2024-04-04T06:05:54.365934+0000","last_fullsized":"2024-04-04T06:05:54.365934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:18:05.337194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594629+0000","last_change":"2024-04-04T06:05:52.372922+0000","last_active":"2024-04-04T06:05:54.594629+0000","last_peered":"2024-04-04T06:05:54.594629+0000","last_clean":"2024-04-04T06:05:54.594629+0000","last_became_active":"2024-04-04T06:05:52.372696+0000","last_became_peered":"2024-04-04T06:05:52.372696+0000","last_unstale":"2024-04-04T06:05:54.594629+0000","last_undegraded":"2024-04-04T06:05:54.594629+0000","last_fullsized":"2024-04-04T06:05:54.594629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:17:53.847287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362796+0000","last_change":"2024-04-04T06:05:53.373066+0000","last_active":"2024-04-04T06:05:54.362796+0000","last_peered":"2024-04-04T06:05:54.362796+0000","last_clean":"2024-04-04T06:05:54.362796+0000","last_became_active":"2024-04-04T06:05:53.367063+0000","last_became_peered":"2024-04-04T06:05:53.367063+0000","last_unstale":"2024-04-04T06:05:54.362796+0000","last_undegraded":"2024-04-04T06:05:54.362796+0000","last_fullsized":"2024-04-04T06:05:54.362796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:10.411661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.363331+0000","last_change":"2024-04-04T06:05:52.365125+0000","last_active":"2024-04-04T06:05:54.363331+0000","last_peered":"2024-04-04T06:05:54.363331+0000","last_clean":"2024-04-04T06:05:54.363331+0000","last_became_active":"2024-04-04T06:05:52.364821+0000","last_became_peered":"2024-04-04T06:05:52.364821+0000","last_unstale":"2024-04-04T06:05:54.363331+0000","last_undegraded":"2024-04-04T06:05:54.363331+0000","last_fullsized":"2024-04-04T06:05:54.363331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:54:42.610854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593547+0000","last_change":"2024-04-04T06:05:53.375887+0000","last_active":"2024-04-04T06:05:54.593547+0000","last_peered":"2024-04-04T06:05:54.593547+0000","last_clean":"2024-04-04T06:05:54.593547+0000","last_became_active":"2024-04-04T06:05:53.374688+0000","last_became_peered":"2024-04-04T06:05:53.374688+0000","last_unstale":"2024-04-04T06:05:54.593547+0000","last_undegraded":"2024-04-04T06:05:54.593547+0000","last_fullsized":"2024-04-04T06:05:54.593547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:45:21.548886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592567+0000","last_change":"2024-04-04T06:05:52.374804+0000","last_active":"2024-04-04T06:05:54.592567+0000","last_peered":"2024-04-04T06:05:54.592567+0000","last_clean":"2024-04-04T06:05:54.592567+0000","last_became_active":"2024-04-04T06:05:52.374476+0000","last_became_peered":"2024-04-04T06:05:52.374476+0000","last_unstale":"2024-04-04T06:05:54.592567+0000","last_undegraded":"2024-04-04T06:05:54.592567+0000","last_fullsized":"2024-04-04T06:05:54.592567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:21:06.579458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590233+0000","last_change":"2024-04-04T06:05:53.358691+0000","last_active":"2024-04-04T06:05:54.590233+0000","last_peered":"2024-04-04T06:05:54.590233+0000","last_clean":"2024-04-04T06:05:54.590233+0000","last_became_active":"2024-04-04T06:05:53.357996+0000","last_became_peered":"2024-04-04T06:05:53.357996+0000","last_unstale":"2024-04-04T06:05:54.590233+0000","last_undegraded":"2024-04-04T06:05:54.590233+0000","last_fullsized":"2024-04-04T06:05:54.590233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:55:58.689192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592379+0000","last_change":"2024-04-04T06:05:52.370047+0000","last_active":"2024-04-04T06:05:54.592379+0000","last_peered":"2024-04-04T06:05:54.592379+0000","last_clean":"2024-04-04T06:05:54.592379+0000","last_became_active":"2024-04-04T06:05:52.369741+0000","last_became_peered":"2024-04-04T06:05:52.369741+0000","last_unstale":"2024-04-04T06:05:54.592379+0000","last_undegraded":"2024-04-04T06:05:54.592379+0000","last_fullsized":"2024-04-04T06:05:54.592379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:36:38.928830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.364951+0000","last_change":"2024-04-04T06:05:53.364563+0000","last_active":"2024-04-04T06:05:54.364951+0000","last_peered":"2024-04-04T06:05:54.364951+0000","last_clean":"2024-04-04T06:05:54.364951+0000","last_became_active":"2024-04-04T06:05:53.362702+0000","last_became_peered":"2024-04-04T06:05:53.362702+0000","last_unstale":"2024-04-04T06:05:54.364951+0000","last_undegraded":"2024-04-04T06:05:54.364951+0000","last_fullsized":"2024-04-04T06:05:54.364951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:54:43.620559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362753+0000","last_change":"2024-04-04T06:05:52.370003+0000","last_active":"2024-04-04T06:05:54.362753+0000","last_peered":"2024-04-04T06:05:54.362753+0000","last_clean":"2024-04-04T06:05:54.362753+0000","last_became_active":"2024-04-04T06:05:52.369724+0000","last_became_peered":"2024-04-04T06:05:52.369724+0000","last_unstale":"2024-04-04T06:05:54.362753+0000","last_undegraded":"2024-04-04T06:05:54.362753+0000","last_fullsized":"2024-04-04T06:05:54.362753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:58:40.874784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592885+0000","last_change":"2024-04-04T06:05:53.361642+0000","last_active":"2024-04-04T06:05:54.592885+0000","last_peered":"2024-04-04T06:05:54.592885+0000","last_clean":"2024-04-04T06:05:54.592885+0000","last_became_active":"2024-04-04T06:05:53.360951+0000","last_became_peered":"2024-04-04T06:05:53.360951+0000","last_unstale":"2024-04-04T06:05:54.592885+0000","last_undegraded":"2024-04-04T06:05:54.592885+0000","last_fullsized":"2024-04-04T06:05:54.592885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:21:23.535889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362582+0000","last_change":"2024-04-04T06:05:52.371223+0000","last_active":"2024-04-04T06:05:54.362582+0000","last_peered":"2024-04-04T06:05:54.362582+0000","last_clean":"2024-04-04T06:05:54.362582+0000","last_became_active":"2024-04-04T06:05:52.370916+0000","last_became_peered":"2024-04-04T06:05:52.370916+0000","last_unstale":"2024-04-04T06:05:54.362582+0000","last_undegraded":"2024-04-04T06:05:54.362582+0000","last_fullsized":"2024-04-04T06:05:54.362582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:53:42.904072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593889+0000","last_change":"2024-04-04T06:05:53.371605+0000","last_active":"2024-04-04T06:05:54.593889+0000","last_peered":"2024-04-04T06:05:54.593889+0000","last_clean":"2024-04-04T06:05:54.593889+0000","last_became_active":"2024-04-04T06:05:53.371328+0000","last_became_peered":"2024-04-04T06:05:53.371328+0000","last_unstale":"2024-04-04T06:05:54.593889+0000","last_undegraded":"2024-04-04T06:05:54.593889+0000","last_fullsized":"2024-04-04T06:05:54.593889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:18:05.337194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593483+0000","last_change":"2024-04-04T06:05:52.368424+0000","last_active":"2024-04-04T06:05:54.593483+0000","last_peered":"2024-04-04T06:05:54.593483+0000","last_clean":"2024-04-04T06:05:54.593483+0000","last_became_active":"2024-04-04T06:05:52.368164+0000","last_became_peered":"2024-04-04T06:05:52.368164+0000","last_unstale":"2024-04-04T06:05:54.593483+0000","last_undegraded":"2024-04-04T06:05:54.593483+0000","last_fullsized":"2024-04-04T06:05:54.593483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:54:18.736812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591138+0000","last_change":"2024-04-04T06:05:53.373894+0000","last_active":"2024-04-04T06:05:54.591138+0000","last_peered":"2024-04-04T06:05:54.591138+0000","last_clean":"2024-04-04T06:05:54.591138+0000","last_became_active":"2024-04-04T06:05:53.373615+0000","last_became_peered":"2024-04-04T06:05:53.373615+0000","last_unstale":"2024-04-04T06:05:54.591138+0000","last_undegraded":"2024-04-04T06:05:54.591138+0000","last_fullsized":"2024-04-04T06:05:54.591138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:03:39.636711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365635+0000","last_change":"2024-04-04T06:05:53.378971+0000","last_active":"2024-04-04T06:05:54.365635+0000","last_peered":"2024-04-04T06:05:54.365635+0000","last_clean":"2024-04-04T06:05:54.365635+0000","last_became_active":"2024-04-04T06:05:53.377287+0000","last_became_peered":"2024-04-04T06:05:53.377287+0000","last_unstale":"2024-04-04T06:05:54.365635+0000","last_undegraded":"2024-04-04T06:05:54.365635+0000","last_fullsized":"2024-04-04T06:05:54.365635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:17:05.576402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594399+0000","last_change":"2024-04-04T06:05:52.376155+0000","last_active":"2024-04-04T06:05:54.594399+0000","last_peered":"2024-04-04T06:05:54.594399+0000","last_clean":"2024-04-04T06:05:54.594399+0000","last_became_active":"2024-04-04T06:05:52.375772+0000","last_became_peered":"2024-04-04T06:05:52.375772+0000","last_unstale":"2024-04-04T06:05:54.594399+0000","last_undegraded":"2024-04-04T06:05:54.594399+0000","last_fullsized":"2024-04-04T06:05:54.594399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:40:09.906862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365922+0000","last_change":"2024-04-04T06:05:53.381179+0000","last_active":"2024-04-04T06:05:54.365922+0000","last_peered":"2024-04-04T06:05:54.365922+0000","last_clean":"2024-04-04T06:05:54.365922+0000","last_became_active":"2024-04-04T06:05:53.374859+0000","last_became_peered":"2024-04-04T06:05:53.374859+0000","last_unstale":"2024-04-04T06:05:54.365922+0000","last_undegraded":"2024-04-04T06:05:54.365922+0000","last_fullsized":"2024-04-04T06:05:54.365922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:20:24.966139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593848+0000","last_change":"2024-04-04T06:05:52.378067+0000","last_active":"2024-04-04T06:05:54.593848+0000","last_peered":"2024-04-04T06:05:54.593848+0000","last_clean":"2024-04-04T06:05:54.593848+0000","last_became_active":"2024-04-04T06:05:52.377777+0000","last_became_peered":"2024-04-04T06:05:52.377777+0000","last_unstale":"2024-04-04T06:05:54.593848+0000","last_undegraded":"2024-04-04T06:05:54.593848+0000","last_fullsized":"2024-04-04T06:05:54.593848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:01:02.629926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590922+0000","last_change":"2024-04-04T06:05:53.368705+0000","last_active":"2024-04-04T06:05:54.590922+0000","last_peered":"2024-04-04T06:05:54.590922+0000","last_clean":"2024-04-04T06:05:54.590922+0000","last_became_active":"2024-04-04T06:05:53.368366+0000","last_became_peered":"2024-04-04T06:05:53.368366+0000","last_unstale":"2024-04-04T06:05:54.590922+0000","last_undegraded":"2024-04-04T06:05:54.590922+0000","last_fullsized":"2024-04-04T06:05:54.590922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:34.766930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593814+0000","last_change":"2024-04-04T06:05:52.367095+0000","last_active":"2024-04-04T06:05:54.593814+0000","last_peered":"2024-04-04T06:05:54.593814+0000","last_clean":"2024-04-04T06:05:54.593814+0000","last_became_active":"2024-04-04T06:05:52.366316+0000","last_became_peered":"2024-04-04T06:05:52.366316+0000","last_unstale":"2024-04-04T06:05:54.593814+0000","last_undegraded":"2024-04-04T06:05:54.593814+0000","last_fullsized":"2024-04-04T06:05:54.593814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:05:06.878796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593646+0000","last_change":"2024-04-04T06:05:53.375226+0000","last_active":"2024-04-04T06:05:54.593646+0000","last_peered":"2024-04-04T06:05:54.593646+0000","last_clean":"2024-04-04T06:05:54.593646+0000","last_became_active":"2024-04-04T06:05:53.374921+0000","last_became_peered":"2024-04-04T06:05:53.374921+0000","last_unstale":"2024-04-04T06:05:54.593646+0000","last_undegraded":"2024-04-04T06:05:54.593646+0000","last_fullsized":"2024-04-04T06:05:54.593646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:20:24.966139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591478+0000","last_change":"2024-04-04T06:05:52.370572+0000","last_active":"2024-04-04T06:05:54.591478+0000","last_peered":"2024-04-04T06:05:54.591478+0000","last_clean":"2024-04-04T06:05:54.591478+0000","last_became_active":"2024-04-04T06:05:52.369378+0000","last_became_peered":"2024-04-04T06:05:52.369378+0000","last_unstale":"2024-04-04T06:05:54.591478+0000","last_undegraded":"2024-04-04T06:05:54.591478+0000","last_fullsized":"2024-04-04T06:05:54.591478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:45:47.508028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":806912,"data_stored":663433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":74095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":74095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":72483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":663489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":72573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":72609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:06:03.298 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-04T06:06:03.298 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-04T06:06:03.298 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-04T06:06:03.298 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-04T06:06:03.299 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-04T06:06:03.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.557+0000 7ff0b0a00640 1 Processor -- start 2024-04-04T06:06:03.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.557+0000 7ff0b0a00640 1 -- start start 2024-04-04T06:06:03.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.558+0000 7ff0b0a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff0ac15c390 0x7ff0ac073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:03.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.558+0000 7ff0b0a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff0ac0af830 0x7ff0ac160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:03.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.558+0000 7ff0b0a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0ac15fe80 con 0x7ff0ac160b80 2024-04-04T06:06:03.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.558+0000 7ff0b0a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7ff0ac15c770 con 0x7ff0ac0af830 2024-04-04T06:06:03.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.558+0000 7ff0b0a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff0ac0ad120 con 0x7ff0ac15c390 2024-04-04T06:06:03.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.559+0000 7ff0a1400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff0ac0af830 0x7ff0ac160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:03.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.559+0000 7ff0aa800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7ff0ac160b80 0x7ff0ac0d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43542/0 (socket says 172.21.15.159:43542) 2024-04-04T06:06:03.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.559+0000 7ff0aa800640 1 -- 172.21.15.159:0/475818609 learned_addr learned my addr 172.21.15.159:0/475818609 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:03.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.559+0000 7ff0a9e00640 1 --2- 172.21.15.159:0/475818609 >> v2:172.21.15.178:3300/0 conn(0x7ff0ac15c390 0x7ff0ac073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:03.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.559+0000 7ff0a1400640 1 -- 172.21.15.159:0/475818609 >> v2:172.21.15.178:3300/0 conn(0x7ff0ac15c390 msgr2=0x7ff0ac073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:03.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.559+0000 7ff0a1400640 1 --2- 172.21.15.159:0/475818609 >> v2:172.21.15.178:3300/0 conn(0x7ff0ac15c390 0x7ff0ac073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:03.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.559+0000 7ff0a1400640 1 -- 172.21.15.159:0/475818609 >> v1:172.21.15.159:6789/0 conn(0x7ff0ac160b80 legacy=0x7ff0ac0d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:03.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.559+0000 7ff0a1400640 1 -- 172.21.15.159:0/475818609 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ac0ad400 con 0x7ff0ac0af830 2024-04-04T06:06:03.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.559+0000 7ff0a9400640 1 -- 172.21.15.159:0/475818609 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3449512438 0 0) 0x7ff0ac15fe80 con 0x7ff0ac160b80 2024-04-04T06:06:03.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.560+0000 7ff0a1400640 1 --2- 172.21.15.159:0/475818609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0ac0af830 0x7ff0ac160640 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac0b63e0 tx=0x7ff09402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d681eb94807c68 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:03.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.561+0000 7ff0a9400640 1 -- 172.21.15.159:0/475818609 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094035020 con 0x7ff0ac0af830 2024-04-04T06:06:03.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.561+0000 7ff0a9400640 1 -- 172.21.15.159:0/475818609 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff094031830 con 0x7ff0ac0af830 2024-04-04T06:06:03.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.561+0000 7ff0a9400640 1 -- 172.21.15.159:0/475818609 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094031c00 con 0x7ff0ac0af830 2024-04-04T06:06:03.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.561+0000 7ff0b0a00640 1 -- 172.21.15.159:0/475818609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0ac0af830 msgr2=0x7ff0ac160640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:03.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.561+0000 7ff0b0a00640 1 --2- 172.21.15.159:0/475818609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0ac0af830 0x7ff0ac160640 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac0b63e0 tx=0x7ff09402b8d0 comp rx=0 tx=0).stop 2024-04-04T06:06:03.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.561+0000 7ff0b0a00640 1 -- 172.21.15.159:0/475818609 shutdown_connections 2024-04-04T06:06:03.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.561+0000 7ff0b0a00640 1 --2- 172.21.15.159:0/475818609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0ac0af830 0x7ff0ac160640 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:03.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.561+0000 7ff0b0a00640 1 --2- 172.21.15.159:0/475818609 >> v2:172.21.15.178:3300/0 conn(0x7ff0ac15c390 0x7ff0ac073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:03.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.561+0000 7ff0b0a00640 1 -- 172.21.15.159:0/475818609 >> 172.21.15.159:0/475818609 conn(0x7ff0ac06b990 msgr2=0x7ff0ac06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:03.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.561+0000 7ff0b0a00640 1 -- 172.21.15.159:0/475818609 shutdown_connections 2024-04-04T06:06:03.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.561+0000 7ff0b0a00640 1 -- 172.21.15.159:0/475818609 wait complete. 2024-04-04T06:06:03.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.563+0000 7ff0b0a00640 1 Processor -- start 2024-04-04T06:06:03.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.563+0000 7ff0b0a00640 1 -- start start 2024-04-04T06:06:03.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.564+0000 7ff0b0a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0ac0af830 0x7ff0ac0b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:03.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.564+0000 7ff0b0a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0ac15c390 0x7ff0ac0afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:03.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.564+0000 7ff0b0a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0ac160b80 0x7ff0ac15db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:03.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.564+0000 7ff0b0a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac0cf980 con 0x7ff0ac15c390 2024-04-04T06:06:03.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.564+0000 7ff0b0a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac160120 con 0x7ff0ac0af830 2024-04-04T06:06:03.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.564+0000 7ff0b0a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff0ac0d0d60 con 0x7ff0ac160b80 2024-04-04T06:06:03.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.564+0000 7ff0a1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0ac15c390 0x7ff0ac0afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:03.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.564+0000 7ff0a1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0ac15c390 0x7ff0ac0afd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43414/0 (socket says 172.21.15.159:43414) 2024-04-04T06:06:03.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.564+0000 7ff0a1400640 1 -- 172.21.15.159:0/3969896029 learned_addr learned my addr 172.21.15.159:0/3969896029 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:03.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.565+0000 7ff0a9e00640 1 --2- 172.21.15.159:0/3969896029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0ac0af830 0x7ff0ac0b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:03.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.565+0000 7ff0aa800640 1 --2- 172.21.15.159:0/3969896029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0ac160b80 0x7ff0ac15db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:03.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.565+0000 7ff0a1400640 1 -- 172.21.15.159:0/3969896029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0ac160b80 msgr2=0x7ff0ac15db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:03.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.565+0000 7ff0a1400640 1 --2- 172.21.15.159:0/3969896029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0ac160b80 0x7ff0ac15db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:03.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.565+0000 7ff0a1400640 1 -- 172.21.15.159:0/3969896029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0ac0af830 msgr2=0x7ff0ac0b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:03.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.565+0000 7ff0a1400640 1 --2- 172.21.15.159:0/3969896029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0ac0af830 0x7ff0ac0b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:03.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.565+0000 7ff0a1400640 1 -- 172.21.15.159:0/3969896029 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ac15e070 con 0x7ff0ac15c390 2024-04-04T06:06:03.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.565+0000 7ff0a9e00640 1 --2- 172.21.15.159:0/3969896029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0ac0af830 0x7ff0ac0b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:03.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.565+0000 7ff0a1400640 1 --2- 172.21.15.159:0/3969896029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0ac15c390 0x7ff0ac0afd80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff094009890 tx=0x7ff0940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:03.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.565+0000 7ff0a2a00640 1 -- 172.21.15.159:0/3969896029 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094041070 con 0x7ff0ac15c390 2024-04-04T06:06:03.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.565+0000 7ff0a2a00640 1 -- 172.21.15.159:0/3969896029 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff09403aa30 con 0x7ff0ac15c390 2024-04-04T06:06:03.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.565+0000 7ff0b0a00640 1 -- 172.21.15.159:0/3969896029 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ac0b0420 con 0x7ff0ac15c390 2024-04-04T06:06:03.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.565+0000 7ff0a2a00640 1 -- 172.21.15.159:0/3969896029 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094035040 con 0x7ff0ac15c390 2024-04-04T06:06:03.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.566+0000 7ff0b0a00640 1 -- 172.21.15.159:0/3969896029 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ac0b09e0 con 0x7ff0ac15c390 2024-04-04T06:06:03.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.567+0000 7ff0b0a00640 1 -- 172.21.15.159:0/3969896029 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0ac0685c0 con 0x7ff0ac15c390 2024-04-04T06:06:03.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.568+0000 7ff0a2a00640 1 -- 172.21.15.159:0/3969896029 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff094038020 con 0x7ff0ac15c390 2024-04-04T06:06:03.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.569+0000 7ff0a2a00640 1 --2- 172.21.15.159:0/3969896029 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff08806c8f0 0x7ff08806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:03.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.569+0000 7ff0a2a00640 1 -- 172.21.15.159:0/3969896029 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff0940ad550 con 0x7ff0ac15c390 2024-04-04T06:06:03.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.569+0000 7ff0a9e00640 1 --2- 172.21.15.159:0/3969896029 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff08806c8f0 0x7ff08806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:03.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.570+0000 7ff0a9e00640 1 --2- 172.21.15.159:0/3969896029 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff08806c8f0 0x7ff08806edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff09c006fd0 tx=0x7ff09c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:03.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.573+0000 7ff0a2a00640 1 -- 172.21.15.159:0/3969896029 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff09407b980 con 0x7ff0ac15c390 2024-04-04T06:06:03.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.827+0000 7ff0b0a00640 1 -- 172.21.15.159:0/3969896029 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7ff0ac06f1b0 con 0x7ff0ac15c390 2024-04-04T06:06:03.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.835+0000 7ff0a2a00640 1 -- 172.21.15.159:0/3969896029 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174143 (secure 0 0 0) 0x7ff09407b280 con 0x7ff0ac15c390 2024-04-04T06:06:03.832 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:03.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.839+0000 7ff0b0a00640 1 -- 172.21.15.159:0/3969896029 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff08806c8f0 msgr2=0x7ff08806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:03.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.839+0000 7ff0b0a00640 1 --2- 172.21.15.159:0/3969896029 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff08806c8f0 0x7ff08806edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff09c006fd0 tx=0x7ff09c008040 comp rx=0 tx=0).stop 2024-04-04T06:06:03.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.839+0000 7ff0b0a00640 1 -- 172.21.15.159:0/3969896029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0ac15c390 msgr2=0x7ff0ac0afd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:03.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.839+0000 7ff0b0a00640 1 --2- 172.21.15.159:0/3969896029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0ac15c390 0x7ff0ac0afd80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff094009890 tx=0x7ff0940046c0 comp rx=0 tx=0).stop 2024-04-04T06:06:03.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.839+0000 7ff0b0a00640 1 -- 172.21.15.159:0/3969896029 shutdown_connections 2024-04-04T06:06:03.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.839+0000 7ff0b0a00640 1 --2- 172.21.15.159:0/3969896029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0ac160b80 0x7ff0ac15db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:03.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.839+0000 7ff0b0a00640 1 --2- 172.21.15.159:0/3969896029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0ac15c390 0x7ff0ac0afd80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:03.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.839+0000 7ff0b0a00640 1 --2- 172.21.15.159:0/3969896029 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff08806c8f0 0x7ff08806edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:03.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.839+0000 7ff0b0a00640 1 --2- 172.21.15.159:0/3969896029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0ac0af830 0x7ff0ac0b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:03.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.840+0000 7ff0b0a00640 1 -- 172.21.15.159:0/3969896029 >> 172.21.15.159:0/3969896029 conn(0x7ff0ac06b990 msgr2=0x7ff0ac15af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:03.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.840+0000 7ff0b0a00640 1 -- 172.21.15.159:0/3969896029 shutdown_connections 2024-04-04T06:06:03.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:03.840+0000 7ff0b0a00640 1 -- 172.21.15.159:0/3969896029 wait complete. 2024-04-04T06:06:03.854 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3272374747},{"type":"v1","addr":"172.21.15.178:6801","nonce":3272374747}]},"active_addr":"172.21.15.178:6801/3272374747","active_change":"2024-04-04T06:05:32.692589+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.178:0","nonce":3723604909}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.178:0","nonce":3426184032}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.178:0","nonce":696432308}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.178:0","nonce":2797488603}]}]} 2024-04-04T06:06:03.858 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-04T06:06:03.858 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-04T06:06:03.858 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:06:04.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.074+0000 7f1bf5c00640 1 Processor -- start 2024-04-04T06:06:04.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.074+0000 7f1bf5c00640 1 -- start start 2024-04-04T06:06:04.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.076+0000 7f1bf5c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1bf015e4a0 0x7f1bf00b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:04.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.076+0000 7f1bf5c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1bf00d2310 0x7f1bf00ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:04.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.076+0000 7f1bf5c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bf015e880 con 0x7f1bf00cf530 2024-04-04T06:06:04.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.076+0000 7f1bf5c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f1bf00d34e0 con 0x7f1bf00d2310 2024-04-04T06:06:04.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.076+0000 7f1bf5c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1bf00d28d0 con 0x7f1bf015e4a0 2024-04-04T06:06:04.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.076+0000 7f1bf5200640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f1bf00cf530 0x7f1bf00d3c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45890/0 (socket says 172.21.15.159:45890) 2024-04-04T06:06:04.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.076+0000 7f1bf5200640 1 -- 172.21.15.159:0/4189198305 learned_addr learned my addr 172.21.15.159:0/4189198305 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:04.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.076+0000 7f1bef400640 1 --2- 172.21.15.159:0/4189198305 >> v2:172.21.15.178:3300/0 conn(0x7f1bf00d2310 0x7f1bf00ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:04.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.076+0000 7f1befe00640 1 --2- 172.21.15.159:0/4189198305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1bf015e4a0 0x7f1bf00b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:04.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.076+0000 7f1beea00640 1 -- 172.21.15.159:0/4189198305 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1702970854 0 0) 0x7f1bf015e880 con 0x7f1bf00cf530 2024-04-04T06:06:04.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.077+0000 7f1beea00640 1 -- 172.21.15.159:0/4189198305 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1bd4003520 con 0x7f1bf00cf530 2024-04-04T06:06:04.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.077+0000 7f1beea00640 1 -- 172.21.15.159:0/4189198305 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3082735142 0 0) 0x7f1bd4003520 con 0x7f1bf00cf530 2024-04-04T06:06:04.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.077+0000 7f1beea00640 1 -- 172.21.15.159:0/4189198305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1bf015e4a0 msgr2=0x7f1bf00b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:04.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.077+0000 7f1beea00640 1 --2- 172.21.15.159:0/4189198305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1bf015e4a0 0x7f1bf00b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:04.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.077+0000 7f1beea00640 1 -- 172.21.15.159:0/4189198305 >> v2:172.21.15.178:3300/0 conn(0x7f1bf00d2310 msgr2=0x7f1bf00ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:04.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.077+0000 7f1beea00640 1 --2- 172.21.15.159:0/4189198305 >> v2:172.21.15.178:3300/0 conn(0x7f1bf00d2310 0x7f1bf00ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:04.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.077+0000 7f1beea00640 1 -- 172.21.15.159:0/4189198305 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf00d2bb0 con 0x7f1bf00cf530 2024-04-04T06:06:04.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.077+0000 7f1beea00640 1 -- 172.21.15.159:0/4189198305 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f1be80019f0 con 0x7f1bf00cf530 2024-04-04T06:06:04.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.077+0000 7f1beea00640 1 -- 172.21.15.159:0/4189198305 >> v1:172.21.15.159:6789/0 conn(0x7f1bf00cf530 legacy=0x7f1bf00d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:04.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.077+0000 7f1beea00640 1 --2- 172.21.15.159:0/4189198305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bd40039b0 0x7f1bd4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:04.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.077+0000 7f1beea00640 1 --2- 172.21.15.159:0/4189198305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1bd4004800 0x7f1bd4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:04.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.077+0000 7f1beea00640 1 --2- 172.21.15.159:0/4189198305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1bf015e4a0 0x7f1bd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:04.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.077+0000 7f1beea00640 1 -- 172.21.15.159:0/4189198305 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf00d34e0 con 0x7f1bd4004800 2024-04-04T06:06:04.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.077+0000 7f1beea00640 1 -- 172.21.15.159:0/4189198305 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf00d28d0 con 0x7f1bd40039b0 2024-04-04T06:06:04.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.077+0000 7f1beea00640 1 -- 172.21.15.159:0/4189198305 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd400b9c0 con 0x7f1bf015e4a0 2024-04-04T06:06:04.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.077+0000 7f1bef400640 1 --2- 172.21.15.159:0/4189198305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1bd4004800 0x7f1bd4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:04.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.077+0000 7f1befe00640 1 --2- 172.21.15.159:0/4189198305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bd40039b0 0x7f1bd4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:04.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.078+0000 7f1bef400640 1 -- 172.21.15.159:0/4189198305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1bf015e4a0 msgr2=0x7f1bd40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:04.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.078+0000 7f1bef400640 1 --2- 172.21.15.159:0/4189198305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1bf015e4a0 0x7f1bd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:04.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.078+0000 7f1bef400640 1 -- 172.21.15.159:0/4189198305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bd40039b0 msgr2=0x7f1bd4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:04.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.078+0000 7f1bef400640 1 --2- 172.21.15.159:0/4189198305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bd40039b0 0x7f1bd4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:04.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.078+0000 7f1bef400640 1 -- 172.21.15.159:0/4189198305 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd4003520 con 0x7f1bd4004800 2024-04-04T06:06:04.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.079+0000 7f1bef400640 1 --2- 172.21.15.159:0/4189198305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1bd4004800 0x7f1bd4004c80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc009ac0 tx=0x7f1bd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35e2cc8ea2746d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:04.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.079+0000 7f1beea00640 1 -- 172.21.15.159:0/4189198305 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc039070 con 0x7f1bd4004800 2024-04-04T06:06:04.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.079+0000 7f1beea00640 1 -- 172.21.15.159:0/4189198305 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bdc02c710 con 0x7f1bd4004800 2024-04-04T06:06:04.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.079+0000 7f1beea00640 1 -- 172.21.15.159:0/4189198305 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc009690 con 0x7f1bd4004800 2024-04-04T06:06:04.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.079+0000 7f1bf5c00640 1 -- 172.21.15.159:0/4189198305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1bd4004800 msgr2=0x7f1bd4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:04.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.079+0000 7f1bf5c00640 1 --2- 172.21.15.159:0/4189198305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1bd4004800 0x7f1bd4004c80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc009ac0 tx=0x7f1bd4005830 comp rx=0 tx=0).stop 2024-04-04T06:06:04.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.079+0000 7f1bf5200640 1 -- 172.21.15.159:0/4189198305 reap_dead start 2024-04-04T06:06:04.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.080+0000 7f1bf5c00640 1 -- 172.21.15.159:0/4189198305 shutdown_connections 2024-04-04T06:06:04.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.080+0000 7f1bf5c00640 1 -- 172.21.15.159:0/4189198305 >> 172.21.15.159:0/4189198305 conn(0x7f1bf00676d0 msgr2=0x7f1bf0067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:04.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.080+0000 7f1bf5c00640 1 -- 172.21.15.159:0/4189198305 shutdown_connections 2024-04-04T06:06:04.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.080+0000 7f1bf5c00640 1 -- 172.21.15.159:0/4189198305 wait complete. 2024-04-04T06:06:04.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.081+0000 7f1bf5c00640 1 Processor -- start 2024-04-04T06:06:04.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.082+0000 7f1bf5c00640 1 -- start start 2024-04-04T06:06:04.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.082+0000 7f1bf5c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bd4004800 0x7f1bf0160320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:04.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.082+0000 7f1bf5c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1bf0153720 0x7f1bf01f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:04.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.082+0000 7f1bf5c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1bf01f7ce0 0x7f1bf01f8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:04.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.082+0000 7f1bf5c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf00d3970 con 0x7f1bf0153720 2024-04-04T06:06:04.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.082+0000 7f1bf5c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf00d3170 con 0x7f1bd4004800 2024-04-04T06:06:04.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.082+0000 7f1bf5c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1bf00b83f0 con 0x7f1bf01f7ce0 2024-04-04T06:06:04.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.083+0000 7f1bef400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1bf0153720 0x7f1bf01f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:04.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.083+0000 7f1bef400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1bf0153720 0x7f1bf01f77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36856/0 (socket says 172.21.15.159:36856) 2024-04-04T06:06:04.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.083+0000 7f1bef400640 1 -- 172.21.15.159:0/2074449419 learned_addr learned my addr 172.21.15.159:0/2074449419 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:04.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.083+0000 7f1befe00640 1 --2- 172.21.15.159:0/2074449419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bd4004800 0x7f1bf0160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:04.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.083+0000 7f1bf5200640 1 --2- 172.21.15.159:0/2074449419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1bf01f7ce0 0x7f1bf01f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:04.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.083+0000 7f1bef400640 1 -- 172.21.15.159:0/2074449419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1bf01f7ce0 msgr2=0x7f1bf01f8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:04.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.083+0000 7f1bef400640 1 --2- 172.21.15.159:0/2074449419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1bf01f7ce0 0x7f1bf01f8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:04.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.083+0000 7f1bef400640 1 -- 172.21.15.159:0/2074449419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bd4004800 msgr2=0x7f1bf0160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:04.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.083+0000 7f1bef400640 1 --2- 172.21.15.159:0/2074449419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bd4004800 0x7f1bf0160320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:04.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.083+0000 7f1bef400640 1 -- 172.21.15.159:0/2074449419 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf0152030 con 0x7f1bf0153720 2024-04-04T06:06:04.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.083+0000 7f1bf5200640 1 --2- 172.21.15.159:0/2074449419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1bf01f7ce0 0x7f1bf01f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:04.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.083+0000 7f1befe00640 1 --2- 172.21.15.159:0/2074449419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bd4004800 0x7f1bf0160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:04.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.084+0000 7f1bef400640 1 --2- 172.21.15.159:0/2074449419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1bf0153720 0x7f1bf01f77a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc000c00 tx=0x7f1bdc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:04.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.084+0000 7f1be7e00640 1 -- 172.21.15.159:0/2074449419 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc039050 con 0x7f1bf0153720 2024-04-04T06:06:04.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.084+0000 7f1be7e00640 1 -- 172.21.15.159:0/2074449419 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bdc045070 con 0x7f1bf0153720 2024-04-04T06:06:04.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.084+0000 7f1be7e00640 1 -- 172.21.15.159:0/2074449419 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc0044e0 con 0x7f1bf0153720 2024-04-04T06:06:04.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.084+0000 7f1bf5c00640 1 -- 172.21.15.159:0/2074449419 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf0153d70 con 0x7f1bf0153720 2024-04-04T06:06:04.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.084+0000 7f1bf5c00640 1 -- 172.21.15.159:0/2074449419 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf0154330 con 0x7f1bf0153720 2024-04-04T06:06:04.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.085+0000 7f1bf5c00640 1 -- 172.21.15.159:0/2074449419 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bf0075940 con 0x7f1bf0153720 2024-04-04T06:06:04.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.086+0000 7f1be7e00640 1 -- 172.21.15.159:0/2074449419 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1bdc003a50 con 0x7f1bf0153720 2024-04-04T06:06:04.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.087+0000 7f1be7e00640 1 --2- 172.21.15.159:0/2074449419 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1bd006c920 0x7f1bd006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:04.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.087+0000 7f1be7e00640 1 -- 172.21.15.159:0/2074449419 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1bdc0ad6e0 con 0x7f1bf0153720 2024-04-04T06:06:04.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.088+0000 7f1befe00640 1 --2- 172.21.15.159:0/2074449419 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1bd006c920 0x7f1bd006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:04.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.089+0000 7f1befe00640 1 --2- 172.21.15.159:0/2074449419 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1bd006c920 0x7f1bd006ede0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1be0006ce0 tx=0x7f1be0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:04.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.092+0000 7f1be7e00640 1 -- 172.21.15.159:0/2074449419 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1bdc04e020 con 0x7f1bf0153720 2024-04-04T06:06:04.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.293+0000 7f1bf5c00640 1 -- 172.21.15.159:0/2074449419 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1bf006cee0 con 0x7f1bf0153720 2024-04-04T06:06:04.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.296+0000 7f1be7e00640 1 -- 172.21.15.159:0/2074449419 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f1bdc07bb00 con 0x7f1bf0153720 2024-04-04T06:06:04.291 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:04.291 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":21,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:05:54.347275+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:05:51.297293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-04T06:05:51.770792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:06:04.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.299+0000 7f1bf5c00640 1 -- 172.21.15.159:0/2074449419 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1bd006c920 msgr2=0x7f1bd006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:04.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.299+0000 7f1bf5c00640 1 --2- 172.21.15.159:0/2074449419 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1bd006c920 0x7f1bd006ede0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1be0006ce0 tx=0x7f1be0008040 comp rx=0 tx=0).stop 2024-04-04T06:06:04.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.299+0000 7f1bf5c00640 1 -- 172.21.15.159:0/2074449419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1bf0153720 msgr2=0x7f1bf01f77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:04.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.299+0000 7f1bf5c00640 1 --2- 172.21.15.159:0/2074449419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1bf0153720 0x7f1bf01f77a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc000c00 tx=0x7f1bdc004730 comp rx=0 tx=0).stop 2024-04-04T06:06:04.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.299+0000 7f1bf5c00640 1 -- 172.21.15.159:0/2074449419 shutdown_connections 2024-04-04T06:06:04.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.299+0000 7f1bf5c00640 1 --2- 172.21.15.159:0/2074449419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1bf01f7ce0 0x7f1bf01f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:04.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.299+0000 7f1bf5c00640 1 --2- 172.21.15.159:0/2074449419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1bf0153720 0x7f1bf01f77a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:04.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.299+0000 7f1bf5c00640 1 --2- 172.21.15.159:0/2074449419 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1bd006c920 0x7f1bd006ede0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:04.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.299+0000 7f1bf5c00640 1 --2- 172.21.15.159:0/2074449419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1bd4004800 0x7f1bf0160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:04.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.299+0000 7f1bf5c00640 1 -- 172.21.15.159:0/2074449419 >> 172.21.15.159:0/2074449419 conn(0x7f1bf00676d0 msgr2=0x7f1bf00d4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:04.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.299+0000 7f1bf5c00640 1 -- 172.21.15.159:0/2074449419 shutdown_connections 2024-04-04T06:06:04.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.299+0000 7f1bf5c00640 1 -- 172.21.15.159:0/2074449419 wait complete. 2024-04-04T06:06:04.314 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-04T06:06:04.314 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:06:04.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.527+0000 7fe666200640 1 Processor -- start 2024-04-04T06:06:04.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.527+0000 7fe666200640 1 -- start start 2024-04-04T06:06:04.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.529+0000 7fe666200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe66015e4a0 0x7fe6600b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:04.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.529+0000 7fe666200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6600cf700 0x7fe6600b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:04.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.529+0000 7fe666200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe66015e880 con 0x7fe6600d2310 2024-04-04T06:06:04.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.529+0000 7fe666200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe6600d34e0 con 0x7fe66015e4a0 2024-04-04T06:06:04.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.529+0000 7fe666200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe6600d28d0 con 0x7fe6600cf700 2024-04-04T06:06:04.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.529+0000 7fe65fe00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fe6600d2310 0x7fe6600ceff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45902/0 (socket says 172.21.15.159:45902) 2024-04-04T06:06:04.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.529+0000 7fe65fe00640 1 -- 172.21.15.159:0/3599342168 learned_addr learned my addr 172.21.15.159:0/3599342168 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:04.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.530+0000 7fe665800640 1 --2- 172.21.15.159:0/3599342168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6600cf700 0x7fe6600b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:04.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.530+0000 7fe664e00640 1 --2- 172.21.15.159:0/3599342168 >> v2:172.21.15.178:3300/0 conn(0x7fe66015e4a0 0x7fe6600b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:04.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.530+0000 7fe65f400640 1 -- 172.21.15.159:0/3599342168 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3657599932 0 0) 0x7fe66015e880 con 0x7fe6600d2310 2024-04-04T06:06:04.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.530+0000 7fe65f400640 1 -- 172.21.15.159:0/3599342168 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe640003520 con 0x7fe6600d2310 2024-04-04T06:06:04.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.530+0000 7fe65f400640 1 -- 172.21.15.159:0/3599342168 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1469636771 0 0) 0x7fe640003520 con 0x7fe6600d2310 2024-04-04T06:06:04.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.530+0000 7fe65f400640 1 -- 172.21.15.159:0/3599342168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6600cf700 msgr2=0x7fe6600b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:04.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.530+0000 7fe65f400640 1 --2- 172.21.15.159:0/3599342168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6600cf700 0x7fe6600b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:04.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.530+0000 7fe65f400640 1 -- 172.21.15.159:0/3599342168 >> v2:172.21.15.178:3300/0 conn(0x7fe66015e4a0 msgr2=0x7fe6600b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:04.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.530+0000 7fe65f400640 1 --2- 172.21.15.159:0/3599342168 >> v2:172.21.15.178:3300/0 conn(0x7fe66015e4a0 0x7fe6600b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:04.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.530+0000 7fe65f400640 1 -- 172.21.15.159:0/3599342168 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6600d2bb0 con 0x7fe6600d2310 2024-04-04T06:06:04.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.530+0000 7fe65f400640 1 -- 172.21.15.159:0/3599342168 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fe6480019f0 con 0x7fe6600d2310 2024-04-04T06:06:04.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.530+0000 7fe65f400640 1 -- 172.21.15.159:0/3599342168 >> v1:172.21.15.159:6789/0 conn(0x7fe6600d2310 legacy=0x7fe6600ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:04.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.530+0000 7fe65f400640 1 --2- 172.21.15.159:0/3599342168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6600cf700 0x7fe6400039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:04.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.530+0000 7fe65f400640 1 --2- 172.21.15.159:0/3599342168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe640003fa0 0x7fe640004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:04.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.530+0000 7fe65f400640 1 --2- 172.21.15.159:0/3599342168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe640004d40 0x7fe6400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:04.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.530+0000 7fe65f400640 1 -- 172.21.15.159:0/3599342168 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe6600d34e0 con 0x7fe640003fa0 2024-04-04T06:06:04.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.530+0000 7fe65f400640 1 -- 172.21.15.159:0/3599342168 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6600d28d0 con 0x7fe640004d40 2024-04-04T06:06:04.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.530+0000 7fe65f400640 1 -- 172.21.15.159:0/3599342168 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe64000ba70 con 0x7fe6600cf700 2024-04-04T06:06:04.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.531+0000 7fe65fe00640 1 --2- 172.21.15.159:0/3599342168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe640003fa0 0x7fe640004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:04.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.531+0000 7fe664e00640 1 --2- 172.21.15.159:0/3599342168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6600cf700 0x7fe6400039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:04.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.531+0000 7fe65fe00640 1 -- 172.21.15.159:0/3599342168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6600cf700 msgr2=0x7fe6400039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:04.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.531+0000 7fe65fe00640 1 --2- 172.21.15.159:0/3599342168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6600cf700 0x7fe6400039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:04.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.531+0000 7fe65fe00640 1 -- 172.21.15.159:0/3599342168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe640004d40 msgr2=0x7fe6400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:04.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.531+0000 7fe65fe00640 1 --2- 172.21.15.159:0/3599342168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe640004d40 0x7fe6400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:04.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.531+0000 7fe65fe00640 1 -- 172.21.15.159:0/3599342168 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe640003520 con 0x7fe640003fa0 2024-04-04T06:06:04.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.531+0000 7fe664e00640 1 --2- 172.21.15.159:0/3599342168 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6600cf700 0x7fe6400039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:04.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.532+0000 7fe65fe00640 1 --2- 172.21.15.159:0/3599342168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe640003fa0 0x7fe640004800 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe64800a7b0 tx=0x7fe640005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2aae5c258fc5a4e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:04.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.532+0000 7fe65f400640 1 -- 172.21.15.159:0/3599342168 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64800a3d0 con 0x7fe640003fa0 2024-04-04T06:06:04.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.532+0000 7fe65f400640 1 -- 172.21.15.159:0/3599342168 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe648034650 con 0x7fe640003fa0 2024-04-04T06:06:04.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.532+0000 7fe65f400640 1 -- 172.21.15.159:0/3599342168 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe648034a80 con 0x7fe640003fa0 2024-04-04T06:06:04.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.533+0000 7fe666200640 1 -- 172.21.15.159:0/3599342168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe640003fa0 msgr2=0x7fe640004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:04.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.533+0000 7fe666200640 1 --2- 172.21.15.159:0/3599342168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe640003fa0 0x7fe640004800 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe64800a7b0 tx=0x7fe640005830 comp rx=0 tx=0).stop 2024-04-04T06:06:04.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.533+0000 7fe665800640 1 -- 172.21.15.159:0/3599342168 reap_dead start 2024-04-04T06:06:04.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.533+0000 7fe666200640 1 -- 172.21.15.159:0/3599342168 shutdown_connections 2024-04-04T06:06:04.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.533+0000 7fe666200640 1 -- 172.21.15.159:0/3599342168 >> 172.21.15.159:0/3599342168 conn(0x7fe6600676d0 msgr2=0x7fe660067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:04.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.533+0000 7fe666200640 1 -- 172.21.15.159:0/3599342168 shutdown_connections 2024-04-04T06:06:04.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.533+0000 7fe666200640 1 -- 172.21.15.159:0/3599342168 wait complete. 2024-04-04T06:06:04.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.535+0000 7fe666200640 1 Processor -- start 2024-04-04T06:06:04.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.535+0000 7fe666200640 1 -- start start 2024-04-04T06:06:04.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.536+0000 7fe666200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe640003fa0 0x7fe6601f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:04.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.536+0000 7fe666200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6600c7dc0 0x7fe6601542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:04.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.536+0000 7fe666200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6600c93d0 0x7fe6600c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:04.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.536+0000 7fe666200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe6600d3970 con 0x7fe640003fa0 2024-04-04T06:06:04.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.536+0000 7fe666200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6600d3170 con 0x7fe6600c7dc0 2024-04-04T06:06:04.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.536+0000 7fe666200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe6600ba1a0 con 0x7fe6600c93d0 2024-04-04T06:06:04.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.536+0000 7fe664e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe640003fa0 0x7fe6601f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:04.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.536+0000 7fe664e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe640003fa0 0x7fe6601f3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36876/0 (socket says 172.21.15.159:36876) 2024-04-04T06:06:04.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.536+0000 7fe664e00640 1 -- 172.21.15.159:0/3913170643 learned_addr learned my addr 172.21.15.159:0/3913170643 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:04.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.537+0000 7fe65fe00640 1 --2- 172.21.15.159:0/3913170643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6600c7dc0 0x7fe6601542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:04.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.537+0000 7fe665800640 1 --2- 172.21.15.159:0/3913170643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6600c93d0 0x7fe6600c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:04.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.537+0000 7fe664e00640 1 -- 172.21.15.159:0/3913170643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6600c93d0 msgr2=0x7fe6600c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:04.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.537+0000 7fe664e00640 1 --2- 172.21.15.159:0/3913170643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6600c93d0 0x7fe6600c9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:04.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.537+0000 7fe664e00640 1 -- 172.21.15.159:0/3913170643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6600c7dc0 msgr2=0x7fe6601542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:04.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.537+0000 7fe664e00640 1 --2- 172.21.15.159:0/3913170643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6600c7dc0 0x7fe6601542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:04.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.537+0000 7fe664e00640 1 -- 172.21.15.159:0/3913170643 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6600c8330 con 0x7fe640003fa0 2024-04-04T06:06:04.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.537+0000 7fe664e00640 1 --2- 172.21.15.159:0/3913170643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe640003fa0 0x7fe6601f3f60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe650010a00 tx=0x7fe650010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:04.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.537+0000 7fe65cc00640 1 -- 172.21.15.159:0/3913170643 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65000f040 con 0x7fe640003fa0 2024-04-04T06:06:04.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.537+0000 7fe65cc00640 1 -- 172.21.15.159:0/3913170643 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe65000ebc0 con 0x7fe640003fa0 2024-04-04T06:06:04.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.537+0000 7fe666200640 1 -- 172.21.15.159:0/3913170643 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6600c8610 con 0x7fe640003fa0 2024-04-04T06:06:04.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.537+0000 7fe65cc00640 1 -- 172.21.15.159:0/3913170643 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65001c400 con 0x7fe640003fa0 2024-04-04T06:06:04.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.537+0000 7fe666200640 1 -- 172.21.15.159:0/3913170643 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6600c8bd0 con 0x7fe640003fa0 2024-04-04T06:06:04.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.538+0000 7fe666200640 1 -- 172.21.15.159:0/3913170643 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe660075940 con 0x7fe640003fa0 2024-04-04T06:06:04.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.539+0000 7fe65cc00640 1 -- 172.21.15.159:0/3913170643 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe65001c5a0 con 0x7fe640003fa0 2024-04-04T06:06:04.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.540+0000 7fe65cc00640 1 --2- 172.21.15.159:0/3913170643 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe63c06c920 0x7fe63c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:04.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.540+0000 7fe65cc00640 1 -- 172.21.15.159:0/3913170643 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe65008c210 con 0x7fe640003fa0 2024-04-04T06:06:04.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.541+0000 7fe65fe00640 1 --2- 172.21.15.159:0/3913170643 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe63c06c920 0x7fe63c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:04.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.542+0000 7fe65fe00640 1 --2- 172.21.15.159:0/3913170643 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe63c06c920 0x7fe63c06ede0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe648034000 tx=0x7fe648009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:04.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.545+0000 7fe65cc00640 1 -- 172.21.15.159:0/3913170643 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe650018030 con 0x7fe640003fa0 2024-04-04T06:06:04.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.749+0000 7fe666200640 1 -- 172.21.15.159:0/3913170643 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6600b4010 con 0x7fe640003fa0 2024-04-04T06:06:04.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.751+0000 7fe65cc00640 1 -- 172.21.15.159:0/3913170643 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7fe65005a640 con 0x7fe640003fa0 2024-04-04T06:06:04.746 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:04.746 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":21,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:05:54.347275+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:05:51.297293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-04T06:05:51.770792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:06:04.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.754+0000 7fe666200640 1 -- 172.21.15.159:0/3913170643 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe63c06c920 msgr2=0x7fe63c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:04.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.754+0000 7fe666200640 1 --2- 172.21.15.159:0/3913170643 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe63c06c920 0x7fe63c06ede0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe648034000 tx=0x7fe648009030 comp rx=0 tx=0).stop 2024-04-04T06:06:04.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.754+0000 7fe666200640 1 -- 172.21.15.159:0/3913170643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe640003fa0 msgr2=0x7fe6601f3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:04.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.754+0000 7fe666200640 1 --2- 172.21.15.159:0/3913170643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe640003fa0 0x7fe6601f3f60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe650010a00 tx=0x7fe650010ec0 comp rx=0 tx=0).stop 2024-04-04T06:06:04.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.754+0000 7fe666200640 1 -- 172.21.15.159:0/3913170643 shutdown_connections 2024-04-04T06:06:04.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.754+0000 7fe666200640 1 --2- 172.21.15.159:0/3913170643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6600c93d0 0x7fe6600c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:04.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.754+0000 7fe666200640 1 --2- 172.21.15.159:0/3913170643 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe63c06c920 0x7fe63c06ede0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:04.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.754+0000 7fe666200640 1 --2- 172.21.15.159:0/3913170643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6600c7dc0 0x7fe6601542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:04.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.754+0000 7fe666200640 1 --2- 172.21.15.159:0/3913170643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe640003fa0 0x7fe6601f3f60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:04.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.754+0000 7fe666200640 1 -- 172.21.15.159:0/3913170643 >> 172.21.15.159:0/3913170643 conn(0x7fe6600676d0 msgr2=0x7fe66015b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:04.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.754+0000 7fe666200640 1 -- 172.21.15.159:0/3913170643 shutdown_connections 2024-04-04T06:06:04.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:04.754+0000 7fe666200640 1 -- 172.21.15.159:0/3913170643 wait complete. 2024-04-04T06:06:04.767 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-04T06:06:04.768 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-04T06:06:04.768 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-04T06:06:04.768 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-04T06:06:04.769 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-04T06:06:04.769 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-04T06:06:04.769 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-04T06:06:04.770 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-04T06:06:05.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.081+0000 7f0eaa800640 1 Processor -- start 2024-04-04T06:06:05.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.082+0000 7f0eaa800640 1 -- start start 2024-04-04T06:06:05.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.082+0000 7f6c6e200640 1 Processor -- start 2024-04-04T06:06:05.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.083+0000 7f7092600640 1 Processor -- start 2024-04-04T06:06:05.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.083+0000 7f7092600640 1 -- start start 2024-04-04T06:06:05.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.083+0000 7f0eaa800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e9c0df420 0x7f0e9c04fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.084+0000 7f0eaa800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0e9c04ffd0 0x7f0e9c054270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.084+0000 7f0eaa800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e9c006af0 con 0x7f0e9c00aaa0 2024-04-04T06:06:05.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.084+0000 7f0eaa800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0e9c0068a0 con 0x7f0e9c04ffd0 2024-04-04T06:06:05.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.084+0000 7f0eaa800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0e9c006dd0 con 0x7f0e9c0df420 2024-04-04T06:06:05.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.084+0000 7f0ea9400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e9c0df420 0x7f0e9c04fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.084+0000 7f0ea9400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e9c0df420 0x7f0e9c04fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:54910/0 (socket says 172.21.15.159:54910) 2024-04-04T06:06:05.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.084+0000 7f0ea9400640 1 -- 172.21.15.159:0/835883188 learned_addr learned my addr 172.21.15.159:0/835883188 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.085+0000 7f0ea9e00640 1 --2- 172.21.15.159:0/835883188 >> v2:172.21.15.178:3300/0 conn(0x7f0e9c04ffd0 0x7f0e9c054270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.083+0000 7f6c6e200640 1 -- start start 2024-04-04T06:06:05.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.085+0000 7f7092600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f70840ead00 0x7f708404fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.086+0000 7f7092600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f708404ffd0 0x7f7084037bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.086+0000 7f7092600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7084038110 0x7f70840385a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.086+0000 7f7092600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f7084054330 con 0x7f70840ead00 2024-04-04T06:06:05.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.086+0000 7f7092600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7084053720 con 0x7f7084038110 2024-04-04T06:06:05.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.086+0000 7f7092600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7084053a00 con 0x7f708404ffd0 2024-04-04T06:06:05.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.085+0000 7f6c6e200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6c6806cc90 0x7f6c68163cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.085+0000 7f6c6e200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c6806fd30 con 0x7f6c6806f950 2024-04-04T06:06:05.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.085+0000 7f6c6e200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c680b3b30 con 0x7f6c680d0390 2024-04-04T06:06:05.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.085+0000 7f6c6e200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f6c680b48b0 con 0x7f6c6806cc90 2024-04-04T06:06:05.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.086+0000 7f3193200640 1 Processor -- start 2024-04-04T06:06:05.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.086+0000 7f6c5fe00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6c6806cc90 0x7f6c68163cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.087+0000 7f6c5fe00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6c6806cc90 0x7f6c68163cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:40894/0 (socket says 172.21.15.159:40894) 2024-04-04T06:06:05.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.087+0000 7f6c5fe00640 1 -- 172.21.15.159:0/1253946139 learned_addr learned my addr 172.21.15.159:0/1253946139 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.086+0000 7f3193200640 1 -- start start 2024-04-04T06:06:05.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.087+0000 7f7091200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f70840ead00 0x7f708404fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.087+0000 7f7091200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f70840ead00 0x7f708404fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36880/0 (socket says 172.21.15.159:36880) 2024-04-04T06:06:05.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.086+0000 7f0ea9400640 1 -- 172.21.15.159:0/835883188 >> v2:172.21.15.178:3300/0 conn(0x7f0e9c04ffd0 msgr2=0x7f0e9c054270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.086+0000 7f0ea9400640 1 --2- 172.21.15.159:0/835883188 >> v2:172.21.15.178:3300/0 conn(0x7f0e9c04ffd0 0x7f0e9c054270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.086+0000 7f0ea9400640 1 -- 172.21.15.159:0/835883188 >> v1:172.21.15.159:6789/0 conn(0x7f0e9c00aaa0 legacy=0x7f0e9c0382c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.086+0000 7f0ea9400640 1 -- 172.21.15.159:0/835883188 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9c0070b0 con 0x7f0e9c0df420 2024-04-04T06:06:05.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.087+0000 7f7091200640 1 -- 172.21.15.159:0/2715001238 learned_addr learned my addr 172.21.15.159:0/2715001238 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.087+0000 7f708be00640 1 --2- 172.21.15.159:0/2715001238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f708404ffd0 0x7f7084037bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.088+0000 7f7091c00640 1 --2- 172.21.15.159:0/2715001238 >> v2:172.21.15.178:3300/0 conn(0x7f7084038110 0x7f70840385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.087+0000 7f6c66a00640 1 -- 172.21.15.159:0/1253946139 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2613442793 0 0) 0x7f6c680b3b30 con 0x7f6c680d0390 2024-04-04T06:06:05.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.088+0000 7f6c66a00640 1 -- 172.21.15.159:0/1253946139 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c4c003520 con 0x7f6c680d0390 2024-04-04T06:06:05.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.088+0000 7f6c5fe00640 1 -- 172.21.15.159:0/1253946139 >> v1:172.21.15.178:6789/0 conn(0x7f6c680d0390 legacy=0x7f6c681593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.088+0000 7f7507e00640 1 Processor -- start 2024-04-04T06:06:05.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.088+0000 7f0ea9400640 1 --2- 172.21.15.159:0/835883188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e9c0df420 0x7f0e9c04fa90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40511b0 tx=0x7f0ea40ab4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29c902f970a136c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.088+0000 7f3193200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f318c054200 0x7f318c050740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.088+0000 7f3193200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f318c006720 0x7f318c006bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.088+0000 7f3193200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f318c0526a0 con 0x7f318c0df800 2024-04-04T06:06:05.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.088+0000 7f3193200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f318c0dfbe0 con 0x7f318c006720 2024-04-04T06:06:05.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.088+0000 7f3193200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f318c053850 con 0x7f318c054200 2024-04-04T06:06:05.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.089+0000 7fc7a6a00640 1 Processor -- start 2024-04-04T06:06:05.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.089+0000 7f0ea3e00640 1 -- 172.21.15.159:0/835883188 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40b7070 con 0x7f0e9c0df420 2024-04-04T06:06:05.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.089+0000 7f0ea3e00640 1 -- 172.21.15.159:0/835883188 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ea40b0810 con 0x7f0e9c0df420 2024-04-04T06:06:05.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.089+0000 7f0ea3e00640 1 -- 172.21.15.159:0/835883188 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40b0c80 con 0x7f0e9c0df420 2024-04-04T06:06:05.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.089+0000 7f3192800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f318c006720 0x7f318c006bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.089+0000 7f3192800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f318c006720 0x7f318c006bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36896/0 (socket says 172.21.15.159:36896) 2024-04-04T06:06:05.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.088+0000 7f6c5fe00640 1 -- 172.21.15.159:0/1253946139 >> v1:172.21.15.159:6789/0 conn(0x7f6c6806f950 legacy=0x7f6c6806c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.089+0000 7f6c5fe00640 1 -- 172.21.15.159:0/1253946139 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c680b3e10 con 0x7f6c6806cc90 2024-04-04T06:06:05.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.089+0000 7f3192800640 1 -- 172.21.15.159:0/2599280194 learned_addr learned my addr 172.21.15.159:0/2599280194 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.089+0000 7f3191400640 1 --2- 172.21.15.159:0/2599280194 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f318c054200 0x7f318c050740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.089+0000 7fc7a6a00640 1 -- start start 2024-04-04T06:06:05.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.090+0000 7f3191400640 1 -- 172.21.15.159:0/2599280194 >> v2:172.21.15.159:3300/0 conn(0x7f318c006720 msgr2=0x7f318c006bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.090+0000 7f7507e00640 1 -- start start 2024-04-04T06:06:05.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.090+0000 7f0eaa800640 1 -- 172.21.15.159:0/835883188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e9c0df420 msgr2=0x7f0e9c04fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.090+0000 7f0eaa800640 1 --2- 172.21.15.159:0/835883188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e9c0df420 0x7f0e9c04fa90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40511b0 tx=0x7f0ea40ab4f0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.090+0000 7f3191400640 1 --2- 172.21.15.159:0/2599280194 >> v2:172.21.15.159:3300/0 conn(0x7f318c006720 0x7f318c006bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.090+0000 7f3190a00640 1 -- 172.21.15.159:0/2599280194 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2102061338 0 0) 0x7f318c0526a0 con 0x7f318c0df800 2024-04-04T06:06:05.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.090+0000 7f3191400640 1 -- 172.21.15.159:0/2599280194 >> v1:172.21.15.178:6789/0 conn(0x7f318c0df800 legacy=0x7f318c053140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.090+0000 7f3191400640 1 -- 172.21.15.159:0/2599280194 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318c053c80 con 0x7f318c054200 2024-04-04T06:06:05.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.090+0000 7f3192800640 1 --2- 172.21.15.159:0/2599280194 >> v2:172.21.15.159:3300/0 conn(0x7f318c006720 0x7f318c006bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:06:05.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.088+0000 7f7091200640 1 -- 172.21.15.159:0/2715001238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f708404ffd0 msgr2=0x7f7084037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.088+0000 7f7091200640 1 --2- 172.21.15.159:0/2715001238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f708404ffd0 0x7f7084037bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.088+0000 7f7091200640 1 -- 172.21.15.159:0/2715001238 >> v2:172.21.15.178:3300/0 conn(0x7f7084038110 msgr2=0x7f70840385a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.088+0000 7f7091200640 1 --2- 172.21.15.159:0/2715001238 >> v2:172.21.15.178:3300/0 conn(0x7f7084038110 0x7f70840385a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.088+0000 7f7091200640 1 -- 172.21.15.159:0/2715001238 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7084053ce0 con 0x7f70840ead00 2024-04-04T06:06:05.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.090+0000 7f6c5fe00640 1 --2- 172.21.15.159:0/1253946139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c6806cc90 0x7f6c68163cf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009870 tx=0x7f6c5002b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e48aef4ea8a214c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.090+0000 7f0eaa800640 1 -- 172.21.15.159:0/835883188 shutdown_connections 2024-04-04T06:06:05.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.090+0000 7f0eaa800640 1 --2- 172.21.15.159:0/835883188 >> v2:172.21.15.178:3300/0 conn(0x7f0e9c04ffd0 0x7f0e9c054270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.090+0000 7f0eaa800640 1 --2- 172.21.15.159:0/835883188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e9c0df420 0x7f0e9c04fa90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.090+0000 7f0eaa800640 1 -- 172.21.15.159:0/835883188 >> 172.21.15.159:0/835883188 conn(0x7f0e9c009fb0 msgr2=0x7f0e9c00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.091+0000 7fded3200640 1 Processor -- start 2024-04-04T06:06:05.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.091+0000 7f0eaa800640 1 -- 172.21.15.159:0/835883188 shutdown_connections 2024-04-04T06:06:05.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.091+0000 7f0eaa800640 1 -- 172.21.15.159:0/835883188 wait complete. 2024-04-04T06:06:05.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.091+0000 7fded3200640 1 -- start start 2024-04-04T06:06:05.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.091+0000 7f7091200640 1 --2- 172.21.15.159:0/2715001238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70840ead00 0x7f708404fa90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f708400bd00 tx=0x7f708002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0126395ae63e83 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.091+0000 7fc7a6a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc7980e3260 0x7fc79804f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.091+0000 7fc7a6a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc798053380 0x7fc798037bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.091+0000 7fc7a6a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7980e3640 con 0x7fc79804fdb0 2024-04-04T06:06:05.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.091+0000 7fc7a6a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc798053930 con 0x7fc7980e3260 2024-04-04T06:06:05.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.091+0000 7fc7a6a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc798053c10 con 0x7fc798053380 2024-04-04T06:06:05.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.092+0000 7f7507e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f75080d3e40 0x7f75080b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.092+0000 7f7507e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75080cef20 0x7f75080cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.092+0000 7f7507e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75080d2da0 con 0x7f750815e460 2024-04-04T06:06:05.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.092+0000 7f7507e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f75080cf8d0 con 0x7f75080d3e40 2024-04-04T06:06:05.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.092+0000 7f7507e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f75080b8320 con 0x7f75080cef20 2024-04-04T06:06:05.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.091+0000 7f6c66a00640 1 -- 172.21.15.159:0/1253946139 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c50034070 con 0x7f6c6806cc90 2024-04-04T06:06:05.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.092+0000 7f708b400640 1 -- 172.21.15.159:0/2715001238 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7080035020 con 0x7f70840ead00 2024-04-04T06:06:05.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.092+0000 7f708b400640 1 -- 172.21.15.159:0/2715001238 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7080031870 con 0x7f70840ead00 2024-04-04T06:06:05.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.092+0000 7f708b400640 1 -- 172.21.15.159:0/2715001238 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7080031c40 con 0x7f70840ead00 2024-04-04T06:06:05.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.092+0000 7f7506a00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f750815e460 0x7f75080d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45932/0 (socket says 172.21.15.159:45932) 2024-04-04T06:06:05.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.092+0000 7f7506a00640 1 -- 172.21.15.159:0/1381549599 learned_addr learned my addr 172.21.15.159:0/1381549599 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:05.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.092+0000 7f6c66a00640 1 -- 172.21.15.159:0/1253946139 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c50030860 con 0x7f6c6806cc90 2024-04-04T06:06:05.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.092+0000 7f6c66a00640 1 -- 172.21.15.159:0/1253946139 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c50030c30 con 0x7f6c6806cc90 2024-04-04T06:06:05.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.092+0000 7f7506000640 1 --2- 172.21.15.159:0/1381549599 >> v2:172.21.15.178:3300/0 conn(0x7f75080d3e40 0x7f75080b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.092+0000 7fc7a6000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc798053380 0x7fc798037bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.092+0000 7fc7a6000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc798053380 0x7fc798037bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:54938/0 (socket says 172.21.15.159:54938) 2024-04-04T06:06:05.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.092+0000 7fc7a6000640 1 -- 172.21.15.159:0/3568706441 learned_addr learned my addr 172.21.15.159:0/3568706441 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.092+0000 7f3191400640 1 --2- 172.21.15.159:0/2599280194 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f318c054200 0x7f318c050740 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f31800097a0 tx=0x7f318002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f09d59cd1273a534 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7fc7a5600640 1 --2- 172.21.15.159:0/3568706441 >> v2:172.21.15.178:3300/0 conn(0x7fc7980e3260 0x7fc79804f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7f7092600640 1 -- 172.21.15.159:0/2715001238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70840ead00 msgr2=0x7f708404fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7f7092600640 1 --2- 172.21.15.159:0/2715001238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70840ead00 0x7f708404fa90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f708400bd00 tx=0x7f708002b910 comp rx=0 tx=0).stop 2024-04-04T06:06:05.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7f7505600640 1 -- 172.21.15.159:0/1381549599 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 725013345 0 0) 0x7f75080d2da0 con 0x7f750815e460 2024-04-04T06:06:05.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7f7505600640 1 -- 172.21.15.159:0/1381549599 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74ec003520 con 0x7f750815e460 2024-04-04T06:06:05.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7f3190a00640 1 -- 172.21.15.159:0/2599280194 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3180035020 con 0x7f318c054200 2024-04-04T06:06:05.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7f090a600640 1 Processor -- start 2024-04-04T06:06:05.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7f3190a00640 1 -- 172.21.15.159:0/2599280194 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3180031810 con 0x7f318c054200 2024-04-04T06:06:05.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7f3190a00640 1 -- 172.21.15.159:0/2599280194 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3180031be0 con 0x7f318c054200 2024-04-04T06:06:05.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7f7507400640 1 --2- 172.21.15.159:0/1381549599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75080cef20 0x7f75080cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7fded3200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdecc0ead00 0x7fdecc04fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7fded3200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fdecc006ea0 0x7fdecc007310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7fded3200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdecc053720 con 0x7fdecc04ffd0 2024-04-04T06:06:05.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7fded3200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fdecc007850 con 0x7fdecc006ea0 2024-04-04T06:06:05.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7fded3200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fdecc054330 con 0x7fdecc0ead00 2024-04-04T06:06:05.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7fc7a6000640 1 -- 172.21.15.159:0/3568706441 >> v2:172.21.15.178:3300/0 conn(0x7fc7980e3260 msgr2=0x7fc79804f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7fc7a6000640 1 --2- 172.21.15.159:0/3568706441 >> v2:172.21.15.178:3300/0 conn(0x7fc7980e3260 0x7fc79804f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7fc7a6000640 1 -- 172.21.15.159:0/3568706441 >> v1:172.21.15.178:6789/0 conn(0x7fc79804fdb0 legacy=0x7fc798052c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7fc7a6000640 1 -- 172.21.15.159:0/3568706441 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc798053ef0 con 0x7fc798053380 2024-04-04T06:06:05.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7f090a600640 1 -- start start 2024-04-04T06:06:05.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7f7505600640 1 -- 172.21.15.159:0/1381549599 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2240235244 0 0) 0x7f74ec003520 con 0x7f750815e460 2024-04-04T06:06:05.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7f7505600640 1 -- 172.21.15.159:0/1381549599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75080cef20 msgr2=0x7f75080cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7f7505600640 1 --2- 172.21.15.159:0/1381549599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75080cef20 0x7f75080cf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7f7505600640 1 -- 172.21.15.159:0/1381549599 >> v2:172.21.15.178:3300/0 conn(0x7f75080d3e40 msgr2=0x7f75080b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.093+0000 7f7505600640 1 --2- 172.21.15.159:0/1381549599 >> v2:172.21.15.178:3300/0 conn(0x7f75080d3e40 0x7f75080b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.094+0000 7f7505600640 1 -- 172.21.15.159:0/1381549599 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75080d3080 con 0x7f750815e460 2024-04-04T06:06:05.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.094+0000 7f7505600640 1 -- 172.21.15.159:0/1381549599 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f74f4001670 con 0x7f750815e460 2024-04-04T06:06:05.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.094+0000 7f7505600640 1 -- 172.21.15.159:0/1381549599 >> v1:172.21.15.159:6789/0 conn(0x7f750815e460 legacy=0x7f75080d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.094+0000 7f7505600640 1 --2- 172.21.15.159:0/1381549599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74ec0039b0 0x7f74ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.094+0000 7f7505600640 1 --2- 172.21.15.159:0/1381549599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74ec004800 0x7f74ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.094+0000 7f7505600640 1 --2- 172.21.15.159:0/1381549599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74ec0051c0 0x7f74ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.094+0000 7f7505600640 1 -- 172.21.15.159:0/1381549599 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f75080cf8d0 con 0x7f74ec0039b0 2024-04-04T06:06:05.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.094+0000 7f7505600640 1 -- 172.21.15.159:0/1381549599 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f75080b8320 con 0x7f74ec004800 2024-04-04T06:06:05.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.094+0000 7f7505600640 1 -- 172.21.15.159:0/1381549599 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f74ec00c030 con 0x7f74ec0051c0 2024-04-04T06:06:05.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.094+0000 7f6c6e200640 1 -- 172.21.15.159:0/1253946139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c6806cc90 msgr2=0x7f6c68163cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.094+0000 7f6c6e200640 1 --2- 172.21.15.159:0/1253946139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c6806cc90 0x7f6c68163cf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009870 tx=0x7f6c5002b9a0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.094+0000 7fded1400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fdecc04ffd0 0x7fdecc037bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45936/0 (socket says 172.21.15.159:45936) 2024-04-04T06:06:05.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.094+0000 7fded1400640 1 -- 172.21.15.159:0/704126335 learned_addr learned my addr 172.21.15.159:0/704126335 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:05.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.094+0000 7fded1e00640 1 --2- 172.21.15.159:0/704126335 >> v2:172.21.15.178:3300/0 conn(0x7fdecc0ead00 0x7fdecc04fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.094+0000 7fded2800640 1 --2- 172.21.15.159:0/704126335 >> v2:172.21.15.159:3300/0 conn(0x7fdecc006ea0 0x7fdecc007310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f7092600640 1 -- 172.21.15.159:0/2715001238 shutdown_connections 2024-04-04T06:06:05.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f7092600640 1 --2- 172.21.15.159:0/2715001238 >> v2:172.21.15.178:3300/0 conn(0x7f7084038110 0x7f70840385a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f7092600640 1 --2- 172.21.15.159:0/2715001238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f708404ffd0 0x7f7084037bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f7092600640 1 --2- 172.21.15.159:0/2715001238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70840ead00 0x7f708404fa90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f7092600640 1 -- 172.21.15.159:0/2715001238 >> 172.21.15.159:0/2715001238 conn(0x7f7084009fb0 msgr2=0x7f708400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.094+0000 7f7507400640 1 --2- 172.21.15.159:0/1381549599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75080cef20 0x7f75080cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:05.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.094+0000 7f7506a00640 1 --2- 172.21.15.159:0/1381549599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74ec0039b0 0x7f74ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.094+0000 7f3193200640 1 -- 172.21.15.159:0/2599280194 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f318c054200 msgr2=0x7f318c050740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.094+0000 7f3193200640 1 --2- 172.21.15.159:0/2599280194 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f318c054200 0x7f318c050740 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f31800097a0 tx=0x7f318002b8b0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f3193200640 1 -- 172.21.15.159:0/2599280194 shutdown_connections 2024-04-04T06:06:05.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f3193200640 1 --2- 172.21.15.159:0/2599280194 >> v2:172.21.15.159:3300/0 conn(0x7f318c006720 0x7f318c006bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f3193200640 1 --2- 172.21.15.159:0/2599280194 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f318c054200 0x7f318c050740 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f3193200640 1 -- 172.21.15.159:0/2599280194 >> 172.21.15.159:0/2599280194 conn(0x7f318c009fb0 msgr2=0x7f318c00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f7507400640 1 --2- 172.21.15.159:0/1381549599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74ec0051c0 0x7f74ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f7506000640 1 --2- 172.21.15.159:0/1381549599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74ec004800 0x7f74ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f090a600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08fc0e3160 0x7f08fc0e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f0eaa800640 1 Processor -- start 2024-04-04T06:06:05.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7fc7a6000640 1 --2- 172.21.15.159:0/3568706441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc798053380 0x7fc798037bb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0000b60 tx=0x7fc7a00a7100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5127a15fbd0a1f98 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f3193200640 1 -- 172.21.15.159:0/2599280194 shutdown_connections 2024-04-04T06:06:05.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f090a600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f08fc03c0a0 0x7f08fc058050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f090a600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f08fc053cb0 0x7f08fc054120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f090a600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f08fc058590 con 0x7f08fc03c0a0 2024-04-04T06:06:05.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f090a600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f08fc054660 con 0x7f08fc053cb0 2024-04-04T06:06:05.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f090a600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f08fc058810 con 0x7f08fc0e3160 2024-04-04T06:06:05.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f7507400640 1 -- 172.21.15.159:0/1381549599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74ec004800 msgr2=0x7f74ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f7507400640 1 --2- 172.21.15.159:0/1381549599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74ec004800 0x7f74ec004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f7507400640 1 -- 172.21.15.159:0/1381549599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74ec0039b0 msgr2=0x7f74ec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f0eaa800640 1 -- start start 2024-04-04T06:06:05.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.096+0000 7f3193200640 1 -- 172.21.15.159:0/2599280194 wait complete. 2024-04-04T06:06:05.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7f0909200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08fc0e3160 0x7f08fc0e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7fded2800640 1 -- 172.21.15.159:0/704126335 >> v2:172.21.15.178:3300/0 conn(0x7fdecc0ead00 msgr2=0x7fdecc04fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7fded2800640 1 --2- 172.21.15.159:0/704126335 >> v2:172.21.15.178:3300/0 conn(0x7fdecc0ead00 0x7fdecc04fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7fded2800640 1 -- 172.21.15.159:0/704126335 >> v1:172.21.15.159:6789/0 conn(0x7fdecc04ffd0 legacy=0x7fdecc037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.095+0000 7fded2800640 1 -- 172.21.15.159:0/704126335 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdecc053a00 con 0x7fdecc006ea0 2024-04-04T06:06:05.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.096+0000 7f7092600640 1 -- 172.21.15.159:0/2715001238 shutdown_connections 2024-04-04T06:06:05.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.096+0000 7f6c6e200640 1 -- 172.21.15.159:0/1253946139 shutdown_connections 2024-04-04T06:06:05.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.096+0000 7f6c6e200640 1 --2- 172.21.15.159:0/1253946139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c6806cc90 0x7f6c68163cf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.096+0000 7f0903e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f08fc03c0a0 0x7f08fc058050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.096+0000 7f0903e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f08fc03c0a0 0x7f08fc058050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36932/0 (socket says 172.21.15.159:36932) 2024-04-04T06:06:05.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.096+0000 7f0903e00640 1 -- 172.21.15.159:0/2713980745 learned_addr learned my addr 172.21.15.159:0/2713980745 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.096+0000 7f6c6e200640 1 -- 172.21.15.159:0/1253946139 >> 172.21.15.159:0/1253946139 conn(0x7f6c68071e60 msgr2=0x7f6c68072270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.096+0000 7fc79fe00640 1 -- 172.21.15.159:0/3568706441 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00b4070 con 0x7fc798053380 2024-04-04T06:06:05.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.096+0000 7fc79fe00640 1 -- 172.21.15.159:0/3568706441 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7a00af440 con 0x7fc798053380 2024-04-04T06:06:05.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.096+0000 7fc79fe00640 1 -- 172.21.15.159:0/3568706441 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00af8b0 con 0x7fc798053380 2024-04-04T06:06:05.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.096+0000 7f0903e00640 1 -- 172.21.15.159:0/2713980745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08fc0e3160 msgr2=0x7f08fc0e7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.096+0000 7f0903e00640 1 --2- 172.21.15.159:0/2713980745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08fc0e3160 0x7f08fc0e7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.096+0000 7f0903e00640 1 -- 172.21.15.159:0/2713980745 >> v2:172.21.15.178:3300/0 conn(0x7f08fc053cb0 msgr2=0x7f08fc054120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.096+0000 7f0903e00640 1 --2- 172.21.15.159:0/2713980745 >> v2:172.21.15.178:3300/0 conn(0x7f08fc053cb0 0x7f08fc054120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.096+0000 7f0903e00640 1 -- 172.21.15.159:0/2713980745 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08fc058af0 con 0x7f08fc03c0a0 2024-04-04T06:06:05.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.096+0000 7f7507400640 1 --2- 172.21.15.159:0/1381549599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74ec0039b0 0x7f74ec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.096+0000 7f7507400640 1 -- 172.21.15.159:0/1381549599 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75080d3080 con 0x7f74ec0051c0 2024-04-04T06:06:05.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.096+0000 7f7506a00640 1 --2- 172.21.15.159:0/1381549599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74ec0039b0 0x7f74ec003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:06:05.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.097+0000 7f6c6e200640 1 -- 172.21.15.159:0/1253946139 shutdown_connections 2024-04-04T06:06:05.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.097+0000 7fc7a6a00640 1 -- 172.21.15.159:0/3568706441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc798053380 msgr2=0x7fc798037bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.097+0000 7fc7a6a00640 1 --2- 172.21.15.159:0/3568706441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc798053380 0x7fc798037bb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0000b60 tx=0x7fc7a00a7100 comp rx=0 tx=0).stop 2024-04-04T06:06:05.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.098+0000 7f0eaa800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e9c00aaa0 0x7f0e9c0485f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.098+0000 7f0eaa800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e9c04ffd0 0x7f0e9c0e2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.098+0000 7fc7a6a00640 1 -- 172.21.15.159:0/3568706441 shutdown_connections 2024-04-04T06:06:05.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.098+0000 7fc7a6a00640 1 --2- 172.21.15.159:0/3568706441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc798053380 0x7fc798037bb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.098+0000 7fc7a6a00640 1 --2- 172.21.15.159:0/3568706441 >> v2:172.21.15.178:3300/0 conn(0x7fc7980e3260 0x7fc79804f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.098+0000 7fc7a6a00640 1 -- 172.21.15.159:0/3568706441 >> 172.21.15.159:0/3568706441 conn(0x7fc798009fb0 msgr2=0x7fc79800a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.098+0000 7f0903e00640 1 --2- 172.21.15.159:0/2713980745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08fc03c0a0 0x7f08fc058050 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f08fc054cd0 tx=0x7f08f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58aedf211ee655c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.098+0000 7f0903400640 1 -- 172.21.15.159:0/2713980745 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0035020 con 0x7f08fc03c0a0 2024-04-04T06:06:05.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.098+0000 7f3193200640 1 Processor -- start 2024-04-04T06:06:05.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.098+0000 7f7507400640 1 --2- 172.21.15.159:0/1381549599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74ec0051c0 0x7f74ec00b9c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f75080cf300 tx=0x7f74ec005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54ff6ba6c5ecf10f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.098+0000 7fc7a6a00640 1 -- 172.21.15.159:0/3568706441 shutdown_connections 2024-04-04T06:06:05.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.098+0000 7f0eaa800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e9c0df420 0x7f0e9c180090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.098+0000 7f0903400640 1 -- 172.21.15.159:0/2713980745 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08f0031870 con 0x7f08fc03c0a0 2024-04-04T06:06:05.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.099+0000 7f0903400640 1 -- 172.21.15.159:0/2713980745 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0031c40 con 0x7f08fc03c0a0 2024-04-04T06:06:05.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.099+0000 7f0ea9400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e9c00aaa0 0x7f0e9c0485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.098+0000 7f3193200640 1 -- start start 2024-04-04T06:06:05.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.099+0000 7f0eaa800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c052b40 con 0x7f0e9c04ffd0 2024-04-04T06:06:05.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.099+0000 7f0eaa800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c0547b0 con 0x7f0e9c0df420 2024-04-04T06:06:05.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.099+0000 7f0eaa800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0e9c052cc0 con 0x7f0e9c00aaa0 2024-04-04T06:06:05.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.099+0000 7f0ea8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e9c04ffd0 0x7f0e9c0e2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.099+0000 7f0ea9400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e9c00aaa0 0x7f0e9c0485f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:54984/0 (socket says 172.21.15.159:54984) 2024-04-04T06:06:05.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.099+0000 7f0ea9400640 1 -- 172.21.15.159:0/2833251566 learned_addr learned my addr 172.21.15.159:0/2833251566 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.098+0000 7fded2800640 1 --2- 172.21.15.159:0/704126335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdecc006ea0 0x7fdecc007310 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fded4050b20 tx=0x7fded40a7050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ba965680df09b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.098+0000 7fded0a00640 1 -- 172.21.15.159:0/704126335 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded40b4070 con 0x7fdecc006ea0 2024-04-04T06:06:05.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.099+0000 7fded0a00640 1 -- 172.21.15.159:0/704126335 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded40af440 con 0x7fdecc006ea0 2024-04-04T06:06:05.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.099+0000 7fded0a00640 1 -- 172.21.15.159:0/704126335 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded40b4070 con 0x7fdecc006ea0 2024-04-04T06:06:05.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.096+0000 7f7092600640 1 -- 172.21.15.159:0/2715001238 wait complete. 2024-04-04T06:06:05.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.099+0000 7f7092600640 1 Processor -- start 2024-04-04T06:06:05.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.097+0000 7f6c6e200640 1 -- 172.21.15.159:0/1253946139 wait complete. 2024-04-04T06:06:05.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.099+0000 7f0ea9e00640 1 --2- 172.21.15.159:0/2833251566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e9c0df420 0x7f0e9c180090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.099+0000 7f6c6e200640 1 Processor -- start 2024-04-04T06:06:05.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.099+0000 7f7092600640 1 -- start start 2024-04-04T06:06:05.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7f7505600640 1 -- 172.21.15.159:0/1381549599 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500007a80 con 0x7f74ec0051c0 2024-04-04T06:06:05.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7f090a600640 1 -- 172.21.15.159:0/2713980745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08fc03c0a0 msgr2=0x7f08fc058050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7f090a600640 1 --2- 172.21.15.159:0/2713980745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08fc03c0a0 0x7f08fc058050 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f08fc054cd0 tx=0x7f08f002b910 comp rx=0 tx=0).stop 2024-04-04T06:06:05.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7f0ea9400640 1 -- 172.21.15.159:0/2833251566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e9c0df420 msgr2=0x7f0e9c180090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7f0ea9400640 1 --2- 172.21.15.159:0/2833251566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e9c0df420 0x7f0e9c180090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7f0ea9400640 1 -- 172.21.15.159:0/2833251566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e9c04ffd0 msgr2=0x7f0e9c0e2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7f0ea9400640 1 --2- 172.21.15.159:0/2833251566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e9c04ffd0 0x7f0e9c0e2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7fded3200640 1 -- 172.21.15.159:0/704126335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdecc006ea0 msgr2=0x7fdecc007310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7fded3200640 1 --2- 172.21.15.159:0/704126335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdecc006ea0 0x7fdecc007310 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fded4050b20 tx=0x7fded40a7050 comp rx=0 tx=0).stop 2024-04-04T06:06:05.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7fded3200640 1 -- 172.21.15.159:0/704126335 shutdown_connections 2024-04-04T06:06:05.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7fded3200640 1 --2- 172.21.15.159:0/704126335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdecc006ea0 0x7fdecc007310 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7fded3200640 1 --2- 172.21.15.159:0/704126335 >> v2:172.21.15.178:3300/0 conn(0x7fdecc0ead00 0x7fdecc04fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7fded3200640 1 -- 172.21.15.159:0/704126335 >> 172.21.15.159:0/704126335 conn(0x7fdecc009fb0 msgr2=0x7fdecc00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7f7505600640 1 -- 172.21.15.159:0/1381549599 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f750002ce50 con 0x7f74ec0051c0 2024-04-04T06:06:05.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7f7505600640 1 -- 172.21.15.159:0/1381549599 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500033820 con 0x7f74ec0051c0 2024-04-04T06:06:05.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7f6c6e200640 1 -- start start 2024-04-04T06:06:05.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7f0ea9400640 1 -- 172.21.15.159:0/2833251566 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9c0e0ff0 con 0x7f0e9c00aaa0 2024-04-04T06:06:05.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7f0ea8a00640 1 --2- 172.21.15.159:0/2833251566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0e9c04ffd0 0x7f0e9c0e2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:06:05.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7f0ea9400640 1 --2- 172.21.15.159:0/2833251566 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e9c00aaa0 0x7f0e9c0485f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4063110 tx=0x7f0ea406ae20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7f0ea9e00640 1 --2- 172.21.15.159:0/2833251566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e9c0df420 0x7f0e9c180090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:06:05.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.098+0000 7fc7a6a00640 1 -- 172.21.15.159:0/3568706441 wait complete. 2024-04-04T06:06:05.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.101+0000 7f3193200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f318c006720 0x7f318c1716a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.101+0000 7f3193200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f318c054200 0x7f318c17d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.101+0000 7f3193200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f318c0df800 0x7f318c0e15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.101+0000 7f3193200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f318c054850 con 0x7f318c0df800 2024-04-04T06:06:05.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.101+0000 7f3193200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f318c052c20 con 0x7f318c006720 2024-04-04T06:06:05.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.101+0000 7f3193200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f318c053f60 con 0x7f318c054200 2024-04-04T06:06:05.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.101+0000 7f0ea1600640 1 -- 172.21.15.159:0/2833251566 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40b7030 con 0x7f0e9c00aaa0 2024-04-04T06:06:05.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.101+0000 7f0ea1600640 1 -- 172.21.15.159:0/2833251566 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ea40b9070 con 0x7f0e9c00aaa0 2024-04-04T06:06:05.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.101+0000 7f6c6e200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c6806cc90 0x7f6c681fe860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.101+0000 7f0ea1600640 1 -- 172.21.15.159:0/2833251566 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40b2b40 con 0x7f0e9c00aaa0 2024-04-04T06:06:05.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.101+0000 7f6c6e200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c6806f950 0x7f6c68164d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.101+0000 7f3191e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f318c006720 0x7f318c1716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.101+0000 7f3191e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f318c006720 0x7f318c1716a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:40994/0 (socket says 172.21.15.159:40994) 2024-04-04T06:06:05.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.101+0000 7f3191e00640 1 -- 172.21.15.159:0/2471495833 learned_addr learned my addr 172.21.15.159:0/2471495833 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.102+0000 7f6c6e200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c680d0390 0x7f6c68165280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.102+0000 7f6c6e200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6806eed0 con 0x7f6c680d0390 2024-04-04T06:06:05.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.102+0000 7f6c6e200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6c680b40f0 con 0x7f6c6806f950 2024-04-04T06:06:05.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.102+0000 7f6c6e200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6c6806f050 con 0x7f6c6806cc90 2024-04-04T06:06:05.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.102+0000 7f6c5fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c6806f950 0x7f6c68164d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.102+0000 7f6c67e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c680d0390 0x7f6c68165280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.102+0000 7f6c5fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c6806f950 0x7f6c68164d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:41004/0 (socket says 172.21.15.159:41004) 2024-04-04T06:06:05.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.102+0000 7f6c5fe00640 1 -- 172.21.15.159:0/3331700055 learned_addr learned my addr 172.21.15.159:0/3331700055 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.102+0000 7f7092600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7084038110 0x7f7084043680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.102+0000 7f3192800640 1 --2- 172.21.15.159:0/2471495833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f318c0df800 0x7f318c0e15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.102+0000 7f3191400640 1 --2- 172.21.15.159:0/2471495833 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f318c054200 0x7f318c17d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7fded3200640 1 -- 172.21.15.159:0/704126335 shutdown_connections 2024-04-04T06:06:05.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.100+0000 7fded3200640 1 -- 172.21.15.159:0/704126335 wait complete. 2024-04-04T06:06:05.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.101+0000 7f7507e00640 1 -- 172.21.15.159:0/1381549599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74ec0051c0 msgr2=0x7f74ec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.101+0000 7f7507e00640 1 --2- 172.21.15.159:0/1381549599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74ec0051c0 0x7f74ec00b9c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f75080cf300 tx=0x7f74ec005830 comp rx=0 tx=0).stop 2024-04-04T06:06:05.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.101+0000 7fc7a6a00640 1 Processor -- start 2024-04-04T06:06:05.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.101+0000 7fc7a6a00640 1 -- start start 2024-04-04T06:06:05.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7fc7a6a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc79804fdb0 0x7fc79804bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7fc7a6a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc798053380 0x7fc79817b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7fc7a6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7980e3260 0x7fc79804a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7fc7a6a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc7980380f0 con 0x7fc7980e3260 2024-04-04T06:06:05.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f7507400640 1 -- 172.21.15.159:0/1381549599 reap_dead start 2024-04-04T06:06:05.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7fc7a6a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc7980544b0 con 0x7fc79804fdb0 2024-04-04T06:06:05.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7fc7a6a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc798038270 con 0x7fc798053380 2024-04-04T06:06:05.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f090a600640 1 -- 172.21.15.159:0/2713980745 shutdown_connections 2024-04-04T06:06:05.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f090a600640 1 --2- 172.21.15.159:0/2713980745 >> v2:172.21.15.178:3300/0 conn(0x7f08fc053cb0 0x7f08fc054120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f090a600640 1 --2- 172.21.15.159:0/2713980745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08fc03c0a0 0x7f08fc058050 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f090a600640 1 --2- 172.21.15.159:0/2713980745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08fc0e3160 0x7f08fc0e7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f090a600640 1 -- 172.21.15.159:0/2713980745 >> 172.21.15.159:0/2713980745 conn(0x7f08fc058ee0 msgr2=0x7f08fc0592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f7507e00640 1 -- 172.21.15.159:0/1381549599 shutdown_connections 2024-04-04T06:06:05.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f7507e00640 1 -- 172.21.15.159:0/1381549599 >> 172.21.15.159:0/1381549599 conn(0x7f75080678b0 msgr2=0x7f75080d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f090a600640 1 -- 172.21.15.159:0/2713980745 shutdown_connections 2024-04-04T06:06:05.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.102+0000 7f0eaa800640 1 -- 172.21.15.159:0/2833251566 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e9c0e12d0 con 0x7f0e9c00aaa0 2024-04-04T06:06:05.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f0eaa800640 1 -- 172.21.15.159:0/2833251566 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e9c0e1890 con 0x7f0e9c00aaa0 2024-04-04T06:06:05.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f3192800640 1 -- 172.21.15.159:0/2471495833 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f318c054200 msgr2=0x7f318c17d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f3192800640 1 --2- 172.21.15.159:0/2471495833 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f318c054200 0x7f318c17d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f7507e00640 1 -- 172.21.15.159:0/1381549599 shutdown_connections 2024-04-04T06:06:05.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f3192800640 1 -- 172.21.15.159:0/2471495833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f318c006720 msgr2=0x7f318c1716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f3192800640 1 --2- 172.21.15.159:0/2471495833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f318c006720 0x7f318c1716a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f3192800640 1 -- 172.21.15.159:0/2471495833 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318c171be0 con 0x7f318c0df800 2024-04-04T06:06:05.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f7507e00640 1 -- 172.21.15.159:0/1381549599 wait complete. 2024-04-04T06:06:05.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.102+0000 7f7092600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f708404ffd0 0x7f70840070f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.102+0000 7f7092600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70840ead00 0x7f7084046ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7fc7a6000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7980e3260 0x7fc79804a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7fc7a6000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7980e3260 0x7fc79804a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36962/0 (socket says 172.21.15.159:36962) 2024-04-04T06:06:05.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7fc7a6000640 1 -- 172.21.15.159:0/175256534 learned_addr learned my addr 172.21.15.159:0/175256534 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.104+0000 7f7091200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7084038110 0x7f7084043680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.104+0000 7f708be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f708404ffd0 0x7f70840070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.104+0000 7f708be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f708404ffd0 0x7f70840070f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36954/0 (socket says 172.21.15.159:36954) 2024-04-04T06:06:05.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f6c5fe00640 1 -- 172.21.15.159:0/3331700055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c6806cc90 msgr2=0x7f6c681fe860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.102+0000 7f6c67400640 1 --2- 172.21.15.159:0/3331700055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c6806cc90 0x7f6c681fe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f6c5fe00640 1 --2- 172.21.15.159:0/3331700055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c6806cc90 0x7f6c681fe860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f6c5fe00640 1 -- 172.21.15.159:0/3331700055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c680d0390 msgr2=0x7f6c68165280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f6c5fe00640 1 --2- 172.21.15.159:0/3331700055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c680d0390 0x7f6c68165280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f6c5fe00640 1 -- 172.21.15.159:0/3331700055 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c68151790 con 0x7f6c6806f950 2024-04-04T06:06:05.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f6c67400640 1 --2- 172.21.15.159:0/3331700055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c6806cc90 0x7f6c681fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:05.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.104+0000 7f6c67e00640 1 --2- 172.21.15.159:0/3331700055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c680d0390 0x7f6c68165280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:05.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.104+0000 7f7092600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7084054940 con 0x7f708404ffd0 2024-04-04T06:06:05.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.104+0000 7f6c5fe00640 1 --2- 172.21.15.159:0/3331700055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c6806f950 0x7f6c68164d40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6c50000c00 tx=0x7f6c500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.104+0000 7f7092600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f70840547c0 con 0x7f70840ead00 2024-04-04T06:06:05.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.104+0000 7f7092600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f708404f550 con 0x7f7084038110 2024-04-04T06:06:05.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.104+0000 7f708be00640 1 -- 172.21.15.159:0/1476810441 learned_addr learned my addr 172.21.15.159:0/1476810441 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.104+0000 7f3192800640 1 --2- 172.21.15.159:0/2471495833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f318c0df800 0x7f318c0e15b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3194075e10 tx=0x7f319404ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.104+0000 7f0eaa800640 1 -- 172.21.15.159:0/2833251566 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0e9c005390 con 0x7f0e9c00aaa0 2024-04-04T06:06:05.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.103+0000 7f090a600640 1 -- 172.21.15.159:0/2713980745 wait complete. 2024-04-04T06:06:05.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.104+0000 7fc7a5600640 1 --2- 172.21.15.159:0/175256534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc79804fdb0 0x7fc79804bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.104+0000 7f6c5f400640 1 -- 172.21.15.159:0/3331700055 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c50034040 con 0x7f6c6806f950 2024-04-04T06:06:05.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.105+0000 7f6c5f400640 1 -- 172.21.15.159:0/3331700055 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c50039d90 con 0x7f6c6806f950 2024-04-04T06:06:05.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.105+0000 7fc7a6000640 1 -- 172.21.15.159:0/175256534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc798053380 msgr2=0x7fc79817b6d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-04T06:06:05.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.105+0000 7fc7a6000640 1 --2- 172.21.15.159:0/175256534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc798053380 0x7fc79817b6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.105+0000 7fc7a6000640 1 -- 172.21.15.159:0/175256534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc79804fdb0 msgr2=0x7fc79804bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.105+0000 7fc7a6000640 1 --2- 172.21.15.159:0/175256534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc79804fdb0 0x7fc79804bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.105+0000 7fc7a6000640 1 -- 172.21.15.159:0/175256534 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7980e1570 con 0x7fc7980e3260 2024-04-04T06:06:05.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.105+0000 7f6c5f400640 1 -- 172.21.15.159:0/3331700055 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c50043610 con 0x7f6c6806f950 2024-04-04T06:06:05.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.105+0000 7f6c6e200640 1 -- 172.21.15.159:0/3331700055 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c68151a70 con 0x7f6c6806f950 2024-04-04T06:06:05.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.105+0000 7f708be00640 1 -- 172.21.15.159:0/1476810441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7084038110 msgr2=0x7f7084043680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.105+0000 7f708be00640 1 --2- 172.21.15.159:0/1476810441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7084038110 0x7f7084043680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.105+0000 7f708be00640 1 -- 172.21.15.159:0/1476810441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70840ead00 msgr2=0x7f7084046ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-04T06:06:05.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.105+0000 7f708be00640 1 --2- 172.21.15.159:0/1476810441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70840ead00 0x7f7084046ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.105+0000 7f708be00640 1 -- 172.21.15.159:0/1476810441 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708417df60 con 0x7f708404ffd0 2024-04-04T06:06:05.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.105+0000 7fc7a6000640 1 --2- 172.21.15.159:0/175256534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7980e3260 0x7fc79804a7e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc7a00511b0 tx=0x7fc7a006ae20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.105+0000 7f6c6e200640 1 -- 172.21.15.159:0/3331700055 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c68152030 con 0x7f6c6806f950 2024-04-04T06:06:05.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.105+0000 7f318a000640 1 -- 172.21.15.159:0/2471495833 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319406b980 con 0x7f318c0df800 2024-04-04T06:06:05.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.105+0000 7f318a000640 1 -- 172.21.15.159:0/2471495833 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3194071070 con 0x7f318c0df800 2024-04-04T06:06:05.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.105+0000 7f318a000640 1 -- 172.21.15.159:0/2471495833 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319406b980 con 0x7f318c0df800 2024-04-04T06:06:05.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.106+0000 7f090a600640 1 Processor -- start 2024-04-04T06:06:05.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.106+0000 7f090a600640 1 -- start start 2024-04-04T06:06:05.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.106+0000 7f7507e00640 1 Processor -- start 2024-04-04T06:06:05.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.106+0000 7fded3200640 1 Processor -- start 2024-04-04T06:06:05.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.105+0000 7f708be00640 1 --2- 172.21.15.159:0/1476810441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f708404ffd0 0x7f70840070f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f707800d800 tx=0x7f707800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.107+0000 7f6c5cc00640 1 -- 172.21.15.159:0/3331700055 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6c38001030 con 0x7f6c6806f950 2024-04-04T06:06:05.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.107+0000 7fc79d600640 1 -- 172.21.15.159:0/175256534 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00b4050 con 0x7fc7980e3260 2024-04-04T06:06:05.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.107+0000 7fc79d600640 1 -- 172.21.15.159:0/175256534 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7a00b5040 con 0x7fc7980e3260 2024-04-04T06:06:05.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.107+0000 7fc79d600640 1 -- 172.21.15.159:0/175256534 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00aeaa0 con 0x7fc7980e3260 2024-04-04T06:06:05.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.106+0000 7f3193200640 1 -- 172.21.15.159:0/2471495833 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f318c17dab0 con 0x7f318c0df800 2024-04-04T06:06:05.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.107+0000 7f090a600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08fc03c0a0 0x7f08fc0dfce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.107+0000 7f090a600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08fc053cb0 0x7f08fc0e0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.107+0000 7f7088c00640 1 -- 172.21.15.159:0/1476810441 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707800b6c0 con 0x7f708404ffd0 2024-04-04T06:06:05.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.107+0000 7f7088c00640 1 -- 172.21.15.159:0/1476810441 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7078019070 con 0x7f708404ffd0 2024-04-04T06:06:05.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.107+0000 7f7088c00640 1 -- 172.21.15.159:0/1476810441 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707800ba80 con 0x7f708404ffd0 2024-04-04T06:06:05.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.107+0000 7fc7a6a00640 1 -- 172.21.15.159:0/175256534 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc798049a20 con 0x7fc7980e3260 2024-04-04T06:06:05.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.106+0000 7f3193200640 1 -- 172.21.15.159:0/2471495833 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f318c17e070 con 0x7f318c0df800 2024-04-04T06:06:05.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.107+0000 7f090a600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08fc0e3160 0x7f08fc0d3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.107+0000 7f090a600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc0e3540 con 0x7f08fc03c0a0 2024-04-04T06:06:05.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.107+0000 7f090a600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc006640 con 0x7f08fc0e3160 2024-04-04T06:06:05.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.107+0000 7fded3200640 1 -- start start 2024-04-04T06:06:05.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.107+0000 7f090a600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f08fc056e10 con 0x7f08fc053cb0 2024-04-04T06:06:05.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.107+0000 7f3193200640 1 -- 172.21.15.159:0/2471495833 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f318c005390 con 0x7f318c0df800 2024-04-04T06:06:05.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.107+0000 7f0903e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08fc053cb0 0x7f08fc0e0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.108+0000 7f7092600640 1 -- 172.21.15.159:0/1476810441 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708417d3e0 con 0x7f708404ffd0 2024-04-04T06:06:05.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.108+0000 7f0903e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08fc053cb0 0x7f08fc0e0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:55044/0 (socket says 172.21.15.159:55044) 2024-04-04T06:06:05.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.108+0000 7f0903e00640 1 -- 172.21.15.159:0/1553080322 learned_addr learned my addr 172.21.15.159:0/1553080322 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.108+0000 7f7092600640 1 -- 172.21.15.159:0/1476810441 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f708417d9a0 con 0x7f708404ffd0 2024-04-04T06:06:05.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.107+0000 7fc7a6a00640 1 -- 172.21.15.159:0/175256534 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc798049fe0 con 0x7fc7980e3260 2024-04-04T06:06:05.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.108+0000 7fc7a6a00640 1 -- 172.21.15.159:0/175256534 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc798005390 con 0x7fc7980e3260 2024-04-04T06:06:05.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.109+0000 7fded3200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdecc006ea0 0x7fdecc04cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.109+0000 7f0903e00640 1 -- 172.21.15.159:0/1553080322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08fc0e3160 msgr2=0x7f08fc0d3b90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-04T06:06:05.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.109+0000 7f0903e00640 1 --2- 172.21.15.159:0/1553080322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08fc0e3160 0x7f08fc0d3b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.109+0000 7f0903e00640 1 -- 172.21.15.159:0/1553080322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08fc03c0a0 msgr2=0x7f08fc0dfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.109+0000 7f0903e00640 1 --2- 172.21.15.159:0/1553080322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08fc03c0a0 0x7f08fc0dfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.109+0000 7f0903e00640 1 -- 172.21.15.159:0/1553080322 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08fc0d67e0 con 0x7f08fc053cb0 2024-04-04T06:06:05.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.109+0000 7f0903e00640 1 --2- 172.21.15.159:0/1553080322 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08fc053cb0 0x7f08fc0e0220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f08f002b950 tx=0x7f08f00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.109+0000 7fded3200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdecc04ffd0 0x7fdecc04e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.109+0000 7fded3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdecc0ead00 0x7fdecc0e85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.109+0000 7fded3200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc0547c0 con 0x7fdecc0ead00 2024-04-04T06:06:05.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.109+0000 7fded3200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc053fc0 con 0x7fdecc04ffd0 2024-04-04T06:06:05.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.109+0000 7fded3200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdecc054940 con 0x7fdecc006ea0 2024-04-04T06:06:05.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.109+0000 7f7092600640 1 -- 172.21.15.159:0/1476810441 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7084005390 con 0x7f708404ffd0 2024-04-04T06:06:05.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.107+0000 7f7507e00640 1 -- start start 2024-04-04T06:06:05.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.109+0000 7f7507e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74ec0051c0 0x7f750815c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.109+0000 7f7507e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f750815c320 0x7f75080c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.109+0000 7f7507e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75081ed8f0 0x7f75081edd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.109+0000 7f7507e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f75080d3920 con 0x7f75081ed8f0 2024-04-04T06:06:05.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.109+0000 7f7507e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f75080d3640 con 0x7f750815c320 2024-04-04T06:06:05.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.109+0000 7f7507e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f75080cf390 con 0x7f74ec0051c0 2024-04-04T06:06:05.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.110+0000 7fded1400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdecc04ffd0 0x7fdecc04e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.110+0000 7fded1400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdecc04ffd0 0x7fdecc04e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:41038/0 (socket says 172.21.15.159:41038) 2024-04-04T06:06:05.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.110+0000 7fded1400640 1 -- 172.21.15.159:0/860740905 learned_addr learned my addr 172.21.15.159:0/860740905 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.110+0000 7fded1e00640 1 --2- 172.21.15.159:0/860740905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdecc006ea0 0x7fdecc04cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.110+0000 7f0900c00640 1 -- 172.21.15.159:0/1553080322 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0040070 con 0x7f08fc053cb0 2024-04-04T06:06:05.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.110+0000 7f0900c00640 1 -- 172.21.15.159:0/1553080322 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08f0030d00 con 0x7f08fc053cb0 2024-04-04T06:06:05.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.110+0000 7f090a600640 1 -- 172.21.15.159:0/1553080322 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08fc0396a0 con 0x7f08fc053cb0 2024-04-04T06:06:05.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.110+0000 7f0900c00640 1 -- 172.21.15.159:0/1553080322 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0035040 con 0x7f08fc053cb0 2024-04-04T06:06:05.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.110+0000 7f090a600640 1 -- 172.21.15.159:0/1553080322 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f08fc0393c0 con 0x7f08fc053cb0 2024-04-04T06:06:05.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.108+0000 7f0ea1600640 1 -- 172.21.15.159:0/2833251566 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ea40c4060 con 0x7f0e9c00aaa0 2024-04-04T06:06:05.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.110+0000 7f7506a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74ec0051c0 0x7f750815c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.111+0000 7f7506000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f750815c320 0x7f75080c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.111+0000 7f7506a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74ec0051c0 0x7f750815c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:55056/0 (socket says 172.21.15.159:55056) 2024-04-04T06:06:05.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.111+0000 7f7506a00640 1 -- 172.21.15.159:0/3595645589 learned_addr learned my addr 172.21.15.159:0/3595645589 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.112+0000 7f0ea1600640 1 --2- 172.21.15.159:0/2833251566 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0e8806c940 0x7f0e8806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.111+0000 7f090a600640 1 -- 172.21.15.159:0/1553080322 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f08e0001030 con 0x7f08fc053cb0 2024-04-04T06:06:05.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.112+0000 7f0ea1600640 1 -- 172.21.15.159:0/2833251566 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0ea4128c70 con 0x7f0e9c00aaa0 2024-04-04T06:06:05.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.111+0000 7fded1400640 1 -- 172.21.15.159:0/860740905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdecc006ea0 msgr2=0x7fdecc04cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.111+0000 7fded1400640 1 --2- 172.21.15.159:0/860740905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdecc006ea0 0x7fdecc04cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.111+0000 7fded1400640 1 -- 172.21.15.159:0/860740905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdecc0ead00 msgr2=0x7fdecc0e85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.111+0000 7fded1400640 1 --2- 172.21.15.159:0/860740905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdecc0ead00 0x7fdecc0e85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.111+0000 7fded1400640 1 -- 172.21.15.159:0/860740905 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdecc04d250 con 0x7fdecc04ffd0 2024-04-04T06:06:05.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.112+0000 7f7507400640 1 --2- 172.21.15.159:0/3595645589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75081ed8f0 0x7f75081edd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.112+0000 7f7506a00640 1 -- 172.21.15.159:0/3595645589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f750815c320 msgr2=0x7f75080c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.112+0000 7f7506a00640 1 --2- 172.21.15.159:0/3595645589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f750815c320 0x7f75080c9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.112+0000 7f7506a00640 1 -- 172.21.15.159:0/3595645589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75081ed8f0 msgr2=0x7f75081edd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.112+0000 7f7506a00640 1 --2- 172.21.15.159:0/3595645589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75081ed8f0 0x7f75081edd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.112+0000 7f7506a00640 1 -- 172.21.15.159:0/3595645589 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75080c9ce0 con 0x7f74ec0051c0 2024-04-04T06:06:05.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.110+0000 7f6c5f400640 1 -- 172.21.15.159:0/3331700055 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c5004a060 con 0x7f6c6806f950 2024-04-04T06:06:05.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.112+0000 7f6c5f400640 1 --2- 172.21.15.159:0/3331700055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6c4406c8f0 0x7f6c4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.112+0000 7f7507400640 1 --2- 172.21.15.159:0/3595645589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75081ed8f0 0x7f75081edd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:05.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.112+0000 7f6c5f400640 1 -- 172.21.15.159:0/3331700055 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6c50079f10 con 0x7f6c6806f950 2024-04-04T06:06:05.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.111+0000 7fc79d600640 1 -- 172.21.15.159:0/175256534 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7a00c2030 con 0x7fc7980e3260 2024-04-04T06:06:05.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.112+0000 7f0ea1600640 1 --2- 172.21.15.159:0/2833251566 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x7f0e88074b60 0x7f0e88076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.112+0000 7f0ea1600640 1 -- 172.21.15.159:0/2833251566 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e88077630 con 0x7f0e88074b60 2024-04-04T06:06:05.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.112+0000 7f0ea1600640 1 -- 172.21.15.159:0/2833251566 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0ea4129020 con 0x7f0e9c00aaa0 2024-04-04T06:06:05.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.113+0000 7f6c5f400640 1 --2- 172.21.15.159:0/3331700055 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x7f6c44075760 0x7f6c44077c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.113+0000 7fded1400640 1 --2- 172.21.15.159:0/860740905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdecc04ffd0 0x7fdecc04e7f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdec000c4a0 tx=0x7fdec000c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.113+0000 7f6c5f400640 1 -- 172.21.15.159:0/3331700055 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6c440782d0 con 0x7f6c44075760 2024-04-04T06:06:05.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.112+0000 7f318a000640 1 -- 172.21.15.159:0/2471495833 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31940723e0 con 0x7f318c0df800 2024-04-04T06:06:05.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.113+0000 7fc79d600640 1 --2- 172.21.15.159:0/175256534 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc78006c940 0x7fc78006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.113+0000 7f0ea9e00640 1 --2- 172.21.15.159:0/2833251566 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x7f0e88074b60 0x7f0e88076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.113+0000 7f0ea8a00640 1 --2- 172.21.15.159:0/2833251566 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0e8806c940 0x7f0e8806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.113+0000 7f7506a00640 1 --2- 172.21.15.159:0/3595645589 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74ec0051c0 0x7f750815c790 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f74f4007fd0 tx=0x7f74f4007860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.113+0000 7fdeca000640 1 -- 172.21.15.159:0/860740905 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0017070 con 0x7fdecc04ffd0 2024-04-04T06:06:05.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.114+0000 7fdeca000640 1 -- 172.21.15.159:0/860740905 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec0005ce0 con 0x7fdecc04ffd0 2024-04-04T06:06:05.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.114+0000 7fc79d600640 1 -- 172.21.15.159:0/175256534 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc7a012d170 con 0x7fc7980e3260 2024-04-04T06:06:05.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.114+0000 7fc7a5600640 1 --2- 172.21.15.159:0/175256534 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc78006c940 0x7fc78006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.114+0000 7f318a000640 1 --2- 172.21.15.159:0/2471495833 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f317406caf0 0x7f317406efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.114+0000 7f0900c00640 1 -- 172.21.15.159:0/1553080322 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08f0038020 con 0x7f08fc053cb0 2024-04-04T06:06:05.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.114+0000 7fdeca000640 1 -- 172.21.15.159:0/860740905 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0007270 con 0x7fdecc04ffd0 2024-04-04T06:06:05.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.113+0000 7f6c5f400640 1 -- 172.21.15.159:0/3331700055 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6c500788a0 con 0x7f6c6806f950 2024-04-04T06:06:05.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.113+0000 7f6c67e00640 1 --2- 172.21.15.159:0/3331700055 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x7f6c44075760 0x7f6c44077c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.113+0000 7f6c67400640 1 --2- 172.21.15.159:0/3331700055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6c4406c8f0 0x7f6c4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.114+0000 7fc79d600640 1 --2- 172.21.15.159:0/175256534 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x7fc780074bc0 0x7fc780076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.114+0000 7fc79d600640 1 -- 172.21.15.159:0/175256534 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc7a00c2ec0 con 0x7fc780074bc0 2024-04-04T06:06:05.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.114+0000 7fc79d600640 1 -- 172.21.15.159:0/175256534 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc7a012e050 con 0x7fc7980e3260 2024-04-04T06:06:05.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.114+0000 7f7088c00640 1 -- 172.21.15.159:0/1476810441 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f707800bc20 con 0x7f708404ffd0 2024-04-04T06:06:05.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.114+0000 7f318a000640 1 -- 172.21.15.159:0/2471495833 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3194106ab0 con 0x7f318c0df800 2024-04-04T06:06:05.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.114+0000 7f318a000640 1 --2- 172.21.15.159:0/2471495833 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x7f3174074c90 0x7f31740770b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.114+0000 7f318a000640 1 -- 172.21.15.159:0/2471495833 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f319408ee50 con 0x7f3174074c90 2024-04-04T06:06:05.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.114+0000 7f318a000640 1 -- 172.21.15.159:0/2471495833 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3194106e60 con 0x7f318c0df800 2024-04-04T06:06:05.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.114+0000 7fded3200640 1 -- 172.21.15.159:0/860740905 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdecc04d530 con 0x7fdecc04ffd0 2024-04-04T06:06:05.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.114+0000 7fded3200640 1 -- 172.21.15.159:0/860740905 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdecc04daf0 con 0x7fdecc04ffd0 2024-04-04T06:06:05.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.115+0000 7f0ea8a00640 1 --2- 172.21.15.159:0/2833251566 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0e8806c940 0x7f0e8806ee00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c048b30 tx=0x7f0e98009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.115+0000 7f74fea00640 1 -- 172.21.15.159:0/3595645589 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f400fb10 con 0x7f74ec0051c0 2024-04-04T06:06:05.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.115+0000 7f74fea00640 1 -- 172.21.15.159:0/3595645589 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74f4018070 con 0x7f74ec0051c0 2024-04-04T06:06:05.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.115+0000 7f74fea00640 1 -- 172.21.15.159:0/3595645589 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f40133f0 con 0x7f74ec0051c0 2024-04-04T06:06:05.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.115+0000 7f0900c00640 1 --2- 172.21.15.159:0/1553080322 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f08e406c8f0 0x7f08e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.115+0000 7f0900c00640 1 -- 172.21.15.159:0/1553080322 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f08f00ae3f0 con 0x7f08fc053cb0 2024-04-04T06:06:05.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.115+0000 7f3191400640 1 --2- 172.21.15.159:0/2471495833 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x7f3174074c90 0x7f31740770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.115+0000 7f7507e00640 1 -- 172.21.15.159:0/3595645589 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75080c9fc0 con 0x7f74ec0051c0 2024-04-04T06:06:05.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.115+0000 7f7507e00640 1 -- 172.21.15.159:0/3595645589 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f75080ca580 con 0x7f74ec0051c0 2024-04-04T06:06:05.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.115+0000 7f0900c00640 1 --2- 172.21.15.159:0/1553080322 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x7f08e4074a90 0x7f08e4076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.115+0000 7f0900c00640 1 -- 172.21.15.159:0/1553080322 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f08e4077560 con 0x7f08e4074a90 2024-04-04T06:06:05.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.115+0000 7f0900c00640 1 -- 172.21.15.159:0/1553080322 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f08f00ae880 con 0x7f08fc053cb0 2024-04-04T06:06:05.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.115+0000 7fc7a4c00640 1 --2- 172.21.15.159:0/175256534 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x7fc780074bc0 0x7fc780076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.115+0000 7fc7a5600640 1 --2- 172.21.15.159:0/175256534 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc78006c940 0x7fc78006ee00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc794006fd0 tx=0x7fc794008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.116+0000 7f7088c00640 1 --2- 172.21.15.159:0/1476810441 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f706c06c9c0 0x7f706c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.116+0000 7f0909200640 1 --2- 172.21.15.159:0/1553080322 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f08e406c8f0 0x7f08e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.116+0000 7f7088c00640 1 -- 172.21.15.159:0/1476810441 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f707808c9e0 con 0x7f708404ffd0 2024-04-04T06:06:05.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.116+0000 7f7088c00640 1 --2- 172.21.15.159:0/1476810441 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x7f706c074be0 0x7f706c077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.116+0000 7f7088c00640 1 -- 172.21.15.159:0/1476810441 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f706c0776b0 con 0x7f706c074be0 2024-04-04T06:06:05.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.116+0000 7f7088c00640 1 -- 172.21.15.159:0/1476810441 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f707808cdd0 con 0x7f708404ffd0 2024-04-04T06:06:05.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.116+0000 7f7507e00640 1 -- 172.21.15.159:0/3595645589 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7508075940 con 0x7f74ec0051c0 2024-04-04T06:06:05.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.116+0000 7fded3200640 1 -- 172.21.15.159:0/860740905 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdecc005390 con 0x7fdecc04ffd0 2024-04-04T06:06:05.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.116+0000 7f6c67e00640 1 --2- 172.21.15.159:0/3331700055 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x7f6c44075760 0x7f6c44077c00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.117+0000 7f3191400640 1 --2- 172.21.15.159:0/2471495833 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x7f3174074c90 0x7f31740770b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.116+0000 7f6c67400640 1 --2- 172.21.15.159:0/3331700055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6c4406c8f0 0x7f6c4406edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6c6815cea0 tx=0x7f6c58009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.117+0000 7f0ea9e00640 1 --2- 172.21.15.159:0/2833251566 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x7f0e88074b60 0x7f0e88076f80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.117+0000 7f0909c00640 1 --2- 172.21.15.159:0/1553080322 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x7f08e4074a90 0x7f08e4076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.117+0000 7f3191e00640 1 --2- 172.21.15.159:0/2471495833 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f317406caf0 0x7f317406efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.117+0000 7f7091c00640 1 --2- 172.21.15.159:0/1476810441 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x7f706c074be0 0x7f706c077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.117+0000 7f7091200640 1 --2- 172.21.15.159:0/1476810441 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f706c06c9c0 0x7f706c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.118+0000 7f7091c00640 1 --2- 172.21.15.159:0/1476810441 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x7f706c074be0 0x7f706c077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.118+0000 7f318a000640 1 -- 172.21.15.159:0/2471495833 <== osd.3 v2:172.21.15.159:6820/2070760205 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f319408ee50 con 0x7f3174074c90 2024-04-04T06:06:05.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.118+0000 7fc7a4c00640 1 --2- 172.21.15.159:0/175256534 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x7fc780074bc0 0x7fc780076fe0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.118+0000 7f0909200640 1 --2- 172.21.15.159:0/1553080322 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f08e406c8f0 0x7f08e406edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f08f8006a90 tx=0x7f08f8006a20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.119+0000 7f7091200640 1 --2- 172.21.15.159:0/1476810441 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f706c06c9c0 0x7f706c06ee80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7080036000 tx=0x7f708003c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.119+0000 7f3191e00640 1 --2- 172.21.15.159:0/2471495833 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f317406caf0 0x7f317406efb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f318c180760 tx=0x7f3184009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.118+0000 7f0ea1600640 1 -- 172.21.15.159:0/2833251566 <== osd.2 v2:172.21.15.159:6808/2619731047 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e88077630 con 0x7f0e88074b60 2024-04-04T06:06:05.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.119+0000 7f7088c00640 1 -- 172.21.15.159:0/1476810441 <== osd.0 v2:172.21.15.159:6800/455382518 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f706c0776b0 con 0x7f706c074be0 2024-04-04T06:06:05.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.119+0000 7f0909c00640 1 --2- 172.21.15.159:0/1553080322 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x7f08e4074a90 0x7f08e4076eb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.120+0000 7f6c5f400640 1 -- 172.21.15.159:0/3331700055 <== osd.1 v2:172.21.15.159:6816/3765338297 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6c440782d0 con 0x7f6c44075760 2024-04-04T06:06:05.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.120+0000 7fdeca000640 1 -- 172.21.15.159:0/860740905 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdec001a070 con 0x7fdecc04ffd0 2024-04-04T06:06:05.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.121+0000 7fc79d600640 1 -- 172.21.15.159:0/175256534 <== osd.6 v2:172.21.15.178:6826/198949157 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc7a00c2ec0 con 0x7fc780074bc0 2024-04-04T06:06:05.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.120+0000 7f74fea00640 1 -- 172.21.15.159:0/3595645589 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74f4013590 con 0x7f74ec0051c0 2024-04-04T06:06:05.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.122+0000 7f0900c00640 1 -- 172.21.15.159:0/1553080322 <== osd.7 v2:172.21.15.178:6810/1066817938 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f08e4077560 con 0x7f08e4074a90 2024-04-04T06:06:05.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.121+0000 7fdeca000640 1 --2- 172.21.15.159:0/860740905 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdeb406c9f0 0x7fdeb406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.122+0000 7f74fea00640 1 --2- 172.21.15.159:0/3595645589 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f74e806c9f0 0x7f74e806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.121+0000 7fdeca000640 1 -- 172.21.15.159:0/860740905 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdec008ae40 con 0x7fdecc04ffd0 2024-04-04T06:06:05.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.122+0000 7fdeca000640 1 --2- 172.21.15.159:0/860740905 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x7fdeb4074c10 0x7fdeb4077030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.122+0000 7fdeca000640 1 -- 172.21.15.159:0/860740905 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdec0020e50 con 0x7fdeb4074c10 2024-04-04T06:06:05.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.122+0000 7fdeca000640 1 -- 172.21.15.159:0/860740905 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdec008b230 con 0x7fdecc04ffd0 2024-04-04T06:06:05.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.122+0000 7fded2800640 1 --2- 172.21.15.159:0/860740905 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x7fdeb4074c10 0x7fdeb4077030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.122+0000 7f74fea00640 1 -- 172.21.15.159:0/3595645589 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f74f408c8c0 con 0x7f74ec0051c0 2024-04-04T06:06:05.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.122+0000 7f74fea00640 1 --2- 172.21.15.159:0/3595645589 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x7f74e8074c10 0x7f74e8077030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.123+0000 7f74fea00640 1 -- 172.21.15.159:0/3595645589 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f74f4012ec0 con 0x7f74e8074c10 2024-04-04T06:06:05.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.123+0000 7f74fea00640 1 -- 172.21.15.159:0/3595645589 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f74f408ccb0 con 0x7f74ec0051c0 2024-04-04T06:06:05.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.123+0000 7f7506000640 1 --2- 172.21.15.159:0/3595645589 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f74e806c9f0 0x7f74e806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.123+0000 7fded1e00640 1 --2- 172.21.15.159:0/860740905 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdeb406c9f0 0x7fdeb406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.123+0000 7f7507400640 1 --2- 172.21.15.159:0/3595645589 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x7f74e8074c10 0x7f74e8077030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.124+0000 7fded2800640 1 --2- 172.21.15.159:0/860740905 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x7fdeb4074c10 0x7fdeb4077030 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.124+0000 7fded1e00640 1 --2- 172.21.15.159:0/860740905 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdeb406c9f0 0x7fdeb406eeb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009a30 tx=0x7fdec4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.124+0000 7f7507400640 1 --2- 172.21.15.159:0/3595645589 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x7f74e8074c10 0x7f74e8077030 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.125+0000 7f7506000640 1 --2- 172.21.15.159:0/3595645589 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f74e806c9f0 0x7f74e806eeb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f750815c740 tx=0x7f74f8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.126+0000 7fdeca000640 1 -- 172.21.15.159:0/860740905 <== osd.5 v2:172.21.15.178:6802/63746765 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdec0020e50 con 0x7fdeb4074c10 2024-04-04T06:06:05.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.127+0000 7f74fea00640 1 -- 172.21.15.159:0/3595645589 <== osd.4 v2:172.21.15.178:6812/4035991429 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f74f4012ec0 con 0x7f74e8074c10 2024-04-04T06:06:05.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.144+0000 7f3193200640 1 -- 172.21.15.159:0/2471495833 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f318c0074f0 con 0x7f3174074c90 2024-04-04T06:06:05.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.145+0000 7f318a000640 1 -- 172.21.15.159:0/2471495833 <== osd.3 v2:172.21.15.159:6820/2070760205 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f318c0074f0 con 0x7f3174074c90 2024-04-04T06:06:05.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.149+0000 7f3193200640 1 -- 172.21.15.159:0/2471495833 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x7f3174074c90 msgr2=0x7f31740770b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.149+0000 7f3193200640 1 --2- 172.21.15.159:0/2471495833 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x7f3174074c90 0x7f31740770b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.149+0000 7f3193200640 1 -- 172.21.15.159:0/2471495833 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f317406caf0 msgr2=0x7f317406efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.149+0000 7f3193200640 1 --2- 172.21.15.159:0/2471495833 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f317406caf0 0x7f317406efb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f318c180760 tx=0x7f3184009040 comp rx=0 tx=0).stop 2024-04-04T06:06:05.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.149+0000 7f3193200640 1 -- 172.21.15.159:0/2471495833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f318c0df800 msgr2=0x7f318c0e15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.149+0000 7f3193200640 1 --2- 172.21.15.159:0/2471495833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f318c0df800 0x7f318c0e15b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3194075e10 tx=0x7f319404ff60 comp rx=0 tx=0).stop 2024-04-04T06:06:05.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.149+0000 7f3192800640 1 -- 172.21.15.159:0/2471495833 reap_dead start 2024-04-04T06:06:05.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.150+0000 7f6c5cc00640 1 -- 172.21.15.159:0/3331700055 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6c38002dd0 con 0x7f6c44075760 2024-04-04T06:06:05.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.150+0000 7f3193200640 1 -- 172.21.15.159:0/2471495833 shutdown_connections 2024-04-04T06:06:05.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.150+0000 7f3193200640 1 -- 172.21.15.159:0/2471495833 >> 172.21.15.159:0/2471495833 conn(0x7f318c009fb0 msgr2=0x7f318c005dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.151+0000 7f0eaa800640 1 -- 172.21.15.159:0/2833251566 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0e9c054150 con 0x7f0e88074b60 2024-04-04T06:06:05.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.151+0000 7f3193200640 1 -- 172.21.15.159:0/2471495833 shutdown_connections 2024-04-04T06:06:05.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.151+0000 7f3193200640 1 -- 172.21.15.159:0/2471495833 wait complete. 2024-04-04T06:06:05.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.152+0000 7f6c5f400640 1 -- 172.21.15.159:0/3331700055 <== osd.1 v2:172.21.15.159:6816/3765338297 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6c38002dd0 con 0x7f6c44075760 2024-04-04T06:06:05.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.153+0000 7f6c5cc00640 1 -- 172.21.15.159:0/3331700055 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x7f6c44075760 msgr2=0x7f6c44077c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.153+0000 7f6c5cc00640 1 --2- 172.21.15.159:0/3331700055 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x7f6c44075760 0x7f6c44077c00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.153+0000 7f0ea1600640 1 -- 172.21.15.159:0/2833251566 <== osd.2 v2:172.21.15.159:6808/2619731047 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0e9c054150 con 0x7f0e88074b60 2024-04-04T06:06:05.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.153+0000 7f6c5cc00640 1 -- 172.21.15.159:0/3331700055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6c4406c8f0 msgr2=0x7f6c4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.153+0000 7f6c5cc00640 1 --2- 172.21.15.159:0/3331700055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6c4406c8f0 0x7f6c4406edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6c6815cea0 tx=0x7f6c58009040 comp rx=0 tx=0).stop 2024-04-04T06:06:05.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.153+0000 7f6c5cc00640 1 -- 172.21.15.159:0/3331700055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c6806f950 msgr2=0x7f6c68164d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.153+0000 7f6c5cc00640 1 --2- 172.21.15.159:0/3331700055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c6806f950 0x7f6c68164d40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6c50000c00 tx=0x7f6c500046c0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.153+0000 7f6c67e00640 1 -- 172.21.15.159:0/3331700055 reap_dead start 2024-04-04T06:06:05.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.153+0000 7f6c5cc00640 1 -- 172.21.15.159:0/3331700055 shutdown_connections 2024-04-04T06:06:05.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.153+0000 7f6c5cc00640 1 -- 172.21.15.159:0/3331700055 >> 172.21.15.159:0/3331700055 conn(0x7f6c68071e60 msgr2=0x7f6c68163050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.153+0000 7f7092600640 1 -- 172.21.15.159:0/1476810441 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7084033f70 con 0x7f706c074be0 2024-04-04T06:06:05.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.154+0000 7f6c5cc00640 1 -- 172.21.15.159:0/3331700055 shutdown_connections 2024-04-04T06:06:05.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.154+0000 7f0eaa800640 1 -- 172.21.15.159:0/2833251566 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x7f0e88074b60 msgr2=0x7f0e88076f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.154+0000 7f0eaa800640 1 --2- 172.21.15.159:0/2833251566 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x7f0e88074b60 0x7f0e88076f80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.154+0000 7f7088c00640 1 -- 172.21.15.159:0/1476810441 <== osd.0 v2:172.21.15.159:6800/455382518 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7084033f70 con 0x7f706c074be0 2024-04-04T06:06:05.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.154+0000 7f0eaa800640 1 -- 172.21.15.159:0/2833251566 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0e8806c940 msgr2=0x7f0e8806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.154+0000 7f0eaa800640 1 --2- 172.21.15.159:0/2833251566 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0e8806c940 0x7f0e8806ee00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c048b30 tx=0x7f0e98009040 comp rx=0 tx=0).stop 2024-04-04T06:06:05.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.154+0000 7f0eaa800640 1 -- 172.21.15.159:0/2833251566 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e9c00aaa0 msgr2=0x7f0e9c0485f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.154+0000 7f0eaa800640 1 --2- 172.21.15.159:0/2833251566 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e9c00aaa0 0x7f0e9c0485f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4063110 tx=0x7f0ea406ae20 comp rx=0 tx=0).stop 2024-04-04T06:06:05.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.154+0000 7fc7a6a00640 1 -- 172.21.15.159:0/175256534 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc798048390 con 0x7fc780074bc0 2024-04-04T06:06:05.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.154+0000 7f0ea9e00640 1 -- 172.21.15.159:0/2833251566 reap_dead start 2024-04-04T06:06:05.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.154+0000 7f6c5cc00640 1 -- 172.21.15.159:0/3331700055 wait complete. 2024-04-04T06:06:05.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.155+0000 7f0eaa800640 1 -- 172.21.15.159:0/2833251566 shutdown_connections 2024-04-04T06:06:05.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.155+0000 7f0eaa800640 1 -- 172.21.15.159:0/2833251566 >> 172.21.15.159:0/2833251566 conn(0x7f0e9c009fb0 msgr2=0x7f0e9c034d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.155+0000 7f0eaa800640 1 -- 172.21.15.159:0/2833251566 shutdown_connections 2024-04-04T06:06:05.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.155+0000 7f707e000640 1 -- 172.21.15.159:0/1476810441 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x7f706c074be0 msgr2=0x7f706c077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.155+0000 7f707e000640 1 --2- 172.21.15.159:0/1476810441 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x7f706c074be0 0x7f706c077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.155+0000 7f0eaa800640 1 -- 172.21.15.159:0/2833251566 wait complete. 2024-04-04T06:06:05.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.155+0000 7f707e000640 1 -- 172.21.15.159:0/1476810441 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f706c06c9c0 msgr2=0x7f706c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.155+0000 7f707e000640 1 --2- 172.21.15.159:0/1476810441 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f706c06c9c0 0x7f706c06ee80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7080036000 tx=0x7f708003c040 comp rx=0 tx=0).stop 2024-04-04T06:06:05.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.155+0000 7f707e000640 1 -- 172.21.15.159:0/1476810441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f708404ffd0 msgr2=0x7f70840070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.155+0000 7f707e000640 1 --2- 172.21.15.159:0/1476810441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f708404ffd0 0x7f70840070f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f707800d800 tx=0x7f707800dcc0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.155+0000 7f7091c00640 1 -- 172.21.15.159:0/1476810441 reap_dead start 2024-04-04T06:06:05.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.155+0000 7f707e000640 1 -- 172.21.15.159:0/1476810441 shutdown_connections 2024-04-04T06:06:05.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.155+0000 7f707e000640 1 -- 172.21.15.159:0/1476810441 >> 172.21.15.159:0/1476810441 conn(0x7f7084009fb0 msgr2=0x7f7084049720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.156+0000 7fc79d600640 1 -- 172.21.15.159:0/175256534 <== osd.6 v2:172.21.15.178:6826/198949157 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc798048390 con 0x7fc780074bc0 2024-04-04T06:06:05.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.156+0000 7f090a600640 1 -- 172.21.15.159:0/1553080322 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f08e0002dd0 con 0x7f08e4074a90 2024-04-04T06:06:05.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.156+0000 7f707e000640 1 -- 172.21.15.159:0/1476810441 shutdown_connections 2024-04-04T06:06:05.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.156+0000 7f707e000640 1 -- 172.21.15.159:0/1476810441 wait complete. 2024-04-04T06:06:05.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.156+0000 7fc7a6a00640 1 -- 172.21.15.159:0/175256534 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x7fc780074bc0 msgr2=0x7fc780076fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.156+0000 7fc7a6a00640 1 --2- 172.21.15.159:0/175256534 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x7fc780074bc0 0x7fc780076fe0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.157+0000 7fc7a6a00640 1 -- 172.21.15.159:0/175256534 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc78006c940 msgr2=0x7fc78006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.157+0000 7fc7a6a00640 1 --2- 172.21.15.159:0/175256534 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc78006c940 0x7fc78006ee00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc794006fd0 tx=0x7fc794008040 comp rx=0 tx=0).stop 2024-04-04T06:06:05.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.157+0000 7fc7a6a00640 1 -- 172.21.15.159:0/175256534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7980e3260 msgr2=0x7fc79804a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.157+0000 7fc7a6a00640 1 --2- 172.21.15.159:0/175256534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7980e3260 0x7fc79804a7e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc7a00511b0 tx=0x7fc7a006ae20 comp rx=0 tx=0).stop 2024-04-04T06:06:05.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.157+0000 7fc7a6000640 1 -- 172.21.15.159:0/175256534 reap_dead start 2024-04-04T06:06:05.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.157+0000 7fc7a6a00640 1 -- 172.21.15.159:0/175256534 shutdown_connections 2024-04-04T06:06:05.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.157+0000 7fc7a6a00640 1 -- 172.21.15.159:0/175256534 >> 172.21.15.159:0/175256534 conn(0x7fc798009fb0 msgr2=0x7fc798052c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.157+0000 7fc7a6a00640 1 -- 172.21.15.159:0/175256534 shutdown_connections 2024-04-04T06:06:05.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.157+0000 7fc7a6a00640 1 -- 172.21.15.159:0/175256534 wait complete. 2024-04-04T06:06:05.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.157+0000 7f0900c00640 1 -- 172.21.15.159:0/1553080322 <== osd.7 v2:172.21.15.178:6810/1066817938 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f08e0002dd0 con 0x7f08e4074a90 2024-04-04T06:06:05.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.158+0000 7f090a600640 1 -- 172.21.15.159:0/1553080322 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x7f08e4074a90 msgr2=0x7f08e4076eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.158+0000 7f090a600640 1 --2- 172.21.15.159:0/1553080322 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x7f08e4074a90 0x7f08e4076eb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.159+0000 7f090a600640 1 -- 172.21.15.159:0/1553080322 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f08e406c8f0 msgr2=0x7f08e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.159+0000 7f090a600640 1 --2- 172.21.15.159:0/1553080322 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f08e406c8f0 0x7f08e406edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f08f8006a90 tx=0x7f08f8006a20 comp rx=0 tx=0).stop 2024-04-04T06:06:05.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.159+0000 7f090a600640 1 -- 172.21.15.159:0/1553080322 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08fc053cb0 msgr2=0x7f08fc0e0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.159+0000 7f090a600640 1 --2- 172.21.15.159:0/1553080322 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08fc053cb0 0x7f08fc0e0220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f08f002b950 tx=0x7f08f00046c0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.159+0000 7f0909c00640 1 -- 172.21.15.159:0/1553080322 reap_dead start 2024-04-04T06:06:05.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.159+0000 7f090a600640 1 -- 172.21.15.159:0/1553080322 shutdown_connections 2024-04-04T06:06:05.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.159+0000 7f090a600640 1 -- 172.21.15.159:0/1553080322 >> 172.21.15.159:0/1553080322 conn(0x7f08fc058ee0 msgr2=0x7f08fc057240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.159+0000 7fded3200640 1 -- 172.21.15.159:0/860740905 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdecc0456f0 con 0x7fdeb4074c10 2024-04-04T06:06:05.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.160+0000 7f090a600640 1 -- 172.21.15.159:0/1553080322 shutdown_connections 2024-04-04T06:06:05.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.160+0000 7f090a600640 1 -- 172.21.15.159:0/1553080322 wait complete. 2024-04-04T06:06:05.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.160+0000 7fdeca000640 1 -- 172.21.15.159:0/860740905 <== osd.5 v2:172.21.15.178:6802/63746765 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdecc0456f0 con 0x7fdeb4074c10 2024-04-04T06:06:05.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.161+0000 7fded3200640 1 -- 172.21.15.159:0/860740905 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x7fdeb4074c10 msgr2=0x7fdeb4077030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.161+0000 7fded3200640 1 --2- 172.21.15.159:0/860740905 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x7fdeb4074c10 0x7fdeb4077030 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.162+0000 7fded3200640 1 -- 172.21.15.159:0/860740905 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdeb406c9f0 msgr2=0x7fdeb406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.162+0000 7fded3200640 1 --2- 172.21.15.159:0/860740905 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdeb406c9f0 0x7fdeb406eeb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009a30 tx=0x7fdec4008040 comp rx=0 tx=0).stop 2024-04-04T06:06:05.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.162+0000 7fded3200640 1 -- 172.21.15.159:0/860740905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdecc04ffd0 msgr2=0x7fdecc04e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.162+0000 7fded3200640 1 --2- 172.21.15.159:0/860740905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdecc04ffd0 0x7fdecc04e7f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdec000c4a0 tx=0x7fdec000c960 comp rx=0 tx=0).stop 2024-04-04T06:06:05.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.162+0000 7fded2800640 1 -- 172.21.15.159:0/860740905 reap_dead start 2024-04-04T06:06:05.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.162+0000 7fded3200640 1 -- 172.21.15.159:0/860740905 shutdown_connections 2024-04-04T06:06:05.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.162+0000 7fded3200640 1 -- 172.21.15.159:0/860740905 >> 172.21.15.159:0/860740905 conn(0x7fdecc009fb0 msgr2=0x7fdecc0e9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.163+0000 7f7507e00640 1 -- 172.21.15.159:0/3595645589 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f75080b8700 con 0x7f74e8074c10 2024-04-04T06:06:05.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.162+0000 7fded3200640 1 -- 172.21.15.159:0/860740905 shutdown_connections 2024-04-04T06:06:05.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.162+0000 7fded3200640 1 -- 172.21.15.159:0/860740905 wait complete. 2024-04-04T06:06:05.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.164+0000 7f74fea00640 1 -- 172.21.15.159:0/3595645589 <== osd.4 v2:172.21.15.178:6812/4035991429 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f75080b8700 con 0x7f74e8074c10 2024-04-04T06:06:05.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.165+0000 7f7507e00640 1 -- 172.21.15.159:0/3595645589 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x7f74e8074c10 msgr2=0x7f74e8077030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.165+0000 7f7507e00640 1 --2- 172.21.15.159:0/3595645589 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x7f74e8074c10 0x7f74e8077030 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.166+0000 7f7507e00640 1 -- 172.21.15.159:0/3595645589 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f74e806c9f0 msgr2=0x7f74e806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.166+0000 7f7507e00640 1 --2- 172.21.15.159:0/3595645589 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f74e806c9f0 0x7f74e806eeb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f750815c740 tx=0x7f74f8008040 comp rx=0 tx=0).stop 2024-04-04T06:06:05.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.166+0000 7f7507e00640 1 -- 172.21.15.159:0/3595645589 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74ec0051c0 msgr2=0x7f750815c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.166+0000 7f7507e00640 1 --2- 172.21.15.159:0/3595645589 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74ec0051c0 0x7f750815c790 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f74f4007fd0 tx=0x7f74f4007860 comp rx=0 tx=0).stop 2024-04-04T06:06:05.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.166+0000 7f7507400640 1 -- 172.21.15.159:0/3595645589 reap_dead start 2024-04-04T06:06:05.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.166+0000 7f7507e00640 1 -- 172.21.15.159:0/3595645589 shutdown_connections 2024-04-04T06:06:05.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.166+0000 7f7507e00640 1 -- 172.21.15.159:0/3595645589 >> 172.21.15.159:0/3595645589 conn(0x7f75080678b0 msgr2=0x7f75080d1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.169+0000 7f7507e00640 1 -- 172.21.15.159:0/3595645589 shutdown_connections 2024-04-04T06:06:05.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.169+0000 7f7507e00640 1 -- 172.21.15.159:0/3595645589 wait complete. 2024-04-04T06:06:05.197 INFO:teuthology.orchestra.run.smithi159.stdout:64424509449 2024-04-04T06:06:05.198 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-04T06:06:05.198 INFO:teuthology.orchestra.run.smithi159.stdout:64424509447 2024-04-04T06:06:05.199 INFO:teuthology.orchestra.run.smithi159.stdout:64424509448 2024-04-04T06:06:05.200 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-04T06:06:05.200 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-04T06:06:05.201 INFO:teuthology.orchestra.run.smithi159.stdout:64424509449 2024-04-04T06:06:05.201 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-04T06:06:05.202 INFO:teuthology.orchestra.run.smithi159.stdout:64424509446 2024-04-04T06:06:05.202 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-04T06:06:05.203 INFO:teuthology.orchestra.run.smithi159.stdout:64424509448 2024-04-04T06:06:05.203 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-04T06:06:05.204 INFO:teuthology.orchestra.run.smithi159.stdout:60129542152 2024-04-04T06:06:05.204 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-04T06:06:05.205 INFO:teuthology.orchestra.run.smithi159.stdout:60129542150 2024-04-04T06:06:05.206 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-04T06:06:05.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.534+0000 7fcc7ca00640 1 Processor -- start 2024-04-04T06:06:05.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.534+0000 7f5267e00640 1 Processor -- start 2024-04-04T06:06:05.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.534+0000 7fcc7ca00640 1 -- start start 2024-04-04T06:06:05.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.534+0000 7f5267e00640 1 -- start start 2024-04-04T06:06:05.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.536+0000 7fcc7ca00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc7815b4e0 0x7fcc780b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.536+0000 7fcc7ca00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcc780afee0 0x7fcc7815eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.536+0000 7fcc7ca00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fcc7815f530 0x7fcc780ae700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.536+0000 7fcc7ca00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fcc780b5810 con 0x7fcc7815f530 2024-04-04T06:06:05.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.536+0000 7fcc7ca00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fcc7815b8c0 con 0x7fcc780afee0 2024-04-04T06:06:05.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.536+0000 7fcc7ca00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcc780afab0 con 0x7fcc7815b4e0 2024-04-04T06:06:05.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.537+0000 7fcc76800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fcc7815f530 0x7fcc780ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.537+0000 7fcc76800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fcc7815f530 0x7fcc780ae700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36994/0 (socket says 172.21.15.159:36994) 2024-04-04T06:06:05.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.537+0000 7fcc76800640 1 -- 172.21.15.159:0/1563380770 learned_addr learned my addr 172.21.15.159:0/1563380770 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.537+0000 7fcc75e00640 1 --2- 172.21.15.159:0/1563380770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc7815b4e0 0x7fcc780b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.537+0000 7fcc75400640 1 --2- 172.21.15.159:0/1563380770 >> v2:172.21.15.178:3300/0 conn(0x7fcc780afee0 0x7fcc7815eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.537+0000 7fcc76800640 1 -- 172.21.15.159:0/1563380770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc7815b4e0 msgr2=0x7fcc780b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.537+0000 7fcc76800640 1 --2- 172.21.15.159:0/1563380770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc7815b4e0 0x7fcc780b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.537+0000 7fcc76800640 1 -- 172.21.15.159:0/1563380770 >> v2:172.21.15.178:3300/0 conn(0x7fcc780afee0 msgr2=0x7fcc7815eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.537+0000 7fcc76800640 1 --2- 172.21.15.159:0/1563380770 >> v2:172.21.15.178:3300/0 conn(0x7fcc780afee0 0x7fcc7815eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.537+0000 7fcc76800640 1 -- 172.21.15.159:0/1563380770 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7815fd70 con 0x7fcc7815f530 2024-04-04T06:06:05.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.538+0000 7f5267e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f526015e950 0x7f52600b3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.538+0000 7f5267e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5260073520 0x7f52600b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.538+0000 7f5267e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f52600ae150 0x7f52600ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.538+0000 7f5267e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f526015ed30 con 0x7f52600ae150 2024-04-04T06:06:05.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.538+0000 7f5267e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f52600ad0b0 con 0x7f526015e950 2024-04-04T06:06:05.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.538+0000 7f5267e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f52600ad390 con 0x7f5260073520 2024-04-04T06:06:05.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.539+0000 7f5265e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f52600ae150 0x7f52600ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.539+0000 7f5265400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f526015e950 0x7f52600b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.539+0000 7f5264a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5260073520 0x7f52600b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.539+0000 7f5265400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f526015e950 0x7f52600b3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:41070/0 (socket says 172.21.15.159:41070) 2024-04-04T06:06:05.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.539+0000 7f5265400640 1 -- 172.21.15.159:0/504487287 learned_addr learned my addr 172.21.15.159:0/504487287 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.539+0000 7fcc76800640 1 --2- 172.21.15.159:0/1563380770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc7815f530 0x7fcc780ae700 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcc7806d660 tx=0x7fcc6802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c9fa434775579d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.540+0000 7f5265e00640 1 -- 172.21.15.159:0/504487287 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5260073520 msgr2=0x7f52600b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.540+0000 7f5265e00640 1 --2- 172.21.15.159:0/504487287 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5260073520 0x7f52600b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.540+0000 7fcc74a00640 1 -- 172.21.15.159:0/1563380770 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc68035020 con 0x7fcc7815f530 2024-04-04T06:06:05.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.540+0000 7fcc74a00640 1 -- 172.21.15.159:0/1563380770 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc68031640 con 0x7fcc7815f530 2024-04-04T06:06:05.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.540+0000 7fcc74a00640 1 -- 172.21.15.159:0/1563380770 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6802d040 con 0x7fcc7815f530 2024-04-04T06:06:05.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.540+0000 7f5265e00640 1 -- 172.21.15.159:0/504487287 >> v2:172.21.15.178:3300/0 conn(0x7f526015e950 msgr2=0x7f52600b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.540+0000 7f5265e00640 1 --2- 172.21.15.159:0/504487287 >> v2:172.21.15.178:3300/0 conn(0x7f526015e950 0x7f52600b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.540+0000 7f5265e00640 1 -- 172.21.15.159:0/504487287 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52600ad670 con 0x7f52600ae150 2024-04-04T06:06:05.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.541+0000 7fcc7ca00640 1 -- 172.21.15.159:0/1563380770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc7815f530 msgr2=0x7fcc780ae700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.541+0000 7fcc7ca00640 1 --2- 172.21.15.159:0/1563380770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc7815f530 0x7fcc780ae700 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcc7806d660 tx=0x7fcc6802b8f0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.542+0000 7fcc7ca00640 1 -- 172.21.15.159:0/1563380770 shutdown_connections 2024-04-04T06:06:05.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.542+0000 7fcc7ca00640 1 --2- 172.21.15.159:0/1563380770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc7815f530 0x7fcc780ae700 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.542+0000 7fcc7ca00640 1 --2- 172.21.15.159:0/1563380770 >> v2:172.21.15.178:3300/0 conn(0x7fcc780afee0 0x7fcc7815eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.542+0000 7fcc7ca00640 1 --2- 172.21.15.159:0/1563380770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc7815b4e0 0x7fcc780b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.542+0000 7fcc7ca00640 1 -- 172.21.15.159:0/1563380770 >> 172.21.15.159:0/1563380770 conn(0x7fcc780b6560 msgr2=0x7fcc780b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.541+0000 7f5265e00640 1 --2- 172.21.15.159:0/504487287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52600ae150 0x7f52600ae5e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f52600691b0 tx=0x7f525802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7f02f0993ba268 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.542+0000 7f525fe00640 1 -- 172.21.15.159:0/504487287 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258039070 con 0x7f52600ae150 2024-04-04T06:06:05.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.542+0000 7fcc7ca00640 1 -- 172.21.15.159:0/1563380770 shutdown_connections 2024-04-04T06:06:05.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.543+0000 7f525fe00640 1 -- 172.21.15.159:0/504487287 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f525802cbe0 con 0x7f52600ae150 2024-04-04T06:06:05.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.543+0000 7f525fe00640 1 -- 172.21.15.159:0/504487287 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258034440 con 0x7f52600ae150 2024-04-04T06:06:05.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.544+0000 7f5267e00640 1 -- 172.21.15.159:0/504487287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52600ae150 msgr2=0x7f52600ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.544+0000 7f5267e00640 1 --2- 172.21.15.159:0/504487287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52600ae150 0x7f52600ae5e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f52600691b0 tx=0x7f525802be50 comp rx=0 tx=0).stop 2024-04-04T06:06:05.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.544+0000 7f5267e00640 1 -- 172.21.15.159:0/504487287 shutdown_connections 2024-04-04T06:06:05.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.542+0000 7fcc7ca00640 1 -- 172.21.15.159:0/1563380770 wait complete. 2024-04-04T06:06:05.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.544+0000 7f5267e00640 1 --2- 172.21.15.159:0/504487287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52600ae150 0x7f52600ae5e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.544+0000 7f5267e00640 1 --2- 172.21.15.159:0/504487287 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5260073520 0x7f52600b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.545+0000 7f5267e00640 1 --2- 172.21.15.159:0/504487287 >> v2:172.21.15.178:3300/0 conn(0x7f526015e950 0x7f52600b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.545+0000 7f5267e00640 1 -- 172.21.15.159:0/504487287 >> 172.21.15.159:0/504487287 conn(0x7f5260072a00 msgr2=0x7f5260072e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.545+0000 7f5267e00640 1 -- 172.21.15.159:0/504487287 shutdown_connections 2024-04-04T06:06:05.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.545+0000 7f5267e00640 1 -- 172.21.15.159:0/504487287 wait complete. 2024-04-04T06:06:05.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.545+0000 7fcc7ca00640 1 Processor -- start 2024-04-04T06:06:05.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.545+0000 7fcc7ca00640 1 -- start start 2024-04-04T06:06:05.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.547+0000 7fcc7ca00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc780afee0 0x7fcc7815a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.547+0000 7f0b1be00640 1 Processor -- start 2024-04-04T06:06:05.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.548+0000 7fcc7ca00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc7815b4e0 0x7fcc782009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.548+0000 7f0b1be00640 1 -- start start 2024-04-04T06:06:05.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.547+0000 7f5267e00640 1 Processor -- start 2024-04-04T06:06:05.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.547+0000 7f5267e00640 1 -- start start 2024-04-04T06:06:05.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.548+0000 7fcc7ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc7815f530 0x7fcc780b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.548+0000 7fcc75e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc780afee0 0x7fcc7815a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.548+0000 7fcc75e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc780afee0 0x7fcc7815a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:55074/0 (socket says 172.21.15.159:55074) 2024-04-04T06:06:05.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.549+0000 7fcc75400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc7815b4e0 0x7fcc782009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.549+0000 7fcc75400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc7815b4e0 0x7fcc782009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:41074/0 (socket says 172.21.15.159:41074) 2024-04-04T06:06:05.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.549+0000 7f5267e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5260073520 0x7f52600cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.549+0000 7f5267e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52600ae150 0x7f5260144710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.549+0000 7f5267e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f526015e950 0x7f5260144c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.549+0000 7f5267e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f52600c93b0 con 0x7f526015e950 2024-04-04T06:06:05.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.549+0000 7f5267e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f52600adc30 con 0x7f5260073520 2024-04-04T06:06:05.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.549+0000 7f5267e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f52600aeb20 con 0x7f52600ae150 2024-04-04T06:06:05.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.549+0000 7fcc75e00640 1 -- 172.21.15.159:0/2352115872 learned_addr learned my addr 172.21.15.159:0/2352115872 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.549+0000 7f5265400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5260073520 0x7f52600cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.549+0000 7f5265400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5260073520 0x7f52600cbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:41076/0 (socket says 172.21.15.159:41076) 2024-04-04T06:06:05.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.549+0000 7f5265400640 1 -- 172.21.15.159:0/3276464247 learned_addr learned my addr 172.21.15.159:0/3276464247 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.550+0000 7f5265e00640 1 --2- 172.21.15.159:0/3276464247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f526015e950 0x7f5260144c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.550+0000 7f0b1be00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0b140b3b70 0x7f0b140d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.550+0000 7f5264a00640 1 --2- 172.21.15.159:0/3276464247 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52600ae150 0x7f5260144710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.550+0000 7f7b86600640 1 Processor -- start 2024-04-04T06:06:05.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.550+0000 7f0b1be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0b140b1530 0x7f0b1415c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.550+0000 7f7b86600640 1 -- start start 2024-04-04T06:06:05.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.551+0000 7f0b19e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0b140b1530 0x7f0b1415c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.551+0000 7f5265400640 1 -- 172.21.15.159:0/3276464247 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52600ae150 msgr2=0x7f5260144710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.551+0000 7f0b19e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0b140b1530 0x7f0b1415c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37038/0 (socket says 172.21.15.159:37038) 2024-04-04T06:06:05.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.551+0000 7f0b19e00640 1 -- 172.21.15.159:0/3351519768 learned_addr learned my addr 172.21.15.159:0/3351519768 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.551+0000 7f5265400640 1 --2- 172.21.15.159:0/3276464247 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52600ae150 0x7f5260144710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.551+0000 7f5265400640 1 -- 172.21.15.159:0/3276464247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f526015e950 msgr2=0x7f5260144c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.551+0000 7f5265400640 1 --2- 172.21.15.159:0/3276464247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f526015e950 0x7f5260144c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.551+0000 7f0b18a00640 1 --2- 172.21.15.159:0/3351519768 >> v2:172.21.15.178:3300/0 conn(0x7f0b140b3b70 0x7f0b140d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.551+0000 7f5265400640 1 -- 172.21.15.159:0/3276464247 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5260154d70 con 0x7f5260073520 2024-04-04T06:06:05.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.551+0000 7f5265e00640 1 --2- 172.21.15.159:0/3276464247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f526015e950 0x7f5260144c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:06:05.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.551+0000 7fcc7ca00640 1 -- 172.21.15.159:0/2352115872 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcc780b5010 con 0x7fcc7815f530 2024-04-04T06:06:05.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.552+0000 7f5265400640 1 --2- 172.21.15.159:0/3276464247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5260073520 0x7f52600cbd70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f525400d7d0 tx=0x7f525400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.551+0000 7fcc7ca00640 1 -- 172.21.15.159:0/2352115872 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcc780b4d30 con 0x7fcc7815b4e0 2024-04-04T06:06:05.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.551+0000 7fcc7ca00640 1 -- 172.21.15.159:0/2352115872 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcc780b52f0 con 0x7fcc780afee0 2024-04-04T06:06:05.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.551+0000 7fcc76800640 1 --2- 172.21.15.159:0/2352115872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc7815f530 0x7fcc780b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.552+0000 7fcc76800640 1 -- 172.21.15.159:0/2352115872 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc780afee0 msgr2=0x7fcc7815a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.552+0000 7fcc76800640 1 --2- 172.21.15.159:0/2352115872 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc780afee0 0x7fcc7815a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.552+0000 7fcc76800640 1 -- 172.21.15.159:0/2352115872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc7815b4e0 msgr2=0x7fcc782009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.552+0000 7fcc76800640 1 --2- 172.21.15.159:0/2352115872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc7815b4e0 0x7fcc782009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.552+0000 7fcc76800640 1 -- 172.21.15.159:0/2352115872 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7815bc50 con 0x7fcc7815f530 2024-04-04T06:06:05.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.552+0000 7fcc75400640 1 --2- 172.21.15.159:0/2352115872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc7815b4e0 0x7fcc782009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:05.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.552+0000 7fcc75e00640 1 --2- 172.21.15.159:0/2352115872 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc780afee0 0x7fcc7815a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:05.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.552+0000 7fcc76800640 1 --2- 172.21.15.159:0/2352115872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc7815f530 0x7fcc780b1ce0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcc68033000 tx=0x7fcc680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.553+0000 7f525d600640 1 -- 172.21.15.159:0/3276464247 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5254018020 con 0x7f5260073520 2024-04-04T06:06:05.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.553+0000 7f0b1be00640 1 -- 172.21.15.159:0/3351519768 --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b1415b240 con 0x7f0b1415de50 2024-04-04T06:06:05.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.553+0000 7f525d600640 1 -- 172.21.15.159:0/3276464247 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52540096a0 con 0x7f5260073520 2024-04-04T06:06:05.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.553+0000 7f525d600640 1 -- 172.21.15.159:0/3276464247 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5254009a90 con 0x7f5260073520 2024-04-04T06:06:05.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.552+0000 7f7b86600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7b8015e860 0x7f7b8006d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.552+0000 7f7b86600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7b8006da30 0x7f7b800d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.553+0000 7f7b86600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b8006dfa0 con 0x7f7b800d4910 2024-04-04T06:06:05.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.553+0000 7f7b86600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f7b800cf220 con 0x7f7b8015e860 2024-04-04T06:06:05.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.553+0000 7f7b86600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7b800d4cf0 con 0x7f7b8006da30 2024-04-04T06:06:05.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.553+0000 7fcc6e000640 1 -- 172.21.15.159:0/2352115872 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc68035040 con 0x7fcc7815f530 2024-04-04T06:06:05.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.553+0000 7f5267e00640 1 -- 172.21.15.159:0/3276464247 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5260155050 con 0x7f5260073520 2024-04-04T06:06:05.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.553+0000 7fcc7ca00640 1 -- 172.21.15.159:0/2352115872 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc78201420 con 0x7fcc7815f530 2024-04-04T06:06:05.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.553+0000 7fcc7ca00640 1 -- 172.21.15.159:0/2352115872 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc7815bf10 con 0x7fcc7815f530 2024-04-04T06:06:05.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.553+0000 7f7b7fe00640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f7b800d4910 0x7f7b800d0e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.159:44226/0 (socket says 172.21.15.159:44226) 2024-04-04T06:06:05.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.554+0000 7f7b7fe00640 1 -- 172.21.15.159:0/590126719 learned_addr learned my addr 172.21.15.159:0/590126719 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:05.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.554+0000 7f7b7ea00640 1 --2- 172.21.15.159:0/590126719 >> v2:172.21.15.178:3300/0 conn(0x7f7b8006da30 0x7f7b800d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.553+0000 7f0b1be00640 1 -- 172.21.15.159:0/3351519768 --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f0b1415e230 con 0x7f0b140b1530 2024-04-04T06:06:05.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.553+0000 7f0b1be00640 1 -- 172.21.15.159:0/3351519768 --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0b1415b520 con 0x7f0b140b3b70 2024-04-04T06:06:05.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.554+0000 7f970b200640 1 Processor -- start 2024-04-04T06:06:05.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.554+0000 7f5267e00640 1 -- 172.21.15.159:0/3276464247 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f526015f870 con 0x7f5260073520 2024-04-04T06:06:05.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.554+0000 7f7b7f400640 1 --2- 172.21.15.159:0/590126719 >> v2:172.21.15.159:3300/0 conn(0x7f7b8015e860 0x7f7b8006d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.554+0000 7f0b18a00640 1 -- 172.21.15.159:0/3351519768 >> v2:172.21.15.159:3300/0 conn(0x7f0b140b1530 msgr2=0x7f0b1415c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.554+0000 7f0b18a00640 1 --2- 172.21.15.159:0/3351519768 >> v2:172.21.15.159:3300/0 conn(0x7f0b140b1530 0x7f0b1415c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.554+0000 7f0b18a00640 1 -- 172.21.15.159:0/3351519768 >> v1:172.21.15.159:6789/0 conn(0x7f0b1415de50 legacy=0x7f0b140b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.554+0000 7f0b18a00640 1 -- 172.21.15.159:0/3351519768 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1415b800 con 0x7f0b140b3b70 2024-04-04T06:06:05.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.554+0000 7fcc6e000640 1 -- 172.21.15.159:0/2352115872 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc68031640 con 0x7fcc7815f530 2024-04-04T06:06:05.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.554+0000 7fcc6e000640 1 -- 172.21.15.159:0/2352115872 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc68039b70 con 0x7fcc7815f530 2024-04-04T06:06:05.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.554+0000 7f970b200640 1 -- start start 2024-04-04T06:06:05.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.555+0000 7f7b7e000640 1 -- 172.21.15.159:0/590126719 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 791431086 0 0) 0x7f7b8006dfa0 con 0x7f7b800d4910 2024-04-04T06:06:05.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.555+0000 7f7b7e000640 1 -- 172.21.15.159:0/590126719 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b64003520 con 0x7f7b800d4910 2024-04-04T06:06:05.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.555+0000 7f7b7ea00640 1 -- 172.21.15.159:0/590126719 >> v2:172.21.15.159:3300/0 conn(0x7f7b8015e860 msgr2=0x7f7b8006d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.555+0000 7f7b7ea00640 1 --2- 172.21.15.159:0/590126719 >> v2:172.21.15.159:3300/0 conn(0x7f7b8015e860 0x7f7b8006d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.555+0000 7f7b7ea00640 1 -- 172.21.15.159:0/590126719 >> v1:172.21.15.178:6789/0 conn(0x7f7b800d4910 legacy=0x7f7b800d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.556+0000 7f7b7ea00640 1 -- 172.21.15.159:0/590126719 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8006e240 con 0x7f7b8006da30 2024-04-04T06:06:05.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.556+0000 7f5267e00640 1 -- 172.21.15.159:0/3276464247 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52600685c0 con 0x7f5260073520 2024-04-04T06:06:05.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.556+0000 7f0b18a00640 1 --2- 172.21.15.159:0/3351519768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b140b3b70 0x7f0b140d0990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0b140691b0 tx=0x7f0b0002ff90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6a8d1308fdc0841 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.557+0000 7f0b13e00640 1 -- 172.21.15.159:0/3351519768 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0003d070 con 0x7f0b140b3b70 2024-04-04T06:06:05.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.557+0000 7f970b200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f97040e30c0 0x7f9704057d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.557+0000 7f0b13e00640 1 -- 172.21.15.159:0/3351519768 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b00030d40 con 0x7f0b140b3b70 2024-04-04T06:06:05.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.557+0000 7f0b13e00640 1 -- 172.21.15.159:0/3351519768 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00038660 con 0x7f0b140b3b70 2024-04-04T06:06:05.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.557+0000 7f970b200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9704054300 0x7f97040068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.557+0000 7f970b200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97040582d0 con 0x7f970403e840 2024-04-04T06:06:05.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.557+0000 7f970b200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f9704058570 con 0x7f97040e30c0 2024-04-04T06:06:05.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.557+0000 7f970b200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9704058850 con 0x7f9704054300 2024-04-04T06:06:05.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.558+0000 7f9709e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f97040e30c0 0x7f9704057d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.558+0000 7f970a800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9704054300 0x7f97040068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.558+0000 7f8d3e200640 1 Processor -- start 2024-04-04T06:06:05.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.558+0000 7f525d600640 1 -- 172.21.15.159:0/3276464247 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f525401b020 con 0x7f5260073520 2024-04-04T06:06:05.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.558+0000 7f7b7ea00640 1 --2- 172.21.15.159:0/590126719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b8006da30 0x7f7b800d43d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7b80162160 tx=0x7f7b7002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fb919f763cfe65 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.558+0000 7f8d3e200640 1 -- start start 2024-04-04T06:06:05.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.558+0000 7f9709e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f97040e30c0 0x7f9704057d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:41114/0 (socket says 172.21.15.159:41114) 2024-04-04T06:06:05.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.558+0000 7f9709e00640 1 -- 172.21.15.159:0/1247813474 learned_addr learned my addr 172.21.15.159:0/1247813474 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.557+0000 7fcc6e000640 1 -- 172.21.15.159:0/2352115872 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc68038020 con 0x7fcc7815f530 2024-04-04T06:06:05.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.558+0000 7fcc7ca00640 1 -- 172.21.15.159:0/2352115872 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc50005290 con 0x7fcc7815f530 2024-04-04T06:06:05.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.559+0000 7f0b1be00640 1 -- 172.21.15.159:0/3351519768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b140b3b70 msgr2=0x7f0b140d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.559+0000 7f0b1be00640 1 --2- 172.21.15.159:0/3351519768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b140b3b70 0x7f0b140d0990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0b140691b0 tx=0x7f0b0002ff90 comp rx=0 tx=0).stop 2024-04-04T06:06:05.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.559+0000 7f0b1be00640 1 -- 172.21.15.159:0/3351519768 shutdown_connections 2024-04-04T06:06:05.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.559+0000 7f0b1be00640 1 --2- 172.21.15.159:0/3351519768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b140b3b70 0x7f0b140d0990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.559+0000 7f0b1be00640 1 --2- 172.21.15.159:0/3351519768 >> v2:172.21.15.159:3300/0 conn(0x7f0b140b1530 0x7f0b1415c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.559+0000 7f0b1be00640 1 -- 172.21.15.159:0/3351519768 >> 172.21.15.159:0/3351519768 conn(0x7f0b14072a00 msgr2=0x7f0b14072e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.559+0000 7f9708a00640 1 -- 172.21.15.159:0/1247813474 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1028942919 0 0) 0x7f97040582d0 con 0x7f970403e840 2024-04-04T06:06:05.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.559+0000 7f9708a00640 1 -- 172.21.15.159:0/1247813474 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96f0003520 con 0x7f970403e840 2024-04-04T06:06:05.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.560+0000 7f7a69a00640 1 Processor -- start 2024-04-04T06:06:05.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.559+0000 7f525d600640 1 --2- 172.21.15.159:0/3276464247 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f524006c8f0 0x7f524006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.559+0000 7f525d600640 1 -- 172.21.15.159:0/3276464247 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f525408c1a0 con 0x7f5260073520 2024-04-04T06:06:05.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.560+0000 7fcc6e000640 1 --2- 172.21.15.159:0/2352115872 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcc5806c8f0 0x7fcc5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.560+0000 7f8d3e200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d3815e860 0x7f8d3806d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.560+0000 7f8d3e200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f8d3806da30 0x7f8d380d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.560+0000 7f8d3e200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8d380d4910 0x7f8d380d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.560+0000 7f8d3e200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f8d380d4d60 con 0x7f8d3806da30 2024-04-04T06:06:05.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.560+0000 7f8d3e200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8d380cf220 con 0x7f8d380d4910 2024-04-04T06:06:05.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.560+0000 7f8d3e200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8d3806dfa0 con 0x7f8d3815e860 2024-04-04T06:06:05.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.560+0000 7f0b1be00640 1 -- 172.21.15.159:0/3351519768 shutdown_connections 2024-04-04T06:06:05.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.559+0000 7f9709e00640 1 -- 172.21.15.159:0/1247813474 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9704054300 msgr2=0x7f97040068d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.559+0000 7f9709e00640 1 --2- 172.21.15.159:0/1247813474 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9704054300 0x7f97040068d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.559+0000 7f9709e00640 1 -- 172.21.15.159:0/1247813474 >> v1:172.21.15.159:6789/0 conn(0x7f970403e840 legacy=0x7f9704053bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.559+0000 7f9709e00640 1 -- 172.21.15.159:0/1247813474 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9704058b30 con 0x7f97040e30c0 2024-04-04T06:06:05.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.561+0000 7f8d37400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d3815e860 0x7f8d3806d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.561+0000 7f8d37400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d3815e860 0x7f8d3806d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:55106/0 (socket says 172.21.15.159:55106) 2024-04-04T06:06:05.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.561+0000 7f8d37400640 1 -- 172.21.15.159:0/4232365460 learned_addr learned my addr 172.21.15.159:0/4232365460 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.561+0000 7fcc6e000640 1 -- 172.21.15.159:0/2352115872 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcc680ad7b0 con 0x7fcc7815f530 2024-04-04T06:06:05.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.561+0000 7fcc75e00640 1 --2- 172.21.15.159:0/2352115872 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcc5806c8f0 0x7fcc5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.560+0000 7f0b1be00640 1 -- 172.21.15.159:0/3351519768 wait complete. 2024-04-04T06:06:05.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.561+0000 7f8d37e00640 1 --2- 172.21.15.159:0/4232365460 >> v2:172.21.15.178:3300/0 conn(0x7f8d380d4910 0x7f8d380d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.561+0000 7f7a69a00640 1 -- start start 2024-04-04T06:06:05.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.561+0000 7f8d37400640 1 -- 172.21.15.159:0/4232365460 >> v2:172.21.15.178:3300/0 conn(0x7f8d380d4910 msgr2=0x7f8d380d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.561+0000 7f8d37400640 1 --2- 172.21.15.159:0/4232365460 >> v2:172.21.15.178:3300/0 conn(0x7f8d380d4910 0x7f8d380d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.561+0000 7f9709e00640 1 --2- 172.21.15.159:0/1247813474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97040e30c0 0x7f9704057d90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009870 tx=0x7f96fc02ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8968b5ab4d3d5ceb server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.561+0000 7f8d37400640 1 -- 172.21.15.159:0/4232365460 >> v2:172.21.15.159:3300/0 conn(0x7f8d3806da30 msgr2=0x7f8d380d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.561+0000 7f8d37400640 1 --2- 172.21.15.159:0/4232365460 >> v2:172.21.15.159:3300/0 conn(0x7f8d3806da30 0x7f8d380d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.561+0000 7f8d37400640 1 -- 172.21.15.159:0/4232365460 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d3806e240 con 0x7f8d3815e860 2024-04-04T06:06:05.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.562+0000 7f115ec00640 1 Processor -- start 2024-04-04T06:06:05.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.562+0000 7fcc75e00640 1 --2- 172.21.15.159:0/2352115872 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcc5806c8f0 0x7fcc5806edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009b80 tx=0x7fcc64008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.562+0000 7f5264a00640 1 --2- 172.21.15.159:0/3276464247 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f524006c8f0 0x7f524006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.562+0000 7f7a69a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7a640d3e40 0x7f7a640b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.562+0000 7f7a69a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a640cef20 0x7f7a640cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.562+0000 7f7a69a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a640d2da0 con 0x7f7a6415e460 2024-04-04T06:06:05.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.562+0000 7f7a69a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f7a640cf8d0 con 0x7f7a640d3e40 2024-04-04T06:06:05.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.562+0000 7f7a69a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7a640b8320 con 0x7f7a640cef20 2024-04-04T06:06:05.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.562+0000 7f115ec00640 1 -- start start 2024-04-04T06:06:05.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.563+0000 7f7a63400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7a640d3e40 0x7f7a640b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.563+0000 7f7a63400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7a640d3e40 0x7f7a640b9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37068/0 (socket says 172.21.15.159:37068) 2024-04-04T06:06:05.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.563+0000 7f7a63400640 1 -- 172.21.15.159:0/2714614512 learned_addr learned my addr 172.21.15.159:0/2714614512 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.563+0000 7f8d37400640 1 --2- 172.21.15.159:0/4232365460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d3815e860 0x7f8d3806d4f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8d380d5520 tx=0x7f8d2802b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e19d9429da61ca0e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.563+0000 7f5264a00640 1 --2- 172.21.15.159:0/3276464247 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f524006c8f0 0x7f524006edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5260161e90 tx=0x7f524c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.563+0000 7f7a63400640 1 -- 172.21.15.159:0/2714614512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a640cef20 msgr2=0x7f7a640cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.563+0000 7f7a63400640 1 --2- 172.21.15.159:0/2714614512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a640cef20 0x7f7a640cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.563+0000 7f7a63400640 1 -- 172.21.15.159:0/2714614512 >> v1:172.21.15.178:6789/0 conn(0x7f7a6415e460 legacy=0x7f7a640d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.563+0000 7f7a63400640 1 -- 172.21.15.159:0/2714614512 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a640d3080 con 0x7f7a640d3e40 2024-04-04T06:06:05.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.563+0000 7f0b1be00640 1 Processor -- start 2024-04-04T06:06:05.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.564+0000 7f8d36000640 1 -- 172.21.15.159:0/4232365460 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28035020 con 0x7f8d3815e860 2024-04-04T06:06:05.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.564+0000 7f115ec00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f1158162920 0x7f11581670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.564+0000 7f115ec00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1158163ac0 0x7f1158071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.564+0000 7f8d36000640 1 -- 172.21.15.159:0/4232365460 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d28031870 con 0x7f8d3815e860 2024-04-04T06:06:05.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.564+0000 7f8d36000640 1 -- 172.21.15.159:0/4232365460 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28031c40 con 0x7f8d3815e860 2024-04-04T06:06:05.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.564+0000 7f115ec00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1158071d60 con 0x7f11580d0280 2024-04-04T06:06:05.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.564+0000 7f115ec00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f1158163f20 con 0x7f1158162920 2024-04-04T06:06:05.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.564+0000 7f115ec00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f11581665d0 con 0x7f1158163ac0 2024-04-04T06:06:05.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.564+0000 7f115cc00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f11580d0280 0x7f11580ad010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45956/0 (socket says 172.21.15.159:45956) 2024-04-04T06:06:05.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.564+0000 7f115cc00640 1 -- 172.21.15.159:0/2702164315 learned_addr learned my addr 172.21.15.159:0/2702164315 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:05.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.564+0000 7f1157e00640 1 --2- 172.21.15.159:0/2702164315 >> v2:172.21.15.159:3300/0 conn(0x7f1158162920 0x7f11581670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.564+0000 7f1157400640 1 --2- 172.21.15.159:0/2702164315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1158163ac0 0x7f1158071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.565+0000 7f1157e00640 1 -- 172.21.15.159:0/2702164315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1158163ac0 msgr2=0x7f1158071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.565+0000 7f8d3e200640 1 -- 172.21.15.159:0/4232365460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d3815e860 msgr2=0x7f8d3806d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.565+0000 7f8d3e200640 1 --2- 172.21.15.159:0/4232365460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d3815e860 0x7f8d3806d4f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8d380d5520 tx=0x7f8d2802b910 comp rx=0 tx=0).stop 2024-04-04T06:06:05.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.565+0000 7f1157e00640 1 --2- 172.21.15.159:0/2702164315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1158163ac0 0x7f1158071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.565+0000 7f1157e00640 1 -- 172.21.15.159:0/2702164315 >> v1:172.21.15.159:6789/0 conn(0x7f11580d0280 legacy=0x7f11580ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:05.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.565+0000 7f1157e00640 1 -- 172.21.15.159:0/2702164315 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11581668b0 con 0x7f1158162920 2024-04-04T06:06:05.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.565+0000 7f8d3e200640 1 -- 172.21.15.159:0/4232365460 shutdown_connections 2024-04-04T06:06:05.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.565+0000 7f8d3e200640 1 --2- 172.21.15.159:0/4232365460 >> v2:172.21.15.178:3300/0 conn(0x7f8d380d4910 0x7f8d380d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.565+0000 7f8d3e200640 1 --2- 172.21.15.159:0/4232365460 >> v2:172.21.15.159:3300/0 conn(0x7f8d3806da30 0x7f8d380d43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.565+0000 7f8d3e200640 1 --2- 172.21.15.159:0/4232365460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d3815e860 0x7f8d3806d4f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.565+0000 7f8d3e200640 1 -- 172.21.15.159:0/4232365460 >> 172.21.15.159:0/4232365460 conn(0x7f8d3809d7d0 msgr2=0x7f8d3808bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.565+0000 7f7a63400640 1 --2- 172.21.15.159:0/2714614512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a640d3e40 0x7f7a640b9e20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009ad0 tx=0x7f7a5002bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f763cb4194058ca server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.565+0000 7f8d3e200640 1 -- 172.21.15.159:0/4232365460 shutdown_connections 2024-04-04T06:06:05.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.564+0000 7f0b1be00640 1 -- start start 2024-04-04T06:06:05.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.565+0000 7f0b1be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b140b1530 0x7f0b141f89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.565+0000 7f8d3e200640 1 -- 172.21.15.159:0/4232365460 wait complete. 2024-04-04T06:06:05.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.565+0000 7f0b1be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b140b3b70 0x7f0b1414c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.565+0000 7f0b1be00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b1415de50 0x7f0b141ee6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.565+0000 7f0b1be00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1415ae30 con 0x7f0b140b3b70 2024-04-04T06:06:05.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.566+0000 7f0b1be00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0b140b4150 con 0x7f0b140b1530 2024-04-04T06:06:05.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.566+0000 7f0b1be00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0b1415afb0 con 0x7f0b1415de50 2024-04-04T06:06:05.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.566+0000 7f0b18a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b140b3b70 0x7f0b1414c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.566+0000 7f1157e00640 1 --2- 172.21.15.159:0/2702164315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1158162920 0x7f11581670b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f11480097a0 tx=0x7f114802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5103d906632c6914 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.566+0000 7f0b18a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b140b3b70 0x7f0b1414c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37094/0 (socket says 172.21.15.159:37094) 2024-04-04T06:06:05.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.566+0000 7f0b18a00640 1 -- 172.21.15.159:0/1356033645 learned_addr learned my addr 172.21.15.159:0/1356033645 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.566+0000 7f0b19e00640 1 --2- 172.21.15.159:0/1356033645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b1415de50 0x7f0b141ee6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.566+0000 7f0b19400640 1 --2- 172.21.15.159:0/1356033645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b140b1530 0x7f0b141f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.566+0000 7f0b18a00640 1 -- 172.21.15.159:0/1356033645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b1415de50 msgr2=0x7f0b141ee6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.566+0000 7f0b18a00640 1 --2- 172.21.15.159:0/1356033645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b1415de50 0x7f0b141ee6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.566+0000 7f0b18a00640 1 -- 172.21.15.159:0/1356033645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b140b1530 msgr2=0x7f0b141f89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.566+0000 7f0b18a00640 1 --2- 172.21.15.159:0/1356033645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b140b1530 0x7f0b141f89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.566+0000 7f0b18a00640 1 -- 172.21.15.159:0/1356033645 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b141598b0 con 0x7f0b140b3b70 2024-04-04T06:06:05.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.566+0000 7f0b19400640 1 --2- 172.21.15.159:0/1356033645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b140b1530 0x7f0b141f89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:05.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.567+0000 7f0b18a00640 1 --2- 172.21.15.159:0/1356033645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b140b3b70 0x7f0b1414c510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0b00010c80 tx=0x7f0b000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.567+0000 7f8d3e200640 1 Processor -- start 2024-04-04T06:06:05.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.568+0000 7f8d3e200640 1 -- start start 2024-04-04T06:06:05.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.568+0000 7f7a62a00640 1 -- 172.21.15.159:0/2714614512 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5002d700 con 0x7f7a640d3e40 2024-04-04T06:06:05.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.568+0000 7f1156a00640 1 -- 172.21.15.159:0/2702164315 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148038070 con 0x7f1158162920 2024-04-04T06:06:05.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.568+0000 7f1156a00640 1 -- 172.21.15.159:0/2702164315 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1148031700 con 0x7f1158162920 2024-04-04T06:06:05.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.568+0000 7f7a62a00640 1 -- 172.21.15.159:0/2714614512 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a5002d8a0 con 0x7f7a640d3e40 2024-04-04T06:06:05.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.568+0000 7f7a62a00640 1 -- 172.21.15.159:0/2714614512 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5002dc70 con 0x7f7a640d3e40 2024-04-04T06:06:05.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.568+0000 7f1156a00640 1 -- 172.21.15.159:0/2702164315 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1148031ad0 con 0x7f1158162920 2024-04-04T06:06:05.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.568+0000 7f0b11600640 1 -- 172.21.15.159:0/1356033645 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0003d070 con 0x7f0b140b3b70 2024-04-04T06:06:05.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.568+0000 7f0b11600640 1 -- 172.21.15.159:0/1356033645 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b0003f070 con 0x7f0b140b3b70 2024-04-04T06:06:05.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.568+0000 7f0b11600640 1 -- 172.21.15.159:0/1356033645 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00044470 con 0x7f0b140b3b70 2024-04-04T06:06:05.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.569+0000 7f8d3e200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d3806da30 0x7f8d381f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.569+0000 7f8d3e200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d380d4910 0x7f8d380c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.569+0000 7f8d3e200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d3815e860 0x7f8d380c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.569+0000 7f8d3e200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8d380cedb0 con 0x7f8d3815e860 2024-04-04T06:06:05.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.569+0000 7f8d3e200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8d380d2c50 con 0x7f8d380d4910 2024-04-04T06:06:05.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.569+0000 7f8d3e200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8d380cef30 con 0x7f8d3806da30 2024-04-04T06:06:05.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.569+0000 7f115ec00640 1 -- 172.21.15.159:0/2702164315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1158162920 msgr2=0x7f11581670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.569+0000 7f115ec00640 1 --2- 172.21.15.159:0/2702164315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1158162920 0x7f11581670b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f11480097a0 tx=0x7f114802b7a0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.569+0000 7f115ec00640 1 -- 172.21.15.159:0/2702164315 shutdown_connections 2024-04-04T06:06:05.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.569+0000 7f115ec00640 1 --2- 172.21.15.159:0/2702164315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1158163ac0 0x7f1158071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.569+0000 7f115ec00640 1 --2- 172.21.15.159:0/2702164315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1158162920 0x7f11581670b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.569+0000 7f115ec00640 1 -- 172.21.15.159:0/2702164315 >> 172.21.15.159:0/2702164315 conn(0x7f115806b750 msgr2=0x7f115806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.569+0000 7f115ec00640 1 -- 172.21.15.159:0/2702164315 shutdown_connections 2024-04-04T06:06:05.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.569+0000 7f115ec00640 1 -- 172.21.15.159:0/2702164315 wait complete. 2024-04-04T06:06:05.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.569+0000 7f7a69a00640 1 -- 172.21.15.159:0/2714614512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a640d3e40 msgr2=0x7f7a640b9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.569+0000 7f7a69a00640 1 --2- 172.21.15.159:0/2714614512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a640d3e40 0x7f7a640b9e20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009ad0 tx=0x7f7a5002bb50 comp rx=0 tx=0).stop 2024-04-04T06:06:05.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.568+0000 7f0b1be00640 1 -- 172.21.15.159:0/1356033645 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b141f8ff0 con 0x7f0b140b3b70 2024-04-04T06:06:05.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.570+0000 7fcc6e000640 1 -- 172.21.15.159:0/2352115872 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcc6807bb40 con 0x7fcc7815f530 2024-04-04T06:06:05.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.569+0000 7f0b1be00640 1 -- 172.21.15.159:0/1356033645 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b141f95b0 con 0x7f0b140b3b70 2024-04-04T06:06:05.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.570+0000 7f7a69a00640 1 -- 172.21.15.159:0/2714614512 shutdown_connections 2024-04-04T06:06:05.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.570+0000 7f7a69a00640 1 --2- 172.21.15.159:0/2714614512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a640cef20 0x7f7a640cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.570+0000 7f7a69a00640 1 --2- 172.21.15.159:0/2714614512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a640d3e40 0x7f7a640b9e20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.570+0000 7f7a69a00640 1 -- 172.21.15.159:0/2714614512 >> 172.21.15.159:0/2714614512 conn(0x7f7a640678b0 msgr2=0x7f7a640d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.570+0000 7f0b1be00640 1 -- 172.21.15.159:0/1356033645 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b140685c0 con 0x7f0b140b3b70 2024-04-04T06:06:05.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.570+0000 7f8d37e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d3815e860 0x7f8d380c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.570+0000 7f8d37400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d3806da30 0x7f8d381f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.570+0000 7f8d36a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d380d4910 0x7f8d380c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.570+0000 7f7a69a00640 1 -- 172.21.15.159:0/2714614512 shutdown_connections 2024-04-04T06:06:05.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.570+0000 7f7a69a00640 1 -- 172.21.15.159:0/2714614512 wait complete. 2024-04-04T06:06:05.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.570+0000 7f8d36a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d380d4910 0x7f8d380c8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:41150/0 (socket says 172.21.15.159:41150) 2024-04-04T06:06:05.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.570+0000 7f8d37400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d3806da30 0x7f8d381f4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:55124/0 (socket says 172.21.15.159:55124) 2024-04-04T06:06:05.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.570+0000 7f8d36a00640 1 -- 172.21.15.159:0/3830282015 learned_addr learned my addr 172.21.15.159:0/3830282015 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.571+0000 7f8d36a00640 1 -- 172.21.15.159:0/3830282015 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d3806da30 msgr2=0x7f8d381f4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.571+0000 7f8d36a00640 1 --2- 172.21.15.159:0/3830282015 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d3806da30 0x7f8d381f4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.571+0000 7f8d36a00640 1 -- 172.21.15.159:0/3830282015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d3815e860 msgr2=0x7f8d380c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.571+0000 7f8d36a00640 1 --2- 172.21.15.159:0/3830282015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d3815e860 0x7f8d380c8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.571+0000 7f8d36a00640 1 -- 172.21.15.159:0/3830282015 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d381ee7e0 con 0x7f8d380d4910 2024-04-04T06:06:05.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.571+0000 7f8d37e00640 1 --2- 172.21.15.159:0/3830282015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d3815e860 0x7f8d380c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:06:05.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.571+0000 7f115ec00640 1 Processor -- start 2024-04-04T06:06:05.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.572+0000 7f115ec00640 1 -- start start 2024-04-04T06:06:05.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.572+0000 7f8d36a00640 1 --2- 172.21.15.159:0/3830282015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d380d4910 0x7f8d380c8250 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400b4a0 tx=0x7f8d2400b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.572+0000 7f0b11600640 1 -- 172.21.15.159:0/1356033645 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b00030bc0 con 0x7f0b140b3b70 2024-04-04T06:06:05.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.573+0000 7f0b11600640 1 --2- 172.21.15.159:0/1356033645 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0af406c970 0x7f0af406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.573+0000 7f0b11600640 1 -- 172.21.15.159:0/1356033645 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0b000ae1b0 con 0x7f0b140b3b70 2024-04-04T06:06:05.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.573+0000 7f115ec00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11580d0280 0x7f1158099970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.573+0000 7f7a69a00640 1 Processor -- start 2024-04-04T06:06:05.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.573+0000 7f115ec00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1158162920 0x7f115815f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.573+0000 7f7a69a00640 1 -- start start 2024-04-04T06:06:05.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.573+0000 7f115ec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1158163ac0 0x7f1158159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.573+0000 7f115ec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1158072040 con 0x7f1158163ac0 2024-04-04T06:06:05.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.573+0000 7f115ec00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1158162d00 con 0x7f1158162920 2024-04-04T06:06:05.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.573+0000 7f115ec00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f11580721c0 con 0x7f11580d0280 2024-04-04T06:06:05.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.574+0000 7f0b19400640 1 --2- 172.21.15.159:0/1356033645 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0af406c970 0x7f0af406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.574+0000 7f115cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1158163ac0 0x7f1158159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.574+0000 7f115cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1158163ac0 0x7f1158159160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37102/0 (socket says 172.21.15.159:37102) 2024-04-04T06:06:05.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.574+0000 7f115cc00640 1 -- 172.21.15.159:0/699788316 learned_addr learned my addr 172.21.15.159:0/699788316 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.574+0000 7f1157e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11580d0280 0x7f1158099970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.574+0000 7f1157400640 1 --2- 172.21.15.159:0/699788316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1158162920 0x7f115815f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.574+0000 7f115cc00640 1 -- 172.21.15.159:0/699788316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11580d0280 msgr2=0x7f1158099970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.574+0000 7f115cc00640 1 --2- 172.21.15.159:0/699788316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11580d0280 0x7f1158099970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.574+0000 7f115cc00640 1 -- 172.21.15.159:0/699788316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1158162920 msgr2=0x7f115815f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.574+0000 7f115cc00640 1 --2- 172.21.15.159:0/699788316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1158162920 0x7f115815f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.574+0000 7f115cc00640 1 -- 172.21.15.159:0/699788316 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11581ee5e0 con 0x7f1158163ac0 2024-04-04T06:06:05.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.575+0000 7f115cc00640 1 --2- 172.21.15.159:0/699788316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1158163ac0 0x7f1158159160 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f115000caf0 tx=0x7f115000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.575+0000 7f0b19400640 1 --2- 172.21.15.159:0/1356033645 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0af406c970 0x7f0af406ee30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0b080015e0 tx=0x7f0b08009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.575+0000 7f7a69a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a640cef20 0x7f7a642012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.576+0000 7f7a69a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a640d3e40 0x7f7a640ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.576+0000 7f7a69a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a6415e460 0x7f7a640cd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.576+0000 7f7a69a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7a640d3920 con 0x7f7a640d3e40 2024-04-04T06:06:05.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.576+0000 7f7a63e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a640cef20 0x7f7a642012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.576+0000 7f7a63e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a640cef20 0x7f7a642012c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:41158/0 (socket says 172.21.15.159:41158) 2024-04-04T06:06:05.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.576+0000 7f7a63e00640 1 -- 172.21.15.159:0/441770322 learned_addr learned my addr 172.21.15.159:0/441770322 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.576+0000 7f7a63400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a640d3e40 0x7f7a640ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.576+0000 7f7a69a00640 1 -- 172.21.15.159:0/441770322 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7a640d3640 con 0x7f7a640cef20 2024-04-04T06:06:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.576+0000 7f7a69a00640 1 -- 172.21.15.159:0/441770322 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7a640b7310 con 0x7f7a6415e460 2024-04-04T06:06:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.576+0000 7f7a69000640 1 --2- 172.21.15.159:0/441770322 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a6415e460 0x7f7a640cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.577+0000 7f7a63400640 1 -- 172.21.15.159:0/441770322 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a6415e460 msgr2=0x7f7a640cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.577+0000 7f7a63400640 1 --2- 172.21.15.159:0/441770322 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a6415e460 0x7f7a640cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.577+0000 7f7a63400640 1 -- 172.21.15.159:0/441770322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a640cef20 msgr2=0x7f7a642012c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.577+0000 7f7a63400640 1 --2- 172.21.15.159:0/441770322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a640cef20 0x7f7a642012c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.577+0000 7f7a63400640 1 -- 172.21.15.159:0/441770322 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a64204210 con 0x7f7a640d3e40 2024-04-04T06:06:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.577+0000 7f7a63e00640 1 --2- 172.21.15.159:0/441770322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a640cef20 0x7f7a642012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:06:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.577+0000 7f7a63400640 1 --2- 172.21.15.159:0/441770322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a640d3e40 0x7f7a640ccd00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7a50033000 tx=0x7f7a500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.579+0000 7f7a5be00640 1 -- 172.21.15.159:0/441770322 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a50045070 con 0x7f7a640d3e40 2024-04-04T06:06:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.579+0000 7f7a5be00640 1 -- 172.21.15.159:0/441770322 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a50035c80 con 0x7f7a640d3e40 2024-04-04T06:06:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.579+0000 7f7a5be00640 1 -- 172.21.15.159:0/441770322 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5003e660 con 0x7f7a640d3e40 2024-04-04T06:06:05.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.579+0000 7f114fe00640 1 -- 172.21.15.159:0/699788316 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115000aca0 con 0x7f1158163ac0 2024-04-04T06:06:05.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.579+0000 7f114fe00640 1 -- 172.21.15.159:0/699788316 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1150012400 con 0x7f1158163ac0 2024-04-04T06:06:05.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.579+0000 7f114fe00640 1 -- 172.21.15.159:0/699788316 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115000ae60 con 0x7f1158163ac0 2024-04-04T06:06:05.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.579+0000 7f115ec00640 1 -- 172.21.15.159:0/699788316 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11581ee8c0 con 0x7f1158163ac0 2024-04-04T06:06:05.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.579+0000 7f115ec00640 1 -- 172.21.15.159:0/699788316 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11581eee80 con 0x7f1158163ac0 2024-04-04T06:06:05.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.579+0000 7f0b11600640 1 -- 172.21.15.159:0/1356033645 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0b141598b0 con 0x7f0b140b3b70 2024-04-04T06:06:05.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.579+0000 7f7a69a00640 1 -- 172.21.15.159:0/441770322 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a642044f0 con 0x7f7a640d3e40 2024-04-04T06:06:05.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.579+0000 7f7a69a00640 1 -- 172.21.15.159:0/441770322 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a64204ab0 con 0x7f7a640d3e40 2024-04-04T06:06:05.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.580+0000 7f115ec00640 1 -- 172.21.15.159:0/699788316 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f115806bb30 con 0x7f1158163ac0 2024-04-04T06:06:05.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.581+0000 7f7a69a00640 1 -- 172.21.15.159:0/441770322 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a64075940 con 0x7f7a640d3e40 2024-04-04T06:06:05.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.581+0000 7f9708a00640 1 -- 172.21.15.159:0/1247813474 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc02f850 con 0x7f97040e30c0 2024-04-04T06:06:05.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.581+0000 7f9708a00640 1 -- 172.21.15.159:0/1247813474 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96fc02f9f0 con 0x7f97040e30c0 2024-04-04T06:06:05.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.581+0000 7f525d600640 1 -- 172.21.15.159:0/3276464247 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5254022020 con 0x7f5260073520 2024-04-04T06:06:05.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.581+0000 7f9708a00640 1 -- 172.21.15.159:0/1247813474 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc02fde0 con 0x7f97040e30c0 2024-04-04T06:06:05.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.582+0000 7f970b200640 1 -- 172.21.15.159:0/1247813474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97040e30c0 msgr2=0x7f9704057d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.582+0000 7f970b200640 1 --2- 172.21.15.159:0/1247813474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97040e30c0 0x7f9704057d90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009870 tx=0x7f96fc02ba30 comp rx=0 tx=0).stop 2024-04-04T06:06:05.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.582+0000 7f114fe00640 1 -- 172.21.15.159:0/699788316 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f11500127f0 con 0x7f1158163ac0 2024-04-04T06:06:05.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.584+0000 7f7a5be00640 1 -- 172.21.15.159:0/441770322 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a50048070 con 0x7f7a640d3e40 2024-04-04T06:06:05.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.584+0000 7f7b7e000640 1 -- 172.21.15.159:0/590126719 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b70035020 con 0x7f7b8006da30 2024-04-04T06:06:05.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.584+0000 7f7b7e000640 1 -- 172.21.15.159:0/590126719 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b70031830 con 0x7f7b8006da30 2024-04-04T06:06:05.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.584+0000 7f7b7e000640 1 -- 172.21.15.159:0/590126719 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b70031c00 con 0x7f7b8006da30 2024-04-04T06:06:05.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.584+0000 7f114fe00640 1 --2- 172.21.15.159:0/699788316 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f113806c970 0x7f113806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.584+0000 7f114fe00640 1 -- 172.21.15.159:0/699788316 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f115008c020 con 0x7f1158163ac0 2024-04-04T06:06:05.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.585+0000 7f7a5be00640 1 --2- 172.21.15.159:0/441770322 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a4406c8f0 0x7f7a4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.585+0000 7f1157e00640 1 --2- 172.21.15.159:0/699788316 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f113806c970 0x7f113806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.585+0000 7f7a5be00640 1 -- 172.21.15.159:0/441770322 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7a500ada40 con 0x7f7a640d3e40 2024-04-04T06:06:05.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.585+0000 7f7b86600640 1 -- 172.21.15.159:0/590126719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b8006da30 msgr2=0x7f7b800d43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.585+0000 7f7b86600640 1 --2- 172.21.15.159:0/590126719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b8006da30 0x7f7b800d43d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7b80162160 tx=0x7f7b7002b8d0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.585+0000 7f970b200640 1 -- 172.21.15.159:0/1247813474 shutdown_connections 2024-04-04T06:06:05.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.585+0000 7f970b200640 1 --2- 172.21.15.159:0/1247813474 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9704054300 0x7f97040068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.586+0000 7f970b200640 1 --2- 172.21.15.159:0/1247813474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97040e30c0 0x7f9704057d90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.586+0000 7f970b200640 1 -- 172.21.15.159:0/1247813474 >> 172.21.15.159:0/1247813474 conn(0x7f97040559e0 msgr2=0x7f9704055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.586+0000 7f7b86600640 1 -- 172.21.15.159:0/590126719 shutdown_connections 2024-04-04T06:06:05.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.586+0000 7f7b86600640 1 --2- 172.21.15.159:0/590126719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b8006da30 0x7f7b800d43d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.586+0000 7f7b86600640 1 --2- 172.21.15.159:0/590126719 >> v2:172.21.15.159:3300/0 conn(0x7f7b8015e860 0x7f7b8006d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.586+0000 7f7b86600640 1 -- 172.21.15.159:0/590126719 >> 172.21.15.159:0/590126719 conn(0x7f7b8009d7d0 msgr2=0x7f7b8008bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.586+0000 7f7a63e00640 1 --2- 172.21.15.159:0/441770322 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a4406c8f0 0x7f7a4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.586+0000 7f8d2f400640 1 -- 172.21.15.159:0/3830282015 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24014070 con 0x7f8d380d4910 2024-04-04T06:06:05.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.586+0000 7f7b86600640 1 -- 172.21.15.159:0/590126719 shutdown_connections 2024-04-04T06:06:05.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.586+0000 7f8d2f400640 1 -- 172.21.15.159:0/3830282015 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d24010610 con 0x7f8d380d4910 2024-04-04T06:06:05.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.586+0000 7f8d2f400640 1 -- 172.21.15.159:0/3830282015 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24010a00 con 0x7f8d380d4910 2024-04-04T06:06:05.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.586+0000 7f1157e00640 1 --2- 172.21.15.159:0/699788316 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f113806c970 0x7f113806ee30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1148035000 tx=0x7f1148008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.587+0000 7f970b200640 1 -- 172.21.15.159:0/1247813474 shutdown_connections 2024-04-04T06:06:05.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.587+0000 7f970b200640 1 -- 172.21.15.159:0/1247813474 wait complete. 2024-04-04T06:06:05.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.587+0000 7f8d3e200640 1 -- 172.21.15.159:0/3830282015 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d381f52c0 con 0x7f8d380d4910 2024-04-04T06:06:05.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.587+0000 7f8d3e200640 1 -- 172.21.15.159:0/3830282015 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d380ca9d0 con 0x7f8d380d4910 2024-04-04T06:06:05.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.586+0000 7f7b86600640 1 -- 172.21.15.159:0/590126719 wait complete. 2024-04-04T06:06:05.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.588+0000 7f7b86600640 1 Processor -- start 2024-04-04T06:06:05.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.589+0000 7f8d3e200640 1 -- 172.21.15.159:0/3830282015 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d380685c0 con 0x7f8d380d4910 2024-04-04T06:06:05.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.589+0000 7f7b86600640 1 -- start start 2024-04-04T06:06:05.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.590+0000 7f970b200640 1 Processor -- start 2024-04-04T06:06:05.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.589+0000 7f7a63e00640 1 --2- 172.21.15.159:0/441770322 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a4406c8f0 0x7f7a4406edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009d80 tx=0x7f7a54009110 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.591+0000 7f970b200640 1 -- start start 2024-04-04T06:06:05.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.591+0000 7f7b86600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b8006da30 0x7f7b80160a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.591+0000 7f7b86600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b800d4910 0x7f7b800ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.591+0000 7f7b86600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b8015e860 0x7f7b800ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.591+0000 7f7b7f400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b8006da30 0x7f7b80160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.591+0000 7f7b7f400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b8006da30 0x7f7b80160a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:41160/0 (socket says 172.21.15.159:41160) 2024-04-04T06:06:05.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.591+0000 7f8d2f400640 1 -- 172.21.15.159:0/3830282015 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8d24017070 con 0x7f8d380d4910 2024-04-04T06:06:05.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.591+0000 7f7b7f400640 1 -- 172.21.15.159:0/2388562104 learned_addr learned my addr 172.21.15.159:0/2388562104 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.592+0000 7f7b86600640 1 -- 172.21.15.159:0/2388562104 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7b800cedb0 con 0x7f7b8015e860 2024-04-04T06:06:05.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.592+0000 7f7b86600640 1 -- 172.21.15.159:0/2388562104 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7b800d2c50 con 0x7f7b8006da30 2024-04-04T06:06:05.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.592+0000 7f7b86600640 1 -- 172.21.15.159:0/2388562104 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7b800cef30 con 0x7f7b800d4910 2024-04-04T06:06:05.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.592+0000 7f7b7fe00640 1 --2- 172.21.15.159:0/2388562104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b8015e860 0x7f7b800ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.592+0000 7f970b200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f970403e840 0x7f970417ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.592+0000 7f970b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9704054300 0x7f970417d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.592+0000 7f970b200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97040e30c0 0x7f97040da3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.592+0000 7f970b200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f97040e34a0 con 0x7f9704054300 2024-04-04T06:06:05.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.592+0000 7f970b200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f970403bfc0 con 0x7f97040e30c0 2024-04-04T06:06:05.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.592+0000 7f970b200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9704006e10 con 0x7f970403e840 2024-04-04T06:06:05.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.593+0000 7f7b7f400640 1 -- 172.21.15.159:0/2388562104 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b800d4910 msgr2=0x7f7b800ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.593+0000 7f7b7f400640 1 --2- 172.21.15.159:0/2388562104 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b800d4910 0x7f7b800ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.593+0000 7f7b7f400640 1 -- 172.21.15.159:0/2388562104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b8015e860 msgr2=0x7f7b800ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.593+0000 7f7b7f400640 1 --2- 172.21.15.159:0/2388562104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b8015e860 0x7f7b800ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.593+0000 7f7b7f400640 1 -- 172.21.15.159:0/2388562104 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b800cbbe0 con 0x7f7b8006da30 2024-04-04T06:06:05.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.593+0000 7f7b7fe00640 1 --2- 172.21.15.159:0/2388562104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b8015e860 0x7f7b800ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:05.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.593+0000 7f9709e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f970403e840 0x7f970417ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.593+0000 7f8d2f400640 1 --2- 172.21.15.159:0/3830282015 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8d1806c9f0 0x7f8d1806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.593+0000 7f8d2f400640 1 -- 172.21.15.159:0/3830282015 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8d2408b960 con 0x7f8d380d4910 2024-04-04T06:06:05.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.593+0000 7f9709e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f970403e840 0x7f970417ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:55150/0 (socket says 172.21.15.159:55150) 2024-04-04T06:06:05.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.593+0000 7f9709e00640 1 -- 172.21.15.159:0/2914464060 learned_addr learned my addr 172.21.15.159:0/2914464060 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:05.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.594+0000 7f9709400640 1 --2- 172.21.15.159:0/2914464060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9704054300 0x7f970417d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.594+0000 7f7b7f400640 1 --2- 172.21.15.159:0/2388562104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b8006da30 0x7f7b80160a40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7b78007e40 tx=0x7f7b7800e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.594+0000 7f970a800640 1 --2- 172.21.15.159:0/2914464060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97040e30c0 0x7f97040da3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.594+0000 7f8d37400640 1 --2- 172.21.15.159:0/3830282015 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8d1806c9f0 0x7f8d1806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.595+0000 7f9709400640 1 -- 172.21.15.159:0/2914464060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f970403e840 msgr2=0x7f970417ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.595+0000 7f9709400640 1 --2- 172.21.15.159:0/2914464060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f970403e840 0x7f970417ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.595+0000 7f9709400640 1 -- 172.21.15.159:0/2914464060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97040e30c0 msgr2=0x7f97040da3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.595+0000 7f9709400640 1 --2- 172.21.15.159:0/2914464060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97040e30c0 0x7f97040da3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.595+0000 7f9709400640 1 -- 172.21.15.159:0/2914464060 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97040db020 con 0x7f9704054300 2024-04-04T06:06:05.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.595+0000 7f9709400640 1 --2- 172.21.15.159:0/2914464060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9704054300 0x7f970417d350 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f96f800c400 tx=0x7f96f800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.595+0000 7f8d37400640 1 --2- 172.21.15.159:0/3830282015 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8d1806c9f0 0x7f8d1806eeb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8d28036000 tx=0x7f8d2803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.595+0000 7f114fe00640 1 -- 172.21.15.159:0/699788316 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f115806bb30 con 0x7f1158163ac0 2024-04-04T06:06:05.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.603+0000 7f9702000640 1 -- 172.21.15.159:0/2914464060 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f8017070 con 0x7f9704054300 2024-04-04T06:06:05.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.603+0000 7f9702000640 1 -- 172.21.15.159:0/2914464060 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96f80052d0 con 0x7f9704054300 2024-04-04T06:06:05.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.603+0000 7f9702000640 1 -- 172.21.15.159:0/2914464060 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f8003830 con 0x7f9704054300 2024-04-04T06:06:05.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.603+0000 7f970b200640 1 -- 172.21.15.159:0/2914464060 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97040db300 con 0x7f9704054300 2024-04-04T06:06:05.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.603+0000 7f970b200640 1 -- 172.21.15.159:0/2914464060 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97040db8c0 con 0x7f9704054300 2024-04-04T06:06:05.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.603+0000 7f7a5be00640 1 -- 172.21.15.159:0/441770322 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7a5004f070 con 0x7f7a640d3e40 2024-04-04T06:06:05.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.605+0000 7f7b77400640 1 -- 172.21.15.159:0/2388562104 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b78014620 con 0x7f7b8006da30 2024-04-04T06:06:05.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.605+0000 7f7b77400640 1 -- 172.21.15.159:0/2388562104 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b78019070 con 0x7f7b8006da30 2024-04-04T06:06:05.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.605+0000 7f7b86600640 1 -- 172.21.15.159:0/2388562104 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b800cbec0 con 0x7f7b8006da30 2024-04-04T06:06:05.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.605+0000 7f7b86600640 1 -- 172.21.15.159:0/2388562104 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b800cc480 con 0x7f7b8006da30 2024-04-04T06:06:05.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.605+0000 7f8d2f400640 1 -- 172.21.15.159:0/3830282015 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8d381f52c0 con 0x7f8d380d4910 2024-04-04T06:06:05.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.606+0000 7f7b77400640 1 -- 172.21.15.159:0/2388562104 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b78014a00 con 0x7f7b8006da30 2024-04-04T06:06:05.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.605+0000 7f970b200640 1 -- 172.21.15.159:0/2914464060 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97040dbcc0 con 0x7f9704054300 2024-04-04T06:06:05.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.606+0000 7f9702000640 1 -- 172.21.15.159:0/2914464060 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96f801a070 con 0x7f9704054300 2024-04-04T06:06:05.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.608+0000 7f9702000640 1 --2- 172.21.15.159:0/2914464060 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f96ec06c940 0x7f96ec06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.608+0000 7f9702000640 1 -- 172.21.15.159:0/2914464060 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f96f808af00 con 0x7f9704054300 2024-04-04T06:06:05.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.609+0000 7f7b74c00640 1 -- 172.21.15.159:0/2388562104 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b58005290 con 0x7f7b8006da30 2024-04-04T06:06:05.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.609+0000 7f9709e00640 1 --2- 172.21.15.159:0/2914464060 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f96ec06c940 0x7f96ec06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.610+0000 7f7b77400640 1 -- 172.21.15.159:0/2388562104 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b78014ba0 con 0x7f7b8006da30 2024-04-04T06:06:05.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.611+0000 7f7b77400640 1 --2- 172.21.15.159:0/2388562104 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7b6006c9c0 0x7f7b6006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:05.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.612+0000 7f7b77400640 1 -- 172.21.15.159:0/2388562104 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7b7808c3f0 con 0x7f7b8006da30 2024-04-04T06:06:05.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.612+0000 7f7b7ea00640 1 --2- 172.21.15.159:0/2388562104 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7b6006c9c0 0x7f7b6006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:05.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.613+0000 7f7b7ea00640 1 --2- 172.21.15.159:0/2388562104 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7b6006c9c0 0x7f7b6006ee80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7b70036000 tx=0x7f7b70030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.613+0000 7f9709e00640 1 --2- 172.21.15.159:0/2914464060 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f96ec06c940 0x7f96ec06ee00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009870 tx=0x7f96fc039040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:05.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.616+0000 7f9702000640 1 -- 172.21.15.159:0/2914464060 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f97040dbcc0 con 0x7f9704054300 2024-04-04T06:06:05.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.624+0000 7f7b77400640 1 -- 172.21.15.159:0/2388562104 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7b7805a800 con 0x7f7b8006da30 2024-04-04T06:06:05.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.987+0000 7fcc7ca00640 1 -- 172.21.15.159:0/2352115872 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fcc50005580 con 0x7fcc7815f530 2024-04-04T06:06:05.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.988+0000 7fcc6e000640 1 -- 172.21.15.159:0/2352115872 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcc6807b440 con 0x7fcc7815f530 2024-04-04T06:06:05.984 INFO:teuthology.orchestra.run.smithi159.stdout:64424509447 2024-04-04T06:06:05.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.992+0000 7fcc7ca00640 1 -- 172.21.15.159:0/2352115872 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcc5806c8f0 msgr2=0x7fcc5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.992+0000 7fcc7ca00640 1 --2- 172.21.15.159:0/2352115872 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcc5806c8f0 0x7fcc5806edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009b80 tx=0x7fcc64008040 comp rx=0 tx=0).stop 2024-04-04T06:06:05.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.992+0000 7fcc7ca00640 1 -- 172.21.15.159:0/2352115872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc7815f530 msgr2=0x7fcc780b1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:05.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.992+0000 7fcc7ca00640 1 --2- 172.21.15.159:0/2352115872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc7815f530 0x7fcc780b1ce0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcc68033000 tx=0x7fcc680046c0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.993+0000 7fcc7ca00640 1 -- 172.21.15.159:0/2352115872 shutdown_connections 2024-04-04T06:06:05.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.993+0000 7fcc7ca00640 1 --2- 172.21.15.159:0/2352115872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc7815f530 0x7fcc780b1ce0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.993+0000 7fcc7ca00640 1 --2- 172.21.15.159:0/2352115872 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcc5806c8f0 0x7fcc5806edb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.993+0000 7fcc7ca00640 1 --2- 172.21.15.159:0/2352115872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc7815b4e0 0x7fcc782009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.993+0000 7fcc7ca00640 1 --2- 172.21.15.159:0/2352115872 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcc780afee0 0x7fcc7815a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:05.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.993+0000 7fcc7ca00640 1 -- 172.21.15.159:0/2352115872 >> 172.21.15.159:0/2352115872 conn(0x7fcc780b6560 msgr2=0x7fcc78152bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:05.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.994+0000 7fcc7ca00640 1 -- 172.21.15.159:0/2352115872 shutdown_connections 2024-04-04T06:06:05.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:05.994+0000 7fcc7ca00640 1 -- 172.21.15.159:0/2352115872 wait complete. 2024-04-04T06:06:06.014 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.2 2024-04-04T06:06:06.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.019+0000 7f5267e00640 1 -- 172.21.15.159:0/3276464247 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f526015fa40 con 0x7f5260073520 2024-04-04T06:06:06.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.020+0000 7f0b1be00640 1 -- 172.21.15.159:0/1356033645 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f0b1406cd20 con 0x7f0b140b3b70 2024-04-04T06:06:06.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.021+0000 7f525d600640 1 -- 172.21.15.159:0/3276464247 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f525405a5b0 con 0x7f5260073520 2024-04-04T06:06:06.016 INFO:teuthology.orchestra.run.smithi159.stdout:64424509448 2024-04-04T06:06:06.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.021+0000 7f0b11600640 1 -- 172.21.15.159:0/1356033645 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0b0007c5c0 con 0x7f0b140b3b70 2024-04-04T06:06:06.016 INFO:teuthology.orchestra.run.smithi159.stdout:64424509448 2024-04-04T06:06:06.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.024+0000 7f5267e00640 1 -- 172.21.15.159:0/3276464247 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f524006c8f0 msgr2=0x7f524006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:06.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.025+0000 7f5267e00640 1 --2- 172.21.15.159:0/3276464247 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f524006c8f0 0x7f524006edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5260161e90 tx=0x7f524c008040 comp rx=0 tx=0).stop 2024-04-04T06:06:06.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.025+0000 7f5267e00640 1 -- 172.21.15.159:0/3276464247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5260073520 msgr2=0x7f52600cbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:06.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.025+0000 7f5267e00640 1 --2- 172.21.15.159:0/3276464247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5260073520 0x7f52600cbd70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f525400d7d0 tx=0x7f525400dc90 comp rx=0 tx=0).stop 2024-04-04T06:06:06.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.024+0000 7f0b1be00640 1 -- 172.21.15.159:0/1356033645 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0af406c970 msgr2=0x7f0af406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:06.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.024+0000 7f0b1be00640 1 --2- 172.21.15.159:0/1356033645 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0af406c970 0x7f0af406ee30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0b080015e0 tx=0x7f0b08009040 comp rx=0 tx=0).stop 2024-04-04T06:06:06.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.025+0000 7f0b1be00640 1 -- 172.21.15.159:0/1356033645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b140b3b70 msgr2=0x7f0b1414c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:06.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.025+0000 7f0b1be00640 1 --2- 172.21.15.159:0/1356033645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b140b3b70 0x7f0b1414c510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0b00010c80 tx=0x7f0b000046c0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.025+0000 7f5267e00640 1 -- 172.21.15.159:0/3276464247 shutdown_connections 2024-04-04T06:06:06.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.025+0000 7f5267e00640 1 --2- 172.21.15.159:0/3276464247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f526015e950 0x7f5260144c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.025+0000 7f5267e00640 1 --2- 172.21.15.159:0/3276464247 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52600ae150 0x7f5260144710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.025+0000 7f5267e00640 1 --2- 172.21.15.159:0/3276464247 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f524006c8f0 0x7f524006edb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.025+0000 7f5267e00640 1 --2- 172.21.15.159:0/3276464247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5260073520 0x7f52600cbd70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.025+0000 7f0b1be00640 1 -- 172.21.15.159:0/1356033645 shutdown_connections 2024-04-04T06:06:06.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.025+0000 7f0b1be00640 1 --2- 172.21.15.159:0/1356033645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b1415de50 0x7f0b141ee6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.025+0000 7f0b1be00640 1 --2- 172.21.15.159:0/1356033645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b140b3b70 0x7f0b1414c510 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.025+0000 7f0b1be00640 1 --2- 172.21.15.159:0/1356033645 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0af406c970 0x7f0af406ee30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.025+0000 7f0b1be00640 1 --2- 172.21.15.159:0/1356033645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b140b1530 0x7f0b141f89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.025+0000 7f0b1be00640 1 -- 172.21.15.159:0/1356033645 >> 172.21.15.159:0/1356033645 conn(0x7f0b14072a00 msgr2=0x7f0b1406ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:06.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.025+0000 7f5267e00640 1 -- 172.21.15.159:0/3276464247 >> 172.21.15.159:0/3276464247 conn(0x7f5260072a00 msgr2=0x7f526015d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:06.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.025+0000 7f5267e00640 1 -- 172.21.15.159:0/3276464247 shutdown_connections 2024-04-04T06:06:06.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.025+0000 7f5267e00640 1 -- 172.21.15.159:0/3276464247 wait complete. 2024-04-04T06:06:06.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.025+0000 7f0b1be00640 1 -- 172.21.15.159:0/1356033645 shutdown_connections 2024-04-04T06:06:06.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.025+0000 7f0b1be00640 1 -- 172.21.15.159:0/1356033645 wait complete. 2024-04-04T06:06:06.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.037+0000 7f8d3e200640 1 -- 172.21.15.159:0/3830282015 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f8d380d0160 con 0x7f8d380d4910 2024-04-04T06:06:06.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.037+0000 7f7a69a00640 1 -- 172.21.15.159:0/441770322 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f7a64204c80 con 0x7f7a640d3e40 2024-04-04T06:06:06.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.038+0000 7f8d2f400640 1 -- 172.21.15.159:0/3830282015 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8d381ee7e0 con 0x7f8d380d4910 2024-04-04T06:06:06.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.038+0000 7f115ec00640 1 -- 172.21.15.159:0/699788316 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f11581ef080 con 0x7f1158163ac0 2024-04-04T06:06:06.033 INFO:teuthology.orchestra.run.smithi159.stdout:64424509445 2024-04-04T06:06:06.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.038+0000 7f7a5be00640 1 -- 172.21.15.159:0/441770322 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7a5007b840 con 0x7f7a640d3e40 2024-04-04T06:06:06.034 INFO:teuthology.orchestra.run.smithi159.stdout:60129542151 2024-04-04T06:06:06.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.039+0000 7f970b200640 1 -- 172.21.15.159:0/2914464060 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f97040057e0 con 0x7f9704054300 2024-04-04T06:06:06.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f8d3e200640 1 -- 172.21.15.159:0/3830282015 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8d1806c9f0 msgr2=0x7f8d1806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:06.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f7a69a00640 1 -- 172.21.15.159:0/441770322 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a4406c8f0 msgr2=0x7f7a4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:06.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f7a69a00640 1 --2- 172.21.15.159:0/441770322 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a4406c8f0 0x7f7a4406edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009d80 tx=0x7f7a54009110 comp rx=0 tx=0).stop 2024-04-04T06:06:06.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f7a69a00640 1 -- 172.21.15.159:0/441770322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a640d3e40 msgr2=0x7f7a640ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:06.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f7a69a00640 1 --2- 172.21.15.159:0/441770322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a640d3e40 0x7f7a640ccd00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7a50033000 tx=0x7f7a500046c0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f7a69a00640 1 -- 172.21.15.159:0/441770322 shutdown_connections 2024-04-04T06:06:06.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f8d3e200640 1 --2- 172.21.15.159:0/3830282015 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8d1806c9f0 0x7f8d1806eeb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8d28036000 tx=0x7f8d2803c040 comp rx=0 tx=0).stop 2024-04-04T06:06:06.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f8d3e200640 1 -- 172.21.15.159:0/3830282015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d380d4910 msgr2=0x7f8d380c8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:06.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f8d3e200640 1 --2- 172.21.15.159:0/3830282015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d380d4910 0x7f8d380c8250 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400b4a0 tx=0x7f8d2400b960 comp rx=0 tx=0).stop 2024-04-04T06:06:06.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f8d3e200640 1 -- 172.21.15.159:0/3830282015 shutdown_connections 2024-04-04T06:06:06.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f8d3e200640 1 --2- 172.21.15.159:0/3830282015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d3815e860 0x7f8d380c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f8d3e200640 1 --2- 172.21.15.159:0/3830282015 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8d1806c9f0 0x7f8d1806eeb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f8d3e200640 1 --2- 172.21.15.159:0/3830282015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d380d4910 0x7f8d380c8250 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f8d3e200640 1 --2- 172.21.15.159:0/3830282015 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d3806da30 0x7f8d381f4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f8d3e200640 1 -- 172.21.15.159:0/3830282015 >> 172.21.15.159:0/3830282015 conn(0x7f8d3809d7d0 msgr2=0x7f8d380cf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:06.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f7a69a00640 1 --2- 172.21.15.159:0/441770322 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a6415e460 0x7f7a640cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f7a69a00640 1 --2- 172.21.15.159:0/441770322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a640d3e40 0x7f7a640ccd00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f7a69a00640 1 --2- 172.21.15.159:0/441770322 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a4406c8f0 0x7f7a4406edb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f7a69a00640 1 --2- 172.21.15.159:0/441770322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a640cef20 0x7f7a642012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f7a69a00640 1 -- 172.21.15.159:0/441770322 >> 172.21.15.159:0/441770322 conn(0x7f7a640678b0 msgr2=0x7f7a640d4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:06.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f7a69a00640 1 -- 172.21.15.159:0/441770322 shutdown_connections 2024-04-04T06:06:06.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f7a69a00640 1 -- 172.21.15.159:0/441770322 wait complete. 2024-04-04T06:06:06.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f8d3e200640 1 -- 172.21.15.159:0/3830282015 shutdown_connections 2024-04-04T06:06:06.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.041+0000 7f8d3e200640 1 -- 172.21.15.159:0/3830282015 wait complete. 2024-04-04T06:06:06.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.042+0000 7f114fe00640 1 -- 172.21.15.159:0/699788316 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f11581ef080 con 0x7f1158163ac0 2024-04-04T06:06:06.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.042+0000 7f9702000640 1 -- 172.21.15.159:0/2914464060 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f96f8058cc0 con 0x7f9704054300 2024-04-04T06:06:06.040 INFO:teuthology.orchestra.run.smithi159.stdout:60129542149 2024-04-04T06:06:06.040 INFO:teuthology.orchestra.run.smithi159.stdout:64424509447 2024-04-04T06:06:06.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.045+0000 7f115ec00640 1 -- 172.21.15.159:0/699788316 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f113806c970 msgr2=0x7f113806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:06.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.045+0000 7f115ec00640 1 --2- 172.21.15.159:0/699788316 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f113806c970 0x7f113806ee30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1148035000 tx=0x7f1148008030 comp rx=0 tx=0).stop 2024-04-04T06:06:06.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.045+0000 7f115ec00640 1 -- 172.21.15.159:0/699788316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1158163ac0 msgr2=0x7f1158159160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:06.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.045+0000 7f115ec00640 1 --2- 172.21.15.159:0/699788316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1158163ac0 0x7f1158159160 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f115000caf0 tx=0x7f115000a060 comp rx=0 tx=0).stop 2024-04-04T06:06:06.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.045+0000 7f115ec00640 1 -- 172.21.15.159:0/699788316 shutdown_connections 2024-04-04T06:06:06.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.045+0000 7f115ec00640 1 --2- 172.21.15.159:0/699788316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1158163ac0 0x7f1158159160 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.045+0000 7f115ec00640 1 --2- 172.21.15.159:0/699788316 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f113806c970 0x7f113806ee30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.045+0000 7f115ec00640 1 --2- 172.21.15.159:0/699788316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1158162920 0x7f115815f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.045+0000 7f115ec00640 1 --2- 172.21.15.159:0/699788316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11580d0280 0x7f1158099970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.045+0000 7f115ec00640 1 -- 172.21.15.159:0/699788316 >> 172.21.15.159:0/699788316 conn(0x7f115806b750 msgr2=0x7f11580ae430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:06.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.046+0000 7f115ec00640 1 -- 172.21.15.159:0/699788316 shutdown_connections 2024-04-04T06:06:06.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.046+0000 7f115ec00640 1 -- 172.21.15.159:0/699788316 wait complete. 2024-04-04T06:06:06.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.047+0000 7f7b74c00640 1 -- 172.21.15.159:0/2388562104 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7b58005580 con 0x7f7b8006da30 2024-04-04T06:06:06.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.047+0000 7f970b200640 1 -- 172.21.15.159:0/2914464060 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f96ec06c940 msgr2=0x7f96ec06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:06.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.047+0000 7f970b200640 1 --2- 172.21.15.159:0/2914464060 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f96ec06c940 0x7f96ec06ee00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009870 tx=0x7f96fc039040 comp rx=0 tx=0).stop 2024-04-04T06:06:06.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.047+0000 7f970b200640 1 -- 172.21.15.159:0/2914464060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9704054300 msgr2=0x7f970417d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:06.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.047+0000 7f970b200640 1 --2- 172.21.15.159:0/2914464060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9704054300 0x7f970417d350 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f96f800c400 tx=0x7f96f800c8c0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.047+0000 7f970b200640 1 -- 172.21.15.159:0/2914464060 shutdown_connections 2024-04-04T06:06:06.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.047+0000 7f970b200640 1 --2- 172.21.15.159:0/2914464060 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f96ec06c940 0x7f96ec06ee00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.047+0000 7f970b200640 1 --2- 172.21.15.159:0/2914464060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97040e30c0 0x7f97040da3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.047+0000 7f970b200640 1 --2- 172.21.15.159:0/2914464060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9704054300 0x7f970417d350 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.047+0000 7f970b200640 1 --2- 172.21.15.159:0/2914464060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f970403e840 0x7f970417ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.047+0000 7f970b200640 1 -- 172.21.15.159:0/2914464060 >> 172.21.15.159:0/2914464060 conn(0x7f97040559e0 msgr2=0x7f97040071f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:06.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.047+0000 7f970b200640 1 -- 172.21.15.159:0/2914464060 shutdown_connections 2024-04-04T06:06:06.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.048+0000 7f970b200640 1 -- 172.21.15.159:0/2914464060 wait complete. 2024-04-04T06:06:06.046 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.0 2024-04-04T06:06:06.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.052+0000 7f7b77400640 1 -- 172.21.15.159:0/2388562104 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7b7805a100 con 0x7f7b8006da30 2024-04-04T06:06:06.050 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.3 2024-04-04T06:06:06.050 INFO:teuthology.orchestra.run.smithi159.stdout:64424509446 2024-04-04T06:06:06.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.059+0000 7f7b74c00640 1 -- 172.21.15.159:0/2388562104 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7b6006c9c0 msgr2=0x7f7b6006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:06.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.059+0000 7f7b74c00640 1 --2- 172.21.15.159:0/2388562104 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7b6006c9c0 0x7f7b6006ee80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7b70036000 tx=0x7f7b70030000 comp rx=0 tx=0).stop 2024-04-04T06:06:06.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.059+0000 7f7b74c00640 1 -- 172.21.15.159:0/2388562104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b8006da30 msgr2=0x7f7b80160a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:06.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.059+0000 7f7b74c00640 1 --2- 172.21.15.159:0/2388562104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b8006da30 0x7f7b80160a40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7b78007e40 tx=0x7f7b7800e4b0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.059+0000 7f7b74c00640 1 -- 172.21.15.159:0/2388562104 shutdown_connections 2024-04-04T06:06:06.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.059+0000 7f7b74c00640 1 --2- 172.21.15.159:0/2388562104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b8015e860 0x7f7b800ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.059+0000 7f7b74c00640 1 --2- 172.21.15.159:0/2388562104 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b800d4910 0x7f7b800ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.059+0000 7f7b74c00640 1 --2- 172.21.15.159:0/2388562104 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7b6006c9c0 0x7f7b6006ee80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.059+0000 7f7b74c00640 1 --2- 172.21.15.159:0/2388562104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b8006da30 0x7f7b80160a40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:06.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.059+0000 7f7b74c00640 1 -- 172.21.15.159:0/2388562104 >> 172.21.15.159:0/2388562104 conn(0x7f7b8009d7d0 msgr2=0x7f7b800b5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:06.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.059+0000 7f7b74c00640 1 -- 172.21.15.159:0/2388562104 shutdown_connections 2024-04-04T06:06:06.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:06.059+0000 7f7b74c00640 1 -- 172.21.15.159:0/2388562104 wait complete. 2024-04-04T06:06:06.062 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.7 2024-04-04T06:06:06.063 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.5 2024-04-04T06:06:06.068 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.4 2024-04-04T06:06:06.069 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.6 2024-04-04T06:06:06.074 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.1 2024-04-04T06:06:07.015 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-04T06:06:07.047 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-04T06:06:07.051 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-04T06:06:07.063 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-04T06:06:07.064 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-04T06:06:07.068 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-04T06:06:07.070 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-04T06:06:07.075 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-04T06:06:07.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.298+0000 7f35ee200640 1 Processor -- start 2024-04-04T06:06:07.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.298+0000 7f35ee200640 1 -- start start 2024-04-04T06:06:07.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.301+0000 7f35ee200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e00e3150 0x7f35e00e7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.301+0000 7f35ee200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f35e003c090 0x7f35e0058040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.302+0000 7f35ee200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f35e0053ca0 0x7f35e0054110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.302+0000 7f35e7400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e00e3150 0x7f35e00e7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.302+0000 7f35e7400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e00e3150 0x7f35e00e7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:55158/0 (socket says 172.21.15.159:55158) 2024-04-04T06:06:07.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.302+0000 7f35ee200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f35e0058580 con 0x7f35e003c090 2024-04-04T06:06:07.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.302+0000 7f35e7e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f35e0053ca0 0x7f35e0054110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.303+0000 7f35e7400640 1 -- 172.21.15.159:0/2963429282 learned_addr learned my addr 172.21.15.159:0/2963429282 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:07.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.303+0000 7f35e6a00640 1 --2- 172.21.15.159:0/2963429282 >> v2:172.21.15.159:3300/0 conn(0x7f35e003c090 0x7f35e0058040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.303+0000 7f35ee200640 1 -- 172.21.15.159:0/2963429282 --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f35e0054650 con 0x7f35e0053ca0 2024-04-04T06:06:07.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.303+0000 7f35ee200640 1 -- 172.21.15.159:0/2963429282 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f35e0058800 con 0x7f35e00e3150 2024-04-04T06:06:07.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.304+0000 7f35e6a00640 1 -- 172.21.15.159:0/2963429282 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e00e3150 msgr2=0x7f35e00e7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:07.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.304+0000 7f35e6a00640 1 --2- 172.21.15.159:0/2963429282 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e00e3150 0x7f35e00e7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.304+0000 7f35e6a00640 1 -- 172.21.15.159:0/2963429282 >> v2:172.21.15.178:3300/0 conn(0x7f35e0053ca0 msgr2=0x7f35e0054110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:07.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.304+0000 7f35e6a00640 1 --2- 172.21.15.159:0/2963429282 >> v2:172.21.15.178:3300/0 conn(0x7f35e0053ca0 0x7f35e0054110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.304+0000 7f35e6a00640 1 -- 172.21.15.159:0/2963429282 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e0058ae0 con 0x7f35e003c090 2024-04-04T06:06:07.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.305+0000 7f35e7e00640 1 --2- 172.21.15.159:0/2963429282 >> v2:172.21.15.178:3300/0 conn(0x7f35e0053ca0 0x7f35e0054110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:07.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.306+0000 7f35e7400640 1 --2- 172.21.15.159:0/2963429282 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e00e3150 0x7f35e00e7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:07.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.307+0000 7f35e6a00640 1 --2- 172.21.15.159:0/2963429282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e003c090 0x7f35e0058040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f35e0054cc0 tx=0x7f35d402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7f6329009312f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.308+0000 7f35e6000640 1 -- 172.21.15.159:0/2963429282 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d40317c0 con 0x7f35e003c090 2024-04-04T06:06:07.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.308+0000 7f35e6000640 1 -- 172.21.15.159:0/2963429282 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d4037070 con 0x7f35e003c090 2024-04-04T06:06:07.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.309+0000 7f35e6000640 1 -- 172.21.15.159:0/2963429282 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4031bc0 con 0x7f35e003c090 2024-04-04T06:06:07.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.310+0000 7f35ee200640 1 -- 172.21.15.159:0/2963429282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e003c090 msgr2=0x7f35e0058040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.310+0000 7f35ee200640 1 --2- 172.21.15.159:0/2963429282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e003c090 0x7f35e0058040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f35e0054cc0 tx=0x7f35d402b860 comp rx=0 tx=0).stop 2024-04-04T06:06:07.305 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.310+0000 7f35ee200640 1 -- 172.21.15.159:0/2963429282 shutdown_connections 2024-04-04T06:06:07.305 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.310+0000 7f35ee200640 1 --2- 172.21.15.159:0/2963429282 >> v2:172.21.15.178:3300/0 conn(0x7f35e0053ca0 0x7f35e0054110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.305 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.311+0000 7f35ee200640 1 --2- 172.21.15.159:0/2963429282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e003c090 0x7f35e0058040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.305 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.311+0000 7f35ee200640 1 --2- 172.21.15.159:0/2963429282 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e00e3150 0x7f35e00e7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.311+0000 7f35ee200640 1 -- 172.21.15.159:0/2963429282 >> 172.21.15.159:0/2963429282 conn(0x7f35e0058ed0 msgr2=0x7f35e00592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:07.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.311+0000 7f35ee200640 1 -- 172.21.15.159:0/2963429282 shutdown_connections 2024-04-04T06:06:07.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.312+0000 7f35ee200640 1 -- 172.21.15.159:0/2963429282 wait complete. 2024-04-04T06:06:07.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.315+0000 7f35ee200640 1 Processor -- start 2024-04-04T06:06:07.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.316+0000 7f35ee200640 1 -- start start 2024-04-04T06:06:07.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.317+0000 7f35ee200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e003c090 0x7f35e00db1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.317+0000 7f35ee200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e0053ca0 0x7f35e004e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.317+0000 7f35ee200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e00e3150 0x7f35e004edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.317+0000 7f35ee200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f35e00e3530 con 0x7f35e0053ca0 2024-04-04T06:06:07.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.317+0000 7f35ee200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35e0006630 con 0x7f35e003c090 2024-04-04T06:06:07.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.317+0000 7f35ee200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f35e0056e00 con 0x7f35e00e3150 2024-04-04T06:06:07.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.319+0000 7f35e6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e0053ca0 0x7f35e004e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.319+0000 7f35e6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e0053ca0 0x7f35e004e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37172/0 (socket says 172.21.15.159:37172) 2024-04-04T06:06:07.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.319+0000 7f35e6a00640 1 -- 172.21.15.159:0/3317413281 learned_addr learned my addr 172.21.15.159:0/3317413281 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:07.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.319+0000 7f35e7e00640 1 --2- 172.21.15.159:0/3317413281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e00e3150 0x7f35e004edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.320+0000 7f35e7e00640 1 -- 172.21.15.159:0/3317413281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e003c090 msgr2=0x7f35e00db1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.321+0000 7f35e7e00640 1 --2- 172.21.15.159:0/3317413281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e003c090 0x7f35e00db1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.321+0000 7f35e7e00640 1 -- 172.21.15.159:0/3317413281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e0053ca0 msgr2=0x7f35e004e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.321+0000 7f35e7e00640 1 --2- 172.21.15.159:0/3317413281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e0053ca0 0x7f35e004e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.321+0000 7f35e7e00640 1 -- 172.21.15.159:0/3317413281 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e004d7c0 con 0x7f35e00e3150 2024-04-04T06:06:07.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.321+0000 7f35e6a00640 1 --2- 172.21.15.159:0/3317413281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e0053ca0 0x7f35e004e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:06:07.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.322+0000 7f35e7e00640 1 --2- 172.21.15.159:0/3317413281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e00e3150 0x7f35e004edc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00c540 tx=0x7f35dc00ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.323+0000 7f35db400640 1 -- 172.21.15.159:0/3317413281 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc015070 con 0x7f35e00e3150 2024-04-04T06:06:07.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.323+0000 7f35db400640 1 -- 172.21.15.159:0/3317413281 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35dc011800 con 0x7f35e00e3150 2024-04-04T06:06:07.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.323+0000 7f35db400640 1 -- 172.21.15.159:0/3317413281 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc011bf0 con 0x7f35e00e3150 2024-04-04T06:06:07.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.324+0000 7f35ee200640 1 -- 172.21.15.159:0/3317413281 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e004daa0 con 0x7f35e00e3150 2024-04-04T06:06:07.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.325+0000 7f35ee200640 1 -- 172.21.15.159:0/3317413281 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e004e060 con 0x7f35e00e3150 2024-04-04T06:06:07.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.329+0000 7f35db400640 1 -- 172.21.15.159:0/3317413281 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35dc018070 con 0x7f35e00e3150 2024-04-04T06:06:07.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.329+0000 7f35ee200640 1 -- 172.21.15.159:0/3317413281 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e0037d90 con 0x7f35e00e3150 2024-04-04T06:06:07.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.331+0000 7f35db400640 1 --2- 172.21.15.159:0/3317413281 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35c806c8f0 0x7f35c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.331+0000 7f35db400640 1 -- 172.21.15.159:0/3317413281 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f35dc08b950 con 0x7f35e00e3150 2024-04-04T06:06:07.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.332+0000 7f35e7400640 1 --2- 172.21.15.159:0/3317413281 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35c806c8f0 0x7f35c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.333+0000 7f35e7400640 1 --2- 172.21.15.159:0/3317413281 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35c806c8f0 0x7f35c806edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f35e8064b70 tx=0x7f35e806e040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.338+0000 7f5ade000640 1 Processor -- start 2024-04-04T06:06:07.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.338+0000 7f5ade000640 1 -- start start 2024-04-04T06:06:07.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.340+0000 7f5ade000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ad80d37b0 0x7f5ad80b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.340+0000 7f5ade000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5ad80b67f0 0x7f5ad80b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.340+0000 7f5ade000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ad80d2710 con 0x7f5ad815e4c0 2024-04-04T06:06:07.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.340+0000 7f5ade000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f5ad815e8a0 con 0x7f5ad80b67f0 2024-04-04T06:06:07.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.340+0000 7f5ade000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad80d4040 con 0x7f5ad80d37b0 2024-04-04T06:06:07.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.341+0000 7f5adcc00640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f5ad815e4c0 0x7f5ad80b9bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.159:44256/0 (socket says 172.21.15.159:44256) 2024-04-04T06:06:07.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.341+0000 7f5adcc00640 1 -- 172.21.15.159:0/3112603924 learned_addr learned my addr 172.21.15.159:0/3112603924 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:07.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.341+0000 7f5ad7e00640 1 --2- 172.21.15.159:0/3112603924 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ad80d37b0 0x7f5ad80b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.341+0000 7f5add600640 1 --2- 172.21.15.159:0/3112603924 >> v2:172.21.15.159:3300/0 conn(0x7f5ad80b67f0 0x7f5ad80b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.341+0000 7f5ad7400640 1 -- 172.21.15.159:0/3112603924 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 132777284 0 0) 0x7f5ad80d2710 con 0x7f5ad815e4c0 2024-04-04T06:06:07.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.341+0000 7f5ad7400640 1 -- 172.21.15.159:0/3112603924 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ac0003520 con 0x7f5ad815e4c0 2024-04-04T06:06:07.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.342+0000 7f5ad7e00640 1 -- 172.21.15.159:0/3112603924 >> v2:172.21.15.159:3300/0 conn(0x7f5ad80b67f0 msgr2=0x7f5ad80b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:07.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.342+0000 7f5ad7e00640 1 --2- 172.21.15.159:0/3112603924 >> v2:172.21.15.159:3300/0 conn(0x7f5ad80b67f0 0x7f5ad80b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.342+0000 7f5ad7e00640 1 -- 172.21.15.159:0/3112603924 >> v1:172.21.15.178:6789/0 conn(0x7f5ad815e4c0 legacy=0x7f5ad80b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.342+0000 7f5ad7e00640 1 -- 172.21.15.159:0/3112603924 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad80d29f0 con 0x7f5ad80d37b0 2024-04-04T06:06:07.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.342+0000 7f5add600640 1 --2- 172.21.15.159:0/3112603924 >> v2:172.21.15.159:3300/0 conn(0x7f5ad80b67f0 0x7f5ad80b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:06:07.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.343+0000 7f5ad7e00640 1 --2- 172.21.15.159:0/3112603924 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ad80d37b0 0x7f5ad80b62b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40097a0 tx=0x7f5ac402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63290db3abe27e58 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.345+0000 7f3d8ba00640 1 Processor -- start 2024-04-04T06:06:07.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.345+0000 7f3d8ba00640 1 -- start start 2024-04-04T06:06:07.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.346+0000 7f3d8ba00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3d840d43d0 0x7f3d840d47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.346+0000 7f3d8ba00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d840d4d20 con 0x7f3d840cf6b0 2024-04-04T06:06:07.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.346+0000 7f3d8ba00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d8406dfa0 con 0x7f3d8415e860 2024-04-04T06:06:07.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.346+0000 7f3d8ba00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f3d840cf220 con 0x7f3d840d43d0 2024-04-04T06:06:07.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.347+0000 7f3d89000640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f3d8415e860 0x7f3d8406d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.159:44262/0 (socket says 172.21.15.159:44262) 2024-04-04T06:06:07.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.347+0000 7f3d89000640 1 -- 172.21.15.159:0/3104232888 learned_addr learned my addr 172.21.15.159:0/3104232888 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:07.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.347+0000 7f3d83e00640 1 --2- 172.21.15.159:0/3104232888 >> v2:172.21.15.159:3300/0 conn(0x7f3d840d43d0 0x7f3d840d47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.347+0000 7f3d83400640 1 -- 172.21.15.159:0/3104232888 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2426287850 0 0) 0x7f3d8406dfa0 con 0x7f3d8415e860 2024-04-04T06:06:07.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.347+0000 7f3d83400640 1 -- 172.21.15.159:0/3104232888 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d6c003520 con 0x7f3d8415e860 2024-04-04T06:06:07.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.348+0000 7f3d83e00640 1 -- 172.21.15.159:0/3104232888 >> v1:172.21.15.178:6789/0 conn(0x7f3d8415e860 legacy=0x7f3d8406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.348+0000 7f3d83e00640 1 -- 172.21.15.159:0/3104232888 >> v1:172.21.15.159:6789/0 conn(0x7f3d840cf6b0 legacy=0x7f3d840cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.348+0000 7f3d83400640 1 -- 172.21.15.159:0/3104232888 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3221369710 0 0) 0x7f3d7c001490 con 0x7f3d840cf6b0 2024-04-04T06:06:07.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.348+0000 7f3d83e00640 1 -- 172.21.15.159:0/3104232888 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8406e240 con 0x7f3d840d43d0 2024-04-04T06:06:07.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.349+0000 7fa4dac00640 1 Processor -- start 2024-04-04T06:06:07.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.349+0000 7fa4dac00640 1 -- start start 2024-04-04T06:06:07.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.349+0000 7f3d83e00640 1 --2- 172.21.15.159:0/3104232888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d840d43d0 0x7f3d840d47e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3d8406e170 tx=0x7f3d7002b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9a9817cc86a911 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.350+0000 7f3d83400640 1 -- 172.21.15.159:0/3104232888 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70034070 con 0x7f3d840d43d0 2024-04-04T06:06:07.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.350+0000 7f3d83400640 1 -- 172.21.15.159:0/3104232888 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d70030800 con 0x7f3d840d43d0 2024-04-04T06:06:07.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.350+0000 7f3d83400640 1 -- 172.21.15.159:0/3104232888 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70030bd0 con 0x7f3d840d43d0 2024-04-04T06:06:07.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.351+0000 7f3d8ba00640 1 -- 172.21.15.159:0/3104232888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d840d43d0 msgr2=0x7f3d840d47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.351+0000 7f3d8ba00640 1 --2- 172.21.15.159:0/3104232888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d840d43d0 0x7f3d840d47e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3d8406e170 tx=0x7f3d7002b940 comp rx=0 tx=0).stop 2024-04-04T06:06:07.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.351+0000 7fa4dac00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa4cc0e30c0 0x7fa4cc057d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.351+0000 7f3d8ba00640 1 -- 172.21.15.159:0/3104232888 shutdown_connections 2024-04-04T06:06:07.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.351+0000 7f3d8ba00640 1 --2- 172.21.15.159:0/3104232888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d840d43d0 0x7f3d840d47e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.351+0000 7f3d8ba00640 1 -- 172.21.15.159:0/3104232888 >> 172.21.15.159:0/3104232888 conn(0x7f3d8409d7d0 msgr2=0x7f3d8408bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:07.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.351+0000 7fa4dac00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4cc03e840 0x7fa4cc053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.351+0000 7fa4dac00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4cc0582d0 con 0x7fa4cc054130 2024-04-04T06:06:07.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.351+0000 7fa4dac00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fa4cc058570 con 0x7fa4cc0e30c0 2024-04-04T06:06:07.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.351+0000 7fa4dac00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa4cc058850 con 0x7fa4cc03e840 2024-04-04T06:06:07.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.351+0000 7f3d8ba00640 1 -- 172.21.15.159:0/3104232888 shutdown_connections 2024-04-04T06:06:07.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.352+0000 7f3d8ba00640 1 -- 172.21.15.159:0/3104232888 wait complete. 2024-04-04T06:06:07.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.351+0000 7fa4d9800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa4cc0e30c0 0x7fa4cc057d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.351+0000 7fa4d9800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa4cc0e30c0 0x7fa4cc057d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37186/0 (socket says 172.21.15.159:37186) 2024-04-04T06:06:07.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.351+0000 7fa4d9800640 1 -- 172.21.15.159:0/169321208 learned_addr learned my addr 172.21.15.159:0/169321208 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:07.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.352+0000 7fa4d8e00640 1 --2- 172.21.15.159:0/169321208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4cc03e840 0x7fa4cc053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.352+0000 7fa4d9800640 1 -- 172.21.15.159:0/169321208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4cc03e840 msgr2=0x7fa4cc053bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:07.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.352+0000 7fa4d9800640 1 --2- 172.21.15.159:0/169321208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4cc03e840 0x7fa4cc053bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.352+0000 7fa4d9800640 1 -- 172.21.15.159:0/169321208 >> v1:172.21.15.178:6789/0 conn(0x7fa4cc054130 legacy=0x7fa4cc0e6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:07.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.352+0000 7fa4d9800640 1 -- 172.21.15.159:0/169321208 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc058b30 con 0x7fa4cc0e30c0 2024-04-04T06:06:07.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.353+0000 7f3d8ba00640 1 Processor -- start 2024-04-04T06:06:07.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.354+0000 7fa4d9800640 1 --2- 172.21.15.159:0/169321208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4cc0e30c0 0x7fa4cc057d90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8009870 tx=0x7fa4c802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7323b0d90bd1e813 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.354+0000 7f3d8ba00640 1 -- start start 2024-04-04T06:06:07.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.354+0000 7fa4d3e00640 1 -- 172.21.15.159:0/169321208 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8034070 con 0x7fa4cc0e30c0 2024-04-04T06:06:07.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.354+0000 7fa4d3e00640 1 -- 172.21.15.159:0/169321208 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c8030840 con 0x7fa4cc0e30c0 2024-04-04T06:06:07.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.354+0000 7fa4d3e00640 1 -- 172.21.15.159:0/169321208 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8030c10 con 0x7fa4cc0e30c0 2024-04-04T06:06:07.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.355+0000 7f3d8ba00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d840cf6b0 0x7f3d84151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.355+0000 7f3d8ba00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d840d43d0 0x7f3d841559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.355+0000 7f3d8ba00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d8415e860 0x7f3d8415b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.355+0000 7f3d8ba00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8406dc00 con 0x7f3d8415e860 2024-04-04T06:06:07.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.355+0000 7f3d8ba00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3d840d2c50 con 0x7f3d840cf6b0 2024-04-04T06:06:07.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.355+0000 7f3d8ba00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3d8406dd80 con 0x7f3d840d43d0 2024-04-04T06:06:07.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.355+0000 7fa4dac00640 1 -- 172.21.15.159:0/169321208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4cc0e30c0 msgr2=0x7fa4cc057d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.355+0000 7fa4dac00640 1 --2- 172.21.15.159:0/169321208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4cc0e30c0 0x7fa4cc057d90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8009870 tx=0x7fa4c802b980 comp rx=0 tx=0).stop 2024-04-04T06:06:07.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.355+0000 7fb6f8600640 1 Processor -- start 2024-04-04T06:06:07.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.355+0000 7fa4dac00640 1 -- 172.21.15.159:0/169321208 shutdown_connections 2024-04-04T06:06:07.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.355+0000 7fa4dac00640 1 --2- 172.21.15.159:0/169321208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4cc03e840 0x7fa4cc053bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.356+0000 7fa4dac00640 1 --2- 172.21.15.159:0/169321208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4cc0e30c0 0x7fa4cc057d90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.356+0000 7fa4dac00640 1 -- 172.21.15.159:0/169321208 >> 172.21.15.159:0/169321208 conn(0x7fa4cc0559e0 msgr2=0x7fa4cc055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:07.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.356+0000 7fa4dac00640 1 -- 172.21.15.159:0/169321208 shutdown_connections 2024-04-04T06:06:07.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.356+0000 7fa4dac00640 1 -- 172.21.15.159:0/169321208 wait complete. 2024-04-04T06:06:07.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.356+0000 7f3d89000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d840cf6b0 0x7f3d84151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.356+0000 7f3d89000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d840cf6b0 0x7f3d84151a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:41204/0 (socket says 172.21.15.159:41204) 2024-04-04T06:06:07.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.356+0000 7f3d89000640 1 -- 172.21.15.159:0/368462380 learned_addr learned my addr 172.21.15.159:0/368462380 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:07.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.356+0000 7f3d89a00640 1 --2- 172.21.15.159:0/368462380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d8415e860 0x7f3d8415b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.356+0000 7f3d83e00640 1 --2- 172.21.15.159:0/368462380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d840d43d0 0x7f3d841559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.357+0000 7f3d89a00640 1 -- 172.21.15.159:0/368462380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d840d43d0 msgr2=0x7f3d841559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.357+0000 7f3d89a00640 1 --2- 172.21.15.159:0/368462380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d840d43d0 0x7f3d841559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.357+0000 7f3d89a00640 1 -- 172.21.15.159:0/368462380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d840cf6b0 msgr2=0x7f3d84151a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.357+0000 7f3d89a00640 1 --2- 172.21.15.159:0/368462380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d840cf6b0 0x7f3d84151a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.357+0000 7f3d89a00640 1 -- 172.21.15.159:0/368462380 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d84151f70 con 0x7f3d8415e860 2024-04-04T06:06:07.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.357+0000 7f3d89a00640 1 --2- 172.21.15.159:0/368462380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d8415e860 0x7f3d8415b960 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00c780 tx=0x7f3d7c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.358+0000 7f3d80c00640 1 -- 172.21.15.159:0/368462380 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c017070 con 0x7f3d8415e860 2024-04-04T06:06:07.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.358+0000 7f35db400640 1 -- 172.21.15.159:0/3317413281 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f35e0037d90 con 0x7f35e00e3150 2024-04-04T06:06:07.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.358+0000 7f3d80c00640 1 -- 172.21.15.159:0/368462380 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d7c003b70 con 0x7f3d8415e860 2024-04-04T06:06:07.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.358+0000 7f3d80c00640 1 -- 172.21.15.159:0/368462380 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c005070 con 0x7f3d8415e860 2024-04-04T06:06:07.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.358+0000 7f5ad7400640 1 -- 172.21.15.159:0/3112603924 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4035020 con 0x7f5ad80d37b0 2024-04-04T06:06:07.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.358+0000 7f5ad7400640 1 -- 172.21.15.159:0/3112603924 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac4031810 con 0x7f5ad80d37b0 2024-04-04T06:06:07.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.358+0000 7f5ad7400640 1 -- 172.21.15.159:0/3112603924 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4031be0 con 0x7f5ad80d37b0 2024-04-04T06:06:07.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.358+0000 7fa4dac00640 1 Processor -- start 2024-04-04T06:06:07.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.356+0000 7fb6f8600640 1 -- start start 2024-04-04T06:06:07.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.358+0000 7fb6f8600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb6f015e860 0x7fb6f006d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.358+0000 7fb6f8600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6f006dfa0 con 0x7fb6f006da30 2024-04-04T06:06:07.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.358+0000 7fb6f8600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6f006e240 con 0x7fb6f00d4ae0 2024-04-04T06:06:07.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.358+0000 7fb6f8600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb6f00cf220 con 0x7fb6f015e860 2024-04-04T06:06:07.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.358+0000 7fb6f5c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb6f015e860 0x7fb6f006d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.358+0000 7fb6f5c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb6f015e860 0x7fb6f006d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:41212/0 (socket says 172.21.15.159:41212) 2024-04-04T06:06:07.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.358+0000 7fb6f5c00640 1 -- 172.21.15.159:0/3641846492 learned_addr learned my addr 172.21.15.159:0/3641846492 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:07.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.358+0000 7fa4dac00640 1 -- start start 2024-04-04T06:06:07.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.359+0000 7fb6efe00640 1 -- 172.21.15.159:0/3641846492 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1622566285 0 0) 0x7fb6f006e240 con 0x7fb6f00d4ae0 2024-04-04T06:06:07.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.360+0000 7fb6efe00640 1 -- 172.21.15.159:0/3641846492 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6d8003520 con 0x7fb6f00d4ae0 2024-04-04T06:06:07.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.360+0000 7fb6f5c00640 1 -- 172.21.15.159:0/3641846492 >> v1:172.21.15.178:6789/0 conn(0x7fb6f00d4ae0 legacy=0x7fb6f00cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.360+0000 7f5ade000640 1 -- 172.21.15.159:0/3112603924 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ad80d37b0 msgr2=0x7f5ad80b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.360+0000 7f5ade000640 1 --2- 172.21.15.159:0/3112603924 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ad80d37b0 0x7f5ad80b62b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40097a0 tx=0x7f5ac402b8b0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.361+0000 7f5ade000640 1 -- 172.21.15.159:0/3112603924 shutdown_connections 2024-04-04T06:06:07.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.361+0000 7f5ade000640 1 --2- 172.21.15.159:0/3112603924 >> v2:172.21.15.159:3300/0 conn(0x7f5ad80b67f0 0x7f5ad80b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.361+0000 7f5ade000640 1 --2- 172.21.15.159:0/3112603924 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ad80d37b0 0x7f5ad80b62b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.361+0000 7f5ade000640 1 -- 172.21.15.159:0/3112603924 >> 172.21.15.159:0/3112603924 conn(0x7f5ad80676a0 msgr2=0x7f5ad8067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:07.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.360+0000 7f3d8ba00640 1 -- 172.21.15.159:0/368462380 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d840b7c40 con 0x7f3d8415e860 2024-04-04T06:06:07.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.360+0000 7f3d8ba00640 1 -- 172.21.15.159:0/368462380 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d840b98e0 con 0x7f3d8415e860 2024-04-04T06:06:07.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.360+0000 7fa4dac00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4cc03e840 0x7fa4cc04eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.360+0000 7fa4dac00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4cc054130 0x7fa4cc051fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.360+0000 7fa4dac00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4cc0e30c0 0x7fa4cc0524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.360+0000 7fa4dac00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc0e34a0 con 0x7fa4cc0e30c0 2024-04-04T06:06:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.360+0000 7fa4dac00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc03bfc0 con 0x7fa4cc03e840 2024-04-04T06:06:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.360+0000 7fa4dac00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa4cc0078e0 con 0x7fa4cc054130 2024-04-04T06:06:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.360+0000 7fa4da200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4cc0e30c0 0x7fa4cc0524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.360+0000 7fa4da200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4cc0e30c0 0x7fa4cc0524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37208/0 (socket says 172.21.15.159:37208) 2024-04-04T06:06:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.360+0000 7fa4da200640 1 -- 172.21.15.159:0/2988567757 learned_addr learned my addr 172.21.15.159:0/2988567757 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.361+0000 7fa4d9800640 1 --2- 172.21.15.159:0/2988567757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4cc03e840 0x7fa4cc04eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.361+0000 7fa4d8e00640 1 --2- 172.21.15.159:0/2988567757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4cc054130 0x7fa4cc051fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.361+0000 7fa4da200640 1 -- 172.21.15.159:0/2988567757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4cc054130 msgr2=0x7fa4cc051fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.361+0000 7fa4da200640 1 --2- 172.21.15.159:0/2988567757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4cc054130 0x7fa4cc051fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.361+0000 7fa4da200640 1 -- 172.21.15.159:0/2988567757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4cc03e840 msgr2=0x7fa4cc04eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.361+0000 7fa4da200640 1 --2- 172.21.15.159:0/2988567757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4cc03e840 0x7fa4cc04eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.361+0000 7fa4da200640 1 -- 172.21.15.159:0/2988567757 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc04efe0 con 0x7fa4cc0e30c0 2024-04-04T06:06:07.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.362+0000 7fa4da200640 1 --2- 172.21.15.159:0/2988567757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4cc0e30c0 0x7fa4cc0524f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa4d406a3e0 tx=0x7fa4d406c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.362+0000 7f5ade000640 1 -- 172.21.15.159:0/3112603924 shutdown_connections 2024-04-04T06:06:07.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.362+0000 7fa4d1600640 1 -- 172.21.15.159:0/2988567757 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d40733c0 con 0x7fa4cc0e30c0 2024-04-04T06:06:07.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.362+0000 7fa4d1600640 1 -- 172.21.15.159:0/2988567757 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4d40735a0 con 0x7fa4cc0e30c0 2024-04-04T06:06:07.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.363+0000 7f3d8ba00640 1 -- 172.21.15.159:0/368462380 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d840685c0 con 0x7f3d8415e860 2024-04-04T06:06:07.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.360+0000 7fb6f5c00640 1 -- 172.21.15.159:0/3641846492 >> v1:172.21.15.159:6789/0 conn(0x7fb6f006da30 legacy=0x7fb6f00d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.360+0000 7fb6f5c00640 1 -- 172.21.15.159:0/3641846492 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f006e520 con 0x7fb6f015e860 2024-04-04T06:06:07.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.362+0000 7fa4d1600640 1 -- 172.21.15.159:0/2988567757 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4073a50 con 0x7fa4cc0e30c0 2024-04-04T06:06:07.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.364+0000 7f5ade000640 1 -- 172.21.15.159:0/3112603924 wait complete. 2024-04-04T06:06:07.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.364+0000 7fb6f5c00640 1 --2- 172.21.15.159:0/3641846492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6f015e860 0x7fb6f006d4f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009870 tx=0x7fb6e402ba80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bec72d0611a02843 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.362+0000 7fa4dac00640 1 -- 172.21.15.159:0/2988567757 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4cc0e3830 con 0x7fa4cc0e30c0 2024-04-04T06:06:07.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.362+0000 7fa4dac00640 1 -- 172.21.15.159:0/2988567757 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4cc0e3df0 con 0x7fa4cc0e30c0 2024-04-04T06:06:07.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.365+0000 7fb6efe00640 1 -- 172.21.15.159:0/3641846492 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e402d840 con 0x7fb6f015e860 2024-04-04T06:06:07.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.365+0000 7fb6efe00640 1 -- 172.21.15.159:0/3641846492 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e402d9e0 con 0x7fb6f015e860 2024-04-04T06:06:07.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.365+0000 7fb6efe00640 1 -- 172.21.15.159:0/3641846492 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e402d840 con 0x7fb6f015e860 2024-04-04T06:06:07.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.365+0000 7fa4dac00640 1 -- 172.21.15.159:0/2988567757 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b0005290 con 0x7fa4cc0e30c0 2024-04-04T06:06:07.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.367+0000 7fb6f8600640 1 -- 172.21.15.159:0/3641846492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6f015e860 msgr2=0x7fb6f006d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.367+0000 7fb6f8600640 1 --2- 172.21.15.159:0/3641846492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6f015e860 0x7fb6f006d4f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009870 tx=0x7fb6e402ba80 comp rx=0 tx=0).stop 2024-04-04T06:06:07.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.367+0000 7fa4d1600640 1 -- 172.21.15.159:0/2988567757 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4d409c3f0 con 0x7fa4cc0e30c0 2024-04-04T06:06:07.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.367+0000 7fb6f8600640 1 -- 172.21.15.159:0/3641846492 shutdown_connections 2024-04-04T06:06:07.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.367+0000 7fb6f8600640 1 --2- 172.21.15.159:0/3641846492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6f015e860 0x7fb6f006d4f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.367+0000 7fb6f8600640 1 -- 172.21.15.159:0/3641846492 >> 172.21.15.159:0/3641846492 conn(0x7fb6f009d7d0 msgr2=0x7fb6f008bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:07.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.367+0000 7fb6f8600640 1 -- 172.21.15.159:0/3641846492 shutdown_connections 2024-04-04T06:06:07.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.368+0000 7fb6f8600640 1 -- 172.21.15.159:0/3641846492 wait complete. 2024-04-04T06:06:07.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.368+0000 7fa4d1600640 1 --2- 172.21.15.159:0/2988567757 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa4b406c9c0 0x7fa4b406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.368+0000 7fa4d1600640 1 -- 172.21.15.159:0/2988567757 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa4d4107040 con 0x7fa4cc0e30c0 2024-04-04T06:06:07.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.370+0000 7fb6f8600640 1 Processor -- start 2024-04-04T06:06:07.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.370+0000 7fb6f8600640 1 -- start start 2024-04-04T06:06:07.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.372+0000 7fb6f8600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6f006da30 0x7fb6f0151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.372+0000 7f5ade000640 1 Processor -- start 2024-04-04T06:06:07.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.372+0000 7fa4d9800640 1 --2- 172.21.15.159:0/2988567757 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa4b406c9c0 0x7fa4b406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.372+0000 7fb6f8600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6f00d4ae0 0x7fb6f00ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.369+0000 7f3d80c00640 1 -- 172.21.15.159:0/368462380 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d7c01a070 con 0x7f3d8415e860 2024-04-04T06:06:07.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.370+0000 7f3d80c00640 1 --2- 172.21.15.159:0/368462380 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3d6406c9f0 0x7f3d6406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.370+0000 7f3d80c00640 1 -- 172.21.15.159:0/368462380 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3d7c08b150 con 0x7f3d8415e860 2024-04-04T06:06:07.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.372+0000 7fb6f8600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6f015e860 0x7fb6f00cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.372+0000 7fb6f8600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f00cef30 con 0x7fb6f00d4ae0 2024-04-04T06:06:07.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.372+0000 7fb6f8600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f00cedb0 con 0x7fb6f006da30 2024-04-04T06:06:07.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.372+0000 7fb6f8600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f00cfdc0 con 0x7fb6f015e860 2024-04-04T06:06:07.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.372+0000 7f5ade000640 1 -- start start 2024-04-04T06:06:07.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.373+0000 7fb6f5c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6f006da30 0x7fb6f0151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.373+0000 7f5ade000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad80b67f0 0x7f5ad815a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.373+0000 7f5ade000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ad80d37b0 0x7f5ad80cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.374+0000 7f3d89000640 1 --2- 172.21.15.159:0/368462380 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3d6406c9f0 0x7f3d6406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.374+0000 7fa4d9800640 1 --2- 172.21.15.159:0/2988567757 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa4b406c9c0 0x7fa4b406ee80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8036000 tx=0x7fa4c803b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.374+0000 7fb6f5200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6f00d4ae0 0x7fb6f00ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.374+0000 7fb6f5200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6f00d4ae0 0x7fb6f00ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37212/0 (socket says 172.21.15.159:37212) 2024-04-04T06:06:07.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.374+0000 7fb6f5200640 1 -- 172.21.15.159:0/4139637500 learned_addr learned my addr 172.21.15.159:0/4139637500 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:07.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.374+0000 7f5ade000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ad815e4c0 0x7f5ad80cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.374+0000 7f5ade000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad80d2fb0 con 0x7f5ad815e4c0 2024-04-04T06:06:07.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.374+0000 7f5ade000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad80d3320 con 0x7f5ad80b67f0 2024-04-04T06:06:07.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.374+0000 7f5ade000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad80d2350 con 0x7f5ad80d37b0 2024-04-04T06:06:07.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.374+0000 7f5ad7e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ad80d37b0 0x7f5ad80cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.375+0000 7f5ad7e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ad80d37b0 0x7f5ad80cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:55190/0 (socket says 172.21.15.159:55190) 2024-04-04T06:06:07.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.375+0000 7f5ad7e00640 1 -- 172.21.15.159:0/603593680 learned_addr learned my addr 172.21.15.159:0/603593680 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:07.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.375+0000 7f5adcc00640 1 --2- 172.21.15.159:0/603593680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad80b67f0 0x7f5ad815a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.375+0000 7f5ad7e00640 1 -- 172.21.15.159:0/603593680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad80b67f0 msgr2=0x7f5ad815a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.375+0000 7f5ad7e00640 1 --2- 172.21.15.159:0/603593680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad80b67f0 0x7f5ad815a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.374+0000 7fb6f5200640 1 -- 172.21.15.159:0/4139637500 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6f015e860 msgr2=0x7fb6f00cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.374+0000 7fb6f5200640 1 --2- 172.21.15.159:0/4139637500 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6f015e860 0x7fb6f00cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.376+0000 7f3d80c00640 1 -- 172.21.15.159:0/368462380 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3d840d4d20 con 0x7f3d8415e860 2024-04-04T06:06:07.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.374+0000 7fb6f5200640 1 -- 172.21.15.159:0/4139637500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6f006da30 msgr2=0x7fb6f0151b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.374+0000 7fb6f5200640 1 --2- 172.21.15.159:0/4139637500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6f006da30 0x7fb6f0151b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.374+0000 7fb6f5200640 1 -- 172.21.15.159:0/4139637500 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f0152070 con 0x7fb6f00d4ae0 2024-04-04T06:06:07.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.375+0000 7fb6f5200640 1 --2- 172.21.15.159:0/4139637500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6f00d4ae0 0x7fb6f00ccc80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00c480 tx=0x7fb6dc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.375+0000 7f5ad7e00640 1 -- 172.21.15.159:0/603593680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ad815e4c0 msgr2=0x7f5ad80cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.376+0000 7f5add600640 1 --2- 172.21.15.159:0/603593680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ad815e4c0 0x7f5ad80cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.376+0000 7f5ad7e00640 1 --2- 172.21.15.159:0/603593680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ad815e4c0 0x7f5ad80cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.376+0000 7f5ad7e00640 1 -- 172.21.15.159:0/603593680 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad81f76e0 con 0x7f5ad80d37b0 2024-04-04T06:06:07.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.376+0000 7f5add600640 1 --2- 172.21.15.159:0/603593680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ad815e4c0 0x7f5ad80cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:07.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.377+0000 7f5ad7e00640 1 --2- 172.21.15.159:0/603593680 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ad80d37b0 0x7f5ad80cd570 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009890 tx=0x7f5ac40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.378+0000 7f5ad4c00640 1 -- 172.21.15.159:0/603593680 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4035040 con 0x7f5ad80d37b0 2024-04-04T06:06:07.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.378+0000 7f3d89000640 1 --2- 172.21.15.159:0/368462380 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3d6406c9f0 0x7f3d6406eeb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3d78005880 tx=0x7f3d78009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.378+0000 7f5ad4c00640 1 -- 172.21.15.159:0/603593680 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac4046480 con 0x7f5ad80d37b0 2024-04-04T06:06:07.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.378+0000 7f5ade000640 1 -- 172.21.15.159:0/603593680 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad81f79a0 con 0x7f5ad80d37b0 2024-04-04T06:06:07.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.378+0000 7f5ade000640 1 -- 172.21.15.159:0/603593680 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad81f7f60 con 0x7f5ad80d37b0 2024-04-04T06:06:07.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.379+0000 7f5ad4c00640 1 -- 172.21.15.159:0/603593680 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4046870 con 0x7f5ad80d37b0 2024-04-04T06:06:07.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.379+0000 7fb6ed600640 1 -- 172.21.15.159:0/4139637500 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc0052d0 con 0x7fb6f00d4ae0 2024-04-04T06:06:07.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.379+0000 7fb6ed600640 1 -- 172.21.15.159:0/4139637500 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6dc005470 con 0x7fb6f00d4ae0 2024-04-04T06:06:07.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.379+0000 7fb6ed600640 1 -- 172.21.15.159:0/4139637500 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc012830 con 0x7fb6f00d4ae0 2024-04-04T06:06:07.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.380+0000 7fb6f8600640 1 -- 172.21.15.159:0/4139637500 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f00cdb00 con 0x7fb6f00d4ae0 2024-04-04T06:06:07.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.380+0000 7fb6f8600640 1 -- 172.21.15.159:0/4139637500 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f00ce0c0 con 0x7fb6f00d4ae0 2024-04-04T06:06:07.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.381+0000 7fa4d1600640 1 -- 172.21.15.159:0/2988567757 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa4d40a2080 con 0x7fa4cc0e30c0 2024-04-04T06:06:07.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.382+0000 7fb6f8600640 1 -- 172.21.15.159:0/4139637500 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6cc005290 con 0x7fb6f00d4ae0 2024-04-04T06:06:07.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.382+0000 7f5ade000640 1 -- 172.21.15.159:0/603593680 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab4005290 con 0x7f5ad80d37b0 2024-04-04T06:06:07.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.383+0000 7f5ad4c00640 1 -- 172.21.15.159:0/603593680 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ac4038020 con 0x7f5ad80d37b0 2024-04-04T06:06:07.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.385+0000 7f5ad4c00640 1 --2- 172.21.15.159:0/603593680 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5ab806c8f0 0x7f5ab806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.385+0000 7f5ad4c00640 1 -- 172.21.15.159:0/603593680 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5ac40adc40 con 0x7f5ad80d37b0 2024-04-04T06:06:07.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.386+0000 7f5adcc00640 1 --2- 172.21.15.159:0/603593680 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5ab806c8f0 0x7f5ab806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.387+0000 7f5adcc00640 1 --2- 172.21.15.159:0/603593680 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5ab806c8f0 0x7f5ab806edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5acc000c00 tx=0x7f5acc007b50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.385+0000 7fb6ed600640 1 -- 172.21.15.159:0/4139637500 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6dc0129d0 con 0x7fb6f00d4ae0 2024-04-04T06:06:07.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.387+0000 7fb6ed600640 1 --2- 172.21.15.159:0/4139637500 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb6d006c9c0 0x7fb6d006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.387+0000 7fb6ed600640 1 -- 172.21.15.159:0/4139637500 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb6dc08b750 con 0x7fb6f00d4ae0 2024-04-04T06:06:07.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.388+0000 7fb6f5c00640 1 --2- 172.21.15.159:0/4139637500 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb6d006c9c0 0x7fb6d006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.389+0000 7fb6f5c00640 1 --2- 172.21.15.159:0/4139637500 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb6d006c9c0 0x7fb6d006ee80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4036000 tx=0x7fb6e4008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.397+0000 7fb6ed600640 1 -- 172.21.15.159:0/4139637500 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb6dc0182b0 con 0x7fb6f00d4ae0 2024-04-04T06:06:07.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.401+0000 7f5ad4c00640 1 -- 172.21.15.159:0/603593680 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5ac407c080 con 0x7f5ad80d37b0 2024-04-04T06:06:07.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.423+0000 7f32f1a00640 1 Processor -- start 2024-04-04T06:06:07.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.423+0000 7f32f1a00640 1 -- start start 2024-04-04T06:06:07.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.425+0000 7f32f1a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f32e40e30b0 0x7f32e4057d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.425+0000 7f32f1a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f32e40542f0 0x7f32e40068c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.425+0000 7f32f1a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32e40582c0 con 0x7f32e403e830 2024-04-04T06:06:07.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.425+0000 7f32f1a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f32e4058560 con 0x7f32e40542f0 2024-04-04T06:06:07.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.425+0000 7f32f1a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f32e4058840 con 0x7f32e40e30b0 2024-04-04T06:06:07.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.425+0000 7f2531c00640 1 Processor -- start 2024-04-04T06:06:07.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.426+0000 7f2531c00640 1 -- start start 2024-04-04T06:06:07.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.426+0000 7f32eaa00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f32e40e30b0 0x7f32e4057d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.426+0000 7f32ea000640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f32e403e830 0x7f32e4053be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.159:44302/0 (socket says 172.21.15.159:44302) 2024-04-04T06:06:07.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.426+0000 7f32ea000640 1 -- 172.21.15.159:0/2211697746 learned_addr learned my addr 172.21.15.159:0/2211697746 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:07.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.426+0000 7fe41c600640 1 Processor -- start 2024-04-04T06:06:07.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.426+0000 7fe41c600640 1 -- start start 2024-04-04T06:06:07.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.427+0000 7f32eb400640 1 --2- 172.21.15.159:0/2211697746 >> v2:172.21.15.159:3300/0 conn(0x7f32e40542f0 0x7f32e40068c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.427+0000 7f2531c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f252c15e860 0x7f252c06d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.427+0000 7f2531c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f252c0d4ae0 0x7f252c0cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.427+0000 7f2531c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f252c06dfa0 con 0x7f252c06da30 2024-04-04T06:06:07.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.427+0000 7f2531c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f252c0cf220 con 0x7f252c15e860 2024-04-04T06:06:07.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.427+0000 7f2531c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f252c06e240 con 0x7f252c0d4ae0 2024-04-04T06:06:07.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.428+0000 7f252a000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f252c06da30 0x7f252c0d43d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:46000/0 (socket says 172.21.15.159:46000) 2024-04-04T06:06:07.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.428+0000 7f252a000640 1 -- 172.21.15.159:0/4193952521 learned_addr learned my addr 172.21.15.159:0/4193952521 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:07.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.428+0000 7f252b400640 1 --2- 172.21.15.159:0/4193952521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f252c0d4ae0 0x7f252c0cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.427+0000 7f32eaa00640 1 -- 172.21.15.159:0/2211697746 >> v2:172.21.15.159:3300/0 conn(0x7f32e40542f0 msgr2=0x7f32e40068c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:07.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.427+0000 7f32eaa00640 1 --2- 172.21.15.159:0/2211697746 >> v2:172.21.15.159:3300/0 conn(0x7f32e40542f0 0x7f32e40068c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.427+0000 7f32e9600640 1 -- 172.21.15.159:0/2211697746 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3924934038 0 0) 0x7f32e40582c0 con 0x7f32e403e830 2024-04-04T06:06:07.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.427+0000 7f32eaa00640 1 -- 172.21.15.159:0/2211697746 >> v1:172.21.15.178:6789/0 conn(0x7f32e403e830 legacy=0x7f32e4053be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.427+0000 7f32eaa00640 1 -- 172.21.15.159:0/2211697746 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e4058b20 con 0x7f32e40e30b0 2024-04-04T06:06:07.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.428+0000 7fe41c600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe4140d0a00 0x7fe4141666d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.429+0000 7fe41c600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe414166c10 0x7fe41406e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.429+0000 7fe41c600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4140cefb0 con 0x7fe41415d060 2024-04-04T06:06:07.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.429+0000 7fe41c600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe414167460 con 0x7fe414166c10 2024-04-04T06:06:07.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.429+0000 7fe41c600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe4140cf250 con 0x7fe4140d0a00 2024-04-04T06:06:07.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.428+0000 7f252aa00640 1 --2- 172.21.15.159:0/4193952521 >> v2:172.21.15.159:3300/0 conn(0x7f252c15e860 0x7f252c06d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.429+0000 7fe419c00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fe41415d060 0x7fe4140d02f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:46002/0 (socket says 172.21.15.159:46002) 2024-04-04T06:06:07.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.429+0000 7fe419c00640 1 -- 172.21.15.159:0/3102804278 learned_addr learned my addr 172.21.15.159:0/3102804278 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:07.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.430+0000 7f252aa00640 1 -- 172.21.15.159:0/4193952521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f252c0d4ae0 msgr2=0x7f252c0cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:07.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.430+0000 7f252aa00640 1 --2- 172.21.15.159:0/4193952521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f252c0d4ae0 0x7f252c0cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.430+0000 7f252aa00640 1 -- 172.21.15.159:0/4193952521 >> v1:172.21.15.159:6789/0 conn(0x7f252c06da30 legacy=0x7f252c0d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.430+0000 7f252aa00640 1 -- 172.21.15.159:0/4193952521 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252c06e520 con 0x7f252c15e860 2024-04-04T06:06:07.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.429+0000 7f32eaa00640 1 --2- 172.21.15.159:0/2211697746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32e40e30b0 0x7f32e4057d80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f32e4056950 tx=0x7f32dc02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=745e60026e6ac7d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.429+0000 7fe419200640 1 --2- 172.21.15.159:0/3102804278 >> v2:172.21.15.178:3300/0 conn(0x7fe4140d0a00 0x7fe4141666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.430+0000 7fe41a600640 1 --2- 172.21.15.159:0/3102804278 >> v2:172.21.15.159:3300/0 conn(0x7fe414166c10 0x7fe41406e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.430+0000 7f32e9600640 1 -- 172.21.15.159:0/2211697746 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc034070 con 0x7f32e40e30b0 2024-04-04T06:06:07.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.430+0000 7f32e9600640 1 -- 172.21.15.159:0/2211697746 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32dc030840 con 0x7f32e40e30b0 2024-04-04T06:06:07.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.430+0000 7f32e9600640 1 -- 172.21.15.159:0/2211697746 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc034070 con 0x7f32e40e30b0 2024-04-04T06:06:07.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.431+0000 7f32f1a00640 1 -- 172.21.15.159:0/2211697746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32e40e30b0 msgr2=0x7f32e4057d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.431+0000 7f32f1a00640 1 --2- 172.21.15.159:0/2211697746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32e40e30b0 0x7f32e4057d80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f32e4056950 tx=0x7f32dc02b980 comp rx=0 tx=0).stop 2024-04-04T06:06:07.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.430+0000 7fe419200640 1 -- 172.21.15.159:0/3102804278 >> v2:172.21.15.159:3300/0 conn(0x7fe414166c10 msgr2=0x7fe41406e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:07.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.430+0000 7fe419200640 1 --2- 172.21.15.159:0/3102804278 >> v2:172.21.15.159:3300/0 conn(0x7fe414166c10 0x7fe41406e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.430+0000 7fe419200640 1 -- 172.21.15.159:0/3102804278 >> v1:172.21.15.159:6789/0 conn(0x7fe41415d060 legacy=0x7fe4140d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.430+0000 7fe419200640 1 -- 172.21.15.159:0/3102804278 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4140cf530 con 0x7fe4140d0a00 2024-04-04T06:06:07.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.431+0000 7f32f1a00640 1 -- 172.21.15.159:0/2211697746 shutdown_connections 2024-04-04T06:06:07.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.431+0000 7f32f1a00640 1 --2- 172.21.15.159:0/2211697746 >> v2:172.21.15.159:3300/0 conn(0x7f32e40542f0 0x7f32e40068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.431+0000 7f32f1a00640 1 --2- 172.21.15.159:0/2211697746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32e40e30b0 0x7f32e4057d80 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f32e4056950 tx=0x7f32dc02b980 comp rx=0 tx=0).stop 2024-04-04T06:06:07.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.431+0000 7f32f1a00640 1 -- 172.21.15.159:0/2211697746 >> 172.21.15.159:0/2211697746 conn(0x7f32e40559d0 msgr2=0x7f32e4055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:07.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.432+0000 7fe41a600640 1 --2- 172.21.15.159:0/3102804278 >> v2:172.21.15.159:3300/0 conn(0x7fe414166c10 0x7fe41406e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:06:07.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.432+0000 7f32f1a00640 1 -- 172.21.15.159:0/2211697746 shutdown_connections 2024-04-04T06:06:07.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.432+0000 7f252aa00640 1 --2- 172.21.15.159:0/4193952521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f252c15e860 0x7f252c06d4f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f25140097a0 tx=0x7f251402b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997210f539a8f068 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.433+0000 7f2529600640 1 -- 172.21.15.159:0/4193952521 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514037070 con 0x7f252c15e860 2024-04-04T06:06:07.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.433+0000 7f2529600640 1 -- 172.21.15.159:0/4193952521 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2514030710 con 0x7f252c15e860 2024-04-04T06:06:07.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.433+0000 7f2529600640 1 -- 172.21.15.159:0/4193952521 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514037070 con 0x7f252c15e860 2024-04-04T06:06:07.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.434+0000 7f2531c00640 1 -- 172.21.15.159:0/4193952521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f252c15e860 msgr2=0x7f252c06d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.434+0000 7f2531c00640 1 --2- 172.21.15.159:0/4193952521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f252c15e860 0x7f252c06d4f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f25140097a0 tx=0x7f251402b710 comp rx=0 tx=0).stop 2024-04-04T06:06:07.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.434+0000 7f2531c00640 1 -- 172.21.15.159:0/4193952521 shutdown_connections 2024-04-04T06:06:07.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.434+0000 7f2531c00640 1 --2- 172.21.15.159:0/4193952521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f252c0d4ae0 0x7f252c0cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.434+0000 7f2531c00640 1 --2- 172.21.15.159:0/4193952521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f252c15e860 0x7f252c06d4f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.434+0000 7f2531c00640 1 -- 172.21.15.159:0/4193952521 >> 172.21.15.159:0/4193952521 conn(0x7f252c09d7d0 msgr2=0x7f252c08bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:07.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.432+0000 7f32f1a00640 1 -- 172.21.15.159:0/2211697746 wait complete. 2024-04-04T06:06:07.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.435+0000 7f32f1a00640 1 Processor -- start 2024-04-04T06:06:07.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.433+0000 7fe419200640 1 --2- 172.21.15.159:0/3102804278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4140d0a00 0x7fe4141666d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe4140af750 tx=0x7fe40002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=989ed49a64b6ec41 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.434+0000 7fe413e00640 1 -- 172.21.15.159:0/3102804278 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400035020 con 0x7fe4140d0a00 2024-04-04T06:06:07.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.434+0000 7fe413e00640 1 -- 172.21.15.159:0/3102804278 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe400031830 con 0x7fe4140d0a00 2024-04-04T06:06:07.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.434+0000 7fe413e00640 1 -- 172.21.15.159:0/3102804278 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400031c00 con 0x7fe4140d0a00 2024-04-04T06:06:07.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.435+0000 7fe41c600640 1 -- 172.21.15.159:0/3102804278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4140d0a00 msgr2=0x7fe4141666d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.435+0000 7fe41c600640 1 --2- 172.21.15.159:0/3102804278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4140d0a00 0x7fe4141666d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe4140af750 tx=0x7fe40002b8d0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.435+0000 7f32f1a00640 1 -- start start 2024-04-04T06:06:07.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.435+0000 7f2531c00640 1 -- 172.21.15.159:0/4193952521 shutdown_connections 2024-04-04T06:06:07.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.435+0000 7f2531c00640 1 -- 172.21.15.159:0/4193952521 wait complete. 2024-04-04T06:06:07.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.436+0000 7fe41c600640 1 -- 172.21.15.159:0/3102804278 shutdown_connections 2024-04-04T06:06:07.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.436+0000 7fe41c600640 1 --2- 172.21.15.159:0/3102804278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4140d0a00 0x7fe4141666d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.436+0000 7fe41c600640 1 --2- 172.21.15.159:0/3102804278 >> v2:172.21.15.159:3300/0 conn(0x7fe414166c10 0x7fe41406e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.436+0000 7fe41c600640 1 -- 172.21.15.159:0/3102804278 >> 172.21.15.159:0/3102804278 conn(0x7fe41406b750 msgr2=0x7fe41406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:07.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.436+0000 7f32f1a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32e403e830 0x7f32e40cff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.436+0000 7f32f1a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32e40542f0 0x7f32e40dcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.436+0000 7f32f1a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32e40dd520 0x7f32e40e0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.436+0000 7f32f1a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f32e40e3490 con 0x7f32e403e830 2024-04-04T06:06:07.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.436+0000 7f32f1a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f32e403bfb0 con 0x7f32e40542f0 2024-04-04T06:06:07.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.436+0000 7f32f1a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f32e4006e00 con 0x7f32e40dd520 2024-04-04T06:06:07.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.437+0000 7f32eaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32e403e830 0x7f32e40cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.437+0000 7fe41c600640 1 -- 172.21.15.159:0/3102804278 shutdown_connections 2024-04-04T06:06:07.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.437+0000 7f32ea000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32e40542f0 0x7f32e40dcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.437+0000 7f32ea000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32e40542f0 0x7f32e40dcfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:41270/0 (socket says 172.21.15.159:41270) 2024-04-04T06:06:07.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.437+0000 7f32ea000640 1 -- 172.21.15.159:0/854206465 learned_addr learned my addr 172.21.15.159:0/854206465 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:07.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.437+0000 7f32eb400640 1 --2- 172.21.15.159:0/854206465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32e40dd520 0x7f32e40e0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.438+0000 7fe41c600640 1 -- 172.21.15.159:0/3102804278 wait complete. 2024-04-04T06:06:07.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.438+0000 7f32ea000640 1 -- 172.21.15.159:0/854206465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32e40dd520 msgr2=0x7f32e40e0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.438+0000 7f32ea000640 1 --2- 172.21.15.159:0/854206465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32e40dd520 0x7f32e40e0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.438+0000 7f32ea000640 1 -- 172.21.15.159:0/854206465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32e403e830 msgr2=0x7f32e40cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.438+0000 7f32ea000640 1 --2- 172.21.15.159:0/854206465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32e403e830 0x7f32e40cff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.438+0000 7f32ea000640 1 -- 172.21.15.159:0/854206465 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e40deb00 con 0x7f32e40542f0 2024-04-04T06:06:07.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.438+0000 7f32ea000640 1 --2- 172.21.15.159:0/854206465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32e40542f0 0x7f32e40dcfe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f32d800c460 tx=0x7f32d800c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.439+0000 7f2531c00640 1 Processor -- start 2024-04-04T06:06:07.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.439+0000 7f32e2a00640 1 -- 172.21.15.159:0/854206465 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8005ce0 con 0x7f32e40542f0 2024-04-04T06:06:07.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.439+0000 7f32e2a00640 1 -- 172.21.15.159:0/854206465 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d8007070 con 0x7f32e40542f0 2024-04-04T06:06:07.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.439+0000 7f32e2a00640 1 -- 172.21.15.159:0/854206465 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8007460 con 0x7f32e40542f0 2024-04-04T06:06:07.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.439+0000 7f32f1a00640 1 -- 172.21.15.159:0/854206465 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e40dda60 con 0x7f32e40542f0 2024-04-04T06:06:07.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.439+0000 7f32f1a00640 1 -- 172.21.15.159:0/854206465 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e40de020 con 0x7f32e40542f0 2024-04-04T06:06:07.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.440+0000 7fe41c600640 1 Processor -- start 2024-04-04T06:06:07.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.439+0000 7f2531c00640 1 -- start start 2024-04-04T06:06:07.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.440+0000 7f2531c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f252c06da30 0x7f252c0caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.440+0000 7f2531c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f252c0d4ae0 0x7f252c0cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.441+0000 7f2531c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f252c15e860 0x7f252c0cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.441+0000 7f252aa00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f252c06da30 0x7f252c0caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.441+0000 7f252a000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f252c0d4ae0 0x7f252c0cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.441+0000 7f252a000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f252c0d4ae0 0x7f252c0cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:41278/0 (socket says 172.21.15.159:41278) 2024-04-04T06:06:07.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.441+0000 7f252a000640 1 -- 172.21.15.159:0/4033952621 learned_addr learned my addr 172.21.15.159:0/4033952621 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:07.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.442+0000 7f2531c00640 1 -- 172.21.15.159:0/4033952621 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f252c0cef30 con 0x7f252c15e860 2024-04-04T06:06:07.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.442+0000 7f2531c00640 1 -- 172.21.15.159:0/4033952621 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f252c0cedb0 con 0x7f252c0d4ae0 2024-04-04T06:06:07.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.442+0000 7f2531c00640 1 -- 172.21.15.159:0/4033952621 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f252c0cfbf0 con 0x7f252c06da30 2024-04-04T06:06:07.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.442+0000 7f252b400640 1 --2- 172.21.15.159:0/4033952621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f252c15e860 0x7f252c0cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.442+0000 7f32f1a00640 1 -- 172.21.15.159:0/854206465 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c4005290 con 0x7f32e40542f0 2024-04-04T06:06:07.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.441+0000 7fe41c600640 1 -- start start 2024-04-04T06:06:07.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.442+0000 7fe41c600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4140d0a00 0x7fe41409d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.442+0000 7fe41c600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe41415d060 0x7fe414162a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.442+0000 7fe41c600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe414166c10 0x7fe41409ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.442+0000 7fe41c600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe414167200 con 0x7fe4140d0a00 2024-04-04T06:06:07.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.442+0000 7fe41c600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe414167080 con 0x7fe414166c10 2024-04-04T06:06:07.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.442+0000 7fe41c600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe4140cfd30 con 0x7fe41415d060 2024-04-04T06:06:07.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.443+0000 7f252aa00640 1 -- 172.21.15.159:0/4033952621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f252c0d4ae0 msgr2=0x7f252c0cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.443+0000 7f252aa00640 1 --2- 172.21.15.159:0/4033952621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f252c0d4ae0 0x7f252c0cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.443+0000 7f252aa00640 1 -- 172.21.15.159:0/4033952621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f252c15e860 msgr2=0x7f252c0cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.443+0000 7fe419c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4140d0a00 0x7fe41409d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.443+0000 7f252aa00640 1 --2- 172.21.15.159:0/4033952621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f252c15e860 0x7f252c0cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.443+0000 7f252aa00640 1 -- 172.21.15.159:0/4033952621 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252c0b18d0 con 0x7f252c06da30 2024-04-04T06:06:07.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.443+0000 7f252b400640 1 --2- 172.21.15.159:0/4033952621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f252c15e860 0x7f252c0cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:06:07.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.443+0000 7fe419c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4140d0a00 0x7fe41409d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37294/0 (socket says 172.21.15.159:37294) 2024-04-04T06:06:07.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.443+0000 7fe419200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe41415d060 0x7fe414162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.443+0000 7fe419200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe41415d060 0x7fe414162a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:55220/0 (socket says 172.21.15.159:55220) 2024-04-04T06:06:07.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.443+0000 7fe419200640 1 -- 172.21.15.159:0/340553130 learned_addr learned my addr 172.21.15.159:0/340553130 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:07.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.443+0000 7fe41a600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe414166c10 0x7fe41409ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.444+0000 7f252aa00640 1 --2- 172.21.15.159:0/4033952621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f252c06da30 0x7f252c0caa20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2514032000 tx=0x7f25140046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.444+0000 7fe41a600640 1 -- 172.21.15.159:0/340553130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe41415d060 msgr2=0x7fe414162a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.444+0000 7fe41a600640 1 --2- 172.21.15.159:0/340553130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe41415d060 0x7fe414162a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.444+0000 7fe41a600640 1 -- 172.21.15.159:0/340553130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4140d0a00 msgr2=0x7fe41409d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.444+0000 7fe41a600640 1 --2- 172.21.15.159:0/340553130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4140d0a00 0x7fe41409d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.444+0000 7fe41a600640 1 -- 172.21.15.159:0/340553130 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41409b290 con 0x7fe414166c10 2024-04-04T06:06:07.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.445+0000 7fe419c00640 1 --2- 172.21.15.159:0/340553130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4140d0a00 0x7fe41409d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:06:07.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.445+0000 7f2522a00640 1 -- 172.21.15.159:0/4033952621 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514041070 con 0x7f252c06da30 2024-04-04T06:06:07.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.445+0000 7f2522a00640 1 -- 172.21.15.159:0/4033952621 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2514032b00 con 0x7f252c06da30 2024-04-04T06:06:07.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.445+0000 7f2522a00640 1 -- 172.21.15.159:0/4033952621 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514037070 con 0x7f252c06da30 2024-04-04T06:06:07.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.445+0000 7f2531c00640 1 -- 172.21.15.159:0/4033952621 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f252c0b1bb0 con 0x7f252c06da30 2024-04-04T06:06:07.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.445+0000 7f2531c00640 1 -- 172.21.15.159:0/4033952621 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f252c0b2170 con 0x7f252c06da30 2024-04-04T06:06:07.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.447+0000 7f2531c00640 1 -- 172.21.15.159:0/4033952621 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f252c0685c0 con 0x7f252c06da30 2024-04-04T06:06:07.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.447+0000 7fe41a600640 1 --2- 172.21.15.159:0/340553130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe414166c10 0x7fe41409ad50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe40c007db0 tx=0x7fe40c00d4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.447+0000 7f32e2a00640 1 -- 172.21.15.159:0/854206465 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32d8007600 con 0x7f32e40542f0 2024-04-04T06:06:07.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.448+0000 7f32e2a00640 1 --2- 172.21.15.159:0/854206465 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f32cc06c8f0 0x7f32cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.449+0000 7f32e2a00640 1 -- 172.21.15.159:0/854206465 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f32d808b550 con 0x7f32e40542f0 2024-04-04T06:06:07.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.450+0000 7f32eaa00640 1 --2- 172.21.15.159:0/854206465 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f32cc06c8f0 0x7f32cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.451+0000 7f2522a00640 1 -- 172.21.15.159:0/4033952621 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2514064790 con 0x7f252c06da30 2024-04-04T06:06:07.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.451+0000 7f32eaa00640 1 --2- 172.21.15.159:0/854206465 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f32cc06c8f0 0x7f32cc06edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f32dc036000 tx=0x7f32dc008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.452+0000 7f2522a00640 1 --2- 172.21.15.159:0/4033952621 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f250806c940 0x7f250806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.453+0000 7f2522a00640 1 -- 172.21.15.159:0/4033952621 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f25140ad280 con 0x7f252c06da30 2024-04-04T06:06:07.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.453+0000 7f252a000640 1 --2- 172.21.15.159:0/4033952621 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f250806c940 0x7f250806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.454+0000 7f252a000640 1 --2- 172.21.15.159:0/4033952621 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f250806c940 0x7f250806ee00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f252c0c84f0 tx=0x7f2524009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.462+0000 7fe411600640 1 -- 172.21.15.159:0/340553130 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c01a070 con 0x7fe414166c10 2024-04-04T06:06:07.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.462+0000 7fe411600640 1 -- 172.21.15.159:0/340553130 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe40c003990 con 0x7fe414166c10 2024-04-04T06:06:07.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.462+0000 7fe411600640 1 -- 172.21.15.159:0/340553130 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c003d80 con 0x7fe414166c10 2024-04-04T06:06:07.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.462+0000 7fe41c600640 1 -- 172.21.15.159:0/340553130 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41409d980 con 0x7fe414166c10 2024-04-04T06:06:07.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.462+0000 7fe41c600640 1 -- 172.21.15.159:0/340553130 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe41409df40 con 0x7fe414166c10 2024-04-04T06:06:07.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.463+0000 7f32e2a00640 1 -- 172.21.15.159:0/854206465 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f32d80593c0 con 0x7f32e40542f0 2024-04-04T06:06:07.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.464+0000 7fe41c600640 1 -- 172.21.15.159:0/340553130 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe414071c80 con 0x7fe414166c10 2024-04-04T06:06:07.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.467+0000 7fe411600640 1 -- 172.21.15.159:0/340553130 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe40c01d070 con 0x7fe414166c10 2024-04-04T06:06:07.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.468+0000 7fe411600640 1 --2- 172.21.15.159:0/340553130 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe3f406c940 0x7fe3f406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:07.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.468+0000 7fe411600640 1 -- 172.21.15.159:0/340553130 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe40c08bcc0 con 0x7fe414166c10 2024-04-04T06:06:07.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.469+0000 7fe419c00640 1 --2- 172.21.15.159:0/340553130 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe3f406c940 0x7fe3f406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:07.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.470+0000 7fe419c00640 1 --2- 172.21.15.159:0/340553130 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe3f406c940 0x7fe3f406ee00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe408000f60 tx=0x7fe408009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:07.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.473+0000 7f2522a00640 1 -- 172.21.15.159:0/4033952621 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f251407b0c0 con 0x7f252c06da30 2024-04-04T06:06:07.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.479+0000 7fe411600640 1 -- 172.21.15.159:0/340553130 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe414071c80 con 0x7fe414166c10 2024-04-04T06:06:07.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.750+0000 7f35ee200640 1 -- 172.21.15.159:0/3317413281 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f35e0005380 con 0x7f35e00e3150 2024-04-04T06:06:07.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.751+0000 7f35db400640 1 -- 172.21.15.159:0/3317413281 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f35dc059720 con 0x7f35e00e3150 2024-04-04T06:06:07.746 INFO:teuthology.orchestra.run.smithi159.stdout:64424509448 2024-04-04T06:06:07.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.755+0000 7f35ee200640 1 -- 172.21.15.159:0/3317413281 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35c806c8f0 msgr2=0x7f35c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.755+0000 7f35ee200640 1 --2- 172.21.15.159:0/3317413281 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35c806c8f0 0x7f35c806edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f35e8064b70 tx=0x7f35e806e040 comp rx=0 tx=0).stop 2024-04-04T06:06:07.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.755+0000 7f35ee200640 1 -- 172.21.15.159:0/3317413281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e00e3150 msgr2=0x7f35e004edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.755+0000 7f35ee200640 1 --2- 172.21.15.159:0/3317413281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e00e3150 0x7f35e004edc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00c540 tx=0x7f35dc00ca00 comp rx=0 tx=0).stop 2024-04-04T06:06:07.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.756+0000 7f35ee200640 1 -- 172.21.15.159:0/3317413281 shutdown_connections 2024-04-04T06:06:07.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.756+0000 7f35ee200640 1 --2- 172.21.15.159:0/3317413281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e00e3150 0x7f35e004edc0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.756+0000 7f35ee200640 1 --2- 172.21.15.159:0/3317413281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e0053ca0 0x7f35e004e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.756+0000 7f35ee200640 1 --2- 172.21.15.159:0/3317413281 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35c806c8f0 0x7f35c806edb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.756+0000 7f35ee200640 1 --2- 172.21.15.159:0/3317413281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e003c090 0x7f35e00db1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.757+0000 7f35ee200640 1 -- 172.21.15.159:0/3317413281 >> 172.21.15.159:0/3317413281 conn(0x7f35e0058ed0 msgr2=0x7f35e0039a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:07.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.757+0000 7f35ee200640 1 -- 172.21.15.159:0/3317413281 shutdown_connections 2024-04-04T06:06:07.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.757+0000 7f35ee200640 1 -- 172.21.15.159:0/3317413281 wait complete. 2024-04-04T06:06:07.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.768+0000 7f3d8ba00640 1 -- 172.21.15.159:0/368462380 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3d840b4950 con 0x7f3d8415e860 2024-04-04T06:06:07.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.769+0000 7f3d80c00640 1 -- 172.21.15.159:0/368462380 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3d7c058f10 con 0x7f3d8415e860 2024-04-04T06:06:07.764 INFO:teuthology.orchestra.run.smithi159.stdout:64424509449 2024-04-04T06:06:07.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.775+0000 7f3d8ba00640 1 -- 172.21.15.159:0/368462380 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3d6406c9f0 msgr2=0x7f3d6406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.775+0000 7f3d8ba00640 1 --2- 172.21.15.159:0/368462380 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3d6406c9f0 0x7f3d6406eeb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3d78005880 tx=0x7f3d78009040 comp rx=0 tx=0).stop 2024-04-04T06:06:07.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.775+0000 7f3d8ba00640 1 -- 172.21.15.159:0/368462380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d8415e860 msgr2=0x7f3d8415b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.775+0000 7f3d8ba00640 1 --2- 172.21.15.159:0/368462380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d8415e860 0x7f3d8415b960 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00c780 tx=0x7f3d7c00cc40 comp rx=0 tx=0).stop 2024-04-04T06:06:07.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.776+0000 7f3d8ba00640 1 -- 172.21.15.159:0/368462380 shutdown_connections 2024-04-04T06:06:07.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.776+0000 7f3d8ba00640 1 --2- 172.21.15.159:0/368462380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d8415e860 0x7f3d8415b960 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.776+0000 7f3d8ba00640 1 --2- 172.21.15.159:0/368462380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d840d43d0 0x7f3d841559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.776+0000 7f3d8ba00640 1 --2- 172.21.15.159:0/368462380 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3d6406c9f0 0x7f3d6406eeb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.776+0000 7f3d8ba00640 1 --2- 172.21.15.159:0/368462380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d840cf6b0 0x7f3d84151a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.776+0000 7f3d8ba00640 1 -- 172.21.15.159:0/368462380 >> 172.21.15.159:0/368462380 conn(0x7f3d8409d7d0 msgr2=0x7f3d840b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:07.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.776+0000 7f3d8ba00640 1 -- 172.21.15.159:0/368462380 shutdown_connections 2024-04-04T06:06:07.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.776+0000 7f3d8ba00640 1 -- 172.21.15.159:0/368462380 wait complete. 2024-04-04T06:06:07.779 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.2 2024-04-04T06:06:07.779 DEBUG:teuthology.parallel:result is None 2024-04-04T06:06:07.799 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.3 2024-04-04T06:06:07.799 DEBUG:teuthology.parallel:result is None 2024-04-04T06:06:07.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.808+0000 7fa4dac00640 1 -- 172.21.15.159:0/2988567757 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fa4b0005850 con 0x7fa4cc0e30c0 2024-04-04T06:06:07.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.809+0000 7fa4d1600640 1 -- 172.21.15.159:0/2988567757 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa4cc04efe0 con 0x7fa4cc0e30c0 2024-04-04T06:06:07.804 INFO:teuthology.orchestra.run.smithi159.stdout:64424509446 2024-04-04T06:06:07.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.812+0000 7fa4dac00640 1 -- 172.21.15.159:0/2988567757 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa4b406c9c0 msgr2=0x7fa4b406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.812+0000 7fa4dac00640 1 --2- 172.21.15.159:0/2988567757 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa4b406c9c0 0x7fa4b406ee80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8036000 tx=0x7fa4c803b040 comp rx=0 tx=0).stop 2024-04-04T06:06:07.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.812+0000 7fa4dac00640 1 -- 172.21.15.159:0/2988567757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4cc0e30c0 msgr2=0x7fa4cc0524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.812+0000 7fa4dac00640 1 --2- 172.21.15.159:0/2988567757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4cc0e30c0 0x7fa4cc0524f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa4d406a3e0 tx=0x7fa4d406c950 comp rx=0 tx=0).stop 2024-04-04T06:06:07.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.813+0000 7fa4dac00640 1 -- 172.21.15.159:0/2988567757 shutdown_connections 2024-04-04T06:06:07.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.813+0000 7fa4dac00640 1 --2- 172.21.15.159:0/2988567757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4cc0e30c0 0x7fa4cc0524f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.813+0000 7fa4dac00640 1 --2- 172.21.15.159:0/2988567757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4cc054130 0x7fa4cc051fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.813+0000 7fa4dac00640 1 --2- 172.21.15.159:0/2988567757 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa4b406c9c0 0x7fa4b406ee80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.813+0000 7fa4dac00640 1 --2- 172.21.15.159:0/2988567757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4cc03e840 0x7fa4cc04eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.813+0000 7fa4dac00640 1 -- 172.21.15.159:0/2988567757 >> 172.21.15.159:0/2988567757 conn(0x7fa4cc0559e0 msgr2=0x7fa4cc006720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:07.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.813+0000 7fa4dac00640 1 -- 172.21.15.159:0/2988567757 shutdown_connections 2024-04-04T06:06:07.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.813+0000 7fa4dac00640 1 -- 172.21.15.159:0/2988567757 wait complete. 2024-04-04T06:06:07.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.818+0000 7f5ade000640 1 -- 172.21.15.159:0/603593680 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f5ab4005850 con 0x7f5ad80d37b0 2024-04-04T06:06:07.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.818+0000 7f5ad4c00640 1 -- 172.21.15.159:0/603593680 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5ac407b980 con 0x7f5ad80d37b0 2024-04-04T06:06:07.813 INFO:teuthology.orchestra.run.smithi159.stdout:64424509449 2024-04-04T06:06:07.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.820+0000 7fb6f8600640 1 -- 172.21.15.159:0/4139637500 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fb6cc005580 con 0x7fb6f00d4ae0 2024-04-04T06:06:07.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.820+0000 7fb6ed600640 1 -- 172.21.15.159:0/4139637500 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb6dc059bf0 con 0x7fb6f00d4ae0 2024-04-04T06:06:07.815 INFO:teuthology.orchestra.run.smithi159.stdout:60129542153 2024-04-04T06:06:07.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.821+0000 7f5ade000640 1 -- 172.21.15.159:0/603593680 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5ab806c8f0 msgr2=0x7f5ab806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.821+0000 7f5ade000640 1 --2- 172.21.15.159:0/603593680 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5ab806c8f0 0x7f5ab806edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5acc000c00 tx=0x7f5acc007b50 comp rx=0 tx=0).stop 2024-04-04T06:06:07.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.821+0000 7f5ade000640 1 -- 172.21.15.159:0/603593680 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ad80d37b0 msgr2=0x7f5ad80cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.821+0000 7f5ade000640 1 --2- 172.21.15.159:0/603593680 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ad80d37b0 0x7f5ad80cd570 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009890 tx=0x7f5ac40046c0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.821+0000 7f5ade000640 1 -- 172.21.15.159:0/603593680 shutdown_connections 2024-04-04T06:06:07.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.821+0000 7f5ade000640 1 --2- 172.21.15.159:0/603593680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ad815e4c0 0x7f5ad80cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.821+0000 7f5ade000640 1 --2- 172.21.15.159:0/603593680 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ad80d37b0 0x7f5ad80cd570 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.821+0000 7f5ade000640 1 --2- 172.21.15.159:0/603593680 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5ab806c8f0 0x7f5ab806edb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.821+0000 7f5ade000640 1 --2- 172.21.15.159:0/603593680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad80b67f0 0x7f5ad815a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.821+0000 7f5ade000640 1 -- 172.21.15.159:0/603593680 >> 172.21.15.159:0/603593680 conn(0x7f5ad80676a0 msgr2=0x7f5ad8067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:07.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.822+0000 7f5ade000640 1 -- 172.21.15.159:0/603593680 shutdown_connections 2024-04-04T06:06:07.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.822+0000 7f5ade000640 1 -- 172.21.15.159:0/603593680 wait complete. 2024-04-04T06:06:07.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.823+0000 7fb6f8600640 1 -- 172.21.15.159:0/4139637500 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb6d006c9c0 msgr2=0x7fb6d006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.824+0000 7fb6f8600640 1 --2- 172.21.15.159:0/4139637500 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb6d006c9c0 0x7fb6d006ee80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4036000 tx=0x7fb6e4008030 comp rx=0 tx=0).stop 2024-04-04T06:06:07.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.824+0000 7fb6f8600640 1 -- 172.21.15.159:0/4139637500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6f00d4ae0 msgr2=0x7fb6f00ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.824+0000 7fb6f8600640 1 --2- 172.21.15.159:0/4139637500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6f00d4ae0 0x7fb6f00ccc80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00c480 tx=0x7fb6dc00c940 comp rx=0 tx=0).stop 2024-04-04T06:06:07.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.824+0000 7fb6f8600640 1 -- 172.21.15.159:0/4139637500 shutdown_connections 2024-04-04T06:06:07.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.824+0000 7fb6f8600640 1 --2- 172.21.15.159:0/4139637500 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6f015e860 0x7fb6f00cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.824+0000 7fb6f8600640 1 --2- 172.21.15.159:0/4139637500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6f00d4ae0 0x7fb6f00ccc80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.824+0000 7fb6f8600640 1 --2- 172.21.15.159:0/4139637500 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb6d006c9c0 0x7fb6d006ee80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.824+0000 7fb6f8600640 1 --2- 172.21.15.159:0/4139637500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6f006da30 0x7fb6f0151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.824+0000 7fb6f8600640 1 -- 172.21.15.159:0/4139637500 >> 172.21.15.159:0/4139637500 conn(0x7fb6f009d7d0 msgr2=0x7fb6f00b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:07.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.824+0000 7fb6f8600640 1 -- 172.21.15.159:0/4139637500 shutdown_connections 2024-04-04T06:06:07.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.824+0000 7fb6f8600640 1 -- 172.21.15.159:0/4139637500 wait complete. 2024-04-04T06:06:07.834 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.7 2024-04-04T06:06:07.834 DEBUG:teuthology.parallel:result is None 2024-04-04T06:06:07.835 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.0 2024-04-04T06:06:07.835 DEBUG:teuthology.parallel:result is None 2024-04-04T06:06:07.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.842+0000 7fe41c600640 1 -- 172.21.15.159:0/340553130 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe414075940 con 0x7fe414166c10 2024-04-04T06:06:07.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.843+0000 7fe411600640 1 -- 172.21.15.159:0/340553130 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe40c010070 con 0x7fe414166c10 2024-04-04T06:06:07.837 INFO:teuthology.orchestra.run.smithi159.stdout:64424509447 2024-04-04T06:06:07.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.845+0000 7fe41c600640 1 -- 172.21.15.159:0/340553130 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe3f406c940 msgr2=0x7fe3f406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.845+0000 7fe41c600640 1 --2- 172.21.15.159:0/340553130 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe3f406c940 0x7fe3f406ee00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe408000f60 tx=0x7fe408009040 comp rx=0 tx=0).stop 2024-04-04T06:06:07.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.845+0000 7fe41c600640 1 -- 172.21.15.159:0/340553130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe414166c10 msgr2=0x7fe41409ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.845+0000 7fe41c600640 1 --2- 172.21.15.159:0/340553130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe414166c10 0x7fe41409ad50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe40c007db0 tx=0x7fe40c00d4a0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.845+0000 7fe41c600640 1 -- 172.21.15.159:0/340553130 shutdown_connections 2024-04-04T06:06:07.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.845+0000 7fe41c600640 1 --2- 172.21.15.159:0/340553130 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe3f406c940 0x7fe3f406ee00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.845+0000 7fe41c600640 1 --2- 172.21.15.159:0/340553130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe414166c10 0x7fe41409ad50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.845+0000 7fe41c600640 1 --2- 172.21.15.159:0/340553130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe41415d060 0x7fe414162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.845+0000 7fe41c600640 1 --2- 172.21.15.159:0/340553130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4140d0a00 0x7fe41409d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.845+0000 7fe41c600640 1 -- 172.21.15.159:0/340553130 >> 172.21.15.159:0/340553130 conn(0x7fe41406b750 msgr2=0x7fe414072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:07.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.845+0000 7fe41c600640 1 -- 172.21.15.159:0/340553130 shutdown_connections 2024-04-04T06:06:07.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.845+0000 7fe41c600640 1 -- 172.21.15.159:0/340553130 wait complete. 2024-04-04T06:06:07.844 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542153 for osd.5 2024-04-04T06:06:07.844 DEBUG:teuthology.parallel:result is None 2024-04-04T06:06:07.857 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.1 2024-04-04T06:06:07.858 DEBUG:teuthology.parallel:result is None 2024-04-04T06:06:07.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.865+0000 7f32f1a00640 1 -- 172.21.15.159:0/854206465 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f32c4005850 con 0x7f32e40542f0 2024-04-04T06:06:07.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.866+0000 7f32e2a00640 1 -- 172.21.15.159:0/854206465 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f32d805d6c0 con 0x7f32e40542f0 2024-04-04T06:06:07.861 INFO:teuthology.orchestra.run.smithi159.stdout:64424509448 2024-04-04T06:06:07.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.869+0000 7f32f1a00640 1 -- 172.21.15.159:0/854206465 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f32cc06c8f0 msgr2=0x7f32cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.869+0000 7f32f1a00640 1 --2- 172.21.15.159:0/854206465 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f32cc06c8f0 0x7f32cc06edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f32dc036000 tx=0x7f32dc008030 comp rx=0 tx=0).stop 2024-04-04T06:06:07.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.869+0000 7f32f1a00640 1 -- 172.21.15.159:0/854206465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32e40542f0 msgr2=0x7f32e40dcfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.869+0000 7f32f1a00640 1 --2- 172.21.15.159:0/854206465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32e40542f0 0x7f32e40dcfe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f32d800c460 tx=0x7f32d800c920 comp rx=0 tx=0).stop 2024-04-04T06:06:07.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.869+0000 7f32f1a00640 1 -- 172.21.15.159:0/854206465 shutdown_connections 2024-04-04T06:06:07.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.869+0000 7f32f1a00640 1 --2- 172.21.15.159:0/854206465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32e40dd520 0x7f32e40e0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.869+0000 7f32f1a00640 1 --2- 172.21.15.159:0/854206465 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f32cc06c8f0 0x7f32cc06edb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.869+0000 7f32f1a00640 1 --2- 172.21.15.159:0/854206465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32e40542f0 0x7f32e40dcfe0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.869+0000 7f32f1a00640 1 --2- 172.21.15.159:0/854206465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32e403e830 0x7f32e40cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.869+0000 7f32f1a00640 1 -- 172.21.15.159:0/854206465 >> 172.21.15.159:0/854206465 conn(0x7f32e40559d0 msgr2=0x7f32e40579e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:07.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.869+0000 7f32f1a00640 1 -- 172.21.15.159:0/854206465 shutdown_connections 2024-04-04T06:06:07.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.870+0000 7f32f1a00640 1 -- 172.21.15.159:0/854206465 wait complete. 2024-04-04T06:06:07.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.875+0000 7f2531c00640 1 -- 172.21.15.159:0/4033952621 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f252c0cb2d0 con 0x7f252c06da30 2024-04-04T06:06:07.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.875+0000 7f2522a00640 1 -- 172.21.15.159:0/4033952621 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f251407f3c0 con 0x7f252c06da30 2024-04-04T06:06:07.870 INFO:teuthology.orchestra.run.smithi159.stdout:60129542151 2024-04-04T06:06:07.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.879+0000 7f2531c00640 1 -- 172.21.15.159:0/4033952621 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f250806c940 msgr2=0x7f250806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.879+0000 7f2531c00640 1 --2- 172.21.15.159:0/4033952621 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f250806c940 0x7f250806ee00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f252c0c84f0 tx=0x7f2524009040 comp rx=0 tx=0).stop 2024-04-04T06:06:07.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.879+0000 7f2531c00640 1 -- 172.21.15.159:0/4033952621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f252c06da30 msgr2=0x7f252c0caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:07.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.879+0000 7f2531c00640 1 --2- 172.21.15.159:0/4033952621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f252c06da30 0x7f252c0caa20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2514032000 tx=0x7f25140046c0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.879+0000 7f2531c00640 1 -- 172.21.15.159:0/4033952621 shutdown_connections 2024-04-04T06:06:07.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.879+0000 7f2531c00640 1 --2- 172.21.15.159:0/4033952621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f252c15e860 0x7f252c0cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.879+0000 7f2531c00640 1 --2- 172.21.15.159:0/4033952621 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f250806c940 0x7f250806ee00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.879+0000 7f2531c00640 1 --2- 172.21.15.159:0/4033952621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f252c0d4ae0 0x7f252c0cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.879+0000 7f2531c00640 1 --2- 172.21.15.159:0/4033952621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f252c06da30 0x7f252c0caa20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:07.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.879+0000 7f2531c00640 1 -- 172.21.15.159:0/4033952621 >> 172.21.15.159:0/4033952621 conn(0x7f252c09d7d0 msgr2=0x7f252c0b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:07.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.879+0000 7f2531c00640 1 -- 172.21.15.159:0/4033952621 shutdown_connections 2024-04-04T06:06:07.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:07.879+0000 7f2531c00640 1 -- 172.21.15.159:0/4033952621 wait complete. 2024-04-04T06:06:07.887 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.6 2024-04-04T06:06:07.887 DEBUG:teuthology.parallel:result is None 2024-04-04T06:06:07.891 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.4 2024-04-04T06:06:07.892 DEBUG:teuthology.parallel:result is None 2024-04-04T06:06:07.892 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-04T06:06:07.892 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:06:08.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.140+0000 7f5dc8400640 1 Processor -- start 2024-04-04T06:06:08.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.140+0000 7f5dc8400640 1 -- start start 2024-04-04T06:06:08.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.142+0000 7f5dc8400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5dc015c390 0x7f5dc0073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:08.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.142+0000 7f5dc8400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5dc00d0650 0x7f5dc00d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:08.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.142+0000 7f5dc8400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dc015fe80 con 0x7f5dc00af830 2024-04-04T06:06:08.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.142+0000 7f5dc8400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f5dc015c770 con 0x7f5dc015c390 2024-04-04T06:06:08.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.142+0000 7f5dc8400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc00ad120 con 0x7f5dc00d0650 2024-04-04T06:06:08.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.142+0000 7f5dc5a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5dc015c390 0x7f5dc0073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:08.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.142+0000 7f5dc5a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5dc015c390 0x7f5dc0073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37304/0 (socket says 172.21.15.159:37304) 2024-04-04T06:06:08.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.142+0000 7f5dc5a00640 1 -- 172.21.15.159:0/4191517489 learned_addr learned my addr 172.21.15.159:0/4191517489 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:08.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.142+0000 7f5dc5a00640 1 -- 172.21.15.159:0/4191517489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5dc00d0650 msgr2=0x7f5dc00d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:08.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.142+0000 7f5dc5a00640 1 --2- 172.21.15.159:0/4191517489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5dc00d0650 0x7f5dc00d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:08.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.142+0000 7f5dc5a00640 1 -- 172.21.15.159:0/4191517489 >> v1:172.21.15.178:6789/0 conn(0x7f5dc00af830 legacy=0x7f5dc0160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:08.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.143+0000 7f5dc5a00640 1 -- 172.21.15.159:0/4191517489 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc00ad400 con 0x7f5dc015c390 2024-04-04T06:06:08.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.144+0000 7f5dc5a00640 1 --2- 172.21.15.159:0/4191517489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dc015c390 0x7f5dc0073790 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009870 tx=0x7f5db402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc0d949c8f00d90 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:08.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.144+0000 7f5dbfe00640 1 -- 172.21.15.159:0/4191517489 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db4035020 con 0x7f5dc015c390 2024-04-04T06:06:08.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.144+0000 7f5dbfe00640 1 -- 172.21.15.159:0/4191517489 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db4031640 con 0x7f5dc015c390 2024-04-04T06:06:08.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.144+0000 7f5dbfe00640 1 -- 172.21.15.159:0/4191517489 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db402d040 con 0x7f5dc015c390 2024-04-04T06:06:08.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.144+0000 7f5dc8400640 1 -- 172.21.15.159:0/4191517489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dc015c390 msgr2=0x7f5dc0073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:08.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.144+0000 7f5dc8400640 1 --2- 172.21.15.159:0/4191517489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dc015c390 0x7f5dc0073790 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009870 tx=0x7f5db402b8f0 comp rx=0 tx=0).stop 2024-04-04T06:06:08.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.145+0000 7f5dc8400640 1 -- 172.21.15.159:0/4191517489 shutdown_connections 2024-04-04T06:06:08.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.145+0000 7f5dc8400640 1 --2- 172.21.15.159:0/4191517489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5dc00d0650 0x7f5dc00d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:08.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.145+0000 7f5dc8400640 1 --2- 172.21.15.159:0/4191517489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dc015c390 0x7f5dc0073790 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:08.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.145+0000 7f5dc8400640 1 -- 172.21.15.159:0/4191517489 >> 172.21.15.159:0/4191517489 conn(0x7f5dc006b990 msgr2=0x7f5dc006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:08.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.145+0000 7f5dc8400640 1 -- 172.21.15.159:0/4191517489 shutdown_connections 2024-04-04T06:06:08.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.145+0000 7f5dc8400640 1 -- 172.21.15.159:0/4191517489 wait complete. 2024-04-04T06:06:08.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.146+0000 7f5dc8400640 1 Processor -- start 2024-04-04T06:06:08.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.146+0000 7f5dc8400640 1 -- start start 2024-04-04T06:06:08.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.147+0000 7f5dc8400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dc00af830 0x7f5dc0157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:08.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.147+0000 7f5dc8400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5dc00d0650 0x7f5dc01e7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:08.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.147+0000 7f5dc8400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5dc015c390 0x7f5dc014c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:08.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.147+0000 7f5dc8400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc0160d50 con 0x7f5dc00af830 2024-04-04T06:06:08.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.147+0000 7f5dc8400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc0160120 con 0x7f5dc015c390 2024-04-04T06:06:08.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.147+0000 7f5dc8400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc00cf980 con 0x7f5dc00d0650 2024-04-04T06:06:08.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.148+0000 7f5dc5a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dc00af830 0x7f5dc0157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:08.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.148+0000 7f5dc5a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dc00af830 0x7f5dc0157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37316/0 (socket says 172.21.15.159:37316) 2024-04-04T06:06:08.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.148+0000 7f5dc5a00640 1 -- 172.21.15.159:0/3639335760 learned_addr learned my addr 172.21.15.159:0/3639335760 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:08.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.148+0000 7f5dc6400640 1 --2- 172.21.15.159:0/3639335760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5dc015c390 0x7f5dc014c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:08.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.148+0000 7f5dc5a00640 1 -- 172.21.15.159:0/3639335760 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5dc00d0650 msgr2=0x7f5dc01e7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:08.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.148+0000 7f5dc5000640 1 --2- 172.21.15.159:0/3639335760 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5dc00d0650 0x7f5dc01e7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:08.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.148+0000 7f5dc5a00640 1 --2- 172.21.15.159:0/3639335760 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5dc00d0650 0x7f5dc01e7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:08.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.148+0000 7f5dc5a00640 1 -- 172.21.15.159:0/3639335760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5dc015c390 msgr2=0x7f5dc014c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:08.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.148+0000 7f5dc5a00640 1 --2- 172.21.15.159:0/3639335760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5dc015c390 0x7f5dc014c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:08.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.148+0000 7f5dc5a00640 1 -- 172.21.15.159:0/3639335760 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc01e8740 con 0x7f5dc00af830 2024-04-04T06:06:08.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.148+0000 7f5dc6400640 1 --2- 172.21.15.159:0/3639335760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5dc015c390 0x7f5dc014c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:08.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.148+0000 7f5dc5a00640 1 --2- 172.21.15.159:0/3639335760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dc00af830 0x7f5dc0157770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5db4000c00 tx=0x7f5db40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:08.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.149+0000 7f5dbd600640 1 -- 172.21.15.159:0/3639335760 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db403ac60 con 0x7f5dc00af830 2024-04-04T06:06:08.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.149+0000 7f5dbd600640 1 -- 172.21.15.159:0/3639335760 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db403ae20 con 0x7f5dc00af830 2024-04-04T06:06:08.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.149+0000 7f5dbd600640 1 -- 172.21.15.159:0/3639335760 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db4044630 con 0x7f5dc00af830 2024-04-04T06:06:08.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.149+0000 7f5dc8400640 1 -- 172.21.15.159:0/3639335760 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc01e8a20 con 0x7f5dc00af830 2024-04-04T06:06:08.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.149+0000 7f5dc8400640 1 -- 172.21.15.159:0/3639335760 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc01e8450 con 0x7f5dc00af830 2024-04-04T06:06:08.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.150+0000 7f5dc8400640 1 -- 172.21.15.159:0/3639335760 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc00add30 con 0x7f5dc00af830 2024-04-04T06:06:08.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.151+0000 7f5dbd600640 1 -- 172.21.15.159:0/3639335760 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5db4038020 con 0x7f5dc00af830 2024-04-04T06:06:08.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.151+0000 7f5dbd600640 1 --2- 172.21.15.159:0/3639335760 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5da006c8f0 0x7f5da006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:08.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.151+0000 7f5dbd600640 1 -- 172.21.15.159:0/3639335760 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5db40ad830 con 0x7f5dc00af830 2024-04-04T06:06:08.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.152+0000 7f5dc5000640 1 --2- 172.21.15.159:0/3639335760 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5da006c8f0 0x7f5da006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:08.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.153+0000 7f5dc5000640 1 --2- 172.21.15.159:0/3639335760 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5da006c8f0 0x7f5da006edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0157490 tx=0x7f5dac009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:08.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.156+0000 7f5dbd600640 1 -- 172.21.15.159:0/3639335760 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5dc00add30 con 0x7f5dc00af830 2024-04-04T06:06:08.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.336+0000 7f5dc8400640 1 -- 172.21.15.159:0/3639335760 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5dc01e8f40 con 0x7f5da006c8f0 2024-04-04T06:06:08.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.356+0000 7f5dbd600640 1 -- 172.21.15.159:0/3639335760 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7f5dc01e8f40 con 0x7f5da006c8f0 2024-04-04T06:06:08.354 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:08.356 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:06:08.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.362+0000 7f5dc8400640 1 -- 172.21.15.159:0/3639335760 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5da006c8f0 msgr2=0x7f5da006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:08.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.362+0000 7f5dc8400640 1 --2- 172.21.15.159:0/3639335760 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5da006c8f0 0x7f5da006edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0157490 tx=0x7f5dac009040 comp rx=0 tx=0).stop 2024-04-04T06:06:08.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.362+0000 7f5dc8400640 1 -- 172.21.15.159:0/3639335760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dc00af830 msgr2=0x7f5dc0157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:08.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.362+0000 7f5dc8400640 1 --2- 172.21.15.159:0/3639335760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dc00af830 0x7f5dc0157770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5db4000c00 tx=0x7f5db40046c0 comp rx=0 tx=0).stop 2024-04-04T06:06:08.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.362+0000 7f5dc8400640 1 -- 172.21.15.159:0/3639335760 shutdown_connections 2024-04-04T06:06:08.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.362+0000 7f5dc8400640 1 --2- 172.21.15.159:0/3639335760 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5da006c8f0 0x7f5da006edb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:08.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.362+0000 7f5dc8400640 1 --2- 172.21.15.159:0/3639335760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5dc015c390 0x7f5dc014c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:08.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.362+0000 7f5dc8400640 1 --2- 172.21.15.159:0/3639335760 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5dc00d0650 0x7f5dc01e7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:08.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.362+0000 7f5dc8400640 1 --2- 172.21.15.159:0/3639335760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dc00af830 0x7f5dc0157770 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:08.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.362+0000 7f5dc8400640 1 -- 172.21.15.159:0/3639335760 >> 172.21.15.159:0/3639335760 conn(0x7f5dc006b990 msgr2=0x7f5dc00afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:08.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.362+0000 7f5dc8400640 1 -- 172.21.15.159:0/3639335760 shutdown_connections 2024-04-04T06:06:08.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.363+0000 7f5dc8400640 1 -- 172.21.15.159:0/3639335760 wait complete. 2024-04-04T06:06:08.374 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-04-04T06:06:06.715377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235672,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739048,"statfs":{"total":773094113280,"available":772852785152,"internally_reserved":0,"allocated":2924544,"data_stored":1765420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004274"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594198+0000","last_change":"2024-04-04T06:05:53.366079+0000","last_active":"2024-04-04T06:05:54.594198+0000","last_peered":"2024-04-04T06:05:54.594198+0000","last_clean":"2024-04-04T06:05:54.594198+0000","last_became_active":"2024-04-04T06:05:53.365810+0000","last_became_peered":"2024-04-04T06:05:53.365810+0000","last_unstale":"2024-04-04T06:05:54.594198+0000","last_undegraded":"2024-04-04T06:05:54.594198+0000","last_fullsized":"2024-04-04T06:05:54.594198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:46:01.152077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362281+0000","last_change":"2024-04-04T06:05:53.376777+0000","last_active":"2024-04-04T06:05:54.362281+0000","last_peered":"2024-04-04T06:05:54.362281+0000","last_clean":"2024-04-04T06:05:54.362281+0000","last_became_active":"2024-04-04T06:05:53.374947+0000","last_became_peered":"2024-04-04T06:05:53.374947+0000","last_unstale":"2024-04-04T06:05:54.362281+0000","last_undegraded":"2024-04-04T06:05:54.362281+0000","last_fullsized":"2024-04-04T06:05:54.362281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:34.766930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591301+0000","last_change":"2024-04-04T06:05:53.362328+0000","last_active":"2024-04-04T06:05:54.591301+0000","last_peered":"2024-04-04T06:05:54.591301+0000","last_clean":"2024-04-04T06:05:54.591301+0000","last_became_active":"2024-04-04T06:05:53.359587+0000","last_became_peered":"2024-04-04T06:05:53.359587+0000","last_unstale":"2024-04-04T06:05:54.591301+0000","last_undegraded":"2024-04-04T06:05:54.591301+0000","last_fullsized":"2024-04-04T06:05:54.591301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:10.411661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365717+0000","last_change":"2024-04-04T06:05:53.373115+0000","last_active":"2024-04-04T06:05:54.365717+0000","last_peered":"2024-04-04T06:05:54.365717+0000","last_clean":"2024-04-04T06:05:54.365717+0000","last_became_active":"2024-04-04T06:05:53.372848+0000","last_became_peered":"2024-04-04T06:05:53.372848+0000","last_unstale":"2024-04-04T06:05:54.365717+0000","last_undegraded":"2024-04-04T06:05:54.365717+0000","last_fullsized":"2024-04-04T06:05:54.365717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:45:21.548886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593377+0000","last_change":"2024-04-04T06:05:53.370721+0000","last_active":"2024-04-04T06:05:54.593377+0000","last_peered":"2024-04-04T06:05:54.593377+0000","last_clean":"2024-04-04T06:05:54.593377+0000","last_became_active":"2024-04-04T06:05:53.370183+0000","last_became_peered":"2024-04-04T06:05:53.370183+0000","last_unstale":"2024-04-04T06:05:54.593377+0000","last_undegraded":"2024-04-04T06:05:54.593377+0000","last_fullsized":"2024-04-04T06:05:54.593377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:30:56.836366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367929+0000","last_change":"2024-04-04T06:05:53.387263+0000","last_active":"2024-04-04T06:05:54.367929+0000","last_peered":"2024-04-04T06:05:54.367929+0000","last_clean":"2024-04-04T06:05:54.367929+0000","last_became_active":"2024-04-04T06:05:53.387012+0000","last_became_peered":"2024-04-04T06:05:53.387012+0000","last_unstale":"2024-04-04T06:05:54.367929+0000","last_undegraded":"2024-04-04T06:05:54.367929+0000","last_fullsized":"2024-04-04T06:05:54.367929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:13.472940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592309+0000","last_change":"2024-04-04T06:05:53.370203+0000","last_active":"2024-04-04T06:05:54.592309+0000","last_peered":"2024-04-04T06:05:54.592309+0000","last_clean":"2024-04-04T06:05:54.592309+0000","last_became_active":"2024-04-04T06:05:53.369800+0000","last_became_peered":"2024-04-04T06:05:53.369800+0000","last_unstale":"2024-04-04T06:05:54.592309+0000","last_undegraded":"2024-04-04T06:05:54.592309+0000","last_fullsized":"2024-04-04T06:05:54.592309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:39:30.391173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592465+0000","last_change":"2024-04-04T06:05:53.369028+0000","last_active":"2024-04-04T06:05:54.592465+0000","last_peered":"2024-04-04T06:05:54.592465+0000","last_clean":"2024-04-04T06:05:54.592465+0000","last_became_active":"2024-04-04T06:05:53.365998+0000","last_became_peered":"2024-04-04T06:05:53.365998+0000","last_unstale":"2024-04-04T06:05:54.592465+0000","last_undegraded":"2024-04-04T06:05:54.592465+0000","last_fullsized":"2024-04-04T06:05:54.592465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:34.766930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594227+0000","last_change":"2024-04-04T06:05:53.369886+0000","last_active":"2024-04-04T06:05:54.594227+0000","last_peered":"2024-04-04T06:05:54.594227+0000","last_clean":"2024-04-04T06:05:54.594227+0000","last_became_active":"2024-04-04T06:05:53.368815+0000","last_became_peered":"2024-04-04T06:05:53.368815+0000","last_unstale":"2024-04-04T06:05:54.594227+0000","last_undegraded":"2024-04-04T06:05:54.594227+0000","last_fullsized":"2024-04-04T06:05:54.594227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:37:59.044530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592641+0000","last_change":"2024-04-04T06:05:53.366948+0000","last_active":"2024-04-04T06:05:54.592641+0000","last_peered":"2024-04-04T06:05:54.592641+0000","last_clean":"2024-04-04T06:05:54.592641+0000","last_became_active":"2024-04-04T06:05:53.366550+0000","last_became_peered":"2024-04-04T06:05:53.366550+0000","last_unstale":"2024-04-04T06:05:54.592641+0000","last_undegraded":"2024-04-04T06:05:54.592641+0000","last_fullsized":"2024-04-04T06:05:54.592641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:47:43.615957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592078+0000","last_change":"2024-04-04T06:05:53.376979+0000","last_active":"2024-04-04T06:05:54.592078+0000","last_peered":"2024-04-04T06:05:54.592078+0000","last_clean":"2024-04-04T06:05:54.592078+0000","last_became_active":"2024-04-04T06:05:53.376622+0000","last_became_peered":"2024-04-04T06:05:53.376622+0000","last_unstale":"2024-04-04T06:05:54.592078+0000","last_undegraded":"2024-04-04T06:05:54.592078+0000","last_fullsized":"2024-04-04T06:05:54.592078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:42:00.014108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367238+0000","last_change":"2024-04-04T06:05:53.364063+0000","last_active":"2024-04-04T06:05:54.367238+0000","last_peered":"2024-04-04T06:05:54.367238+0000","last_clean":"2024-04-04T06:05:54.367238+0000","last_became_active":"2024-04-04T06:05:53.363715+0000","last_became_peered":"2024-04-04T06:05:53.363715+0000","last_unstale":"2024-04-04T06:05:54.367238+0000","last_undegraded":"2024-04-04T06:05:54.367238+0000","last_fullsized":"2024-04-04T06:05:54.367238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:21.889411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593467+0000","last_change":"2024-04-04T06:05:53.362035+0000","last_active":"2024-04-04T06:05:54.593467+0000","last_peered":"2024-04-04T06:05:54.593467+0000","last_clean":"2024-04-04T06:05:54.593467+0000","last_became_active":"2024-04-04T06:05:53.361614+0000","last_became_peered":"2024-04-04T06:05:53.361614+0000","last_unstale":"2024-04-04T06:05:54.593467+0000","last_undegraded":"2024-04-04T06:05:54.593467+0000","last_fullsized":"2024-04-04T06:05:54.593467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:17:05.576402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590275+0000","last_change":"2024-04-04T06:05:53.355832+0000","last_active":"2024-04-04T06:05:54.590275+0000","last_peered":"2024-04-04T06:05:54.590275+0000","last_clean":"2024-04-04T06:05:54.590275+0000","last_became_active":"2024-04-04T06:05:53.355508+0000","last_became_peered":"2024-04-04T06:05:53.355508+0000","last_unstale":"2024-04-04T06:05:54.590275+0000","last_undegraded":"2024-04-04T06:05:54.590275+0000","last_fullsized":"2024-04-04T06:05:54.590275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:11:00.324524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362799+0000","last_change":"2024-04-04T06:05:53.353630+0000","last_active":"2024-04-04T06:05:54.362799+0000","last_peered":"2024-04-04T06:05:54.362799+0000","last_clean":"2024-04-04T06:05:54.362799+0000","last_became_active":"2024-04-04T06:05:53.353315+0000","last_became_peered":"2024-04-04T06:05:53.353315+0000","last_unstale":"2024-04-04T06:05:54.362799+0000","last_undegraded":"2024-04-04T06:05:54.362799+0000","last_fullsized":"2024-04-04T06:05:54.362799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.364261+0000","last_change":"2024-04-04T06:05:53.387499+0000","last_active":"2024-04-04T06:05:54.364261+0000","last_peered":"2024-04-04T06:05:54.364261+0000","last_clean":"2024-04-04T06:05:54.364261+0000","last_became_active":"2024-04-04T06:05:53.387187+0000","last_became_peered":"2024-04-04T06:05:53.387187+0000","last_unstale":"2024-04-04T06:05:54.364261+0000","last_undegraded":"2024-04-04T06:05:54.364261+0000","last_fullsized":"2024-04-04T06:05:54.364261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.366558+0000","last_change":"2024-04-04T06:05:53.378705+0000","last_active":"2024-04-04T06:05:54.366558+0000","last_peered":"2024-04-04T06:05:54.366558+0000","last_clean":"2024-04-04T06:05:54.366558+0000","last_became_active":"2024-04-04T06:05:53.378428+0000","last_became_peered":"2024-04-04T06:05:53.378428+0000","last_unstale":"2024-04-04T06:05:54.366558+0000","last_undegraded":"2024-04-04T06:05:54.366558+0000","last_fullsized":"2024-04-04T06:05:54.366558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:17:42.995360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591477+0000","last_change":"2024-04-04T06:05:53.372718+0000","last_active":"2024-04-04T06:05:54.591477+0000","last_peered":"2024-04-04T06:05:54.591477+0000","last_clean":"2024-04-04T06:05:54.591477+0000","last_became_active":"2024-04-04T06:05:53.372458+0000","last_became_peered":"2024-04-04T06:05:53.372458+0000","last_unstale":"2024-04-04T06:05:54.591477+0000","last_undegraded":"2024-04-04T06:05:54.591477+0000","last_fullsized":"2024-04-04T06:05:54.591477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:31.723861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594485+0000","last_change":"2024-04-04T06:05:53.360929+0000","last_active":"2024-04-04T06:05:54.594485+0000","last_peered":"2024-04-04T06:05:54.594485+0000","last_clean":"2024-04-04T06:05:54.594485+0000","last_became_active":"2024-04-04T06:05:53.360661+0000","last_became_peered":"2024-04-04T06:05:53.360661+0000","last_unstale":"2024-04-04T06:05:54.594485+0000","last_undegraded":"2024-04-04T06:05:54.594485+0000","last_fullsized":"2024-04-04T06:05:54.594485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:06:23.755362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591699+0000","last_change":"2024-04-04T06:05:53.374155+0000","last_active":"2024-04-04T06:05:54.591699+0000","last_peered":"2024-04-04T06:05:54.591699+0000","last_clean":"2024-04-04T06:05:54.591699+0000","last_became_active":"2024-04-04T06:05:53.373872+0000","last_became_peered":"2024-04-04T06:05:53.373872+0000","last_unstale":"2024-04-04T06:05:54.591699+0000","last_undegraded":"2024-04-04T06:05:54.591699+0000","last_fullsized":"2024-04-04T06:05:54.591699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:19:20.840644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590730+0000","last_change":"2024-04-04T06:05:53.375944+0000","last_active":"2024-04-04T06:05:54.590730+0000","last_peered":"2024-04-04T06:05:54.590730+0000","last_clean":"2024-04-04T06:05:54.590730+0000","last_became_active":"2024-04-04T06:05:53.375551+0000","last_became_peered":"2024-04-04T06:05:53.375551+0000","last_unstale":"2024-04-04T06:05:54.590730+0000","last_undegraded":"2024-04-04T06:05:54.590730+0000","last_fullsized":"2024-04-04T06:05:54.590730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:33:17.187978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367881+0000","last_change":"2024-04-04T06:05:53.377200+0000","last_active":"2024-04-04T06:05:54.367881+0000","last_peered":"2024-04-04T06:05:54.367881+0000","last_clean":"2024-04-04T06:05:54.367881+0000","last_became_active":"2024-04-04T06:05:53.373063+0000","last_became_peered":"2024-04-04T06:05:53.373063+0000","last_unstale":"2024-04-04T06:05:54.367881+0000","last_undegraded":"2024-04-04T06:05:54.367881+0000","last_fullsized":"2024-04-04T06:05:54.367881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:28:48.569552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592389+0000","last_change":"2024-04-04T06:05:53.358659+0000","last_active":"2024-04-04T06:05:54.592389+0000","last_peered":"2024-04-04T06:05:54.592389+0000","last_clean":"2024-04-04T06:05:54.592389+0000","last_became_active":"2024-04-04T06:05:53.358372+0000","last_became_peered":"2024-04-04T06:05:53.358372+0000","last_unstale":"2024-04-04T06:05:54.592389+0000","last_undegraded":"2024-04-04T06:05:54.592389+0000","last_fullsized":"2024-04-04T06:05:54.592389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:14:41.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365536+0000","last_change":"2024-04-04T06:05:53.381591+0000","last_active":"2024-04-04T06:05:54.365536+0000","last_peered":"2024-04-04T06:05:54.365536+0000","last_clean":"2024-04-04T06:05:54.365536+0000","last_became_active":"2024-04-04T06:05:53.377261+0000","last_became_peered":"2024-04-04T06:05:53.377261+0000","last_unstale":"2024-04-04T06:05:54.365536+0000","last_undegraded":"2024-04-04T06:05:54.365536+0000","last_fullsized":"2024-04-04T06:05:54.365536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:07:53.196090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593367+0000","last_change":"2024-04-04T06:05:53.371961+0000","last_active":"2024-04-04T06:05:54.593367+0000","last_peered":"2024-04-04T06:05:54.593367+0000","last_clean":"2024-04-04T06:05:54.593367+0000","last_became_active":"2024-04-04T06:05:53.371702+0000","last_became_peered":"2024-04-04T06:05:53.371702+0000","last_unstale":"2024-04-04T06:05:54.593367+0000","last_undegraded":"2024-04-04T06:05:54.593367+0000","last_fullsized":"2024-04-04T06:05:54.593367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592985+0000","last_change":"2024-04-04T06:05:53.383384+0000","last_active":"2024-04-04T06:05:54.592985+0000","last_peered":"2024-04-04T06:05:54.592985+0000","last_clean":"2024-04-04T06:05:54.592985+0000","last_became_active":"2024-04-04T06:05:53.382809+0000","last_became_peered":"2024-04-04T06:05:53.382809+0000","last_unstale":"2024-04-04T06:05:54.592985+0000","last_undegraded":"2024-04-04T06:05:54.592985+0000","last_fullsized":"2024-04-04T06:05:54.592985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:15:27.732386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362466+0000","last_change":"2024-04-04T06:05:53.373499+0000","last_active":"2024-04-04T06:05:54.362466+0000","last_peered":"2024-04-04T06:05:54.362466+0000","last_clean":"2024-04-04T06:05:54.362466+0000","last_became_active":"2024-04-04T06:05:53.367324+0000","last_became_peered":"2024-04-04T06:05:53.367324+0000","last_unstale":"2024-04-04T06:05:54.362466+0000","last_undegraded":"2024-04-04T06:05:54.362466+0000","last_fullsized":"2024-04-04T06:05:54.362466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:21:23.535889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593436+0000","last_change":"2024-04-04T06:05:53.371688+0000","last_active":"2024-04-04T06:05:54.593436+0000","last_peered":"2024-04-04T06:05:54.593436+0000","last_clean":"2024-04-04T06:05:54.593436+0000","last_became_active":"2024-04-04T06:05:53.371350+0000","last_became_peered":"2024-04-04T06:05:53.371350+0000","last_unstale":"2024-04-04T06:05:54.593436+0000","last_undegraded":"2024-04-04T06:05:54.593436+0000","last_fullsized":"2024-04-04T06:05:54.593436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:07:53.196090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590942+0000","last_change":"2024-04-04T06:05:53.370489+0000","last_active":"2024-04-04T06:05:54.590942+0000","last_peered":"2024-04-04T06:05:54.590942+0000","last_clean":"2024-04-04T06:05:54.590942+0000","last_became_active":"2024-04-04T06:05:53.367223+0000","last_became_peered":"2024-04-04T06:05:53.367223+0000","last_unstale":"2024-04-04T06:05:54.590942+0000","last_undegraded":"2024-04-04T06:05:54.590942+0000","last_fullsized":"2024-04-04T06:05:54.590942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:28:13.818701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368055+0000","last_change":"2024-04-04T06:05:53.364565+0000","last_active":"2024-04-04T06:05:54.368055+0000","last_peered":"2024-04-04T06:05:54.368055+0000","last_clean":"2024-04-04T06:05:54.368055+0000","last_became_active":"2024-04-04T06:05:53.364292+0000","last_became_peered":"2024-04-04T06:05:53.364292+0000","last_unstale":"2024-04-04T06:05:54.368055+0000","last_undegraded":"2024-04-04T06:05:54.368055+0000","last_fullsized":"2024-04-04T06:05:54.368055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:37:54.850791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590194+0000","last_change":"2024-04-04T06:05:53.366686+0000","last_active":"2024-04-04T06:05:54.590194+0000","last_peered":"2024-04-04T06:05:54.590194+0000","last_clean":"2024-04-04T06:05:54.590194+0000","last_became_active":"2024-04-04T06:05:53.366409+0000","last_became_peered":"2024-04-04T06:05:53.366409+0000","last_unstale":"2024-04-04T06:05:54.590194+0000","last_undegraded":"2024-04-04T06:05:54.590194+0000","last_fullsized":"2024-04-04T06:05:54.590194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:21.889411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591239+0000","last_change":"2024-04-04T06:05:53.361681+0000","last_active":"2024-04-04T06:05:54.591239+0000","last_peered":"2024-04-04T06:05:54.591239+0000","last_clean":"2024-04-04T06:05:54.591239+0000","last_became_active":"2024-04-04T06:05:53.361307+0000","last_became_peered":"2024-04-04T06:05:53.361307+0000","last_unstale":"2024-04-04T06:05:54.591239+0000","last_undegraded":"2024-04-04T06:05:54.591239+0000","last_fullsized":"2024-04-04T06:05:54.591239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:16:20.634573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.364842+0000","last_change":"2024-04-04T06:05:53.361164+0000","last_active":"2024-04-04T06:05:54.364842+0000","last_peered":"2024-04-04T06:05:54.364842+0000","last_clean":"2024-04-04T06:05:54.364842+0000","last_became_active":"2024-04-04T06:05:53.357301+0000","last_became_peered":"2024-04-04T06:05:53.357301+0000","last_unstale":"2024-04-04T06:05:54.364842+0000","last_undegraded":"2024-04-04T06:05:54.364842+0000","last_fullsized":"2024-04-04T06:05:54.364842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:56:10.393146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594020+0000","last_change":"2024-04-04T06:05:52.368795+0000","last_active":"2024-04-04T06:05:54.594020+0000","last_peered":"2024-04-04T06:05:54.594020+0000","last_clean":"2024-04-04T06:05:54.594020+0000","last_became_active":"2024-04-04T06:05:52.368526+0000","last_became_peered":"2024-04-04T06:05:52.368526+0000","last_unstale":"2024-04-04T06:05:54.594020+0000","last_undegraded":"2024-04-04T06:05:54.594020+0000","last_fullsized":"2024-04-04T06:05:54.594020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:45:43.480308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591181+0000","last_change":"2024-04-04T06:05:53.377207+0000","last_active":"2024-04-04T06:05:54.591181+0000","last_peered":"2024-04-04T06:05:54.591181+0000","last_clean":"2024-04-04T06:05:54.591181+0000","last_became_active":"2024-04-04T06:05:53.376853+0000","last_became_peered":"2024-04-04T06:05:53.376853+0000","last_unstale":"2024-04-04T06:05:54.591181+0000","last_undegraded":"2024-04-04T06:05:54.591181+0000","last_fullsized":"2024-04-04T06:05:54.591181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:21:23.535889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591256+0000","last_change":"2024-04-04T06:05:52.372922+0000","last_active":"2024-04-04T06:05:54.591256+0000","last_peered":"2024-04-04T06:05:54.591256+0000","last_clean":"2024-04-04T06:05:54.591256+0000","last_became_active":"2024-04-04T06:05:52.372615+0000","last_became_peered":"2024-04-04T06:05:52.372615+0000","last_unstale":"2024-04-04T06:05:54.591256+0000","last_undegraded":"2024-04-04T06:05:54.591256+0000","last_fullsized":"2024-04-04T06:05:54.591256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:36:38.928830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592633+0000","last_change":"2024-04-04T06:05:52.371943+0000","last_active":"2024-04-04T06:05:54.592633+0000","last_peered":"2024-04-04T06:05:54.592633+0000","last_clean":"2024-04-04T06:05:54.592633+0000","last_became_active":"2024-04-04T06:05:52.371686+0000","last_became_peered":"2024-04-04T06:05:52.371686+0000","last_unstale":"2024-04-04T06:05:54.592633+0000","last_undegraded":"2024-04-04T06:05:54.592633+0000","last_fullsized":"2024-04-04T06:05:54.592633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:52:05.788594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592504+0000","last_change":"2024-04-04T06:05:53.363121+0000","last_active":"2024-04-04T06:05:54.592504+0000","last_peered":"2024-04-04T06:05:54.592504+0000","last_clean":"2024-04-04T06:05:54.592504+0000","last_became_active":"2024-04-04T06:05:53.362861+0000","last_became_peered":"2024-04-04T06:05:53.362861+0000","last_unstale":"2024-04-04T06:05:54.592504+0000","last_undegraded":"2024-04-04T06:05:54.592504+0000","last_fullsized":"2024-04-04T06:05:54.592504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:28:13.818701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365617+0000","last_change":"2024-04-04T06:05:52.371090+0000","last_active":"2024-04-04T06:05:54.365617+0000","last_peered":"2024-04-04T06:05:54.365617+0000","last_clean":"2024-04-04T06:05:54.365617+0000","last_became_active":"2024-04-04T06:05:52.369650+0000","last_became_peered":"2024-04-04T06:05:52.369650+0000","last_unstale":"2024-04-04T06:05:54.365617+0000","last_undegraded":"2024-04-04T06:05:54.365617+0000","last_fullsized":"2024-04-04T06:05:54.365617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:45:43.480308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362432+0000","last_change":"2024-04-04T06:05:53.357537+0000","last_active":"2024-04-04T06:05:54.362432+0000","last_peered":"2024-04-04T06:05:54.362432+0000","last_clean":"2024-04-04T06:05:54.362432+0000","last_became_active":"2024-04-04T06:05:53.357284+0000","last_became_peered":"2024-04-04T06:05:53.357284+0000","last_unstale":"2024-04-04T06:05:54.362432+0000","last_undegraded":"2024-04-04T06:05:54.362432+0000","last_fullsized":"2024-04-04T06:05:54.362432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:28:13.818701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592398+0000","last_change":"2024-04-04T06:05:52.374921+0000","last_active":"2024-04-04T06:05:54.592398+0000","last_peered":"2024-04-04T06:05:54.592398+0000","last_clean":"2024-04-04T06:05:54.592398+0000","last_became_active":"2024-04-04T06:05:52.374555+0000","last_became_peered":"2024-04-04T06:05:52.374555+0000","last_unstale":"2024-04-04T06:05:54.592398+0000","last_undegraded":"2024-04-04T06:05:54.592398+0000","last_fullsized":"2024-04-04T06:05:54.592398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:05.925335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368597+0000","last_change":"2024-04-04T06:05:53.377612+0000","last_active":"2024-04-04T06:05:54.368597+0000","last_peered":"2024-04-04T06:05:54.368597+0000","last_clean":"2024-04-04T06:05:54.368597+0000","last_became_active":"2024-04-04T06:05:53.374258+0000","last_became_peered":"2024-04-04T06:05:53.374258+0000","last_unstale":"2024-04-04T06:05:54.368597+0000","last_undegraded":"2024-04-04T06:05:54.368597+0000","last_fullsized":"2024-04-04T06:05:54.368597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:08:44.769578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368664+0000","last_change":"2024-04-04T06:05:52.373979+0000","last_active":"2024-04-04T06:05:54.368664+0000","last_peered":"2024-04-04T06:05:54.368664+0000","last_clean":"2024-04-04T06:05:54.368664+0000","last_became_active":"2024-04-04T06:05:52.373408+0000","last_became_peered":"2024-04-04T06:05:52.373408+0000","last_unstale":"2024-04-04T06:05:54.368664+0000","last_undegraded":"2024-04-04T06:05:54.368664+0000","last_fullsized":"2024-04-04T06:05:54.368664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:56:09.383441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591338+0000","last_change":"2024-04-04T06:05:53.368728+0000","last_active":"2024-04-04T06:05:54.591338+0000","last_peered":"2024-04-04T06:05:54.591338+0000","last_clean":"2024-04-04T06:05:54.591338+0000","last_became_active":"2024-04-04T06:05:53.367971+0000","last_became_peered":"2024-04-04T06:05:53.367971+0000","last_unstale":"2024-04-04T06:05:54.591338+0000","last_undegraded":"2024-04-04T06:05:54.591338+0000","last_fullsized":"2024-04-04T06:05:54.591338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362602+0000","last_change":"2024-04-04T06:05:52.368574+0000","last_active":"2024-04-04T06:05:54.362602+0000","last_peered":"2024-04-04T06:05:54.362602+0000","last_clean":"2024-04-04T06:05:54.362602+0000","last_became_active":"2024-04-04T06:05:52.367460+0000","last_became_peered":"2024-04-04T06:05:52.367460+0000","last_unstale":"2024-04-04T06:05:54.362602+0000","last_undegraded":"2024-04-04T06:05:54.362602+0000","last_fullsized":"2024-04-04T06:05:54.362602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:56:09.383441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367547+0000","last_change":"2024-04-04T06:05:53.379475+0000","last_active":"2024-04-04T06:05:54.367547+0000","last_peered":"2024-04-04T06:05:54.367547+0000","last_clean":"2024-04-04T06:05:54.367547+0000","last_became_active":"2024-04-04T06:05:53.376075+0000","last_became_peered":"2024-04-04T06:05:53.376075+0000","last_unstale":"2024-04-04T06:05:54.367547+0000","last_undegraded":"2024-04-04T06:05:54.367547+0000","last_fullsized":"2024-04-04T06:05:54.367547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:03:00.104400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591818+0000","last_change":"2024-04-04T06:05:52.370133+0000","last_active":"2024-04-04T06:05:54.591818+0000","last_peered":"2024-04-04T06:05:54.591818+0000","last_clean":"2024-04-04T06:05:54.591818+0000","last_became_active":"2024-04-04T06:05:52.369825+0000","last_became_peered":"2024-04-04T06:05:52.369825+0000","last_unstale":"2024-04-04T06:05:54.591818+0000","last_undegraded":"2024-04-04T06:05:54.591818+0000","last_fullsized":"2024-04-04T06:05:54.591818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:01:02.629926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594503+0000","last_change":"2024-04-04T06:05:53.371020+0000","last_active":"2024-04-04T06:05:54.594503+0000","last_peered":"2024-04-04T06:05:54.594503+0000","last_clean":"2024-04-04T06:05:54.594503+0000","last_became_active":"2024-04-04T06:05:53.370755+0000","last_became_peered":"2024-04-04T06:05:53.370755+0000","last_unstale":"2024-04-04T06:05:54.594503+0000","last_undegraded":"2024-04-04T06:05:54.594503+0000","last_fullsized":"2024-04-04T06:05:54.594503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:54:36.353079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591316+0000","last_change":"2024-04-04T06:05:52.367020+0000","last_active":"2024-04-04T06:05:54.591316+0000","last_peered":"2024-04-04T06:05:54.591316+0000","last_clean":"2024-04-04T06:05:54.591316+0000","last_became_active":"2024-04-04T06:05:52.366654+0000","last_became_peered":"2024-04-04T06:05:52.366654+0000","last_unstale":"2024-04-04T06:05:54.591316+0000","last_undegraded":"2024-04-04T06:05:54.591316+0000","last_fullsized":"2024-04-04T06:05:54.591316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:26:03.163432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590433+0000","last_change":"2024-04-04T06:05:53.373111+0000","last_active":"2024-04-04T06:05:54.590433+0000","last_peered":"2024-04-04T06:05:54.590433+0000","last_clean":"2024-04-04T06:05:54.590433+0000","last_became_active":"2024-04-04T06:05:53.372761+0000","last_became_peered":"2024-04-04T06:05:53.372761+0000","last_unstale":"2024-04-04T06:05:54.590433+0000","last_undegraded":"2024-04-04T06:05:54.590433+0000","last_fullsized":"2024-04-04T06:05:54.590433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:27:06.279930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368191+0000","last_change":"2024-04-04T06:05:52.365334+0000","last_active":"2024-04-04T06:05:54.368191+0000","last_peered":"2024-04-04T06:05:54.368191+0000","last_clean":"2024-04-04T06:05:54.368191+0000","last_became_active":"2024-04-04T06:05:52.363154+0000","last_became_peered":"2024-04-04T06:05:52.363154+0000","last_unstale":"2024-04-04T06:05:54.368191+0000","last_undegraded":"2024-04-04T06:05:54.368191+0000","last_fullsized":"2024-04-04T06:05:54.368191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:53:42.904072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593559+0000","last_change":"2024-04-04T06:05:53.380196+0000","last_active":"2024-04-04T06:05:54.593559+0000","last_peered":"2024-04-04T06:05:54.593559+0000","last_clean":"2024-04-04T06:05:54.593559+0000","last_became_active":"2024-04-04T06:05:53.379800+0000","last_became_peered":"2024-04-04T06:05:53.379800+0000","last_unstale":"2024-04-04T06:05:54.593559+0000","last_undegraded":"2024-04-04T06:05:54.593559+0000","last_fullsized":"2024-04-04T06:05:54.593559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:41:09.510239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591089+0000","last_change":"2024-04-04T06:05:52.377332+0000","last_active":"2024-04-04T06:05:54.591089+0000","last_peered":"2024-04-04T06:05:54.591089+0000","last_clean":"2024-04-04T06:05:54.591089+0000","last_became_active":"2024-04-04T06:05:52.377002+0000","last_became_peered":"2024-04-04T06:05:52.377002+0000","last_unstale":"2024-04-04T06:05:54.591089+0000","last_undegraded":"2024-04-04T06:05:54.591089+0000","last_fullsized":"2024-04-04T06:05:54.591089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:05:57.500385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591711+0000","last_change":"2024-04-04T06:05:53.378591+0000","last_active":"2024-04-04T06:05:54.591711+0000","last_peered":"2024-04-04T06:05:54.591711+0000","last_clean":"2024-04-04T06:05:54.591711+0000","last_became_active":"2024-04-04T06:05:53.378227+0000","last_became_peered":"2024-04-04T06:05:53.378227+0000","last_unstale":"2024-04-04T06:05:54.591711+0000","last_undegraded":"2024-04-04T06:05:54.591711+0000","last_fullsized":"2024-04-04T06:05:54.591711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:42:54.347331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.589595+0000","last_change":"2024-04-04T06:05:52.357745+0000","last_active":"2024-04-04T06:05:54.589595+0000","last_peered":"2024-04-04T06:05:54.589595+0000","last_clean":"2024-04-04T06:05:54.589595+0000","last_became_active":"2024-04-04T06:05:52.357280+0000","last_became_peered":"2024-04-04T06:05:52.357280+0000","last_unstale":"2024-04-04T06:05:54.589595+0000","last_undegraded":"2024-04-04T06:05:54.589595+0000","last_fullsized":"2024-04-04T06:05:54.589595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:45:43.480308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592420+0000","last_change":"2024-04-04T06:05:53.387060+0000","last_active":"2024-04-04T06:05:54.592420+0000","last_peered":"2024-04-04T06:05:54.592420+0000","last_clean":"2024-04-04T06:05:54.592420+0000","last_became_active":"2024-04-04T06:05:53.386804+0000","last_became_peered":"2024-04-04T06:05:53.386804+0000","last_unstale":"2024-04-04T06:05:54.592420+0000","last_undegraded":"2024-04-04T06:05:54.592420+0000","last_fullsized":"2024-04-04T06:05:54.592420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:37:36.681617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367992+0000","last_change":"2024-04-04T06:05:52.362010+0000","last_active":"2024-04-04T06:05:54.367992+0000","last_peered":"2024-04-04T06:05:54.367992+0000","last_clean":"2024-04-04T06:05:54.367992+0000","last_became_active":"2024-04-04T06:05:52.361424+0000","last_became_peered":"2024-04-04T06:05:52.361424+0000","last_unstale":"2024-04-04T06:05:54.367992+0000","last_undegraded":"2024-04-04T06:05:54.367992+0000","last_fullsized":"2024-04-04T06:05:54.367992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:26:03.163432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594202+0000","last_change":"2024-04-04T06:05:53.375944+0000","last_active":"2024-04-04T06:05:54.594202+0000","last_peered":"2024-04-04T06:05:54.594202+0000","last_clean":"2024-04-04T06:05:54.594202+0000","last_became_active":"2024-04-04T06:05:53.375684+0000","last_became_peered":"2024-04-04T06:05:53.375684+0000","last_unstale":"2024-04-04T06:05:54.594202+0000","last_undegraded":"2024-04-04T06:05:54.594202+0000","last_fullsized":"2024-04-04T06:05:54.594202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:02:57.074580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591274+0000","last_change":"2024-04-04T06:05:52.358735+0000","last_active":"2024-04-04T06:05:54.591274+0000","last_peered":"2024-04-04T06:05:54.591274+0000","last_clean":"2024-04-04T06:05:54.591274+0000","last_became_active":"2024-04-04T06:05:52.358203+0000","last_became_peered":"2024-04-04T06:05:52.358203+0000","last_unstale":"2024-04-04T06:05:54.591274+0000","last_undegraded":"2024-04-04T06:05:54.591274+0000","last_fullsized":"2024-04-04T06:05:54.591274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:53:42.904072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594617+0000","last_change":"2024-04-04T06:05:53.388782+0000","last_active":"2024-04-04T06:05:54.594617+0000","last_peered":"2024-04-04T06:05:54.594617+0000","last_clean":"2024-04-04T06:05:54.594617+0000","last_became_active":"2024-04-04T06:05:53.388230+0000","last_became_peered":"2024-04-04T06:05:53.388230+0000","last_unstale":"2024-04-04T06:05:54.594617+0000","last_undegraded":"2024-04-04T06:05:54.594617+0000","last_fullsized":"2024-04-04T06:05:54.594617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:31:19.282600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362887+0000","last_change":"2024-04-04T06:05:52.366026+0000","last_active":"2024-04-04T06:05:54.362887+0000","last_peered":"2024-04-04T06:05:54.362887+0000","last_clean":"2024-04-04T06:05:54.362887+0000","last_became_active":"2024-04-04T06:05:52.365663+0000","last_became_peered":"2024-04-04T06:05:52.365663+0000","last_unstale":"2024-04-04T06:05:54.362887+0000","last_undegraded":"2024-04-04T06:05:54.362887+0000","last_fullsized":"2024-04-04T06:05:54.362887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:30.714156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.366141+0000","last_change":"2024-04-04T06:05:53.381556+0000","last_active":"2024-04-04T06:05:54.366141+0000","last_peered":"2024-04-04T06:05:54.366141+0000","last_clean":"2024-04-04T06:05:54.366141+0000","last_became_active":"2024-04-04T06:05:53.373492+0000","last_became_peered":"2024-04-04T06:05:53.373492+0000","last_unstale":"2024-04-04T06:05:54.366141+0000","last_undegraded":"2024-04-04T06:05:54.366141+0000","last_fullsized":"2024-04-04T06:05:54.366141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:53:19.419468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590914+0000","last_change":"2024-04-04T06:05:52.353584+0000","last_active":"2024-04-04T06:05:54.590914+0000","last_peered":"2024-04-04T06:05:54.590914+0000","last_clean":"2024-04-04T06:05:54.590914+0000","last_became_active":"2024-04-04T06:05:52.353017+0000","last_became_peered":"2024-04-04T06:05:52.353017+0000","last_unstale":"2024-04-04T06:05:54.590914+0000","last_undegraded":"2024-04-04T06:05:54.590914+0000","last_fullsized":"2024-04-04T06:05:54.590914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:58:40.874784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593698+0000","last_change":"2024-04-04T06:05:53.368622+0000","last_active":"2024-04-04T06:05:54.593698+0000","last_peered":"2024-04-04T06:05:54.593698+0000","last_clean":"2024-04-04T06:05:54.593698+0000","last_became_active":"2024-04-04T06:05:53.368311+0000","last_became_peered":"2024-04-04T06:05:53.368311+0000","last_unstale":"2024-04-04T06:05:54.593698+0000","last_undegraded":"2024-04-04T06:05:54.593698+0000","last_fullsized":"2024-04-04T06:05:54.593698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:01:24.602038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591729+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:05:54.591729+0000","last_peered":"2024-04-04T06:05:54.591729+0000","last_clean":"2024-04-04T06:05:54.591729+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:05:54.591729+0000","last_undegraded":"2024-04-04T06:05:54.591729+0000","last_fullsized":"2024-04-04T06:05:54.591729+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365145+0000","last_change":"2024-04-04T06:05:52.356730+0000","last_active":"2024-04-04T06:05:54.365145+0000","last_peered":"2024-04-04T06:05:54.365145+0000","last_clean":"2024-04-04T06:05:54.365145+0000","last_became_active":"2024-04-04T06:05:52.352963+0000","last_became_peered":"2024-04-04T06:05:52.352963+0000","last_unstale":"2024-04-04T06:05:54.365145+0000","last_undegraded":"2024-04-04T06:05:54.365145+0000","last_fullsized":"2024-04-04T06:05:54.365145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:05:06.878796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592869+0000","last_change":"2024-04-04T06:05:53.366506+0000","last_active":"2024-04-04T06:05:54.592869+0000","last_peered":"2024-04-04T06:05:54.592869+0000","last_clean":"2024-04-04T06:05:54.592869+0000","last_became_active":"2024-04-04T06:05:53.365677+0000","last_became_peered":"2024-04-04T06:05:53.365677+0000","last_unstale":"2024-04-04T06:05:54.592869+0000","last_undegraded":"2024-04-04T06:05:54.592869+0000","last_fullsized":"2024-04-04T06:05:54.592869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:10.411661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591456+0000","last_change":"2024-04-04T06:05:52.374798+0000","last_active":"2024-04-04T06:05:54.591456+0000","last_peered":"2024-04-04T06:05:54.591456+0000","last_clean":"2024-04-04T06:05:54.591456+0000","last_became_active":"2024-04-04T06:05:52.373838+0000","last_became_peered":"2024-04-04T06:05:52.373838+0000","last_unstale":"2024-04-04T06:05:54.591456+0000","last_undegraded":"2024-04-04T06:05:54.591456+0000","last_fullsized":"2024-04-04T06:05:54.591456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:51:50.301107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590751+0000","last_change":"2024-04-04T06:05:53.385393+0000","last_active":"2024-04-04T06:05:54.590751+0000","last_peered":"2024-04-04T06:05:54.590751+0000","last_clean":"2024-04-04T06:05:54.590751+0000","last_became_active":"2024-04-04T06:05:53.385062+0000","last_became_peered":"2024-04-04T06:05:53.385062+0000","last_unstale":"2024-04-04T06:05:54.590751+0000","last_undegraded":"2024-04-04T06:05:54.590751+0000","last_fullsized":"2024-04-04T06:05:54.590751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:14:41.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590309+0000","last_change":"2024-04-04T06:05:52.360018+0000","last_active":"2024-04-04T06:05:54.590309+0000","last_peered":"2024-04-04T06:05:54.590309+0000","last_clean":"2024-04-04T06:05:54.590309+0000","last_became_active":"2024-04-04T06:05:52.359735+0000","last_became_peered":"2024-04-04T06:05:52.359735+0000","last_unstale":"2024-04-04T06:05:54.590309+0000","last_undegraded":"2024-04-04T06:05:54.590309+0000","last_fullsized":"2024-04-04T06:05:54.590309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:05:06.878796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.363467+0000","last_change":"2024-04-04T06:05:53.356771+0000","last_active":"2024-04-04T06:05:54.363467+0000","last_peered":"2024-04-04T06:05:54.363467+0000","last_clean":"2024-04-04T06:05:54.363467+0000","last_became_active":"2024-04-04T06:05:53.356431+0000","last_became_peered":"2024-04-04T06:05:53.356431+0000","last_unstale":"2024-04-04T06:05:54.363467+0000","last_undegraded":"2024-04-04T06:05:54.363467+0000","last_fullsized":"2024-04-04T06:05:54.363467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:07:53.196090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368163+0000","last_change":"2024-04-04T06:05:52.370387+0000","last_active":"2024-04-04T06:05:54.368163+0000","last_peered":"2024-04-04T06:05:54.368163+0000","last_clean":"2024-04-04T06:05:54.368163+0000","last_became_active":"2024-04-04T06:05:52.369905+0000","last_became_peered":"2024-04-04T06:05:52.369905+0000","last_unstale":"2024-04-04T06:05:54.368163+0000","last_undegraded":"2024-04-04T06:05:54.368163+0000","last_fullsized":"2024-04-04T06:05:54.368163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:58:40.874784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367307+0000","last_change":"2024-04-04T06:05:53.371957+0000","last_active":"2024-04-04T06:05:54.367307+0000","last_peered":"2024-04-04T06:05:54.367307+0000","last_clean":"2024-04-04T06:05:54.367307+0000","last_became_active":"2024-04-04T06:05:53.371121+0000","last_became_peered":"2024-04-04T06:05:53.371121+0000","last_unstale":"2024-04-04T06:05:54.367307+0000","last_undegraded":"2024-04-04T06:05:54.367307+0000","last_fullsized":"2024-04-04T06:05:54.367307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:31.723861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362920+0000","last_change":"2024-04-04T06:05:52.361392+0000","last_active":"2024-04-04T06:05:54.362920+0000","last_peered":"2024-04-04T06:05:54.362920+0000","last_clean":"2024-04-04T06:05:54.362920+0000","last_became_active":"2024-04-04T06:05:52.360932+0000","last_became_peered":"2024-04-04T06:05:52.360932+0000","last_unstale":"2024-04-04T06:05:54.362920+0000","last_undegraded":"2024-04-04T06:05:54.362920+0000","last_fullsized":"2024-04-04T06:05:54.362920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:20.879706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365934+0000","last_change":"2024-04-04T06:05:53.379438+0000","last_active":"2024-04-04T06:05:54.365934+0000","last_peered":"2024-04-04T06:05:54.365934+0000","last_clean":"2024-04-04T06:05:54.365934+0000","last_became_active":"2024-04-04T06:05:53.378781+0000","last_became_peered":"2024-04-04T06:05:53.378781+0000","last_unstale":"2024-04-04T06:05:54.365934+0000","last_undegraded":"2024-04-04T06:05:54.365934+0000","last_fullsized":"2024-04-04T06:05:54.365934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:18:05.337194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594629+0000","last_change":"2024-04-04T06:05:52.372922+0000","last_active":"2024-04-04T06:05:54.594629+0000","last_peered":"2024-04-04T06:05:54.594629+0000","last_clean":"2024-04-04T06:05:54.594629+0000","last_became_active":"2024-04-04T06:05:52.372696+0000","last_became_peered":"2024-04-04T06:05:52.372696+0000","last_unstale":"2024-04-04T06:05:54.594629+0000","last_undegraded":"2024-04-04T06:05:54.594629+0000","last_fullsized":"2024-04-04T06:05:54.594629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:17:53.847287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362796+0000","last_change":"2024-04-04T06:05:53.373066+0000","last_active":"2024-04-04T06:05:54.362796+0000","last_peered":"2024-04-04T06:05:54.362796+0000","last_clean":"2024-04-04T06:05:54.362796+0000","last_became_active":"2024-04-04T06:05:53.367063+0000","last_became_peered":"2024-04-04T06:05:53.367063+0000","last_unstale":"2024-04-04T06:05:54.362796+0000","last_undegraded":"2024-04-04T06:05:54.362796+0000","last_fullsized":"2024-04-04T06:05:54.362796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:10.411661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.363331+0000","last_change":"2024-04-04T06:05:52.365125+0000","last_active":"2024-04-04T06:05:54.363331+0000","last_peered":"2024-04-04T06:05:54.363331+0000","last_clean":"2024-04-04T06:05:54.363331+0000","last_became_active":"2024-04-04T06:05:52.364821+0000","last_became_peered":"2024-04-04T06:05:52.364821+0000","last_unstale":"2024-04-04T06:05:54.363331+0000","last_undegraded":"2024-04-04T06:05:54.363331+0000","last_fullsized":"2024-04-04T06:05:54.363331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:54:42.610854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593547+0000","last_change":"2024-04-04T06:05:53.375887+0000","last_active":"2024-04-04T06:05:54.593547+0000","last_peered":"2024-04-04T06:05:54.593547+0000","last_clean":"2024-04-04T06:05:54.593547+0000","last_became_active":"2024-04-04T06:05:53.374688+0000","last_became_peered":"2024-04-04T06:05:53.374688+0000","last_unstale":"2024-04-04T06:05:54.593547+0000","last_undegraded":"2024-04-04T06:05:54.593547+0000","last_fullsized":"2024-04-04T06:05:54.593547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:45:21.548886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592567+0000","last_change":"2024-04-04T06:05:52.374804+0000","last_active":"2024-04-04T06:05:54.592567+0000","last_peered":"2024-04-04T06:05:54.592567+0000","last_clean":"2024-04-04T06:05:54.592567+0000","last_became_active":"2024-04-04T06:05:52.374476+0000","last_became_peered":"2024-04-04T06:05:52.374476+0000","last_unstale":"2024-04-04T06:05:54.592567+0000","last_undegraded":"2024-04-04T06:05:54.592567+0000","last_fullsized":"2024-04-04T06:05:54.592567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:21:06.579458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590233+0000","last_change":"2024-04-04T06:05:53.358691+0000","last_active":"2024-04-04T06:05:54.590233+0000","last_peered":"2024-04-04T06:05:54.590233+0000","last_clean":"2024-04-04T06:05:54.590233+0000","last_became_active":"2024-04-04T06:05:53.357996+0000","last_became_peered":"2024-04-04T06:05:53.357996+0000","last_unstale":"2024-04-04T06:05:54.590233+0000","last_undegraded":"2024-04-04T06:05:54.590233+0000","last_fullsized":"2024-04-04T06:05:54.590233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:55:58.689192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592379+0000","last_change":"2024-04-04T06:05:52.370047+0000","last_active":"2024-04-04T06:05:54.592379+0000","last_peered":"2024-04-04T06:05:54.592379+0000","last_clean":"2024-04-04T06:05:54.592379+0000","last_became_active":"2024-04-04T06:05:52.369741+0000","last_became_peered":"2024-04-04T06:05:52.369741+0000","last_unstale":"2024-04-04T06:05:54.592379+0000","last_undegraded":"2024-04-04T06:05:54.592379+0000","last_fullsized":"2024-04-04T06:05:54.592379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:36:38.928830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.364951+0000","last_change":"2024-04-04T06:05:53.364563+0000","last_active":"2024-04-04T06:05:54.364951+0000","last_peered":"2024-04-04T06:05:54.364951+0000","last_clean":"2024-04-04T06:05:54.364951+0000","last_became_active":"2024-04-04T06:05:53.362702+0000","last_became_peered":"2024-04-04T06:05:53.362702+0000","last_unstale":"2024-04-04T06:05:54.364951+0000","last_undegraded":"2024-04-04T06:05:54.364951+0000","last_fullsized":"2024-04-04T06:05:54.364951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:54:43.620559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362753+0000","last_change":"2024-04-04T06:05:52.370003+0000","last_active":"2024-04-04T06:05:54.362753+0000","last_peered":"2024-04-04T06:05:54.362753+0000","last_clean":"2024-04-04T06:05:54.362753+0000","last_became_active":"2024-04-04T06:05:52.369724+0000","last_became_peered":"2024-04-04T06:05:52.369724+0000","last_unstale":"2024-04-04T06:05:54.362753+0000","last_undegraded":"2024-04-04T06:05:54.362753+0000","last_fullsized":"2024-04-04T06:05:54.362753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:58:40.874784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592885+0000","last_change":"2024-04-04T06:05:53.361642+0000","last_active":"2024-04-04T06:05:54.592885+0000","last_peered":"2024-04-04T06:05:54.592885+0000","last_clean":"2024-04-04T06:05:54.592885+0000","last_became_active":"2024-04-04T06:05:53.360951+0000","last_became_peered":"2024-04-04T06:05:53.360951+0000","last_unstale":"2024-04-04T06:05:54.592885+0000","last_undegraded":"2024-04-04T06:05:54.592885+0000","last_fullsized":"2024-04-04T06:05:54.592885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:21:23.535889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362582+0000","last_change":"2024-04-04T06:05:52.371223+0000","last_active":"2024-04-04T06:05:54.362582+0000","last_peered":"2024-04-04T06:05:54.362582+0000","last_clean":"2024-04-04T06:05:54.362582+0000","last_became_active":"2024-04-04T06:05:52.370916+0000","last_became_peered":"2024-04-04T06:05:52.370916+0000","last_unstale":"2024-04-04T06:05:54.362582+0000","last_undegraded":"2024-04-04T06:05:54.362582+0000","last_fullsized":"2024-04-04T06:05:54.362582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:53:42.904072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593889+0000","last_change":"2024-04-04T06:05:53.371605+0000","last_active":"2024-04-04T06:05:54.593889+0000","last_peered":"2024-04-04T06:05:54.593889+0000","last_clean":"2024-04-04T06:05:54.593889+0000","last_became_active":"2024-04-04T06:05:53.371328+0000","last_became_peered":"2024-04-04T06:05:53.371328+0000","last_unstale":"2024-04-04T06:05:54.593889+0000","last_undegraded":"2024-04-04T06:05:54.593889+0000","last_fullsized":"2024-04-04T06:05:54.593889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:18:05.337194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593483+0000","last_change":"2024-04-04T06:05:52.368424+0000","last_active":"2024-04-04T06:05:54.593483+0000","last_peered":"2024-04-04T06:05:54.593483+0000","last_clean":"2024-04-04T06:05:54.593483+0000","last_became_active":"2024-04-04T06:05:52.368164+0000","last_became_peered":"2024-04-04T06:05:52.368164+0000","last_unstale":"2024-04-04T06:05:54.593483+0000","last_undegraded":"2024-04-04T06:05:54.593483+0000","last_fullsized":"2024-04-04T06:05:54.593483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:54:18.736812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591138+0000","last_change":"2024-04-04T06:05:53.373894+0000","last_active":"2024-04-04T06:05:54.591138+0000","last_peered":"2024-04-04T06:05:54.591138+0000","last_clean":"2024-04-04T06:05:54.591138+0000","last_became_active":"2024-04-04T06:05:53.373615+0000","last_became_peered":"2024-04-04T06:05:53.373615+0000","last_unstale":"2024-04-04T06:05:54.591138+0000","last_undegraded":"2024-04-04T06:05:54.591138+0000","last_fullsized":"2024-04-04T06:05:54.591138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:03:39.636711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365635+0000","last_change":"2024-04-04T06:05:53.378971+0000","last_active":"2024-04-04T06:05:54.365635+0000","last_peered":"2024-04-04T06:05:54.365635+0000","last_clean":"2024-04-04T06:05:54.365635+0000","last_became_active":"2024-04-04T06:05:53.377287+0000","last_became_peered":"2024-04-04T06:05:53.377287+0000","last_unstale":"2024-04-04T06:05:54.365635+0000","last_undegraded":"2024-04-04T06:05:54.365635+0000","last_fullsized":"2024-04-04T06:05:54.365635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:17:05.576402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594399+0000","last_change":"2024-04-04T06:05:52.376155+0000","last_active":"2024-04-04T06:05:54.594399+0000","last_peered":"2024-04-04T06:05:54.594399+0000","last_clean":"2024-04-04T06:05:54.594399+0000","last_became_active":"2024-04-04T06:05:52.375772+0000","last_became_peered":"2024-04-04T06:05:52.375772+0000","last_unstale":"2024-04-04T06:05:54.594399+0000","last_undegraded":"2024-04-04T06:05:54.594399+0000","last_fullsized":"2024-04-04T06:05:54.594399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:40:09.906862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365922+0000","last_change":"2024-04-04T06:05:53.381179+0000","last_active":"2024-04-04T06:05:54.365922+0000","last_peered":"2024-04-04T06:05:54.365922+0000","last_clean":"2024-04-04T06:05:54.365922+0000","last_became_active":"2024-04-04T06:05:53.374859+0000","last_became_peered":"2024-04-04T06:05:53.374859+0000","last_unstale":"2024-04-04T06:05:54.365922+0000","last_undegraded":"2024-04-04T06:05:54.365922+0000","last_fullsized":"2024-04-04T06:05:54.365922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:20:24.966139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593848+0000","last_change":"2024-04-04T06:05:52.378067+0000","last_active":"2024-04-04T06:05:54.593848+0000","last_peered":"2024-04-04T06:05:54.593848+0000","last_clean":"2024-04-04T06:05:54.593848+0000","last_became_active":"2024-04-04T06:05:52.377777+0000","last_became_peered":"2024-04-04T06:05:52.377777+0000","last_unstale":"2024-04-04T06:05:54.593848+0000","last_undegraded":"2024-04-04T06:05:54.593848+0000","last_fullsized":"2024-04-04T06:05:54.593848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:01:02.629926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590922+0000","last_change":"2024-04-04T06:05:53.368705+0000","last_active":"2024-04-04T06:05:54.590922+0000","last_peered":"2024-04-04T06:05:54.590922+0000","last_clean":"2024-04-04T06:05:54.590922+0000","last_became_active":"2024-04-04T06:05:53.368366+0000","last_became_peered":"2024-04-04T06:05:53.368366+0000","last_unstale":"2024-04-04T06:05:54.590922+0000","last_undegraded":"2024-04-04T06:05:54.590922+0000","last_fullsized":"2024-04-04T06:05:54.590922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:34.766930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593814+0000","last_change":"2024-04-04T06:05:52.367095+0000","last_active":"2024-04-04T06:05:54.593814+0000","last_peered":"2024-04-04T06:05:54.593814+0000","last_clean":"2024-04-04T06:05:54.593814+0000","last_became_active":"2024-04-04T06:05:52.366316+0000","last_became_peered":"2024-04-04T06:05:52.366316+0000","last_unstale":"2024-04-04T06:05:54.593814+0000","last_undegraded":"2024-04-04T06:05:54.593814+0000","last_fullsized":"2024-04-04T06:05:54.593814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:05:06.878796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593646+0000","last_change":"2024-04-04T06:05:53.375226+0000","last_active":"2024-04-04T06:05:54.593646+0000","last_peered":"2024-04-04T06:05:54.593646+0000","last_clean":"2024-04-04T06:05:54.593646+0000","last_became_active":"2024-04-04T06:05:53.374921+0000","last_became_peered":"2024-04-04T06:05:53.374921+0000","last_unstale":"2024-04-04T06:05:54.593646+0000","last_undegraded":"2024-04-04T06:05:54.593646+0000","last_fullsized":"2024-04-04T06:05:54.593646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:20:24.966139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591478+0000","last_change":"2024-04-04T06:05:52.370572+0000","last_active":"2024-04-04T06:05:54.591478+0000","last_peered":"2024-04-04T06:05:54.591478+0000","last_clean":"2024-04-04T06:05:54.591478+0000","last_became_active":"2024-04-04T06:05:52.369378+0000","last_became_peered":"2024-04-04T06:05:52.369378+0000","last_unstale":"2024-04-04T06:05:54.591478+0000","last_undegraded":"2024-04-04T06:05:54.591478+0000","last_fullsized":"2024-04-04T06:05:54.591478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:45:47.508028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":806912,"data_stored":663433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":74095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":74095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":72483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":663489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":72573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":72609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:06:08.377 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:06:08.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.629+0000 7fbdbf200640 1 Processor -- start 2024-04-04T06:06:08.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.629+0000 7fbdbf200640 1 -- start start 2024-04-04T06:06:08.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.631+0000 7fbdbf200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fbdc00d2310 0x7fbdc00ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:08.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.631+0000 7fbdbf200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbdc00cf530 0x7fbdc00b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:08.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.631+0000 7fbdbf200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdc015e880 con 0x7fbdc015e4a0 2024-04-04T06:06:08.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.631+0000 7fbdbf200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fbdc00cf9a0 con 0x7fbdc00d2310 2024-04-04T06:06:08.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.631+0000 7fbdbf200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fbdc00d34e0 con 0x7fbdc00cf530 2024-04-04T06:06:08.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.631+0000 7fbdbde00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fbdc015e4a0 0x7fbdc00b9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:46006/0 (socket says 172.21.15.159:46006) 2024-04-04T06:06:08.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.631+0000 7fbdbde00640 1 -- 172.21.15.159:0/3944975777 learned_addr learned my addr 172.21.15.159:0/3944975777 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:08.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.631+0000 7fbdb5400640 1 --2- 172.21.15.159:0/3944975777 >> v2:172.21.15.159:3300/0 conn(0x7fbdc00d2310 0x7fbdc00ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:08.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.631+0000 7fbdbe800640 1 --2- 172.21.15.159:0/3944975777 >> v2:172.21.15.178:3300/0 conn(0x7fbdc00cf530 0x7fbdc00b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:08.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.631+0000 7fbdbd400640 1 -- 172.21.15.159:0/3944975777 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 929209178 0 0) 0x7fbdc015e880 con 0x7fbdc015e4a0 2024-04-04T06:06:08.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.631+0000 7fbdbd400640 1 -- 172.21.15.159:0/3944975777 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbda4003520 con 0x7fbdc015e4a0 2024-04-04T06:06:08.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.632+0000 7fbdb5400640 1 -- 172.21.15.159:0/3944975777 >> v2:172.21.15.178:3300/0 conn(0x7fbdc00cf530 msgr2=0x7fbdc00b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:08.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.632+0000 7fbdb5400640 1 --2- 172.21.15.159:0/3944975777 >> v2:172.21.15.178:3300/0 conn(0x7fbdc00cf530 0x7fbdc00b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:08.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.632+0000 7fbdb5400640 1 -- 172.21.15.159:0/3944975777 >> v1:172.21.15.159:6789/0 conn(0x7fbdc015e4a0 legacy=0x7fbdc00b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:08.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.632+0000 7fbdb5400640 1 -- 172.21.15.159:0/3944975777 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc00d28d0 con 0x7fbdc00d2310 2024-04-04T06:06:08.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.633+0000 7fbdb5400640 1 --2- 172.21.15.159:0/3944975777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdc00d2310 0x7fbdc00ceff0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00d3660 tx=0x7fbda802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19545a3bfc8f24fa server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:08.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.633+0000 7fbdbd400640 1 -- 172.21.15.159:0/3944975777 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda80315a0 con 0x7fbdc00d2310 2024-04-04T06:06:08.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.633+0000 7fbdbd400640 1 -- 172.21.15.159:0/3944975777 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbda8037070 con 0x7fbdc00d2310 2024-04-04T06:06:08.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.633+0000 7fbdbd400640 1 -- 172.21.15.159:0/3944975777 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8031960 con 0x7fbdc00d2310 2024-04-04T06:06:08.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.633+0000 7fbdbf200640 1 -- 172.21.15.159:0/3944975777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdc00d2310 msgr2=0x7fbdc00ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:08.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.633+0000 7fbdbf200640 1 --2- 172.21.15.159:0/3944975777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdc00d2310 0x7fbdc00ceff0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00d3660 tx=0x7fbda802b840 comp rx=0 tx=0).stop 2024-04-04T06:06:08.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.634+0000 7fbdbf200640 1 -- 172.21.15.159:0/3944975777 shutdown_connections 2024-04-04T06:06:08.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.634+0000 7fbdbf200640 1 --2- 172.21.15.159:0/3944975777 >> v2:172.21.15.178:3300/0 conn(0x7fbdc00cf530 0x7fbdc00b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:08.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.634+0000 7fbdbf200640 1 --2- 172.21.15.159:0/3944975777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdc00d2310 0x7fbdc00ceff0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:08.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.634+0000 7fbdbf200640 1 -- 172.21.15.159:0/3944975777 >> 172.21.15.159:0/3944975777 conn(0x7fbdc00676d0 msgr2=0x7fbdc0067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:08.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.634+0000 7fbdbf200640 1 -- 172.21.15.159:0/3944975777 shutdown_connections 2024-04-04T06:06:08.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.634+0000 7fbdbf200640 1 -- 172.21.15.159:0/3944975777 wait complete. 2024-04-04T06:06:08.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.635+0000 7fbdbf200640 1 Processor -- start 2024-04-04T06:06:08.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.635+0000 7fbdbf200640 1 -- start start 2024-04-04T06:06:08.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.636+0000 7fbdbf200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdc00cf530 0x7fbdc00c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:08.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.636+0000 7fbdbf200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbdc00d2310 0x7fbdc00c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:08.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.636+0000 7fbdbf200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdc015e4a0 0x7fbdc015fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:08.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.636+0000 7fbdbf200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc00d3170 con 0x7fbdc00cf530 2024-04-04T06:06:08.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.636+0000 7fbdbf200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc00d2e90 con 0x7fbdc015e4a0 2024-04-04T06:06:08.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.636+0000 7fbdbf200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc00d3970 con 0x7fbdc00d2310 2024-04-04T06:06:08.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.637+0000 7fbdbde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdc00cf530 0x7fbdc00c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:08.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.637+0000 7fbdbde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdc00cf530 0x7fbdc00c97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37328/0 (socket says 172.21.15.159:37328) 2024-04-04T06:06:08.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.637+0000 7fbdbde00640 1 -- 172.21.15.159:0/513563827 learned_addr learned my addr 172.21.15.159:0/513563827 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:08.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.637+0000 7fbdbe800640 1 --2- 172.21.15.159:0/513563827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdc015e4a0 0x7fbdc015fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:08.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.637+0000 7fbdb5400640 1 --2- 172.21.15.159:0/513563827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbdc00d2310 0x7fbdc00c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:08.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.637+0000 7fbdbde00640 1 -- 172.21.15.159:0/513563827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbdc00d2310 msgr2=0x7fbdc00c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:08.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.637+0000 7fbdbde00640 1 --2- 172.21.15.159:0/513563827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbdc00d2310 0x7fbdc00c6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:08.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.637+0000 7fbdbde00640 1 -- 172.21.15.159:0/513563827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdc015e4a0 msgr2=0x7fbdc015fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:08.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.637+0000 7fbdbde00640 1 --2- 172.21.15.159:0/513563827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdc015e4a0 0x7fbdc015fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:08.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.637+0000 7fbdbde00640 1 -- 172.21.15.159:0/513563827 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc00c9ce0 con 0x7fbdc00cf530 2024-04-04T06:06:08.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.637+0000 7fbdbde00640 1 --2- 172.21.15.159:0/513563827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdc00cf530 0x7fbdc00c97a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000c3c0 tx=0x7fbdb000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:08.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.637+0000 7fbdb6a00640 1 -- 172.21.15.159:0/513563827 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0017070 con 0x7fbdc00cf530 2024-04-04T06:06:08.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.638+0000 7fbdb6a00640 1 -- 172.21.15.159:0/513563827 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdb0005ce0 con 0x7fbdc00cf530 2024-04-04T06:06:08.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.638+0000 7fbdb6a00640 1 -- 172.21.15.159:0/513563827 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb00072f0 con 0x7fbdc00cf530 2024-04-04T06:06:08.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.638+0000 7fbdbf200640 1 -- 172.21.15.159:0/513563827 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc00c6fd0 con 0x7fbdc00cf530 2024-04-04T06:06:08.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.638+0000 7fbdbf200640 1 -- 172.21.15.159:0/513563827 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc00c8cc0 con 0x7fbdc00cf530 2024-04-04T06:06:08.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.639+0000 7fbdbf200640 1 -- 172.21.15.159:0/513563827 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc0075940 con 0x7fbdc00cf530 2024-04-04T06:06:08.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.640+0000 7fbdb6a00640 1 -- 172.21.15.159:0/513563827 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbdb001a070 con 0x7fbdc00cf530 2024-04-04T06:06:08.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.640+0000 7fbdb6a00640 1 --2- 172.21.15.159:0/513563827 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd9c06c8f0 0x7fbd9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:08.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.641+0000 7fbdb6a00640 1 -- 172.21.15.159:0/513563827 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbdb008afa0 con 0x7fbdc00cf530 2024-04-04T06:06:08.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.641+0000 7fbdb5400640 1 --2- 172.21.15.159:0/513563827 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd9c06c8f0 0x7fbd9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:08.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.642+0000 7fbdb5400640 1 --2- 172.21.15.159:0/513563827 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd9c06c8f0 0x7fbd9c06edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbda8036000 tx=0x7fbda803d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:08.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.646+0000 7fbdb6a00640 1 -- 172.21.15.159:0/513563827 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbdb0059320 con 0x7fbdc00cf530 2024-04-04T06:06:08.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.828+0000 7fbdbf200640 1 -- 172.21.15.159:0/513563827 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbdc0143e60 con 0x7fbd9c06c8f0 2024-04-04T06:06:08.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.846+0000 7fbdb6a00640 1 -- 172.21.15.159:0/513563827 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7fbdc0143e60 con 0x7fbd9c06c8f0 2024-04-04T06:06:08.843 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:08.846 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:06:08.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.852+0000 7fbdbf200640 1 -- 172.21.15.159:0/513563827 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd9c06c8f0 msgr2=0x7fbd9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:08.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.852+0000 7fbdbf200640 1 --2- 172.21.15.159:0/513563827 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd9c06c8f0 0x7fbd9c06edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbda8036000 tx=0x7fbda803d040 comp rx=0 tx=0).stop 2024-04-04T06:06:08.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.852+0000 7fbdbf200640 1 -- 172.21.15.159:0/513563827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdc00cf530 msgr2=0x7fbdc00c97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:08.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.852+0000 7fbdbf200640 1 --2- 172.21.15.159:0/513563827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdc00cf530 0x7fbdc00c97a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000c3c0 tx=0x7fbdb000c880 comp rx=0 tx=0).stop 2024-04-04T06:06:08.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.852+0000 7fbdbf200640 1 -- 172.21.15.159:0/513563827 shutdown_connections 2024-04-04T06:06:08.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.852+0000 7fbdbf200640 1 --2- 172.21.15.159:0/513563827 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd9c06c8f0 0x7fbd9c06edb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:08.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.852+0000 7fbdbf200640 1 --2- 172.21.15.159:0/513563827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbdc015e4a0 0x7fbdc015fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:08.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.852+0000 7fbdbf200640 1 --2- 172.21.15.159:0/513563827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbdc00d2310 0x7fbdc00c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:08.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.852+0000 7fbdbf200640 1 --2- 172.21.15.159:0/513563827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdc00cf530 0x7fbdc00c97a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:08.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.852+0000 7fbdbf200640 1 -- 172.21.15.159:0/513563827 >> 172.21.15.159:0/513563827 conn(0x7fbdc00676d0 msgr2=0x7fbdc00b4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:08.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.853+0000 7fbdbf200640 1 -- 172.21.15.159:0/513563827 shutdown_connections 2024-04-04T06:06:08.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:08.853+0000 7fbdbf200640 1 -- 172.21.15.159:0/513563827 wait complete. 2024-04-04T06:06:08.867 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-04-04T06:06:08.716458+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235672,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739048,"statfs":{"total":773094113280,"available":772852785152,"internally_reserved":0,"allocated":2924544,"data_stored":1765420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004759"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594198+0000","last_change":"2024-04-04T06:05:53.366079+0000","last_active":"2024-04-04T06:05:54.594198+0000","last_peered":"2024-04-04T06:05:54.594198+0000","last_clean":"2024-04-04T06:05:54.594198+0000","last_became_active":"2024-04-04T06:05:53.365810+0000","last_became_peered":"2024-04-04T06:05:53.365810+0000","last_unstale":"2024-04-04T06:05:54.594198+0000","last_undegraded":"2024-04-04T06:05:54.594198+0000","last_fullsized":"2024-04-04T06:05:54.594198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:46:01.152077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362281+0000","last_change":"2024-04-04T06:05:53.376777+0000","last_active":"2024-04-04T06:05:54.362281+0000","last_peered":"2024-04-04T06:05:54.362281+0000","last_clean":"2024-04-04T06:05:54.362281+0000","last_became_active":"2024-04-04T06:05:53.374947+0000","last_became_peered":"2024-04-04T06:05:53.374947+0000","last_unstale":"2024-04-04T06:05:54.362281+0000","last_undegraded":"2024-04-04T06:05:54.362281+0000","last_fullsized":"2024-04-04T06:05:54.362281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:34.766930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591301+0000","last_change":"2024-04-04T06:05:53.362328+0000","last_active":"2024-04-04T06:05:54.591301+0000","last_peered":"2024-04-04T06:05:54.591301+0000","last_clean":"2024-04-04T06:05:54.591301+0000","last_became_active":"2024-04-04T06:05:53.359587+0000","last_became_peered":"2024-04-04T06:05:53.359587+0000","last_unstale":"2024-04-04T06:05:54.591301+0000","last_undegraded":"2024-04-04T06:05:54.591301+0000","last_fullsized":"2024-04-04T06:05:54.591301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:10.411661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365717+0000","last_change":"2024-04-04T06:05:53.373115+0000","last_active":"2024-04-04T06:05:54.365717+0000","last_peered":"2024-04-04T06:05:54.365717+0000","last_clean":"2024-04-04T06:05:54.365717+0000","last_became_active":"2024-04-04T06:05:53.372848+0000","last_became_peered":"2024-04-04T06:05:53.372848+0000","last_unstale":"2024-04-04T06:05:54.365717+0000","last_undegraded":"2024-04-04T06:05:54.365717+0000","last_fullsized":"2024-04-04T06:05:54.365717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:45:21.548886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593377+0000","last_change":"2024-04-04T06:05:53.370721+0000","last_active":"2024-04-04T06:05:54.593377+0000","last_peered":"2024-04-04T06:05:54.593377+0000","last_clean":"2024-04-04T06:05:54.593377+0000","last_became_active":"2024-04-04T06:05:53.370183+0000","last_became_peered":"2024-04-04T06:05:53.370183+0000","last_unstale":"2024-04-04T06:05:54.593377+0000","last_undegraded":"2024-04-04T06:05:54.593377+0000","last_fullsized":"2024-04-04T06:05:54.593377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:30:56.836366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367929+0000","last_change":"2024-04-04T06:05:53.387263+0000","last_active":"2024-04-04T06:05:54.367929+0000","last_peered":"2024-04-04T06:05:54.367929+0000","last_clean":"2024-04-04T06:05:54.367929+0000","last_became_active":"2024-04-04T06:05:53.387012+0000","last_became_peered":"2024-04-04T06:05:53.387012+0000","last_unstale":"2024-04-04T06:05:54.367929+0000","last_undegraded":"2024-04-04T06:05:54.367929+0000","last_fullsized":"2024-04-04T06:05:54.367929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:13.472940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592309+0000","last_change":"2024-04-04T06:05:53.370203+0000","last_active":"2024-04-04T06:05:54.592309+0000","last_peered":"2024-04-04T06:05:54.592309+0000","last_clean":"2024-04-04T06:05:54.592309+0000","last_became_active":"2024-04-04T06:05:53.369800+0000","last_became_peered":"2024-04-04T06:05:53.369800+0000","last_unstale":"2024-04-04T06:05:54.592309+0000","last_undegraded":"2024-04-04T06:05:54.592309+0000","last_fullsized":"2024-04-04T06:05:54.592309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:39:30.391173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592465+0000","last_change":"2024-04-04T06:05:53.369028+0000","last_active":"2024-04-04T06:05:54.592465+0000","last_peered":"2024-04-04T06:05:54.592465+0000","last_clean":"2024-04-04T06:05:54.592465+0000","last_became_active":"2024-04-04T06:05:53.365998+0000","last_became_peered":"2024-04-04T06:05:53.365998+0000","last_unstale":"2024-04-04T06:05:54.592465+0000","last_undegraded":"2024-04-04T06:05:54.592465+0000","last_fullsized":"2024-04-04T06:05:54.592465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:34.766930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594227+0000","last_change":"2024-04-04T06:05:53.369886+0000","last_active":"2024-04-04T06:05:54.594227+0000","last_peered":"2024-04-04T06:05:54.594227+0000","last_clean":"2024-04-04T06:05:54.594227+0000","last_became_active":"2024-04-04T06:05:53.368815+0000","last_became_peered":"2024-04-04T06:05:53.368815+0000","last_unstale":"2024-04-04T06:05:54.594227+0000","last_undegraded":"2024-04-04T06:05:54.594227+0000","last_fullsized":"2024-04-04T06:05:54.594227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:37:59.044530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592641+0000","last_change":"2024-04-04T06:05:53.366948+0000","last_active":"2024-04-04T06:05:54.592641+0000","last_peered":"2024-04-04T06:05:54.592641+0000","last_clean":"2024-04-04T06:05:54.592641+0000","last_became_active":"2024-04-04T06:05:53.366550+0000","last_became_peered":"2024-04-04T06:05:53.366550+0000","last_unstale":"2024-04-04T06:05:54.592641+0000","last_undegraded":"2024-04-04T06:05:54.592641+0000","last_fullsized":"2024-04-04T06:05:54.592641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:47:43.615957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592078+0000","last_change":"2024-04-04T06:05:53.376979+0000","last_active":"2024-04-04T06:05:54.592078+0000","last_peered":"2024-04-04T06:05:54.592078+0000","last_clean":"2024-04-04T06:05:54.592078+0000","last_became_active":"2024-04-04T06:05:53.376622+0000","last_became_peered":"2024-04-04T06:05:53.376622+0000","last_unstale":"2024-04-04T06:05:54.592078+0000","last_undegraded":"2024-04-04T06:05:54.592078+0000","last_fullsized":"2024-04-04T06:05:54.592078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:42:00.014108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367238+0000","last_change":"2024-04-04T06:05:53.364063+0000","last_active":"2024-04-04T06:05:54.367238+0000","last_peered":"2024-04-04T06:05:54.367238+0000","last_clean":"2024-04-04T06:05:54.367238+0000","last_became_active":"2024-04-04T06:05:53.363715+0000","last_became_peered":"2024-04-04T06:05:53.363715+0000","last_unstale":"2024-04-04T06:05:54.367238+0000","last_undegraded":"2024-04-04T06:05:54.367238+0000","last_fullsized":"2024-04-04T06:05:54.367238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:21.889411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593467+0000","last_change":"2024-04-04T06:05:53.362035+0000","last_active":"2024-04-04T06:05:54.593467+0000","last_peered":"2024-04-04T06:05:54.593467+0000","last_clean":"2024-04-04T06:05:54.593467+0000","last_became_active":"2024-04-04T06:05:53.361614+0000","last_became_peered":"2024-04-04T06:05:53.361614+0000","last_unstale":"2024-04-04T06:05:54.593467+0000","last_undegraded":"2024-04-04T06:05:54.593467+0000","last_fullsized":"2024-04-04T06:05:54.593467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:17:05.576402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590275+0000","last_change":"2024-04-04T06:05:53.355832+0000","last_active":"2024-04-04T06:05:54.590275+0000","last_peered":"2024-04-04T06:05:54.590275+0000","last_clean":"2024-04-04T06:05:54.590275+0000","last_became_active":"2024-04-04T06:05:53.355508+0000","last_became_peered":"2024-04-04T06:05:53.355508+0000","last_unstale":"2024-04-04T06:05:54.590275+0000","last_undegraded":"2024-04-04T06:05:54.590275+0000","last_fullsized":"2024-04-04T06:05:54.590275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:11:00.324524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362799+0000","last_change":"2024-04-04T06:05:53.353630+0000","last_active":"2024-04-04T06:05:54.362799+0000","last_peered":"2024-04-04T06:05:54.362799+0000","last_clean":"2024-04-04T06:05:54.362799+0000","last_became_active":"2024-04-04T06:05:53.353315+0000","last_became_peered":"2024-04-04T06:05:53.353315+0000","last_unstale":"2024-04-04T06:05:54.362799+0000","last_undegraded":"2024-04-04T06:05:54.362799+0000","last_fullsized":"2024-04-04T06:05:54.362799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.364261+0000","last_change":"2024-04-04T06:05:53.387499+0000","last_active":"2024-04-04T06:05:54.364261+0000","last_peered":"2024-04-04T06:05:54.364261+0000","last_clean":"2024-04-04T06:05:54.364261+0000","last_became_active":"2024-04-04T06:05:53.387187+0000","last_became_peered":"2024-04-04T06:05:53.387187+0000","last_unstale":"2024-04-04T06:05:54.364261+0000","last_undegraded":"2024-04-04T06:05:54.364261+0000","last_fullsized":"2024-04-04T06:05:54.364261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.366558+0000","last_change":"2024-04-04T06:05:53.378705+0000","last_active":"2024-04-04T06:05:54.366558+0000","last_peered":"2024-04-04T06:05:54.366558+0000","last_clean":"2024-04-04T06:05:54.366558+0000","last_became_active":"2024-04-04T06:05:53.378428+0000","last_became_peered":"2024-04-04T06:05:53.378428+0000","last_unstale":"2024-04-04T06:05:54.366558+0000","last_undegraded":"2024-04-04T06:05:54.366558+0000","last_fullsized":"2024-04-04T06:05:54.366558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:17:42.995360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591477+0000","last_change":"2024-04-04T06:05:53.372718+0000","last_active":"2024-04-04T06:05:54.591477+0000","last_peered":"2024-04-04T06:05:54.591477+0000","last_clean":"2024-04-04T06:05:54.591477+0000","last_became_active":"2024-04-04T06:05:53.372458+0000","last_became_peered":"2024-04-04T06:05:53.372458+0000","last_unstale":"2024-04-04T06:05:54.591477+0000","last_undegraded":"2024-04-04T06:05:54.591477+0000","last_fullsized":"2024-04-04T06:05:54.591477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:31.723861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594485+0000","last_change":"2024-04-04T06:05:53.360929+0000","last_active":"2024-04-04T06:05:54.594485+0000","last_peered":"2024-04-04T06:05:54.594485+0000","last_clean":"2024-04-04T06:05:54.594485+0000","last_became_active":"2024-04-04T06:05:53.360661+0000","last_became_peered":"2024-04-04T06:05:53.360661+0000","last_unstale":"2024-04-04T06:05:54.594485+0000","last_undegraded":"2024-04-04T06:05:54.594485+0000","last_fullsized":"2024-04-04T06:05:54.594485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:06:23.755362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591699+0000","last_change":"2024-04-04T06:05:53.374155+0000","last_active":"2024-04-04T06:05:54.591699+0000","last_peered":"2024-04-04T06:05:54.591699+0000","last_clean":"2024-04-04T06:05:54.591699+0000","last_became_active":"2024-04-04T06:05:53.373872+0000","last_became_peered":"2024-04-04T06:05:53.373872+0000","last_unstale":"2024-04-04T06:05:54.591699+0000","last_undegraded":"2024-04-04T06:05:54.591699+0000","last_fullsized":"2024-04-04T06:05:54.591699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:19:20.840644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590730+0000","last_change":"2024-04-04T06:05:53.375944+0000","last_active":"2024-04-04T06:05:54.590730+0000","last_peered":"2024-04-04T06:05:54.590730+0000","last_clean":"2024-04-04T06:05:54.590730+0000","last_became_active":"2024-04-04T06:05:53.375551+0000","last_became_peered":"2024-04-04T06:05:53.375551+0000","last_unstale":"2024-04-04T06:05:54.590730+0000","last_undegraded":"2024-04-04T06:05:54.590730+0000","last_fullsized":"2024-04-04T06:05:54.590730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:33:17.187978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367881+0000","last_change":"2024-04-04T06:05:53.377200+0000","last_active":"2024-04-04T06:05:54.367881+0000","last_peered":"2024-04-04T06:05:54.367881+0000","last_clean":"2024-04-04T06:05:54.367881+0000","last_became_active":"2024-04-04T06:05:53.373063+0000","last_became_peered":"2024-04-04T06:05:53.373063+0000","last_unstale":"2024-04-04T06:05:54.367881+0000","last_undegraded":"2024-04-04T06:05:54.367881+0000","last_fullsized":"2024-04-04T06:05:54.367881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:28:48.569552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592389+0000","last_change":"2024-04-04T06:05:53.358659+0000","last_active":"2024-04-04T06:05:54.592389+0000","last_peered":"2024-04-04T06:05:54.592389+0000","last_clean":"2024-04-04T06:05:54.592389+0000","last_became_active":"2024-04-04T06:05:53.358372+0000","last_became_peered":"2024-04-04T06:05:53.358372+0000","last_unstale":"2024-04-04T06:05:54.592389+0000","last_undegraded":"2024-04-04T06:05:54.592389+0000","last_fullsized":"2024-04-04T06:05:54.592389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:14:41.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365536+0000","last_change":"2024-04-04T06:05:53.381591+0000","last_active":"2024-04-04T06:05:54.365536+0000","last_peered":"2024-04-04T06:05:54.365536+0000","last_clean":"2024-04-04T06:05:54.365536+0000","last_became_active":"2024-04-04T06:05:53.377261+0000","last_became_peered":"2024-04-04T06:05:53.377261+0000","last_unstale":"2024-04-04T06:05:54.365536+0000","last_undegraded":"2024-04-04T06:05:54.365536+0000","last_fullsized":"2024-04-04T06:05:54.365536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:07:53.196090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593367+0000","last_change":"2024-04-04T06:05:53.371961+0000","last_active":"2024-04-04T06:05:54.593367+0000","last_peered":"2024-04-04T06:05:54.593367+0000","last_clean":"2024-04-04T06:05:54.593367+0000","last_became_active":"2024-04-04T06:05:53.371702+0000","last_became_peered":"2024-04-04T06:05:53.371702+0000","last_unstale":"2024-04-04T06:05:54.593367+0000","last_undegraded":"2024-04-04T06:05:54.593367+0000","last_fullsized":"2024-04-04T06:05:54.593367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592985+0000","last_change":"2024-04-04T06:05:53.383384+0000","last_active":"2024-04-04T06:05:54.592985+0000","last_peered":"2024-04-04T06:05:54.592985+0000","last_clean":"2024-04-04T06:05:54.592985+0000","last_became_active":"2024-04-04T06:05:53.382809+0000","last_became_peered":"2024-04-04T06:05:53.382809+0000","last_unstale":"2024-04-04T06:05:54.592985+0000","last_undegraded":"2024-04-04T06:05:54.592985+0000","last_fullsized":"2024-04-04T06:05:54.592985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:15:27.732386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362466+0000","last_change":"2024-04-04T06:05:53.373499+0000","last_active":"2024-04-04T06:05:54.362466+0000","last_peered":"2024-04-04T06:05:54.362466+0000","last_clean":"2024-04-04T06:05:54.362466+0000","last_became_active":"2024-04-04T06:05:53.367324+0000","last_became_peered":"2024-04-04T06:05:53.367324+0000","last_unstale":"2024-04-04T06:05:54.362466+0000","last_undegraded":"2024-04-04T06:05:54.362466+0000","last_fullsized":"2024-04-04T06:05:54.362466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:21:23.535889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593436+0000","last_change":"2024-04-04T06:05:53.371688+0000","last_active":"2024-04-04T06:05:54.593436+0000","last_peered":"2024-04-04T06:05:54.593436+0000","last_clean":"2024-04-04T06:05:54.593436+0000","last_became_active":"2024-04-04T06:05:53.371350+0000","last_became_peered":"2024-04-04T06:05:53.371350+0000","last_unstale":"2024-04-04T06:05:54.593436+0000","last_undegraded":"2024-04-04T06:05:54.593436+0000","last_fullsized":"2024-04-04T06:05:54.593436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:07:53.196090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590942+0000","last_change":"2024-04-04T06:05:53.370489+0000","last_active":"2024-04-04T06:05:54.590942+0000","last_peered":"2024-04-04T06:05:54.590942+0000","last_clean":"2024-04-04T06:05:54.590942+0000","last_became_active":"2024-04-04T06:05:53.367223+0000","last_became_peered":"2024-04-04T06:05:53.367223+0000","last_unstale":"2024-04-04T06:05:54.590942+0000","last_undegraded":"2024-04-04T06:05:54.590942+0000","last_fullsized":"2024-04-04T06:05:54.590942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:28:13.818701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368055+0000","last_change":"2024-04-04T06:05:53.364565+0000","last_active":"2024-04-04T06:05:54.368055+0000","last_peered":"2024-04-04T06:05:54.368055+0000","last_clean":"2024-04-04T06:05:54.368055+0000","last_became_active":"2024-04-04T06:05:53.364292+0000","last_became_peered":"2024-04-04T06:05:53.364292+0000","last_unstale":"2024-04-04T06:05:54.368055+0000","last_undegraded":"2024-04-04T06:05:54.368055+0000","last_fullsized":"2024-04-04T06:05:54.368055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:37:54.850791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590194+0000","last_change":"2024-04-04T06:05:53.366686+0000","last_active":"2024-04-04T06:05:54.590194+0000","last_peered":"2024-04-04T06:05:54.590194+0000","last_clean":"2024-04-04T06:05:54.590194+0000","last_became_active":"2024-04-04T06:05:53.366409+0000","last_became_peered":"2024-04-04T06:05:53.366409+0000","last_unstale":"2024-04-04T06:05:54.590194+0000","last_undegraded":"2024-04-04T06:05:54.590194+0000","last_fullsized":"2024-04-04T06:05:54.590194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:21.889411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591239+0000","last_change":"2024-04-04T06:05:53.361681+0000","last_active":"2024-04-04T06:05:54.591239+0000","last_peered":"2024-04-04T06:05:54.591239+0000","last_clean":"2024-04-04T06:05:54.591239+0000","last_became_active":"2024-04-04T06:05:53.361307+0000","last_became_peered":"2024-04-04T06:05:53.361307+0000","last_unstale":"2024-04-04T06:05:54.591239+0000","last_undegraded":"2024-04-04T06:05:54.591239+0000","last_fullsized":"2024-04-04T06:05:54.591239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:16:20.634573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.364842+0000","last_change":"2024-04-04T06:05:53.361164+0000","last_active":"2024-04-04T06:05:54.364842+0000","last_peered":"2024-04-04T06:05:54.364842+0000","last_clean":"2024-04-04T06:05:54.364842+0000","last_became_active":"2024-04-04T06:05:53.357301+0000","last_became_peered":"2024-04-04T06:05:53.357301+0000","last_unstale":"2024-04-04T06:05:54.364842+0000","last_undegraded":"2024-04-04T06:05:54.364842+0000","last_fullsized":"2024-04-04T06:05:54.364842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:56:10.393146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594020+0000","last_change":"2024-04-04T06:05:52.368795+0000","last_active":"2024-04-04T06:05:54.594020+0000","last_peered":"2024-04-04T06:05:54.594020+0000","last_clean":"2024-04-04T06:05:54.594020+0000","last_became_active":"2024-04-04T06:05:52.368526+0000","last_became_peered":"2024-04-04T06:05:52.368526+0000","last_unstale":"2024-04-04T06:05:54.594020+0000","last_undegraded":"2024-04-04T06:05:54.594020+0000","last_fullsized":"2024-04-04T06:05:54.594020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:45:43.480308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591181+0000","last_change":"2024-04-04T06:05:53.377207+0000","last_active":"2024-04-04T06:05:54.591181+0000","last_peered":"2024-04-04T06:05:54.591181+0000","last_clean":"2024-04-04T06:05:54.591181+0000","last_became_active":"2024-04-04T06:05:53.376853+0000","last_became_peered":"2024-04-04T06:05:53.376853+0000","last_unstale":"2024-04-04T06:05:54.591181+0000","last_undegraded":"2024-04-04T06:05:54.591181+0000","last_fullsized":"2024-04-04T06:05:54.591181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:21:23.535889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591256+0000","last_change":"2024-04-04T06:05:52.372922+0000","last_active":"2024-04-04T06:05:54.591256+0000","last_peered":"2024-04-04T06:05:54.591256+0000","last_clean":"2024-04-04T06:05:54.591256+0000","last_became_active":"2024-04-04T06:05:52.372615+0000","last_became_peered":"2024-04-04T06:05:52.372615+0000","last_unstale":"2024-04-04T06:05:54.591256+0000","last_undegraded":"2024-04-04T06:05:54.591256+0000","last_fullsized":"2024-04-04T06:05:54.591256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:36:38.928830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592633+0000","last_change":"2024-04-04T06:05:52.371943+0000","last_active":"2024-04-04T06:05:54.592633+0000","last_peered":"2024-04-04T06:05:54.592633+0000","last_clean":"2024-04-04T06:05:54.592633+0000","last_became_active":"2024-04-04T06:05:52.371686+0000","last_became_peered":"2024-04-04T06:05:52.371686+0000","last_unstale":"2024-04-04T06:05:54.592633+0000","last_undegraded":"2024-04-04T06:05:54.592633+0000","last_fullsized":"2024-04-04T06:05:54.592633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:52:05.788594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592504+0000","last_change":"2024-04-04T06:05:53.363121+0000","last_active":"2024-04-04T06:05:54.592504+0000","last_peered":"2024-04-04T06:05:54.592504+0000","last_clean":"2024-04-04T06:05:54.592504+0000","last_became_active":"2024-04-04T06:05:53.362861+0000","last_became_peered":"2024-04-04T06:05:53.362861+0000","last_unstale":"2024-04-04T06:05:54.592504+0000","last_undegraded":"2024-04-04T06:05:54.592504+0000","last_fullsized":"2024-04-04T06:05:54.592504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:28:13.818701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365617+0000","last_change":"2024-04-04T06:05:52.371090+0000","last_active":"2024-04-04T06:05:54.365617+0000","last_peered":"2024-04-04T06:05:54.365617+0000","last_clean":"2024-04-04T06:05:54.365617+0000","last_became_active":"2024-04-04T06:05:52.369650+0000","last_became_peered":"2024-04-04T06:05:52.369650+0000","last_unstale":"2024-04-04T06:05:54.365617+0000","last_undegraded":"2024-04-04T06:05:54.365617+0000","last_fullsized":"2024-04-04T06:05:54.365617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:45:43.480308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362432+0000","last_change":"2024-04-04T06:05:53.357537+0000","last_active":"2024-04-04T06:05:54.362432+0000","last_peered":"2024-04-04T06:05:54.362432+0000","last_clean":"2024-04-04T06:05:54.362432+0000","last_became_active":"2024-04-04T06:05:53.357284+0000","last_became_peered":"2024-04-04T06:05:53.357284+0000","last_unstale":"2024-04-04T06:05:54.362432+0000","last_undegraded":"2024-04-04T06:05:54.362432+0000","last_fullsized":"2024-04-04T06:05:54.362432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:28:13.818701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592398+0000","last_change":"2024-04-04T06:05:52.374921+0000","last_active":"2024-04-04T06:05:54.592398+0000","last_peered":"2024-04-04T06:05:54.592398+0000","last_clean":"2024-04-04T06:05:54.592398+0000","last_became_active":"2024-04-04T06:05:52.374555+0000","last_became_peered":"2024-04-04T06:05:52.374555+0000","last_unstale":"2024-04-04T06:05:54.592398+0000","last_undegraded":"2024-04-04T06:05:54.592398+0000","last_fullsized":"2024-04-04T06:05:54.592398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:05.925335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368597+0000","last_change":"2024-04-04T06:05:53.377612+0000","last_active":"2024-04-04T06:05:54.368597+0000","last_peered":"2024-04-04T06:05:54.368597+0000","last_clean":"2024-04-04T06:05:54.368597+0000","last_became_active":"2024-04-04T06:05:53.374258+0000","last_became_peered":"2024-04-04T06:05:53.374258+0000","last_unstale":"2024-04-04T06:05:54.368597+0000","last_undegraded":"2024-04-04T06:05:54.368597+0000","last_fullsized":"2024-04-04T06:05:54.368597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:08:44.769578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368664+0000","last_change":"2024-04-04T06:05:52.373979+0000","last_active":"2024-04-04T06:05:54.368664+0000","last_peered":"2024-04-04T06:05:54.368664+0000","last_clean":"2024-04-04T06:05:54.368664+0000","last_became_active":"2024-04-04T06:05:52.373408+0000","last_became_peered":"2024-04-04T06:05:52.373408+0000","last_unstale":"2024-04-04T06:05:54.368664+0000","last_undegraded":"2024-04-04T06:05:54.368664+0000","last_fullsized":"2024-04-04T06:05:54.368664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:56:09.383441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591338+0000","last_change":"2024-04-04T06:05:53.368728+0000","last_active":"2024-04-04T06:05:54.591338+0000","last_peered":"2024-04-04T06:05:54.591338+0000","last_clean":"2024-04-04T06:05:54.591338+0000","last_became_active":"2024-04-04T06:05:53.367971+0000","last_became_peered":"2024-04-04T06:05:53.367971+0000","last_unstale":"2024-04-04T06:05:54.591338+0000","last_undegraded":"2024-04-04T06:05:54.591338+0000","last_fullsized":"2024-04-04T06:05:54.591338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:52.720034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362602+0000","last_change":"2024-04-04T06:05:52.368574+0000","last_active":"2024-04-04T06:05:54.362602+0000","last_peered":"2024-04-04T06:05:54.362602+0000","last_clean":"2024-04-04T06:05:54.362602+0000","last_became_active":"2024-04-04T06:05:52.367460+0000","last_became_peered":"2024-04-04T06:05:52.367460+0000","last_unstale":"2024-04-04T06:05:54.362602+0000","last_undegraded":"2024-04-04T06:05:54.362602+0000","last_fullsized":"2024-04-04T06:05:54.362602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:56:09.383441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367547+0000","last_change":"2024-04-04T06:05:53.379475+0000","last_active":"2024-04-04T06:05:54.367547+0000","last_peered":"2024-04-04T06:05:54.367547+0000","last_clean":"2024-04-04T06:05:54.367547+0000","last_became_active":"2024-04-04T06:05:53.376075+0000","last_became_peered":"2024-04-04T06:05:53.376075+0000","last_unstale":"2024-04-04T06:05:54.367547+0000","last_undegraded":"2024-04-04T06:05:54.367547+0000","last_fullsized":"2024-04-04T06:05:54.367547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:03:00.104400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591818+0000","last_change":"2024-04-04T06:05:52.370133+0000","last_active":"2024-04-04T06:05:54.591818+0000","last_peered":"2024-04-04T06:05:54.591818+0000","last_clean":"2024-04-04T06:05:54.591818+0000","last_became_active":"2024-04-04T06:05:52.369825+0000","last_became_peered":"2024-04-04T06:05:52.369825+0000","last_unstale":"2024-04-04T06:05:54.591818+0000","last_undegraded":"2024-04-04T06:05:54.591818+0000","last_fullsized":"2024-04-04T06:05:54.591818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:01:02.629926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594503+0000","last_change":"2024-04-04T06:05:53.371020+0000","last_active":"2024-04-04T06:05:54.594503+0000","last_peered":"2024-04-04T06:05:54.594503+0000","last_clean":"2024-04-04T06:05:54.594503+0000","last_became_active":"2024-04-04T06:05:53.370755+0000","last_became_peered":"2024-04-04T06:05:53.370755+0000","last_unstale":"2024-04-04T06:05:54.594503+0000","last_undegraded":"2024-04-04T06:05:54.594503+0000","last_fullsized":"2024-04-04T06:05:54.594503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:54:36.353079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591316+0000","last_change":"2024-04-04T06:05:52.367020+0000","last_active":"2024-04-04T06:05:54.591316+0000","last_peered":"2024-04-04T06:05:54.591316+0000","last_clean":"2024-04-04T06:05:54.591316+0000","last_became_active":"2024-04-04T06:05:52.366654+0000","last_became_peered":"2024-04-04T06:05:52.366654+0000","last_unstale":"2024-04-04T06:05:54.591316+0000","last_undegraded":"2024-04-04T06:05:54.591316+0000","last_fullsized":"2024-04-04T06:05:54.591316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:26:03.163432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590433+0000","last_change":"2024-04-04T06:05:53.373111+0000","last_active":"2024-04-04T06:05:54.590433+0000","last_peered":"2024-04-04T06:05:54.590433+0000","last_clean":"2024-04-04T06:05:54.590433+0000","last_became_active":"2024-04-04T06:05:53.372761+0000","last_became_peered":"2024-04-04T06:05:53.372761+0000","last_unstale":"2024-04-04T06:05:54.590433+0000","last_undegraded":"2024-04-04T06:05:54.590433+0000","last_fullsized":"2024-04-04T06:05:54.590433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:27:06.279930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368191+0000","last_change":"2024-04-04T06:05:52.365334+0000","last_active":"2024-04-04T06:05:54.368191+0000","last_peered":"2024-04-04T06:05:54.368191+0000","last_clean":"2024-04-04T06:05:54.368191+0000","last_became_active":"2024-04-04T06:05:52.363154+0000","last_became_peered":"2024-04-04T06:05:52.363154+0000","last_unstale":"2024-04-04T06:05:54.368191+0000","last_undegraded":"2024-04-04T06:05:54.368191+0000","last_fullsized":"2024-04-04T06:05:54.368191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:53:42.904072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593559+0000","last_change":"2024-04-04T06:05:53.380196+0000","last_active":"2024-04-04T06:05:54.593559+0000","last_peered":"2024-04-04T06:05:54.593559+0000","last_clean":"2024-04-04T06:05:54.593559+0000","last_became_active":"2024-04-04T06:05:53.379800+0000","last_became_peered":"2024-04-04T06:05:53.379800+0000","last_unstale":"2024-04-04T06:05:54.593559+0000","last_undegraded":"2024-04-04T06:05:54.593559+0000","last_fullsized":"2024-04-04T06:05:54.593559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:41:09.510239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591089+0000","last_change":"2024-04-04T06:05:52.377332+0000","last_active":"2024-04-04T06:05:54.591089+0000","last_peered":"2024-04-04T06:05:54.591089+0000","last_clean":"2024-04-04T06:05:54.591089+0000","last_became_active":"2024-04-04T06:05:52.377002+0000","last_became_peered":"2024-04-04T06:05:52.377002+0000","last_unstale":"2024-04-04T06:05:54.591089+0000","last_undegraded":"2024-04-04T06:05:54.591089+0000","last_fullsized":"2024-04-04T06:05:54.591089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:05:57.500385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591711+0000","last_change":"2024-04-04T06:05:53.378591+0000","last_active":"2024-04-04T06:05:54.591711+0000","last_peered":"2024-04-04T06:05:54.591711+0000","last_clean":"2024-04-04T06:05:54.591711+0000","last_became_active":"2024-04-04T06:05:53.378227+0000","last_became_peered":"2024-04-04T06:05:53.378227+0000","last_unstale":"2024-04-04T06:05:54.591711+0000","last_undegraded":"2024-04-04T06:05:54.591711+0000","last_fullsized":"2024-04-04T06:05:54.591711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:42:54.347331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.589595+0000","last_change":"2024-04-04T06:05:52.357745+0000","last_active":"2024-04-04T06:05:54.589595+0000","last_peered":"2024-04-04T06:05:54.589595+0000","last_clean":"2024-04-04T06:05:54.589595+0000","last_became_active":"2024-04-04T06:05:52.357280+0000","last_became_peered":"2024-04-04T06:05:52.357280+0000","last_unstale":"2024-04-04T06:05:54.589595+0000","last_undegraded":"2024-04-04T06:05:54.589595+0000","last_fullsized":"2024-04-04T06:05:54.589595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:45:43.480308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592420+0000","last_change":"2024-04-04T06:05:53.387060+0000","last_active":"2024-04-04T06:05:54.592420+0000","last_peered":"2024-04-04T06:05:54.592420+0000","last_clean":"2024-04-04T06:05:54.592420+0000","last_became_active":"2024-04-04T06:05:53.386804+0000","last_became_peered":"2024-04-04T06:05:53.386804+0000","last_unstale":"2024-04-04T06:05:54.592420+0000","last_undegraded":"2024-04-04T06:05:54.592420+0000","last_fullsized":"2024-04-04T06:05:54.592420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:37:36.681617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367992+0000","last_change":"2024-04-04T06:05:52.362010+0000","last_active":"2024-04-04T06:05:54.367992+0000","last_peered":"2024-04-04T06:05:54.367992+0000","last_clean":"2024-04-04T06:05:54.367992+0000","last_became_active":"2024-04-04T06:05:52.361424+0000","last_became_peered":"2024-04-04T06:05:52.361424+0000","last_unstale":"2024-04-04T06:05:54.367992+0000","last_undegraded":"2024-04-04T06:05:54.367992+0000","last_fullsized":"2024-04-04T06:05:54.367992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:26:03.163432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594202+0000","last_change":"2024-04-04T06:05:53.375944+0000","last_active":"2024-04-04T06:05:54.594202+0000","last_peered":"2024-04-04T06:05:54.594202+0000","last_clean":"2024-04-04T06:05:54.594202+0000","last_became_active":"2024-04-04T06:05:53.375684+0000","last_became_peered":"2024-04-04T06:05:53.375684+0000","last_unstale":"2024-04-04T06:05:54.594202+0000","last_undegraded":"2024-04-04T06:05:54.594202+0000","last_fullsized":"2024-04-04T06:05:54.594202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:02:57.074580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591274+0000","last_change":"2024-04-04T06:05:52.358735+0000","last_active":"2024-04-04T06:05:54.591274+0000","last_peered":"2024-04-04T06:05:54.591274+0000","last_clean":"2024-04-04T06:05:54.591274+0000","last_became_active":"2024-04-04T06:05:52.358203+0000","last_became_peered":"2024-04-04T06:05:52.358203+0000","last_unstale":"2024-04-04T06:05:54.591274+0000","last_undegraded":"2024-04-04T06:05:54.591274+0000","last_fullsized":"2024-04-04T06:05:54.591274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:53:42.904072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594617+0000","last_change":"2024-04-04T06:05:53.388782+0000","last_active":"2024-04-04T06:05:54.594617+0000","last_peered":"2024-04-04T06:05:54.594617+0000","last_clean":"2024-04-04T06:05:54.594617+0000","last_became_active":"2024-04-04T06:05:53.388230+0000","last_became_peered":"2024-04-04T06:05:53.388230+0000","last_unstale":"2024-04-04T06:05:54.594617+0000","last_undegraded":"2024-04-04T06:05:54.594617+0000","last_fullsized":"2024-04-04T06:05:54.594617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:31:19.282600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362887+0000","last_change":"2024-04-04T06:05:52.366026+0000","last_active":"2024-04-04T06:05:54.362887+0000","last_peered":"2024-04-04T06:05:54.362887+0000","last_clean":"2024-04-04T06:05:54.362887+0000","last_became_active":"2024-04-04T06:05:52.365663+0000","last_became_peered":"2024-04-04T06:05:52.365663+0000","last_unstale":"2024-04-04T06:05:54.362887+0000","last_undegraded":"2024-04-04T06:05:54.362887+0000","last_fullsized":"2024-04-04T06:05:54.362887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:30.714156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.366141+0000","last_change":"2024-04-04T06:05:53.381556+0000","last_active":"2024-04-04T06:05:54.366141+0000","last_peered":"2024-04-04T06:05:54.366141+0000","last_clean":"2024-04-04T06:05:54.366141+0000","last_became_active":"2024-04-04T06:05:53.373492+0000","last_became_peered":"2024-04-04T06:05:53.373492+0000","last_unstale":"2024-04-04T06:05:54.366141+0000","last_undegraded":"2024-04-04T06:05:54.366141+0000","last_fullsized":"2024-04-04T06:05:54.366141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:53:19.419468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590914+0000","last_change":"2024-04-04T06:05:52.353584+0000","last_active":"2024-04-04T06:05:54.590914+0000","last_peered":"2024-04-04T06:05:54.590914+0000","last_clean":"2024-04-04T06:05:54.590914+0000","last_became_active":"2024-04-04T06:05:52.353017+0000","last_became_peered":"2024-04-04T06:05:52.353017+0000","last_unstale":"2024-04-04T06:05:54.590914+0000","last_undegraded":"2024-04-04T06:05:54.590914+0000","last_fullsized":"2024-04-04T06:05:54.590914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:58:40.874784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593698+0000","last_change":"2024-04-04T06:05:53.368622+0000","last_active":"2024-04-04T06:05:54.593698+0000","last_peered":"2024-04-04T06:05:54.593698+0000","last_clean":"2024-04-04T06:05:54.593698+0000","last_became_active":"2024-04-04T06:05:53.368311+0000","last_became_peered":"2024-04-04T06:05:53.368311+0000","last_unstale":"2024-04-04T06:05:54.593698+0000","last_undegraded":"2024-04-04T06:05:54.593698+0000","last_fullsized":"2024-04-04T06:05:54.593698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:01:24.602038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591729+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:05:54.591729+0000","last_peered":"2024-04-04T06:05:54.591729+0000","last_clean":"2024-04-04T06:05:54.591729+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:05:54.591729+0000","last_undegraded":"2024-04-04T06:05:54.591729+0000","last_fullsized":"2024-04-04T06:05:54.591729+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365145+0000","last_change":"2024-04-04T06:05:52.356730+0000","last_active":"2024-04-04T06:05:54.365145+0000","last_peered":"2024-04-04T06:05:54.365145+0000","last_clean":"2024-04-04T06:05:54.365145+0000","last_became_active":"2024-04-04T06:05:52.352963+0000","last_became_peered":"2024-04-04T06:05:52.352963+0000","last_unstale":"2024-04-04T06:05:54.365145+0000","last_undegraded":"2024-04-04T06:05:54.365145+0000","last_fullsized":"2024-04-04T06:05:54.365145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:05:06.878796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592869+0000","last_change":"2024-04-04T06:05:53.366506+0000","last_active":"2024-04-04T06:05:54.592869+0000","last_peered":"2024-04-04T06:05:54.592869+0000","last_clean":"2024-04-04T06:05:54.592869+0000","last_became_active":"2024-04-04T06:05:53.365677+0000","last_became_peered":"2024-04-04T06:05:53.365677+0000","last_unstale":"2024-04-04T06:05:54.592869+0000","last_undegraded":"2024-04-04T06:05:54.592869+0000","last_fullsized":"2024-04-04T06:05:54.592869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:10.411661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591456+0000","last_change":"2024-04-04T06:05:52.374798+0000","last_active":"2024-04-04T06:05:54.591456+0000","last_peered":"2024-04-04T06:05:54.591456+0000","last_clean":"2024-04-04T06:05:54.591456+0000","last_became_active":"2024-04-04T06:05:52.373838+0000","last_became_peered":"2024-04-04T06:05:52.373838+0000","last_unstale":"2024-04-04T06:05:54.591456+0000","last_undegraded":"2024-04-04T06:05:54.591456+0000","last_fullsized":"2024-04-04T06:05:54.591456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:51:50.301107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590751+0000","last_change":"2024-04-04T06:05:53.385393+0000","last_active":"2024-04-04T06:05:54.590751+0000","last_peered":"2024-04-04T06:05:54.590751+0000","last_clean":"2024-04-04T06:05:54.590751+0000","last_became_active":"2024-04-04T06:05:53.385062+0000","last_became_peered":"2024-04-04T06:05:53.385062+0000","last_unstale":"2024-04-04T06:05:54.590751+0000","last_undegraded":"2024-04-04T06:05:54.590751+0000","last_fullsized":"2024-04-04T06:05:54.590751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:14:41.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590309+0000","last_change":"2024-04-04T06:05:52.360018+0000","last_active":"2024-04-04T06:05:54.590309+0000","last_peered":"2024-04-04T06:05:54.590309+0000","last_clean":"2024-04-04T06:05:54.590309+0000","last_became_active":"2024-04-04T06:05:52.359735+0000","last_became_peered":"2024-04-04T06:05:52.359735+0000","last_unstale":"2024-04-04T06:05:54.590309+0000","last_undegraded":"2024-04-04T06:05:54.590309+0000","last_fullsized":"2024-04-04T06:05:54.590309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:05:06.878796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.363467+0000","last_change":"2024-04-04T06:05:53.356771+0000","last_active":"2024-04-04T06:05:54.363467+0000","last_peered":"2024-04-04T06:05:54.363467+0000","last_clean":"2024-04-04T06:05:54.363467+0000","last_became_active":"2024-04-04T06:05:53.356431+0000","last_became_peered":"2024-04-04T06:05:53.356431+0000","last_unstale":"2024-04-04T06:05:54.363467+0000","last_undegraded":"2024-04-04T06:05:54.363467+0000","last_fullsized":"2024-04-04T06:05:54.363467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:07:53.196090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.368163+0000","last_change":"2024-04-04T06:05:52.370387+0000","last_active":"2024-04-04T06:05:54.368163+0000","last_peered":"2024-04-04T06:05:54.368163+0000","last_clean":"2024-04-04T06:05:54.368163+0000","last_became_active":"2024-04-04T06:05:52.369905+0000","last_became_peered":"2024-04-04T06:05:52.369905+0000","last_unstale":"2024-04-04T06:05:54.368163+0000","last_undegraded":"2024-04-04T06:05:54.368163+0000","last_fullsized":"2024-04-04T06:05:54.368163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:58:40.874784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.367307+0000","last_change":"2024-04-04T06:05:53.371957+0000","last_active":"2024-04-04T06:05:54.367307+0000","last_peered":"2024-04-04T06:05:54.367307+0000","last_clean":"2024-04-04T06:05:54.367307+0000","last_became_active":"2024-04-04T06:05:53.371121+0000","last_became_peered":"2024-04-04T06:05:53.371121+0000","last_unstale":"2024-04-04T06:05:54.367307+0000","last_undegraded":"2024-04-04T06:05:54.367307+0000","last_fullsized":"2024-04-04T06:05:54.367307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:28:31.723861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362920+0000","last_change":"2024-04-04T06:05:52.361392+0000","last_active":"2024-04-04T06:05:54.362920+0000","last_peered":"2024-04-04T06:05:54.362920+0000","last_clean":"2024-04-04T06:05:54.362920+0000","last_became_active":"2024-04-04T06:05:52.360932+0000","last_became_peered":"2024-04-04T06:05:52.360932+0000","last_unstale":"2024-04-04T06:05:54.362920+0000","last_undegraded":"2024-04-04T06:05:54.362920+0000","last_fullsized":"2024-04-04T06:05:54.362920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:20.879706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365934+0000","last_change":"2024-04-04T06:05:53.379438+0000","last_active":"2024-04-04T06:05:54.365934+0000","last_peered":"2024-04-04T06:05:54.365934+0000","last_clean":"2024-04-04T06:05:54.365934+0000","last_became_active":"2024-04-04T06:05:53.378781+0000","last_became_peered":"2024-04-04T06:05:53.378781+0000","last_unstale":"2024-04-04T06:05:54.365934+0000","last_undegraded":"2024-04-04T06:05:54.365934+0000","last_fullsized":"2024-04-04T06:05:54.365934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:18:05.337194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594629+0000","last_change":"2024-04-04T06:05:52.372922+0000","last_active":"2024-04-04T06:05:54.594629+0000","last_peered":"2024-04-04T06:05:54.594629+0000","last_clean":"2024-04-04T06:05:54.594629+0000","last_became_active":"2024-04-04T06:05:52.372696+0000","last_became_peered":"2024-04-04T06:05:52.372696+0000","last_unstale":"2024-04-04T06:05:54.594629+0000","last_undegraded":"2024-04-04T06:05:54.594629+0000","last_fullsized":"2024-04-04T06:05:54.594629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:17:53.847287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362796+0000","last_change":"2024-04-04T06:05:53.373066+0000","last_active":"2024-04-04T06:05:54.362796+0000","last_peered":"2024-04-04T06:05:54.362796+0000","last_clean":"2024-04-04T06:05:54.362796+0000","last_became_active":"2024-04-04T06:05:53.367063+0000","last_became_peered":"2024-04-04T06:05:53.367063+0000","last_unstale":"2024-04-04T06:05:54.362796+0000","last_undegraded":"2024-04-04T06:05:54.362796+0000","last_fullsized":"2024-04-04T06:05:54.362796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:49:10.411661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.363331+0000","last_change":"2024-04-04T06:05:52.365125+0000","last_active":"2024-04-04T06:05:54.363331+0000","last_peered":"2024-04-04T06:05:54.363331+0000","last_clean":"2024-04-04T06:05:54.363331+0000","last_became_active":"2024-04-04T06:05:52.364821+0000","last_became_peered":"2024-04-04T06:05:52.364821+0000","last_unstale":"2024-04-04T06:05:54.363331+0000","last_undegraded":"2024-04-04T06:05:54.363331+0000","last_fullsized":"2024-04-04T06:05:54.363331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:54:42.610854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593547+0000","last_change":"2024-04-04T06:05:53.375887+0000","last_active":"2024-04-04T06:05:54.593547+0000","last_peered":"2024-04-04T06:05:54.593547+0000","last_clean":"2024-04-04T06:05:54.593547+0000","last_became_active":"2024-04-04T06:05:53.374688+0000","last_became_peered":"2024-04-04T06:05:53.374688+0000","last_unstale":"2024-04-04T06:05:54.593547+0000","last_undegraded":"2024-04-04T06:05:54.593547+0000","last_fullsized":"2024-04-04T06:05:54.593547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:45:21.548886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592567+0000","last_change":"2024-04-04T06:05:52.374804+0000","last_active":"2024-04-04T06:05:54.592567+0000","last_peered":"2024-04-04T06:05:54.592567+0000","last_clean":"2024-04-04T06:05:54.592567+0000","last_became_active":"2024-04-04T06:05:52.374476+0000","last_became_peered":"2024-04-04T06:05:52.374476+0000","last_unstale":"2024-04-04T06:05:54.592567+0000","last_undegraded":"2024-04-04T06:05:54.592567+0000","last_fullsized":"2024-04-04T06:05:54.592567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:21:06.579458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590233+0000","last_change":"2024-04-04T06:05:53.358691+0000","last_active":"2024-04-04T06:05:54.590233+0000","last_peered":"2024-04-04T06:05:54.590233+0000","last_clean":"2024-04-04T06:05:54.590233+0000","last_became_active":"2024-04-04T06:05:53.357996+0000","last_became_peered":"2024-04-04T06:05:53.357996+0000","last_unstale":"2024-04-04T06:05:54.590233+0000","last_undegraded":"2024-04-04T06:05:54.590233+0000","last_fullsized":"2024-04-04T06:05:54.590233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:55:58.689192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592379+0000","last_change":"2024-04-04T06:05:52.370047+0000","last_active":"2024-04-04T06:05:54.592379+0000","last_peered":"2024-04-04T06:05:54.592379+0000","last_clean":"2024-04-04T06:05:54.592379+0000","last_became_active":"2024-04-04T06:05:52.369741+0000","last_became_peered":"2024-04-04T06:05:52.369741+0000","last_unstale":"2024-04-04T06:05:54.592379+0000","last_undegraded":"2024-04-04T06:05:54.592379+0000","last_fullsized":"2024-04-04T06:05:54.592379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:36:38.928830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.364951+0000","last_change":"2024-04-04T06:05:53.364563+0000","last_active":"2024-04-04T06:05:54.364951+0000","last_peered":"2024-04-04T06:05:54.364951+0000","last_clean":"2024-04-04T06:05:54.364951+0000","last_became_active":"2024-04-04T06:05:53.362702+0000","last_became_peered":"2024-04-04T06:05:53.362702+0000","last_unstale":"2024-04-04T06:05:54.364951+0000","last_undegraded":"2024-04-04T06:05:54.364951+0000","last_fullsized":"2024-04-04T06:05:54.364951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:54:43.620559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362753+0000","last_change":"2024-04-04T06:05:52.370003+0000","last_active":"2024-04-04T06:05:54.362753+0000","last_peered":"2024-04-04T06:05:54.362753+0000","last_clean":"2024-04-04T06:05:54.362753+0000","last_became_active":"2024-04-04T06:05:52.369724+0000","last_became_peered":"2024-04-04T06:05:52.369724+0000","last_unstale":"2024-04-04T06:05:54.362753+0000","last_undegraded":"2024-04-04T06:05:54.362753+0000","last_fullsized":"2024-04-04T06:05:54.362753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:58:40.874784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.592885+0000","last_change":"2024-04-04T06:05:53.361642+0000","last_active":"2024-04-04T06:05:54.592885+0000","last_peered":"2024-04-04T06:05:54.592885+0000","last_clean":"2024-04-04T06:05:54.592885+0000","last_became_active":"2024-04-04T06:05:53.360951+0000","last_became_peered":"2024-04-04T06:05:53.360951+0000","last_unstale":"2024-04-04T06:05:54.592885+0000","last_undegraded":"2024-04-04T06:05:54.592885+0000","last_fullsized":"2024-04-04T06:05:54.592885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:21:23.535889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.362582+0000","last_change":"2024-04-04T06:05:52.371223+0000","last_active":"2024-04-04T06:05:54.362582+0000","last_peered":"2024-04-04T06:05:54.362582+0000","last_clean":"2024-04-04T06:05:54.362582+0000","last_became_active":"2024-04-04T06:05:52.370916+0000","last_became_peered":"2024-04-04T06:05:52.370916+0000","last_unstale":"2024-04-04T06:05:54.362582+0000","last_undegraded":"2024-04-04T06:05:54.362582+0000","last_fullsized":"2024-04-04T06:05:54.362582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:53:42.904072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593889+0000","last_change":"2024-04-04T06:05:53.371605+0000","last_active":"2024-04-04T06:05:54.593889+0000","last_peered":"2024-04-04T06:05:54.593889+0000","last_clean":"2024-04-04T06:05:54.593889+0000","last_became_active":"2024-04-04T06:05:53.371328+0000","last_became_peered":"2024-04-04T06:05:53.371328+0000","last_unstale":"2024-04-04T06:05:54.593889+0000","last_undegraded":"2024-04-04T06:05:54.593889+0000","last_fullsized":"2024-04-04T06:05:54.593889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:18:05.337194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593483+0000","last_change":"2024-04-04T06:05:52.368424+0000","last_active":"2024-04-04T06:05:54.593483+0000","last_peered":"2024-04-04T06:05:54.593483+0000","last_clean":"2024-04-04T06:05:54.593483+0000","last_became_active":"2024-04-04T06:05:52.368164+0000","last_became_peered":"2024-04-04T06:05:52.368164+0000","last_unstale":"2024-04-04T06:05:54.593483+0000","last_undegraded":"2024-04-04T06:05:54.593483+0000","last_fullsized":"2024-04-04T06:05:54.593483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:54:18.736812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591138+0000","last_change":"2024-04-04T06:05:53.373894+0000","last_active":"2024-04-04T06:05:54.591138+0000","last_peered":"2024-04-04T06:05:54.591138+0000","last_clean":"2024-04-04T06:05:54.591138+0000","last_became_active":"2024-04-04T06:05:53.373615+0000","last_became_peered":"2024-04-04T06:05:53.373615+0000","last_unstale":"2024-04-04T06:05:54.591138+0000","last_undegraded":"2024-04-04T06:05:54.591138+0000","last_fullsized":"2024-04-04T06:05:54.591138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:03:39.636711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365635+0000","last_change":"2024-04-04T06:05:53.378971+0000","last_active":"2024-04-04T06:05:54.365635+0000","last_peered":"2024-04-04T06:05:54.365635+0000","last_clean":"2024-04-04T06:05:54.365635+0000","last_became_active":"2024-04-04T06:05:53.377287+0000","last_became_peered":"2024-04-04T06:05:53.377287+0000","last_unstale":"2024-04-04T06:05:54.365635+0000","last_undegraded":"2024-04-04T06:05:54.365635+0000","last_fullsized":"2024-04-04T06:05:54.365635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:17:05.576402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.594399+0000","last_change":"2024-04-04T06:05:52.376155+0000","last_active":"2024-04-04T06:05:54.594399+0000","last_peered":"2024-04-04T06:05:54.594399+0000","last_clean":"2024-04-04T06:05:54.594399+0000","last_became_active":"2024-04-04T06:05:52.375772+0000","last_became_peered":"2024-04-04T06:05:52.375772+0000","last_unstale":"2024-04-04T06:05:54.594399+0000","last_undegraded":"2024-04-04T06:05:54.594399+0000","last_fullsized":"2024-04-04T06:05:54.594399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:40:09.906862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.365922+0000","last_change":"2024-04-04T06:05:53.381179+0000","last_active":"2024-04-04T06:05:54.365922+0000","last_peered":"2024-04-04T06:05:54.365922+0000","last_clean":"2024-04-04T06:05:54.365922+0000","last_became_active":"2024-04-04T06:05:53.374859+0000","last_became_peered":"2024-04-04T06:05:53.374859+0000","last_unstale":"2024-04-04T06:05:54.365922+0000","last_undegraded":"2024-04-04T06:05:54.365922+0000","last_fullsized":"2024-04-04T06:05:54.365922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:20:24.966139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593848+0000","last_change":"2024-04-04T06:05:52.378067+0000","last_active":"2024-04-04T06:05:54.593848+0000","last_peered":"2024-04-04T06:05:54.593848+0000","last_clean":"2024-04-04T06:05:54.593848+0000","last_became_active":"2024-04-04T06:05:52.377777+0000","last_became_peered":"2024-04-04T06:05:52.377777+0000","last_unstale":"2024-04-04T06:05:54.593848+0000","last_undegraded":"2024-04-04T06:05:54.593848+0000","last_fullsized":"2024-04-04T06:05:54.593848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:01:02.629926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.590922+0000","last_change":"2024-04-04T06:05:53.368705+0000","last_active":"2024-04-04T06:05:54.590922+0000","last_peered":"2024-04-04T06:05:54.590922+0000","last_clean":"2024-04-04T06:05:54.590922+0000","last_became_active":"2024-04-04T06:05:53.368366+0000","last_became_peered":"2024-04-04T06:05:53.368366+0000","last_unstale":"2024-04-04T06:05:54.590922+0000","last_undegraded":"2024-04-04T06:05:54.590922+0000","last_fullsized":"2024-04-04T06:05:54.590922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:34.766930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593814+0000","last_change":"2024-04-04T06:05:52.367095+0000","last_active":"2024-04-04T06:05:54.593814+0000","last_peered":"2024-04-04T06:05:54.593814+0000","last_clean":"2024-04-04T06:05:54.593814+0000","last_became_active":"2024-04-04T06:05:52.366316+0000","last_became_peered":"2024-04-04T06:05:52.366316+0000","last_unstale":"2024-04-04T06:05:54.593814+0000","last_undegraded":"2024-04-04T06:05:54.593814+0000","last_fullsized":"2024-04-04T06:05:54.593814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:05:06.878796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.593646+0000","last_change":"2024-04-04T06:05:53.375226+0000","last_active":"2024-04-04T06:05:54.593646+0000","last_peered":"2024-04-04T06:05:54.593646+0000","last_clean":"2024-04-04T06:05:54.593646+0000","last_became_active":"2024-04-04T06:05:53.374921+0000","last_became_peered":"2024-04-04T06:05:53.374921+0000","last_unstale":"2024-04-04T06:05:54.593646+0000","last_undegraded":"2024-04-04T06:05:54.593646+0000","last_fullsized":"2024-04-04T06:05:54.593646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:52.317707+0000","last_clean_scrub_stamp":"2024-04-04T06:05:52.317707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:20:24.966139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-04T06:05:54.591478+0000","last_change":"2024-04-04T06:05:52.370572+0000","last_active":"2024-04-04T06:05:54.591478+0000","last_peered":"2024-04-04T06:05:54.591478+0000","last_clean":"2024-04-04T06:05:54.591478+0000","last_became_active":"2024-04-04T06:05:52.369378+0000","last_became_peered":"2024-04-04T06:05:52.369378+0000","last_unstale":"2024-04-04T06:05:54.591478+0000","last_undegraded":"2024-04-04T06:05:54.591478+0000","last_fullsized":"2024-04-04T06:05:54.591478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:51.308002+0000","last_clean_scrub_stamp":"2024-04-04T06:05:51.308002+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:45:47.508028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":806912,"data_stored":663433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":74095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":74095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":72483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":663489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":72573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":72609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:06:08.870 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-04T06:06:08.871 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-04T06:06:08.871 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-04T06:06:08.871 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-04T06:06:09.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.118+0000 7f26d5400640 1 Processor -- start 2024-04-04T06:06:09.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.119+0000 7f26d5400640 1 -- start start 2024-04-04T06:06:09.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.120+0000 7f26d5400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f26d0153580 0x7f26d00d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:09.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.120+0000 7f26d5400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f26d00af4b0 0x7f26d01600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:09.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.120+0000 7f26d5400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d00afab0 con 0x7f26d00d07c0 2024-04-04T06:06:09.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.120+0000 7f26d5400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f26d00d0c30 con 0x7f26d00af4b0 2024-04-04T06:06:09.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.120+0000 7f26d5400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f26d015cc80 con 0x7f26d0153580 2024-04-04T06:06:09.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.120+0000 7f26cf200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f26d00af4b0 0x7f26d01600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:09.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.120+0000 7f26cf200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f26d00af4b0 0x7f26d01600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37340/0 (socket says 172.21.15.159:37340) 2024-04-04T06:06:09.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.120+0000 7f26cf200640 1 -- 172.21.15.159:0/3004935331 learned_addr learned my addr 172.21.15.159:0/3004935331 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:09.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.121+0000 7f26cf200640 1 -- 172.21.15.159:0/3004935331 >> v2:172.21.15.178:3300/0 conn(0x7f26d0153580 msgr2=0x7f26d00d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:09.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.121+0000 7f26cf200640 1 --2- 172.21.15.159:0/3004935331 >> v2:172.21.15.178:3300/0 conn(0x7f26d0153580 0x7f26d00d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:09.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.121+0000 7f26cf200640 1 -- 172.21.15.159:0/3004935331 >> v1:172.21.15.178:6789/0 conn(0x7f26d00d07c0 legacy=0x7f26d00aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:09.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.121+0000 7f26cf200640 1 -- 172.21.15.159:0/3004935331 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d015da00 con 0x7f26d00af4b0 2024-04-04T06:06:09.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.122+0000 7f26cf200640 1 --2- 172.21.15.159:0/3004935331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26d00af4b0 0x7f26d01600d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f26d00b5270 tx=0x7f26c802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74efb55cf71ac413 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:09.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.122+0000 7f26cde00640 1 -- 172.21.15.159:0/3004935331 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c802c9f0 con 0x7f26d00af4b0 2024-04-04T06:06:09.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.122+0000 7f26cde00640 1 -- 172.21.15.159:0/3004935331 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c802cb90 con 0x7f26d00af4b0 2024-04-04T06:06:09.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.122+0000 7f26cde00640 1 -- 172.21.15.159:0/3004935331 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c80343f0 con 0x7f26d00af4b0 2024-04-04T06:06:09.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.123+0000 7f26d5400640 1 -- 172.21.15.159:0/3004935331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26d00af4b0 msgr2=0x7f26d01600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:09.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.123+0000 7f26d5400640 1 --2- 172.21.15.159:0/3004935331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26d00af4b0 0x7f26d01600d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f26d00b5270 tx=0x7f26c802be50 comp rx=0 tx=0).stop 2024-04-04T06:06:09.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.123+0000 7f26d5400640 1 -- 172.21.15.159:0/3004935331 shutdown_connections 2024-04-04T06:06:09.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.123+0000 7f26d5400640 1 --2- 172.21.15.159:0/3004935331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26d00af4b0 0x7f26d01600d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:09.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.123+0000 7f26d5400640 1 --2- 172.21.15.159:0/3004935331 >> v2:172.21.15.178:3300/0 conn(0x7f26d0153580 0x7f26d00d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:09.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.123+0000 7f26d5400640 1 -- 172.21.15.159:0/3004935331 >> 172.21.15.159:0/3004935331 conn(0x7f26d006b750 msgr2=0x7f26d006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:09.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.123+0000 7f26d5400640 1 -- 172.21.15.159:0/3004935331 shutdown_connections 2024-04-04T06:06:09.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.123+0000 7f26d5400640 1 -- 172.21.15.159:0/3004935331 wait complete. 2024-04-04T06:06:09.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.125+0000 7f26d5400640 1 Processor -- start 2024-04-04T06:06:09.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.125+0000 7f26d5400640 1 -- start start 2024-04-04T06:06:09.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.126+0000 7f26d5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26d00af4b0 0x7f26d01554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:09.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.126+0000 7f26d5400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26d00d07c0 0x7f26d0153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:09.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.126+0000 7f26d5400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26d0153580 0x7f26d00b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:09.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.126+0000 7f26d5400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f26d015d4e0 con 0x7f26d00af4b0 2024-04-04T06:06:09.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.126+0000 7f26d5400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f26d015d200 con 0x7f26d00d07c0 2024-04-04T06:06:09.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.126+0000 7f26d5400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f26d0153960 con 0x7f26d0153580 2024-04-04T06:06:09.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.126+0000 7f26ce800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26d00af4b0 0x7f26d01554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:09.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.126+0000 7f26ce800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26d00af4b0 0x7f26d01554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37348/0 (socket says 172.21.15.159:37348) 2024-04-04T06:06:09.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.126+0000 7f26ce800640 1 -- 172.21.15.159:0/1955608652 learned_addr learned my addr 172.21.15.159:0/1955608652 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:09.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.126+0000 7f26c5e00640 1 --2- 172.21.15.159:0/1955608652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26d00d07c0 0x7f26d0153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:09.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.126+0000 7f26cf200640 1 --2- 172.21.15.159:0/1955608652 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26d0153580 0x7f26d00b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:09.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.126+0000 7f26ce800640 1 -- 172.21.15.159:0/1955608652 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26d0153580 msgr2=0x7f26d00b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:09.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.126+0000 7f26ce800640 1 --2- 172.21.15.159:0/1955608652 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26d0153580 0x7f26d00b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:09.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.126+0000 7f26ce800640 1 -- 172.21.15.159:0/1955608652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26d00d07c0 msgr2=0x7f26d0153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:09.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.126+0000 7f26ce800640 1 --2- 172.21.15.159:0/1955608652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26d00d07c0 0x7f26d0153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:09.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.126+0000 7f26ce800640 1 -- 172.21.15.159:0/1955608652 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d0154450 con 0x7f26d00af4b0 2024-04-04T06:06:09.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.127+0000 7f26ce800640 1 --2- 172.21.15.159:0/1955608652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26d00af4b0 0x7f26d01554f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f26c000b4d0 tx=0x7f26c000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:09.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.127+0000 7f26c7400640 1 -- 172.21.15.159:0/1955608652 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0014070 con 0x7f26d00af4b0 2024-04-04T06:06:09.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.127+0000 7f26c7400640 1 -- 172.21.15.159:0/1955608652 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c0010610 con 0x7f26d00af4b0 2024-04-04T06:06:09.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.127+0000 7f26c7400640 1 -- 172.21.15.159:0/1955608652 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0010a00 con 0x7f26d00af4b0 2024-04-04T06:06:09.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.127+0000 7f26d5400640 1 -- 172.21.15.159:0/1955608652 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d0154730 con 0x7f26d00af4b0 2024-04-04T06:06:09.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.127+0000 7f26d5400640 1 -- 172.21.15.159:0/1955608652 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d0154cf0 con 0x7f26d00af4b0 2024-04-04T06:06:09.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.128+0000 7f26d5400640 1 -- 172.21.15.159:0/1955608652 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26d0155120 con 0x7f26d00af4b0 2024-04-04T06:06:09.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.129+0000 7f26c7400640 1 -- 172.21.15.159:0/1955608652 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26c0017070 con 0x7f26d00af4b0 2024-04-04T06:06:09.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.130+0000 7f26c7400640 1 --2- 172.21.15.159:0/1955608652 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f26ac06c8f0 0x7f26ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:09.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.130+0000 7f26c7400640 1 -- 172.21.15.159:0/1955608652 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f26c008b700 con 0x7f26d00af4b0 2024-04-04T06:06:09.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.130+0000 7f26c5e00640 1 --2- 172.21.15.159:0/1955608652 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f26ac06c8f0 0x7f26ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:09.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.131+0000 7f26c5e00640 1 --2- 172.21.15.159:0/1955608652 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f26ac06c8f0 0x7f26ac06edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f26d015ca00 tx=0x7f26b8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:09.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.134+0000 7f26c7400640 1 -- 172.21.15.159:0/1955608652 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f26c0059a00 con 0x7f26d00af4b0 2024-04-04T06:06:09.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.380+0000 7f26d5400640 1 -- 172.21.15.159:0/1955608652 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f26d00685c0 con 0x7f26d00af4b0 2024-04-04T06:06:09.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.381+0000 7f26c7400640 1 -- 172.21.15.159:0/1955608652 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f26c0059300 con 0x7f26d00af4b0 2024-04-04T06:06:09.376 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:09.376 INFO:teuthology.orchestra.run.smithi159.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-04T06:06:09.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.383+0000 7f26d5400640 1 -- 172.21.15.159:0/1955608652 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f26ac06c8f0 msgr2=0x7f26ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:09.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.383+0000 7f26d5400640 1 --2- 172.21.15.159:0/1955608652 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f26ac06c8f0 0x7f26ac06edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f26d015ca00 tx=0x7f26b8009040 comp rx=0 tx=0).stop 2024-04-04T06:06:09.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.383+0000 7f26d5400640 1 -- 172.21.15.159:0/1955608652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26d00af4b0 msgr2=0x7f26d01554f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:09.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.383+0000 7f26d5400640 1 --2- 172.21.15.159:0/1955608652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26d00af4b0 0x7f26d01554f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f26c000b4d0 tx=0x7f26c000b990 comp rx=0 tx=0).stop 2024-04-04T06:06:09.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.384+0000 7f26d5400640 1 -- 172.21.15.159:0/1955608652 shutdown_connections 2024-04-04T06:06:09.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.384+0000 7f26d5400640 1 --2- 172.21.15.159:0/1955608652 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26d0153580 0x7f26d00b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:09.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.384+0000 7f26d5400640 1 --2- 172.21.15.159:0/1955608652 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f26ac06c8f0 0x7f26ac06edb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:09.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.384+0000 7f26d5400640 1 --2- 172.21.15.159:0/1955608652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26d00d07c0 0x7f26d0153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:09.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.384+0000 7f26d5400640 1 --2- 172.21.15.159:0/1955608652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26d00af4b0 0x7f26d01554f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:09.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.384+0000 7f26d5400640 1 -- 172.21.15.159:0/1955608652 >> 172.21.15.159:0/1955608652 conn(0x7f26d006b750 msgr2=0x7f26d015e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:09.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.384+0000 7f26d5400640 1 -- 172.21.15.159:0/1955608652 shutdown_connections 2024-04-04T06:06:09.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.384+0000 7f26d5400640 1 -- 172.21.15.159:0/1955608652 wait complete. 2024-04-04T06:06:09.397 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-04T06:06:09.398 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:06:09.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.605+0000 7f7dd0600640 1 Processor -- start 2024-04-04T06:06:09.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.605+0000 7f7dd0600640 1 -- start start 2024-04-04T06:06:09.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.606+0000 7f7dd0600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7dc8161b00 0x7f7dc8162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:09.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.606+0000 7f7dd0600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7dc80ad120 0x7f7dc80d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:09.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.606+0000 7f7dd0600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dc8071b60 con 0x7f7dc80ad580 2024-04-04T06:06:09.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.606+0000 7f7dd0600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f7dc8161ee0 con 0x7f7dc8161b00 2024-04-04T06:06:09.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.606+0000 7f7dd0600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7dc80b3450 con 0x7f7dc80ad120 2024-04-04T06:06:09.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.607+0000 7f7dcdc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7dc8161b00 0x7f7dc8162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:09.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.607+0000 7f7dcdc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7dc8161b00 0x7f7dc8162ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37360/0 (socket says 172.21.15.159:37360) 2024-04-04T06:06:09.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.607+0000 7f7dcdc00640 1 -- 172.21.15.159:0/3667348400 learned_addr learned my addr 172.21.15.159:0/3667348400 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:09.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.607+0000 7f7dcdc00640 1 -- 172.21.15.159:0/3667348400 >> v2:172.21.15.178:3300/0 conn(0x7f7dc80ad120 msgr2=0x7f7dc80d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:09.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.607+0000 7f7dcdc00640 1 --2- 172.21.15.159:0/3667348400 >> v2:172.21.15.178:3300/0 conn(0x7f7dc80ad120 0x7f7dc80d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:09.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.607+0000 7f7dcdc00640 1 -- 172.21.15.159:0/3667348400 >> v1:172.21.15.178:6789/0 conn(0x7f7dc80ad580 legacy=0x7f7dc80b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:09.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.607+0000 7f7dcdc00640 1 -- 172.21.15.159:0/3667348400 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc8071e40 con 0x7f7dc8161b00 2024-04-04T06:06:09.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.608+0000 7f7dcdc00640 1 --2- 172.21.15.159:0/3667348400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dc8161b00 0x7f7dc8162ab0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7dc806ea10 tx=0x7f7db802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be223d8dfb684544 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:09.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.609+0000 7f7dc7e00640 1 -- 172.21.15.159:0/3667348400 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8035020 con 0x7f7dc8161b00 2024-04-04T06:06:09.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.609+0000 7f7dc7e00640 1 -- 172.21.15.159:0/3667348400 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db8031870 con 0x7f7dc8161b00 2024-04-04T06:06:09.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.609+0000 7f7dc7e00640 1 -- 172.21.15.159:0/3667348400 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8031c40 con 0x7f7dc8161b00 2024-04-04T06:06:09.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.609+0000 7f7dd0600640 1 -- 172.21.15.159:0/3667348400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dc8161b00 msgr2=0x7f7dc8162ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:09.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.609+0000 7f7dd0600640 1 --2- 172.21.15.159:0/3667348400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dc8161b00 0x7f7dc8162ab0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7dc806ea10 tx=0x7f7db802b910 comp rx=0 tx=0).stop 2024-04-04T06:06:09.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.609+0000 7f7dd0600640 1 -- 172.21.15.159:0/3667348400 shutdown_connections 2024-04-04T06:06:09.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.609+0000 7f7dd0600640 1 --2- 172.21.15.159:0/3667348400 >> v2:172.21.15.178:3300/0 conn(0x7f7dc80ad120 0x7f7dc80d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:09.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.609+0000 7f7dd0600640 1 --2- 172.21.15.159:0/3667348400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dc8161b00 0x7f7dc8162ab0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:09.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.609+0000 7f7dd0600640 1 -- 172.21.15.159:0/3667348400 >> 172.21.15.159:0/3667348400 conn(0x7f7dc806b990 msgr2=0x7f7dc806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:09.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.610+0000 7f7dd0600640 1 -- 172.21.15.159:0/3667348400 shutdown_connections 2024-04-04T06:06:09.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.610+0000 7f7dd0600640 1 -- 172.21.15.159:0/3667348400 wait complete. 2024-04-04T06:06:09.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.612+0000 7f7dd0600640 1 Processor -- start 2024-04-04T06:06:09.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.612+0000 7f7dd0600640 1 -- start start 2024-04-04T06:06:09.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.613+0000 7f7dd0600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7dc80ad120 0x7f7dc81526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:09.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.613+0000 7f7dd0600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dc80ad580 0x7f7dc8156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:09.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.613+0000 7f7dd0600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7dc8161b00 0x7f7dc8157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:09.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.613+0000 7f7dd0600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc8162ff0 con 0x7f7dc80ad580 2024-04-04T06:06:09.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.613+0000 7f7dd0600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc80b3730 con 0x7f7dc80ad120 2024-04-04T06:06:09.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.613+0000 7f7dd0600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7dc80717f0 con 0x7f7dc8161b00 2024-04-04T06:06:09.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.613+0000 7f7dcd200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dc80ad580 0x7f7dc8156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:09.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.613+0000 7f7dcd200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dc80ad580 0x7f7dc8156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37368/0 (socket says 172.21.15.159:37368) 2024-04-04T06:06:09.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.613+0000 7f7dcd200640 1 -- 172.21.15.159:0/2301632292 learned_addr learned my addr 172.21.15.159:0/2301632292 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:09.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.613+0000 7f7dcdc00640 1 --2- 172.21.15.159:0/2301632292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7dc80ad120 0x7f7dc81526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:09.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.613+0000 7f7dce600640 1 --2- 172.21.15.159:0/2301632292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7dc8161b00 0x7f7dc8157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:09.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.613+0000 7f7dcd200640 1 -- 172.21.15.159:0/2301632292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7dc8161b00 msgr2=0x7f7dc8157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:09.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.613+0000 7f7dcd200640 1 --2- 172.21.15.159:0/2301632292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7dc8161b00 0x7f7dc8157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:09.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.613+0000 7f7dcd200640 1 -- 172.21.15.159:0/2301632292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7dc80ad120 msgr2=0x7f7dc81526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:09.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.613+0000 7f7dcd200640 1 --2- 172.21.15.159:0/2301632292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7dc80ad120 0x7f7dc81526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:09.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.613+0000 7f7dcd200640 1 -- 172.21.15.159:0/2301632292 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc8157b40 con 0x7f7dc80ad580 2024-04-04T06:06:09.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.614+0000 7f7dcd200640 1 --2- 172.21.15.159:0/2301632292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dc80ad580 0x7f7dc8156fd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7db400ba50 tx=0x7f7db400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:09.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.614+0000 7f7dc5600640 1 -- 172.21.15.159:0/2301632292 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4019070 con 0x7f7dc80ad580 2024-04-04T06:06:09.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.614+0000 7f7dc5600640 1 -- 172.21.15.159:0/2301632292 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db400cc00 con 0x7f7dc80ad580 2024-04-04T06:06:09.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.614+0000 7f7dc5600640 1 -- 172.21.15.159:0/2301632292 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4014440 con 0x7f7dc80ad580 2024-04-04T06:06:09.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.614+0000 7f7dd0600640 1 -- 172.21.15.159:0/2301632292 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc8157e20 con 0x7f7dc80ad580 2024-04-04T06:06:09.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.614+0000 7f7dd0600640 1 -- 172.21.15.159:0/2301632292 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc81583e0 con 0x7f7dc80ad580 2024-04-04T06:06:09.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.615+0000 7f7dd0600640 1 -- 172.21.15.159:0/2301632292 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dc80685c0 con 0x7f7dc80ad580 2024-04-04T06:06:09.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.616+0000 7f7dc5600640 1 -- 172.21.15.159:0/2301632292 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7db401c070 con 0x7f7dc80ad580 2024-04-04T06:06:09.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.617+0000 7f7dc5600640 1 --2- 172.21.15.159:0/2301632292 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7da806c8f0 0x7f7da806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:09.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.617+0000 7f7dc5600640 1 -- 172.21.15.159:0/2301632292 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7db408b8a0 con 0x7f7dc80ad580 2024-04-04T06:06:09.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.617+0000 7f7dcdc00640 1 --2- 172.21.15.159:0/2301632292 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7da806c8f0 0x7f7da806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:09.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.618+0000 7f7dcdc00640 1 --2- 172.21.15.159:0/2301632292 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7da806c8f0 0x7f7da806edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7db8036000 tx=0x7f7db803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:09.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.621+0000 7f7dc5600640 1 -- 172.21.15.159:0/2301632292 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7db4059cd0 con 0x7f7dc80ad580 2024-04-04T06:06:09.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.823+0000 7f7dd0600640 1 -- 172.21.15.159:0/2301632292 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7dc80b2360 con 0x7f7dc80ad580 2024-04-04T06:06:09.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.826+0000 7f7dc5600640 1 -- 172.21.15.159:0/2301632292 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f7db40595d0 con 0x7f7dc80ad580 2024-04-04T06:06:09.821 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:09.821 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":21,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:05:54.347275+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:05:51.297293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-04T06:05:51.770792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:06:09.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.828+0000 7f7dd0600640 1 -- 172.21.15.159:0/2301632292 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7da806c8f0 msgr2=0x7f7da806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:09.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.828+0000 7f7dd0600640 1 --2- 172.21.15.159:0/2301632292 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7da806c8f0 0x7f7da806edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7db8036000 tx=0x7f7db803c040 comp rx=0 tx=0).stop 2024-04-04T06:06:09.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.829+0000 7f7dd0600640 1 -- 172.21.15.159:0/2301632292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dc80ad580 msgr2=0x7f7dc8156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:09.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.829+0000 7f7dd0600640 1 --2- 172.21.15.159:0/2301632292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dc80ad580 0x7f7dc8156fd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7db400ba50 tx=0x7f7db400bf10 comp rx=0 tx=0).stop 2024-04-04T06:06:09.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.829+0000 7f7dd0600640 1 -- 172.21.15.159:0/2301632292 shutdown_connections 2024-04-04T06:06:09.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.829+0000 7f7dd0600640 1 --2- 172.21.15.159:0/2301632292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7dc8161b00 0x7f7dc8157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:09.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.829+0000 7f7dd0600640 1 --2- 172.21.15.159:0/2301632292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dc80ad580 0x7f7dc8156fd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:09.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.829+0000 7f7dd0600640 1 --2- 172.21.15.159:0/2301632292 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7da806c8f0 0x7f7da806edb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:09.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.829+0000 7f7dd0600640 1 --2- 172.21.15.159:0/2301632292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7dc80ad120 0x7f7dc81526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:09.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.829+0000 7f7dd0600640 1 -- 172.21.15.159:0/2301632292 >> 172.21.15.159:0/2301632292 conn(0x7f7dc806b990 msgr2=0x7f7dc80b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:09.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.829+0000 7f7dd0600640 1 -- 172.21.15.159:0/2301632292 shutdown_connections 2024-04-04T06:06:09.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:09.829+0000 7f7dd0600640 1 -- 172.21.15.159:0/2301632292 wait complete. 2024-04-04T06:06:09.842 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:05:51.297293+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:05:51.770792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:06:09.843 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:06:10.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.055+0000 7fdd1de00640 1 Processor -- start 2024-04-04T06:06:10.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.055+0000 7fdd1de00640 1 -- start start 2024-04-04T06:06:10.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.057+0000 7fdd1de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fdd1813f190 0x7fdd180cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:10.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.057+0000 7fdd1de00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdd180cc880 0x7fdd180cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:10.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.057+0000 7fdd1de00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd180ce890 con 0x7fdd1806d620 2024-04-04T06:06:10.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.057+0000 7fdd1de00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fdd180cd230 con 0x7fdd1813f190 2024-04-04T06:06:10.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.057+0000 7fdd1de00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fdd180ceb50 con 0x7fdd180cc880 2024-04-04T06:06:10.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.057+0000 7fdd1ca00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fdd1813f190 0x7fdd180cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:10.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.057+0000 7fdd1ca00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fdd1813f190 0x7fdd180cf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37384/0 (socket says 172.21.15.159:37384) 2024-04-04T06:06:10.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.057+0000 7fdd1ca00640 1 -- 172.21.15.159:0/3595597692 learned_addr learned my addr 172.21.15.159:0/3595597692 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:10.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.058+0000 7fdd0fe00640 1 --2- 172.21.15.159:0/3595597692 >> v2:172.21.15.178:3300/0 conn(0x7fdd180cc880 0x7fdd180cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:10.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.058+0000 7fdd1ca00640 1 -- 172.21.15.159:0/3595597692 >> v2:172.21.15.178:3300/0 conn(0x7fdd180cc880 msgr2=0x7fdd180cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:10.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.058+0000 7fdd1ca00640 1 --2- 172.21.15.159:0/3595597692 >> v2:172.21.15.178:3300/0 conn(0x7fdd180cc880 0x7fdd180cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.058+0000 7fdd1ca00640 1 -- 172.21.15.159:0/3595597692 >> v1:172.21.15.178:6789/0 conn(0x7fdd1806d620 legacy=0x7fdd1806da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:10.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.058+0000 7fdd1ca00640 1 -- 172.21.15.159:0/3595597692 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd180cee30 con 0x7fdd1813f190 2024-04-04T06:06:10.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.059+0000 7fdd1ca00640 1 --2- 172.21.15.159:0/3595597692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd1813f190 0x7fdd180cf630 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdd180cc7b0 tx=0x7fdd0802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ceadeca8be76bf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:10.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.060+0000 7fdd17e00640 1 -- 172.21.15.159:0/3595597692 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08035020 con 0x7fdd1813f190 2024-04-04T06:06:10.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.060+0000 7fdd17e00640 1 -- 172.21.15.159:0/3595597692 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd08031870 con 0x7fdd1813f190 2024-04-04T06:06:10.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.060+0000 7fdd17e00640 1 -- 172.21.15.159:0/3595597692 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08031c40 con 0x7fdd1813f190 2024-04-04T06:06:10.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.060+0000 7fdd1de00640 1 -- 172.21.15.159:0/3595597692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd1813f190 msgr2=0x7fdd180cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:10.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.060+0000 7fdd1de00640 1 --2- 172.21.15.159:0/3595597692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd1813f190 0x7fdd180cf630 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdd180cc7b0 tx=0x7fdd0802b910 comp rx=0 tx=0).stop 2024-04-04T06:06:10.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.061+0000 7fdd1de00640 1 -- 172.21.15.159:0/3595597692 shutdown_connections 2024-04-04T06:06:10.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.061+0000 7fdd1de00640 1 --2- 172.21.15.159:0/3595597692 >> v2:172.21.15.178:3300/0 conn(0x7fdd180cc880 0x7fdd180cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.061+0000 7fdd1de00640 1 --2- 172.21.15.159:0/3595597692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd1813f190 0x7fdd180cf630 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.061+0000 7fdd1de00640 1 -- 172.21.15.159:0/3595597692 >> 172.21.15.159:0/3595597692 conn(0x7fdd180b1ba0 msgr2=0x7fdd180b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:10.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.061+0000 7fdd1de00640 1 -- 172.21.15.159:0/3595597692 shutdown_connections 2024-04-04T06:06:10.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.061+0000 7fdd1de00640 1 -- 172.21.15.159:0/3595597692 wait complete. 2024-04-04T06:06:10.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.062+0000 7fdd1de00640 1 Processor -- start 2024-04-04T06:06:10.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.063+0000 7fdd1de00640 1 -- start start 2024-04-04T06:06:10.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.064+0000 7fdd1de00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd1806d620 0x7fdd1813cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:10.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.064+0000 7fdd1de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd180cc880 0x7fdd181e79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:10.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.064+0000 7fdd1de00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd1813f190 0x7fdd18072430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:10.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.064+0000 7fdd1de00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1813f570 con 0x7fdd180cc880 2024-04-04T06:06:10.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.064+0000 7fdd1de00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd18160c70 con 0x7fdd1806d620 2024-04-04T06:06:10.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.064+0000 7fdd1de00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdd180cfb70 con 0x7fdd1813f190 2024-04-04T06:06:10.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.064+0000 7fdd0fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd180cc880 0x7fdd181e79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:10.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.064+0000 7fdd0fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd180cc880 0x7fdd181e79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37392/0 (socket says 172.21.15.159:37392) 2024-04-04T06:06:10.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.064+0000 7fdd0fe00640 1 -- 172.21.15.159:0/29519124 learned_addr learned my addr 172.21.15.159:0/29519124 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:10.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.064+0000 7fdd1ca00640 1 --2- 172.21.15.159:0/29519124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd1806d620 0x7fdd1813cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:10.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.064+0000 7fdd1d400640 1 --2- 172.21.15.159:0/29519124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd1813f190 0x7fdd18072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:10.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.064+0000 7fdd0fe00640 1 -- 172.21.15.159:0/29519124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd1813f190 msgr2=0x7fdd18072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:10.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.064+0000 7fdd0fe00640 1 --2- 172.21.15.159:0/29519124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd1813f190 0x7fdd18072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.064+0000 7fdd0fe00640 1 -- 172.21.15.159:0/29519124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd1806d620 msgr2=0x7fdd1813cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:10.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.064+0000 7fdd0fe00640 1 --2- 172.21.15.159:0/29519124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd1806d620 0x7fdd1813cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.064+0000 7fdd0fe00640 1 -- 172.21.15.159:0/29519124 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd181e5b00 con 0x7fdd180cc880 2024-04-04T06:06:10.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.065+0000 7fdd0fe00640 1 --2- 172.21.15.159:0/29519124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd180cc880 0x7fdd181e79c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000d5f0 tx=0x7fdd0000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:10.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.065+0000 7fdd15600640 1 -- 172.21.15.159:0/29519124 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd000125f0 con 0x7fdd180cc880 2024-04-04T06:06:10.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.065+0000 7fdd15600640 1 -- 172.21.15.159:0/29519124 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd00019070 con 0x7fdd180cc880 2024-04-04T06:06:10.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.065+0000 7fdd15600640 1 -- 172.21.15.159:0/29519124 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd000129d0 con 0x7fdd180cc880 2024-04-04T06:06:10.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.065+0000 7fdd1de00640 1 -- 172.21.15.159:0/29519124 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd181e5de0 con 0x7fdd180cc880 2024-04-04T06:06:10.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.065+0000 7fdd1de00640 1 -- 172.21.15.159:0/29519124 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd181e63a0 con 0x7fdd180cc880 2024-04-04T06:06:10.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.066+0000 7fdd1de00640 1 -- 172.21.15.159:0/29519124 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd18075940 con 0x7fdd180cc880 2024-04-04T06:06:10.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.067+0000 7fdd15600640 1 -- 172.21.15.159:0/29519124 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd00012b70 con 0x7fdd180cc880 2024-04-04T06:06:10.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.068+0000 7fdd15600640 1 --2- 172.21.15.159:0/29519124 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdcf406c8f0 0x7fdcf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:10.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.068+0000 7fdd15600640 1 -- 172.21.15.159:0/29519124 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdd0008bfd0 con 0x7fdd180cc880 2024-04-04T06:06:10.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.068+0000 7fdd1ca00640 1 --2- 172.21.15.159:0/29519124 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdcf406c8f0 0x7fdcf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:10.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.070+0000 7fdd1ca00640 1 --2- 172.21.15.159:0/29519124 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdcf406c8f0 0x7fdcf406edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdd08036000 tx=0x7fdd0803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:10.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.073+0000 7fdd15600640 1 -- 172.21.15.159:0/29519124 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fdd0005a360 con 0x7fdd180cc880 2024-04-04T06:06:10.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.273+0000 7fdd1de00640 1 -- 172.21.15.159:0/29519124 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdd1806c580 con 0x7fdd180cc880 2024-04-04T06:06:10.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.274+0000 7fdd15600640 1 -- 172.21.15.159:0/29519124 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fdd00059c60 con 0x7fdd180cc880 2024-04-04T06:06:10.268 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:06:10.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.276+0000 7fdd1de00640 1 -- 172.21.15.159:0/29519124 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdcf406c8f0 msgr2=0x7fdcf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:10.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.276+0000 7fdd1de00640 1 --2- 172.21.15.159:0/29519124 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdcf406c8f0 0x7fdcf406edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdd08036000 tx=0x7fdd0803c040 comp rx=0 tx=0).stop 2024-04-04T06:06:10.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.276+0000 7fdd1de00640 1 -- 172.21.15.159:0/29519124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd180cc880 msgr2=0x7fdd181e79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:10.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.276+0000 7fdd1de00640 1 --2- 172.21.15.159:0/29519124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd180cc880 0x7fdd181e79c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000d5f0 tx=0x7fdd0000dab0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.276+0000 7fdd1de00640 1 -- 172.21.15.159:0/29519124 shutdown_connections 2024-04-04T06:06:10.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.276+0000 7fdd1de00640 1 --2- 172.21.15.159:0/29519124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd1813f190 0x7fdd18072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.276+0000 7fdd1de00640 1 --2- 172.21.15.159:0/29519124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd180cc880 0x7fdd181e79c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.276+0000 7fdd1de00640 1 --2- 172.21.15.159:0/29519124 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdcf406c8f0 0x7fdcf406edb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.276+0000 7fdd1de00640 1 --2- 172.21.15.159:0/29519124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd1806d620 0x7fdd1813cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.276+0000 7fdd1de00640 1 -- 172.21.15.159:0/29519124 >> 172.21.15.159:0/29519124 conn(0x7fdd180b1ba0 msgr2=0x7fdd1815eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:10.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.276+0000 7fdd1de00640 1 -- 172.21.15.159:0/29519124 shutdown_connections 2024-04-04T06:06:10.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.277+0000 7fdd1de00640 1 -- 172.21.15.159:0/29519124 wait complete. 2024-04-04T06:06:10.288 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:06:10.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.501+0000 7fef11a00640 1 Processor -- start 2024-04-04T06:06:10.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.501+0000 7fef11a00640 1 -- start start 2024-04-04T06:06:10.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.502+0000 7fef11a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fef0c155550 0x7fef0c155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:10.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.502+0000 7fef11a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef0c0d0990 0x7fef0c0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:10.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.502+0000 7fef11a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef0c0b40b0 con 0x7fef0c15f800 2024-04-04T06:06:10.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.502+0000 7fef11a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fef0c0b4390 con 0x7fef0c155550 2024-04-04T06:06:10.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.502+0000 7fef11a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fef0c0b4670 con 0x7fef0c0d0990 2024-04-04T06:06:10.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.503+0000 7fef0aa00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fef0c155550 0x7fef0c155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:10.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.503+0000 7fef0a000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fef0c15f800 0x7fef0c0d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:46008/0 (socket says 172.21.15.159:46008) 2024-04-04T06:06:10.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.503+0000 7fef0a000640 1 -- 172.21.15.159:0/4008320776 learned_addr learned my addr 172.21.15.159:0/4008320776 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:10.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.503+0000 7fef0b400640 1 --2- 172.21.15.159:0/4008320776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef0c0d0990 0x7fef0c0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:10.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.503+0000 7fef09600640 1 -- 172.21.15.159:0/4008320776 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3134999102 0 0) 0x7fef0c0b40b0 con 0x7fef0c15f800 2024-04-04T06:06:10.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.503+0000 7fef0aa00640 1 -- 172.21.15.159:0/4008320776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef0c0d0990 msgr2=0x7fef0c0aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:10.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.503+0000 7fef0aa00640 1 --2- 172.21.15.159:0/4008320776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef0c0d0990 0x7fef0c0aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.503+0000 7fef0aa00640 1 -- 172.21.15.159:0/4008320776 >> v1:172.21.15.159:6789/0 conn(0x7fef0c15f800 legacy=0x7fef0c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:10.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.503+0000 7fef0aa00640 1 -- 172.21.15.159:0/4008320776 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0c0b4950 con 0x7fef0c155550 2024-04-04T06:06:10.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.505+0000 7fef0aa00640 1 --2- 172.21.15.159:0/4008320776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef0c155550 0x7fef0c155930 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7feefc0097a0 tx=0x7feefc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e4ddc07ba463f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:10.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.505+0000 7fef09600640 1 -- 172.21.15.159:0/4008320776 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc035020 con 0x7fef0c155550 2024-04-04T06:06:10.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.505+0000 7fef09600640 1 -- 172.21.15.159:0/4008320776 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feefc031830 con 0x7fef0c155550 2024-04-04T06:06:10.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.505+0000 7fef09600640 1 -- 172.21.15.159:0/4008320776 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc031c00 con 0x7fef0c155550 2024-04-04T06:06:10.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.505+0000 7fef11a00640 1 -- 172.21.15.159:0/4008320776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef0c155550 msgr2=0x7fef0c155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:10.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.505+0000 7fef11a00640 1 --2- 172.21.15.159:0/4008320776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef0c155550 0x7fef0c155930 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7feefc0097a0 tx=0x7feefc02b8d0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.505+0000 7fef11a00640 1 -- 172.21.15.159:0/4008320776 shutdown_connections 2024-04-04T06:06:10.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.505+0000 7fef11a00640 1 --2- 172.21.15.159:0/4008320776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef0c0d0990 0x7fef0c0aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.505+0000 7fef11a00640 1 --2- 172.21.15.159:0/4008320776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef0c155550 0x7fef0c155930 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.505+0000 7fef11a00640 1 -- 172.21.15.159:0/4008320776 >> 172.21.15.159:0/4008320776 conn(0x7fef0c06b750 msgr2=0x7fef0c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:10.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.506+0000 7fef11a00640 1 -- 172.21.15.159:0/4008320776 shutdown_connections 2024-04-04T06:06:10.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.506+0000 7fef11a00640 1 -- 172.21.15.159:0/4008320776 wait complete. 2024-04-04T06:06:10.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.507+0000 7fef11a00640 1 Processor -- start 2024-04-04T06:06:10.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.507+0000 7fef11a00640 1 -- start start 2024-04-04T06:06:10.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.508+0000 7fef11a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef0c0d0990 0x7fef0c147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:10.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.508+0000 7fef11a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef0c155550 0x7fef0c09a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:10.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.508+0000 7fef11a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef0c15f800 0x7fef0c09a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:10.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.508+0000 7fef11a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c15ba20 con 0x7fef0c155550 2024-04-04T06:06:10.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.508+0000 7fef11a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c15fc80 con 0x7fef0c0d0990 2024-04-04T06:06:10.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.508+0000 7fef11a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fef0c15bba0 con 0x7fef0c15f800 2024-04-04T06:06:10.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.508+0000 7fef0a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef0c155550 0x7fef0c09a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:10.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.508+0000 7fef0a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef0c155550 0x7fef0c09a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37408/0 (socket says 172.21.15.159:37408) 2024-04-04T06:06:10.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.508+0000 7fef0a000640 1 -- 172.21.15.159:0/2980931338 learned_addr learned my addr 172.21.15.159:0/2980931338 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:10.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.509+0000 7fef0a000640 1 -- 172.21.15.159:0/2980931338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef0c15f800 msgr2=0x7fef0c09a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:10.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.509+0000 7fef0a000640 1 --2- 172.21.15.159:0/2980931338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef0c15f800 0x7fef0c09a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.509+0000 7fef0a000640 1 -- 172.21.15.159:0/2980931338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef0c0d0990 msgr2=0x7fef0c147ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:10.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.509+0000 7fef0a000640 1 --2- 172.21.15.159:0/2980931338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef0c0d0990 0x7fef0c147ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.509+0000 7fef0a000640 1 -- 172.21.15.159:0/2980931338 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0c146a10 con 0x7fef0c155550 2024-04-04T06:06:10.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.509+0000 7fef0a000640 1 --2- 172.21.15.159:0/2980931338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef0c155550 0x7fef0c09a190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7feef800c4d0 tx=0x7feef800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:10.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.509+0000 7fef02a00640 1 -- 172.21.15.159:0/2980931338 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef8018070 con 0x7fef0c155550 2024-04-04T06:06:10.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.509+0000 7fef02a00640 1 -- 172.21.15.159:0/2980931338 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef8007500 con 0x7fef0c155550 2024-04-04T06:06:10.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.509+0000 7fef02a00640 1 -- 172.21.15.159:0/2980931338 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef8004a20 con 0x7fef0c155550 2024-04-04T06:06:10.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.509+0000 7fef11a00640 1 -- 172.21.15.159:0/2980931338 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0c146cf0 con 0x7fef0c155550 2024-04-04T06:06:10.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.510+0000 7fef11a00640 1 -- 172.21.15.159:0/2980931338 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef0c1472b0 con 0x7fef0c155550 2024-04-04T06:06:10.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.511+0000 7fef11a00640 1 -- 172.21.15.159:0/2980931338 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef0c0685c0 con 0x7fef0c155550 2024-04-04T06:06:10.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.512+0000 7fef02a00640 1 -- 172.21.15.159:0/2980931338 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feef801b070 con 0x7fef0c155550 2024-04-04T06:06:10.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.512+0000 7fef02a00640 1 --2- 172.21.15.159:0/2980931338 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feeec06c8f0 0x7feeec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:10.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.512+0000 7fef02a00640 1 -- 172.21.15.159:0/2980931338 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7feef808af10 con 0x7fef0c155550 2024-04-04T06:06:10.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.513+0000 7fef0aa00640 1 --2- 172.21.15.159:0/2980931338 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feeec06c8f0 0x7feeec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:10.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.514+0000 7fef0aa00640 1 --2- 172.21.15.159:0/2980931338 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feeec06c8f0 0x7feeec06edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feefc036000 tx=0x7feefc030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:10.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.517+0000 7fef02a00640 1 -- 172.21.15.159:0/2980931338 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7feef80592a0 con 0x7fef0c155550 2024-04-04T06:06:10.548 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:06:10.554 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:06:10.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.711+0000 7fef11a00640 1 -- 172.21.15.159:0/2980931338 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fef0c0b3ad0 con 0x7fef0c155550 2024-04-04T06:06:10.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.711+0000 7fef02a00640 1 -- 172.21.15.159:0/2980931338 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7feef8058ba0 con 0x7fef0c155550 2024-04-04T06:06:10.707 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:06:10.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.714+0000 7fef11a00640 1 -- 172.21.15.159:0/2980931338 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feeec06c8f0 msgr2=0x7feeec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:10.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.714+0000 7fef11a00640 1 --2- 172.21.15.159:0/2980931338 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feeec06c8f0 0x7feeec06edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feefc036000 tx=0x7feefc030000 comp rx=0 tx=0).stop 2024-04-04T06:06:10.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.714+0000 7fef11a00640 1 -- 172.21.15.159:0/2980931338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef0c155550 msgr2=0x7fef0c09a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:10.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.714+0000 7fef11a00640 1 --2- 172.21.15.159:0/2980931338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef0c155550 0x7fef0c09a190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7feef800c4d0 tx=0x7feef800c990 comp rx=0 tx=0).stop 2024-04-04T06:06:10.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.714+0000 7fef11a00640 1 -- 172.21.15.159:0/2980931338 shutdown_connections 2024-04-04T06:06:10.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.714+0000 7fef11a00640 1 --2- 172.21.15.159:0/2980931338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef0c15f800 0x7fef0c09a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.714+0000 7fef11a00640 1 --2- 172.21.15.159:0/2980931338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef0c155550 0x7fef0c09a190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.714+0000 7fef11a00640 1 --2- 172.21.15.159:0/2980931338 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feeec06c8f0 0x7feeec06edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.714+0000 7fef11a00640 1 --2- 172.21.15.159:0/2980931338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef0c0d0990 0x7fef0c147ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.714+0000 7fef11a00640 1 -- 172.21.15.159:0/2980931338 >> 172.21.15.159:0/2980931338 conn(0x7fef0c06b750 msgr2=0x7fef0c06d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:10.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.714+0000 7fef11a00640 1 -- 172.21.15.159:0/2980931338 shutdown_connections 2024-04-04T06:06:10.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.714+0000 7fef11a00640 1 -- 172.21.15.159:0/2980931338 wait complete. 2024-04-04T06:06:10.728 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:06:10.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.942+0000 7ff54ca00640 1 Processor -- start 2024-04-04T06:06:10.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.942+0000 7ff54ca00640 1 -- start start 2024-04-04T06:06:10.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.944+0000 7ff54ca00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff5480d0280 0x7ff5480d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:10.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.944+0000 7ff54ca00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff548141c10 0x7ff548071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:10.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.944+0000 7ff54ca00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff54815ec80 con 0x7ff548071eb0 2024-04-04T06:06:10.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.944+0000 7ff54ca00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff5480d0ba0 con 0x7ff5480d0280 2024-04-04T06:06:10.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.944+0000 7ff54ca00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff54815ef20 con 0x7ff548141c10 2024-04-04T06:06:10.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.945+0000 7ff545400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff548141c10 0x7ff548071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:10.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.945+0000 7ff545400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff548141c10 0x7ff548071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:55292/0 (socket says 172.21.15.159:55292) 2024-04-04T06:06:10.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.945+0000 7ff545400640 1 -- 172.21.15.159:0/1627138897 learned_addr learned my addr 172.21.15.159:0/1627138897 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:10.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.945+0000 7ff545e00640 1 --2- 172.21.15.159:0/1627138897 >> v2:172.21.15.178:3300/0 conn(0x7ff5480d0280 0x7ff5480d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:10.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.945+0000 7ff545400640 1 -- 172.21.15.159:0/1627138897 >> v2:172.21.15.178:3300/0 conn(0x7ff5480d0280 msgr2=0x7ff5480d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:10.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.946+0000 7ff545400640 1 --2- 172.21.15.159:0/1627138897 >> v2:172.21.15.178:3300/0 conn(0x7ff5480d0280 0x7ff5480d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.946+0000 7ff545400640 1 -- 172.21.15.159:0/1627138897 >> v1:172.21.15.178:6789/0 conn(0x7ff548071eb0 legacy=0x7ff54806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:10.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.946+0000 7ff545400640 1 -- 172.21.15.159:0/1627138897 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54815f200 con 0x7ff548141c10 2024-04-04T06:06:10.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.946+0000 7ff544a00640 1 -- 172.21.15.159:0/1627138897 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4097258242 0 0) 0x7ff54815ec80 con 0x7ff548071eb0 2024-04-04T06:06:10.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.947+0000 7ff545400640 1 --2- 172.21.15.159:0/1627138897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff548141c10 0x7ff548071970 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff530009e50 tx=0x7ff53002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84744b1f4c10e75e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:10.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.947+0000 7ff544a00640 1 -- 172.21.15.159:0/1627138897 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53002cb00 con 0x7ff548141c10 2024-04-04T06:06:10.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.947+0000 7ff544a00640 1 -- 172.21.15.159:0/1627138897 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff53002cca0 con 0x7ff548141c10 2024-04-04T06:06:10.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.947+0000 7ff544a00640 1 -- 172.21.15.159:0/1627138897 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff530034610 con 0x7ff548141c10 2024-04-04T06:06:10.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.948+0000 7ff54ca00640 1 -- 172.21.15.159:0/1627138897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff548141c10 msgr2=0x7ff548071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:10.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.948+0000 7ff54ca00640 1 --2- 172.21.15.159:0/1627138897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff548141c10 0x7ff548071970 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff530009e50 tx=0x7ff53002bf60 comp rx=0 tx=0).stop 2024-04-04T06:06:10.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.948+0000 7ff54ca00640 1 -- 172.21.15.159:0/1627138897 shutdown_connections 2024-04-04T06:06:10.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.948+0000 7ff54ca00640 1 --2- 172.21.15.159:0/1627138897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff548141c10 0x7ff548071970 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.948+0000 7ff54ca00640 1 --2- 172.21.15.159:0/1627138897 >> v2:172.21.15.178:3300/0 conn(0x7ff5480d0280 0x7ff5480d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.948+0000 7ff54ca00640 1 -- 172.21.15.159:0/1627138897 >> 172.21.15.159:0/1627138897 conn(0x7ff54806b750 msgr2=0x7ff54806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:10.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.948+0000 7ff54ca00640 1 -- 172.21.15.159:0/1627138897 shutdown_connections 2024-04-04T06:06:10.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.948+0000 7ff54ca00640 1 -- 172.21.15.159:0/1627138897 wait complete. 2024-04-04T06:06:10.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.950+0000 7ff54ca00640 1 Processor -- start 2024-04-04T06:06:10.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.950+0000 7ff54ca00640 1 -- start start 2024-04-04T06:06:10.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.951+0000 7ff54ca00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff548071eb0 0x7ff54809c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:10.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.951+0000 7ff54ca00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5480d0280 0x7ff5481576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:10.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.951+0000 7ff54ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff548141c10 0x7ff54809aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:10.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.951+0000 7ff54ca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff548158e90 con 0x7ff548141c10 2024-04-04T06:06:10.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.951+0000 7ff54ca00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff54815f7c0 con 0x7ff5480d0280 2024-04-04T06:06:10.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.951+0000 7ff54ca00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff548099660 con 0x7ff548071eb0 2024-04-04T06:06:10.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.951+0000 7ff546800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff548141c10 0x7ff54809aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:10.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.951+0000 7ff546800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff548141c10 0x7ff54809aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37420/0 (socket says 172.21.15.159:37420) 2024-04-04T06:06:10.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.952+0000 7ff546800640 1 -- 172.21.15.159:0/1249476511 learned_addr learned my addr 172.21.15.159:0/1249476511 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:10.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.952+0000 7ff545400640 1 --2- 172.21.15.159:0/1249476511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5480d0280 0x7ff5481576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:10.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.952+0000 7ff545e00640 1 --2- 172.21.15.159:0/1249476511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff548071eb0 0x7ff54809c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:10.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.952+0000 7ff546800640 1 -- 172.21.15.159:0/1249476511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff548071eb0 msgr2=0x7ff54809c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:10.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.952+0000 7ff546800640 1 --2- 172.21.15.159:0/1249476511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff548071eb0 0x7ff54809c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.952+0000 7ff546800640 1 -- 172.21.15.159:0/1249476511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5480d0280 msgr2=0x7ff5481576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:10.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.952+0000 7ff546800640 1 --2- 172.21.15.159:0/1249476511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5480d0280 0x7ff5481576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:10.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.952+0000 7ff546800640 1 -- 172.21.15.159:0/1249476511 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54809b4a0 con 0x7ff548141c10 2024-04-04T06:06:10.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.952+0000 7ff545400640 1 --2- 172.21.15.159:0/1249476511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5480d0280 0x7ff5481576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:06:10.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.952+0000 7ff546800640 1 --2- 172.21.15.159:0/1249476511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff548141c10 0x7ff54809aba0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff54000c480 tx=0x7ff54000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:10.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.953+0000 7ff53e000640 1 -- 172.21.15.159:0/1249476511 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff540005480 con 0x7ff548141c10 2024-04-04T06:06:10.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.953+0000 7ff53e000640 1 -- 172.21.15.159:0/1249476511 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5400035f0 con 0x7ff548141c10 2024-04-04T06:06:10.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.953+0000 7ff54ca00640 1 -- 172.21.15.159:0/1249476511 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff54809b780 con 0x7ff548141c10 2024-04-04T06:06:10.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.953+0000 7ff53e000640 1 -- 172.21.15.159:0/1249476511 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff540003a00 con 0x7ff548141c10 2024-04-04T06:06:10.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.953+0000 7ff54ca00640 1 -- 172.21.15.159:0/1249476511 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff54809c020 con 0x7ff548141c10 2024-04-04T06:06:10.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.954+0000 7ff54ca00640 1 -- 172.21.15.159:0/1249476511 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5480685c0 con 0x7ff548141c10 2024-04-04T06:06:10.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.955+0000 7ff53e000640 1 -- 172.21.15.159:0/1249476511 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff540003ba0 con 0x7ff548141c10 2024-04-04T06:06:10.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.956+0000 7ff53e000640 1 --2- 172.21.15.159:0/1249476511 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff52406ca90 0x7ff52406ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:10.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.956+0000 7ff53e000640 1 -- 172.21.15.159:0/1249476511 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff54008b830 con 0x7ff548141c10 2024-04-04T06:06:10.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.956+0000 7ff545e00640 1 --2- 172.21.15.159:0/1249476511 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff52406ca90 0x7ff52406ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:10.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.957+0000 7ff545e00640 1 --2- 172.21.15.159:0/1249476511 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff52406ca90 0x7ff52406ef50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff538009ad0 tx=0x7ff538008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:10.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:10.960+0000 7ff53e000640 1 -- 172.21.15.159:0/1249476511 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff540059c50 con 0x7ff548141c10 2024-04-04T06:06:11.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.157+0000 7ff54ca00640 1 -- 172.21.15.159:0/1249476511 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff5480cfdb0 con 0x7ff548141c10 2024-04-04T06:06:11.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.157+0000 7ff53e000640 1 -- 172.21.15.159:0/1249476511 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7ff540059550 con 0x7ff548141c10 2024-04-04T06:06:11.152 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:06:11.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.160+0000 7ff54ca00640 1 -- 172.21.15.159:0/1249476511 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff52406ca90 msgr2=0x7ff52406ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:11.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.160+0000 7ff54ca00640 1 --2- 172.21.15.159:0/1249476511 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff52406ca90 0x7ff52406ef50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff538009ad0 tx=0x7ff538008040 comp rx=0 tx=0).stop 2024-04-04T06:06:11.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.160+0000 7ff54ca00640 1 -- 172.21.15.159:0/1249476511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff548141c10 msgr2=0x7ff54809aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:11.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.160+0000 7ff54ca00640 1 --2- 172.21.15.159:0/1249476511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff548141c10 0x7ff54809aba0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff54000c480 tx=0x7ff54000c940 comp rx=0 tx=0).stop 2024-04-04T06:06:11.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.160+0000 7ff54ca00640 1 -- 172.21.15.159:0/1249476511 shutdown_connections 2024-04-04T06:06:11.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.160+0000 7ff54ca00640 1 --2- 172.21.15.159:0/1249476511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff548141c10 0x7ff54809aba0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:11.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.160+0000 7ff54ca00640 1 --2- 172.21.15.159:0/1249476511 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff52406ca90 0x7ff52406ef50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:11.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.160+0000 7ff54ca00640 1 --2- 172.21.15.159:0/1249476511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5480d0280 0x7ff5481576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:11.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.160+0000 7ff54ca00640 1 --2- 172.21.15.159:0/1249476511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff548071eb0 0x7ff54809c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:11.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.160+0000 7ff54ca00640 1 -- 172.21.15.159:0/1249476511 >> 172.21.15.159:0/1249476511 conn(0x7ff54806b750 msgr2=0x7ff54806de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:11.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.160+0000 7ff54ca00640 1 -- 172.21.15.159:0/1249476511 shutdown_connections 2024-04-04T06:06:11.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.160+0000 7ff54ca00640 1 -- 172.21.15.159:0/1249476511 wait complete. 2024-04-04T06:06:11.172 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:06:11.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.379+0000 7f0fab000640 1 Processor -- start 2024-04-04T06:06:11.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.379+0000 7f0fab000640 1 -- start start 2024-04-04T06:06:11.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.380+0000 7f0fab000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0fa40ad120 0x7f0fa40d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:11.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.380+0000 7f0fab000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fa4071b60 con 0x7f0fa4161b00 2024-04-04T06:06:11.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.380+0000 7f0fab000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fa4071e40 con 0x7f0fa40ad580 2024-04-04T06:06:11.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.380+0000 7f0fab000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0fa4161ee0 con 0x7f0fa40ad120 2024-04-04T06:06:11.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.381+0000 7f0fa3e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f0fa4161b00 0x7f0fa4162ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:46010/0 (socket says 172.21.15.159:46010) 2024-04-04T06:06:11.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.381+0000 7f0fa3e00640 1 -- 172.21.15.159:0/259614029 learned_addr learned my addr 172.21.15.159:0/259614029 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:11.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.381+0000 7f0fa2a00640 1 -- 172.21.15.159:0/259614029 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3282277957 0 0) 0x7f0fa4071b60 con 0x7f0fa4161b00 2024-04-04T06:06:11.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.381+0000 7f0fa2a00640 1 -- 172.21.15.159:0/259614029 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f88003520 con 0x7f0fa4161b00 2024-04-04T06:06:11.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.381+0000 7f0fa3400640 1 --2- 172.21.15.159:0/259614029 >> v2:172.21.15.178:3300/0 conn(0x7f0fa40ad120 0x7f0fa40d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:11.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.381+0000 7f0fa2a00640 1 -- 172.21.15.159:0/259614029 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 406534956 0 0) 0x7f0f88003520 con 0x7f0fa4161b00 2024-04-04T06:06:11.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.381+0000 7f0fa2a00640 1 -- 172.21.15.159:0/259614029 >> v2:172.21.15.178:3300/0 conn(0x7f0fa40ad120 msgr2=0x7f0fa40d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:11.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.381+0000 7f0fa2a00640 1 --2- 172.21.15.159:0/259614029 >> v2:172.21.15.178:3300/0 conn(0x7f0fa40ad120 0x7f0fa40d0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:11.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.381+0000 7f0fa2a00640 1 -- 172.21.15.159:0/259614029 >> v1:172.21.15.178:6789/0 conn(0x7f0fa40ad580 legacy=0x7f0fa40b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:11.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.381+0000 7f0fa2a00640 1 -- 172.21.15.159:0/259614029 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa4072120 con 0x7f0fa4161b00 2024-04-04T06:06:11.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.381+0000 7f0fa2a00640 1 -- 172.21.15.159:0/259614029 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f0f8c001670 con 0x7f0fa4161b00 2024-04-04T06:06:11.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.381+0000 7f0fa2a00640 1 -- 172.21.15.159:0/259614029 >> v1:172.21.15.159:6789/0 conn(0x7f0fa4161b00 legacy=0x7f0fa4162ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:11.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.382+0000 7f0fa2a00640 1 --2- 172.21.15.159:0/259614029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f88003a00 0x7f0f88003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:11.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.382+0000 7f0fa2a00640 1 --2- 172.21.15.159:0/259614029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f88004800 0x7f0f88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:11.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.382+0000 7f0fa2a00640 1 --2- 172.21.15.159:0/259614029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f880052e0 0x7f0f8800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:11.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.382+0000 7f0fa2a00640 1 -- 172.21.15.159:0/259614029 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa4161ee0 con 0x7f0f88003a00 2024-04-04T06:06:11.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.382+0000 7f0fa2a00640 1 -- 172.21.15.159:0/259614029 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8800c200 con 0x7f0f880052e0 2024-04-04T06:06:11.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.382+0000 7f0fa2a00640 1 -- 172.21.15.159:0/259614029 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0f8800c400 con 0x7f0f88004800 2024-04-04T06:06:11.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.382+0000 7f0fa3e00640 1 --2- 172.21.15.159:0/259614029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f88003a00 0x7f0f88003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:11.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.382+0000 7f0fa3e00640 1 -- 172.21.15.159:0/259614029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f88004800 msgr2=0x7f0f88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:11.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.382+0000 7f0fa3e00640 1 --2- 172.21.15.159:0/259614029 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f88004800 0x7f0f88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:11.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.382+0000 7f0fa3e00640 1 -- 172.21.15.159:0/259614029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f880052e0 msgr2=0x7f0f8800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:11.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.382+0000 7f0fa9000640 1 --2- 172.21.15.159:0/259614029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f880052e0 0x7f0f8800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:11.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.382+0000 7f0fa3e00640 1 --2- 172.21.15.159:0/259614029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f880052e0 0x7f0f8800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:11.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.382+0000 7f0fa3e00640 1 -- 172.21.15.159:0/259614029 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f88003520 con 0x7f0f88003a00 2024-04-04T06:06:11.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.382+0000 7f0fa9000640 1 --2- 172.21.15.159:0/259614029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f880052e0 0x7f0f8800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:11.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.382+0000 7f0fa9000640 1 -- 172.21.15.159:0/259614029 reap_dead start 2024-04-04T06:06:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.383+0000 7f0fa3e00640 1 --2- 172.21.15.159:0/259614029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f88003a00 0x7f0f88003e20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00c110 tx=0x7f0f88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b76227803ca0e91 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.383+0000 7f0fa2a00640 1 -- 172.21.15.159:0/259614029 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c007c40 con 0x7f0f88003a00 2024-04-04T06:06:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.384+0000 7f0fa2a00640 1 -- 172.21.15.159:0/259614029 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f8c030070 con 0x7f0f88003a00 2024-04-04T06:06:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.384+0000 7f0fa2a00640 1 -- 172.21.15.159:0/259614029 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c005d70 con 0x7f0f88003a00 2024-04-04T06:06:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.384+0000 7f0fab000640 1 -- 172.21.15.159:0/259614029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f88003a00 msgr2=0x7f0f88003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.384+0000 7f0fab000640 1 --2- 172.21.15.159:0/259614029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f88003a00 0x7f0f88003e20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00c110 tx=0x7f0f88005830 comp rx=0 tx=0).stop 2024-04-04T06:06:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.384+0000 7f0fab000640 1 -- 172.21.15.159:0/259614029 shutdown_connections 2024-04-04T06:06:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.384+0000 7f0fab000640 1 --2- 172.21.15.159:0/259614029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f88003a00 0x7f0f88003e20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:11.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.384+0000 7f0fab000640 1 -- 172.21.15.159:0/259614029 >> 172.21.15.159:0/259614029 conn(0x7f0fa406b990 msgr2=0x7f0fa406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:11.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.384+0000 7f0fab000640 1 -- 172.21.15.159:0/259614029 shutdown_connections 2024-04-04T06:06:11.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.384+0000 7f0fab000640 1 -- 172.21.15.159:0/259614029 wait complete. 2024-04-04T06:06:11.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.386+0000 7f0fab000640 1 Processor -- start 2024-04-04T06:06:11.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.386+0000 7f0fab000640 1 -- start start 2024-04-04T06:06:11.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.387+0000 7f0fab000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f88003a00 0x7f0fa41549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:11.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.387+0000 7f0fab000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fa4159ac0 0x7f0fa41606d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:11.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.387+0000 7f0fab000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fa41f44e0 0x7f0fa415dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:11.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.387+0000 7f0fab000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa40b3730 con 0x7f0fa41f44e0 2024-04-04T06:06:11.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.387+0000 7f0fab000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa40b3450 con 0x7f0f88003a00 2024-04-04T06:06:11.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.387+0000 7f0fab000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0fa40b5020 con 0x7f0fa4159ac0 2024-04-04T06:06:11.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.387+0000 7f0fa9000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fa41f44e0 0x7f0fa415dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:11.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.387+0000 7f0fa9000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fa41f44e0 0x7f0fa415dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37432/0 (socket says 172.21.15.159:37432) 2024-04-04T06:06:11.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.387+0000 7f0fa9000640 1 -- 172.21.15.159:0/3211054490 learned_addr learned my addr 172.21.15.159:0/3211054490 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:11.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.387+0000 7f0fa3e00640 1 --2- 172.21.15.159:0/3211054490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f88003a00 0x7f0fa41549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:11.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.387+0000 7f0fa3400640 1 --2- 172.21.15.159:0/3211054490 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fa4159ac0 0x7f0fa41606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:11.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.388+0000 7f0fa9000640 1 -- 172.21.15.159:0/3211054490 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fa4159ac0 msgr2=0x7f0fa41606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:11.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.388+0000 7f0fa9000640 1 --2- 172.21.15.159:0/3211054490 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fa4159ac0 0x7f0fa41606d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:11.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.388+0000 7f0fa9000640 1 -- 172.21.15.159:0/3211054490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f88003a00 msgr2=0x7f0fa41549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:11.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.388+0000 7f0fa9000640 1 --2- 172.21.15.159:0/3211054490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f88003a00 0x7f0fa41549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:11.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.388+0000 7f0fa9000640 1 -- 172.21.15.159:0/3211054490 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa41577e0 con 0x7f0fa41f44e0 2024-04-04T06:06:11.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.388+0000 7f0fa9000640 1 --2- 172.21.15.159:0/3211054490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fa41f44e0 0x7f0fa415dbd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0fa406fd00 tx=0x7f0f9c00d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:11.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.388+0000 7f0f9be00640 1 -- 172.21.15.159:0/3211054490 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c018070 con 0x7f0fa41f44e0 2024-04-04T06:06:11.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.388+0000 7f0f9be00640 1 -- 172.21.15.159:0/3211054490 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f9c003da0 con 0x7f0fa41f44e0 2024-04-04T06:06:11.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.388+0000 7f0f9be00640 1 -- 172.21.15.159:0/3211054490 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c005310 con 0x7f0fa41f44e0 2024-04-04T06:06:11.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.388+0000 7f0fab000640 1 -- 172.21.15.159:0/3211054490 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa4161040 con 0x7f0fa41f44e0 2024-04-04T06:06:11.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.388+0000 7f0fab000640 1 -- 172.21.15.159:0/3211054490 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa4160c10 con 0x7f0fa41f44e0 2024-04-04T06:06:11.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.389+0000 7f0fab000640 1 -- 172.21.15.159:0/3211054490 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa40685c0 con 0x7f0fa41f44e0 2024-04-04T06:06:11.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.390+0000 7f0f9be00640 1 -- 172.21.15.159:0/3211054490 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f9c01b070 con 0x7f0fa41f44e0 2024-04-04T06:06:11.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.391+0000 7f0f9be00640 1 --2- 172.21.15.159:0/3211054490 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f8006c8f0 0x7f0f8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:11.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.391+0000 7f0f9be00640 1 -- 172.21.15.159:0/3211054490 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0f9c08b320 con 0x7f0fa41f44e0 2024-04-04T06:06:11.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.392+0000 7f0fa3e00640 1 --2- 172.21.15.159:0/3211054490 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f8006c8f0 0x7f0f8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:11.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.393+0000 7f0fa3e00640 1 --2- 172.21.15.159:0/3211054490 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f8006c8f0 0x7f0f8006edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c034000 tx=0x7f0f8c009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:11.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.396+0000 7f0f9be00640 1 -- 172.21.15.159:0/3211054490 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0f9c059750 con 0x7f0fa41f44e0 2024-04-04T06:06:11.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.618+0000 7f0fab000640 1 -- 172.21.15.159:0/3211054490 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fa4071b60 con 0x7f0fa41f44e0 2024-04-04T06:06:11.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.619+0000 7f0f9be00640 1 -- 172.21.15.159:0/3211054490 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5184 (secure 0 0 0) 0x7f0f9c059050 con 0x7f0fa41f44e0 2024-04-04T06:06:11.614 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:11.614 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/789759868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":789759868},{"type":"v1","addr":"172.21.15.178:6836","nonce":789759868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2438288045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2438288045},{"type":"v1","addr":"172.21.15.159:6833","nonce":2438288045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/1705205607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1705205607},{"type":"v1","addr":"172.21.15.159:6835","nonce":1705205607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:05:52.801529+0000","modified":"2024-04-04T06:05:55.866207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6837/3362141613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":3362141613},{"type":"v1","addr":"172.21.15.178:6837","nonce":3362141613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-04T06:06:11.615 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 8 2024-04-04T06:06:11.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.622+0000 7f0fab000640 1 -- 172.21.15.159:0/3211054490 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f8006c8f0 msgr2=0x7f0f8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:11.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.622+0000 7f0fab000640 1 --2- 172.21.15.159:0/3211054490 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f8006c8f0 0x7f0f8006edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c034000 tx=0x7f0f8c009030 comp rx=0 tx=0).stop 2024-04-04T06:06:11.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.622+0000 7f0fab000640 1 -- 172.21.15.159:0/3211054490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fa41f44e0 msgr2=0x7f0fa415dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:11.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.622+0000 7f0fab000640 1 --2- 172.21.15.159:0/3211054490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fa41f44e0 0x7f0fa415dbd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0fa406fd00 tx=0x7f0f9c00d900 comp rx=0 tx=0).stop 2024-04-04T06:06:11.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.622+0000 7f0fab000640 1 -- 172.21.15.159:0/3211054490 shutdown_connections 2024-04-04T06:06:11.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.622+0000 7f0fab000640 1 --2- 172.21.15.159:0/3211054490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fa41f44e0 0x7f0fa415dbd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:11.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.622+0000 7f0fab000640 1 --2- 172.21.15.159:0/3211054490 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fa4159ac0 0x7f0fa41606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:11.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.622+0000 7f0fab000640 1 --2- 172.21.15.159:0/3211054490 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f8006c8f0 0x7f0f8006edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:11.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.622+0000 7f0fab000640 1 --2- 172.21.15.159:0/3211054490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f88003a00 0x7f0fa41549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:11.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.622+0000 7f0fab000640 1 -- 172.21.15.159:0/3211054490 >> 172.21.15.159:0/3211054490 conn(0x7f0fa406b990 msgr2=0x7f0fa40aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:11.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.622+0000 7f0fab000640 1 -- 172.21.15.159:0/3211054490 shutdown_connections 2024-04-04T06:06:11.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.622+0000 7f0fab000640 1 -- 172.21.15.159:0/3211054490 wait complete. 2024-04-04T06:06:11.634 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:06:11.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.846+0000 7eff3fc00640 1 Processor -- start 2024-04-04T06:06:11.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.846+0000 7eff3fc00640 1 -- start start 2024-04-04T06:06:11.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.848+0000 7eff3fc00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7eff3815c390 0x7eff38073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:11.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.848+0000 7eff3fc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff380d0650 0x7eff380d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:11.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.848+0000 7eff3fc00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff3815fe80 con 0x7eff380af830 2024-04-04T06:06:11.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.848+0000 7eff3fc00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7eff3815c770 con 0x7eff3815c390 2024-04-04T06:06:11.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.848+0000 7eff3fc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7eff380ad120 con 0x7eff380d0650 2024-04-04T06:06:11.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.848+0000 7eff37e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7eff380af830 0x7eff38160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:46016/0 (socket says 172.21.15.159:46016) 2024-04-04T06:06:11.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.848+0000 7eff37e00640 1 -- 172.21.15.159:0/3333258796 learned_addr learned my addr 172.21.15.159:0/3333258796 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:11.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.848+0000 7eff3dc00640 1 --2- 172.21.15.159:0/3333258796 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff380d0650 0x7eff380d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:11.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.848+0000 7eff3d200640 1 --2- 172.21.15.159:0/3333258796 >> v2:172.21.15.178:3300/0 conn(0x7eff3815c390 0x7eff38073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:11.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.849+0000 7eff37400640 1 -- 172.21.15.159:0/3333258796 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 766132921 0 0) 0x7eff3815fe80 con 0x7eff380af830 2024-04-04T06:06:11.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.849+0000 7eff37400640 1 -- 172.21.15.159:0/3333258796 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff20003520 con 0x7eff380af830 2024-04-04T06:06:11.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.849+0000 7eff37400640 1 -- 172.21.15.159:0/3333258796 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 239488208 0 0) 0x7eff20003520 con 0x7eff380af830 2024-04-04T06:06:11.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.849+0000 7eff37400640 1 -- 172.21.15.159:0/3333258796 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff380d0650 msgr2=0x7eff380d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:11.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.849+0000 7eff37400640 1 --2- 172.21.15.159:0/3333258796 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff380d0650 0x7eff380d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:11.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.849+0000 7eff37400640 1 -- 172.21.15.159:0/3333258796 >> v2:172.21.15.178:3300/0 conn(0x7eff3815c390 msgr2=0x7eff38073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:11.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.849+0000 7eff37400640 1 --2- 172.21.15.159:0/3333258796 >> v2:172.21.15.178:3300/0 conn(0x7eff3815c390 0x7eff38073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:11.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.849+0000 7eff37400640 1 -- 172.21.15.159:0/3333258796 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff380ad400 con 0x7eff380af830 2024-04-04T06:06:11.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.849+0000 7eff37400640 1 -- 172.21.15.159:0/3333258796 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7eff24001670 con 0x7eff380af830 2024-04-04T06:06:11.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.849+0000 7eff37400640 1 -- 172.21.15.159:0/3333258796 >> v1:172.21.15.159:6789/0 conn(0x7eff380af830 legacy=0x7eff38160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:11.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.849+0000 7eff37400640 1 --2- 172.21.15.159:0/3333258796 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff380d0650 0x7eff200039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:11.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.849+0000 7eff37400640 1 --2- 172.21.15.159:0/3333258796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff20003f30 0x7eff20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:11.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.849+0000 7eff37400640 1 --2- 172.21.15.159:0/3333258796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff20004d40 0x7eff200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:11.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.849+0000 7eff37400640 1 -- 172.21.15.159:0/3333258796 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7eff380ad120 con 0x7eff20004d40 2024-04-04T06:06:11.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.849+0000 7eff37400640 1 -- 172.21.15.159:0/3333258796 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7eff2000b9c0 con 0x7eff20003f30 2024-04-04T06:06:11.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.849+0000 7eff37400640 1 -- 172.21.15.159:0/3333258796 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7eff2000bb40 con 0x7eff380d0650 2024-04-04T06:06:11.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.849+0000 7eff3dc00640 1 --2- 172.21.15.159:0/3333258796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff20004d40 0x7eff200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:11.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.850+0000 7eff3d200640 1 --2- 172.21.15.159:0/3333258796 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff380d0650 0x7eff200039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:11.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.850+0000 7eff3dc00640 1 -- 172.21.15.159:0/3333258796 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff380d0650 msgr2=0x7eff200039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:11.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.850+0000 7eff3dc00640 1 --2- 172.21.15.159:0/3333258796 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff380d0650 0x7eff200039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:11.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.850+0000 7eff3dc00640 1 -- 172.21.15.159:0/3333258796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff20003f30 msgr2=0x7eff20004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:11.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.850+0000 7eff37e00640 1 --2- 172.21.15.159:0/3333258796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff20003f30 0x7eff20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:11.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.850+0000 7eff3dc00640 1 --2- 172.21.15.159:0/3333258796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff20003f30 0x7eff20004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:11.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.850+0000 7eff3dc00640 1 -- 172.21.15.159:0/3333258796 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff20003520 con 0x7eff20004d40 2024-04-04T06:06:11.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.851+0000 7eff3dc00640 1 --2- 172.21.15.159:0/3333258796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff20004d40 0x7eff200051c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7eff30009fc0 tx=0x7eff20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6c8c4a412e331d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:11.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.851+0000 7eff37400640 1 -- 172.21.15.159:0/3333258796 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30039070 con 0x7eff20004d40 2024-04-04T06:06:11.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.851+0000 7eff37400640 1 -- 172.21.15.159:0/3333258796 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff3002cc00 con 0x7eff20004d40 2024-04-04T06:06:11.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.851+0000 7eff37400640 1 -- 172.21.15.159:0/3333258796 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30009c10 con 0x7eff20004d40 2024-04-04T06:06:11.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.852+0000 7eff3fc00640 1 -- 172.21.15.159:0/3333258796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff20004d40 msgr2=0x7eff200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:11.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.852+0000 7eff3fc00640 1 --2- 172.21.15.159:0/3333258796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff20004d40 0x7eff200051c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7eff30009fc0 tx=0x7eff20005830 comp rx=0 tx=0).stop 2024-04-04T06:06:11.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.852+0000 7eff3dc00640 1 -- 172.21.15.159:0/3333258796 reap_dead start 2024-04-04T06:06:11.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.852+0000 7eff3fc00640 1 -- 172.21.15.159:0/3333258796 shutdown_connections 2024-04-04T06:06:11.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.852+0000 7eff3fc00640 1 -- 172.21.15.159:0/3333258796 >> 172.21.15.159:0/3333258796 conn(0x7eff3806b990 msgr2=0x7eff3806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:11.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.852+0000 7eff3fc00640 1 -- 172.21.15.159:0/3333258796 shutdown_connections 2024-04-04T06:06:11.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.852+0000 7eff3fc00640 1 -- 172.21.15.159:0/3333258796 wait complete. 2024-04-04T06:06:11.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.854+0000 7eff3fc00640 1 Processor -- start 2024-04-04T06:06:11.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.854+0000 7eff3fc00640 1 -- start start 2024-04-04T06:06:11.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.855+0000 7eff3fc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff20004d40 0x7eff381f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:11.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.855+0000 7eff3fc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff3815f1a0 0x7eff380b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:11.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.855+0000 7eff3fc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff38152900 0x7eff3815a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:11.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.855+0000 7eff3fc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7eff38160d50 con 0x7eff38152900 2024-04-04T06:06:11.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.855+0000 7eff3fc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7eff38160120 con 0x7eff3815f1a0 2024-04-04T06:06:11.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.855+0000 7eff3fc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7eff380cf980 con 0x7eff20004d40 2024-04-04T06:06:11.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.855+0000 7eff3dc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff38152900 0x7eff3815a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:11.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.855+0000 7eff3dc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff38152900 0x7eff3815a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37450/0 (socket says 172.21.15.159:37450) 2024-04-04T06:06:11.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.855+0000 7eff3dc00640 1 -- 172.21.15.159:0/3148567422 learned_addr learned my addr 172.21.15.159:0/3148567422 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:11.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.855+0000 7eff3d200640 1 --2- 172.21.15.159:0/3148567422 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff20004d40 0x7eff381f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:11.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.855+0000 7eff37e00640 1 --2- 172.21.15.159:0/3148567422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff3815f1a0 0x7eff380b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:11.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.855+0000 7eff3dc00640 1 -- 172.21.15.159:0/3148567422 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff20004d40 msgr2=0x7eff381f7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:11.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.855+0000 7eff3dc00640 1 --2- 172.21.15.159:0/3148567422 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff20004d40 0x7eff381f7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:11.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.855+0000 7eff3dc00640 1 -- 172.21.15.159:0/3148567422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff3815f1a0 msgr2=0x7eff380b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:11.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.855+0000 7eff3dc00640 1 --2- 172.21.15.159:0/3148567422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff3815f1a0 0x7eff380b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:11.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.855+0000 7eff3dc00640 1 -- 172.21.15.159:0/3148567422 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff3815f620 con 0x7eff38152900 2024-04-04T06:06:11.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.855+0000 7eff3d200640 1 --2- 172.21.15.159:0/3148567422 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff20004d40 0x7eff381f7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:11.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.856+0000 7eff3dc00640 1 --2- 172.21.15.159:0/3148567422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff38152900 0x7eff3815a570 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7eff3002c6d0 tx=0x7eff30009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:11.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.856+0000 7eff34c00640 1 -- 172.21.15.159:0/3148567422 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30039070 con 0x7eff38152900 2024-04-04T06:06:11.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.856+0000 7eff34c00640 1 -- 172.21.15.159:0/3148567422 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff3003a040 con 0x7eff38152900 2024-04-04T06:06:11.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.856+0000 7eff34c00640 1 -- 172.21.15.159:0/3148567422 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30003990 con 0x7eff38152900 2024-04-04T06:06:11.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.856+0000 7eff3fc00640 1 -- 172.21.15.159:0/3148567422 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff380b3780 con 0x7eff38152900 2024-04-04T06:06:11.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.856+0000 7eff3fc00640 1 -- 172.21.15.159:0/3148567422 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff381f8850 con 0x7eff38152900 2024-04-04T06:06:11.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.857+0000 7eff3fc00640 1 -- 172.21.15.159:0/3148567422 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff380685c0 con 0x7eff38152900 2024-04-04T06:06:11.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.858+0000 7eff34c00640 1 -- 172.21.15.159:0/3148567422 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff30046030 con 0x7eff38152900 2024-04-04T06:06:11.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.859+0000 7eff34c00640 1 --2- 172.21.15.159:0/3148567422 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7eff1806c8f0 0x7eff1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:11.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.860+0000 7eff34c00640 1 -- 172.21.15.159:0/3148567422 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7eff300ada40 con 0x7eff38152900 2024-04-04T06:06:11.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.860+0000 7eff3d200640 1 --2- 172.21.15.159:0/3148567422 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7eff1806c8f0 0x7eff1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:11.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.861+0000 7eff3d200640 1 --2- 172.21.15.159:0/3148567422 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7eff1806c8f0 0x7eff1806edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009a60 tx=0x7eff2c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:11.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:11.865+0000 7eff34c00640 1 -- 172.21.15.159:0/3148567422 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7eff3004d020 con 0x7eff38152900 2024-04-04T06:06:12.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.059+0000 7eff3fc00640 1 -- 172.21.15.159:0/3148567422 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff3806f1b0 con 0x7eff38152900 2024-04-04T06:06:12.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.061+0000 7eff34c00640 1 -- 172.21.15.159:0/3148567422 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7eff3815f620 con 0x7eff38152900 2024-04-04T06:06:12.056 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:12.056 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":21,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:05:54.347275+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:05:51.297293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-04T06:05:51.770792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:06:12.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.063+0000 7eff3fc00640 1 -- 172.21.15.159:0/3148567422 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7eff1806c8f0 msgr2=0x7eff1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:12.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.063+0000 7eff3fc00640 1 --2- 172.21.15.159:0/3148567422 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7eff1806c8f0 0x7eff1806edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009a60 tx=0x7eff2c008040 comp rx=0 tx=0).stop 2024-04-04T06:06:12.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.063+0000 7eff3fc00640 1 -- 172.21.15.159:0/3148567422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff38152900 msgr2=0x7eff3815a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:12.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.063+0000 7eff3fc00640 1 --2- 172.21.15.159:0/3148567422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff38152900 0x7eff3815a570 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7eff3002c6d0 tx=0x7eff30009750 comp rx=0 tx=0).stop 2024-04-04T06:06:12.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.064+0000 7eff3fc00640 1 -- 172.21.15.159:0/3148567422 shutdown_connections 2024-04-04T06:06:12.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.064+0000 7eff3fc00640 1 --2- 172.21.15.159:0/3148567422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff38152900 0x7eff3815a570 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.064+0000 7eff3fc00640 1 --2- 172.21.15.159:0/3148567422 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7eff1806c8f0 0x7eff1806edb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.064+0000 7eff3fc00640 1 --2- 172.21.15.159:0/3148567422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff3815f1a0 0x7eff380b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.064+0000 7eff3fc00640 1 --2- 172.21.15.159:0/3148567422 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff20004d40 0x7eff381f7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.064+0000 7eff3fc00640 1 -- 172.21.15.159:0/3148567422 >> 172.21.15.159:0/3148567422 conn(0x7eff3806b990 msgr2=0x7eff380afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:12.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.064+0000 7eff3fc00640 1 -- 172.21.15.159:0/3148567422 shutdown_connections 2024-04-04T06:06:12.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.064+0000 7eff3fc00640 1 -- 172.21.15.159:0/3148567422 wait complete. 2024-04-04T06:06:12.077 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:05:51.297293+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:05:51.770792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:06:12.077 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:06:12.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.286+0000 7f3064000640 1 Processor -- start 2024-04-04T06:06:12.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.286+0000 7f3064000640 1 -- start start 2024-04-04T06:06:12.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.287+0000 7f3064000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f305c0af4b0 0x7f305c167460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:12.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.287+0000 7f3064000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f305c0afab0 con 0x7f305c0d0990 2024-04-04T06:06:12.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.287+0000 7f3064000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f305c15cc80 con 0x7f305c153580 2024-04-04T06:06:12.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.287+0000 7f3064000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f305c15da00 con 0x7f305c0af4b0 2024-04-04T06:06:12.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.287+0000 7f3060c00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f305c0d0990 0x7f305c0aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:46026/0 (socket says 172.21.15.159:46026) 2024-04-04T06:06:12.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.287+0000 7f3060c00640 1 -- 172.21.15.159:0/1554611368 learned_addr learned my addr 172.21.15.159:0/1554611368 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:12.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.288+0000 7f3062000640 1 --2- 172.21.15.159:0/1554611368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f305c0af4b0 0x7f305c167460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:12.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.288+0000 7f305be00640 1 -- 172.21.15.159:0/1554611368 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2587551148 0 0) 0x7f305c0afab0 con 0x7f305c0d0990 2024-04-04T06:06:12.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.288+0000 7f305be00640 1 -- 172.21.15.159:0/1554611368 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3044003520 con 0x7f305c0d0990 2024-04-04T06:06:12.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.288+0000 7f305be00640 1 -- 172.21.15.159:0/1554611368 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1128578763 0 0) 0x7f3044003520 con 0x7f305c0d0990 2024-04-04T06:06:12.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.288+0000 7f305be00640 1 -- 172.21.15.159:0/1554611368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f305c0af4b0 msgr2=0x7f305c167460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:12.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.288+0000 7f305be00640 1 --2- 172.21.15.159:0/1554611368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f305c0af4b0 0x7f305c167460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.288+0000 7f305be00640 1 -- 172.21.15.159:0/1554611368 >> v1:172.21.15.178:6789/0 conn(0x7f305c153580 legacy=0x7f305c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:12.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.288+0000 7f305be00640 1 -- 172.21.15.159:0/1554611368 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305c15cf20 con 0x7f305c0d0990 2024-04-04T06:06:12.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.288+0000 7f305be00640 1 -- 172.21.15.159:0/1554611368 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f3048001670 con 0x7f305c0d0990 2024-04-04T06:06:12.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.288+0000 7f305be00640 1 -- 172.21.15.159:0/1554611368 >> v1:172.21.15.159:6789/0 conn(0x7f305c0d0990 legacy=0x7f305c0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:12.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.288+0000 7f305be00640 1 --2- 172.21.15.159:0/1554611368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30440039e0 0x7f3044003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:12.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.288+0000 7f305be00640 1 --2- 172.21.15.159:0/1554611368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f305c0af4b0 0x7f3044004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:12.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.288+0000 7f305be00640 1 --2- 172.21.15.159:0/1554611368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3044004d40 0x7f30440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:12.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.288+0000 7f305be00640 1 -- 172.21.15.159:0/1554611368 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f305c15da00 con 0x7f3044004d40 2024-04-04T06:06:12.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.288+0000 7f305be00640 1 -- 172.21.15.159:0/1554611368 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f304400bb50 con 0x7f30440039e0 2024-04-04T06:06:12.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.288+0000 7f305be00640 1 -- 172.21.15.159:0/1554611368 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f304400bd50 con 0x7f305c0af4b0 2024-04-04T06:06:12.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.288+0000 7f3062000640 1 --2- 172.21.15.159:0/1554611368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3044004d40 0x7f30440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:12.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.289+0000 7f3061600640 1 --2- 172.21.15.159:0/1554611368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30440039e0 0x7f3044003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:12.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.289+0000 7f3062000640 1 -- 172.21.15.159:0/1554611368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f305c0af4b0 msgr2=0x7f3044004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:12.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.289+0000 7f3060c00640 1 --2- 172.21.15.159:0/1554611368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f305c0af4b0 0x7f3044004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:12.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.289+0000 7f3062000640 1 --2- 172.21.15.159:0/1554611368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f305c0af4b0 0x7f3044004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.289+0000 7f3062000640 1 -- 172.21.15.159:0/1554611368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30440039e0 msgr2=0x7f3044003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:12.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.289+0000 7f3062000640 1 --2- 172.21.15.159:0/1554611368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30440039e0 0x7f3044003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.289+0000 7f3062000640 1 -- 172.21.15.159:0/1554611368 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3044003520 con 0x7f3044004d40 2024-04-04T06:06:12.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.290+0000 7f3062000640 1 --2- 172.21.15.159:0/1554611368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3044004d40 0x7f30440051c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3054009fc0 tx=0x7f3044005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd89834fa7f5bb54 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:12.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.290+0000 7f305be00640 1 -- 172.21.15.159:0/1554611368 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054039070 con 0x7f3044004d40 2024-04-04T06:06:12.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.290+0000 7f305be00640 1 -- 172.21.15.159:0/1554611368 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f305402cc20 con 0x7f3044004d40 2024-04-04T06:06:12.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.290+0000 7f305be00640 1 -- 172.21.15.159:0/1554611368 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054009c10 con 0x7f3044004d40 2024-04-04T06:06:12.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.290+0000 7f3064000640 1 -- 172.21.15.159:0/1554611368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3044004d40 msgr2=0x7f30440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:12.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.290+0000 7f3064000640 1 --2- 172.21.15.159:0/1554611368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3044004d40 0x7f30440051c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3054009fc0 tx=0x7f3044005830 comp rx=0 tx=0).stop 2024-04-04T06:06:12.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.291+0000 7f3062000640 1 -- 172.21.15.159:0/1554611368 reap_dead start 2024-04-04T06:06:12.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.291+0000 7f3064000640 1 -- 172.21.15.159:0/1554611368 shutdown_connections 2024-04-04T06:06:12.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.291+0000 7f3064000640 1 -- 172.21.15.159:0/1554611368 >> 172.21.15.159:0/1554611368 conn(0x7f305c06b750 msgr2=0x7f305c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:12.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.291+0000 7f3064000640 1 -- 172.21.15.159:0/1554611368 shutdown_connections 2024-04-04T06:06:12.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.291+0000 7f3064000640 1 -- 172.21.15.159:0/1554611368 wait complete. 2024-04-04T06:06:12.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.292+0000 7f3064000640 1 Processor -- start 2024-04-04T06:06:12.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.293+0000 7f3064000640 1 -- start start 2024-04-04T06:06:12.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.293+0000 7f3064000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3044004d40 0x7f305c14bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:12.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.293+0000 7f3064000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f305c072020 0x7f305c0cf100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:12.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.293+0000 7f3064000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f305c0cf640 0x7f305c15b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:12.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.294+0000 7f3064000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f305c153960 con 0x7f3044004d40 2024-04-04T06:06:12.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.294+0000 7f3064000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f305c15d4e0 con 0x7f305c072020 2024-04-04T06:06:12.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.294+0000 7f3064000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f305c165450 con 0x7f305c0cf640 2024-04-04T06:06:12.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.294+0000 7f3061600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3044004d40 0x7f305c14bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:12.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.294+0000 7f3061600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3044004d40 0x7f305c14bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37472/0 (socket says 172.21.15.159:37472) 2024-04-04T06:06:12.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.294+0000 7f3061600640 1 -- 172.21.15.159:0/202917289 learned_addr learned my addr 172.21.15.159:0/202917289 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:12.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.294+0000 7f3060c00640 1 --2- 172.21.15.159:0/202917289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f305c072020 0x7f305c0cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:12.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.294+0000 7f3061600640 1 -- 172.21.15.159:0/202917289 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f305c0cf640 msgr2=0x7f305c15b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:12.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.294+0000 7f3061600640 1 --2- 172.21.15.159:0/202917289 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f305c0cf640 0x7f305c15b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.294+0000 7f3061600640 1 -- 172.21.15.159:0/202917289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f305c072020 msgr2=0x7f305c0cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:12.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.294+0000 7f3061600640 1 --2- 172.21.15.159:0/202917289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f305c072020 0x7f305c0cf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.294+0000 7f3061600640 1 -- 172.21.15.159:0/202917289 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305c072ef0 con 0x7f3044004d40 2024-04-04T06:06:12.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.294+0000 7f3061600640 1 --2- 172.21.15.159:0/202917289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3044004d40 0x7f305c14bea0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f305000e4d0 tx=0x7f305000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:12.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.294+0000 7f3059600640 1 -- 172.21.15.159:0/202917289 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30500052e0 con 0x7f3044004d40 2024-04-04T06:06:12.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.295+0000 7f3059600640 1 -- 172.21.15.159:0/202917289 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3050007020 con 0x7f3044004d40 2024-04-04T06:06:12.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.295+0000 7f3059600640 1 -- 172.21.15.159:0/202917289 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30500071f0 con 0x7f3044004d40 2024-04-04T06:06:12.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.295+0000 7f3064000640 1 -- 172.21.15.159:0/202917289 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305c0731d0 con 0x7f3044004d40 2024-04-04T06:06:12.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.295+0000 7f3064000640 1 -- 172.21.15.159:0/202917289 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f305c0ce900 con 0x7f3044004d40 2024-04-04T06:06:12.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.296+0000 7f3064000640 1 -- 172.21.15.159:0/202917289 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f305c0685c0 con 0x7f3044004d40 2024-04-04T06:06:12.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.296+0000 7f3059600640 1 -- 172.21.15.159:0/202917289 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f305001a020 con 0x7f3044004d40 2024-04-04T06:06:12.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.297+0000 7f3059600640 1 --2- 172.21.15.159:0/202917289 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f303c06c920 0x7f303c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:12.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.297+0000 7f3059600640 1 -- 172.21.15.159:0/202917289 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f305008b110 con 0x7f3044004d40 2024-04-04T06:06:12.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.298+0000 7f3060c00640 1 --2- 172.21.15.159:0/202917289 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f303c06c920 0x7f303c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:12.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.299+0000 7f3060c00640 1 --2- 172.21.15.159:0/202917289 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f303c06c920 0x7f303c06ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f305c14cc00 tx=0x7f3048009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:12.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.301+0000 7f3059600640 1 -- 172.21.15.159:0/202917289 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3050059530 con 0x7f3044004d40 2024-04-04T06:06:12.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.497+0000 7f3064000640 1 -- 172.21.15.159:0/202917289 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f305c0cc5d0 con 0x7f3044004d40 2024-04-04T06:06:12.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.498+0000 7f3059600640 1 -- 172.21.15.159:0/202917289 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f3050058e30 con 0x7f3044004d40 2024-04-04T06:06:12.493 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:06:12.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.500+0000 7f3064000640 1 -- 172.21.15.159:0/202917289 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f303c06c920 msgr2=0x7f303c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:12.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.500+0000 7f3064000640 1 --2- 172.21.15.159:0/202917289 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f303c06c920 0x7f303c06ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f305c14cc00 tx=0x7f3048009040 comp rx=0 tx=0).stop 2024-04-04T06:06:12.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.500+0000 7f3064000640 1 -- 172.21.15.159:0/202917289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3044004d40 msgr2=0x7f305c14bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:12.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.500+0000 7f3064000640 1 --2- 172.21.15.159:0/202917289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3044004d40 0x7f305c14bea0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f305000e4d0 tx=0x7f305000e990 comp rx=0 tx=0).stop 2024-04-04T06:06:12.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.501+0000 7f3064000640 1 -- 172.21.15.159:0/202917289 shutdown_connections 2024-04-04T06:06:12.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.501+0000 7f3064000640 1 --2- 172.21.15.159:0/202917289 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f305c0cf640 0x7f305c15b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.501+0000 7f3064000640 1 --2- 172.21.15.159:0/202917289 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f303c06c920 0x7f303c06ede0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.501+0000 7f3064000640 1 --2- 172.21.15.159:0/202917289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f305c072020 0x7f305c0cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.501+0000 7f3064000640 1 --2- 172.21.15.159:0/202917289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3044004d40 0x7f305c14bea0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.501+0000 7f3064000640 1 -- 172.21.15.159:0/202917289 >> 172.21.15.159:0/202917289 conn(0x7f305c06b750 msgr2=0x7f305c0afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:12.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.501+0000 7f3064000640 1 -- 172.21.15.159:0/202917289 shutdown_connections 2024-04-04T06:06:12.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.501+0000 7f3064000640 1 -- 172.21.15.159:0/202917289 wait complete. 2024-04-04T06:06:12.514 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:06:12.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.726+0000 7f05e9400640 1 Processor -- start 2024-04-04T06:06:12.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.727+0000 7f05e9400640 1 -- start start 2024-04-04T06:06:12.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.728+0000 7f05e9400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f05e4153580 0x7f05e40d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:12.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.728+0000 7f05e9400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f05e40d07c0 0x7f05e40aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:12.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.728+0000 7f05e9400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05e40afab0 con 0x7f05e40af2e0 2024-04-04T06:06:12.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.728+0000 7f05e9400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f05e40d0c30 con 0x7f05e40d07c0 2024-04-04T06:06:12.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.728+0000 7f05e9400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f05e415cc80 con 0x7f05e4153580 2024-04-04T06:06:12.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.728+0000 7f05dbe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f05e40d07c0 0x7f05e40aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:12.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.729+0000 7f05dbe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f05e40d07c0 0x7f05e40aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37482/0 (socket says 172.21.15.159:37482) 2024-04-04T06:06:12.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.729+0000 7f05dbe00640 1 -- 172.21.15.159:0/2121807392 learned_addr learned my addr 172.21.15.159:0/2121807392 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:12.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.729+0000 7f05e2800640 1 --2- 172.21.15.159:0/2121807392 >> v2:172.21.15.178:3300/0 conn(0x7f05e4153580 0x7f05e40d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:12.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.729+0000 7f05dbe00640 1 -- 172.21.15.159:0/2121807392 >> v2:172.21.15.178:3300/0 conn(0x7f05e4153580 msgr2=0x7f05e40d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:12.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.729+0000 7f05dbe00640 1 --2- 172.21.15.159:0/2121807392 >> v2:172.21.15.178:3300/0 conn(0x7f05e4153580 0x7f05e40d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.729+0000 7f05dbe00640 1 -- 172.21.15.159:0/2121807392 >> v1:172.21.15.178:6789/0 conn(0x7f05e40af2e0 legacy=0x7f05e406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:12.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.729+0000 7f05dbe00640 1 -- 172.21.15.159:0/2121807392 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e415da00 con 0x7f05e40d07c0 2024-04-04T06:06:12.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.730+0000 7f05dbe00640 1 --2- 172.21.15.159:0/2121807392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05e40d07c0 0x7f05e40aeda0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f05e40b5270 tx=0x7f05cc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9df51e6e1839d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:12.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.730+0000 7f05e1e00640 1 -- 172.21.15.159:0/2121807392 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc035020 con 0x7f05e40d07c0 2024-04-04T06:06:12.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.730+0000 7f05e1e00640 1 -- 172.21.15.159:0/2121807392 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05cc031870 con 0x7f05e40d07c0 2024-04-04T06:06:12.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.730+0000 7f05e1e00640 1 -- 172.21.15.159:0/2121807392 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc031c40 con 0x7f05e40d07c0 2024-04-04T06:06:12.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.731+0000 7f05e9400640 1 -- 172.21.15.159:0/2121807392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05e40d07c0 msgr2=0x7f05e40aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:12.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.731+0000 7f05e9400640 1 --2- 172.21.15.159:0/2121807392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05e40d07c0 0x7f05e40aeda0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f05e40b5270 tx=0x7f05cc02b910 comp rx=0 tx=0).stop 2024-04-04T06:06:12.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.731+0000 7f05e9400640 1 -- 172.21.15.159:0/2121807392 shutdown_connections 2024-04-04T06:06:12.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.731+0000 7f05e9400640 1 --2- 172.21.15.159:0/2121807392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05e40d07c0 0x7f05e40aeda0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.731+0000 7f05e9400640 1 --2- 172.21.15.159:0/2121807392 >> v2:172.21.15.178:3300/0 conn(0x7f05e4153580 0x7f05e40d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.731+0000 7f05e9400640 1 -- 172.21.15.159:0/2121807392 >> 172.21.15.159:0/2121807392 conn(0x7f05e406b750 msgr2=0x7f05e406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:12.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.731+0000 7f05e9400640 1 -- 172.21.15.159:0/2121807392 shutdown_connections 2024-04-04T06:06:12.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.731+0000 7f05e9400640 1 -- 172.21.15.159:0/2121807392 wait complete. 2024-04-04T06:06:12.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.733+0000 7f05e9400640 1 Processor -- start 2024-04-04T06:06:12.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.733+0000 7f05e9400640 1 -- start start 2024-04-04T06:06:12.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.734+0000 7f05e9400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05e40af2e0 0x7f05e4072470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.734+0000 7f05e9400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05e40d07c0 0x7f05e40c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.734+0000 7f05e9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05e4153580 0x7f05e415b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.734+0000 7f05e9400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f05e415d4e0 con 0x7f05e4153580 2024-04-04T06:06:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.734+0000 7f05e9400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f05e415d200 con 0x7f05e40d07c0 2024-04-04T06:06:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.734+0000 7f05e9400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f05e4153960 con 0x7f05e40af2e0 2024-04-04T06:06:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.734+0000 7f05e3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05e4153580 0x7f05e415b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.734+0000 7f05e3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05e4153580 0x7f05e415b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37488/0 (socket says 172.21.15.159:37488) 2024-04-04T06:06:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.734+0000 7f05e3200640 1 -- 172.21.15.159:0/3201952096 learned_addr learned my addr 172.21.15.159:0/3201952096 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.734+0000 7f05dbe00640 1 --2- 172.21.15.159:0/3201952096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05e40d07c0 0x7f05e40c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.734+0000 7f05e2800640 1 --2- 172.21.15.159:0/3201952096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05e40af2e0 0x7f05e4072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.734+0000 7f05e3200640 1 -- 172.21.15.159:0/3201952096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05e40af2e0 msgr2=0x7f05e4072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.734+0000 7f05e3200640 1 --2- 172.21.15.159:0/3201952096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05e40af2e0 0x7f05e4072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.734+0000 7f05e3200640 1 -- 172.21.15.159:0/3201952096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05e40d07c0 msgr2=0x7f05e40c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.734+0000 7f05e3200640 1 --2- 172.21.15.159:0/3201952096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05e40d07c0 0x7f05e40c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.734+0000 7f05e3200640 1 -- 172.21.15.159:0/3201952096 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e40c9c90 con 0x7f05e4153580 2024-04-04T06:06:12.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.735+0000 7f05e3200640 1 --2- 172.21.15.159:0/3201952096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05e4153580 0x7f05e415b840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00c6e0 tx=0x7f05dc00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:12.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.735+0000 7f05daa00640 1 -- 172.21.15.159:0/3201952096 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc003b70 con 0x7f05e4153580 2024-04-04T06:06:12.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.735+0000 7f05daa00640 1 -- 172.21.15.159:0/3201952096 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05dc003d30 con 0x7f05e4153580 2024-04-04T06:06:12.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.735+0000 7f05daa00640 1 -- 172.21.15.159:0/3201952096 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc005310 con 0x7f05e4153580 2024-04-04T06:06:12.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.735+0000 7f05e9400640 1 -- 172.21.15.159:0/3201952096 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e40cb420 con 0x7f05e4153580 2024-04-04T06:06:12.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.735+0000 7f05e9400640 1 -- 172.21.15.159:0/3201952096 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e40cb9e0 con 0x7f05e4153580 2024-04-04T06:06:12.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.736+0000 7f05e9400640 1 -- 172.21.15.159:0/3201952096 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05e40ade60 con 0x7f05e4153580 2024-04-04T06:06:12.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.737+0000 7f05daa00640 1 -- 172.21.15.159:0/3201952096 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05dc0054b0 con 0x7f05e4153580 2024-04-04T06:06:12.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.738+0000 7f05daa00640 1 --2- 172.21.15.159:0/3201952096 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f05c006c8f0 0x7f05c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:12.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.738+0000 7f05daa00640 1 -- 172.21.15.159:0/3201952096 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f05dc08ba60 con 0x7f05e4153580 2024-04-04T06:06:12.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.738+0000 7f05e2800640 1 --2- 172.21.15.159:0/3201952096 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f05c006c8f0 0x7f05c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:12.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.739+0000 7f05e2800640 1 --2- 172.21.15.159:0/3201952096 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f05c006c8f0 0x7f05c006edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05d4006fd0 tx=0x7f05d4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:12.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.743+0000 7f05daa00640 1 -- 172.21.15.159:0/3201952096 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f05e40ade60 con 0x7f05e4153580 2024-04-04T06:06:12.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.941+0000 7f05e9400640 1 -- 172.21.15.159:0/3201952096 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f05e40685c0 con 0x7f05e4153580 2024-04-04T06:06:12.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.942+0000 7f05daa00640 1 -- 172.21.15.159:0/3201952096 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f05dc059df0 con 0x7f05e4153580 2024-04-04T06:06:12.937 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:06:12.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.945+0000 7f05e9400640 1 -- 172.21.15.159:0/3201952096 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f05c006c8f0 msgr2=0x7f05c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:12.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.945+0000 7f05e9400640 1 --2- 172.21.15.159:0/3201952096 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f05c006c8f0 0x7f05c006edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05d4006fd0 tx=0x7f05d4008040 comp rx=0 tx=0).stop 2024-04-04T06:06:12.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.945+0000 7f05e9400640 1 -- 172.21.15.159:0/3201952096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05e4153580 msgr2=0x7f05e415b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:12.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.945+0000 7f05e9400640 1 --2- 172.21.15.159:0/3201952096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05e4153580 0x7f05e415b840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00c6e0 tx=0x7f05dc00cba0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.945+0000 7f05e9400640 1 -- 172.21.15.159:0/3201952096 shutdown_connections 2024-04-04T06:06:12.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.945+0000 7f05e9400640 1 --2- 172.21.15.159:0/3201952096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05e4153580 0x7f05e415b840 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.945+0000 7f05e9400640 1 --2- 172.21.15.159:0/3201952096 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f05c006c8f0 0x7f05c006edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.945+0000 7f05e9400640 1 --2- 172.21.15.159:0/3201952096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05e40d07c0 0x7f05e40c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.945+0000 7f05e9400640 1 --2- 172.21.15.159:0/3201952096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05e40af2e0 0x7f05e4072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:12.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.945+0000 7f05e9400640 1 -- 172.21.15.159:0/3201952096 >> 172.21.15.159:0/3201952096 conn(0x7f05e406b750 msgr2=0x7f05e406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:12.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.945+0000 7f05e9400640 1 -- 172.21.15.159:0/3201952096 shutdown_connections 2024-04-04T06:06:12.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:12.945+0000 7f05e9400640 1 -- 172.21.15.159:0/3201952096 wait complete. 2024-04-04T06:06:12.957 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:06:13.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.172+0000 7fd9a9000640 1 Processor -- start 2024-04-04T06:06:13.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.172+0000 7fd9a9000640 1 -- start start 2024-04-04T06:06:13.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.174+0000 7fd9a9000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd9a40d0990 0x7fd9a40aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:13.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.174+0000 7fd9a9000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9a40af2e0 0x7fd9a41600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:13.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.174+0000 7fd9a9000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9a40afab0 con 0x7fd9a4153580 2024-04-04T06:06:13.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.174+0000 7fd9a9000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd9a415cc80 con 0x7fd9a40d0990 2024-04-04T06:06:13.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.174+0000 7fd9a9000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a415da00 con 0x7fd9a40af2e0 2024-04-04T06:06:13.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.174+0000 7fd9a2800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9a40af2e0 0x7fd9a41600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:13.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.174+0000 7fd9a1400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd9a40d0990 0x7fd9a40aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:13.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.174+0000 7fd9a1e00640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fd9a4153580 0x7fd9a40d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.159:44378/0 (socket says 172.21.15.159:44378) 2024-04-04T06:06:13.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.175+0000 7fd9a1e00640 1 -- 172.21.15.159:0/3830258894 learned_addr learned my addr 172.21.15.159:0/3830258894 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:13.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.175+0000 7fd9a0a00640 1 -- 172.21.15.159:0/3830258894 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 305368088 0 0) 0x7fd9a40afab0 con 0x7fd9a4153580 2024-04-04T06:06:13.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.175+0000 7fd9a0a00640 1 -- 172.21.15.159:0/3830258894 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd988003520 con 0x7fd9a4153580 2024-04-04T06:06:13.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.175+0000 7fd9a1400640 1 -- 172.21.15.159:0/3830258894 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9a40af2e0 msgr2=0x7fd9a41600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:13.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.175+0000 7fd9a1400640 1 --2- 172.21.15.159:0/3830258894 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9a40af2e0 0x7fd9a41600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:13.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.175+0000 7fd9a1400640 1 -- 172.21.15.159:0/3830258894 >> v1:172.21.15.178:6789/0 conn(0x7fd9a4153580 legacy=0x7fd9a40d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:13.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.175+0000 7fd9a1400640 1 -- 172.21.15.159:0/3830258894 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a415cf20 con 0x7fd9a40d0990 2024-04-04T06:06:13.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.175+0000 7fd9a2800640 1 --2- 172.21.15.159:0/3830258894 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9a40af2e0 0x7fd9a41600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:06:13.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.177+0000 7fd9a1400640 1 --2- 172.21.15.159:0/3830258894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9a40d0990 0x7fd9a40aeda0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd9900097a0 tx=0x7fd99002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58db97eb8e38ce51 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:13.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.177+0000 7fd9a0a00640 1 -- 172.21.15.159:0/3830258894 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd990031780 con 0x7fd9a40d0990 2024-04-04T06:06:13.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.177+0000 7fd9a0a00640 1 -- 172.21.15.159:0/3830258894 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd990037070 con 0x7fd9a40d0990 2024-04-04T06:06:13.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.177+0000 7fd9a0a00640 1 -- 172.21.15.159:0/3830258894 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd990031b40 con 0x7fd9a40d0990 2024-04-04T06:06:13.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.178+0000 7fd9a9000640 1 -- 172.21.15.159:0/3830258894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9a40d0990 msgr2=0x7fd9a40aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:13.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.178+0000 7fd9a9000640 1 --2- 172.21.15.159:0/3830258894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9a40d0990 0x7fd9a40aeda0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd9900097a0 tx=0x7fd99002b820 comp rx=0 tx=0).stop 2024-04-04T06:06:13.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.178+0000 7fd9a9000640 1 -- 172.21.15.159:0/3830258894 shutdown_connections 2024-04-04T06:06:13.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.178+0000 7fd9a9000640 1 --2- 172.21.15.159:0/3830258894 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9a40af2e0 0x7fd9a41600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:13.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.178+0000 7fd9a9000640 1 --2- 172.21.15.159:0/3830258894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9a40d0990 0x7fd9a40aeda0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:13.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.178+0000 7fd9a9000640 1 -- 172.21.15.159:0/3830258894 >> 172.21.15.159:0/3830258894 conn(0x7fd9a406b750 msgr2=0x7fd9a406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:13.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.178+0000 7fd9a9000640 1 -- 172.21.15.159:0/3830258894 shutdown_connections 2024-04-04T06:06:13.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.178+0000 7fd9a9000640 1 -- 172.21.15.159:0/3830258894 wait complete. 2024-04-04T06:06:13.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.180+0000 7fd9a9000640 1 Processor -- start 2024-04-04T06:06:13.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.180+0000 7fd9a9000640 1 -- start start 2024-04-04T06:06:13.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.181+0000 7fd9a9000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9a40af2e0 0x7fd9a414b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:13.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.181+0000 7fd9a9000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a40d0990 0x7fd9a41562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:13.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.181+0000 7fd9a9000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9a4153580 0x7fd9a415ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:13.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.181+0000 7fd9a9000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4153960 con 0x7fd9a40d0990 2024-04-04T06:06:13.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.181+0000 7fd9a9000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a415d4e0 con 0x7fd9a4153580 2024-04-04T06:06:13.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.181+0000 7fd9a9000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a40af710 con 0x7fd9a40af2e0 2024-04-04T06:06:13.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.181+0000 7fd9a1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a40d0990 0x7fd9a41562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:13.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.181+0000 7fd9a1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a40d0990 0x7fd9a41562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37504/0 (socket says 172.21.15.159:37504) 2024-04-04T06:06:13.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.181+0000 7fd9a1400640 1 -- 172.21.15.159:0/1340024981 learned_addr learned my addr 172.21.15.159:0/1340024981 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:13.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.181+0000 7fd9a1e00640 1 --2- 172.21.15.159:0/1340024981 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9a40af2e0 0x7fd9a414b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:13.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.181+0000 7fd9a2800640 1 --2- 172.21.15.159:0/1340024981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9a4153580 0x7fd9a415ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:13.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.181+0000 7fd9a1400640 1 -- 172.21.15.159:0/1340024981 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9a40af2e0 msgr2=0x7fd9a414b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:13.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.181+0000 7fd9a1400640 1 --2- 172.21.15.159:0/1340024981 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9a40af2e0 0x7fd9a414b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:13.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.182+0000 7fd9a1400640 1 -- 172.21.15.159:0/1340024981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9a4153580 msgr2=0x7fd9a415ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:13.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.182+0000 7fd9a1400640 1 --2- 172.21.15.159:0/1340024981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9a4153580 0x7fd9a415ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:13.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.182+0000 7fd9a1400640 1 -- 172.21.15.159:0/1340024981 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a40b2260 con 0x7fd9a40d0990 2024-04-04T06:06:13.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.182+0000 7fd9a2800640 1 --2- 172.21.15.159:0/1340024981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9a4153580 0x7fd9a415ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:13.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.182+0000 7fd9a1400640 1 --2- 172.21.15.159:0/1340024981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a40d0990 0x7fd9a41562f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd990034040 tx=0x7fd9900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:13.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.182+0000 7fd99a000640 1 -- 172.21.15.159:0/1340024981 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd990032610 con 0x7fd9a40d0990 2024-04-04T06:06:13.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.182+0000 7fd99a000640 1 -- 172.21.15.159:0/1340024981 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9900327d0 con 0x7fd9a40d0990 2024-04-04T06:06:13.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.182+0000 7fd99a000640 1 -- 172.21.15.159:0/1340024981 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd990032bc0 con 0x7fd9a40d0990 2024-04-04T06:06:13.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.182+0000 7fd9a9000640 1 -- 172.21.15.159:0/1340024981 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a40b2540 con 0x7fd9a40d0990 2024-04-04T06:06:13.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.182+0000 7fd9a9000640 1 -- 172.21.15.159:0/1340024981 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a40b2b00 con 0x7fd9a40d0990 2024-04-04T06:06:13.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.183+0000 7fd9a9000640 1 -- 172.21.15.159:0/1340024981 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a40685c0 con 0x7fd9a40d0990 2024-04-04T06:06:13.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.184+0000 7fd99a000640 1 -- 172.21.15.159:0/1340024981 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd99004e440 con 0x7fd9a40d0990 2024-04-04T06:06:13.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.185+0000 7fd99a000640 1 --2- 172.21.15.159:0/1340024981 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd98406c8f0 0x7fd98406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:13.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.186+0000 7fd99a000640 1 -- 172.21.15.159:0/1340024981 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd9900ad810 con 0x7fd9a40d0990 2024-04-04T06:06:13.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.186+0000 7fd9a1e00640 1 --2- 172.21.15.159:0/1340024981 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd98406c8f0 0x7fd98406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:13.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.187+0000 7fd9a1e00640 1 --2- 172.21.15.159:0/1340024981 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd98406c8f0 0x7fd98406edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd9a409e360 tx=0x7fd994009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:13.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.189+0000 7fd99a000640 1 -- 172.21.15.159:0/1340024981 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd990040070 con 0x7fd9a40d0990 2024-04-04T06:06:13.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.387+0000 7fd9a9000640 1 -- 172.21.15.159:0/1340024981 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd9a406df50 con 0x7fd9a40d0990 2024-04-04T06:06:13.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.389+0000 7fd99a000640 1 -- 172.21.15.159:0/1340024981 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fd99002f050 con 0x7fd9a40d0990 2024-04-04T06:06:13.383 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:06:13.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.391+0000 7fd9a9000640 1 -- 172.21.15.159:0/1340024981 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd98406c8f0 msgr2=0x7fd98406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:13.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.391+0000 7fd9a9000640 1 --2- 172.21.15.159:0/1340024981 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd98406c8f0 0x7fd98406edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd9a409e360 tx=0x7fd994009040 comp rx=0 tx=0).stop 2024-04-04T06:06:13.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.391+0000 7fd9a9000640 1 -- 172.21.15.159:0/1340024981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a40d0990 msgr2=0x7fd9a41562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:13.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.391+0000 7fd9a9000640 1 --2- 172.21.15.159:0/1340024981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a40d0990 0x7fd9a41562f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd990034040 tx=0x7fd9900046c0 comp rx=0 tx=0).stop 2024-04-04T06:06:13.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.392+0000 7fd9a9000640 1 -- 172.21.15.159:0/1340024981 shutdown_connections 2024-04-04T06:06:13.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.392+0000 7fd9a9000640 1 --2- 172.21.15.159:0/1340024981 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd98406c8f0 0x7fd98406edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:13.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.392+0000 7fd9a9000640 1 --2- 172.21.15.159:0/1340024981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9a4153580 0x7fd9a415ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:13.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.392+0000 7fd9a9000640 1 --2- 172.21.15.159:0/1340024981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a40d0990 0x7fd9a41562f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:13.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.392+0000 7fd9a9000640 1 --2- 172.21.15.159:0/1340024981 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9a40af2e0 0x7fd9a414b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:13.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.392+0000 7fd9a9000640 1 -- 172.21.15.159:0/1340024981 >> 172.21.15.159:0/1340024981 conn(0x7fd9a406b750 msgr2=0x7fd9a415dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:13.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.392+0000 7fd9a9000640 1 -- 172.21.15.159:0/1340024981 shutdown_connections 2024-04-04T06:06:13.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.392+0000 7fd9a9000640 1 -- 172.21.15.159:0/1340024981 wait complete. 2024-04-04T06:06:13.406 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:06:13.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.621+0000 7fc85e600640 1 Processor -- start 2024-04-04T06:06:13.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.621+0000 7fc85e600640 1 -- start start 2024-04-04T06:06:13.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.622+0000 7fc85e600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc858147a40 0x7fc8580cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:13.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.622+0000 7fc85e600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8580ce810 con 0x7fc8580d01c0 2024-04-04T06:06:13.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.622+0000 7fc85e600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc858161f30 con 0x7fc8580ce380 2024-04-04T06:06:13.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.622+0000 7fc85e600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc858162b00 con 0x7fc858147a40 2024-04-04T06:06:13.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.623+0000 7fc857e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fc8580d01c0 0x7fc8580cdc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:46038/0 (socket says 172.21.15.159:46038) 2024-04-04T06:06:13.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.623+0000 7fc857e00640 1 -- 172.21.15.159:0/1569081160 learned_addr learned my addr 172.21.15.159:0/1569081160 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:13.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.623+0000 7fc85d200640 1 --2- 172.21.15.159:0/1569081160 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc858147a40 0x7fc8580cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:13.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.623+0000 7fc857400640 1 -- 172.21.15.159:0/1569081160 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1832699305 0 0) 0x7fc8580ce810 con 0x7fc8580d01c0 2024-04-04T06:06:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.623+0000 7fc857400640 1 -- 172.21.15.159:0/1569081160 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc83c003520 con 0x7fc8580d01c0 2024-04-04T06:06:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.624+0000 7fc857400640 1 -- 172.21.15.159:0/1569081160 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2932886932 0 0) 0x7fc83c003520 con 0x7fc8580d01c0 2024-04-04T06:06:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.624+0000 7fc857400640 1 -- 172.21.15.159:0/1569081160 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc858147a40 msgr2=0x7fc8580cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.624+0000 7fc857400640 1 --2- 172.21.15.159:0/1569081160 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc858147a40 0x7fc8580cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.624+0000 7fc857400640 1 -- 172.21.15.159:0/1569081160 >> v1:172.21.15.178:6789/0 conn(0x7fc8580ce380 legacy=0x7fc85806da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.624+0000 7fc857400640 1 -- 172.21.15.159:0/1569081160 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8581621d0 con 0x7fc8580d01c0 2024-04-04T06:06:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.624+0000 7fc857400640 1 -- 172.21.15.159:0/1569081160 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fc844001670 con 0x7fc8580d01c0 2024-04-04T06:06:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.624+0000 7fc857400640 1 -- 172.21.15.159:0/1569081160 >> v1:172.21.15.159:6789/0 conn(0x7fc8580d01c0 legacy=0x7fc8580cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.624+0000 7fc857400640 1 --2- 172.21.15.159:0/1569081160 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc858147a40 0x7fc83c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.624+0000 7fc857400640 1 --2- 172.21.15.159:0/1569081160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc83c003f80 0x7fc83c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.624+0000 7fc857400640 1 --2- 172.21.15.159:0/1569081160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc83c004d40 0x7fc83c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.624+0000 7fc857400640 1 -- 172.21.15.159:0/1569081160 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc858162b00 con 0x7fc83c003f80 2024-04-04T06:06:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.624+0000 7fc857400640 1 -- 172.21.15.159:0/1569081160 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c00bad0 con 0x7fc83c004d40 2024-04-04T06:06:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.624+0000 7fc857400640 1 -- 172.21.15.159:0/1569081160 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc83c00bcd0 con 0x7fc858147a40 2024-04-04T06:06:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.624+0000 7fc857e00640 1 --2- 172.21.15.159:0/1569081160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc83c003f80 0x7fc83c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.624+0000 7fc85d200640 1 --2- 172.21.15.159:0/1569081160 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc858147a40 0x7fc83c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.624+0000 7fc85dc00640 1 --2- 172.21.15.159:0/1569081160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc83c004d40 0x7fc83c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.624+0000 7fc857e00640 1 -- 172.21.15.159:0/1569081160 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc858147a40 msgr2=0x7fc83c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.624+0000 7fc857e00640 1 --2- 172.21.15.159:0/1569081160 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc858147a40 0x7fc83c003a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.624+0000 7fc857e00640 1 -- 172.21.15.159:0/1569081160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc83c004d40 msgr2=0x7fc83c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.625+0000 7fc857e00640 1 --2- 172.21.15.159:0/1569081160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc83c004d40 0x7fc83c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.625+0000 7fc857e00640 1 -- 172.21.15.159:0/1569081160 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83c003520 con 0x7fc83c003f80 2024-04-04T06:06:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.625+0000 7fc85dc00640 1 --2- 172.21.15.159:0/1569081160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc83c004d40 0x7fc83c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:13.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.626+0000 7fc857e00640 1 --2- 172.21.15.159:0/1569081160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc83c003f80 0x7fc83c004800 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc84400c210 tx=0x7fc83c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456f4908848bc3c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:13.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.626+0000 7fc857400640 1 -- 172.21.15.159:0/1569081160 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844007d10 con 0x7fc83c003f80 2024-04-04T06:06:13.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.626+0000 7fc857400640 1 -- 172.21.15.159:0/1569081160 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc844030070 con 0x7fc83c003f80 2024-04-04T06:06:13.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.626+0000 7fc857400640 1 -- 172.21.15.159:0/1569081160 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844005d70 con 0x7fc83c003f80 2024-04-04T06:06:13.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.626+0000 7fc85e600640 1 -- 172.21.15.159:0/1569081160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc83c003f80 msgr2=0x7fc83c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:13.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.626+0000 7fc85e600640 1 --2- 172.21.15.159:0/1569081160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc83c003f80 0x7fc83c004800 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc84400c210 tx=0x7fc83c005830 comp rx=0 tx=0).stop 2024-04-04T06:06:13.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.626+0000 7fc85dc00640 1 -- 172.21.15.159:0/1569081160 reap_dead start 2024-04-04T06:06:13.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.626+0000 7fc85e600640 1 -- 172.21.15.159:0/1569081160 shutdown_connections 2024-04-04T06:06:13.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.626+0000 7fc85e600640 1 -- 172.21.15.159:0/1569081160 >> 172.21.15.159:0/1569081160 conn(0x7fc8580ccad0 msgr2=0x7fc8580cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:13.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.627+0000 7fc85e600640 1 -- 172.21.15.159:0/1569081160 shutdown_connections 2024-04-04T06:06:13.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.627+0000 7fc85e600640 1 -- 172.21.15.159:0/1569081160 wait complete. 2024-04-04T06:06:13.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.628+0000 7fc85e600640 1 Processor -- start 2024-04-04T06:06:13.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.628+0000 7fc85e600640 1 -- start start 2024-04-04T06:06:13.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.629+0000 7fc85e600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc83c003f80 0x7fc8581ea190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:13.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.629+0000 7fc85e600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8580b5ac0 0x7fc858140e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:13.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.629+0000 7fc85e600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8581413a0 0x7fc858141820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:13.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.629+0000 7fc85e600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc8580d0620 con 0x7fc83c003f80 2024-04-04T06:06:13.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.629+0000 7fc85e600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc858162790 con 0x7fc8580b5ac0 2024-04-04T06:06:13.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.629+0000 7fc85e600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc858147e20 con 0x7fc8581413a0 2024-04-04T06:06:13.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.630+0000 7fc85d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc83c003f80 0x7fc8581ea190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:13.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.630+0000 7fc85d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc83c003f80 0x7fc8581ea190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37512/0 (socket says 172.21.15.159:37512) 2024-04-04T06:06:13.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.630+0000 7fc85d200640 1 -- 172.21.15.159:0/4119570603 learned_addr learned my addr 172.21.15.159:0/4119570603 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:13.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.630+0000 7fc857e00640 1 --2- 172.21.15.159:0/4119570603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8580b5ac0 0x7fc858140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:13.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.630+0000 7fc85d200640 1 -- 172.21.15.159:0/4119570603 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8581413a0 msgr2=0x7fc858141820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:13.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.630+0000 7fc85d200640 1 --2- 172.21.15.159:0/4119570603 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8581413a0 0x7fc858141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:13.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.630+0000 7fc85d200640 1 -- 172.21.15.159:0/4119570603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8580b5ac0 msgr2=0x7fc858140e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:13.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.630+0000 7fc85d200640 1 --2- 172.21.15.159:0/4119570603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8580b5ac0 0x7fc858140e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:13.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.630+0000 7fc85d200640 1 -- 172.21.15.159:0/4119570603 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8580b36a0 con 0x7fc83c003f80 2024-04-04T06:06:13.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.630+0000 7fc857e00640 1 --2- 172.21.15.159:0/4119570603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8580b5ac0 0x7fc858140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:13.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.630+0000 7fc85d200640 1 --2- 172.21.15.159:0/4119570603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc83c003f80 0x7fc8581ea190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc848010860 tx=0x7fc848010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:13.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.631+0000 7fc854c00640 1 -- 172.21.15.159:0/4119570603 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84800f040 con 0x7fc83c003f80 2024-04-04T06:06:13.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.631+0000 7fc854c00640 1 -- 172.21.15.159:0/4119570603 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc84800eb60 con 0x7fc83c003f80 2024-04-04T06:06:13.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.631+0000 7fc854c00640 1 -- 172.21.15.159:0/4119570603 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84801c400 con 0x7fc83c003f80 2024-04-04T06:06:13.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.631+0000 7fc85e600640 1 -- 172.21.15.159:0/4119570603 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8580b3980 con 0x7fc83c003f80 2024-04-04T06:06:13.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.631+0000 7fc85e600640 1 -- 172.21.15.159:0/4119570603 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8580b3f40 con 0x7fc83c003f80 2024-04-04T06:06:13.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.632+0000 7fc85e600640 1 -- 172.21.15.159:0/4119570603 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8580b61c0 con 0x7fc83c003f80 2024-04-04T06:06:13.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.633+0000 7fc854c00640 1 -- 172.21.15.159:0/4119570603 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc84801c5a0 con 0x7fc83c003f80 2024-04-04T06:06:13.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.634+0000 7fc854c00640 1 --2- 172.21.15.159:0/4119570603 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc83806c920 0x7fc83806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:13.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.634+0000 7fc854c00640 1 -- 172.21.15.159:0/4119570603 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc84808c280 con 0x7fc83c003f80 2024-04-04T06:06:13.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.635+0000 7fc857e00640 1 --2- 172.21.15.159:0/4119570603 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc83806c920 0x7fc83806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:13.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.636+0000 7fc857e00640 1 --2- 172.21.15.159:0/4119570603 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc83806c920 0x7fc83806ede0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc844034000 tx=0x7fc844039040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:13.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.639+0000 7fc854c00640 1 -- 172.21.15.159:0/4119570603 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc8580b61c0 con 0x7fc83c003f80 2024-04-04T06:06:13.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.864+0000 7fc85e600640 1 -- 172.21.15.159:0/4119570603 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc858075940 con 0x7fc83c003f80 2024-04-04T06:06:13.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.865+0000 7fc854c00640 1 -- 172.21.15.159:0/4119570603 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5184 (secure 0 0 0) 0x7fc848018030 con 0x7fc83c003f80 2024-04-04T06:06:13.860 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:13.861 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/789759868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":789759868},{"type":"v1","addr":"172.21.15.178:6836","nonce":789759868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2438288045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2438288045},{"type":"v1","addr":"172.21.15.159:6833","nonce":2438288045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/1705205607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1705205607},{"type":"v1","addr":"172.21.15.159:6835","nonce":1705205607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:05:52.801529+0000","modified":"2024-04-04T06:05:55.866207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6837/3362141613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":3362141613},{"type":"v1","addr":"172.21.15.178:6837","nonce":3362141613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-04T06:06:13.861 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 8 2024-04-04T06:06:13.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.868+0000 7fc85e600640 1 -- 172.21.15.159:0/4119570603 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc83806c920 msgr2=0x7fc83806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:13.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.868+0000 7fc85e600640 1 --2- 172.21.15.159:0/4119570603 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc83806c920 0x7fc83806ede0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc844034000 tx=0x7fc844039040 comp rx=0 tx=0).stop 2024-04-04T06:06:13.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.868+0000 7fc85e600640 1 -- 172.21.15.159:0/4119570603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc83c003f80 msgr2=0x7fc8581ea190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:13.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.868+0000 7fc85e600640 1 --2- 172.21.15.159:0/4119570603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc83c003f80 0x7fc8581ea190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc848010860 tx=0x7fc848010d20 comp rx=0 tx=0).stop 2024-04-04T06:06:13.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.868+0000 7fc85e600640 1 -- 172.21.15.159:0/4119570603 shutdown_connections 2024-04-04T06:06:13.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.868+0000 7fc85e600640 1 --2- 172.21.15.159:0/4119570603 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8581413a0 0x7fc858141820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:13.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.868+0000 7fc85e600640 1 --2- 172.21.15.159:0/4119570603 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc83806c920 0x7fc83806ede0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:13.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.868+0000 7fc85e600640 1 --2- 172.21.15.159:0/4119570603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8580b5ac0 0x7fc858140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:13.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.868+0000 7fc85e600640 1 --2- 172.21.15.159:0/4119570603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc83c003f80 0x7fc8581ea190 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:13.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.868+0000 7fc85e600640 1 -- 172.21.15.159:0/4119570603 >> 172.21.15.159:0/4119570603 conn(0x7fc8580ccad0 msgr2=0x7fc85806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:13.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.868+0000 7fc85e600640 1 -- 172.21.15.159:0/4119570603 shutdown_connections 2024-04-04T06:06:13.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:13.868+0000 7fc85e600640 1 -- 172.21.15.159:0/4119570603 wait complete. 2024-04-04T06:06:13.880 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:06:14.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.092+0000 7f4d83c00640 1 Processor -- start 2024-04-04T06:06:14.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.093+0000 7f4d83c00640 1 -- start start 2024-04-04T06:06:14.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.094+0000 7f4d83c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4d7c0af830 0x7f4d7c073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:14.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.094+0000 7f4d83c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4d7c0d0650 0x7f4d7c0d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:14.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.094+0000 7f4d83c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d7c15fe80 con 0x7f4d7c15c390 2024-04-04T06:06:14.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.094+0000 7f4d83c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f4d7c15c770 con 0x7f4d7c0d0650 2024-04-04T06:06:14.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.094+0000 7f4d83c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4d7c0d0fc0 con 0x7f4d7c0af830 2024-04-04T06:06:14.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.094+0000 7f4d81c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4d7c0d0650 0x7f4d7c0d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:14.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.095+0000 7f4d81c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4d7c0d0650 0x7f4d7c0d0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40648/0 (socket says 172.21.15.159:40648) 2024-04-04T06:06:14.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.095+0000 7f4d81c00640 1 -- 172.21.15.159:0/1814720974 learned_addr learned my addr 172.21.15.159:0/1814720974 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:14.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.095+0000 7f4d7be00640 1 --2- 172.21.15.159:0/1814720974 >> v2:172.21.15.178:3300/0 conn(0x7f4d7c0af830 0x7f4d7c073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:14.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.095+0000 7f4d81c00640 1 -- 172.21.15.159:0/1814720974 >> v2:172.21.15.178:3300/0 conn(0x7f4d7c0af830 msgr2=0x7f4d7c073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:14.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.095+0000 7f4d81c00640 1 --2- 172.21.15.159:0/1814720974 >> v2:172.21.15.178:3300/0 conn(0x7f4d7c0af830 0x7f4d7c073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:14.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.095+0000 7f4d81c00640 1 -- 172.21.15.159:0/1814720974 >> v1:172.21.15.178:6789/0 conn(0x7f4d7c15c390 legacy=0x7f4d7c160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:14.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.095+0000 7f4d81c00640 1 -- 172.21.15.159:0/1814720974 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7c0ad120 con 0x7f4d7c0d0650 2024-04-04T06:06:14.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.095+0000 7f4d7be00640 1 --2- 172.21.15.159:0/1814720974 >> v2:172.21.15.178:3300/0 conn(0x7f4d7c0af830 0x7f4d7c073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:14.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.096+0000 7f4d81c00640 1 --2- 172.21.15.159:0/1814720974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d7c0d0650 0x7f4d7c0d0a80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0b5450 tx=0x7f4d7402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf65ec1c9b4c3af7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:14.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.096+0000 7f4d7b400640 1 -- 172.21.15.159:0/1814720974 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7402ca50 con 0x7f4d7c0d0650 2024-04-04T06:06:14.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.096+0000 7f4d7b400640 1 -- 172.21.15.159:0/1814720974 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d7402cbf0 con 0x7f4d7c0d0650 2024-04-04T06:06:14.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.096+0000 7f4d7b400640 1 -- 172.21.15.159:0/1814720974 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d740343f0 con 0x7f4d7c0d0650 2024-04-04T06:06:14.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.097+0000 7f4d83c00640 1 -- 172.21.15.159:0/1814720974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d7c0d0650 msgr2=0x7f4d7c0d0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:14.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.097+0000 7f4d83c00640 1 --2- 172.21.15.159:0/1814720974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d7c0d0650 0x7f4d7c0d0a80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0b5450 tx=0x7f4d7402beb0 comp rx=0 tx=0).stop 2024-04-04T06:06:14.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.097+0000 7f4d83c00640 1 -- 172.21.15.159:0/1814720974 shutdown_connections 2024-04-04T06:06:14.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.097+0000 7f4d83c00640 1 --2- 172.21.15.159:0/1814720974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d7c0d0650 0x7f4d7c0d0a80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:14.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.097+0000 7f4d83c00640 1 --2- 172.21.15.159:0/1814720974 >> v2:172.21.15.178:3300/0 conn(0x7f4d7c0af830 0x7f4d7c073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:14.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.097+0000 7f4d83c00640 1 -- 172.21.15.159:0/1814720974 >> 172.21.15.159:0/1814720974 conn(0x7f4d7c06b990 msgr2=0x7f4d7c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:14.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.097+0000 7f4d83c00640 1 -- 172.21.15.159:0/1814720974 shutdown_connections 2024-04-04T06:06:14.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.097+0000 7f4d83c00640 1 -- 172.21.15.159:0/1814720974 wait complete. 2024-04-04T06:06:14.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.099+0000 7f4d83c00640 1 Processor -- start 2024-04-04T06:06:14.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.099+0000 7f4d83c00640 1 -- start start 2024-04-04T06:06:14.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.100+0000 7f4d83c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d7c0af830 0x7f4d7c09d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:14.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.100+0000 7f4d83c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d7c0d0650 0x7f4d7c14a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:14.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.100+0000 7f4d83c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d7c15c390 0x7f4d7c09a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:14.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.100+0000 7f4d83c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c160120 con 0x7f4d7c0af830 2024-04-04T06:06:14.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.100+0000 7f4d83c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c0ad6e0 con 0x7f4d7c15c390 2024-04-04T06:06:14.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.100+0000 7f4d83c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4d7c160d50 con 0x7f4d7c0d0650 2024-04-04T06:06:14.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.101+0000 7f4d81200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d7c0af830 0x7f4d7c09d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:14.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.101+0000 7f4d81200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d7c0af830 0x7f4d7c09d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40652/0 (socket says 172.21.15.159:40652) 2024-04-04T06:06:14.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.101+0000 7f4d81200640 1 -- 172.21.15.159:0/842700 learned_addr learned my addr 172.21.15.159:0/842700 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:14.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.101+0000 7f4d81c00640 1 --2- 172.21.15.159:0/842700 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d7c15c390 0x7f4d7c09a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:14.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.101+0000 7f4d81200640 1 -- 172.21.15.159:0/842700 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d7c0d0650 msgr2=0x7f4d7c14a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:14.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.101+0000 7f4d81200640 1 --2- 172.21.15.159:0/842700 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d7c0d0650 0x7f4d7c14a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:14.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.101+0000 7f4d81200640 1 -- 172.21.15.159:0/842700 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d7c15c390 msgr2=0x7f4d7c09a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:14.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.101+0000 7f4d81200640 1 --2- 172.21.15.159:0/842700 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d7c15c390 0x7f4d7c09a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:14.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.101+0000 7f4d81200640 1 -- 172.21.15.159:0/842700 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7c14abb0 con 0x7f4d7c0af830 2024-04-04T06:06:14.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.101+0000 7f4d81200640 1 --2- 172.21.15.159:0/842700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d7c0af830 0x7f4d7c09d140 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c00c430 tx=0x7f4d6c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:14.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.102+0000 7f4d78c00640 1 -- 172.21.15.159:0/842700 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c011790 con 0x7f4d7c0af830 2024-04-04T06:06:14.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.102+0000 7f4d78c00640 1 -- 172.21.15.159:0/842700 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d6c017070 con 0x7f4d7c0af830 2024-04-04T06:06:14.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.102+0000 7f4d78c00640 1 -- 172.21.15.159:0/842700 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c011bb0 con 0x7f4d7c0af830 2024-04-04T06:06:14.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.102+0000 7f4d83c00640 1 -- 172.21.15.159:0/842700 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d7c09d820 con 0x7f4d7c0af830 2024-04-04T06:06:14.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.102+0000 7f4d83c00640 1 -- 172.21.15.159:0/842700 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d7c09dde0 con 0x7f4d7c0af830 2024-04-04T06:06:14.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.103+0000 7f4d83c00640 1 -- 172.21.15.159:0/842700 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d7c0685c0 con 0x7f4d7c0af830 2024-04-04T06:06:14.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.104+0000 7f4d78c00640 1 -- 172.21.15.159:0/842700 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d6c01c460 con 0x7f4d7c0af830 2024-04-04T06:06:14.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.105+0000 7f4d78c00640 1 --2- 172.21.15.159:0/842700 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4d5c06c8f0 0x7f4d5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:14.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.105+0000 7f4d78c00640 1 -- 172.21.15.159:0/842700 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4d6c08bab0 con 0x7f4d7c0af830 2024-04-04T06:06:14.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.105+0000 7f4d7be00640 1 --2- 172.21.15.159:0/842700 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4d5c06c8f0 0x7f4d5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:14.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.106+0000 7f4d7be00640 1 --2- 172.21.15.159:0/842700 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4d5c06c8f0 0x7f4d5c06edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4d68006fd0 tx=0x7f4d68008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:14.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.109+0000 7f4d78c00640 1 -- 172.21.15.159:0/842700 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4d7c14abb0 con 0x7f4d7c0af830 2024-04-04T06:06:14.305 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.310+0000 7f4d83c00640 1 -- 172.21.15.159:0/842700 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d7c06bf80 con 0x7f4d7c0af830 2024-04-04T06:06:14.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.312+0000 7f4d78c00640 1 -- 172.21.15.159:0/842700 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7f4d6c059e40 con 0x7f4d7c0af830 2024-04-04T06:06:14.307 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:14.307 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":21,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:05:54.347275+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:05:51.297293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-04T06:05:51.770792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:06:14.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.314+0000 7f4d83c00640 1 -- 172.21.15.159:0/842700 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4d5c06c8f0 msgr2=0x7f4d5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:14.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.314+0000 7f4d83c00640 1 --2- 172.21.15.159:0/842700 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4d5c06c8f0 0x7f4d5c06edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4d68006fd0 tx=0x7f4d68008040 comp rx=0 tx=0).stop 2024-04-04T06:06:14.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.314+0000 7f4d83c00640 1 -- 172.21.15.159:0/842700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d7c0af830 msgr2=0x7f4d7c09d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:14.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.314+0000 7f4d83c00640 1 --2- 172.21.15.159:0/842700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d7c0af830 0x7f4d7c09d140 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c00c430 tx=0x7f4d6c00c8f0 comp rx=0 tx=0).stop 2024-04-04T06:06:14.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.314+0000 7f4d83c00640 1 -- 172.21.15.159:0/842700 shutdown_connections 2024-04-04T06:06:14.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.314+0000 7f4d83c00640 1 --2- 172.21.15.159:0/842700 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4d5c06c8f0 0x7f4d5c06edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:14.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.314+0000 7f4d83c00640 1 --2- 172.21.15.159:0/842700 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d7c15c390 0x7f4d7c09a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:14.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.314+0000 7f4d83c00640 1 --2- 172.21.15.159:0/842700 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d7c0d0650 0x7f4d7c14a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:14.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.314+0000 7f4d83c00640 1 --2- 172.21.15.159:0/842700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d7c0af830 0x7f4d7c09d140 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:14.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.314+0000 7f4d83c00640 1 -- 172.21.15.159:0/842700 >> 172.21.15.159:0/842700 conn(0x7f4d7c06b990 msgr2=0x7f4d7c06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:14.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.315+0000 7f4d83c00640 1 -- 172.21.15.159:0/842700 shutdown_connections 2024-04-04T06:06:14.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.315+0000 7f4d83c00640 1 -- 172.21.15.159:0/842700 wait complete. 2024-04-04T06:06:14.329 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:06:14.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.538+0000 7fef5f400640 1 Processor -- start 2024-04-04T06:06:14.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.539+0000 7fef5f400640 1 -- start start 2024-04-04T06:06:14.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.540+0000 7fef5f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fef5815e980 0x7fef580cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:14.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.540+0000 7fef5f400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef580d2530 con 0x7fef580d4f10 2024-04-04T06:06:14.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.540+0000 7fef5f400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef580d5370 con 0x7fef580d3130 2024-04-04T06:06:14.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.540+0000 7fef5f400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fef580cf630 con 0x7fef5815e980 2024-04-04T06:06:14.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.540+0000 7fef5ca00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fef5815e980 0x7fef580cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:14.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.540+0000 7fef4fe00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fef580d4f10 0x7fef580d2a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41340/0 (socket says 172.21.15.159:41340) 2024-04-04T06:06:14.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.540+0000 7fef4fe00640 1 -- 172.21.15.159:0/3335715197 learned_addr learned my addr 172.21.15.159:0/3335715197 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:14.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.541+0000 7fef5ca00640 1 -- 172.21.15.159:0/3335715197 >> v1:172.21.15.178:6789/0 conn(0x7fef580d3130 legacy=0x7fef580b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:14.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.541+0000 7fef5ca00640 1 -- 172.21.15.159:0/3335715197 >> v1:172.21.15.159:6789/0 conn(0x7fef580d4f10 legacy=0x7fef580d2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:14.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.541+0000 7fef5ca00640 1 -- 172.21.15.159:0/3335715197 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef580cf910 con 0x7fef5815e980 2024-04-04T06:06:14.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.541+0000 7fef57e00640 1 -- 172.21.15.159:0/3335715197 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 622053090 0 0) 0x7fef580d2530 con 0x7fef580d4f10 2024-04-04T06:06:14.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.542+0000 7fef5ca00640 1 --2- 172.21.15.159:0/3335715197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef5815e980 0x7fef580cefe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fef480097a0 tx=0x7fef4802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dba4145c73ec4ea server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:14.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.542+0000 7fef57e00640 1 -- 172.21.15.159:0/3335715197 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48034070 con 0x7fef5815e980 2024-04-04T06:06:14.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.542+0000 7fef57e00640 1 -- 172.21.15.159:0/3335715197 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef48030610 con 0x7fef5815e980 2024-04-04T06:06:14.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.542+0000 7fef57e00640 1 -- 172.21.15.159:0/3335715197 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef480309e0 con 0x7fef5815e980 2024-04-04T06:06:14.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.543+0000 7fef5f400640 1 -- 172.21.15.159:0/3335715197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef5815e980 msgr2=0x7fef580cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:14.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.543+0000 7fef5f400640 1 --2- 172.21.15.159:0/3335715197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef5815e980 0x7fef580cefe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fef480097a0 tx=0x7fef4802b960 comp rx=0 tx=0).stop 2024-04-04T06:06:14.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.543+0000 7fef5f400640 1 -- 172.21.15.159:0/3335715197 shutdown_connections 2024-04-04T06:06:14.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.543+0000 7fef5f400640 1 --2- 172.21.15.159:0/3335715197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef5815e980 0x7fef580cefe0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:14.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.543+0000 7fef5f400640 1 -- 172.21.15.159:0/3335715197 >> 172.21.15.159:0/3335715197 conn(0x7fef5808bfb0 msgr2=0x7fef580d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:14.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.543+0000 7fef5f400640 1 -- 172.21.15.159:0/3335715197 shutdown_connections 2024-04-04T06:06:14.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.543+0000 7fef5f400640 1 -- 172.21.15.159:0/3335715197 wait complete. 2024-04-04T06:06:14.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.544+0000 7fef5f400640 1 Processor -- start 2024-04-04T06:06:14.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.545+0000 7fef5f400640 1 -- start start 2024-04-04T06:06:14.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.545+0000 7fef5f400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef580d3130 0x7fef580cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:14.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.546+0000 7fef5f400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef580d4f10 0x7fef5814f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:14.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.546+0000 7fef5f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef5815e980 0x7fef580b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:14.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.546+0000 7fef5f400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fef580d01b0 con 0x7fef5815e980 2024-04-04T06:06:14.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.546+0000 7fef5f400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fef580cfed0 con 0x7fef580d3130 2024-04-04T06:06:14.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.546+0000 7fef5f400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fef580b40b0 con 0x7fef580d4f10 2024-04-04T06:06:14.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.546+0000 7fef5d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef5815e980 0x7fef580b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:14.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.546+0000 7fef5d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef5815e980 0x7fef580b5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40672/0 (socket says 172.21.15.159:40672) 2024-04-04T06:06:14.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.546+0000 7fef5d400640 1 -- 172.21.15.159:0/3812221622 learned_addr learned my addr 172.21.15.159:0/3812221622 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:14.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.546+0000 7fef5d400640 1 -- 172.21.15.159:0/3812221622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef580d4f10 msgr2=0x7fef5814f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:14.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.546+0000 7fef5ca00640 1 --2- 172.21.15.159:0/3812221622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef580d3130 0x7fef580cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:14.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.546+0000 7fef4fe00640 1 --2- 172.21.15.159:0/3812221622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef580d4f10 0x7fef5814f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:14.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.546+0000 7fef5d400640 1 --2- 172.21.15.159:0/3812221622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef580d4f10 0x7fef5814f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:14.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.546+0000 7fef5d400640 1 -- 172.21.15.159:0/3812221622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef580d3130 msgr2=0x7fef580cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:14.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.546+0000 7fef5d400640 1 --2- 172.21.15.159:0/3812221622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef580d3130 0x7fef580cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:14.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.546+0000 7fef5d400640 1 -- 172.21.15.159:0/3812221622 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef580ce920 con 0x7fef5815e980 2024-04-04T06:06:14.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.547+0000 7fef5d400640 1 --2- 172.21.15.159:0/3812221622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef5815e980 0x7fef580b5af0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fef5000c730 tx=0x7fef5000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:14.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.547+0000 7fef55600640 1 -- 172.21.15.159:0/3812221622 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50003b70 con 0x7fef5815e980 2024-04-04T06:06:14.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.547+0000 7fef55600640 1 -- 172.21.15.159:0/3812221622 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef50003d10 con 0x7fef5815e980 2024-04-04T06:06:14.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.547+0000 7fef55600640 1 -- 172.21.15.159:0/3812221622 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5000aa90 con 0x7fef5815e980 2024-04-04T06:06:14.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.547+0000 7fef5f400640 1 -- 172.21.15.159:0/3812221622 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef580ce640 con 0x7fef5815e980 2024-04-04T06:06:14.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.547+0000 7fef5f400640 1 -- 172.21.15.159:0/3812221622 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef581f7e60 con 0x7fef5815e980 2024-04-04T06:06:14.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.548+0000 7fef5f400640 1 -- 172.21.15.159:0/3812221622 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef580685c0 con 0x7fef5815e980 2024-04-04T06:06:14.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.549+0000 7fef55600640 1 -- 172.21.15.159:0/3812221622 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef5000ac30 con 0x7fef5815e980 2024-04-04T06:06:14.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.550+0000 7fef55600640 1 --2- 172.21.15.159:0/3812221622 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fef3406c8f0 0x7fef3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:14.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.550+0000 7fef55600640 1 -- 172.21.15.159:0/3812221622 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fef5008ba50 con 0x7fef5815e980 2024-04-04T06:06:14.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.550+0000 7fef5ca00640 1 --2- 172.21.15.159:0/3812221622 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fef3406c8f0 0x7fef3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:14.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.551+0000 7fef5ca00640 1 --2- 172.21.15.159:0/3812221622 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fef3406c8f0 0x7fef3406edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fef48036000 tx=0x7fef48008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:14.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.554+0000 7fef55600640 1 -- 172.21.15.159:0/3812221622 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fef50059e70 con 0x7fef5815e980 2024-04-04T06:06:14.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.791+0000 7fef5f400640 1 -- 172.21.15.159:0/3812221622 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef580d19f0 con 0x7fef5815e980 2024-04-04T06:06:14.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.791+0000 7fef55600640 1 -- 172.21.15.159:0/3812221622 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5184 (secure 0 0 0) 0x7fef50059770 con 0x7fef5815e980 2024-04-04T06:06:14.786 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:14.787 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/789759868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":789759868},{"type":"v1","addr":"172.21.15.178:6836","nonce":789759868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2438288045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2438288045},{"type":"v1","addr":"172.21.15.159:6833","nonce":2438288045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/1705205607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1705205607},{"type":"v1","addr":"172.21.15.159:6835","nonce":1705205607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:05:52.801529+0000","modified":"2024-04-04T06:05:55.866207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6837/3362141613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":3362141613},{"type":"v1","addr":"172.21.15.178:6837","nonce":3362141613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-04T06:06:14.787 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 8 2024-04-04T06:06:14.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.794+0000 7fef5f400640 1 -- 172.21.15.159:0/3812221622 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fef3406c8f0 msgr2=0x7fef3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:14.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.794+0000 7fef5f400640 1 --2- 172.21.15.159:0/3812221622 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fef3406c8f0 0x7fef3406edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fef48036000 tx=0x7fef48008030 comp rx=0 tx=0).stop 2024-04-04T06:06:14.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.794+0000 7fef5f400640 1 -- 172.21.15.159:0/3812221622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef5815e980 msgr2=0x7fef580b5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:14.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.794+0000 7fef5f400640 1 --2- 172.21.15.159:0/3812221622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef5815e980 0x7fef580b5af0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fef5000c730 tx=0x7fef5000cbf0 comp rx=0 tx=0).stop 2024-04-04T06:06:14.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.794+0000 7fef5f400640 1 -- 172.21.15.159:0/3812221622 shutdown_connections 2024-04-04T06:06:14.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.794+0000 7fef5f400640 1 --2- 172.21.15.159:0/3812221622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef5815e980 0x7fef580b5af0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:14.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.794+0000 7fef5f400640 1 --2- 172.21.15.159:0/3812221622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef580d4f10 0x7fef5814f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:14.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.794+0000 7fef5f400640 1 --2- 172.21.15.159:0/3812221622 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fef3406c8f0 0x7fef3406edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:14.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.794+0000 7fef5f400640 1 --2- 172.21.15.159:0/3812221622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef580d3130 0x7fef580cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:14.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.794+0000 7fef5f400640 1 -- 172.21.15.159:0/3812221622 >> 172.21.15.159:0/3812221622 conn(0x7fef5808bfb0 msgr2=0x7fef580b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:14.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.794+0000 7fef5f400640 1 -- 172.21.15.159:0/3812221622 shutdown_connections 2024-04-04T06:06:14.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:14.794+0000 7fef5f400640 1 -- 172.21.15.159:0/3812221622 wait complete. 2024-04-04T06:06:15.210 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:05:52.801529+0000', 'modified': '2024-04-04T06:05:55.866207+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4293}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4293': {'gid': 4293, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.178:6837/3362141613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6835', 'nonce': 3362141613}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 3362141613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4293, 'qdb_cluster': [4293]} 2024-04-04T06:06:15.210 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-04T06:06:15.210 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-04T06:06:15.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.436+0000 7fec9f800640 1 Processor -- start 2024-04-04T06:06:15.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.437+0000 7fec9f800640 1 -- start start 2024-04-04T06:06:15.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.438+0000 7fec9f800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec98141c10 0x7fec98071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.438+0000 7fec9f800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec9815ec80 con 0x7fec98071eb0 2024-04-04T06:06:15.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.438+0000 7fec9f800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec9815ef20 con 0x7fec980d0280 2024-04-04T06:06:15.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.438+0000 7fec9f800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fec9815f200 con 0x7fec98141c10 2024-04-04T06:06:15.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.438+0000 7fec9d800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fec98071eb0 0x7fec98162160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41356/0 (socket says 172.21.15.159:41356) 2024-04-04T06:06:15.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.438+0000 7fec9d800640 1 -- 172.21.15.159:0/3162121928 learned_addr learned my addr 172.21.15.159:0/3162121928 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:15.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.439+0000 7fec8fe00640 1 --2- 172.21.15.159:0/3162121928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec98141c10 0x7fec98071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:15.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.439+0000 7fec97e00640 1 -- 172.21.15.159:0/3162121928 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1192747158 0 0) 0x7fec9815ec80 con 0x7fec98071eb0 2024-04-04T06:06:15.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.439+0000 7fec97e00640 1 -- 172.21.15.159:0/3162121928 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec7c003520 con 0x7fec98071eb0 2024-04-04T06:06:15.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.439+0000 7fec97e00640 1 -- 172.21.15.159:0/3162121928 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2342628742 0 0) 0x7fec7c003520 con 0x7fec98071eb0 2024-04-04T06:06:15.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.439+0000 7fec97e00640 1 -- 172.21.15.159:0/3162121928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec98141c10 msgr2=0x7fec98071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:15.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.439+0000 7fec97e00640 1 --2- 172.21.15.159:0/3162121928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec98141c10 0x7fec98071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.439+0000 7fec97e00640 1 -- 172.21.15.159:0/3162121928 >> v1:172.21.15.178:6789/0 conn(0x7fec980d0280 legacy=0x7fec980d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:15.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.439+0000 7fec97e00640 1 -- 172.21.15.159:0/3162121928 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9815f4e0 con 0x7fec98071eb0 2024-04-04T06:06:15.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.439+0000 7fec97e00640 1 -- 172.21.15.159:0/3162121928 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fec900019f0 con 0x7fec98071eb0 2024-04-04T06:06:15.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.439+0000 7fec97e00640 1 -- 172.21.15.159:0/3162121928 >> v1:172.21.15.159:6789/0 conn(0x7fec98071eb0 legacy=0x7fec98162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:15.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.439+0000 7fec97e00640 1 --2- 172.21.15.159:0/3162121928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec98141c10 0x7fec7c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.439+0000 7fec97e00640 1 --2- 172.21.15.159:0/3162121928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec7c003fb0 0x7fec7c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.439+0000 7fec97e00640 1 --2- 172.21.15.159:0/3162121928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec7c004d40 0x7fec7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.439+0000 7fec97e00640 1 -- 172.21.15.159:0/3162121928 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fec9815f200 con 0x7fec7c004d40 2024-04-04T06:06:15.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.439+0000 7fec97e00640 1 -- 172.21.15.159:0/3162121928 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c00bb00 con 0x7fec7c003fb0 2024-04-04T06:06:15.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.439+0000 7fec97e00640 1 -- 172.21.15.159:0/3162121928 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fec7c00bd00 con 0x7fec98141c10 2024-04-04T06:06:15.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.440+0000 7fec9d800640 1 --2- 172.21.15.159:0/3162121928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec7c004d40 0x7fec7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:15.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.440+0000 7fec8fe00640 1 --2- 172.21.15.159:0/3162121928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec7c003fb0 0x7fec7c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:15.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.440+0000 7fec9d800640 1 -- 172.21.15.159:0/3162121928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec98141c10 msgr2=0x7fec7c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:15.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.440+0000 7fec9ce00640 1 --2- 172.21.15.159:0/3162121928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec98141c10 0x7fec7c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:15.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.440+0000 7fec9d800640 1 --2- 172.21.15.159:0/3162121928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec98141c10 0x7fec7c0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.440+0000 7fec9d800640 1 -- 172.21.15.159:0/3162121928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec7c003fb0 msgr2=0x7fec7c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:15.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.440+0000 7fec9d800640 1 --2- 172.21.15.159:0/3162121928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec7c003fb0 0x7fec7c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.440+0000 7fec9d800640 1 -- 172.21.15.159:0/3162121928 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7c003520 con 0x7fec7c004d40 2024-04-04T06:06:15.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.440+0000 7fec8fe00640 1 --2- 172.21.15.159:0/3162121928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec7c003fb0 0x7fec7c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:15.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.441+0000 7fec9d800640 1 --2- 172.21.15.159:0/3162121928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec7c004d40 0x7fec7c0051c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fec9000a790 tx=0x7fec7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef4bf51f0a80e2c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:15.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.441+0000 7fec97e00640 1 -- 172.21.15.159:0/3162121928 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9000a3b0 con 0x7fec7c004d40 2024-04-04T06:06:15.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.441+0000 7fec97e00640 1 -- 172.21.15.159:0/3162121928 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec90034650 con 0x7fec7c004d40 2024-04-04T06:06:15.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.441+0000 7fec97e00640 1 -- 172.21.15.159:0/3162121928 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90034a80 con 0x7fec7c004d40 2024-04-04T06:06:15.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.442+0000 7fec9f800640 1 -- 172.21.15.159:0/3162121928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec7c004d40 msgr2=0x7fec7c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:15.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.442+0000 7fec9f800640 1 --2- 172.21.15.159:0/3162121928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec7c004d40 0x7fec7c0051c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fec9000a790 tx=0x7fec7c005830 comp rx=0 tx=0).stop 2024-04-04T06:06:15.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.442+0000 7fec9d800640 1 -- 172.21.15.159:0/3162121928 reap_dead start 2024-04-04T06:06:15.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.442+0000 7fec9f800640 1 -- 172.21.15.159:0/3162121928 shutdown_connections 2024-04-04T06:06:15.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.442+0000 7fec9f800640 1 -- 172.21.15.159:0/3162121928 >> 172.21.15.159:0/3162121928 conn(0x7fec9806b750 msgr2=0x7fec9806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:15.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.442+0000 7fec9f800640 1 -- 172.21.15.159:0/3162121928 shutdown_connections 2024-04-04T06:06:15.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.442+0000 7fec9f800640 1 -- 172.21.15.159:0/3162121928 wait complete. 2024-04-04T06:06:15.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.444+0000 7fec9f800640 1 Processor -- start 2024-04-04T06:06:15.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.444+0000 7fec9f800640 1 -- start start 2024-04-04T06:06:15.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.445+0000 7fec9f800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec7c004d40 0x7fec98161970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.445+0000 7fec9f800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec980adc20 0x7fec98156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.445+0000 7fec9f800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec980b3680 0x7fec980b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.445+0000 7fec9f800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fec98158e90 con 0x7fec980adc20 2024-04-04T06:06:15.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.445+0000 7fec9f800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fec9806f340 con 0x7fec980b3680 2024-04-04T06:06:15.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.445+0000 7fec9f800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fec98161eb0 con 0x7fec7c004d40 2024-04-04T06:06:15.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.445+0000 7fec8fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec980adc20 0x7fec98156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:15.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.445+0000 7fec8fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec980adc20 0x7fec98156b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40682/0 (socket says 172.21.15.159:40682) 2024-04-04T06:06:15.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.445+0000 7fec8fe00640 1 -- 172.21.15.159:0/454475419 learned_addr learned my addr 172.21.15.159:0/454475419 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:15.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.446+0000 7fec9ce00640 1 --2- 172.21.15.159:0/454475419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec7c004d40 0x7fec98161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:15.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.446+0000 7fec9d800640 1 --2- 172.21.15.159:0/454475419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec980b3680 0x7fec980b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:15.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.446+0000 7fec8fe00640 1 -- 172.21.15.159:0/454475419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec7c004d40 msgr2=0x7fec98161970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:15.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.446+0000 7fec8fe00640 1 --2- 172.21.15.159:0/454475419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec7c004d40 0x7fec98161970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.446+0000 7fec8fe00640 1 -- 172.21.15.159:0/454475419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec980b3680 msgr2=0x7fec980b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:15.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.446+0000 7fec8fe00640 1 --2- 172.21.15.159:0/454475419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec980b3680 0x7fec980b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.446+0000 7fec8fe00640 1 -- 172.21.15.159:0/454475419 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec981f8080 con 0x7fec980adc20 2024-04-04T06:06:15.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.446+0000 7fec9ce00640 1 --2- 172.21.15.159:0/454475419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec7c004d40 0x7fec98161970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:15.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.446+0000 7fec8fe00640 1 --2- 172.21.15.159:0/454475419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec980adc20 0x7fec98156b30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fec80010890 tx=0x7fec80010c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:15.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.446+0000 7fec95600640 1 -- 172.21.15.159:0/454475419 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8000f040 con 0x7fec980adc20 2024-04-04T06:06:15.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.446+0000 7fec95600640 1 -- 172.21.15.159:0/454475419 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec8000eb60 con 0x7fec980adc20 2024-04-04T06:06:15.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.446+0000 7fec95600640 1 -- 172.21.15.159:0/454475419 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8001c400 con 0x7fec980adc20 2024-04-04T06:06:15.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.446+0000 7fec9f800640 1 -- 172.21.15.159:0/454475419 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec981f8360 con 0x7fec980adc20 2024-04-04T06:06:15.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.446+0000 7fec9f800640 1 -- 172.21.15.159:0/454475419 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec981f8920 con 0x7fec980adc20 2024-04-04T06:06:15.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.449+0000 7fec95600640 1 -- 172.21.15.159:0/454475419 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec8001c5a0 con 0x7fec980adc20 2024-04-04T06:06:15.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.449+0000 7fec95600640 1 --2- 172.21.15.159:0/454475419 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fec7406c8f0 0x7fec7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.449+0000 7fec95600640 1 -- 172.21.15.159:0/454475419 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fec8008c170 con 0x7fec980adc20 2024-04-04T06:06:15.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.450+0000 7fec9ce00640 1 --2- 172.21.15.159:0/454475419 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fec7406c8f0 0x7fec7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:15.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.450+0000 7feca25c8740 1 Processor -- start 2024-04-04T06:06:15.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.450+0000 7feca25c8740 1 -- start start 2024-04-04T06:06:15.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.451+0000 7fec9ce00640 1 --2- 172.21.15.159:0/454475419 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fec7406c8f0 0x7fec7406edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fec880016f0 tx=0x7fec88009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:15.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.451+0000 7feca25c8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7130 0x55a66af9a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.451+0000 7feca25c8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66afc7510 0x55a66afc9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.451+0000 7feca25c8740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a66af9a610 0x55a66afca400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.451+0000 7feca25c8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a66af61ab0 con 0x55a66afc7130 2024-04-04T06:06:15.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.451+0000 7feca25c8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a66af03fa0 con 0x55a66afc7510 2024-04-04T06:06:15.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.451+0000 7feca25c8740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a66aeac560 con 0x55a66af9a610 2024-04-04T06:06:15.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.452+0000 7fec9ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7130 0x55a66af9a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:15.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.452+0000 7fec9ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7130 0x55a66af9a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40696/0 (socket says 172.21.15.159:40696) 2024-04-04T06:06:15.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.452+0000 7fec9ce00640 1 -- 172.21.15.159:0/1378206644 learned_addr learned my addr 172.21.15.159:0/1378206644 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:15.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.452+0000 7fec8fe00640 1 --2- 172.21.15.159:0/1378206644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66afc7510 0x55a66afc9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:15.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.452+0000 7fec9d800640 1 --2- 172.21.15.159:0/1378206644 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a66af9a610 0x55a66afca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:15.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.452+0000 7fec9ce00640 1 -- 172.21.15.159:0/1378206644 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a66af9a610 msgr2=0x55a66afca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:15.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.452+0000 7fec9ce00640 1 --2- 172.21.15.159:0/1378206644 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a66af9a610 0x55a66afca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.452+0000 7fec9ce00640 1 -- 172.21.15.159:0/1378206644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66afc7510 msgr2=0x55a66afc9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:15.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.452+0000 7fec9ce00640 1 --2- 172.21.15.159:0/1378206644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66afc7510 0x55a66afc9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.452+0000 7fec9ce00640 1 -- 172.21.15.159:0/1378206644 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a66ac182d0 con 0x55a66afc7130 2024-04-04T06:06:15.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.452+0000 7fec8fe00640 1 --2- 172.21.15.159:0/1378206644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66afc7510 0x55a66afc9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:15.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.452+0000 7fec9d800640 1 --2- 172.21.15.159:0/1378206644 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a66af9a610 0x55a66afca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:15.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.452+0000 7fec9ce00640 1 --2- 172.21.15.159:0/1378206644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7130 0x55a66af9a0d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fec8800eb00 tx=0x7fec88011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df68302d64c8ab4d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:15.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.453+0000 7fec8d600640 1 -- 172.21.15.159:0/1378206644 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8801a050 con 0x55a66afc7130 2024-04-04T06:06:15.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.453+0000 7fec8d600640 1 -- 172.21.15.159:0/1378206644 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec88024070 con 0x55a66afc7130 2024-04-04T06:06:15.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.453+0000 7fec8d600640 1 -- 172.21.15.159:0/1378206644 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88013bf0 con 0x55a66afc7130 2024-04-04T06:06:15.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.453+0000 7feca25c8740 1 -- 172.21.15.159:0/1378206644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7130 msgr2=0x55a66af9a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:15.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.453+0000 7feca25c8740 1 --2- 172.21.15.159:0/1378206644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7130 0x55a66af9a0d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fec8800eb00 tx=0x7fec88011000 comp rx=0 tx=0).stop 2024-04-04T06:06:15.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.453+0000 7feca25c8740 1 -- 172.21.15.159:0/1378206644 shutdown_connections 2024-04-04T06:06:15.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.453+0000 7feca25c8740 1 --2- 172.21.15.159:0/1378206644 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a66af9a610 0x55a66afca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.453+0000 7feca25c8740 1 --2- 172.21.15.159:0/1378206644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66afc7510 0x55a66afc9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.453+0000 7feca25c8740 1 --2- 172.21.15.159:0/1378206644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7130 0x55a66af9a0d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.453+0000 7feca25c8740 1 -- 172.21.15.159:0/1378206644 >> 172.21.15.159:0/1378206644 conn(0x55a66af9c340 msgr2=0x55a66afe4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:15.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.454+0000 7feca25c8740 1 -- 172.21.15.159:0/1378206644 shutdown_connections 2024-04-04T06:06:15.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.454+0000 7feca25c8740 1 -- 172.21.15.159:0/1378206644 wait complete. 2024-04-04T06:06:15.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.454+0000 7feca25c8740 1 -- start start 2024-04-04T06:06:15.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.455+0000 7feca25c8740 1 Processor -- start 2024-04-04T06:06:15.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.455+0000 7feca25c8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66afc7130 0x55a66aff89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.456+0000 7feca25c8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7510 0x55a66afcbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.456+0000 7feca25c8740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a66afcc4e0 0x55a66afb4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.456+0000 7feca25c8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a66aebf920 con 0x55a66afc7510 2024-04-04T06:06:15.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.456+0000 7feca25c8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a66aefff50 con 0x55a66afc7130 2024-04-04T06:06:15.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.456+0000 7feca25c8740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a66af9e8e0 con 0x55a66afcc4e0 2024-04-04T06:06:15.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.456+0000 7feca25c8740 10 client.0 _ll_register_callbacks cb 0x55a66aebf310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:06:15.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.456+0000 7fec8fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7510 0x55a66afcbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:15.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.456+0000 7fec8fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7510 0x55a66afcbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40708/0 (socket says 172.21.15.159:40708) 2024-04-04T06:06:15.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.456+0000 7fec8fe00640 1 -- 172.21.15.159:0/1578190535 learned_addr learned my addr 172.21.15.159:0/1578190535 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:15.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.456+0000 7fec8fe00640 1 -- 172.21.15.159:0/1578190535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a66afcc4e0 msgr2=0x55a66afb4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:15.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.456+0000 7fec8fe00640 1 --2- 172.21.15.159:0/1578190535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a66afcc4e0 0x55a66afb4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.456+0000 7fec8fe00640 1 -- 172.21.15.159:0/1578190535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66afc7130 msgr2=0x55a66aff89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:15.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.456+0000 7fec8fe00640 1 --2- 172.21.15.159:0/1578190535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66afc7130 0x55a66aff89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.456+0000 7fec8fe00640 1 -- 172.21.15.159:0/1578190535 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a66ace26b0 con 0x55a66afc7510 2024-04-04T06:06:15.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.456+0000 7fec8fe00640 1 --2- 172.21.15.159:0/1578190535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7510 0x55a66afcbfa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fec8008a540 tx=0x7fec80059eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1ba1692f991536 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:15.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.456+0000 7fec86000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:06:15.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.457+0000 7fec86000640 1 -- 172.21.15.159:0/1578190535 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec80088060 con 0x55a66afc7510 2024-04-04T06:06:15.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.457+0000 7fec86000640 1 -- 172.21.15.159:0/1578190535 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fec80062550 con 0x55a66afc7510 2024-04-04T06:06:15.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.457+0000 7feca25c8740 1 -- 172.21.15.159:0/1578190535 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec88024070 con 0x55a66afc7510 2024-04-04T06:06:15.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.457+0000 7fec86000640 1 -- 172.21.15.159:0/1578190535 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fec80077050 con 0x55a66afc7510 2024-04-04T06:06:15.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.457+0000 7feca25c8740 10 client.4692 fetch_fsmap learned FSMap version 8 2024-04-04T06:06:15.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.457+0000 7feca25c8740 1 -- 172.21.15.159:0/1578190535 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55a66afcd810 con 0x55a66afc7510 2024-04-04T06:06:15.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.457+0000 7fec86000640 1 -- 172.21.15.159:0/1578190535 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec88024070 con 0x55a66afc7510 2024-04-04T06:06:15.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.457+0000 7fec86000640 1 -- 172.21.15.159:0/1578190535 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fec800792e0 con 0x55a66afc7510 2024-04-04T06:06:15.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.457+0000 7feca25c8740 10 client.4692 fetch_fsmap finished waiting for FSMap version 8 2024-04-04T06:06:15.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.457+0000 7feca25c8740 10 client.4692 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-04T06:06:15.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.457+0000 7feca25c8740 20 client.4692 populate_metadata read hostname 'smithi159' 2024-04-04T06:06:15.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.457+0000 7feca25c8740 1 --2- 172.21.15.159:0/1578190535 >> [v2:172.21.15.178:6835/3362141613,v1:172.21.15.178:6837/3362141613] conn(0x55a66aff77f0 0x55a66afb7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.457+0000 7feca25c8740 4 client.4692 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:06:15.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.457+0000 7feca25c8740 1 -- 172.21.15.159:0/1578190535 --> [v2:172.21.15.178:6835/3362141613,v1:172.21.15.178:6837/3362141613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a66af114e0 con 0x55a66aff77f0 2024-04-04T06:06:15.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.458+0000 7fec9ce00640 1 --2- 172.21.15.159:0/1578190535 >> [v2:172.21.15.178:6835/3362141613,v1:172.21.15.178:6837/3362141613] conn(0x55a66aff77f0 0x55a66afb7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:15.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.459+0000 7fec9ce00640 1 --2- 172.21.15.159:0/1578190535 >> [v2:172.21.15.178:6835/3362141613,v1:172.21.15.178:6837/3362141613] conn(0x55a66aff77f0 0x55a66afb7c30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b523c33ba940a8b1 server_cookie=74adb9bf20462e23 in_seq=0 out_seq=0 2024-04-04T06:06:15.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.459+0000 7fec86000640 10 client.4692 ms_handle_connect on v2:172.21.15.178:6835/3362141613 2024-04-04T06:06:15.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.461+0000 7fec86000640 1 -- 172.21.15.159:0/1578190535 <== mds.0 v2:172.21.15.178:6835/3362141613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a66af114e0 con 0x55a66aff77f0 2024-04-04T06:06:15.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.461+0000 7fec86000640 10 client.4692 handle_command_reply: tid=0 2024-04-04T06:06:15.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.461+0000 7feca25c8740 1 client.4692 shutdown 2024-04-04T06:06:15.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.461+0000 7feca25c8740 1 -- 172.21.15.159:0/1578190535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7510 msgr2=0x55a66afcbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:15.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.461+0000 7feca25c8740 1 --2- 172.21.15.159:0/1578190535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7510 0x55a66afcbfa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fec8008a540 tx=0x7fec80059eb0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.461+0000 7feca25c8740 1 -- 172.21.15.159:0/1578190535 shutdown_connections 2024-04-04T06:06:15.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.461+0000 7feca25c8740 1 --2- 172.21.15.159:0/1578190535 >> [v2:172.21.15.178:6835/3362141613,v1:172.21.15.178:6837/3362141613] conn(0x55a66aff77f0 0x55a66afb7c30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.461+0000 7feca25c8740 1 --2- 172.21.15.159:0/1578190535 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a66afcc4e0 0x55a66afb4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.461+0000 7feca25c8740 1 --2- 172.21.15.159:0/1578190535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7510 0x55a66afcbfa0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.461+0000 7feca25c8740 1 --2- 172.21.15.159:0/1578190535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66afc7130 0x55a66aff89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.461+0000 7fec86000640 0 client.4692 ms_handle_reset on v2:172.21.15.178:6835/3362141613 2024-04-04T06:06:15.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.461+0000 7feca25c8740 1 -- 172.21.15.159:0/1578190535 >> 172.21.15.159:0/1578190535 conn(0x55a66af9a610 msgr2=0x55a66afc9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:15.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.462+0000 7feca25c8740 1 -- 172.21.15.159:0/1578190535 shutdown_connections 2024-04-04T06:06:15.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.462+0000 7feca25c8740 1 -- 172.21.15.159:0/1578190535 wait complete. 2024-04-04T06:06:15.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.462+0000 7feca25c8740 20 client.4692 trim_cache size 0 max 16384 2024-04-04T06:06:15.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.482+0000 7feca25c8740 1 Processor -- start 2024-04-04T06:06:15.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.482+0000 7feca25c8740 1 -- start start 2024-04-04T06:06:15.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.482+0000 7feca25c8740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a66afc7130 0x55a66af995d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.483+0000 7feca25c8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7510 0x55a66afc9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.483+0000 7feca25c8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66afcc4e0 0x55a66afe7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.483+0000 7feca25c8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fec98200500 con 0x55a66afc7510 2024-04-04T06:06:15.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.483+0000 7feca25c8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fec981474d0 con 0x55a66afcc4e0 2024-04-04T06:06:15.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.483+0000 7feca25c8740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fec980b03a0 con 0x55a66afc7130 2024-04-04T06:06:15.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.483+0000 7fec8fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7510 0x55a66afc9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:15.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.483+0000 7fec8fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7510 0x55a66afc9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40710/0 (socket says 172.21.15.159:40710) 2024-04-04T06:06:15.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.483+0000 7fec8fe00640 1 -- 172.21.15.159:0/370690508 learned_addr learned my addr 172.21.15.159:0/370690508 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:15.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.483+0000 7fec9ce00640 1 --2- 172.21.15.159:0/370690508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a66afc7130 0x55a66af995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:15.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.483+0000 7fec8fe00640 1 -- 172.21.15.159:0/370690508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a66afc7130 msgr2=0x55a66af995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:15.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.483+0000 7fec8fe00640 1 --2- 172.21.15.159:0/370690508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a66afc7130 0x55a66af995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.483+0000 7fec8fe00640 1 -- 172.21.15.159:0/370690508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66afcc4e0 msgr2=0x55a66afe7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:15.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.483+0000 7fec8fe00640 1 --2- 172.21.15.159:0/370690508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66afcc4e0 0x55a66afe7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.483+0000 7fec8fe00640 1 -- 172.21.15.159:0/370690508 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a66afccda0 con 0x55a66afc7510 2024-04-04T06:06:15.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.484+0000 7fec8fe00640 1 --2- 172.21.15.159:0/370690508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7510 0x55a66afc9b10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fec8000e540 tx=0x7fec8006f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb00650a41d4618b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:15.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.484+0000 7fec8cc00640 1 -- 172.21.15.159:0/370690508 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec80072070 con 0x55a66afc7510 2024-04-04T06:06:15.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.484+0000 7fec8cc00640 1 -- 172.21.15.159:0/370690508 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec800792e0 con 0x55a66afc7510 2024-04-04T06:06:15.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.484+0000 7fec8cc00640 1 -- 172.21.15.159:0/370690508 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec80021040 con 0x55a66afc7510 2024-04-04T06:06:15.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.484+0000 7feca25c8740 1 -- 172.21.15.159:0/370690508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7510 msgr2=0x55a66afc9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:15.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.484+0000 7feca25c8740 1 --2- 172.21.15.159:0/370690508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7510 0x55a66afc9b10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fec8000e540 tx=0x7fec8006f000 comp rx=0 tx=0).stop 2024-04-04T06:06:15.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.485+0000 7feca25c8740 1 -- 172.21.15.159:0/370690508 shutdown_connections 2024-04-04T06:06:15.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.485+0000 7feca25c8740 1 --2- 172.21.15.159:0/370690508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66afcc4e0 0x55a66afe7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.485+0000 7feca25c8740 1 --2- 172.21.15.159:0/370690508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7510 0x55a66afc9b10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.485+0000 7feca25c8740 1 --2- 172.21.15.159:0/370690508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a66afc7130 0x55a66af995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.485+0000 7feca25c8740 1 -- 172.21.15.159:0/370690508 >> 172.21.15.159:0/370690508 conn(0x55a66af9a610 msgr2=0x55a66afb6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:15.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.485+0000 7feca25c8740 1 -- 172.21.15.159:0/370690508 shutdown_connections 2024-04-04T06:06:15.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.485+0000 7feca25c8740 1 -- 172.21.15.159:0/370690508 wait complete. 2024-04-04T06:06:15.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.485+0000 7feca25c8740 1 -- start start 2024-04-04T06:06:15.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.485+0000 7feca25c8740 1 Processor -- start 2024-04-04T06:06:15.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.486+0000 7feca25c8740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a66afc7130 0x55a66afcb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.486+0000 7feca25c8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7510 0x55a66affe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.486+0000 7feca25c8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66afcc4e0 0x55a66afe7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.486+0000 7feca25c8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a66aff7670 con 0x55a66afc7510 2024-04-04T06:06:15.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.486+0000 7feca25c8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a66afb6a20 con 0x55a66afcc4e0 2024-04-04T06:06:15.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.486+0000 7feca25c8740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a66aff7c90 con 0x55a66afc7130 2024-04-04T06:06:15.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.486+0000 7feca25c8740 10 client.0 _ll_register_callbacks cb 0x7fec800599f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:06:15.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.487+0000 7fec8fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7510 0x55a66affe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:15.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.487+0000 7fec8fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7510 0x55a66affe800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40712/0 (socket says 172.21.15.159:40712) 2024-04-04T06:06:15.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.487+0000 7fec8fe00640 1 -- 172.21.15.159:0/2096749820 learned_addr learned my addr 172.21.15.159:0/2096749820 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:15.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.487+0000 7fec8fe00640 1 -- 172.21.15.159:0/2096749820 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a66afc7130 msgr2=0x55a66afcb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:15.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.487+0000 7fec8fe00640 1 --2- 172.21.15.159:0/2096749820 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a66afc7130 0x55a66afcb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.487+0000 7fec8fe00640 1 -- 172.21.15.159:0/2096749820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66afcc4e0 msgr2=0x55a66afe7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:15.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.487+0000 7fec9d800640 1 --2- 172.21.15.159:0/2096749820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66afcc4e0 0x55a66afe7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:15.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.487+0000 7fec8fe00640 1 --2- 172.21.15.159:0/2096749820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66afcc4e0 0x55a66afe7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.487+0000 7fec8fe00640 1 -- 172.21.15.159:0/2096749820 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a66afdf500 con 0x55a66afc7510 2024-04-04T06:06:15.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.487+0000 7fec8fe00640 1 --2- 172.21.15.159:0/2096749820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7510 0x55a66affe800 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fec80000c00 tx=0x7fec8008a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba6661284564d0b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:15.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.487+0000 7fec86000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:06:15.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.488+0000 7fec86000640 1 -- 172.21.15.159:0/2096749820 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec80021070 con 0x55a66afc7510 2024-04-04T06:06:15.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.488+0000 7fec86000640 1 -- 172.21.15.159:0/2096749820 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fec80059300 con 0x55a66afc7510 2024-04-04T06:06:15.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.488+0000 7feca25c8740 1 -- 172.21.15.159:0/2096749820 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec800792e0 con 0x55a66afc7510 2024-04-04T06:06:15.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.488+0000 7fec86000640 1 -- 172.21.15.159:0/2096749820 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fec80072040 con 0x55a66afc7510 2024-04-04T06:06:15.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.488+0000 7feca25c8740 10 client.4698 fetch_fsmap learned FSMap version 8 2024-04-04T06:06:15.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.488+0000 7feca25c8740 1 -- 172.21.15.159:0/2096749820 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55a66afdf500 con 0x55a66afc7510 2024-04-04T06:06:15.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.488+0000 7fec86000640 1 -- 172.21.15.159:0/2096749820 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec800792e0 con 0x55a66afc7510 2024-04-04T06:06:15.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.488+0000 7fec86000640 1 -- 172.21.15.159:0/2096749820 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fec800603b0 con 0x55a66afc7510 2024-04-04T06:06:15.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.488+0000 7feca25c8740 10 client.4698 fetch_fsmap finished waiting for FSMap version 8 2024-04-04T06:06:15.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.488+0000 7feca25c8740 10 client.4698 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-04T06:06:15.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.488+0000 7feca25c8740 20 client.4698 populate_metadata read hostname 'smithi159' 2024-04-04T06:06:15.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.488+0000 7feca25c8740 1 --2- 172.21.15.159:0/2096749820 >> [v2:172.21.15.178:6835/3362141613,v1:172.21.15.178:6837/3362141613] conn(0x55a66af9e3b0 0x55a66afe7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:15.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.488+0000 7feca25c8740 4 client.4698 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-04T06:06:15.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.488+0000 7feca25c8740 1 -- 172.21.15.159:0/2096749820 --> [v2:172.21.15.178:6835/3362141613,v1:172.21.15.178:6837/3362141613] -- command(tid 0: {"prefix": "status"}) -- 0x55a66af0b830 con 0x55a66af9e3b0 2024-04-04T06:06:15.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.489+0000 7fec9ce00640 1 --2- 172.21.15.159:0/2096749820 >> [v2:172.21.15.178:6835/3362141613,v1:172.21.15.178:6837/3362141613] conn(0x55a66af9e3b0 0x55a66afe7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:15.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.490+0000 7fec9ce00640 1 --2- 172.21.15.159:0/2096749820 >> [v2:172.21.15.178:6835/3362141613,v1:172.21.15.178:6837/3362141613] conn(0x55a66af9e3b0 0x55a66afe7960 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdc26af8d6f42d68 server_cookie=918cb04190ded846 in_seq=0 out_seq=0 2024-04-04T06:06:15.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.490+0000 7fec86000640 10 client.4698 ms_handle_connect on v2:172.21.15.178:6835/3362141613 2024-04-04T06:06:15.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.490+0000 7fec86000640 1 -- 172.21.15.159:0/2096749820 <== mds.0 v2:172.21.15.178:6835/3362141613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55a66af0b830 con 0x55a66af9e3b0 2024-04-04T06:06:15.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.490+0000 7fec86000640 10 client.4698 handle_command_reply: tid=0 2024-04-04T06:06:15.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.490+0000 7feca25c8740 1 client.4698 shutdown 2024-04-04T06:06:15.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.490+0000 7feca25c8740 1 -- 172.21.15.159:0/2096749820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7510 msgr2=0x55a66affe800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:15.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.490+0000 7feca25c8740 1 --2- 172.21.15.159:0/2096749820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7510 0x55a66affe800 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fec80000c00 tx=0x7fec8008a4d0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.490+0000 7feca25c8740 1 -- 172.21.15.159:0/2096749820 shutdown_connections 2024-04-04T06:06:15.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.490+0000 7feca25c8740 1 --2- 172.21.15.159:0/2096749820 >> [v2:172.21.15.178:6835/3362141613,v1:172.21.15.178:6837/3362141613] conn(0x55a66af9e3b0 0x55a66afe7960 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.490+0000 7feca25c8740 1 --2- 172.21.15.159:0/2096749820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a66afcc4e0 0x55a66afe7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.490+0000 7feca25c8740 1 --2- 172.21.15.159:0/2096749820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a66afc7510 0x55a66affe800 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.490+0000 7feca25c8740 1 --2- 172.21.15.159:0/2096749820 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a66afc7130 0x55a66afcb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.490+0000 7feca25c8740 1 -- 172.21.15.159:0/2096749820 >> 172.21.15.159:0/2096749820 conn(0x55a66af9a610 msgr2=0x55a66afc9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:15.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.490+0000 7fec86000640 0 client.4698 ms_handle_reset on v2:172.21.15.178:6835/3362141613 2024-04-04T06:06:15.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.490+0000 7feca25c8740 1 -- 172.21.15.159:0/2096749820 shutdown_connections 2024-04-04T06:06:15.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.490+0000 7feca25c8740 1 -- 172.21.15.159:0/2096749820 wait complete. 2024-04-04T06:06:15.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.491+0000 7feca25c8740 20 client.4698 trim_cache size 0 max 16384 2024-04-04T06:06:15.488 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:06:15.488 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_fsid": "30311045-f714-48fa-832b-4cb2ba8bf0bf", 2024-04-04T06:06:15.488 INFO:teuthology.orchestra.run.smithi159.stdout: "whoami": 0, 2024-04-04T06:06:15.488 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 4293, 2024-04-04T06:06:15.489 INFO:teuthology.orchestra.run.smithi159.stdout: "want_state": "up:active", 2024-04-04T06:06:15.489 INFO:teuthology.orchestra.run.smithi159.stdout: "state": "up:active", 2024-04-04T06:06:15.489 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs", 2024-04-04T06:06:15.489 INFO:teuthology.orchestra.run.smithi159.stdout: "rank_uptime": 22.153440123999999, 2024-04-04T06:06:15.489 INFO:teuthology.orchestra.run.smithi159.stdout: "mdsmap_epoch": 8, 2024-04-04T06:06:15.489 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch": 21, 2024-04-04T06:06:15.489 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch_barrier": 21, 2024-04-04T06:06:15.489 INFO:teuthology.orchestra.run.smithi159.stdout: "uptime": 27.171152108000001 2024-04-04T06:06:15.489 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:06:15.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.491+0000 7fec9f800640 1 -- 172.21.15.159:0/454475419 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fec7406c8f0 msgr2=0x7fec7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:15.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.491+0000 7fec9f800640 1 --2- 172.21.15.159:0/454475419 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fec7406c8f0 0x7fec7406edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fec880016f0 tx=0x7fec88009040 comp rx=0 tx=0).stop 2024-04-04T06:06:15.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.491+0000 7fec9f800640 1 -- 172.21.15.159:0/454475419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec980adc20 msgr2=0x7fec98156b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:15.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.491+0000 7fec9f800640 1 --2- 172.21.15.159:0/454475419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec980adc20 0x7fec98156b30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fec80010890 tx=0x7fec80010c90 comp rx=0 tx=0).stop 2024-04-04T06:06:15.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.491+0000 7fec9f800640 1 -- 172.21.15.159:0/454475419 shutdown_connections 2024-04-04T06:06:15.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.491+0000 7fec9f800640 1 --2- 172.21.15.159:0/454475419 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fec7406c8f0 0x7fec7406edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.491+0000 7fec9f800640 1 --2- 172.21.15.159:0/454475419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec980b3680 0x7fec980b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.491+0000 7fec9f800640 1 --2- 172.21.15.159:0/454475419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec980adc20 0x7fec98156b30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.491+0000 7fec9f800640 1 --2- 172.21.15.159:0/454475419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fec7c004d40 0x7fec98161970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:15.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.491+0000 7fec9f800640 1 -- 172.21.15.159:0/454475419 >> 172.21.15.159:0/454475419 conn(0x7fec9806b750 msgr2=0x7fec980b49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:15.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.492+0000 7fec9f800640 1 -- 172.21.15.159:0/454475419 shutdown_connections 2024-04-04T06:06:15.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:15.492+0000 7fec9f800640 1 -- 172.21.15.159:0/454475419 wait complete. 2024-04-04T06:06:15.505 INFO:teuthology.run_tasks:Running task sequential... 2024-04-04T06:06:15.515 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-04T06:06:15.515 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-04T06:06:15.515 INFO:teuthology.task.print:Enabling mgr modules 2024-04-04T06:06:15.515 INFO:teuthology.run_tasks:Running task kclient... 2024-04-04T06:06:15.542 INFO:tasks.kclient:Mounting kernel clients... 2024-04-04T06:06:15.542 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-04T06:06:15.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:15.542 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-04-04T06:06:15.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:15.597 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link delete ceph-brx 2024-04-04T06:06:15.692 INFO:teuthology.orchestra.run.smithi159.stderr:Cannot find device "ceph-brx" 2024-04-04T06:06:15.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-04T06:06:15.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:15.697 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-04-04T06:06:15.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:15.747 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link delete ceph-brx 2024-04-04T06:06:15.840 INFO:teuthology.orchestra.run.smithi159.stderr:Cannot find device "ceph-brx" 2024-04-04T06:06:15.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-04T06:06:15.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:15.844 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2024-04-04T06:06:15.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:15.899 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link delete ceph-brx 2024-04-04T06:06:15.986 INFO:teuthology.orchestra.run.smithi178.stderr:Cannot find device "ceph-brx" 2024-04-04T06:06:15.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-04T06:06:15.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:15.990 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2024-04-04T06:06:16.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:16.038 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link delete ceph-brx 2024-04-04T06:06:16.123 INFO:teuthology.orchestra.run.smithi178.stderr:Cannot find device "ceph-brx" 2024-04-04T06:06:16.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-04T06:06:16.127 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-04T06:06:16.127 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-04T06:06:16.127 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-04T06:06:16.128 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-04T06:06:16.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.350+0000 7fdd57000640 1 Processor -- start 2024-04-04T06:06:16.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.350+0000 7fdd57000640 1 -- start start 2024-04-04T06:06:16.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.351+0000 7fdd57000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fdd50167450 0x7fdd50169840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:16.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.351+0000 7fdd57000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd5015ec80 con 0x7fdd50141c10 2024-04-04T06:06:16.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.351+0000 7fdd57000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd5015ef20 con 0x7fdd500d0280 2024-04-04T06:06:16.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.351+0000 7fdd57000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fdd5015f200 con 0x7fdd50167450 2024-04-04T06:06:16.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.352+0000 7fdd55000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fdd50167450 0x7fdd50169840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:16.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.352+0000 7fdd4f400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fdd50141c10 0x7fdd50071970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41364/0 (socket says 172.21.15.159:41364) 2024-04-04T06:06:16.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.352+0000 7fdd4f400640 1 -- 172.21.15.159:0/2162276656 learned_addr learned my addr 172.21.15.159:0/2162276656 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:16.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.352+0000 7fdd55000640 1 -- 172.21.15.159:0/2162276656 >> v1:172.21.15.178:6789/0 conn(0x7fdd500d0280 legacy=0x7fdd500d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:16.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.352+0000 7fdd55000640 1 -- 172.21.15.159:0/2162276656 >> v1:172.21.15.159:6789/0 conn(0x7fdd50141c10 legacy=0x7fdd50071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:16.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.352+0000 7fdd55000640 1 -- 172.21.15.159:0/2162276656 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd5015f4e0 con 0x7fdd50167450 2024-04-04T06:06:16.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.352+0000 7fdd4ea00640 1 -- 172.21.15.159:0/2162276656 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3501440121 0 0) 0x7fdd5015ec80 con 0x7fdd50141c10 2024-04-04T06:06:16.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.354+0000 7fdd55000640 1 --2- 172.21.15.159:0/2162276656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd50167450 0x7fdd50169840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009d90 tx=0x7fdd4802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8269144f7a58e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:16.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.354+0000 7fdd4ea00640 1 -- 172.21.15.159:0/2162276656 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48039070 con 0x7fdd50167450 2024-04-04T06:06:16.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.354+0000 7fdd4ea00640 1 -- 172.21.15.159:0/2162276656 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd4802cc60 con 0x7fdd50167450 2024-04-04T06:06:16.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.354+0000 7fdd4ea00640 1 -- 172.21.15.159:0/2162276656 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48034660 con 0x7fdd50167450 2024-04-04T06:06:16.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.354+0000 7fdd57000640 1 -- 172.21.15.159:0/2162276656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd50167450 msgr2=0x7fdd50169840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:16.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.354+0000 7fdd57000640 1 --2- 172.21.15.159:0/2162276656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd50167450 0x7fdd50169840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009d90 tx=0x7fdd4802beb0 comp rx=0 tx=0).stop 2024-04-04T06:06:16.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.355+0000 7fdd57000640 1 -- 172.21.15.159:0/2162276656 shutdown_connections 2024-04-04T06:06:16.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.355+0000 7fdd57000640 1 --2- 172.21.15.159:0/2162276656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd50167450 0x7fdd50169840 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:16.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.355+0000 7fdd57000640 1 -- 172.21.15.159:0/2162276656 >> 172.21.15.159:0/2162276656 conn(0x7fdd5006b750 msgr2=0x7fdd5006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:16.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.355+0000 7fdd57000640 1 -- 172.21.15.159:0/2162276656 shutdown_connections 2024-04-04T06:06:16.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.355+0000 7fdd57000640 1 -- 172.21.15.159:0/2162276656 wait complete. 2024-04-04T06:06:16.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.356+0000 7fdd57000640 1 Processor -- start 2024-04-04T06:06:16.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.357+0000 7fdd57000640 1 -- start start 2024-04-04T06:06:16.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.358+0000 7fdd57000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd500d0280 0x7fdd50143fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:16.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.358+0000 7fdd57000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd50141c10 0x7fdd50145000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:16.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.358+0000 7fdd57000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd50167450 0x7fdd50163b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:16.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.358+0000 7fdd57000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdd50158e90 con 0x7fdd50141c10 2024-04-04T06:06:16.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.358+0000 7fdd57000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5006f340 con 0x7fdd500d0280 2024-04-04T06:06:16.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.358+0000 7fdd57000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdd50160e50 con 0x7fdd50167450 2024-04-04T06:06:16.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.358+0000 7fdd4f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd50141c10 0x7fdd50145000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:16.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.358+0000 7fdd4f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd50141c10 0x7fdd50145000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40728/0 (socket says 172.21.15.159:40728) 2024-04-04T06:06:16.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.358+0000 7fdd4f400640 1 -- 172.21.15.159:0/1713793088 learned_addr learned my addr 172.21.15.159:0/1713793088 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:16.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.358+0000 7fdd4fe00640 1 --2- 172.21.15.159:0/1713793088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd500d0280 0x7fdd50143fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:16.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.358+0000 7fdd4f400640 1 -- 172.21.15.159:0/1713793088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd50167450 msgr2=0x7fdd50163b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:16.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.358+0000 7fdd4f400640 1 --2- 172.21.15.159:0/1713793088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd50167450 0x7fdd50163b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:16.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.358+0000 7fdd4f400640 1 -- 172.21.15.159:0/1713793088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd500d0280 msgr2=0x7fdd50143fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:16.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.358+0000 7fdd4f400640 1 --2- 172.21.15.159:0/1713793088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd500d0280 0x7fdd50143fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:16.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.358+0000 7fdd4f400640 1 -- 172.21.15.159:0/1713793088 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd50160b70 con 0x7fdd50141c10 2024-04-04T06:06:16.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.358+0000 7fdd4fe00640 1 --2- 172.21.15.159:0/1713793088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd500d0280 0x7fdd50143fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:16.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.359+0000 7fdd4f400640 1 --2- 172.21.15.159:0/1713793088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd50141c10 0x7fdd50145000 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800c470 tx=0x7fdd3800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:16.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.359+0000 7fdd47e00640 1 -- 172.21.15.159:0/1713793088 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38017070 con 0x7fdd50141c10 2024-04-04T06:06:16.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.359+0000 7fdd47e00640 1 -- 172.21.15.159:0/1713793088 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd38005ce0 con 0x7fdd50141c10 2024-04-04T06:06:16.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.359+0000 7fdd57000640 1 -- 172.21.15.159:0/1713793088 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd50145960 con 0x7fdd50141c10 2024-04-04T06:06:16.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.359+0000 7fdd47e00640 1 -- 172.21.15.159:0/1713793088 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38007270 con 0x7fdd50141c10 2024-04-04T06:06:16.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.359+0000 7fdd57000640 1 -- 172.21.15.159:0/1713793088 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd501444e0 con 0x7fdd50141c10 2024-04-04T06:06:16.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.360+0000 7fdd57000640 1 -- 172.21.15.159:0/1713793088 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd500685c0 con 0x7fdd50141c10 2024-04-04T06:06:16.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.361+0000 7fdd47e00640 1 -- 172.21.15.159:0/1713793088 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd3801a070 con 0x7fdd50141c10 2024-04-04T06:06:16.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.362+0000 7fdd47e00640 1 --2- 172.21.15.159:0/1713793088 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdd2c06c8f0 0x7fdd2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:16.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.363+0000 7fdd47e00640 1 -- 172.21.15.159:0/1713793088 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdd3808ae40 con 0x7fdd50141c10 2024-04-04T06:06:16.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.363+0000 7fdd4fe00640 1 --2- 172.21.15.159:0/1713793088 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdd2c06c8f0 0x7fdd2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:16.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.364+0000 7fdd4fe00640 1 --2- 172.21.15.159:0/1713793088 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdd2c06c8f0 0x7fdd2c06edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdd50148200 tx=0x7fdd40009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:16.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.367+0000 7fdd47e00640 1 -- 172.21.15.159:0/1713793088 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fdd38058b80 con 0x7fdd50141c10 2024-04-04T06:06:16.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.611+0000 7fdd57000640 1 -- 172.21.15.159:0/1713793088 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdd500cfdb0 con 0x7fdd50141c10 2024-04-04T06:06:16.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.612+0000 7fdd47e00640 1 -- 172.21.15.159:0/1713793088 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fdd3805ce80 con 0x7fdd50141c10 2024-04-04T06:06:16.606 INFO:teuthology.orchestra.run.smithi159.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-04T06:06:16.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.614+0000 7fdd57000640 1 -- 172.21.15.159:0/1713793088 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdd2c06c8f0 msgr2=0x7fdd2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:16.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.614+0000 7fdd57000640 1 --2- 172.21.15.159:0/1713793088 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdd2c06c8f0 0x7fdd2c06edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdd50148200 tx=0x7fdd40009040 comp rx=0 tx=0).stop 2024-04-04T06:06:16.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.614+0000 7fdd57000640 1 -- 172.21.15.159:0/1713793088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd50141c10 msgr2=0x7fdd50145000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:16.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.614+0000 7fdd57000640 1 --2- 172.21.15.159:0/1713793088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd50141c10 0x7fdd50145000 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800c470 tx=0x7fdd3800c930 comp rx=0 tx=0).stop 2024-04-04T06:06:16.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.614+0000 7fdd57000640 1 -- 172.21.15.159:0/1713793088 shutdown_connections 2024-04-04T06:06:16.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.614+0000 7fdd57000640 1 --2- 172.21.15.159:0/1713793088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd50167450 0x7fdd50163b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:16.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.614+0000 7fdd57000640 1 --2- 172.21.15.159:0/1713793088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd50141c10 0x7fdd50145000 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:16.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.614+0000 7fdd57000640 1 --2- 172.21.15.159:0/1713793088 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdd2c06c8f0 0x7fdd2c06edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:16.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.614+0000 7fdd57000640 1 --2- 172.21.15.159:0/1713793088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd500d0280 0x7fdd50143fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:16.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.614+0000 7fdd57000640 1 -- 172.21.15.159:0/1713793088 >> 172.21.15.159:0/1713793088 conn(0x7fdd5006b750 msgr2=0x7fdd50160230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:16.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.614+0000 7fdd57000640 1 -- 172.21.15.159:0/1713793088 shutdown_connections 2024-04-04T06:06:16.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16.614+0000 7fdd57000640 1 -- 172.21.15.159:0/1713793088 wait complete. 2024-04-04T06:06:16.626 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-04T06:06:16.626 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-04T06:06:16.626 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi159.front.sepia.ceph.com 2024-04-04T06:06:16.626 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-04T06:06:16.626 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-04T06:06:16.626 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-04T06:06:16.626 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-04T06:06:16.626 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-04T06:06:16.626 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:06:16.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:16.627 DEBUG:teuthology.orchestra.run.smithi159:> ip addr 2024-04-04T06:06:16.672 INFO:teuthology.orchestra.run.smithi159.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:06:16.672 INFO:teuthology.orchestra.run.smithi159.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:06:16.672 INFO:teuthology.orchestra.run.smithi159.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:06:16.672 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:16.672 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 ::1/128 scope host 2024-04-04T06:06:16.672 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:16.672 INFO:teuthology.orchestra.run.smithi159.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:06:16.672 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c6 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:06:16.672 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f0 2024-04-04T06:06:16.672 INFO:teuthology.orchestra.run.smithi159.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:06:16.672 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c7 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:06:16.672 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f1 2024-04-04T06:06:16.673 INFO:teuthology.orchestra.run.smithi159.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:06:16.673 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4e brd ff:ff:ff:ff:ff:ff 2024-04-04T06:06:16.673 INFO:teuthology.orchestra.run.smithi159.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-04T06:06:16.673 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:06:16.673 INFO:teuthology.orchestra.run.smithi159.stdout: inet 172.21.15.159/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-04T06:06:16.673 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft 42556sec preferred_lft 42556sec 2024-04-04T06:06:16.673 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::ec4:7aff:fe88:724f/64 scope link 2024-04-04T06:06:16.673 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:16.674 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-04T06:06:16.674 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:06:16.674 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:06:16.674 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link add name ceph-brx type bridge 2024-04-04T06:06:16.674 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip addr flush dev ceph-brx 2024-04-04T06:06:16.675 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set ceph-brx up 2024-04-04T06:06:16.675 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-04T06:06:16.675 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:06:16.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:17.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:17.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:17.246 DEBUG:teuthology.orchestra.run.smithi159:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-04T06:06:17.322 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:06:17.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:17.327 DEBUG:teuthology.orchestra.run.smithi159:> route 2024-04-04T06:06:17.376 INFO:teuthology.orchestra.run.smithi159.stdout:Kernel IP routing table 2024-04-04T06:06:17.376 INFO:teuthology.orchestra.run.smithi159.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-04T06:06:17.377 INFO:teuthology.orchestra.run.smithi159.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-04T06:06:17.377 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-04T06:06:17.377 INFO:teuthology.orchestra.run.smithi159.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-04T06:06:17.377 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:06:17.377 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:06:17.377 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-04T06:06:17.377 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-04T06:06:17.377 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-04T06:06:17.378 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:06:17.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:18.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:18.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:18.116 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-04-04T06:06:18.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:18.160 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list-id 2024-04-04T06:06:18.251 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:06:18.251 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:06:18.251 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-04T06:06:18.251 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-04T06:06:18.251 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:06:18.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:18.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:18.437 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-04T06:06:18.437 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:06:18.437 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:06:18.437 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-04T06:06:18.438 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-04T06:06:18.438 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-04T06:06:18.438 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-04T06:06:18.438 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-04T06:06:18.438 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:06:18.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:18.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:18.759 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:06:18.759 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:06:18.759 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set brx.0 up 2024-04-04T06:06:18.759 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set dev brx.0 master ceph-brx 2024-04-04T06:06:18.759 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:06:18.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:18.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:18.970 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:06:18.970 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-04T06:06:19.011 INFO:teuthology.orchestra.run.smithi159.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:06:19.012 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:06:19.012 DEBUG:teuthology.orchestra.run.smithi159:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:06:19.103 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-04T06:06:19.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:19.103 DEBUG:teuthology.orchestra.run.smithi159:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:06:19.237 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:06:19.237 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:06:19.237 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:06:19.835 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:06:19.835 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:06:19.836 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:06:19.836 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:06:19.836 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:06:19.871 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:06:19.872 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:06:19.872 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:06:19.872 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:06:19.872 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:06:19.872 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-04T06:06:19.872 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-04T06:06:19.872 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30311045-f714-48fa-832b-4cb2ba8bf0bf" will pass to kernel 2024-04-04T06:06:19.874 INFO:teuthology.orchestra.run.smithi159.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-04T06:06:19.874 INFO:teuthology.orchestra.run.smithi159.stdout: You just mounted a file system that supports labels which does not 2024-04-04T06:06:19.874 INFO:teuthology.orchestra.run.smithi159.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-04T06:06:19.874 INFO:teuthology.orchestra.run.smithi159.stdout: applications will generate AVC messages and not be allowed access to 2024-04-04T06:06:19.875 INFO:teuthology.orchestra.run.smithi159.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-04T06:06:19.879 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-04T06:06:19.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:19.879 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:06:19.969 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:06:20.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:06:20.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:20.110 INFO:teuthology.orchestra.run.smithi159.stdout:client4540 2024-04-04T06:06:20.111 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:06:20.111 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:06:20.166 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:06:20.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:06:20.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:20.309 INFO:teuthology.orchestra.run.smithi159.stdout:client4540 2024-04-04T06:06:20.309 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:06:20.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:06:20.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:20.448 INFO:teuthology.orchestra.run.smithi159.stdout:client4540 2024-04-04T06:06:20.448 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/30311045-f714-48fa-832b-4cb2ba8bf0bf.client4540/status') 2024-04-04T06:06:20.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:20.609 INFO:teuthology.orchestra.run.smithi159.stdout:instance: client.4540 (3)172.21.15.159:0/2506763738 2024-04-04T06:06:20.609 INFO:teuthology.orchestra.run.smithi159.stdout:blocklisted: false 2024-04-04T06:06:20.609 INFO:teuthology.orchestra.run.smithi159.stderr:0+1 records in 2024-04-04T06:06:20.609 INFO:teuthology.orchestra.run.smithi159.stderr:0+1 records out 2024-04-04T06:06:20.609 INFO:teuthology.orchestra.run.smithi159.stderr:71 bytes copied, 6.3929e-05 s, 1.1 MB/s 2024-04-04T06:06:20.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:20.623 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-04T06:06:20.623 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-04T06:06:20.623 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-04T06:06:20.623 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-04T06:06:20.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.837+0000 7fe15b200640 1 Processor -- start 2024-04-04T06:06:20.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.838+0000 7fe15b200640 1 -- start start 2024-04-04T06:06:20.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.839+0000 7fe15b200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe15c13f300 0x7fe15c06e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:20.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.839+0000 7fe15b200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe15c1419e0 0x7fe15c15fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:20.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.839+0000 7fe15b200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe15c142880 con 0x7fe15c141e40 2024-04-04T06:06:20.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.839+0000 7fe15b200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe15c0cee90 con 0x7fe15c1419e0 2024-04-04T06:06:20.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.839+0000 7fe15b200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe15c1422c0 con 0x7fe15c13f300 2024-04-04T06:06:20.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.840+0000 7fe15a800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fe15c141e40 0x7fe15c15c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41380/0 (socket says 172.21.15.159:41380) 2024-04-04T06:06:20.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.840+0000 7fe153e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe15c1419e0 0x7fe15c15fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:20.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.840+0000 7fe15a800640 1 -- 172.21.15.159:0/2995010994 learned_addr learned my addr 172.21.15.159:0/2995010994 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:20.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.840+0000 7fe159e00640 1 --2- 172.21.15.159:0/2995010994 >> v2:172.21.15.178:3300/0 conn(0x7fe15c13f300 0x7fe15c06e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:20.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.840+0000 7fe153e00640 1 -- 172.21.15.159:0/2995010994 >> v2:172.21.15.178:3300/0 conn(0x7fe15c13f300 msgr2=0x7fe15c06e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:20.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.840+0000 7fe153e00640 1 --2- 172.21.15.159:0/2995010994 >> v2:172.21.15.178:3300/0 conn(0x7fe15c13f300 0x7fe15c06e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:20.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.840+0000 7fe153e00640 1 -- 172.21.15.159:0/2995010994 >> v1:172.21.15.159:6789/0 conn(0x7fe15c141e40 legacy=0x7fe15c15c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:20.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.840+0000 7fe153e00640 1 -- 172.21.15.159:0/2995010994 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15c1425a0 con 0x7fe15c1419e0 2024-04-04T06:06:20.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.840+0000 7fe159400640 1 -- 172.21.15.159:0/2995010994 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2268313706 0 0) 0x7fe15c142880 con 0x7fe15c141e40 2024-04-04T06:06:20.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.840+0000 7fe159e00640 1 --2- 172.21.15.159:0/2995010994 >> v2:172.21.15.178:3300/0 conn(0x7fe15c13f300 0x7fe15c06e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:06:20.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.841+0000 7fe153e00640 1 --2- 172.21.15.159:0/2995010994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe15c1419e0 0x7fe15c15fd90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0cf010 tx=0x7fe14402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327b104510e6ba06 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:20.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.842+0000 7fe159400640 1 -- 172.21.15.159:0/2995010994 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144035020 con 0x7fe15c1419e0 2024-04-04T06:06:20.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.842+0000 7fe159400640 1 -- 172.21.15.159:0/2995010994 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe144031830 con 0x7fe15c1419e0 2024-04-04T06:06:20.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.842+0000 7fe159400640 1 -- 172.21.15.159:0/2995010994 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144031c00 con 0x7fe15c1419e0 2024-04-04T06:06:20.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.842+0000 7fe15b200640 1 -- 172.21.15.159:0/2995010994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe15c1419e0 msgr2=0x7fe15c15fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:20.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.842+0000 7fe15b200640 1 --2- 172.21.15.159:0/2995010994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe15c1419e0 0x7fe15c15fd90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0cf010 tx=0x7fe14402b8d0 comp rx=0 tx=0).stop 2024-04-04T06:06:20.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.842+0000 7fe15b200640 1 -- 172.21.15.159:0/2995010994 shutdown_connections 2024-04-04T06:06:20.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.842+0000 7fe15b200640 1 --2- 172.21.15.159:0/2995010994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe15c1419e0 0x7fe15c15fd90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:20.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.842+0000 7fe15b200640 1 --2- 172.21.15.159:0/2995010994 >> v2:172.21.15.178:3300/0 conn(0x7fe15c13f300 0x7fe15c06e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:20.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.842+0000 7fe15b200640 1 -- 172.21.15.159:0/2995010994 >> 172.21.15.159:0/2995010994 conn(0x7fe15c06b570 msgr2=0x7fe15c06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:20.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.842+0000 7fe15b200640 1 -- 172.21.15.159:0/2995010994 shutdown_connections 2024-04-04T06:06:20.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.842+0000 7fe15b200640 1 -- 172.21.15.159:0/2995010994 wait complete. 2024-04-04T06:06:20.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.844+0000 7fe15b200640 1 Processor -- start 2024-04-04T06:06:20.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.844+0000 7fe15b200640 1 -- start start 2024-04-04T06:06:20.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.845+0000 7fe15b200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe15c13f300 0x7fe15c134100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:20.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.845+0000 7fe15b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe15c1419e0 0x7fe15c0b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:20.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.845+0000 7fe15b200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe15c141e40 0x7fe15c13bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:20.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.845+0000 7fe15b200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c0d04b0 con 0x7fe15c1419e0 2024-04-04T06:06:20.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.845+0000 7fe15b200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c0d0330 con 0x7fe15c141e40 2024-04-04T06:06:20.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.845+0000 7fe15b200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe15c0d0630 con 0x7fe15c13f300 2024-04-04T06:06:20.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.845+0000 7fe153e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe15c1419e0 0x7fe15c0b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:20.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.845+0000 7fe153e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe15c1419e0 0x7fe15c0b2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40744/0 (socket says 172.21.15.159:40744) 2024-04-04T06:06:20.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.845+0000 7fe153e00640 1 -- 172.21.15.159:0/3356697905 learned_addr learned my addr 172.21.15.159:0/3356697905 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:20.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.846+0000 7fe15a800640 1 --2- 172.21.15.159:0/3356697905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe15c141e40 0x7fe15c13bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:20.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.846+0000 7fe159e00640 1 --2- 172.21.15.159:0/3356697905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe15c13f300 0x7fe15c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:20.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.846+0000 7fe153e00640 1 -- 172.21.15.159:0/3356697905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe15c13f300 msgr2=0x7fe15c134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:20.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.846+0000 7fe153e00640 1 --2- 172.21.15.159:0/3356697905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe15c13f300 0x7fe15c134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:20.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.846+0000 7fe153e00640 1 -- 172.21.15.159:0/3356697905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe15c141e40 msgr2=0x7fe15c13bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:20.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.846+0000 7fe153e00640 1 --2- 172.21.15.159:0/3356697905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe15c141e40 0x7fe15c13bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:20.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.846+0000 7fe153e00640 1 -- 172.21.15.159:0/3356697905 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15c134640 con 0x7fe15c1419e0 2024-04-04T06:06:20.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.846+0000 7fe159e00640 1 --2- 172.21.15.159:0/3356697905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe15c13f300 0x7fe15c134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:20.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.846+0000 7fe153e00640 1 --2- 172.21.15.159:0/3356697905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe15c1419e0 0x7fe15c0b2830 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe144009860 tx=0x7fe1440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:20.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.847+0000 7fe152000640 1 -- 172.21.15.159:0/3356697905 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144041070 con 0x7fe15c1419e0 2024-04-04T06:06:20.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.847+0000 7fe152000640 1 -- 172.21.15.159:0/3356697905 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe14403a950 con 0x7fe15c1419e0 2024-04-04T06:06:20.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.847+0000 7fe15b200640 1 -- 172.21.15.159:0/3356697905 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15c0b1790 con 0x7fe15c1419e0 2024-04-04T06:06:20.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.847+0000 7fe152000640 1 -- 172.21.15.159:0/3356697905 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144035040 con 0x7fe15c1419e0 2024-04-04T06:06:20.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.847+0000 7fe15b200640 1 -- 172.21.15.159:0/3356697905 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe15c0b1d50 con 0x7fe15c1419e0 2024-04-04T06:06:20.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.848+0000 7fe15b200640 1 -- 172.21.15.159:0/3356697905 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe15c075940 con 0x7fe15c1419e0 2024-04-04T06:06:20.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.849+0000 7fe152000640 1 -- 172.21.15.159:0/3356697905 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe144038020 con 0x7fe15c1419e0 2024-04-04T06:06:20.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.850+0000 7fe152000640 1 --2- 172.21.15.159:0/3356697905 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe13806c8f0 0x7fe13806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:20.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.850+0000 7fe152000640 1 -- 172.21.15.159:0/3356697905 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe1440ad5f0 con 0x7fe15c1419e0 2024-04-04T06:06:20.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.850+0000 7fe159e00640 1 --2- 172.21.15.159:0/3356697905 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe13806c8f0 0x7fe13806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:20.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.851+0000 7fe159e00640 1 --2- 172.21.15.159:0/3356697905 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe13806c8f0 0x7fe13806edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe14c006fd0 tx=0x7fe14c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:20.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:20.854+0000 7fe152000640 1 -- 172.21.15.159:0/3356697905 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe14407ba10 con 0x7fe15c1419e0 2024-04-04T06:06:21.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:21.094+0000 7fe15b200640 1 -- 172.21.15.159:0/3356697905 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe15c0cd430 con 0x7fe15c1419e0 2024-04-04T06:06:21.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:21.095+0000 7fe152000640 1 -- 172.21.15.159:0/3356697905 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fe14407b310 con 0x7fe15c1419e0 2024-04-04T06:06:21.090 INFO:teuthology.orchestra.run.smithi159.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-04T06:06:21.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:21.098+0000 7fe15b200640 1 -- 172.21.15.159:0/3356697905 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe13806c8f0 msgr2=0x7fe13806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:21.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:21.098+0000 7fe15b200640 1 --2- 172.21.15.159:0/3356697905 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe13806c8f0 0x7fe13806edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe14c006fd0 tx=0x7fe14c008040 comp rx=0 tx=0).stop 2024-04-04T06:06:21.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:21.098+0000 7fe15b200640 1 -- 172.21.15.159:0/3356697905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe15c1419e0 msgr2=0x7fe15c0b2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:21.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:21.098+0000 7fe15b200640 1 --2- 172.21.15.159:0/3356697905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe15c1419e0 0x7fe15c0b2830 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe144009860 tx=0x7fe1440046c0 comp rx=0 tx=0).stop 2024-04-04T06:06:21.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:21.098+0000 7fe15b200640 1 -- 172.21.15.159:0/3356697905 shutdown_connections 2024-04-04T06:06:21.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:21.098+0000 7fe15b200640 1 --2- 172.21.15.159:0/3356697905 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe13806c8f0 0x7fe13806edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:21.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:21.098+0000 7fe15b200640 1 --2- 172.21.15.159:0/3356697905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe15c141e40 0x7fe15c13bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:21.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:21.098+0000 7fe15b200640 1 --2- 172.21.15.159:0/3356697905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe15c1419e0 0x7fe15c0b2830 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:21.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:21.098+0000 7fe15b200640 1 --2- 172.21.15.159:0/3356697905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe15c13f300 0x7fe15c134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:21.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:21.098+0000 7fe15b200640 1 -- 172.21.15.159:0/3356697905 >> 172.21.15.159:0/3356697905 conn(0x7fe15c06b570 msgr2=0x7fe15c15ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:21.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:21.099+0000 7fe15b200640 1 -- 172.21.15.159:0/3356697905 shutdown_connections 2024-04-04T06:06:21.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:21.099+0000 7fe15b200640 1 -- 172.21.15.159:0/3356697905 wait complete. 2024-04-04T06:06:21.114 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-04T06:06:21.114 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-04T06:06:21.114 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi159.front.sepia.ceph.com 2024-04-04T06:06:21.114 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-04T06:06:21.114 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-04T06:06:21.114 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-04T06:06:21.114 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-04T06:06:21.114 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-04T06:06:21.114 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-04T06:06:21.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:21.115 DEBUG:teuthology.orchestra.run.smithi159:> ip addr 2024-04-04T06:06:21.160 INFO:teuthology.orchestra.run.smithi159.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:06:21.160 INFO:teuthology.orchestra.run.smithi159.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:06:21.160 INFO:teuthology.orchestra.run.smithi159.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:06:21.160 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:21.160 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 ::1/128 scope host 2024-04-04T06:06:21.161 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:21.161 INFO:teuthology.orchestra.run.smithi159.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:06:21.161 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c6 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:06:21.161 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f0 2024-04-04T06:06:21.161 INFO:teuthology.orchestra.run.smithi159.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:06:21.161 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c7 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:06:21.161 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f1 2024-04-04T06:06:21.161 INFO:teuthology.orchestra.run.smithi159.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:06:21.161 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4e brd ff:ff:ff:ff:ff:ff 2024-04-04T06:06:21.161 INFO:teuthology.orchestra.run.smithi159.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-04T06:06:21.161 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:06:21.161 INFO:teuthology.orchestra.run.smithi159.stdout: inet 172.21.15.159/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-04T06:06:21.161 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft 42552sec preferred_lft 42552sec 2024-04-04T06:06:21.161 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::ec4:7aff:fe88:724f/64 scope link 2024-04-04T06:06:21.162 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:21.162 INFO:teuthology.orchestra.run.smithi159.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-04T06:06:21.162 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether a2:e3:e0:cb:0a:73 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:06:21.162 INFO:teuthology.orchestra.run.smithi159.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-04T06:06:21.162 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:21.162 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::989e:e4ff:fea5:7568/64 scope link 2024-04-04T06:06:21.162 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:21.162 INFO:teuthology.orchestra.run.smithi159.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:06:21.162 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether a2:e3:e0:cb:0a:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-04T06:06:21.162 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::a0e3:e0ff:fecb:a73/64 scope link 2024-04-04T06:06:21.162 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:21.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:21.164 DEBUG:teuthology.orchestra.run.smithi159:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-04T06:06:21.295 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:06:21.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:21.299 DEBUG:teuthology.orchestra.run.smithi159:> route 2024-04-04T06:06:21.352 INFO:teuthology.orchestra.run.smithi159.stdout:Kernel IP routing table 2024-04-04T06:06:21.352 INFO:teuthology.orchestra.run.smithi159.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-04T06:06:21.352 INFO:teuthology.orchestra.run.smithi159.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-04T06:06:21.353 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-04T06:06:21.353 INFO:teuthology.orchestra.run.smithi159.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-04T06:06:21.353 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:06:21.353 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:06:21.353 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-04T06:06:21.353 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-04T06:06:21.353 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-04T06:06:21.353 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:06:21.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:21.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:21.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:21.598 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-04-04T06:06:21.645 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-04T06:06:21.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:21.647 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list-id 2024-04-04T06:06:21.695 INFO:teuthology.orchestra.run.smithi159.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-04T06:06:21.696 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:06:21.697 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:06:21.697 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-04T06:06:21.697 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-04T06:06:21.697 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:06:21.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:21.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:21.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:21.900 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-04T06:06:21.989 INFO:teuthology.orchestra.run.smithi159.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:06:21.989 INFO:teuthology.orchestra.run.smithi159.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:06:21.989 INFO:teuthology.orchestra.run.smithi159.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:06:21.989 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:21.989 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 ::1/128 scope host 2024-04-04T06:06:21.989 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:21.989 INFO:teuthology.orchestra.run.smithi159.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-04T06:06:21.989 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 9a:94:2c:b8:7b:11 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-04T06:06:21.990 INFO:teuthology.orchestra.run.smithi159.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-04T06:06:21.990 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:21.990 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::9894:2cff:feb8:7b11/64 scope link 2024-04-04T06:06:21.990 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:21.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:21.993 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-04T06:06:22.085 INFO:teuthology.orchestra.run.smithi159.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:06:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:06:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:06:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 ::1/128 scope host 2024-04-04T06:06:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:22.085 INFO:teuthology.orchestra.run.smithi159.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-04T06:06:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 9a:94:2c:b8:7b:11 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-04T06:06:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-04T06:06:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:22.085 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::9894:2cff:feb8:7b11/64 scope link 2024-04-04T06:06:22.086 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:22.089 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-04T06:06:22.090 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:06:22.090 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:06:22.090 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-04T06:06:22.090 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-04T06:06:22.090 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-04T06:06:22.090 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-04T06:06:22.090 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-04T06:06:22.090 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:06:22.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:22.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:22.416 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:06:22.416 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:06:22.416 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set brx.1 up 2024-04-04T06:06:22.416 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set dev brx.1 master ceph-brx 2024-04-04T06:06:22.416 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:06:22.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:22.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:22.645 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:06:22.645 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-04T06:06:22.689 INFO:teuthology.orchestra.run.smithi159.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-04T06:06:22.690 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:06:22.690 DEBUG:teuthology.orchestra.run.smithi159:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-04T06:06:22.783 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-04T06:06:22.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:22.784 DEBUG:teuthology.orchestra.run.smithi159:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-04T06:06:22.914 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-04T06:06:22.914 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-04T06:06:22.914 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:06:22.933 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-04T06:06:22.934 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-04T06:06:22.934 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:06:22.934 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:06:22.934 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:06:22.962 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-04T06:06:22.963 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-04T06:06:22.963 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:06:22.963 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:06:22.963 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:06:22.963 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-04T06:06:22.963 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-04T06:06:22.963 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=30311045-f714-48fa-832b-4cb2ba8bf0bf" will pass to kernel 2024-04-04T06:06:22.964 INFO:teuthology.orchestra.run.smithi159.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-04T06:06:22.965 INFO:teuthology.orchestra.run.smithi159.stdout: You just mounted a file system that supports labels which does not 2024-04-04T06:06:22.965 INFO:teuthology.orchestra.run.smithi159.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-04T06:06:22.965 INFO:teuthology.orchestra.run.smithi159.stdout: applications will generate AVC messages and not be allowed access to 2024-04-04T06:06:22.965 INFO:teuthology.orchestra.run.smithi159.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-04T06:06:22.970 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-04T06:06:22.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:22.970 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-04T06:06:23.050 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:06:23.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:06:23.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:23.167 INFO:teuthology.orchestra.run.smithi159.stdout:client4436 2024-04-04T06:06:23.168 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:06:23.168 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:06:23.220 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:06:23.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:06:23.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:23.339 INFO:teuthology.orchestra.run.smithi159.stdout:client4436 2024-04-04T06:06:23.339 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:06:23.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:06:23.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:23.457 INFO:teuthology.orchestra.run.smithi159.stdout:client4436 2024-04-04T06:06:23.458 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/30311045-f714-48fa-832b-4cb2ba8bf0bf.client4436/status') 2024-04-04T06:06:23.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:23.595 INFO:teuthology.orchestra.run.smithi159.stdout:instance: client.4436 (3)172.21.15.159:0/696552485 2024-04-04T06:06:23.595 INFO:teuthology.orchestra.run.smithi159.stdout:blocklisted: false 2024-04-04T06:06:23.595 INFO:teuthology.orchestra.run.smithi159.stderr:0+1 records in 2024-04-04T06:06:23.595 INFO:teuthology.orchestra.run.smithi159.stderr:0+1 records out 2024-04-04T06:06:23.595 INFO:teuthology.orchestra.run.smithi159.stderr:70 bytes copied, 7.2943e-05 s, 960 kB/s 2024-04-04T06:06:23.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:23.608 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-04T06:06:23.608 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-04T06:06:23.608 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-04T06:06:23.609 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-04T06:06:23.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.821+0000 7f4ecfe00640 1 Processor -- start 2024-04-04T06:06:23.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.821+0000 7f4ecfe00640 1 -- start start 2024-04-04T06:06:23.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.822+0000 7f4ecfe00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4ec8153580 0x7f4ec80d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:23.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.822+0000 7f4ecfe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4ec80d07c0 0x7f4ec80aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:23.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.822+0000 7f4ecfe00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ec80afab0 con 0x7f4ec80af2e0 2024-04-04T06:06:23.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.822+0000 7f4ecfe00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f4ec80d0c30 con 0x7f4ec80d07c0 2024-04-04T06:06:23.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.822+0000 7f4ecfe00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4ec815cc80 con 0x7f4ec8153580 2024-04-04T06:06:23.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.822+0000 7f4ecca00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4ec80d07c0 0x7f4ec80aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:23.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.822+0000 7f4ecca00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4ec80d07c0 0x7f4ec80aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40750/0 (socket says 172.21.15.159:40750) 2024-04-04T06:06:23.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.822+0000 7f4ecca00640 1 -- 172.21.15.159:0/2492558829 learned_addr learned my addr 172.21.15.159:0/2492558829 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:23.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.823+0000 7f4ecd400640 1 --2- 172.21.15.159:0/2492558829 >> v2:172.21.15.178:3300/0 conn(0x7f4ec8153580 0x7f4ec80d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:23.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.823+0000 7f4ecca00640 1 -- 172.21.15.159:0/2492558829 >> v2:172.21.15.178:3300/0 conn(0x7f4ec8153580 msgr2=0x7f4ec80d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:23.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.823+0000 7f4ecca00640 1 --2- 172.21.15.159:0/2492558829 >> v2:172.21.15.178:3300/0 conn(0x7f4ec8153580 0x7f4ec80d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:23.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.823+0000 7f4ecca00640 1 -- 172.21.15.159:0/2492558829 >> v1:172.21.15.178:6789/0 conn(0x7f4ec80af2e0 legacy=0x7f4ec806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:23.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.823+0000 7f4ecca00640 1 -- 172.21.15.159:0/2492558829 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec815da00 con 0x7f4ec80d07c0 2024-04-04T06:06:23.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.824+0000 7f4ecca00640 1 --2- 172.21.15.159:0/2492558829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ec80d07c0 0x7f4ec80aeda0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80b5270 tx=0x7f4eb402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7dd081cc10df92c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:23.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.824+0000 7f4ec7e00640 1 -- 172.21.15.159:0/2492558829 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb402e070 con 0x7f4ec80d07c0 2024-04-04T06:06:23.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.824+0000 7f4ec7e00640 1 -- 172.21.15.159:0/2492558829 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb4037070 con 0x7f4ec80d07c0 2024-04-04T06:06:23.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.824+0000 7f4ec7e00640 1 -- 172.21.15.159:0/2492558829 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4031a00 con 0x7f4ec80d07c0 2024-04-04T06:06:23.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.825+0000 7f4ecfe00640 1 -- 172.21.15.159:0/2492558829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ec80d07c0 msgr2=0x7f4ec80aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:23.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.825+0000 7f4ecfe00640 1 --2- 172.21.15.159:0/2492558829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ec80d07c0 0x7f4ec80aeda0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80b5270 tx=0x7f4eb402b880 comp rx=0 tx=0).stop 2024-04-04T06:06:23.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.825+0000 7f4ecfe00640 1 -- 172.21.15.159:0/2492558829 shutdown_connections 2024-04-04T06:06:23.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.825+0000 7f4ecfe00640 1 --2- 172.21.15.159:0/2492558829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ec80d07c0 0x7f4ec80aeda0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:23.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.825+0000 7f4ecfe00640 1 --2- 172.21.15.159:0/2492558829 >> v2:172.21.15.178:3300/0 conn(0x7f4ec8153580 0x7f4ec80d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:23.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.825+0000 7f4ecfe00640 1 -- 172.21.15.159:0/2492558829 >> 172.21.15.159:0/2492558829 conn(0x7f4ec806b750 msgr2=0x7f4ec806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:23.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.825+0000 7f4ecfe00640 1 -- 172.21.15.159:0/2492558829 shutdown_connections 2024-04-04T06:06:23.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.825+0000 7f4ecfe00640 1 -- 172.21.15.159:0/2492558829 wait complete. 2024-04-04T06:06:23.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.827+0000 7f4ecfe00640 1 Processor -- start 2024-04-04T06:06:23.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.827+0000 7f4ecfe00640 1 -- start start 2024-04-04T06:06:23.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.828+0000 7f4ecfe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ec80af2e0 0x7f4ec80cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:23.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.828+0000 7f4ecfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ec80d07c0 0x7f4ec8156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:23.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.828+0000 7f4ecfe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ec8153580 0x7f4ec80cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:23.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.828+0000 7f4ecfe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec815d4e0 con 0x7f4ec80d07c0 2024-04-04T06:06:23.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.828+0000 7f4ecfe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec815d200 con 0x7f4ec80af2e0 2024-04-04T06:06:23.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.828+0000 7f4ecfe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec8153960 con 0x7f4ec8153580 2024-04-04T06:06:23.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.828+0000 7f4ecca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ec80d07c0 0x7f4ec8156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:23.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.828+0000 7f4ecca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ec80d07c0 0x7f4ec8156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40764/0 (socket says 172.21.15.159:40764) 2024-04-04T06:06:23.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.828+0000 7f4ecca00640 1 -- 172.21.15.159:0/1594592294 learned_addr learned my addr 172.21.15.159:0/1594592294 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:23.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.828+0000 7f4ecd400640 1 --2- 172.21.15.159:0/1594592294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ec80af2e0 0x7f4ec80cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:23.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.829+0000 7f4ecca00640 1 -- 172.21.15.159:0/1594592294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ec8153580 msgr2=0x7f4ec80cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:23.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.829+0000 7f4ecde00640 1 --2- 172.21.15.159:0/1594592294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ec8153580 0x7f4ec80cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:23.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.829+0000 7f4ecca00640 1 --2- 172.21.15.159:0/1594592294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ec8153580 0x7f4ec80cf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:23.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.829+0000 7f4ecca00640 1 -- 172.21.15.159:0/1594592294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ec80af2e0 msgr2=0x7f4ec80cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:23.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.829+0000 7f4ecca00640 1 --2- 172.21.15.159:0/1594592294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ec80af2e0 0x7f4ec80cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:23.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.829+0000 7f4ecca00640 1 -- 172.21.15.159:0/1594592294 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec80cca50 con 0x7f4ec80d07c0 2024-04-04T06:06:23.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.829+0000 7f4ecca00640 1 --2- 172.21.15.159:0/1594592294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ec80d07c0 0x7f4ec8156680 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4eb402b8c0 tx=0x7f4eb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:23.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.829+0000 7f4ec5600640 1 -- 172.21.15.159:0/1594592294 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4032880 con 0x7f4ec80d07c0 2024-04-04T06:06:23.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.829+0000 7f4ec5600640 1 -- 172.21.15.159:0/1594592294 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb4032a20 con 0x7f4ec80d07c0 2024-04-04T06:06:23.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.829+0000 7f4ec5600640 1 -- 172.21.15.159:0/1594592294 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4032e10 con 0x7f4ec80d07c0 2024-04-04T06:06:23.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.829+0000 7f4ecfe00640 1 -- 172.21.15.159:0/1594592294 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec80ccd30 con 0x7f4ec80d07c0 2024-04-04T06:06:23.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.830+0000 7f4ecfe00640 1 -- 172.21.15.159:0/1594592294 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec80cd2f0 con 0x7f4ec80d07c0 2024-04-04T06:06:23.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.831+0000 7f4ecfe00640 1 -- 172.21.15.159:0/1594592294 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec80685c0 con 0x7f4ec80d07c0 2024-04-04T06:06:23.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.832+0000 7f4ec5600640 1 -- 172.21.15.159:0/1594592294 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4eb40453f0 con 0x7f4ec80d07c0 2024-04-04T06:06:23.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.833+0000 7f4ec5600640 1 --2- 172.21.15.159:0/1594592294 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ea806c8f0 0x7f4ea806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:23.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.833+0000 7f4ec5600640 1 -- 172.21.15.159:0/1594592294 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4eb40adb60 con 0x7f4ec80d07c0 2024-04-04T06:06:23.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.834+0000 7f4ecd400640 1 --2- 172.21.15.159:0/1594592294 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ea806c8f0 0x7f4ea806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:23.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.835+0000 7f4ecd400640 1 --2- 172.21.15.159:0/1594592294 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ea806c8f0 0x7f4ea806edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc006fd0 tx=0x7f4ebc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:23.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:23.839+0000 7f4ec5600640 1 -- 172.21.15.159:0/1594592294 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4eb4040070 con 0x7f4ec80d07c0 2024-04-04T06:06:24.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:24.082+0000 7f4ecfe00640 1 -- 172.21.15.159:0/1594592294 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4ec806eca0 con 0x7f4ec80d07c0 2024-04-04T06:06:24.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:24.082+0000 7f4ec5600640 1 -- 172.21.15.159:0/1594592294 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f4eb407c010 con 0x7f4ec80d07c0 2024-04-04T06:06:24.077 INFO:teuthology.orchestra.run.smithi159.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-04T06:06:24.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:24.085+0000 7f4ecfe00640 1 -- 172.21.15.159:0/1594592294 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ea806c8f0 msgr2=0x7f4ea806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:24.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:24.085+0000 7f4ecfe00640 1 --2- 172.21.15.159:0/1594592294 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ea806c8f0 0x7f4ea806edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc006fd0 tx=0x7f4ebc008040 comp rx=0 tx=0).stop 2024-04-04T06:06:24.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:24.085+0000 7f4ecfe00640 1 -- 172.21.15.159:0/1594592294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ec80d07c0 msgr2=0x7f4ec8156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:24.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:24.085+0000 7f4ecfe00640 1 --2- 172.21.15.159:0/1594592294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ec80d07c0 0x7f4ec8156680 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4eb402b8c0 tx=0x7f4eb40046c0 comp rx=0 tx=0).stop 2024-04-04T06:06:24.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:24.085+0000 7f4ecfe00640 1 -- 172.21.15.159:0/1594592294 shutdown_connections 2024-04-04T06:06:24.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:24.085+0000 7f4ecfe00640 1 --2- 172.21.15.159:0/1594592294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ec8153580 0x7f4ec80cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:24.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:24.085+0000 7f4ecfe00640 1 --2- 172.21.15.159:0/1594592294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ec80d07c0 0x7f4ec8156680 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:24.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:24.085+0000 7f4ecfe00640 1 --2- 172.21.15.159:0/1594592294 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ea806c8f0 0x7f4ea806edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:24.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:24.085+0000 7f4ecfe00640 1 --2- 172.21.15.159:0/1594592294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ec80af2e0 0x7f4ec80cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:24.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:24.085+0000 7f4ecfe00640 1 -- 172.21.15.159:0/1594592294 >> 172.21.15.159:0/1594592294 conn(0x7f4ec806b750 msgr2=0x7f4ec806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:24.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:24.085+0000 7f4ecfe00640 1 -- 172.21.15.159:0/1594592294 shutdown_connections 2024-04-04T06:06:24.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:24.085+0000 7f4ecfe00640 1 -- 172.21.15.159:0/1594592294 wait complete. 2024-04-04T06:06:24.100 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-04T06:06:24.101 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-04T06:06:24.101 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi178.front.sepia.ceph.com 2024-04-04T06:06:24.101 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-04T06:06:24.101 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-04T06:06:24.101 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-04T06:06:24.101 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-04T06:06:24.101 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-04T06:06:24.101 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-04T06:06:24.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:24.101 DEBUG:teuthology.orchestra.run.smithi178:> ip addr 2024-04-04T06:06:24.158 INFO:teuthology.orchestra.run.smithi178.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:06:24.158 INFO:teuthology.orchestra.run.smithi178.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:06:24.158 INFO:teuthology.orchestra.run.smithi178.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:06:24.158 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:24.158 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 ::1/128 scope host 2024-04-04T06:06:24.159 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:24.159 INFO:teuthology.orchestra.run.smithi178.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:06:24.159 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:d9:3b:20 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:06:24.159 INFO:teuthology.orchestra.run.smithi178.stdout: altname enp7s0f0 2024-04-04T06:06:24.159 INFO:teuthology.orchestra.run.smithi178.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:06:24.159 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:d9:3b:21 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:06:24.159 INFO:teuthology.orchestra.run.smithi178.stdout: altname enp7s0f1 2024-04-04T06:06:24.159 INFO:teuthology.orchestra.run.smithi178.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:06:24.159 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:88:7e:f8 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:06:24.159 INFO:teuthology.orchestra.run.smithi178.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-04T06:06:24.159 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:88:7e:f9 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:06:24.159 INFO:teuthology.orchestra.run.smithi178.stdout: inet 172.21.15.178/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-04T06:06:24.159 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft 42575sec preferred_lft 42575sec 2024-04-04T06:06:24.159 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::ec4:7aff:fe88:7ef9/64 scope link 2024-04-04T06:06:24.160 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:24.160 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-04T06:06:24.160 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:06:24.160 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-04-04T06:06:24.160 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link add name ceph-brx type bridge 2024-04-04T06:06:24.160 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip addr flush dev ceph-brx 2024-04-04T06:06:24.160 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set ceph-brx up 2024-04-04T06:06:24.160 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-04T06:06:24.160 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-04-04T06:06:24.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:24.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:24.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:24.655 DEBUG:teuthology.orchestra.run.smithi178:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-04T06:06:24.731 INFO:teuthology.orchestra.run.smithi178.stdout:1 2024-04-04T06:06:24.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:24.734 DEBUG:teuthology.orchestra.run.smithi178:> route 2024-04-04T06:06:24.810 INFO:teuthology.orchestra.run.smithi178.stdout:Kernel IP routing table 2024-04-04T06:06:24.811 INFO:teuthology.orchestra.run.smithi178.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-04T06:06:24.811 INFO:teuthology.orchestra.run.smithi178.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-04T06:06:24.811 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-04T06:06:24.811 INFO:teuthology.orchestra.run.smithi178.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-04T06:06:24.812 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:06:24.812 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-04-04T06:06:24.812 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-04T06:06:24.812 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-04T06:06:24.812 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-04T06:06:24.812 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-04-04T06:06:24.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:25.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:25.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:25.613 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2024-04-04T06:06:25.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:25.665 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list-id 2024-04-04T06:06:25.712 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:06:25.712 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-04-04T06:06:25.712 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-04T06:06:25.712 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-04T06:06:25.712 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-04-04T06:06:25.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:25.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:25.902 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-04T06:06:25.902 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:06:25.902 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-04-04T06:06:25.902 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-04T06:06:25.902 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-04T06:06:25.902 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-04T06:06:25.902 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-04T06:06:25.902 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-04T06:06:25.903 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-04-04T06:06:26.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:26.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:26.235 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:06:26.236 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-04-04T06:06:26.236 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set brx.0 up 2024-04-04T06:06:26.236 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set dev brx.0 master ceph-brx 2024-04-04T06:06:26.236 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-04-04T06:06:26.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:26.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:26.450 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:06:26.450 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-04T06:06:26.493 INFO:teuthology.orchestra.run.smithi178.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-04T06:06:26.493 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:06:26.493 DEBUG:teuthology.orchestra.run.smithi178:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-04T06:06:26.576 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-04T06:06:26.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:26.577 DEBUG:teuthology.orchestra.run.smithi178:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-04T06:06:26.741 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-04T06:06:26.742 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-04T06:06:26.742 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2024-04-04T06:06:27.380 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-04T06:06:27.380 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-04T06:06:27.380 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2024-04-04T06:06:27.380 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:06:27.381 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:06:27.412 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-04T06:06:27.412 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-04T06:06:27.412 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2024-04-04T06:06:27.412 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:06:27.413 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:06:27.413 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-04T06:06:27.413 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-04T06:06:27.413 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=30311045-f714-48fa-832b-4cb2ba8bf0bf" will pass to kernel 2024-04-04T06:06:27.416 INFO:teuthology.orchestra.run.smithi178.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-04T06:06:27.417 INFO:teuthology.orchestra.run.smithi178.stdout: You just mounted a file system that supports labels which does not 2024-04-04T06:06:27.417 INFO:teuthology.orchestra.run.smithi178.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-04T06:06:27.417 INFO:teuthology.orchestra.run.smithi178.stdout: applications will generate AVC messages and not be allowed access to 2024-04-04T06:06:27.417 INFO:teuthology.orchestra.run.smithi178.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-04T06:06:27.420 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-04T06:06:27.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:27.421 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-04T06:06:27.503 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:06:27.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:06:27.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:27.616 INFO:teuthology.orchestra.run.smithi178.stdout:client4555 2024-04-04T06:06:27.616 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:06:27.616 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:06:27.671 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:06:27.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:06:27.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:27.780 INFO:teuthology.orchestra.run.smithi178.stdout:client4555 2024-04-04T06:06:27.780 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:06:27.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:06:27.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:27.894 INFO:teuthology.orchestra.run.smithi178.stdout:client4555 2024-04-04T06:06:27.895 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/30311045-f714-48fa-832b-4cb2ba8bf0bf.client4555/status') 2024-04-04T06:06:27.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:28.030 INFO:teuthology.orchestra.run.smithi178.stdout:instance: client.4555 (3)192.168.0.1:0/4241860897 2024-04-04T06:06:28.030 INFO:teuthology.orchestra.run.smithi178.stdout:blocklisted: false 2024-04-04T06:06:28.030 INFO:teuthology.orchestra.run.smithi178.stderr:0+1 records in 2024-04-04T06:06:28.031 INFO:teuthology.orchestra.run.smithi178.stderr:0+1 records out 2024-04-04T06:06:28.031 INFO:teuthology.orchestra.run.smithi178.stderr:69 bytes copied, 6.3233e-05 s, 1.1 MB/s 2024-04-04T06:06:28.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:28.041 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-04T06:06:28.041 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-04T06:06:28.041 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-04T06:06:28.042 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-04T06:06:28.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.265+0000 7eff8e600640 1 Processor -- start 2024-04-04T06:06:28.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.265+0000 7eff8e600640 1 -- start start 2024-04-04T06:06:28.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.266+0000 7eff8e600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7eff88155550 0x7eff88155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:28.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.266+0000 7eff8e600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff880b40b0 con 0x7eff880d0990 2024-04-04T06:06:28.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.266+0000 7eff8e600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff880b4390 con 0x7eff8815f800 2024-04-04T06:06:28.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.266+0000 7eff8e600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7eff880b4670 con 0x7eff88155550 2024-04-04T06:06:28.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.267+0000 7eff87400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7eff88155550 0x7eff88155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:28.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.267+0000 7eff87e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7eff880d0990 0x7eff880aede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58342/0 (socket says 172.21.15.159:58342) 2024-04-04T06:06:28.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.267+0000 7eff87e00640 1 -- 172.21.15.159:0/3369088230 learned_addr learned my addr 172.21.15.159:0/3369088230 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:28.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.267+0000 7eff87400640 1 -- 172.21.15.159:0/3369088230 >> v1:172.21.15.178:6789/0 conn(0x7eff8815f800 legacy=0x7eff880d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:28.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.267+0000 7eff87400640 1 -- 172.21.15.159:0/3369088230 >> v1:172.21.15.159:6789/0 conn(0x7eff880d0990 legacy=0x7eff880aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:28.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.267+0000 7eff87400640 1 -- 172.21.15.159:0/3369088230 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff880b4950 con 0x7eff88155550 2024-04-04T06:06:28.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.268+0000 7eff87400640 1 --2- 172.21.15.159:0/3369088230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff88155550 0x7eff88155930 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7eff78009d90 tx=0x7eff7802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1e2c9f7b91435f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:28.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.268+0000 7eff86000640 1 -- 172.21.15.159:0/3369088230 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff78039070 con 0x7eff88155550 2024-04-04T06:06:28.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.268+0000 7eff86000640 1 -- 172.21.15.159:0/3369088230 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff7802cbd0 con 0x7eff88155550 2024-04-04T06:06:28.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.268+0000 7eff86000640 1 -- 172.21.15.159:0/3369088230 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff78034440 con 0x7eff88155550 2024-04-04T06:06:28.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.269+0000 7eff8e600640 1 -- 172.21.15.159:0/3369088230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff88155550 msgr2=0x7eff88155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:28.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.269+0000 7eff8e600640 1 --2- 172.21.15.159:0/3369088230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff88155550 0x7eff88155930 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7eff78009d90 tx=0x7eff7802be20 comp rx=0 tx=0).stop 2024-04-04T06:06:28.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.269+0000 7eff8e600640 1 -- 172.21.15.159:0/3369088230 shutdown_connections 2024-04-04T06:06:28.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.269+0000 7eff8e600640 1 --2- 172.21.15.159:0/3369088230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff88155550 0x7eff88155930 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:28.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.269+0000 7eff8e600640 1 -- 172.21.15.159:0/3369088230 >> 172.21.15.159:0/3369088230 conn(0x7eff8806b750 msgr2=0x7eff8806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:28.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.269+0000 7eff8e600640 1 -- 172.21.15.159:0/3369088230 shutdown_connections 2024-04-04T06:06:28.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.269+0000 7eff8e600640 1 -- 172.21.15.159:0/3369088230 wait complete. 2024-04-04T06:06:28.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.271+0000 7eff8e600640 1 Processor -- start 2024-04-04T06:06:28.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.271+0000 7eff8e600640 1 -- start start 2024-04-04T06:06:28.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.273+0000 7eff8e600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff880d0990 0x7eff8809db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:28.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.273+0000 7eff8e600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff88155550 0x7eff880b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:28.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.273+0000 7eff8e600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff8815f800 0x7eff880b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:28.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.273+0000 7eff8e600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7eff8815ba20 con 0x7eff88155550 2024-04-04T06:06:28.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.273+0000 7eff8e600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7eff8815fc80 con 0x7eff8815f800 2024-04-04T06:06:28.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.273+0000 7eff8e600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7eff8815bba0 con 0x7eff880d0990 2024-04-04T06:06:28.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.273+0000 7eff86a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff88155550 0x7eff880b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:28.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.273+0000 7eff86a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff88155550 0x7eff880b28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45264/0 (socket says 172.21.15.159:45264) 2024-04-04T06:06:28.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.273+0000 7eff86a00640 1 -- 172.21.15.159:0/951247333 learned_addr learned my addr 172.21.15.159:0/951247333 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:28.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.273+0000 7eff87400640 1 --2- 172.21.15.159:0/951247333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff880d0990 0x7eff8809db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:28.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.273+0000 7eff87e00640 1 --2- 172.21.15.159:0/951247333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff8815f800 0x7eff880b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:28.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.273+0000 7eff86a00640 1 -- 172.21.15.159:0/951247333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff880d0990 msgr2=0x7eff8809db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:28.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.273+0000 7eff86a00640 1 --2- 172.21.15.159:0/951247333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff880d0990 0x7eff8809db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:28.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.273+0000 7eff86a00640 1 -- 172.21.15.159:0/951247333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff8815f800 msgr2=0x7eff880b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:28.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.273+0000 7eff86a00640 1 --2- 172.21.15.159:0/951247333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff8815f800 0x7eff880b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:28.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.273+0000 7eff86a00640 1 -- 172.21.15.159:0/951247333 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8809a230 con 0x7eff88155550 2024-04-04T06:06:28.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.273+0000 7eff86a00640 1 --2- 172.21.15.159:0/951247333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff88155550 0x7eff880b28b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7eff7400c460 tx=0x7eff7400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:28.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.274+0000 7eff7f400640 1 -- 172.21.15.159:0/951247333 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74016020 con 0x7eff88155550 2024-04-04T06:06:28.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.274+0000 7eff7f400640 1 -- 172.21.15.159:0/951247333 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff74012870 con 0x7eff88155550 2024-04-04T06:06:28.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.274+0000 7eff7f400640 1 -- 172.21.15.159:0/951247333 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74012c60 con 0x7eff88155550 2024-04-04T06:06:28.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.274+0000 7eff8e600640 1 -- 172.21.15.159:0/951247333 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff880b13e0 con 0x7eff88155550 2024-04-04T06:06:28.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.274+0000 7eff8e600640 1 -- 172.21.15.159:0/951247333 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff8809e090 con 0x7eff88155550 2024-04-04T06:06:28.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.275+0000 7eff8e600640 1 -- 172.21.15.159:0/951247333 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff880b0700 con 0x7eff88155550 2024-04-04T06:06:28.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.276+0000 7eff7f400640 1 -- 172.21.15.159:0/951247333 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff74019020 con 0x7eff88155550 2024-04-04T06:06:28.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.277+0000 7eff7f400640 1 --2- 172.21.15.159:0/951247333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7eff6806c8f0 0x7eff6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:28.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.277+0000 7eff7f400640 1 -- 172.21.15.159:0/951247333 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7eff7408b630 con 0x7eff88155550 2024-04-04T06:06:28.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.278+0000 7eff87400640 1 --2- 172.21.15.159:0/951247333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7eff6806c8f0 0x7eff6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:28.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.279+0000 7eff87400640 1 --2- 172.21.15.159:0/951247333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7eff6806c8f0 0x7eff6806edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7eff78036000 tx=0x7eff78030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:28.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.283+0000 7eff7f400640 1 -- 172.21.15.159:0/951247333 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7eff880b0700 con 0x7eff88155550 2024-04-04T06:06:28.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.528+0000 7eff8e600640 1 -- 172.21.15.159:0/951247333 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7eff880685c0 con 0x7eff88155550 2024-04-04T06:06:28.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.529+0000 7eff7f400640 1 -- 172.21.15.159:0/951247333 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7eff74020070 con 0x7eff88155550 2024-04-04T06:06:28.524 INFO:teuthology.orchestra.run.smithi159.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-04T06:06:28.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.532+0000 7eff8e600640 1 -- 172.21.15.159:0/951247333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7eff6806c8f0 msgr2=0x7eff6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:28.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.532+0000 7eff8e600640 1 --2- 172.21.15.159:0/951247333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7eff6806c8f0 0x7eff6806edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7eff78036000 tx=0x7eff78030000 comp rx=0 tx=0).stop 2024-04-04T06:06:28.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.532+0000 7eff8e600640 1 -- 172.21.15.159:0/951247333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff88155550 msgr2=0x7eff880b28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:28.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.532+0000 7eff8e600640 1 --2- 172.21.15.159:0/951247333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff88155550 0x7eff880b28b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7eff7400c460 tx=0x7eff7400c920 comp rx=0 tx=0).stop 2024-04-04T06:06:28.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.532+0000 7eff8e600640 1 -- 172.21.15.159:0/951247333 shutdown_connections 2024-04-04T06:06:28.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.532+0000 7eff8e600640 1 --2- 172.21.15.159:0/951247333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7eff6806c8f0 0x7eff6806edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:28.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.532+0000 7eff8e600640 1 --2- 172.21.15.159:0/951247333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff8815f800 0x7eff880b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:28.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.532+0000 7eff8e600640 1 --2- 172.21.15.159:0/951247333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff88155550 0x7eff880b28b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:28.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.532+0000 7eff8e600640 1 --2- 172.21.15.159:0/951247333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7eff880d0990 0x7eff8809db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:28.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.532+0000 7eff8e600640 1 -- 172.21.15.159:0/951247333 >> 172.21.15.159:0/951247333 conn(0x7eff8806b750 msgr2=0x7eff880cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:28.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.532+0000 7eff8e600640 1 -- 172.21.15.159:0/951247333 shutdown_connections 2024-04-04T06:06:28.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:28.532+0000 7eff8e600640 1 -- 172.21.15.159:0/951247333 wait complete. 2024-04-04T06:06:28.547 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-04T06:06:28.547 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-04T06:06:28.547 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi178.front.sepia.ceph.com 2024-04-04T06:06:28.547 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-04T06:06:28.547 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-04T06:06:28.547 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-04T06:06:28.547 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-04T06:06:28.547 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-04T06:06:28.547 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-04T06:06:28.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:28.547 DEBUG:teuthology.orchestra.run.smithi178:> ip addr 2024-04-04T06:06:28.601 INFO:teuthology.orchestra.run.smithi178.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:06:28.601 INFO:teuthology.orchestra.run.smithi178.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:06:28.601 INFO:teuthology.orchestra.run.smithi178.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:06:28.601 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:28.601 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 ::1/128 scope host 2024-04-04T06:06:28.601 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:28.602 INFO:teuthology.orchestra.run.smithi178.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:06:28.602 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:d9:3b:20 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:06:28.602 INFO:teuthology.orchestra.run.smithi178.stdout: altname enp7s0f0 2024-04-04T06:06:28.602 INFO:teuthology.orchestra.run.smithi178.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:06:28.602 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:d9:3b:21 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:06:28.602 INFO:teuthology.orchestra.run.smithi178.stdout: altname enp7s0f1 2024-04-04T06:06:28.602 INFO:teuthology.orchestra.run.smithi178.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:06:28.602 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:88:7e:f8 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:06:28.602 INFO:teuthology.orchestra.run.smithi178.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-04T06:06:28.602 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:88:7e:f9 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:06:28.602 INFO:teuthology.orchestra.run.smithi178.stdout: inet 172.21.15.178/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-04T06:06:28.602 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft 42571sec preferred_lft 42571sec 2024-04-04T06:06:28.602 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::ec4:7aff:fe88:7ef9/64 scope link 2024-04-04T06:06:28.602 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:28.603 INFO:teuthology.orchestra.run.smithi178.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-04T06:06:28.603 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 16:19:50:99:69:2d brd ff:ff:ff:ff:ff:ff 2024-04-04T06:06:28.603 INFO:teuthology.orchestra.run.smithi178.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-04T06:06:28.603 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:28.603 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::b87a:8dff:fe8f:e92c/64 scope link 2024-04-04T06:06:28.603 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:28.603 INFO:teuthology.orchestra.run.smithi178.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:06:28.603 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 16:19:50:99:69:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-04T06:06:28.603 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::1419:50ff:fe99:692d/64 scope link 2024-04-04T06:06:28.603 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:28.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:28.604 DEBUG:teuthology.orchestra.run.smithi178:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-04T06:06:28.689 INFO:teuthology.orchestra.run.smithi178.stdout:1 2024-04-04T06:06:28.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:28.693 DEBUG:teuthology.orchestra.run.smithi178:> route 2024-04-04T06:06:28.742 INFO:teuthology.orchestra.run.smithi178.stdout:Kernel IP routing table 2024-04-04T06:06:28.742 INFO:teuthology.orchestra.run.smithi178.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-04T06:06:28.742 INFO:teuthology.orchestra.run.smithi178.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-04T06:06:28.742 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-04T06:06:28.742 INFO:teuthology.orchestra.run.smithi178.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-04T06:06:28.743 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:06:28.743 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-04-04T06:06:28.743 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-04T06:06:28.743 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-04T06:06:28.743 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-04T06:06:28.743 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-04-04T06:06:28.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:28.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:28.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:28.978 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2024-04-04T06:06:29.022 INFO:teuthology.orchestra.run.smithi178.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-04T06:06:29.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:29.023 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list-id 2024-04-04T06:06:29.111 INFO:teuthology.orchestra.run.smithi178.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-04T06:06:29.112 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:06:29.112 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-04-04T06:06:29.112 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-04T06:06:29.112 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-04T06:06:29.112 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-04-04T06:06:29.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:29.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:29.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:29.314 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-04T06:06:29.398 INFO:teuthology.orchestra.run.smithi178.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:06:29.399 INFO:teuthology.orchestra.run.smithi178.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:06:29.399 INFO:teuthology.orchestra.run.smithi178.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:06:29.399 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:29.399 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 ::1/128 scope host 2024-04-04T06:06:29.399 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:29.399 INFO:teuthology.orchestra.run.smithi178.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-04T06:06:29.399 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether d2:35:64:62:aa:6a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-04T06:06:29.399 INFO:teuthology.orchestra.run.smithi178.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-04T06:06:29.399 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:29.399 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::d035:64ff:fe62:aa6a/64 scope link 2024-04-04T06:06:29.399 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:29.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:29.403 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-04T06:06:29.490 INFO:teuthology.orchestra.run.smithi178.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:06:29.490 INFO:teuthology.orchestra.run.smithi178.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:06:29.490 INFO:teuthology.orchestra.run.smithi178.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:06:29.490 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:29.490 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 ::1/128 scope host 2024-04-04T06:06:29.490 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:29.490 INFO:teuthology.orchestra.run.smithi178.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-04T06:06:29.490 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether d2:35:64:62:aa:6a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-04T06:06:29.491 INFO:teuthology.orchestra.run.smithi178.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-04T06:06:29.491 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:29.491 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::d035:64ff:fe62:aa6a/64 scope link 2024-04-04T06:06:29.491 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:06:29.494 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-04T06:06:29.495 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:06:29.495 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-04-04T06:06:29.495 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-04T06:06:29.495 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-04T06:06:29.495 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-04T06:06:29.495 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-04T06:06:29.495 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-04T06:06:29.495 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-04-04T06:06:29.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:29.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:29.809 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:06:29.809 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-04-04T06:06:29.809 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set brx.1 up 2024-04-04T06:06:29.809 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set dev brx.1 master ceph-brx 2024-04-04T06:06:29.809 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-04-04T06:06:29.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:30.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:30.024 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:06:30.024 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-04T06:06:30.064 INFO:teuthology.orchestra.run.smithi178.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-04T06:06:30.065 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:06:30.065 DEBUG:teuthology.orchestra.run.smithi178:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-04T06:06:30.158 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-04T06:06:30.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:30.159 DEBUG:teuthology.orchestra.run.smithi178:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-04T06:06:30.293 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-04T06:06:30.294 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-04T06:06:30.294 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2024-04-04T06:06:30.316 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-04T06:06:30.316 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-04T06:06:30.316 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2024-04-04T06:06:30.316 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:06:30.316 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:06:30.344 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-04T06:06:30.344 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-04T06:06:30.344 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2024-04-04T06:06:30.344 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:06:30.344 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:06:30.344 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-04T06:06:30.344 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-04T06:06:30.344 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=30311045-f714-48fa-832b-4cb2ba8bf0bf" will pass to kernel 2024-04-04T06:06:30.347 INFO:teuthology.orchestra.run.smithi178.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-04T06:06:30.347 INFO:teuthology.orchestra.run.smithi178.stdout: You just mounted a file system that supports labels which does not 2024-04-04T06:06:30.347 INFO:teuthology.orchestra.run.smithi178.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-04T06:06:30.347 INFO:teuthology.orchestra.run.smithi178.stdout: applications will generate AVC messages and not be allowed access to 2024-04-04T06:06:30.347 INFO:teuthology.orchestra.run.smithi178.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-04T06:06:30.351 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-04T06:06:30.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:30.351 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-04T06:06:30.429 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:06:30.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:06:30.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:30.541 INFO:teuthology.orchestra.run.smithi178.stdout:client4734 2024-04-04T06:06:30.541 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:06:30.541 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:06:30.587 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:06:30.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:06:30.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:30.751 INFO:teuthology.orchestra.run.smithi178.stdout:client4734 2024-04-04T06:06:30.751 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:06:30.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:06:30.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:30.860 INFO:teuthology.orchestra.run.smithi178.stdout:client4734 2024-04-04T06:06:30.860 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/30311045-f714-48fa-832b-4cb2ba8bf0bf.client4734/status') 2024-04-04T06:06:30.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:06:30.992 INFO:teuthology.orchestra.run.smithi178.stdout:instance: client.4734 (3)172.21.15.178:0/1929906417 2024-04-04T06:06:30.992 INFO:teuthology.orchestra.run.smithi178.stdout:blocklisted: false 2024-04-04T06:06:30.992 INFO:teuthology.orchestra.run.smithi178.stderr:0+1 records in 2024-04-04T06:06:30.992 INFO:teuthology.orchestra.run.smithi178.stderr:0+1 records out 2024-04-04T06:06:30.992 INFO:teuthology.orchestra.run.smithi178.stderr:71 bytes copied, 5.7015e-05 s, 1.2 MB/s 2024-04-04T06:06:30.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-04T06:06:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:06:31.004 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-04T06:06:31.023 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:06:31.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.248+0000 7fd835a00640 1 Processor -- start 2024-04-04T06:06:31.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.248+0000 7fd835a00640 1 -- start start 2024-04-04T06:06:31.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.249+0000 7fd835a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd830161b00 0x7fd830162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:31.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.249+0000 7fd835a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd8300ad120 0x7fd8300d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:31.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.249+0000 7fd835a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd830071b60 con 0x7fd8300ad580 2024-04-04T06:06:31.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.249+0000 7fd835a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fd830161ee0 con 0x7fd830161b00 2024-04-04T06:06:31.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.249+0000 7fd835a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd8300b3450 con 0x7fd8300ad120 2024-04-04T06:06:31.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.250+0000 7fd82ea00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd830161b00 0x7fd830162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:31.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.250+0000 7fd82ea00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd830161b00 0x7fd830162ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45278/0 (socket says 172.21.15.159:45278) 2024-04-04T06:06:31.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.250+0000 7fd82f400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fd8300ad580 0x7fd8300b3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58350/0 (socket says 172.21.15.159:58350) 2024-04-04T06:06:31.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.250+0000 7fd82ea00640 1 -- 172.21.15.159:0/3520541517 learned_addr learned my addr 172.21.15.159:0/3520541517 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:31.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.250+0000 7fd826000640 1 --2- 172.21.15.159:0/3520541517 >> v2:172.21.15.178:3300/0 conn(0x7fd8300ad120 0x7fd8300d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:31.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.250+0000 7fd82ea00640 1 -- 172.21.15.159:0/3520541517 >> v2:172.21.15.178:3300/0 conn(0x7fd8300ad120 msgr2=0x7fd8300d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:31.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.250+0000 7fd82ea00640 1 --2- 172.21.15.159:0/3520541517 >> v2:172.21.15.178:3300/0 conn(0x7fd8300ad120 0x7fd8300d0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:31.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.250+0000 7fd82ea00640 1 -- 172.21.15.159:0/3520541517 >> v1:172.21.15.159:6789/0 conn(0x7fd8300ad580 legacy=0x7fd8300b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:31.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.250+0000 7fd82ea00640 1 -- 172.21.15.159:0/3520541517 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd830071e40 con 0x7fd830161b00 2024-04-04T06:06:31.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.251+0000 7fd82ea00640 1 --2- 172.21.15.159:0/3520541517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd830161b00 0x7fd830162ab0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd83006ea10 tx=0x7fd82002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa003a0341ea875 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:31.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.252+0000 7fd82e000640 1 -- 172.21.15.159:0/3520541517 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820034070 con 0x7fd830161b00 2024-04-04T06:06:31.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.252+0000 7fd82e000640 1 -- 172.21.15.159:0/3520541517 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd820030860 con 0x7fd830161b00 2024-04-04T06:06:31.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.252+0000 7fd82e000640 1 -- 172.21.15.159:0/3520541517 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820030c30 con 0x7fd830161b00 2024-04-04T06:06:31.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.252+0000 7fd835a00640 1 -- 172.21.15.159:0/3520541517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd830161b00 msgr2=0x7fd830162ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:31.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.252+0000 7fd835a00640 1 --2- 172.21.15.159:0/3520541517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd830161b00 0x7fd830162ab0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd83006ea10 tx=0x7fd82002b9a0 comp rx=0 tx=0).stop 2024-04-04T06:06:31.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.252+0000 7fd835a00640 1 -- 172.21.15.159:0/3520541517 shutdown_connections 2024-04-04T06:06:31.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.252+0000 7fd835a00640 1 --2- 172.21.15.159:0/3520541517 >> v2:172.21.15.178:3300/0 conn(0x7fd8300ad120 0x7fd8300d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:31.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.252+0000 7fd835a00640 1 --2- 172.21.15.159:0/3520541517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd830161b00 0x7fd830162ab0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:31.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.252+0000 7fd835a00640 1 -- 172.21.15.159:0/3520541517 >> 172.21.15.159:0/3520541517 conn(0x7fd83006b990 msgr2=0x7fd83006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:31.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.252+0000 7fd835a00640 1 -- 172.21.15.159:0/3520541517 shutdown_connections 2024-04-04T06:06:31.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.253+0000 7fd835a00640 1 -- 172.21.15.159:0/3520541517 wait complete. 2024-04-04T06:06:31.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.254+0000 7fd835a00640 1 Processor -- start 2024-04-04T06:06:31.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.254+0000 7fd835a00640 1 -- start start 2024-04-04T06:06:31.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.255+0000 7fd835a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8300ad120 0x7fd8301527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:31.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.255+0000 7fd835a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8300ad580 0x7fd830159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:31.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.255+0000 7fd835a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd830161b00 0x7fd830159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:31.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.255+0000 7fd835a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd830162ff0 con 0x7fd8300ad120 2024-04-04T06:06:31.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.255+0000 7fd835a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd8300b3730 con 0x7fd830161b00 2024-04-04T06:06:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.255+0000 7fd835a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd8300717f0 con 0x7fd8300ad580 2024-04-04T06:06:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.255+0000 7fd82ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8300ad120 0x7fd8301527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.255+0000 7fd82ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8300ad120 0x7fd8301527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45292/0 (socket says 172.21.15.159:45292) 2024-04-04T06:06:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.255+0000 7fd82ea00640 1 -- 172.21.15.159:0/4170665583 learned_addr learned my addr 172.21.15.159:0/4170665583 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.256+0000 7fd826000640 1 --2- 172.21.15.159:0/4170665583 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8300ad580 0x7fd830159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.256+0000 7fd82ea00640 1 -- 172.21.15.159:0/4170665583 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8300ad580 msgr2=0x7fd830159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.256+0000 7fd82ea00640 1 --2- 172.21.15.159:0/4170665583 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8300ad580 0x7fd830159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.256+0000 7fd82f400640 1 --2- 172.21.15.159:0/4170665583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd830161b00 0x7fd830159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.256+0000 7fd82ea00640 1 -- 172.21.15.159:0/4170665583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd830161b00 msgr2=0x7fd830159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.256+0000 7fd82ea00640 1 --2- 172.21.15.159:0/4170665583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd830161b00 0x7fd830159550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.256+0000 7fd82ea00640 1 -- 172.21.15.159:0/4170665583 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd830155790 con 0x7fd8300ad120 2024-04-04T06:06:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.256+0000 7fd82f400640 1 --2- 172.21.15.159:0/4170665583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd830161b00 0x7fd830159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:31.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.256+0000 7fd82ea00640 1 --2- 172.21.15.159:0/4170665583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8300ad120 0x7fd8301527f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd820000c00 tx=0x7fd820004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:31.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.256+0000 7fd827400640 1 -- 172.21.15.159:0/4170665583 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820048070 con 0x7fd8300ad120 2024-04-04T06:06:31.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.257+0000 7fd827400640 1 -- 172.21.15.159:0/4170665583 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd820030660 con 0x7fd8300ad120 2024-04-04T06:06:31.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.257+0000 7fd827400640 1 -- 172.21.15.159:0/4170665583 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820034040 con 0x7fd8300ad120 2024-04-04T06:06:31.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.257+0000 7fd835a00640 1 -- 172.21.15.159:0/4170665583 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd830152d30 con 0x7fd8300ad120 2024-04-04T06:06:31.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.257+0000 7fd835a00640 1 -- 172.21.15.159:0/4170665583 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd830154cb0 con 0x7fd8300ad120 2024-04-04T06:06:31.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.258+0000 7fd835a00640 1 -- 172.21.15.159:0/4170665583 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8300685c0 con 0x7fd8300ad120 2024-04-04T06:06:31.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.259+0000 7fd827400640 1 -- 172.21.15.159:0/4170665583 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd82004b070 con 0x7fd8300ad120 2024-04-04T06:06:31.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.260+0000 7fd827400640 1 --2- 172.21.15.159:0/4170665583 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd80c06c8f0 0x7fd80c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:31.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.260+0000 7fd827400640 1 -- 172.21.15.159:0/4170665583 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd8200add90 con 0x7fd8300ad120 2024-04-04T06:06:31.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.260+0000 7fd826000640 1 --2- 172.21.15.159:0/4170665583 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd80c06c8f0 0x7fd80c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:31.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.261+0000 7fd826000640 1 --2- 172.21.15.159:0/4170665583 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd80c06c8f0 0x7fd80c06edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd818006fd0 tx=0x7fd818008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:31.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.264+0000 7fd827400640 1 -- 172.21.15.159:0/4170665583 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd82007c090 con 0x7fd8300ad120 2024-04-04T06:06:31.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.465+0000 7fd835a00640 1 -- 172.21.15.159:0/4170665583 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd83006f3a0 con 0x7fd8300ad120 2024-04-04T06:06:31.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.467+0000 7fd827400640 1 -- 172.21.15.159:0/4170665583 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16617 (secure 0 0 0) 0x7fd820030830 con 0x7fd8300ad120 2024-04-04T06:06:31.461 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:31.461 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":21,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:05:54.347275+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:05:51.297293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-04T06:05:51.770792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:06:31.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.469+0000 7fd835a00640 1 -- 172.21.15.159:0/4170665583 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd80c06c8f0 msgr2=0x7fd80c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:31.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.469+0000 7fd835a00640 1 --2- 172.21.15.159:0/4170665583 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd80c06c8f0 0x7fd80c06edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd818006fd0 tx=0x7fd818008040 comp rx=0 tx=0).stop 2024-04-04T06:06:31.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.469+0000 7fd835a00640 1 -- 172.21.15.159:0/4170665583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8300ad120 msgr2=0x7fd8301527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:31.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.469+0000 7fd835a00640 1 --2- 172.21.15.159:0/4170665583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8300ad120 0x7fd8301527f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd820000c00 tx=0x7fd820004730 comp rx=0 tx=0).stop 2024-04-04T06:06:31.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.470+0000 7fd835a00640 1 -- 172.21.15.159:0/4170665583 shutdown_connections 2024-04-04T06:06:31.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.470+0000 7fd835a00640 1 --2- 172.21.15.159:0/4170665583 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd80c06c8f0 0x7fd80c06edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:31.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.470+0000 7fd835a00640 1 --2- 172.21.15.159:0/4170665583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd830161b00 0x7fd830159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:31.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.470+0000 7fd835a00640 1 --2- 172.21.15.159:0/4170665583 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8300ad580 0x7fd830159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:31.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.470+0000 7fd835a00640 1 --2- 172.21.15.159:0/4170665583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8300ad120 0x7fd8301527f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:31.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.470+0000 7fd835a00640 1 -- 172.21.15.159:0/4170665583 >> 172.21.15.159:0/4170665583 conn(0x7fd83006b990 msgr2=0x7fd8300b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:31.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.470+0000 7fd835a00640 1 -- 172.21.15.159:0/4170665583 shutdown_connections 2024-04-04T06:06:31.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.470+0000 7fd835a00640 1 -- 172.21.15.159:0/4170665583 wait complete. 2024-04-04T06:06:31.484 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:05:51.297293+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:05:51.770792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:06:31.484 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:06:31.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.703+0000 7f47b9000640 1 Processor -- start 2024-04-04T06:06:31.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.704+0000 7f47b9000640 1 -- start start 2024-04-04T06:06:31.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.705+0000 7f47b9000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f47b40d0e70 0x7f47b40cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:31.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.705+0000 7f47b9000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f47b40d2110 0x7f47b40d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:31.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.705+0000 7f47b9000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47b40cf6c0 con 0x7f47b415f6e0 2024-04-04T06:06:31.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.705+0000 7f47b9000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f47b40cf9a0 con 0x7f47b40d2110 2024-04-04T06:06:31.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.705+0000 7f47b9000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f47b40cfc80 con 0x7f47b40d0e70 2024-04-04T06:06:31.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.705+0000 7f47b2800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f47b40d2110 0x7f47b40d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:31.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.705+0000 7f47b1400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f47b415f6e0 0x7f47b40d1250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58352/0 (socket says 172.21.15.159:58352) 2024-04-04T06:06:31.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.705+0000 7f47b2800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f47b40d2110 0x7f47b40d25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45298/0 (socket says 172.21.15.159:45298) 2024-04-04T06:06:31.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.705+0000 7f47b1400640 1 -- 172.21.15.159:0/1115098817 learned_addr learned my addr 172.21.15.159:0/1115098817 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:31.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.706+0000 7f47b1e00640 1 --2- 172.21.15.159:0/1115098817 >> v2:172.21.15.178:3300/0 conn(0x7f47b40d0e70 0x7f47b40cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:31.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.706+0000 7f47b2800640 1 -- 172.21.15.159:0/1115098817 >> v2:172.21.15.178:3300/0 conn(0x7f47b40d0e70 msgr2=0x7f47b40cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:31.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.706+0000 7f47b2800640 1 --2- 172.21.15.159:0/1115098817 >> v2:172.21.15.178:3300/0 conn(0x7f47b40d0e70 0x7f47b40cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:31.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.706+0000 7f47b2800640 1 -- 172.21.15.159:0/1115098817 >> v1:172.21.15.159:6789/0 conn(0x7f47b415f6e0 legacy=0x7f47b40d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:31.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.706+0000 7f47b2800640 1 -- 172.21.15.159:0/1115098817 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b40cff60 con 0x7f47b40d2110 2024-04-04T06:06:31.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.706+0000 7f47b1e00640 1 --2- 172.21.15.159:0/1115098817 >> v2:172.21.15.178:3300/0 conn(0x7f47b40d0e70 0x7f47b40cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:31.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.707+0000 7f47b2800640 1 --2- 172.21.15.159:0/1115098817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47b40d2110 0x7f47b40d25a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f47b408cf80 tx=0x7f47ac02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46cd6b6e155ba705 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:31.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.707+0000 7f47b0a00640 1 -- 172.21.15.159:0/1115098817 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac0315a0 con 0x7f47b40d2110 2024-04-04T06:06:31.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.707+0000 7f47b0a00640 1 -- 172.21.15.159:0/1115098817 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ac037070 con 0x7f47b40d2110 2024-04-04T06:06:31.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.707+0000 7f47b0a00640 1 -- 172.21.15.159:0/1115098817 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac031960 con 0x7f47b40d2110 2024-04-04T06:06:31.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.708+0000 7f47b9000640 1 -- 172.21.15.159:0/1115098817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47b40d2110 msgr2=0x7f47b40d25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:31.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.708+0000 7f47b9000640 1 --2- 172.21.15.159:0/1115098817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47b40d2110 0x7f47b40d25a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f47b408cf80 tx=0x7f47ac02b840 comp rx=0 tx=0).stop 2024-04-04T06:06:31.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.708+0000 7f47b9000640 1 -- 172.21.15.159:0/1115098817 shutdown_connections 2024-04-04T06:06:31.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.708+0000 7f47b9000640 1 --2- 172.21.15.159:0/1115098817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47b40d2110 0x7f47b40d25a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:31.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.708+0000 7f47b9000640 1 --2- 172.21.15.159:0/1115098817 >> v2:172.21.15.178:3300/0 conn(0x7f47b40d0e70 0x7f47b40cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:31.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.708+0000 7f47b9000640 1 -- 172.21.15.159:0/1115098817 >> 172.21.15.159:0/1115098817 conn(0x7f47b408bfb0 msgr2=0x7f47b40d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:31.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.708+0000 7f47b9000640 1 -- 172.21.15.159:0/1115098817 shutdown_connections 2024-04-04T06:06:31.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.708+0000 7f47b9000640 1 -- 172.21.15.159:0/1115098817 wait complete. 2024-04-04T06:06:31.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.709+0000 7f47b9000640 1 Processor -- start 2024-04-04T06:06:31.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.710+0000 7f47b9000640 1 -- start start 2024-04-04T06:06:31.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.710+0000 7f47b9000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47b40d0e70 0x7f47b41499e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:31.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.710+0000 7f47b9000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47b40d2110 0x7f47b406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:31.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.711+0000 7f47b9000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47b415f6e0 0x7f47b406cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:31.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.711+0000 7f47b9000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f47b415e910 con 0x7f47b40d0e70 2024-04-04T06:06:31.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.711+0000 7f47b9000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f47b415fb40 con 0x7f47b415f6e0 2024-04-04T06:06:31.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.711+0000 7f47b9000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f47b415ea90 con 0x7f47b40d2110 2024-04-04T06:06:31.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.711+0000 7f47b1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47b40d0e70 0x7f47b41499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:31.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.711+0000 7f47b1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47b40d0e70 0x7f47b41499e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45312/0 (socket says 172.21.15.159:45312) 2024-04-04T06:06:31.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.711+0000 7f47b1e00640 1 -- 172.21.15.159:0/2979364600 learned_addr learned my addr 172.21.15.159:0/2979364600 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:31.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.711+0000 7f47b2800640 1 --2- 172.21.15.159:0/2979364600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47b415f6e0 0x7f47b406cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:31.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.711+0000 7f47b1400640 1 --2- 172.21.15.159:0/2979364600 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47b40d2110 0x7f47b406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:31.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.711+0000 7f47b1e00640 1 -- 172.21.15.159:0/2979364600 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47b40d2110 msgr2=0x7f47b406c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:31.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.711+0000 7f47b1e00640 1 --2- 172.21.15.159:0/2979364600 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47b40d2110 0x7f47b406c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:31.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.711+0000 7f47b1e00640 1 -- 172.21.15.159:0/2979364600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47b415f6e0 msgr2=0x7f47b406cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:31.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.711+0000 7f47b1e00640 1 --2- 172.21.15.159:0/2979364600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47b415f6e0 0x7f47b406cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:31.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.711+0000 7f47b1e00640 1 -- 172.21.15.159:0/2979364600 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b40ca140 con 0x7f47b40d0e70 2024-04-04T06:06:31.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.712+0000 7f47b1e00640 1 --2- 172.21.15.159:0/2979364600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47b40d0e70 0x7f47b41499e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f47a400c7f0 tx=0x7f47a400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:31.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.712+0000 7f47aa000640 1 -- 172.21.15.159:0/2979364600 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4015070 con 0x7f47b40d0e70 2024-04-04T06:06:31.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.712+0000 7f47aa000640 1 -- 172.21.15.159:0/2979364600 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47a4007820 con 0x7f47b40d0e70 2024-04-04T06:06:31.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.712+0000 7f47aa000640 1 -- 172.21.15.159:0/2979364600 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4007bd0 con 0x7f47b40d0e70 2024-04-04T06:06:31.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.712+0000 7f47b9000640 1 -- 172.21.15.159:0/2979364600 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b40ca420 con 0x7f47b40d0e70 2024-04-04T06:06:31.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.712+0000 7f47b9000640 1 -- 172.21.15.159:0/2979364600 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b40ca9e0 con 0x7f47b40d0e70 2024-04-04T06:06:31.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.713+0000 7f47b9000640 1 -- 172.21.15.159:0/2979364600 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47b4068cd0 con 0x7f47b40d0e70 2024-04-04T06:06:31.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.714+0000 7f47aa000640 1 -- 172.21.15.159:0/2979364600 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47a4018070 con 0x7f47b40d0e70 2024-04-04T06:06:31.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.715+0000 7f47aa000640 1 --2- 172.21.15.159:0/2979364600 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f479006c8f0 0x7f479006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:31.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.716+0000 7f47aa000640 1 -- 172.21.15.159:0/2979364600 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f47a408bcd0 con 0x7f47b40d0e70 2024-04-04T06:06:31.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.716+0000 7f47b1400640 1 --2- 172.21.15.159:0/2979364600 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f479006c8f0 0x7f479006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:31.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.717+0000 7f47b1400640 1 --2- 172.21.15.159:0/2979364600 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f479006c8f0 0x7f479006edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f47b406d590 tx=0x7f479c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:31.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.720+0000 7f47aa000640 1 -- 172.21.15.159:0/2979364600 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f47a405a0f0 con 0x7f47b40d0e70 2024-04-04T06:06:31.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.919+0000 7f47b9000640 1 -- 172.21.15.159:0/2979364600 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f47b40d1b80 con 0x7f47b40d0e70 2024-04-04T06:06:31.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.920+0000 7f47aa000640 1 -- 172.21.15.159:0/2979364600 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f47a40599f0 con 0x7f47b40d0e70 2024-04-04T06:06:31.914 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:06:31.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.923+0000 7f47b9000640 1 -- 172.21.15.159:0/2979364600 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f479006c8f0 msgr2=0x7f479006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:31.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.923+0000 7f47b9000640 1 --2- 172.21.15.159:0/2979364600 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f479006c8f0 0x7f479006edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f47b406d590 tx=0x7f479c009040 comp rx=0 tx=0).stop 2024-04-04T06:06:31.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.923+0000 7f47b9000640 1 -- 172.21.15.159:0/2979364600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47b40d0e70 msgr2=0x7f47b41499e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:31.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.923+0000 7f47b9000640 1 --2- 172.21.15.159:0/2979364600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47b40d0e70 0x7f47b41499e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f47a400c7f0 tx=0x7f47a400ccb0 comp rx=0 tx=0).stop 2024-04-04T06:06:31.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.923+0000 7f47b9000640 1 -- 172.21.15.159:0/2979364600 shutdown_connections 2024-04-04T06:06:31.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.923+0000 7f47b9000640 1 --2- 172.21.15.159:0/2979364600 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f479006c8f0 0x7f479006edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:31.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.923+0000 7f47b9000640 1 --2- 172.21.15.159:0/2979364600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47b415f6e0 0x7f47b406cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:31.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.923+0000 7f47b9000640 1 --2- 172.21.15.159:0/2979364600 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47b40d2110 0x7f47b406c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:31.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.923+0000 7f47b9000640 1 --2- 172.21.15.159:0/2979364600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47b40d0e70 0x7f47b41499e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:31.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.923+0000 7f47b9000640 1 -- 172.21.15.159:0/2979364600 >> 172.21.15.159:0/2979364600 conn(0x7f47b408bfb0 msgr2=0x7f47b4160870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:31.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.923+0000 7f47b9000640 1 -- 172.21.15.159:0/2979364600 shutdown_connections 2024-04-04T06:06:31.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:31.923+0000 7f47b9000640 1 -- 172.21.15.159:0/2979364600 wait complete. 2024-04-04T06:06:31.936 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:06:32.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.135+0000 7f7565200640 1 Processor -- start 2024-04-04T06:06:32.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.135+0000 7f7565200640 1 -- start start 2024-04-04T06:06:32.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.136+0000 7f7565200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7560153580 0x7f75600d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:32.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.136+0000 7f7565200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75600af4b0 0x7f75601600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:32.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.136+0000 7f7565200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75600afab0 con 0x7f75600d07c0 2024-04-04T06:06:32.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.136+0000 7f7565200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f75600d0c30 con 0x7f7560153580 2024-04-04T06:06:32.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.137+0000 7f7565200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f756015cc80 con 0x7f75600af4b0 2024-04-04T06:06:32.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.137+0000 7f755de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7560153580 0x7f75600d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:32.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.137+0000 7f755de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7560153580 0x7f75600d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45322/0 (socket says 172.21.15.159:45322) 2024-04-04T06:06:32.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.137+0000 7f755de00640 1 -- 172.21.15.159:0/3718316384 learned_addr learned my addr 172.21.15.159:0/3718316384 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:32.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.137+0000 7f755de00640 1 -- 172.21.15.159:0/3718316384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75600af4b0 msgr2=0x7f75601600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:32.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.137+0000 7f755de00640 1 --2- 172.21.15.159:0/3718316384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75600af4b0 0x7f75601600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.137+0000 7f755de00640 1 -- 172.21.15.159:0/3718316384 >> v1:172.21.15.178:6789/0 conn(0x7f75600d07c0 legacy=0x7f75600aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:32.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.137+0000 7f755de00640 1 -- 172.21.15.159:0/3718316384 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756015da00 con 0x7f7560153580 2024-04-04T06:06:32.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.138+0000 7f755de00640 1 --2- 172.21.15.159:0/3718316384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7560153580 0x7f75600d0280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7550009870 tx=0x7f755002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88735806779f94c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:32.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.138+0000 7f755d400640 1 -- 172.21.15.159:0/3718316384 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75500317c0 con 0x7f7560153580 2024-04-04T06:06:32.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.138+0000 7f755d400640 1 -- 172.21.15.159:0/3718316384 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7550037070 con 0x7f7560153580 2024-04-04T06:06:32.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.139+0000 7f755d400640 1 -- 172.21.15.159:0/3718316384 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550031b80 con 0x7f7560153580 2024-04-04T06:06:32.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.139+0000 7f7565200640 1 -- 172.21.15.159:0/3718316384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7560153580 msgr2=0x7f75600d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:32.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.139+0000 7f7565200640 1 --2- 172.21.15.159:0/3718316384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7560153580 0x7f75600d0280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7550009870 tx=0x7f755002b860 comp rx=0 tx=0).stop 2024-04-04T06:06:32.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.139+0000 7f7565200640 1 -- 172.21.15.159:0/3718316384 shutdown_connections 2024-04-04T06:06:32.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.139+0000 7f7565200640 1 --2- 172.21.15.159:0/3718316384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75600af4b0 0x7f75601600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.139+0000 7f7565200640 1 --2- 172.21.15.159:0/3718316384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7560153580 0x7f75600d0280 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.139+0000 7f7565200640 1 -- 172.21.15.159:0/3718316384 >> 172.21.15.159:0/3718316384 conn(0x7f756006b750 msgr2=0x7f756006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:32.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.139+0000 7f7565200640 1 -- 172.21.15.159:0/3718316384 shutdown_connections 2024-04-04T06:06:32.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.139+0000 7f7565200640 1 -- 172.21.15.159:0/3718316384 wait complete. 2024-04-04T06:06:32.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.141+0000 7f7565200640 1 Processor -- start 2024-04-04T06:06:32.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.141+0000 7f7565200640 1 -- start start 2024-04-04T06:06:32.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.142+0000 7f7565200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75600af4b0 0x7f756014e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:32.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.143+0000 7f7565200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75600d07c0 0x7f756015a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:32.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.143+0000 7f7565200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7560153580 0x7f756015ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:32.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.143+0000 7f7565200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f756015d4e0 con 0x7f75600d07c0 2024-04-04T06:06:32.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.143+0000 7f7565200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f756015d200 con 0x7f7560153580 2024-04-04T06:06:32.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.143+0000 7f7565200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7560153960 con 0x7f75600af4b0 2024-04-04T06:06:32.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.143+0000 7f7555400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75600d07c0 0x7f756015a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:32.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.143+0000 7f7555400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75600d07c0 0x7f756015a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45338/0 (socket says 172.21.15.159:45338) 2024-04-04T06:06:32.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.143+0000 7f7555400640 1 -- 172.21.15.159:0/719148738 learned_addr learned my addr 172.21.15.159:0/719148738 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:32.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.143+0000 7f755e800640 1 --2- 172.21.15.159:0/719148738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7560153580 0x7f756015ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:32.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.143+0000 7f755de00640 1 --2- 172.21.15.159:0/719148738 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75600af4b0 0x7f756014e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:32.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.143+0000 7f7555400640 1 -- 172.21.15.159:0/719148738 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75600af4b0 msgr2=0x7f756014e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:32.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.143+0000 7f7555400640 1 --2- 172.21.15.159:0/719148738 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75600af4b0 0x7f756014e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.143+0000 7f7555400640 1 -- 172.21.15.159:0/719148738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7560153580 msgr2=0x7f756015ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:32.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.143+0000 7f7555400640 1 --2- 172.21.15.159:0/719148738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7560153580 0x7f756015ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.143+0000 7f7555400640 1 -- 172.21.15.159:0/719148738 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756014ecb0 con 0x7f75600d07c0 2024-04-04T06:06:32.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.143+0000 7f755e800640 1 --2- 172.21.15.159:0/719148738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7560153580 0x7f756015ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:32.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.144+0000 7f7555400640 1 --2- 172.21.15.159:0/719148738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75600d07c0 0x7f756015a700 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f754800b420 tx=0x7f754800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:32.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.144+0000 7f7556a00640 1 -- 172.21.15.159:0/719148738 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548011830 con 0x7f75600d07c0 2024-04-04T06:06:32.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.144+0000 7f7556a00640 1 -- 172.21.15.159:0/719148738 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7548016070 con 0x7f75600d07c0 2024-04-04T06:06:32.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.144+0000 7f7556a00640 1 -- 172.21.15.159:0/719148738 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548011c10 con 0x7f75600d07c0 2024-04-04T06:06:32.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.144+0000 7f7565200640 1 -- 172.21.15.159:0/719148738 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7560155c70 con 0x7f75600d07c0 2024-04-04T06:06:32.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.144+0000 7f7565200640 1 -- 172.21.15.159:0/719148738 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7560154eb0 con 0x7f75600d07c0 2024-04-04T06:06:32.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.145+0000 7f7565200640 1 -- 172.21.15.159:0/719148738 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75600685c0 con 0x7f75600d07c0 2024-04-04T06:06:32.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.146+0000 7f7556a00640 1 -- 172.21.15.159:0/719148738 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f754801b460 con 0x7f75600d07c0 2024-04-04T06:06:32.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.147+0000 7f7556a00640 1 --2- 172.21.15.159:0/719148738 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f753c06c8f0 0x7f753c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:32.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.147+0000 7f7556a00640 1 -- 172.21.15.159:0/719148738 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f754808b650 con 0x7f75600d07c0 2024-04-04T06:06:32.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.147+0000 7f755de00640 1 --2- 172.21.15.159:0/719148738 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f753c06c8f0 0x7f753c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:32.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.149+0000 7f755de00640 1 --2- 172.21.15.159:0/719148738 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f753c06c8f0 0x7f753c06edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7550036000 tx=0x7f7550008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:32.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.151+0000 7f7556a00640 1 -- 172.21.15.159:0/719148738 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7548020020 con 0x7f75600d07c0 2024-04-04T06:06:32.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.334+0000 7f7565200640 1 -- 172.21.15.159:0/719148738 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f75600af170 con 0x7f75600d07c0 2024-04-04T06:06:32.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.335+0000 7f7556a00640 1 -- 172.21.15.159:0/719148738 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f7548059a70 con 0x7f75600d07c0 2024-04-04T06:06:32.329 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:06:32.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.337+0000 7f7565200640 1 -- 172.21.15.159:0/719148738 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f753c06c8f0 msgr2=0x7f753c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:32.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.337+0000 7f7565200640 1 --2- 172.21.15.159:0/719148738 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f753c06c8f0 0x7f753c06edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7550036000 tx=0x7f7550008030 comp rx=0 tx=0).stop 2024-04-04T06:06:32.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.337+0000 7f7565200640 1 -- 172.21.15.159:0/719148738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75600d07c0 msgr2=0x7f756015a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:32.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.337+0000 7f7565200640 1 --2- 172.21.15.159:0/719148738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75600d07c0 0x7f756015a700 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f754800b420 tx=0x7f754800b8e0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.337+0000 7f7565200640 1 -- 172.21.15.159:0/719148738 shutdown_connections 2024-04-04T06:06:32.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.337+0000 7f7565200640 1 --2- 172.21.15.159:0/719148738 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f753c06c8f0 0x7f753c06edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.337+0000 7f7565200640 1 --2- 172.21.15.159:0/719148738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7560153580 0x7f756015ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.337+0000 7f7565200640 1 --2- 172.21.15.159:0/719148738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75600d07c0 0x7f756015a700 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.337+0000 7f7565200640 1 --2- 172.21.15.159:0/719148738 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75600af4b0 0x7f756014e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.337+0000 7f7565200640 1 -- 172.21.15.159:0/719148738 >> 172.21.15.159:0/719148738 conn(0x7f756006b750 msgr2=0x7f756015bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:32.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.337+0000 7f7565200640 1 -- 172.21.15.159:0/719148738 shutdown_connections 2024-04-04T06:06:32.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.338+0000 7f7565200640 1 -- 172.21.15.159:0/719148738 wait complete. 2024-04-04T06:06:32.350 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:06:32.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.553+0000 7fa57ce00640 1 Processor -- start 2024-04-04T06:06:32.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.553+0000 7fa57ce00640 1 -- start start 2024-04-04T06:06:32.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.554+0000 7fa57ce00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa5780b4890 0x7fa578072570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:32.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.555+0000 7fa57ce00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa57806d620 0x7fa578072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:32.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.555+0000 7fa57ce00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5780b4c70 con 0x7fa57815f6a0 2024-04-04T06:06:32.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.555+0000 7fa57ce00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fa5780b3eb0 con 0x7fa5780b4890 2024-04-04T06:06:32.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.555+0000 7fa57ce00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa57815fb30 con 0x7fa57806d620 2024-04-04T06:06:32.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.555+0000 7fa577400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa5780b4890 0x7fa578072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:32.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.555+0000 7fa577400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa5780b4890 0x7fa578072570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45350/0 (socket says 172.21.15.159:45350) 2024-04-04T06:06:32.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.555+0000 7fa56fe00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fa57815f6a0 0x7fa5780713e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58362/0 (socket says 172.21.15.159:58362) 2024-04-04T06:06:32.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.555+0000 7fa577400640 1 -- 172.21.15.159:0/3901831490 learned_addr learned my addr 172.21.15.159:0/3901831490 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:32.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.555+0000 7fa577e00640 1 --2- 172.21.15.159:0/3901831490 >> v2:172.21.15.178:3300/0 conn(0x7fa57806d620 0x7fa578072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:32.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.555+0000 7fa577400640 1 -- 172.21.15.159:0/3901831490 >> v2:172.21.15.178:3300/0 conn(0x7fa57806d620 msgr2=0x7fa578072ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:32.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.555+0000 7fa577400640 1 --2- 172.21.15.159:0/3901831490 >> v2:172.21.15.178:3300/0 conn(0x7fa57806d620 0x7fa578072ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.555+0000 7fa577400640 1 -- 172.21.15.159:0/3901831490 >> v1:172.21.15.159:6789/0 conn(0x7fa57815f6a0 legacy=0x7fa5780713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:32.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.555+0000 7fa577400640 1 -- 172.21.15.159:0/3901831490 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57815b7b0 con 0x7fa5780b4890 2024-04-04T06:06:32.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.556+0000 7fa577400640 1 --2- 172.21.15.159:0/3901831490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5780b4890 0x7fa578072570 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa578159270 tx=0x7fa56802b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bbe208bef9ec33b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:32.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.557+0000 7fa576a00640 1 -- 172.21.15.159:0/3901831490 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568034070 con 0x7fa5780b4890 2024-04-04T06:06:32.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.557+0000 7fa576a00640 1 -- 172.21.15.159:0/3901831490 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa568030870 con 0x7fa5780b4890 2024-04-04T06:06:32.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.557+0000 7fa576a00640 1 -- 172.21.15.159:0/3901831490 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568030c40 con 0x7fa5780b4890 2024-04-04T06:06:32.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.557+0000 7fa57ce00640 1 -- 172.21.15.159:0/3901831490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5780b4890 msgr2=0x7fa578072570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:32.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.557+0000 7fa57ce00640 1 --2- 172.21.15.159:0/3901831490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5780b4890 0x7fa578072570 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa578159270 tx=0x7fa56802b9b0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.557+0000 7fa57ce00640 1 -- 172.21.15.159:0/3901831490 shutdown_connections 2024-04-04T06:06:32.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.557+0000 7fa57ce00640 1 --2- 172.21.15.159:0/3901831490 >> v2:172.21.15.178:3300/0 conn(0x7fa57806d620 0x7fa578072ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.557+0000 7fa57ce00640 1 --2- 172.21.15.159:0/3901831490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5780b4890 0x7fa578072570 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.557+0000 7fa57ce00640 1 -- 172.21.15.159:0/3901831490 >> 172.21.15.159:0/3901831490 conn(0x7fa5780b1ba0 msgr2=0x7fa5780b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:32.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.557+0000 7fa57ce00640 1 -- 172.21.15.159:0/3901831490 shutdown_connections 2024-04-04T06:06:32.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.558+0000 7fa57ce00640 1 -- 172.21.15.159:0/3901831490 wait complete. 2024-04-04T06:06:32.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.560+0000 7fa57ce00640 1 Processor -- start 2024-04-04T06:06:32.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.560+0000 7fa57ce00640 1 -- start start 2024-04-04T06:06:32.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.561+0000 7fa57ce00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa57806d620 0x7fa57814d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:32.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.561+0000 7fa57ce00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5780b4890 0x7fa57814fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:32.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.561+0000 7fa57ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa578146480 0x7fa57813d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:32.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.561+0000 7fa57ce00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa57815c050 con 0x7fa578146480 2024-04-04T06:06:32.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.561+0000 7fa57ce00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa57815bd70 con 0x7fa57806d620 2024-04-04T06:06:32.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.561+0000 7fa57ce00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa57815c570 con 0x7fa5780b4890 2024-04-04T06:06:32.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.561+0000 7fa577e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa578146480 0x7fa57813d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:32.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.561+0000 7fa577e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa578146480 0x7fa57813d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45352/0 (socket says 172.21.15.159:45352) 2024-04-04T06:06:32.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.561+0000 7fa577e00640 1 -- 172.21.15.159:0/1804998716 learned_addr learned my addr 172.21.15.159:0/1804998716 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:32.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.562+0000 7fa577e00640 1 -- 172.21.15.159:0/1804998716 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5780b4890 msgr2=0x7fa57814fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:32.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.562+0000 7fa577e00640 1 --2- 172.21.15.159:0/1804998716 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5780b4890 0x7fa57814fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.562+0000 7fa577e00640 1 -- 172.21.15.159:0/1804998716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa57806d620 msgr2=0x7fa57814d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:32.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.562+0000 7fa577e00640 1 --2- 172.21.15.159:0/1804998716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa57806d620 0x7fa57814d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.562+0000 7fa577e00640 1 -- 172.21.15.159:0/1804998716 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa578152d70 con 0x7fa578146480 2024-04-04T06:06:32.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.562+0000 7fa577e00640 1 --2- 172.21.15.159:0/1804998716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa578146480 0x7fa57813d9d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa57000cc30 tx=0x7fa57000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:32.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.562+0000 7fa56f400640 1 -- 172.21.15.159:0/1804998716 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57000ce40 con 0x7fa578146480 2024-04-04T06:06:32.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.562+0000 7fa56f400640 1 -- 172.21.15.159:0/1804998716 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa570003990 con 0x7fa578146480 2024-04-04T06:06:32.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.562+0000 7fa57ce00640 1 -- 172.21.15.159:0/1804998716 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa578146c80 con 0x7fa578146480 2024-04-04T06:06:32.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.562+0000 7fa56f400640 1 -- 172.21.15.159:0/1804998716 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570003d40 con 0x7fa578146480 2024-04-04T06:06:32.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.562+0000 7fa57ce00640 1 -- 172.21.15.159:0/1804998716 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5781532f0 con 0x7fa578146480 2024-04-04T06:06:32.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.563+0000 7fa57ce00640 1 -- 172.21.15.159:0/1804998716 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa578075940 con 0x7fa578146480 2024-04-04T06:06:32.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.564+0000 7fa56f400640 1 -- 172.21.15.159:0/1804998716 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa570003fa0 con 0x7fa578146480 2024-04-04T06:06:32.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.565+0000 7fa56f400640 1 --2- 172.21.15.159:0/1804998716 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa55406c8f0 0x7fa55406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:32.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.565+0000 7fa56f400640 1 -- 172.21.15.159:0/1804998716 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa57008c080 con 0x7fa578146480 2024-04-04T06:06:32.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.566+0000 7fa577400640 1 --2- 172.21.15.159:0/1804998716 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa55406c8f0 0x7fa55406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:32.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.567+0000 7fa577400640 1 --2- 172.21.15.159:0/1804998716 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa55406c8f0 0x7fa55406edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa568036000 tx=0x7fa568030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:32.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.569+0000 7fa56f400640 1 -- 172.21.15.159:0/1804998716 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa570014030 con 0x7fa578146480 2024-04-04T06:06:32.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.770+0000 7fa57ce00640 1 -- 172.21.15.159:0/1804998716 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa578072850 con 0x7fa578146480 2024-04-04T06:06:32.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.771+0000 7fa56f400640 1 -- 172.21.15.159:0/1804998716 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fa57005a410 con 0x7fa578146480 2024-04-04T06:06:32.765 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:06:32.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.774+0000 7fa57ce00640 1 -- 172.21.15.159:0/1804998716 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa55406c8f0 msgr2=0x7fa55406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:32.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.774+0000 7fa57ce00640 1 --2- 172.21.15.159:0/1804998716 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa55406c8f0 0x7fa55406edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa568036000 tx=0x7fa568030000 comp rx=0 tx=0).stop 2024-04-04T06:06:32.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.774+0000 7fa57ce00640 1 -- 172.21.15.159:0/1804998716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa578146480 msgr2=0x7fa57813d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:32.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.774+0000 7fa57ce00640 1 --2- 172.21.15.159:0/1804998716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa578146480 0x7fa57813d9d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa57000cc30 tx=0x7fa57000a5c0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.774+0000 7fa57ce00640 1 -- 172.21.15.159:0/1804998716 shutdown_connections 2024-04-04T06:06:32.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.774+0000 7fa57ce00640 1 --2- 172.21.15.159:0/1804998716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa578146480 0x7fa57813d9d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.774+0000 7fa57ce00640 1 --2- 172.21.15.159:0/1804998716 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5780b4890 0x7fa57814fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.774+0000 7fa57ce00640 1 --2- 172.21.15.159:0/1804998716 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa55406c8f0 0x7fa55406edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.774+0000 7fa57ce00640 1 --2- 172.21.15.159:0/1804998716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa57806d620 0x7fa57814d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.774+0000 7fa57ce00640 1 -- 172.21.15.159:0/1804998716 >> 172.21.15.159:0/1804998716 conn(0x7fa5780b1ba0 msgr2=0x7fa57806ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:32.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.774+0000 7fa57ce00640 1 -- 172.21.15.159:0/1804998716 shutdown_connections 2024-04-04T06:06:32.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.774+0000 7fa57ce00640 1 -- 172.21.15.159:0/1804998716 wait complete. 2024-04-04T06:06:32.786 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:06:32.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.985+0000 7f2c30e00640 1 Processor -- start 2024-04-04T06:06:32.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.985+0000 7f2c30e00640 1 -- start start 2024-04-04T06:06:32.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.986+0000 7f2c30e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2c2c161b00 0x7f2c2c162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:32.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.986+0000 7f2c30e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c2c0d0d60 0x7f2c2c0b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:32.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.987+0000 7f2c30e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c2c071b60 con 0x7f2c2c0ad120 2024-04-04T06:06:32.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.987+0000 7f2c30e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2c2c161ee0 con 0x7f2c2c161b00 2024-04-04T06:06:32.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.987+0000 7f2c30e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2c2c0b3450 con 0x7f2c2c0d0d60 2024-04-04T06:06:32.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.987+0000 7f2c29400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f2c2c0ad120 0x7f2c2c0d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58370/0 (socket says 172.21.15.159:58370) 2024-04-04T06:06:32.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.987+0000 7f2c29400640 1 -- 172.21.15.159:0/2514015240 learned_addr learned my addr 172.21.15.159:0/2514015240 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:32.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.987+0000 7f2c29e00640 1 --2- 172.21.15.159:0/2514015240 >> v2:172.21.15.178:3300/0 conn(0x7f2c2c161b00 0x7f2c2c162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:32.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.987+0000 7f2c2a800640 1 --2- 172.21.15.159:0/2514015240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c2c0d0d60 0x7f2c2c0b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:32.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.987+0000 7f2c28a00640 1 -- 172.21.15.159:0/2514015240 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3329362614 0 0) 0x7f2c2c071b60 con 0x7f2c2c0ad120 2024-04-04T06:06:32.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.987+0000 7f2c28a00640 1 -- 172.21.15.159:0/2514015240 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c10003520 con 0x7f2c2c0ad120 2024-04-04T06:06:32.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.988+0000 7f2c28a00640 1 -- 172.21.15.159:0/2514015240 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1010228963 0 0) 0x7f2c10003520 con 0x7f2c2c0ad120 2024-04-04T06:06:32.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.988+0000 7f2c28a00640 1 -- 172.21.15.159:0/2514015240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c2c0d0d60 msgr2=0x7f2c2c0b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:32.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.988+0000 7f2c28a00640 1 --2- 172.21.15.159:0/2514015240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c2c0d0d60 0x7f2c2c0b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.988+0000 7f2c28a00640 1 -- 172.21.15.159:0/2514015240 >> v2:172.21.15.178:3300/0 conn(0x7f2c2c161b00 msgr2=0x7f2c2c162ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:32.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.988+0000 7f2c28a00640 1 --2- 172.21.15.159:0/2514015240 >> v2:172.21.15.178:3300/0 conn(0x7f2c2c161b00 0x7f2c2c162ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.988+0000 7f2c28a00640 1 -- 172.21.15.159:0/2514015240 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2c071e40 con 0x7f2c2c0ad120 2024-04-04T06:06:32.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.988+0000 7f2c28a00640 1 -- 172.21.15.159:0/2514015240 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f2c18001670 con 0x7f2c2c0ad120 2024-04-04T06:06:32.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.988+0000 7f2c28a00640 1 -- 172.21.15.159:0/2514015240 >> v1:172.21.15.159:6789/0 conn(0x7f2c2c0ad120 legacy=0x7f2c2c0d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:32.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.988+0000 7f2c28a00640 1 --2- 172.21.15.159:0/2514015240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c100039b0 0x7f2c10003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:32.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.988+0000 7f2c28a00640 1 --2- 172.21.15.159:0/2514015240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c10004800 0x7f2c10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:32.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.988+0000 7f2c28a00640 1 --2- 172.21.15.159:0/2514015240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c2c0d0d60 0x7f2c100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:32.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.988+0000 7f2c28a00640 1 -- 172.21.15.159:0/2514015240 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c161ee0 con 0x7f2c100039b0 2024-04-04T06:06:32.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.988+0000 7f2c28a00640 1 -- 172.21.15.159:0/2514015240 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c0b3450 con 0x7f2c10004800 2024-04-04T06:06:32.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.988+0000 7f2c28a00640 1 -- 172.21.15.159:0/2514015240 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2c1000b9c0 con 0x7f2c2c0d0d60 2024-04-04T06:06:32.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.988+0000 7f2c29e00640 1 --2- 172.21.15.159:0/2514015240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c100039b0 0x7f2c10003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:32.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.988+0000 7f2c29400640 1 --2- 172.21.15.159:0/2514015240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c10004800 0x7f2c10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:32.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.989+0000 7f2c29e00640 1 -- 172.21.15.159:0/2514015240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c2c0d0d60 msgr2=0x7f2c100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:32.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.989+0000 7f2c2a800640 1 --2- 172.21.15.159:0/2514015240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c2c0d0d60 0x7f2c100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:32.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.989+0000 7f2c29e00640 1 --2- 172.21.15.159:0/2514015240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c2c0d0d60 0x7f2c100051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.989+0000 7f2c29e00640 1 -- 172.21.15.159:0/2514015240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c10004800 msgr2=0x7f2c10004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:32.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.989+0000 7f2c29e00640 1 --2- 172.21.15.159:0/2514015240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c10004800 0x7f2c10004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.989+0000 7f2c29e00640 1 -- 172.21.15.159:0/2514015240 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c10003520 con 0x7f2c100039b0 2024-04-04T06:06:32.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.989+0000 7f2c29400640 1 --2- 172.21.15.159:0/2514015240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c10004800 0x7f2c10004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:32.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.990+0000 7f2c29e00640 1 --2- 172.21.15.159:0/2514015240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c100039b0 0x7f2c10003dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009ac0 tx=0x7f2c10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a52d89200c5acdb server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:32.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.990+0000 7f2c28a00640 1 -- 172.21.15.159:0/2514015240 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c039070 con 0x7f2c100039b0 2024-04-04T06:06:32.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.990+0000 7f2c28a00640 1 -- 172.21.15.159:0/2514015240 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c1c02c710 con 0x7f2c100039b0 2024-04-04T06:06:32.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.990+0000 7f2c28a00640 1 -- 172.21.15.159:0/2514015240 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c009690 con 0x7f2c100039b0 2024-04-04T06:06:32.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.990+0000 7f2c30e00640 1 -- 172.21.15.159:0/2514015240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c100039b0 msgr2=0x7f2c10003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:32.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.990+0000 7f2c30e00640 1 --2- 172.21.15.159:0/2514015240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c100039b0 0x7f2c10003dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009ac0 tx=0x7f2c10005830 comp rx=0 tx=0).stop 2024-04-04T06:06:32.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.990+0000 7f2c2a800640 1 -- 172.21.15.159:0/2514015240 reap_dead start 2024-04-04T06:06:32.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.990+0000 7f2c30e00640 1 -- 172.21.15.159:0/2514015240 shutdown_connections 2024-04-04T06:06:32.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.990+0000 7f2c30e00640 1 -- 172.21.15.159:0/2514015240 >> 172.21.15.159:0/2514015240 conn(0x7f2c2c06b990 msgr2=0x7f2c2c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:32.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.991+0000 7f2c30e00640 1 -- 172.21.15.159:0/2514015240 shutdown_connections 2024-04-04T06:06:32.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.991+0000 7f2c30e00640 1 -- 172.21.15.159:0/2514015240 wait complete. 2024-04-04T06:06:32.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.992+0000 7f2c30e00640 1 Processor -- start 2024-04-04T06:06:32.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.992+0000 7f2c30e00640 1 -- start start 2024-04-04T06:06:32.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.993+0000 7f2c30e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c100039b0 0x7f2c2c1f74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:32.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.993+0000 7f2c30e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c2c15f610 0x7f2c2c15fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:32.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.993+0000 7f2c30e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c2c1560d0 0x7f2c2c156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:32.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.993+0000 7f2c30e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c162ff0 con 0x7f2c100039b0 2024-04-04T06:06:32.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.993+0000 7f2c30e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c0b3730 con 0x7f2c2c1560d0 2024-04-04T06:06:32.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.993+0000 7f2c30e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2c2c0717f0 con 0x7f2c2c15f610 2024-04-04T06:06:32.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.993+0000 7f2c29e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c100039b0 0x7f2c2c1f74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:32.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.994+0000 7f2c29e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c100039b0 0x7f2c2c1f74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45368/0 (socket says 172.21.15.159:45368) 2024-04-04T06:06:32.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.994+0000 7f2c29e00640 1 -- 172.21.15.159:0/2339348338 learned_addr learned my addr 172.21.15.159:0/2339348338 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:32.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.994+0000 7f2c29e00640 1 -- 172.21.15.159:0/2339348338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c2c15f610 msgr2=0x7f2c2c15fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:32.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.994+0000 7f2c29e00640 1 --2- 172.21.15.159:0/2339348338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c2c15f610 0x7f2c2c15fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.994+0000 7f2c29e00640 1 -- 172.21.15.159:0/2339348338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c2c1560d0 msgr2=0x7f2c2c156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:32.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.994+0000 7f2c29e00640 1 --2- 172.21.15.159:0/2339348338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c2c1560d0 0x7f2c2c156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:32.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.994+0000 7f2c29e00640 1 -- 172.21.15.159:0/2339348338 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2c15eaf0 con 0x7f2c100039b0 2024-04-04T06:06:32.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.994+0000 7f2c29e00640 1 --2- 172.21.15.159:0/2339348338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c100039b0 0x7f2c2c1f74c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009be0 tx=0x7f2c1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:32.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.994+0000 7f2c22000640 1 -- 172.21.15.159:0/2339348338 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c042020 con 0x7f2c100039b0 2024-04-04T06:06:32.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.995+0000 7f2c22000640 1 -- 172.21.15.159:0/2339348338 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c1c03a040 con 0x7f2c100039b0 2024-04-04T06:06:32.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.995+0000 7f2c22000640 1 -- 172.21.15.159:0/2339348338 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c039070 con 0x7f2c100039b0 2024-04-04T06:06:32.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.995+0000 7f2c30e00640 1 -- 172.21.15.159:0/2339348338 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c2c156cb0 con 0x7f2c100039b0 2024-04-04T06:06:32.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.995+0000 7f2c30e00640 1 -- 172.21.15.159:0/2339348338 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c2c157270 con 0x7f2c100039b0 2024-04-04T06:06:32.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.996+0000 7f2c30e00640 1 -- 172.21.15.159:0/2339348338 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c2c0685c0 con 0x7f2c100039b0 2024-04-04T06:06:32.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.997+0000 7f2c22000640 1 -- 172.21.15.159:0/2339348338 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c1c045020 con 0x7f2c100039b0 2024-04-04T06:06:32.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.998+0000 7f2c22000640 1 --2- 172.21.15.159:0/2339348338 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2c0c06c920 0x7f2c0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:32.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.998+0000 7f2c22000640 1 -- 172.21.15.159:0/2339348338 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f2c1c0ad320 con 0x7f2c100039b0 2024-04-04T06:06:32.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.998+0000 7f2c29400640 1 --2- 172.21.15.159:0/2339348338 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2c0c06c920 0x7f2c0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:32.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:32.999+0000 7f2c29400640 1 --2- 172.21.15.159:0/2339348338 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2c0c06c920 0x7f2c0c06ede0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2c18001920 tx=0x7f2c18009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:32.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.003+0000 7f2c22000640 1 -- 172.21.15.159:0/2339348338 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2c1c04c030 con 0x7f2c100039b0 2024-04-04T06:06:33.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.203+0000 7f2c30e00640 1 -- 172.21.15.159:0/2339348338 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c2c06f3a0 con 0x7f2c100039b0 2024-04-04T06:06:33.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.206+0000 7f2c22000640 1 -- 172.21.15.159:0/2339348338 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16958 (secure 0 0 0) 0x7f2c1c07b600 con 0x7f2c100039b0 2024-04-04T06:06:33.200 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:33.201 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":22,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:06:32.788060+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:05:51.297293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-04T06:05:51.770792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":0,"to":2}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:06:33.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.209+0000 7f2c30e00640 1 -- 172.21.15.159:0/2339348338 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2c0c06c920 msgr2=0x7f2c0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:33.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.209+0000 7f2c30e00640 1 --2- 172.21.15.159:0/2339348338 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2c0c06c920 0x7f2c0c06ede0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2c18001920 tx=0x7f2c18009040 comp rx=0 tx=0).stop 2024-04-04T06:06:33.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.209+0000 7f2c30e00640 1 -- 172.21.15.159:0/2339348338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c100039b0 msgr2=0x7f2c2c1f74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:33.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.209+0000 7f2c30e00640 1 --2- 172.21.15.159:0/2339348338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c100039b0 0x7f2c2c1f74c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009be0 tx=0x7f2c1c004730 comp rx=0 tx=0).stop 2024-04-04T06:06:33.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.209+0000 7f2c30e00640 1 -- 172.21.15.159:0/2339348338 shutdown_connections 2024-04-04T06:06:33.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.209+0000 7f2c30e00640 1 --2- 172.21.15.159:0/2339348338 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2c0c06c920 0x7f2c0c06ede0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:33.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.209+0000 7f2c30e00640 1 --2- 172.21.15.159:0/2339348338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c2c1560d0 0x7f2c2c156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:33.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.209+0000 7f2c30e00640 1 --2- 172.21.15.159:0/2339348338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2c2c15f610 0x7f2c2c15fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:33.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.209+0000 7f2c30e00640 1 --2- 172.21.15.159:0/2339348338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c100039b0 0x7f2c2c1f74c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:33.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.209+0000 7f2c30e00640 1 -- 172.21.15.159:0/2339348338 >> 172.21.15.159:0/2339348338 conn(0x7f2c2c06b990 msgr2=0x7f2c2c0b0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:33.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.209+0000 7f2c30e00640 1 -- 172.21.15.159:0/2339348338 shutdown_connections 2024-04-04T06:06:33.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.209+0000 7f2c30e00640 1 -- 172.21.15.159:0/2339348338 wait complete. 2024-04-04T06:06:33.222 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:05:51.297293+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:05:51.770792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:06:33.222 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:06:33.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.448+0000 7f538a200640 1 Processor -- start 2024-04-04T06:06:33.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.448+0000 7f538a200640 1 -- start start 2024-04-04T06:06:33.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.450+0000 7f538a200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f53840d0990 0x7f53841670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:33.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.450+0000 7f538a200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5384163ac0 0x7f5384071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:33.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.450+0000 7f538a200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5384071d60 con 0x7f5384162920 2024-04-04T06:06:33.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.450+0000 7f538a200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f5384163f30 con 0x7f53840d0990 2024-04-04T06:06:33.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.450+0000 7f538a200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f53841665d0 con 0x7f5384163ac0 2024-04-04T06:06:33.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.450+0000 7f5383400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f5384162920 0x7f53840d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58372/0 (socket says 172.21.15.159:58372) 2024-04-04T06:06:33.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.450+0000 7f5383400640 1 -- 172.21.15.159:0/895674049 learned_addr learned my addr 172.21.15.159:0/895674049 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:33.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.450+0000 7f5382a00640 1 --2- 172.21.15.159:0/895674049 >> v2:172.21.15.159:3300/0 conn(0x7f53840d0990 0x7f53841670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:33.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.450+0000 7f5383e00640 1 --2- 172.21.15.159:0/895674049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5384163ac0 0x7f5384071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:33.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.450+0000 7f5382000640 1 -- 172.21.15.159:0/895674049 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1004032798 0 0) 0x7f5384071d60 con 0x7f5384162920 2024-04-04T06:06:33.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.451+0000 7f5382000640 1 -- 172.21.15.159:0/895674049 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5368003520 con 0x7f5384162920 2024-04-04T06:06:33.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.451+0000 7f5382a00640 1 -- 172.21.15.159:0/895674049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5384163ac0 msgr2=0x7f5384071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:33.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.451+0000 7f5382a00640 1 --2- 172.21.15.159:0/895674049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5384163ac0 0x7f5384071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:33.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.451+0000 7f5382a00640 1 -- 172.21.15.159:0/895674049 >> v1:172.21.15.159:6789/0 conn(0x7f5384162920 legacy=0x7f53840d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:33.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.451+0000 7f5382a00640 1 -- 172.21.15.159:0/895674049 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53841668b0 con 0x7f53840d0990 2024-04-04T06:06:33.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.452+0000 7f5382a00640 1 --2- 172.21.15.159:0/895674049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53840d0990 0x7f53841670b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f53700097a0 tx=0x7f537002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bb31889e75d2e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:33.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.452+0000 7f5382000640 1 -- 172.21.15.159:0/895674049 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5370031780 con 0x7f53840d0990 2024-04-04T06:06:33.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.452+0000 7f5382000640 1 -- 172.21.15.159:0/895674049 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5370037070 con 0x7f53840d0990 2024-04-04T06:06:33.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.452+0000 7f5382000640 1 -- 172.21.15.159:0/895674049 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5370031b20 con 0x7f53840d0990 2024-04-04T06:06:33.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.452+0000 7f538a200640 1 -- 172.21.15.159:0/895674049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53840d0990 msgr2=0x7f53841670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:33.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.452+0000 7f538a200640 1 --2- 172.21.15.159:0/895674049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53840d0990 0x7f53841670b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f53700097a0 tx=0x7f537002b820 comp rx=0 tx=0).stop 2024-04-04T06:06:33.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.453+0000 7f538a200640 1 -- 172.21.15.159:0/895674049 shutdown_connections 2024-04-04T06:06:33.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.453+0000 7f538a200640 1 --2- 172.21.15.159:0/895674049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5384163ac0 0x7f5384071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:33.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.453+0000 7f538a200640 1 --2- 172.21.15.159:0/895674049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53840d0990 0x7f53841670b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:33.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.453+0000 7f538a200640 1 -- 172.21.15.159:0/895674049 >> 172.21.15.159:0/895674049 conn(0x7f538406b750 msgr2=0x7f538406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:33.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.453+0000 7f538a200640 1 -- 172.21.15.159:0/895674049 shutdown_connections 2024-04-04T06:06:33.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.453+0000 7f538a200640 1 -- 172.21.15.159:0/895674049 wait complete. 2024-04-04T06:06:33.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.454+0000 7f538a200640 1 Processor -- start 2024-04-04T06:06:33.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.454+0000 7f538a200640 1 -- start start 2024-04-04T06:06:33.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.455+0000 7f538a200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53840d0990 0x7f538414d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:33.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.455+0000 7f538a200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5384162920 0x7f53841fc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:33.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.455+0000 7f538a200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5384163ac0 0x7f538414fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:33.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.455+0000 7f538a200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5384072040 con 0x7f5384163ac0 2024-04-04T06:06:33.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.455+0000 7f538a200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5384162d00 con 0x7f53840d0990 2024-04-04T06:06:33.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.455+0000 7f538a200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f53840721c0 con 0x7f5384162920 2024-04-04T06:06:33.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.456+0000 7f5383e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5384163ac0 0x7f538414fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:33.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.456+0000 7f5383e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5384163ac0 0x7f538414fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45378/0 (socket says 172.21.15.159:45378) 2024-04-04T06:06:33.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.456+0000 7f5383e00640 1 -- 172.21.15.159:0/2963161324 learned_addr learned my addr 172.21.15.159:0/2963161324 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:33.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.456+0000 7f5382a00640 1 --2- 172.21.15.159:0/2963161324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5384162920 0x7f53841fc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:33.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.456+0000 7f5383e00640 1 -- 172.21.15.159:0/2963161324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5384162920 msgr2=0x7f53841fc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:33.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.456+0000 7f5383400640 1 --2- 172.21.15.159:0/2963161324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53840d0990 0x7f538414d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:33.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.456+0000 7f5383e00640 1 --2- 172.21.15.159:0/2963161324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5384162920 0x7f53841fc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:33.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.456+0000 7f5383e00640 1 -- 172.21.15.159:0/2963161324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53840d0990 msgr2=0x7f538414d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:33.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.456+0000 7f5383e00640 1 --2- 172.21.15.159:0/2963161324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53840d0990 0x7f538414d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:33.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.456+0000 7f5383e00640 1 -- 172.21.15.159:0/2963161324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f538414d750 con 0x7f5384163ac0 2024-04-04T06:06:33.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.456+0000 7f5383e00640 1 --2- 172.21.15.159:0/2963161324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5384163ac0 0x7f538414fa80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f537c007e60 tx=0x7f537c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:33.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.457+0000 7f537b400640 1 -- 172.21.15.159:0/2963161324 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c014420 con 0x7f5384163ac0 2024-04-04T06:06:33.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.457+0000 7f537b400640 1 -- 172.21.15.159:0/2963161324 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f537c019070 con 0x7f5384163ac0 2024-04-04T06:06:33.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.457+0000 7f537b400640 1 -- 172.21.15.159:0/2963161324 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c014820 con 0x7f5384163ac0 2024-04-04T06:06:33.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.457+0000 7f538a200640 1 -- 172.21.15.159:0/2963161324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53841504e0 con 0x7f5384163ac0 2024-04-04T06:06:33.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.457+0000 7f538a200640 1 -- 172.21.15.159:0/2963161324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53841fbc30 con 0x7f5384163ac0 2024-04-04T06:06:33.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.458+0000 7f538a200640 1 -- 172.21.15.159:0/2963161324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5384150a00 con 0x7f5384163ac0 2024-04-04T06:06:33.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.459+0000 7f537b400640 1 -- 172.21.15.159:0/2963161324 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f537c0149c0 con 0x7f5384163ac0 2024-04-04T06:06:33.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.460+0000 7f537b400640 1 --2- 172.21.15.159:0/2963161324 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f536406c920 0x7f536406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:33.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.460+0000 7f537b400640 1 -- 172.21.15.159:0/2963161324 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5883+0+0 (secure 0 0 0) 0x7f537c08c450 con 0x7f5384163ac0 2024-04-04T06:06:33.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.460+0000 7f5383400640 1 --2- 172.21.15.159:0/2963161324 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f536406c920 0x7f536406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:33.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.462+0000 7f5383400640 1 --2- 172.21.15.159:0/2963161324 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f536406c920 0x7f536406ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5374000f60 tx=0x7f5374009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:33.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.464+0000 7f537b400640 1 -- 172.21.15.159:0/2963161324 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f537c05a7b0 con 0x7f5384163ac0 2024-04-04T06:06:33.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.663+0000 7f538a200640 1 -- 172.21.15.159:0/2963161324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f53840685c0 con 0x7f5384163ac0 2024-04-04T06:06:33.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.664+0000 7f537b400640 1 -- 172.21.15.159:0/2963161324 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f537c05a0b0 con 0x7f5384163ac0 2024-04-04T06:06:33.657 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:06:33.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.666+0000 7f538a200640 1 -- 172.21.15.159:0/2963161324 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f536406c920 msgr2=0x7f536406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:33.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.666+0000 7f538a200640 1 --2- 172.21.15.159:0/2963161324 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f536406c920 0x7f536406ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5374000f60 tx=0x7f5374009040 comp rx=0 tx=0).stop 2024-04-04T06:06:33.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.666+0000 7f538a200640 1 -- 172.21.15.159:0/2963161324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5384163ac0 msgr2=0x7f538414fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:33.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.666+0000 7f538a200640 1 --2- 172.21.15.159:0/2963161324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5384163ac0 0x7f538414fa80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f537c007e60 tx=0x7f537c00e4b0 comp rx=0 tx=0).stop 2024-04-04T06:06:33.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.667+0000 7f538a200640 1 -- 172.21.15.159:0/2963161324 shutdown_connections 2024-04-04T06:06:33.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.667+0000 7f538a200640 1 --2- 172.21.15.159:0/2963161324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5384163ac0 0x7f538414fa80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:33.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.667+0000 7f538a200640 1 --2- 172.21.15.159:0/2963161324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5384162920 0x7f53841fc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:33.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.667+0000 7f538a200640 1 --2- 172.21.15.159:0/2963161324 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f536406c920 0x7f536406ede0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:33.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.667+0000 7f538a200640 1 --2- 172.21.15.159:0/2963161324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53840d0990 0x7f538414d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:33.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.667+0000 7f538a200640 1 -- 172.21.15.159:0/2963161324 >> 172.21.15.159:0/2963161324 conn(0x7f538406b750 msgr2=0x7f5384165eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:33.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.667+0000 7f538a200640 1 -- 172.21.15.159:0/2963161324 shutdown_connections 2024-04-04T06:06:33.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.667+0000 7f538a200640 1 -- 172.21.15.159:0/2963161324 wait complete. 2024-04-04T06:06:33.680 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:06:33.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.894+0000 7ff12da00640 1 Processor -- start 2024-04-04T06:06:33.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.894+0000 7ff12da00640 1 -- start start 2024-04-04T06:06:33.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.896+0000 7ff12da00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff12815e590 0x7ff1280d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:33.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.896+0000 7ff12da00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff12806cd90 0x7ff1280cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:33.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.896+0000 7ff12da00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff128071060 con 0x7ff1280b47d0 2024-04-04T06:06:33.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.896+0000 7ff12da00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7ff128071340 con 0x7ff12806cd90 2024-04-04T06:06:33.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.896+0000 7ff12da00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff128071620 con 0x7ff12815e590 2024-04-04T06:06:33.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.896+0000 7ff127400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff12806cd90 0x7ff1280cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:33.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.896+0000 7ff126000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7ff1280b47d0 0x7ff12806c680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37050/0 (socket says 172.21.15.159:37050) 2024-04-04T06:06:33.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.896+0000 7ff126000640 1 -- 172.21.15.159:0/660611884 learned_addr learned my addr 172.21.15.159:0/660611884 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:33.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.896+0000 7ff126a00640 1 --2- 172.21.15.159:0/660611884 >> v2:172.21.15.178:3300/0 conn(0x7ff12815e590 0x7ff1280d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:33.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.896+0000 7ff127400640 1 -- 172.21.15.159:0/660611884 >> v2:172.21.15.178:3300/0 conn(0x7ff12815e590 msgr2=0x7ff1280d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:33.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.896+0000 7ff127400640 1 --2- 172.21.15.159:0/660611884 >> v2:172.21.15.178:3300/0 conn(0x7ff12815e590 0x7ff1280d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:33.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.896+0000 7ff127400640 1 -- 172.21.15.159:0/660611884 >> v1:172.21.15.159:6789/0 conn(0x7ff1280b47d0 legacy=0x7ff12806c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:33.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.896+0000 7ff127400640 1 -- 172.21.15.159:0/660611884 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff128071900 con 0x7ff12806cd90 2024-04-04T06:06:33.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.897+0000 7ff127400640 1 --2- 172.21.15.159:0/660611884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff12806cd90 0x7ff1280cfd80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff1280691c0 tx=0x7ff12002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5a6b87a32a6c78 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:33.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.898+0000 7ff125600640 1 -- 172.21.15.159:0/660611884 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120035020 con 0x7ff12806cd90 2024-04-04T06:06:33.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.898+0000 7ff125600640 1 -- 172.21.15.159:0/660611884 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff120031830 con 0x7ff12806cd90 2024-04-04T06:06:33.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.898+0000 7ff125600640 1 -- 172.21.15.159:0/660611884 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120031c00 con 0x7ff12806cd90 2024-04-04T06:06:33.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.898+0000 7ff12da00640 1 -- 172.21.15.159:0/660611884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff12806cd90 msgr2=0x7ff1280cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:33.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.898+0000 7ff12da00640 1 --2- 172.21.15.159:0/660611884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff12806cd90 0x7ff1280cfd80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff1280691c0 tx=0x7ff12002b8d0 comp rx=0 tx=0).stop 2024-04-04T06:06:33.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.898+0000 7ff12da00640 1 -- 172.21.15.159:0/660611884 shutdown_connections 2024-04-04T06:06:33.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.898+0000 7ff12da00640 1 --2- 172.21.15.159:0/660611884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff12806cd90 0x7ff1280cfd80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:33.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.898+0000 7ff12da00640 1 --2- 172.21.15.159:0/660611884 >> v2:172.21.15.178:3300/0 conn(0x7ff12815e590 0x7ff1280d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:33.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.898+0000 7ff12da00640 1 -- 172.21.15.159:0/660611884 >> 172.21.15.159:0/660611884 conn(0x7ff1280d2240 msgr2=0x7ff1280d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:33.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.898+0000 7ff12da00640 1 -- 172.21.15.159:0/660611884 shutdown_connections 2024-04-04T06:06:33.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.899+0000 7ff12da00640 1 -- 172.21.15.159:0/660611884 wait complete. 2024-04-04T06:06:33.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.900+0000 7ff12da00640 1 Processor -- start 2024-04-04T06:06:33.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.900+0000 7ff12da00640 1 -- start start 2024-04-04T06:06:33.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.901+0000 7ff12da00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12806cd90 0x7ff1280b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:33.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.901+0000 7ff12da00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1280b47d0 0x7ff1281f8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:33.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.901+0000 7ff12da00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff12815e590 0x7ff128153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:33.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.901+0000 7ff12da00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff12815e970 con 0x7ff1280b47d0 2024-04-04T06:06:33.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.901+0000 7ff12da00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff1280b4c60 con 0x7ff12806cd90 2024-04-04T06:06:33.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.901+0000 7ff12da00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff12815eaf0 con 0x7ff12815e590 2024-04-04T06:06:33.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.902+0000 7ff126000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1280b47d0 0x7ff1281f8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:33.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.902+0000 7ff126000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1280b47d0 0x7ff1281f8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36056/0 (socket says 172.21.15.159:36056) 2024-04-04T06:06:33.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.902+0000 7ff126000640 1 -- 172.21.15.159:0/309300730 learned_addr learned my addr 172.21.15.159:0/309300730 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:33.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.902+0000 7ff126a00640 1 --2- 172.21.15.159:0/309300730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12806cd90 0x7ff1280b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:33.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.902+0000 7ff127400640 1 --2- 172.21.15.159:0/309300730 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff12815e590 0x7ff128153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:33.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.902+0000 7ff126000640 1 -- 172.21.15.159:0/309300730 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff12815e590 msgr2=0x7ff128153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:33.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.902+0000 7ff126000640 1 --2- 172.21.15.159:0/309300730 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff12815e590 0x7ff128153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:33.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.902+0000 7ff126000640 1 -- 172.21.15.159:0/309300730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12806cd90 msgr2=0x7ff1280b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:33.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.902+0000 7ff126000640 1 --2- 172.21.15.159:0/309300730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12806cd90 0x7ff1280b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:33.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.902+0000 7ff126000640 1 -- 172.21.15.159:0/309300730 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff128161370 con 0x7ff1280b47d0 2024-04-04T06:06:33.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.903+0000 7ff126000640 1 --2- 172.21.15.159:0/309300730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1280b47d0 0x7ff1281f8080 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff11400c400 tx=0x7ff11400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:33.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.903+0000 7ff11ea00640 1 -- 172.21.15.159:0/309300730 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114017070 con 0x7ff1280b47d0 2024-04-04T06:06:33.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.903+0000 7ff11ea00640 1 -- 172.21.15.159:0/309300730 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff114005ce0 con 0x7ff1280b47d0 2024-04-04T06:06:33.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.903+0000 7ff11ea00640 1 -- 172.21.15.159:0/309300730 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1140072f0 con 0x7ff1280b47d0 2024-04-04T06:06:33.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.903+0000 7ff12da00640 1 -- 172.21.15.159:0/309300730 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1281f85c0 con 0x7ff1280b47d0 2024-04-04T06:06:33.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.903+0000 7ff12da00640 1 -- 172.21.15.159:0/309300730 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1281609d0 con 0x7ff1280b47d0 2024-04-04T06:06:33.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.904+0000 7ff12da00640 1 -- 172.21.15.159:0/309300730 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1280685c0 con 0x7ff1280b47d0 2024-04-04T06:06:33.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.905+0000 7ff11ea00640 1 -- 172.21.15.159:0/309300730 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff11401a070 con 0x7ff1280b47d0 2024-04-04T06:06:33.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.906+0000 7ff11ea00640 1 --2- 172.21.15.159:0/309300730 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff10806c8f0 0x7ff10806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:33.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.906+0000 7ff11ea00640 1 -- 172.21.15.159:0/309300730 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7ff11408af40 con 0x7ff1280b47d0 2024-04-04T06:06:33.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.906+0000 7ff126a00640 1 --2- 172.21.15.159:0/309300730 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff10806c8f0 0x7ff10806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:33.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.908+0000 7ff126a00640 1 --2- 172.21.15.159:0/309300730 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff10806c8f0 0x7ff10806edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff128162ea0 tx=0x7ff118009830 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:33.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:33.911+0000 7ff11ea00640 1 -- 172.21.15.159:0/309300730 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff114058c60 con 0x7ff1280b47d0 2024-04-04T06:06:34.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.113+0000 7ff12da00640 1 -- 172.21.15.159:0/309300730 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff128161fa0 con 0x7ff1280b47d0 2024-04-04T06:06:34.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.114+0000 7ff11ea00640 1 -- 172.21.15.159:0/309300730 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7ff11405cf60 con 0x7ff1280b47d0 2024-04-04T06:06:34.107 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:06:34.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.116+0000 7ff12da00640 1 -- 172.21.15.159:0/309300730 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff10806c8f0 msgr2=0x7ff10806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:34.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.116+0000 7ff12da00640 1 --2- 172.21.15.159:0/309300730 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff10806c8f0 0x7ff10806edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff128162ea0 tx=0x7ff118009830 comp rx=0 tx=0).stop 2024-04-04T06:06:34.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.116+0000 7ff12da00640 1 -- 172.21.15.159:0/309300730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1280b47d0 msgr2=0x7ff1281f8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:34.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.116+0000 7ff12da00640 1 --2- 172.21.15.159:0/309300730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1280b47d0 0x7ff1281f8080 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff11400c400 tx=0x7ff11400c8c0 comp rx=0 tx=0).stop 2024-04-04T06:06:34.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.117+0000 7ff12da00640 1 -- 172.21.15.159:0/309300730 shutdown_connections 2024-04-04T06:06:34.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.117+0000 7ff12da00640 1 --2- 172.21.15.159:0/309300730 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff12815e590 0x7ff128153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:34.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.117+0000 7ff12da00640 1 --2- 172.21.15.159:0/309300730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1280b47d0 0x7ff1281f8080 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:34.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.117+0000 7ff12da00640 1 --2- 172.21.15.159:0/309300730 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff10806c8f0 0x7ff10806edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:34.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.117+0000 7ff12da00640 1 --2- 172.21.15.159:0/309300730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12806cd90 0x7ff1280b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:34.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.117+0000 7ff12da00640 1 -- 172.21.15.159:0/309300730 >> 172.21.15.159:0/309300730 conn(0x7ff1280d2240 msgr2=0x7ff1280cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:34.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.117+0000 7ff12da00640 1 -- 172.21.15.159:0/309300730 shutdown_connections 2024-04-04T06:06:34.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.117+0000 7ff12da00640 1 -- 172.21.15.159:0/309300730 wait complete. 2024-04-04T06:06:34.129 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:06:34.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.349+0000 7fc397000640 1 Processor -- start 2024-04-04T06:06:34.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.349+0000 7fc397000640 1 -- start start 2024-04-04T06:06:34.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.351+0000 7fc397000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc3900cc880 0x7fc3900cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:34.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.351+0000 7fc397000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc39006d620 0x7fc39006da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:34.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.351+0000 7fc397000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3900ce890 con 0x7fc39013f190 2024-04-04T06:06:34.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.351+0000 7fc397000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fc3900cd230 con 0x7fc39006d620 2024-04-04T06:06:34.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.351+0000 7fc397000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc3900ceb50 con 0x7fc3900cc880 2024-04-04T06:06:34.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.351+0000 7fc395c00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fc39013f190 0x7fc3900cf630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37062/0 (socket says 172.21.15.159:37062) 2024-04-04T06:06:34.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.351+0000 7fc395c00640 1 -- 172.21.15.159:0/3903005911 learned_addr learned my addr 172.21.15.159:0/3903005911 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:34.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.351+0000 7fc396600640 1 --2- 172.21.15.159:0/3903005911 >> v2:172.21.15.159:3300/0 conn(0x7fc39006d620 0x7fc39006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:34.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.351+0000 7fc395200640 1 --2- 172.21.15.159:0/3903005911 >> v2:172.21.15.178:3300/0 conn(0x7fc3900cc880 0x7fc3900cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:34.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.351+0000 7fc38fe00640 1 -- 172.21.15.159:0/3903005911 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 716588524 0 0) 0x7fc3900ce890 con 0x7fc39013f190 2024-04-04T06:06:34.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.352+0000 7fc396600640 1 -- 172.21.15.159:0/3903005911 >> v2:172.21.15.178:3300/0 conn(0x7fc3900cc880 msgr2=0x7fc3900cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:34.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.352+0000 7fc396600640 1 --2- 172.21.15.159:0/3903005911 >> v2:172.21.15.178:3300/0 conn(0x7fc3900cc880 0x7fc3900cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:34.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.352+0000 7fc396600640 1 -- 172.21.15.159:0/3903005911 >> v1:172.21.15.159:6789/0 conn(0x7fc39013f190 legacy=0x7fc3900cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:34.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.352+0000 7fc396600640 1 -- 172.21.15.159:0/3903005911 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3900cee30 con 0x7fc39006d620 2024-04-04T06:06:34.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.353+0000 7fc396600640 1 --2- 172.21.15.159:0/3903005911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc39006d620 0x7fc39006da50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc3900cc7b0 tx=0x7fc38802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1ff363b8caa39d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:34.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.353+0000 7fc38fe00640 1 -- 172.21.15.159:0/3903005911 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388039070 con 0x7fc39006d620 2024-04-04T06:06:34.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.353+0000 7fc38fe00640 1 -- 172.21.15.159:0/3903005911 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc38802cc80 con 0x7fc39006d620 2024-04-04T06:06:34.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.353+0000 7fc38fe00640 1 -- 172.21.15.159:0/3903005911 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388034660 con 0x7fc39006d620 2024-04-04T06:06:34.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.354+0000 7fc397000640 1 -- 172.21.15.159:0/3903005911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc39006d620 msgr2=0x7fc39006da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:34.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.354+0000 7fc397000640 1 --2- 172.21.15.159:0/3903005911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc39006d620 0x7fc39006da50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc3900cc7b0 tx=0x7fc38802bed0 comp rx=0 tx=0).stop 2024-04-04T06:06:34.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.354+0000 7fc397000640 1 -- 172.21.15.159:0/3903005911 shutdown_connections 2024-04-04T06:06:34.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.354+0000 7fc397000640 1 --2- 172.21.15.159:0/3903005911 >> v2:172.21.15.178:3300/0 conn(0x7fc3900cc880 0x7fc3900cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:34.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.354+0000 7fc397000640 1 --2- 172.21.15.159:0/3903005911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc39006d620 0x7fc39006da50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:34.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.354+0000 7fc397000640 1 -- 172.21.15.159:0/3903005911 >> 172.21.15.159:0/3903005911 conn(0x7fc3900b1ba0 msgr2=0x7fc3900b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:34.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.354+0000 7fc397000640 1 -- 172.21.15.159:0/3903005911 shutdown_connections 2024-04-04T06:06:34.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.354+0000 7fc397000640 1 -- 172.21.15.159:0/3903005911 wait complete. 2024-04-04T06:06:34.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.356+0000 7fc397000640 1 Processor -- start 2024-04-04T06:06:34.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.356+0000 7fc397000640 1 -- start start 2024-04-04T06:06:34.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.357+0000 7fc397000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc39006d620 0x7fc39015a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:34.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.357+0000 7fc397000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3900cc880 0x7fc39015ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:34.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.357+0000 7fc397000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc39013f190 0x7fc3901f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:34.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.357+0000 7fc397000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc39013f570 con 0x7fc3900cc880 2024-04-04T06:06:34.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.357+0000 7fc397000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc390160c70 con 0x7fc39013f190 2024-04-04T06:06:34.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.357+0000 7fc397000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc39006df90 con 0x7fc39006d620 2024-04-04T06:06:34.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.357+0000 7fc395200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3900cc880 0x7fc39015ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:34.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.357+0000 7fc395200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3900cc880 0x7fc39015ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36084/0 (socket says 172.21.15.159:36084) 2024-04-04T06:06:34.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.357+0000 7fc395200640 1 -- 172.21.15.159:0/1381815994 learned_addr learned my addr 172.21.15.159:0/1381815994 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:34.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.358+0000 7fc395200640 1 -- 172.21.15.159:0/1381815994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc39006d620 msgr2=0x7fc39015a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:34.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.358+0000 7fc396600640 1 --2- 172.21.15.159:0/1381815994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc39013f190 0x7fc3901f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:34.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.358+0000 7fc395c00640 1 --2- 172.21.15.159:0/1381815994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc39006d620 0x7fc39015a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:34.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.358+0000 7fc395200640 1 --2- 172.21.15.159:0/1381815994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc39006d620 0x7fc39015a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:34.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.358+0000 7fc395200640 1 -- 172.21.15.159:0/1381815994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc39013f190 msgr2=0x7fc3901f1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:34.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.358+0000 7fc395200640 1 --2- 172.21.15.159:0/1381815994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc39013f190 0x7fc3901f1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:34.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.358+0000 7fc395200640 1 -- 172.21.15.159:0/1381815994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3900a9c40 con 0x7fc3900cc880 2024-04-04T06:06:34.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.358+0000 7fc395c00640 1 --2- 172.21.15.159:0/1381815994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc39006d620 0x7fc39015a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:34.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.358+0000 7fc396600640 1 --2- 172.21.15.159:0/1381815994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc39013f190 0x7fc3901f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:34.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.358+0000 7fc395200640 1 --2- 172.21.15.159:0/1381815994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3900cc880 0x7fc39015ae50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00d620 tx=0x7fc37c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:34.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.358+0000 7fc38d600640 1 -- 172.21.15.159:0/1381815994 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c00b620 con 0x7fc3900cc880 2024-04-04T06:06:34.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.358+0000 7fc38d600640 1 -- 172.21.15.159:0/1381815994 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc37c019070 con 0x7fc3900cc880 2024-04-04T06:06:34.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.358+0000 7fc397000640 1 -- 172.21.15.159:0/1381815994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39015b480 con 0x7fc3900cc880 2024-04-04T06:06:34.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.358+0000 7fc38d600640 1 -- 172.21.15.159:0/1381815994 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c00ba00 con 0x7fc3900cc880 2024-04-04T06:06:34.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.359+0000 7fc397000640 1 -- 172.21.15.159:0/1381815994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc39015ba40 con 0x7fc3900cc880 2024-04-04T06:06:34.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.360+0000 7fc397000640 1 -- 172.21.15.159:0/1381815994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc390075940 con 0x7fc3900cc880 2024-04-04T06:06:34.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.361+0000 7fc38d600640 1 -- 172.21.15.159:0/1381815994 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc37c00bba0 con 0x7fc3900cc880 2024-04-04T06:06:34.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.362+0000 7fc38d600640 1 --2- 172.21.15.159:0/1381815994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc37006c8f0 0x7fc37006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:34.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.362+0000 7fc38d600640 1 -- 172.21.15.159:0/1381815994 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7fc37c08bff0 con 0x7fc3900cc880 2024-04-04T06:06:34.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.362+0000 7fc395c00640 1 --2- 172.21.15.159:0/1381815994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc37006c8f0 0x7fc37006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:34.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.363+0000 7fc395c00640 1 --2- 172.21.15.159:0/1381815994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc37006c8f0 0x7fc37006edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc384000f60 tx=0x7fc384009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:34.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.366+0000 7fc38d600640 1 -- 172.21.15.159:0/1381815994 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc37c05a370 con 0x7fc3900cc880 2024-04-04T06:06:34.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.567+0000 7fc397000640 1 -- 172.21.15.159:0/1381815994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc3900b13b0 con 0x7fc3900cc880 2024-04-04T06:06:34.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.568+0000 7fc38d600640 1 -- 172.21.15.159:0/1381815994 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fc37c060030 con 0x7fc3900cc880 2024-04-04T06:06:34.562 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:06:34.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.571+0000 7fc397000640 1 -- 172.21.15.159:0/1381815994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc37006c8f0 msgr2=0x7fc37006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:34.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.571+0000 7fc397000640 1 --2- 172.21.15.159:0/1381815994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc37006c8f0 0x7fc37006edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc384000f60 tx=0x7fc384009040 comp rx=0 tx=0).stop 2024-04-04T06:06:34.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.571+0000 7fc397000640 1 -- 172.21.15.159:0/1381815994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3900cc880 msgr2=0x7fc39015ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:34.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.571+0000 7fc397000640 1 --2- 172.21.15.159:0/1381815994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3900cc880 0x7fc39015ae50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00d620 tx=0x7fc37c00dae0 comp rx=0 tx=0).stop 2024-04-04T06:06:34.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.571+0000 7fc397000640 1 -- 172.21.15.159:0/1381815994 shutdown_connections 2024-04-04T06:06:34.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.571+0000 7fc397000640 1 --2- 172.21.15.159:0/1381815994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc37006c8f0 0x7fc37006edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:34.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.571+0000 7fc397000640 1 --2- 172.21.15.159:0/1381815994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc39013f190 0x7fc3901f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:34.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.571+0000 7fc397000640 1 --2- 172.21.15.159:0/1381815994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3900cc880 0x7fc39015ae50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:34.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.571+0000 7fc397000640 1 --2- 172.21.15.159:0/1381815994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc39006d620 0x7fc39015a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:34.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.571+0000 7fc397000640 1 -- 172.21.15.159:0/1381815994 >> 172.21.15.159:0/1381815994 conn(0x7fc3900b1ba0 msgr2=0x7fc39015eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:34.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.571+0000 7fc397000640 1 -- 172.21.15.159:0/1381815994 shutdown_connections 2024-04-04T06:06:34.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.571+0000 7fc397000640 1 -- 172.21.15.159:0/1381815994 wait complete. 2024-04-04T06:06:34.586 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:06:34.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.795+0000 7fa76d000640 1 Processor -- start 2024-04-04T06:06:34.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.795+0000 7fa76d000640 1 -- start start 2024-04-04T06:06:34.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.796+0000 7fa76d000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7680ad120 0x7fa7680d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:34.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.796+0000 7fa76d000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa7680ad580 0x7fa7680d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:34.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.796+0000 7fa76d000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa768071b60 con 0x7fa768161b00 2024-04-04T06:06:34.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.796+0000 7fa76d000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fa768161ee0 con 0x7fa7680ad580 2024-04-04T06:06:34.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.796+0000 7fa76d000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa7680b3450 con 0x7fa7680ad120 2024-04-04T06:06:34.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.797+0000 7fa765e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fa768161b00 0x7fa768162ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37070/0 (socket says 172.21.15.159:37070) 2024-04-04T06:06:34.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.797+0000 7fa765e00640 1 -- 172.21.15.159:0/1897909021 learned_addr learned my addr 172.21.15.159:0/1897909021 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:34.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.797+0000 7fa766800640 1 --2- 172.21.15.159:0/1897909021 >> v2:172.21.15.159:3300/0 conn(0x7fa7680ad580 0x7fa7680d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:34.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.797+0000 7fa75d400640 1 --2- 172.21.15.159:0/1897909021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7680ad120 0x7fa7680d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:34.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.797+0000 7fa765400640 1 -- 172.21.15.159:0/1897909021 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3577588824 0 0) 0x7fa768071b60 con 0x7fa768161b00 2024-04-04T06:06:34.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.797+0000 7fa765400640 1 -- 172.21.15.159:0/1897909021 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa74c003520 con 0x7fa768161b00 2024-04-04T06:06:34.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.797+0000 7fa766800640 1 -- 172.21.15.159:0/1897909021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7680ad120 msgr2=0x7fa7680d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:34.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.797+0000 7fa766800640 1 --2- 172.21.15.159:0/1897909021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7680ad120 0x7fa7680d0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:34.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.797+0000 7fa766800640 1 -- 172.21.15.159:0/1897909021 >> v1:172.21.15.159:6789/0 conn(0x7fa768161b00 legacy=0x7fa768162ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:34.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.797+0000 7fa766800640 1 -- 172.21.15.159:0/1897909021 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa768071e40 con 0x7fa7680ad580 2024-04-04T06:06:34.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.797+0000 7fa75d400640 1 --2- 172.21.15.159:0/1897909021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7680ad120 0x7fa7680d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:06:34.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.799+0000 7fa766800640 1 --2- 172.21.15.159:0/1897909021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7680ad580 0x7fa7680d0b90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa760009d90 tx=0x7fa76002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da4e53b72f91c5a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:34.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.799+0000 7fa765400640 1 -- 172.21.15.159:0/1897909021 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760039070 con 0x7fa7680ad580 2024-04-04T06:06:34.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.799+0000 7fa765400640 1 -- 172.21.15.159:0/1897909021 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa76002cbc0 con 0x7fa7680ad580 2024-04-04T06:06:34.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.799+0000 7fa765400640 1 -- 172.21.15.159:0/1897909021 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760034440 con 0x7fa7680ad580 2024-04-04T06:06:34.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.799+0000 7fa76d000640 1 -- 172.21.15.159:0/1897909021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7680ad580 msgr2=0x7fa7680d0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:34.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.799+0000 7fa76d000640 1 --2- 172.21.15.159:0/1897909021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7680ad580 0x7fa7680d0b90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa760009d90 tx=0x7fa76002be10 comp rx=0 tx=0).stop 2024-04-04T06:06:34.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.799+0000 7fa76d000640 1 -- 172.21.15.159:0/1897909021 shutdown_connections 2024-04-04T06:06:34.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.799+0000 7fa76d000640 1 --2- 172.21.15.159:0/1897909021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7680ad120 0x7fa7680d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:34.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.799+0000 7fa76d000640 1 --2- 172.21.15.159:0/1897909021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7680ad580 0x7fa7680d0b90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:34.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.799+0000 7fa76d000640 1 -- 172.21.15.159:0/1897909021 >> 172.21.15.159:0/1897909021 conn(0x7fa76806b990 msgr2=0x7fa76806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:34.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.800+0000 7fa76d000640 1 -- 172.21.15.159:0/1897909021 shutdown_connections 2024-04-04T06:06:34.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.800+0000 7fa76d000640 1 -- 172.21.15.159:0/1897909021 wait complete. 2024-04-04T06:06:34.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.801+0000 7fa76d000640 1 Processor -- start 2024-04-04T06:06:34.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.801+0000 7fa76d000640 1 -- start start 2024-04-04T06:06:34.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.802+0000 7fa76d000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7680ad120 0x7fa76809d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:34.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.802+0000 7fa76d000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7680ad580 0x7fa76814e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:34.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.802+0000 7fa76d000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa768161b00 0x7fa7680ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:34.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.802+0000 7fa76d000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa7680717f0 con 0x7fa7680ad580 2024-04-04T06:06:34.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.802+0000 7fa76d000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa7680b3730 con 0x7fa768161b00 2024-04-04T06:06:34.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.802+0000 7fa76d000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa7680b5020 con 0x7fa7680ad120 2024-04-04T06:06:34.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.803+0000 7fa75d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7680ad580 0x7fa76814e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:34.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.803+0000 7fa75d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7680ad580 0x7fa76814e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36092/0 (socket says 172.21.15.159:36092) 2024-04-04T06:06:34.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.803+0000 7fa75d400640 1 -- 172.21.15.159:0/3765638055 learned_addr learned my addr 172.21.15.159:0/3765638055 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:34.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.803+0000 7fa765e00640 1 --2- 172.21.15.159:0/3765638055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7680ad120 0x7fa76809d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:34.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.803+0000 7fa75d400640 1 -- 172.21.15.159:0/3765638055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7680ad120 msgr2=0x7fa76809d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:34.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.803+0000 7fa766800640 1 --2- 172.21.15.159:0/3765638055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa768161b00 0x7fa7680ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:34.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.803+0000 7fa75d400640 1 --2- 172.21.15.159:0/3765638055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7680ad120 0x7fa76809d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:34.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.803+0000 7fa75d400640 1 -- 172.21.15.159:0/3765638055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa768161b00 msgr2=0x7fa7680ca290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:34.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.803+0000 7fa75d400640 1 --2- 172.21.15.159:0/3765638055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa768161b00 0x7fa7680ca290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:34.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.803+0000 7fa75d400640 1 -- 172.21.15.159:0/3765638055 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76809a2a0 con 0x7fa7680ad580 2024-04-04T06:06:34.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.803+0000 7fa765e00640 1 --2- 172.21.15.159:0/3765638055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7680ad120 0x7fa76809d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:34.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.803+0000 7fa75d400640 1 --2- 172.21.15.159:0/3765638055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7680ad580 0x7fa76814e580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa75000d7d0 tx=0x7fa75000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:34.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.804+0000 7fa75ea00640 1 -- 172.21.15.159:0/3765638055 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750004030 con 0x7fa7680ad580 2024-04-04T06:06:34.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.804+0000 7fa75ea00640 1 -- 172.21.15.159:0/3765638055 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7500041d0 con 0x7fa7680ad580 2024-04-04T06:06:34.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.804+0000 7fa76d000640 1 -- 172.21.15.159:0/3765638055 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76809d830 con 0x7fa7680ad580 2024-04-04T06:06:34.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.804+0000 7fa75ea00640 1 -- 172.21.15.159:0/3765638055 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7500096a0 con 0x7fa7680ad580 2024-04-04T06:06:34.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.804+0000 7fa76d000640 1 -- 172.21.15.159:0/3765638055 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa76809ddc0 con 0x7fa7680ad580 2024-04-04T06:06:34.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.805+0000 7fa76d000640 1 -- 172.21.15.159:0/3765638055 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7680b53f0 con 0x7fa7680ad580 2024-04-04T06:06:34.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.806+0000 7fa75ea00640 1 -- 172.21.15.159:0/3765638055 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa750009840 con 0x7fa7680ad580 2024-04-04T06:06:34.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.807+0000 7fa75ea00640 1 --2- 172.21.15.159:0/3765638055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa74406c8f0 0x7fa74406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:34.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.807+0000 7fa75ea00640 1 -- 172.21.15.159:0/3765638055 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7fa75008bf90 con 0x7fa7680ad580 2024-04-04T06:06:34.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.808+0000 7fa765e00640 1 --2- 172.21.15.159:0/3765638055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa74406c8f0 0x7fa74406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:34.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.809+0000 7fa765e00640 1 --2- 172.21.15.159:0/3765638055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa74406c8f0 0x7fa74406edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa758005880 tx=0x7fa758009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:34.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:34.812+0000 7fa75ea00640 1 -- 172.21.15.159:0/3765638055 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa75005a270 con 0x7fa7680ad580 2024-04-04T06:06:35.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.016+0000 7fa76d000640 1 -- 172.21.15.159:0/3765638055 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7680cffc0 con 0x7fa7680ad580 2024-04-04T06:06:35.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.018+0000 7fa75ea00640 1 -- 172.21.15.159:0/3765638055 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16958 (secure 0 0 0) 0x7fa750059b70 con 0x7fa7680ad580 2024-04-04T06:06:35.012 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:35.013 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":23,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:06:33.796979+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:05:51.297293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-04T06:05:51.770792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":0,"to":2}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:06:35.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.021+0000 7fa76d000640 1 -- 172.21.15.159:0/3765638055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa74406c8f0 msgr2=0x7fa74406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:35.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.021+0000 7fa76d000640 1 --2- 172.21.15.159:0/3765638055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa74406c8f0 0x7fa74406edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa758005880 tx=0x7fa758009040 comp rx=0 tx=0).stop 2024-04-04T06:06:35.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.021+0000 7fa76d000640 1 -- 172.21.15.159:0/3765638055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7680ad580 msgr2=0x7fa76814e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:35.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.021+0000 7fa76d000640 1 --2- 172.21.15.159:0/3765638055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7680ad580 0x7fa76814e580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa75000d7d0 tx=0x7fa75000dc90 comp rx=0 tx=0).stop 2024-04-04T06:06:35.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.021+0000 7fa76d000640 1 -- 172.21.15.159:0/3765638055 shutdown_connections 2024-04-04T06:06:35.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.021+0000 7fa76d000640 1 --2- 172.21.15.159:0/3765638055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa74406c8f0 0x7fa74406edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.021+0000 7fa76d000640 1 --2- 172.21.15.159:0/3765638055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa768161b00 0x7fa7680ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.021+0000 7fa76d000640 1 --2- 172.21.15.159:0/3765638055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7680ad580 0x7fa76814e580 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.021+0000 7fa76d000640 1 --2- 172.21.15.159:0/3765638055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7680ad120 0x7fa76809d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.021+0000 7fa76d000640 1 -- 172.21.15.159:0/3765638055 >> 172.21.15.159:0/3765638055 conn(0x7fa76806b990 msgr2=0x7fa7680aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:35.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.021+0000 7fa76d000640 1 -- 172.21.15.159:0/3765638055 shutdown_connections 2024-04-04T06:06:35.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.021+0000 7fa76d000640 1 -- 172.21.15.159:0/3765638055 wait complete. 2024-04-04T06:06:35.034 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:05:51.297293+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:05:51.770792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:06:35.035 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:06:35.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.245+0000 7f63f5c00640 1 Processor -- start 2024-04-04T06:06:35.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.245+0000 7f63f5c00640 1 -- start start 2024-04-04T06:06:35.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.247+0000 7f63f5c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63f00b3ec0 0x7f63f006d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:35.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.247+0000 7f63f5c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63f015c330 con 0x7f63f006db60 2024-04-04T06:06:35.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.247+0000 7f63f5c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63f006e230 con 0x7f63f015bf50 2024-04-04T06:06:35.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.247+0000 7f63f5c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f63f006e010 con 0x7f63f00b3ec0 2024-04-04T06:06:35.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.247+0000 7f63f5200640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f63f006db60 0x7f63f00add30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37084/0 (socket says 172.21.15.159:37084) 2024-04-04T06:06:35.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.247+0000 7f63f5200640 1 -- 172.21.15.159:0/3542528557 learned_addr learned my addr 172.21.15.159:0/3542528557 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:35.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.247+0000 7f63e7e00640 1 --2- 172.21.15.159:0/3542528557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63f00b3ec0 0x7f63f006d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:35.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.248+0000 7f63ef400640 1 -- 172.21.15.159:0/3542528557 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1165777386 0 0) 0x7f63f015c330 con 0x7f63f006db60 2024-04-04T06:06:35.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.248+0000 7f63ef400640 1 -- 172.21.15.159:0/3542528557 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63d4003520 con 0x7f63f006db60 2024-04-04T06:06:35.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.248+0000 7f63ef400640 1 -- 172.21.15.159:0/3542528557 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2691254958 0 0) 0x7f63d4003520 con 0x7f63f006db60 2024-04-04T06:06:35.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.248+0000 7f63e7e00640 1 -- 172.21.15.159:0/3542528557 >> v1:172.21.15.178:6789/0 conn(0x7f63f015bf50 legacy=0x7f63f00b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:35.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.248+0000 7f63e7e00640 1 -- 172.21.15.159:0/3542528557 >> v1:172.21.15.159:6789/0 conn(0x7f63f006db60 legacy=0x7f63f00add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:35.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.248+0000 7f63e7e00640 1 -- 172.21.15.159:0/3542528557 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f006e510 con 0x7f63f00b3ec0 2024-04-04T06:06:35.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.249+0000 7f63e7e00640 1 --2- 172.21.15.159:0/3542528557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63f00b3ec0 0x7f63f006d620 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f63d80097d0 tx=0x7f63d802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd94cb2fb0ffbfa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:35.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.250+0000 7f63ef400640 1 -- 172.21.15.159:0/3542528557 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e80019f0 con 0x7f63f00b3ec0 2024-04-04T06:06:35.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.250+0000 7f63ef400640 1 -- 172.21.15.159:0/3542528557 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63d8037070 con 0x7f63f00b3ec0 2024-04-04T06:06:35.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.250+0000 7f63ef400640 1 -- 172.21.15.159:0/3542528557 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d80317a0 con 0x7f63f00b3ec0 2024-04-04T06:06:35.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.250+0000 7f63f5c00640 1 -- 172.21.15.159:0/3542528557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63f00b3ec0 msgr2=0x7f63f006d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:35.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.250+0000 7f63f5c00640 1 --2- 172.21.15.159:0/3542528557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63f00b3ec0 0x7f63f006d620 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f63d80097d0 tx=0x7f63d802b8a0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.251+0000 7f63f5c00640 1 -- 172.21.15.159:0/3542528557 shutdown_connections 2024-04-04T06:06:35.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.251+0000 7f63f5c00640 1 --2- 172.21.15.159:0/3542528557 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63f00b3ec0 0x7f63f006d620 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.251+0000 7f63f5c00640 1 -- 172.21.15.159:0/3542528557 >> 172.21.15.159:0/3542528557 conn(0x7f63f00b1ba0 msgr2=0x7f63f00b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:35.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.251+0000 7f63f5c00640 1 -- 172.21.15.159:0/3542528557 shutdown_connections 2024-04-04T06:06:35.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.251+0000 7f63f5c00640 1 -- 172.21.15.159:0/3542528557 wait complete. 2024-04-04T06:06:35.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.252+0000 7f63f5c00640 1 Processor -- start 2024-04-04T06:06:35.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.252+0000 7f63f5c00640 1 -- start start 2024-04-04T06:06:35.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.253+0000 7f63f5c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63f006db60 0x7f63f0147960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:35.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.253+0000 7f63f5c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63f00b3ec0 0x7f63f0157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:35.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.253+0000 7f63f5c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63f015bf50 0x7f63f01576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:35.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.253+0000 7f63f5c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f63f015ed30 con 0x7f63f015bf50 2024-04-04T06:06:35.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.253+0000 7f63f5c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f63f006ee40 con 0x7f63f006db60 2024-04-04T06:06:35.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.253+0000 7f63f5c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f63f006ead0 con 0x7f63f00b3ec0 2024-04-04T06:06:35.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.254+0000 7f63f5200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63f015bf50 0x7f63f01576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:35.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.254+0000 7f63f5200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63f015bf50 0x7f63f01576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36100/0 (socket says 172.21.15.159:36100) 2024-04-04T06:06:35.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.254+0000 7f63f5200640 1 -- 172.21.15.159:0/2789222857 learned_addr learned my addr 172.21.15.159:0/2789222857 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:35.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.254+0000 7f63e7e00640 1 --2- 172.21.15.159:0/2789222857 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63f00b3ec0 0x7f63f0157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:35.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.254+0000 7f63efe00640 1 --2- 172.21.15.159:0/2789222857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63f006db60 0x7f63f0147960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:35.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.254+0000 7f63f5200640 1 -- 172.21.15.159:0/2789222857 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63f00b3ec0 msgr2=0x7f63f0157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:35.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.254+0000 7f63f5200640 1 --2- 172.21.15.159:0/2789222857 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63f00b3ec0 0x7f63f0157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.254+0000 7f63f5200640 1 -- 172.21.15.159:0/2789222857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63f006db60 msgr2=0x7f63f0147960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:35.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.254+0000 7f63f5200640 1 --2- 172.21.15.159:0/2789222857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63f006db60 0x7f63f0147960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.254+0000 7f63f5200640 1 -- 172.21.15.159:0/2789222857 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f015a040 con 0x7f63f015bf50 2024-04-04T06:06:35.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.255+0000 7f63f5200640 1 --2- 172.21.15.159:0/2789222857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63f015bf50 0x7f63f01576a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f63e8001890 tx=0x7f63e800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:35.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.255+0000 7f63ecc00640 1 -- 172.21.15.159:0/2789222857 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e8012400 con 0x7f63f015bf50 2024-04-04T06:06:35.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.255+0000 7f63ecc00640 1 -- 172.21.15.159:0/2789222857 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e80125c0 con 0x7f63f015bf50 2024-04-04T06:06:35.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.255+0000 7f63f5c00640 1 -- 172.21.15.159:0/2789222857 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f015a320 con 0x7f63f015bf50 2024-04-04T06:06:35.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.255+0000 7f63ecc00640 1 -- 172.21.15.159:0/2789222857 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e80129b0 con 0x7f63f015bf50 2024-04-04T06:06:35.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.255+0000 7f63f5c00640 1 -- 172.21.15.159:0/2789222857 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f00aeff0 con 0x7f63f015bf50 2024-04-04T06:06:35.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.256+0000 7f63f5c00640 1 -- 172.21.15.159:0/2789222857 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f0075940 con 0x7f63f015bf50 2024-04-04T06:06:35.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.257+0000 7f63ecc00640 1 -- 172.21.15.159:0/2789222857 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f63e8012b50 con 0x7f63f015bf50 2024-04-04T06:06:35.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.258+0000 7f63ecc00640 1 --2- 172.21.15.159:0/2789222857 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f63cc06c8f0 0x7f63cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:35.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.258+0000 7f63ecc00640 1 -- 172.21.15.159:0/2789222857 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f63e808c550 con 0x7f63f015bf50 2024-04-04T06:06:35.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.259+0000 7f63efe00640 1 --2- 172.21.15.159:0/2789222857 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f63cc06c8f0 0x7f63cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:35.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.260+0000 7f63efe00640 1 --2- 172.21.15.159:0/2789222857 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f63cc06c8f0 0x7f63cc06edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f63e0000f60 tx=0x7f63e0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:35.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.263+0000 7f63ecc00640 1 -- 172.21.15.159:0/2789222857 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f63e80182b0 con 0x7f63f015bf50 2024-04-04T06:06:35.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.457+0000 7f63f5c00640 1 -- 172.21.15.159:0/2789222857 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f63f015f3e0 con 0x7f63f015bf50 2024-04-04T06:06:35.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.458+0000 7f63ecc00640 1 -- 172.21.15.159:0/2789222857 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f63e805a840 con 0x7f63f015bf50 2024-04-04T06:06:35.452 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:06:35.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.460+0000 7f63f5c00640 1 -- 172.21.15.159:0/2789222857 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f63cc06c8f0 msgr2=0x7f63cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:35.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.460+0000 7f63f5c00640 1 --2- 172.21.15.159:0/2789222857 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f63cc06c8f0 0x7f63cc06edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f63e0000f60 tx=0x7f63e0009040 comp rx=0 tx=0).stop 2024-04-04T06:06:35.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.460+0000 7f63f5c00640 1 -- 172.21.15.159:0/2789222857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63f015bf50 msgr2=0x7f63f01576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:35.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.460+0000 7f63f5c00640 1 --2- 172.21.15.159:0/2789222857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63f015bf50 0x7f63f01576a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f63e8001890 tx=0x7f63e800a430 comp rx=0 tx=0).stop 2024-04-04T06:06:35.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.461+0000 7f63f5c00640 1 -- 172.21.15.159:0/2789222857 shutdown_connections 2024-04-04T06:06:35.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.461+0000 7f63f5c00640 1 --2- 172.21.15.159:0/2789222857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63f015bf50 0x7f63f01576a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.461+0000 7f63f5c00640 1 --2- 172.21.15.159:0/2789222857 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f63f00b3ec0 0x7f63f0157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.461+0000 7f63f5c00640 1 --2- 172.21.15.159:0/2789222857 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f63cc06c8f0 0x7f63cc06edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.461+0000 7f63f5c00640 1 --2- 172.21.15.159:0/2789222857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63f006db60 0x7f63f0147960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.461+0000 7f63f5c00640 1 -- 172.21.15.159:0/2789222857 >> 172.21.15.159:0/2789222857 conn(0x7f63f00b1ba0 msgr2=0x7f63f015dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:35.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.461+0000 7f63f5c00640 1 -- 172.21.15.159:0/2789222857 shutdown_connections 2024-04-04T06:06:35.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.461+0000 7f63f5c00640 1 -- 172.21.15.159:0/2789222857 wait complete. 2024-04-04T06:06:35.472 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:06:35.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.691+0000 7f16be400640 1 Processor -- start 2024-04-04T06:06:35.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.691+0000 7f16be400640 1 -- start start 2024-04-04T06:06:35.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.692+0000 7f16be400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f16b8153580 0x7f16b80d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:35.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.693+0000 7f16be400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f16b80af4b0 0x7f16b81600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:35.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.693+0000 7f16be400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16b80afab0 con 0x7f16b80d07c0 2024-04-04T06:06:35.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.693+0000 7f16be400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f16b80d0c30 con 0x7f16b80af4b0 2024-04-04T06:06:35.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.693+0000 7f16be400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f16b815cc80 con 0x7f16b8153580 2024-04-04T06:06:35.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.693+0000 7f16b7e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f16b80af4b0 0x7f16b81600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:35.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.693+0000 7f16afe00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f16b80d07c0 0x7f16b80aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37096/0 (socket says 172.21.15.159:37096) 2024-04-04T06:06:35.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.693+0000 7f16afe00640 1 -- 172.21.15.159:0/1100142254 learned_addr learned my addr 172.21.15.159:0/1100142254 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:35.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.693+0000 7f16b7400640 1 --2- 172.21.15.159:0/1100142254 >> v2:172.21.15.178:3300/0 conn(0x7f16b8153580 0x7f16b80d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:35.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.694+0000 7f16b7e00640 1 -- 172.21.15.159:0/1100142254 >> v2:172.21.15.178:3300/0 conn(0x7f16b8153580 msgr2=0x7f16b80d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:35.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.694+0000 7f16b7e00640 1 --2- 172.21.15.159:0/1100142254 >> v2:172.21.15.178:3300/0 conn(0x7f16b8153580 0x7f16b80d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.694+0000 7f16b7e00640 1 -- 172.21.15.159:0/1100142254 >> v1:172.21.15.159:6789/0 conn(0x7f16b80d07c0 legacy=0x7f16b80aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:35.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.694+0000 7f16b7e00640 1 -- 172.21.15.159:0/1100142254 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b815da00 con 0x7f16b80af4b0 2024-04-04T06:06:35.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.694+0000 7f16b6a00640 1 -- 172.21.15.159:0/1100142254 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2840093594 0 0) 0x7f16b80afab0 con 0x7f16b80d07c0 2024-04-04T06:06:35.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.695+0000 7f16b7e00640 1 --2- 172.21.15.159:0/1100142254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16b80af4b0 0x7f16b81600d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f16b80b5270 tx=0x7f16b002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1c44a6574ed32e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:35.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.695+0000 7f16b6a00640 1 -- 172.21.15.159:0/1100142254 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0039070 con 0x7f16b80af4b0 2024-04-04T06:06:35.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.695+0000 7f16b6a00640 1 -- 172.21.15.159:0/1100142254 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b002cbf0 con 0x7f16b80af4b0 2024-04-04T06:06:35.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.695+0000 7f16b6a00640 1 -- 172.21.15.159:0/1100142254 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0034440 con 0x7f16b80af4b0 2024-04-04T06:06:35.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.696+0000 7f16be400640 1 -- 172.21.15.159:0/1100142254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16b80af4b0 msgr2=0x7f16b81600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:35.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.696+0000 7f16be400640 1 --2- 172.21.15.159:0/1100142254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16b80af4b0 0x7f16b81600d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f16b80b5270 tx=0x7f16b002be40 comp rx=0 tx=0).stop 2024-04-04T06:06:35.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.696+0000 7f16be400640 1 -- 172.21.15.159:0/1100142254 shutdown_connections 2024-04-04T06:06:35.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.696+0000 7f16be400640 1 --2- 172.21.15.159:0/1100142254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16b80af4b0 0x7f16b81600d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.696+0000 7f16be400640 1 --2- 172.21.15.159:0/1100142254 >> v2:172.21.15.178:3300/0 conn(0x7f16b8153580 0x7f16b80d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.696+0000 7f16be400640 1 -- 172.21.15.159:0/1100142254 >> 172.21.15.159:0/1100142254 conn(0x7f16b806b750 msgr2=0x7f16b806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:35.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.696+0000 7f16be400640 1 -- 172.21.15.159:0/1100142254 shutdown_connections 2024-04-04T06:06:35.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.696+0000 7f16be400640 1 -- 172.21.15.159:0/1100142254 wait complete. 2024-04-04T06:06:35.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.698+0000 7f16be400640 1 Processor -- start 2024-04-04T06:06:35.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.698+0000 7f16be400640 1 -- start start 2024-04-04T06:06:35.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.699+0000 7f16be400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16b80af4b0 0x7f16b80ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:35.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.699+0000 7f16be400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16b80d07c0 0x7f16b81f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:35.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.699+0000 7f16be400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16b8153580 0x7f16b80b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:35.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.699+0000 7f16be400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f16b815d4e0 con 0x7f16b80af4b0 2024-04-04T06:06:35.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.699+0000 7f16be400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f16b815d200 con 0x7f16b80d07c0 2024-04-04T06:06:35.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.699+0000 7f16be400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f16b8153960 con 0x7f16b8153580 2024-04-04T06:06:35.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.699+0000 7f16b7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16b80af4b0 0x7f16b80ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:35.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.699+0000 7f16b7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16b80af4b0 0x7f16b80ce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36104/0 (socket says 172.21.15.159:36104) 2024-04-04T06:06:35.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.699+0000 7f16b7400640 1 -- 172.21.15.159:0/2109648303 learned_addr learned my addr 172.21.15.159:0/2109648303 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:35.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.699+0000 7f16b7400640 1 -- 172.21.15.159:0/2109648303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16b8153580 msgr2=0x7f16b80b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:35.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.699+0000 7f16b7400640 1 --2- 172.21.15.159:0/2109648303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16b8153580 0x7f16b80b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.699+0000 7f16b7400640 1 -- 172.21.15.159:0/2109648303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16b80d07c0 msgr2=0x7f16b81f37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:35.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.699+0000 7f16b7400640 1 --2- 172.21.15.159:0/2109648303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16b80d07c0 0x7f16b81f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.699+0000 7f16b7400640 1 -- 172.21.15.159:0/2109648303 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b80cec60 con 0x7f16b80af4b0 2024-04-04T06:06:35.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.700+0000 7f16b7400640 1 --2- 172.21.15.159:0/2109648303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16b80af4b0 0x7f16b80ce680 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f16a800e9e0 tx=0x7f16a800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:35.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.700+0000 7f16af400640 1 -- 172.21.15.159:0/2109648303 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8017070 con 0x7f16b80af4b0 2024-04-04T06:06:35.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.700+0000 7f16af400640 1 -- 172.21.15.159:0/2109648303 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16a800c9d0 con 0x7f16b80af4b0 2024-04-04T06:06:35.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.700+0000 7f16af400640 1 -- 172.21.15.159:0/2109648303 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a800cdc0 con 0x7f16b80af4b0 2024-04-04T06:06:35.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.700+0000 7f16be400640 1 -- 172.21.15.159:0/2109648303 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b8154c50 con 0x7f16b80af4b0 2024-04-04T06:06:35.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.700+0000 7f16be400640 1 -- 172.21.15.159:0/2109648303 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b8155210 con 0x7f16b80af4b0 2024-04-04T06:06:35.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.701+0000 7f16be400640 1 -- 172.21.15.159:0/2109648303 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b80685c0 con 0x7f16b80af4b0 2024-04-04T06:06:35.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.702+0000 7f16af400640 1 -- 172.21.15.159:0/2109648303 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f16a801a070 con 0x7f16b80af4b0 2024-04-04T06:06:35.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.703+0000 7f16af400640 1 --2- 172.21.15.159:0/2109648303 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f169406c8f0 0x7f169406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:35.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.703+0000 7f16af400640 1 -- 172.21.15.159:0/2109648303 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f16a808bfb0 con 0x7f16b80af4b0 2024-04-04T06:06:35.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.704+0000 7f16afe00640 1 --2- 172.21.15.159:0/2109648303 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f169406c8f0 0x7f169406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:35.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.705+0000 7f16afe00640 1 --2- 172.21.15.159:0/2109648303 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f169406c8f0 0x7f169406edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f16a0007fd0 tx=0x7f16a0007920 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:35.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.708+0000 7f16af400640 1 -- 172.21.15.159:0/2109648303 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f16a805a320 con 0x7f16b80af4b0 2024-04-04T06:06:35.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.893+0000 7f16be400640 1 -- 172.21.15.159:0/2109648303 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f16b8152ca0 con 0x7f16b80af4b0 2024-04-04T06:06:35.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.894+0000 7f16af400640 1 -- 172.21.15.159:0/2109648303 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f16a8059c20 con 0x7f16b80af4b0 2024-04-04T06:06:35.888 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:06:35.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.897+0000 7f16be400640 1 -- 172.21.15.159:0/2109648303 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f169406c8f0 msgr2=0x7f169406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:35.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.897+0000 7f16be400640 1 --2- 172.21.15.159:0/2109648303 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f169406c8f0 0x7f169406edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f16a0007fd0 tx=0x7f16a0007920 comp rx=0 tx=0).stop 2024-04-04T06:06:35.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.897+0000 7f16be400640 1 -- 172.21.15.159:0/2109648303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16b80af4b0 msgr2=0x7f16b80ce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:35.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.897+0000 7f16be400640 1 --2- 172.21.15.159:0/2109648303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16b80af4b0 0x7f16b80ce680 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f16a800e9e0 tx=0x7f16a800eea0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.897+0000 7f16be400640 1 -- 172.21.15.159:0/2109648303 shutdown_connections 2024-04-04T06:06:35.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.897+0000 7f16be400640 1 --2- 172.21.15.159:0/2109648303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f16b8153580 0x7f16b80b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.897+0000 7f16be400640 1 --2- 172.21.15.159:0/2109648303 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f169406c8f0 0x7f169406edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.897+0000 7f16be400640 1 --2- 172.21.15.159:0/2109648303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16b80d07c0 0x7f16b81f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.897+0000 7f16be400640 1 --2- 172.21.15.159:0/2109648303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16b80af4b0 0x7f16b80ce680 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:35.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.897+0000 7f16be400640 1 -- 172.21.15.159:0/2109648303 >> 172.21.15.159:0/2109648303 conn(0x7f16b806b750 msgr2=0x7f16b815b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:35.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.897+0000 7f16be400640 1 -- 172.21.15.159:0/2109648303 shutdown_connections 2024-04-04T06:06:35.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:35.897+0000 7f16be400640 1 -- 172.21.15.159:0/2109648303 wait complete. 2024-04-04T06:06:35.908 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:06:36.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.109+0000 7f3a58600640 1 Processor -- start 2024-04-04T06:06:36.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.109+0000 7f3a58600640 1 -- start start 2024-04-04T06:06:36.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.111+0000 7f3a58600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3a500d4760 0x7f3a50169550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:36.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.111+0000 7f3a58600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a500ceef0 con 0x7f3a500d0cb0 2024-04-04T06:06:36.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.111+0000 7f3a58600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a5015eab0 con 0x7f3a5015e6d0 2024-04-04T06:06:36.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.111+0000 7f3a58600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f3a500d1220 con 0x7f3a500d4760 2024-04-04T06:06:36.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.111+0000 7f3a56600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3a500d4760 0x7f3a50169550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:36.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.111+0000 7f3a56600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3a500d4760 0x7f3a50169550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36110/0 (socket says 172.21.15.159:36110) 2024-04-04T06:06:36.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.111+0000 7f3a56600640 1 -- 172.21.15.159:0/514269248 learned_addr learned my addr 172.21.15.159:0/514269248 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:36.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.112+0000 7f3a56600640 1 -- 172.21.15.159:0/514269248 >> v1:172.21.15.178:6789/0 conn(0x7f3a5015e6d0 legacy=0x7f3a500d21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:36.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.112+0000 7f3a56600640 1 -- 172.21.15.159:0/514269248 >> v1:172.21.15.159:6789/0 conn(0x7f3a500d0cb0 legacy=0x7f3a500d4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:36.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.112+0000 7f3a56600640 1 -- 172.21.15.159:0/514269248 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a500d1500 con 0x7f3a500d4760 2024-04-04T06:06:36.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.112+0000 7f3a4fe00640 1 -- 172.21.15.159:0/514269248 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 328656629 0 0) 0x7f3a500ceef0 con 0x7f3a500d0cb0 2024-04-04T06:06:36.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.113+0000 7f3a56600640 1 --2- 172.21.15.159:0/514269248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a500d4760 0x7f3a50169550 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009870 tx=0x7f3a4802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2874f35b8a90076 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:36.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.113+0000 7f3a4fe00640 1 -- 172.21.15.159:0/514269248 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48034070 con 0x7f3a500d4760 2024-04-04T06:06:36.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.113+0000 7f3a4fe00640 1 -- 172.21.15.159:0/514269248 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a48030860 con 0x7f3a500d4760 2024-04-04T06:06:36.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.113+0000 7f3a4fe00640 1 -- 172.21.15.159:0/514269248 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48030c30 con 0x7f3a500d4760 2024-04-04T06:06:36.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.114+0000 7f3a58600640 1 -- 172.21.15.159:0/514269248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a500d4760 msgr2=0x7f3a50169550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:36.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.114+0000 7f3a58600640 1 --2- 172.21.15.159:0/514269248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a500d4760 0x7f3a50169550 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009870 tx=0x7f3a4802b9a0 comp rx=0 tx=0).stop 2024-04-04T06:06:36.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.114+0000 7f3a58600640 1 -- 172.21.15.159:0/514269248 shutdown_connections 2024-04-04T06:06:36.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.114+0000 7f3a58600640 1 --2- 172.21.15.159:0/514269248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a500d4760 0x7f3a50169550 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:36.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.114+0000 7f3a58600640 1 -- 172.21.15.159:0/514269248 >> 172.21.15.159:0/514269248 conn(0x7f3a5009d7b0 msgr2=0x7f3a5008bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:36.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.114+0000 7f3a58600640 1 -- 172.21.15.159:0/514269248 shutdown_connections 2024-04-04T06:06:36.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.114+0000 7f3a58600640 1 -- 172.21.15.159:0/514269248 wait complete. 2024-04-04T06:06:36.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.115+0000 7f3a58600640 1 Processor -- start 2024-04-04T06:06:36.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.115+0000 7f3a58600640 1 -- start start 2024-04-04T06:06:36.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.116+0000 7f3a58600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a500d0cb0 0x7f3a5014f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:36.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.116+0000 7f3a58600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a500d4760 0x7f3a500b9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:36.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.116+0000 7f3a58600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a5015e6d0 0x7f3a500b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:36.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.116+0000 7f3a58600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5006c890 con 0x7f3a5015e6d0 2024-04-04T06:06:36.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.116+0000 7f3a58600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3a500d1ac0 con 0x7f3a500d0cb0 2024-04-04T06:06:36.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.116+0000 7f3a58600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3a5006ca10 con 0x7f3a500d4760 2024-04-04T06:06:36.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.117+0000 7f3a56600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a5015e6d0 0x7f3a500b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:36.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.117+0000 7f3a56600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a5015e6d0 0x7f3a500b72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36112/0 (socket says 172.21.15.159:36112) 2024-04-04T06:06:36.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.117+0000 7f3a56600640 1 -- 172.21.15.159:0/2784009645 learned_addr learned my addr 172.21.15.159:0/2784009645 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:36.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.117+0000 7f3a56600640 1 -- 172.21.15.159:0/2784009645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a500d4760 msgr2=0x7f3a500b9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:36.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.117+0000 7f3a56600640 1 --2- 172.21.15.159:0/2784009645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a500d4760 0x7f3a500b9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:36.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.117+0000 7f3a56600640 1 -- 172.21.15.159:0/2784009645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a500d0cb0 msgr2=0x7f3a5014f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:36.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.117+0000 7f3a55c00640 1 --2- 172.21.15.159:0/2784009645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a500d0cb0 0x7f3a5014f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:36.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.117+0000 7f3a56600640 1 --2- 172.21.15.159:0/2784009645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a500d0cb0 0x7f3a5014f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:36.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.117+0000 7f3a56600640 1 -- 172.21.15.159:0/2784009645 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5015b810 con 0x7f3a5015e6d0 2024-04-04T06:06:36.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.117+0000 7f3a56600640 1 --2- 172.21.15.159:0/2784009645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a5015e6d0 0x7f3a500b72a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009870 tx=0x7f3a48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:36.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.117+0000 7f3a4d600640 1 -- 172.21.15.159:0/2784009645 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48048070 con 0x7f3a5015e6d0 2024-04-04T06:06:36.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.118+0000 7f3a4d600640 1 -- 172.21.15.159:0/2784009645 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a48030660 con 0x7f3a5015e6d0 2024-04-04T06:06:36.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.118+0000 7f3a4d600640 1 -- 172.21.15.159:0/2784009645 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48034040 con 0x7f3a5015e6d0 2024-04-04T06:06:36.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.118+0000 7f3a58600640 1 -- 172.21.15.159:0/2784009645 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5015baf0 con 0x7f3a5015e6d0 2024-04-04T06:06:36.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.118+0000 7f3a58600640 1 -- 172.21.15.159:0/2784009645 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5015c0b0 con 0x7f3a5015e6d0 2024-04-04T06:06:36.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.119+0000 7f3a58600640 1 -- 172.21.15.159:0/2784009645 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a500685c0 con 0x7f3a5015e6d0 2024-04-04T06:06:36.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.120+0000 7f3a4d600640 1 -- 172.21.15.159:0/2784009645 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a4804b070 con 0x7f3a5015e6d0 2024-04-04T06:06:36.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.121+0000 7f3a4d600640 1 --2- 172.21.15.159:0/2784009645 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3a3006c8f0 0x7f3a3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:36.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.121+0000 7f3a4d600640 1 -- 172.21.15.159:0/2784009645 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f3a480adeb0 con 0x7f3a5015e6d0 2024-04-04T06:06:36.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.121+0000 7f3a55c00640 1 --2- 172.21.15.159:0/2784009645 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3a3006c8f0 0x7f3a3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:36.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.122+0000 7f3a55c00640 1 --2- 172.21.15.159:0/2784009645 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3a3006c8f0 0x7f3a3006edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3a40007bf0 tx=0x7f3a40007240 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:36.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.125+0000 7f3a4d600640 1 -- 172.21.15.159:0/2784009645 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3a4807c3e0 con 0x7f3a5015e6d0 2024-04-04T06:06:36.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.308+0000 7f3a58600640 1 -- 172.21.15.159:0/2784009645 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3a5006c670 con 0x7f3a5015e6d0 2024-04-04T06:06:36.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.309+0000 7f3a4d600640 1 -- 172.21.15.159:0/2784009645 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f3a4807bce0 con 0x7f3a5015e6d0 2024-04-04T06:06:36.303 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:06:36.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.312+0000 7f3a58600640 1 -- 172.21.15.159:0/2784009645 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3a3006c8f0 msgr2=0x7f3a3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:36.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.312+0000 7f3a58600640 1 --2- 172.21.15.159:0/2784009645 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3a3006c8f0 0x7f3a3006edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3a40007bf0 tx=0x7f3a40007240 comp rx=0 tx=0).stop 2024-04-04T06:06:36.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.312+0000 7f3a58600640 1 -- 172.21.15.159:0/2784009645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a5015e6d0 msgr2=0x7f3a500b72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:36.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.312+0000 7f3a58600640 1 --2- 172.21.15.159:0/2784009645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a5015e6d0 0x7f3a500b72a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009870 tx=0x7f3a48004730 comp rx=0 tx=0).stop 2024-04-04T06:06:36.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.312+0000 7f3a58600640 1 -- 172.21.15.159:0/2784009645 shutdown_connections 2024-04-04T06:06:36.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.312+0000 7f3a58600640 1 --2- 172.21.15.159:0/2784009645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a5015e6d0 0x7f3a500b72a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:36.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.312+0000 7f3a58600640 1 --2- 172.21.15.159:0/2784009645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a500d4760 0x7f3a500b9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:36.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.312+0000 7f3a58600640 1 --2- 172.21.15.159:0/2784009645 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3a3006c8f0 0x7f3a3006edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:36.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.312+0000 7f3a58600640 1 --2- 172.21.15.159:0/2784009645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a500d0cb0 0x7f3a5014f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:36.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.312+0000 7f3a58600640 1 -- 172.21.15.159:0/2784009645 >> 172.21.15.159:0/2784009645 conn(0x7f3a5009d7b0 msgr2=0x7f3a5015f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:36.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.312+0000 7f3a58600640 1 -- 172.21.15.159:0/2784009645 shutdown_connections 2024-04-04T06:06:36.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.312+0000 7f3a58600640 1 -- 172.21.15.159:0/2784009645 wait complete. 2024-04-04T06:06:36.322 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:06:36.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.532+0000 7f3ef3a00640 1 Processor -- start 2024-04-04T06:06:36.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.532+0000 7f3ef3a00640 1 -- start start 2024-04-04T06:06:36.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.533+0000 7f3ef3a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3eec0d0d60 0x7f3eec073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:36.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.533+0000 7f3ef3a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3eec15fe80 con 0x7f3eec15c390 2024-04-04T06:06:36.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.533+0000 7f3ef3a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3eec0ad120 con 0x7f3eec0af830 2024-04-04T06:06:36.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.533+0000 7f3ef3a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3eec15c770 con 0x7f3eec0d0d60 2024-04-04T06:06:36.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.534+0000 7f3ef1000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f3eec15c390 0x7f3eec160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37118/0 (socket says 172.21.15.159:37118) 2024-04-04T06:06:36.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.534+0000 7f3ef1000640 1 -- 172.21.15.159:0/1779101300 learned_addr learned my addr 172.21.15.159:0/1779101300 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:36.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.534+0000 7f3ef1a00640 1 --2- 172.21.15.159:0/1779101300 >> v2:172.21.15.178:3300/0 conn(0x7f3eec0d0d60 0x7f3eec073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:36.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.534+0000 7f3eeb400640 1 -- 172.21.15.159:0/1779101300 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1124428769 0 0) 0x7f3eec15fe80 con 0x7f3eec15c390 2024-04-04T06:06:36.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.534+0000 7f3eeb400640 1 -- 172.21.15.159:0/1779101300 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ed4003520 con 0x7f3eec15c390 2024-04-04T06:06:36.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.535+0000 7f3ef1a00640 1 -- 172.21.15.159:0/1779101300 >> v1:172.21.15.178:6789/0 conn(0x7f3eec0af830 legacy=0x7f3eec0d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:36.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.535+0000 7f3ef1a00640 1 -- 172.21.15.159:0/1779101300 >> v1:172.21.15.159:6789/0 conn(0x7f3eec15c390 legacy=0x7f3eec160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:36.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.535+0000 7f3eeb400640 1 -- 172.21.15.159:0/1779101300 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 653164973 0 0) 0x7f3ed4003520 con 0x7f3eec15c390 2024-04-04T06:06:36.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.535+0000 7f3ef1a00640 1 -- 172.21.15.159:0/1779101300 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eec0ad400 con 0x7f3eec0d0d60 2024-04-04T06:06:36.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.536+0000 7f3ef1a00640 1 --2- 172.21.15.159:0/1779101300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eec0d0d60 0x7f3eec073790 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009d90 tx=0x7f3ee402be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0d1dbb26018499a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:36.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.536+0000 7f3eeb400640 1 -- 172.21.15.159:0/1779101300 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4039070 con 0x7f3eec0d0d60 2024-04-04T06:06:36.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.536+0000 7f3eeb400640 1 -- 172.21.15.159:0/1779101300 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee402c940 con 0x7f3eec0d0d60 2024-04-04T06:06:36.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.536+0000 7f3eeb400640 1 -- 172.21.15.159:0/1779101300 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0001690 con 0x7f3eec0d0d60 2024-04-04T06:06:36.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.537+0000 7f3ef3a00640 1 -- 172.21.15.159:0/1779101300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eec0d0d60 msgr2=0x7f3eec073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:36.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.537+0000 7f3ef3a00640 1 --2- 172.21.15.159:0/1779101300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eec0d0d60 0x7f3eec073790 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009d90 tx=0x7f3ee402be10 comp rx=0 tx=0).stop 2024-04-04T06:06:36.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.537+0000 7f3ef3a00640 1 -- 172.21.15.159:0/1779101300 shutdown_connections 2024-04-04T06:06:36.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.537+0000 7f3ef3a00640 1 --2- 172.21.15.159:0/1779101300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eec0d0d60 0x7f3eec073790 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:36.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.537+0000 7f3ef3a00640 1 -- 172.21.15.159:0/1779101300 >> 172.21.15.159:0/1779101300 conn(0x7f3eec06b990 msgr2=0x7f3eec06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:36.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.537+0000 7f3ef3a00640 1 -- 172.21.15.159:0/1779101300 shutdown_connections 2024-04-04T06:06:36.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.537+0000 7f3ef3a00640 1 -- 172.21.15.159:0/1779101300 wait complete. 2024-04-04T06:06:36.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.539+0000 7f3ef3a00640 1 Processor -- start 2024-04-04T06:06:36.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.539+0000 7f3ef3a00640 1 -- start start 2024-04-04T06:06:36.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.540+0000 7f3ef3a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eec0af830 0x7f3eec153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:36.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.540+0000 7f3ef3a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eec0d0d60 0x7f3eec153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:36.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.540+0000 7f3ef3a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3eec15c390 0x7f3eec157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:36.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.540+0000 7f3ef3a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec160d50 con 0x7f3eec15c390 2024-04-04T06:06:36.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.540+0000 7f3ef3a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec160120 con 0x7f3eec0af830 2024-04-04T06:06:36.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.540+0000 7f3ef3a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3eec0cf980 con 0x7f3eec0d0d60 2024-04-04T06:06:36.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.540+0000 7f3ef1a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3eec15c390 0x7f3eec157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:36.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.540+0000 7f3ef1a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3eec15c390 0x7f3eec157b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36120/0 (socket says 172.21.15.159:36120) 2024-04-04T06:06:36.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.540+0000 7f3ef1a00640 1 -- 172.21.15.159:0/3157811434 learned_addr learned my addr 172.21.15.159:0/3157811434 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:36.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.541+0000 7f3eebe00640 1 --2- 172.21.15.159:0/3157811434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eec0d0d60 0x7f3eec153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:36.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.541+0000 7f3ef1000640 1 --2- 172.21.15.159:0/3157811434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eec0af830 0x7f3eec153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:36.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.541+0000 7f3ef1a00640 1 -- 172.21.15.159:0/3157811434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eec0d0d60 msgr2=0x7f3eec153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:36.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.541+0000 7f3ef1a00640 1 --2- 172.21.15.159:0/3157811434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eec0d0d60 0x7f3eec153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:36.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.541+0000 7f3ef1a00640 1 -- 172.21.15.159:0/3157811434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eec0af830 msgr2=0x7f3eec153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:36.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.541+0000 7f3ef1a00640 1 --2- 172.21.15.159:0/3157811434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eec0af830 0x7f3eec153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:36.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.541+0000 7f3ef1a00640 1 -- 172.21.15.159:0/3157811434 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eec14eb00 con 0x7f3eec15c390 2024-04-04T06:06:36.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.541+0000 7f3ef1a00640 1 --2- 172.21.15.159:0/3157811434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3eec15c390 0x7f3eec157b30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009eb0 tx=0x7f3ee4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:36.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.541+0000 7f3ee8c00640 1 -- 172.21.15.159:0/3157811434 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4044020 con 0x7f3eec15c390 2024-04-04T06:06:36.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.541+0000 7f3ee8c00640 1 -- 172.21.15.159:0/3157811434 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee4004240 con 0x7f3eec15c390 2024-04-04T06:06:36.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.541+0000 7f3ef3a00640 1 -- 172.21.15.159:0/3157811434 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eec0b3390 con 0x7f3eec15c390 2024-04-04T06:06:36.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.541+0000 7f3ee8c00640 1 -- 172.21.15.159:0/3157811434 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4039040 con 0x7f3eec15c390 2024-04-04T06:06:36.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.541+0000 7f3ef3a00640 1 -- 172.21.15.159:0/3157811434 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eec0b3950 con 0x7f3eec15c390 2024-04-04T06:06:36.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.542+0000 7f3ef3a00640 1 -- 172.21.15.159:0/3157811434 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eec0685c0 con 0x7f3eec15c390 2024-04-04T06:06:36.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.544+0000 7f3ee8c00640 1 -- 172.21.15.159:0/3157811434 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ee4047020 con 0x7f3eec15c390 2024-04-04T06:06:36.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.544+0000 7f3ee8c00640 1 --2- 172.21.15.159:0/3157811434 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3ecc06c8f0 0x7f3ecc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:36.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.545+0000 7f3ee8c00640 1 -- 172.21.15.159:0/3157811434 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f3ee40ae160 con 0x7f3eec15c390 2024-04-04T06:06:36.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.545+0000 7f3ef1000640 1 --2- 172.21.15.159:0/3157811434 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3ecc06c8f0 0x7f3ecc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:36.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.546+0000 7f3ef1000640 1 --2- 172.21.15.159:0/3157811434 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3ecc06c8f0 0x7f3ecc06edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0001eb0 tx=0x7f3ee0001c60 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:36.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.549+0000 7f3ee8c00640 1 -- 172.21.15.159:0/3157811434 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3ee404e070 con 0x7f3eec15c390 2024-04-04T06:06:36.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.734+0000 7f3ef3a00640 1 -- 172.21.15.159:0/3157811434 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3eec06f1b0 con 0x7f3eec15c390 2024-04-04T06:06:36.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.737+0000 7f3ee8c00640 1 -- 172.21.15.159:0/3157811434 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16958 (secure 0 0 0) 0x7f3eec14eb00 con 0x7f3eec15c390 2024-04-04T06:06:36.731 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:36.731 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":23,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:06:33.796979+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:05:51.297293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-04T06:05:51.770792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":0,"to":2}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:06:36.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.739+0000 7f3ef3a00640 1 -- 172.21.15.159:0/3157811434 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3ecc06c8f0 msgr2=0x7f3ecc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:36.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.739+0000 7f3ef3a00640 1 --2- 172.21.15.159:0/3157811434 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3ecc06c8f0 0x7f3ecc06edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0001eb0 tx=0x7f3ee0001c60 comp rx=0 tx=0).stop 2024-04-04T06:06:36.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.739+0000 7f3ef3a00640 1 -- 172.21.15.159:0/3157811434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3eec15c390 msgr2=0x7f3eec157b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:36.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.739+0000 7f3ef3a00640 1 --2- 172.21.15.159:0/3157811434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3eec15c390 0x7f3eec157b30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009eb0 tx=0x7f3ee4004a40 comp rx=0 tx=0).stop 2024-04-04T06:06:36.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.740+0000 7f3ef3a00640 1 -- 172.21.15.159:0/3157811434 shutdown_connections 2024-04-04T06:06:36.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.740+0000 7f3ef3a00640 1 --2- 172.21.15.159:0/3157811434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3eec15c390 0x7f3eec157b30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:36.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.740+0000 7f3ef3a00640 1 --2- 172.21.15.159:0/3157811434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3eec0d0d60 0x7f3eec153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:36.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.740+0000 7f3ef3a00640 1 --2- 172.21.15.159:0/3157811434 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3ecc06c8f0 0x7f3ecc06edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:36.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.740+0000 7f3ef3a00640 1 --2- 172.21.15.159:0/3157811434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eec0af830 0x7f3eec153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:36.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.740+0000 7f3ef3a00640 1 -- 172.21.15.159:0/3157811434 >> 172.21.15.159:0/3157811434 conn(0x7f3eec06b990 msgr2=0x7f3eec15a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:36.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.740+0000 7f3ef3a00640 1 -- 172.21.15.159:0/3157811434 shutdown_connections 2024-04-04T06:06:36.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.740+0000 7f3ef3a00640 1 -- 172.21.15.159:0/3157811434 wait complete. 2024-04-04T06:06:36.751 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:05:51.297293+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:05:51.770792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:06:36.752 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:06:36.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.970+0000 7fea60600640 1 Processor -- start 2024-04-04T06:06:36.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.970+0000 7fea60600640 1 -- start start 2024-04-04T06:06:36.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.972+0000 7fea60600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea580d0280 0x7fea580d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:36.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.972+0000 7fea60600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea5815ec80 con 0x7fea58162160 2024-04-04T06:06:36.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.972+0000 7fea60600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea580d0ba0 con 0x7fea58141c10 2024-04-04T06:06:36.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.972+0000 7fea60600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea5815ef20 con 0x7fea580d0280 2024-04-04T06:06:36.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.972+0000 7fea5e600640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fea58162160 0x7fea581625e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37132/0 (socket says 172.21.15.159:37132) 2024-04-04T06:06:36.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.972+0000 7fea5e600640 1 -- 172.21.15.159:0/916413002 learned_addr learned my addr 172.21.15.159:0/916413002 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:36.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.972+0000 7fea5dc00640 1 --2- 172.21.15.159:0/916413002 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea580d0280 0x7fea580d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:36.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.973+0000 7fea57e00640 1 -- 172.21.15.159:0/916413002 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2379639773 0 0) 0x7fea5815ec80 con 0x7fea58162160 2024-04-04T06:06:36.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.973+0000 7fea57e00640 1 -- 172.21.15.159:0/916413002 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea40003520 con 0x7fea58162160 2024-04-04T06:06:36.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.973+0000 7fea57e00640 1 -- 172.21.15.159:0/916413002 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 34905822 0 0) 0x7fea40003520 con 0x7fea58162160 2024-04-04T06:06:36.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.973+0000 7fea57e00640 1 -- 172.21.15.159:0/916413002 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea580d0280 msgr2=0x7fea580d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:36.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.973+0000 7fea57e00640 1 --2- 172.21.15.159:0/916413002 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea580d0280 0x7fea580d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:36.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.973+0000 7fea57e00640 1 -- 172.21.15.159:0/916413002 >> v1:172.21.15.178:6789/0 conn(0x7fea58141c10 legacy=0x7fea58071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:36.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.973+0000 7fea57e00640 1 -- 172.21.15.159:0/916413002 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5815f200 con 0x7fea58162160 2024-04-04T06:06:36.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.973+0000 7fea57e00640 1 -- 172.21.15.159:0/916413002 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fea500019f0 con 0x7fea58162160 2024-04-04T06:06:36.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.973+0000 7fea57e00640 1 -- 172.21.15.159:0/916413002 >> v1:172.21.15.159:6789/0 conn(0x7fea58162160 legacy=0x7fea581625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:36.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.973+0000 7fea57e00640 1 --2- 172.21.15.159:0/916413002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea400039e0 0x7fea40003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:36.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.973+0000 7fea57e00640 1 --2- 172.21.15.159:0/916413002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea40004800 0x7fea40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:36.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.973+0000 7fea57e00640 1 --2- 172.21.15.159:0/916413002 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea580d0280 0x7fea400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:36.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.973+0000 7fea57e00640 1 -- 172.21.15.159:0/916413002 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fea5815ef20 con 0x7fea400039e0 2024-04-04T06:06:36.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.973+0000 7fea57e00640 1 -- 172.21.15.159:0/916413002 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea4000ba70 con 0x7fea40004800 2024-04-04T06:06:36.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.973+0000 7fea57e00640 1 -- 172.21.15.159:0/916413002 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea4000bc70 con 0x7fea580d0280 2024-04-04T06:06:36.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.973+0000 7fea5dc00640 1 --2- 172.21.15.159:0/916413002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea400039e0 0x7fea40003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:36.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.974+0000 7fea5d200640 1 --2- 172.21.15.159:0/916413002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea40004800 0x7fea40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:36.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.974+0000 7fea5dc00640 1 -- 172.21.15.159:0/916413002 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea580d0280 msgr2=0x7fea400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:36.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.974+0000 7fea5dc00640 1 --2- 172.21.15.159:0/916413002 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea580d0280 0x7fea400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:36.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.974+0000 7fea5dc00640 1 -- 172.21.15.159:0/916413002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea40004800 msgr2=0x7fea40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:36.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.974+0000 7fea5dc00640 1 --2- 172.21.15.159:0/916413002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea40004800 0x7fea40004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:36.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.974+0000 7fea5dc00640 1 -- 172.21.15.159:0/916413002 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea40003520 con 0x7fea400039e0 2024-04-04T06:06:36.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.975+0000 7fea5dc00640 1 --2- 172.21.15.159:0/916413002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea400039e0 0x7fea40003e00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009a90 tx=0x7fea40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca98e022cb0d0e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:36.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.975+0000 7fea57e00640 1 -- 172.21.15.159:0/916413002 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c039070 con 0x7fea400039e0 2024-04-04T06:06:36.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.975+0000 7fea57e00640 1 -- 172.21.15.159:0/916413002 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea4c02c6b0 con 0x7fea400039e0 2024-04-04T06:06:36.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.975+0000 7fea57e00640 1 -- 172.21.15.159:0/916413002 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c009690 con 0x7fea400039e0 2024-04-04T06:06:36.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.976+0000 7fea60600640 1 -- 172.21.15.159:0/916413002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea400039e0 msgr2=0x7fea40003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:36.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.976+0000 7fea60600640 1 --2- 172.21.15.159:0/916413002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea400039e0 0x7fea40003e00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009a90 tx=0x7fea40005830 comp rx=0 tx=0).stop 2024-04-04T06:06:36.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.976+0000 7fea5e600640 1 -- 172.21.15.159:0/916413002 reap_dead start 2024-04-04T06:06:36.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.976+0000 7fea60600640 1 -- 172.21.15.159:0/916413002 shutdown_connections 2024-04-04T06:06:36.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.976+0000 7fea60600640 1 -- 172.21.15.159:0/916413002 >> 172.21.15.159:0/916413002 conn(0x7fea5806b750 msgr2=0x7fea5806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:36.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.976+0000 7fea60600640 1 -- 172.21.15.159:0/916413002 shutdown_connections 2024-04-04T06:06:36.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.976+0000 7fea60600640 1 -- 172.21.15.159:0/916413002 wait complete. 2024-04-04T06:06:36.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.978+0000 7fea60600640 1 Processor -- start 2024-04-04T06:06:36.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.978+0000 7fea60600640 1 -- start start 2024-04-04T06:06:36.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.979+0000 7fea60600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea400039e0 0x7fea58148290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:36.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.979+0000 7fea60600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea581f4810 0x7fea5809c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:36.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.979+0000 7fea60600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea5809cec0 0x7fea581ea3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:36.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.979+0000 7fea60600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fea58158e90 con 0x7fea400039e0 2024-04-04T06:06:36.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.979+0000 7fea60600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea5815f7c0 con 0x7fea581f4810 2024-04-04T06:06:36.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.979+0000 7fea60600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea5809af80 con 0x7fea5809cec0 2024-04-04T06:06:36.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.979+0000 7fea5dc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea400039e0 0x7fea58148290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:36.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.979+0000 7fea5dc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea400039e0 0x7fea58148290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36150/0 (socket says 172.21.15.159:36150) 2024-04-04T06:06:36.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.979+0000 7fea5dc00640 1 -- 172.21.15.159:0/1802769001 learned_addr learned my addr 172.21.15.159:0/1802769001 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:36.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.979+0000 7fea5d200640 1 --2- 172.21.15.159:0/1802769001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea581f4810 0x7fea5809c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:36.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.979+0000 7fea5dc00640 1 -- 172.21.15.159:0/1802769001 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea5809cec0 msgr2=0x7fea581ea3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:36.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.979+0000 7fea5dc00640 1 --2- 172.21.15.159:0/1802769001 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea5809cec0 0x7fea581ea3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:36.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.979+0000 7fea5dc00640 1 -- 172.21.15.159:0/1802769001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea581f4810 msgr2=0x7fea5809c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:36.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.979+0000 7fea5dc00640 1 --2- 172.21.15.159:0/1802769001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea581f4810 0x7fea5809c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:36.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.979+0000 7fea5dc00640 1 -- 172.21.15.159:0/1802769001 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5809a370 con 0x7fea400039e0 2024-04-04T06:06:36.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.980+0000 7fea5dc00640 1 --2- 172.21.15.159:0/1802769001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea400039e0 0x7fea58148290 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009bb0 tx=0x7fea4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:36.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.980+0000 7fea55600640 1 -- 172.21.15.159:0/1802769001 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c039050 con 0x7fea400039e0 2024-04-04T06:06:36.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.980+0000 7fea55600640 1 -- 172.21.15.159:0/1802769001 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea4c045070 con 0x7fea400039e0 2024-04-04T06:06:36.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.980+0000 7fea55600640 1 -- 172.21.15.159:0/1802769001 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c003850 con 0x7fea400039e0 2024-04-04T06:06:36.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.980+0000 7fea60600640 1 -- 172.21.15.159:0/1802769001 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5809a650 con 0x7fea400039e0 2024-04-04T06:06:36.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.980+0000 7fea60600640 1 -- 172.21.15.159:0/1802769001 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea5809ac10 con 0x7fea400039e0 2024-04-04T06:06:36.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.981+0000 7fea60600640 1 -- 172.21.15.159:0/1802769001 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea5809d2e0 con 0x7fea400039e0 2024-04-04T06:06:36.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.982+0000 7fea55600640 1 -- 172.21.15.159:0/1802769001 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea4c0039f0 con 0x7fea400039e0 2024-04-04T06:06:36.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.983+0000 7fea55600640 1 --2- 172.21.15.159:0/1802769001 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea3806c8f0 0x7fea3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:36.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.983+0000 7fea55600640 1 -- 172.21.15.159:0/1802769001 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7fea4c0ad680 con 0x7fea400039e0 2024-04-04T06:06:36.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.984+0000 7fea5d200640 1 --2- 172.21.15.159:0/1802769001 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea3806c8f0 0x7fea3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:36.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.985+0000 7fea5d200640 1 --2- 172.21.15.159:0/1802769001 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea3806c8f0 0x7fea3806edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fea581f8410 tx=0x7fea44009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:36.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:36.987+0000 7fea55600640 1 -- 172.21.15.159:0/1802769001 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fea4c04e020 con 0x7fea400039e0 2024-04-04T06:06:37.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.186+0000 7fea60600640 1 -- 172.21.15.159:0/1802769001 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fea580685c0 con 0x7fea400039e0 2024-04-04T06:06:37.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.186+0000 7fea55600640 1 -- 172.21.15.159:0/1802769001 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7fea4c07b9f0 con 0x7fea400039e0 2024-04-04T06:06:37.180 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:06:37.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.189+0000 7fea60600640 1 -- 172.21.15.159:0/1802769001 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea3806c8f0 msgr2=0x7fea3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:37.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.189+0000 7fea60600640 1 --2- 172.21.15.159:0/1802769001 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea3806c8f0 0x7fea3806edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fea581f8410 tx=0x7fea44009040 comp rx=0 tx=0).stop 2024-04-04T06:06:37.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.189+0000 7fea60600640 1 -- 172.21.15.159:0/1802769001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea400039e0 msgr2=0x7fea58148290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:37.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.189+0000 7fea60600640 1 --2- 172.21.15.159:0/1802769001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea400039e0 0x7fea58148290 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009bb0 tx=0x7fea4c004730 comp rx=0 tx=0).stop 2024-04-04T06:06:37.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.189+0000 7fea60600640 1 -- 172.21.15.159:0/1802769001 shutdown_connections 2024-04-04T06:06:37.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.189+0000 7fea60600640 1 --2- 172.21.15.159:0/1802769001 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea5809cec0 0x7fea581ea3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:37.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.189+0000 7fea60600640 1 --2- 172.21.15.159:0/1802769001 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea3806c8f0 0x7fea3806edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:37.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.189+0000 7fea60600640 1 --2- 172.21.15.159:0/1802769001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea581f4810 0x7fea5809c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:37.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.189+0000 7fea60600640 1 --2- 172.21.15.159:0/1802769001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea400039e0 0x7fea58148290 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:37.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.189+0000 7fea60600640 1 -- 172.21.15.159:0/1802769001 >> 172.21.15.159:0/1802769001 conn(0x7fea5806b750 msgr2=0x7fea581613b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:37.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.189+0000 7fea60600640 1 -- 172.21.15.159:0/1802769001 shutdown_connections 2024-04-04T06:06:37.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.189+0000 7fea60600640 1 -- 172.21.15.159:0/1802769001 wait complete. 2024-04-04T06:06:37.203 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:06:37.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.410+0000 7f44cdc00640 1 Processor -- start 2024-04-04T06:06:37.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.410+0000 7f44cdc00640 1 -- start start 2024-04-04T06:06:37.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.411+0000 7f44cdc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f44c813f190 0x7f44c80cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:37.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.411+0000 7f44cdc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44c806dd30 0x7f44c80cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:37.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.411+0000 7f44cdc00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44c80ce890 con 0x7f44c80cc880 2024-04-04T06:06:37.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.411+0000 7f44cdc00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f44c80cd230 con 0x7f44c813f190 2024-04-04T06:06:37.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.411+0000 7f44cdc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f44c80ceb50 con 0x7f44c806dd30 2024-04-04T06:06:37.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.412+0000 7f44c7e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f44c813f190 0x7f44c80cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:37.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.412+0000 7f44c7e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f44c813f190 0x7f44c80cf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36158/0 (socket says 172.21.15.159:36158) 2024-04-04T06:06:37.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.412+0000 7f44c7e00640 1 -- 172.21.15.159:0/1101223272 learned_addr learned my addr 172.21.15.159:0/1101223272 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:37.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.412+0000 7f44cd200640 1 --2- 172.21.15.159:0/1101223272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44c806dd30 0x7f44c80cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:37.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.412+0000 7f44c7e00640 1 -- 172.21.15.159:0/1101223272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44c806dd30 msgr2=0x7f44c80cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:37.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.412+0000 7f44c7e00640 1 --2- 172.21.15.159:0/1101223272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44c806dd30 0x7f44c80cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:37.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.412+0000 7f44c7e00640 1 -- 172.21.15.159:0/1101223272 >> v1:172.21.15.178:6789/0 conn(0x7f44c80cc880 legacy=0x7f44c806d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:37.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.412+0000 7f44c7e00640 1 -- 172.21.15.159:0/1101223272 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c80cee30 con 0x7f44c813f190 2024-04-04T06:06:37.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.414+0000 7f44c7e00640 1 --2- 172.21.15.159:0/1101223272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c813f190 0x7f44c80cf630 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f44c806f660 tx=0x7f44b802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5498556a67c1594 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:37.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.414+0000 7f44c6a00640 1 -- 172.21.15.159:0/1101223272 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8035020 con 0x7f44c813f190 2024-04-04T06:06:37.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.414+0000 7f44c6a00640 1 -- 172.21.15.159:0/1101223272 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44b8031830 con 0x7f44c813f190 2024-04-04T06:06:37.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.414+0000 7f44c6a00640 1 -- 172.21.15.159:0/1101223272 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8031c00 con 0x7f44c813f190 2024-04-04T06:06:37.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.415+0000 7f44cdc00640 1 -- 172.21.15.159:0/1101223272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c813f190 msgr2=0x7f44c80cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:37.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.415+0000 7f44cdc00640 1 --2- 172.21.15.159:0/1101223272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c813f190 0x7f44c80cf630 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f44c806f660 tx=0x7f44b802b8d0 comp rx=0 tx=0).stop 2024-04-04T06:06:37.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.415+0000 7f44cdc00640 1 -- 172.21.15.159:0/1101223272 shutdown_connections 2024-04-04T06:06:37.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.415+0000 7f44cdc00640 1 --2- 172.21.15.159:0/1101223272 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44c806dd30 0x7f44c80cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:37.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.415+0000 7f44cdc00640 1 --2- 172.21.15.159:0/1101223272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c813f190 0x7f44c80cf630 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:37.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.415+0000 7f44cdc00640 1 -- 172.21.15.159:0/1101223272 >> 172.21.15.159:0/1101223272 conn(0x7f44c80b1ba0 msgr2=0x7f44c80b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:37.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.415+0000 7f44cdc00640 1 -- 172.21.15.159:0/1101223272 shutdown_connections 2024-04-04T06:06:37.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.415+0000 7f44cdc00640 1 -- 172.21.15.159:0/1101223272 wait complete. 2024-04-04T06:06:37.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.417+0000 7f44cdc00640 1 Processor -- start 2024-04-04T06:06:37.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.417+0000 7f44cdc00640 1 -- start start 2024-04-04T06:06:37.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.418+0000 7f44cdc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44c806dd30 0x7f44c8155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:37.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.418+0000 7f44cdc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44c80cc880 0x7f44c80b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:37.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.418+0000 7f44cdc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c813f190 0x7f44c80b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:37.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.418+0000 7f44cdc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f44c813f570 con 0x7f44c813f190 2024-04-04T06:06:37.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.418+0000 7f44cdc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f44c8160c70 con 0x7f44c80cc880 2024-04-04T06:06:37.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.418+0000 7f44cdc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f44c80cfb70 con 0x7f44c806dd30 2024-04-04T06:06:37.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.418+0000 7f44cd200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c813f190 0x7f44c80b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:37.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.418+0000 7f44cd200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c813f190 0x7f44c80b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36162/0 (socket says 172.21.15.159:36162) 2024-04-04T06:06:37.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.418+0000 7f44cd200640 1 -- 172.21.15.159:0/871709235 learned_addr learned my addr 172.21.15.159:0/871709235 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:37.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.418+0000 7f44c7400640 1 --2- 172.21.15.159:0/871709235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44c80cc880 0x7f44c80b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:37.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.418+0000 7f44c7e00640 1 --2- 172.21.15.159:0/871709235 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44c806dd30 0x7f44c8155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:37.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.419+0000 7f44cd200640 1 -- 172.21.15.159:0/871709235 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44c806dd30 msgr2=0x7f44c8155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:37.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.419+0000 7f44cd200640 1 --2- 172.21.15.159:0/871709235 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44c806dd30 0x7f44c8155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:37.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.419+0000 7f44cd200640 1 -- 172.21.15.159:0/871709235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44c80cc880 msgr2=0x7f44c80b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:37.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.419+0000 7f44cd200640 1 --2- 172.21.15.159:0/871709235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44c80cc880 0x7f44c80b3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:37.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.419+0000 7f44cd200640 1 -- 172.21.15.159:0/871709235 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c813da70 con 0x7f44c813f190 2024-04-04T06:06:37.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.419+0000 7f44cd200640 1 --2- 172.21.15.159:0/871709235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c813f190 0x7f44c80b4220 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f44c0007e60 tx=0x7f44c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:37.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.419+0000 7f44bfe00640 1 -- 172.21.15.159:0/871709235 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c0014420 con 0x7f44c813f190 2024-04-04T06:06:37.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.419+0000 7f44bfe00640 1 -- 172.21.15.159:0/871709235 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44c0019070 con 0x7f44c813f190 2024-04-04T06:06:37.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.419+0000 7f44bfe00640 1 -- 172.21.15.159:0/871709235 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c0014820 con 0x7f44c813f190 2024-04-04T06:06:37.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.419+0000 7f44cdc00640 1 -- 172.21.15.159:0/871709235 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c813dd50 con 0x7f44c813f190 2024-04-04T06:06:37.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.419+0000 7f44cdc00640 1 -- 172.21.15.159:0/871709235 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c813e310 con 0x7f44c813f190 2024-04-04T06:06:37.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.420+0000 7f44cdc00640 1 -- 172.21.15.159:0/871709235 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44c8075940 con 0x7f44c813f190 2024-04-04T06:06:37.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.421+0000 7f44bfe00640 1 -- 172.21.15.159:0/871709235 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f44c00149c0 con 0x7f44c813f190 2024-04-04T06:06:37.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.422+0000 7f44bfe00640 1 --2- 172.21.15.159:0/871709235 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f44a806c8f0 0x7f44a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:37.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.422+0000 7f44bfe00640 1 -- 172.21.15.159:0/871709235 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f44c008c500 con 0x7f44c813f190 2024-04-04T06:06:37.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.423+0000 7f44c7e00640 1 --2- 172.21.15.159:0/871709235 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f44a806c8f0 0x7f44a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:37.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.424+0000 7f44c7e00640 1 --2- 172.21.15.159:0/871709235 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f44a806c8f0 0x7f44a806edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f44b8036000 tx=0x7f44b803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:37.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.427+0000 7f44bfe00640 1 -- 172.21.15.159:0/871709235 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f44c005a7e0 con 0x7f44c813f190 2024-04-04T06:06:37.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.627+0000 7f44cdc00640 1 -- 172.21.15.159:0/871709235 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f44c815cff0 con 0x7f44c813f190 2024-04-04T06:06:37.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.629+0000 7f44bfe00640 1 -- 172.21.15.159:0/871709235 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f44c005a0e0 con 0x7f44c813f190 2024-04-04T06:06:37.622 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:06:37.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.631+0000 7f44cdc00640 1 -- 172.21.15.159:0/871709235 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f44a806c8f0 msgr2=0x7f44a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:37.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.631+0000 7f44cdc00640 1 --2- 172.21.15.159:0/871709235 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f44a806c8f0 0x7f44a806edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f44b8036000 tx=0x7f44b803c040 comp rx=0 tx=0).stop 2024-04-04T06:06:37.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.631+0000 7f44cdc00640 1 -- 172.21.15.159:0/871709235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c813f190 msgr2=0x7f44c80b4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:37.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.631+0000 7f44cdc00640 1 --2- 172.21.15.159:0/871709235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c813f190 0x7f44c80b4220 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f44c0007e60 tx=0x7f44c000e4b0 comp rx=0 tx=0).stop 2024-04-04T06:06:37.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.631+0000 7f44cdc00640 1 -- 172.21.15.159:0/871709235 shutdown_connections 2024-04-04T06:06:37.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.631+0000 7f44cdc00640 1 --2- 172.21.15.159:0/871709235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c813f190 0x7f44c80b4220 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:37.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.631+0000 7f44cdc00640 1 --2- 172.21.15.159:0/871709235 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f44a806c8f0 0x7f44a806edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:37.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.631+0000 7f44cdc00640 1 --2- 172.21.15.159:0/871709235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44c80cc880 0x7f44c80b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:37.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.631+0000 7f44cdc00640 1 --2- 172.21.15.159:0/871709235 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44c806dd30 0x7f44c8155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:37.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.631+0000 7f44cdc00640 1 -- 172.21.15.159:0/871709235 >> 172.21.15.159:0/871709235 conn(0x7f44c80b1ba0 msgr2=0x7f44c806d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:37.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.631+0000 7f44cdc00640 1 -- 172.21.15.159:0/871709235 shutdown_connections 2024-04-04T06:06:37.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.631+0000 7f44cdc00640 1 -- 172.21.15.159:0/871709235 wait complete. 2024-04-04T06:06:37.644 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:06:37.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.863+0000 7fb27be00640 1 Processor -- start 2024-04-04T06:06:37.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.863+0000 7fb27be00640 1 -- start start 2024-04-04T06:06:37.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.865+0000 7fb27be00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb2740d0990 0x7fb2740aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:37.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.865+0000 7fb27be00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2740af2e0 0x7fb2741600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:37.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.865+0000 7fb27be00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2740afab0 con 0x7fb274153580 2024-04-04T06:06:37.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.865+0000 7fb27be00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb27415cc80 con 0x7fb2740d0990 2024-04-04T06:06:37.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.865+0000 7fb27be00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb27415da00 con 0x7fb2740af2e0 2024-04-04T06:06:37.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.865+0000 7fb279400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fb274153580 0x7fb2740d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37144/0 (socket says 172.21.15.159:37144) 2024-04-04T06:06:37.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.865+0000 7fb279400640 1 -- 172.21.15.159:0/3003227699 learned_addr learned my addr 172.21.15.159:0/3003227699 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:37.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.866+0000 7fb278a00640 1 --2- 172.21.15.159:0/3003227699 >> v2:172.21.15.178:3300/0 conn(0x7fb2740d0990 0x7fb2740aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:37.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.866+0000 7fb279e00640 1 --2- 172.21.15.159:0/3003227699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2740af2e0 0x7fb2741600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:37.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.866+0000 7fb273e00640 1 -- 172.21.15.159:0/3003227699 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2364696708 0 0) 0x7fb2740afab0 con 0x7fb274153580 2024-04-04T06:06:37.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.866+0000 7fb273e00640 1 -- 172.21.15.159:0/3003227699 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb25c003520 con 0x7fb274153580 2024-04-04T06:06:37.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.866+0000 7fb273e00640 1 -- 172.21.15.159:0/3003227699 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1742651801 0 0) 0x7fb25c003520 con 0x7fb274153580 2024-04-04T06:06:37.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.866+0000 7fb273e00640 1 -- 172.21.15.159:0/3003227699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2740af2e0 msgr2=0x7fb2741600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:37.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.866+0000 7fb273e00640 1 --2- 172.21.15.159:0/3003227699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2740af2e0 0x7fb2741600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:37.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.866+0000 7fb273e00640 1 -- 172.21.15.159:0/3003227699 >> v2:172.21.15.178:3300/0 conn(0x7fb2740d0990 msgr2=0x7fb2740aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:37.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.866+0000 7fb273e00640 1 --2- 172.21.15.159:0/3003227699 >> v2:172.21.15.178:3300/0 conn(0x7fb2740d0990 0x7fb2740aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:37.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.866+0000 7fb273e00640 1 -- 172.21.15.159:0/3003227699 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27415cf20 con 0x7fb274153580 2024-04-04T06:06:37.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.866+0000 7fb273e00640 1 -- 172.21.15.159:0/3003227699 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fb268001670 con 0x7fb274153580 2024-04-04T06:06:37.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.866+0000 7fb273e00640 1 -- 172.21.15.159:0/3003227699 >> v1:172.21.15.159:6789/0 conn(0x7fb274153580 legacy=0x7fb2740d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:37.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.866+0000 7fb273e00640 1 --2- 172.21.15.159:0/3003227699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb25c0039b0 0x7fb25c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:37.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.866+0000 7fb273e00640 1 --2- 172.21.15.159:0/3003227699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb25c004800 0x7fb25c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:37.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.866+0000 7fb273e00640 1 --2- 172.21.15.159:0/3003227699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2740af2e0 0x7fb25c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:37.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.866+0000 7fb273e00640 1 -- 172.21.15.159:0/3003227699 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb27415cc80 con 0x7fb25c0039b0 2024-04-04T06:06:37.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.867+0000 7fb273e00640 1 -- 172.21.15.159:0/3003227699 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb27415da00 con 0x7fb25c004800 2024-04-04T06:06:37.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.867+0000 7fb273e00640 1 -- 172.21.15.159:0/3003227699 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb25c00b9c0 con 0x7fb2740af2e0 2024-04-04T06:06:37.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.867+0000 7fb279400640 1 --2- 172.21.15.159:0/3003227699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb25c0039b0 0x7fb25c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:37.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.867+0000 7fb279e00640 1 --2- 172.21.15.159:0/3003227699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2740af2e0 0x7fb25c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:37.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.867+0000 7fb278a00640 1 --2- 172.21.15.159:0/3003227699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb25c004800 0x7fb25c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:37.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.867+0000 7fb279400640 1 -- 172.21.15.159:0/3003227699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2740af2e0 msgr2=0x7fb25c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:37.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.867+0000 7fb279400640 1 --2- 172.21.15.159:0/3003227699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2740af2e0 0x7fb25c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:37.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.867+0000 7fb279400640 1 -- 172.21.15.159:0/3003227699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb25c004800 msgr2=0x7fb25c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:37.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.867+0000 7fb279400640 1 --2- 172.21.15.159:0/3003227699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb25c004800 0x7fb25c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:37.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.867+0000 7fb279400640 1 -- 172.21.15.159:0/3003227699 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25c003520 con 0x7fb25c0039b0 2024-04-04T06:06:37.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.868+0000 7fb279400640 1 --2- 172.21.15.159:0/3003227699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb25c0039b0 0x7fb25c003dd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb26800c210 tx=0x7fb25c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c497503243690a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:37.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.869+0000 7fb273e00640 1 -- 172.21.15.159:0/3003227699 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268007d50 con 0x7fb25c0039b0 2024-04-04T06:06:37.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.869+0000 7fb273e00640 1 -- 172.21.15.159:0/3003227699 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb268030070 con 0x7fb25c0039b0 2024-04-04T06:06:37.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.869+0000 7fb273e00640 1 -- 172.21.15.159:0/3003227699 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26802cd20 con 0x7fb25c0039b0 2024-04-04T06:06:37.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.869+0000 7fb27be00640 1 -- 172.21.15.159:0/3003227699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb25c0039b0 msgr2=0x7fb25c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:37.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.869+0000 7fb27be00640 1 --2- 172.21.15.159:0/3003227699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb25c0039b0 0x7fb25c003dd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb26800c210 tx=0x7fb25c005830 comp rx=0 tx=0).stop 2024-04-04T06:06:37.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.869+0000 7fb279e00640 1 -- 172.21.15.159:0/3003227699 reap_dead start 2024-04-04T06:06:37.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.869+0000 7fb27be00640 1 -- 172.21.15.159:0/3003227699 shutdown_connections 2024-04-04T06:06:37.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.869+0000 7fb27be00640 1 -- 172.21.15.159:0/3003227699 >> 172.21.15.159:0/3003227699 conn(0x7fb27406b750 msgr2=0x7fb27406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:37.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.869+0000 7fb27be00640 1 -- 172.21.15.159:0/3003227699 shutdown_connections 2024-04-04T06:06:37.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.869+0000 7fb27be00640 1 -- 172.21.15.159:0/3003227699 wait complete. 2024-04-04T06:06:37.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.871+0000 7fb27be00640 1 Processor -- start 2024-04-04T06:06:37.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.871+0000 7fb27be00640 1 -- start start 2024-04-04T06:06:37.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.872+0000 7fb27be00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb25c0039b0 0x7fb2741f1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:37.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.872+0000 7fb27be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2740b0640 0x7fb2740b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:37.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.873+0000 7fb27be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb27409b6f0 0x7fb27409bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:37.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.873+0000 7fb27be00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb274153960 con 0x7fb27409b6f0 2024-04-04T06:06:37.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.873+0000 7fb27be00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb27415d4e0 con 0x7fb2740b0640 2024-04-04T06:06:37.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.873+0000 7fb27be00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb2740af710 con 0x7fb25c0039b0 2024-04-04T06:06:37.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.873+0000 7fb279e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb27409b6f0 0x7fb27409bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:37.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.873+0000 7fb279e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb27409b6f0 0x7fb27409bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36190/0 (socket says 172.21.15.159:36190) 2024-04-04T06:06:37.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.873+0000 7fb279e00640 1 -- 172.21.15.159:0/2325941035 learned_addr learned my addr 172.21.15.159:0/2325941035 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:37.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.873+0000 7fb279400640 1 --2- 172.21.15.159:0/2325941035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb25c0039b0 0x7fb2741f1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:37.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.873+0000 7fb278a00640 1 --2- 172.21.15.159:0/2325941035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2740b0640 0x7fb2740b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:37.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.873+0000 7fb279e00640 1 -- 172.21.15.159:0/2325941035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb25c0039b0 msgr2=0x7fb2741f1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:37.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.873+0000 7fb279e00640 1 --2- 172.21.15.159:0/2325941035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb25c0039b0 0x7fb2741f1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:37.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.873+0000 7fb279e00640 1 -- 172.21.15.159:0/2325941035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2740b0640 msgr2=0x7fb2740b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:37.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.873+0000 7fb279e00640 1 --2- 172.21.15.159:0/2325941035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2740b0640 0x7fb2740b0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:37.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.873+0000 7fb279e00640 1 -- 172.21.15.159:0/2325941035 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27409c420 con 0x7fb27409b6f0 2024-04-04T06:06:37.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.874+0000 7fb279e00640 1 --2- 172.21.15.159:0/2325941035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb27409b6f0 0x7fb27409bb70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb27406c530 tx=0x7fb26c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:37.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.874+0000 7fb271600640 1 -- 172.21.15.159:0/2325941035 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26c01c070 con 0x7fb27409b6f0 2024-04-04T06:06:37.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.874+0000 7fb271600640 1 -- 172.21.15.159:0/2325941035 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb26c017400 con 0x7fb27409b6f0 2024-04-04T06:06:37.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.874+0000 7fb27be00640 1 -- 172.21.15.159:0/2325941035 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27409c700 con 0x7fb27409b6f0 2024-04-04T06:06:37.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.874+0000 7fb271600640 1 -- 172.21.15.159:0/2325941035 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26c0177f0 con 0x7fb27409b6f0 2024-04-04T06:06:37.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.874+0000 7fb27be00640 1 -- 172.21.15.159:0/2325941035 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb27409ccc0 con 0x7fb27409b6f0 2024-04-04T06:06:37.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.875+0000 7fb27be00640 1 -- 172.21.15.159:0/2325941035 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2740685c0 con 0x7fb27409b6f0 2024-04-04T06:06:37.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.876+0000 7fb271600640 1 -- 172.21.15.159:0/2325941035 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb26c01f070 con 0x7fb27409b6f0 2024-04-04T06:06:37.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.877+0000 7fb271600640 1 --2- 172.21.15.159:0/2325941035 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb25406c8f0 0x7fb25406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:37.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.877+0000 7fb271600640 1 -- 172.21.15.159:0/2325941035 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7fb26c08bda0 con 0x7fb27409b6f0 2024-04-04T06:06:37.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.877+0000 7fb279400640 1 --2- 172.21.15.159:0/2325941035 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb25406c8f0 0x7fb25406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:37.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.878+0000 7fb279400640 1 --2- 172.21.15.159:0/2325941035 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb25406c8f0 0x7fb25406edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb268034000 tx=0x7fb268007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:37.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:37.882+0000 7fb271600640 1 -- 172.21.15.159:0/2325941035 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb26c012070 con 0x7fb27409b6f0 2024-04-04T06:06:38.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.084+0000 7fb27be00640 1 -- 172.21.15.159:0/2325941035 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb2740b41b0 con 0x7fb27409b6f0 2024-04-04T06:06:38.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.085+0000 7fb271600640 1 -- 172.21.15.159:0/2325941035 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fb26c059a30 con 0x7fb27409b6f0 2024-04-04T06:06:38.079 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:06:38.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.087+0000 7fb27be00640 1 -- 172.21.15.159:0/2325941035 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb25406c8f0 msgr2=0x7fb25406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:38.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.087+0000 7fb27be00640 1 --2- 172.21.15.159:0/2325941035 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb25406c8f0 0x7fb25406edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb268034000 tx=0x7fb268007000 comp rx=0 tx=0).stop 2024-04-04T06:06:38.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.087+0000 7fb27be00640 1 -- 172.21.15.159:0/2325941035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb27409b6f0 msgr2=0x7fb27409bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:38.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.087+0000 7fb27be00640 1 --2- 172.21.15.159:0/2325941035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb27409b6f0 0x7fb27409bb70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb27406c530 tx=0x7fb26c00f4a0 comp rx=0 tx=0).stop 2024-04-04T06:06:38.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.087+0000 7fb27be00640 1 -- 172.21.15.159:0/2325941035 shutdown_connections 2024-04-04T06:06:38.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.087+0000 7fb27be00640 1 --2- 172.21.15.159:0/2325941035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb27409b6f0 0x7fb27409bb70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:38.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.087+0000 7fb27be00640 1 --2- 172.21.15.159:0/2325941035 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb25406c8f0 0x7fb25406edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:38.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.087+0000 7fb27be00640 1 --2- 172.21.15.159:0/2325941035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2740b0640 0x7fb2740b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:38.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.087+0000 7fb27be00640 1 --2- 172.21.15.159:0/2325941035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb25c0039b0 0x7fb2741f1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:38.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.087+0000 7fb27be00640 1 -- 172.21.15.159:0/2325941035 >> 172.21.15.159:0/2325941035 conn(0x7fb27406b750 msgr2=0x7fb2740b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:38.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.087+0000 7fb27be00640 1 -- 172.21.15.159:0/2325941035 shutdown_connections 2024-04-04T06:06:38.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.087+0000 7fb27be00640 1 -- 172.21.15.159:0/2325941035 wait complete. 2024-04-04T06:06:38.109 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-04T06:06:38.109 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:06:38.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.320+0000 7f06d2c00640 1 Processor -- start 2024-04-04T06:06:38.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.321+0000 7f06d2c00640 1 -- start start 2024-04-04T06:06:38.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.322+0000 7f06d2c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f06cc0af830 0x7f06cc073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:38.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.322+0000 7f06d2c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f06cc0d0650 0x7f06cc0d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:38.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.322+0000 7f06d2c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06cc15fe80 con 0x7f06cc15c390 2024-04-04T06:06:38.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.322+0000 7f06d2c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f06cc15c770 con 0x7f06cc0af830 2024-04-04T06:06:38.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.322+0000 7f06d2c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f06cc0d0fc0 con 0x7f06cc0d0650 2024-04-04T06:06:38.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.323+0000 7f06cb400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f06cc0af830 0x7f06cc073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:38.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.323+0000 7f06cb400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f06cc0af830 0x7f06cc073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36194/0 (socket says 172.21.15.159:36194) 2024-04-04T06:06:38.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.323+0000 7f06cb400640 1 -- 172.21.15.159:0/700199565 learned_addr learned my addr 172.21.15.159:0/700199565 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:38.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.323+0000 7f06cb400640 1 -- 172.21.15.159:0/700199565 >> v2:172.21.15.178:3300/0 conn(0x7f06cc0d0650 msgr2=0x7f06cc0d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:38.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.323+0000 7f06d0c00640 1 --2- 172.21.15.159:0/700199565 >> v2:172.21.15.178:3300/0 conn(0x7f06cc0d0650 0x7f06cc0d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:38.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.323+0000 7f06cb400640 1 --2- 172.21.15.159:0/700199565 >> v2:172.21.15.178:3300/0 conn(0x7f06cc0d0650 0x7f06cc0d0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:38.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.323+0000 7f06cb400640 1 -- 172.21.15.159:0/700199565 >> v1:172.21.15.178:6789/0 conn(0x7f06cc15c390 legacy=0x7f06cc160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:38.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.323+0000 7f06cb400640 1 -- 172.21.15.159:0/700199565 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06cc0ad120 con 0x7f06cc0af830 2024-04-04T06:06:38.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.324+0000 7f06cb400640 1 --2- 172.21.15.159:0/700199565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06cc0af830 0x7f06cc073790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f06cc06eaa0 tx=0x7f06b802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443a7b84261d8cbd server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:38.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.325+0000 7f06caa00640 1 -- 172.21.15.159:0/700199565 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b802ca70 con 0x7f06cc0af830 2024-04-04T06:06:38.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.325+0000 7f06caa00640 1 -- 172.21.15.159:0/700199565 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06b802cc10 con 0x7f06cc0af830 2024-04-04T06:06:38.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.325+0000 7f06caa00640 1 -- 172.21.15.159:0/700199565 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b80343f0 con 0x7f06cc0af830 2024-04-04T06:06:38.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.325+0000 7f06d2c00640 1 -- 172.21.15.159:0/700199565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06cc0af830 msgr2=0x7f06cc073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:38.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.325+0000 7f06d2c00640 1 --2- 172.21.15.159:0/700199565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06cc0af830 0x7f06cc073790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f06cc06eaa0 tx=0x7f06b802bed0 comp rx=0 tx=0).stop 2024-04-04T06:06:38.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.326+0000 7f06d2c00640 1 -- 172.21.15.159:0/700199565 shutdown_connections 2024-04-04T06:06:38.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.326+0000 7f06d2c00640 1 --2- 172.21.15.159:0/700199565 >> v2:172.21.15.178:3300/0 conn(0x7f06cc0d0650 0x7f06cc0d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:38.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.326+0000 7f06d2c00640 1 --2- 172.21.15.159:0/700199565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06cc0af830 0x7f06cc073790 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:38.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.326+0000 7f06d2c00640 1 -- 172.21.15.159:0/700199565 >> 172.21.15.159:0/700199565 conn(0x7f06cc06b990 msgr2=0x7f06cc06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:38.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.326+0000 7f06d2c00640 1 -- 172.21.15.159:0/700199565 shutdown_connections 2024-04-04T06:06:38.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.326+0000 7f06d2c00640 1 -- 172.21.15.159:0/700199565 wait complete. 2024-04-04T06:06:38.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.327+0000 7f06d2c00640 1 Processor -- start 2024-04-04T06:06:38.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.327+0000 7f06d2c00640 1 -- start start 2024-04-04T06:06:38.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.328+0000 7f06d2c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06cc0af830 0x7f06cc14e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:38.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.328+0000 7f06d2c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06cc0d0650 0x7f06cc0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:38.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.328+0000 7f06d2c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06cc15c390 0x7f06cc1f4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:38.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.328+0000 7f06d2c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc160120 con 0x7f06cc0d0650 2024-04-04T06:06:38.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.328+0000 7f06d2c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc0ad6e0 con 0x7f06cc0af830 2024-04-04T06:06:38.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.328+0000 7f06d2c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f06cc160d50 con 0x7f06cc15c390 2024-04-04T06:06:38.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.329+0000 7f06cb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06cc0d0650 0x7f06cc0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:38.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.329+0000 7f06cb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06cc0d0650 0x7f06cc0b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36196/0 (socket says 172.21.15.159:36196) 2024-04-04T06:06:38.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.329+0000 7f06cb400640 1 -- 172.21.15.159:0/3682689430 learned_addr learned my addr 172.21.15.159:0/3682689430 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:38.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.329+0000 7f06cbe00640 1 --2- 172.21.15.159:0/3682689430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06cc0af830 0x7f06cc14e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:38.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.329+0000 7f06d0c00640 1 --2- 172.21.15.159:0/3682689430 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06cc15c390 0x7f06cc1f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:38.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.329+0000 7f06cb400640 1 -- 172.21.15.159:0/3682689430 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06cc15c390 msgr2=0x7f06cc1f4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:38.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.329+0000 7f06cb400640 1 --2- 172.21.15.159:0/3682689430 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06cc15c390 0x7f06cc1f4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:38.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.329+0000 7f06cb400640 1 -- 172.21.15.159:0/3682689430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06cc0af830 msgr2=0x7f06cc14e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:38.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.329+0000 7f06cb400640 1 --2- 172.21.15.159:0/3682689430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06cc0af830 0x7f06cc14e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:38.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.329+0000 7f06cb400640 1 -- 172.21.15.159:0/3682689430 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06cc0b0fc0 con 0x7f06cc0d0650 2024-04-04T06:06:38.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.329+0000 7f06cbe00640 1 --2- 172.21.15.159:0/3682689430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06cc0af830 0x7f06cc14e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:38.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.329+0000 7f06d0c00640 1 --2- 172.21.15.159:0/3682689430 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06cc15c390 0x7f06cc1f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:38.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.329+0000 7f06cb400640 1 --2- 172.21.15.159:0/3682689430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06cc0d0650 0x7f06cc0b43f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009e50 tx=0x7f06b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:38.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.330+0000 7f06c3e00640 1 -- 172.21.15.159:0/3682689430 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8043020 con 0x7f06cc0d0650 2024-04-04T06:06:38.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.330+0000 7f06c3e00640 1 -- 172.21.15.159:0/3682689430 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06b803e450 con 0x7f06cc0d0650 2024-04-04T06:06:38.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.330+0000 7f06c3e00640 1 -- 172.21.15.159:0/3682689430 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b803e840 con 0x7f06cc0d0650 2024-04-04T06:06:38.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.330+0000 7f06d2c00640 1 -- 172.21.15.159:0/3682689430 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06cc0b12a0 con 0x7f06cc0d0650 2024-04-04T06:06:38.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.330+0000 7f06d2c00640 1 -- 172.21.15.159:0/3682689430 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06cc0b3bf0 con 0x7f06cc0d0650 2024-04-04T06:06:38.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.331+0000 7f06d2c00640 1 -- 172.21.15.159:0/3682689430 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06cc0685c0 con 0x7f06cc0d0650 2024-04-04T06:06:38.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.332+0000 7f06c3e00640 1 -- 172.21.15.159:0/3682689430 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f06b8046020 con 0x7f06cc0d0650 2024-04-04T06:06:38.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.333+0000 7f06c3e00640 1 --2- 172.21.15.159:0/3682689430 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f06ac06c8f0 0x7f06ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:38.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.333+0000 7f06c3e00640 1 -- 172.21.15.159:0/3682689430 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f06b80ae570 con 0x7f06cc0d0650 2024-04-04T06:06:38.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.334+0000 7f06cbe00640 1 --2- 172.21.15.159:0/3682689430 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f06ac06c8f0 0x7f06ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:38.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.335+0000 7f06cbe00640 1 --2- 172.21.15.159:0/3682689430 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f06ac06c8f0 0x7f06ac06edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f06bc007fd0 tx=0x7f06bc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:38.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.338+0000 7f06c3e00640 1 -- 172.21.15.159:0/3682689430 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f06b804d030 con 0x7f06cc0d0650 2024-04-04T06:06:38.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.541+0000 7f06d2c00640 1 -- 172.21.15.159:0/3682689430 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06cc06f1b0 con 0x7f06cc0d0650 2024-04-04T06:06:38.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.543+0000 7f06c3e00640 1 -- 172.21.15.159:0/3682689430 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16958 (secure 0 0 0) 0x7f06b807c8f0 con 0x7f06cc0d0650 2024-04-04T06:06:38.539 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:38.539 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":23,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:06:33.796979+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:05:51.297293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-04T06:05:51.770792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":0,"to":2}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:06:38.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.546+0000 7f06d2c00640 1 -- 172.21.15.159:0/3682689430 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f06ac06c8f0 msgr2=0x7f06ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:38.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.546+0000 7f06d2c00640 1 --2- 172.21.15.159:0/3682689430 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f06ac06c8f0 0x7f06ac06edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f06bc007fd0 tx=0x7f06bc009040 comp rx=0 tx=0).stop 2024-04-04T06:06:38.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.546+0000 7f06d2c00640 1 -- 172.21.15.159:0/3682689430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06cc0d0650 msgr2=0x7f06cc0b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:38.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.546+0000 7f06d2c00640 1 --2- 172.21.15.159:0/3682689430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06cc0d0650 0x7f06cc0b43f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009e50 tx=0x7f06b8004a40 comp rx=0 tx=0).stop 2024-04-04T06:06:38.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.546+0000 7f06d2c00640 1 -- 172.21.15.159:0/3682689430 shutdown_connections 2024-04-04T06:06:38.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.547+0000 7f06d2c00640 1 --2- 172.21.15.159:0/3682689430 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06cc15c390 0x7f06cc1f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:38.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.547+0000 7f06d2c00640 1 --2- 172.21.15.159:0/3682689430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06cc0d0650 0x7f06cc0b43f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:38.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.547+0000 7f06d2c00640 1 --2- 172.21.15.159:0/3682689430 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f06ac06c8f0 0x7f06ac06edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:38.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.547+0000 7f06d2c00640 1 --2- 172.21.15.159:0/3682689430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06cc0af830 0x7f06cc14e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:38.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.547+0000 7f06d2c00640 1 -- 172.21.15.159:0/3682689430 >> 172.21.15.159:0/3682689430 conn(0x7f06cc06b990 msgr2=0x7f06cc15a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:38.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.547+0000 7f06d2c00640 1 -- 172.21.15.159:0/3682689430 shutdown_connections 2024-04-04T06:06:38.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.547+0000 7f06d2c00640 1 -- 172.21.15.159:0/3682689430 wait complete. 2024-04-04T06:06:38.561 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:05:51.297293+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:05:51.770792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:06:38.562 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:06:38.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.787+0000 7fca6ca00640 1 Processor -- start 2024-04-04T06:06:38.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.788+0000 7fca6ca00640 1 -- start start 2024-04-04T06:06:38.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.789+0000 7fca6ca00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fca680d0990 0x7fca680aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:38.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.789+0000 7fca6ca00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fca680af2e0 0x7fca681600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:38.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.789+0000 7fca6ca00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca680afab0 con 0x7fca68153580 2024-04-04T06:06:38.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.789+0000 7fca6ca00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fca6815cc80 con 0x7fca680d0990 2024-04-04T06:06:38.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.789+0000 7fca6ca00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fca6815da00 con 0x7fca680af2e0 2024-04-04T06:06:38.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.789+0000 7fca5d400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fca680d0990 0x7fca680aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:38.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.789+0000 7fca65e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fca68153580 0x7fca680d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37146/0 (socket says 172.21.15.159:37146) 2024-04-04T06:06:38.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.789+0000 7fca65e00640 1 -- 172.21.15.159:0/3968598859 learned_addr learned my addr 172.21.15.159:0/3968598859 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:38.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.790+0000 7fca66800640 1 --2- 172.21.15.159:0/3968598859 >> v2:172.21.15.178:3300/0 conn(0x7fca680af2e0 0x7fca681600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:38.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.790+0000 7fca5d400640 1 -- 172.21.15.159:0/3968598859 >> v2:172.21.15.178:3300/0 conn(0x7fca680af2e0 msgr2=0x7fca681600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:38.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.790+0000 7fca5d400640 1 --2- 172.21.15.159:0/3968598859 >> v2:172.21.15.178:3300/0 conn(0x7fca680af2e0 0x7fca681600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:38.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.790+0000 7fca5d400640 1 -- 172.21.15.159:0/3968598859 >> v1:172.21.15.159:6789/0 conn(0x7fca68153580 legacy=0x7fca680d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:38.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.790+0000 7fca5d400640 1 -- 172.21.15.159:0/3968598859 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6815cf20 con 0x7fca680d0990 2024-04-04T06:06:38.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.790+0000 7fca65400640 1 -- 172.21.15.159:0/3968598859 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 596745171 0 0) 0x7fca680afab0 con 0x7fca68153580 2024-04-04T06:06:38.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.791+0000 7fca5d400640 1 --2- 172.21.15.159:0/3968598859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca680d0990 0x7fca680aeda0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fca6806efa0 tx=0x7fca5002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266fc7464f4916f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:38.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.792+0000 7fca65400640 1 -- 172.21.15.159:0/3968598859 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50039070 con 0x7fca680d0990 2024-04-04T06:06:38.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.792+0000 7fca65400640 1 -- 172.21.15.159:0/3968598859 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca5002cbf0 con 0x7fca680d0990 2024-04-04T06:06:38.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.792+0000 7fca65400640 1 -- 172.21.15.159:0/3968598859 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50034440 con 0x7fca680d0990 2024-04-04T06:06:38.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.792+0000 7fca6ca00640 1 -- 172.21.15.159:0/3968598859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca680d0990 msgr2=0x7fca680aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:38.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.792+0000 7fca6ca00640 1 --2- 172.21.15.159:0/3968598859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca680d0990 0x7fca680aeda0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fca6806efa0 tx=0x7fca5002be40 comp rx=0 tx=0).stop 2024-04-04T06:06:38.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.792+0000 7fca6ca00640 1 -- 172.21.15.159:0/3968598859 shutdown_connections 2024-04-04T06:06:38.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.792+0000 7fca6ca00640 1 --2- 172.21.15.159:0/3968598859 >> v2:172.21.15.178:3300/0 conn(0x7fca680af2e0 0x7fca681600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:38.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.792+0000 7fca6ca00640 1 --2- 172.21.15.159:0/3968598859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca680d0990 0x7fca680aeda0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:38.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.792+0000 7fca6ca00640 1 -- 172.21.15.159:0/3968598859 >> 172.21.15.159:0/3968598859 conn(0x7fca6806b750 msgr2=0x7fca6806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:38.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.792+0000 7fca6ca00640 1 -- 172.21.15.159:0/3968598859 shutdown_connections 2024-04-04T06:06:38.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.792+0000 7fca6ca00640 1 -- 172.21.15.159:0/3968598859 wait complete. 2024-04-04T06:06:38.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.794+0000 7fca6ca00640 1 Processor -- start 2024-04-04T06:06:38.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.794+0000 7fca6ca00640 1 -- start start 2024-04-04T06:06:38.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.795+0000 7fca6ca00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca680af2e0 0x7fca681f7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:38.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.795+0000 7fca6ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca680d0990 0x7fca68154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:38.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.795+0000 7fca6ca00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca68153580 0x7fca68159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:38.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.795+0000 7fca6ca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fca68153960 con 0x7fca680d0990 2024-04-04T06:06:38.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.795+0000 7fca6ca00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fca6815d4e0 con 0x7fca68153580 2024-04-04T06:06:38.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.795+0000 7fca6ca00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fca680af6c0 con 0x7fca680af2e0 2024-04-04T06:06:38.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.796+0000 7fca5d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca680d0990 0x7fca68154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:38.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.796+0000 7fca5d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca680d0990 0x7fca68154b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36216/0 (socket says 172.21.15.159:36216) 2024-04-04T06:06:38.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.796+0000 7fca5d400640 1 -- 172.21.15.159:0/2589602642 learned_addr learned my addr 172.21.15.159:0/2589602642 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:38.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.796+0000 7fca66800640 1 --2- 172.21.15.159:0/2589602642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca68153580 0x7fca68159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:38.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.796+0000 7fca5d400640 1 -- 172.21.15.159:0/2589602642 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca680af2e0 msgr2=0x7fca681f7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:38.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.796+0000 7fca65e00640 1 --2- 172.21.15.159:0/2589602642 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca680af2e0 0x7fca681f7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:38.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.796+0000 7fca5d400640 1 --2- 172.21.15.159:0/2589602642 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca680af2e0 0x7fca681f7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:38.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.796+0000 7fca5d400640 1 -- 172.21.15.159:0/2589602642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca68153580 msgr2=0x7fca68159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:38.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.796+0000 7fca5d400640 1 --2- 172.21.15.159:0/2589602642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca68153580 0x7fca68159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:38.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.796+0000 7fca5d400640 1 -- 172.21.15.159:0/2589602642 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca68155260 con 0x7fca680d0990 2024-04-04T06:06:38.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.796+0000 7fca66800640 1 --2- 172.21.15.159:0/2589602642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca68153580 0x7fca68159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:38.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.796+0000 7fca65e00640 1 --2- 172.21.15.159:0/2589602642 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca680af2e0 0x7fca681f7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:38.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.796+0000 7fca5d400640 1 --2- 172.21.15.159:0/2589602642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca680d0990 0x7fca68154b30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fca50009e80 tx=0x7fca50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:38.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.797+0000 7fca5ea00640 1 -- 172.21.15.159:0/2589602642 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50039050 con 0x7fca680d0990 2024-04-04T06:06:38.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.797+0000 7fca5ea00640 1 -- 172.21.15.159:0/2589602642 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca5003a040 con 0x7fca680d0990 2024-04-04T06:06:38.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.797+0000 7fca5ea00640 1 -- 172.21.15.159:0/2589602642 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5003e610 con 0x7fca680d0990 2024-04-04T06:06:38.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.797+0000 7fca6ca00640 1 -- 172.21.15.159:0/2589602642 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca68155540 con 0x7fca680d0990 2024-04-04T06:06:38.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.797+0000 7fca6ca00640 1 -- 172.21.15.159:0/2589602642 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca68155b00 con 0x7fca680d0990 2024-04-04T06:06:38.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.798+0000 7fca6ca00640 1 -- 172.21.15.159:0/2589602642 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca680685c0 con 0x7fca680d0990 2024-04-04T06:06:38.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.799+0000 7fca5ea00640 1 -- 172.21.15.159:0/2589602642 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca50047020 con 0x7fca680d0990 2024-04-04T06:06:38.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.800+0000 7fca5ea00640 1 --2- 172.21.15.159:0/2589602642 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fca4406c8f0 0x7fca4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:38.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.800+0000 7fca5ea00640 1 -- 172.21.15.159:0/2589602642 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7fca500adf30 con 0x7fca680d0990 2024-04-04T06:06:38.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.801+0000 7fca65e00640 1 --2- 172.21.15.159:0/2589602642 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fca4406c8f0 0x7fca4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:38.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.802+0000 7fca65e00640 1 --2- 172.21.15.159:0/2589602642 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fca4406c8f0 0x7fca4406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fca58000f60 tx=0x7fca58009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:38.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:38.805+0000 7fca5ea00640 1 -- 172.21.15.159:0/2589602642 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fca5004e020 con 0x7fca680d0990 2024-04-04T06:06:39.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.007+0000 7fca6ca00640 1 -- 172.21.15.159:0/2589602642 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fca6806eca0 con 0x7fca680d0990 2024-04-04T06:06:39.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.007+0000 7fca5ea00640 1 -- 172.21.15.159:0/2589602642 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7fca5007c340 con 0x7fca680d0990 2024-04-04T06:06:39.002 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:06:39.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.010+0000 7fca6ca00640 1 -- 172.21.15.159:0/2589602642 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fca4406c8f0 msgr2=0x7fca4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:39.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.010+0000 7fca6ca00640 1 --2- 172.21.15.159:0/2589602642 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fca4406c8f0 0x7fca4406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fca58000f60 tx=0x7fca58009040 comp rx=0 tx=0).stop 2024-04-04T06:06:39.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.010+0000 7fca6ca00640 1 -- 172.21.15.159:0/2589602642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca680d0990 msgr2=0x7fca68154b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:39.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.010+0000 7fca6ca00640 1 --2- 172.21.15.159:0/2589602642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca680d0990 0x7fca68154b30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fca50009e80 tx=0x7fca50004a40 comp rx=0 tx=0).stop 2024-04-04T06:06:39.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.010+0000 7fca6ca00640 1 -- 172.21.15.159:0/2589602642 shutdown_connections 2024-04-04T06:06:39.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.010+0000 7fca6ca00640 1 --2- 172.21.15.159:0/2589602642 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fca4406c8f0 0x7fca4406edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:39.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.010+0000 7fca6ca00640 1 --2- 172.21.15.159:0/2589602642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca68153580 0x7fca68159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:39.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.010+0000 7fca6ca00640 1 --2- 172.21.15.159:0/2589602642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca680d0990 0x7fca68154b30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:39.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.010+0000 7fca6ca00640 1 --2- 172.21.15.159:0/2589602642 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca680af2e0 0x7fca681f7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:39.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.010+0000 7fca6ca00640 1 -- 172.21.15.159:0/2589602642 >> 172.21.15.159:0/2589602642 conn(0x7fca6806b750 msgr2=0x7fca6806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:39.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.010+0000 7fca6ca00640 1 -- 172.21.15.159:0/2589602642 shutdown_connections 2024-04-04T06:06:39.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.010+0000 7fca6ca00640 1 -- 172.21.15.159:0/2589602642 wait complete. 2024-04-04T06:06:39.025 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:06:39.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.236+0000 7f97dc600640 1 Processor -- start 2024-04-04T06:06:39.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.236+0000 7f97dc600640 1 -- start start 2024-04-04T06:06:39.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.238+0000 7f97dc600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97d4155550 0x7f97d4155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:39.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.238+0000 7f97dc600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97d40b40b0 con 0x7f97d40d0990 2024-04-04T06:06:39.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.238+0000 7f97dc600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97d40b4390 con 0x7f97d415f800 2024-04-04T06:06:39.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.238+0000 7f97dc600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f97d40b4670 con 0x7f97d4155550 2024-04-04T06:06:39.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.238+0000 7f97da600640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f97d40d0990 0x7f97d40aede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37162/0 (socket says 172.21.15.159:37162) 2024-04-04T06:06:39.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.238+0000 7f97da600640 1 -- 172.21.15.159:0/2666218507 learned_addr learned my addr 172.21.15.159:0/2666218507 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:39.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.238+0000 7f97d3e00640 1 -- 172.21.15.159:0/2666218507 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 441364728 0 0) 0x7f97d40b40b0 con 0x7f97d40d0990 2024-04-04T06:06:39.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.239+0000 7f97d3e00640 1 -- 172.21.15.159:0/2666218507 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97bc003520 con 0x7f97d40d0990 2024-04-04T06:06:39.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.239+0000 7f97d9c00640 1 --2- 172.21.15.159:0/2666218507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97d4155550 0x7f97d4155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:39.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.239+0000 7f97d3e00640 1 -- 172.21.15.159:0/2666218507 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1586676946 0 0) 0x7f97bc003520 con 0x7f97d40d0990 2024-04-04T06:06:39.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.239+0000 7f97d3e00640 1 -- 172.21.15.159:0/2666218507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97d4155550 msgr2=0x7f97d4155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:39.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.239+0000 7f97d3e00640 1 --2- 172.21.15.159:0/2666218507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97d4155550 0x7f97d4155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:39.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.239+0000 7f97d3e00640 1 -- 172.21.15.159:0/2666218507 >> v1:172.21.15.178:6789/0 conn(0x7f97d415f800 legacy=0x7f97d40d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:39.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.239+0000 7f97d3e00640 1 -- 172.21.15.159:0/2666218507 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d40b4950 con 0x7f97d40d0990 2024-04-04T06:06:39.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.239+0000 7f97d3e00640 1 -- 172.21.15.159:0/2666218507 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f97cc001a10 con 0x7f97d40d0990 2024-04-04T06:06:39.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.239+0000 7f97d3e00640 1 -- 172.21.15.159:0/2666218507 >> v1:172.21.15.159:6789/0 conn(0x7f97d40d0990 legacy=0x7f97d40aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:39.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.239+0000 7f97d3e00640 1 --2- 172.21.15.159:0/2666218507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97d4155550 0x7f97bc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:39.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.239+0000 7f97d3e00640 1 --2- 172.21.15.159:0/2666218507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97bc004060 0x7f97bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:39.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.239+0000 7f97d3e00640 1 --2- 172.21.15.159:0/2666218507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97bc004d40 0x7f97bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:39.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.239+0000 7f97d3e00640 1 -- 172.21.15.159:0/2666218507 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f97d40b4670 con 0x7f97bc004060 2024-04-04T06:06:39.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.239+0000 7f97d3e00640 1 -- 172.21.15.159:0/2666218507 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc00bca0 con 0x7f97bc004d40 2024-04-04T06:06:39.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.239+0000 7f97d3e00640 1 -- 172.21.15.159:0/2666218507 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f97bc00e0b0 con 0x7f97d4155550 2024-04-04T06:06:39.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.239+0000 7f97d9200640 1 --2- 172.21.15.159:0/2666218507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97bc004060 0x7f97bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:39.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.240+0000 7f97d9c00640 1 --2- 172.21.15.159:0/2666218507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97d4155550 0x7f97bc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:39.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.240+0000 7f97da600640 1 --2- 172.21.15.159:0/2666218507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97bc004d40 0x7f97bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:39.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.240+0000 7f97d9200640 1 -- 172.21.15.159:0/2666218507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97d4155550 msgr2=0x7f97bc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:39.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.240+0000 7f97d9200640 1 --2- 172.21.15.159:0/2666218507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97d4155550 0x7f97bc0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:39.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.240+0000 7f97d9200640 1 -- 172.21.15.159:0/2666218507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97bc004d40 msgr2=0x7f97bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:39.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.240+0000 7f97d9200640 1 --2- 172.21.15.159:0/2666218507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97bc004d40 0x7f97bc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:39.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.240+0000 7f97d9200640 1 -- 172.21.15.159:0/2666218507 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97bc003520 con 0x7f97bc004060 2024-04-04T06:06:39.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.241+0000 7f97d9200640 1 --2- 172.21.15.159:0/2666218507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97bc004060 0x7f97bc004800 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f97c0001750 tx=0x7f97bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac1a168d192c70b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:39.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.241+0000 7f97d3e00640 1 -- 172.21.15.159:0/2666218507 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c0001370 con 0x7f97bc004060 2024-04-04T06:06:39.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.241+0000 7f97d3e00640 1 -- 172.21.15.159:0/2666218507 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97c00052d0 con 0x7f97bc004060 2024-04-04T06:06:39.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.241+0000 7f97d3e00640 1 -- 172.21.15.159:0/2666218507 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c0003830 con 0x7f97bc004060 2024-04-04T06:06:39.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.242+0000 7f97dc600640 1 -- 172.21.15.159:0/2666218507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97bc004060 msgr2=0x7f97bc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:39.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.242+0000 7f97dc600640 1 --2- 172.21.15.159:0/2666218507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97bc004060 0x7f97bc004800 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f97c0001750 tx=0x7f97bc005830 comp rx=0 tx=0).stop 2024-04-04T06:06:39.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.242+0000 7f97da600640 1 -- 172.21.15.159:0/2666218507 reap_dead start 2024-04-04T06:06:39.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.242+0000 7f97dc600640 1 -- 172.21.15.159:0/2666218507 shutdown_connections 2024-04-04T06:06:39.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.242+0000 7f97dc600640 1 -- 172.21.15.159:0/2666218507 >> 172.21.15.159:0/2666218507 conn(0x7f97d406b750 msgr2=0x7f97d406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:39.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.242+0000 7f97dc600640 1 -- 172.21.15.159:0/2666218507 shutdown_connections 2024-04-04T06:06:39.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.242+0000 7f97dc600640 1 -- 172.21.15.159:0/2666218507 wait complete. 2024-04-04T06:06:39.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.244+0000 7f97dc600640 1 Processor -- start 2024-04-04T06:06:39.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.244+0000 7f97dc600640 1 -- start start 2024-04-04T06:06:39.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.245+0000 7f97dc600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97bc004060 0x7f97d415d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:39.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.245+0000 7f97dc600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97d415a920 0x7f97d41f7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:39.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.245+0000 7f97dc600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97d40b2170 0x7f97d40b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:39.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.245+0000 7f97dc600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f97d415ba20 con 0x7f97d40b2170 2024-04-04T06:06:39.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.245+0000 7f97dc600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97d415fc80 con 0x7f97d415a920 2024-04-04T06:06:39.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.245+0000 7f97dc600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f97d415bba0 con 0x7f97bc004060 2024-04-04T06:06:39.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.245+0000 7f97da600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97d40b2170 0x7f97d40b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:39.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.245+0000 7f97da600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97d40b2170 0x7f97d40b25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36242/0 (socket says 172.21.15.159:36242) 2024-04-04T06:06:39.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.245+0000 7f97da600640 1 -- 172.21.15.159:0/3773728966 learned_addr learned my addr 172.21.15.159:0/3773728966 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:39.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.245+0000 7f97d9c00640 1 --2- 172.21.15.159:0/3773728966 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97bc004060 0x7f97d415d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:39.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.245+0000 7f97d9200640 1 --2- 172.21.15.159:0/3773728966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97d415a920 0x7f97d41f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:39.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.245+0000 7f97da600640 1 -- 172.21.15.159:0/3773728966 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97bc004060 msgr2=0x7f97d415d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:39.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.245+0000 7f97da600640 1 --2- 172.21.15.159:0/3773728966 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97bc004060 0x7f97d415d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:39.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.245+0000 7f97da600640 1 -- 172.21.15.159:0/3773728966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97d415a920 msgr2=0x7f97d41f7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:39.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.245+0000 7f97da600640 1 --2- 172.21.15.159:0/3773728966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97d415a920 0x7f97d41f7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:39.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.245+0000 7f97da600640 1 -- 172.21.15.159:0/3773728966 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d406e620 con 0x7f97d40b2170 2024-04-04T06:06:39.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.246+0000 7f97da600640 1 --2- 172.21.15.159:0/3773728966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97d40b2170 0x7f97d40b25f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f97d4155e90 tx=0x7f97cc00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:39.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.246+0000 7f97d1600640 1 -- 172.21.15.159:0/3773728966 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc004d60 con 0x7f97d40b2170 2024-04-04T06:06:39.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.246+0000 7f97d1600640 1 -- 172.21.15.159:0/3773728966 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97cc00bb80 con 0x7f97d40b2170 2024-04-04T06:06:39.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.246+0000 7f97d1600640 1 -- 172.21.15.159:0/3773728966 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc007110 con 0x7f97d40b2170 2024-04-04T06:06:39.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.246+0000 7f97dc600640 1 -- 172.21.15.159:0/3773728966 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d406e8c0 con 0x7f97d40b2170 2024-04-04T06:06:39.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.246+0000 7f97dc600640 1 -- 172.21.15.159:0/3773728966 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d409be60 con 0x7f97d40b2170 2024-04-04T06:06:39.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.247+0000 7f97dc600640 1 -- 172.21.15.159:0/3773728966 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97d40685c0 con 0x7f97d40b2170 2024-04-04T06:06:39.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.248+0000 7f97d1600640 1 -- 172.21.15.159:0/3773728966 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f97cc01b020 con 0x7f97d40b2170 2024-04-04T06:06:39.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.249+0000 7f97d1600640 1 --2- 172.21.15.159:0/3773728966 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f97b406c8f0 0x7f97b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:39.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.249+0000 7f97d1600640 1 -- 172.21.15.159:0/3773728966 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f97cc08bdf0 con 0x7f97d40b2170 2024-04-04T06:06:39.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.250+0000 7f97d9c00640 1 --2- 172.21.15.159:0/3773728966 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f97b406c8f0 0x7f97b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:39.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.251+0000 7f97d9c00640 1 --2- 172.21.15.159:0/3773728966 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f97b406c8f0 0x7f97b406edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f97c8006fd0 tx=0x7f97c8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:39.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.254+0000 7f97d1600640 1 -- 172.21.15.159:0/3773728966 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f97cc05a160 con 0x7f97d40b2170 2024-04-04T06:06:39.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.456+0000 7f97dc600640 1 -- 172.21.15.159:0/3773728966 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f97d40b3ad0 con 0x7f97d40b2170 2024-04-04T06:06:39.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.457+0000 7f97d1600640 1 -- 172.21.15.159:0/3773728966 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f97cc059a60 con 0x7f97d40b2170 2024-04-04T06:06:39.452 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:06:39.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.460+0000 7f97dc600640 1 -- 172.21.15.159:0/3773728966 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f97b406c8f0 msgr2=0x7f97b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:39.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.460+0000 7f97dc600640 1 --2- 172.21.15.159:0/3773728966 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f97b406c8f0 0x7f97b406edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f97c8006fd0 tx=0x7f97c8008040 comp rx=0 tx=0).stop 2024-04-04T06:06:39.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.460+0000 7f97dc600640 1 -- 172.21.15.159:0/3773728966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97d40b2170 msgr2=0x7f97d40b25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:39.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.460+0000 7f97dc600640 1 --2- 172.21.15.159:0/3773728966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97d40b2170 0x7f97d40b25f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f97d4155e90 tx=0x7f97cc00b4a0 comp rx=0 tx=0).stop 2024-04-04T06:06:39.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.460+0000 7f97dc600640 1 -- 172.21.15.159:0/3773728966 shutdown_connections 2024-04-04T06:06:39.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.460+0000 7f97dc600640 1 --2- 172.21.15.159:0/3773728966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97d40b2170 0x7f97d40b25f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:39.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.460+0000 7f97dc600640 1 --2- 172.21.15.159:0/3773728966 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f97b406c8f0 0x7f97b406edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:39.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.460+0000 7f97dc600640 1 --2- 172.21.15.159:0/3773728966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97d415a920 0x7f97d41f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:39.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.460+0000 7f97dc600640 1 --2- 172.21.15.159:0/3773728966 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97bc004060 0x7f97d415d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:39.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.460+0000 7f97dc600640 1 -- 172.21.15.159:0/3773728966 >> 172.21.15.159:0/3773728966 conn(0x7f97d406b750 msgr2=0x7f97d41527c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:39.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.460+0000 7f97dc600640 1 -- 172.21.15.159:0/3773728966 shutdown_connections 2024-04-04T06:06:39.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.460+0000 7f97dc600640 1 -- 172.21.15.159:0/3773728966 wait complete. 2024-04-04T06:06:39.472 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:06:39.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.685+0000 7fe06ec00640 1 Processor -- start 2024-04-04T06:06:39.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.685+0000 7fe06ec00640 1 -- start start 2024-04-04T06:06:39.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.686+0000 7fe06ec00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe06806dd30 0x7fe0680cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:39.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.686+0000 7fe06ec00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0680ce890 con 0x7fe0680cc880 2024-04-04T06:06:39.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.686+0000 7fe06ec00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0680ceb50 con 0x7fe06813f190 2024-04-04T06:06:39.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.686+0000 7fe06ec00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe0680cd230 con 0x7fe06806dd30 2024-04-04T06:06:39.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.687+0000 7fe06ce00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fe0680cc880 0x7fe06806d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37176/0 (socket says 172.21.15.159:37176) 2024-04-04T06:06:39.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.687+0000 7fe06ce00640 1 -- 172.21.15.159:0/4166871900 learned_addr learned my addr 172.21.15.159:0/4166871900 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:39.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.687+0000 7fe06e200640 1 --2- 172.21.15.159:0/4166871900 >> v2:172.21.15.159:3300/0 conn(0x7fe06806dd30 0x7fe0680cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:39.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.687+0000 7fe067e00640 1 -- 172.21.15.159:0/4166871900 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2206452977 0 0) 0x7fe0680ce890 con 0x7fe0680cc880 2024-04-04T06:06:39.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.687+0000 7fe067e00640 1 -- 172.21.15.159:0/4166871900 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe050003520 con 0x7fe0680cc880 2024-04-04T06:06:39.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.687+0000 7fe06e200640 1 -- 172.21.15.159:0/4166871900 >> v1:172.21.15.178:6789/0 conn(0x7fe06813f190 legacy=0x7fe0680cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:39.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.687+0000 7fe06e200640 1 -- 172.21.15.159:0/4166871900 >> v1:172.21.15.159:6789/0 conn(0x7fe0680cc880 legacy=0x7fe06806d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:39.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.687+0000 7fe06e200640 1 -- 172.21.15.159:0/4166871900 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0680cee30 con 0x7fe06806dd30 2024-04-04T06:06:39.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.688+0000 7fe06e200640 1 --2- 172.21.15.159:0/4166871900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe06806dd30 0x7fe0680cccf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe060009d90 tx=0x7fe06002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d754239e4e829b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:39.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.689+0000 7fe067e00640 1 -- 172.21.15.159:0/4166871900 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060039070 con 0x7fe06806dd30 2024-04-04T06:06:39.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.689+0000 7fe067e00640 1 -- 172.21.15.159:0/4166871900 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe06002cd00 con 0x7fe06806dd30 2024-04-04T06:06:39.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.689+0000 7fe067e00640 1 -- 172.21.15.159:0/4166871900 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060034660 con 0x7fe06806dd30 2024-04-04T06:06:39.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.689+0000 7fe06ec00640 1 -- 172.21.15.159:0/4166871900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe06806dd30 msgr2=0x7fe0680cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:39.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.689+0000 7fe06ec00640 1 --2- 172.21.15.159:0/4166871900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe06806dd30 0x7fe0680cccf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe060009d90 tx=0x7fe06002bf50 comp rx=0 tx=0).stop 2024-04-04T06:06:39.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.689+0000 7fe06ec00640 1 -- 172.21.15.159:0/4166871900 shutdown_connections 2024-04-04T06:06:39.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.689+0000 7fe06ec00640 1 --2- 172.21.15.159:0/4166871900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe06806dd30 0x7fe0680cccf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:39.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.689+0000 7fe06ec00640 1 -- 172.21.15.159:0/4166871900 >> 172.21.15.159:0/4166871900 conn(0x7fe0680b1ba0 msgr2=0x7fe0680b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:39.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.689+0000 7fe06ec00640 1 -- 172.21.15.159:0/4166871900 shutdown_connections 2024-04-04T06:06:39.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.689+0000 7fe06ec00640 1 -- 172.21.15.159:0/4166871900 wait complete. 2024-04-04T06:06:39.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.691+0000 7fe06ec00640 1 Processor -- start 2024-04-04T06:06:39.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.691+0000 7fe06ec00640 1 -- start start 2024-04-04T06:06:39.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.692+0000 7fe06ec00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe06806dd30 0x7fe0681ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:39.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.692+0000 7fe06ec00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0680cc880 0x7fe0681580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:39.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.692+0000 7fe06ec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe06813f190 0x7fe0681585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:39.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.692+0000 7fe06ec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe06813f570 con 0x7fe06813f190 2024-04-04T06:06:39.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.692+0000 7fe06ec00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe068160c70 con 0x7fe0680cc880 2024-04-04T06:06:39.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.692+0000 7fe06ec00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe06815ff10 con 0x7fe06806dd30 2024-04-04T06:06:39.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.693+0000 7fe06e200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe06813f190 0x7fe0681585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:39.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.693+0000 7fe06e200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe06813f190 0x7fe0681585e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36250/0 (socket says 172.21.15.159:36250) 2024-04-04T06:06:39.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.693+0000 7fe06e200640 1 -- 172.21.15.159:0/977631462 learned_addr learned my addr 172.21.15.159:0/977631462 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:39.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.693+0000 7fe06d800640 1 --2- 172.21.15.159:0/977631462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe06806dd30 0x7fe0681ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:39.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.693+0000 7fe06e200640 1 -- 172.21.15.159:0/977631462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe06806dd30 msgr2=0x7fe0681ef2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:39.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.693+0000 7fe06e200640 1 --2- 172.21.15.159:0/977631462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe06806dd30 0x7fe0681ef2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:39.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.693+0000 7fe06ce00640 1 --2- 172.21.15.159:0/977631462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0680cc880 0x7fe0681580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:39.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.693+0000 7fe06e200640 1 -- 172.21.15.159:0/977631462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0680cc880 msgr2=0x7fe0681580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:39.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.693+0000 7fe06e200640 1 --2- 172.21.15.159:0/977631462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0680cc880 0x7fe0681580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:39.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.693+0000 7fe06e200640 1 -- 172.21.15.159:0/977631462 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0681572e0 con 0x7fe06813f190 2024-04-04T06:06:39.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.693+0000 7fe06e200640 1 --2- 172.21.15.159:0/977631462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe06813f190 0x7fe0681585e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe060009d90 tx=0x7fe060004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:39.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.694+0000 7fe065600640 1 -- 172.21.15.159:0/977631462 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060039050 con 0x7fe06813f190 2024-04-04T06:06:39.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.694+0000 7fe065600640 1 -- 172.21.15.159:0/977631462 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe060045070 con 0x7fe06813f190 2024-04-04T06:06:39.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.694+0000 7fe06ec00640 1 -- 172.21.15.159:0/977631462 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0681575c0 con 0x7fe06813f190 2024-04-04T06:06:39.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.694+0000 7fe065600640 1 -- 172.21.15.159:0/977631462 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060009450 con 0x7fe06813f190 2024-04-04T06:06:39.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.694+0000 7fe06ec00640 1 -- 172.21.15.159:0/977631462 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0681ef820 con 0x7fe06813f190 2024-04-04T06:06:39.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.695+0000 7fe06ec00640 1 -- 172.21.15.159:0/977631462 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe068075940 con 0x7fe06813f190 2024-04-04T06:06:39.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.696+0000 7fe065600640 1 -- 172.21.15.159:0/977631462 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe06003e850 con 0x7fe06813f190 2024-04-04T06:06:39.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.697+0000 7fe065600640 1 --2- 172.21.15.159:0/977631462 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe04806c8f0 0x7fe04806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:39.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.697+0000 7fe065600640 1 -- 172.21.15.159:0/977631462 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7fe0600ae850 con 0x7fe06813f190 2024-04-04T06:06:39.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.697+0000 7fe06d800640 1 --2- 172.21.15.159:0/977631462 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe04806c8f0 0x7fe04806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:39.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.699+0000 7fe06d800640 1 --2- 172.21.15.159:0/977631462 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe04806c8f0 0x7fe04806edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe068156560 tx=0x7fe05c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:39.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.702+0000 7fe065600640 1 -- 172.21.15.159:0/977631462 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe06007cbc0 con 0x7fe06813f190 2024-04-04T06:06:39.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.904+0000 7fe06ec00640 1 -- 172.21.15.159:0/977631462 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe0680b2870 con 0x7fe06813f190 2024-04-04T06:06:39.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.906+0000 7fe065600640 1 -- 172.21.15.159:0/977631462 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fe06007c4c0 con 0x7fe06813f190 2024-04-04T06:06:39.900 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:06:39.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.908+0000 7fe06ec00640 1 -- 172.21.15.159:0/977631462 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe04806c8f0 msgr2=0x7fe04806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:39.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.908+0000 7fe06ec00640 1 --2- 172.21.15.159:0/977631462 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe04806c8f0 0x7fe04806edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe068156560 tx=0x7fe05c009040 comp rx=0 tx=0).stop 2024-04-04T06:06:39.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.908+0000 7fe06ec00640 1 -- 172.21.15.159:0/977631462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe06813f190 msgr2=0x7fe0681585e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:39.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.908+0000 7fe06ec00640 1 --2- 172.21.15.159:0/977631462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe06813f190 0x7fe0681585e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe060009d90 tx=0x7fe060004a40 comp rx=0 tx=0).stop 2024-04-04T06:06:39.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.908+0000 7fe06ec00640 1 -- 172.21.15.159:0/977631462 shutdown_connections 2024-04-04T06:06:39.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.908+0000 7fe06ec00640 1 --2- 172.21.15.159:0/977631462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe06813f190 0x7fe0681585e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:39.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.908+0000 7fe06ec00640 1 --2- 172.21.15.159:0/977631462 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe04806c8f0 0x7fe04806edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:39.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.908+0000 7fe06ec00640 1 --2- 172.21.15.159:0/977631462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0680cc880 0x7fe0681580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:39.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.908+0000 7fe06ec00640 1 --2- 172.21.15.159:0/977631462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe06806dd30 0x7fe0681ef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:39.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.908+0000 7fe06ec00640 1 -- 172.21.15.159:0/977631462 >> 172.21.15.159:0/977631462 conn(0x7fe0680b1ba0 msgr2=0x7fe0681602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:39.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.908+0000 7fe06ec00640 1 -- 172.21.15.159:0/977631462 shutdown_connections 2024-04-04T06:06:39.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:39.909+0000 7fe06ec00640 1 -- 172.21.15.159:0/977631462 wait complete. 2024-04-04T06:06:39.920 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-04T06:06:40.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.128+0000 7f4087000640 1 Processor -- start 2024-04-04T06:06:40.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.129+0000 7f4087000640 1 -- start start 2024-04-04T06:06:40.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.130+0000 7f4087000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f40800d0280 0x7f40800d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:40.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.130+0000 7f4087000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4080141c10 0x7f4080071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:40.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.130+0000 7f4087000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4080071eb0 0x7f408006d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:40.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.130+0000 7f4087000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f408015ec80 con 0x7f4080141c10 2024-04-04T06:06:40.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.130+0000 7f4087000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f40800d0ba0 con 0x7f40800d0280 2024-04-04T06:06:40.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.130+0000 7f4087000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f408015ef20 con 0x7f4080071eb0 2024-04-04T06:06:40.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.130+0000 7f407f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4080141c10 0x7f4080071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:40.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.130+0000 7f407f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4080141c10 0x7f4080071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36258/0 (socket says 172.21.15.159:36258) 2024-04-04T06:06:40.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.130+0000 7f407f400640 1 -- 172.21.15.159:0/3460772954 learned_addr learned my addr 172.21.15.159:0/3460772954 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:40.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.131+0000 7f407fe00640 1 --2- 172.21.15.159:0/3460772954 >> v2:172.21.15.178:3300/0 conn(0x7f40800d0280 0x7f40800d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:40.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.131+0000 7f4085000640 1 --2- 172.21.15.159:0/3460772954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4080071eb0 0x7f408006d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:40.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.131+0000 7f407f400640 1 -- 172.21.15.159:0/3460772954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4080071eb0 msgr2=0x7f408006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:40.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.131+0000 7f407f400640 1 --2- 172.21.15.159:0/3460772954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4080071eb0 0x7f408006d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:40.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.131+0000 7f407f400640 1 -- 172.21.15.159:0/3460772954 >> v2:172.21.15.178:3300/0 conn(0x7f40800d0280 msgr2=0x7f40800d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:40.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.131+0000 7f407f400640 1 --2- 172.21.15.159:0/3460772954 >> v2:172.21.15.178:3300/0 conn(0x7f40800d0280 0x7f40800d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:40.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.131+0000 7f407f400640 1 -- 172.21.15.159:0/3460772954 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408015f200 con 0x7f4080141c10 2024-04-04T06:06:40.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.131+0000 7f407fe00640 1 --2- 172.21.15.159:0/3460772954 >> v2:172.21.15.178:3300/0 conn(0x7f40800d0280 0x7f40800d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:40.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.132+0000 7f407f400640 1 --2- 172.21.15.159:0/3460772954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4080141c10 0x7f4080071970 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f40800d1050 tx=0x7f406802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c4a5c7f9ae9d42 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:40.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.132+0000 7f407ea00640 1 -- 172.21.15.159:0/3460772954 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068035020 con 0x7f4080141c10 2024-04-04T06:06:40.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.132+0000 7f407ea00640 1 -- 172.21.15.159:0/3460772954 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4068031870 con 0x7f4080141c10 2024-04-04T06:06:40.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.132+0000 7f407ea00640 1 -- 172.21.15.159:0/3460772954 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068031c40 con 0x7f4080141c10 2024-04-04T06:06:40.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.133+0000 7f4087000640 1 -- 172.21.15.159:0/3460772954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4080141c10 msgr2=0x7f4080071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:40.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.133+0000 7f4087000640 1 --2- 172.21.15.159:0/3460772954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4080141c10 0x7f4080071970 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f40800d1050 tx=0x7f406802b910 comp rx=0 tx=0).stop 2024-04-04T06:06:40.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.133+0000 7f4087000640 1 -- 172.21.15.159:0/3460772954 shutdown_connections 2024-04-04T06:06:40.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.133+0000 7f4087000640 1 --2- 172.21.15.159:0/3460772954 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4080071eb0 0x7f408006d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:40.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.133+0000 7f4087000640 1 --2- 172.21.15.159:0/3460772954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4080141c10 0x7f4080071970 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:40.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.133+0000 7f4087000640 1 --2- 172.21.15.159:0/3460772954 >> v2:172.21.15.178:3300/0 conn(0x7f40800d0280 0x7f40800d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:40.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.133+0000 7f4087000640 1 -- 172.21.15.159:0/3460772954 >> 172.21.15.159:0/3460772954 conn(0x7f408006b750 msgr2=0x7f408006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:40.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.133+0000 7f4087000640 1 -- 172.21.15.159:0/3460772954 shutdown_connections 2024-04-04T06:06:40.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.133+0000 7f4087000640 1 -- 172.21.15.159:0/3460772954 wait complete. 2024-04-04T06:06:40.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.135+0000 7f4087000640 1 Processor -- start 2024-04-04T06:06:40.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.135+0000 7f4087000640 1 -- start start 2024-04-04T06:06:40.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.136+0000 7f4087000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4080071eb0 0x7f4080156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:40.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.136+0000 7f4087000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40800d0280 0x7f40800995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:40.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.136+0000 7f4087000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4080141c10 0x7f40801578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:40.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.136+0000 7f4087000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4080158e90 con 0x7f40800d0280 2024-04-04T06:06:40.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.136+0000 7f4087000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f408015f7c0 con 0x7f4080071eb0 2024-04-04T06:06:40.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.136+0000 7f4087000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4080157430 con 0x7f4080141c10 2024-04-04T06:06:40.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.136+0000 7f407f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40800d0280 0x7f40800995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:40.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.136+0000 7f407f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40800d0280 0x7f40800995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36274/0 (socket says 172.21.15.159:36274) 2024-04-04T06:06:40.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.136+0000 7f407f400640 1 -- 172.21.15.159:0/3568666114 learned_addr learned my addr 172.21.15.159:0/3568666114 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:40.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.136+0000 7f407fe00640 1 --2- 172.21.15.159:0/3568666114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4080071eb0 0x7f4080156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:40.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.136+0000 7f407f400640 1 -- 172.21.15.159:0/3568666114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4080141c10 msgr2=0x7f40801578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:40.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.136+0000 7f407f400640 1 --2- 172.21.15.159:0/3568666114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4080141c10 0x7f40801578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:40.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.136+0000 7f407f400640 1 -- 172.21.15.159:0/3568666114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4080071eb0 msgr2=0x7f4080156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:40.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.136+0000 7f407f400640 1 --2- 172.21.15.159:0/3568666114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4080071eb0 0x7f4080156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:40.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.136+0000 7f407f400640 1 -- 172.21.15.159:0/3568666114 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40801565a0 con 0x7f40800d0280 2024-04-04T06:06:40.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.137+0000 7f407fe00640 1 --2- 172.21.15.159:0/3568666114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4080071eb0 0x7f4080156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:40.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.137+0000 7f407f400640 1 --2- 172.21.15.159:0/3568666114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40800d0280 0x7f40800995b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f406802b950 tx=0x7f40680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:40.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.137+0000 7f4077e00640 1 -- 172.21.15.159:0/3568666114 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068040070 con 0x7f40800d0280 2024-04-04T06:06:40.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.137+0000 7f4077e00640 1 -- 172.21.15.159:0/3568666114 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f406803a420 con 0x7f40800d0280 2024-04-04T06:06:40.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.137+0000 7f4077e00640 1 -- 172.21.15.159:0/3568666114 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068035040 con 0x7f40800d0280 2024-04-04T06:06:40.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.137+0000 7f4087000640 1 -- 172.21.15.159:0/3568666114 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4080156820 con 0x7f40800d0280 2024-04-04T06:06:40.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.137+0000 7f4087000640 1 -- 172.21.15.159:0/3568666114 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4080156de0 con 0x7f40800d0280 2024-04-04T06:06:40.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.139+0000 7f4087000640 1 -- 172.21.15.159:0/3568666114 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40801575b0 con 0x7f40800d0280 2024-04-04T06:06:40.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.139+0000 7f4077e00640 1 -- 172.21.15.159:0/3568666114 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f406804b060 con 0x7f40800d0280 2024-04-04T06:06:40.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.140+0000 7f4077e00640 1 --2- 172.21.15.159:0/3568666114 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f405c06c8f0 0x7f405c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:40.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.140+0000 7f4077e00640 1 -- 172.21.15.159:0/3568666114 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f40680adaf0 con 0x7f40800d0280 2024-04-04T06:06:40.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.141+0000 7f407fe00640 1 --2- 172.21.15.159:0/3568666114 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f405c06c8f0 0x7f405c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:40.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.142+0000 7f407fe00640 1 --2- 172.21.15.159:0/3568666114 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f405c06c8f0 0x7f405c06edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f407000d430 tx=0x7f4070007530 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:40.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.144+0000 7f4077e00640 1 -- 172.21.15.159:0/3568666114 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f40801565a0 con 0x7f40800d0280 2024-04-04T06:06:40.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.418+0000 7f4087000640 1 -- 172.21.15.159:0/3568666114 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7f40800685c0 con 0x7f40800d0280 2024-04-04T06:06:40.671 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:06:40.680 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:06:40.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.859+0000 7f4077e00640 1 -- 172.21.15.159:0/3568666114 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v69) ==== 145+0+0 (secure 0 0 0) 0x7f406807bee0 con 0x7f40800d0280 2024-04-04T06:06:40.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.863+0000 7f4087000640 1 -- 172.21.15.159:0/3568666114 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f405c06c8f0 msgr2=0x7f405c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:40.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.863+0000 7f4087000640 1 --2- 172.21.15.159:0/3568666114 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f405c06c8f0 0x7f405c06edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f407000d430 tx=0x7f4070007530 comp rx=0 tx=0).stop 2024-04-04T06:06:40.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.864+0000 7f4087000640 1 -- 172.21.15.159:0/3568666114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40800d0280 msgr2=0x7f40800995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:40.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.864+0000 7f4087000640 1 --2- 172.21.15.159:0/3568666114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40800d0280 0x7f40800995b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f406802b950 tx=0x7f40680046c0 comp rx=0 tx=0).stop 2024-04-04T06:06:40.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.864+0000 7f4087000640 1 -- 172.21.15.159:0/3568666114 shutdown_connections 2024-04-04T06:06:40.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.864+0000 7f4087000640 1 --2- 172.21.15.159:0/3568666114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4080141c10 0x7f40801578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:40.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.864+0000 7f4087000640 1 --2- 172.21.15.159:0/3568666114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40800d0280 0x7f40800995b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:40.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.864+0000 7f4087000640 1 --2- 172.21.15.159:0/3568666114 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f405c06c8f0 0x7f405c06edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:40.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.864+0000 7f4087000640 1 --2- 172.21.15.159:0/3568666114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4080071eb0 0x7f4080156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:40.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.864+0000 7f4087000640 1 -- 172.21.15.159:0/3568666114 >> 172.21.15.159:0/3568666114 conn(0x7f408006b750 msgr2=0x7f40800b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:40.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.864+0000 7f4087000640 1 -- 172.21.15.159:0/3568666114 shutdown_connections 2024-04-04T06:06:40.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:40.864+0000 7f4087000640 1 -- 172.21.15.159:0/3568666114 wait complete. 2024-04-04T06:06:40.881 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-04T06:06:41.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.095+0000 7f74c4000640 1 Processor -- start 2024-04-04T06:06:41.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.096+0000 7f74c4000640 1 -- start start 2024-04-04T06:06:41.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.097+0000 7f74c4000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74bc0d0280 0x7f74bc0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:41.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.097+0000 7f74c4000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74bc15ec80 con 0x7f74bc162160 2024-04-04T06:06:41.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.097+0000 7f74c4000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74bc0d0ba0 con 0x7f74bc141c10 2024-04-04T06:06:41.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.097+0000 7f74c4000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f74bc15ef20 con 0x7f74bc0d0280 2024-04-04T06:06:41.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.097+0000 7f74c2000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f74bc162160 0x7f74bc1625e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37184/0 (socket says 172.21.15.159:37184) 2024-04-04T06:06:41.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.097+0000 7f74c2000640 1 -- 172.21.15.159:0/196328460 learned_addr learned my addr 172.21.15.159:0/196328460 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:41.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.098+0000 7f74c1600640 1 --2- 172.21.15.159:0/196328460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74bc0d0280 0x7f74bc0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:41.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.098+0000 7f74bbe00640 1 -- 172.21.15.159:0/196328460 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4112019101 0 0) 0x7f74bc15ec80 con 0x7f74bc162160 2024-04-04T06:06:41.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.098+0000 7f74bbe00640 1 -- 172.21.15.159:0/196328460 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74a4003520 con 0x7f74bc162160 2024-04-04T06:06:41.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.098+0000 7f74bbe00640 1 -- 172.21.15.159:0/196328460 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 665900547 0 0) 0x7f74a4003520 con 0x7f74bc162160 2024-04-04T06:06:41.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.098+0000 7f74bbe00640 1 -- 172.21.15.159:0/196328460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74bc0d0280 msgr2=0x7f74bc0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:41.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.098+0000 7f74bbe00640 1 --2- 172.21.15.159:0/196328460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74bc0d0280 0x7f74bc0d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:41.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.098+0000 7f74bbe00640 1 -- 172.21.15.159:0/196328460 >> v1:172.21.15.178:6789/0 conn(0x7f74bc141c10 legacy=0x7f74bc071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:41.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.098+0000 7f74bbe00640 1 -- 172.21.15.159:0/196328460 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc15f200 con 0x7f74bc162160 2024-04-04T06:06:41.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.098+0000 7f74bbe00640 1 -- 172.21.15.159:0/196328460 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f74b40019f0 con 0x7f74bc162160 2024-04-04T06:06:41.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.098+0000 7f74bbe00640 1 -- 172.21.15.159:0/196328460 >> v1:172.21.15.159:6789/0 conn(0x7f74bc162160 legacy=0x7f74bc1625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:41.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.098+0000 7f74bbe00640 1 --2- 172.21.15.159:0/196328460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74bc0d0280 0x7f74a40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:41.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.098+0000 7f74bbe00640 1 --2- 172.21.15.159:0/196328460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74a4003f60 0x7f74a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:41.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.099+0000 7f74bbe00640 1 --2- 172.21.15.159:0/196328460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74a4004d40 0x7f74a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:41.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.099+0000 7f74bbe00640 1 -- 172.21.15.159:0/196328460 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc15ef20 con 0x7f74a4003f60 2024-04-04T06:06:41.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.099+0000 7f74bbe00640 1 -- 172.21.15.159:0/196328460 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f74a400baa0 con 0x7f74a4004d40 2024-04-04T06:06:41.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.099+0000 7f74bbe00640 1 -- 172.21.15.159:0/196328460 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f74a400bca0 con 0x7f74bc0d0280 2024-04-04T06:06:41.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.099+0000 7f74c0c00640 1 --2- 172.21.15.159:0/196328460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74a4003f60 0x7f74a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:41.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.099+0000 7f74c0c00640 1 -- 172.21.15.159:0/196328460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74bc0d0280 msgr2=0x7f74a40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:41.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.099+0000 7f74c0c00640 1 --2- 172.21.15.159:0/196328460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74bc0d0280 0x7f74a40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:41.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.099+0000 7f74c0c00640 1 -- 172.21.15.159:0/196328460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74a4004d40 msgr2=0x7f74a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:41.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.099+0000 7f74c0c00640 1 --2- 172.21.15.159:0/196328460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74a4004d40 0x7f74a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:41.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.099+0000 7f74c0c00640 1 -- 172.21.15.159:0/196328460 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a4003520 con 0x7f74a4003f60 2024-04-04T06:06:41.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.100+0000 7f74c0c00640 1 --2- 172.21.15.159:0/196328460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74a4003f60 0x7f74a4004800 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f74bc148260 tx=0x7f74a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48673ab6c0e3ad1e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:41.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.101+0000 7f74bbe00640 1 -- 172.21.15.159:0/196328460 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a8001370 con 0x7f74a4003f60 2024-04-04T06:06:41.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.101+0000 7f74bbe00640 1 -- 172.21.15.159:0/196328460 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74a80052d0 con 0x7f74a4003f60 2024-04-04T06:06:41.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.101+0000 7f74bbe00640 1 -- 172.21.15.159:0/196328460 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a8003830 con 0x7f74a4003f60 2024-04-04T06:06:41.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.101+0000 7f74c4000640 1 -- 172.21.15.159:0/196328460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74a4003f60 msgr2=0x7f74a4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:41.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.101+0000 7f74c4000640 1 --2- 172.21.15.159:0/196328460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74a4003f60 0x7f74a4004800 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f74bc148260 tx=0x7f74a4005830 comp rx=0 tx=0).stop 2024-04-04T06:06:41.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.101+0000 7f74c2000640 1 -- 172.21.15.159:0/196328460 reap_dead start 2024-04-04T06:06:41.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.101+0000 7f74c4000640 1 -- 172.21.15.159:0/196328460 shutdown_connections 2024-04-04T06:06:41.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.101+0000 7f74c4000640 1 -- 172.21.15.159:0/196328460 >> 172.21.15.159:0/196328460 conn(0x7f74bc06b750 msgr2=0x7f74bc06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:41.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.101+0000 7f74c4000640 1 -- 172.21.15.159:0/196328460 shutdown_connections 2024-04-04T06:06:41.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.101+0000 7f74c4000640 1 -- 172.21.15.159:0/196328460 wait complete. 2024-04-04T06:06:41.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.103+0000 7f74c4000640 1 Processor -- start 2024-04-04T06:06:41.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.103+0000 7f74c4000640 1 -- start start 2024-04-04T06:06:41.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.104+0000 7f74c4000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74a4003f60 0x7f74bc15cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:41.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.104+0000 7f74c4000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74bc09a190 0x7f74bc155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:41.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.104+0000 7f74c4000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74bc159e30 0x7f74bc15a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:41.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.104+0000 7f74c4000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc158e90 con 0x7f74bc09a190 2024-04-04T06:06:41.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.104+0000 7f74c4000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc15f7c0 con 0x7f74a4003f60 2024-04-04T06:06:41.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.104+0000 7f74c4000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f74bc156010 con 0x7f74bc159e30 2024-04-04T06:06:41.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.104+0000 7f74c0c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74bc09a190 0x7f74bc155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:41.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.104+0000 7f74c0c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74bc09a190 0x7f74bc155ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36296/0 (socket says 172.21.15.159:36296) 2024-04-04T06:06:41.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.104+0000 7f74c0c00640 1 -- 172.21.15.159:0/1065856450 learned_addr learned my addr 172.21.15.159:0/1065856450 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:41.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.105+0000 7f74c0c00640 1 -- 172.21.15.159:0/1065856450 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74bc159e30 msgr2=0x7f74bc15a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:41.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.105+0000 7f74c2000640 1 --2- 172.21.15.159:0/1065856450 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74bc159e30 0x7f74bc15a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:41.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.105+0000 7f74c1600640 1 --2- 172.21.15.159:0/1065856450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74a4003f60 0x7f74bc15cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:41.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.105+0000 7f74c0c00640 1 --2- 172.21.15.159:0/1065856450 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74bc159e30 0x7f74bc15a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:41.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.105+0000 7f74c0c00640 1 -- 172.21.15.159:0/1065856450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74a4003f60 msgr2=0x7f74bc15cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:41.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.105+0000 7f74c0c00640 1 --2- 172.21.15.159:0/1065856450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74a4003f60 0x7f74bc15cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:41.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.105+0000 7f74c0c00640 1 -- 172.21.15.159:0/1065856450 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc09a950 con 0x7f74bc09a190 2024-04-04T06:06:41.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.105+0000 7f74c0c00640 1 --2- 172.21.15.159:0/1065856450 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74bc09a190 0x7f74bc155ad0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f74a8008fd0 tx=0x7f74a8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:41.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.105+0000 7f74b9600640 1 -- 172.21.15.159:0/1065856450 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a8041070 con 0x7f74bc09a190 2024-04-04T06:06:41.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.105+0000 7f74b9600640 1 -- 172.21.15.159:0/1065856450 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74a8005480 con 0x7f74bc09a190 2024-04-04T06:06:41.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.105+0000 7f74b9600640 1 -- 172.21.15.159:0/1065856450 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a8007a30 con 0x7f74bc09a190 2024-04-04T06:06:41.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.105+0000 7f74c4000640 1 -- 172.21.15.159:0/1065856450 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74bc09ac30 con 0x7f74bc09a190 2024-04-04T06:06:41.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.106+0000 7f74c4000640 1 -- 172.21.15.159:0/1065856450 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74bc09b1f0 con 0x7f74bc09a190 2024-04-04T06:06:41.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.107+0000 7f74c4000640 1 -- 172.21.15.159:0/1065856450 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74bc0685c0 con 0x7f74bc09a190 2024-04-04T06:06:41.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.107+0000 7f74b9600640 1 -- 172.21.15.159:0/1065856450 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74a8044070 con 0x7f74bc09a190 2024-04-04T06:06:41.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.109+0000 7f74b9600640 1 --2- 172.21.15.159:0/1065856450 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f749c06c920 0x7f749c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:41.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.109+0000 7f74b9600640 1 -- 172.21.15.159:0/1065856450 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f74a80ad890 con 0x7f74bc09a190 2024-04-04T06:06:41.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.109+0000 7f74c1600640 1 --2- 172.21.15.159:0/1065856450 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f749c06c920 0x7f749c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:41.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.110+0000 7f74c1600640 1 --2- 172.21.15.159:0/1065856450 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f749c06c920 0x7f749c06ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009970 tx=0x7f74b0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:41.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.114+0000 7f74b9600640 1 -- 172.21.15.159:0/1065856450 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f74a807bb70 con 0x7f74bc09a190 2024-04-04T06:06:41.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.303+0000 7f74c4000640 1 -- 172.21.15.159:0/1065856450 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f74bc06e370 con 0x7f74bc09a190 2024-04-04T06:06:41.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.309+0000 7f74b9600640 1 -- 172.21.15.159:0/1065856450 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f74a807b470 con 0x7f74bc09a190 2024-04-04T06:06:41.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.311+0000 7f74c4000640 1 -- 172.21.15.159:0/1065856450 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f749c06c920 msgr2=0x7f749c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:41.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.311+0000 7f74c4000640 1 --2- 172.21.15.159:0/1065856450 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f749c06c920 0x7f749c06ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009970 tx=0x7f74b0008040 comp rx=0 tx=0).stop 2024-04-04T06:06:41.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.312+0000 7f74c4000640 1 -- 172.21.15.159:0/1065856450 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74bc09a190 msgr2=0x7f74bc155ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:41.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.312+0000 7f74c4000640 1 --2- 172.21.15.159:0/1065856450 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74bc09a190 0x7f74bc155ad0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f74a8008fd0 tx=0x7f74a8001120 comp rx=0 tx=0).stop 2024-04-04T06:06:41.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.312+0000 7f74c4000640 1 -- 172.21.15.159:0/1065856450 shutdown_connections 2024-04-04T06:06:41.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.312+0000 7f74c4000640 1 --2- 172.21.15.159:0/1065856450 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f74bc159e30 0x7f74bc15a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:41.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.312+0000 7f74c4000640 1 --2- 172.21.15.159:0/1065856450 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74bc09a190 0x7f74bc155ad0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:41.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.312+0000 7f74c4000640 1 --2- 172.21.15.159:0/1065856450 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f749c06c920 0x7f749c06ede0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:41.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.312+0000 7f74c4000640 1 --2- 172.21.15.159:0/1065856450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74a4003f60 0x7f74bc15cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:41.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.312+0000 7f74c4000640 1 -- 172.21.15.159:0/1065856450 >> 172.21.15.159:0/1065856450 conn(0x7f74bc06b750 msgr2=0x7f74bc0b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:41.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.312+0000 7f74c4000640 1 -- 172.21.15.159:0/1065856450 shutdown_connections 2024-04-04T06:06:41.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.312+0000 7f74c4000640 1 -- 172.21.15.159:0/1065856450 wait complete. 2024-04-04T06:06:41.326 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-04T06:06:41.326 DEBUG:teuthology.orchestra.run.smithi159:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:06:41.436 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:06:41.552 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-04T06:06:41.552 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:06:41.553 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:06:41.608 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:06:41.609 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:06:41.662 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:06:41.663 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:06:41.711 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:06:41.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.976+0000 7f0fb4000640 1 Processor -- start 2024-04-04T06:06:41.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.976+0000 7f0fb4000640 1 -- start start 2024-04-04T06:06:41.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.977+0000 7f0fb4000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0fac0af830 0x7f0fac073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:41.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.977+0000 7f0fb4000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fac0d0650 0x7f0fac0d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:41.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.977+0000 7f0fb4000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fac15fe80 con 0x7f0fac15c390 2024-04-04T06:06:41.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.977+0000 7f0fb4000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f0fac0d1000 con 0x7f0fac0af830 2024-04-04T06:06:41.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.977+0000 7f0fb4000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0fac15c770 con 0x7f0fac0d0650 2024-04-04T06:06:41.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.978+0000 7f0fb0c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0fac0af830 0x7f0fac073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:41.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.978+0000 7f0fb0c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0fac0af830 0x7f0fac073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36298/0 (socket says 172.21.15.159:36298) 2024-04-04T06:06:41.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.978+0000 7f0fb0c00640 1 -- 172.21.15.159:0/2107669705 learned_addr learned my addr 172.21.15.159:0/2107669705 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:41.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.978+0000 7f0fb1600640 1 --1- 172.21.15.159:0/2107669705 >> v1:172.21.15.159:6789/0 conn(0x7f0fac15c390 0x7f0fac160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37194/0 (socket says 172.21.15.159:37194) 2024-04-04T06:06:41.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.978+0000 7f0fb2000640 1 --2- 172.21.15.159:0/2107669705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fac0d0650 0x7f0fac0d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:41.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.978+0000 7f0fb0c00640 1 -- 172.21.15.159:0/2107669705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fac0d0650 msgr2=0x7f0fac0d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:41.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.978+0000 7f0fb0c00640 1 --2- 172.21.15.159:0/2107669705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fac0d0650 0x7f0fac0d0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:41.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.978+0000 7f0fb0c00640 1 -- 172.21.15.159:0/2107669705 >> v1:172.21.15.159:6789/0 conn(0x7f0fac15c390 legacy=0x7f0fac160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:41.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.978+0000 7f0fb0c00640 1 -- 172.21.15.159:0/2107669705 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fac0ad120 con 0x7f0fac0af830 2024-04-04T06:06:41.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.978+0000 7f0fb2000640 1 --2- 172.21.15.159:0/2107669705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fac0d0650 0x7f0fac0d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:06:41.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.979+0000 7f0fb0c00640 1 --2- 172.21.15.159:0/2107669705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fac0af830 0x7f0fac073790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009870 tx=0x7f0f9802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a814e3d546c804e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:41.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.980+0000 7f0fabe00640 1 -- 172.21.15.159:0/2107669705 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98034070 con 0x7f0fac0af830 2024-04-04T06:06:41.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.980+0000 7f0fabe00640 1 -- 172.21.15.159:0/2107669705 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f98030840 con 0x7f0fac0af830 2024-04-04T06:06:41.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.980+0000 7f0fabe00640 1 -- 172.21.15.159:0/2107669705 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98030c10 con 0x7f0fac0af830 2024-04-04T06:06:41.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.980+0000 7f0fb4000640 1 -- 172.21.15.159:0/2107669705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fac0af830 msgr2=0x7f0fac073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:41.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.980+0000 7f0fb4000640 1 --2- 172.21.15.159:0/2107669705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fac0af830 0x7f0fac073790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009870 tx=0x7f0f9802b980 comp rx=0 tx=0).stop 2024-04-04T06:06:41.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.980+0000 7f0fb4000640 1 -- 172.21.15.159:0/2107669705 shutdown_connections 2024-04-04T06:06:41.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.980+0000 7f0fb4000640 1 --2- 172.21.15.159:0/2107669705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fac0d0650 0x7f0fac0d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:41.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.980+0000 7f0fb4000640 1 --2- 172.21.15.159:0/2107669705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fac0af830 0x7f0fac073790 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:41.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.980+0000 7f0fb4000640 1 -- 172.21.15.159:0/2107669705 >> 172.21.15.159:0/2107669705 conn(0x7f0fac06b990 msgr2=0x7f0fac06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:41.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.980+0000 7f0fb4000640 1 -- 172.21.15.159:0/2107669705 shutdown_connections 2024-04-04T06:06:41.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.980+0000 7f0fb4000640 1 -- 172.21.15.159:0/2107669705 wait complete. 2024-04-04T06:06:41.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.982+0000 7f0fb4000640 1 Processor -- start 2024-04-04T06:06:41.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.982+0000 7f0fb4000640 1 -- start start 2024-04-04T06:06:41.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.983+0000 7f0fb4000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fac0af830 0x7f0fac15f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:41.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.983+0000 7f0fb4000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fac0d0650 0x7f0fac0b0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:41.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.983+0000 7f0fb4000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fac15c390 0x7f0fac099900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:41.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.983+0000 7f0fb4000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac160120 con 0x7f0fac15c390 2024-04-04T06:06:41.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.983+0000 7f0fb4000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac0ad6e0 con 0x7f0fac0af830 2024-04-04T06:06:41.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.983+0000 7f0fb4000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0fac160d50 con 0x7f0fac0d0650 2024-04-04T06:06:41.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.983+0000 7f0fb2000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fac15c390 0x7f0fac099900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:41.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.983+0000 7f0fb2000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fac15c390 0x7f0fac099900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36304/0 (socket says 172.21.15.159:36304) 2024-04-04T06:06:41.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.983+0000 7f0fb2000640 1 -- 172.21.15.159:0/511097633 learned_addr learned my addr 172.21.15.159:0/511097633 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:41.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.984+0000 7f0fb1600640 1 --2- 172.21.15.159:0/511097633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fac0af830 0x7f0fac15f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:41.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.984+0000 7f0fb0c00640 1 --2- 172.21.15.159:0/511097633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fac0d0650 0x7f0fac0b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:41.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.984+0000 7f0fb2000640 1 -- 172.21.15.159:0/511097633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fac0d0650 msgr2=0x7f0fac0b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:41.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.984+0000 7f0fb2000640 1 --2- 172.21.15.159:0/511097633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fac0d0650 0x7f0fac0b0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:41.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.984+0000 7f0fb2000640 1 -- 172.21.15.159:0/511097633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fac0af830 msgr2=0x7f0fac15f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:41.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.984+0000 7f0fb2000640 1 --2- 172.21.15.159:0/511097633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fac0af830 0x7f0fac15f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:41.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.984+0000 7f0fb2000640 1 -- 172.21.15.159:0/511097633 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fac15c9a0 con 0x7f0fac15c390 2024-04-04T06:06:41.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.984+0000 7f0fb0c00640 1 --2- 172.21.15.159:0/511097633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fac0d0650 0x7f0fac0b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:41.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.984+0000 7f0fb2000640 1 --2- 172.21.15.159:0/511097633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fac15c390 0x7f0fac099900 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009fa0 tx=0x7f0fa400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:41.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.984+0000 7f0fa9600640 1 -- 172.21.15.159:0/511097633 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa401a070 con 0x7f0fac15c390 2024-04-04T06:06:41.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.984+0000 7f0fa9600640 1 -- 172.21.15.159:0/511097633 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa4003bb0 con 0x7f0fac15c390 2024-04-04T06:06:41.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.984+0000 7f0fb4000640 1 -- 172.21.15.159:0/511097633 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fac0b0cd0 con 0x7f0fac15c390 2024-04-04T06:06:41.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.984+0000 7f0fa9600640 1 -- 172.21.15.159:0/511097633 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4004030 con 0x7f0fac15c390 2024-04-04T06:06:41.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.985+0000 7f0fb4000640 1 -- 172.21.15.159:0/511097633 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fac0b1250 con 0x7f0fac15c390 2024-04-04T06:06:41.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.986+0000 7f0fb4000640 1 -- 172.21.15.159:0/511097633 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fac0685c0 con 0x7f0fac15c390 2024-04-04T06:06:41.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.987+0000 7f0fa9600640 1 -- 172.21.15.159:0/511097633 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0fa401d070 con 0x7f0fac15c390 2024-04-04T06:06:41.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.988+0000 7f0fa9600640 1 --2- 172.21.15.159:0/511097633 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f8c06c8f0 0x7f0f8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:41.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.988+0000 7f0fa9600640 1 -- 172.21.15.159:0/511097633 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f0fa408bf30 con 0x7f0fac15c390 2024-04-04T06:06:41.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.988+0000 7f0fb1600640 1 --2- 172.21.15.159:0/511097633 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f8c06c8f0 0x7f0f8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:41.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.989+0000 7f0fb1600640 1 --2- 172.21.15.159:0/511097633 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f8c06c8f0 0x7f0f8c06edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0000f60 tx=0x7f0fa0007cc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:41.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:41.993+0000 7f0fa9600640 1 -- 172.21.15.159:0/511097633 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0fa4010070 con 0x7f0fac15c390 2024-04-04T06:06:42.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.191+0000 7f0fb4000640 1 -- 172.21.15.159:0/511097633 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0fac0cf760 con 0x7f0fac15c390 2024-04-04T06:06:42.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.192+0000 7f0fa9600640 1 -- 172.21.15.159:0/511097633 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16958 (secure 0 0 0) 0x7f0fa405a290 con 0x7f0fac15c390 2024-04-04T06:06:42.187 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:42.187 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":23,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:06:33.796979+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:05:51.297293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-04T06:05:51.770792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":0,"to":2}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:06:42.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.195+0000 7f0fb4000640 1 -- 172.21.15.159:0/511097633 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f8c06c8f0 msgr2=0x7f0f8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:42.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.195+0000 7f0fb4000640 1 --2- 172.21.15.159:0/511097633 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f8c06c8f0 0x7f0f8c06edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0000f60 tx=0x7f0fa0007cc0 comp rx=0 tx=0).stop 2024-04-04T06:06:42.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.195+0000 7f0fb4000640 1 -- 172.21.15.159:0/511097633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fac15c390 msgr2=0x7f0fac099900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:42.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.195+0000 7f0fb4000640 1 --2- 172.21.15.159:0/511097633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fac15c390 0x7f0fac099900 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009fa0 tx=0x7f0fa400d560 comp rx=0 tx=0).stop 2024-04-04T06:06:42.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.195+0000 7f0fb4000640 1 -- 172.21.15.159:0/511097633 shutdown_connections 2024-04-04T06:06:42.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.195+0000 7f0fb4000640 1 --2- 172.21.15.159:0/511097633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fac15c390 0x7f0fac099900 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:42.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.195+0000 7f0fb4000640 1 --2- 172.21.15.159:0/511097633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fac0d0650 0x7f0fac0b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:42.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.195+0000 7f0fb4000640 1 --2- 172.21.15.159:0/511097633 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f8c06c8f0 0x7f0f8c06edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:42.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.195+0000 7f0fb4000640 1 --2- 172.21.15.159:0/511097633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fac0af830 0x7f0fac15f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:42.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.196+0000 7f0fb4000640 1 -- 172.21.15.159:0/511097633 >> 172.21.15.159:0/511097633 conn(0x7f0fac06b990 msgr2=0x7f0fac06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:42.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.196+0000 7f0fb4000640 1 -- 172.21.15.159:0/511097633 shutdown_connections 2024-04-04T06:06:42.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.196+0000 7f0fb4000640 1 -- 172.21.15.159:0/511097633 wait complete. 2024-04-04T06:06:42.211 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:05:51.297293+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:05:51.770792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:06:42.211 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:06:42.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.427+0000 7f10d9e00640 1 Processor -- start 2024-04-04T06:06:42.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.428+0000 7f10d9e00640 1 -- start start 2024-04-04T06:06:42.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.429+0000 7f10d9e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10d40afd80 0x7f10d415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:42.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.429+0000 7f10d9e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f10d406cc90 0x7f10d40ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:42.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.429+0000 7f10d9e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10d40b4710 con 0x7f10d40ccc40 2024-04-04T06:06:42.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.429+0000 7f10d9e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f10d40b5740 con 0x7f10d406cc90 2024-04-04T06:06:42.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.429+0000 7f10d9e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f10d415da00 con 0x7f10d40afd80 2024-04-04T06:06:42.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.430+0000 7f10d3400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f10d406cc90 0x7f10d40ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:42.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.430+0000 7f10d2a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10d40afd80 0x7f10d415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:42.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.430+0000 7f10d3400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f10d406cc90 0x7f10d40ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:49918/0 (socket says 172.21.15.159:49918) 2024-04-04T06:06:42.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.430+0000 7f10d3400640 1 -- 172.21.15.159:0/2890726629 learned_addr learned my addr 172.21.15.159:0/2890726629 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:42.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.430+0000 7f10d2000640 1 --1- 172.21.15.159:0/2890726629 >> v1:172.21.15.178:6789/0 conn(0x7f10d40ccc40 0x7f10d406c580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.159:39044/0 (socket says 172.21.15.159:39044) 2024-04-04T06:06:42.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.431+0000 7f10d3400640 1 -- 172.21.15.159:0/2890726629 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10d40afd80 msgr2=0x7f10d415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:42.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.431+0000 7f10d3400640 1 --2- 172.21.15.159:0/2890726629 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10d40afd80 0x7f10d415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:42.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.431+0000 7f10d3400640 1 -- 172.21.15.159:0/2890726629 >> v1:172.21.15.178:6789/0 conn(0x7f10d40ccc40 legacy=0x7f10d406c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:42.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.431+0000 7f10d3400640 1 -- 172.21.15.159:0/2890726629 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d40b4980 con 0x7f10d406cc90 2024-04-04T06:06:42.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.432+0000 7f10d3400640 1 --2- 172.21.15.159:0/2890726629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10d406cc90 0x7f10d40ce8a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f10cc009870 tx=0x7f10cc02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2418b639148f94c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:42.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.432+0000 7f10d1600640 1 -- 172.21.15.159:0/2890726629 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc034070 con 0x7f10d406cc90 2024-04-04T06:06:42.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.432+0000 7f10d1600640 1 -- 172.21.15.159:0/2890726629 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10cc030840 con 0x7f10d406cc90 2024-04-04T06:06:42.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.432+0000 7f10d1600640 1 -- 172.21.15.159:0/2890726629 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc030c10 con 0x7f10d406cc90 2024-04-04T06:06:42.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.433+0000 7f10d9e00640 1 -- 172.21.15.159:0/2890726629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10d406cc90 msgr2=0x7f10d40ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:42.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.433+0000 7f10d9e00640 1 --2- 172.21.15.159:0/2890726629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10d406cc90 0x7f10d40ce8a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f10cc009870 tx=0x7f10cc02b980 comp rx=0 tx=0).stop 2024-04-04T06:06:42.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.433+0000 7f10d9e00640 1 -- 172.21.15.159:0/2890726629 shutdown_connections 2024-04-04T06:06:42.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.433+0000 7f10d9e00640 1 --2- 172.21.15.159:0/2890726629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10d406cc90 0x7f10d40ce8a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:42.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.433+0000 7f10d9e00640 1 --2- 172.21.15.159:0/2890726629 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10d40afd80 0x7f10d415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:42.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.433+0000 7f10d9e00640 1 -- 172.21.15.159:0/2890726629 >> 172.21.15.159:0/2890726629 conn(0x7f10d4071e20 msgr2=0x7f10d4072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:42.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.433+0000 7f10d9e00640 1 -- 172.21.15.159:0/2890726629 shutdown_connections 2024-04-04T06:06:42.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.433+0000 7f10d9e00640 1 -- 172.21.15.159:0/2890726629 wait complete. 2024-04-04T06:06:42.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.435+0000 7f10d9e00640 1 Processor -- start 2024-04-04T06:06:42.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.436+0000 7f10d9e00640 1 -- start start 2024-04-04T06:06:42.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.437+0000 7f10d9e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10d406cc90 0x7f10d415e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:42.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.437+0000 7f10d9e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10d40afd80 0x7f10d40aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:42.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.437+0000 7f10d9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10d40ccc40 0x7f10d414bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:42.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.437+0000 7f10d9e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f10d40b5220 con 0x7f10d40ccc40 2024-04-04T06:06:42.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.437+0000 7f10d9e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f10d40b4f40 con 0x7f10d40afd80 2024-04-04T06:06:42.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.437+0000 7f10d9e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f10d40b0160 con 0x7f10d406cc90 2024-04-04T06:06:42.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.437+0000 7f10d3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10d40ccc40 0x7f10d414bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:42.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.437+0000 7f10d3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10d40ccc40 0x7f10d414bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36316/0 (socket says 172.21.15.159:36316) 2024-04-04T06:06:42.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.437+0000 7f10d3400640 1 -- 172.21.15.159:0/3257200026 learned_addr learned my addr 172.21.15.159:0/3257200026 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:42.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.437+0000 7f10d2000640 1 --2- 172.21.15.159:0/3257200026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10d40afd80 0x7f10d40aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:42.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.437+0000 7f10d2a00640 1 --2- 172.21.15.159:0/3257200026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10d406cc90 0x7f10d415e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:42.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.438+0000 7f10d3400640 1 -- 172.21.15.159:0/3257200026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10d406cc90 msgr2=0x7f10d415e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:42.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.438+0000 7f10d3400640 1 --2- 172.21.15.159:0/3257200026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10d406cc90 0x7f10d415e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:42.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.438+0000 7f10d3400640 1 -- 172.21.15.159:0/3257200026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10d40afd80 msgr2=0x7f10d40aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:42.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.438+0000 7f10d3400640 1 --2- 172.21.15.159:0/3257200026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10d40afd80 0x7f10d40aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:42.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.438+0000 7f10d3400640 1 -- 172.21.15.159:0/3257200026 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d4158b80 con 0x7f10d40ccc40 2024-04-04T06:06:42.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.438+0000 7f10d3400640 1 --2- 172.21.15.159:0/3257200026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10d40ccc40 0x7f10d414bcf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f10cc009870 tx=0x7f10cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:42.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.438+0000 7f10cb400640 1 -- 172.21.15.159:0/3257200026 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc034040 con 0x7f10d40ccc40 2024-04-04T06:06:42.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.438+0000 7f10cb400640 1 -- 172.21.15.159:0/3257200026 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10cc039d90 con 0x7f10d40ccc40 2024-04-04T06:06:42.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.438+0000 7f10cb400640 1 -- 172.21.15.159:0/3257200026 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc043610 con 0x7f10d40ccc40 2024-04-04T06:06:42.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.439+0000 7f10d9e00640 1 -- 172.21.15.159:0/3257200026 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d4158e60 con 0x7f10d40ccc40 2024-04-04T06:06:42.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.439+0000 7f10d9e00640 1 -- 172.21.15.159:0/3257200026 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d4159420 con 0x7f10d40ccc40 2024-04-04T06:06:42.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.440+0000 7f10d9e00640 1 -- 172.21.15.159:0/3257200026 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d4068cd0 con 0x7f10d40ccc40 2024-04-04T06:06:42.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.441+0000 7f10cb400640 1 -- 172.21.15.159:0/3257200026 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10cc04a060 con 0x7f10d40ccc40 2024-04-04T06:06:42.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.442+0000 7f10cb400640 1 --2- 172.21.15.159:0/3257200026 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f10b006c920 0x7f10b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:42.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.442+0000 7f10cb400640 1 -- 172.21.15.159:0/3257200026 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f10cc0ad930 con 0x7f10d40ccc40 2024-04-04T06:06:42.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.442+0000 7f10d2a00640 1 --2- 172.21.15.159:0/3257200026 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f10b006c920 0x7f10b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:42.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.443+0000 7f10d2a00640 1 --2- 172.21.15.159:0/3257200026 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f10b006c920 0x7f10b006ede0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f10c4009b40 tx=0x7f10c4009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:42.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.446+0000 7f10cb400640 1 -- 172.21.15.159:0/3257200026 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f10cc03e070 con 0x7f10d40ccc40 2024-04-04T06:06:42.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.649+0000 7f10d9e00640 1 -- 172.21.15.159:0/3257200026 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f10d406d0b0 con 0x7f10d40ccc40 2024-04-04T06:06:42.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.650+0000 7f10cb400640 1 -- 172.21.15.159:0/3257200026 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f10cc07b6f0 con 0x7f10d40ccc40 2024-04-04T06:06:42.645 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:06:42.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.652+0000 7f10d9e00640 1 -- 172.21.15.159:0/3257200026 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f10b006c920 msgr2=0x7f10b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:42.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.653+0000 7f10d9e00640 1 --2- 172.21.15.159:0/3257200026 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f10b006c920 0x7f10b006ede0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f10c4009b40 tx=0x7f10c4009190 comp rx=0 tx=0).stop 2024-04-04T06:06:42.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.653+0000 7f10d9e00640 1 -- 172.21.15.159:0/3257200026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10d40ccc40 msgr2=0x7f10d414bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:42.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.653+0000 7f10d9e00640 1 --2- 172.21.15.159:0/3257200026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10d40ccc40 0x7f10d414bcf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f10cc009870 tx=0x7f10cc0046c0 comp rx=0 tx=0).stop 2024-04-04T06:06:42.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.653+0000 7f10d9e00640 1 -- 172.21.15.159:0/3257200026 shutdown_connections 2024-04-04T06:06:42.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.653+0000 7f10d9e00640 1 --2- 172.21.15.159:0/3257200026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10d40ccc40 0x7f10d414bcf0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:42.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.653+0000 7f10d9e00640 1 --2- 172.21.15.159:0/3257200026 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f10b006c920 0x7f10b006ede0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:42.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.653+0000 7f10d9e00640 1 --2- 172.21.15.159:0/3257200026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10d40afd80 0x7f10d40aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:42.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.653+0000 7f10d9e00640 1 --2- 172.21.15.159:0/3257200026 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10d406cc90 0x7f10d415e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:42.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.653+0000 7f10d9e00640 1 -- 172.21.15.159:0/3257200026 >> 172.21.15.159:0/3257200026 conn(0x7f10d4071e20 msgr2=0x7f10d415dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:42.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.653+0000 7f10d9e00640 1 -- 172.21.15.159:0/3257200026 shutdown_connections 2024-04-04T06:06:42.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.653+0000 7f10d9e00640 1 -- 172.21.15.159:0/3257200026 wait complete. 2024-04-04T06:06:42.667 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:06:42.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.890+0000 7f0b7dc00640 1 Processor -- start 2024-04-04T06:06:42.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.890+0000 7f0b7dc00640 1 -- start start 2024-04-04T06:06:42.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.891+0000 7f0b7dc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0b7815eca0 0x7f0b780b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:42.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.891+0000 7f0b7dc00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b780b4c00 con 0x7f0b7815ac50 2024-04-04T06:06:42.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.891+0000 7f0b7dc00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b780b4ee0 con 0x7f0b7815a870 2024-04-04T06:06:42.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.891+0000 7f0b7dc00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f0b7806d5b0 con 0x7f0b7815eca0 2024-04-04T06:06:42.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.892+0000 7f0b76000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0b7815eca0 0x7f0b780b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:42.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.892+0000 7f0b76000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0b7815eca0 0x7f0b780b3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36320/0 (socket says 172.21.15.159:36320) 2024-04-04T06:06:42.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.892+0000 7f0b76000640 1 -- 172.21.15.159:0/3522575691 learned_addr learned my addr 172.21.15.159:0/3522575691 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:42.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.892+0000 7f0b76000640 1 -- 172.21.15.159:0/3522575691 >> v1:172.21.15.178:6789/0 conn(0x7f0b7815a870 legacy=0x7f0b7815e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:42.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.892+0000 7f0b76000640 1 -- 172.21.15.159:0/3522575691 >> v1:172.21.15.159:6789/0 conn(0x7f0b7815ac50 legacy=0x7f0b780b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:42.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.892+0000 7f0b76000640 1 -- 172.21.15.159:0/3522575691 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7806d800 con 0x7f0b7815eca0 2024-04-04T06:06:42.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.892+0000 7f0b75600640 1 -- 172.21.15.159:0/3522575691 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1068043429 0 0) 0x7f0b780b4c00 con 0x7f0b7815ac50 2024-04-04T06:06:42.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.893+0000 7f0b76000640 1 --2- 172.21.15.159:0/3522575691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b7815eca0 0x7f0b780b3f10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0b7806d730 tx=0x7f0b6002dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff206af1565336f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:42.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.894+0000 7f0b75600640 1 -- 172.21.15.159:0/3522575691 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6003b070 con 0x7f0b7815eca0 2024-04-04T06:06:42.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.894+0000 7f0b75600640 1 -- 172.21.15.159:0/3522575691 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b6002e9e0 con 0x7f0b7815eca0 2024-04-04T06:06:42.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.894+0000 7f0b75600640 1 -- 172.21.15.159:0/3522575691 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6002edb0 con 0x7f0b7815eca0 2024-04-04T06:06:42.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.894+0000 7f0b7dc00640 1 -- 172.21.15.159:0/3522575691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b7815eca0 msgr2=0x7f0b780b3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:42.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.894+0000 7f0b7dc00640 1 --2- 172.21.15.159:0/3522575691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b7815eca0 0x7f0b780b3f10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0b7806d730 tx=0x7f0b6002dc30 comp rx=0 tx=0).stop 2024-04-04T06:06:42.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.894+0000 7f0b7dc00640 1 -- 172.21.15.159:0/3522575691 shutdown_connections 2024-04-04T06:06:42.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.894+0000 7f0b7dc00640 1 --2- 172.21.15.159:0/3522575691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b7815eca0 0x7f0b780b3f10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:42.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.894+0000 7f0b7dc00640 1 -- 172.21.15.159:0/3522575691 >> 172.21.15.159:0/3522575691 conn(0x7f0b7806b750 msgr2=0x7f0b7806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:42.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.895+0000 7f0b7dc00640 1 -- 172.21.15.159:0/3522575691 shutdown_connections 2024-04-04T06:06:42.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.895+0000 7f0b7dc00640 1 -- 172.21.15.159:0/3522575691 wait complete. 2024-04-04T06:06:42.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.896+0000 7f0b7dc00640 1 Processor -- start 2024-04-04T06:06:42.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.896+0000 7f0b7dc00640 1 -- start start 2024-04-04T06:06:42.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.897+0000 7f0b7dc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b7815a870 0x7f0b7815da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:42.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.897+0000 7f0b7dc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b7815ac50 0x7f0b781f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:42.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.897+0000 7f0b7dc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b7815eca0 0x7f0b7815b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:42.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.897+0000 7f0b7dc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7806db70 con 0x7f0b7815ac50 2024-04-04T06:06:42.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.897+0000 7f0b7dc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7806f2a0 con 0x7f0b7815a870 2024-04-04T06:06:42.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.897+0000 7f0b7dc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0b780d0280 con 0x7f0b7815eca0 2024-04-04T06:06:42.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.898+0000 7f0b76000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b7815ac50 0x7f0b781f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:42.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.898+0000 7f0b76000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b7815ac50 0x7f0b781f4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36334/0 (socket says 172.21.15.159:36334) 2024-04-04T06:06:42.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.898+0000 7f0b76000640 1 -- 172.21.15.159:0/1380377293 learned_addr learned my addr 172.21.15.159:0/1380377293 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:42.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.898+0000 7f0b77400640 1 --2- 172.21.15.159:0/1380377293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b7815eca0 0x7f0b7815b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:42.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.898+0000 7f0b76a00640 1 --2- 172.21.15.159:0/1380377293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b7815a870 0x7f0b7815da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:42.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.898+0000 7f0b76000640 1 -- 172.21.15.159:0/1380377293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b7815eca0 msgr2=0x7f0b7815b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:42.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.898+0000 7f0b76000640 1 --2- 172.21.15.159:0/1380377293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b7815eca0 0x7f0b7815b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:42.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.898+0000 7f0b76000640 1 -- 172.21.15.159:0/1380377293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b7815a870 msgr2=0x7f0b7815da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:42.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.898+0000 7f0b76000640 1 --2- 172.21.15.159:0/1380377293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b7815a870 0x7f0b7815da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:42.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.898+0000 7f0b76000640 1 -- 172.21.15.159:0/1380377293 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7815dff0 con 0x7f0b7815ac50 2024-04-04T06:06:42.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.898+0000 7f0b76000640 1 --2- 172.21.15.159:0/1380377293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b7815ac50 0x7f0b781f4370 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0b6002dc70 tx=0x7f0b600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:42.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.899+0000 7f0b6ea00640 1 -- 172.21.15.159:0/1380377293 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6003b070 con 0x7f0b7815ac50 2024-04-04T06:06:42.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.899+0000 7f0b6ea00640 1 -- 172.21.15.159:0/1380377293 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b60041440 con 0x7f0b7815ac50 2024-04-04T06:06:42.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.899+0000 7f0b6ea00640 1 -- 172.21.15.159:0/1380377293 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b60041830 con 0x7f0b7815ac50 2024-04-04T06:06:42.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.899+0000 7f0b7dc00640 1 -- 172.21.15.159:0/1380377293 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b780cbd90 con 0x7f0b7815ac50 2024-04-04T06:06:42.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.899+0000 7f0b7dc00640 1 -- 172.21.15.159:0/1380377293 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b780c9670 con 0x7f0b7815ac50 2024-04-04T06:06:42.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.900+0000 7f0b7dc00640 1 -- 172.21.15.159:0/1380377293 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b780ce480 con 0x7f0b7815ac50 2024-04-04T06:06:42.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.901+0000 7f0b6ea00640 1 -- 172.21.15.159:0/1380377293 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b600419d0 con 0x7f0b7815ac50 2024-04-04T06:06:42.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.902+0000 7f0b6ea00640 1 --2- 172.21.15.159:0/1380377293 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0b5406c8f0 0x7f0b5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:42.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.902+0000 7f0b6ea00640 1 -- 172.21.15.159:0/1380377293 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f0b600ae320 con 0x7f0b7815ac50 2024-04-04T06:06:42.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.902+0000 7f0b76a00640 1 --2- 172.21.15.159:0/1380377293 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0b5406c8f0 0x7f0b5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:42.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.903+0000 7f0b76a00640 1 --2- 172.21.15.159:0/1380377293 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0b5406c8f0 0x7f0b5406edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0b68000f60 tx=0x7f0b68009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:42.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:42.907+0000 7f0b6ea00640 1 -- 172.21.15.159:0/1380377293 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0b6007c720 con 0x7f0b7815ac50 2024-04-04T06:06:43.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.104+0000 7f0b7dc00640 1 -- 172.21.15.159:0/1380377293 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0b780685c0 con 0x7f0b7815ac50 2024-04-04T06:06:43.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.105+0000 7f0b6ea00640 1 -- 172.21.15.159:0/1380377293 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f0b6002e9b0 con 0x7f0b7815ac50 2024-04-04T06:06:43.100 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:06:43.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.108+0000 7f0b7dc00640 1 -- 172.21.15.159:0/1380377293 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0b5406c8f0 msgr2=0x7f0b5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:43.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.108+0000 7f0b7dc00640 1 --2- 172.21.15.159:0/1380377293 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0b5406c8f0 0x7f0b5406edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0b68000f60 tx=0x7f0b68009040 comp rx=0 tx=0).stop 2024-04-04T06:06:43.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.108+0000 7f0b7dc00640 1 -- 172.21.15.159:0/1380377293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b7815ac50 msgr2=0x7f0b781f4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:43.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.108+0000 7f0b7dc00640 1 --2- 172.21.15.159:0/1380377293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b7815ac50 0x7f0b781f4370 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0b6002dc70 tx=0x7f0b600046c0 comp rx=0 tx=0).stop 2024-04-04T06:06:43.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.108+0000 7f0b7dc00640 1 -- 172.21.15.159:0/1380377293 shutdown_connections 2024-04-04T06:06:43.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.108+0000 7f0b7dc00640 1 --2- 172.21.15.159:0/1380377293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b7815eca0 0x7f0b7815b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:43.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.108+0000 7f0b7dc00640 1 --2- 172.21.15.159:0/1380377293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b7815ac50 0x7f0b781f4370 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:43.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.108+0000 7f0b7dc00640 1 --2- 172.21.15.159:0/1380377293 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0b5406c8f0 0x7f0b5406edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:43.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.108+0000 7f0b7dc00640 1 --2- 172.21.15.159:0/1380377293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b7815a870 0x7f0b7815da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:43.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.108+0000 7f0b7dc00640 1 -- 172.21.15.159:0/1380377293 >> 172.21.15.159:0/1380377293 conn(0x7f0b7806b750 msgr2=0x7f0b780d0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:43.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.108+0000 7f0b7dc00640 1 -- 172.21.15.159:0/1380377293 shutdown_connections 2024-04-04T06:06:43.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.109+0000 7f0b7dc00640 1 -- 172.21.15.159:0/1380377293 wait complete. 2024-04-04T06:06:43.122 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:06:43.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.343+0000 7fa712a00640 1 Processor -- start 2024-04-04T06:06:43.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.344+0000 7fa712a00640 1 -- start start 2024-04-04T06:06:43.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.345+0000 7fa712a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa70c1670b0 0x7fa70c163ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:43.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.345+0000 7fa712a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa70c071d60 con 0x7fa70c162920 2024-04-04T06:06:43.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.345+0000 7fa712a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa70c1665d0 con 0x7fa70c0d0990 2024-04-04T06:06:43.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.345+0000 7fa712a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa70c1674d0 con 0x7fa70c1670b0 2024-04-04T06:06:43.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.345+0000 7fa70be00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fa70c162920 0x7fa70c0d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37204/0 (socket says 172.21.15.159:37204) 2024-04-04T06:06:43.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.345+0000 7fa70be00640 1 -- 172.21.15.159:0/1993356622 learned_addr learned my addr 172.21.15.159:0/1993356622 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:43.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.346+0000 7fa710a00640 1 --2- 172.21.15.159:0/1993356622 >> v2:172.21.15.178:3300/0 conn(0x7fa70c1670b0 0x7fa70c163ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:43.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.346+0000 7fa70b400640 1 -- 172.21.15.159:0/1993356622 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2161740380 0 0) 0x7fa70c071d60 con 0x7fa70c162920 2024-04-04T06:06:43.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.346+0000 7fa70b400640 1 -- 172.21.15.159:0/1993356622 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6f0003520 con 0x7fa70c162920 2024-04-04T06:06:43.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.346+0000 7fa70b400640 1 -- 172.21.15.159:0/1993356622 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2581649165 0 0) 0x7fa6f0003520 con 0x7fa70c162920 2024-04-04T06:06:43.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.346+0000 7fa70b400640 1 -- 172.21.15.159:0/1993356622 >> v2:172.21.15.178:3300/0 conn(0x7fa70c1670b0 msgr2=0x7fa70c163ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:43.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.346+0000 7fa70b400640 1 --2- 172.21.15.159:0/1993356622 >> v2:172.21.15.178:3300/0 conn(0x7fa70c1670b0 0x7fa70c163ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:43.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.346+0000 7fa70b400640 1 -- 172.21.15.159:0/1993356622 >> v1:172.21.15.178:6789/0 conn(0x7fa70c0d0990 legacy=0x7fa70c0ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:43.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.346+0000 7fa70b400640 1 -- 172.21.15.159:0/1993356622 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa70c1668b0 con 0x7fa70c162920 2024-04-04T06:06:43.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.346+0000 7fa70b400640 1 -- 172.21.15.159:0/1993356622 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fa6fc001670 con 0x7fa70c162920 2024-04-04T06:06:43.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.346+0000 7fa70b400640 1 -- 172.21.15.159:0/1993356622 >> v1:172.21.15.159:6789/0 conn(0x7fa70c162920 legacy=0x7fa70c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:43.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.346+0000 7fa70b400640 1 --2- 172.21.15.159:0/1993356622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6f00039b0 0x7fa6f0003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:43.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.347+0000 7fa70b400640 1 --2- 172.21.15.159:0/1993356622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6f0004800 0x7fa6f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:43.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.347+0000 7fa70b400640 1 --2- 172.21.15.159:0/1993356622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6f0005210 0x7fa6f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:43.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.347+0000 7fa70b400640 1 -- 172.21.15.159:0/1993356622 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c1674d0 con 0x7fa6f00039b0 2024-04-04T06:06:43.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.347+0000 7fa70b400640 1 -- 172.21.15.159:0/1993356622 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f000c140 con 0x7fa6f0004800 2024-04-04T06:06:43.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.347+0000 7fa70b400640 1 -- 172.21.15.159:0/1993356622 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f000c340 con 0x7fa6f0005210 2024-04-04T06:06:43.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.347+0000 7fa70be00640 1 --2- 172.21.15.159:0/1993356622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6f00039b0 0x7fa6f0003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:43.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.347+0000 7fa703400640 1 --2- 172.21.15.159:0/1993356622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6f0004800 0x7fa6f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:43.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.347+0000 7fa710a00640 1 --2- 172.21.15.159:0/1993356622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6f0005210 0x7fa6f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:43.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.347+0000 7fa70be00640 1 -- 172.21.15.159:0/1993356622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6f0005210 msgr2=0x7fa6f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:43.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.347+0000 7fa70be00640 1 --2- 172.21.15.159:0/1993356622 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6f0005210 0x7fa6f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:43.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.347+0000 7fa70be00640 1 -- 172.21.15.159:0/1993356622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6f0004800 msgr2=0x7fa6f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:43.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.347+0000 7fa70be00640 1 --2- 172.21.15.159:0/1993356622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6f0004800 0x7fa6f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:43.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.347+0000 7fa70be00640 1 -- 172.21.15.159:0/1993356622 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f0003520 con 0x7fa6f00039b0 2024-04-04T06:06:43.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.347+0000 7fa710a00640 1 -- 172.21.15.159:0/1993356622 reap_dead start 2024-04-04T06:06:43.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.348+0000 7fa70be00640 1 --2- 172.21.15.159:0/1993356622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6f00039b0 0x7fa6f0003df0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc00c240 tx=0x7fa6f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c6eccdecaf4b84 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:43.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.349+0000 7fa70b400640 1 -- 172.21.15.159:0/1993356622 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc007cf0 con 0x7fa6f00039b0 2024-04-04T06:06:43.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.349+0000 7fa70b400640 1 -- 172.21.15.159:0/1993356622 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6fc030070 con 0x7fa6f00039b0 2024-04-04T06:06:43.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.349+0000 7fa70b400640 1 -- 172.21.15.159:0/1993356622 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc005d70 con 0x7fa6f00039b0 2024-04-04T06:06:43.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.349+0000 7fa712a00640 1 -- 172.21.15.159:0/1993356622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6f00039b0 msgr2=0x7fa6f0003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:43.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.349+0000 7fa712a00640 1 --2- 172.21.15.159:0/1993356622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6f00039b0 0x7fa6f0003df0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc00c240 tx=0x7fa6f0005830 comp rx=0 tx=0).stop 2024-04-04T06:06:43.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.349+0000 7fa712a00640 1 -- 172.21.15.159:0/1993356622 shutdown_connections 2024-04-04T06:06:43.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.349+0000 7fa712a00640 1 --2- 172.21.15.159:0/1993356622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6f00039b0 0x7fa6f0003df0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:43.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.349+0000 7fa712a00640 1 -- 172.21.15.159:0/1993356622 >> 172.21.15.159:0/1993356622 conn(0x7fa70c06b750 msgr2=0x7fa70c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:43.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.349+0000 7fa712a00640 1 -- 172.21.15.159:0/1993356622 shutdown_connections 2024-04-04T06:06:43.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.349+0000 7fa712a00640 1 -- 172.21.15.159:0/1993356622 wait complete. 2024-04-04T06:06:43.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.351+0000 7fa712a00640 1 Processor -- start 2024-04-04T06:06:43.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.351+0000 7fa712a00640 1 -- start start 2024-04-04T06:06:43.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.352+0000 7fa712a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6f00039b0 0x7fa70c15e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:43.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.352+0000 7fa712a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa70c15ed70 0x7fa70c0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:43.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.352+0000 7fa712a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa70c15f1d0 0x7fa70c0b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:43.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.352+0000 7fa712a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c072040 con 0x7fa70c15f1d0 2024-04-04T06:06:43.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.352+0000 7fa712a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c162d00 con 0x7fa6f00039b0 2024-04-04T06:06:43.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.352+0000 7fa712a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa70c0721c0 con 0x7fa70c15ed70 2024-04-04T06:06:43.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.353+0000 7fa710a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa70c15f1d0 0x7fa70c0b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:43.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.353+0000 7fa710a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa70c15f1d0 0x7fa70c0b09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36366/0 (socket says 172.21.15.159:36366) 2024-04-04T06:06:43.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.353+0000 7fa710a00640 1 -- 172.21.15.159:0/1595803082 learned_addr learned my addr 172.21.15.159:0/1595803082 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:43.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.353+0000 7fa70be00640 1 --2- 172.21.15.159:0/1595803082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6f00039b0 0x7fa70c15e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:43.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.353+0000 7fa703400640 1 --2- 172.21.15.159:0/1595803082 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa70c15ed70 0x7fa70c0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:43.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.353+0000 7fa710a00640 1 -- 172.21.15.159:0/1595803082 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa70c15ed70 msgr2=0x7fa70c0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:43.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.353+0000 7fa710a00640 1 --2- 172.21.15.159:0/1595803082 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa70c15ed70 0x7fa70c0b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:43.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.353+0000 7fa710a00640 1 -- 172.21.15.159:0/1595803082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6f00039b0 msgr2=0x7fa70c15e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:43.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.353+0000 7fa710a00640 1 --2- 172.21.15.159:0/1595803082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6f00039b0 0x7fa70c15e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:43.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.353+0000 7fa710a00640 1 -- 172.21.15.159:0/1595803082 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa70c15f750 con 0x7fa70c15f1d0 2024-04-04T06:06:43.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.354+0000 7fa710a00640 1 --2- 172.21.15.159:0/1595803082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa70c15f1d0 0x7fa70c0b09a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa70c072810 tx=0x7fa704007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:43.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.354+0000 7fa708c00640 1 -- 172.21.15.159:0/1595803082 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704014410 con 0x7fa70c15f1d0 2024-04-04T06:06:43.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.354+0000 7fa708c00640 1 -- 172.21.15.159:0/1595803082 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa704019070 con 0x7fa70c15f1d0 2024-04-04T06:06:43.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.354+0000 7fa712a00640 1 -- 172.21.15.159:0/1595803082 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa70c15fa30 con 0x7fa70c15f1d0 2024-04-04T06:06:43.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.354+0000 7fa708c00640 1 -- 172.21.15.159:0/1595803082 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704014810 con 0x7fa70c15f1d0 2024-04-04T06:06:43.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.354+0000 7fa712a00640 1 -- 172.21.15.159:0/1595803082 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa70c1f8ef0 con 0x7fa70c15f1d0 2024-04-04T06:06:43.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.355+0000 7fa712a00640 1 -- 172.21.15.159:0/1595803082 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa70c15ff50 con 0x7fa70c15f1d0 2024-04-04T06:06:43.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.356+0000 7fa708c00640 1 -- 172.21.15.159:0/1595803082 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7040149b0 con 0x7fa70c15f1d0 2024-04-04T06:06:43.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.357+0000 7fa708c00640 1 --2- 172.21.15.159:0/1595803082 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa6e806c920 0x7fa6e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:43.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.358+0000 7fa708c00640 1 -- 172.21.15.159:0/1595803082 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7fa70408c150 con 0x7fa70c15f1d0 2024-04-04T06:06:43.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.358+0000 7fa70be00640 1 --2- 172.21.15.159:0/1595803082 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa6e806c920 0x7fa6e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:43.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.359+0000 7fa70be00640 1 --2- 172.21.15.159:0/1595803082 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa6e806c920 0x7fa6e806ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc034000 tx=0x7fa6fc007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:43.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.362+0000 7fa708c00640 1 -- 172.21.15.159:0/1595803082 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa70c15ff50 con 0x7fa70c15f1d0 2024-04-04T06:06:43.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.563+0000 7fa712a00640 1 -- 172.21.15.159:0/1595803082 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa70c0685c0 con 0x7fa70c15f1d0 2024-04-04T06:06:43.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.564+0000 7fa708c00640 1 -- 172.21.15.159:0/1595803082 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fa70405a440 con 0x7fa70c15f1d0 2024-04-04T06:06:43.559 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:06:43.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.567+0000 7fa712a00640 1 -- 172.21.15.159:0/1595803082 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa6e806c920 msgr2=0x7fa6e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:43.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.567+0000 7fa712a00640 1 --2- 172.21.15.159:0/1595803082 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa6e806c920 0x7fa6e806ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc034000 tx=0x7fa6fc007000 comp rx=0 tx=0).stop 2024-04-04T06:06:43.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.567+0000 7fa712a00640 1 -- 172.21.15.159:0/1595803082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa70c15f1d0 msgr2=0x7fa70c0b09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:43.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.567+0000 7fa712a00640 1 --2- 172.21.15.159:0/1595803082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa70c15f1d0 0x7fa70c0b09a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa70c072810 tx=0x7fa704007700 comp rx=0 tx=0).stop 2024-04-04T06:06:43.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.567+0000 7fa712a00640 1 -- 172.21.15.159:0/1595803082 shutdown_connections 2024-04-04T06:06:43.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.567+0000 7fa712a00640 1 --2- 172.21.15.159:0/1595803082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa70c15f1d0 0x7fa70c0b09a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:43.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.567+0000 7fa712a00640 1 --2- 172.21.15.159:0/1595803082 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa70c15ed70 0x7fa70c0b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:43.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.567+0000 7fa712a00640 1 --2- 172.21.15.159:0/1595803082 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa6e806c920 0x7fa6e806ede0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:43.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.567+0000 7fa712a00640 1 --2- 172.21.15.159:0/1595803082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6f00039b0 0x7fa70c15e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:43.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.567+0000 7fa712a00640 1 -- 172.21.15.159:0/1595803082 >> 172.21.15.159:0/1595803082 conn(0x7fa70c06b750 msgr2=0x7fa70c0ad010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:43.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.567+0000 7fa712a00640 1 -- 172.21.15.159:0/1595803082 shutdown_connections 2024-04-04T06:06:43.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.567+0000 7fa712a00640 1 -- 172.21.15.159:0/1595803082 wait complete. 2024-04-04T06:06:43.581 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-04T06:06:43.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.798+0000 7fe3a1800640 1 Processor -- start 2024-04-04T06:06:43.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.799+0000 7fe3a1800640 1 -- start start 2024-04-04T06:06:43.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.800+0000 7fe3a1800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe39c0b4a30 0x7fe39c0b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:43.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.800+0000 7fe3a1800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe39c158580 0x7fe39c1589f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:43.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.800+0000 7fe3a1800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe39c15ede0 con 0x7fe39c0b4650 2024-04-04T06:06:43.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.800+0000 7fe3a1800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe39c141b30 con 0x7fe39c0b4a30 2024-04-04T06:06:43.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.800+0000 7fe3a1800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe39c141d80 con 0x7fe39c158580 2024-04-04T06:06:43.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.801+0000 7fe39a000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe39c0b4a30 0x7fe39c0b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:43.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.801+0000 7fe39a000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe39c0b4a30 0x7fe39c0b5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36368/0 (socket says 172.21.15.159:36368) 2024-04-04T06:06:43.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.801+0000 7fe39a000640 1 -- 172.21.15.159:0/3056187373 learned_addr learned my addr 172.21.15.159:0/3056187373 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:43.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.801+0000 7fe39b400640 1 --2- 172.21.15.159:0/3056187373 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe39c158580 0x7fe39c1589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:43.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.801+0000 7fe39a000640 1 -- 172.21.15.159:0/3056187373 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe39c158580 msgr2=0x7fe39c1589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:43.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.801+0000 7fe39a000640 1 --2- 172.21.15.159:0/3056187373 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe39c158580 0x7fe39c1589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:43.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.801+0000 7fe39a000640 1 -- 172.21.15.159:0/3056187373 >> v1:172.21.15.178:6789/0 conn(0x7fe39c0b4650 legacy=0x7fe39c06c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:43.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.801+0000 7fe39a000640 1 -- 172.21.15.159:0/3056187373 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39c142060 con 0x7fe39c0b4a30 2024-04-04T06:06:43.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.802+0000 7fe39a000640 1 --2- 172.21.15.159:0/3056187373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe39c0b4a30 0x7fe39c0b5c10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe39c141cb0 tx=0x7fe38402be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdfdaa9db9bfa73 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:43.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.803+0000 7fe399600640 1 -- 172.21.15.159:0/3056187373 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38402ca20 con 0x7fe39c0b4a30 2024-04-04T06:06:43.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.803+0000 7fe399600640 1 -- 172.21.15.159:0/3056187373 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe38402cbc0 con 0x7fe39c0b4a30 2024-04-04T06:06:43.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.803+0000 7fe399600640 1 -- 172.21.15.159:0/3056187373 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3840343f0 con 0x7fe39c0b4a30 2024-04-04T06:06:43.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.803+0000 7fe3a1800640 1 -- 172.21.15.159:0/3056187373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe39c0b4a30 msgr2=0x7fe39c0b5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:43.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.803+0000 7fe3a1800640 1 --2- 172.21.15.159:0/3056187373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe39c0b4a30 0x7fe39c0b5c10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe39c141cb0 tx=0x7fe38402be80 comp rx=0 tx=0).stop 2024-04-04T06:06:43.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.803+0000 7fe3a1800640 1 -- 172.21.15.159:0/3056187373 shutdown_connections 2024-04-04T06:06:43.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.803+0000 7fe3a1800640 1 --2- 172.21.15.159:0/3056187373 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe39c158580 0x7fe39c1589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:43.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.803+0000 7fe3a1800640 1 --2- 172.21.15.159:0/3056187373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe39c0b4a30 0x7fe39c0b5c10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:43.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.803+0000 7fe3a1800640 1 -- 172.21.15.159:0/3056187373 >> 172.21.15.159:0/3056187373 conn(0x7fe39c067670 msgr2=0x7fe39c067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:43.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.803+0000 7fe3a1800640 1 -- 172.21.15.159:0/3056187373 shutdown_connections 2024-04-04T06:06:43.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.803+0000 7fe3a1800640 1 -- 172.21.15.159:0/3056187373 wait complete. 2024-04-04T06:06:43.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.805+0000 7fe3a1800640 1 Processor -- start 2024-04-04T06:06:43.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.805+0000 7fe3a1800640 1 -- start start 2024-04-04T06:06:43.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.806+0000 7fe3a1800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe39c0b4650 0x7fe39c159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:43.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.806+0000 7fe3a1800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe39c0b4a30 0x7fe39c15b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:43.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.806+0000 7fe3a1800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe39c158580 0x7fe39c1f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:43.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.806+0000 7fe3a1800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c142900 con 0x7fe39c0b4650 2024-04-04T06:06:43.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.806+0000 7fe3a1800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c142620 con 0x7fe39c158580 2024-04-04T06:06:43.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.806+0000 7fe3a1800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe39c158f30 con 0x7fe39c0b4a30 2024-04-04T06:06:43.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.807+0000 7fe39aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe39c0b4650 0x7fe39c159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:43.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.807+0000 7fe39aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe39c0b4650 0x7fe39c159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36370/0 (socket says 172.21.15.159:36370) 2024-04-04T06:06:43.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.807+0000 7fe39aa00640 1 -- 172.21.15.159:0/1233454563 learned_addr learned my addr 172.21.15.159:0/1233454563 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:43.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.807+0000 7fe39b400640 1 --2- 172.21.15.159:0/1233454563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe39c158580 0x7fe39c1f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:43.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.807+0000 7fe39a000640 1 --2- 172.21.15.159:0/1233454563 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe39c0b4a30 0x7fe39c15b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:43.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.807+0000 7fe39aa00640 1 -- 172.21.15.159:0/1233454563 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe39c0b4a30 msgr2=0x7fe39c15b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:43.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.807+0000 7fe39aa00640 1 --2- 172.21.15.159:0/1233454563 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe39c0b4a30 0x7fe39c15b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:43.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.807+0000 7fe39aa00640 1 -- 172.21.15.159:0/1233454563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe39c158580 msgr2=0x7fe39c1f71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:43.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.807+0000 7fe39aa00640 1 --2- 172.21.15.159:0/1233454563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe39c158580 0x7fe39c1f71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:43.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.807+0000 7fe39aa00640 1 -- 172.21.15.159:0/1233454563 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39c0cdc10 con 0x7fe39c0b4650 2024-04-04T06:06:43.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.807+0000 7fe39a000640 1 --2- 172.21.15.159:0/1233454563 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe39c0b4a30 0x7fe39c15b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:43.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.807+0000 7fe39b400640 1 --2- 172.21.15.159:0/1233454563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe39c158580 0x7fe39c1f71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:43.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.807+0000 7fe39aa00640 1 --2- 172.21.15.159:0/1233454563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe39c0b4650 0x7fe39c159140 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00c430 tx=0x7fe38c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:43.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.808+0000 7fe392a00640 1 -- 172.21.15.159:0/1233454563 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c011790 con 0x7fe39c0b4650 2024-04-04T06:06:43.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.808+0000 7fe392a00640 1 -- 172.21.15.159:0/1233454563 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe38c017070 con 0x7fe39c0b4650 2024-04-04T06:06:43.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.808+0000 7fe392a00640 1 -- 172.21.15.159:0/1233454563 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c011b70 con 0x7fe39c0b4650 2024-04-04T06:06:43.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.808+0000 7fe3a1800640 1 -- 172.21.15.159:0/1233454563 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39c159790 con 0x7fe39c0b4650 2024-04-04T06:06:43.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.808+0000 7fe3a1800640 1 -- 172.21.15.159:0/1233454563 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe39c159d50 con 0x7fe39c0b4650 2024-04-04T06:06:43.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.809+0000 7fe3a1800640 1 -- 172.21.15.159:0/1233454563 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe39c0b6850 con 0x7fe39c0b4650 2024-04-04T06:06:43.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.810+0000 7fe392a00640 1 -- 172.21.15.159:0/1233454563 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe38c011d10 con 0x7fe39c0b4650 2024-04-04T06:06:43.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.811+0000 7fe392a00640 1 --2- 172.21.15.159:0/1233454563 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe37806c8f0 0x7fe37806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:43.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.811+0000 7fe392a00640 1 -- 172.21.15.159:0/1233454563 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7fe38c08bb80 con 0x7fe39c0b4650 2024-04-04T06:06:43.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.811+0000 7fe39a000640 1 --2- 172.21.15.159:0/1233454563 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe37806c8f0 0x7fe37806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:43.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.813+0000 7fe39a000640 1 --2- 172.21.15.159:0/1233454563 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe37806c8f0 0x7fe37806edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe384035000 tx=0x7fe384030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:43.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:43.816+0000 7fe392a00640 1 -- 172.21.15.159:0/1233454563 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe39c0b6850 con 0x7fe39c0b4650 2024-04-04T06:06:44.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.029+0000 7fe3a1800640 1 -- 172.21.15.159:0/1233454563 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe39c0b2e00 con 0x7fe39c0b4650 2024-04-04T06:06:44.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.029+0000 7fe392a00640 1 -- 172.21.15.159:0/1233454563 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fe38c059dd0 con 0x7fe39c0b4650 2024-04-04T06:06:44.024 INFO:teuthology.orchestra.run.smithi159.stderr:listed 0 entries 2024-04-04T06:06:44.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.032+0000 7fe3a1800640 1 -- 172.21.15.159:0/1233454563 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe37806c8f0 msgr2=0x7fe37806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:44.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.032+0000 7fe3a1800640 1 --2- 172.21.15.159:0/1233454563 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe37806c8f0 0x7fe37806edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe384035000 tx=0x7fe384030000 comp rx=0 tx=0).stop 2024-04-04T06:06:44.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.032+0000 7fe3a1800640 1 -- 172.21.15.159:0/1233454563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe39c0b4650 msgr2=0x7fe39c159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:44.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.032+0000 7fe3a1800640 1 --2- 172.21.15.159:0/1233454563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe39c0b4650 0x7fe39c159140 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00c430 tx=0x7fe38c00c8f0 comp rx=0 tx=0).stop 2024-04-04T06:06:44.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.032+0000 7fe3a1800640 1 -- 172.21.15.159:0/1233454563 shutdown_connections 2024-04-04T06:06:44.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.032+0000 7fe3a1800640 1 --2- 172.21.15.159:0/1233454563 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe37806c8f0 0x7fe37806edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:44.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.032+0000 7fe3a1800640 1 --2- 172.21.15.159:0/1233454563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe39c158580 0x7fe39c1f71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:44.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.032+0000 7fe3a1800640 1 --2- 172.21.15.159:0/1233454563 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe39c0b4a30 0x7fe39c15b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:44.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.032+0000 7fe3a1800640 1 --2- 172.21.15.159:0/1233454563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe39c0b4650 0x7fe39c159140 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:44.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.032+0000 7fe3a1800640 1 -- 172.21.15.159:0/1233454563 >> 172.21.15.159:0/1233454563 conn(0x7fe39c067670 msgr2=0x7fe39c06df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:44.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.032+0000 7fe3a1800640 1 -- 172.21.15.159:0/1233454563 shutdown_connections 2024-04-04T06:06:44.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.032+0000 7fe3a1800640 1 -- 172.21.15.159:0/1233454563 wait complete. 2024-04-04T06:06:44.048 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-04T06:06:44.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:44.048 DEBUG:teuthology.orchestra.run.smithi159:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-04T06:06:44.147 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:06:44.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:44.147 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-04T06:06:44.212 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:06:44.212 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:06:44.268 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:06:44.268 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:06:44.318 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:06:44.318 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:06:44.367 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:06:44.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.595+0000 7fb013600640 1 Processor -- start 2024-04-04T06:06:44.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.595+0000 7fb013600640 1 -- start start 2024-04-04T06:06:44.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.596+0000 7fb013600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb00c0af830 0x7fb00c073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:44.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.597+0000 7fb013600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb00c0d0650 0x7fb00c0d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:44.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.597+0000 7fb013600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb00c15fe80 con 0x7fb00c15c390 2024-04-04T06:06:44.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.597+0000 7fb013600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fb00c0d1000 con 0x7fb00c0af830 2024-04-04T06:06:44.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.597+0000 7fb013600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb00c15c770 con 0x7fb00c0d0650 2024-04-04T06:06:44.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.597+0000 7fb00be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb00c0af830 0x7fb00c073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:44.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.597+0000 7fb00be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb00c0af830 0x7fb00c073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41062/0 (socket says 172.21.15.159:41062) 2024-04-04T06:06:44.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.597+0000 7fb010c00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fb00c15c390 0x7fb00c160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:32858/0 (socket says 172.21.15.159:32858) 2024-04-04T06:06:44.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.597+0000 7fb00be00640 1 -- 172.21.15.159:0/3066074178 learned_addr learned my addr 172.21.15.159:0/3066074178 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:44.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.597+0000 7fb011600640 1 --2- 172.21.15.159:0/3066074178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb00c0d0650 0x7fb00c0d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:44.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.598+0000 7fb00be00640 1 -- 172.21.15.159:0/3066074178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb00c0d0650 msgr2=0x7fb00c0d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:44.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.598+0000 7fb00be00640 1 --2- 172.21.15.159:0/3066074178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb00c0d0650 0x7fb00c0d0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:44.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.598+0000 7fb00be00640 1 -- 172.21.15.159:0/3066074178 >> v1:172.21.15.159:6789/0 conn(0x7fb00c15c390 legacy=0x7fb00c160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:44.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.598+0000 7fb00be00640 1 -- 172.21.15.159:0/3066074178 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00c0ad120 con 0x7fb00c0af830 2024-04-04T06:06:44.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.599+0000 7fb00be00640 1 --2- 172.21.15.159:0/3066074178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c0af830 0x7fb00c073790 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faff8009870 tx=0x7faff802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4259355fd41a96c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:44.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.599+0000 7fb00b400640 1 -- 172.21.15.159:0/3066074178 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff8034070 con 0x7fb00c0af830 2024-04-04T06:06:44.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.599+0000 7fb00b400640 1 -- 172.21.15.159:0/3066074178 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff8030840 con 0x7fb00c0af830 2024-04-04T06:06:44.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.599+0000 7fb00b400640 1 -- 172.21.15.159:0/3066074178 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff8030c10 con 0x7fb00c0af830 2024-04-04T06:06:44.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.599+0000 7fb013600640 1 -- 172.21.15.159:0/3066074178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c0af830 msgr2=0x7fb00c073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:44.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.600+0000 7fb013600640 1 --2- 172.21.15.159:0/3066074178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c0af830 0x7fb00c073790 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faff8009870 tx=0x7faff802b980 comp rx=0 tx=0).stop 2024-04-04T06:06:44.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.600+0000 7fb013600640 1 -- 172.21.15.159:0/3066074178 shutdown_connections 2024-04-04T06:06:44.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.600+0000 7fb013600640 1 --2- 172.21.15.159:0/3066074178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb00c0d0650 0x7fb00c0d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:44.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.600+0000 7fb013600640 1 --2- 172.21.15.159:0/3066074178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c0af830 0x7fb00c073790 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:44.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.600+0000 7fb013600640 1 -- 172.21.15.159:0/3066074178 >> 172.21.15.159:0/3066074178 conn(0x7fb00c06b990 msgr2=0x7fb00c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:44.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.600+0000 7fb013600640 1 -- 172.21.15.159:0/3066074178 shutdown_connections 2024-04-04T06:06:44.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.600+0000 7fb013600640 1 -- 172.21.15.159:0/3066074178 wait complete. 2024-04-04T06:06:44.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.601+0000 7fb013600640 1 Processor -- start 2024-04-04T06:06:44.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.602+0000 7fb013600640 1 -- start start 2024-04-04T06:06:44.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.602+0000 7fb013600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c0af830 0x7fb00c1421a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:44.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.602+0000 7fb013600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb00c0d0650 0x7fb00c144a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:44.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.603+0000 7fb013600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb00c15c390 0x7fb00c144f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:44.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.603+0000 7fb013600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c160120 con 0x7fb00c0af830 2024-04-04T06:06:44.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.603+0000 7fb013600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c0ad6e0 con 0x7fb00c0d0650 2024-04-04T06:06:44.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.603+0000 7fb013600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb00c160d50 con 0x7fb00c15c390 2024-04-04T06:06:44.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.603+0000 7fb010c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c0af830 0x7fb00c1421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:44.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.603+0000 7fb010c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c0af830 0x7fb00c1421a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41068/0 (socket says 172.21.15.159:41068) 2024-04-04T06:06:44.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.603+0000 7fb010c00640 1 -- 172.21.15.159:0/1782203375 learned_addr learned my addr 172.21.15.159:0/1782203375 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:44.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.603+0000 7fb00be00640 1 --2- 172.21.15.159:0/1782203375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb00c0d0650 0x7fb00c144a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:44.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.603+0000 7fb011600640 1 --2- 172.21.15.159:0/1782203375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb00c15c390 0x7fb00c144f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:44.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.603+0000 7fb010c00640 1 -- 172.21.15.159:0/1782203375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb00c15c390 msgr2=0x7fb00c144f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:44.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.603+0000 7fb010c00640 1 --2- 172.21.15.159:0/1782203375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb00c15c390 0x7fb00c144f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:44.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.603+0000 7fb010c00640 1 -- 172.21.15.159:0/1782203375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb00c0d0650 msgr2=0x7fb00c144a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:44.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.603+0000 7fb010c00640 1 --2- 172.21.15.159:0/1782203375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb00c0d0650 0x7fb00c144a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:44.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.603+0000 7fb010c00640 1 -- 172.21.15.159:0/1782203375 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00c15e7d0 con 0x7fb00c0af830 2024-04-04T06:06:44.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.604+0000 7fb010c00640 1 --2- 172.21.15.159:0/1782203375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c0af830 0x7fb00c1421a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb00000c430 tx=0x7fb00000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:44.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.604+0000 7fb008c00640 1 -- 172.21.15.159:0/1782203375 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0000074e0 con 0x7fb00c0af830 2024-04-04T06:06:44.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.604+0000 7fb008c00640 1 -- 172.21.15.159:0/1782203375 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb000007680 con 0x7fb00c0af830 2024-04-04T06:06:44.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.604+0000 7fb013600640 1 -- 172.21.15.159:0/1782203375 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb00c142b40 con 0x7fb00c0af830 2024-04-04T06:06:44.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.604+0000 7fb008c00640 1 -- 172.21.15.159:0/1782203375 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000004a00 con 0x7fb00c0af830 2024-04-04T06:06:44.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.604+0000 7fb013600640 1 -- 172.21.15.159:0/1782203375 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb00c15e4f0 con 0x7fb00c0af830 2024-04-04T06:06:44.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.605+0000 7fb013600640 1 -- 172.21.15.159:0/1782203375 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb00c0685c0 con 0x7fb00c0af830 2024-04-04T06:06:44.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.606+0000 7fb008c00640 1 -- 172.21.15.159:0/1782203375 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb00001a020 con 0x7fb00c0af830 2024-04-04T06:06:44.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.607+0000 7fb008c00640 1 --2- 172.21.15.159:0/1782203375 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fafec06c8f0 0x7fafec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:44.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.607+0000 7fb008c00640 1 -- 172.21.15.159:0/1782203375 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7fb00008b3a0 con 0x7fb00c0af830 2024-04-04T06:06:44.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.608+0000 7fb00be00640 1 --2- 172.21.15.159:0/1782203375 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fafec06c8f0 0x7fafec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:44.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.609+0000 7fb00be00640 1 --2- 172.21.15.159:0/1782203375 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fafec06c8f0 0x7fafec06edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faff8036000 tx=0x7faff803b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:44.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.612+0000 7fb008c00640 1 -- 172.21.15.159:0/1782203375 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb0000597b0 con 0x7fb00c0af830 2024-04-04T06:06:44.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.819+0000 7fb013600640 1 -- 172.21.15.159:0/1782203375 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb00c06f1b0 con 0x7fb00c0af830 2024-04-04T06:06:44.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.821+0000 7fb008c00640 1 -- 172.21.15.159:0/1782203375 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16958 (secure 0 0 0) 0x7fb0000590b0 con 0x7fb00c0af830 2024-04-04T06:06:44.816 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:44.816 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":23,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:06:33.796979+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:05:51.297293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-04T06:05:51.770792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":0,"to":2}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:06:44.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.823+0000 7fb013600640 1 -- 172.21.15.159:0/1782203375 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fafec06c8f0 msgr2=0x7fafec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:44.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.823+0000 7fb013600640 1 --2- 172.21.15.159:0/1782203375 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fafec06c8f0 0x7fafec06edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faff8036000 tx=0x7faff803b040 comp rx=0 tx=0).stop 2024-04-04T06:06:44.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.823+0000 7fb013600640 1 -- 172.21.15.159:0/1782203375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c0af830 msgr2=0x7fb00c1421a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:44.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.823+0000 7fb013600640 1 --2- 172.21.15.159:0/1782203375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c0af830 0x7fb00c1421a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb00000c430 tx=0x7fb00000c8f0 comp rx=0 tx=0).stop 2024-04-04T06:06:44.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.824+0000 7fb013600640 1 -- 172.21.15.159:0/1782203375 shutdown_connections 2024-04-04T06:06:44.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.824+0000 7fb013600640 1 --2- 172.21.15.159:0/1782203375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb00c15c390 0x7fb00c144f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:44.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.824+0000 7fb013600640 1 --2- 172.21.15.159:0/1782203375 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fafec06c8f0 0x7fafec06edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:44.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.824+0000 7fb013600640 1 --2- 172.21.15.159:0/1782203375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb00c0d0650 0x7fb00c144a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:44.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.824+0000 7fb013600640 1 --2- 172.21.15.159:0/1782203375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c0af830 0x7fb00c1421a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:44.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.824+0000 7fb013600640 1 -- 172.21.15.159:0/1782203375 >> 172.21.15.159:0/1782203375 conn(0x7fb00c06b990 msgr2=0x7fb00c0728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:44.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.824+0000 7fb013600640 1 -- 172.21.15.159:0/1782203375 shutdown_connections 2024-04-04T06:06:44.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:44.824+0000 7fb013600640 1 -- 172.21.15.159:0/1782203375 wait complete. 2024-04-04T06:06:44.839 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:05:51.297293+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:05:51.770792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:06:44.839 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:06:45.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.056+0000 7f42c0c00640 1 Processor -- start 2024-04-04T06:06:45.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.056+0000 7f42c0c00640 1 -- start start 2024-04-04T06:06:45.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.058+0000 7f42c0c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42bc153580 0x7f42bc0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:45.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.058+0000 7f42c0c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f42bc0af4b0 0x7f42bc1600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:45.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.058+0000 7f42c0c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42bc0afab0 con 0x7f42bc0d07c0 2024-04-04T06:06:45.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.058+0000 7f42c0c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f42bc0d0c00 con 0x7f42bc0af4b0 2024-04-04T06:06:45.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.058+0000 7f42c0c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f42bc15cc80 con 0x7f42bc153580 2024-04-04T06:06:45.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.058+0000 7f42b9400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f42bc0d07c0 0x7f42bc0aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:32870/0 (socket says 172.21.15.159:32870) 2024-04-04T06:06:45.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.058+0000 7f42b9400640 1 -- 172.21.15.159:0/71878721 learned_addr learned my addr 172.21.15.159:0/71878721 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:45.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.058+0000 7f42b9e00640 1 --2- 172.21.15.159:0/71878721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42bc153580 0x7f42bc0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:45.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.058+0000 7f42ba800640 1 --2- 172.21.15.159:0/71878721 >> v2:172.21.15.178:3300/0 conn(0x7f42bc0af4b0 0x7f42bc1600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:45.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.059+0000 7f42b8a00640 1 -- 172.21.15.159:0/71878721 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1645979755 0 0) 0x7f42bc0afab0 con 0x7f42bc0d07c0 2024-04-04T06:06:45.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.059+0000 7f42b8a00640 1 -- 172.21.15.159:0/71878721 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42a0003520 con 0x7f42bc0d07c0 2024-04-04T06:06:45.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.059+0000 7f42b8a00640 1 -- 172.21.15.159:0/71878721 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 345810047 0 0) 0x7f42a0003520 con 0x7f42bc0d07c0 2024-04-04T06:06:45.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.059+0000 7f42b8a00640 1 -- 172.21.15.159:0/71878721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42bc153580 msgr2=0x7f42bc0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:45.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.059+0000 7f42b8a00640 1 --2- 172.21.15.159:0/71878721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42bc153580 0x7f42bc0d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.059+0000 7f42b8a00640 1 -- 172.21.15.159:0/71878721 >> v2:172.21.15.178:3300/0 conn(0x7f42bc0af4b0 msgr2=0x7f42bc1600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:45.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.059+0000 7f42b8a00640 1 --2- 172.21.15.159:0/71878721 >> v2:172.21.15.178:3300/0 conn(0x7f42bc0af4b0 0x7f42bc1600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.059+0000 7f42b8a00640 1 -- 172.21.15.159:0/71878721 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42bc15da00 con 0x7f42bc0d07c0 2024-04-04T06:06:45.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.059+0000 7f42b8a00640 1 -- 172.21.15.159:0/71878721 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f42a8001670 con 0x7f42bc0d07c0 2024-04-04T06:06:45.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.059+0000 7f42b8a00640 1 -- 172.21.15.159:0/71878721 >> v1:172.21.15.159:6789/0 conn(0x7f42bc0d07c0 legacy=0x7f42bc0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:45.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.059+0000 7f42b8a00640 1 --2- 172.21.15.159:0/71878721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42a00039b0 0x7f42a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:45.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.059+0000 7f42b8a00640 1 --2- 172.21.15.159:0/71878721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a0004800 0x7f42a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:45.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.059+0000 7f42b8a00640 1 --2- 172.21.15.159:0/71878721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42bc153580 0x7f42a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:45.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.059+0000 7f42b8a00640 1 -- 172.21.15.159:0/71878721 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc0d0c00 con 0x7f42a0004800 2024-04-04T06:06:45.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.059+0000 7f42b8a00640 1 -- 172.21.15.159:0/71878721 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc15cc80 con 0x7f42a00039b0 2024-04-04T06:06:45.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.059+0000 7f42b8a00640 1 -- 172.21.15.159:0/71878721 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f42a000b9c0 con 0x7f42bc153580 2024-04-04T06:06:45.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.059+0000 7f42b9400640 1 --2- 172.21.15.159:0/71878721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a0004800 0x7f42a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:45.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.060+0000 7f42ba800640 1 --2- 172.21.15.159:0/71878721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42bc153580 0x7f42a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:45.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.060+0000 7f42b9400640 1 -- 172.21.15.159:0/71878721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42bc153580 msgr2=0x7f42a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:45.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.060+0000 7f42b9400640 1 --2- 172.21.15.159:0/71878721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42bc153580 0x7f42a00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.060+0000 7f42b9400640 1 -- 172.21.15.159:0/71878721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42a00039b0 msgr2=0x7f42a0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:45.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.060+0000 7f42b9400640 1 --2- 172.21.15.159:0/71878721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42a00039b0 0x7f42a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.060+0000 7f42b9400640 1 -- 172.21.15.159:0/71878721 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a0003520 con 0x7f42a0004800 2024-04-04T06:06:45.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.061+0000 7f42b9400640 1 --2- 172.21.15.159:0/71878721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a0004800 0x7f42a0004c80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f42a800c240 tx=0x7f42a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c340f9afb0905400 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:45.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.061+0000 7f42b8a00640 1 -- 172.21.15.159:0/71878721 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a8007d50 con 0x7f42a0004800 2024-04-04T06:06:45.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.061+0000 7f42b8a00640 1 -- 172.21.15.159:0/71878721 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42a8030070 con 0x7f42a0004800 2024-04-04T06:06:45.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.061+0000 7f42b8a00640 1 -- 172.21.15.159:0/71878721 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a802cd80 con 0x7f42a0004800 2024-04-04T06:06:45.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.062+0000 7f42c0c00640 1 -- 172.21.15.159:0/71878721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a0004800 msgr2=0x7f42a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:45.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.062+0000 7f42c0c00640 1 --2- 172.21.15.159:0/71878721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a0004800 0x7f42a0004c80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f42a800c240 tx=0x7f42a0005830 comp rx=0 tx=0).stop 2024-04-04T06:06:45.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.062+0000 7f42ba800640 1 -- 172.21.15.159:0/71878721 reap_dead start 2024-04-04T06:06:45.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.062+0000 7f42c0c00640 1 -- 172.21.15.159:0/71878721 shutdown_connections 2024-04-04T06:06:45.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.062+0000 7f42c0c00640 1 -- 172.21.15.159:0/71878721 >> 172.21.15.159:0/71878721 conn(0x7f42bc06b750 msgr2=0x7f42bc06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:45.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.062+0000 7f42c0c00640 1 -- 172.21.15.159:0/71878721 shutdown_connections 2024-04-04T06:06:45.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.062+0000 7f42c0c00640 1 -- 172.21.15.159:0/71878721 wait complete. 2024-04-04T06:06:45.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.064+0000 7f42c0c00640 1 Processor -- start 2024-04-04T06:06:45.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.064+0000 7f42c0c00640 1 -- start start 2024-04-04T06:06:45.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.065+0000 7f42c0c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a0004800 0x7f42bc1f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:45.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.065+0000 7f42c0c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42bc1556e0 0x7f42bc151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:45.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.065+0000 7f42c0c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42bc071610 0x7f42bc0ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:45.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.065+0000 7f42c0c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc15d4e0 con 0x7f42a0004800 2024-04-04T06:06:45.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.065+0000 7f42c0c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc15d200 con 0x7f42bc071610 2024-04-04T06:06:45.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.065+0000 7f42c0c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f42bc153960 con 0x7f42bc1556e0 2024-04-04T06:06:45.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.065+0000 7f42b9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a0004800 0x7f42bc1f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:45.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.065+0000 7f42b9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a0004800 0x7f42bc1f4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41078/0 (socket says 172.21.15.159:41078) 2024-04-04T06:06:45.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.065+0000 7f42b9e00640 1 -- 172.21.15.159:0/3353719614 learned_addr learned my addr 172.21.15.159:0/3353719614 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:45.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.066+0000 7f42b9400640 1 --2- 172.21.15.159:0/3353719614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42bc1556e0 0x7f42bc151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:45.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.066+0000 7f42b9e00640 1 -- 172.21.15.159:0/3353719614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42bc1556e0 msgr2=0x7f42bc151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:45.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.066+0000 7f42b9e00640 1 --2- 172.21.15.159:0/3353719614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42bc1556e0 0x7f42bc151df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.066+0000 7f42b9e00640 1 -- 172.21.15.159:0/3353719614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42bc071610 msgr2=0x7f42bc0ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:45.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.066+0000 7f42ba800640 1 --2- 172.21.15.159:0/3353719614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42bc071610 0x7f42bc0ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:45.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.066+0000 7f42b9e00640 1 --2- 172.21.15.159:0/3353719614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42bc071610 0x7f42bc0ca600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.066+0000 7f42b9e00640 1 -- 172.21.15.159:0/3353719614 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42bc1f4fb0 con 0x7f42a0004800 2024-04-04T06:06:45.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.066+0000 7f42b9e00640 1 --2- 172.21.15.159:0/3353719614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a0004800 0x7f42bc1f4a70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00e7a0 tx=0x7f42ac00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:45.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.066+0000 7f42b2000640 1 -- 172.21.15.159:0/3353719614 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac017070 con 0x7f42a0004800 2024-04-04T06:06:45.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.066+0000 7f42b2000640 1 -- 172.21.15.159:0/3353719614 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42ac00c940 con 0x7f42a0004800 2024-04-04T06:06:45.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.066+0000 7f42b2000640 1 -- 172.21.15.159:0/3353719614 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac00cd10 con 0x7f42a0004800 2024-04-04T06:06:45.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.066+0000 7f42c0c00640 1 -- 172.21.15.159:0/3353719614 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42bc152330 con 0x7f42a0004800 2024-04-04T06:06:45.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.067+0000 7f42c0c00640 1 -- 172.21.15.159:0/3353719614 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42bc159d80 con 0x7f42a0004800 2024-04-04T06:06:45.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.068+0000 7f42c0c00640 1 -- 172.21.15.159:0/3353719614 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42bc0685c0 con 0x7f42a0004800 2024-04-04T06:06:45.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.069+0000 7f42b2000640 1 -- 172.21.15.159:0/3353719614 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f42ac01a070 con 0x7f42a0004800 2024-04-04T06:06:45.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.069+0000 7f42b2000640 1 --2- 172.21.15.159:0/3353719614 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f429c06c8f0 0x7f429c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:45.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.070+0000 7f42b2000640 1 -- 172.21.15.159:0/3353719614 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f42ac08ba10 con 0x7f42a0004800 2024-04-04T06:06:45.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.070+0000 7f42b9400640 1 --2- 172.21.15.159:0/3353719614 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f429c06c8f0 0x7f429c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:45.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.071+0000 7f42b9400640 1 --2- 172.21.15.159:0/3353719614 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f429c06c8f0 0x7f429c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f42a8034000 tx=0x7f42a8007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:45.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.074+0000 7f42b2000640 1 -- 172.21.15.159:0/3353719614 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f42bc1f4fb0 con 0x7f42a0004800 2024-04-04T06:06:45.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.274+0000 7f42c0c00640 1 -- 172.21.15.159:0/3353719614 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f42bc15b1f0 con 0x7f42a0004800 2024-04-04T06:06:45.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.275+0000 7f42b2000640 1 -- 172.21.15.159:0/3353719614 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f42ac059d80 con 0x7f42a0004800 2024-04-04T06:06:45.270 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:06:45.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.277+0000 7f42c0c00640 1 -- 172.21.15.159:0/3353719614 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f429c06c8f0 msgr2=0x7f429c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:45.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.277+0000 7f42c0c00640 1 --2- 172.21.15.159:0/3353719614 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f429c06c8f0 0x7f429c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f42a8034000 tx=0x7f42a8007000 comp rx=0 tx=0).stop 2024-04-04T06:06:45.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.277+0000 7f42c0c00640 1 -- 172.21.15.159:0/3353719614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a0004800 msgr2=0x7f42bc1f4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:45.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.277+0000 7f42c0c00640 1 --2- 172.21.15.159:0/3353719614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a0004800 0x7f42bc1f4a70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00e7a0 tx=0x7f42ac00ec60 comp rx=0 tx=0).stop 2024-04-04T06:06:45.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.278+0000 7f42c0c00640 1 -- 172.21.15.159:0/3353719614 shutdown_connections 2024-04-04T06:06:45.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.278+0000 7f42c0c00640 1 --2- 172.21.15.159:0/3353719614 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f429c06c8f0 0x7f429c06edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.278+0000 7f42c0c00640 1 --2- 172.21.15.159:0/3353719614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42bc071610 0x7f42bc0ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.278+0000 7f42c0c00640 1 --2- 172.21.15.159:0/3353719614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42bc1556e0 0x7f42bc151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.278+0000 7f42c0c00640 1 --2- 172.21.15.159:0/3353719614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a0004800 0x7f42bc1f4a70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.278+0000 7f42c0c00640 1 -- 172.21.15.159:0/3353719614 >> 172.21.15.159:0/3353719614 conn(0x7f42bc06b750 msgr2=0x7f42bc15b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:45.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.278+0000 7f42c0c00640 1 -- 172.21.15.159:0/3353719614 shutdown_connections 2024-04-04T06:06:45.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.278+0000 7f42c0c00640 1 -- 172.21.15.159:0/3353719614 wait complete. 2024-04-04T06:06:45.291 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:06:45.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.509+0000 7f15f4e00640 1 Processor -- start 2024-04-04T06:06:45.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.509+0000 7f15f4e00640 1 -- start start 2024-04-04T06:06:45.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.511+0000 7f15f4e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f15f006bc80 0x7f15f00702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:45.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.511+0000 7f15f4e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15f0070c10 con 0x7f15f0147c80 2024-04-04T06:06:45.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.511+0000 7f15f4e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15f00cfe80 con 0x7f15f00707e0 2024-04-04T06:06:45.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.511+0000 7f15f4e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f15f00d0f20 con 0x7f15f006bc80 2024-04-04T06:06:45.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.511+0000 7f15eea00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f15f006bc80 0x7f15f00702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:45.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.511+0000 7f15ef400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f15f0147c80 0x7f15f006b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:32884/0 (socket says 172.21.15.159:32884) 2024-04-04T06:06:45.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.511+0000 7f15ef400640 1 -- 172.21.15.159:0/2921682991 learned_addr learned my addr 172.21.15.159:0/2921682991 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:45.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.512+0000 7f15eea00640 1 -- 172.21.15.159:0/2921682991 >> v1:172.21.15.178:6789/0 conn(0x7f15f00707e0 legacy=0x7f15f00724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:45.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.512+0000 7f15eea00640 1 -- 172.21.15.159:0/2921682991 >> v1:172.21.15.159:6789/0 conn(0x7f15f0147c80 legacy=0x7f15f006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:45.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.512+0000 7f15eea00640 1 -- 172.21.15.159:0/2921682991 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f00d0160 con 0x7f15f006bc80 2024-04-04T06:06:45.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.512+0000 7f15ee000640 1 -- 172.21.15.159:0/2921682991 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1012116106 0 0) 0x7f15f0070c10 con 0x7f15f0147c80 2024-04-04T06:06:45.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.513+0000 7f15eea00640 1 --2- 172.21.15.159:0/2921682991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15f006bc80 0x7f15f00702a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f15d80097a0 tx=0x7f15d802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718895e1b63181a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:45.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.513+0000 7f15ee000640 1 -- 172.21.15.159:0/2921682991 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8035020 con 0x7f15f006bc80 2024-04-04T06:06:45.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.513+0000 7f15ee000640 1 -- 172.21.15.159:0/2921682991 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15d8031830 con 0x7f15f006bc80 2024-04-04T06:06:45.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.513+0000 7f15ee000640 1 -- 172.21.15.159:0/2921682991 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8031c00 con 0x7f15f006bc80 2024-04-04T06:06:45.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.514+0000 7f15f4e00640 1 -- 172.21.15.159:0/2921682991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15f006bc80 msgr2=0x7f15f00702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:45.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.514+0000 7f15f4e00640 1 --2- 172.21.15.159:0/2921682991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15f006bc80 0x7f15f00702a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f15d80097a0 tx=0x7f15d802b8d0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.514+0000 7f15f4e00640 1 -- 172.21.15.159:0/2921682991 shutdown_connections 2024-04-04T06:06:45.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.514+0000 7f15f4e00640 1 --2- 172.21.15.159:0/2921682991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15f006bc80 0x7f15f00702a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.514+0000 7f15f4e00640 1 -- 172.21.15.159:0/2921682991 >> 172.21.15.159:0/2921682991 conn(0x7f15f006e730 msgr2=0x7f15f006eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:45.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.514+0000 7f15f4e00640 1 -- 172.21.15.159:0/2921682991 shutdown_connections 2024-04-04T06:06:45.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.514+0000 7f15f4e00640 1 -- 172.21.15.159:0/2921682991 wait complete. 2024-04-04T06:06:45.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.516+0000 7f15f4e00640 1 Processor -- start 2024-04-04T06:06:45.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.516+0000 7f15f4e00640 1 -- start start 2024-04-04T06:06:45.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.517+0000 7f15f4e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f15f006bc80 0x7f15f013ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:45.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.517+0000 7f15f4e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15f00707e0 0x7f15f01420f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:45.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.517+0000 7f15f4e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15f0147c80 0x7f15f00afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:45.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.517+0000 7f15f4e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f15f00d0720 con 0x7f15f0147c80 2024-04-04T06:06:45.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.517+0000 7f15f4e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f15f00d0a90 con 0x7f15f00707e0 2024-04-04T06:06:45.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.517+0000 7f15f4e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f15f00cfb50 con 0x7f15f006bc80 2024-04-04T06:06:45.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.517+0000 7f15efe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15f0147c80 0x7f15f00afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:45.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.517+0000 7f15efe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15f0147c80 0x7f15f00afaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41086/0 (socket says 172.21.15.159:41086) 2024-04-04T06:06:45.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.517+0000 7f15efe00640 1 -- 172.21.15.159:0/2966794391 learned_addr learned my addr 172.21.15.159:0/2966794391 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:45.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.517+0000 7f15efe00640 1 -- 172.21.15.159:0/2966794391 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f15f006bc80 msgr2=0x7f15f013ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:45.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.517+0000 7f15ef400640 1 --2- 172.21.15.159:0/2966794391 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f15f006bc80 0x7f15f013ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:45.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.518+0000 7f15eea00640 1 --2- 172.21.15.159:0/2966794391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15f00707e0 0x7f15f01420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:45.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.518+0000 7f15efe00640 1 --2- 172.21.15.159:0/2966794391 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f15f006bc80 0x7f15f013ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.518+0000 7f15efe00640 1 -- 172.21.15.159:0/2966794391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15f00707e0 msgr2=0x7f15f01420f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:45.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.518+0000 7f15efe00640 1 --2- 172.21.15.159:0/2966794391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15f00707e0 0x7f15f01420f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.518+0000 7f15efe00640 1 -- 172.21.15.159:0/2966794391 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f013f110 con 0x7f15f0147c80 2024-04-04T06:06:45.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.518+0000 7f15efe00640 1 --2- 172.21.15.159:0/2966794391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15f0147c80 0x7f15f00afaf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f15e800c410 tx=0x7f15e800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:45.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.518+0000 7f15e7400640 1 -- 172.21.15.159:0/2966794391 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8011790 con 0x7f15f0147c80 2024-04-04T06:06:45.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.518+0000 7f15e7400640 1 -- 172.21.15.159:0/2966794391 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e8017070 con 0x7f15f0147c80 2024-04-04T06:06:45.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.518+0000 7f15f4e00640 1 -- 172.21.15.159:0/2966794391 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f01414a0 con 0x7f15f0147c80 2024-04-04T06:06:45.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.518+0000 7f15e7400640 1 -- 172.21.15.159:0/2966794391 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8011bb0 con 0x7f15f0147c80 2024-04-04T06:06:45.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.518+0000 7f15f4e00640 1 -- 172.21.15.159:0/2966794391 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f00b0da0 con 0x7f15f0147c80 2024-04-04T06:06:45.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.519+0000 7f15f4e00640 1 -- 172.21.15.159:0/2966794391 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f01419c0 con 0x7f15f0147c80 2024-04-04T06:06:45.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.520+0000 7f15e7400640 1 -- 172.21.15.159:0/2966794391 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f15e801c460 con 0x7f15f0147c80 2024-04-04T06:06:45.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.521+0000 7f15e7400640 1 --2- 172.21.15.159:0/2966794391 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f15cc06c8f0 0x7f15cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:45.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.521+0000 7f15e7400640 1 -- 172.21.15.159:0/2966794391 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f15e808b7e0 con 0x7f15f0147c80 2024-04-04T06:06:45.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.522+0000 7f15ef400640 1 --2- 172.21.15.159:0/2966794391 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f15cc06c8f0 0x7f15cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:45.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.523+0000 7f15ef400640 1 --2- 172.21.15.159:0/2966794391 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f15cc06c8f0 0x7f15cc06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f15e00012e0 tx=0x7f15e0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:45.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.526+0000 7f15e7400640 1 -- 172.21.15.159:0/2966794391 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f15e8021020 con 0x7f15f0147c80 2024-04-04T06:06:45.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.724+0000 7f15f4e00640 1 -- 172.21.15.159:0/2966794391 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f15f00d14e0 con 0x7f15f0147c80 2024-04-04T06:06:45.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.725+0000 7f15e7400640 1 -- 172.21.15.159:0/2966794391 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f15f013f110 con 0x7f15f0147c80 2024-04-04T06:06:45.719 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:06:45.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.727+0000 7f15f4e00640 1 -- 172.21.15.159:0/2966794391 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f15cc06c8f0 msgr2=0x7f15cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:45.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.727+0000 7f15f4e00640 1 --2- 172.21.15.159:0/2966794391 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f15cc06c8f0 0x7f15cc06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f15e00012e0 tx=0x7f15e0009040 comp rx=0 tx=0).stop 2024-04-04T06:06:45.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.727+0000 7f15f4e00640 1 -- 172.21.15.159:0/2966794391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15f0147c80 msgr2=0x7f15f00afaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:45.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.727+0000 7f15f4e00640 1 --2- 172.21.15.159:0/2966794391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15f0147c80 0x7f15f00afaf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f15e800c410 tx=0x7f15e800c8d0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.728+0000 7f15f4e00640 1 -- 172.21.15.159:0/2966794391 shutdown_connections 2024-04-04T06:06:45.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.728+0000 7f15f4e00640 1 --2- 172.21.15.159:0/2966794391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15f0147c80 0x7f15f00afaf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.728+0000 7f15f4e00640 1 --2- 172.21.15.159:0/2966794391 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f15cc06c8f0 0x7f15cc06edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.728+0000 7f15f4e00640 1 --2- 172.21.15.159:0/2966794391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15f00707e0 0x7f15f01420f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.728+0000 7f15f4e00640 1 --2- 172.21.15.159:0/2966794391 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f15f006bc80 0x7f15f013ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.728+0000 7f15f4e00640 1 -- 172.21.15.159:0/2966794391 >> 172.21.15.159:0/2966794391 conn(0x7f15f006e730 msgr2=0x7f15f00ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:45.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.728+0000 7f15f4e00640 1 -- 172.21.15.159:0/2966794391 shutdown_connections 2024-04-04T06:06:45.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.728+0000 7f15f4e00640 1 -- 172.21.15.159:0/2966794391 wait complete. 2024-04-04T06:06:45.740 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:06:45.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.952+0000 7fbcab200640 1 Processor -- start 2024-04-04T06:06:45.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.952+0000 7fbcab200640 1 -- start start 2024-04-04T06:06:45.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.954+0000 7fbcab200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fbcac0b69c0 0x7fbcac168f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:45.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.954+0000 7fbcab200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcac0d2710 con 0x7fbcac15e4c0 2024-04-04T06:06:45.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.954+0000 7fbcab200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcac0d29f0 con 0x7fbcac0d37b0 2024-04-04T06:06:45.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.954+0000 7fbcab200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fbcac15e8a0 con 0x7fbcac0b69c0 2024-04-04T06:06:45.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.954+0000 7fbca9e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fbcac15e4c0 0x7fbcac0b9bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:32892/0 (socket says 172.21.15.159:32892) 2024-04-04T06:06:45.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.954+0000 7fbca9e00640 1 -- 172.21.15.159:0/2724354605 learned_addr learned my addr 172.21.15.159:0/2724354605 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:45.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.954+0000 7fbcaa800640 1 --2- 172.21.15.159:0/2724354605 >> v2:172.21.15.159:3300/0 conn(0x7fbcac0b69c0 0x7fbcac168f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:45.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.955+0000 7fbcaa800640 1 -- 172.21.15.159:0/2724354605 >> v1:172.21.15.178:6789/0 conn(0x7fbcac0d37b0 legacy=0x7fbcac0b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:45.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.955+0000 7fbcaa800640 1 -- 172.21.15.159:0/2724354605 >> v1:172.21.15.159:6789/0 conn(0x7fbcac15e4c0 legacy=0x7fbcac0b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:45.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.955+0000 7fbca9400640 1 -- 172.21.15.159:0/2724354605 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 101372470 0 0) 0x7fbcac0d2710 con 0x7fbcac15e4c0 2024-04-04T06:06:45.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.955+0000 7fbcaa800640 1 -- 172.21.15.159:0/2724354605 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcac0d2cd0 con 0x7fbcac0b69c0 2024-04-04T06:06:45.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.956+0000 7fbcaa800640 1 --2- 172.21.15.159:0/2724354605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcac0b69c0 0x7fbcac168f70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009d90 tx=0x7fbca402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90db0237f19dfc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:45.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.956+0000 7fbca9400640 1 -- 172.21.15.159:0/2724354605 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4039070 con 0x7fbcac0b69c0 2024-04-04T06:06:45.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.956+0000 7fbca9400640 1 -- 172.21.15.159:0/2724354605 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca402cc60 con 0x7fbcac0b69c0 2024-04-04T06:06:45.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.956+0000 7fbca9400640 1 -- 172.21.15.159:0/2724354605 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4034660 con 0x7fbcac0b69c0 2024-04-04T06:06:45.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.957+0000 7fbcab200640 1 -- 172.21.15.159:0/2724354605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcac0b69c0 msgr2=0x7fbcac168f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:45.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.957+0000 7fbcab200640 1 --2- 172.21.15.159:0/2724354605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcac0b69c0 0x7fbcac168f70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009d90 tx=0x7fbca402beb0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.957+0000 7fbcab200640 1 -- 172.21.15.159:0/2724354605 shutdown_connections 2024-04-04T06:06:45.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.957+0000 7fbcab200640 1 --2- 172.21.15.159:0/2724354605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcac0b69c0 0x7fbcac168f70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.957+0000 7fbcab200640 1 -- 172.21.15.159:0/2724354605 >> 172.21.15.159:0/2724354605 conn(0x7fbcac0676a0 msgr2=0x7fbcac067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:45.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.957+0000 7fbcab200640 1 -- 172.21.15.159:0/2724354605 shutdown_connections 2024-04-04T06:06:45.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.957+0000 7fbcab200640 1 -- 172.21.15.159:0/2724354605 wait complete. 2024-04-04T06:06:45.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.959+0000 7fbcab200640 1 Processor -- start 2024-04-04T06:06:45.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.959+0000 7fbcab200640 1 -- start start 2024-04-04T06:06:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.960+0000 7fbcab200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcac0b69c0 0x7fbcac0b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.960+0000 7fbcab200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcac0d37b0 0x7fbcac0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.960+0000 7fbcab200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcac15e4c0 0x7fbcac0ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.960+0000 7fbcab200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbcac0d3320 con 0x7fbcac0b69c0 2024-04-04T06:06:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.960+0000 7fbcab200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbcac0d4040 con 0x7fbcac0d37b0 2024-04-04T06:06:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.960+0000 7fbcab200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbcac0d2350 con 0x7fbcac15e4c0 2024-04-04T06:06:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.960+0000 7fbca9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcac0b69c0 0x7fbcac0b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.960+0000 7fbca9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcac0b69c0 0x7fbcac0b51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41096/0 (socket says 172.21.15.159:41096) 2024-04-04T06:06:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.960+0000 7fbca9e00640 1 -- 172.21.15.159:0/804949036 learned_addr learned my addr 172.21.15.159:0/804949036 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.960+0000 7fbcaa800640 1 --2- 172.21.15.159:0/804949036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcac15e4c0 0x7fbcac0ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.960+0000 7fbca3e00640 1 --2- 172.21.15.159:0/804949036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcac0d37b0 0x7fbcac0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.960+0000 7fbca9e00640 1 -- 172.21.15.159:0/804949036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcac15e4c0 msgr2=0x7fbcac0ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.960+0000 7fbca9e00640 1 --2- 172.21.15.159:0/804949036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcac15e4c0 0x7fbcac0ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.960+0000 7fbca9e00640 1 -- 172.21.15.159:0/804949036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcac0d37b0 msgr2=0x7fbcac0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:45.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.960+0000 7fbca9e00640 1 --2- 172.21.15.159:0/804949036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcac0d37b0 0x7fbcac0cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:45.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.960+0000 7fbca9e00640 1 -- 172.21.15.159:0/804949036 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcac1f4220 con 0x7fbcac0b69c0 2024-04-04T06:06:45.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.961+0000 7fbca9e00640 1 --2- 172.21.15.159:0/804949036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcac0b69c0 0x7fbcac0b51a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00c4d0 tx=0x7fbc9c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:45.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.961+0000 7fbca2000640 1 -- 172.21.15.159:0/804949036 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c018070 con 0x7fbcac0b69c0 2024-04-04T06:06:45.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.961+0000 7fbca2000640 1 -- 172.21.15.159:0/804949036 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc9c007500 con 0x7fbcac0b69c0 2024-04-04T06:06:45.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.961+0000 7fbca2000640 1 -- 172.21.15.159:0/804949036 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c004a20 con 0x7fbcac0b69c0 2024-04-04T06:06:45.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.961+0000 7fbcab200640 1 -- 172.21.15.159:0/804949036 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcac1f4500 con 0x7fbcac0b69c0 2024-04-04T06:06:45.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.961+0000 7fbcab200640 1 -- 172.21.15.159:0/804949036 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcac1f4ac0 con 0x7fbcac0b69c0 2024-04-04T06:06:45.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.962+0000 7fbcab200640 1 -- 172.21.15.159:0/804949036 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcac075940 con 0x7fbcac0b69c0 2024-04-04T06:06:45.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.963+0000 7fbca2000640 1 -- 172.21.15.159:0/804949036 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc9c01b070 con 0x7fbcac0b69c0 2024-04-04T06:06:45.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.964+0000 7fbca2000640 1 --2- 172.21.15.159:0/804949036 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc8806c8f0 0x7fbc8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:45.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.965+0000 7fbca2000640 1 -- 172.21.15.159:0/804949036 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7fbc9c08afa0 con 0x7fbcac0b69c0 2024-04-04T06:06:45.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.965+0000 7fbca3e00640 1 --2- 172.21.15.159:0/804949036 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc8806c8f0 0x7fbc8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:45.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.966+0000 7fbca3e00640 1 --2- 172.21.15.159:0/804949036 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc8806c8f0 0x7fbc8806edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0cae20 tx=0x7fbc94009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:45.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:45.969+0000 7fbca2000640 1 -- 172.21.15.159:0/804949036 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbc9c059280 con 0x7fbcac0b69c0 2024-04-04T06:06:46.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.168+0000 7fbcab200640 1 -- 172.21.15.159:0/804949036 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbcac0b4240 con 0x7fbcac0b69c0 2024-04-04T06:06:46.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.169+0000 7fbca2000640 1 -- 172.21.15.159:0/804949036 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fbc9c058b80 con 0x7fbcac0b69c0 2024-04-04T06:06:46.164 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:06:46.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.172+0000 7fbcab200640 1 -- 172.21.15.159:0/804949036 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc8806c8f0 msgr2=0x7fbc8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:46.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.172+0000 7fbcab200640 1 --2- 172.21.15.159:0/804949036 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc8806c8f0 0x7fbc8806edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0cae20 tx=0x7fbc94009040 comp rx=0 tx=0).stop 2024-04-04T06:06:46.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.172+0000 7fbcab200640 1 -- 172.21.15.159:0/804949036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcac0b69c0 msgr2=0x7fbcac0b51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:46.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.172+0000 7fbcab200640 1 --2- 172.21.15.159:0/804949036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcac0b69c0 0x7fbcac0b51a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00c4d0 tx=0x7fbc9c00c990 comp rx=0 tx=0).stop 2024-04-04T06:06:46.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.172+0000 7fbcab200640 1 -- 172.21.15.159:0/804949036 shutdown_connections 2024-04-04T06:06:46.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.172+0000 7fbcab200640 1 --2- 172.21.15.159:0/804949036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbcac15e4c0 0x7fbcac0ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:46.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.172+0000 7fbcab200640 1 --2- 172.21.15.159:0/804949036 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc8806c8f0 0x7fbc8806edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:46.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.172+0000 7fbcab200640 1 --2- 172.21.15.159:0/804949036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbcac0d37b0 0x7fbcac0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:46.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.172+0000 7fbcab200640 1 --2- 172.21.15.159:0/804949036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcac0b69c0 0x7fbcac0b51a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:46.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.172+0000 7fbcab200640 1 -- 172.21.15.159:0/804949036 >> 172.21.15.159:0/804949036 conn(0x7fbcac0676a0 msgr2=0x7fbcac0b4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:46.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.172+0000 7fbcab200640 1 -- 172.21.15.159:0/804949036 shutdown_connections 2024-04-04T06:06:46.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.172+0000 7fbcab200640 1 -- 172.21.15.159:0/804949036 wait complete. 2024-04-04T06:06:46.186 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-04T06:06:46.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.396+0000 7fdea1000640 1 Processor -- start 2024-04-04T06:06:46.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.396+0000 7fdea1000640 1 -- start start 2024-04-04T06:06:46.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.397+0000 7fdea1000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fde9c0d0280 0x7fde9c0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:46.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.397+0000 7fdea1000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fde9c141c10 0x7fde9c071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:46.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.397+0000 7fdea1000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde9c15ec80 con 0x7fde9c071eb0 2024-04-04T06:06:46.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.397+0000 7fdea1000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fde9c0d0ba0 con 0x7fde9c141c10 2024-04-04T06:06:46.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.397+0000 7fdea1000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fde9c15ef20 con 0x7fde9c0d0280 2024-04-04T06:06:46.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.398+0000 7fde91400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fde9c141c10 0x7fde9c071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:46.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.398+0000 7fde91400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fde9c141c10 0x7fde9c071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41108/0 (socket says 172.21.15.159:41108) 2024-04-04T06:06:46.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.398+0000 7fde91400640 1 -- 172.21.15.159:0/3554006919 learned_addr learned my addr 172.21.15.159:0/3554006919 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:46.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.398+0000 7fde99e00640 1 --2- 172.21.15.159:0/3554006919 >> v2:172.21.15.178:3300/0 conn(0x7fde9c0d0280 0x7fde9c0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:46.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.398+0000 7fde91400640 1 -- 172.21.15.159:0/3554006919 >> v2:172.21.15.178:3300/0 conn(0x7fde9c0d0280 msgr2=0x7fde9c0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:46.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.398+0000 7fde91400640 1 --2- 172.21.15.159:0/3554006919 >> v2:172.21.15.178:3300/0 conn(0x7fde9c0d0280 0x7fde9c0d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:46.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.398+0000 7fde91400640 1 -- 172.21.15.159:0/3554006919 >> v1:172.21.15.159:6789/0 conn(0x7fde9c071eb0 legacy=0x7fde9c06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:46.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.398+0000 7fde91400640 1 -- 172.21.15.159:0/3554006919 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c15f200 con 0x7fde9c141c10 2024-04-04T06:06:46.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.398+0000 7fde99e00640 1 --2- 172.21.15.159:0/3554006919 >> v2:172.21.15.178:3300/0 conn(0x7fde9c0d0280 0x7fde9c0d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:46.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.399+0000 7fde91400640 1 --2- 172.21.15.159:0/3554006919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde9c141c10 0x7fde9c071970 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0d1050 tx=0x7fde8402b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e560047673c9ad server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:46.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.400+0000 7fde99400640 1 -- 172.21.15.159:0/3554006919 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84034070 con 0x7fde9c141c10 2024-04-04T06:06:46.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.400+0000 7fde99400640 1 -- 172.21.15.159:0/3554006919 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde84030870 con 0x7fde9c141c10 2024-04-04T06:06:46.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.400+0000 7fde99400640 1 -- 172.21.15.159:0/3554006919 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84030c40 con 0x7fde9c141c10 2024-04-04T06:06:46.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.400+0000 7fdea1000640 1 -- 172.21.15.159:0/3554006919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde9c141c10 msgr2=0x7fde9c071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:46.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.400+0000 7fdea1000640 1 --2- 172.21.15.159:0/3554006919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde9c141c10 0x7fde9c071970 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0d1050 tx=0x7fde8402b9b0 comp rx=0 tx=0).stop 2024-04-04T06:06:46.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.401+0000 7fdea1000640 1 -- 172.21.15.159:0/3554006919 shutdown_connections 2024-04-04T06:06:46.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.401+0000 7fdea1000640 1 --2- 172.21.15.159:0/3554006919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde9c141c10 0x7fde9c071970 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:46.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.401+0000 7fdea1000640 1 --2- 172.21.15.159:0/3554006919 >> v2:172.21.15.178:3300/0 conn(0x7fde9c0d0280 0x7fde9c0d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:46.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.401+0000 7fdea1000640 1 -- 172.21.15.159:0/3554006919 >> 172.21.15.159:0/3554006919 conn(0x7fde9c06b750 msgr2=0x7fde9c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:46.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.401+0000 7fdea1000640 1 -- 172.21.15.159:0/3554006919 shutdown_connections 2024-04-04T06:06:46.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.401+0000 7fdea1000640 1 -- 172.21.15.159:0/3554006919 wait complete. 2024-04-04T06:06:46.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.402+0000 7fdea1000640 1 Processor -- start 2024-04-04T06:06:46.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.403+0000 7fdea1000640 1 -- start start 2024-04-04T06:06:46.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.403+0000 7fdea1000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde9c071eb0 0x7fde9c0ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:46.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.403+0000 7fdea1000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde9c0d0280 0x7fde9c0ad550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:46.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.404+0000 7fdea1000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde9c141c10 0x7fde9c0ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:46.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.404+0000 7fdea1000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c158e90 con 0x7fde9c0d0280 2024-04-04T06:06:46.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.404+0000 7fdea1000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c15f7c0 con 0x7fde9c141c10 2024-04-04T06:06:46.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.404+0000 7fdea1000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fde9c0af2e0 con 0x7fde9c071eb0 2024-04-04T06:06:46.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.404+0000 7fde91400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde9c0d0280 0x7fde9c0ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:46.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.404+0000 7fde91400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde9c0d0280 0x7fde9c0ad550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41112/0 (socket says 172.21.15.159:41112) 2024-04-04T06:06:46.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.404+0000 7fde91400640 1 -- 172.21.15.159:0/197463429 learned_addr learned my addr 172.21.15.159:0/197463429 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:46.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.404+0000 7fde99e00640 1 --2- 172.21.15.159:0/197463429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde9c071eb0 0x7fde9c0ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:46.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.404+0000 7fde9a800640 1 --2- 172.21.15.159:0/197463429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde9c141c10 0x7fde9c0ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:46.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.404+0000 7fde91400640 1 -- 172.21.15.159:0/197463429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde9c071eb0 msgr2=0x7fde9c0ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:46.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.404+0000 7fde91400640 1 --2- 172.21.15.159:0/197463429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde9c071eb0 0x7fde9c0ad010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:46.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.404+0000 7fde91400640 1 -- 172.21.15.159:0/197463429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde9c141c10 msgr2=0x7fde9c0ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:46.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.404+0000 7fde91400640 1 --2- 172.21.15.159:0/197463429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde9c141c10 0x7fde9c0ae230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:46.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.404+0000 7fde91400640 1 -- 172.21.15.159:0/197463429 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c0af5c0 con 0x7fde9c0d0280 2024-04-04T06:06:46.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.405+0000 7fde91400640 1 --2- 172.21.15.159:0/197463429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde9c0d0280 0x7fde9c0ad550 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fde8402b9f0 tx=0x7fde840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:46.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.405+0000 7fde92a00640 1 -- 172.21.15.159:0/197463429 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84034040 con 0x7fde9c0d0280 2024-04-04T06:06:46.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.405+0000 7fde92a00640 1 -- 172.21.15.159:0/197463429 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde840433f0 con 0x7fde9c0d0280 2024-04-04T06:06:46.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.405+0000 7fde92a00640 1 -- 172.21.15.159:0/197463429 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde840437e0 con 0x7fde9c0d0280 2024-04-04T06:06:46.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.405+0000 7fdea1000640 1 -- 172.21.15.159:0/197463429 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9c14fc50 con 0x7fde9c0d0280 2024-04-04T06:06:46.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.405+0000 7fdea1000640 1 -- 172.21.15.159:0/197463429 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde9c150210 con 0x7fde9c0d0280 2024-04-04T06:06:46.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.406+0000 7fdea1000640 1 -- 172.21.15.159:0/197463429 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde9c0685c0 con 0x7fde9c0d0280 2024-04-04T06:06:46.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.407+0000 7fde92a00640 1 -- 172.21.15.159:0/197463429 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde84037070 con 0x7fde9c0d0280 2024-04-04T06:06:46.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.408+0000 7fde92a00640 1 --2- 172.21.15.159:0/197463429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde7806c8f0 0x7fde7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:46.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.408+0000 7fde92a00640 1 -- 172.21.15.159:0/197463429 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7fde840adfc0 con 0x7fde9c0d0280 2024-04-04T06:06:46.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.409+0000 7fde99e00640 1 --2- 172.21.15.159:0/197463429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde7806c8f0 0x7fde7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:46.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.410+0000 7fde99e00640 1 --2- 172.21.15.159:0/197463429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde7806c8f0 0x7fde7806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00a990 tx=0x7fde8c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:46.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.413+0000 7fde92a00640 1 -- 172.21.15.159:0/197463429 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fde8403e070 con 0x7fde9c0d0280 2024-04-04T06:06:46.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.625+0000 7fdea1000640 1 -- 172.21.15.159:0/197463429 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fde9c156810 con 0x7fde9c0d0280 2024-04-04T06:06:46.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.626+0000 7fde92a00640 1 -- 172.21.15.159:0/197463429 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fde8407c2a0 con 0x7fde9c0d0280 2024-04-04T06:06:46.621 INFO:teuthology.orchestra.run.smithi159.stderr:listed 0 entries 2024-04-04T06:06:46.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.629+0000 7fdea1000640 1 -- 172.21.15.159:0/197463429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde7806c8f0 msgr2=0x7fde7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:46.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.629+0000 7fdea1000640 1 --2- 172.21.15.159:0/197463429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde7806c8f0 0x7fde7806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00a990 tx=0x7fde8c008040 comp rx=0 tx=0).stop 2024-04-04T06:06:46.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.629+0000 7fdea1000640 1 -- 172.21.15.159:0/197463429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde9c0d0280 msgr2=0x7fde9c0ad550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:46.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.629+0000 7fdea1000640 1 --2- 172.21.15.159:0/197463429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde9c0d0280 0x7fde9c0ad550 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fde8402b9f0 tx=0x7fde840046c0 comp rx=0 tx=0).stop 2024-04-04T06:06:46.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.629+0000 7fdea1000640 1 -- 172.21.15.159:0/197463429 shutdown_connections 2024-04-04T06:06:46.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.629+0000 7fdea1000640 1 --2- 172.21.15.159:0/197463429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde7806c8f0 0x7fde7806edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:46.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.629+0000 7fdea1000640 1 --2- 172.21.15.159:0/197463429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde9c141c10 0x7fde9c0ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:46.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.629+0000 7fdea1000640 1 --2- 172.21.15.159:0/197463429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde9c0d0280 0x7fde9c0ad550 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:46.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.629+0000 7fdea1000640 1 --2- 172.21.15.159:0/197463429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde9c071eb0 0x7fde9c0ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:46.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.629+0000 7fdea1000640 1 -- 172.21.15.159:0/197463429 >> 172.21.15.159:0/197463429 conn(0x7fde9c06b750 msgr2=0x7fde9c0b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:46.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.629+0000 7fdea1000640 1 -- 172.21.15.159:0/197463429 shutdown_connections 2024-04-04T06:06:46.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:46.629+0000 7fdea1000640 1 -- 172.21.15.159:0/197463429 wait complete. 2024-04-04T06:06:46.642 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-04T06:06:46.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:46.645 DEBUG:teuthology.orchestra.run.smithi178:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-04T06:06:46.761 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi178.front.sepia.ceph.com 2024-04-04T06:06:46.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:46.761 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-04T06:06:46.820 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:06:46.820 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:06:46.877 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:06:46.877 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:06:46.930 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:06:46.930 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:06:46.977 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:06:47.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.239+0000 7f9bb6400640 1 Processor -- start 2024-04-04T06:06:47.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.239+0000 7f9bb6400640 1 -- start start 2024-04-04T06:06:47.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.241+0000 7f9bb6400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9bb015c390 0x7f9bb0073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:47.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.241+0000 7f9bb6400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9bb00af830 0x7f9bb0160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:47.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.241+0000 7f9bb6400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bb015fe80 con 0x7f9bb0160b80 2024-04-04T06:06:47.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.241+0000 7f9bb6400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f9bb015c770 con 0x7f9bb015c390 2024-04-04T06:06:47.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.241+0000 7f9bb6400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9bb00ad120 con 0x7f9bb00af830 2024-04-04T06:06:47.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.241+0000 7f9baf400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9bb015c390 0x7f9bb0073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:47.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.241+0000 7f9bafe00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f9bb0160b80 0x7f9bb00d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:32912/0 (socket says 172.21.15.159:32912) 2024-04-04T06:06:47.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.241+0000 7f9bafe00640 1 -- 172.21.15.159:0/2918656676 learned_addr learned my addr 172.21.15.159:0/2918656676 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:47.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.241+0000 7f9baf400640 1 -- 172.21.15.159:0/2918656676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9bb00af830 msgr2=0x7f9bb0160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:47.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.241+0000 7f9baf400640 1 --2- 172.21.15.159:0/2918656676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9bb00af830 0x7f9bb0160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:47.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.241+0000 7f9baf400640 1 -- 172.21.15.159:0/2918656676 >> v1:172.21.15.159:6789/0 conn(0x7f9bb0160b80 legacy=0x7f9bb00d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:47.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.241+0000 7f9baf400640 1 -- 172.21.15.159:0/2918656676 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb00ad400 con 0x7f9bb015c390 2024-04-04T06:06:47.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.243+0000 7f9baf400640 1 --2- 172.21.15.159:0/2918656676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bb015c390 0x7f9bb0073790 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9ba00097a0 tx=0x7f9ba002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725c9fb7f48310bd server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:47.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.243+0000 7f9bae000640 1 -- 172.21.15.159:0/2918656676 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0035020 con 0x7f9bb015c390 2024-04-04T06:06:47.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.243+0000 7f9bae000640 1 -- 172.21.15.159:0/2918656676 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba0031810 con 0x7f9bb015c390 2024-04-04T06:06:47.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.243+0000 7f9bae000640 1 -- 172.21.15.159:0/2918656676 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0031be0 con 0x7f9bb015c390 2024-04-04T06:06:47.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.243+0000 7f9bb6400640 1 -- 172.21.15.159:0/2918656676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bb015c390 msgr2=0x7f9bb0073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:47.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.243+0000 7f9bb6400640 1 --2- 172.21.15.159:0/2918656676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bb015c390 0x7f9bb0073790 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9ba00097a0 tx=0x7f9ba002b8b0 comp rx=0 tx=0).stop 2024-04-04T06:06:47.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.244+0000 7f9bb6400640 1 -- 172.21.15.159:0/2918656676 shutdown_connections 2024-04-04T06:06:47.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.244+0000 7f9bb6400640 1 --2- 172.21.15.159:0/2918656676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9bb00af830 0x7f9bb0160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:47.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.244+0000 7f9bb6400640 1 --2- 172.21.15.159:0/2918656676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bb015c390 0x7f9bb0073790 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:47.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.244+0000 7f9bb6400640 1 -- 172.21.15.159:0/2918656676 >> 172.21.15.159:0/2918656676 conn(0x7f9bb006b990 msgr2=0x7f9bb006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:47.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.244+0000 7f9bb6400640 1 -- 172.21.15.159:0/2918656676 shutdown_connections 2024-04-04T06:06:47.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.244+0000 7f9bb6400640 1 -- 172.21.15.159:0/2918656676 wait complete. 2024-04-04T06:06:47.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.245+0000 7f9bb6400640 1 Processor -- start 2024-04-04T06:06:47.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.246+0000 7f9bb6400640 1 -- start start 2024-04-04T06:06:47.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.246+0000 7f9bb6400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bb00af830 0x7f9bb0099900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:47.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.246+0000 7f9bb6400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bb015c390 0x7f9bb015a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:47.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.246+0000 7f9bb6400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9bb0160b80 0x7f9bb015acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:47.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.246+0000 7f9bb6400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb00cf980 con 0x7f9bb015c390 2024-04-04T06:06:47.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.246+0000 7f9bb6400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb0160120 con 0x7f9bb00af830 2024-04-04T06:06:47.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.246+0000 7f9bb6400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9bb00d0d60 con 0x7f9bb0160b80 2024-04-04T06:06:47.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.247+0000 7f9baea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bb015c390 0x7f9bb015a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:47.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.247+0000 7f9baea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bb015c390 0x7f9bb015a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41136/0 (socket says 172.21.15.159:41136) 2024-04-04T06:06:47.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.247+0000 7f9baea00640 1 -- 172.21.15.159:0/1528116873 learned_addr learned my addr 172.21.15.159:0/1528116873 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:47.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.247+0000 7f9baf400640 1 --2- 172.21.15.159:0/1528116873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bb00af830 0x7f9bb0099900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:47.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.247+0000 7f9bafe00640 1 --2- 172.21.15.159:0/1528116873 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9bb0160b80 0x7f9bb015acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:47.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.247+0000 7f9baea00640 1 -- 172.21.15.159:0/1528116873 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9bb0160b80 msgr2=0x7f9bb015acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:47.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.247+0000 7f9baea00640 1 --2- 172.21.15.159:0/1528116873 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9bb0160b80 0x7f9bb015acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:47.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.247+0000 7f9baea00640 1 -- 172.21.15.159:0/1528116873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bb00af830 msgr2=0x7f9bb0099900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:47.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.247+0000 7f9baea00640 1 --2- 172.21.15.159:0/1528116873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bb00af830 0x7f9bb0099900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:47.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.247+0000 7f9baea00640 1 -- 172.21.15.159:0/1528116873 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb01e6b10 con 0x7f9bb015c390 2024-04-04T06:06:47.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.247+0000 7f9baea00640 1 --2- 172.21.15.159:0/1528116873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bb015c390 0x7f9bb015a770 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00b4d0 tx=0x7f9b9c00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:47.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.248+0000 7f9ba7400640 1 -- 172.21.15.159:0/1528116873 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c014070 con 0x7f9bb015c390 2024-04-04T06:06:47.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.248+0000 7f9ba7400640 1 -- 172.21.15.159:0/1528116873 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b9c010870 con 0x7f9bb015c390 2024-04-04T06:06:47.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.248+0000 7f9ba7400640 1 -- 172.21.15.159:0/1528116873 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c010c60 con 0x7f9bb015c390 2024-04-04T06:06:47.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.248+0000 7f9bb6400640 1 -- 172.21.15.159:0/1528116873 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb01e6df0 con 0x7f9bb015c390 2024-04-04T06:06:47.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.248+0000 7f9bb6400640 1 -- 172.21.15.159:0/1528116873 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb01e73b0 con 0x7f9bb015c390 2024-04-04T06:06:47.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.249+0000 7f9bb6400640 1 -- 172.21.15.159:0/1528116873 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bb00685c0 con 0x7f9bb015c390 2024-04-04T06:06:47.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.250+0000 7f9ba7400640 1 -- 172.21.15.159:0/1528116873 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b9c017070 con 0x7f9bb015c390 2024-04-04T06:06:47.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.251+0000 7f9ba7400640 1 --2- 172.21.15.159:0/1528116873 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b9006c8f0 0x7f9b9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:47.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.251+0000 7f9ba7400640 1 -- 172.21.15.159:0/1528116873 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f9b9c08ba10 con 0x7f9bb015c390 2024-04-04T06:06:47.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.251+0000 7f9baf400640 1 --2- 172.21.15.159:0/1528116873 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b9006c8f0 0x7f9b9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:47.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.253+0000 7f9baf400640 1 --2- 172.21.15.159:0/1528116873 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b9006c8f0 0x7f9b9006edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0036000 tx=0x7f9ba0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:47.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.256+0000 7f9ba7400640 1 -- 172.21.15.159:0/1528116873 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9b9c059d80 con 0x7f9bb015c390 2024-04-04T06:06:47.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.460+0000 7f9bb6400640 1 -- 172.21.15.159:0/1528116873 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9bb006f1b0 con 0x7f9bb015c390 2024-04-04T06:06:47.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.463+0000 7f9ba7400640 1 -- 172.21.15.159:0/1528116873 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16958 (secure 0 0 0) 0x7f9b9c059680 con 0x7f9bb015c390 2024-04-04T06:06:47.458 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:47.458 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":23,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:06:33.796979+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:05:51.297293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-04T06:05:51.770792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":0,"to":2}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:06:47.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.466+0000 7f9bb6400640 1 -- 172.21.15.159:0/1528116873 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b9006c8f0 msgr2=0x7f9b9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:47.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.466+0000 7f9bb6400640 1 --2- 172.21.15.159:0/1528116873 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b9006c8f0 0x7f9b9006edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0036000 tx=0x7f9ba0008030 comp rx=0 tx=0).stop 2024-04-04T06:06:47.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.466+0000 7f9bb6400640 1 -- 172.21.15.159:0/1528116873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bb015c390 msgr2=0x7f9bb015a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:47.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.466+0000 7f9bb6400640 1 --2- 172.21.15.159:0/1528116873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bb015c390 0x7f9bb015a770 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00b4d0 tx=0x7f9b9c00b990 comp rx=0 tx=0).stop 2024-04-04T06:06:47.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.466+0000 7f9bb6400640 1 -- 172.21.15.159:0/1528116873 shutdown_connections 2024-04-04T06:06:47.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.466+0000 7f9bb6400640 1 --2- 172.21.15.159:0/1528116873 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9bb0160b80 0x7f9bb015acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:47.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.466+0000 7f9bb6400640 1 --2- 172.21.15.159:0/1528116873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bb015c390 0x7f9bb015a770 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:47.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.466+0000 7f9bb6400640 1 --2- 172.21.15.159:0/1528116873 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b9006c8f0 0x7f9b9006edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:47.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.466+0000 7f9bb6400640 1 --2- 172.21.15.159:0/1528116873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bb00af830 0x7f9bb0099900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:47.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.466+0000 7f9bb6400640 1 -- 172.21.15.159:0/1528116873 >> 172.21.15.159:0/1528116873 conn(0x7f9bb006b990 msgr2=0x7f9bb00b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:47.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.466+0000 7f9bb6400640 1 -- 172.21.15.159:0/1528116873 shutdown_connections 2024-04-04T06:06:47.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.466+0000 7f9bb6400640 1 -- 172.21.15.159:0/1528116873 wait complete. 2024-04-04T06:06:47.479 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:05:51.297293+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:05:51.770792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:06:47.479 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:06:47.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.697+0000 7f5409a00640 1 Processor -- start 2024-04-04T06:06:47.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.697+0000 7f5409a00640 1 -- start start 2024-04-04T06:06:47.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.699+0000 7f5409a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5404153580 0x7f54040d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:47.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.699+0000 7f5409a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f54040af4b0 0x7f54041600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:47.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.699+0000 7f5409a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54040afab0 con 0x7f54040d07c0 2024-04-04T06:06:47.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.699+0000 7f5409a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f54040d0c00 con 0x7f54040af4b0 2024-04-04T06:06:47.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.699+0000 7f5409a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f540415cc80 con 0x7f5404153580 2024-04-04T06:06:47.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.700+0000 7f5402a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5404153580 0x7f54040d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:47.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.700+0000 7f5402000640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f54040d07c0 0x7f54040aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.159:44740/0 (socket says 172.21.15.159:44740) 2024-04-04T06:06:47.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.700+0000 7f5402000640 1 -- 172.21.15.159:0/4108332425 learned_addr learned my addr 172.21.15.159:0/4108332425 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:47.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.700+0000 7f5403400640 1 --2- 172.21.15.159:0/4108332425 >> v2:172.21.15.178:3300/0 conn(0x7f54040af4b0 0x7f54041600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:47.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.701+0000 7f5401600640 1 -- 172.21.15.159:0/4108332425 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3912820573 0 0) 0x7f54040afab0 con 0x7f54040d07c0 2024-04-04T06:06:47.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.701+0000 7f5401600640 1 -- 172.21.15.159:0/4108332425 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53e8003520 con 0x7f54040d07c0 2024-04-04T06:06:47.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.701+0000 7f5402a00640 1 -- 172.21.15.159:0/4108332425 >> v2:172.21.15.178:3300/0 conn(0x7f54040af4b0 msgr2=0x7f54041600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:47.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.701+0000 7f5402a00640 1 --2- 172.21.15.159:0/4108332425 >> v2:172.21.15.178:3300/0 conn(0x7f54040af4b0 0x7f54041600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:47.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.701+0000 7f5402a00640 1 -- 172.21.15.159:0/4108332425 >> v1:172.21.15.178:6789/0 conn(0x7f54040d07c0 legacy=0x7f54040aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:47.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.701+0000 7f5402a00640 1 -- 172.21.15.159:0/4108332425 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540415da00 con 0x7f5404153580 2024-04-04T06:06:47.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.701+0000 7f5403400640 1 --2- 172.21.15.159:0/4108332425 >> v2:172.21.15.178:3300/0 conn(0x7f54040af4b0 0x7f54041600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:06:47.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.702+0000 7f5402a00640 1 --2- 172.21.15.159:0/4108332425 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5404153580 0x7f54040d0280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0097a0 tx=0x7f53ec02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe7aed4576f6fdc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:47.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.702+0000 7f5401600640 1 -- 172.21.15.159:0/4108332425 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec0315a0 con 0x7f5404153580 2024-04-04T06:06:47.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.702+0000 7f5401600640 1 -- 172.21.15.159:0/4108332425 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53ec037070 con 0x7f5404153580 2024-04-04T06:06:47.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.702+0000 7f5401600640 1 -- 172.21.15.159:0/4108332425 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec031940 con 0x7f5404153580 2024-04-04T06:06:47.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.703+0000 7f5409a00640 1 -- 172.21.15.159:0/4108332425 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5404153580 msgr2=0x7f54040d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:47.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.703+0000 7f5409a00640 1 --2- 172.21.15.159:0/4108332425 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5404153580 0x7f54040d0280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0097a0 tx=0x7f53ec02b840 comp rx=0 tx=0).stop 2024-04-04T06:06:47.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.703+0000 7f5409a00640 1 -- 172.21.15.159:0/4108332425 shutdown_connections 2024-04-04T06:06:47.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.703+0000 7f5409a00640 1 --2- 172.21.15.159:0/4108332425 >> v2:172.21.15.178:3300/0 conn(0x7f54040af4b0 0x7f54041600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:47.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.703+0000 7f5409a00640 1 --2- 172.21.15.159:0/4108332425 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5404153580 0x7f54040d0280 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:47.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.703+0000 7f5409a00640 1 -- 172.21.15.159:0/4108332425 >> 172.21.15.159:0/4108332425 conn(0x7f540406b750 msgr2=0x7f540406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:47.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.703+0000 7f5409a00640 1 -- 172.21.15.159:0/4108332425 shutdown_connections 2024-04-04T06:06:47.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.703+0000 7f5409a00640 1 -- 172.21.15.159:0/4108332425 wait complete. 2024-04-04T06:06:47.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.705+0000 7f5409a00640 1 Processor -- start 2024-04-04T06:06:47.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.705+0000 7f5409a00640 1 -- start start 2024-04-04T06:06:47.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.706+0000 7f5409a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54040af4b0 0x7f5404072dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:47.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.706+0000 7f5409a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54040d07c0 0x7f54040b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:47.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.706+0000 7f5409a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5404153580 0x7f54040cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:47.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.706+0000 7f5409a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f540415d4e0 con 0x7f54040d07c0 2024-04-04T06:06:47.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.706+0000 7f5409a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f540415d200 con 0x7f54040af4b0 2024-04-04T06:06:47.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.706+0000 7f5409a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5404153960 con 0x7f5404153580 2024-04-04T06:06:47.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.706+0000 7f5402000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54040d07c0 0x7f54040b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:47.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.706+0000 7f5402000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54040d07c0 0x7f54040b01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41150/0 (socket says 172.21.15.159:41150) 2024-04-04T06:06:47.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.706+0000 7f5402000640 1 -- 172.21.15.159:0/2882668994 learned_addr learned my addr 172.21.15.159:0/2882668994 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:47.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.707+0000 7f5402a00640 1 --2- 172.21.15.159:0/2882668994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54040af4b0 0x7f5404072dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:47.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.707+0000 7f5403400640 1 --2- 172.21.15.159:0/2882668994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5404153580 0x7f54040cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:47.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.707+0000 7f5402000640 1 -- 172.21.15.159:0/2882668994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5404153580 msgr2=0x7f54040cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:47.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.707+0000 7f5402000640 1 --2- 172.21.15.159:0/2882668994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5404153580 0x7f54040cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:47.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.707+0000 7f5402000640 1 -- 172.21.15.159:0/2882668994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54040af4b0 msgr2=0x7f5404072dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:47.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.707+0000 7f5402000640 1 --2- 172.21.15.159:0/2882668994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54040af4b0 0x7f5404072dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:47.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.707+0000 7f5402000640 1 -- 172.21.15.159:0/2882668994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54040b0910 con 0x7f54040d07c0 2024-04-04T06:06:47.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.707+0000 7f5402000640 1 --2- 172.21.15.159:0/2882668994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54040d07c0 0x7f54040b01e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f53f400cad0 tx=0x7f53f400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:47.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.707+0000 7f53faa00640 1 -- 172.21.15.159:0/2882668994 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4017070 con 0x7f54040d07c0 2024-04-04T06:06:47.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.707+0000 7f53faa00640 1 -- 172.21.15.159:0/2882668994 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53f400ac30 con 0x7f54040d07c0 2024-04-04T06:06:47.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.707+0000 7f53faa00640 1 -- 172.21.15.159:0/2882668994 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4012400 con 0x7f54040d07c0 2024-04-04T06:06:47.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.707+0000 7f5409a00640 1 -- 172.21.15.159:0/2882668994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54040b0bf0 con 0x7f54040d07c0 2024-04-04T06:06:47.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.708+0000 7f5409a00640 1 -- 172.21.15.159:0/2882668994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54040b11b0 con 0x7f54040d07c0 2024-04-04T06:06:47.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.709+0000 7f5409a00640 1 -- 172.21.15.159:0/2882668994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54040ae680 con 0x7f54040d07c0 2024-04-04T06:06:47.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.710+0000 7f53faa00640 1 -- 172.21.15.159:0/2882668994 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53f401a070 con 0x7f54040d07c0 2024-04-04T06:06:47.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.711+0000 7f53faa00640 1 --2- 172.21.15.159:0/2882668994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f53e006c920 0x7f53e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:47.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.711+0000 7f53faa00640 1 -- 172.21.15.159:0/2882668994 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f53f408ba20 con 0x7f54040d07c0 2024-04-04T06:06:47.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.712+0000 7f5402a00640 1 --2- 172.21.15.159:0/2882668994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f53e006c920 0x7f53e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:47.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.713+0000 7f5402a00640 1 --2- 172.21.15.159:0/2882668994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f53e006c920 0x7f53e006ede0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f53ec036000 tx=0x7f53ec03d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:47.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.716+0000 7f53faa00640 1 -- 172.21.15.159:0/2882668994 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f54040ae680 con 0x7f54040d07c0 2024-04-04T06:06:47.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.919+0000 7f5409a00640 1 -- 172.21.15.159:0/2882668994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f54040685c0 con 0x7f54040d07c0 2024-04-04T06:06:47.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.920+0000 7f53faa00640 1 -- 172.21.15.159:0/2882668994 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f53f4059d10 con 0x7f54040d07c0 2024-04-04T06:06:47.914 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:06:47.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.922+0000 7f5409a00640 1 -- 172.21.15.159:0/2882668994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f53e006c920 msgr2=0x7f53e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:47.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.922+0000 7f5409a00640 1 --2- 172.21.15.159:0/2882668994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f53e006c920 0x7f53e006ede0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f53ec036000 tx=0x7f53ec03d040 comp rx=0 tx=0).stop 2024-04-04T06:06:47.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.922+0000 7f5409a00640 1 -- 172.21.15.159:0/2882668994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54040d07c0 msgr2=0x7f54040b01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:47.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.922+0000 7f5409a00640 1 --2- 172.21.15.159:0/2882668994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54040d07c0 0x7f54040b01e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f53f400cad0 tx=0x7f53f400cf90 comp rx=0 tx=0).stop 2024-04-04T06:06:47.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.923+0000 7f5409a00640 1 -- 172.21.15.159:0/2882668994 shutdown_connections 2024-04-04T06:06:47.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.923+0000 7f5409a00640 1 --2- 172.21.15.159:0/2882668994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5404153580 0x7f54040cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:47.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.923+0000 7f5409a00640 1 --2- 172.21.15.159:0/2882668994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54040d07c0 0x7f54040b01e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:47.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.923+0000 7f5409a00640 1 --2- 172.21.15.159:0/2882668994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f53e006c920 0x7f53e006ede0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:47.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.923+0000 7f5409a00640 1 --2- 172.21.15.159:0/2882668994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54040af4b0 0x7f5404072dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:47.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.923+0000 7f5409a00640 1 -- 172.21.15.159:0/2882668994 >> 172.21.15.159:0/2882668994 conn(0x7f540406b750 msgr2=0x7f540415e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:47.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.923+0000 7f5409a00640 1 -- 172.21.15.159:0/2882668994 shutdown_connections 2024-04-04T06:06:47.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:47.923+0000 7f5409a00640 1 -- 172.21.15.159:0/2882668994 wait complete. 2024-04-04T06:06:47.937 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:06:48.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.155+0000 7f35d2a00640 1 Processor -- start 2024-04-04T06:06:48.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.156+0000 7f35d2a00640 1 -- start start 2024-04-04T06:06:48.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.157+0000 7f35d2a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f35cc0d0280 0x7f35cc0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:48.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.157+0000 7f35d2a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35cc162160 0x7f35cc162610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:48.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.157+0000 7f35d2a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35cc15ec80 con 0x7f35cc141c10 2024-04-04T06:06:48.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.157+0000 7f35d2a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f35cc0d0ba0 con 0x7f35cc0d0280 2024-04-04T06:06:48.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.157+0000 7f35d2a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f35cc15ef20 con 0x7f35cc162160 2024-04-04T06:06:48.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.157+0000 7f35cb400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f35cc141c10 0x7f35cc071970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:32924/0 (socket says 172.21.15.159:32924) 2024-04-04T06:06:48.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.157+0000 7f35cb400640 1 -- 172.21.15.159:0/3484488316 learned_addr learned my addr 172.21.15.159:0/3484488316 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:48.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.158+0000 7f35cbe00640 1 --2- 172.21.15.159:0/3484488316 >> v2:172.21.15.178:3300/0 conn(0x7f35cc0d0280 0x7f35cc0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:48.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.158+0000 7f35d0a00640 1 --2- 172.21.15.159:0/3484488316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35cc162160 0x7f35cc162610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:48.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.158+0000 7f35caa00640 1 -- 172.21.15.159:0/3484488316 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3172910261 0 0) 0x7f35cc15ec80 con 0x7f35cc141c10 2024-04-04T06:06:48.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.158+0000 7f35caa00640 1 -- 172.21.15.159:0/3484488316 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35b0003520 con 0x7f35cc141c10 2024-04-04T06:06:48.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.158+0000 7f35caa00640 1 -- 172.21.15.159:0/3484488316 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4265348628 0 0) 0x7f35b0003520 con 0x7f35cc141c10 2024-04-04T06:06:48.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.158+0000 7f35caa00640 1 -- 172.21.15.159:0/3484488316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35cc162160 msgr2=0x7f35cc162610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:48.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.158+0000 7f35caa00640 1 --2- 172.21.15.159:0/3484488316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35cc162160 0x7f35cc162610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:48.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.158+0000 7f35caa00640 1 -- 172.21.15.159:0/3484488316 >> v2:172.21.15.178:3300/0 conn(0x7f35cc0d0280 msgr2=0x7f35cc0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:48.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.158+0000 7f35caa00640 1 --2- 172.21.15.159:0/3484488316 >> v2:172.21.15.178:3300/0 conn(0x7f35cc0d0280 0x7f35cc0d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:48.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.158+0000 7f35caa00640 1 -- 172.21.15.159:0/3484488316 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35cc15f200 con 0x7f35cc141c10 2024-04-04T06:06:48.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.158+0000 7f35caa00640 1 -- 172.21.15.159:0/3484488316 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f35bc0019f0 con 0x7f35cc141c10 2024-04-04T06:06:48.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.158+0000 7f35caa00640 1 -- 172.21.15.159:0/3484488316 >> v1:172.21.15.159:6789/0 conn(0x7f35cc141c10 legacy=0x7f35cc071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:48.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.158+0000 7f35caa00640 1 --2- 172.21.15.159:0/3484488316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35cc162160 0x7f35b00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:48.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.158+0000 7f35caa00640 1 --2- 172.21.15.159:0/3484488316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35b0003f80 0x7f35b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:48.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.158+0000 7f35caa00640 1 --2- 172.21.15.159:0/3484488316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b0004d40 0x7f35b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:48.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.158+0000 7f35caa00640 1 -- 172.21.15.159:0/3484488316 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc0d0ba0 con 0x7f35b0003f80 2024-04-04T06:06:48.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.158+0000 7f35caa00640 1 -- 172.21.15.159:0/3484488316 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc15ef20 con 0x7f35b0004d40 2024-04-04T06:06:48.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.158+0000 7f35caa00640 1 -- 172.21.15.159:0/3484488316 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f35b000b9c0 con 0x7f35cc162160 2024-04-04T06:06:48.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.159+0000 7f35cb400640 1 --2- 172.21.15.159:0/3484488316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35b0003f80 0x7f35b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:48.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.159+0000 7f35cbe00640 1 --2- 172.21.15.159:0/3484488316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35cc162160 0x7f35b00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:48.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.159+0000 7f35d0a00640 1 --2- 172.21.15.159:0/3484488316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b0004d40 0x7f35b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:48.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.159+0000 7f35cb400640 1 -- 172.21.15.159:0/3484488316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35cc162160 msgr2=0x7f35b00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:48.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.159+0000 7f35cb400640 1 --2- 172.21.15.159:0/3484488316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35cc162160 0x7f35b00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:48.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.159+0000 7f35cb400640 1 -- 172.21.15.159:0/3484488316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b0004d40 msgr2=0x7f35b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:48.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.159+0000 7f35cb400640 1 --2- 172.21.15.159:0/3484488316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b0004d40 0x7f35b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:48.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.159+0000 7f35cb400640 1 -- 172.21.15.159:0/3484488316 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b0003520 con 0x7f35b0003f80 2024-04-04T06:06:48.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.159+0000 7f35cbe00640 1 --2- 172.21.15.159:0/3484488316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35cc162160 0x7f35b00039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:48.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.159+0000 7f35d0a00640 1 --2- 172.21.15.159:0/3484488316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b0004d40 0x7f35b00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:48.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.160+0000 7f35cb400640 1 --2- 172.21.15.159:0/3484488316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35b0003f80 0x7f35b0004800 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f35cc148260 tx=0x7f35b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac61ccc7e045919 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:48.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.161+0000 7f35caa00640 1 -- 172.21.15.159:0/3484488316 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc00a3f0 con 0x7f35b0003f80 2024-04-04T06:06:48.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.161+0000 7f35caa00640 1 -- 172.21.15.159:0/3484488316 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35bc034650 con 0x7f35b0003f80 2024-04-04T06:06:48.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.161+0000 7f35caa00640 1 -- 172.21.15.159:0/3484488316 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc034a40 con 0x7f35b0003f80 2024-04-04T06:06:48.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.161+0000 7f35d2a00640 1 -- 172.21.15.159:0/3484488316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35b0003f80 msgr2=0x7f35b0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:48.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.161+0000 7f35d2a00640 1 --2- 172.21.15.159:0/3484488316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35b0003f80 0x7f35b0004800 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f35cc148260 tx=0x7f35b0005830 comp rx=0 tx=0).stop 2024-04-04T06:06:48.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.161+0000 7f35d0a00640 1 -- 172.21.15.159:0/3484488316 reap_dead start 2024-04-04T06:06:48.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.161+0000 7f35d2a00640 1 -- 172.21.15.159:0/3484488316 shutdown_connections 2024-04-04T06:06:48.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.161+0000 7f35d2a00640 1 -- 172.21.15.159:0/3484488316 >> 172.21.15.159:0/3484488316 conn(0x7f35cc06b750 msgr2=0x7f35cc06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:48.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.161+0000 7f35d2a00640 1 -- 172.21.15.159:0/3484488316 shutdown_connections 2024-04-04T06:06:48.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.161+0000 7f35d2a00640 1 -- 172.21.15.159:0/3484488316 wait complete. 2024-04-04T06:06:48.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.163+0000 7f35d2a00640 1 Processor -- start 2024-04-04T06:06:48.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.163+0000 7f35d2a00640 1 -- start start 2024-04-04T06:06:48.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.164+0000 7f35d2a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35b0003f80 0x7f35cc1577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:48.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.164+0000 7f35d2a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35cc09c9b0 0x7f35cc0ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:48.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.164+0000 7f35d2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35cc15dc90 0x7f35cc15e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:48.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.164+0000 7f35d2a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc158e90 con 0x7f35cc15dc90 2024-04-04T06:06:48.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.164+0000 7f35d2a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc15f7c0 con 0x7f35cc09c9b0 2024-04-04T06:06:48.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.164+0000 7f35d2a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f35cc15e650 con 0x7f35b0003f80 2024-04-04T06:06:48.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.164+0000 7f35d0a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35cc15dc90 0x7f35cc15e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:48.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.164+0000 7f35d0a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35cc15dc90 0x7f35cc15e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41170/0 (socket says 172.21.15.159:41170) 2024-04-04T06:06:48.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.164+0000 7f35d0a00640 1 -- 172.21.15.159:0/2679658789 learned_addr learned my addr 172.21.15.159:0/2679658789 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:48.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.165+0000 7f35cbe00640 1 --2- 172.21.15.159:0/2679658789 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35b0003f80 0x7f35cc1577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:48.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.165+0000 7f35cb400640 1 --2- 172.21.15.159:0/2679658789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35cc09c9b0 0x7f35cc0ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:48.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.165+0000 7f35d0a00640 1 -- 172.21.15.159:0/2679658789 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35b0003f80 msgr2=0x7f35cc1577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:48.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.165+0000 7f35d0a00640 1 --2- 172.21.15.159:0/2679658789 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35b0003f80 0x7f35cc1577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:48.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.165+0000 7f35d0a00640 1 -- 172.21.15.159:0/2679658789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35cc09c9b0 msgr2=0x7f35cc0ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:48.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.165+0000 7f35d0a00640 1 --2- 172.21.15.159:0/2679658789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35cc09c9b0 0x7f35cc0ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:48.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.165+0000 7f35d0a00640 1 -- 172.21.15.159:0/2679658789 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35cc09b910 con 0x7f35cc15dc90 2024-04-04T06:06:48.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.165+0000 7f35cb400640 1 --2- 172.21.15.159:0/2679658789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35cc09c9b0 0x7f35cc0ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:48.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.165+0000 7f35d0a00640 1 --2- 172.21.15.159:0/2679658789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35cc15dc90 0x7f35cc15e110 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f35cc06ef30 tx=0x7f35c4010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:48.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.165+0000 7f35c3e00640 1 -- 172.21.15.159:0/2679658789 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c400f040 con 0x7f35cc15dc90 2024-04-04T06:06:48.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.166+0000 7f35c3e00640 1 -- 172.21.15.159:0/2679658789 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c401c070 con 0x7f35cc15dc90 2024-04-04T06:06:48.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.166+0000 7f35c3e00640 1 -- 172.21.15.159:0/2679658789 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c400eb30 con 0x7f35cc15dc90 2024-04-04T06:06:48.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.166+0000 7f35d2a00640 1 -- 172.21.15.159:0/2679658789 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35cc09bbf0 con 0x7f35cc15dc90 2024-04-04T06:06:48.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.166+0000 7f35d2a00640 1 -- 172.21.15.159:0/2679658789 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35cc09c1b0 con 0x7f35cc15dc90 2024-04-04T06:06:48.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.167+0000 7f35d2a00640 1 -- 172.21.15.159:0/2679658789 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35cc0685c0 con 0x7f35cc15dc90 2024-04-04T06:06:48.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.168+0000 7f35c3e00640 1 -- 172.21.15.159:0/2679658789 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35c400ecd0 con 0x7f35cc15dc90 2024-04-04T06:06:48.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.169+0000 7f35c3e00640 1 --2- 172.21.15.159:0/2679658789 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35a806c8f0 0x7f35a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:48.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.169+0000 7f35c3e00640 1 -- 172.21.15.159:0/2679658789 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f35c408b330 con 0x7f35cc15dc90 2024-04-04T06:06:48.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.169+0000 7f35cbe00640 1 --2- 172.21.15.159:0/2679658789 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35a806c8f0 0x7f35a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:48.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.170+0000 7f35cbe00640 1 --2- 172.21.15.159:0/2679658789 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35a806c8f0 0x7f35a806edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009b20 tx=0x7f35b4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:48.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.173+0000 7f35c3e00640 1 -- 172.21.15.159:0/2679658789 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f35c4025020 con 0x7f35cc15dc90 2024-04-04T06:06:48.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.375+0000 7f35d2a00640 1 -- 172.21.15.159:0/2679658789 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f35cc0cfdb0 con 0x7f35cc15dc90 2024-04-04T06:06:48.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.376+0000 7f35c3e00640 1 -- 172.21.15.159:0/2679658789 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f35c4059620 con 0x7f35cc15dc90 2024-04-04T06:06:48.371 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:06:48.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.379+0000 7f35d2a00640 1 -- 172.21.15.159:0/2679658789 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35a806c8f0 msgr2=0x7f35a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:48.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.379+0000 7f35d2a00640 1 --2- 172.21.15.159:0/2679658789 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35a806c8f0 0x7f35a806edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009b20 tx=0x7f35b4008040 comp rx=0 tx=0).stop 2024-04-04T06:06:48.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.379+0000 7f35d2a00640 1 -- 172.21.15.159:0/2679658789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35cc15dc90 msgr2=0x7f35cc15e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:48.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.379+0000 7f35d2a00640 1 --2- 172.21.15.159:0/2679658789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35cc15dc90 0x7f35cc15e110 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f35cc06ef30 tx=0x7f35c4010c20 comp rx=0 tx=0).stop 2024-04-04T06:06:48.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.379+0000 7f35d2a00640 1 -- 172.21.15.159:0/2679658789 shutdown_connections 2024-04-04T06:06:48.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.379+0000 7f35d2a00640 1 --2- 172.21.15.159:0/2679658789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35cc15dc90 0x7f35cc15e110 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:48.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.379+0000 7f35d2a00640 1 --2- 172.21.15.159:0/2679658789 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35a806c8f0 0x7f35a806edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:48.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.379+0000 7f35d2a00640 1 --2- 172.21.15.159:0/2679658789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35cc09c9b0 0x7f35cc0ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:48.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.379+0000 7f35d2a00640 1 --2- 172.21.15.159:0/2679658789 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35b0003f80 0x7f35cc1577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:48.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.379+0000 7f35d2a00640 1 -- 172.21.15.159:0/2679658789 >> 172.21.15.159:0/2679658789 conn(0x7f35cc06b750 msgr2=0x7f35cc160ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:48.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.379+0000 7f35d2a00640 1 -- 172.21.15.159:0/2679658789 shutdown_connections 2024-04-04T06:06:48.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.379+0000 7f35d2a00640 1 -- 172.21.15.159:0/2679658789 wait complete. 2024-04-04T06:06:48.394 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:06:48.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.615+0000 7fbd92000640 1 Processor -- start 2024-04-04T06:06:48.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.615+0000 7fbd92000640 1 -- start start 2024-04-04T06:06:48.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.616+0000 7fbd92000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd8c155550 0x7fbd8c155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:48.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.616+0000 7fbd92000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbd8c0d0990 0x7fbd8c0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:48.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.616+0000 7fbd92000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd8c0b40b0 con 0x7fbd8c15f800 2024-04-04T06:06:48.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.616+0000 7fbd92000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fbd8c0b4390 con 0x7fbd8c0d0990 2024-04-04T06:06:48.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.616+0000 7fbd92000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbd8c0b4670 con 0x7fbd8c155550 2024-04-04T06:06:48.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.617+0000 7fbd8aa00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fbd8c15f800 0x7fbd8c0d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:32930/0 (socket says 172.21.15.159:32930) 2024-04-04T06:06:48.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.617+0000 7fbd8aa00640 1 -- 172.21.15.159:0/964150347 learned_addr learned my addr 172.21.15.159:0/964150347 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:48.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.617+0000 7fbd8b400640 1 --2- 172.21.15.159:0/964150347 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd8c155550 0x7fbd8c155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:48.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.617+0000 7fbd8be00640 1 --2- 172.21.15.159:0/964150347 >> v2:172.21.15.178:3300/0 conn(0x7fbd8c0d0990 0x7fbd8c0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:48.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.617+0000 7fbd8a000640 1 -- 172.21.15.159:0/964150347 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1772023472 0 0) 0x7fbd8c0b40b0 con 0x7fbd8c15f800 2024-04-04T06:06:48.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.617+0000 7fbd8a000640 1 -- 172.21.15.159:0/964150347 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd70003520 con 0x7fbd8c15f800 2024-04-04T06:06:48.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.617+0000 7fbd8a000640 1 -- 172.21.15.159:0/964150347 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3228493693 0 0) 0x7fbd70003520 con 0x7fbd8c15f800 2024-04-04T06:06:48.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.617+0000 7fbd8a000640 1 -- 172.21.15.159:0/964150347 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd8c155550 msgr2=0x7fbd8c155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:48.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.617+0000 7fbd8a000640 1 --2- 172.21.15.159:0/964150347 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd8c155550 0x7fbd8c155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:48.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.617+0000 7fbd8a000640 1 -- 172.21.15.159:0/964150347 >> v2:172.21.15.178:3300/0 conn(0x7fbd8c0d0990 msgr2=0x7fbd8c0aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:48.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.617+0000 7fbd8a000640 1 --2- 172.21.15.159:0/964150347 >> v2:172.21.15.178:3300/0 conn(0x7fbd8c0d0990 0x7fbd8c0aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:48.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.617+0000 7fbd8a000640 1 -- 172.21.15.159:0/964150347 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8c0b4950 con 0x7fbd8c15f800 2024-04-04T06:06:48.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.617+0000 7fbd8a000640 1 -- 172.21.15.159:0/964150347 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fbd74001690 con 0x7fbd8c15f800 2024-04-04T06:06:48.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.617+0000 7fbd8a000640 1 -- 172.21.15.159:0/964150347 >> v1:172.21.15.159:6789/0 conn(0x7fbd8c15f800 legacy=0x7fbd8c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:48.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.618+0000 7fbd8a000640 1 --2- 172.21.15.159:0/964150347 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd8c155550 0x7fbd700039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:48.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.618+0000 7fbd8a000640 1 --2- 172.21.15.159:0/964150347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd70003f80 0x7fbd70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:48.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.618+0000 7fbd8a000640 1 --2- 172.21.15.159:0/964150347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd70004d40 0x7fbd700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:48.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.618+0000 7fbd8a000640 1 -- 172.21.15.159:0/964150347 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c0b4390 con 0x7fbd70003f80 2024-04-04T06:06:48.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.618+0000 7fbd8a000640 1 -- 172.21.15.159:0/964150347 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c0b4670 con 0x7fbd70004d40 2024-04-04T06:06:48.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.618+0000 7fbd8a000640 1 -- 172.21.15.159:0/964150347 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbd7000dd30 con 0x7fbd8c155550 2024-04-04T06:06:48.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.618+0000 7fbd8aa00640 1 --2- 172.21.15.159:0/964150347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd70003f80 0x7fbd70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:48.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.618+0000 7fbd8be00640 1 --2- 172.21.15.159:0/964150347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd70004d40 0x7fbd700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:48.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.618+0000 7fbd8b400640 1 --2- 172.21.15.159:0/964150347 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd8c155550 0x7fbd700039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:48.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.618+0000 7fbd8aa00640 1 -- 172.21.15.159:0/964150347 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd8c155550 msgr2=0x7fbd700039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:48.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.618+0000 7fbd8aa00640 1 --2- 172.21.15.159:0/964150347 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd8c155550 0x7fbd700039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:48.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.618+0000 7fbd8aa00640 1 -- 172.21.15.159:0/964150347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd70004d40 msgr2=0x7fbd700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:48.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.618+0000 7fbd8aa00640 1 --2- 172.21.15.159:0/964150347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd70004d40 0x7fbd700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:48.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.618+0000 7fbd8aa00640 1 -- 172.21.15.159:0/964150347 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd70003520 con 0x7fbd70003f80 2024-04-04T06:06:48.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.620+0000 7fbd8aa00640 1 --2- 172.21.15.159:0/964150347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd70003f80 0x7fbd70004800 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbd7400c160 tx=0x7fbd70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70798a6334f4653b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:48.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.620+0000 7fbd8a000640 1 -- 172.21.15.159:0/964150347 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd74007c80 con 0x7fbd70003f80 2024-04-04T06:06:48.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.620+0000 7fbd8a000640 1 -- 172.21.15.159:0/964150347 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd74030070 con 0x7fbd70003f80 2024-04-04T06:06:48.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.620+0000 7fbd8a000640 1 -- 172.21.15.159:0/964150347 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd74005d10 con 0x7fbd70003f80 2024-04-04T06:06:48.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.620+0000 7fbd92000640 1 -- 172.21.15.159:0/964150347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd70003f80 msgr2=0x7fbd70004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:48.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.620+0000 7fbd92000640 1 --2- 172.21.15.159:0/964150347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd70003f80 0x7fbd70004800 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbd7400c160 tx=0x7fbd70005830 comp rx=0 tx=0).stop 2024-04-04T06:06:48.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.620+0000 7fbd8be00640 1 -- 172.21.15.159:0/964150347 reap_dead start 2024-04-04T06:06:48.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.620+0000 7fbd92000640 1 -- 172.21.15.159:0/964150347 shutdown_connections 2024-04-04T06:06:48.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.620+0000 7fbd92000640 1 -- 172.21.15.159:0/964150347 >> 172.21.15.159:0/964150347 conn(0x7fbd8c06b750 msgr2=0x7fbd8c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:48.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.621+0000 7fbd92000640 1 -- 172.21.15.159:0/964150347 shutdown_connections 2024-04-04T06:06:48.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.621+0000 7fbd92000640 1 -- 172.21.15.159:0/964150347 wait complete. 2024-04-04T06:06:48.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.623+0000 7fbd92000640 1 Processor -- start 2024-04-04T06:06:48.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.623+0000 7fbd92000640 1 -- start start 2024-04-04T06:06:48.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.624+0000 7fbd92000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd70003f80 0x7fbd8c1f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:48.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.624+0000 7fbd92000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd8c0cee40 0x7fbd8c0cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:48.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.624+0000 7fbd92000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd8c150580 0x7fbd8c150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:48.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.624+0000 7fbd92000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c15ba20 con 0x7fbd70003f80 2024-04-04T06:06:48.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.624+0000 7fbd92000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c15fc80 con 0x7fbd8c150580 2024-04-04T06:06:48.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.624+0000 7fbd92000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbd8c15bba0 con 0x7fbd8c0cee40 2024-04-04T06:06:48.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.624+0000 7fbd8b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd70003f80 0x7fbd8c1f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:48.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.624+0000 7fbd8b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd70003f80 0x7fbd8c1f3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41180/0 (socket says 172.21.15.159:41180) 2024-04-04T06:06:48.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.624+0000 7fbd8b400640 1 -- 172.21.15.159:0/71586092 learned_addr learned my addr 172.21.15.159:0/71586092 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:48.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.625+0000 7fbd8be00640 1 --2- 172.21.15.159:0/71586092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd8c150580 0x7fbd8c150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:48.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.625+0000 7fbd8aa00640 1 --2- 172.21.15.159:0/71586092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd8c0cee40 0x7fbd8c0cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:48.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.625+0000 7fbd8b400640 1 -- 172.21.15.159:0/71586092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd8c0cee40 msgr2=0x7fbd8c0cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:48.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.625+0000 7fbd8b400640 1 --2- 172.21.15.159:0/71586092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd8c0cee40 0x7fbd8c0cf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:48.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.625+0000 7fbd8b400640 1 -- 172.21.15.159:0/71586092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd8c150580 msgr2=0x7fbd8c150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:48.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.625+0000 7fbd8b400640 1 --2- 172.21.15.159:0/71586092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd8c150580 0x7fbd8c150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:48.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.625+0000 7fbd8b400640 1 -- 172.21.15.159:0/71586092 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8c0ae000 con 0x7fbd70003f80 2024-04-04T06:06:48.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.625+0000 7fbd8aa00640 1 --2- 172.21.15.159:0/71586092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd8c0cee40 0x7fbd8c0cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:48.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.625+0000 7fbd8be00640 1 --2- 172.21.15.159:0/71586092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd8c150580 0x7fbd8c150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:48.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.625+0000 7fbd8b400640 1 --2- 172.21.15.159:0/71586092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd70003f80 0x7fbd8c1f3e70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c007f60 tx=0x7fbd7c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:48.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.625+0000 7fbd83400640 1 -- 172.21.15.159:0/71586092 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c01a020 con 0x7fbd70003f80 2024-04-04T06:06:48.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.626+0000 7fbd83400640 1 -- 172.21.15.159:0/71586092 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd7c016470 con 0x7fbd70003f80 2024-04-04T06:06:48.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.626+0000 7fbd92000640 1 -- 172.21.15.159:0/71586092 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd8c0ae2e0 con 0x7fbd70003f80 2024-04-04T06:06:48.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.626+0000 7fbd83400640 1 -- 172.21.15.159:0/71586092 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c016860 con 0x7fbd70003f80 2024-04-04T06:06:48.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.626+0000 7fbd92000640 1 -- 172.21.15.159:0/71586092 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd8c0ae8a0 con 0x7fbd70003f80 2024-04-04T06:06:48.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.627+0000 7fbd92000640 1 -- 172.21.15.159:0/71586092 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd8c0685c0 con 0x7fbd70003f80 2024-04-04T06:06:48.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.628+0000 7fbd83400640 1 -- 172.21.15.159:0/71586092 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd7c005d10 con 0x7fbd70003f80 2024-04-04T06:06:48.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.629+0000 7fbd83400640 1 --2- 172.21.15.159:0/71586092 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd6806c920 0x7fbd6806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:48.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.630+0000 7fbd83400640 1 -- 172.21.15.159:0/71586092 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7fbd7c08ca60 con 0x7fbd70003f80 2024-04-04T06:06:48.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.630+0000 7fbd8aa00640 1 --2- 172.21.15.159:0/71586092 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd6806c920 0x7fbd6806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:48.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.631+0000 7fbd8aa00640 1 --2- 172.21.15.159:0/71586092 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd6806c920 0x7fbd6806ede0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbd74034000 tx=0x7fbd74007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:48.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.634+0000 7fbd83400640 1 -- 172.21.15.159:0/71586092 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbd7c024030 con 0x7fbd70003f80 2024-04-04T06:06:48.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.834+0000 7fbd92000640 1 -- 172.21.15.159:0/71586092 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbd8c0b1aa0 con 0x7fbd70003f80 2024-04-04T06:06:48.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.835+0000 7fbd83400640 1 -- 172.21.15.159:0/71586092 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fbd7c05ad40 con 0x7fbd70003f80 2024-04-04T06:06:48.830 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:06:48.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.837+0000 7fbd92000640 1 -- 172.21.15.159:0/71586092 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd6806c920 msgr2=0x7fbd6806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:48.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.837+0000 7fbd92000640 1 --2- 172.21.15.159:0/71586092 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd6806c920 0x7fbd6806ede0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbd74034000 tx=0x7fbd74007000 comp rx=0 tx=0).stop 2024-04-04T06:06:48.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.837+0000 7fbd92000640 1 -- 172.21.15.159:0/71586092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd70003f80 msgr2=0x7fbd8c1f3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:48.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.837+0000 7fbd92000640 1 --2- 172.21.15.159:0/71586092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd70003f80 0x7fbd8c1f3e70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c007f60 tx=0x7fbd7c007f90 comp rx=0 tx=0).stop 2024-04-04T06:06:48.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.837+0000 7fbd92000640 1 -- 172.21.15.159:0/71586092 shutdown_connections 2024-04-04T06:06:48.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.837+0000 7fbd92000640 1 --2- 172.21.15.159:0/71586092 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd6806c920 0x7fbd6806ede0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:48.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.837+0000 7fbd92000640 1 --2- 172.21.15.159:0/71586092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd8c150580 0x7fbd8c150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:48.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.837+0000 7fbd92000640 1 --2- 172.21.15.159:0/71586092 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd8c0cee40 0x7fbd8c0cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:48.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.837+0000 7fbd92000640 1 --2- 172.21.15.159:0/71586092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd70003f80 0x7fbd8c1f3e70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:48.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.837+0000 7fbd92000640 1 -- 172.21.15.159:0/71586092 >> 172.21.15.159:0/71586092 conn(0x7fbd8c06b750 msgr2=0x7fbd8c06df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:48.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.838+0000 7fbd92000640 1 -- 172.21.15.159:0/71586092 shutdown_connections 2024-04-04T06:06:48.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:48.838+0000 7fbd92000640 1 -- 172.21.15.159:0/71586092 wait complete. 2024-04-04T06:06:48.851 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-04T06:06:49.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.068+0000 7fb04f200640 1 Processor -- start 2024-04-04T06:06:49.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.068+0000 7fb04f200640 1 -- start start 2024-04-04T06:06:49.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.069+0000 7fb04f200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb048155930 0x7fb0480d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:49.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.069+0000 7fb04f200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0480d07c0 0x7fb0480aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:49.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.069+0000 7fb04f200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0480b40b0 con 0x7fb048155550 2024-04-04T06:06:49.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.069+0000 7fb04f200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb0480d0c30 con 0x7fb048155930 2024-04-04T06:06:49.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.069+0000 7fb04f200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb0480b4390 con 0x7fb0480d07c0 2024-04-04T06:06:49.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.070+0000 7fb04d200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0480d07c0 0x7fb0480aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:49.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.070+0000 7fb04d200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0480d07c0 0x7fb0480aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:34100/0 (socket says 172.21.15.159:34100) 2024-04-04T06:06:49.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.070+0000 7fb04d200640 1 -- 172.21.15.159:0/2589170237 learned_addr learned my addr 172.21.15.159:0/2589170237 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:49.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.070+0000 7fb047400640 1 --2- 172.21.15.159:0/2589170237 >> v2:172.21.15.178:3300/0 conn(0x7fb048155930 0x7fb0480d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:49.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.070+0000 7fb04d200640 1 -- 172.21.15.159:0/2589170237 >> v2:172.21.15.178:3300/0 conn(0x7fb048155930 msgr2=0x7fb0480d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:49.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.070+0000 7fb04d200640 1 --2- 172.21.15.159:0/2589170237 >> v2:172.21.15.178:3300/0 conn(0x7fb048155930 0x7fb0480d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:49.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.070+0000 7fb04d200640 1 -- 172.21.15.159:0/2589170237 >> v1:172.21.15.178:6789/0 conn(0x7fb048155550 legacy=0x7fb04815f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:49.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.070+0000 7fb04d200640 1 -- 172.21.15.159:0/2589170237 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0480b4670 con 0x7fb0480d07c0 2024-04-04T06:06:49.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.070+0000 7fb047400640 1 --2- 172.21.15.159:0/2589170237 >> v2:172.21.15.178:3300/0 conn(0x7fb048155930 0x7fb0480d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:49.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.072+0000 7fb04d200640 1 --2- 172.21.15.159:0/2589170237 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0480d07c0 0x7fb0480aede0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb040009e50 tx=0x7fb04002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=950697d0f39a39d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:49.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.072+0000 7fb046a00640 1 -- 172.21.15.159:0/2589170237 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04002cb00 con 0x7fb0480d07c0 2024-04-04T06:06:49.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.072+0000 7fb046a00640 1 -- 172.21.15.159:0/2589170237 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb04002cca0 con 0x7fb0480d07c0 2024-04-04T06:06:49.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.072+0000 7fb046a00640 1 -- 172.21.15.159:0/2589170237 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb040034610 con 0x7fb0480d07c0 2024-04-04T06:06:49.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.073+0000 7fb04f200640 1 -- 172.21.15.159:0/2589170237 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0480d07c0 msgr2=0x7fb0480aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:49.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.073+0000 7fb04f200640 1 --2- 172.21.15.159:0/2589170237 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0480d07c0 0x7fb0480aede0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb040009e50 tx=0x7fb04002bf60 comp rx=0 tx=0).stop 2024-04-04T06:06:49.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.073+0000 7fb04f200640 1 -- 172.21.15.159:0/2589170237 shutdown_connections 2024-04-04T06:06:49.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.073+0000 7fb04f200640 1 --2- 172.21.15.159:0/2589170237 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0480d07c0 0x7fb0480aede0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:49.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.073+0000 7fb04f200640 1 --2- 172.21.15.159:0/2589170237 >> v2:172.21.15.178:3300/0 conn(0x7fb048155930 0x7fb0480d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:49.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.073+0000 7fb04f200640 1 -- 172.21.15.159:0/2589170237 >> 172.21.15.159:0/2589170237 conn(0x7fb04806b750 msgr2=0x7fb04806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:49.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.073+0000 7fb04f200640 1 -- 172.21.15.159:0/2589170237 shutdown_connections 2024-04-04T06:06:49.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.073+0000 7fb04f200640 1 -- 172.21.15.159:0/2589170237 wait complete. 2024-04-04T06:06:49.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.075+0000 7fb04f200640 1 Processor -- start 2024-04-04T06:06:49.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.075+0000 7fb04f200640 1 -- start start 2024-04-04T06:06:49.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.076+0000 7fb04f200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0480d07c0 0x7fb04815e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:49.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.076+0000 7fb04f200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb048155550 0x7fb04815a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:49.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.076+0000 7fb04f200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb048155930 0x7fb0480aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:49.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.076+0000 7fb04f200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb04815ba20 con 0x7fb048155930 2024-04-04T06:06:49.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.076+0000 7fb04f200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb0480b4c30 con 0x7fb048155550 2024-04-04T06:06:49.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.076+0000 7fb04f200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb04815bba0 con 0x7fb0480d07c0 2024-04-04T06:06:49.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.076+0000 7fb04d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb048155930 0x7fb0480aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:49.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.076+0000 7fb04d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb048155930 0x7fb0480aff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41192/0 (socket says 172.21.15.159:41192) 2024-04-04T06:06:49.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.076+0000 7fb04d200640 1 -- 172.21.15.159:0/1840260965 learned_addr learned my addr 172.21.15.159:0/1840260965 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:49.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.076+0000 7fb047e00640 1 --2- 172.21.15.159:0/1840260965 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0480d07c0 0x7fb04815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:49.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.077+0000 7fb047400640 1 --2- 172.21.15.159:0/1840260965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb048155550 0x7fb04815a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:49.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.077+0000 7fb04d200640 1 -- 172.21.15.159:0/1840260965 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0480d07c0 msgr2=0x7fb04815e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:49.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.077+0000 7fb04d200640 1 --2- 172.21.15.159:0/1840260965 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0480d07c0 0x7fb04815e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:49.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.077+0000 7fb04d200640 1 -- 172.21.15.159:0/1840260965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb048155550 msgr2=0x7fb04815a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:49.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.077+0000 7fb04d200640 1 --2- 172.21.15.159:0/1840260965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb048155550 0x7fb04815a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:49.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.077+0000 7fb04d200640 1 -- 172.21.15.159:0/1840260965 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0480b1030 con 0x7fb048155930 2024-04-04T06:06:49.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.077+0000 7fb047e00640 1 --2- 172.21.15.159:0/1840260965 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0480d07c0 0x7fb04815e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:06:49.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.077+0000 7fb04d200640 1 --2- 172.21.15.159:0/1840260965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb048155930 0x7fb0480aff80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb040036040 tx=0x7fb040004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:49.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.077+0000 7fb03fe00640 1 -- 172.21.15.159:0/1840260965 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb040043020 con 0x7fb048155930 2024-04-04T06:06:49.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.078+0000 7fb03fe00640 1 -- 172.21.15.159:0/1840260965 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb04002ce50 con 0x7fb048155930 2024-04-04T06:06:49.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.078+0000 7fb03fe00640 1 -- 172.21.15.159:0/1840260965 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04003e690 con 0x7fb048155930 2024-04-04T06:06:49.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.078+0000 7fb04f200640 1 -- 172.21.15.159:0/1840260965 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb04815db50 con 0x7fb048155930 2024-04-04T06:06:49.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.078+0000 7fb04f200640 1 -- 172.21.15.159:0/1840260965 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0480b15b0 con 0x7fb048155930 2024-04-04T06:06:49.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.079+0000 7fb04f200640 1 -- 172.21.15.159:0/1840260965 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0480685c0 con 0x7fb048155930 2024-04-04T06:06:49.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.080+0000 7fb03fe00640 1 -- 172.21.15.159:0/1840260965 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb040046020 con 0x7fb048155930 2024-04-04T06:06:49.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.081+0000 7fb03fe00640 1 --2- 172.21.15.159:0/1840260965 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb02806c920 0x7fb02806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:49.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.081+0000 7fb03fe00640 1 -- 172.21.15.159:0/1840260965 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7fb0400ae3f0 con 0x7fb048155930 2024-04-04T06:06:49.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.081+0000 7fb047e00640 1 --2- 172.21.15.159:0/1840260965 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb02806c920 0x7fb02806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:49.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.082+0000 7fb047e00640 1 --2- 172.21.15.159:0/1840260965 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb02806c920 0x7fb02806ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb038001700 tx=0x7fb038009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:49.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.085+0000 7fb03fe00640 1 -- 172.21.15.159:0/1840260965 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb04004d030 con 0x7fb048155930 2024-04-04T06:06:49.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.297+0000 7fb04f200640 1 -- 172.21.15.159:0/1840260965 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb04815de30 con 0x7fb048155930 2024-04-04T06:06:49.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.298+0000 7fb03fe00640 1 -- 172.21.15.159:0/1840260965 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fb04007c780 con 0x7fb048155930 2024-04-04T06:06:49.293 INFO:teuthology.orchestra.run.smithi159.stderr:listed 0 entries 2024-04-04T06:06:49.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.301+0000 7fb04f200640 1 -- 172.21.15.159:0/1840260965 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb02806c920 msgr2=0x7fb02806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:49.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.301+0000 7fb04f200640 1 --2- 172.21.15.159:0/1840260965 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb02806c920 0x7fb02806ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb038001700 tx=0x7fb038009040 comp rx=0 tx=0).stop 2024-04-04T06:06:49.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.301+0000 7fb04f200640 1 -- 172.21.15.159:0/1840260965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb048155930 msgr2=0x7fb0480aff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:49.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.301+0000 7fb04f200640 1 --2- 172.21.15.159:0/1840260965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb048155930 0x7fb0480aff80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb040036040 tx=0x7fb040004a40 comp rx=0 tx=0).stop 2024-04-04T06:06:49.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.301+0000 7fb04f200640 1 -- 172.21.15.159:0/1840260965 shutdown_connections 2024-04-04T06:06:49.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.301+0000 7fb04f200640 1 --2- 172.21.15.159:0/1840260965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb048155930 0x7fb0480aff80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:49.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.301+0000 7fb04f200640 1 --2- 172.21.15.159:0/1840260965 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb02806c920 0x7fb02806ede0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:49.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.301+0000 7fb04f200640 1 --2- 172.21.15.159:0/1840260965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb048155550 0x7fb04815a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:49.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.302+0000 7fb04f200640 1 --2- 172.21.15.159:0/1840260965 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0480d07c0 0x7fb04815e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:49.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.302+0000 7fb04f200640 1 -- 172.21.15.159:0/1840260965 >> 172.21.15.159:0/1840260965 conn(0x7fb04806b750 msgr2=0x7fb04815cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:49.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.302+0000 7fb04f200640 1 -- 172.21.15.159:0/1840260965 shutdown_connections 2024-04-04T06:06:49.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.302+0000 7fb04f200640 1 -- 172.21.15.159:0/1840260965 wait complete. 2024-04-04T06:06:49.315 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-04T06:06:49.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:49.316 DEBUG:teuthology.orchestra.run.smithi159:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-04T06:06:49.416 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:06:49.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:49.417 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-04T06:06:49.471 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:06:49.471 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:06:49.527 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:06:49.528 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:06:49.574 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:06:49.574 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:06:49.668 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:06:49.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.893+0000 7f91b5800640 1 Processor -- start 2024-04-04T06:06:49.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.893+0000 7f91b5800640 1 -- start start 2024-04-04T06:06:49.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.895+0000 7f91b5800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91b00727f0 0x7f91b00b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:49.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.895+0000 7f91b5800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f91b00d0780 0x7f91b00d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:49.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.895+0000 7f91b5800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91b015b6f0 con 0x7f91b015b310 2024-04-04T06:06:49.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.895+0000 7f91b5800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f91b0072c50 con 0x7f91b00d0780 2024-04-04T06:06:49.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.895+0000 7f91b5800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f91b0072e80 con 0x7f91b00727f0 2024-04-04T06:06:49.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.895+0000 7f91aea00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f91b015b310 0x7f91b015f7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:32942/0 (socket says 172.21.15.159:32942) 2024-04-04T06:06:49.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.895+0000 7f91aea00640 1 -- 172.21.15.159:0/3468133419 learned_addr learned my addr 172.21.15.159:0/3468133419 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:49.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.895+0000 7f91af400640 1 --2- 172.21.15.159:0/3468133419 >> v2:172.21.15.159:3300/0 conn(0x7f91b00d0780 0x7f91b00d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:49.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.895+0000 7f91ae000640 1 --2- 172.21.15.159:0/3468133419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91b00727f0 0x7f91b00b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:49.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.895+0000 7f91ad600640 1 -- 172.21.15.159:0/3468133419 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1407658286 0 0) 0x7f91b015b6f0 con 0x7f91b015b310 2024-04-04T06:06:49.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.895+0000 7f91ad600640 1 -- 172.21.15.159:0/3468133419 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9194003520 con 0x7f91b015b310 2024-04-04T06:06:49.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.896+0000 7f91af400640 1 -- 172.21.15.159:0/3468133419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91b00727f0 msgr2=0x7f91b00b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:49.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.896+0000 7f91af400640 1 --2- 172.21.15.159:0/3468133419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91b00727f0 0x7f91b00b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:49.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.896+0000 7f91af400640 1 -- 172.21.15.159:0/3468133419 >> v1:172.21.15.159:6789/0 conn(0x7f91b015b310 legacy=0x7f91b015f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:49.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.896+0000 7f91af400640 1 -- 172.21.15.159:0/3468133419 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b0073160 con 0x7f91b00d0780 2024-04-04T06:06:49.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.897+0000 7f91af400640 1 --2- 172.21.15.159:0/3468133419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91b00d0780 0x7f91b00d0c10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009d90 tx=0x7f91a802bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223af3418f857bd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:49.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.897+0000 7f91ad600640 1 -- 172.21.15.159:0/3468133419 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8039070 con 0x7f91b00d0780 2024-04-04T06:06:49.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.897+0000 7f91ad600640 1 -- 172.21.15.159:0/3468133419 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a802cb40 con 0x7f91b00d0780 2024-04-04T06:06:49.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.897+0000 7f91ad600640 1 -- 172.21.15.159:0/3468133419 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8034440 con 0x7f91b00d0780 2024-04-04T06:06:49.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.898+0000 7f91b5800640 1 -- 172.21.15.159:0/3468133419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91b00d0780 msgr2=0x7f91b00d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:49.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.898+0000 7f91b5800640 1 --2- 172.21.15.159:0/3468133419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91b00d0780 0x7f91b00d0c10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009d90 tx=0x7f91a802bd90 comp rx=0 tx=0).stop 2024-04-04T06:06:49.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.898+0000 7f91b5800640 1 -- 172.21.15.159:0/3468133419 shutdown_connections 2024-04-04T06:06:49.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.898+0000 7f91b5800640 1 --2- 172.21.15.159:0/3468133419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91b00727f0 0x7f91b00b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:49.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.898+0000 7f91b5800640 1 --2- 172.21.15.159:0/3468133419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91b00d0780 0x7f91b00d0c10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:49.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.898+0000 7f91b5800640 1 -- 172.21.15.159:0/3468133419 >> 172.21.15.159:0/3468133419 conn(0x7f91b006b990 msgr2=0x7f91b006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:49.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.898+0000 7f91b5800640 1 -- 172.21.15.159:0/3468133419 shutdown_connections 2024-04-04T06:06:49.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.898+0000 7f91b5800640 1 -- 172.21.15.159:0/3468133419 wait complete. 2024-04-04T06:06:49.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.899+0000 7f91b5800640 1 Processor -- start 2024-04-04T06:06:49.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.900+0000 7f91b5800640 1 -- start start 2024-04-04T06:06:49.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.901+0000 7f91b5800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91b00727f0 0x7f91b009db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:49.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.901+0000 7f91b5800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91b00d0780 0x7f91b0155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:49.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.901+0000 7f91b5800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91b015b310 0x7f91b0159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:49.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.901+0000 7f91b5800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f91b0073440 con 0x7f91b00727f0 2024-04-04T06:06:49.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.901+0000 7f91b5800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f91b015f2c0 con 0x7f91b00d0780 2024-04-04T06:06:49.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.901+0000 7f91b5800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f91b00adc80 con 0x7f91b015b310 2024-04-04T06:06:49.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.901+0000 7f91aea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91b00727f0 0x7f91b009db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:49.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.901+0000 7f91aea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91b00727f0 0x7f91b009db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41220/0 (socket says 172.21.15.159:41220) 2024-04-04T06:06:49.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.901+0000 7f91aea00640 1 -- 172.21.15.159:0/980188607 learned_addr learned my addr 172.21.15.159:0/980188607 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:49.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.901+0000 7f91af400640 1 --2- 172.21.15.159:0/980188607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91b015b310 0x7f91b0159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:49.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.901+0000 7f91aea00640 1 -- 172.21.15.159:0/980188607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91b015b310 msgr2=0x7f91b0159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:49.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.901+0000 7f91aea00640 1 --2- 172.21.15.159:0/980188607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91b015b310 0x7f91b0159c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:49.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.901+0000 7f91aea00640 1 -- 172.21.15.159:0/980188607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91b00d0780 msgr2=0x7f91b0155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:49.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.901+0000 7f91aea00640 1 --2- 172.21.15.159:0/980188607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91b00d0780 0x7f91b0155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:49.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.901+0000 7f91aea00640 1 -- 172.21.15.159:0/980188607 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b009e0c0 con 0x7f91b00727f0 2024-04-04T06:06:49.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.902+0000 7f91aea00640 1 --2- 172.21.15.159:0/980188607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91b00727f0 0x7f91b009db80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f91a000c390 tx=0x7f91a000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:49.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.902+0000 7f91a6a00640 1 -- 172.21.15.159:0/980188607 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a0017070 con 0x7f91b00727f0 2024-04-04T06:06:49.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.902+0000 7f91a6a00640 1 -- 172.21.15.159:0/980188607 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a0005ce0 con 0x7f91b00727f0 2024-04-04T06:06:49.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.902+0000 7f91a6a00640 1 -- 172.21.15.159:0/980188607 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a00072f0 con 0x7f91b00727f0 2024-04-04T06:06:49.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.902+0000 7f91b5800640 1 -- 172.21.15.159:0/980188607 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b0155e30 con 0x7f91b00727f0 2024-04-04T06:06:49.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.902+0000 7f91b5800640 1 -- 172.21.15.159:0/980188607 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b0156670 con 0x7f91b00727f0 2024-04-04T06:06:49.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.903+0000 7f91b5800640 1 -- 172.21.15.159:0/980188607 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b00685c0 con 0x7f91b00727f0 2024-04-04T06:06:49.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.904+0000 7f91a6a00640 1 -- 172.21.15.159:0/980188607 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91a001a070 con 0x7f91b00727f0 2024-04-04T06:06:49.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.905+0000 7f91a6a00640 1 --2- 172.21.15.159:0/980188607 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f919006c8f0 0x7f919006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:49.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.905+0000 7f91a6a00640 1 -- 172.21.15.159:0/980188607 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f91a008b0c0 con 0x7f91b00727f0 2024-04-04T06:06:49.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.906+0000 7f91ae000640 1 --2- 172.21.15.159:0/980188607 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f919006c8f0 0x7f919006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:49.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.907+0000 7f91ae000640 1 --2- 172.21.15.159:0/980188607 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f919006c8f0 0x7f919006edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f91b015ac60 tx=0x7f919c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:49.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:49.909+0000 7f91a6a00640 1 -- 172.21.15.159:0/980188607 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f91b0155e30 con 0x7f91b00727f0 2024-04-04T06:06:50.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.112+0000 7f91b5800640 1 -- 172.21.15.159:0/980188607 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91b0157370 con 0x7f91b00727f0 2024-04-04T06:06:50.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.114+0000 7f91a6a00640 1 -- 172.21.15.159:0/980188607 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16958 (secure 0 0 0) 0x7f91a00593a0 con 0x7f91b00727f0 2024-04-04T06:06:50.109 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:50.109 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":23,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:06:33.796979+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:05:51.297293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-04T06:05:51.770792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":0,"to":2}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:06:50.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.116+0000 7f91b5800640 1 -- 172.21.15.159:0/980188607 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f919006c8f0 msgr2=0x7f919006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:50.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.116+0000 7f91b5800640 1 --2- 172.21.15.159:0/980188607 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f919006c8f0 0x7f919006edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f91b015ac60 tx=0x7f919c008040 comp rx=0 tx=0).stop 2024-04-04T06:06:50.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.116+0000 7f91b5800640 1 -- 172.21.15.159:0/980188607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91b00727f0 msgr2=0x7f91b009db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:50.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.116+0000 7f91b5800640 1 --2- 172.21.15.159:0/980188607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91b00727f0 0x7f91b009db80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f91a000c390 tx=0x7f91a000c850 comp rx=0 tx=0).stop 2024-04-04T06:06:50.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.117+0000 7f91b5800640 1 -- 172.21.15.159:0/980188607 shutdown_connections 2024-04-04T06:06:50.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.117+0000 7f91b5800640 1 --2- 172.21.15.159:0/980188607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91b015b310 0x7f91b0159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.117+0000 7f91b5800640 1 --2- 172.21.15.159:0/980188607 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f919006c8f0 0x7f919006edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.117+0000 7f91b5800640 1 --2- 172.21.15.159:0/980188607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91b00d0780 0x7f91b0155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.117+0000 7f91b5800640 1 --2- 172.21.15.159:0/980188607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91b00727f0 0x7f91b009db80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.117+0000 7f91b5800640 1 -- 172.21.15.159:0/980188607 >> 172.21.15.159:0/980188607 conn(0x7f91b006b990 msgr2=0x7f91b00ae820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:50.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.117+0000 7f91b5800640 1 -- 172.21.15.159:0/980188607 shutdown_connections 2024-04-04T06:06:50.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.117+0000 7f91b5800640 1 -- 172.21.15.159:0/980188607 wait complete. 2024-04-04T06:06:50.130 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:05:51.297293+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:05:51.770792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:06:50.130 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:06:50.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.344+0000 7f1b83200640 1 Processor -- start 2024-04-04T06:06:50.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.344+0000 7f1b83200640 1 -- start start 2024-04-04T06:06:50.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.346+0000 7f1b83200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f1b8413f190 0x7f1b840cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:50.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.346+0000 7f1b83200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1b840cc880 0x7f1b840cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:50.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.346+0000 7f1b83200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b8406d620 0x7f1b8406da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:50.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.346+0000 7f1b83200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f1b840cd230 con 0x7f1b8413f190 2024-04-04T06:06:50.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.346+0000 7f1b83200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f1b840ce890 con 0x7f1b840cc880 2024-04-04T06:06:50.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.346+0000 7f1b83200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1b840ceb50 con 0x7f1b8406d620 2024-04-04T06:06:50.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.346+0000 7f1b81e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f1b8413f190 0x7f1b840cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:50.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.346+0000 7f1b81e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f1b8413f190 0x7f1b840cf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41228/0 (socket says 172.21.15.159:41228) 2024-04-04T06:06:50.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.346+0000 7f1b81e00640 1 -- 172.21.15.159:0/578922920 learned_addr learned my addr 172.21.15.159:0/578922920 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:50.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.346+0000 7f1b81400640 1 --2- 172.21.15.159:0/578922920 >> v2:172.21.15.178:3300/0 conn(0x7f1b840cc880 0x7f1b840cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:50.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.346+0000 7f1b82800640 1 --2- 172.21.15.159:0/578922920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b8406d620 0x7f1b8406da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:50.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.346+0000 7f1b81e00640 1 -- 172.21.15.159:0/578922920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b8406d620 msgr2=0x7f1b8406da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:50.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.346+0000 7f1b81e00640 1 --2- 172.21.15.159:0/578922920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b8406d620 0x7f1b8406da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.346+0000 7f1b81e00640 1 -- 172.21.15.159:0/578922920 >> v2:172.21.15.178:3300/0 conn(0x7f1b840cc880 msgr2=0x7f1b840cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:50.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.346+0000 7f1b81e00640 1 --2- 172.21.15.159:0/578922920 >> v2:172.21.15.178:3300/0 conn(0x7f1b840cc880 0x7f1b840cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.346+0000 7f1b81e00640 1 -- 172.21.15.159:0/578922920 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b840cee30 con 0x7f1b8413f190 2024-04-04T06:06:50.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.346+0000 7f1b82800640 1 --2- 172.21.15.159:0/578922920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b8406d620 0x7f1b8406da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:50.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.348+0000 7f1b81e00640 1 --2- 172.21.15.159:0/578922920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b8413f190 0x7f1b840cf630 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1b840cc7b0 tx=0x7f1b7402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68c47186cf0ce9c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:50.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.348+0000 7f1b80a00640 1 -- 172.21.15.159:0/578922920 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74035020 con 0x7f1b8413f190 2024-04-04T06:06:50.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.348+0000 7f1b80a00640 1 -- 172.21.15.159:0/578922920 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b74031870 con 0x7f1b8413f190 2024-04-04T06:06:50.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.348+0000 7f1b80a00640 1 -- 172.21.15.159:0/578922920 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74031c40 con 0x7f1b8413f190 2024-04-04T06:06:50.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.349+0000 7f1b83200640 1 -- 172.21.15.159:0/578922920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b8413f190 msgr2=0x7f1b840cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:50.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.349+0000 7f1b83200640 1 --2- 172.21.15.159:0/578922920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b8413f190 0x7f1b840cf630 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1b840cc7b0 tx=0x7f1b7402b910 comp rx=0 tx=0).stop 2024-04-04T06:06:50.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.349+0000 7f1b83200640 1 -- 172.21.15.159:0/578922920 shutdown_connections 2024-04-04T06:06:50.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.349+0000 7f1b83200640 1 --2- 172.21.15.159:0/578922920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b8406d620 0x7f1b8406da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.349+0000 7f1b83200640 1 --2- 172.21.15.159:0/578922920 >> v2:172.21.15.178:3300/0 conn(0x7f1b840cc880 0x7f1b840cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.349+0000 7f1b83200640 1 --2- 172.21.15.159:0/578922920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b8413f190 0x7f1b840cf630 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.349+0000 7f1b83200640 1 -- 172.21.15.159:0/578922920 >> 172.21.15.159:0/578922920 conn(0x7f1b840b1ba0 msgr2=0x7f1b840b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:50.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.349+0000 7f1b83200640 1 -- 172.21.15.159:0/578922920 shutdown_connections 2024-04-04T06:06:50.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.349+0000 7f1b83200640 1 -- 172.21.15.159:0/578922920 wait complete. 2024-04-04T06:06:50.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.351+0000 7f1b83200640 1 Processor -- start 2024-04-04T06:06:50.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.351+0000 7f1b83200640 1 -- start start 2024-04-04T06:06:50.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.352+0000 7f1b83200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b8406d620 0x7f1b8413add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:50.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.352+0000 7f1b83200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b840cc880 0x7f1b8415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:50.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.352+0000 7f1b83200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b8413f190 0x7f1b841391c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:50.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.352+0000 7f1b83200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8413f570 con 0x7f1b840cc880 2024-04-04T06:06:50.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.352+0000 7f1b83200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1b84160c70 con 0x7f1b8413f190 2024-04-04T06:06:50.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.352+0000 7f1b83200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1b840cfb70 con 0x7f1b8406d620 2024-04-04T06:06:50.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.352+0000 7f1b81400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b840cc880 0x7f1b8415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:50.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.352+0000 7f1b81400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b840cc880 0x7f1b8415b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41236/0 (socket says 172.21.15.159:41236) 2024-04-04T06:06:50.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.352+0000 7f1b81400640 1 -- 172.21.15.159:0/4207043986 learned_addr learned my addr 172.21.15.159:0/4207043986 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:50.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.352+0000 7f1b81e00640 1 --2- 172.21.15.159:0/4207043986 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b8406d620 0x7f1b8413add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:50.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.352+0000 7f1b82800640 1 --2- 172.21.15.159:0/4207043986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b8413f190 0x7f1b841391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:50.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.353+0000 7f1b81400640 1 -- 172.21.15.159:0/4207043986 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b8406d620 msgr2=0x7f1b8413add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:50.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.353+0000 7f1b81400640 1 --2- 172.21.15.159:0/4207043986 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b8406d620 0x7f1b8413add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.353+0000 7f1b81400640 1 -- 172.21.15.159:0/4207043986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b8413f190 msgr2=0x7f1b841391c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:50.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.353+0000 7f1b81400640 1 --2- 172.21.15.159:0/4207043986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b8413f190 0x7f1b841391c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.353+0000 7f1b81400640 1 -- 172.21.15.159:0/4207043986 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b840b42e0 con 0x7f1b840cc880 2024-04-04T06:06:50.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.353+0000 7f1b81400640 1 --2- 172.21.15.159:0/4207043986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b840cc880 0x7f1b8415b4d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1b7000c7f0 tx=0x7f1b7000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:50.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.353+0000 7f1b7a000640 1 -- 172.21.15.159:0/4207043986 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70015070 con 0x7f1b840cc880 2024-04-04T06:06:50.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.353+0000 7f1b7a000640 1 -- 172.21.15.159:0/4207043986 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b70004240 con 0x7f1b840cc880 2024-04-04T06:06:50.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.353+0000 7f1b7a000640 1 -- 172.21.15.159:0/4207043986 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70007620 con 0x7f1b840cc880 2024-04-04T06:06:50.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.353+0000 7f1b83200640 1 -- 172.21.15.159:0/4207043986 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b8413a010 con 0x7f1b840cc880 2024-04-04T06:06:50.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.353+0000 7f1b83200640 1 -- 172.21.15.159:0/4207043986 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b8413a5d0 con 0x7f1b840cc880 2024-04-04T06:06:50.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.355+0000 7f1b83200640 1 -- 172.21.15.159:0/4207043986 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b84075940 con 0x7f1b840cc880 2024-04-04T06:06:50.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.356+0000 7f1b7a000640 1 -- 172.21.15.159:0/4207043986 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b700079d0 con 0x7f1b840cc880 2024-04-04T06:06:50.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.356+0000 7f1b7a000640 1 --2- 172.21.15.159:0/4207043986 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1b6406c8f0 0x7f1b6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:50.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.357+0000 7f1b7a000640 1 -- 172.21.15.159:0/4207043986 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f1b7008bfc0 con 0x7f1b840cc880 2024-04-04T06:06:50.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.357+0000 7f1b81e00640 1 --2- 172.21.15.159:0/4207043986 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1b6406c8f0 0x7f1b6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:50.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.358+0000 7f1b81e00640 1 --2- 172.21.15.159:0/4207043986 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1b6406c8f0 0x7f1b6406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1b74036000 tx=0x7f1b7403c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:50.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.361+0000 7f1b7a000640 1 -- 172.21.15.159:0/4207043986 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1b840b42e0 con 0x7f1b840cc880 2024-04-04T06:06:50.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.563+0000 7f1b83200640 1 -- 172.21.15.159:0/4207043986 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1b840b2870 con 0x7f1b840cc880 2024-04-04T06:06:50.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.564+0000 7f1b7a000640 1 -- 172.21.15.159:0/4207043986 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f1b7005a340 con 0x7f1b840cc880 2024-04-04T06:06:50.559 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:06:50.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.567+0000 7f1b83200640 1 -- 172.21.15.159:0/4207043986 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1b6406c8f0 msgr2=0x7f1b6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:50.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.567+0000 7f1b83200640 1 --2- 172.21.15.159:0/4207043986 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1b6406c8f0 0x7f1b6406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1b74036000 tx=0x7f1b7403c040 comp rx=0 tx=0).stop 2024-04-04T06:06:50.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.567+0000 7f1b83200640 1 -- 172.21.15.159:0/4207043986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b840cc880 msgr2=0x7f1b8415b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:50.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.567+0000 7f1b83200640 1 --2- 172.21.15.159:0/4207043986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b840cc880 0x7f1b8415b4d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1b7000c7f0 tx=0x7f1b7000ccb0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.567+0000 7f1b83200640 1 -- 172.21.15.159:0/4207043986 shutdown_connections 2024-04-04T06:06:50.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.567+0000 7f1b83200640 1 --2- 172.21.15.159:0/4207043986 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1b6406c8f0 0x7f1b6406edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.567+0000 7f1b83200640 1 --2- 172.21.15.159:0/4207043986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b8413f190 0x7f1b841391c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.567+0000 7f1b83200640 1 --2- 172.21.15.159:0/4207043986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b840cc880 0x7f1b8415b4d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.567+0000 7f1b83200640 1 --2- 172.21.15.159:0/4207043986 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b8406d620 0x7f1b8413add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.567+0000 7f1b83200640 1 -- 172.21.15.159:0/4207043986 >> 172.21.15.159:0/4207043986 conn(0x7f1b840b1ba0 msgr2=0x7f1b840cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:50.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.567+0000 7f1b83200640 1 -- 172.21.15.159:0/4207043986 shutdown_connections 2024-04-04T06:06:50.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.567+0000 7f1b83200640 1 -- 172.21.15.159:0/4207043986 wait complete. 2024-04-04T06:06:50.582 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:06:50.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.796+0000 7f32b3e00640 1 Processor -- start 2024-04-04T06:06:50.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.796+0000 7f32b3e00640 1 -- start start 2024-04-04T06:06:50.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.797+0000 7f32b3e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32b413f190 0x7f32b40cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:50.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.798+0000 7f32b3e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f32b406dd30 0x7f32b40cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:50.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.798+0000 7f32b3e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32b40ce890 con 0x7f32b40cc880 2024-04-04T06:06:50.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.798+0000 7f32b3e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f32b40cd200 con 0x7f32b406dd30 2024-04-04T06:06:50.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.798+0000 7f32b3e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f32b40ceb50 con 0x7f32b413f190 2024-04-04T06:06:50.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.798+0000 7f32abe00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f32b40cc880 0x7f32b406d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:32952/0 (socket says 172.21.15.159:32952) 2024-04-04T06:06:50.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.798+0000 7f32abe00640 1 -- 172.21.15.159:0/800754054 learned_addr learned my addr 172.21.15.159:0/800754054 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:50.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.798+0000 7f32b3400640 1 --2- 172.21.15.159:0/800754054 >> v2:172.21.15.159:3300/0 conn(0x7f32b406dd30 0x7f32b40cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:50.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.798+0000 7f32b2a00640 1 --2- 172.21.15.159:0/800754054 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32b413f190 0x7f32b40cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:50.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.798+0000 7f32b2000640 1 -- 172.21.15.159:0/800754054 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4125147864 0 0) 0x7f32b40ce890 con 0x7f32b40cc880 2024-04-04T06:06:50.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.798+0000 7f32b2000640 1 -- 172.21.15.159:0/800754054 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3298003520 con 0x7f32b40cc880 2024-04-04T06:06:50.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.799+0000 7f32b3400640 1 -- 172.21.15.159:0/800754054 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32b413f190 msgr2=0x7f32b40cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:50.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.799+0000 7f32b3400640 1 --2- 172.21.15.159:0/800754054 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32b413f190 0x7f32b40cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.799+0000 7f32b3400640 1 -- 172.21.15.159:0/800754054 >> v1:172.21.15.159:6789/0 conn(0x7f32b40cc880 legacy=0x7f32b406d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:50.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.799+0000 7f32b3400640 1 -- 172.21.15.159:0/800754054 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b40cee30 con 0x7f32b406dd30 2024-04-04T06:06:50.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.799+0000 7f32b2a00640 1 --2- 172.21.15.159:0/800754054 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32b413f190 0x7f32b40cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:06:50.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.800+0000 7f32b3400640 1 --2- 172.21.15.159:0/800754054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b406dd30 0x7f32b40cccc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009d90 tx=0x7f32ac02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e1a93cead8f40f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:50.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.800+0000 7f32b2000640 1 -- 172.21.15.159:0/800754054 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac039070 con 0x7f32b406dd30 2024-04-04T06:06:50.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.800+0000 7f32b2000640 1 -- 172.21.15.159:0/800754054 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32ac02cc50 con 0x7f32b406dd30 2024-04-04T06:06:50.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.800+0000 7f32b2000640 1 -- 172.21.15.159:0/800754054 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac034440 con 0x7f32b406dd30 2024-04-04T06:06:50.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.800+0000 7f32b3e00640 1 -- 172.21.15.159:0/800754054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b406dd30 msgr2=0x7f32b40cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:50.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.800+0000 7f32b3e00640 1 --2- 172.21.15.159:0/800754054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b406dd30 0x7f32b40cccc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009d90 tx=0x7f32ac02bea0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.801+0000 7f32b3e00640 1 -- 172.21.15.159:0/800754054 shutdown_connections 2024-04-04T06:06:50.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.801+0000 7f32b3e00640 1 --2- 172.21.15.159:0/800754054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b406dd30 0x7f32b40cccc0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.801+0000 7f32b3e00640 1 --2- 172.21.15.159:0/800754054 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32b413f190 0x7f32b40cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.801+0000 7f32b3e00640 1 -- 172.21.15.159:0/800754054 >> 172.21.15.159:0/800754054 conn(0x7f32b40b1ba0 msgr2=0x7f32b40b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:50.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.801+0000 7f32b3e00640 1 -- 172.21.15.159:0/800754054 shutdown_connections 2024-04-04T06:06:50.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.801+0000 7f32b3e00640 1 -- 172.21.15.159:0/800754054 wait complete. 2024-04-04T06:06:50.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.802+0000 7f32b3e00640 1 Processor -- start 2024-04-04T06:06:50.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.802+0000 7f32b3e00640 1 -- start start 2024-04-04T06:06:50.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.803+0000 7f32b3e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32b406dd30 0x7f32b40713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:50.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.803+0000 7f32b3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b40cc880 0x7f32b4136f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:50.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.803+0000 7f32b3e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32b413f190 0x7f32b41559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:50.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.803+0000 7f32b3e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f32b413f570 con 0x7f32b40cc880 2024-04-04T06:06:50.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.803+0000 7f32b3e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f32b4160c70 con 0x7f32b406dd30 2024-04-04T06:06:50.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.803+0000 7f32b3e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f32b40cfb70 con 0x7f32b413f190 2024-04-04T06:06:50.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.804+0000 7f32abe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b40cc880 0x7f32b4136f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:50.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.804+0000 7f32abe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b40cc880 0x7f32b4136f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41258/0 (socket says 172.21.15.159:41258) 2024-04-04T06:06:50.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.804+0000 7f32abe00640 1 -- 172.21.15.159:0/892485216 learned_addr learned my addr 172.21.15.159:0/892485216 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:50.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.804+0000 7f32b3400640 1 --2- 172.21.15.159:0/892485216 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32b413f190 0x7f32b41559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:50.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.804+0000 7f32b2a00640 1 --2- 172.21.15.159:0/892485216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32b406dd30 0x7f32b40713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:50.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.804+0000 7f32abe00640 1 -- 172.21.15.159:0/892485216 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32b413f190 msgr2=0x7f32b41559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:50.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.804+0000 7f32abe00640 1 --2- 172.21.15.159:0/892485216 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32b413f190 0x7f32b41559a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.804+0000 7f32abe00640 1 -- 172.21.15.159:0/892485216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32b406dd30 msgr2=0x7f32b40713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:50.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.804+0000 7f32abe00640 1 --2- 172.21.15.159:0/892485216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32b406dd30 0x7f32b40713e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.804+0000 7f32abe00640 1 -- 172.21.15.159:0/892485216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b4132e10 con 0x7f32b40cc880 2024-04-04T06:06:50.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.805+0000 7f32abe00640 1 --2- 172.21.15.159:0/892485216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b40cc880 0x7f32b4136f60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f329c00c460 tx=0x7f329c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:50.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.805+0000 7f32aaa00640 1 -- 172.21.15.159:0/892485216 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c017070 con 0x7f32b40cc880 2024-04-04T06:06:50.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.805+0000 7f32aaa00640 1 -- 172.21.15.159:0/892485216 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f329c005ce0 con 0x7f32b40cc880 2024-04-04T06:06:50.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.805+0000 7f32b3e00640 1 -- 172.21.15.159:0/892485216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b41330f0 con 0x7f32b40cc880 2024-04-04T06:06:50.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.805+0000 7f32aaa00640 1 -- 172.21.15.159:0/892485216 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c007290 con 0x7f32b40cc880 2024-04-04T06:06:50.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.805+0000 7f32b3e00640 1 -- 172.21.15.159:0/892485216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b41336b0 con 0x7f32b40cc880 2024-04-04T06:06:50.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.806+0000 7f32b3e00640 1 -- 172.21.15.159:0/892485216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b4075940 con 0x7f32b40cc880 2024-04-04T06:06:50.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.807+0000 7f32aaa00640 1 -- 172.21.15.159:0/892485216 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f329c01a070 con 0x7f32b40cc880 2024-04-04T06:06:50.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.808+0000 7f32aaa00640 1 --2- 172.21.15.159:0/892485216 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f329006c8f0 0x7f329006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:50.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.808+0000 7f32aaa00640 1 -- 172.21.15.159:0/892485216 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f329c08afe0 con 0x7f32b40cc880 2024-04-04T06:06:50.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.809+0000 7f32b2a00640 1 --2- 172.21.15.159:0/892485216 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f329006c8f0 0x7f329006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:50.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.810+0000 7f32b2a00640 1 --2- 172.21.15.159:0/892485216 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f329006c8f0 0x7f329006edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f32a4006fd0 tx=0x7f32a4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:50.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.813+0000 7f32aaa00640 1 -- 172.21.15.159:0/892485216 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f329c059390 con 0x7f32b40cc880 2024-04-04T06:06:50.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.994+0000 7f32b3e00640 1 -- 172.21.15.159:0/892485216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f32b415c240 con 0x7f32b40cc880 2024-04-04T06:06:50.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.995+0000 7f32aaa00640 1 -- 172.21.15.159:0/892485216 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f329c058c90 con 0x7f32b40cc880 2024-04-04T06:06:50.990 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:06:50.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.998+0000 7f32b3e00640 1 -- 172.21.15.159:0/892485216 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f329006c8f0 msgr2=0x7f329006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:50.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.998+0000 7f32b3e00640 1 --2- 172.21.15.159:0/892485216 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f329006c8f0 0x7f329006edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f32a4006fd0 tx=0x7f32a4008040 comp rx=0 tx=0).stop 2024-04-04T06:06:50.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.998+0000 7f32b3e00640 1 -- 172.21.15.159:0/892485216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b40cc880 msgr2=0x7f32b4136f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:50.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.998+0000 7f32b3e00640 1 --2- 172.21.15.159:0/892485216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b40cc880 0x7f32b4136f60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f329c00c460 tx=0x7f329c00c920 comp rx=0 tx=0).stop 2024-04-04T06:06:50.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.998+0000 7f32b3e00640 1 -- 172.21.15.159:0/892485216 shutdown_connections 2024-04-04T06:06:50.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.998+0000 7f32b3e00640 1 --2- 172.21.15.159:0/892485216 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32b413f190 0x7f32b41559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.998+0000 7f32b3e00640 1 --2- 172.21.15.159:0/892485216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b40cc880 0x7f32b4136f60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.998+0000 7f32b3e00640 1 --2- 172.21.15.159:0/892485216 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f329006c8f0 0x7f329006edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.998+0000 7f32b3e00640 1 --2- 172.21.15.159:0/892485216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32b406dd30 0x7f32b40713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:50.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.998+0000 7f32b3e00640 1 -- 172.21.15.159:0/892485216 >> 172.21.15.159:0/892485216 conn(0x7f32b40b1ba0 msgr2=0x7f32b415f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:50.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.998+0000 7f32b3e00640 1 -- 172.21.15.159:0/892485216 shutdown_connections 2024-04-04T06:06:50.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:50.998+0000 7f32b3e00640 1 -- 172.21.15.159:0/892485216 wait complete. 2024-04-04T06:06:51.010 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:06:51.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.226+0000 7f3c74a00640 1 Processor -- start 2024-04-04T06:06:51.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.226+0000 7f3c74a00640 1 -- start start 2024-04-04T06:06:51.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.228+0000 7f3c74a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3c700d0280 0x7f3c700d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:51.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.228+0000 7f3c74a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c70141c10 0x7f3c70071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:51.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.228+0000 7f3c74a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3c70071eb0 0x7f3c7006d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:51.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.228+0000 7f3c74a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f3c7015ec80 con 0x7f3c70071eb0 2024-04-04T06:06:51.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.228+0000 7f3c74a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3c700d0ba0 con 0x7f3c700d0280 2024-04-04T06:06:51.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.228+0000 7f3c74a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3c7015ef20 con 0x7f3c70141c10 2024-04-04T06:06:51.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.228+0000 7f3c6e800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3c70071eb0 0x7f3c7006d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:51.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.228+0000 7f3c6e800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3c70071eb0 0x7f3c7006d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41270/0 (socket says 172.21.15.159:41270) 2024-04-04T06:06:51.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.228+0000 7f3c6e800640 1 -- 172.21.15.159:0/1636521370 learned_addr learned my addr 172.21.15.159:0/1636521370 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:51.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.229+0000 7f3c6de00640 1 --2- 172.21.15.159:0/1636521370 >> v2:172.21.15.178:3300/0 conn(0x7f3c700d0280 0x7f3c700d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:51.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.229+0000 7f3c65400640 1 --2- 172.21.15.159:0/1636521370 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c70141c10 0x7f3c70071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:51.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.229+0000 7f3c6e800640 1 -- 172.21.15.159:0/1636521370 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c70141c10 msgr2=0x7f3c70071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:51.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.229+0000 7f3c6e800640 1 --2- 172.21.15.159:0/1636521370 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c70141c10 0x7f3c70071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.229+0000 7f3c6e800640 1 -- 172.21.15.159:0/1636521370 >> v2:172.21.15.178:3300/0 conn(0x7f3c700d0280 msgr2=0x7f3c700d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:51.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.229+0000 7f3c6e800640 1 --2- 172.21.15.159:0/1636521370 >> v2:172.21.15.178:3300/0 conn(0x7f3c700d0280 0x7f3c700d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.229+0000 7f3c6e800640 1 -- 172.21.15.159:0/1636521370 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7015f200 con 0x7f3c70071eb0 2024-04-04T06:06:51.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.230+0000 7f3c6e800640 1 --2- 172.21.15.159:0/1636521370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c70071eb0 0x7f3c7006d5b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3c700d1050 tx=0x7f3c6802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81254f536c94b88 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:51.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.230+0000 7f3c6d400640 1 -- 172.21.15.159:0/1636521370 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6802ca70 con 0x7f3c70071eb0 2024-04-04T06:06:51.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.230+0000 7f3c6d400640 1 -- 172.21.15.159:0/1636521370 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c6802cc10 con 0x7f3c70071eb0 2024-04-04T06:06:51.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.230+0000 7f3c6d400640 1 -- 172.21.15.159:0/1636521370 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c680343f0 con 0x7f3c70071eb0 2024-04-04T06:06:51.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.230+0000 7f3c74a00640 1 -- 172.21.15.159:0/1636521370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c70071eb0 msgr2=0x7f3c7006d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:51.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.230+0000 7f3c74a00640 1 --2- 172.21.15.159:0/1636521370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c70071eb0 0x7f3c7006d5b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3c700d1050 tx=0x7f3c6802bed0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.231+0000 7f3c74a00640 1 -- 172.21.15.159:0/1636521370 shutdown_connections 2024-04-04T06:06:51.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.231+0000 7f3c74a00640 1 --2- 172.21.15.159:0/1636521370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c70071eb0 0x7f3c7006d5b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.231+0000 7f3c74a00640 1 --2- 172.21.15.159:0/1636521370 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c70141c10 0x7f3c70071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.231+0000 7f3c74a00640 1 --2- 172.21.15.159:0/1636521370 >> v2:172.21.15.178:3300/0 conn(0x7f3c700d0280 0x7f3c700d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.231+0000 7f3c74a00640 1 -- 172.21.15.159:0/1636521370 >> 172.21.15.159:0/1636521370 conn(0x7f3c7006b750 msgr2=0x7f3c7006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:51.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.231+0000 7f3c74a00640 1 -- 172.21.15.159:0/1636521370 shutdown_connections 2024-04-04T06:06:51.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.231+0000 7f3c74a00640 1 -- 172.21.15.159:0/1636521370 wait complete. 2024-04-04T06:06:51.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.232+0000 7f3c74a00640 1 Processor -- start 2024-04-04T06:06:51.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.232+0000 7f3c74a00640 1 -- start start 2024-04-04T06:06:51.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.233+0000 7f3c74a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c70071eb0 0x7f3c7009d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:51.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.233+0000 7f3c74a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c700d0280 0x7f3c70099740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:51.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.233+0000 7f3c74a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c70141c10 0x7f3c70146050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:51.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.233+0000 7f3c74a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70158e90 con 0x7f3c70141c10 2024-04-04T06:06:51.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.233+0000 7f3c74a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7015f7c0 con 0x7f3c70071eb0 2024-04-04T06:06:51.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.233+0000 7f3c74a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3c7009d750 con 0x7f3c700d0280 2024-04-04T06:06:51.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.234+0000 7f3c6e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c70141c10 0x7f3c70146050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:51.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.234+0000 7f3c6e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c70141c10 0x7f3c70146050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41274/0 (socket says 172.21.15.159:41274) 2024-04-04T06:06:51.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.234+0000 7f3c6e800640 1 -- 172.21.15.159:0/1650706109 learned_addr learned my addr 172.21.15.159:0/1650706109 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:51.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.234+0000 7f3c6e800640 1 -- 172.21.15.159:0/1650706109 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c700d0280 msgr2=0x7f3c70099740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:51.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.234+0000 7f3c6e800640 1 --2- 172.21.15.159:0/1650706109 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c700d0280 0x7f3c70099740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.234+0000 7f3c6e800640 1 -- 172.21.15.159:0/1650706109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c70071eb0 msgr2=0x7f3c7009d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:51.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.234+0000 7f3c6e800640 1 --2- 172.21.15.159:0/1650706109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c70071eb0 0x7f3c7009d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.234+0000 7f3c6e800640 1 -- 172.21.15.159:0/1650706109 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7009d980 con 0x7f3c70141c10 2024-04-04T06:06:51.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.234+0000 7f3c6e800640 1 --2- 172.21.15.159:0/1650706109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c70141c10 0x7f3c70146050 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009f40 tx=0x7f3c68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:51.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.235+0000 7f3c66a00640 1 -- 172.21.15.159:0/1650706109 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c68043020 con 0x7f3c70141c10 2024-04-04T06:06:51.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.235+0000 7f3c66a00640 1 -- 172.21.15.159:0/1650706109 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c6803e450 con 0x7f3c70141c10 2024-04-04T06:06:51.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.235+0000 7f3c74a00640 1 -- 172.21.15.159:0/1650706109 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7009dc60 con 0x7f3c70141c10 2024-04-04T06:06:51.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.235+0000 7f3c66a00640 1 -- 172.21.15.159:0/1650706109 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c68009510 con 0x7f3c70141c10 2024-04-04T06:06:51.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.235+0000 7f3c74a00640 1 -- 172.21.15.159:0/1650706109 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c70145b30 con 0x7f3c70141c10 2024-04-04T06:06:51.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.236+0000 7f3c74a00640 1 -- 172.21.15.159:0/1650706109 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c700685c0 con 0x7f3c70141c10 2024-04-04T06:06:51.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.237+0000 7f3c66a00640 1 -- 172.21.15.159:0/1650706109 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c68046020 con 0x7f3c70141c10 2024-04-04T06:06:51.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.237+0000 7f3c66a00640 1 --2- 172.21.15.159:0/1650706109 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c4c06c8f0 0x7f3c4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:51.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.237+0000 7f3c66a00640 1 -- 172.21.15.159:0/1650706109 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f3c680ae4b0 con 0x7f3c70141c10 2024-04-04T06:06:51.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.238+0000 7f3c6de00640 1 --2- 172.21.15.159:0/1650706109 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c4c06c8f0 0x7f3c4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:51.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.239+0000 7f3c6de00640 1 --2- 172.21.15.159:0/1650706109 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c4c06c8f0 0x7f3c4c06edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3c60006b10 tx=0x7f3c60008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:51.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.242+0000 7f3c66a00640 1 -- 172.21.15.159:0/1650706109 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3c6804d030 con 0x7f3c70141c10 2024-04-04T06:06:51.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.424+0000 7f3c74a00640 1 -- 172.21.15.159:0/1650706109 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3c700cfdb0 con 0x7f3c70141c10 2024-04-04T06:06:51.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.425+0000 7f3c66a00640 1 -- 172.21.15.159:0/1650706109 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f3c6807c8c0 con 0x7f3c70141c10 2024-04-04T06:06:51.419 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:06:51.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.427+0000 7f3c74a00640 1 -- 172.21.15.159:0/1650706109 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c4c06c8f0 msgr2=0x7f3c4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:51.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.427+0000 7f3c74a00640 1 --2- 172.21.15.159:0/1650706109 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c4c06c8f0 0x7f3c4c06edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3c60006b10 tx=0x7f3c60008040 comp rx=0 tx=0).stop 2024-04-04T06:06:51.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.427+0000 7f3c74a00640 1 -- 172.21.15.159:0/1650706109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c70141c10 msgr2=0x7f3c70146050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:51.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.427+0000 7f3c74a00640 1 --2- 172.21.15.159:0/1650706109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c70141c10 0x7f3c70146050 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009f40 tx=0x7f3c68004a40 comp rx=0 tx=0).stop 2024-04-04T06:06:51.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.427+0000 7f3c74a00640 1 -- 172.21.15.159:0/1650706109 shutdown_connections 2024-04-04T06:06:51.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.428+0000 7f3c74a00640 1 --2- 172.21.15.159:0/1650706109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c70141c10 0x7f3c70146050 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.428+0000 7f3c74a00640 1 --2- 172.21.15.159:0/1650706109 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c700d0280 0x7f3c70099740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.428+0000 7f3c74a00640 1 --2- 172.21.15.159:0/1650706109 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c4c06c8f0 0x7f3c4c06edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.428+0000 7f3c74a00640 1 --2- 172.21.15.159:0/1650706109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c70071eb0 0x7f3c7009d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.428+0000 7f3c74a00640 1 -- 172.21.15.159:0/1650706109 >> 172.21.15.159:0/1650706109 conn(0x7f3c7006b750 msgr2=0x7f3c70144030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:51.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.428+0000 7f3c74a00640 1 -- 172.21.15.159:0/1650706109 shutdown_connections 2024-04-04T06:06:51.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.428+0000 7f3c74a00640 1 -- 172.21.15.159:0/1650706109 wait complete. 2024-04-04T06:06:51.439 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-04T06:06:51.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.651+0000 7fe5ae400640 1 Processor -- start 2024-04-04T06:06:51.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.652+0000 7fe5ae400640 1 -- start start 2024-04-04T06:06:51.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.653+0000 7fe5ae400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe5a8153580 0x7fe5a80d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:51.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.653+0000 7fe5ae400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5a80d07c0 0x7fe5a80aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:51.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.653+0000 7fe5ae400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5a80afab0 con 0x7fe5a80af2e0 2024-04-04T06:06:51.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.653+0000 7fe5ae400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe5a80d0c00 con 0x7fe5a8153580 2024-04-04T06:06:51.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.653+0000 7fe5ae400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe5a815cc80 con 0x7fe5a80d07c0 2024-04-04T06:06:51.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.654+0000 7fe5a7e00640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fe5a80af2e0 0x7fe5a806d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.159:44762/0 (socket says 172.21.15.159:44762) 2024-04-04T06:06:51.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.654+0000 7fe5a7400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe5a8153580 0x7fe5a80d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:51.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.654+0000 7fe5a7e00640 1 -- 172.21.15.159:0/2281728782 learned_addr learned my addr 172.21.15.159:0/2281728782 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:51.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.654+0000 7fe59ea00640 1 --2- 172.21.15.159:0/2281728782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5a80d07c0 0x7fe5a80aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:51.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.655+0000 7fe5a7400640 1 -- 172.21.15.159:0/2281728782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5a80d07c0 msgr2=0x7fe5a80aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:51.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.655+0000 7fe5a7400640 1 --2- 172.21.15.159:0/2281728782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5a80d07c0 0x7fe5a80aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.655+0000 7fe5a7400640 1 -- 172.21.15.159:0/2281728782 >> v1:172.21.15.178:6789/0 conn(0x7fe5a80af2e0 legacy=0x7fe5a806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:51.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.655+0000 7fe5a7400640 1 -- 172.21.15.159:0/2281728782 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a815da00 con 0x7fe5a8153580 2024-04-04T06:06:51.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.656+0000 7fe5a7400640 1 --2- 172.21.15.159:0/2281728782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5a8153580 0x7fe5a80d0280 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe5980097a0 tx=0x7fe59802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1af1beb57f2bf server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:51.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.656+0000 7fe5a6a00640 1 -- 172.21.15.159:0/2281728782 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598031780 con 0x7fe5a8153580 2024-04-04T06:06:51.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.656+0000 7fe5a6a00640 1 -- 172.21.15.159:0/2281728782 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe598037070 con 0x7fe5a8153580 2024-04-04T06:06:51.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.656+0000 7fe5a6a00640 1 -- 172.21.15.159:0/2281728782 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598031b40 con 0x7fe5a8153580 2024-04-04T06:06:51.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.657+0000 7fe5ae400640 1 -- 172.21.15.159:0/2281728782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5a8153580 msgr2=0x7fe5a80d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:51.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.657+0000 7fe5ae400640 1 --2- 172.21.15.159:0/2281728782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5a8153580 0x7fe5a80d0280 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe5980097a0 tx=0x7fe59802b820 comp rx=0 tx=0).stop 2024-04-04T06:06:51.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.657+0000 7fe5ae400640 1 -- 172.21.15.159:0/2281728782 shutdown_connections 2024-04-04T06:06:51.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.657+0000 7fe5ae400640 1 --2- 172.21.15.159:0/2281728782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5a80d07c0 0x7fe5a80aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.657+0000 7fe5ae400640 1 --2- 172.21.15.159:0/2281728782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5a8153580 0x7fe5a80d0280 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.657+0000 7fe5ae400640 1 -- 172.21.15.159:0/2281728782 >> 172.21.15.159:0/2281728782 conn(0x7fe5a806b750 msgr2=0x7fe5a806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:51.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.657+0000 7fe5ae400640 1 -- 172.21.15.159:0/2281728782 shutdown_connections 2024-04-04T06:06:51.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.658+0000 7fe5ae400640 1 -- 172.21.15.159:0/2281728782 wait complete. 2024-04-04T06:06:51.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.659+0000 7fe5ae400640 1 Processor -- start 2024-04-04T06:06:51.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.660+0000 7fe5ae400640 1 -- start start 2024-04-04T06:06:51.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.660+0000 7fe5ae400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5a80af2e0 0x7fe5a81479a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:51.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.660+0000 7fe5ae400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5a80d07c0 0x7fe5a814c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:51.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.661+0000 7fe5ae400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5a8153580 0x7fe5a80cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:51.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.661+0000 7fe5ae400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a815d4e0 con 0x7fe5a80af2e0 2024-04-04T06:06:51.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.661+0000 7fe5ae400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a815d200 con 0x7fe5a80d07c0 2024-04-04T06:06:51.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.661+0000 7fe5ae400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe5a8153960 con 0x7fe5a8153580 2024-04-04T06:06:51.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.661+0000 7fe5a7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5a80af2e0 0x7fe5a81479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:51.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.661+0000 7fe5a7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5a80af2e0 0x7fe5a81479a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41276/0 (socket says 172.21.15.159:41276) 2024-04-04T06:06:51.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.661+0000 7fe5a7400640 1 -- 172.21.15.159:0/1994448621 learned_addr learned my addr 172.21.15.159:0/1994448621 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:51.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.661+0000 7fe5a7e00640 1 --2- 172.21.15.159:0/1994448621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5a8153580 0x7fe5a80cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:51.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.661+0000 7fe59ea00640 1 --2- 172.21.15.159:0/1994448621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5a80d07c0 0x7fe5a814c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:51.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.661+0000 7fe5a7400640 1 -- 172.21.15.159:0/1994448621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5a8153580 msgr2=0x7fe5a80cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:51.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.661+0000 7fe5a7400640 1 --2- 172.21.15.159:0/1994448621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5a8153580 0x7fe5a80cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.661+0000 7fe5a7400640 1 -- 172.21.15.159:0/1994448621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5a80d07c0 msgr2=0x7fe5a814c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:51.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.661+0000 7fe5a7400640 1 --2- 172.21.15.159:0/1994448621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5a80d07c0 0x7fe5a814c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.661+0000 7fe5a7400640 1 -- 172.21.15.159:0/1994448621 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a814ceb0 con 0x7fe5a80af2e0 2024-04-04T06:06:51.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.662+0000 7fe5a7400640 1 --2- 172.21.15.159:0/1994448621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5a80af2e0 0x7fe5a81479a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe598034040 tx=0x7fe5980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:51.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.662+0000 7fe59fe00640 1 -- 172.21.15.159:0/1994448621 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598032610 con 0x7fe5a80af2e0 2024-04-04T06:06:51.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.662+0000 7fe59fe00640 1 -- 172.21.15.159:0/1994448621 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe598031ec0 con 0x7fe5a80af2e0 2024-04-04T06:06:51.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.662+0000 7fe59fe00640 1 -- 172.21.15.159:0/1994448621 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598032a10 con 0x7fe5a80af2e0 2024-04-04T06:06:51.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.662+0000 7fe5ae400640 1 -- 172.21.15.159:0/1994448621 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a814d190 con 0x7fe5a80af2e0 2024-04-04T06:06:51.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.662+0000 7fe5ae400640 1 -- 172.21.15.159:0/1994448621 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a814d750 con 0x7fe5a80af2e0 2024-04-04T06:06:51.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.663+0000 7fe5ae400640 1 -- 172.21.15.159:0/1994448621 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5a80685c0 con 0x7fe5a80af2e0 2024-04-04T06:06:51.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.664+0000 7fe59fe00640 1 -- 172.21.15.159:0/1994448621 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe598032bb0 con 0x7fe5a80af2e0 2024-04-04T06:06:51.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.665+0000 7fe59fe00640 1 --2- 172.21.15.159:0/1994448621 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe58406c8f0 0x7fe58406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:51.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.665+0000 7fe59fe00640 1 -- 172.21.15.159:0/1994448621 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7fe5980ad850 con 0x7fe5a80af2e0 2024-04-04T06:06:51.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.666+0000 7fe59ea00640 1 --2- 172.21.15.159:0/1994448621 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe58406c8f0 0x7fe58406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:51.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.667+0000 7fe59ea00640 1 --2- 172.21.15.159:0/1994448621 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe58406c8f0 0x7fe58406edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8147ee0 tx=0x7fe590008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:51.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.669+0000 7fe59fe00640 1 -- 172.21.15.159:0/1994448621 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe598040070 con 0x7fe5a80af2e0 2024-04-04T06:06:51.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.866+0000 7fe5ae400640 1 -- 172.21.15.159:0/1994448621 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe5a80ad9c0 con 0x7fe5a80af2e0 2024-04-04T06:06:51.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.867+0000 7fe59fe00640 1 -- 172.21.15.159:0/1994448621 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fe59802f050 con 0x7fe5a80af2e0 2024-04-04T06:06:51.861 INFO:teuthology.orchestra.run.smithi159.stderr:listed 0 entries 2024-04-04T06:06:51.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.870+0000 7fe5ae400640 1 -- 172.21.15.159:0/1994448621 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe58406c8f0 msgr2=0x7fe58406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:51.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.870+0000 7fe5ae400640 1 --2- 172.21.15.159:0/1994448621 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe58406c8f0 0x7fe58406edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8147ee0 tx=0x7fe590008040 comp rx=0 tx=0).stop 2024-04-04T06:06:51.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.870+0000 7fe5ae400640 1 -- 172.21.15.159:0/1994448621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5a80af2e0 msgr2=0x7fe5a81479a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:51.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.870+0000 7fe5ae400640 1 --2- 172.21.15.159:0/1994448621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5a80af2e0 0x7fe5a81479a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe598034040 tx=0x7fe5980046c0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.870+0000 7fe5ae400640 1 -- 172.21.15.159:0/1994448621 shutdown_connections 2024-04-04T06:06:51.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.870+0000 7fe5ae400640 1 --2- 172.21.15.159:0/1994448621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5a8153580 0x7fe5a80cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.870+0000 7fe5ae400640 1 --2- 172.21.15.159:0/1994448621 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe58406c8f0 0x7fe58406edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.870+0000 7fe5ae400640 1 --2- 172.21.15.159:0/1994448621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5a80d07c0 0x7fe5a814c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.870+0000 7fe5ae400640 1 --2- 172.21.15.159:0/1994448621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5a80af2e0 0x7fe5a81479a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:51.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.870+0000 7fe5ae400640 1 -- 172.21.15.159:0/1994448621 >> 172.21.15.159:0/1994448621 conn(0x7fe5a806b750 msgr2=0x7fe5a815b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:51.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.870+0000 7fe5ae400640 1 -- 172.21.15.159:0/1994448621 shutdown_connections 2024-04-04T06:06:51.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:51.870+0000 7fe5ae400640 1 -- 172.21.15.159:0/1994448621 wait complete. 2024-04-04T06:06:51.885 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-04T06:06:51.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:51.885 DEBUG:teuthology.orchestra.run.smithi178:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-04T06:06:51.989 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi178.front.sepia.ceph.com 2024-04-04T06:06:51.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:06:51.989 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-04T06:06:52.034 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-04T06:06:52.034 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:06:52.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.249+0000 7f9637c00640 1 Processor -- start 2024-04-04T06:06:52.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.249+0000 7f9637c00640 1 -- start start 2024-04-04T06:06:52.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.251+0000 7f9637c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f96300d0d60 0x7f9630073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:52.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.251+0000 7f9637c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f963015fe80 con 0x7f96300af830 2024-04-04T06:06:52.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.251+0000 7f9637c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96300ad120 con 0x7f963015c390 2024-04-04T06:06:52.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.251+0000 7f9637c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f963015c770 con 0x7f96300d0d60 2024-04-04T06:06:52.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.251+0000 7f9627e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f96300af830 0x7f96300d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:32954/0 (socket says 172.21.15.159:32954) 2024-04-04T06:06:52.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.251+0000 7f9627e00640 1 -- 172.21.15.159:0/3412660373 learned_addr learned my addr 172.21.15.159:0/3412660373 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:52.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.251+0000 7f962fe00640 1 -- 172.21.15.159:0/3412660373 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1331038372 0 0) 0x7f963015fe80 con 0x7f96300af830 2024-04-04T06:06:52.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.251+0000 7f962fe00640 1 -- 172.21.15.159:0/3412660373 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9614003520 con 0x7f96300af830 2024-04-04T06:06:52.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.252+0000 7f9635c00640 1 --2- 172.21.15.159:0/3412660373 >> v2:172.21.15.178:3300/0 conn(0x7f96300d0d60 0x7f9630073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:52.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.252+0000 7f962fe00640 1 -- 172.21.15.159:0/3412660373 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2881711127 0 0) 0x7f9614003520 con 0x7f96300af830 2024-04-04T06:06:52.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.252+0000 7f962fe00640 1 -- 172.21.15.159:0/3412660373 >> v2:172.21.15.178:3300/0 conn(0x7f96300d0d60 msgr2=0x7f9630073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:52.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.252+0000 7f962fe00640 1 --2- 172.21.15.159:0/3412660373 >> v2:172.21.15.178:3300/0 conn(0x7f96300d0d60 0x7f9630073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.252+0000 7f962fe00640 1 -- 172.21.15.159:0/3412660373 >> v1:172.21.15.178:6789/0 conn(0x7f963015c390 legacy=0x7f9630160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:52.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.252+0000 7f962fe00640 1 -- 172.21.15.159:0/3412660373 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96300ad400 con 0x7f96300af830 2024-04-04T06:06:52.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.252+0000 7f962fe00640 1 -- 172.21.15.159:0/3412660373 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f9618001690 con 0x7f96300af830 2024-04-04T06:06:52.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.252+0000 7f962fe00640 1 -- 172.21.15.159:0/3412660373 >> v1:172.21.15.159:6789/0 conn(0x7f96300af830 legacy=0x7f96300d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:52.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.252+0000 7f962fe00640 1 --2- 172.21.15.159:0/3412660373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9614003a00 0x7f9614003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:52.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.252+0000 7f962fe00640 1 --2- 172.21.15.159:0/3412660373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9614004800 0x7f9614004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:52.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.252+0000 7f962fe00640 1 --2- 172.21.15.159:0/3412660373 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9614005210 0x7f961400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:52.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.252+0000 7f962fe00640 1 -- 172.21.15.159:0/3412660373 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f961400c130 con 0x7f9614003a00 2024-04-04T06:06:52.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.252+0000 7f962fe00640 1 -- 172.21.15.159:0/3412660373 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f961400c2e0 con 0x7f9614004800 2024-04-04T06:06:52.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.252+0000 7f962fe00640 1 -- 172.21.15.159:0/3412660373 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f961400c4e0 con 0x7f9614005210 2024-04-04T06:06:52.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.252+0000 7f9635200640 1 --2- 172.21.15.159:0/3412660373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9614003a00 0x7f9614003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:52.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.253+0000 7f9627e00640 1 --2- 172.21.15.159:0/3412660373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9614004800 0x7f9614004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:52.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.253+0000 7f9635200640 1 -- 172.21.15.159:0/3412660373 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9614005210 msgr2=0x7f961400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:52.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.253+0000 7f9635200640 1 --2- 172.21.15.159:0/3412660373 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9614005210 0x7f961400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.253+0000 7f9635200640 1 -- 172.21.15.159:0/3412660373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9614004800 msgr2=0x7f9614004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:52.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.253+0000 7f9635200640 1 --2- 172.21.15.159:0/3412660373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9614004800 0x7f9614004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.253+0000 7f9635200640 1 -- 172.21.15.159:0/3412660373 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9614003520 con 0x7f9614003a00 2024-04-04T06:06:52.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.253+0000 7f9635c00640 1 -- 172.21.15.159:0/3412660373 reap_dead start 2024-04-04T06:06:52.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.254+0000 7f9635200640 1 --2- 172.21.15.159:0/3412660373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9614003a00 0x7f9614003e20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9620001e50 tx=0x7f9614005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55f07190e1c9f75 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:52.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.254+0000 7f962fe00640 1 -- 172.21.15.159:0/3412660373 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620001a70 con 0x7f9614003a00 2024-04-04T06:06:52.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.254+0000 7f962fe00640 1 -- 172.21.15.159:0/3412660373 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9620003b70 con 0x7f9614003a00 2024-04-04T06:06:52.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.254+0000 7f962fe00640 1 -- 172.21.15.159:0/3412660373 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96200050d0 con 0x7f9614003a00 2024-04-04T06:06:52.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.254+0000 7f9637c00640 1 -- 172.21.15.159:0/3412660373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9614003a00 msgr2=0x7f9614003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:52.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.254+0000 7f9637c00640 1 --2- 172.21.15.159:0/3412660373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9614003a00 0x7f9614003e20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9620001e50 tx=0x7f9614005830 comp rx=0 tx=0).stop 2024-04-04T06:06:52.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.255+0000 7f9637c00640 1 -- 172.21.15.159:0/3412660373 shutdown_connections 2024-04-04T06:06:52.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.255+0000 7f9637c00640 1 --2- 172.21.15.159:0/3412660373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9614003a00 0x7f9614003e20 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.255+0000 7f9637c00640 1 -- 172.21.15.159:0/3412660373 >> 172.21.15.159:0/3412660373 conn(0x7f963006b990 msgr2=0x7f963006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:52.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.255+0000 7f9637c00640 1 -- 172.21.15.159:0/3412660373 shutdown_connections 2024-04-04T06:06:52.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.255+0000 7f9637c00640 1 -- 172.21.15.159:0/3412660373 wait complete. 2024-04-04T06:06:52.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.256+0000 7f9637c00640 1 Processor -- start 2024-04-04T06:06:52.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.257+0000 7f9637c00640 1 -- start start 2024-04-04T06:06:52.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.257+0000 7f9637c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9614003a00 0x7f963015adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:52.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.257+0000 7f9637c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f963014fe70 0x7f9630159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:52.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.257+0000 7f9637c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9630159bc0 0x7f963015d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:52.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.257+0000 7f9637c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f96300cf980 con 0x7f9614003a00 2024-04-04T06:06:52.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.257+0000 7f9637c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9630160120 con 0x7f9630159bc0 2024-04-04T06:06:52.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.257+0000 7f9637c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9630073790 con 0x7f963014fe70 2024-04-04T06:06:52.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.258+0000 7f9635200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9614003a00 0x7f963015adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:52.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.258+0000 7f9635200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9614003a00 0x7f963015adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41306/0 (socket says 172.21.15.159:41306) 2024-04-04T06:06:52.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.258+0000 7f9635200640 1 -- 172.21.15.159:0/2767576766 learned_addr learned my addr 172.21.15.159:0/2767576766 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:52.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.258+0000 7f9635c00640 1 --2- 172.21.15.159:0/2767576766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9630159bc0 0x7f963015d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:52.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.258+0000 7f9627e00640 1 --2- 172.21.15.159:0/2767576766 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f963014fe70 0x7f9630159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:52.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.258+0000 7f9635200640 1 -- 172.21.15.159:0/2767576766 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f963014fe70 msgr2=0x7f9630159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:52.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.258+0000 7f9635200640 1 --2- 172.21.15.159:0/2767576766 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f963014fe70 0x7f9630159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.258+0000 7f9635200640 1 -- 172.21.15.159:0/2767576766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9630159bc0 msgr2=0x7f963015d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:52.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.258+0000 7f9635200640 1 --2- 172.21.15.159:0/2767576766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9630159bc0 0x7f963015d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.258+0000 7f9635200640 1 -- 172.21.15.159:0/2767576766 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963015c7a0 con 0x7f9614003a00 2024-04-04T06:06:52.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.258+0000 7f9635200640 1 --2- 172.21.15.159:0/2767576766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9614003a00 0x7f963015adb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9620001f70 tx=0x7f96200014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:52.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.259+0000 7f962d600640 1 -- 172.21.15.159:0/2767576766 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620003970 con 0x7f9614003a00 2024-04-04T06:06:52.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.259+0000 7f962d600640 1 -- 172.21.15.159:0/2767576766 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9620006b50 con 0x7f9614003a00 2024-04-04T06:06:52.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.259+0000 7f9637c00640 1 -- 172.21.15.159:0/2767576766 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963015ca80 con 0x7f9614003a00 2024-04-04T06:06:52.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.259+0000 7f962d600640 1 -- 172.21.15.159:0/2767576766 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620007030 con 0x7f9614003a00 2024-04-04T06:06:52.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.259+0000 7f9637c00640 1 -- 172.21.15.159:0/2767576766 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963015d320 con 0x7f9614003a00 2024-04-04T06:06:52.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.260+0000 7f9637c00640 1 -- 172.21.15.159:0/2767576766 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96300685c0 con 0x7f9614003a00 2024-04-04T06:06:52.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.261+0000 7f962d600640 1 -- 172.21.15.159:0/2767576766 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96200071d0 con 0x7f9614003a00 2024-04-04T06:06:52.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.262+0000 7f962d600640 1 --2- 172.21.15.159:0/2767576766 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f960c06c920 0x7f960c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:52.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.262+0000 7f962d600640 1 -- 172.21.15.159:0/2767576766 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f96200ad7e0 con 0x7f9614003a00 2024-04-04T06:06:52.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.263+0000 7f9627e00640 1 --2- 172.21.15.159:0/2767576766 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f960c06c920 0x7f960c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:52.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.264+0000 7f9627e00640 1 --2- 172.21.15.159:0/2767576766 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f960c06c920 0x7f960c06ede0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9630157f50 tx=0x7f9618009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:52.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.267+0000 7f962d600640 1 -- 172.21.15.159:0/2767576766 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9620033020 con 0x7f9614003a00 2024-04-04T06:06:52.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.511+0000 7f9637c00640 1 -- 172.21.15.159:0/2767576766 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f963006f1b0 con 0x7f9614003a00 2024-04-04T06:06:52.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.513+0000 7f962d600640 1 -- 172.21.15.159:0/2767576766 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5184 (secure 0 0 0) 0x7f962007bac0 con 0x7f9614003a00 2024-04-04T06:06:52.507 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:52.507 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/789759868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":789759868},{"type":"v1","addr":"172.21.15.178:6836","nonce":789759868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2438288045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2438288045},{"type":"v1","addr":"172.21.15.159:6833","nonce":2438288045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/1705205607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1705205607},{"type":"v1","addr":"172.21.15.159:6835","nonce":1705205607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:05:52.801529+0000","modified":"2024-04-04T06:05:55.866207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6837/3362141613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":3362141613},{"type":"v1","addr":"172.21.15.178:6837","nonce":3362141613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-04T06:06:52.507 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 8 2024-04-04T06:06:52.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.515+0000 7f9637c00640 1 -- 172.21.15.159:0/2767576766 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f960c06c920 msgr2=0x7f960c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:52.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.515+0000 7f9637c00640 1 --2- 172.21.15.159:0/2767576766 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f960c06c920 0x7f960c06ede0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9630157f50 tx=0x7f9618009040 comp rx=0 tx=0).stop 2024-04-04T06:06:52.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.515+0000 7f9637c00640 1 -- 172.21.15.159:0/2767576766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9614003a00 msgr2=0x7f963015adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:52.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.516+0000 7f9637c00640 1 --2- 172.21.15.159:0/2767576766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9614003a00 0x7f963015adb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9620001f70 tx=0x7f96200014a0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.516+0000 7f9637c00640 1 -- 172.21.15.159:0/2767576766 shutdown_connections 2024-04-04T06:06:52.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.516+0000 7f9637c00640 1 --2- 172.21.15.159:0/2767576766 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f960c06c920 0x7f960c06ede0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.516+0000 7f9637c00640 1 --2- 172.21.15.159:0/2767576766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9630159bc0 0x7f963015d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.516+0000 7f9637c00640 1 --2- 172.21.15.159:0/2767576766 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f963014fe70 0x7f9630159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.516+0000 7f9637c00640 1 --2- 172.21.15.159:0/2767576766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9614003a00 0x7f963015adb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.516+0000 7f9637c00640 1 -- 172.21.15.159:0/2767576766 >> 172.21.15.159:0/2767576766 conn(0x7f963006b990 msgr2=0x7f96300b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:52.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.516+0000 7f9637c00640 1 -- 172.21.15.159:0/2767576766 shutdown_connections 2024-04-04T06:06:52.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.516+0000 7f9637c00640 1 -- 172.21.15.159:0/2767576766 wait complete. 2024-04-04T06:06:52.529 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:06:52.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.739+0000 7fdfd3200640 1 Processor -- start 2024-04-04T06:06:52.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.739+0000 7fdfd3200640 1 -- start start 2024-04-04T06:06:52.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.740+0000 7fdfd3200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fdfd41442c0 0x7fdfd40ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:52.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.740+0000 7fdfd3200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdfd40cefe0 0x7fdfd40cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:52.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.740+0000 7fdfd3200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdfd40d00d0 0x7fdfd406e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:52.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.740+0000 7fdfd3200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd4071d20 con 0x7fdfd41442c0 2024-04-04T06:06:52.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.740+0000 7fdfd3200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40cf420 con 0x7fdfd40cefe0 2024-04-04T06:06:52.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.740+0000 7fdfd3200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd4071460 con 0x7fdfd40d00d0 2024-04-04T06:06:52.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.741+0000 7fdfd1e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fdfd41442c0 0x7fdfd40ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:52.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.741+0000 7fdfd1e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fdfd41442c0 0x7fdfd40ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41314/0 (socket says 172.21.15.159:41314) 2024-04-04T06:06:52.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.741+0000 7fdfd1e00640 1 -- 172.21.15.159:0/345211109 learned_addr learned my addr 172.21.15.159:0/345211109 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:52.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.741+0000 7fdfd2800640 1 --2- 172.21.15.159:0/345211109 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdfd40d00d0 0x7fdfd406e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:52.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.741+0000 7fdfd1e00640 1 -- 172.21.15.159:0/345211109 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdfd40d00d0 msgr2=0x7fdfd406e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:52.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.741+0000 7fdfd1e00640 1 --2- 172.21.15.159:0/345211109 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdfd40d00d0 0x7fdfd406e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.741+0000 7fdfd1e00640 1 -- 172.21.15.159:0/345211109 >> v2:172.21.15.178:3300/0 conn(0x7fdfd40cefe0 msgr2=0x7fdfd40cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:52.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.741+0000 7fdfd1e00640 1 --2- 172.21.15.159:0/345211109 >> v2:172.21.15.178:3300/0 conn(0x7fdfd40cefe0 0x7fdfd40cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.741+0000 7fdfd1e00640 1 -- 172.21.15.159:0/345211109 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd40721b0 con 0x7fdfd41442c0 2024-04-04T06:06:52.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.742+0000 7fdfd1e00640 1 --2- 172.21.15.159:0/345211109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd41442c0 0x7fdfd40ceaa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40b2c60 tx=0x7fdfc402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d45e6c43b4ba1fb server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:52.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.743+0000 7fdfd1400640 1 -- 172.21.15.159:0/345211109 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4034070 con 0x7fdfd41442c0 2024-04-04T06:06:52.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.743+0000 7fdfd1400640 1 -- 172.21.15.159:0/345211109 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc4030860 con 0x7fdfd41442c0 2024-04-04T06:06:52.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.743+0000 7fdfd1400640 1 -- 172.21.15.159:0/345211109 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4030c30 con 0x7fdfd41442c0 2024-04-04T06:06:52.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.743+0000 7fdfd3200640 1 -- 172.21.15.159:0/345211109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd41442c0 msgr2=0x7fdfd40ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:52.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.743+0000 7fdfd3200640 1 --2- 172.21.15.159:0/345211109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd41442c0 0x7fdfd40ceaa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40b2c60 tx=0x7fdfc402b9a0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.743+0000 7fdfd3200640 1 -- 172.21.15.159:0/345211109 shutdown_connections 2024-04-04T06:06:52.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.744+0000 7fdfd3200640 1 --2- 172.21.15.159:0/345211109 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdfd40d00d0 0x7fdfd406e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.744+0000 7fdfd3200640 1 --2- 172.21.15.159:0/345211109 >> v2:172.21.15.178:3300/0 conn(0x7fdfd40cefe0 0x7fdfd40cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.744+0000 7fdfd3200640 1 --2- 172.21.15.159:0/345211109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd41442c0 0x7fdfd40ceaa0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.744+0000 7fdfd3200640 1 -- 172.21.15.159:0/345211109 >> 172.21.15.159:0/345211109 conn(0x7fdfd40ccad0 msgr2=0x7fdfd40cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:52.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.744+0000 7fdfd3200640 1 -- 172.21.15.159:0/345211109 shutdown_connections 2024-04-04T06:06:52.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.744+0000 7fdfd3200640 1 -- 172.21.15.159:0/345211109 wait complete. 2024-04-04T06:06:52.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.745+0000 7fdfd3200640 1 Processor -- start 2024-04-04T06:06:52.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.746+0000 7fdfd3200640 1 -- start start 2024-04-04T06:06:52.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.746+0000 7fdfd3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd40cefe0 0x7fdfd4135740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:52.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.746+0000 7fdfd3200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdfd40d00d0 0x7fdfd41e4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:52.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.747+0000 7fdfd3200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfd41442c0 0x7fdfd41e5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:52.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.747+0000 7fdfd3200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd40d0770 con 0x7fdfd40cefe0 2024-04-04T06:06:52.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.747+0000 7fdfd3200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd40719b0 con 0x7fdfd41442c0 2024-04-04T06:06:52.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.747+0000 7fdfd3200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd41446a0 con 0x7fdfd40d00d0 2024-04-04T06:06:52.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.747+0000 7fdfd1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd40cefe0 0x7fdfd4135740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:52.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.747+0000 7fdfd1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd40cefe0 0x7fdfd4135740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41316/0 (socket says 172.21.15.159:41316) 2024-04-04T06:06:52.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.747+0000 7fdfd1e00640 1 -- 172.21.15.159:0/1471448309 learned_addr learned my addr 172.21.15.159:0/1471448309 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:52.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.747+0000 7fdfd1e00640 1 -- 172.21.15.159:0/1471448309 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdfd40d00d0 msgr2=0x7fdfd41e4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:52.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.747+0000 7fdfd2800640 1 --2- 172.21.15.159:0/1471448309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfd41442c0 0x7fdfd41e5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:52.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.747+0000 7fdfd1e00640 1 --2- 172.21.15.159:0/1471448309 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdfd40d00d0 0x7fdfd41e4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.747+0000 7fdfd1e00640 1 -- 172.21.15.159:0/1471448309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfd41442c0 msgr2=0x7fdfd41e5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:52.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.747+0000 7fdfd1e00640 1 --2- 172.21.15.159:0/1471448309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfd41442c0 0x7fdfd41e5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.747+0000 7fdfd1e00640 1 -- 172.21.15.159:0/1471448309 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd41e37a0 con 0x7fdfd40cefe0 2024-04-04T06:06:52.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.748+0000 7fdfd1e00640 1 --2- 172.21.15.159:0/1471448309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd40cefe0 0x7fdfd4135740 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdfc402b9e0 tx=0x7fdfc4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:52.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.748+0000 7fdfca000640 1 -- 172.21.15.159:0/1471448309 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4048070 con 0x7fdfd40cefe0 2024-04-04T06:06:52.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.748+0000 7fdfca000640 1 -- 172.21.15.159:0/1471448309 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc4030660 con 0x7fdfd40cefe0 2024-04-04T06:06:52.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.748+0000 7fdfca000640 1 -- 172.21.15.159:0/1471448309 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4034040 con 0x7fdfd40cefe0 2024-04-04T06:06:52.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.748+0000 7fdfd3200640 1 -- 172.21.15.159:0/1471448309 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd4135c80 con 0x7fdfd40cefe0 2024-04-04T06:06:52.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.748+0000 7fdfd3200640 1 -- 172.21.15.159:0/1471448309 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd41e3a10 con 0x7fdfd40cefe0 2024-04-04T06:06:52.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.749+0000 7fdfd3200640 1 -- 172.21.15.159:0/1471448309 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd4075940 con 0x7fdfd40cefe0 2024-04-04T06:06:52.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.750+0000 7fdfca000640 1 -- 172.21.15.159:0/1471448309 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdfc404b070 con 0x7fdfd40cefe0 2024-04-04T06:06:52.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.751+0000 7fdfca000640 1 --2- 172.21.15.159:0/1471448309 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdfb006c8f0 0x7fdfb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:52.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.752+0000 7fdfca000640 1 -- 172.21.15.159:0/1471448309 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7fdfc40adda0 con 0x7fdfd40cefe0 2024-04-04T06:06:52.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.752+0000 7fdfcbe00640 1 --2- 172.21.15.159:0/1471448309 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdfb006c8f0 0x7fdfb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:52.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.753+0000 7fdfcbe00640 1 --2- 172.21.15.159:0/1471448309 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdfb006c8f0 0x7fdfb006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4143680 tx=0x7fdfbc006bf0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:52.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.756+0000 7fdfca000640 1 -- 172.21.15.159:0/1471448309 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fdfc407c1a0 con 0x7fdfd40cefe0 2024-04-04T06:06:52.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.959+0000 7fdfd3200640 1 -- 172.21.15.159:0/1471448309 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfd406db00 con 0x7fdfd40cefe0 2024-04-04T06:06:52.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.962+0000 7fdfca000640 1 -- 172.21.15.159:0/1471448309 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16958 (secure 0 0 0) 0x7fdfc407baa0 con 0x7fdfd40cefe0 2024-04-04T06:06:52.957 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:52.957 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":23,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:06:33.796979+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:05:51.297293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-04T06:05:51.770792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":0,"to":2}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:06:52.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.964+0000 7fdfd3200640 1 -- 172.21.15.159:0/1471448309 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdfb006c8f0 msgr2=0x7fdfb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:52.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.964+0000 7fdfd3200640 1 --2- 172.21.15.159:0/1471448309 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdfb006c8f0 0x7fdfb006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4143680 tx=0x7fdfbc006bf0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.964+0000 7fdfd3200640 1 -- 172.21.15.159:0/1471448309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd40cefe0 msgr2=0x7fdfd4135740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:52.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.964+0000 7fdfd3200640 1 --2- 172.21.15.159:0/1471448309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd40cefe0 0x7fdfd4135740 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdfc402b9e0 tx=0x7fdfc4004730 comp rx=0 tx=0).stop 2024-04-04T06:06:52.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.965+0000 7fdfd3200640 1 -- 172.21.15.159:0/1471448309 shutdown_connections 2024-04-04T06:06:52.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.965+0000 7fdfd3200640 1 --2- 172.21.15.159:0/1471448309 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdfb006c8f0 0x7fdfb006edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.965+0000 7fdfd3200640 1 --2- 172.21.15.159:0/1471448309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfd41442c0 0x7fdfd41e5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.965+0000 7fdfd3200640 1 --2- 172.21.15.159:0/1471448309 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdfd40d00d0 0x7fdfd41e4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.965+0000 7fdfd3200640 1 --2- 172.21.15.159:0/1471448309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd40cefe0 0x7fdfd4135740 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:52.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.965+0000 7fdfd3200640 1 -- 172.21.15.159:0/1471448309 >> 172.21.15.159:0/1471448309 conn(0x7fdfd40ccad0 msgr2=0x7fdfd40afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:52.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.965+0000 7fdfd3200640 1 -- 172.21.15.159:0/1471448309 shutdown_connections 2024-04-04T06:06:52.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:52.965+0000 7fdfd3200640 1 -- 172.21.15.159:0/1471448309 wait complete. 2024-04-04T06:06:52.979 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:05:51.297293+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:05:51.770792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:06:52.979 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:06:53.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.200+0000 7f095b000640 1 Processor -- start 2024-04-04T06:06:53.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.200+0000 7f095b000640 1 -- start start 2024-04-04T06:06:53.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.201+0000 7f095b000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0954153580 0x7f09540d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:53.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.201+0000 7f095b000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f09540d07c0 0x7f09540aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:53.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.201+0000 7f095b000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f09540af2e0 0x7f095406d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:53.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.201+0000 7f095b000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f09540d0c00 con 0x7f09540d07c0 2024-04-04T06:06:53.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.201+0000 7f095b000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f09540afab0 con 0x7f09540af2e0 2024-04-04T06:06:53.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.201+0000 7f095b000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f095415cc80 con 0x7f0954153580 2024-04-04T06:06:53.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.201+0000 7f094b400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f09540d07c0 0x7f09540aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:53.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.202+0000 7f094b400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f09540d07c0 0x7f09540aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41318/0 (socket says 172.21.15.159:41318) 2024-04-04T06:06:53.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.202+0000 7f094b400640 1 -- 172.21.15.159:0/4205276400 learned_addr learned my addr 172.21.15.159:0/4205276400 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:53.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.202+0000 7f0953e00640 1 --2- 172.21.15.159:0/4205276400 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0954153580 0x7f09540d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:53.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.202+0000 7f094b400640 1 -- 172.21.15.159:0/4205276400 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0954153580 msgr2=0x7f09540d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:53.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.202+0000 7f094b400640 1 --2- 172.21.15.159:0/4205276400 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0954153580 0x7f09540d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:53.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.202+0000 7f094b400640 1 -- 172.21.15.159:0/4205276400 >> v2:172.21.15.178:3300/0 conn(0x7f09540af2e0 msgr2=0x7f095406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:53.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.202+0000 7f094b400640 1 --2- 172.21.15.159:0/4205276400 >> v2:172.21.15.178:3300/0 conn(0x7f09540af2e0 0x7f095406d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:53.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.202+0000 7f094b400640 1 -- 172.21.15.159:0/4205276400 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095415da00 con 0x7f09540d07c0 2024-04-04T06:06:53.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.203+0000 7f094b400640 1 --2- 172.21.15.159:0/4205276400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09540d07c0 0x7f09540aeda0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f09540cff70 tx=0x7f093c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117a09d1804d5ea3 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:53.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.203+0000 7f0953400640 1 -- 172.21.15.159:0/4205276400 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c02e070 con 0x7f09540d07c0 2024-04-04T06:06:53.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.203+0000 7f0953400640 1 -- 172.21.15.159:0/4205276400 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f093c037070 con 0x7f09540d07c0 2024-04-04T06:06:53.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.203+0000 7f0953400640 1 -- 172.21.15.159:0/4205276400 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c031a00 con 0x7f09540d07c0 2024-04-04T06:06:53.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.204+0000 7f095b000640 1 -- 172.21.15.159:0/4205276400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09540d07c0 msgr2=0x7f09540aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:53.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.204+0000 7f095b000640 1 --2- 172.21.15.159:0/4205276400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09540d07c0 0x7f09540aeda0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f09540cff70 tx=0x7f093c02b880 comp rx=0 tx=0).stop 2024-04-04T06:06:53.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.204+0000 7f095b000640 1 -- 172.21.15.159:0/4205276400 shutdown_connections 2024-04-04T06:06:53.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.204+0000 7f095b000640 1 --2- 172.21.15.159:0/4205276400 >> v2:172.21.15.178:3300/0 conn(0x7f09540af2e0 0x7f095406d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:53.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.204+0000 7f095b000640 1 --2- 172.21.15.159:0/4205276400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09540d07c0 0x7f09540aeda0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:53.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.204+0000 7f095b000640 1 --2- 172.21.15.159:0/4205276400 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0954153580 0x7f09540d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:53.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.204+0000 7f095b000640 1 -- 172.21.15.159:0/4205276400 >> 172.21.15.159:0/4205276400 conn(0x7f095406b750 msgr2=0x7f095406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:53.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.204+0000 7f095b000640 1 -- 172.21.15.159:0/4205276400 shutdown_connections 2024-04-04T06:06:53.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.204+0000 7f095b000640 1 -- 172.21.15.159:0/4205276400 wait complete. 2024-04-04T06:06:53.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.206+0000 7f095b000640 1 Processor -- start 2024-04-04T06:06:53.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.206+0000 7f095b000640 1 -- start start 2024-04-04T06:06:53.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.207+0000 7f095b000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09540af2e0 0x7f0954150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:53.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.207+0000 7f095b000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09540d07c0 0x7f09541f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:53.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.207+0000 7f095b000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0954153580 0x7f09540ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:53.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.207+0000 7f095b000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f095415d4e0 con 0x7f0954153580 2024-04-04T06:06:53.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.207+0000 7f095b000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f095415d200 con 0x7f09540af2e0 2024-04-04T06:06:53.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.207+0000 7f095b000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0954153960 con 0x7f09540d07c0 2024-04-04T06:06:53.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.207+0000 7f0959000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0954153580 0x7f09540ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:53.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.207+0000 7f0959000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0954153580 0x7f09540ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41326/0 (socket says 172.21.15.159:41326) 2024-04-04T06:06:53.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.207+0000 7f0959000640 1 -- 172.21.15.159:0/206860349 learned_addr learned my addr 172.21.15.159:0/206860349 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:53.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.207+0000 7f094b400640 1 --2- 172.21.15.159:0/206860349 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09540d07c0 0x7f09541f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:53.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.208+0000 7f0953e00640 1 --2- 172.21.15.159:0/206860349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09540af2e0 0x7f0954150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:53.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.208+0000 7f0959000640 1 -- 172.21.15.159:0/206860349 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09540d07c0 msgr2=0x7f09541f31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:53.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.208+0000 7f0959000640 1 --2- 172.21.15.159:0/206860349 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09540d07c0 0x7f09541f31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:53.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.208+0000 7f0959000640 1 -- 172.21.15.159:0/206860349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09540af2e0 msgr2=0x7f0954150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:53.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.208+0000 7f0959000640 1 --2- 172.21.15.159:0/206860349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09540af2e0 0x7f0954150ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:53.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.208+0000 7f0959000640 1 -- 172.21.15.159:0/206860349 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09540cd0e0 con 0x7f0954153580 2024-04-04T06:06:53.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.208+0000 7f094b400640 1 --2- 172.21.15.159:0/206860349 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09540d07c0 0x7f09541f31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:06:53.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.208+0000 7f0959000640 1 --2- 172.21.15.159:0/206860349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0954153580 0x7f09540ceeb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f094c00b970 tx=0x7f094c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:53.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.208+0000 7f0950c00640 1 -- 172.21.15.159:0/206860349 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c019070 con 0x7f0954153580 2024-04-04T06:06:53.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.208+0000 7f0950c00640 1 -- 172.21.15.159:0/206860349 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f094c00cb60 con 0x7f0954153580 2024-04-04T06:06:53.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.209+0000 7f0950c00640 1 -- 172.21.15.159:0/206860349 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c014440 con 0x7f0954153580 2024-04-04T06:06:53.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.209+0000 7f095b000640 1 -- 172.21.15.159:0/206860349 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09540cd3c0 con 0x7f0954153580 2024-04-04T06:06:53.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.209+0000 7f095b000640 1 -- 172.21.15.159:0/206860349 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09540cd980 con 0x7f0954153580 2024-04-04T06:06:53.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.210+0000 7f095b000640 1 -- 172.21.15.159:0/206860349 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09540685c0 con 0x7f0954153580 2024-04-04T06:06:53.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.211+0000 7f0950c00640 1 -- 172.21.15.159:0/206860349 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f094c01c070 con 0x7f0954153580 2024-04-04T06:06:53.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.212+0000 7f0950c00640 1 --2- 172.21.15.159:0/206860349 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f093006c920 0x7f093006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:53.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.212+0000 7f0950c00640 1 -- 172.21.15.159:0/206860349 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f094c08b9d0 con 0x7f0954153580 2024-04-04T06:06:53.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.213+0000 7f0953e00640 1 --2- 172.21.15.159:0/206860349 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f093006c920 0x7f093006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:53.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.214+0000 7f0953e00640 1 --2- 172.21.15.159:0/206860349 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f093006c920 0x7f093006ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0944006fd0 tx=0x7f0944008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:53.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.217+0000 7f0950c00640 1 -- 172.21.15.159:0/206860349 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f094c059cb0 con 0x7f0954153580 2024-04-04T06:06:53.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.418+0000 7f095b000640 1 -- 172.21.15.159:0/206860349 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f095415f3c0 con 0x7f0954153580 2024-04-04T06:06:53.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.419+0000 7f0950c00640 1 -- 172.21.15.159:0/206860349 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f094c0595b0 con 0x7f0954153580 2024-04-04T06:06:53.413 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:06:53.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.421+0000 7f095b000640 1 -- 172.21.15.159:0/206860349 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f093006c920 msgr2=0x7f093006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:53.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.421+0000 7f095b000640 1 --2- 172.21.15.159:0/206860349 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f093006c920 0x7f093006ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0944006fd0 tx=0x7f0944008040 comp rx=0 tx=0).stop 2024-04-04T06:06:53.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.421+0000 7f095b000640 1 -- 172.21.15.159:0/206860349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0954153580 msgr2=0x7f09540ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:53.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.421+0000 7f095b000640 1 --2- 172.21.15.159:0/206860349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0954153580 0x7f09540ceeb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f094c00b970 tx=0x7f094c00be30 comp rx=0 tx=0).stop 2024-04-04T06:06:53.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.421+0000 7f095b000640 1 -- 172.21.15.159:0/206860349 shutdown_connections 2024-04-04T06:06:53.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.421+0000 7f095b000640 1 --2- 172.21.15.159:0/206860349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0954153580 0x7f09540ceeb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:53.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.421+0000 7f095b000640 1 --2- 172.21.15.159:0/206860349 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09540d07c0 0x7f09541f31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:53.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.421+0000 7f095b000640 1 --2- 172.21.15.159:0/206860349 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f093006c920 0x7f093006ede0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:53.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.421+0000 7f095b000640 1 --2- 172.21.15.159:0/206860349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09540af2e0 0x7f0954150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:53.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.421+0000 7f095b000640 1 -- 172.21.15.159:0/206860349 >> 172.21.15.159:0/206860349 conn(0x7f095406b750 msgr2=0x7f09540b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:53.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.422+0000 7f095b000640 1 -- 172.21.15.159:0/206860349 shutdown_connections 2024-04-04T06:06:53.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.422+0000 7f095b000640 1 -- 172.21.15.159:0/206860349 wait complete. 2024-04-04T06:06:53.433 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:06:53.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.641+0000 7f6b64400640 1 Processor -- start 2024-04-04T06:06:53.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.642+0000 7f6b64400640 1 -- start start 2024-04-04T06:06:53.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.643+0000 7f6b64400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6b5c0af4b0 0x7f6b5c167460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:53.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.643+0000 7f6b64400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b5c0afab0 con 0x7f6b5c0d0990 2024-04-04T06:06:53.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.643+0000 7f6b64400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b5c15cc80 con 0x7f6b5c153580 2024-04-04T06:06:53.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.643+0000 7f6b64400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f6b5c0af890 con 0x7f6b5c0af4b0 2024-04-04T06:06:53.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.643+0000 7f6b61000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f6b5c0d0990 0x7f6b5c0aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:32968/0 (socket says 172.21.15.159:32968) 2024-04-04T06:06:53.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.643+0000 7f6b61000640 1 -- 172.21.15.159:0/3509593749 learned_addr learned my addr 172.21.15.159:0/3509593749 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:53.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.644+0000 7f6b5be00640 1 -- 172.21.15.159:0/3509593749 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3259852239 0 0) 0x7f6b5c0afab0 con 0x7f6b5c0d0990 2024-04-04T06:06:53.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.644+0000 7f6b5be00640 1 -- 172.21.15.159:0/3509593749 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b44003520 con 0x7f6b5c0d0990 2024-04-04T06:06:53.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.644+0000 7f6b62400640 1 --2- 172.21.15.159:0/3509593749 >> v2:172.21.15.178:3300/0 conn(0x7f6b5c0af4b0 0x7f6b5c167460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:53.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.644+0000 7f6b5be00640 1 -- 172.21.15.159:0/3509593749 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 326407114 0 0) 0x7f6b44003520 con 0x7f6b5c0d0990 2024-04-04T06:06:53.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.644+0000 7f6b5be00640 1 -- 172.21.15.159:0/3509593749 >> v2:172.21.15.178:3300/0 conn(0x7f6b5c0af4b0 msgr2=0x7f6b5c167460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:53.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.644+0000 7f6b5be00640 1 --2- 172.21.15.159:0/3509593749 >> v2:172.21.15.178:3300/0 conn(0x7f6b5c0af4b0 0x7f6b5c167460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:53.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.644+0000 7f6b5be00640 1 -- 172.21.15.159:0/3509593749 >> v1:172.21.15.178:6789/0 conn(0x7f6b5c153580 legacy=0x7f6b5c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:53.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.644+0000 7f6b5be00640 1 -- 172.21.15.159:0/3509593749 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5c15da00 con 0x7f6b5c0d0990 2024-04-04T06:06:53.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.644+0000 7f6b5be00640 1 -- 172.21.15.159:0/3509593749 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f6b48001670 con 0x7f6b5c0d0990 2024-04-04T06:06:53.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.644+0000 7f6b5be00640 1 -- 172.21.15.159:0/3509593749 >> v1:172.21.15.159:6789/0 conn(0x7f6b5c0d0990 legacy=0x7f6b5c0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:53.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.644+0000 7f6b5be00640 1 --2- 172.21.15.159:0/3509593749 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b440039b0 0x7f6b44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:53.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.645+0000 7f6b5be00640 1 --2- 172.21.15.159:0/3509593749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b44004800 0x7f6b44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:53.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.645+0000 7f6b5be00640 1 --2- 172.21.15.159:0/3509593749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b440052e0 0x7f6b4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:53.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.645+0000 7f6b5be00640 1 -- 172.21.15.159:0/3509593749 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c0af890 con 0x7f6b44004800 2024-04-04T06:06:53.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.645+0000 7f6b5be00640 1 -- 172.21.15.159:0/3509593749 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4400c1b0 con 0x7f6b440052e0 2024-04-04T06:06:53.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.645+0000 7f6b5be00640 1 -- 172.21.15.159:0/3509593749 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6b4400c3b0 con 0x7f6b440039b0 2024-04-04T06:06:53.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.645+0000 7f6b61000640 1 --2- 172.21.15.159:0/3509593749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b44004800 0x7f6b44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:53.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.645+0000 7f6b61a00640 1 --2- 172.21.15.159:0/3509593749 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b440039b0 0x7f6b44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:53.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.645+0000 7f6b62400640 1 --2- 172.21.15.159:0/3509593749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b440052e0 0x7f6b4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:53.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.645+0000 7f6b61000640 1 -- 172.21.15.159:0/3509593749 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b440039b0 msgr2=0x7f6b44003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:53.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.645+0000 7f6b61000640 1 --2- 172.21.15.159:0/3509593749 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b440039b0 0x7f6b44003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:53.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.645+0000 7f6b61000640 1 -- 172.21.15.159:0/3509593749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b440052e0 msgr2=0x7f6b4400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:53.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.645+0000 7f6b61000640 1 --2- 172.21.15.159:0/3509593749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b440052e0 0x7f6b4400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:53.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.645+0000 7f6b61000640 1 -- 172.21.15.159:0/3509593749 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b44003520 con 0x7f6b44004800 2024-04-04T06:06:53.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.645+0000 7f6b62400640 1 --2- 172.21.15.159:0/3509593749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b440052e0 0x7f6b4400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:53.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.645+0000 7f6b62400640 1 -- 172.21.15.159:0/3509593749 reap_dead start 2024-04-04T06:06:53.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.646+0000 7f6b61000640 1 --2- 172.21.15.159:0/3509593749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b44004800 0x7f6b44004c80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6b48001eb0 tx=0x7f6b44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9280aa64309b6de server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:53.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.647+0000 7f6b5be00640 1 -- 172.21.15.159:0/3509593749 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48007d30 con 0x7f6b44004800 2024-04-04T06:06:53.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.647+0000 7f6b5be00640 1 -- 172.21.15.159:0/3509593749 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b48030070 con 0x7f6b44004800 2024-04-04T06:06:53.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.647+0000 7f6b5be00640 1 -- 172.21.15.159:0/3509593749 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48005d70 con 0x7f6b44004800 2024-04-04T06:06:53.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.647+0000 7f6b64400640 1 -- 172.21.15.159:0/3509593749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b44004800 msgr2=0x7f6b44004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:53.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.647+0000 7f6b64400640 1 --2- 172.21.15.159:0/3509593749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b44004800 0x7f6b44004c80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6b48001eb0 tx=0x7f6b44005830 comp rx=0 tx=0).stop 2024-04-04T06:06:53.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.647+0000 7f6b64400640 1 -- 172.21.15.159:0/3509593749 shutdown_connections 2024-04-04T06:06:53.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.647+0000 7f6b64400640 1 --2- 172.21.15.159:0/3509593749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b44004800 0x7f6b44004c80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:53.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.647+0000 7f6b64400640 1 -- 172.21.15.159:0/3509593749 >> 172.21.15.159:0/3509593749 conn(0x7f6b5c06b750 msgr2=0x7f6b5c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:53.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.647+0000 7f6b64400640 1 -- 172.21.15.159:0/3509593749 shutdown_connections 2024-04-04T06:06:53.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.647+0000 7f6b64400640 1 -- 172.21.15.159:0/3509593749 wait complete. 2024-04-04T06:06:53.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.649+0000 7f6b64400640 1 Processor -- start 2024-04-04T06:06:53.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.649+0000 7f6b64400640 1 -- start start 2024-04-04T06:06:53.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.650+0000 7f6b64400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b44004800 0x7f6b5c0ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:53.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.650+0000 7f6b64400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b5c0cf560 0x7f6b5c13d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:53.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.650+0000 7f6b64400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b5c0cf9c0 0x7f6b5c13d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:53.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.650+0000 7f6b64400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c15d4e0 con 0x7f6b44004800 2024-04-04T06:06:53.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.650+0000 7f6b64400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c15d200 con 0x7f6b5c0cf560 2024-04-04T06:06:53.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.650+0000 7f6b64400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6b5c153960 con 0x7f6b5c0cf9c0 2024-04-04T06:06:53.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.650+0000 7f6b61a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b44004800 0x7f6b5c0ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:53.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.650+0000 7f6b61a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b44004800 0x7f6b5c0ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41344/0 (socket says 172.21.15.159:41344) 2024-04-04T06:06:53.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.650+0000 7f6b61a00640 1 -- 172.21.15.159:0/3048472674 learned_addr learned my addr 172.21.15.159:0/3048472674 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:53.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.651+0000 7f6b61a00640 1 -- 172.21.15.159:0/3048472674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b5c0cf9c0 msgr2=0x7f6b5c13d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:53.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.651+0000 7f6b61a00640 1 --2- 172.21.15.159:0/3048472674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b5c0cf9c0 0x7f6b5c13d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:53.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.651+0000 7f6b61a00640 1 -- 172.21.15.159:0/3048472674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b5c0cf560 msgr2=0x7f6b5c13d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:53.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.651+0000 7f6b61a00640 1 --2- 172.21.15.159:0/3048472674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b5c0cf560 0x7f6b5c13d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:53.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.651+0000 7f6b61a00640 1 -- 172.21.15.159:0/3048472674 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5c140140 con 0x7f6b44004800 2024-04-04T06:06:53.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.651+0000 7f6b61a00640 1 --2- 172.21.15.159:0/3048472674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b44004800 0x7f6b5c0ccef0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000e490 tx=0x7f6b5000e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:53.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.651+0000 7f6b59600640 1 -- 172.21.15.159:0/3048472674 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b500052e0 con 0x7f6b44004800 2024-04-04T06:06:53.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.651+0000 7f6b59600640 1 -- 172.21.15.159:0/3048472674 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b50018070 con 0x7f6b44004800 2024-04-04T06:06:53.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.651+0000 7f6b59600640 1 -- 172.21.15.159:0/3048472674 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b50003830 con 0x7f6b44004800 2024-04-04T06:06:53.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.651+0000 7f6b64400640 1 -- 172.21.15.159:0/3048472674 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b5c0cea80 con 0x7f6b44004800 2024-04-04T06:06:53.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.651+0000 7f6b64400640 1 -- 172.21.15.159:0/3048472674 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b5c0cf040 con 0x7f6b44004800 2024-04-04T06:06:53.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.653+0000 7f6b64400640 1 -- 172.21.15.159:0/3048472674 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b5c0685c0 con 0x7f6b44004800 2024-04-04T06:06:53.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.653+0000 7f6b59600640 1 -- 172.21.15.159:0/3048472674 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b500039d0 con 0x7f6b44004800 2024-04-04T06:06:53.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.654+0000 7f6b59600640 1 --2- 172.21.15.159:0/3048472674 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6b3c06c920 0x7f6b3c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:53.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.654+0000 7f6b59600640 1 -- 172.21.15.159:0/3048472674 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f6b5008b9a0 con 0x7f6b44004800 2024-04-04T06:06:53.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.655+0000 7f6b61000640 1 --2- 172.21.15.159:0/3048472674 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6b3c06c920 0x7f6b3c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:53.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.656+0000 7f6b61000640 1 --2- 172.21.15.159:0/3048472674 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6b3c06c920 0x7f6b3c06ede0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6b48034000 tx=0x7f6b48007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:53.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.659+0000 7f6b59600640 1 -- 172.21.15.159:0/3048472674 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6b50059bf0 con 0x7f6b44004800 2024-04-04T06:06:53.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.842+0000 7f6b64400640 1 -- 172.21.15.159:0/3048472674 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6b5c1546f0 con 0x7f6b44004800 2024-04-04T06:06:53.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.843+0000 7f6b59600640 1 -- 172.21.15.159:0/3048472674 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f6b500594f0 con 0x7f6b44004800 2024-04-04T06:06:53.838 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:06:53.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.845+0000 7f6b64400640 1 -- 172.21.15.159:0/3048472674 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6b3c06c920 msgr2=0x7f6b3c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:53.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.845+0000 7f6b64400640 1 --2- 172.21.15.159:0/3048472674 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6b3c06c920 0x7f6b3c06ede0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6b48034000 tx=0x7f6b48007000 comp rx=0 tx=0).stop 2024-04-04T06:06:53.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.845+0000 7f6b64400640 1 -- 172.21.15.159:0/3048472674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b44004800 msgr2=0x7f6b5c0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:53.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.845+0000 7f6b64400640 1 --2- 172.21.15.159:0/3048472674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b44004800 0x7f6b5c0ccef0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000e490 tx=0x7f6b5000e950 comp rx=0 tx=0).stop 2024-04-04T06:06:53.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.846+0000 7f6b64400640 1 -- 172.21.15.159:0/3048472674 shutdown_connections 2024-04-04T06:06:53.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.846+0000 7f6b64400640 1 --2- 172.21.15.159:0/3048472674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b5c0cf9c0 0x7f6b5c13d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:53.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.846+0000 7f6b64400640 1 --2- 172.21.15.159:0/3048472674 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6b3c06c920 0x7f6b3c06ede0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:53.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.846+0000 7f6b64400640 1 --2- 172.21.15.159:0/3048472674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b5c0cf560 0x7f6b5c13d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:53.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.846+0000 7f6b64400640 1 --2- 172.21.15.159:0/3048472674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b44004800 0x7f6b5c0ccef0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:53.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.846+0000 7f6b64400640 1 -- 172.21.15.159:0/3048472674 >> 172.21.15.159:0/3048472674 conn(0x7f6b5c06b750 msgr2=0x7f6b5c0afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:53.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.846+0000 7f6b64400640 1 -- 172.21.15.159:0/3048472674 shutdown_connections 2024-04-04T06:06:53.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:53.846+0000 7f6b64400640 1 -- 172.21.15.159:0/3048472674 wait complete. 2024-04-04T06:06:53.857 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:06:54.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.065+0000 7f2b16800640 1 Processor -- start 2024-04-04T06:06:54.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.065+0000 7f2b16800640 1 -- start start 2024-04-04T06:06:54.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.066+0000 7f2b16800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2b1013f190 0x7f2b100cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:54.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.067+0000 7f2b16800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2b1006dd30 0x7f2b100cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:54.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.067+0000 7f2b16800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b100ce890 con 0x7f2b100cc880 2024-04-04T06:06:54.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.067+0000 7f2b16800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f2b100cd230 con 0x7f2b1006dd30 2024-04-04T06:06:54.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.067+0000 7f2b16800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2b100ceb50 con 0x7f2b1013f190 2024-04-04T06:06:54.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.067+0000 7f2b14a00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f2b100cc880 0x7f2b1006d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58552/0 (socket says 172.21.15.159:58552) 2024-04-04T06:06:54.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.067+0000 7f2b14a00640 1 -- 172.21.15.159:0/2209114261 learned_addr learned my addr 172.21.15.159:0/2209114261 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:54.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.067+0000 7f2b15e00640 1 --2- 172.21.15.159:0/2209114261 >> v2:172.21.15.159:3300/0 conn(0x7f2b1006dd30 0x7f2b100cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:54.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.067+0000 7f2b15400640 1 --2- 172.21.15.159:0/2209114261 >> v2:172.21.15.178:3300/0 conn(0x7f2b1013f190 0x7f2b100cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:54.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.067+0000 7f2b0fe00640 1 -- 172.21.15.159:0/2209114261 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1993053961 0 0) 0x7f2b100ce890 con 0x7f2b100cc880 2024-04-04T06:06:54.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.068+0000 7f2b0fe00640 1 -- 172.21.15.159:0/2209114261 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2af8003520 con 0x7f2b100cc880 2024-04-04T06:06:54.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.068+0000 7f2b15e00640 1 -- 172.21.15.159:0/2209114261 >> v2:172.21.15.178:3300/0 conn(0x7f2b1013f190 msgr2=0x7f2b100cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:54.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.068+0000 7f2b15e00640 1 --2- 172.21.15.159:0/2209114261 >> v2:172.21.15.178:3300/0 conn(0x7f2b1013f190 0x7f2b100cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:54.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.068+0000 7f2b15e00640 1 -- 172.21.15.159:0/2209114261 >> v1:172.21.15.159:6789/0 conn(0x7f2b100cc880 legacy=0x7f2b1006d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:54.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.068+0000 7f2b0fe00640 1 -- 172.21.15.159:0/2209114261 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 937416423 0 0) 0x7f2af8003520 con 0x7f2b100cc880 2024-04-04T06:06:54.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.068+0000 7f2b15e00640 1 -- 172.21.15.159:0/2209114261 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b100cee30 con 0x7f2b1006dd30 2024-04-04T06:06:54.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.069+0000 7f2b15e00640 1 --2- 172.21.15.159:0/2209114261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b1006dd30 0x7f2b100cccf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2b1006f660 tx=0x7f2b0802bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98191d0ed15eb7b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:54.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.070+0000 7f2b0fe00640 1 -- 172.21.15.159:0/2209114261 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b08039070 con 0x7f2b1006dd30 2024-04-04T06:06:54.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.070+0000 7f2b0fe00640 1 -- 172.21.15.159:0/2209114261 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b0802c8c0 con 0x7f2b1006dd30 2024-04-04T06:06:54.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.070+0000 7f2b0fe00640 1 -- 172.21.15.159:0/2209114261 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc001670 con 0x7f2b1006dd30 2024-04-04T06:06:54.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.070+0000 7f2b16800640 1 -- 172.21.15.159:0/2209114261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b1006dd30 msgr2=0x7f2b100cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:54.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.070+0000 7f2b16800640 1 --2- 172.21.15.159:0/2209114261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b1006dd30 0x7f2b100cccf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2b1006f660 tx=0x7f2b0802bd90 comp rx=0 tx=0).stop 2024-04-04T06:06:54.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.070+0000 7f2b16800640 1 -- 172.21.15.159:0/2209114261 shutdown_connections 2024-04-04T06:06:54.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.070+0000 7f2b16800640 1 --2- 172.21.15.159:0/2209114261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b1006dd30 0x7f2b100cccf0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:54.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.070+0000 7f2b16800640 1 --2- 172.21.15.159:0/2209114261 >> v2:172.21.15.178:3300/0 conn(0x7f2b1013f190 0x7f2b100cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:54.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.070+0000 7f2b16800640 1 -- 172.21.15.159:0/2209114261 >> 172.21.15.159:0/2209114261 conn(0x7f2b100b1ba0 msgr2=0x7f2b100b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:54.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.070+0000 7f2b16800640 1 -- 172.21.15.159:0/2209114261 shutdown_connections 2024-04-04T06:06:54.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.070+0000 7f2b16800640 1 -- 172.21.15.159:0/2209114261 wait complete. 2024-04-04T06:06:54.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.072+0000 7f2b16800640 1 Processor -- start 2024-04-04T06:06:54.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.072+0000 7f2b16800640 1 -- start start 2024-04-04T06:06:54.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.073+0000 7f2b16800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b1006dd30 0x7f2b1013e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:54.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.073+0000 7f2b16800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b100cc880 0x7f2b101e7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:54.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.073+0000 7f2b16800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b1013f190 0x7f2b1015ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:54.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.073+0000 7f2b16800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1013f570 con 0x7f2b1006dd30 2024-04-04T06:06:54.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.073+0000 7f2b16800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2b10160c70 con 0x7f2b1013f190 2024-04-04T06:06:54.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.073+0000 7f2b16800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2b100cfb70 con 0x7f2b100cc880 2024-04-04T06:06:54.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.073+0000 7f2b15400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b1006dd30 0x7f2b1013e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:54.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.073+0000 7f2b15400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b1006dd30 0x7f2b1013e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42908/0 (socket says 172.21.15.159:42908) 2024-04-04T06:06:54.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.074+0000 7f2b15400640 1 -- 172.21.15.159:0/842875542 learned_addr learned my addr 172.21.15.159:0/842875542 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:54.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.074+0000 7f2b15e00640 1 --2- 172.21.15.159:0/842875542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b1013f190 0x7f2b1015ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:54.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.074+0000 7f2b14a00640 1 --2- 172.21.15.159:0/842875542 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b100cc880 0x7f2b101e7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:54.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.074+0000 7f2b15400640 1 -- 172.21.15.159:0/842875542 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b100cc880 msgr2=0x7f2b101e7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:54.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.074+0000 7f2b15400640 1 --2- 172.21.15.159:0/842875542 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b100cc880 0x7f2b101e7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:54.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.074+0000 7f2b15400640 1 -- 172.21.15.159:0/842875542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b1013f190 msgr2=0x7f2b1015ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:54.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.074+0000 7f2b15400640 1 --2- 172.21.15.159:0/842875542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b1013f190 0x7f2b1015ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:54.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.074+0000 7f2b15400640 1 -- 172.21.15.159:0/842875542 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b101e75b0 con 0x7f2b1006dd30 2024-04-04T06:06:54.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.074+0000 7f2b15400640 1 --2- 172.21.15.159:0/842875542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b1006dd30 0x7f2b1013e8b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400e9b0 tx=0x7f2b0400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:54.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.074+0000 7f2b0d600640 1 -- 172.21.15.159:0/842875542 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04017070 con 0x7f2b1006dd30 2024-04-04T06:06:54.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.074+0000 7f2b0d600640 1 -- 172.21.15.159:0/842875542 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b0400c9a0 con 0x7f2b1006dd30 2024-04-04T06:06:54.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.074+0000 7f2b16800640 1 -- 172.21.15.159:0/842875542 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b101e7860 con 0x7f2b1006dd30 2024-04-04T06:06:54.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.074+0000 7f2b0d600640 1 -- 172.21.15.159:0/842875542 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0400cd90 con 0x7f2b1006dd30 2024-04-04T06:06:54.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.075+0000 7f2b16800640 1 -- 172.21.15.159:0/842875542 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b101e7e00 con 0x7f2b1006dd30 2024-04-04T06:06:54.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.076+0000 7f2b16800640 1 -- 172.21.15.159:0/842875542 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b10075940 con 0x7f2b1006dd30 2024-04-04T06:06:54.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.076+0000 7f2b0d600640 1 -- 172.21.15.159:0/842875542 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b0401a070 con 0x7f2b1006dd30 2024-04-04T06:06:54.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.077+0000 7f2b0d600640 1 --2- 172.21.15.159:0/842875542 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2af006c8f0 0x7f2af006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:54.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.077+0000 7f2b0d600640 1 -- 172.21.15.159:0/842875542 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f2b0408bba0 con 0x7f2b1006dd30 2024-04-04T06:06:54.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.078+0000 7f2b14a00640 1 --2- 172.21.15.159:0/842875542 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2af006c8f0 0x7f2af006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:54.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.079+0000 7f2b14a00640 1 --2- 172.21.15.159:0/842875542 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2af006c8f0 0x7f2af006edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2b101e6fc0 tx=0x7f2afc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:54.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.082+0000 7f2b0d600640 1 -- 172.21.15.159:0/842875542 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2b04059fb0 con 0x7f2b1006dd30 2024-04-04T06:06:54.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.282+0000 7f2b16800640 1 -- 172.21.15.159:0/842875542 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2b100b2870 con 0x7f2b1006dd30 2024-04-04T06:06:54.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.282+0000 7f2b0d600640 1 -- 172.21.15.159:0/842875542 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f2b040598b0 con 0x7f2b1006dd30 2024-04-04T06:06:54.378 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:06:54.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.285+0000 7f2b16800640 1 -- 172.21.15.159:0/842875542 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2af006c8f0 msgr2=0x7f2af006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:54.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.285+0000 7f2b16800640 1 --2- 172.21.15.159:0/842875542 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2af006c8f0 0x7f2af006edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2b101e6fc0 tx=0x7f2afc009040 comp rx=0 tx=0).stop 2024-04-04T06:06:54.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.285+0000 7f2b16800640 1 -- 172.21.15.159:0/842875542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b1006dd30 msgr2=0x7f2b1013e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:54.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.285+0000 7f2b16800640 1 --2- 172.21.15.159:0/842875542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b1006dd30 0x7f2b1013e8b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400e9b0 tx=0x7f2b0400ee70 comp rx=0 tx=0).stop 2024-04-04T06:06:54.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.285+0000 7f2b16800640 1 -- 172.21.15.159:0/842875542 shutdown_connections 2024-04-04T06:06:54.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.285+0000 7f2b16800640 1 --2- 172.21.15.159:0/842875542 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2af006c8f0 0x7f2af006edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:54.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.285+0000 7f2b16800640 1 --2- 172.21.15.159:0/842875542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b1013f190 0x7f2b1015ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:54.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.285+0000 7f2b16800640 1 --2- 172.21.15.159:0/842875542 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b100cc880 0x7f2b101e7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:54.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.285+0000 7f2b16800640 1 --2- 172.21.15.159:0/842875542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b1006dd30 0x7f2b1013e8b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:54.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.285+0000 7f2b16800640 1 -- 172.21.15.159:0/842875542 >> 172.21.15.159:0/842875542 conn(0x7f2b100b1ba0 msgr2=0x7f2b100cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:54.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.285+0000 7f2b16800640 1 -- 172.21.15.159:0/842875542 shutdown_connections 2024-04-04T06:06:54.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.285+0000 7f2b16800640 1 -- 172.21.15.159:0/842875542 wait complete. 2024-04-04T06:06:54.381 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:06:54.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.599+0000 7fd377000640 1 Processor -- start 2024-04-04T06:06:54.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.600+0000 7fd377000640 1 -- start start 2024-04-04T06:06:54.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.601+0000 7fd377000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd37015b310 0x7fd3700727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:54.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.601+0000 7fd377000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd37015b6f0 con 0x7fd3700d0780 2024-04-04T06:06:54.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.601+0000 7fd377000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd370072e80 con 0x7fd3700b4f90 2024-04-04T06:06:54.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.601+0000 7fd377000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd370073160 con 0x7fd37015b310 2024-04-04T06:06:54.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.601+0000 7fd375000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fd3700d0780 0x7fd3700adc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58554/0 (socket says 172.21.15.159:58554) 2024-04-04T06:06:54.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.601+0000 7fd375000640 1 -- 172.21.15.159:0/2183805696 learned_addr learned my addr 172.21.15.159:0/2183805696 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:54.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.602+0000 7fd36fe00640 1 --2- 172.21.15.159:0/2183805696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd37015b310 0x7fd3700727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:54.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.602+0000 7fd36f400640 1 -- 172.21.15.159:0/2183805696 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1369760892 0 0) 0x7fd37015b6f0 con 0x7fd3700d0780 2024-04-04T06:06:54.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.602+0000 7fd36f400640 1 -- 172.21.15.159:0/2183805696 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd354003520 con 0x7fd3700d0780 2024-04-04T06:06:54.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.602+0000 7fd36f400640 1 -- 172.21.15.159:0/2183805696 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2638451014 0 0) 0x7fd354003520 con 0x7fd3700d0780 2024-04-04T06:06:54.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.602+0000 7fd36f400640 1 -- 172.21.15.159:0/2183805696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd37015b310 msgr2=0x7fd3700727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:54.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.602+0000 7fd36f400640 1 --2- 172.21.15.159:0/2183805696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd37015b310 0x7fd3700727f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:54.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.602+0000 7fd36f400640 1 -- 172.21.15.159:0/2183805696 >> v1:172.21.15.178:6789/0 conn(0x7fd3700b4f90 legacy=0x7fd37015f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:54.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.602+0000 7fd36f400640 1 -- 172.21.15.159:0/2183805696 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd37015efe0 con 0x7fd3700d0780 2024-04-04T06:06:54.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.602+0000 7fd36f400640 1 -- 172.21.15.159:0/2183805696 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fd3680019f0 con 0x7fd3700d0780 2024-04-04T06:06:54.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.602+0000 7fd36f400640 1 -- 172.21.15.159:0/2183805696 >> v1:172.21.15.159:6789/0 conn(0x7fd3700d0780 legacy=0x7fd3700adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:54.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.602+0000 7fd36f400640 1 --2- 172.21.15.159:0/2183805696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd37015b310 0x7fd3540039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:54.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.602+0000 7fd36f400640 1 --2- 172.21.15.159:0/2183805696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd354003f60 0x7fd354004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:54.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.602+0000 7fd36f400640 1 --2- 172.21.15.159:0/2183805696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd354004d40 0x7fd3540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:54.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.602+0000 7fd36f400640 1 -- 172.21.15.159:0/2183805696 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd370073160 con 0x7fd354003f60 2024-04-04T06:06:54.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.602+0000 7fd36f400640 1 -- 172.21.15.159:0/2183805696 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd35400bc00 con 0x7fd354004d40 2024-04-04T06:06:54.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.602+0000 7fd36f400640 1 -- 172.21.15.159:0/2183805696 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd35400be00 con 0x7fd37015b310 2024-04-04T06:06:54.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.602+0000 7fd367400640 1 --2- 172.21.15.159:0/2183805696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd354003f60 0x7fd354004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:54.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.603+0000 7fd375000640 1 --2- 172.21.15.159:0/2183805696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd354004d40 0x7fd3540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:54.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.603+0000 7fd367400640 1 -- 172.21.15.159:0/2183805696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd37015b310 msgr2=0x7fd3540039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:54.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.603+0000 7fd367400640 1 --2- 172.21.15.159:0/2183805696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd37015b310 0x7fd3540039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:54.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.603+0000 7fd367400640 1 -- 172.21.15.159:0/2183805696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd354004d40 msgr2=0x7fd3540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:54.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.603+0000 7fd367400640 1 --2- 172.21.15.159:0/2183805696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd354004d40 0x7fd3540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:54.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.603+0000 7fd367400640 1 -- 172.21.15.159:0/2183805696 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd354003520 con 0x7fd354003f60 2024-04-04T06:06:54.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.603+0000 7fd375000640 1 --2- 172.21.15.159:0/2183805696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd354004d40 0x7fd3540051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:54.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.604+0000 7fd367400640 1 --2- 172.21.15.159:0/2183805696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd354003f60 0x7fd354004800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd358001750 tx=0x7fd354005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598362949b6e207d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:54.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.604+0000 7fd36f400640 1 -- 172.21.15.159:0/2183805696 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358001370 con 0x7fd354003f60 2024-04-04T06:06:54.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.604+0000 7fd36f400640 1 -- 172.21.15.159:0/2183805696 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3580052d0 con 0x7fd354003f60 2024-04-04T06:06:54.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.604+0000 7fd36f400640 1 -- 172.21.15.159:0/2183805696 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358003830 con 0x7fd354003f60 2024-04-04T06:06:54.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.605+0000 7fd377000640 1 -- 172.21.15.159:0/2183805696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd354003f60 msgr2=0x7fd354004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:54.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.605+0000 7fd377000640 1 --2- 172.21.15.159:0/2183805696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd354003f60 0x7fd354004800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd358001750 tx=0x7fd354005830 comp rx=0 tx=0).stop 2024-04-04T06:06:54.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.605+0000 7fd375000640 1 -- 172.21.15.159:0/2183805696 reap_dead start 2024-04-04T06:06:54.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.605+0000 7fd377000640 1 -- 172.21.15.159:0/2183805696 shutdown_connections 2024-04-04T06:06:54.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.605+0000 7fd377000640 1 -- 172.21.15.159:0/2183805696 >> 172.21.15.159:0/2183805696 conn(0x7fd37006b990 msgr2=0x7fd37006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:54.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.605+0000 7fd377000640 1 -- 172.21.15.159:0/2183805696 shutdown_connections 2024-04-04T06:06:54.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.605+0000 7fd377000640 1 -- 172.21.15.159:0/2183805696 wait complete. 2024-04-04T06:06:54.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.607+0000 7fd377000640 1 Processor -- start 2024-04-04T06:06:54.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.607+0000 7fd377000640 1 -- start start 2024-04-04T06:06:54.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.608+0000 7fd377000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd354003f60 0x7fd3700b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:54.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.608+0000 7fd377000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3701f59f0 0x7fd3700cf380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:54.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.608+0000 7fd377000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3700ca680 0x7fd3700cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:54.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.608+0000 7fd377000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd370073440 con 0x7fd3701f59f0 2024-04-04T06:06:54.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.608+0000 7fd377000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3700b53f0 con 0x7fd3700ca680 2024-04-04T06:06:54.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.608+0000 7fd377000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd3700727f0 con 0x7fd354003f60 2024-04-04T06:06:54.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.608+0000 7fd367400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3701f59f0 0x7fd3700cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:54.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.608+0000 7fd367400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3701f59f0 0x7fd3700cf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42934/0 (socket says 172.21.15.159:42934) 2024-04-04T06:06:54.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.608+0000 7fd367400640 1 -- 172.21.15.159:0/484837901 learned_addr learned my addr 172.21.15.159:0/484837901 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:54.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.608+0000 7fd375000640 1 --2- 172.21.15.159:0/484837901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3700ca680 0x7fd3700cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:54.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.608+0000 7fd367400640 1 -- 172.21.15.159:0/484837901 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd354003f60 msgr2=0x7fd3700b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:54.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.608+0000 7fd367400640 1 --2- 172.21.15.159:0/484837901 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd354003f60 0x7fd3700b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:54.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.608+0000 7fd367400640 1 -- 172.21.15.159:0/484837901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3700ca680 msgr2=0x7fd3700cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:54.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.608+0000 7fd367400640 1 --2- 172.21.15.159:0/484837901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3700ca680 0x7fd3700cab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:54.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.608+0000 7fd367400640 1 -- 172.21.15.159:0/484837901 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3701f5e70 con 0x7fd3701f59f0 2024-04-04T06:06:54.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.609+0000 7fd367400640 1 --2- 172.21.15.159:0/484837901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3701f59f0 0x7fd3700cf380 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd358008fd0 tx=0x7fd358001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:54.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.609+0000 7fd36cc00640 1 -- 172.21.15.159:0/484837901 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358041070 con 0x7fd3701f59f0 2024-04-04T06:06:54.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.609+0000 7fd36cc00640 1 -- 172.21.15.159:0/484837901 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd358005480 con 0x7fd3701f59f0 2024-04-04T06:06:54.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.609+0000 7fd36cc00640 1 -- 172.21.15.159:0/484837901 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358001370 con 0x7fd3701f59f0 2024-04-04T06:06:54.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.609+0000 7fd377000640 1 -- 172.21.15.159:0/484837901 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3700b0f10 con 0x7fd3701f59f0 2024-04-04T06:06:54.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.609+0000 7fd377000640 1 -- 172.21.15.159:0/484837901 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3700c9840 con 0x7fd3701f59f0 2024-04-04T06:06:54.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.610+0000 7fd377000640 1 -- 172.21.15.159:0/484837901 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3700685c0 con 0x7fd3701f59f0 2024-04-04T06:06:54.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.611+0000 7fd36cc00640 1 -- 172.21.15.159:0/484837901 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd358044070 con 0x7fd3701f59f0 2024-04-04T06:06:54.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.612+0000 7fd36cc00640 1 --2- 172.21.15.159:0/484837901 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd34c06c920 0x7fd34c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:54.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.612+0000 7fd36cc00640 1 -- 172.21.15.159:0/484837901 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7fd3580ad670 con 0x7fd3701f59f0 2024-04-04T06:06:54.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.612+0000 7fd36fe00640 1 --2- 172.21.15.159:0/484837901 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd34c06c920 0x7fd34c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:54.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.614+0000 7fd36fe00640 1 --2- 172.21.15.159:0/484837901 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd34c06c920 0x7fd34c06ede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd360006a90 tx=0x7fd360006a20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:54.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.616+0000 7fd36cc00640 1 -- 172.21.15.159:0/484837901 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd35807b9f0 con 0x7fd3701f59f0 2024-04-04T06:06:54.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.856+0000 7fd377000640 1 -- 172.21.15.159:0/484837901 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd37015ec20 con 0x7fd3701f59f0 2024-04-04T06:06:54.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.857+0000 7fd36cc00640 1 -- 172.21.15.159:0/484837901 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5184 (secure 0 0 0) 0x7fd35807b2f0 con 0x7fd3701f59f0 2024-04-04T06:06:54.852 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:54.852 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/789759868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":789759868},{"type":"v1","addr":"172.21.15.178:6836","nonce":789759868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2438288045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2438288045},{"type":"v1","addr":"172.21.15.159:6833","nonce":2438288045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/1705205607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1705205607},{"type":"v1","addr":"172.21.15.159:6835","nonce":1705205607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:05:52.801529+0000","modified":"2024-04-04T06:05:55.866207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6837/3362141613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":3362141613},{"type":"v1","addr":"172.21.15.178:6837","nonce":3362141613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-04T06:06:54.852 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 8 2024-04-04T06:06:54.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.860+0000 7fd377000640 1 -- 172.21.15.159:0/484837901 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd34c06c920 msgr2=0x7fd34c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:54.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.860+0000 7fd377000640 1 --2- 172.21.15.159:0/484837901 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd34c06c920 0x7fd34c06ede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd360006a90 tx=0x7fd360006a20 comp rx=0 tx=0).stop 2024-04-04T06:06:54.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.860+0000 7fd377000640 1 -- 172.21.15.159:0/484837901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3701f59f0 msgr2=0x7fd3700cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:54.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.860+0000 7fd377000640 1 --2- 172.21.15.159:0/484837901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3701f59f0 0x7fd3700cf380 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd358008fd0 tx=0x7fd358001120 comp rx=0 tx=0).stop 2024-04-04T06:06:54.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.860+0000 7fd377000640 1 -- 172.21.15.159:0/484837901 shutdown_connections 2024-04-04T06:06:54.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.860+0000 7fd377000640 1 --2- 172.21.15.159:0/484837901 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd34c06c920 0x7fd34c06ede0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:54.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.860+0000 7fd377000640 1 --2- 172.21.15.159:0/484837901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3700ca680 0x7fd3700cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:54.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.860+0000 7fd377000640 1 --2- 172.21.15.159:0/484837901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3701f59f0 0x7fd3700cf380 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:54.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.860+0000 7fd377000640 1 --2- 172.21.15.159:0/484837901 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd354003f60 0x7fd3700b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:54.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.860+0000 7fd377000640 1 -- 172.21.15.159:0/484837901 >> 172.21.15.159:0/484837901 conn(0x7fd37006b990 msgr2=0x7fd3700ae100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:54.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.860+0000 7fd377000640 1 -- 172.21.15.159:0/484837901 shutdown_connections 2024-04-04T06:06:54.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:54.860+0000 7fd377000640 1 -- 172.21.15.159:0/484837901 wait complete. 2024-04-04T06:06:54.872 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:06:55.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.081+0000 7f99e7e00640 1 Processor -- start 2024-04-04T06:06:55.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.081+0000 7f99e7e00640 1 -- start start 2024-04-04T06:06:55.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.083+0000 7f99e7e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f99e80cc0b0 0x7f99e806dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:55.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.083+0000 7f99e7e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99e815fc50 0x7f99e80cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:55.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.083+0000 7f99e7e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99e806e4f0 con 0x7f99e80cc490 2024-04-04T06:06:55.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.083+0000 7f99e7e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f99e815c610 con 0x7f99e80cc0b0 2024-04-04T06:06:55.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.083+0000 7f99e7e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f99e80b3150 con 0x7f99e815fc50 2024-04-04T06:06:55.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.083+0000 7f99e7400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f99e80cc490 0x7f99e815ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58556/0 (socket says 172.21.15.159:58556) 2024-04-04T06:06:55.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.083+0000 7f99e7400640 1 -- 172.21.15.159:0/2105246416 learned_addr learned my addr 172.21.15.159:0/2105246416 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:55.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.083+0000 7f99e6a00640 1 --2- 172.21.15.159:0/2105246416 >> v2:172.21.15.178:3300/0 conn(0x7f99e80cc0b0 0x7f99e806dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:55.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.083+0000 7f99e6000640 1 --2- 172.21.15.159:0/2105246416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99e815fc50 0x7f99e80cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:55.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.083+0000 7f99e5600640 1 -- 172.21.15.159:0/2105246416 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 452350323 0 0) 0x7f99e806e4f0 con 0x7f99e80cc490 2024-04-04T06:06:55.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.083+0000 7f99e5600640 1 -- 172.21.15.159:0/2105246416 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99cc003520 con 0x7f99e80cc490 2024-04-04T06:06:55.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.084+0000 7f99e5600640 1 -- 172.21.15.159:0/2105246416 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2522035101 0 0) 0x7f99cc003520 con 0x7f99e80cc490 2024-04-04T06:06:55.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.084+0000 7f99e5600640 1 -- 172.21.15.159:0/2105246416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99e815fc50 msgr2=0x7f99e80cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:55.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.084+0000 7f99e5600640 1 --2- 172.21.15.159:0/2105246416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99e815fc50 0x7f99e80cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:55.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.084+0000 7f99e5600640 1 -- 172.21.15.159:0/2105246416 >> v2:172.21.15.178:3300/0 conn(0x7f99e80cc0b0 msgr2=0x7f99e806dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:55.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.084+0000 7f99e5600640 1 --2- 172.21.15.159:0/2105246416 >> v2:172.21.15.178:3300/0 conn(0x7f99e80cc0b0 0x7f99e806dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:55.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.084+0000 7f99e5600640 1 -- 172.21.15.159:0/2105246416 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e81600e0 con 0x7f99e80cc490 2024-04-04T06:06:55.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.084+0000 7f99e5600640 1 -- 172.21.15.159:0/2105246416 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f99e00019f0 con 0x7f99e80cc490 2024-04-04T06:06:55.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.084+0000 7f99e5600640 1 -- 172.21.15.159:0/2105246416 >> v1:172.21.15.159:6789/0 conn(0x7f99e80cc490 legacy=0x7f99e815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:55.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.084+0000 7f99e5600640 1 --2- 172.21.15.159:0/2105246416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99cc0039b0 0x7f99cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:55.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.084+0000 7f99e5600640 1 --2- 172.21.15.159:0/2105246416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99cc004800 0x7f99cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:55.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.084+0000 7f99e5600640 1 --2- 172.21.15.159:0/2105246416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99cc0051c0 0x7f99cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:55.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.084+0000 7f99e5600640 1 -- 172.21.15.159:0/2105246416 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f99e815c610 con 0x7f99cc0039b0 2024-04-04T06:06:55.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.084+0000 7f99e5600640 1 -- 172.21.15.159:0/2105246416 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f99e80b3150 con 0x7f99cc004800 2024-04-04T06:06:55.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.084+0000 7f99e5600640 1 -- 172.21.15.159:0/2105246416 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f99cc00bf00 con 0x7f99cc0051c0 2024-04-04T06:06:55.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.084+0000 7f99e6a00640 1 --2- 172.21.15.159:0/2105246416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99cc0039b0 0x7f99cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:55.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.085+0000 7f99e6000640 1 --2- 172.21.15.159:0/2105246416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99cc004800 0x7f99cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:55.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.085+0000 7f99e7400640 1 --2- 172.21.15.159:0/2105246416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99cc0051c0 0x7f99cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:55.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.085+0000 7f99e6a00640 1 -- 172.21.15.159:0/2105246416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99cc0051c0 msgr2=0x7f99cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:55.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.085+0000 7f99e6a00640 1 --2- 172.21.15.159:0/2105246416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99cc0051c0 0x7f99cc00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:55.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.085+0000 7f99e6a00640 1 -- 172.21.15.159:0/2105246416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99cc004800 msgr2=0x7f99cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:55.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.085+0000 7f99e6a00640 1 --2- 172.21.15.159:0/2105246416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99cc004800 0x7f99cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:55.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.085+0000 7f99e6a00640 1 -- 172.21.15.159:0/2105246416 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99cc003520 con 0x7f99cc0039b0 2024-04-04T06:06:55.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.086+0000 7f99e6a00640 1 --2- 172.21.15.159:0/2105246416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99cc0039b0 0x7f99cc003dd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f99d8009ac0 tx=0x7f99cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f6ee39f349dc35 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:55.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.086+0000 7f99e5600640 1 -- 172.21.15.159:0/2105246416 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8039070 con 0x7f99cc0039b0 2024-04-04T06:06:55.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.086+0000 7f99e5600640 1 -- 172.21.15.159:0/2105246416 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d802c710 con 0x7f99cc0039b0 2024-04-04T06:06:55.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.086+0000 7f99e5600640 1 -- 172.21.15.159:0/2105246416 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8009690 con 0x7f99cc0039b0 2024-04-04T06:06:55.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.087+0000 7f99e7e00640 1 -- 172.21.15.159:0/2105246416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99cc0039b0 msgr2=0x7f99cc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:55.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.087+0000 7f99e7e00640 1 --2- 172.21.15.159:0/2105246416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99cc0039b0 0x7f99cc003dd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f99d8009ac0 tx=0x7f99cc005830 comp rx=0 tx=0).stop 2024-04-04T06:06:55.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.087+0000 7f99e7400640 1 -- 172.21.15.159:0/2105246416 reap_dead start 2024-04-04T06:06:55.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.087+0000 7f99e7e00640 1 -- 172.21.15.159:0/2105246416 shutdown_connections 2024-04-04T06:06:55.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.087+0000 7f99e7e00640 1 -- 172.21.15.159:0/2105246416 >> 172.21.15.159:0/2105246416 conn(0x7f99e80ccad0 msgr2=0x7f99e80cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:55.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.087+0000 7f99e7e00640 1 -- 172.21.15.159:0/2105246416 shutdown_connections 2024-04-04T06:06:55.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.087+0000 7f99e7e00640 1 -- 172.21.15.159:0/2105246416 wait complete. 2024-04-04T06:06:55.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.089+0000 7f99e7e00640 1 Processor -- start 2024-04-04T06:06:55.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.089+0000 7f99e7e00640 1 -- start start 2024-04-04T06:06:55.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.090+0000 7f99e7e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99cc0039b0 0x7f99e80b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:55.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.090+0000 7f99e7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99e813e9a0 0x7f99e815eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:55.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.090+0000 7f99e7e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99e813dd30 0x7f99e813e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:55.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.090+0000 7f99e7e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f99e8160980 con 0x7f99e813e9a0 2024-04-04T06:06:55.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.090+0000 7f99e7e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f99e81606a0 con 0x7f99cc0039b0 2024-04-04T06:06:55.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.090+0000 7f99e7e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f99e815d180 con 0x7f99e813dd30 2024-04-04T06:06:55.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.090+0000 7f99e6000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99e813e9a0 0x7f99e815eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:55.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.090+0000 7f99e6000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99e813e9a0 0x7f99e815eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42952/0 (socket says 172.21.15.159:42952) 2024-04-04T06:06:55.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.090+0000 7f99e6000640 1 -- 172.21.15.159:0/2421371840 learned_addr learned my addr 172.21.15.159:0/2421371840 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:55.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.090+0000 7f99e6a00640 1 --2- 172.21.15.159:0/2421371840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99cc0039b0 0x7f99e80b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:55.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.090+0000 7f99e7400640 1 --2- 172.21.15.159:0/2421371840 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99e813dd30 0x7f99e813e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:55.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.090+0000 7f99e6000640 1 -- 172.21.15.159:0/2421371840 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99e813dd30 msgr2=0x7f99e813e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:55.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.090+0000 7f99e6000640 1 --2- 172.21.15.159:0/2421371840 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99e813dd30 0x7f99e813e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:55.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.090+0000 7f99e6000640 1 -- 172.21.15.159:0/2421371840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99cc0039b0 msgr2=0x7f99e80b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:55.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.090+0000 7f99e6000640 1 --2- 172.21.15.159:0/2421371840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99cc0039b0 0x7f99e80b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:55.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.090+0000 7f99e6000640 1 -- 172.21.15.159:0/2421371840 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e813e700 con 0x7f99e813e9a0 2024-04-04T06:06:55.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.091+0000 7f99e6000640 1 --2- 172.21.15.159:0/2421371840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99e813e9a0 0x7f99e815eac0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f99d400f5b0 tx=0x7f99d400fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:55.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.091+0000 7f99df400640 1 -- 172.21.15.159:0/2421371840 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4019020 con 0x7f99e813e9a0 2024-04-04T06:06:55.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.091+0000 7f99df400640 1 -- 172.21.15.159:0/2421371840 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d4014870 con 0x7f99e813e9a0 2024-04-04T06:06:55.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.091+0000 7f99e7e00640 1 -- 172.21.15.159:0/2421371840 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e80b1a60 con 0x7f99e813e9a0 2024-04-04T06:06:55.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.091+0000 7f99df400640 1 -- 172.21.15.159:0/2421371840 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4014c60 con 0x7f99e813e9a0 2024-04-04T06:06:55.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.091+0000 7f99e7e00640 1 -- 172.21.15.159:0/2421371840 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e80b2300 con 0x7f99e813e9a0 2024-04-04T06:06:55.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.092+0000 7f99e7e00640 1 -- 172.21.15.159:0/2421371840 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e8075940 con 0x7f99e813e9a0 2024-04-04T06:06:55.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.093+0000 7f99df400640 1 -- 172.21.15.159:0/2421371840 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f99d401c020 con 0x7f99e813e9a0 2024-04-04T06:06:55.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.095+0000 7f99df400640 1 --2- 172.21.15.159:0/2421371840 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f99c806c8f0 0x7f99c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:55.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.095+0000 7f99df400640 1 -- 172.21.15.159:0/2421371840 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f99d408c450 con 0x7f99e813e9a0 2024-04-04T06:06:55.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.095+0000 7f99e6a00640 1 --2- 172.21.15.159:0/2421371840 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f99c806c8f0 0x7f99c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:55.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.096+0000 7f99e6a00640 1 --2- 172.21.15.159:0/2421371840 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f99c806c8f0 0x7f99c806edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f99d8036000 tx=0x7f99d8030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:55.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.099+0000 7f99df400640 1 -- 172.21.15.159:0/2421371840 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f99d4023020 con 0x7f99e813e9a0 2024-04-04T06:06:55.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.297+0000 7f99e7e00640 1 -- 172.21.15.159:0/2421371840 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99e8073830 con 0x7f99e813e9a0 2024-04-04T06:06:55.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.301+0000 7f99df400640 1 -- 172.21.15.159:0/2421371840 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16958 (secure 0 0 0) 0x7f99d405a7d0 con 0x7f99e813e9a0 2024-04-04T06:06:55.295 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:55.296 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":23,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:06:33.796979+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:05:51.297293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-04T06:05:51.770792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":0,"to":2}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:06:55.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.304+0000 7f99e7e00640 1 -- 172.21.15.159:0/2421371840 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f99c806c8f0 msgr2=0x7f99c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:55.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.304+0000 7f99e7e00640 1 --2- 172.21.15.159:0/2421371840 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f99c806c8f0 0x7f99c806edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f99d8036000 tx=0x7f99d8030000 comp rx=0 tx=0).stop 2024-04-04T06:06:55.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.304+0000 7f99e7e00640 1 -- 172.21.15.159:0/2421371840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99e813e9a0 msgr2=0x7f99e815eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:55.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.304+0000 7f99e7e00640 1 --2- 172.21.15.159:0/2421371840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99e813e9a0 0x7f99e815eac0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f99d400f5b0 tx=0x7f99d400fa70 comp rx=0 tx=0).stop 2024-04-04T06:06:55.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.304+0000 7f99e7e00640 1 -- 172.21.15.159:0/2421371840 shutdown_connections 2024-04-04T06:06:55.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.304+0000 7f99e7e00640 1 --2- 172.21.15.159:0/2421371840 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99e813dd30 0x7f99e813e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:55.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.304+0000 7f99e7e00640 1 --2- 172.21.15.159:0/2421371840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99e813e9a0 0x7f99e815eac0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:55.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.304+0000 7f99e7e00640 1 --2- 172.21.15.159:0/2421371840 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f99c806c8f0 0x7f99c806edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:55.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.304+0000 7f99e7e00640 1 --2- 172.21.15.159:0/2421371840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99cc0039b0 0x7f99e80b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:55.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.304+0000 7f99e7e00640 1 -- 172.21.15.159:0/2421371840 >> 172.21.15.159:0/2421371840 conn(0x7f99e80ccad0 msgr2=0x7f99e80d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:55.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.304+0000 7f99e7e00640 1 -- 172.21.15.159:0/2421371840 shutdown_connections 2024-04-04T06:06:55.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.304+0000 7f99e7e00640 1 -- 172.21.15.159:0/2421371840 wait complete. 2024-04-04T06:06:55.319 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-04T06:06:55.319 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:06:55.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.538+0000 7f36ca400640 1 Processor -- start 2024-04-04T06:06:55.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.538+0000 7f36ca400640 1 -- start start 2024-04-04T06:06:55.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.540+0000 7f36ca400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c4161b00 0x7f36c4162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:55.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.540+0000 7f36ca400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f36c40d0d60 0x7f36c40b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:55.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.540+0000 7f36ca400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36c4071b60 con 0x7f36c40ad120 2024-04-04T06:06:55.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.540+0000 7f36ca400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f36c4161ee0 con 0x7f36c40d0d60 2024-04-04T06:06:55.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.540+0000 7f36ca400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f36c40b3450 con 0x7f36c4161b00 2024-04-04T06:06:55.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.540+0000 7f36c3e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f36c40d0d60 0x7f36c40b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:55.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.540+0000 7f36c3e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f36c40d0d60 0x7f36c40b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42954/0 (socket says 172.21.15.159:42954) 2024-04-04T06:06:55.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.540+0000 7f36c3e00640 1 -- 172.21.15.159:0/3712351162 learned_addr learned my addr 172.21.15.159:0/3712351162 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:55.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.541+0000 7f36c3400640 1 --2- 172.21.15.159:0/3712351162 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c4161b00 0x7f36c4162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:55.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.541+0000 7f36c3e00640 1 -- 172.21.15.159:0/3712351162 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c4161b00 msgr2=0x7f36c4162ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:55.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.541+0000 7f36c3e00640 1 --2- 172.21.15.159:0/3712351162 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c4161b00 0x7f36c4162ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:55.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.541+0000 7f36c3e00640 1 -- 172.21.15.159:0/3712351162 >> v1:172.21.15.178:6789/0 conn(0x7f36c40ad120 legacy=0x7f36c40d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:55.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.541+0000 7f36c3e00640 1 -- 172.21.15.159:0/3712351162 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c4071e40 con 0x7f36c40d0d60 2024-04-04T06:06:55.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.541+0000 7f36c3400640 1 --2- 172.21.15.159:0/3712351162 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c4161b00 0x7f36c4162ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:55.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.543+0000 7f36c3e00640 1 --2- 172.21.15.159:0/3712351162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36c40d0d60 0x7f36c40b3bc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f36c4162060 tx=0x7f36bc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a7677d4c5c3c5d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:55.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.543+0000 7f36c2000640 1 -- 172.21.15.159:0/3712351162 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc034070 con 0x7f36c40d0d60 2024-04-04T06:06:55.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.543+0000 7f36c2000640 1 -- 172.21.15.159:0/3712351162 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36bc030840 con 0x7f36c40d0d60 2024-04-04T06:06:55.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.543+0000 7f36c2000640 1 -- 172.21.15.159:0/3712351162 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc030c10 con 0x7f36c40d0d60 2024-04-04T06:06:55.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.544+0000 7f36ca400640 1 -- 172.21.15.159:0/3712351162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36c40d0d60 msgr2=0x7f36c40b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:55.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.544+0000 7f36ca400640 1 --2- 172.21.15.159:0/3712351162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36c40d0d60 0x7f36c40b3bc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f36c4162060 tx=0x7f36bc02b980 comp rx=0 tx=0).stop 2024-04-04T06:06:55.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.544+0000 7f36ca400640 1 -- 172.21.15.159:0/3712351162 shutdown_connections 2024-04-04T06:06:55.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.544+0000 7f36ca400640 1 --2- 172.21.15.159:0/3712351162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36c40d0d60 0x7f36c40b3bc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:55.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.544+0000 7f36ca400640 1 --2- 172.21.15.159:0/3712351162 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c4161b00 0x7f36c4162ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:55.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.544+0000 7f36ca400640 1 -- 172.21.15.159:0/3712351162 >> 172.21.15.159:0/3712351162 conn(0x7f36c406b990 msgr2=0x7f36c406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:55.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.544+0000 7f36ca400640 1 -- 172.21.15.159:0/3712351162 shutdown_connections 2024-04-04T06:06:55.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.544+0000 7f36ca400640 1 -- 172.21.15.159:0/3712351162 wait complete. 2024-04-04T06:06:55.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.546+0000 7f36ca400640 1 Processor -- start 2024-04-04T06:06:55.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.546+0000 7f36ca400640 1 -- start start 2024-04-04T06:06:55.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.547+0000 7f36ca400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c40ad120 0x7f36c4160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:55.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.547+0000 7f36ca400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c40d0d60 0x7f36c409cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:55.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.547+0000 7f36ca400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36c4161b00 0x7f36c414f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:55.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.547+0000 7f36ca400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f36c4162ff0 con 0x7f36c4161b00 2024-04-04T06:06:55.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.547+0000 7f36ca400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f36c40b3730 con 0x7f36c40d0d60 2024-04-04T06:06:55.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.547+0000 7f36ca400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f36c40717f0 con 0x7f36c40ad120 2024-04-04T06:06:55.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.547+0000 7f36c3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36c4161b00 0x7f36c414f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:55.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.547+0000 7f36c3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36c4161b00 0x7f36c414f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42958/0 (socket says 172.21.15.159:42958) 2024-04-04T06:06:55.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.547+0000 7f36c3e00640 1 -- 172.21.15.159:0/1282692043 learned_addr learned my addr 172.21.15.159:0/1282692043 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:55.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.548+0000 7f36c3400640 1 --2- 172.21.15.159:0/1282692043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c40ad120 0x7f36c4160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:55.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.548+0000 7f36c2a00640 1 --2- 172.21.15.159:0/1282692043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c40d0d60 0x7f36c409cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:55.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.548+0000 7f36c3e00640 1 -- 172.21.15.159:0/1282692043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c40ad120 msgr2=0x7f36c4160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:55.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.548+0000 7f36c3e00640 1 --2- 172.21.15.159:0/1282692043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c40ad120 0x7f36c4160fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:55.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.548+0000 7f36c3e00640 1 -- 172.21.15.159:0/1282692043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c40d0d60 msgr2=0x7f36c409cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:55.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.548+0000 7f36c3e00640 1 --2- 172.21.15.159:0/1282692043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c40d0d60 0x7f36c409cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:55.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.548+0000 7f36c3e00640 1 -- 172.21.15.159:0/1282692043 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c409a930 con 0x7f36c4161b00 2024-04-04T06:06:55.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.548+0000 7f36c3e00640 1 --2- 172.21.15.159:0/1282692043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36c4161b00 0x7f36c414f590 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f36bc02b9c0 tx=0x7f36bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:55.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.549+0000 7f36bb400640 1 -- 172.21.15.159:0/1282692043 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc048070 con 0x7f36c4161b00 2024-04-04T06:06:55.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.549+0000 7f36bb400640 1 -- 172.21.15.159:0/1282692043 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36bc030640 con 0x7f36c4161b00 2024-04-04T06:06:55.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.549+0000 7f36ca400640 1 -- 172.21.15.159:0/1282692043 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c409ac10 con 0x7f36c4161b00 2024-04-04T06:06:55.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.549+0000 7f36bb400640 1 -- 172.21.15.159:0/1282692043 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc034040 con 0x7f36c4161b00 2024-04-04T06:06:55.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.549+0000 7f36ca400640 1 -- 172.21.15.159:0/1282692043 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c409b1d0 con 0x7f36c4161b00 2024-04-04T06:06:55.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.550+0000 7f36ca400640 1 -- 172.21.15.159:0/1282692043 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c40685c0 con 0x7f36c4161b00 2024-04-04T06:06:55.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.551+0000 7f36bb400640 1 -- 172.21.15.159:0/1282692043 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36bc04b070 con 0x7f36c4161b00 2024-04-04T06:06:55.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.552+0000 7f36bb400640 1 --2- 172.21.15.159:0/1282692043 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f36a406c8f0 0x7f36a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:55.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.552+0000 7f36bb400640 1 -- 172.21.15.159:0/1282692043 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f36bc0adf10 con 0x7f36c4161b00 2024-04-04T06:06:55.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.552+0000 7f36c3400640 1 --2- 172.21.15.159:0/1282692043 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f36a406c8f0 0x7f36a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:55.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.553+0000 7f36c3400640 1 --2- 172.21.15.159:0/1282692043 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f36a406c8f0 0x7f36a406edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f36b4007510 tx=0x7f36b40074a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:55.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.556+0000 7f36bb400640 1 -- 172.21.15.159:0/1282692043 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f36bc07c3b0 con 0x7f36c4161b00 2024-04-04T06:06:55.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.794+0000 7f36ca400640 1 -- 172.21.15.159:0/1282692043 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36c406f3a0 con 0x7f36c4161b00 2024-04-04T06:06:55.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.795+0000 7f36bb400640 1 -- 172.21.15.159:0/1282692043 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5184 (secure 0 0 0) 0x7f36bc07bcb0 con 0x7f36c4161b00 2024-04-04T06:06:55.790 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:55.790 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/789759868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":789759868},{"type":"v1","addr":"172.21.15.178:6836","nonce":789759868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2438288045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2438288045},{"type":"v1","addr":"172.21.15.159:6833","nonce":2438288045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/1705205607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1705205607},{"type":"v1","addr":"172.21.15.159:6835","nonce":1705205607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:05:52.801529+0000","modified":"2024-04-04T06:05:55.866207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6837/3362141613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":3362141613},{"type":"v1","addr":"172.21.15.178:6837","nonce":3362141613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-04T06:06:55.790 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 8 2024-04-04T06:06:55.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.798+0000 7f36ca400640 1 -- 172.21.15.159:0/1282692043 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f36a406c8f0 msgr2=0x7f36a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:55.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.798+0000 7f36ca400640 1 --2- 172.21.15.159:0/1282692043 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f36a406c8f0 0x7f36a406edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f36b4007510 tx=0x7f36b40074a0 comp rx=0 tx=0).stop 2024-04-04T06:06:55.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.798+0000 7f36ca400640 1 -- 172.21.15.159:0/1282692043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36c4161b00 msgr2=0x7f36c414f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:55.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.798+0000 7f36ca400640 1 --2- 172.21.15.159:0/1282692043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36c4161b00 0x7f36c414f590 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f36bc02b9c0 tx=0x7f36bc004730 comp rx=0 tx=0).stop 2024-04-04T06:06:55.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.798+0000 7f36ca400640 1 -- 172.21.15.159:0/1282692043 shutdown_connections 2024-04-04T06:06:55.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.798+0000 7f36ca400640 1 --2- 172.21.15.159:0/1282692043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36c4161b00 0x7f36c414f590 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:55.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.798+0000 7f36ca400640 1 --2- 172.21.15.159:0/1282692043 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f36a406c8f0 0x7f36a406edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:55.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.798+0000 7f36ca400640 1 --2- 172.21.15.159:0/1282692043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c40d0d60 0x7f36c409cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:55.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.798+0000 7f36ca400640 1 --2- 172.21.15.159:0/1282692043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c40ad120 0x7f36c4160fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:55.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.798+0000 7f36ca400640 1 -- 172.21.15.159:0/1282692043 >> 172.21.15.159:0/1282692043 conn(0x7f36c406b990 msgr2=0x7f36c40ae1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:55.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.798+0000 7f36ca400640 1 -- 172.21.15.159:0/1282692043 shutdown_connections 2024-04-04T06:06:55.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:55.798+0000 7f36ca400640 1 -- 172.21.15.159:0/1282692043 wait complete. 2024-04-04T06:06:55.813 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:06:56.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.024+0000 7f2b69c00640 1 Processor -- start 2024-04-04T06:06:56.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.024+0000 7f2b69c00640 1 -- start start 2024-04-04T06:06:56.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.025+0000 7f2b69c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2b64161b00 0x7f2b64162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:56.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.025+0000 7f2b69c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2b640ad120 0x7f2b640d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:56.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.025+0000 7f2b69c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b64071b60 con 0x7f2b640ad580 2024-04-04T06:06:56.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.025+0000 7f2b69c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f2b64161ee0 con 0x7f2b640ad120 2024-04-04T06:06:56.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.025+0000 7f2b69c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2b640b3450 con 0x7f2b64161b00 2024-04-04T06:06:56.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.026+0000 7f2b63400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f2b640ad580 0x7f2b640b3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58560/0 (socket says 172.21.15.159:58560) 2024-04-04T06:06:56.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.026+0000 7f2b63400640 1 -- 172.21.15.159:0/2367862392 learned_addr learned my addr 172.21.15.159:0/2367862392 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:56.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.026+0000 7f2b5be00640 1 --2- 172.21.15.159:0/2367862392 >> v2:172.21.15.159:3300/0 conn(0x7f2b640ad120 0x7f2b640d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:56.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.026+0000 7f2b62a00640 1 --2- 172.21.15.159:0/2367862392 >> v2:172.21.15.178:3300/0 conn(0x7f2b64161b00 0x7f2b64162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:56.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.026+0000 7f2b62000640 1 -- 172.21.15.159:0/2367862392 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1942559893 0 0) 0x7f2b64071b60 con 0x7f2b640ad580 2024-04-04T06:06:56.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.026+0000 7f2b62000640 1 -- 172.21.15.159:0/2367862392 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b44003520 con 0x7f2b640ad580 2024-04-04T06:06:56.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.026+0000 7f2b5be00640 1 -- 172.21.15.159:0/2367862392 >> v2:172.21.15.178:3300/0 conn(0x7f2b64161b00 msgr2=0x7f2b64162ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:56.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.026+0000 7f2b5be00640 1 --2- 172.21.15.159:0/2367862392 >> v2:172.21.15.178:3300/0 conn(0x7f2b64161b00 0x7f2b64162ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.026+0000 7f2b5be00640 1 -- 172.21.15.159:0/2367862392 >> v1:172.21.15.159:6789/0 conn(0x7f2b640ad580 legacy=0x7f2b640b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:56.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.026+0000 7f2b5be00640 1 -- 172.21.15.159:0/2367862392 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b64071e40 con 0x7f2b640ad120 2024-04-04T06:06:56.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.027+0000 7f2b5be00640 1 --2- 172.21.15.159:0/2367862392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b640ad120 0x7f2b640d0650 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2b640b63b0 tx=0x7f2b4c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a61496c68b3222b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:56.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.028+0000 7f2b62000640 1 -- 172.21.15.159:0/2367862392 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c035020 con 0x7f2b640ad120 2024-04-04T06:06:56.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.028+0000 7f2b62000640 1 -- 172.21.15.159:0/2367862392 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b4c031830 con 0x7f2b640ad120 2024-04-04T06:06:56.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.028+0000 7f2b62000640 1 -- 172.21.15.159:0/2367862392 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c031be0 con 0x7f2b640ad120 2024-04-04T06:06:56.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.028+0000 7f2b69c00640 1 -- 172.21.15.159:0/2367862392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b640ad120 msgr2=0x7f2b640d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:56.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.028+0000 7f2b69c00640 1 --2- 172.21.15.159:0/2367862392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b640ad120 0x7f2b640d0650 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2b640b63b0 tx=0x7f2b4c02b8d0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.028+0000 7f2b69c00640 1 -- 172.21.15.159:0/2367862392 shutdown_connections 2024-04-04T06:06:56.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.028+0000 7f2b69c00640 1 --2- 172.21.15.159:0/2367862392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b640ad120 0x7f2b640d0650 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.028+0000 7f2b69c00640 1 --2- 172.21.15.159:0/2367862392 >> v2:172.21.15.178:3300/0 conn(0x7f2b64161b00 0x7f2b64162ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.028+0000 7f2b69c00640 1 -- 172.21.15.159:0/2367862392 >> 172.21.15.159:0/2367862392 conn(0x7f2b6406b990 msgr2=0x7f2b6406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:56.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.028+0000 7f2b69c00640 1 -- 172.21.15.159:0/2367862392 shutdown_connections 2024-04-04T06:06:56.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.028+0000 7f2b69c00640 1 -- 172.21.15.159:0/2367862392 wait complete. 2024-04-04T06:06:56.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.030+0000 7f2b69c00640 1 Processor -- start 2024-04-04T06:06:56.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.030+0000 7f2b69c00640 1 -- start start 2024-04-04T06:06:56.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.031+0000 7f2b69c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b640ad120 0x7f2b64159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:56.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.031+0000 7f2b69c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b640ad580 0x7f2b6415da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:56.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.031+0000 7f2b69c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b64161b00 0x7f2b64153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:56.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.031+0000 7f2b69c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2b64162ff0 con 0x7f2b640ad120 2024-04-04T06:06:56.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.031+0000 7f2b69c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2b640b3730 con 0x7f2b640ad580 2024-04-04T06:06:56.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.031+0000 7f2b69c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2b640717f0 con 0x7f2b64161b00 2024-04-04T06:06:56.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.031+0000 7f2b62a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b640ad120 0x7f2b64159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:56.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.031+0000 7f2b62a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b640ad120 0x7f2b64159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42982/0 (socket says 172.21.15.159:42982) 2024-04-04T06:06:56.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.031+0000 7f2b62a00640 1 -- 172.21.15.159:0/2811063671 learned_addr learned my addr 172.21.15.159:0/2811063671 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:56.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.032+0000 7f2b63400640 1 --2- 172.21.15.159:0/2811063671 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b64161b00 0x7f2b64153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:56.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.032+0000 7f2b62a00640 1 -- 172.21.15.159:0/2811063671 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b64161b00 msgr2=0x7f2b64153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:56.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.032+0000 7f2b62a00640 1 --2- 172.21.15.159:0/2811063671 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b64161b00 0x7f2b64153570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.032+0000 7f2b5be00640 1 --2- 172.21.15.159:0/2811063671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b640ad580 0x7f2b6415da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:56.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.032+0000 7f2b62a00640 1 -- 172.21.15.159:0/2811063671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b640ad580 msgr2=0x7f2b6415da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:56.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.032+0000 7f2b62a00640 1 --2- 172.21.15.159:0/2811063671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b640ad580 0x7f2b6415da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.032+0000 7f2b62a00640 1 -- 172.21.15.159:0/2811063671 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b64160500 con 0x7f2b640ad120 2024-04-04T06:06:56.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.032+0000 7f2b62a00640 1 --2- 172.21.15.159:0/2811063671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b640ad120 0x7f2b64159630 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400e9b0 tx=0x7f2b5400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:56.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.032+0000 7f2b5aa00640 1 -- 172.21.15.159:0/2811063671 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54017070 con 0x7f2b640ad120 2024-04-04T06:06:56.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.033+0000 7f2b5aa00640 1 -- 172.21.15.159:0/2811063671 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b5400c9a0 con 0x7f2b640ad120 2024-04-04T06:06:56.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.033+0000 7f2b5aa00640 1 -- 172.21.15.159:0/2811063671 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5400cd90 con 0x7f2b640ad120 2024-04-04T06:06:56.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.033+0000 7f2b69c00640 1 -- 172.21.15.159:0/2811063671 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b641607a0 con 0x7f2b640ad120 2024-04-04T06:06:56.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.033+0000 7f2b69c00640 1 -- 172.21.15.159:0/2811063671 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b641f44d0 con 0x7f2b640ad120 2024-04-04T06:06:56.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.034+0000 7f2b69c00640 1 -- 172.21.15.159:0/2811063671 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b640685c0 con 0x7f2b640ad120 2024-04-04T06:06:56.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.035+0000 7f2b5aa00640 1 -- 172.21.15.159:0/2811063671 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b5401a070 con 0x7f2b640ad120 2024-04-04T06:06:56.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.036+0000 7f2b5aa00640 1 --2- 172.21.15.159:0/2811063671 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2b4006c8f0 0x7f2b4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:56.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.036+0000 7f2b5aa00640 1 -- 172.21.15.159:0/2811063671 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f2b5408bc20 con 0x7f2b640ad120 2024-04-04T06:06:56.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.036+0000 7f2b5be00640 1 --2- 172.21.15.159:0/2811063671 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2b4006c8f0 0x7f2b4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:56.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.037+0000 7f2b5be00640 1 --2- 172.21.15.159:0/2811063671 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2b4006c8f0 0x7f2b4006edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c036000 tx=0x7f2b4c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:56.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.040+0000 7f2b5aa00640 1 -- 172.21.15.159:0/2811063671 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2b54059f90 con 0x7f2b640ad120 2024-04-04T06:06:56.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.283+0000 7f2b69c00640 1 -- 172.21.15.159:0/2811063671 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b6406f3a0 con 0x7f2b640ad120 2024-04-04T06:06:56.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.284+0000 7f2b5aa00640 1 -- 172.21.15.159:0/2811063671 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5184 (secure 0 0 0) 0x7f2b54059890 con 0x7f2b640ad120 2024-04-04T06:06:56.279 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:56.279 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/789759868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":789759868},{"type":"v1","addr":"172.21.15.178:6836","nonce":789759868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2438288045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2438288045},{"type":"v1","addr":"172.21.15.159:6833","nonce":2438288045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/1705205607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1705205607},{"type":"v1","addr":"172.21.15.159:6835","nonce":1705205607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:05:52.801529+0000","modified":"2024-04-04T06:05:55.866207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.178:6837/3362141613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":3362141613},{"type":"v1","addr":"172.21.15.178:6837","nonce":3362141613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293]},"id":1}]} 2024-04-04T06:06:56.280 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 8 2024-04-04T06:06:56.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.287+0000 7f2b69c00640 1 -- 172.21.15.159:0/2811063671 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2b4006c8f0 msgr2=0x7f2b4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:56.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.287+0000 7f2b69c00640 1 --2- 172.21.15.159:0/2811063671 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2b4006c8f0 0x7f2b4006edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c036000 tx=0x7f2b4c008030 comp rx=0 tx=0).stop 2024-04-04T06:06:56.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.288+0000 7f2b69c00640 1 -- 172.21.15.159:0/2811063671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b640ad120 msgr2=0x7f2b64159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:56.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.288+0000 7f2b69c00640 1 --2- 172.21.15.159:0/2811063671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b640ad120 0x7f2b64159630 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400e9b0 tx=0x7f2b5400ee70 comp rx=0 tx=0).stop 2024-04-04T06:06:56.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.288+0000 7f2b69c00640 1 -- 172.21.15.159:0/2811063671 shutdown_connections 2024-04-04T06:06:56.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.288+0000 7f2b69c00640 1 --2- 172.21.15.159:0/2811063671 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b64161b00 0x7f2b64153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.288+0000 7f2b69c00640 1 --2- 172.21.15.159:0/2811063671 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2b4006c8f0 0x7f2b4006edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.288+0000 7f2b69c00640 1 --2- 172.21.15.159:0/2811063671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b640ad580 0x7f2b6415da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.288+0000 7f2b69c00640 1 --2- 172.21.15.159:0/2811063671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b640ad120 0x7f2b64159630 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.288+0000 7f2b69c00640 1 -- 172.21.15.159:0/2811063671 >> 172.21.15.159:0/2811063671 conn(0x7f2b6406b990 msgr2=0x7f2b640b1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:56.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.288+0000 7f2b69c00640 1 -- 172.21.15.159:0/2811063671 shutdown_connections 2024-04-04T06:06:56.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.288+0000 7f2b69c00640 1 -- 172.21.15.159:0/2811063671 wait complete. 2024-04-04T06:06:56.302 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:06:56.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.517+0000 7fe9f1400640 1 Processor -- start 2024-04-04T06:06:56.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.518+0000 7fe9f1400640 1 -- start start 2024-04-04T06:06:56.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.519+0000 7fe9f1400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe9ec0af830 0x7fe9ec073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:56.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.519+0000 7fe9f1400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe9ec0d0650 0x7fe9ec0d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:56.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.519+0000 7fe9f1400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9ec15fe80 con 0x7fe9ec15c390 2024-04-04T06:06:56.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.519+0000 7fe9f1400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe9ec15c770 con 0x7fe9ec0d0650 2024-04-04T06:06:56.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.519+0000 7fe9f1400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe9ec0d0fc0 con 0x7fe9ec0af830 2024-04-04T06:06:56.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.519+0000 7fe9ea800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fe9ec15c390 0x7fe9ec160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58572/0 (socket says 172.21.15.159:58572) 2024-04-04T06:06:56.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.519+0000 7fe9ea800640 1 -- 172.21.15.159:0/1310103962 learned_addr learned my addr 172.21.15.159:0/1310103962 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:06:56.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.519+0000 7fe9eb200640 1 --2- 172.21.15.159:0/1310103962 >> v2:172.21.15.159:3300/0 conn(0x7fe9ec0d0650 0x7fe9ec0d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:56.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.520+0000 7fe9e1e00640 1 --2- 172.21.15.159:0/1310103962 >> v2:172.21.15.178:3300/0 conn(0x7fe9ec0af830 0x7fe9ec073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:56.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.520+0000 7fe9e9e00640 1 -- 172.21.15.159:0/1310103962 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1112106803 0 0) 0x7fe9ec15fe80 con 0x7fe9ec15c390 2024-04-04T06:06:56.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.520+0000 7fe9e9e00640 1 -- 172.21.15.159:0/1310103962 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9d0003520 con 0x7fe9ec15c390 2024-04-04T06:06:56.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.520+0000 7fe9eb200640 1 -- 172.21.15.159:0/1310103962 >> v2:172.21.15.178:3300/0 conn(0x7fe9ec0af830 msgr2=0x7fe9ec073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:56.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.520+0000 7fe9eb200640 1 --2- 172.21.15.159:0/1310103962 >> v2:172.21.15.178:3300/0 conn(0x7fe9ec0af830 0x7fe9ec073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.520+0000 7fe9eb200640 1 -- 172.21.15.159:0/1310103962 >> v1:172.21.15.159:6789/0 conn(0x7fe9ec15c390 legacy=0x7fe9ec160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:56.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.520+0000 7fe9eb200640 1 -- 172.21.15.159:0/1310103962 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ec0ad120 con 0x7fe9ec0d0650 2024-04-04T06:06:56.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.521+0000 7fe9eb200640 1 --2- 172.21.15.159:0/1310103962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ec0d0650 0x7fe9ec0d0a80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec06eaa0 tx=0x7fe9e402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1276427bda6e301 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:56.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.521+0000 7fe9e9e00640 1 -- 172.21.15.159:0/1310103962 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4039070 con 0x7fe9ec0d0650 2024-04-04T06:06:56.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.521+0000 7fe9e9e00640 1 -- 172.21.15.159:0/1310103962 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9e402cc60 con 0x7fe9ec0d0650 2024-04-04T06:06:56.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.521+0000 7fe9e9e00640 1 -- 172.21.15.159:0/1310103962 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4034660 con 0x7fe9ec0d0650 2024-04-04T06:06:56.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.522+0000 7fe9f1400640 1 -- 172.21.15.159:0/1310103962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ec0d0650 msgr2=0x7fe9ec0d0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:56.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.522+0000 7fe9f1400640 1 --2- 172.21.15.159:0/1310103962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ec0d0650 0x7fe9ec0d0a80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec06eaa0 tx=0x7fe9e402beb0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.522+0000 7fe9f1400640 1 -- 172.21.15.159:0/1310103962 shutdown_connections 2024-04-04T06:06:56.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.522+0000 7fe9f1400640 1 --2- 172.21.15.159:0/1310103962 >> v2:172.21.15.178:3300/0 conn(0x7fe9ec0af830 0x7fe9ec073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.522+0000 7fe9f1400640 1 --2- 172.21.15.159:0/1310103962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ec0d0650 0x7fe9ec0d0a80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.522+0000 7fe9f1400640 1 -- 172.21.15.159:0/1310103962 >> 172.21.15.159:0/1310103962 conn(0x7fe9ec06b990 msgr2=0x7fe9ec06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:56.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.522+0000 7fe9f1400640 1 -- 172.21.15.159:0/1310103962 shutdown_connections 2024-04-04T06:06:56.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.522+0000 7fe9f1400640 1 -- 172.21.15.159:0/1310103962 wait complete. 2024-04-04T06:06:56.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.524+0000 7fe9f1400640 1 Processor -- start 2024-04-04T06:06:56.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.524+0000 7fe9f1400640 1 -- start start 2024-04-04T06:06:56.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.525+0000 7fe9f1400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9ec0af830 0x7fe9ec15edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:56.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.525+0000 7fe9f1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ec0d0650 0x7fe9ec1453e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:56.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.525+0000 7fe9f1400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9ec15c390 0x7fe9ec157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:56.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.525+0000 7fe9f1400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec160120 con 0x7fe9ec0d0650 2024-04-04T06:06:56.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.525+0000 7fe9f1400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec0ad6e0 con 0x7fe9ec15c390 2024-04-04T06:06:56.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.525+0000 7fe9f1400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ec160d50 con 0x7fe9ec0af830 2024-04-04T06:06:56.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.525+0000 7fe9e1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ec0d0650 0x7fe9ec1453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:56.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.525+0000 7fe9e1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ec0d0650 0x7fe9ec1453e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43006/0 (socket says 172.21.15.159:43006) 2024-04-04T06:06:56.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.525+0000 7fe9e1e00640 1 -- 172.21.15.159:0/1728271066 learned_addr learned my addr 172.21.15.159:0/1728271066 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:56.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.525+0000 7fe9ea800640 1 --2- 172.21.15.159:0/1728271066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9ec0af830 0x7fe9ec15edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:56.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.526+0000 7fe9eb200640 1 --2- 172.21.15.159:0/1728271066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9ec15c390 0x7fe9ec157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:56.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.526+0000 7fe9e1e00640 1 -- 172.21.15.159:0/1728271066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9ec0af830 msgr2=0x7fe9ec15edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:56.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.526+0000 7fe9e1e00640 1 --2- 172.21.15.159:0/1728271066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9ec0af830 0x7fe9ec15edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.526+0000 7fe9e1e00640 1 -- 172.21.15.159:0/1728271066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9ec15c390 msgr2=0x7fe9ec157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:56.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.526+0000 7fe9e1e00640 1 --2- 172.21.15.159:0/1728271066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9ec15c390 0x7fe9ec157580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.526+0000 7fe9e1e00640 1 -- 172.21.15.159:0/1728271066 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ec145920 con 0x7fe9ec0d0650 2024-04-04T06:06:56.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.526+0000 7fe9e1e00640 1 --2- 172.21.15.159:0/1728271066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ec0d0650 0x7fe9ec1453e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400d620 tx=0x7fe9d400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:56.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.526+0000 7fe9e3400640 1 -- 172.21.15.159:0/1728271066 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400b620 con 0x7fe9ec0d0650 2024-04-04T06:06:56.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.526+0000 7fe9e3400640 1 -- 172.21.15.159:0/1728271066 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9d4019070 con 0x7fe9ec0d0650 2024-04-04T06:06:56.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.526+0000 7fe9e3400640 1 -- 172.21.15.159:0/1728271066 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400ba00 con 0x7fe9ec0d0650 2024-04-04T06:06:56.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.526+0000 7fe9f1400640 1 -- 172.21.15.159:0/1728271066 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ec0b0120 con 0x7fe9ec0d0650 2024-04-04T06:06:56.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.527+0000 7fe9f1400640 1 -- 172.21.15.159:0/1728271066 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ec0b06e0 con 0x7fe9ec0d0650 2024-04-04T06:06:56.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.528+0000 7fe9f1400640 1 -- 172.21.15.159:0/1728271066 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ec070100 con 0x7fe9ec0d0650 2024-04-04T06:06:56.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.529+0000 7fe9e3400640 1 -- 172.21.15.159:0/1728271066 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9d400bba0 con 0x7fe9ec0d0650 2024-04-04T06:06:56.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.529+0000 7fe9e3400640 1 --2- 172.21.15.159:0/1728271066 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe9c806c8f0 0x7fe9c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:56.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.530+0000 7fe9e3400640 1 -- 172.21.15.159:0/1728271066 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7fe9d408bfe0 con 0x7fe9ec0d0650 2024-04-04T06:06:56.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.530+0000 7fe9ea800640 1 --2- 172.21.15.159:0/1728271066 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe9c806c8f0 0x7fe9c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:56.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.531+0000 7fe9ea800640 1 --2- 172.21.15.159:0/1728271066 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe9c806c8f0 0x7fe9c806edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc007fd0 tx=0x7fe9dc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:56.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.534+0000 7fe9e3400640 1 -- 172.21.15.159:0/1728271066 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe9ec070100 con 0x7fe9ec0d0650 2024-04-04T06:06:56.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.735+0000 7fe9f1400640 1 -- 172.21.15.159:0/1728271066 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9ec0685c0 con 0x7fe9ec0d0650 2024-04-04T06:06:56.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.738+0000 7fe9e3400640 1 -- 172.21.15.159:0/1728271066 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16958 (secure 0 0 0) 0x7fe9d405a3b0 con 0x7fe9ec0d0650 2024-04-04T06:06:56.733 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:06:56.733 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":23,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:06:33.796979+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:05:51.297293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-04T06:05:51.770792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":0,"to":2}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:06:56.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.741+0000 7fe9f1400640 1 -- 172.21.15.159:0/1728271066 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe9c806c8f0 msgr2=0x7fe9c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:56.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.741+0000 7fe9f1400640 1 --2- 172.21.15.159:0/1728271066 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe9c806c8f0 0x7fe9c806edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc007fd0 tx=0x7fe9dc009040 comp rx=0 tx=0).stop 2024-04-04T06:06:56.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.741+0000 7fe9f1400640 1 -- 172.21.15.159:0/1728271066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ec0d0650 msgr2=0x7fe9ec1453e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:56.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.741+0000 7fe9f1400640 1 --2- 172.21.15.159:0/1728271066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ec0d0650 0x7fe9ec1453e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400d620 tx=0x7fe9d400dae0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.741+0000 7fe9f1400640 1 -- 172.21.15.159:0/1728271066 shutdown_connections 2024-04-04T06:06:56.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.741+0000 7fe9f1400640 1 --2- 172.21.15.159:0/1728271066 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe9c806c8f0 0x7fe9c806edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.741+0000 7fe9f1400640 1 --2- 172.21.15.159:0/1728271066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9ec15c390 0x7fe9ec157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.741+0000 7fe9f1400640 1 --2- 172.21.15.159:0/1728271066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ec0d0650 0x7fe9ec1453e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.741+0000 7fe9f1400640 1 --2- 172.21.15.159:0/1728271066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9ec0af830 0x7fe9ec15edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.741+0000 7fe9f1400640 1 -- 172.21.15.159:0/1728271066 >> 172.21.15.159:0/1728271066 conn(0x7fe9ec06b990 msgr2=0x7fe9ec071cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:56.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.741+0000 7fe9f1400640 1 -- 172.21.15.159:0/1728271066 shutdown_connections 2024-04-04T06:06:56.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.741+0000 7fe9f1400640 1 -- 172.21.15.159:0/1728271066 wait complete. 2024-04-04T06:06:56.754 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-04T06:06:56.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.978+0000 7f3380200640 1 Processor -- start 2024-04-04T06:06:56.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.978+0000 7f3380200640 1 -- start start 2024-04-04T06:06:56.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.980+0000 7f3380200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f337815d490 0x7f33780cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:56.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.980+0000 7f3380200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f33780716d0 0x7f33780d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:56.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.980+0000 7f3380200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3378071c80 con 0x7f33780b4a30 2024-04-04T06:06:56.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.980+0000 7f3380200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f3378071f60 con 0x7f33780716d0 2024-04-04T06:06:56.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.980+0000 7f3380200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f33780b3f50 con 0x7f337815d490 2024-04-04T06:06:56.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.980+0000 7f337e200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f33780716d0 0x7f33780d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:56.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.980+0000 7f337e200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f33780716d0 0x7f33780d02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43022/0 (socket says 172.21.15.159:43022) 2024-04-04T06:06:56.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.980+0000 7f337e200640 1 -- 172.21.15.159:0/2751241011 learned_addr learned my addr 172.21.15.159:0/2751241011 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:56.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.981+0000 7f337e200640 1 -- 172.21.15.159:0/2751241011 >> v2:172.21.15.178:3300/0 conn(0x7f337815d490 msgr2=0x7f33780cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:56.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.981+0000 7f337d800640 1 --2- 172.21.15.159:0/2751241011 >> v2:172.21.15.178:3300/0 conn(0x7f337815d490 0x7f33780cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:56.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.981+0000 7f337e200640 1 --2- 172.21.15.159:0/2751241011 >> v2:172.21.15.178:3300/0 conn(0x7f337815d490 0x7f33780cfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.981+0000 7f337e200640 1 -- 172.21.15.159:0/2751241011 >> v1:172.21.15.178:6789/0 conn(0x7f33780b4a30 legacy=0x7f3378070fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:56.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.981+0000 7f337e200640 1 -- 172.21.15.159:0/2751241011 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33780b4230 con 0x7f33780716d0 2024-04-04T06:06:56.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.981+0000 7f337d800640 1 --2- 172.21.15.159:0/2751241011 >> v2:172.21.15.178:3300/0 conn(0x7f337815d490 0x7f33780cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:56.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.982+0000 7f337e200640 1 --2- 172.21.15.159:0/2751241011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33780716d0 0x7f33780d02c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f337806c520 tx=0x7f337002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbeacc4b7872e78 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:56.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.982+0000 7f3377e00640 1 -- 172.21.15.159:0/2751241011 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337002c9f0 con 0x7f33780716d0 2024-04-04T06:06:56.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.982+0000 7f3377e00640 1 -- 172.21.15.159:0/2751241011 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f337002cb90 con 0x7f33780716d0 2024-04-04T06:06:56.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.982+0000 7f3377e00640 1 -- 172.21.15.159:0/2751241011 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33700343f0 con 0x7f33780716d0 2024-04-04T06:06:56.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.983+0000 7f3380200640 1 -- 172.21.15.159:0/2751241011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33780716d0 msgr2=0x7f33780d02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:56.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.983+0000 7f3380200640 1 --2- 172.21.15.159:0/2751241011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33780716d0 0x7f33780d02c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f337806c520 tx=0x7f337002be50 comp rx=0 tx=0).stop 2024-04-04T06:06:56.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.983+0000 7f3380200640 1 -- 172.21.15.159:0/2751241011 shutdown_connections 2024-04-04T06:06:56.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.983+0000 7f3380200640 1 --2- 172.21.15.159:0/2751241011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33780716d0 0x7f33780d02c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.983+0000 7f3380200640 1 --2- 172.21.15.159:0/2751241011 >> v2:172.21.15.178:3300/0 conn(0x7f337815d490 0x7f33780cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.983+0000 7f3380200640 1 -- 172.21.15.159:0/2751241011 >> 172.21.15.159:0/2751241011 conn(0x7f33780d2240 msgr2=0x7f33780d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:56.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.983+0000 7f3380200640 1 -- 172.21.15.159:0/2751241011 shutdown_connections 2024-04-04T06:06:56.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.983+0000 7f3380200640 1 -- 172.21.15.159:0/2751241011 wait complete. 2024-04-04T06:06:56.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.985+0000 7f3380200640 1 Processor -- start 2024-04-04T06:06:56.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.985+0000 7f3380200640 1 -- start start 2024-04-04T06:06:56.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.986+0000 7f3380200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33780716d0 0x7f33780b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:56.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.986+0000 7f3380200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33780b4a30 0x7f337814bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:56.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.986+0000 7f3380200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f337815d490 0x7f33780cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:56.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.986+0000 7f3380200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f337815d9f0 con 0x7f337815d490 2024-04-04T06:06:56.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.986+0000 7f3380200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f337815d870 con 0x7f33780716d0 2024-04-04T06:06:56.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.986+0000 7f3380200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f33780c74a0 con 0x7f33780b4a30 2024-04-04T06:06:56.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.986+0000 7f337e200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f337815d490 0x7f33780cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:56.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.986+0000 7f337e200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f337815d490 0x7f33780cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43036/0 (socket says 172.21.15.159:43036) 2024-04-04T06:06:56.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.986+0000 7f337e200640 1 -- 172.21.15.159:0/616825517 learned_addr learned my addr 172.21.15.159:0/616825517 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:56.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.987+0000 7f337d800640 1 --2- 172.21.15.159:0/616825517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33780716d0 0x7f33780b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:56.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.987+0000 7f337ce00640 1 --2- 172.21.15.159:0/616825517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33780b4a30 0x7f337814bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:56.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.987+0000 7f337e200640 1 -- 172.21.15.159:0/616825517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33780b4a30 msgr2=0x7f337814bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:56.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.987+0000 7f337e200640 1 --2- 172.21.15.159:0/616825517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33780b4a30 0x7f337814bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.987+0000 7f337e200640 1 -- 172.21.15.159:0/616825517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33780716d0 msgr2=0x7f33780b6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:56.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.987+0000 7f337e200640 1 --2- 172.21.15.159:0/616825517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33780716d0 0x7f33780b6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:56.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.987+0000 7f337e200640 1 -- 172.21.15.159:0/616825517 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337814c2b0 con 0x7f337815d490 2024-04-04T06:06:56.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.987+0000 7f337e200640 1 --2- 172.21.15.159:0/616825517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f337815d490 0x7f33780cbd00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3370036040 tx=0x7f3370004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:56.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.987+0000 7f3375600640 1 -- 172.21.15.159:0/616825517 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370043020 con 0x7f337815d490 2024-04-04T06:06:56.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.987+0000 7f3375600640 1 -- 172.21.15.159:0/616825517 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f337003e450 con 0x7f337815d490 2024-04-04T06:06:56.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.987+0000 7f3375600640 1 -- 172.21.15.159:0/616825517 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337003e820 con 0x7f337815d490 2024-04-04T06:06:56.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.987+0000 7f3380200640 1 -- 172.21.15.159:0/616825517 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33780b5aa0 con 0x7f337815d490 2024-04-04T06:06:56.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.988+0000 7f3380200640 1 -- 172.21.15.159:0/616825517 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33780b6060 con 0x7f337815d490 2024-04-04T06:06:56.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.989+0000 7f3380200640 1 -- 172.21.15.159:0/616825517 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33780685c0 con 0x7f337815d490 2024-04-04T06:06:56.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.990+0000 7f3375600640 1 -- 172.21.15.159:0/616825517 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3370046020 con 0x7f337815d490 2024-04-04T06:06:56.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.990+0000 7f3375600640 1 --2- 172.21.15.159:0/616825517 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f335c06c8f0 0x7f335c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:56.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.991+0000 7f3375600640 1 -- 172.21.15.159:0/616825517 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5883+0+0 (secure 0 0 0) 0x7f33700ae550 con 0x7f337815d490 2024-04-04T06:06:56.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.991+0000 7f337d800640 1 --2- 172.21.15.159:0/616825517 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f335c06c8f0 0x7f335c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:56.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.992+0000 7f337d800640 1 --2- 172.21.15.159:0/616825517 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f335c06c8f0 0x7f335c06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f336c00a950 tx=0x7f336c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:56.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:56.995+0000 7f3375600640 1 -- 172.21.15.159:0/616825517 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f337004d030 con 0x7f337815d490 2024-04-04T06:06:57.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.230+0000 7f3380200640 1 -- 172.21.15.159:0/616825517 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f33780c8fc0 con 0x7f337815d490 2024-04-04T06:06:57.226 INFO:tasks.ceph.mon.a.smithi159.stderr:2024-04-04T06:06:57.231+0000 7f1995c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-04T06:06:57.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.235+0000 7f3375600640 1 -- 172.21.15.159:0/616825517 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f337814c2b0 con 0x7f337815d490 2024-04-04T06:06:57.230 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-04T06:06:57.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.238+0000 7f3380200640 1 -- 172.21.15.159:0/616825517 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f335c06c8f0 msgr2=0x7f335c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:57.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.238+0000 7f3380200640 1 --2- 172.21.15.159:0/616825517 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f335c06c8f0 0x7f335c06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f336c00a950 tx=0x7f336c008040 comp rx=0 tx=0).stop 2024-04-04T06:06:57.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.238+0000 7f3380200640 1 -- 172.21.15.159:0/616825517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f337815d490 msgr2=0x7f33780cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:57.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.238+0000 7f3380200640 1 --2- 172.21.15.159:0/616825517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f337815d490 0x7f33780cbd00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3370036040 tx=0x7f3370004a40 comp rx=0 tx=0).stop 2024-04-04T06:06:57.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.238+0000 7f3380200640 1 -- 172.21.15.159:0/616825517 shutdown_connections 2024-04-04T06:06:57.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.238+0000 7f3380200640 1 --2- 172.21.15.159:0/616825517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f337815d490 0x7f33780cbd00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:57.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.238+0000 7f3380200640 1 --2- 172.21.15.159:0/616825517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33780b4a30 0x7f337814bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:57.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.238+0000 7f3380200640 1 --2- 172.21.15.159:0/616825517 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f335c06c8f0 0x7f335c06edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:57.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.238+0000 7f3380200640 1 --2- 172.21.15.159:0/616825517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33780716d0 0x7f33780b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:57.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.238+0000 7f3380200640 1 -- 172.21.15.159:0/616825517 >> 172.21.15.159:0/616825517 conn(0x7f33780d2240 msgr2=0x7f3378155400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:57.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.238+0000 7f3380200640 1 -- 172.21.15.159:0/616825517 shutdown_connections 2024-04-04T06:06:57.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.238+0000 7f3380200640 1 -- 172.21.15.159:0/616825517 wait complete. 2024-04-04T06:06:57.251 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-04T06:06:57.270 INFO:tasks.ceph.mds.b.smithi178.stderr: -8674> 2024-04-04T06:05:55.309+0000 7f36b5000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:06:57.346 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-04T06:06:57.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.452+0000 7fdd7d400640 1 Processor -- start 2024-04-04T06:06:57.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.452+0000 7fdd7d400640 1 -- start start 2024-04-04T06:06:57.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.453+0000 7fdd7d400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdd780d0990 0x7fdd780aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:57.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.453+0000 7fdd7d400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fdd780af2e0 0x7fdd781600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:57.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.453+0000 7fdd7d400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd780afab0 con 0x7fdd78153580 2024-04-04T06:06:57.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.453+0000 7fdd7d400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fdd7815cc80 con 0x7fdd780af2e0 2024-04-04T06:06:57.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.453+0000 7fdd7d400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fdd7815da00 con 0x7fdd780d0990 2024-04-04T06:06:57.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.454+0000 7fdd77200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fdd780af2e0 0x7fdd781600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:57.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.454+0000 7fdd77200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fdd780af2e0 0x7fdd781600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43040/0 (socket says 172.21.15.159:43040) 2024-04-04T06:06:57.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.454+0000 7fdd77200640 1 -- 172.21.15.159:0/3552233088 learned_addr learned my addr 172.21.15.159:0/3552233088 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:57.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.454+0000 7fdd77200640 1 -- 172.21.15.159:0/3552233088 >> v2:172.21.15.178:3300/0 conn(0x7fdd780d0990 msgr2=0x7fdd780aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:57.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.454+0000 7fdd75e00640 1 --2- 172.21.15.159:0/3552233088 >> v2:172.21.15.178:3300/0 conn(0x7fdd780d0990 0x7fdd780aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:57.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.454+0000 7fdd77200640 1 --2- 172.21.15.159:0/3552233088 >> v2:172.21.15.178:3300/0 conn(0x7fdd780d0990 0x7fdd780aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:57.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.454+0000 7fdd77200640 1 -- 172.21.15.159:0/3552233088 >> v1:172.21.15.178:6789/0 conn(0x7fdd78153580 legacy=0x7fdd780d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:57.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.454+0000 7fdd77200640 1 -- 172.21.15.159:0/3552233088 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7815cf20 con 0x7fdd780af2e0 2024-04-04T06:06:57.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.454+0000 7fdd75e00640 1 --2- 172.21.15.159:0/3552233088 >> v2:172.21.15.178:3300/0 conn(0x7fdd780d0990 0x7fdd780aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:57.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.455+0000 7fdd77200640 1 --2- 172.21.15.159:0/3552233088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd780af2e0 0x7fdd781600d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdd7815a6b0 tx=0x7fdd7002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0f7df753f38349 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:57.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.456+0000 7fdd75400640 1 -- 172.21.15.159:0/3552233088 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7002c9f0 con 0x7fdd780af2e0 2024-04-04T06:06:57.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.456+0000 7fdd75400640 1 -- 172.21.15.159:0/3552233088 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd7002cb90 con 0x7fdd780af2e0 2024-04-04T06:06:57.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.456+0000 7fdd75400640 1 -- 172.21.15.159:0/3552233088 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd700343f0 con 0x7fdd780af2e0 2024-04-04T06:06:57.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.456+0000 7fdd7d400640 1 -- 172.21.15.159:0/3552233088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd780af2e0 msgr2=0x7fdd781600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:57.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.456+0000 7fdd7d400640 1 --2- 172.21.15.159:0/3552233088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd780af2e0 0x7fdd781600d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdd7815a6b0 tx=0x7fdd7002be50 comp rx=0 tx=0).stop 2024-04-04T06:06:57.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.456+0000 7fdd7d400640 1 -- 172.21.15.159:0/3552233088 shutdown_connections 2024-04-04T06:06:57.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.456+0000 7fdd7d400640 1 --2- 172.21.15.159:0/3552233088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd780af2e0 0x7fdd781600d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:57.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.456+0000 7fdd7d400640 1 --2- 172.21.15.159:0/3552233088 >> v2:172.21.15.178:3300/0 conn(0x7fdd780d0990 0x7fdd780aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:57.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.456+0000 7fdd7d400640 1 -- 172.21.15.159:0/3552233088 >> 172.21.15.159:0/3552233088 conn(0x7fdd7806b750 msgr2=0x7fdd7806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:57.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.456+0000 7fdd7d400640 1 -- 172.21.15.159:0/3552233088 shutdown_connections 2024-04-04T06:06:57.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.456+0000 7fdd7d400640 1 -- 172.21.15.159:0/3552233088 wait complete. 2024-04-04T06:06:57.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.458+0000 7fdd7d400640 1 Processor -- start 2024-04-04T06:06:57.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.458+0000 7fdd7d400640 1 -- start start 2024-04-04T06:06:57.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.459+0000 7fdd7d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd780af2e0 0x7fdd78156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:57.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.459+0000 7fdd7d400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd780d0990 0x7fdd7815b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:57.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.459+0000 7fdd7d400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd78153580 0x7fdd78153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:57.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.459+0000 7fdd7d400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdd78153960 con 0x7fdd780af2e0 2024-04-04T06:06:57.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.459+0000 7fdd7d400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7815d4e0 con 0x7fdd78153580 2024-04-04T06:06:57.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.459+0000 7fdd7d400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdd780af6c0 con 0x7fdd780d0990 2024-04-04T06:06:57.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.459+0000 7fdd76800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd780af2e0 0x7fdd78156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:57.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.459+0000 7fdd76800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd780af2e0 0x7fdd78156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43048/0 (socket says 172.21.15.159:43048) 2024-04-04T06:06:57.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.459+0000 7fdd76800640 1 -- 172.21.15.159:0/1413987589 learned_addr learned my addr 172.21.15.159:0/1413987589 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:57.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.460+0000 7fdd76800640 1 -- 172.21.15.159:0/1413987589 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd780d0990 msgr2=0x7fdd7815b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:57.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.460+0000 7fdd76800640 1 --2- 172.21.15.159:0/1413987589 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd780d0990 0x7fdd7815b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:57.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.460+0000 7fdd77200640 1 --2- 172.21.15.159:0/1413987589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd78153580 0x7fdd78153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:57.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.460+0000 7fdd76800640 1 -- 172.21.15.159:0/1413987589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd78153580 msgr2=0x7fdd78153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:57.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.460+0000 7fdd76800640 1 --2- 172.21.15.159:0/1413987589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd78153580 0x7fdd78153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:57.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.460+0000 7fdd76800640 1 -- 172.21.15.159:0/1413987589 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd78151f50 con 0x7fdd780af2e0 2024-04-04T06:06:57.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.460+0000 7fdd76800640 1 --2- 172.21.15.159:0/1413987589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd780af2e0 0x7fdd78156950 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800c430 tx=0x7fdd6800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:57.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.460+0000 7fdd6ea00640 1 -- 172.21.15.159:0/1413987589 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68011790 con 0x7fdd780af2e0 2024-04-04T06:06:57.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.460+0000 7fdd6ea00640 1 -- 172.21.15.159:0/1413987589 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd68017070 con 0x7fdd780af2e0 2024-04-04T06:06:57.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.460+0000 7fdd7d400640 1 -- 172.21.15.159:0/1413987589 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd78155e70 con 0x7fdd780af2e0 2024-04-04T06:06:57.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.460+0000 7fdd6ea00640 1 -- 172.21.15.159:0/1413987589 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68011bb0 con 0x7fdd780af2e0 2024-04-04T06:06:57.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.460+0000 7fdd7d400640 1 -- 172.21.15.159:0/1413987589 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd78156430 con 0x7fdd780af2e0 2024-04-04T06:06:57.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.461+0000 7fdd7d400640 1 -- 172.21.15.159:0/1413987589 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd780685c0 con 0x7fdd780af2e0 2024-04-04T06:06:57.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.462+0000 7fdd6ea00640 1 -- 172.21.15.159:0/1413987589 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd6801c460 con 0x7fdd780af2e0 2024-04-04T06:06:57.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.463+0000 7fdd6ea00640 1 --2- 172.21.15.159:0/1413987589 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdd5806c8f0 0x7fdd5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:57.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.463+0000 7fdd6ea00640 1 -- 172.21.15.159:0/1413987589 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5969+0+0 (secure 0 0 0) 0x7fdd6808bbf0 con 0x7fdd780af2e0 2024-04-04T06:06:57.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.464+0000 7fdd75e00640 1 --2- 172.21.15.159:0/1413987589 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdd5806c8f0 0x7fdd5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:57.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.465+0000 7fdd75e00640 1 --2- 172.21.15.159:0/1413987589 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdd5806c8f0 0x7fdd5806edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdd780730c0 tx=0x7fdd64008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:57.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.468+0000 7fdd6ea00640 1 -- 172.21.15.159:0/1413987589 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fdd68059e70 con 0x7fdd780af2e0 2024-04-04T06:06:57.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:57.689+0000 7fdd7d400640 1 -- 172.21.15.159:0/1413987589 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fdd7806eca0 con 0x7fdd780af2e0 2024-04-04T06:06:58.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.238+0000 7fdd6ea00640 1 -- 172.21.15.159:0/1413987589 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7fdd68059770 con 0x7fdd780af2e0 2024-04-04T06:06:58.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.242+0000 7fdd7d400640 1 -- 172.21.15.159:0/1413987589 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdd5806c8f0 msgr2=0x7fdd5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:58.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.242+0000 7fdd7d400640 1 --2- 172.21.15.159:0/1413987589 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdd5806c8f0 0x7fdd5806edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdd780730c0 tx=0x7fdd64008040 comp rx=0 tx=0).stop 2024-04-04T06:06:58.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.242+0000 7fdd7d400640 1 -- 172.21.15.159:0/1413987589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd780af2e0 msgr2=0x7fdd78156950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:58.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.242+0000 7fdd7d400640 1 --2- 172.21.15.159:0/1413987589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd780af2e0 0x7fdd78156950 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800c430 tx=0x7fdd6800c8f0 comp rx=0 tx=0).stop 2024-04-04T06:06:58.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.242+0000 7fdd7d400640 1 -- 172.21.15.159:0/1413987589 shutdown_connections 2024-04-04T06:06:58.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.242+0000 7fdd7d400640 1 --2- 172.21.15.159:0/1413987589 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdd5806c8f0 0x7fdd5806edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:58.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.242+0000 7fdd7d400640 1 --2- 172.21.15.159:0/1413987589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd78153580 0x7fdd78153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:58.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.242+0000 7fdd7d400640 1 --2- 172.21.15.159:0/1413987589 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdd780d0990 0x7fdd7815b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:58.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.242+0000 7fdd7d400640 1 --2- 172.21.15.159:0/1413987589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd780af2e0 0x7fdd78156950 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:58.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.242+0000 7fdd7d400640 1 -- 172.21.15.159:0/1413987589 >> 172.21.15.159:0/1413987589 conn(0x7fdd7806b750 msgr2=0x7fdd780d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:58.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.242+0000 7fdd7d400640 1 -- 172.21.15.159:0/1413987589 shutdown_connections 2024-04-04T06:06:58.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.242+0000 7fdd7d400640 1 -- 172.21.15.159:0/1413987589 wait complete. 2024-04-04T06:06:58.258 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-04T06:06:58.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.477+0000 7effcf800640 1 Processor -- start 2024-04-04T06:06:58.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.477+0000 7effcf800640 1 -- start start 2024-04-04T06:06:58.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.479+0000 7effcf800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7effc8162930 0x7effc80b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:58.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.479+0000 7effcf800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effc8072c50 0x7effc80b3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:58.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.479+0000 7effcf800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effc806ee70 con 0x7effc80b3ad0 2024-04-04T06:06:58.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.479+0000 7effcf800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7effc80717b0 con 0x7effc8162930 2024-04-04T06:06:58.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.479+0000 7effcf800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7effc8162d10 con 0x7effc8072c50 2024-04-04T06:06:58.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.479+0000 7effcce00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7effc8162930 0x7effc80b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:58.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.479+0000 7effcce00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7effc8162930 0x7effc80b2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43054/0 (socket says 172.21.15.159:43054) 2024-04-04T06:06:58.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.479+0000 7effcce00640 1 -- 172.21.15.159:0/1310621811 learned_addr learned my addr 172.21.15.159:0/1310621811 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:58.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.479+0000 7effc7e00640 1 --1- 172.21.15.159:0/1310621811 >> v1:172.21.15.159:6789/0 conn(0x7effc80b3ad0 0x7effc81664a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58588/0 (socket says 172.21.15.159:58588) 2024-04-04T06:06:58.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.480+0000 7effcd800640 1 --2- 172.21.15.159:0/1310621811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effc8072c50 0x7effc80b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:58.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.480+0000 7effcce00640 1 -- 172.21.15.159:0/1310621811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effc8072c50 msgr2=0x7effc80b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:58.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.480+0000 7effcce00640 1 --2- 172.21.15.159:0/1310621811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effc8072c50 0x7effc80b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:58.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.480+0000 7effcce00640 1 -- 172.21.15.159:0/1310621811 >> v1:172.21.15.159:6789/0 conn(0x7effc80b3ad0 legacy=0x7effc81664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:58.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.480+0000 7effc7400640 1 -- 172.21.15.159:0/1310621811 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2087722196 0 0) 0x7effc806ee70 con 0x7effc80b3ad0 2024-04-04T06:06:58.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.480+0000 7effcce00640 1 -- 172.21.15.159:0/1310621811 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc806e5d0 con 0x7effc8162930 2024-04-04T06:06:58.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.480+0000 7effcd800640 1 --2- 172.21.15.159:0/1310621811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effc8072c50 0x7effc80b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:58.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.481+0000 7effcce00640 1 --2- 172.21.15.159:0/1310621811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effc8162930 0x7effc80b2f30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7effbc009870 tx=0x7effbc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de99f8fc22eea350 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:58.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.481+0000 7effc7400640 1 -- 172.21.15.159:0/1310621811 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc034070 con 0x7effc8162930 2024-04-04T06:06:58.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.481+0000 7effc7400640 1 -- 172.21.15.159:0/1310621811 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effbc030840 con 0x7effc8162930 2024-04-04T06:06:58.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.481+0000 7effc7400640 1 -- 172.21.15.159:0/1310621811 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc030c10 con 0x7effc8162930 2024-04-04T06:06:58.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.482+0000 7effcf800640 1 -- 172.21.15.159:0/1310621811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effc8162930 msgr2=0x7effc80b2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:58.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.482+0000 7effcf800640 1 --2- 172.21.15.159:0/1310621811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effc8162930 0x7effc80b2f30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7effbc009870 tx=0x7effbc02b980 comp rx=0 tx=0).stop 2024-04-04T06:06:58.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.482+0000 7effcf800640 1 -- 172.21.15.159:0/1310621811 shutdown_connections 2024-04-04T06:06:58.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.482+0000 7effcf800640 1 --2- 172.21.15.159:0/1310621811 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effc8072c50 0x7effc80b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:58.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.482+0000 7effcf800640 1 --2- 172.21.15.159:0/1310621811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effc8162930 0x7effc80b2f30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:58.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.482+0000 7effcf800640 1 -- 172.21.15.159:0/1310621811 >> 172.21.15.159:0/1310621811 conn(0x7effc806bba0 msgr2=0x7effc80b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:58.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.482+0000 7effcf800640 1 -- 172.21.15.159:0/1310621811 shutdown_connections 2024-04-04T06:06:58.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.482+0000 7effcf800640 1 -- 172.21.15.159:0/1310621811 wait complete. 2024-04-04T06:06:58.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.484+0000 7effcf800640 1 Processor -- start 2024-04-04T06:06:58.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.484+0000 7effcf800640 1 -- start start 2024-04-04T06:06:58.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.485+0000 7effcf800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effc8072c50 0x7effc809aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:58.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.485+0000 7effcf800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effc80b3ad0 0x7effc80cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:58.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.485+0000 7effcf800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effc8162930 0x7effc8160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:58.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.485+0000 7effcf800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7effc806e1f0 con 0x7effc8072c50 2024-04-04T06:06:58.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.485+0000 7effcf800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7effc806eb90 con 0x7effc80b3ad0 2024-04-04T06:06:58.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.485+0000 7effcf800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7effc806e370 con 0x7effc8162930 2024-04-04T06:06:58.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.485+0000 7effcce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effc8072c50 0x7effc809aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:58.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.485+0000 7effcce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effc8072c50 0x7effc809aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43058/0 (socket says 172.21.15.159:43058) 2024-04-04T06:06:58.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.485+0000 7effcce00640 1 -- 172.21.15.159:0/3145707922 learned_addr learned my addr 172.21.15.159:0/3145707922 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:58.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.486+0000 7effcce00640 1 -- 172.21.15.159:0/3145707922 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effc8162930 msgr2=0x7effc8160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:58.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.486+0000 7effcce00640 1 --2- 172.21.15.159:0/3145707922 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effc8162930 0x7effc8160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:58.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.486+0000 7effc7e00640 1 --2- 172.21.15.159:0/3145707922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effc80b3ad0 0x7effc80cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:58.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.486+0000 7effcce00640 1 -- 172.21.15.159:0/3145707922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effc80b3ad0 msgr2=0x7effc80cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:58.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.486+0000 7effcce00640 1 --2- 172.21.15.159:0/3145707922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effc80b3ad0 0x7effc80cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:58.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.486+0000 7effcce00640 1 -- 172.21.15.159:0/3145707922 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc815fb40 con 0x7effc8072c50 2024-04-04T06:06:58.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.486+0000 7effc7e00640 1 --2- 172.21.15.159:0/3145707922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effc80b3ad0 0x7effc80cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:58.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.486+0000 7effcce00640 1 --2- 172.21.15.159:0/3145707922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effc8072c50 0x7effc809aa20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7effbc000c00 tx=0x7effbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:58.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.486+0000 7effc4c00640 1 -- 172.21.15.159:0/3145707922 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc034040 con 0x7effc8072c50 2024-04-04T06:06:58.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.487+0000 7effc4c00640 1 -- 172.21.15.159:0/3145707922 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effbc039d90 con 0x7effc8072c50 2024-04-04T06:06:58.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.487+0000 7effc4c00640 1 -- 172.21.15.159:0/3145707922 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc043610 con 0x7effc8072c50 2024-04-04T06:06:58.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.487+0000 7effcf800640 1 -- 172.21.15.159:0/3145707922 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc815fe20 con 0x7effc8072c50 2024-04-04T06:06:58.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.487+0000 7effcf800640 1 -- 172.21.15.159:0/3145707922 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc81603e0 con 0x7effc8072c50 2024-04-04T06:06:58.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.488+0000 7effcf800640 1 -- 172.21.15.159:0/3145707922 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc80685c0 con 0x7effc8072c50 2024-04-04T06:06:58.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.489+0000 7effc4c00640 1 -- 172.21.15.159:0/3145707922 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7effbc04a060 con 0x7effc8072c50 2024-04-04T06:06:58.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.490+0000 7effc4c00640 1 --2- 172.21.15.159:0/3145707922 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7effa806c8f0 0x7effa806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:58.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.490+0000 7effc4c00640 1 -- 172.21.15.159:0/3145707922 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5969+0+0 (secure 0 0 0) 0x7effbc0addf0 con 0x7effc8072c50 2024-04-04T06:06:58.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.490+0000 7effc7e00640 1 --2- 172.21.15.159:0/3145707922 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7effa806c8f0 0x7effa806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:58.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.491+0000 7effc7e00640 1 --2- 172.21.15.159:0/3145707922 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7effa806c8f0 0x7effa806edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7effc815f790 tx=0x7effb4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:58.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.498+0000 7effc4c00640 1 -- 172.21.15.159:0/3145707922 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7effbc03e070 con 0x7effc8072c50 2024-04-04T06:06:58.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:58.702+0000 7effcf800640 1 -- 172.21.15.159:0/3145707922 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7effc815c7d0 con 0x7effc8072c50 2024-04-04T06:06:59.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.248+0000 7effc4c00640 1 -- 172.21.15.159:0/3145707922 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) ==== 181+0+0 (secure 0 0 0) 0x7effbc02d330 con 0x7effc8072c50 2024-04-04T06:06:59.244 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_metadata' removed 2024-04-04T06:06:59.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.254+0000 7effcf800640 1 -- 172.21.15.159:0/3145707922 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7effa806c8f0 msgr2=0x7effa806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:59.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.254+0000 7effcf800640 1 --2- 172.21.15.159:0/3145707922 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7effa806c8f0 0x7effa806edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7effc815f790 tx=0x7effb4009040 comp rx=0 tx=0).stop 2024-04-04T06:06:59.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.254+0000 7effcf800640 1 -- 172.21.15.159:0/3145707922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effc8072c50 msgr2=0x7effc809aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:59.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.254+0000 7effcf800640 1 --2- 172.21.15.159:0/3145707922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effc8072c50 0x7effc809aa20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7effbc000c00 tx=0x7effbc0046c0 comp rx=0 tx=0).stop 2024-04-04T06:06:59.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.255+0000 7effcf800640 1 -- 172.21.15.159:0/3145707922 shutdown_connections 2024-04-04T06:06:59.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.255+0000 7effcf800640 1 --2- 172.21.15.159:0/3145707922 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effc8162930 0x7effc8160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:59.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.255+0000 7effcf800640 1 --2- 172.21.15.159:0/3145707922 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7effa806c8f0 0x7effa806edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:59.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.255+0000 7effcf800640 1 --2- 172.21.15.159:0/3145707922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effc80b3ad0 0x7effc80cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:59.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.255+0000 7effcf800640 1 --2- 172.21.15.159:0/3145707922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effc8072c50 0x7effc809aa20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:59.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.255+0000 7effcf800640 1 -- 172.21.15.159:0/3145707922 >> 172.21.15.159:0/3145707922 conn(0x7effc806bba0 msgr2=0x7effc815bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:59.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.255+0000 7effcf800640 1 -- 172.21.15.159:0/3145707922 shutdown_connections 2024-04-04T06:06:59.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.255+0000 7effcf800640 1 -- 172.21.15.159:0/3145707922 wait complete. 2024-04-04T06:06:59.271 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-04T06:06:59.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.482+0000 7ff875a00640 1 Processor -- start 2024-04-04T06:06:59.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.482+0000 7ff875a00640 1 -- start start 2024-04-04T06:06:59.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.483+0000 7ff875a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff870072720 0x7ff8700b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:59.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.484+0000 7ff875a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff8700aefc0 0x7ff87006dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:59.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.484+0000 7ff875a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff87006ea10 con 0x7ff870072b00 2024-04-04T06:06:59.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.484+0000 7ff875a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7ff8700b3e80 con 0x7ff8700aefc0 2024-04-04T06:06:59.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.484+0000 7ff875a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff87006ecf0 con 0x7ff870072720 2024-04-04T06:06:59.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.484+0000 7ff86f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff8700aefc0 0x7ff87006dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:59.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.484+0000 7ff86f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff8700aefc0 0x7ff87006dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43072/0 (socket says 172.21.15.159:43072) 2024-04-04T06:06:59.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.484+0000 7ff86f400640 1 -- 172.21.15.159:0/3942854040 learned_addr learned my addr 172.21.15.159:0/3942854040 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:59.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.484+0000 7ff86ea00640 1 --2- 172.21.15.159:0/3942854040 >> v2:172.21.15.178:3300/0 conn(0x7ff870072720 0x7ff8700b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:59.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.484+0000 7ff86f400640 1 -- 172.21.15.159:0/3942854040 >> v2:172.21.15.178:3300/0 conn(0x7ff870072720 msgr2=0x7ff8700b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:59.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.484+0000 7ff86f400640 1 --2- 172.21.15.159:0/3942854040 >> v2:172.21.15.178:3300/0 conn(0x7ff870072720 0x7ff8700b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:59.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.484+0000 7ff86f400640 1 -- 172.21.15.159:0/3942854040 >> v1:172.21.15.178:6789/0 conn(0x7ff870072b00 legacy=0x7ff87006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:06:59.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.485+0000 7ff86f400640 1 -- 172.21.15.159:0/3942854040 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87006efd0 con 0x7ff8700aefc0 2024-04-04T06:06:59.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.486+0000 7ff86f400640 1 --2- 172.21.15.159:0/3942854040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8700aefc0 0x7ff87006dce0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff870073390 tx=0x7ff86802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc37deccff2c1c83 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:59.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.486+0000 7ff86e000640 1 -- 172.21.15.159:0/3942854040 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86802ca80 con 0x7ff8700aefc0 2024-04-04T06:06:59.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.486+0000 7ff86e000640 1 -- 172.21.15.159:0/3942854040 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff86802cc20 con 0x7ff8700aefc0 2024-04-04T06:06:59.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.486+0000 7ff86e000640 1 -- 172.21.15.159:0/3942854040 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8680343f0 con 0x7ff8700aefc0 2024-04-04T06:06:59.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.487+0000 7ff875a00640 1 -- 172.21.15.159:0/3942854040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8700aefc0 msgr2=0x7ff87006dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:59.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.487+0000 7ff875a00640 1 --2- 172.21.15.159:0/3942854040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8700aefc0 0x7ff87006dce0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff870073390 tx=0x7ff86802bee0 comp rx=0 tx=0).stop 2024-04-04T06:06:59.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.487+0000 7ff875a00640 1 -- 172.21.15.159:0/3942854040 shutdown_connections 2024-04-04T06:06:59.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.487+0000 7ff875a00640 1 --2- 172.21.15.159:0/3942854040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8700aefc0 0x7ff87006dce0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:59.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.487+0000 7ff875a00640 1 --2- 172.21.15.159:0/3942854040 >> v2:172.21.15.178:3300/0 conn(0x7ff870072720 0x7ff8700b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:59.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.487+0000 7ff875a00640 1 -- 172.21.15.159:0/3942854040 >> 172.21.15.159:0/3942854040 conn(0x7ff87006bba0 msgr2=0x7ff8700b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:06:59.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.487+0000 7ff875a00640 1 -- 172.21.15.159:0/3942854040 shutdown_connections 2024-04-04T06:06:59.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.487+0000 7ff875a00640 1 -- 172.21.15.159:0/3942854040 wait complete. 2024-04-04T06:06:59.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.489+0000 7ff875a00640 1 Processor -- start 2024-04-04T06:06:59.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.489+0000 7ff875a00640 1 -- start start 2024-04-04T06:06:59.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.490+0000 7ff875a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff870072720 0x7ff8700ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:59.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.490+0000 7ff875a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff870072b00 0x7ff87009c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:59.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.490+0000 7ff875a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8700aefc0 0x7ff8701491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:59.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.490+0000 7ff875a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff87006e690 con 0x7ff870072720 2024-04-04T06:06:59.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.490+0000 7ff875a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff87006f590 con 0x7ff8700aefc0 2024-04-04T06:06:59.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.490+0000 7ff875a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff870072f70 con 0x7ff870072b00 2024-04-04T06:06:59.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.491+0000 7ff86ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff870072720 0x7ff8700ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:59.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.491+0000 7ff86ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff870072720 0x7ff8700ae200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43088/0 (socket says 172.21.15.159:43088) 2024-04-04T06:06:59.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.491+0000 7ff86ea00640 1 -- 172.21.15.159:0/2137546085 learned_addr learned my addr 172.21.15.159:0/2137546085 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:06:59.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.491+0000 7ff86f400640 1 --2- 172.21.15.159:0/2137546085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8700aefc0 0x7ff8701491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:59.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.491+0000 7ff866000640 1 --2- 172.21.15.159:0/2137546085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff870072b00 0x7ff87009c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:59.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.491+0000 7ff86ea00640 1 -- 172.21.15.159:0/2137546085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff870072b00 msgr2=0x7ff87009c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:59.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.491+0000 7ff86ea00640 1 --2- 172.21.15.159:0/2137546085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff870072b00 0x7ff87009c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:59.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.491+0000 7ff86ea00640 1 -- 172.21.15.159:0/2137546085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8700aefc0 msgr2=0x7ff8701491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:06:59.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.491+0000 7ff86ea00640 1 --2- 172.21.15.159:0/2137546085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8700aefc0 0x7ff8701491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:06:59.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.491+0000 7ff86ea00640 1 -- 172.21.15.159:0/2137546085 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8700b02e0 con 0x7ff870072720 2024-04-04T06:06:59.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.491+0000 7ff86f400640 1 --2- 172.21.15.159:0/2137546085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8700aefc0 0x7ff8701491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:59.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.491+0000 7ff866000640 1 --2- 172.21.15.159:0/2137546085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff870072b00 0x7ff87009c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:06:59.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.492+0000 7ff86ea00640 1 --2- 172.21.15.159:0/2137546085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff870072720 0x7ff8700ae200 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff86000c7f0 tx=0x7ff86000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:59.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.492+0000 7ff867400640 1 -- 172.21.15.159:0/2137546085 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860015070 con 0x7ff870072720 2024-04-04T06:06:59.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.492+0000 7ff867400640 1 -- 172.21.15.159:0/2137546085 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff860004240 con 0x7ff870072720 2024-04-04T06:06:59.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.492+0000 7ff867400640 1 -- 172.21.15.159:0/2137546085 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860007620 con 0x7ff870072720 2024-04-04T06:06:59.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.492+0000 7ff875a00640 1 -- 172.21.15.159:0/2137546085 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8700b05c0 con 0x7ff870072720 2024-04-04T06:06:59.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.492+0000 7ff875a00640 1 -- 172.21.15.159:0/2137546085 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8700b0b80 con 0x7ff870072720 2024-04-04T06:06:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.493+0000 7ff875a00640 1 -- 172.21.15.159:0/2137546085 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8700685c0 con 0x7ff870072720 2024-04-04T06:06:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.494+0000 7ff867400640 1 -- 172.21.15.159:0/2137546085 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8600079d0 con 0x7ff870072720 2024-04-04T06:06:59.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.495+0000 7ff867400640 1 --2- 172.21.15.159:0/2137546085 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff84c06c8f0 0x7ff84c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:06:59.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.496+0000 7ff867400640 1 -- 172.21.15.159:0/2137546085 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5531+0+0 (secure 0 0 0) 0x7ff86008bf40 con 0x7ff870072720 2024-04-04T06:06:59.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.496+0000 7ff866000640 1 --2- 172.21.15.159:0/2137546085 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff84c06c8f0 0x7ff84c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:06:59.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.497+0000 7ff866000640 1 --2- 172.21.15.159:0/2137546085 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff84c06c8f0 0x7ff84c06edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff870151060 tx=0x7ff858009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:06:59.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.501+0000 7ff867400640 1 -- 172.21.15.159:0/2137546085 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff86005a410 con 0x7ff870072720 2024-04-04T06:06:59.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:06:59.701+0000 7ff875a00640 1 -- 172.21.15.159:0/2137546085 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff87006f930 con 0x7ff870072720 2024-04-04T06:07:00.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.260+0000 7ff867400640 1 -- 172.21.15.159:0/2137546085 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) ==== 169+0+0 (secure 0 0 0) 0x7ff860059d10 con 0x7ff870072720 2024-04-04T06:07:00.255 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_data' removed 2024-04-04T06:07:00.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.266+0000 7ff875a00640 1 -- 172.21.15.159:0/2137546085 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff84c06c8f0 msgr2=0x7ff84c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:00.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.266+0000 7ff875a00640 1 --2- 172.21.15.159:0/2137546085 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff84c06c8f0 0x7ff84c06edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff870151060 tx=0x7ff858009040 comp rx=0 tx=0).stop 2024-04-04T06:07:00.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.266+0000 7ff875a00640 1 -- 172.21.15.159:0/2137546085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff870072720 msgr2=0x7ff8700ae200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:00.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.266+0000 7ff875a00640 1 --2- 172.21.15.159:0/2137546085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff870072720 0x7ff8700ae200 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff86000c7f0 tx=0x7ff86000ccb0 comp rx=0 tx=0).stop 2024-04-04T06:07:00.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.266+0000 7ff875a00640 1 -- 172.21.15.159:0/2137546085 shutdown_connections 2024-04-04T06:07:00.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.266+0000 7ff875a00640 1 --2- 172.21.15.159:0/2137546085 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff84c06c8f0 0x7ff84c06edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:00.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.266+0000 7ff875a00640 1 --2- 172.21.15.159:0/2137546085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8700aefc0 0x7ff8701491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:00.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.266+0000 7ff875a00640 1 --2- 172.21.15.159:0/2137546085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff870072b00 0x7ff87009c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:00.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.266+0000 7ff875a00640 1 --2- 172.21.15.159:0/2137546085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff870072720 0x7ff8700ae200 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:00.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.266+0000 7ff875a00640 1 -- 172.21.15.159:0/2137546085 >> 172.21.15.159:0/2137546085 conn(0x7ff87006bba0 msgr2=0x7ff8700b2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:00.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.267+0000 7ff875a00640 1 -- 172.21.15.159:0/2137546085 shutdown_connections 2024-04-04T06:07:00.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.267+0000 7ff875a00640 1 -- 172.21.15.159:0/2137546085 wait complete. 2024-04-04T06:07:00.285 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-04T06:07:00.285 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-04T06:07:00.285 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-04T06:07:00.286 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:07:00.286 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-04T06:07:00.286 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-04T06:07:00.286 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-04T06:07:00.286 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:07:00.286 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-04T06:07:00.287 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-04T06:07:00.287 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-04T06:07:00.287 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:07:00.287 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-04T06:07:00.287 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-04T06:07:00.287 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-04T06:07:00.287 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:07:00.303 INFO:tasks.ceph.mds.a:Stopped 2024-04-04T06:07:00.303 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-04T06:07:00.306 INFO:tasks.ceph.mds.c:Stopped 2024-04-04T06:07:00.306 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-04T06:07:00.307 INFO:tasks.ceph.mds.b:Stopped 2024-04-04T06:07:00.307 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-04T06:07:00.308 INFO:tasks.ceph.mds.d:Stopped 2024-04-04T06:07:00.308 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-04T06:07:00.353 INFO:tasks.ceph.mds.a:Started 2024-04-04T06:07:00.353 DEBUG:teuthology.parallel:result is None 2024-04-04T06:07:00.355 INFO:tasks.ceph.mds.b:Started 2024-04-04T06:07:00.355 DEBUG:teuthology.parallel:result is None 2024-04-04T06:07:00.359 INFO:tasks.ceph.mds.c:Started 2024-04-04T06:07:00.359 DEBUG:teuthology.parallel:result is None 2024-04-04T06:07:00.360 INFO:tasks.ceph.mds.d:Started 2024-04-04T06:07:00.360 DEBUG:teuthology.parallel:result is None 2024-04-04T06:07:00.361 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-04T06:07:00.508 INFO:tasks.ceph.mds.a.smithi159.stdout:starting mds.a at 2024-04-04T06:07:00.512 INFO:tasks.ceph.mds.c.smithi159.stdout:starting mds.c at 2024-04-04T06:07:00.512 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-04T06:07:00.513 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-04T06:07:00.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.557+0000 7fcd63200640 1 Processor -- start 2024-04-04T06:07:00.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.557+0000 7fcd63200640 1 -- start start 2024-04-04T06:07:00.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.559+0000 7fcd63200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd5c141c10 0x7fcd5c071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:00.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.559+0000 7fcd63200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd5c15ec80 con 0x7fcd5c0d0280 2024-04-04T06:07:00.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.559+0000 7fcd63200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd5c15ef20 con 0x7fcd5c071eb0 2024-04-04T06:07:00.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.559+0000 7fcd63200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcd5c15f200 con 0x7fcd5c141c10 2024-04-04T06:07:00.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.559+0000 7fcd5be00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fcd5c0d0280 0x7fcd5c0d0660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58608/0 (socket says 172.21.15.159:58608) 2024-04-04T06:07:00.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.559+0000 7fcd5be00640 1 -- 172.21.15.159:0/3397198564 learned_addr learned my addr 172.21.15.159:0/3397198564 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:00.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.559+0000 7fcd5b400640 1 --2- 172.21.15.159:0/3397198564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd5c141c10 0x7fcd5c071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:00.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.560+0000 7fcd5aa00640 1 -- 172.21.15.159:0/3397198564 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1434753398 0 0) 0x7fcd5c15ec80 con 0x7fcd5c0d0280 2024-04-04T06:07:00.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.560+0000 7fcd5aa00640 1 -- 172.21.15.159:0/3397198564 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd40003520 con 0x7fcd5c0d0280 2024-04-04T06:07:00.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.560+0000 7fcd5aa00640 1 -- 172.21.15.159:0/3397198564 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1231088320 0 0) 0x7fcd40003520 con 0x7fcd5c0d0280 2024-04-04T06:07:00.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.560+0000 7fcd5aa00640 1 -- 172.21.15.159:0/3397198564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd5c141c10 msgr2=0x7fcd5c071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:00.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.560+0000 7fcd5aa00640 1 --2- 172.21.15.159:0/3397198564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd5c141c10 0x7fcd5c071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:00.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.560+0000 7fcd5aa00640 1 -- 172.21.15.159:0/3397198564 >> v1:172.21.15.178:6789/0 conn(0x7fcd5c071eb0 legacy=0x7fcd5c162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:00.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.560+0000 7fcd5aa00640 1 -- 172.21.15.159:0/3397198564 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5c15f4e0 con 0x7fcd5c0d0280 2024-04-04T06:07:00.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.560+0000 7fcd5aa00640 1 -- 172.21.15.159:0/3397198564 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fcd4c001670 con 0x7fcd5c0d0280 2024-04-04T06:07:00.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.560+0000 7fcd5aa00640 1 -- 172.21.15.159:0/3397198564 >> v1:172.21.15.159:6789/0 conn(0x7fcd5c0d0280 legacy=0x7fcd5c0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:00.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.560+0000 7fcd5aa00640 1 --2- 172.21.15.159:0/3397198564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd5c141c10 0x7fcd400039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:00.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.560+0000 7fcd5aa00640 1 --2- 172.21.15.159:0/3397198564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd40003f20 0x7fcd40004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:00.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.560+0000 7fcd5aa00640 1 --2- 172.21.15.159:0/3397198564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd40004d40 0x7fcd400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:00.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.560+0000 7fcd5aa00640 1 -- 172.21.15.159:0/3397198564 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c15f200 con 0x7fcd40004d40 2024-04-04T06:07:00.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.560+0000 7fcd5aa00640 1 -- 172.21.15.159:0/3397198564 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4000ba20 con 0x7fcd40003f20 2024-04-04T06:07:00.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.560+0000 7fcd5aa00640 1 -- 172.21.15.159:0/3397198564 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcd4000bc20 con 0x7fcd5c141c10 2024-04-04T06:07:00.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.560+0000 7fcd5be00640 1 --2- 172.21.15.159:0/3397198564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd5c141c10 0x7fcd400039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:00.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.560+0000 7fcd5b400640 1 --2- 172.21.15.159:0/3397198564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd40003f20 0x7fcd40004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:00.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.561+0000 7fcd61200640 1 --2- 172.21.15.159:0/3397198564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd40004d40 0x7fcd400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:00.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.561+0000 7fcd5b400640 1 -- 172.21.15.159:0/3397198564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd5c141c10 msgr2=0x7fcd400039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:00.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.561+0000 7fcd5b400640 1 --2- 172.21.15.159:0/3397198564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd5c141c10 0x7fcd400039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:00.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.561+0000 7fcd5b400640 1 -- 172.21.15.159:0/3397198564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd40004d40 msgr2=0x7fcd400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:00.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.561+0000 7fcd5b400640 1 --2- 172.21.15.159:0/3397198564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd40004d40 0x7fcd400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:00.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.561+0000 7fcd5b400640 1 -- 172.21.15.159:0/3397198564 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd40003520 con 0x7fcd40003f20 2024-04-04T06:07:00.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.562+0000 7fcd5b400640 1 --2- 172.21.15.159:0/3397198564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd40003f20 0x7fcd40004800 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c148260 tx=0x7fcd40005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=affdad312ef6e4e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:00.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.562+0000 7fcd5aa00640 1 -- 172.21.15.159:0/3397198564 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48039070 con 0x7fcd40003f20 2024-04-04T06:07:00.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.562+0000 7fcd5aa00640 1 -- 172.21.15.159:0/3397198564 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd4802c580 con 0x7fcd40003f20 2024-04-04T06:07:00.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.562+0000 7fcd5aa00640 1 -- 172.21.15.159:0/3397198564 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48009690 con 0x7fcd40003f20 2024-04-04T06:07:00.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.563+0000 7fcd63200640 1 -- 172.21.15.159:0/3397198564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd40003f20 msgr2=0x7fcd40004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:00.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.563+0000 7fcd63200640 1 --2- 172.21.15.159:0/3397198564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd40003f20 0x7fcd40004800 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c148260 tx=0x7fcd40005830 comp rx=0 tx=0).stop 2024-04-04T06:07:00.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.563+0000 7fcd61200640 1 -- 172.21.15.159:0/3397198564 reap_dead start 2024-04-04T06:07:00.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.563+0000 7fcd63200640 1 -- 172.21.15.159:0/3397198564 shutdown_connections 2024-04-04T06:07:00.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.563+0000 7fcd63200640 1 -- 172.21.15.159:0/3397198564 >> 172.21.15.159:0/3397198564 conn(0x7fcd5c06b750 msgr2=0x7fcd5c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:00.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.563+0000 7fcd63200640 1 -- 172.21.15.159:0/3397198564 shutdown_connections 2024-04-04T06:07:00.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.563+0000 7fcd63200640 1 -- 172.21.15.159:0/3397198564 wait complete. 2024-04-04T06:07:00.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.565+0000 7fcd63200640 1 Processor -- start 2024-04-04T06:07:00.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.565+0000 7fcd63200640 1 -- start start 2024-04-04T06:07:00.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.566+0000 7fcd63200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd40003f20 0x7fcd5c0af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:00.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.566+0000 7fcd63200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd5c157040 0x7fcd5c098e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:00.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.566+0000 7fcd63200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd5c09d1f0 0x7fcd5c09d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:00.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.566+0000 7fcd63200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c158e90 con 0x7fcd40003f20 2024-04-04T06:07:00.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.566+0000 7fcd63200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c06f340 con 0x7fcd5c09d1f0 2024-04-04T06:07:00.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.566+0000 7fcd63200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcd5c1578b0 con 0x7fcd5c157040 2024-04-04T06:07:00.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.566+0000 7fcd5be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd40003f20 0x7fcd5c0af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:00.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.566+0000 7fcd5be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd40003f20 0x7fcd5c0af1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43132/0 (socket says 172.21.15.159:43132) 2024-04-04T06:07:00.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.566+0000 7fcd5be00640 1 -- 172.21.15.159:0/3198253677 learned_addr learned my addr 172.21.15.159:0/3198253677 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:00.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.566+0000 7fcd5b400640 1 --2- 172.21.15.159:0/3198253677 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd5c157040 0x7fcd5c098e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:00.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.566+0000 7fcd61200640 1 --2- 172.21.15.159:0/3198253677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd5c09d1f0 0x7fcd5c09d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:00.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.567+0000 7fcd5be00640 1 -- 172.21.15.159:0/3198253677 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd5c157040 msgr2=0x7fcd5c098e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:00.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.567+0000 7fcd5be00640 1 --2- 172.21.15.159:0/3198253677 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd5c157040 0x7fcd5c098e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:00.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.567+0000 7fcd5be00640 1 -- 172.21.15.159:0/3198253677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd5c09d1f0 msgr2=0x7fcd5c09d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:00.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.567+0000 7fcd5be00640 1 --2- 172.21.15.159:0/3198253677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd5c09d1f0 0x7fcd5c09d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:00.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.567+0000 7fcd5be00640 1 -- 172.21.15.159:0/3198253677 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5c0af960 con 0x7fcd40003f20 2024-04-04T06:07:00.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.567+0000 7fcd5be00640 1 --2- 172.21.15.159:0/3198253677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd40003f20 0x7fcd5c0af1c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c007290 tx=0x7fcd4c007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:00.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.567+0000 7fcd53e00640 1 -- 172.21.15.159:0/3198253677 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c01a070 con 0x7fcd40003f20 2024-04-04T06:07:00.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.567+0000 7fcd53e00640 1 -- 172.21.15.159:0/3198253677 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd4c00fa20 con 0x7fcd40003f20 2024-04-04T06:07:00.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.567+0000 7fcd53e00640 1 -- 172.21.15.159:0/3198253677 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c00fe10 con 0x7fcd40003f20 2024-04-04T06:07:00.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.567+0000 7fcd63200640 1 -- 172.21.15.159:0/3198253677 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5c0afc40 con 0x7fcd40003f20 2024-04-04T06:07:00.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.567+0000 7fcd63200640 1 -- 172.21.15.159:0/3198253677 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd5c0b0200 con 0x7fcd40003f20 2024-04-04T06:07:00.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.568+0000 7fcd63200640 1 -- 172.21.15.159:0/3198253677 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd5c09dbb0 con 0x7fcd40003f20 2024-04-04T06:07:00.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.570+0000 7fcd53e00640 1 -- 172.21.15.159:0/3198253677 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd4c01d070 con 0x7fcd40003f20 2024-04-04T06:07:00.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.570+0000 7fcd53e00640 1 --2- 172.21.15.159:0/3198253677 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcd3c06c940 0x7fcd3c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:00.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.570+0000 7fcd53e00640 1 -- 172.21.15.159:0/3198253677 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4943+0+0 (secure 0 0 0) 0x7fcd4c08b500 con 0x7fcd40003f20 2024-04-04T06:07:00.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.571+0000 7fcd5b400640 1 --2- 172.21.15.159:0/3198253677 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcd3c06c940 0x7fcd3c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:00.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.571+0000 7fcd5b400640 1 --2- 172.21.15.159:0/3198253677 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcd3c06c940 0x7fcd3c06ee00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcd48036000 tx=0x7fcd48030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:00.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.575+0000 7fcd53e00640 1 -- 172.21.15.159:0/3198253677 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcd4c0595c0 con 0x7fcd40003f20 2024-04-04T06:07:00.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:00.791+0000 7fcd63200640 1 -- 172.21.15.159:0/3198253677 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fcd5c0685c0 con 0x7fcd40003f20 2024-04-04T06:07:01.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.284+0000 7fcd53e00640 1 -- 172.21.15.159:0/3198253677 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) ==== 97+0+0 (secure 0 0 0) 0x7fcd4c089e20 con 0x7fcd40003f20 2024-04-04T06:07:01.279 INFO:teuthology.orchestra.run.smithi159.stderr: removed all blocklist entries 2024-04-04T06:07:01.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.288+0000 7fcd63200640 1 -- 172.21.15.159:0/3198253677 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcd3c06c940 msgr2=0x7fcd3c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:01.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.288+0000 7fcd63200640 1 --2- 172.21.15.159:0/3198253677 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcd3c06c940 0x7fcd3c06ee00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcd48036000 tx=0x7fcd48030000 comp rx=0 tx=0).stop 2024-04-04T06:07:01.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.289+0000 7fcd63200640 1 -- 172.21.15.159:0/3198253677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd40003f20 msgr2=0x7fcd5c0af1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:01.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.289+0000 7fcd63200640 1 --2- 172.21.15.159:0/3198253677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd40003f20 0x7fcd5c0af1c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c007290 tx=0x7fcd4c007690 comp rx=0 tx=0).stop 2024-04-04T06:07:01.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.289+0000 7fcd63200640 1 -- 172.21.15.159:0/3198253677 shutdown_connections 2024-04-04T06:07:01.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.289+0000 7fcd63200640 1 --2- 172.21.15.159:0/3198253677 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcd3c06c940 0x7fcd3c06ee00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:01.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.289+0000 7fcd63200640 1 --2- 172.21.15.159:0/3198253677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd5c09d1f0 0x7fcd5c09d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:01.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.289+0000 7fcd63200640 1 --2- 172.21.15.159:0/3198253677 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd5c157040 0x7fcd5c098e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:01.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.289+0000 7fcd63200640 1 --2- 172.21.15.159:0/3198253677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd40003f20 0x7fcd5c0af1c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:01.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.289+0000 7fcd63200640 1 -- 172.21.15.159:0/3198253677 >> 172.21.15.159:0/3198253677 conn(0x7fcd5c06b750 msgr2=0x7fcd5c0b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:01.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.289+0000 7fcd63200640 1 -- 172.21.15.159:0/3198253677 shutdown_connections 2024-04-04T06:07:01.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.289+0000 7fcd63200640 1 -- 172.21.15.159:0/3198253677 wait complete. 2024-04-04T06:07:01.314 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-04T06:07:01.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.506+0000 7f2a49200640 1 Processor -- start 2024-04-04T06:07:01.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.506+0000 7f2a49200640 1 -- start start 2024-04-04T06:07:01.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.507+0000 7f2a49200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a44153580 0x7f2a440d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:01.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.508+0000 7f2a49200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2a440af4b0 0x7f2a441600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:01.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.508+0000 7f2a49200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a440afab0 con 0x7f2a440d07c0 2024-04-04T06:07:01.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.508+0000 7f2a49200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f2a440d0c00 con 0x7f2a440af4b0 2024-04-04T06:07:01.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.508+0000 7f2a49200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2a4415cc80 con 0x7f2a44153580 2024-04-04T06:07:01.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.508+0000 7f2a42800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2a440af4b0 0x7f2a441600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:01.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.508+0000 7f2a42800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2a440af4b0 0x7f2a441600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43140/0 (socket says 172.21.15.159:43140) 2024-04-04T06:07:01.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.508+0000 7f2a42800640 1 -- 172.21.15.159:0/623976752 learned_addr learned my addr 172.21.15.159:0/623976752 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:01.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.508+0000 7f2a42800640 1 -- 172.21.15.159:0/623976752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a44153580 msgr2=0x7f2a440d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:01.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.508+0000 7f2a42800640 1 --2- 172.21.15.159:0/623976752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a44153580 0x7f2a440d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:01.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.508+0000 7f2a42800640 1 -- 172.21.15.159:0/623976752 >> v1:172.21.15.178:6789/0 conn(0x7f2a440d07c0 legacy=0x7f2a440aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:01.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.508+0000 7f2a42800640 1 -- 172.21.15.159:0/623976752 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4415da00 con 0x7f2a440af4b0 2024-04-04T06:07:01.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.509+0000 7f2a42800640 1 --2- 172.21.15.159:0/623976752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a440af4b0 0x7f2a441600d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009e50 tx=0x7f2a3c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c539be490f13b6e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:01.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.510+0000 7f2a40a00640 1 -- 172.21.15.159:0/623976752 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c02c9e0 con 0x7f2a440af4b0 2024-04-04T06:07:01.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.510+0000 7f2a40a00640 1 -- 172.21.15.159:0/623976752 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a3c02cb80 con 0x7f2a440af4b0 2024-04-04T06:07:01.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.510+0000 7f2a40a00640 1 -- 172.21.15.159:0/623976752 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c0343f0 con 0x7f2a440af4b0 2024-04-04T06:07:01.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.510+0000 7f2a49200640 1 -- 172.21.15.159:0/623976752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a440af4b0 msgr2=0x7f2a441600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:01.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.510+0000 7f2a49200640 1 --2- 172.21.15.159:0/623976752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a440af4b0 0x7f2a441600d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009e50 tx=0x7f2a3c02be40 comp rx=0 tx=0).stop 2024-04-04T06:07:01.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.510+0000 7f2a49200640 1 -- 172.21.15.159:0/623976752 shutdown_connections 2024-04-04T06:07:01.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.510+0000 7f2a49200640 1 --2- 172.21.15.159:0/623976752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a440af4b0 0x7f2a441600d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:01.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.510+0000 7f2a49200640 1 --2- 172.21.15.159:0/623976752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a44153580 0x7f2a440d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:01.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.510+0000 7f2a49200640 1 -- 172.21.15.159:0/623976752 >> 172.21.15.159:0/623976752 conn(0x7f2a4406b750 msgr2=0x7f2a4406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:01.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.510+0000 7f2a49200640 1 -- 172.21.15.159:0/623976752 shutdown_connections 2024-04-04T06:07:01.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.510+0000 7f2a49200640 1 -- 172.21.15.159:0/623976752 wait complete. 2024-04-04T06:07:01.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.512+0000 7f2a49200640 1 Processor -- start 2024-04-04T06:07:01.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.512+0000 7f2a49200640 1 -- start start 2024-04-04T06:07:01.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.513+0000 7f2a49200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a440af4b0 0x7f2a441563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:01.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.513+0000 7f2a49200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a440d07c0 0x7f2a440cf190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:01.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.513+0000 7f2a49200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a44153580 0x7f2a44142f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:01.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.513+0000 7f2a49200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4415d4e0 con 0x7f2a440af4b0 2024-04-04T06:07:01.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.513+0000 7f2a49200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4415d200 con 0x7f2a44153580 2024-04-04T06:07:01.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.513+0000 7f2a49200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2a44153960 con 0x7f2a440d07c0 2024-04-04T06:07:01.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.513+0000 7f2a41e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a440af4b0 0x7f2a441563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:01.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.513+0000 7f2a41e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a440af4b0 0x7f2a441563a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43142/0 (socket says 172.21.15.159:43142) 2024-04-04T06:07:01.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.513+0000 7f2a41e00640 1 -- 172.21.15.159:0/485244769 learned_addr learned my addr 172.21.15.159:0/485244769 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:01.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.514+0000 7f2a41e00640 1 -- 172.21.15.159:0/485244769 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a440d07c0 msgr2=0x7f2a440cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:01.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.514+0000 7f2a41e00640 1 --2- 172.21.15.159:0/485244769 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a440d07c0 0x7f2a440cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:01.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.514+0000 7f2a41e00640 1 -- 172.21.15.159:0/485244769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a44153580 msgr2=0x7f2a44142f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:01.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.514+0000 7f2a41e00640 1 --2- 172.21.15.159:0/485244769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a44153580 0x7f2a44142f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:01.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.514+0000 7f2a41e00640 1 -- 172.21.15.159:0/485244769 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a440cde70 con 0x7f2a440af4b0 2024-04-04T06:07:01.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.514+0000 7f2a41e00640 1 --2- 172.21.15.159:0/485244769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a440af4b0 0x7f2a441563a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400b4a0 tx=0x7f2a3400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:01.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.514+0000 7f2a3a000640 1 -- 172.21.15.159:0/485244769 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a34014070 con 0x7f2a440af4b0 2024-04-04T06:07:01.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.514+0000 7f2a3a000640 1 -- 172.21.15.159:0/485244769 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a34010610 con 0x7f2a440af4b0 2024-04-04T06:07:01.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.514+0000 7f2a3a000640 1 -- 172.21.15.159:0/485244769 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a34010a00 con 0x7f2a440af4b0 2024-04-04T06:07:01.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.514+0000 7f2a49200640 1 -- 172.21.15.159:0/485244769 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a440ce3d0 con 0x7f2a440af4b0 2024-04-04T06:07:01.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.514+0000 7f2a49200640 1 -- 172.21.15.159:0/485244769 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a440ce990 con 0x7f2a440af4b0 2024-04-04T06:07:01.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.515+0000 7f2a49200640 1 -- 172.21.15.159:0/485244769 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a4406ea00 con 0x7f2a440af4b0 2024-04-04T06:07:01.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.516+0000 7f2a3a000640 1 -- 172.21.15.159:0/485244769 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a34017070 con 0x7f2a440af4b0 2024-04-04T06:07:01.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.517+0000 7f2a3a000640 1 --2- 172.21.15.159:0/485244769 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2a2406c8f0 0x7f2a2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:01.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.517+0000 7f2a3a000640 1 -- 172.21.15.159:0/485244769 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f2a3408b2f0 con 0x7f2a440af4b0 2024-04-04T06:07:01.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.518+0000 7f2a41400640 1 --2- 172.21.15.159:0/485244769 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2a2406c8f0 0x7f2a2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:01.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.519+0000 7f2a41400640 1 --2- 172.21.15.159:0/485244769 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2a2406c8f0 0x7f2a2406edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2a44142e40 tx=0x7f2a30007990 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:01.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.522+0000 7f2a3a000640 1 -- 172.21.15.159:0/485244769 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2a4406ea00 con 0x7f2a440af4b0 2024-04-04T06:07:01.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.794+0000 7f2a49200640 1 -- 172.21.15.159:0/485244769 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f2a440685c0 con 0x7f2a440af4b0 2024-04-04T06:07:01.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.795+0000 7f2a3a000640 1 -- 172.21.15.159:0/485244769 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f2a440cde70 con 0x7f2a440af4b0 2024-04-04T06:07:01.790 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:01.790 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:07:01.790 INFO:teuthology.orchestra.run.smithi159.stdout: "auth_dump": [ 2024-04-04T06:07:01.790 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.790 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.a", 2024-04-04T06:07:01.790 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5mvoZ2FxAAsVEQrkiICIH6T7ZzaqcWbA==", 2024-04-04T06:07:01.790 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.790 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:07:01.790 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:07:01.791 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:07:01.791 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:01.791 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.791 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.791 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.791 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.b", 2024-04-04T06:07:01.791 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5m+rccORAA6I/lQ+22QfWYL9+aoOFJ0w==", 2024-04-04T06:07:01.791 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.791 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:07:01.791 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:07:01.791 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:07:01.791 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:01.791 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.791 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.792 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.792 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.c", 2024-04-04T06:07:01.792 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5mjmQcKBAA8BjpQce6G7EycXOkyVECpQ==", 2024-04-04T06:07:01.792 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.792 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:07:01.792 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:07:01.792 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:07:01.792 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:01.792 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.792 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.792 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.d", 2024-04-04T06:07:01.792 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5m04hoDRAAKdhRI7nosZp7d4y7dVvzvg==", 2024-04-04T06:07:01.793 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.793 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:07:01.793 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:07:01.793 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:07:01.793 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:01.793 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.793 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.793 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.0", 2024-04-04T06:07:01.793 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQABQw5mCe7fNRAA6ZKxWvnMLf4jhAlpjudFRA==", 2024-04-04T06:07:01.793 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.793 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:07:01.793 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:07:01.793 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:01.794 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.794 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.1", 2024-04-04T06:07:01.794 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAFQw5mvIqTExAAU8Pm5alxFtHEL4qM9MavPw==", 2024-04-04T06:07:01.794 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.794 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:07:01.794 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:07:01.794 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:01.794 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.794 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.2", 2024-04-04T06:07:01.794 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAIQw5mAHblHRAAqEDe5dIGTDpdy2OsQyzJXA==", 2024-04-04T06:07:01.795 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.795 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:07:01.795 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:07:01.795 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:01.795 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.795 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.3", 2024-04-04T06:07:01.795 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQALQw5mwK7rJhAADQBAqE0Eej9rn91Wxhq/KA==", 2024-04-04T06:07:01.795 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.795 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:07:01.795 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:07:01.795 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:01.795 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.795 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.796 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.4", 2024-04-04T06:07:01.796 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAQQw5m/6I8GRAA2P7jI+AXYld2J11nDfhAUQ==", 2024-04-04T06:07:01.796 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.796 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:07:01.796 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:07:01.796 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:01.796 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.796 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.796 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.5", 2024-04-04T06:07:01.796 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQATQw5mv5HoMBAAn2mp2ys5KjS0UygOcX4vvg==", 2024-04-04T06:07:01.796 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.796 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:07:01.796 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:07:01.796 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:01.796 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.797 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.6", 2024-04-04T06:07:01.797 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAWQw5mx12gORAAT2LhrfEB5BJSl0ypgospzg==", 2024-04-04T06:07:01.797 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.797 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:07:01.797 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:07:01.797 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:01.797 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.797 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.797 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.7", 2024-04-04T06:07:01.797 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAaQw5m6HEOBxAAVegLH3bjVM7r1A6tK5eqHQ==", 2024-04-04T06:07:01.797 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.797 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:07:01.797 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:07:01.798 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:01.798 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.798 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.798 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.798 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.0", 2024-04-04T06:07:01.798 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mqNKJGxAA4jP9cadhRIyhQ3lssttWIw==", 2024-04-04T06:07:01.798 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.798 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:07:01.798 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:07:01.798 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:07:01.798 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:07:01.798 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.798 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.798 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.798 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.1", 2024-04-04T06:07:01.798 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mRkdOMBAA1ae1roIwrlB49DFQUoFSTg==", 2024-04-04T06:07:01.799 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.799 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:07:01.799 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:07:01.799 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:07:01.799 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:07:01.799 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.799 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.799 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.799 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.2", 2024-04-04T06:07:01.799 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mtW2/JRAAucxSSN9q0EQtI2a/P2yozQ==", 2024-04-04T06:07:01.799 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.799 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:07:01.799 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:07:01.799 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:07:01.799 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:07:01.799 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.800 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.800 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.3", 2024-04-04T06:07:01.800 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mgqD0OhAAov+3aDGyGEoqxep8+SBpSA==", 2024-04-04T06:07:01.800 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.800 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:07:01.800 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:07:01.800 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:07:01.800 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:07:01.800 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.800 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.800 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.admin", 2024-04-04T06:07:01.800 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD7Qg5m0O/iCBAAsWTzlEA0P60XYuxmJiAb4A==", 2024-04-04T06:07:01.800 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.800 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow *", 2024-04-04T06:07:01.801 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:07:01.801 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:07:01.801 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:01.801 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.801 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.801 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.bootstrap-mds", 2024-04-04T06:07:01.801 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAnQw5mtiTVJxAAKsIKvIeN/NiTalDxuUX7pQ==", 2024-04-04T06:07:01.801 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.801 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile bootstrap-mds" 2024-04-04T06:07:01.801 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.801 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.801 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.bootstrap-mgr", 2024-04-04T06:07:01.801 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAnQw5mjkLVJxAApecrTEDbo4a+2Mdx0uAfDg==", 2024-04-04T06:07:01.801 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.802 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-04T06:07:01.802 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.802 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.802 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.bootstrap-osd", 2024-04-04T06:07:01.802 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAnQw5mm1vVJxAAM3Js5WMu7TyS/0Vj2j9FiA==", 2024-04-04T06:07:01.802 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.802 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile bootstrap-osd" 2024-04-04T06:07:01.802 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.802 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.802 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.bootstrap-rbd", 2024-04-04T06:07:01.802 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAnQw5mAnTVJxAAvrbG+zXC45PX7ccvyUNroA==", 2024-04-04T06:07:01.802 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.802 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-04T06:07:01.803 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.803 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.803 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-04T06:07:01.803 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAnQw5mhpDVJxAA4tYTSqMJaOwYFwiso1Kcsg==", 2024-04-04T06:07:01.803 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.803 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-04T06:07:01.803 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.803 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.803 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.bootstrap-rgw", 2024-04-04T06:07:01.803 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAnQw5mFarVJxAAFVyMcHho8YIxsdQ70+gsbw==", 2024-04-04T06:07:01.803 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.803 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-04T06:07:01.804 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.804 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.804 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mgr.x", 2024-04-04T06:07:01.804 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5mhqbBCxAASBknMIj4uu7OiB3jn7WP8Q==", 2024-04-04T06:07:01.804 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.804 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow *", 2024-04-04T06:07:01.804 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile mgr", 2024-04-04T06:07:01.804 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:01.804 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:01.804 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:01.804 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mgr.y", 2024-04-04T06:07:01.804 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD7Qg5miXT0ORAAL6y306CPAfEGQKAJ0no4uw==", 2024-04-04T06:07:01.804 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:01.804 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow *", 2024-04-04T06:07:01.804 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile mgr", 2024-04-04T06:07:01.805 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:01.805 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.805 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:01.805 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-04-04T06:07:01.805 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:07:01.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.797+0000 7f2a49200640 1 -- 172.21.15.159:0/485244769 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2a2406c8f0 msgr2=0x7f2a2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:01.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.797+0000 7f2a49200640 1 --2- 172.21.15.159:0/485244769 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2a2406c8f0 0x7f2a2406edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2a44142e40 tx=0x7f2a30007990 comp rx=0 tx=0).stop 2024-04-04T06:07:01.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.797+0000 7f2a49200640 1 -- 172.21.15.159:0/485244769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a440af4b0 msgr2=0x7f2a441563a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:01.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.797+0000 7f2a49200640 1 --2- 172.21.15.159:0/485244769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a440af4b0 0x7f2a441563a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400b4a0 tx=0x7f2a3400b960 comp rx=0 tx=0).stop 2024-04-04T06:07:01.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.798+0000 7f2a49200640 1 -- 172.21.15.159:0/485244769 shutdown_connections 2024-04-04T06:07:01.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.798+0000 7f2a49200640 1 --2- 172.21.15.159:0/485244769 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2a2406c8f0 0x7f2a2406edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:01.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.798+0000 7f2a49200640 1 --2- 172.21.15.159:0/485244769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a44153580 0x7f2a44142f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:01.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.798+0000 7f2a49200640 1 --2- 172.21.15.159:0/485244769 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a440d07c0 0x7f2a440cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:01.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.798+0000 7f2a49200640 1 --2- 172.21.15.159:0/485244769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a440af4b0 0x7f2a441563a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:01.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.798+0000 7f2a49200640 1 -- 172.21.15.159:0/485244769 >> 172.21.15.159:0/485244769 conn(0x7f2a4406b750 msgr2=0x7f2a4406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:01.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.798+0000 7f2a49200640 1 -- 172.21.15.159:0/485244769 shutdown_connections 2024-04-04T06:07:01.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:01.798+0000 7f2a49200640 1 -- 172.21.15.159:0/485244769 wait complete. 2024-04-04T06:07:01.811 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-04T06:07:02.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.028+0000 7f4b2de00640 1 Processor -- start 2024-04-04T06:07:02.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.028+0000 7f4b2de00640 1 -- start start 2024-04-04T06:07:02.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.030+0000 7f4b2de00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b2813f190 0x7f4b280cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:02.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.030+0000 7f4b2de00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b280ce890 con 0x7f4b2806dd30 2024-04-04T06:07:02.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.030+0000 7f4b2de00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b280ceb50 con 0x7f4b280cc880 2024-04-04T06:07:02.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.030+0000 7f4b2de00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4b280cee30 con 0x7f4b2813f190 2024-04-04T06:07:02.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.030+0000 7f4b2d400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f4b2806dd30 0x7f4b2815ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58620/0 (socket says 172.21.15.159:58620) 2024-04-04T06:07:02.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.030+0000 7f4b2d400640 1 -- 172.21.15.159:0/1185603571 learned_addr learned my addr 172.21.15.159:0/1185603571 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:02.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.031+0000 7f4b2ca00640 1 --2- 172.21.15.159:0/1185603571 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b2813f190 0x7f4b280cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:02.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.031+0000 7f4b27e00640 1 -- 172.21.15.159:0/1185603571 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3530846312 0 0) 0x7f4b280ce890 con 0x7f4b2806dd30 2024-04-04T06:07:02.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.031+0000 7f4b27e00640 1 -- 172.21.15.159:0/1185603571 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b0c003520 con 0x7f4b2806dd30 2024-04-04T06:07:02.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.031+0000 7f4b27e00640 1 -- 172.21.15.159:0/1185603571 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2353620803 0 0) 0x7f4b0c003520 con 0x7f4b2806dd30 2024-04-04T06:07:02.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.031+0000 7f4b27e00640 1 -- 172.21.15.159:0/1185603571 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b2813f190 msgr2=0x7f4b280cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:02.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.031+0000 7f4b27e00640 1 --2- 172.21.15.159:0/1185603571 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b2813f190 0x7f4b280cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:02.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.031+0000 7f4b27e00640 1 -- 172.21.15.159:0/1185603571 >> v1:172.21.15.178:6789/0 conn(0x7f4b280cc880 legacy=0x7f4b2806d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:02.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.031+0000 7f4b27e00640 1 -- 172.21.15.159:0/1185603571 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b280cf110 con 0x7f4b2806dd30 2024-04-04T06:07:02.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.031+0000 7f4b27e00640 1 -- 172.21.15.159:0/1185603571 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f4b200019f0 con 0x7f4b2806dd30 2024-04-04T06:07:02.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.031+0000 7f4b27e00640 1 -- 172.21.15.159:0/1185603571 >> v1:172.21.15.159:6789/0 conn(0x7f4b2806dd30 legacy=0x7f4b2815ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:02.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.031+0000 7f4b27e00640 1 --2- 172.21.15.159:0/1185603571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b0c0039e0 0x7f4b0c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:02.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.031+0000 7f4b27e00640 1 --2- 172.21.15.159:0/1185603571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b0c004800 0x7f4b0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:02.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.032+0000 7f4b27e00640 1 --2- 172.21.15.159:0/1185603571 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b2813f190 0x7f4b0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:02.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.032+0000 7f4b27e00640 1 -- 172.21.15.159:0/1185603571 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4b280cee30 con 0x7f4b0c004800 2024-04-04T06:07:02.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.032+0000 7f4b27e00640 1 -- 172.21.15.159:0/1185603571 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c00ba70 con 0x7f4b0c0039e0 2024-04-04T06:07:02.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.032+0000 7f4b27e00640 1 -- 172.21.15.159:0/1185603571 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4b0c00bc70 con 0x7f4b2813f190 2024-04-04T06:07:02.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.032+0000 7f4b1fe00640 1 --2- 172.21.15.159:0/1185603571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b0c004800 0x7f4b0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:02.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.032+0000 7f4b2ca00640 1 --2- 172.21.15.159:0/1185603571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b0c0039e0 0x7f4b0c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:02.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.032+0000 7f4b2d400640 1 --2- 172.21.15.159:0/1185603571 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b2813f190 0x7f4b0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:02.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.032+0000 7f4b1fe00640 1 -- 172.21.15.159:0/1185603571 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b2813f190 msgr2=0x7f4b0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:02.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.032+0000 7f4b1fe00640 1 --2- 172.21.15.159:0/1185603571 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b2813f190 0x7f4b0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:02.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.032+0000 7f4b1fe00640 1 -- 172.21.15.159:0/1185603571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b0c0039e0 msgr2=0x7f4b0c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:02.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.032+0000 7f4b1fe00640 1 --2- 172.21.15.159:0/1185603571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b0c0039e0 0x7f4b0c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:02.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.032+0000 7f4b1fe00640 1 -- 172.21.15.159:0/1185603571 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0c003520 con 0x7f4b0c004800 2024-04-04T06:07:02.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.033+0000 7f4b1fe00640 1 --2- 172.21.15.159:0/1185603571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b0c004800 0x7f4b0c004c80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4b10001720 tx=0x7f4b0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f497cdc6624b80c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:02.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.033+0000 7f4b27e00640 1 -- 172.21.15.159:0/1185603571 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b10001370 con 0x7f4b0c004800 2024-04-04T06:07:02.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.033+0000 7f4b27e00640 1 -- 172.21.15.159:0/1185603571 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b100052d0 con 0x7f4b0c004800 2024-04-04T06:07:02.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.034+0000 7f4b27e00640 1 -- 172.21.15.159:0/1185603571 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b10003830 con 0x7f4b0c004800 2024-04-04T06:07:02.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.034+0000 7f4b2de00640 1 -- 172.21.15.159:0/1185603571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b0c004800 msgr2=0x7f4b0c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:02.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.034+0000 7f4b2de00640 1 --2- 172.21.15.159:0/1185603571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b0c004800 0x7f4b0c004c80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4b10001720 tx=0x7f4b0c005830 comp rx=0 tx=0).stop 2024-04-04T06:07:02.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.034+0000 7f4b2d400640 1 -- 172.21.15.159:0/1185603571 reap_dead start 2024-04-04T06:07:02.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.034+0000 7f4b2de00640 1 -- 172.21.15.159:0/1185603571 shutdown_connections 2024-04-04T06:07:02.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.034+0000 7f4b2de00640 1 -- 172.21.15.159:0/1185603571 >> 172.21.15.159:0/1185603571 conn(0x7f4b280b1ba0 msgr2=0x7f4b280b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:02.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.034+0000 7f4b2de00640 1 -- 172.21.15.159:0/1185603571 shutdown_connections 2024-04-04T06:07:02.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.034+0000 7f4b2de00640 1 -- 172.21.15.159:0/1185603571 wait complete. 2024-04-04T06:07:02.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.036+0000 7f4b2de00640 1 Processor -- start 2024-04-04T06:07:02.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.036+0000 7f4b2de00640 1 -- start start 2024-04-04T06:07:02.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.037+0000 7f4b2de00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b0c004800 0x7f4b28138cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:02.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.037+0000 7f4b2de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b280ae850 0x7f4b280adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:02.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.037+0000 7f4b2de00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b2815df10 0x7f4b2815e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:02.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.037+0000 7f4b2de00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4b28160620 con 0x7f4b280ae850 2024-04-04T06:07:02.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.037+0000 7f4b2de00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2813f570 con 0x7f4b2815df10 2024-04-04T06:07:02.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.037+0000 7f4b2de00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4b281607a0 con 0x7f4b0c004800 2024-04-04T06:07:02.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.037+0000 7f4b1fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b280ae850 0x7f4b280adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:02.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.037+0000 7f4b1fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b280ae850 0x7f4b280adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43160/0 (socket says 172.21.15.159:43160) 2024-04-04T06:07:02.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.037+0000 7f4b1fe00640 1 -- 172.21.15.159:0/3136591121 learned_addr learned my addr 172.21.15.159:0/3136591121 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:02.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.038+0000 7f4b1fe00640 1 -- 172.21.15.159:0/3136591121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b0c004800 msgr2=0x7f4b28138cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:02.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.038+0000 7f4b2ca00640 1 --2- 172.21.15.159:0/3136591121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b0c004800 0x7f4b28138cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:02.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.038+0000 7f4b2d400640 1 --2- 172.21.15.159:0/3136591121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b2815df10 0x7f4b2815e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:02.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.038+0000 7f4b1fe00640 1 --2- 172.21.15.159:0/3136591121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b0c004800 0x7f4b28138cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:02.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.038+0000 7f4b1fe00640 1 -- 172.21.15.159:0/3136591121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b2815df10 msgr2=0x7f4b2815e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:02.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.038+0000 7f4b1fe00640 1 --2- 172.21.15.159:0/3136591121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b2815df10 0x7f4b2815e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:02.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.038+0000 7f4b1fe00640 1 -- 172.21.15.159:0/3136591121 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b280aecb0 con 0x7f4b280ae850 2024-04-04T06:07:02.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.038+0000 7f4b1fe00640 1 --2- 172.21.15.159:0/3136591121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b280ae850 0x7f4b280adda0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4b10008fd0 tx=0x7f4b10001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:02.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.038+0000 7f4b25600640 1 -- 172.21.15.159:0/3136591121 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b10041070 con 0x7f4b280ae850 2024-04-04T06:07:02.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.038+0000 7f4b25600640 1 -- 172.21.15.159:0/3136591121 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b10005480 con 0x7f4b280ae850 2024-04-04T06:07:02.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.038+0000 7f4b25600640 1 -- 172.21.15.159:0/3136591121 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b10007a30 con 0x7f4b280ae850 2024-04-04T06:07:02.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.038+0000 7f4b2de00640 1 -- 172.21.15.159:0/3136591121 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b280b3140 con 0x7f4b280ae850 2024-04-04T06:07:02.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.039+0000 7f4b2de00640 1 -- 172.21.15.159:0/3136591121 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b280b3700 con 0x7f4b280ae850 2024-04-04T06:07:02.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.040+0000 7f4b2de00640 1 -- 172.21.15.159:0/3136591121 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b2815e8d0 con 0x7f4b280ae850 2024-04-04T06:07:02.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.041+0000 7f4b25600640 1 -- 172.21.15.159:0/3136591121 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b10044070 con 0x7f4b280ae850 2024-04-04T06:07:02.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.041+0000 7f4b25600640 1 --2- 172.21.15.159:0/3136591121 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4b0406c8f0 0x7f4b0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:02.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.042+0000 7f4b25600640 1 -- 172.21.15.159:0/3136591121 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f4b100ad430 con 0x7f4b280ae850 2024-04-04T06:07:02.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.042+0000 7f4b2ca00640 1 --2- 172.21.15.159:0/3136591121 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4b0406c8f0 0x7f4b0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:02.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.043+0000 7f4b2ca00640 1 --2- 172.21.15.159:0/3136591121 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4b0406c8f0 0x7f4b0406edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4b18009950 tx=0x7f4b18008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:02.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.046+0000 7f4b25600640 1 -- 172.21.15.159:0/3136591121 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4b100052a0 con 0x7f4b280ae850 2024-04-04T06:07:02.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.320+0000 7f4b2de00640 1 -- 172.21.15.159:0/3136591121 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f4b28075940 con 0x7f4b280ae850 2024-04-04T06:07:02.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.324+0000 7f4b25600640 1 -- 172.21.15.159:0/3136591121 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f4b1007bac0 con 0x7f4b280ae850 2024-04-04T06:07:02.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.327+0000 7f4b2de00640 1 -- 172.21.15.159:0/3136591121 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4b0406c8f0 msgr2=0x7f4b0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:02.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.327+0000 7f4b2de00640 1 --2- 172.21.15.159:0/3136591121 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4b0406c8f0 0x7f4b0406edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4b18009950 tx=0x7f4b18008040 comp rx=0 tx=0).stop 2024-04-04T06:07:02.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.327+0000 7f4b2de00640 1 -- 172.21.15.159:0/3136591121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b280ae850 msgr2=0x7f4b280adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:02.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.327+0000 7f4b2de00640 1 --2- 172.21.15.159:0/3136591121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b280ae850 0x7f4b280adda0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4b10008fd0 tx=0x7f4b10001120 comp rx=0 tx=0).stop 2024-04-04T06:07:02.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.327+0000 7f4b2de00640 1 -- 172.21.15.159:0/3136591121 shutdown_connections 2024-04-04T06:07:02.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.327+0000 7f4b2de00640 1 --2- 172.21.15.159:0/3136591121 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4b0406c8f0 0x7f4b0406edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:02.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.327+0000 7f4b2de00640 1 --2- 172.21.15.159:0/3136591121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b2815df10 0x7f4b2815e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:02.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.327+0000 7f4b2de00640 1 --2- 172.21.15.159:0/3136591121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b280ae850 0x7f4b280adda0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:02.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.327+0000 7f4b2de00640 1 --2- 172.21.15.159:0/3136591121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4b0c004800 0x7f4b28138cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:02.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.327+0000 7f4b2de00640 1 -- 172.21.15.159:0/3136591121 >> 172.21.15.159:0/3136591121 conn(0x7f4b280b1ba0 msgr2=0x7f4b2815f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:02.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.328+0000 7f4b2de00640 1 -- 172.21.15.159:0/3136591121 shutdown_connections 2024-04-04T06:07:02.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.328+0000 7f4b2de00640 1 -- 172.21.15.159:0/3136591121 wait complete. 2024-04-04T06:07:02.340 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-04T06:07:02.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.559+0000 7f5c18400640 1 Processor -- start 2024-04-04T06:07:02.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.559+0000 7f5c18400640 1 -- start start 2024-04-04T06:07:02.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.560+0000 7f5c18400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c1015b4e0 0x7f5c100b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:02.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.561+0000 7f5c18400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5c1015f700 0x7f5c1006e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:02.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.561+0000 7f5c18400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c1015b8c0 con 0x7f5c100afee0 2024-04-04T06:07:02.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.561+0000 7f5c18400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f5c100b5810 con 0x7f5c1015f700 2024-04-04T06:07:02.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.561+0000 7f5c18400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5c100afab0 con 0x7f5c1015b4e0 2024-04-04T06:07:02.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.561+0000 7f5c15000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f5c100afee0 0x7f5c1015eff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58636/0 (socket says 172.21.15.159:58636) 2024-04-04T06:07:02.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.561+0000 7f5c15000640 1 -- 172.21.15.159:0/552126994 learned_addr learned my addr 172.21.15.159:0/552126994 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:02.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.561+0000 7f5c15a00640 1 --2- 172.21.15.159:0/552126994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c1015b4e0 0x7f5c100b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:02.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.561+0000 7f5c16400640 1 --2- 172.21.15.159:0/552126994 >> v2:172.21.15.178:3300/0 conn(0x7f5c1015f700 0x7f5c1006e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:02.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.561+0000 7f5c0fe00640 1 -- 172.21.15.159:0/552126994 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1359156598 0 0) 0x7f5c1015b8c0 con 0x7f5c100afee0 2024-04-04T06:07:02.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.561+0000 7f5c0fe00640 1 -- 172.21.15.159:0/552126994 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bf8003520 con 0x7f5c100afee0 2024-04-04T06:07:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.562+0000 7f5c0fe00640 1 -- 172.21.15.159:0/552126994 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1291896337 0 0) 0x7f5bf8003520 con 0x7f5c100afee0 2024-04-04T06:07:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.562+0000 7f5c0fe00640 1 -- 172.21.15.159:0/552126994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c1015b4e0 msgr2=0x7f5c100b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.562+0000 7f5c0fe00640 1 --2- 172.21.15.159:0/552126994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c1015b4e0 0x7f5c100b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.562+0000 7f5c0fe00640 1 -- 172.21.15.159:0/552126994 >> v2:172.21.15.178:3300/0 conn(0x7f5c1015f700 msgr2=0x7f5c1006e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.562+0000 7f5c0fe00640 1 --2- 172.21.15.159:0/552126994 >> v2:172.21.15.178:3300/0 conn(0x7f5c1015f700 0x7f5c1006e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.562+0000 7f5c0fe00640 1 -- 172.21.15.159:0/552126994 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1015fd70 con 0x7f5c100afee0 2024-04-04T06:07:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.562+0000 7f5c0fe00640 1 -- 172.21.15.159:0/552126994 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f5bfc001670 con 0x7f5c100afee0 2024-04-04T06:07:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.562+0000 7f5c0fe00640 1 -- 172.21.15.159:0/552126994 >> v1:172.21.15.159:6789/0 conn(0x7f5c100afee0 legacy=0x7f5c1015eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.562+0000 7f5c0fe00640 1 --2- 172.21.15.159:0/552126994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bf80039b0 0x7f5bf8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.562+0000 7f5c0fe00640 1 --2- 172.21.15.159:0/552126994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bf8004800 0x7f5bf8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.562+0000 7f5c0fe00640 1 --2- 172.21.15.159:0/552126994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bf80051c0 0x7f5bf800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.562+0000 7f5c0fe00640 1 -- 172.21.15.159:0/552126994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5c100afab0 con 0x7f5bf8004800 2024-04-04T06:07:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.562+0000 7f5c0fe00640 1 -- 172.21.15.159:0/552126994 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf800bfb0 con 0x7f5bf80039b0 2024-04-04T06:07:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.562+0000 7f5c0fe00640 1 -- 172.21.15.159:0/552126994 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf800c180 con 0x7f5bf80051c0 2024-04-04T06:07:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.562+0000 7f5c15a00640 1 --2- 172.21.15.159:0/552126994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c1015b4e0 0x7f5c100b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:07:02.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.562+0000 7f5c16400640 1 --2- 172.21.15.159:0/552126994 >> v2:172.21.15.178:3300/0 conn(0x7f5c1015f700 0x7f5c1006e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:07:02.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.562+0000 7f5c15000640 1 --2- 172.21.15.159:0/552126994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bf8004800 0x7f5bf8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:02.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.563+0000 7f5c15000640 1 -- 172.21.15.159:0/552126994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bf80051c0 msgr2=0x7f5bf800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:02.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.563+0000 7f5c15000640 1 --2- 172.21.15.159:0/552126994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bf80051c0 0x7f5bf800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:02.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.563+0000 7f5c15000640 1 -- 172.21.15.159:0/552126994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bf80039b0 msgr2=0x7f5bf8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:02.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.563+0000 7f5c15a00640 1 --2- 172.21.15.159:0/552126994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bf80039b0 0x7f5bf8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:02.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.563+0000 7f5c15000640 1 --2- 172.21.15.159:0/552126994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bf80039b0 0x7f5bf8003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:02.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.563+0000 7f5c15000640 1 -- 172.21.15.159:0/552126994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf8003520 con 0x7f5bf8004800 2024-04-04T06:07:02.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.564+0000 7f5c15000640 1 --2- 172.21.15.159:0/552126994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bf8004800 0x7f5bf8004c80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00c210 tx=0x7f5bf8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f180fd62d11742e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:02.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.564+0000 7f5c0fe00640 1 -- 172.21.15.159:0/552126994 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc007d50 con 0x7f5bf8004800 2024-04-04T06:07:02.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.564+0000 7f5c0fe00640 1 -- 172.21.15.159:0/552126994 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bfc030070 con 0x7f5bf8004800 2024-04-04T06:07:02.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.564+0000 7f5c0fe00640 1 -- 172.21.15.159:0/552126994 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc02cd40 con 0x7f5bf8004800 2024-04-04T06:07:02.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.565+0000 7f5c18400640 1 -- 172.21.15.159:0/552126994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bf8004800 msgr2=0x7f5bf8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:02.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.565+0000 7f5c18400640 1 --2- 172.21.15.159:0/552126994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bf8004800 0x7f5bf8004c80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00c210 tx=0x7f5bf8005830 comp rx=0 tx=0).stop 2024-04-04T06:07:02.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.565+0000 7f5c16400640 1 -- 172.21.15.159:0/552126994 reap_dead start 2024-04-04T06:07:02.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.565+0000 7f5c18400640 1 -- 172.21.15.159:0/552126994 shutdown_connections 2024-04-04T06:07:02.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.565+0000 7f5c18400640 1 -- 172.21.15.159:0/552126994 >> 172.21.15.159:0/552126994 conn(0x7f5c100b6560 msgr2=0x7f5c100b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:02.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.565+0000 7f5c18400640 1 -- 172.21.15.159:0/552126994 shutdown_connections 2024-04-04T06:07:02.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.565+0000 7f5c18400640 1 -- 172.21.15.159:0/552126994 wait complete. 2024-04-04T06:07:02.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.567+0000 7f5c18400640 1 Processor -- start 2024-04-04T06:07:02.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.567+0000 7f5c18400640 1 -- start start 2024-04-04T06:07:02.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.568+0000 7f5c18400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bf8004800 0x7f5c1006e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:02.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.568+0000 7f5c18400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c1014efa0 0x7f5c1014f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:02.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.568+0000 7f5c18400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c10143900 0x7f5c10143d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:02.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.568+0000 7f5c18400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5c100b5010 con 0x7f5c10143900 2024-04-04T06:07:02.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.568+0000 7f5c18400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5c100b4d30 con 0x7f5bf8004800 2024-04-04T06:07:02.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.568+0000 7f5c18400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5c100b52f0 con 0x7f5c1014efa0 2024-04-04T06:07:02.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.568+0000 7f5c16400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c10143900 0x7f5c10143d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:02.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.568+0000 7f5c16400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c10143900 0x7f5c10143d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43182/0 (socket says 172.21.15.159:43182) 2024-04-04T06:07:02.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.568+0000 7f5c16400640 1 -- 172.21.15.159:0/3941328303 learned_addr learned my addr 172.21.15.159:0/3941328303 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:02.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.568+0000 7f5c15a00640 1 --2- 172.21.15.159:0/3941328303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bf8004800 0x7f5c1006e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:02.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.568+0000 7f5c15000640 1 --2- 172.21.15.159:0/3941328303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c1014efa0 0x7f5c1014f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:02.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.568+0000 7f5c16400640 1 -- 172.21.15.159:0/3941328303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c1014efa0 msgr2=0x7f5c1014f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:02.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.568+0000 7f5c16400640 1 --2- 172.21.15.159:0/3941328303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c1014efa0 0x7f5c1014f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:02.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.568+0000 7f5c16400640 1 -- 172.21.15.159:0/3941328303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bf8004800 msgr2=0x7f5c1006e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:02.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.568+0000 7f5c16400640 1 --2- 172.21.15.159:0/3941328303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bf8004800 0x7f5c1006e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:02.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.568+0000 7f5c16400640 1 -- 172.21.15.159:0/3941328303 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c10146000 con 0x7f5c10143900 2024-04-04T06:07:02.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.568+0000 7f5c15000640 1 --2- 172.21.15.159:0/3941328303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c1014efa0 0x7f5c1014f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:02.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.569+0000 7f5c16400640 1 --2- 172.21.15.159:0/3941328303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c10143900 0x7f5c10143d80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5c10070740 tx=0x7f5c0800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:02.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.569+0000 7f5c0d600640 1 -- 172.21.15.159:0/3941328303 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0801a070 con 0x7f5c10143900 2024-04-04T06:07:02.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.569+0000 7f5c0d600640 1 -- 172.21.15.159:0/3941328303 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c0800de90 con 0x7f5c10143900 2024-04-04T06:07:02.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.569+0000 7f5c18400640 1 -- 172.21.15.159:0/3941328303 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c101462e0 con 0x7f5c10143900 2024-04-04T06:07:02.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.569+0000 7f5c0d600640 1 -- 172.21.15.159:0/3941328303 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c08015640 con 0x7f5c10143900 2024-04-04T06:07:02.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.569+0000 7f5c18400640 1 -- 172.21.15.159:0/3941328303 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c101442c0 con 0x7f5c10143900 2024-04-04T06:07:02.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.571+0000 7f5c18400640 1 -- 172.21.15.159:0/3941328303 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c10067750 con 0x7f5c10143900 2024-04-04T06:07:02.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.571+0000 7f5c0d600640 1 -- 172.21.15.159:0/3941328303 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c0801d070 con 0x7f5c10143900 2024-04-04T06:07:02.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.572+0000 7f5c0d600640 1 --2- 172.21.15.159:0/3941328303 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5bf006c920 0x7f5bf006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:02.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.572+0000 7f5c0d600640 1 -- 172.21.15.159:0/3941328303 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f5c0808b750 con 0x7f5c10143900 2024-04-04T06:07:02.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.573+0000 7f5c15a00640 1 --2- 172.21.15.159:0/3941328303 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5bf006c920 0x7f5bf006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:02.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.574+0000 7f5c15a00640 1 --2- 172.21.15.159:0/3941328303 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5bf006c920 0x7f5bf006ede0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5c04006fd0 tx=0x7f5c04008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:02.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.577+0000 7f5c0d600640 1 -- 172.21.15.159:0/3941328303 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5c10067750 con 0x7f5c10143900 2024-04-04T06:07:02.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.856+0000 7f5c18400640 1 -- 172.21.15.159:0/3941328303 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f5c1015d420 con 0x7f5c10143900 2024-04-04T06:07:02.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.860+0000 7f5c0d600640 1 -- 172.21.15.159:0/3941328303 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f5c08010070 con 0x7f5c10143900 2024-04-04T06:07:02.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.863+0000 7f5c18400640 1 -- 172.21.15.159:0/3941328303 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5bf006c920 msgr2=0x7f5bf006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.863+0000 7f5c18400640 1 --2- 172.21.15.159:0/3941328303 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5bf006c920 0x7f5bf006ede0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5c04006fd0 tx=0x7f5c04008040 comp rx=0 tx=0).stop 2024-04-04T06:07:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.863+0000 7f5c18400640 1 -- 172.21.15.159:0/3941328303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c10143900 msgr2=0x7f5c10143d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.863+0000 7f5c18400640 1 --2- 172.21.15.159:0/3941328303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c10143900 0x7f5c10143d80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5c10070740 tx=0x7f5c0800d4a0 comp rx=0 tx=0).stop 2024-04-04T06:07:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.864+0000 7f5c18400640 1 -- 172.21.15.159:0/3941328303 shutdown_connections 2024-04-04T06:07:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.864+0000 7f5c18400640 1 --2- 172.21.15.159:0/3941328303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c10143900 0x7f5c10143d80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.864+0000 7f5c18400640 1 --2- 172.21.15.159:0/3941328303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c1014efa0 0x7f5c1014f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.864+0000 7f5c18400640 1 --2- 172.21.15.159:0/3941328303 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5bf006c920 0x7f5bf006ede0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.864+0000 7f5c18400640 1 --2- 172.21.15.159:0/3941328303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bf8004800 0x7f5c1006e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.864+0000 7f5c18400640 1 -- 172.21.15.159:0/3941328303 >> 172.21.15.159:0/3941328303 conn(0x7f5c100b6560 msgr2=0x7f5c1015cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.864+0000 7f5c18400640 1 -- 172.21.15.159:0/3941328303 shutdown_connections 2024-04-04T06:07:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:02.864+0000 7f5c18400640 1 -- 172.21.15.159:0/3941328303 wait complete. 2024-04-04T06:07:02.877 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-04T06:07:03.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.082+0000 7f7ec1a00640 1 Processor -- start 2024-04-04T06:07:03.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.082+0000 7f7ec1a00640 1 -- start start 2024-04-04T06:07:03.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.084+0000 7f7ec1a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7ebc153580 0x7f7ebc0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:03.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.084+0000 7f7ec1a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ebc0af4b0 0x7f7ebc1600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:03.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.084+0000 7f7ec1a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ebc0afab0 con 0x7f7ebc0d07c0 2024-04-04T06:07:03.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.084+0000 7f7ec1a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7ebc0d0c30 con 0x7f7ebc153580 2024-04-04T06:07:03.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.084+0000 7f7ec1a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7ebc15cc80 con 0x7f7ebc0af4b0 2024-04-04T06:07:03.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.085+0000 7f7ebb400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ebc0af4b0 0x7f7ebc1600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:03.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.085+0000 7f7ebaa00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7ebc153580 0x7f7ebc0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:03.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.085+0000 7f7ebb400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ebc0af4b0 0x7f7ebc1600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:52124/0 (socket says 172.21.15.159:52124) 2024-04-04T06:07:03.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.085+0000 7f7ebb400640 1 -- 172.21.15.159:0/2815008488 learned_addr learned my addr 172.21.15.159:0/2815008488 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:03.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.085+0000 7f7ebb400640 1 -- 172.21.15.159:0/2815008488 >> v2:172.21.15.178:3300/0 conn(0x7f7ebc153580 msgr2=0x7f7ebc0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:03.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.085+0000 7f7ebb400640 1 --2- 172.21.15.159:0/2815008488 >> v2:172.21.15.178:3300/0 conn(0x7f7ebc153580 0x7f7ebc0d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:03.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.085+0000 7f7ebb400640 1 -- 172.21.15.159:0/2815008488 >> v1:172.21.15.178:6789/0 conn(0x7f7ebc0d07c0 legacy=0x7f7ebc0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:03.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.085+0000 7f7ebb400640 1 -- 172.21.15.159:0/2815008488 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc15da00 con 0x7f7ebc0af4b0 2024-04-04T06:07:03.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.087+0000 7f7ebb400640 1 --2- 172.21.15.159:0/2815008488 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ebc0af4b0 0x7f7ebc1600d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009870 tx=0x7f7eb402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5980b22aba99d1d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:03.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.087+0000 7f7eb9600640 1 -- 172.21.15.159:0/2815008488 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4035020 con 0x7f7ebc0af4b0 2024-04-04T06:07:03.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.087+0000 7f7eb9600640 1 -- 172.21.15.159:0/2815008488 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eb4031640 con 0x7f7ebc0af4b0 2024-04-04T06:07:03.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.087+0000 7f7eb9600640 1 -- 172.21.15.159:0/2815008488 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb402d040 con 0x7f7ebc0af4b0 2024-04-04T06:07:03.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.088+0000 7f7ec1a00640 1 -- 172.21.15.159:0/2815008488 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ebc0af4b0 msgr2=0x7f7ebc1600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:03.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.088+0000 7f7ec1a00640 1 --2- 172.21.15.159:0/2815008488 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ebc0af4b0 0x7f7ebc1600d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009870 tx=0x7f7eb402b8f0 comp rx=0 tx=0).stop 2024-04-04T06:07:03.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.088+0000 7f7ec1a00640 1 -- 172.21.15.159:0/2815008488 shutdown_connections 2024-04-04T06:07:03.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.088+0000 7f7ec1a00640 1 --2- 172.21.15.159:0/2815008488 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ebc0af4b0 0x7f7ebc1600d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:03.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.088+0000 7f7ec1a00640 1 --2- 172.21.15.159:0/2815008488 >> v2:172.21.15.178:3300/0 conn(0x7f7ebc153580 0x7f7ebc0d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:03.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.088+0000 7f7ec1a00640 1 -- 172.21.15.159:0/2815008488 >> 172.21.15.159:0/2815008488 conn(0x7f7ebc06b750 msgr2=0x7f7ebc06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:03.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.088+0000 7f7ec1a00640 1 -- 172.21.15.159:0/2815008488 shutdown_connections 2024-04-04T06:07:03.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.088+0000 7f7ec1a00640 1 -- 172.21.15.159:0/2815008488 wait complete. 2024-04-04T06:07:03.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.090+0000 7f7ec1a00640 1 Processor -- start 2024-04-04T06:07:03.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.090+0000 7f7ec1a00640 1 -- start start 2024-04-04T06:07:03.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.091+0000 7f7ec1a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ebc0af4b0 0x7f7ebc1571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:03.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.091+0000 7f7ec1a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ebc0d07c0 0x7f7ebc159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:03.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.091+0000 7f7ec1a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ebc153580 0x7f7ebc0b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:03.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.091+0000 7f7ec1a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc15d4e0 con 0x7f7ebc0af4b0 2024-04-04T06:07:03.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.091+0000 7f7ec1a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc15d200 con 0x7f7ebc153580 2024-04-04T06:07:03.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.091+0000 7f7ec1a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7ebc153960 con 0x7f7ebc0d07c0 2024-04-04T06:07:03.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.091+0000 7f7ebaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ebc0af4b0 0x7f7ebc1571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:03.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.091+0000 7f7ebaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ebc0af4b0 0x7f7ebc1571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43186/0 (socket says 172.21.15.159:43186) 2024-04-04T06:07:03.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.091+0000 7f7ebaa00640 1 -- 172.21.15.159:0/4262381786 learned_addr learned my addr 172.21.15.159:0/4262381786 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:03.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.092+0000 7f7ebb400640 1 --2- 172.21.15.159:0/4262381786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ebc153580 0x7f7ebc0b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:03.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.092+0000 7f7eba000640 1 --2- 172.21.15.159:0/4262381786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ebc0d07c0 0x7f7ebc159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:03.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.092+0000 7f7ebaa00640 1 -- 172.21.15.159:0/4262381786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ebc0d07c0 msgr2=0x7f7ebc159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:03.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.092+0000 7f7ebaa00640 1 --2- 172.21.15.159:0/4262381786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ebc0d07c0 0x7f7ebc159850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:03.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.092+0000 7f7ebaa00640 1 -- 172.21.15.159:0/4262381786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ebc153580 msgr2=0x7f7ebc0b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:03.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.092+0000 7f7ebaa00640 1 --2- 172.21.15.159:0/4262381786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ebc153580 0x7f7ebc0b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:03.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.092+0000 7f7ebaa00640 1 -- 172.21.15.159:0/4262381786 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc157830 con 0x7f7ebc0af4b0 2024-04-04T06:07:03.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.092+0000 7f7eba000640 1 --2- 172.21.15.159:0/4262381786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ebc0d07c0 0x7f7ebc159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:03.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.092+0000 7f7ebaa00640 1 --2- 172.21.15.159:0/4262381786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ebc0af4b0 0x7f7ebc1571a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00e9b0 tx=0x7f7eac00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:03.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.093+0000 7f7eb2a00640 1 -- 172.21.15.159:0/4262381786 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac017070 con 0x7f7ebc0af4b0 2024-04-04T06:07:03.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.093+0000 7f7eb2a00640 1 -- 172.21.15.159:0/4262381786 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eac00c9a0 con 0x7f7ebc0af4b0 2024-04-04T06:07:03.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.093+0000 7f7ec1a00640 1 -- 172.21.15.159:0/4262381786 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ebc157b10 con 0x7f7ebc0af4b0 2024-04-04T06:07:03.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.093+0000 7f7eb2a00640 1 -- 172.21.15.159:0/4262381786 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac00cd90 con 0x7f7ebc0af4b0 2024-04-04T06:07:03.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.093+0000 7f7ec1a00640 1 -- 172.21.15.159:0/4262381786 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ebc1580d0 con 0x7f7ebc0af4b0 2024-04-04T06:07:03.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.094+0000 7f7ec1a00640 1 -- 172.21.15.159:0/4262381786 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ebc1584b0 con 0x7f7ebc0af4b0 2024-04-04T06:07:03.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.095+0000 7f7eb2a00640 1 -- 172.21.15.159:0/4262381786 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7eac01a070 con 0x7f7ebc0af4b0 2024-04-04T06:07:03.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.096+0000 7f7eb2a00640 1 --2- 172.21.15.159:0/4262381786 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e9806c8f0 0x7f7e9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:03.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.096+0000 7f7eb2a00640 1 -- 172.21.15.159:0/4262381786 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f7eac08b910 con 0x7f7ebc0af4b0 2024-04-04T06:07:03.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.096+0000 7f7eba000640 1 --2- 172.21.15.159:0/4262381786 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e9806c8f0 0x7f7e9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:03.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.097+0000 7f7eba000640 1 --2- 172.21.15.159:0/4262381786 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e9806c8f0 0x7f7e9806edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc071f60 tx=0x7f7ea4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:03.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.101+0000 7f7eb2a00640 1 -- 172.21.15.159:0/4262381786 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7ebc1584b0 con 0x7f7ebc0af4b0 2024-04-04T06:07:03.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.369+0000 7f7ec1a00640 1 -- 172.21.15.159:0/4262381786 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f7ebc0685c0 con 0x7f7ebc0af4b0 2024-04-04T06:07:03.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.373+0000 7f7eb2a00640 1 -- 172.21.15.159:0/4262381786 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f7eac05a000 con 0x7f7ebc0af4b0 2024-04-04T06:07:03.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.375+0000 7f7ec1a00640 1 -- 172.21.15.159:0/4262381786 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e9806c8f0 msgr2=0x7f7e9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:03.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.375+0000 7f7ec1a00640 1 --2- 172.21.15.159:0/4262381786 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e9806c8f0 0x7f7e9806edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc071f60 tx=0x7f7ea4009040 comp rx=0 tx=0).stop 2024-04-04T06:07:03.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.375+0000 7f7ec1a00640 1 -- 172.21.15.159:0/4262381786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ebc0af4b0 msgr2=0x7f7ebc1571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:03.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.375+0000 7f7ec1a00640 1 --2- 172.21.15.159:0/4262381786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ebc0af4b0 0x7f7ebc1571a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00e9b0 tx=0x7f7eac00ee70 comp rx=0 tx=0).stop 2024-04-04T06:07:03.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.376+0000 7f7ec1a00640 1 -- 172.21.15.159:0/4262381786 shutdown_connections 2024-04-04T06:07:03.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.376+0000 7f7ec1a00640 1 --2- 172.21.15.159:0/4262381786 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e9806c8f0 0x7f7e9806edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:03.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.376+0000 7f7ec1a00640 1 --2- 172.21.15.159:0/4262381786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ebc153580 0x7f7ebc0b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:03.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.376+0000 7f7ec1a00640 1 --2- 172.21.15.159:0/4262381786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ebc0d07c0 0x7f7ebc159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:03.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.376+0000 7f7ec1a00640 1 --2- 172.21.15.159:0/4262381786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ebc0af4b0 0x7f7ebc1571a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:03.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.376+0000 7f7ec1a00640 1 -- 172.21.15.159:0/4262381786 >> 172.21.15.159:0/4262381786 conn(0x7f7ebc06b750 msgr2=0x7f7ebc0aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:03.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.376+0000 7f7ec1a00640 1 -- 172.21.15.159:0/4262381786 shutdown_connections 2024-04-04T06:07:03.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.376+0000 7f7ec1a00640 1 -- 172.21.15.159:0/4262381786 wait complete. 2024-04-04T06:07:03.388 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-04T06:07:03.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.597+0000 7f779b200640 1 Processor -- start 2024-04-04T06:07:03.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.597+0000 7f779b200640 1 -- start start 2024-04-04T06:07:03.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.599+0000 7f779b200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7794153580 0x7f77940d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:03.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.599+0000 7f779b200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77940d07c0 0x7f77940aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:03.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.599+0000 7f779b200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f77940af2e0 0x7f779406d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:03.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.599+0000 7f779b200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f77940d0c00 con 0x7f7794153580 2024-04-04T06:07:03.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.599+0000 7f779b200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f77940afab0 con 0x7f77940af2e0 2024-04-04T06:07:03.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.599+0000 7f779b200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f779415cc80 con 0x7f77940d07c0 2024-04-04T06:07:03.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.599+0000 7f7793e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7794153580 0x7f77940d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:03.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.599+0000 7f7793e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7794153580 0x7f77940d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43194/0 (socket says 172.21.15.159:43194) 2024-04-04T06:07:03.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.599+0000 7f7793e00640 1 -- 172.21.15.159:0/6410169 learned_addr learned my addr 172.21.15.159:0/6410169 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:03.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.599+0000 7f7793e00640 1 -- 172.21.15.159:0/6410169 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77940d07c0 msgr2=0x7f77940aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:03.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.599+0000 7f7793e00640 1 --2- 172.21.15.159:0/6410169 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77940d07c0 0x7f77940aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:03.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.599+0000 7f7799200640 1 --2- 172.21.15.159:0/6410169 >> v2:172.21.15.178:3300/0 conn(0x7f77940af2e0 0x7f779406d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:03.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.599+0000 7f7793e00640 1 -- 172.21.15.159:0/6410169 >> v2:172.21.15.178:3300/0 conn(0x7f77940af2e0 msgr2=0x7f779406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:03.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.599+0000 7f7793e00640 1 --2- 172.21.15.159:0/6410169 >> v2:172.21.15.178:3300/0 conn(0x7f77940af2e0 0x7f779406d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:03.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.600+0000 7f7793e00640 1 -- 172.21.15.159:0/6410169 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f779415da00 con 0x7f7794153580 2024-04-04T06:07:03.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.600+0000 7f7799200640 1 --2- 172.21.15.159:0/6410169 >> v2:172.21.15.178:3300/0 conn(0x7f77940af2e0 0x7f779406d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:03.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.601+0000 7f7793e00640 1 --2- 172.21.15.159:0/6410169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7794153580 0x7f77940d0280 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f77940cff70 tx=0x7f778402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28faca63ae101898 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:03.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.601+0000 7f7792a00640 1 -- 172.21.15.159:0/6410169 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778402e070 con 0x7f7794153580 2024-04-04T06:07:03.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.601+0000 7f7792a00640 1 -- 172.21.15.159:0/6410169 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7784037070 con 0x7f7794153580 2024-04-04T06:07:03.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.601+0000 7f7792a00640 1 -- 172.21.15.159:0/6410169 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784031a00 con 0x7f7794153580 2024-04-04T06:07:03.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.601+0000 7f779b200640 1 -- 172.21.15.159:0/6410169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7794153580 msgr2=0x7f77940d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:03.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.601+0000 7f779b200640 1 --2- 172.21.15.159:0/6410169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7794153580 0x7f77940d0280 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f77940cff70 tx=0x7f778402b880 comp rx=0 tx=0).stop 2024-04-04T06:07:03.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.602+0000 7f779b200640 1 -- 172.21.15.159:0/6410169 shutdown_connections 2024-04-04T06:07:03.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.602+0000 7f779b200640 1 --2- 172.21.15.159:0/6410169 >> v2:172.21.15.178:3300/0 conn(0x7f77940af2e0 0x7f779406d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:03.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.602+0000 7f779b200640 1 --2- 172.21.15.159:0/6410169 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77940d07c0 0x7f77940aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:03.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.602+0000 7f779b200640 1 --2- 172.21.15.159:0/6410169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7794153580 0x7f77940d0280 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:03.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.602+0000 7f779b200640 1 -- 172.21.15.159:0/6410169 >> 172.21.15.159:0/6410169 conn(0x7f779406b750 msgr2=0x7f779406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:03.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.602+0000 7f779b200640 1 -- 172.21.15.159:0/6410169 shutdown_connections 2024-04-04T06:07:03.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.602+0000 7f779b200640 1 -- 172.21.15.159:0/6410169 wait complete. 2024-04-04T06:07:03.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.603+0000 7f779b200640 1 Processor -- start 2024-04-04T06:07:03.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.603+0000 7f779b200640 1 -- start start 2024-04-04T06:07:03.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.604+0000 7f779b200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77940af2e0 0x7f7794153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:03.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.604+0000 7f779b200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77940d07c0 0x7f7794152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:03.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.604+0000 7f779b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7794153580 0x7f77941564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:03.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.604+0000 7f779b200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f779415d4e0 con 0x7f7794153580 2024-04-04T06:07:03.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.604+0000 7f779b200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f779415d200 con 0x7f77940af2e0 2024-04-04T06:07:03.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.604+0000 7f779b200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7794153960 con 0x7f77940d07c0 2024-04-04T06:07:03.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.605+0000 7f7799200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7794153580 0x7f77941564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:03.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.605+0000 7f7799200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7794153580 0x7f77941564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43206/0 (socket says 172.21.15.159:43206) 2024-04-04T06:07:03.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.605+0000 7f7799200640 1 -- 172.21.15.159:0/1788578364 learned_addr learned my addr 172.21.15.159:0/1788578364 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:03.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.605+0000 7f7799200640 1 -- 172.21.15.159:0/1788578364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77940d07c0 msgr2=0x7f7794152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:03.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.605+0000 7f7799200640 1 --2- 172.21.15.159:0/1788578364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77940d07c0 0x7f7794152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:03.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.605+0000 7f7799200640 1 -- 172.21.15.159:0/1788578364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77940af2e0 msgr2=0x7f7794153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:03.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.605+0000 7f7799200640 1 --2- 172.21.15.159:0/1788578364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77940af2e0 0x7f7794153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:03.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.605+0000 7f7799200640 1 -- 172.21.15.159:0/1788578364 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77941542e0 con 0x7f7794153580 2024-04-04T06:07:03.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.605+0000 7f7799200640 1 --2- 172.21.15.159:0/1788578364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7794153580 0x7f77941564e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f778c00cd50 tx=0x7f778c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:03.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.606+0000 7f778be00640 1 -- 172.21.15.159:0/1788578364 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c00f680 con 0x7f7794153580 2024-04-04T06:07:03.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.606+0000 7f778be00640 1 -- 172.21.15.159:0/1788578364 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f778c00f840 con 0x7f7794153580 2024-04-04T06:07:03.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.606+0000 7f778be00640 1 -- 172.21.15.159:0/1788578364 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c00fc30 con 0x7f7794153580 2024-04-04T06:07:03.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.606+0000 7f779b200640 1 -- 172.21.15.159:0/1788578364 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77941545c0 con 0x7f7794153580 2024-04-04T06:07:03.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.606+0000 7f779b200640 1 -- 172.21.15.159:0/1788578364 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7794154b80 con 0x7f7794153580 2024-04-04T06:07:03.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.607+0000 7f779b200640 1 -- 172.21.15.159:0/1788578364 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77940685c0 con 0x7f7794153580 2024-04-04T06:07:03.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.608+0000 7f778be00640 1 -- 172.21.15.159:0/1788578364 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f778c00e3e0 con 0x7f7794153580 2024-04-04T06:07:03.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.609+0000 7f778be00640 1 --2- 172.21.15.159:0/1788578364 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f777406c8f0 0x7f777406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:03.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.609+0000 7f778be00640 1 -- 172.21.15.159:0/1788578364 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f778c08c680 con 0x7f7794153580 2024-04-04T06:07:03.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.610+0000 7f7793e00640 1 --2- 172.21.15.159:0/1788578364 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f777406c8f0 0x7f777406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:03.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.611+0000 7f7793e00640 1 --2- 172.21.15.159:0/1788578364 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f777406c8f0 0x7f777406edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7784036000 tx=0x7f7784030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:03.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.613+0000 7f778be00640 1 -- 172.21.15.159:0/1788578364 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f778c014030 con 0x7f7794153580 2024-04-04T06:07:03.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.881+0000 7f779b200640 1 -- 172.21.15.159:0/1788578364 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f779406eca0 con 0x7f7794153580 2024-04-04T06:07:03.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.884+0000 7f778be00640 1 -- 172.21.15.159:0/1788578364 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f778c05ad60 con 0x7f7794153580 2024-04-04T06:07:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.887+0000 7f779b200640 1 -- 172.21.15.159:0/1788578364 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f777406c8f0 msgr2=0x7f777406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.887+0000 7f779b200640 1 --2- 172.21.15.159:0/1788578364 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f777406c8f0 0x7f777406edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7784036000 tx=0x7f7784030000 comp rx=0 tx=0).stop 2024-04-04T06:07:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.887+0000 7f779b200640 1 -- 172.21.15.159:0/1788578364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7794153580 msgr2=0x7f77941564e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.887+0000 7f779b200640 1 --2- 172.21.15.159:0/1788578364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7794153580 0x7f77941564e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f778c00cd50 tx=0x7f778c00a650 comp rx=0 tx=0).stop 2024-04-04T06:07:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.887+0000 7f779b200640 1 -- 172.21.15.159:0/1788578364 shutdown_connections 2024-04-04T06:07:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.887+0000 7f779b200640 1 --2- 172.21.15.159:0/1788578364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7794153580 0x7f77941564e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.887+0000 7f779b200640 1 --2- 172.21.15.159:0/1788578364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f77940d07c0 0x7f7794152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.887+0000 7f779b200640 1 --2- 172.21.15.159:0/1788578364 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f777406c8f0 0x7f777406edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.887+0000 7f779b200640 1 --2- 172.21.15.159:0/1788578364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77940af2e0 0x7f7794153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.887+0000 7f779b200640 1 -- 172.21.15.159:0/1788578364 >> 172.21.15.159:0/1788578364 conn(0x7f779406b750 msgr2=0x7f77940b4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.887+0000 7f779b200640 1 -- 172.21.15.159:0/1788578364 shutdown_connections 2024-04-04T06:07:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:03.887+0000 7f779b200640 1 -- 172.21.15.159:0/1788578364 wait complete. 2024-04-04T06:07:03.899 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-04T06:07:04.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.111+0000 7ff857200640 1 Processor -- start 2024-04-04T06:07:04.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.111+0000 7ff857200640 1 -- start start 2024-04-04T06:07:04.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.112+0000 7ff857200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff8580b4890 0x7ff858072570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:04.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.113+0000 7ff857200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff85806d620 0x7ff858072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:04.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.113+0000 7ff857200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8580b4c70 con 0x7ff85815f6a0 2024-04-04T06:07:04.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.113+0000 7ff857200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff8580b3eb0 con 0x7ff8580b4890 2024-04-04T06:07:04.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.113+0000 7ff857200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff85815fb30 con 0x7ff85806d620 2024-04-04T06:07:04.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.113+0000 7ff856800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff85806d620 0x7ff858072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:04.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.113+0000 7ff855e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff8580b4890 0x7ff858072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:04.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.113+0000 7ff855e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff8580b4890 0x7ff858072570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:37890/0 (socket says 172.21.15.159:37890) 2024-04-04T06:07:04.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.113+0000 7ff856800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff85806d620 0x7ff858072ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:43994/0 (socket says 172.21.15.159:43994) 2024-04-04T06:07:04.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.113+0000 7ff855e00640 1 -- 172.21.15.159:0/802412434 learned_addr learned my addr 172.21.15.159:0/802412434 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:04.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.114+0000 7ff855e00640 1 -- 172.21.15.159:0/802412434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff85806d620 msgr2=0x7ff858072ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:04.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.114+0000 7ff855e00640 1 --2- 172.21.15.159:0/802412434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff85806d620 0x7ff858072ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:04.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.114+0000 7ff855e00640 1 -- 172.21.15.159:0/802412434 >> v1:172.21.15.178:6789/0 conn(0x7ff85815f6a0 legacy=0x7ff8580713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:04.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.114+0000 7ff855e00640 1 -- 172.21.15.159:0/802412434 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85815b7b0 con 0x7ff8580b4890 2024-04-04T06:07:04.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.116+0000 7ff855e00640 1 --2- 172.21.15.159:0/802412434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8580b4890 0x7ff858072570 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff848009870 tx=0x7ff84802db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=638c37ea072695ff server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:04.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.116+0000 7ff854a00640 1 -- 172.21.15.159:0/802412434 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84803b070 con 0x7ff8580b4890 2024-04-04T06:07:04.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.116+0000 7ff854a00640 1 -- 172.21.15.159:0/802412434 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff84802e940 con 0x7ff8580b4890 2024-04-04T06:07:04.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.116+0000 7ff854a00640 1 -- 172.21.15.159:0/802412434 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84802ed10 con 0x7ff8580b4890 2024-04-04T06:07:04.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.117+0000 7ff857200640 1 -- 172.21.15.159:0/802412434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8580b4890 msgr2=0x7ff858072570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:04.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.117+0000 7ff857200640 1 --2- 172.21.15.159:0/802412434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8580b4890 0x7ff858072570 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff848009870 tx=0x7ff84802db90 comp rx=0 tx=0).stop 2024-04-04T06:07:04.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.117+0000 7ff857200640 1 -- 172.21.15.159:0/802412434 shutdown_connections 2024-04-04T06:07:04.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.117+0000 7ff857200640 1 --2- 172.21.15.159:0/802412434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff85806d620 0x7ff858072ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:04.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.117+0000 7ff857200640 1 --2- 172.21.15.159:0/802412434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8580b4890 0x7ff858072570 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:04.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.117+0000 7ff857200640 1 -- 172.21.15.159:0/802412434 >> 172.21.15.159:0/802412434 conn(0x7ff8580b1ba0 msgr2=0x7ff8580b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:04.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.117+0000 7ff857200640 1 -- 172.21.15.159:0/802412434 shutdown_connections 2024-04-04T06:07:04.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.117+0000 7ff857200640 1 -- 172.21.15.159:0/802412434 wait complete. 2024-04-04T06:07:04.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.119+0000 7ff857200640 1 Processor -- start 2024-04-04T06:07:04.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.119+0000 7ff857200640 1 -- start start 2024-04-04T06:07:04.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.120+0000 7ff857200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff85806d620 0x7ff8581f1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:04.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.120+0000 7ff857200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8580b4890 0x7ff8581f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:04.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.120+0000 7ff857200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff858148390 0x7ff858146300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:04.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.120+0000 7ff857200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff85815c050 con 0x7ff85806d620 2024-04-04T06:07:04.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.120+0000 7ff857200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff85815bd70 con 0x7ff858148390 2024-04-04T06:07:04.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.120+0000 7ff857200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff85815c570 con 0x7ff8580b4890 2024-04-04T06:07:04.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.120+0000 7ff855e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff85806d620 0x7ff8581f1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:04.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.120+0000 7ff855e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff85806d620 0x7ff8581f1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59290/0 (socket says 172.21.15.159:59290) 2024-04-04T06:07:04.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.120+0000 7ff855e00640 1 -- 172.21.15.159:0/2083629324 learned_addr learned my addr 172.21.15.159:0/2083629324 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:04.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.121+0000 7ff856800640 1 --2- 172.21.15.159:0/2083629324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff858148390 0x7ff858146300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:04.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.121+0000 7ff855e00640 1 -- 172.21.15.159:0/2083629324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8580b4890 msgr2=0x7ff8581f7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:04.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.121+0000 7ff855400640 1 --2- 172.21.15.159:0/2083629324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8580b4890 0x7ff8581f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:04.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.121+0000 7ff855e00640 1 --2- 172.21.15.159:0/2083629324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8580b4890 0x7ff8581f7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:04.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.121+0000 7ff855e00640 1 -- 172.21.15.159:0/2083629324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff858148390 msgr2=0x7ff858146300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:04.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.121+0000 7ff855e00640 1 --2- 172.21.15.159:0/2083629324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff858148390 0x7ff858146300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:04.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.121+0000 7ff855e00640 1 -- 172.21.15.159:0/2083629324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8581f3da0 con 0x7ff85806d620 2024-04-04T06:07:04.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.121+0000 7ff856800640 1 --2- 172.21.15.159:0/2083629324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff858148390 0x7ff858146300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:04.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.121+0000 7ff855400640 1 --2- 172.21.15.159:0/2083629324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8580b4890 0x7ff8581f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:04.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.121+0000 7ff855e00640 1 --2- 172.21.15.159:0/2083629324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff85806d620 0x7ff8581f1450 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff848000c00 tx=0x7ff8480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:04.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.121+0000 7ff84e000640 1 -- 172.21.15.159:0/2083629324 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848046070 con 0x7ff85806d620 2024-04-04T06:07:04.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.121+0000 7ff84e000640 1 -- 172.21.15.159:0/2083629324 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff848036d80 con 0x7ff85806d620 2024-04-04T06:07:04.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.121+0000 7ff857200640 1 -- 172.21.15.159:0/2083629324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8581f82b0 con 0x7ff85806d620 2024-04-04T06:07:04.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.121+0000 7ff84e000640 1 -- 172.21.15.159:0/2083629324 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84803b070 con 0x7ff85806d620 2024-04-04T06:07:04.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.122+0000 7ff857200640 1 -- 172.21.15.159:0/2083629324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff858148b70 con 0x7ff85806d620 2024-04-04T06:07:04.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.123+0000 7ff857200640 1 -- 172.21.15.159:0/2083629324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff858075940 con 0x7ff85806d620 2024-04-04T06:07:04.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.124+0000 7ff84e000640 1 -- 172.21.15.159:0/2083629324 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff848049070 con 0x7ff85806d620 2024-04-04T06:07:04.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.125+0000 7ff84e000640 1 --2- 172.21.15.159:0/2083629324 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff83806c8f0 0x7ff83806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:04.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.125+0000 7ff84e000640 1 -- 172.21.15.159:0/2083629324 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7ff8480acf60 con 0x7ff85806d620 2024-04-04T06:07:04.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.125+0000 7ff855400640 1 --2- 172.21.15.159:0/2083629324 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff83806c8f0 0x7ff83806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:04.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.126+0000 7ff855400640 1 --2- 172.21.15.159:0/2083629324 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff83806c8f0 0x7ff83806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff8581f1990 tx=0x7ff844009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:04.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.129+0000 7ff84e000640 1 -- 172.21.15.159:0/2083629324 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff8581f3da0 con 0x7ff85806d620 2024-04-04T06:07:04.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.406+0000 7ff857200640 1 -- 172.21.15.159:0/2083629324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7ff8580d03b0 con 0x7ff85806d620 2024-04-04T06:07:04.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.411+0000 7ff84e000640 1 -- 172.21.15.159:0/2083629324 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7ff84807b6d0 con 0x7ff85806d620 2024-04-04T06:07:04.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.414+0000 7ff857200640 1 -- 172.21.15.159:0/2083629324 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff83806c8f0 msgr2=0x7ff83806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:04.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.414+0000 7ff857200640 1 --2- 172.21.15.159:0/2083629324 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff83806c8f0 0x7ff83806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff8581f1990 tx=0x7ff844009040 comp rx=0 tx=0).stop 2024-04-04T06:07:04.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.414+0000 7ff857200640 1 -- 172.21.15.159:0/2083629324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff85806d620 msgr2=0x7ff8581f1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:04.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.414+0000 7ff857200640 1 --2- 172.21.15.159:0/2083629324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff85806d620 0x7ff8581f1450 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff848000c00 tx=0x7ff8480046c0 comp rx=0 tx=0).stop 2024-04-04T06:07:04.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.414+0000 7ff857200640 1 -- 172.21.15.159:0/2083629324 shutdown_connections 2024-04-04T06:07:04.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.414+0000 7ff857200640 1 --2- 172.21.15.159:0/2083629324 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff83806c8f0 0x7ff83806edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:04.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.414+0000 7ff857200640 1 --2- 172.21.15.159:0/2083629324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff858148390 0x7ff858146300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:04.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.414+0000 7ff857200640 1 --2- 172.21.15.159:0/2083629324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8580b4890 0x7ff8581f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:04.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.414+0000 7ff857200640 1 --2- 172.21.15.159:0/2083629324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff85806d620 0x7ff8581f1450 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:04.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.414+0000 7ff857200640 1 -- 172.21.15.159:0/2083629324 >> 172.21.15.159:0/2083629324 conn(0x7ff8580b1ba0 msgr2=0x7ff8580ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:04.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.414+0000 7ff857200640 1 -- 172.21.15.159:0/2083629324 shutdown_connections 2024-04-04T06:07:04.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.414+0000 7ff857200640 1 -- 172.21.15.159:0/2083629324 wait complete. 2024-04-04T06:07:04.427 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-04T06:07:04.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.639+0000 7fbb0b200640 1 Processor -- start 2024-04-04T06:07:04.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.639+0000 7fbb0b200640 1 -- start start 2024-04-04T06:07:04.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.641+0000 7fbb0b200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbb0c13f190 0x7fbb0c0cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:04.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.641+0000 7fbb0b200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fbb0c06dd30 0x7fbb0c0cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:04.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.641+0000 7fbb0b200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb0c0ce890 con 0x7fbb0c0cc880 2024-04-04T06:07:04.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.641+0000 7fbb0b200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fbb0c0cd230 con 0x7fbb0c06dd30 2024-04-04T06:07:04.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.641+0000 7fbb0b200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fbb0c0ceb50 con 0x7fbb0c13f190 2024-04-04T06:07:04.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.641+0000 7fbb0a800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fbb0c06dd30 0x7fbb0c0cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:04.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.641+0000 7fbb0a800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fbb0c06dd30 0x7fbb0c0cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59306/0 (socket says 172.21.15.159:59306) 2024-04-04T06:07:04.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.641+0000 7fbb0a800640 1 -- 172.21.15.159:0/3338739440 learned_addr learned my addr 172.21.15.159:0/3338739440 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:04.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.641+0000 7fbb01400640 1 --1- 172.21.15.159:0/3338739440 >> v1:172.21.15.159:6789/0 conn(0x7fbb0c0cc880 0x7fbb0c06d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52410/0 (socket says 172.21.15.159:52410) 2024-04-04T06:07:04.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.642+0000 7fbb09e00640 1 --2- 172.21.15.159:0/3338739440 >> v2:172.21.15.178:3300/0 conn(0x7fbb0c13f190 0x7fbb0c0cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:04.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.642+0000 7fbb0a800640 1 -- 172.21.15.159:0/3338739440 >> v2:172.21.15.178:3300/0 conn(0x7fbb0c13f190 msgr2=0x7fbb0c0cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:04.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.642+0000 7fbb0a800640 1 --2- 172.21.15.159:0/3338739440 >> v2:172.21.15.178:3300/0 conn(0x7fbb0c13f190 0x7fbb0c0cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:04.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.642+0000 7fbb0a800640 1 -- 172.21.15.159:0/3338739440 >> v1:172.21.15.159:6789/0 conn(0x7fbb0c0cc880 legacy=0x7fbb0c06d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:04.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.642+0000 7fbb0a800640 1 -- 172.21.15.159:0/3338739440 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0c0cee30 con 0x7fbb0c06dd30 2024-04-04T06:07:04.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.642+0000 7fbb09e00640 1 --2- 172.21.15.159:0/3338739440 >> v2:172.21.15.178:3300/0 conn(0x7fbb0c13f190 0x7fbb0c0cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:04.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.643+0000 7fbb0a800640 1 --2- 172.21.15.159:0/3338739440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb0c06dd30 0x7fbb0c0cccf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c06f660 tx=0x7fbb0402b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e843745daa8dc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:04.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.643+0000 7fbb09400640 1 -- 172.21.15.159:0/3338739440 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04034070 con 0x7fbb0c06dd30 2024-04-04T06:07:04.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.643+0000 7fbb09400640 1 -- 172.21.15.159:0/3338739440 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb04030870 con 0x7fbb0c06dd30 2024-04-04T06:07:04.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.643+0000 7fbb09400640 1 -- 172.21.15.159:0/3338739440 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04030c40 con 0x7fbb0c06dd30 2024-04-04T06:07:04.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.644+0000 7fbb0b200640 1 -- 172.21.15.159:0/3338739440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb0c06dd30 msgr2=0x7fbb0c0cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:04.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.644+0000 7fbb0b200640 1 --2- 172.21.15.159:0/3338739440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb0c06dd30 0x7fbb0c0cccf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c06f660 tx=0x7fbb0402b9b0 comp rx=0 tx=0).stop 2024-04-04T06:07:04.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.644+0000 7fbb0b200640 1 -- 172.21.15.159:0/3338739440 shutdown_connections 2024-04-04T06:07:04.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.644+0000 7fbb0b200640 1 --2- 172.21.15.159:0/3338739440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb0c06dd30 0x7fbb0c0cccf0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:04.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.644+0000 7fbb0b200640 1 --2- 172.21.15.159:0/3338739440 >> v2:172.21.15.178:3300/0 conn(0x7fbb0c13f190 0x7fbb0c0cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:04.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.644+0000 7fbb0b200640 1 -- 172.21.15.159:0/3338739440 >> 172.21.15.159:0/3338739440 conn(0x7fbb0c0b1ba0 msgr2=0x7fbb0c0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:04.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.644+0000 7fbb0b200640 1 -- 172.21.15.159:0/3338739440 shutdown_connections 2024-04-04T06:07:04.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.644+0000 7fbb0b200640 1 -- 172.21.15.159:0/3338739440 wait complete. 2024-04-04T06:07:04.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.645+0000 7fbb0b200640 1 Processor -- start 2024-04-04T06:07:04.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.646+0000 7fbb0b200640 1 -- start start 2024-04-04T06:07:04.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.646+0000 7fbb0b200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb0c06dd30 0x7fbb0c13e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:04.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.646+0000 7fbb0b200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb0c0cc880 0x7fbb0c0adf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:04.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.647+0000 7fbb0b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb0c13f190 0x7fbb0c0b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:04.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.647+0000 7fbb0b200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c13f570 con 0x7fbb0c13f190 2024-04-04T06:07:04.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.647+0000 7fbb0b200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c160c70 con 0x7fbb0c06dd30 2024-04-04T06:07:04.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.647+0000 7fbb0b200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbb0c0cfb70 con 0x7fbb0c0cc880 2024-04-04T06:07:04.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.647+0000 7fbb0a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb0c13f190 0x7fbb0c0b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:04.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.647+0000 7fbb0a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb0c13f190 0x7fbb0c0b0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59322/0 (socket says 172.21.15.159:59322) 2024-04-04T06:07:04.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.647+0000 7fbb0a800640 1 -- 172.21.15.159:0/3913399060 learned_addr learned my addr 172.21.15.159:0/3913399060 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:04.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.647+0000 7fbb09e00640 1 --2- 172.21.15.159:0/3913399060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb0c06dd30 0x7fbb0c13e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:04.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.647+0000 7fbb0a800640 1 -- 172.21.15.159:0/3913399060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb0c0cc880 msgr2=0x7fbb0c0adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:04.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.647+0000 7fbb01400640 1 --2- 172.21.15.159:0/3913399060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb0c0cc880 0x7fbb0c0adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:04.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.647+0000 7fbb0a800640 1 --2- 172.21.15.159:0/3913399060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb0c0cc880 0x7fbb0c0adf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:04.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.647+0000 7fbb0a800640 1 -- 172.21.15.159:0/3913399060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb0c06dd30 msgr2=0x7fbb0c13e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:04.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.647+0000 7fbb0a800640 1 --2- 172.21.15.159:0/3913399060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb0c06dd30 0x7fbb0c13e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:04.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.647+0000 7fbb0a800640 1 -- 172.21.15.159:0/3913399060 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0c0b4370 con 0x7fbb0c13f190 2024-04-04T06:07:04.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.647+0000 7fbb09e00640 1 --2- 172.21.15.159:0/3913399060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb0c06dd30 0x7fbb0c13e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:04.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.648+0000 7fbb0a800640 1 --2- 172.21.15.159:0/3913399060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb0c13f190 0x7fbb0c0b0c70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbb0402b9f0 tx=0x7fbb040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:04.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.648+0000 7fbb02a00640 1 -- 172.21.15.159:0/3913399060 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04034040 con 0x7fbb0c13f190 2024-04-04T06:07:04.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.648+0000 7fbb02a00640 1 -- 172.21.15.159:0/3913399060 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb040433f0 con 0x7fbb0c13f190 2024-04-04T06:07:04.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.648+0000 7fbb02a00640 1 -- 172.21.15.159:0/3913399060 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb040437e0 con 0x7fbb0c13f190 2024-04-04T06:07:04.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.648+0000 7fbb0b200640 1 -- 172.21.15.159:0/3913399060 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0c0b4650 con 0x7fbb0c13f190 2024-04-04T06:07:04.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.648+0000 7fbb0b200640 1 -- 172.21.15.159:0/3913399060 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb0c0b4c10 con 0x7fbb0c13f190 2024-04-04T06:07:04.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.649+0000 7fbb0b200640 1 -- 172.21.15.159:0/3913399060 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb0c075940 con 0x7fbb0c13f190 2024-04-04T06:07:04.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.650+0000 7fbb02a00640 1 -- 172.21.15.159:0/3913399060 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb04037070 con 0x7fbb0c13f190 2024-04-04T06:07:04.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.651+0000 7fbb02a00640 1 --2- 172.21.15.159:0/3913399060 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbae806c8f0 0x7fbae806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:04.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.651+0000 7fbb02a00640 1 -- 172.21.15.159:0/3913399060 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fbb040adb50 con 0x7fbb0c13f190 2024-04-04T06:07:04.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.652+0000 7fbb09e00640 1 --2- 172.21.15.159:0/3913399060 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbae806c8f0 0x7fbae806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:04.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.653+0000 7fbb09e00640 1 --2- 172.21.15.159:0/3913399060 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbae806c8f0 0x7fbae806edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00d430 tx=0x7fbafc007530 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:04.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.655+0000 7fbb02a00640 1 -- 172.21.15.159:0/3913399060 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbb0403e070 con 0x7fbb0c13f190 2024-04-04T06:07:04.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.932+0000 7fbb0b200640 1 -- 172.21.15.159:0/3913399060 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fbb0c0b2870 con 0x7fbb0c13f190 2024-04-04T06:07:04.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.935+0000 7fbb02a00640 1 -- 172.21.15.159:0/3913399060 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fbb0407c2c0 con 0x7fbb0c13f190 2024-04-04T06:07:04.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.938+0000 7fbb0b200640 1 -- 172.21.15.159:0/3913399060 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbae806c8f0 msgr2=0x7fbae806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:04.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.938+0000 7fbb0b200640 1 --2- 172.21.15.159:0/3913399060 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbae806c8f0 0x7fbae806edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00d430 tx=0x7fbafc007530 comp rx=0 tx=0).stop 2024-04-04T06:07:04.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.938+0000 7fbb0b200640 1 -- 172.21.15.159:0/3913399060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb0c13f190 msgr2=0x7fbb0c0b0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:04.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.938+0000 7fbb0b200640 1 --2- 172.21.15.159:0/3913399060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb0c13f190 0x7fbb0c0b0c70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbb0402b9f0 tx=0x7fbb040046c0 comp rx=0 tx=0).stop 2024-04-04T06:07:04.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.938+0000 7fbb0b200640 1 -- 172.21.15.159:0/3913399060 shutdown_connections 2024-04-04T06:07:04.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.938+0000 7fbb0b200640 1 --2- 172.21.15.159:0/3913399060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb0c13f190 0x7fbb0c0b0c70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:04.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.938+0000 7fbb0b200640 1 --2- 172.21.15.159:0/3913399060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb0c0cc880 0x7fbb0c0adf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:04.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.938+0000 7fbb0b200640 1 --2- 172.21.15.159:0/3913399060 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbae806c8f0 0x7fbae806edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:04.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.938+0000 7fbb0b200640 1 --2- 172.21.15.159:0/3913399060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb0c06dd30 0x7fbb0c13e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:04.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.938+0000 7fbb0b200640 1 -- 172.21.15.159:0/3913399060 >> 172.21.15.159:0/3913399060 conn(0x7fbb0c0b1ba0 msgr2=0x7fbb0c0af880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:04.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.938+0000 7fbb0b200640 1 -- 172.21.15.159:0/3913399060 shutdown_connections 2024-04-04T06:07:04.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:04.938+0000 7fbb0b200640 1 -- 172.21.15.159:0/3913399060 wait complete. 2024-04-04T06:07:04.951 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:07:05.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.155+0000 7f3cbaa00640 1 Processor -- start 2024-04-04T06:07:05.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.155+0000 7f3cbaa00640 1 -- start start 2024-04-04T06:07:05.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.156+0000 7f3cbaa00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb40d0390 0x7f3cb40cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:05.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.157+0000 7f3cbaa00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3cb40ce1b0 0x7f3cb406da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:05.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.157+0000 7f3cbaa00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cb4161f30 con 0x7f3cb4147a40 2024-04-04T06:07:05.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.157+0000 7f3cbaa00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f3cb4162b00 con 0x7f3cb40ce1b0 2024-04-04T06:07:05.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.157+0000 7f3cbaa00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3cb41621d0 con 0x7f3cb40d0390 2024-04-04T06:07:05.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.157+0000 7f3cba000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3cb40ce1b0 0x7f3cb406da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:05.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.157+0000 7f3cba000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3cb40ce1b0 0x7f3cb406da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59338/0 (socket says 172.21.15.159:59338) 2024-04-04T06:07:05.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.157+0000 7f3cba000640 1 -- 172.21.15.159:0/2246616380 learned_addr learned my addr 172.21.15.159:0/2246616380 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:05.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.157+0000 7f3cba000640 1 -- 172.21.15.159:0/2246616380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb40d0390 msgr2=0x7f3cb40cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:05.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.157+0000 7f3cb8c00640 1 --2- 172.21.15.159:0/2246616380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb40d0390 0x7f3cb40cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:05.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.157+0000 7f3cba000640 1 --2- 172.21.15.159:0/2246616380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb40d0390 0x7f3cb40cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:05.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.157+0000 7f3cba000640 1 -- 172.21.15.159:0/2246616380 >> v1:172.21.15.178:6789/0 conn(0x7f3cb4147a40 legacy=0x7f3cb40cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:05.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.158+0000 7f3cba000640 1 -- 172.21.15.159:0/2246616380 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb41624b0 con 0x7f3cb40ce1b0 2024-04-04T06:07:05.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.158+0000 7f3cb8c00640 1 --2- 172.21.15.159:0/2246616380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb40d0390 0x7f3cb40cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:05.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.159+0000 7f3cba000640 1 --2- 172.21.15.159:0/2246616380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cb40ce1b0 0x7f3cb406da80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3cb40cc890 tx=0x7f3cac02be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831bfddfb9031322 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:05.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.159+0000 7f3cb3e00640 1 -- 172.21.15.159:0/2246616380 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac02ca30 con 0x7f3cb40ce1b0 2024-04-04T06:07:05.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.159+0000 7f3cb3e00640 1 -- 172.21.15.159:0/2246616380 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cac02cbd0 con 0x7f3cb40ce1b0 2024-04-04T06:07:05.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.159+0000 7f3cb3e00640 1 -- 172.21.15.159:0/2246616380 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac0343f0 con 0x7f3cb40ce1b0 2024-04-04T06:07:05.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.159+0000 7f3cbaa00640 1 -- 172.21.15.159:0/2246616380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cb40ce1b0 msgr2=0x7f3cb406da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:05.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.159+0000 7f3cbaa00640 1 --2- 172.21.15.159:0/2246616380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cb40ce1b0 0x7f3cb406da80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3cb40cc890 tx=0x7f3cac02be90 comp rx=0 tx=0).stop 2024-04-04T06:07:05.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.160+0000 7f3cbaa00640 1 -- 172.21.15.159:0/2246616380 shutdown_connections 2024-04-04T06:07:05.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.160+0000 7f3cbaa00640 1 --2- 172.21.15.159:0/2246616380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cb40ce1b0 0x7f3cb406da80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:05.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.160+0000 7f3cbaa00640 1 --2- 172.21.15.159:0/2246616380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb40d0390 0x7f3cb40cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:05.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.160+0000 7f3cbaa00640 1 -- 172.21.15.159:0/2246616380 >> 172.21.15.159:0/2246616380 conn(0x7f3cb40ccad0 msgr2=0x7f3cb40cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:05.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.160+0000 7f3cbaa00640 1 -- 172.21.15.159:0/2246616380 shutdown_connections 2024-04-04T06:07:05.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.160+0000 7f3cbaa00640 1 -- 172.21.15.159:0/2246616380 wait complete. 2024-04-04T06:07:05.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.161+0000 7f3cbaa00640 1 Processor -- start 2024-04-04T06:07:05.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.161+0000 7f3cbaa00640 1 -- start start 2024-04-04T06:07:05.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.162+0000 7f3cbaa00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb40ce1b0 0x7f3cb4153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:05.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.162+0000 7f3cbaa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cb40d0390 0x7f3cb413c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:05.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.162+0000 7f3cbaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cb4147a40 0x7f3cb413f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:05.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.162+0000 7f3cbaa00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb4147fa0 con 0x7f3cb4147a40 2024-04-04T06:07:05.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.162+0000 7f3cbaa00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb4147e20 con 0x7f3cb40d0390 2024-04-04T06:07:05.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.162+0000 7f3cbaa00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3cb40ce640 con 0x7f3cb40ce1b0 2024-04-04T06:07:05.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.163+0000 7f3cba000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cb4147a40 0x7f3cb413f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:05.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.163+0000 7f3cba000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cb4147a40 0x7f3cb413f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59348/0 (socket says 172.21.15.159:59348) 2024-04-04T06:07:05.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.163+0000 7f3cba000640 1 -- 172.21.15.159:0/171937630 learned_addr learned my addr 172.21.15.159:0/171937630 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:05.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.163+0000 7f3cb9600640 1 --2- 172.21.15.159:0/171937630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb40ce1b0 0x7f3cb4153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:05.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.163+0000 7f3cba000640 1 -- 172.21.15.159:0/171937630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb40ce1b0 msgr2=0x7f3cb4153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:05.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.163+0000 7f3cba000640 1 --2- 172.21.15.159:0/171937630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb40ce1b0 0x7f3cb4153a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:05.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.163+0000 7f3cba000640 1 -- 172.21.15.159:0/171937630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cb40d0390 msgr2=0x7f3cb413c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:05.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.163+0000 7f3cb8c00640 1 --2- 172.21.15.159:0/171937630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cb40d0390 0x7f3cb413c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:05.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.163+0000 7f3cba000640 1 --2- 172.21.15.159:0/171937630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cb40d0390 0x7f3cb413c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:05.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.163+0000 7f3cba000640 1 -- 172.21.15.159:0/171937630 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb4153f60 con 0x7f3cb4147a40 2024-04-04T06:07:05.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.163+0000 7f3cb8c00640 1 --2- 172.21.15.159:0/171937630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cb40d0390 0x7f3cb413c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:05.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.163+0000 7f3cba000640 1 --2- 172.21.15.159:0/171937630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cb4147a40 0x7f3cb413f090 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009f40 tx=0x7f3cac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:05.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.164+0000 7f3cb1600640 1 -- 172.21.15.159:0/171937630 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac043020 con 0x7f3cb4147a40 2024-04-04T06:07:05.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.164+0000 7f3cb1600640 1 -- 172.21.15.159:0/171937630 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cac03e450 con 0x7f3cb4147a40 2024-04-04T06:07:05.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.164+0000 7f3cb1600640 1 -- 172.21.15.159:0/171937630 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac03e820 con 0x7f3cb4147a40 2024-04-04T06:07:05.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.164+0000 7f3cbaa00640 1 -- 172.21.15.159:0/171937630 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb413d390 con 0x7f3cb4147a40 2024-04-04T06:07:05.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.164+0000 7f3cbaa00640 1 -- 172.21.15.159:0/171937630 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb413d950 con 0x7f3cb4147a40 2024-04-04T06:07:05.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.165+0000 7f3cbaa00640 1 -- 172.21.15.159:0/171937630 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cb4075940 con 0x7f3cb4147a40 2024-04-04T06:07:05.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.166+0000 7f3cb1600640 1 -- 172.21.15.159:0/171937630 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3cac046020 con 0x7f3cb4147a40 2024-04-04T06:07:05.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.167+0000 7f3cb1600640 1 --2- 172.21.15.159:0/171937630 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c9406c920 0x7f3c9406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:05.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.167+0000 7f3cb1600640 1 -- 172.21.15.159:0/171937630 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f3cac0adf70 con 0x7f3cb4147a40 2024-04-04T06:07:05.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.168+0000 7f3cb9600640 1 --2- 172.21.15.159:0/171937630 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c9406c920 0x7f3c9406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:05.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.169+0000 7f3cb9600640 1 --2- 172.21.15.159:0/171937630 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c9406c920 0x7f3c9406ede0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800a750 tx=0x7f3ca8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:05.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.172+0000 7f3cb1600640 1 -- 172.21.15.159:0/171937630 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3cac04d070 con 0x7f3cb4147a40 2024-04-04T06:07:05.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.372+0000 7f3cbaa00640 1 -- 172.21.15.159:0/171937630 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3cb40af6f0 con 0x7f3cb4147a40 2024-04-04T06:07:05.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.374+0000 7f3cb1600640 1 -- 172.21.15.159:0/171937630 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+12903 (secure 0 0 0) 0x7f3cac07c5c0 con 0x7f3cb4147a40 2024-04-04T06:07:05.368 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:05.368 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":30,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:07:01.296735+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:07:05.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.376+0000 7f3cbaa00640 1 -- 172.21.15.159:0/171937630 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c9406c920 msgr2=0x7f3c9406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:05.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.376+0000 7f3cbaa00640 1 --2- 172.21.15.159:0/171937630 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c9406c920 0x7f3c9406ede0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800a750 tx=0x7f3ca8008040 comp rx=0 tx=0).stop 2024-04-04T06:07:05.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.376+0000 7f3cbaa00640 1 -- 172.21.15.159:0/171937630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cb4147a40 msgr2=0x7f3cb413f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:05.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.376+0000 7f3cbaa00640 1 --2- 172.21.15.159:0/171937630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cb4147a40 0x7f3cb413f090 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009f40 tx=0x7f3cac004a40 comp rx=0 tx=0).stop 2024-04-04T06:07:05.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.376+0000 7f3cbaa00640 1 -- 172.21.15.159:0/171937630 shutdown_connections 2024-04-04T06:07:05.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.376+0000 7f3cbaa00640 1 --2- 172.21.15.159:0/171937630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cb4147a40 0x7f3cb413f090 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:05.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.376+0000 7f3cbaa00640 1 --2- 172.21.15.159:0/171937630 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c9406c920 0x7f3c9406ede0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:05.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.376+0000 7f3cbaa00640 1 --2- 172.21.15.159:0/171937630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3cb40d0390 0x7f3cb413c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:05.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.376+0000 7f3cbaa00640 1 --2- 172.21.15.159:0/171937630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3cb40ce1b0 0x7f3cb4153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:05.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.376+0000 7f3cbaa00640 1 -- 172.21.15.159:0/171937630 >> 172.21.15.159:0/171937630 conn(0x7f3cb40ccad0 msgr2=0x7f3cb41608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:05.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.377+0000 7f3cbaa00640 1 -- 172.21.15.159:0/171937630 shutdown_connections 2024-04-04T06:07:05.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.377+0000 7f3cbaa00640 1 -- 172.21.15.159:0/171937630 wait complete. 2024-04-04T06:07:05.389 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:07:05.389 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:07:05.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.600+0000 7fc6a3400640 1 Processor -- start 2024-04-04T06:07:05.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.600+0000 7fc6a3400640 1 -- start start 2024-04-04T06:07:05.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.601+0000 7fc6a3400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc69c162920 0x7fc69c1670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:05.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.601+0000 7fc6a3400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0d0990 0x7fc69c071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:05.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.601+0000 7fc6a3400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc69c071d60 con 0x7fc69c163ac0 2024-04-04T06:07:05.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.601+0000 7fc6a3400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fc69c163f20 con 0x7fc69c162920 2024-04-04T06:07:05.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.601+0000 7fc6a3400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc69c1665d0 con 0x7fc69c0d0990 2024-04-04T06:07:05.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.602+0000 7fc6a0a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc69c162920 0x7fc69c1670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:05.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.602+0000 7fc6a0a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc69c162920 0x7fc69c1670b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59364/0 (socket says 172.21.15.159:59364) 2024-04-04T06:07:05.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.602+0000 7fc6a0a00640 1 -- 172.21.15.159:0/1039253506 learned_addr learned my addr 172.21.15.159:0/1039253506 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:05.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.602+0000 7fc6a1400640 1 --2- 172.21.15.159:0/1039253506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0d0990 0x7fc69c071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:05.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.602+0000 7fc6a0a00640 1 -- 172.21.15.159:0/1039253506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0d0990 msgr2=0x7fc69c071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:05.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.602+0000 7fc6a0a00640 1 --2- 172.21.15.159:0/1039253506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0d0990 0x7fc69c071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:05.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.602+0000 7fc6a0a00640 1 -- 172.21.15.159:0/1039253506 >> v1:172.21.15.178:6789/0 conn(0x7fc69c163ac0 legacy=0x7fc69c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:05.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.602+0000 7fc6a0a00640 1 -- 172.21.15.159:0/1039253506 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c1668b0 con 0x7fc69c162920 2024-04-04T06:07:05.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.602+0000 7fc6a1400640 1 --2- 172.21.15.159:0/1039253506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0d0990 0x7fc69c071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:05.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.603+0000 7fc6a0a00640 1 --2- 172.21.15.159:0/1039253506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc69c162920 0x7fc69c1670b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0d0dd0 tx=0x7fc68402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4dd5b7a2de64e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.604+0000 7fc69be00640 1 -- 172.21.15.159:0/1039253506 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6840315a0 con 0x7fc69c162920 2024-04-04T06:07:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.604+0000 7fc69be00640 1 -- 172.21.15.159:0/1039253506 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc684037070 con 0x7fc69c162920 2024-04-04T06:07:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.604+0000 7fc69be00640 1 -- 172.21.15.159:0/1039253506 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684031960 con 0x7fc69c162920 2024-04-04T06:07:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.604+0000 7fc6a3400640 1 -- 172.21.15.159:0/1039253506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc69c162920 msgr2=0x7fc69c1670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.604+0000 7fc6a3400640 1 --2- 172.21.15.159:0/1039253506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc69c162920 0x7fc69c1670b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0d0dd0 tx=0x7fc68402b840 comp rx=0 tx=0).stop 2024-04-04T06:07:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.604+0000 7fc6a3400640 1 -- 172.21.15.159:0/1039253506 shutdown_connections 2024-04-04T06:07:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.604+0000 7fc6a3400640 1 --2- 172.21.15.159:0/1039253506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0d0990 0x7fc69c071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.604+0000 7fc6a3400640 1 --2- 172.21.15.159:0/1039253506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc69c162920 0x7fc69c1670b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:05.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.604+0000 7fc6a3400640 1 -- 172.21.15.159:0/1039253506 >> 172.21.15.159:0/1039253506 conn(0x7fc69c06b750 msgr2=0x7fc69c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:05.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.604+0000 7fc6a3400640 1 -- 172.21.15.159:0/1039253506 shutdown_connections 2024-04-04T06:07:05.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.605+0000 7fc6a3400640 1 -- 172.21.15.159:0/1039253506 wait complete. 2024-04-04T06:07:05.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.606+0000 7fc6a3400640 1 Processor -- start 2024-04-04T06:07:05.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.606+0000 7fc6a3400640 1 -- start start 2024-04-04T06:07:05.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.607+0000 7fc6a3400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0d0990 0x7fc69c15f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:05.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.607+0000 7fc6a3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc69c162920 0x7fc69c1f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:05.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.607+0000 7fc6a3400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc69c163ac0 0x7fc69c1ed510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:05.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.607+0000 7fc6a3400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c072040 con 0x7fc69c162920 2024-04-04T06:07:05.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.607+0000 7fc6a3400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c162d00 con 0x7fc69c163ac0 2024-04-04T06:07:05.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.607+0000 7fc6a3400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc69c0721c0 con 0x7fc69c0d0990 2024-04-04T06:07:05.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.608+0000 7fc693e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc69c162920 0x7fc69c1f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:05.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.608+0000 7fc693e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc69c162920 0x7fc69c1f86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59368/0 (socket says 172.21.15.159:59368) 2024-04-04T06:07:05.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.608+0000 7fc693e00640 1 -- 172.21.15.159:0/1428019476 learned_addr learned my addr 172.21.15.159:0/1428019476 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:05.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.608+0000 7fc6a0a00640 1 --2- 172.21.15.159:0/1428019476 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0d0990 0x7fc69c15f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:05.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.608+0000 7fc693e00640 1 -- 172.21.15.159:0/1428019476 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0d0990 msgr2=0x7fc69c15f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:05.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.608+0000 7fc6a1400640 1 --2- 172.21.15.159:0/1428019476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc69c163ac0 0x7fc69c1ed510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:05.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.608+0000 7fc693e00640 1 --2- 172.21.15.159:0/1428019476 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0d0990 0x7fc69c15f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:05.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.608+0000 7fc693e00640 1 -- 172.21.15.159:0/1428019476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc69c163ac0 msgr2=0x7fc69c1ed510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:05.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.608+0000 7fc693e00640 1 --2- 172.21.15.159:0/1428019476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc69c163ac0 0x7fc69c1ed510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:05.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.608+0000 7fc693e00640 1 -- 172.21.15.159:0/1428019476 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c09a2c0 con 0x7fc69c162920 2024-04-04T06:07:05.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.608+0000 7fc6a0a00640 1 --2- 172.21.15.159:0/1428019476 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0d0990 0x7fc69c15f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:05.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.608+0000 7fc6a1400640 1 --2- 172.21.15.159:0/1428019476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc69c163ac0 0x7fc69c1ed510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:05.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.608+0000 7fc693e00640 1 --2- 172.21.15.159:0/1428019476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc69c162920 0x7fc69c1f86d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00c430 tx=0x7fc68c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:05.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.609+0000 7fc699600640 1 -- 172.21.15.159:0/1428019476 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c011790 con 0x7fc69c162920 2024-04-04T06:07:05.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.609+0000 7fc699600640 1 -- 172.21.15.159:0/1428019476 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc68c017070 con 0x7fc69c162920 2024-04-04T06:07:05.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.609+0000 7fc6a3400640 1 -- 172.21.15.159:0/1428019476 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69c09a5a0 con 0x7fc69c162920 2024-04-04T06:07:05.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.609+0000 7fc699600640 1 -- 172.21.15.159:0/1428019476 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c011bb0 con 0x7fc69c162920 2024-04-04T06:07:05.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.609+0000 7fc6a3400640 1 -- 172.21.15.159:0/1428019476 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc69c09c010 con 0x7fc69c162920 2024-04-04T06:07:05.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.610+0000 7fc6a3400640 1 -- 172.21.15.159:0/1428019476 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc69c0685c0 con 0x7fc69c162920 2024-04-04T06:07:05.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.611+0000 7fc699600640 1 -- 172.21.15.159:0/1428019476 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc68c01c460 con 0x7fc69c162920 2024-04-04T06:07:05.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.612+0000 7fc699600640 1 --2- 172.21.15.159:0/1428019476 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc67806c8f0 0x7fc67806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:05.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.612+0000 7fc699600640 1 -- 172.21.15.159:0/1428019476 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fc68c08b610 con 0x7fc69c162920 2024-04-04T06:07:05.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.612+0000 7fc6a0a00640 1 --2- 172.21.15.159:0/1428019476 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc67806c8f0 0x7fc67806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:05.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.613+0000 7fc6a0a00640 1 --2- 172.21.15.159:0/1428019476 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc67806c8f0 0x7fc67806edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc684036000 tx=0x7fc684008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:05.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.616+0000 7fc699600640 1 -- 172.21.15.159:0/1428019476 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc68c059e20 con 0x7fc69c162920 2024-04-04T06:07:05.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.801+0000 7fc6a3400640 1 -- 172.21.15.159:0/1428019476 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc69c06e610 con 0x7fc69c162920 2024-04-04T06:07:05.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.802+0000 7fc699600640 1 -- 172.21.15.159:0/1428019476 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) ==== 93+0+10 (secure 0 0 0) 0x7fc68c059720 con 0x7fc69c162920 2024-04-04T06:07:05.796 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:07:05.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.804+0000 7fc6a3400640 1 -- 172.21.15.159:0/1428019476 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc67806c8f0 msgr2=0x7fc67806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:05.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.804+0000 7fc6a3400640 1 --2- 172.21.15.159:0/1428019476 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc67806c8f0 0x7fc67806edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc684036000 tx=0x7fc684008030 comp rx=0 tx=0).stop 2024-04-04T06:07:05.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.804+0000 7fc6a3400640 1 -- 172.21.15.159:0/1428019476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc69c162920 msgr2=0x7fc69c1f86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:05.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.804+0000 7fc6a3400640 1 --2- 172.21.15.159:0/1428019476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc69c162920 0x7fc69c1f86d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00c430 tx=0x7fc68c00c8f0 comp rx=0 tx=0).stop 2024-04-04T06:07:05.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.804+0000 7fc6a3400640 1 -- 172.21.15.159:0/1428019476 shutdown_connections 2024-04-04T06:07:05.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.804+0000 7fc6a3400640 1 --2- 172.21.15.159:0/1428019476 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc67806c8f0 0x7fc67806edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:05.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.804+0000 7fc6a3400640 1 --2- 172.21.15.159:0/1428019476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc69c163ac0 0x7fc69c1ed510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:05.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.804+0000 7fc6a3400640 1 --2- 172.21.15.159:0/1428019476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc69c162920 0x7fc69c1f86d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:05.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.804+0000 7fc6a3400640 1 --2- 172.21.15.159:0/1428019476 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc69c0d0990 0x7fc69c15f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:05.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.804+0000 7fc6a3400640 1 -- 172.21.15.159:0/1428019476 >> 172.21.15.159:0/1428019476 conn(0x7fc69c06b750 msgr2=0x7fc69c1640e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:05.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.805+0000 7fc6a3400640 1 -- 172.21.15.159:0/1428019476 shutdown_connections 2024-04-04T06:07:05.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:05.805+0000 7fc6a3400640 1 -- 172.21.15.159:0/1428019476 wait complete. 2024-04-04T06:07:05.817 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-04T06:07:06.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.026+0000 7fb0e8a00640 1 Processor -- start 2024-04-04T06:07:06.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.026+0000 7fb0e8a00640 1 -- start start 2024-04-04T06:07:06.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.027+0000 7fb0e8a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb0e4153580 0x7fb0e40d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:06.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.027+0000 7fb0e8a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0e40af4b0 0x7fb0e41600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:06.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.027+0000 7fb0e8a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0e40afab0 con 0x7fb0e40d07c0 2024-04-04T06:07:06.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.027+0000 7fb0e8a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fb0e40d0c30 con 0x7fb0e4153580 2024-04-04T06:07:06.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.027+0000 7fb0e8a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb0e415cc80 con 0x7fb0e40af4b0 2024-04-04T06:07:06.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.028+0000 7fb0d9400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fb0e40d07c0 0x7fb0e40aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52414/0 (socket says 172.21.15.159:52414) 2024-04-04T06:07:06.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.028+0000 7fb0d9400640 1 -- 172.21.15.159:0/3241003158 learned_addr learned my addr 172.21.15.159:0/3241003158 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:06.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.028+0000 7fb0e1e00640 1 --2- 172.21.15.159:0/3241003158 >> v2:172.21.15.159:3300/0 conn(0x7fb0e4153580 0x7fb0e40d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:06.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.028+0000 7fb0e2800640 1 --2- 172.21.15.159:0/3241003158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0e40af4b0 0x7fb0e41600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:06.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.028+0000 7fb0e1400640 1 -- 172.21.15.159:0/3241003158 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 799298736 0 0) 0x7fb0e40afab0 con 0x7fb0e40d07c0 2024-04-04T06:07:06.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.028+0000 7fb0e1400640 1 -- 172.21.15.159:0/3241003158 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0c8003520 con 0x7fb0e40d07c0 2024-04-04T06:07:06.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.028+0000 7fb0e1e00640 1 -- 172.21.15.159:0/3241003158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0e40af4b0 msgr2=0x7fb0e41600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:06.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.028+0000 7fb0e1e00640 1 --2- 172.21.15.159:0/3241003158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0e40af4b0 0x7fb0e41600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:06.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.028+0000 7fb0e1e00640 1 -- 172.21.15.159:0/3241003158 >> v1:172.21.15.159:6789/0 conn(0x7fb0e40d07c0 legacy=0x7fb0e40aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:06.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.028+0000 7fb0e1e00640 1 -- 172.21.15.159:0/3241003158 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e415da00 con 0x7fb0e4153580 2024-04-04T06:07:06.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.030+0000 7fb0e1e00640 1 --2- 172.21.15.159:0/3241003158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e4153580 0x7fb0e40d0280 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb0d40097a0 tx=0x7fb0d402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b079fc63ef6ce31e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:06.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.030+0000 7fb0e1400640 1 -- 172.21.15.159:0/3241003158 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4031780 con 0x7fb0e4153580 2024-04-04T06:07:06.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.030+0000 7fb0e1400640 1 -- 172.21.15.159:0/3241003158 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0d4037070 con 0x7fb0e4153580 2024-04-04T06:07:06.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.030+0000 7fb0e1400640 1 -- 172.21.15.159:0/3241003158 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4031b40 con 0x7fb0e4153580 2024-04-04T06:07:06.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.031+0000 7fb0e8a00640 1 -- 172.21.15.159:0/3241003158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e4153580 msgr2=0x7fb0e40d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:06.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.031+0000 7fb0e8a00640 1 --2- 172.21.15.159:0/3241003158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e4153580 0x7fb0e40d0280 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb0d40097a0 tx=0x7fb0d402b820 comp rx=0 tx=0).stop 2024-04-04T06:07:06.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.031+0000 7fb0e8a00640 1 -- 172.21.15.159:0/3241003158 shutdown_connections 2024-04-04T06:07:06.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.031+0000 7fb0e8a00640 1 --2- 172.21.15.159:0/3241003158 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0e40af4b0 0x7fb0e41600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:06.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.031+0000 7fb0e8a00640 1 --2- 172.21.15.159:0/3241003158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e4153580 0x7fb0e40d0280 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:06.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.031+0000 7fb0e8a00640 1 -- 172.21.15.159:0/3241003158 >> 172.21.15.159:0/3241003158 conn(0x7fb0e406b750 msgr2=0x7fb0e406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:06.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.031+0000 7fb0e8a00640 1 -- 172.21.15.159:0/3241003158 shutdown_connections 2024-04-04T06:07:06.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.031+0000 7fb0e8a00640 1 -- 172.21.15.159:0/3241003158 wait complete. 2024-04-04T06:07:06.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.033+0000 7fb0e8a00640 1 Processor -- start 2024-04-04T06:07:06.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.033+0000 7fb0e8a00640 1 -- start start 2024-04-04T06:07:06.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.034+0000 7fb0e8a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0e40af4b0 0x7fb0e40ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:06.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.034+0000 7fb0e8a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0e40d07c0 0x7fb0e414f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:06.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.034+0000 7fb0e8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e4153580 0x7fb0e40b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:06.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.034+0000 7fb0e8a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e415d4e0 con 0x7fb0e4153580 2024-04-04T06:07:06.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.034+0000 7fb0e8a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e415d200 con 0x7fb0e40af4b0 2024-04-04T06:07:06.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.034+0000 7fb0e8a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb0e4153960 con 0x7fb0e40d07c0 2024-04-04T06:07:06.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.034+0000 7fb0e2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e4153580 0x7fb0e40b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:06.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.034+0000 7fb0e2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e4153580 0x7fb0e40b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59392/0 (socket says 172.21.15.159:59392) 2024-04-04T06:07:06.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.034+0000 7fb0e2800640 1 -- 172.21.15.159:0/2365733164 learned_addr learned my addr 172.21.15.159:0/2365733164 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:06.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.035+0000 7fb0d9400640 1 --2- 172.21.15.159:0/2365733164 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0e40d07c0 0x7fb0e414f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:06.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.035+0000 7fb0e2800640 1 -- 172.21.15.159:0/2365733164 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0e40d07c0 msgr2=0x7fb0e414f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:06.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.035+0000 7fb0e1e00640 1 --2- 172.21.15.159:0/2365733164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0e40af4b0 0x7fb0e40ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:06.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.035+0000 7fb0e2800640 1 --2- 172.21.15.159:0/2365733164 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0e40d07c0 0x7fb0e414f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:06.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.035+0000 7fb0e2800640 1 -- 172.21.15.159:0/2365733164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0e40af4b0 msgr2=0x7fb0e40ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:06.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.035+0000 7fb0e2800640 1 --2- 172.21.15.159:0/2365733164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0e40af4b0 0x7fb0e40ca810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:06.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.035+0000 7fb0e2800640 1 -- 172.21.15.159:0/2365733164 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e40cad50 con 0x7fb0e4153580 2024-04-04T06:07:06.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.035+0000 7fb0e2800640 1 --2- 172.21.15.159:0/2365733164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e4153580 0x7fb0e40b3a80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009d30 tx=0x7fb0dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:06.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.035+0000 7fb0daa00640 1 -- 172.21.15.159:0/2365733164 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc018020 con 0x7fb0e4153580 2024-04-04T06:07:06.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.035+0000 7fb0daa00640 1 -- 172.21.15.159:0/2365733164 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0dc014480 con 0x7fb0e4153580 2024-04-04T06:07:06.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.035+0000 7fb0daa00640 1 -- 172.21.15.159:0/2365733164 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc014870 con 0x7fb0e4153580 2024-04-04T06:07:06.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.035+0000 7fb0e8a00640 1 -- 172.21.15.159:0/2365733164 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e40b4520 con 0x7fb0e4153580 2024-04-04T06:07:06.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.035+0000 7fb0e8a00640 1 -- 172.21.15.159:0/2365733164 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e40b4ae0 con 0x7fb0e4153580 2024-04-04T06:07:06.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.037+0000 7fb0e8a00640 1 -- 172.21.15.159:0/2365733164 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0e40685c0 con 0x7fb0e4153580 2024-04-04T06:07:06.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.037+0000 7fb0daa00640 1 -- 172.21.15.159:0/2365733164 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0dc005d10 con 0x7fb0e4153580 2024-04-04T06:07:06.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.038+0000 7fb0daa00640 1 --2- 172.21.15.159:0/2365733164 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb0c006c8f0 0x7fb0c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:06.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.038+0000 7fb0daa00640 1 -- 172.21.15.159:0/2365733164 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fb0dc08c220 con 0x7fb0e4153580 2024-04-04T06:07:06.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.039+0000 7fb0e1e00640 1 --2- 172.21.15.159:0/2365733164 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb0c006c8f0 0x7fb0c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:06.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.040+0000 7fb0e1e00640 1 --2- 172.21.15.159:0/2365733164 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb0c006c8f0 0x7fb0c006edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4036000 tx=0x7fb0d4008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:06.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.044+0000 7fb0daa00640 1 -- 172.21.15.159:0/2365733164 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb0dc022070 con 0x7fb0e4153580 2024-04-04T06:07:06.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.245+0000 7fb0e8a00640 1 -- 172.21.15.159:0/2365733164 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb0e406eca0 con 0x7fb0e4153580 2024-04-04T06:07:06.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.246+0000 7fb0daa00640 1 -- 172.21.15.159:0/2365733164 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) ==== 84+0+65 (secure 0 0 0) 0x7fb0dc010040 con 0x7fb0e4153580 2024-04-04T06:07:06.241 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:06.241 INFO:teuthology.orchestra.run.smithi159.stdout:[ 2024-04-04T06:07:06.241 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:06.241 INFO:teuthology.orchestra.run.smithi159.stdout: "poolnum": 1, 2024-04-04T06:07:06.241 INFO:teuthology.orchestra.run.smithi159.stdout: "poolname": ".mgr" 2024-04-04T06:07:06.241 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:06.241 INFO:teuthology.orchestra.run.smithi159.stdout:] 2024-04-04T06:07:06.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.248+0000 7fb0e8a00640 1 -- 172.21.15.159:0/2365733164 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb0c006c8f0 msgr2=0x7fb0c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:06.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.248+0000 7fb0e8a00640 1 --2- 172.21.15.159:0/2365733164 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb0c006c8f0 0x7fb0c006edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4036000 tx=0x7fb0d4008030 comp rx=0 tx=0).stop 2024-04-04T06:07:06.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.248+0000 7fb0e8a00640 1 -- 172.21.15.159:0/2365733164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e4153580 msgr2=0x7fb0e40b3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:06.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.248+0000 7fb0e8a00640 1 --2- 172.21.15.159:0/2365733164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e4153580 0x7fb0e40b3a80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009d30 tx=0x7fb0dc00c490 comp rx=0 tx=0).stop 2024-04-04T06:07:06.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.249+0000 7fb0e8a00640 1 -- 172.21.15.159:0/2365733164 shutdown_connections 2024-04-04T06:07:06.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.249+0000 7fb0e8a00640 1 --2- 172.21.15.159:0/2365733164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e4153580 0x7fb0e40b3a80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:06.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.249+0000 7fb0e8a00640 1 --2- 172.21.15.159:0/2365733164 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0e40d07c0 0x7fb0e414f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:06.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.249+0000 7fb0e8a00640 1 --2- 172.21.15.159:0/2365733164 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb0c006c8f0 0x7fb0c006edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:06.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.249+0000 7fb0e8a00640 1 --2- 172.21.15.159:0/2365733164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0e40af4b0 0x7fb0e40ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:06.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.249+0000 7fb0e8a00640 1 -- 172.21.15.159:0/2365733164 >> 172.21.15.159:0/2365733164 conn(0x7fb0e406b750 msgr2=0x7fb0e40b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:06.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.249+0000 7fb0e8a00640 1 -- 172.21.15.159:0/2365733164 shutdown_connections 2024-04-04T06:07:06.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.249+0000 7fb0e8a00640 1 -- 172.21.15.159:0/2365733164 wait complete. 2024-04-04T06:07:06.263 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-04T06:07:06.263 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-04T06:07:06.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.474+0000 7f44cb000640 1 Processor -- start 2024-04-04T06:07:06.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.474+0000 7f44cb000640 1 -- start start 2024-04-04T06:07:06.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.475+0000 7f44cb000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f44c40d0d90 0x7f44c40d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:06.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.476+0000 7f44cb000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44c415ea70 con 0x7f44c406c680 2024-04-04T06:07:06.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.476+0000 7f44cb000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44c4071060 con 0x7f44c415e690 2024-04-04T06:07:06.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.476+0000 7f44cb000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f44c4071340 con 0x7f44c40d0d90 2024-04-04T06:07:06.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.476+0000 7f44c9000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f44c406c680 0x7f44c406ca90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52416/0 (socket says 172.21.15.159:52416) 2024-04-04T06:07:06.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.476+0000 7f44c9000640 1 -- 172.21.15.159:0/1555906949 learned_addr learned my addr 172.21.15.159:0/1555906949 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:06.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.476+0000 7f44c3400640 1 --2- 172.21.15.159:0/1555906949 >> v2:172.21.15.178:3300/0 conn(0x7f44c40d0d90 0x7f44c40d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:06.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.476+0000 7f44c2a00640 1 -- 172.21.15.159:0/1555906949 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 16720006 0 0) 0x7f44c415ea70 con 0x7f44c406c680 2024-04-04T06:07:06.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.476+0000 7f44c2a00640 1 -- 172.21.15.159:0/1555906949 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44a8003520 con 0x7f44c406c680 2024-04-04T06:07:06.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.477+0000 7f44c2a00640 1 -- 172.21.15.159:0/1555906949 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3504909238 0 0) 0x7f44a8003520 con 0x7f44c406c680 2024-04-04T06:07:06.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.477+0000 7f44c2a00640 1 -- 172.21.15.159:0/1555906949 >> v2:172.21.15.178:3300/0 conn(0x7f44c40d0d90 msgr2=0x7f44c40d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:06.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.477+0000 7f44c2a00640 1 --2- 172.21.15.159:0/1555906949 >> v2:172.21.15.178:3300/0 conn(0x7f44c40d0d90 0x7f44c40d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:06.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.477+0000 7f44c2a00640 1 -- 172.21.15.159:0/1555906949 >> v1:172.21.15.178:6789/0 conn(0x7f44c415e690 legacy=0x7f44c40b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:06.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.477+0000 7f44c2a00640 1 -- 172.21.15.159:0/1555906949 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c4071620 con 0x7f44c406c680 2024-04-04T06:07:06.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.477+0000 7f44c2a00640 1 -- 172.21.15.159:0/1555906949 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f44bc0019f0 con 0x7f44c406c680 2024-04-04T06:07:06.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.477+0000 7f44c3400640 1 --2- 172.21.15.159:0/1555906949 >> v2:172.21.15.178:3300/0 conn(0x7f44c40d0d90 0x7f44c40d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:07:06.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.477+0000 7f44c2a00640 1 -- 172.21.15.159:0/1555906949 >> v1:172.21.15.159:6789/0 conn(0x7f44c406c680 legacy=0x7f44c406ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:06.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.477+0000 7f44c2a00640 1 --2- 172.21.15.159:0/1555906949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44a80039b0 0x7f44a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:06.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.477+0000 7f44c2a00640 1 --2- 172.21.15.159:0/1555906949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44a8004800 0x7f44a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:06.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.477+0000 7f44c2a00640 1 --2- 172.21.15.159:0/1555906949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44a80051c0 0x7f44a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:06.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.477+0000 7f44c2a00640 1 -- 172.21.15.159:0/1555906949 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4071340 con 0x7f44a80051c0 2024-04-04T06:07:06.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.477+0000 7f44c2a00640 1 -- 172.21.15.159:0/1555906949 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f44a800c110 con 0x7f44a8004800 2024-04-04T06:07:06.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.477+0000 7f44c2a00640 1 -- 172.21.15.159:0/1555906949 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f44a800c310 con 0x7f44a80039b0 2024-04-04T06:07:06.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.477+0000 7f44c9000640 1 --2- 172.21.15.159:0/1555906949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44a80051c0 0x7f44a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:06.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.477+0000 7f44c3e00640 1 --2- 172.21.15.159:0/1555906949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44a80039b0 0x7f44a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:06.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.478+0000 7f44c3400640 1 --2- 172.21.15.159:0/1555906949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44a8004800 0x7f44a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:06.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.478+0000 7f44c9000640 1 -- 172.21.15.159:0/1555906949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44a80039b0 msgr2=0x7f44a8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:06.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.478+0000 7f44c9000640 1 --2- 172.21.15.159:0/1555906949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44a80039b0 0x7f44a8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:06.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.478+0000 7f44c9000640 1 -- 172.21.15.159:0/1555906949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44a8004800 msgr2=0x7f44a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:06.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.478+0000 7f44c9000640 1 --2- 172.21.15.159:0/1555906949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44a8004800 0x7f44a8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:06.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.478+0000 7f44c9000640 1 -- 172.21.15.159:0/1555906949 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a8003520 con 0x7f44a80051c0 2024-04-04T06:07:06.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.479+0000 7f44c9000640 1 -- 172.21.15.159:0/1555906949 reap_dead start 2024-04-04T06:07:06.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.479+0000 7f44c9000640 1 --2- 172.21.15.159:0/1555906949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44a80051c0 0x7f44a800b9c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00a850 tx=0x7f44a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3e29ae794cb690 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:06.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.479+0000 7f44c2a00640 1 -- 172.21.15.159:0/1555906949 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc00a4a0 con 0x7f44a80051c0 2024-04-04T06:07:06.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.479+0000 7f44c2a00640 1 -- 172.21.15.159:0/1555906949 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44bc034650 con 0x7f44a80051c0 2024-04-04T06:07:06.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.479+0000 7f44c2a00640 1 -- 172.21.15.159:0/1555906949 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc034a40 con 0x7f44a80051c0 2024-04-04T06:07:06.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.480+0000 7f44cb000640 1 -- 172.21.15.159:0/1555906949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44a80051c0 msgr2=0x7f44a800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:06.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.480+0000 7f44cb000640 1 --2- 172.21.15.159:0/1555906949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44a80051c0 0x7f44a800b9c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00a850 tx=0x7f44a8005830 comp rx=0 tx=0).stop 2024-04-04T06:07:06.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.480+0000 7f44cb000640 1 -- 172.21.15.159:0/1555906949 shutdown_connections 2024-04-04T06:07:06.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.480+0000 7f44cb000640 1 --2- 172.21.15.159:0/1555906949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44a80051c0 0x7f44a800b9c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:06.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.480+0000 7f44cb000640 1 -- 172.21.15.159:0/1555906949 >> 172.21.15.159:0/1555906949 conn(0x7f44c40d2240 msgr2=0x7f44c40d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:06.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.480+0000 7f44cb000640 1 -- 172.21.15.159:0/1555906949 shutdown_connections 2024-04-04T06:07:06.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.480+0000 7f44cb000640 1 -- 172.21.15.159:0/1555906949 wait complete. 2024-04-04T06:07:06.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.481+0000 7f44cb000640 1 Processor -- start 2024-04-04T06:07:06.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.482+0000 7f44cb000640 1 -- start start 2024-04-04T06:07:06.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.482+0000 7f44cb000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44a80051c0 0x7f44c415da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:06.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.482+0000 7f44cb000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44c41463b0 0x7f44c40b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:06.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.482+0000 7f44cb000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c41e6db0 0x7f44c40b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:06.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.482+0000 7f44cb000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f44c40d15d0 con 0x7f44c41e6db0 2024-04-04T06:07:06.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.482+0000 7f44cb000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4071be0 con 0x7f44a80051c0 2024-04-04T06:07:06.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.482+0000 7f44cb000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f44c40d1750 con 0x7f44c41463b0 2024-04-04T06:07:06.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.483+0000 7f44c9000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c41e6db0 0x7f44c40b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:06.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.483+0000 7f44c9000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c41e6db0 0x7f44c40b5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59416/0 (socket says 172.21.15.159:59416) 2024-04-04T06:07:06.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.483+0000 7f44c9000640 1 -- 172.21.15.159:0/2085215651 learned_addr learned my addr 172.21.15.159:0/2085215651 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:06.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.483+0000 7f44c3e00640 1 --2- 172.21.15.159:0/2085215651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44a80051c0 0x7f44c415da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:06.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.483+0000 7f44c9000640 1 -- 172.21.15.159:0/2085215651 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44c41463b0 msgr2=0x7f44c40b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:06.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.483+0000 7f44c9000640 1 --2- 172.21.15.159:0/2085215651 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44c41463b0 0x7f44c40b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:06.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.483+0000 7f44c9000640 1 -- 172.21.15.159:0/2085215651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44a80051c0 msgr2=0x7f44c415da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:06.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.483+0000 7f44c9000640 1 --2- 172.21.15.159:0/2085215651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44a80051c0 0x7f44c415da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:06.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.483+0000 7f44c9000640 1 -- 172.21.15.159:0/2085215651 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c4145350 con 0x7f44c41e6db0 2024-04-04T06:07:06.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.483+0000 7f44c9000640 1 --2- 172.21.15.159:0/2085215651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c41e6db0 0x7f44c40b5af0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f44bc001cf0 tx=0x7f44bc001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:06.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.484+0000 7f44bbe00640 1 -- 172.21.15.159:0/2085215651 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc039040 con 0x7f44c41e6db0 2024-04-04T06:07:06.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.484+0000 7f44bbe00640 1 -- 172.21.15.159:0/2085215651 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44bc043070 con 0x7f44c41e6db0 2024-04-04T06:07:06.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.484+0000 7f44bbe00640 1 -- 172.21.15.159:0/2085215651 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc004b60 con 0x7f44c41e6db0 2024-04-04T06:07:06.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.484+0000 7f44cb000640 1 -- 172.21.15.159:0/2085215651 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c40b34a0 con 0x7f44c41e6db0 2024-04-04T06:07:06.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.484+0000 7f44cb000640 1 -- 172.21.15.159:0/2085215651 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c41458d0 con 0x7f44c41e6db0 2024-04-04T06:07:06.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.485+0000 7f44cb000640 1 -- 172.21.15.159:0/2085215651 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44c40685c0 con 0x7f44c41e6db0 2024-04-04T06:07:06.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.486+0000 7f44bbe00640 1 -- 172.21.15.159:0/2085215651 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f44bc004d00 con 0x7f44c41e6db0 2024-04-04T06:07:06.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.487+0000 7f44bbe00640 1 --2- 172.21.15.159:0/2085215651 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f44a406c8f0 0x7f44a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:06.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.487+0000 7f44bbe00640 1 -- 172.21.15.159:0/2085215651 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f44bc0ad5a0 con 0x7f44c41e6db0 2024-04-04T06:07:06.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.487+0000 7f44c3e00640 1 --2- 172.21.15.159:0/2085215651 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f44a406c8f0 0x7f44a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:06.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.489+0000 7f44c3e00640 1 --2- 172.21.15.159:0/2085215651 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f44a406c8f0 0x7f44a406edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f44b4007fd0 tx=0x7f44b4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:06.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.492+0000 7f44bbe00640 1 -- 172.21.15.159:0/2085215651 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f44bc07bd10 con 0x7f44c41e6db0 2024-04-04T06:07:06.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:06.692+0000 7f44cb000640 1 -- 172.21.15.159:0/2085215651 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f44c406e890 con 0x7f44c41e6db0 2024-04-04T06:07:07.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.350+0000 7f44bbe00640 1 -- 172.21.15.159:0/2085215651 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) ==== 138+0+0 (secure 0 0 0) 0x7f44bc07b610 con 0x7f44c41e6db0 2024-04-04T06:07:07.345 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_metadata' created 2024-04-04T06:07:07.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.353+0000 7f44cb000640 1 -- 172.21.15.159:0/2085215651 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f44a406c8f0 msgr2=0x7f44a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:07.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.353+0000 7f44cb000640 1 --2- 172.21.15.159:0/2085215651 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f44a406c8f0 0x7f44a406edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f44b4007fd0 tx=0x7f44b4009040 comp rx=0 tx=0).stop 2024-04-04T06:07:07.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.353+0000 7f44cb000640 1 -- 172.21.15.159:0/2085215651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c41e6db0 msgr2=0x7f44c40b5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:07.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.353+0000 7f44cb000640 1 --2- 172.21.15.159:0/2085215651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c41e6db0 0x7f44c40b5af0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f44bc001cf0 tx=0x7f44bc001dc0 comp rx=0 tx=0).stop 2024-04-04T06:07:07.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.353+0000 7f44cb000640 1 -- 172.21.15.159:0/2085215651 shutdown_connections 2024-04-04T06:07:07.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.353+0000 7f44cb000640 1 --2- 172.21.15.159:0/2085215651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44c41e6db0 0x7f44c40b5af0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:07.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.353+0000 7f44cb000640 1 --2- 172.21.15.159:0/2085215651 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44c41463b0 0x7f44c40b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:07.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.353+0000 7f44cb000640 1 --2- 172.21.15.159:0/2085215651 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f44a406c8f0 0x7f44a406edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:07.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.353+0000 7f44cb000640 1 --2- 172.21.15.159:0/2085215651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44a80051c0 0x7f44c415da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:07.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.353+0000 7f44cb000640 1 -- 172.21.15.159:0/2085215651 >> 172.21.15.159:0/2085215651 conn(0x7f44c40d2240 msgr2=0x7f44c40b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:07.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.354+0000 7f44cb000640 1 -- 172.21.15.159:0/2085215651 shutdown_connections 2024-04-04T06:07:07.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.354+0000 7f44cb000640 1 -- 172.21.15.159:0/2085215651 wait complete. 2024-04-04T06:07:07.375 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-04T06:07:07.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.596+0000 7f932f200640 1 Processor -- start 2024-04-04T06:07:07.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.596+0000 7f932f200640 1 -- start start 2024-04-04T06:07:07.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.597+0000 7f932f200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f933013f300 0x7f933006e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:07.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.598+0000 7f932f200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9330142880 con 0x7f9330141e40 2024-04-04T06:07:07.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.598+0000 7f932f200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93301422c0 con 0x7f93301419e0 2024-04-04T06:07:07.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.598+0000 7f932f200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f93300cee90 con 0x7f933013f300 2024-04-04T06:07:07.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.598+0000 7f932de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f933013f300 0x7f933006e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:07.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.598+0000 7f932e800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f9330141e40 0x7f93300cf320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52430/0 (socket says 172.21.15.159:52430) 2024-04-04T06:07:07.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.598+0000 7f932e800640 1 -- 172.21.15.159:0/122699497 learned_addr learned my addr 172.21.15.159:0/122699497 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:07.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.598+0000 7f932de00640 1 -- 172.21.15.159:0/122699497 >> v1:172.21.15.178:6789/0 conn(0x7f93301419e0 legacy=0x7f933015fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:07.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.598+0000 7f932de00640 1 -- 172.21.15.159:0/122699497 >> v1:172.21.15.159:6789/0 conn(0x7f9330141e40 legacy=0x7f93300cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:07.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.598+0000 7f932de00640 1 -- 172.21.15.159:0/122699497 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93301425a0 con 0x7f933013f300 2024-04-04T06:07:07.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.600+0000 7f932de00640 1 --2- 172.21.15.159:0/122699497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f933013f300 0x7f933006e030 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f93200097a0 tx=0x7f932002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e735c91b5c20aa server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:07.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.600+0000 7f932ca00640 1 -- 172.21.15.159:0/122699497 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320035020 con 0x7f933013f300 2024-04-04T06:07:07.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.600+0000 7f932ca00640 1 -- 172.21.15.159:0/122699497 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9320031830 con 0x7f933013f300 2024-04-04T06:07:07.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.600+0000 7f932ca00640 1 -- 172.21.15.159:0/122699497 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320031c00 con 0x7f933013f300 2024-04-04T06:07:07.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.601+0000 7f932f200640 1 -- 172.21.15.159:0/122699497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f933013f300 msgr2=0x7f933006e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:07.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.601+0000 7f932f200640 1 --2- 172.21.15.159:0/122699497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f933013f300 0x7f933006e030 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f93200097a0 tx=0x7f932002b8d0 comp rx=0 tx=0).stop 2024-04-04T06:07:07.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.601+0000 7f932f200640 1 -- 172.21.15.159:0/122699497 shutdown_connections 2024-04-04T06:07:07.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.601+0000 7f932f200640 1 --2- 172.21.15.159:0/122699497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f933013f300 0x7f933006e030 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:07.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.601+0000 7f932f200640 1 -- 172.21.15.159:0/122699497 >> 172.21.15.159:0/122699497 conn(0x7f933006b570 msgr2=0x7f933006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:07.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.601+0000 7f932f200640 1 -- 172.21.15.159:0/122699497 shutdown_connections 2024-04-04T06:07:07.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.601+0000 7f932f200640 1 -- 172.21.15.159:0/122699497 wait complete. 2024-04-04T06:07:07.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.603+0000 7f932f200640 1 Processor -- start 2024-04-04T06:07:07.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.603+0000 7f932f200640 1 -- start start 2024-04-04T06:07:07.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.604+0000 7f932f200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f933013f300 0x7f93300b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:07.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.604+0000 7f932f200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93301419e0 0x7f93300a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:07.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.604+0000 7f932f200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9330141e40 0x7f933015b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:07.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.604+0000 7f932f200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9330160620 con 0x7f933013f300 2024-04-04T06:07:07.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.604+0000 7f932f200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f93301604a0 con 0x7f93301419e0 2024-04-04T06:07:07.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.604+0000 7f932f200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f93300cfa30 con 0x7f9330141e40 2024-04-04T06:07:07.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.605+0000 7f932de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f933013f300 0x7f93300b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:07.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.605+0000 7f932de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f933013f300 0x7f93300b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59436/0 (socket says 172.21.15.159:59436) 2024-04-04T06:07:07.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.605+0000 7f932de00640 1 -- 172.21.15.159:0/2498394532 learned_addr learned my addr 172.21.15.159:0/2498394532 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:07.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.605+0000 7f932d400640 1 --2- 172.21.15.159:0/2498394532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93301419e0 0x7f93300a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:07.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.605+0000 7f932e800640 1 --2- 172.21.15.159:0/2498394532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9330141e40 0x7f933015b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:07.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.605+0000 7f932de00640 1 -- 172.21.15.159:0/2498394532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9330141e40 msgr2=0x7f933015b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:07.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.605+0000 7f932de00640 1 --2- 172.21.15.159:0/2498394532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9330141e40 0x7f933015b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:07.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.605+0000 7f932de00640 1 -- 172.21.15.159:0/2498394532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93301419e0 msgr2=0x7f93300a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:07.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.605+0000 7f932de00640 1 --2- 172.21.15.159:0/2498394532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93301419e0 0x7f93300a9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:07.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.605+0000 7f932de00640 1 -- 172.21.15.159:0/2498394532 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93300b50e0 con 0x7f933013f300 2024-04-04T06:07:07.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.606+0000 7f932de00640 1 --2- 172.21.15.159:0/2498394532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f933013f300 0x7f93300b4ba0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f93200098c0 tx=0x7f93200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:07.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.606+0000 7f9326000640 1 -- 172.21.15.159:0/2498394532 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932003ba80 con 0x7f933013f300 2024-04-04T06:07:07.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.606+0000 7f9326000640 1 -- 172.21.15.159:0/2498394532 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f932003bc20 con 0x7f933013f300 2024-04-04T06:07:07.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.606+0000 7f9326000640 1 -- 172.21.15.159:0/2498394532 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93200453f0 con 0x7f933013f300 2024-04-04T06:07:07.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.606+0000 7f932f200640 1 -- 172.21.15.159:0/2498394532 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f933013cd50 con 0x7f933013f300 2024-04-04T06:07:07.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.606+0000 7f932f200640 1 -- 172.21.15.159:0/2498394532 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93300b3de0 con 0x7f933013f300 2024-04-04T06:07:07.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.607+0000 7f932f200640 1 -- 172.21.15.159:0/2498394532 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9330075940 con 0x7f933013f300 2024-04-04T06:07:07.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.609+0000 7f9326000640 1 -- 172.21.15.159:0/2498394532 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f932004c060 con 0x7f933013f300 2024-04-04T06:07:07.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.610+0000 7f9326000640 1 --2- 172.21.15.159:0/2498394532 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f931006c8f0 0x7f931006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:07.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.611+0000 7f9326000640 1 -- 172.21.15.159:0/2498394532 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5227+0+0 (secure 0 0 0) 0x7f93200adad0 con 0x7f933013f300 2024-04-04T06:07:07.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.611+0000 7f932d400640 1 --2- 172.21.15.159:0/2498394532 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f931006c8f0 0x7f931006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:07.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.612+0000 7f932d400640 1 --2- 172.21.15.159:0/2498394532 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f931006c8f0 0x7f931006edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f93301f77c0 tx=0x7f931c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:07.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.614+0000 7f9326000640 1 -- 172.21.15.159:0/2498394532 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f93200ade20 con 0x7f933013f300 2024-04-04T06:07:07.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:07.817+0000 7f932f200640 1 -- 172.21.15.159:0/2498394532 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f933013cf50 con 0x7f933013f300 2024-04-04T06:07:08.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.357+0000 7f9326000640 1 -- 172.21.15.159:0/2498394532 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) ==== 170+0+0 (secure 0 0 0) 0x7f9320040070 con 0x7f933013f300 2024-04-04T06:07:08.352 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_data' created 2024-04-04T06:07:08.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.360+0000 7f932f200640 1 -- 172.21.15.159:0/2498394532 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f931006c8f0 msgr2=0x7f931006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:08.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.360+0000 7f932f200640 1 --2- 172.21.15.159:0/2498394532 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f931006c8f0 0x7f931006edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f93301f77c0 tx=0x7f931c009040 comp rx=0 tx=0).stop 2024-04-04T06:07:08.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.361+0000 7f932f200640 1 -- 172.21.15.159:0/2498394532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f933013f300 msgr2=0x7f93300b4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:08.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.361+0000 7f932f200640 1 --2- 172.21.15.159:0/2498394532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f933013f300 0x7f93300b4ba0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f93200098c0 tx=0x7f93200046c0 comp rx=0 tx=0).stop 2024-04-04T06:07:08.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.361+0000 7f932f200640 1 -- 172.21.15.159:0/2498394532 shutdown_connections 2024-04-04T06:07:08.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.361+0000 7f932f200640 1 --2- 172.21.15.159:0/2498394532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9330141e40 0x7f933015b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:08.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.361+0000 7f932f200640 1 --2- 172.21.15.159:0/2498394532 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f931006c8f0 0x7f931006edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:08.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.361+0000 7f932f200640 1 --2- 172.21.15.159:0/2498394532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93301419e0 0x7f93300a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:08.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.361+0000 7f932f200640 1 --2- 172.21.15.159:0/2498394532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f933013f300 0x7f93300b4ba0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:08.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.361+0000 7f932f200640 1 -- 172.21.15.159:0/2498394532 >> 172.21.15.159:0/2498394532 conn(0x7f933006b570 msgr2=0x7f93300cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:08.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.361+0000 7f932f200640 1 -- 172.21.15.159:0/2498394532 shutdown_connections 2024-04-04T06:07:08.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.361+0000 7f932f200640 1 -- 172.21.15.159:0/2498394532 wait complete. 2024-04-04T06:07:08.392 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-04T06:07:08.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.579+0000 7f9b3a800640 1 Processor -- start 2024-04-04T06:07:08.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.579+0000 7f9b3a800640 1 -- start start 2024-04-04T06:07:08.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.580+0000 7f9b3a800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9b3406dd30 0x7f9b341690a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:08.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.580+0000 7f9b3a800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b3415c330 con 0x7f9b3415bf50 2024-04-04T06:07:08.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.580+0000 7f9b3a800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b3406e230 con 0x7f9b340b3ec0 2024-04-04T06:07:08.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.580+0000 7f9b3a800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f9b3406ee40 con 0x7f9b3406dd30 2024-04-04T06:07:08.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.581+0000 7f9b39400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f9b3415bf50 0x7f9b340b37b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52444/0 (socket says 172.21.15.159:52444) 2024-04-04T06:07:08.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.581+0000 7f9b39400640 1 -- 172.21.15.159:0/3292847095 learned_addr learned my addr 172.21.15.159:0/3292847095 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:08.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.581+0000 7f9b39e00640 1 --2- 172.21.15.159:0/3292847095 >> v2:172.21.15.178:3300/0 conn(0x7f9b3406dd30 0x7f9b341690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:08.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.581+0000 7f9b33e00640 1 -- 172.21.15.159:0/3292847095 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 481685110 0 0) 0x7f9b3415c330 con 0x7f9b3415bf50 2024-04-04T06:07:08.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.581+0000 7f9b33e00640 1 -- 172.21.15.159:0/3292847095 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b1c003520 con 0x7f9b3415bf50 2024-04-04T06:07:08.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.581+0000 7f9b33e00640 1 -- 172.21.15.159:0/3292847095 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1646385908 0 0) 0x7f9b1c003520 con 0x7f9b3415bf50 2024-04-04T06:07:08.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.581+0000 7f9b33e00640 1 -- 172.21.15.159:0/3292847095 >> v2:172.21.15.178:3300/0 conn(0x7f9b3406dd30 msgr2=0x7f9b341690a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:08.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.581+0000 7f9b33e00640 1 --2- 172.21.15.159:0/3292847095 >> v2:172.21.15.178:3300/0 conn(0x7f9b3406dd30 0x7f9b341690a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:08.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.581+0000 7f9b33e00640 1 -- 172.21.15.159:0/3292847095 >> v1:172.21.15.178:6789/0 conn(0x7f9b340b3ec0 legacy=0x7f9b3406d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:08.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.581+0000 7f9b33e00640 1 -- 172.21.15.159:0/3292847095 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3406e510 con 0x7f9b3415bf50 2024-04-04T06:07:08.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.581+0000 7f9b33e00640 1 -- 172.21.15.159:0/3292847095 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f9b28001670 con 0x7f9b3415bf50 2024-04-04T06:07:08.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.582+0000 7f9b33e00640 1 -- 172.21.15.159:0/3292847095 >> v1:172.21.15.159:6789/0 conn(0x7f9b3415bf50 legacy=0x7f9b340b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:08.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.582+0000 7f9b33e00640 1 --2- 172.21.15.159:0/3292847095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b1c0039b0 0x7f9b1c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:08.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.582+0000 7f9b33e00640 1 --2- 172.21.15.159:0/3292847095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b1c004800 0x7f9b1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:08.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.582+0000 7f9b33e00640 1 --2- 172.21.15.159:0/3292847095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b1c0051c0 0x7f9b1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:08.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.582+0000 7f9b33e00640 1 -- 172.21.15.159:0/3292847095 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3406ee40 con 0x7f9b1c004800 2024-04-04T06:07:08.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.582+0000 7f9b33e00640 1 -- 172.21.15.159:0/3292847095 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c00c040 con 0x7f9b1c0039b0 2024-04-04T06:07:08.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.582+0000 7f9b33e00640 1 -- 172.21.15.159:0/3292847095 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9b1c00c240 con 0x7f9b1c0051c0 2024-04-04T06:07:08.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.582+0000 7f9b38a00640 1 --2- 172.21.15.159:0/3292847095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b1c004800 0x7f9b1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:08.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.582+0000 7f9b39400640 1 --2- 172.21.15.159:0/3292847095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b1c0039b0 0x7f9b1c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:08.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.582+0000 7f9b38a00640 1 -- 172.21.15.159:0/3292847095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b1c0051c0 msgr2=0x7f9b1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:08.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.582+0000 7f9b38a00640 1 --2- 172.21.15.159:0/3292847095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b1c0051c0 0x7f9b1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:08.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.582+0000 7f9b38a00640 1 -- 172.21.15.159:0/3292847095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b1c0039b0 msgr2=0x7f9b1c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:08.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.582+0000 7f9b38a00640 1 --2- 172.21.15.159:0/3292847095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b1c0039b0 0x7f9b1c003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:08.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.582+0000 7f9b38a00640 1 -- 172.21.15.159:0/3292847095 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1c003520 con 0x7f9b1c004800 2024-04-04T06:07:08.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.582+0000 7f9b39e00640 1 -- 172.21.15.159:0/3292847095 reap_dead start 2024-04-04T06:07:08.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.582+0000 7f9b39400640 1 --2- 172.21.15.159:0/3292847095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b1c0039b0 0x7f9b1c003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:08.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.584+0000 7f9b38a00640 1 --2- 172.21.15.159:0/3292847095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b1c004800 0x7f9b1c004c80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9b24003d80 tx=0x7f9b1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36bae61975c91d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:08.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.584+0000 7f9b33e00640 1 -- 172.21.15.159:0/3292847095 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24037070 con 0x7f9b1c004800 2024-04-04T06:07:08.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.584+0000 7f9b33e00640 1 -- 172.21.15.159:0/3292847095 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b24032600 con 0x7f9b1c004800 2024-04-04T06:07:08.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.584+0000 7f9b33e00640 1 -- 172.21.15.159:0/3292847095 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24003980 con 0x7f9b1c004800 2024-04-04T06:07:08.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.584+0000 7f9b3a800640 1 -- 172.21.15.159:0/3292847095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b1c004800 msgr2=0x7f9b1c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:08.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.584+0000 7f9b3a800640 1 --2- 172.21.15.159:0/3292847095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b1c004800 0x7f9b1c004c80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9b24003d80 tx=0x7f9b1c005830 comp rx=0 tx=0).stop 2024-04-04T06:07:08.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.585+0000 7f9b3a800640 1 -- 172.21.15.159:0/3292847095 shutdown_connections 2024-04-04T06:07:08.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.585+0000 7f9b3a800640 1 --2- 172.21.15.159:0/3292847095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b1c004800 0x7f9b1c004c80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:08.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.585+0000 7f9b3a800640 1 -- 172.21.15.159:0/3292847095 >> 172.21.15.159:0/3292847095 conn(0x7f9b340b1ba0 msgr2=0x7f9b340b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:08.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.585+0000 7f9b3a800640 1 -- 172.21.15.159:0/3292847095 shutdown_connections 2024-04-04T06:07:08.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.585+0000 7f9b3a800640 1 -- 172.21.15.159:0/3292847095 wait complete. 2024-04-04T06:07:08.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.586+0000 7f9b3a800640 1 Processor -- start 2024-04-04T06:07:08.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.587+0000 7f9b3a800640 1 -- start start 2024-04-04T06:07:08.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.588+0000 7f9b3a800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b1c004800 0x7f9b34152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:08.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.588+0000 7f9b3a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b340b1f80 0x7f9b340b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:08.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.588+0000 7f9b3a800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b340ae420 0x7f9b340ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:08.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.588+0000 7f9b3a800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3406ead0 con 0x7f9b340b1f80 2024-04-04T06:07:08.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.588+0000 7f9b3a800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3415ed30 con 0x7f9b1c004800 2024-04-04T06:07:08.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.588+0000 7f9b3a800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9b3406d620 con 0x7f9b340ae420 2024-04-04T06:07:08.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.588+0000 7f9b38a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b340b1f80 0x7f9b340b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:08.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.588+0000 7f9b38a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b340b1f80 0x7f9b340b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59466/0 (socket says 172.21.15.159:59466) 2024-04-04T06:07:08.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.588+0000 7f9b38a00640 1 -- 172.21.15.159:0/1241201905 learned_addr learned my addr 172.21.15.159:0/1241201905 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:08.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.588+0000 7f9b39400640 1 --2- 172.21.15.159:0/1241201905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b1c004800 0x7f9b34152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:08.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.588+0000 7f9b39e00640 1 --2- 172.21.15.159:0/1241201905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b340ae420 0x7f9b340ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:08.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.588+0000 7f9b38a00640 1 -- 172.21.15.159:0/1241201905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b340ae420 msgr2=0x7f9b340ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:08.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.588+0000 7f9b38a00640 1 --2- 172.21.15.159:0/1241201905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b340ae420 0x7f9b340ae880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:08.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.588+0000 7f9b38a00640 1 -- 172.21.15.159:0/1241201905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b1c004800 msgr2=0x7f9b34152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:08.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.588+0000 7f9b38a00640 1 --2- 172.21.15.159:0/1241201905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b1c004800 0x7f9b34152bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:08.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.588+0000 7f9b38a00640 1 -- 172.21.15.159:0/1241201905 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b340af190 con 0x7f9b340b1f80 2024-04-04T06:07:08.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.589+0000 7f9b38a00640 1 --2- 172.21.15.159:0/1241201905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b340b1f80 0x7f9b340b23e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9b24003ea0 tx=0x7f9b240036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:08.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.589+0000 7f9b31600640 1 -- 172.21.15.159:0/1241201905 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24042030 con 0x7f9b340b1f80 2024-04-04T06:07:08.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.589+0000 7f9b31600640 1 -- 172.21.15.159:0/1241201905 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b24032400 con 0x7f9b340b1f80 2024-04-04T06:07:08.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.589+0000 7f9b31600640 1 -- 172.21.15.159:0/1241201905 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24037040 con 0x7f9b340b1f80 2024-04-04T06:07:08.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.589+0000 7f9b3a800640 1 -- 172.21.15.159:0/1241201905 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b340af470 con 0x7f9b340b1f80 2024-04-04T06:07:08.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.589+0000 7f9b3a800640 1 -- 172.21.15.159:0/1241201905 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b340afa30 con 0x7f9b340b1f80 2024-04-04T06:07:08.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.590+0000 7f9b3a800640 1 -- 172.21.15.159:0/1241201905 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b34075940 con 0x7f9b340b1f80 2024-04-04T06:07:08.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.592+0000 7f9b31600640 1 -- 172.21.15.159:0/1241201905 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b24045070 con 0x7f9b340b1f80 2024-04-04T06:07:08.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.594+0000 7f9b31600640 1 --2- 172.21.15.159:0/1241201905 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b1806c970 0x7f9b1806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:08.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.594+0000 7f9b31600640 1 -- 172.21.15.159:0/1241201905 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5609+0+0 (secure 0 0 0) 0x7f9b240add50 con 0x7f9b340b1f80 2024-04-04T06:07:08.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.594+0000 7f9b39400640 1 --2- 172.21.15.159:0/1241201905 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b1806c970 0x7f9b1806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:08.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.596+0000 7f9b39400640 1 --2- 172.21.15.159:0/1241201905 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b1806c970 0x7f9b1806ee30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9b3406a480 tx=0x7f9b28009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:08.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.598+0000 7f9b31600640 1 -- 172.21.15.159:0/1241201905 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9b2407c1c0 con 0x7f9b340b1f80 2024-04-04T06:07:08.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:08.837+0000 7f9b3a800640 1 -- 172.21.15.159:0/1241201905 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9b34159c70 con 0x7f9b340b1f80 2024-04-04T06:07:09.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.404+0000 7f9b31600640 1 -- 172.21.15.159:0/1241201905 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f9b2407bac0 con 0x7f9b340b1f80 2024-04-04T06:07:09.401 INFO:teuthology.orchestra.run.smithi159.stderr:new fs with metadata pool 4 and data pool 5 2024-04-04T06:07:09.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.414+0000 7f9b3a800640 1 -- 172.21.15.159:0/1241201905 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b1806c970 msgr2=0x7f9b1806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:09.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.414+0000 7f9b3a800640 1 --2- 172.21.15.159:0/1241201905 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b1806c970 0x7f9b1806ee30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9b3406a480 tx=0x7f9b28009040 comp rx=0 tx=0).stop 2024-04-04T06:07:09.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.415+0000 7f9b3a800640 1 -- 172.21.15.159:0/1241201905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b340b1f80 msgr2=0x7f9b340b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:09.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.415+0000 7f9b3a800640 1 --2- 172.21.15.159:0/1241201905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b340b1f80 0x7f9b340b23e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9b24003ea0 tx=0x7f9b240036f0 comp rx=0 tx=0).stop 2024-04-04T06:07:09.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.417+0000 7f9b3a800640 1 -- 172.21.15.159:0/1241201905 shutdown_connections 2024-04-04T06:07:09.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.417+0000 7f9b3a800640 1 --2- 172.21.15.159:0/1241201905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b340ae420 0x7f9b340ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:09.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.417+0000 7f9b3a800640 1 --2- 172.21.15.159:0/1241201905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b340b1f80 0x7f9b340b23e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:09.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.417+0000 7f9b3a800640 1 --2- 172.21.15.159:0/1241201905 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b1806c970 0x7f9b1806ee30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:09.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.417+0000 7f9b3a800640 1 --2- 172.21.15.159:0/1241201905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b1c004800 0x7f9b34152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:09.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.417+0000 7f9b3a800640 1 -- 172.21.15.159:0/1241201905 >> 172.21.15.159:0/1241201905 conn(0x7f9b340b1ba0 msgr2=0x7f9b3415dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:09.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.417+0000 7f9b3a800640 1 -- 172.21.15.159:0/1241201905 shutdown_connections 2024-04-04T06:07:09.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.418+0000 7f9b3a800640 1 -- 172.21.15.159:0/1241201905 wait complete. 2024-04-04T06:07:09.443 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:07:09.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.619+0000 7fb6fe200640 1 Processor -- start 2024-04-04T06:07:09.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.619+0000 7fb6fe200640 1 -- start start 2024-04-04T06:07:09.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.620+0000 7fb6fe200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb6f80d2310 0x7fb6f80ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:09.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.620+0000 7fb6fe200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6f80cf530 0x7fb6f80b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:09.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.620+0000 7fb6fe200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6f815e880 con 0x7fb6f815e4a0 2024-04-04T06:07:09.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.620+0000 7fb6fe200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb6f80d34e0 con 0x7fb6f80d2310 2024-04-04T06:07:09.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.620+0000 7fb6fe200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f80d28d0 con 0x7fb6f80cf530 2024-04-04T06:07:09.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.621+0000 7fb6fd800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6f80cf530 0x7fb6f80b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:09.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.621+0000 7fb6f7e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb6f80d2310 0x7fb6f80ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:09.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.621+0000 7fb6f7e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb6f80d2310 0x7fb6f80ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:38004/0 (socket says 172.21.15.159:38004) 2024-04-04T06:07:09.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.621+0000 7fb6f7e00640 1 -- 172.21.15.159:0/1372831547 learned_addr learned my addr 172.21.15.159:0/1372831547 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:09.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.622+0000 7fb6f7e00640 1 -- 172.21.15.159:0/1372831547 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6f80cf530 msgr2=0x7fb6f80b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:09.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.622+0000 7fb6f7e00640 1 --2- 172.21.15.159:0/1372831547 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6f80cf530 0x7fb6f80b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:09.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.622+0000 7fb6f7e00640 1 -- 172.21.15.159:0/1372831547 >> v1:172.21.15.178:6789/0 conn(0x7fb6f815e4a0 legacy=0x7fb6f80b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:09.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.622+0000 7fb6f7e00640 1 -- 172.21.15.159:0/1372831547 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f80d2bb0 con 0x7fb6f80d2310 2024-04-04T06:07:09.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.623+0000 7fb6f7e00640 1 --2- 172.21.15.159:0/1372831547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6f80d2310 0x7fb6f80ceff0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009870 tx=0x7fb6e402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=169550adef72b51d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:09.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.623+0000 7fb6f7400640 1 -- 172.21.15.159:0/1372831547 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4035020 con 0x7fb6f80d2310 2024-04-04T06:07:09.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.624+0000 7fb6f7400640 1 -- 172.21.15.159:0/1372831547 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e4031870 con 0x7fb6f80d2310 2024-04-04T06:07:09.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.624+0000 7fb6f7400640 1 -- 172.21.15.159:0/1372831547 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4031c40 con 0x7fb6f80d2310 2024-04-04T06:07:09.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.624+0000 7fb6fe200640 1 -- 172.21.15.159:0/1372831547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6f80d2310 msgr2=0x7fb6f80ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:09.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.624+0000 7fb6fe200640 1 --2- 172.21.15.159:0/1372831547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6f80d2310 0x7fb6f80ceff0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009870 tx=0x7fb6e402b910 comp rx=0 tx=0).stop 2024-04-04T06:07:09.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.624+0000 7fb6fe200640 1 -- 172.21.15.159:0/1372831547 shutdown_connections 2024-04-04T06:07:09.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.624+0000 7fb6fe200640 1 --2- 172.21.15.159:0/1372831547 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6f80cf530 0x7fb6f80b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:09.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.624+0000 7fb6fe200640 1 --2- 172.21.15.159:0/1372831547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6f80d2310 0x7fb6f80ceff0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:09.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.624+0000 7fb6fe200640 1 -- 172.21.15.159:0/1372831547 >> 172.21.15.159:0/1372831547 conn(0x7fb6f80676d0 msgr2=0x7fb6f8067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:09.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.624+0000 7fb6fe200640 1 -- 172.21.15.159:0/1372831547 shutdown_connections 2024-04-04T06:07:09.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.625+0000 7fb6fe200640 1 -- 172.21.15.159:0/1372831547 wait complete. 2024-04-04T06:07:09.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.626+0000 7fb6fe200640 1 Processor -- start 2024-04-04T06:07:09.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.626+0000 7fb6fe200640 1 -- start start 2024-04-04T06:07:09.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.627+0000 7fb6fe200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6f80cf530 0x7fb6f80c6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:09.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.627+0000 7fb6fe200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6f80d2310 0x7fb6f81e6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:09.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.627+0000 7fb6fe200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6f815e4a0 0x7fb6f81e70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:09.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.627+0000 7fb6fe200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f80d3970 con 0x7fb6f80cf530 2024-04-04T06:07:09.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.627+0000 7fb6fe200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f80d3170 con 0x7fb6f80d2310 2024-04-04T06:07:09.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.627+0000 7fb6fe200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f80b7920 con 0x7fb6f815e4a0 2024-04-04T06:07:09.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.627+0000 7fb6fce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6f80cf530 0x7fb6f80c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:09.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.628+0000 7fb6fce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6f80cf530 0x7fb6f80c6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59472/0 (socket says 172.21.15.159:59472) 2024-04-04T06:07:09.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.628+0000 7fb6fce00640 1 -- 172.21.15.159:0/1743684550 learned_addr learned my addr 172.21.15.159:0/1743684550 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:09.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.628+0000 7fb6fd800640 1 --2- 172.21.15.159:0/1743684550 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6f815e4a0 0x7fb6f81e70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:09.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.628+0000 7fb6f7e00640 1 --2- 172.21.15.159:0/1743684550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6f80d2310 0x7fb6f81e6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:09.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.628+0000 7fb6fce00640 1 -- 172.21.15.159:0/1743684550 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6f815e4a0 msgr2=0x7fb6f81e70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:09.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.628+0000 7fb6fce00640 1 --2- 172.21.15.159:0/1743684550 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6f815e4a0 0x7fb6f81e70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:09.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.628+0000 7fb6fce00640 1 -- 172.21.15.159:0/1743684550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6f80d2310 msgr2=0x7fb6f81e6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:09.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.628+0000 7fb6fce00640 1 --2- 172.21.15.159:0/1743684550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6f80d2310 0x7fb6f81e6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:09.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.628+0000 7fb6fce00640 1 -- 172.21.15.159:0/1743684550 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f80cd9f0 con 0x7fb6f80cf530 2024-04-04T06:07:09.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.628+0000 7fb6fce00640 1 --2- 172.21.15.159:0/1743684550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6f80cf530 0x7fb6f80c6740 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00c3c0 tx=0x7fb6ec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:09.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.629+0000 7fb6f4c00640 1 -- 172.21.15.159:0/1743684550 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0052d0 con 0x7fb6f80cf530 2024-04-04T06:07:09.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.629+0000 7fb6f4c00640 1 -- 172.21.15.159:0/1743684550 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6ec005490 con 0x7fb6f80cf530 2024-04-04T06:07:09.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.629+0000 7fb6f4c00640 1 -- 172.21.15.159:0/1743684550 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec007260 con 0x7fb6f80cf530 2024-04-04T06:07:09.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.629+0000 7fb6fe200640 1 -- 172.21.15.159:0/1743684550 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f80cdcd0 con 0x7fb6f80cf530 2024-04-04T06:07:09.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.629+0000 7fb6fe200640 1 -- 172.21.15.159:0/1743684550 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f814b6a0 con 0x7fb6f80cf530 2024-04-04T06:07:09.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.630+0000 7fb6fe200640 1 -- 172.21.15.159:0/1743684550 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f8075940 con 0x7fb6f80cf530 2024-04-04T06:07:09.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.631+0000 7fb6f4c00640 1 -- 172.21.15.159:0/1743684550 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6ec007400 con 0x7fb6f80cf530 2024-04-04T06:07:09.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.632+0000 7fb6f4c00640 1 --2- 172.21.15.159:0/1743684550 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb6d806c8f0 0x7fb6d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:09.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.632+0000 7fb6f4c00640 1 -- 172.21.15.159:0/1743684550 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb6ec08b050 con 0x7fb6f80cf530 2024-04-04T06:07:09.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.632+0000 7fb6f7e00640 1 --2- 172.21.15.159:0/1743684550 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb6d806c8f0 0x7fb6d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:09.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.633+0000 7fb6f7e00640 1 --2- 172.21.15.159:0/1743684550 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb6d806c8f0 0x7fb6d806edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4036000 tx=0x7fb6e4008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:09.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.636+0000 7fb6f4c00640 1 -- 172.21.15.159:0/1743684550 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb6ec059480 con 0x7fb6f80cf530 2024-04-04T06:07:09.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.821+0000 7fb6fe200640 1 -- 172.21.15.159:0/1743684550 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6f80d3c00 con 0x7fb6f80cf530 2024-04-04T06:07:09.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.823+0000 7fb6f4c00640 1 -- 172.21.15.159:0/1743684550 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16694 (secure 0 0 0) 0x7fb6ec058d80 con 0x7fb6f80cf530 2024-04-04T06:07:09.818 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:09.818 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":33,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:07:08.838896+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:07:06.708548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-04T06:07:07.833730+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:07:09.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.825+0000 7fb6fe200640 1 -- 172.21.15.159:0/1743684550 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb6d806c8f0 msgr2=0x7fb6d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:09.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.825+0000 7fb6fe200640 1 --2- 172.21.15.159:0/1743684550 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb6d806c8f0 0x7fb6d806edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4036000 tx=0x7fb6e4008030 comp rx=0 tx=0).stop 2024-04-04T06:07:09.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.825+0000 7fb6fe200640 1 -- 172.21.15.159:0/1743684550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6f80cf530 msgr2=0x7fb6f80c6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:09.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.825+0000 7fb6fe200640 1 --2- 172.21.15.159:0/1743684550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6f80cf530 0x7fb6f80c6740 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00c3c0 tx=0x7fb6ec00c880 comp rx=0 tx=0).stop 2024-04-04T06:07:09.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.825+0000 7fb6fe200640 1 -- 172.21.15.159:0/1743684550 shutdown_connections 2024-04-04T06:07:09.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.825+0000 7fb6fe200640 1 --2- 172.21.15.159:0/1743684550 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6f815e4a0 0x7fb6f81e70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:09.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.825+0000 7fb6fe200640 1 --2- 172.21.15.159:0/1743684550 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb6d806c8f0 0x7fb6d806edb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:09.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.825+0000 7fb6fe200640 1 --2- 172.21.15.159:0/1743684550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6f80d2310 0x7fb6f81e6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:09.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.825+0000 7fb6fe200640 1 --2- 172.21.15.159:0/1743684550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6f80cf530 0x7fb6f80c6740 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:09.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.825+0000 7fb6fe200640 1 -- 172.21.15.159:0/1743684550 >> 172.21.15.159:0/1743684550 conn(0x7fb6f80676d0 msgr2=0x7fb6f80c9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:09.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.825+0000 7fb6fe200640 1 -- 172.21.15.159:0/1743684550 shutdown_connections 2024-04-04T06:07:09.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:09.825+0000 7fb6fe200640 1 -- 172.21.15.159:0/1743684550 wait complete. 2024-04-04T06:07:09.837 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:07:10.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.042+0000 7ff97b200640 1 Processor -- start 2024-04-04T06:07:10.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.042+0000 7ff97b200640 1 -- start start 2024-04-04T06:07:10.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.044+0000 7ff97b200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff97c15fc50 0x7ff97c0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:10.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.044+0000 7ff97b200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff97c0cc490 0x7ff97c0d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:10.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.044+0000 7ff97b200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff97c0b3150 con 0x7ff97c0cc0b0 2024-04-04T06:07:10.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.044+0000 7ff97b200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7ff97c15c610 con 0x7ff97c15fc50 2024-04-04T06:07:10.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.044+0000 7ff97b200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff97c1600e0 con 0x7ff97c0cc490 2024-04-04T06:07:10.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.044+0000 7ff971400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff97c15fc50 0x7ff97c0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:10.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.044+0000 7ff979e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7ff97c0cc0b0 0x7ff97c06dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52454/0 (socket says 172.21.15.159:52454) 2024-04-04T06:07:10.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.044+0000 7ff979e00640 1 -- 172.21.15.159:0/191040460 learned_addr learned my addr 172.21.15.159:0/191040460 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:10.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.044+0000 7ff97a800640 1 --2- 172.21.15.159:0/191040460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff97c0cc490 0x7ff97c0d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:10.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.044+0000 7ff971400640 1 -- 172.21.15.159:0/191040460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff97c0cc490 msgr2=0x7ff97c0d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:10.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.044+0000 7ff971400640 1 --2- 172.21.15.159:0/191040460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff97c0cc490 0x7ff97c0d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:10.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.044+0000 7ff971400640 1 -- 172.21.15.159:0/191040460 >> v1:172.21.15.159:6789/0 conn(0x7ff97c0cc0b0 legacy=0x7ff97c06dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:10.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.044+0000 7ff971400640 1 -- 172.21.15.159:0/191040460 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97c1603c0 con 0x7ff97c15fc50 2024-04-04T06:07:10.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.045+0000 7ff971400640 1 --2- 172.21.15.159:0/191040460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff97c15fc50 0x7ff97c0cfdc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff9640097a0 tx=0x7ff96402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d0ac1ec068b8bd server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:10.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.046+0000 7ff979400640 1 -- 172.21.15.159:0/191040460 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964035020 con 0x7ff97c15fc50 2024-04-04T06:07:10.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.046+0000 7ff979400640 1 -- 172.21.15.159:0/191040460 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff964031810 con 0x7ff97c15fc50 2024-04-04T06:07:10.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.046+0000 7ff979400640 1 -- 172.21.15.159:0/191040460 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964031be0 con 0x7ff97c15fc50 2024-04-04T06:07:10.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.046+0000 7ff97b200640 1 -- 172.21.15.159:0/191040460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff97c15fc50 msgr2=0x7ff97c0cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:10.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.046+0000 7ff97b200640 1 --2- 172.21.15.159:0/191040460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff97c15fc50 0x7ff97c0cfdc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff9640097a0 tx=0x7ff96402b8b0 comp rx=0 tx=0).stop 2024-04-04T06:07:10.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.046+0000 7ff97b200640 1 -- 172.21.15.159:0/191040460 shutdown_connections 2024-04-04T06:07:10.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.046+0000 7ff97b200640 1 --2- 172.21.15.159:0/191040460 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff97c0cc490 0x7ff97c0d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:10.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.046+0000 7ff97b200640 1 --2- 172.21.15.159:0/191040460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff97c15fc50 0x7ff97c0cfdc0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:10.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.046+0000 7ff97b200640 1 -- 172.21.15.159:0/191040460 >> 172.21.15.159:0/191040460 conn(0x7ff97c0ccad0 msgr2=0x7ff97c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:10.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.047+0000 7ff97b200640 1 -- 172.21.15.159:0/191040460 shutdown_connections 2024-04-04T06:07:10.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.047+0000 7ff97b200640 1 -- 172.21.15.159:0/191040460 wait complete. 2024-04-04T06:07:10.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.048+0000 7ff97b200640 1 Processor -- start 2024-04-04T06:07:10.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.048+0000 7ff97b200640 1 -- start start 2024-04-04T06:07:10.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.049+0000 7ff97b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff97c0cc0b0 0x7ff97c13e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:10.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.049+0000 7ff97b200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff97c0cc490 0x7ff97c13d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:10.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.049+0000 7ff97b200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff97c1e7a90 0x7ff97c1e9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:10.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.049+0000 7ff97b200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c15ca70 con 0x7ff97c0cc0b0 2024-04-04T06:07:10.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.049+0000 7ff97b200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c160980 con 0x7ff97c0cc490 2024-04-04T06:07:10.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.049+0000 7ff97b200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff97c15cbf0 con 0x7ff97c1e7a90 2024-04-04T06:07:10.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.049+0000 7ff979e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff97c0cc0b0 0x7ff97c13e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:10.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.049+0000 7ff979e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff97c0cc0b0 0x7ff97c13e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59494/0 (socket says 172.21.15.159:59494) 2024-04-04T06:07:10.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.049+0000 7ff979e00640 1 -- 172.21.15.159:0/1615150049 learned_addr learned my addr 172.21.15.159:0/1615150049 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:10.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.050+0000 7ff979e00640 1 -- 172.21.15.159:0/1615150049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff97c1e7a90 msgr2=0x7ff97c1e9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:10.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.050+0000 7ff979e00640 1 --2- 172.21.15.159:0/1615150049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff97c1e7a90 0x7ff97c1e9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:10.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.050+0000 7ff979e00640 1 -- 172.21.15.159:0/1615150049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff97c0cc490 msgr2=0x7ff97c13d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:10.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.050+0000 7ff979e00640 1 --2- 172.21.15.159:0/1615150049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff97c0cc490 0x7ff97c13d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:10.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.050+0000 7ff979e00640 1 -- 172.21.15.159:0/1615150049 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97c1e7fe0 con 0x7ff97c0cc0b0 2024-04-04T06:07:10.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.050+0000 7ff979e00640 1 --2- 172.21.15.159:0/1615150049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff97c0cc0b0 0x7ff97c13e1e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00c400 tx=0x7ff96c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:10.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.050+0000 7ff972a00640 1 -- 172.21.15.159:0/1615150049 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c017070 con 0x7ff97c0cc0b0 2024-04-04T06:07:10.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.050+0000 7ff972a00640 1 -- 172.21.15.159:0/1615150049 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff96c005ce0 con 0x7ff97c0cc0b0 2024-04-04T06:07:10.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.050+0000 7ff972a00640 1 -- 172.21.15.159:0/1615150049 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c0072f0 con 0x7ff97c0cc0b0 2024-04-04T06:07:10.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.050+0000 7ff97b200640 1 -- 172.21.15.159:0/1615150049 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff97c1e82c0 con 0x7ff97c0cc0b0 2024-04-04T06:07:10.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.050+0000 7ff97b200640 1 -- 172.21.15.159:0/1615150049 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff97c1e8880 con 0x7ff97c0cc0b0 2024-04-04T06:07:10.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.052+0000 7ff97b200640 1 -- 172.21.15.159:0/1615150049 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff97c075940 con 0x7ff97c0cc0b0 2024-04-04T06:07:10.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.053+0000 7ff972a00640 1 -- 172.21.15.159:0/1615150049 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff96c01a070 con 0x7ff97c0cc0b0 2024-04-04T06:07:10.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.054+0000 7ff972a00640 1 --2- 172.21.15.159:0/1615150049 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff95806c8f0 0x7ff95806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:10.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.054+0000 7ff972a00640 1 -- 172.21.15.159:0/1615150049 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff96c08af70 con 0x7ff97c0cc0b0 2024-04-04T06:07:10.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.054+0000 7ff971400640 1 --2- 172.21.15.159:0/1615150049 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff95806c8f0 0x7ff95806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:10.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.055+0000 7ff971400640 1 --2- 172.21.15.159:0/1615150049 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff95806c8f0 0x7ff95806edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff964036000 tx=0x7ff964030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:10.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.058+0000 7ff972a00640 1 -- 172.21.15.159:0/1615150049 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff96c059390 con 0x7ff97c0cc0b0 2024-04-04T06:07:10.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.242+0000 7ff97b200640 1 -- 172.21.15.159:0/1615150049 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff97c073830 con 0x7ff97c0cc0b0 2024-04-04T06:07:10.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.245+0000 7ff972a00640 1 -- 172.21.15.159:0/1615150049 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16694 (secure 0 0 0) 0x7ff96c058c90 con 0x7ff97c0cc0b0 2024-04-04T06:07:10.241 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:10.241 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":33,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:07:08.838896+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:07:06.708548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-04T06:07:07.833730+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:07:10.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.248+0000 7ff97b200640 1 -- 172.21.15.159:0/1615150049 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff95806c8f0 msgr2=0x7ff95806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:10.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.248+0000 7ff97b200640 1 --2- 172.21.15.159:0/1615150049 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff95806c8f0 0x7ff95806edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff964036000 tx=0x7ff964030000 comp rx=0 tx=0).stop 2024-04-04T06:07:10.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.248+0000 7ff97b200640 1 -- 172.21.15.159:0/1615150049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff97c0cc0b0 msgr2=0x7ff97c13e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:10.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.248+0000 7ff97b200640 1 --2- 172.21.15.159:0/1615150049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff97c0cc0b0 0x7ff97c13e1e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00c400 tx=0x7ff96c00c8c0 comp rx=0 tx=0).stop 2024-04-04T06:07:10.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.248+0000 7ff97b200640 1 -- 172.21.15.159:0/1615150049 shutdown_connections 2024-04-04T06:07:10.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.248+0000 7ff97b200640 1 --2- 172.21.15.159:0/1615150049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff97c1e7a90 0x7ff97c1e9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:10.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.248+0000 7ff97b200640 1 --2- 172.21.15.159:0/1615150049 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff95806c8f0 0x7ff95806edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:10.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.248+0000 7ff97b200640 1 --2- 172.21.15.159:0/1615150049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff97c0cc490 0x7ff97c13d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:10.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.248+0000 7ff97b200640 1 --2- 172.21.15.159:0/1615150049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff97c0cc0b0 0x7ff97c13e1e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:10.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.248+0000 7ff97b200640 1 -- 172.21.15.159:0/1615150049 >> 172.21.15.159:0/1615150049 conn(0x7ff97c0ccad0 msgr2=0x7ff97c15f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:10.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.248+0000 7ff97b200640 1 -- 172.21.15.159:0/1615150049 shutdown_connections 2024-04-04T06:07:10.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.248+0000 7ff97b200640 1 -- 172.21.15.159:0/1615150049 wait complete. 2024-04-04T06:07:10.261 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-04T06:07:10.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.479+0000 7fccb6a00640 1 Processor -- start 2024-04-04T06:07:10.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.480+0000 7fccb6a00640 1 -- start start 2024-04-04T06:07:10.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.481+0000 7fccb6a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccb0155550 0x7fccb0155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:10.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.481+0000 7fccb6a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fccb015f800 0x7fccb00d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:10.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.481+0000 7fccb6a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccb00b40b0 con 0x7fccb00d07c0 2024-04-04T06:07:10.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.481+0000 7fccb6a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fccb00d0c30 con 0x7fccb015f800 2024-04-04T06:07:10.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.481+0000 7fccb6a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fccb00b4390 con 0x7fccb0155550 2024-04-04T06:07:10.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.481+0000 7fcca7400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fccb015f800 0x7fccb00d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:10.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.482+0000 7fcca7400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fccb015f800 0x7fccb00d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59496/0 (socket says 172.21.15.159:59496) 2024-04-04T06:07:10.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.482+0000 7fcca7400640 1 -- 172.21.15.159:0/1572180995 learned_addr learned my addr 172.21.15.159:0/1572180995 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:10.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.482+0000 7fccafe00640 1 --2- 172.21.15.159:0/1572180995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccb0155550 0x7fccb0155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:10.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.482+0000 7fcca7400640 1 -- 172.21.15.159:0/1572180995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccb0155550 msgr2=0x7fccb0155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:10.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.482+0000 7fcca7400640 1 --2- 172.21.15.159:0/1572180995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccb0155550 0x7fccb0155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:10.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.482+0000 7fcca7400640 1 -- 172.21.15.159:0/1572180995 >> v1:172.21.15.178:6789/0 conn(0x7fccb00d07c0 legacy=0x7fccb00aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:10.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.482+0000 7fcca7400640 1 -- 172.21.15.159:0/1572180995 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb00b4670 con 0x7fccb015f800 2024-04-04T06:07:10.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.482+0000 7fccafe00640 1 --2- 172.21.15.159:0/1572180995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccb0155550 0x7fccb0155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:07:10.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.483+0000 7fcca7400640 1 --2- 172.21.15.159:0/1572180995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccb015f800 0x7fccb00d0280 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800bb10 tx=0x7fcc9802e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aaf56f41030379 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:10.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.484+0000 7fccaf400640 1 -- 172.21.15.159:0/1572180995 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9803b070 con 0x7fccb015f800 2024-04-04T06:07:10.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.484+0000 7fccaf400640 1 -- 172.21.15.159:0/1572180995 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc98036440 con 0x7fccb015f800 2024-04-04T06:07:10.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.484+0000 7fccaf400640 1 -- 172.21.15.159:0/1572180995 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98036810 con 0x7fccb015f800 2024-04-04T06:07:10.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.484+0000 7fccb6a00640 1 -- 172.21.15.159:0/1572180995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccb015f800 msgr2=0x7fccb00d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:10.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.484+0000 7fccb6a00640 1 --2- 172.21.15.159:0/1572180995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccb015f800 0x7fccb00d0280 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800bb10 tx=0x7fcc9802e150 comp rx=0 tx=0).stop 2024-04-04T06:07:10.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.484+0000 7fccb6a00640 1 -- 172.21.15.159:0/1572180995 shutdown_connections 2024-04-04T06:07:10.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.484+0000 7fccb6a00640 1 --2- 172.21.15.159:0/1572180995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccb015f800 0x7fccb00d0280 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:10.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.484+0000 7fccb6a00640 1 --2- 172.21.15.159:0/1572180995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccb0155550 0x7fccb0155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:10.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.484+0000 7fccb6a00640 1 -- 172.21.15.159:0/1572180995 >> 172.21.15.159:0/1572180995 conn(0x7fccb006b750 msgr2=0x7fccb006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:10.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.484+0000 7fccb6a00640 1 -- 172.21.15.159:0/1572180995 shutdown_connections 2024-04-04T06:07:10.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.484+0000 7fccb6a00640 1 -- 172.21.15.159:0/1572180995 wait complete. 2024-04-04T06:07:10.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.486+0000 7fccb6a00640 1 Processor -- start 2024-04-04T06:07:10.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.486+0000 7fccb6a00640 1 -- start start 2024-04-04T06:07:10.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.487+0000 7fccb6a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccb00d07c0 0x7fccb00afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:10.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.487+0000 7fccb6a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fccb0155550 0x7fccb01539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:10.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.487+0000 7fccb6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccb015f800 0x7fccb0153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:10.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.487+0000 7fccb6a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fccb015fc80 con 0x7fccb015f800 2024-04-04T06:07:10.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.487+0000 7fccb6a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fccb00b4c30 con 0x7fccb0155550 2024-04-04T06:07:10.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.487+0000 7fccb6a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fccb015ba20 con 0x7fccb00d07c0 2024-04-04T06:07:10.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.487+0000 7fccb4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccb015f800 0x7fccb0153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:10.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.487+0000 7fccb4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccb015f800 0x7fccb0153f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59504/0 (socket says 172.21.15.159:59504) 2024-04-04T06:07:10.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.487+0000 7fccb4a00640 1 -- 172.21.15.159:0/1593971877 learned_addr learned my addr 172.21.15.159:0/1593971877 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:10.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.488+0000 7fccafe00640 1 --2- 172.21.15.159:0/1593971877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccb00d07c0 0x7fccb00afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:10.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.488+0000 7fcca7400640 1 --2- 172.21.15.159:0/1593971877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fccb0155550 0x7fccb01539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:10.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.488+0000 7fccb4a00640 1 -- 172.21.15.159:0/1593971877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccb00d07c0 msgr2=0x7fccb00afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:10.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.488+0000 7fccb4a00640 1 --2- 172.21.15.159:0/1593971877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccb00d07c0 0x7fccb00afbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:10.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.488+0000 7fccb4a00640 1 -- 172.21.15.159:0/1593971877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fccb0155550 msgr2=0x7fccb01539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:10.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.488+0000 7fccb4a00640 1 --2- 172.21.15.159:0/1593971877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fccb0155550 0x7fccb01539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:10.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.488+0000 7fccb4a00640 1 -- 172.21.15.159:0/1593971877 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb00b02e0 con 0x7fccb015f800 2024-04-04T06:07:10.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.488+0000 7fccb4a00640 1 --2- 172.21.15.159:0/1593971877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccb015f800 0x7fccb0153f20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcca800c390 tx=0x7fcca800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:10.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.488+0000 7fccacc00640 1 -- 172.21.15.159:0/1593971877 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8017070 con 0x7fccb015f800 2024-04-04T06:07:10.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.489+0000 7fccacc00640 1 -- 172.21.15.159:0/1593971877 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcca80052d0 con 0x7fccb015f800 2024-04-04T06:07:10.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.489+0000 7fccacc00640 1 -- 172.21.15.159:0/1593971877 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8003810 con 0x7fccb015f800 2024-04-04T06:07:10.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.489+0000 7fccb6a00640 1 -- 172.21.15.159:0/1593971877 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb00b05c0 con 0x7fccb015f800 2024-04-04T06:07:10.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.489+0000 7fccb6a00640 1 -- 172.21.15.159:0/1593971877 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb00b0b80 con 0x7fccb015f800 2024-04-04T06:07:10.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.490+0000 7fccb6a00640 1 -- 172.21.15.159:0/1593971877 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccb00685c0 con 0x7fccb015f800 2024-04-04T06:07:10.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.491+0000 7fccacc00640 1 -- 172.21.15.159:0/1593971877 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcca801a070 con 0x7fccb015f800 2024-04-04T06:07:10.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.491+0000 7fccacc00640 1 --2- 172.21.15.159:0/1593971877 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcc8c06c8f0 0x7fcc8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:10.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.492+0000 7fccacc00640 1 -- 172.21.15.159:0/1593971877 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fcca808af10 con 0x7fccb015f800 2024-04-04T06:07:10.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.492+0000 7fccafe00640 1 --2- 172.21.15.159:0/1593971877 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcc8c06c8f0 0x7fcc8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:10.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.493+0000 7fccafe00640 1 --2- 172.21.15.159:0/1593971877 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcc8c06c8f0 0x7fcc8c06edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcca0003fd0 tx=0x7fcca0003ce0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:10.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.496+0000 7fccacc00640 1 -- 172.21.15.159:0/1593971877 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcca80592a0 con 0x7fccb015f800 2024-04-04T06:07:10.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:10.736+0000 7fccb6a00640 1 -- 172.21.15.159:0/1593971877 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fccb00b3ad0 con 0x7fccb015f800 2024-04-04T06:07:10.915 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:07:10.923 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:07:11.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.421+0000 7fccacc00640 1 -- 172.21.15.159:0/1593971877 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fcca8058ba0 con 0x7fccb015f800 2024-04-04T06:07:11.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.424+0000 7fccb6a00640 1 -- 172.21.15.159:0/1593971877 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcc8c06c8f0 msgr2=0x7fcc8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:11.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.424+0000 7fccb6a00640 1 --2- 172.21.15.159:0/1593971877 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcc8c06c8f0 0x7fcc8c06edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcca0003fd0 tx=0x7fcca0003ce0 comp rx=0 tx=0).stop 2024-04-04T06:07:11.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.424+0000 7fccb6a00640 1 -- 172.21.15.159:0/1593971877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccb015f800 msgr2=0x7fccb0153f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:11.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.424+0000 7fccb6a00640 1 --2- 172.21.15.159:0/1593971877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccb015f800 0x7fccb0153f20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcca800c390 tx=0x7fcca800c850 comp rx=0 tx=0).stop 2024-04-04T06:07:11.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.425+0000 7fccb6a00640 1 -- 172.21.15.159:0/1593971877 shutdown_connections 2024-04-04T06:07:11.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.425+0000 7fccb6a00640 1 --2- 172.21.15.159:0/1593971877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccb015f800 0x7fccb0153f20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:11.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.425+0000 7fccb6a00640 1 --2- 172.21.15.159:0/1593971877 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcc8c06c8f0 0x7fcc8c06edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:11.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.425+0000 7fccb6a00640 1 --2- 172.21.15.159:0/1593971877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fccb0155550 0x7fccb01539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:11.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.425+0000 7fccb6a00640 1 --2- 172.21.15.159:0/1593971877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fccb00d07c0 0x7fccb00afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:11.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.425+0000 7fccb6a00640 1 -- 172.21.15.159:0/1593971877 >> 172.21.15.159:0/1593971877 conn(0x7fccb006b750 msgr2=0x7fccb015c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:11.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.425+0000 7fccb6a00640 1 -- 172.21.15.159:0/1593971877 shutdown_connections 2024-04-04T06:07:11.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.425+0000 7fccb6a00640 1 -- 172.21.15.159:0/1593971877 wait complete. 2024-04-04T06:07:11.441 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-04T06:07:11.441 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-04T06:07:11.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.665+0000 7f3e85000640 1 Processor -- start 2024-04-04T06:07:11.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.665+0000 7f3e85000640 1 -- start start 2024-04-04T06:07:11.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.667+0000 7f3e85000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3e800cc880 0x7f3e800cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:11.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.667+0000 7f3e85000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e8006d620 0x7f3e8006da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:11.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.667+0000 7f3e85000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e800ce890 con 0x7f3e8013f190 2024-04-04T06:07:11.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.667+0000 7f3e85000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3e800cd230 con 0x7f3e800cc880 2024-04-04T06:07:11.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.667+0000 7f3e85000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3e800ceb50 con 0x7f3e8006d620 2024-04-04T06:07:11.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.667+0000 7f3e7f400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f3e8013f190 0x7f3e800cf630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52468/0 (socket says 172.21.15.159:52468) 2024-04-04T06:07:11.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.667+0000 7f3e7f400640 1 -- 172.21.15.159:0/1472092650 learned_addr learned my addr 172.21.15.159:0/1472092650 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:11.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.668+0000 7f3e7fe00640 1 --2- 172.21.15.159:0/1472092650 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e8006d620 0x7f3e8006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:11.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.668+0000 7f3e7ea00640 1 --2- 172.21.15.159:0/1472092650 >> v2:172.21.15.178:3300/0 conn(0x7f3e800cc880 0x7f3e800cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:11.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.668+0000 7f3e7e000640 1 -- 172.21.15.159:0/1472092650 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1249039482 0 0) 0x7f3e800ce890 con 0x7f3e8013f190 2024-04-04T06:07:11.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.668+0000 7f3e7e000640 1 -- 172.21.15.159:0/1472092650 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e64003520 con 0x7f3e8013f190 2024-04-04T06:07:11.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.668+0000 7f3e7e000640 1 -- 172.21.15.159:0/1472092650 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3079078343 0 0) 0x7f3e64003520 con 0x7f3e8013f190 2024-04-04T06:07:11.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.668+0000 7f3e7e000640 1 -- 172.21.15.159:0/1472092650 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e8006d620 msgr2=0x7f3e8006da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:11.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.668+0000 7f3e7e000640 1 --2- 172.21.15.159:0/1472092650 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e8006d620 0x7f3e8006da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:11.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.668+0000 7f3e7e000640 1 -- 172.21.15.159:0/1472092650 >> v2:172.21.15.178:3300/0 conn(0x7f3e800cc880 msgr2=0x7f3e800cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:11.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.668+0000 7f3e7e000640 1 --2- 172.21.15.159:0/1472092650 >> v2:172.21.15.178:3300/0 conn(0x7f3e800cc880 0x7f3e800cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:11.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.668+0000 7f3e7e000640 1 -- 172.21.15.159:0/1472092650 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e800cee30 con 0x7f3e8013f190 2024-04-04T06:07:11.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.668+0000 7f3e7e000640 1 -- 172.21.15.159:0/1472092650 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f3e70001690 con 0x7f3e8013f190 2024-04-04T06:07:11.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.668+0000 7f3e7e000640 1 -- 172.21.15.159:0/1472092650 >> v1:172.21.15.159:6789/0 conn(0x7f3e8013f190 legacy=0x7f3e800cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:11.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.668+0000 7f3e7e000640 1 --2- 172.21.15.159:0/1472092650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e640039b0 0x7f3e64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:11.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.669+0000 7f3e7e000640 1 --2- 172.21.15.159:0/1472092650 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e8006d620 0x7f3e64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:11.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.669+0000 7f3e7e000640 1 --2- 172.21.15.159:0/1472092650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e64004d40 0x7f3e640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:11.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.669+0000 7f3e7e000640 1 -- 172.21.15.159:0/1472092650 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3e800cd230 con 0x7f3e64004d40 2024-04-04T06:07:11.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.669+0000 7f3e7e000640 1 -- 172.21.15.159:0/1472092650 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3e800ceb50 con 0x7f3e640039b0 2024-04-04T06:07:11.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.669+0000 7f3e7e000640 1 -- 172.21.15.159:0/1472092650 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3e6400b9c0 con 0x7f3e8006d620 2024-04-04T06:07:11.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.669+0000 7f3e7fe00640 1 --2- 172.21.15.159:0/1472092650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e64004d40 0x7f3e640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:11.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.669+0000 7f3e7f400640 1 --2- 172.21.15.159:0/1472092650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e640039b0 0x7f3e64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:11.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.669+0000 7f3e7ea00640 1 --2- 172.21.15.159:0/1472092650 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e8006d620 0x7f3e64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:11.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.669+0000 7f3e7fe00640 1 -- 172.21.15.159:0/1472092650 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e8006d620 msgr2=0x7f3e64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:11.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.669+0000 7f3e7fe00640 1 --2- 172.21.15.159:0/1472092650 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e8006d620 0x7f3e64004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:11.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.669+0000 7f3e7fe00640 1 -- 172.21.15.159:0/1472092650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e640039b0 msgr2=0x7f3e64003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:11.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.669+0000 7f3e7fe00640 1 --2- 172.21.15.159:0/1472092650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e640039b0 0x7f3e64003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:11.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.669+0000 7f3e7fe00640 1 -- 172.21.15.159:0/1472092650 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e64003520 con 0x7f3e64004d40 2024-04-04T06:07:11.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.670+0000 7f3e7fe00640 1 --2- 172.21.15.159:0/1472092650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e64004d40 0x7f3e640051c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800e170 tx=0x7f3e64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e810e2c5b8f3f4e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:11.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.671+0000 7f3e7e000640 1 -- 172.21.15.159:0/1472092650 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78039070 con 0x7f3e64004d40 2024-04-04T06:07:11.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.671+0000 7f3e7e000640 1 -- 172.21.15.159:0/1472092650 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e7802cbf0 con 0x7f3e64004d40 2024-04-04T06:07:11.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.671+0000 7f3e7e000640 1 -- 172.21.15.159:0/1472092650 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78009c10 con 0x7f3e64004d40 2024-04-04T06:07:11.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.671+0000 7f3e85000640 1 -- 172.21.15.159:0/1472092650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e64004d40 msgr2=0x7f3e640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:11.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.671+0000 7f3e85000640 1 --2- 172.21.15.159:0/1472092650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e64004d40 0x7f3e640051c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800e170 tx=0x7f3e64005830 comp rx=0 tx=0).stop 2024-04-04T06:07:11.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.671+0000 7f3e7fe00640 1 -- 172.21.15.159:0/1472092650 reap_dead start 2024-04-04T06:07:11.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.671+0000 7f3e85000640 1 -- 172.21.15.159:0/1472092650 shutdown_connections 2024-04-04T06:07:11.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.671+0000 7f3e85000640 1 -- 172.21.15.159:0/1472092650 >> 172.21.15.159:0/1472092650 conn(0x7f3e800b1ba0 msgr2=0x7f3e800b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:11.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.671+0000 7f3e85000640 1 -- 172.21.15.159:0/1472092650 shutdown_connections 2024-04-04T06:07:11.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.671+0000 7f3e85000640 1 -- 172.21.15.159:0/1472092650 wait complete. 2024-04-04T06:07:11.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.673+0000 7f3e85000640 1 Processor -- start 2024-04-04T06:07:11.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.673+0000 7f3e85000640 1 -- start start 2024-04-04T06:07:11.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.674+0000 7f3e85000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e64004d40 0x7f3e801f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:11.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.674+0000 7f3e85000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e801f4cd0 0x7f3e80158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:11.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.674+0000 7f3e85000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e801f5130 0x7f3e80158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:11.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.674+0000 7f3e85000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8013f570 con 0x7f3e801f5130 2024-04-04T06:07:11.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.674+0000 7f3e85000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3e80160c70 con 0x7f3e801f4cd0 2024-04-04T06:07:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.674+0000 7f3e85000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3e8006da50 con 0x7f3e64004d40 2024-04-04T06:07:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.674+0000 7f3e7fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e801f5130 0x7f3e80158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.674+0000 7f3e7fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e801f5130 0x7f3e80158ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59522/0 (socket says 172.21.15.159:59522) 2024-04-04T06:07:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.674+0000 7f3e7fe00640 1 -- 172.21.15.159:0/748580506 learned_addr learned my addr 172.21.15.159:0/748580506 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.675+0000 7f3e7ea00640 1 --2- 172.21.15.159:0/748580506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e801f4cd0 0x7f3e80158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.675+0000 7f3e7f400640 1 --2- 172.21.15.159:0/748580506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e64004d40 0x7f3e801f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.675+0000 7f3e7fe00640 1 -- 172.21.15.159:0/748580506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e64004d40 msgr2=0x7f3e801f1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.675+0000 7f3e7fe00640 1 --2- 172.21.15.159:0/748580506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e64004d40 0x7f3e801f1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.675+0000 7f3e7fe00640 1 -- 172.21.15.159:0/748580506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e801f4cd0 msgr2=0x7f3e80158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.675+0000 7f3e7fe00640 1 --2- 172.21.15.159:0/748580506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e801f4cd0 0x7f3e80158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.675+0000 7f3e7fe00640 1 -- 172.21.15.159:0/748580506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e801590e0 con 0x7f3e801f5130 2024-04-04T06:07:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.675+0000 7f3e7ea00640 1 --2- 172.21.15.159:0/748580506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e801f4cd0 0x7f3e80158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.675+0000 7f3e7fe00640 1 --2- 172.21.15.159:0/748580506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e801f5130 0x7f3e80158ab0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800e170 tx=0x7f3e78009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.675+0000 7f3e77400640 1 -- 172.21.15.159:0/748580506 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78039030 con 0x7f3e801f5130 2024-04-04T06:07:11.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.675+0000 7f3e77400640 1 -- 172.21.15.159:0/748580506 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e7803a040 con 0x7f3e801f5130 2024-04-04T06:07:11.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.675+0000 7f3e85000640 1 -- 172.21.15.159:0/748580506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e801593c0 con 0x7f3e801f5130 2024-04-04T06:07:11.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.675+0000 7f3e77400640 1 -- 172.21.15.159:0/748580506 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78003990 con 0x7f3e801f5130 2024-04-04T06:07:11.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.676+0000 7f3e85000640 1 -- 172.21.15.159:0/748580506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e80159980 con 0x7f3e801f5130 2024-04-04T06:07:11.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.677+0000 7f3e85000640 1 -- 172.21.15.159:0/748580506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e80075940 con 0x7f3e801f5130 2024-04-04T06:07:11.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.678+0000 7f3e77400640 1 -- 172.21.15.159:0/748580506 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e78045030 con 0x7f3e801f5130 2024-04-04T06:07:11.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.678+0000 7f3e77400640 1 --2- 172.21.15.159:0/748580506 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3e6006c920 0x7f3e6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:11.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.679+0000 7f3e77400640 1 -- 172.21.15.159:0/748580506 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f3e780adbd0 con 0x7f3e801f5130 2024-04-04T06:07:11.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.679+0000 7f3e7f400640 1 --2- 172.21.15.159:0/748580506 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3e6006c920 0x7f3e6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:11.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.680+0000 7f3e7f400640 1 --2- 172.21.15.159:0/748580506 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3e6006c920 0x7f3e6006ede0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3e70001eb0 tx=0x7f3e70001450 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:11.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.683+0000 7f3e77400640 1 -- 172.21.15.159:0/748580506 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3e7804c020 con 0x7f3e801f5130 2024-04-04T06:07:11.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:11.925+0000 7f3e85000640 1 -- 172.21.15.159:0/748580506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3e800d0790 con 0x7f3e801f5130 2024-04-04T06:07:12.288 INFO:tasks.ceph.mds.b.smithi178.stderr:2024-04-04T06:07:12.294+0000 7fd6d6400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:07:12.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.424+0000 7f3e77400640 1 -- 172.21.15.159:0/748580506 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f3e7807bff0 con 0x7f3e801f5130 2024-04-04T06:07:12.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.427+0000 7f3e85000640 1 -- 172.21.15.159:0/748580506 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3e6006c920 msgr2=0x7f3e6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:12.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.427+0000 7f3e85000640 1 --2- 172.21.15.159:0/748580506 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3e6006c920 0x7f3e6006ede0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3e70001eb0 tx=0x7f3e70001450 comp rx=0 tx=0).stop 2024-04-04T06:07:12.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.427+0000 7f3e85000640 1 -- 172.21.15.159:0/748580506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e801f5130 msgr2=0x7f3e80158ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:12.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.427+0000 7f3e85000640 1 --2- 172.21.15.159:0/748580506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e801f5130 0x7f3e80158ab0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800e170 tx=0x7f3e78009750 comp rx=0 tx=0).stop 2024-04-04T06:07:12.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.427+0000 7f3e85000640 1 -- 172.21.15.159:0/748580506 shutdown_connections 2024-04-04T06:07:12.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.427+0000 7f3e85000640 1 --2- 172.21.15.159:0/748580506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e801f5130 0x7f3e80158ab0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:12.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.427+0000 7f3e85000640 1 --2- 172.21.15.159:0/748580506 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3e6006c920 0x7f3e6006ede0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:12.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.427+0000 7f3e85000640 1 --2- 172.21.15.159:0/748580506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e801f4cd0 0x7f3e80158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:12.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.427+0000 7f3e85000640 1 --2- 172.21.15.159:0/748580506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e64004d40 0x7f3e801f1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:12.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.427+0000 7f3e85000640 1 -- 172.21.15.159:0/748580506 >> 172.21.15.159:0/748580506 conn(0x7f3e800b1ba0 msgr2=0x7f3e8015e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:12.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.427+0000 7f3e85000640 1 -- 172.21.15.159:0/748580506 shutdown_connections 2024-04-04T06:07:12.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.427+0000 7f3e85000640 1 -- 172.21.15.159:0/748580506 wait complete. 2024-04-04T06:07:12.443 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-04T06:07:12.443 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:07:12.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.658+0000 7fc7e1600640 1 Processor -- start 2024-04-04T06:07:12.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.658+0000 7fc7e1600640 1 -- start start 2024-04-04T06:07:12.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.660+0000 7fc7e1600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc7dc0c8040 0x7fc7dc15fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:12.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.660+0000 7fc7e1600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7dc0b4c00 con 0x7fc7dc06c680 2024-04-04T06:07:12.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.660+0000 7fc7e1600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7dc0c7560 con 0x7fc7dc0c84a0 2024-04-04T06:07:12.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.660+0000 7fc7e1600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fc7dc0c7310 con 0x7fc7dc0c8040 2024-04-04T06:07:12.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.660+0000 7fc7da000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc7dc0c8040 0x7fc7dc15fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:12.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.660+0000 7fc7daa00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fc7dc06c680 0x7fc7dc06ca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52478/0 (socket says 172.21.15.159:52478) 2024-04-04T06:07:12.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.660+0000 7fc7daa00640 1 -- 172.21.15.159:0/1761898920 learned_addr learned my addr 172.21.15.159:0/1761898920 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:12.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.661+0000 7fc7da000640 1 -- 172.21.15.159:0/1761898920 >> v1:172.21.15.178:6789/0 conn(0x7fc7dc0c84a0 legacy=0x7fc7dc0b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:12.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.661+0000 7fc7da000640 1 -- 172.21.15.159:0/1761898920 >> v1:172.21.15.159:6789/0 conn(0x7fc7dc06c680 legacy=0x7fc7dc06ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:12.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.661+0000 7fc7da000640 1 -- 172.21.15.159:0/1761898920 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7dc0c7840 con 0x7fc7dc0c8040 2024-04-04T06:07:12.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.662+0000 7fc7da000640 1 --2- 172.21.15.159:0/1761898920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7dc0c8040 0x7fc7dc15fd30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80097a0 tx=0x7fc7c802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a805a25773b66d55 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:12.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.662+0000 7fc7d9600640 1 -- 172.21.15.159:0/1761898920 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8035020 con 0x7fc7dc0c8040 2024-04-04T06:07:12.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.662+0000 7fc7d9600640 1 -- 172.21.15.159:0/1761898920 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7c8031830 con 0x7fc7dc0c8040 2024-04-04T06:07:12.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.662+0000 7fc7d9600640 1 -- 172.21.15.159:0/1761898920 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8031c00 con 0x7fc7dc0c8040 2024-04-04T06:07:12.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.663+0000 7fc7e1600640 1 -- 172.21.15.159:0/1761898920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7dc0c8040 msgr2=0x7fc7dc15fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:12.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.663+0000 7fc7e1600640 1 --2- 172.21.15.159:0/1761898920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7dc0c8040 0x7fc7dc15fd30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80097a0 tx=0x7fc7c802b8d0 comp rx=0 tx=0).stop 2024-04-04T06:07:12.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.663+0000 7fc7e1600640 1 -- 172.21.15.159:0/1761898920 shutdown_connections 2024-04-04T06:07:12.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.663+0000 7fc7e1600640 1 --2- 172.21.15.159:0/1761898920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7dc0c8040 0x7fc7dc15fd30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:12.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.663+0000 7fc7e1600640 1 -- 172.21.15.159:0/1761898920 >> 172.21.15.159:0/1761898920 conn(0x7fc7dc0d2240 msgr2=0x7fc7dc0d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:12.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.663+0000 7fc7e1600640 1 -- 172.21.15.159:0/1761898920 shutdown_connections 2024-04-04T06:07:12.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.663+0000 7fc7e1600640 1 -- 172.21.15.159:0/1761898920 wait complete. 2024-04-04T06:07:12.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.665+0000 7fc7e1600640 1 Processor -- start 2024-04-04T06:07:12.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.665+0000 7fc7e1600640 1 -- start start 2024-04-04T06:07:12.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.666+0000 7fc7e1600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7dc06c680 0x7fc7dc1430b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:12.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.666+0000 7fc7e1600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7dc0c8040 0x7fc7dc0c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:12.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.667+0000 7fc7e1600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7dc0c84a0 0x7fc7dc13e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:12.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.667+0000 7fc7e1600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc1481b0 con 0x7fc7dc06c680 2024-04-04T06:07:12.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.667+0000 7fc7e1600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc148030 con 0x7fc7dc0c84a0 2024-04-04T06:07:12.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.667+0000 7fc7e1600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc7dc0b45a0 con 0x7fc7dc0c8040 2024-04-04T06:07:12.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.667+0000 7fc7daa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7dc06c680 0x7fc7dc1430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:12.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.667+0000 7fc7daa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7dc06c680 0x7fc7dc1430b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59538/0 (socket says 172.21.15.159:59538) 2024-04-04T06:07:12.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.667+0000 7fc7daa00640 1 -- 172.21.15.159:0/1027600739 learned_addr learned my addr 172.21.15.159:0/1027600739 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:12.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.667+0000 7fc7da000640 1 --2- 172.21.15.159:0/1027600739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7dc0c8040 0x7fc7dc0c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:12.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.667+0000 7fc7daa00640 1 -- 172.21.15.159:0/1027600739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7dc0c8040 msgr2=0x7fc7dc0c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:12.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.667+0000 7fc7daa00640 1 --2- 172.21.15.159:0/1027600739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7dc0c8040 0x7fc7dc0c9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:12.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.667+0000 7fc7daa00640 1 -- 172.21.15.159:0/1027600739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7dc0c84a0 msgr2=0x7fc7dc13e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:12.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.667+0000 7fc7daa00640 1 --2- 172.21.15.159:0/1027600739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7dc0c84a0 0x7fc7dc13e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:12.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.667+0000 7fc7daa00640 1 -- 172.21.15.159:0/1027600739 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7dc143690 con 0x7fc7dc06c680 2024-04-04T06:07:12.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.668+0000 7fc7daa00640 1 --2- 172.21.15.159:0/1027600739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7dc06c680 0x7fc7dc1430b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00c430 tx=0x7fc7cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:12.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.668+0000 7fc7d2a00640 1 -- 172.21.15.159:0/1027600739 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc017070 con 0x7fc7dc06c680 2024-04-04T06:07:12.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.668+0000 7fc7d2a00640 1 -- 172.21.15.159:0/1027600739 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7cc005ce0 con 0x7fc7dc06c680 2024-04-04T06:07:12.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.668+0000 7fc7d2a00640 1 -- 172.21.15.159:0/1027600739 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc0072f0 con 0x7fc7dc06c680 2024-04-04T06:07:12.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.668+0000 7fc7e1600640 1 -- 172.21.15.159:0/1027600739 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7dc143970 con 0x7fc7dc06c680 2024-04-04T06:07:12.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.668+0000 7fc7e1600640 1 -- 172.21.15.159:0/1027600739 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7dc143f30 con 0x7fc7dc06c680 2024-04-04T06:07:12.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.669+0000 7fc7e1600640 1 -- 172.21.15.159:0/1027600739 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7dc0685c0 con 0x7fc7dc06c680 2024-04-04T06:07:12.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.670+0000 7fc7d2a00640 1 -- 172.21.15.159:0/1027600739 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7cc01a070 con 0x7fc7dc06c680 2024-04-04T06:07:12.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.671+0000 7fc7d2a00640 1 --2- 172.21.15.159:0/1027600739 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc7bc06c8f0 0x7fc7bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:12.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.671+0000 7fc7d2a00640 1 -- 172.21.15.159:0/1027600739 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fc7cc08afa0 con 0x7fc7dc06c680 2024-04-04T06:07:12.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.672+0000 7fc7da000640 1 --2- 172.21.15.159:0/1027600739 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc7bc06c8f0 0x7fc7bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:12.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.673+0000 7fc7da000640 1 --2- 172.21.15.159:0/1027600739 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc7bc06c8f0 0x7fc7bc06edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8036000 tx=0x7fc7c8008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:12.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.676+0000 7fc7d2a00640 1 -- 172.21.15.159:0/1027600739 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc7cc0593c0 con 0x7fc7dc06c680 2024-04-04T06:07:12.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.921+0000 7fc7e1600640 1 -- 172.21.15.159:0/1027600739 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7dc0d02b0 con 0x7fc7dc06c680 2024-04-04T06:07:12.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.922+0000 7fc7d2a00640 1 -- 172.21.15.159:0/1027600739 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5193 (secure 0 0 0) 0x7fc7cc058cc0 con 0x7fc7dc06c680 2024-04-04T06:07:12.917 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:12.917 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1643905381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1643905381},{"type":"v1","addr":"172.21.15.178:6837","nonce":1643905381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4744,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2629539137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2629539137},{"type":"v1","addr":"172.21.15.159:6835","nonce":2629539137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3402172592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3402172592},{"type":"v1","addr":"172.21.15.159:6833","nonce":3402172592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:07:08.839645+0000","modified":"2024-04-04T06:07:11.927601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5157},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5157":{"gid":5157,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.178:6836/3282963883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3282963883},{"type":"v1","addr":"172.21.15.178:6836","nonce":3282963883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5157,"qdb_cluster":[5157]},"id":2}]} 2024-04-04T06:07:12.917 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 17 2024-04-04T06:07:12.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.925+0000 7fc7e1600640 1 -- 172.21.15.159:0/1027600739 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc7bc06c8f0 msgr2=0x7fc7bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:12.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.925+0000 7fc7e1600640 1 --2- 172.21.15.159:0/1027600739 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc7bc06c8f0 0x7fc7bc06edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8036000 tx=0x7fc7c8008030 comp rx=0 tx=0).stop 2024-04-04T06:07:12.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.925+0000 7fc7e1600640 1 -- 172.21.15.159:0/1027600739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7dc06c680 msgr2=0x7fc7dc1430b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:12.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.925+0000 7fc7e1600640 1 --2- 172.21.15.159:0/1027600739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7dc06c680 0x7fc7dc1430b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00c430 tx=0x7fc7cc00c8f0 comp rx=0 tx=0).stop 2024-04-04T06:07:12.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.925+0000 7fc7e1600640 1 -- 172.21.15.159:0/1027600739 shutdown_connections 2024-04-04T06:07:12.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.925+0000 7fc7e1600640 1 --2- 172.21.15.159:0/1027600739 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc7bc06c8f0 0x7fc7bc06edb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:12.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.925+0000 7fc7e1600640 1 --2- 172.21.15.159:0/1027600739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7dc0c84a0 0x7fc7dc13e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:12.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.925+0000 7fc7e1600640 1 --2- 172.21.15.159:0/1027600739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7dc0c8040 0x7fc7dc0c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:12.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.925+0000 7fc7e1600640 1 --2- 172.21.15.159:0/1027600739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7dc06c680 0x7fc7dc1430b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:12.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.925+0000 7fc7e1600640 1 -- 172.21.15.159:0/1027600739 >> 172.21.15.159:0/1027600739 conn(0x7fc7dc0d2240 msgr2=0x7fc7dc146ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:12.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.925+0000 7fc7e1600640 1 -- 172.21.15.159:0/1027600739 shutdown_connections 2024-04-04T06:07:12.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:12.926+0000 7fc7e1600640 1 -- 172.21.15.159:0/1027600739 wait complete. 2024-04-04T06:07:12.939 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:07:13.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.151+0000 7fef81000640 1 Processor -- start 2024-04-04T06:07:13.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.151+0000 7fef81000640 1 -- start start 2024-04-04T06:07:13.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.152+0000 7fef81000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fef7c0cc0b0 0x7fef7c06dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:13.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.152+0000 7fef81000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fef7c15fc50 0x7fef7c0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:13.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.152+0000 7fef81000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef7c0cc490 0x7fef7c0d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:13.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.152+0000 7fef81000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fef7c15c610 con 0x7fef7c15fc50 2024-04-04T06:07:13.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.152+0000 7fef81000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fef7c06e4f0 con 0x7fef7c0cc0b0 2024-04-04T06:07:13.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.152+0000 7fef81000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fef7c0b3150 con 0x7fef7c0cc490 2024-04-04T06:07:13.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.152+0000 7fef72a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fef7c15fc50 0x7fef7c0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:13.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.152+0000 7fef72a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fef7c15fc50 0x7fef7c0cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59554/0 (socket says 172.21.15.159:59554) 2024-04-04T06:07:13.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.152+0000 7fef72a00640 1 -- 172.21.15.159:0/1037020891 learned_addr learned my addr 172.21.15.159:0/1037020891 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:13.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.153+0000 7fef7b400640 1 --2- 172.21.15.159:0/1037020891 >> v2:172.21.15.178:3300/0 conn(0x7fef7c0cc0b0 0x7fef7c06dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:13.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.153+0000 7fef7be00640 1 --2- 172.21.15.159:0/1037020891 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef7c0cc490 0x7fef7c0d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:13.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.153+0000 7fef72a00640 1 -- 172.21.15.159:0/1037020891 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef7c0cc490 msgr2=0x7fef7c0d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:13.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.153+0000 7fef72a00640 1 --2- 172.21.15.159:0/1037020891 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef7c0cc490 0x7fef7c0d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.153+0000 7fef72a00640 1 -- 172.21.15.159:0/1037020891 >> v2:172.21.15.178:3300/0 conn(0x7fef7c0cc0b0 msgr2=0x7fef7c06dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:13.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.153+0000 7fef72a00640 1 --2- 172.21.15.159:0/1037020891 >> v2:172.21.15.178:3300/0 conn(0x7fef7c0cc0b0 0x7fef7c06dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.153+0000 7fef72a00640 1 -- 172.21.15.159:0/1037020891 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c1600e0 con 0x7fef7c15fc50 2024-04-04T06:07:13.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.154+0000 7fef72a00640 1 --2- 172.21.15.159:0/1037020891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef7c15fc50 0x7fef7c0cfdc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0cfd10 tx=0x7fef6402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b593d31b2572c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:13.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.154+0000 7fef7aa00640 1 -- 172.21.15.159:0/1037020891 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64035020 con 0x7fef7c15fc50 2024-04-04T06:07:13.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.154+0000 7fef7aa00640 1 -- 172.21.15.159:0/1037020891 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef6402d040 con 0x7fef7c15fc50 2024-04-04T06:07:13.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.154+0000 7fef7aa00640 1 -- 172.21.15.159:0/1037020891 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64031a80 con 0x7fef7c15fc50 2024-04-04T06:07:13.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.155+0000 7fef81000640 1 -- 172.21.15.159:0/1037020891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef7c15fc50 msgr2=0x7fef7c0cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:13.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.155+0000 7fef81000640 1 --2- 172.21.15.159:0/1037020891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef7c15fc50 0x7fef7c0cfdc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0cfd10 tx=0x7fef6402b900 comp rx=0 tx=0).stop 2024-04-04T06:07:13.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.155+0000 7fef81000640 1 -- 172.21.15.159:0/1037020891 shutdown_connections 2024-04-04T06:07:13.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.155+0000 7fef81000640 1 --2- 172.21.15.159:0/1037020891 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef7c0cc490 0x7fef7c0d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.155+0000 7fef81000640 1 --2- 172.21.15.159:0/1037020891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef7c15fc50 0x7fef7c0cfdc0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.155+0000 7fef81000640 1 --2- 172.21.15.159:0/1037020891 >> v2:172.21.15.178:3300/0 conn(0x7fef7c0cc0b0 0x7fef7c06dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.155+0000 7fef81000640 1 -- 172.21.15.159:0/1037020891 >> 172.21.15.159:0/1037020891 conn(0x7fef7c0ccad0 msgr2=0x7fef7c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:13.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.155+0000 7fef81000640 1 -- 172.21.15.159:0/1037020891 shutdown_connections 2024-04-04T06:07:13.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.155+0000 7fef81000640 1 -- 172.21.15.159:0/1037020891 wait complete. 2024-04-04T06:07:13.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.157+0000 7fef81000640 1 Processor -- start 2024-04-04T06:07:13.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.157+0000 7fef81000640 1 -- start start 2024-04-04T06:07:13.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.158+0000 7fef81000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef7c0cc0b0 0x7fef7c12d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:13.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.158+0000 7fef81000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef7c0cc490 0x7fef7c12b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:13.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.158+0000 7fef81000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef7c12b690 0x7fef7c155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:13.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.158+0000 7fef81000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c160980 con 0x7fef7c0cc490 2024-04-04T06:07:13.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.158+0000 7fef81000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1606a0 con 0x7fef7c12b690 2024-04-04T06:07:13.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.158+0000 7fef81000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fef7c15ca70 con 0x7fef7c0cc0b0 2024-04-04T06:07:13.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.158+0000 7fef72a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef7c0cc490 0x7fef7c12b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:13.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.158+0000 7fef72a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef7c0cc490 0x7fef7c12b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59556/0 (socket says 172.21.15.159:59556) 2024-04-04T06:07:13.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.158+0000 7fef72a00640 1 -- 172.21.15.159:0/1479635184 learned_addr learned my addr 172.21.15.159:0/1479635184 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:13.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.158+0000 7fef7b400640 1 --2- 172.21.15.159:0/1479635184 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef7c0cc0b0 0x7fef7c12d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:13.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.158+0000 7fef72a00640 1 -- 172.21.15.159:0/1479635184 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef7c0cc0b0 msgr2=0x7fef7c12d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:13.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.158+0000 7fef72a00640 1 --2- 172.21.15.159:0/1479635184 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef7c0cc0b0 0x7fef7c12d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.158+0000 7fef72a00640 1 -- 172.21.15.159:0/1479635184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef7c12b690 msgr2=0x7fef7c155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:13.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.158+0000 7fef72a00640 1 --2- 172.21.15.159:0/1479635184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef7c12b690 0x7fef7c155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.158+0000 7fef72a00640 1 -- 172.21.15.159:0/1479635184 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c12bd30 con 0x7fef7c0cc490 2024-04-04T06:07:13.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.159+0000 7fef72a00640 1 --2- 172.21.15.159:0/1479635184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef7c0cc490 0x7fef7c12b150 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fef6402b940 tx=0x7fef640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:13.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.159+0000 7fef73e00640 1 -- 172.21.15.159:0/1479635184 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64040070 con 0x7fef7c0cc490 2024-04-04T06:07:13.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.159+0000 7fef73e00640 1 -- 172.21.15.159:0/1479635184 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef64039bf0 con 0x7fef7c0cc490 2024-04-04T06:07:13.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.159+0000 7fef73e00640 1 -- 172.21.15.159:0/1479635184 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64035040 con 0x7fef7c0cc490 2024-04-04T06:07:13.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.159+0000 7fef81000640 1 -- 172.21.15.159:0/1479635184 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef7c12c010 con 0x7fef7c0cc490 2024-04-04T06:07:13.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.159+0000 7fef81000640 1 -- 172.21.15.159:0/1479635184 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef7c12c5d0 con 0x7fef7c0cc490 2024-04-04T06:07:13.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.160+0000 7fef81000640 1 -- 172.21.15.159:0/1479635184 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef7c12bac0 con 0x7fef7c0cc490 2024-04-04T06:07:13.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.161+0000 7fef73e00640 1 -- 172.21.15.159:0/1479635184 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef6404b060 con 0x7fef7c0cc490 2024-04-04T06:07:13.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.162+0000 7fef73e00640 1 --2- 172.21.15.159:0/1479635184 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fef5806c920 0x7fef5806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:13.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.162+0000 7fef73e00640 1 -- 172.21.15.159:0/1479635184 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fef640ad730 con 0x7fef7c0cc490 2024-04-04T06:07:13.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.163+0000 7fef7b400640 1 --2- 172.21.15.159:0/1479635184 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fef5806c920 0x7fef5806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:13.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.164+0000 7fef7b400640 1 --2- 172.21.15.159:0/1479635184 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fef5806c920 0x7fef5806ede0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fef6c006fd0 tx=0x7fef6c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:13.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.166+0000 7fef73e00640 1 -- 172.21.15.159:0/1479635184 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fef6407bac0 con 0x7fef7c0cc490 2024-04-04T06:07:13.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.352+0000 7fef81000640 1 -- 172.21.15.159:0/1479635184 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef7c075940 con 0x7fef7c0cc490 2024-04-04T06:07:13.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.355+0000 7fef73e00640 1 -- 172.21.15.159:0/1479635184 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16681 (secure 0 0 0) 0x7fef6407b3c0 con 0x7fef7c0cc490 2024-04-04T06:07:13.350 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:13.350 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":34,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:07:10.401521+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:07:06.708548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-04T06:07:07.833730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:07:13.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.357+0000 7fef81000640 1 -- 172.21.15.159:0/1479635184 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fef5806c920 msgr2=0x7fef5806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:13.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.357+0000 7fef81000640 1 --2- 172.21.15.159:0/1479635184 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fef5806c920 0x7fef5806ede0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fef6c006fd0 tx=0x7fef6c008040 comp rx=0 tx=0).stop 2024-04-04T06:07:13.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.357+0000 7fef81000640 1 -- 172.21.15.159:0/1479635184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef7c0cc490 msgr2=0x7fef7c12b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:13.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.357+0000 7fef81000640 1 --2- 172.21.15.159:0/1479635184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef7c0cc490 0x7fef7c12b150 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fef6402b940 tx=0x7fef640046c0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.357+0000 7fef81000640 1 -- 172.21.15.159:0/1479635184 shutdown_connections 2024-04-04T06:07:13.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.357+0000 7fef81000640 1 --2- 172.21.15.159:0/1479635184 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fef5806c920 0x7fef5806ede0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.357+0000 7fef81000640 1 --2- 172.21.15.159:0/1479635184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef7c12b690 0x7fef7c155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.357+0000 7fef81000640 1 --2- 172.21.15.159:0/1479635184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef7c0cc490 0x7fef7c12b150 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.357+0000 7fef81000640 1 --2- 172.21.15.159:0/1479635184 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef7c0cc0b0 0x7fef7c12d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.357+0000 7fef81000640 1 -- 172.21.15.159:0/1479635184 >> 172.21.15.159:0/1479635184 conn(0x7fef7c0ccad0 msgr2=0x7fef7c0b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:13.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.357+0000 7fef81000640 1 -- 172.21.15.159:0/1479635184 shutdown_connections 2024-04-04T06:07:13.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.358+0000 7fef81000640 1 -- 172.21.15.159:0/1479635184 wait complete. 2024-04-04T06:07:13.371 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-04T06:07:13.371 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:07:13.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.591+0000 7fca73e00640 1 Processor -- start 2024-04-04T06:07:13.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.591+0000 7fca73e00640 1 -- start start 2024-04-04T06:07:13.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.592+0000 7fca73e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca7416af40 0x7fca74071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:13.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.592+0000 7fca73e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fca740721e0 0x7fca7406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:13.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.592+0000 7fca73e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca74072f00 con 0x7fca7406db10 2024-04-04T06:07:13.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.592+0000 7fca73e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fca7406df80 con 0x7fca740721e0 2024-04-04T06:07:13.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.592+0000 7fca73e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fca74072660 con 0x7fca7416af40 2024-04-04T06:07:13.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.593+0000 7fca6a000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fca740721e0 0x7fca7406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:13.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.593+0000 7fca6a000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fca740721e0 0x7fca7406d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:38100/0 (socket says 172.21.15.159:38100) 2024-04-04T06:07:13.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.593+0000 7fca6a000640 1 -- 172.21.15.159:0/2883941901 learned_addr learned my addr 172.21.15.159:0/2883941901 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:13.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.594+0000 7fca72000640 1 -- 172.21.15.159:0/2883941901 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3941996795 0 0) 0x7fca74072f00 con 0x7fca7406db10 2024-04-04T06:07:13.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.593+0000 7fca72a00640 1 --2- 172.21.15.159:0/2883941901 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca7416af40 0x7fca74071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:13.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.594+0000 7fca72000640 1 -- 172.21.15.159:0/2883941901 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca58003500 con 0x7fca7406db10 2024-04-04T06:07:13.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.595+0000 7fca6a000640 1 -- 172.21.15.159:0/2883941901 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca7416af40 msgr2=0x7fca74071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:13.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.595+0000 7fca6a000640 1 --2- 172.21.15.159:0/2883941901 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca7416af40 0x7fca74071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.595+0000 7fca6a000640 1 -- 172.21.15.159:0/2883941901 >> v1:172.21.15.178:6789/0 conn(0x7fca7406db10 legacy=0x7fca7416eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:13.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.595+0000 7fca6a000640 1 -- 172.21.15.159:0/2883941901 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca74072940 con 0x7fca740721e0 2024-04-04T06:07:13.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.596+0000 7fca6a000640 1 --2- 172.21.15.159:0/2883941901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca740721e0 0x7fca7406d5d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fca5c008120 tx=0x7fca5c02bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=698edfa01fbd38c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:13.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.597+0000 7fca72000640 1 -- 172.21.15.159:0/2883941901 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c00b040 con 0x7fca740721e0 2024-04-04T06:07:13.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.597+0000 7fca72000640 1 -- 172.21.15.159:0/2883941901 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca5c004230 con 0x7fca740721e0 2024-04-04T06:07:13.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.597+0000 7fca72000640 1 -- 172.21.15.159:0/2883941901 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c02c650 con 0x7fca740721e0 2024-04-04T06:07:13.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.598+0000 7fca73e00640 1 -- 172.21.15.159:0/2883941901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca740721e0 msgr2=0x7fca7406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:13.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.598+0000 7fca73e00640 1 --2- 172.21.15.159:0/2883941901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca740721e0 0x7fca7406d5d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fca5c008120 tx=0x7fca5c02bcb0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.598+0000 7fca73e00640 1 -- 172.21.15.159:0/2883941901 shutdown_connections 2024-04-04T06:07:13.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.598+0000 7fca73e00640 1 --2- 172.21.15.159:0/2883941901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca740721e0 0x7fca7406d5d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.598+0000 7fca73e00640 1 --2- 172.21.15.159:0/2883941901 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca7416af40 0x7fca74071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.598+0000 7fca73e00640 1 -- 172.21.15.159:0/2883941901 >> 172.21.15.159:0/2883941901 conn(0x7fca740b1ba0 msgr2=0x7fca740b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:13.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.598+0000 7fca73e00640 1 -- 172.21.15.159:0/2883941901 shutdown_connections 2024-04-04T06:07:13.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.598+0000 7fca73e00640 1 -- 172.21.15.159:0/2883941901 wait complete. 2024-04-04T06:07:13.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.600+0000 7fca73e00640 1 Processor -- start 2024-04-04T06:07:13.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.600+0000 7fca73e00640 1 -- start start 2024-04-04T06:07:13.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.601+0000 7fca73e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca7406db10 0x7fca74163dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:13.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.601+0000 7fca73e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca740721e0 0x7fca741f5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:13.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.601+0000 7fca73e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca7416af40 0x7fca741f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:13.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.601+0000 7fca73e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fca7406e250 con 0x7fca7406db10 2024-04-04T06:07:13.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.601+0000 7fca73e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fca7416b320 con 0x7fca7416af40 2024-04-04T06:07:13.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.601+0000 7fca73e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fca7406e3d0 con 0x7fca740721e0 2024-04-04T06:07:13.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.601+0000 7fca72a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca7406db10 0x7fca74163dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:13.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.601+0000 7fca72a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca7406db10 0x7fca74163dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59564/0 (socket says 172.21.15.159:59564) 2024-04-04T06:07:13.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.601+0000 7fca72a00640 1 -- 172.21.15.159:0/691276444 learned_addr learned my addr 172.21.15.159:0/691276444 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:13.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.601+0000 7fca73400640 1 --2- 172.21.15.159:0/691276444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca7416af40 0x7fca741f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:13.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.601+0000 7fca6a000640 1 --2- 172.21.15.159:0/691276444 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca740721e0 0x7fca741f5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:13.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.601+0000 7fca72a00640 1 -- 172.21.15.159:0/691276444 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca740721e0 msgr2=0x7fca741f5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:13.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.601+0000 7fca72a00640 1 --2- 172.21.15.159:0/691276444 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca740721e0 0x7fca741f5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.601+0000 7fca72a00640 1 -- 172.21.15.159:0/691276444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca7416af40 msgr2=0x7fca741f60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:13.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.601+0000 7fca72a00640 1 --2- 172.21.15.159:0/691276444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca7416af40 0x7fca741f60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.601+0000 7fca72a00640 1 -- 172.21.15.159:0/691276444 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca741f4fb0 con 0x7fca7406db10 2024-04-04T06:07:13.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.601+0000 7fca73400640 1 --2- 172.21.15.159:0/691276444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca7416af40 0x7fca741f60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:13.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.602+0000 7fca72a00640 1 --2- 172.21.15.159:0/691276444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca7406db10 0x7fca74163dd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fca6400fa80 tx=0x7fca6400fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:13.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.602+0000 7fca6b400640 1 -- 172.21.15.159:0/691276444 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64017070 con 0x7fca7406db10 2024-04-04T06:07:13.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.602+0000 7fca6b400640 1 -- 172.21.15.159:0/691276444 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca64007a10 con 0x7fca7406db10 2024-04-04T06:07:13.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.602+0000 7fca6b400640 1 -- 172.21.15.159:0/691276444 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6400d040 con 0x7fca7406db10 2024-04-04T06:07:13.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.602+0000 7fca73e00640 1 -- 172.21.15.159:0/691276444 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca740b3e10 con 0x7fca7406db10 2024-04-04T06:07:13.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.602+0000 7fca73e00640 1 -- 172.21.15.159:0/691276444 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca741f49f0 con 0x7fca7406db10 2024-04-04T06:07:13.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.603+0000 7fca73e00640 1 -- 172.21.15.159:0/691276444 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca74075940 con 0x7fca7406db10 2024-04-04T06:07:13.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.604+0000 7fca6b400640 1 -- 172.21.15.159:0/691276444 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca6401a070 con 0x7fca7406db10 2024-04-04T06:07:13.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.605+0000 7fca6b400640 1 --2- 172.21.15.159:0/691276444 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fca5006c8f0 0x7fca5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:13.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.605+0000 7fca6b400640 1 -- 172.21.15.159:0/691276444 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fca6408bde0 con 0x7fca7406db10 2024-04-04T06:07:13.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.606+0000 7fca6a000640 1 --2- 172.21.15.159:0/691276444 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fca5006c8f0 0x7fca5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:13.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.607+0000 7fca6a000640 1 --2- 172.21.15.159:0/691276444 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fca5006c8f0 0x7fca5006edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fca5c035000 tx=0x7fca5c030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:13.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.610+0000 7fca6b400640 1 -- 172.21.15.159:0/691276444 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fca6405a200 con 0x7fca7406db10 2024-04-04T06:07:13.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.806+0000 7fca73e00640 1 -- 172.21.15.159:0/691276444 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fca7416e8f0 con 0x7fca5006c8f0 2024-04-04T06:07:13.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.826+0000 7fca6b400640 1 -- 172.21.15.159:0/691276444 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7fca7416e8f0 con 0x7fca5006c8f0 2024-04-04T06:07:13.824 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:13.825 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:07:13.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.831+0000 7fca73e00640 1 -- 172.21.15.159:0/691276444 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fca5006c8f0 msgr2=0x7fca5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:13.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.831+0000 7fca73e00640 1 --2- 172.21.15.159:0/691276444 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fca5006c8f0 0x7fca5006edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fca5c035000 tx=0x7fca5c030000 comp rx=0 tx=0).stop 2024-04-04T06:07:13.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.831+0000 7fca73e00640 1 -- 172.21.15.159:0/691276444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca7406db10 msgr2=0x7fca74163dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:13.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.831+0000 7fca73e00640 1 --2- 172.21.15.159:0/691276444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca7406db10 0x7fca74163dd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fca6400fa80 tx=0x7fca6400fb70 comp rx=0 tx=0).stop 2024-04-04T06:07:13.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.831+0000 7fca73e00640 1 -- 172.21.15.159:0/691276444 shutdown_connections 2024-04-04T06:07:13.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.831+0000 7fca73e00640 1 --2- 172.21.15.159:0/691276444 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fca5006c8f0 0x7fca5006edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.831+0000 7fca73e00640 1 --2- 172.21.15.159:0/691276444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca7416af40 0x7fca741f60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.831+0000 7fca73e00640 1 --2- 172.21.15.159:0/691276444 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fca740721e0 0x7fca741f5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.831+0000 7fca73e00640 1 --2- 172.21.15.159:0/691276444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca7406db10 0x7fca74163dd0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:13.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.831+0000 7fca73e00640 1 -- 172.21.15.159:0/691276444 >> 172.21.15.159:0/691276444 conn(0x7fca740b1ba0 msgr2=0x7fca7416cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:13.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.831+0000 7fca73e00640 1 -- 172.21.15.159:0/691276444 shutdown_connections 2024-04-04T06:07:13.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:13.831+0000 7fca73e00640 1 -- 172.21.15.159:0/691276444 wait complete. 2024-04-04T06:07:13.846 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-04-04T06:07:12.737952+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236968,"kb_used_data":4144,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737752,"statfs":{"total":773094113280,"available":772851458048,"internally_reserved":0,"allocated":4243456,"data_stored":2360868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.384537"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126160+0000","last_change":"2024-04-04T06:07:09.415112+0000","last_active":"2024-04-04T06:07:11.126160+0000","last_peered":"2024-04-04T06:07:11.126160+0000","last_clean":"2024-04-04T06:07:11.126160+0000","last_became_active":"2024-04-04T06:07:09.414812+0000","last_became_peered":"2024-04-04T06:07:09.414812+0000","last_unstale":"2024-04-04T06:07:11.126160+0000","last_undegraded":"2024-04-04T06:07:11.126160+0000","last_fullsized":"2024-04-04T06:07:11.126160+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:04:36.434377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.128487+0000","last_change":"2024-04-04T06:07:09.424013+0000","last_active":"2024-04-04T06:07:11.128487+0000","last_peered":"2024-04-04T06:07:11.128487+0000","last_clean":"2024-04-04T06:07:11.128487+0000","last_became_active":"2024-04-04T06:07:09.423100+0000","last_became_peered":"2024-04-04T06:07:09.423100+0000","last_unstale":"2024-04-04T06:07:11.128487+0000","last_undegraded":"2024-04-04T06:07:11.128487+0000","last_fullsized":"2024-04-04T06:07:11.128487+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:38:33.849951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.420075+0000","last_change":"2024-04-04T06:07:09.429037+0000","last_active":"2024-04-04T06:07:10.420075+0000","last_peered":"2024-04-04T06:07:10.420075+0000","last_clean":"2024-04-04T06:07:10.420075+0000","last_became_active":"2024-04-04T06:07:09.428784+0000","last_became_peered":"2024-04-04T06:07:09.428784+0000","last_unstale":"2024-04-04T06:07:10.420075+0000","last_undegraded":"2024-04-04T06:07:10.420075+0000","last_fullsized":"2024-04-04T06:07:10.420075+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:14:07.097099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417626+0000","last_change":"2024-04-04T06:07:09.425498+0000","last_active":"2024-04-04T06:07:10.417626+0000","last_peered":"2024-04-04T06:07:10.417626+0000","last_clean":"2024-04-04T06:07:10.417626+0000","last_became_active":"2024-04-04T06:07:09.425232+0000","last_became_peered":"2024-04-04T06:07:09.425232+0000","last_unstale":"2024-04-04T06:07:10.417626+0000","last_undegraded":"2024-04-04T06:07:10.417626+0000","last_fullsized":"2024-04-04T06:07:10.417626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:08:47.952291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.127199+0000","last_change":"2024-04-04T06:07:09.417405+0000","last_active":"2024-04-04T06:07:11.127199+0000","last_peered":"2024-04-04T06:07:11.127199+0000","last_clean":"2024-04-04T06:07:11.127199+0000","last_became_active":"2024-04-04T06:07:09.417047+0000","last_became_peered":"2024-04-04T06:07:09.417047+0000","last_unstale":"2024-04-04T06:07:11.127199+0000","last_undegraded":"2024-04-04T06:07:11.127199+0000","last_fullsized":"2024-04-04T06:07:11.127199+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:44:15.339915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126401+0000","last_change":"2024-04-04T06:07:09.406370+0000","last_active":"2024-04-04T06:07:11.126401+0000","last_peered":"2024-04-04T06:07:11.126401+0000","last_clean":"2024-04-04T06:07:11.126401+0000","last_became_active":"2024-04-04T06:07:09.405814+0000","last_became_peered":"2024-04-04T06:07:09.405814+0000","last_unstale":"2024-04-04T06:07:11.126401+0000","last_undegraded":"2024-04-04T06:07:11.126401+0000","last_fullsized":"2024-04-04T06:07:11.126401+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:57:22.011487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.667703+0000","last_change":"2024-04-04T06:07:09.431344+0000","last_active":"2024-04-04T06:07:10.667703+0000","last_peered":"2024-04-04T06:07:10.667703+0000","last_clean":"2024-04-04T06:07:10.667703+0000","last_became_active":"2024-04-04T06:07:09.431041+0000","last_became_peered":"2024-04-04T06:07:09.431041+0000","last_unstale":"2024-04-04T06:07:10.667703+0000","last_undegraded":"2024-04-04T06:07:10.667703+0000","last_fullsized":"2024-04-04T06:07:10.667703+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:21:22.868588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.666769+0000","last_change":"2024-04-04T06:07:09.424869+0000","last_active":"2024-04-04T06:07:10.666769+0000","last_peered":"2024-04-04T06:07:10.666769+0000","last_clean":"2024-04-04T06:07:10.666769+0000","last_became_active":"2024-04-04T06:07:09.424384+0000","last_became_peered":"2024-04-04T06:07:09.424384+0000","last_unstale":"2024-04-04T06:07:10.666769+0000","last_undegraded":"2024-04-04T06:07:10.666769+0000","last_fullsized":"2024-04-04T06:07:10.666769+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:54:36.113422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418995+0000","last_change":"2024-04-04T06:07:09.420985+0000","last_active":"2024-04-04T06:07:10.418995+0000","last_peered":"2024-04-04T06:07:10.418995+0000","last_clean":"2024-04-04T06:07:10.418995+0000","last_became_active":"2024-04-04T06:07:09.420629+0000","last_became_peered":"2024-04-04T06:07:09.420629+0000","last_unstale":"2024-04-04T06:07:10.418995+0000","last_undegraded":"2024-04-04T06:07:10.418995+0000","last_fullsized":"2024-04-04T06:07:10.418995+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:04:43.784625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417424+0000","last_change":"2024-04-04T06:07:09.422931+0000","last_active":"2024-04-04T06:07:10.417424+0000","last_peered":"2024-04-04T06:07:10.417424+0000","last_clean":"2024-04-04T06:07:10.417424+0000","last_became_active":"2024-04-04T06:07:09.422628+0000","last_became_peered":"2024-04-04T06:07:09.422628+0000","last_unstale":"2024-04-04T06:07:10.417424+0000","last_undegraded":"2024-04-04T06:07:10.417424+0000","last_fullsized":"2024-04-04T06:07:10.417424+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:41:35.695728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.667342+0000","last_change":"2024-04-04T06:07:09.430515+0000","last_active":"2024-04-04T06:07:10.667342+0000","last_peered":"2024-04-04T06:07:10.667342+0000","last_clean":"2024-04-04T06:07:10.667342+0000","last_became_active":"2024-04-04T06:07:09.429760+0000","last_became_peered":"2024-04-04T06:07:09.429760+0000","last_unstale":"2024-04-04T06:07:10.667342+0000","last_undegraded":"2024-04-04T06:07:10.667342+0000","last_fullsized":"2024-04-04T06:07:10.667342+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:13:29.321236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.419566+0000","last_change":"2024-04-04T06:07:09.419125+0000","last_active":"2024-04-04T06:07:10.419566+0000","last_peered":"2024-04-04T06:07:10.419566+0000","last_clean":"2024-04-04T06:07:10.419566+0000","last_became_active":"2024-04-04T06:07:09.416934+0000","last_became_peered":"2024-04-04T06:07:09.416934+0000","last_unstale":"2024-04-04T06:07:10.419566+0000","last_undegraded":"2024-04-04T06:07:10.419566+0000","last_fullsized":"2024-04-04T06:07:10.419566+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:51:20.551855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126885+0000","last_change":"2024-04-04T06:07:09.421613+0000","last_active":"2024-04-04T06:07:11.126885+0000","last_peered":"2024-04-04T06:07:11.126885+0000","last_clean":"2024-04-04T06:07:11.126885+0000","last_became_active":"2024-04-04T06:07:09.421273+0000","last_became_peered":"2024-04-04T06:07:09.421273+0000","last_unstale":"2024-04-04T06:07:11.126885+0000","last_undegraded":"2024-04-04T06:07:11.126885+0000","last_fullsized":"2024-04-04T06:07:11.126885+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:05:49.177763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126567+0000","last_change":"2024-04-04T06:07:09.413255+0000","last_active":"2024-04-04T06:07:11.126567+0000","last_peered":"2024-04-04T06:07:11.126567+0000","last_clean":"2024-04-04T06:07:11.126567+0000","last_became_active":"2024-04-04T06:07:09.412791+0000","last_became_peered":"2024-04-04T06:07:09.412791+0000","last_unstale":"2024-04-04T06:07:11.126567+0000","last_undegraded":"2024-04-04T06:07:11.126567+0000","last_fullsized":"2024-04-04T06:07:11.126567+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:24:57.295679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418532+0000","last_change":"2024-04-04T06:07:09.413108+0000","last_active":"2024-04-04T06:07:10.418532+0000","last_peered":"2024-04-04T06:07:10.418532+0000","last_clean":"2024-04-04T06:07:10.418532+0000","last_became_active":"2024-04-04T06:07:09.412778+0000","last_became_peered":"2024-04-04T06:07:09.412778+0000","last_unstale":"2024-04-04T06:07:10.418532+0000","last_undegraded":"2024-04-04T06:07:10.418532+0000","last_fullsized":"2024-04-04T06:07:10.418532+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:08:01.796146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.667999+0000","last_change":"2024-04-04T06:07:09.424640+0000","last_active":"2024-04-04T06:07:10.667999+0000","last_peered":"2024-04-04T06:07:10.667999+0000","last_clean":"2024-04-04T06:07:10.667999+0000","last_became_active":"2024-04-04T06:07:09.423981+0000","last_became_peered":"2024-04-04T06:07:09.423981+0000","last_unstale":"2024-04-04T06:07:10.667999+0000","last_undegraded":"2024-04-04T06:07:10.667999+0000","last_fullsized":"2024-04-04T06:07:10.667999+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:34:51.034798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.667828+0000","last_change":"2024-04-04T06:07:09.433800+0000","last_active":"2024-04-04T06:07:10.667828+0000","last_peered":"2024-04-04T06:07:10.667828+0000","last_clean":"2024-04-04T06:07:10.667828+0000","last_became_active":"2024-04-04T06:07:09.431162+0000","last_became_peered":"2024-04-04T06:07:09.431162+0000","last_unstale":"2024-04-04T06:07:10.667828+0000","last_undegraded":"2024-04-04T06:07:10.667828+0000","last_fullsized":"2024-04-04T06:07:10.667828+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:48:08.743467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.128738+0000","last_change":"2024-04-04T06:07:09.428272+0000","last_active":"2024-04-04T06:07:11.128738+0000","last_peered":"2024-04-04T06:07:11.128738+0000","last_clean":"2024-04-04T06:07:11.128738+0000","last_became_active":"2024-04-04T06:07:09.423584+0000","last_became_peered":"2024-04-04T06:07:09.423584+0000","last_unstale":"2024-04-04T06:07:11.128738+0000","last_undegraded":"2024-04-04T06:07:11.128738+0000","last_fullsized":"2024-04-04T06:07:11.128738+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:30:20.672617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.127594+0000","last_change":"2024-04-04T06:07:09.432751+0000","last_active":"2024-04-04T06:07:11.127594+0000","last_peered":"2024-04-04T06:07:11.127594+0000","last_clean":"2024-04-04T06:07:11.127594+0000","last_became_active":"2024-04-04T06:07:09.432487+0000","last_became_peered":"2024-04-04T06:07:09.432487+0000","last_unstale":"2024-04-04T06:07:11.127594+0000","last_undegraded":"2024-04-04T06:07:11.127594+0000","last_fullsized":"2024-04-04T06:07:11.127594+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:14:07.097099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.127229+0000","last_change":"2024-04-04T06:07:09.417098+0000","last_active":"2024-04-04T06:07:11.127229+0000","last_peered":"2024-04-04T06:07:11.127229+0000","last_clean":"2024-04-04T06:07:11.127229+0000","last_became_active":"2024-04-04T06:07:09.416830+0000","last_became_peered":"2024-04-04T06:07:09.416830+0000","last_unstale":"2024-04-04T06:07:11.127229+0000","last_undegraded":"2024-04-04T06:07:11.127229+0000","last_fullsized":"2024-04-04T06:07:11.127229+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:53:43.115602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417847+0000","last_change":"2024-04-04T06:07:09.421377+0000","last_active":"2024-04-04T06:07:10.417847+0000","last_peered":"2024-04-04T06:07:10.417847+0000","last_clean":"2024-04-04T06:07:10.417847+0000","last_became_active":"2024-04-04T06:07:09.421064+0000","last_became_peered":"2024-04-04T06:07:09.421064+0000","last_unstale":"2024-04-04T06:07:10.417847+0000","last_undegraded":"2024-04-04T06:07:10.417847+0000","last_fullsized":"2024-04-04T06:07:10.417847+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:41:35.695728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418156+0000","last_change":"2024-04-04T06:07:09.427146+0000","last_active":"2024-04-04T06:07:10.418156+0000","last_peered":"2024-04-04T06:07:10.418156+0000","last_clean":"2024-04-04T06:07:10.418156+0000","last_became_active":"2024-04-04T06:07:09.424868+0000","last_became_peered":"2024-04-04T06:07:09.424868+0000","last_unstale":"2024-04-04T06:07:10.418156+0000","last_undegraded":"2024-04-04T06:07:10.418156+0000","last_fullsized":"2024-04-04T06:07:10.418156+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:49:38.280388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.423758+0000","last_change":"2024-04-04T06:07:09.435325+0000","last_active":"2024-04-04T06:07:10.423758+0000","last_peered":"2024-04-04T06:07:10.423758+0000","last_clean":"2024-04-04T06:07:10.423758+0000","last_became_active":"2024-04-04T06:07:09.430681+0000","last_became_peered":"2024-04-04T06:07:09.430681+0000","last_unstale":"2024-04-04T06:07:10.423758+0000","last_undegraded":"2024-04-04T06:07:10.423758+0000","last_fullsized":"2024-04-04T06:07:10.423758+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:52:08.974731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417708+0000","last_change":"2024-04-04T06:07:09.417505+0000","last_active":"2024-04-04T06:07:10.417708+0000","last_peered":"2024-04-04T06:07:10.417708+0000","last_clean":"2024-04-04T06:07:10.417708+0000","last_became_active":"2024-04-04T06:07:09.417260+0000","last_became_peered":"2024-04-04T06:07:09.417260+0000","last_unstale":"2024-04-04T06:07:10.417708+0000","last_undegraded":"2024-04-04T06:07:10.417708+0000","last_fullsized":"2024-04-04T06:07:10.417708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:47:54.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418889+0000","last_change":"2024-04-04T06:07:09.431743+0000","last_active":"2024-04-04T06:07:10.418889+0000","last_peered":"2024-04-04T06:07:10.418889+0000","last_clean":"2024-04-04T06:07:10.418889+0000","last_became_active":"2024-04-04T06:07:09.431463+0000","last_became_peered":"2024-04-04T06:07:09.431463+0000","last_unstale":"2024-04-04T06:07:10.418889+0000","last_undegraded":"2024-04-04T06:07:10.418889+0000","last_fullsized":"2024-04-04T06:07:10.418889+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:51:20.551855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.128713+0000","last_change":"2024-04-04T06:07:09.420002+0000","last_active":"2024-04-04T06:07:11.128713+0000","last_peered":"2024-04-04T06:07:11.128713+0000","last_clean":"2024-04-04T06:07:11.128713+0000","last_became_active":"2024-04-04T06:07:09.419685+0000","last_became_peered":"2024-04-04T06:07:09.419685+0000","last_unstale":"2024-04-04T06:07:11.128713+0000","last_undegraded":"2024-04-04T06:07:11.128713+0000","last_fullsized":"2024-04-04T06:07:11.128713+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:30:04.395085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418308+0000","last_change":"2024-04-04T06:07:09.433952+0000","last_active":"2024-04-04T06:07:10.418308+0000","last_peered":"2024-04-04T06:07:10.418308+0000","last_clean":"2024-04-04T06:07:10.418308+0000","last_became_active":"2024-04-04T06:07:09.433656+0000","last_became_peered":"2024-04-04T06:07:09.433656+0000","last_unstale":"2024-04-04T06:07:10.418308+0000","last_undegraded":"2024-04-04T06:07:10.418308+0000","last_fullsized":"2024-04-04T06:07:10.418308+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:08:01.796146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.419995+0000","last_change":"2024-04-04T06:07:09.427397+0000","last_active":"2024-04-04T06:07:10.419995+0000","last_peered":"2024-04-04T06:07:10.419995+0000","last_clean":"2024-04-04T06:07:10.419995+0000","last_became_active":"2024-04-04T06:07:09.427137+0000","last_became_peered":"2024-04-04T06:07:09.427137+0000","last_unstale":"2024-04-04T06:07:10.419995+0000","last_undegraded":"2024-04-04T06:07:10.419995+0000","last_fullsized":"2024-04-04T06:07:10.419995+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:24:57.295679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417871+0000","last_change":"2024-04-04T06:07:09.422128+0000","last_active":"2024-04-04T06:07:10.417871+0000","last_peered":"2024-04-04T06:07:10.417871+0000","last_clean":"2024-04-04T06:07:10.417871+0000","last_became_active":"2024-04-04T06:07:09.421687+0000","last_became_peered":"2024-04-04T06:07:09.421687+0000","last_unstale":"2024-04-04T06:07:10.417871+0000","last_undegraded":"2024-04-04T06:07:10.417871+0000","last_fullsized":"2024-04-04T06:07:10.417871+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:08:47.952291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.667063+0000","last_change":"2024-04-04T06:07:09.431813+0000","last_active":"2024-04-04T06:07:10.667063+0000","last_peered":"2024-04-04T06:07:10.667063+0000","last_clean":"2024-04-04T06:07:10.667063+0000","last_became_active":"2024-04-04T06:07:09.431562+0000","last_became_peered":"2024-04-04T06:07:09.431562+0000","last_unstale":"2024-04-04T06:07:10.667063+0000","last_undegraded":"2024-04-04T06:07:10.667063+0000","last_fullsized":"2024-04-04T06:07:10.667063+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:56:24.411309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.125768+0000","last_change":"2024-04-04T06:07:09.421243+0000","last_active":"2024-04-04T06:07:11.125768+0000","last_peered":"2024-04-04T06:07:11.125768+0000","last_clean":"2024-04-04T06:07:11.125768+0000","last_became_active":"2024-04-04T06:07:09.420978+0000","last_became_peered":"2024-04-04T06:07:09.420978+0000","last_unstale":"2024-04-04T06:07:11.125768+0000","last_undegraded":"2024-04-04T06:07:11.125768+0000","last_fullsized":"2024-04-04T06:07:11.125768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:55:41.865738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.667329+0000","last_change":"2024-04-04T06:07:09.424249+0000","last_active":"2024-04-04T06:07:10.667329+0000","last_peered":"2024-04-04T06:07:10.667329+0000","last_clean":"2024-04-04T06:07:10.667329+0000","last_became_active":"2024-04-04T06:07:09.423645+0000","last_became_peered":"2024-04-04T06:07:09.423645+0000","last_unstale":"2024-04-04T06:07:10.667329+0000","last_undegraded":"2024-04-04T06:07:10.667329+0000","last_fullsized":"2024-04-04T06:07:10.667329+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:41:40.027163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.424121+0000","last_change":"2024-04-04T06:07:09.435761+0000","last_active":"2024-04-04T06:07:10.424121+0000","last_peered":"2024-04-04T06:07:10.424121+0000","last_clean":"2024-04-04T06:07:10.424121+0000","last_became_active":"2024-04-04T06:07:09.431040+0000","last_became_peered":"2024-04-04T06:07:09.431040+0000","last_unstale":"2024-04-04T06:07:10.424121+0000","last_undegraded":"2024-04-04T06:07:10.424121+0000","last_fullsized":"2024-04-04T06:07:10.424121+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:24:08.463853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126416+0000","last_change":"2024-04-04T06:07:08.395896+0000","last_active":"2024-04-04T06:07:11.126416+0000","last_peered":"2024-04-04T06:07:11.126416+0000","last_clean":"2024-04-04T06:07:11.126416+0000","last_became_active":"2024-04-04T06:07:08.395526+0000","last_became_peered":"2024-04-04T06:07:08.395526+0000","last_unstale":"2024-04-04T06:07:11.126416+0000","last_undegraded":"2024-04-04T06:07:11.126416+0000","last_fullsized":"2024-04-04T06:07:11.126416+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:15:31.320764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126289+0000","last_change":"2024-04-04T06:07:09.426596+0000","last_active":"2024-04-04T06:07:11.126289+0000","last_peered":"2024-04-04T06:07:11.126289+0000","last_clean":"2024-04-04T06:07:11.126289+0000","last_became_active":"2024-04-04T06:07:09.426252+0000","last_became_peered":"2024-04-04T06:07:09.426252+0000","last_unstale":"2024-04-04T06:07:11.126289+0000","last_undegraded":"2024-04-04T06:07:11.126289+0000","last_fullsized":"2024-04-04T06:07:11.126289+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:27:48.592561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.424311+0000","last_change":"2024-04-04T06:07:08.393414+0000","last_active":"2024-04-04T06:07:10.424311+0000","last_peered":"2024-04-04T06:07:10.424311+0000","last_clean":"2024-04-04T06:07:10.424311+0000","last_became_active":"2024-04-04T06:07:08.393161+0000","last_became_peered":"2024-04-04T06:07:08.393161+0000","last_unstale":"2024-04-04T06:07:10.424311+0000","last_undegraded":"2024-04-04T06:07:10.424311+0000","last_fullsized":"2024-04-04T06:07:10.424311+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:53:42.102358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418401+0000","last_change":"2024-04-04T06:07:09.420683+0000","last_active":"2024-04-04T06:07:10.418401+0000","last_peered":"2024-04-04T06:07:10.418401+0000","last_clean":"2024-04-04T06:07:10.418401+0000","last_became_active":"2024-04-04T06:07:09.420379+0000","last_became_peered":"2024-04-04T06:07:09.420379+0000","last_unstale":"2024-04-04T06:07:10.418401+0000","last_undegraded":"2024-04-04T06:07:10.418401+0000","last_fullsized":"2024-04-04T06:07:10.418401+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:47:54.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.127892+0000","last_change":"2024-04-04T06:07:08.396008+0000","last_active":"2024-04-04T06:07:11.127892+0000","last_peered":"2024-04-04T06:07:11.127892+0000","last_clean":"2024-04-04T06:07:11.127892+0000","last_became_active":"2024-04-04T06:07:08.394919+0000","last_became_peered":"2024-04-04T06:07:08.394919+0000","last_unstale":"2024-04-04T06:07:11.127892+0000","last_undegraded":"2024-04-04T06:07:11.127892+0000","last_fullsized":"2024-04-04T06:07:11.127892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:20:05.428424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418450+0000","last_change":"2024-04-04T06:07:09.421981+0000","last_active":"2024-04-04T06:07:10.418450+0000","last_peered":"2024-04-04T06:07:10.418450+0000","last_clean":"2024-04-04T06:07:10.418450+0000","last_became_active":"2024-04-04T06:07:09.421716+0000","last_became_peered":"2024-04-04T06:07:09.421716+0000","last_unstale":"2024-04-04T06:07:10.418450+0000","last_undegraded":"2024-04-04T06:07:10.418450+0000","last_fullsized":"2024-04-04T06:07:10.418450+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:04:43.784625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.127006+0000","last_change":"2024-04-04T06:07:08.386261+0000","last_active":"2024-04-04T06:07:11.127006+0000","last_peered":"2024-04-04T06:07:11.127006+0000","last_clean":"2024-04-04T06:07:11.127006+0000","last_became_active":"2024-04-04T06:07:08.385957+0000","last_became_peered":"2024-04-04T06:07:08.385957+0000","last_unstale":"2024-04-04T06:07:11.127006+0000","last_undegraded":"2024-04-04T06:07:11.127006+0000","last_fullsized":"2024-04-04T06:07:11.127006+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:06:21.181613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.667298+0000","last_change":"2024-04-04T06:07:09.431316+0000","last_active":"2024-04-04T06:07:10.667298+0000","last_peered":"2024-04-04T06:07:10.667298+0000","last_clean":"2024-04-04T06:07:10.667298+0000","last_became_active":"2024-04-04T06:07:09.430996+0000","last_became_peered":"2024-04-04T06:07:09.430996+0000","last_unstale":"2024-04-04T06:07:10.667298+0000","last_undegraded":"2024-04-04T06:07:10.667298+0000","last_fullsized":"2024-04-04T06:07:10.667298+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:36:37.218048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417962+0000","last_change":"2024-04-04T06:07:08.385627+0000","last_active":"2024-04-04T06:07:10.417962+0000","last_peered":"2024-04-04T06:07:10.417962+0000","last_clean":"2024-04-04T06:07:10.417962+0000","last_became_active":"2024-04-04T06:07:08.384853+0000","last_became_peered":"2024-04-04T06:07:08.384853+0000","last_unstale":"2024-04-04T06:07:10.417962+0000","last_undegraded":"2024-04-04T06:07:10.417962+0000","last_fullsized":"2024-04-04T06:07:10.417962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:44:14.326671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.419465+0000","last_change":"2024-04-04T06:07:09.422774+0000","last_active":"2024-04-04T06:07:10.419465+0000","last_peered":"2024-04-04T06:07:10.419465+0000","last_clean":"2024-04-04T06:07:10.419465+0000","last_became_active":"2024-04-04T06:07:09.422526+0000","last_became_peered":"2024-04-04T06:07:09.422526+0000","last_unstale":"2024-04-04T06:07:10.419465+0000","last_undegraded":"2024-04-04T06:07:10.419465+0000","last_fullsized":"2024-04-04T06:07:10.419465+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:04:43.784625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.419498+0000","last_change":"2024-04-04T06:07:08.388038+0000","last_active":"2024-04-04T06:07:10.419498+0000","last_peered":"2024-04-04T06:07:10.419498+0000","last_clean":"2024-04-04T06:07:10.419498+0000","last_became_active":"2024-04-04T06:07:08.387579+0000","last_became_peered":"2024-04-04T06:07:08.387579+0000","last_unstale":"2024-04-04T06:07:10.419498+0000","last_undegraded":"2024-04-04T06:07:10.419498+0000","last_fullsized":"2024-04-04T06:07:10.419498+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:36:10.005773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417818+0000","last_change":"2024-04-04T06:07:09.418402+0000","last_active":"2024-04-04T06:07:10.417818+0000","last_peered":"2024-04-04T06:07:10.417818+0000","last_clean":"2024-04-04T06:07:10.417818+0000","last_became_active":"2024-04-04T06:07:09.417990+0000","last_became_peered":"2024-04-04T06:07:09.417990+0000","last_unstale":"2024-04-04T06:07:10.417818+0000","last_undegraded":"2024-04-04T06:07:10.417818+0000","last_fullsized":"2024-04-04T06:07:10.417818+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:57:12.182305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126772+0000","last_change":"2024-04-04T06:07:08.385346+0000","last_active":"2024-04-04T06:07:11.126772+0000","last_peered":"2024-04-04T06:07:11.126772+0000","last_clean":"2024-04-04T06:07:11.126772+0000","last_became_active":"2024-04-04T06:07:08.385090+0000","last_became_peered":"2024-04-04T06:07:08.385090+0000","last_unstale":"2024-04-04T06:07:11.126772+0000","last_undegraded":"2024-04-04T06:07:11.126772+0000","last_fullsized":"2024-04-04T06:07:11.126772+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:54:31.814804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418304+0000","last_change":"2024-04-04T06:07:09.419242+0000","last_active":"2024-04-04T06:07:10.418304+0000","last_peered":"2024-04-04T06:07:10.418304+0000","last_clean":"2024-04-04T06:07:10.418304+0000","last_became_active":"2024-04-04T06:07:09.418911+0000","last_became_peered":"2024-04-04T06:07:09.418911+0000","last_unstale":"2024-04-04T06:07:10.418304+0000","last_undegraded":"2024-04-04T06:07:10.418304+0000","last_fullsized":"2024-04-04T06:07:10.418304+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:47:54.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417990+0000","last_change":"2024-04-04T06:07:08.392237+0000","last_active":"2024-04-04T06:07:10.417990+0000","last_peered":"2024-04-04T06:07:10.417990+0000","last_clean":"2024-04-04T06:07:10.417990+0000","last_became_active":"2024-04-04T06:07:08.391959+0000","last_became_peered":"2024-04-04T06:07:08.391959+0000","last_unstale":"2024-04-04T06:07:10.417990+0000","last_undegraded":"2024-04-04T06:07:10.417990+0000","last_fullsized":"2024-04-04T06:07:10.417990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:22:30.070585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.420013+0000","last_change":"2024-04-04T06:07:09.417161+0000","last_active":"2024-04-04T06:07:10.420013+0000","last_peered":"2024-04-04T06:07:10.420013+0000","last_clean":"2024-04-04T06:07:10.420013+0000","last_became_active":"2024-04-04T06:07:09.416862+0000","last_became_peered":"2024-04-04T06:07:09.416862+0000","last_unstale":"2024-04-04T06:07:10.420013+0000","last_undegraded":"2024-04-04T06:07:10.420013+0000","last_fullsized":"2024-04-04T06:07:10.420013+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:16:00.835478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.128517+0000","last_change":"2024-04-04T06:07:08.395854+0000","last_active":"2024-04-04T06:07:11.128517+0000","last_peered":"2024-04-04T06:07:11.128517+0000","last_clean":"2024-04-04T06:07:11.128517+0000","last_became_active":"2024-04-04T06:07:08.395053+0000","last_became_peered":"2024-04-04T06:07:08.395053+0000","last_unstale":"2024-04-04T06:07:11.128517+0000","last_undegraded":"2024-04-04T06:07:11.128517+0000","last_fullsized":"2024-04-04T06:07:11.128517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:55:03.146417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.128093+0000","last_change":"2024-04-04T06:07:09.426054+0000","last_active":"2024-04-04T06:07:11.128093+0000","last_peered":"2024-04-04T06:07:11.128093+0000","last_clean":"2024-04-04T06:07:11.128093+0000","last_became_active":"2024-04-04T06:07:09.423539+0000","last_became_peered":"2024-04-04T06:07:09.423539+0000","last_unstale":"2024-04-04T06:07:11.128093+0000","last_undegraded":"2024-04-04T06:07:11.128093+0000","last_fullsized":"2024-04-04T06:07:11.128093+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:53:13.683704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.424352+0000","last_change":"2024-04-04T06:07:08.394398+0000","last_active":"2024-04-04T06:07:10.424352+0000","last_peered":"2024-04-04T06:07:10.424352+0000","last_clean":"2024-04-04T06:07:10.424352+0000","last_became_active":"2024-04-04T06:07:08.392409+0000","last_became_peered":"2024-04-04T06:07:08.392409+0000","last_unstale":"2024-04-04T06:07:10.424352+0000","last_undegraded":"2024-04-04T06:07:10.424352+0000","last_fullsized":"2024-04-04T06:07:10.424352+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:23:43.557850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.128059+0000","last_change":"2024-04-04T06:07:09.420571+0000","last_active":"2024-04-04T06:07:11.128059+0000","last_peered":"2024-04-04T06:07:11.128059+0000","last_clean":"2024-04-04T06:07:11.128059+0000","last_became_active":"2024-04-04T06:07:09.420238+0000","last_became_peered":"2024-04-04T06:07:09.420238+0000","last_unstale":"2024-04-04T06:07:11.128059+0000","last_undegraded":"2024-04-04T06:07:11.128059+0000","last_fullsized":"2024-04-04T06:07:11.128059+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:31:23.568304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.420006+0000","last_change":"2024-04-04T06:07:08.388024+0000","last_active":"2024-04-04T06:07:10.420006+0000","last_peered":"2024-04-04T06:07:10.420006+0000","last_clean":"2024-04-04T06:07:10.420006+0000","last_became_active":"2024-04-04T06:07:08.387356+0000","last_became_peered":"2024-04-04T06:07:08.387356+0000","last_unstale":"2024-04-04T06:07:10.420006+0000","last_undegraded":"2024-04-04T06:07:10.420006+0000","last_fullsized":"2024-04-04T06:07:10.420006+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:20:05.428424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.419850+0000","last_change":"2024-04-04T06:07:09.425331+0000","last_active":"2024-04-04T06:07:10.419850+0000","last_peered":"2024-04-04T06:07:10.419850+0000","last_clean":"2024-04-04T06:07:10.419850+0000","last_became_active":"2024-04-04T06:07:09.424674+0000","last_became_peered":"2024-04-04T06:07:09.424674+0000","last_unstale":"2024-04-04T06:07:10.419850+0000","last_undegraded":"2024-04-04T06:07:10.419850+0000","last_fullsized":"2024-04-04T06:07:10.419850+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:57:22.011487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.128364+0000","last_change":"2024-04-04T06:07:08.390768+0000","last_active":"2024-04-04T06:07:11.128364+0000","last_peered":"2024-04-04T06:07:11.128364+0000","last_clean":"2024-04-04T06:07:11.128364+0000","last_became_active":"2024-04-04T06:07:08.390487+0000","last_became_peered":"2024-04-04T06:07:08.390487+0000","last_unstale":"2024-04-04T06:07:11.128364+0000","last_undegraded":"2024-04-04T06:07:11.128364+0000","last_fullsized":"2024-04-04T06:07:11.128364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:52:07.961487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417238+0000","last_change":"2024-04-04T06:07:09.420879+0000","last_active":"2024-04-04T06:07:10.417238+0000","last_peered":"2024-04-04T06:07:10.417238+0000","last_clean":"2024-04-04T06:07:10.417238+0000","last_became_active":"2024-04-04T06:07:09.418739+0000","last_became_peered":"2024-04-04T06:07:09.418739+0000","last_unstale":"2024-04-04T06:07:10.417238+0000","last_undegraded":"2024-04-04T06:07:10.417238+0000","last_fullsized":"2024-04-04T06:07:10.417238+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:57:12.182305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417967+0000","last_change":"2024-04-04T06:07:08.393867+0000","last_active":"2024-04-04T06:07:10.417967+0000","last_peered":"2024-04-04T06:07:10.417967+0000","last_clean":"2024-04-04T06:07:10.417967+0000","last_became_active":"2024-04-04T06:07:08.393204+0000","last_became_peered":"2024-04-04T06:07:08.393204+0000","last_unstale":"2024-04-04T06:07:10.417967+0000","last_undegraded":"2024-04-04T06:07:10.417967+0000","last_fullsized":"2024-04-04T06:07:10.417967+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:53:42.102358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.128004+0000","last_change":"2024-04-04T06:07:09.434342+0000","last_active":"2024-04-04T06:07:11.128004+0000","last_peered":"2024-04-04T06:07:11.128004+0000","last_clean":"2024-04-04T06:07:11.128004+0000","last_became_active":"2024-04-04T06:07:09.432138+0000","last_became_peered":"2024-04-04T06:07:09.432138+0000","last_unstale":"2024-04-04T06:07:11.128004+0000","last_undegraded":"2024-04-04T06:07:11.128004+0000","last_fullsized":"2024-04-04T06:07:11.128004+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:53:59.963734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.416963+0000","last_change":"2024-04-04T06:07:08.393385+0000","last_active":"2024-04-04T06:07:10.416963+0000","last_peered":"2024-04-04T06:07:10.416963+0000","last_clean":"2024-04-04T06:07:10.416963+0000","last_became_active":"2024-04-04T06:07:08.393078+0000","last_became_peered":"2024-04-04T06:07:08.393078+0000","last_unstale":"2024-04-04T06:07:10.416963+0000","last_undegraded":"2024-04-04T06:07:10.416963+0000","last_fullsized":"2024-04-04T06:07:10.416963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:17:01.423214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.667504+0000","last_change":"2024-04-04T06:07:09.426880+0000","last_active":"2024-04-04T06:07:10.667504+0000","last_peered":"2024-04-04T06:07:10.667504+0000","last_clean":"2024-04-04T06:07:10.667504+0000","last_became_active":"2024-04-04T06:07:09.426586+0000","last_became_peered":"2024-04-04T06:07:09.426586+0000","last_unstale":"2024-04-04T06:07:10.667504+0000","last_undegraded":"2024-04-04T06:07:10.667504+0000","last_fullsized":"2024-04-04T06:07:10.667504+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:03:01.372537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126228+0000","last_change":"2024-04-04T06:07:08.393997+0000","last_active":"2024-04-04T06:07:11.126228+0000","last_peered":"2024-04-04T06:07:11.126228+0000","last_clean":"2024-04-04T06:07:11.126228+0000","last_became_active":"2024-04-04T06:07:08.393594+0000","last_became_peered":"2024-04-04T06:07:08.393594+0000","last_unstale":"2024-04-04T06:07:11.126228+0000","last_undegraded":"2024-04-04T06:07:11.126228+0000","last_fullsized":"2024-04-04T06:07:11.126228+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:29:14.442681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.125930+0000","last_change":"2024-04-04T06:07:09.426992+0000","last_active":"2024-04-04T06:07:11.125930+0000","last_peered":"2024-04-04T06:07:11.125930+0000","last_clean":"2024-04-04T06:07:11.125930+0000","last_became_active":"2024-04-04T06:07:09.426692+0000","last_became_peered":"2024-04-04T06:07:09.426692+0000","last_unstale":"2024-04-04T06:07:11.125930+0000","last_undegraded":"2024-04-04T06:07:11.125930+0000","last_fullsized":"2024-04-04T06:07:11.125930+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:37:44.944983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.127468+0000","last_change":"2024-04-04T06:07:08.391760+0000","last_active":"2024-04-04T06:07:11.127468+0000","last_peered":"2024-04-04T06:07:11.127468+0000","last_clean":"2024-04-04T06:07:11.127468+0000","last_became_active":"2024-04-04T06:07:08.391123+0000","last_became_peered":"2024-04-04T06:07:08.391123+0000","last_unstale":"2024-04-04T06:07:11.127468+0000","last_undegraded":"2024-04-04T06:07:11.127468+0000","last_fullsized":"2024-04-04T06:07:11.127468+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:52:10.694546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418638+0000","last_change":"2024-04-04T06:07:09.419012+0000","last_active":"2024-04-04T06:07:10.418638+0000","last_peered":"2024-04-04T06:07:10.418638+0000","last_clean":"2024-04-04T06:07:10.418638+0000","last_became_active":"2024-04-04T06:07:09.418708+0000","last_became_peered":"2024-04-04T06:07:09.418708+0000","last_unstale":"2024-04-04T06:07:10.418638+0000","last_undegraded":"2024-04-04T06:07:10.418638+0000","last_fullsized":"2024-04-04T06:07:10.418638+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:08:01.796146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.419142+0000","last_change":"2024-04-04T06:07:08.389249+0000","last_active":"2024-04-04T06:07:10.419142+0000","last_peered":"2024-04-04T06:07:10.419142+0000","last_clean":"2024-04-04T06:07:10.419142+0000","last_became_active":"2024-04-04T06:07:08.387652+0000","last_became_peered":"2024-04-04T06:07:08.387652+0000","last_unstale":"2024-04-04T06:07:10.419142+0000","last_undegraded":"2024-04-04T06:07:10.419142+0000","last_fullsized":"2024-04-04T06:07:10.419142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:08:14.750770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.420454+0000","last_change":"2024-04-04T06:07:09.408046+0000","last_active":"2024-04-04T06:07:10.420454+0000","last_peered":"2024-04-04T06:07:10.420454+0000","last_clean":"2024-04-04T06:07:10.420454+0000","last_became_active":"2024-04-04T06:07:09.407728+0000","last_became_peered":"2024-04-04T06:07:09.407728+0000","last_unstale":"2024-04-04T06:07:10.420454+0000","last_undegraded":"2024-04-04T06:07:10.420454+0000","last_fullsized":"2024-04-04T06:07:10.420454+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:31:23.568304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418566+0000","last_change":"2024-04-04T06:07:08.392144+0000","last_active":"2024-04-04T06:07:10.418566+0000","last_peered":"2024-04-04T06:07:10.418566+0000","last_clean":"2024-04-04T06:07:10.418566+0000","last_became_active":"2024-04-04T06:07:08.391854+0000","last_became_peered":"2024-04-04T06:07:08.391854+0000","last_unstale":"2024-04-04T06:07:10.418566+0000","last_undegraded":"2024-04-04T06:07:10.418566+0000","last_fullsized":"2024-04-04T06:07:10.418566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:23:43.557850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417616+0000","last_change":"2024-04-04T06:07:09.425499+0000","last_active":"2024-04-04T06:07:10.417616+0000","last_peered":"2024-04-04T06:07:10.417616+0000","last_clean":"2024-04-04T06:07:10.417616+0000","last_became_active":"2024-04-04T06:07:09.425221+0000","last_became_peered":"2024-04-04T06:07:09.425221+0000","last_unstale":"2024-04-04T06:07:10.417616+0000","last_undegraded":"2024-04-04T06:07:10.417616+0000","last_fullsized":"2024-04-04T06:07:10.417616+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:49:38.280388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418697+0000","last_change":"2024-04-04T06:07:08.384551+0000","last_active":"2024-04-04T06:07:10.418697+0000","last_peered":"2024-04-04T06:07:10.418697+0000","last_clean":"2024-04-04T06:07:10.418697+0000","last_became_active":"2024-04-04T06:07:08.383977+0000","last_became_peered":"2024-04-04T06:07:08.383977+0000","last_unstale":"2024-04-04T06:07:10.418697+0000","last_undegraded":"2024-04-04T06:07:10.418697+0000","last_fullsized":"2024-04-04T06:07:10.418697+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:51:41.223975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.127586+0000","last_change":"2024-04-04T06:07:09.427057+0000","last_active":"2024-04-04T06:07:11.127586+0000","last_peered":"2024-04-04T06:07:11.127586+0000","last_clean":"2024-04-04T06:07:11.127586+0000","last_became_active":"2024-04-04T06:07:09.426800+0000","last_became_peered":"2024-04-04T06:07:09.426800+0000","last_unstale":"2024-04-04T06:07:11.127586+0000","last_undegraded":"2024-04-04T06:07:11.127586+0000","last_fullsized":"2024-04-04T06:07:11.127586+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:08:15.764014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418519+0000","last_change":"2024-04-04T06:07:08.385778+0000","last_active":"2024-04-04T06:07:10.418519+0000","last_peered":"2024-04-04T06:07:10.418519+0000","last_clean":"2024-04-04T06:07:10.418519+0000","last_became_active":"2024-04-04T06:07:08.385169+0000","last_became_peered":"2024-04-04T06:07:08.385169+0000","last_unstale":"2024-04-04T06:07:10.418519+0000","last_undegraded":"2024-04-04T06:07:10.418519+0000","last_fullsized":"2024-04-04T06:07:10.418519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:17:01.423214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126543+0000","last_change":"2024-04-04T06:07:09.424114+0000","last_active":"2024-04-04T06:07:11.126543+0000","last_peered":"2024-04-04T06:07:11.126543+0000","last_clean":"2024-04-04T06:07:11.126543+0000","last_became_active":"2024-04-04T06:07:09.423801+0000","last_became_peered":"2024-04-04T06:07:09.423801+0000","last_unstale":"2024-04-04T06:07:11.126543+0000","last_undegraded":"2024-04-04T06:07:11.126543+0000","last_fullsized":"2024-04-04T06:07:11.126543+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:46:14.937588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.423960+0000","last_change":"2024-04-04T06:07:08.385377+0000","last_active":"2024-04-04T06:07:10.423960+0000","last_peered":"2024-04-04T06:07:10.423960+0000","last_clean":"2024-04-04T06:07:10.423960+0000","last_became_active":"2024-04-04T06:07:08.385120+0000","last_became_peered":"2024-04-04T06:07:08.385120+0000","last_unstale":"2024-04-04T06:07:10.423960+0000","last_undegraded":"2024-04-04T06:07:10.423960+0000","last_fullsized":"2024-04-04T06:07:10.423960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:08:14.750770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.127301+0000","last_change":"2024-04-04T06:07:09.420552+0000","last_active":"2024-04-04T06:07:11.127301+0000","last_peered":"2024-04-04T06:07:11.127301+0000","last_clean":"2024-04-04T06:07:11.127301+0000","last_became_active":"2024-04-04T06:07:09.420098+0000","last_became_peered":"2024-04-04T06:07:09.420098+0000","last_unstale":"2024-04-04T06:07:11.127301+0000","last_undegraded":"2024-04-04T06:07:11.127301+0000","last_fullsized":"2024-04-04T06:07:11.127301+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:17:02.436458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126024+0000","last_change":"2024-04-04T06:07:08.392816+0000","last_active":"2024-04-04T06:07:11.126024+0000","last_peered":"2024-04-04T06:07:11.126024+0000","last_clean":"2024-04-04T06:07:11.126024+0000","last_became_active":"2024-04-04T06:07:08.391134+0000","last_became_peered":"2024-04-04T06:07:08.391134+0000","last_unstale":"2024-04-04T06:07:11.126024+0000","last_undegraded":"2024-04-04T06:07:11.126024+0000","last_fullsized":"2024-04-04T06:07:11.126024+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:15:04.696399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126821+0000","last_change":"2024-04-04T06:07:09.423215+0000","last_active":"2024-04-04T06:07:11.126821+0000","last_peered":"2024-04-04T06:07:11.126821+0000","last_clean":"2024-04-04T06:07:11.126821+0000","last_became_active":"2024-04-04T06:07:09.421307+0000","last_became_peered":"2024-04-04T06:07:09.421307+0000","last_unstale":"2024-04-04T06:07:11.126821+0000","last_undegraded":"2024-04-04T06:07:11.126821+0000","last_fullsized":"2024-04-04T06:07:11.126821+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:51:42.237220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.419553+0000","last_change":"2024-04-04T06:07:08.388908+0000","last_active":"2024-04-04T06:07:10.419553+0000","last_peered":"2024-04-04T06:07:10.419553+0000","last_clean":"2024-04-04T06:07:10.419553+0000","last_became_active":"2024-04-04T06:07:08.388561+0000","last_became_peered":"2024-04-04T06:07:08.388561+0000","last_unstale":"2024-04-04T06:07:10.419553+0000","last_undegraded":"2024-04-04T06:07:10.419553+0000","last_fullsized":"2024-04-04T06:07:10.419553+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:46:13.924344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.667011+0000","last_change":"2024-04-04T06:07:09.432471+0000","last_active":"2024-04-04T06:07:10.667011+0000","last_peered":"2024-04-04T06:07:10.667011+0000","last_clean":"2024-04-04T06:07:10.667011+0000","last_became_active":"2024-04-04T06:07:09.432191+0000","last_became_peered":"2024-04-04T06:07:09.432191+0000","last_unstale":"2024-04-04T06:07:10.667011+0000","last_undegraded":"2024-04-04T06:07:10.667011+0000","last_fullsized":"2024-04-04T06:07:10.667011+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:56:30.405307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418178+0000","last_change":"2024-04-04T06:07:08.394203+0000","last_active":"2024-04-04T06:07:10.418178+0000","last_peered":"2024-04-04T06:07:10.418178+0000","last_clean":"2024-04-04T06:07:10.418178+0000","last_became_active":"2024-04-04T06:07:08.393913+0000","last_became_peered":"2024-04-04T06:07:08.393913+0000","last_unstale":"2024-04-04T06:07:10.418178+0000","last_undegraded":"2024-04-04T06:07:10.418178+0000","last_fullsized":"2024-04-04T06:07:10.418178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:11:10.659591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.127523+0000","last_change":"2024-04-04T06:07:09.417965+0000","last_active":"2024-04-04T06:07:11.127523+0000","last_peered":"2024-04-04T06:07:11.127523+0000","last_clean":"2024-04-04T06:07:11.127523+0000","last_became_active":"2024-04-04T06:07:09.417641+0000","last_became_peered":"2024-04-04T06:07:09.417641+0000","last_unstale":"2024-04-04T06:07:11.127523+0000","last_undegraded":"2024-04-04T06:07:11.127523+0000","last_fullsized":"2024-04-04T06:07:11.127523+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:23:44.571094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.419117+0000","last_change":"2024-04-04T06:07:08.389862+0000","last_active":"2024-04-04T06:07:10.419117+0000","last_peered":"2024-04-04T06:07:10.419117+0000","last_clean":"2024-04-04T06:07:10.419117+0000","last_became_active":"2024-04-04T06:07:08.389574+0000","last_became_peered":"2024-04-04T06:07:08.389574+0000","last_unstale":"2024-04-04T06:07:10.419117+0000","last_undegraded":"2024-04-04T06:07:10.419117+0000","last_fullsized":"2024-04-04T06:07:10.419117+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:29:14.442681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.420421+0000","last_change":"2024-04-04T06:07:09.428824+0000","last_active":"2024-04-04T06:07:10.420421+0000","last_peered":"2024-04-04T06:07:10.420421+0000","last_clean":"2024-04-04T06:07:10.420421+0000","last_became_active":"2024-04-04T06:07:09.428511+0000","last_became_peered":"2024-04-04T06:07:09.428511+0000","last_unstale":"2024-04-04T06:07:10.420421+0000","last_undegraded":"2024-04-04T06:07:10.420421+0000","last_fullsized":"2024-04-04T06:07:10.420421+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:53:59.963734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.427561+0000","last_change":"2024-04-04T06:07:08.388555+0000","last_active":"2024-04-04T06:07:10.427561+0000","last_peered":"2024-04-04T06:07:10.427561+0000","last_clean":"2024-04-04T06:07:10.427561+0000","last_became_active":"2024-04-04T06:07:08.388280+0000","last_became_peered":"2024-04-04T06:07:08.388280+0000","last_unstale":"2024-04-04T06:07:10.427561+0000","last_undegraded":"2024-04-04T06:07:10.427561+0000","last_fullsized":"2024-04-04T06:07:10.427561+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:17:01.423214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.424287+0000","last_change":"2024-04-04T06:07:09.426956+0000","last_active":"2024-04-04T06:07:10.424287+0000","last_peered":"2024-04-04T06:07:10.424287+0000","last_clean":"2024-04-04T06:07:10.424287+0000","last_became_active":"2024-04-04T06:07:09.426638+0000","last_became_peered":"2024-04-04T06:07:09.426638+0000","last_unstale":"2024-04-04T06:07:10.424287+0000","last_undegraded":"2024-04-04T06:07:10.424287+0000","last_fullsized":"2024-04-04T06:07:10.424287+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:04:03.680120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417712+0000","last_change":"2024-04-04T06:07:08.382978+0000","last_active":"2024-04-04T06:07:10.417712+0000","last_peered":"2024-04-04T06:07:10.417712+0000","last_clean":"2024-04-04T06:07:10.417712+0000","last_became_active":"2024-04-04T06:07:08.382623+0000","last_became_peered":"2024-04-04T06:07:08.382623+0000","last_unstale":"2024-04-04T06:07:10.417712+0000","last_undegraded":"2024-04-04T06:07:10.417712+0000","last_fullsized":"2024-04-04T06:07:10.417712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:53:42.102358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.125585+0000","last_change":"2024-04-04T06:07:09.423847+0000","last_active":"2024-04-04T06:07:11.125585+0000","last_peered":"2024-04-04T06:07:11.125585+0000","last_clean":"2024-04-04T06:07:11.125585+0000","last_became_active":"2024-04-04T06:07:09.423575+0000","last_became_peered":"2024-04-04T06:07:09.423575+0000","last_unstale":"2024-04-04T06:07:11.125585+0000","last_undegraded":"2024-04-04T06:07:11.125585+0000","last_fullsized":"2024-04-04T06:07:11.125585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:24:27.600006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.424124+0000","last_change":"2024-04-04T06:07:08.388610+0000","last_active":"2024-04-04T06:07:10.424124+0000","last_peered":"2024-04-04T06:07:10.424124+0000","last_clean":"2024-04-04T06:07:10.424124+0000","last_became_active":"2024-04-04T06:07:08.388297+0000","last_became_peered":"2024-04-04T06:07:08.388297+0000","last_unstale":"2024-04-04T06:07:10.424124+0000","last_undegraded":"2024-04-04T06:07:10.424124+0000","last_fullsized":"2024-04-04T06:07:10.424124+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:29:14.442681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.423816+0000","last_change":"2024-04-04T06:07:09.428804+0000","last_active":"2024-04-04T06:07:10.423816+0000","last_peered":"2024-04-04T06:07:10.423816+0000","last_clean":"2024-04-04T06:07:10.423816+0000","last_became_active":"2024-04-04T06:07:09.428547+0000","last_became_peered":"2024-04-04T06:07:09.428547+0000","last_unstale":"2024-04-04T06:07:10.423816+0000","last_undegraded":"2024-04-04T06:07:10.423816+0000","last_fullsized":"2024-04-04T06:07:10.423816+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:55:04.159661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417081+0000","last_change":"2024-04-04T06:07:08.388072+0000","last_active":"2024-04-04T06:07:10.417081+0000","last_peered":"2024-04-04T06:07:10.417081+0000","last_clean":"2024-04-04T06:07:10.417081+0000","last_became_active":"2024-04-04T06:07:08.387790+0000","last_became_peered":"2024-04-04T06:07:08.387790+0000","last_unstale":"2024-04-04T06:07:10.417081+0000","last_undegraded":"2024-04-04T06:07:10.417081+0000","last_fullsized":"2024-04-04T06:07:10.417081+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:23:43.557850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126913+0000","last_change":"2024-04-04T06:07:09.420955+0000","last_active":"2024-04-04T06:07:11.126913+0000","last_peered":"2024-04-04T06:07:11.126913+0000","last_clean":"2024-04-04T06:07:11.126913+0000","last_became_active":"2024-04-04T06:07:09.420687+0000","last_became_peered":"2024-04-04T06:07:09.420687+0000","last_unstale":"2024-04-04T06:07:11.126913+0000","last_undegraded":"2024-04-04T06:07:11.126913+0000","last_fullsized":"2024-04-04T06:07:11.126913+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:16:00.835478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.419615+0000","last_change":"2024-04-04T06:07:08.390168+0000","last_active":"2024-04-04T06:07:10.419615+0000","last_peered":"2024-04-04T06:07:10.419615+0000","last_clean":"2024-04-04T06:07:10.419615+0000","last_became_active":"2024-04-04T06:07:08.389813+0000","last_became_peered":"2024-04-04T06:07:08.389813+0000","last_unstale":"2024-04-04T06:07:10.419615+0000","last_undegraded":"2024-04-04T06:07:10.419615+0000","last_fullsized":"2024-04-04T06:07:10.419615+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:05:48.164519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.419502+0000","last_change":"2024-04-04T06:07:09.421402+0000","last_active":"2024-04-04T06:07:10.419502+0000","last_peered":"2024-04-04T06:07:10.419502+0000","last_clean":"2024-04-04T06:07:10.419502+0000","last_became_active":"2024-04-04T06:07:09.421115+0000","last_became_peered":"2024-04-04T06:07:09.421115+0000","last_unstale":"2024-04-04T06:07:10.419502+0000","last_undegraded":"2024-04-04T06:07:10.419502+0000","last_fullsized":"2024-04-04T06:07:10.419502+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:51:20.551855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.127155+0000","last_change":"2024-04-04T06:07:08.395148+0000","last_active":"2024-04-04T06:07:11.127155+0000","last_peered":"2024-04-04T06:07:11.127155+0000","last_clean":"2024-04-04T06:07:11.127155+0000","last_became_active":"2024-04-04T06:07:08.394882+0000","last_became_peered":"2024-04-04T06:07:08.394882+0000","last_unstale":"2024-04-04T06:07:11.127155+0000","last_undegraded":"2024-04-04T06:07:11.127155+0000","last_fullsized":"2024-04-04T06:07:11.127155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:11:10.659591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.667335+0000","last_change":"2024-04-04T06:07:09.429518+0000","last_active":"2024-04-04T06:07:10.667335+0000","last_peered":"2024-04-04T06:07:10.667335+0000","last_clean":"2024-04-04T06:07:10.667335+0000","last_became_active":"2024-04-04T06:07:09.428398+0000","last_became_peered":"2024-04-04T06:07:09.428398+0000","last_unstale":"2024-04-04T06:07:10.667335+0000","last_undegraded":"2024-04-04T06:07:10.667335+0000","last_fullsized":"2024-04-04T06:07:10.667335+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:08:18.706750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.125733+0000","last_change":"2024-04-04T06:07:08.397414+0000","last_active":"2024-04-04T06:07:11.125733+0000","last_peered":"2024-04-04T06:07:11.125733+0000","last_clean":"2024-04-04T06:07:11.125733+0000","last_became_active":"2024-04-04T06:07:08.397084+0000","last_became_peered":"2024-04-04T06:07:08.397084+0000","last_unstale":"2024-04-04T06:07:11.125733+0000","last_undegraded":"2024-04-04T06:07:11.125733+0000","last_fullsized":"2024-04-04T06:07:11.125733+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:18:48.789573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":192,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.666453+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:07:10.666453+0000","last_peered":"2024-04-04T06:07:10.666453+0000","last_clean":"2024-04-04T06:07:10.666453+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:07:10.666453+0000","last_undegraded":"2024-04-04T06:07:10.666453+0000","last_fullsized":"2024-04-04T06:07:10.666453+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509460,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":974848,"data_stored":738947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":372736,"data_stored":142805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":150123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":148559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":150123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":738857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542166,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":385024,"data_stored":149161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542164,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":372736,"data_stored":142293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:07:13.850 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:07:14.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.111+0000 7f2509400640 1 Processor -- start 2024-04-04T06:07:14.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.111+0000 7f2509400640 1 -- start start 2024-04-04T06:07:14.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.112+0000 7f2509400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f250415c390 0x7f2504073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:14.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.112+0000 7f2509400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25040d0650 0x7f25040d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:14.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.112+0000 7f2509400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f250415fe80 con 0x7f25040af830 2024-04-04T06:07:14.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.112+0000 7f2509400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f250415c770 con 0x7f250415c390 2024-04-04T06:07:14.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.112+0000 7f2509400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f25040ad120 con 0x7f25040d0650 2024-04-04T06:07:14.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.113+0000 7f2502800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f250415c390 0x7f2504073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:14.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.113+0000 7f2502800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f250415c390 0x7f2504073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:35512/0 (socket says 172.21.15.159:35512) 2024-04-04T06:07:14.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.113+0000 7f2502800640 1 -- 172.21.15.159:0/2995881806 learned_addr learned my addr 172.21.15.159:0/2995881806 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:14.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.113+0000 7f2503200640 1 --2- 172.21.15.159:0/2995881806 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25040d0650 0x7f25040d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:14.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.114+0000 7f2502800640 1 -- 172.21.15.159:0/2995881806 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25040d0650 msgr2=0x7f25040d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:14.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.114+0000 7f2502800640 1 --2- 172.21.15.159:0/2995881806 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25040d0650 0x7f25040d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.114+0000 7f2502800640 1 -- 172.21.15.159:0/2995881806 >> v1:172.21.15.178:6789/0 conn(0x7f25040af830 legacy=0x7f2504160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:14.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.114+0000 7f2502800640 1 -- 172.21.15.159:0/2995881806 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25040ad400 con 0x7f250415c390 2024-04-04T06:07:14.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.115+0000 7f2502800640 1 --2- 172.21.15.159:0/2995881806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f250415c390 0x7f2504073790 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f24f4009870 tx=0x7f24f402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=769ba02bebf492a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:14.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.116+0000 7f2501400640 1 -- 172.21.15.159:0/2995881806 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f4034070 con 0x7f250415c390 2024-04-04T06:07:14.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.116+0000 7f2501400640 1 -- 172.21.15.159:0/2995881806 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f4030840 con 0x7f250415c390 2024-04-04T06:07:14.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.116+0000 7f2501400640 1 -- 172.21.15.159:0/2995881806 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f4030c10 con 0x7f250415c390 2024-04-04T06:07:14.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.116+0000 7f2509400640 1 -- 172.21.15.159:0/2995881806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f250415c390 msgr2=0x7f2504073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:14.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.116+0000 7f2509400640 1 --2- 172.21.15.159:0/2995881806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f250415c390 0x7f2504073790 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f24f4009870 tx=0x7f24f402b980 comp rx=0 tx=0).stop 2024-04-04T06:07:14.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.117+0000 7f2509400640 1 -- 172.21.15.159:0/2995881806 shutdown_connections 2024-04-04T06:07:14.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.117+0000 7f2509400640 1 --2- 172.21.15.159:0/2995881806 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f25040d0650 0x7f25040d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.117+0000 7f2509400640 1 --2- 172.21.15.159:0/2995881806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f250415c390 0x7f2504073790 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.117+0000 7f2509400640 1 -- 172.21.15.159:0/2995881806 >> 172.21.15.159:0/2995881806 conn(0x7f250406b990 msgr2=0x7f250406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:14.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.117+0000 7f2509400640 1 -- 172.21.15.159:0/2995881806 shutdown_connections 2024-04-04T06:07:14.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.117+0000 7f2509400640 1 -- 172.21.15.159:0/2995881806 wait complete. 2024-04-04T06:07:14.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.119+0000 7f2509400640 1 Processor -- start 2024-04-04T06:07:14.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.119+0000 7f2509400640 1 -- start start 2024-04-04T06:07:14.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.120+0000 7f2509400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25040af830 0x7f25040727d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:14.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.120+0000 7f2509400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25040d0650 0x7f250415a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:14.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.120+0000 7f2509400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f250415c390 0x7f250414cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:14.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.120+0000 7f2509400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2504160d50 con 0x7f25040d0650 2024-04-04T06:07:14.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.120+0000 7f2509400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2504160120 con 0x7f25040af830 2024-04-04T06:07:14.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.120+0000 7f2509400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f25040cf980 con 0x7f250415c390 2024-04-04T06:07:14.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.120+0000 7f2501e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25040d0650 0x7f250415a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:14.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.120+0000 7f2501e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25040d0650 0x7f250415a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53194/0 (socket says 172.21.15.159:53194) 2024-04-04T06:07:14.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.120+0000 7f2501e00640 1 -- 172.21.15.159:0/1556527959 learned_addr learned my addr 172.21.15.159:0/1556527959 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:14.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.121+0000 7f2502800640 1 --2- 172.21.15.159:0/1556527959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25040af830 0x7f25040727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:14.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.121+0000 7f2503200640 1 --2- 172.21.15.159:0/1556527959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f250415c390 0x7f250414cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:14.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.121+0000 7f2501e00640 1 -- 172.21.15.159:0/1556527959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f250415c390 msgr2=0x7f250414cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:14.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.121+0000 7f2501e00640 1 --2- 172.21.15.159:0/1556527959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f250415c390 0x7f250414cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.121+0000 7f2501e00640 1 -- 172.21.15.159:0/1556527959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25040af830 msgr2=0x7f25040727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:14.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.121+0000 7f2501e00640 1 --2- 172.21.15.159:0/1556527959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25040af830 0x7f25040727d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.121+0000 7f2501e00640 1 -- 172.21.15.159:0/1556527959 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25040b3370 con 0x7f25040d0650 2024-04-04T06:07:14.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.121+0000 7f2503200640 1 --2- 172.21.15.159:0/1556527959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f250415c390 0x7f250414cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:14.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.121+0000 7f2501e00640 1 --2- 172.21.15.159:0/1556527959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25040d0650 0x7f250415a450 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f24f000c400 tx=0x7f24f000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:14.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.121+0000 7f24faa00640 1 -- 172.21.15.159:0/1556527959 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f00052d0 con 0x7f25040d0650 2024-04-04T06:07:14.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.121+0000 7f24faa00640 1 -- 172.21.15.159:0/1556527959 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f0005490 con 0x7f25040d0650 2024-04-04T06:07:14.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.121+0000 7f24faa00640 1 -- 172.21.15.159:0/1556527959 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f0007260 con 0x7f25040d0650 2024-04-04T06:07:14.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.121+0000 7f2509400640 1 -- 172.21.15.159:0/1556527959 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25040b3650 con 0x7f25040d0650 2024-04-04T06:07:14.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.122+0000 7f2509400640 1 -- 172.21.15.159:0/1556527959 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25040b3c10 con 0x7f25040d0650 2024-04-04T06:07:14.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.123+0000 7f2509400640 1 -- 172.21.15.159:0/1556527959 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25040685c0 con 0x7f25040d0650 2024-04-04T06:07:14.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.124+0000 7f24faa00640 1 -- 172.21.15.159:0/1556527959 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24f0007400 con 0x7f25040d0650 2024-04-04T06:07:14.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.124+0000 7f24faa00640 1 --2- 172.21.15.159:0/1556527959 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f24e406c920 0x7f24e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:14.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.125+0000 7f24faa00640 1 -- 172.21.15.159:0/1556527959 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f24f008b070 con 0x7f25040d0650 2024-04-04T06:07:14.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.125+0000 7f2502800640 1 --2- 172.21.15.159:0/1556527959 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f24e406c920 0x7f24e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:14.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.126+0000 7f2502800640 1 --2- 172.21.15.159:0/1556527959 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f24e406c920 0x7f24e406ede0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f24f4036000 tx=0x7f24f403b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:14.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.129+0000 7f24faa00640 1 -- 172.21.15.159:0/1556527959 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f24f0059490 con 0x7f25040d0650 2024-04-04T06:07:14.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.326+0000 7f2509400640 1 -- 172.21.15.159:0/1556527959 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25040b3de0 con 0x7f24e406c920 2024-04-04T06:07:14.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.346+0000 7f24faa00640 1 -- 172.21.15.159:0/1556527959 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7f25040b3de0 con 0x7f24e406c920 2024-04-04T06:07:14.345 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:14.346 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:07:14.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.353+0000 7f2509400640 1 -- 172.21.15.159:0/1556527959 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f24e406c920 msgr2=0x7f24e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:14.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.353+0000 7f2509400640 1 --2- 172.21.15.159:0/1556527959 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f24e406c920 0x7f24e406ede0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f24f4036000 tx=0x7f24f403b040 comp rx=0 tx=0).stop 2024-04-04T06:07:14.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.353+0000 7f2509400640 1 -- 172.21.15.159:0/1556527959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25040d0650 msgr2=0x7f250415a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:14.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.353+0000 7f2509400640 1 --2- 172.21.15.159:0/1556527959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25040d0650 0x7f250415a450 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f24f000c400 tx=0x7f24f000c8c0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.353+0000 7f2509400640 1 -- 172.21.15.159:0/1556527959 shutdown_connections 2024-04-04T06:07:14.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.353+0000 7f2509400640 1 --2- 172.21.15.159:0/1556527959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f250415c390 0x7f250414cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.353+0000 7f2509400640 1 --2- 172.21.15.159:0/1556527959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25040d0650 0x7f250415a450 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.353+0000 7f2509400640 1 --2- 172.21.15.159:0/1556527959 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f24e406c920 0x7f24e406ede0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.353+0000 7f2509400640 1 --2- 172.21.15.159:0/1556527959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25040af830 0x7f25040727d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.353+0000 7f2509400640 1 -- 172.21.15.159:0/1556527959 >> 172.21.15.159:0/1556527959 conn(0x7f250406b990 msgr2=0x7f25040b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:14.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.353+0000 7f2509400640 1 -- 172.21.15.159:0/1556527959 shutdown_connections 2024-04-04T06:07:14.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.353+0000 7f2509400640 1 -- 172.21.15.159:0/1556527959 wait complete. 2024-04-04T06:07:14.368 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-04-04T06:07:12.737952+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236968,"kb_used_data":4144,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737752,"statfs":{"total":773094113280,"available":772851458048,"internally_reserved":0,"allocated":4243456,"data_stored":2360868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.384537"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126160+0000","last_change":"2024-04-04T06:07:09.415112+0000","last_active":"2024-04-04T06:07:11.126160+0000","last_peered":"2024-04-04T06:07:11.126160+0000","last_clean":"2024-04-04T06:07:11.126160+0000","last_became_active":"2024-04-04T06:07:09.414812+0000","last_became_peered":"2024-04-04T06:07:09.414812+0000","last_unstale":"2024-04-04T06:07:11.126160+0000","last_undegraded":"2024-04-04T06:07:11.126160+0000","last_fullsized":"2024-04-04T06:07:11.126160+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:04:36.434377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.128487+0000","last_change":"2024-04-04T06:07:09.424013+0000","last_active":"2024-04-04T06:07:11.128487+0000","last_peered":"2024-04-04T06:07:11.128487+0000","last_clean":"2024-04-04T06:07:11.128487+0000","last_became_active":"2024-04-04T06:07:09.423100+0000","last_became_peered":"2024-04-04T06:07:09.423100+0000","last_unstale":"2024-04-04T06:07:11.128487+0000","last_undegraded":"2024-04-04T06:07:11.128487+0000","last_fullsized":"2024-04-04T06:07:11.128487+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:38:33.849951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.420075+0000","last_change":"2024-04-04T06:07:09.429037+0000","last_active":"2024-04-04T06:07:10.420075+0000","last_peered":"2024-04-04T06:07:10.420075+0000","last_clean":"2024-04-04T06:07:10.420075+0000","last_became_active":"2024-04-04T06:07:09.428784+0000","last_became_peered":"2024-04-04T06:07:09.428784+0000","last_unstale":"2024-04-04T06:07:10.420075+0000","last_undegraded":"2024-04-04T06:07:10.420075+0000","last_fullsized":"2024-04-04T06:07:10.420075+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:14:07.097099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417626+0000","last_change":"2024-04-04T06:07:09.425498+0000","last_active":"2024-04-04T06:07:10.417626+0000","last_peered":"2024-04-04T06:07:10.417626+0000","last_clean":"2024-04-04T06:07:10.417626+0000","last_became_active":"2024-04-04T06:07:09.425232+0000","last_became_peered":"2024-04-04T06:07:09.425232+0000","last_unstale":"2024-04-04T06:07:10.417626+0000","last_undegraded":"2024-04-04T06:07:10.417626+0000","last_fullsized":"2024-04-04T06:07:10.417626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:08:47.952291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.127199+0000","last_change":"2024-04-04T06:07:09.417405+0000","last_active":"2024-04-04T06:07:11.127199+0000","last_peered":"2024-04-04T06:07:11.127199+0000","last_clean":"2024-04-04T06:07:11.127199+0000","last_became_active":"2024-04-04T06:07:09.417047+0000","last_became_peered":"2024-04-04T06:07:09.417047+0000","last_unstale":"2024-04-04T06:07:11.127199+0000","last_undegraded":"2024-04-04T06:07:11.127199+0000","last_fullsized":"2024-04-04T06:07:11.127199+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:44:15.339915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126401+0000","last_change":"2024-04-04T06:07:09.406370+0000","last_active":"2024-04-04T06:07:11.126401+0000","last_peered":"2024-04-04T06:07:11.126401+0000","last_clean":"2024-04-04T06:07:11.126401+0000","last_became_active":"2024-04-04T06:07:09.405814+0000","last_became_peered":"2024-04-04T06:07:09.405814+0000","last_unstale":"2024-04-04T06:07:11.126401+0000","last_undegraded":"2024-04-04T06:07:11.126401+0000","last_fullsized":"2024-04-04T06:07:11.126401+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:57:22.011487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.667703+0000","last_change":"2024-04-04T06:07:09.431344+0000","last_active":"2024-04-04T06:07:10.667703+0000","last_peered":"2024-04-04T06:07:10.667703+0000","last_clean":"2024-04-04T06:07:10.667703+0000","last_became_active":"2024-04-04T06:07:09.431041+0000","last_became_peered":"2024-04-04T06:07:09.431041+0000","last_unstale":"2024-04-04T06:07:10.667703+0000","last_undegraded":"2024-04-04T06:07:10.667703+0000","last_fullsized":"2024-04-04T06:07:10.667703+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:21:22.868588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.666769+0000","last_change":"2024-04-04T06:07:09.424869+0000","last_active":"2024-04-04T06:07:10.666769+0000","last_peered":"2024-04-04T06:07:10.666769+0000","last_clean":"2024-04-04T06:07:10.666769+0000","last_became_active":"2024-04-04T06:07:09.424384+0000","last_became_peered":"2024-04-04T06:07:09.424384+0000","last_unstale":"2024-04-04T06:07:10.666769+0000","last_undegraded":"2024-04-04T06:07:10.666769+0000","last_fullsized":"2024-04-04T06:07:10.666769+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:54:36.113422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418995+0000","last_change":"2024-04-04T06:07:09.420985+0000","last_active":"2024-04-04T06:07:10.418995+0000","last_peered":"2024-04-04T06:07:10.418995+0000","last_clean":"2024-04-04T06:07:10.418995+0000","last_became_active":"2024-04-04T06:07:09.420629+0000","last_became_peered":"2024-04-04T06:07:09.420629+0000","last_unstale":"2024-04-04T06:07:10.418995+0000","last_undegraded":"2024-04-04T06:07:10.418995+0000","last_fullsized":"2024-04-04T06:07:10.418995+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:04:43.784625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417424+0000","last_change":"2024-04-04T06:07:09.422931+0000","last_active":"2024-04-04T06:07:10.417424+0000","last_peered":"2024-04-04T06:07:10.417424+0000","last_clean":"2024-04-04T06:07:10.417424+0000","last_became_active":"2024-04-04T06:07:09.422628+0000","last_became_peered":"2024-04-04T06:07:09.422628+0000","last_unstale":"2024-04-04T06:07:10.417424+0000","last_undegraded":"2024-04-04T06:07:10.417424+0000","last_fullsized":"2024-04-04T06:07:10.417424+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:41:35.695728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.667342+0000","last_change":"2024-04-04T06:07:09.430515+0000","last_active":"2024-04-04T06:07:10.667342+0000","last_peered":"2024-04-04T06:07:10.667342+0000","last_clean":"2024-04-04T06:07:10.667342+0000","last_became_active":"2024-04-04T06:07:09.429760+0000","last_became_peered":"2024-04-04T06:07:09.429760+0000","last_unstale":"2024-04-04T06:07:10.667342+0000","last_undegraded":"2024-04-04T06:07:10.667342+0000","last_fullsized":"2024-04-04T06:07:10.667342+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:13:29.321236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.419566+0000","last_change":"2024-04-04T06:07:09.419125+0000","last_active":"2024-04-04T06:07:10.419566+0000","last_peered":"2024-04-04T06:07:10.419566+0000","last_clean":"2024-04-04T06:07:10.419566+0000","last_became_active":"2024-04-04T06:07:09.416934+0000","last_became_peered":"2024-04-04T06:07:09.416934+0000","last_unstale":"2024-04-04T06:07:10.419566+0000","last_undegraded":"2024-04-04T06:07:10.419566+0000","last_fullsized":"2024-04-04T06:07:10.419566+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:51:20.551855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126885+0000","last_change":"2024-04-04T06:07:09.421613+0000","last_active":"2024-04-04T06:07:11.126885+0000","last_peered":"2024-04-04T06:07:11.126885+0000","last_clean":"2024-04-04T06:07:11.126885+0000","last_became_active":"2024-04-04T06:07:09.421273+0000","last_became_peered":"2024-04-04T06:07:09.421273+0000","last_unstale":"2024-04-04T06:07:11.126885+0000","last_undegraded":"2024-04-04T06:07:11.126885+0000","last_fullsized":"2024-04-04T06:07:11.126885+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:05:49.177763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126567+0000","last_change":"2024-04-04T06:07:09.413255+0000","last_active":"2024-04-04T06:07:11.126567+0000","last_peered":"2024-04-04T06:07:11.126567+0000","last_clean":"2024-04-04T06:07:11.126567+0000","last_became_active":"2024-04-04T06:07:09.412791+0000","last_became_peered":"2024-04-04T06:07:09.412791+0000","last_unstale":"2024-04-04T06:07:11.126567+0000","last_undegraded":"2024-04-04T06:07:11.126567+0000","last_fullsized":"2024-04-04T06:07:11.126567+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:24:57.295679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418532+0000","last_change":"2024-04-04T06:07:09.413108+0000","last_active":"2024-04-04T06:07:10.418532+0000","last_peered":"2024-04-04T06:07:10.418532+0000","last_clean":"2024-04-04T06:07:10.418532+0000","last_became_active":"2024-04-04T06:07:09.412778+0000","last_became_peered":"2024-04-04T06:07:09.412778+0000","last_unstale":"2024-04-04T06:07:10.418532+0000","last_undegraded":"2024-04-04T06:07:10.418532+0000","last_fullsized":"2024-04-04T06:07:10.418532+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:08:01.796146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.667999+0000","last_change":"2024-04-04T06:07:09.424640+0000","last_active":"2024-04-04T06:07:10.667999+0000","last_peered":"2024-04-04T06:07:10.667999+0000","last_clean":"2024-04-04T06:07:10.667999+0000","last_became_active":"2024-04-04T06:07:09.423981+0000","last_became_peered":"2024-04-04T06:07:09.423981+0000","last_unstale":"2024-04-04T06:07:10.667999+0000","last_undegraded":"2024-04-04T06:07:10.667999+0000","last_fullsized":"2024-04-04T06:07:10.667999+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:34:51.034798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.667828+0000","last_change":"2024-04-04T06:07:09.433800+0000","last_active":"2024-04-04T06:07:10.667828+0000","last_peered":"2024-04-04T06:07:10.667828+0000","last_clean":"2024-04-04T06:07:10.667828+0000","last_became_active":"2024-04-04T06:07:09.431162+0000","last_became_peered":"2024-04-04T06:07:09.431162+0000","last_unstale":"2024-04-04T06:07:10.667828+0000","last_undegraded":"2024-04-04T06:07:10.667828+0000","last_fullsized":"2024-04-04T06:07:10.667828+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:48:08.743467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.128738+0000","last_change":"2024-04-04T06:07:09.428272+0000","last_active":"2024-04-04T06:07:11.128738+0000","last_peered":"2024-04-04T06:07:11.128738+0000","last_clean":"2024-04-04T06:07:11.128738+0000","last_became_active":"2024-04-04T06:07:09.423584+0000","last_became_peered":"2024-04-04T06:07:09.423584+0000","last_unstale":"2024-04-04T06:07:11.128738+0000","last_undegraded":"2024-04-04T06:07:11.128738+0000","last_fullsized":"2024-04-04T06:07:11.128738+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:30:20.672617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.127594+0000","last_change":"2024-04-04T06:07:09.432751+0000","last_active":"2024-04-04T06:07:11.127594+0000","last_peered":"2024-04-04T06:07:11.127594+0000","last_clean":"2024-04-04T06:07:11.127594+0000","last_became_active":"2024-04-04T06:07:09.432487+0000","last_became_peered":"2024-04-04T06:07:09.432487+0000","last_unstale":"2024-04-04T06:07:11.127594+0000","last_undegraded":"2024-04-04T06:07:11.127594+0000","last_fullsized":"2024-04-04T06:07:11.127594+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:14:07.097099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.127229+0000","last_change":"2024-04-04T06:07:09.417098+0000","last_active":"2024-04-04T06:07:11.127229+0000","last_peered":"2024-04-04T06:07:11.127229+0000","last_clean":"2024-04-04T06:07:11.127229+0000","last_became_active":"2024-04-04T06:07:09.416830+0000","last_became_peered":"2024-04-04T06:07:09.416830+0000","last_unstale":"2024-04-04T06:07:11.127229+0000","last_undegraded":"2024-04-04T06:07:11.127229+0000","last_fullsized":"2024-04-04T06:07:11.127229+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:53:43.115602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417847+0000","last_change":"2024-04-04T06:07:09.421377+0000","last_active":"2024-04-04T06:07:10.417847+0000","last_peered":"2024-04-04T06:07:10.417847+0000","last_clean":"2024-04-04T06:07:10.417847+0000","last_became_active":"2024-04-04T06:07:09.421064+0000","last_became_peered":"2024-04-04T06:07:09.421064+0000","last_unstale":"2024-04-04T06:07:10.417847+0000","last_undegraded":"2024-04-04T06:07:10.417847+0000","last_fullsized":"2024-04-04T06:07:10.417847+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:41:35.695728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418156+0000","last_change":"2024-04-04T06:07:09.427146+0000","last_active":"2024-04-04T06:07:10.418156+0000","last_peered":"2024-04-04T06:07:10.418156+0000","last_clean":"2024-04-04T06:07:10.418156+0000","last_became_active":"2024-04-04T06:07:09.424868+0000","last_became_peered":"2024-04-04T06:07:09.424868+0000","last_unstale":"2024-04-04T06:07:10.418156+0000","last_undegraded":"2024-04-04T06:07:10.418156+0000","last_fullsized":"2024-04-04T06:07:10.418156+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:49:38.280388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.423758+0000","last_change":"2024-04-04T06:07:09.435325+0000","last_active":"2024-04-04T06:07:10.423758+0000","last_peered":"2024-04-04T06:07:10.423758+0000","last_clean":"2024-04-04T06:07:10.423758+0000","last_became_active":"2024-04-04T06:07:09.430681+0000","last_became_peered":"2024-04-04T06:07:09.430681+0000","last_unstale":"2024-04-04T06:07:10.423758+0000","last_undegraded":"2024-04-04T06:07:10.423758+0000","last_fullsized":"2024-04-04T06:07:10.423758+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:52:08.974731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417708+0000","last_change":"2024-04-04T06:07:09.417505+0000","last_active":"2024-04-04T06:07:10.417708+0000","last_peered":"2024-04-04T06:07:10.417708+0000","last_clean":"2024-04-04T06:07:10.417708+0000","last_became_active":"2024-04-04T06:07:09.417260+0000","last_became_peered":"2024-04-04T06:07:09.417260+0000","last_unstale":"2024-04-04T06:07:10.417708+0000","last_undegraded":"2024-04-04T06:07:10.417708+0000","last_fullsized":"2024-04-04T06:07:10.417708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:47:54.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418889+0000","last_change":"2024-04-04T06:07:09.431743+0000","last_active":"2024-04-04T06:07:10.418889+0000","last_peered":"2024-04-04T06:07:10.418889+0000","last_clean":"2024-04-04T06:07:10.418889+0000","last_became_active":"2024-04-04T06:07:09.431463+0000","last_became_peered":"2024-04-04T06:07:09.431463+0000","last_unstale":"2024-04-04T06:07:10.418889+0000","last_undegraded":"2024-04-04T06:07:10.418889+0000","last_fullsized":"2024-04-04T06:07:10.418889+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:51:20.551855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.128713+0000","last_change":"2024-04-04T06:07:09.420002+0000","last_active":"2024-04-04T06:07:11.128713+0000","last_peered":"2024-04-04T06:07:11.128713+0000","last_clean":"2024-04-04T06:07:11.128713+0000","last_became_active":"2024-04-04T06:07:09.419685+0000","last_became_peered":"2024-04-04T06:07:09.419685+0000","last_unstale":"2024-04-04T06:07:11.128713+0000","last_undegraded":"2024-04-04T06:07:11.128713+0000","last_fullsized":"2024-04-04T06:07:11.128713+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:30:04.395085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418308+0000","last_change":"2024-04-04T06:07:09.433952+0000","last_active":"2024-04-04T06:07:10.418308+0000","last_peered":"2024-04-04T06:07:10.418308+0000","last_clean":"2024-04-04T06:07:10.418308+0000","last_became_active":"2024-04-04T06:07:09.433656+0000","last_became_peered":"2024-04-04T06:07:09.433656+0000","last_unstale":"2024-04-04T06:07:10.418308+0000","last_undegraded":"2024-04-04T06:07:10.418308+0000","last_fullsized":"2024-04-04T06:07:10.418308+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:08:01.796146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.419995+0000","last_change":"2024-04-04T06:07:09.427397+0000","last_active":"2024-04-04T06:07:10.419995+0000","last_peered":"2024-04-04T06:07:10.419995+0000","last_clean":"2024-04-04T06:07:10.419995+0000","last_became_active":"2024-04-04T06:07:09.427137+0000","last_became_peered":"2024-04-04T06:07:09.427137+0000","last_unstale":"2024-04-04T06:07:10.419995+0000","last_undegraded":"2024-04-04T06:07:10.419995+0000","last_fullsized":"2024-04-04T06:07:10.419995+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:24:57.295679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417871+0000","last_change":"2024-04-04T06:07:09.422128+0000","last_active":"2024-04-04T06:07:10.417871+0000","last_peered":"2024-04-04T06:07:10.417871+0000","last_clean":"2024-04-04T06:07:10.417871+0000","last_became_active":"2024-04-04T06:07:09.421687+0000","last_became_peered":"2024-04-04T06:07:09.421687+0000","last_unstale":"2024-04-04T06:07:10.417871+0000","last_undegraded":"2024-04-04T06:07:10.417871+0000","last_fullsized":"2024-04-04T06:07:10.417871+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:08:47.952291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.667063+0000","last_change":"2024-04-04T06:07:09.431813+0000","last_active":"2024-04-04T06:07:10.667063+0000","last_peered":"2024-04-04T06:07:10.667063+0000","last_clean":"2024-04-04T06:07:10.667063+0000","last_became_active":"2024-04-04T06:07:09.431562+0000","last_became_peered":"2024-04-04T06:07:09.431562+0000","last_unstale":"2024-04-04T06:07:10.667063+0000","last_undegraded":"2024-04-04T06:07:10.667063+0000","last_fullsized":"2024-04-04T06:07:10.667063+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:56:24.411309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.125768+0000","last_change":"2024-04-04T06:07:09.421243+0000","last_active":"2024-04-04T06:07:11.125768+0000","last_peered":"2024-04-04T06:07:11.125768+0000","last_clean":"2024-04-04T06:07:11.125768+0000","last_became_active":"2024-04-04T06:07:09.420978+0000","last_became_peered":"2024-04-04T06:07:09.420978+0000","last_unstale":"2024-04-04T06:07:11.125768+0000","last_undegraded":"2024-04-04T06:07:11.125768+0000","last_fullsized":"2024-04-04T06:07:11.125768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:55:41.865738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.667329+0000","last_change":"2024-04-04T06:07:09.424249+0000","last_active":"2024-04-04T06:07:10.667329+0000","last_peered":"2024-04-04T06:07:10.667329+0000","last_clean":"2024-04-04T06:07:10.667329+0000","last_became_active":"2024-04-04T06:07:09.423645+0000","last_became_peered":"2024-04-04T06:07:09.423645+0000","last_unstale":"2024-04-04T06:07:10.667329+0000","last_undegraded":"2024-04-04T06:07:10.667329+0000","last_fullsized":"2024-04-04T06:07:10.667329+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:41:40.027163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.424121+0000","last_change":"2024-04-04T06:07:09.435761+0000","last_active":"2024-04-04T06:07:10.424121+0000","last_peered":"2024-04-04T06:07:10.424121+0000","last_clean":"2024-04-04T06:07:10.424121+0000","last_became_active":"2024-04-04T06:07:09.431040+0000","last_became_peered":"2024-04-04T06:07:09.431040+0000","last_unstale":"2024-04-04T06:07:10.424121+0000","last_undegraded":"2024-04-04T06:07:10.424121+0000","last_fullsized":"2024-04-04T06:07:10.424121+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:24:08.463853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126416+0000","last_change":"2024-04-04T06:07:08.395896+0000","last_active":"2024-04-04T06:07:11.126416+0000","last_peered":"2024-04-04T06:07:11.126416+0000","last_clean":"2024-04-04T06:07:11.126416+0000","last_became_active":"2024-04-04T06:07:08.395526+0000","last_became_peered":"2024-04-04T06:07:08.395526+0000","last_unstale":"2024-04-04T06:07:11.126416+0000","last_undegraded":"2024-04-04T06:07:11.126416+0000","last_fullsized":"2024-04-04T06:07:11.126416+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:15:31.320764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126289+0000","last_change":"2024-04-04T06:07:09.426596+0000","last_active":"2024-04-04T06:07:11.126289+0000","last_peered":"2024-04-04T06:07:11.126289+0000","last_clean":"2024-04-04T06:07:11.126289+0000","last_became_active":"2024-04-04T06:07:09.426252+0000","last_became_peered":"2024-04-04T06:07:09.426252+0000","last_unstale":"2024-04-04T06:07:11.126289+0000","last_undegraded":"2024-04-04T06:07:11.126289+0000","last_fullsized":"2024-04-04T06:07:11.126289+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:27:48.592561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.424311+0000","last_change":"2024-04-04T06:07:08.393414+0000","last_active":"2024-04-04T06:07:10.424311+0000","last_peered":"2024-04-04T06:07:10.424311+0000","last_clean":"2024-04-04T06:07:10.424311+0000","last_became_active":"2024-04-04T06:07:08.393161+0000","last_became_peered":"2024-04-04T06:07:08.393161+0000","last_unstale":"2024-04-04T06:07:10.424311+0000","last_undegraded":"2024-04-04T06:07:10.424311+0000","last_fullsized":"2024-04-04T06:07:10.424311+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:53:42.102358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418401+0000","last_change":"2024-04-04T06:07:09.420683+0000","last_active":"2024-04-04T06:07:10.418401+0000","last_peered":"2024-04-04T06:07:10.418401+0000","last_clean":"2024-04-04T06:07:10.418401+0000","last_became_active":"2024-04-04T06:07:09.420379+0000","last_became_peered":"2024-04-04T06:07:09.420379+0000","last_unstale":"2024-04-04T06:07:10.418401+0000","last_undegraded":"2024-04-04T06:07:10.418401+0000","last_fullsized":"2024-04-04T06:07:10.418401+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:47:54.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.127892+0000","last_change":"2024-04-04T06:07:08.396008+0000","last_active":"2024-04-04T06:07:11.127892+0000","last_peered":"2024-04-04T06:07:11.127892+0000","last_clean":"2024-04-04T06:07:11.127892+0000","last_became_active":"2024-04-04T06:07:08.394919+0000","last_became_peered":"2024-04-04T06:07:08.394919+0000","last_unstale":"2024-04-04T06:07:11.127892+0000","last_undegraded":"2024-04-04T06:07:11.127892+0000","last_fullsized":"2024-04-04T06:07:11.127892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:20:05.428424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418450+0000","last_change":"2024-04-04T06:07:09.421981+0000","last_active":"2024-04-04T06:07:10.418450+0000","last_peered":"2024-04-04T06:07:10.418450+0000","last_clean":"2024-04-04T06:07:10.418450+0000","last_became_active":"2024-04-04T06:07:09.421716+0000","last_became_peered":"2024-04-04T06:07:09.421716+0000","last_unstale":"2024-04-04T06:07:10.418450+0000","last_undegraded":"2024-04-04T06:07:10.418450+0000","last_fullsized":"2024-04-04T06:07:10.418450+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:04:43.784625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.127006+0000","last_change":"2024-04-04T06:07:08.386261+0000","last_active":"2024-04-04T06:07:11.127006+0000","last_peered":"2024-04-04T06:07:11.127006+0000","last_clean":"2024-04-04T06:07:11.127006+0000","last_became_active":"2024-04-04T06:07:08.385957+0000","last_became_peered":"2024-04-04T06:07:08.385957+0000","last_unstale":"2024-04-04T06:07:11.127006+0000","last_undegraded":"2024-04-04T06:07:11.127006+0000","last_fullsized":"2024-04-04T06:07:11.127006+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:06:21.181613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.667298+0000","last_change":"2024-04-04T06:07:09.431316+0000","last_active":"2024-04-04T06:07:10.667298+0000","last_peered":"2024-04-04T06:07:10.667298+0000","last_clean":"2024-04-04T06:07:10.667298+0000","last_became_active":"2024-04-04T06:07:09.430996+0000","last_became_peered":"2024-04-04T06:07:09.430996+0000","last_unstale":"2024-04-04T06:07:10.667298+0000","last_undegraded":"2024-04-04T06:07:10.667298+0000","last_fullsized":"2024-04-04T06:07:10.667298+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:36:37.218048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417962+0000","last_change":"2024-04-04T06:07:08.385627+0000","last_active":"2024-04-04T06:07:10.417962+0000","last_peered":"2024-04-04T06:07:10.417962+0000","last_clean":"2024-04-04T06:07:10.417962+0000","last_became_active":"2024-04-04T06:07:08.384853+0000","last_became_peered":"2024-04-04T06:07:08.384853+0000","last_unstale":"2024-04-04T06:07:10.417962+0000","last_undegraded":"2024-04-04T06:07:10.417962+0000","last_fullsized":"2024-04-04T06:07:10.417962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:44:14.326671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.419465+0000","last_change":"2024-04-04T06:07:09.422774+0000","last_active":"2024-04-04T06:07:10.419465+0000","last_peered":"2024-04-04T06:07:10.419465+0000","last_clean":"2024-04-04T06:07:10.419465+0000","last_became_active":"2024-04-04T06:07:09.422526+0000","last_became_peered":"2024-04-04T06:07:09.422526+0000","last_unstale":"2024-04-04T06:07:10.419465+0000","last_undegraded":"2024-04-04T06:07:10.419465+0000","last_fullsized":"2024-04-04T06:07:10.419465+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:04:43.784625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.419498+0000","last_change":"2024-04-04T06:07:08.388038+0000","last_active":"2024-04-04T06:07:10.419498+0000","last_peered":"2024-04-04T06:07:10.419498+0000","last_clean":"2024-04-04T06:07:10.419498+0000","last_became_active":"2024-04-04T06:07:08.387579+0000","last_became_peered":"2024-04-04T06:07:08.387579+0000","last_unstale":"2024-04-04T06:07:10.419498+0000","last_undegraded":"2024-04-04T06:07:10.419498+0000","last_fullsized":"2024-04-04T06:07:10.419498+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:36:10.005773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417818+0000","last_change":"2024-04-04T06:07:09.418402+0000","last_active":"2024-04-04T06:07:10.417818+0000","last_peered":"2024-04-04T06:07:10.417818+0000","last_clean":"2024-04-04T06:07:10.417818+0000","last_became_active":"2024-04-04T06:07:09.417990+0000","last_became_peered":"2024-04-04T06:07:09.417990+0000","last_unstale":"2024-04-04T06:07:10.417818+0000","last_undegraded":"2024-04-04T06:07:10.417818+0000","last_fullsized":"2024-04-04T06:07:10.417818+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:57:12.182305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126772+0000","last_change":"2024-04-04T06:07:08.385346+0000","last_active":"2024-04-04T06:07:11.126772+0000","last_peered":"2024-04-04T06:07:11.126772+0000","last_clean":"2024-04-04T06:07:11.126772+0000","last_became_active":"2024-04-04T06:07:08.385090+0000","last_became_peered":"2024-04-04T06:07:08.385090+0000","last_unstale":"2024-04-04T06:07:11.126772+0000","last_undegraded":"2024-04-04T06:07:11.126772+0000","last_fullsized":"2024-04-04T06:07:11.126772+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:54:31.814804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418304+0000","last_change":"2024-04-04T06:07:09.419242+0000","last_active":"2024-04-04T06:07:10.418304+0000","last_peered":"2024-04-04T06:07:10.418304+0000","last_clean":"2024-04-04T06:07:10.418304+0000","last_became_active":"2024-04-04T06:07:09.418911+0000","last_became_peered":"2024-04-04T06:07:09.418911+0000","last_unstale":"2024-04-04T06:07:10.418304+0000","last_undegraded":"2024-04-04T06:07:10.418304+0000","last_fullsized":"2024-04-04T06:07:10.418304+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:47:54.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417990+0000","last_change":"2024-04-04T06:07:08.392237+0000","last_active":"2024-04-04T06:07:10.417990+0000","last_peered":"2024-04-04T06:07:10.417990+0000","last_clean":"2024-04-04T06:07:10.417990+0000","last_became_active":"2024-04-04T06:07:08.391959+0000","last_became_peered":"2024-04-04T06:07:08.391959+0000","last_unstale":"2024-04-04T06:07:10.417990+0000","last_undegraded":"2024-04-04T06:07:10.417990+0000","last_fullsized":"2024-04-04T06:07:10.417990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:22:30.070585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.420013+0000","last_change":"2024-04-04T06:07:09.417161+0000","last_active":"2024-04-04T06:07:10.420013+0000","last_peered":"2024-04-04T06:07:10.420013+0000","last_clean":"2024-04-04T06:07:10.420013+0000","last_became_active":"2024-04-04T06:07:09.416862+0000","last_became_peered":"2024-04-04T06:07:09.416862+0000","last_unstale":"2024-04-04T06:07:10.420013+0000","last_undegraded":"2024-04-04T06:07:10.420013+0000","last_fullsized":"2024-04-04T06:07:10.420013+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:16:00.835478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.128517+0000","last_change":"2024-04-04T06:07:08.395854+0000","last_active":"2024-04-04T06:07:11.128517+0000","last_peered":"2024-04-04T06:07:11.128517+0000","last_clean":"2024-04-04T06:07:11.128517+0000","last_became_active":"2024-04-04T06:07:08.395053+0000","last_became_peered":"2024-04-04T06:07:08.395053+0000","last_unstale":"2024-04-04T06:07:11.128517+0000","last_undegraded":"2024-04-04T06:07:11.128517+0000","last_fullsized":"2024-04-04T06:07:11.128517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:55:03.146417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.128093+0000","last_change":"2024-04-04T06:07:09.426054+0000","last_active":"2024-04-04T06:07:11.128093+0000","last_peered":"2024-04-04T06:07:11.128093+0000","last_clean":"2024-04-04T06:07:11.128093+0000","last_became_active":"2024-04-04T06:07:09.423539+0000","last_became_peered":"2024-04-04T06:07:09.423539+0000","last_unstale":"2024-04-04T06:07:11.128093+0000","last_undegraded":"2024-04-04T06:07:11.128093+0000","last_fullsized":"2024-04-04T06:07:11.128093+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:53:13.683704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.424352+0000","last_change":"2024-04-04T06:07:08.394398+0000","last_active":"2024-04-04T06:07:10.424352+0000","last_peered":"2024-04-04T06:07:10.424352+0000","last_clean":"2024-04-04T06:07:10.424352+0000","last_became_active":"2024-04-04T06:07:08.392409+0000","last_became_peered":"2024-04-04T06:07:08.392409+0000","last_unstale":"2024-04-04T06:07:10.424352+0000","last_undegraded":"2024-04-04T06:07:10.424352+0000","last_fullsized":"2024-04-04T06:07:10.424352+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:23:43.557850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.128059+0000","last_change":"2024-04-04T06:07:09.420571+0000","last_active":"2024-04-04T06:07:11.128059+0000","last_peered":"2024-04-04T06:07:11.128059+0000","last_clean":"2024-04-04T06:07:11.128059+0000","last_became_active":"2024-04-04T06:07:09.420238+0000","last_became_peered":"2024-04-04T06:07:09.420238+0000","last_unstale":"2024-04-04T06:07:11.128059+0000","last_undegraded":"2024-04-04T06:07:11.128059+0000","last_fullsized":"2024-04-04T06:07:11.128059+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:31:23.568304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.420006+0000","last_change":"2024-04-04T06:07:08.388024+0000","last_active":"2024-04-04T06:07:10.420006+0000","last_peered":"2024-04-04T06:07:10.420006+0000","last_clean":"2024-04-04T06:07:10.420006+0000","last_became_active":"2024-04-04T06:07:08.387356+0000","last_became_peered":"2024-04-04T06:07:08.387356+0000","last_unstale":"2024-04-04T06:07:10.420006+0000","last_undegraded":"2024-04-04T06:07:10.420006+0000","last_fullsized":"2024-04-04T06:07:10.420006+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:20:05.428424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.419850+0000","last_change":"2024-04-04T06:07:09.425331+0000","last_active":"2024-04-04T06:07:10.419850+0000","last_peered":"2024-04-04T06:07:10.419850+0000","last_clean":"2024-04-04T06:07:10.419850+0000","last_became_active":"2024-04-04T06:07:09.424674+0000","last_became_peered":"2024-04-04T06:07:09.424674+0000","last_unstale":"2024-04-04T06:07:10.419850+0000","last_undegraded":"2024-04-04T06:07:10.419850+0000","last_fullsized":"2024-04-04T06:07:10.419850+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:57:22.011487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.128364+0000","last_change":"2024-04-04T06:07:08.390768+0000","last_active":"2024-04-04T06:07:11.128364+0000","last_peered":"2024-04-04T06:07:11.128364+0000","last_clean":"2024-04-04T06:07:11.128364+0000","last_became_active":"2024-04-04T06:07:08.390487+0000","last_became_peered":"2024-04-04T06:07:08.390487+0000","last_unstale":"2024-04-04T06:07:11.128364+0000","last_undegraded":"2024-04-04T06:07:11.128364+0000","last_fullsized":"2024-04-04T06:07:11.128364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:52:07.961487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417238+0000","last_change":"2024-04-04T06:07:09.420879+0000","last_active":"2024-04-04T06:07:10.417238+0000","last_peered":"2024-04-04T06:07:10.417238+0000","last_clean":"2024-04-04T06:07:10.417238+0000","last_became_active":"2024-04-04T06:07:09.418739+0000","last_became_peered":"2024-04-04T06:07:09.418739+0000","last_unstale":"2024-04-04T06:07:10.417238+0000","last_undegraded":"2024-04-04T06:07:10.417238+0000","last_fullsized":"2024-04-04T06:07:10.417238+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:57:12.182305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417967+0000","last_change":"2024-04-04T06:07:08.393867+0000","last_active":"2024-04-04T06:07:10.417967+0000","last_peered":"2024-04-04T06:07:10.417967+0000","last_clean":"2024-04-04T06:07:10.417967+0000","last_became_active":"2024-04-04T06:07:08.393204+0000","last_became_peered":"2024-04-04T06:07:08.393204+0000","last_unstale":"2024-04-04T06:07:10.417967+0000","last_undegraded":"2024-04-04T06:07:10.417967+0000","last_fullsized":"2024-04-04T06:07:10.417967+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:53:42.102358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.128004+0000","last_change":"2024-04-04T06:07:09.434342+0000","last_active":"2024-04-04T06:07:11.128004+0000","last_peered":"2024-04-04T06:07:11.128004+0000","last_clean":"2024-04-04T06:07:11.128004+0000","last_became_active":"2024-04-04T06:07:09.432138+0000","last_became_peered":"2024-04-04T06:07:09.432138+0000","last_unstale":"2024-04-04T06:07:11.128004+0000","last_undegraded":"2024-04-04T06:07:11.128004+0000","last_fullsized":"2024-04-04T06:07:11.128004+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:53:59.963734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.416963+0000","last_change":"2024-04-04T06:07:08.393385+0000","last_active":"2024-04-04T06:07:10.416963+0000","last_peered":"2024-04-04T06:07:10.416963+0000","last_clean":"2024-04-04T06:07:10.416963+0000","last_became_active":"2024-04-04T06:07:08.393078+0000","last_became_peered":"2024-04-04T06:07:08.393078+0000","last_unstale":"2024-04-04T06:07:10.416963+0000","last_undegraded":"2024-04-04T06:07:10.416963+0000","last_fullsized":"2024-04-04T06:07:10.416963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:17:01.423214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.667504+0000","last_change":"2024-04-04T06:07:09.426880+0000","last_active":"2024-04-04T06:07:10.667504+0000","last_peered":"2024-04-04T06:07:10.667504+0000","last_clean":"2024-04-04T06:07:10.667504+0000","last_became_active":"2024-04-04T06:07:09.426586+0000","last_became_peered":"2024-04-04T06:07:09.426586+0000","last_unstale":"2024-04-04T06:07:10.667504+0000","last_undegraded":"2024-04-04T06:07:10.667504+0000","last_fullsized":"2024-04-04T06:07:10.667504+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:03:01.372537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126228+0000","last_change":"2024-04-04T06:07:08.393997+0000","last_active":"2024-04-04T06:07:11.126228+0000","last_peered":"2024-04-04T06:07:11.126228+0000","last_clean":"2024-04-04T06:07:11.126228+0000","last_became_active":"2024-04-04T06:07:08.393594+0000","last_became_peered":"2024-04-04T06:07:08.393594+0000","last_unstale":"2024-04-04T06:07:11.126228+0000","last_undegraded":"2024-04-04T06:07:11.126228+0000","last_fullsized":"2024-04-04T06:07:11.126228+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:29:14.442681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.125930+0000","last_change":"2024-04-04T06:07:09.426992+0000","last_active":"2024-04-04T06:07:11.125930+0000","last_peered":"2024-04-04T06:07:11.125930+0000","last_clean":"2024-04-04T06:07:11.125930+0000","last_became_active":"2024-04-04T06:07:09.426692+0000","last_became_peered":"2024-04-04T06:07:09.426692+0000","last_unstale":"2024-04-04T06:07:11.125930+0000","last_undegraded":"2024-04-04T06:07:11.125930+0000","last_fullsized":"2024-04-04T06:07:11.125930+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:37:44.944983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.127468+0000","last_change":"2024-04-04T06:07:08.391760+0000","last_active":"2024-04-04T06:07:11.127468+0000","last_peered":"2024-04-04T06:07:11.127468+0000","last_clean":"2024-04-04T06:07:11.127468+0000","last_became_active":"2024-04-04T06:07:08.391123+0000","last_became_peered":"2024-04-04T06:07:08.391123+0000","last_unstale":"2024-04-04T06:07:11.127468+0000","last_undegraded":"2024-04-04T06:07:11.127468+0000","last_fullsized":"2024-04-04T06:07:11.127468+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:52:10.694546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418638+0000","last_change":"2024-04-04T06:07:09.419012+0000","last_active":"2024-04-04T06:07:10.418638+0000","last_peered":"2024-04-04T06:07:10.418638+0000","last_clean":"2024-04-04T06:07:10.418638+0000","last_became_active":"2024-04-04T06:07:09.418708+0000","last_became_peered":"2024-04-04T06:07:09.418708+0000","last_unstale":"2024-04-04T06:07:10.418638+0000","last_undegraded":"2024-04-04T06:07:10.418638+0000","last_fullsized":"2024-04-04T06:07:10.418638+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:08:01.796146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.419142+0000","last_change":"2024-04-04T06:07:08.389249+0000","last_active":"2024-04-04T06:07:10.419142+0000","last_peered":"2024-04-04T06:07:10.419142+0000","last_clean":"2024-04-04T06:07:10.419142+0000","last_became_active":"2024-04-04T06:07:08.387652+0000","last_became_peered":"2024-04-04T06:07:08.387652+0000","last_unstale":"2024-04-04T06:07:10.419142+0000","last_undegraded":"2024-04-04T06:07:10.419142+0000","last_fullsized":"2024-04-04T06:07:10.419142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:08:14.750770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.420454+0000","last_change":"2024-04-04T06:07:09.408046+0000","last_active":"2024-04-04T06:07:10.420454+0000","last_peered":"2024-04-04T06:07:10.420454+0000","last_clean":"2024-04-04T06:07:10.420454+0000","last_became_active":"2024-04-04T06:07:09.407728+0000","last_became_peered":"2024-04-04T06:07:09.407728+0000","last_unstale":"2024-04-04T06:07:10.420454+0000","last_undegraded":"2024-04-04T06:07:10.420454+0000","last_fullsized":"2024-04-04T06:07:10.420454+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:31:23.568304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418566+0000","last_change":"2024-04-04T06:07:08.392144+0000","last_active":"2024-04-04T06:07:10.418566+0000","last_peered":"2024-04-04T06:07:10.418566+0000","last_clean":"2024-04-04T06:07:10.418566+0000","last_became_active":"2024-04-04T06:07:08.391854+0000","last_became_peered":"2024-04-04T06:07:08.391854+0000","last_unstale":"2024-04-04T06:07:10.418566+0000","last_undegraded":"2024-04-04T06:07:10.418566+0000","last_fullsized":"2024-04-04T06:07:10.418566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:23:43.557850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417616+0000","last_change":"2024-04-04T06:07:09.425499+0000","last_active":"2024-04-04T06:07:10.417616+0000","last_peered":"2024-04-04T06:07:10.417616+0000","last_clean":"2024-04-04T06:07:10.417616+0000","last_became_active":"2024-04-04T06:07:09.425221+0000","last_became_peered":"2024-04-04T06:07:09.425221+0000","last_unstale":"2024-04-04T06:07:10.417616+0000","last_undegraded":"2024-04-04T06:07:10.417616+0000","last_fullsized":"2024-04-04T06:07:10.417616+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:49:38.280388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418697+0000","last_change":"2024-04-04T06:07:08.384551+0000","last_active":"2024-04-04T06:07:10.418697+0000","last_peered":"2024-04-04T06:07:10.418697+0000","last_clean":"2024-04-04T06:07:10.418697+0000","last_became_active":"2024-04-04T06:07:08.383977+0000","last_became_peered":"2024-04-04T06:07:08.383977+0000","last_unstale":"2024-04-04T06:07:10.418697+0000","last_undegraded":"2024-04-04T06:07:10.418697+0000","last_fullsized":"2024-04-04T06:07:10.418697+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:51:41.223975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.127586+0000","last_change":"2024-04-04T06:07:09.427057+0000","last_active":"2024-04-04T06:07:11.127586+0000","last_peered":"2024-04-04T06:07:11.127586+0000","last_clean":"2024-04-04T06:07:11.127586+0000","last_became_active":"2024-04-04T06:07:09.426800+0000","last_became_peered":"2024-04-04T06:07:09.426800+0000","last_unstale":"2024-04-04T06:07:11.127586+0000","last_undegraded":"2024-04-04T06:07:11.127586+0000","last_fullsized":"2024-04-04T06:07:11.127586+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:08:15.764014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418519+0000","last_change":"2024-04-04T06:07:08.385778+0000","last_active":"2024-04-04T06:07:10.418519+0000","last_peered":"2024-04-04T06:07:10.418519+0000","last_clean":"2024-04-04T06:07:10.418519+0000","last_became_active":"2024-04-04T06:07:08.385169+0000","last_became_peered":"2024-04-04T06:07:08.385169+0000","last_unstale":"2024-04-04T06:07:10.418519+0000","last_undegraded":"2024-04-04T06:07:10.418519+0000","last_fullsized":"2024-04-04T06:07:10.418519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:17:01.423214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126543+0000","last_change":"2024-04-04T06:07:09.424114+0000","last_active":"2024-04-04T06:07:11.126543+0000","last_peered":"2024-04-04T06:07:11.126543+0000","last_clean":"2024-04-04T06:07:11.126543+0000","last_became_active":"2024-04-04T06:07:09.423801+0000","last_became_peered":"2024-04-04T06:07:09.423801+0000","last_unstale":"2024-04-04T06:07:11.126543+0000","last_undegraded":"2024-04-04T06:07:11.126543+0000","last_fullsized":"2024-04-04T06:07:11.126543+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:46:14.937588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.423960+0000","last_change":"2024-04-04T06:07:08.385377+0000","last_active":"2024-04-04T06:07:10.423960+0000","last_peered":"2024-04-04T06:07:10.423960+0000","last_clean":"2024-04-04T06:07:10.423960+0000","last_became_active":"2024-04-04T06:07:08.385120+0000","last_became_peered":"2024-04-04T06:07:08.385120+0000","last_unstale":"2024-04-04T06:07:10.423960+0000","last_undegraded":"2024-04-04T06:07:10.423960+0000","last_fullsized":"2024-04-04T06:07:10.423960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:08:14.750770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.127301+0000","last_change":"2024-04-04T06:07:09.420552+0000","last_active":"2024-04-04T06:07:11.127301+0000","last_peered":"2024-04-04T06:07:11.127301+0000","last_clean":"2024-04-04T06:07:11.127301+0000","last_became_active":"2024-04-04T06:07:09.420098+0000","last_became_peered":"2024-04-04T06:07:09.420098+0000","last_unstale":"2024-04-04T06:07:11.127301+0000","last_undegraded":"2024-04-04T06:07:11.127301+0000","last_fullsized":"2024-04-04T06:07:11.127301+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:17:02.436458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126024+0000","last_change":"2024-04-04T06:07:08.392816+0000","last_active":"2024-04-04T06:07:11.126024+0000","last_peered":"2024-04-04T06:07:11.126024+0000","last_clean":"2024-04-04T06:07:11.126024+0000","last_became_active":"2024-04-04T06:07:08.391134+0000","last_became_peered":"2024-04-04T06:07:08.391134+0000","last_unstale":"2024-04-04T06:07:11.126024+0000","last_undegraded":"2024-04-04T06:07:11.126024+0000","last_fullsized":"2024-04-04T06:07:11.126024+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:15:04.696399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126821+0000","last_change":"2024-04-04T06:07:09.423215+0000","last_active":"2024-04-04T06:07:11.126821+0000","last_peered":"2024-04-04T06:07:11.126821+0000","last_clean":"2024-04-04T06:07:11.126821+0000","last_became_active":"2024-04-04T06:07:09.421307+0000","last_became_peered":"2024-04-04T06:07:09.421307+0000","last_unstale":"2024-04-04T06:07:11.126821+0000","last_undegraded":"2024-04-04T06:07:11.126821+0000","last_fullsized":"2024-04-04T06:07:11.126821+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:51:42.237220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.419553+0000","last_change":"2024-04-04T06:07:08.388908+0000","last_active":"2024-04-04T06:07:10.419553+0000","last_peered":"2024-04-04T06:07:10.419553+0000","last_clean":"2024-04-04T06:07:10.419553+0000","last_became_active":"2024-04-04T06:07:08.388561+0000","last_became_peered":"2024-04-04T06:07:08.388561+0000","last_unstale":"2024-04-04T06:07:10.419553+0000","last_undegraded":"2024-04-04T06:07:10.419553+0000","last_fullsized":"2024-04-04T06:07:10.419553+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:46:13.924344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.667011+0000","last_change":"2024-04-04T06:07:09.432471+0000","last_active":"2024-04-04T06:07:10.667011+0000","last_peered":"2024-04-04T06:07:10.667011+0000","last_clean":"2024-04-04T06:07:10.667011+0000","last_became_active":"2024-04-04T06:07:09.432191+0000","last_became_peered":"2024-04-04T06:07:09.432191+0000","last_unstale":"2024-04-04T06:07:10.667011+0000","last_undegraded":"2024-04-04T06:07:10.667011+0000","last_fullsized":"2024-04-04T06:07:10.667011+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:56:30.405307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.418178+0000","last_change":"2024-04-04T06:07:08.394203+0000","last_active":"2024-04-04T06:07:10.418178+0000","last_peered":"2024-04-04T06:07:10.418178+0000","last_clean":"2024-04-04T06:07:10.418178+0000","last_became_active":"2024-04-04T06:07:08.393913+0000","last_became_peered":"2024-04-04T06:07:08.393913+0000","last_unstale":"2024-04-04T06:07:10.418178+0000","last_undegraded":"2024-04-04T06:07:10.418178+0000","last_fullsized":"2024-04-04T06:07:10.418178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:11:10.659591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.127523+0000","last_change":"2024-04-04T06:07:09.417965+0000","last_active":"2024-04-04T06:07:11.127523+0000","last_peered":"2024-04-04T06:07:11.127523+0000","last_clean":"2024-04-04T06:07:11.127523+0000","last_became_active":"2024-04-04T06:07:09.417641+0000","last_became_peered":"2024-04-04T06:07:09.417641+0000","last_unstale":"2024-04-04T06:07:11.127523+0000","last_undegraded":"2024-04-04T06:07:11.127523+0000","last_fullsized":"2024-04-04T06:07:11.127523+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:23:44.571094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.419117+0000","last_change":"2024-04-04T06:07:08.389862+0000","last_active":"2024-04-04T06:07:10.419117+0000","last_peered":"2024-04-04T06:07:10.419117+0000","last_clean":"2024-04-04T06:07:10.419117+0000","last_became_active":"2024-04-04T06:07:08.389574+0000","last_became_peered":"2024-04-04T06:07:08.389574+0000","last_unstale":"2024-04-04T06:07:10.419117+0000","last_undegraded":"2024-04-04T06:07:10.419117+0000","last_fullsized":"2024-04-04T06:07:10.419117+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:29:14.442681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.420421+0000","last_change":"2024-04-04T06:07:09.428824+0000","last_active":"2024-04-04T06:07:10.420421+0000","last_peered":"2024-04-04T06:07:10.420421+0000","last_clean":"2024-04-04T06:07:10.420421+0000","last_became_active":"2024-04-04T06:07:09.428511+0000","last_became_peered":"2024-04-04T06:07:09.428511+0000","last_unstale":"2024-04-04T06:07:10.420421+0000","last_undegraded":"2024-04-04T06:07:10.420421+0000","last_fullsized":"2024-04-04T06:07:10.420421+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:53:59.963734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.427561+0000","last_change":"2024-04-04T06:07:08.388555+0000","last_active":"2024-04-04T06:07:10.427561+0000","last_peered":"2024-04-04T06:07:10.427561+0000","last_clean":"2024-04-04T06:07:10.427561+0000","last_became_active":"2024-04-04T06:07:08.388280+0000","last_became_peered":"2024-04-04T06:07:08.388280+0000","last_unstale":"2024-04-04T06:07:10.427561+0000","last_undegraded":"2024-04-04T06:07:10.427561+0000","last_fullsized":"2024-04-04T06:07:10.427561+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:17:01.423214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.424287+0000","last_change":"2024-04-04T06:07:09.426956+0000","last_active":"2024-04-04T06:07:10.424287+0000","last_peered":"2024-04-04T06:07:10.424287+0000","last_clean":"2024-04-04T06:07:10.424287+0000","last_became_active":"2024-04-04T06:07:09.426638+0000","last_became_peered":"2024-04-04T06:07:09.426638+0000","last_unstale":"2024-04-04T06:07:10.424287+0000","last_undegraded":"2024-04-04T06:07:10.424287+0000","last_fullsized":"2024-04-04T06:07:10.424287+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:04:03.680120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417712+0000","last_change":"2024-04-04T06:07:08.382978+0000","last_active":"2024-04-04T06:07:10.417712+0000","last_peered":"2024-04-04T06:07:10.417712+0000","last_clean":"2024-04-04T06:07:10.417712+0000","last_became_active":"2024-04-04T06:07:08.382623+0000","last_became_peered":"2024-04-04T06:07:08.382623+0000","last_unstale":"2024-04-04T06:07:10.417712+0000","last_undegraded":"2024-04-04T06:07:10.417712+0000","last_fullsized":"2024-04-04T06:07:10.417712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:53:42.102358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.125585+0000","last_change":"2024-04-04T06:07:09.423847+0000","last_active":"2024-04-04T06:07:11.125585+0000","last_peered":"2024-04-04T06:07:11.125585+0000","last_clean":"2024-04-04T06:07:11.125585+0000","last_became_active":"2024-04-04T06:07:09.423575+0000","last_became_peered":"2024-04-04T06:07:09.423575+0000","last_unstale":"2024-04-04T06:07:11.125585+0000","last_undegraded":"2024-04-04T06:07:11.125585+0000","last_fullsized":"2024-04-04T06:07:11.125585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:24:27.600006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.424124+0000","last_change":"2024-04-04T06:07:08.388610+0000","last_active":"2024-04-04T06:07:10.424124+0000","last_peered":"2024-04-04T06:07:10.424124+0000","last_clean":"2024-04-04T06:07:10.424124+0000","last_became_active":"2024-04-04T06:07:08.388297+0000","last_became_peered":"2024-04-04T06:07:08.388297+0000","last_unstale":"2024-04-04T06:07:10.424124+0000","last_undegraded":"2024-04-04T06:07:10.424124+0000","last_fullsized":"2024-04-04T06:07:10.424124+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:29:14.442681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.423816+0000","last_change":"2024-04-04T06:07:09.428804+0000","last_active":"2024-04-04T06:07:10.423816+0000","last_peered":"2024-04-04T06:07:10.423816+0000","last_clean":"2024-04-04T06:07:10.423816+0000","last_became_active":"2024-04-04T06:07:09.428547+0000","last_became_peered":"2024-04-04T06:07:09.428547+0000","last_unstale":"2024-04-04T06:07:10.423816+0000","last_undegraded":"2024-04-04T06:07:10.423816+0000","last_fullsized":"2024-04-04T06:07:10.423816+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:55:04.159661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.417081+0000","last_change":"2024-04-04T06:07:08.388072+0000","last_active":"2024-04-04T06:07:10.417081+0000","last_peered":"2024-04-04T06:07:10.417081+0000","last_clean":"2024-04-04T06:07:10.417081+0000","last_became_active":"2024-04-04T06:07:08.387790+0000","last_became_peered":"2024-04-04T06:07:08.387790+0000","last_unstale":"2024-04-04T06:07:10.417081+0000","last_undegraded":"2024-04-04T06:07:10.417081+0000","last_fullsized":"2024-04-04T06:07:10.417081+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:23:43.557850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.126913+0000","last_change":"2024-04-04T06:07:09.420955+0000","last_active":"2024-04-04T06:07:11.126913+0000","last_peered":"2024-04-04T06:07:11.126913+0000","last_clean":"2024-04-04T06:07:11.126913+0000","last_became_active":"2024-04-04T06:07:09.420687+0000","last_became_peered":"2024-04-04T06:07:09.420687+0000","last_unstale":"2024-04-04T06:07:11.126913+0000","last_undegraded":"2024-04-04T06:07:11.126913+0000","last_fullsized":"2024-04-04T06:07:11.126913+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:16:00.835478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.419615+0000","last_change":"2024-04-04T06:07:08.390168+0000","last_active":"2024-04-04T06:07:10.419615+0000","last_peered":"2024-04-04T06:07:10.419615+0000","last_clean":"2024-04-04T06:07:10.419615+0000","last_became_active":"2024-04-04T06:07:08.389813+0000","last_became_peered":"2024-04-04T06:07:08.389813+0000","last_unstale":"2024-04-04T06:07:10.419615+0000","last_undegraded":"2024-04-04T06:07:10.419615+0000","last_fullsized":"2024-04-04T06:07:10.419615+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:05:48.164519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.419502+0000","last_change":"2024-04-04T06:07:09.421402+0000","last_active":"2024-04-04T06:07:10.419502+0000","last_peered":"2024-04-04T06:07:10.419502+0000","last_clean":"2024-04-04T06:07:10.419502+0000","last_became_active":"2024-04-04T06:07:09.421115+0000","last_became_peered":"2024-04-04T06:07:09.421115+0000","last_unstale":"2024-04-04T06:07:10.419502+0000","last_undegraded":"2024-04-04T06:07:10.419502+0000","last_fullsized":"2024-04-04T06:07:10.419502+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:51:20.551855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.127155+0000","last_change":"2024-04-04T06:07:08.395148+0000","last_active":"2024-04-04T06:07:11.127155+0000","last_peered":"2024-04-04T06:07:11.127155+0000","last_clean":"2024-04-04T06:07:11.127155+0000","last_became_active":"2024-04-04T06:07:08.394882+0000","last_became_peered":"2024-04-04T06:07:08.394882+0000","last_unstale":"2024-04-04T06:07:11.127155+0000","last_undegraded":"2024-04-04T06:07:11.127155+0000","last_fullsized":"2024-04-04T06:07:11.127155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:11:10.659591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.667335+0000","last_change":"2024-04-04T06:07:09.429518+0000","last_active":"2024-04-04T06:07:10.667335+0000","last_peered":"2024-04-04T06:07:10.667335+0000","last_clean":"2024-04-04T06:07:10.667335+0000","last_became_active":"2024-04-04T06:07:09.428398+0000","last_became_peered":"2024-04-04T06:07:09.428398+0000","last_unstale":"2024-04-04T06:07:10.667335+0000","last_undegraded":"2024-04-04T06:07:10.667335+0000","last_fullsized":"2024-04-04T06:07:10.667335+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:08.350982+0000","last_clean_scrub_stamp":"2024-04-04T06:07:08.350982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:08:18.706750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:11.125733+0000","last_change":"2024-04-04T06:07:08.397414+0000","last_active":"2024-04-04T06:07:11.125733+0000","last_peered":"2024-04-04T06:07:11.125733+0000","last_clean":"2024-04-04T06:07:11.125733+0000","last_became_active":"2024-04-04T06:07:08.397084+0000","last_became_peered":"2024-04-04T06:07:08.397084+0000","last_unstale":"2024-04-04T06:07:11.125733+0000","last_undegraded":"2024-04-04T06:07:11.125733+0000","last_fullsized":"2024-04-04T06:07:11.125733+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:07.337738+0000","last_clean_scrub_stamp":"2024-04-04T06:07:07.337738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:18:48.789573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":192,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-04T06:07:10.666453+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:07:10.666453+0000","last_peered":"2024-04-04T06:07:10.666453+0000","last_clean":"2024-04-04T06:07:10.666453+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:07:10.666453+0000","last_undegraded":"2024-04-04T06:07:10.666453+0000","last_fullsized":"2024-04-04T06:07:10.666453+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509460,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":974848,"data_stored":738947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":372736,"data_stored":142805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":150123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":148559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":150123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":738857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542166,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":385024,"data_stored":149161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542164,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":372736,"data_stored":142293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:07:14.372 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-04T06:07:14.372 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-04T06:07:14.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.641+0000 7f4af5a00640 1 Processor -- start 2024-04-04T06:07:14.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.642+0000 7f4af5a00640 1 -- start start 2024-04-04T06:07:14.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.643+0000 7f4af5a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4af015e440 0x7f4af00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:14.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.643+0000 7f4af5a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4af00cf0f0 0x7f4af00b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:14.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.643+0000 7f4af5a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4af015e820 0x7f4af00b6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:14.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.643+0000 7f4af5a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f4af00b7580 con 0x7f4af015e820 2024-04-04T06:07:14.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.643+0000 7f4af5a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4af00cf670 con 0x7f4af00cf0f0 2024-04-04T06:07:14.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.643+0000 7f4af5a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4af00cf950 con 0x7f4af015e440 2024-04-04T06:07:14.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.644+0000 7f4af5000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4af015e820 0x7f4af00b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:14.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.644+0000 7f4af5000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4af015e820 0x7f4af00b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53206/0 (socket says 172.21.15.159:53206) 2024-04-04T06:07:14.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.644+0000 7f4af5000640 1 -- 172.21.15.159:0/1183710722 learned_addr learned my addr 172.21.15.159:0/1183710722 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:14.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.644+0000 7f4aef400640 1 --2- 172.21.15.159:0/1183710722 >> v2:172.21.15.178:3300/0 conn(0x7f4af00cf0f0 0x7f4af00b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:14.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.644+0000 7f4aefe00640 1 --2- 172.21.15.159:0/1183710722 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4af015e440 0x7f4af00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:14.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.644+0000 7f4af5000640 1 -- 172.21.15.159:0/1183710722 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4af015e440 msgr2=0x7f4af00cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:14.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.644+0000 7f4af5000640 1 --2- 172.21.15.159:0/1183710722 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4af015e440 0x7f4af00cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.644+0000 7f4af5000640 1 -- 172.21.15.159:0/1183710722 >> v2:172.21.15.178:3300/0 conn(0x7f4af00cf0f0 msgr2=0x7f4af00b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:14.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.644+0000 7f4af5000640 1 --2- 172.21.15.159:0/1183710722 >> v2:172.21.15.178:3300/0 conn(0x7f4af00cf0f0 0x7f4af00b5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.644+0000 7f4af5000640 1 -- 172.21.15.159:0/1183710722 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af00d2670 con 0x7f4af015e820 2024-04-04T06:07:14.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.644+0000 7f4aef400640 1 --2- 172.21.15.159:0/1183710722 >> v2:172.21.15.178:3300/0 conn(0x7f4af00cf0f0 0x7f4af00b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:14.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.645+0000 7f4af5000640 1 --2- 172.21.15.159:0/1183710722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4af015e820 0x7f4af00b6310 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4af00b6f40 tx=0x7f4ae802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913968765b0f3643 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:14.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.646+0000 7f4aeea00640 1 -- 172.21.15.159:0/1183710722 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae802ca70 con 0x7f4af015e820 2024-04-04T06:07:14.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.646+0000 7f4aeea00640 1 -- 172.21.15.159:0/1183710722 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ae802cc10 con 0x7f4af015e820 2024-04-04T06:07:14.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.646+0000 7f4aeea00640 1 -- 172.21.15.159:0/1183710722 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae80343f0 con 0x7f4af015e820 2024-04-04T06:07:14.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.647+0000 7f4af5a00640 1 -- 172.21.15.159:0/1183710722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4af015e820 msgr2=0x7f4af00b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:14.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.647+0000 7f4af5a00640 1 --2- 172.21.15.159:0/1183710722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4af015e820 0x7f4af00b6310 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4af00b6f40 tx=0x7f4ae802bed0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.647+0000 7f4af5a00640 1 -- 172.21.15.159:0/1183710722 shutdown_connections 2024-04-04T06:07:14.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.647+0000 7f4af5a00640 1 --2- 172.21.15.159:0/1183710722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4af015e820 0x7f4af00b6310 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.647+0000 7f4af5a00640 1 --2- 172.21.15.159:0/1183710722 >> v2:172.21.15.178:3300/0 conn(0x7f4af00cf0f0 0x7f4af00b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.647+0000 7f4af5a00640 1 --2- 172.21.15.159:0/1183710722 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4af015e440 0x7f4af00cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.647+0000 7f4af5a00640 1 -- 172.21.15.159:0/1183710722 >> 172.21.15.159:0/1183710722 conn(0x7f4af00676a0 msgr2=0x7f4af0067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:14.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.647+0000 7f4af5a00640 1 -- 172.21.15.159:0/1183710722 shutdown_connections 2024-04-04T06:07:14.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.647+0000 7f4af5a00640 1 -- 172.21.15.159:0/1183710722 wait complete. 2024-04-04T06:07:14.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.649+0000 7f4af5a00640 1 Processor -- start 2024-04-04T06:07:14.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.649+0000 7f4af5a00640 1 -- start start 2024-04-04T06:07:14.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.651+0000 7f4af5a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4af00cf0f0 0x7f4af01f52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:14.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.651+0000 7f4af5a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4af015e440 0x7f4af01f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:14.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.651+0000 7f4af5a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4af015e820 0x7f4af01f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:14.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.651+0000 7f4af5a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4af0162560 con 0x7f4af015e440 2024-04-04T06:07:14.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.651+0000 7f4af5a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4af01623e0 con 0x7f4af015e820 2024-04-04T06:07:14.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.651+0000 7f4af5a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4af006c610 con 0x7f4af00cf0f0 2024-04-04T06:07:14.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.651+0000 7f4aef400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4af015e440 0x7f4af01f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:14.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.651+0000 7f4aef400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4af015e440 0x7f4af01f3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53208/0 (socket says 172.21.15.159:53208) 2024-04-04T06:07:14.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.651+0000 7f4aef400640 1 -- 172.21.15.159:0/366933805 learned_addr learned my addr 172.21.15.159:0/366933805 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:14.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.651+0000 7f4aefe00640 1 --2- 172.21.15.159:0/366933805 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4af00cf0f0 0x7f4af01f52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:14.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.651+0000 7f4af5000640 1 --2- 172.21.15.159:0/366933805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4af015e820 0x7f4af01f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:14.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.651+0000 7f4aef400640 1 -- 172.21.15.159:0/366933805 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4af00cf0f0 msgr2=0x7f4af01f52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:14.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.651+0000 7f4aef400640 1 --2- 172.21.15.159:0/366933805 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4af00cf0f0 0x7f4af01f52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.651+0000 7f4aef400640 1 -- 172.21.15.159:0/366933805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4af015e820 msgr2=0x7f4af01f4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:14.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.651+0000 7f4aef400640 1 --2- 172.21.15.159:0/366933805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4af015e820 0x7f4af01f4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.651+0000 7f4aef400640 1 -- 172.21.15.159:0/366933805 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af01f5070 con 0x7f4af015e440 2024-04-04T06:07:14.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.651+0000 7f4af5000640 1 --2- 172.21.15.159:0/366933805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4af015e820 0x7f4af01f4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:14.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.652+0000 7f4aef400640 1 --2- 172.21.15.159:0/366933805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4af015e440 0x7f4af01f3a60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800c7f0 tx=0x7f4ad800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:14.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.652+0000 7f4ae7e00640 1 -- 172.21.15.159:0/366933805 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8015070 con 0x7f4af015e440 2024-04-04T06:07:14.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.652+0000 7f4ae7e00640 1 -- 172.21.15.159:0/366933805 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ad8007a30 con 0x7f4af015e440 2024-04-04T06:07:14.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.652+0000 7f4ae7e00640 1 -- 172.21.15.159:0/366933805 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8007e00 con 0x7f4af015e440 2024-04-04T06:07:14.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.652+0000 7f4af5a00640 1 -- 172.21.15.159:0/366933805 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af00c80e0 con 0x7f4af015e440 2024-04-04T06:07:14.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.652+0000 7f4af5a00640 1 -- 172.21.15.159:0/366933805 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af00c86a0 con 0x7f4af015e440 2024-04-04T06:07:14.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.653+0000 7f4af5a00640 1 -- 172.21.15.159:0/366933805 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af0075940 con 0x7f4af015e440 2024-04-04T06:07:14.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.654+0000 7f4ae7e00640 1 -- 172.21.15.159:0/366933805 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ad8018070 con 0x7f4af015e440 2024-04-04T06:07:14.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.655+0000 7f4ae7e00640 1 --2- 172.21.15.159:0/366933805 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4acc06c8f0 0x7f4acc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:14.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.655+0000 7f4ae7e00640 1 -- 172.21.15.159:0/366933805 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f4ad808bad0 con 0x7f4af015e440 2024-04-04T06:07:14.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.656+0000 7f4aefe00640 1 --2- 172.21.15.159:0/366933805 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4acc06c8f0 0x7f4acc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:14.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.657+0000 7f4aefe00640 1 --2- 172.21.15.159:0/366933805 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4acc06c8f0 0x7f4acc06edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0006fd0 tx=0x7f4ae0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:14.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.660+0000 7f4ae7e00640 1 -- 172.21.15.159:0/366933805 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4ad8059ef0 con 0x7f4af015e440 2024-04-04T06:07:14.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.933+0000 7f4af5a00640 1 -- 172.21.15.159:0/366933805 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f4af00d1c90 con 0x7f4af015e440 2024-04-04T06:07:14.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.937+0000 7f4ae7e00640 1 -- 172.21.15.159:0/366933805 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f4ad80597f0 con 0x7f4af015e440 2024-04-04T06:07:14.932 INFO:teuthology.orchestra.run.smithi159.stdout:[client.0] 2024-04-04T06:07:14.932 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQD9Qg5mqNKJGxAA4jP9cadhRIyhQ3lssttWIw== 2024-04-04T06:07:14.932 INFO:teuthology.orchestra.run.smithi159.stdout: caps mds = "allow" 2024-04-04T06:07:14.932 INFO:teuthology.orchestra.run.smithi159.stdout: caps mon = "allow r" 2024-04-04T06:07:14.932 INFO:teuthology.orchestra.run.smithi159.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-04T06:07:14.932 INFO:teuthology.orchestra.run.smithi159.stderr:updated caps for client.0 2024-04-04T06:07:14.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.940+0000 7f4af5a00640 1 -- 172.21.15.159:0/366933805 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4acc06c8f0 msgr2=0x7f4acc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:14.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.940+0000 7f4af5a00640 1 --2- 172.21.15.159:0/366933805 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4acc06c8f0 0x7f4acc06edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0006fd0 tx=0x7f4ae0008040 comp rx=0 tx=0).stop 2024-04-04T06:07:14.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.940+0000 7f4af5a00640 1 -- 172.21.15.159:0/366933805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4af015e440 msgr2=0x7f4af01f3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:14.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.940+0000 7f4af5a00640 1 --2- 172.21.15.159:0/366933805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4af015e440 0x7f4af01f3a60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800c7f0 tx=0x7f4ad800cbf0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.940+0000 7f4af5a00640 1 -- 172.21.15.159:0/366933805 shutdown_connections 2024-04-04T06:07:14.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.940+0000 7f4af5a00640 1 --2- 172.21.15.159:0/366933805 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4acc06c8f0 0x7f4acc06edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.940+0000 7f4af5a00640 1 --2- 172.21.15.159:0/366933805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4af015e820 0x7f4af01f4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.940+0000 7f4af5a00640 1 --2- 172.21.15.159:0/366933805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4af015e440 0x7f4af01f3a60 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.940+0000 7f4af5a00640 1 --2- 172.21.15.159:0/366933805 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4af00cf0f0 0x7f4af01f52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:14.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.940+0000 7f4af5a00640 1 -- 172.21.15.159:0/366933805 >> 172.21.15.159:0/366933805 conn(0x7f4af00676a0 msgr2=0x7f4af0067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:14.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.940+0000 7f4af5a00640 1 -- 172.21.15.159:0/366933805 shutdown_connections 2024-04-04T06:07:14.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:14.940+0000 7f4af5a00640 1 -- 172.21.15.159:0/366933805 wait complete. 2024-04-04T06:07:14.955 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:07:15.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.166+0000 7f43d7200640 1 Processor -- start 2024-04-04T06:07:15.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.166+0000 7f43d7200640 1 -- start start 2024-04-04T06:07:15.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.168+0000 7f43d7200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43d815e4a0 0x7f43d80b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:15.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.168+0000 7f43d7200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f43d80cf700 0x7f43d80b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:15.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.168+0000 7f43d7200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d815e880 con 0x7f43d80d2310 2024-04-04T06:07:15.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.168+0000 7f43d7200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f43d80d34e0 con 0x7f43d80cf700 2024-04-04T06:07:15.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.168+0000 7f43d7200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f43d80d28d0 con 0x7f43d815e4a0 2024-04-04T06:07:15.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.168+0000 7f43d6800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f43d80cf700 0x7f43d80b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:15.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.168+0000 7f43d6800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f43d80cf700 0x7f43d80b73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53214/0 (socket says 172.21.15.159:53214) 2024-04-04T06:07:15.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.168+0000 7f43d6800640 1 -- 172.21.15.159:0/3430550569 learned_addr learned my addr 172.21.15.159:0/3430550569 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:15.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.169+0000 7f43d5e00640 1 --2- 172.21.15.159:0/3430550569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43d815e4a0 0x7f43d80b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:15.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.169+0000 7f43d6800640 1 -- 172.21.15.159:0/3430550569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43d815e4a0 msgr2=0x7f43d80b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:15.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.169+0000 7f43d6800640 1 --2- 172.21.15.159:0/3430550569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43d815e4a0 0x7f43d80b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:15.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.169+0000 7f43d6800640 1 -- 172.21.15.159:0/3430550569 >> v1:172.21.15.178:6789/0 conn(0x7f43d80d2310 legacy=0x7f43d80ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:15.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.169+0000 7f43d6800640 1 -- 172.21.15.159:0/3430550569 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d80d2bb0 con 0x7f43d80cf700 2024-04-04T06:07:15.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.170+0000 7f43d6800640 1 --2- 172.21.15.159:0/3430550569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43d80cf700 0x7f43d80b73e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f43d80d3660 tx=0x7f43d002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d7d5d41acf9f3c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:15.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.170+0000 7f43d5400640 1 -- 172.21.15.159:0/3430550569 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d002caf0 con 0x7f43d80cf700 2024-04-04T06:07:15.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.170+0000 7f43d5400640 1 -- 172.21.15.159:0/3430550569 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43d002cc90 con 0x7f43d80cf700 2024-04-04T06:07:15.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.170+0000 7f43d5400640 1 -- 172.21.15.159:0/3430550569 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d0034610 con 0x7f43d80cf700 2024-04-04T06:07:15.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.171+0000 7f43d7200640 1 -- 172.21.15.159:0/3430550569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43d80cf700 msgr2=0x7f43d80b73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:15.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.171+0000 7f43d7200640 1 --2- 172.21.15.159:0/3430550569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43d80cf700 0x7f43d80b73e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f43d80d3660 tx=0x7f43d002bf50 comp rx=0 tx=0).stop 2024-04-04T06:07:15.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.171+0000 7f43d7200640 1 -- 172.21.15.159:0/3430550569 shutdown_connections 2024-04-04T06:07:15.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.171+0000 7f43d7200640 1 --2- 172.21.15.159:0/3430550569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43d80cf700 0x7f43d80b73e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:15.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.171+0000 7f43d7200640 1 --2- 172.21.15.159:0/3430550569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43d815e4a0 0x7f43d80b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:15.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.171+0000 7f43d7200640 1 -- 172.21.15.159:0/3430550569 >> 172.21.15.159:0/3430550569 conn(0x7f43d80676d0 msgr2=0x7f43d8067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:15.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.171+0000 7f43d7200640 1 -- 172.21.15.159:0/3430550569 shutdown_connections 2024-04-04T06:07:15.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.171+0000 7f43d7200640 1 -- 172.21.15.159:0/3430550569 wait complete. 2024-04-04T06:07:15.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.173+0000 7f43d7200640 1 Processor -- start 2024-04-04T06:07:15.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.173+0000 7f43d7200640 1 -- start start 2024-04-04T06:07:15.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.174+0000 7f43d7200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43d80cf700 0x7f43d81f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:15.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.174+0000 7f43d7200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43d80d2310 0x7f43d80ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:15.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.174+0000 7f43d7200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43d815e4a0 0x7f43d81613a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:15.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.174+0000 7f43d7200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f43d80d3970 con 0x7f43d80cf700 2024-04-04T06:07:15.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.174+0000 7f43d7200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43d80d3170 con 0x7f43d815e4a0 2024-04-04T06:07:15.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.174+0000 7f43d7200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f43d80ba1a0 con 0x7f43d80d2310 2024-04-04T06:07:15.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.175+0000 7f43d5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43d80cf700 0x7f43d81f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:15.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.175+0000 7f43d5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43d80cf700 0x7f43d81f3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53216/0 (socket says 172.21.15.159:53216) 2024-04-04T06:07:15.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.175+0000 7f43d5e00640 1 -- 172.21.15.159:0/3187700950 learned_addr learned my addr 172.21.15.159:0/3187700950 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:15.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.175+0000 7f43d6800640 1 --2- 172.21.15.159:0/3187700950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43d815e4a0 0x7f43d81613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:15.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.175+0000 7f43d5e00640 1 -- 172.21.15.159:0/3187700950 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43d80d2310 msgr2=0x7f43d80ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:15.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.175+0000 7f43d5e00640 1 --2- 172.21.15.159:0/3187700950 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43d80d2310 0x7f43d80ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:15.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.175+0000 7f43d5e00640 1 -- 172.21.15.159:0/3187700950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43d815e4a0 msgr2=0x7f43d81613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:15.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.175+0000 7f43d5e00640 1 --2- 172.21.15.159:0/3187700950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43d815e4a0 0x7f43d81613a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:15.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.175+0000 7f43d5e00640 1 -- 172.21.15.159:0/3187700950 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d8160060 con 0x7f43d80cf700 2024-04-04T06:07:15.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.176+0000 7f43d5e00640 1 --2- 172.21.15.159:0/3187700950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43d80cf700 0x7f43d81f3ea0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f43c800e930 tx=0x7f43c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:15.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.176+0000 7f43cea00640 1 -- 172.21.15.159:0/3187700950 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8017070 con 0x7f43d80cf700 2024-04-04T06:07:15.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.176+0000 7f43cea00640 1 -- 172.21.15.159:0/3187700950 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c800ca00 con 0x7f43d80cf700 2024-04-04T06:07:15.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.176+0000 7f43d7200640 1 -- 172.21.15.159:0/3187700950 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d80cd320 con 0x7f43d80cf700 2024-04-04T06:07:15.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.176+0000 7f43cea00640 1 -- 172.21.15.159:0/3187700950 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c800cdf0 con 0x7f43d80cf700 2024-04-04T06:07:15.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.176+0000 7f43d7200640 1 -- 172.21.15.159:0/3187700950 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d81605e0 con 0x7f43d80cf700 2024-04-04T06:07:15.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.177+0000 7f43d7200640 1 -- 172.21.15.159:0/3187700950 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d8075940 con 0x7f43d80cf700 2024-04-04T06:07:15.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.178+0000 7f43cea00640 1 -- 172.21.15.159:0/3187700950 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f43c801a070 con 0x7f43d80cf700 2024-04-04T06:07:15.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.179+0000 7f43cea00640 1 --2- 172.21.15.159:0/3187700950 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43b406c920 0x7f43b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:15.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.180+0000 7f43cea00640 1 -- 172.21.15.159:0/3187700950 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f43c808bd00 con 0x7f43d80cf700 2024-04-04T06:07:15.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.180+0000 7f43cd400640 1 --2- 172.21.15.159:0/3187700950 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43b406c920 0x7f43b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:15.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.181+0000 7f43cd400640 1 --2- 172.21.15.159:0/3187700950 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43b406c920 0x7f43b406ede0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f43c0000f60 tx=0x7f43c00079b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:15.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.184+0000 7f43cea00640 1 -- 172.21.15.159:0/3187700950 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f43c805a0c0 con 0x7f43d80cf700 2024-04-04T06:07:15.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.420+0000 7f43d7200640 1 -- 172.21.15.159:0/3187700950 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43d8067e10 con 0x7f43d80cf700 2024-04-04T06:07:15.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.421+0000 7f43cea00640 1 -- 172.21.15.159:0/3187700950 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5193 (secure 0 0 0) 0x7f43c80599c0 con 0x7f43d80cf700 2024-04-04T06:07:15.415 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:15.416 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1643905381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1643905381},{"type":"v1","addr":"172.21.15.178:6837","nonce":1643905381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4744,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2629539137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2629539137},{"type":"v1","addr":"172.21.15.159:6835","nonce":2629539137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3402172592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3402172592},{"type":"v1","addr":"172.21.15.159:6833","nonce":3402172592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:07:08.839645+0000","modified":"2024-04-04T06:07:11.927601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5157},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5157":{"gid":5157,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.178:6836/3282963883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3282963883},{"type":"v1","addr":"172.21.15.178:6836","nonce":3282963883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5157,"qdb_cluster":[5157]},"id":2}]} 2024-04-04T06:07:15.416 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 17 2024-04-04T06:07:15.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.423+0000 7f43d7200640 1 -- 172.21.15.159:0/3187700950 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43b406c920 msgr2=0x7f43b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:15.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.424+0000 7f43d7200640 1 --2- 172.21.15.159:0/3187700950 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43b406c920 0x7f43b406ede0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f43c0000f60 tx=0x7f43c00079b0 comp rx=0 tx=0).stop 2024-04-04T06:07:15.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.424+0000 7f43d7200640 1 -- 172.21.15.159:0/3187700950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43d80cf700 msgr2=0x7f43d81f3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.424+0000 7f43d7200640 1 --2- 172.21.15.159:0/3187700950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43d80cf700 0x7f43d81f3ea0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f43c800e930 tx=0x7f43c800edf0 comp rx=0 tx=0).stop 2024-04-04T06:07:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.424+0000 7f43d7200640 1 -- 172.21.15.159:0/3187700950 shutdown_connections 2024-04-04T06:07:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.424+0000 7f43d7200640 1 --2- 172.21.15.159:0/3187700950 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43b406c920 0x7f43b406ede0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.424+0000 7f43d7200640 1 --2- 172.21.15.159:0/3187700950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43d815e4a0 0x7f43d81613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.424+0000 7f43d7200640 1 --2- 172.21.15.159:0/3187700950 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43d80d2310 0x7f43d80ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.424+0000 7f43d7200640 1 --2- 172.21.15.159:0/3187700950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43d80cf700 0x7f43d81f3ea0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.424+0000 7f43d7200640 1 -- 172.21.15.159:0/3187700950 >> 172.21.15.159:0/3187700950 conn(0x7f43d80676d0 msgr2=0x7f43d80ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.424+0000 7f43d7200640 1 -- 172.21.15.159:0/3187700950 shutdown_connections 2024-04-04T06:07:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:15.424+0000 7f43d7200640 1 -- 172.21.15.159:0/3187700950 wait complete. 2024-04-04T06:07:15.839 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:07:08.839645+0000', 'modified': '2024-04-04T06:07:11.927601+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5157}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5157': {'gid': 5157, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.178:6836/3282963883', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6834', 'nonce': 3282963883}, {'type': 'v1', 'addr': '172.21.15.178:6836', 'nonce': 3282963883}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5157, 'qdb_cluster': [5157]} 2024-04-04T06:07:15.839 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-04T06:07:15.839 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-04T06:07:16.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.063+0000 7f5d99800640 1 Processor -- start 2024-04-04T06:07:16.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.064+0000 7f5d99800640 1 -- start start 2024-04-04T06:07:16.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.065+0000 7f5d99800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5d94155930 0x7f5d940d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.065+0000 7f5d99800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d940b40b0 con 0x7f5d940d07c0 2024-04-04T06:07:16.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.065+0000 7f5d99800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d940b4390 con 0x7f5d94155550 2024-04-04T06:07:16.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.065+0000 7f5d99800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f5d940d0c10 con 0x7f5d94155930 2024-04-04T06:07:16.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.065+0000 7f5d8a000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5d94155930 0x7f5d940d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:16.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.065+0000 7f5d93400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f5d940d07c0 0x7f5d940aede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:49198/0 (socket says 172.21.15.159:49198) 2024-04-04T06:07:16.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.065+0000 7f5d93400640 1 -- 172.21.15.159:0/3396581470 learned_addr learned my addr 172.21.15.159:0/3396581470 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:16.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.066+0000 7f5d8a000640 1 -- 172.21.15.159:0/3396581470 >> v1:172.21.15.178:6789/0 conn(0x7f5d94155550 legacy=0x7f5d9415f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:16.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.066+0000 7f5d8a000640 1 -- 172.21.15.159:0/3396581470 >> v1:172.21.15.159:6789/0 conn(0x7f5d940d07c0 legacy=0x7f5d940aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:16.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.066+0000 7f5d8a000640 1 -- 172.21.15.159:0/3396581470 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d940b4670 con 0x7f5d94155930 2024-04-04T06:07:16.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.066+0000 7f5d92000640 1 -- 172.21.15.159:0/3396581470 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4008899950 0 0) 0x7f5d940b40b0 con 0x7f5d940d07c0 2024-04-04T06:07:16.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.067+0000 7f5d8a000640 1 --2- 172.21.15.159:0/3396581470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d94155930 0x7f5d940d0280 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0097a0 tx=0x7f5d7c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d7bb315527dfdf server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:16.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.067+0000 7f5d92000640 1 -- 172.21.15.159:0/3396581470 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0315a0 con 0x7f5d94155930 2024-04-04T06:07:16.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.067+0000 7f5d92000640 1 -- 172.21.15.159:0/3396581470 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d7c037070 con 0x7f5d94155930 2024-04-04T06:07:16.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.067+0000 7f5d92000640 1 -- 172.21.15.159:0/3396581470 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c031960 con 0x7f5d94155930 2024-04-04T06:07:16.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.068+0000 7f5d99800640 1 -- 172.21.15.159:0/3396581470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d94155930 msgr2=0x7f5d940d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:16.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.068+0000 7f5d99800640 1 --2- 172.21.15.159:0/3396581470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d94155930 0x7f5d940d0280 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0097a0 tx=0x7f5d7c02b840 comp rx=0 tx=0).stop 2024-04-04T06:07:16.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.068+0000 7f5d99800640 1 -- 172.21.15.159:0/3396581470 shutdown_connections 2024-04-04T06:07:16.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.068+0000 7f5d99800640 1 --2- 172.21.15.159:0/3396581470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d94155930 0x7f5d940d0280 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.068+0000 7f5d99800640 1 -- 172.21.15.159:0/3396581470 >> 172.21.15.159:0/3396581470 conn(0x7f5d9406b750 msgr2=0x7f5d9406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:16.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.068+0000 7f5d99800640 1 -- 172.21.15.159:0/3396581470 shutdown_connections 2024-04-04T06:07:16.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.068+0000 7f5d99800640 1 -- 172.21.15.159:0/3396581470 wait complete. 2024-04-04T06:07:16.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.070+0000 7f5d99800640 1 Processor -- start 2024-04-04T06:07:16.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.070+0000 7f5d99800640 1 -- start start 2024-04-04T06:07:16.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.071+0000 7f5d99800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d940d07c0 0x7f5d9414f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.071+0000 7f5d99800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d94155550 0x7f5d9414d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.071+0000 7f5d99800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d94155930 0x7f5d9414c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.071+0000 7f5d99800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9415ba20 con 0x7f5d94155930 2024-04-04T06:07:16.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.071+0000 7f5d99800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5d940b4c30 con 0x7f5d94155550 2024-04-04T06:07:16.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.071+0000 7f5d99800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5d9415bba0 con 0x7f5d940d07c0 2024-04-04T06:07:16.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.071+0000 7f5d93400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d94155930 0x7f5d9414c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:16.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.071+0000 7f5d93400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d94155930 0x7f5d9414c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53222/0 (socket says 172.21.15.159:53222) 2024-04-04T06:07:16.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.071+0000 7f5d93400640 1 -- 172.21.15.159:0/1480413041 learned_addr learned my addr 172.21.15.159:0/1480413041 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:16.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.071+0000 7f5d8a000640 1 --2- 172.21.15.159:0/1480413041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d94155550 0x7f5d9414d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:16.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.072+0000 7f5d93400640 1 -- 172.21.15.159:0/1480413041 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d940d07c0 msgr2=0x7f5d9414f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:16.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.072+0000 7f5d93400640 1 --2- 172.21.15.159:0/1480413041 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d940d07c0 0x7f5d9414f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.072+0000 7f5d93400640 1 -- 172.21.15.159:0/1480413041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d94155550 msgr2=0x7f5d9414d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:16.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.072+0000 7f5d93400640 1 --2- 172.21.15.159:0/1480413041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d94155550 0x7f5d9414d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.072+0000 7f5d93400640 1 -- 172.21.15.159:0/1480413041 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d94147d00 con 0x7f5d94155930 2024-04-04T06:07:16.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.072+0000 7f5d8a000640 1 --2- 172.21.15.159:0/1480413041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d94155550 0x7f5d9414d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:16.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.072+0000 7f5d93400640 1 --2- 172.21.15.159:0/1480413041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d94155930 0x7f5d9414c0d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00c700 tx=0x7f5d8c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:16.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.072+0000 7f5d8b400640 1 -- 172.21.15.159:0/1480413041 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c018070 con 0x7f5d94155930 2024-04-04T06:07:16.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.072+0000 7f5d8b400640 1 -- 172.21.15.159:0/1480413041 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d8c00ce10 con 0x7f5d94155930 2024-04-04T06:07:16.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.072+0000 7f5d8b400640 1 -- 172.21.15.159:0/1480413041 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c00a880 con 0x7f5d94155930 2024-04-04T06:07:16.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.072+0000 7f5d99800640 1 -- 172.21.15.159:0/1480413041 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d94147fe0 con 0x7f5d94155930 2024-04-04T06:07:16.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.072+0000 7f5d99800640 1 -- 172.21.15.159:0/1480413041 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d941485a0 con 0x7f5d94155930 2024-04-04T06:07:16.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.075+0000 7f5d8b400640 1 -- 172.21.15.159:0/1480413041 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d8c00aa20 con 0x7f5d94155930 2024-04-04T06:07:16.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.076+0000 7f5d8b400640 1 --2- 172.21.15.159:0/1480413041 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d7006c8f0 0x7f5d7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.077+0000 7f5d8b400640 1 -- 172.21.15.159:0/1480413041 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f5d8c08b1f0 con 0x7f5d94155930 2024-04-04T06:07:16.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.077+0000 7f5d9c432740 1 Processor -- start 2024-04-04T06:07:16.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.077+0000 7f5d9c432740 1 -- start start 2024-04-04T06:07:16.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.077+0000 7f5d92a00640 1 --2- 172.21.15.159:0/1480413041 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d7006c8f0 0x7f5d7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:16.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.078+0000 7f5d9c432740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f791540 0x55bd8f7644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.078+0000 7f5d9c432740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd8f791920 0x55bd8f7942d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.078+0000 7f5d9c432740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bd8f764a20 0x55bd8f794810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.078+0000 7f5d9c432740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bd8f727ab0 con 0x55bd8f791540 2024-04-04T06:07:16.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.078+0000 7f5d9c432740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bd8f6c9fa0 con 0x55bd8f791920 2024-04-04T06:07:16.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.078+0000 7f5d9c432740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bd8f672560 con 0x55bd8f764a20 2024-04-04T06:07:16.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.078+0000 7f5d92a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f791540 0x55bd8f7644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:16.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.078+0000 7f5d92a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f791540 0x55bd8f7644e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53238/0 (socket says 172.21.15.159:53238) 2024-04-04T06:07:16.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.078+0000 7f5d92a00640 1 -- 172.21.15.159:0/2806449018 learned_addr learned my addr 172.21.15.159:0/2806449018 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:16.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.078+0000 7f5d93400640 1 --2- 172.21.15.159:0/2806449018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bd8f764a20 0x55bd8f794810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:16.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.078+0000 7f5d8a000640 1 --2- 172.21.15.159:0/2806449018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd8f791920 0x55bd8f7942d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:16.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.078+0000 7f5d92a00640 1 -- 172.21.15.159:0/2806449018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bd8f764a20 msgr2=0x55bd8f794810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:16.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.078+0000 7f5d92a00640 1 --2- 172.21.15.159:0/2806449018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bd8f764a20 0x55bd8f794810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.079+0000 7f5d92a00640 1 -- 172.21.15.159:0/2806449018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd8f791920 msgr2=0x55bd8f7942d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:16.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.079+0000 7f5d92a00640 1 --2- 172.21.15.159:0/2806449018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd8f791920 0x55bd8f7942d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.079+0000 7f5d92a00640 1 -- 172.21.15.159:0/2806449018 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd8f3de2d0 con 0x55bd8f791540 2024-04-04T06:07:16.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.079+0000 7f5d92a00640 1 --2- 172.21.15.159:0/1480413041 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d7006c8f0 0x7f5d7006edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5d941f5b00 tx=0x7f5d84004120 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:16.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.079+0000 7f5d92a00640 1 --2- 172.21.15.159:0/2806449018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f791540 0x55bd8f7644e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400c8d0 tx=0x7f5d84017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc8fc6d294572d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:16.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.079+0000 7f5d83400640 1 -- 172.21.15.159:0/2806449018 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84024070 con 0x55bd8f791540 2024-04-04T06:07:16.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.079+0000 7f5d83400640 1 -- 172.21.15.159:0/2806449018 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d84017720 con 0x55bd8f791540 2024-04-04T06:07:16.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.079+0000 7f5d83400640 1 -- 172.21.15.159:0/2806449018 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84010040 con 0x55bd8f791540 2024-04-04T06:07:16.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.080+0000 7f5d9c432740 1 -- 172.21.15.159:0/2806449018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f791540 msgr2=0x55bd8f7644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:16.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.080+0000 7f5d9c432740 1 --2- 172.21.15.159:0/2806449018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f791540 0x55bd8f7644e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400c8d0 tx=0x7f5d84017000 comp rx=0 tx=0).stop 2024-04-04T06:07:16.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.080+0000 7f5d9c432740 1 -- 172.21.15.159:0/2806449018 shutdown_connections 2024-04-04T06:07:16.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.080+0000 7f5d9c432740 1 --2- 172.21.15.159:0/2806449018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bd8f764a20 0x55bd8f794810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.080+0000 7f5d9c432740 1 --2- 172.21.15.159:0/2806449018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd8f791920 0x55bd8f7942d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.080+0000 7f5d9c432740 1 --2- 172.21.15.159:0/2806449018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f791540 0x55bd8f7644e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.080+0000 7f5d9c432740 1 -- 172.21.15.159:0/2806449018 >> 172.21.15.159:0/2806449018 conn(0x55bd8f766750 msgr2=0x55bd8f7af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:16.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.080+0000 7f5d9c432740 1 -- 172.21.15.159:0/2806449018 shutdown_connections 2024-04-04T06:07:16.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.080+0000 7f5d9c432740 1 -- 172.21.15.159:0/2806449018 wait complete. 2024-04-04T06:07:16.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.081+0000 7f5d9c432740 1 -- start start 2024-04-04T06:07:16.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.081+0000 7f5d9c432740 1 Processor -- start 2024-04-04T06:07:16.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.082+0000 7f5d9c432740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f791540 0x55bd8f7892f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.082+0000 7f5d9c432740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bd8f791920 0x55bd8f7c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.082+0000 7f5d9c432740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd8f7c3410 0x55bd8f796f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.082+0000 7f5d9c432740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bd8f685920 con 0x55bd8f791540 2024-04-04T06:07:16.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.082+0000 7f5d9c432740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bd8f6c5f50 con 0x55bd8f7c3410 2024-04-04T06:07:16.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.082+0000 7f5d9c432740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bd8f768cf0 con 0x55bd8f791920 2024-04-04T06:07:16.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.082+0000 7f5d9c432740 10 client.0 _ll_register_callbacks cb 0x55bd8f685310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:07:16.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.082+0000 7f5d92a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f791540 0x55bd8f7892f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:16.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.082+0000 7f5d92a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f791540 0x55bd8f7892f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53244/0 (socket says 172.21.15.159:53244) 2024-04-04T06:07:16.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.082+0000 7f5d92a00640 1 -- 172.21.15.159:0/3011976257 learned_addr learned my addr 172.21.15.159:0/3011976257 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:16.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.082+0000 7f5d8a000640 1 --2- 172.21.15.159:0/3011976257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bd8f791920 0x55bd8f7c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:16.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.082+0000 7f5d92a00640 1 -- 172.21.15.159:0/3011976257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bd8f791920 msgr2=0x55bd8f7c2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:16.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.082+0000 7f5d92a00640 1 --2- 172.21.15.159:0/3011976257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bd8f791920 0x55bd8f7c2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.082+0000 7f5d92a00640 1 -- 172.21.15.159:0/3011976257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd8f7c3410 msgr2=0x55bd8f796f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:16.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.082+0000 7f5d93400640 1 --2- 172.21.15.159:0/3011976257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd8f7c3410 0x55bd8f796f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:16.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.082+0000 7f5d92a00640 1 --2- 172.21.15.159:0/3011976257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd8f7c3410 0x55bd8f796f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.082+0000 7f5d92a00640 1 -- 172.21.15.159:0/3011976257 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd8f4a86b0 con 0x55bd8f791540 2024-04-04T06:07:16.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.082+0000 7f5d8a000640 1 --2- 172.21.15.159:0/3011976257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bd8f791920 0x55bd8f7c2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:16.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.083+0000 7f5d92a00640 1 --2- 172.21.15.159:0/3011976257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f791540 0x55bd8f7892f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5d84007d20 tx=0x7f5d840037d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7905683d3fe95ea9 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:16.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.083+0000 7f5d77e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:07:16.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.083+0000 7f5d77e00640 1 -- 172.21.15.159:0/3011976257 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8402e020 con 0x55bd8f791540 2024-04-04T06:07:16.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.083+0000 7f5d77e00640 1 -- 172.21.15.159:0/3011976257 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f5d84007240 con 0x55bd8f791540 2024-04-04T06:07:16.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.083+0000 7f5d9c432740 1 -- 172.21.15.159:0/3011976257 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d84017720 con 0x55bd8f791540 2024-04-04T06:07:16.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.083+0000 7f5d77e00640 1 -- 172.21.15.159:0/3011976257 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f5d84038050 con 0x55bd8f791540 2024-04-04T06:07:16.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.083+0000 7f5d9c432740 10 client.5319 fetch_fsmap learned FSMap version 17 2024-04-04T06:07:16.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.083+0000 7f5d9c432740 1 -- 172.21.15.159:0/3011976257 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55bd8f796710 con 0x55bd8f791540 2024-04-04T06:07:16.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.083+0000 7f5d77e00640 1 -- 172.21.15.159:0/3011976257 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d84017720 con 0x55bd8f791540 2024-04-04T06:07:16.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.084+0000 7f5d77e00640 1 -- 172.21.15.159:0/3011976257 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f5d8403b020 con 0x55bd8f791540 2024-04-04T06:07:16.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.084+0000 7f5d9c432740 10 client.5319 fetch_fsmap finished waiting for FSMap version 17 2024-04-04T06:07:16.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.084+0000 7f5d9c432740 10 client.5319 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-04T06:07:16.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.084+0000 7f5d9c432740 20 client.5319 populate_metadata read hostname 'smithi159' 2024-04-04T06:07:16.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.084+0000 7f5d9c432740 1 --2- 172.21.15.159:0/3011976257 >> [v2:172.21.15.178:6834/3282963883,v1:172.21.15.178:6836/3282963883] conn(0x55bd8f797aa0 0x55bd8f7cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.084+0000 7f5d9c432740 4 client.5319 mds_command: new command op to 5157 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:07:16.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.084+0000 7f5d9c432740 1 -- 172.21.15.159:0/3011976257 --> [v2:172.21.15.178:6834/3282963883,v1:172.21.15.178:6836/3282963883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd8f6d74e0 con 0x55bd8f797aa0 2024-04-04T06:07:16.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.084+0000 7f5d8a000640 1 --2- 172.21.15.159:0/3011976257 >> [v2:172.21.15.178:6834/3282963883,v1:172.21.15.178:6836/3282963883] conn(0x55bd8f797aa0 0x55bd8f7cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:16.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.085+0000 7f5d8a000640 1 --2- 172.21.15.159:0/3011976257 >> [v2:172.21.15.178:6834/3282963883,v1:172.21.15.178:6836/3282963883] conn(0x55bd8f797aa0 0x55bd8f7cd580 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3468b9013cf1f7fe server_cookie=7d759ae4f243be21 in_seq=0 out_seq=0 2024-04-04T06:07:16.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.085+0000 7f5d77e00640 10 client.5319 ms_handle_connect on v2:172.21.15.178:6834/3282963883 2024-04-04T06:07:16.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.087+0000 7f5d77e00640 1 -- 172.21.15.159:0/3011976257 <== mds.0 v2:172.21.15.178:6834/3282963883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd8f6d74e0 con 0x55bd8f797aa0 2024-04-04T06:07:16.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.087+0000 7f5d77e00640 10 client.5319 handle_command_reply: tid=0 2024-04-04T06:07:16.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.087+0000 7f5d9c432740 1 client.5319 shutdown 2024-04-04T06:07:16.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.087+0000 7f5d9c432740 1 -- 172.21.15.159:0/3011976257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f791540 msgr2=0x55bd8f7892f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:16.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.087+0000 7f5d9c432740 1 --2- 172.21.15.159:0/3011976257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f791540 0x55bd8f7892f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5d84007d20 tx=0x7f5d840037d0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.087+0000 7f5d9c432740 1 -- 172.21.15.159:0/3011976257 shutdown_connections 2024-04-04T06:07:16.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.087+0000 7f5d9c432740 1 --2- 172.21.15.159:0/3011976257 >> [v2:172.21.15.178:6834/3282963883,v1:172.21.15.178:6836/3282963883] conn(0x55bd8f797aa0 0x55bd8f7cd580 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.087+0000 7f5d9c432740 1 --2- 172.21.15.159:0/3011976257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd8f7c3410 0x55bd8f796f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.087+0000 7f5d9c432740 1 --2- 172.21.15.159:0/3011976257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bd8f791920 0x55bd8f7c2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.087+0000 7f5d9c432740 1 --2- 172.21.15.159:0/3011976257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f791540 0x55bd8f7892f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.087+0000 7f5d77e00640 0 client.5319 ms_handle_reset on v2:172.21.15.178:6834/3282963883 2024-04-04T06:07:16.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.087+0000 7f5d9c432740 1 -- 172.21.15.159:0/3011976257 >> 172.21.15.159:0/3011976257 conn(0x55bd8f764a20 msgr2=0x55bd8f75eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:16.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.087+0000 7f5d9c432740 1 -- 172.21.15.159:0/3011976257 shutdown_connections 2024-04-04T06:07:16.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.087+0000 7f5d9c432740 1 -- 172.21.15.159:0/3011976257 wait complete. 2024-04-04T06:07:16.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.087+0000 7f5d9c432740 20 client.5319 trim_cache size 0 max 16384 2024-04-04T06:07:16.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.107+0000 7f5d9c432740 1 Processor -- start 2024-04-04T06:07:16.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.107+0000 7f5d9c432740 1 -- start start 2024-04-04T06:07:16.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.108+0000 7f5d9c432740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bd8f791540 0x55bd8f791e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.108+0000 7f5d9c432740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd8f791920 0x55bd8f7bf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.108+0000 7f5d9c432740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f7c3410 0x55bd8f7a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.108+0000 7f5d9c432740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5d94167190 con 0x55bd8f7c3410 2024-04-04T06:07:16.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.108+0000 7f5d9c432740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5d940b0700 con 0x55bd8f791920 2024-04-04T06:07:16.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.108+0000 7f5d9c432740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5d940995e0 con 0x55bd8f791540 2024-04-04T06:07:16.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.108+0000 7f5d93400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f7c3410 0x55bd8f7a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:16.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.108+0000 7f5d93400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f7c3410 0x55bd8f7a4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53256/0 (socket says 172.21.15.159:53256) 2024-04-04T06:07:16.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.109+0000 7f5d93400640 1 -- 172.21.15.159:0/3919239164 learned_addr learned my addr 172.21.15.159:0/3919239164 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:16.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.109+0000 7f5d93400640 1 -- 172.21.15.159:0/3919239164 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bd8f791540 msgr2=0x55bd8f791e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:16.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.109+0000 7f5d93400640 1 --2- 172.21.15.159:0/3919239164 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bd8f791540 0x55bd8f791e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.109+0000 7f5d93400640 1 -- 172.21.15.159:0/3919239164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd8f791920 msgr2=0x55bd8f7bf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:16.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.109+0000 7f5d93400640 1 --2- 172.21.15.159:0/3919239164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd8f791920 0x55bd8f7bf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.109+0000 7f5d93400640 1 -- 172.21.15.159:0/3919239164 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd8f7a8330 con 0x55bd8f7c3410 2024-04-04T06:07:16.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.109+0000 7f5d93400640 1 --2- 172.21.15.159:0/3919239164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f7c3410 0x55bd8f7a4c60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c089620 tx=0x7f5d8c0588e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf94e979b4880f09 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:16.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.109+0000 7f5d82a00640 1 -- 172.21.15.159:0/3919239164 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c071020 con 0x55bd8f7c3410 2024-04-04T06:07:16.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.109+0000 7f5d82a00640 1 -- 172.21.15.159:0/3919239164 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d8c05d230 con 0x55bd8f7c3410 2024-04-04T06:07:16.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.109+0000 7f5d82a00640 1 -- 172.21.15.159:0/3919239164 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c061a60 con 0x55bd8f7c3410 2024-04-04T06:07:16.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.110+0000 7f5d9c432740 1 -- 172.21.15.159:0/3919239164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f7c3410 msgr2=0x55bd8f7a4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:16.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.110+0000 7f5d9c432740 1 --2- 172.21.15.159:0/3919239164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f7c3410 0x55bd8f7a4c60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c089620 tx=0x7f5d8c0588e0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.110+0000 7f5d9c432740 1 -- 172.21.15.159:0/3919239164 shutdown_connections 2024-04-04T06:07:16.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.110+0000 7f5d9c432740 1 --2- 172.21.15.159:0/3919239164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f7c3410 0x55bd8f7a4c60 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.110+0000 7f5d9c432740 1 --2- 172.21.15.159:0/3919239164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd8f791920 0x55bd8f7bf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.110+0000 7f5d9c432740 1 --2- 172.21.15.159:0/3919239164 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bd8f791540 0x55bd8f791e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.110+0000 7f5d9c432740 1 -- 172.21.15.159:0/3919239164 >> 172.21.15.159:0/3919239164 conn(0x55bd8f764a20 msgr2=0x55bd8f789100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:16.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.110+0000 7f5d9c432740 1 -- 172.21.15.159:0/3919239164 shutdown_connections 2024-04-04T06:07:16.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.110+0000 7f5d9c432740 1 -- 172.21.15.159:0/3919239164 wait complete. 2024-04-04T06:07:16.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.111+0000 7f5d9c432740 1 -- start start 2024-04-04T06:07:16.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.111+0000 7f5d9c432740 1 Processor -- start 2024-04-04T06:07:16.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.112+0000 7f5d9c432740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd8f791540 0x55bd8f7a6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.112+0000 7f5d9c432740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f791920 0x55bd8f75f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.112+0000 7f5d9c432740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bd8f7c3410 0x55bd8f7b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.112+0000 7f5d9c432740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bd8f797920 con 0x55bd8f791920 2024-04-04T06:07:16.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.112+0000 7f5d9c432740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bd8f7969f0 con 0x55bd8f791540 2024-04-04T06:07:16.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.112+0000 7f5d9c432740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bd8f793d40 con 0x55bd8f7c3410 2024-04-04T06:07:16.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.112+0000 7f5d9c432740 10 client.0 _ll_register_callbacks cb 0x7f5d84017c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:07:16.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.112+0000 7f5d8a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f791920 0x55bd8f75f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:16.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.112+0000 7f5d8a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f791920 0x55bd8f75f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53264/0 (socket says 172.21.15.159:53264) 2024-04-04T06:07:16.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.112+0000 7f5d8a000640 1 -- 172.21.15.159:0/2017631410 learned_addr learned my addr 172.21.15.159:0/2017631410 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:16.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.112+0000 7f5d8a000640 1 -- 172.21.15.159:0/2017631410 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bd8f7c3410 msgr2=0x55bd8f7b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:16.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.112+0000 7f5d8a000640 1 --2- 172.21.15.159:0/2017631410 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bd8f7c3410 0x55bd8f7b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.112+0000 7f5d8a000640 1 -- 172.21.15.159:0/2017631410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd8f791540 msgr2=0x55bd8f7a6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:16.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.113+0000 7f5d8a000640 1 --2- 172.21.15.159:0/2017631410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd8f791540 0x55bd8f7a6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.113+0000 7f5d8a000640 1 -- 172.21.15.159:0/2017631410 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd8f75f5c0 con 0x55bd8f791920 2024-04-04T06:07:16.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.113+0000 7f5d8a000640 1 --2- 172.21.15.159:0/2017631410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f791920 0x55bd8f75f080 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c000c00 tx=0x7f5d7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e561e35f68d11f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:16.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.113+0000 7f5d77e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:07:16.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.113+0000 7f5d77e00640 1 -- 172.21.15.159:0/2017631410 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c048900 con 0x55bd8f791920 2024-04-04T06:07:16.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.113+0000 7f5d77e00640 1 -- 172.21.15.159:0/2017631410 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f5d7c0318f0 con 0x55bd8f791920 2024-04-04T06:07:16.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.113+0000 7f5d9c432740 1 -- 172.21.15.159:0/2017631410 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d8c05d230 con 0x55bd8f791920 2024-04-04T06:07:16.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.113+0000 7f5d77e00640 1 -- 172.21.15.159:0/2017631410 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f5d7c031b10 con 0x55bd8f791920 2024-04-04T06:07:16.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.113+0000 7f5d9c432740 10 client.5325 fetch_fsmap learned FSMap version 17 2024-04-04T06:07:16.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.113+0000 7f5d9c432740 1 -- 172.21.15.159:0/2017631410 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55bd8f7895b0 con 0x55bd8f791920 2024-04-04T06:07:16.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.114+0000 7f5d77e00640 1 -- 172.21.15.159:0/2017631410 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d8c05d230 con 0x55bd8f791920 2024-04-04T06:07:16.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.114+0000 7f5d77e00640 1 -- 172.21.15.159:0/2017631410 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f5d7c031070 con 0x55bd8f791920 2024-04-04T06:07:16.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.114+0000 7f5d9c432740 10 client.5325 fetch_fsmap finished waiting for FSMap version 17 2024-04-04T06:07:16.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.114+0000 7f5d9c432740 10 client.5325 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-04T06:07:16.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.114+0000 7f5d9c432740 20 client.5325 populate_metadata read hostname 'smithi159' 2024-04-04T06:07:16.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.114+0000 7f5d9c432740 1 --2- 172.21.15.159:0/2017631410 >> [v2:172.21.15.178:6834/3282963883,v1:172.21.15.178:6836/3282963883] conn(0x55bd8f78c0a0 0x55bd8f78c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.114+0000 7f5d9c432740 4 client.5325 mds_command: new command op to 5157 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-04T06:07:16.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.114+0000 7f5d9c432740 1 -- 172.21.15.159:0/2017631410 --> [v2:172.21.15.178:6834/3282963883,v1:172.21.15.178:6836/3282963883] -- command(tid 0: {"prefix": "status"}) -- 0x55bd8f6d1830 con 0x55bd8f78c0a0 2024-04-04T06:07:16.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.114+0000 7f5d92a00640 1 --2- 172.21.15.159:0/2017631410 >> [v2:172.21.15.178:6834/3282963883,v1:172.21.15.178:6836/3282963883] conn(0x55bd8f78c0a0 0x55bd8f78c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:16.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.115+0000 7f5d92a00640 1 --2- 172.21.15.159:0/2017631410 >> [v2:172.21.15.178:6834/3282963883,v1:172.21.15.178:6836/3282963883] conn(0x55bd8f78c0a0 0x55bd8f78c480 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98d215932dc71f70 server_cookie=70c93f7795ab4ed in_seq=0 out_seq=0 2024-04-04T06:07:16.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.115+0000 7f5d77e00640 10 client.5325 ms_handle_connect on v2:172.21.15.178:6834/3282963883 2024-04-04T06:07:16.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.116+0000 7f5d77e00640 1 -- 172.21.15.159:0/2017631410 <== mds.0 v2:172.21.15.178:6834/3282963883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55bd8f6d1830 con 0x55bd8f78c0a0 2024-04-04T06:07:16.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.116+0000 7f5d77e00640 10 client.5325 handle_command_reply: tid=0 2024-04-04T06:07:16.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.116+0000 7f5d9c432740 1 client.5325 shutdown 2024-04-04T06:07:16.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.116+0000 7f5d9c432740 1 -- 172.21.15.159:0/2017631410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f791920 msgr2=0x55bd8f75f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:16.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.116+0000 7f5d9c432740 1 --2- 172.21.15.159:0/2017631410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f791920 0x55bd8f75f080 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c000c00 tx=0x7f5d7c004730 comp rx=0 tx=0).stop 2024-04-04T06:07:16.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.116+0000 7f5d9c432740 1 -- 172.21.15.159:0/2017631410 shutdown_connections 2024-04-04T06:07:16.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.116+0000 7f5d9c432740 1 --2- 172.21.15.159:0/2017631410 >> [v2:172.21.15.178:6834/3282963883,v1:172.21.15.178:6836/3282963883] conn(0x55bd8f78c0a0 0x55bd8f78c480 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.116+0000 7f5d9c432740 1 --2- 172.21.15.159:0/2017631410 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bd8f7c3410 0x55bd8f7b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.116+0000 7f5d9c432740 1 --2- 172.21.15.159:0/2017631410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd8f791920 0x55bd8f75f080 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.116+0000 7f5d9c432740 1 --2- 172.21.15.159:0/2017631410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd8f791540 0x55bd8f7a6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.116+0000 7f5d77e00640 0 client.5325 ms_handle_reset on v2:172.21.15.178:6834/3282963883 2024-04-04T06:07:16.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.116+0000 7f5d9c432740 1 -- 172.21.15.159:0/2017631410 >> 172.21.15.159:0/2017631410 conn(0x55bd8f764a20 msgr2=0x55bd8f7bf150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:16.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.116+0000 7f5d9c432740 1 -- 172.21.15.159:0/2017631410 shutdown_connections 2024-04-04T06:07:16.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.116+0000 7f5d9c432740 1 -- 172.21.15.159:0/2017631410 wait complete. 2024-04-04T06:07:16.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.116+0000 7f5d9c432740 20 client.5325 trim_cache size 0 max 16384 2024-04-04T06:07:16.114 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:07:16.114 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_fsid": "30311045-f714-48fa-832b-4cb2ba8bf0bf", 2024-04-04T06:07:16.114 INFO:teuthology.orchestra.run.smithi159.stdout: "whoami": 0, 2024-04-04T06:07:16.114 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 5157, 2024-04-04T06:07:16.114 INFO:teuthology.orchestra.run.smithi159.stdout: "want_state": "up:active", 2024-04-04T06:07:16.114 INFO:teuthology.orchestra.run.smithi159.stdout: "state": "up:active", 2024-04-04T06:07:16.114 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs", 2024-04-04T06:07:16.114 INFO:teuthology.orchestra.run.smithi159.stdout: "rank_uptime": 6.7097343079999998, 2024-04-04T06:07:16.114 INFO:teuthology.orchestra.run.smithi159.stdout: "mdsmap_epoch": 17, 2024-04-04T06:07:16.114 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch": 34, 2024-04-04T06:07:16.114 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch_barrier": 34, 2024-04-04T06:07:16.114 INFO:teuthology.orchestra.run.smithi159.stdout: "uptime": 15.591853085 2024-04-04T06:07:16.114 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:07:16.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.117+0000 7f5d99800640 1 -- 172.21.15.159:0/1480413041 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d7006c8f0 msgr2=0x7f5d7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:16.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.117+0000 7f5d99800640 1 --2- 172.21.15.159:0/1480413041 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d7006c8f0 0x7f5d7006edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5d941f5b00 tx=0x7f5d84004120 comp rx=0 tx=0).stop 2024-04-04T06:07:16.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.117+0000 7f5d99800640 1 -- 172.21.15.159:0/1480413041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d94155930 msgr2=0x7f5d9414c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:16.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.117+0000 7f5d99800640 1 --2- 172.21.15.159:0/1480413041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d94155930 0x7f5d9414c0d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00c700 tx=0x7f5d8c00cbc0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.117+0000 7f5d99800640 1 -- 172.21.15.159:0/1480413041 shutdown_connections 2024-04-04T06:07:16.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.117+0000 7f5d99800640 1 --2- 172.21.15.159:0/1480413041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d94155930 0x7f5d9414c0d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.117+0000 7f5d99800640 1 --2- 172.21.15.159:0/1480413041 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d7006c8f0 0x7f5d7006edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.117+0000 7f5d99800640 1 --2- 172.21.15.159:0/1480413041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d94155550 0x7f5d9414d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.117+0000 7f5d99800640 1 --2- 172.21.15.159:0/1480413041 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d940d07c0 0x7f5d9414f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.117+0000 7f5d99800640 1 -- 172.21.15.159:0/1480413041 >> 172.21.15.159:0/1480413041 conn(0x7f5d9406b750 msgr2=0x7f5d9406e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:16.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.117+0000 7f5d99800640 1 -- 172.21.15.159:0/1480413041 shutdown_connections 2024-04-04T06:07:16.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.117+0000 7f5d99800640 1 -- 172.21.15.159:0/1480413041 wait complete. 2024-04-04T06:07:16.128 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-04T06:07:16.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.326+0000 7f5d74200640 1 Processor -- start 2024-04-04T06:07:16.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.326+0000 7f5d74200640 1 -- start start 2024-04-04T06:07:16.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.328+0000 7f5d74200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5d6c15b4e0 0x7f5d6c0b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.328+0000 7f5d74200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5d6c0afee0 0x7f5d6c15eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.328+0000 7f5d74200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d6c15b8c0 con 0x7f5d6c15f530 2024-04-04T06:07:16.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.328+0000 7f5d74200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f5d6c0b5810 con 0x7f5d6c15b4e0 2024-04-04T06:07:16.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.328+0000 7f5d74200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f5d6c0afab0 con 0x7f5d6c0afee0 2024-04-04T06:07:16.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.328+0000 7f5d71800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5d6c15b4e0 0x7f5d6c0b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:16.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.328+0000 7f5d71800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5d6c15b4e0 0x7f5d6c0b0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53266/0 (socket says 172.21.15.159:53266) 2024-04-04T06:07:16.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.328+0000 7f5d71800640 1 -- 172.21.15.159:0/2546644905 learned_addr learned my addr 172.21.15.159:0/2546644905 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:16.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.329+0000 7f5d70e00640 1 --2- 172.21.15.159:0/2546644905 >> v2:172.21.15.178:3300/0 conn(0x7f5d6c0afee0 0x7f5d6c15eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:16.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.329+0000 7f5d71800640 1 -- 172.21.15.159:0/2546644905 >> v2:172.21.15.178:3300/0 conn(0x7f5d6c0afee0 msgr2=0x7f5d6c15eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:16.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.329+0000 7f5d71800640 1 --2- 172.21.15.159:0/2546644905 >> v2:172.21.15.178:3300/0 conn(0x7f5d6c0afee0 0x7f5d6c15eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.329+0000 7f5d71800640 1 -- 172.21.15.159:0/2546644905 >> v1:172.21.15.178:6789/0 conn(0x7f5d6c15f530 legacy=0x7f5d6c0ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:16.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.329+0000 7f5d71800640 1 -- 172.21.15.159:0/2546644905 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6c15fd70 con 0x7f5d6c15b4e0 2024-04-04T06:07:16.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.330+0000 7f5d71800640 1 --2- 172.21.15.159:0/2546644905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d6c15b4e0 0x7f5d6c0b0570 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c070740 tx=0x7f5d6002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83573cc41b522eec server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:16.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.330+0000 7f5d6be00640 1 -- 172.21.15.159:0/2546644905 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6003b070 con 0x7f5d6c15b4e0 2024-04-04T06:07:16.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.330+0000 7f5d6be00640 1 -- 172.21.15.159:0/2546644905 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d60036440 con 0x7f5d6c15b4e0 2024-04-04T06:07:16.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.330+0000 7f5d6be00640 1 -- 172.21.15.159:0/2546644905 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60036810 con 0x7f5d6c15b4e0 2024-04-04T06:07:16.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.331+0000 7f5d74200640 1 -- 172.21.15.159:0/2546644905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d6c15b4e0 msgr2=0x7f5d6c0b0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:16.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.331+0000 7f5d74200640 1 --2- 172.21.15.159:0/2546644905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d6c15b4e0 0x7f5d6c0b0570 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c070740 tx=0x7f5d6002e0e0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.331+0000 7f5d74200640 1 -- 172.21.15.159:0/2546644905 shutdown_connections 2024-04-04T06:07:16.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.331+0000 7f5d74200640 1 --2- 172.21.15.159:0/2546644905 >> v2:172.21.15.178:3300/0 conn(0x7f5d6c0afee0 0x7f5d6c15eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.331+0000 7f5d74200640 1 --2- 172.21.15.159:0/2546644905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d6c15b4e0 0x7f5d6c0b0570 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.331+0000 7f5d74200640 1 -- 172.21.15.159:0/2546644905 >> 172.21.15.159:0/2546644905 conn(0x7f5d6c0b6560 msgr2=0x7f5d6c0b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:16.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.331+0000 7f5d74200640 1 -- 172.21.15.159:0/2546644905 shutdown_connections 2024-04-04T06:07:16.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.331+0000 7f5d74200640 1 -- 172.21.15.159:0/2546644905 wait complete. 2024-04-04T06:07:16.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.333+0000 7f5d74200640 1 Processor -- start 2024-04-04T06:07:16.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.333+0000 7f5d74200640 1 -- start start 2024-04-04T06:07:16.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.334+0000 7f5d74200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d6c0afee0 0x7f5d6c15ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.334+0000 7f5d74200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d6c15b4e0 0x7f5d6c15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.334+0000 7f5d74200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d6c15f530 0x7f5d6c15d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.334+0000 7f5d74200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c0b5010 con 0x7f5d6c0afee0 2024-04-04T06:07:16.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.334+0000 7f5d74200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c0b4d30 con 0x7f5d6c15b4e0 2024-04-04T06:07:16.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.334+0000 7f5d74200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5d6c0b52f0 con 0x7f5d6c15f530 2024-04-04T06:07:16.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.334+0000 7f5d71800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d6c0afee0 0x7f5d6c15ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:16.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.334+0000 7f5d71800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d6c0afee0 0x7f5d6c15ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53278/0 (socket says 172.21.15.159:53278) 2024-04-04T06:07:16.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.335+0000 7f5d71800640 1 -- 172.21.15.159:0/1962847748 learned_addr learned my addr 172.21.15.159:0/1962847748 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:16.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.335+0000 7f5d71800640 1 -- 172.21.15.159:0/1962847748 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d6c15f530 msgr2=0x7f5d6c15d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.335+0000 7f5d71800640 1 --2- 172.21.15.159:0/1962847748 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d6c15f530 0x7f5d6c15d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.335+0000 7f5d70e00640 1 --2- 172.21.15.159:0/1962847748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d6c15b4e0 0x7f5d6c15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.335+0000 7f5d71800640 1 -- 172.21.15.159:0/1962847748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d6c15b4e0 msgr2=0x7f5d6c15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.335+0000 7f5d71800640 1 --2- 172.21.15.159:0/1962847748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d6c15b4e0 0x7f5d6c15bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.335+0000 7f5d71800640 1 -- 172.21.15.159:0/1962847748 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6c15a200 con 0x7f5d6c0afee0 2024-04-04T06:07:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.335+0000 7f5d71800640 1 --2- 172.21.15.159:0/1962847748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d6c0afee0 0x7f5d6c15ba90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5d6002e120 tx=0x7f5d600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.335+0000 7f5d69600640 1 -- 172.21.15.159:0/1962847748 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60045030 con 0x7f5d6c0afee0 2024-04-04T06:07:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.336+0000 7f5d69600640 1 -- 172.21.15.159:0/1962847748 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d60034440 con 0x7f5d6c0afee0 2024-04-04T06:07:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.336+0000 7f5d69600640 1 -- 172.21.15.159:0/1962847748 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6003b040 con 0x7f5d6c0afee0 2024-04-04T06:07:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.336+0000 7f5d74200640 1 -- 172.21.15.159:0/1962847748 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6c159160 con 0x7f5d6c0afee0 2024-04-04T06:07:16.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.336+0000 7f5d74200640 1 -- 172.21.15.159:0/1962847748 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d6c159720 con 0x7f5d6c0afee0 2024-04-04T06:07:16.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.337+0000 7f5d74200640 1 -- 172.21.15.159:0/1962847748 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d6c0685c0 con 0x7f5d6c0afee0 2024-04-04T06:07:16.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.338+0000 7f5d69600640 1 -- 172.21.15.159:0/1962847748 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d60048070 con 0x7f5d6c0afee0 2024-04-04T06:07:16.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.339+0000 7f5d69600640 1 --2- 172.21.15.159:0/1962847748 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d5006c8f0 0x7f5d5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:16.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.339+0000 7f5d69600640 1 -- 172.21.15.159:0/1962847748 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f5d600b0270 con 0x7f5d6c0afee0 2024-04-04T06:07:16.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.340+0000 7f5d70e00640 1 --2- 172.21.15.159:0/1962847748 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d5006c8f0 0x7f5d5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:16.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.341+0000 7f5d70e00640 1 --2- 172.21.15.159:0/1962847748 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d5006c8f0 0x7f5d5006edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c15a720 tx=0x7f5d5c00b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:16.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.344+0000 7f5d69600640 1 -- 172.21.15.159:0/1962847748 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5d6004f070 con 0x7f5d6c0afee0 2024-04-04T06:07:16.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.588+0000 7f5d74200640 1 -- 172.21.15.159:0/1962847748 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5d6c0b1110 con 0x7f5d6c0afee0 2024-04-04T06:07:16.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.589+0000 7f5d69600640 1 -- 172.21.15.159:0/1962847748 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f5d6007e600 con 0x7f5d6c0afee0 2024-04-04T06:07:16.583 INFO:teuthology.orchestra.run.smithi159.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-04T06:07:16.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.591+0000 7f5d74200640 1 -- 172.21.15.159:0/1962847748 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d5006c8f0 msgr2=0x7f5d5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:16.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.591+0000 7f5d74200640 1 --2- 172.21.15.159:0/1962847748 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d5006c8f0 0x7f5d5006edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c15a720 tx=0x7f5d5c00b040 comp rx=0 tx=0).stop 2024-04-04T06:07:16.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.591+0000 7f5d74200640 1 -- 172.21.15.159:0/1962847748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d6c0afee0 msgr2=0x7f5d6c15ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:16.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.591+0000 7f5d74200640 1 --2- 172.21.15.159:0/1962847748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d6c0afee0 0x7f5d6c15ba90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5d6002e120 tx=0x7f5d600046c0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.592+0000 7f5d74200640 1 -- 172.21.15.159:0/1962847748 shutdown_connections 2024-04-04T06:07:16.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.592+0000 7f5d74200640 1 --2- 172.21.15.159:0/1962847748 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d6c15f530 0x7f5d6c15d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.592+0000 7f5d74200640 1 --2- 172.21.15.159:0/1962847748 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d5006c8f0 0x7f5d5006edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.592+0000 7f5d74200640 1 --2- 172.21.15.159:0/1962847748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d6c15b4e0 0x7f5d6c15bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.592+0000 7f5d74200640 1 --2- 172.21.15.159:0/1962847748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d6c0afee0 0x7f5d6c15ba90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:16.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.592+0000 7f5d74200640 1 -- 172.21.15.159:0/1962847748 >> 172.21.15.159:0/1962847748 conn(0x7f5d6c0b6560 msgr2=0x7f5d6c06be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:16.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.592+0000 7f5d74200640 1 -- 172.21.15.159:0/1962847748 shutdown_connections 2024-04-04T06:07:16.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16.592+0000 7f5d74200640 1 -- 172.21.15.159:0/1962847748 wait complete. 2024-04-04T06:07:16.604 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-04T06:07:16.604 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-04T06:07:16.604 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi159.front.sepia.ceph.com 2024-04-04T06:07:16.605 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-04T06:07:16.605 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-04T06:07:16.605 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-04T06:07:16.605 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-04T06:07:16.605 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-04T06:07:16.605 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:07:16.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:07:16.605 DEBUG:teuthology.orchestra.run.smithi159:> ip addr 2024-04-04T06:07:16.652 INFO:teuthology.orchestra.run.smithi159.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:07:16.652 INFO:teuthology.orchestra.run.smithi159.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:07:16.653 INFO:teuthology.orchestra.run.smithi159.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:07:16.653 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:07:16.653 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 ::1/128 scope host 2024-04-04T06:07:16.653 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:07:16.653 INFO:teuthology.orchestra.run.smithi159.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:07:16.653 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c6 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:07:16.653 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f0 2024-04-04T06:07:16.653 INFO:teuthology.orchestra.run.smithi159.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:07:16.653 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c7 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:07:16.653 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f1 2024-04-04T06:07:16.653 INFO:teuthology.orchestra.run.smithi159.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:07:16.653 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4e brd ff:ff:ff:ff:ff:ff 2024-04-04T06:07:16.653 INFO:teuthology.orchestra.run.smithi159.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-04T06:07:16.653 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:07:16.654 INFO:teuthology.orchestra.run.smithi159.stdout: inet 172.21.15.159/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-04T06:07:16.654 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft 42496sec preferred_lft 42496sec 2024-04-04T06:07:16.654 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::ec4:7aff:fe88:724f/64 scope link 2024-04-04T06:07:16.654 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:07:16.654 INFO:teuthology.orchestra.run.smithi159.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-04T06:07:16.654 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:07:16.654 INFO:teuthology.orchestra.run.smithi159.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-04T06:07:16.654 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:07:16.654 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::989e:e4ff:fea5:7568/64 scope link 2024-04-04T06:07:16.654 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:07:16.654 INFO:teuthology.orchestra.run.smithi159.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:07:16.654 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether a2:e3:e0:cb:0a:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-04T06:07:16.654 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::a0e3:e0ff:fecb:a73/64 scope link 2024-04-04T06:07:16.655 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:07:16.655 INFO:teuthology.orchestra.run.smithi159.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:07:16.655 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-04T06:07:16.655 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::c0a:8aff:fe54:b58f/64 scope link 2024-04-04T06:07:16.655 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:07:16.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:07:16.656 DEBUG:teuthology.orchestra.run.smithi159:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-04T06:07:16.739 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:07:16.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:07:16.743 DEBUG:teuthology.orchestra.run.smithi159:> route 2024-04-04T06:07:16.795 INFO:teuthology.orchestra.run.smithi159.stdout:Kernel IP routing table 2024-04-04T06:07:16.795 INFO:teuthology.orchestra.run.smithi159.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-04T06:07:16.795 INFO:teuthology.orchestra.run.smithi159.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-04T06:07:16.795 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-04T06:07:16.795 INFO:teuthology.orchestra.run.smithi159.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-04T06:07:16.796 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:07:16.796 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:07:16.796 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-04T06:07:16.796 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-04T06:07:16.796 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-04T06:07:16.796 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:07:16.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:07:17.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:07:17.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:07:17.040 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-04-04T06:07:17.091 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-04T06:07:17.091 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-04T06:07:17.092 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:07:17.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:07:17.092 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set brx.0 up 2024-04-04T06:07:17.185 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:07:17.185 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-04T06:07:17.239 INFO:teuthology.orchestra.run.smithi159.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:07:17.240 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:07:17.240 DEBUG:teuthology.orchestra.run.smithi159:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:07:17.291 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-04T06:07:17.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:07:17.292 DEBUG:teuthology.orchestra.run.smithi159:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:07:17.431 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:07:17.431 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:07:17.432 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:07:17.450 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:07:17.450 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:07:17.450 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:07:17.451 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:07:17.451 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:07:17.477 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:07:17.477 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:07:17.477 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:07:17.477 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:07:17.477 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:07:17.477 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-04T06:07:17.477 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-04T06:07:17.477 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30311045-f714-48fa-832b-4cb2ba8bf0bf" will pass to kernel 2024-04-04T06:07:17.479 INFO:teuthology.orchestra.run.smithi159.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-04T06:07:17.479 INFO:teuthology.orchestra.run.smithi159.stdout: You just mounted a file system that supports labels which does not 2024-04-04T06:07:17.479 INFO:teuthology.orchestra.run.smithi159.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-04T06:07:17.479 INFO:teuthology.orchestra.run.smithi159.stdout: applications will generate AVC messages and not be allowed access to 2024-04-04T06:07:17.479 INFO:teuthology.orchestra.run.smithi159.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-04T06:07:17.485 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-04T06:07:17.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:07:17.486 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:07:17.567 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:07:17.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:07:17.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:07:17.713 INFO:teuthology.orchestra.run.smithi159.stdout:client5334 2024-04-04T06:07:17.713 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:07:17.713 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:07:17.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:07:17.764 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-04-04T06:07:17.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:07:17.880 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-04-04T06:07:17.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:07:17.939 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-04-04T06:07:17.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:07:17.998 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-04-04T06:07:18.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:07:18.060 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-04-04T06:07:18.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:07:18.115 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-04T06:07:18.177 INFO:teuthology.orchestra.run.smithi159.stdout:./ 2024-04-04T06:07:18.177 INFO:teuthology.orchestra.run.smithi159.stdout:./rfile 2024-04-04T06:07:18.177 INFO:teuthology.orchestra.run.smithi159.stdout:./parentdir 2024-04-04T06:07:18.177 INFO:teuthology.orchestra.run.smithi159.stdout:./parentdir/pfile 2024-04-04T06:07:18.177 INFO:teuthology.orchestra.run.smithi159.stdout:./parentdir/childdir 2024-04-04T06:07:18.177 INFO:teuthology.orchestra.run.smithi159.stdout:./parentdir/childdir/cfile 2024-04-04T06:07:18.179 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:07:18.179 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:07:18.179 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:07:18.179 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:07:18.179 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-04T06:07:18.179 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:07:18.316 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:07:18.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:07:18.329 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:07:18.330 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:07:18.330 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:07:18.330 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:07:18.330 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-04-04T06:07:18.330 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:07:18.502 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627778 2024-04-04T06:07:18.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:07:18.516 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:07:18.516 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:07:18.516 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:07:18.516 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:07:18.516 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-04-04T06:07:18.516 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:07:18.702 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627776 2024-04-04T06:07:18.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:07:18.716 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:07:18.716 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:07:18.717 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:07:18.717 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:07:18.717 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-04-04T06:07:18.717 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:07:18.898 INFO:teuthology.orchestra.run.smithi159.stdout:1099511628279 2024-04-04T06:07:18.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:07:18.911 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:07:18.911 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:07:18.911 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:07:18.911 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:07:18.911 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-04-04T06:07:18.911 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:07:19.094 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627777 2024-04-04T06:07:19.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:07:19.107 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:07:19.107 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:07:19.107 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:07:19.107 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:07:19.107 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-04-04T06:07:19.107 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:07:19.292 INFO:teuthology.orchestra.run.smithi159.stdout:1099511628280 2024-04-04T06:07:19.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:07:19.306 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:07:19.306 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:07:19.403 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:07:19.403 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:07:19.456 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:07:19.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.676+0000 7f0354a00640 1 Processor -- start 2024-04-04T06:07:19.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.677+0000 7f0354a00640 1 -- start start 2024-04-04T06:07:19.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.678+0000 7f0354a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f035015fc50 0x7f03500cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:19.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.678+0000 7f0354a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f03500cc490 0x7f03500d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:19.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.678+0000 7f0354a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03500b3150 con 0x7f03500cc0b0 2024-04-04T06:07:19.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.678+0000 7f0354a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f035015c610 con 0x7f035015fc50 2024-04-04T06:07:19.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.678+0000 7f0354a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f03501600e0 con 0x7f03500cc490 2024-04-04T06:07:19.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.679+0000 7f034de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f035015fc50 0x7f03500cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:19.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.679+0000 7f034de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f035015fc50 0x7f03500cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53282/0 (socket says 172.21.15.159:53282) 2024-04-04T06:07:19.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.679+0000 7f034de00640 1 -- 172.21.15.159:0/3165117464 learned_addr learned my addr 172.21.15.159:0/3165117464 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:19.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.679+0000 7f034f200640 1 --2- 172.21.15.159:0/3165117464 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f03500cc490 0x7f03500d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:19.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.679+0000 7f034de00640 1 -- 172.21.15.159:0/3165117464 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f03500cc490 msgr2=0x7f03500d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:19.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.679+0000 7f034de00640 1 --2- 172.21.15.159:0/3165117464 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f03500cc490 0x7f03500d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:19.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.679+0000 7f034de00640 1 -- 172.21.15.159:0/3165117464 >> v1:172.21.15.178:6789/0 conn(0x7f03500cc0b0 legacy=0x7f035006dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:19.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.679+0000 7f034de00640 1 -- 172.21.15.159:0/3165117464 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03501603c0 con 0x7f035015fc50 2024-04-04T06:07:19.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.680+0000 7f034de00640 1 --2- 172.21.15.159:0/3165117464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f035015fc50 0x7f03500cfdc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f033c009e50 tx=0x7f033c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d40d064ff6248a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:19.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.681+0000 7f034d400640 1 -- 172.21.15.159:0/3165117464 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c02ca70 con 0x7f035015fc50 2024-04-04T06:07:19.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.681+0000 7f034d400640 1 -- 172.21.15.159:0/3165117464 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f033c02cc10 con 0x7f035015fc50 2024-04-04T06:07:19.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.681+0000 7f034d400640 1 -- 172.21.15.159:0/3165117464 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c0343f0 con 0x7f035015fc50 2024-04-04T06:07:19.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.681+0000 7f0354a00640 1 -- 172.21.15.159:0/3165117464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f035015fc50 msgr2=0x7f03500cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:19.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.681+0000 7f0354a00640 1 --2- 172.21.15.159:0/3165117464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f035015fc50 0x7f03500cfdc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f033c009e50 tx=0x7f033c02bed0 comp rx=0 tx=0).stop 2024-04-04T06:07:19.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.681+0000 7f0354a00640 1 -- 172.21.15.159:0/3165117464 shutdown_connections 2024-04-04T06:07:19.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.681+0000 7f0354a00640 1 --2- 172.21.15.159:0/3165117464 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f03500cc490 0x7f03500d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:19.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.681+0000 7f0354a00640 1 --2- 172.21.15.159:0/3165117464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f035015fc50 0x7f03500cfdc0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:19.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.681+0000 7f0354a00640 1 -- 172.21.15.159:0/3165117464 >> 172.21.15.159:0/3165117464 conn(0x7f03500ccad0 msgr2=0x7f03500cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:19.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.681+0000 7f0354a00640 1 -- 172.21.15.159:0/3165117464 shutdown_connections 2024-04-04T06:07:19.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.681+0000 7f0354a00640 1 -- 172.21.15.159:0/3165117464 wait complete. 2024-04-04T06:07:19.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.683+0000 7f0354a00640 1 Processor -- start 2024-04-04T06:07:19.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.683+0000 7f0354a00640 1 -- start start 2024-04-04T06:07:19.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.684+0000 7f0354a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03500cc0b0 0x7f0350155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:19.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.684+0000 7f0354a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f03500cc490 0x7f0350159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:19.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.684+0000 7f0354a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f035015baa0 0x7f035015a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:19.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.684+0000 7f0354a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f035015ca70 con 0x7f035015baa0 2024-04-04T06:07:19.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.684+0000 7f0354a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0350160980 con 0x7f03500cc0b0 2024-04-04T06:07:19.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.684+0000 7f0354a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f035015cbf0 con 0x7f03500cc490 2024-04-04T06:07:19.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.684+0000 7f034f200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f035015baa0 0x7f035015a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:19.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.684+0000 7f034f200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f035015baa0 0x7f035015a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53296/0 (socket says 172.21.15.159:53296) 2024-04-04T06:07:19.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.685+0000 7f034f200640 1 -- 172.21.15.159:0/1397840716 learned_addr learned my addr 172.21.15.159:0/1397840716 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:19.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.685+0000 7f034de00640 1 --2- 172.21.15.159:0/1397840716 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f03500cc490 0x7f0350159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:19.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.685+0000 7f034e800640 1 --2- 172.21.15.159:0/1397840716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03500cc0b0 0x7f0350155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:19.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.685+0000 7f034f200640 1 -- 172.21.15.159:0/1397840716 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f03500cc490 msgr2=0x7f0350159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:19.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.685+0000 7f034f200640 1 --2- 172.21.15.159:0/1397840716 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f03500cc490 0x7f0350159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:19.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.685+0000 7f034f200640 1 -- 172.21.15.159:0/1397840716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03500cc0b0 msgr2=0x7f0350155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:19.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.685+0000 7f034f200640 1 --2- 172.21.15.159:0/1397840716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03500cc0b0 0x7f0350155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:19.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.685+0000 7f034f200640 1 -- 172.21.15.159:0/1397840716 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035015aa00 con 0x7f035015baa0 2024-04-04T06:07:19.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.685+0000 7f034e800640 1 --2- 172.21.15.159:0/1397840716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03500cc0b0 0x7f0350155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:19.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.685+0000 7f034f200640 1 --2- 172.21.15.159:0/1397840716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f035015baa0 0x7f035015a360 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f034800c5c0 tx=0x7f034800ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:19.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.685+0000 7f0346a00640 1 -- 172.21.15.159:0/1397840716 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0348015070 con 0x7f035015baa0 2024-04-04T06:07:19.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.685+0000 7f0346a00640 1 -- 172.21.15.159:0/1397840716 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0348004220 con 0x7f035015baa0 2024-04-04T06:07:19.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.686+0000 7f0346a00640 1 -- 172.21.15.159:0/1397840716 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0348003610 con 0x7f035015baa0 2024-04-04T06:07:19.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.686+0000 7f0354a00640 1 -- 172.21.15.159:0/1397840716 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f035015ace0 con 0x7f035015baa0 2024-04-04T06:07:19.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.686+0000 7f0354a00640 1 -- 172.21.15.159:0/1397840716 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f035015b2a0 con 0x7f035015baa0 2024-04-04T06:07:19.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.687+0000 7f0354a00640 1 -- 172.21.15.159:0/1397840716 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0350075940 con 0x7f035015baa0 2024-04-04T06:07:19.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.688+0000 7f0346a00640 1 -- 172.21.15.159:0/1397840716 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f03480037b0 con 0x7f035015baa0 2024-04-04T06:07:19.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.689+0000 7f0346a00640 1 --2- 172.21.15.159:0/1397840716 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f033006c8f0 0x7f033006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:19.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.689+0000 7f0346a00640 1 -- 172.21.15.159:0/1397840716 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f034808bd80 con 0x7f035015baa0 2024-04-04T06:07:19.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.689+0000 7f034e800640 1 --2- 172.21.15.159:0/1397840716 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f033006c8f0 0x7f033006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:19.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.690+0000 7f034e800640 1 --2- 172.21.15.159:0/1397840716 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f033006c8f0 0x7f033006edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0340003eb0 tx=0x7f0340009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:19.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.694+0000 7f0346a00640 1 -- 172.21.15.159:0/1397840716 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f034805a110 con 0x7f035015baa0 2024-04-04T06:07:19.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.898+0000 7f0354a00640 1 -- 172.21.15.159:0/1397840716 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f035006f2a0 con 0x7f035015baa0 2024-04-04T06:07:19.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.900+0000 7f0346a00640 1 -- 172.21.15.159:0/1397840716 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16681 (secure 0 0 0) 0x7f0348059a10 con 0x7f035015baa0 2024-04-04T06:07:19.895 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:19.895 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":34,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:07:10.401521+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:07:06.708548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-04T06:07:07.833730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:07:19.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.903+0000 7f0354a00640 1 -- 172.21.15.159:0/1397840716 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f033006c8f0 msgr2=0x7f033006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:19.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.903+0000 7f0354a00640 1 --2- 172.21.15.159:0/1397840716 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f033006c8f0 0x7f033006edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0340003eb0 tx=0x7f0340009040 comp rx=0 tx=0).stop 2024-04-04T06:07:19.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.903+0000 7f0354a00640 1 -- 172.21.15.159:0/1397840716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f035015baa0 msgr2=0x7f035015a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:19.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.903+0000 7f0354a00640 1 --2- 172.21.15.159:0/1397840716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f035015baa0 0x7f035015a360 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f034800c5c0 tx=0x7f034800ca80 comp rx=0 tx=0).stop 2024-04-04T06:07:19.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.903+0000 7f0354a00640 1 -- 172.21.15.159:0/1397840716 shutdown_connections 2024-04-04T06:07:19.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.903+0000 7f0354a00640 1 --2- 172.21.15.159:0/1397840716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f035015baa0 0x7f035015a360 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:19.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.903+0000 7f0354a00640 1 --2- 172.21.15.159:0/1397840716 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f03500cc490 0x7f0350159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:19.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.903+0000 7f0354a00640 1 --2- 172.21.15.159:0/1397840716 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f033006c8f0 0x7f033006edb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:19.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.903+0000 7f0354a00640 1 --2- 172.21.15.159:0/1397840716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03500cc0b0 0x7f0350155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:19.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.903+0000 7f0354a00640 1 -- 172.21.15.159:0/1397840716 >> 172.21.15.159:0/1397840716 conn(0x7f03500ccad0 msgr2=0x7f035015ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:19.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.903+0000 7f0354a00640 1 -- 172.21.15.159:0/1397840716 shutdown_connections 2024-04-04T06:07:19.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:19.903+0000 7f0354a00640 1 -- 172.21.15.159:0/1397840716 wait complete. 2024-04-04T06:07:19.917 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:07:06.708548+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:07:07.833730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:07:19.918 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:07:20.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.137+0000 7fe94b000640 1 Processor -- start 2024-04-04T06:07:20.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.137+0000 7fe94b000640 1 -- start start 2024-04-04T06:07:20.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.139+0000 7fe94b000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe944155550 0x7fe944155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:20.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.139+0000 7fe94b000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe94415f800 0x7fe9440d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:20.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.139+0000 7fe94b000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe9440d07c0 0x7fe9440aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:20.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.139+0000 7fe94b000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe9440d0c30 con 0x7fe94415f800 2024-04-04T06:07:20.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.139+0000 7fe94b000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe9440b40b0 con 0x7fe9440d07c0 2024-04-04T06:07:20.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.139+0000 7fe94b000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe9440b4390 con 0x7fe944155550 2024-04-04T06:07:20.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.139+0000 7fe93b400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe94415f800 0x7fe9440d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:20.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.140+0000 7fe93b400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe94415f800 0x7fe9440d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53310/0 (socket says 172.21.15.159:53310) 2024-04-04T06:07:20.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.140+0000 7fe93b400640 1 -- 172.21.15.159:0/1208343086 learned_addr learned my addr 172.21.15.159:0/1208343086 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:20.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.140+0000 7fe949000640 1 --2- 172.21.15.159:0/1208343086 >> v2:172.21.15.178:3300/0 conn(0x7fe9440d07c0 0x7fe9440aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:20.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.140+0000 7fe943e00640 1 --2- 172.21.15.159:0/1208343086 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe944155550 0x7fe944155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:20.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.140+0000 7fe93b400640 1 -- 172.21.15.159:0/1208343086 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe944155550 msgr2=0x7fe944155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:20.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.140+0000 7fe93b400640 1 --2- 172.21.15.159:0/1208343086 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe944155550 0x7fe944155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.140+0000 7fe93b400640 1 -- 172.21.15.159:0/1208343086 >> v2:172.21.15.178:3300/0 conn(0x7fe9440d07c0 msgr2=0x7fe9440aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:20.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.140+0000 7fe93b400640 1 --2- 172.21.15.159:0/1208343086 >> v2:172.21.15.178:3300/0 conn(0x7fe9440d07c0 0x7fe9440aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.140+0000 7fe93b400640 1 -- 172.21.15.159:0/1208343086 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9440b4670 con 0x7fe94415f800 2024-04-04T06:07:20.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.140+0000 7fe949000640 1 --2- 172.21.15.159:0/1208343086 >> v2:172.21.15.178:3300/0 conn(0x7fe9440d07c0 0x7fe9440aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:07:20.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.141+0000 7fe93b400640 1 --2- 172.21.15.159:0/1208343086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe94415f800 0x7fe9440d0280 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe9440713d0 tx=0x7fe92c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a766ff3619e3d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:20.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.141+0000 7fe943400640 1 -- 172.21.15.159:0/1208343086 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c035020 con 0x7fe94415f800 2024-04-04T06:07:20.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.141+0000 7fe943400640 1 -- 172.21.15.159:0/1208343086 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe92c02d040 con 0x7fe94415f800 2024-04-04T06:07:20.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.141+0000 7fe943400640 1 -- 172.21.15.159:0/1208343086 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c031a80 con 0x7fe94415f800 2024-04-04T06:07:20.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.142+0000 7fe94b000640 1 -- 172.21.15.159:0/1208343086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe94415f800 msgr2=0x7fe9440d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:20.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.142+0000 7fe94b000640 1 --2- 172.21.15.159:0/1208343086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe94415f800 0x7fe9440d0280 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe9440713d0 tx=0x7fe92c02b900 comp rx=0 tx=0).stop 2024-04-04T06:07:20.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.142+0000 7fe94b000640 1 -- 172.21.15.159:0/1208343086 shutdown_connections 2024-04-04T06:07:20.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.142+0000 7fe94b000640 1 --2- 172.21.15.159:0/1208343086 >> v2:172.21.15.178:3300/0 conn(0x7fe9440d07c0 0x7fe9440aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.142+0000 7fe94b000640 1 --2- 172.21.15.159:0/1208343086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe94415f800 0x7fe9440d0280 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.142+0000 7fe94b000640 1 --2- 172.21.15.159:0/1208343086 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe944155550 0x7fe944155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.142+0000 7fe94b000640 1 -- 172.21.15.159:0/1208343086 >> 172.21.15.159:0/1208343086 conn(0x7fe94406b750 msgr2=0x7fe94406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:20.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.142+0000 7fe94b000640 1 -- 172.21.15.159:0/1208343086 shutdown_connections 2024-04-04T06:07:20.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.142+0000 7fe94b000640 1 -- 172.21.15.159:0/1208343086 wait complete. 2024-04-04T06:07:20.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.144+0000 7fe94b000640 1 Processor -- start 2024-04-04T06:07:20.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.144+0000 7fe94b000640 1 -- start start 2024-04-04T06:07:20.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.145+0000 7fe94b000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9440d07c0 0x7fe9441f87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:20.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.145+0000 7fe94b000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe944155550 0x7fe9440b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:20.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.145+0000 7fe94b000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe94415f800 0x7fe9441f4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:20.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.145+0000 7fe94b000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe94415fc80 con 0x7fe9440d07c0 2024-04-04T06:07:20.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.145+0000 7fe94b000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe9440b4c30 con 0x7fe94415f800 2024-04-04T06:07:20.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.145+0000 7fe94b000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe94415ba20 con 0x7fe944155550 2024-04-04T06:07:20.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.145+0000 7fe943e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9440d07c0 0x7fe9441f87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:20.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.145+0000 7fe943e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9440d07c0 0x7fe9441f87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53314/0 (socket says 172.21.15.159:53314) 2024-04-04T06:07:20.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.145+0000 7fe943e00640 1 -- 172.21.15.159:0/1186172854 learned_addr learned my addr 172.21.15.159:0/1186172854 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:20.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.146+0000 7fe93b400640 1 --2- 172.21.15.159:0/1186172854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe944155550 0x7fe9440b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:20.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.146+0000 7fe949000640 1 --2- 172.21.15.159:0/1186172854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe94415f800 0x7fe9441f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:20.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.146+0000 7fe943e00640 1 -- 172.21.15.159:0/1186172854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe944155550 msgr2=0x7fe9440b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:20.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.146+0000 7fe943e00640 1 --2- 172.21.15.159:0/1186172854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe944155550 0x7fe9440b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.146+0000 7fe943e00640 1 -- 172.21.15.159:0/1186172854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe94415f800 msgr2=0x7fe9441f4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:20.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.146+0000 7fe943e00640 1 --2- 172.21.15.159:0/1186172854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe94415f800 0x7fe9441f4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.146+0000 7fe943e00640 1 -- 172.21.15.159:0/1186172854 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9441f5240 con 0x7fe9440d07c0 2024-04-04T06:07:20.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.146+0000 7fe943e00640 1 --2- 172.21.15.159:0/1186172854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9440d07c0 0x7fe9441f87a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe934007e40 tx=0x7fe93400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:20.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.146+0000 7fe940c00640 1 -- 172.21.15.159:0/1186172854 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934014620 con 0x7fe9440d07c0 2024-04-04T06:07:20.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.146+0000 7fe940c00640 1 -- 172.21.15.159:0/1186172854 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe934019070 con 0x7fe9440d07c0 2024-04-04T06:07:20.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.146+0000 7fe940c00640 1 -- 172.21.15.159:0/1186172854 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934014a00 con 0x7fe9440d07c0 2024-04-04T06:07:20.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.146+0000 7fe94b000640 1 -- 172.21.15.159:0/1186172854 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9441f5520 con 0x7fe9440d07c0 2024-04-04T06:07:20.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.147+0000 7fe94b000640 1 -- 172.21.15.159:0/1186172854 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9441f7fa0 con 0x7fe9440d07c0 2024-04-04T06:07:20.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.148+0000 7fe94b000640 1 -- 172.21.15.159:0/1186172854 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9440685c0 con 0x7fe9440d07c0 2024-04-04T06:07:20.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.149+0000 7fe940c00640 1 -- 172.21.15.159:0/1186172854 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe934014ba0 con 0x7fe9440d07c0 2024-04-04T06:07:20.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.150+0000 7fe940c00640 1 --2- 172.21.15.159:0/1186172854 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe92006c8f0 0x7fe92006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:20.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.150+0000 7fe940c00640 1 -- 172.21.15.159:0/1186172854 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fe93408c380 con 0x7fe9440d07c0 2024-04-04T06:07:20.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.150+0000 7fe93b400640 1 --2- 172.21.15.159:0/1186172854 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe92006c8f0 0x7fe92006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:20.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.151+0000 7fe93b400640 1 --2- 172.21.15.159:0/1186172854 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe92006c8f0 0x7fe92006edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe92c036000 tx=0x7fe92c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:20.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.154+0000 7fe940c00640 1 -- 172.21.15.159:0/1186172854 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe93405a7a0 con 0x7fe9440d07c0 2024-04-04T06:07:20.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.355+0000 7fe94b000640 1 -- 172.21.15.159:0/1186172854 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe944155330 con 0x7fe9440d07c0 2024-04-04T06:07:20.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.356+0000 7fe940c00640 1 -- 172.21.15.159:0/1186172854 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7fe93405a0a0 con 0x7fe9440d07c0 2024-04-04T06:07:20.350 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:07:20.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.358+0000 7fe94b000640 1 -- 172.21.15.159:0/1186172854 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe92006c8f0 msgr2=0x7fe92006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:20.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.358+0000 7fe94b000640 1 --2- 172.21.15.159:0/1186172854 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe92006c8f0 0x7fe92006edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe92c036000 tx=0x7fe92c008030 comp rx=0 tx=0).stop 2024-04-04T06:07:20.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.358+0000 7fe94b000640 1 -- 172.21.15.159:0/1186172854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9440d07c0 msgr2=0x7fe9441f87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:20.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.358+0000 7fe94b000640 1 --2- 172.21.15.159:0/1186172854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9440d07c0 0x7fe9441f87a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe934007e40 tx=0x7fe93400e4b0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.358+0000 7fe94b000640 1 -- 172.21.15.159:0/1186172854 shutdown_connections 2024-04-04T06:07:20.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.358+0000 7fe94b000640 1 --2- 172.21.15.159:0/1186172854 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe92006c8f0 0x7fe92006edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.358+0000 7fe94b000640 1 --2- 172.21.15.159:0/1186172854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe94415f800 0x7fe9441f4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.358+0000 7fe94b000640 1 --2- 172.21.15.159:0/1186172854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe944155550 0x7fe9440b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.358+0000 7fe94b000640 1 --2- 172.21.15.159:0/1186172854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9440d07c0 0x7fe9441f87a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.358+0000 7fe94b000640 1 -- 172.21.15.159:0/1186172854 >> 172.21.15.159:0/1186172854 conn(0x7fe94406b750 msgr2=0x7fe944071610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:20.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.359+0000 7fe94b000640 1 -- 172.21.15.159:0/1186172854 shutdown_connections 2024-04-04T06:07:20.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.359+0000 7fe94b000640 1 -- 172.21.15.159:0/1186172854 wait complete. 2024-04-04T06:07:20.371 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:07:20.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.591+0000 7f7e2f000640 1 Processor -- start 2024-04-04T06:07:20.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.591+0000 7f7e2f000640 1 -- start start 2024-04-04T06:07:20.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.592+0000 7f7e2f000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e28153580 0x7f7e280d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:20.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.592+0000 7f7e2f000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7e280af4b0 0x7f7e281600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:20.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.593+0000 7f7e2f000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e280afab0 con 0x7f7e280d07c0 2024-04-04T06:07:20.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.593+0000 7f7e2f000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7e280d0c00 con 0x7f7e280af4b0 2024-04-04T06:07:20.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.593+0000 7f7e2f000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7e2815cc80 con 0x7f7e28153580 2024-04-04T06:07:20.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.593+0000 7f7e1f400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f7e280d07c0 0x7f7e280aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:49212/0 (socket says 172.21.15.159:49212) 2024-04-04T06:07:20.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.593+0000 7f7e1f400640 1 -- 172.21.15.159:0/1447552210 learned_addr learned my addr 172.21.15.159:0/1447552210 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:20.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.593+0000 7f7e27e00640 1 --2- 172.21.15.159:0/1447552210 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e28153580 0x7f7e280d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:20.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.593+0000 7f7e2d000640 1 --2- 172.21.15.159:0/1447552210 >> v2:172.21.15.178:3300/0 conn(0x7f7e280af4b0 0x7f7e281600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:20.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.593+0000 7f7e27400640 1 -- 172.21.15.159:0/1447552210 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2962509818 0 0) 0x7f7e280afab0 con 0x7f7e280d07c0 2024-04-04T06:07:20.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.594+0000 7f7e27400640 1 -- 172.21.15.159:0/1447552210 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e0c003520 con 0x7f7e280d07c0 2024-04-04T06:07:20.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.594+0000 7f7e27400640 1 -- 172.21.15.159:0/1447552210 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 903733706 0 0) 0x7f7e0c003520 con 0x7f7e280d07c0 2024-04-04T06:07:20.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.594+0000 7f7e27400640 1 -- 172.21.15.159:0/1447552210 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e28153580 msgr2=0x7f7e280d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:20.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.594+0000 7f7e27400640 1 --2- 172.21.15.159:0/1447552210 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e28153580 0x7f7e280d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.594+0000 7f7e27400640 1 -- 172.21.15.159:0/1447552210 >> v2:172.21.15.178:3300/0 conn(0x7f7e280af4b0 msgr2=0x7f7e281600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:20.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.594+0000 7f7e27400640 1 --2- 172.21.15.159:0/1447552210 >> v2:172.21.15.178:3300/0 conn(0x7f7e280af4b0 0x7f7e281600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.594+0000 7f7e27400640 1 -- 172.21.15.159:0/1447552210 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2815da00 con 0x7f7e280d07c0 2024-04-04T06:07:20.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.594+0000 7f7e27400640 1 -- 172.21.15.159:0/1447552210 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f7e10001670 con 0x7f7e280d07c0 2024-04-04T06:07:20.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.594+0000 7f7e2d000640 1 --2- 172.21.15.159:0/1447552210 >> v2:172.21.15.178:3300/0 conn(0x7f7e280af4b0 0x7f7e281600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:07:20.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.594+0000 7f7e27400640 1 -- 172.21.15.159:0/1447552210 >> v1:172.21.15.159:6789/0 conn(0x7f7e280d07c0 legacy=0x7f7e280aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:20.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.594+0000 7f7e27400640 1 --2- 172.21.15.159:0/1447552210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e0c0039b0 0x7f7e0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:20.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.594+0000 7f7e27400640 1 --2- 172.21.15.159:0/1447552210 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e28153580 0x7f7e0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:20.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.594+0000 7f7e27400640 1 --2- 172.21.15.159:0/1447552210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e0c004d40 0x7f7e0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:20.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.594+0000 7f7e27400640 1 -- 172.21.15.159:0/1447552210 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7e280d0c00 con 0x7f7e0c004d40 2024-04-04T06:07:20.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.594+0000 7f7e27400640 1 -- 172.21.15.159:0/1447552210 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2815cc80 con 0x7f7e0c0039b0 2024-04-04T06:07:20.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.594+0000 7f7e27400640 1 -- 172.21.15.159:0/1447552210 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7e0c00b9f0 con 0x7f7e28153580 2024-04-04T06:07:20.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.594+0000 7f7e2d000640 1 --2- 172.21.15.159:0/1447552210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e0c004d40 0x7f7e0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:20.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.595+0000 7f7e27e00640 1 --2- 172.21.15.159:0/1447552210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e0c0039b0 0x7f7e0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:20.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.595+0000 7f7e1f400640 1 --2- 172.21.15.159:0/1447552210 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e28153580 0x7f7e0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:20.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.595+0000 7f7e2d000640 1 -- 172.21.15.159:0/1447552210 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e28153580 msgr2=0x7f7e0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:20.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.595+0000 7f7e2d000640 1 --2- 172.21.15.159:0/1447552210 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e28153580 0x7f7e0c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.595+0000 7f7e2d000640 1 -- 172.21.15.159:0/1447552210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e0c0039b0 msgr2=0x7f7e0c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:20.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.595+0000 7f7e2d000640 1 --2- 172.21.15.159:0/1447552210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e0c0039b0 0x7f7e0c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.595+0000 7f7e2d000640 1 -- 172.21.15.159:0/1447552210 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e0c003520 con 0x7f7e0c004d40 2024-04-04T06:07:20.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.595+0000 7f7e1f400640 1 --2- 172.21.15.159:0/1447552210 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e28153580 0x7f7e0c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:20.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.596+0000 7f7e2d000640 1 --2- 172.21.15.159:0/1447552210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e0c004d40 0x7f7e0c0051c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009c90 tx=0x7f7e0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa471b7abbb1dd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:20.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.596+0000 7f7e27400640 1 -- 172.21.15.159:0/1447552210 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e20039070 con 0x7f7e0c004d40 2024-04-04T06:07:20.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.596+0000 7f7e27400640 1 -- 172.21.15.159:0/1447552210 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e2002c980 con 0x7f7e0c004d40 2024-04-04T06:07:20.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.596+0000 7f7e27400640 1 -- 172.21.15.159:0/1447552210 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e20009860 con 0x7f7e0c004d40 2024-04-04T06:07:20.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.597+0000 7f7e2f000640 1 -- 172.21.15.159:0/1447552210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e0c004d40 msgr2=0x7f7e0c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:20.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.597+0000 7f7e2f000640 1 --2- 172.21.15.159:0/1447552210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e0c004d40 0x7f7e0c0051c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009c90 tx=0x7f7e0c005830 comp rx=0 tx=0).stop 2024-04-04T06:07:20.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.597+0000 7f7e2d000640 1 -- 172.21.15.159:0/1447552210 reap_dead start 2024-04-04T06:07:20.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.597+0000 7f7e2f000640 1 -- 172.21.15.159:0/1447552210 shutdown_connections 2024-04-04T06:07:20.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.597+0000 7f7e2f000640 1 -- 172.21.15.159:0/1447552210 >> 172.21.15.159:0/1447552210 conn(0x7f7e2806b750 msgr2=0x7f7e2806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:20.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.597+0000 7f7e2f000640 1 -- 172.21.15.159:0/1447552210 shutdown_connections 2024-04-04T06:07:20.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.597+0000 7f7e2f000640 1 -- 172.21.15.159:0/1447552210 wait complete. 2024-04-04T06:07:20.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.599+0000 7f7e2f000640 1 Processor -- start 2024-04-04T06:07:20.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.599+0000 7f7e2f000640 1 -- start start 2024-04-04T06:07:20.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.600+0000 7f7e2f000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e0c004d40 0x7f7e28151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:20.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.600+0000 7f7e2f000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e28153bc0 0x7f7e28071f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:20.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.600+0000 7f7e2f000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e28154040 0x7f7e280724d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:20.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.600+0000 7f7e2f000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2815d4e0 con 0x7f7e28153bc0 2024-04-04T06:07:20.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.600+0000 7f7e2f000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2815d200 con 0x7f7e28154040 2024-04-04T06:07:20.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.600+0000 7f7e2f000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7e28153960 con 0x7f7e0c004d40 2024-04-04T06:07:20.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.600+0000 7f7e1f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e28153bc0 0x7f7e28071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:20.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.600+0000 7f7e1f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e28153bc0 0x7f7e28071f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53330/0 (socket says 172.21.15.159:53330) 2024-04-04T06:07:20.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.600+0000 7f7e1f400640 1 -- 172.21.15.159:0/2642982214 learned_addr learned my addr 172.21.15.159:0/2642982214 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:20.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.600+0000 7f7e2d000640 1 --2- 172.21.15.159:0/2642982214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e28154040 0x7f7e280724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:20.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.600+0000 7f7e1f400640 1 -- 172.21.15.159:0/2642982214 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e0c004d40 msgr2=0x7f7e28151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:20.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.600+0000 7f7e27e00640 1 --2- 172.21.15.159:0/2642982214 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e0c004d40 0x7f7e28151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:20.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.601+0000 7f7e1f400640 1 --2- 172.21.15.159:0/2642982214 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e0c004d40 0x7f7e28151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.601+0000 7f7e1f400640 1 -- 172.21.15.159:0/2642982214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e28154040 msgr2=0x7f7e280724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:20.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.601+0000 7f7e1f400640 1 --2- 172.21.15.159:0/2642982214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e28154040 0x7f7e280724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.601+0000 7f7e1f400640 1 -- 172.21.15.159:0/2642982214 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e281571a0 con 0x7f7e28153bc0 2024-04-04T06:07:20.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.601+0000 7f7e2d000640 1 --2- 172.21.15.159:0/2642982214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e28154040 0x7f7e280724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:20.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.601+0000 7f7e27e00640 1 --2- 172.21.15.159:0/2642982214 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e0c004d40 0x7f7e28151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:20.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.601+0000 7f7e1f400640 1 --2- 172.21.15.159:0/2642982214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e28153bc0 0x7f7e28071f90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000d8e0 tx=0x7f7e1000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:20.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.601+0000 7f7e24c00640 1 -- 172.21.15.159:0/2642982214 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10019070 con 0x7f7e28153bc0 2024-04-04T06:07:20.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.601+0000 7f7e24c00640 1 -- 172.21.15.159:0/2642982214 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e100074e0 con 0x7f7e28153bc0 2024-04-04T06:07:20.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.601+0000 7f7e24c00640 1 -- 172.21.15.159:0/2642982214 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10007690 con 0x7f7e28153bc0 2024-04-04T06:07:20.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.601+0000 7f7e2f000640 1 -- 172.21.15.159:0/2642982214 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e28157480 con 0x7f7e28153bc0 2024-04-04T06:07:20.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.601+0000 7f7e2f000640 1 -- 172.21.15.159:0/2642982214 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e28157a40 con 0x7f7e28153bc0 2024-04-04T06:07:20.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.603+0000 7f7e2f000640 1 -- 172.21.15.159:0/2642982214 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e280685c0 con 0x7f7e28153bc0 2024-04-04T06:07:20.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.604+0000 7f7e24c00640 1 -- 172.21.15.159:0/2642982214 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e100049e0 con 0x7f7e28153bc0 2024-04-04T06:07:20.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.604+0000 7f7e24c00640 1 --2- 172.21.15.159:0/2642982214 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e0406c8f0 0x7f7e0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:20.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.605+0000 7f7e24c00640 1 -- 172.21.15.159:0/2642982214 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f7e1008b780 con 0x7f7e28153bc0 2024-04-04T06:07:20.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.605+0000 7f7e27e00640 1 --2- 172.21.15.159:0/2642982214 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e0406c8f0 0x7f7e0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:20.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.606+0000 7f7e27e00640 1 --2- 172.21.15.159:0/2642982214 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e0406c8f0 0x7f7e0406edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7e18007440 tx=0x7f7e1800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:20.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.609+0000 7f7e24c00640 1 -- 172.21.15.159:0/2642982214 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7e10059bb0 con 0x7f7e28153bc0 2024-04-04T06:07:20.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.812+0000 7f7e2f000640 1 -- 172.21.15.159:0/2642982214 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7e280ad820 con 0x7f7e28153bc0 2024-04-04T06:07:20.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.813+0000 7f7e24c00640 1 -- 172.21.15.159:0/2642982214 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7f7e100594b0 con 0x7f7e28153bc0 2024-04-04T06:07:20.808 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:07:20.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.817+0000 7f7e2f000640 1 -- 172.21.15.159:0/2642982214 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e0406c8f0 msgr2=0x7f7e0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:20.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.817+0000 7f7e2f000640 1 --2- 172.21.15.159:0/2642982214 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e0406c8f0 0x7f7e0406edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7e18007440 tx=0x7f7e1800b040 comp rx=0 tx=0).stop 2024-04-04T06:07:20.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.817+0000 7f7e2f000640 1 -- 172.21.15.159:0/2642982214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e28153bc0 msgr2=0x7f7e28071f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:20.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.817+0000 7f7e2f000640 1 --2- 172.21.15.159:0/2642982214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e28153bc0 0x7f7e28071f90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000d8e0 tx=0x7f7e1000dda0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.817+0000 7f7e2f000640 1 -- 172.21.15.159:0/2642982214 shutdown_connections 2024-04-04T06:07:20.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.817+0000 7f7e2f000640 1 --2- 172.21.15.159:0/2642982214 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e0406c8f0 0x7f7e0406edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.817+0000 7f7e2f000640 1 --2- 172.21.15.159:0/2642982214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e28154040 0x7f7e280724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.817+0000 7f7e2f000640 1 --2- 172.21.15.159:0/2642982214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e28153bc0 0x7f7e28071f90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.817+0000 7f7e2f000640 1 --2- 172.21.15.159:0/2642982214 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e0c004d40 0x7f7e28151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:20.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.817+0000 7f7e2f000640 1 -- 172.21.15.159:0/2642982214 >> 172.21.15.159:0/2642982214 conn(0x7f7e2806b750 msgr2=0x7f7e2815bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:20.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.817+0000 7f7e2f000640 1 -- 172.21.15.159:0/2642982214 shutdown_connections 2024-04-04T06:07:20.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:20.817+0000 7f7e2f000640 1 -- 172.21.15.159:0/2642982214 wait complete. 2024-04-04T06:07:20.829 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:07:21.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.049+0000 7fbc73200640 1 Processor -- start 2024-04-04T06:07:21.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.050+0000 7fbc73200640 1 -- start start 2024-04-04T06:07:21.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.051+0000 7fbc73200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc6c141c10 0x7fbc6c071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:21.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.051+0000 7fbc73200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbc6c071eb0 0x7fbc6c162160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:21.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.051+0000 7fbc73200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc6c15ec80 con 0x7fbc6c0d0280 2024-04-04T06:07:21.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.051+0000 7fbc73200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fbc6c15ef20 con 0x7fbc6c071eb0 2024-04-04T06:07:21.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.051+0000 7fbc73200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbc6c15f200 con 0x7fbc6c141c10 2024-04-04T06:07:21.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.052+0000 7fbc71200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbc6c071eb0 0x7fbc6c162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:21.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.052+0000 7fbc6b400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc6c141c10 0x7fbc6c071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:21.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.052+0000 7fbc6b400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc6c141c10 0x7fbc6c071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:40270/0 (socket says 172.21.15.159:40270) 2024-04-04T06:07:21.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.052+0000 7fbc6b400640 1 -- 172.21.15.159:0/1857129823 learned_addr learned my addr 172.21.15.159:0/1857129823 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:21.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.052+0000 7fbc6b400640 1 -- 172.21.15.159:0/1857129823 >> v2:172.21.15.178:3300/0 conn(0x7fbc6c071eb0 msgr2=0x7fbc6c162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:21.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.052+0000 7fbc6b400640 1 --2- 172.21.15.159:0/1857129823 >> v2:172.21.15.178:3300/0 conn(0x7fbc6c071eb0 0x7fbc6c162160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:21.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.052+0000 7fbc6b400640 1 -- 172.21.15.159:0/1857129823 >> v1:172.21.15.178:6789/0 conn(0x7fbc6c0d0280 legacy=0x7fbc6c0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:21.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.052+0000 7fbc6b400640 1 -- 172.21.15.159:0/1857129823 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6c15f4e0 con 0x7fbc6c141c10 2024-04-04T06:07:21.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.054+0000 7fbc6b400640 1 --2- 172.21.15.159:0/1857129823 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc6c141c10 0x7fbc6c071970 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009e50 tx=0x7fbc5402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e34378e6b862a161 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:21.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.054+0000 7fbc6aa00640 1 -- 172.21.15.159:0/1857129823 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5402cb00 con 0x7fbc6c141c10 2024-04-04T06:07:21.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.054+0000 7fbc6aa00640 1 -- 172.21.15.159:0/1857129823 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc5402cca0 con 0x7fbc6c141c10 2024-04-04T06:07:21.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.054+0000 7fbc6aa00640 1 -- 172.21.15.159:0/1857129823 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54034610 con 0x7fbc6c141c10 2024-04-04T06:07:21.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.055+0000 7fbc73200640 1 -- 172.21.15.159:0/1857129823 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc6c141c10 msgr2=0x7fbc6c071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:21.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.055+0000 7fbc73200640 1 --2- 172.21.15.159:0/1857129823 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc6c141c10 0x7fbc6c071970 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009e50 tx=0x7fbc5402bf60 comp rx=0 tx=0).stop 2024-04-04T06:07:21.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.055+0000 7fbc73200640 1 -- 172.21.15.159:0/1857129823 shutdown_connections 2024-04-04T06:07:21.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.055+0000 7fbc73200640 1 --2- 172.21.15.159:0/1857129823 >> v2:172.21.15.178:3300/0 conn(0x7fbc6c071eb0 0x7fbc6c162160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:21.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.055+0000 7fbc73200640 1 --2- 172.21.15.159:0/1857129823 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc6c141c10 0x7fbc6c071970 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:21.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.055+0000 7fbc73200640 1 -- 172.21.15.159:0/1857129823 >> 172.21.15.159:0/1857129823 conn(0x7fbc6c06b750 msgr2=0x7fbc6c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:21.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.055+0000 7fbc73200640 1 -- 172.21.15.159:0/1857129823 shutdown_connections 2024-04-04T06:07:21.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.055+0000 7fbc73200640 1 -- 172.21.15.159:0/1857129823 wait complete. 2024-04-04T06:07:21.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.057+0000 7fbc73200640 1 Processor -- start 2024-04-04T06:07:21.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.057+0000 7fbc73200640 1 -- start start 2024-04-04T06:07:21.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.059+0000 7fbc73200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc6c071eb0 0x7fbc6c09a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:21.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.059+0000 7fbc73200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc6c0d0280 0x7fbc6c148840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:21.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.059+0000 7fbc73200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc6c141c10 0x7fbc6c0ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:21.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.059+0000 7fbc73200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c158e90 con 0x7fbc6c0d0280 2024-04-04T06:07:21.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.059+0000 7fbc73200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c06f340 con 0x7fbc6c141c10 2024-04-04T06:07:21.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.059+0000 7fbc73200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbc6c09b550 con 0x7fbc6c071eb0 2024-04-04T06:07:21.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.059+0000 7fbc6b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc6c0d0280 0x7fbc6c148840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:21.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.059+0000 7fbc6b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc6c0d0280 0x7fbc6c148840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53336/0 (socket says 172.21.15.159:53336) 2024-04-04T06:07:21.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.059+0000 7fbc6b400640 1 -- 172.21.15.159:0/3671911009 learned_addr learned my addr 172.21.15.159:0/3671911009 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:21.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.059+0000 7fbc6be00640 1 --2- 172.21.15.159:0/3671911009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc6c071eb0 0x7fbc6c09a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:21.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.059+0000 7fbc71200640 1 --2- 172.21.15.159:0/3671911009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc6c141c10 0x7fbc6c0ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:21.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.060+0000 7fbc6b400640 1 -- 172.21.15.159:0/3671911009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc6c071eb0 msgr2=0x7fbc6c09a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:21.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.060+0000 7fbc6b400640 1 --2- 172.21.15.159:0/3671911009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc6c071eb0 0x7fbc6c09a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:21.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.060+0000 7fbc6b400640 1 -- 172.21.15.159:0/3671911009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc6c141c10 msgr2=0x7fbc6c0ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:21.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.060+0000 7fbc6b400640 1 --2- 172.21.15.159:0/3671911009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc6c141c10 0x7fbc6c0ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:21.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.060+0000 7fbc6b400640 1 -- 172.21.15.159:0/3671911009 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6c148d80 con 0x7fbc6c0d0280 2024-04-04T06:07:21.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.060+0000 7fbc6b400640 1 --2- 172.21.15.159:0/3671911009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc6c0d0280 0x7fbc6c148840 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbc54000f80 tx=0x7fbc54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:21.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.060+0000 7fbc63e00640 1 -- 172.21.15.159:0/3671911009 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54043020 con 0x7fbc6c0d0280 2024-04-04T06:07:21.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.060+0000 7fbc63e00640 1 -- 172.21.15.159:0/3671911009 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc5402ce50 con 0x7fbc6c0d0280 2024-04-04T06:07:21.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.060+0000 7fbc63e00640 1 -- 172.21.15.159:0/3671911009 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54009510 con 0x7fbc6c0d0280 2024-04-04T06:07:21.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.060+0000 7fbc73200640 1 -- 172.21.15.159:0/3671911009 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc6c09a770 con 0x7fbc6c0d0280 2024-04-04T06:07:21.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.061+0000 7fbc73200640 1 -- 172.21.15.159:0/3671911009 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc6c09ad30 con 0x7fbc6c0d0280 2024-04-04T06:07:21.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.062+0000 7fbc73200640 1 -- 172.21.15.159:0/3671911009 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc6c06e660 con 0x7fbc6c0d0280 2024-04-04T06:07:21.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.063+0000 7fbc63e00640 1 -- 172.21.15.159:0/3671911009 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc54046020 con 0x7fbc6c0d0280 2024-04-04T06:07:21.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.063+0000 7fbc63e00640 1 --2- 172.21.15.159:0/3671911009 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc4806c8f0 0x7fbc4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:21.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.064+0000 7fbc63e00640 1 -- 172.21.15.159:0/3671911009 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fbc540ae170 con 0x7fbc6c0d0280 2024-04-04T06:07:21.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.064+0000 7fbc6be00640 1 --2- 172.21.15.159:0/3671911009 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc4806c8f0 0x7fbc4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:21.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.065+0000 7fbc6be00640 1 --2- 172.21.15.159:0/3671911009 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc4806c8f0 0x7fbc4806edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c001640 tx=0x7fbc5c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:21.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.068+0000 7fbc63e00640 1 -- 172.21.15.159:0/3671911009 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbc6c06e660 con 0x7fbc6c0d0280 2024-04-04T06:07:21.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.269+0000 7fbc73200640 1 -- 172.21.15.159:0/3671911009 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbc6c0685c0 con 0x7fbc6c0d0280 2024-04-04T06:07:21.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.270+0000 7fbc63e00640 1 -- 172.21.15.159:0/3671911009 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7fbc5404d030 con 0x7fbc6c0d0280 2024-04-04T06:07:21.265 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:07:21.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.272+0000 7fbc73200640 1 -- 172.21.15.159:0/3671911009 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc4806c8f0 msgr2=0x7fbc4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:21.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.272+0000 7fbc73200640 1 --2- 172.21.15.159:0/3671911009 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc4806c8f0 0x7fbc4806edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c001640 tx=0x7fbc5c009040 comp rx=0 tx=0).stop 2024-04-04T06:07:21.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.272+0000 7fbc73200640 1 -- 172.21.15.159:0/3671911009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc6c0d0280 msgr2=0x7fbc6c148840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:21.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.272+0000 7fbc73200640 1 --2- 172.21.15.159:0/3671911009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc6c0d0280 0x7fbc6c148840 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbc54000f80 tx=0x7fbc54004a40 comp rx=0 tx=0).stop 2024-04-04T06:07:21.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.272+0000 7fbc73200640 1 -- 172.21.15.159:0/3671911009 shutdown_connections 2024-04-04T06:07:21.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.272+0000 7fbc73200640 1 --2- 172.21.15.159:0/3671911009 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc4806c8f0 0x7fbc4806edb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:21.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.272+0000 7fbc73200640 1 --2- 172.21.15.159:0/3671911009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc6c141c10 0x7fbc6c0ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:21.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.272+0000 7fbc73200640 1 --2- 172.21.15.159:0/3671911009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc6c0d0280 0x7fbc6c148840 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:21.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.272+0000 7fbc73200640 1 --2- 172.21.15.159:0/3671911009 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc6c071eb0 0x7fbc6c09a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:21.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.272+0000 7fbc73200640 1 -- 172.21.15.159:0/3671911009 >> 172.21.15.159:0/3671911009 conn(0x7fbc6c06b750 msgr2=0x7fbc6c158030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:21.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.272+0000 7fbc73200640 1 -- 172.21.15.159:0/3671911009 shutdown_connections 2024-04-04T06:07:21.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.272+0000 7fbc73200640 1 -- 172.21.15.159:0/3671911009 wait complete. 2024-04-04T06:07:21.287 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-04T06:07:21.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.490+0000 7fa774800640 1 Processor -- start 2024-04-04T06:07:21.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.490+0000 7fa774800640 1 -- start start 2024-04-04T06:07:21.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.492+0000 7fa774800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa76c0b4650 0x7fa76c06c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:21.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.492+0000 7fa774800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa76c0b4a30 0x7fa76c0b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:21.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.492+0000 7fa774800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa76c158580 0x7fa76c1589f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:21.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.492+0000 7fa774800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fa76c141b30 con 0x7fa76c0b4a30 2024-04-04T06:07:21.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.492+0000 7fa774800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa76c15ede0 con 0x7fa76c0b4650 2024-04-04T06:07:21.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.492+0000 7fa774800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa76c141d80 con 0x7fa76c158580 2024-04-04T06:07:21.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.492+0000 7fa771400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa76c0b4a30 0x7fa76c0b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:21.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.492+0000 7fa771400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa76c0b4a30 0x7fa76c0b5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53352/0 (socket says 172.21.15.159:53352) 2024-04-04T06:07:21.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.492+0000 7fa771400640 1 -- 172.21.15.159:0/3926650192 learned_addr learned my addr 172.21.15.159:0/3926650192 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:21.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.493+0000 7fa772800640 1 --2- 172.21.15.159:0/3926650192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa76c158580 0x7fa76c1589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:21.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.493+0000 7fa771e00640 1 --2- 172.21.15.159:0/3926650192 >> v2:172.21.15.178:3300/0 conn(0x7fa76c0b4650 0x7fa76c06c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:21.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.493+0000 7fa771400640 1 -- 172.21.15.159:0/3926650192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa76c158580 msgr2=0x7fa76c1589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:21.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.493+0000 7fa771400640 1 --2- 172.21.15.159:0/3926650192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa76c158580 0x7fa76c1589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:21.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.493+0000 7fa771400640 1 -- 172.21.15.159:0/3926650192 >> v2:172.21.15.178:3300/0 conn(0x7fa76c0b4650 msgr2=0x7fa76c06c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:21.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.493+0000 7fa771400640 1 --2- 172.21.15.159:0/3926650192 >> v2:172.21.15.178:3300/0 conn(0x7fa76c0b4650 0x7fa76c06c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:21.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.493+0000 7fa771400640 1 -- 172.21.15.159:0/3926650192 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76c142060 con 0x7fa76c0b4a30 2024-04-04T06:07:21.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.494+0000 7fa771400640 1 --2- 172.21.15.159:0/3926650192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa76c0b4a30 0x7fa76c0b5c10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0d32b0 tx=0x7fa75c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e654a1bfe4b669 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:21.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.495+0000 7fa770a00640 1 -- 172.21.15.159:0/3926650192 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c035020 con 0x7fa76c0b4a30 2024-04-04T06:07:21.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.495+0000 7fa770a00640 1 -- 172.21.15.159:0/3926650192 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa75c02d040 con 0x7fa76c0b4a30 2024-04-04T06:07:21.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.495+0000 7fa770a00640 1 -- 172.21.15.159:0/3926650192 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c031a80 con 0x7fa76c0b4a30 2024-04-04T06:07:21.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.496+0000 7fa774800640 1 -- 172.21.15.159:0/3926650192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa76c0b4a30 msgr2=0x7fa76c0b5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:21.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.496+0000 7fa774800640 1 --2- 172.21.15.159:0/3926650192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa76c0b4a30 0x7fa76c0b5c10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0d32b0 tx=0x7fa75c02b900 comp rx=0 tx=0).stop 2024-04-04T06:07:21.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.496+0000 7fa774800640 1 -- 172.21.15.159:0/3926650192 shutdown_connections 2024-04-04T06:07:21.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.496+0000 7fa774800640 1 --2- 172.21.15.159:0/3926650192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa76c158580 0x7fa76c1589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:21.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.496+0000 7fa774800640 1 --2- 172.21.15.159:0/3926650192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa76c0b4a30 0x7fa76c0b5c10 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:21.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.496+0000 7fa774800640 1 --2- 172.21.15.159:0/3926650192 >> v2:172.21.15.178:3300/0 conn(0x7fa76c0b4650 0x7fa76c06c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:21.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.496+0000 7fa774800640 1 -- 172.21.15.159:0/3926650192 >> 172.21.15.159:0/3926650192 conn(0x7fa76c067670 msgr2=0x7fa76c067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:21.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.496+0000 7fa774800640 1 -- 172.21.15.159:0/3926650192 shutdown_connections 2024-04-04T06:07:21.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.496+0000 7fa774800640 1 -- 172.21.15.159:0/3926650192 wait complete. 2024-04-04T06:07:21.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.498+0000 7fa774800640 1 Processor -- start 2024-04-04T06:07:21.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.498+0000 7fa774800640 1 -- start start 2024-04-04T06:07:21.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.499+0000 7fa774800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa76c0b4650 0x7fa76c15bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:21.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.499+0000 7fa774800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa76c0b4a30 0x7fa76c0cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:21.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.499+0000 7fa774800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa76c158580 0x7fa76c0cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:21.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.499+0000 7fa774800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c142900 con 0x7fa76c158580 2024-04-04T06:07:21.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.499+0000 7fa774800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c142620 con 0x7fa76c0b4a30 2024-04-04T06:07:21.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.499+0000 7fa774800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa76c158f30 con 0x7fa76c0b4650 2024-04-04T06:07:21.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.499+0000 7fa772800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa76c158580 0x7fa76c0cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:21.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.499+0000 7fa772800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa76c158580 0x7fa76c0cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53362/0 (socket says 172.21.15.159:53362) 2024-04-04T06:07:21.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.499+0000 7fa772800640 1 -- 172.21.15.159:0/3019545025 learned_addr learned my addr 172.21.15.159:0/3019545025 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:21.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.499+0000 7fa771400640 1 --2- 172.21.15.159:0/3019545025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa76c0b4a30 0x7fa76c0cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:21.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.499+0000 7fa771e00640 1 --2- 172.21.15.159:0/3019545025 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa76c0b4650 0x7fa76c15bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:21.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.499+0000 7fa772800640 1 -- 172.21.15.159:0/3019545025 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa76c0b4650 msgr2=0x7fa76c15bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:21.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.499+0000 7fa772800640 1 --2- 172.21.15.159:0/3019545025 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa76c0b4650 0x7fa76c15bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:21.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.499+0000 7fa772800640 1 -- 172.21.15.159:0/3019545025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa76c0b4a30 msgr2=0x7fa76c0cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:21.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.499+0000 7fa772800640 1 --2- 172.21.15.159:0/3019545025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa76c0b4a30 0x7fa76c0cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:21.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.499+0000 7fa772800640 1 -- 172.21.15.159:0/3019545025 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76c161070 con 0x7fa76c158580 2024-04-04T06:07:21.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.499+0000 7fa771400640 1 --2- 172.21.15.159:0/3019545025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa76c0b4a30 0x7fa76c0cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:21.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.499+0000 7fa771e00640 1 --2- 172.21.15.159:0/3019545025 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa76c0b4650 0x7fa76c15bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:21.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.500+0000 7fa772800640 1 --2- 172.21.15.159:0/3019545025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa76c158580 0x7fa76c0cb580 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa76800d800 tx=0x7fa76800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:21.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.500+0000 7fa762000640 1 -- 172.21.15.159:0/3019545025 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76800b6c0 con 0x7fa76c158580 2024-04-04T06:07:21.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.500+0000 7fa762000640 1 -- 172.21.15.159:0/3019545025 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa768019070 con 0x7fa76c158580 2024-04-04T06:07:21.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.500+0000 7fa762000640 1 -- 172.21.15.159:0/3019545025 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76800bac0 con 0x7fa76c158580 2024-04-04T06:07:21.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.500+0000 7fa774800640 1 -- 172.21.15.159:0/3019545025 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76c161350 con 0x7fa76c158580 2024-04-04T06:07:21.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.500+0000 7fa774800640 1 -- 172.21.15.159:0/3019545025 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa76c15c0f0 con 0x7fa76c158580 2024-04-04T06:07:21.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.501+0000 7fa774800640 1 -- 172.21.15.159:0/3019545025 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa76c0685c0 con 0x7fa76c158580 2024-04-04T06:07:21.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.502+0000 7fa762000640 1 -- 172.21.15.159:0/3019545025 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa76800bc60 con 0x7fa76c158580 2024-04-04T06:07:21.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.503+0000 7fa762000640 1 --2- 172.21.15.159:0/3019545025 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa75006c8f0 0x7fa75006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:21.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.503+0000 7fa762000640 1 -- 172.21.15.159:0/3019545025 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fa76808bfe0 con 0x7fa76c158580 2024-04-04T06:07:21.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.504+0000 7fa771e00640 1 --2- 172.21.15.159:0/3019545025 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa75006c8f0 0x7fa75006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:21.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.505+0000 7fa771e00640 1 --2- 172.21.15.159:0/3019545025 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa75006c8f0 0x7fa75006edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa764009f50 tx=0x7fa76400b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:21.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.508+0000 7fa762000640 1 -- 172.21.15.159:0/3019545025 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa76805a390 con 0x7fa76c158580 2024-04-04T06:07:21.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.718+0000 7fa774800640 1 -- 172.21.15.159:0/3019545025 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa76c1589f0 con 0x7fa76c158580 2024-04-04T06:07:21.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.719+0000 7fa762000640 1 -- 172.21.15.159:0/3019545025 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) ==== 80+0+0 (secure 0 0 0) 0x7fa768059c90 con 0x7fa76c158580 2024-04-04T06:07:21.713 INFO:teuthology.orchestra.run.smithi159.stderr:listed 0 entries 2024-04-04T06:07:21.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.723+0000 7fa774800640 1 -- 172.21.15.159:0/3019545025 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa75006c8f0 msgr2=0x7fa75006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:21.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.723+0000 7fa774800640 1 --2- 172.21.15.159:0/3019545025 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa75006c8f0 0x7fa75006edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa764009f50 tx=0x7fa76400b040 comp rx=0 tx=0).stop 2024-04-04T06:07:21.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.723+0000 7fa774800640 1 -- 172.21.15.159:0/3019545025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa76c158580 msgr2=0x7fa76c0cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:21.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.723+0000 7fa774800640 1 --2- 172.21.15.159:0/3019545025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa76c158580 0x7fa76c0cb580 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa76800d800 tx=0x7fa76800dcc0 comp rx=0 tx=0).stop 2024-04-04T06:07:21.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.723+0000 7fa774800640 1 -- 172.21.15.159:0/3019545025 shutdown_connections 2024-04-04T06:07:21.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.723+0000 7fa774800640 1 --2- 172.21.15.159:0/3019545025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa76c158580 0x7fa76c0cb580 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:21.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.723+0000 7fa774800640 1 --2- 172.21.15.159:0/3019545025 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa75006c8f0 0x7fa75006edb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:21.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.723+0000 7fa774800640 1 --2- 172.21.15.159:0/3019545025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa76c0b4a30 0x7fa76c0cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:21.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.723+0000 7fa774800640 1 --2- 172.21.15.159:0/3019545025 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa76c0b4650 0x7fa76c15bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:21.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.723+0000 7fa774800640 1 -- 172.21.15.159:0/3019545025 >> 172.21.15.159:0/3019545025 conn(0x7fa76c067670 msgr2=0x7fa76c0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:21.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.723+0000 7fa774800640 1 -- 172.21.15.159:0/3019545025 shutdown_connections 2024-04-04T06:07:21.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:21.723+0000 7fa774800640 1 -- 172.21.15.159:0/3019545025 wait complete. 2024-04-04T06:07:21.736 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-04T06:07:21.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:07:21.736 DEBUG:teuthology.orchestra.run.smithi159:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-04T06:07:21.877 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:07:21.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:07:21.878 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-04T06:07:21.924 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:07:22.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.155+0000 7f11c3e00640 1 Processor -- start 2024-04-04T06:07:22.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.155+0000 7f11c3e00640 1 -- start start 2024-04-04T06:07:22.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.156+0000 7f11c3e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f11c40cc0b0 0x7f11c406dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:22.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.156+0000 7f11c3e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11c40d04d0 0x7f11c415ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:22.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.156+0000 7f11c3e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11c406e4f0 con 0x7f11c415fc50 2024-04-04T06:07:22.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.156+0000 7f11c3e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f11c415c610 con 0x7f11c40cc0b0 2024-04-04T06:07:22.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.156+0000 7f11c3e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f11c40b3150 con 0x7f11c40d04d0 2024-04-04T06:07:22.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.156+0000 7f11ba000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f11c415fc50 0x7f11c40cfdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:49228/0 (socket says 172.21.15.159:49228) 2024-04-04T06:07:22.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.156+0000 7f11ba000640 1 -- 172.21.15.159:0/4274105381 learned_addr learned my addr 172.21.15.159:0/4274105381 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:22.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.157+0000 7f11c3400640 1 --2- 172.21.15.159:0/4274105381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11c40d04d0 0x7f11c415ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:22.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.157+0000 7f11c2a00640 1 --2- 172.21.15.159:0/4274105381 >> v2:172.21.15.178:3300/0 conn(0x7f11c40cc0b0 0x7f11c406dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:22.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.157+0000 7f11c2000640 1 -- 172.21.15.159:0/4274105381 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3976203639 0 0) 0x7f11c406e4f0 con 0x7f11c415fc50 2024-04-04T06:07:22.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.157+0000 7f11c2000640 1 -- 172.21.15.159:0/4274105381 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11a8003520 con 0x7f11c415fc50 2024-04-04T06:07:22.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.157+0000 7f11c2000640 1 -- 172.21.15.159:0/4274105381 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2378637866 0 0) 0x7f11a8003520 con 0x7f11c415fc50 2024-04-04T06:07:22.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.157+0000 7f11c2000640 1 -- 172.21.15.159:0/4274105381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11c40d04d0 msgr2=0x7f11c415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:22.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.157+0000 7f11c2000640 1 --2- 172.21.15.159:0/4274105381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11c40d04d0 0x7f11c415ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:22.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.157+0000 7f11c2000640 1 -- 172.21.15.159:0/4274105381 >> v2:172.21.15.178:3300/0 conn(0x7f11c40cc0b0 msgr2=0x7f11c406dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:22.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.157+0000 7f11c2000640 1 --2- 172.21.15.159:0/4274105381 >> v2:172.21.15.178:3300/0 conn(0x7f11c40cc0b0 0x7f11c406dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:22.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.157+0000 7f11c2000640 1 -- 172.21.15.159:0/4274105381 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c41600e0 con 0x7f11c415fc50 2024-04-04T06:07:22.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.157+0000 7f11c2000640 1 -- 172.21.15.159:0/4274105381 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f11ac001670 con 0x7f11c415fc50 2024-04-04T06:07:22.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.157+0000 7f11c2000640 1 -- 172.21.15.159:0/4274105381 >> v1:172.21.15.159:6789/0 conn(0x7f11c415fc50 legacy=0x7f11c40cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:22.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.157+0000 7f11c2000640 1 --2- 172.21.15.159:0/4274105381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11a80039b0 0x7f11a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:22.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.157+0000 7f11c2000640 1 --2- 172.21.15.159:0/4274105381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11c40d04d0 0x7f11a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:22.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.157+0000 7f11c2000640 1 --2- 172.21.15.159:0/4274105381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a8004d40 0x7f11a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:22.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.157+0000 7f11c2000640 1 -- 172.21.15.159:0/4274105381 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f11c415c610 con 0x7f11a80039b0 2024-04-04T06:07:22.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.157+0000 7f11c2000640 1 -- 172.21.15.159:0/4274105381 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f11c40b3150 con 0x7f11a8004d40 2024-04-04T06:07:22.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.157+0000 7f11c2000640 1 -- 172.21.15.159:0/4274105381 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f11a800b9c0 con 0x7f11c40d04d0 2024-04-04T06:07:22.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.158+0000 7f11c2a00640 1 --2- 172.21.15.159:0/4274105381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11a80039b0 0x7f11a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:22.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.158+0000 7f11c3400640 1 --2- 172.21.15.159:0/4274105381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a8004d40 0x7f11a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:22.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.158+0000 7f11ba000640 1 --2- 172.21.15.159:0/4274105381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11c40d04d0 0x7f11a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:22.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.158+0000 7f11c2a00640 1 -- 172.21.15.159:0/4274105381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11c40d04d0 msgr2=0x7f11a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:22.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.158+0000 7f11c2a00640 1 --2- 172.21.15.159:0/4274105381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11c40d04d0 0x7f11a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:22.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.158+0000 7f11c2a00640 1 -- 172.21.15.159:0/4274105381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a8004d40 msgr2=0x7f11a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:22.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.158+0000 7f11c2a00640 1 --2- 172.21.15.159:0/4274105381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a8004d40 0x7f11a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:22.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.158+0000 7f11c2a00640 1 -- 172.21.15.159:0/4274105381 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a8003520 con 0x7f11a80039b0 2024-04-04T06:07:22.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.158+0000 7f11c3400640 1 --2- 172.21.15.159:0/4274105381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a8004d40 0x7f11a80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:22.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.158+0000 7f11ba000640 1 --2- 172.21.15.159:0/4274105381 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11c40d04d0 0x7f11a8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:22.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.159+0000 7f11c2a00640 1 --2- 172.21.15.159:0/4274105381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11a80039b0 0x7f11a8003dd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009af0 tx=0x7f11a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead83c510962a571 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:22.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.159+0000 7f11c2000640 1 -- 172.21.15.159:0/4274105381 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4039070 con 0x7f11a80039b0 2024-04-04T06:07:22.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.159+0000 7f11c2000640 1 -- 172.21.15.159:0/4274105381 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11b402c710 con 0x7f11a80039b0 2024-04-04T06:07:22.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.159+0000 7f11c2000640 1 -- 172.21.15.159:0/4274105381 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4009690 con 0x7f11a80039b0 2024-04-04T06:07:22.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.160+0000 7f11c3e00640 1 -- 172.21.15.159:0/4274105381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11a80039b0 msgr2=0x7f11a8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:22.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.160+0000 7f11c3e00640 1 --2- 172.21.15.159:0/4274105381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11a80039b0 0x7f11a8003dd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009af0 tx=0x7f11a8005830 comp rx=0 tx=0).stop 2024-04-04T06:07:22.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.160+0000 7f11c3400640 1 -- 172.21.15.159:0/4274105381 reap_dead start 2024-04-04T06:07:22.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.160+0000 7f11c3e00640 1 -- 172.21.15.159:0/4274105381 shutdown_connections 2024-04-04T06:07:22.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.160+0000 7f11c3e00640 1 -- 172.21.15.159:0/4274105381 >> 172.21.15.159:0/4274105381 conn(0x7f11c40ccad0 msgr2=0x7f11c40cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:22.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.160+0000 7f11c3e00640 1 -- 172.21.15.159:0/4274105381 shutdown_connections 2024-04-04T06:07:22.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.160+0000 7f11c3e00640 1 -- 172.21.15.159:0/4274105381 wait complete. 2024-04-04T06:07:22.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.162+0000 7f11c3e00640 1 Processor -- start 2024-04-04T06:07:22.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.162+0000 7f11c3e00640 1 -- start start 2024-04-04T06:07:22.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.163+0000 7f11c3e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a80039b0 0x7f11c41583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:22.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.163+0000 7f11c3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11c413e4e0 0x7f11c41f3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:22.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.163+0000 7f11c3e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11c40b1810 0x7f11c40b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:22.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.163+0000 7f11c3e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f11c4160980 con 0x7f11c413e4e0 2024-04-04T06:07:22.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.163+0000 7f11c3e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f11c41606a0 con 0x7f11a80039b0 2024-04-04T06:07:22.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.163+0000 7f11c3e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f11c40cc490 con 0x7f11c40b1810 2024-04-04T06:07:22.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.163+0000 7f11ba000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11c413e4e0 0x7f11c41f3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:22.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.163+0000 7f11ba000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11c413e4e0 0x7f11c41f3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53370/0 (socket says 172.21.15.159:53370) 2024-04-04T06:07:22.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.163+0000 7f11ba000640 1 -- 172.21.15.159:0/369260713 learned_addr learned my addr 172.21.15.159:0/369260713 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:22.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.164+0000 7f11c2a00640 1 --2- 172.21.15.159:0/369260713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a80039b0 0x7f11c41583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:22.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.164+0000 7f11c3400640 1 --2- 172.21.15.159:0/369260713 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11c40b1810 0x7f11c40b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:22.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.164+0000 7f11ba000640 1 -- 172.21.15.159:0/369260713 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11c40b1810 msgr2=0x7f11c40b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:22.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.164+0000 7f11ba000640 1 --2- 172.21.15.159:0/369260713 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11c40b1810 0x7f11c40b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:22.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.164+0000 7f11ba000640 1 -- 172.21.15.159:0/369260713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a80039b0 msgr2=0x7f11c41583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:22.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.164+0000 7f11ba000640 1 --2- 172.21.15.159:0/369260713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a80039b0 0x7f11c41583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:22.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.164+0000 7f11ba000640 1 -- 172.21.15.159:0/369260713 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c415a020 con 0x7f11c413e4e0 2024-04-04T06:07:22.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.164+0000 7f11ba000640 1 --2- 172.21.15.159:0/369260713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11c413e4e0 0x7f11c41f3c10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00d810 tx=0x7f11ac00dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:22.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.164+0000 7f11bb400640 1 -- 172.21.15.159:0/369260713 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac019070 con 0x7f11c413e4e0 2024-04-04T06:07:22.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.164+0000 7f11bb400640 1 -- 172.21.15.159:0/369260713 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ac0074e0 con 0x7f11c413e4e0 2024-04-04T06:07:22.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.164+0000 7f11bb400640 1 -- 172.21.15.159:0/369260713 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac0049e0 con 0x7f11c413e4e0 2024-04-04T06:07:22.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.164+0000 7f11c3e00640 1 -- 172.21.15.159:0/369260713 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c415a300 con 0x7f11c413e4e0 2024-04-04T06:07:22.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.164+0000 7f11c3e00640 1 -- 172.21.15.159:0/369260713 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c415a8c0 con 0x7f11c413e4e0 2024-04-04T06:07:22.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.166+0000 7f11c3e00640 1 -- 172.21.15.159:0/369260713 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11c41562e0 con 0x7f11c413e4e0 2024-04-04T06:07:22.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.167+0000 7f11bb400640 1 -- 172.21.15.159:0/369260713 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f11ac01c070 con 0x7f11c413e4e0 2024-04-04T06:07:22.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.167+0000 7f11bb400640 1 --2- 172.21.15.159:0/369260713 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f11a006c920 0x7f11a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:22.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.168+0000 7f11bb400640 1 -- 172.21.15.159:0/369260713 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f11ac08b600 con 0x7f11c413e4e0 2024-04-04T06:07:22.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.168+0000 7f11c2a00640 1 --2- 172.21.15.159:0/369260713 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f11a006c920 0x7f11a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:22.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.169+0000 7f11c2a00640 1 --2- 172.21.15.159:0/369260713 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f11a006c920 0x7f11a006ede0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f11b4036000 tx=0x7f11b4030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:22.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.172+0000 7f11bb400640 1 -- 172.21.15.159:0/369260713 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f11ac059a50 con 0x7f11c413e4e0 2024-04-04T06:07:22.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.406+0000 7f11c3e00640 1 -- 172.21.15.159:0/369260713 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11c4075940 con 0x7f11c413e4e0 2024-04-04T06:07:22.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.407+0000 7f11bb400640 1 -- 172.21.15.159:0/369260713 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5193 (secure 0 0 0) 0x7f11ac059350 con 0x7f11c413e4e0 2024-04-04T06:07:22.401 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:22.401 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1643905381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1643905381},{"type":"v1","addr":"172.21.15.178:6837","nonce":1643905381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4744,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2629539137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2629539137},{"type":"v1","addr":"172.21.15.159:6835","nonce":2629539137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3402172592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3402172592},{"type":"v1","addr":"172.21.15.159:6833","nonce":3402172592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:07:08.839645+0000","modified":"2024-04-04T06:07:11.927601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5157},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5157":{"gid":5157,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.178:6836/3282963883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3282963883},{"type":"v1","addr":"172.21.15.178:6836","nonce":3282963883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5157,"qdb_cluster":[5157]},"id":2}]} 2024-04-04T06:07:22.401 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 17 2024-04-04T06:07:22.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.410+0000 7f11c3e00640 1 -- 172.21.15.159:0/369260713 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f11a006c920 msgr2=0x7f11a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:22.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.410+0000 7f11c3e00640 1 --2- 172.21.15.159:0/369260713 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f11a006c920 0x7f11a006ede0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f11b4036000 tx=0x7f11b4030000 comp rx=0 tx=0).stop 2024-04-04T06:07:22.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.410+0000 7f11c3e00640 1 -- 172.21.15.159:0/369260713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11c413e4e0 msgr2=0x7f11c41f3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:22.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.410+0000 7f11c3e00640 1 --2- 172.21.15.159:0/369260713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11c413e4e0 0x7f11c41f3c10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00d810 tx=0x7f11ac00dcd0 comp rx=0 tx=0).stop 2024-04-04T06:07:22.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.410+0000 7f11c3e00640 1 -- 172.21.15.159:0/369260713 shutdown_connections 2024-04-04T06:07:22.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.410+0000 7f11c3e00640 1 --2- 172.21.15.159:0/369260713 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11c40b1810 0x7f11c40b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:22.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.410+0000 7f11c3e00640 1 --2- 172.21.15.159:0/369260713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11c413e4e0 0x7f11c41f3c10 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:22.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.410+0000 7f11c3e00640 1 --2- 172.21.15.159:0/369260713 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f11a006c920 0x7f11a006ede0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:22.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.410+0000 7f11c3e00640 1 --2- 172.21.15.159:0/369260713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a80039b0 0x7f11c41583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:22.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.410+0000 7f11c3e00640 1 -- 172.21.15.159:0/369260713 >> 172.21.15.159:0/369260713 conn(0x7f11c40ccad0 msgr2=0x7f11c40724e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:22.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.410+0000 7f11c3e00640 1 -- 172.21.15.159:0/369260713 shutdown_connections 2024-04-04T06:07:22.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.410+0000 7f11c3e00640 1 -- 172.21.15.159:0/369260713 wait complete. 2024-04-04T06:07:22.422 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-04T06:07:22.657 INFO:teuthology.orchestra.run.smithi178.stdout:{"message":"","return_code":0} 2024-04-04T06:07:22.658 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-04T06:07:22.658 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:07:22.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.874+0000 7f43bb800640 1 Processor -- start 2024-04-04T06:07:22.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.874+0000 7f43bb800640 1 -- start start 2024-04-04T06:07:22.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.876+0000 7f43bb800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43b40727f0 0x7f43b40b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:22.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.876+0000 7f43bb800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43b415b6f0 con 0x7f43b40d0780 2024-04-04T06:07:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.876+0000 7f43bb800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43b4072e80 con 0x7f43b415b310 2024-04-04T06:07:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.876+0000 7f43bb800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f43b4072c50 con 0x7f43b40727f0 2024-04-04T06:07:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.876+0000 7f43b9800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f43b40d0780 0x7f43b40adc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:49236/0 (socket says 172.21.15.159:49236) 2024-04-04T06:07:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.876+0000 7f43b9800640 1 -- 172.21.15.159:0/3991755900 learned_addr learned my addr 172.21.15.159:0/3991755900 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.876+0000 7f43abe00640 1 --2- 172.21.15.159:0/3991755900 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43b40727f0 0x7f43b40b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:22.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.877+0000 7f43b3e00640 1 -- 172.21.15.159:0/3991755900 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1761232379 0 0) 0x7f43b415b6f0 con 0x7f43b40d0780 2024-04-04T06:07:22.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.877+0000 7f43b3e00640 1 -- 172.21.15.159:0/3991755900 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4398003520 con 0x7f43b40d0780 2024-04-04T06:07:22.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.877+0000 7f43b3e00640 1 -- 172.21.15.159:0/3991755900 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2647706200 0 0) 0x7f4398003520 con 0x7f43b40d0780 2024-04-04T06:07:22.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.877+0000 7f43b3e00640 1 -- 172.21.15.159:0/3991755900 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43b40727f0 msgr2=0x7f43b40b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:22.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.877+0000 7f43b3e00640 1 --2- 172.21.15.159:0/3991755900 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43b40727f0 0x7f43b40b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:22.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.877+0000 7f43b3e00640 1 -- 172.21.15.159:0/3991755900 >> v1:172.21.15.178:6789/0 conn(0x7f43b415b310 legacy=0x7f43b415f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:22.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.877+0000 7f43b3e00640 1 -- 172.21.15.159:0/3991755900 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b4073160 con 0x7f43b40d0780 2024-04-04T06:07:22.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.877+0000 7f43b3e00640 1 -- 172.21.15.159:0/3991755900 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f43ac0019f0 con 0x7f43b40d0780 2024-04-04T06:07:22.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.877+0000 7f43b3e00640 1 -- 172.21.15.159:0/3991755900 >> v1:172.21.15.159:6789/0 conn(0x7f43b40d0780 legacy=0x7f43b40adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:22.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.877+0000 7f43b3e00640 1 --2- 172.21.15.159:0/3991755900 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43b40727f0 0x7f43980039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:22.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.877+0000 7f43b3e00640 1 --2- 172.21.15.159:0/3991755900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4398003fb0 0x7f4398004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:22.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.877+0000 7f43b3e00640 1 --2- 172.21.15.159:0/3991755900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4398004d40 0x7f43980051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:22.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.877+0000 7f43b3e00640 1 -- 172.21.15.159:0/3991755900 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f439800bc20 con 0x7f4398003fb0 2024-04-04T06:07:22.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.877+0000 7f43b3e00640 1 -- 172.21.15.159:0/3991755900 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f439800dfe0 con 0x7f4398004d40 2024-04-04T06:07:22.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.877+0000 7f43b3e00640 1 -- 172.21.15.159:0/3991755900 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f439800e1e0 con 0x7f43b40727f0 2024-04-04T06:07:22.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.878+0000 7f43abe00640 1 --2- 172.21.15.159:0/3991755900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4398003fb0 0x7f4398004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:22.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.878+0000 7f43b8e00640 1 --2- 172.21.15.159:0/3991755900 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43b40727f0 0x7f43980039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:22.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.878+0000 7f43abe00640 1 -- 172.21.15.159:0/3991755900 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43b40727f0 msgr2=0x7f43980039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:22.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.878+0000 7f43abe00640 1 --2- 172.21.15.159:0/3991755900 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43b40727f0 0x7f43980039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:22.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.878+0000 7f43abe00640 1 -- 172.21.15.159:0/3991755900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4398004d40 msgr2=0x7f43980051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:22.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.878+0000 7f43abe00640 1 --2- 172.21.15.159:0/3991755900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4398004d40 0x7f43980051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:22.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.878+0000 7f43abe00640 1 -- 172.21.15.159:0/3991755900 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4398003520 con 0x7f4398003fb0 2024-04-04T06:07:22.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.879+0000 7f43abe00640 1 --2- 172.21.15.159:0/3991755900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4398003fb0 0x7f4398004800 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f439c009ac0 tx=0x7f4398005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4845578c5fd984d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:22.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.879+0000 7f43b3e00640 1 -- 172.21.15.159:0/3991755900 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c039070 con 0x7f4398003fb0 2024-04-04T06:07:22.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.879+0000 7f43b3e00640 1 -- 172.21.15.159:0/3991755900 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439c02c6b0 con 0x7f4398003fb0 2024-04-04T06:07:22.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.879+0000 7f43b3e00640 1 -- 172.21.15.159:0/3991755900 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c009690 con 0x7f4398003fb0 2024-04-04T06:07:22.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.880+0000 7f43bb800640 1 -- 172.21.15.159:0/3991755900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4398003fb0 msgr2=0x7f4398004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:22.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.880+0000 7f43bb800640 1 --2- 172.21.15.159:0/3991755900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4398003fb0 0x7f4398004800 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f439c009ac0 tx=0x7f4398005830 comp rx=0 tx=0).stop 2024-04-04T06:07:22.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.880+0000 7f43b9800640 1 -- 172.21.15.159:0/3991755900 reap_dead start 2024-04-04T06:07:22.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.880+0000 7f43bb800640 1 -- 172.21.15.159:0/3991755900 shutdown_connections 2024-04-04T06:07:22.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.880+0000 7f43bb800640 1 -- 172.21.15.159:0/3991755900 >> 172.21.15.159:0/3991755900 conn(0x7f43b406b990 msgr2=0x7f43b406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:22.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.880+0000 7f43bb800640 1 -- 172.21.15.159:0/3991755900 shutdown_connections 2024-04-04T06:07:22.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.880+0000 7f43bb800640 1 -- 172.21.15.159:0/3991755900 wait complete. 2024-04-04T06:07:22.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.882+0000 7f43bb800640 1 Processor -- start 2024-04-04T06:07:22.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.882+0000 7f43bb800640 1 -- start start 2024-04-04T06:07:22.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.883+0000 7f43bb800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4398003fb0 0x7f43b41e7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:22.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.883+0000 7f43bb800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43b4147210 0x7f43b4147690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:22.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.883+0000 7f43bb800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43b41f4750 0x7f43b41f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:22.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.883+0000 7f43bb800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f43b4073440 con 0x7f43b41f4750 2024-04-04T06:07:22.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.883+0000 7f43bb800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43b415f2c0 con 0x7f4398003fb0 2024-04-04T06:07:22.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.883+0000 7f43bb800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f43b40ae390 con 0x7f43b4147210 2024-04-04T06:07:22.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.883+0000 7f43b9800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43b41f4750 0x7f43b41f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:22.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.883+0000 7f43b9800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43b41f4750 0x7f43b41f4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53382/0 (socket says 172.21.15.159:53382) 2024-04-04T06:07:22.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.883+0000 7f43b9800640 1 -- 172.21.15.159:0/1308558442 learned_addr learned my addr 172.21.15.159:0/1308558442 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:22.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.883+0000 7f43b8e00640 1 --2- 172.21.15.159:0/1308558442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4398003fb0 0x7f43b41e7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:22.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.883+0000 7f43abe00640 1 --2- 172.21.15.159:0/1308558442 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43b4147210 0x7f43b4147690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:22.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.883+0000 7f43b9800640 1 -- 172.21.15.159:0/1308558442 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43b4147210 msgr2=0x7f43b4147690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:22.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.883+0000 7f43b9800640 1 --2- 172.21.15.159:0/1308558442 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43b4147210 0x7f43b4147690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:22.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.883+0000 7f43b9800640 1 -- 172.21.15.159:0/1308558442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4398003fb0 msgr2=0x7f43b41e7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:22.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.883+0000 7f43b9800640 1 --2- 172.21.15.159:0/1308558442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4398003fb0 0x7f43b41e7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:22.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.883+0000 7f43b9800640 1 -- 172.21.15.159:0/1308558442 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b40b0f80 con 0x7f43b41f4750 2024-04-04T06:07:22.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.884+0000 7f43b9800640 1 --2- 172.21.15.159:0/1308558442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43b41f4750 0x7f43b41f4be0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f43b40b2e80 tx=0x7f43ac00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:22.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.884+0000 7f43b1600640 1 -- 172.21.15.159:0/1308558442 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac00a610 con 0x7f43b41f4750 2024-04-04T06:07:22.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.884+0000 7f43b1600640 1 -- 172.21.15.159:0/1308558442 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43ac00a7b0 con 0x7f43b41f4750 2024-04-04T06:07:22.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.884+0000 7f43bb800640 1 -- 172.21.15.159:0/1308558442 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b41f5530 con 0x7f43b41f4750 2024-04-04T06:07:22.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.884+0000 7f43b1600640 1 -- 172.21.15.159:0/1308558442 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac00aba0 con 0x7f43b41f4750 2024-04-04T06:07:22.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.884+0000 7f43bb800640 1 -- 172.21.15.159:0/1308558442 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b40b1520 con 0x7f43b41f4750 2024-04-04T06:07:22.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.885+0000 7f43bb800640 1 -- 172.21.15.159:0/1308558442 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43b40685c0 con 0x7f43b41f4750 2024-04-04T06:07:22.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.886+0000 7f43b1600640 1 -- 172.21.15.159:0/1308558442 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f43ac011070 con 0x7f43b41f4750 2024-04-04T06:07:22.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.887+0000 7f43b1600640 1 --2- 172.21.15.159:0/1308558442 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f439006c8f0 0x7f439006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:22.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.887+0000 7f43b1600640 1 -- 172.21.15.159:0/1308558442 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f43ac08b4b0 con 0x7f43b41f4750 2024-04-04T06:07:22.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.888+0000 7f43b8e00640 1 --2- 172.21.15.159:0/1308558442 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f439006c8f0 0x7f439006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:22.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.889+0000 7f43b8e00640 1 --2- 172.21.15.159:0/1308558442 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f439006c8f0 0x7f439006edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f43b415a9b0 tx=0x7f43a4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:22.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:22.892+0000 7f43b1600640 1 -- 172.21.15.159:0/1308558442 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f43ac01a060 con 0x7f43b41f4750 2024-04-04T06:07:23.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:23.127+0000 7f43bb800640 1 -- 172.21.15.159:0/1308558442 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43b40b16f0 con 0x7f43b41f4750 2024-04-04T06:07:23.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:23.128+0000 7f43b1600640 1 -- 172.21.15.159:0/1308558442 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5193 (secure 0 0 0) 0x7f43ac059840 con 0x7f43b41f4750 2024-04-04T06:07:23.121 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:23.122 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1643905381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1643905381},{"type":"v1","addr":"172.21.15.178:6837","nonce":1643905381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4744,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2629539137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2629539137},{"type":"v1","addr":"172.21.15.159:6835","nonce":2629539137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3402172592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3402172592},{"type":"v1","addr":"172.21.15.159:6833","nonce":3402172592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:07:08.839645+0000","modified":"2024-04-04T06:07:11.927601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5157},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5157":{"gid":5157,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.178:6836/3282963883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3282963883},{"type":"v1","addr":"172.21.15.178:6836","nonce":3282963883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5157,"qdb_cluster":[5157]},"id":2}]} 2024-04-04T06:07:23.122 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 17 2024-04-04T06:07:23.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:23.130+0000 7f43bb800640 1 -- 172.21.15.159:0/1308558442 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f439006c8f0 msgr2=0x7f439006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:23.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:23.130+0000 7f43bb800640 1 --2- 172.21.15.159:0/1308558442 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f439006c8f0 0x7f439006edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f43b415a9b0 tx=0x7f43a4009040 comp rx=0 tx=0).stop 2024-04-04T06:07:23.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:23.130+0000 7f43bb800640 1 -- 172.21.15.159:0/1308558442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43b41f4750 msgr2=0x7f43b41f4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:23.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:23.130+0000 7f43bb800640 1 --2- 172.21.15.159:0/1308558442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43b41f4750 0x7f43b41f4be0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f43b40b2e80 tx=0x7f43ac00cd50 comp rx=0 tx=0).stop 2024-04-04T06:07:23.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:23.130+0000 7f43bb800640 1 -- 172.21.15.159:0/1308558442 shutdown_connections 2024-04-04T06:07:23.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:23.130+0000 7f43bb800640 1 --2- 172.21.15.159:0/1308558442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43b41f4750 0x7f43b41f4be0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:23.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:23.130+0000 7f43bb800640 1 --2- 172.21.15.159:0/1308558442 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43b4147210 0x7f43b4147690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:23.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:23.130+0000 7f43bb800640 1 --2- 172.21.15.159:0/1308558442 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f439006c8f0 0x7f439006edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:23.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:23.130+0000 7f43bb800640 1 --2- 172.21.15.159:0/1308558442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4398003fb0 0x7f43b41e7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:23.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:23.131+0000 7f43bb800640 1 -- 172.21.15.159:0/1308558442 >> 172.21.15.159:0/1308558442 conn(0x7f43b406b990 msgr2=0x7f43b4153820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:23.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:23.131+0000 7f43bb800640 1 -- 172.21.15.159:0/1308558442 shutdown_connections 2024-04-04T06:07:23.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:23.131+0000 7f43bb800640 1 -- 172.21.15.159:0/1308558442 wait complete. 2024-04-04T06:07:23.143 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json tag path /parentdir mytag 2024-04-04T06:07:23.384 INFO:teuthology.orchestra.run.smithi178.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-04-04T06:07:23.385 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-04-04T06:07:33.386 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-04-04T06:07:33.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.499+0000 7f5c1da4cf40 1 Processor -- start 2024-04-04T06:07:33.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.499+0000 7f5c1da4cf40 1 -- start start 2024-04-04T06:07:33.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.500+0000 7f5c1da4cf40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fb21480 0x55ff6faf0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.500+0000 7f5c1da4cf40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff6fbbc7d0 0x55ff6fb1c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.500+0000 7f5c1da4cf40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ff6fbbcbb0 0x55ff6fbb28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.500+0000 7f5c1da4cf40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ff6fa982c0 con 0x55ff6fbbc7d0 2024-04-04T06:07:33.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.500+0000 7f5c1da4cf40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ff6fa9c310 con 0x55ff6fb21480 2024-04-04T06:07:33.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.500+0000 7f5c1da4cf40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ff6fb206c0 con 0x55ff6fbbcbb0 2024-04-04T06:07:33.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.500+0000 7f5c19a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff6fbbc7d0 0x55ff6fb1c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.500+0000 7f5c19a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff6fbbc7d0 0x55ff6fb1c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48206/0 (socket says 172.21.15.159:48206) 2024-04-04T06:07:33.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.500+0000 7f5c19a00640 1 -- 172.21.15.159:0/742260568 learned_addr learned my addr 172.21.15.159:0/742260568 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:33.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.501+0000 7f5c1a400640 1 --2- 172.21.15.159:0/742260568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fb21480 0x55ff6faf0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.501+0000 7f5c19a00640 1 -- 172.21.15.159:0/742260568 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ff6fbbcbb0 msgr2=0x55ff6fbb28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:33.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.501+0000 7f5c19a00640 1 --2- 172.21.15.159:0/742260568 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ff6fbbcbb0 0x55ff6fbb28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.501+0000 7f5c19a00640 1 -- 172.21.15.159:0/742260568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fb21480 msgr2=0x55ff6faf0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:33.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.501+0000 7f5c19a00640 1 --2- 172.21.15.159:0/742260568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fb21480 0x55ff6faf0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.501+0000 7f5c19a00640 1 -- 172.21.15.159:0/742260568 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff6fbc72a0 con 0x55ff6fbbc7d0 2024-04-04T06:07:33.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.501+0000 7f5c19a00640 1 --2- 172.21.15.159:0/742260568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff6fbbc7d0 0x55ff6fb1c7a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5c04001e20 tx=0x7f5c0400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91ab8a98e3bdebd server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:33.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.501+0000 7f5c13e00640 1 -- 172.21.15.159:0/742260568 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0400be30 con 0x55ff6fbbc7d0 2024-04-04T06:07:33.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.501+0000 7f5c13e00640 1 -- 172.21.15.159:0/742260568 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c04006070 con 0x55ff6fbbc7d0 2024-04-04T06:07:33.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.501+0000 7f5c13e00640 1 -- 172.21.15.159:0/742260568 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0400e800 con 0x55ff6fbbc7d0 2024-04-04T06:07:33.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.502+0000 7f5c1da4cf40 1 -- 172.21.15.159:0/742260568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff6fbbc7d0 msgr2=0x55ff6fb1c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.502+0000 7f5c1da4cf40 1 --2- 172.21.15.159:0/742260568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff6fbbc7d0 0x55ff6fb1c7a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5c04001e20 tx=0x7f5c0400b6c0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.502+0000 7f5c1da4cf40 1 -- 172.21.15.159:0/742260568 shutdown_connections 2024-04-04T06:07:33.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.502+0000 7f5c1da4cf40 1 --2- 172.21.15.159:0/742260568 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ff6fbbcbb0 0x55ff6fbb28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.502+0000 7f5c1da4cf40 1 --2- 172.21.15.159:0/742260568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff6fbbc7d0 0x55ff6fb1c7a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.502+0000 7f5c1da4cf40 1 --2- 172.21.15.159:0/742260568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fb21480 0x55ff6faf0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.502+0000 7f5c1da4cf40 1 -- 172.21.15.159:0/742260568 >> 172.21.15.159:0/742260568 conn(0x55ff6faf08a0 msgr2=0x55ff6fb3af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:33.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.502+0000 7f5c1da4cf40 1 -- 172.21.15.159:0/742260568 shutdown_connections 2024-04-04T06:07:33.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.502+0000 7f5c1da4cf40 1 -- 172.21.15.159:0/742260568 wait complete. 2024-04-04T06:07:33.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.503+0000 7f5c1da4cf40 1 Processor -- start 2024-04-04T06:07:33.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.503+0000 7f5c1da4cf40 1 -- start start 2024-04-04T06:07:33.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.504+0000 7f5c1da4cf40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fb21480 0x55ff6fbbfde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.504+0000 7f5c1da4cf40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff6fbbc7d0 0x55ff6fbbcf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.504+0000 7f5c1da4cf40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ff6fbbcbb0 0x55ff6fb3af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.504+0000 7f5c1da4cf40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ff6fbc7580 con 0x55ff6fbbc7d0 2024-04-04T06:07:33.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.504+0000 7f5c1da4cf40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ff6fbc7050 con 0x55ff6fb21480 2024-04-04T06:07:33.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.504+0000 7f5c1da4cf40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ff6fb1d5c0 con 0x55ff6fbbcbb0 2024-04-04T06:07:33.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.504+0000 7f5c19a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff6fbbc7d0 0x55ff6fbbcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.504+0000 7f5c19a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff6fbbc7d0 0x55ff6fbbcf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48218/0 (socket says 172.21.15.159:48218) 2024-04-04T06:07:33.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.504+0000 7f5c19a00640 1 -- 172.21.15.159:0/4135356008 learned_addr learned my addr 172.21.15.159:0/4135356008 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:33.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.504+0000 7f5c1a400640 1 --2- 172.21.15.159:0/4135356008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fb21480 0x55ff6fbbfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.504+0000 7f5c1ae00640 1 --2- 172.21.15.159:0/4135356008 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ff6fbbcbb0 0x55ff6fb3af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.505+0000 7f5c19a00640 1 -- 172.21.15.159:0/4135356008 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ff6fbbcbb0 msgr2=0x55ff6fb3af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.505+0000 7f5c19a00640 1 --2- 172.21.15.159:0/4135356008 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ff6fbbcbb0 0x55ff6fb3af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.505+0000 7f5c19a00640 1 -- 172.21.15.159:0/4135356008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fb21480 msgr2=0x55ff6fbbfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.505+0000 7f5c19a00640 1 --2- 172.21.15.159:0/4135356008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ff6fb21480 0x55ff6fbbfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.505+0000 7f5c19a00640 1 -- 172.21.15.159:0/4135356008 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff6fbba3e0 con 0x55ff6fbbc7d0 2024-04-04T06:07:33.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.505+0000 7f5c19a00640 1 --2- 172.21.15.159:0/4135356008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff6fbbc7d0 0x55ff6fbbcf90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400b700 tx=0x7f5c04000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:33.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.505+0000 7f5c12a00640 1 -- 172.21.15.159:0/4135356008 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c040011f0 con 0x55ff6fbbc7d0 2024-04-04T06:07:33.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.505+0000 7f5c12a00640 1 -- 172.21.15.159:0/4135356008 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c04021020 con 0x55ff6fbbc7d0 2024-04-04T06:07:33.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.505+0000 7f5c1da4cf40 1 -- 172.21.15.159:0/4135356008 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ff6fbba6c0 con 0x55ff6fbbc7d0 2024-04-04T06:07:33.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.505+0000 7f5c12a00640 1 -- 172.21.15.159:0/4135356008 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0400f450 con 0x55ff6fbbc7d0 2024-04-04T06:07:33.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.505+0000 7f5c1da4cf40 1 -- 172.21.15.159:0/4135356008 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff6fbbac80 con 0x55ff6fbbc7d0 2024-04-04T06:07:33.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.508+0000 7f5c12a00640 1 -- 172.21.15.159:0/4135356008 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c04017070 con 0x55ff6fbbc7d0 2024-04-04T06:07:33.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.509+0000 7f5c12a00640 1 --2- 172.21.15.159:0/4135356008 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5bf806b2d0 0x7f5bf806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.509+0000 7f5c12a00640 1 -- 172.21.15.159:0/4135356008 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f5c0408c160 con 0x55ff6fbbc7d0 2024-04-04T06:07:33.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.509+0000 7f5c1da4cf40 1 --2- 172.21.15.159:0/4135356008 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55ff6fb3b470 0x55ff6fbbb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.509+0000 7f5c1da4cf40 1 -- 172.21.15.159:0/4135356008 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) -- 0x55ff6fbbb9c0 con 0x55ff6fb3b470 2024-04-04T06:07:33.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.509+0000 7f5c1a400640 1 --2- 172.21.15.159:0/4135356008 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5bf806b2d0 0x7f5bf806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.509+0000 7f5c1ae00640 1 --2- 172.21.15.159:0/4135356008 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55ff6fb3b470 0x55ff6fbbb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.510+0000 7f5c1ae00640 1 --2- 172.21.15.159:0/4135356008 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55ff6fb3b470 0x55ff6fbbb480 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:33.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.510+0000 7f5c1ae00640 1 -- 172.21.15.159:0/4135356008 <== osd.3 v2:172.21.15.159:6820/2070760205 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f5c1404d780 con 0x55ff6fb3b470 2024-04-04T06:07:33.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.511+0000 7f5c1a400640 1 --2- 172.21.15.159:0/4135356008 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5bf806b2d0 0x7f5bf806d790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00f4d0 tx=0x7f5c0c003630 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:33.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.511+0000 7f5c1da4cf40 1 -- 172.21.15.159:0/4135356008 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55ff6fb3b470 msgr2=0x55ff6fbbb480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.511+0000 7f5c1da4cf40 1 --2- 172.21.15.159:0/4135356008 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55ff6fb3b470 0x55ff6fbbb480 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.511+0000 7f5c1da4cf40 1 -- 172.21.15.159:0/4135356008 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5bf806b2d0 msgr2=0x7f5bf806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.511+0000 7f5c1da4cf40 1 --2- 172.21.15.159:0/4135356008 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5bf806b2d0 0x7f5bf806d790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00f4d0 tx=0x7f5c0c003630 comp rx=0 tx=0).stop 2024-04-04T06:07:33.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.511+0000 7f5c1da4cf40 1 -- 172.21.15.159:0/4135356008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff6fbbc7d0 msgr2=0x55ff6fbbcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.511+0000 7f5c1da4cf40 1 --2- 172.21.15.159:0/4135356008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff6fbbc7d0 0x55ff6fbbcf90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400b700 tx=0x7f5c04000bc0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.511+0000 7f5c1ae00640 1 -- 172.21.15.159:0/4135356008 reap_dead start 2024-04-04T06:07:33.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.511+0000 7f5c1da4cf40 1 -- 172.21.15.159:0/4135356008 shutdown_connections 2024-04-04T06:07:33.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.511+0000 7f5c1da4cf40 1 -- 172.21.15.159:0/4135356008 >> 172.21.15.159:0/4135356008 conn(0x55ff6faf08a0 msgr2=0x55ff6fb1c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:33.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.511+0000 7f5c1da4cf40 1 -- 172.21.15.159:0/4135356008 shutdown_connections 2024-04-04T06:07:33.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.511+0000 7f5c1da4cf40 1 -- 172.21.15.159:0/4135356008 wait complete. 2024-04-04T06:07:33.510 INFO:teuthology.orchestra.run.smithi159.stdout:mytag 2024-04-04T06:07:33.510 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-04-04T06:07:33.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.606+0000 7fec626f5f40 1 Processor -- start 2024-04-04T06:07:33.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.606+0000 7fec626f5f40 1 -- start start 2024-04-04T06:07:33.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.607+0000 7fec626f5f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082f6bb240 0x56082f6b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.607+0000 7fec626f5f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56082f6c1270 0x56082f6b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.607+0000 7fec626f5f40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082f6c1be0 0x56082f6b5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.607+0000 7fec626f5f40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56082f5ea8d0 con 0x56082f6c1be0 2024-04-04T06:07:33.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.607+0000 7fec626f5f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56082f634160 con 0x56082f6bb240 2024-04-04T06:07:33.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.607+0000 7fec626f5f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56082f5ea390 con 0x56082f6c1270 2024-04-04T06:07:33.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.607+0000 7fec60200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082f6c1be0 0x56082f6b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.607+0000 7fec60200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082f6c1be0 0x56082f6b5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48228/0 (socket says 172.21.15.159:48228) 2024-04-04T06:07:33.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.607+0000 7fec60200640 1 -- 172.21.15.159:0/437598735 learned_addr learned my addr 172.21.15.159:0/437598735 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:33.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.607+0000 7fec5f800640 1 --2- 172.21.15.159:0/437598735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082f6bb240 0x56082f6b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.607+0000 7fec5ee00640 1 --2- 172.21.15.159:0/437598735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56082f6c1270 0x56082f6b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.607+0000 7fec60200640 1 -- 172.21.15.159:0/437598735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56082f6c1270 msgr2=0x56082f6b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:33.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.608+0000 7fec60200640 1 --2- 172.21.15.159:0/437598735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56082f6c1270 0x56082f6b2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.608+0000 7fec60200640 1 -- 172.21.15.159:0/437598735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082f6bb240 msgr2=0x56082f6b6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:33.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.608+0000 7fec60200640 1 --2- 172.21.15.159:0/437598735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082f6bb240 0x56082f6b6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.608+0000 7fec60200640 1 -- 172.21.15.159:0/437598735 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56082f52c4b0 con 0x56082f6c1be0 2024-04-04T06:07:33.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.608+0000 7fec5ee00640 1 --2- 172.21.15.159:0/437598735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56082f6c1270 0x56082f6b2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:33.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.608+0000 7fec5f800640 1 --2- 172.21.15.159:0/437598735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082f6bb240 0x56082f6b6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:33.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.608+0000 7fec60200640 1 --2- 172.21.15.159:0/437598735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082f6c1be0 0x56082f6b5590 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fec58009510 tx=0x7fec580099d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282acdd283b2e2d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:33.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.608+0000 7fec5da00640 1 -- 172.21.15.159:0/437598735 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58017070 con 0x56082f6c1be0 2024-04-04T06:07:33.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.608+0000 7fec5da00640 1 -- 172.21.15.159:0/437598735 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec58006580 con 0x56082f6c1be0 2024-04-04T06:07:33.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.608+0000 7fec5da00640 1 -- 172.21.15.159:0/437598735 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58006970 con 0x56082f6c1be0 2024-04-04T06:07:33.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.609+0000 7fec626f5f40 1 -- 172.21.15.159:0/437598735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082f6c1be0 msgr2=0x56082f6b5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.609+0000 7fec626f5f40 1 --2- 172.21.15.159:0/437598735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082f6c1be0 0x56082f6b5590 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fec58009510 tx=0x7fec580099d0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.609+0000 7fec626f5f40 1 -- 172.21.15.159:0/437598735 shutdown_connections 2024-04-04T06:07:33.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.609+0000 7fec626f5f40 1 --2- 172.21.15.159:0/437598735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082f6c1be0 0x56082f6b5590 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.609+0000 7fec626f5f40 1 --2- 172.21.15.159:0/437598735 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56082f6c1270 0x56082f6b2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.609+0000 7fec626f5f40 1 --2- 172.21.15.159:0/437598735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082f6bb240 0x56082f6b6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.609+0000 7fec626f5f40 1 -- 172.21.15.159:0/437598735 >> 172.21.15.159:0/437598735 conn(0x56082f637630 msgr2=0x56082f6c47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:33.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.609+0000 7fec626f5f40 1 -- 172.21.15.159:0/437598735 shutdown_connections 2024-04-04T06:07:33.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.609+0000 7fec626f5f40 1 -- 172.21.15.159:0/437598735 wait complete. 2024-04-04T06:07:33.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.610+0000 7fec626f5f40 1 Processor -- start 2024-04-04T06:07:33.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.610+0000 7fec626f5f40 1 -- start start 2024-04-04T06:07:33.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.610+0000 7fec626f5f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56082f6bb240 0x56082f6bb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.611+0000 7fec626f5f40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082f6c1270 0x56082f75c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.611+0000 7fec626f5f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082f6c1be0 0x56082f75ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.611+0000 7fec626f5f40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56082f5e94f0 con 0x56082f6c1270 2024-04-04T06:07:33.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.611+0000 7fec626f5f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56082f6b2640 con 0x56082f6c1be0 2024-04-04T06:07:33.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.611+0000 7fec626f5f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56082f6b22d0 con 0x56082f6bb240 2024-04-04T06:07:33.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.611+0000 7fec5ee00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082f6c1270 0x56082f75c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.611+0000 7fec5ee00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082f6c1270 0x56082f75c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48244/0 (socket says 172.21.15.159:48244) 2024-04-04T06:07:33.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.611+0000 7fec5ee00640 1 -- 172.21.15.159:0/3475019463 learned_addr learned my addr 172.21.15.159:0/3475019463 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:33.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.611+0000 7fec5ee00640 1 -- 172.21.15.159:0/3475019463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56082f6bb240 msgr2=0x56082f6bb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.611+0000 7fec5ee00640 1 --2- 172.21.15.159:0/3475019463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56082f6bb240 0x56082f6bb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.611+0000 7fec5ee00640 1 -- 172.21.15.159:0/3475019463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082f6c1be0 msgr2=0x56082f75ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.611+0000 7fec5ee00640 1 --2- 172.21.15.159:0/3475019463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56082f6c1be0 0x56082f75ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.611+0000 7fec5ee00640 1 -- 172.21.15.159:0/3475019463 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56082f75ba50 con 0x56082f6c1270 2024-04-04T06:07:33.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.611+0000 7fec5ee00640 1 --2- 172.21.15.159:0/3475019463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082f6c1270 0x56082f75c530 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fec4c007dd0 tx=0x7fec4c006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:33.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.612+0000 7fec57e00640 1 -- 172.21.15.159:0/3475019463 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c006e60 con 0x56082f6c1270 2024-04-04T06:07:33.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.612+0000 7fec57e00640 1 -- 172.21.15.159:0/3475019463 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec4c010710 con 0x56082f6c1270 2024-04-04T06:07:33.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.612+0000 7fec57e00640 1 -- 172.21.15.159:0/3475019463 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c010ac0 con 0x56082f6c1270 2024-04-04T06:07:33.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.612+0000 7fec626f5f40 1 -- 172.21.15.159:0/3475019463 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56082f75bd30 con 0x56082f6c1270 2024-04-04T06:07:33.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.612+0000 7fec626f5f40 1 -- 172.21.15.159:0/3475019463 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56082f618680 con 0x56082f6c1270 2024-04-04T06:07:33.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.614+0000 7fec57e00640 1 -- 172.21.15.159:0/3475019463 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec4c010c60 con 0x56082f6c1270 2024-04-04T06:07:33.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.615+0000 7fec57e00640 1 --2- 172.21.15.159:0/3475019463 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fec4006b300 0x7fec4006d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.615+0000 7fec57e00640 1 -- 172.21.15.159:0/3475019463 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fec4c089f00 con 0x56082f6c1270 2024-04-04T06:07:33.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.615+0000 7fec626f5f40 1 --2- 172.21.15.159:0/3475019463 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x56082f763070 0x56082f765460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.615+0000 7fec626f5f40 1 -- 172.21.15.159:0/3475019463 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:1 4.7 4:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) -- 0x56082f7659a0 con 0x56082f763070 2024-04-04T06:07:33.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.615+0000 7fec5f800640 1 --2- 172.21.15.159:0/3475019463 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fec4006b300 0x7fec4006d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.615+0000 7fec60200640 1 --2- 172.21.15.159:0/3475019463 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x56082f763070 0x56082f765460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.616+0000 7fec60200640 1 --2- 172.21.15.159:0/3475019463 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x56082f763070 0x56082f765460 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:33.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.616+0000 7fec60200640 1 -- 172.21.15.159:0/3475019463 <== osd.1 v2:172.21.15.159:6816/3765338297 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fec58006810 con 0x56082f763070 2024-04-04T06:07:33.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.617+0000 7fec626f5f40 1 -- 172.21.15.159:0/3475019463 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x56082f763070 msgr2=0x56082f765460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.617+0000 7fec626f5f40 1 --2- 172.21.15.159:0/3475019463 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x56082f763070 0x56082f765460 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.617+0000 7fec5f800640 1 --2- 172.21.15.159:0/3475019463 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fec4006b300 0x7fec4006d7c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fec50052000 tx=0x7fec5004c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:33.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.617+0000 7fec626f5f40 1 -- 172.21.15.159:0/3475019463 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fec4006b300 msgr2=0x7fec4006d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.617+0000 7fec626f5f40 1 --2- 172.21.15.159:0/3475019463 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fec4006b300 0x7fec4006d7c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fec50052000 tx=0x7fec5004c000 comp rx=0 tx=0).stop 2024-04-04T06:07:33.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.617+0000 7fec626f5f40 1 -- 172.21.15.159:0/3475019463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082f6c1270 msgr2=0x56082f75c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.617+0000 7fec626f5f40 1 --2- 172.21.15.159:0/3475019463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56082f6c1270 0x56082f75c530 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fec4c007dd0 tx=0x7fec4c006750 comp rx=0 tx=0).stop 2024-04-04T06:07:33.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.617+0000 7fec60200640 1 -- 172.21.15.159:0/3475019463 reap_dead start 2024-04-04T06:07:33.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.617+0000 7fec626f5f40 1 -- 172.21.15.159:0/3475019463 shutdown_connections 2024-04-04T06:07:33.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.617+0000 7fec626f5f40 1 -- 172.21.15.159:0/3475019463 >> 172.21.15.159:0/3475019463 conn(0x56082f637630 msgr2=0x56082f6c3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:33.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.617+0000 7fec626f5f40 1 -- 172.21.15.159:0/3475019463 shutdown_connections 2024-04-04T06:07:33.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.617+0000 7fec626f5f40 1 -- 172.21.15.159:0/3475019463 wait complete. 2024-04-04T06:07:33.616 INFO:teuthology.orchestra.run.smithi159.stdout:mytag 2024-04-04T06:07:33.616 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-04T06:07:33.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.706+0000 7f19d983bf40 1 Processor -- start 2024-04-04T06:07:33.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.706+0000 7f19d983bf40 1 -- start start 2024-04-04T06:07:33.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.707+0000 7f19d983bf40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55878849d2b0 0x55878853a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.707+0000 7f19d983bf40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55878853c230 0x55878854c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.707+0000 7f19d983bf40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558788543970 0x558788538fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.707+0000 7f19d983bf40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5587884b71c0 con 0x55878853c230 2024-04-04T06:07:33.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.707+0000 7f19d983bf40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5587884112c0 con 0x558788543970 2024-04-04T06:07:33.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.707+0000 7f19d983bf40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5587884b7340 con 0x55878849d2b0 2024-04-04T06:07:33.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.707+0000 7f19d5800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55878853c230 0x55878854c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.707+0000 7f19d5800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55878853c230 0x55878854c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48266/0 (socket says 172.21.15.159:48266) 2024-04-04T06:07:33.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.707+0000 7f19d5800640 1 -- 172.21.15.159:0/3207925859 learned_addr learned my addr 172.21.15.159:0/3207925859 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:33.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.708+0000 7f19d6c00640 1 --2- 172.21.15.159:0/3207925859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558788543970 0x558788538fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.708+0000 7f19d6200640 1 --2- 172.21.15.159:0/3207925859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55878849d2b0 0x55878853a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.708+0000 7f19d5800640 1 -- 172.21.15.159:0/3207925859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55878849d2b0 msgr2=0x55878853a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:33.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.708+0000 7f19d5800640 1 --2- 172.21.15.159:0/3207925859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55878849d2b0 0x55878853a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.708+0000 7f19d5800640 1 -- 172.21.15.159:0/3207925859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558788543970 msgr2=0x558788538fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:33.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.708+0000 7f19d5800640 1 --2- 172.21.15.159:0/3207925859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558788543970 0x558788538fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.708+0000 7f19d5800640 1 -- 172.21.15.159:0/3207925859 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55878854be10 con 0x55878853c230 2024-04-04T06:07:33.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.708+0000 7f19d5800640 1 --2- 172.21.15.159:0/3207925859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55878853c230 0x55878854c0b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f19c800a4b0 tx=0x7f19c804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8363d1d00f3a6871 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:33.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.708+0000 7f19cfe00640 1 -- 172.21.15.159:0/3207925859 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8052980 con 0x55878853c230 2024-04-04T06:07:33.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.708+0000 7f19cfe00640 1 -- 172.21.15.159:0/3207925859 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19c8062070 con 0x55878853c230 2024-04-04T06:07:33.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.708+0000 7f19cfe00640 1 -- 172.21.15.159:0/3207925859 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8052d60 con 0x55878853c230 2024-04-04T06:07:33.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.709+0000 7f19d983bf40 1 -- 172.21.15.159:0/3207925859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55878853c230 msgr2=0x55878854c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.709+0000 7f19d983bf40 1 --2- 172.21.15.159:0/3207925859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55878853c230 0x55878854c0b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f19c800a4b0 tx=0x7f19c804c000 comp rx=0 tx=0).stop 2024-04-04T06:07:33.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.709+0000 7f19d983bf40 1 -- 172.21.15.159:0/3207925859 shutdown_connections 2024-04-04T06:07:33.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.709+0000 7f19d983bf40 1 --2- 172.21.15.159:0/3207925859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558788543970 0x558788538fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.709+0000 7f19d983bf40 1 --2- 172.21.15.159:0/3207925859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55878853c230 0x55878854c0b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.709+0000 7f19d983bf40 1 --2- 172.21.15.159:0/3207925859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55878849d2b0 0x55878853a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.709+0000 7f19d983bf40 1 -- 172.21.15.159:0/3207925859 >> 172.21.15.159:0/3207925859 conn(0x5587884b8360 msgr2=0x558788544a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:33.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.709+0000 7f19d983bf40 1 -- 172.21.15.159:0/3207925859 shutdown_connections 2024-04-04T06:07:33.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.709+0000 7f19d983bf40 1 -- 172.21.15.159:0/3207925859 wait complete. 2024-04-04T06:07:33.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.710+0000 7f19d983bf40 1 Processor -- start 2024-04-04T06:07:33.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.710+0000 7f19d983bf40 1 -- start start 2024-04-04T06:07:33.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.711+0000 7f19d983bf40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55878849d2b0 0x558788532ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.711+0000 7f19d983bf40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55878853c230 0x5587885304c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.711+0000 7f19d983bf40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558788543970 0x55878846adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.711+0000 7f19d983bf40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558788539d40 con 0x55878853c230 2024-04-04T06:07:33.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.711+0000 7f19d983bf40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558788544160 con 0x55878849d2b0 2024-04-04T06:07:33.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.711+0000 7f19d983bf40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5587885443b0 con 0x558788543970 2024-04-04T06:07:33.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.711+0000 7f19d5800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55878853c230 0x5587885304c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.711+0000 7f19d5800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55878853c230 0x5587885304c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48278/0 (socket says 172.21.15.159:48278) 2024-04-04T06:07:33.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.711+0000 7f19d5800640 1 -- 172.21.15.159:0/893487788 learned_addr learned my addr 172.21.15.159:0/893487788 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:33.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.711+0000 7f19d6200640 1 --2- 172.21.15.159:0/893487788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55878849d2b0 0x558788532ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.711+0000 7f19d6c00640 1 --2- 172.21.15.159:0/893487788 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558788543970 0x55878846adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.712+0000 7f19d5800640 1 -- 172.21.15.159:0/893487788 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558788543970 msgr2=0x55878846adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.712+0000 7f19d5800640 1 --2- 172.21.15.159:0/893487788 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558788543970 0x55878846adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.712+0000 7f19d5800640 1 -- 172.21.15.159:0/893487788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55878849d2b0 msgr2=0x558788532ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.712+0000 7f19d5800640 1 --2- 172.21.15.159:0/893487788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55878849d2b0 0x558788532ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.712+0000 7f19d5800640 1 -- 172.21.15.159:0/893487788 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558788469db0 con 0x55878853c230 2024-04-04T06:07:33.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.712+0000 7f19d5800640 1 --2- 172.21.15.159:0/893487788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55878853c230 0x5587885304c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f19c801ba40 tx=0x7f19c8003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:33.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.712+0000 7f19cea00640 1 -- 172.21.15.159:0/893487788 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8052c40 con 0x55878853c230 2024-04-04T06:07:33.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.712+0000 7f19cea00640 1 -- 172.21.15.159:0/893487788 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19c8058040 con 0x55878853c230 2024-04-04T06:07:33.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.712+0000 7f19cea00640 1 -- 172.21.15.159:0/893487788 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8071430 con 0x55878853c230 2024-04-04T06:07:33.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.712+0000 7f19d983bf40 1 -- 172.21.15.159:0/893487788 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55878846a030 con 0x55878853c230 2024-04-04T06:07:33.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.712+0000 7f19d983bf40 1 -- 172.21.15.159:0/893487788 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55878846a5f0 con 0x55878853c230 2024-04-04T06:07:33.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.714+0000 7f19cea00640 1 -- 172.21.15.159:0/893487788 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19c8050070 con 0x55878853c230 2024-04-04T06:07:33.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.715+0000 7f19cea00640 1 --2- 172.21.15.159:0/893487788 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f19b406b2d0 0x7f19b406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.715+0000 7f19cea00640 1 -- 172.21.15.159:0/893487788 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f19c80cde60 con 0x55878853c230 2024-04-04T06:07:33.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.715+0000 7f19d983bf40 1 --2- 172.21.15.159:0/893487788 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x558788525510 0x55878846b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.715+0000 7f19d983bf40 1 -- 172.21.15.159:0/893487788 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:1 5.36 5:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) -- 0x558788533430 con 0x558788525510 2024-04-04T06:07:33.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.716+0000 7f19d6200640 1 --2- 172.21.15.159:0/893487788 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f19b406b2d0 0x7f19b406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.716+0000 7f19d6c00640 1 --2- 172.21.15.159:0/893487788 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x558788525510 0x55878846b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.716+0000 7f19d6c00640 1 --2- 172.21.15.159:0/893487788 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x558788525510 0x55878846b330 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:33.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.717+0000 7f19d6c00640 1 -- 172.21.15.159:0/893487788 <== osd.1 v2:172.21.15.159:6816/3765338297 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f19d00074f0 con 0x558788525510 2024-04-04T06:07:33.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.717+0000 7f19d983bf40 1 -- 172.21.15.159:0/893487788 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x558788525510 msgr2=0x55878846b330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.717+0000 7f19d983bf40 1 --2- 172.21.15.159:0/893487788 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x558788525510 0x55878846b330 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.717+0000 7f19d6200640 1 --2- 172.21.15.159:0/893487788 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f19b406b2d0 0x7f19b406d790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x5587884b6a60 tx=0x7f19c0003610 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:33.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.717+0000 7f19d983bf40 1 -- 172.21.15.159:0/893487788 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f19b406b2d0 msgr2=0x7f19b406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.717+0000 7f19d983bf40 1 --2- 172.21.15.159:0/893487788 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f19b406b2d0 0x7f19b406d790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x5587884b6a60 tx=0x7f19c0003610 comp rx=0 tx=0).stop 2024-04-04T06:07:33.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.717+0000 7f19d983bf40 1 -- 172.21.15.159:0/893487788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55878853c230 msgr2=0x5587885304c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.717+0000 7f19d983bf40 1 --2- 172.21.15.159:0/893487788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55878853c230 0x5587885304c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f19c801ba40 tx=0x7f19c8003330 comp rx=0 tx=0).stop 2024-04-04T06:07:33.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.717+0000 7f19d6c00640 1 -- 172.21.15.159:0/893487788 reap_dead start 2024-04-04T06:07:33.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.717+0000 7f19d983bf40 1 -- 172.21.15.159:0/893487788 shutdown_connections 2024-04-04T06:07:33.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.717+0000 7f19d983bf40 1 -- 172.21.15.159:0/893487788 >> 172.21.15.159:0/893487788 conn(0x5587884b8360 msgr2=0x55878849bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:33.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.717+0000 7f19d983bf40 1 -- 172.21.15.159:0/893487788 shutdown_connections 2024-04-04T06:07:33.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.717+0000 7f19d983bf40 1 -- 172.21.15.159:0/893487788 wait complete. 2024-04-04T06:07:33.716 INFO:teuthology.orchestra.run.smithi159.stdout:mytag 2024-04-04T06:07:33.716 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-04T06:07:33.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.807+0000 7f0e1ced2f40 1 Processor -- start 2024-04-04T06:07:33.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.807+0000 7f0e1ced2f40 1 -- start start 2024-04-04T06:07:33.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.808+0000 7f0e1ced2f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9ff49a2b0 0x55c9ff5d06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.808+0000 7f0e1ced2f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c9ff539230 0x55c9ff537100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.808+0000 7f0e1ced2f40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9ff540970 0x55c9ff52d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.808+0000 7f0e1ced2f40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c9ff53fe00 con 0x55c9ff540970 2024-04-04T06:07:33.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.808+0000 7f0e1ced2f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c9ff4b3b20 con 0x55c9ff49a2b0 2024-04-04T06:07:33.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.808+0000 7f0e1ced2f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c9ff4b3570 con 0x55c9ff539230 2024-04-04T06:07:33.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.808+0000 7f0e1aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9ff540970 0x55c9ff52d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.809+0000 7f0e1aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9ff540970 0x55c9ff52d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48284/0 (socket says 172.21.15.159:48284) 2024-04-04T06:07:33.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.809+0000 7f0e1aa00640 1 -- 172.21.15.159:0/1968018565 learned_addr learned my addr 172.21.15.159:0/1968018565 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:33.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.809+0000 7f0e1a000640 1 --2- 172.21.15.159:0/1968018565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9ff49a2b0 0x55c9ff5d06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.809+0000 7f0e19600640 1 --2- 172.21.15.159:0/1968018565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c9ff539230 0x55c9ff537100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.809+0000 7f0e1aa00640 1 -- 172.21.15.159:0/1968018565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c9ff539230 msgr2=0x55c9ff537100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:33.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.809+0000 7f0e1aa00640 1 --2- 172.21.15.159:0/1968018565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c9ff539230 0x55c9ff537100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.809+0000 7f0e1aa00640 1 -- 172.21.15.159:0/1968018565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9ff49a2b0 msgr2=0x55c9ff5d06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:33.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.809+0000 7f0e1aa00640 1 --2- 172.21.15.159:0/1968018565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9ff49a2b0 0x55c9ff5d06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.809+0000 7f0e1aa00640 1 -- 172.21.15.159:0/1968018565 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9ff3aa4b0 con 0x55c9ff540970 2024-04-04T06:07:33.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.809+0000 7f0e1a000640 1 --2- 172.21.15.159:0/1968018565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9ff49a2b0 0x55c9ff5d06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:33.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.809+0000 7f0e1aa00640 1 --2- 172.21.15.159:0/1968018565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9ff540970 0x55c9ff52d7f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400c440 tx=0x7f0e1400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f084d6618124d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:33.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.810+0000 7f0e13e00640 1 -- 172.21.15.159:0/1968018565 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e140116b0 con 0x55c9ff540970 2024-04-04T06:07:33.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.810+0000 7f0e13e00640 1 -- 172.21.15.159:0/1968018565 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e14017070 con 0x55c9ff540970 2024-04-04T06:07:33.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.810+0000 7f0e13e00640 1 -- 172.21.15.159:0/1968018565 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14011a90 con 0x55c9ff540970 2024-04-04T06:07:33.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.810+0000 7f0e1ced2f40 1 -- 172.21.15.159:0/1968018565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9ff540970 msgr2=0x55c9ff52d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.810+0000 7f0e1ced2f40 1 --2- 172.21.15.159:0/1968018565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9ff540970 0x55c9ff52d7f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400c440 tx=0x7f0e1400c900 comp rx=0 tx=0).stop 2024-04-04T06:07:33.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.810+0000 7f0e1ced2f40 1 -- 172.21.15.159:0/1968018565 shutdown_connections 2024-04-04T06:07:33.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.810+0000 7f0e1ced2f40 1 --2- 172.21.15.159:0/1968018565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9ff540970 0x55c9ff52d7f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.810+0000 7f0e1ced2f40 1 --2- 172.21.15.159:0/1968018565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c9ff539230 0x55c9ff537100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.810+0000 7f0e1ced2f40 1 --2- 172.21.15.159:0/1968018565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9ff49a2b0 0x55c9ff5d06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.810+0000 7f0e1ced2f40 1 -- 172.21.15.159:0/1968018565 >> 172.21.15.159:0/1968018565 conn(0x55c9ff4b5360 msgr2=0x55c9ff534ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:33.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.810+0000 7f0e1ced2f40 1 -- 172.21.15.159:0/1968018565 shutdown_connections 2024-04-04T06:07:33.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.810+0000 7f0e1ced2f40 1 -- 172.21.15.159:0/1968018565 wait complete. 2024-04-04T06:07:33.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.811+0000 7f0e1ced2f40 1 Processor -- start 2024-04-04T06:07:33.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.811+0000 7f0e1ced2f40 1 -- start start 2024-04-04T06:07:33.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.812+0000 7f0e1ced2f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9ff49a2b0 0x55c9ff52b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.812+0000 7f0e1ced2f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c9ff539230 0x55c9ff465f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.812+0000 7f0e1ced2f40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9ff540970 0x55c9ff52c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.812+0000 7f0e1ced2f40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c9ff548d70 con 0x55c9ff540970 2024-04-04T06:07:33.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.812+0000 7f0e1ced2f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c9ff5366f0 con 0x55c9ff49a2b0 2024-04-04T06:07:33.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.812+0000 7f0e1ced2f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c9ff496540 con 0x55c9ff539230 2024-04-04T06:07:33.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.812+0000 7f0e1aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9ff540970 0x55c9ff52c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.813+0000 7f0e1aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9ff540970 0x55c9ff52c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48294/0 (socket says 172.21.15.159:48294) 2024-04-04T06:07:33.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.813+0000 7f0e1aa00640 1 -- 172.21.15.159:0/2870280958 learned_addr learned my addr 172.21.15.159:0/2870280958 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:33.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.813+0000 7f0e1a000640 1 --2- 172.21.15.159:0/2870280958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9ff49a2b0 0x55c9ff52b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.813+0000 7f0e19600640 1 --2- 172.21.15.159:0/2870280958 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c9ff539230 0x55c9ff465f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.813+0000 7f0e1aa00640 1 -- 172.21.15.159:0/2870280958 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c9ff539230 msgr2=0x55c9ff465f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.813+0000 7f0e1aa00640 1 --2- 172.21.15.159:0/2870280958 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c9ff539230 0x55c9ff465f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.813+0000 7f0e1aa00640 1 -- 172.21.15.159:0/2870280958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9ff49a2b0 msgr2=0x55c9ff52b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.813+0000 7f0e1aa00640 1 --2- 172.21.15.159:0/2870280958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c9ff49a2b0 0x55c9ff52b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.813+0000 7f0e1aa00640 1 -- 172.21.15.159:0/2870280958 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9ff538a30 con 0x55c9ff540970 2024-04-04T06:07:33.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.813+0000 7f0e19600640 1 --2- 172.21.15.159:0/2870280958 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c9ff539230 0x55c9ff465f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:33.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.813+0000 7f0e1aa00640 1 --2- 172.21.15.159:0/2870280958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9ff540970 0x55c9ff52c040 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0e14014040 tx=0x7f0e140033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:33.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.814+0000 7f0e12a00640 1 -- 172.21.15.159:0/2870280958 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14012c40 con 0x55c9ff540970 2024-04-04T06:07:33.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.814+0000 7f0e12a00640 1 -- 172.21.15.159:0/2870280958 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e14012de0 con 0x55c9ff540970 2024-04-04T06:07:33.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.814+0000 7f0e12a00640 1 -- 172.21.15.159:0/2870280958 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14026650 con 0x55c9ff540970 2024-04-04T06:07:33.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.814+0000 7f0e1ced2f40 1 -- 172.21.15.159:0/2870280958 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c9ff52d1f0 con 0x55c9ff540970 2024-04-04T06:07:33.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.814+0000 7f0e1ced2f40 1 -- 172.21.15.159:0/2870280958 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9ff468d00 con 0x55c9ff540970 2024-04-04T06:07:33.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.816+0000 7f0e12a00640 1 -- 172.21.15.159:0/2870280958 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e14024020 con 0x55c9ff540970 2024-04-04T06:07:33.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.816+0000 7f0e12a00640 1 --2- 172.21.15.159:0/2870280958 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0dfc06b300 0x7f0dfc06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.817+0000 7f0e12a00640 1 -- 172.21.15.159:0/2870280958 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f0e1400e020 con 0x55c9ff540970 2024-04-04T06:07:33.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.817+0000 7f0e1ced2f40 1 --2- 172.21.15.159:0/2870280958 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x55c9ff468310 0x55c9ff52ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.817+0000 7f0e1ced2f40 1 -- 172.21.15.159:0/2870280958 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:1 5.19 5:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) -- 0x55c9ff52bc70 con 0x55c9ff468310 2024-04-04T06:07:33.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.817+0000 7f0e1a000640 1 --2- 172.21.15.159:0/2870280958 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0dfc06b300 0x7f0dfc06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.817+0000 7f0e19600640 1 --2- 172.21.15.159:0/2870280958 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x55c9ff468310 0x55c9ff52ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.818+0000 7f0e19600640 1 --2- 172.21.15.159:0/2870280958 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x55c9ff468310 0x55c9ff52ca90 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:33.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.818+0000 7f0e19600640 1 -- 172.21.15.159:0/2870280958 <== osd.1 v2:172.21.15.159:6816/3765338297 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f0e0800a1e0 con 0x55c9ff468310 2024-04-04T06:07:33.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.818+0000 7f0e1a000640 1 --2- 172.21.15.159:0/2870280958 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0dfc06b300 0x7f0dfc06d7c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c005fd0 tx=0x7f0e0c005ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:33.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.818+0000 7f0e1ced2f40 1 -- 172.21.15.159:0/2870280958 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x55c9ff468310 msgr2=0x55c9ff52ca90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.818+0000 7f0e1ced2f40 1 --2- 172.21.15.159:0/2870280958 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x55c9ff468310 0x55c9ff52ca90 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.818+0000 7f0e1ced2f40 1 -- 172.21.15.159:0/2870280958 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0dfc06b300 msgr2=0x7f0dfc06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.818+0000 7f0e1ced2f40 1 --2- 172.21.15.159:0/2870280958 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0dfc06b300 0x7f0dfc06d7c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c005fd0 tx=0x7f0e0c005ac0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.818+0000 7f0e1ced2f40 1 -- 172.21.15.159:0/2870280958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9ff540970 msgr2=0x55c9ff52c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.818+0000 7f0e1ced2f40 1 --2- 172.21.15.159:0/2870280958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9ff540970 0x55c9ff52c040 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0e14014040 tx=0x7f0e140033a0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.818+0000 7f0e1aa00640 1 -- 172.21.15.159:0/2870280958 reap_dead start 2024-04-04T06:07:33.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.818+0000 7f0e1ced2f40 1 -- 172.21.15.159:0/2870280958 shutdown_connections 2024-04-04T06:07:33.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.818+0000 7f0e1ced2f40 1 -- 172.21.15.159:0/2870280958 >> 172.21.15.159:0/2870280958 conn(0x55c9ff4b5360 msgr2=0x55c9ff548f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:33.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.819+0000 7f0e1ced2f40 1 -- 172.21.15.159:0/2870280958 shutdown_connections 2024-04-04T06:07:33.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.819+0000 7f0e1ced2f40 1 -- 172.21.15.159:0/2870280958 wait complete. 2024-04-04T06:07:33.818 INFO:teuthology.orchestra.run.smithi159.stdout:mytag 2024-04-04T06:07:33.818 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-04T06:07:33.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.906+0000 7fbb08bedf40 1 Processor -- start 2024-04-04T06:07:33.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.906+0000 7fbb08bedf40 1 -- start start 2024-04-04T06:07:33.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.908+0000 7fbb08bedf40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56164507d240 0x561645086a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.908+0000 7fbb08bedf40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561645083270 0x5616450843a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.908+0000 7fbb08bedf40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561645083be0 0x561645078d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.908+0000 7fbb08bedf40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561644fac8d0 con 0x56164507d240 2024-04-04T06:07:33.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.908+0000 7fbb08bedf40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561644ff6160 con 0x561645083270 2024-04-04T06:07:33.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.908+0000 7fbb08bedf40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x561644fac390 con 0x561645083be0 2024-04-04T06:07:33.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.908+0000 7fbb05e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56164507d240 0x561645086a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.908+0000 7fbb05e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56164507d240 0x561645086a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56684/0 (socket says 172.21.15.159:56684) 2024-04-04T06:07:33.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.908+0000 7fbb05e00640 1 -- 172.21.15.159:0/3379755560 learned_addr learned my addr 172.21.15.159:0/3379755560 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:33.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.908+0000 7fbb05e00640 1 -- 172.21.15.159:0/3379755560 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561645083be0 msgr2=0x561645078d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:33.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.908+0000 7fbb05e00640 1 --2- 172.21.15.159:0/3379755560 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561645083be0 0x561645078d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.908+0000 7fbb05e00640 1 -- 172.21.15.159:0/3379755560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561645083270 msgr2=0x5616450843a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:33.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.908+0000 7fbb05400640 1 --2- 172.21.15.159:0/3379755560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561645083270 0x5616450843a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.908+0000 7fbb05e00640 1 --2- 172.21.15.159:0/3379755560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561645083270 0x5616450843a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.909+0000 7fbb05e00640 1 -- 172.21.15.159:0/3379755560 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561644eee4b0 con 0x56164507d240 2024-04-04T06:07:33.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.909+0000 7fbb05e00640 1 --2- 172.21.15.159:0/3379755560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56164507d240 0x561645086a70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800a7a0 tx=0x7fbaf800a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416b948af61d193c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:33.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.909+0000 7fbaffe00640 1 -- 172.21.15.159:0/3379755560 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8019070 con 0x56164507d240 2024-04-04T06:07:33.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.909+0000 7fbaffe00640 1 -- 172.21.15.159:0/3379755560 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf8007600 con 0x56164507d240 2024-04-04T06:07:33.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.909+0000 7fbaffe00640 1 -- 172.21.15.159:0/3379755560 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf80079d0 con 0x56164507d240 2024-04-04T06:07:33.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.910+0000 7fbb08bedf40 1 -- 172.21.15.159:0/3379755560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56164507d240 msgr2=0x561645086a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.910+0000 7fbb08bedf40 1 --2- 172.21.15.159:0/3379755560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56164507d240 0x561645086a70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800a7a0 tx=0x7fbaf800a890 comp rx=0 tx=0).stop 2024-04-04T06:07:33.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.910+0000 7fbb08bedf40 1 -- 172.21.15.159:0/3379755560 shutdown_connections 2024-04-04T06:07:33.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.910+0000 7fbb08bedf40 1 --2- 172.21.15.159:0/3379755560 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561645083be0 0x561645078d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.910+0000 7fbb08bedf40 1 --2- 172.21.15.159:0/3379755560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561645083270 0x5616450843a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.910+0000 7fbb08bedf40 1 --2- 172.21.15.159:0/3379755560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56164507d240 0x561645086a70 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.910+0000 7fbb08bedf40 1 -- 172.21.15.159:0/3379755560 >> 172.21.15.159:0/3379755560 conn(0x561644ff9630 msgr2=0x561645085d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:33.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.910+0000 7fbb08bedf40 1 -- 172.21.15.159:0/3379755560 shutdown_connections 2024-04-04T06:07:33.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.910+0000 7fbb08bedf40 1 -- 172.21.15.159:0/3379755560 wait complete. 2024-04-04T06:07:33.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.911+0000 7fbb08bedf40 1 Processor -- start 2024-04-04T06:07:33.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.911+0000 7fbb08bedf40 1 -- start start 2024-04-04T06:07:33.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.912+0000 7fbb08bedf40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56164507d240 0x56164507e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.912+0000 7fbb08bedf40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561645083270 0x561645085970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.912+0000 7fbb08bedf40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561645083be0 0x56164506a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.912+0000 7fbb08bedf40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561645085350 con 0x561645083270 2024-04-04T06:07:33.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.912+0000 7fbb08bedf40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561644ff8f10 con 0x561645083be0 2024-04-04T06:07:33.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.912+0000 7fbb08bedf40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x561644ff8c30 con 0x56164507d240 2024-04-04T06:07:33.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.912+0000 7fbb05400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561645083270 0x561645085970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.912+0000 7fbb05400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561645083270 0x561645085970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56688/0 (socket says 172.21.15.159:56688) 2024-04-04T06:07:33.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.912+0000 7fbb05400640 1 -- 172.21.15.159:0/3264551225 learned_addr learned my addr 172.21.15.159:0/3264551225 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:33.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.912+0000 7fbb05e00640 1 --2- 172.21.15.159:0/3264551225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56164507d240 0x56164507e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.912+0000 7fbb06800640 1 --2- 172.21.15.159:0/3264551225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561645083be0 0x56164506a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.912+0000 7fbb05400640 1 -- 172.21.15.159:0/3264551225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56164507d240 msgr2=0x56164507e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.912+0000 7fbb05400640 1 --2- 172.21.15.159:0/3264551225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56164507d240 0x56164507e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.912+0000 7fbb05400640 1 -- 172.21.15.159:0/3264551225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561645083be0 msgr2=0x56164506a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.912+0000 7fbb05400640 1 --2- 172.21.15.159:0/3264551225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561645083be0 0x56164506a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.912+0000 7fbb05400640 1 -- 172.21.15.159:0/3264551225 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164507e7a0 con 0x561645083270 2024-04-04T06:07:33.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.912+0000 7fbb06800640 1 --2- 172.21.15.159:0/3264551225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561645083be0 0x56164506a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:33.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.913+0000 7fbb05400640 1 --2- 172.21.15.159:0/3264551225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561645083270 0x561645085970 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400a4f0 tx=0x7fbaf4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:33.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.913+0000 7fbafea00640 1 -- 172.21.15.159:0/3264551225 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4061020 con 0x561645083270 2024-04-04T06:07:33.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.913+0000 7fbafea00640 1 -- 172.21.15.159:0/3264551225 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf4052ad0 con 0x561645083270 2024-04-04T06:07:33.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.913+0000 7fbafea00640 1 -- 172.21.15.159:0/3264551225 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4057040 con 0x561645083270 2024-04-04T06:07:33.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.913+0000 7fbb08bedf40 1 -- 172.21.15.159:0/3264551225 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561645078740 con 0x561645083270 2024-04-04T06:07:33.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.913+0000 7fbb08bedf40 1 -- 172.21.15.159:0/3264551225 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561645078ed0 con 0x561645083270 2024-04-04T06:07:33.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.915+0000 7fbafea00640 1 -- 172.21.15.159:0/3264551225 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbaf4064020 con 0x561645083270 2024-04-04T06:07:33.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.916+0000 7fbafea00640 1 --2- 172.21.15.159:0/3264551225 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbae806b2d0 0x7fbae806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.916+0000 7fbafea00640 1 -- 172.21.15.159:0/3264551225 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fbaf40cc270 con 0x561645083270 2024-04-04T06:07:33.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.916+0000 7fbb08bedf40 1 --2- 172.21.15.159:0/3264551225 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x561645085eb0 0x561645080850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:33.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.916+0000 7fbb08bedf40 1 -- 172.21.15.159:0/3264551225 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:1 5.12 5:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) -- 0x561645078030 con 0x561645085eb0 2024-04-04T06:07:33.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.916+0000 7fbb05e00640 1 --2- 172.21.15.159:0/3264551225 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbae806b2d0 0x7fbae806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.916+0000 7fbb06800640 1 --2- 172.21.15.159:0/3264551225 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x561645085eb0 0x561645080850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:33.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.917+0000 7fbb06800640 1 --2- 172.21.15.159:0/3264551225 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x561645085eb0 0x561645080850 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:33.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.917+0000 7fbb06800640 1 -- 172.21.15.159:0/3264551225 <== osd.1 v2:172.21.15.159:6816/3765338297 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7fbb0000d530 con 0x561645085eb0 2024-04-04T06:07:33.914 INFO:teuthology.orchestra.run.smithi159.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-04-04T06:07:33.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.917+0000 7fbb08bedf40 1 -- 172.21.15.159:0/3264551225 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x561645085eb0 msgr2=0x561645080850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.917+0000 7fbb08bedf40 1 --2- 172.21.15.159:0/3264551225 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x561645085eb0 0x561645080850 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.917+0000 7fbb05e00640 1 --2- 172.21.15.159:0/3264551225 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbae806b2d0 0x7fbae806d790 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800a7a0 tx=0x7fbaf800c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:33.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.917+0000 7fbb08bedf40 1 -- 172.21.15.159:0/3264551225 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbae806b2d0 msgr2=0x7fbae806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.917+0000 7fbb08bedf40 1 --2- 172.21.15.159:0/3264551225 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbae806b2d0 0x7fbae806d790 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800a7a0 tx=0x7fbaf800c000 comp rx=0 tx=0).stop 2024-04-04T06:07:33.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.917+0000 7fbb08bedf40 1 -- 172.21.15.159:0/3264551225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561645083270 msgr2=0x561645085970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:33.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.917+0000 7fbb08bedf40 1 --2- 172.21.15.159:0/3264551225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561645083270 0x561645085970 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400a4f0 tx=0x7fbaf4002fb0 comp rx=0 tx=0).stop 2024-04-04T06:07:33.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.917+0000 7fbb06800640 1 -- 172.21.15.159:0/3264551225 reap_dead start 2024-04-04T06:07:33.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.918+0000 7fbb08bedf40 1 -- 172.21.15.159:0/3264551225 shutdown_connections 2024-04-04T06:07:33.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.918+0000 7fbb08bedf40 1 -- 172.21.15.159:0/3264551225 >> 172.21.15.159:0/3264551225 conn(0x561644ff9630 msgr2=0x561644fddf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:33.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.918+0000 7fbb08bedf40 1 -- 172.21.15.159:0/3264551225 shutdown_connections 2024-04-04T06:07:33.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:33.918+0000 7fbb08bedf40 1 -- 172.21.15.159:0/3264551225 wait complete. 2024-04-04T06:07:33.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-04T06:07:33.916 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-04T06:07:33.916 DEBUG:teuthology.orchestra.run.smithi159:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:07:34.012 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:07:34.134 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-04T06:07:34.134 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:07:34.134 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:07:34.135 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:07:34.190 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:07:34.190 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:07:34.190 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:07:34.191 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:07:34.247 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:07:34.248 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:07:34.248 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:07:34.248 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:07:34.303 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:07:34.303 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:07:34.303 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:07:34.303 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:07:34.353 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:07:34.353 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:07:34.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.576+0000 7fd2cb000640 1 Processor -- start 2024-04-04T06:07:34.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.576+0000 7fd2cb000640 1 -- start start 2024-04-04T06:07:34.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.578+0000 7fd2cb000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd2c415b310 0x7fd2c40727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:34.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.578+0000 7fd2cb000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2c415b6f0 con 0x7fd2c40d0780 2024-04-04T06:07:34.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.578+0000 7fd2cb000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2c4072e80 con 0x7fd2c40b4f90 2024-04-04T06:07:34.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.578+0000 7fd2cb000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fd2c4073160 con 0x7fd2c415b310 2024-04-04T06:07:34.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.578+0000 7fd2c3e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd2c415b310 0x7fd2c40727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:34.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.578+0000 7fd2c9000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fd2c40d0780 0x7fd2c40adc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43846/0 (socket says 172.21.15.159:43846) 2024-04-04T06:07:34.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.578+0000 7fd2c9000640 1 -- 172.21.15.159:0/2766544572 learned_addr learned my addr 172.21.15.159:0/2766544572 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:34.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.578+0000 7fd2c3e00640 1 -- 172.21.15.159:0/2766544572 >> v1:172.21.15.178:6789/0 conn(0x7fd2c40b4f90 legacy=0x7fd2c415f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:34.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.578+0000 7fd2c3e00640 1 -- 172.21.15.159:0/2766544572 >> v1:172.21.15.159:6789/0 conn(0x7fd2c40d0780 legacy=0x7fd2c40adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:34.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.578+0000 7fd2c3e00640 1 -- 172.21.15.159:0/2766544572 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c415efe0 con 0x7fd2c415b310 2024-04-04T06:07:34.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.580+0000 7fd2c3e00640 1 --2- 172.21.15.159:0/2766544572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2c415b310 0x7fd2c40727f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40097a0 tx=0x7fd2b402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4c3c52c4ea56e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:34.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.580+0000 7fd2c2a00640 1 -- 172.21.15.159:0/2766544572 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4038070 con 0x7fd2c415b310 2024-04-04T06:07:34.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.580+0000 7fd2c2a00640 1 -- 172.21.15.159:0/2766544572 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2b4031700 con 0x7fd2c415b310 2024-04-04T06:07:34.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.580+0000 7fd2c2a00640 1 -- 172.21.15.159:0/2766544572 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4031ad0 con 0x7fd2c415b310 2024-04-04T06:07:34.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.580+0000 7fd2cb000640 1 -- 172.21.15.159:0/2766544572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2c415b310 msgr2=0x7fd2c40727f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:34.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.580+0000 7fd2cb000640 1 --2- 172.21.15.159:0/2766544572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2c415b310 0x7fd2c40727f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40097a0 tx=0x7fd2b402b7a0 comp rx=0 tx=0).stop 2024-04-04T06:07:34.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.580+0000 7fd2cb000640 1 -- 172.21.15.159:0/2766544572 shutdown_connections 2024-04-04T06:07:34.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.580+0000 7fd2cb000640 1 --2- 172.21.15.159:0/2766544572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2c415b310 0x7fd2c40727f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:34.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.580+0000 7fd2cb000640 1 -- 172.21.15.159:0/2766544572 >> 172.21.15.159:0/2766544572 conn(0x7fd2c406b990 msgr2=0x7fd2c406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:34.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.580+0000 7fd2cb000640 1 -- 172.21.15.159:0/2766544572 shutdown_connections 2024-04-04T06:07:34.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.581+0000 7fd2cb000640 1 -- 172.21.15.159:0/2766544572 wait complete. 2024-04-04T06:07:34.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.582+0000 7fd2cb000640 1 Processor -- start 2024-04-04T06:07:34.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.582+0000 7fd2cb000640 1 -- start start 2024-04-04T06:07:34.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.583+0000 7fd2cb000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2c40b4f90 0x7fd2c41f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:34.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.583+0000 7fd2cb000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2c40d0780 0x7fd2c414bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:34.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.583+0000 7fd2cb000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2c415b310 0x7fd2c415b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:34.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.583+0000 7fd2cb000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c4073440 con 0x7fd2c415b310 2024-04-04T06:07:34.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.583+0000 7fd2cb000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c40b53f0 con 0x7fd2c40d0780 2024-04-04T06:07:34.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.583+0000 7fd2cb000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c40ae390 con 0x7fd2c40b4f90 2024-04-04T06:07:34.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.583+0000 7fd2c9000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2c415b310 0x7fd2c415b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:34.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.583+0000 7fd2c9000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2c415b310 0x7fd2c415b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56714/0 (socket says 172.21.15.159:56714) 2024-04-04T06:07:34.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.583+0000 7fd2c9000640 1 -- 172.21.15.159:0/4169623994 learned_addr learned my addr 172.21.15.159:0/4169623994 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:34.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.584+0000 7fd2c3e00640 1 --2- 172.21.15.159:0/4169623994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2c40b4f90 0x7fd2c41f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:34.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.584+0000 7fd2c3400640 1 --2- 172.21.15.159:0/4169623994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2c40d0780 0x7fd2c414bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:34.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.584+0000 7fd2c9000640 1 -- 172.21.15.159:0/4169623994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2c40b4f90 msgr2=0x7fd2c41f48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:34.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.584+0000 7fd2c9000640 1 --2- 172.21.15.159:0/4169623994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2c40b4f90 0x7fd2c41f48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:34.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.584+0000 7fd2c9000640 1 -- 172.21.15.159:0/4169623994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2c40d0780 msgr2=0x7fd2c414bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:34.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.584+0000 7fd2c9000640 1 --2- 172.21.15.159:0/4169623994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2c40d0780 0x7fd2c414bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:34.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.584+0000 7fd2c9000640 1 -- 172.21.15.159:0/4169623994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c41481a0 con 0x7fd2c415b310 2024-04-04T06:07:34.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.584+0000 7fd2c9000640 1 --2- 172.21.15.159:0/4169623994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2c415b310 0x7fd2c415b8c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00cbc0 tx=0x7fd2bc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:34.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.584+0000 7fd2bbe00640 1 -- 172.21.15.159:0/4169623994 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc017070 con 0x7fd2c415b310 2024-04-04T06:07:34.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.585+0000 7fd2bbe00640 1 -- 172.21.15.159:0/4169623994 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2bc00ad40 con 0x7fd2c415b310 2024-04-04T06:07:34.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.585+0000 7fd2bbe00640 1 -- 172.21.15.159:0/4169623994 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc012600 con 0x7fd2c415b310 2024-04-04T06:07:34.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.585+0000 7fd2cb000640 1 -- 172.21.15.159:0/4169623994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c4155e30 con 0x7fd2c415b310 2024-04-04T06:07:34.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.585+0000 7fd2cb000640 1 -- 172.21.15.159:0/4169623994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c41563f0 con 0x7fd2c415b310 2024-04-04T06:07:34.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.586+0000 7fd2cb000640 1 -- 172.21.15.159:0/4169623994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2c40685c0 con 0x7fd2c415b310 2024-04-04T06:07:34.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.587+0000 7fd2bbe00640 1 -- 172.21.15.159:0/4169623994 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2bc01a070 con 0x7fd2c415b310 2024-04-04T06:07:34.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.587+0000 7fd2bbe00640 1 --2- 172.21.15.159:0/4169623994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd2a406c8f0 0x7fd2a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:34.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.588+0000 7fd2bbe00640 1 -- 172.21.15.159:0/4169623994 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fd2bc08b7f0 con 0x7fd2c415b310 2024-04-04T06:07:34.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.588+0000 7fd2c3e00640 1 --2- 172.21.15.159:0/4169623994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd2a406c8f0 0x7fd2a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:34.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.589+0000 7fd2c3e00640 1 --2- 172.21.15.159:0/4169623994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd2a406c8f0 0x7fd2a406edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4035000 tx=0x7fd2b4008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:34.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.592+0000 7fd2bbe00640 1 -- 172.21.15.159:0/4169623994 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd2bc0595c0 con 0x7fd2c415b310 2024-04-04T06:07:34.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.832+0000 7fd2cb000640 1 -- 172.21.15.159:0/4169623994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2c415ec20 con 0x7fd2c415b310 2024-04-04T06:07:34.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.833+0000 7fd2bbe00640 1 -- 172.21.15.159:0/4169623994 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5193 (secure 0 0 0) 0x7fd2bc05d8c0 con 0x7fd2c415b310 2024-04-04T06:07:34.828 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:34.828 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1643905381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1643905381},{"type":"v1","addr":"172.21.15.178:6837","nonce":1643905381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4744,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2629539137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2629539137},{"type":"v1","addr":"172.21.15.159:6835","nonce":2629539137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3402172592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3402172592},{"type":"v1","addr":"172.21.15.159:6833","nonce":3402172592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:07:08.839645+0000","modified":"2024-04-04T06:07:11.927601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5157},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5157":{"gid":5157,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.178:6836/3282963883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3282963883},{"type":"v1","addr":"172.21.15.178:6836","nonce":3282963883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5157,"qdb_cluster":[5157]},"id":2}]} 2024-04-04T06:07:34.828 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 17 2024-04-04T06:07:34.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.836+0000 7fd2cb000640 1 -- 172.21.15.159:0/4169623994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd2a406c8f0 msgr2=0x7fd2a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:34.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.836+0000 7fd2cb000640 1 --2- 172.21.15.159:0/4169623994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd2a406c8f0 0x7fd2a406edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4035000 tx=0x7fd2b4008030 comp rx=0 tx=0).stop 2024-04-04T06:07:34.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.836+0000 7fd2cb000640 1 -- 172.21.15.159:0/4169623994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2c415b310 msgr2=0x7fd2c415b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:34.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.836+0000 7fd2cb000640 1 --2- 172.21.15.159:0/4169623994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2c415b310 0x7fd2c415b8c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00cbc0 tx=0x7fd2bc00a430 comp rx=0 tx=0).stop 2024-04-04T06:07:34.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.836+0000 7fd2cb000640 1 -- 172.21.15.159:0/4169623994 shutdown_connections 2024-04-04T06:07:34.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.836+0000 7fd2cb000640 1 --2- 172.21.15.159:0/4169623994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2c415b310 0x7fd2c415b8c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:34.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.836+0000 7fd2cb000640 1 --2- 172.21.15.159:0/4169623994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd2a406c8f0 0x7fd2a406edb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:34.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.836+0000 7fd2cb000640 1 --2- 172.21.15.159:0/4169623994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2c40d0780 0x7fd2c414bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:34.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.836+0000 7fd2cb000640 1 --2- 172.21.15.159:0/4169623994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2c40b4f90 0x7fd2c41f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:34.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.836+0000 7fd2cb000640 1 -- 172.21.15.159:0/4169623994 >> 172.21.15.159:0/4169623994 conn(0x7fd2c406b990 msgr2=0x7fd2c406da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:34.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.836+0000 7fd2cb000640 1 -- 172.21.15.159:0/4169623994 shutdown_connections 2024-04-04T06:07:34.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:34.836+0000 7fd2cb000640 1 -- 172.21.15.159:0/4169623994 wait complete. 2024-04-04T06:07:34.850 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:07:35.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.068+0000 7fe6de000640 1 Processor -- start 2024-04-04T06:07:35.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.068+0000 7fe6de000640 1 -- start start 2024-04-04T06:07:35.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.070+0000 7fe6de000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6d815b310 0x7fe6d80727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.070+0000 7fe6de000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6d815b6f0 con 0x7fe6d80d0780 2024-04-04T06:07:35.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.070+0000 7fe6de000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6d8072e80 con 0x7fe6d80b4f90 2024-04-04T06:07:35.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.070+0000 7fe6de000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d8073160 con 0x7fe6d815b310 2024-04-04T06:07:35.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.070+0000 7fe6d7e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fe6d80d0780 0x7fe6d80adc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43860/0 (socket says 172.21.15.159:43860) 2024-04-04T06:07:35.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.070+0000 7fe6d7e00640 1 -- 172.21.15.159:0/3826162303 learned_addr learned my addr 172.21.15.159:0/3826162303 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:35.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.071+0000 7fe6d7400640 1 --2- 172.21.15.159:0/3826162303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6d815b310 0x7fe6d80727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.071+0000 7fe6d6000640 1 -- 172.21.15.159:0/3826162303 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2351728600 0 0) 0x7fe6d815b6f0 con 0x7fe6d80d0780 2024-04-04T06:07:35.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.071+0000 7fe6d6000640 1 -- 172.21.15.159:0/3826162303 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6bc003520 con 0x7fe6d80d0780 2024-04-04T06:07:35.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.071+0000 7fe6d6000640 1 -- 172.21.15.159:0/3826162303 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 749940272 0 0) 0x7fe6bc003520 con 0x7fe6d80d0780 2024-04-04T06:07:35.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.071+0000 7fe6d6000640 1 -- 172.21.15.159:0/3826162303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6d815b310 msgr2=0x7fe6d80727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:35.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.071+0000 7fe6d6000640 1 --2- 172.21.15.159:0/3826162303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6d815b310 0x7fe6d80727f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.071+0000 7fe6d6000640 1 -- 172.21.15.159:0/3826162303 >> v1:172.21.15.178:6789/0 conn(0x7fe6d80b4f90 legacy=0x7fe6d815f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:35.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.071+0000 7fe6d6000640 1 -- 172.21.15.159:0/3826162303 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d815efe0 con 0x7fe6d80d0780 2024-04-04T06:07:35.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.071+0000 7fe6d6000640 1 -- 172.21.15.159:0/3826162303 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fe6d0001670 con 0x7fe6d80d0780 2024-04-04T06:07:35.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.071+0000 7fe6d6000640 1 -- 172.21.15.159:0/3826162303 >> v1:172.21.15.159:6789/0 conn(0x7fe6d80d0780 legacy=0x7fe6d80adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:35.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.071+0000 7fe6d6000640 1 --2- 172.21.15.159:0/3826162303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6bc0039e0 0x7fe6bc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.071+0000 7fe6d6000640 1 --2- 172.21.15.159:0/3826162303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6d815b310 0x7fe6bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.071+0000 7fe6d6000640 1 --2- 172.21.15.159:0/3826162303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6bc004d40 0x7fe6bc0051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.071+0000 7fe6d6000640 1 -- 172.21.15.159:0/3826162303 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8073160 con 0x7fe6bc004d40 2024-04-04T06:07:35.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.071+0000 7fe6d6000640 1 -- 172.21.15.159:0/3826162303 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6bc00dde0 con 0x7fe6bc0039e0 2024-04-04T06:07:35.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.071+0000 7fe6d6000640 1 -- 172.21.15.159:0/3826162303 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe6bc00dfe0 con 0x7fe6d815b310 2024-04-04T06:07:35.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.072+0000 7fe6d7e00640 1 --2- 172.21.15.159:0/3826162303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6bc004d40 0x7fe6bc0051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.072+0000 7fe6d7400640 1 --2- 172.21.15.159:0/3826162303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6bc0039e0 0x7fe6bc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.072+0000 7fe6d6a00640 1 --2- 172.21.15.159:0/3826162303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6d815b310 0x7fe6bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.072+0000 7fe6d7e00640 1 -- 172.21.15.159:0/3826162303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6d815b310 msgr2=0x7fe6bc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:35.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.072+0000 7fe6d7e00640 1 --2- 172.21.15.159:0/3826162303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6d815b310 0x7fe6bc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.072+0000 7fe6d7e00640 1 -- 172.21.15.159:0/3826162303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6bc0039e0 msgr2=0x7fe6bc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:35.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.072+0000 7fe6d7e00640 1 --2- 172.21.15.159:0/3826162303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6bc0039e0 0x7fe6bc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.072+0000 7fe6d7e00640 1 -- 172.21.15.159:0/3826162303 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6bc003520 con 0x7fe6bc004d40 2024-04-04T06:07:35.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.073+0000 7fe6d7e00640 1 --2- 172.21.15.159:0/3826162303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6bc004d40 0x7fe6bc0051a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000c0e0 tx=0x7fe6bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f819b34cde320be2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:35.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.073+0000 7fe6d6000640 1 -- 172.21.15.159:0/3826162303 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0007c20 con 0x7fe6bc004d40 2024-04-04T06:07:35.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.074+0000 7fe6d6000640 1 -- 172.21.15.159:0/3826162303 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d0030070 con 0x7fe6bc004d40 2024-04-04T06:07:35.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.074+0000 7fe6d6000640 1 -- 172.21.15.159:0/3826162303 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0005d30 con 0x7fe6bc004d40 2024-04-04T06:07:35.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.074+0000 7fe6de000640 1 -- 172.21.15.159:0/3826162303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6bc004d40 msgr2=0x7fe6bc0051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:35.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.074+0000 7fe6de000640 1 --2- 172.21.15.159:0/3826162303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6bc004d40 0x7fe6bc0051a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000c0e0 tx=0x7fe6bc005830 comp rx=0 tx=0).stop 2024-04-04T06:07:35.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.074+0000 7fe6d7e00640 1 -- 172.21.15.159:0/3826162303 reap_dead start 2024-04-04T06:07:35.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.074+0000 7fe6de000640 1 -- 172.21.15.159:0/3826162303 shutdown_connections 2024-04-04T06:07:35.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.074+0000 7fe6de000640 1 -- 172.21.15.159:0/3826162303 >> 172.21.15.159:0/3826162303 conn(0x7fe6d806b990 msgr2=0x7fe6d806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:35.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.074+0000 7fe6de000640 1 -- 172.21.15.159:0/3826162303 shutdown_connections 2024-04-04T06:07:35.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.074+0000 7fe6de000640 1 -- 172.21.15.159:0/3826162303 wait complete. 2024-04-04T06:07:35.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.076+0000 7fe6de000640 1 Processor -- start 2024-04-04T06:07:35.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.076+0000 7fe6de000640 1 -- start start 2024-04-04T06:07:35.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.077+0000 7fe6de000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6bc004d40 0x7fe6d80c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.077+0000 7fe6de000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6d815ca50 0x7fe6d815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.077+0000 7fe6de000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6d815d410 0x7fe6d80cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.077+0000 7fe6de000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8073440 con 0x7fe6d815d410 2024-04-04T06:07:35.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.077+0000 7fe6de000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d80b53f0 con 0x7fe6bc004d40 2024-04-04T06:07:35.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.077+0000 7fe6de000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d80727f0 con 0x7fe6d815ca50 2024-04-04T06:07:35.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.077+0000 7fe6d7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6d815d410 0x7fe6d80cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.077+0000 7fe6d7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6d815d410 0x7fe6d80cd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56738/0 (socket says 172.21.15.159:56738) 2024-04-04T06:07:35.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.077+0000 7fe6d7e00640 1 -- 172.21.15.159:0/1431697922 learned_addr learned my addr 172.21.15.159:0/1431697922 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:35.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.078+0000 7fe6d7400640 1 --2- 172.21.15.159:0/1431697922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6bc004d40 0x7fe6d80c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.078+0000 7fe6d6a00640 1 --2- 172.21.15.159:0/1431697922 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6d815ca50 0x7fe6d815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.078+0000 7fe6d7e00640 1 -- 172.21.15.159:0/1431697922 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6d815ca50 msgr2=0x7fe6d815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:35.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.078+0000 7fe6d7e00640 1 --2- 172.21.15.159:0/1431697922 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6d815ca50 0x7fe6d815ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.078+0000 7fe6d7e00640 1 -- 172.21.15.159:0/1431697922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6bc004d40 msgr2=0x7fe6d80c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:35.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.078+0000 7fe6d7e00640 1 --2- 172.21.15.159:0/1431697922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6bc004d40 0x7fe6d80c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.078+0000 7fe6d7e00640 1 -- 172.21.15.159:0/1431697922 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d815d890 con 0x7fe6d815d410 2024-04-04T06:07:35.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.078+0000 7fe6d6a00640 1 --2- 172.21.15.159:0/1431697922 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6d815ca50 0x7fe6d815ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:35.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.078+0000 7fe6d7400640 1 --2- 172.21.15.159:0/1431697922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6bc004d40 0x7fe6d80c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:35.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.078+0000 7fe6d7e00640 1 --2- 172.21.15.159:0/1431697922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6d815d410 0x7fe6d80cd130 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0000c00 tx=0x7fe6d815b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:35.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.078+0000 7fe6cf400640 1 -- 172.21.15.159:0/1431697922 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d00047e0 con 0x7fe6d815d410 2024-04-04T06:07:35.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.078+0000 7fe6cf400640 1 -- 172.21.15.159:0/1431697922 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d0004980 con 0x7fe6d815d410 2024-04-04T06:07:35.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.078+0000 7fe6cf400640 1 -- 172.21.15.159:0/1431697922 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0004d30 con 0x7fe6d815d410 2024-04-04T06:07:35.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.079+0000 7fe6de000640 1 -- 172.21.15.159:0/1431697922 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d80ca840 con 0x7fe6d815d410 2024-04-04T06:07:35.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.079+0000 7fe6de000640 1 -- 172.21.15.159:0/1431697922 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d80cae00 con 0x7fe6d815d410 2024-04-04T06:07:35.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.080+0000 7fe6de000640 1 -- 172.21.15.159:0/1431697922 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d80685c0 con 0x7fe6d815d410 2024-04-04T06:07:35.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.081+0000 7fe6cf400640 1 -- 172.21.15.159:0/1431697922 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe6d0007070 con 0x7fe6d815d410 2024-04-04T06:07:35.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.082+0000 7fe6cf400640 1 --2- 172.21.15.159:0/1431697922 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe6b806c920 0x7fe6b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.082+0000 7fe6cf400640 1 -- 172.21.15.159:0/1431697922 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fe6d00ac400 con 0x7fe6d815d410 2024-04-04T06:07:35.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.082+0000 7fe6d7400640 1 --2- 172.21.15.159:0/1431697922 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe6b806c920 0x7fe6b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.083+0000 7fe6d7400640 1 --2- 172.21.15.159:0/1431697922 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe6b806c920 0x7fe6b806ede0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80af790 tx=0x7fe6c800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:35.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.086+0000 7fe6cf400640 1 -- 172.21.15.159:0/1431697922 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe6d0048070 con 0x7fe6d815d410 2024-04-04T06:07:35.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.284+0000 7fe6de000640 1 -- 172.21.15.159:0/1431697922 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6d815ec20 con 0x7fe6d815d410 2024-04-04T06:07:35.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.286+0000 7fe6cf400640 1 -- 172.21.15.159:0/1431697922 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16681 (secure 0 0 0) 0x7fe6d007a820 con 0x7fe6d815d410 2024-04-04T06:07:35.281 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:35.281 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":34,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:07:10.401521+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:07:06.708548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-04T06:07:07.833730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:07:35.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.288+0000 7fe6de000640 1 -- 172.21.15.159:0/1431697922 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe6b806c920 msgr2=0x7fe6b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:35.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.288+0000 7fe6de000640 1 --2- 172.21.15.159:0/1431697922 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe6b806c920 0x7fe6b806ede0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80af790 tx=0x7fe6c800b040 comp rx=0 tx=0).stop 2024-04-04T06:07:35.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.288+0000 7fe6de000640 1 -- 172.21.15.159:0/1431697922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6d815d410 msgr2=0x7fe6d80cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:35.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.288+0000 7fe6de000640 1 --2- 172.21.15.159:0/1431697922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6d815d410 0x7fe6d80cd130 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0000c00 tx=0x7fe6d815b8c0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.289+0000 7fe6de000640 1 -- 172.21.15.159:0/1431697922 shutdown_connections 2024-04-04T06:07:35.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.289+0000 7fe6de000640 1 --2- 172.21.15.159:0/1431697922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6d815d410 0x7fe6d80cd130 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.289+0000 7fe6de000640 1 --2- 172.21.15.159:0/1431697922 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6d815ca50 0x7fe6d815ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.289+0000 7fe6de000640 1 --2- 172.21.15.159:0/1431697922 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe6b806c920 0x7fe6b806ede0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.289+0000 7fe6de000640 1 --2- 172.21.15.159:0/1431697922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6bc004d40 0x7fe6d80c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.289+0000 7fe6de000640 1 -- 172.21.15.159:0/1431697922 >> 172.21.15.159:0/1431697922 conn(0x7fe6d806b990 msgr2=0x7fe6d80b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:35.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.289+0000 7fe6de000640 1 -- 172.21.15.159:0/1431697922 shutdown_connections 2024-04-04T06:07:35.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.289+0000 7fe6de000640 1 -- 172.21.15.159:0/1431697922 wait complete. 2024-04-04T06:07:35.302 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:07:06.708548+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:07:07.833730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:07:35.302 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:07:35.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.510+0000 7f9e1f800640 1 Processor -- start 2024-04-04T06:07:35.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.511+0000 7f9e1f800640 1 -- start start 2024-04-04T06:07:35.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.512+0000 7f9e1f800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e18155550 0x7f9e18155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.512+0000 7f9e1f800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9e1815f800 0x7f9e180d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.512+0000 7f9e1f800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e180b40b0 con 0x7f9e180d07c0 2024-04-04T06:07:35.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.512+0000 7f9e1f800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f9e180d0c30 con 0x7f9e1815f800 2024-04-04T06:07:35.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.512+0000 7f9e1f800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9e180b4390 con 0x7f9e18155550 2024-04-04T06:07:35.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.512+0000 7f9e17e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9e1815f800 0x7f9e180d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.513+0000 7f9e17e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9e1815f800 0x7f9e180d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56750/0 (socket says 172.21.15.159:56750) 2024-04-04T06:07:35.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.513+0000 7f9e17e00640 1 -- 172.21.15.159:0/283273352 learned_addr learned my addr 172.21.15.159:0/283273352 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:35.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.513+0000 7f9e1ce00640 1 --2- 172.21.15.159:0/283273352 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e18155550 0x7f9e18155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.513+0000 7f9e17e00640 1 -- 172.21.15.159:0/283273352 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e18155550 msgr2=0x7f9e18155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:35.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.513+0000 7f9e17e00640 1 --2- 172.21.15.159:0/283273352 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e18155550 0x7f9e18155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.513+0000 7f9e17e00640 1 -- 172.21.15.159:0/283273352 >> v1:172.21.15.178:6789/0 conn(0x7f9e180d07c0 legacy=0x7f9e180aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:35.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.513+0000 7f9e17e00640 1 -- 172.21.15.159:0/283273352 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e180b4670 con 0x7f9e1815f800 2024-04-04T06:07:35.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.514+0000 7f9e17e00640 1 --2- 172.21.15.159:0/283273352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e1815f800 0x7f9e180d0280 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009e50 tx=0x7f9e0002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757d5d4c1cfc9cba server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:35.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.514+0000 7f9e17400640 1 -- 172.21.15.159:0/283273352 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0002ca80 con 0x7f9e1815f800 2024-04-04T06:07:35.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.514+0000 7f9e17400640 1 -- 172.21.15.159:0/283273352 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e0002cc20 con 0x7f9e1815f800 2024-04-04T06:07:35.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.514+0000 7f9e17400640 1 -- 172.21.15.159:0/283273352 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e000343f0 con 0x7f9e1815f800 2024-04-04T06:07:35.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.515+0000 7f9e1f800640 1 -- 172.21.15.159:0/283273352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e1815f800 msgr2=0x7f9e180d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:35.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.515+0000 7f9e1f800640 1 --2- 172.21.15.159:0/283273352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e1815f800 0x7f9e180d0280 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009e50 tx=0x7f9e0002bee0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.515+0000 7f9e1f800640 1 -- 172.21.15.159:0/283273352 shutdown_connections 2024-04-04T06:07:35.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.515+0000 7f9e1f800640 1 --2- 172.21.15.159:0/283273352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e1815f800 0x7f9e180d0280 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.515+0000 7f9e1f800640 1 --2- 172.21.15.159:0/283273352 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e18155550 0x7f9e18155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.515+0000 7f9e1f800640 1 -- 172.21.15.159:0/283273352 >> 172.21.15.159:0/283273352 conn(0x7f9e1806b750 msgr2=0x7f9e1806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:35.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.515+0000 7f9e1f800640 1 -- 172.21.15.159:0/283273352 shutdown_connections 2024-04-04T06:07:35.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.515+0000 7f9e1f800640 1 -- 172.21.15.159:0/283273352 wait complete. 2024-04-04T06:07:35.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.517+0000 7f9e1f800640 1 Processor -- start 2024-04-04T06:07:35.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.518+0000 7f9e1f800640 1 -- start start 2024-04-04T06:07:35.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.519+0000 7f9e1f800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e180d07c0 0x7f9e1814e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.519+0000 7f9e1f800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e18155550 0x7f9e18072660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.519+0000 7f9e1f800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e1815f800 0x7f9e18072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.519+0000 7f9e1f800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1815fc80 con 0x7f9e180d07c0 2024-04-04T06:07:35.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.519+0000 7f9e1f800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9e180b4c30 con 0x7f9e1815f800 2024-04-04T06:07:35.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.519+0000 7f9e1f800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9e1815ba20 con 0x7f9e18155550 2024-04-04T06:07:35.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.519+0000 7f9e1ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e180d07c0 0x7f9e1814e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.519+0000 7f9e1ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e180d07c0 0x7f9e1814e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56752/0 (socket says 172.21.15.159:56752) 2024-04-04T06:07:35.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.519+0000 7f9e1ce00640 1 -- 172.21.15.159:0/4253996660 learned_addr learned my addr 172.21.15.159:0/4253996660 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:35.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.519+0000 7f9e17e00640 1 --2- 172.21.15.159:0/4253996660 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e18155550 0x7f9e18072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.519+0000 7f9e1d800640 1 --2- 172.21.15.159:0/4253996660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e1815f800 0x7f9e18072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.519+0000 7f9e1ce00640 1 -- 172.21.15.159:0/4253996660 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e18155550 msgr2=0x7f9e18072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:35.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.519+0000 7f9e1ce00640 1 --2- 172.21.15.159:0/4253996660 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e18155550 0x7f9e18072660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.519+0000 7f9e1ce00640 1 -- 172.21.15.159:0/4253996660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e1815f800 msgr2=0x7f9e18072ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:35.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.519+0000 7f9e1ce00640 1 --2- 172.21.15.159:0/4253996660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e1815f800 0x7f9e18072ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.519+0000 7f9e1ce00640 1 -- 172.21.15.159:0/4253996660 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e180cc7b0 con 0x7f9e180d07c0 2024-04-04T06:07:35.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.520+0000 7f9e1ce00640 1 --2- 172.21.15.159:0/4253996660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e180d07c0 0x7f9e1814e900 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000c7f0 tx=0x7f9e1000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:35.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.520+0000 7f9e14c00640 1 -- 172.21.15.159:0/4253996660 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e10015070 con 0x7f9e180d07c0 2024-04-04T06:07:35.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.520+0000 7f9e14c00640 1 -- 172.21.15.159:0/4253996660 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e10004240 con 0x7f9e180d07c0 2024-04-04T06:07:35.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.520+0000 7f9e14c00640 1 -- 172.21.15.159:0/4253996660 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e10007620 con 0x7f9e180d07c0 2024-04-04T06:07:35.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.520+0000 7f9e1f800640 1 -- 172.21.15.159:0/4253996660 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e18153a90 con 0x7f9e180d07c0 2024-04-04T06:07:35.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.520+0000 7f9e1f800640 1 -- 172.21.15.159:0/4253996660 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e18154050 con 0x7f9e180d07c0 2024-04-04T06:07:35.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.521+0000 7f9e1f800640 1 -- 172.21.15.159:0/4253996660 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e180b12c0 con 0x7f9e180d07c0 2024-04-04T06:07:35.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.522+0000 7f9e14c00640 1 -- 172.21.15.159:0/4253996660 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e100079d0 con 0x7f9e180d07c0 2024-04-04T06:07:35.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.523+0000 7f9e14c00640 1 --2- 172.21.15.159:0/4253996660 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9df406c920 0x7f9df406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.523+0000 7f9e14c00640 1 -- 172.21.15.159:0/4253996660 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f9e1008bbc0 con 0x7f9e180d07c0 2024-04-04T06:07:35.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.524+0000 7f9e17e00640 1 --2- 172.21.15.159:0/4253996660 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9df406c920 0x7f9df406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.525+0000 7f9e17e00640 1 --2- 172.21.15.159:0/4253996660 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9df406c920 0x7f9df406ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9e00035000 tx=0x7f9e00030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:35.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.528+0000 7f9e14c00640 1 -- 172.21.15.159:0/4253996660 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9e180b12c0 con 0x7f9e180d07c0 2024-04-04T06:07:35.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.729+0000 7f9e1f800640 1 -- 172.21.15.159:0/4253996660 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9e180685c0 con 0x7f9e180d07c0 2024-04-04T06:07:35.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.730+0000 7f9e14c00640 1 -- 172.21.15.159:0/4253996660 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f9e180cc7b0 con 0x7f9e180d07c0 2024-04-04T06:07:35.725 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:07:35.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.733+0000 7f9e1f800640 1 -- 172.21.15.159:0/4253996660 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9df406c920 msgr2=0x7f9df406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:35.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.733+0000 7f9e1f800640 1 --2- 172.21.15.159:0/4253996660 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9df406c920 0x7f9df406ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9e00035000 tx=0x7f9e00030000 comp rx=0 tx=0).stop 2024-04-04T06:07:35.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.733+0000 7f9e1f800640 1 -- 172.21.15.159:0/4253996660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e180d07c0 msgr2=0x7f9e1814e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:35.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.733+0000 7f9e1f800640 1 --2- 172.21.15.159:0/4253996660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e180d07c0 0x7f9e1814e900 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000c7f0 tx=0x7f9e1000ccb0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.733+0000 7f9e1f800640 1 -- 172.21.15.159:0/4253996660 shutdown_connections 2024-04-04T06:07:35.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.733+0000 7f9e1f800640 1 --2- 172.21.15.159:0/4253996660 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9df406c920 0x7f9df406ede0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.733+0000 7f9e1f800640 1 --2- 172.21.15.159:0/4253996660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e1815f800 0x7f9e18072ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.733+0000 7f9e1f800640 1 --2- 172.21.15.159:0/4253996660 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e18155550 0x7f9e18072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.733+0000 7f9e1f800640 1 --2- 172.21.15.159:0/4253996660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e180d07c0 0x7f9e1814e900 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.733+0000 7f9e1f800640 1 -- 172.21.15.159:0/4253996660 >> 172.21.15.159:0/4253996660 conn(0x7f9e1806b750 msgr2=0x7f9e1806dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:35.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.733+0000 7f9e1f800640 1 -- 172.21.15.159:0/4253996660 shutdown_connections 2024-04-04T06:07:35.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.733+0000 7f9e1f800640 1 -- 172.21.15.159:0/4253996660 wait complete. 2024-04-04T06:07:35.747 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:07:35.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.962+0000 7f821fe00640 1 Processor -- start 2024-04-04T06:07:35.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.962+0000 7f821fe00640 1 -- start start 2024-04-04T06:07:35.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.964+0000 7f821fe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f822006dd30 0x7f82200cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.964+0000 7f821fe00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82200ce890 con 0x7f82200cc880 2024-04-04T06:07:35.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.964+0000 7f821fe00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82200ceb50 con 0x7f822013f190 2024-04-04T06:07:35.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.964+0000 7f821fe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f82200cd230 con 0x7f822006dd30 2024-04-04T06:07:35.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.964+0000 7f8216000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f82200cc880 0x7f822006d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43876/0 (socket says 172.21.15.159:43876) 2024-04-04T06:07:35.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.964+0000 7f8216000640 1 -- 172.21.15.159:0/2524453896 learned_addr learned my addr 172.21.15.159:0/2524453896 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:35.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.965+0000 7f821f400640 1 --2- 172.21.15.159:0/2524453896 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f822006dd30 0x7f82200cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.965+0000 7f821e000640 1 -- 172.21.15.159:0/2524453896 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1964119266 0 0) 0x7f82200ce890 con 0x7f82200cc880 2024-04-04T06:07:35.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.965+0000 7f821e000640 1 -- 172.21.15.159:0/2524453896 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8204003520 con 0x7f82200cc880 2024-04-04T06:07:35.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.965+0000 7f821e000640 1 -- 172.21.15.159:0/2524453896 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1290687831 0 0) 0x7f8204003520 con 0x7f82200cc880 2024-04-04T06:07:35.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.965+0000 7f821e000640 1 -- 172.21.15.159:0/2524453896 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f822006dd30 msgr2=0x7f82200cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:35.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.965+0000 7f821e000640 1 --2- 172.21.15.159:0/2524453896 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f822006dd30 0x7f82200cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.965+0000 7f821e000640 1 -- 172.21.15.159:0/2524453896 >> v1:172.21.15.178:6789/0 conn(0x7f822013f190 legacy=0x7f82200cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:35.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.965+0000 7f821e000640 1 -- 172.21.15.159:0/2524453896 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82200cee30 con 0x7f82200cc880 2024-04-04T06:07:35.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.965+0000 7f821e000640 1 -- 172.21.15.159:0/2524453896 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f8208001690 con 0x7f82200cc880 2024-04-04T06:07:35.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.965+0000 7f821e000640 1 -- 172.21.15.159:0/2524453896 >> v1:172.21.15.159:6789/0 conn(0x7f82200cc880 legacy=0x7f822006d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:35.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.965+0000 7f821e000640 1 --2- 172.21.15.159:0/2524453896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82040039e0 0x7f8204003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.965+0000 7f821e000640 1 --2- 172.21.15.159:0/2524453896 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f822006dd30 0x7f8204004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.965+0000 7f821e000640 1 --2- 172.21.15.159:0/2524453896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8204004d40 0x7f82040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.965+0000 7f821e000640 1 -- 172.21.15.159:0/2524453896 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f82200cd230 con 0x7f8204004d40 2024-04-04T06:07:35.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.965+0000 7f821e000640 1 -- 172.21.15.159:0/2524453896 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f820400bad0 con 0x7f82040039e0 2024-04-04T06:07:35.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.965+0000 7f821e000640 1 -- 172.21.15.159:0/2524453896 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f820400bcd0 con 0x7f822006dd30 2024-04-04T06:07:35.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.966+0000 7f821f400640 1 --2- 172.21.15.159:0/2524453896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8204004d40 0x7f82040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.966+0000 7f821ea00640 1 --2- 172.21.15.159:0/2524453896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82040039e0 0x7f8204003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.966+0000 7f8216000640 1 --2- 172.21.15.159:0/2524453896 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f822006dd30 0x7f8204004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.966+0000 7f821f400640 1 -- 172.21.15.159:0/2524453896 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f822006dd30 msgr2=0x7f8204004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:35.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.966+0000 7f821f400640 1 --2- 172.21.15.159:0/2524453896 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f822006dd30 0x7f8204004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.966+0000 7f821f400640 1 -- 172.21.15.159:0/2524453896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82040039e0 msgr2=0x7f8204003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:35.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.966+0000 7f821f400640 1 --2- 172.21.15.159:0/2524453896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82040039e0 0x7f8204003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.966+0000 7f821f400640 1 -- 172.21.15.159:0/2524453896 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8204003520 con 0x7f8204004d40 2024-04-04T06:07:35.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.966+0000 7f821ea00640 1 --2- 172.21.15.159:0/2524453896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82040039e0 0x7f8204003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:35.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.967+0000 7f821f400640 1 --2- 172.21.15.159:0/2524453896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8204004d40 0x7f82040051c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8218009f90 tx=0x7f8204005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfde38b22d4979a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:35.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.967+0000 7f821e000640 1 -- 172.21.15.159:0/2524453896 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218039070 con 0x7f8204004d40 2024-04-04T06:07:35.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.968+0000 7f821e000640 1 -- 172.21.15.159:0/2524453896 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f821802cbd0 con 0x7f8204004d40 2024-04-04T06:07:35.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.968+0000 7f821e000640 1 -- 172.21.15.159:0/2524453896 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218009c10 con 0x7f8204004d40 2024-04-04T06:07:35.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.968+0000 7f821fe00640 1 -- 172.21.15.159:0/2524453896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8204004d40 msgr2=0x7f82040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:35.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.968+0000 7f821fe00640 1 --2- 172.21.15.159:0/2524453896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8204004d40 0x7f82040051c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8218009f90 tx=0x7f8204005830 comp rx=0 tx=0).stop 2024-04-04T06:07:35.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.968+0000 7f821f400640 1 -- 172.21.15.159:0/2524453896 reap_dead start 2024-04-04T06:07:35.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.968+0000 7f821fe00640 1 -- 172.21.15.159:0/2524453896 shutdown_connections 2024-04-04T06:07:35.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.968+0000 7f821fe00640 1 -- 172.21.15.159:0/2524453896 >> 172.21.15.159:0/2524453896 conn(0x7f82200b1ba0 msgr2=0x7f82200b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:35.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.968+0000 7f821fe00640 1 -- 172.21.15.159:0/2524453896 shutdown_connections 2024-04-04T06:07:35.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.968+0000 7f821fe00640 1 -- 172.21.15.159:0/2524453896 wait complete. 2024-04-04T06:07:35.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.970+0000 7f821fe00640 1 Processor -- start 2024-04-04T06:07:35.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.970+0000 7f821fe00640 1 -- start start 2024-04-04T06:07:35.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.971+0000 7f821fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8204004d40 0x7f82201e8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.971+0000 7f821fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82201f45b0 0x7f82201f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.971+0000 7f821fe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82200d0080 0x7f82200d0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.971+0000 7f821fe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f822013f570 con 0x7f82201f45b0 2024-04-04T06:07:35.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.971+0000 7f821fe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8220160c70 con 0x7f8204004d40 2024-04-04T06:07:35.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.971+0000 7f821fe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f82200cccf0 con 0x7f82200d0080 2024-04-04T06:07:35.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.972+0000 7f8216000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82201f45b0 0x7f82201f4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.972+0000 7f8216000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82201f45b0 0x7f82201f4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56762/0 (socket says 172.21.15.159:56762) 2024-04-04T06:07:35.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.972+0000 7f8216000640 1 -- 172.21.15.159:0/256639482 learned_addr learned my addr 172.21.15.159:0/256639482 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:35.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.972+0000 7f821f400640 1 --2- 172.21.15.159:0/256639482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82200d0080 0x7f82200d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.972+0000 7f8216000640 1 -- 172.21.15.159:0/256639482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82200d0080 msgr2=0x7f82200d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:35.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.972+0000 7f8216000640 1 --2- 172.21.15.159:0/256639482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82200d0080 0x7f82200d0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.972+0000 7f8216000640 1 -- 172.21.15.159:0/256639482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8204004d40 msgr2=0x7f82201e8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:35.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.972+0000 7f821ea00640 1 --2- 172.21.15.159:0/256639482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8204004d40 0x7f82201e8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.972+0000 7f8216000640 1 --2- 172.21.15.159:0/256639482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8204004d40 0x7f82201e8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:35.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.972+0000 7f8216000640 1 -- 172.21.15.159:0/256639482 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82201f5100 con 0x7f82201f45b0 2024-04-04T06:07:35.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.972+0000 7f8216000640 1 --2- 172.21.15.159:0/256639482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82201f45b0 0x7f82201f4a30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f820800d810 tx=0x7f820800dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:35.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.973+0000 7f8217e00640 1 -- 172.21.15.159:0/256639482 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8208019070 con 0x7f82201f45b0 2024-04-04T06:07:35.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.973+0000 7f8217e00640 1 -- 172.21.15.159:0/256639482 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82080074e0 con 0x7f82201f45b0 2024-04-04T06:07:35.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.973+0000 7f821fe00640 1 -- 172.21.15.159:0/256639482 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82201f53e0 con 0x7f82201f45b0 2024-04-04T06:07:35.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.973+0000 7f8217e00640 1 -- 172.21.15.159:0/256639482 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82080049e0 con 0x7f82201f45b0 2024-04-04T06:07:35.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.973+0000 7f821fe00640 1 -- 172.21.15.159:0/256639482 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82201f5de0 con 0x7f82201f45b0 2024-04-04T06:07:35.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.974+0000 7f821fe00640 1 -- 172.21.15.159:0/256639482 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82200d0a40 con 0x7f82201f45b0 2024-04-04T06:07:35.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.975+0000 7f8217e00640 1 -- 172.21.15.159:0/256639482 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f820801c070 con 0x7f82201f45b0 2024-04-04T06:07:35.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.976+0000 7f8217e00640 1 --2- 172.21.15.159:0/256639482 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f81fc06c8f0 0x7f81fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:35.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.976+0000 7f8217e00640 1 -- 172.21.15.159:0/256639482 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f820808b6c0 con 0x7f82201f45b0 2024-04-04T06:07:35.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.976+0000 7f821ea00640 1 --2- 172.21.15.159:0/256639482 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f81fc06c8f0 0x7f81fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:35.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.978+0000 7f821ea00640 1 --2- 172.21.15.159:0/256639482 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f81fc06c8f0 0x7f81fc06edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8210000f60 tx=0x7f8210009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:35.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:35.981+0000 7f8217e00640 1 -- 172.21.15.159:0/256639482 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8208059b10 con 0x7f82201f45b0 2024-04-04T06:07:36.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.179+0000 7f821fe00640 1 -- 172.21.15.159:0/256639482 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8220075940 con 0x7f82201f45b0 2024-04-04T06:07:36.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.180+0000 7f8217e00640 1 -- 172.21.15.159:0/256639482 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7f8208059410 con 0x7f82201f45b0 2024-04-04T06:07:36.174 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:07:36.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.182+0000 7f821fe00640 1 -- 172.21.15.159:0/256639482 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f81fc06c8f0 msgr2=0x7f81fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.182+0000 7f821fe00640 1 --2- 172.21.15.159:0/256639482 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f81fc06c8f0 0x7f81fc06edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8210000f60 tx=0x7f8210009040 comp rx=0 tx=0).stop 2024-04-04T06:07:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.182+0000 7f821fe00640 1 -- 172.21.15.159:0/256639482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82201f45b0 msgr2=0x7f82201f4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.182+0000 7f821fe00640 1 --2- 172.21.15.159:0/256639482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82201f45b0 0x7f82201f4a30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f820800d810 tx=0x7f820800dcd0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.182+0000 7f821fe00640 1 -- 172.21.15.159:0/256639482 shutdown_connections 2024-04-04T06:07:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.182+0000 7f821fe00640 1 --2- 172.21.15.159:0/256639482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82200d0080 0x7f82200d0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.182+0000 7f821fe00640 1 --2- 172.21.15.159:0/256639482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82201f45b0 0x7f82201f4a30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.182+0000 7f821fe00640 1 --2- 172.21.15.159:0/256639482 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f81fc06c8f0 0x7f81fc06edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.182+0000 7f821fe00640 1 --2- 172.21.15.159:0/256639482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8204004d40 0x7f82201e8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.182+0000 7f821fe00640 1 -- 172.21.15.159:0/256639482 >> 172.21.15.159:0/256639482 conn(0x7f82200b1ba0 msgr2=0x7f822006e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.182+0000 7f821fe00640 1 -- 172.21.15.159:0/256639482 shutdown_connections 2024-04-04T06:07:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.183+0000 7f821fe00640 1 -- 172.21.15.159:0/256639482 wait complete. 2024-04-04T06:07:36.194 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:07:36.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.412+0000 7f2794000640 1 Processor -- start 2024-04-04T06:07:36.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.413+0000 7f2794000640 1 -- start start 2024-04-04T06:07:36.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.414+0000 7f2794000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f278c15e590 0x7f278c0d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:36.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.414+0000 7f2794000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f278c0b47d0 0x7f278c06c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:36.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.414+0000 7f2794000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f278c06cbc0 0x7f278c162230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:36.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.414+0000 7f2794000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f278c06cff0 con 0x7f278c0b47d0 2024-04-04T06:07:36.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.414+0000 7f2794000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f278c071060 con 0x7f278c06cbc0 2024-04-04T06:07:36.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.414+0000 7f2794000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f278c071340 con 0x7f278c15e590 2024-04-04T06:07:36.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.414+0000 7f2790c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f278c0b47d0 0x7f278c06c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:36.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.415+0000 7f2790c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f278c0b47d0 0x7f278c06c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56770/0 (socket says 172.21.15.159:56770) 2024-04-04T06:07:36.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.415+0000 7f2790c00640 1 -- 172.21.15.159:0/2239246266 learned_addr learned my addr 172.21.15.159:0/2239246266 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:36.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.415+0000 7f2791600640 1 --2- 172.21.15.159:0/2239246266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f278c15e590 0x7f278c0d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:36.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.415+0000 7f2792000640 1 --2- 172.21.15.159:0/2239246266 >> v2:172.21.15.178:3300/0 conn(0x7f278c06cbc0 0x7f278c162230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:36.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.415+0000 7f2790c00640 1 -- 172.21.15.159:0/2239246266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f278c15e590 msgr2=0x7f278c0d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:36.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.415+0000 7f2790c00640 1 --2- 172.21.15.159:0/2239246266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f278c15e590 0x7f278c0d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.415+0000 7f2790c00640 1 -- 172.21.15.159:0/2239246266 >> v2:172.21.15.178:3300/0 conn(0x7f278c06cbc0 msgr2=0x7f278c162230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:36.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.415+0000 7f2790c00640 1 --2- 172.21.15.159:0/2239246266 >> v2:172.21.15.178:3300/0 conn(0x7f278c06cbc0 0x7f278c162230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.415+0000 7f2790c00640 1 -- 172.21.15.159:0/2239246266 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278c071620 con 0x7f278c0b47d0 2024-04-04T06:07:36.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.416+0000 7f2790c00640 1 --2- 172.21.15.159:0/2239246266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f278c0b47d0 0x7f278c06c680 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f278c070c90 tx=0x7f277802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb8bb73895403b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:36.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.416+0000 7f278be00640 1 -- 172.21.15.159:0/2239246266 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778035020 con 0x7f278c0b47d0 2024-04-04T06:07:36.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.416+0000 7f278be00640 1 -- 172.21.15.159:0/2239246266 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2778031870 con 0x7f278c0b47d0 2024-04-04T06:07:36.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.416+0000 7f278be00640 1 -- 172.21.15.159:0/2239246266 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778031c40 con 0x7f278c0b47d0 2024-04-04T06:07:36.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.417+0000 7f2794000640 1 -- 172.21.15.159:0/2239246266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f278c0b47d0 msgr2=0x7f278c06c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:36.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.417+0000 7f2794000640 1 --2- 172.21.15.159:0/2239246266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f278c0b47d0 0x7f278c06c680 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f278c070c90 tx=0x7f277802b910 comp rx=0 tx=0).stop 2024-04-04T06:07:36.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.417+0000 7f2794000640 1 -- 172.21.15.159:0/2239246266 shutdown_connections 2024-04-04T06:07:36.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.417+0000 7f2794000640 1 --2- 172.21.15.159:0/2239246266 >> v2:172.21.15.178:3300/0 conn(0x7f278c06cbc0 0x7f278c162230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.417+0000 7f2794000640 1 --2- 172.21.15.159:0/2239246266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f278c0b47d0 0x7f278c06c680 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.417+0000 7f2794000640 1 --2- 172.21.15.159:0/2239246266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f278c15e590 0x7f278c0d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.417+0000 7f2794000640 1 -- 172.21.15.159:0/2239246266 >> 172.21.15.159:0/2239246266 conn(0x7f278c0d2240 msgr2=0x7f278c0d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:36.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.417+0000 7f2794000640 1 -- 172.21.15.159:0/2239246266 shutdown_connections 2024-04-04T06:07:36.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.417+0000 7f2794000640 1 -- 172.21.15.159:0/2239246266 wait complete. 2024-04-04T06:07:36.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.419+0000 7f2794000640 1 Processor -- start 2024-04-04T06:07:36.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.420+0000 7f2794000640 1 -- start start 2024-04-04T06:07:36.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.421+0000 7f2794000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f278c06cbc0 0x7f278c14cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:36.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.421+0000 7f2794000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f278c0b47d0 0x7f278c0b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:36.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.421+0000 7f2794000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f278c15e590 0x7f278c0b3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:36.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.421+0000 7f2794000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f278c0b4c30 con 0x7f278c15e590 2024-04-04T06:07:36.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.421+0000 7f2794000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f278c071be0 con 0x7f278c06cbc0 2024-04-04T06:07:36.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.421+0000 7f2794000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f278c15e970 con 0x7f278c0b47d0 2024-04-04T06:07:36.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.421+0000 7f2792000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f278c15e590 0x7f278c0b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:36.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.421+0000 7f2792000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f278c15e590 0x7f278c0b3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56778/0 (socket says 172.21.15.159:56778) 2024-04-04T06:07:36.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.421+0000 7f2792000640 1 -- 172.21.15.159:0/3782412661 learned_addr learned my addr 172.21.15.159:0/3782412661 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:36.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.421+0000 7f2790c00640 1 --2- 172.21.15.159:0/3782412661 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f278c0b47d0 0x7f278c0b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:36.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.421+0000 7f2791600640 1 --2- 172.21.15.159:0/3782412661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f278c06cbc0 0x7f278c14cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:36.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.421+0000 7f2792000640 1 -- 172.21.15.159:0/3782412661 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f278c0b47d0 msgr2=0x7f278c0b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:36.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.421+0000 7f2792000640 1 --2- 172.21.15.159:0/3782412661 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f278c0b47d0 0x7f278c0b33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.421+0000 7f2792000640 1 -- 172.21.15.159:0/3782412661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f278c06cbc0 msgr2=0x7f278c14cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:36.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.422+0000 7f2792000640 1 --2- 172.21.15.159:0/3782412661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f278c06cbc0 0x7f278c14cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.422+0000 7f2792000640 1 -- 172.21.15.159:0/3782412661 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278c0b5540 con 0x7f278c15e590 2024-04-04T06:07:36.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.422+0000 7f2791600640 1 --2- 172.21.15.159:0/3782412661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f278c06cbc0 0x7f278c14cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:36.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.422+0000 7f2792000640 1 --2- 172.21.15.159:0/3782412661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f278c15e590 0x7f278c0b3910 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f278400cc30 tx=0x7f278400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:36.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.422+0000 7f2789600640 1 -- 172.21.15.159:0/3782412661 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278400ce40 con 0x7f278c15e590 2024-04-04T06:07:36.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.422+0000 7f2789600640 1 -- 172.21.15.159:0/3782412661 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2784003990 con 0x7f278c15e590 2024-04-04T06:07:36.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.422+0000 7f2789600640 1 -- 172.21.15.159:0/3782412661 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2784003d40 con 0x7f278c15e590 2024-04-04T06:07:36.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.422+0000 7f2794000640 1 -- 172.21.15.159:0/3782412661 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278c0b5820 con 0x7f278c15e590 2024-04-04T06:07:36.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.422+0000 7f2794000640 1 -- 172.21.15.159:0/3782412661 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f278c0b5de0 con 0x7f278c15e590 2024-04-04T06:07:36.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.423+0000 7f2794000640 1 -- 172.21.15.159:0/3782412661 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f278c0685c0 con 0x7f278c15e590 2024-04-04T06:07:36.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.424+0000 7f2789600640 1 -- 172.21.15.159:0/3782412661 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2784003ee0 con 0x7f278c15e590 2024-04-04T06:07:36.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.426+0000 7f2789600640 1 --2- 172.21.15.159:0/3782412661 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f276c06c940 0x7f276c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:36.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.426+0000 7f2789600640 1 -- 172.21.15.159:0/3782412661 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f278408bfa0 con 0x7f278c15e590 2024-04-04T06:07:36.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.427+0000 7f2791600640 1 --2- 172.21.15.159:0/3782412661 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f276c06c940 0x7f276c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:36.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.428+0000 7f2791600640 1 --2- 172.21.15.159:0/3782412661 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f276c06c940 0x7f276c06ee00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f277c006fd0 tx=0x7f277c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:36.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.432+0000 7f2789600640 1 -- 172.21.15.159:0/3782412661 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2784014030 con 0x7f278c15e590 2024-04-04T06:07:36.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.633+0000 7f2794000640 1 -- 172.21.15.159:0/3782412661 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f278c162f50 con 0x7f278c15e590 2024-04-04T06:07:36.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.633+0000 7f2789600640 1 -- 172.21.15.159:0/3782412661 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7f278405a440 con 0x7f278c15e590 2024-04-04T06:07:36.628 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:07:36.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.636+0000 7f2794000640 1 -- 172.21.15.159:0/3782412661 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f276c06c940 msgr2=0x7f276c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:36.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.636+0000 7f2794000640 1 --2- 172.21.15.159:0/3782412661 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f276c06c940 0x7f276c06ee00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f277c006fd0 tx=0x7f277c008040 comp rx=0 tx=0).stop 2024-04-04T06:07:36.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.636+0000 7f2794000640 1 -- 172.21.15.159:0/3782412661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f278c15e590 msgr2=0x7f278c0b3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:36.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.636+0000 7f2794000640 1 --2- 172.21.15.159:0/3782412661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f278c15e590 0x7f278c0b3910 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f278400cc30 tx=0x7f278400a5c0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.636+0000 7f2794000640 1 -- 172.21.15.159:0/3782412661 shutdown_connections 2024-04-04T06:07:36.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.636+0000 7f2794000640 1 --2- 172.21.15.159:0/3782412661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f278c15e590 0x7f278c0b3910 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.636+0000 7f2794000640 1 --2- 172.21.15.159:0/3782412661 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f278c0b47d0 0x7f278c0b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.636+0000 7f2794000640 1 --2- 172.21.15.159:0/3782412661 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f276c06c940 0x7f276c06ee00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.636+0000 7f2794000640 1 --2- 172.21.15.159:0/3782412661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f278c06cbc0 0x7f278c14cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.636+0000 7f2794000640 1 -- 172.21.15.159:0/3782412661 >> 172.21.15.159:0/3782412661 conn(0x7f278c0d2240 msgr2=0x7f278c158950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:36.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.636+0000 7f2794000640 1 -- 172.21.15.159:0/3782412661 shutdown_connections 2024-04-04T06:07:36.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.636+0000 7f2794000640 1 -- 172.21.15.159:0/3782412661 wait complete. 2024-04-04T06:07:36.649 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:07:36.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.852+0000 7f45ac600640 1 Processor -- start 2024-04-04T06:07:36.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.852+0000 7f45ac600640 1 -- start start 2024-04-04T06:07:36.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.854+0000 7f45ac600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f45a415c390 0x7f45a4073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:36.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.854+0000 7f45ac600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45a40d0650 0x7f45a40d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:36.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.854+0000 7f45ac600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45a415fe80 con 0x7f45a40af830 2024-04-04T06:07:36.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.854+0000 7f45ac600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f45a415c770 con 0x7f45a415c390 2024-04-04T06:07:36.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.854+0000 7f45ac600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f45a40ad120 con 0x7f45a40d0650 2024-04-04T06:07:36.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.854+0000 7f45a9c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f45a415c390 0x7f45a4073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:36.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.854+0000 7f45a9c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f45a415c390 0x7f45a4073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56782/0 (socket says 172.21.15.159:56782) 2024-04-04T06:07:36.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.854+0000 7f45a9c00640 1 -- 172.21.15.159:0/2832063393 learned_addr learned my addr 172.21.15.159:0/2832063393 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:36.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.854+0000 7f45aa600640 1 --2- 172.21.15.159:0/2832063393 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45a40d0650 0x7f45a40d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:36.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.855+0000 7f45a9c00640 1 -- 172.21.15.159:0/2832063393 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45a40d0650 msgr2=0x7f45a40d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:36.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.855+0000 7f45a9c00640 1 --2- 172.21.15.159:0/2832063393 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45a40d0650 0x7f45a40d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.855+0000 7f45a9c00640 1 -- 172.21.15.159:0/2832063393 >> v1:172.21.15.178:6789/0 conn(0x7f45a40af830 legacy=0x7f45a4160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:36.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.855+0000 7f45a9c00640 1 -- 172.21.15.159:0/2832063393 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a40ad400 con 0x7f45a415c390 2024-04-04T06:07:36.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.856+0000 7f45a9c00640 1 --2- 172.21.15.159:0/2832063393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45a415c390 0x7f45a4073790 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4598009870 tx=0x7f459802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bfc52339509a6b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:36.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.856+0000 7f45a3e00640 1 -- 172.21.15.159:0/2832063393 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598034070 con 0x7f45a415c390 2024-04-04T06:07:36.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.856+0000 7f45a3e00640 1 -- 172.21.15.159:0/2832063393 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4598030840 con 0x7f45a415c390 2024-04-04T06:07:36.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.856+0000 7f45a3e00640 1 -- 172.21.15.159:0/2832063393 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598030c10 con 0x7f45a415c390 2024-04-04T06:07:36.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.857+0000 7f45ac600640 1 -- 172.21.15.159:0/2832063393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45a415c390 msgr2=0x7f45a4073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:36.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.857+0000 7f45ac600640 1 --2- 172.21.15.159:0/2832063393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45a415c390 0x7f45a4073790 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4598009870 tx=0x7f459802b980 comp rx=0 tx=0).stop 2024-04-04T06:07:36.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.857+0000 7f45ac600640 1 -- 172.21.15.159:0/2832063393 shutdown_connections 2024-04-04T06:07:36.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.857+0000 7f45ac600640 1 --2- 172.21.15.159:0/2832063393 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45a40d0650 0x7f45a40d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.857+0000 7f45ac600640 1 --2- 172.21.15.159:0/2832063393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45a415c390 0x7f45a4073790 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.857+0000 7f45ac600640 1 -- 172.21.15.159:0/2832063393 >> 172.21.15.159:0/2832063393 conn(0x7f45a406b990 msgr2=0x7f45a406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:36.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.857+0000 7f45ac600640 1 -- 172.21.15.159:0/2832063393 shutdown_connections 2024-04-04T06:07:36.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.857+0000 7f45ac600640 1 -- 172.21.15.159:0/2832063393 wait complete. 2024-04-04T06:07:36.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.859+0000 7f45ac600640 1 Processor -- start 2024-04-04T06:07:36.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.859+0000 7f45ac600640 1 -- start start 2024-04-04T06:07:36.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.860+0000 7f45ac600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45a40af830 0x7f45a41f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:36.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.860+0000 7f45ac600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a40d0650 0x7f45a40ce720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:36.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.860+0000 7f45ac600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45a415c390 0x7f45a4150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:36.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.860+0000 7f45ac600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f45a4160d50 con 0x7f45a40af830 2024-04-04T06:07:36.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.860+0000 7f45ac600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45a4160120 con 0x7f45a40d0650 2024-04-04T06:07:36.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.860+0000 7f45ac600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f45a40cf980 con 0x7f45a415c390 2024-04-04T06:07:36.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.860+0000 7f45a9c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45a40af830 0x7f45a41f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:36.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.860+0000 7f45a9c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45a40af830 0x7f45a41f3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56792/0 (socket says 172.21.15.159:56792) 2024-04-04T06:07:36.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.860+0000 7f45a9c00640 1 -- 172.21.15.159:0/451925429 learned_addr learned my addr 172.21.15.159:0/451925429 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:36.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.860+0000 7f45aa600640 1 --2- 172.21.15.159:0/451925429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45a415c390 0x7f45a4150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:36.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.860+0000 7f45a9200640 1 --2- 172.21.15.159:0/451925429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a40d0650 0x7f45a40ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:36.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.861+0000 7f45a9c00640 1 -- 172.21.15.159:0/451925429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45a415c390 msgr2=0x7f45a4150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:36.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.861+0000 7f45a9c00640 1 --2- 172.21.15.159:0/451925429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45a415c390 0x7f45a4150330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.861+0000 7f45a9c00640 1 -- 172.21.15.159:0/451925429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a40d0650 msgr2=0x7f45a40ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:36.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.861+0000 7f45a9c00640 1 --2- 172.21.15.159:0/451925429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a40d0650 0x7f45a40ce720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:36.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.861+0000 7f45a9c00640 1 -- 172.21.15.159:0/451925429 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a41f4350 con 0x7f45a40af830 2024-04-04T06:07:36.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.861+0000 7f45a9c00640 1 --2- 172.21.15.159:0/451925429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45a40af830 0x7f45a41f3e10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4598000c00 tx=0x7f45980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:36.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.861+0000 7f45a1600640 1 -- 172.21.15.159:0/451925429 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598034040 con 0x7f45a40af830 2024-04-04T06:07:36.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.861+0000 7f45a1600640 1 -- 172.21.15.159:0/451925429 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4598039d90 con 0x7f45a40af830 2024-04-04T06:07:36.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.861+0000 7f45ac600640 1 -- 172.21.15.159:0/451925429 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a40b3d80 con 0x7f45a40af830 2024-04-04T06:07:36.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.861+0000 7f45a1600640 1 -- 172.21.15.159:0/451925429 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598043610 con 0x7f45a40af830 2024-04-04T06:07:36.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.861+0000 7f45ac600640 1 -- 172.21.15.159:0/451925429 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a40b4340 con 0x7f45a40af830 2024-04-04T06:07:36.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.862+0000 7f45ac600640 1 -- 172.21.15.159:0/451925429 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45a40685c0 con 0x7f45a40af830 2024-04-04T06:07:36.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.863+0000 7f45a1600640 1 -- 172.21.15.159:0/451925429 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f459804a060 con 0x7f45a40af830 2024-04-04T06:07:36.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.865+0000 7f45a1600640 1 --2- 172.21.15.159:0/451925429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f458406c8f0 0x7f458406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:36.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.865+0000 7f45a1600640 1 -- 172.21.15.159:0/451925429 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f45980adbf0 con 0x7f45a40af830 2024-04-04T06:07:36.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.865+0000 7f45a9200640 1 --2- 172.21.15.159:0/451925429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f458406c8f0 0x7f458406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:36.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.867+0000 7f45a9200640 1 --2- 172.21.15.159:0/451925429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f458406c8f0 0x7f458406edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f45a40b0b80 tx=0x7f4590009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:36.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:36.869+0000 7f45a1600640 1 -- 172.21.15.159:0/451925429 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f459803e070 con 0x7f45a40af830 2024-04-04T06:07:37.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.113+0000 7f45ac600640 1 -- 172.21.15.159:0/451925429 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45a406f1b0 con 0x7f45a40af830 2024-04-04T06:07:37.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.115+0000 7f45a1600640 1 -- 172.21.15.159:0/451925429 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5193 (secure 0 0 0) 0x7f459802d330 con 0x7f45a40af830 2024-04-04T06:07:37.109 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:37.109 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1643905381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1643905381},{"type":"v1","addr":"172.21.15.178:6837","nonce":1643905381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4744,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2629539137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2629539137},{"type":"v1","addr":"172.21.15.159:6835","nonce":2629539137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3402172592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3402172592},{"type":"v1","addr":"172.21.15.159:6833","nonce":3402172592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:07:08.839645+0000","modified":"2024-04-04T06:07:11.927601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5157},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5157":{"gid":5157,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.178:6836/3282963883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3282963883},{"type":"v1","addr":"172.21.15.178:6836","nonce":3282963883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5157,"qdb_cluster":[5157]},"id":2}]} 2024-04-04T06:07:37.109 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 17 2024-04-04T06:07:37.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.117+0000 7f45ac600640 1 -- 172.21.15.159:0/451925429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f458406c8f0 msgr2=0x7f458406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:37.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.117+0000 7f45ac600640 1 --2- 172.21.15.159:0/451925429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f458406c8f0 0x7f458406edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f45a40b0b80 tx=0x7f4590009040 comp rx=0 tx=0).stop 2024-04-04T06:07:37.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.117+0000 7f45ac600640 1 -- 172.21.15.159:0/451925429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45a40af830 msgr2=0x7f45a41f3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:37.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.117+0000 7f45ac600640 1 --2- 172.21.15.159:0/451925429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45a40af830 0x7f45a41f3e10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4598000c00 tx=0x7f45980046c0 comp rx=0 tx=0).stop 2024-04-04T06:07:37.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.117+0000 7f45ac600640 1 -- 172.21.15.159:0/451925429 shutdown_connections 2024-04-04T06:07:37.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.117+0000 7f45ac600640 1 --2- 172.21.15.159:0/451925429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45a415c390 0x7f45a4150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:37.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.117+0000 7f45ac600640 1 --2- 172.21.15.159:0/451925429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f458406c8f0 0x7f458406edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:37.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.117+0000 7f45ac600640 1 --2- 172.21.15.159:0/451925429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45a40d0650 0x7f45a40ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:37.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.117+0000 7f45ac600640 1 --2- 172.21.15.159:0/451925429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45a40af830 0x7f45a41f3e10 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:37.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.117+0000 7f45ac600640 1 -- 172.21.15.159:0/451925429 >> 172.21.15.159:0/451925429 conn(0x7f45a406b990 msgr2=0x7f45a40afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:37.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.117+0000 7f45ac600640 1 -- 172.21.15.159:0/451925429 shutdown_connections 2024-04-04T06:07:37.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.117+0000 7f45ac600640 1 -- 172.21.15.159:0/451925429 wait complete. 2024-04-04T06:07:37.133 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:07:37.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.349+0000 7f5129400640 1 Processor -- start 2024-04-04T06:07:37.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.349+0000 7f5129400640 1 -- start start 2024-04-04T06:07:37.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.351+0000 7f5129400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f51240727f0 0x7f51240b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:37.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.351+0000 7f5129400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f512415b6f0 con 0x7f51240d0780 2024-04-04T06:07:37.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.351+0000 7f5129400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5124072e80 con 0x7f512415b310 2024-04-04T06:07:37.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.351+0000 7f5129400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f5124072c50 con 0x7f51240727f0 2024-04-04T06:07:37.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.351+0000 7f5119e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f51240727f0 0x7f51240b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:37.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.351+0000 7f5123200640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f51240d0780 0x7f51240adc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43878/0 (socket says 172.21.15.159:43878) 2024-04-04T06:07:37.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.351+0000 7f5119e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f51240727f0 0x7f51240b4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56798/0 (socket says 172.21.15.159:56798) 2024-04-04T06:07:37.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.351+0000 7f5123200640 1 -- 172.21.15.159:0/4241975931 learned_addr learned my addr 172.21.15.159:0/4241975931 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:37.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.352+0000 7f5119e00640 1 -- 172.21.15.159:0/4241975931 >> v1:172.21.15.178:6789/0 conn(0x7f512415b310 legacy=0x7f512415f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:37.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.352+0000 7f5119e00640 1 -- 172.21.15.159:0/4241975931 >> v1:172.21.15.159:6789/0 conn(0x7f51240d0780 legacy=0x7f51240adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:37.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.352+0000 7f5119e00640 1 -- 172.21.15.159:0/4241975931 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5124073160 con 0x7f51240727f0 2024-04-04T06:07:37.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.353+0000 7f5119e00640 1 --2- 172.21.15.159:0/4241975931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51240727f0 0x7f51240b4f90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f510c0097a0 tx=0x7f510c02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fa3936af9e154f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:37.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.353+0000 7f5121e00640 1 -- 172.21.15.159:0/4241975931 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c038070 con 0x7f51240727f0 2024-04-04T06:07:37.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.353+0000 7f5121e00640 1 -- 172.21.15.159:0/4241975931 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f510c031700 con 0x7f51240727f0 2024-04-04T06:07:37.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.353+0000 7f5121e00640 1 -- 172.21.15.159:0/4241975931 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c031ad0 con 0x7f51240727f0 2024-04-04T06:07:37.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.354+0000 7f5129400640 1 -- 172.21.15.159:0/4241975931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51240727f0 msgr2=0x7f51240b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:37.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.354+0000 7f5129400640 1 --2- 172.21.15.159:0/4241975931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51240727f0 0x7f51240b4f90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f510c0097a0 tx=0x7f510c02b7a0 comp rx=0 tx=0).stop 2024-04-04T06:07:37.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.354+0000 7f5129400640 1 -- 172.21.15.159:0/4241975931 shutdown_connections 2024-04-04T06:07:37.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.354+0000 7f5129400640 1 --2- 172.21.15.159:0/4241975931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51240727f0 0x7f51240b4f90 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:37.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.354+0000 7f5129400640 1 -- 172.21.15.159:0/4241975931 >> 172.21.15.159:0/4241975931 conn(0x7f512406b990 msgr2=0x7f512406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:37.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.354+0000 7f5129400640 1 -- 172.21.15.159:0/4241975931 shutdown_connections 2024-04-04T06:07:37.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.354+0000 7f5129400640 1 -- 172.21.15.159:0/4241975931 wait complete. 2024-04-04T06:07:37.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.356+0000 7f5129400640 1 Processor -- start 2024-04-04T06:07:37.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.356+0000 7f5129400640 1 -- start start 2024-04-04T06:07:37.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.357+0000 7f5129400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51240727f0 0x7f51241f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:37.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.357+0000 7f5129400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51240d0780 0x7f512415e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:37.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.357+0000 7f5129400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f512415b310 0x7f51241527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:37.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.357+0000 7f5129400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5124073440 con 0x7f51240d0780 2024-04-04T06:07:37.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.357+0000 7f5129400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f512415f2c0 con 0x7f51240727f0 2024-04-04T06:07:37.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.357+0000 7f5129400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f51240ae390 con 0x7f512415b310 2024-04-04T06:07:37.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.357+0000 7f5119e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51240d0780 0x7f512415e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:37.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.357+0000 7f5119e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51240d0780 0x7f512415e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56800/0 (socket says 172.21.15.159:56800) 2024-04-04T06:07:37.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.357+0000 7f5119e00640 1 -- 172.21.15.159:0/2855042682 learned_addr learned my addr 172.21.15.159:0/2855042682 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:37.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.357+0000 7f5122800640 1 --2- 172.21.15.159:0/2855042682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51240727f0 0x7f51241f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:37.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.357+0000 7f5123200640 1 --2- 172.21.15.159:0/2855042682 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f512415b310 0x7f51241527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:37.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.357+0000 7f5119e00640 1 -- 172.21.15.159:0/2855042682 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f512415b310 msgr2=0x7f51241527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:37.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.357+0000 7f5119e00640 1 --2- 172.21.15.159:0/2855042682 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f512415b310 0x7f51241527d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:37.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.357+0000 7f5119e00640 1 -- 172.21.15.159:0/2855042682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51240727f0 msgr2=0x7f51241f8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:37.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.357+0000 7f5119e00640 1 --2- 172.21.15.159:0/2855042682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51240727f0 0x7f51241f8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:37.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.357+0000 7f5119e00640 1 -- 172.21.15.159:0/2855042682 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512414df40 con 0x7f51240d0780 2024-04-04T06:07:37.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.358+0000 7f5119e00640 1 --2- 172.21.15.159:0/2855042682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51240d0780 0x7f512415e350 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f510c0098c0 tx=0x7f510c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:37.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.358+0000 7f511b400640 1 -- 172.21.15.159:0/2855042682 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c042070 con 0x7f51240d0780 2024-04-04T06:07:37.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.358+0000 7f511b400640 1 -- 172.21.15.159:0/2855042682 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f510c033a30 con 0x7f51240d0780 2024-04-04T06:07:37.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.358+0000 7f5129400640 1 -- 172.21.15.159:0/2855042682 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5124151a10 con 0x7f51240d0780 2024-04-04T06:07:37.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.358+0000 7f511b400640 1 -- 172.21.15.159:0/2855042682 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c038040 con 0x7f51240d0780 2024-04-04T06:07:37.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.358+0000 7f5129400640 1 -- 172.21.15.159:0/2855042682 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5124151fd0 con 0x7f51240d0780 2024-04-04T06:07:37.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.360+0000 7f5129400640 1 -- 172.21.15.159:0/2855042682 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51240685c0 con 0x7f51240d0780 2024-04-04T06:07:37.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.360+0000 7f511b400640 1 -- 172.21.15.159:0/2855042682 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f510c045070 con 0x7f51240d0780 2024-04-04T06:07:37.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.361+0000 7f511b400640 1 --2- 172.21.15.159:0/2855042682 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f510006c8f0 0x7f510006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:37.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.361+0000 7f511b400640 1 -- 172.21.15.159:0/2855042682 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f510c0ad8a0 con 0x7f51240d0780 2024-04-04T06:07:37.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.362+0000 7f5122800640 1 --2- 172.21.15.159:0/2855042682 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f510006c8f0 0x7f510006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:37.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.363+0000 7f5122800640 1 --2- 172.21.15.159:0/2855042682 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f510006c8f0 0x7f510006edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5114006fd0 tx=0x7f5114008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:37.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.366+0000 7f511b400640 1 -- 172.21.15.159:0/2855042682 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f510c030070 con 0x7f51240d0780 2024-04-04T06:07:37.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.569+0000 7f5129400640 1 -- 172.21.15.159:0/2855042682 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f512415ec20 con 0x7f51240d0780 2024-04-04T06:07:37.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.571+0000 7f511b400640 1 -- 172.21.15.159:0/2855042682 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16681 (secure 0 0 0) 0x7f510c07bdf0 con 0x7f51240d0780 2024-04-04T06:07:37.565 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:37.565 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":34,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:07:10.401521+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:07:06.708548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-04T06:07:07.833730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:07:37.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.573+0000 7f5129400640 1 -- 172.21.15.159:0/2855042682 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f510006c8f0 msgr2=0x7f510006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:37.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.573+0000 7f5129400640 1 --2- 172.21.15.159:0/2855042682 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f510006c8f0 0x7f510006edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5114006fd0 tx=0x7f5114008040 comp rx=0 tx=0).stop 2024-04-04T06:07:37.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.573+0000 7f5129400640 1 -- 172.21.15.159:0/2855042682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51240d0780 msgr2=0x7f512415e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:37.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.573+0000 7f5129400640 1 --2- 172.21.15.159:0/2855042682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51240d0780 0x7f512415e350 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f510c0098c0 tx=0x7f510c0046c0 comp rx=0 tx=0).stop 2024-04-04T06:07:37.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.574+0000 7f5129400640 1 -- 172.21.15.159:0/2855042682 shutdown_connections 2024-04-04T06:07:37.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.574+0000 7f5129400640 1 --2- 172.21.15.159:0/2855042682 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f512415b310 0x7f51241527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:37.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.574+0000 7f5129400640 1 --2- 172.21.15.159:0/2855042682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51240d0780 0x7f512415e350 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:37.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.574+0000 7f5129400640 1 --2- 172.21.15.159:0/2855042682 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f510006c8f0 0x7f510006edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:37.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.574+0000 7f5129400640 1 --2- 172.21.15.159:0/2855042682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51240727f0 0x7f51241f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:37.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.574+0000 7f5129400640 1 -- 172.21.15.159:0/2855042682 >> 172.21.15.159:0/2855042682 conn(0x7f512406b990 msgr2=0x7f512406bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:37.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.574+0000 7f5129400640 1 -- 172.21.15.159:0/2855042682 shutdown_connections 2024-04-04T06:07:37.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.574+0000 7f5129400640 1 -- 172.21.15.159:0/2855042682 wait complete. 2024-04-04T06:07:37.587 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-04T06:07:37.587 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:07:37.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.808+0000 7fd4d9800640 1 Processor -- start 2024-04-04T06:07:37.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.808+0000 7fd4d9800640 1 -- start start 2024-04-04T06:07:37.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.809+0000 7fd4d9800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd4d40cc0b0 0x7fd4d406dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:37.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.809+0000 7fd4d9800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4d415fc50 0x7fd4d40cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:37.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.809+0000 7fd4d9800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4d406e4f0 con 0x7fd4d40cc490 2024-04-04T06:07:37.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.809+0000 7fd4d9800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd4d415c610 con 0x7fd4d40cc0b0 2024-04-04T06:07:37.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.810+0000 7fd4d9800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d40b3150 con 0x7fd4d415fc50 2024-04-04T06:07:37.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.810+0000 7fd4d8e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fd4d40cc490 0x7fd4d415ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43882/0 (socket says 172.21.15.159:43882) 2024-04-04T06:07:37.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.810+0000 7fd4d8e00640 1 -- 172.21.15.159:0/4224200947 learned_addr learned my addr 172.21.15.159:0/4224200947 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:37.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.810+0000 7fd4d3e00640 1 --2- 172.21.15.159:0/4224200947 >> v2:172.21.15.178:3300/0 conn(0x7fd4d40cc0b0 0x7fd4d406dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:37.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.810+0000 7fd4cb400640 1 --2- 172.21.15.159:0/4224200947 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4d415fc50 0x7fd4d40cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:37.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.810+0000 7fd4d3400640 1 -- 172.21.15.159:0/4224200947 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3721590671 0 0) 0x7fd4d406e4f0 con 0x7fd4d40cc490 2024-04-04T06:07:37.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.810+0000 7fd4d3400640 1 -- 172.21.15.159:0/4224200947 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4b8003520 con 0x7fd4d40cc490 2024-04-04T06:07:37.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.811+0000 7fd4d3400640 1 -- 172.21.15.159:0/4224200947 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3559258689 0 0) 0x7fd4b8003520 con 0x7fd4d40cc490 2024-04-04T06:07:37.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.811+0000 7fd4d3400640 1 -- 172.21.15.159:0/4224200947 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4d415fc50 msgr2=0x7fd4d40cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:37.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.811+0000 7fd4d3400640 1 --2- 172.21.15.159:0/4224200947 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4d415fc50 0x7fd4d40cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:37.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.811+0000 7fd4d3400640 1 -- 172.21.15.159:0/4224200947 >> v2:172.21.15.178:3300/0 conn(0x7fd4d40cc0b0 msgr2=0x7fd4d406dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:37.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.811+0000 7fd4d3400640 1 --2- 172.21.15.159:0/4224200947 >> v2:172.21.15.178:3300/0 conn(0x7fd4d40cc0b0 0x7fd4d406dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:37.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.811+0000 7fd4d3400640 1 -- 172.21.15.159:0/4224200947 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d41600e0 con 0x7fd4d40cc490 2024-04-04T06:07:37.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.811+0000 7fd4d3400640 1 -- 172.21.15.159:0/4224200947 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fd4cc0019f0 con 0x7fd4d40cc490 2024-04-04T06:07:37.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.811+0000 7fd4d3400640 1 -- 172.21.15.159:0/4224200947 >> v1:172.21.15.159:6789/0 conn(0x7fd4d40cc490 legacy=0x7fd4d415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:37.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.811+0000 7fd4d3400640 1 --2- 172.21.15.159:0/4224200947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4b80039b0 0x7fd4b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:37.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.811+0000 7fd4d3400640 1 --2- 172.21.15.159:0/4224200947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4b8004800 0x7fd4b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:37.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.811+0000 7fd4d3400640 1 --2- 172.21.15.159:0/4224200947 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4b80051c0 0x7fd4b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:37.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.811+0000 7fd4d3400640 1 -- 172.21.15.159:0/4224200947 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d415c610 con 0x7fd4b80039b0 2024-04-04T06:07:37.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.811+0000 7fd4d3400640 1 -- 172.21.15.159:0/4224200947 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d40b3150 con 0x7fd4b8004800 2024-04-04T06:07:37.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.811+0000 7fd4d3400640 1 -- 172.21.15.159:0/4224200947 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd4b8004600 con 0x7fd4b80051c0 2024-04-04T06:07:37.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.811+0000 7fd4d3e00640 1 --2- 172.21.15.159:0/4224200947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4b80039b0 0x7fd4b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:37.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.812+0000 7fd4d3e00640 1 -- 172.21.15.159:0/4224200947 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4b80051c0 msgr2=0x7fd4b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:37.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.812+0000 7fd4d8e00640 1 --2- 172.21.15.159:0/4224200947 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4b80051c0 0x7fd4b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:37.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.812+0000 7fd4cb400640 1 --2- 172.21.15.159:0/4224200947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4b8004800 0x7fd4b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:37.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.812+0000 7fd4d3e00640 1 --2- 172.21.15.159:0/4224200947 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4b80051c0 0x7fd4b800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:37.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.812+0000 7fd4d3e00640 1 -- 172.21.15.159:0/4224200947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4b8004800 msgr2=0x7fd4b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:37.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.812+0000 7fd4d3e00640 1 --2- 172.21.15.159:0/4224200947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4b8004800 0x7fd4b8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:37.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.812+0000 7fd4d3e00640 1 -- 172.21.15.159:0/4224200947 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b8003520 con 0x7fd4b80039b0 2024-04-04T06:07:37.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.813+0000 7fd4d3e00640 1 --2- 172.21.15.159:0/4224200947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4b80039b0 0x7fd4b8003dd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4009a90 tx=0x7fd4b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4ccf0d2628d6a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:37.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.813+0000 7fd4d3400640 1 -- 172.21.15.159:0/4224200947 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4039070 con 0x7fd4b80039b0 2024-04-04T06:07:37.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.813+0000 7fd4d3400640 1 -- 172.21.15.159:0/4224200947 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c402c6b0 con 0x7fd4b80039b0 2024-04-04T06:07:37.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.813+0000 7fd4d3400640 1 -- 172.21.15.159:0/4224200947 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4009690 con 0x7fd4b80039b0 2024-04-04T06:07:37.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.814+0000 7fd4d9800640 1 -- 172.21.15.159:0/4224200947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4b80039b0 msgr2=0x7fd4b8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:37.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.814+0000 7fd4d9800640 1 --2- 172.21.15.159:0/4224200947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4b80039b0 0x7fd4b8003dd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4009a90 tx=0x7fd4b8005830 comp rx=0 tx=0).stop 2024-04-04T06:07:37.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.814+0000 7fd4d8e00640 1 -- 172.21.15.159:0/4224200947 reap_dead start 2024-04-04T06:07:37.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.814+0000 7fd4d9800640 1 -- 172.21.15.159:0/4224200947 shutdown_connections 2024-04-04T06:07:37.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.814+0000 7fd4d9800640 1 -- 172.21.15.159:0/4224200947 >> 172.21.15.159:0/4224200947 conn(0x7fd4d40ccad0 msgr2=0x7fd4d40cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:37.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.814+0000 7fd4d9800640 1 -- 172.21.15.159:0/4224200947 shutdown_connections 2024-04-04T06:07:37.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.814+0000 7fd4d9800640 1 -- 172.21.15.159:0/4224200947 wait complete. 2024-04-04T06:07:37.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.816+0000 7fd4d9800640 1 Processor -- start 2024-04-04T06:07:37.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.816+0000 7fd4d9800640 1 -- start start 2024-04-04T06:07:37.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.817+0000 7fd4d9800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4b80039b0 0x7fd4d41f3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:37.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.817+0000 7fd4d9800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4d413e210 0x7fd4d4129810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:37.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.817+0000 7fd4d9800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4d412b300 0x7fd4d412b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:37.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.817+0000 7fd4d9800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d4160980 con 0x7fd4d413e210 2024-04-04T06:07:37.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.817+0000 7fd4d9800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d41606a0 con 0x7fd4b80039b0 2024-04-04T06:07:37.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.817+0000 7fd4d9800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d415d180 con 0x7fd4d412b300 2024-04-04T06:07:37.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.817+0000 7fd4cb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4d413e210 0x7fd4d4129810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:37.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.817+0000 7fd4cb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4d413e210 0x7fd4d4129810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56826/0 (socket says 172.21.15.159:56826) 2024-04-04T06:07:37.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.817+0000 7fd4cb400640 1 -- 172.21.15.159:0/2808228961 learned_addr learned my addr 172.21.15.159:0/2808228961 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:37.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.817+0000 7fd4d3e00640 1 --2- 172.21.15.159:0/2808228961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4b80039b0 0x7fd4d41f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:37.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.817+0000 7fd4cb400640 1 -- 172.21.15.159:0/2808228961 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4d412b300 msgr2=0x7fd4d412b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:37.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.817+0000 7fd4cb400640 1 --2- 172.21.15.159:0/2808228961 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4d412b300 0x7fd4d412b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:37.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.817+0000 7fd4cb400640 1 -- 172.21.15.159:0/2808228961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4b80039b0 msgr2=0x7fd4d41f3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:37.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.817+0000 7fd4cb400640 1 --2- 172.21.15.159:0/2808228961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4b80039b0 0x7fd4d41f3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:37.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.817+0000 7fd4cb400640 1 -- 172.21.15.159:0/2808228961 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d412a180 con 0x7fd4d413e210 2024-04-04T06:07:37.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.818+0000 7fd4cb400640 1 --2- 172.21.15.159:0/2808228961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4d413e210 0x7fd4d4129810 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc010890 tx=0x7fd4bc010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:37.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.818+0000 7fd4d0c00640 1 -- 172.21.15.159:0/2808228961 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc00f040 con 0x7fd4d413e210 2024-04-04T06:07:37.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.818+0000 7fd4d0c00640 1 -- 172.21.15.159:0/2808228961 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4bc00eb60 con 0x7fd4d413e210 2024-04-04T06:07:37.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.818+0000 7fd4d0c00640 1 -- 172.21.15.159:0/2808228961 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc01c400 con 0x7fd4d413e210 2024-04-04T06:07:37.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.818+0000 7fd4d9800640 1 -- 172.21.15.159:0/2808228961 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d412a460 con 0x7fd4d413e210 2024-04-04T06:07:37.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.818+0000 7fd4d9800640 1 -- 172.21.15.159:0/2808228961 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d412aa20 con 0x7fd4d413e210 2024-04-04T06:07:37.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.819+0000 7fd4d9800640 1 -- 172.21.15.159:0/2808228961 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d4075940 con 0x7fd4d413e210 2024-04-04T06:07:37.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.820+0000 7fd4d0c00640 1 -- 172.21.15.159:0/2808228961 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd4bc01c5a0 con 0x7fd4d413e210 2024-04-04T06:07:37.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.822+0000 7fd4d0c00640 1 --2- 172.21.15.159:0/2808228961 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd4b006c920 0x7fd4b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:37.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.822+0000 7fd4d0c00640 1 -- 172.21.15.159:0/2808228961 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fd4bc08c150 con 0x7fd4d413e210 2024-04-04T06:07:37.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.822+0000 7fd4d3e00640 1 --2- 172.21.15.159:0/2808228961 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd4b006c920 0x7fd4b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:37.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.823+0000 7fd4d3e00640 1 --2- 172.21.15.159:0/2808228961 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd4b006c920 0x7fd4b006ede0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4036000 tx=0x7fd4c4030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:37.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:37.827+0000 7fd4d0c00640 1 -- 172.21.15.159:0/2808228961 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd4bc018030 con 0x7fd4d413e210 2024-04-04T06:07:38.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.073+0000 7fd4d9800640 1 -- 172.21.15.159:0/2808228961 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4d4073830 con 0x7fd4d413e210 2024-04-04T06:07:38.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.074+0000 7fd4d0c00640 1 -- 172.21.15.159:0/2808228961 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5193 (secure 0 0 0) 0x7fd4bc05d460 con 0x7fd4d413e210 2024-04-04T06:07:38.069 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:38.069 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1643905381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1643905381},{"type":"v1","addr":"172.21.15.178:6837","nonce":1643905381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4744,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2629539137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2629539137},{"type":"v1","addr":"172.21.15.159:6835","nonce":2629539137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3402172592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3402172592},{"type":"v1","addr":"172.21.15.159:6833","nonce":3402172592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:07:08.839645+0000","modified":"2024-04-04T06:07:11.927601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5157},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5157":{"gid":5157,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.178:6836/3282963883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3282963883},{"type":"v1","addr":"172.21.15.178:6836","nonce":3282963883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5157,"qdb_cluster":[5157]},"id":2}]} 2024-04-04T06:07:38.069 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 17 2024-04-04T06:07:38.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.077+0000 7fd4d9800640 1 -- 172.21.15.159:0/2808228961 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd4b006c920 msgr2=0x7fd4b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:38.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.077+0000 7fd4d9800640 1 --2- 172.21.15.159:0/2808228961 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd4b006c920 0x7fd4b006ede0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4036000 tx=0x7fd4c4030000 comp rx=0 tx=0).stop 2024-04-04T06:07:38.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.077+0000 7fd4d9800640 1 -- 172.21.15.159:0/2808228961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4d413e210 msgr2=0x7fd4d4129810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:38.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.077+0000 7fd4d9800640 1 --2- 172.21.15.159:0/2808228961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4d413e210 0x7fd4d4129810 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc010890 tx=0x7fd4bc010d50 comp rx=0 tx=0).stop 2024-04-04T06:07:38.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.077+0000 7fd4d9800640 1 -- 172.21.15.159:0/2808228961 shutdown_connections 2024-04-04T06:07:38.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.077+0000 7fd4d9800640 1 --2- 172.21.15.159:0/2808228961 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4d412b300 0x7fd4d412b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:38.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.077+0000 7fd4d9800640 1 --2- 172.21.15.159:0/2808228961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4d413e210 0x7fd4d4129810 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:38.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.077+0000 7fd4d9800640 1 --2- 172.21.15.159:0/2808228961 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd4b006c920 0x7fd4b006ede0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:38.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.077+0000 7fd4d9800640 1 --2- 172.21.15.159:0/2808228961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4b80039b0 0x7fd4d41f3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:38.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.077+0000 7fd4d9800640 1 -- 172.21.15.159:0/2808228961 >> 172.21.15.159:0/2808228961 conn(0x7fd4d40ccad0 msgr2=0x7fd4d40ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:38.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.077+0000 7fd4d9800640 1 -- 172.21.15.159:0/2808228961 shutdown_connections 2024-04-04T06:07:38.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.078+0000 7fd4d9800640 1 -- 172.21.15.159:0/2808228961 wait complete. 2024-04-04T06:07:38.092 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:07:38.305 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.311+0000 7fc2dca00640 1 Processor -- start 2024-04-04T06:07:38.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.311+0000 7fc2dca00640 1 -- start start 2024-04-04T06:07:38.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.312+0000 7fc2dca00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2d815fc50 0x7fc2d80cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:38.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.312+0000 7fc2dca00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc2d80cc490 0x7fc2d80d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:38.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.312+0000 7fc2dca00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2d80b3150 con 0x7fc2d80cc0b0 2024-04-04T06:07:38.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.312+0000 7fc2dca00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc2d815c610 con 0x7fc2d80cc490 2024-04-04T06:07:38.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.312+0000 7fc2dca00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc2d81600e0 con 0x7fc2d815fc50 2024-04-04T06:07:38.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.313+0000 7fc2d7e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc2d80cc490 0x7fc2d80d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:38.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.313+0000 7fc2cea00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2d815fc50 0x7fc2d80cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:38.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.313+0000 7fc2d7e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc2d80cc490 0x7fc2d80d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:42460/0 (socket says 172.21.15.159:42460) 2024-04-04T06:07:38.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.313+0000 7fc2d7400640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fc2d80cc0b0 0x7fc2d806dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.159:48428/0 (socket says 172.21.15.159:48428) 2024-04-04T06:07:38.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.313+0000 7fc2d7e00640 1 -- 172.21.15.159:0/2293453374 learned_addr learned my addr 172.21.15.159:0/2293453374 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:38.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.314+0000 7fc2d7e00640 1 -- 172.21.15.159:0/2293453374 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2d815fc50 msgr2=0x7fc2d80cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:38.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.314+0000 7fc2d7e00640 1 --2- 172.21.15.159:0/2293453374 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2d815fc50 0x7fc2d80cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:38.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.314+0000 7fc2d7e00640 1 -- 172.21.15.159:0/2293453374 >> v1:172.21.15.178:6789/0 conn(0x7fc2d80cc0b0 legacy=0x7fc2d806dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:38.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.314+0000 7fc2d7e00640 1 -- 172.21.15.159:0/2293453374 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d81603c0 con 0x7fc2d80cc490 2024-04-04T06:07:38.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.316+0000 7fc2d7e00640 1 --2- 172.21.15.159:0/2293453374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2d80cc490 0x7fc2d80d0300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009e50 tx=0x7fc2d002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a145864db1d52d84 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:38.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.316+0000 7fc2d6a00640 1 -- 172.21.15.159:0/2293453374 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d002cb00 con 0x7fc2d80cc490 2024-04-04T06:07:38.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.316+0000 7fc2d6a00640 1 -- 172.21.15.159:0/2293453374 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2d002cca0 con 0x7fc2d80cc490 2024-04-04T06:07:38.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.316+0000 7fc2d6a00640 1 -- 172.21.15.159:0/2293453374 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0034610 con 0x7fc2d80cc490 2024-04-04T06:07:38.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.317+0000 7fc2dca00640 1 -- 172.21.15.159:0/2293453374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2d80cc490 msgr2=0x7fc2d80d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:38.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.317+0000 7fc2dca00640 1 --2- 172.21.15.159:0/2293453374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2d80cc490 0x7fc2d80d0300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009e50 tx=0x7fc2d002bf60 comp rx=0 tx=0).stop 2024-04-04T06:07:38.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.317+0000 7fc2dca00640 1 -- 172.21.15.159:0/2293453374 shutdown_connections 2024-04-04T06:07:38.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.317+0000 7fc2dca00640 1 --2- 172.21.15.159:0/2293453374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2d80cc490 0x7fc2d80d0300 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:38.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.317+0000 7fc2dca00640 1 --2- 172.21.15.159:0/2293453374 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2d815fc50 0x7fc2d80cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:38.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.317+0000 7fc2dca00640 1 -- 172.21.15.159:0/2293453374 >> 172.21.15.159:0/2293453374 conn(0x7fc2d80ccad0 msgr2=0x7fc2d80cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:38.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.317+0000 7fc2dca00640 1 -- 172.21.15.159:0/2293453374 shutdown_connections 2024-04-04T06:07:38.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.317+0000 7fc2dca00640 1 -- 172.21.15.159:0/2293453374 wait complete. 2024-04-04T06:07:38.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.318+0000 7fc2dca00640 1 Processor -- start 2024-04-04T06:07:38.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.319+0000 7fc2dca00640 1 -- start start 2024-04-04T06:07:38.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.319+0000 7fc2dca00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2d80cc0b0 0x7fc2d81386f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:38.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.319+0000 7fc2dca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2d80cc490 0x7fc2d8139f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:38.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.319+0000 7fc2dca00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2d814c800 0x7fc2d80b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:38.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.319+0000 7fc2dca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d815ca70 con 0x7fc2d80cc490 2024-04-04T06:07:38.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.319+0000 7fc2dca00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d8160980 con 0x7fc2d814c800 2024-04-04T06:07:38.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.319+0000 7fc2dca00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc2d815cbf0 con 0x7fc2d80cc0b0 2024-04-04T06:07:38.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.320+0000 7fc2cea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2d80cc490 0x7fc2d8139f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:38.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.320+0000 7fc2cea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2d80cc490 0x7fc2d8139f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56838/0 (socket says 172.21.15.159:56838) 2024-04-04T06:07:38.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.320+0000 7fc2cea00640 1 -- 172.21.15.159:0/3157290302 learned_addr learned my addr 172.21.15.159:0/3157290302 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:38.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.320+0000 7fc2d7400640 1 --2- 172.21.15.159:0/3157290302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2d80cc0b0 0x7fc2d81386f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:38.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.320+0000 7fc2d7e00640 1 --2- 172.21.15.159:0/3157290302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2d814c800 0x7fc2d80b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:38.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.320+0000 7fc2cea00640 1 -- 172.21.15.159:0/3157290302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2d80cc0b0 msgr2=0x7fc2d81386f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:38.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.320+0000 7fc2cea00640 1 --2- 172.21.15.159:0/3157290302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2d80cc0b0 0x7fc2d81386f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:38.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.320+0000 7fc2cea00640 1 -- 172.21.15.159:0/3157290302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2d814c800 msgr2=0x7fc2d80b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:38.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.320+0000 7fc2cea00640 1 --2- 172.21.15.159:0/3157290302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2d814c800 0x7fc2d80b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:38.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.320+0000 7fc2cea00640 1 -- 172.21.15.159:0/3157290302 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d80b10e0 con 0x7fc2d80cc490 2024-04-04T06:07:38.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.321+0000 7fc2cea00640 1 --2- 172.21.15.159:0/3157290302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2d80cc490 0x7fc2d8139f60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000e870 tx=0x7fc2c000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:38.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.321+0000 7fc2cfe00640 1 -- 172.21.15.159:0/3157290302 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0017070 con 0x7fc2d80cc490 2024-04-04T06:07:38.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.321+0000 7fc2cfe00640 1 -- 172.21.15.159:0/3157290302 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2c000c8e0 con 0x7fc2d80cc490 2024-04-04T06:07:38.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.321+0000 7fc2dca00640 1 -- 172.21.15.159:0/3157290302 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d80b13c0 con 0x7fc2d80cc490 2024-04-04T06:07:38.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.321+0000 7fc2cfe00640 1 -- 172.21.15.159:0/3157290302 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c000ccd0 con 0x7fc2d80cc490 2024-04-04T06:07:38.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.321+0000 7fc2dca00640 1 -- 172.21.15.159:0/3157290302 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d80b1980 con 0x7fc2d80cc490 2024-04-04T06:07:38.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.323+0000 7fc2dca00640 1 -- 172.21.15.159:0/3157290302 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2d8075940 con 0x7fc2d80cc490 2024-04-04T06:07:38.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.323+0000 7fc2cfe00640 1 -- 172.21.15.159:0/3157290302 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc2c001a070 con 0x7fc2d80cc490 2024-04-04T06:07:38.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.324+0000 7fc2cfe00640 1 --2- 172.21.15.159:0/3157290302 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc2b406c920 0x7fc2b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:38.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.324+0000 7fc2cfe00640 1 -- 172.21.15.159:0/3157290302 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fc2c008bc40 con 0x7fc2d80cc490 2024-04-04T06:07:38.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.325+0000 7fc2d7400640 1 --2- 172.21.15.159:0/3157290302 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc2b406c920 0x7fc2b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:38.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.326+0000 7fc2d7400640 1 --2- 172.21.15.159:0/3157290302 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc2b406c920 0x7fc2b406ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8000f60 tx=0x7fc2c8007cc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:38.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.329+0000 7fc2cfe00640 1 -- 172.21.15.159:0/3157290302 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc2c005a100 con 0x7fc2d80cc490 2024-04-04T06:07:38.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.566+0000 7fc2dca00640 1 -- 172.21.15.159:0/3157290302 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2d8145820 con 0x7fc2d80cc490 2024-04-04T06:07:38.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.566+0000 7fc2cfe00640 1 -- 172.21.15.159:0/3157290302 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5193 (secure 0 0 0) 0x7fc2c0059a00 con 0x7fc2d80cc490 2024-04-04T06:07:38.561 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:38.561 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1643905381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1643905381},{"type":"v1","addr":"172.21.15.178:6837","nonce":1643905381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4744,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2629539137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2629539137},{"type":"v1","addr":"172.21.15.159:6835","nonce":2629539137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3402172592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3402172592},{"type":"v1","addr":"172.21.15.159:6833","nonce":3402172592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:07:08.839645+0000","modified":"2024-04-04T06:07:11.927601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5157},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5157":{"gid":5157,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.178:6836/3282963883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3282963883},{"type":"v1","addr":"172.21.15.178:6836","nonce":3282963883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5157,"qdb_cluster":[5157]},"id":2}]} 2024-04-04T06:07:38.561 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 17 2024-04-04T06:07:38.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.569+0000 7fc2dca00640 1 -- 172.21.15.159:0/3157290302 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc2b406c920 msgr2=0x7fc2b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:38.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.569+0000 7fc2dca00640 1 --2- 172.21.15.159:0/3157290302 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc2b406c920 0x7fc2b406ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8000f60 tx=0x7fc2c8007cc0 comp rx=0 tx=0).stop 2024-04-04T06:07:38.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.569+0000 7fc2dca00640 1 -- 172.21.15.159:0/3157290302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2d80cc490 msgr2=0x7fc2d8139f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:38.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.569+0000 7fc2dca00640 1 --2- 172.21.15.159:0/3157290302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2d80cc490 0x7fc2d8139f60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000e870 tx=0x7fc2c000ed30 comp rx=0 tx=0).stop 2024-04-04T06:07:38.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.569+0000 7fc2dca00640 1 -- 172.21.15.159:0/3157290302 shutdown_connections 2024-04-04T06:07:38.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.569+0000 7fc2dca00640 1 --2- 172.21.15.159:0/3157290302 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc2b406c920 0x7fc2b406ede0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:38.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.569+0000 7fc2dca00640 1 --2- 172.21.15.159:0/3157290302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2d814c800 0x7fc2d80b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:38.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.569+0000 7fc2dca00640 1 --2- 172.21.15.159:0/3157290302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2d80cc490 0x7fc2d8139f60 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:38.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.569+0000 7fc2dca00640 1 --2- 172.21.15.159:0/3157290302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2d80cc0b0 0x7fc2d81386f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:38.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.569+0000 7fc2dca00640 1 -- 172.21.15.159:0/3157290302 >> 172.21.15.159:0/3157290302 conn(0x7fc2d80ccad0 msgr2=0x7fc2d80cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:38.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.569+0000 7fc2dca00640 1 -- 172.21.15.159:0/3157290302 shutdown_connections 2024-04-04T06:07:38.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.569+0000 7fc2dca00640 1 -- 172.21.15.159:0/3157290302 wait complete. 2024-04-04T06:07:38.582 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:07:38.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.781+0000 7f1dc9400640 1 Processor -- start 2024-04-04T06:07:38.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.781+0000 7f1dc9400640 1 -- start start 2024-04-04T06:07:38.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.783+0000 7f1dc9400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1dc40af830 0x7f1dc4073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:38.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.783+0000 7f1dc9400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc415fe80 con 0x7f1dc415c390 2024-04-04T06:07:38.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.783+0000 7f1dc9400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc40ad120 con 0x7f1dc40d0650 2024-04-04T06:07:38.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.783+0000 7f1dc9400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc415c770 con 0x7f1dc40af830 2024-04-04T06:07:38.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.783+0000 7f1dc2800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f1dc415c390 0x7f1dc4160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43884/0 (socket says 172.21.15.159:43884) 2024-04-04T06:07:38.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.783+0000 7f1dc2800640 1 -- 172.21.15.159:0/3861919470 learned_addr learned my addr 172.21.15.159:0/3861919470 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:38.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.783+0000 7f1dc1e00640 1 --2- 172.21.15.159:0/3861919470 >> v2:172.21.15.178:3300/0 conn(0x7f1dc40af830 0x7f1dc4073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:38.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.783+0000 7f1dc1400640 1 -- 172.21.15.159:0/3861919470 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1173872704 0 0) 0x7f1dc415fe80 con 0x7f1dc415c390 2024-04-04T06:07:38.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.783+0000 7f1dc1400640 1 -- 172.21.15.159:0/3861919470 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1da8003520 con 0x7f1dc415c390 2024-04-04T06:07:38.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.784+0000 7f1dc1400640 1 -- 172.21.15.159:0/3861919470 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1888194745 0 0) 0x7f1da8003520 con 0x7f1dc415c390 2024-04-04T06:07:38.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.784+0000 7f1dc1400640 1 -- 172.21.15.159:0/3861919470 >> v2:172.21.15.178:3300/0 conn(0x7f1dc40af830 msgr2=0x7f1dc4073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:38.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.784+0000 7f1dc1400640 1 --2- 172.21.15.159:0/3861919470 >> v2:172.21.15.178:3300/0 conn(0x7f1dc40af830 0x7f1dc4073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:38.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.784+0000 7f1dc1400640 1 -- 172.21.15.159:0/3861919470 >> v1:172.21.15.178:6789/0 conn(0x7f1dc40d0650 legacy=0x7f1dc4071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:38.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.784+0000 7f1dc1400640 1 -- 172.21.15.159:0/3861919470 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc40ad400 con 0x7f1dc415c390 2024-04-04T06:07:38.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.784+0000 7f1dc1400640 1 -- 172.21.15.159:0/3861919470 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f1db4001690 con 0x7f1dc415c390 2024-04-04T06:07:38.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.784+0000 7f1dc1400640 1 -- 172.21.15.159:0/3861919470 >> v1:172.21.15.159:6789/0 conn(0x7f1dc415c390 legacy=0x7f1dc4160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:38.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.784+0000 7f1dc1400640 1 --2- 172.21.15.159:0/3861919470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1da8003a00 0x7f1da8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:38.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.784+0000 7f1dc1400640 1 --2- 172.21.15.159:0/3861919470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1da8004800 0x7f1da8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:38.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.784+0000 7f1dc1400640 1 --2- 172.21.15.159:0/3861919470 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1da80051c0 0x7f1da800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:38.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.784+0000 7f1dc1400640 1 -- 172.21.15.159:0/3861919470 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1da800c090 con 0x7f1da8003a00 2024-04-04T06:07:38.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.784+0000 7f1dc1400640 1 -- 172.21.15.159:0/3861919470 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1da800c240 con 0x7f1da8004800 2024-04-04T06:07:38.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.784+0000 7f1dc1400640 1 -- 172.21.15.159:0/3861919470 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1da800c440 con 0x7f1da80051c0 2024-04-04T06:07:38.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.784+0000 7f1dc2800640 1 --2- 172.21.15.159:0/3861919470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1da8003a00 0x7f1da8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:38.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.785+0000 7f1dc1e00640 1 --2- 172.21.15.159:0/3861919470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1da8004800 0x7f1da8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:38.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.785+0000 7f1dc2800640 1 -- 172.21.15.159:0/3861919470 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1da80051c0 msgr2=0x7f1da800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:38.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.785+0000 7f1dc2800640 1 --2- 172.21.15.159:0/3861919470 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1da80051c0 0x7f1da800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:38.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.785+0000 7f1dc2800640 1 -- 172.21.15.159:0/3861919470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1da8004800 msgr2=0x7f1da8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:38.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.785+0000 7f1dc2800640 1 --2- 172.21.15.159:0/3861919470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1da8004800 0x7f1da8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:38.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.785+0000 7f1dc2800640 1 -- 172.21.15.159:0/3861919470 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da8003520 con 0x7f1da8003a00 2024-04-04T06:07:38.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.785+0000 7f1dc1e00640 1 --2- 172.21.15.159:0/3861919470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1da8004800 0x7f1da8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:38.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.785+0000 7f1dc3200640 1 -- 172.21.15.159:0/3861919470 reap_dead start 2024-04-04T06:07:38.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.786+0000 7f1dc2800640 1 --2- 172.21.15.159:0/3861919470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1da8003a00 0x7f1da8003e20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1db400c160 tx=0x7f1da8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf8bca0f4704a3a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:38.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.786+0000 7f1dc1400640 1 -- 172.21.15.159:0/3861919470 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4007c60 con 0x7f1da8003a00 2024-04-04T06:07:38.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.786+0000 7f1dc1400640 1 -- 172.21.15.159:0/3861919470 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db4030070 con 0x7f1da8003a00 2024-04-04T06:07:38.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.786+0000 7f1dc1400640 1 -- 172.21.15.159:0/3861919470 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4005d30 con 0x7f1da8003a00 2024-04-04T06:07:38.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.787+0000 7f1dc9400640 1 -- 172.21.15.159:0/3861919470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1da8003a00 msgr2=0x7f1da8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:38.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.787+0000 7f1dc9400640 1 --2- 172.21.15.159:0/3861919470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1da8003a00 0x7f1da8003e20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1db400c160 tx=0x7f1da8005830 comp rx=0 tx=0).stop 2024-04-04T06:07:38.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.787+0000 7f1dc9400640 1 -- 172.21.15.159:0/3861919470 shutdown_connections 2024-04-04T06:07:38.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.787+0000 7f1dc9400640 1 --2- 172.21.15.159:0/3861919470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1da8003a00 0x7f1da8003e20 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:38.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.787+0000 7f1dc9400640 1 -- 172.21.15.159:0/3861919470 >> 172.21.15.159:0/3861919470 conn(0x7f1dc406b990 msgr2=0x7f1dc406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:38.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.787+0000 7f1dc9400640 1 -- 172.21.15.159:0/3861919470 shutdown_connections 2024-04-04T06:07:38.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.787+0000 7f1dc9400640 1 -- 172.21.15.159:0/3861919470 wait complete. 2024-04-04T06:07:38.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.788+0000 7f1dc9400640 1 Processor -- start 2024-04-04T06:07:38.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.789+0000 7f1dc9400640 1 -- start start 2024-04-04T06:07:38.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.789+0000 7f1dc9400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1da8003a00 0x7f1dc4146630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:38.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.789+0000 7f1dc9400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dc41f2750 0x7f1dc41569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:38.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.789+0000 7f1dc9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1dc4156f10 0x7f1dc415ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:38.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.790+0000 7f1dc9400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc40cf980 con 0x7f1dc4156f10 2024-04-04T06:07:38.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.790+0000 7f1dc9400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4160120 con 0x7f1dc41f2750 2024-04-04T06:07:38.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.790+0000 7f1dc9400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc4073790 con 0x7f1da8003a00 2024-04-04T06:07:38.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.790+0000 7f1dc3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1dc4156f10 0x7f1dc415ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:38.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.790+0000 7f1dc3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1dc4156f10 0x7f1dc415ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56866/0 (socket says 172.21.15.159:56866) 2024-04-04T06:07:38.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.790+0000 7f1dc3200640 1 -- 172.21.15.159:0/855129429 learned_addr learned my addr 172.21.15.159:0/855129429 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:38.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.790+0000 7f1dc2800640 1 --2- 172.21.15.159:0/855129429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1da8003a00 0x7f1dc4146630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:38.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.790+0000 7f1dc3200640 1 -- 172.21.15.159:0/855129429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1da8003a00 msgr2=0x7f1dc4146630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:38.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.790+0000 7f1dc3200640 1 --2- 172.21.15.159:0/855129429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1da8003a00 0x7f1dc4146630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:38.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.790+0000 7f1dc3200640 1 -- 172.21.15.159:0/855129429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dc41f2750 msgr2=0x7f1dc41569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:38.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.790+0000 7f1dc3200640 1 --2- 172.21.15.159:0/855129429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dc41f2750 0x7f1dc41569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:38.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.790+0000 7f1dc3200640 1 -- 172.21.15.159:0/855129429 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc415dbf0 con 0x7f1dc4156f10 2024-04-04T06:07:38.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.790+0000 7f1dc3200640 1 --2- 172.21.15.159:0/855129429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1dc4156f10 0x7f1dc415ec90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40b63b0 tx=0x7f1dbc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:38.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.791+0000 7f1dbaa00640 1 -- 172.21.15.159:0/855129429 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc017070 con 0x7f1dc4156f10 2024-04-04T06:07:38.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.791+0000 7f1dbaa00640 1 -- 172.21.15.159:0/855129429 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dbc005d60 con 0x7f1dc4156f10 2024-04-04T06:07:38.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.791+0000 7f1dbaa00640 1 -- 172.21.15.159:0/855129429 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc00a850 con 0x7f1dc4156f10 2024-04-04T06:07:38.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.791+0000 7f1dc9400640 1 -- 172.21.15.159:0/855129429 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc415ded0 con 0x7f1dc4156f10 2024-04-04T06:07:38.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.791+0000 7f1dc9400640 1 -- 172.21.15.159:0/855129429 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc415e490 con 0x7f1dc4156f10 2024-04-04T06:07:38.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.792+0000 7f1dc9400640 1 -- 172.21.15.159:0/855129429 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc40685c0 con 0x7f1dc4156f10 2024-04-04T06:07:38.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.793+0000 7f1dbaa00640 1 -- 172.21.15.159:0/855129429 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1dbc01a070 con 0x7f1dc4156f10 2024-04-04T06:07:38.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.794+0000 7f1dbaa00640 1 --2- 172.21.15.159:0/855129429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1da406c8f0 0x7f1da406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:38.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.794+0000 7f1dbaa00640 1 -- 172.21.15.159:0/855129429 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f1dbc08ac90 con 0x7f1dc4156f10 2024-04-04T06:07:38.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.794+0000 7f1dc2800640 1 --2- 172.21.15.159:0/855129429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1da406c8f0 0x7f1da406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:38.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.795+0000 7f1dc2800640 1 --2- 172.21.15.159:0/855129429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1da406c8f0 0x7f1da406edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1db4034000 tx=0x7f1db4007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:38.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:38.798+0000 7f1dbaa00640 1 -- 172.21.15.159:0/855129429 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1dbc059020 con 0x7f1dc4156f10 2024-04-04T06:07:38.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.001+0000 7f1dc9400640 1 -- 172.21.15.159:0/855129429 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1dc406f1b0 con 0x7f1dc4156f10 2024-04-04T06:07:38.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.004+0000 7f1dbaa00640 1 -- 172.21.15.159:0/855129429 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16681 (secure 0 0 0) 0x7f1dbc058920 con 0x7f1dc4156f10 2024-04-04T06:07:38.999 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:38.999 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":34,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:07:10.401521+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:07:06.708548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-04T06:07:07.833730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:07:39.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.006+0000 7f1dc9400640 1 -- 172.21.15.159:0/855129429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1da406c8f0 msgr2=0x7f1da406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:39.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.006+0000 7f1dc9400640 1 --2- 172.21.15.159:0/855129429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1da406c8f0 0x7f1da406edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1db4034000 tx=0x7f1db4007000 comp rx=0 tx=0).stop 2024-04-04T06:07:39.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.006+0000 7f1dc9400640 1 -- 172.21.15.159:0/855129429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1dc4156f10 msgr2=0x7f1dc415ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:39.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.006+0000 7f1dc9400640 1 --2- 172.21.15.159:0/855129429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1dc4156f10 0x7f1dc415ec90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40b63b0 tx=0x7f1dbc00cb90 comp rx=0 tx=0).stop 2024-04-04T06:07:39.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.006+0000 7f1dc9400640 1 -- 172.21.15.159:0/855129429 shutdown_connections 2024-04-04T06:07:39.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.006+0000 7f1dc9400640 1 --2- 172.21.15.159:0/855129429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1dc4156f10 0x7f1dc415ec90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:39.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.006+0000 7f1dc9400640 1 --2- 172.21.15.159:0/855129429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1da406c8f0 0x7f1da406edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:39.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.006+0000 7f1dc9400640 1 --2- 172.21.15.159:0/855129429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dc41f2750 0x7f1dc41569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:39.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.006+0000 7f1dc9400640 1 --2- 172.21.15.159:0/855129429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1da8003a00 0x7f1dc4146630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:39.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.006+0000 7f1dc9400640 1 -- 172.21.15.159:0/855129429 >> 172.21.15.159:0/855129429 conn(0x7f1dc406b990 msgr2=0x7f1dc4158950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:39.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.007+0000 7f1dc9400640 1 -- 172.21.15.159:0/855129429 shutdown_connections 2024-04-04T06:07:39.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.007+0000 7f1dc9400640 1 -- 172.21.15.159:0/855129429 wait complete. 2024-04-04T06:07:39.021 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-04T06:07:39.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.238+0000 7f35ce600640 1 Processor -- start 2024-04-04T06:07:39.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.238+0000 7f35ce600640 1 -- start start 2024-04-04T06:07:39.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.239+0000 7f35ce600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35c8155550 0x7f35c8155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:39.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.239+0000 7f35ce600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f35c80d0990 0x7f35c80aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:39.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.239+0000 7f35ce600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35c80b40b0 con 0x7f35c815f800 2024-04-04T06:07:39.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.239+0000 7f35ce600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f35c80b4390 con 0x7f35c80d0990 2024-04-04T06:07:39.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.239+0000 7f35ce600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f35c80b4670 con 0x7f35c8155550 2024-04-04T06:07:39.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.240+0000 7f35c7e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f35c80d0990 0x7f35c80aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:39.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.240+0000 7f35c7e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f35c80d0990 0x7f35c80aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56882/0 (socket says 172.21.15.159:56882) 2024-04-04T06:07:39.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.240+0000 7f35c7e00640 1 -- 172.21.15.159:0/4083795495 learned_addr learned my addr 172.21.15.159:0/4083795495 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:39.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.240+0000 7f35c7400640 1 --2- 172.21.15.159:0/4083795495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35c8155550 0x7f35c8155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:39.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.240+0000 7f35c7e00640 1 -- 172.21.15.159:0/4083795495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35c8155550 msgr2=0x7f35c8155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:39.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.240+0000 7f35c7e00640 1 --2- 172.21.15.159:0/4083795495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35c8155550 0x7f35c8155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:39.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.240+0000 7f35c7e00640 1 -- 172.21.15.159:0/4083795495 >> v1:172.21.15.178:6789/0 conn(0x7f35c815f800 legacy=0x7f35c80d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:39.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.240+0000 7f35c7e00640 1 -- 172.21.15.159:0/4083795495 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c80b4950 con 0x7f35c80d0990 2024-04-04T06:07:39.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.242+0000 7f35c7e00640 1 --2- 172.21.15.159:0/4083795495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35c80d0990 0x7f35c80aede0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009e50 tx=0x7f35c002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b22500cc71a40e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:39.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.242+0000 7f35c6a00640 1 -- 172.21.15.159:0/4083795495 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c002ca70 con 0x7f35c80d0990 2024-04-04T06:07:39.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.242+0000 7f35c6a00640 1 -- 172.21.15.159:0/4083795495 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c002cc10 con 0x7f35c80d0990 2024-04-04T06:07:39.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.242+0000 7f35c6a00640 1 -- 172.21.15.159:0/4083795495 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c00343f0 con 0x7f35c80d0990 2024-04-04T06:07:39.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.243+0000 7f35ce600640 1 -- 172.21.15.159:0/4083795495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35c80d0990 msgr2=0x7f35c80aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:39.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.243+0000 7f35ce600640 1 --2- 172.21.15.159:0/4083795495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35c80d0990 0x7f35c80aede0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009e50 tx=0x7f35c002bed0 comp rx=0 tx=0).stop 2024-04-04T06:07:39.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.243+0000 7f35ce600640 1 -- 172.21.15.159:0/4083795495 shutdown_connections 2024-04-04T06:07:39.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.243+0000 7f35ce600640 1 --2- 172.21.15.159:0/4083795495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35c80d0990 0x7f35c80aede0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:39.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.243+0000 7f35ce600640 1 --2- 172.21.15.159:0/4083795495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35c8155550 0x7f35c8155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:39.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.243+0000 7f35ce600640 1 -- 172.21.15.159:0/4083795495 >> 172.21.15.159:0/4083795495 conn(0x7f35c806b750 msgr2=0x7f35c806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:39.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.243+0000 7f35ce600640 1 -- 172.21.15.159:0/4083795495 shutdown_connections 2024-04-04T06:07:39.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.243+0000 7f35ce600640 1 -- 172.21.15.159:0/4083795495 wait complete. 2024-04-04T06:07:39.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.244+0000 7f35ce600640 1 Processor -- start 2024-04-04T06:07:39.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.245+0000 7f35ce600640 1 -- start start 2024-04-04T06:07:39.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.245+0000 7f35ce600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35c80d0990 0x7f35c809c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:39.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.246+0000 7f35ce600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35c8155550 0x7f35c81f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:39.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.246+0000 7f35ce600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35c815f800 0x7f35c81f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:39.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.246+0000 7f35ce600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f35c815ba20 con 0x7f35c815f800 2024-04-04T06:07:39.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.246+0000 7f35ce600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35c815fc80 con 0x7f35c80d0990 2024-04-04T06:07:39.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.246+0000 7f35ce600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f35c815bba0 con 0x7f35c8155550 2024-04-04T06:07:39.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.246+0000 7f35c7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35c815f800 0x7f35c81f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:39.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.246+0000 7f35c7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35c815f800 0x7f35c81f66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56886/0 (socket says 172.21.15.159:56886) 2024-04-04T06:07:39.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.246+0000 7f35c7e00640 1 -- 172.21.15.159:0/1924882318 learned_addr learned my addr 172.21.15.159:0/1924882318 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:39.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.246+0000 7f35c7400640 1 --2- 172.21.15.159:0/1924882318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35c80d0990 0x7f35c809c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:39.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.246+0000 7f35bea00640 1 --2- 172.21.15.159:0/1924882318 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35c8155550 0x7f35c81f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:39.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.246+0000 7f35c7e00640 1 -- 172.21.15.159:0/1924882318 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35c8155550 msgr2=0x7f35c81f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:39.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.246+0000 7f35c7e00640 1 --2- 172.21.15.159:0/1924882318 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35c8155550 0x7f35c81f61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:39.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.246+0000 7f35c7e00640 1 -- 172.21.15.159:0/1924882318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35c80d0990 msgr2=0x7f35c809c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:39.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.246+0000 7f35c7e00640 1 --2- 172.21.15.159:0/1924882318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35c80d0990 0x7f35c809c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:39.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.246+0000 7f35c7e00640 1 -- 172.21.15.159:0/1924882318 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c809b3f0 con 0x7f35c815f800 2024-04-04T06:07:39.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.246+0000 7f35c7400640 1 --2- 172.21.15.159:0/1924882318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35c80d0990 0x7f35c809c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:39.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.247+0000 7f35c7e00640 1 --2- 172.21.15.159:0/1924882318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35c815f800 0x7f35c81f66e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f35c0036040 tx=0x7f35c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:39.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.247+0000 7f35bfe00640 1 -- 172.21.15.159:0/1924882318 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c0043020 con 0x7f35c815f800 2024-04-04T06:07:39.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.247+0000 7f35bfe00640 1 -- 172.21.15.159:0/1924882318 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c003e450 con 0x7f35c815f800 2024-04-04T06:07:39.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.247+0000 7f35ce600640 1 -- 172.21.15.159:0/1924882318 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c80af0f0 con 0x7f35c815f800 2024-04-04T06:07:39.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.247+0000 7f35bfe00640 1 -- 172.21.15.159:0/1924882318 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c0009510 con 0x7f35c815f800 2024-04-04T06:07:39.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.247+0000 7f35ce600640 1 -- 172.21.15.159:0/1924882318 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c80998d0 con 0x7f35c815f800 2024-04-04T06:07:39.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.248+0000 7f35ce600640 1 -- 172.21.15.159:0/1924882318 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35c80685c0 con 0x7f35c815f800 2024-04-04T06:07:39.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.249+0000 7f35bfe00640 1 -- 172.21.15.159:0/1924882318 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35c0046020 con 0x7f35c815f800 2024-04-04T06:07:39.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.250+0000 7f35bfe00640 1 --2- 172.21.15.159:0/1924882318 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35a406c8f0 0x7f35a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:39.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.250+0000 7f35bfe00640 1 -- 172.21.15.159:0/1924882318 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f35c00ae3f0 con 0x7f35c815f800 2024-04-04T06:07:39.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.251+0000 7f35c7400640 1 --2- 172.21.15.159:0/1924882318 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35a406c8f0 0x7f35a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:39.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.252+0000 7f35c7400640 1 --2- 172.21.15.159:0/1924882318 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35a406c8f0 0x7f35a406edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f35b8006fd0 tx=0x7f35b8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:39.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.254+0000 7f35bfe00640 1 -- 172.21.15.159:0/1924882318 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f35c004d030 con 0x7f35c815f800 2024-04-04T06:07:39.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.494+0000 7f35ce600640 1 -- 172.21.15.159:0/1924882318 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f35c809c4a0 con 0x7f35c815f800 2024-04-04T06:07:39.489 INFO:tasks.ceph.mon.a.smithi159.stderr:2024-04-04T06:07:39.495+0000 7f1995c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-04T06:07:39.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.498+0000 7f35bfe00640 1 -- 172.21.15.159:0/1924882318 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7f35c007c780 con 0x7f35c815f800 2024-04-04T06:07:39.492 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-04T06:07:39.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.502+0000 7f35ce600640 1 -- 172.21.15.159:0/1924882318 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35a406c8f0 msgr2=0x7f35a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:39.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.502+0000 7f35ce600640 1 --2- 172.21.15.159:0/1924882318 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35a406c8f0 0x7f35a406edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f35b8006fd0 tx=0x7f35b8008040 comp rx=0 tx=0).stop 2024-04-04T06:07:39.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.502+0000 7f35ce600640 1 -- 172.21.15.159:0/1924882318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35c815f800 msgr2=0x7f35c81f66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:39.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.502+0000 7f35ce600640 1 --2- 172.21.15.159:0/1924882318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35c815f800 0x7f35c81f66e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f35c0036040 tx=0x7f35c0004a40 comp rx=0 tx=0).stop 2024-04-04T06:07:39.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.502+0000 7f35ce600640 1 -- 172.21.15.159:0/1924882318 shutdown_connections 2024-04-04T06:07:39.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.502+0000 7f35ce600640 1 --2- 172.21.15.159:0/1924882318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35c815f800 0x7f35c81f66e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:39.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.502+0000 7f35ce600640 1 --2- 172.21.15.159:0/1924882318 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35c8155550 0x7f35c81f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:39.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.502+0000 7f35ce600640 1 --2- 172.21.15.159:0/1924882318 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35a406c8f0 0x7f35a406edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:39.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.502+0000 7f35ce600640 1 --2- 172.21.15.159:0/1924882318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35c80d0990 0x7f35c809c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:39.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.502+0000 7f35ce600640 1 -- 172.21.15.159:0/1924882318 >> 172.21.15.159:0/1924882318 conn(0x7f35c806b750 msgr2=0x7f35c80720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:39.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.503+0000 7f35ce600640 1 -- 172.21.15.159:0/1924882318 shutdown_connections 2024-04-04T06:07:39.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.503+0000 7f35ce600640 1 -- 172.21.15.159:0/1924882318 wait complete. 2024-04-04T06:07:39.517 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-04T06:07:39.525 INFO:tasks.ceph.mds.b.smithi178.stderr: -6795> 2024-04-04T06:07:12.294+0000 7fd6d6400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:07:39.609 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-04T06:07:39.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.725+0000 7ff438a00640 1 Processor -- start 2024-04-04T06:07:39.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.725+0000 7ff438a00640 1 -- start start 2024-04-04T06:07:39.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.726+0000 7ff438a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff434141c10 0x7ff434071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:39.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.727+0000 7ff438a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff43415ec80 con 0x7ff434071eb0 2024-04-04T06:07:39.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.727+0000 7ff438a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff43415ef20 con 0x7ff4340d0280 2024-04-04T06:07:39.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.727+0000 7ff438a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff43415f200 con 0x7ff434141c10 2024-04-04T06:07:39.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.727+0000 7ff432800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7ff434071eb0 0x7ff434162160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43896/0 (socket says 172.21.15.159:43896) 2024-04-04T06:07:39.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.727+0000 7ff432800640 1 -- 172.21.15.159:0/3334030725 learned_addr learned my addr 172.21.15.159:0/3334030725 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:39.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.727+0000 7ff431400640 1 --2- 172.21.15.159:0/3334030725 >> v2:172.21.15.178:3300/0 conn(0x7ff434141c10 0x7ff434071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:39.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.727+0000 7ff430a00640 1 -- 172.21.15.159:0/3334030725 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3166842658 0 0) 0x7ff43415ec80 con 0x7ff434071eb0 2024-04-04T06:07:39.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.727+0000 7ff430a00640 1 -- 172.21.15.159:0/3334030725 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff418003520 con 0x7ff434071eb0 2024-04-04T06:07:39.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.728+0000 7ff430a00640 1 -- 172.21.15.159:0/3334030725 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1887858084 0 0) 0x7ff418003520 con 0x7ff434071eb0 2024-04-04T06:07:39.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.728+0000 7ff430a00640 1 -- 172.21.15.159:0/3334030725 >> v2:172.21.15.178:3300/0 conn(0x7ff434141c10 msgr2=0x7ff434071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:39.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.728+0000 7ff430a00640 1 --2- 172.21.15.159:0/3334030725 >> v2:172.21.15.178:3300/0 conn(0x7ff434141c10 0x7ff434071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:39.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.728+0000 7ff430a00640 1 -- 172.21.15.159:0/3334030725 >> v1:172.21.15.178:6789/0 conn(0x7ff4340d0280 legacy=0x7ff4340d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:39.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.728+0000 7ff430a00640 1 -- 172.21.15.159:0/3334030725 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43415f4e0 con 0x7ff434071eb0 2024-04-04T06:07:39.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.728+0000 7ff430a00640 1 -- 172.21.15.159:0/3334030725 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7ff42c0019f0 con 0x7ff434071eb0 2024-04-04T06:07:39.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.728+0000 7ff430a00640 1 -- 172.21.15.159:0/3334030725 >> v1:172.21.15.159:6789/0 conn(0x7ff434071eb0 legacy=0x7ff434162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:39.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.728+0000 7ff430a00640 1 --2- 172.21.15.159:0/3334030725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff418003a00 0x7ff418003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:39.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.728+0000 7ff430a00640 1 --2- 172.21.15.159:0/3334030725 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff418004800 0x7ff418004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:39.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.728+0000 7ff430a00640 1 --2- 172.21.15.159:0/3334030725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4180051c0 0x7ff41800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:39.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.728+0000 7ff430a00640 1 -- 172.21.15.159:0/3334030725 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff43415f200 con 0x7ff418003a00 2024-04-04T06:07:39.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.728+0000 7ff430a00640 1 -- 172.21.15.159:0/3334030725 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff41800c0c0 con 0x7ff4180051c0 2024-04-04T06:07:39.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.728+0000 7ff430a00640 1 -- 172.21.15.159:0/3334030725 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff41800c2c0 con 0x7ff418004800 2024-04-04T06:07:39.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.728+0000 7ff431e00640 1 --2- 172.21.15.159:0/3334030725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff418003a00 0x7ff418003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:39.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.729+0000 7ff431400640 1 --2- 172.21.15.159:0/3334030725 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff418004800 0x7ff418004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:39.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.729+0000 7ff431e00640 1 -- 172.21.15.159:0/3334030725 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff418004800 msgr2=0x7ff418004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:39.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.729+0000 7ff432800640 1 --2- 172.21.15.159:0/3334030725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4180051c0 0x7ff41800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:39.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.729+0000 7ff431e00640 1 --2- 172.21.15.159:0/3334030725 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff418004800 0x7ff418004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:39.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.729+0000 7ff431e00640 1 -- 172.21.15.159:0/3334030725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4180051c0 msgr2=0x7ff41800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:39.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.729+0000 7ff431e00640 1 --2- 172.21.15.159:0/3334030725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4180051c0 0x7ff41800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:39.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.729+0000 7ff432800640 1 -- 172.21.15.159:0/3334030725 reap_dead start 2024-04-04T06:07:39.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.729+0000 7ff431e00640 1 -- 172.21.15.159:0/3334030725 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff418003520 con 0x7ff418003a00 2024-04-04T06:07:39.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.729+0000 7ff431400640 1 --2- 172.21.15.159:0/3334030725 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff418004800 0x7ff418004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:39.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.730+0000 7ff431e00640 1 --2- 172.21.15.159:0/3334030725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff418003a00 0x7ff418003e20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff424001750 tx=0x7ff418005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd2d5f74e71995d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:39.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.730+0000 7ff430a00640 1 -- 172.21.15.159:0/3334030725 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424001370 con 0x7ff418003a00 2024-04-04T06:07:39.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.730+0000 7ff430a00640 1 -- 172.21.15.159:0/3334030725 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4240052d0 con 0x7ff418003a00 2024-04-04T06:07:39.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.730+0000 7ff430a00640 1 -- 172.21.15.159:0/3334030725 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424003830 con 0x7ff418003a00 2024-04-04T06:07:39.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.731+0000 7ff438a00640 1 -- 172.21.15.159:0/3334030725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff418003a00 msgr2=0x7ff418003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:39.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.731+0000 7ff438a00640 1 --2- 172.21.15.159:0/3334030725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff418003a00 0x7ff418003e20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff424001750 tx=0x7ff418005830 comp rx=0 tx=0).stop 2024-04-04T06:07:39.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.731+0000 7ff438a00640 1 -- 172.21.15.159:0/3334030725 shutdown_connections 2024-04-04T06:07:39.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.731+0000 7ff438a00640 1 --2- 172.21.15.159:0/3334030725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff418003a00 0x7ff418003e20 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:39.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.731+0000 7ff438a00640 1 -- 172.21.15.159:0/3334030725 >> 172.21.15.159:0/3334030725 conn(0x7ff43406b750 msgr2=0x7ff43406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:39.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.731+0000 7ff438a00640 1 -- 172.21.15.159:0/3334030725 shutdown_connections 2024-04-04T06:07:39.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.731+0000 7ff438a00640 1 -- 172.21.15.159:0/3334030725 wait complete. 2024-04-04T06:07:39.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.733+0000 7ff438a00640 1 Processor -- start 2024-04-04T06:07:39.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.733+0000 7ff438a00640 1 -- start start 2024-04-04T06:07:39.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.734+0000 7ff438a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff418003a00 0x7ff4341f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:39.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.734+0000 7ff438a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff43414de50 0x7ff434143610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:39.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.734+0000 7ff438a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff434143b50 0x7ff434143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:39.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.734+0000 7ff438a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff434158e90 con 0x7ff418003a00 2024-04-04T06:07:39.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.734+0000 7ff438a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff43406f340 con 0x7ff434143b50 2024-04-04T06:07:39.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.734+0000 7ff438a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff43414d710 con 0x7ff43414de50 2024-04-04T06:07:39.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.734+0000 7ff431e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff418003a00 0x7ff4341f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:39.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.734+0000 7ff431e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff418003a00 0x7ff4341f78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56910/0 (socket says 172.21.15.159:56910) 2024-04-04T06:07:39.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.734+0000 7ff431e00640 1 -- 172.21.15.159:0/1847065031 learned_addr learned my addr 172.21.15.159:0/1847065031 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:39.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.734+0000 7ff431400640 1 --2- 172.21.15.159:0/1847065031 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff43414de50 0x7ff434143610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:39.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.734+0000 7ff432800640 1 --2- 172.21.15.159:0/1847065031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff434143b50 0x7ff434143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:39.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.734+0000 7ff431e00640 1 -- 172.21.15.159:0/1847065031 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff43414de50 msgr2=0x7ff434143610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:39.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.734+0000 7ff431e00640 1 --2- 172.21.15.159:0/1847065031 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff43414de50 0x7ff434143610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:39.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.734+0000 7ff431e00640 1 -- 172.21.15.159:0/1847065031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff434143b50 msgr2=0x7ff434143fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:39.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.734+0000 7ff431e00640 1 --2- 172.21.15.159:0/1847065031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff434143b50 0x7ff434143fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:39.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.734+0000 7ff431e00640 1 -- 172.21.15.159:0/1847065031 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4341f7e30 con 0x7ff418003a00 2024-04-04T06:07:39.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.735+0000 7ff431e00640 1 --2- 172.21.15.159:0/1847065031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff418003a00 0x7ff4341f78f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff424005040 tx=0x7ff424001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:39.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.735+0000 7ff42a000640 1 -- 172.21.15.159:0/1847065031 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424041020 con 0x7ff418003a00 2024-04-04T06:07:39.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.735+0000 7ff42a000640 1 -- 172.21.15.159:0/1847065031 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff424005480 con 0x7ff418003a00 2024-04-04T06:07:39.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.735+0000 7ff438a00640 1 -- 172.21.15.159:0/1847065031 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43414c950 con 0x7ff418003a00 2024-04-04T06:07:39.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.735+0000 7ff42a000640 1 -- 172.21.15.159:0/1847065031 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424001370 con 0x7ff418003a00 2024-04-04T06:07:39.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.735+0000 7ff438a00640 1 -- 172.21.15.159:0/1847065031 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff43414cf10 con 0x7ff418003a00 2024-04-04T06:07:39.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.736+0000 7ff438a00640 1 -- 172.21.15.159:0/1847065031 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4340685c0 con 0x7ff418003a00 2024-04-04T06:07:39.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.737+0000 7ff42a000640 1 -- 172.21.15.159:0/1847065031 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff424044020 con 0x7ff418003a00 2024-04-04T06:07:39.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.738+0000 7ff42a000640 1 --2- 172.21.15.159:0/1847065031 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff41406c920 0x7ff41406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:39.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.738+0000 7ff42a000640 1 -- 172.21.15.159:0/1847065031 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7ff4240ad430 con 0x7ff418003a00 2024-04-04T06:07:39.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.739+0000 7ff431400640 1 --2- 172.21.15.159:0/1847065031 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff41406c920 0x7ff41406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:39.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.740+0000 7ff431400640 1 --2- 172.21.15.159:0/1847065031 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff41406c920 0x7ff41406ede0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff43406c080 tx=0x7ff420008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:39.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.742+0000 7ff42a000640 1 -- 172.21.15.159:0/1847065031 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff42404b030 con 0x7ff418003a00 2024-04-04T06:07:39.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:39.963+0000 7ff438a00640 1 -- 172.21.15.159:0/1847065031 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff4340cfdb0 con 0x7ff418003a00 2024-04-04T06:07:40.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.502+0000 7ff42a000640 1 -- 172.21.15.159:0/1847065031 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7ff424006e20 con 0x7ff418003a00 2024-04-04T06:07:40.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.506+0000 7ff438a00640 1 -- 172.21.15.159:0/1847065031 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff41406c920 msgr2=0x7ff41406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:40.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.506+0000 7ff438a00640 1 --2- 172.21.15.159:0/1847065031 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff41406c920 0x7ff41406ede0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff43406c080 tx=0x7ff420008040 comp rx=0 tx=0).stop 2024-04-04T06:07:40.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.506+0000 7ff438a00640 1 -- 172.21.15.159:0/1847065031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff418003a00 msgr2=0x7ff4341f78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:40.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.506+0000 7ff438a00640 1 --2- 172.21.15.159:0/1847065031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff418003a00 0x7ff4341f78f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff424005040 tx=0x7ff424001120 comp rx=0 tx=0).stop 2024-04-04T06:07:40.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.506+0000 7ff438a00640 1 -- 172.21.15.159:0/1847065031 shutdown_connections 2024-04-04T06:07:40.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.506+0000 7ff438a00640 1 --2- 172.21.15.159:0/1847065031 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff41406c920 0x7ff41406ede0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:40.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.506+0000 7ff438a00640 1 --2- 172.21.15.159:0/1847065031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff434143b50 0x7ff434143fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:40.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.506+0000 7ff438a00640 1 --2- 172.21.15.159:0/1847065031 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff43414de50 0x7ff434143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:40.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.506+0000 7ff438a00640 1 --2- 172.21.15.159:0/1847065031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff418003a00 0x7ff4341f78f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:40.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.506+0000 7ff438a00640 1 -- 172.21.15.159:0/1847065031 >> 172.21.15.159:0/1847065031 conn(0x7ff43406b750 msgr2=0x7ff434161490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:40.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.506+0000 7ff438a00640 1 -- 172.21.15.159:0/1847065031 shutdown_connections 2024-04-04T06:07:40.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.506+0000 7ff438a00640 1 -- 172.21.15.159:0/1847065031 wait complete. 2024-04-04T06:07:40.524 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-04T06:07:40.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.743+0000 7f7f2ce00640 1 Processor -- start 2024-04-04T06:07:40.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.743+0000 7f7f2ce00640 1 -- start start 2024-04-04T06:07:40.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.744+0000 7f7f2ce00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f2815e420 0x7f7f280b9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:40.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.745+0000 7f7f2ce00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7f280b6310 0x7f7f280b67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:40.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.745+0000 7f7f2ce00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f280d23b0 con 0x7f7f280d3d50 2024-04-04T06:07:40.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.745+0000 7f7f2ce00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7f280d2fc0 con 0x7f7f280b6310 2024-04-04T06:07:40.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.745+0000 7f7f2ce00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7f280d2690 con 0x7f7f2815e420 2024-04-04T06:07:40.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.745+0000 7f7f1fe00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f7f280d3d50 0x7f7f280d3450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43898/0 (socket says 172.21.15.159:43898) 2024-04-04T06:07:40.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.745+0000 7f7f1fe00640 1 -- 172.21.15.159:0/894292744 learned_addr learned my addr 172.21.15.159:0/894292744 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:40.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.745+0000 7f7f26a00640 1 -- 172.21.15.159:0/894292744 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 933887234 0 0) 0x7f7f280d23b0 con 0x7f7f280d3d50 2024-04-04T06:07:40.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.745+0000 7f7f26a00640 1 -- 172.21.15.159:0/894292744 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f0c003520 con 0x7f7f280d3d50 2024-04-04T06:07:40.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.745+0000 7f7f27400640 1 --2- 172.21.15.159:0/894292744 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f2815e420 0x7f7f280b9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:40.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.745+0000 7f7f27e00640 1 --2- 172.21.15.159:0/894292744 >> v2:172.21.15.178:3300/0 conn(0x7f7f280b6310 0x7f7f280b67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:40.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.746+0000 7f7f26a00640 1 -- 172.21.15.159:0/894292744 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3855185247 0 0) 0x7f7f0c003520 con 0x7f7f280d3d50 2024-04-04T06:07:40.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.746+0000 7f7f26a00640 1 -- 172.21.15.159:0/894292744 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f2815e420 msgr2=0x7f7f280b9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:40.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.746+0000 7f7f26a00640 1 --2- 172.21.15.159:0/894292744 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f2815e420 0x7f7f280b9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:40.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.746+0000 7f7f26a00640 1 -- 172.21.15.159:0/894292744 >> v2:172.21.15.178:3300/0 conn(0x7f7f280b6310 msgr2=0x7f7f280b67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:40.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.746+0000 7f7f26a00640 1 --2- 172.21.15.159:0/894292744 >> v2:172.21.15.178:3300/0 conn(0x7f7f280b6310 0x7f7f280b67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:40.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.746+0000 7f7f26a00640 1 -- 172.21.15.159:0/894292744 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f280d2970 con 0x7f7f280d3d50 2024-04-04T06:07:40.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.746+0000 7f7f26a00640 1 -- 172.21.15.159:0/894292744 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f7f10001670 con 0x7f7f280d3d50 2024-04-04T06:07:40.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.746+0000 7f7f26a00640 1 -- 172.21.15.159:0/894292744 >> v1:172.21.15.159:6789/0 conn(0x7f7f280d3d50 legacy=0x7f7f280d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:40.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.746+0000 7f7f26a00640 1 --2- 172.21.15.159:0/894292744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f0c0039b0 0x7f7f0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:40.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.746+0000 7f7f26a00640 1 --2- 172.21.15.159:0/894292744 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f2815e420 0x7f7f0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:40.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.746+0000 7f7f26a00640 1 --2- 172.21.15.159:0/894292744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f0c004d40 0x7f7f0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:40.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.746+0000 7f7f26a00640 1 -- 172.21.15.159:0/894292744 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7f280d2fc0 con 0x7f7f0c0039b0 2024-04-04T06:07:40.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.746+0000 7f7f26a00640 1 -- 172.21.15.159:0/894292744 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7f280d2690 con 0x7f7f0c004d40 2024-04-04T06:07:40.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.746+0000 7f7f26a00640 1 -- 172.21.15.159:0/894292744 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7f0c00b9c0 con 0x7f7f2815e420 2024-04-04T06:07:40.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.746+0000 7f7f27400640 1 --2- 172.21.15.159:0/894292744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f0c0039b0 0x7f7f0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:40.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.746+0000 7f7f1fe00640 1 --2- 172.21.15.159:0/894292744 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f2815e420 0x7f7f0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:40.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.747+0000 7f7f27e00640 1 --2- 172.21.15.159:0/894292744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f0c004d40 0x7f7f0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:40.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.747+0000 7f7f27400640 1 -- 172.21.15.159:0/894292744 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f2815e420 msgr2=0x7f7f0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:40.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.747+0000 7f7f27400640 1 --2- 172.21.15.159:0/894292744 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f2815e420 0x7f7f0c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:40.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.747+0000 7f7f27400640 1 -- 172.21.15.159:0/894292744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f0c004d40 msgr2=0x7f7f0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:40.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.747+0000 7f7f27400640 1 --2- 172.21.15.159:0/894292744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f0c004d40 0x7f7f0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:40.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.747+0000 7f7f27400640 1 -- 172.21.15.159:0/894292744 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0c003520 con 0x7f7f0c0039b0 2024-04-04T06:07:40.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.747+0000 7f7f27e00640 1 --2- 172.21.15.159:0/894292744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f0c004d40 0x7f7f0c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:40.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.748+0000 7f7f27400640 1 --2- 172.21.15.159:0/894292744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f0c0039b0 0x7f7f0c003dd0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7f18009ac0 tx=0x7f7f0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6518361c9ea90c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:40.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.748+0000 7f7f26a00640 1 -- 172.21.15.159:0/894292744 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18039070 con 0x7f7f0c0039b0 2024-04-04T06:07:40.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.748+0000 7f7f26a00640 1 -- 172.21.15.159:0/894292744 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f1802c6b0 con 0x7f7f0c0039b0 2024-04-04T06:07:40.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.748+0000 7f7f26a00640 1 -- 172.21.15.159:0/894292744 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18009690 con 0x7f7f0c0039b0 2024-04-04T06:07:40.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.749+0000 7f7f2ce00640 1 -- 172.21.15.159:0/894292744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f0c0039b0 msgr2=0x7f7f0c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:40.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.749+0000 7f7f2ce00640 1 --2- 172.21.15.159:0/894292744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f0c0039b0 0x7f7f0c003dd0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7f18009ac0 tx=0x7f7f0c005830 comp rx=0 tx=0).stop 2024-04-04T06:07:40.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.749+0000 7f7f27e00640 1 -- 172.21.15.159:0/894292744 reap_dead start 2024-04-04T06:07:40.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.749+0000 7f7f2ce00640 1 -- 172.21.15.159:0/894292744 shutdown_connections 2024-04-04T06:07:40.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.749+0000 7f7f2ce00640 1 -- 172.21.15.159:0/894292744 >> 172.21.15.159:0/894292744 conn(0x7f7f280676a0 msgr2=0x7f7f28067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:40.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.749+0000 7f7f2ce00640 1 -- 172.21.15.159:0/894292744 shutdown_connections 2024-04-04T06:07:40.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.749+0000 7f7f2ce00640 1 -- 172.21.15.159:0/894292744 wait complete. 2024-04-04T06:07:40.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.751+0000 7f7f2ce00640 1 Processor -- start 2024-04-04T06:07:40.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.751+0000 7f7f2ce00640 1 -- start start 2024-04-04T06:07:40.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.752+0000 7f7f2ce00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f0c0039b0 0x7f7f281f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:40.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.752+0000 7f7f2ce00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f28158e70 0x7f7f28153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:40.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.752+0000 7f7f2ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f280b8c40 0x7f7f280b90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:40.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.752+0000 7f7f2ce00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7f280b9c50 con 0x7f7f280b8c40 2024-04-04T06:07:40.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.752+0000 7f7f2ce00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2815e800 con 0x7f7f28158e70 2024-04-04T06:07:40.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.752+0000 7f7f2ce00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7f280b9dd0 con 0x7f7f0c0039b0 2024-04-04T06:07:40.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.752+0000 7f7f27e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f280b8c40 0x7f7f280b90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:40.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.752+0000 7f7f27e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f280b8c40 0x7f7f280b90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56936/0 (socket says 172.21.15.159:56936) 2024-04-04T06:07:40.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.752+0000 7f7f27e00640 1 -- 172.21.15.159:0/978371491 learned_addr learned my addr 172.21.15.159:0/978371491 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:40.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.752+0000 7f7f27400640 1 --2- 172.21.15.159:0/978371491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f0c0039b0 0x7f7f281f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:40.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.752+0000 7f7f1fe00640 1 --2- 172.21.15.159:0/978371491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f28158e70 0x7f7f28153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:40.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.752+0000 7f7f27e00640 1 -- 172.21.15.159:0/978371491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f0c0039b0 msgr2=0x7f7f281f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:40.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.752+0000 7f7f27e00640 1 --2- 172.21.15.159:0/978371491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f0c0039b0 0x7f7f281f70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:40.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.752+0000 7f7f27e00640 1 -- 172.21.15.159:0/978371491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f28158e70 msgr2=0x7f7f28153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:40.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.752+0000 7f7f27e00640 1 --2- 172.21.15.159:0/978371491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f28158e70 0x7f7f28153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:40.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.752+0000 7f7f27e00640 1 -- 172.21.15.159:0/978371491 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f281592f0 con 0x7f7f280b8c40 2024-04-04T06:07:40.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.753+0000 7f7f27e00640 1 --2- 172.21.15.159:0/978371491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f280b8c40 0x7f7f280b90c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009fd0 tx=0x7f7f2000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:40.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.753+0000 7f7f1f400640 1 -- 172.21.15.159:0/978371491 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2001c070 con 0x7f7f280b8c40 2024-04-04T06:07:40.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.753+0000 7f7f1f400640 1 -- 172.21.15.159:0/978371491 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f20017400 con 0x7f7f280b8c40 2024-04-04T06:07:40.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.753+0000 7f7f1f400640 1 -- 172.21.15.159:0/978371491 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f200177d0 con 0x7f7f280b8c40 2024-04-04T06:07:40.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.753+0000 7f7f2ce00640 1 -- 172.21.15.159:0/978371491 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f281f7fb0 con 0x7f7f280b8c40 2024-04-04T06:07:40.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.753+0000 7f7f2ce00640 1 -- 172.21.15.159:0/978371491 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f281f79f0 con 0x7f7f280b8c40 2024-04-04T06:07:40.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.754+0000 7f7f2ce00640 1 -- 172.21.15.159:0/978371491 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f280b9600 con 0x7f7f280b8c40 2024-04-04T06:07:40.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.755+0000 7f7f1f400640 1 -- 172.21.15.159:0/978371491 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f20005d10 con 0x7f7f280b8c40 2024-04-04T06:07:40.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.756+0000 7f7f1f400640 1 --2- 172.21.15.159:0/978371491 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7f0406c8f0 0x7f7f0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:40.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.756+0000 7f7f1f400640 1 -- 172.21.15.159:0/978371491 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f7f2008be10 con 0x7f7f280b8c40 2024-04-04T06:07:40.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.757+0000 7f7f27400640 1 --2- 172.21.15.159:0/978371491 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7f0406c8f0 0x7f7f0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:40.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.758+0000 7f7f27400640 1 --2- 172.21.15.159:0/978371491 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7f0406c8f0 0x7f7f0406edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7f18036000 tx=0x7f7f18030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:40.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.761+0000 7f7f1f400640 1 -- 172.21.15.159:0/978371491 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7f2008f030 con 0x7f7f280b8c40 2024-04-04T06:07:40.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:40.961+0000 7f7f2ce00640 1 -- 172.21.15.159:0/978371491 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7f28075940 con 0x7f7f280b8c40 2024-04-04T06:07:41.047 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:07:41.055 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:07:41.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:41.932+0000 7f7f1f400640 1 -- 172.21.15.159:0/978371491 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) ==== 181+0+0 (secure 0 0 0) 0x7f7f281592f0 con 0x7f7f280b8c40 2024-04-04T06:07:41.927 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_metadata' removed 2024-04-04T06:07:41.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:41.936+0000 7f7f2ce00640 1 -- 172.21.15.159:0/978371491 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7f0406c8f0 msgr2=0x7f7f0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:41.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:41.936+0000 7f7f2ce00640 1 --2- 172.21.15.159:0/978371491 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7f0406c8f0 0x7f7f0406edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7f18036000 tx=0x7f7f18030000 comp rx=0 tx=0).stop 2024-04-04T06:07:41.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:41.936+0000 7f7f2ce00640 1 -- 172.21.15.159:0/978371491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f280b8c40 msgr2=0x7f7f280b90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:41.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:41.936+0000 7f7f2ce00640 1 --2- 172.21.15.159:0/978371491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f280b8c40 0x7f7f280b90c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009fd0 tx=0x7f7f2000f4a0 comp rx=0 tx=0).stop 2024-04-04T06:07:41.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:41.938+0000 7f7f2ce00640 1 -- 172.21.15.159:0/978371491 shutdown_connections 2024-04-04T06:07:41.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:41.938+0000 7f7f2ce00640 1 --2- 172.21.15.159:0/978371491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f280b8c40 0x7f7f280b90c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:41.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:41.938+0000 7f7f2ce00640 1 --2- 172.21.15.159:0/978371491 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7f0406c8f0 0x7f7f0406edb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:41.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:41.938+0000 7f7f2ce00640 1 --2- 172.21.15.159:0/978371491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f28158e70 0x7f7f28153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:41.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:41.938+0000 7f7f2ce00640 1 --2- 172.21.15.159:0/978371491 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f0c0039b0 0x7f7f281f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:41.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:41.938+0000 7f7f2ce00640 1 -- 172.21.15.159:0/978371491 >> 172.21.15.159:0/978371491 conn(0x7f7f280676a0 msgr2=0x7f7f280cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:41.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:41.939+0000 7f7f2ce00640 1 -- 172.21.15.159:0/978371491 shutdown_connections 2024-04-04T06:07:41.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:41.939+0000 7f7f2ce00640 1 -- 172.21.15.159:0/978371491 wait complete. 2024-04-04T06:07:41.957 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-04T06:07:42.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.171+0000 7f1745600640 1 Processor -- start 2024-04-04T06:07:42.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.172+0000 7f1745600640 1 -- start start 2024-04-04T06:07:42.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.173+0000 7f1745600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17400b3940 0x7f17400adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:42.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.173+0000 7f1745600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f17400ae4f0 0x7f17400ae930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:42.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.173+0000 7f1745600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f174006ea10 con 0x7f1740072720 2024-04-04T06:07:42.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.173+0000 7f1745600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f174006ecf0 con 0x7f17400ae4f0 2024-04-04T06:07:42.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.173+0000 7f1745600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f174006efd0 con 0x7f17400b3940 2024-04-04T06:07:42.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.174+0000 7f173f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f17400ae4f0 0x7f17400ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:42.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.174+0000 7f173f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f17400ae4f0 0x7f17400ae930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56938/0 (socket says 172.21.15.159:56938) 2024-04-04T06:07:42.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.174+0000 7f173ea00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f1740072720 0x7f1740072b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43914/0 (socket says 172.21.15.159:43914) 2024-04-04T06:07:42.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.174+0000 7f173f400640 1 -- 172.21.15.159:0/655029565 learned_addr learned my addr 172.21.15.159:0/655029565 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:42.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.174+0000 7f1736000640 1 --2- 172.21.15.159:0/655029565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17400b3940 0x7f17400adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:42.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.174+0000 7f173f400640 1 -- 172.21.15.159:0/655029565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17400b3940 msgr2=0x7f17400adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:42.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.174+0000 7f173f400640 1 --2- 172.21.15.159:0/655029565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17400b3940 0x7f17400adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:42.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.174+0000 7f173f400640 1 -- 172.21.15.159:0/655029565 >> v1:172.21.15.159:6789/0 conn(0x7f1740072720 legacy=0x7f1740072b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:42.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.174+0000 7f173f400640 1 -- 172.21.15.159:0/655029565 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174006f2b0 con 0x7f17400ae4f0 2024-04-04T06:07:42.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.174+0000 7f1736000640 1 --2- 172.21.15.159:0/655029565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17400b3940 0x7f17400adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:42.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.175+0000 7f173f400640 1 --2- 172.21.15.159:0/655029565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17400ae4f0 0x7f17400ae930 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1738009e50 tx=0x7f173802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1620e6d2bba10a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:42.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.175+0000 7f173e000640 1 -- 172.21.15.159:0/655029565 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173802cb00 con 0x7f17400ae4f0 2024-04-04T06:07:42.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.176+0000 7f173e000640 1 -- 172.21.15.159:0/655029565 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f173802cca0 con 0x7f17400ae4f0 2024-04-04T06:07:42.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.176+0000 7f173e000640 1 -- 172.21.15.159:0/655029565 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738034610 con 0x7f17400ae4f0 2024-04-04T06:07:42.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.176+0000 7f1745600640 1 -- 172.21.15.159:0/655029565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17400ae4f0 msgr2=0x7f17400ae930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:42.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.176+0000 7f1745600640 1 --2- 172.21.15.159:0/655029565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17400ae4f0 0x7f17400ae930 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1738009e50 tx=0x7f173802bf60 comp rx=0 tx=0).stop 2024-04-04T06:07:42.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.176+0000 7f1745600640 1 -- 172.21.15.159:0/655029565 shutdown_connections 2024-04-04T06:07:42.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.176+0000 7f1745600640 1 --2- 172.21.15.159:0/655029565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17400b3940 0x7f17400adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:42.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.176+0000 7f1745600640 1 --2- 172.21.15.159:0/655029565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17400ae4f0 0x7f17400ae930 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:42.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.176+0000 7f1745600640 1 -- 172.21.15.159:0/655029565 >> 172.21.15.159:0/655029565 conn(0x7f174006bba0 msgr2=0x7f17400b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:42.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.176+0000 7f1745600640 1 -- 172.21.15.159:0/655029565 shutdown_connections 2024-04-04T06:07:42.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.176+0000 7f1745600640 1 -- 172.21.15.159:0/655029565 wait complete. 2024-04-04T06:07:42.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.178+0000 7f1745600640 1 Processor -- start 2024-04-04T06:07:42.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.178+0000 7f1745600640 1 -- start start 2024-04-04T06:07:42.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.179+0000 7f1745600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1740072720 0x7f1740149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:42.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.179+0000 7f1745600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17400ae4f0 0x7f17400ce830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:42.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.179+0000 7f1745600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17400b3940 0x7f17400ced70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:42.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.179+0000 7f1745600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f174006e690 con 0x7f1740072720 2024-04-04T06:07:42.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.179+0000 7f1745600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f17400b3d80 con 0x7f17400ae4f0 2024-04-04T06:07:42.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.179+0000 7f1745600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f174015eab0 con 0x7f17400b3940 2024-04-04T06:07:42.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.179+0000 7f173ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1740072720 0x7f1740149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:42.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.179+0000 7f173ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1740072720 0x7f1740149450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56940/0 (socket says 172.21.15.159:56940) 2024-04-04T06:07:42.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.179+0000 7f173ea00640 1 -- 172.21.15.159:0/1569962699 learned_addr learned my addr 172.21.15.159:0/1569962699 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:42.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.180+0000 7f173ea00640 1 -- 172.21.15.159:0/1569962699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17400b3940 msgr2=0x7f17400ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:42.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.180+0000 7f173ea00640 1 --2- 172.21.15.159:0/1569962699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17400b3940 0x7f17400ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:42.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.180+0000 7f173ea00640 1 -- 172.21.15.159:0/1569962699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17400ae4f0 msgr2=0x7f17400ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:42.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.180+0000 7f173ea00640 1 --2- 172.21.15.159:0/1569962699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17400ae4f0 0x7f17400ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:42.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.180+0000 7f173ea00640 1 -- 172.21.15.159:0/1569962699 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1740149ea0 con 0x7f1740072720 2024-04-04T06:07:42.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.180+0000 7f173ea00640 1 --2- 172.21.15.159:0/1569962699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1740072720 0x7f1740149450 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f173000c3c0 tx=0x7f173000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:42.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.180+0000 7f1737400640 1 -- 172.21.15.159:0/1569962699 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17300074e0 con 0x7f1740072720 2024-04-04T06:07:42.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.180+0000 7f1737400640 1 -- 172.21.15.159:0/1569962699 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1730007680 con 0x7f1740072720 2024-04-04T06:07:42.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.180+0000 7f1737400640 1 -- 172.21.15.159:0/1569962699 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1730004a00 con 0x7f1740072720 2024-04-04T06:07:42.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.181+0000 7f1745600640 1 -- 172.21.15.159:0/1569962699 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17400cf3d0 con 0x7f1740072720 2024-04-04T06:07:42.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.181+0000 7f1745600640 1 -- 172.21.15.159:0/1569962699 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f174009afa0 con 0x7f1740072720 2024-04-04T06:07:42.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.182+0000 7f1745600640 1 -- 172.21.15.159:0/1569962699 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17400685c0 con 0x7f1740072720 2024-04-04T06:07:42.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.183+0000 7f1737400640 1 -- 172.21.15.159:0/1569962699 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f173001a020 con 0x7f1740072720 2024-04-04T06:07:42.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.183+0000 7f1737400640 1 --2- 172.21.15.159:0/1569962699 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f171c06c8f0 0x7f171c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:42.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.184+0000 7f1737400640 1 -- 172.21.15.159:0/1569962699 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5357+0+0 (secure 0 0 0) 0x7f173008b1c0 con 0x7f1740072720 2024-04-04T06:07:42.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.184+0000 7f1736000640 1 --2- 172.21.15.159:0/1569962699 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f171c06c8f0 0x7f171c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:42.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.185+0000 7f1736000640 1 --2- 172.21.15.159:0/1569962699 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f171c06c8f0 0x7f171c06edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1728006fd0 tx=0x7f1728008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:42.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.188+0000 7f1737400640 1 -- 172.21.15.159:0/1569962699 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1730059740 con 0x7f1740072720 2024-04-04T06:07:42.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.397+0000 7f1745600640 1 -- 172.21.15.159:0/1569962699 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f17400c9970 con 0x7f1740072720 2024-04-04T06:07:42.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.942+0000 7f1737400640 1 -- 172.21.15.159:0/1569962699 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) ==== 169+0+0 (secure 0 0 0) 0x7f1730059040 con 0x7f1740072720 2024-04-04T06:07:42.936 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_data' removed 2024-04-04T06:07:42.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.945+0000 7f1745600640 1 -- 172.21.15.159:0/1569962699 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f171c06c8f0 msgr2=0x7f171c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:42.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.945+0000 7f1745600640 1 --2- 172.21.15.159:0/1569962699 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f171c06c8f0 0x7f171c06edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1728006fd0 tx=0x7f1728008040 comp rx=0 tx=0).stop 2024-04-04T06:07:42.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.946+0000 7f1745600640 1 -- 172.21.15.159:0/1569962699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1740072720 msgr2=0x7f1740149450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:42.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.946+0000 7f1745600640 1 --2- 172.21.15.159:0/1569962699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1740072720 0x7f1740149450 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f173000c3c0 tx=0x7f173000c880 comp rx=0 tx=0).stop 2024-04-04T06:07:42.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.946+0000 7f1745600640 1 -- 172.21.15.159:0/1569962699 shutdown_connections 2024-04-04T06:07:42.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.946+0000 7f1745600640 1 --2- 172.21.15.159:0/1569962699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17400b3940 0x7f17400ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:42.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.946+0000 7f1745600640 1 --2- 172.21.15.159:0/1569962699 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f171c06c8f0 0x7f171c06edb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:42.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.946+0000 7f1745600640 1 --2- 172.21.15.159:0/1569962699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17400ae4f0 0x7f17400ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:42.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.946+0000 7f1745600640 1 --2- 172.21.15.159:0/1569962699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1740072720 0x7f1740149450 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:42.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.946+0000 7f1745600640 1 -- 172.21.15.159:0/1569962699 >> 172.21.15.159:0/1569962699 conn(0x7f174006bba0 msgr2=0x7f174015dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:42.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.947+0000 7f1745600640 1 -- 172.21.15.159:0/1569962699 shutdown_connections 2024-04-04T06:07:42.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:42.948+0000 7f1745600640 1 -- 172.21.15.159:0/1569962699 wait complete. 2024-04-04T06:07:42.969 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-04T06:07:43.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.179+0000 7f1953200640 1 Processor -- start 2024-04-04T06:07:43.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.179+0000 7f1953200640 1 -- start start 2024-04-04T06:07:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.181+0000 7f1953200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f195415fc50 0x7f19540cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.181+0000 7f1953200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f19540cc490 0x7f19540d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.181+0000 7f1953200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19540b3150 con 0x7f19540cc0b0 2024-04-04T06:07:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.181+0000 7f1953200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f195415c610 con 0x7f19540cc490 2024-04-04T06:07:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.181+0000 7f1953200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f19541600e0 con 0x7f195415fc50 2024-04-04T06:07:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.181+0000 7f1951e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f19540cc0b0 0x7f195406dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43930/0 (socket says 172.21.15.159:43930) 2024-04-04T06:07:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.181+0000 7f1951e00640 1 -- 172.21.15.159:0/1040490065 learned_addr learned my addr 172.21.15.159:0/1040490065 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.181+0000 7f1952800640 1 --2- 172.21.15.159:0/1040490065 >> v2:172.21.15.159:3300/0 conn(0x7f19540cc490 0x7f19540d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.181+0000 7f1951400640 1 -- 172.21.15.159:0/1040490065 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 114082806 0 0) 0x7f19540b3150 con 0x7f19540cc0b0 2024-04-04T06:07:43.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.181+0000 7f1951400640 1 -- 172.21.15.159:0/1040490065 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1938003520 con 0x7f19540cc0b0 2024-04-04T06:07:43.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.181+0000 7f194be00640 1 --2- 172.21.15.159:0/1040490065 >> v2:172.21.15.178:3300/0 conn(0x7f195415fc50 0x7f19540cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:43.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.182+0000 7f1952800640 1 -- 172.21.15.159:0/1040490065 >> v2:172.21.15.178:3300/0 conn(0x7f195415fc50 msgr2=0x7f19540cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:43.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.182+0000 7f1952800640 1 --2- 172.21.15.159:0/1040490065 >> v2:172.21.15.178:3300/0 conn(0x7f195415fc50 0x7f19540cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:43.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.182+0000 7f1952800640 1 -- 172.21.15.159:0/1040490065 >> v1:172.21.15.159:6789/0 conn(0x7f19540cc0b0 legacy=0x7f195406dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:43.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.182+0000 7f1952800640 1 -- 172.21.15.159:0/1040490065 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19541603c0 con 0x7f19540cc490 2024-04-04T06:07:43.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.183+0000 7f1952800640 1 --2- 172.21.15.159:0/1040490065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19540cc490 0x7f19540d0300 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f195406ee00 tx=0x7f194c02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b86ea2f7ed65e1e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:43.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.183+0000 7f1951400640 1 -- 172.21.15.159:0/1040490065 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c039070 con 0x7f19540cc490 2024-04-04T06:07:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.183+0000 7f1951400640 1 -- 172.21.15.159:0/1040490065 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f194c02cc70 con 0x7f19540cc490 2024-04-04T06:07:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.183+0000 7f1951400640 1 -- 172.21.15.159:0/1040490065 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c034640 con 0x7f19540cc490 2024-04-04T06:07:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.184+0000 7f1953200640 1 -- 172.21.15.159:0/1040490065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19540cc490 msgr2=0x7f19540d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.184+0000 7f1953200640 1 --2- 172.21.15.159:0/1040490065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19540cc490 0x7f19540d0300 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f195406ee00 tx=0x7f194c02bec0 comp rx=0 tx=0).stop 2024-04-04T06:07:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.184+0000 7f1953200640 1 -- 172.21.15.159:0/1040490065 shutdown_connections 2024-04-04T06:07:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.184+0000 7f1953200640 1 --2- 172.21.15.159:0/1040490065 >> v2:172.21.15.178:3300/0 conn(0x7f195415fc50 0x7f19540cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.184+0000 7f1953200640 1 --2- 172.21.15.159:0/1040490065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19540cc490 0x7f19540d0300 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.184+0000 7f1953200640 1 -- 172.21.15.159:0/1040490065 >> 172.21.15.159:0/1040490065 conn(0x7f19540ccad0 msgr2=0x7f19540cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.184+0000 7f1953200640 1 -- 172.21.15.159:0/1040490065 shutdown_connections 2024-04-04T06:07:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.184+0000 7f1953200640 1 -- 172.21.15.159:0/1040490065 wait complete. 2024-04-04T06:07:43.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.185+0000 7f1953200640 1 Processor -- start 2024-04-04T06:07:43.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.186+0000 7f1953200640 1 -- start start 2024-04-04T06:07:43.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.186+0000 7f1953200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19540cc0b0 0x7f19541f7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:43.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.186+0000 7f1953200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19540cc490 0x7f19541e64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:43.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.187+0000 7f1953200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19541e69f0 0x7f1954130e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:43.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.187+0000 7f1953200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f195415ca70 con 0x7f19540cc0b0 2024-04-04T06:07:43.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.187+0000 7f1953200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1954160980 con 0x7f19540cc490 2024-04-04T06:07:43.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.187+0000 7f1953200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f195415cbf0 con 0x7f19541e69f0 2024-04-04T06:07:43.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.187+0000 7f1951e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19540cc0b0 0x7f19541f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:43.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.187+0000 7f1951e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19540cc0b0 0x7f19541f7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56950/0 (socket says 172.21.15.159:56950) 2024-04-04T06:07:43.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.187+0000 7f1951e00640 1 -- 172.21.15.159:0/631028238 learned_addr learned my addr 172.21.15.159:0/631028238 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:43.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.187+0000 7f1952800640 1 --2- 172.21.15.159:0/631028238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19541e69f0 0x7f1954130e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:43.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.187+0000 7f1951e00640 1 -- 172.21.15.159:0/631028238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19541e69f0 msgr2=0x7f1954130e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:43.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.187+0000 7f1951e00640 1 --2- 172.21.15.159:0/631028238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19541e69f0 0x7f1954130e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:43.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.187+0000 7f194be00640 1 --2- 172.21.15.159:0/631028238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19540cc490 0x7f19541e64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:43.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.187+0000 7f1951e00640 1 -- 172.21.15.159:0/631028238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19540cc490 msgr2=0x7f19541e64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:43.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.187+0000 7f1951e00640 1 --2- 172.21.15.159:0/631028238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19540cc490 0x7f19541e64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:43.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.187+0000 7f1951e00640 1 -- 172.21.15.159:0/631028238 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19541f6070 con 0x7f19540cc0b0 2024-04-04T06:07:43.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.188+0000 7f1951e00640 1 --2- 172.21.15.159:0/631028238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19540cc0b0 0x7f19541f7110 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f194400c460 tx=0x7f194400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:43.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.188+0000 7f194a000640 1 -- 172.21.15.159:0/631028238 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1944017070 con 0x7f19540cc0b0 2024-04-04T06:07:43.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.188+0000 7f194a000640 1 -- 172.21.15.159:0/631028238 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1944005ce0 con 0x7f19540cc0b0 2024-04-04T06:07:43.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.188+0000 7f194a000640 1 -- 172.21.15.159:0/631028238 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1944007290 con 0x7f19540cc0b0 2024-04-04T06:07:43.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.188+0000 7f1953200640 1 -- 172.21.15.159:0/631028238 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19541f6350 con 0x7f19540cc0b0 2024-04-04T06:07:43.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.188+0000 7f1953200640 1 -- 172.21.15.159:0/631028238 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19541f6910 con 0x7f19540cc0b0 2024-04-04T06:07:43.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.189+0000 7f1953200640 1 -- 172.21.15.159:0/631028238 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1954075940 con 0x7f19540cc0b0 2024-04-04T06:07:43.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.190+0000 7f194a000640 1 -- 172.21.15.159:0/631028238 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f194401a070 con 0x7f19540cc0b0 2024-04-04T06:07:43.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.191+0000 7f194a000640 1 --2- 172.21.15.159:0/631028238 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f193006c8f0 0x7f193006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:43.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.191+0000 7f194a000640 1 -- 172.21.15.159:0/631028238 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f194408ac60 con 0x7f19540cc0b0 2024-04-04T06:07:43.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.192+0000 7f194be00640 1 --2- 172.21.15.159:0/631028238 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f193006c8f0 0x7f193006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:43.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.193+0000 7f194be00640 1 --2- 172.21.15.159:0/631028238 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f193006c8f0 0x7f193006edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f193c006fd0 tx=0x7f193c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:43.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.196+0000 7f194a000640 1 -- 172.21.15.159:0/631028238 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f19440592f0 con 0x7f19540cc0b0 2024-04-04T06:07:43.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.394+0000 7f1953200640 1 -- 172.21.15.159:0/631028238 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f1954073830 con 0x7f19540cc0b0 2024-04-04T06:07:43.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.400+0000 7f194a000640 1 -- 172.21.15.159:0/631028238 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f1944058bf0 con 0x7f19540cc0b0 2024-04-04T06:07:43.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.403+0000 7f1953200640 1 -- 172.21.15.159:0/631028238 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f193006c8f0 msgr2=0x7f193006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:43.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.403+0000 7f1953200640 1 --2- 172.21.15.159:0/631028238 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f193006c8f0 0x7f193006edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f193c006fd0 tx=0x7f193c008040 comp rx=0 tx=0).stop 2024-04-04T06:07:43.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.403+0000 7f1953200640 1 -- 172.21.15.159:0/631028238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19540cc0b0 msgr2=0x7f19541f7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:43.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.403+0000 7f1953200640 1 --2- 172.21.15.159:0/631028238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19540cc0b0 0x7f19541f7110 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f194400c460 tx=0x7f194400c920 comp rx=0 tx=0).stop 2024-04-04T06:07:43.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.403+0000 7f1953200640 1 -- 172.21.15.159:0/631028238 shutdown_connections 2024-04-04T06:07:43.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.403+0000 7f1953200640 1 --2- 172.21.15.159:0/631028238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19541e69f0 0x7f1954130e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:43.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.403+0000 7f1953200640 1 --2- 172.21.15.159:0/631028238 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f193006c8f0 0x7f193006edb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:43.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.403+0000 7f1953200640 1 --2- 172.21.15.159:0/631028238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19540cc490 0x7f19541e64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:43.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.403+0000 7f1953200640 1 --2- 172.21.15.159:0/631028238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19540cc0b0 0x7f19541f7110 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:43.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.403+0000 7f1953200640 1 -- 172.21.15.159:0/631028238 >> 172.21.15.159:0/631028238 conn(0x7f19540ccad0 msgr2=0x7f19540728a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:43.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.403+0000 7f1953200640 1 -- 172.21.15.159:0/631028238 shutdown_connections 2024-04-04T06:07:43.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.403+0000 7f1953200640 1 -- 172.21.15.159:0/631028238 wait complete. 2024-04-04T06:07:43.417 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-04T06:07:43.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.626+0000 7f053a000640 1 Processor -- start 2024-04-04T06:07:43.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.627+0000 7f053a000640 1 -- start start 2024-04-04T06:07:43.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.628+0000 7f053a000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f053406e030 0x7f053415fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:43.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.628+0000 7f053a000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f05341602d0 0x7f05340cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:43.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.628+0000 7f053a000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0534142880 con 0x7f053413f300 2024-04-04T06:07:43.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.628+0000 7f053a000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f0534160720 con 0x7f05341602d0 2024-04-04T06:07:43.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.628+0000 7f053a000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f05340cee90 con 0x7f053406e030 2024-04-04T06:07:43.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.629+0000 7f0539600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f05341602d0 0x7f05340cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:43.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.629+0000 7f0539600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f05341602d0 0x7f05340cf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56958/0 (socket says 172.21.15.159:56958) 2024-04-04T06:07:43.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.629+0000 7f0539600640 1 -- 172.21.15.159:0/2173108111 learned_addr learned my addr 172.21.15.159:0/2173108111 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:43.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.629+0000 7f052be00640 1 --2- 172.21.15.159:0/2173108111 >> v2:172.21.15.178:3300/0 conn(0x7f053406e030 0x7f053415fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:43.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.629+0000 7f0539600640 1 -- 172.21.15.159:0/2173108111 >> v2:172.21.15.178:3300/0 conn(0x7f053406e030 msgr2=0x7f053415fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:43.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.629+0000 7f0539600640 1 --2- 172.21.15.159:0/2173108111 >> v2:172.21.15.178:3300/0 conn(0x7f053406e030 0x7f053415fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:43.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.629+0000 7f0539600640 1 -- 172.21.15.159:0/2173108111 >> v1:172.21.15.178:6789/0 conn(0x7f053413f300 legacy=0x7f05341419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:43.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.629+0000 7f0539600640 1 -- 172.21.15.159:0/2173108111 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05341422c0 con 0x7f05341602d0 2024-04-04T06:07:43.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.629+0000 7f052be00640 1 --2- 172.21.15.159:0/2173108111 >> v2:172.21.15.178:3300/0 conn(0x7f053406e030 0x7f053415fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:07:43.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.631+0000 7f0539600640 1 --2- 172.21.15.159:0/2173108111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05341602d0 0x7f05340cf320 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f053406c4a0 tx=0x7f052c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1466d58c85adce server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:43.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.631+0000 7f0533e00640 1 -- 172.21.15.159:0/2173108111 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c02ca80 con 0x7f05341602d0 2024-04-04T06:07:43.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.631+0000 7f0533e00640 1 -- 172.21.15.159:0/2173108111 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f052c02cc20 con 0x7f05341602d0 2024-04-04T06:07:43.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.631+0000 7f0533e00640 1 -- 172.21.15.159:0/2173108111 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c0343f0 con 0x7f05341602d0 2024-04-04T06:07:43.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.632+0000 7f053a000640 1 -- 172.21.15.159:0/2173108111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05341602d0 msgr2=0x7f05340cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:43.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.632+0000 7f053a000640 1 --2- 172.21.15.159:0/2173108111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05341602d0 0x7f05340cf320 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f053406c4a0 tx=0x7f052c02bee0 comp rx=0 tx=0).stop 2024-04-04T06:07:43.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.632+0000 7f053a000640 1 -- 172.21.15.159:0/2173108111 shutdown_connections 2024-04-04T06:07:43.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.632+0000 7f053a000640 1 --2- 172.21.15.159:0/2173108111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05341602d0 0x7f05340cf320 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:43.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.632+0000 7f053a000640 1 --2- 172.21.15.159:0/2173108111 >> v2:172.21.15.178:3300/0 conn(0x7f053406e030 0x7f053415fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:43.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.632+0000 7f053a000640 1 -- 172.21.15.159:0/2173108111 >> 172.21.15.159:0/2173108111 conn(0x7f053406b570 msgr2=0x7f053406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:43.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.632+0000 7f053a000640 1 -- 172.21.15.159:0/2173108111 shutdown_connections 2024-04-04T06:07:43.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.632+0000 7f053a000640 1 -- 172.21.15.159:0/2173108111 wait complete. 2024-04-04T06:07:43.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.634+0000 7f053a000640 1 Processor -- start 2024-04-04T06:07:43.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.634+0000 7f053a000640 1 -- start start 2024-04-04T06:07:43.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.635+0000 7f053a000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f053406e030 0x7f05341f7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:43.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.635+0000 7f053a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f053413f300 0x7f05340b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:43.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.635+0000 7f053a000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05341602d0 0x7f05340a96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:43.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.635+0000 7f053a000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f05340cf860 con 0x7f053413f300 2024-04-04T06:07:43.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.635+0000 7f053a000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f05340ceb20 con 0x7f05341602d0 2024-04-04T06:07:43.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.635+0000 7f053a000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f05340cf9e0 con 0x7f053406e030 2024-04-04T06:07:43.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.635+0000 7f052be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f053413f300 0x7f05340b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:43.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.635+0000 7f052be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f053413f300 0x7f05340b0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56966/0 (socket says 172.21.15.159:56966) 2024-04-04T06:07:43.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.635+0000 7f052be00640 1 -- 172.21.15.159:0/3958162616 learned_addr learned my addr 172.21.15.159:0/3958162616 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:43.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.635+0000 7f052be00640 1 -- 172.21.15.159:0/3958162616 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f053406e030 msgr2=0x7f05341f7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:43.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.635+0000 7f0538c00640 1 --2- 172.21.15.159:0/3958162616 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f053406e030 0x7f05341f7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:43.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.635+0000 7f0539600640 1 --2- 172.21.15.159:0/3958162616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05341602d0 0x7f05340a96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:43.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.635+0000 7f052be00640 1 --2- 172.21.15.159:0/3958162616 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f053406e030 0x7f05341f7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:43.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.636+0000 7f052be00640 1 -- 172.21.15.159:0/3958162616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05341602d0 msgr2=0x7f05340a96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:43.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.636+0000 7f052be00640 1 --2- 172.21.15.159:0/3958162616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05341602d0 0x7f05340a96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:43.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.636+0000 7f052be00640 1 -- 172.21.15.159:0/3958162616 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05340b11f0 con 0x7f053413f300 2024-04-04T06:07:43.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.636+0000 7f0539600640 1 --2- 172.21.15.159:0/3958162616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05341602d0 0x7f05340a96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:43.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.636+0000 7f052be00640 1 --2- 172.21.15.159:0/3958162616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f053413f300 0x7f05340b0cb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f051c00e730 tx=0x7f051c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:43.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.636+0000 7f0531600640 1 -- 172.21.15.159:0/3958162616 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c00f040 con 0x7f053413f300 2024-04-04T06:07:43.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.636+0000 7f0531600640 1 -- 172.21.15.159:0/3958162616 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f051c004230 con 0x7f053413f300 2024-04-04T06:07:43.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.636+0000 7f0531600640 1 -- 172.21.15.159:0/3958162616 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c00ee40 con 0x7f053413f300 2024-04-04T06:07:43.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.636+0000 7f053a000640 1 -- 172.21.15.159:0/3958162616 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05340b14c0 con 0x7f053413f300 2024-04-04T06:07:43.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.636+0000 7f053a000640 1 -- 172.21.15.159:0/3958162616 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05340b1a80 con 0x7f053413f300 2024-04-04T06:07:43.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.638+0000 7f053a000640 1 -- 172.21.15.159:0/3958162616 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0534075940 con 0x7f053413f300 2024-04-04T06:07:43.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.638+0000 7f0531600640 1 -- 172.21.15.159:0/3958162616 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f051c00c6d0 con 0x7f053413f300 2024-04-04T06:07:43.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.640+0000 7f0531600640 1 --2- 172.21.15.159:0/3958162616 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f051006c8f0 0x7f051006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:43.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.640+0000 7f0531600640 1 -- 172.21.15.159:0/3958162616 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f051c08b650 con 0x7f053413f300 2024-04-04T06:07:43.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.640+0000 7f0538c00640 1 --2- 172.21.15.159:0/3958162616 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f051006c8f0 0x7f051006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:43.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.641+0000 7f0538c00640 1 --2- 172.21.15.159:0/3958162616 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f051006c8f0 0x7f051006edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0524003d90 tx=0x7f0524009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:43.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.644+0000 7f0531600640 1 -- 172.21.15.159:0/3958162616 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f051c017020 con 0x7f053413f300 2024-04-04T06:07:43.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.922+0000 7f053a000640 1 -- 172.21.15.159:0/3958162616 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7f0534072a00 con 0x7f053413f300 2024-04-04T06:07:43.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.945+0000 7f0531600640 1 -- 172.21.15.159:0/3958162616 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v125) ==== 142+0+0 (secure 0 0 0) 0x7f051c059ce0 con 0x7f053413f300 2024-04-04T06:07:43.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.948+0000 7f053a000640 1 -- 172.21.15.159:0/3958162616 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f051006c8f0 msgr2=0x7f051006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:43.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.948+0000 7f053a000640 1 --2- 172.21.15.159:0/3958162616 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f051006c8f0 0x7f051006edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0524003d90 tx=0x7f0524009040 comp rx=0 tx=0).stop 2024-04-04T06:07:43.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.948+0000 7f053a000640 1 -- 172.21.15.159:0/3958162616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f053413f300 msgr2=0x7f05340b0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:43.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.948+0000 7f053a000640 1 --2- 172.21.15.159:0/3958162616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f053413f300 0x7f05340b0cb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f051c00e730 tx=0x7f051c00ebf0 comp rx=0 tx=0).stop 2024-04-04T06:07:43.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.948+0000 7f053a000640 1 -- 172.21.15.159:0/3958162616 shutdown_connections 2024-04-04T06:07:43.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.948+0000 7f053a000640 1 --2- 172.21.15.159:0/3958162616 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f051006c8f0 0x7f051006edb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:43.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.948+0000 7f053a000640 1 --2- 172.21.15.159:0/3958162616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05341602d0 0x7f05340a96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:43.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.948+0000 7f053a000640 1 --2- 172.21.15.159:0/3958162616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f053413f300 0x7f05340b0cb0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:43.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.948+0000 7f053a000640 1 --2- 172.21.15.159:0/3958162616 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f053406e030 0x7f05341f7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:43.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.948+0000 7f053a000640 1 -- 172.21.15.159:0/3958162616 >> 172.21.15.159:0/3958162616 conn(0x7f053406b570 msgr2=0x7f053413f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:43.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.948+0000 7f053a000640 1 -- 172.21.15.159:0/3958162616 shutdown_connections 2024-04-04T06:07:43.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:43.948+0000 7f053a000640 1 -- 172.21.15.159:0/3958162616 wait complete. 2024-04-04T06:07:43.963 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-04T06:07:43.963 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-04-04T06:07:43.963 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-04T06:07:43.964 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:07:44.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.180+0000 7f9316600640 1 Processor -- start 2024-04-04T06:07:44.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.180+0000 7f9316600640 1 -- start start 2024-04-04T06:07:44.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.182+0000 7f9316600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f931015b310 0x7f93100727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:44.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.182+0000 7f9316600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f931015b6f0 con 0x7f93100d0780 2024-04-04T06:07:44.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.182+0000 7f9316600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9310072e80 con 0x7f93100b4f90 2024-04-04T06:07:44.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.182+0000 7f9316600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f9310073160 con 0x7f931015b310 2024-04-04T06:07:44.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.182+0000 7f930f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f931015b310 0x7f93100727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:44.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.182+0000 7f930f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f931015b310 0x7f93100727f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47158/0 (socket says 172.21.15.159:47158) 2024-04-04T06:07:44.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.182+0000 7f930f400640 1 -- 172.21.15.159:0/3596098198 learned_addr learned my addr 172.21.15.159:0/3596098198 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:44.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.182+0000 7f930fe00640 1 --1- 172.21.15.159:0/3596098198 >> v1:172.21.15.159:6789/0 conn(0x7f93100d0780 0x7f93100adc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:33284/0 (socket says 172.21.15.159:33284) 2024-04-04T06:07:44.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.183+0000 7f930f400640 1 -- 172.21.15.159:0/3596098198 >> v1:172.21.15.178:6789/0 conn(0x7f93100b4f90 legacy=0x7f931015f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:44.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.183+0000 7f930f400640 1 -- 172.21.15.159:0/3596098198 >> v1:172.21.15.159:6789/0 conn(0x7f93100d0780 legacy=0x7f93100adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:44.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.183+0000 7f930f400640 1 -- 172.21.15.159:0/3596098198 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931015efe0 con 0x7f931015b310 2024-04-04T06:07:44.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.184+0000 7f930f400640 1 --2- 172.21.15.159:0/3596098198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f931015b310 0x7f93100727f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9308009e50 tx=0x7f930802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c03d3eaaeb013a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:44.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.184+0000 7f930e000640 1 -- 172.21.15.159:0/3596098198 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930802c9f0 con 0x7f931015b310 2024-04-04T06:07:44.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.184+0000 7f930e000640 1 -- 172.21.15.159:0/3596098198 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f930802cb90 con 0x7f931015b310 2024-04-04T06:07:44.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.184+0000 7f930e000640 1 -- 172.21.15.159:0/3596098198 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93080343f0 con 0x7f931015b310 2024-04-04T06:07:44.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.185+0000 7f9316600640 1 -- 172.21.15.159:0/3596098198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f931015b310 msgr2=0x7f93100727f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:44.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.185+0000 7f9316600640 1 --2- 172.21.15.159:0/3596098198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f931015b310 0x7f93100727f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9308009e50 tx=0x7f930802be50 comp rx=0 tx=0).stop 2024-04-04T06:07:44.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.185+0000 7f9316600640 1 -- 172.21.15.159:0/3596098198 shutdown_connections 2024-04-04T06:07:44.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.185+0000 7f9316600640 1 --2- 172.21.15.159:0/3596098198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f931015b310 0x7f93100727f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:44.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.185+0000 7f9316600640 1 -- 172.21.15.159:0/3596098198 >> 172.21.15.159:0/3596098198 conn(0x7f931006b990 msgr2=0x7f931006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:44.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.185+0000 7f9316600640 1 -- 172.21.15.159:0/3596098198 shutdown_connections 2024-04-04T06:07:44.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.185+0000 7f9316600640 1 -- 172.21.15.159:0/3596098198 wait complete. 2024-04-04T06:07:44.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.187+0000 7f9316600640 1 Processor -- start 2024-04-04T06:07:44.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.187+0000 7f9316600640 1 -- start start 2024-04-04T06:07:44.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.188+0000 7f9316600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93100b4f90 0x7f931015e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:44.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.188+0000 7f9316600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93100d0780 0x7f93101e8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:44.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.189+0000 7f9316600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f931015b310 0x7f93101e8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:44.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.189+0000 7f9316600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9310073440 con 0x7f931015b310 2024-04-04T06:07:44.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.189+0000 7f9316600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f93100b53f0 con 0x7f93100d0780 2024-04-04T06:07:44.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.189+0000 7f9316600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f93100ae390 con 0x7f93100b4f90 2024-04-04T06:07:44.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.189+0000 7f930fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f931015b310 0x7f93101e8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:44.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.189+0000 7f930fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f931015b310 0x7f93101e8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47164/0 (socket says 172.21.15.159:47164) 2024-04-04T06:07:44.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.189+0000 7f930fe00640 1 -- 172.21.15.159:0/320408904 learned_addr learned my addr 172.21.15.159:0/320408904 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:44.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.189+0000 7f930f400640 1 --2- 172.21.15.159:0/320408904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93100b4f90 0x7f931015e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:44.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.189+0000 7f930ea00640 1 --2- 172.21.15.159:0/320408904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93100d0780 0x7f93101e8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:44.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.189+0000 7f930fe00640 1 -- 172.21.15.159:0/320408904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93100b4f90 msgr2=0x7f931015e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:44.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.189+0000 7f930fe00640 1 --2- 172.21.15.159:0/320408904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93100b4f90 0x7f931015e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:44.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.189+0000 7f930fe00640 1 -- 172.21.15.159:0/320408904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93100d0780 msgr2=0x7f93101e8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:44.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.189+0000 7f930fe00640 1 --2- 172.21.15.159:0/320408904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93100d0780 0x7f93101e8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:44.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.189+0000 7f930fe00640 1 -- 172.21.15.159:0/320408904 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931015bb10 con 0x7f931015b310 2024-04-04T06:07:44.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.189+0000 7f930f400640 1 --2- 172.21.15.159:0/320408904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93100b4f90 0x7f931015e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:44.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.190+0000 7f930fe00640 1 --2- 172.21.15.159:0/320408904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f931015b310 0x7f93101e8e40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f930000c400 tx=0x7f930000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:44.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.190+0000 7f9307400640 1 -- 172.21.15.159:0/320408904 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93000074e0 con 0x7f931015b310 2024-04-04T06:07:44.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.190+0000 7f9307400640 1 -- 172.21.15.159:0/320408904 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9300007680 con 0x7f931015b310 2024-04-04T06:07:44.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.190+0000 7f9316600640 1 -- 172.21.15.159:0/320408904 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931015bdf0 con 0x7f931015b310 2024-04-04T06:07:44.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.190+0000 7f9307400640 1 -- 172.21.15.159:0/320408904 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93000049e0 con 0x7f931015b310 2024-04-04T06:07:44.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.190+0000 7f9316600640 1 -- 172.21.15.159:0/320408904 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f931015c3b0 con 0x7f931015b310 2024-04-04T06:07:44.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.191+0000 7f9316600640 1 -- 172.21.15.159:0/320408904 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93100cfce0 con 0x7f931015b310 2024-04-04T06:07:44.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.192+0000 7f9307400640 1 -- 172.21.15.159:0/320408904 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f930001a020 con 0x7f931015b310 2024-04-04T06:07:44.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.193+0000 7f9307400640 1 --2- 172.21.15.159:0/320408904 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f92ec06c8f0 0x7f92ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:44.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.193+0000 7f9307400640 1 -- 172.21.15.159:0/320408904 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f930008afe0 con 0x7f931015b310 2024-04-04T06:07:44.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.194+0000 7f930f400640 1 --2- 172.21.15.159:0/320408904 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f92ec06c8f0 0x7f92ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:44.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.195+0000 7f930f400640 1 --2- 172.21.15.159:0/320408904 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f92ec06c8f0 0x7f92ec06edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9308035000 tx=0x7f9308030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:44.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.197+0000 7f9307400640 1 -- 172.21.15.159:0/320408904 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f93100cfce0 con 0x7f931015b310 2024-04-04T06:07:44.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.397+0000 7f9316600640 1 -- 172.21.15.159:0/320408904 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f931015c5b0 con 0x7f931015b310 2024-04-04T06:07:44.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.399+0000 7f9307400640 1 -- 172.21.15.159:0/320408904 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+13034 (secure 0 0 0) 0x7f931015c5b0 con 0x7f931015b310 2024-04-04T06:07:44.393 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:44.393 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":37,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:07:42.933839+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6836/3282963883":"2024-04-05T06:07:39.496464+0000","172.21.15.178:6834/3282963883":"2024-04-05T06:07:39.496464+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:07:44.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.402+0000 7f9316600640 1 -- 172.21.15.159:0/320408904 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f92ec06c8f0 msgr2=0x7f92ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:44.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.402+0000 7f9316600640 1 --2- 172.21.15.159:0/320408904 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f92ec06c8f0 0x7f92ec06edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9308035000 tx=0x7f9308030000 comp rx=0 tx=0).stop 2024-04-04T06:07:44.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.402+0000 7f9316600640 1 -- 172.21.15.159:0/320408904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f931015b310 msgr2=0x7f93101e8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:44.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.402+0000 7f9316600640 1 --2- 172.21.15.159:0/320408904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f931015b310 0x7f93101e8e40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f930000c400 tx=0x7f930000c8c0 comp rx=0 tx=0).stop 2024-04-04T06:07:44.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.402+0000 7f9316600640 1 -- 172.21.15.159:0/320408904 shutdown_connections 2024-04-04T06:07:44.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.402+0000 7f9316600640 1 --2- 172.21.15.159:0/320408904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f931015b310 0x7f93101e8e40 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:44.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.402+0000 7f9316600640 1 --2- 172.21.15.159:0/320408904 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f92ec06c8f0 0x7f92ec06edb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:44.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.402+0000 7f9316600640 1 --2- 172.21.15.159:0/320408904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93100d0780 0x7f93101e8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:44.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.402+0000 7f9316600640 1 --2- 172.21.15.159:0/320408904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f93100b4f90 0x7f931015e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:44.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.402+0000 7f9316600640 1 -- 172.21.15.159:0/320408904 >> 172.21.15.159:0/320408904 conn(0x7f931006b990 msgr2=0x7f93100aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:44.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.402+0000 7f9316600640 1 -- 172.21.15.159:0/320408904 shutdown_connections 2024-04-04T06:07:44.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.402+0000 7f9316600640 1 -- 172.21.15.159:0/320408904 wait complete. 2024-04-04T06:07:44.417 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:07:44.417 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:07:44.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.640+0000 7faa8f200640 1 Processor -- start 2024-04-04T06:07:44.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.640+0000 7faa8f200640 1 -- start start 2024-04-04T06:07:44.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.642+0000 7faa8f200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7faa8815e6d0 0x7faa880d21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:44.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.642+0000 7faa8f200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7faa880d4760 0x7faa880cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:44.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.642+0000 7faa8f200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa880ceef0 con 0x7faa880d0cb0 2024-04-04T06:07:44.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.642+0000 7faa8f200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7faa8815eab0 con 0x7faa880d4760 2024-04-04T06:07:44.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.642+0000 7faa8f200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7faa880d1220 con 0x7faa8815e6d0 2024-04-04T06:07:44.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.643+0000 7faa8d200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7faa880d4760 0x7faa880cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:44.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.643+0000 7faa8d200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7faa880d4760 0x7faa880cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47168/0 (socket says 172.21.15.159:47168) 2024-04-04T06:07:44.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.643+0000 7faa8d200640 1 -- 172.21.15.159:0/325898983 learned_addr learned my addr 172.21.15.159:0/325898983 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:44.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.643+0000 7faa87e00640 1 --2- 172.21.15.159:0/325898983 >> v2:172.21.15.178:3300/0 conn(0x7faa8815e6d0 0x7faa880d21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:44.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.643+0000 7faa8d200640 1 -- 172.21.15.159:0/325898983 >> v2:172.21.15.178:3300/0 conn(0x7faa8815e6d0 msgr2=0x7faa880d21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:44.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.643+0000 7faa8d200640 1 --2- 172.21.15.159:0/325898983 >> v2:172.21.15.178:3300/0 conn(0x7faa8815e6d0 0x7faa880d21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:44.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.643+0000 7faa8d200640 1 -- 172.21.15.159:0/325898983 >> v1:172.21.15.178:6789/0 conn(0x7faa880d0cb0 legacy=0x7faa880d4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:44.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.643+0000 7faa8d200640 1 -- 172.21.15.159:0/325898983 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa880d1500 con 0x7faa880d4760 2024-04-04T06:07:44.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.644+0000 7faa8d200640 1 --2- 172.21.15.159:0/325898983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa880d4760 0x7faa880cf3d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7faa8808cdc0 tx=0x7faa8002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d26806fa0f05a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:44.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.645+0000 7faa86a00640 1 -- 172.21.15.159:0/325898983 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80034070 con 0x7faa880d4760 2024-04-04T06:07:44.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.645+0000 7faa86a00640 1 -- 172.21.15.159:0/325898983 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa80030860 con 0x7faa880d4760 2024-04-04T06:07:44.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.645+0000 7faa86a00640 1 -- 172.21.15.159:0/325898983 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80030c30 con 0x7faa880d4760 2024-04-04T06:07:44.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.645+0000 7faa8f200640 1 -- 172.21.15.159:0/325898983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa880d4760 msgr2=0x7faa880cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:44.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.645+0000 7faa8f200640 1 --2- 172.21.15.159:0/325898983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa880d4760 0x7faa880cf3d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7faa8808cdc0 tx=0x7faa8002b9a0 comp rx=0 tx=0).stop 2024-04-04T06:07:44.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.645+0000 7faa8f200640 1 -- 172.21.15.159:0/325898983 shutdown_connections 2024-04-04T06:07:44.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.645+0000 7faa8f200640 1 --2- 172.21.15.159:0/325898983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa880d4760 0x7faa880cf3d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:44.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.645+0000 7faa8f200640 1 --2- 172.21.15.159:0/325898983 >> v2:172.21.15.178:3300/0 conn(0x7faa8815e6d0 0x7faa880d21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:44.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.645+0000 7faa8f200640 1 -- 172.21.15.159:0/325898983 >> 172.21.15.159:0/325898983 conn(0x7faa8809d7b0 msgr2=0x7faa8808bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:44.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.645+0000 7faa8f200640 1 -- 172.21.15.159:0/325898983 shutdown_connections 2024-04-04T06:07:44.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.645+0000 7faa8f200640 1 -- 172.21.15.159:0/325898983 wait complete. 2024-04-04T06:07:44.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.647+0000 7faa8f200640 1 Processor -- start 2024-04-04T06:07:44.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.647+0000 7faa8f200640 1 -- start start 2024-04-04T06:07:44.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.648+0000 7faa8f200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa880d0cb0 0x7faa880ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:44.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.648+0000 7faa8f200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa880d4760 0x7faa88159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:44.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.648+0000 7faa8f200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa8815e6d0 0x7faa8815a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:44.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.648+0000 7faa8f200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faa880d2720 con 0x7faa880d4760 2024-04-04T06:07:44.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.648+0000 7faa8f200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faa880d1ac0 con 0x7faa880d0cb0 2024-04-04T06:07:44.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.648+0000 7faa8f200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7faa8806c890 con 0x7faa8815e6d0 2024-04-04T06:07:44.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.648+0000 7faa87400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa880d4760 0x7faa88159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:44.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.648+0000 7faa87400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa880d4760 0x7faa88159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47172/0 (socket says 172.21.15.159:47172) 2024-04-04T06:07:44.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.648+0000 7faa87400640 1 -- 172.21.15.159:0/493662408 learned_addr learned my addr 172.21.15.159:0/493662408 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:44.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.649+0000 7faa87e00640 1 --2- 172.21.15.159:0/493662408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa880d0cb0 0x7faa880ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:44.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.649+0000 7faa87400640 1 -- 172.21.15.159:0/493662408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa8815e6d0 msgr2=0x7faa8815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:44.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.649+0000 7faa87400640 1 --2- 172.21.15.159:0/493662408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa8815e6d0 0x7faa8815a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:44.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.649+0000 7faa87400640 1 -- 172.21.15.159:0/493662408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa880d0cb0 msgr2=0x7faa880ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:44.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.649+0000 7faa87400640 1 --2- 172.21.15.159:0/493662408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa880d0cb0 0x7faa880ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:44.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.649+0000 7faa87400640 1 -- 172.21.15.159:0/493662408 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa880b9360 con 0x7faa880d4760 2024-04-04T06:07:44.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.649+0000 7faa87e00640 1 --2- 172.21.15.159:0/493662408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa880d0cb0 0x7faa880ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:44.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.649+0000 7faa87400640 1 --2- 172.21.15.159:0/493662408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa880d4760 0x7faa88159bb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faa7000c440 tx=0x7faa7000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:44.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.649+0000 7faa7fe00640 1 -- 172.21.15.159:0/493662408 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70017070 con 0x7faa880d4760 2024-04-04T06:07:44.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.650+0000 7faa7fe00640 1 -- 172.21.15.159:0/493662408 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa700052f0 con 0x7faa880d4760 2024-04-04T06:07:44.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.650+0000 7faa8f200640 1 -- 172.21.15.159:0/493662408 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa880b9640 con 0x7faa880d4760 2024-04-04T06:07:44.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.650+0000 7faa7fe00640 1 -- 172.21.15.159:0/493662408 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70003660 con 0x7faa880d4760 2024-04-04T06:07:44.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.650+0000 7faa8f200640 1 -- 172.21.15.159:0/493662408 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa880c81d0 con 0x7faa880d4760 2024-04-04T06:07:44.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.651+0000 7faa8f200640 1 -- 172.21.15.159:0/493662408 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa880685c0 con 0x7faa880d4760 2024-04-04T06:07:44.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.652+0000 7faa7fe00640 1 -- 172.21.15.159:0/493662408 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa7001a070 con 0x7faa880d4760 2024-04-04T06:07:44.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.653+0000 7faa7fe00640 1 --2- 172.21.15.159:0/493662408 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faa6406c920 0x7faa6406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:44.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.653+0000 7faa7fe00640 1 -- 172.21.15.159:0/493662408 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7faa7008ac10 con 0x7faa880d4760 2024-04-04T06:07:44.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.653+0000 7faa87e00640 1 --2- 172.21.15.159:0/493662408 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faa6406c920 0x7faa6406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:44.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.654+0000 7faa87e00640 1 --2- 172.21.15.159:0/493662408 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faa6406c920 0x7faa6406ede0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faa78007520 tx=0x7faa7800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:44.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.657+0000 7faa7fe00640 1 -- 172.21.15.159:0/493662408 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7faa700592b0 con 0x7faa880d4760 2024-04-04T06:07:44.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.857+0000 7faa8f200640 1 -- 172.21.15.159:0/493662408 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faa8806c670 con 0x7faa880d4760 2024-04-04T06:07:44.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.858+0000 7faa7fe00640 1 -- 172.21.15.159:0/493662408 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) ==== 93+0+10 (secure 0 0 0) 0x7faa70058bb0 con 0x7faa880d4760 2024-04-04T06:07:44.852 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:07:44.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.860+0000 7faa8f200640 1 -- 172.21.15.159:0/493662408 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faa6406c920 msgr2=0x7faa6406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:44.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.860+0000 7faa8f200640 1 --2- 172.21.15.159:0/493662408 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faa6406c920 0x7faa6406ede0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faa78007520 tx=0x7faa7800b040 comp rx=0 tx=0).stop 2024-04-04T06:07:44.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.860+0000 7faa8f200640 1 -- 172.21.15.159:0/493662408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa880d4760 msgr2=0x7faa88159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:44.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.860+0000 7faa8f200640 1 --2- 172.21.15.159:0/493662408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa880d4760 0x7faa88159bb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faa7000c440 tx=0x7faa7000c900 comp rx=0 tx=0).stop 2024-04-04T06:07:44.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.860+0000 7faa8f200640 1 -- 172.21.15.159:0/493662408 shutdown_connections 2024-04-04T06:07:44.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.860+0000 7faa8f200640 1 --2- 172.21.15.159:0/493662408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa8815e6d0 0x7faa8815a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:44.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.860+0000 7faa8f200640 1 --2- 172.21.15.159:0/493662408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa880d4760 0x7faa88159bb0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:44.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.860+0000 7faa8f200640 1 --2- 172.21.15.159:0/493662408 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faa6406c920 0x7faa6406ede0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:44.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.860+0000 7faa8f200640 1 --2- 172.21.15.159:0/493662408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa880d0cb0 0x7faa880ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:44.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.860+0000 7faa8f200640 1 -- 172.21.15.159:0/493662408 >> 172.21.15.159:0/493662408 conn(0x7faa8809d7b0 msgr2=0x7faa880cf0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:44.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.861+0000 7faa8f200640 1 -- 172.21.15.159:0/493662408 shutdown_connections 2024-04-04T06:07:44.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:44.861+0000 7faa8f200640 1 -- 172.21.15.159:0/493662408 wait complete. 2024-04-04T06:07:44.875 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-04T06:07:45.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.092+0000 7f4410600640 1 Processor -- start 2024-04-04T06:07:45.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.092+0000 7f4410600640 1 -- start start 2024-04-04T06:07:45.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.094+0000 7f4410600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f44080d0990 0x7f44080aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:45.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.094+0000 7f4410600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44080b40b0 con 0x7f4408155930 2024-04-04T06:07:45.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.094+0000 7f4410600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44080b4390 con 0x7f4408155550 2024-04-04T06:07:45.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.094+0000 7f4410600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f44080b4670 con 0x7f44080d0990 2024-04-04T06:07:45.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.094+0000 7f440d200640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f4408155930 0x7f44080d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:33294/0 (socket says 172.21.15.159:33294) 2024-04-04T06:07:45.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.094+0000 7f440d200640 1 -- 172.21.15.159:0/1503631901 learned_addr learned my addr 172.21.15.159:0/1503631901 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:45.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.095+0000 7f440e600640 1 --2- 172.21.15.159:0/1503631901 >> v2:172.21.15.178:3300/0 conn(0x7f44080d0990 0x7f44080aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:45.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.095+0000 7f4407e00640 1 -- 172.21.15.159:0/1503631901 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3804470603 0 0) 0x7f44080b40b0 con 0x7f4408155930 2024-04-04T06:07:45.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.095+0000 7f4407e00640 1 -- 172.21.15.159:0/1503631901 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43ec003520 con 0x7f4408155930 2024-04-04T06:07:45.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.095+0000 7f4407e00640 1 -- 172.21.15.159:0/1503631901 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1734149776 0 0) 0x7f43ec003520 con 0x7f4408155930 2024-04-04T06:07:45.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.095+0000 7f4407e00640 1 -- 172.21.15.159:0/1503631901 >> v2:172.21.15.178:3300/0 conn(0x7f44080d0990 msgr2=0x7f44080aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:45.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.095+0000 7f4407e00640 1 --2- 172.21.15.159:0/1503631901 >> v2:172.21.15.178:3300/0 conn(0x7f44080d0990 0x7f44080aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:45.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.095+0000 7f4407e00640 1 -- 172.21.15.159:0/1503631901 >> v1:172.21.15.178:6789/0 conn(0x7f4408155550 legacy=0x7f440815f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:45.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.095+0000 7f4407e00640 1 -- 172.21.15.159:0/1503631901 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44080b4950 con 0x7f4408155930 2024-04-04T06:07:45.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.095+0000 7f4407e00640 1 -- 172.21.15.159:0/1503631901 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f43f4001670 con 0x7f4408155930 2024-04-04T06:07:45.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.095+0000 7f4407e00640 1 -- 172.21.15.159:0/1503631901 >> v1:172.21.15.159:6789/0 conn(0x7f4408155930 legacy=0x7f44080d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:45.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.095+0000 7f4407e00640 1 --2- 172.21.15.159:0/1503631901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43ec0039b0 0x7f43ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:45.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.095+0000 7f4407e00640 1 --2- 172.21.15.159:0/1503631901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43ec004800 0x7f43ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:45.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.095+0000 7f4407e00640 1 --2- 172.21.15.159:0/1503631901 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43ec0051c0 0x7f43ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:45.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.095+0000 7f4407e00640 1 -- 172.21.15.159:0/1503631901 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f44080b4670 con 0x7f43ec0039b0 2024-04-04T06:07:45.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.095+0000 7f4407e00640 1 -- 172.21.15.159:0/1503631901 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec00c160 con 0x7f43ec004800 2024-04-04T06:07:45.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.095+0000 7f4407e00640 1 -- 172.21.15.159:0/1503631901 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f43ec00e570 con 0x7f43ec0051c0 2024-04-04T06:07:45.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.095+0000 7f440dc00640 1 --2- 172.21.15.159:0/1503631901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43ec0039b0 0x7f43ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:45.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.096+0000 7f440d200640 1 --2- 172.21.15.159:0/1503631901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43ec004800 0x7f43ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:45.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.096+0000 7f440dc00640 1 -- 172.21.15.159:0/1503631901 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43ec0051c0 msgr2=0x7f43ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:45.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.096+0000 7f440dc00640 1 --2- 172.21.15.159:0/1503631901 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43ec0051c0 0x7f43ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:45.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.096+0000 7f440dc00640 1 -- 172.21.15.159:0/1503631901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43ec004800 msgr2=0x7f43ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:45.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.096+0000 7f440dc00640 1 --2- 172.21.15.159:0/1503631901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43ec004800 0x7f43ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:45.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.096+0000 7f440dc00640 1 -- 172.21.15.159:0/1503631901 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ec003520 con 0x7f43ec0039b0 2024-04-04T06:07:45.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.096+0000 7f440e600640 1 -- 172.21.15.159:0/1503631901 reap_dead start 2024-04-04T06:07:45.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.097+0000 7f440dc00640 1 --2- 172.21.15.159:0/1503631901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43ec0039b0 0x7f43ec003dd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f43f8001750 tx=0x7f43ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40444bdf069253fb server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:45.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.097+0000 7f4407e00640 1 -- 172.21.15.159:0/1503631901 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8001370 con 0x7f43ec0039b0 2024-04-04T06:07:45.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.097+0000 7f4407e00640 1 -- 172.21.15.159:0/1503631901 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43f80052d0 con 0x7f43ec0039b0 2024-04-04T06:07:45.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.097+0000 7f4407e00640 1 -- 172.21.15.159:0/1503631901 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8003830 con 0x7f43ec0039b0 2024-04-04T06:07:45.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.098+0000 7f4410600640 1 -- 172.21.15.159:0/1503631901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43ec0039b0 msgr2=0x7f43ec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:45.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.098+0000 7f4410600640 1 --2- 172.21.15.159:0/1503631901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43ec0039b0 0x7f43ec003dd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f43f8001750 tx=0x7f43ec005830 comp rx=0 tx=0).stop 2024-04-04T06:07:45.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.098+0000 7f4410600640 1 -- 172.21.15.159:0/1503631901 shutdown_connections 2024-04-04T06:07:45.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.098+0000 7f4410600640 1 --2- 172.21.15.159:0/1503631901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43ec0039b0 0x7f43ec003dd0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:45.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.098+0000 7f4410600640 1 -- 172.21.15.159:0/1503631901 >> 172.21.15.159:0/1503631901 conn(0x7f440806b750 msgr2=0x7f440806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:45.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.098+0000 7f4410600640 1 -- 172.21.15.159:0/1503631901 shutdown_connections 2024-04-04T06:07:45.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.098+0000 7f4410600640 1 -- 172.21.15.159:0/1503631901 wait complete. 2024-04-04T06:07:45.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.100+0000 7f4410600640 1 Processor -- start 2024-04-04T06:07:45.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.100+0000 7f4410600640 1 -- start start 2024-04-04T06:07:45.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.101+0000 7f4410600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43ec0039b0 0x7f44080992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:45.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.101+0000 7f4410600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44081e6d10 0x7f440815e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.101+0000 7f4410600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f440815e790 0x7f44081f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.101+0000 7f4410600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f440815bba0 con 0x7f44081e6d10 2024-04-04T06:07:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.101+0000 7f4410600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f440815ba20 con 0x7f43ec0039b0 2024-04-04T06:07:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.101+0000 7f4410600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f44080b3cf0 con 0x7f440815e790 2024-04-04T06:07:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.101+0000 7f440d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44081e6d10 0x7f440815e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.101+0000 7f440d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44081e6d10 0x7f440815e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47194/0 (socket says 172.21.15.159:47194) 2024-04-04T06:07:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.101+0000 7f440d200640 1 -- 172.21.15.159:0/1039818836 learned_addr learned my addr 172.21.15.159:0/1039818836 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.101+0000 7f440dc00640 1 --2- 172.21.15.159:0/1039818836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43ec0039b0 0x7f44080992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.101+0000 7f440d200640 1 -- 172.21.15.159:0/1039818836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f440815e790 msgr2=0x7f44081f45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.101+0000 7f440d200640 1 --2- 172.21.15.159:0/1039818836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f440815e790 0x7f44081f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.102+0000 7f440d200640 1 -- 172.21.15.159:0/1039818836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43ec0039b0 msgr2=0x7f44080992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.102+0000 7f440d200640 1 --2- 172.21.15.159:0/1039818836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43ec0039b0 0x7f44080992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.102+0000 7f440d200640 1 -- 172.21.15.159:0/1039818836 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44080ae6a0 con 0x7f44081e6d10 2024-04-04T06:07:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.102+0000 7f440dc00640 1 --2- 172.21.15.159:0/1039818836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43ec0039b0 0x7f44080992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.102+0000 7f440d200640 1 --2- 172.21.15.159:0/1039818836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44081e6d10 0x7f440815e250 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f43f4001050 tx=0x7f43f400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:45.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.102+0000 7f4405600640 1 -- 172.21.15.159:0/1039818836 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f400ee70 con 0x7f44081e6d10 2024-04-04T06:07:45.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.102+0000 7f4405600640 1 -- 172.21.15.159:0/1039818836 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43f4018070 con 0x7f44081e6d10 2024-04-04T06:07:45.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.102+0000 7f4405600640 1 -- 172.21.15.159:0/1039818836 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f400ca50 con 0x7f44081e6d10 2024-04-04T06:07:45.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.102+0000 7f4410600640 1 -- 172.21.15.159:0/1039818836 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44080ae960 con 0x7f44081e6d10 2024-04-04T06:07:45.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.102+0000 7f4410600640 1 -- 172.21.15.159:0/1039818836 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44080aef20 con 0x7f44081e6d10 2024-04-04T06:07:45.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.103+0000 7f4410600640 1 -- 172.21.15.159:0/1039818836 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4408154c80 con 0x7f44081e6d10 2024-04-04T06:07:45.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.104+0000 7f4405600640 1 -- 172.21.15.159:0/1039818836 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f43f400cbf0 con 0x7f44081e6d10 2024-04-04T06:07:45.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.105+0000 7f4405600640 1 --2- 172.21.15.159:0/1039818836 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43e806c8f0 0x7f43e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:45.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.105+0000 7f4405600640 1 -- 172.21.15.159:0/1039818836 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f43f408b6e0 con 0x7f44081e6d10 2024-04-04T06:07:45.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.106+0000 7f440dc00640 1 --2- 172.21.15.159:0/1039818836 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43e806c8f0 0x7f43e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:45.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.107+0000 7f440dc00640 1 --2- 172.21.15.159:0/1039818836 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43e806c8f0 0x7f43e806edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f43f8033000 tx=0x7f43f8009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:45.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.110+0000 7f4405600640 1 -- 172.21.15.159:0/1039818836 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4408154c80 con 0x7f44081e6d10 2024-04-04T06:07:45.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.384+0000 7f4410600640 1 -- 172.21.15.159:0/1039818836 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f44080685c0 con 0x7f44081e6d10 2024-04-04T06:07:45.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.958+0000 7f4405600640 1 -- 172.21.15.159:0/1039818836 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v127) ==== 152+0+0 (secure 0 0 0) 0x7f43f4059d70 con 0x7f44081e6d10 2024-04-04T06:07:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.961+0000 7f4410600640 1 -- 172.21.15.159:0/1039818836 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43e806c8f0 msgr2=0x7f43e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.961+0000 7f4410600640 1 --2- 172.21.15.159:0/1039818836 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43e806c8f0 0x7f43e806edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f43f8033000 tx=0x7f43f8009030 comp rx=0 tx=0).stop 2024-04-04T06:07:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.961+0000 7f4410600640 1 -- 172.21.15.159:0/1039818836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44081e6d10 msgr2=0x7f440815e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.961+0000 7f4410600640 1 --2- 172.21.15.159:0/1039818836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44081e6d10 0x7f440815e250 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f43f4001050 tx=0x7f43f400ec80 comp rx=0 tx=0).stop 2024-04-04T06:07:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.962+0000 7f4410600640 1 -- 172.21.15.159:0/1039818836 shutdown_connections 2024-04-04T06:07:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.962+0000 7f4410600640 1 --2- 172.21.15.159:0/1039818836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f440815e790 0x7f44081f45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.962+0000 7f4410600640 1 --2- 172.21.15.159:0/1039818836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44081e6d10 0x7f440815e250 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.962+0000 7f4410600640 1 --2- 172.21.15.159:0/1039818836 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43e806c8f0 0x7f43e806edb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.962+0000 7f4410600640 1 --2- 172.21.15.159:0/1039818836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43ec0039b0 0x7f44080992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:45.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.962+0000 7f4410600640 1 -- 172.21.15.159:0/1039818836 >> 172.21.15.159:0/1039818836 conn(0x7f440806b750 msgr2=0x7f440815aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:45.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.962+0000 7f4410600640 1 -- 172.21.15.159:0/1039818836 shutdown_connections 2024-04-04T06:07:45.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:45.962+0000 7f4410600640 1 -- 172.21.15.159:0/1039818836 wait complete. 2024-04-04T06:07:45.980 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-04T06:07:46.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.203+0000 7fc8f9600640 1 Processor -- start 2024-04-04T06:07:46.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.203+0000 7fc8f9600640 1 -- start start 2024-04-04T06:07:46.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.205+0000 7fc8f9600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc8f40d0280 0x7fc8f40d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:46.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.205+0000 7fc8f9600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc8f4141c10 0x7fc8f4071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:46.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.205+0000 7fc8f9600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8f415ec80 con 0x7fc8f4071eb0 2024-04-04T06:07:46.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.205+0000 7fc8f9600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fc8f40d0ba0 con 0x7fc8f40d0280 2024-04-04T06:07:46.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.205+0000 7fc8f9600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc8f415ef20 con 0x7fc8f4141c10 2024-04-04T06:07:46.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.205+0000 7fc8f2a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc8f40d0280 0x7fc8f40d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:46.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.205+0000 7fc8f2a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc8f40d0280 0x7fc8f40d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47208/0 (socket says 172.21.15.159:47208) 2024-04-04T06:07:46.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.205+0000 7fc8f2a00640 1 -- 172.21.15.159:0/4039696020 learned_addr learned my addr 172.21.15.159:0/4039696020 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:46.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.205+0000 7fc8f2000640 1 --2- 172.21.15.159:0/4039696020 >> v2:172.21.15.178:3300/0 conn(0x7fc8f4141c10 0x7fc8f4071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:46.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.205+0000 7fc8f2a00640 1 -- 172.21.15.159:0/4039696020 >> v2:172.21.15.178:3300/0 conn(0x7fc8f4141c10 msgr2=0x7fc8f4071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:46.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.205+0000 7fc8f2a00640 1 --2- 172.21.15.159:0/4039696020 >> v2:172.21.15.178:3300/0 conn(0x7fc8f4141c10 0x7fc8f4071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:46.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.206+0000 7fc8f2a00640 1 -- 172.21.15.159:0/4039696020 >> v1:172.21.15.178:6789/0 conn(0x7fc8f4071eb0 legacy=0x7fc8f406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:46.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.206+0000 7fc8f2a00640 1 -- 172.21.15.159:0/4039696020 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f415f200 con 0x7fc8f40d0280 2024-04-04T06:07:46.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.207+0000 7fc8f2a00640 1 --2- 172.21.15.159:0/4039696020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8f40d0280 0x7fc8f40d0660 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40b5270 tx=0x7fc8e402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3397052d314f44e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:46.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.207+0000 7fc8f1600640 1 -- 172.21.15.159:0/4039696020 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4035020 con 0x7fc8f40d0280 2024-04-04T06:07:46.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.207+0000 7fc8f1600640 1 -- 172.21.15.159:0/4039696020 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e4031870 con 0x7fc8f40d0280 2024-04-04T06:07:46.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.207+0000 7fc8f1600640 1 -- 172.21.15.159:0/4039696020 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4031c40 con 0x7fc8f40d0280 2024-04-04T06:07:46.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.208+0000 7fc8f9600640 1 -- 172.21.15.159:0/4039696020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8f40d0280 msgr2=0x7fc8f40d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:46.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.208+0000 7fc8f9600640 1 --2- 172.21.15.159:0/4039696020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8f40d0280 0x7fc8f40d0660 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40b5270 tx=0x7fc8e402b910 comp rx=0 tx=0).stop 2024-04-04T06:07:46.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.208+0000 7fc8f9600640 1 -- 172.21.15.159:0/4039696020 shutdown_connections 2024-04-04T06:07:46.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.208+0000 7fc8f9600640 1 --2- 172.21.15.159:0/4039696020 >> v2:172.21.15.178:3300/0 conn(0x7fc8f4141c10 0x7fc8f4071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:46.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.208+0000 7fc8f9600640 1 --2- 172.21.15.159:0/4039696020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8f40d0280 0x7fc8f40d0660 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:46.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.208+0000 7fc8f9600640 1 -- 172.21.15.159:0/4039696020 >> 172.21.15.159:0/4039696020 conn(0x7fc8f406b750 msgr2=0x7fc8f406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:46.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.208+0000 7fc8f9600640 1 -- 172.21.15.159:0/4039696020 shutdown_connections 2024-04-04T06:07:46.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.208+0000 7fc8f9600640 1 -- 172.21.15.159:0/4039696020 wait complete. 2024-04-04T06:07:46.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.209+0000 7fc8f9600640 1 Processor -- start 2024-04-04T06:07:46.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.210+0000 7fc8f9600640 1 -- start start 2024-04-04T06:07:46.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.211+0000 7fc8f9600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8f4071eb0 0x7fc8f4157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:46.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.211+0000 7fc8f9600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8f40d0280 0x7fc8f4143070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:46.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.211+0000 7fc8f9600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8f4141c10 0x7fc8f41435b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:46.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.211+0000 7fc8f9600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4158e90 con 0x7fc8f40d0280 2024-04-04T06:07:46.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.211+0000 7fc8f9600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f415f7c0 con 0x7fc8f4141c10 2024-04-04T06:07:46.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.211+0000 7fc8f9600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f41ea060 con 0x7fc8f4071eb0 2024-04-04T06:07:46.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.211+0000 7fc8f2000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8f40d0280 0x7fc8f4143070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:46.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.211+0000 7fc8f2000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8f40d0280 0x7fc8f4143070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47210/0 (socket says 172.21.15.159:47210) 2024-04-04T06:07:46.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.211+0000 7fc8f2000640 1 -- 172.21.15.159:0/3114785130 learned_addr learned my addr 172.21.15.159:0/3114785130 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:46.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.211+0000 7fc8f2a00640 1 --2- 172.21.15.159:0/3114785130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8f4071eb0 0x7fc8f4157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:46.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.211+0000 7fc8f3400640 1 --2- 172.21.15.159:0/3114785130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8f4141c10 0x7fc8f41435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:46.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.211+0000 7fc8f2000640 1 -- 172.21.15.159:0/3114785130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8f4071eb0 msgr2=0x7fc8f4157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:46.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.211+0000 7fc8f2000640 1 --2- 172.21.15.159:0/3114785130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8f4071eb0 0x7fc8f4157d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:46.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.211+0000 7fc8f2000640 1 -- 172.21.15.159:0/3114785130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8f4141c10 msgr2=0x7fc8f41435b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:46.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.211+0000 7fc8f2000640 1 --2- 172.21.15.159:0/3114785130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8f4141c10 0x7fc8f41435b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:46.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.211+0000 7fc8f2000640 1 -- 172.21.15.159:0/3114785130 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f41e9450 con 0x7fc8f40d0280 2024-04-04T06:07:46.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.212+0000 7fc8f2000640 1 --2- 172.21.15.159:0/3114785130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8f40d0280 0x7fc8f4143070 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc8e000d5d0 tx=0x7fc8e000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:46.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.212+0000 7fc8eaa00640 1 -- 172.21.15.159:0/3114785130 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00125f0 con 0x7fc8f40d0280 2024-04-04T06:07:46.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.212+0000 7fc8eaa00640 1 -- 172.21.15.159:0/3114785130 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e0019070 con 0x7fc8f40d0280 2024-04-04T06:07:46.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.212+0000 7fc8f9600640 1 -- 172.21.15.159:0/3114785130 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f41e9730 con 0x7fc8f40d0280 2024-04-04T06:07:46.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.212+0000 7fc8eaa00640 1 -- 172.21.15.159:0/3114785130 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00129b0 con 0x7fc8f40d0280 2024-04-04T06:07:46.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.212+0000 7fc8f9600640 1 -- 172.21.15.159:0/3114785130 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f41e9cf0 con 0x7fc8f40d0280 2024-04-04T06:07:46.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.213+0000 7fc8f9600640 1 -- 172.21.15.159:0/3114785130 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f40685c0 con 0x7fc8f40d0280 2024-04-04T06:07:46.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.214+0000 7fc8eaa00640 1 -- 172.21.15.159:0/3114785130 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8e0012b50 con 0x7fc8f40d0280 2024-04-04T06:07:46.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.215+0000 7fc8eaa00640 1 --2- 172.21.15.159:0/3114785130 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc8d406c8f0 0x7fc8d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:46.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.215+0000 7fc8eaa00640 1 -- 172.21.15.159:0/3114785130 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fc8e008bc20 con 0x7fc8f40d0280 2024-04-04T06:07:46.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.216+0000 7fc8f2a00640 1 --2- 172.21.15.159:0/3114785130 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc8d406c8f0 0x7fc8d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:46.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.217+0000 7fc8f2a00640 1 --2- 172.21.15.159:0/3114785130 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc8d406c8f0 0x7fc8d406edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4036000 tx=0x7fc8e403c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:46.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.220+0000 7fc8eaa00640 1 -- 172.21.15.159:0/3114785130 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc8e005a2c0 con 0x7fc8f40d0280 2024-04-04T06:07:46.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.426+0000 7fc8f9600640 1 -- 172.21.15.159:0/3114785130 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc8f4158640 con 0x7fc8f40d0280 2024-04-04T06:07:46.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.433+0000 7fc8eaa00640 1 -- 172.21.15.159:0/3114785130 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7fc8e0059bc0 con 0x7fc8f40d0280 2024-04-04T06:07:46.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.436+0000 7fc8f9600640 1 -- 172.21.15.159:0/3114785130 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc8d406c8f0 msgr2=0x7fc8d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:46.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.436+0000 7fc8f9600640 1 --2- 172.21.15.159:0/3114785130 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc8d406c8f0 0x7fc8d406edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4036000 tx=0x7fc8e403c040 comp rx=0 tx=0).stop 2024-04-04T06:07:46.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.436+0000 7fc8f9600640 1 -- 172.21.15.159:0/3114785130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8f40d0280 msgr2=0x7fc8f4143070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:46.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.436+0000 7fc8f9600640 1 --2- 172.21.15.159:0/3114785130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8f40d0280 0x7fc8f4143070 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc8e000d5d0 tx=0x7fc8e000da90 comp rx=0 tx=0).stop 2024-04-04T06:07:46.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.436+0000 7fc8f9600640 1 -- 172.21.15.159:0/3114785130 shutdown_connections 2024-04-04T06:07:46.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.436+0000 7fc8f9600640 1 --2- 172.21.15.159:0/3114785130 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc8d406c8f0 0x7fc8d406edb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:46.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.436+0000 7fc8f9600640 1 --2- 172.21.15.159:0/3114785130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8f4141c10 0x7fc8f41435b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:46.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.436+0000 7fc8f9600640 1 --2- 172.21.15.159:0/3114785130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8f40d0280 0x7fc8f4143070 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:46.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.436+0000 7fc8f9600640 1 --2- 172.21.15.159:0/3114785130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc8f4071eb0 0x7fc8f4157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:46.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.436+0000 7fc8f9600640 1 -- 172.21.15.159:0/3114785130 >> 172.21.15.159:0/3114785130 conn(0x7fc8f406b750 msgr2=0x7fc8f40b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:46.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.436+0000 7fc8f9600640 1 -- 172.21.15.159:0/3114785130 shutdown_connections 2024-04-04T06:07:46.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:46.436+0000 7fc8f9600640 1 -- 172.21.15.159:0/3114785130 wait complete. 2024-04-04T06:07:46.451 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-04T06:07:46.451 DEBUG:teuthology.orchestra.run.smithi159:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:07:46.565 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:07:46.690 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-04T06:07:46.690 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:07:46.690 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:07:46.747 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:07:46.747 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:07:46.747 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:07:46.803 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:07:46.804 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:07:46.804 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:07:46.860 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:07:46.860 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:07:46.860 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:07:46.916 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:07:46.916 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-04T06:07:46.916 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:07:47.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.139+0000 7f9da6800640 1 Processor -- start 2024-04-04T06:07:47.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.139+0000 7f9da6800640 1 -- start start 2024-04-04T06:07:47.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.140+0000 7f9da6800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9da00d0390 0x7f9da00cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:47.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.140+0000 7f9da6800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9da0161f30 con 0x7f9da00ce1b0 2024-04-04T06:07:47.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.140+0000 7f9da6800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9da01621d0 con 0x7f9da0147a40 2024-04-04T06:07:47.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.140+0000 7f9da6800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f9da0162b00 con 0x7f9da00d0390 2024-04-04T06:07:47.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.141+0000 7f9da5e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f9da00ce1b0 0x7f9da006da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:33308/0 (socket says 172.21.15.159:33308) 2024-04-04T06:07:47.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.141+0000 7f9da5e00640 1 -- 172.21.15.159:0/3546308489 learned_addr learned my addr 172.21.15.159:0/3546308489 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:47.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.141+0000 7f9da4a00640 1 --2- 172.21.15.159:0/3546308489 >> v2:172.21.15.178:3300/0 conn(0x7f9da00d0390 0x7f9da00cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:47.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.141+0000 7f9d9fe00640 1 -- 172.21.15.159:0/3546308489 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1475376664 0 0) 0x7f9da0161f30 con 0x7f9da00ce1b0 2024-04-04T06:07:47.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.141+0000 7f9d9fe00640 1 -- 172.21.15.159:0/3546308489 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d88003520 con 0x7f9da00ce1b0 2024-04-04T06:07:47.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.142+0000 7f9d9fe00640 1 -- 172.21.15.159:0/3546308489 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 359670013 0 0) 0x7f9d88003520 con 0x7f9da00ce1b0 2024-04-04T06:07:47.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.142+0000 7f9d9fe00640 1 -- 172.21.15.159:0/3546308489 >> v2:172.21.15.178:3300/0 conn(0x7f9da00d0390 msgr2=0x7f9da00cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:47.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.142+0000 7f9d9fe00640 1 --2- 172.21.15.159:0/3546308489 >> v2:172.21.15.178:3300/0 conn(0x7f9da00d0390 0x7f9da00cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:47.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.142+0000 7f9d9fe00640 1 -- 172.21.15.159:0/3546308489 >> v1:172.21.15.178:6789/0 conn(0x7f9da0147a40 legacy=0x7f9da00cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:47.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.142+0000 7f9d9fe00640 1 -- 172.21.15.159:0/3546308489 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da01624b0 con 0x7f9da00ce1b0 2024-04-04T06:07:47.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.142+0000 7f9d9fe00640 1 -- 172.21.15.159:0/3546308489 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f9d98001a10 con 0x7f9da00ce1b0 2024-04-04T06:07:47.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.142+0000 7f9d9fe00640 1 -- 172.21.15.159:0/3546308489 >> v1:172.21.15.159:6789/0 conn(0x7f9da00ce1b0 legacy=0x7f9da006da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:47.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.142+0000 7f9d9fe00640 1 --2- 172.21.15.159:0/3546308489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d880039b0 0x7f9d88003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:47.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.142+0000 7f9d9fe00640 1 --2- 172.21.15.159:0/3546308489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d88004800 0x7f9d88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:47.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.142+0000 7f9d9fe00640 1 --2- 172.21.15.159:0/3546308489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d880051c0 0x7f9d8800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:47.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.142+0000 7f9d9fe00640 1 -- 172.21.15.159:0/3546308489 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9da0162b00 con 0x7f9d880039b0 2024-04-04T06:07:47.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.142+0000 7f9d9fe00640 1 -- 172.21.15.159:0/3546308489 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8800c0f0 con 0x7f9d88004800 2024-04-04T06:07:47.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.142+0000 7f9d9fe00640 1 -- 172.21.15.159:0/3546308489 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9d8800c2f0 con 0x7f9d880051c0 2024-04-04T06:07:47.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.142+0000 7f9da5400640 1 --2- 172.21.15.159:0/3546308489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d880039b0 0x7f9d88003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:47.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.142+0000 7f9da5400640 1 -- 172.21.15.159:0/3546308489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d880051c0 msgr2=0x7f9d8800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:47.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.142+0000 7f9da5400640 1 --2- 172.21.15.159:0/3546308489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d880051c0 0x7f9d8800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:47.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.142+0000 7f9da5400640 1 -- 172.21.15.159:0/3546308489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d88004800 msgr2=0x7f9d88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:47.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.142+0000 7f9da4a00640 1 --2- 172.21.15.159:0/3546308489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d88004800 0x7f9d88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:47.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.142+0000 7f9da5400640 1 --2- 172.21.15.159:0/3546308489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d88004800 0x7f9d88004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:47.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.142+0000 7f9da5400640 1 -- 172.21.15.159:0/3546308489 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d88003520 con 0x7f9d880039b0 2024-04-04T06:07:47.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.142+0000 7f9da5e00640 1 -- 172.21.15.159:0/3546308489 reap_dead start 2024-04-04T06:07:47.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.144+0000 7f9da5400640 1 --2- 172.21.15.159:0/3546308489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d880039b0 0x7f9d88003db0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9d94003d80 tx=0x7f9d88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e103cb24fc1060a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:47.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.144+0000 7f9d9fe00640 1 -- 172.21.15.159:0/3546308489 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94037070 con 0x7f9d880039b0 2024-04-04T06:07:47.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.144+0000 7f9d9fe00640 1 -- 172.21.15.159:0/3546308489 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d94032600 con 0x7f9d880039b0 2024-04-04T06:07:47.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.144+0000 7f9d9fe00640 1 -- 172.21.15.159:0/3546308489 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94003980 con 0x7f9d880039b0 2024-04-04T06:07:47.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.144+0000 7f9da6800640 1 -- 172.21.15.159:0/3546308489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d880039b0 msgr2=0x7f9d88003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:47.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.144+0000 7f9da6800640 1 --2- 172.21.15.159:0/3546308489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d880039b0 0x7f9d88003db0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9d94003d80 tx=0x7f9d88005830 comp rx=0 tx=0).stop 2024-04-04T06:07:47.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.144+0000 7f9da6800640 1 -- 172.21.15.159:0/3546308489 shutdown_connections 2024-04-04T06:07:47.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.144+0000 7f9da6800640 1 --2- 172.21.15.159:0/3546308489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d880039b0 0x7f9d88003db0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:47.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.145+0000 7f9da6800640 1 -- 172.21.15.159:0/3546308489 >> 172.21.15.159:0/3546308489 conn(0x7f9da00ccad0 msgr2=0x7f9da00cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:47.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.145+0000 7f9da6800640 1 -- 172.21.15.159:0/3546308489 shutdown_connections 2024-04-04T06:07:47.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.145+0000 7f9da6800640 1 -- 172.21.15.159:0/3546308489 wait complete. 2024-04-04T06:07:47.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.146+0000 7f9da6800640 1 Processor -- start 2024-04-04T06:07:47.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.146+0000 7f9da6800640 1 -- start start 2024-04-04T06:07:47.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.147+0000 7f9da6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d880039b0 0x7f9da0133010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:47.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.147+0000 7f9da6800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9da0136b40 0x7f9da01424a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:47.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.147+0000 7f9da6800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9da00b01b0 0x7f9da0130f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:47.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.147+0000 7f9da6800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9da0147fa0 con 0x7f9d880039b0 2024-04-04T06:07:47.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.147+0000 7f9da6800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9da0147e20 con 0x7f9da00b01b0 2024-04-04T06:07:47.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.147+0000 7f9da6800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9da00ce610 con 0x7f9da0136b40 2024-04-04T06:07:47.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.148+0000 7f9da5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d880039b0 0x7f9da0133010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:47.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.148+0000 7f9da5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d880039b0 0x7f9da0133010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47232/0 (socket says 172.21.15.159:47232) 2024-04-04T06:07:47.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.148+0000 7f9da5400640 1 -- 172.21.15.159:0/2506135931 learned_addr learned my addr 172.21.15.159:0/2506135931 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:47.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.148+0000 7f9da5400640 1 -- 172.21.15.159:0/2506135931 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9da0136b40 msgr2=0x7f9da01424a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:47.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.148+0000 7f9da5400640 1 --2- 172.21.15.159:0/2506135931 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9da0136b40 0x7f9da01424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:47.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.148+0000 7f9da5400640 1 -- 172.21.15.159:0/2506135931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9da00b01b0 msgr2=0x7f9da0130f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:47.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.148+0000 7f9da5400640 1 --2- 172.21.15.159:0/2506135931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9da00b01b0 0x7f9da0130f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:47.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.148+0000 7f9da5400640 1 -- 172.21.15.159:0/2506135931 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da0133910 con 0x7f9d880039b0 2024-04-04T06:07:47.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.148+0000 7f9da5400640 1 --2- 172.21.15.159:0/2506135931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d880039b0 0x7f9da0133010 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9d94035040 tx=0x7f9d940036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:47.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.149+0000 7f9d9d600640 1 -- 172.21.15.159:0/2506135931 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94042030 con 0x7f9d880039b0 2024-04-04T06:07:47.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.149+0000 7f9d9d600640 1 -- 172.21.15.159:0/2506135931 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d94032400 con 0x7f9d880039b0 2024-04-04T06:07:47.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.149+0000 7f9d9d600640 1 -- 172.21.15.159:0/2506135931 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94037040 con 0x7f9d880039b0 2024-04-04T06:07:47.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.149+0000 7f9da6800640 1 -- 172.21.15.159:0/2506135931 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da0133bf0 con 0x7f9d880039b0 2024-04-04T06:07:47.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.149+0000 7f9da6800640 1 -- 172.21.15.159:0/2506135931 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da00af9b0 con 0x7f9d880039b0 2024-04-04T06:07:47.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.150+0000 7f9da6800640 1 -- 172.21.15.159:0/2506135931 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da0075940 con 0x7f9d880039b0 2024-04-04T06:07:47.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.151+0000 7f9d9d600640 1 -- 172.21.15.159:0/2506135931 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d94045070 con 0x7f9d880039b0 2024-04-04T06:07:47.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.152+0000 7f9d9d600640 1 --2- 172.21.15.159:0/2506135931 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9d8006c920 0x7f9d8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:47.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.152+0000 7f9d9d600640 1 -- 172.21.15.159:0/2506135931 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f9d940acf50 con 0x7f9d880039b0 2024-04-04T06:07:47.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.152+0000 7f9da4a00640 1 --2- 172.21.15.159:0/2506135931 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9d8006c920 0x7f9d8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:47.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.153+0000 7f9da4a00640 1 --2- 172.21.15.159:0/2506135931 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9d8006c920 0x7f9d8006ede0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c006a90 tx=0x7f9d8c006a20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:47.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.158+0000 7f9d9d600640 1 -- 172.21.15.159:0/2506135931 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9d9404c070 con 0x7f9d880039b0 2024-04-04T06:07:47.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.400+0000 7f9da6800640 1 -- 172.21.15.159:0/2506135931 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9da00008d0 con 0x7f9d880039b0 2024-04-04T06:07:47.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.402+0000 7f9d9d600640 1 -- 172.21.15.159:0/2506135931 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3881 (secure 0 0 0) 0x7f9da00008d0 con 0x7f9d880039b0 2024-04-04T06:07:47.396 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:47.396 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1643905381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1643905381},{"type":"v1","addr":"172.21.15.178:6837","nonce":1643905381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4744,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2629539137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2629539137},{"type":"v1","addr":"172.21.15.159:6835","nonce":2629539137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1287769267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1287769267},{"type":"v1","addr":"172.21.15.178:6836","nonce":1287769267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3402172592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3402172592},{"type":"v1","addr":"172.21.15.159:6833","nonce":3402172592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-04T06:07:47.396 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 19 2024-04-04T06:07:47.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.404+0000 7f9da6800640 1 -- 172.21.15.159:0/2506135931 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9d8006c920 msgr2=0x7f9d8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:47.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.404+0000 7f9da6800640 1 --2- 172.21.15.159:0/2506135931 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9d8006c920 0x7f9d8006ede0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c006a90 tx=0x7f9d8c006a20 comp rx=0 tx=0).stop 2024-04-04T06:07:47.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.404+0000 7f9da6800640 1 -- 172.21.15.159:0/2506135931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d880039b0 msgr2=0x7f9da0133010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:47.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.404+0000 7f9da6800640 1 --2- 172.21.15.159:0/2506135931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d880039b0 0x7f9da0133010 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9d94035040 tx=0x7f9d940036f0 comp rx=0 tx=0).stop 2024-04-04T06:07:47.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.405+0000 7f9da6800640 1 -- 172.21.15.159:0/2506135931 shutdown_connections 2024-04-04T06:07:47.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.405+0000 7f9da6800640 1 --2- 172.21.15.159:0/2506135931 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9d8006c920 0x7f9d8006ede0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:47.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.405+0000 7f9da6800640 1 --2- 172.21.15.159:0/2506135931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9da00b01b0 0x7f9da0130f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:47.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.405+0000 7f9da6800640 1 --2- 172.21.15.159:0/2506135931 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9da0136b40 0x7f9da01424a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:47.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.405+0000 7f9da6800640 1 --2- 172.21.15.159:0/2506135931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d880039b0 0x7f9da0133010 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:47.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.405+0000 7f9da6800640 1 -- 172.21.15.159:0/2506135931 >> 172.21.15.159:0/2506135931 conn(0x7f9da00ccad0 msgr2=0x7f9da00cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:47.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.405+0000 7f9da6800640 1 -- 172.21.15.159:0/2506135931 shutdown_connections 2024-04-04T06:07:47.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.405+0000 7f9da6800640 1 -- 172.21.15.159:0/2506135931 wait complete. 2024-04-04T06:07:47.420 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-04T06:07:47.420 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-04T06:07:47.420 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-04T06:07:47.420 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:07:47.420 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-04T06:07:47.420 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-04T06:07:47.421 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-04T06:07:47.421 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:07:47.421 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-04T06:07:47.421 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-04T06:07:47.421 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-04T06:07:47.421 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:07:47.421 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-04T06:07:47.421 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-04T06:07:47.422 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-04T06:07:47.422 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:07:47.438 INFO:tasks.ceph.mds.c:Stopped 2024-04-04T06:07:47.438 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-04T06:07:47.443 INFO:tasks.ceph.mds.a:Stopped 2024-04-04T06:07:47.443 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-04T06:07:47.445 INFO:tasks.ceph.mds.b:Stopped 2024-04-04T06:07:47.445 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-04T06:07:47.446 INFO:tasks.ceph.mds.d:Stopped 2024-04-04T06:07:47.446 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-04T06:07:47.489 INFO:tasks.ceph.mds.c:Started 2024-04-04T06:07:47.490 DEBUG:teuthology.parallel:result is None 2024-04-04T06:07:47.493 INFO:tasks.ceph.mds.b:Started 2024-04-04T06:07:47.493 DEBUG:teuthology.parallel:result is None 2024-04-04T06:07:47.495 INFO:tasks.ceph.mds.a:Started 2024-04-04T06:07:47.495 DEBUG:teuthology.parallel:result is None 2024-04-04T06:07:47.499 INFO:tasks.ceph.mds.d:Started 2024-04-04T06:07:47.499 DEBUG:teuthology.parallel:result is None 2024-04-04T06:07:47.499 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-04T06:07:47.646 INFO:tasks.ceph.mds.c.smithi159.stdout:starting mds.c at 2024-04-04T06:07:47.648 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-04T06:07:47.650 INFO:tasks.ceph.mds.a.smithi159.stdout:starting mds.a at 2024-04-04T06:07:47.657 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-04T06:07:47.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.693+0000 7f23bf200640 1 Processor -- start 2024-04-04T06:07:47.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.693+0000 7f23bf200640 1 -- start start 2024-04-04T06:07:47.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.695+0000 7f23bf200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f23b80d0990 0x7f23b81670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:47.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.695+0000 7f23bf200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f23b8163ac0 0x7f23b8071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:47.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.695+0000 7f23bf200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23b8163ee0 con 0x7f23b8162920 2024-04-04T06:07:47.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.695+0000 7f23bf200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f23b8071d60 con 0x7f23b80d0990 2024-04-04T06:07:47.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.695+0000 7f23bf200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f23b81665d0 con 0x7f23b8163ac0 2024-04-04T06:07:47.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.695+0000 7f23b7400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f23b80d0990 0x7f23b81670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:47.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.695+0000 7f23b7e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f23b8162920 0x7f23b80d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:33314/0 (socket says 172.21.15.159:33314) 2024-04-04T06:07:47.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.695+0000 7f23b7e00640 1 -- 172.21.15.159:0/799816428 learned_addr learned my addr 172.21.15.159:0/799816428 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:47.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.695+0000 7f23bd200640 1 --2- 172.21.15.159:0/799816428 >> v2:172.21.15.178:3300/0 conn(0x7f23b8163ac0 0x7f23b8071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:47.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.696+0000 7f23b7400640 1 -- 172.21.15.159:0/799816428 >> v2:172.21.15.178:3300/0 conn(0x7f23b8163ac0 msgr2=0x7f23b8071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:47.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.696+0000 7f23b7400640 1 --2- 172.21.15.159:0/799816428 >> v2:172.21.15.178:3300/0 conn(0x7f23b8163ac0 0x7f23b8071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:47.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.696+0000 7f23b7400640 1 -- 172.21.15.159:0/799816428 >> v1:172.21.15.159:6789/0 conn(0x7f23b8162920 legacy=0x7f23b80d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:47.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.696+0000 7f23b7400640 1 -- 172.21.15.159:0/799816428 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b81668b0 con 0x7f23b80d0990 2024-04-04T06:07:47.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.696+0000 7f23b6a00640 1 -- 172.21.15.159:0/799816428 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 990696164 0 0) 0x7f23b8163ee0 con 0x7f23b8162920 2024-04-04T06:07:47.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.696+0000 7f23bd200640 1 --2- 172.21.15.159:0/799816428 >> v2:172.21.15.178:3300/0 conn(0x7f23b8163ac0 0x7f23b8071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:07:47.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.697+0000 7f23b7400640 1 --2- 172.21.15.159:0/799816428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23b80d0990 0x7f23b81670b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f23b806e8b0 tx=0x7f23a402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb6eceae61fc01 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:47.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.697+0000 7f23b6a00640 1 -- 172.21.15.159:0/799816428 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a40315a0 con 0x7f23b80d0990 2024-04-04T06:07:47.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.697+0000 7f23b6a00640 1 -- 172.21.15.159:0/799816428 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23a4037070 con 0x7f23b80d0990 2024-04-04T06:07:47.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.697+0000 7f23b6a00640 1 -- 172.21.15.159:0/799816428 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a4031960 con 0x7f23b80d0990 2024-04-04T06:07:47.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.698+0000 7f23bf200640 1 -- 172.21.15.159:0/799816428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23b80d0990 msgr2=0x7f23b81670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:47.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.698+0000 7f23bf200640 1 --2- 172.21.15.159:0/799816428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23b80d0990 0x7f23b81670b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f23b806e8b0 tx=0x7f23a402b840 comp rx=0 tx=0).stop 2024-04-04T06:07:47.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.698+0000 7f23bf200640 1 -- 172.21.15.159:0/799816428 shutdown_connections 2024-04-04T06:07:47.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.698+0000 7f23bf200640 1 --2- 172.21.15.159:0/799816428 >> v2:172.21.15.178:3300/0 conn(0x7f23b8163ac0 0x7f23b8071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:47.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.698+0000 7f23bf200640 1 --2- 172.21.15.159:0/799816428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23b80d0990 0x7f23b81670b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:47.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.698+0000 7f23bf200640 1 -- 172.21.15.159:0/799816428 >> 172.21.15.159:0/799816428 conn(0x7f23b806b750 msgr2=0x7f23b806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:47.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.698+0000 7f23bf200640 1 -- 172.21.15.159:0/799816428 shutdown_connections 2024-04-04T06:07:47.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.698+0000 7f23bf200640 1 -- 172.21.15.159:0/799816428 wait complete. 2024-04-04T06:07:47.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.699+0000 7f23bf200640 1 Processor -- start 2024-04-04T06:07:47.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.700+0000 7f23bf200640 1 -- start start 2024-04-04T06:07:47.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.700+0000 7f23bf200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23b80d0990 0x7f23b80b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:47.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.700+0000 7f23bf200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23b8162920 0x7f23b80cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:47.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.701+0000 7f23bf200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23b8163ac0 0x7f23b81640e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:47.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.701+0000 7f23bf200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f23b8072040 con 0x7f23b8162920 2024-04-04T06:07:47.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.701+0000 7f23bf200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f23b8162d00 con 0x7f23b8163ac0 2024-04-04T06:07:47.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.701+0000 7f23bf200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f23b80721c0 con 0x7f23b80d0990 2024-04-04T06:07:47.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.701+0000 7f23b7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23b8162920 0x7f23b80cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:47.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.701+0000 7f23b7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23b8162920 0x7f23b80cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47278/0 (socket says 172.21.15.159:47278) 2024-04-04T06:07:47.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.701+0000 7f23b7400640 1 -- 172.21.15.159:0/3192413427 learned_addr learned my addr 172.21.15.159:0/3192413427 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:47.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.701+0000 7f23bd200640 1 --2- 172.21.15.159:0/3192413427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23b8163ac0 0x7f23b81640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:47.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.701+0000 7f23b7e00640 1 --2- 172.21.15.159:0/3192413427 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23b80d0990 0x7f23b80b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:47.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.701+0000 7f23b7400640 1 -- 172.21.15.159:0/3192413427 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23b80d0990 msgr2=0x7f23b80b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:47.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.701+0000 7f23b7400640 1 --2- 172.21.15.159:0/3192413427 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23b80d0990 0x7f23b80b40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:47.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.701+0000 7f23b7400640 1 -- 172.21.15.159:0/3192413427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23b8163ac0 msgr2=0x7f23b81640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:47.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.701+0000 7f23b7400640 1 --2- 172.21.15.159:0/3192413427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23b8163ac0 0x7f23b81640e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:47.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.701+0000 7f23b7400640 1 -- 172.21.15.159:0/3192413427 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b81ec9c0 con 0x7f23b8162920 2024-04-04T06:07:47.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.702+0000 7f23b7400640 1 --2- 172.21.15.159:0/3192413427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23b8162920 0x7f23b80cc560 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f23a4034040 tx=0x7f23a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:47.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.702+0000 7f23afe00640 1 -- 172.21.15.159:0/3192413427 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a4004030 con 0x7f23b8162920 2024-04-04T06:07:47.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.702+0000 7f23afe00640 1 -- 172.21.15.159:0/3192413427 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23a40041d0 con 0x7f23b8162920 2024-04-04T06:07:47.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.702+0000 7f23afe00640 1 -- 172.21.15.159:0/3192413427 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a4003610 con 0x7f23b8162920 2024-04-04T06:07:47.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.702+0000 7f23bf200640 1 -- 172.21.15.159:0/3192413427 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b80ccf30 con 0x7f23b8162920 2024-04-04T06:07:47.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.702+0000 7f23bf200640 1 -- 172.21.15.159:0/3192413427 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b80cd770 con 0x7f23b8162920 2024-04-04T06:07:47.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.703+0000 7f23bf200640 1 -- 172.21.15.159:0/3192413427 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23b80685c0 con 0x7f23b8162920 2024-04-04T06:07:47.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.704+0000 7f23afe00640 1 -- 172.21.15.159:0/3192413427 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f23a40037b0 con 0x7f23b8162920 2024-04-04T06:07:47.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.705+0000 7f23afe00640 1 --2- 172.21.15.159:0/3192413427 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f239806c8f0 0x7f239806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:47.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.705+0000 7f23afe00640 1 -- 172.21.15.159:0/3192413427 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f23a402f030 con 0x7f23b8162920 2024-04-04T06:07:47.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.705+0000 7f23b7e00640 1 --2- 172.21.15.159:0/3192413427 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f239806c8f0 0x7f239806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:47.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.706+0000 7f23b7e00640 1 --2- 172.21.15.159:0/3192413427 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f239806c8f0 0x7f239806edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f23a80012e0 tx=0x7f23a8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:47.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.709+0000 7f23afe00640 1 -- 172.21.15.159:0/3192413427 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f23b80ccf30 con 0x7f23b8162920 2024-04-04T06:07:47.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:47.918+0000 7f23bf200640 1 -- 172.21.15.159:0/3192413427 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f23b806ef60 con 0x7f23b8162920 2024-04-04T06:07:48.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:48.995+0000 7f23afe00640 1 -- 172.21.15.159:0/3192413427 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v42) ==== 97+0+0 (secure 0 0 0) 0x7f23a4041070 con 0x7f23b8162920 2024-04-04T06:07:48.990 INFO:teuthology.orchestra.run.smithi159.stderr: removed all blocklist entries 2024-04-04T06:07:48.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:48.999+0000 7f23bf200640 1 -- 172.21.15.159:0/3192413427 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f239806c8f0 msgr2=0x7f239806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:48.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:48.999+0000 7f23bf200640 1 --2- 172.21.15.159:0/3192413427 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f239806c8f0 0x7f239806edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f23a80012e0 tx=0x7f23a8009040 comp rx=0 tx=0).stop 2024-04-04T06:07:48.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.000+0000 7f23bf200640 1 -- 172.21.15.159:0/3192413427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23b8162920 msgr2=0x7f23b80cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:48.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.000+0000 7f23bf200640 1 --2- 172.21.15.159:0/3192413427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23b8162920 0x7f23b80cc560 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f23a4034040 tx=0x7f23a40046c0 comp rx=0 tx=0).stop 2024-04-04T06:07:48.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.000+0000 7f23bf200640 1 -- 172.21.15.159:0/3192413427 shutdown_connections 2024-04-04T06:07:48.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.000+0000 7f23bf200640 1 --2- 172.21.15.159:0/3192413427 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f239806c8f0 0x7f239806edb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:48.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.000+0000 7f23bf200640 1 --2- 172.21.15.159:0/3192413427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23b8163ac0 0x7f23b81640e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:48.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.000+0000 7f23bf200640 1 --2- 172.21.15.159:0/3192413427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23b8162920 0x7f23b80cc560 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:48.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.000+0000 7f23bf200640 1 --2- 172.21.15.159:0/3192413427 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23b80d0990 0x7f23b80b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:48.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.000+0000 7f23bf200640 1 -- 172.21.15.159:0/3192413427 >> 172.21.15.159:0/3192413427 conn(0x7f23b806b750 msgr2=0x7f23b80d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:48.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.000+0000 7f23bf200640 1 -- 172.21.15.159:0/3192413427 shutdown_connections 2024-04-04T06:07:48.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.000+0000 7f23bf200640 1 -- 172.21.15.159:0/3192413427 wait complete. 2024-04-04T06:07:49.013 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-04T06:07:49.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.229+0000 7f84f9c00640 1 Processor -- start 2024-04-04T06:07:49.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.230+0000 7f84f9c00640 1 -- start start 2024-04-04T06:07:49.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.231+0000 7f84f9c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f84f40afee0 0x7f84f40b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:49.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.231+0000 7f84f9c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f84f415f700 0x7f84f406e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:49.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.231+0000 7f84f9c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84f415b8c0 con 0x7f84f415b4e0 2024-04-04T06:07:49.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.231+0000 7f84f9c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f84f40b5810 con 0x7f84f40afee0 2024-04-04T06:07:49.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.231+0000 7f84f9c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f84f40afab0 con 0x7f84f415f700 2024-04-04T06:07:49.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.232+0000 7f84f2a00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f84f415b4e0 0x7f84f415eff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:33330/0 (socket says 172.21.15.159:33330) 2024-04-04T06:07:49.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.232+0000 7f84f2a00640 1 -- 172.21.15.159:0/547202894 learned_addr learned my addr 172.21.15.159:0/547202894 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:49.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.232+0000 7f84f2000640 1 --2- 172.21.15.159:0/547202894 >> v2:172.21.15.159:3300/0 conn(0x7f84f40afee0 0x7f84f40b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:49.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.232+0000 7f84f3400640 1 --2- 172.21.15.159:0/547202894 >> v2:172.21.15.178:3300/0 conn(0x7f84f415f700 0x7f84f406e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:49.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.232+0000 7f84f1600640 1 -- 172.21.15.159:0/547202894 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1752768545 0 0) 0x7f84f415b8c0 con 0x7f84f415b4e0 2024-04-04T06:07:49.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.232+0000 7f84f1600640 1 -- 172.21.15.159:0/547202894 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84d8005730 con 0x7f84f415b4e0 2024-04-04T06:07:49.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.232+0000 7f84f2000640 1 -- 172.21.15.159:0/547202894 >> v2:172.21.15.178:3300/0 conn(0x7f84f415f700 msgr2=0x7f84f406e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:49.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.232+0000 7f84f2000640 1 --2- 172.21.15.159:0/547202894 >> v2:172.21.15.178:3300/0 conn(0x7f84f415f700 0x7f84f406e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:49.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.232+0000 7f84f2000640 1 -- 172.21.15.159:0/547202894 >> v1:172.21.15.159:6789/0 conn(0x7f84f415b4e0 legacy=0x7f84f415eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:49.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.232+0000 7f84f2000640 1 -- 172.21.15.159:0/547202894 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f415fd70 con 0x7f84f40afee0 2024-04-04T06:07:49.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.233+0000 7f84f2000640 1 --2- 172.21.15.159:0/547202894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f40afee0 0x7f84f40b0570 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f84f40680f0 tx=0x7f84dc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8fffa63b779aa server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:49.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.234+0000 7f84f1600640 1 -- 172.21.15.159:0/547202894 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc035020 con 0x7f84f40afee0 2024-04-04T06:07:49.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.234+0000 7f84f1600640 1 -- 172.21.15.159:0/547202894 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84dc031830 con 0x7f84f40afee0 2024-04-04T06:07:49.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.234+0000 7f84f1600640 1 -- 172.21.15.159:0/547202894 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc031c00 con 0x7f84f40afee0 2024-04-04T06:07:49.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.234+0000 7f84f9c00640 1 -- 172.21.15.159:0/547202894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f40afee0 msgr2=0x7f84f40b0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:49.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.234+0000 7f84f9c00640 1 --2- 172.21.15.159:0/547202894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f40afee0 0x7f84f40b0570 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f84f40680f0 tx=0x7f84dc02b8d0 comp rx=0 tx=0).stop 2024-04-04T06:07:49.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.235+0000 7f84f9c00640 1 -- 172.21.15.159:0/547202894 shutdown_connections 2024-04-04T06:07:49.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.235+0000 7f84f9c00640 1 --2- 172.21.15.159:0/547202894 >> v2:172.21.15.178:3300/0 conn(0x7f84f415f700 0x7f84f406e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:49.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.235+0000 7f84f9c00640 1 --2- 172.21.15.159:0/547202894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f40afee0 0x7f84f40b0570 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:49.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.235+0000 7f84f9c00640 1 -- 172.21.15.159:0/547202894 >> 172.21.15.159:0/547202894 conn(0x7f84f40b6560 msgr2=0x7f84f40b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:49.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.235+0000 7f84f9c00640 1 -- 172.21.15.159:0/547202894 shutdown_connections 2024-04-04T06:07:49.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.235+0000 7f84f9c00640 1 -- 172.21.15.159:0/547202894 wait complete. 2024-04-04T06:07:49.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.236+0000 7f84f9c00640 1 Processor -- start 2024-04-04T06:07:49.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.236+0000 7f84f9c00640 1 -- start start 2024-04-04T06:07:49.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.237+0000 7f84f9c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f40afee0 0x7f84f406f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:49.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.237+0000 7f84f9c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84f415b4e0 0x7f84f414aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:49.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.237+0000 7f84f9c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84f415f700 0x7f84f40cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:49.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.237+0000 7f84f9c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84f40b5010 con 0x7f84f40afee0 2024-04-04T06:07:49.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.237+0000 7f84f9c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f84f40b4d30 con 0x7f84f415f700 2024-04-04T06:07:49.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.237+0000 7f84f9c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f84f40b52f0 con 0x7f84f415b4e0 2024-04-04T06:07:49.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.238+0000 7f84f2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f40afee0 0x7f84f406f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:49.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.238+0000 7f84f2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f40afee0 0x7f84f406f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47282/0 (socket says 172.21.15.159:47282) 2024-04-04T06:07:49.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.238+0000 7f84f2a00640 1 -- 172.21.15.159:0/2308093087 learned_addr learned my addr 172.21.15.159:0/2308093087 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:49.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.238+0000 7f84f2a00640 1 -- 172.21.15.159:0/2308093087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84f415b4e0 msgr2=0x7f84f414aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:49.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.238+0000 7f84f2a00640 1 --2- 172.21.15.159:0/2308093087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84f415b4e0 0x7f84f414aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:49.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.238+0000 7f84f2a00640 1 -- 172.21.15.159:0/2308093087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84f415f700 msgr2=0x7f84f40cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:49.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.238+0000 7f84f3400640 1 --2- 172.21.15.159:0/2308093087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84f415f700 0x7f84f40cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:49.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.238+0000 7f84f2a00640 1 --2- 172.21.15.159:0/2308093087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84f415f700 0x7f84f40cadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:49.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.238+0000 7f84f2a00640 1 -- 172.21.15.159:0/2308093087 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f414b010 con 0x7f84f40afee0 2024-04-04T06:07:49.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.238+0000 7f84f2a00640 1 --2- 172.21.15.159:0/2308093087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f40afee0 0x7f84f406f520 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f84e400c430 tx=0x7f84e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:49.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.239+0000 7f84eaa00640 1 -- 172.21.15.159:0/2308093087 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4017070 con 0x7f84f40afee0 2024-04-04T06:07:49.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.239+0000 7f84eaa00640 1 -- 172.21.15.159:0/2308093087 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e4005ce0 con 0x7f84f40afee0 2024-04-04T06:07:49.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.239+0000 7f84eaa00640 1 -- 172.21.15.159:0/2308093087 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e40072f0 con 0x7f84f40afee0 2024-04-04T06:07:49.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.239+0000 7f84f9c00640 1 -- 172.21.15.159:0/2308093087 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f40cb620 con 0x7f84f40afee0 2024-04-04T06:07:49.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.239+0000 7f84f9c00640 1 -- 172.21.15.159:0/2308093087 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f40cbbe0 con 0x7f84f40afee0 2024-04-04T06:07:49.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.240+0000 7f84f9c00640 1 -- 172.21.15.159:0/2308093087 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f40685c0 con 0x7f84f40afee0 2024-04-04T06:07:49.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.241+0000 7f84eaa00640 1 -- 172.21.15.159:0/2308093087 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84e401a070 con 0x7f84f40afee0 2024-04-04T06:07:49.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.242+0000 7f84eaa00640 1 --2- 172.21.15.159:0/2308093087 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f84d006c8f0 0x7f84d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:49.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.243+0000 7f84eaa00640 1 -- 172.21.15.159:0/2308093087 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f84e408ad60 con 0x7f84f40afee0 2024-04-04T06:07:49.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.243+0000 7f84f2000640 1 --2- 172.21.15.159:0/2308093087 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f84d006c8f0 0x7f84d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:49.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.244+0000 7f84f2000640 1 --2- 172.21.15.159:0/2308093087 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f84d006c8f0 0x7f84d006edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f84dc036000 tx=0x7f84dc030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:49.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.247+0000 7f84eaa00640 1 -- 172.21.15.159:0/2308093087 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f84e4059440 con 0x7f84f40afee0 2024-04-04T06:07:49.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.524+0000 7f84f9c00640 1 -- 172.21.15.159:0/2308093087 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f84f4153320 con 0x7f84f40afee0 2024-04-04T06:07:49.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.525+0000 7f84eaa00640 1 -- 172.21.15.159:0/2308093087 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7f84e4058d40 con 0x7f84f40afee0 2024-04-04T06:07:49.519 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:49.520 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:07:49.520 INFO:teuthology.orchestra.run.smithi159.stdout: "auth_dump": [ 2024-04-04T06:07:49.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:49.520 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.a", 2024-04-04T06:07:49.520 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5mvoZ2FxAAsVEQrkiICIH6T7ZzaqcWbA==", 2024-04-04T06:07:49.520 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:49.520 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:07:49.520 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:07:49.520 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:07:49.520 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:49.520 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:49.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:49.520 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:49.520 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.b", 2024-04-04T06:07:49.521 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5m+rccORAA6I/lQ+22QfWYL9+aoOFJ0w==", 2024-04-04T06:07:49.521 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:49.521 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:07:49.521 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:07:49.521 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:07:49.521 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:49.521 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:49.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:49.521 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:49.521 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.c", 2024-04-04T06:07:49.521 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5mjmQcKBAA8BjpQce6G7EycXOkyVECpQ==", 2024-04-04T06:07:49.521 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:49.521 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:07:49.521 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:07:49.521 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:07:49.522 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:49.522 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:49.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:49.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:49.522 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.d", 2024-04-04T06:07:49.522 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5m04hoDRAAKdhRI7nosZp7d4y7dVvzvg==", 2024-04-04T06:07:49.522 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:49.522 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:07:49.522 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:07:49.522 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:07:49.522 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:49.522 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:49.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:49.522 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:49.523 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.0", 2024-04-04T06:07:49.523 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQABQw5mCe7fNRAA6ZKxWvnMLf4jhAlpjudFRA==", 2024-04-04T06:07:49.523 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:49.523 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:07:49.523 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:07:49.523 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:49.523 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:49.523 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:49.523 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:49.523 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.1", 2024-04-04T06:07:49.523 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAFQw5mvIqTExAAU8Pm5alxFtHEL4qM9MavPw==", 2024-04-04T06:07:49.523 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:49.523 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:07:49.523 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:07:49.523 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:49.524 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:49.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:49.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:49.524 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.2", 2024-04-04T06:07:49.524 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAIQw5mAHblHRAAqEDe5dIGTDpdy2OsQyzJXA==", 2024-04-04T06:07:49.524 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:49.524 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:07:49.524 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:07:49.524 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:49.524 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:49.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:49.524 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:49.524 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.3", 2024-04-04T06:07:49.524 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQALQw5mwK7rJhAADQBAqE0Eej9rn91Wxhq/KA==", 2024-04-04T06:07:49.525 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:49.525 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:07:49.525 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:07:49.525 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:49.525 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:49.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:49.525 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:49.525 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.4", 2024-04-04T06:07:49.525 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAQQw5m/6I8GRAA2P7jI+AXYld2J11nDfhAUQ==", 2024-04-04T06:07:49.525 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:49.525 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:07:49.525 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:07:49.525 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:49.525 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:49.525 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:49.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:49.526 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.5", 2024-04-04T06:07:49.526 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQATQw5mv5HoMBAAn2mp2ys5KjS0UygOcX4vvg==", 2024-04-04T06:07:49.526 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:49.526 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:07:49.526 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:07:49.526 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:49.526 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:49.526 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:49.526 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:49.526 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.6", 2024-04-04T06:07:49.526 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAWQw5mx12gORAAT2LhrfEB5BJSl0ypgospzg==", 2024-04-04T06:07:49.526 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:49.526 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:07:49.527 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:07:49.527 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:49.527 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:49.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:49.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:49.527 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.7", 2024-04-04T06:07:49.527 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAaQw5m6HEOBxAAVegLH3bjVM7r1A6tK5eqHQ==", 2024-04-04T06:07:49.527 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:49.527 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:07:49.527 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:07:49.527 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:49.527 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:49.527 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:49.527 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:49.527 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.0", 2024-04-04T06:07:49.528 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mqNKJGxAA4jP9cadhRIyhQ3lssttWIw==", 2024-04-04T06:07:49.528 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:49.528 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:07:49.528 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow r", 2024-04-04T06:07:49.528 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-04T06:07:49.528 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:49.528 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:49.528 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:49.528 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.1", 2024-04-04T06:07:49.528 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mRkdOMBAA1ae1roIwrlB49DFQUoFSTg==", 2024-04-04T06:07:49.528 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:49.528 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:07:49.528 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:07:49.528 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:07:49.528 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:07:49.529 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:49.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:49.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:49.529 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.2", 2024-04-04T06:07:49.529 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mtW2/JRAAucxSSN9q0EQtI2a/P2yozQ==", 2024-04-04T06:07:49.529 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:49.529 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:07:49.529 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:07:49.529 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:07:49.529 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:07:49.529 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:49.529 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:49.529 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:49.529 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.3", 2024-04-04T06:07:49.530 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mgqD0OhAAov+3aDGyGEoqxep8+SBpSA==", 2024-04-04T06:07:49.530 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:49.530 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:07:49.530 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:07:49.530 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:07:49.530 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:07:49.530 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:49.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:49.530 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:49.530 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.admin", 2024-04-04T06:07:49.530 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD7Qg5m0O/iCBAAsWTzlEA0P60XYuxmJiAb4A==", 2024-04-04T06:07:49.530 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:49.530 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow *", 2024-04-04T06:07:49.530 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:07:49.531 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:07:49.531 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:49.531 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:49.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:49.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:49.531 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mgr.x", 2024-04-04T06:07:49.531 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5mhqbBCxAASBknMIj4uu7OiB3jn7WP8Q==", 2024-04-04T06:07:49.531 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:49.531 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow *", 2024-04-04T06:07:49.531 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile mgr", 2024-04-04T06:07:49.531 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:49.531 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:49.531 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:07:49.531 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:49.532 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mgr.y", 2024-04-04T06:07:49.532 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD7Qg5miXT0ORAAL6y306CPAfEGQKAJ0no4uw==", 2024-04-04T06:07:49.532 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:07:49.532 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow *", 2024-04-04T06:07:49.532 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile mgr", 2024-04-04T06:07:49.532 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:07:49.532 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:49.532 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:49.532 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-04-04T06:07:49.532 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:07:49.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.528+0000 7f84f9c00640 1 -- 172.21.15.159:0/2308093087 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f84d006c8f0 msgr2=0x7f84d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:49.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.528+0000 7f84f9c00640 1 --2- 172.21.15.159:0/2308093087 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f84d006c8f0 0x7f84d006edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f84dc036000 tx=0x7f84dc030000 comp rx=0 tx=0).stop 2024-04-04T06:07:49.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.528+0000 7f84f9c00640 1 -- 172.21.15.159:0/2308093087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f40afee0 msgr2=0x7f84f406f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:49.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.528+0000 7f84f9c00640 1 --2- 172.21.15.159:0/2308093087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f40afee0 0x7f84f406f520 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f84e400c430 tx=0x7f84e400c8f0 comp rx=0 tx=0).stop 2024-04-04T06:07:49.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.528+0000 7f84f9c00640 1 -- 172.21.15.159:0/2308093087 shutdown_connections 2024-04-04T06:07:49.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.528+0000 7f84f9c00640 1 --2- 172.21.15.159:0/2308093087 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f84d006c8f0 0x7f84d006edb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:49.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.528+0000 7f84f9c00640 1 --2- 172.21.15.159:0/2308093087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84f415f700 0x7f84f40cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:49.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.528+0000 7f84f9c00640 1 --2- 172.21.15.159:0/2308093087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84f415b4e0 0x7f84f414aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:49.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.528+0000 7f84f9c00640 1 --2- 172.21.15.159:0/2308093087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f40afee0 0x7f84f406f520 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:49.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.528+0000 7f84f9c00640 1 -- 172.21.15.159:0/2308093087 >> 172.21.15.159:0/2308093087 conn(0x7f84f40b6560 msgr2=0x7f84f4158b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:49.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.528+0000 7f84f9c00640 1 -- 172.21.15.159:0/2308093087 shutdown_connections 2024-04-04T06:07:49.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.528+0000 7f84f9c00640 1 -- 172.21.15.159:0/2308093087 wait complete. 2024-04-04T06:07:49.540 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:07:49.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.748+0000 7f8564c00640 1 Processor -- start 2024-04-04T06:07:49.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.749+0000 7f8564c00640 1 -- start start 2024-04-04T06:07:49.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.750+0000 7f8564c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f856015d120 0x7f8560162e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:49.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.750+0000 7f8564c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f85600af630 0x7f8560166b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:49.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.750+0000 7f8564c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85601633d0 con 0x7f8560167080 2024-04-04T06:07:49.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.750+0000 7f8564c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f85600723c0 con 0x7f85600af630 2024-04-04T06:07:49.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.750+0000 7f8564c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8560163650 con 0x7f856015d120 2024-04-04T06:07:49.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.750+0000 7f855e800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f8560167080 0x7f85600b3500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:33336/0 (socket says 172.21.15.159:33336) 2024-04-04T06:07:49.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.750+0000 7f855e800640 1 -- 172.21.15.159:0/2358190145 learned_addr learned my addr 172.21.15.159:0/2358190145 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:49.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.751+0000 7f855de00640 1 --2- 172.21.15.159:0/2358190145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f856015d120 0x7f8560162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:49.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.751+0000 7f855d400640 1 --2- 172.21.15.159:0/2358190145 >> v2:172.21.15.178:3300/0 conn(0x7f85600af630 0x7f8560166b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:49.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.751+0000 7f855ca00640 1 -- 172.21.15.159:0/2358190145 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 42597058 0 0) 0x7f8558001490 con 0x7f8560167080 2024-04-04T06:07:49.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.751+0000 7f855ca00640 1 -- 172.21.15.159:0/2358190145 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8544003520 con 0x7f8560167080 2024-04-04T06:07:49.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.751+0000 7f855ca00640 1 -- 172.21.15.159:0/2358190145 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3052917977 0 0) 0x7f8544003520 con 0x7f8560167080 2024-04-04T06:07:49.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.751+0000 7f855ca00640 1 -- 172.21.15.159:0/2358190145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f856015d120 msgr2=0x7f8560162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:49.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.751+0000 7f855ca00640 1 --2- 172.21.15.159:0/2358190145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f856015d120 0x7f8560162e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:49.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.751+0000 7f855ca00640 1 -- 172.21.15.159:0/2358190145 >> v2:172.21.15.178:3300/0 conn(0x7f85600af630 msgr2=0x7f8560166b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:49.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.751+0000 7f855ca00640 1 --2- 172.21.15.159:0/2358190145 >> v2:172.21.15.178:3300/0 conn(0x7f85600af630 0x7f8560166b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:49.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.751+0000 7f855ca00640 1 -- 172.21.15.159:0/2358190145 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8560163930 con 0x7f8560167080 2024-04-04T06:07:49.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.751+0000 7f855ca00640 1 -- 172.21.15.159:0/2358190145 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f8558001bc0 con 0x7f8560167080 2024-04-04T06:07:49.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.751+0000 7f855ca00640 1 -- 172.21.15.159:0/2358190145 >> v1:172.21.15.159:6789/0 conn(0x7f8560167080 legacy=0x7f85600b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:49.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.752+0000 7f855ca00640 1 --2- 172.21.15.159:0/2358190145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85440039b0 0x7f8544003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:49.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.752+0000 7f855ca00640 1 --2- 172.21.15.159:0/2358190145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8544004800 0x7f8544004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:49.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.752+0000 7f855ca00640 1 --2- 172.21.15.159:0/2358190145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f856015d120 0x7f85440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:49.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.752+0000 7f855ca00640 1 -- 172.21.15.159:0/2358190145 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f85600723c0 con 0x7f85440039b0 2024-04-04T06:07:49.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.752+0000 7f855ca00640 1 -- 172.21.15.159:0/2358190145 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8560163650 con 0x7f8544004800 2024-04-04T06:07:49.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.752+0000 7f855ca00640 1 -- 172.21.15.159:0/2358190145 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f854400b9c0 con 0x7f856015d120 2024-04-04T06:07:49.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.752+0000 7f855de00640 1 --2- 172.21.15.159:0/2358190145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85440039b0 0x7f8544003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:49.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.752+0000 7f855d400640 1 --2- 172.21.15.159:0/2358190145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8544004800 0x7f8544004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:49.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.752+0000 7f855de00640 1 -- 172.21.15.159:0/2358190145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f856015d120 msgr2=0x7f85440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:49.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.752+0000 7f855de00640 1 --2- 172.21.15.159:0/2358190145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f856015d120 0x7f85440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:49.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.752+0000 7f855de00640 1 -- 172.21.15.159:0/2358190145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8544004800 msgr2=0x7f8544004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:49.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.752+0000 7f855de00640 1 --2- 172.21.15.159:0/2358190145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8544004800 0x7f8544004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:49.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.752+0000 7f855de00640 1 -- 172.21.15.159:0/2358190145 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8544003520 con 0x7f85440039b0 2024-04-04T06:07:49.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.753+0000 7f855de00640 1 --2- 172.21.15.159:0/2358190145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85440039b0 0x7f8544003dd0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8550009ac0 tx=0x7f8544005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdeba080f9423ee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:49.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.754+0000 7f855ca00640 1 -- 172.21.15.159:0/2358190145 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550039070 con 0x7f85440039b0 2024-04-04T06:07:49.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.754+0000 7f855ca00640 1 -- 172.21.15.159:0/2358190145 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f855002c710 con 0x7f85440039b0 2024-04-04T06:07:49.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.754+0000 7f855ca00640 1 -- 172.21.15.159:0/2358190145 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550009690 con 0x7f85440039b0 2024-04-04T06:07:49.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.754+0000 7f8564c00640 1 -- 172.21.15.159:0/2358190145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85440039b0 msgr2=0x7f8544003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:49.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.754+0000 7f8564c00640 1 --2- 172.21.15.159:0/2358190145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85440039b0 0x7f8544003dd0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8550009ac0 tx=0x7f8544005830 comp rx=0 tx=0).stop 2024-04-04T06:07:49.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.754+0000 7f855e800640 1 -- 172.21.15.159:0/2358190145 reap_dead start 2024-04-04T06:07:49.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.754+0000 7f8564c00640 1 -- 172.21.15.159:0/2358190145 shutdown_connections 2024-04-04T06:07:49.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.754+0000 7f8564c00640 1 -- 172.21.15.159:0/2358190145 >> 172.21.15.159:0/2358190145 conn(0x7f856006b990 msgr2=0x7f856006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:49.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.754+0000 7f8564c00640 1 -- 172.21.15.159:0/2358190145 shutdown_connections 2024-04-04T06:07:49.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.754+0000 7f8564c00640 1 -- 172.21.15.159:0/2358190145 wait complete. 2024-04-04T06:07:49.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.756+0000 7f8564c00640 1 Processor -- start 2024-04-04T06:07:49.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.756+0000 7f8564c00640 1 -- start start 2024-04-04T06:07:49.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.757+0000 7f8564c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85440039b0 0x7f856015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:49.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.757+0000 7f8564c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f856009a2a0 0x7f85601f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:49.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.757+0000 7f8564c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85601f9330 0x7f856009c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:49.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.757+0000 7f8564c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f85600b3c10 con 0x7f85440039b0 2024-04-04T06:07:49.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.757+0000 7f8564c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8560166620 con 0x7f85601f9330 2024-04-04T06:07:49.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.757+0000 7f8564c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f85600b3d90 con 0x7f856009a2a0 2024-04-04T06:07:49.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.758+0000 7f855de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85440039b0 0x7f856015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:49.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.758+0000 7f855de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85440039b0 0x7f856015baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47308/0 (socket says 172.21.15.159:47308) 2024-04-04T06:07:49.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.758+0000 7f855de00640 1 -- 172.21.15.159:0/44323786 learned_addr learned my addr 172.21.15.159:0/44323786 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:49.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.758+0000 7f855de00640 1 -- 172.21.15.159:0/44323786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f856009a2a0 msgr2=0x7f85601f8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:49.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.758+0000 7f855de00640 1 --2- 172.21.15.159:0/44323786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f856009a2a0 0x7f85601f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:49.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.758+0000 7f855de00640 1 -- 172.21.15.159:0/44323786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85601f9330 msgr2=0x7f856009c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:49.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.758+0000 7f855de00640 1 --2- 172.21.15.159:0/44323786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85601f9330 0x7f856009c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:49.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.758+0000 7f855de00640 1 -- 172.21.15.159:0/44323786 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856009a7f0 con 0x7f85440039b0 2024-04-04T06:07:49.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.758+0000 7f855de00640 1 --2- 172.21.15.159:0/44323786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85440039b0 0x7f856015baa0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8550000c00 tx=0x7f8550004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:49.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.758+0000 7f8556000640 1 -- 172.21.15.159:0/44323786 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550039050 con 0x7f85440039b0 2024-04-04T06:07:49.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.759+0000 7f8556000640 1 -- 172.21.15.159:0/44323786 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8550045070 con 0x7f85440039b0 2024-04-04T06:07:49.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.759+0000 7f8556000640 1 -- 172.21.15.159:0/44323786 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550003a50 con 0x7f85440039b0 2024-04-04T06:07:49.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.759+0000 7f8564c00640 1 -- 172.21.15.159:0/44323786 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85600cf2c0 con 0x7f85440039b0 2024-04-04T06:07:49.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.759+0000 7f8564c00640 1 -- 172.21.15.159:0/44323786 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85600cf880 con 0x7f85440039b0 2024-04-04T06:07:49.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.760+0000 7f8564c00640 1 -- 172.21.15.159:0/44323786 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85600685c0 con 0x7f85440039b0 2024-04-04T06:07:49.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.761+0000 7f8556000640 1 -- 172.21.15.159:0/44323786 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8550003bf0 con 0x7f85440039b0 2024-04-04T06:07:49.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.762+0000 7f8556000640 1 --2- 172.21.15.159:0/44323786 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f853c06c8f0 0x7f853c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:49.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.762+0000 7f8556000640 1 -- 172.21.15.159:0/44323786 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f85500ad4c0 con 0x7f85440039b0 2024-04-04T06:07:49.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.763+0000 7f855d400640 1 --2- 172.21.15.159:0/44323786 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f853c06c8f0 0x7f853c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:49.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.764+0000 7f855d400640 1 --2- 172.21.15.159:0/44323786 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f853c06c8f0 0x7f853c06edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f85600cff50 tx=0x7f8548008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:49.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.767+0000 7f8556000640 1 -- 172.21.15.159:0/44323786 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f855004e020 con 0x7f85440039b0 2024-04-04T06:07:49.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.969+0000 7f8564c00640 1 -- 172.21.15.159:0/44323786 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f856006f3a0 con 0x7f85440039b0 2024-04-04T06:07:49.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.971+0000 7f8556000640 1 -- 172.21.15.159:0/44323786 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+12903 (secure 0 0 0) 0x7f855007bb10 con 0x7f85440039b0 2024-04-04T06:07:49.966 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:49.966 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":42,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:07:48.992872+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:07:49.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.974+0000 7f8564c00640 1 -- 172.21.15.159:0/44323786 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f853c06c8f0 msgr2=0x7f853c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:49.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.974+0000 7f8564c00640 1 --2- 172.21.15.159:0/44323786 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f853c06c8f0 0x7f853c06edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f85600cff50 tx=0x7f8548008040 comp rx=0 tx=0).stop 2024-04-04T06:07:49.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.974+0000 7f8564c00640 1 -- 172.21.15.159:0/44323786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85440039b0 msgr2=0x7f856015baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:49.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.974+0000 7f8564c00640 1 --2- 172.21.15.159:0/44323786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85440039b0 0x7f856015baa0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8550000c00 tx=0x7f8550004730 comp rx=0 tx=0).stop 2024-04-04T06:07:49.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.974+0000 7f8564c00640 1 -- 172.21.15.159:0/44323786 shutdown_connections 2024-04-04T06:07:49.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.974+0000 7f8564c00640 1 --2- 172.21.15.159:0/44323786 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f853c06c8f0 0x7f853c06edb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:49.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.974+0000 7f8564c00640 1 --2- 172.21.15.159:0/44323786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85601f9330 0x7f856009c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:49.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.974+0000 7f8564c00640 1 --2- 172.21.15.159:0/44323786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f856009a2a0 0x7f85601f8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:49.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.974+0000 7f8564c00640 1 --2- 172.21.15.159:0/44323786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85440039b0 0x7f856015baa0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:49.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.974+0000 7f8564c00640 1 -- 172.21.15.159:0/44323786 >> 172.21.15.159:0/44323786 conn(0x7f856006b990 msgr2=0x7f85600ae060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:49.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.975+0000 7f8564c00640 1 -- 172.21.15.159:0/44323786 shutdown_connections 2024-04-04T06:07:49.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:49.975+0000 7f8564c00640 1 -- 172.21.15.159:0/44323786 wait complete. 2024-04-04T06:07:49.986 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:07:49.987 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:07:50.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.209+0000 7f75ccc00640 1 Processor -- start 2024-04-04T06:07:50.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.209+0000 7f75ccc00640 1 -- start start 2024-04-04T06:07:50.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.211+0000 7f75ccc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c8155930 0x7f75c80d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:50.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.211+0000 7f75ccc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f75c80d07c0 0x7f75c80aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:50.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.211+0000 7f75ccc00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75c80b40b0 con 0x7f75c8155550 2024-04-04T06:07:50.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.211+0000 7f75ccc00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f75c80b4390 con 0x7f75c80d07c0 2024-04-04T06:07:50.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.211+0000 7f75ccc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f75c80b4670 con 0x7f75c8155930 2024-04-04T06:07:50.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.211+0000 7f75c6800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f75c80d07c0 0x7f75c80aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:50.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.211+0000 7f75c6800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f75c80d07c0 0x7f75c80aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47322/0 (socket says 172.21.15.159:47322) 2024-04-04T06:07:50.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.211+0000 7f75c6800640 1 -- 172.21.15.159:0/4072819059 learned_addr learned my addr 172.21.15.159:0/4072819059 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:50.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.212+0000 7f75bfe00640 1 --2- 172.21.15.159:0/4072819059 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c8155930 0x7f75c80d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:50.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.212+0000 7f75c6800640 1 -- 172.21.15.159:0/4072819059 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c8155930 msgr2=0x7f75c80d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:50.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.212+0000 7f75c6800640 1 --2- 172.21.15.159:0/4072819059 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c8155930 0x7f75c80d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:50.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.212+0000 7f75c6800640 1 -- 172.21.15.159:0/4072819059 >> v1:172.21.15.178:6789/0 conn(0x7f75c8155550 legacy=0x7f75c815f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:50.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.212+0000 7f75c6800640 1 -- 172.21.15.159:0/4072819059 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c80b4950 con 0x7f75c80d07c0 2024-04-04T06:07:50.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.213+0000 7f75c6800640 1 --2- 172.21.15.159:0/4072819059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75c80d07c0 0x7f75c80aede0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009e50 tx=0x7f75c002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3fe5b2e905ba91 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:50.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.213+0000 7f75c5400640 1 -- 172.21.15.159:0/4072819059 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c002cb00 con 0x7f75c80d07c0 2024-04-04T06:07:50.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.213+0000 7f75c5400640 1 -- 172.21.15.159:0/4072819059 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c002cca0 con 0x7f75c80d07c0 2024-04-04T06:07:50.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.213+0000 7f75c5400640 1 -- 172.21.15.159:0/4072819059 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0034610 con 0x7f75c80d07c0 2024-04-04T06:07:50.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.214+0000 7f75ccc00640 1 -- 172.21.15.159:0/4072819059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75c80d07c0 msgr2=0x7f75c80aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:50.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.214+0000 7f75ccc00640 1 --2- 172.21.15.159:0/4072819059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75c80d07c0 0x7f75c80aede0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009e50 tx=0x7f75c002bf60 comp rx=0 tx=0).stop 2024-04-04T06:07:50.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.214+0000 7f75ccc00640 1 -- 172.21.15.159:0/4072819059 shutdown_connections 2024-04-04T06:07:50.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.214+0000 7f75ccc00640 1 --2- 172.21.15.159:0/4072819059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75c80d07c0 0x7f75c80aede0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:50.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.214+0000 7f75ccc00640 1 --2- 172.21.15.159:0/4072819059 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c8155930 0x7f75c80d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:50.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.214+0000 7f75ccc00640 1 -- 172.21.15.159:0/4072819059 >> 172.21.15.159:0/4072819059 conn(0x7f75c806b750 msgr2=0x7f75c806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:50.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.214+0000 7f75ccc00640 1 -- 172.21.15.159:0/4072819059 shutdown_connections 2024-04-04T06:07:50.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.214+0000 7f75ccc00640 1 -- 172.21.15.159:0/4072819059 wait complete. 2024-04-04T06:07:50.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.216+0000 7f75ccc00640 1 Processor -- start 2024-04-04T06:07:50.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.216+0000 7f75ccc00640 1 -- start start 2024-04-04T06:07:50.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.217+0000 7f75ccc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c80d07c0 0x7f75c8072670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:50.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.217+0000 7f75ccc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75c8155550 0x7f75c809aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:50.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.217+0000 7f75ccc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75c8155930 0x7f75c809b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:50.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.217+0000 7f75ccc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f75c815bba0 con 0x7f75c8155930 2024-04-04T06:07:50.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.217+0000 7f75ccc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f75c815ba20 con 0x7f75c8155550 2024-04-04T06:07:50.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.217+0000 7f75ccc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f75c80b3cf0 con 0x7f75c80d07c0 2024-04-04T06:07:50.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.217+0000 7f75c6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75c8155930 0x7f75c809b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:50.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.217+0000 7f75c6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75c8155930 0x7f75c809b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47330/0 (socket says 172.21.15.159:47330) 2024-04-04T06:07:50.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.217+0000 7f75c6800640 1 -- 172.21.15.159:0/2112452208 learned_addr learned my addr 172.21.15.159:0/2112452208 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:50.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.217+0000 7f75c5e00640 1 --2- 172.21.15.159:0/2112452208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c80d07c0 0x7f75c8072670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:50.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.217+0000 7f75bfe00640 1 --2- 172.21.15.159:0/2112452208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75c8155550 0x7f75c809aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:50.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.218+0000 7f75c6800640 1 -- 172.21.15.159:0/2112452208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c80d07c0 msgr2=0x7f75c8072670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:50.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.218+0000 7f75c6800640 1 --2- 172.21.15.159:0/2112452208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c80d07c0 0x7f75c8072670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:50.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.218+0000 7f75c6800640 1 -- 172.21.15.159:0/2112452208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75c8155550 msgr2=0x7f75c809aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:50.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.218+0000 7f75c6800640 1 --2- 172.21.15.159:0/2112452208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75c8155550 0x7f75c809aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:50.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.218+0000 7f75c6800640 1 -- 172.21.15.159:0/2112452208 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c8099c00 con 0x7f75c8155930 2024-04-04T06:07:50.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.218+0000 7f75c6800640 1 --2- 172.21.15.159:0/2112452208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75c8155930 0x7f75c809b030 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009e50 tx=0x7f75c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:50.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.218+0000 7f75be000640 1 -- 172.21.15.159:0/2112452208 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0043020 con 0x7f75c8155930 2024-04-04T06:07:50.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.218+0000 7f75be000640 1 -- 172.21.15.159:0/2112452208 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c002ce50 con 0x7f75c8155930 2024-04-04T06:07:50.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.218+0000 7f75be000640 1 -- 172.21.15.159:0/2112452208 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0009510 con 0x7f75c8155930 2024-04-04T06:07:50.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.218+0000 7f75ccc00640 1 -- 172.21.15.159:0/2112452208 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c809d8a0 con 0x7f75c8155930 2024-04-04T06:07:50.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.218+0000 7f75ccc00640 1 -- 172.21.15.159:0/2112452208 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c809de40 con 0x7f75c8155930 2024-04-04T06:07:50.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.220+0000 7f75ccc00640 1 -- 172.21.15.159:0/2112452208 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75c80685c0 con 0x7f75c8155930 2024-04-04T06:07:50.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.221+0000 7f75be000640 1 -- 172.21.15.159:0/2112452208 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75c0046020 con 0x7f75c8155930 2024-04-04T06:07:50.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.221+0000 7f75be000640 1 --2- 172.21.15.159:0/2112452208 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f75a406c8f0 0x7f75a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:50.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.222+0000 7f75be000640 1 -- 172.21.15.159:0/2112452208 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f75c00adde0 con 0x7f75c8155930 2024-04-04T06:07:50.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.222+0000 7f75c5e00640 1 --2- 172.21.15.159:0/2112452208 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f75a406c8f0 0x7f75a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:50.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.223+0000 7f75c5e00640 1 --2- 172.21.15.159:0/2112452208 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f75a406c8f0 0x7f75a406edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f75b80015a0 tx=0x7f75b8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:50.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.226+0000 7f75be000640 1 -- 172.21.15.159:0/2112452208 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f75c004d030 con 0x7f75c8155930 2024-04-04T06:07:50.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.424+0000 7f75ccc00640 1 -- 172.21.15.159:0/2112452208 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f75c8144d70 con 0x7f75c8155930 2024-04-04T06:07:50.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.425+0000 7f75be000640 1 -- 172.21.15.159:0/2112452208 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) ==== 93+0+10 (secure 0 0 0) 0x7f75c007c5f0 con 0x7f75c8155930 2024-04-04T06:07:50.419 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:07:50.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.427+0000 7f75ccc00640 1 -- 172.21.15.159:0/2112452208 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f75a406c8f0 msgr2=0x7f75a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:50.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.427+0000 7f75ccc00640 1 --2- 172.21.15.159:0/2112452208 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f75a406c8f0 0x7f75a406edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f75b80015a0 tx=0x7f75b8009040 comp rx=0 tx=0).stop 2024-04-04T06:07:50.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.427+0000 7f75ccc00640 1 -- 172.21.15.159:0/2112452208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75c8155930 msgr2=0x7f75c809b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:50.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.427+0000 7f75ccc00640 1 --2- 172.21.15.159:0/2112452208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75c8155930 0x7f75c809b030 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009e50 tx=0x7f75c0004a40 comp rx=0 tx=0).stop 2024-04-04T06:07:50.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.427+0000 7f75ccc00640 1 -- 172.21.15.159:0/2112452208 shutdown_connections 2024-04-04T06:07:50.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.427+0000 7f75ccc00640 1 --2- 172.21.15.159:0/2112452208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75c8155930 0x7f75c809b030 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:50.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.427+0000 7f75ccc00640 1 --2- 172.21.15.159:0/2112452208 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f75a406c8f0 0x7f75a406edb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:50.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.427+0000 7f75ccc00640 1 --2- 172.21.15.159:0/2112452208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75c8155550 0x7f75c809aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:50.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.427+0000 7f75ccc00640 1 --2- 172.21.15.159:0/2112452208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f75c80d07c0 0x7f75c8072670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:50.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.427+0000 7f75ccc00640 1 -- 172.21.15.159:0/2112452208 >> 172.21.15.159:0/2112452208 conn(0x7f75c806b750 msgr2=0x7f75c815d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:50.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.428+0000 7f75ccc00640 1 -- 172.21.15.159:0/2112452208 shutdown_connections 2024-04-04T06:07:50.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.428+0000 7f75ccc00640 1 -- 172.21.15.159:0/2112452208 wait complete. 2024-04-04T06:07:50.441 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-04T06:07:50.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.661+0000 7f1015e00640 1 Processor -- start 2024-04-04T06:07:50.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.661+0000 7f1015e00640 1 -- start start 2024-04-04T06:07:50.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.663+0000 7f1015e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1010141c10 0x7f1010071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:50.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.663+0000 7f1015e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f101015ec80 con 0x7f10100d0280 2024-04-04T06:07:50.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.663+0000 7f1015e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f101015ef20 con 0x7f1010071eb0 2024-04-04T06:07:50.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.663+0000 7f1015e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f101015f200 con 0x7f1010141c10 2024-04-04T06:07:50.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.663+0000 7f100ea00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f10100d0280 0x7f10100d0660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:33338/0 (socket says 172.21.15.159:33338) 2024-04-04T06:07:50.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.663+0000 7f100ea00640 1 -- 172.21.15.159:0/2577592973 learned_addr learned my addr 172.21.15.159:0/2577592973 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:50.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.664+0000 7f1006000640 1 --2- 172.21.15.159:0/2577592973 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1010141c10 0x7f1010071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:50.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.664+0000 7f100e000640 1 -- 172.21.15.159:0/2577592973 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3295789170 0 0) 0x7f101015ec80 con 0x7f10100d0280 2024-04-04T06:07:50.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.664+0000 7f100e000640 1 -- 172.21.15.159:0/2577592973 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ff4003520 con 0x7f10100d0280 2024-04-04T06:07:50.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.664+0000 7f100e000640 1 -- 172.21.15.159:0/2577592973 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 406621582 0 0) 0x7f0ff4003520 con 0x7f10100d0280 2024-04-04T06:07:50.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.664+0000 7f100e000640 1 -- 172.21.15.159:0/2577592973 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1010141c10 msgr2=0x7f1010071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:50.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.664+0000 7f100e000640 1 --2- 172.21.15.159:0/2577592973 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1010141c10 0x7f1010071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:50.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.664+0000 7f100e000640 1 -- 172.21.15.159:0/2577592973 >> v1:172.21.15.178:6789/0 conn(0x7f1010071eb0 legacy=0x7f1010162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:50.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.664+0000 7f100e000640 1 -- 172.21.15.159:0/2577592973 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101015f4e0 con 0x7f10100d0280 2024-04-04T06:07:50.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.664+0000 7f100e000640 1 -- 172.21.15.159:0/2577592973 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f1000001670 con 0x7f10100d0280 2024-04-04T06:07:50.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.664+0000 7f100e000640 1 -- 172.21.15.159:0/2577592973 >> v1:172.21.15.159:6789/0 conn(0x7f10100d0280 legacy=0x7f10100d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:50.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.664+0000 7f100e000640 1 --2- 172.21.15.159:0/2577592973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff40039e0 0x7f0ff4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:50.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.664+0000 7f100e000640 1 --2- 172.21.15.159:0/2577592973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ff4004800 0x7f0ff4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:50.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.664+0000 7f100e000640 1 --2- 172.21.15.159:0/2577592973 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1010141c10 0x7f0ff40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:50.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.664+0000 7f100e000640 1 -- 172.21.15.159:0/2577592973 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f101015f200 con 0x7f0ff40039e0 2024-04-04T06:07:50.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.664+0000 7f100e000640 1 -- 172.21.15.159:0/2577592973 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff400ba70 con 0x7f0ff4004800 2024-04-04T06:07:50.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.664+0000 7f100e000640 1 -- 172.21.15.159:0/2577592973 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff400bc20 con 0x7f1010141c10 2024-04-04T06:07:50.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.665+0000 7f100ea00640 1 --2- 172.21.15.159:0/2577592973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff40039e0 0x7f0ff4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:50.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.665+0000 7f1006000640 1 --2- 172.21.15.159:0/2577592973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ff4004800 0x7f0ff4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:50.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.665+0000 7f100ea00640 1 -- 172.21.15.159:0/2577592973 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1010141c10 msgr2=0x7f0ff40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:50.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.665+0000 7f100ea00640 1 --2- 172.21.15.159:0/2577592973 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1010141c10 0x7f0ff40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:50.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.665+0000 7f100ea00640 1 -- 172.21.15.159:0/2577592973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ff4004800 msgr2=0x7f0ff4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:50.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.665+0000 7f100ea00640 1 --2- 172.21.15.159:0/2577592973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ff4004800 0x7f0ff4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:50.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.665+0000 7f100ea00640 1 -- 172.21.15.159:0/2577592973 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff4003520 con 0x7f0ff40039e0 2024-04-04T06:07:50.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.665+0000 7f1006000640 1 --2- 172.21.15.159:0/2577592973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ff4004800 0x7f0ff4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:50.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.666+0000 7f100ea00640 1 --2- 172.21.15.159:0/2577592973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff40039e0 0x7f0ff4003dc0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f100000c160 tx=0x7f0ff4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758a6eff50d2952b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:50.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.666+0000 7f100e000640 1 -- 172.21.15.159:0/2577592973 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000007c60 con 0x7f0ff40039e0 2024-04-04T06:07:50.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.666+0000 7f100e000640 1 -- 172.21.15.159:0/2577592973 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1000030070 con 0x7f0ff40039e0 2024-04-04T06:07:50.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.666+0000 7f100e000640 1 -- 172.21.15.159:0/2577592973 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000005d10 con 0x7f0ff40039e0 2024-04-04T06:07:50.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.667+0000 7f1015e00640 1 -- 172.21.15.159:0/2577592973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff40039e0 msgr2=0x7f0ff4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:50.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.667+0000 7f1015e00640 1 --2- 172.21.15.159:0/2577592973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff40039e0 0x7f0ff4003dc0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f100000c160 tx=0x7f0ff4005830 comp rx=0 tx=0).stop 2024-04-04T06:07:50.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.667+0000 7f100f400640 1 -- 172.21.15.159:0/2577592973 reap_dead start 2024-04-04T06:07:50.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.667+0000 7f1015e00640 1 -- 172.21.15.159:0/2577592973 shutdown_connections 2024-04-04T06:07:50.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.667+0000 7f1015e00640 1 -- 172.21.15.159:0/2577592973 >> 172.21.15.159:0/2577592973 conn(0x7f101006b750 msgr2=0x7f101006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:50.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.667+0000 7f1015e00640 1 -- 172.21.15.159:0/2577592973 shutdown_connections 2024-04-04T06:07:50.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.667+0000 7f1015e00640 1 -- 172.21.15.159:0/2577592973 wait complete. 2024-04-04T06:07:50.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.669+0000 7f1015e00640 1 Processor -- start 2024-04-04T06:07:50.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.669+0000 7f1015e00640 1 -- start start 2024-04-04T06:07:50.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.670+0000 7f1015e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ff40039e0 0x7f101009ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:50.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.670+0000 7f1015e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1010157f00 0x7f101009b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:50.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.670+0000 7f1015e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f101009ba20 0x7f10101f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:50.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.670+0000 7f1015e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1010158e90 con 0x7f101009ba20 2024-04-04T06:07:50.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.670+0000 7f1015e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f101006f340 con 0x7f0ff40039e0 2024-04-04T06:07:50.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.670+0000 7f1015e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f101009be60 con 0x7f1010157f00 2024-04-04T06:07:50.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.671+0000 7f100f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f101009ba20 0x7f10101f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:50.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.671+0000 7f100f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f101009ba20 0x7f10101f3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47358/0 (socket says 172.21.15.159:47358) 2024-04-04T06:07:50.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.671+0000 7f100f400640 1 -- 172.21.15.159:0/3879377685 learned_addr learned my addr 172.21.15.159:0/3879377685 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:50.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.671+0000 7f100ea00640 1 --2- 172.21.15.159:0/3879377685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ff40039e0 0x7f101009ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:50.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.671+0000 7f1006000640 1 --2- 172.21.15.159:0/3879377685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1010157f00 0x7f101009b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:50.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.671+0000 7f100f400640 1 -- 172.21.15.159:0/3879377685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1010157f00 msgr2=0x7f101009b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:50.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.671+0000 7f100f400640 1 --2- 172.21.15.159:0/3879377685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1010157f00 0x7f101009b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:50.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.671+0000 7f100f400640 1 -- 172.21.15.159:0/3879377685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ff40039e0 msgr2=0x7f101009ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:50.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.671+0000 7f100f400640 1 --2- 172.21.15.159:0/3879377685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ff40039e0 0x7f101009ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:50.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.671+0000 7f100f400640 1 -- 172.21.15.159:0/3879377685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10101f4c20 con 0x7f101009ba20 2024-04-04T06:07:50.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.671+0000 7f1006000640 1 --2- 172.21.15.159:0/3879377685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1010157f00 0x7f101009b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:50.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.671+0000 7f100f400640 1 --2- 172.21.15.159:0/3879377685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f101009ba20 0x7f10101f3f80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f10100b5270 tx=0x7f100800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:50.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.672+0000 7f1007e00640 1 -- 172.21.15.159:0/3879377685 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1008017070 con 0x7f101009ba20 2024-04-04T06:07:50.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.672+0000 7f1007e00640 1 -- 172.21.15.159:0/3879377685 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1008005d40 con 0x7f101009ba20 2024-04-04T06:07:50.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.672+0000 7f1007e00640 1 -- 172.21.15.159:0/3879377685 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100800a850 con 0x7f101009ba20 2024-04-04T06:07:50.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.672+0000 7f1015e00640 1 -- 172.21.15.159:0/3879377685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10101f4f00 con 0x7f101009ba20 2024-04-04T06:07:50.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.672+0000 7f1015e00640 1 -- 172.21.15.159:0/3879377685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f101009a510 con 0x7f101009ba20 2024-04-04T06:07:50.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.673+0000 7f1015e00640 1 -- 172.21.15.159:0/3879377685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10100685c0 con 0x7f101009ba20 2024-04-04T06:07:50.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.674+0000 7f1007e00640 1 -- 172.21.15.159:0/3879377685 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f100800a9f0 con 0x7f101009ba20 2024-04-04T06:07:50.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.675+0000 7f1007e00640 1 --2- 172.21.15.159:0/3879377685 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0fec06c8f0 0x7f0fec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:50.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.675+0000 7f1007e00640 1 -- 172.21.15.159:0/3879377685 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f100808ab70 con 0x7f101009ba20 2024-04-04T06:07:50.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.675+0000 7f100ea00640 1 --2- 172.21.15.159:0/3879377685 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0fec06c8f0 0x7f0fec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:50.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.676+0000 7f100ea00640 1 --2- 172.21.15.159:0/3879377685 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0fec06c8f0 0x7f0fec06edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1000034000 tx=0x7f1000009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:50.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.680+0000 7f1007e00640 1 -- 172.21.15.159:0/3879377685 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1008059250 con 0x7f101009ba20 2024-04-04T06:07:50.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.883+0000 7f1015e00640 1 -- 172.21.15.159:0/3879377685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f10100cfdb0 con 0x7f101009ba20 2024-04-04T06:07:50.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.884+0000 7f1007e00640 1 -- 172.21.15.159:0/3879377685 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) ==== 84+0+65 (secure 0 0 0) 0x7f1008058b50 con 0x7f101009ba20 2024-04-04T06:07:50.878 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:50.878 INFO:teuthology.orchestra.run.smithi159.stdout:[ 2024-04-04T06:07:50.878 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:07:50.878 INFO:teuthology.orchestra.run.smithi159.stdout: "poolnum": 1, 2024-04-04T06:07:50.878 INFO:teuthology.orchestra.run.smithi159.stdout: "poolname": ".mgr" 2024-04-04T06:07:50.878 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:07:50.878 INFO:teuthology.orchestra.run.smithi159.stdout:] 2024-04-04T06:07:50.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.886+0000 7f1015e00640 1 -- 172.21.15.159:0/3879377685 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0fec06c8f0 msgr2=0x7f0fec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:50.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.886+0000 7f1015e00640 1 --2- 172.21.15.159:0/3879377685 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0fec06c8f0 0x7f0fec06edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1000034000 tx=0x7f1000009030 comp rx=0 tx=0).stop 2024-04-04T06:07:50.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.886+0000 7f1015e00640 1 -- 172.21.15.159:0/3879377685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f101009ba20 msgr2=0x7f10101f3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:50.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.886+0000 7f1015e00640 1 --2- 172.21.15.159:0/3879377685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f101009ba20 0x7f10101f3f80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f10100b5270 tx=0x7f100800cb90 comp rx=0 tx=0).stop 2024-04-04T06:07:50.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.886+0000 7f1015e00640 1 -- 172.21.15.159:0/3879377685 shutdown_connections 2024-04-04T06:07:50.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.886+0000 7f1015e00640 1 --2- 172.21.15.159:0/3879377685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f101009ba20 0x7f10101f3f80 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:50.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.886+0000 7f1015e00640 1 --2- 172.21.15.159:0/3879377685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1010157f00 0x7f101009b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:50.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.886+0000 7f1015e00640 1 --2- 172.21.15.159:0/3879377685 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0fec06c8f0 0x7f0fec06edb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:50.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.886+0000 7f1015e00640 1 --2- 172.21.15.159:0/3879377685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ff40039e0 0x7f101009ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:50.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.886+0000 7f1015e00640 1 -- 172.21.15.159:0/3879377685 >> 172.21.15.159:0/3879377685 conn(0x7f101006b750 msgr2=0x7f101006d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:50.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.886+0000 7f1015e00640 1 -- 172.21.15.159:0/3879377685 shutdown_connections 2024-04-04T06:07:50.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:50.886+0000 7f1015e00640 1 -- 172.21.15.159:0/3879377685 wait complete. 2024-04-04T06:07:50.900 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-04T06:07:50.900 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-04T06:07:51.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.120+0000 7f34b4000640 1 Processor -- start 2024-04-04T06:07:51.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.120+0000 7f34b4000640 1 -- start start 2024-04-04T06:07:51.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.122+0000 7f34b4000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f34ac06e310 0x7f34ac0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:51.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.122+0000 7f34b4000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34ac0d00b0 0x7f34ac0d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:51.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.122+0000 7f34b4000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f34ac0d11d0 0x7f34ac1610d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:51.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.122+0000 7f34b4000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f34ac0d53c0 con 0x7f34ac0d11d0 2024-04-04T06:07:51.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.122+0000 7f34b4000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f34ac0d21c0 con 0x7f34ac06e310 2024-04-04T06:07:51.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.122+0000 7f34b4000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f34ac0d24a0 con 0x7f34ac0d00b0 2024-04-04T06:07:51.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.122+0000 7f34b2000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f34ac0d11d0 0x7f34ac1610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:51.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.123+0000 7f34b2000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f34ac0d11d0 0x7f34ac1610d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47374/0 (socket says 172.21.15.159:47374) 2024-04-04T06:07:51.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.123+0000 7f34b2000640 1 -- 172.21.15.159:0/3628120518 learned_addr learned my addr 172.21.15.159:0/3628120518 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:51.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.123+0000 7f34b1600640 1 --2- 172.21.15.159:0/3628120518 >> v2:172.21.15.178:3300/0 conn(0x7f34ac06e310 0x7f34ac0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:51.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.123+0000 7f34b0c00640 1 --2- 172.21.15.159:0/3628120518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34ac0d00b0 0x7f34ac0d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:51.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.123+0000 7f34b2000640 1 -- 172.21.15.159:0/3628120518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34ac0d00b0 msgr2=0x7f34ac0d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:51.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.123+0000 7f34b2000640 1 --2- 172.21.15.159:0/3628120518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34ac0d00b0 0x7f34ac0d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:51.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.123+0000 7f34b2000640 1 -- 172.21.15.159:0/3628120518 >> v2:172.21.15.178:3300/0 conn(0x7f34ac06e310 msgr2=0x7f34ac0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:51.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.123+0000 7f34b2000640 1 --2- 172.21.15.159:0/3628120518 >> v2:172.21.15.178:3300/0 conn(0x7f34ac06e310 0x7f34ac0cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:51.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.123+0000 7f34b2000640 1 -- 172.21.15.159:0/3628120518 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ac0d2780 con 0x7f34ac0d11d0 2024-04-04T06:07:51.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.123+0000 7f34b0c00640 1 --2- 172.21.15.159:0/3628120518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34ac0d00b0 0x7f34ac0d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:51.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.124+0000 7f34b2000640 1 --2- 172.21.15.159:0/3628120518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34ac0d11d0 0x7f34ac1610d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f34ac08e650 tx=0x7f34a402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef4068251e117d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:51.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.124+0000 7f34abe00640 1 -- 172.21.15.159:0/3628120518 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a402ca70 con 0x7f34ac0d11d0 2024-04-04T06:07:51.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.124+0000 7f34abe00640 1 -- 172.21.15.159:0/3628120518 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a402cc10 con 0x7f34ac0d11d0 2024-04-04T06:07:51.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.124+0000 7f34abe00640 1 -- 172.21.15.159:0/3628120518 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a40343f0 con 0x7f34ac0d11d0 2024-04-04T06:07:51.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.125+0000 7f34b4000640 1 -- 172.21.15.159:0/3628120518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34ac0d11d0 msgr2=0x7f34ac1610d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:51.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.125+0000 7f34b4000640 1 --2- 172.21.15.159:0/3628120518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34ac0d11d0 0x7f34ac1610d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f34ac08e650 tx=0x7f34a402bed0 comp rx=0 tx=0).stop 2024-04-04T06:07:51.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.125+0000 7f34b4000640 1 -- 172.21.15.159:0/3628120518 shutdown_connections 2024-04-04T06:07:51.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.125+0000 7f34b4000640 1 --2- 172.21.15.159:0/3628120518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34ac0d11d0 0x7f34ac1610d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:51.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.125+0000 7f34b4000640 1 --2- 172.21.15.159:0/3628120518 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34ac0d00b0 0x7f34ac0d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:51.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.125+0000 7f34b4000640 1 --2- 172.21.15.159:0/3628120518 >> v2:172.21.15.178:3300/0 conn(0x7f34ac06e310 0x7f34ac0cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:51.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.125+0000 7f34b4000640 1 -- 172.21.15.159:0/3628120518 >> 172.21.15.159:0/3628120518 conn(0x7f34ac08bfb0 msgr2=0x7f34ac0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:51.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.125+0000 7f34b4000640 1 -- 172.21.15.159:0/3628120518 shutdown_connections 2024-04-04T06:07:51.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.125+0000 7f34b4000640 1 -- 172.21.15.159:0/3628120518 wait complete. 2024-04-04T06:07:51.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.127+0000 7f34b4000640 1 Processor -- start 2024-04-04T06:07:51.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.127+0000 7f34b4000640 1 -- start start 2024-04-04T06:07:51.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.128+0000 7f34b4000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34ac0d00b0 0x7f34ac1e6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:51.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.128+0000 7f34b4000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34ac0d11d0 0x7f34ac1e2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:51.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.128+0000 7f34b4000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34ac1f78c0 0x7f34ac1f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:51.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.128+0000 7f34b4000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac0d3260 con 0x7f34ac0d00b0 2024-04-04T06:07:51.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.128+0000 7f34b4000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac0d2d40 con 0x7f34ac1f78c0 2024-04-04T06:07:51.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.128+0000 7f34b4000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f34ac1f8380 con 0x7f34ac0d11d0 2024-04-04T06:07:51.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.128+0000 7f34b1600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34ac0d00b0 0x7f34ac1e6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:51.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.129+0000 7f34b1600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34ac0d00b0 0x7f34ac1e6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47382/0 (socket says 172.21.15.159:47382) 2024-04-04T06:07:51.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.129+0000 7f34b1600640 1 -- 172.21.15.159:0/2979260640 learned_addr learned my addr 172.21.15.159:0/2979260640 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:51.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.129+0000 7f34b2000640 1 --2- 172.21.15.159:0/2979260640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34ac1f78c0 0x7f34ac1f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:51.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.129+0000 7f34b1600640 1 -- 172.21.15.159:0/2979260640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34ac0d11d0 msgr2=0x7f34ac1e2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:51.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.129+0000 7f34b1600640 1 --2- 172.21.15.159:0/2979260640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34ac0d11d0 0x7f34ac1e2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:51.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.129+0000 7f34b1600640 1 -- 172.21.15.159:0/2979260640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34ac1f78c0 msgr2=0x7f34ac1f7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:51.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.129+0000 7f34b1600640 1 --2- 172.21.15.159:0/2979260640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34ac1f78c0 0x7f34ac1f7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:51.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.129+0000 7f34b1600640 1 -- 172.21.15.159:0/2979260640 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ac1f8660 con 0x7f34ac0d00b0 2024-04-04T06:07:51.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.129+0000 7f34b2000640 1 --2- 172.21.15.159:0/2979260640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34ac1f78c0 0x7f34ac1f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:51.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.129+0000 7f34b1600640 1 --2- 172.21.15.159:0/2979260640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34ac0d00b0 0x7f34ac1e6e30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f34a000d5f0 tx=0x7f34a000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:51.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.130+0000 7f34a9600640 1 -- 172.21.15.159:0/2979260640 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a00125f0 con 0x7f34ac0d00b0 2024-04-04T06:07:51.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.130+0000 7f34a9600640 1 -- 172.21.15.159:0/2979260640 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a0019070 con 0x7f34ac0d00b0 2024-04-04T06:07:51.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.130+0000 7f34a9600640 1 -- 172.21.15.159:0/2979260640 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a00129d0 con 0x7f34ac0d00b0 2024-04-04T06:07:51.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.130+0000 7f34b4000640 1 -- 172.21.15.159:0/2979260640 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34ac1e6630 con 0x7f34ac0d00b0 2024-04-04T06:07:51.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.130+0000 7f34b4000640 1 -- 172.21.15.159:0/2979260640 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34ac1e7370 con 0x7f34ac0d00b0 2024-04-04T06:07:51.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.131+0000 7f34b4000640 1 -- 172.21.15.159:0/2979260640 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34ac1e6420 con 0x7f34ac0d00b0 2024-04-04T06:07:51.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.132+0000 7f34a9600640 1 -- 172.21.15.159:0/2979260640 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34a0012b70 con 0x7f34ac0d00b0 2024-04-04T06:07:51.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.133+0000 7f34a9600640 1 --2- 172.21.15.159:0/2979260640 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f348c06c8f0 0x7f348c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:51.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.133+0000 7f34a9600640 1 -- 172.21.15.159:0/2979260640 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f34a008bcf0 con 0x7f34ac0d00b0 2024-04-04T06:07:51.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.133+0000 7f34b0c00640 1 --2- 172.21.15.159:0/2979260640 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f348c06c8f0 0x7f348c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:51.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.134+0000 7f34b0c00640 1 --2- 172.21.15.159:0/2979260640 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f348c06c8f0 0x7f348c06edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3498006b10 tx=0x7f3498008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:51.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.137+0000 7f34a9600640 1 -- 172.21.15.159:0/2979260640 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f34a005a3d0 con 0x7f34ac0d00b0 2024-04-04T06:07:51.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:51.340+0000 7f34b4000640 1 -- 172.21.15.159:0/2979260640 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f34ac0685c0 con 0x7f34ac0d00b0 2024-04-04T06:07:52.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.023+0000 7f34a9600640 1 -- 172.21.15.159:0/2979260640 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v43) ==== 138+0+0 (secure 0 0 0) 0x7f34a0059cd0 con 0x7f34ac0d00b0 2024-04-04T06:07:52.018 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_metadata' created 2024-04-04T06:07:52.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.029+0000 7f34b4000640 1 -- 172.21.15.159:0/2979260640 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f348c06c8f0 msgr2=0x7f348c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:52.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.029+0000 7f34b4000640 1 --2- 172.21.15.159:0/2979260640 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f348c06c8f0 0x7f348c06edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3498006b10 tx=0x7f3498008040 comp rx=0 tx=0).stop 2024-04-04T06:07:52.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.029+0000 7f34b4000640 1 -- 172.21.15.159:0/2979260640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34ac0d00b0 msgr2=0x7f34ac1e6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:52.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.029+0000 7f34b4000640 1 --2- 172.21.15.159:0/2979260640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34ac0d00b0 0x7f34ac1e6e30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f34a000d5f0 tx=0x7f34a000dab0 comp rx=0 tx=0).stop 2024-04-04T06:07:52.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.030+0000 7f34b4000640 1 -- 172.21.15.159:0/2979260640 shutdown_connections 2024-04-04T06:07:52.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.030+0000 7f34b4000640 1 --2- 172.21.15.159:0/2979260640 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f348c06c8f0 0x7f348c06edb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:52.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.030+0000 7f34b4000640 1 --2- 172.21.15.159:0/2979260640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34ac1f78c0 0x7f34ac1f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:52.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.030+0000 7f34b4000640 1 --2- 172.21.15.159:0/2979260640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34ac0d11d0 0x7f34ac1e2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:52.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.030+0000 7f34b4000640 1 --2- 172.21.15.159:0/2979260640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34ac0d00b0 0x7f34ac1e6e30 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:52.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.030+0000 7f34b4000640 1 -- 172.21.15.159:0/2979260640 >> 172.21.15.159:0/2979260640 conn(0x7f34ac08bfb0 msgr2=0x7f34ac0b7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:52.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.030+0000 7f34b4000640 1 -- 172.21.15.159:0/2979260640 shutdown_connections 2024-04-04T06:07:52.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.030+0000 7f34b4000640 1 -- 172.21.15.159:0/2979260640 wait complete. 2024-04-04T06:07:52.052 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-04T06:07:52.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.263+0000 7fda3be00640 1 Processor -- start 2024-04-04T06:07:52.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.264+0000 7fda3be00640 1 -- start start 2024-04-04T06:07:52.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.265+0000 7fda3be00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fda34153580 0x7fda340d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:52.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.265+0000 7fda3be00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda340afab0 con 0x7fda340d07c0 2024-04-04T06:07:52.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.265+0000 7fda3be00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda3415cc80 con 0x7fda340af4b0 2024-04-04T06:07:52.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.265+0000 7fda3be00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fda340d0c30 con 0x7fda34153580 2024-04-04T06:07:52.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.265+0000 7fda38a00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fda340d07c0 0x7fda340aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:33342/0 (socket says 172.21.15.159:33342) 2024-04-04T06:07:52.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.265+0000 7fda38a00640 1 -- 172.21.15.159:0/1257773061 learned_addr learned my addr 172.21.15.159:0/1257773061 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:52.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.266+0000 7fda39400640 1 --2- 172.21.15.159:0/1257773061 >> v2:172.21.15.178:3300/0 conn(0x7fda34153580 0x7fda340d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:52.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.266+0000 7fda33e00640 1 -- 172.21.15.159:0/1257773061 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 464472833 0 0) 0x7fda340afab0 con 0x7fda340d07c0 2024-04-04T06:07:52.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.266+0000 7fda33e00640 1 -- 172.21.15.159:0/1257773061 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda18003520 con 0x7fda340d07c0 2024-04-04T06:07:52.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.267+0000 7fda33e00640 1 -- 172.21.15.159:0/1257773061 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 859349365 0 0) 0x7fda18003520 con 0x7fda340d07c0 2024-04-04T06:07:52.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.267+0000 7fda33e00640 1 -- 172.21.15.159:0/1257773061 >> v2:172.21.15.178:3300/0 conn(0x7fda34153580 msgr2=0x7fda340d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:52.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.267+0000 7fda33e00640 1 --2- 172.21.15.159:0/1257773061 >> v2:172.21.15.178:3300/0 conn(0x7fda34153580 0x7fda340d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:52.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.267+0000 7fda33e00640 1 -- 172.21.15.159:0/1257773061 >> v1:172.21.15.178:6789/0 conn(0x7fda340af4b0 legacy=0x7fda341600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:52.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.267+0000 7fda33e00640 1 -- 172.21.15.159:0/1257773061 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3415da00 con 0x7fda340d07c0 2024-04-04T06:07:52.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.267+0000 7fda33e00640 1 -- 172.21.15.159:0/1257773061 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fda20001670 con 0x7fda340d07c0 2024-04-04T06:07:52.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.267+0000 7fda33e00640 1 -- 172.21.15.159:0/1257773061 >> v1:172.21.15.159:6789/0 conn(0x7fda340d07c0 legacy=0x7fda340aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:52.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.267+0000 7fda33e00640 1 --2- 172.21.15.159:0/1257773061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda180039b0 0x7fda18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:52.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.267+0000 7fda33e00640 1 --2- 172.21.15.159:0/1257773061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda18004800 0x7fda18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:52.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.267+0000 7fda33e00640 1 --2- 172.21.15.159:0/1257773061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda180051c0 0x7fda1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:52.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.267+0000 7fda33e00640 1 -- 172.21.15.159:0/1257773061 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fda340d0c30 con 0x7fda18004800 2024-04-04T06:07:52.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.267+0000 7fda33e00640 1 -- 172.21.15.159:0/1257773061 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fda1800c160 con 0x7fda180051c0 2024-04-04T06:07:52.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.267+0000 7fda33e00640 1 -- 172.21.15.159:0/1257773061 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fda1800c360 con 0x7fda180039b0 2024-04-04T06:07:52.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.267+0000 7fda38a00640 1 --2- 172.21.15.159:0/1257773061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda18004800 0x7fda18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:52.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.267+0000 7fda39400640 1 --2- 172.21.15.159:0/1257773061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda180039b0 0x7fda18003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:52.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.268+0000 7fda38a00640 1 -- 172.21.15.159:0/1257773061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda180039b0 msgr2=0x7fda18003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:52.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.267+0000 7fda39e00640 1 --2- 172.21.15.159:0/1257773061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda180051c0 0x7fda1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:52.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.268+0000 7fda38a00640 1 --2- 172.21.15.159:0/1257773061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda180039b0 0x7fda18003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:52.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.268+0000 7fda38a00640 1 -- 172.21.15.159:0/1257773061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda180051c0 msgr2=0x7fda1800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:52.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.268+0000 7fda38a00640 1 --2- 172.21.15.159:0/1257773061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda180051c0 0x7fda1800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:52.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.268+0000 7fda38a00640 1 -- 172.21.15.159:0/1257773061 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda18003520 con 0x7fda18004800 2024-04-04T06:07:52.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.268+0000 7fda39e00640 1 --2- 172.21.15.159:0/1257773061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda180051c0 0x7fda1800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:52.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.268+0000 7fda39e00640 1 -- 172.21.15.159:0/1257773061 reap_dead start 2024-04-04T06:07:52.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.269+0000 7fda38a00640 1 --2- 172.21.15.159:0/1257773061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda18004800 0x7fda18004c80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fda2000c240 tx=0x7fda18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe805f36c1d88217 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:52.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.269+0000 7fda33e00640 1 -- 172.21.15.159:0/1257773061 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda20007d10 con 0x7fda18004800 2024-04-04T06:07:52.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.269+0000 7fda33e00640 1 -- 172.21.15.159:0/1257773061 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda20030070 con 0x7fda18004800 2024-04-04T06:07:52.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.269+0000 7fda33e00640 1 -- 172.21.15.159:0/1257773061 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2002cdc0 con 0x7fda18004800 2024-04-04T06:07:52.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.269+0000 7fda3be00640 1 -- 172.21.15.159:0/1257773061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda18004800 msgr2=0x7fda18004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:52.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.269+0000 7fda3be00640 1 --2- 172.21.15.159:0/1257773061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda18004800 0x7fda18004c80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fda2000c240 tx=0x7fda18005830 comp rx=0 tx=0).stop 2024-04-04T06:07:52.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.270+0000 7fda3be00640 1 -- 172.21.15.159:0/1257773061 shutdown_connections 2024-04-04T06:07:52.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.270+0000 7fda3be00640 1 --2- 172.21.15.159:0/1257773061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda18004800 0x7fda18004c80 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:52.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.270+0000 7fda3be00640 1 -- 172.21.15.159:0/1257773061 >> 172.21.15.159:0/1257773061 conn(0x7fda3406b750 msgr2=0x7fda3406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:52.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.270+0000 7fda3be00640 1 -- 172.21.15.159:0/1257773061 shutdown_connections 2024-04-04T06:07:52.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.270+0000 7fda3be00640 1 -- 172.21.15.159:0/1257773061 wait complete. 2024-04-04T06:07:52.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.271+0000 7fda3be00640 1 Processor -- start 2024-04-04T06:07:52.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.272+0000 7fda3be00640 1 -- start start 2024-04-04T06:07:52.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.272+0000 7fda3be00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda18004800 0x7fda3409bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:52.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.272+0000 7fda3be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda3409a190 0x7fda34155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:52.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.272+0000 7fda3be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda341546d0 0x7fda34154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:52.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.272+0000 7fda3be00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fda3415d4e0 con 0x7fda3409a190 2024-04-04T06:07:52.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.272+0000 7fda3be00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fda3415d200 con 0x7fda341546d0 2024-04-04T06:07:52.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.272+0000 7fda3be00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fda34153960 con 0x7fda18004800 2024-04-04T06:07:52.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.273+0000 7fda38a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda3409a190 0x7fda34155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:52.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.273+0000 7fda38a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda3409a190 0x7fda34155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47402/0 (socket says 172.21.15.159:47402) 2024-04-04T06:07:52.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.273+0000 7fda38a00640 1 -- 172.21.15.159:0/582069296 learned_addr learned my addr 172.21.15.159:0/582069296 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:52.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.273+0000 7fda39400640 1 --2- 172.21.15.159:0/582069296 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda18004800 0x7fda3409bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:52.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.273+0000 7fda38a00640 1 -- 172.21.15.159:0/582069296 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda18004800 msgr2=0x7fda3409bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:52.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.273+0000 7fda38a00640 1 --2- 172.21.15.159:0/582069296 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda18004800 0x7fda3409bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:52.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.273+0000 7fda38a00640 1 -- 172.21.15.159:0/582069296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda341546d0 msgr2=0x7fda34154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:52.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.273+0000 7fda39e00640 1 --2- 172.21.15.159:0/582069296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda341546d0 0x7fda34154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:52.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.273+0000 7fda38a00640 1 --2- 172.21.15.159:0/582069296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda341546d0 0x7fda34154b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:52.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.273+0000 7fda38a00640 1 -- 172.21.15.159:0/582069296 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3409a610 con 0x7fda3409a190 2024-04-04T06:07:52.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.274+0000 7fda38a00640 1 --2- 172.21.15.159:0/582069296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda3409a190 0x7fda34155bc0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fda20001970 tx=0x7fda340b46b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:52.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.274+0000 7fda31600640 1 -- 172.21.15.159:0/582069296 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda20045070 con 0x7fda3409a190 2024-04-04T06:07:52.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.274+0000 7fda31600640 1 -- 172.21.15.159:0/582069296 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda200049f0 con 0x7fda3409a190 2024-04-04T06:07:52.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.274+0000 7fda3be00640 1 -- 172.21.15.159:0/582069296 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda34156100 con 0x7fda3409a190 2024-04-04T06:07:52.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.274+0000 7fda31600640 1 -- 172.21.15.159:0/582069296 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda20007d10 con 0x7fda3409a190 2024-04-04T06:07:52.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.274+0000 7fda3be00640 1 -- 172.21.15.159:0/582069296 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda34145940 con 0x7fda3409a190 2024-04-04T06:07:52.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.275+0000 7fda3be00640 1 -- 172.21.15.159:0/582069296 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda340685c0 con 0x7fda3409a190 2024-04-04T06:07:52.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.276+0000 7fda31600640 1 -- 172.21.15.159:0/582069296 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda20007070 con 0x7fda3409a190 2024-04-04T06:07:52.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.277+0000 7fda31600640 1 --2- 172.21.15.159:0/582069296 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fda1406c8f0 0x7fda1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:52.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.277+0000 7fda31600640 1 -- 172.21.15.159:0/582069296 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5227+0+0 (secure 0 0 0) 0x7fda200ac8a0 con 0x7fda3409a190 2024-04-04T06:07:52.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.278+0000 7fda39400640 1 --2- 172.21.15.159:0/582069296 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fda1406c8f0 0x7fda1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:52.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.279+0000 7fda39400640 1 --2- 172.21.15.159:0/582069296 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fda1406c8f0 0x7fda1406edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fda340af0a0 tx=0x7fda24008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:52.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.282+0000 7fda31600640 1 -- 172.21.15.159:0/582069296 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fda2007aea0 con 0x7fda3409a190 2024-04-04T06:07:52.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:52.482+0000 7fda3be00640 1 -- 172.21.15.159:0/582069296 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fda3406bc30 con 0x7fda3409a190 2024-04-04T06:07:53.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.029+0000 7fda31600640 1 -- 172.21.15.159:0/582069296 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v44) ==== 170+0+0 (secure 0 0 0) 0x7fda2007a7a0 con 0x7fda3409a190 2024-04-04T06:07:53.024 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_data' created 2024-04-04T06:07:53.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.033+0000 7fda3be00640 1 -- 172.21.15.159:0/582069296 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fda1406c8f0 msgr2=0x7fda1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:53.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.033+0000 7fda3be00640 1 --2- 172.21.15.159:0/582069296 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fda1406c8f0 0x7fda1406edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fda340af0a0 tx=0x7fda24008040 comp rx=0 tx=0).stop 2024-04-04T06:07:53.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.033+0000 7fda3be00640 1 -- 172.21.15.159:0/582069296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda3409a190 msgr2=0x7fda34155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:53.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.033+0000 7fda3be00640 1 --2- 172.21.15.159:0/582069296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda3409a190 0x7fda34155bc0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fda20001970 tx=0x7fda340b46b0 comp rx=0 tx=0).stop 2024-04-04T06:07:53.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.033+0000 7fda3be00640 1 -- 172.21.15.159:0/582069296 shutdown_connections 2024-04-04T06:07:53.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.033+0000 7fda3be00640 1 --2- 172.21.15.159:0/582069296 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fda1406c8f0 0x7fda1406edb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:53.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.033+0000 7fda3be00640 1 --2- 172.21.15.159:0/582069296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda341546d0 0x7fda34154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:53.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.033+0000 7fda3be00640 1 --2- 172.21.15.159:0/582069296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda3409a190 0x7fda34155bc0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:53.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.033+0000 7fda3be00640 1 --2- 172.21.15.159:0/582069296 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda18004800 0x7fda3409bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:53.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.033+0000 7fda3be00640 1 -- 172.21.15.159:0/582069296 >> 172.21.15.159:0/582069296 conn(0x7fda3406b750 msgr2=0x7fda340afd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:53.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.033+0000 7fda3be00640 1 -- 172.21.15.159:0/582069296 shutdown_connections 2024-04-04T06:07:53.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.034+0000 7fda3be00640 1 -- 172.21.15.159:0/582069296 wait complete. 2024-04-04T06:07:53.066 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-04T06:07:53.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.255+0000 7f8f3ec00640 1 Processor -- start 2024-04-04T06:07:53.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.255+0000 7f8f3ec00640 1 -- start start 2024-04-04T06:07:53.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.256+0000 7f8f3ec00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f8f38153580 0x7f8f380d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:53.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.256+0000 7f8f3ec00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f380d07c0 0x7f8f380aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:53.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.256+0000 7f8f3ec00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f380afab0 con 0x7f8f380af2e0 2024-04-04T06:07:53.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.256+0000 7f8f3ec00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f8f380d0c00 con 0x7f8f38153580 2024-04-04T06:07:53.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.256+0000 7f8f3ec00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8f3815cc80 con 0x7f8f380d07c0 2024-04-04T06:07:53.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.257+0000 7f8f37e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f8f38153580 0x7f8f380d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:53.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.257+0000 7f8f3cc00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f8f380af2e0 0x7f8f3806d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:33348/0 (socket says 172.21.15.159:33348) 2024-04-04T06:07:53.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.257+0000 7f8f3cc00640 1 -- 172.21.15.159:0/2716765338 learned_addr learned my addr 172.21.15.159:0/2716765338 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:53.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.257+0000 7f8f37e00640 1 -- 172.21.15.159:0/2716765338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f380d07c0 msgr2=0x7f8f380aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:53.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.257+0000 7f8f37e00640 1 --2- 172.21.15.159:0/2716765338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f380d07c0 0x7f8f380aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:53.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.257+0000 7f8f37e00640 1 -- 172.21.15.159:0/2716765338 >> v1:172.21.15.159:6789/0 conn(0x7f8f380af2e0 legacy=0x7f8f3806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:53.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.257+0000 7f8f37e00640 1 -- 172.21.15.159:0/2716765338 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3815da00 con 0x7f8f38153580 2024-04-04T06:07:53.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.258+0000 7f8f37e00640 1 --2- 172.21.15.159:0/2716765338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f38153580 0x7f8f380d0280 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8f280097a0 tx=0x7f8f2802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14fa45a62234943 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:53.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.259+0000 7f8f36a00640 1 -- 172.21.15.159:0/2716765338 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28031780 con 0x7f8f38153580 2024-04-04T06:07:53.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.259+0000 7f8f36a00640 1 -- 172.21.15.159:0/2716765338 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f28037070 con 0x7f8f38153580 2024-04-04T06:07:53.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.259+0000 7f8f36a00640 1 -- 172.21.15.159:0/2716765338 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28031b40 con 0x7f8f38153580 2024-04-04T06:07:53.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.259+0000 7f8f3ec00640 1 -- 172.21.15.159:0/2716765338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f38153580 msgr2=0x7f8f380d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:53.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.259+0000 7f8f3ec00640 1 --2- 172.21.15.159:0/2716765338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f38153580 0x7f8f380d0280 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8f280097a0 tx=0x7f8f2802b820 comp rx=0 tx=0).stop 2024-04-04T06:07:53.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.259+0000 7f8f3ec00640 1 -- 172.21.15.159:0/2716765338 shutdown_connections 2024-04-04T06:07:53.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.259+0000 7f8f3ec00640 1 --2- 172.21.15.159:0/2716765338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f380d07c0 0x7f8f380aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:53.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.259+0000 7f8f3ec00640 1 --2- 172.21.15.159:0/2716765338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f38153580 0x7f8f380d0280 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:53.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.259+0000 7f8f3ec00640 1 -- 172.21.15.159:0/2716765338 >> 172.21.15.159:0/2716765338 conn(0x7f8f3806b750 msgr2=0x7f8f3806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:53.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.259+0000 7f8f3ec00640 1 -- 172.21.15.159:0/2716765338 shutdown_connections 2024-04-04T06:07:53.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.260+0000 7f8f3ec00640 1 -- 172.21.15.159:0/2716765338 wait complete. 2024-04-04T06:07:53.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.262+0000 7f8f3ec00640 1 Processor -- start 2024-04-04T06:07:53.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.262+0000 7f8f3ec00640 1 -- start start 2024-04-04T06:07:53.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.264+0000 7f8f3ec00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f380af2e0 0x7f8f380b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:53.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.264+0000 7f8f3ec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f380d07c0 0x7f8f380b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:53.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.264+0000 7f8f3ec00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f38072470 0x7f8f380cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:53.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.264+0000 7f8f3ec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3815d4e0 con 0x7f8f380d07c0 2024-04-04T06:07:53.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.264+0000 7f8f3ec00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3815d200 con 0x7f8f380af2e0 2024-04-04T06:07:53.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.264+0000 7f8f3ec00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8f38153960 con 0x7f8f38072470 2024-04-04T06:07:53.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.264+0000 7f8f37400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f380d07c0 0x7f8f380b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:53.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.264+0000 7f8f37400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f380d07c0 0x7f8f380b34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47414/0 (socket says 172.21.15.159:47414) 2024-04-04T06:07:53.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.264+0000 7f8f37400640 1 -- 172.21.15.159:0/3212046676 learned_addr learned my addr 172.21.15.159:0/3212046676 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:53.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.264+0000 7f8f3cc00640 1 --2- 172.21.15.159:0/3212046676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f38072470 0x7f8f380cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:53.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.264+0000 7f8f37e00640 1 --2- 172.21.15.159:0/3212046676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f380af2e0 0x7f8f380b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:53.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.264+0000 7f8f37400640 1 -- 172.21.15.159:0/3212046676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f38072470 msgr2=0x7f8f380cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:53.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.264+0000 7f8f37400640 1 --2- 172.21.15.159:0/3212046676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f38072470 0x7f8f380cade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:53.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.264+0000 7f8f37400640 1 -- 172.21.15.159:0/3212046676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f380af2e0 msgr2=0x7f8f380b2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:53.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.264+0000 7f8f37400640 1 --2- 172.21.15.159:0/3212046676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f380af2e0 0x7f8f380b2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:53.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.264+0000 7f8f37400640 1 -- 172.21.15.159:0/3212046676 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f380716b0 con 0x7f8f380d07c0 2024-04-04T06:07:53.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.264+0000 7f8f37e00640 1 --2- 172.21.15.159:0/3212046676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f380af2e0 0x7f8f380b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:53.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.265+0000 7f8f37400640 1 --2- 172.21.15.159:0/3212046676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f380d07c0 0x7f8f380b34b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000b4d0 tx=0x7f8f2000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:53.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.265+0000 7f8f2fe00640 1 -- 172.21.15.159:0/3212046676 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20014070 con 0x7f8f380d07c0 2024-04-04T06:07:53.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.265+0000 7f8f2fe00640 1 -- 172.21.15.159:0/3212046676 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f20010870 con 0x7f8f380d07c0 2024-04-04T06:07:53.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.265+0000 7f8f2fe00640 1 -- 172.21.15.159:0/3212046676 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20010c60 con 0x7f8f380d07c0 2024-04-04T06:07:53.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.265+0000 7f8f3ec00640 1 -- 172.21.15.159:0/3212046676 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f38071990 con 0x7f8f380d07c0 2024-04-04T06:07:53.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.265+0000 7f8f3ec00640 1 -- 172.21.15.159:0/3212046676 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f38071f50 con 0x7f8f380d07c0 2024-04-04T06:07:53.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.267+0000 7f8f3ec00640 1 -- 172.21.15.159:0/3212046676 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f380685c0 con 0x7f8f380d07c0 2024-04-04T06:07:53.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.268+0000 7f8f2fe00640 1 -- 172.21.15.159:0/3212046676 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f20017070 con 0x7f8f380d07c0 2024-04-04T06:07:53.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.269+0000 7f8f2fe00640 1 --2- 172.21.15.159:0/3212046676 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8f1406c8f0 0x7f8f1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:53.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.269+0000 7f8f2fe00640 1 -- 172.21.15.159:0/3212046676 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5609+0+0 (secure 0 0 0) 0x7f8f2008b8f0 con 0x7f8f380d07c0 2024-04-04T06:07:53.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.269+0000 7f8f37e00640 1 --2- 172.21.15.159:0/3212046676 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8f1406c8f0 0x7f8f1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:53.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.271+0000 7f8f37e00640 1 --2- 172.21.15.159:0/3212046676 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8f1406c8f0 0x7f8f1406edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8f28036000 tx=0x7f8f28008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:53.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.274+0000 7f8f2fe00640 1 -- 172.21.15.159:0/3212046676 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8f20059d80 con 0x7f8f380d07c0 2024-04-04T06:07:53.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:53.520+0000 7f8f3ec00640 1 -- 172.21.15.159:0/3212046676 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8f38072120 con 0x7f8f380d07c0 2024-04-04T06:07:54.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.071+0000 7f8f2fe00640 1 -- 172.21.15.159:0/3212046676 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) ==== 172+0+0 (secure 0 0 0) 0x7f8f20059680 con 0x7f8f380d07c0 2024-04-04T06:07:54.070 INFO:teuthology.orchestra.run.smithi159.stderr:new fs with metadata pool 6 and data pool 7 2024-04-04T06:07:54.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.078+0000 7f8f3ec00640 1 -- 172.21.15.159:0/3212046676 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8f1406c8f0 msgr2=0x7f8f1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:54.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.078+0000 7f8f3ec00640 1 --2- 172.21.15.159:0/3212046676 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8f1406c8f0 0x7f8f1406edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8f28036000 tx=0x7f8f28008030 comp rx=0 tx=0).stop 2024-04-04T06:07:54.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.078+0000 7f8f3ec00640 1 -- 172.21.15.159:0/3212046676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f380d07c0 msgr2=0x7f8f380b34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:54.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.078+0000 7f8f3ec00640 1 --2- 172.21.15.159:0/3212046676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f380d07c0 0x7f8f380b34b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000b4d0 tx=0x7f8f2000b990 comp rx=0 tx=0).stop 2024-04-04T06:07:54.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.078+0000 7f8f3ec00640 1 -- 172.21.15.159:0/3212046676 shutdown_connections 2024-04-04T06:07:54.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.078+0000 7f8f3ec00640 1 --2- 172.21.15.159:0/3212046676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f38072470 0x7f8f380cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.078+0000 7f8f3ec00640 1 --2- 172.21.15.159:0/3212046676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f380d07c0 0x7f8f380b34b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.078+0000 7f8f3ec00640 1 --2- 172.21.15.159:0/3212046676 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8f1406c8f0 0x7f8f1406edb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.078+0000 7f8f3ec00640 1 --2- 172.21.15.159:0/3212046676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f380af2e0 0x7f8f380b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.078+0000 7f8f3ec00640 1 -- 172.21.15.159:0/3212046676 >> 172.21.15.159:0/3212046676 conn(0x7f8f3806b750 msgr2=0x7f8f3815e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:54.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.079+0000 7f8f3ec00640 1 -- 172.21.15.159:0/3212046676 shutdown_connections 2024-04-04T06:07:54.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.079+0000 7f8f3ec00640 1 -- 172.21.15.159:0/3212046676 wait complete. 2024-04-04T06:07:54.111 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:07:54.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.294+0000 7f53a9800640 1 Processor -- start 2024-04-04T06:07:54.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.295+0000 7f53a9800640 1 -- start start 2024-04-04T06:07:54.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.296+0000 7f53a9800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f53a414ba20 0x7f53a40b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:54.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.296+0000 7f53a9800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f53a414c1b0 0x7f53a4166290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:54.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.296+0000 7f53a9800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53a40b33e0 con 0x7f53a40cfda0 2024-04-04T06:07:54.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.296+0000 7f53a9800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f53a40cde20 con 0x7f53a414c1b0 2024-04-04T06:07:54.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.296+0000 7f53a9800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f53a4166ae0 con 0x7f53a414ba20 2024-04-04T06:07:54.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.296+0000 7f539b400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f53a414c1b0 0x7f53a4166290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:54.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.296+0000 7f539b400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f53a414c1b0 0x7f53a4166290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50350/0 (socket says 172.21.15.159:50350) 2024-04-04T06:07:54.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.296+0000 7f539b400640 1 -- 172.21.15.159:0/2723648632 learned_addr learned my addr 172.21.15.159:0/2723648632 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:54.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.297+0000 7f539b400640 1 -- 172.21.15.159:0/2723648632 >> v2:172.21.15.178:3300/0 conn(0x7f53a414ba20 msgr2=0x7f53a40b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:54.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.297+0000 7f539b400640 1 --2- 172.21.15.159:0/2723648632 >> v2:172.21.15.178:3300/0 conn(0x7f53a414ba20 0x7f53a40b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.297+0000 7f539b400640 1 -- 172.21.15.159:0/2723648632 >> v1:172.21.15.178:6789/0 conn(0x7f53a40cfda0 legacy=0x7f53a40d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:54.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.297+0000 7f539b400640 1 -- 172.21.15.159:0/2723648632 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a4166dc0 con 0x7f53a414c1b0 2024-04-04T06:07:54.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.298+0000 7f539b400640 1 --2- 172.21.15.159:0/2723648632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53a414c1b0 0x7f53a4166290 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f53a406f160 tx=0x7f538c02b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c239f923cf6317e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:54.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.298+0000 7f53a3400640 1 -- 172.21.15.159:0/2723648632 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c035020 con 0x7f53a414c1b0 2024-04-04T06:07:54.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.298+0000 7f53a3400640 1 -- 172.21.15.159:0/2723648632 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f538c031890 con 0x7f53a414c1b0 2024-04-04T06:07:54.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.298+0000 7f53a3400640 1 -- 172.21.15.159:0/2723648632 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c031c60 con 0x7f53a414c1b0 2024-04-04T06:07:54.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.299+0000 7f53a9800640 1 -- 172.21.15.159:0/2723648632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53a414c1b0 msgr2=0x7f53a4166290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:54.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.299+0000 7f53a9800640 1 --2- 172.21.15.159:0/2723648632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53a414c1b0 0x7f53a4166290 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f53a406f160 tx=0x7f538c02b930 comp rx=0 tx=0).stop 2024-04-04T06:07:54.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.299+0000 7f53a9800640 1 -- 172.21.15.159:0/2723648632 shutdown_connections 2024-04-04T06:07:54.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.299+0000 7f53a9800640 1 --2- 172.21.15.159:0/2723648632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53a414c1b0 0x7f53a4166290 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.299+0000 7f53a9800640 1 --2- 172.21.15.159:0/2723648632 >> v2:172.21.15.178:3300/0 conn(0x7f53a414ba20 0x7f53a40b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.299+0000 7f53a9800640 1 -- 172.21.15.159:0/2723648632 >> 172.21.15.159:0/2723648632 conn(0x7f53a40ccad0 msgr2=0x7f53a40cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:54.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.299+0000 7f53a9800640 1 -- 172.21.15.159:0/2723648632 shutdown_connections 2024-04-04T06:07:54.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.299+0000 7f53a9800640 1 -- 172.21.15.159:0/2723648632 wait complete. 2024-04-04T06:07:54.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.301+0000 7f53a9800640 1 Processor -- start 2024-04-04T06:07:54.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.301+0000 7f53a9800640 1 -- start start 2024-04-04T06:07:54.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.302+0000 7f53a9800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53a40cfda0 0x7f53a41458d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:54.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.302+0000 7f53a9800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53a414ba20 0x7f53a4146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:54.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.302+0000 7f53a9800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53a414c1b0 0x7f53a4148b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:54.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.302+0000 7f53a9800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f53a41667d0 con 0x7f53a40cfda0 2024-04-04T06:07:54.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.302+0000 7f53a9800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f53a40ce0c0 con 0x7f53a414c1b0 2024-04-04T06:07:54.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.302+0000 7f53a9800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f53a414be00 con 0x7f53a414ba20 2024-04-04T06:07:54.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.302+0000 7f53a3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53a40cfda0 0x7f53a41458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:54.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.302+0000 7f53a3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53a40cfda0 0x7f53a41458d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50358/0 (socket says 172.21.15.159:50358) 2024-04-04T06:07:54.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.302+0000 7f53a3e00640 1 -- 172.21.15.159:0/2230669299 learned_addr learned my addr 172.21.15.159:0/2230669299 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:54.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.302+0000 7f53a3e00640 1 -- 172.21.15.159:0/2230669299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53a414ba20 msgr2=0x7f53a4146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:54.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.302+0000 7f539b400640 1 --2- 172.21.15.159:0/2230669299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53a414ba20 0x7f53a4146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:54.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.302+0000 7f53a8e00640 1 --2- 172.21.15.159:0/2230669299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53a414c1b0 0x7f53a4148b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:54.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.302+0000 7f53a3e00640 1 --2- 172.21.15.159:0/2230669299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53a414ba20 0x7f53a4146860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.302+0000 7f53a3e00640 1 -- 172.21.15.159:0/2230669299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53a414c1b0 msgr2=0x7f53a4148b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:54.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.302+0000 7f53a3e00640 1 --2- 172.21.15.159:0/2230669299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53a414c1b0 0x7f53a4148b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.302+0000 7f53a3e00640 1 -- 172.21.15.159:0/2230669299 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a4146da0 con 0x7f53a40cfda0 2024-04-04T06:07:54.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.303+0000 7f53a3e00640 1 --2- 172.21.15.159:0/2230669299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53a40cfda0 0x7f53a41458d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f539400b4a0 tx=0x7f539400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:54.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.303+0000 7f53a0c00640 1 -- 172.21.15.159:0/2230669299 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394014070 con 0x7f53a40cfda0 2024-04-04T06:07:54.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.303+0000 7f53a0c00640 1 -- 172.21.15.159:0/2230669299 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5394010610 con 0x7f53a40cfda0 2024-04-04T06:07:54.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.303+0000 7f53a0c00640 1 -- 172.21.15.159:0/2230669299 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394010a00 con 0x7f53a40cfda0 2024-04-04T06:07:54.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.303+0000 7f53a9800640 1 -- 172.21.15.159:0/2230669299 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a41ec200 con 0x7f53a40cfda0 2024-04-04T06:07:54.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.303+0000 7f53a9800640 1 -- 172.21.15.159:0/2230669299 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a41ec7c0 con 0x7f53a40cfda0 2024-04-04T06:07:54.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.304+0000 7f53a9800640 1 -- 172.21.15.159:0/2230669299 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a4073450 con 0x7f53a40cfda0 2024-04-04T06:07:54.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.305+0000 7f53a0c00640 1 -- 172.21.15.159:0/2230669299 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5394017070 con 0x7f53a40cfda0 2024-04-04T06:07:54.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.307+0000 7f53a0c00640 1 --2- 172.21.15.159:0/2230669299 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f538006c8f0 0x7f538006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:54.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.307+0000 7f53a0c00640 1 -- 172.21.15.159:0/2230669299 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f539408b5b0 con 0x7f53a40cfda0 2024-04-04T06:07:54.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.307+0000 7f539b400640 1 --2- 172.21.15.159:0/2230669299 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f538006c8f0 0x7f538006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:54.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.308+0000 7f539b400640 1 --2- 172.21.15.159:0/2230669299 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f538006c8f0 0x7f538006edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f538c036000 tx=0x7f538c03c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:54.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.312+0000 7f53a0c00640 1 -- 172.21.15.159:0/2230669299 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f53a4073450 con 0x7f53a40cfda0 2024-04-04T06:07:54.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.505+0000 7f53a9800640 1 -- 172.21.15.159:0/2230669299 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53a4075940 con 0x7f53a40cfda0 2024-04-04T06:07:54.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.506+0000 7f53a0c00640 1 -- 172.21.15.159:0/2230669299 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16638 (secure 0 0 0) 0x7f53a4146da0 con 0x7f53a40cfda0 2024-04-04T06:07:54.501 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:54.501 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":45,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:07:53.522340+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:07:51.357366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-04T06:07:52.495884+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:07:54.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.509+0000 7f53a9800640 1 -- 172.21.15.159:0/2230669299 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f538006c8f0 msgr2=0x7f538006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:54.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.509+0000 7f53a9800640 1 --2- 172.21.15.159:0/2230669299 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f538006c8f0 0x7f538006edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f538c036000 tx=0x7f538c03c040 comp rx=0 tx=0).stop 2024-04-04T06:07:54.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.509+0000 7f53a9800640 1 -- 172.21.15.159:0/2230669299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53a40cfda0 msgr2=0x7f53a41458d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:54.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.509+0000 7f53a9800640 1 --2- 172.21.15.159:0/2230669299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53a40cfda0 0x7f53a41458d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f539400b4a0 tx=0x7f539400b960 comp rx=0 tx=0).stop 2024-04-04T06:07:54.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.509+0000 7f53a9800640 1 -- 172.21.15.159:0/2230669299 shutdown_connections 2024-04-04T06:07:54.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.509+0000 7f53a9800640 1 --2- 172.21.15.159:0/2230669299 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f538006c8f0 0x7f538006edb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.509+0000 7f53a9800640 1 --2- 172.21.15.159:0/2230669299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53a414c1b0 0x7f53a4148b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.509+0000 7f53a9800640 1 --2- 172.21.15.159:0/2230669299 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53a414ba20 0x7f53a4146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.509+0000 7f53a9800640 1 --2- 172.21.15.159:0/2230669299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53a40cfda0 0x7f53a41458d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.509+0000 7f53a9800640 1 -- 172.21.15.159:0/2230669299 >> 172.21.15.159:0/2230669299 conn(0x7f53a40ccad0 msgr2=0x7f53a40ce5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:54.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.510+0000 7f53a9800640 1 -- 172.21.15.159:0/2230669299 shutdown_connections 2024-04-04T06:07:54.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.510+0000 7f53a9800640 1 -- 172.21.15.159:0/2230669299 wait complete. 2024-04-04T06:07:54.521 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:07:54.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.728+0000 7fb78f000640 1 Processor -- start 2024-04-04T06:07:54.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.728+0000 7fb78f000640 1 -- start start 2024-04-04T06:07:54.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.730+0000 7fb78f000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb7880727f0 0x7fb7880b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:54.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.730+0000 7fb78f000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb78815b6f0 con 0x7fb7880d0780 2024-04-04T06:07:54.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.730+0000 7fb78f000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb788072e80 con 0x7fb78815b310 2024-04-04T06:07:54.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.730+0000 7fb78f000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fb788072c50 con 0x7fb7880727f0 2024-04-04T06:07:54.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.730+0000 7fb78d000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fb7880d0780 0x7fb7880adc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37024/0 (socket says 172.21.15.159:37024) 2024-04-04T06:07:54.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.730+0000 7fb78d000640 1 -- 172.21.15.159:0/3345228075 learned_addr learned my addr 172.21.15.159:0/3345228075 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:54.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.730+0000 7fb787400640 1 --2- 172.21.15.159:0/3345228075 >> v2:172.21.15.159:3300/0 conn(0x7fb7880727f0 0x7fb7880b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:54.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.730+0000 7fb786a00640 1 -- 172.21.15.159:0/3345228075 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3470273488 0 0) 0x7fb78815b6f0 con 0x7fb7880d0780 2024-04-04T06:07:54.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.730+0000 7fb786a00640 1 -- 172.21.15.159:0/3345228075 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb76c003520 con 0x7fb7880d0780 2024-04-04T06:07:54.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.730+0000 7fb787400640 1 -- 172.21.15.159:0/3345228075 >> v1:172.21.15.178:6789/0 conn(0x7fb78815b310 legacy=0x7fb78815f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:54.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.731+0000 7fb787400640 1 -- 172.21.15.159:0/3345228075 >> v1:172.21.15.159:6789/0 conn(0x7fb7880d0780 legacy=0x7fb7880adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:54.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.731+0000 7fb787400640 1 -- 172.21.15.159:0/3345228075 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb788073160 con 0x7fb7880727f0 2024-04-04T06:07:54.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.732+0000 7fb787400640 1 --2- 172.21.15.159:0/3345228075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7880727f0 0x7fb7880b4f90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb7740097a0 tx=0x7fb77402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31abf395595a9ff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:54.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.732+0000 7fb786a00640 1 -- 172.21.15.159:0/3345228075 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774038070 con 0x7fb7880727f0 2024-04-04T06:07:54.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.732+0000 7fb786a00640 1 -- 172.21.15.159:0/3345228075 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb774031700 con 0x7fb7880727f0 2024-04-04T06:07:54.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.732+0000 7fb786a00640 1 -- 172.21.15.159:0/3345228075 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774031ad0 con 0x7fb7880727f0 2024-04-04T06:07:54.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.732+0000 7fb78f000640 1 -- 172.21.15.159:0/3345228075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7880727f0 msgr2=0x7fb7880b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:54.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.732+0000 7fb78f000640 1 --2- 172.21.15.159:0/3345228075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7880727f0 0x7fb7880b4f90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb7740097a0 tx=0x7fb77402b7a0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.733+0000 7fb78f000640 1 -- 172.21.15.159:0/3345228075 shutdown_connections 2024-04-04T06:07:54.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.733+0000 7fb78f000640 1 --2- 172.21.15.159:0/3345228075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7880727f0 0x7fb7880b4f90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.733+0000 7fb78f000640 1 -- 172.21.15.159:0/3345228075 >> 172.21.15.159:0/3345228075 conn(0x7fb78806b990 msgr2=0x7fb78806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:54.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.733+0000 7fb78f000640 1 -- 172.21.15.159:0/3345228075 shutdown_connections 2024-04-04T06:07:54.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.733+0000 7fb78f000640 1 -- 172.21.15.159:0/3345228075 wait complete. 2024-04-04T06:07:54.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.734+0000 7fb78f000640 1 Processor -- start 2024-04-04T06:07:54.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.734+0000 7fb78f000640 1 -- start start 2024-04-04T06:07:54.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.735+0000 7fb78f000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7880727f0 0x7fb7881f2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:54.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.735+0000 7fb78f000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7880d0780 0x7fb78809b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:54.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.735+0000 7fb78f000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb78815b310 0x7fb78815beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:54.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.735+0000 7fb78f000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb788073440 con 0x7fb7880d0780 2024-04-04T06:07:54.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.735+0000 7fb78f000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb78815f2c0 con 0x7fb7880727f0 2024-04-04T06:07:54.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.735+0000 7fb78f000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb7880ae390 con 0x7fb78815b310 2024-04-04T06:07:54.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.735+0000 7fb787400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7880d0780 0x7fb78809b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:54.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.735+0000 7fb787400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7880d0780 0x7fb78809b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50376/0 (socket says 172.21.15.159:50376) 2024-04-04T06:07:54.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.735+0000 7fb787400640 1 -- 172.21.15.159:0/2870242093 learned_addr learned my addr 172.21.15.159:0/2870242093 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:54.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.736+0000 7fb787400640 1 -- 172.21.15.159:0/2870242093 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb78815b310 msgr2=0x7fb78815beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:54.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.736+0000 7fb787400640 1 --2- 172.21.15.159:0/2870242093 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb78815b310 0x7fb78815beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.736+0000 7fb787400640 1 -- 172.21.15.159:0/2870242093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7880727f0 msgr2=0x7fb7881f2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:54.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.736+0000 7fb787400640 1 --2- 172.21.15.159:0/2870242093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7880727f0 0x7fb7881f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.736+0000 7fb787400640 1 -- 172.21.15.159:0/2870242093 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7881f2ee0 con 0x7fb7880d0780 2024-04-04T06:07:54.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.736+0000 7fb787400640 1 --2- 172.21.15.159:0/2870242093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7880d0780 0x7fb78809b5b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb774000c00 tx=0x7fb7740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:54.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.736+0000 7fb77fe00640 1 -- 172.21.15.159:0/2870242093 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774042070 con 0x7fb7880d0780 2024-04-04T06:07:54.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.736+0000 7fb77fe00640 1 -- 172.21.15.159:0/2870242093 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77403d890 con 0x7fb7880d0780 2024-04-04T06:07:54.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.736+0000 7fb77fe00640 1 -- 172.21.15.159:0/2870242093 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774038040 con 0x7fb7880d0780 2024-04-04T06:07:54.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.736+0000 7fb78f000640 1 -- 172.21.15.159:0/2870242093 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7881f31c0 con 0x7fb7880d0780 2024-04-04T06:07:54.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.737+0000 7fb78f000640 1 -- 172.21.15.159:0/2870242093 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb78815b990 con 0x7fb7880d0780 2024-04-04T06:07:54.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.738+0000 7fb78f000640 1 -- 172.21.15.159:0/2870242093 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb78806c2d0 con 0x7fb7880d0780 2024-04-04T06:07:54.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.738+0000 7fb77fe00640 1 -- 172.21.15.159:0/2870242093 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb774045070 con 0x7fb7880d0780 2024-04-04T06:07:54.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.739+0000 7fb77fe00640 1 --2- 172.21.15.159:0/2870242093 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb76806c8f0 0x7fb76806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:54.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.739+0000 7fb77fe00640 1 -- 172.21.15.159:0/2870242093 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fb7740ad700 con 0x7fb7880d0780 2024-04-04T06:07:54.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.740+0000 7fb787e00640 1 --2- 172.21.15.159:0/2870242093 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb76806c8f0 0x7fb76806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:54.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.741+0000 7fb787e00640 1 --2- 172.21.15.159:0/2870242093 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb76806c8f0 0x7fb76806edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb778000f60 tx=0x7fb7780079b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:54.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.744+0000 7fb77fe00640 1 -- 172.21.15.159:0/2870242093 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb78806c2d0 con 0x7fb7880d0780 2024-04-04T06:07:54.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.931+0000 7fb78f000640 1 -- 172.21.15.159:0/2870242093 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb78815bb90 con 0x7fb7880d0780 2024-04-04T06:07:54.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.933+0000 7fb77fe00640 1 -- 172.21.15.159:0/2870242093 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16638 (secure 0 0 0) 0x7fb78815bb90 con 0x7fb7880d0780 2024-04-04T06:07:54.928 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:54.928 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":45,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:07:53.522340+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:07:51.357366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-04T06:07:52.495884+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:07:54.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.936+0000 7fb78f000640 1 -- 172.21.15.159:0/2870242093 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb76806c8f0 msgr2=0x7fb76806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:54.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.936+0000 7fb78f000640 1 --2- 172.21.15.159:0/2870242093 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb76806c8f0 0x7fb76806edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb778000f60 tx=0x7fb7780079b0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.936+0000 7fb78f000640 1 -- 172.21.15.159:0/2870242093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7880d0780 msgr2=0x7fb78809b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:54.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.936+0000 7fb78f000640 1 --2- 172.21.15.159:0/2870242093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7880d0780 0x7fb78809b5b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb774000c00 tx=0x7fb7740046c0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.936+0000 7fb78f000640 1 -- 172.21.15.159:0/2870242093 shutdown_connections 2024-04-04T06:07:54.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.936+0000 7fb78f000640 1 --2- 172.21.15.159:0/2870242093 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb78815b310 0x7fb78815beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.936+0000 7fb78f000640 1 --2- 172.21.15.159:0/2870242093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7880d0780 0x7fb78809b5b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.936+0000 7fb78f000640 1 --2- 172.21.15.159:0/2870242093 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb76806c8f0 0x7fb76806edb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.936+0000 7fb78f000640 1 --2- 172.21.15.159:0/2870242093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7880727f0 0x7fb7881f2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:54.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.936+0000 7fb78f000640 1 -- 172.21.15.159:0/2870242093 >> 172.21.15.159:0/2870242093 conn(0x7fb78806b990 msgr2=0x7fb788157760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:54.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.936+0000 7fb78f000640 1 -- 172.21.15.159:0/2870242093 shutdown_connections 2024-04-04T06:07:54.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:54.936+0000 7fb78f000640 1 -- 172.21.15.159:0/2870242093 wait complete. 2024-04-04T06:07:54.948 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-04T06:07:55.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.195+0000 7f7054c00640 1 Processor -- start 2024-04-04T06:07:55.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.195+0000 7f7054c00640 1 -- start start 2024-04-04T06:07:55.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.197+0000 7f7054c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f705015e860 0x7f705006d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:55.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.197+0000 7f7054c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f705006dfa0 con 0x7f705006da30 2024-04-04T06:07:55.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.197+0000 7f7054c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f705006e240 con 0x7f70500d4ae0 2024-04-04T06:07:55.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.197+0000 7f7054c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f70500cf220 con 0x7f705015e860 2024-04-04T06:07:55.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.197+0000 7f704de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f705015e860 0x7f705006d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:55.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.197+0000 7f704de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f705015e860 0x7f705006d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50384/0 (socket says 172.21.15.159:50384) 2024-04-04T06:07:55.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.197+0000 7f704de00640 1 -- 172.21.15.159:0/1583238775 learned_addr learned my addr 172.21.15.159:0/1583238775 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:55.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.197+0000 7f7045400640 1 --1- 172.21.15.159:0/1583238775 >> v1:172.21.15.159:6789/0 conn(0x7f705006da30 0x7f70500d43d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37040/0 (socket says 172.21.15.159:37040) 2024-04-04T06:07:55.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.198+0000 7f704de00640 1 -- 172.21.15.159:0/1583238775 >> v1:172.21.15.178:6789/0 conn(0x7f70500d4ae0 legacy=0x7f70500cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:55.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.198+0000 7f704de00640 1 -- 172.21.15.159:0/1583238775 >> v1:172.21.15.159:6789/0 conn(0x7f705006da30 legacy=0x7f70500d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:55.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.198+0000 7f704de00640 1 -- 172.21.15.159:0/1583238775 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705006e520 con 0x7f705015e860 2024-04-04T06:07:55.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.198+0000 7f704d400640 1 -- 172.21.15.159:0/1583238775 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2275989460 0 0) 0x7f705006dfa0 con 0x7f705006da30 2024-04-04T06:07:55.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.199+0000 7f704de00640 1 --2- 172.21.15.159:0/1583238775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f705015e860 0x7f705006d4f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7040009870 tx=0x7f704002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d604df975bd69450 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:55.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.199+0000 7f704d400640 1 -- 172.21.15.159:0/1583238775 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040034070 con 0x7f705015e860 2024-04-04T06:07:55.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.199+0000 7f704d400640 1 -- 172.21.15.159:0/1583238775 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7040030860 con 0x7f705015e860 2024-04-04T06:07:55.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.199+0000 7f704d400640 1 -- 172.21.15.159:0/1583238775 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040030c30 con 0x7f705015e860 2024-04-04T06:07:55.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.200+0000 7f7054c00640 1 -- 172.21.15.159:0/1583238775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f705015e860 msgr2=0x7f705006d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:55.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.200+0000 7f7054c00640 1 --2- 172.21.15.159:0/1583238775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f705015e860 0x7f705006d4f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7040009870 tx=0x7f704002b9a0 comp rx=0 tx=0).stop 2024-04-04T06:07:55.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.200+0000 7f7054c00640 1 -- 172.21.15.159:0/1583238775 shutdown_connections 2024-04-04T06:07:55.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.200+0000 7f7054c00640 1 --2- 172.21.15.159:0/1583238775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f705015e860 0x7f705006d4f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:55.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.200+0000 7f7054c00640 1 -- 172.21.15.159:0/1583238775 >> 172.21.15.159:0/1583238775 conn(0x7f705009d7d0 msgr2=0x7f705008bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:55.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.200+0000 7f7054c00640 1 -- 172.21.15.159:0/1583238775 shutdown_connections 2024-04-04T06:07:55.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.200+0000 7f7054c00640 1 -- 172.21.15.159:0/1583238775 wait complete. 2024-04-04T06:07:55.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.202+0000 7f7054c00640 1 Processor -- start 2024-04-04T06:07:55.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.202+0000 7f7054c00640 1 -- start start 2024-04-04T06:07:55.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.203+0000 7f7054c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f705006da30 0x7f705014b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:55.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.203+0000 7f7054c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70500d4ae0 0x7f70500b9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:55.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.203+0000 7f7054c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f705015e860 0x7f70500b7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:55.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.203+0000 7f7054c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f70500cef30 con 0x7f705015e860 2024-04-04T06:07:55.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.203+0000 7f7054c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f70500cedb0 con 0x7f70500d4ae0 2024-04-04T06:07:55.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.203+0000 7f7054c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f70500cfdc0 con 0x7f705006da30 2024-04-04T06:07:55.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.203+0000 7f704e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f705015e860 0x7f70500b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:55.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.203+0000 7f704e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f705015e860 0x7f70500b7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50388/0 (socket says 172.21.15.159:50388) 2024-04-04T06:07:55.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.203+0000 7f704e800640 1 -- 172.21.15.159:0/776239931 learned_addr learned my addr 172.21.15.159:0/776239931 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:55.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.203+0000 7f704de00640 1 --2- 172.21.15.159:0/776239931 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f705006da30 0x7f705014b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:55.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.203+0000 7f7045400640 1 --2- 172.21.15.159:0/776239931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70500d4ae0 0x7f70500b9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:55.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.204+0000 7f704e800640 1 -- 172.21.15.159:0/776239931 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f705006da30 msgr2=0x7f705014b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:55.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.204+0000 7f704e800640 1 --2- 172.21.15.159:0/776239931 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f705006da30 0x7f705014b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:55.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.204+0000 7f704e800640 1 -- 172.21.15.159:0/776239931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70500d4ae0 msgr2=0x7f70500b9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:55.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.204+0000 7f704e800640 1 --2- 172.21.15.159:0/776239931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70500d4ae0 0x7f70500b9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:55.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.204+0000 7f704e800640 1 -- 172.21.15.159:0/776239931 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70500c94f0 con 0x7f705015e860 2024-04-04T06:07:55.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.204+0000 7f704e800640 1 --2- 172.21.15.159:0/776239931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f705015e860 0x7f70500b7020 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f704800c780 tx=0x7f704800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:55.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.204+0000 7f7046a00640 1 -- 172.21.15.159:0/776239931 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048004b60 con 0x7f705015e860 2024-04-04T06:07:55.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.204+0000 7f7046a00640 1 -- 172.21.15.159:0/776239931 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7048004d00 con 0x7f705015e860 2024-04-04T06:07:55.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.204+0000 7f7054c00640 1 -- 172.21.15.159:0/776239931 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70500c97d0 con 0x7f705015e860 2024-04-04T06:07:55.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.204+0000 7f7046a00640 1 -- 172.21.15.159:0/776239931 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704800aa90 con 0x7f705015e860 2024-04-04T06:07:55.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.204+0000 7f7054c00640 1 -- 172.21.15.159:0/776239931 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70500c9d90 con 0x7f705015e860 2024-04-04T06:07:55.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.206+0000 7f7054c00640 1 -- 172.21.15.159:0/776239931 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70500685c0 con 0x7f705015e860 2024-04-04T06:07:55.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.207+0000 7f7046a00640 1 -- 172.21.15.159:0/776239931 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f704800ac30 con 0x7f705015e860 2024-04-04T06:07:55.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.207+0000 7f7046a00640 1 --2- 172.21.15.159:0/776239931 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f702c06c920 0x7f702c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:55.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.208+0000 7f7046a00640 1 -- 172.21.15.159:0/776239931 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f704808be60 con 0x7f705015e860 2024-04-04T06:07:55.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.208+0000 7f704de00640 1 --2- 172.21.15.159:0/776239931 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f702c06c920 0x7f702c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:55.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.209+0000 7f704de00640 1 --2- 172.21.15.159:0/776239931 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f702c06c920 0x7f702c06ede0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7040036000 tx=0x7f704003b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:55.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.212+0000 7f7046a00640 1 -- 172.21.15.159:0/776239931 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7048020070 con 0x7f705015e860 2024-04-04T06:07:55.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:55.509+0000 7f7054c00640 1 -- 172.21.15.159:0/776239931 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f70500cf750 con 0x7f705015e860 2024-04-04T06:07:56.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.092+0000 7f7046a00640 1 -- 172.21.15.159:0/776239931 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) ==== 118+0+0 (secure 0 0 0) 0x7f704805a200 con 0x7f705015e860 2024-04-04T06:07:56.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.097+0000 7f7054c00640 1 -- 172.21.15.159:0/776239931 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f702c06c920 msgr2=0x7f702c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:56.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.097+0000 7f7054c00640 1 --2- 172.21.15.159:0/776239931 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f702c06c920 0x7f702c06ede0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7040036000 tx=0x7f704003b040 comp rx=0 tx=0).stop 2024-04-04T06:07:56.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.097+0000 7f7054c00640 1 -- 172.21.15.159:0/776239931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f705015e860 msgr2=0x7f70500b7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:56.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.097+0000 7f7054c00640 1 --2- 172.21.15.159:0/776239931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f705015e860 0x7f70500b7020 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f704800c780 tx=0x7f704800cc40 comp rx=0 tx=0).stop 2024-04-04T06:07:56.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.097+0000 7f7054c00640 1 -- 172.21.15.159:0/776239931 shutdown_connections 2024-04-04T06:07:56.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.097+0000 7f7054c00640 1 --2- 172.21.15.159:0/776239931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f705015e860 0x7f70500b7020 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:56.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.097+0000 7f7054c00640 1 --2- 172.21.15.159:0/776239931 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f702c06c920 0x7f702c06ede0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:56.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.097+0000 7f7054c00640 1 --2- 172.21.15.159:0/776239931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70500d4ae0 0x7f70500b9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:56.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.097+0000 7f7054c00640 1 --2- 172.21.15.159:0/776239931 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f705006da30 0x7f705014b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:56.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.097+0000 7f7054c00640 1 -- 172.21.15.159:0/776239931 >> 172.21.15.159:0/776239931 conn(0x7f705009d7d0 msgr2=0x7f70500b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:56.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.097+0000 7f7054c00640 1 -- 172.21.15.159:0/776239931 shutdown_connections 2024-04-04T06:07:56.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.097+0000 7f7054c00640 1 -- 172.21.15.159:0/776239931 wait complete. 2024-04-04T06:07:56.125 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-04T06:07:56.125 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-04T06:07:56.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.339+0000 7f13dde00640 1 Processor -- start 2024-04-04T06:07:56.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.339+0000 7f13dde00640 1 -- start start 2024-04-04T06:07:56.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.341+0000 7f13dde00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13d81590c0 0x7f13d815f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:56.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.341+0000 7f13dde00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f13d806fa50 0x7f13d806ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:56.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.341+0000 7f13dde00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13d815fe10 con 0x7f13d806c580 2024-04-04T06:07:56.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.341+0000 7f13dde00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f13d806e5b0 con 0x7f13d806fa50 2024-04-04T06:07:56.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.341+0000 7f13dde00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f13d8067900 con 0x7f13d81590c0 2024-04-04T06:07:56.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.342+0000 7f13d6a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13d81590c0 0x7f13d815f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:56.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.342+0000 7f13d6a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13d81590c0 0x7f13d815f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:47156/0 (socket says 172.21.15.159:47156) 2024-04-04T06:07:56.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.342+0000 7f13d6a00640 1 -- 172.21.15.159:0/3721232582 learned_addr learned my addr 172.21.15.159:0/3721232582 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:56.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.342+0000 7f13d7400640 1 --2- 172.21.15.159:0/3721232582 >> v2:172.21.15.178:3300/0 conn(0x7f13d806fa50 0x7f13d806ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:56.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.343+0000 7f13d6a00640 1 -- 172.21.15.159:0/3721232582 >> v2:172.21.15.178:3300/0 conn(0x7f13d806fa50 msgr2=0x7f13d806ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:56.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.343+0000 7f13d6a00640 1 --2- 172.21.15.159:0/3721232582 >> v2:172.21.15.178:3300/0 conn(0x7f13d806fa50 0x7f13d806ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:56.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.343+0000 7f13d6a00640 1 -- 172.21.15.159:0/3721232582 >> v1:172.21.15.178:6789/0 conn(0x7f13d806c580 legacy=0x7f13d8162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:56.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.343+0000 7f13d6a00640 1 -- 172.21.15.159:0/3721232582 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d80b47d0 con 0x7f13d81590c0 2024-04-04T06:07:56.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.344+0000 7f13d6a00640 1 --2- 172.21.15.159:0/3721232582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13d81590c0 0x7f13d815f8d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009870 tx=0x7f13c802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8d02db4f393a015 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:56.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.344+0000 7f13d6000640 1 -- 172.21.15.159:0/3721232582 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8034070 con 0x7f13d81590c0 2024-04-04T06:07:56.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.344+0000 7f13d6000640 1 -- 172.21.15.159:0/3721232582 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c8030840 con 0x7f13d81590c0 2024-04-04T06:07:56.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.344+0000 7f13d6000640 1 -- 172.21.15.159:0/3721232582 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8030c10 con 0x7f13d81590c0 2024-04-04T06:07:56.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.345+0000 7f13dde00640 1 -- 172.21.15.159:0/3721232582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13d81590c0 msgr2=0x7f13d815f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:56.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.345+0000 7f13dde00640 1 --2- 172.21.15.159:0/3721232582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13d81590c0 0x7f13d815f8d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009870 tx=0x7f13c802b980 comp rx=0 tx=0).stop 2024-04-04T06:07:56.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.345+0000 7f13dde00640 1 -- 172.21.15.159:0/3721232582 shutdown_connections 2024-04-04T06:07:56.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.345+0000 7f13dde00640 1 --2- 172.21.15.159:0/3721232582 >> v2:172.21.15.178:3300/0 conn(0x7f13d806fa50 0x7f13d806ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:56.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.345+0000 7f13dde00640 1 --2- 172.21.15.159:0/3721232582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13d81590c0 0x7f13d815f8d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:56.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.345+0000 7f13dde00640 1 -- 172.21.15.159:0/3721232582 >> 172.21.15.159:0/3721232582 conn(0x7f13d8071e60 msgr2=0x7f13d8072270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:56.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.345+0000 7f13dde00640 1 -- 172.21.15.159:0/3721232582 shutdown_connections 2024-04-04T06:07:56.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.345+0000 7f13dde00640 1 -- 172.21.15.159:0/3721232582 wait complete. 2024-04-04T06:07:56.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.347+0000 7f13dde00640 1 Processor -- start 2024-04-04T06:07:56.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.347+0000 7f13dde00640 1 -- start start 2024-04-04T06:07:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.348+0000 7f13dde00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13d806c580 0x7f13d81f5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.348+0000 7f13dde00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13d806fa50 0x7f13d81f7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.348+0000 7f13dde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13d81590c0 0x7f13d80cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.348+0000 7f13dde00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f13d806e240 con 0x7f13d81590c0 2024-04-04T06:07:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.348+0000 7f13dde00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f13d806df60 con 0x7f13d806fa50 2024-04-04T06:07:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.348+0000 7f13dde00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f13d80b4380 con 0x7f13d806c580 2024-04-04T06:07:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.348+0000 7f13d7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13d81590c0 0x7f13d80cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.348+0000 7f13d7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13d81590c0 0x7f13d80cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50396/0 (socket says 172.21.15.159:50396) 2024-04-04T06:07:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.348+0000 7f13d7400640 1 -- 172.21.15.159:0/3282884423 learned_addr learned my addr 172.21.15.159:0/3282884423 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.349+0000 7f13d6a00640 1 --2- 172.21.15.159:0/3282884423 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13d806c580 0x7f13d81f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.349+0000 7f13ce000640 1 --2- 172.21.15.159:0/3282884423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13d806fa50 0x7f13d81f7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.349+0000 7f13d7400640 1 -- 172.21.15.159:0/3282884423 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13d806c580 msgr2=0x7f13d81f5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.349+0000 7f13d7400640 1 --2- 172.21.15.159:0/3282884423 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13d806c580 0x7f13d81f5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.349+0000 7f13d7400640 1 -- 172.21.15.159:0/3282884423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13d806fa50 msgr2=0x7f13d81f7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.349+0000 7f13d7400640 1 --2- 172.21.15.159:0/3282884423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13d806fa50 0x7f13d81f7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.349+0000 7f13d7400640 1 -- 172.21.15.159:0/3282884423 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d8153f00 con 0x7f13d81590c0 2024-04-04T06:07:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.349+0000 7f13ce000640 1 --2- 172.21.15.159:0/3282884423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13d806fa50 0x7f13d81f7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:07:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.349+0000 7f13d7400640 1 --2- 172.21.15.159:0/3282884423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13d81590c0 0x7f13d80cdcd0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009e30 tx=0x7f13d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.349+0000 7f13cfe00640 1 -- 172.21.15.159:0/3282884423 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0018020 con 0x7f13d81590c0 2024-04-04T06:07:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.349+0000 7f13cfe00640 1 -- 172.21.15.159:0/3282884423 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d0014480 con 0x7f13d81590c0 2024-04-04T06:07:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.349+0000 7f13cfe00640 1 -- 172.21.15.159:0/3282884423 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0014870 con 0x7f13d81590c0 2024-04-04T06:07:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.350+0000 7f13dde00640 1 -- 172.21.15.159:0/3282884423 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d80ae930 con 0x7f13d81590c0 2024-04-04T06:07:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.350+0000 7f13dde00640 1 -- 172.21.15.159:0/3282884423 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d81f78b0 con 0x7f13d81590c0 2024-04-04T06:07:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.351+0000 7f13dde00640 1 -- 172.21.15.159:0/3282884423 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13d80685c0 con 0x7f13d81590c0 2024-04-04T06:07:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.352+0000 7f13cfe00640 1 -- 172.21.15.159:0/3282884423 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13d0005d10 con 0x7f13d81590c0 2024-04-04T06:07:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.353+0000 7f13cfe00640 1 --2- 172.21.15.159:0/3282884423 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f13b406c8f0 0x7f13b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.353+0000 7f13cfe00640 1 -- 172.21.15.159:0/3282884423 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f13d008ca80 con 0x7f13d81590c0 2024-04-04T06:07:56.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.354+0000 7f13d6a00640 1 --2- 172.21.15.159:0/3282884423 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f13b406c8f0 0x7f13b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:56.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.355+0000 7f13d6a00640 1 --2- 172.21.15.159:0/3282884423 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f13b406c8f0 0x7f13b406edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f13c8036000 tx=0x7f13c803b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:56.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.358+0000 7f13cfe00640 1 -- 172.21.15.159:0/3282884423 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f13d008ee20 con 0x7f13d81590c0 2024-04-04T06:07:56.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:56.599+0000 7f13dde00640 1 -- 172.21.15.159:0/3282884423 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f13d8067590 con 0x7f13d81590c0 2024-04-04T06:07:57.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.104+0000 7f13cfe00640 1 -- 172.21.15.159:0/3282884423 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) ==== 122+0+0 (secure 0 0 0) 0x7f13d0022030 con 0x7f13d81590c0 2024-04-04T06:07:57.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.108+0000 7f13dde00640 1 -- 172.21.15.159:0/3282884423 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f13b406c8f0 msgr2=0x7f13b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:57.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.108+0000 7f13dde00640 1 --2- 172.21.15.159:0/3282884423 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f13b406c8f0 0x7f13b406edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f13c8036000 tx=0x7f13c803b040 comp rx=0 tx=0).stop 2024-04-04T06:07:57.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.108+0000 7f13dde00640 1 -- 172.21.15.159:0/3282884423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13d81590c0 msgr2=0x7f13d80cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:57.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.108+0000 7f13dde00640 1 --2- 172.21.15.159:0/3282884423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13d81590c0 0x7f13d80cdcd0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009e30 tx=0x7f13d000c490 comp rx=0 tx=0).stop 2024-04-04T06:07:57.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.108+0000 7f13dde00640 1 -- 172.21.15.159:0/3282884423 shutdown_connections 2024-04-04T06:07:57.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.108+0000 7f13dde00640 1 --2- 172.21.15.159:0/3282884423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13d81590c0 0x7f13d80cdcd0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:57.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.108+0000 7f13dde00640 1 --2- 172.21.15.159:0/3282884423 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f13b406c8f0 0x7f13b406edb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:57.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.108+0000 7f13dde00640 1 --2- 172.21.15.159:0/3282884423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13d806fa50 0x7f13d81f7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:57.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.108+0000 7f13dde00640 1 --2- 172.21.15.159:0/3282884423 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13d806c580 0x7f13d81f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:57.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.108+0000 7f13dde00640 1 -- 172.21.15.159:0/3282884423 >> 172.21.15.159:0/3282884423 conn(0x7f13d8071e60 msgr2=0x7f13d8072240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:57.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.108+0000 7f13dde00640 1 -- 172.21.15.159:0/3282884423 shutdown_connections 2024-04-04T06:07:57.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.108+0000 7f13dde00640 1 -- 172.21.15.159:0/3282884423 wait complete. 2024-04-04T06:07:57.123 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-04T06:07:57.123 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:07:57.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.353+0000 7ff5e3200640 1 Processor -- start 2024-04-04T06:07:57.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.354+0000 7ff5e3200640 1 -- start start 2024-04-04T06:07:57.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.355+0000 7ff5e3200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5e40d2310 0x7ff5e40ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:57.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.355+0000 7ff5e3200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5e415e880 con 0x7ff5e40cf530 2024-04-04T06:07:57.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.355+0000 7ff5e3200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5e40d28d0 con 0x7ff5e415e4a0 2024-04-04T06:07:57.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.355+0000 7ff5e3200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e40d34e0 con 0x7ff5e40d2310 2024-04-04T06:07:57.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.355+0000 7ff5e2800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7ff5e40cf530 0x7ff5e40b73e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37044/0 (socket says 172.21.15.159:37044) 2024-04-04T06:07:57.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.355+0000 7ff5e2800640 1 -- 172.21.15.159:0/430969201 learned_addr learned my addr 172.21.15.159:0/430969201 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.356+0000 7ff5e1400640 1 --2- 172.21.15.159:0/430969201 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5e40d2310 0x7ff5e40ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.356+0000 7ff5e0a00640 1 -- 172.21.15.159:0/430969201 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3012375268 0 0) 0x7ff5e415e880 con 0x7ff5e40cf530 2024-04-04T06:07:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.356+0000 7ff5e0a00640 1 -- 172.21.15.159:0/430969201 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5c4003520 con 0x7ff5e40cf530 2024-04-04T06:07:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.356+0000 7ff5e0a00640 1 -- 172.21.15.159:0/430969201 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1036556184 0 0) 0x7ff5c4003520 con 0x7ff5e40cf530 2024-04-04T06:07:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.356+0000 7ff5e0a00640 1 -- 172.21.15.159:0/430969201 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5e40d2310 msgr2=0x7ff5e40ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.356+0000 7ff5e0a00640 1 --2- 172.21.15.159:0/430969201 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5e40d2310 0x7ff5e40ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.356+0000 7ff5e0a00640 1 -- 172.21.15.159:0/430969201 >> v1:172.21.15.178:6789/0 conn(0x7ff5e415e4a0 legacy=0x7ff5e40b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.356+0000 7ff5e0a00640 1 -- 172.21.15.159:0/430969201 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e40d2bb0 con 0x7ff5e40cf530 2024-04-04T06:07:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.356+0000 7ff5e0a00640 1 -- 172.21.15.159:0/430969201 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7ff5dc001670 con 0x7ff5e40cf530 2024-04-04T06:07:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.356+0000 7ff5e0a00640 1 -- 172.21.15.159:0/430969201 >> v1:172.21.15.159:6789/0 conn(0x7ff5e40cf530 legacy=0x7ff5e40b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.357+0000 7ff5e0a00640 1 --2- 172.21.15.159:0/430969201 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5e40d2310 0x7ff5c40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.357+0000 7ff5e0a00640 1 --2- 172.21.15.159:0/430969201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5c4003f60 0x7ff5c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.357+0000 7ff5e0a00640 1 --2- 172.21.15.159:0/430969201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5c4004d40 0x7ff5c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.357+0000 7ff5e0a00640 1 -- 172.21.15.159:0/430969201 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e40d34e0 con 0x7ff5c4003f60 2024-04-04T06:07:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.357+0000 7ff5e0a00640 1 -- 172.21.15.159:0/430969201 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c400baa0 con 0x7ff5c4004d40 2024-04-04T06:07:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.357+0000 7ff5e0a00640 1 -- 172.21.15.159:0/430969201 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c400bca0 con 0x7ff5e40d2310 2024-04-04T06:07:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.357+0000 7ff5e1400640 1 --2- 172.21.15.159:0/430969201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5c4003f60 0x7ff5c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.357+0000 7ff5e1e00640 1 --2- 172.21.15.159:0/430969201 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5e40d2310 0x7ff5c40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.357+0000 7ff5e2800640 1 --2- 172.21.15.159:0/430969201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5c4004d40 0x7ff5c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.357+0000 7ff5e1400640 1 -- 172.21.15.159:0/430969201 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5e40d2310 msgr2=0x7ff5c40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.357+0000 7ff5e1400640 1 --2- 172.21.15.159:0/430969201 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5e40d2310 0x7ff5c40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.357+0000 7ff5e1400640 1 -- 172.21.15.159:0/430969201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5c4004d40 msgr2=0x7ff5c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.357+0000 7ff5e1400640 1 --2- 172.21.15.159:0/430969201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5c4004d40 0x7ff5c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.357+0000 7ff5e1400640 1 -- 172.21.15.159:0/430969201 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c4003520 con 0x7ff5c4003f60 2024-04-04T06:07:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.357+0000 7ff5e2800640 1 --2- 172.21.15.159:0/430969201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5c4004d40 0x7ff5c40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.359+0000 7ff5e1400640 1 --2- 172.21.15.159:0/430969201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5c4003f60 0x7ff5c4004800 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc009fc0 tx=0x7ff5c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b47213fc8b8a79 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.359+0000 7ff5e0a00640 1 -- 172.21.15.159:0/430969201 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc039070 con 0x7ff5c4003f60 2024-04-04T06:07:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.359+0000 7ff5e0a00640 1 -- 172.21.15.159:0/430969201 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5cc02cc20 con 0x7ff5c4003f60 2024-04-04T06:07:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.359+0000 7ff5e0a00640 1 -- 172.21.15.159:0/430969201 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc009c10 con 0x7ff5c4003f60 2024-04-04T06:07:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.360+0000 7ff5e3200640 1 -- 172.21.15.159:0/430969201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5c4003f60 msgr2=0x7ff5c4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.360+0000 7ff5e3200640 1 --2- 172.21.15.159:0/430969201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5c4003f60 0x7ff5c4004800 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc009fc0 tx=0x7ff5c4005830 comp rx=0 tx=0).stop 2024-04-04T06:07:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.360+0000 7ff5e2800640 1 -- 172.21.15.159:0/430969201 reap_dead start 2024-04-04T06:07:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.360+0000 7ff5e3200640 1 -- 172.21.15.159:0/430969201 shutdown_connections 2024-04-04T06:07:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.360+0000 7ff5e3200640 1 -- 172.21.15.159:0/430969201 >> 172.21.15.159:0/430969201 conn(0x7ff5e40676d0 msgr2=0x7ff5e4067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.360+0000 7ff5e3200640 1 -- 172.21.15.159:0/430969201 shutdown_connections 2024-04-04T06:07:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.360+0000 7ff5e3200640 1 -- 172.21.15.159:0/430969201 wait complete. 2024-04-04T06:07:57.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.361+0000 7ff5e3200640 1 Processor -- start 2024-04-04T06:07:57.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.362+0000 7ff5e3200640 1 -- start start 2024-04-04T06:07:57.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.362+0000 7ff5e3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5c4003f60 0x7ff5e4161370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:57.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.362+0000 7ff5e3200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5e414f080 0x7ff5e415ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:57.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.363+0000 7ff5e3200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5e415f1d0 0x7ff5e415f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:57.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.363+0000 7ff5e3200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e40d3970 con 0x7ff5c4003f60 2024-04-04T06:07:57.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.363+0000 7ff5e3200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e40d3170 con 0x7ff5e415f1d0 2024-04-04T06:07:57.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.363+0000 7ff5e3200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e40ceff0 con 0x7ff5e414f080 2024-04-04T06:07:57.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.363+0000 7ff5e1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5c4003f60 0x7ff5e4161370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:57.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.363+0000 7ff5e1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5c4003f60 0x7ff5e4161370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50412/0 (socket says 172.21.15.159:50412) 2024-04-04T06:07:57.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.363+0000 7ff5e1e00640 1 -- 172.21.15.159:0/332584770 learned_addr learned my addr 172.21.15.159:0/332584770 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:57.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.363+0000 7ff5e2800640 1 --2- 172.21.15.159:0/332584770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5e415f1d0 0x7ff5e415f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:57.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.363+0000 7ff5e1e00640 1 -- 172.21.15.159:0/332584770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5e414f080 msgr2=0x7ff5e415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:57.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.363+0000 7ff5e1e00640 1 --2- 172.21.15.159:0/332584770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5e414f080 0x7ff5e415ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:57.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.363+0000 7ff5e1e00640 1 -- 172.21.15.159:0/332584770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5e415f1d0 msgr2=0x7ff5e415f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:57.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.363+0000 7ff5e1e00640 1 --2- 172.21.15.159:0/332584770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5e415f1d0 0x7ff5e415f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:57.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.363+0000 7ff5e1e00640 1 -- 172.21.15.159:0/332584770 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e41533e0 con 0x7ff5c4003f60 2024-04-04T06:07:57.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.363+0000 7ff5e2800640 1 --2- 172.21.15.159:0/332584770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5e415f1d0 0x7ff5e415f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:57.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.364+0000 7ff5e1e00640 1 --2- 172.21.15.159:0/332584770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5c4003f60 0x7ff5e4161370 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400e580 tx=0x7ff5d400ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:57.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.364+0000 7ff5da000640 1 -- 172.21.15.159:0/332584770 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d40052e0 con 0x7ff5c4003f60 2024-04-04T06:07:57.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.364+0000 7ff5da000640 1 -- 172.21.15.159:0/332584770 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d4018070 con 0x7ff5c4003f60 2024-04-04T06:07:57.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.364+0000 7ff5da000640 1 -- 172.21.15.159:0/332584770 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4003810 con 0x7ff5c4003f60 2024-04-04T06:07:57.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.364+0000 7ff5e3200640 1 -- 172.21.15.159:0/332584770 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e41536c0 con 0x7ff5c4003f60 2024-04-04T06:07:57.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.364+0000 7ff5e3200640 1 -- 172.21.15.159:0/332584770 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e414f8e0 con 0x7ff5c4003f60 2024-04-04T06:07:57.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.365+0000 7ff5e3200640 1 -- 172.21.15.159:0/332584770 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e4075940 con 0x7ff5c4003f60 2024-04-04T06:07:57.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.366+0000 7ff5da000640 1 -- 172.21.15.159:0/332584770 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5d40039b0 con 0x7ff5c4003f60 2024-04-04T06:07:57.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.368+0000 7ff5da000640 1 --2- 172.21.15.159:0/332584770 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff5c006c920 0x7ff5c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:57.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.368+0000 7ff5da000640 1 -- 172.21.15.159:0/332584770 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7ff5d408b840 con 0x7ff5c4003f60 2024-04-04T06:07:57.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.368+0000 7ff5e1400640 1 --2- 172.21.15.159:0/332584770 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff5c006c920 0x7ff5c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:57.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.369+0000 7ff5e1400640 1 --2- 172.21.15.159:0/332584770 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff5c006c920 0x7ff5c006ede0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc036000 tx=0x7ff5cc02e070 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:57.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.372+0000 7ff5da000640 1 -- 172.21.15.159:0/332584770 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff5d4059be0 con 0x7ff5c4003f60 2024-04-04T06:07:57.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.609+0000 7ff5e3200640 1 -- 172.21.15.159:0/332584770 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5e415a690 con 0x7ff5c4003f60 2024-04-04T06:07:57.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.610+0000 7ff5da000640 1 -- 172.21.15.159:0/332584770 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5193 (secure 0 0 0) 0x7ff5d40594e0 con 0x7ff5c4003f60 2024-04-04T06:07:57.604 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:57.604 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3765578393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3765578393},{"type":"v1","addr":"172.21.15.178:6835","nonce":3765578393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3260412913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3260412913},{"type":"v1","addr":"172.21.15.178:6837","nonce":3260412913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3149916718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3149916718},{"type":"v1","addr":"172.21.15.159:6833","nonce":3149916718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:07:53.523212+0000","modified":"2024-04-04T06:07:56.601750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6835/1867245367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1867245367},{"type":"v1","addr":"172.21.15.159:6835","nonce":1867245367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5589,"qdb_cluster":[5589]},"id":3}]} 2024-04-04T06:07:57.604 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 25 2024-04-04T06:07:57.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.612+0000 7ff5e3200640 1 -- 172.21.15.159:0/332584770 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff5c006c920 msgr2=0x7ff5c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:57.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.612+0000 7ff5e3200640 1 --2- 172.21.15.159:0/332584770 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff5c006c920 0x7ff5c006ede0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc036000 tx=0x7ff5cc02e070 comp rx=0 tx=0).stop 2024-04-04T06:07:57.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.612+0000 7ff5e3200640 1 -- 172.21.15.159:0/332584770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5c4003f60 msgr2=0x7ff5e4161370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:57.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.612+0000 7ff5e3200640 1 --2- 172.21.15.159:0/332584770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5c4003f60 0x7ff5e4161370 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400e580 tx=0x7ff5d400ea40 comp rx=0 tx=0).stop 2024-04-04T06:07:57.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.612+0000 7ff5e3200640 1 -- 172.21.15.159:0/332584770 shutdown_connections 2024-04-04T06:07:57.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.612+0000 7ff5e3200640 1 --2- 172.21.15.159:0/332584770 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff5c006c920 0x7ff5c006ede0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:57.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.612+0000 7ff5e3200640 1 --2- 172.21.15.159:0/332584770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5e415f1d0 0x7ff5e415f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:57.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.612+0000 7ff5e3200640 1 --2- 172.21.15.159:0/332584770 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5e414f080 0x7ff5e415ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:57.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.612+0000 7ff5e3200640 1 --2- 172.21.15.159:0/332584770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5c4003f60 0x7ff5e4161370 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:57.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.612+0000 7ff5e3200640 1 -- 172.21.15.159:0/332584770 >> 172.21.15.159:0/332584770 conn(0x7ff5e40676d0 msgr2=0x7ff5e415cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:57.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.613+0000 7ff5e3200640 1 -- 172.21.15.159:0/332584770 shutdown_connections 2024-04-04T06:07:57.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.613+0000 7ff5e3200640 1 -- 172.21.15.159:0/332584770 wait complete. 2024-04-04T06:07:57.626 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:07:57.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.833+0000 7f9738600640 1 Processor -- start 2024-04-04T06:07:57.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.834+0000 7f9738600640 1 -- start start 2024-04-04T06:07:57.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.835+0000 7f9738600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f97300af830 0x7f9730073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:57.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.835+0000 7f9738600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f97300d0650 0x7f97300d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:57.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.835+0000 7f9738600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f973015fe80 con 0x7f973015c390 2024-04-04T06:07:57.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.835+0000 7f9738600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f973015c770 con 0x7f97300d0650 2024-04-04T06:07:57.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.835+0000 7f9738600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f97300d0fc0 con 0x7f97300af830 2024-04-04T06:07:57.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.835+0000 7f9736600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f97300d0650 0x7f97300d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:57.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.835+0000 7f9736600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f97300d0650 0x7f97300d0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50424/0 (socket says 172.21.15.159:50424) 2024-04-04T06:07:57.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.835+0000 7f9736600640 1 -- 172.21.15.159:0/3736589225 learned_addr learned my addr 172.21.15.159:0/3736589225 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:57.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.836+0000 7f9735200640 1 --2- 172.21.15.159:0/3736589225 >> v2:172.21.15.178:3300/0 conn(0x7f97300af830 0x7f9730073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:57.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.836+0000 7f9736600640 1 -- 172.21.15.159:0/3736589225 >> v2:172.21.15.178:3300/0 conn(0x7f97300af830 msgr2=0x7f9730073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:57.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.836+0000 7f9736600640 1 --2- 172.21.15.159:0/3736589225 >> v2:172.21.15.178:3300/0 conn(0x7f97300af830 0x7f9730073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:57.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.836+0000 7f9736600640 1 -- 172.21.15.159:0/3736589225 >> v1:172.21.15.178:6789/0 conn(0x7f973015c390 legacy=0x7f9730160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:57.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.836+0000 7f9736600640 1 -- 172.21.15.159:0/3736589225 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97300ad120 con 0x7f97300d0650 2024-04-04T06:07:57.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.838+0000 7f9736600640 1 --2- 172.21.15.159:0/3736589225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97300d0650 0x7f97300d0a80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f973006eaa0 tx=0x7f972802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e24e1849610a005 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:57.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.838+0000 7f972fe00640 1 -- 172.21.15.159:0/3736589225 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972802cb20 con 0x7f97300d0650 2024-04-04T06:07:57.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.838+0000 7f972fe00640 1 -- 172.21.15.159:0/3736589225 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f972802ccc0 con 0x7f97300d0650 2024-04-04T06:07:57.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.838+0000 7f972fe00640 1 -- 172.21.15.159:0/3736589225 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728034610 con 0x7f97300d0650 2024-04-04T06:07:57.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.838+0000 7f9738600640 1 -- 172.21.15.159:0/3736589225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97300d0650 msgr2=0x7f97300d0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:57.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.838+0000 7f9738600640 1 --2- 172.21.15.159:0/3736589225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97300d0650 0x7f97300d0a80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f973006eaa0 tx=0x7f972802bf80 comp rx=0 tx=0).stop 2024-04-04T06:07:57.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.839+0000 7f9738600640 1 -- 172.21.15.159:0/3736589225 shutdown_connections 2024-04-04T06:07:57.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.839+0000 7f9738600640 1 --2- 172.21.15.159:0/3736589225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97300d0650 0x7f97300d0a80 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:57.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.839+0000 7f9738600640 1 --2- 172.21.15.159:0/3736589225 >> v2:172.21.15.178:3300/0 conn(0x7f97300af830 0x7f9730073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:57.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.839+0000 7f9738600640 1 -- 172.21.15.159:0/3736589225 >> 172.21.15.159:0/3736589225 conn(0x7f973006b990 msgr2=0x7f973006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:57.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.839+0000 7f9738600640 1 -- 172.21.15.159:0/3736589225 shutdown_connections 2024-04-04T06:07:57.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.839+0000 7f9738600640 1 -- 172.21.15.159:0/3736589225 wait complete. 2024-04-04T06:07:57.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.840+0000 7f9738600640 1 Processor -- start 2024-04-04T06:07:57.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.841+0000 7f9738600640 1 -- start start 2024-04-04T06:07:57.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.841+0000 7f9738600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97300af830 0x7f973015e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:57.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.842+0000 7f9738600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97300d0650 0x7f97301455d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:57.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.842+0000 7f9738600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f973015c390 0x7f973014e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:57.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.842+0000 7f9738600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9730160120 con 0x7f973015c390 2024-04-04T06:07:57.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.842+0000 7f9738600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97300ad6e0 con 0x7f97300d0650 2024-04-04T06:07:57.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.842+0000 7f9738600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9730160d50 con 0x7f97300af830 2024-04-04T06:07:57.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.842+0000 7f9736600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f973015c390 0x7f973014e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:57.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.842+0000 7f9736600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f973015c390 0x7f973014e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50438/0 (socket says 172.21.15.159:50438) 2024-04-04T06:07:57.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.842+0000 7f9736600640 1 -- 172.21.15.159:0/428102251 learned_addr learned my addr 172.21.15.159:0/428102251 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:57.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.842+0000 7f9735200640 1 --2- 172.21.15.159:0/428102251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97300d0650 0x7f97301455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:57.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.842+0000 7f9736600640 1 -- 172.21.15.159:0/428102251 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97300af830 msgr2=0x7f973015e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:57.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.842+0000 7f9735c00640 1 --2- 172.21.15.159:0/428102251 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97300af830 0x7f973015e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:57.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.842+0000 7f9736600640 1 --2- 172.21.15.159:0/428102251 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97300af830 0x7f973015e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:57.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.842+0000 7f9736600640 1 -- 172.21.15.159:0/428102251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97300d0650 msgr2=0x7f97301455d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:57.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.842+0000 7f9736600640 1 --2- 172.21.15.159:0/428102251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97300d0650 0x7f97301455d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:57.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.842+0000 7f9736600640 1 -- 172.21.15.159:0/428102251 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973015efc0 con 0x7f973015c390 2024-04-04T06:07:57.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.842+0000 7f9735200640 1 --2- 172.21.15.159:0/428102251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97300d0650 0x7f97301455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:57.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.842+0000 7f9735c00640 1 --2- 172.21.15.159:0/428102251 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97300af830 0x7f973015e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:57.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.843+0000 7f9736600640 1 --2- 172.21.15.159:0/428102251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f973015c390 0x7f973014e480 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9728009e50 tx=0x7f9728004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:57.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.843+0000 7f972d600640 1 -- 172.21.15.159:0/428102251 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728043020 con 0x7f973015c390 2024-04-04T06:07:57.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.843+0000 7f972d600640 1 -- 172.21.15.159:0/428102251 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f972803e450 con 0x7f973015c390 2024-04-04T06:07:57.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.843+0000 7f9738600640 1 -- 172.21.15.159:0/428102251 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97300b1940 con 0x7f973015c390 2024-04-04T06:07:57.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.843+0000 7f972d600640 1 -- 172.21.15.159:0/428102251 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728009510 con 0x7f973015c390 2024-04-04T06:07:57.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.843+0000 7f9738600640 1 -- 172.21.15.159:0/428102251 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97300b1f00 con 0x7f973015c390 2024-04-04T06:07:57.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.844+0000 7f9738600640 1 -- 172.21.15.159:0/428102251 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97300685c0 con 0x7f973015c390 2024-04-04T06:07:57.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.845+0000 7f972d600640 1 -- 172.21.15.159:0/428102251 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9728046020 con 0x7f973015c390 2024-04-04T06:07:57.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.846+0000 7f972d600640 1 --2- 172.21.15.159:0/428102251 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f971006c8f0 0x7f971006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:57.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.846+0000 7f972d600640 1 -- 172.21.15.159:0/428102251 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f97280ae3b0 con 0x7f973015c390 2024-04-04T06:07:57.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.847+0000 7f9735c00640 1 --2- 172.21.15.159:0/428102251 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f971006c8f0 0x7f971006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:57.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.848+0000 7f9735c00640 1 --2- 172.21.15.159:0/428102251 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f971006c8f0 0x7f971006edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9724000f60 tx=0x7f9724009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:57.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:57.851+0000 7f972d600640 1 -- 172.21.15.159:0/428102251 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f972804d030 con 0x7f973015c390 2024-04-04T06:07:58.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.050+0000 7f9738600640 1 -- 172.21.15.159:0/428102251 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f973006f1b0 con 0x7f973015c390 2024-04-04T06:07:58.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.052+0000 7f972d600640 1 -- 172.21.15.159:0/428102251 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16625 (secure 0 0 0) 0x7f972807c7d0 con 0x7f973015c390 2024-04-04T06:07:58.046 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:58.047 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":46,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:07:55.072304+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:07:51.357366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-04T06:07:52.495884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:07:58.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.054+0000 7f9738600640 1 -- 172.21.15.159:0/428102251 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f971006c8f0 msgr2=0x7f971006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:58.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.055+0000 7f9738600640 1 --2- 172.21.15.159:0/428102251 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f971006c8f0 0x7f971006edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9724000f60 tx=0x7f9724009040 comp rx=0 tx=0).stop 2024-04-04T06:07:58.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.055+0000 7f9738600640 1 -- 172.21.15.159:0/428102251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f973015c390 msgr2=0x7f973014e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:58.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.055+0000 7f9738600640 1 --2- 172.21.15.159:0/428102251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f973015c390 0x7f973014e480 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9728009e50 tx=0x7f9728004a40 comp rx=0 tx=0).stop 2024-04-04T06:07:58.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.055+0000 7f9738600640 1 -- 172.21.15.159:0/428102251 shutdown_connections 2024-04-04T06:07:58.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.055+0000 7f9738600640 1 --2- 172.21.15.159:0/428102251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f973015c390 0x7f973014e480 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:58.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.055+0000 7f9738600640 1 --2- 172.21.15.159:0/428102251 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f971006c8f0 0x7f971006edb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:58.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.055+0000 7f9738600640 1 --2- 172.21.15.159:0/428102251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97300d0650 0x7f97301455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:58.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.055+0000 7f9738600640 1 --2- 172.21.15.159:0/428102251 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97300af830 0x7f973015e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:58.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.055+0000 7f9738600640 1 -- 172.21.15.159:0/428102251 >> 172.21.15.159:0/428102251 conn(0x7f973006b990 msgr2=0x7f973015aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:58.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.055+0000 7f9738600640 1 -- 172.21.15.159:0/428102251 shutdown_connections 2024-04-04T06:07:58.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.055+0000 7f9738600640 1 -- 172.21.15.159:0/428102251 wait complete. 2024-04-04T06:07:58.067 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-04T06:07:58.067 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:07:58.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.268+0000 7fbaaf200640 1 Processor -- start 2024-04-04T06:07:58.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.268+0000 7fbaaf200640 1 -- start start 2024-04-04T06:07:58.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.270+0000 7fbaaf200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbaa815f900 0x7fbaa815c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:58.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.270+0000 7fbaaf200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbaa815da60 0x7fbaa80cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:58.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.270+0000 7fbaaf200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaa815cd90 con 0x7fbaa815fe90 2024-04-04T06:07:58.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.270+0000 7fbaaf200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fbaa815d060 con 0x7fbaa815f900 2024-04-04T06:07:58.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.270+0000 7fbaaf200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa815d340 con 0x7fbaa815da60 2024-04-04T06:07:58.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.270+0000 7fbaad200640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fbaa815fe90 0x7fbaa80ce8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37056/0 (socket says 172.21.15.159:37056) 2024-04-04T06:07:58.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.270+0000 7fbaad200640 1 -- 172.21.15.159:0/1369421121 learned_addr learned my addr 172.21.15.159:0/1369421121 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:58.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.271+0000 7fbaa7e00640 1 --2- 172.21.15.159:0/1369421121 >> v2:172.21.15.178:3300/0 conn(0x7fbaa815f900 0x7fbaa815c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:58.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.271+0000 7fba9f400640 1 --2- 172.21.15.159:0/1369421121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbaa815da60 0x7fbaa80cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:58.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.271+0000 7fbaa7400640 1 -- 172.21.15.159:0/1369421121 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 792286811 0 0) 0x7fbaa815cd90 con 0x7fbaa815fe90 2024-04-04T06:07:58.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.271+0000 7fbaa7400640 1 -- 172.21.15.159:0/1369421121 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba8c003520 con 0x7fbaa815fe90 2024-04-04T06:07:58.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.271+0000 7fbaa7400640 1 -- 172.21.15.159:0/1369421121 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 175513731 0 0) 0x7fba8c003520 con 0x7fbaa815fe90 2024-04-04T06:07:58.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.271+0000 7fbaa7400640 1 -- 172.21.15.159:0/1369421121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbaa815da60 msgr2=0x7fbaa80cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:58.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.271+0000 7fbaa7400640 1 --2- 172.21.15.159:0/1369421121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbaa815da60 0x7fbaa80cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:58.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.271+0000 7fbaa7400640 1 -- 172.21.15.159:0/1369421121 >> v2:172.21.15.178:3300/0 conn(0x7fbaa815f900 msgr2=0x7fbaa815c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:58.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.271+0000 7fbaa7400640 1 --2- 172.21.15.159:0/1369421121 >> v2:172.21.15.178:3300/0 conn(0x7fbaa815f900 0x7fbaa815c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:58.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.271+0000 7fbaa7400640 1 -- 172.21.15.159:0/1369421121 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa815d620 con 0x7fbaa815fe90 2024-04-04T06:07:58.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.271+0000 7fbaa7400640 1 -- 172.21.15.159:0/1369421121 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fbaa0001670 con 0x7fbaa815fe90 2024-04-04T06:07:58.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.271+0000 7fbaa7400640 1 -- 172.21.15.159:0/1369421121 >> v1:172.21.15.159:6789/0 conn(0x7fbaa815fe90 legacy=0x7fbaa80ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:58.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.271+0000 7fbaa7400640 1 --2- 172.21.15.159:0/1369421121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba8c0039b0 0x7fba8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:58.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.271+0000 7fbaa7400640 1 --2- 172.21.15.159:0/1369421121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba8c004800 0x7fba8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:58.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.271+0000 7fbaa7400640 1 --2- 172.21.15.159:0/1369421121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbaa815da60 0x7fba8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:58.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.271+0000 7fbaa7400640 1 -- 172.21.15.159:0/1369421121 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa815d060 con 0x7fba8c0039b0 2024-04-04T06:07:58.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.271+0000 7fbaa7400640 1 -- 172.21.15.159:0/1369421121 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa815d340 con 0x7fba8c004800 2024-04-04T06:07:58.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.271+0000 7fbaa7400640 1 -- 172.21.15.159:0/1369421121 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fba8c00b9c0 con 0x7fbaa815da60 2024-04-04T06:07:58.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.272+0000 7fbaa7e00640 1 --2- 172.21.15.159:0/1369421121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba8c0039b0 0x7fba8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:58.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.272+0000 7fbaa7e00640 1 -- 172.21.15.159:0/1369421121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbaa815da60 msgr2=0x7fba8c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:58.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.272+0000 7fba9f400640 1 --2- 172.21.15.159:0/1369421121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba8c004800 0x7fba8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:58.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.272+0000 7fbaa7e00640 1 --2- 172.21.15.159:0/1369421121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbaa815da60 0x7fba8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:58.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.272+0000 7fbaa7e00640 1 -- 172.21.15.159:0/1369421121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba8c004800 msgr2=0x7fba8c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:58.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.272+0000 7fbaa7e00640 1 --2- 172.21.15.159:0/1369421121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba8c004800 0x7fba8c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:58.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.272+0000 7fbaa7e00640 1 -- 172.21.15.159:0/1369421121 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8c003520 con 0x7fba8c0039b0 2024-04-04T06:07:58.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.274+0000 7fbaa7e00640 1 --2- 172.21.15.159:0/1369421121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba8c0039b0 0x7fba8c003dd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fba98009fc0 tx=0x7fba8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3983190eb5b8a9b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:58.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.274+0000 7fbaa7400640 1 -- 172.21.15.159:0/1369421121 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98039070 con 0x7fba8c0039b0 2024-04-04T06:07:58.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.274+0000 7fbaa7400640 1 -- 172.21.15.159:0/1369421121 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba9802cc60 con 0x7fba8c0039b0 2024-04-04T06:07:58.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.274+0000 7fbaa7400640 1 -- 172.21.15.159:0/1369421121 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98009c10 con 0x7fba8c0039b0 2024-04-04T06:07:58.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.274+0000 7fbaaf200640 1 -- 172.21.15.159:0/1369421121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba8c0039b0 msgr2=0x7fba8c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:58.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.274+0000 7fbaaf200640 1 --2- 172.21.15.159:0/1369421121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba8c0039b0 0x7fba8c003dd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fba98009fc0 tx=0x7fba8c005830 comp rx=0 tx=0).stop 2024-04-04T06:07:58.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.275+0000 7fbaad200640 1 -- 172.21.15.159:0/1369421121 reap_dead start 2024-04-04T06:07:58.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.275+0000 7fbaaf200640 1 -- 172.21.15.159:0/1369421121 shutdown_connections 2024-04-04T06:07:58.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.275+0000 7fbaaf200640 1 -- 172.21.15.159:0/1369421121 >> 172.21.15.159:0/1369421121 conn(0x7fbaa80b3e90 msgr2=0x7fbaa80b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:58.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.275+0000 7fbaaf200640 1 -- 172.21.15.159:0/1369421121 shutdown_connections 2024-04-04T06:07:58.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.275+0000 7fbaaf200640 1 -- 172.21.15.159:0/1369421121 wait complete. 2024-04-04T06:07:58.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.277+0000 7fbaaf200640 1 Processor -- start 2024-04-04T06:07:58.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.277+0000 7fbaaf200640 1 -- start start 2024-04-04T06:07:58.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.278+0000 7fbaaf200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba8c0039b0 0x7fbaa80c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:58.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.278+0000 7fbaaf200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbaa8157620 0x7fbaa81567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:58.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.278+0000 7fbaaf200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbaa8156d10 0x7fbaa80b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:58.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.278+0000 7fbaaf200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa80cf160 con 0x7fbaa8157620 2024-04-04T06:07:58.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.278+0000 7fbaaf200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa80cefe0 con 0x7fba8c0039b0 2024-04-04T06:07:58.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.278+0000 7fbaaf200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa80cf2e0 con 0x7fbaa8156d10 2024-04-04T06:07:58.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.279+0000 7fba9f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbaa8157620 0x7fbaa81567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:58.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.279+0000 7fba9f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbaa8157620 0x7fbaa81567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50452/0 (socket says 172.21.15.159:50452) 2024-04-04T06:07:58.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.279+0000 7fba9f400640 1 -- 172.21.15.159:0/890844392 learned_addr learned my addr 172.21.15.159:0/890844392 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:58.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.279+0000 7fbaa7e00640 1 --2- 172.21.15.159:0/890844392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba8c0039b0 0x7fbaa80c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:58.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.279+0000 7fbaad200640 1 --2- 172.21.15.159:0/890844392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbaa8156d10 0x7fbaa80b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:58.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.279+0000 7fba9f400640 1 -- 172.21.15.159:0/890844392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbaa8156d10 msgr2=0x7fbaa80b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:58.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.279+0000 7fba9f400640 1 --2- 172.21.15.159:0/890844392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbaa8156d10 0x7fbaa80b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:58.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.279+0000 7fba9f400640 1 -- 172.21.15.159:0/890844392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba8c0039b0 msgr2=0x7fbaa80c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:58.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.279+0000 7fba9f400640 1 --2- 172.21.15.159:0/890844392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba8c0039b0 0x7fbaa80c9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:58.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.279+0000 7fba9f400640 1 -- 172.21.15.159:0/890844392 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa80b5300 con 0x7fbaa8157620 2024-04-04T06:07:58.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.279+0000 7fbaad200640 1 --2- 172.21.15.159:0/890844392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbaa8156d10 0x7fbaa80b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:58.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.280+0000 7fba9f400640 1 --2- 172.21.15.159:0/890844392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbaa8157620 0x7fbaa81567d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fba90010860 tx=0x7fba90010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:58.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.280+0000 7fbaa4c00640 1 -- 172.21.15.159:0/890844392 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9000f040 con 0x7fbaa8157620 2024-04-04T06:07:58.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.280+0000 7fbaa4c00640 1 -- 172.21.15.159:0/890844392 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba9000eb00 con 0x7fbaa8157620 2024-04-04T06:07:58.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.280+0000 7fbaa4c00640 1 -- 172.21.15.159:0/890844392 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9001c400 con 0x7fbaa8157620 2024-04-04T06:07:58.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.280+0000 7fbaaf200640 1 -- 172.21.15.159:0/890844392 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa80b55e0 con 0x7fbaa8157620 2024-04-04T06:07:58.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.280+0000 7fbaaf200640 1 -- 172.21.15.159:0/890844392 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa81520a0 con 0x7fbaa8157620 2024-04-04T06:07:58.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.281+0000 7fbaaf200640 1 -- 172.21.15.159:0/890844392 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaa80b57b0 con 0x7fbaa8157620 2024-04-04T06:07:58.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.283+0000 7fbaa4c00640 1 -- 172.21.15.159:0/890844392 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba9001c5a0 con 0x7fbaa8157620 2024-04-04T06:07:58.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.284+0000 7fbaa4c00640 1 --2- 172.21.15.159:0/890844392 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fba8406c8f0 0x7fba8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:58.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.284+0000 7fbaa4c00640 1 -- 172.21.15.159:0/890844392 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fba9008c200 con 0x7fbaa8157620 2024-04-04T06:07:58.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.285+0000 7fbaa7e00640 1 --2- 172.21.15.159:0/890844392 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fba8406c8f0 0x7fba8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:58.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.286+0000 7fbaa7e00640 1 --2- 172.21.15.159:0/890844392 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fba8406c8f0 0x7fba8406edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fba98036000 tx=0x7fba9802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:58.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.289+0000 7fbaa4c00640 1 -- 172.21.15.159:0/890844392 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fba90018030 con 0x7fbaa8157620 2024-04-04T06:07:58.444 INFO:tasks.ceph.mds.a.smithi159.stderr:2024-04-04T06:07:58.450+0000 7f0f66600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:07:58.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.580+0000 7fbaaf200640 1 -- 172.21.15.159:0/890844392 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbaa8068cd0 con 0x7fba8406c8f0 2024-04-04T06:07:58.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.599+0000 7fbaa4c00640 1 -- 172.21.15.159:0/890844392 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240775 (secure 0 0 0) 0x7fbaa8068cd0 con 0x7fba8406c8f0 2024-04-04T06:07:58.596 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:58.597 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:07:58.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.604+0000 7fbaaf200640 1 -- 172.21.15.159:0/890844392 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fba8406c8f0 msgr2=0x7fba8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:58.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.604+0000 7fbaaf200640 1 --2- 172.21.15.159:0/890844392 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fba8406c8f0 0x7fba8406edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fba98036000 tx=0x7fba9802e000 comp rx=0 tx=0).stop 2024-04-04T06:07:58.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.604+0000 7fbaaf200640 1 -- 172.21.15.159:0/890844392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbaa8157620 msgr2=0x7fbaa81567d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:58.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.604+0000 7fbaaf200640 1 --2- 172.21.15.159:0/890844392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbaa8157620 0x7fbaa81567d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fba90010860 tx=0x7fba90010d20 comp rx=0 tx=0).stop 2024-04-04T06:07:58.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.604+0000 7fbaaf200640 1 -- 172.21.15.159:0/890844392 shutdown_connections 2024-04-04T06:07:58.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.604+0000 7fbaaf200640 1 --2- 172.21.15.159:0/890844392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbaa8156d10 0x7fbaa80b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:58.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.604+0000 7fbaaf200640 1 --2- 172.21.15.159:0/890844392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbaa8157620 0x7fbaa81567d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:58.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.604+0000 7fbaaf200640 1 --2- 172.21.15.159:0/890844392 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fba8406c8f0 0x7fba8406edb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:58.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.604+0000 7fbaaf200640 1 --2- 172.21.15.159:0/890844392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba8c0039b0 0x7fbaa80c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:58.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.604+0000 7fbaaf200640 1 -- 172.21.15.159:0/890844392 >> 172.21.15.159:0/890844392 conn(0x7fbaa80b3e90 msgr2=0x7fbaa8158230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:58.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.604+0000 7fbaaf200640 1 -- 172.21.15.159:0/890844392 shutdown_connections 2024-04-04T06:07:58.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.604+0000 7fbaaf200640 1 -- 172.21.15.159:0/890844392 wait complete. 2024-04-04T06:07:58.616 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-04-04T06:07:56.753415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590438,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":166,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":237868,"kb_used_data":5036,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736852,"statfs":{"total":773094113280,"available":772850536448,"internally_reserved":0,"allocated":5156864,"data_stored":2761132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.726465"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.088901+0000","last_change":"2024-04-04T06:07:54.086343+0000","last_active":"2024-04-04T06:07:54.088901+0000","last_peered":"2024-04-04T06:07:54.088901+0000","last_clean":"2024-04-04T06:07:54.088901+0000","last_became_active":"2024-04-04T06:07:54.086058+0000","last_became_peered":"2024-04-04T06:07:54.086058+0000","last_unstale":"2024-04-04T06:07:54.088901+0000","last_undegraded":"2024-04-04T06:07:54.088901+0000","last_fullsized":"2024-04-04T06:07:54.088901+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:05:58.618597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089138+0000","last_change":"2024-04-04T06:07:54.098960+0000","last_active":"2024-04-04T06:07:55.089138+0000","last_peered":"2024-04-04T06:07:55.089138+0000","last_clean":"2024-04-04T06:07:55.089138+0000","last_became_active":"2024-04-04T06:07:54.098596+0000","last_became_peered":"2024-04-04T06:07:54.098596+0000","last_unstale":"2024-04-04T06:07:55.089138+0000","last_undegraded":"2024-04-04T06:07:55.089138+0000","last_fullsized":"2024-04-04T06:07:55.089138+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:21:40.197464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.088621+0000","last_change":"2024-04-04T06:07:54.086509+0000","last_active":"2024-04-04T06:07:54.088621+0000","last_peered":"2024-04-04T06:07:54.088621+0000","last_clean":"2024-04-04T06:07:54.088621+0000","last_became_active":"2024-04-04T06:07:54.085418+0000","last_became_peered":"2024-04-04T06:07:54.085418+0000","last_unstale":"2024-04-04T06:07:54.088621+0000","last_undegraded":"2024-04-04T06:07:54.088621+0000","last_fullsized":"2024-04-04T06:07:54.088621+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:55:34.902884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.097046+0000","last_change":"2024-04-04T06:07:54.093793+0000","last_active":"2024-04-04T06:07:54.097046+0000","last_peered":"2024-04-04T06:07:54.097046+0000","last_clean":"2024-04-04T06:07:54.097046+0000","last_became_active":"2024-04-04T06:07:54.090439+0000","last_became_peered":"2024-04-04T06:07:54.090439+0000","last_unstale":"2024-04-04T06:07:54.097046+0000","last_undegraded":"2024-04-04T06:07:54.097046+0000","last_fullsized":"2024-04-04T06:07:54.097046+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:23:47.563294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089965+0000","last_change":"2024-04-04T06:07:54.094405+0000","last_active":"2024-04-04T06:07:55.089965+0000","last_peered":"2024-04-04T06:07:55.089965+0000","last_clean":"2024-04-04T06:07:55.089965+0000","last_became_active":"2024-04-04T06:07:54.093952+0000","last_became_peered":"2024-04-04T06:07:54.093952+0000","last_unstale":"2024-04-04T06:07:55.089965+0000","last_undegraded":"2024-04-04T06:07:55.089965+0000","last_fullsized":"2024-04-04T06:07:55.089965+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:51:38.648784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089301+0000","last_change":"2024-04-04T06:07:54.114967+0000","last_active":"2024-04-04T06:07:55.089301+0000","last_peered":"2024-04-04T06:07:55.089301+0000","last_clean":"2024-04-04T06:07:55.089301+0000","last_became_active":"2024-04-04T06:07:54.114621+0000","last_became_peered":"2024-04-04T06:07:54.114621+0000","last_unstale":"2024-04-04T06:07:55.089301+0000","last_undegraded":"2024-04-04T06:07:55.089301+0000","last_fullsized":"2024-04-04T06:07:55.089301+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:10:39.176046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089328+0000","last_change":"2024-04-04T06:07:54.090360+0000","last_active":"2024-04-04T06:07:55.089328+0000","last_peered":"2024-04-04T06:07:55.089328+0000","last_clean":"2024-04-04T06:07:55.089328+0000","last_became_active":"2024-04-04T06:07:54.090063+0000","last_became_peered":"2024-04-04T06:07:54.090063+0000","last_unstale":"2024-04-04T06:07:55.089328+0000","last_undegraded":"2024-04-04T06:07:55.089328+0000","last_fullsized":"2024-04-04T06:07:55.089328+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:28:42.604702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.097024+0000","last_change":"2024-04-04T06:07:54.092448+0000","last_active":"2024-04-04T06:07:54.097024+0000","last_peered":"2024-04-04T06:07:54.097024+0000","last_clean":"2024-04-04T06:07:54.097024+0000","last_became_active":"2024-04-04T06:07:54.092129+0000","last_became_peered":"2024-04-04T06:07:54.092129+0000","last_unstale":"2024-04-04T06:07:54.097024+0000","last_undegraded":"2024-04-04T06:07:54.097024+0000","last_fullsized":"2024-04-04T06:07:54.097024+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:14:41.451368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.088939+0000","last_change":"2024-04-04T06:07:54.096978+0000","last_active":"2024-04-04T06:07:55.088939+0000","last_peered":"2024-04-04T06:07:55.088939+0000","last_clean":"2024-04-04T06:07:55.088939+0000","last_became_active":"2024-04-04T06:07:54.095879+0000","last_became_peered":"2024-04-04T06:07:54.095879+0000","last_unstale":"2024-04-04T06:07:55.088939+0000","last_undegraded":"2024-04-04T06:07:55.088939+0000","last_fullsized":"2024-04-04T06:07:55.088939+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:23:47.563294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089957+0000","last_change":"2024-04-04T06:07:53.062446+0000","last_active":"2024-04-04T06:07:55.089957+0000","last_peered":"2024-04-04T06:07:55.089957+0000","last_clean":"2024-04-04T06:07:55.089957+0000","last_became_active":"2024-04-04T06:07:53.062148+0000","last_became_peered":"2024-04-04T06:07:53.062148+0000","last_unstale":"2024-04-04T06:07:55.089957+0000","last_undegraded":"2024-04-04T06:07:55.089957+0000","last_fullsized":"2024-04-04T06:07:55.089957+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:21:53.730124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.087355+0000","last_change":"2024-04-04T06:07:54.083643+0000","last_active":"2024-04-04T06:07:54.087355+0000","last_peered":"2024-04-04T06:07:54.087355+0000","last_clean":"2024-04-04T06:07:54.087355+0000","last_became_active":"2024-04-04T06:07:54.083349+0000","last_became_peered":"2024-04-04T06:07:54.083349+0000","last_unstale":"2024-04-04T06:07:54.087355+0000","last_undegraded":"2024-04-04T06:07:54.087355+0000","last_fullsized":"2024-04-04T06:07:54.087355+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:24:27.790884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089388+0000","last_change":"2024-04-04T06:07:54.099487+0000","last_active":"2024-04-04T06:07:55.089388+0000","last_peered":"2024-04-04T06:07:55.089388+0000","last_clean":"2024-04-04T06:07:55.089388+0000","last_became_active":"2024-04-04T06:07:54.096608+0000","last_became_peered":"2024-04-04T06:07:54.096608+0000","last_unstale":"2024-04-04T06:07:55.089388+0000","last_undegraded":"2024-04-04T06:07:55.089388+0000","last_fullsized":"2024-04-04T06:07:55.089388+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:07:45.530166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089350+0000","last_change":"2024-04-04T06:07:53.067674+0000","last_active":"2024-04-04T06:07:55.089350+0000","last_peered":"2024-04-04T06:07:55.089350+0000","last_clean":"2024-04-04T06:07:55.089350+0000","last_became_active":"2024-04-04T06:07:53.067324+0000","last_became_peered":"2024-04-04T06:07:53.067324+0000","last_unstale":"2024-04-04T06:07:55.089350+0000","last_undegraded":"2024-04-04T06:07:55.089350+0000","last_fullsized":"2024-04-04T06:07:55.089350+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:24:58.400986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.071716+0000","last_change":"2024-04-04T06:07:53.065862+0000","last_active":"2024-04-04T06:07:54.071716+0000","last_peered":"2024-04-04T06:07:54.071716+0000","last_clean":"2024-04-04T06:07:54.071716+0000","last_became_active":"2024-04-04T06:07:53.065576+0000","last_became_peered":"2024-04-04T06:07:53.065576+0000","last_unstale":"2024-04-04T06:07:54.071716+0000","last_undegraded":"2024-04-04T06:07:54.071716+0000","last_fullsized":"2024-04-04T06:07:54.071716+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:30:35.808232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089594+0000","last_change":"2024-04-04T06:07:54.110939+0000","last_active":"2024-04-04T06:07:55.089594+0000","last_peered":"2024-04-04T06:07:55.089594+0000","last_clean":"2024-04-04T06:07:55.089594+0000","last_became_active":"2024-04-04T06:07:54.110586+0000","last_became_peered":"2024-04-04T06:07:54.110586+0000","last_unstale":"2024-04-04T06:07:55.089594+0000","last_undegraded":"2024-04-04T06:07:55.089594+0000","last_fullsized":"2024-04-04T06:07:55.089594+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:42:04.674995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.090022+0000","last_change":"2024-04-04T06:07:53.061916+0000","last_active":"2024-04-04T06:07:55.090022+0000","last_peered":"2024-04-04T06:07:55.090022+0000","last_clean":"2024-04-04T06:07:55.090022+0000","last_became_active":"2024-04-04T06:07:53.061616+0000","last_became_peered":"2024-04-04T06:07:53.061616+0000","last_unstale":"2024-04-04T06:07:55.090022+0000","last_undegraded":"2024-04-04T06:07:55.090022+0000","last_fullsized":"2024-04-04T06:07:55.090022+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:08.463507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089601+0000","last_change":"2024-04-04T06:07:53.057040+0000","last_active":"2024-04-04T06:07:55.089601+0000","last_peered":"2024-04-04T06:07:55.089601+0000","last_clean":"2024-04-04T06:07:55.089601+0000","last_became_active":"2024-04-04T06:07:53.055583+0000","last_became_peered":"2024-04-04T06:07:53.055583+0000","last_unstale":"2024-04-04T06:07:55.089601+0000","last_undegraded":"2024-04-04T06:07:55.089601+0000","last_fullsized":"2024-04-04T06:07:55.089601+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:25:52.241221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.111490+0000","last_change":"2024-04-04T06:07:53.067589+0000","last_active":"2024-04-04T06:07:54.111490+0000","last_peered":"2024-04-04T06:07:54.111490+0000","last_clean":"2024-04-04T06:07:54.111490+0000","last_became_active":"2024-04-04T06:07:53.066905+0000","last_became_peered":"2024-04-04T06:07:53.066905+0000","last_unstale":"2024-04-04T06:07:54.111490+0000","last_undegraded":"2024-04-04T06:07:54.111490+0000","last_fullsized":"2024-04-04T06:07:54.111490+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:11:21.687705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.073477+0000","last_change":"2024-04-04T06:07:53.066241+0000","last_active":"2024-04-04T06:07:54.073477+0000","last_peered":"2024-04-04T06:07:54.073477+0000","last_clean":"2024-04-04T06:07:54.073477+0000","last_became_active":"2024-04-04T06:07:53.065627+0000","last_became_peered":"2024-04-04T06:07:53.065627+0000","last_unstale":"2024-04-04T06:07:54.073477+0000","last_undegraded":"2024-04-04T06:07:54.073477+0000","last_fullsized":"2024-04-04T06:07:54.073477+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:08:22.170647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089601+0000","last_change":"2024-04-04T06:07:53.056395+0000","last_active":"2024-04-04T06:07:55.089601+0000","last_peered":"2024-04-04T06:07:55.089601+0000","last_clean":"2024-04-04T06:07:55.089601+0000","last_became_active":"2024-04-04T06:07:53.055821+0000","last_became_peered":"2024-04-04T06:07:53.055821+0000","last_unstale":"2024-04-04T06:07:55.089601+0000","last_undegraded":"2024-04-04T06:07:55.089601+0000","last_fullsized":"2024-04-04T06:07:55.089601+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:35:48.409918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.096706+0000","last_change":"2024-04-04T06:07:54.094110+0000","last_active":"2024-04-04T06:07:54.096706+0000","last_peered":"2024-04-04T06:07:54.096706+0000","last_clean":"2024-04-04T06:07:54.096706+0000","last_became_active":"2024-04-04T06:07:54.093316+0000","last_became_peered":"2024-04-04T06:07:54.093316+0000","last_unstale":"2024-04-04T06:07:54.096706+0000","last_undegraded":"2024-04-04T06:07:54.096706+0000","last_fullsized":"2024-04-04T06:07:54.096706+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:04:27.387380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089021+0000","last_change":"2024-04-04T06:07:54.099176+0000","last_active":"2024-04-04T06:07:55.089021+0000","last_peered":"2024-04-04T06:07:55.089021+0000","last_clean":"2024-04-04T06:07:55.089021+0000","last_became_active":"2024-04-04T06:07:54.098797+0000","last_became_peered":"2024-04-04T06:07:54.098797+0000","last_unstale":"2024-04-04T06:07:55.089021+0000","last_undegraded":"2024-04-04T06:07:55.089021+0000","last_fullsized":"2024-04-04T06:07:55.089021+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:04:27.387380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:52.009951+0000","last_change":"2024-04-04T06:07:52.009951+0000","last_active":"2024-04-04T06:07:52.009951+0000","last_peered":"2024-04-04T06:07:52.009951+0000","last_clean":"2024-04-04T06:07:52.009951+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:52.009951+0000","last_undegraded":"2024-04-04T06:07:52.009951+0000","last_fullsized":"2024-04-04T06:07:52.009951+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:07:53.024258+0000","last_change":"2024-04-04T06:07:53.024258+0000","last_active":"2024-04-04T06:07:53.024258+0000","last_peered":"2024-04-04T06:07:53.024258+0000","last_clean":"2024-04-04T06:07:53.024258+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:07:53.024258+0000","last_undegraded":"2024-04-04T06:07:53.024258+0000","last_fullsized":"2024-04-04T06:07:53.024258+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.115042+0000","last_change":"2024-04-04T06:07:53.067506+0000","last_active":"2024-04-04T06:07:54.115042+0000","last_peered":"2024-04-04T06:07:54.115042+0000","last_clean":"2024-04-04T06:07:54.115042+0000","last_became_active":"2024-04-04T06:07:53.067063+0000","last_became_peered":"2024-04-04T06:07:53.067063+0000","last_unstale":"2024-04-04T06:07:54.115042+0000","last_undegraded":"2024-04-04T06:07:54.115042+0000","last_fullsized":"2024-04-04T06:07:54.115042+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:23:33.331413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":206,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-04T06:07:49.003106+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:07:49.003106+0000","last_peered":"2024-04-04T06:07:49.003106+0000","last_clean":"2024-04-04T06:07:49.003106+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:07:49.003106+0000","last_undegraded":"2024-04-04T06:07:49.003106+0000","last_fullsized":"2024-04-04T06:07:49.003106+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":28,"acting":28,"num_store_stats":2},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":70,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":18,"acting":18,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509468,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1077248,"data_stored":782205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31652,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":483328,"data_stored":191837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":191837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27504,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":483328,"data_stored":191837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509469,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27496,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":483328,"data_stored":191837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509471,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1077248,"data_stored":782205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31692,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":524288,"data_stored":210766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542173,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":544768,"data_stored":218608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:07:58.621 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:07:58.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.874+0000 7f6443e00640 1 Processor -- start 2024-04-04T06:07:58.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.874+0000 7f6443e00640 1 -- start start 2024-04-04T06:07:58.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.876+0000 7f6443e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f6444070250 0x7f64440d21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:58.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.876+0000 7f6443e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64440d26e0 0x7f64440d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:58.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.876+0000 7f6443e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64440d2ed0 con 0x7f644413f100 2024-04-04T06:07:58.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.876+0000 7f6443e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f6444142270 con 0x7f6444070250 2024-04-04T06:07:58.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.876+0000 7f6443e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f644413f4e0 con 0x7f64440d26e0 2024-04-04T06:07:58.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.876+0000 7f6442000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f6444070250 0x7f64440d21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:58.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.876+0000 7f6442a00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f644413f100 0x7f644406fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37058/0 (socket says 172.21.15.159:37058) 2024-04-04T06:07:58.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.876+0000 7f6442a00640 1 -- 172.21.15.159:0/114940561 learned_addr learned my addr 172.21.15.159:0/114940561 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:07:58.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.876+0000 7f6441600640 1 -- 172.21.15.159:0/114940561 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 533831699 0 0) 0x7f64440d2ed0 con 0x7f644413f100 2024-04-04T06:07:58.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.876+0000 7f6441600640 1 -- 172.21.15.159:0/114940561 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6428003520 con 0x7f644413f100 2024-04-04T06:07:58.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.876+0000 7f6443400640 1 --2- 172.21.15.159:0/114940561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64440d26e0 0x7f64440d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:58.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.876+0000 7f6442000640 1 -- 172.21.15.159:0/114940561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64440d26e0 msgr2=0x7f64440d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:58.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.877+0000 7f6442000640 1 --2- 172.21.15.159:0/114940561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64440d26e0 0x7f64440d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:58.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.877+0000 7f6442000640 1 -- 172.21.15.159:0/114940561 >> v1:172.21.15.159:6789/0 conn(0x7f644413f100 legacy=0x7f644406fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:58.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.877+0000 7f6442000640 1 -- 172.21.15.159:0/114940561 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6444141940 con 0x7f6444070250 2024-04-04T06:07:58.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.878+0000 7f6442000640 1 --2- 172.21.15.159:0/114940561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6444070250 0x7f64440d21a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f64300097a0 tx=0x7f643002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e9ff8cb5695ad0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:58.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.878+0000 7f6441600640 1 -- 172.21.15.159:0/114940561 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430035020 con 0x7f6444070250 2024-04-04T06:07:58.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.878+0000 7f6441600640 1 -- 172.21.15.159:0/114940561 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6430031810 con 0x7f6444070250 2024-04-04T06:07:58.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.878+0000 7f6441600640 1 -- 172.21.15.159:0/114940561 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430031bc0 con 0x7f6444070250 2024-04-04T06:07:58.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.878+0000 7f6443e00640 1 -- 172.21.15.159:0/114940561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6444070250 msgr2=0x7f64440d21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:58.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.878+0000 7f6443e00640 1 --2- 172.21.15.159:0/114940561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6444070250 0x7f64440d21a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f64300097a0 tx=0x7f643002b8b0 comp rx=0 tx=0).stop 2024-04-04T06:07:58.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.879+0000 7f6443e00640 1 -- 172.21.15.159:0/114940561 shutdown_connections 2024-04-04T06:07:58.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.879+0000 7f6443e00640 1 --2- 172.21.15.159:0/114940561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64440d26e0 0x7f64440d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:58.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.879+0000 7f6443e00640 1 --2- 172.21.15.159:0/114940561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6444070250 0x7f64440d21a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:58.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.879+0000 7f6443e00640 1 -- 172.21.15.159:0/114940561 >> 172.21.15.159:0/114940561 conn(0x7f6444067460 msgr2=0x7f6444067840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:58.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.879+0000 7f6443e00640 1 -- 172.21.15.159:0/114940561 shutdown_connections 2024-04-04T06:07:58.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.879+0000 7f6443e00640 1 -- 172.21.15.159:0/114940561 wait complete. 2024-04-04T06:07:58.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.880+0000 7f6443e00640 1 Processor -- start 2024-04-04T06:07:58.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.880+0000 7f6443e00640 1 -- start start 2024-04-04T06:07:58.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.881+0000 7f6443e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6444070250 0x7f6444153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:58.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.881+0000 7f6443e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64440d26e0 0x7f64440b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:58.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.881+0000 7f6443e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f644413f100 0x7f644413f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:58.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.881+0000 7f6443e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6444142700 con 0x7f644413f100 2024-04-04T06:07:58.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.881+0000 7f6443e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6444141f00 con 0x7f64440d26e0 2024-04-04T06:07:58.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.881+0000 7f6443e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f64440d2b70 con 0x7f6444070250 2024-04-04T06:07:58.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.882+0000 7f6443400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f644413f100 0x7f644413f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:58.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.882+0000 7f6443400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f644413f100 0x7f644413f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50468/0 (socket says 172.21.15.159:50468) 2024-04-04T06:07:58.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.882+0000 7f6443400640 1 -- 172.21.15.159:0/1728324231 learned_addr learned my addr 172.21.15.159:0/1728324231 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:58.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.882+0000 7f6442000640 1 --2- 172.21.15.159:0/1728324231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64440d26e0 0x7f64440b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:58.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.882+0000 7f6443400640 1 -- 172.21.15.159:0/1728324231 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6444070250 msgr2=0x7f6444153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:58.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.882+0000 7f6443400640 1 --2- 172.21.15.159:0/1728324231 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6444070250 0x7f6444153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:58.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.882+0000 7f6443400640 1 -- 172.21.15.159:0/1728324231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64440d26e0 msgr2=0x7f64440b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:58.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.882+0000 7f6443400640 1 --2- 172.21.15.159:0/1728324231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64440d26e0 0x7f64440b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:58.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.882+0000 7f6443400640 1 -- 172.21.15.159:0/1728324231 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6444143830 con 0x7f644413f100 2024-04-04T06:07:58.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.882+0000 7f6443400640 1 --2- 172.21.15.159:0/1728324231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f644413f100 0x7f644413f9b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f643c007e40 tx=0x7f643c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:58.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.883+0000 7f643aa00640 1 -- 172.21.15.159:0/1728324231 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c014620 con 0x7f644413f100 2024-04-04T06:07:58.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.883+0000 7f643aa00640 1 -- 172.21.15.159:0/1728324231 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f643c019070 con 0x7f644413f100 2024-04-04T06:07:58.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.883+0000 7f6443e00640 1 -- 172.21.15.159:0/1728324231 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6444143b10 con 0x7f644413f100 2024-04-04T06:07:58.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.883+0000 7f643aa00640 1 -- 172.21.15.159:0/1728324231 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c014a20 con 0x7f644413f100 2024-04-04T06:07:58.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.883+0000 7f6443e00640 1 -- 172.21.15.159:0/1728324231 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64441440d0 con 0x7f644413f100 2024-04-04T06:07:58.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.884+0000 7f6443e00640 1 -- 172.21.15.159:0/1728324231 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6444075940 con 0x7f644413f100 2024-04-04T06:07:58.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.885+0000 7f643aa00640 1 -- 172.21.15.159:0/1728324231 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f643c014bc0 con 0x7f644413f100 2024-04-04T06:07:58.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.886+0000 7f643aa00640 1 --2- 172.21.15.159:0/1728324231 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f642406c8f0 0x7f642406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:58.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.886+0000 7f643aa00640 1 -- 172.21.15.159:0/1728324231 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f643c08c500 con 0x7f644413f100 2024-04-04T06:07:58.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.886+0000 7f6442a00640 1 --2- 172.21.15.159:0/1728324231 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f642406c8f0 0x7f642406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:58.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.888+0000 7f6442a00640 1 --2- 172.21.15.159:0/1728324231 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f642406c8f0 0x7f642406edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6434000f60 tx=0x7f64340078f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:58.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:58.890+0000 7f643aa00640 1 -- 172.21.15.159:0/1728324231 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f643c05a890 con 0x7f644413f100 2024-04-04T06:07:59.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.089+0000 7f6443e00640 1 -- 172.21.15.159:0/1728324231 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6444144560 con 0x7f642406c8f0 2024-04-04T06:07:59.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.108+0000 7f643aa00640 1 -- 172.21.15.159:0/1728324231 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251937 (secure 0 0 0) 0x7f6434005890 con 0x7f642406c8f0 2024-04-04T06:07:59.106 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:07:59.108 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:07:59.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.115+0000 7f6443e00640 1 -- 172.21.15.159:0/1728324231 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f642406c8f0 msgr2=0x7f642406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:59.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.115+0000 7f6443e00640 1 --2- 172.21.15.159:0/1728324231 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f642406c8f0 0x7f642406edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6434000f60 tx=0x7f64340078f0 comp rx=0 tx=0).stop 2024-04-04T06:07:59.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.115+0000 7f6443e00640 1 -- 172.21.15.159:0/1728324231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f644413f100 msgr2=0x7f644413f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:59.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.115+0000 7f6443e00640 1 --2- 172.21.15.159:0/1728324231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f644413f100 0x7f644413f9b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f643c007e40 tx=0x7f643c00e4b0 comp rx=0 tx=0).stop 2024-04-04T06:07:59.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.115+0000 7f6443e00640 1 -- 172.21.15.159:0/1728324231 shutdown_connections 2024-04-04T06:07:59.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.115+0000 7f6443e00640 1 --2- 172.21.15.159:0/1728324231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f644413f100 0x7f644413f9b0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:59.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.115+0000 7f6443e00640 1 --2- 172.21.15.159:0/1728324231 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f642406c8f0 0x7f642406edb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:59.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.115+0000 7f6443e00640 1 --2- 172.21.15.159:0/1728324231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64440d26e0 0x7f64440b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:59.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.115+0000 7f6443e00640 1 --2- 172.21.15.159:0/1728324231 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6444070250 0x7f6444153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:59.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.115+0000 7f6443e00640 1 -- 172.21.15.159:0/1728324231 >> 172.21.15.159:0/1728324231 conn(0x7f6444067460 msgr2=0x7f64440d0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:59.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.115+0000 7f6443e00640 1 -- 172.21.15.159:0/1728324231 shutdown_connections 2024-04-04T06:07:59.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.115+0000 7f6443e00640 1 -- 172.21.15.159:0/1728324231 wait complete. 2024-04-04T06:07:59.127 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-04-04T06:07:58.754131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238156,"kb_used_data":5336,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736564,"statfs":{"total":773094113280,"available":772850241536,"internally_reserved":0,"allocated":5464064,"data_stored":2896324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002075"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:56.838897+0000","last_change":"2024-04-04T06:07:54.094926+0000","last_active":"2024-04-04T06:07:56.838897+0000","last_peered":"2024-04-04T06:07:56.838897+0000","last_clean":"2024-04-04T06:07:56.838897+0000","last_became_active":"2024-04-04T06:07:54.094639+0000","last_became_peered":"2024-04-04T06:07:54.094639+0000","last_unstale":"2024-04-04T06:07:56.838897+0000","last_undegraded":"2024-04-04T06:07:56.838897+0000","last_fullsized":"2024-04-04T06:07:56.838897+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:16:19.346319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.088901+0000","last_change":"2024-04-04T06:07:54.086343+0000","last_active":"2024-04-04T06:07:54.088901+0000","last_peered":"2024-04-04T06:07:54.088901+0000","last_clean":"2024-04-04T06:07:54.088901+0000","last_became_active":"2024-04-04T06:07:54.086058+0000","last_became_peered":"2024-04-04T06:07:54.086058+0000","last_unstale":"2024-04-04T06:07:54.088901+0000","last_undegraded":"2024-04-04T06:07:54.088901+0000","last_fullsized":"2024-04-04T06:07:54.088901+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:05:58.618597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089138+0000","last_change":"2024-04-04T06:07:54.098960+0000","last_active":"2024-04-04T06:07:55.089138+0000","last_peered":"2024-04-04T06:07:55.089138+0000","last_clean":"2024-04-04T06:07:55.089138+0000","last_became_active":"2024-04-04T06:07:54.098596+0000","last_became_peered":"2024-04-04T06:07:54.098596+0000","last_unstale":"2024-04-04T06:07:55.089138+0000","last_undegraded":"2024-04-04T06:07:55.089138+0000","last_fullsized":"2024-04-04T06:07:55.089138+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:21:40.197464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.086975+0000","last_change":"2024-04-04T06:07:54.094662+0000","last_active":"2024-04-04T06:07:55.086975+0000","last_peered":"2024-04-04T06:07:55.086975+0000","last_clean":"2024-04-04T06:07:55.086975+0000","last_became_active":"2024-04-04T06:07:54.092661+0000","last_became_peered":"2024-04-04T06:07:54.092661+0000","last_unstale":"2024-04-04T06:07:55.086975+0000","last_undegraded":"2024-04-04T06:07:55.086975+0000","last_fullsized":"2024-04-04T06:07:55.086975+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:14:41.451368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.088621+0000","last_change":"2024-04-04T06:07:54.086509+0000","last_active":"2024-04-04T06:07:54.088621+0000","last_peered":"2024-04-04T06:07:54.088621+0000","last_clean":"2024-04-04T06:07:54.088621+0000","last_became_active":"2024-04-04T06:07:54.085418+0000","last_became_peered":"2024-04-04T06:07:54.085418+0000","last_unstale":"2024-04-04T06:07:54.088621+0000","last_undegraded":"2024-04-04T06:07:54.088621+0000","last_fullsized":"2024-04-04T06:07:54.088621+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:55:34.902884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.091904+0000","last_change":"2024-04-04T06:07:54.098088+0000","last_active":"2024-04-04T06:07:55.091904+0000","last_peered":"2024-04-04T06:07:55.091904+0000","last_clean":"2024-04-04T06:07:55.091904+0000","last_became_active":"2024-04-04T06:07:54.096292+0000","last_became_peered":"2024-04-04T06:07:54.096292+0000","last_unstale":"2024-04-04T06:07:55.091904+0000","last_undegraded":"2024-04-04T06:07:55.091904+0000","last_fullsized":"2024-04-04T06:07:55.091904+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:22:26.493599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:56.838342+0000","last_change":"2024-04-04T06:07:54.105499+0000","last_active":"2024-04-04T06:07:56.838342+0000","last_peered":"2024-04-04T06:07:56.838342+0000","last_clean":"2024-04-04T06:07:56.838342+0000","last_became_active":"2024-04-04T06:07:54.105108+0000","last_became_peered":"2024-04-04T06:07:54.105108+0000","last_unstale":"2024-04-04T06:07:56.838342+0000","last_undegraded":"2024-04-04T06:07:56.838342+0000","last_fullsized":"2024-04-04T06:07:56.838342+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:37:33.982134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.086763+0000","last_change":"2024-04-04T06:07:54.089975+0000","last_active":"2024-04-04T06:07:55.086763+0000","last_peered":"2024-04-04T06:07:55.086763+0000","last_clean":"2024-04-04T06:07:55.086763+0000","last_became_active":"2024-04-04T06:07:54.089741+0000","last_became_peered":"2024-04-04T06:07:54.089741+0000","last_unstale":"2024-04-04T06:07:55.086763+0000","last_undegraded":"2024-04-04T06:07:55.086763+0000","last_fullsized":"2024-04-04T06:07:55.086763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:05:58.618597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.088142+0000","last_change":"2024-04-04T06:07:54.094276+0000","last_active":"2024-04-04T06:07:55.088142+0000","last_peered":"2024-04-04T06:07:55.088142+0000","last_clean":"2024-04-04T06:07:55.088142+0000","last_became_active":"2024-04-04T06:07:54.093665+0000","last_became_peered":"2024-04-04T06:07:54.093665+0000","last_unstale":"2024-04-04T06:07:55.088142+0000","last_undegraded":"2024-04-04T06:07:55.088142+0000","last_fullsized":"2024-04-04T06:07:55.088142+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:51:38.648784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.088208+0000","last_change":"2024-04-04T06:07:54.088671+0000","last_active":"2024-04-04T06:07:55.088208+0000","last_peered":"2024-04-04T06:07:55.088208+0000","last_clean":"2024-04-04T06:07:55.088208+0000","last_became_active":"2024-04-04T06:07:54.081723+0000","last_became_peered":"2024-04-04T06:07:54.081723+0000","last_unstale":"2024-04-04T06:07:55.088208+0000","last_undegraded":"2024-04-04T06:07:55.088208+0000","last_fullsized":"2024-04-04T06:07:55.088208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:30:57.851801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.087788+0000","last_change":"2024-04-04T06:07:54.104793+0000","last_active":"2024-04-04T06:07:55.087788+0000","last_peered":"2024-04-04T06:07:55.087788+0000","last_clean":"2024-04-04T06:07:55.087788+0000","last_became_active":"2024-04-04T06:07:54.104090+0000","last_became_peered":"2024-04-04T06:07:54.104090+0000","last_unstale":"2024-04-04T06:07:55.087788+0000","last_undegraded":"2024-04-04T06:07:55.087788+0000","last_fullsized":"2024-04-04T06:07:55.087788+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:42:04.674995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.097046+0000","last_change":"2024-04-04T06:07:54.093793+0000","last_active":"2024-04-04T06:07:54.097046+0000","last_peered":"2024-04-04T06:07:54.097046+0000","last_clean":"2024-04-04T06:07:54.097046+0000","last_became_active":"2024-04-04T06:07:54.090439+0000","last_became_peered":"2024-04-04T06:07:54.090439+0000","last_unstale":"2024-04-04T06:07:54.097046+0000","last_undegraded":"2024-04-04T06:07:54.097046+0000","last_fullsized":"2024-04-04T06:07:54.097046+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:23:47.563294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.085545+0000","last_change":"2024-04-04T06:07:54.088953+0000","last_active":"2024-04-04T06:07:55.085545+0000","last_peered":"2024-04-04T06:07:55.085545+0000","last_clean":"2024-04-04T06:07:55.085545+0000","last_became_active":"2024-04-04T06:07:54.087734+0000","last_became_peered":"2024-04-04T06:07:54.087734+0000","last_unstale":"2024-04-04T06:07:55.085545+0000","last_undegraded":"2024-04-04T06:07:55.085545+0000","last_fullsized":"2024-04-04T06:07:55.085545+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:30:49.068361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.087283+0000","last_change":"2024-04-04T06:07:54.108869+0000","last_active":"2024-04-04T06:07:55.087283+0000","last_peered":"2024-04-04T06:07:55.087283+0000","last_clean":"2024-04-04T06:07:55.087283+0000","last_became_active":"2024-04-04T06:07:54.105572+0000","last_became_peered":"2024-04-04T06:07:54.105572+0000","last_unstale":"2024-04-04T06:07:55.087283+0000","last_undegraded":"2024-04-04T06:07:55.087283+0000","last_fullsized":"2024-04-04T06:07:55.087283+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:28:42.604702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.091575+0000","last_change":"2024-04-04T06:07:54.087750+0000","last_active":"2024-04-04T06:07:55.091575+0000","last_peered":"2024-04-04T06:07:55.091575+0000","last_clean":"2024-04-04T06:07:55.091575+0000","last_became_active":"2024-04-04T06:07:54.087441+0000","last_became_peered":"2024-04-04T06:07:54.087441+0000","last_unstale":"2024-04-04T06:07:55.091575+0000","last_undegraded":"2024-04-04T06:07:55.091575+0000","last_fullsized":"2024-04-04T06:07:55.091575+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:08:24.773446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.093510+0000","last_change":"2024-04-04T06:07:54.084315+0000","last_active":"2024-04-04T06:07:54.093510+0000","last_peered":"2024-04-04T06:07:54.093510+0000","last_clean":"2024-04-04T06:07:54.093510+0000","last_became_active":"2024-04-04T06:07:54.084028+0000","last_became_peered":"2024-04-04T06:07:54.084028+0000","last_unstale":"2024-04-04T06:07:54.093510+0000","last_undegraded":"2024-04-04T06:07:54.093510+0000","last_fullsized":"2024-04-04T06:07:54.093510+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:48:40.226640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089965+0000","last_change":"2024-04-04T06:07:54.094405+0000","last_active":"2024-04-04T06:07:55.089965+0000","last_peered":"2024-04-04T06:07:55.089965+0000","last_clean":"2024-04-04T06:07:55.089965+0000","last_became_active":"2024-04-04T06:07:54.093952+0000","last_became_peered":"2024-04-04T06:07:54.093952+0000","last_unstale":"2024-04-04T06:07:55.089965+0000","last_undegraded":"2024-04-04T06:07:55.089965+0000","last_fullsized":"2024-04-04T06:07:55.089965+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:51:38.648784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089301+0000","last_change":"2024-04-04T06:07:54.114967+0000","last_active":"2024-04-04T06:07:55.089301+0000","last_peered":"2024-04-04T06:07:55.089301+0000","last_clean":"2024-04-04T06:07:55.089301+0000","last_became_active":"2024-04-04T06:07:54.114621+0000","last_became_peered":"2024-04-04T06:07:54.114621+0000","last_unstale":"2024-04-04T06:07:55.089301+0000","last_undegraded":"2024-04-04T06:07:55.089301+0000","last_fullsized":"2024-04-04T06:07:55.089301+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:10:39.176046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.094181+0000","last_change":"2024-04-04T06:07:54.085006+0000","last_active":"2024-04-04T06:07:54.094181+0000","last_peered":"2024-04-04T06:07:54.094181+0000","last_clean":"2024-04-04T06:07:54.094181+0000","last_became_active":"2024-04-04T06:07:54.083659+0000","last_became_peered":"2024-04-04T06:07:54.083659+0000","last_unstale":"2024-04-04T06:07:54.094181+0000","last_undegraded":"2024-04-04T06:07:54.094181+0000","last_fullsized":"2024-04-04T06:07:54.094181+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:00:34.868598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.085911+0000","last_change":"2024-04-04T06:07:54.083820+0000","last_active":"2024-04-04T06:07:55.085911+0000","last_peered":"2024-04-04T06:07:55.085911+0000","last_clean":"2024-04-04T06:07:55.085911+0000","last_became_active":"2024-04-04T06:07:54.083496+0000","last_became_peered":"2024-04-04T06:07:54.083496+0000","last_unstale":"2024-04-04T06:07:55.085911+0000","last_undegraded":"2024-04-04T06:07:55.085911+0000","last_fullsized":"2024-04-04T06:07:55.085911+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:25:53.287827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.091972+0000","last_change":"2024-04-04T06:07:54.086768+0000","last_active":"2024-04-04T06:07:55.091972+0000","last_peered":"2024-04-04T06:07:55.091972+0000","last_clean":"2024-04-04T06:07:55.091972+0000","last_became_active":"2024-04-04T06:07:54.086569+0000","last_became_peered":"2024-04-04T06:07:54.086569+0000","last_unstale":"2024-04-04T06:07:55.091972+0000","last_undegraded":"2024-04-04T06:07:55.091972+0000","last_fullsized":"2024-04-04T06:07:55.091972+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:45:23.235349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.087406+0000","last_change":"2024-04-04T06:07:54.079682+0000","last_active":"2024-04-04T06:07:55.087406+0000","last_peered":"2024-04-04T06:07:55.087406+0000","last_clean":"2024-04-04T06:07:55.087406+0000","last_became_active":"2024-04-04T06:07:54.079420+0000","last_became_peered":"2024-04-04T06:07:54.079420+0000","last_unstale":"2024-04-04T06:07:55.087406+0000","last_undegraded":"2024-04-04T06:07:55.087406+0000","last_fullsized":"2024-04-04T06:07:55.087406+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:07:45.530166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.094327+0000","last_change":"2024-04-04T06:07:54.087481+0000","last_active":"2024-04-04T06:07:54.094327+0000","last_peered":"2024-04-04T06:07:54.094327+0000","last_clean":"2024-04-04T06:07:54.094327+0000","last_became_active":"2024-04-04T06:07:54.085726+0000","last_became_peered":"2024-04-04T06:07:54.085726+0000","last_unstale":"2024-04-04T06:07:54.094327+0000","last_undegraded":"2024-04-04T06:07:54.094327+0000","last_fullsized":"2024-04-04T06:07:54.094327+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:43:33.066735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.094451+0000","last_change":"2024-04-04T06:07:54.090674+0000","last_active":"2024-04-04T06:07:54.094451+0000","last_peered":"2024-04-04T06:07:54.094451+0000","last_clean":"2024-04-04T06:07:54.094451+0000","last_became_active":"2024-04-04T06:07:54.090408+0000","last_became_peered":"2024-04-04T06:07:54.090408+0000","last_unstale":"2024-04-04T06:07:54.094451+0000","last_undegraded":"2024-04-04T06:07:54.094451+0000","last_fullsized":"2024-04-04T06:07:54.094451+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:21:51.735092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.094578+0000","last_change":"2024-04-04T06:07:54.091625+0000","last_active":"2024-04-04T06:07:54.094578+0000","last_peered":"2024-04-04T06:07:54.094578+0000","last_clean":"2024-04-04T06:07:54.094578+0000","last_became_active":"2024-04-04T06:07:54.090921+0000","last_became_peered":"2024-04-04T06:07:54.090921+0000","last_unstale":"2024-04-04T06:07:54.094578+0000","last_undegraded":"2024-04-04T06:07:54.094578+0000","last_fullsized":"2024-04-04T06:07:54.094578+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:52:00.385203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.085924+0000","last_change":"2024-04-04T06:07:54.088134+0000","last_active":"2024-04-04T06:07:55.085924+0000","last_peered":"2024-04-04T06:07:55.085924+0000","last_clean":"2024-04-04T06:07:55.085924+0000","last_became_active":"2024-04-04T06:07:54.087820+0000","last_became_peered":"2024-04-04T06:07:54.087820+0000","last_unstale":"2024-04-04T06:07:55.085924+0000","last_undegraded":"2024-04-04T06:07:55.085924+0000","last_fullsized":"2024-04-04T06:07:55.085924+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:40:43.153816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089328+0000","last_change":"2024-04-04T06:07:54.090360+0000","last_active":"2024-04-04T06:07:55.089328+0000","last_peered":"2024-04-04T06:07:55.089328+0000","last_clean":"2024-04-04T06:07:55.089328+0000","last_became_active":"2024-04-04T06:07:54.090063+0000","last_became_peered":"2024-04-04T06:07:54.090063+0000","last_unstale":"2024-04-04T06:07:55.089328+0000","last_undegraded":"2024-04-04T06:07:55.089328+0000","last_fullsized":"2024-04-04T06:07:55.089328+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:28:42.604702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.097024+0000","last_change":"2024-04-04T06:07:54.092448+0000","last_active":"2024-04-04T06:07:54.097024+0000","last_peered":"2024-04-04T06:07:54.097024+0000","last_clean":"2024-04-04T06:07:54.097024+0000","last_became_active":"2024-04-04T06:07:54.092129+0000","last_became_peered":"2024-04-04T06:07:54.092129+0000","last_unstale":"2024-04-04T06:07:54.097024+0000","last_undegraded":"2024-04-04T06:07:54.097024+0000","last_fullsized":"2024-04-04T06:07:54.097024+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:14:41.451368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.095089+0000","last_change":"2024-04-04T06:07:54.092412+0000","last_active":"2024-04-04T06:07:54.095089+0000","last_peered":"2024-04-04T06:07:54.095089+0000","last_clean":"2024-04-04T06:07:54.095089+0000","last_became_active":"2024-04-04T06:07:54.091741+0000","last_became_peered":"2024-04-04T06:07:54.091741+0000","last_unstale":"2024-04-04T06:07:54.095089+0000","last_undegraded":"2024-04-04T06:07:54.095089+0000","last_fullsized":"2024-04-04T06:07:54.095089+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:39:06.266157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.088939+0000","last_change":"2024-04-04T06:07:54.096978+0000","last_active":"2024-04-04T06:07:55.088939+0000","last_peered":"2024-04-04T06:07:55.088939+0000","last_clean":"2024-04-04T06:07:55.088939+0000","last_became_active":"2024-04-04T06:07:54.095879+0000","last_became_peered":"2024-04-04T06:07:54.095879+0000","last_unstale":"2024-04-04T06:07:55.088939+0000","last_undegraded":"2024-04-04T06:07:55.088939+0000","last_fullsized":"2024-04-04T06:07:55.088939+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:23:47.563294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.091689+0000","last_change":"2024-04-04T06:07:54.096121+0000","last_active":"2024-04-04T06:07:55.091689+0000","last_peered":"2024-04-04T06:07:55.091689+0000","last_clean":"2024-04-04T06:07:55.091689+0000","last_became_active":"2024-04-04T06:07:54.090805+0000","last_became_peered":"2024-04-04T06:07:54.090805+0000","last_unstale":"2024-04-04T06:07:55.091689+0000","last_undegraded":"2024-04-04T06:07:55.091689+0000","last_fullsized":"2024-04-04T06:07:55.091689+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:27:23.004079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.085369+0000","last_change":"2024-04-04T06:07:54.092672+0000","last_active":"2024-04-04T06:07:55.085369+0000","last_peered":"2024-04-04T06:07:55.085369+0000","last_clean":"2024-04-04T06:07:55.085369+0000","last_became_active":"2024-04-04T06:07:54.092303+0000","last_became_peered":"2024-04-04T06:07:54.092303+0000","last_unstale":"2024-04-04T06:07:55.085369+0000","last_undegraded":"2024-04-04T06:07:55.085369+0000","last_fullsized":"2024-04-04T06:07:55.085369+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:53:58.356980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.113545+0000","last_change":"2024-04-04T06:07:54.111751+0000","last_active":"2024-04-04T06:07:54.113545+0000","last_peered":"2024-04-04T06:07:54.113545+0000","last_clean":"2024-04-04T06:07:54.113545+0000","last_became_active":"2024-04-04T06:07:54.111494+0000","last_became_peered":"2024-04-04T06:07:54.111494+0000","last_unstale":"2024-04-04T06:07:54.113545+0000","last_undegraded":"2024-04-04T06:07:54.113545+0000","last_fullsized":"2024-04-04T06:07:54.113545+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:25:53.255528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.092393+0000","last_change":"2024-04-04T06:07:53.062022+0000","last_active":"2024-04-04T06:07:55.092393+0000","last_peered":"2024-04-04T06:07:55.092393+0000","last_clean":"2024-04-04T06:07:55.092393+0000","last_became_active":"2024-04-04T06:07:53.061721+0000","last_became_peered":"2024-04-04T06:07:53.061721+0000","last_unstale":"2024-04-04T06:07:55.092393+0000","last_undegraded":"2024-04-04T06:07:55.092393+0000","last_fullsized":"2024-04-04T06:07:55.092393+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:21:39.183157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.096239+0000","last_change":"2024-04-04T06:07:54.094341+0000","last_active":"2024-04-04T06:07:54.096239+0000","last_peered":"2024-04-04T06:07:54.096239+0000","last_clean":"2024-04-04T06:07:54.096239+0000","last_became_active":"2024-04-04T06:07:54.093243+0000","last_became_peered":"2024-04-04T06:07:54.093243+0000","last_unstale":"2024-04-04T06:07:54.096239+0000","last_undegraded":"2024-04-04T06:07:54.096239+0000","last_fullsized":"2024-04-04T06:07:54.096239+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:14:52.965537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089957+0000","last_change":"2024-04-04T06:07:53.062446+0000","last_active":"2024-04-04T06:07:55.089957+0000","last_peered":"2024-04-04T06:07:55.089957+0000","last_clean":"2024-04-04T06:07:55.089957+0000","last_became_active":"2024-04-04T06:07:53.062148+0000","last_became_peered":"2024-04-04T06:07:53.062148+0000","last_unstale":"2024-04-04T06:07:55.089957+0000","last_undegraded":"2024-04-04T06:07:55.089957+0000","last_fullsized":"2024-04-04T06:07:55.089957+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:21:53.730124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.087355+0000","last_change":"2024-04-04T06:07:54.083643+0000","last_active":"2024-04-04T06:07:54.087355+0000","last_peered":"2024-04-04T06:07:54.087355+0000","last_clean":"2024-04-04T06:07:54.087355+0000","last_became_active":"2024-04-04T06:07:54.083349+0000","last_became_peered":"2024-04-04T06:07:54.083349+0000","last_unstale":"2024-04-04T06:07:54.087355+0000","last_undegraded":"2024-04-04T06:07:54.087355+0000","last_fullsized":"2024-04-04T06:07:54.087355+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:24:27.790884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.110564+0000","last_change":"2024-04-04T06:07:53.068475+0000","last_active":"2024-04-04T06:07:54.110564+0000","last_peered":"2024-04-04T06:07:54.110564+0000","last_clean":"2024-04-04T06:07:54.110564+0000","last_became_active":"2024-04-04T06:07:53.068192+0000","last_became_peered":"2024-04-04T06:07:53.068192+0000","last_unstale":"2024-04-04T06:07:54.110564+0000","last_undegraded":"2024-04-04T06:07:54.110564+0000","last_fullsized":"2024-04-04T06:07:54.110564+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:35:48.409918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.107219+0000","last_change":"2024-04-04T06:07:54.101683+0000","last_active":"2024-04-04T06:07:54.107219+0000","last_peered":"2024-04-04T06:07:54.107219+0000","last_clean":"2024-04-04T06:07:54.107219+0000","last_became_active":"2024-04-04T06:07:54.101393+0000","last_became_peered":"2024-04-04T06:07:54.101393+0000","last_unstale":"2024-04-04T06:07:54.107219+0000","last_undegraded":"2024-04-04T06:07:54.107219+0000","last_fullsized":"2024-04-04T06:07:54.107219+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:21:54.744432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:56.838284+0000","last_change":"2024-04-04T06:07:53.071353+0000","last_active":"2024-04-04T06:07:56.838284+0000","last_peered":"2024-04-04T06:07:56.838284+0000","last_clean":"2024-04-04T06:07:56.838284+0000","last_became_active":"2024-04-04T06:07:53.070870+0000","last_became_peered":"2024-04-04T06:07:53.070870+0000","last_unstale":"2024-04-04T06:07:56.838284+0000","last_undegraded":"2024-04-04T06:07:56.838284+0000","last_fullsized":"2024-04-04T06:07:56.838284+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:54:09.859054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089388+0000","last_change":"2024-04-04T06:07:54.099487+0000","last_active":"2024-04-04T06:07:55.089388+0000","last_peered":"2024-04-04T06:07:55.089388+0000","last_clean":"2024-04-04T06:07:55.089388+0000","last_became_active":"2024-04-04T06:07:54.096608+0000","last_became_peered":"2024-04-04T06:07:54.096608+0000","last_unstale":"2024-04-04T06:07:55.089388+0000","last_undegraded":"2024-04-04T06:07:55.089388+0000","last_fullsized":"2024-04-04T06:07:55.089388+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:07:45.530166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.092208+0000","last_change":"2024-04-04T06:07:53.070897+0000","last_active":"2024-04-04T06:07:55.092208+0000","last_peered":"2024-04-04T06:07:55.092208+0000","last_clean":"2024-04-04T06:07:55.092208+0000","last_became_active":"2024-04-04T06:07:53.070574+0000","last_became_peered":"2024-04-04T06:07:53.070574+0000","last_unstale":"2024-04-04T06:07:55.092208+0000","last_undegraded":"2024-04-04T06:07:55.092208+0000","last_fullsized":"2024-04-04T06:07:55.092208+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:10:38.161739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.087063+0000","last_change":"2024-04-04T06:07:54.090327+0000","last_active":"2024-04-04T06:07:55.087063+0000","last_peered":"2024-04-04T06:07:55.087063+0000","last_clean":"2024-04-04T06:07:55.087063+0000","last_became_active":"2024-04-04T06:07:54.090040+0000","last_became_peered":"2024-04-04T06:07:54.090040+0000","last_unstale":"2024-04-04T06:07:55.087063+0000","last_undegraded":"2024-04-04T06:07:55.087063+0000","last_fullsized":"2024-04-04T06:07:55.087063+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:04:27.387380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089350+0000","last_change":"2024-04-04T06:07:53.067674+0000","last_active":"2024-04-04T06:07:55.089350+0000","last_peered":"2024-04-04T06:07:55.089350+0000","last_clean":"2024-04-04T06:07:55.089350+0000","last_became_active":"2024-04-04T06:07:53.067324+0000","last_became_peered":"2024-04-04T06:07:53.067324+0000","last_unstale":"2024-04-04T06:07:55.089350+0000","last_undegraded":"2024-04-04T06:07:55.089350+0000","last_fullsized":"2024-04-04T06:07:55.089350+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:24:58.400986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.086782+0000","last_change":"2024-04-04T06:07:54.076434+0000","last_active":"2024-04-04T06:07:55.086782+0000","last_peered":"2024-04-04T06:07:55.086782+0000","last_clean":"2024-04-04T06:07:55.086782+0000","last_became_active":"2024-04-04T06:07:54.076146+0000","last_became_peered":"2024-04-04T06:07:54.076146+0000","last_unstale":"2024-04-04T06:07:55.086782+0000","last_undegraded":"2024-04-04T06:07:55.086782+0000","last_fullsized":"2024-04-04T06:07:55.086782+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:55:34.902884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.071716+0000","last_change":"2024-04-04T06:07:53.065862+0000","last_active":"2024-04-04T06:07:54.071716+0000","last_peered":"2024-04-04T06:07:54.071716+0000","last_clean":"2024-04-04T06:07:54.071716+0000","last_became_active":"2024-04-04T06:07:53.065576+0000","last_became_peered":"2024-04-04T06:07:53.065576+0000","last_unstale":"2024-04-04T06:07:54.071716+0000","last_undegraded":"2024-04-04T06:07:54.071716+0000","last_fullsized":"2024-04-04T06:07:54.071716+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:30:35.808232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.094345+0000","last_change":"2024-04-04T06:07:54.087700+0000","last_active":"2024-04-04T06:07:54.094345+0000","last_peered":"2024-04-04T06:07:54.094345+0000","last_clean":"2024-04-04T06:07:54.094345+0000","last_became_active":"2024-04-04T06:07:54.087223+0000","last_became_peered":"2024-04-04T06:07:54.087223+0000","last_unstale":"2024-04-04T06:07:54.094345+0000","last_undegraded":"2024-04-04T06:07:54.094345+0000","last_fullsized":"2024-04-04T06:07:54.094345+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:57:04.209665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.086886+0000","last_change":"2024-04-04T06:07:53.051664+0000","last_active":"2024-04-04T06:07:55.086886+0000","last_peered":"2024-04-04T06:07:55.086886+0000","last_clean":"2024-04-04T06:07:55.086886+0000","last_became_active":"2024-04-04T06:07:53.050137+0000","last_became_peered":"2024-04-04T06:07:53.050137+0000","last_unstale":"2024-04-04T06:07:55.086886+0000","last_undegraded":"2024-04-04T06:07:55.086886+0000","last_fullsized":"2024-04-04T06:07:55.086886+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:31:04.331585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089594+0000","last_change":"2024-04-04T06:07:54.110939+0000","last_active":"2024-04-04T06:07:55.089594+0000","last_peered":"2024-04-04T06:07:55.089594+0000","last_clean":"2024-04-04T06:07:55.089594+0000","last_became_active":"2024-04-04T06:07:54.110586+0000","last_became_peered":"2024-04-04T06:07:54.110586+0000","last_unstale":"2024-04-04T06:07:55.089594+0000","last_undegraded":"2024-04-04T06:07:55.089594+0000","last_fullsized":"2024-04-04T06:07:55.089594+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:42:04.674995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.090022+0000","last_change":"2024-04-04T06:07:53.061916+0000","last_active":"2024-04-04T06:07:55.090022+0000","last_peered":"2024-04-04T06:07:55.090022+0000","last_clean":"2024-04-04T06:07:55.090022+0000","last_became_active":"2024-04-04T06:07:53.061616+0000","last_became_peered":"2024-04-04T06:07:53.061616+0000","last_unstale":"2024-04-04T06:07:55.090022+0000","last_undegraded":"2024-04-04T06:07:55.090022+0000","last_fullsized":"2024-04-04T06:07:55.090022+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:08.463507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:56.838966+0000","last_change":"2024-04-04T06:07:54.089736+0000","last_active":"2024-04-04T06:07:56.838966+0000","last_peered":"2024-04-04T06:07:56.838966+0000","last_clean":"2024-04-04T06:07:56.838966+0000","last_became_active":"2024-04-04T06:07:54.089364+0000","last_became_peered":"2024-04-04T06:07:54.089364+0000","last_unstale":"2024-04-04T06:07:56.838966+0000","last_undegraded":"2024-04-04T06:07:56.838966+0000","last_fullsized":"2024-04-04T06:07:56.838966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:34:42.879240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.088434+0000","last_change":"2024-04-04T06:07:53.069205+0000","last_active":"2024-04-04T06:07:55.088434+0000","last_peered":"2024-04-04T06:07:55.088434+0000","last_clean":"2024-04-04T06:07:55.088434+0000","last_became_active":"2024-04-04T06:07:53.068912+0000","last_became_peered":"2024-04-04T06:07:53.068912+0000","last_unstale":"2024-04-04T06:07:55.088434+0000","last_undegraded":"2024-04-04T06:07:55.088434+0000","last_fullsized":"2024-04-04T06:07:55.088434+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:26:59.686552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.092233+0000","last_change":"2024-04-04T06:07:54.094849+0000","last_active":"2024-04-04T06:07:55.092233+0000","last_peered":"2024-04-04T06:07:55.092233+0000","last_clean":"2024-04-04T06:07:55.092233+0000","last_became_active":"2024-04-04T06:07:54.094580+0000","last_became_peered":"2024-04-04T06:07:54.094580+0000","last_unstale":"2024-04-04T06:07:55.092233+0000","last_undegraded":"2024-04-04T06:07:55.092233+0000","last_fullsized":"2024-04-04T06:07:55.092233+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:38:07.815061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:56.839050+0000","last_change":"2024-04-04T06:07:53.070627+0000","last_active":"2024-04-04T06:07:56.839050+0000","last_peered":"2024-04-04T06:07:56.839050+0000","last_clean":"2024-04-04T06:07:56.839050+0000","last_became_active":"2024-04-04T06:07:53.070321+0000","last_became_peered":"2024-04-04T06:07:53.070321+0000","last_unstale":"2024-04-04T06:07:56.839050+0000","last_undegraded":"2024-04-04T06:07:56.839050+0000","last_fullsized":"2024-04-04T06:07:56.839050+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:44:27.520002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:56.838768+0000","last_change":"2024-04-04T06:07:54.105424+0000","last_active":"2024-04-04T06:07:56.838768+0000","last_peered":"2024-04-04T06:07:56.838768+0000","last_clean":"2024-04-04T06:07:56.838768+0000","last_became_active":"2024-04-04T06:07:54.105098+0000","last_became_peered":"2024-04-04T06:07:54.105098+0000","last_unstale":"2024-04-04T06:07:56.838768+0000","last_undegraded":"2024-04-04T06:07:56.838768+0000","last_fullsized":"2024-04-04T06:07:56.838768+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:39:09.423226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.085980+0000","last_change":"2024-04-04T06:07:53.053745+0000","last_active":"2024-04-04T06:07:55.085980+0000","last_peered":"2024-04-04T06:07:55.085980+0000","last_clean":"2024-04-04T06:07:55.085980+0000","last_became_active":"2024-04-04T06:07:53.053541+0000","last_became_peered":"2024-04-04T06:07:53.053541+0000","last_unstale":"2024-04-04T06:07:55.085980+0000","last_undegraded":"2024-04-04T06:07:55.085980+0000","last_fullsized":"2024-04-04T06:07:55.085980+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:08:45.455115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.091711+0000","last_change":"2024-04-04T06:07:54.082875+0000","last_active":"2024-04-04T06:07:54.091711+0000","last_peered":"2024-04-04T06:07:54.091711+0000","last_clean":"2024-04-04T06:07:54.091711+0000","last_became_active":"2024-04-04T06:07:54.082587+0000","last_became_peered":"2024-04-04T06:07:54.082587+0000","last_unstale":"2024-04-04T06:07:54.091711+0000","last_undegraded":"2024-04-04T06:07:54.091711+0000","last_fullsized":"2024-04-04T06:07:54.091711+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:48:53.416743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.092296+0000","last_change":"2024-04-04T06:07:53.052090+0000","last_active":"2024-04-04T06:07:55.092296+0000","last_peered":"2024-04-04T06:07:55.092296+0000","last_clean":"2024-04-04T06:07:55.092296+0000","last_became_active":"2024-04-04T06:07:53.051551+0000","last_became_peered":"2024-04-04T06:07:53.051551+0000","last_unstale":"2024-04-04T06:07:55.092296+0000","last_undegraded":"2024-04-04T06:07:55.092296+0000","last_fullsized":"2024-04-04T06:07:55.092296+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:28:41.590395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.087222+0000","last_change":"2024-04-04T06:07:54.106902+0000","last_active":"2024-04-04T06:07:55.087222+0000","last_peered":"2024-04-04T06:07:55.087222+0000","last_clean":"2024-04-04T06:07:55.087222+0000","last_became_active":"2024-04-04T06:07:54.096665+0000","last_became_peered":"2024-04-04T06:07:54.096665+0000","last_unstale":"2024-04-04T06:07:55.087222+0000","last_undegraded":"2024-04-04T06:07:55.087222+0000","last_fullsized":"2024-04-04T06:07:55.087222+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:10:39.176046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:56.838461+0000","last_change":"2024-04-04T06:07:53.070054+0000","last_active":"2024-04-04T06:07:56.838461+0000","last_peered":"2024-04-04T06:07:56.838461+0000","last_clean":"2024-04-04T06:07:56.838461+0000","last_became_active":"2024-04-04T06:07:53.069771+0000","last_became_peered":"2024-04-04T06:07:53.069771+0000","last_unstale":"2024-04-04T06:07:56.838461+0000","last_undegraded":"2024-04-04T06:07:56.838461+0000","last_fullsized":"2024-04-04T06:07:56.838461+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:38:06.549749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.091253+0000","last_change":"2024-04-04T06:07:54.099471+0000","last_active":"2024-04-04T06:07:55.091253+0000","last_peered":"2024-04-04T06:07:55.091253+0000","last_clean":"2024-04-04T06:07:55.091253+0000","last_became_active":"2024-04-04T06:07:54.097215+0000","last_became_peered":"2024-04-04T06:07:54.097215+0000","last_unstale":"2024-04-04T06:07:55.091253+0000","last_undegraded":"2024-04-04T06:07:55.091253+0000","last_fullsized":"2024-04-04T06:07:55.091253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:25:54.597791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.091674+0000","last_change":"2024-04-04T06:07:53.051859+0000","last_active":"2024-04-04T06:07:55.091674+0000","last_peered":"2024-04-04T06:07:55.091674+0000","last_clean":"2024-04-04T06:07:55.091674+0000","last_became_active":"2024-04-04T06:07:53.051559+0000","last_became_peered":"2024-04-04T06:07:53.051559+0000","last_unstale":"2024-04-04T06:07:55.091674+0000","last_undegraded":"2024-04-04T06:07:55.091674+0000","last_fullsized":"2024-04-04T06:07:55.091674+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:23:46.548987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:56.838427+0000","last_change":"2024-04-04T06:07:54.091251+0000","last_active":"2024-04-04T06:07:56.838427+0000","last_peered":"2024-04-04T06:07:56.838427+0000","last_clean":"2024-04-04T06:07:56.838427+0000","last_became_active":"2024-04-04T06:07:54.090931+0000","last_became_peered":"2024-04-04T06:07:54.090931+0000","last_unstale":"2024-04-04T06:07:56.838427+0000","last_undegraded":"2024-04-04T06:07:56.838427+0000","last_fullsized":"2024-04-04T06:07:56.838427+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:27:46.855417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":22,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.087414+0000","last_change":"2024-04-04T06:07:53.065107+0000","last_active":"2024-04-04T06:07:55.087414+0000","last_peered":"2024-04-04T06:07:55.087414+0000","last_clean":"2024-04-04T06:07:55.087414+0000","last_became_active":"2024-04-04T06:07:53.064851+0000","last_became_peered":"2024-04-04T06:07:53.064851+0000","last_unstale":"2024-04-04T06:07:55.087414+0000","last_undegraded":"2024-04-04T06:07:55.087414+0000","last_fullsized":"2024-04-04T06:07:55.087414+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:25:52.273520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.085964+0000","last_change":"2024-04-04T06:07:54.086384+0000","last_active":"2024-04-04T06:07:55.085964+0000","last_peered":"2024-04-04T06:07:55.085964+0000","last_clean":"2024-04-04T06:07:55.085964+0000","last_became_active":"2024-04-04T06:07:54.084930+0000","last_became_peered":"2024-04-04T06:07:54.084930+0000","last_unstale":"2024-04-04T06:07:55.085964+0000","last_undegraded":"2024-04-04T06:07:55.085964+0000","last_fullsized":"2024-04-04T06:07:55.085964+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:31:05.345893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.085904+0000","last_change":"2024-04-04T06:07:53.051565+0000","last_active":"2024-04-04T06:07:55.085904+0000","last_peered":"2024-04-04T06:07:55.085904+0000","last_clean":"2024-04-04T06:07:55.085904+0000","last_became_active":"2024-04-04T06:07:53.051331+0000","last_became_peered":"2024-04-04T06:07:53.051331+0000","last_unstale":"2024-04-04T06:07:55.085904+0000","last_undegraded":"2024-04-04T06:07:55.085904+0000","last_fullsized":"2024-04-04T06:07:55.085904+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:05:27.443594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.088317+0000","last_change":"2024-04-04T06:07:54.091977+0000","last_active":"2024-04-04T06:07:55.088317+0000","last_peered":"2024-04-04T06:07:55.088317+0000","last_clean":"2024-04-04T06:07:55.088317+0000","last_became_active":"2024-04-04T06:07:54.082801+0000","last_became_peered":"2024-04-04T06:07:54.082801+0000","last_unstale":"2024-04-04T06:07:55.088317+0000","last_undegraded":"2024-04-04T06:07:55.088317+0000","last_fullsized":"2024-04-04T06:07:55.088317+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:00:04.938563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089601+0000","last_change":"2024-04-04T06:07:53.057040+0000","last_active":"2024-04-04T06:07:55.089601+0000","last_peered":"2024-04-04T06:07:55.089601+0000","last_clean":"2024-04-04T06:07:55.089601+0000","last_became_active":"2024-04-04T06:07:53.055583+0000","last_became_peered":"2024-04-04T06:07:53.055583+0000","last_unstale":"2024-04-04T06:07:55.089601+0000","last_undegraded":"2024-04-04T06:07:55.089601+0000","last_fullsized":"2024-04-04T06:07:55.089601+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:25:52.241221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.093251+0000","last_change":"2024-04-04T06:07:54.083642+0000","last_active":"2024-04-04T06:07:54.093251+0000","last_peered":"2024-04-04T06:07:54.093251+0000","last_clean":"2024-04-04T06:07:54.093251+0000","last_became_active":"2024-04-04T06:07:54.083303+0000","last_became_peered":"2024-04-04T06:07:54.083303+0000","last_unstale":"2024-04-04T06:07:54.093251+0000","last_undegraded":"2024-04-04T06:07:54.093251+0000","last_fullsized":"2024-04-04T06:07:54.093251+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:35:35.708074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.111490+0000","last_change":"2024-04-04T06:07:53.067589+0000","last_active":"2024-04-04T06:07:54.111490+0000","last_peered":"2024-04-04T06:07:54.111490+0000","last_clean":"2024-04-04T06:07:54.111490+0000","last_became_active":"2024-04-04T06:07:53.066905+0000","last_became_peered":"2024-04-04T06:07:53.066905+0000","last_unstale":"2024-04-04T06:07:54.111490+0000","last_undegraded":"2024-04-04T06:07:54.111490+0000","last_fullsized":"2024-04-04T06:07:54.111490+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:11:21.687705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.104915+0000","last_change":"2024-04-04T06:07:54.103356+0000","last_active":"2024-04-04T06:07:54.104915+0000","last_peered":"2024-04-04T06:07:54.104915+0000","last_clean":"2024-04-04T06:07:54.104915+0000","last_became_active":"2024-04-04T06:07:54.102865+0000","last_became_peered":"2024-04-04T06:07:54.102865+0000","last_unstale":"2024-04-04T06:07:54.104915+0000","last_undegraded":"2024-04-04T06:07:54.104915+0000","last_fullsized":"2024-04-04T06:07:54.104915+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:25:32.222311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.092252+0000","last_change":"2024-04-04T06:07:53.062361+0000","last_active":"2024-04-04T06:07:55.092252+0000","last_peered":"2024-04-04T06:07:55.092252+0000","last_clean":"2024-04-04T06:07:55.092252+0000","last_became_active":"2024-04-04T06:07:53.061776+0000","last_became_peered":"2024-04-04T06:07:53.061776+0000","last_unstale":"2024-04-04T06:07:55.092252+0000","last_undegraded":"2024-04-04T06:07:55.092252+0000","last_fullsized":"2024-04-04T06:07:55.092252+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:42:03.660687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.091900+0000","last_change":"2024-04-04T06:07:54.091895+0000","last_active":"2024-04-04T06:07:55.091900+0000","last_peered":"2024-04-04T06:07:55.091900+0000","last_clean":"2024-04-04T06:07:55.091900+0000","last_became_active":"2024-04-04T06:07:54.091593+0000","last_became_peered":"2024-04-04T06:07:54.091593+0000","last_unstale":"2024-04-04T06:07:55.091900+0000","last_undegraded":"2024-04-04T06:07:55.091900+0000","last_fullsized":"2024-04-04T06:07:55.091900+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:14:11.051327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.092442+0000","last_change":"2024-04-04T06:07:53.066828+0000","last_active":"2024-04-04T06:07:55.092442+0000","last_peered":"2024-04-04T06:07:55.092442+0000","last_clean":"2024-04-04T06:07:55.092442+0000","last_became_active":"2024-04-04T06:07:53.066337+0000","last_became_peered":"2024-04-04T06:07:53.066337+0000","last_unstale":"2024-04-04T06:07:55.092442+0000","last_undegraded":"2024-04-04T06:07:55.092442+0000","last_fullsized":"2024-04-04T06:07:55.092442+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:07:44.515859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.100361+0000","last_change":"2024-04-04T06:07:54.093554+0000","last_active":"2024-04-04T06:07:54.100361+0000","last_peered":"2024-04-04T06:07:54.100361+0000","last_clean":"2024-04-04T06:07:54.100361+0000","last_became_active":"2024-04-04T06:07:54.093282+0000","last_became_peered":"2024-04-04T06:07:54.093282+0000","last_unstale":"2024-04-04T06:07:54.100361+0000","last_undegraded":"2024-04-04T06:07:54.100361+0000","last_fullsized":"2024-04-04T06:07:54.100361+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:39:09.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.073477+0000","last_change":"2024-04-04T06:07:53.066241+0000","last_active":"2024-04-04T06:07:54.073477+0000","last_peered":"2024-04-04T06:07:54.073477+0000","last_clean":"2024-04-04T06:07:54.073477+0000","last_became_active":"2024-04-04T06:07:53.065627+0000","last_became_peered":"2024-04-04T06:07:53.065627+0000","last_unstale":"2024-04-04T06:07:54.073477+0000","last_undegraded":"2024-04-04T06:07:54.073477+0000","last_fullsized":"2024-04-04T06:07:54.073477+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:08:22.170647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.085519+0000","last_change":"2024-04-04T06:07:54.085740+0000","last_active":"2024-04-04T06:07:55.085519+0000","last_peered":"2024-04-04T06:07:55.085519+0000","last_clean":"2024-04-04T06:07:55.085519+0000","last_became_active":"2024-04-04T06:07:54.085403+0000","last_became_peered":"2024-04-04T06:07:54.085403+0000","last_unstale":"2024-04-04T06:07:55.085519+0000","last_undegraded":"2024-04-04T06:07:55.085519+0000","last_fullsized":"2024-04-04T06:07:55.085519+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:39:18.523227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.087339+0000","last_change":"2024-04-04T06:07:53.069282+0000","last_active":"2024-04-04T06:07:55.087339+0000","last_peered":"2024-04-04T06:07:55.087339+0000","last_clean":"2024-04-04T06:07:55.087339+0000","last_became_active":"2024-04-04T06:07:53.068961+0000","last_became_peered":"2024-04-04T06:07:53.068961+0000","last_unstale":"2024-04-04T06:07:55.087339+0000","last_undegraded":"2024-04-04T06:07:55.087339+0000","last_fullsized":"2024-04-04T06:07:55.087339+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:47:40.551697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.087067+0000","last_change":"2024-04-04T06:07:54.094233+0000","last_active":"2024-04-04T06:07:55.087067+0000","last_peered":"2024-04-04T06:07:55.087067+0000","last_clean":"2024-04-04T06:07:55.087067+0000","last_became_active":"2024-04-04T06:07:54.093943+0000","last_became_peered":"2024-04-04T06:07:54.093943+0000","last_unstale":"2024-04-04T06:07:55.087067+0000","last_undegraded":"2024-04-04T06:07:55.087067+0000","last_fullsized":"2024-04-04T06:07:55.087067+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:23:47.563294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:56.838007+0000","last_change":"2024-04-04T06:07:53.068825+0000","last_active":"2024-04-04T06:07:56.838007+0000","last_peered":"2024-04-04T06:07:56.838007+0000","last_clean":"2024-04-04T06:07:56.838007+0000","last_became_active":"2024-04-04T06:07:53.068557+0000","last_became_peered":"2024-04-04T06:07:53.068557+0000","last_unstale":"2024-04-04T06:07:56.838007+0000","last_undegraded":"2024-04-04T06:07:56.838007+0000","last_fullsized":"2024-04-04T06:07:56.838007+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:11:09.338088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.089094+0000","last_change":"2024-04-04T06:07:54.082447+0000","last_active":"2024-04-04T06:07:54.089094+0000","last_peered":"2024-04-04T06:07:54.089094+0000","last_clean":"2024-04-04T06:07:54.089094+0000","last_became_active":"2024-04-04T06:07:54.082110+0000","last_became_peered":"2024-04-04T06:07:54.082110+0000","last_unstale":"2024-04-04T06:07:54.089094+0000","last_undegraded":"2024-04-04T06:07:54.089094+0000","last_fullsized":"2024-04-04T06:07:54.089094+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:22:07.541864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.095432+0000","last_change":"2024-04-04T06:07:53.062615+0000","last_active":"2024-04-04T06:07:55.095432+0000","last_peered":"2024-04-04T06:07:55.095432+0000","last_clean":"2024-04-04T06:07:55.095432+0000","last_became_active":"2024-04-04T06:07:53.062311+0000","last_became_peered":"2024-04-04T06:07:53.062311+0000","last_unstale":"2024-04-04T06:07:55.095432+0000","last_undegraded":"2024-04-04T06:07:55.095432+0000","last_fullsized":"2024-04-04T06:07:55.095432+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:04:26.373072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.087118+0000","last_change":"2024-04-04T06:07:54.105858+0000","last_active":"2024-04-04T06:07:55.087118+0000","last_peered":"2024-04-04T06:07:55.087118+0000","last_clean":"2024-04-04T06:07:55.087118+0000","last_became_active":"2024-04-04T06:07:54.105008+0000","last_became_peered":"2024-04-04T06:07:54.105008+0000","last_unstale":"2024-04-04T06:07:55.087118+0000","last_undegraded":"2024-04-04T06:07:55.087118+0000","last_fullsized":"2024-04-04T06:07:55.087118+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:21:40.197464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:56.839042+0000","last_change":"2024-04-04T06:07:53.071277+0000","last_active":"2024-04-04T06:07:56.839042+0000","last_peered":"2024-04-04T06:07:56.839042+0000","last_clean":"2024-04-04T06:07:56.839042+0000","last_became_active":"2024-04-04T06:07:53.070965+0000","last_became_peered":"2024-04-04T06:07:53.070965+0000","last_unstale":"2024-04-04T06:07:56.839042+0000","last_undegraded":"2024-04-04T06:07:56.839042+0000","last_fullsized":"2024-04-04T06:07:56.839042+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:50:02.880629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:56.838787+0000","last_change":"2024-04-04T06:07:54.095188+0000","last_active":"2024-04-04T06:07:56.838787+0000","last_peered":"2024-04-04T06:07:56.838787+0000","last_clean":"2024-04-04T06:07:56.838787+0000","last_became_active":"2024-04-04T06:07:54.094849+0000","last_became_peered":"2024-04-04T06:07:54.094849+0000","last_unstale":"2024-04-04T06:07:56.838787+0000","last_undegraded":"2024-04-04T06:07:56.838787+0000","last_fullsized":"2024-04-04T06:07:56.838787+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:55:13.180186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.099084+0000","last_change":"2024-04-04T06:07:53.068894+0000","last_active":"2024-04-04T06:07:54.099084+0000","last_peered":"2024-04-04T06:07:54.099084+0000","last_clean":"2024-04-04T06:07:54.099084+0000","last_became_active":"2024-04-04T06:07:53.068074+0000","last_became_peered":"2024-04-04T06:07:53.068074+0000","last_unstale":"2024-04-04T06:07:54.099084+0000","last_undegraded":"2024-04-04T06:07:54.099084+0000","last_fullsized":"2024-04-04T06:07:54.099084+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:08.463507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.092458+0000","last_change":"2024-04-04T06:07:54.106345+0000","last_active":"2024-04-04T06:07:55.092458+0000","last_peered":"2024-04-04T06:07:55.092458+0000","last_clean":"2024-04-04T06:07:55.092458+0000","last_became_active":"2024-04-04T06:07:54.106061+0000","last_became_peered":"2024-04-04T06:07:54.106061+0000","last_unstale":"2024-04-04T06:07:55.092458+0000","last_undegraded":"2024-04-04T06:07:55.092458+0000","last_fullsized":"2024-04-04T06:07:55.092458+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:45:53.396045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:56.838815+0000","last_change":"2024-04-04T06:07:53.070349+0000","last_active":"2024-04-04T06:07:56.838815+0000","last_peered":"2024-04-04T06:07:56.838815+0000","last_clean":"2024-04-04T06:07:56.838815+0000","last_became_active":"2024-04-04T06:07:53.070050+0000","last_became_peered":"2024-04-04T06:07:53.070050+0000","last_unstale":"2024-04-04T06:07:56.838815+0000","last_undegraded":"2024-04-04T06:07:56.838815+0000","last_fullsized":"2024-04-04T06:07:56.838815+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:12:19.693836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.092140+0000","last_change":"2024-04-04T06:07:54.093604+0000","last_active":"2024-04-04T06:07:55.092140+0000","last_peered":"2024-04-04T06:07:55.092140+0000","last_clean":"2024-04-04T06:07:55.092140+0000","last_became_active":"2024-04-04T06:07:54.092915+0000","last_became_peered":"2024-04-04T06:07:54.092915+0000","last_unstale":"2024-04-04T06:07:55.092140+0000","last_undegraded":"2024-04-04T06:07:55.092140+0000","last_fullsized":"2024-04-04T06:07:55.092140+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:36:54.849628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089601+0000","last_change":"2024-04-04T06:07:53.056395+0000","last_active":"2024-04-04T06:07:55.089601+0000","last_peered":"2024-04-04T06:07:55.089601+0000","last_clean":"2024-04-04T06:07:55.089601+0000","last_became_active":"2024-04-04T06:07:53.055821+0000","last_became_peered":"2024-04-04T06:07:53.055821+0000","last_unstale":"2024-04-04T06:07:55.089601+0000","last_undegraded":"2024-04-04T06:07:55.089601+0000","last_fullsized":"2024-04-04T06:07:55.089601+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:35:48.409918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.096706+0000","last_change":"2024-04-04T06:07:54.094110+0000","last_active":"2024-04-04T06:07:54.096706+0000","last_peered":"2024-04-04T06:07:54.096706+0000","last_clean":"2024-04-04T06:07:54.096706+0000","last_became_active":"2024-04-04T06:07:54.093316+0000","last_became_peered":"2024-04-04T06:07:54.093316+0000","last_unstale":"2024-04-04T06:07:54.096706+0000","last_undegraded":"2024-04-04T06:07:54.096706+0000","last_fullsized":"2024-04-04T06:07:54.096706+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:04:27.387380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.095070+0000","last_change":"2024-04-04T06:07:53.061528+0000","last_active":"2024-04-04T06:07:54.095070+0000","last_peered":"2024-04-04T06:07:54.095070+0000","last_clean":"2024-04-04T06:07:54.095070+0000","last_became_active":"2024-04-04T06:07:53.061248+0000","last_became_peered":"2024-04-04T06:07:53.061248+0000","last_unstale":"2024-04-04T06:07:54.095070+0000","last_undegraded":"2024-04-04T06:07:54.095070+0000","last_fullsized":"2024-04-04T06:07:54.095070+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:24:58.400986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.089021+0000","last_change":"2024-04-04T06:07:54.099176+0000","last_active":"2024-04-04T06:07:55.089021+0000","last_peered":"2024-04-04T06:07:55.089021+0000","last_clean":"2024-04-04T06:07:55.089021+0000","last_became_active":"2024-04-04T06:07:54.098797+0000","last_became_peered":"2024-04-04T06:07:54.098797+0000","last_unstale":"2024-04-04T06:07:55.089021+0000","last_undegraded":"2024-04-04T06:07:55.089021+0000","last_fullsized":"2024-04-04T06:07:55.089021+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:04:27.387380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.087394+0000","last_change":"2024-04-04T06:07:53.053336+0000","last_active":"2024-04-04T06:07:55.087394+0000","last_peered":"2024-04-04T06:07:55.087394+0000","last_clean":"2024-04-04T06:07:55.087394+0000","last_became_active":"2024-04-04T06:07:53.053051+0000","last_became_peered":"2024-04-04T06:07:53.053051+0000","last_unstale":"2024-04-04T06:07:55.087394+0000","last_undegraded":"2024-04-04T06:07:55.087394+0000","last_fullsized":"2024-04-04T06:07:55.087394+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:40:42.139509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:55.088363+0000","last_change":"2024-04-04T06:07:54.109879+0000","last_active":"2024-04-04T06:07:55.088363+0000","last_peered":"2024-04-04T06:07:55.088363+0000","last_clean":"2024-04-04T06:07:55.088363+0000","last_became_active":"2024-04-04T06:07:54.103057+0000","last_became_peered":"2024-04-04T06:07:54.103057+0000","last_unstale":"2024-04-04T06:07:55.088363+0000","last_undegraded":"2024-04-04T06:07:55.088363+0000","last_fullsized":"2024-04-04T06:07:55.088363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:53.024258+0000","last_clean_scrub_stamp":"2024-04-04T06:07:53.024258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:24:28.778323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-04T06:07:54.115042+0000","last_change":"2024-04-04T06:07:53.067506+0000","last_active":"2024-04-04T06:07:54.115042+0000","last_peered":"2024-04-04T06:07:54.115042+0000","last_clean":"2024-04-04T06:07:54.115042+0000","last_became_active":"2024-04-04T06:07:53.067063+0000","last_became_peered":"2024-04-04T06:07:53.067063+0000","last_unstale":"2024-04-04T06:07:54.115042+0000","last_undegraded":"2024-04-04T06:07:54.115042+0000","last_fullsized":"2024-04-04T06:07:54.115042+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:07:52.009951+0000","last_clean_scrub_stamp":"2024-04-04T06:07:52.009951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:23:33.331413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":214,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-04T06:07:56.837879+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:07:56.837879+0000","last_peered":"2024-04-04T06:07:56.837879+0000","last_clean":"2024-04-04T06:07:56.837879+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:07:56.837879+0000","last_undegraded":"2024-04-04T06:07:56.837879+0000","last_fullsized":"2024-04-04T06:07:56.837879+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1118208,"data_stored":801154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509471,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31700,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":528384,"data_stored":211368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509472,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":217634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":548864,"data_stored":218630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":210766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509472,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28124,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343716,"statfs":{"total":96636764160,"available":96607965184,"internally_reserved":0,"allocated":1130496,"data_stored":807398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31692,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":524288,"data_stored":210766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542173,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":544768,"data_stored":218608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:07:59.131 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-04T06:07:59.132 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-04T06:07:59.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.393+0000 7f173cc00640 1 Processor -- start 2024-04-04T06:07:59.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.393+0000 7f173cc00640 1 -- start start 2024-04-04T06:07:59.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.395+0000 7f173cc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f173815e860 0x7f173806d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:59.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.395+0000 7f173cc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f173806da30 0x7f17380d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:59.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.395+0000 7f173cc00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f173806dfa0 con 0x7f17380d4910 2024-04-04T06:07:59.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.395+0000 7f173cc00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f17380d4d60 con 0x7f173806da30 2024-04-04T06:07:59.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.395+0000 7f173cc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f17380cf220 con 0x7f173815e860 2024-04-04T06:07:59.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.395+0000 7f172d400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f173806da30 0x7f17380d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:59.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.395+0000 7f172d400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f173806da30 0x7f17380d43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50474/0 (socket says 172.21.15.159:50474) 2024-04-04T06:07:59.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.395+0000 7f172d400640 1 -- 172.21.15.159:0/3700149657 learned_addr learned my addr 172.21.15.159:0/3700149657 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:59.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.396+0000 7f172d400640 1 -- 172.21.15.159:0/3700149657 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f173815e860 msgr2=0x7f173806d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:59.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.396+0000 7f172d400640 1 --2- 172.21.15.159:0/3700149657 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f173815e860 0x7f173806d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:59.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.396+0000 7f172d400640 1 -- 172.21.15.159:0/3700149657 >> v1:172.21.15.178:6789/0 conn(0x7f17380d4910 legacy=0x7f17380d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:07:59.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.396+0000 7f172d400640 1 -- 172.21.15.159:0/3700149657 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173806e240 con 0x7f173806da30 2024-04-04T06:07:59.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.397+0000 7f172d400640 1 --2- 172.21.15.159:0/3700149657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f173806da30 0x7f17380d43d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1720009870 tx=0x7f172002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3312dba2e9d89dce server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:59.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.398+0000 7f1735400640 1 -- 172.21.15.159:0/3700149657 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720035020 con 0x7f173806da30 2024-04-04T06:07:59.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.398+0000 7f1735400640 1 -- 172.21.15.159:0/3700149657 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1720031640 con 0x7f173806da30 2024-04-04T06:07:59.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.398+0000 7f1735400640 1 -- 172.21.15.159:0/3700149657 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172002d040 con 0x7f173806da30 2024-04-04T06:07:59.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.398+0000 7f173cc00640 1 -- 172.21.15.159:0/3700149657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f173806da30 msgr2=0x7f17380d43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:59.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.398+0000 7f173cc00640 1 --2- 172.21.15.159:0/3700149657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f173806da30 0x7f17380d43d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1720009870 tx=0x7f172002b8f0 comp rx=0 tx=0).stop 2024-04-04T06:07:59.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.398+0000 7f173cc00640 1 -- 172.21.15.159:0/3700149657 shutdown_connections 2024-04-04T06:07:59.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.399+0000 7f173cc00640 1 --2- 172.21.15.159:0/3700149657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f173806da30 0x7f17380d43d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:59.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.399+0000 7f173cc00640 1 --2- 172.21.15.159:0/3700149657 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f173815e860 0x7f173806d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:59.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.399+0000 7f173cc00640 1 -- 172.21.15.159:0/3700149657 >> 172.21.15.159:0/3700149657 conn(0x7f173809d7d0 msgr2=0x7f173808bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:59.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.399+0000 7f173cc00640 1 -- 172.21.15.159:0/3700149657 shutdown_connections 2024-04-04T06:07:59.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.399+0000 7f173cc00640 1 -- 172.21.15.159:0/3700149657 wait complete. 2024-04-04T06:07:59.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.402+0000 7f173cc00640 1 Processor -- start 2024-04-04T06:07:59.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.403+0000 7f173cc00640 1 -- start start 2024-04-04T06:07:59.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.405+0000 7f173cc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f173806da30 0x7f1738158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:59.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.405+0000 7f173cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17380d4910 0x7f17380b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:59.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.405+0000 7f173cc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f173815e860 0x7f17380b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:59.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.405+0000 7f173cc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f17380cedb0 con 0x7f17380d4910 2024-04-04T06:07:59.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.405+0000 7f173cc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f17380d2c50 con 0x7f173815e860 2024-04-04T06:07:59.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.405+0000 7f173cc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f17380cef30 con 0x7f173806da30 2024-04-04T06:07:59.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.405+0000 7f172d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17380d4910 0x7f17380b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:59.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.405+0000 7f172d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17380d4910 0x7f17380b3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50490/0 (socket says 172.21.15.159:50490) 2024-04-04T06:07:59.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.405+0000 7f172d400640 1 -- 172.21.15.159:0/3559758084 learned_addr learned my addr 172.21.15.159:0/3559758084 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:59.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.405+0000 7f1735e00640 1 --2- 172.21.15.159:0/3559758084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f173806da30 0x7f1738158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:59.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.406+0000 7f172d400640 1 -- 172.21.15.159:0/3559758084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f173806da30 msgr2=0x7f1738158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:59.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.406+0000 7f172d400640 1 --2- 172.21.15.159:0/3559758084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f173806da30 0x7f1738158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:59.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.406+0000 7f172d400640 1 -- 172.21.15.159:0/3559758084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f173815e860 msgr2=0x7f17380b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:59.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.406+0000 7f172d400640 1 --2- 172.21.15.159:0/3559758084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f173815e860 0x7f17380b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:59.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.406+0000 7f172d400640 1 -- 172.21.15.159:0/3559758084 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17380cb5b0 con 0x7f17380d4910 2024-04-04T06:07:59.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.406+0000 7f1735e00640 1 --2- 172.21.15.159:0/3559758084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f173806da30 0x7f1738158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:07:59.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.406+0000 7f172d400640 1 --2- 172.21.15.159:0/3559758084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17380d4910 0x7f17380b3050 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1720000c00 tx=0x7f17200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:59.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.406+0000 7f172ea00640 1 -- 172.21.15.159:0/3559758084 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172003ac60 con 0x7f17380d4910 2024-04-04T06:07:59.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.406+0000 7f172ea00640 1 -- 172.21.15.159:0/3559758084 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f172003ae20 con 0x7f17380d4910 2024-04-04T06:07:59.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.406+0000 7f172ea00640 1 -- 172.21.15.159:0/3559758084 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720044630 con 0x7f17380d4910 2024-04-04T06:07:59.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.406+0000 7f173cc00640 1 -- 172.21.15.159:0/3559758084 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17380cb890 con 0x7f17380d4910 2024-04-04T06:07:59.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.407+0000 7f173cc00640 1 -- 172.21.15.159:0/3559758084 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17380cbe50 con 0x7f17380d4910 2024-04-04T06:07:59.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.408+0000 7f173cc00640 1 -- 172.21.15.159:0/3559758084 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1738159c70 con 0x7f17380d4910 2024-04-04T06:07:59.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.409+0000 7f172ea00640 1 -- 172.21.15.159:0/3559758084 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1720038020 con 0x7f17380d4910 2024-04-04T06:07:59.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.409+0000 7f172ea00640 1 --2- 172.21.15.159:0/3559758084 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f171406c8f0 0x7f171406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:59.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.410+0000 7f172ea00640 1 -- 172.21.15.159:0/3559758084 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f17200ad9a0 con 0x7f17380d4910 2024-04-04T06:07:59.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.410+0000 7f1735e00640 1 --2- 172.21.15.159:0/3559758084 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f171406c8f0 0x7f171406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:59.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.411+0000 7f1735e00640 1 --2- 172.21.15.159:0/3559758084 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f171406c8f0 0x7f171406edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f17380b6c40 tx=0x7f1728008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:59.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.423+0000 7f172ea00640 1 -- 172.21.15.159:0/3559758084 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1738159c70 con 0x7f17380d4910 2024-04-04T06:07:59.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.754+0000 7f173cc00640 1 -- 172.21.15.159:0/3559758084 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f17380b1b40 con 0x7f17380d4910 2024-04-04T06:07:59.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.757+0000 7f172ea00640 1 -- 172.21.15.159:0/3559758084 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7f172003f070 con 0x7f17380d4910 2024-04-04T06:07:59.751 INFO:teuthology.orchestra.run.smithi159.stdout:[client.0] 2024-04-04T06:07:59.751 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQD9Qg5mqNKJGxAA4jP9cadhRIyhQ3lssttWIw== 2024-04-04T06:07:59.752 INFO:teuthology.orchestra.run.smithi159.stdout: caps mds = "allow" 2024-04-04T06:07:59.752 INFO:teuthology.orchestra.run.smithi159.stdout: caps mon = "allow r" 2024-04-04T06:07:59.752 INFO:teuthology.orchestra.run.smithi159.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-04T06:07:59.752 INFO:teuthology.orchestra.run.smithi159.stderr:updated caps for client.0 2024-04-04T06:07:59.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.760+0000 7f173cc00640 1 -- 172.21.15.159:0/3559758084 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f171406c8f0 msgr2=0x7f171406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:59.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.760+0000 7f173cc00640 1 --2- 172.21.15.159:0/3559758084 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f171406c8f0 0x7f171406edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f17380b6c40 tx=0x7f1728008040 comp rx=0 tx=0).stop 2024-04-04T06:07:59.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.760+0000 7f173cc00640 1 -- 172.21.15.159:0/3559758084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17380d4910 msgr2=0x7f17380b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:59.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.760+0000 7f173cc00640 1 --2- 172.21.15.159:0/3559758084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17380d4910 0x7f17380b3050 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1720000c00 tx=0x7f17200046c0 comp rx=0 tx=0).stop 2024-04-04T06:07:59.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.760+0000 7f173cc00640 1 -- 172.21.15.159:0/3559758084 shutdown_connections 2024-04-04T06:07:59.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.760+0000 7f173cc00640 1 --2- 172.21.15.159:0/3559758084 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f171406c8f0 0x7f171406edb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:59.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.760+0000 7f173cc00640 1 --2- 172.21.15.159:0/3559758084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f173815e860 0x7f17380b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:59.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.760+0000 7f173cc00640 1 --2- 172.21.15.159:0/3559758084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17380d4910 0x7f17380b3050 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:59.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.760+0000 7f173cc00640 1 --2- 172.21.15.159:0/3559758084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f173806da30 0x7f1738158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:59.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.760+0000 7f173cc00640 1 -- 172.21.15.159:0/3559758084 >> 172.21.15.159:0/3559758084 conn(0x7f173809d7d0 msgr2=0x7f17380b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:59.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.760+0000 7f173cc00640 1 -- 172.21.15.159:0/3559758084 shutdown_connections 2024-04-04T06:07:59.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.760+0000 7f173cc00640 1 -- 172.21.15.159:0/3559758084 wait complete. 2024-04-04T06:07:59.772 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:07:59.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.977+0000 7f1d19c00640 1 Processor -- start 2024-04-04T06:07:59.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.978+0000 7f1d19c00640 1 -- start start 2024-04-04T06:07:59.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.979+0000 7f1d19c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f1d1415e980 0x7f1d140cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:59.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.979+0000 7f1d19c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d140d2530 con 0x7f1d140d3130 2024-04-04T06:07:59.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.979+0000 7f1d19c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d140d5370 con 0x7f1d140d4f10 2024-04-04T06:07:59.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.979+0000 7f1d19c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f1d140cf630 con 0x7f1d1415e980 2024-04-04T06:07:59.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.979+0000 7f1d12a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f1d1415e980 0x7f1d140cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:59.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.979+0000 7f1d12a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f1d1415e980 0x7f1d140cefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50498/0 (socket says 172.21.15.159:50498) 2024-04-04T06:07:59.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.979+0000 7f1d12a00640 1 -- 172.21.15.159:0/4007579242 learned_addr learned my addr 172.21.15.159:0/4007579242 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:59.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.979+0000 7f1d13400640 1 --1- 172.21.15.159:0/4007579242 >> v1:172.21.15.159:6789/0 conn(0x7f1d140d3130 0x7f1d140b39a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37060/0 (socket says 172.21.15.159:37060) 2024-04-04T06:07:59.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.980+0000 7f1d12a00640 1 -- 172.21.15.159:0/4007579242 >> v1:172.21.15.178:6789/0 conn(0x7f1d140d4f10 legacy=0x7f1d140d2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:59.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.980+0000 7f1d12a00640 1 -- 172.21.15.159:0/4007579242 >> v1:172.21.15.159:6789/0 conn(0x7f1d140d3130 legacy=0x7f1d140b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:59.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.980+0000 7f1d12a00640 1 -- 172.21.15.159:0/4007579242 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d140cf910 con 0x7f1d1415e980 2024-04-04T06:07:59.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.980+0000 7f1d11600640 1 -- 172.21.15.159:0/4007579242 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3545331409 0 0) 0x7f1d140d2530 con 0x7f1d140d3130 2024-04-04T06:07:59.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.981+0000 7f1d12a00640 1 --2- 172.21.15.159:0/4007579242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d1415e980 0x7f1d140cefe0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1d04009870 tx=0x7f1d0402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a210858b4d3ba3f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:59.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.981+0000 7f1d11600640 1 -- 172.21.15.159:0/4007579242 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0402f840 con 0x7f1d1415e980 2024-04-04T06:07:59.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.981+0000 7f1d11600640 1 -- 172.21.15.159:0/4007579242 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d0402f9e0 con 0x7f1d1415e980 2024-04-04T06:07:59.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.981+0000 7f1d11600640 1 -- 172.21.15.159:0/4007579242 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0402fdb0 con 0x7f1d1415e980 2024-04-04T06:07:59.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.982+0000 7f1d19c00640 1 -- 172.21.15.159:0/4007579242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d1415e980 msgr2=0x7f1d140cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:59.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.982+0000 7f1d19c00640 1 --2- 172.21.15.159:0/4007579242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d1415e980 0x7f1d140cefe0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1d04009870 tx=0x7f1d0402ba20 comp rx=0 tx=0).stop 2024-04-04T06:07:59.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.982+0000 7f1d19c00640 1 -- 172.21.15.159:0/4007579242 shutdown_connections 2024-04-04T06:07:59.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.982+0000 7f1d19c00640 1 --2- 172.21.15.159:0/4007579242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d1415e980 0x7f1d140cefe0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:59.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.982+0000 7f1d19c00640 1 -- 172.21.15.159:0/4007579242 >> 172.21.15.159:0/4007579242 conn(0x7f1d1408bfb0 msgr2=0x7f1d140d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:07:59.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.982+0000 7f1d19c00640 1 -- 172.21.15.159:0/4007579242 shutdown_connections 2024-04-04T06:07:59.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.982+0000 7f1d19c00640 1 -- 172.21.15.159:0/4007579242 wait complete. 2024-04-04T06:07:59.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.985+0000 7f1d19c00640 1 Processor -- start 2024-04-04T06:07:59.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.985+0000 7f1d19c00640 1 -- start start 2024-04-04T06:07:59.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.986+0000 7f1d19c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d140d3130 0x7f1d141584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:59.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.987+0000 7f1d19c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d140d4f10 0x7f1d1415bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:59.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.987+0000 7f1d19c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d1415e980 0x7f1d1415c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:59.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.987+0000 7f1d19c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1d140d01b0 con 0x7f1d1415e980 2024-04-04T06:07:59.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.987+0000 7f1d19c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1d140cfed0 con 0x7f1d140d4f10 2024-04-04T06:07:59.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.987+0000 7f1d19c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1d140b40b0 con 0x7f1d140d3130 2024-04-04T06:07:59.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.987+0000 7f1d13400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d1415e980 0x7f1d1415c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:59.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.987+0000 7f1d13400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d1415e980 0x7f1d1415c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50500/0 (socket says 172.21.15.159:50500) 2024-04-04T06:07:59.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.987+0000 7f1d13400640 1 -- 172.21.15.159:0/2328516055 learned_addr learned my addr 172.21.15.159:0/2328516055 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:07:59.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.987+0000 7f1d12a00640 1 --2- 172.21.15.159:0/2328516055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d140d3130 0x7f1d141584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:59.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.987+0000 7f1d12000640 1 --2- 172.21.15.159:0/2328516055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d140d4f10 0x7f1d1415bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:59.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.987+0000 7f1d13400640 1 -- 172.21.15.159:0/2328516055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d140d3130 msgr2=0x7f1d141584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:59.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.987+0000 7f1d13400640 1 --2- 172.21.15.159:0/2328516055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d140d3130 0x7f1d141584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:59.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.987+0000 7f1d13400640 1 -- 172.21.15.159:0/2328516055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d140d4f10 msgr2=0x7f1d1415bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:07:59.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.987+0000 7f1d13400640 1 --2- 172.21.15.159:0/2328516055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d140d4f10 0x7f1d1415bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:07:59.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.987+0000 7f1d13400640 1 -- 172.21.15.159:0/2328516055 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d141607f0 con 0x7f1d1415e980 2024-04-04T06:07:59.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.988+0000 7f1d13400640 1 --2- 172.21.15.159:0/2328516055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d1415e980 0x7f1d1415c0e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c00c760 tx=0x7f1d0c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:59.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.988+0000 7f1d0aa00640 1 -- 172.21.15.159:0/2328516055 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c00a640 con 0x7f1d1415e980 2024-04-04T06:07:59.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.988+0000 7f1d0aa00640 1 -- 172.21.15.159:0/2328516055 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d0c00a7e0 con 0x7f1d1415e980 2024-04-04T06:07:59.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.988+0000 7f1d19c00640 1 -- 172.21.15.159:0/2328516055 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d1415fbe0 con 0x7f1d1415e980 2024-04-04T06:07:59.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.988+0000 7f1d0aa00640 1 -- 172.21.15.159:0/2328516055 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c00ac10 con 0x7f1d1415e980 2024-04-04T06:07:59.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.988+0000 7f1d19c00640 1 -- 172.21.15.159:0/2328516055 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d141601a0 con 0x7f1d1415e980 2024-04-04T06:07:59.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.989+0000 7f1d19c00640 1 -- 172.21.15.159:0/2328516055 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d140685c0 con 0x7f1d1415e980 2024-04-04T06:07:59.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.990+0000 7f1d0aa00640 1 -- 172.21.15.159:0/2328516055 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d0c007070 con 0x7f1d1415e980 2024-04-04T06:07:59.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.992+0000 7f1d0aa00640 1 --2- 172.21.15.159:0/2328516055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1cf406c8f0 0x7f1cf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:07:59.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.992+0000 7f1d0aa00640 1 -- 172.21.15.159:0/2328516055 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f1d0c08baf0 con 0x7f1d1415e980 2024-04-04T06:07:59.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.993+0000 7f1d12a00640 1 --2- 172.21.15.159:0/2328516055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1cf406c8f0 0x7f1cf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:07:59.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.994+0000 7f1d12a00640 1 --2- 172.21.15.159:0/2328516055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1cf406c8f0 0x7f1cf406edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1d04036000 tx=0x7f1d04008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:07:59.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:07:59.996+0000 7f1d0aa00640 1 -- 172.21.15.159:0/2328516055 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1d0c059e80 con 0x7f1d1415e980 2024-04-04T06:08:00.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.225+0000 7f1d19c00640 1 -- 172.21.15.159:0/2328516055 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d140d2800 con 0x7f1d1415e980 2024-04-04T06:08:00.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.226+0000 7f1d0aa00640 1 -- 172.21.15.159:0/2328516055 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5193 (secure 0 0 0) 0x7f1d0c059780 con 0x7f1d1415e980 2024-04-04T06:08:00.221 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:00.221 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3765578393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3765578393},{"type":"v1","addr":"172.21.15.178:6835","nonce":3765578393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3260412913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3260412913},{"type":"v1","addr":"172.21.15.178:6837","nonce":3260412913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3149916718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3149916718},{"type":"v1","addr":"172.21.15.159:6833","nonce":3149916718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:07:53.523212+0000","modified":"2024-04-04T06:07:56.601750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6835/1867245367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1867245367},{"type":"v1","addr":"172.21.15.159:6835","nonce":1867245367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5589,"qdb_cluster":[5589]},"id":3}]} 2024-04-04T06:08:00.221 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 25 2024-04-04T06:08:00.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.230+0000 7f1d19c00640 1 -- 172.21.15.159:0/2328516055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1cf406c8f0 msgr2=0x7f1cf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:00.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.230+0000 7f1d19c00640 1 --2- 172.21.15.159:0/2328516055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1cf406c8f0 0x7f1cf406edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1d04036000 tx=0x7f1d04008030 comp rx=0 tx=0).stop 2024-04-04T06:08:00.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.230+0000 7f1d19c00640 1 -- 172.21.15.159:0/2328516055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d1415e980 msgr2=0x7f1d1415c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:00.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.230+0000 7f1d19c00640 1 --2- 172.21.15.159:0/2328516055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d1415e980 0x7f1d1415c0e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c00c760 tx=0x7f1d0c00cc20 comp rx=0 tx=0).stop 2024-04-04T06:08:00.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.230+0000 7f1d19c00640 1 -- 172.21.15.159:0/2328516055 shutdown_connections 2024-04-04T06:08:00.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.230+0000 7f1d19c00640 1 --2- 172.21.15.159:0/2328516055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d1415e980 0x7f1d1415c0e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.230+0000 7f1d19c00640 1 --2- 172.21.15.159:0/2328516055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1cf406c8f0 0x7f1cf406edb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.230+0000 7f1d19c00640 1 --2- 172.21.15.159:0/2328516055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d140d4f10 0x7f1d1415bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.230+0000 7f1d19c00640 1 --2- 172.21.15.159:0/2328516055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d140d3130 0x7f1d141584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.230+0000 7f1d19c00640 1 -- 172.21.15.159:0/2328516055 >> 172.21.15.159:0/2328516055 conn(0x7f1d1408bfb0 msgr2=0x7f1d140b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:00.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.230+0000 7f1d19c00640 1 -- 172.21.15.159:0/2328516055 shutdown_connections 2024-04-04T06:08:00.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.230+0000 7f1d19c00640 1 -- 172.21.15.159:0/2328516055 wait complete. 2024-04-04T06:08:00.645 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:07:53.523212+0000', 'modified': '2024-04-04T06:07:56.601750+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5589}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5589': {'gid': 5589, 'name': 'a', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.159:6835/1867245367', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 1867245367}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 1867245367}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5589, 'qdb_cluster': [5589]} 2024-04-04T06:08:00.645 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-04T06:08:00.645 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-04T06:08:00.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.858+0000 7fe759400640 1 Processor -- start 2024-04-04T06:08:00.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.858+0000 7fe759400640 1 -- start start 2024-04-04T06:08:00.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.860+0000 7fe759400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe7540cc0b0 0x7fe75406dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:00.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.860+0000 7fe759400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe75415fc50 0x7fe7540cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:00.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.860+0000 7fe759400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe75406e4f0 con 0x7fe7540cc490 2024-04-04T06:08:00.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.860+0000 7fe759400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe75415c610 con 0x7fe75415fc50 2024-04-04T06:08:00.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.860+0000 7fe759400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe7540b3150 con 0x7fe7540cc0b0 2024-04-04T06:08:00.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.860+0000 7fe753400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe75415fc50 0x7fe7540cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:00.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.860+0000 7fe758a00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fe7540cc490 0x7fe75415ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37072/0 (socket says 172.21.15.159:37072) 2024-04-04T06:08:00.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.860+0000 7fe758a00640 1 -- 172.21.15.159:0/1825176367 learned_addr learned my addr 172.21.15.159:0/1825176367 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:00.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.860+0000 7fe753e00640 1 --2- 172.21.15.159:0/1825176367 >> v2:172.21.15.178:3300/0 conn(0x7fe7540cc0b0 0x7fe75406dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:00.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.861+0000 7fe753400640 1 -- 172.21.15.159:0/1825176367 >> v2:172.21.15.178:3300/0 conn(0x7fe7540cc0b0 msgr2=0x7fe75406dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:00.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.861+0000 7fe753400640 1 --2- 172.21.15.159:0/1825176367 >> v2:172.21.15.178:3300/0 conn(0x7fe7540cc0b0 0x7fe75406dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.861+0000 7fe753400640 1 -- 172.21.15.159:0/1825176367 >> v1:172.21.15.159:6789/0 conn(0x7fe7540cc490 legacy=0x7fe75415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:00.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.861+0000 7fe753400640 1 -- 172.21.15.159:0/1825176367 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7541600e0 con 0x7fe75415fc50 2024-04-04T06:08:00.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.861+0000 7fe752a00640 1 -- 172.21.15.159:0/1825176367 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1374986876 0 0) 0x7fe75406e4f0 con 0x7fe7540cc490 2024-04-04T06:08:00.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.862+0000 7fe753400640 1 --2- 172.21.15.159:0/1825176367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe75415fc50 0x7fe7540cfdc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe7540cfd10 tx=0x7fe74002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c642e80d8ed194 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:00.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.862+0000 7fe752a00640 1 -- 172.21.15.159:0/1825176367 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe740035020 con 0x7fe75415fc50 2024-04-04T06:08:00.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.862+0000 7fe752a00640 1 -- 172.21.15.159:0/1825176367 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe740031830 con 0x7fe75415fc50 2024-04-04T06:08:00.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.862+0000 7fe752a00640 1 -- 172.21.15.159:0/1825176367 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe740031c00 con 0x7fe75415fc50 2024-04-04T06:08:00.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.863+0000 7fe759400640 1 -- 172.21.15.159:0/1825176367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe75415fc50 msgr2=0x7fe7540cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:00.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.863+0000 7fe759400640 1 --2- 172.21.15.159:0/1825176367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe75415fc50 0x7fe7540cfdc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe7540cfd10 tx=0x7fe74002b8d0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.863+0000 7fe759400640 1 -- 172.21.15.159:0/1825176367 shutdown_connections 2024-04-04T06:08:00.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.863+0000 7fe759400640 1 --2- 172.21.15.159:0/1825176367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe75415fc50 0x7fe7540cfdc0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.863+0000 7fe759400640 1 --2- 172.21.15.159:0/1825176367 >> v2:172.21.15.178:3300/0 conn(0x7fe7540cc0b0 0x7fe75406dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.863+0000 7fe759400640 1 -- 172.21.15.159:0/1825176367 >> 172.21.15.159:0/1825176367 conn(0x7fe7540ccad0 msgr2=0x7fe7540cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:00.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.863+0000 7fe759400640 1 -- 172.21.15.159:0/1825176367 shutdown_connections 2024-04-04T06:08:00.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.863+0000 7fe759400640 1 -- 172.21.15.159:0/1825176367 wait complete. 2024-04-04T06:08:00.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.864+0000 7fe759400640 1 Processor -- start 2024-04-04T06:08:00.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.865+0000 7fe759400640 1 -- start start 2024-04-04T06:08:00.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.865+0000 7fe759400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7540cc0b0 0x7fe7540b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:00.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.865+0000 7fe759400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7540cc490 0x7fe754156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:00.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.865+0000 7fe759400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7541e6380 0x7fe7540b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:00.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.865+0000 7fe759400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe754160980 con 0x7fe7540cc490 2024-04-04T06:08:00.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.865+0000 7fe759400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe7541606a0 con 0x7fe7541e6380 2024-04-04T06:08:00.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.866+0000 7fe759400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe7540d0300 con 0x7fe7540cc0b0 2024-04-04T06:08:00.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.866+0000 7fe753400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7540cc490 0x7fe754156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:00.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.866+0000 7fe753400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7540cc490 0x7fe754156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50512/0 (socket says 172.21.15.159:50512) 2024-04-04T06:08:00.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.866+0000 7fe753400640 1 -- 172.21.15.159:0/2659159826 learned_addr learned my addr 172.21.15.159:0/2659159826 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:00.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.866+0000 7fe753e00640 1 --2- 172.21.15.159:0/2659159826 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7540cc0b0 0x7fe7540b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:00.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.866+0000 7fe753400640 1 -- 172.21.15.159:0/2659159826 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7540cc0b0 msgr2=0x7fe7540b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:00.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.866+0000 7fe753400640 1 --2- 172.21.15.159:0/2659159826 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7540cc0b0 0x7fe7540b1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.866+0000 7fe753400640 1 -- 172.21.15.159:0/2659159826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7541e6380 msgr2=0x7fe7540b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:00.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.866+0000 7fe753400640 1 --2- 172.21.15.159:0/2659159826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7541e6380 0x7fe7540b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.866+0000 7fe753400640 1 -- 172.21.15.159:0/2659159826 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7541e6800 con 0x7fe7540cc490 2024-04-04T06:08:00.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.866+0000 7fe753400640 1 --2- 172.21.15.159:0/2659159826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7540cc490 0x7fe754156100 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe740034040 tx=0x7fe7400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:00.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.867+0000 7fe74be00640 1 -- 172.21.15.159:0/2659159826 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe740041070 con 0x7fe7540cc490 2024-04-04T06:08:00.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.867+0000 7fe74be00640 1 -- 172.21.15.159:0/2659159826 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe740004250 con 0x7fe7540cc490 2024-04-04T06:08:00.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.867+0000 7fe74be00640 1 -- 172.21.15.159:0/2659159826 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe740035040 con 0x7fe7540cc490 2024-04-04T06:08:00.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.867+0000 7fe759400640 1 -- 172.21.15.159:0/2659159826 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7541f7820 con 0x7fe7540cc490 2024-04-04T06:08:00.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.867+0000 7fe759400640 1 -- 172.21.15.159:0/2659159826 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7541f7de0 con 0x7fe7540cc490 2024-04-04T06:08:00.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.869+0000 7fe74be00640 1 -- 172.21.15.159:0/2659159826 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe740038020 con 0x7fe7540cc490 2024-04-04T06:08:00.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.870+0000 7fe74be00640 1 --2- 172.21.15.159:0/2659159826 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe73406c8f0 0x7fe73406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:00.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.870+0000 7fe74be00640 1 -- 172.21.15.159:0/2659159826 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fe7400ad510 con 0x7fe7540cc490 2024-04-04T06:08:00.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.870+0000 7fe753e00640 1 --2- 172.21.15.159:0/2659159826 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe73406c8f0 0x7fe73406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:00.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.871+0000 7fe75d7d6740 1 Processor -- start 2024-04-04T06:08:00.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.871+0000 7fe75d7d6740 1 -- start start 2024-04-04T06:08:00.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.872+0000 7fe75d7d6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2ab300 0x55dd8d27e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:00.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.872+0000 7fe75d7d6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd8d2ab6e0 0x55dd8d2ae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:00.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.872+0000 7fe75d7d6740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd8d27e7e0 0x55dd8d2ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:00.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.872+0000 7fe75d7d6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dd8d245ab0 con 0x55dd8d2ab300 2024-04-04T06:08:00.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.872+0000 7fe75d7d6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dd8d1e7fa0 con 0x55dd8d2ab6e0 2024-04-04T06:08:00.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.872+0000 7fe75d7d6740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55dd8d190560 con 0x55dd8d27e7e0 2024-04-04T06:08:00.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.872+0000 7fe753e00640 1 --2- 172.21.15.159:0/2659159826 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe73406c8f0 0x7fe73406edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe744006fd0 tx=0x7fe744008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:00.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.872+0000 7fe753e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2ab300 0x55dd8d27e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:00.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.872+0000 7fe753e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2ab300 0x55dd8d27e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50518/0 (socket says 172.21.15.159:50518) 2024-04-04T06:08:00.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.872+0000 7fe753e00640 1 -- 172.21.15.159:0/2939230024 learned_addr learned my addr 172.21.15.159:0/2939230024 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:00.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.872+0000 7fe753e00640 1 -- 172.21.15.159:0/2939230024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd8d27e7e0 msgr2=0x55dd8d2ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:00.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.872+0000 7fe753e00640 1 --2- 172.21.15.159:0/2939230024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd8d27e7e0 0x55dd8d2ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.872+0000 7fe753e00640 1 -- 172.21.15.159:0/2939230024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd8d2ab6e0 msgr2=0x55dd8d2ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:00.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.872+0000 7fe753e00640 1 --2- 172.21.15.159:0/2939230024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd8d2ab6e0 0x55dd8d2ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.872+0000 7fe753e00640 1 -- 172.21.15.159:0/2939230024 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd8cefc2d0 con 0x55dd8d2ab300 2024-04-04T06:08:00.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.872+0000 7fe753e00640 1 --2- 172.21.15.159:0/2939230024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2ab300 0x55dd8d27e2a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe74400eab0 tx=0x7fe744011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d384a278344d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:00.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.873+0000 7fe73fe00640 1 -- 172.21.15.159:0/2939230024 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe744025070 con 0x55dd8d2ab300 2024-04-04T06:08:00.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.873+0000 7fe73fe00640 1 -- 172.21.15.159:0/2939230024 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe744015700 con 0x55dd8d2ab300 2024-04-04T06:08:00.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.873+0000 7fe73fe00640 1 -- 172.21.15.159:0/2939230024 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74400f040 con 0x55dd8d2ab300 2024-04-04T06:08:00.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.873+0000 7fe75d7d6740 1 -- 172.21.15.159:0/2939230024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2ab300 msgr2=0x55dd8d27e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:00.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.873+0000 7fe75d7d6740 1 --2- 172.21.15.159:0/2939230024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2ab300 0x55dd8d27e2a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe74400eab0 tx=0x7fe744011000 comp rx=0 tx=0).stop 2024-04-04T06:08:00.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.873+0000 7fe75d7d6740 1 -- 172.21.15.159:0/2939230024 shutdown_connections 2024-04-04T06:08:00.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.873+0000 7fe75d7d6740 1 --2- 172.21.15.159:0/2939230024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd8d27e7e0 0x55dd8d2ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.873+0000 7fe75d7d6740 1 --2- 172.21.15.159:0/2939230024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd8d2ab6e0 0x55dd8d2ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.873+0000 7fe75d7d6740 1 --2- 172.21.15.159:0/2939230024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2ab300 0x55dd8d27e2a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.873+0000 7fe75d7d6740 1 -- 172.21.15.159:0/2939230024 >> 172.21.15.159:0/2939230024 conn(0x55dd8d280510 msgr2=0x55dd8d2c9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:00.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.874+0000 7fe75d7d6740 1 -- 172.21.15.159:0/2939230024 shutdown_connections 2024-04-04T06:08:00.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.874+0000 7fe75d7d6740 1 -- 172.21.15.159:0/2939230024 wait complete. 2024-04-04T06:08:00.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.874+0000 7fe75d7d6740 1 -- start start 2024-04-04T06:08:00.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.874+0000 7fe75d7d6740 1 Processor -- start 2024-04-04T06:08:00.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.875+0000 7fe75d7d6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd8d2ab300 0x55dd8d2d9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:00.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.875+0000 7fe75d7d6740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd8d2ab6e0 0x55dd8d2b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:00.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.875+0000 7fe75d7d6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2c40b0 0x55dd8d2aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:00.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.875+0000 7fe75d7d6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dd8d1a3920 con 0x55dd8d2c40b0 2024-04-04T06:08:00.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.875+0000 7fe75d7d6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dd8d1e3f50 con 0x55dd8d2ab300 2024-04-04T06:08:00.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.875+0000 7fe75d7d6740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55dd8d282ab0 con 0x55dd8d2ab6e0 2024-04-04T06:08:00.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.875+0000 7fe75d7d6740 10 client.0 _ll_register_callbacks cb 0x55dd8d1a3310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:08:00.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.876+0000 7fe758a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2c40b0 0x55dd8d2aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:00.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.876+0000 7fe758a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2c40b0 0x55dd8d2aa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50522/0 (socket says 172.21.15.159:50522) 2024-04-04T06:08:00.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.876+0000 7fe758a00640 1 -- 172.21.15.159:0/477968398 learned_addr learned my addr 172.21.15.159:0/477968398 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:00.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.876+0000 7fe753e00640 1 --2- 172.21.15.159:0/477968398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd8d2ab300 0x55dd8d2d9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:00.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.876+0000 7fe753400640 1 --2- 172.21.15.159:0/477968398 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd8d2ab6e0 0x55dd8d2b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:00.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.876+0000 7fe758a00640 1 -- 172.21.15.159:0/477968398 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd8d2ab6e0 msgr2=0x55dd8d2b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:00.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.876+0000 7fe758a00640 1 --2- 172.21.15.159:0/477968398 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd8d2ab6e0 0x55dd8d2b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.876+0000 7fe758a00640 1 -- 172.21.15.159:0/477968398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd8d2ab300 msgr2=0x55dd8d2d9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:00.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.876+0000 7fe758a00640 1 --2- 172.21.15.159:0/477968398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd8d2ab300 0x55dd8d2d9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.876+0000 7fe758a00640 1 -- 172.21.15.159:0/477968398 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd8cfc66b0 con 0x55dd8d2c40b0 2024-04-04T06:08:00.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.876+0000 7fe753400640 1 --2- 172.21.15.159:0/477968398 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd8d2ab6e0 0x55dd8d2b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:00.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.876+0000 7fe758a00640 1 --2- 172.21.15.159:0/477968398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2c40b0 0x55dd8d2aa0e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00c210 tx=0x7fe74c00c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23a7fefd46443eb server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:00.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.876+0000 7fe73cc00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:08:00.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.877+0000 7fe73cc00640 1 -- 172.21.15.159:0/477968398 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c0070d0 con 0x55dd8d2c40b0 2024-04-04T06:08:00.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.877+0000 7fe73cc00640 1 -- 172.21.15.159:0/477968398 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fe74c015ad0 con 0x55dd8d2c40b0 2024-04-04T06:08:00.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.877+0000 7fe75d7d6740 1 -- 172.21.15.159:0/477968398 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe744015700 con 0x55dd8d2c40b0 2024-04-04T06:08:00.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.877+0000 7fe73cc00640 1 -- 172.21.15.159:0/477968398 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe74c015cd0 con 0x55dd8d2c40b0 2024-04-04T06:08:00.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.877+0000 7fe75d7d6740 10 client.5733 fetch_fsmap learned FSMap version 25 2024-04-04T06:08:00.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.877+0000 7fe75d7d6740 1 -- 172.21.15.159:0/477968398 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x55dd8d2c2600 con 0x55dd8d2c40b0 2024-04-04T06:08:00.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.877+0000 7fe73cc00640 1 -- 172.21.15.159:0/477968398 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe744015700 con 0x55dd8d2c40b0 2024-04-04T06:08:00.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.877+0000 7fe73cc00640 1 -- 172.21.15.159:0/477968398 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7fe74c01c050 con 0x55dd8d2c40b0 2024-04-04T06:08:00.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.877+0000 7fe75d7d6740 10 client.5733 fetch_fsmap finished waiting for FSMap version 25 2024-04-04T06:08:00.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.877+0000 7fe75d7d6740 10 client.5733 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:08:00.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.877+0000 7fe75d7d6740 20 client.5733 populate_metadata read hostname 'smithi159' 2024-04-04T06:08:00.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.877+0000 7fe75d7d6740 1 --2- 172.21.15.159:0/477968398 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55dd8d2c01a0 0x55dd8d2c1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:00.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.877+0000 7fe75d7d6740 4 client.5733 mds_command: new command op to 5589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:08:00.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.877+0000 7fe75d7d6740 1 -- 172.21.15.159:0/477968398 --> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd8d1f54e0 con 0x55dd8d2c01a0 2024-04-04T06:08:00.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.878+0000 7fe753e00640 1 --2- 172.21.15.159:0/477968398 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55dd8d2c01a0 0x55dd8d2c1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:00.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.878+0000 7fe753e00640 1 --2- 172.21.15.159:0/477968398 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55dd8d2c01a0 0x55dd8d2c1c50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da5eb65016af5541 server_cookie=1fa3131917e9f3fa in_seq=0 out_seq=0 2024-04-04T06:08:00.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.878+0000 7fe73cc00640 10 client.5733 ms_handle_connect on v2:172.21.15.159:6834/1867245367 2024-04-04T06:08:00.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.879+0000 7fe73cc00640 1 -- 172.21.15.159:0/477968398 <== mds.0 v2:172.21.15.159:6834/1867245367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd8d1f54e0 con 0x55dd8d2c01a0 2024-04-04T06:08:00.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.879+0000 7fe73cc00640 10 client.5733 handle_command_reply: tid=0 2024-04-04T06:08:00.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.879+0000 7fe75d7d6740 1 client.5733 shutdown 2024-04-04T06:08:00.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.879+0000 7fe75d7d6740 1 -- 172.21.15.159:0/477968398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2c40b0 msgr2=0x55dd8d2aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:00.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.879+0000 7fe75d7d6740 1 --2- 172.21.15.159:0/477968398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2c40b0 0x55dd8d2aa0e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00c210 tx=0x7fe74c00c6d0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.880+0000 7fe75d7d6740 1 -- 172.21.15.159:0/477968398 shutdown_connections 2024-04-04T06:08:00.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.880+0000 7fe75d7d6740 1 --2- 172.21.15.159:0/477968398 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55dd8d2c01a0 0x55dd8d2c1c50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.880+0000 7fe75d7d6740 1 --2- 172.21.15.159:0/477968398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2c40b0 0x55dd8d2aa0e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.880+0000 7fe75d7d6740 1 --2- 172.21.15.159:0/477968398 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd8d2ab6e0 0x55dd8d2b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.880+0000 7fe75d7d6740 1 --2- 172.21.15.159:0/477968398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd8d2ab300 0x55dd8d2d9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.880+0000 7fe73cc00640 0 client.5733 ms_handle_reset on v2:172.21.15.159:6834/1867245367 2024-04-04T06:08:00.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.880+0000 7fe75d7d6740 1 -- 172.21.15.159:0/477968398 >> 172.21.15.159:0/477968398 conn(0x55dd8d27e7e0 msgr2=0x55dd8d2ae080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:00.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.880+0000 7fe75d7d6740 1 -- 172.21.15.159:0/477968398 shutdown_connections 2024-04-04T06:08:00.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.880+0000 7fe75d7d6740 1 -- 172.21.15.159:0/477968398 wait complete. 2024-04-04T06:08:00.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.880+0000 7fe75d7d6740 20 client.5733 trim_cache size 0 max 16384 2024-04-04T06:08:00.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.899+0000 7fe75d7d6740 1 Processor -- start 2024-04-04T06:08:00.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.899+0000 7fe75d7d6740 1 -- start start 2024-04-04T06:08:00.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.899+0000 7fe75d7d6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd8d2ab300 0x55dd8d2c2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:00.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.900+0000 7fe75d7d6740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd8d2ab6e0 0x55dd8d2eca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:00.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.900+0000 7fe75d7d6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2c40b0 0x55dd8d2ecfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:00.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.900+0000 7fe75d7d6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe75412e2b0 con 0x55dd8d2c40b0 2024-04-04T06:08:00.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.900+0000 7fe75d7d6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe75412fca0 con 0x55dd8d2ab300 2024-04-04T06:08:00.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.900+0000 7fe75d7d6740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe75412dc60 con 0x55dd8d2ab6e0 2024-04-04T06:08:00.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.900+0000 7fe758a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2c40b0 0x55dd8d2ecfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:00.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.900+0000 7fe758a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2c40b0 0x55dd8d2ecfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50538/0 (socket says 172.21.15.159:50538) 2024-04-04T06:08:00.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.900+0000 7fe758a00640 1 -- 172.21.15.159:0/1843879350 learned_addr learned my addr 172.21.15.159:0/1843879350 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:00.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.900+0000 7fe758a00640 1 -- 172.21.15.159:0/1843879350 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd8d2ab6e0 msgr2=0x55dd8d2eca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:00.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.900+0000 7fe758a00640 1 --2- 172.21.15.159:0/1843879350 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd8d2ab6e0 0x55dd8d2eca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.900+0000 7fe758a00640 1 -- 172.21.15.159:0/1843879350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd8d2ab300 msgr2=0x55dd8d2c2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:00.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.900+0000 7fe758a00640 1 --2- 172.21.15.159:0/1843879350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd8d2ab300 0x55dd8d2c2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.900+0000 7fe758a00640 1 -- 172.21.15.159:0/1843879350 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd8d27fc70 con 0x55dd8d2c40b0 2024-04-04T06:08:00.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.900+0000 7fe758a00640 1 --2- 172.21.15.159:0/1843879350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2c40b0 0x55dd8d2ecfb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0017b0 tx=0x7fe74c006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a247ae5c96f46fe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:00.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.901+0000 7fe73f400640 1 -- 172.21.15.159:0/1843879350 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c007250 con 0x55dd8d2c40b0 2024-04-04T06:08:00.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.901+0000 7fe73f400640 1 -- 172.21.15.159:0/1843879350 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe74c0074b0 con 0x55dd8d2c40b0 2024-04-04T06:08:00.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.901+0000 7fe73f400640 1 -- 172.21.15.159:0/1843879350 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c0078a0 con 0x55dd8d2c40b0 2024-04-04T06:08:00.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.901+0000 7fe75d7d6740 1 -- 172.21.15.159:0/1843879350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2c40b0 msgr2=0x55dd8d2ecfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:00.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.901+0000 7fe75d7d6740 1 --2- 172.21.15.159:0/1843879350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2c40b0 0x55dd8d2ecfb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0017b0 tx=0x7fe74c006000 comp rx=0 tx=0).stop 2024-04-04T06:08:00.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.901+0000 7fe75d7d6740 1 -- 172.21.15.159:0/1843879350 shutdown_connections 2024-04-04T06:08:00.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.901+0000 7fe75d7d6740 1 --2- 172.21.15.159:0/1843879350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2c40b0 0x55dd8d2ecfb0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.901+0000 7fe75d7d6740 1 --2- 172.21.15.159:0/1843879350 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd8d2ab6e0 0x55dd8d2eca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.901+0000 7fe75d7d6740 1 --2- 172.21.15.159:0/1843879350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd8d2ab300 0x55dd8d2c2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.901+0000 7fe75d7d6740 1 -- 172.21.15.159:0/1843879350 >> 172.21.15.159:0/1843879350 conn(0x55dd8d27e7e0 msgr2=0x55dd8d2c48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:00.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.901+0000 7fe75d7d6740 1 -- 172.21.15.159:0/1843879350 shutdown_connections 2024-04-04T06:08:00.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.901+0000 7fe75d7d6740 1 -- 172.21.15.159:0/1843879350 wait complete. 2024-04-04T06:08:00.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.902+0000 7fe75d7d6740 1 -- start start 2024-04-04T06:08:00.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.903+0000 7fe75d7d6740 1 Processor -- start 2024-04-04T06:08:00.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.904+0000 7fe75d7d6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd8d2ab300 0x55dd8d2cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:00.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.904+0000 7fe75d7d6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2ab6e0 0x55dd8d3422d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:00.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.904+0000 7fe75d7d6740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd8d2c40b0 0x55dd8d281e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:00.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.904+0000 7fe75d7d6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dd8d2c0020 con 0x55dd8d2ab6e0 2024-04-04T06:08:00.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.904+0000 7fe75d7d6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dd8d2bfb20 con 0x55dd8d2ab300 2024-04-04T06:08:00.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.904+0000 7fe75d7d6740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55dd8d2c0670 con 0x55dd8d2c40b0 2024-04-04T06:08:00.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.904+0000 7fe75d7d6740 10 client.0 _ll_register_callbacks cb 0x7fe74c001ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:08:00.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.905+0000 7fe753400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2ab6e0 0x55dd8d3422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:00.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.905+0000 7fe753400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2ab6e0 0x55dd8d3422d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50544/0 (socket says 172.21.15.159:50544) 2024-04-04T06:08:00.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.905+0000 7fe753400640 1 -- 172.21.15.159:0/1766400981 learned_addr learned my addr 172.21.15.159:0/1766400981 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:00.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.905+0000 7fe753400640 1 -- 172.21.15.159:0/1766400981 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd8d2c40b0 msgr2=0x55dd8d281e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:00.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.905+0000 7fe753400640 1 --2- 172.21.15.159:0/1766400981 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd8d2c40b0 0x55dd8d281e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.905+0000 7fe753400640 1 -- 172.21.15.159:0/1766400981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd8d2ab300 msgr2=0x55dd8d2cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:00.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.905+0000 7fe753400640 1 --2- 172.21.15.159:0/1766400981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd8d2ab300 0x55dd8d2cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.905+0000 7fe753400640 1 -- 172.21.15.159:0/1766400981 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd8d2d9950 con 0x55dd8d2ab6e0 2024-04-04T06:08:00.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.905+0000 7fe753400640 1 --2- 172.21.15.159:0/1766400981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2ab6e0 0x55dd8d3422d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe740077690 tx=0x7fe74007b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c1ea24d8df5a30 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:00.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.905+0000 7fe73cc00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:08:00.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.905+0000 7fe73cc00640 1 -- 172.21.15.159:0/1766400981 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe740083e10 con 0x55dd8d2ab6e0 2024-04-04T06:08:00.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.905+0000 7fe73cc00640 1 -- 172.21.15.159:0/1766400981 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fe7400317e0 con 0x55dd8d2ab6e0 2024-04-04T06:08:00.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.906+0000 7fe75d7d6740 1 -- 172.21.15.159:0/1766400981 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe74c0074b0 con 0x55dd8d2ab6e0 2024-04-04T06:08:00.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.906+0000 7fe73cc00640 1 -- 172.21.15.159:0/1766400981 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe7400778a0 con 0x55dd8d2ab6e0 2024-04-04T06:08:00.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.906+0000 7fe75d7d6740 10 client.5739 fetch_fsmap learned FSMap version 25 2024-04-04T06:08:00.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.906+0000 7fe75d7d6740 1 -- 172.21.15.159:0/1766400981 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x55dd8d2d8a50 con 0x55dd8d2ab6e0 2024-04-04T06:08:00.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.906+0000 7fe73cc00640 1 -- 172.21.15.159:0/1766400981 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe74c0074b0 con 0x55dd8d2ab6e0 2024-04-04T06:08:00.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.906+0000 7fe73cc00640 1 -- 172.21.15.159:0/1766400981 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7fe74007f170 con 0x55dd8d2ab6e0 2024-04-04T06:08:00.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.906+0000 7fe75d7d6740 10 client.5739 fetch_fsmap finished waiting for FSMap version 25 2024-04-04T06:08:00.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.906+0000 7fe75d7d6740 10 client.5739 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:08:00.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.906+0000 7fe75d7d6740 20 client.5739 populate_metadata read hostname 'smithi159' 2024-04-04T06:08:00.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.906+0000 7fe75d7d6740 1 --2- 172.21.15.159:0/1766400981 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55dd8d2c8460 0x55dd8d2d79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:00.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.906+0000 7fe75d7d6740 4 client.5739 mds_command: new command op to 5589 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-04T06:08:00.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.906+0000 7fe75d7d6740 1 -- 172.21.15.159:0/1766400981 --> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] -- command(tid 0: {"prefix": "status"}) -- 0x55dd8d1ef830 con 0x55dd8d2c8460 2024-04-04T06:08:00.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.906+0000 7fe753e00640 1 --2- 172.21.15.159:0/1766400981 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55dd8d2c8460 0x55dd8d2d79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:00.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.907+0000 7fe753e00640 1 --2- 172.21.15.159:0/1766400981 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55dd8d2c8460 0x55dd8d2d79d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8e8e37ebfbedca3 server_cookie=618e5d388cdc8825 in_seq=0 out_seq=0 2024-04-04T06:08:00.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.907+0000 7fe73cc00640 10 client.5739 ms_handle_connect on v2:172.21.15.159:6834/1867245367 2024-04-04T06:08:00.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.907+0000 7fe73cc00640 1 -- 172.21.15.159:0/1766400981 <== mds.0 v2:172.21.15.159:6834/1867245367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55dd8d1ef830 con 0x55dd8d2c8460 2024-04-04T06:08:00.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.907+0000 7fe73cc00640 10 client.5739 handle_command_reply: tid=0 2024-04-04T06:08:00.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.907+0000 7fe75d7d6740 1 client.5739 shutdown 2024-04-04T06:08:00.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.907+0000 7fe75d7d6740 1 -- 172.21.15.159:0/1766400981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2ab6e0 msgr2=0x55dd8d3422d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:00.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.907+0000 7fe75d7d6740 1 --2- 172.21.15.159:0/1766400981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2ab6e0 0x55dd8d3422d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe740077690 tx=0x7fe74007b9f0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.907+0000 7fe75d7d6740 1 -- 172.21.15.159:0/1766400981 shutdown_connections 2024-04-04T06:08:00.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.907+0000 7fe75d7d6740 1 --2- 172.21.15.159:0/1766400981 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55dd8d2c8460 0x55dd8d2d79d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.907+0000 7fe75d7d6740 1 --2- 172.21.15.159:0/1766400981 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd8d2c40b0 0x55dd8d281e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.907+0000 7fe75d7d6740 1 --2- 172.21.15.159:0/1766400981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd8d2ab6e0 0x55dd8d3422d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.907+0000 7fe75d7d6740 1 --2- 172.21.15.159:0/1766400981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd8d2ab300 0x55dd8d2cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.907+0000 7fe75d7d6740 1 -- 172.21.15.159:0/1766400981 >> 172.21.15.159:0/1766400981 conn(0x55dd8d27e7e0 msgr2=0x55dd8d2c0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:00.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.907+0000 7fe73cc00640 0 client.5739 ms_handle_reset on v2:172.21.15.159:6834/1867245367 2024-04-04T06:08:00.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.907+0000 7fe75d7d6740 1 -- 172.21.15.159:0/1766400981 shutdown_connections 2024-04-04T06:08:00.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.907+0000 7fe75d7d6740 1 -- 172.21.15.159:0/1766400981 wait complete. 2024-04-04T06:08:00.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.908+0000 7fe75d7d6740 20 client.5739 trim_cache size 0 max 16384 2024-04-04T06:08:00.905 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:08:00.905 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_fsid": "30311045-f714-48fa-832b-4cb2ba8bf0bf", 2024-04-04T06:08:00.905 INFO:teuthology.orchestra.run.smithi159.stdout: "whoami": 0, 2024-04-04T06:08:00.905 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 5589, 2024-04-04T06:08:00.905 INFO:teuthology.orchestra.run.smithi159.stdout: "want_state": "up:active", 2024-04-04T06:08:00.905 INFO:teuthology.orchestra.run.smithi159.stdout: "state": "up:active", 2024-04-04T06:08:00.905 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs", 2024-04-04T06:08:00.905 INFO:teuthology.orchestra.run.smithi159.stdout: "rank_uptime": 6.8327611580000003, 2024-04-04T06:08:00.906 INFO:teuthology.orchestra.run.smithi159.stdout: "mdsmap_epoch": 25, 2024-04-04T06:08:00.906 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch": 46, 2024-04-04T06:08:00.906 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch_barrier": 46, 2024-04-04T06:08:00.906 INFO:teuthology.orchestra.run.smithi159.stdout: "uptime": 13.246123441 2024-04-04T06:08:00.906 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:08:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.908+0000 7fe759400640 1 -- 172.21.15.159:0/2659159826 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe73406c8f0 msgr2=0x7fe73406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.908+0000 7fe759400640 1 --2- 172.21.15.159:0/2659159826 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe73406c8f0 0x7fe73406edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe744006fd0 tx=0x7fe744008040 comp rx=0 tx=0).stop 2024-04-04T06:08:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.908+0000 7fe759400640 1 -- 172.21.15.159:0/2659159826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7540cc490 msgr2=0x7fe754156100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.908+0000 7fe759400640 1 --2- 172.21.15.159:0/2659159826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7540cc490 0x7fe754156100 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe740034040 tx=0x7fe7400046c0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.908+0000 7fe759400640 1 -- 172.21.15.159:0/2659159826 shutdown_connections 2024-04-04T06:08:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.908+0000 7fe759400640 1 --2- 172.21.15.159:0/2659159826 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe73406c8f0 0x7fe73406edb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.908+0000 7fe759400640 1 --2- 172.21.15.159:0/2659159826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7541e6380 0x7fe7540b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.908+0000 7fe759400640 1 --2- 172.21.15.159:0/2659159826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7540cc490 0x7fe754156100 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.908+0000 7fe759400640 1 --2- 172.21.15.159:0/2659159826 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7540cc0b0 0x7fe7540b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.908+0000 7fe759400640 1 -- 172.21.15.159:0/2659159826 >> 172.21.15.159:0/2659159826 conn(0x7fe7540ccad0 msgr2=0x7fe75415c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.908+0000 7fe759400640 1 -- 172.21.15.159:0/2659159826 shutdown_connections 2024-04-04T06:08:00.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:00.908+0000 7fe759400640 1 -- 172.21.15.159:0/2659159826 wait complete. 2024-04-04T06:08:00.920 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-04T06:08:01.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.132+0000 7ff988200640 1 Processor -- start 2024-04-04T06:08:01.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.133+0000 7ff988200640 1 -- start start 2024-04-04T06:08:01.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.134+0000 7ff988200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff9800d43d0 0x7ff9800d47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:01.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.134+0000 7ff988200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9800d4d20 con 0x7ff98015e860 2024-04-04T06:08:01.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.134+0000 7ff988200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff98006dfa0 con 0x7ff9800cf6b0 2024-04-04T06:08:01.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.134+0000 7ff988200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7ff9800cf220 con 0x7ff9800d43d0 2024-04-04T06:08:01.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.135+0000 7ff984e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff9800d43d0 0x7ff9800d47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:01.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.135+0000 7ff984e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff9800d43d0 0x7ff9800d47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50550/0 (socket says 172.21.15.159:50550) 2024-04-04T06:08:01.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.135+0000 7ff984e00640 1 -- 172.21.15.159:0/3309118598 learned_addr learned my addr 172.21.15.159:0/3309118598 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:01.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.135+0000 7ff985800640 1 --1- 172.21.15.159:0/3309118598 >> v1:172.21.15.159:6789/0 conn(0x7ff98015e860 0x7ff98006d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37074/0 (socket says 172.21.15.159:37074) 2024-04-04T06:08:01.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.135+0000 7ff984e00640 1 -- 172.21.15.159:0/3309118598 >> v1:172.21.15.178:6789/0 conn(0x7ff9800cf6b0 legacy=0x7ff9800cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:01.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.135+0000 7ff984e00640 1 -- 172.21.15.159:0/3309118598 >> v1:172.21.15.159:6789/0 conn(0x7ff98015e860 legacy=0x7ff98006d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:01.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.135+0000 7ff97fe00640 1 -- 172.21.15.159:0/3309118598 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2793092109 0 0) 0x7ff974001110 con 0x7ff98015e860 2024-04-04T06:08:01.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.135+0000 7ff984e00640 1 -- 172.21.15.159:0/3309118598 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff98006e240 con 0x7ff9800d43d0 2024-04-04T06:08:01.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.136+0000 7ff984e00640 1 --2- 172.21.15.159:0/3309118598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9800d43d0 0x7ff9800d47e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff98006e170 tx=0x7ff96c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d4fac623e4232d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:01.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.137+0000 7ff97fe00640 1 -- 172.21.15.159:0/3309118598 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c02f840 con 0x7ff9800d43d0 2024-04-04T06:08:01.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.137+0000 7ff97fe00640 1 -- 172.21.15.159:0/3309118598 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff96c02f9e0 con 0x7ff9800d43d0 2024-04-04T06:08:01.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.137+0000 7ff97fe00640 1 -- 172.21.15.159:0/3309118598 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c02fdb0 con 0x7ff9800d43d0 2024-04-04T06:08:01.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.137+0000 7ff988200640 1 -- 172.21.15.159:0/3309118598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9800d43d0 msgr2=0x7ff9800d47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:01.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.137+0000 7ff988200640 1 --2- 172.21.15.159:0/3309118598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9800d43d0 0x7ff9800d47e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff98006e170 tx=0x7ff96c02ba20 comp rx=0 tx=0).stop 2024-04-04T06:08:01.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.137+0000 7ff988200640 1 -- 172.21.15.159:0/3309118598 shutdown_connections 2024-04-04T06:08:01.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.137+0000 7ff988200640 1 --2- 172.21.15.159:0/3309118598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9800d43d0 0x7ff9800d47e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:01.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.137+0000 7ff988200640 1 -- 172.21.15.159:0/3309118598 >> 172.21.15.159:0/3309118598 conn(0x7ff98009d7d0 msgr2=0x7ff98008bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:01.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.137+0000 7ff988200640 1 -- 172.21.15.159:0/3309118598 shutdown_connections 2024-04-04T06:08:01.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.137+0000 7ff988200640 1 -- 172.21.15.159:0/3309118598 wait complete. 2024-04-04T06:08:01.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.139+0000 7ff988200640 1 Processor -- start 2024-04-04T06:08:01.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.139+0000 7ff988200640 1 -- start start 2024-04-04T06:08:01.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.140+0000 7ff988200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9800cf6b0 0x7ff9801615a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:01.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.140+0000 7ff988200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9800d43d0 0x7ff9801586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:01.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.140+0000 7ff988200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff98015e860 0x7ff9800cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:01.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.140+0000 7ff988200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff98006dc00 con 0x7ff98015e860 2024-04-04T06:08:01.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.140+0000 7ff988200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9800d2c50 con 0x7ff9800d43d0 2024-04-04T06:08:01.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.140+0000 7ff988200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff98006dd80 con 0x7ff9800cf6b0 2024-04-04T06:08:01.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.140+0000 7ff986200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff98015e860 0x7ff9800cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:01.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.141+0000 7ff986200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff98015e860 0x7ff9800cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50556/0 (socket says 172.21.15.159:50556) 2024-04-04T06:08:01.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.141+0000 7ff986200640 1 -- 172.21.15.159:0/2906434988 learned_addr learned my addr 172.21.15.159:0/2906434988 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:01.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.141+0000 7ff984e00640 1 --2- 172.21.15.159:0/2906434988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9800d43d0 0x7ff9801586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:01.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.141+0000 7ff985800640 1 --2- 172.21.15.159:0/2906434988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9800cf6b0 0x7ff9801615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:01.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.141+0000 7ff986200640 1 -- 172.21.15.159:0/2906434988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9800cf6b0 msgr2=0x7ff9801615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:01.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.141+0000 7ff986200640 1 --2- 172.21.15.159:0/2906434988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9800cf6b0 0x7ff9801615a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:01.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.141+0000 7ff986200640 1 -- 172.21.15.159:0/2906434988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9800d43d0 msgr2=0x7ff9801586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:01.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.141+0000 7ff986200640 1 --2- 172.21.15.159:0/2906434988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9800d43d0 0x7ff9801586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:01.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.141+0000 7ff986200640 1 -- 172.21.15.159:0/2906434988 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9800c8010 con 0x7ff98015e860 2024-04-04T06:08:01.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.141+0000 7ff986200640 1 --2- 172.21.15.159:0/2906434988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff98015e860 0x7ff9800cd520 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff97800c6b0 tx=0x7ff97800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:01.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.141+0000 7ff97d600640 1 -- 172.21.15.159:0/2906434988 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978003b70 con 0x7ff98015e860 2024-04-04T06:08:01.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.142+0000 7ff97d600640 1 -- 172.21.15.159:0/2906434988 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff978003d30 con 0x7ff98015e860 2024-04-04T06:08:01.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.142+0000 7ff97d600640 1 -- 172.21.15.159:0/2906434988 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978005310 con 0x7ff98015e860 2024-04-04T06:08:01.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.142+0000 7ff988200640 1 -- 172.21.15.159:0/2906434988 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9801606e0 con 0x7ff98015e860 2024-04-04T06:08:01.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.142+0000 7ff988200640 1 -- 172.21.15.159:0/2906434988 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9800c8280 con 0x7ff98015e860 2024-04-04T06:08:01.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.143+0000 7ff988200640 1 -- 172.21.15.159:0/2906434988 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9800d31e0 con 0x7ff98015e860 2024-04-04T06:08:01.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.144+0000 7ff97d600640 1 -- 172.21.15.159:0/2906434988 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9780054b0 con 0x7ff98015e860 2024-04-04T06:08:01.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.145+0000 7ff97d600640 1 --2- 172.21.15.159:0/2906434988 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff96006c8f0 0x7ff96006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:01.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.145+0000 7ff97d600640 1 -- 172.21.15.159:0/2906434988 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7ff97808ba40 con 0x7ff98015e860 2024-04-04T06:08:01.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.145+0000 7ff985800640 1 --2- 172.21.15.159:0/2906434988 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff96006c8f0 0x7ff96006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:01.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.147+0000 7ff985800640 1 --2- 172.21.15.159:0/2906434988 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff96006c8f0 0x7ff96006edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff974000f60 tx=0x7ff974007970 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:01.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.149+0000 7ff97d600640 1 -- 172.21.15.159:0/2906434988 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff9800d31e0 con 0x7ff98015e860 2024-04-04T06:08:01.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.388+0000 7ff988200640 1 -- 172.21.15.159:0/2906434988 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff9800c8450 con 0x7ff98015e860 2024-04-04T06:08:01.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.388+0000 7ff97d600640 1 -- 172.21.15.159:0/2906434988 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7ff97808be20 con 0x7ff98015e860 2024-04-04T06:08:01.382 INFO:teuthology.orchestra.run.smithi159.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-04T06:08:01.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.390+0000 7ff988200640 1 -- 172.21.15.159:0/2906434988 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff96006c8f0 msgr2=0x7ff96006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:01.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.390+0000 7ff988200640 1 --2- 172.21.15.159:0/2906434988 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff96006c8f0 0x7ff96006edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff974000f60 tx=0x7ff974007970 comp rx=0 tx=0).stop 2024-04-04T06:08:01.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.390+0000 7ff988200640 1 -- 172.21.15.159:0/2906434988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff98015e860 msgr2=0x7ff9800cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:01.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.390+0000 7ff988200640 1 --2- 172.21.15.159:0/2906434988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff98015e860 0x7ff9800cd520 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff97800c6b0 tx=0x7ff97800cb70 comp rx=0 tx=0).stop 2024-04-04T06:08:01.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.390+0000 7ff988200640 1 -- 172.21.15.159:0/2906434988 shutdown_connections 2024-04-04T06:08:01.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.390+0000 7ff988200640 1 --2- 172.21.15.159:0/2906434988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff98015e860 0x7ff9800cd520 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:01.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.390+0000 7ff988200640 1 --2- 172.21.15.159:0/2906434988 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff96006c8f0 0x7ff96006edb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:01.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.390+0000 7ff988200640 1 --2- 172.21.15.159:0/2906434988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9800d43d0 0x7ff9801586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:01.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.390+0000 7ff988200640 1 --2- 172.21.15.159:0/2906434988 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9800cf6b0 0x7ff9801615a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:01.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.390+0000 7ff988200640 1 -- 172.21.15.159:0/2906434988 >> 172.21.15.159:0/2906434988 conn(0x7ff98009d7d0 msgr2=0x7ff9800b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:01.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.391+0000 7ff988200640 1 -- 172.21.15.159:0/2906434988 shutdown_connections 2024-04-04T06:08:01.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01.391+0000 7ff988200640 1 -- 172.21.15.159:0/2906434988 wait complete. 2024-04-04T06:08:01.401 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-04T06:08:01.402 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-04T06:08:01.402 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi159.front.sepia.ceph.com 2024-04-04T06:08:01.402 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-04T06:08:01.402 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-04T06:08:01.402 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-04T06:08:01.402 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-04T06:08:01.402 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-04T06:08:01.402 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:08:01.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:08:01.402 DEBUG:teuthology.orchestra.run.smithi159:> ip addr 2024-04-04T06:08:01.451 INFO:teuthology.orchestra.run.smithi159.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:08:01.451 INFO:teuthology.orchestra.run.smithi159.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:08:01.451 INFO:teuthology.orchestra.run.smithi159.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:08:01.452 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:08:01.452 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 ::1/128 scope host 2024-04-04T06:08:01.452 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:08:01.452 INFO:teuthology.orchestra.run.smithi159.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:08:01.452 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c6 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:08:01.452 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f0 2024-04-04T06:08:01.452 INFO:teuthology.orchestra.run.smithi159.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:08:01.452 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c7 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:08:01.452 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f1 2024-04-04T06:08:01.452 INFO:teuthology.orchestra.run.smithi159.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:08:01.452 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4e brd ff:ff:ff:ff:ff:ff 2024-04-04T06:08:01.452 INFO:teuthology.orchestra.run.smithi159.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-04T06:08:01.452 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:08:01.452 INFO:teuthology.orchestra.run.smithi159.stdout: inet 172.21.15.159/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-04T06:08:01.452 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft 42451sec preferred_lft 42451sec 2024-04-04T06:08:01.452 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::ec4:7aff:fe88:724f/64 scope link 2024-04-04T06:08:01.453 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:08:01.453 INFO:teuthology.orchestra.run.smithi159.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-04T06:08:01.453 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:08:01.453 INFO:teuthology.orchestra.run.smithi159.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-04T06:08:01.453 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:08:01.453 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::989e:e4ff:fea5:7568/64 scope link 2024-04-04T06:08:01.453 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:08:01.453 INFO:teuthology.orchestra.run.smithi159.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:08:01.453 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether a2:e3:e0:cb:0a:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-04T06:08:01.453 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::a0e3:e0ff:fecb:a73/64 scope link 2024-04-04T06:08:01.453 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:08:01.453 INFO:teuthology.orchestra.run.smithi159.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:08:01.453 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-04T06:08:01.453 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::c0a:8aff:fe54:b58f/64 scope link 2024-04-04T06:08:01.454 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:08:01.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:08:01.454 DEBUG:teuthology.orchestra.run.smithi159:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-04T06:08:01.582 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:08:01.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:08:01.588 DEBUG:teuthology.orchestra.run.smithi159:> route 2024-04-04T06:08:01.634 INFO:teuthology.orchestra.run.smithi159.stdout:Kernel IP routing table 2024-04-04T06:08:01.634 INFO:teuthology.orchestra.run.smithi159.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-04T06:08:01.634 INFO:teuthology.orchestra.run.smithi159.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-04T06:08:01.634 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-04T06:08:01.635 INFO:teuthology.orchestra.run.smithi159.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-04T06:08:01.635 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:08:01.635 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:08:01.635 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-04T06:08:01.635 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-04T06:08:01.635 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-04T06:08:01.635 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:08:01.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:08:01.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:08:01.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:08:01.928 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-04-04T06:08:01.971 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-04T06:08:01.971 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-04T06:08:01.972 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:08:01.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:08:01.972 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set brx.0 up 2024-04-04T06:08:02.107 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:08:02.108 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-04T06:08:02.151 INFO:teuthology.orchestra.run.smithi159.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:08:02.152 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:08:02.152 DEBUG:teuthology.orchestra.run.smithi159:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:08:02.240 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-04T06:08:02.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:08:02.240 DEBUG:teuthology.orchestra.run.smithi159:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:08:02.414 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:08:02.414 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:08:02.414 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:08:02.434 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:08:02.435 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:08:02.435 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:08:02.435 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:08:02.435 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:08:02.461 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:08:02.461 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:08:02.461 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:08:02.461 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:08:02.461 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:08:02.461 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-04T06:08:02.461 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-04T06:08:02.461 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30311045-f714-48fa-832b-4cb2ba8bf0bf" will pass to kernel 2024-04-04T06:08:02.466 INFO:teuthology.orchestra.run.smithi159.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-04T06:08:02.466 INFO:teuthology.orchestra.run.smithi159.stdout: You just mounted a file system that supports labels which does not 2024-04-04T06:08:02.470 INFO:teuthology.orchestra.run.smithi159.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-04T06:08:02.470 INFO:teuthology.orchestra.run.smithi159.stdout: applications will generate AVC messages and not be allowed access to 2024-04-04T06:08:02.470 INFO:teuthology.orchestra.run.smithi159.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-04T06:08:02.471 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-04T06:08:02.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:08:02.471 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:08:02.555 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:08:02.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:08:02.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:08:02.675 INFO:teuthology.orchestra.run.smithi159.stdout:client5751 2024-04-04T06:08:02.675 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:08:02.676 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:08:02.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:08:02.723 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-04-04T06:08:02.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:08:02.822 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-04-04T06:08:02.871 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:08:02.872 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:08:02.872 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:08:02.872 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:08:02.872 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-04-04T06:08:02.872 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:08:02.994 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627777 2024-04-04T06:08:02.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:08:03.006 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:08:03.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.262+0000 7f0262e00640 1 Processor -- start 2024-04-04T06:08:03.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.263+0000 7f0262e00640 1 -- start start 2024-04-04T06:08:03.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.264+0000 7f0262e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f025c15c390 0x7f025c073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:03.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.264+0000 7f0262e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f025c15fe80 con 0x7f025c0d0650 2024-04-04T06:08:03.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.264+0000 7f0262e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f025c0ad120 con 0x7f025c0af830 2024-04-04T06:08:03.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.264+0000 7f0262e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f025c15c770 con 0x7f025c15c390 2024-04-04T06:08:03.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.264+0000 7f0260e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f025c0d0650 0x7f025c071cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37084/0 (socket says 172.21.15.159:37084) 2024-04-04T06:08:03.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.264+0000 7f0260e00640 1 -- 172.21.15.159:0/3912058420 learned_addr learned my addr 172.21.15.159:0/3912058420 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:03.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.265+0000 7f025be00640 1 --2- 172.21.15.159:0/3912058420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f025c15c390 0x7f025c073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:03.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.265+0000 7f025b400640 1 -- 172.21.15.159:0/3912058420 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2213322520 0 0) 0x7f025c15fe80 con 0x7f025c0d0650 2024-04-04T06:08:03.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.265+0000 7f025b400640 1 -- 172.21.15.159:0/3912058420 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0240003520 con 0x7f025c0d0650 2024-04-04T06:08:03.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.265+0000 7f025b400640 1 -- 172.21.15.159:0/3912058420 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 989968723 0 0) 0x7f0240003520 con 0x7f025c0d0650 2024-04-04T06:08:03.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.265+0000 7f025b400640 1 -- 172.21.15.159:0/3912058420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f025c15c390 msgr2=0x7f025c073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:03.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.265+0000 7f025b400640 1 --2- 172.21.15.159:0/3912058420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f025c15c390 0x7f025c073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:03.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.265+0000 7f025b400640 1 -- 172.21.15.159:0/3912058420 >> v1:172.21.15.178:6789/0 conn(0x7f025c0af830 legacy=0x7f025c160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:03.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.265+0000 7f025b400640 1 -- 172.21.15.159:0/3912058420 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025c0ad400 con 0x7f025c0d0650 2024-04-04T06:08:03.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.265+0000 7f025b400640 1 -- 172.21.15.159:0/3912058420 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f02540019f0 con 0x7f025c0d0650 2024-04-04T06:08:03.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.265+0000 7f025b400640 1 -- 172.21.15.159:0/3912058420 >> v1:172.21.15.159:6789/0 conn(0x7f025c0d0650 legacy=0x7f025c071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:03.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.266+0000 7f025b400640 1 --2- 172.21.15.159:0/3912058420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f025c15c390 0x7f02400039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:03.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.266+0000 7f025b400640 1 --2- 172.21.15.159:0/3912058420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0240003f40 0x7f0240004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:03.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.266+0000 7f025b400640 1 --2- 172.21.15.159:0/3912058420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0240004d40 0x7f02400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:03.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.266+0000 7f025b400640 1 -- 172.21.15.159:0/3912058420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f024000fe80 con 0x7f0240003f40 2024-04-04T06:08:03.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.266+0000 7f025b400640 1 -- 172.21.15.159:0/3912058420 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0240010030 con 0x7f0240004d40 2024-04-04T06:08:03.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.266+0000 7f025b400640 1 -- 172.21.15.159:0/3912058420 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0240010230 con 0x7f025c15c390 2024-04-04T06:08:03.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.266+0000 7f0253e00640 1 --2- 172.21.15.159:0/3912058420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0240003f40 0x7f0240004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:03.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.266+0000 7f0260e00640 1 --2- 172.21.15.159:0/3912058420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0240004d40 0x7f02400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:03.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.266+0000 7f025be00640 1 --2- 172.21.15.159:0/3912058420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f025c15c390 0x7f02400039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:03.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.266+0000 7f0253e00640 1 -- 172.21.15.159:0/3912058420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f025c15c390 msgr2=0x7f02400039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:03.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.266+0000 7f0253e00640 1 --2- 172.21.15.159:0/3912058420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f025c15c390 0x7f02400039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:03.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.266+0000 7f0253e00640 1 -- 172.21.15.159:0/3912058420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0240004d40 msgr2=0x7f02400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:03.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.266+0000 7f0253e00640 1 --2- 172.21.15.159:0/3912058420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0240004d40 0x7f02400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:03.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.266+0000 7f0253e00640 1 -- 172.21.15.159:0/3912058420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0240003520 con 0x7f0240003f40 2024-04-04T06:08:03.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.267+0000 7f0253e00640 1 --2- 172.21.15.159:0/3912058420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0240003f40 0x7f0240004800 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0244003db0 tx=0x7f0240005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324f50e1a1517c96 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:03.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.268+0000 7f025b400640 1 -- 172.21.15.159:0/3912058420 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244037070 con 0x7f0240003f40 2024-04-04T06:08:03.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.268+0000 7f025b400640 1 -- 172.21.15.159:0/3912058420 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0244032600 con 0x7f0240003f40 2024-04-04T06:08:03.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.268+0000 7f025b400640 1 -- 172.21.15.159:0/3912058420 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244003980 con 0x7f0240003f40 2024-04-04T06:08:03.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.268+0000 7f0262e00640 1 -- 172.21.15.159:0/3912058420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0240003f40 msgr2=0x7f0240004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:03.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.268+0000 7f0262e00640 1 --2- 172.21.15.159:0/3912058420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0240003f40 0x7f0240004800 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0244003db0 tx=0x7f0240005830 comp rx=0 tx=0).stop 2024-04-04T06:08:03.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.268+0000 7f0260e00640 1 -- 172.21.15.159:0/3912058420 reap_dead start 2024-04-04T06:08:03.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.269+0000 7f0262e00640 1 -- 172.21.15.159:0/3912058420 shutdown_connections 2024-04-04T06:08:03.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.269+0000 7f0262e00640 1 -- 172.21.15.159:0/3912058420 >> 172.21.15.159:0/3912058420 conn(0x7f025c06b990 msgr2=0x7f025c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:03.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.269+0000 7f0262e00640 1 -- 172.21.15.159:0/3912058420 shutdown_connections 2024-04-04T06:08:03.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.269+0000 7f0262e00640 1 -- 172.21.15.159:0/3912058420 wait complete. 2024-04-04T06:08:03.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.270+0000 7f0262e00640 1 Processor -- start 2024-04-04T06:08:03.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.271+0000 7f0262e00640 1 -- start start 2024-04-04T06:08:03.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.271+0000 7f0262e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0240003f40 0x7f025c1524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:03.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.271+0000 7f0262e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f025c15f1a0 0x7f025c158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:03.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.271+0000 7f0262e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f025c1556b0 0x7f025c155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:03.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.271+0000 7f0262e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f025c160d50 con 0x7f025c1556b0 2024-04-04T06:08:03.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.271+0000 7f0262e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f025c160120 con 0x7f025c15f1a0 2024-04-04T06:08:03.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.271+0000 7f0262e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f025c0cf980 con 0x7f0240003f40 2024-04-04T06:08:03.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.272+0000 7f0260e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f025c1556b0 0x7f025c155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:03.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.272+0000 7f0260e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f025c1556b0 0x7f025c155b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50578/0 (socket says 172.21.15.159:50578) 2024-04-04T06:08:03.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.272+0000 7f0260e00640 1 -- 172.21.15.159:0/1862880502 learned_addr learned my addr 172.21.15.159:0/1862880502 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:03.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.272+0000 7f0253e00640 1 --2- 172.21.15.159:0/1862880502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f025c15f1a0 0x7f025c158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:03.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.272+0000 7f025be00640 1 --2- 172.21.15.159:0/1862880502 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0240003f40 0x7f025c1524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:03.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.272+0000 7f0260e00640 1 -- 172.21.15.159:0/1862880502 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0240003f40 msgr2=0x7f025c1524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:03.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.272+0000 7f0260e00640 1 --2- 172.21.15.159:0/1862880502 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0240003f40 0x7f025c1524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:03.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.272+0000 7f0260e00640 1 -- 172.21.15.159:0/1862880502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f025c15f1a0 msgr2=0x7f025c158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:03.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.272+0000 7f0260e00640 1 --2- 172.21.15.159:0/1862880502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f025c15f1a0 0x7f025c158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:03.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.272+0000 7f0260e00640 1 -- 172.21.15.159:0/1862880502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025c15ace0 con 0x7f025c1556b0 2024-04-04T06:08:03.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.273+0000 7f0260e00640 1 --2- 172.21.15.159:0/1862880502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f025c1556b0 0x7f025c155b30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f025c0d1330 tx=0x7f025400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:03.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.273+0000 7f0258c00640 1 -- 172.21.15.159:0/1862880502 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0254007aa0 con 0x7f025c1556b0 2024-04-04T06:08:03.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.273+0000 7f0258c00640 1 -- 172.21.15.159:0/1862880502 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0254018070 con 0x7f025c1556b0 2024-04-04T06:08:03.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.273+0000 7f0258c00640 1 -- 172.21.15.159:0/1862880502 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0254013420 con 0x7f025c1556b0 2024-04-04T06:08:03.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.273+0000 7f0262e00640 1 -- 172.21.15.159:0/1862880502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025c15afc0 con 0x7f025c1556b0 2024-04-04T06:08:03.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.273+0000 7f0262e00640 1 -- 172.21.15.159:0/1862880502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f025c15a9d0 con 0x7f025c1556b0 2024-04-04T06:08:03.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.274+0000 7f0262e00640 1 -- 172.21.15.159:0/1862880502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f025c15b680 con 0x7f025c1556b0 2024-04-04T06:08:03.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.275+0000 7f0258c00640 1 -- 172.21.15.159:0/1862880502 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f02540135e0 con 0x7f025c1556b0 2024-04-04T06:08:03.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.276+0000 7f0258c00640 1 --2- 172.21.15.159:0/1862880502 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f023806c940 0x7f023806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:03.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.276+0000 7f0258c00640 1 -- 172.21.15.159:0/1862880502 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f025408bf10 con 0x7f025c1556b0 2024-04-04T06:08:03.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.277+0000 7f025be00640 1 --2- 172.21.15.159:0/1862880502 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f023806c940 0x7f023806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:03.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.278+0000 7f025be00640 1 --2- 172.21.15.159:0/1862880502 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f023806c940 0x7f023806ee00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f024c009950 tx=0x7f024c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:03.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.281+0000 7f0258c00640 1 -- 172.21.15.159:0/1862880502 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f025c15b680 con 0x7f025c1556b0 2024-04-04T06:08:03.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.519+0000 7f0262e00640 1 -- 172.21.15.159:0/1862880502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f025c0685c0 con 0x7f025c1556b0 2024-04-04T06:08:03.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.520+0000 7f0258c00640 1 -- 172.21.15.159:0/1862880502 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5193 (secure 0 0 0) 0x7f025405a3b0 con 0x7f025c1556b0 2024-04-04T06:08:03.514 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:03.514 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3765578393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3765578393},{"type":"v1","addr":"172.21.15.178:6835","nonce":3765578393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3260412913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3260412913},{"type":"v1","addr":"172.21.15.178:6837","nonce":3260412913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3149916718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3149916718},{"type":"v1","addr":"172.21.15.159:6833","nonce":3149916718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:07:53.523212+0000","modified":"2024-04-04T06:07:56.601750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6835/1867245367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1867245367},{"type":"v1","addr":"172.21.15.159:6835","nonce":1867245367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5589,"qdb_cluster":[5589]},"id":3}]} 2024-04-04T06:08:03.515 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 25 2024-04-04T06:08:03.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.522+0000 7f0262e00640 1 -- 172.21.15.159:0/1862880502 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f023806c940 msgr2=0x7f023806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:03.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.522+0000 7f0262e00640 1 --2- 172.21.15.159:0/1862880502 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f023806c940 0x7f023806ee00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f024c009950 tx=0x7f024c008040 comp rx=0 tx=0).stop 2024-04-04T06:08:03.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.522+0000 7f0262e00640 1 -- 172.21.15.159:0/1862880502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f025c1556b0 msgr2=0x7f025c155b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:03.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.522+0000 7f0262e00640 1 --2- 172.21.15.159:0/1862880502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f025c1556b0 0x7f025c155b30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f025c0d1330 tx=0x7f025400e530 comp rx=0 tx=0).stop 2024-04-04T06:08:03.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.523+0000 7f0262e00640 1 -- 172.21.15.159:0/1862880502 shutdown_connections 2024-04-04T06:08:03.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.523+0000 7f0262e00640 1 --2- 172.21.15.159:0/1862880502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f025c1556b0 0x7f025c155b30 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:03.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.523+0000 7f0262e00640 1 --2- 172.21.15.159:0/1862880502 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f023806c940 0x7f023806ee00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:03.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.523+0000 7f0262e00640 1 --2- 172.21.15.159:0/1862880502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f025c15f1a0 0x7f025c158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:03.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.523+0000 7f0262e00640 1 --2- 172.21.15.159:0/1862880502 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0240003f40 0x7f025c1524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:03.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.523+0000 7f0262e00640 1 -- 172.21.15.159:0/1862880502 >> 172.21.15.159:0/1862880502 conn(0x7f025c06b990 msgr2=0x7f025c06e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:03.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.523+0000 7f0262e00640 1 -- 172.21.15.159:0/1862880502 shutdown_connections 2024-04-04T06:08:03.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.523+0000 7f0262e00640 1 -- 172.21.15.159:0/1862880502 wait complete. 2024-04-04T06:08:03.538 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-04T06:08:03.783 INFO:teuthology.orchestra.run.smithi159.stdout:{"message":"","return_code":0} 2024-04-04T06:08:03.783 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-04T06:08:03.783 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-04T06:08:03.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.881+0000 7f1d97e0ef40 1 Processor -- start 2024-04-04T06:08:03.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.881+0000 7f1d97e0ef40 1 -- start start 2024-04-04T06:08:03.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.881+0000 7f1d97e0ef40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d780022d0 0x55d64b0e7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:03.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.882+0000 7f1d97e0ef40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64b04df30 0x55d64b04caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:03.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.882+0000 7f1d97e0ef40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d64b04d030 0x55d64b019ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:03.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.882+0000 7f1d97e0ef40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d64afc22c0 con 0x55d64b04d030 2024-04-04T06:08:03.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.882+0000 7f1d97e0ef40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d64afc6310 con 0x55d64b04df30 2024-04-04T06:08:03.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.882+0000 7f1d97e0ef40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55d64b01c390 con 0x7f1d780022d0 2024-04-04T06:08:03.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.882+0000 7f1d94e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d64b04d030 0x55d64b019ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:03.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.882+0000 7f1d94e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d64b04d030 0x55d64b019ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45432/0 (socket says 172.21.15.159:45432) 2024-04-04T06:08:03.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.882+0000 7f1d94e00640 1 -- 172.21.15.159:0/2575437430 learned_addr learned my addr 172.21.15.159:0/2575437430 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:03.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.882+0000 7f1d94400640 1 --2- 172.21.15.159:0/2575437430 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d780022d0 0x55d64b0e7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:03.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.882+0000 7f1d93a00640 1 --2- 172.21.15.159:0/2575437430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64b04df30 0x55d64b04caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:03.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.882+0000 7f1d94e00640 1 -- 172.21.15.159:0/2575437430 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d780022d0 msgr2=0x55d64b0e7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:03.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.882+0000 7f1d94e00640 1 --2- 172.21.15.159:0/2575437430 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d780022d0 0x55d64b0e7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:03.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.882+0000 7f1d94e00640 1 -- 172.21.15.159:0/2575437430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64b04df30 msgr2=0x55d64b04caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:03.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.882+0000 7f1d94e00640 1 --2- 172.21.15.159:0/2575437430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64b04df30 0x55d64b04caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:03.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.882+0000 7f1d94e00640 1 -- 172.21.15.159:0/2575437430 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d64b049e60 con 0x55d64b04d030 2024-04-04T06:08:03.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.882+0000 7f1d93a00640 1 --2- 172.21.15.159:0/2575437430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64b04df30 0x55d64b04caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:03.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.883+0000 7f1d94e00640 1 --2- 172.21.15.159:0/2575437430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d64b04d030 0x55d64b019ad0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c005fd0 tx=0x7f1d8c002aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481204668d7709a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:03.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.883+0000 7f1d92600640 1 -- 172.21.15.159:0/2575437430 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c012680 con 0x55d64b04d030 2024-04-04T06:08:03.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.883+0000 7f1d92600640 1 -- 172.21.15.159:0/2575437430 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d8c018070 con 0x55d64b04d030 2024-04-04T06:08:03.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.883+0000 7f1d92600640 1 -- 172.21.15.159:0/2575437430 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c012a60 con 0x55d64b04d030 2024-04-04T06:08:03.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.883+0000 7f1d97e0ef40 1 -- 172.21.15.159:0/2575437430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d64b04d030 msgr2=0x55d64b019ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:03.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.883+0000 7f1d97e0ef40 1 --2- 172.21.15.159:0/2575437430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d64b04d030 0x55d64b019ad0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c005fd0 tx=0x7f1d8c002aa0 comp rx=0 tx=0).stop 2024-04-04T06:08:03.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.884+0000 7f1d97e0ef40 1 -- 172.21.15.159:0/2575437430 shutdown_connections 2024-04-04T06:08:03.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.884+0000 7f1d97e0ef40 1 --2- 172.21.15.159:0/2575437430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d64b04d030 0x55d64b019ad0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:03.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.884+0000 7f1d97e0ef40 1 --2- 172.21.15.159:0/2575437430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64b04df30 0x55d64b04caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:03.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.884+0000 7f1d97e0ef40 1 --2- 172.21.15.159:0/2575437430 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d780022d0 0x55d64b0e7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:03.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.884+0000 7f1d97e0ef40 1 -- 172.21.15.159:0/2575437430 >> 172.21.15.159:0/2575437430 conn(0x55d64b069630 msgr2=0x55d64b0f66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:03.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.884+0000 7f1d97e0ef40 1 -- 172.21.15.159:0/2575437430 shutdown_connections 2024-04-04T06:08:03.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.884+0000 7f1d97e0ef40 1 -- 172.21.15.159:0/2575437430 wait complete. 2024-04-04T06:08:03.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.885+0000 7f1d97e0ef40 1 Processor -- start 2024-04-04T06:08:03.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.885+0000 7f1d97e0ef40 1 -- start start 2024-04-04T06:08:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.886+0000 7f1d97e0ef40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64b04d030 0x55d64b0f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.887+0000 7f1d97e0ef40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d64b04df30 0x55d64b0da870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.887+0000 7f1d97e0ef40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d780022d0 0x55d64b0dadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.887+0000 7f1d97e0ef40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d64b0dc3f0 con 0x7f1d780022d0 2024-04-04T06:08:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.887+0000 7f1d97e0ef40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d64b069e90 con 0x55d64b04d030 2024-04-04T06:08:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.887+0000 7f1d97e0ef40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55d64b0e9380 con 0x55d64b04df30 2024-04-04T06:08:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.887+0000 7f1d94e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d780022d0 0x55d64b0dadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.887+0000 7f1d94e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d780022d0 0x55d64b0dadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45436/0 (socket says 172.21.15.159:45436) 2024-04-04T06:08:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.887+0000 7f1d94e00640 1 -- 172.21.15.159:0/3911603612 learned_addr learned my addr 172.21.15.159:0/3911603612 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.887+0000 7f1d93a00640 1 --2- 172.21.15.159:0/3911603612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d64b04df30 0x55d64b0da870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.887+0000 7f1d94400640 1 --2- 172.21.15.159:0/3911603612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64b04d030 0x55d64b0f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.887+0000 7f1d94e00640 1 -- 172.21.15.159:0/3911603612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d64b04df30 msgr2=0x55d64b0da870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.887+0000 7f1d94e00640 1 --2- 172.21.15.159:0/3911603612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d64b04df30 0x55d64b0da870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.887+0000 7f1d94e00640 1 -- 172.21.15.159:0/3911603612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64b04d030 msgr2=0x55d64b0f68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.887+0000 7f1d94e00640 1 --2- 172.21.15.159:0/3911603612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64b04d030 0x55d64b0f68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.887+0000 7f1d94e00640 1 -- 172.21.15.159:0/3911603612 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d64b0f55c0 con 0x7f1d780022d0 2024-04-04T06:08:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.887+0000 7f1d94400640 1 --2- 172.21.15.159:0/3911603612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d64b04d030 0x55d64b0f68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.888+0000 7f1d94e00640 1 --2- 172.21.15.159:0/3911603612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d780022d0 0x55d64b0dadb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c015040 tx=0x7f1d8c0033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.888+0000 7f1d91200640 1 -- 172.21.15.159:0/3911603612 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c00db90 con 0x7f1d780022d0 2024-04-04T06:08:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.888+0000 7f1d91200640 1 -- 172.21.15.159:0/3911603612 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d8c024070 con 0x7f1d780022d0 2024-04-04T06:08:03.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.888+0000 7f1d91200640 1 -- 172.21.15.159:0/3911603612 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c01c3f0 con 0x7f1d780022d0 2024-04-04T06:08:03.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.888+0000 7f1d97e0ef40 1 -- 172.21.15.159:0/3911603612 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d64b01ab90 con 0x7f1d780022d0 2024-04-04T06:08:03.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.888+0000 7f1d97e0ef40 1 -- 172.21.15.159:0/3911603612 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d64b04cd60 con 0x7f1d780022d0 2024-04-04T06:08:03.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.890+0000 7f1d91200640 1 -- 172.21.15.159:0/3911603612 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d8c01c590 con 0x7f1d780022d0 2024-04-04T06:08:03.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.891+0000 7f1d91200640 1 --2- 172.21.15.159:0/3911603612 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1d7406b300 0x7f1d7406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:03.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.891+0000 7f1d91200640 1 -- 172.21.15.159:0/3911603612 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f1d8c08bd80 con 0x7f1d780022d0 2024-04-04T06:08:03.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.891+0000 7f1d97e0ef40 1 --2- 172.21.15.159:0/3911603612 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x55d64b0f6e20 0x55d64b0def00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:03.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.891+0000 7f1d97e0ef40 1 -- 172.21.15.159:0/3911603612 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46) -- 0x55d64b04a230 con 0x55d64b0f6e20 2024-04-04T06:08:03.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.892+0000 7f1d94400640 1 --2- 172.21.15.159:0/3911603612 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1d7406b300 0x7f1d7406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:03.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.892+0000 7f1d93a00640 1 --2- 172.21.15.159:0/3911603612 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x55d64b0f6e20 0x55d64b0def00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:03.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.893+0000 7f1d94400640 1 --2- 172.21.15.159:0/3911603612 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1d7406b300 0x7f1d7406d7c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1d84052000 tx=0x7f1d8404c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:03.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.893+0000 7f1d93a00640 1 --2- 172.21.15.159:0/3911603612 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x55d64b0f6e20 0x55d64b0def00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:03.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.894+0000 7f1d93a00640 1 -- 172.21.15.159:0/3911603612 <== osd.6 v2:172.21.15.178:6826/198949157 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f1d8000e6d0 con 0x55d64b0f6e20 2024-04-04T06:08:03.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.894+0000 7f1d97e0ef40 1 -- 172.21.15.159:0/3911603612 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x55d64b0f6e20 msgr2=0x55d64b0def00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:03.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.894+0000 7f1d97e0ef40 1 --2- 172.21.15.159:0/3911603612 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x55d64b0f6e20 0x55d64b0def00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:03.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.894+0000 7f1d97e0ef40 1 -- 172.21.15.159:0/3911603612 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1d7406b300 msgr2=0x7f1d7406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:03.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.894+0000 7f1d97e0ef40 1 --2- 172.21.15.159:0/3911603612 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1d7406b300 0x7f1d7406d7c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1d84052000 tx=0x7f1d8404c000 comp rx=0 tx=0).stop 2024-04-04T06:08:03.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.894+0000 7f1d97e0ef40 1 -- 172.21.15.159:0/3911603612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d780022d0 msgr2=0x55d64b0dadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:03.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.894+0000 7f1d97e0ef40 1 --2- 172.21.15.159:0/3911603612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d780022d0 0x55d64b0dadb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c015040 tx=0x7f1d8c0033a0 comp rx=0 tx=0).stop 2024-04-04T06:08:03.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.894+0000 7f1d94e00640 1 -- 172.21.15.159:0/3911603612 reap_dead start 2024-04-04T06:08:03.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.894+0000 7f1d97e0ef40 1 -- 172.21.15.159:0/3911603612 shutdown_connections 2024-04-04T06:08:03.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.894+0000 7f1d97e0ef40 1 -- 172.21.15.159:0/3911603612 >> 172.21.15.159:0/3911603612 conn(0x55d64b069630 msgr2=0x55d64b0dfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:03.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.894+0000 7f1d97e0ef40 1 -- 172.21.15.159:0/3911603612 shutdown_connections 2024-04-04T06:08:03.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:03.894+0000 7f1d97e0ef40 1 -- 172.21.15.159:0/3911603612 wait complete. 2024-04-04T06:08:03.893 INFO:teuthology.orchestra.run.smithi159.stdout:Yalphaparent_a 2024-04-04T06:08:03.893 DEBUG:teuthology.orchestra.run.smithi159:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-04T06:08:04.037 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:08:04.037 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 1099511627777, 2024-04-04T06:08:04.037 INFO:teuthology.orchestra.run.smithi159.stdout: "ancestors": [ 2024-04-04T06:08:04.037 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:04.037 INFO:teuthology.orchestra.run.smithi159.stdout: "dirino": 1099511627776, 2024-04-04T06:08:04.038 INFO:teuthology.orchestra.run.smithi159.stdout: "dname": "alpha", 2024-04-04T06:08:04.038 INFO:teuthology.orchestra.run.smithi159.stdout: "version": 2 2024-04-04T06:08:04.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:08:04.038 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:04.038 INFO:teuthology.orchestra.run.smithi159.stdout: "dirino": 1, 2024-04-04T06:08:04.038 INFO:teuthology.orchestra.run.smithi159.stdout: "dname": "parent_a", 2024-04-04T06:08:04.038 INFO:teuthology.orchestra.run.smithi159.stdout: "version": 4 2024-04-04T06:08:04.038 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:04.038 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-04-04T06:08:04.038 INFO:teuthology.orchestra.run.smithi159.stdout: "pool": 7, 2024-04-04T06:08:04.038 INFO:teuthology.orchestra.run.smithi159.stdout: "old_pools": [] 2024-04-04T06:08:04.038 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:08:04.038 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:04.051 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-04-04T06:08:04.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.146+0000 7f3cbcc39f40 1 Processor -- start 2024-04-04T06:08:04.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.146+0000 7f3cbcc39f40 1 -- start start 2024-04-04T06:08:04.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.147+0000 7f3cbcc39f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b487da4f0 0x560b488acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.147+0000 7f3cbcc39f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560b4881f5d0 0x560b48806590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.147+0000 7f3cbcc39f40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b48821a50 0x560b48806ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.147+0000 7f3cbcc39f40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560b4877d2c0 con 0x560b48821a50 2024-04-04T06:08:04.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.147+0000 7f3cbcc39f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560b48781310 con 0x560b487da4f0 2024-04-04T06:08:04.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.147+0000 7f3cbcc39f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x560b48820560 con 0x560b4881f5d0 2024-04-04T06:08:04.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.148+0000 7f3cba200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b48821a50 0x560b48806ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.148+0000 7f3cba200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b48821a50 0x560b48806ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45454/0 (socket says 172.21.15.159:45454) 2024-04-04T06:08:04.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.148+0000 7f3cba200640 1 -- 172.21.15.159:0/3037234559 learned_addr learned my addr 172.21.15.159:0/3037234559 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:04.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.148+0000 7f3cb9800640 1 --2- 172.21.15.159:0/3037234559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b487da4f0 0x560b488acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.148+0000 7f3cb8e00640 1 --2- 172.21.15.159:0/3037234559 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560b4881f5d0 0x560b48806590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.148+0000 7f3cba200640 1 -- 172.21.15.159:0/3037234559 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560b4881f5d0 msgr2=0x560b48806590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.148+0000 7f3cba200640 1 --2- 172.21.15.159:0/3037234559 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560b4881f5d0 0x560b48806590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.148+0000 7f3cba200640 1 -- 172.21.15.159:0/3037234559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b487da4f0 msgr2=0x560b488acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.148+0000 7f3cba200640 1 --2- 172.21.15.159:0/3037234559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b487da4f0 0x560b488acb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.148+0000 7f3cba200640 1 -- 172.21.15.159:0/3037234559 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b487d6600 con 0x560b48821a50 2024-04-04T06:08:04.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.148+0000 7f3cba200640 1 --2- 172.21.15.159:0/3037234559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b48821a50 0x560b48806ad0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4006f80 tx=0x7f3cb4001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10dbedac6a324c4f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:04.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.149+0000 7f3cb3400640 1 -- 172.21.15.159:0/3037234559 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4003650 con 0x560b48821a50 2024-04-04T06:08:04.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.149+0000 7f3cb3400640 1 -- 172.21.15.159:0/3037234559 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb4003830 con 0x560b48821a50 2024-04-04T06:08:04.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.149+0000 7f3cb3400640 1 -- 172.21.15.159:0/3037234559 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4003ca0 con 0x560b48821a50 2024-04-04T06:08:04.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.149+0000 7f3cbcc39f40 1 -- 172.21.15.159:0/3037234559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b48821a50 msgr2=0x560b48806ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.149+0000 7f3cbcc39f40 1 --2- 172.21.15.159:0/3037234559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b48821a50 0x560b48806ad0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4006f80 tx=0x7f3cb4001ac0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.149+0000 7f3cbcc39f40 1 -- 172.21.15.159:0/3037234559 shutdown_connections 2024-04-04T06:08:04.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.149+0000 7f3cbcc39f40 1 --2- 172.21.15.159:0/3037234559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b48821a50 0x560b48806ad0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.149+0000 7f3cbcc39f40 1 --2- 172.21.15.159:0/3037234559 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560b4881f5d0 0x560b48806590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.149+0000 7f3cbcc39f40 1 --2- 172.21.15.159:0/3037234559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b487da4f0 0x560b488acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.149+0000 7f3cbcc39f40 1 -- 172.21.15.159:0/3037234559 >> 172.21.15.159:0/3037234559 conn(0x560b48801dc0 msgr2=0x560b48821340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:04.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.149+0000 7f3cbcc39f40 1 -- 172.21.15.159:0/3037234559 shutdown_connections 2024-04-04T06:08:04.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.149+0000 7f3cbcc39f40 1 -- 172.21.15.159:0/3037234559 wait complete. 2024-04-04T06:08:04.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.150+0000 7f3cbcc39f40 1 Processor -- start 2024-04-04T06:08:04.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.150+0000 7f3cbcc39f40 1 -- start start 2024-04-04T06:08:04.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.151+0000 7f3cbcc39f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b487da4f0 0x560b488aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.151+0000 7f3cbcc39f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560b4881f5d0 0x560b488a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.151+0000 7f3cbcc39f40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b48821a50 0x560b48894840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.151+0000 7f3cbcc39f40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560b487d68a0 con 0x560b48821a50 2024-04-04T06:08:04.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.151+0000 7f3cbcc39f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560b488a44a0 con 0x560b487da4f0 2024-04-04T06:08:04.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.151+0000 7f3cbcc39f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x560b48806300 con 0x560b4881f5d0 2024-04-04T06:08:04.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.151+0000 7f3cba200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b48821a50 0x560b48894840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.151+0000 7f3cba200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b48821a50 0x560b48894840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45462/0 (socket says 172.21.15.159:45462) 2024-04-04T06:08:04.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.151+0000 7f3cba200640 1 -- 172.21.15.159:0/2150603347 learned_addr learned my addr 172.21.15.159:0/2150603347 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:04.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.152+0000 7f3cb9800640 1 --2- 172.21.15.159:0/2150603347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b487da4f0 0x560b488aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.152+0000 7f3cb8e00640 1 --2- 172.21.15.159:0/2150603347 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560b4881f5d0 0x560b488a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.152+0000 7f3cba200640 1 -- 172.21.15.159:0/2150603347 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560b4881f5d0 msgr2=0x560b488a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.152+0000 7f3cba200640 1 --2- 172.21.15.159:0/2150603347 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560b4881f5d0 0x560b488a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.152+0000 7f3cba200640 1 -- 172.21.15.159:0/2150603347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b487da4f0 msgr2=0x560b488aa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.152+0000 7f3cba200640 1 --2- 172.21.15.159:0/2150603347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560b487da4f0 0x560b488aa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.152+0000 7f3cba200640 1 -- 172.21.15.159:0/2150603347 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b48806df0 con 0x560b48821a50 2024-04-04T06:08:04.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.152+0000 7f3cba200640 1 --2- 172.21.15.159:0/2150603347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b48821a50 0x560b48894840 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4001330 tx=0x7f3cb4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:04.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.152+0000 7f3cb2000640 1 -- 172.21.15.159:0/2150603347 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4024070 con 0x560b48821a50 2024-04-04T06:08:04.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.152+0000 7f3cb2000640 1 -- 172.21.15.159:0/2150603347 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb4019040 con 0x560b48821a50 2024-04-04T06:08:04.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.152+0000 7f3cb2000640 1 -- 172.21.15.159:0/2150603347 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4013820 con 0x560b48821a50 2024-04-04T06:08:04.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.152+0000 7f3cbcc39f40 1 -- 172.21.15.159:0/2150603347 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560b48806550 con 0x560b48821a50 2024-04-04T06:08:04.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.153+0000 7f3cbcc39f40 1 -- 172.21.15.159:0/2150603347 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b48806b10 con 0x560b48821a50 2024-04-04T06:08:04.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.154+0000 7f3cb2000640 1 -- 172.21.15.159:0/2150603347 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3cb4010070 con 0x560b48821a50 2024-04-04T06:08:04.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.155+0000 7f3cb2000640 1 --2- 172.21.15.159:0/2150603347 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c9c06b2d0 0x7f3c9c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.155+0000 7f3cb2000640 1 -- 172.21.15.159:0/2150603347 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f3cb408b3d0 con 0x560b48821a50 2024-04-04T06:08:04.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.156+0000 7f3cbcc39f40 1 --2- 172.21.15.159:0/2150603347 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x560b488a1770 0x560b489538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.156+0000 7f3cbcc39f40 1 -- 172.21.15.159:0/2150603347 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46) -- 0x560b48953de0 con 0x560b488a1770 2024-04-04T06:08:04.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.156+0000 7f3cb9800640 1 --2- 172.21.15.159:0/2150603347 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c9c06b2d0 0x7f3c9c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.156+0000 7f3cb8e00640 1 --2- 172.21.15.159:0/2150603347 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x560b488a1770 0x560b489538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.157+0000 7f3cb8e00640 1 --2- 172.21.15.159:0/2150603347 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x560b488a1770 0x560b489538a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:04.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.158+0000 7f3cb9800640 1 --2- 172.21.15.159:0/2150603347 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c9c06b2d0 0x7f3c9c06d790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3cac054000 tx=0x7f3cac04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:04.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.160+0000 7f3cb8e00640 1 -- 172.21.15.159:0/2150603347 <== osd.6 v2:172.21.15.178:6826/198949157 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v46'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3ca800f3a0 con 0x560b488a1770 2024-04-04T06:08:04.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.160+0000 7f3cbcc39f40 1 -- 172.21.15.159:0/2150603347 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x560b488a1770 msgr2=0x560b489538a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.160+0000 7f3cbcc39f40 1 --2- 172.21.15.159:0/2150603347 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x560b488a1770 0x560b489538a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.160+0000 7f3cbcc39f40 1 -- 172.21.15.159:0/2150603347 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c9c06b2d0 msgr2=0x7f3c9c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.160+0000 7f3cbcc39f40 1 --2- 172.21.15.159:0/2150603347 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c9c06b2d0 0x7f3c9c06d790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3cac054000 tx=0x7f3cac04e000 comp rx=0 tx=0).stop 2024-04-04T06:08:04.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.160+0000 7f3cbcc39f40 1 -- 172.21.15.159:0/2150603347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b48821a50 msgr2=0x560b48894840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.160+0000 7f3cbcc39f40 1 --2- 172.21.15.159:0/2150603347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b48821a50 0x560b48894840 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4001330 tx=0x7f3cb4001250 comp rx=0 tx=0).stop 2024-04-04T06:08:04.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.160+0000 7f3cba200640 1 -- 172.21.15.159:0/2150603347 reap_dead start 2024-04-04T06:08:04.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.161+0000 7f3cbcc39f40 1 -- 172.21.15.159:0/2150603347 shutdown_connections 2024-04-04T06:08:04.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.161+0000 7f3cbcc39f40 1 -- 172.21.15.159:0/2150603347 >> 172.21.15.159:0/2150603347 conn(0x560b48801dc0 msgr2=0x560b487d5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:04.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.161+0000 7f3cbcc39f40 1 -- 172.21.15.159:0/2150603347 shutdown_connections 2024-04-04T06:08:04.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.161+0000 7f3cbcc39f40 1 -- 172.21.15.159:0/2150603347 wait complete. 2024-04-04T06:08:04.159 DEBUG:teuthology.orchestra.run.smithi159:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-04T06:08:04.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:08:04.163 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-04-04T06:08:04.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.363+0000 7fe959000640 1 Processor -- start 2024-04-04T06:08:04.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.363+0000 7fe959000640 1 -- start start 2024-04-04T06:08:04.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.364+0000 7fe959000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe95409dcf0 0x7fe9541453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.364+0000 7fe959000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe954071710 0x7fe954071b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.364+0000 7fe959000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe95415cb90 con 0x7fe9540acf10 2024-04-04T06:08:04.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.364+0000 7fe959000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe954145f60 con 0x7fe954071710 2024-04-04T06:08:04.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.364+0000 7fe959000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe95415c950 con 0x7fe95409dcf0 2024-04-04T06:08:04.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.365+0000 7fe952800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe954071710 0x7fe954071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.365+0000 7fe952800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe954071710 0x7fe954071b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45472/0 (socket says 172.21.15.159:45472) 2024-04-04T06:08:04.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.365+0000 7fe952800640 1 -- 172.21.15.159:0/1446243768 learned_addr learned my addr 172.21.15.159:0/1446243768 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:04.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.365+0000 7fe952800640 1 -- 172.21.15.159:0/1446243768 >> v2:172.21.15.178:3300/0 conn(0x7fe95409dcf0 msgr2=0x7fe9541453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.365+0000 7fe952800640 1 --2- 172.21.15.159:0/1446243768 >> v2:172.21.15.178:3300/0 conn(0x7fe95409dcf0 0x7fe9541453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.365+0000 7fe952800640 1 -- 172.21.15.159:0/1446243768 >> v1:172.21.15.178:6789/0 conn(0x7fe9540acf10 legacy=0x7fe9540ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.365+0000 7fe952800640 1 -- 172.21.15.159:0/1446243768 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95415ce70 con 0x7fe954071710 2024-04-04T06:08:04.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.367+0000 7fe952800640 1 --2- 172.21.15.159:0/1446243768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe954071710 0x7fe954071b40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe9540cf8c0 tx=0x7fe94c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ac05bfdeac4a07 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:04.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.367+0000 7fe950a00640 1 -- 172.21.15.159:0/1446243768 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c02c9f0 con 0x7fe954071710 2024-04-04T06:08:04.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.367+0000 7fe950a00640 1 -- 172.21.15.159:0/1446243768 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe94c02cb90 con 0x7fe954071710 2024-04-04T06:08:04.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.367+0000 7fe950a00640 1 -- 172.21.15.159:0/1446243768 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c0343f0 con 0x7fe954071710 2024-04-04T06:08:04.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.368+0000 7fe959000640 1 -- 172.21.15.159:0/1446243768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe954071710 msgr2=0x7fe954071b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.368+0000 7fe959000640 1 --2- 172.21.15.159:0/1446243768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe954071710 0x7fe954071b40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe9540cf8c0 tx=0x7fe94c02be50 comp rx=0 tx=0).stop 2024-04-04T06:08:04.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.368+0000 7fe959000640 1 -- 172.21.15.159:0/1446243768 shutdown_connections 2024-04-04T06:08:04.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.368+0000 7fe959000640 1 --2- 172.21.15.159:0/1446243768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe954071710 0x7fe954071b40 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.368+0000 7fe959000640 1 --2- 172.21.15.159:0/1446243768 >> v2:172.21.15.178:3300/0 conn(0x7fe95409dcf0 0x7fe9541453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.368+0000 7fe959000640 1 -- 172.21.15.159:0/1446243768 >> 172.21.15.159:0/1446243768 conn(0x7fe95406bba0 msgr2=0x7fe9540b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:04.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.368+0000 7fe959000640 1 -- 172.21.15.159:0/1446243768 shutdown_connections 2024-04-04T06:08:04.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.368+0000 7fe959000640 1 -- 172.21.15.159:0/1446243768 wait complete. 2024-04-04T06:08:04.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.370+0000 7fe959000640 1 Processor -- start 2024-04-04T06:08:04.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.370+0000 7fe959000640 1 -- start start 2024-04-04T06:08:04.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.371+0000 7fe959000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe954071710 0x7fe9540ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.371+0000 7fe959000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe95409dcf0 0x7fe954131890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.371+0000 7fe959000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9540acf10 0x7fe954131dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.371+0000 7fe959000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe95415d430 con 0x7fe9540acf10 2024-04-04T06:08:04.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.371+0000 7fe959000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe95415d7a0 con 0x7fe95409dcf0 2024-04-04T06:08:04.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.371+0000 7fe959000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe954145930 con 0x7fe954071710 2024-04-04T06:08:04.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.371+0000 7fe952800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9540acf10 0x7fe954131dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.371+0000 7fe952800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9540acf10 0x7fe954131dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45480/0 (socket says 172.21.15.159:45480) 2024-04-04T06:08:04.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.371+0000 7fe952800640 1 -- 172.21.15.159:0/652028591 learned_addr learned my addr 172.21.15.159:0/652028591 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:04.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.372+0000 7fe951e00640 1 --2- 172.21.15.159:0/652028591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe954071710 0x7fe9540ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.372+0000 7fe951400640 1 --2- 172.21.15.159:0/652028591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe95409dcf0 0x7fe954131890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.372+0000 7fe952800640 1 -- 172.21.15.159:0/652028591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe954071710 msgr2=0x7fe9540ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.372+0000 7fe952800640 1 --2- 172.21.15.159:0/652028591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe954071710 0x7fe9540ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.372+0000 7fe952800640 1 -- 172.21.15.159:0/652028591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe95409dcf0 msgr2=0x7fe954131890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.372+0000 7fe952800640 1 --2- 172.21.15.159:0/652028591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe95409dcf0 0x7fe954131890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.372+0000 7fe952800640 1 -- 172.21.15.159:0/652028591 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe954132450 con 0x7fe9540acf10 2024-04-04T06:08:04.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.372+0000 7fe951e00640 1 --2- 172.21.15.159:0/652028591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe954071710 0x7fe9540ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:04.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.372+0000 7fe951400640 1 --2- 172.21.15.159:0/652028591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe95409dcf0 0x7fe954131890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:04.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.372+0000 7fe952800640 1 --2- 172.21.15.159:0/652028591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9540acf10 0x7fe954131dd0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009f40 tx=0x7fe94c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:04.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.372+0000 7fe94a000640 1 -- 172.21.15.159:0/652028591 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c043020 con 0x7fe9540acf10 2024-04-04T06:08:04.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.372+0000 7fe94a000640 1 -- 172.21.15.159:0/652028591 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe94c03e450 con 0x7fe9540acf10 2024-04-04T06:08:04.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.372+0000 7fe94a000640 1 -- 172.21.15.159:0/652028591 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c009510 con 0x7fe9540acf10 2024-04-04T06:08:04.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.372+0000 7fe959000640 1 -- 172.21.15.159:0/652028591 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe954132730 con 0x7fe9540acf10 2024-04-04T06:08:04.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.373+0000 7fe959000640 1 -- 172.21.15.159:0/652028591 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe95412fb30 con 0x7fe9540acf10 2024-04-04T06:08:04.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.375+0000 7fe94a000640 1 -- 172.21.15.159:0/652028591 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe94c046020 con 0x7fe9540acf10 2024-04-04T06:08:04.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.376+0000 7fe95bdea740 1 Processor -- start 2024-04-04T06:08:04.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.376+0000 7fe95bdea740 1 -- start start 2024-04-04T06:08:04.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.376+0000 7fe94a000640 1 --2- 172.21.15.159:0/652028591 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe93006c8f0 0x7fe93006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.376+0000 7fe94a000640 1 -- 172.21.15.159:0/652028591 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fe94c0ae4e0 con 0x7fe9540acf10 2024-04-04T06:08:04.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.377+0000 7fe951400640 1 --2- 172.21.15.159:0/652028591 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe93006c8f0 0x7fe93006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.377+0000 7fe95bdea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502e7c10 0x55b7502b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.377+0000 7fe95bdea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7502e7ff0 0x55b7502e4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.377+0000 7fe95bdea740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b7502e5120 0x55b7502b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.377+0000 7fe95bdea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b75021e280 con 0x55b7502e7c10 2024-04-04T06:08:04.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.377+0000 7fe95bdea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b7501c66f0 con 0x55b7502e7ff0 2024-04-04T06:08:04.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.377+0000 7fe95bdea740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55b75021a230 con 0x55b7502e5120 2024-04-04T06:08:04.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.378+0000 7fe951e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502e7c10 0x55b7502b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.378+0000 7fe951e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502e7c10 0x55b7502b83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45482/0 (socket says 172.21.15.159:45482) 2024-04-04T06:08:04.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.378+0000 7fe951e00640 1 -- 172.21.15.159:0/2138108410 learned_addr learned my addr 172.21.15.159:0/2138108410 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:04.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.378+0000 7fe951400640 1 --2- 172.21.15.159:0/2138108410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7502e7ff0 0x55b7502e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.378+0000 7fe952800640 1 --2- 172.21.15.159:0/2138108410 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b7502e5120 0x55b7502b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.378+0000 7fe951e00640 1 -- 172.21.15.159:0/2138108410 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b7502e5120 msgr2=0x55b7502b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.378+0000 7fe951e00640 1 --2- 172.21.15.159:0/2138108410 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b7502e5120 0x55b7502b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.378+0000 7fe951e00640 1 -- 172.21.15.159:0/2138108410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7502e7ff0 msgr2=0x55b7502e4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.378+0000 7fe951e00640 1 --2- 172.21.15.159:0/2138108410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7502e7ff0 0x55b7502e4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.378+0000 7fe951e00640 1 -- 172.21.15.159:0/2138108410 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b74fffa7e0 con 0x55b7502e7c10 2024-04-04T06:08:04.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.378+0000 7fe951400640 1 --2- 172.21.15.159:0/652028591 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe93006c8f0 0x7fe93006edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe93c006fd0 tx=0x7fe93c00acf0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:04.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.379+0000 7fe951e00640 1 --2- 172.21.15.159:0/2138108410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502e7c10 0x55b7502b83d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe944003f50 tx=0x7fe944004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3364a43f5b24b0df server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:04.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.379+0000 7fe942a00640 1 -- 172.21.15.159:0/2138108410 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944005270 con 0x55b7502e7c10 2024-04-04T06:08:04.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.379+0000 7fe942a00640 1 -- 172.21.15.159:0/2138108410 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe944005410 con 0x55b7502e7c10 2024-04-04T06:08:04.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.379+0000 7fe942a00640 1 -- 172.21.15.159:0/2138108410 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94400dbe0 con 0x55b7502e7c10 2024-04-04T06:08:04.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.380+0000 7fe95bdea740 1 -- 172.21.15.159:0/2138108410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502e7c10 msgr2=0x55b7502b83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.380+0000 7fe95bdea740 1 --2- 172.21.15.159:0/2138108410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502e7c10 0x55b7502b83d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe944003f50 tx=0x7fe944004410 comp rx=0 tx=0).stop 2024-04-04T06:08:04.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.380+0000 7fe95bdea740 1 -- 172.21.15.159:0/2138108410 shutdown_connections 2024-04-04T06:08:04.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.380+0000 7fe95bdea740 1 --2- 172.21.15.159:0/2138108410 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b7502e5120 0x55b7502b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.380+0000 7fe95bdea740 1 --2- 172.21.15.159:0/2138108410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7502e7ff0 0x55b7502e4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.380+0000 7fe95bdea740 1 --2- 172.21.15.159:0/2138108410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502e7c10 0x55b7502b83d0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.380+0000 7fe95bdea740 1 -- 172.21.15.159:0/2138108410 >> 172.21.15.159:0/2138108410 conn(0x55b7502b6040 msgr2=0x55b7502b7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:04.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.380+0000 7fe95bdea740 1 -- 172.21.15.159:0/2138108410 shutdown_connections 2024-04-04T06:08:04.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.380+0000 7fe95bdea740 1 -- 172.21.15.159:0/2138108410 wait complete. 2024-04-04T06:08:04.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.381+0000 7fe95bdea740 1 -- start start 2024-04-04T06:08:04.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.381+0000 7fe95bdea740 1 Processor -- start 2024-04-04T06:08:04.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.382+0000 7f92ee600640 1 Processor -- start 2024-04-04T06:08:04.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.382+0000 7f92ee600640 1 -- start start 2024-04-04T06:08:04.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.383+0000 7fe95bdea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7502e7c10 0x55b7502c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.383+0000 7fe95bdea740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b7502e7ff0 0x55b750313200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.383+0000 7fe95bdea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502cac50 0x55b7502e6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.383+0000 7fe95bdea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b750301030 con 0x55b7502cac50 2024-04-04T06:08:04.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.383+0000 7fe95bdea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b7501d9aa0 con 0x55b7502e7c10 2024-04-04T06:08:04.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.383+0000 7fe95bdea740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55b750300180 con 0x55b7502e7ff0 2024-04-04T06:08:04.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.383+0000 7fe95bdea740 10 client.0 _ll_register_callbacks cb 0x55b7501d9490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:08:04.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.383+0000 7fe952800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502cac50 0x55b7502e6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.383+0000 7fe952800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502cac50 0x55b7502e6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45490/0 (socket says 172.21.15.159:45490) 2024-04-04T06:08:04.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.383+0000 7fe952800640 1 -- 172.21.15.159:0/1630249837 learned_addr learned my addr 172.21.15.159:0/1630249837 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:04.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.384+0000 7fe951e00640 1 --2- 172.21.15.159:0/1630249837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7502e7c10 0x55b7502c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.384+0000 7fe952800640 1 -- 172.21.15.159:0/1630249837 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b7502e7ff0 msgr2=0x55b750313200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.384+0000 7fe952800640 1 --2- 172.21.15.159:0/1630249837 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b7502e7ff0 0x55b750313200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.384+0000 7f92ee600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f92e80aebf0 0x7f92e80af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.384+0000 7f92ee600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e8071850 0x7f92e8071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.384+0000 7fe952800640 1 -- 172.21.15.159:0/1630249837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7502e7c10 msgr2=0x55b7502c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.384+0000 7fe952800640 1 --2- 172.21.15.159:0/1630249837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7502e7c10 0x55b7502c7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.384+0000 7fe952800640 1 -- 172.21.15.159:0/1630249837 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b75005af70 con 0x55b7502cac50 2024-04-04T06:08:04.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.384+0000 7f92ee600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92e806e4e0 con 0x7f92e8145ce0 2024-04-04T06:08:04.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.384+0000 7f92ee600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f92e806e7c0 con 0x7f92e80aebf0 2024-04-04T06:08:04.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.384+0000 7f92ee600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f92e806eaa0 con 0x7f92e8071850 2024-04-04T06:08:04.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.384+0000 7f92e7400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f92e8145ce0 0x7f92e80acff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:55688/0 (socket says 172.21.15.159:55688) 2024-04-04T06:08:04.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.384+0000 7f92e7400640 1 -- 172.21.15.159:0/3242152551 learned_addr learned my addr 172.21.15.159:0/3242152551 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:04.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.384+0000 7fe952800640 1 --2- 172.21.15.159:0/1630249837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502cac50 0x55b7502e6e90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0848c0 tx=0x7fe94c07c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecefe859417fc3e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:04.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.384+0000 7fe93b400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:08:04.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.384+0000 7f92e7e00640 1 --2- 172.21.15.159:0/3242152551 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e8071850 0x7f92e8071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.384+0000 7f92e6a00640 1 --2- 172.21.15.159:0/3242152551 >> v2:172.21.15.178:3300/0 conn(0x7f92e80aebf0 0x7f92e80af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7fe93b400640 1 -- 172.21.15.159:0/1630249837 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c084df0 con 0x55b7502cac50 2024-04-04T06:08:04.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7fe93b400640 1 -- 172.21.15.159:0/1630249837 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fe94c03c350 con 0x55b7502cac50 2024-04-04T06:08:04.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7fe95bdea740 1 -- 172.21.15.159:0/1630249837 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe944005410 con 0x55b7502cac50 2024-04-04T06:08:04.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7f92e6000640 1 -- 172.21.15.159:0/3242152551 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3726904376 0 0) 0x7f92e806e4e0 con 0x7f92e8145ce0 2024-04-04T06:08:04.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7f92e6000640 1 -- 172.21.15.159:0/3242152551 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92cc003520 con 0x7f92e8145ce0 2024-04-04T06:08:04.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7fe93b400640 1 -- 172.21.15.159:0/1630249837 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe94c03c550 con 0x55b7502cac50 2024-04-04T06:08:04.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7f92e6000640 1 -- 172.21.15.159:0/3242152551 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4055926320 0 0) 0x7f92cc003520 con 0x7f92e8145ce0 2024-04-04T06:08:04.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7f92e6000640 1 -- 172.21.15.159:0/3242152551 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e8071850 msgr2=0x7f92e8071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7f92e6000640 1 --2- 172.21.15.159:0/3242152551 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e8071850 0x7f92e8071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7fe95bdea740 10 client.5790 fetch_fsmap learned FSMap version 25 2024-04-04T06:08:04.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7fe95bdea740 1 -- 172.21.15.159:0/1630249837 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x55b74fef2170 con 0x55b7502cac50 2024-04-04T06:08:04.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7f92e6000640 1 -- 172.21.15.159:0/3242152551 >> v2:172.21.15.178:3300/0 conn(0x7f92e80aebf0 msgr2=0x7f92e80af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7f92e6000640 1 --2- 172.21.15.159:0/3242152551 >> v2:172.21.15.178:3300/0 conn(0x7f92e80aebf0 0x7f92e80af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7f92e6000640 1 -- 172.21.15.159:0/3242152551 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e806ed80 con 0x7f92e8145ce0 2024-04-04T06:08:04.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7f92e6000640 1 -- 172.21.15.159:0/3242152551 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f92d8001690 con 0x7f92e8145ce0 2024-04-04T06:08:04.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7f92e6000640 1 -- 172.21.15.159:0/3242152551 >> v1:172.21.15.159:6789/0 conn(0x7f92e8145ce0 legacy=0x7f92e80acff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7f92e6000640 1 --2- 172.21.15.159:0/3242152551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92cc0039b0 0x7f92cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7f92e6000640 1 --2- 172.21.15.159:0/3242152551 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e8071850 0x7f92cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7f92e6000640 1 --2- 172.21.15.159:0/3242152551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92cc004d40 0x7f92cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7f92e6000640 1 -- 172.21.15.159:0/3242152551 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f92e806e7c0 con 0x7f92cc0039b0 2024-04-04T06:08:04.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7f92e6000640 1 -- 172.21.15.159:0/3242152551 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f92e806eaa0 con 0x7f92cc004d40 2024-04-04T06:08:04.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7f92e6000640 1 -- 172.21.15.159:0/3242152551 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f92cc00b9c0 con 0x7f92e8071850 2024-04-04T06:08:04.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7fe93b400640 1 -- 172.21.15.159:0/1630249837 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe944005410 con 0x55b7502cac50 2024-04-04T06:08:04.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.385+0000 7fe93b400640 1 -- 172.21.15.159:0/1630249837 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7fe94c080170 con 0x55b7502cac50 2024-04-04T06:08:04.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.386+0000 7fe95bdea740 10 client.5790 fetch_fsmap finished waiting for FSMap version 25 2024-04-04T06:08:04.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.386+0000 7fe95bdea740 10 client.5790 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-04T06:08:04.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.386+0000 7f92e7400640 1 --2- 172.21.15.159:0/3242152551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92cc0039b0 0x7f92cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.386+0000 7fe95bdea740 20 client.5790 populate_metadata read hostname 'smithi159' 2024-04-04T06:08:04.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.386+0000 7fe95bdea740 1 --2- 172.21.15.159:0/1630249837 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55b7502b7dd0 0x55b75030a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.386+0000 7fe95bdea740 4 client.5790 mds_command: new command op to 5589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:08:04.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.386+0000 7fe95bdea740 1 -- 172.21.15.159:0/1630249837 --> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b75022b7c0 con 0x55b7502b7dd0 2024-04-04T06:08:04.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.386+0000 7fe951e00640 1 --2- 172.21.15.159:0/1630249837 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55b7502b7dd0 0x55b75030a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.386+0000 7f92e6a00640 1 --2- 172.21.15.159:0/3242152551 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e8071850 0x7f92cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.386+0000 7f92e7400640 1 -- 172.21.15.159:0/3242152551 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e8071850 msgr2=0x7f92cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.386+0000 7f92e7400640 1 --2- 172.21.15.159:0/3242152551 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e8071850 0x7f92cc004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.386+0000 7f92e7400640 1 -- 172.21.15.159:0/3242152551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92cc004d40 msgr2=0x7f92cc0051c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-04T06:08:04.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.386+0000 7f92e7400640 1 --2- 172.21.15.159:0/3242152551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92cc004d40 0x7f92cc0051c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.386+0000 7f92e7400640 1 -- 172.21.15.159:0/3242152551 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92cc003520 con 0x7f92cc0039b0 2024-04-04T06:08:04.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.387+0000 7fe951e00640 1 --2- 172.21.15.159:0/1630249837 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55b7502b7dd0 0x55b75030a720 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f9b4b253a585bd3 server_cookie=84833875f3d50794 in_seq=0 out_seq=0 2024-04-04T06:08:04.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.387+0000 7fe93b400640 10 client.5790 ms_handle_connect on v2:172.21.15.159:6834/1867245367 2024-04-04T06:08:04.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.387+0000 7fe93b400640 1 -- 172.21.15.159:0/1630249837 <== mds.0 v2:172.21.15.159:6834/1867245367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b75022b7c0 con 0x55b7502b7dd0 2024-04-04T06:08:04.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.387+0000 7fe93b400640 10 client.5790 handle_command_reply: tid=0 2024-04-04T06:08:04.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.387+0000 7f92e7400640 1 --2- 172.21.15.159:0/3242152551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92cc0039b0 0x7f92cc003dd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f92d8007fd0 tx=0x7f92cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=defefa2cefbfe11 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:04.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.388+0000 7fe95bdea740 1 client.5790 shutdown 2024-04-04T06:08:04.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.388+0000 7f92e6000640 1 -- 172.21.15.159:0/3242152551 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8007ca0 con 0x7f92cc0039b0 2024-04-04T06:08:04.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.388+0000 7fe95bdea740 1 -- 172.21.15.159:0/1630249837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502cac50 msgr2=0x55b7502e6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.388+0000 7fe95bdea740 1 --2- 172.21.15.159:0/1630249837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502cac50 0x55b7502e6e90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0848c0 tx=0x7fe94c07c9d0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.388+0000 7fe95bdea740 1 -- 172.21.15.159:0/1630249837 shutdown_connections 2024-04-04T06:08:04.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.388+0000 7fe95bdea740 1 --2- 172.21.15.159:0/1630249837 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55b7502b7dd0 0x55b75030a720 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.388+0000 7fe95bdea740 1 --2- 172.21.15.159:0/1630249837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502cac50 0x55b7502e6e90 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.388+0000 7fe95bdea740 1 --2- 172.21.15.159:0/1630249837 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b7502e7ff0 0x55b750313200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.388+0000 7fe93b400640 0 client.5790 ms_handle_reset on v2:172.21.15.159:6834/1867245367 2024-04-04T06:08:04.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.388+0000 7fe95bdea740 1 --2- 172.21.15.159:0/1630249837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7502e7c10 0x55b7502c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.388+0000 7fe95bdea740 1 -- 172.21.15.159:0/1630249837 >> 172.21.15.159:0/1630249837 conn(0x55b7502e5120 msgr2=0x55b7502b3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:04.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.388+0000 7f92e6000640 1 -- 172.21.15.159:0/3242152551 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d8030070 con 0x7f92cc0039b0 2024-04-04T06:08:04.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.388+0000 7f92e6000640 1 -- 172.21.15.159:0/3242152551 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8005d30 con 0x7f92cc0039b0 2024-04-04T06:08:04.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.388+0000 7fe95bdea740 1 -- 172.21.15.159:0/1630249837 shutdown_connections 2024-04-04T06:08:04.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.388+0000 7f92ee600640 1 -- 172.21.15.159:0/3242152551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92cc0039b0 msgr2=0x7f92cc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.388+0000 7f92ee600640 1 --2- 172.21.15.159:0/3242152551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92cc0039b0 0x7f92cc003dd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f92d8007fd0 tx=0x7f92cc005830 comp rx=0 tx=0).stop 2024-04-04T06:08:04.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.389+0000 7f92e7e00640 1 -- 172.21.15.159:0/3242152551 reap_dead start 2024-04-04T06:08:04.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.388+0000 7fe95bdea740 1 -- 172.21.15.159:0/1630249837 wait complete. 2024-04-04T06:08:04.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.389+0000 7f92ee600640 1 -- 172.21.15.159:0/3242152551 shutdown_connections 2024-04-04T06:08:04.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.389+0000 7f92ee600640 1 -- 172.21.15.159:0/3242152551 >> 172.21.15.159:0/3242152551 conn(0x7f92e806b570 msgr2=0x7f92e806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:04.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.389+0000 7fe95bdea740 20 client.5790 trim_cache size 0 max 16384 2024-04-04T06:08:04.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.389+0000 7f92ee600640 1 -- 172.21.15.159:0/3242152551 shutdown_connections 2024-04-04T06:08:04.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.389+0000 7f92ee600640 1 -- 172.21.15.159:0/3242152551 wait complete. 2024-04-04T06:08:04.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.391+0000 7f92ee600640 1 Processor -- start 2024-04-04T06:08:04.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.391+0000 7f92ee600640 1 -- start start 2024-04-04T06:08:04.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.392+0000 7f92ee600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92cc0039b0 0x7f92e809ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.392+0000 7f92ee600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e813ba40 0x7f92e809cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.392+0000 7f92ee600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92e809d000 0x7f92e81302c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.392+0000 7f92ee600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f92e80ad700 con 0x7f92cc0039b0 2024-04-04T06:08:04.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.392+0000 7f92ee600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f92e806f060 con 0x7f92e809d000 2024-04-04T06:08:04.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.392+0000 7f92ee600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f92e80af570 con 0x7f92e813ba40 2024-04-04T06:08:04.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.392+0000 7f92e7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92cc0039b0 0x7f92e809ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.392+0000 7f92e7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92cc0039b0 0x7f92e809ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45508/0 (socket says 172.21.15.159:45508) 2024-04-04T06:08:04.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.392+0000 7f92e7400640 1 -- 172.21.15.159:0/3370956774 learned_addr learned my addr 172.21.15.159:0/3370956774 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:04.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.392+0000 7f92e6a00640 1 --2- 172.21.15.159:0/3370956774 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e813ba40 0x7f92e809cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.392+0000 7f92e7e00640 1 --2- 172.21.15.159:0/3370956774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92e809d000 0x7f92e81302c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.392+0000 7f92e7400640 1 -- 172.21.15.159:0/3370956774 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e813ba40 msgr2=0x7f92e809cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.392+0000 7f92e7400640 1 --2- 172.21.15.159:0/3370956774 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92e813ba40 0x7f92e809cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.392+0000 7f92e7400640 1 -- 172.21.15.159:0/3370956774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92e809d000 msgr2=0x7f92e81302c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.392+0000 7f92e7400640 1 --2- 172.21.15.159:0/3370956774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92e809d000 0x7f92e81302c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.392+0000 7f92e7400640 1 -- 172.21.15.159:0/3370956774 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e809d4c0 con 0x7f92cc0039b0 2024-04-04T06:08:04.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.393+0000 7f92e7e00640 1 --2- 172.21.15.159:0/3370956774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92e809d000 0x7f92e81302c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:04.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.393+0000 7f92e7400640 1 --2- 172.21.15.159:0/3370956774 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92cc0039b0 0x7f92e809ac90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f92d802c480 tx=0x7f92d8001990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:04.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.393+0000 7f92df400640 1 -- 172.21.15.159:0/3370956774 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8004bf0 con 0x7f92cc0039b0 2024-04-04T06:08:04.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.393+0000 7f92df400640 1 -- 172.21.15.159:0/3370956774 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d8004d90 con 0x7f92cc0039b0 2024-04-04T06:08:04.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.393+0000 7f92df400640 1 -- 172.21.15.159:0/3370956774 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d80072b0 con 0x7f92cc0039b0 2024-04-04T06:08:04.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.393+0000 7f92ee600640 1 -- 172.21.15.159:0/3370956774 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e809d7a0 con 0x7f92cc0039b0 2024-04-04T06:08:04.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.393+0000 7f92ee600640 1 -- 172.21.15.159:0/3370956774 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e815cdd0 con 0x7f92cc0039b0 2024-04-04T06:08:04.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.394+0000 7f92ee600640 1 -- 172.21.15.159:0/3370956774 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f92e80685c0 con 0x7f92cc0039b0 2024-04-04T06:08:04.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.396+0000 7f92df400640 1 -- 172.21.15.159:0/3370956774 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f92d8007450 con 0x7f92cc0039b0 2024-04-04T06:08:04.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.398+0000 7f92df400640 1 --2- 172.21.15.159:0/3370956774 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f92c806c8c0 0x7f92c806ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.398+0000 7f92df400640 1 -- 172.21.15.159:0/3370956774 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f92d80adb60 con 0x7f92cc0039b0 2024-04-04T06:08:04.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.398+0000 7f92df400640 1 -- 172.21.15.159:0/3370956774 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+501 (secure 0 0 0) 0x7f92d804f030 con 0x7f92cc0039b0 2024-04-04T06:08:04.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.399+0000 7f92e6a00640 1 --2- 172.21.15.159:0/3370956774 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f92c806c8c0 0x7f92c806ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.399+0000 7f92ee600640 1 -- 172.21.15.159:0/3370956774 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f92e806f700 con 0x7f92cc0039b0 2024-04-04T06:08:04.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.400+0000 7f92e6a00640 1 --2- 172.21.15.159:0/3370956774 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f92c806c8c0 0x7f92c806ed80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f92e8070110 tx=0x7f92d4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:04.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.410+0000 7fe95bdea740 1 Processor -- start 2024-04-04T06:08:04.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.410+0000 7fe95bdea740 1 -- start start 2024-04-04T06:08:04.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.410+0000 7fe95bdea740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b7502cac50 0x55b750309c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.410+0000 7fe95bdea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7502e7c10 0x55b7502c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.411+0000 7fe95bdea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502e7ff0 0x55b7502b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.411+0000 7fe95bdea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe95406f290 con 0x55b7502e7ff0 2024-04-04T06:08:04.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.411+0000 7fe95bdea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe95406e290 con 0x55b7502e7c10 2024-04-04T06:08:04.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.411+0000 7fe95bdea740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe95406d620 con 0x55b7502cac50 2024-04-04T06:08:04.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.411+0000 7fe952800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502e7ff0 0x55b7502b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.411+0000 7fe952800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502e7ff0 0x55b7502b2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45512/0 (socket says 172.21.15.159:45512) 2024-04-04T06:08:04.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.411+0000 7fe952800640 1 -- 172.21.15.159:0/2314921739 learned_addr learned my addr 172.21.15.159:0/2314921739 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:04.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.411+0000 7fe951400640 1 --2- 172.21.15.159:0/2314921739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7502e7c10 0x55b7502c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.411+0000 7fe951e00640 1 --2- 172.21.15.159:0/2314921739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b7502cac50 0x55b750309c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.411+0000 7fe952800640 1 -- 172.21.15.159:0/2314921739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b7502cac50 msgr2=0x55b750309c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.411+0000 7fe952800640 1 --2- 172.21.15.159:0/2314921739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b7502cac50 0x55b750309c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.411+0000 7fe952800640 1 -- 172.21.15.159:0/2314921739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7502e7c10 msgr2=0x55b7502c7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.411+0000 7fe952800640 1 --2- 172.21.15.159:0/2314921739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7502e7c10 0x55b7502c7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.411+0000 7fe952800640 1 -- 172.21.15.159:0/2314921739 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b750327010 con 0x55b7502e7ff0 2024-04-04T06:08:04.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.412+0000 7fe952800640 1 --2- 172.21.15.159:0/2314921739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502e7ff0 0x55b7502b2cc0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009f10 tx=0x7fe94c04a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78994df14f682a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:04.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.412+0000 7fe942000640 1 -- 172.21.15.159:0/2314921739 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c080170 con 0x55b7502e7ff0 2024-04-04T06:08:04.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.412+0000 7fe942000640 1 -- 172.21.15.159:0/2314921739 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe94c02ca00 con 0x55b7502e7ff0 2024-04-04T06:08:04.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.412+0000 7fe942000640 1 -- 172.21.15.159:0/2314921739 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c03c550 con 0x55b7502e7ff0 2024-04-04T06:08:04.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.413+0000 7fe95bdea740 1 -- 172.21.15.159:0/2314921739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502e7ff0 msgr2=0x55b7502b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.413+0000 7fe95bdea740 1 --2- 172.21.15.159:0/2314921739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502e7ff0 0x55b7502b2cc0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009f10 tx=0x7fe94c04a000 comp rx=0 tx=0).stop 2024-04-04T06:08:04.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.413+0000 7fe95bdea740 1 -- 172.21.15.159:0/2314921739 shutdown_connections 2024-04-04T06:08:04.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.413+0000 7fe95bdea740 1 --2- 172.21.15.159:0/2314921739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502e7ff0 0x55b7502b2cc0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.413+0000 7fe95bdea740 1 --2- 172.21.15.159:0/2314921739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7502e7c10 0x55b7502c7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.413+0000 7fe95bdea740 1 --2- 172.21.15.159:0/2314921739 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b7502cac50 0x55b750309c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.413+0000 7fe95bdea740 1 -- 172.21.15.159:0/2314921739 >> 172.21.15.159:0/2314921739 conn(0x55b7502e5120 msgr2=0x55b750308c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:04.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.413+0000 7fe95bdea740 1 -- 172.21.15.159:0/2314921739 shutdown_connections 2024-04-04T06:08:04.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.413+0000 7fe95bdea740 1 -- 172.21.15.159:0/2314921739 wait complete. 2024-04-04T06:08:04.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.414+0000 7fe95bdea740 1 -- start start 2024-04-04T06:08:04.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.414+0000 7fe95bdea740 1 Processor -- start 2024-04-04T06:08:04.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.415+0000 7fe95bdea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502cac50 0x55b75030ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.415+0000 7fe95bdea740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b7502e7c10 0x55b7502b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.415+0000 7fe95bdea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7502e7ff0 0x55b75031c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.415+0000 7fe95bdea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b7502dc5a0 con 0x55b7502cac50 2024-04-04T06:08:04.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.415+0000 7fe95bdea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe95406dc40 con 0x55b7502e7ff0 2024-04-04T06:08:04.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.415+0000 7fe95bdea740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55b7502e6f10 con 0x55b7502e7c10 2024-04-04T06:08:04.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.415+0000 7fe95bdea740 10 client.0 _ll_register_callbacks cb 0x7fe94c07be00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:08:04.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.415+0000 7fe951e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502cac50 0x55b75030ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.415+0000 7fe951e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502cac50 0x55b75030ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45516/0 (socket says 172.21.15.159:45516) 2024-04-04T06:08:04.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.415+0000 7fe951e00640 1 -- 172.21.15.159:0/352497338 learned_addr learned my addr 172.21.15.159:0/352497338 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:04.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.415+0000 7fe951e00640 1 -- 172.21.15.159:0/352497338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b7502e7c10 msgr2=0x55b7502b8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.415+0000 7fe951e00640 1 --2- 172.21.15.159:0/352497338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b7502e7c10 0x55b7502b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.415+0000 7fe951e00640 1 -- 172.21.15.159:0/352497338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7502e7ff0 msgr2=0x55b75031c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.415+0000 7fe951e00640 1 --2- 172.21.15.159:0/352497338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7502e7ff0 0x55b75031c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.415+0000 7fe951e00640 1 -- 172.21.15.159:0/352497338 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7502b2f10 con 0x55b7502cac50 2024-04-04T06:08:04.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.416+0000 7fe951e00640 1 --2- 172.21.15.159:0/352497338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502cac50 0x55b75030ade0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe944009010 tx=0x7fe9440036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf086bb727912cc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:04.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.416+0000 7fe93b400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:08:04.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.416+0000 7fe93b400640 1 -- 172.21.15.159:0/352497338 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944007cc0 con 0x55b7502cac50 2024-04-04T06:08:04.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.416+0000 7fe93b400640 1 -- 172.21.15.159:0/352497338 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fe94400daa0 con 0x55b7502cac50 2024-04-04T06:08:04.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.416+0000 7fe95bdea740 1 -- 172.21.15.159:0/352497338 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe94c02ca00 con 0x55b7502cac50 2024-04-04T06:08:04.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.416+0000 7fe93b400640 1 -- 172.21.15.159:0/352497338 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe94400dce0 con 0x55b7502cac50 2024-04-04T06:08:04.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.416+0000 7fe95bdea740 10 client.5802 fetch_fsmap learned FSMap version 25 2024-04-04T06:08:04.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.416+0000 7fe95bdea740 1 -- 172.21.15.159:0/352497338 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x55b7502b34d0 con 0x55b7502cac50 2024-04-04T06:08:04.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.416+0000 7fe93b400640 1 -- 172.21.15.159:0/352497338 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe94c02ca00 con 0x55b7502cac50 2024-04-04T06:08:04.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.417+0000 7fe93b400640 1 -- 172.21.15.159:0/352497338 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7fe944026b50 con 0x55b7502cac50 2024-04-04T06:08:04.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.417+0000 7fe95bdea740 10 client.5802 fetch_fsmap finished waiting for FSMap version 25 2024-04-04T06:08:04.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.417+0000 7fe95bdea740 10 client.5802 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-04T06:08:04.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.417+0000 7fe95bdea740 20 client.5802 populate_metadata read hostname 'smithi159' 2024-04-04T06:08:04.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.417+0000 7fe95bdea740 1 --2- 172.21.15.159:0/352497338 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55b7502b9390 0x55b75031c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.417+0000 7fe95bdea740 4 client.5802 mds_command: new command op to 5589 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-04T06:08:04.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.417+0000 7fe95bdea740 1 -- 172.21.15.159:0/352497338 --> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55b750225b10 con 0x55b7502b9390 2024-04-04T06:08:04.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.417+0000 7fe951400640 1 --2- 172.21.15.159:0/352497338 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55b7502b9390 0x55b75031c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.418+0000 7fe93b400640 10 client.5802 ms_handle_connect on v2:172.21.15.159:6834/1867245367 2024-04-04T06:08:04.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.418+0000 7fe951400640 1 --2- 172.21.15.159:0/352497338 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55b7502b9390 0x55b75031c900 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=587a17b00d6b4a3a server_cookie=502791e26b186bca in_seq=0 out_seq=0 2024-04-04T06:08:04.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.419+0000 7fe93b400640 1 -- 172.21.15.159:0/352497338 <== mds.0 v2:172.21.15.159:6834/1867245367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b750225b10 con 0x55b7502b9390 2024-04-04T06:08:04.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.419+0000 7fe93b400640 10 client.5802 handle_command_reply: tid=0 2024-04-04T06:08:04.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.419+0000 7fe95bdea740 1 client.5802 shutdown 2024-04-04T06:08:04.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.419+0000 7fe95bdea740 1 -- 172.21.15.159:0/352497338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502cac50 msgr2=0x55b75030ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.419+0000 7fe95bdea740 1 --2- 172.21.15.159:0/352497338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502cac50 0x55b75030ade0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe944009010 tx=0x7fe9440036f0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.419+0000 7fe95bdea740 1 -- 172.21.15.159:0/352497338 shutdown_connections 2024-04-04T06:08:04.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.419+0000 7fe95bdea740 1 --2- 172.21.15.159:0/352497338 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55b7502b9390 0x55b75031c900 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.419+0000 7fe95bdea740 1 --2- 172.21.15.159:0/352497338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b7502e7ff0 0x55b75031c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.419+0000 7fe95bdea740 1 --2- 172.21.15.159:0/352497338 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b7502e7c10 0x55b7502b8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.419+0000 7fe95bdea740 1 --2- 172.21.15.159:0/352497338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7502cac50 0x55b75030ade0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.419+0000 7fe93b400640 0 client.5802 ms_handle_reset on v2:172.21.15.159:6834/1867245367 2024-04-04T06:08:04.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.419+0000 7fe95bdea740 1 -- 172.21.15.159:0/352497338 >> 172.21.15.159:0/352497338 conn(0x55b7502e5120 msgr2=0x55b7502c7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:04.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.419+0000 7fe95bdea740 1 -- 172.21.15.159:0/352497338 shutdown_connections 2024-04-04T06:08:04.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.419+0000 7fe95bdea740 1 -- 172.21.15.159:0/352497338 wait complete. 2024-04-04T06:08:04.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.420+0000 7fe95bdea740 20 client.5802 trim_cache size 0 max 16384 2024-04-04T06:08:04.417 INFO:tasks.ceph.mds.a.smithi159.stderr:2024-04-04T06:08:04.420+0000 7f0f61600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5751=0-4194304@1} caps={5751=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5751 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x5623139ecc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-04T06:08:04.417 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:08:04.417 INFO:teuthology.orchestra.run.smithi159.stdout: "return_code": 0, 2024-04-04T06:08:04.417 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_tag": "896f50e3-4433-426d-a6f7-ef43bb302a66", 2024-04-04T06:08:04.417 INFO:teuthology.orchestra.run.smithi159.stdout: "mode": "asynchronous" 2024-04-04T06:08:04.417 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:08:04.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.420+0000 7fe959000640 1 -- 172.21.15.159:0/652028591 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe93006c8f0 msgr2=0x7fe93006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.420+0000 7fe959000640 1 --2- 172.21.15.159:0/652028591 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe93006c8f0 0x7fe93006edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe93c006fd0 tx=0x7fe93c00acf0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.420+0000 7fe959000640 1 -- 172.21.15.159:0/652028591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9540acf10 msgr2=0x7fe954131dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.420+0000 7fe959000640 1 --2- 172.21.15.159:0/652028591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9540acf10 0x7fe954131dd0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009f40 tx=0x7fe94c004a40 comp rx=0 tx=0).stop 2024-04-04T06:08:04.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.420+0000 7fe959000640 1 -- 172.21.15.159:0/652028591 shutdown_connections 2024-04-04T06:08:04.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.420+0000 7fe959000640 1 --2- 172.21.15.159:0/652028591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9540acf10 0x7fe954131dd0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.420+0000 7fe959000640 1 --2- 172.21.15.159:0/652028591 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe93006c8f0 0x7fe93006edb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.420+0000 7fe959000640 1 --2- 172.21.15.159:0/652028591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe95409dcf0 0x7fe954131890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.420+0000 7fe959000640 1 --2- 172.21.15.159:0/652028591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe954071710 0x7fe9540ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.420+0000 7fe959000640 1 -- 172.21.15.159:0/652028591 >> 172.21.15.159:0/652028591 conn(0x7fe95406bba0 msgr2=0x7fe95415fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:04.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.421+0000 7fe959000640 1 -- 172.21.15.159:0/652028591 shutdown_connections 2024-04-04T06:08:04.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.421+0000 7fe959000640 1 -- 172.21.15.159:0/652028591 wait complete. 2024-04-04T06:08:04.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:08:04.432 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-04T06:08:04.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.627+0000 7fe6e9800640 1 Processor -- start 2024-04-04T06:08:04.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.627+0000 7fe6e9800640 1 -- start start 2024-04-04T06:08:04.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.628+0000 7fe6e9800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe6e415e660 0x7fe6e406f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.628+0000 7fe6e9800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe6e406f8b0 0x7fe6e406c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.628+0000 7fe6e9800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e406cac0 0x7fe6e406e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.629+0000 7fe6e9800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe6e40703d0 con 0x7fe6e415e660 2024-04-04T06:08:04.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.629+0000 7fe6e9800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe6e40ae6f0 con 0x7fe6e406f8b0 2024-04-04T06:08:04.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.629+0000 7fe6e9800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e406fe10 con 0x7fe6e406cac0 2024-04-04T06:08:04.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.629+0000 7fe6e2a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe6e415e660 0x7fe6e406f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.629+0000 7fe6e2a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe6e415e660 0x7fe6e406f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45526/0 (socket says 172.21.15.159:45526) 2024-04-04T06:08:04.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.629+0000 7fe6e2a00640 1 -- 172.21.15.159:0/2925864015 learned_addr learned my addr 172.21.15.159:0/2925864015 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:04.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.629+0000 7fe6e2000640 1 --2- 172.21.15.159:0/2925864015 >> v2:172.21.15.178:3300/0 conn(0x7fe6e406f8b0 0x7fe6e406c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.629+0000 7fe6e3400640 1 --2- 172.21.15.159:0/2925864015 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e406cac0 0x7fe6e406e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.629+0000 7fe6e2a00640 1 -- 172.21.15.159:0/2925864015 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e406cac0 msgr2=0x7fe6e406e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.629+0000 7fe6e2a00640 1 --2- 172.21.15.159:0/2925864015 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e406cac0 0x7fe6e406e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.629+0000 7fe6e2a00640 1 -- 172.21.15.159:0/2925864015 >> v2:172.21.15.178:3300/0 conn(0x7fe6e406f8b0 msgr2=0x7fe6e406c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.629+0000 7fe6e2a00640 1 --2- 172.21.15.159:0/2925864015 >> v2:172.21.15.178:3300/0 conn(0x7fe6e406f8b0 0x7fe6e406c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.629+0000 7fe6e2a00640 1 -- 172.21.15.159:0/2925864015 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e40700f0 con 0x7fe6e415e660 2024-04-04T06:08:04.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.630+0000 7fe6e2a00640 1 --2- 172.21.15.159:0/2925864015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6e415e660 0x7fe6e406f370 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4067480 tx=0x7fe6d402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664d859ebdca6c7f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:04.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.631+0000 7fe6e1600640 1 -- 172.21.15.159:0/2925864015 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d402e070 con 0x7fe6e415e660 2024-04-04T06:08:04.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.631+0000 7fe6e1600640 1 -- 172.21.15.159:0/2925864015 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d4037070 con 0x7fe6e415e660 2024-04-04T06:08:04.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.631+0000 7fe6e1600640 1 -- 172.21.15.159:0/2925864015 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4031a00 con 0x7fe6e415e660 2024-04-04T06:08:04.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.632+0000 7fe6e9800640 1 -- 172.21.15.159:0/2925864015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6e415e660 msgr2=0x7fe6e406f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.632+0000 7fe6e9800640 1 --2- 172.21.15.159:0/2925864015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6e415e660 0x7fe6e406f370 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4067480 tx=0x7fe6d402b880 comp rx=0 tx=0).stop 2024-04-04T06:08:04.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.632+0000 7fe6e9800640 1 -- 172.21.15.159:0/2925864015 shutdown_connections 2024-04-04T06:08:04.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.632+0000 7fe6e9800640 1 --2- 172.21.15.159:0/2925864015 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e406cac0 0x7fe6e406e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.632+0000 7fe6e9800640 1 --2- 172.21.15.159:0/2925864015 >> v2:172.21.15.178:3300/0 conn(0x7fe6e406f8b0 0x7fe6e406c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.632+0000 7fe6e9800640 1 --2- 172.21.15.159:0/2925864015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6e415e660 0x7fe6e406f370 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.632+0000 7fe6e9800640 1 -- 172.21.15.159:0/2925864015 >> 172.21.15.159:0/2925864015 conn(0x7fe6e4071e20 msgr2=0x7fe6e4072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:04.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.632+0000 7fe6e9800640 1 -- 172.21.15.159:0/2925864015 shutdown_connections 2024-04-04T06:08:04.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.632+0000 7fe6e9800640 1 -- 172.21.15.159:0/2925864015 wait complete. 2024-04-04T06:08:04.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.634+0000 7fe6e9800640 1 Processor -- start 2024-04-04T06:08:04.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.634+0000 7fe6e9800640 1 -- start start 2024-04-04T06:08:04.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.635+0000 7fe6e9800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6e406cac0 0x7fe6e41583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.635+0000 7fe6e9800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e406f8b0 0x7fe6e4161370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.635+0000 7fe6e9800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6e415e660 0x7fe6e4144e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.635+0000 7fe6e9800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e415ea40 con 0x7fe6e406cac0 2024-04-04T06:08:04.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.635+0000 7fe6e9800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e40aec50 con 0x7fe6e415e660 2024-04-04T06:08:04.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.635+0000 7fe6e9800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e40cea90 con 0x7fe6e406f8b0 2024-04-04T06:08:04.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.636+0000 7fe6e2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6e406cac0 0x7fe6e41583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.636+0000 7fe6e2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6e406cac0 0x7fe6e41583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45528/0 (socket says 172.21.15.159:45528) 2024-04-04T06:08:04.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.636+0000 7fe6e2a00640 1 -- 172.21.15.159:0/2570270165 learned_addr learned my addr 172.21.15.159:0/2570270165 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:04.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.636+0000 7fe6e3400640 1 --2- 172.21.15.159:0/2570270165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6e415e660 0x7fe6e4144e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.636+0000 7fe6e2000640 1 --2- 172.21.15.159:0/2570270165 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e406f8b0 0x7fe6e4161370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.636+0000 7fe6e2a00640 1 -- 172.21.15.159:0/2570270165 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e406f8b0 msgr2=0x7fe6e4161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.636+0000 7fe6e2a00640 1 --2- 172.21.15.159:0/2570270165 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e406f8b0 0x7fe6e4161370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.636+0000 7fe6e2a00640 1 -- 172.21.15.159:0/2570270165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6e415e660 msgr2=0x7fe6e4144e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.636+0000 7fe6e2a00640 1 --2- 172.21.15.159:0/2570270165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6e415e660 0x7fe6e4144e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.636+0000 7fe6e2a00640 1 -- 172.21.15.159:0/2570270165 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e415fb90 con 0x7fe6e406cac0 2024-04-04T06:08:04.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.637+0000 7fe6e2a00640 1 --2- 172.21.15.159:0/2570270165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6e406cac0 0x7fe6e41583b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4000c00 tx=0x7fe6d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:04.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.637+0000 7fe6daa00640 1 -- 172.21.15.159:0/2570270165 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4032670 con 0x7fe6e406cac0 2024-04-04T06:08:04.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.637+0000 7fe6daa00640 1 -- 172.21.15.159:0/2570270165 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d4032850 con 0x7fe6e406cac0 2024-04-04T06:08:04.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.637+0000 7fe6daa00640 1 -- 172.21.15.159:0/2570270165 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4032c40 con 0x7fe6e406cac0 2024-04-04T06:08:04.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.637+0000 7fe6e9800640 1 -- 172.21.15.159:0/2570270165 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e415fe70 con 0x7fe6e406cac0 2024-04-04T06:08:04.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.637+0000 7fe6e9800640 1 -- 172.21.15.159:0/2570270165 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e4160430 con 0x7fe6e406cac0 2024-04-04T06:08:04.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.639+0000 7fe6daa00640 1 -- 172.21.15.159:0/2570270165 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe6d40453f0 con 0x7fe6e406cac0 2024-04-04T06:08:04.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.640+0000 7fe6daa00640 1 --2- 172.21.15.159:0/2570270165 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe6c406c920 0x7fe6c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.640+0000 7fe6daa00640 1 -- 172.21.15.159:0/2570270165 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fe6d40ad960 con 0x7fe6e406cac0 2024-04-04T06:08:04.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.641+0000 7fe6e2000640 1 --2- 172.21.15.159:0/2570270165 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe6c406c920 0x7fe6c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.641+0000 7fe6ec591740 1 Processor -- start 2024-04-04T06:08:04.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.641+0000 7fe6ec591740 1 -- start start 2024-04-04T06:08:04.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.642+0000 7fe6e2000640 1 --2- 172.21.15.159:0/2570270165 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe6c406c920 0x7fe6c406ede0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe6e415dde0 tx=0x7fe6d0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:04.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.642+0000 7fe6ec591740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2c6aea580 0x55a2c6abd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.642+0000 7fe6ec591740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a2c6aea960 0x55a2c6aed310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.642+0000 7fe6ec591740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6abda60 0x55a2c6aed850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.642+0000 7fe6ec591740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a2c6a286d0 con 0x55a2c6abda60 2024-04-04T06:08:04.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.642+0000 7fe6ec591740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a2c69d0770 con 0x55a2c6aea580 2024-04-04T06:08:04.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.642+0000 7fe6ec591740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a2c6a24680 con 0x55a2c6aea960 2024-04-04T06:08:04.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.643+0000 7fe6e3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6abda60 0x55a2c6aed850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.643+0000 7fe6e3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6abda60 0x55a2c6aed850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45538/0 (socket says 172.21.15.159:45538) 2024-04-04T06:08:04.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.643+0000 7fe6e3400640 1 -- 172.21.15.159:0/4244856993 learned_addr learned my addr 172.21.15.159:0/4244856993 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:04.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.643+0000 7fe6e2a00640 1 --2- 172.21.15.159:0/4244856993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2c6aea580 0x55a2c6abd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.643+0000 7fe6e2000640 1 --2- 172.21.15.159:0/4244856993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a2c6aea960 0x55a2c6aed310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.643+0000 7fe6e3400640 1 -- 172.21.15.159:0/4244856993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a2c6aea960 msgr2=0x55a2c6aed310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.643+0000 7fe6e3400640 1 --2- 172.21.15.159:0/4244856993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a2c6aea960 0x55a2c6aed310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.643+0000 7fe6e3400640 1 -- 172.21.15.159:0/4244856993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2c6aea580 msgr2=0x55a2c6abd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.643+0000 7fe6e3400640 1 --2- 172.21.15.159:0/4244856993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2c6aea580 0x55a2c6abd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.643+0000 7fe6e3400640 1 -- 172.21.15.159:0/4244856993 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2c68067d0 con 0x55a2c6abda60 2024-04-04T06:08:04.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.643+0000 7fe6e2000640 1 --2- 172.21.15.159:0/4244856993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a2c6aea960 0x55a2c6aed310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:04.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.643+0000 7fe6e3400640 1 --2- 172.21.15.159:0/4244856993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6abda60 0x55a2c6aed850 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00e3f0 tx=0x7fe6dc00e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1d58c3f8e9be43 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:04.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.644+0000 7fe6cf400640 1 -- 172.21.15.159:0/4244856993 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc00f6c0 con 0x55a2c6abda60 2024-04-04T06:08:04.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.644+0000 7fe6cf400640 1 -- 172.21.15.159:0/4244856993 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6dc00f860 con 0x55a2c6abda60 2024-04-04T06:08:04.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.644+0000 7fe6cf400640 1 -- 172.21.15.159:0/4244856993 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc00fc10 con 0x55a2c6abda60 2024-04-04T06:08:04.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.644+0000 7fe6ec591740 1 -- 172.21.15.159:0/4244856993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6abda60 msgr2=0x55a2c6aed850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.644+0000 7fe6ec591740 1 --2- 172.21.15.159:0/4244856993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6abda60 0x55a2c6aed850 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00e3f0 tx=0x7fe6dc00e8b0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.644+0000 7fe6ec591740 1 -- 172.21.15.159:0/4244856993 shutdown_connections 2024-04-04T06:08:04.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.644+0000 7fe6ec591740 1 --2- 172.21.15.159:0/4244856993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6abda60 0x55a2c6aed850 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.644+0000 7fe6ec591740 1 --2- 172.21.15.159:0/4244856993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a2c6aea960 0x55a2c6aed310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.644+0000 7fe6ec591740 1 --2- 172.21.15.159:0/4244856993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2c6aea580 0x55a2c6abd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.644+0000 7fe6ec591740 1 -- 172.21.15.159:0/4244856993 >> 172.21.15.159:0/4244856993 conn(0x55a2c6abf790 msgr2=0x55a2c6b08310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:04.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.645+0000 7fe6ec591740 1 -- 172.21.15.159:0/4244856993 shutdown_connections 2024-04-04T06:08:04.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.645+0000 7fe6ec591740 1 -- 172.21.15.159:0/4244856993 wait complete. 2024-04-04T06:08:04.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.645+0000 7fe6ec591740 1 -- start start 2024-04-04T06:08:04.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.645+0000 7fe6ec591740 1 Processor -- start 2024-04-04T06:08:04.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.646+0000 7fe6ec591740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2c6aea580 0x55a2c6af4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.646+0000 7fe6ec591740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6aea960 0x55a2c6aee510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.646+0000 7fe6ec591740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a2c6b188d0 0x55a2c6b18d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.646+0000 7fe6ec591740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a2c6ac1e30 con 0x55a2c6aea960 2024-04-04T06:08:04.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.647+0000 7fe6ec591740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a2c69e3aa0 con 0x55a2c6aea580 2024-04-04T06:08:04.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.647+0000 7fe6ec591740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a2c6ac4670 con 0x55a2c6b188d0 2024-04-04T06:08:04.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.647+0000 7fe6ec591740 10 client.0 _ll_register_callbacks cb 0x55a2c69e3490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:08:04.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.647+0000 7fe6e2000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6aea960 0x55a2c6aee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.647+0000 7fe6e2000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6aea960 0x55a2c6aee510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45546/0 (socket says 172.21.15.159:45546) 2024-04-04T06:08:04.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.647+0000 7fe6e2000640 1 -- 172.21.15.159:0/1242035082 learned_addr learned my addr 172.21.15.159:0/1242035082 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:04.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.647+0000 7fe6e2000640 1 -- 172.21.15.159:0/1242035082 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a2c6b188d0 msgr2=0x55a2c6b18d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.647+0000 7fe6e2a00640 1 --2- 172.21.15.159:0/1242035082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2c6aea580 0x55a2c6af4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.647+0000 7fe6e2000640 1 --2- 172.21.15.159:0/1242035082 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a2c6b188d0 0x55a2c6b18d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.647+0000 7fe6e2000640 1 -- 172.21.15.159:0/1242035082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2c6aea580 msgr2=0x55a2c6af4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.647+0000 7fe6e2000640 1 --2- 172.21.15.159:0/1242035082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2c6aea580 0x55a2c6af4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.647+0000 7fe6e2000640 1 -- 172.21.15.159:0/1242035082 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2c673c3f0 con 0x55a2c6aea960 2024-04-04T06:08:04.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.647+0000 7fe6e2000640 1 --2- 172.21.15.159:0/1242035082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6aea960 0x55a2c6aee510 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0014490 tx=0x7fe6d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3858106959db7817 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:04.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.648+0000 7fe6c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:08:04.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.648+0000 7fe6c3e00640 1 -- 172.21.15.159:0/1242035082 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0015820 con 0x55a2c6aea960 2024-04-04T06:08:04.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.648+0000 7fe6c3e00640 1 -- 172.21.15.159:0/1242035082 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fe6d00159c0 con 0x55a2c6aea960 2024-04-04T06:08:04.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.648+0000 7fe6ec591740 1 -- 172.21.15.159:0/1242035082 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6dc00f860 con 0x55a2c6aea960 2024-04-04T06:08:04.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.648+0000 7fe6c3e00640 1 -- 172.21.15.159:0/1242035082 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe6d0015bc0 con 0x55a2c6aea960 2024-04-04T06:08:04.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.648+0000 7fe6ec591740 10 client.5814 fetch_fsmap learned FSMap version 25 2024-04-04T06:08:04.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.648+0000 7fe6ec591740 1 -- 172.21.15.159:0/1242035082 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x55a2c66fc0f0 con 0x55a2c6aea960 2024-04-04T06:08:04.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.648+0000 7fe6c3e00640 1 -- 172.21.15.159:0/1242035082 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6dc00f860 con 0x55a2c6aea960 2024-04-04T06:08:04.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.649+0000 7fe6c3e00640 1 -- 172.21.15.159:0/1242035082 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7fe6d0031020 con 0x55a2c6aea960 2024-04-04T06:08:04.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.649+0000 7fe6ec591740 10 client.5814 fetch_fsmap finished waiting for FSMap version 25 2024-04-04T06:08:04.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.649+0000 7fe6ec591740 10 client.5814 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-04T06:08:04.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.649+0000 7fe6ec591740 20 client.5814 populate_metadata read hostname 'smithi159' 2024-04-04T06:08:04.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.649+0000 7fe6ec591740 1 --2- 172.21.15.159:0/1242035082 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55a2c6af2110 0x55a2c6af25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.649+0000 7fe6ec591740 4 client.5814 mds_command: new command op to 5589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:08:04.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.649+0000 7fe6ec591740 1 -- 172.21.15.159:0/1242035082 --> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2c6a35c10 con 0x55a2c6af2110 2024-04-04T06:08:04.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.649+0000 7fe6e2a00640 1 --2- 172.21.15.159:0/1242035082 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55a2c6af2110 0x55a2c6af25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.649+0000 7fe6e2a00640 1 --2- 172.21.15.159:0/1242035082 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55a2c6af2110 0x55a2c6af25a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c487ef7b5eda84a2 server_cookie=46d31517fcd0b02f in_seq=0 out_seq=0 2024-04-04T06:08:04.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.649+0000 7fe6c3e00640 10 client.5814 ms_handle_connect on v2:172.21.15.159:6834/1867245367 2024-04-04T06:08:04.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.650+0000 7fe6c3e00640 1 -- 172.21.15.159:0/1242035082 <== mds.0 v2:172.21.15.159:6834/1867245367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a2c6a35c10 con 0x55a2c6af2110 2024-04-04T06:08:04.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.650+0000 7fe6c3e00640 10 client.5814 handle_command_reply: tid=0 2024-04-04T06:08:04.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.650+0000 7fe6ec591740 1 client.5814 shutdown 2024-04-04T06:08:04.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.650+0000 7fe6ec591740 1 -- 172.21.15.159:0/1242035082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6aea960 msgr2=0x55a2c6aee510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.650+0000 7fe6ec591740 1 --2- 172.21.15.159:0/1242035082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6aea960 0x55a2c6aee510 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0014490 tx=0x7fe6d0004730 comp rx=0 tx=0).stop 2024-04-04T06:08:04.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.650+0000 7fe6ec591740 1 -- 172.21.15.159:0/1242035082 shutdown_connections 2024-04-04T06:08:04.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.650+0000 7fe6ec591740 1 --2- 172.21.15.159:0/1242035082 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55a2c6af2110 0x55a2c6af25a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.650+0000 7fe6ec591740 1 --2- 172.21.15.159:0/1242035082 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a2c6b188d0 0x55a2c6b18d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.650+0000 7fe6ec591740 1 --2- 172.21.15.159:0/1242035082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6aea960 0x55a2c6aee510 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.650+0000 7fe6ec591740 1 --2- 172.21.15.159:0/1242035082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2c6aea580 0x55a2c6af4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.650+0000 7fe6c3e00640 0 client.5814 ms_handle_reset on v2:172.21.15.159:6834/1867245367 2024-04-04T06:08:04.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.650+0000 7fe6ec591740 1 -- 172.21.15.159:0/1242035082 >> 172.21.15.159:0/1242035082 conn(0x55a2c6abda60 msgr2=0x55a2c6b0b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:04.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.651+0000 7fe6ec591740 1 -- 172.21.15.159:0/1242035082 shutdown_connections 2024-04-04T06:08:04.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.651+0000 7fe6ec591740 1 -- 172.21.15.159:0/1242035082 wait complete. 2024-04-04T06:08:04.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.651+0000 7fe6ec591740 20 client.5814 trim_cache size 0 max 16384 2024-04-04T06:08:04.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.670+0000 7fe6ec591740 1 Processor -- start 2024-04-04T06:08:04.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.670+0000 7fe6ec591740 1 -- start start 2024-04-04T06:08:04.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.671+0000 7fe6ec591740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a2c6aea580 0x55a2c6b0b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.671+0000 7fe6ec591740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2c6aea960 0x55a2c6af4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.671+0000 7fe6ec591740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6b188d0 0x55a2c6ac2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.671+0000 7fe6ec591740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e4200a60 con 0x55a2c6b188d0 2024-04-04T06:08:04.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.671+0000 7fe6ec591740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e41578c0 con 0x55a2c6aea960 2024-04-04T06:08:04.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.671+0000 7fe6ec591740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e415acd0 con 0x55a2c6aea580 2024-04-04T06:08:04.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.671+0000 7fe6e3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6b188d0 0x55a2c6ac2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.671+0000 7fe6e3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6b188d0 0x55a2c6ac2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45562/0 (socket says 172.21.15.159:45562) 2024-04-04T06:08:04.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.671+0000 7fe6e3400640 1 -- 172.21.15.159:0/1575544345 learned_addr learned my addr 172.21.15.159:0/1575544345 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:04.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.671+0000 7fe6e3400640 1 -- 172.21.15.159:0/1575544345 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a2c6aea580 msgr2=0x55a2c6b0b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.671+0000 7fe6e3400640 1 --2- 172.21.15.159:0/1575544345 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a2c6aea580 0x55a2c6b0b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.671+0000 7fe6e3400640 1 -- 172.21.15.159:0/1575544345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2c6aea960 msgr2=0x55a2c6af4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.671+0000 7fe6e3400640 1 --2- 172.21.15.159:0/1575544345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2c6aea960 0x55a2c6af4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.671+0000 7fe6e3400640 1 -- 172.21.15.159:0/1575544345 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2c6b19080 con 0x55a2c6b188d0 2024-04-04T06:08:04.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.672+0000 7fe6e3400640 1 --2- 172.21.15.159:0/1575544345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6b188d0 0x55a2c6ac2950 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00e8f0 tx=0x7fe6dc011070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aca906d86767758 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:04.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.672+0000 7fe6cea00640 1 -- 172.21.15.159:0/1575544345 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc026070 con 0x55a2c6b188d0 2024-04-04T06:08:04.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.672+0000 7fe6cea00640 1 -- 172.21.15.159:0/1575544345 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6dc007c40 con 0x55a2c6b188d0 2024-04-04T06:08:04.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.672+0000 7fe6cea00640 1 -- 172.21.15.159:0/1575544345 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc003990 con 0x55a2c6b188d0 2024-04-04T06:08:04.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.672+0000 7fe6ec591740 1 -- 172.21.15.159:0/1575544345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6b188d0 msgr2=0x55a2c6ac2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.672+0000 7fe6ec591740 1 --2- 172.21.15.159:0/1575544345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6b188d0 0x55a2c6ac2950 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00e8f0 tx=0x7fe6dc011070 comp rx=0 tx=0).stop 2024-04-04T06:08:04.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.672+0000 7fe6ec591740 1 -- 172.21.15.159:0/1575544345 shutdown_connections 2024-04-04T06:08:04.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.672+0000 7fe6ec591740 1 --2- 172.21.15.159:0/1575544345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6b188d0 0x55a2c6ac2950 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.672+0000 7fe6ec591740 1 --2- 172.21.15.159:0/1575544345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2c6aea960 0x55a2c6af4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.672+0000 7fe6ec591740 1 --2- 172.21.15.159:0/1575544345 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a2c6aea580 0x55a2c6b0b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.672+0000 7fe6ec591740 1 -- 172.21.15.159:0/1575544345 >> 172.21.15.159:0/1575544345 conn(0x55a2c6abda60 msgr2=0x55a2c6b08a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:04.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.673+0000 7fe6ec591740 1 -- 172.21.15.159:0/1575544345 shutdown_connections 2024-04-04T06:08:04.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.673+0000 7fe6ec591740 1 -- 172.21.15.159:0/1575544345 wait complete. 2024-04-04T06:08:04.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.673+0000 7fe6ec591740 1 -- start start 2024-04-04T06:08:04.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.673+0000 7fe6ec591740 1 Processor -- start 2024-04-04T06:08:04.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.674+0000 7fe6ec591740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2c6aea580 0x55a2c6af4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.674+0000 7fe6ec591740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a2c6aea960 0x55a2c6b09fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.674+0000 7fe6ec591740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6b188d0 0x55a2c6ae87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.674+0000 7fe6ec591740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a2c6a5c8b0 con 0x55a2c6b188d0 2024-04-04T06:08:04.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.674+0000 7fe6ec591740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e4157270 con 0x55a2c6aea580 2024-04-04T06:08:04.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.674+0000 7fe6ec591740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a2c6af2620 con 0x55a2c6aea960 2024-04-04T06:08:04.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.674+0000 7fe6ec591740 10 client.0 _ll_register_callbacks cb 0x7fe6d000fca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:08:04.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.675+0000 7fe6e3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6b188d0 0x55a2c6ae87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.675+0000 7fe6e3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6b188d0 0x55a2c6ae87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45568/0 (socket says 172.21.15.159:45568) 2024-04-04T06:08:04.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.675+0000 7fe6e3400640 1 -- 172.21.15.159:0/1890288909 learned_addr learned my addr 172.21.15.159:0/1890288909 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:04.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.675+0000 7fe6e2a00640 1 --2- 172.21.15.159:0/1890288909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2c6aea580 0x55a2c6af4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.675+0000 7fe6e2000640 1 --2- 172.21.15.159:0/1890288909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a2c6aea960 0x55a2c6b09fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.675+0000 7fe6e3400640 1 -- 172.21.15.159:0/1890288909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a2c6aea960 msgr2=0x55a2c6b09fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.675+0000 7fe6e3400640 1 --2- 172.21.15.159:0/1890288909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a2c6aea960 0x55a2c6b09fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.675+0000 7fe6e3400640 1 -- 172.21.15.159:0/1890288909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2c6aea580 msgr2=0x55a2c6af4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:04.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.675+0000 7fe6e3400640 1 --2- 172.21.15.159:0/1890288909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2c6aea580 0x55a2c6af4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.675+0000 7fe6e3400640 1 -- 172.21.15.159:0/1890288909 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2c6b85de0 con 0x55a2c6b188d0 2024-04-04T06:08:04.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.676+0000 7fe6e3400640 1 --2- 172.21.15.159:0/1890288909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6b188d0 0x55a2c6ae87d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00e8f0 tx=0x7fe6dc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3489898793df27 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:04.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.676+0000 7fe6c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:08:04.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.676+0000 7fe6c3e00640 1 -- 172.21.15.159:0/1890288909 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc02f070 con 0x55a2c6b188d0 2024-04-04T06:08:04.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.676+0000 7fe6c3e00640 1 -- 172.21.15.159:0/1890288909 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fe6dc003b90 con 0x55a2c6b188d0 2024-04-04T06:08:04.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.676+0000 7fe6ec591740 1 -- 172.21.15.159:0/1890288909 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6dc007c40 con 0x55a2c6b188d0 2024-04-04T06:08:04.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.676+0000 7fe6c3e00640 1 -- 172.21.15.159:0/1890288909 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe6dc03a050 con 0x55a2c6b188d0 2024-04-04T06:08:04.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.676+0000 7fe6ec591740 10 client.5820 fetch_fsmap learned FSMap version 25 2024-04-04T06:08:04.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.676+0000 7fe6ec591740 1 -- 172.21.15.159:0/1890288909 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x55a2c6b863a0 con 0x55a2c6b188d0 2024-04-04T06:08:04.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.676+0000 7fe6c3e00640 1 -- 172.21.15.159:0/1890288909 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6dc007c40 con 0x55a2c6b188d0 2024-04-04T06:08:04.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.677+0000 7fe6c3e00640 1 -- 172.21.15.159:0/1890288909 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7fe6dc03d020 con 0x55a2c6b188d0 2024-04-04T06:08:04.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.677+0000 7fe6ec591740 10 client.5820 fetch_fsmap finished waiting for FSMap version 25 2024-04-04T06:08:04.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.677+0000 7fe6ec591740 10 client.5820 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-04T06:08:04.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.677+0000 7fe6ec591740 20 client.5820 populate_metadata read hostname 'smithi159' 2024-04-04T06:08:04.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.677+0000 7fe6ec591740 1 --2- 172.21.15.159:0/1890288909 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55a2c6ae8d10 0x55a2c6b825a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:04.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.677+0000 7fe6ec591740 4 client.5820 mds_command: new command op to 5589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-04T06:08:04.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.677+0000 7fe6ec591740 1 -- 172.21.15.159:0/1890288909 --> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a2c6a2ff60 con 0x55a2c6ae8d10 2024-04-04T06:08:04.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.677+0000 7fe6e2a00640 1 --2- 172.21.15.159:0/1890288909 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55a2c6ae8d10 0x55a2c6b825a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:04.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.678+0000 7fe6c3e00640 10 client.5820 ms_handle_connect on v2:172.21.15.159:6834/1867245367 2024-04-04T06:08:04.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.678+0000 7fe6e2a00640 1 --2- 172.21.15.159:0/1890288909 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55a2c6ae8d10 0x55a2c6b825a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b523fad7c5c5d16f server_cookie=26344c48ce27d0f1 in_seq=0 out_seq=0 2024-04-04T06:08:04.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.678+0000 7fe6c3e00640 1 -- 172.21.15.159:0/1890288909 <== mds.0 v2:172.21.15.159:6834/1867245367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55a2c6a2ff60 con 0x55a2c6ae8d10 2024-04-04T06:08:04.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.678+0000 7fe6c3e00640 10 client.5820 handle_command_reply: tid=0 2024-04-04T06:08:04.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.678+0000 7fe6ec591740 1 client.5820 shutdown 2024-04-04T06:08:04.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.678+0000 7fe6ec591740 1 -- 172.21.15.159:0/1890288909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6b188d0 msgr2=0x55a2c6ae87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.678+0000 7fe6ec591740 1 --2- 172.21.15.159:0/1890288909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6b188d0 0x55a2c6ae87d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00e8f0 tx=0x7fe6dc004ab0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.678+0000 7fe6ec591740 1 -- 172.21.15.159:0/1890288909 shutdown_connections 2024-04-04T06:08:04.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.678+0000 7fe6ec591740 1 --2- 172.21.15.159:0/1890288909 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55a2c6ae8d10 0x55a2c6b825a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.678+0000 7fe6ec591740 1 --2- 172.21.15.159:0/1890288909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2c6b188d0 0x55a2c6ae87d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.678+0000 7fe6ec591740 1 --2- 172.21.15.159:0/1890288909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a2c6aea960 0x55a2c6b09fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.678+0000 7fe6ec591740 1 --2- 172.21.15.159:0/1890288909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2c6aea580 0x55a2c6af4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.678+0000 7fe6c3e00640 0 client.5820 ms_handle_reset on v2:172.21.15.159:6834/1867245367 2024-04-04T06:08:04.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.678+0000 7fe6ec591740 1 -- 172.21.15.159:0/1890288909 >> 172.21.15.159:0/1890288909 conn(0x55a2c6abda60 msgr2=0x55a2c6ac41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:04.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.678+0000 7fe6ec591740 1 -- 172.21.15.159:0/1890288909 shutdown_connections 2024-04-04T06:08:04.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.678+0000 7fe6ec591740 1 -- 172.21.15.159:0/1890288909 wait complete. 2024-04-04T06:08:04.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.679+0000 7fe6ec591740 20 client.5820 trim_cache size 0 max 16384 2024-04-04T06:08:04.676 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:08:04.676 INFO:teuthology.orchestra.run.smithi159.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-04T06:08:04.676 INFO:teuthology.orchestra.run.smithi159.stdout: "scrubs": { 2024-04-04T06:08:04.676 INFO:teuthology.orchestra.run.smithi159.stdout: "896f50e3-4433-426d-a6f7-ef43bb302a66": { 2024-04-04T06:08:04.676 INFO:teuthology.orchestra.run.smithi159.stdout: "path": "/", 2024-04-04T06:08:04.676 INFO:teuthology.orchestra.run.smithi159.stdout: "tag": "896f50e3-4433-426d-a6f7-ef43bb302a66", 2024-04-04T06:08:04.677 INFO:teuthology.orchestra.run.smithi159.stdout: "options": "recursive,repair" 2024-04-04T06:08:04.677 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:04.677 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:04.677 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:08:04.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.679+0000 7fe6e9800640 1 -- 172.21.15.159:0/2570270165 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe6c406c920 msgr2=0x7fe6c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.679+0000 7fe6e9800640 1 --2- 172.21.15.159:0/2570270165 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe6c406c920 0x7fe6c406ede0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe6e415dde0 tx=0x7fe6d0008040 comp rx=0 tx=0).stop 2024-04-04T06:08:04.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.679+0000 7fe6e9800640 1 -- 172.21.15.159:0/2570270165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6e406cac0 msgr2=0x7fe6e41583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:04.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.679+0000 7fe6e9800640 1 --2- 172.21.15.159:0/2570270165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6e406cac0 0x7fe6e41583b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4000c00 tx=0x7fe6d40046c0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.679+0000 7fe6e9800640 1 -- 172.21.15.159:0/2570270165 shutdown_connections 2024-04-04T06:08:04.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.679+0000 7fe6e9800640 1 --2- 172.21.15.159:0/2570270165 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe6c406c920 0x7fe6c406ede0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.679+0000 7fe6e9800640 1 --2- 172.21.15.159:0/2570270165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6e415e660 0x7fe6e4144e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.679+0000 7fe6e9800640 1 --2- 172.21.15.159:0/2570270165 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6e406f8b0 0x7fe6e4161370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.679+0000 7fe6e9800640 1 --2- 172.21.15.159:0/2570270165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6e406cac0 0x7fe6e41583b0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:04.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.679+0000 7fe6e9800640 1 -- 172.21.15.159:0/2570270165 >> 172.21.15.159:0/2570270165 conn(0x7fe6e4071e20 msgr2=0x7fe6e4072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:04.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.679+0000 7fe6e9800640 1 -- 172.21.15.159:0/2570270165 shutdown_connections 2024-04-04T06:08:04.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:04.680+0000 7fe6e9800640 1 -- 172.21.15.159:0/2570270165 wait complete. 2024-04-04T06:08:04.690 INFO:tasks.cephfs.filesystem:scrub status for tag:896f50e3-4433-426d-a6f7-ef43bb302a66 - {'path': '/', 'tag': '896f50e3-4433-426d-a6f7-ef43bb302a66', 'options': 'recursive,repair'} 2024-04-04T06:08:05.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:05.141+0000 7f92df400640 1 -- 172.21.15.159:0/3370956774 <== mon.0 v2:172.21.15.159:3300/0 7 ==== log(6 entries from seq 1 at 2024-04-04T06:08:04.419529+0000) ==== 1078+0+0 (secure 0 0 0) 0x7f92d80ae000 con 0x7f92cc0039b0 2024-04-04T06:08:05.136 INFO:teuthology.orchestra.run.smithi159.stdout: cluster: 2024-04-04T06:08:05.136 INFO:teuthology.orchestra.run.smithi159.stdout: id: 30311045-f714-48fa-832b-4cb2ba8bf0bf 2024-04-04T06:08:05.136 INFO:teuthology.orchestra.run.smithi159.stdout: health: HEALTH_OK 2024-04-04T06:08:05.136 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:05.136 INFO:teuthology.orchestra.run.smithi159.stdout: services: 2024-04-04T06:08:05.136 INFO:teuthology.orchestra.run.smithi159.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-04-04T06:08:05.136 INFO:teuthology.orchestra.run.smithi159.stdout: mgr: x(active, since 2m), standbys: y 2024-04-04T06:08:05.136 INFO:teuthology.orchestra.run.smithi159.stdout: mds: 1/1 daemons up, 3 standby 2024-04-04T06:08:05.136 INFO:teuthology.orchestra.run.smithi159.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-04-04T06:08:05.136 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:05.137 INFO:teuthology.orchestra.run.smithi159.stdout: data: 2024-04-04T06:08:05.137 INFO:teuthology.orchestra.run.smithi159.stdout: volumes: 1/1 healthy 2024-04-04T06:08:05.137 INFO:teuthology.orchestra.run.smithi159.stdout: pools: 3 pools, 97 pgs 2024-04-04T06:08:05.137 INFO:teuthology.orchestra.run.smithi159.stdout: objects: 24 objects, 579 KiB 2024-04-04T06:08:05.137 INFO:teuthology.orchestra.run.smithi159.stdout: usage: 233 MiB used, 720 GiB / 720 GiB avail 2024-04-04T06:08:05.137 INFO:teuthology.orchestra.run.smithi159.stdout: pgs: 97 active+clean 2024-04-04T06:08:05.137 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:05.137 INFO:teuthology.orchestra.run.smithi159.stdout: io: 2024-04-04T06:08:05.137 INFO:teuthology.orchestra.run.smithi159.stdout: client: 1.6 KiB/s wr, 0 op/s rd, 5 op/s wr 2024-04-04T06:08:05.137 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:05.137 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:05.137 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:08:04.419529+0000 mds.a [INF] scrub queued for path: / 2024-04-04T06:08:05.137 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:08:04.419532+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-04T06:08:05.138 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:08:04.419536+0000 mds.a [INF] scrub summary: active paths [/] 2024-04-04T06:08:05.138 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:08:04.421313+0000 mds.a [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-04-04T06:08:05.138 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:08:04.421324+0000 mds.a [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-04-04T06:08:05.138 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:08:04.421364+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-04T06:08:09.162 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:08:07.666152+0000 mds.a [INF] scrub summary: idle 2024-04-04T06:08:09.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:09.167+0000 7f92df400640 1 -- 172.21.15.159:0/3370956774 <== mon.0 v2:172.21.15.159:3300/0 8 ==== log(1 entries from seq 7 at 2024-04-04T06:08:07.666152+0000) ==== 189+0+0 (secure 0 0 0) 0x7f92d8079a30 con 0x7f92cc0039b0 2024-04-04T06:08:11.188 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:08:11.195 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:08:33.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:33.249+0000 7f92df400640 1 -- 172.21.15.159:0/3370956774 <== mon.0 v2:172.21.15.159:3300/0 9 ==== log(4 entries from seq 131 at 2024-04-04T06:08:32.838696+0000) ==== 1122+0+0 (secure 0 0 0) 0x7f92d8079c90 con 0x7f92cc0039b0 2024-04-04T06:08:34.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.262+0000 7f92df400640 1 -- 172.21.15.159:0/3370956774 <== mon.0 v2:172.21.15.159:3300/0 10 ==== log(2 entries from seq 450 at 2024-04-04T06:08:33.246913+0000) ==== 558+0+0 (secure 0 0 0) 0x7f92d8007c80 con 0x7f92cc0039b0 2024-04-04T06:08:34.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:08:34.691 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-04T06:08:34.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.919+0000 7fe3dca00640 1 Processor -- start 2024-04-04T06:08:34.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.920+0000 7fe3dca00640 1 -- start start 2024-04-04T06:08:34.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.921+0000 7fe3dca00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe3d8153580 0x7fe3d80d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:34.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.921+0000 7fe3dca00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe3d80d07c0 0x7fe3d80aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:34.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.921+0000 7fe3dca00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3d80afab0 con 0x7fe3d80af2e0 2024-04-04T06:08:34.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.921+0000 7fe3dca00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe3d80d0c30 con 0x7fe3d80d07c0 2024-04-04T06:08:34.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.921+0000 7fe3dca00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe3d815cc80 con 0x7fe3d8153580 2024-04-04T06:08:34.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.921+0000 7fe3d6800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fe3d80af2e0 0x7fe3d806d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:32772/0 (socket says 172.21.15.159:32772) 2024-04-04T06:08:34.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.922+0000 7fe3d6800640 1 -- 172.21.15.159:0/4293862943 learned_addr learned my addr 172.21.15.159:0/4293862943 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:34.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.921+0000 7fe3cfe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe3d80d07c0 0x7fe3d80aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:34.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.922+0000 7fe3d5e00640 1 --2- 172.21.15.159:0/4293862943 >> v2:172.21.15.178:3300/0 conn(0x7fe3d8153580 0x7fe3d80d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:34.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.922+0000 7fe3cfe00640 1 -- 172.21.15.159:0/4293862943 >> v2:172.21.15.178:3300/0 conn(0x7fe3d8153580 msgr2=0x7fe3d80d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:34.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.922+0000 7fe3cfe00640 1 --2- 172.21.15.159:0/4293862943 >> v2:172.21.15.178:3300/0 conn(0x7fe3d8153580 0x7fe3d80d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.922+0000 7fe3cfe00640 1 -- 172.21.15.159:0/4293862943 >> v1:172.21.15.159:6789/0 conn(0x7fe3d80af2e0 legacy=0x7fe3d806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:34.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.922+0000 7fe3d5400640 1 -- 172.21.15.159:0/4293862943 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2355752904 0 0) 0x7fe3d80afab0 con 0x7fe3d80af2e0 2024-04-04T06:08:34.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.922+0000 7fe3cfe00640 1 -- 172.21.15.159:0/4293862943 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d815da00 con 0x7fe3d80d07c0 2024-04-04T06:08:34.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.923+0000 7fe3cfe00640 1 --2- 172.21.15.159:0/4293862943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3d80d07c0 0x7fe3d80aeda0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80b5270 tx=0x7fe3c002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb89d1dcfc69478 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:34.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.923+0000 7fe3d5400640 1 -- 172.21.15.159:0/4293862943 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c00315a0 con 0x7fe3d80d07c0 2024-04-04T06:08:34.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.924+0000 7fe3d5400640 1 -- 172.21.15.159:0/4293862943 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c0037070 con 0x7fe3d80d07c0 2024-04-04T06:08:34.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.924+0000 7fe3d5400640 1 -- 172.21.15.159:0/4293862943 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0031960 con 0x7fe3d80d07c0 2024-04-04T06:08:34.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.924+0000 7fe3dca00640 1 -- 172.21.15.159:0/4293862943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3d80d07c0 msgr2=0x7fe3d80aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:34.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.924+0000 7fe3dca00640 1 --2- 172.21.15.159:0/4293862943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3d80d07c0 0x7fe3d80aeda0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80b5270 tx=0x7fe3c002b840 comp rx=0 tx=0).stop 2024-04-04T06:08:34.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.924+0000 7fe3dca00640 1 -- 172.21.15.159:0/4293862943 shutdown_connections 2024-04-04T06:08:34.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.924+0000 7fe3dca00640 1 --2- 172.21.15.159:0/4293862943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3d80d07c0 0x7fe3d80aeda0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.924+0000 7fe3dca00640 1 --2- 172.21.15.159:0/4293862943 >> v2:172.21.15.178:3300/0 conn(0x7fe3d8153580 0x7fe3d80d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.924+0000 7fe3dca00640 1 -- 172.21.15.159:0/4293862943 >> 172.21.15.159:0/4293862943 conn(0x7fe3d806b750 msgr2=0x7fe3d806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:34.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.924+0000 7fe3dca00640 1 -- 172.21.15.159:0/4293862943 shutdown_connections 2024-04-04T06:08:34.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.924+0000 7fe3dca00640 1 -- 172.21.15.159:0/4293862943 wait complete. 2024-04-04T06:08:34.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.926+0000 7fe3dca00640 1 Processor -- start 2024-04-04T06:08:34.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.926+0000 7fe3dca00640 1 -- start start 2024-04-04T06:08:34.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.927+0000 7fe3dca00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3d80af2e0 0x7fe3d80cf130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:34.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.927+0000 7fe3dca00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3d80d07c0 0x7fe3d8071610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:34.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.927+0000 7fe3dca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3d8153580 0x7fe3d814a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:34.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.927+0000 7fe3dca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d815d4e0 con 0x7fe3d8153580 2024-04-04T06:08:34.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.927+0000 7fe3dca00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d815d200 con 0x7fe3d80af2e0 2024-04-04T06:08:34.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.927+0000 7fe3dca00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d8153960 con 0x7fe3d80d07c0 2024-04-04T06:08:34.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.927+0000 7fe3d6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3d8153580 0x7fe3d814a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:34.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.927+0000 7fe3d6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3d8153580 0x7fe3d814a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47142/0 (socket says 172.21.15.159:47142) 2024-04-04T06:08:34.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.927+0000 7fe3d6800640 1 -- 172.21.15.159:0/3187740425 learned_addr learned my addr 172.21.15.159:0/3187740425 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:34.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.928+0000 7fe3d5e00640 1 --2- 172.21.15.159:0/3187740425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3d80af2e0 0x7fe3d80cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:34.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.928+0000 7fe3cfe00640 1 --2- 172.21.15.159:0/3187740425 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3d80d07c0 0x7fe3d8071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:34.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.928+0000 7fe3d6800640 1 -- 172.21.15.159:0/3187740425 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3d80d07c0 msgr2=0x7fe3d8071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:34.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.928+0000 7fe3d6800640 1 --2- 172.21.15.159:0/3187740425 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3d80d07c0 0x7fe3d8071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.928+0000 7fe3d6800640 1 -- 172.21.15.159:0/3187740425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3d80af2e0 msgr2=0x7fe3d80cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:34.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.928+0000 7fe3d6800640 1 --2- 172.21.15.159:0/3187740425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3d80af2e0 0x7fe3d80cf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.928+0000 7fe3d6800640 1 -- 172.21.15.159:0/3187740425 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d8071f80 con 0x7fe3d8153580 2024-04-04T06:08:34.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.928+0000 7fe3cfe00640 1 --2- 172.21.15.159:0/3187740425 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3d80d07c0 0x7fe3d8071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:34.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.928+0000 7fe3d6800640 1 --2- 172.21.15.159:0/3187740425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3d8153580 0x7fe3d814a310 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000c700 tx=0x7fe3d000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:34.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.928+0000 7fe3ce000640 1 -- 172.21.15.159:0/3187740425 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0018070 con 0x7fe3d8153580 2024-04-04T06:08:34.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.929+0000 7fe3ce000640 1 -- 172.21.15.159:0/3187740425 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3d000ce10 con 0x7fe3d8153580 2024-04-04T06:08:34.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.929+0000 7fe3ce000640 1 -- 172.21.15.159:0/3187740425 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d000a860 con 0x7fe3d8153580 2024-04-04T06:08:34.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.929+0000 7fe3dca00640 1 -- 172.21.15.159:0/3187740425 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d8072260 con 0x7fe3d8153580 2024-04-04T06:08:34.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.929+0000 7fe3dca00640 1 -- 172.21.15.159:0/3187740425 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d8072820 con 0x7fe3d8153580 2024-04-04T06:08:34.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.931+0000 7fe3ce000640 1 -- 172.21.15.159:0/3187740425 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3d000aa00 con 0x7fe3d8153580 2024-04-04T06:08:34.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.932+0000 7fe3ce000640 1 --2- 172.21.15.159:0/3187740425 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe3b406c8f0 0x7fe3b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:34.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.932+0000 7fe3ce000640 1 -- 172.21.15.159:0/3187740425 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fe3d008b230 con 0x7fe3d8153580 2024-04-04T06:08:34.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.933+0000 7fe3d5e00640 1 --2- 172.21.15.159:0/3187740425 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe3b406c8f0 0x7fe3b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:34.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.933+0000 7fe3df5ea740 1 Processor -- start 2024-04-04T06:08:34.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.933+0000 7fe3df5ea740 1 -- start start 2024-04-04T06:08:34.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.934+0000 7fe3df5ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f015e53580 0x55f015e26520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:34.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.934+0000 7fe3df5ea740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f015e53960 0x55f015e56310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:34.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.934+0000 7fe3df5ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e26a60 0x55f015e56850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:34.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.934+0000 7fe3df5ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f015d916d0 con 0x55f015e26a60 2024-04-04T06:08:34.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.934+0000 7fe3df5ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f015d39770 con 0x55f015e53580 2024-04-04T06:08:34.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.934+0000 7fe3df5ea740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55f015d8d680 con 0x55f015e53960 2024-04-04T06:08:34.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.934+0000 7fe3d5e00640 1 --2- 172.21.15.159:0/3187740425 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe3b406c8f0 0x7fe3b406edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8006fd0 tx=0x7fe3c8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:34.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.934+0000 7fe3d6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e26a60 0x55f015e56850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:34.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.934+0000 7fe3d6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e26a60 0x55f015e56850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47148/0 (socket says 172.21.15.159:47148) 2024-04-04T06:08:34.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.934+0000 7fe3d6800640 1 -- 172.21.15.159:0/3258722016 learned_addr learned my addr 172.21.15.159:0/3258722016 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:34.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.935+0000 7fe3cfe00640 1 --2- 172.21.15.159:0/3258722016 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f015e53960 0x55f015e56310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:34.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.935+0000 7fe3d5e00640 1 --2- 172.21.15.159:0/3258722016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f015e53580 0x55f015e26520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:34.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.935+0000 7fe3d6800640 1 -- 172.21.15.159:0/3258722016 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f015e53960 msgr2=0x55f015e56310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:34.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.935+0000 7fe3d6800640 1 --2- 172.21.15.159:0/3258722016 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f015e53960 0x55f015e56310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.935+0000 7fe3d6800640 1 -- 172.21.15.159:0/3258722016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f015e53580 msgr2=0x55f015e26520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:34.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.935+0000 7fe3d6800640 1 --2- 172.21.15.159:0/3258722016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f015e53580 0x55f015e26520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.935+0000 7fe3d6800640 1 -- 172.21.15.159:0/3258722016 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f015b6f7d0 con 0x55f015e26a60 2024-04-04T06:08:34.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.935+0000 7fe3d6800640 1 --2- 172.21.15.159:0/3258722016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e26a60 0x55f015e56850 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00895f0 tx=0x7fe3d0058d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569b648a8d5f4ef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:34.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.935+0000 7fe3c6600640 1 -- 172.21.15.159:0/3258722016 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0061a60 con 0x55f015e26a60 2024-04-04T06:08:34.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.935+0000 7fe3c6600640 1 -- 172.21.15.159:0/3258722016 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3d0057180 con 0x55f015e26a60 2024-04-04T06:08:34.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.935+0000 7fe3c6600640 1 -- 172.21.15.159:0/3258722016 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d00554f0 con 0x55f015e26a60 2024-04-04T06:08:34.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.936+0000 7fe3df5ea740 1 -- 172.21.15.159:0/3258722016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e26a60 msgr2=0x55f015e56850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:34.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.936+0000 7fe3df5ea740 1 --2- 172.21.15.159:0/3258722016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e26a60 0x55f015e56850 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00895f0 tx=0x7fe3d0058d80 comp rx=0 tx=0).stop 2024-04-04T06:08:34.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.936+0000 7fe3df5ea740 1 -- 172.21.15.159:0/3258722016 shutdown_connections 2024-04-04T06:08:34.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.936+0000 7fe3df5ea740 1 --2- 172.21.15.159:0/3258722016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e26a60 0x55f015e56850 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.936+0000 7fe3df5ea740 1 --2- 172.21.15.159:0/3258722016 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f015e53960 0x55f015e56310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.936+0000 7fe3df5ea740 1 --2- 172.21.15.159:0/3258722016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f015e53580 0x55f015e26520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.936+0000 7fe3df5ea740 1 -- 172.21.15.159:0/3258722016 >> 172.21.15.159:0/3258722016 conn(0x55f015e28790 msgr2=0x55f015e71310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:34.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.936+0000 7fe3df5ea740 1 -- 172.21.15.159:0/3258722016 shutdown_connections 2024-04-04T06:08:34.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.936+0000 7fe3df5ea740 1 -- 172.21.15.159:0/3258722016 wait complete. 2024-04-04T06:08:34.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.937+0000 7fe3df5ea740 1 -- start start 2024-04-04T06:08:34.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.938+0000 7fe3df5ea740 1 Processor -- start 2024-04-04T06:08:34.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.939+0000 7fe3df5ea740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f015e53580 0x55f015e82390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:34.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.939+0000 7fe3df5ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f015e53960 0x55f015e4ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:34.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.939+0000 7fe3df5ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e4cc50 0x55f015e4b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:34.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.939+0000 7fe3df5ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f015e2ae30 con 0x55f015e4cc50 2024-04-04T06:08:34.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.939+0000 7fe3df5ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f015d4caa0 con 0x55f015e53960 2024-04-04T06:08:34.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.939+0000 7fe3df5ea740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55f015e2d670 con 0x55f015e53580 2024-04-04T06:08:34.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.939+0000 7fe3df5ea740 10 client.0 _ll_register_callbacks cb 0x55f015d4c490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:08:34.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.939+0000 7fe3d6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e4cc50 0x55f015e4b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:34.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.940+0000 7fe3d6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e4cc50 0x55f015e4b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47160/0 (socket says 172.21.15.159:47160) 2024-04-04T06:08:34.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.940+0000 7fe3d6800640 1 -- 172.21.15.159:0/1763743221 learned_addr learned my addr 172.21.15.159:0/1763743221 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:34.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.940+0000 7fe3d5e00640 1 --2- 172.21.15.159:0/1763743221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f015e53580 0x55f015e82390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:34.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.940+0000 7fe3d6800640 1 -- 172.21.15.159:0/1763743221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f015e53580 msgr2=0x55f015e82390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:34.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.940+0000 7fe3d6800640 1 --2- 172.21.15.159:0/1763743221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f015e53580 0x55f015e82390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.940+0000 7fe3cfe00640 1 --2- 172.21.15.159:0/1763743221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f015e53960 0x55f015e4ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:34.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.940+0000 7fe3d6800640 1 -- 172.21.15.159:0/1763743221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f015e53960 msgr2=0x55f015e4ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:34.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.940+0000 7fe3d6800640 1 --2- 172.21.15.159:0/1763743221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f015e53960 0x55f015e4ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.940+0000 7fe3d6800640 1 -- 172.21.15.159:0/1763743221 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f015aa53f0 con 0x55f015e4cc50 2024-04-04T06:08:34.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.940+0000 7fe3d6800640 1 --2- 172.21.15.159:0/1763743221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e4cc50 0x55f015e4b640 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0058e10 tx=0x7fe3d0001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4947d410f04951ae server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:34.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.940+0000 7fe3baa00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:08:34.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.940+0000 7fe3baa00640 1 -- 172.21.15.159:0/1763743221 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d00893b0 con 0x55f015e4cc50 2024-04-04T06:08:34.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.940+0000 7fe3baa00640 1 -- 172.21.15.159:0/1763743221 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fe3d0071170 con 0x55f015e4cc50 2024-04-04T06:08:34.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.940+0000 7fe3df5ea740 1 -- 172.21.15.159:0/1763743221 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3d0057180 con 0x55f015e4cc50 2024-04-04T06:08:34.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.941+0000 7fe3baa00640 1 -- 172.21.15.159:0/1763743221 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe3d0076050 con 0x55f015e4cc50 2024-04-04T06:08:34.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.941+0000 7fe3df5ea740 10 client.5835 fetch_fsmap learned FSMap version 25 2024-04-04T06:08:34.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.941+0000 7fe3df5ea740 1 -- 172.21.15.159:0/1763743221 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x55f015a650f0 con 0x55f015e4cc50 2024-04-04T06:08:34.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.941+0000 7fe3baa00640 1 -- 172.21.15.159:0/1763743221 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3d0057180 con 0x55f015e4cc50 2024-04-04T06:08:34.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.941+0000 7fe3baa00640 1 -- 172.21.15.159:0/1763743221 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7fe3d00782f0 con 0x55f015e4cc50 2024-04-04T06:08:34.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.941+0000 7fe3df5ea740 10 client.5835 fetch_fsmap finished waiting for FSMap version 25 2024-04-04T06:08:34.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.941+0000 7fe3df5ea740 10 client.5835 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-04T06:08:34.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.941+0000 7fe3df5ea740 20 client.5835 populate_metadata read hostname 'smithi159' 2024-04-04T06:08:34.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.941+0000 7fe3df5ea740 1 --2- 172.21.15.159:0/1763743221 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55f015e85420 0x55f015e85800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:34.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.941+0000 7fe3df5ea740 4 client.5835 mds_command: new command op to 5589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:08:34.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.941+0000 7fe3df5ea740 1 -- 172.21.15.159:0/1763743221 --> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f015d9ec10 con 0x55f015e85420 2024-04-04T06:08:34.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.942+0000 7fe3d5e00640 1 --2- 172.21.15.159:0/1763743221 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55f015e85420 0x55f015e85800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:34.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.942+0000 7fe3baa00640 10 client.5835 ms_handle_connect on v2:172.21.15.159:6834/1867245367 2024-04-04T06:08:34.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.942+0000 7fe3d5e00640 1 --2- 172.21.15.159:0/1763743221 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55f015e85420 0x55f015e85800 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35bdd49eacad8d63 server_cookie=fafe97d9555004f7 in_seq=0 out_seq=0 2024-04-04T06:08:34.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.943+0000 7fe3baa00640 1 -- 172.21.15.159:0/1763743221 <== mds.0 v2:172.21.15.159:6834/1867245367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f015d9ec10 con 0x55f015e85420 2024-04-04T06:08:34.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.943+0000 7fe3baa00640 10 client.5835 handle_command_reply: tid=0 2024-04-04T06:08:34.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.943+0000 7fe3df5ea740 1 client.5835 shutdown 2024-04-04T06:08:34.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.943+0000 7fe3df5ea740 1 -- 172.21.15.159:0/1763743221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e4cc50 msgr2=0x55f015e4b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:34.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.943+0000 7fe3df5ea740 1 --2- 172.21.15.159:0/1763743221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e4cc50 0x55f015e4b640 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0058e10 tx=0x7fe3d0001250 comp rx=0 tx=0).stop 2024-04-04T06:08:34.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.943+0000 7fe3df5ea740 1 -- 172.21.15.159:0/1763743221 shutdown_connections 2024-04-04T06:08:34.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.943+0000 7fe3df5ea740 1 --2- 172.21.15.159:0/1763743221 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55f015e85420 0x55f015e85800 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.943+0000 7fe3df5ea740 1 --2- 172.21.15.159:0/1763743221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e4cc50 0x55f015e4b640 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.943+0000 7fe3df5ea740 1 --2- 172.21.15.159:0/1763743221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f015e53960 0x55f015e4ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.943+0000 7fe3df5ea740 1 --2- 172.21.15.159:0/1763743221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f015e53580 0x55f015e82390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.943+0000 7fe3df5ea740 1 -- 172.21.15.159:0/1763743221 >> 172.21.15.159:0/1763743221 conn(0x55f015e26a60 msgr2=0x55f015e2a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:34.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.943+0000 7fe3baa00640 0 client.5835 ms_handle_reset on v2:172.21.15.159:6834/1867245367 2024-04-04T06:08:34.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.943+0000 7fe3df5ea740 1 -- 172.21.15.159:0/1763743221 shutdown_connections 2024-04-04T06:08:34.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.943+0000 7fe3df5ea740 1 -- 172.21.15.159:0/1763743221 wait complete. 2024-04-04T06:08:34.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.944+0000 7fe3df5ea740 20 client.5835 trim_cache size 0 max 16384 2024-04-04T06:08:34.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.963+0000 7fe3df5ea740 1 Processor -- start 2024-04-04T06:08:34.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.963+0000 7fe3df5ea740 1 -- start start 2024-04-04T06:08:34.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.964+0000 7fe3df5ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e4cc50 0x55f015e54530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:34.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.964+0000 7fe3df5ea740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f015e53580 0x55f015e74b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:34.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.964+0000 7fe3df5ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f015e53960 0x55f015e750a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:34.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.964+0000 7fe3df5ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d8200a80 con 0x55f015e4cc50 2024-04-04T06:08:34.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.964+0000 7fe3df5ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81ffa60 con 0x55f015e53960 2024-04-04T06:08:34.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.964+0000 7fe3df5ea740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d80ca370 con 0x55f015e53580 2024-04-04T06:08:34.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.964+0000 7fe3d5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e4cc50 0x55f015e54530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:34.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.964+0000 7fe3d5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e4cc50 0x55f015e54530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47166/0 (socket says 172.21.15.159:47166) 2024-04-04T06:08:34.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.964+0000 7fe3d5e00640 1 -- 172.21.15.159:0/465912634 learned_addr learned my addr 172.21.15.159:0/465912634 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:34.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.964+0000 7fe3d5e00640 1 -- 172.21.15.159:0/465912634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f015e53580 msgr2=0x55f015e74b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:34.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.964+0000 7fe3d5e00640 1 --2- 172.21.15.159:0/465912634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f015e53580 0x55f015e74b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.964+0000 7fe3d5e00640 1 -- 172.21.15.159:0/465912634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f015e53960 msgr2=0x55f015e750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:34.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.964+0000 7fe3d5e00640 1 --2- 172.21.15.159:0/465912634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f015e53960 0x55f015e750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.964+0000 7fe3d5e00640 1 -- 172.21.15.159:0/465912634 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f015e28030 con 0x55f015e4cc50 2024-04-04T06:08:34.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.965+0000 7fe3d5e00640 1 --2- 172.21.15.159:0/465912634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e4cc50 0x55f015e54530 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800b360 tx=0x7fe3c8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae2c442effbb3a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:34.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.965+0000 7fe3c5c00640 1 -- 172.21.15.159:0/465912634 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80149f0 con 0x55f015e4cc50 2024-04-04T06:08:34.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.965+0000 7fe3c5c00640 1 -- 172.21.15.159:0/465912634 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c8014b90 con 0x55f015e4cc50 2024-04-04T06:08:34.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.965+0000 7fe3c5c00640 1 -- 172.21.15.159:0/465912634 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80281d0 con 0x55f015e4cc50 2024-04-04T06:08:34.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.966+0000 7fe3df5ea740 1 -- 172.21.15.159:0/465912634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e4cc50 msgr2=0x55f015e54530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:34.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.966+0000 7fe3df5ea740 1 --2- 172.21.15.159:0/465912634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e4cc50 0x55f015e54530 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800b360 tx=0x7fe3c8015000 comp rx=0 tx=0).stop 2024-04-04T06:08:34.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.966+0000 7fe3df5ea740 1 -- 172.21.15.159:0/465912634 shutdown_connections 2024-04-04T06:08:34.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.966+0000 7fe3df5ea740 1 --2- 172.21.15.159:0/465912634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f015e53960 0x55f015e750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.966+0000 7fe3df5ea740 1 --2- 172.21.15.159:0/465912634 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f015e53580 0x55f015e74b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.966+0000 7fe3df5ea740 1 --2- 172.21.15.159:0/465912634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e4cc50 0x55f015e54530 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.966+0000 7fe3df5ea740 1 -- 172.21.15.159:0/465912634 >> 172.21.15.159:0/465912634 conn(0x55f015e26a60 msgr2=0x55f015e4ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:34.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.966+0000 7fe3df5ea740 1 -- 172.21.15.159:0/465912634 shutdown_connections 2024-04-04T06:08:34.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.966+0000 7fe3df5ea740 1 -- 172.21.15.159:0/465912634 wait complete. 2024-04-04T06:08:34.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.966+0000 7fe3df5ea740 1 -- start start 2024-04-04T06:08:34.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.967+0000 7fe3df5ea740 1 Processor -- start 2024-04-04T06:08:34.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.968+0000 7fe3df5ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e4cc50 0x55f015ef2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:34.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.968+0000 7fe3df5ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f015e53580 0x55f015e518f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:34.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.968+0000 7fe3df5ea740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f015e53960 0x55f015e51e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:34.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.968+0000 7fe3df5ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d80ca9c0 con 0x55f015e4cc50 2024-04-04T06:08:34.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.968+0000 7fe3df5ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f015e25c40 con 0x55f015e53580 2024-04-04T06:08:34.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.968+0000 7fe3df5ea740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55f015ef3c60 con 0x55f015e53960 2024-04-04T06:08:34.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.968+0000 7fe3df5ea740 10 client.0 _ll_register_callbacks cb 0x7fe3d0058870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:08:34.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.968+0000 7fe3d5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e4cc50 0x55f015ef2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:34.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.968+0000 7fe3d5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e4cc50 0x55f015ef2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47184/0 (socket says 172.21.15.159:47184) 2024-04-04T06:08:34.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.968+0000 7fe3d5e00640 1 -- 172.21.15.159:0/2946891685 learned_addr learned my addr 172.21.15.159:0/2946891685 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:34.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.968+0000 7fe3cfe00640 1 --2- 172.21.15.159:0/2946891685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f015e53580 0x55f015e518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:34.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.968+0000 7fe3d6800640 1 --2- 172.21.15.159:0/2946891685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f015e53960 0x55f015e51e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:34.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.968+0000 7fe3d5e00640 1 -- 172.21.15.159:0/2946891685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f015e53960 msgr2=0x55f015e51e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:34.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.968+0000 7fe3d5e00640 1 --2- 172.21.15.159:0/2946891685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f015e53960 0x55f015e51e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.968+0000 7fe3d5e00640 1 -- 172.21.15.159:0/2946891685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f015e53580 msgr2=0x55f015e518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:34.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.968+0000 7fe3d5e00640 1 --2- 172.21.15.159:0/2946891685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f015e53580 0x55f015e518f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.968+0000 7fe3d5e00640 1 -- 172.21.15.159:0/2946891685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f015e7e3f0 con 0x55f015e4cc50 2024-04-04T06:08:34.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.968+0000 7fe3cfe00640 1 --2- 172.21.15.159:0/2946891685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f015e53580 0x55f015e518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:34.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.969+0000 7fe3d5e00640 1 --2- 172.21.15.159:0/2946891685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e4cc50 0x55f015ef2ff0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8014dc0 tx=0x7fe3c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4723fa11fb0ea7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:34.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.969+0000 7fe3baa00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:08:34.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.969+0000 7fe3baa00640 1 -- 172.21.15.159:0/2946891685 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80285e0 con 0x55f015e4cc50 2024-04-04T06:08:34.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.969+0000 7fe3baa00640 1 -- 172.21.15.159:0/2946891685 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fe3c8028780 con 0x55f015e4cc50 2024-04-04T06:08:34.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.969+0000 7fe3df5ea740 1 -- 172.21.15.159:0/2946891685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c8014b90 con 0x55f015e4cc50 2024-04-04T06:08:34.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.969+0000 7fe3baa00640 1 -- 172.21.15.159:0/2946891685 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe3c8039050 con 0x55f015e4cc50 2024-04-04T06:08:34.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.969+0000 7fe3df5ea740 10 client.5841 fetch_fsmap learned FSMap version 25 2024-04-04T06:08:34.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.969+0000 7fe3df5ea740 1 -- 172.21.15.159:0/2946891685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) -- 0x55f015ef3a60 con 0x55f015e4cc50 2024-04-04T06:08:34.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.970+0000 7fe3baa00640 1 -- 172.21.15.159:0/2946891685 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c8014b90 con 0x55f015e4cc50 2024-04-04T06:08:34.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.970+0000 7fe3baa00640 1 -- 172.21.15.159:0/2946891685 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 25) ==== 3494+0+0 (secure 0 0 0) 0x7fe3c803c020 con 0x55f015e4cc50 2024-04-04T06:08:34.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.970+0000 7fe3df5ea740 10 client.5841 fetch_fsmap finished waiting for FSMap version 25 2024-04-04T06:08:34.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.970+0000 7fe3df5ea740 10 client.5841 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-04T06:08:34.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.970+0000 7fe3df5ea740 20 client.5841 populate_metadata read hostname 'smithi159' 2024-04-04T06:08:34.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.970+0000 7fe3df5ea740 1 --2- 172.21.15.159:0/2946891685 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55f015e74f20 0x55f015e4ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:34.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.970+0000 7fe3df5ea740 4 client.5841 mds_command: new command op to 5589 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-04T06:08:34.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.970+0000 7fe3df5ea740 1 -- 172.21.15.159:0/2946891685 --> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f015d98f60 con 0x55f015e74f20 2024-04-04T06:08:34.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.970+0000 7fe3cfe00640 1 --2- 172.21.15.159:0/2946891685 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55f015e74f20 0x55f015e4ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:34.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.971+0000 7fe3cfe00640 1 --2- 172.21.15.159:0/2946891685 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55f015e74f20 0x55f015e4ae40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3db8bdcd6087c883 server_cookie=7c37ab4ee4c2b97f in_seq=0 out_seq=0 2024-04-04T06:08:34.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.971+0000 7fe3baa00640 10 client.5841 ms_handle_connect on v2:172.21.15.159:6834/1867245367 2024-04-04T06:08:34.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.971+0000 7fe3baa00640 1 -- 172.21.15.159:0/2946891685 <== mds.0 v2:172.21.15.159:6834/1867245367 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f015d98f60 con 0x55f015e74f20 2024-04-04T06:08:34.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.971+0000 7fe3baa00640 10 client.5841 handle_command_reply: tid=0 2024-04-04T06:08:34.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.971+0000 7fe3df5ea740 1 client.5841 shutdown 2024-04-04T06:08:34.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.971+0000 7fe3df5ea740 1 -- 172.21.15.159:0/2946891685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e4cc50 msgr2=0x55f015ef2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:34.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.971+0000 7fe3df5ea740 1 --2- 172.21.15.159:0/2946891685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e4cc50 0x55f015ef2ff0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8014dc0 tx=0x7fe3c8004730 comp rx=0 tx=0).stop 2024-04-04T06:08:34.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.971+0000 7fe3df5ea740 1 -- 172.21.15.159:0/2946891685 shutdown_connections 2024-04-04T06:08:34.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.971+0000 7fe3df5ea740 1 --2- 172.21.15.159:0/2946891685 >> [v2:172.21.15.159:6834/1867245367,v1:172.21.15.159:6835/1867245367] conn(0x55f015e74f20 0x55f015e4ae40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.971+0000 7fe3df5ea740 1 --2- 172.21.15.159:0/2946891685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f015e53960 0x55f015e51e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.971+0000 7fe3df5ea740 1 --2- 172.21.15.159:0/2946891685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f015e53580 0x55f015e518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.971+0000 7fe3df5ea740 1 --2- 172.21.15.159:0/2946891685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f015e4cc50 0x55f015ef2ff0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.971+0000 7fe3df5ea740 1 -- 172.21.15.159:0/2946891685 >> 172.21.15.159:0/2946891685 conn(0x55f015e26a60 msgr2=0x55f015e4ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:34.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.971+0000 7fe3baa00640 0 client.5841 ms_handle_reset on v2:172.21.15.159:6834/1867245367 2024-04-04T06:08:34.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.971+0000 7fe3df5ea740 1 -- 172.21.15.159:0/2946891685 shutdown_connections 2024-04-04T06:08:34.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.971+0000 7fe3df5ea740 1 -- 172.21.15.159:0/2946891685 wait complete. 2024-04-04T06:08:34.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.971+0000 7fe3df5ea740 20 client.5841 trim_cache size 0 max 16384 2024-04-04T06:08:34.969 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:08:34.969 INFO:teuthology.orchestra.run.smithi159.stdout: "status": "no active scrubs running", 2024-04-04T06:08:34.969 INFO:teuthology.orchestra.run.smithi159.stdout: "scrubs": {} 2024-04-04T06:08:34.970 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:08:34.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.972+0000 7fe3dca00640 1 -- 172.21.15.159:0/3187740425 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe3b406c8f0 msgr2=0x7fe3b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:34.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.972+0000 7fe3dca00640 1 --2- 172.21.15.159:0/3187740425 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe3b406c8f0 0x7fe3b406edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8006fd0 tx=0x7fe3c8008040 comp rx=0 tx=0).stop 2024-04-04T06:08:34.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.972+0000 7fe3dca00640 1 -- 172.21.15.159:0/3187740425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3d8153580 msgr2=0x7fe3d814a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:34.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.972+0000 7fe3dca00640 1 --2- 172.21.15.159:0/3187740425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3d8153580 0x7fe3d814a310 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000c700 tx=0x7fe3d000cbc0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.972+0000 7fe3dca00640 1 -- 172.21.15.159:0/3187740425 shutdown_connections 2024-04-04T06:08:34.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.972+0000 7fe3dca00640 1 --2- 172.21.15.159:0/3187740425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3d8153580 0x7fe3d814a310 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.972+0000 7fe3dca00640 1 --2- 172.21.15.159:0/3187740425 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3d80d07c0 0x7fe3d8071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.972+0000 7fe3dca00640 1 --2- 172.21.15.159:0/3187740425 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe3b406c8f0 0x7fe3b406edb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.972+0000 7fe3dca00640 1 --2- 172.21.15.159:0/3187740425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3d80af2e0 0x7fe3d80cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:34.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.972+0000 7fe3dca00640 1 -- 172.21.15.159:0/3187740425 >> 172.21.15.159:0/3187740425 conn(0x7fe3d806b750 msgr2=0x7fe3d806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:34.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.972+0000 7fe3dca00640 1 -- 172.21.15.159:0/3187740425 shutdown_connections 2024-04-04T06:08:34.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:34.972+0000 7fe3dca00640 1 -- 172.21.15.159:0/3187740425 wait complete. 2024-04-04T06:08:34.984 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-04T06:08:35.085 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:08:35.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.297+0000 7fb72ec00640 1 Processor -- start 2024-04-04T06:08:35.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.297+0000 7fb72ec00640 1 -- start start 2024-04-04T06:08:35.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.299+0000 7fb72ec00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb7280d04d0 0x7fb7280b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:35.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.299+0000 7fb72ec00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7280b3150 con 0x7fb7280cc0b0 2024-04-04T06:08:35.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.299+0000 7fb72ec00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7281600e0 con 0x7fb72815fc50 2024-04-04T06:08:35.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.299+0000 7fb72ec00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fb72815c610 con 0x7fb7280d04d0 2024-04-04T06:08:35.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.299+0000 7fb72d800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fb7280cc0b0 0x7fb72806dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:32774/0 (socket says 172.21.15.159:32774) 2024-04-04T06:08:35.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.299+0000 7fb72d800640 1 -- 172.21.15.159:0/2026461592 learned_addr learned my addr 172.21.15.159:0/2026461592 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:35.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.299+0000 7fb72e200640 1 --2- 172.21.15.159:0/2026461592 >> v2:172.21.15.159:3300/0 conn(0x7fb7280d04d0 0x7fb7280b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:35.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.299+0000 7fb727e00640 1 -- 172.21.15.159:0/2026461592 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1767666054 0 0) 0x7fb7280b3150 con 0x7fb7280cc0b0 2024-04-04T06:08:35.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.300+0000 7fb727e00640 1 -- 172.21.15.159:0/2026461592 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb710003520 con 0x7fb7280cc0b0 2024-04-04T06:08:35.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.300+0000 7fb72e200640 1 -- 172.21.15.159:0/2026461592 >> v1:172.21.15.178:6789/0 conn(0x7fb72815fc50 legacy=0x7fb7280cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:35.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.300+0000 7fb72e200640 1 -- 172.21.15.159:0/2026461592 >> v1:172.21.15.159:6789/0 conn(0x7fb7280cc0b0 legacy=0x7fb72806dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:35.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.300+0000 7fb72e200640 1 -- 172.21.15.159:0/2026461592 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7281603c0 con 0x7fb7280d04d0 2024-04-04T06:08:35.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.301+0000 7fb72e200640 1 --2- 172.21.15.159:0/2026461592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7280d04d0 0x7fb7280b5b40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb7200097a0 tx=0x7fb72002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae523955771e1337 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:35.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.301+0000 7fb727e00640 1 -- 172.21.15.159:0/2026461592 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7200315a0 con 0x7fb7280d04d0 2024-04-04T06:08:35.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.301+0000 7fb727e00640 1 -- 172.21.15.159:0/2026461592 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb720037070 con 0x7fb7280d04d0 2024-04-04T06:08:35.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.301+0000 7fb727e00640 1 -- 172.21.15.159:0/2026461592 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720031940 con 0x7fb7280d04d0 2024-04-04T06:08:35.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.302+0000 7fb72ec00640 1 -- 172.21.15.159:0/2026461592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7280d04d0 msgr2=0x7fb7280b5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:35.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.302+0000 7fb72ec00640 1 --2- 172.21.15.159:0/2026461592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7280d04d0 0x7fb7280b5b40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb7200097a0 tx=0x7fb72002b840 comp rx=0 tx=0).stop 2024-04-04T06:08:35.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.302+0000 7fb72ec00640 1 -- 172.21.15.159:0/2026461592 shutdown_connections 2024-04-04T06:08:35.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.302+0000 7fb72ec00640 1 --2- 172.21.15.159:0/2026461592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7280d04d0 0x7fb7280b5b40 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:35.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.302+0000 7fb72ec00640 1 -- 172.21.15.159:0/2026461592 >> 172.21.15.159:0/2026461592 conn(0x7fb7280ccad0 msgr2=0x7fb7280cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:35.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.302+0000 7fb72ec00640 1 -- 172.21.15.159:0/2026461592 shutdown_connections 2024-04-04T06:08:35.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.302+0000 7fb72ec00640 1 -- 172.21.15.159:0/2026461592 wait complete. 2024-04-04T06:08:35.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.303+0000 7fb72ec00640 1 Processor -- start 2024-04-04T06:08:35.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.304+0000 7fb72ec00640 1 -- start start 2024-04-04T06:08:35.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.304+0000 7fb72ec00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7280cc0b0 0x7fb728137860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:35.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.305+0000 7fb72ec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7280d04d0 0x7fb7280b1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:35.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.305+0000 7fb72ec00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb72815e650 0x7fb7280b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:35.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.305+0000 7fb72ec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb7280cc490 con 0x7fb7280d04d0 2024-04-04T06:08:35.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.305+0000 7fb72ec00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb728160980 con 0x7fb7280cc0b0 2024-04-04T06:08:35.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.305+0000 7fb72ec00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb7280cc610 con 0x7fb72815e650 2024-04-04T06:08:35.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.305+0000 7fb72ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7280d04d0 0x7fb7280b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:35.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.305+0000 7fb72ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7280d04d0 0x7fb7280b1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47204/0 (socket says 172.21.15.159:47204) 2024-04-04T06:08:35.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.305+0000 7fb72ce00640 1 -- 172.21.15.159:0/3399395216 learned_addr learned my addr 172.21.15.159:0/3399395216 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:35.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.305+0000 7fb72d800640 1 --2- 172.21.15.159:0/3399395216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7280cc0b0 0x7fb728137860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:35.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.305+0000 7fb72e200640 1 --2- 172.21.15.159:0/3399395216 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb72815e650 0x7fb7280b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:35.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.305+0000 7fb72ce00640 1 -- 172.21.15.159:0/3399395216 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb72815e650 msgr2=0x7fb7280b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:35.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.305+0000 7fb72ce00640 1 --2- 172.21.15.159:0/3399395216 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb72815e650 0x7fb7280b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:35.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.305+0000 7fb72ce00640 1 -- 172.21.15.159:0/3399395216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7280cc0b0 msgr2=0x7fb728137860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:35.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.305+0000 7fb72ce00640 1 --2- 172.21.15.159:0/3399395216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7280cc0b0 0x7fb728137860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:35.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.305+0000 7fb72ce00640 1 -- 172.21.15.159:0/3399395216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7281561f0 con 0x7fb7280d04d0 2024-04-04T06:08:35.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.306+0000 7fb72ce00640 1 --2- 172.21.15.159:0/3399395216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7280d04d0 0x7fb7280b1340 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb71400b3f0 tx=0x7fb71400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:35.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.306+0000 7fb725600640 1 -- 172.21.15.159:0/3399395216 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714015020 con 0x7fb7280d04d0 2024-04-04T06:08:35.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.306+0000 7fb725600640 1 -- 172.21.15.159:0/3399395216 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb714011830 con 0x7fb7280d04d0 2024-04-04T06:08:35.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.306+0000 7fb72ec00640 1 -- 172.21.15.159:0/3399395216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb728155430 con 0x7fb7280d04d0 2024-04-04T06:08:35.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.306+0000 7fb725600640 1 -- 172.21.15.159:0/3399395216 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714011c20 con 0x7fb7280d04d0 2024-04-04T06:08:35.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.306+0000 7fb72ec00640 1 -- 172.21.15.159:0/3399395216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7281559f0 con 0x7fb7280d04d0 2024-04-04T06:08:35.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.307+0000 7fb72ec00640 1 -- 172.21.15.159:0/3399395216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb728075940 con 0x7fb7280d04d0 2024-04-04T06:08:35.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.308+0000 7fb725600640 1 -- 172.21.15.159:0/3399395216 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb714018020 con 0x7fb7280d04d0 2024-04-04T06:08:35.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.309+0000 7fb725600640 1 --2- 172.21.15.159:0/3399395216 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb70806c8f0 0x7fb70806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:35.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.310+0000 7fb725600640 1 -- 172.21.15.159:0/3399395216 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fb71408b760 con 0x7fb7280d04d0 2024-04-04T06:08:35.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.310+0000 7fb72d800640 1 --2- 172.21.15.159:0/3399395216 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb70806c8f0 0x7fb70806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:35.305 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.311+0000 7fb72d800640 1 --2- 172.21.15.159:0/3399395216 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb70806c8f0 0x7fb70806edb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb71c005c00 tx=0x7fb71c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:35.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.315+0000 7fb725600640 1 -- 172.21.15.159:0/3399395216 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb71401f030 con 0x7fb7280d04d0 2024-04-04T06:08:35.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.552+0000 7fb72ec00640 1 -- 172.21.15.159:0/3399395216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb728068e40 con 0x7fb7280d04d0 2024-04-04T06:08:35.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.553+0000 7fb725600640 1 -- 172.21.15.159:0/3399395216 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5193 (secure 0 0 0) 0x7fb7140172c0 con 0x7fb7280d04d0 2024-04-04T06:08:35.547 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:35.548 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3765578393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3765578393},{"type":"v1","addr":"172.21.15.178:6835","nonce":3765578393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3260412913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3260412913},{"type":"v1","addr":"172.21.15.178:6837","nonce":3260412913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3149916718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3149916718},{"type":"v1","addr":"172.21.15.159:6833","nonce":3149916718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:07:53.523212+0000","modified":"2024-04-04T06:07:56.601750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6835/1867245367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1867245367},{"type":"v1","addr":"172.21.15.159:6835","nonce":1867245367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5589,"qdb_cluster":[5589]},"id":3}]} 2024-04-04T06:08:35.548 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 25 2024-04-04T06:08:35.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.557+0000 7fb72ec00640 1 -- 172.21.15.159:0/3399395216 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb70806c8f0 msgr2=0x7fb70806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:35.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.557+0000 7fb72ec00640 1 --2- 172.21.15.159:0/3399395216 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb70806c8f0 0x7fb70806edb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb71c005c00 tx=0x7fb71c009040 comp rx=0 tx=0).stop 2024-04-04T06:08:35.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.557+0000 7fb72ec00640 1 -- 172.21.15.159:0/3399395216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7280d04d0 msgr2=0x7fb7280b1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:35.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.557+0000 7fb72ec00640 1 --2- 172.21.15.159:0/3399395216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7280d04d0 0x7fb7280b1340 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb71400b3f0 tx=0x7fb71400b8b0 comp rx=0 tx=0).stop 2024-04-04T06:08:35.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.557+0000 7fb72ec00640 1 -- 172.21.15.159:0/3399395216 shutdown_connections 2024-04-04T06:08:35.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.557+0000 7fb72ec00640 1 --2- 172.21.15.159:0/3399395216 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb72815e650 0x7fb7280b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:35.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.557+0000 7fb72ec00640 1 --2- 172.21.15.159:0/3399395216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7280d04d0 0x7fb7280b1340 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:35.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.557+0000 7fb72ec00640 1 --2- 172.21.15.159:0/3399395216 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb70806c8f0 0x7fb70806edb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:35.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.557+0000 7fb72ec00640 1 --2- 172.21.15.159:0/3399395216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7280cc0b0 0x7fb728137860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:35.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.557+0000 7fb72ec00640 1 -- 172.21.15.159:0/3399395216 >> 172.21.15.159:0/3399395216 conn(0x7fb7280ccad0 msgr2=0x7fb728072b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:35.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.557+0000 7fb72ec00640 1 -- 172.21.15.159:0/3399395216 shutdown_connections 2024-04-04T06:08:35.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.557+0000 7fb72ec00640 1 -- 172.21.15.159:0/3399395216 wait complete. 2024-04-04T06:08:35.571 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-04T06:08:35.813 INFO:teuthology.orchestra.run.smithi159.stdout:{"message":"","return_code":0} 2024-04-04T06:08:35.813 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-04T06:08:35.813 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-04T06:08:35.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.909+0000 7fce2de3af40 1 Processor -- start 2024-04-04T06:08:35.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.909+0000 7fce2de3af40 1 -- start start 2024-04-04T06:08:35.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.910+0000 7fce2de3af40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55882389d8b0 0x558823885120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:35.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.910+0000 7fce2de3af40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55882389e290 0x55882392d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:35.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.910+0000 7fce2de3af40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55882392fc10 0x55882392d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:35.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.910+0000 7fce2de3af40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5588239315e0 con 0x55882392fc10 2024-04-04T06:08:35.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.910+0000 7fce2de3af40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5588237ff2c0 con 0x55882389e290 2024-04-04T06:08:35.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.910+0000 7fce2de3af40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x558823931880 con 0x55882389d8b0 2024-04-04T06:08:35.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.911+0000 7fce2b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55882392fc10 0x55882392d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:35.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.911+0000 7fce2b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55882392fc10 0x55882392d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47220/0 (socket says 172.21.15.159:47220) 2024-04-04T06:08:35.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.911+0000 7fce2b200640 1 -- 172.21.15.159:0/1703894243 learned_addr learned my addr 172.21.15.159:0/1703894243 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:35.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.911+0000 7fce23e00640 1 --2- 172.21.15.159:0/1703894243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55882389e290 0x55882392d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:35.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.911+0000 7fce2a800640 1 --2- 172.21.15.159:0/1703894243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55882389d8b0 0x558823885120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:35.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.911+0000 7fce2b200640 1 -- 172.21.15.159:0/1703894243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55882389d8b0 msgr2=0x558823885120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:35.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.911+0000 7fce2b200640 1 --2- 172.21.15.159:0/1703894243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55882389d8b0 0x558823885120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:35.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.911+0000 7fce2b200640 1 -- 172.21.15.159:0/1703894243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55882389e290 msgr2=0x55882392d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:35.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.911+0000 7fce2b200640 1 --2- 172.21.15.159:0/1703894243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55882389e290 0x55882392d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:35.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.911+0000 7fce2b200640 1 -- 172.21.15.159:0/1703894243 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55882392ec80 con 0x55882392fc10 2024-04-04T06:08:35.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.911+0000 7fce23e00640 1 --2- 172.21.15.159:0/1703894243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55882389e290 0x55882392d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:35.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.911+0000 7fce2b200640 1 --2- 172.21.15.159:0/1703894243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55882392fc10 0x55882392d7d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fce24002880 tx=0x7fce24002d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be09852dcdfd56c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:35.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.912+0000 7fce29400640 1 -- 172.21.15.159:0/1703894243 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24016020 con 0x55882392fc10 2024-04-04T06:08:35.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.912+0000 7fce29400640 1 -- 172.21.15.159:0/1703894243 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce2400d9c0 con 0x55882392fc10 2024-04-04T06:08:35.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.912+0000 7fce29400640 1 -- 172.21.15.159:0/1703894243 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2400ddb0 con 0x55882392fc10 2024-04-04T06:08:35.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.912+0000 7fce2de3af40 1 -- 172.21.15.159:0/1703894243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55882392fc10 msgr2=0x55882392d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:35.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.912+0000 7fce2de3af40 1 --2- 172.21.15.159:0/1703894243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55882392fc10 0x55882392d7d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fce24002880 tx=0x7fce24002d40 comp rx=0 tx=0).stop 2024-04-04T06:08:35.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.912+0000 7fce2de3af40 1 -- 172.21.15.159:0/1703894243 shutdown_connections 2024-04-04T06:08:35.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.912+0000 7fce2de3af40 1 --2- 172.21.15.159:0/1703894243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55882392fc10 0x55882392d7d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:35.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.912+0000 7fce2de3af40 1 --2- 172.21.15.159:0/1703894243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55882389e290 0x55882392d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:35.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.912+0000 7fce2de3af40 1 --2- 172.21.15.159:0/1703894243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55882389d8b0 0x558823885120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:35.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.912+0000 7fce2de3af40 1 -- 172.21.15.159:0/1703894243 >> 172.21.15.159:0/1703894243 conn(0x558823883cf0 msgr2=0x5588238834a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:35.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.912+0000 7fce2de3af40 1 -- 172.21.15.159:0/1703894243 shutdown_connections 2024-04-04T06:08:35.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.912+0000 7fce2de3af40 1 -- 172.21.15.159:0/1703894243 wait complete. 2024-04-04T06:08:35.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.914+0000 7fce2de3af40 1 Processor -- start 2024-04-04T06:08:35.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.914+0000 7fce2de3af40 1 -- start start 2024-04-04T06:08:35.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.915+0000 7fce2de3af40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55882389d8b0 0x5588239c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:35.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.915+0000 7fce2de3af40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55882389e290 0x558823924550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:35.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.915+0000 7fce2de3af40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55882392fc10 0x558823923580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:35.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.915+0000 7fce2de3af40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558823933db0 con 0x55882389e290 2024-04-04T06:08:35.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.915+0000 7fce2de3af40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558823933b60 con 0x55882392fc10 2024-04-04T06:08:35.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.915+0000 7fce2de3af40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55882392ef20 con 0x55882389d8b0 2024-04-04T06:08:35.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.916+0000 7fce23e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55882389e290 0x558823924550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:35.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.916+0000 7fce23e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55882389e290 0x558823924550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47234/0 (socket says 172.21.15.159:47234) 2024-04-04T06:08:35.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.916+0000 7fce23e00640 1 -- 172.21.15.159:0/2483819748 learned_addr learned my addr 172.21.15.159:0/2483819748 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:35.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.916+0000 7fce2a800640 1 --2- 172.21.15.159:0/2483819748 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55882389d8b0 0x5588239c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:35.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.916+0000 7fce2b200640 1 --2- 172.21.15.159:0/2483819748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55882392fc10 0x558823923580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:35.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.916+0000 7fce23e00640 1 -- 172.21.15.159:0/2483819748 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55882389d8b0 msgr2=0x5588239c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:35.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.916+0000 7fce23e00640 1 --2- 172.21.15.159:0/2483819748 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55882389d8b0 0x5588239c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:35.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.916+0000 7fce23e00640 1 -- 172.21.15.159:0/2483819748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55882392fc10 msgr2=0x558823923580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:35.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.916+0000 7fce23e00640 1 --2- 172.21.15.159:0/2483819748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55882392fc10 0x558823923580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:35.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.916+0000 7fce23e00640 1 -- 172.21.15.159:0/2483819748 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558823933820 con 0x55882389e290 2024-04-04T06:08:35.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.917+0000 7fce23e00640 1 --2- 172.21.15.159:0/2483819748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55882389e290 0x558823924550 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fce14055040 tx=0x7fce14002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:35.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.917+0000 7fce23400640 1 -- 172.21.15.159:0/2483819748 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce14063070 con 0x55882389e290 2024-04-04T06:08:35.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.917+0000 7fce23400640 1 -- 172.21.15.159:0/2483819748 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce14052a70 con 0x55882389e290 2024-04-04T06:08:35.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.917+0000 7fce2de3af40 1 -- 172.21.15.159:0/2483819748 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5588238a2cd0 con 0x55882389e290 2024-04-04T06:08:35.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.917+0000 7fce23400640 1 -- 172.21.15.159:0/2483819748 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce14057040 con 0x55882389e290 2024-04-04T06:08:35.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.917+0000 7fce2de3af40 1 -- 172.21.15.159:0/2483819748 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558823929f80 con 0x55882389e290 2024-04-04T06:08:35.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.919+0000 7fce23400640 1 -- 172.21.15.159:0/2483819748 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce14066070 con 0x55882389e290 2024-04-04T06:08:35.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.920+0000 7fce23400640 1 --2- 172.21.15.159:0/2483819748 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce0806b320 0x7fce0806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:35.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.920+0000 7fce23400640 1 -- 172.21.15.159:0/2483819748 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fce140cc620 con 0x55882389e290 2024-04-04T06:08:35.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.920+0000 7fce2de3af40 1 --2- 172.21.15.159:0/2483819748 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x558823924a90 0x558823926a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:35.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.920+0000 7fce2de3af40 1 -- 172.21.15.159:0/2483819748 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48) -- 0x5588238a0160 con 0x558823924a90 2024-04-04T06:08:35.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.920+0000 7fce2a800640 1 --2- 172.21.15.159:0/2483819748 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce0806b320 0x7fce0806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:35.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.921+0000 7fce2b200640 1 --2- 172.21.15.159:0/2483819748 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x558823924a90 0x558823926a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:35.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.922+0000 7fce2a800640 1 --2- 172.21.15.159:0/2483819748 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce0806b320 0x7fce0806d7e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0033e0 tx=0x7fce1c003050 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:35.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.922+0000 7fce2b200640 1 --2- 172.21.15.159:0/2483819748 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x558823924a90 0x558823926a20 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:35.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.923+0000 7fce2b200640 1 -- 172.21.15.159:0/2483819748 <== osd.6 v2:172.21.15.178:6826/198949157 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7fce24012470 con 0x558823924a90 2024-04-04T06:08:35.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.923+0000 7fce2de3af40 1 -- 172.21.15.159:0/2483819748 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x558823924a90 msgr2=0x558823926a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:35.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.923+0000 7fce2de3af40 1 --2- 172.21.15.159:0/2483819748 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x558823924a90 0x558823926a20 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:35.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.923+0000 7fce2de3af40 1 -- 172.21.15.159:0/2483819748 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce0806b320 msgr2=0x7fce0806d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:35.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.923+0000 7fce2de3af40 1 --2- 172.21.15.159:0/2483819748 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce0806b320 0x7fce0806d7e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0033e0 tx=0x7fce1c003050 comp rx=0 tx=0).stop 2024-04-04T06:08:35.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.923+0000 7fce2de3af40 1 -- 172.21.15.159:0/2483819748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55882389e290 msgr2=0x558823924550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:35.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.923+0000 7fce2de3af40 1 --2- 172.21.15.159:0/2483819748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55882389e290 0x558823924550 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fce14055040 tx=0x7fce14002fb0 comp rx=0 tx=0).stop 2024-04-04T06:08:35.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.923+0000 7fce2b200640 1 -- 172.21.15.159:0/2483819748 reap_dead start 2024-04-04T06:08:35.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.923+0000 7fce2de3af40 1 -- 172.21.15.159:0/2483819748 shutdown_connections 2024-04-04T06:08:35.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.923+0000 7fce2de3af40 1 -- 172.21.15.159:0/2483819748 >> 172.21.15.159:0/2483819748 conn(0x558823883cf0 msgr2=0x5588238a3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:35.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.923+0000 7fce2de3af40 1 -- 172.21.15.159:0/2483819748 shutdown_connections 2024-04-04T06:08:35.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:35.923+0000 7fce2de3af40 1 -- 172.21.15.159:0/2483819748 wait complete. 2024-04-04T06:08:35.923 INFO:teuthology.orchestra.run.smithi159.stdout:Yalphaparent_a 2024-04-04T06:08:35.923 DEBUG:teuthology.orchestra.run.smithi159:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-04T06:08:36.018 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:08:36.019 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 1099511627777, 2024-04-04T06:08:36.019 INFO:teuthology.orchestra.run.smithi159.stdout: "ancestors": [ 2024-04-04T06:08:36.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:36.019 INFO:teuthology.orchestra.run.smithi159.stdout: "dirino": 1099511627776, 2024-04-04T06:08:36.019 INFO:teuthology.orchestra.run.smithi159.stdout: "dname": "alpha", 2024-04-04T06:08:36.019 INFO:teuthology.orchestra.run.smithi159.stdout: "version": 2 2024-04-04T06:08:36.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:08:36.019 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:36.019 INFO:teuthology.orchestra.run.smithi159.stdout: "dirino": 1, 2024-04-04T06:08:36.019 INFO:teuthology.orchestra.run.smithi159.stdout: "dname": "parent_a", 2024-04-04T06:08:36.019 INFO:teuthology.orchestra.run.smithi159.stdout: "version": 4 2024-04-04T06:08:36.019 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:36.020 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-04-04T06:08:36.020 INFO:teuthology.orchestra.run.smithi159.stdout: "pool": 7, 2024-04-04T06:08:36.020 INFO:teuthology.orchestra.run.smithi159.stdout: "old_pools": [] 2024-04-04T06:08:36.020 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:08:36.020 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:36.026 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-04T06:08:36.026 DEBUG:teuthology.orchestra.run.smithi159:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:08:36.133 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:08:36.254 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-04T06:08:36.254 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:08:36.255 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:08:36.255 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:08:36.307 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:08:36.307 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:08:36.396 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:08:36.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.644+0000 7f2653a00640 1 Processor -- start 2024-04-04T06:08:36.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.644+0000 7f2653a00640 1 -- start start 2024-04-04T06:08:36.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.645+0000 7f2653a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f264c15fa10 0x7f264c0b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:36.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.645+0000 7f2653a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f264c0d0c30 0x7f264c0d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:36.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.645+0000 7f2653a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f264c0d3140 con 0x7f264c0d2bf0 2024-04-04T06:08:36.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.645+0000 7f2653a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f264c0d15e0 con 0x7f264c15fa10 2024-04-04T06:08:36.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.645+0000 7f2653a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f264c0d04a0 con 0x7f264c0d0c30 2024-04-04T06:08:36.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.646+0000 7f2643e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f264c0d2bf0 0x7f264c0cfd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:32776/0 (socket says 172.21.15.159:32776) 2024-04-04T06:08:36.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.646+0000 7f2643e00640 1 -- 172.21.15.159:0/1686484662 learned_addr learned my addr 172.21.15.159:0/1686484662 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:36.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.646+0000 7f2651000640 1 --2- 172.21.15.159:0/1686484662 >> v2:172.21.15.178:3300/0 conn(0x7f264c15fa10 0x7f264c0b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:36.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.646+0000 7f2651a00640 1 --2- 172.21.15.159:0/1686484662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f264c0d0c30 0x7f264c0d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:36.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.646+0000 7f264be00640 1 -- 172.21.15.159:0/1686484662 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4047315721 0 0) 0x7f264c0d3140 con 0x7f264c0d2bf0 2024-04-04T06:08:36.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.647+0000 7f264be00640 1 -- 172.21.15.159:0/1686484662 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2630003520 con 0x7f264c0d2bf0 2024-04-04T06:08:36.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.647+0000 7f264be00640 1 -- 172.21.15.159:0/1686484662 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3351317102 0 0) 0x7f2630003520 con 0x7f264c0d2bf0 2024-04-04T06:08:36.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.647+0000 7f264be00640 1 -- 172.21.15.159:0/1686484662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f264c0d0c30 msgr2=0x7f264c0d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:36.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.647+0000 7f264be00640 1 --2- 172.21.15.159:0/1686484662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f264c0d0c30 0x7f264c0d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:36.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.647+0000 7f264be00640 1 -- 172.21.15.159:0/1686484662 >> v2:172.21.15.178:3300/0 conn(0x7f264c15fa10 msgr2=0x7f264c0b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:36.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.647+0000 7f264be00640 1 --2- 172.21.15.159:0/1686484662 >> v2:172.21.15.178:3300/0 conn(0x7f264c15fa10 0x7f264c0b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:36.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.647+0000 7f264be00640 1 -- 172.21.15.159:0/1686484662 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c0d3420 con 0x7f264c0d2bf0 2024-04-04T06:08:36.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.647+0000 7f264be00640 1 -- 172.21.15.159:0/1686484662 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f26340050b0 con 0x7f264c0d2bf0 2024-04-04T06:08:36.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.647+0000 7f264be00640 1 -- 172.21.15.159:0/1686484662 >> v1:172.21.15.159:6789/0 conn(0x7f264c0d2bf0 legacy=0x7f264c0cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:36.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.647+0000 7f264be00640 1 --2- 172.21.15.159:0/1686484662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f264c0d0c30 0x7f26300039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:36.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.647+0000 7f264be00640 1 --2- 172.21.15.159:0/1686484662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2630003f30 0x7f2630004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:36.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.647+0000 7f264be00640 1 --2- 172.21.15.159:0/1686484662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2630004d40 0x7f26300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:36.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.647+0000 7f264be00640 1 -- 172.21.15.159:0/1686484662 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f264c0d15e0 con 0x7f2630003f30 2024-04-04T06:08:36.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.647+0000 7f264be00640 1 -- 172.21.15.159:0/1686484662 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f264c0d04a0 con 0x7f2630004d40 2024-04-04T06:08:36.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.647+0000 7f264be00640 1 -- 172.21.15.159:0/1686484662 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f263000b9f0 con 0x7f264c0d0c30 2024-04-04T06:08:36.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.647+0000 7f2643e00640 1 --2- 172.21.15.159:0/1686484662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2630003f30 0x7f2630004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:36.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.648+0000 7f2651000640 1 --2- 172.21.15.159:0/1686484662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f264c0d0c30 0x7f26300039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:36.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.648+0000 7f2651a00640 1 --2- 172.21.15.159:0/1686484662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2630004d40 0x7f26300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:36.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.648+0000 7f2643e00640 1 -- 172.21.15.159:0/1686484662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f264c0d0c30 msgr2=0x7f26300039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:36.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.648+0000 7f2643e00640 1 --2- 172.21.15.159:0/1686484662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f264c0d0c30 0x7f26300039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:36.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.648+0000 7f2643e00640 1 -- 172.21.15.159:0/1686484662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2630004d40 msgr2=0x7f26300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:36.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.648+0000 7f2643e00640 1 --2- 172.21.15.159:0/1686484662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2630004d40 0x7f26300051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:36.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.648+0000 7f2643e00640 1 -- 172.21.15.159:0/1686484662 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2630003520 con 0x7f2630003f30 2024-04-04T06:08:36.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.649+0000 7f2643e00640 1 --2- 172.21.15.159:0/1686484662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2630003f30 0x7f2630004800 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f263400c210 tx=0x7f2630005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ff93d8b08cc57 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:36.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.649+0000 7f264be00640 1 -- 172.21.15.159:0/1686484662 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2634007d70 con 0x7f2630003f30 2024-04-04T06:08:36.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.649+0000 7f264be00640 1 -- 172.21.15.159:0/1686484662 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2634030070 con 0x7f2630003f30 2024-04-04T06:08:36.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.649+0000 7f264be00640 1 -- 172.21.15.159:0/1686484662 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263402cd40 con 0x7f2630003f30 2024-04-04T06:08:36.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.650+0000 7f2653a00640 1 -- 172.21.15.159:0/1686484662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2630003f30 msgr2=0x7f2630004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:36.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.650+0000 7f2653a00640 1 --2- 172.21.15.159:0/1686484662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2630003f30 0x7f2630004800 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f263400c210 tx=0x7f2630005830 comp rx=0 tx=0).stop 2024-04-04T06:08:36.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.650+0000 7f2651a00640 1 -- 172.21.15.159:0/1686484662 reap_dead start 2024-04-04T06:08:36.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.650+0000 7f2653a00640 1 -- 172.21.15.159:0/1686484662 shutdown_connections 2024-04-04T06:08:36.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.650+0000 7f2653a00640 1 -- 172.21.15.159:0/1686484662 >> 172.21.15.159:0/1686484662 conn(0x7f264c08bfb0 msgr2=0x7f264c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:36.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.650+0000 7f2653a00640 1 -- 172.21.15.159:0/1686484662 shutdown_connections 2024-04-04T06:08:36.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.650+0000 7f2653a00640 1 -- 172.21.15.159:0/1686484662 wait complete. 2024-04-04T06:08:36.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.652+0000 7f2653a00640 1 Processor -- start 2024-04-04T06:08:36.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.652+0000 7f2653a00640 1 -- start start 2024-04-04T06:08:36.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.653+0000 7f2653a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2630003f30 0x7f264c0cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:36.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.653+0000 7f2653a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f264c157ab0 0x7f264c157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:36.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.653+0000 7f2653a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f264c155690 0x7f264c155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:36.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.653+0000 7f2653a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f264c0cf870 con 0x7f264c155690 2024-04-04T06:08:36.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.653+0000 7f2653a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f264c0cf590 con 0x7f2630003f30 2024-04-04T06:08:36.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.653+0000 7f2653a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f264c0b5ae0 con 0x7f264c157ab0 2024-04-04T06:08:36.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.653+0000 7f2651a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f264c155690 0x7f264c155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:36.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.653+0000 7f2651a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f264c155690 0x7f264c155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47260/0 (socket says 172.21.15.159:47260) 2024-04-04T06:08:36.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.653+0000 7f2651a00640 1 -- 172.21.15.159:0/205800170 learned_addr learned my addr 172.21.15.159:0/205800170 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:36.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.653+0000 7f2651000640 1 --2- 172.21.15.159:0/205800170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2630003f30 0x7f264c0cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:36.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.653+0000 7f2643e00640 1 --2- 172.21.15.159:0/205800170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f264c157ab0 0x7f264c157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:36.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.654+0000 7f2651a00640 1 -- 172.21.15.159:0/205800170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f264c157ab0 msgr2=0x7f264c157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:36.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.654+0000 7f2651a00640 1 --2- 172.21.15.159:0/205800170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f264c157ab0 0x7f264c157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:36.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.654+0000 7f2651a00640 1 -- 172.21.15.159:0/205800170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2630003f30 msgr2=0x7f264c0cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:36.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.654+0000 7f2651a00640 1 --2- 172.21.15.159:0/205800170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2630003f30 0x7f264c0cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:36.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.654+0000 7f2651a00640 1 -- 172.21.15.159:0/205800170 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c14b790 con 0x7f264c155690 2024-04-04T06:08:36.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.654+0000 7f2651000640 1 --2- 172.21.15.159:0/205800170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2630003f30 0x7f264c0cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:36.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.654+0000 7f2643e00640 1 --2- 172.21.15.159:0/205800170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f264c157ab0 0x7f264c157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:36.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.654+0000 7f2651a00640 1 --2- 172.21.15.159:0/205800170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f264c155690 0x7f264c155b40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f264c0d4390 tx=0x7f264400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:36.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.654+0000 7f2649600640 1 -- 172.21.15.159:0/205800170 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264401c070 con 0x7f264c155690 2024-04-04T06:08:36.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.654+0000 7f2649600640 1 -- 172.21.15.159:0/205800170 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2644017400 con 0x7f264c155690 2024-04-04T06:08:36.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.654+0000 7f2653a00640 1 -- 172.21.15.159:0/205800170 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264c14ba70 con 0x7f264c155690 2024-04-04T06:08:36.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.654+0000 7f2649600640 1 -- 172.21.15.159:0/205800170 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26440177f0 con 0x7f264c155690 2024-04-04T06:08:36.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.654+0000 7f2653a00640 1 -- 172.21.15.159:0/205800170 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f264c14c030 con 0x7f264c155690 2024-04-04T06:08:36.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.656+0000 7f2653a00640 1 -- 172.21.15.159:0/205800170 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f264c068cd0 con 0x7f264c155690 2024-04-04T06:08:36.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.657+0000 7f2649600640 1 -- 172.21.15.159:0/205800170 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f264401f070 con 0x7f264c155690 2024-04-04T06:08:36.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.657+0000 7f2649600640 1 --2- 172.21.15.159:0/205800170 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f262806c8f0 0x7f262806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:36.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.658+0000 7f2649600640 1 -- 172.21.15.159:0/205800170 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f264408ba50 con 0x7f264c155690 2024-04-04T06:08:36.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.658+0000 7f2651000640 1 --2- 172.21.15.159:0/205800170 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f262806c8f0 0x7f262806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:36.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.659+0000 7f2651000640 1 --2- 172.21.15.159:0/205800170 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f262806c8f0 0x7f262806edb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f263c009a60 tx=0x7f263c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:36.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.662+0000 7f2649600640 1 -- 172.21.15.159:0/205800170 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2644012070 con 0x7f264c155690 2024-04-04T06:08:36.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.866+0000 7f2653a00640 1 -- 172.21.15.159:0/205800170 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f264c0ba3d0 con 0x7f264c155690 2024-04-04T06:08:36.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.868+0000 7f2649600640 1 -- 172.21.15.159:0/205800170 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16777 (secure 0 0 0) 0x7f2644059e40 con 0x7f264c155690 2024-04-04T06:08:36.862 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:36.863 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":48,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:08:34.246895+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:07:51.357366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-04T06:07:52.495884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.30","mappings":[{"from":0,"to":4}]},{"pgid":"7.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:08:36.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.871+0000 7f2653a00640 1 -- 172.21.15.159:0/205800170 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f262806c8f0 msgr2=0x7f262806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:36.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.871+0000 7f2653a00640 1 --2- 172.21.15.159:0/205800170 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f262806c8f0 0x7f262806edb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f263c009a60 tx=0x7f263c008040 comp rx=0 tx=0).stop 2024-04-04T06:08:36.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.871+0000 7f2653a00640 1 -- 172.21.15.159:0/205800170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f264c155690 msgr2=0x7f264c155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:36.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.871+0000 7f2653a00640 1 --2- 172.21.15.159:0/205800170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f264c155690 0x7f264c155b40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f264c0d4390 tx=0x7f264400f4a0 comp rx=0 tx=0).stop 2024-04-04T06:08:36.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.871+0000 7f2653a00640 1 -- 172.21.15.159:0/205800170 shutdown_connections 2024-04-04T06:08:36.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.871+0000 7f2653a00640 1 --2- 172.21.15.159:0/205800170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f264c155690 0x7f264c155b40 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:36.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.871+0000 7f2653a00640 1 --2- 172.21.15.159:0/205800170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f264c157ab0 0x7f264c157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:36.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.871+0000 7f2653a00640 1 --2- 172.21.15.159:0/205800170 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f262806c8f0 0x7f262806edb0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:36.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.871+0000 7f2653a00640 1 --2- 172.21.15.159:0/205800170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2630003f30 0x7f264c0cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:36.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.871+0000 7f2653a00640 1 -- 172.21.15.159:0/205800170 >> 172.21.15.159:0/205800170 conn(0x7f264c08bfb0 msgr2=0x7f264c0d2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:36.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.871+0000 7f2653a00640 1 -- 172.21.15.159:0/205800170 shutdown_connections 2024-04-04T06:08:36.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:36.871+0000 7f2653a00640 1 -- 172.21.15.159:0/205800170 wait complete. 2024-04-04T06:08:36.884 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:07:51.357366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:07:52.495884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:08:36.885 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:08:37.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.087+0000 7f359c200640 1 Processor -- start 2024-04-04T06:08:37.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.088+0000 7f359c200640 1 -- start start 2024-04-04T06:08:37.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.089+0000 7f359c200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3594153580 0x7f35940d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:37.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.089+0000 7f359c200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f35940d07c0 0x7f35940aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:37.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.089+0000 7f359c200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35940af2e0 0x7f359406d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:37.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.089+0000 7f359c200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f35940d0c30 con 0x7f3594153580 2024-04-04T06:08:37.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.089+0000 7f359c200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f35940afab0 con 0x7f35940d07c0 2024-04-04T06:08:37.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.089+0000 7f359c200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f359415cc80 con 0x7f35940af2e0 2024-04-04T06:08:37.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.089+0000 7f3599800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3594153580 0x7f35940d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:37.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.089+0000 7f3599800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3594153580 0x7f35940d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47272/0 (socket says 172.21.15.159:47272) 2024-04-04T06:08:37.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.089+0000 7f3599800640 1 -- 172.21.15.159:0/562187279 learned_addr learned my addr 172.21.15.159:0/562187279 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:37.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.090+0000 7f359a200640 1 --2- 172.21.15.159:0/562187279 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35940af2e0 0x7f359406d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:37.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.090+0000 7f3598e00640 1 --2- 172.21.15.159:0/562187279 >> v2:172.21.15.178:3300/0 conn(0x7f35940d07c0 0x7f35940aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:37.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.090+0000 7f3599800640 1 -- 172.21.15.159:0/562187279 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35940af2e0 msgr2=0x7f359406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:37.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.090+0000 7f3599800640 1 --2- 172.21.15.159:0/562187279 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35940af2e0 0x7f359406d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.090+0000 7f3599800640 1 -- 172.21.15.159:0/562187279 >> v2:172.21.15.178:3300/0 conn(0x7f35940d07c0 msgr2=0x7f35940aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:37.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.090+0000 7f3599800640 1 --2- 172.21.15.159:0/562187279 >> v2:172.21.15.178:3300/0 conn(0x7f35940d07c0 0x7f35940aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.090+0000 7f3599800640 1 -- 172.21.15.159:0/562187279 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359415da00 con 0x7f3594153580 2024-04-04T06:08:37.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.091+0000 7f3599800640 1 --2- 172.21.15.159:0/562187279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3594153580 0x7f35940d0280 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f35940cff70 tx=0x7f358802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133b9f8511dec92 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:37.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.091+0000 7f3593e00640 1 -- 172.21.15.159:0/562187279 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358802e070 con 0x7f3594153580 2024-04-04T06:08:37.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.091+0000 7f3593e00640 1 -- 172.21.15.159:0/562187279 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3588037070 con 0x7f3594153580 2024-04-04T06:08:37.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.091+0000 7f3593e00640 1 -- 172.21.15.159:0/562187279 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588031a00 con 0x7f3594153580 2024-04-04T06:08:37.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.092+0000 7f359c200640 1 -- 172.21.15.159:0/562187279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3594153580 msgr2=0x7f35940d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:37.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.092+0000 7f359c200640 1 --2- 172.21.15.159:0/562187279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3594153580 0x7f35940d0280 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f35940cff70 tx=0x7f358802b880 comp rx=0 tx=0).stop 2024-04-04T06:08:37.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.092+0000 7f359c200640 1 -- 172.21.15.159:0/562187279 shutdown_connections 2024-04-04T06:08:37.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.092+0000 7f359c200640 1 --2- 172.21.15.159:0/562187279 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35940af2e0 0x7f359406d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.092+0000 7f359c200640 1 --2- 172.21.15.159:0/562187279 >> v2:172.21.15.178:3300/0 conn(0x7f35940d07c0 0x7f35940aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.092+0000 7f359c200640 1 --2- 172.21.15.159:0/562187279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3594153580 0x7f35940d0280 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.092+0000 7f359c200640 1 -- 172.21.15.159:0/562187279 >> 172.21.15.159:0/562187279 conn(0x7f359406b750 msgr2=0x7f359406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:37.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.092+0000 7f359c200640 1 -- 172.21.15.159:0/562187279 shutdown_connections 2024-04-04T06:08:37.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.092+0000 7f359c200640 1 -- 172.21.15.159:0/562187279 wait complete. 2024-04-04T06:08:37.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.094+0000 7f359c200640 1 Processor -- start 2024-04-04T06:08:37.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.094+0000 7f359c200640 1 -- start start 2024-04-04T06:08:37.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.095+0000 7f359c200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35940af2e0 0x7f35941522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:37.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.095+0000 7f359c200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35940d07c0 0x7f3594155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:37.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.095+0000 7f359c200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3594153580 0x7f3594159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:37.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.095+0000 7f359c200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f359415d4e0 con 0x7f35940af2e0 2024-04-04T06:08:37.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.095+0000 7f359c200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f359415d200 con 0x7f3594153580 2024-04-04T06:08:37.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.095+0000 7f359c200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3594153960 con 0x7f35940d07c0 2024-04-04T06:08:37.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.095+0000 7f3599800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35940af2e0 0x7f35941522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:37.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.095+0000 7f3599800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35940af2e0 0x7f35941522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47284/0 (socket says 172.21.15.159:47284) 2024-04-04T06:08:37.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.096+0000 7f3599800640 1 -- 172.21.15.159:0/3326692917 learned_addr learned my addr 172.21.15.159:0/3326692917 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:37.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.096+0000 7f3598e00640 1 --2- 172.21.15.159:0/3326692917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35940d07c0 0x7f3594155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:37.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.096+0000 7f359a200640 1 --2- 172.21.15.159:0/3326692917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3594153580 0x7f3594159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:37.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.096+0000 7f3599800640 1 -- 172.21.15.159:0/3326692917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35940d07c0 msgr2=0x7f3594155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:37.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.096+0000 7f3599800640 1 --2- 172.21.15.159:0/3326692917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35940d07c0 0x7f3594155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.096+0000 7f3599800640 1 -- 172.21.15.159:0/3326692917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3594153580 msgr2=0x7f3594159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:37.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.096+0000 7f3599800640 1 --2- 172.21.15.159:0/3326692917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3594153580 0x7f3594159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.096+0000 7f3599800640 1 -- 172.21.15.159:0/3326692917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3594158120 con 0x7f35940af2e0 2024-04-04T06:08:37.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.096+0000 7f3599800640 1 --2- 172.21.15.159:0/3326692917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35940af2e0 0x7f35941522c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3588000c00 tx=0x7f35880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:37.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.097+0000 7f3591600640 1 -- 172.21.15.159:0/3326692917 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588032670 con 0x7f35940af2e0 2024-04-04T06:08:37.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.097+0000 7f3591600640 1 -- 172.21.15.159:0/3326692917 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3588032850 con 0x7f35940af2e0 2024-04-04T06:08:37.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.097+0000 7f3591600640 1 -- 172.21.15.159:0/3326692917 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588032c40 con 0x7f35940af2e0 2024-04-04T06:08:37.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.097+0000 7f359c200640 1 -- 172.21.15.159:0/3326692917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3594158400 con 0x7f35940af2e0 2024-04-04T06:08:37.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.097+0000 7f359c200640 1 -- 172.21.15.159:0/3326692917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35941589c0 con 0x7f35940af2e0 2024-04-04T06:08:37.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.098+0000 7f359c200640 1 -- 172.21.15.159:0/3326692917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35940685c0 con 0x7f35940af2e0 2024-04-04T06:08:37.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.099+0000 7f3591600640 1 -- 172.21.15.159:0/3326692917 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f358804c060 con 0x7f35940af2e0 2024-04-04T06:08:37.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.100+0000 7f3591600640 1 --2- 172.21.15.159:0/3326692917 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f357406c920 0x7f357406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:37.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.100+0000 7f3591600640 1 -- 172.21.15.159:0/3326692917 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f35880ad5c0 con 0x7f35940af2e0 2024-04-04T06:08:37.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.101+0000 7f3598e00640 1 --2- 172.21.15.159:0/3326692917 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f357406c920 0x7f357406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:37.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.102+0000 7f3598e00640 1 --2- 172.21.15.159:0/3326692917 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f357406c920 0x7f357406ede0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3580006fd0 tx=0x7f3580008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:37.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.108+0000 7f3591600640 1 -- 172.21.15.159:0/3326692917 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3588040070 con 0x7f35940af2e0 2024-04-04T06:08:37.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.309+0000 7f359c200640 1 -- 172.21.15.159:0/3326692917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f35940cc9d0 con 0x7f35940af2e0 2024-04-04T06:08:37.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.310+0000 7f3591600640 1 -- 172.21.15.159:0/3326692917 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) ==== 93+0+10 (secure 0 0 0) 0x7f358807b9b0 con 0x7f35940af2e0 2024-04-04T06:08:37.304 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:08:37.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.313+0000 7f359c200640 1 -- 172.21.15.159:0/3326692917 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f357406c920 msgr2=0x7f357406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:37.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.313+0000 7f359c200640 1 --2- 172.21.15.159:0/3326692917 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f357406c920 0x7f357406ede0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3580006fd0 tx=0x7f3580008040 comp rx=0 tx=0).stop 2024-04-04T06:08:37.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.313+0000 7f359c200640 1 -- 172.21.15.159:0/3326692917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35940af2e0 msgr2=0x7f35941522c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:37.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.313+0000 7f359c200640 1 --2- 172.21.15.159:0/3326692917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35940af2e0 0x7f35941522c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3588000c00 tx=0x7f35880046c0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.313+0000 7f359c200640 1 -- 172.21.15.159:0/3326692917 shutdown_connections 2024-04-04T06:08:37.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.313+0000 7f359c200640 1 --2- 172.21.15.159:0/3326692917 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f357406c920 0x7f357406ede0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.313+0000 7f359c200640 1 --2- 172.21.15.159:0/3326692917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3594153580 0x7f3594159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.313+0000 7f359c200640 1 --2- 172.21.15.159:0/3326692917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35940d07c0 0x7f3594155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.313+0000 7f359c200640 1 --2- 172.21.15.159:0/3326692917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35940af2e0 0x7f35941522c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.313+0000 7f359c200640 1 -- 172.21.15.159:0/3326692917 >> 172.21.15.159:0/3326692917 conn(0x7f359406b750 msgr2=0x7f359406d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:37.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.313+0000 7f359c200640 1 -- 172.21.15.159:0/3326692917 shutdown_connections 2024-04-04T06:08:37.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.313+0000 7f359c200640 1 -- 172.21.15.159:0/3326692917 wait complete. 2024-04-04T06:08:37.326 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:08:37.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.534+0000 7fa7c6a00640 1 Processor -- start 2024-04-04T06:08:37.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.535+0000 7fa7c6a00640 1 -- start start 2024-04-04T06:08:37.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.536+0000 7fa7c6a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa7c0153580 0x7fa7c00d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:37.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.536+0000 7fa7c6a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa7c00d07c0 0x7fa7c00aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:37.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.536+0000 7fa7c6a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7c00af2e0 0x7fa7c006d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:37.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.536+0000 7fa7c6a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fa7c00d0c30 con 0x7fa7c00d07c0 2024-04-04T06:08:37.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.536+0000 7fa7c6a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa7c00afab0 con 0x7fa7c0153580 2024-04-04T06:08:37.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.536+0000 7fa7c6a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c015cc80 con 0x7fa7c00af2e0 2024-04-04T06:08:37.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.536+0000 7fa7b7400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa7c00d07c0 0x7fa7c00aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:37.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.536+0000 7fa7b7400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa7c00d07c0 0x7fa7c00aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47290/0 (socket says 172.21.15.159:47290) 2024-04-04T06:08:37.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.536+0000 7fa7b7400640 1 -- 172.21.15.159:0/811320510 learned_addr learned my addr 172.21.15.159:0/811320510 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:37.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.537+0000 7fa7bfe00640 1 --2- 172.21.15.159:0/811320510 >> v2:172.21.15.178:3300/0 conn(0x7fa7c0153580 0x7fa7c00d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:37.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.537+0000 7fa7c4a00640 1 --2- 172.21.15.159:0/811320510 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7c00af2e0 0x7fa7c006d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:37.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.537+0000 7fa7b7400640 1 -- 172.21.15.159:0/811320510 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7c00af2e0 msgr2=0x7fa7c006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:37.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.537+0000 7fa7b7400640 1 --2- 172.21.15.159:0/811320510 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7c00af2e0 0x7fa7c006d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.537+0000 7fa7b7400640 1 -- 172.21.15.159:0/811320510 >> v2:172.21.15.178:3300/0 conn(0x7fa7c0153580 msgr2=0x7fa7c00d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:37.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.537+0000 7fa7b7400640 1 --2- 172.21.15.159:0/811320510 >> v2:172.21.15.178:3300/0 conn(0x7fa7c0153580 0x7fa7c00d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.537+0000 7fa7b7400640 1 -- 172.21.15.159:0/811320510 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c015da00 con 0x7fa7c00d07c0 2024-04-04T06:08:37.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.537+0000 7fa7bfe00640 1 --2- 172.21.15.159:0/811320510 >> v2:172.21.15.178:3300/0 conn(0x7fa7c0153580 0x7fa7c00d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:37.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.537+0000 7fa7c4a00640 1 --2- 172.21.15.159:0/811320510 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7c00af2e0 0x7fa7c006d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:37.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.538+0000 7fa7b7400640 1 --2- 172.21.15.159:0/811320510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7c00d07c0 0x7fa7c00aeda0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00b5270 tx=0x7fa7a802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1add16d166962a44 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:37.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.539+0000 7fa7bf400640 1 -- 172.21.15.159:0/811320510 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a802e070 con 0x7fa7c00d07c0 2024-04-04T06:08:37.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.539+0000 7fa7bf400640 1 -- 172.21.15.159:0/811320510 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a8037070 con 0x7fa7c00d07c0 2024-04-04T06:08:37.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.539+0000 7fa7bf400640 1 -- 172.21.15.159:0/811320510 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8031a00 con 0x7fa7c00d07c0 2024-04-04T06:08:37.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.539+0000 7fa7c6a00640 1 -- 172.21.15.159:0/811320510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7c00d07c0 msgr2=0x7fa7c00aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:37.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.539+0000 7fa7c6a00640 1 --2- 172.21.15.159:0/811320510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7c00d07c0 0x7fa7c00aeda0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00b5270 tx=0x7fa7a802b880 comp rx=0 tx=0).stop 2024-04-04T06:08:37.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.540+0000 7fa7c6a00640 1 -- 172.21.15.159:0/811320510 shutdown_connections 2024-04-04T06:08:37.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.540+0000 7fa7c6a00640 1 --2- 172.21.15.159:0/811320510 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7c00af2e0 0x7fa7c006d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.540+0000 7fa7c6a00640 1 --2- 172.21.15.159:0/811320510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7c00d07c0 0x7fa7c00aeda0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.540+0000 7fa7c6a00640 1 --2- 172.21.15.159:0/811320510 >> v2:172.21.15.178:3300/0 conn(0x7fa7c0153580 0x7fa7c00d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.540+0000 7fa7c6a00640 1 -- 172.21.15.159:0/811320510 >> 172.21.15.159:0/811320510 conn(0x7fa7c006b750 msgr2=0x7fa7c006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:37.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.540+0000 7fa7c6a00640 1 -- 172.21.15.159:0/811320510 shutdown_connections 2024-04-04T06:08:37.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.540+0000 7fa7c6a00640 1 -- 172.21.15.159:0/811320510 wait complete. 2024-04-04T06:08:37.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.542+0000 7fa7c6a00640 1 Processor -- start 2024-04-04T06:08:37.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.542+0000 7fa7c6a00640 1 -- start start 2024-04-04T06:08:37.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.543+0000 7fa7c6a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7c00af2e0 0x7fa7c00b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:37.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.543+0000 7fa7c6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7c00d07c0 0x7fa7c014b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:37.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.543+0000 7fa7c6a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7c0153580 0x7fa7c015b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:37.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.543+0000 7fa7c6a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c015d4e0 con 0x7fa7c00d07c0 2024-04-04T06:08:37.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.543+0000 7fa7c6a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c015d200 con 0x7fa7c00af2e0 2024-04-04T06:08:37.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.543+0000 7fa7c6a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c0153960 con 0x7fa7c0153580 2024-04-04T06:08:37.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.543+0000 7fa7b7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7c00d07c0 0x7fa7c014b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:37.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.543+0000 7fa7b7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7c00d07c0 0x7fa7c014b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47304/0 (socket says 172.21.15.159:47304) 2024-04-04T06:08:37.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.543+0000 7fa7b7400640 1 -- 172.21.15.159:0/1827031920 learned_addr learned my addr 172.21.15.159:0/1827031920 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:37.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.544+0000 7fa7bfe00640 1 --2- 172.21.15.159:0/1827031920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7c00af2e0 0x7fa7c00b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:37.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.544+0000 7fa7c4a00640 1 --2- 172.21.15.159:0/1827031920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7c0153580 0x7fa7c015b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:37.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.544+0000 7fa7b7400640 1 -- 172.21.15.159:0/1827031920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7c0153580 msgr2=0x7fa7c015b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:37.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.544+0000 7fa7b7400640 1 --2- 172.21.15.159:0/1827031920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7c0153580 0x7fa7c015b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.544+0000 7fa7b7400640 1 -- 172.21.15.159:0/1827031920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7c00af2e0 msgr2=0x7fa7c00b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:37.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.544+0000 7fa7b7400640 1 --2- 172.21.15.159:0/1827031920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7c00af2e0 0x7fa7c00b45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.544+0000 7fa7b7400640 1 -- 172.21.15.159:0/1827031920 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c00b4b00 con 0x7fa7c00d07c0 2024-04-04T06:08:37.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.544+0000 7fa7b7400640 1 --2- 172.21.15.159:0/1827031920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7c00d07c0 0x7fa7c014b500 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8000c00 tx=0x7fa7a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:37.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.545+0000 7fa7bcc00640 1 -- 172.21.15.159:0/1827031920 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8032880 con 0x7fa7c00d07c0 2024-04-04T06:08:37.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.545+0000 7fa7bcc00640 1 -- 172.21.15.159:0/1827031920 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a8032a20 con 0x7fa7c00d07c0 2024-04-04T06:08:37.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.545+0000 7fa7c6a00640 1 -- 172.21.15.159:0/1827031920 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c0154090 con 0x7fa7c00d07c0 2024-04-04T06:08:37.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.545+0000 7fa7bcc00640 1 -- 172.21.15.159:0/1827031920 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8032e10 con 0x7fa7c00d07c0 2024-04-04T06:08:37.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.545+0000 7fa7c6a00640 1 -- 172.21.15.159:0/1827031920 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c0154650 con 0x7fa7c00d07c0 2024-04-04T06:08:37.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.546+0000 7fa7c6a00640 1 -- 172.21.15.159:0/1827031920 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c00685c0 con 0x7fa7c00d07c0 2024-04-04T06:08:37.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.547+0000 7fa7bcc00640 1 -- 172.21.15.159:0/1827031920 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7a804c060 con 0x7fa7c00d07c0 2024-04-04T06:08:37.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.548+0000 7fa7bcc00640 1 --2- 172.21.15.159:0/1827031920 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa79c06c8f0 0x7fa79c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:37.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.548+0000 7fa7bcc00640 1 -- 172.21.15.159:0/1827031920 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fa7a80ad880 con 0x7fa7c00d07c0 2024-04-04T06:08:37.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.548+0000 7fa7bfe00640 1 --2- 172.21.15.159:0/1827031920 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa79c06c8f0 0x7fa79c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:37.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.550+0000 7fa7bfe00640 1 --2- 172.21.15.159:0/1827031920 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa79c06c8f0 0x7fa79c06edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00075b0 tx=0x7fa7b0007540 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:37.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.552+0000 7fa7bcc00640 1 -- 172.21.15.159:0/1827031920 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa7a8040070 con 0x7fa7c00d07c0 2024-04-04T06:08:37.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.751+0000 7fa7c6a00640 1 -- 172.21.15.159:0/1827031920 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa7c0072d00 con 0x7fa7c00d07c0 2024-04-04T06:08:37.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.752+0000 7fa7bcc00640 1 -- 172.21.15.159:0/1827031920 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) ==== 104+0+11 (secure 0 0 0) 0x7fa7a807bd00 con 0x7fa7c00d07c0 2024-04-04T06:08:37.746 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:08:37.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.754+0000 7fa7c6a00640 1 -- 172.21.15.159:0/1827031920 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa79c06c8f0 msgr2=0x7fa79c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:37.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.754+0000 7fa7c6a00640 1 --2- 172.21.15.159:0/1827031920 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa79c06c8f0 0x7fa79c06edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00075b0 tx=0x7fa7b0007540 comp rx=0 tx=0).stop 2024-04-04T06:08:37.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.754+0000 7fa7c6a00640 1 -- 172.21.15.159:0/1827031920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7c00d07c0 msgr2=0x7fa7c014b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:37.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.754+0000 7fa7c6a00640 1 --2- 172.21.15.159:0/1827031920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7c00d07c0 0x7fa7c014b500 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8000c00 tx=0x7fa7a80046c0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.755+0000 7fa7c6a00640 1 -- 172.21.15.159:0/1827031920 shutdown_connections 2024-04-04T06:08:37.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.755+0000 7fa7c6a00640 1 --2- 172.21.15.159:0/1827031920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7c0153580 0x7fa7c015b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.755+0000 7fa7c6a00640 1 --2- 172.21.15.159:0/1827031920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7c00d07c0 0x7fa7c014b500 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.755+0000 7fa7c6a00640 1 --2- 172.21.15.159:0/1827031920 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa79c06c8f0 0x7fa79c06edb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.755+0000 7fa7c6a00640 1 --2- 172.21.15.159:0/1827031920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7c00af2e0 0x7fa7c00b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:37.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.755+0000 7fa7c6a00640 1 -- 172.21.15.159:0/1827031920 >> 172.21.15.159:0/1827031920 conn(0x7fa7c006b750 msgr2=0x7fa7c006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:37.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.755+0000 7fa7c6a00640 1 -- 172.21.15.159:0/1827031920 shutdown_connections 2024-04-04T06:08:37.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:37.755+0000 7fa7c6a00640 1 -- 172.21.15.159:0/1827031920 wait complete. 2024-04-04T06:08:37.768 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:08:38.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.010+0000 7fe79f200640 1 Processor -- start 2024-04-04T06:08:38.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.010+0000 7fe79f200640 1 -- start start 2024-04-04T06:08:38.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.013+0000 7fe79f200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe7a00cc880 0x7fe7a00cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:38.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.013+0000 7fe79f200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe7a006d620 0x7fe7a006da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:38.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.013+0000 7fe79f200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7a00ce890 con 0x7fe7a013f190 2024-04-04T06:08:38.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.013+0000 7fe79f200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe7a00cd230 con 0x7fe7a00cc880 2024-04-04T06:08:38.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.013+0000 7fe79f200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe7a00ceb50 con 0x7fe7a006d620 2024-04-04T06:08:38.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.013+0000 7fe79d400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe7a00cc880 0x7fe7a00cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:38.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.013+0000 7fe79d400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe7a00cc880 0x7fe7a00cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47320/0 (socket says 172.21.15.159:47320) 2024-04-04T06:08:38.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.013+0000 7fe79d400640 1 -- 172.21.15.159:0/3125118601 learned_addr learned my addr 172.21.15.159:0/3125118601 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:38.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.013+0000 7fe79e800640 1 --2- 172.21.15.159:0/3125118601 >> v2:172.21.15.178:3300/0 conn(0x7fe7a006d620 0x7fe7a006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:38.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.014+0000 7fe79d400640 1 -- 172.21.15.159:0/3125118601 >> v2:172.21.15.178:3300/0 conn(0x7fe7a006d620 msgr2=0x7fe7a006da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:38.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.014+0000 7fe79d400640 1 --2- 172.21.15.159:0/3125118601 >> v2:172.21.15.178:3300/0 conn(0x7fe7a006d620 0x7fe7a006da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:38.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.014+0000 7fe79d400640 1 -- 172.21.15.159:0/3125118601 >> v1:172.21.15.178:6789/0 conn(0x7fe7a013f190 legacy=0x7fe7a00cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:38.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.014+0000 7fe79d400640 1 -- 172.21.15.159:0/3125118601 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a00cee30 con 0x7fe7a00cc880 2024-04-04T06:08:38.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.015+0000 7fe79d400640 1 --2- 172.21.15.159:0/3125118601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7a00cc880 0x7fe7a00cccf0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00b54a0 tx=0x7fe78802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6669c211ba2f18 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:38.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.016+0000 7fe79ca00640 1 -- 172.21.15.159:0/3125118601 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788035020 con 0x7fe7a00cc880 2024-04-04T06:08:38.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.016+0000 7fe79ca00640 1 -- 172.21.15.159:0/3125118601 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe788031870 con 0x7fe7a00cc880 2024-04-04T06:08:38.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.016+0000 7fe79ca00640 1 -- 172.21.15.159:0/3125118601 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788031c40 con 0x7fe7a00cc880 2024-04-04T06:08:38.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.016+0000 7fe79f200640 1 -- 172.21.15.159:0/3125118601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7a00cc880 msgr2=0x7fe7a00cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:38.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.016+0000 7fe79f200640 1 --2- 172.21.15.159:0/3125118601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7a00cc880 0x7fe7a00cccf0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00b54a0 tx=0x7fe78802b910 comp rx=0 tx=0).stop 2024-04-04T06:08:38.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.016+0000 7fe79f200640 1 -- 172.21.15.159:0/3125118601 shutdown_connections 2024-04-04T06:08:38.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.016+0000 7fe79f200640 1 --2- 172.21.15.159:0/3125118601 >> v2:172.21.15.178:3300/0 conn(0x7fe7a006d620 0x7fe7a006da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:38.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.016+0000 7fe79f200640 1 --2- 172.21.15.159:0/3125118601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7a00cc880 0x7fe7a00cccf0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:38.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.016+0000 7fe79f200640 1 -- 172.21.15.159:0/3125118601 >> 172.21.15.159:0/3125118601 conn(0x7fe7a00b1ba0 msgr2=0x7fe7a00b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:38.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.016+0000 7fe79f200640 1 -- 172.21.15.159:0/3125118601 shutdown_connections 2024-04-04T06:08:38.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.016+0000 7fe79f200640 1 -- 172.21.15.159:0/3125118601 wait complete. 2024-04-04T06:08:38.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.018+0000 7fe79f200640 1 Processor -- start 2024-04-04T06:08:38.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.018+0000 7fe79f200640 1 -- start start 2024-04-04T06:08:38.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.019+0000 7fe79f200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7a006d620 0x7fe7a01f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:38.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.019+0000 7fe79f200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7a00cc880 0x7fe7a0139710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:38.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.019+0000 7fe79f200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7a013f190 0x7fe7a0155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:38.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.019+0000 7fe79f200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a013f570 con 0x7fe7a006d620 2024-04-04T06:08:38.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.019+0000 7fe79f200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a0160c70 con 0x7fe7a013f190 2024-04-04T06:08:38.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.019+0000 7fe79f200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe7a006df90 con 0x7fe7a00cc880 2024-04-04T06:08:38.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.019+0000 7fe79de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7a006d620 0x7fe7a01f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:38.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.019+0000 7fe79de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7a006d620 0x7fe7a01f21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47332/0 (socket says 172.21.15.159:47332) 2024-04-04T06:08:38.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.019+0000 7fe79de00640 1 -- 172.21.15.159:0/1922182222 learned_addr learned my addr 172.21.15.159:0/1922182222 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:38.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.020+0000 7fe79de00640 1 -- 172.21.15.159:0/1922182222 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7a00cc880 msgr2=0x7fe7a0139710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:38.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.020+0000 7fe79de00640 1 --2- 172.21.15.159:0/1922182222 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7a00cc880 0x7fe7a0139710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:38.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.020+0000 7fe79e800640 1 --2- 172.21.15.159:0/1922182222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7a013f190 0x7fe7a0155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:38.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.020+0000 7fe79de00640 1 -- 172.21.15.159:0/1922182222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7a013f190 msgr2=0x7fe7a0155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:38.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.020+0000 7fe79de00640 1 --2- 172.21.15.159:0/1922182222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7a013f190 0x7fe7a0155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:38.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.020+0000 7fe79de00640 1 -- 172.21.15.159:0/1922182222 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a01f28e0 con 0x7fe7a006d620 2024-04-04T06:08:38.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.020+0000 7fe79de00640 1 --2- 172.21.15.159:0/1922182222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7a006d620 0x7fe7a01f21e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe79000c460 tx=0x7fe79000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:38.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.020+0000 7fe796000640 1 -- 172.21.15.159:0/1922182222 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe790016020 con 0x7fe7a006d620 2024-04-04T06:08:38.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.020+0000 7fe796000640 1 -- 172.21.15.159:0/1922182222 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe790012870 con 0x7fe7a006d620 2024-04-04T06:08:38.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.020+0000 7fe79f200640 1 -- 172.21.15.159:0/1922182222 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7a01f2bc0 con 0x7fe7a006d620 2024-04-04T06:08:38.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.021+0000 7fe796000640 1 -- 172.21.15.159:0/1922182222 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe790012c60 con 0x7fe7a006d620 2024-04-04T06:08:38.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.021+0000 7fe79f200640 1 -- 172.21.15.159:0/1922182222 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a01f3180 con 0x7fe7a006d620 2024-04-04T06:08:38.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.022+0000 7fe79f200640 1 -- 172.21.15.159:0/1922182222 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7a0075940 con 0x7fe7a006d620 2024-04-04T06:08:38.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.023+0000 7fe796000640 1 -- 172.21.15.159:0/1922182222 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe790019020 con 0x7fe7a006d620 2024-04-04T06:08:38.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.024+0000 7fe796000640 1 --2- 172.21.15.159:0/1922182222 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe77c06c8f0 0x7fe77c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:38.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.024+0000 7fe796000640 1 -- 172.21.15.159:0/1922182222 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fe79008b900 con 0x7fe7a006d620 2024-04-04T06:08:38.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.024+0000 7fe79d400640 1 --2- 172.21.15.159:0/1922182222 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe77c06c8f0 0x7fe77c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:38.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.025+0000 7fe79d400640 1 --2- 172.21.15.159:0/1922182222 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe77c06c8f0 0x7fe77c06edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe788036000 tx=0x7fe788030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:38.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.029+0000 7fe796000640 1 -- 172.21.15.159:0/1922182222 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe790020030 con 0x7fe7a006d620 2024-04-04T06:08:38.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.225+0000 7fe79f200640 1 -- 172.21.15.159:0/1922182222 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe7a00b2870 con 0x7fe7a006d620 2024-04-04T06:08:38.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.226+0000 7fe796000640 1 -- 172.21.15.159:0/1922182222 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) ==== 100+0+11 (secure 0 0 0) 0x7fe790059c60 con 0x7fe7a006d620 2024-04-04T06:08:38.221 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:08:38.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.228+0000 7fe79f200640 1 -- 172.21.15.159:0/1922182222 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe77c06c8f0 msgr2=0x7fe77c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:38.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.228+0000 7fe79f200640 1 --2- 172.21.15.159:0/1922182222 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe77c06c8f0 0x7fe77c06edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe788036000 tx=0x7fe788030000 comp rx=0 tx=0).stop 2024-04-04T06:08:38.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.228+0000 7fe79f200640 1 -- 172.21.15.159:0/1922182222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7a006d620 msgr2=0x7fe7a01f21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:38.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.228+0000 7fe79f200640 1 --2- 172.21.15.159:0/1922182222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7a006d620 0x7fe7a01f21e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe79000c460 tx=0x7fe79000c920 comp rx=0 tx=0).stop 2024-04-04T06:08:38.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.228+0000 7fe79f200640 1 -- 172.21.15.159:0/1922182222 shutdown_connections 2024-04-04T06:08:38.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.228+0000 7fe79f200640 1 --2- 172.21.15.159:0/1922182222 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe77c06c8f0 0x7fe77c06edb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:38.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.228+0000 7fe79f200640 1 --2- 172.21.15.159:0/1922182222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7a013f190 0x7fe7a0155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:38.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.228+0000 7fe79f200640 1 --2- 172.21.15.159:0/1922182222 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7a00cc880 0x7fe7a0139710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:38.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.228+0000 7fe79f200640 1 --2- 172.21.15.159:0/1922182222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7a006d620 0x7fe7a01f21e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:38.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.228+0000 7fe79f200640 1 -- 172.21.15.159:0/1922182222 >> 172.21.15.159:0/1922182222 conn(0x7fe7a00b1ba0 msgr2=0x7fe7a00afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:38.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.229+0000 7fe79f200640 1 -- 172.21.15.159:0/1922182222 shutdown_connections 2024-04-04T06:08:38.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.229+0000 7fe79f200640 1 -- 172.21.15.159:0/1922182222 wait complete. 2024-04-04T06:08:38.240 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-04T06:08:38.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.444+0000 7ff506800640 1 Processor -- start 2024-04-04T06:08:38.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.444+0000 7ff506800640 1 -- start start 2024-04-04T06:08:38.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.445+0000 7ff506800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff5000d0990 0x7ff5000aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:38.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.445+0000 7ff506800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5000afab0 con 0x7ff5000af2e0 2024-04-04T06:08:38.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.445+0000 7ff506800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff50015cc80 con 0x7ff500153580 2024-04-04T06:08:38.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.445+0000 7ff506800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff50015da00 con 0x7ff5000d0990 2024-04-04T06:08:38.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.446+0000 7ff4ffe00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7ff5000af2e0 0x7ff5001600d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:32782/0 (socket says 172.21.15.159:32782) 2024-04-04T06:08:38.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.446+0000 7ff4ffe00640 1 -- 172.21.15.159:0/2899819417 learned_addr learned my addr 172.21.15.159:0/2899819417 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:38.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.446+0000 7ff4f7e00640 1 --2- 172.21.15.159:0/2899819417 >> v2:172.21.15.178:3300/0 conn(0x7ff5000d0990 0x7ff5000aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:38.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.446+0000 7ff4fea00640 1 -- 172.21.15.159:0/2899819417 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3109340761 0 0) 0x7ff5000afab0 con 0x7ff5000af2e0 2024-04-04T06:08:38.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.446+0000 7ff4fea00640 1 -- 172.21.15.159:0/2899819417 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4e4003520 con 0x7ff5000af2e0 2024-04-04T06:08:38.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.447+0000 7ff4fea00640 1 -- 172.21.15.159:0/2899819417 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3816879943 0 0) 0x7ff4e4003520 con 0x7ff5000af2e0 2024-04-04T06:08:38.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.447+0000 7ff4fea00640 1 -- 172.21.15.159:0/2899819417 >> v2:172.21.15.178:3300/0 conn(0x7ff5000d0990 msgr2=0x7ff5000aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:38.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.447+0000 7ff4fea00640 1 --2- 172.21.15.159:0/2899819417 >> v2:172.21.15.178:3300/0 conn(0x7ff5000d0990 0x7ff5000aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:38.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.447+0000 7ff4fea00640 1 -- 172.21.15.159:0/2899819417 >> v1:172.21.15.178:6789/0 conn(0x7ff500153580 legacy=0x7ff5000d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:38.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.447+0000 7ff4fea00640 1 -- 172.21.15.159:0/2899819417 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50015cf20 con 0x7ff5000af2e0 2024-04-04T06:08:38.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.447+0000 7ff4fea00640 1 -- 172.21.15.159:0/2899819417 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7ff4f80019f0 con 0x7ff5000af2e0 2024-04-04T06:08:38.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.447+0000 7ff4fea00640 1 -- 172.21.15.159:0/2899819417 >> v1:172.21.15.159:6789/0 conn(0x7ff5000af2e0 legacy=0x7ff5001600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:38.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.447+0000 7ff4fea00640 1 --2- 172.21.15.159:0/2899819417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4e40039b0 0x7ff4e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:38.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.447+0000 7ff4fea00640 1 --2- 172.21.15.159:0/2899819417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4e4004800 0x7ff4e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:38.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.447+0000 7ff4fea00640 1 --2- 172.21.15.159:0/2899819417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4e40051c0 0x7ff4e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:38.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.447+0000 7ff4fea00640 1 -- 172.21.15.159:0/2899819417 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff50015da00 con 0x7ff4e4004800 2024-04-04T06:08:38.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.447+0000 7ff4fea00640 1 -- 172.21.15.159:0/2899819417 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e400c160 con 0x7ff4e40051c0 2024-04-04T06:08:38.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.447+0000 7ff4fea00640 1 -- 172.21.15.159:0/2899819417 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e400c360 con 0x7ff4e40039b0 2024-04-04T06:08:38.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.447+0000 7ff4f7e00640 1 --2- 172.21.15.159:0/2899819417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4e4004800 0x7ff4e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:38.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.447+0000 7ff4ff400640 1 --2- 172.21.15.159:0/2899819417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4e40039b0 0x7ff4e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:38.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.448+0000 7ff4f7e00640 1 -- 172.21.15.159:0/2899819417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4e40039b0 msgr2=0x7ff4e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:38.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.448+0000 7ff4ffe00640 1 --2- 172.21.15.159:0/2899819417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4e40051c0 0x7ff4e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:38.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.448+0000 7ff4f7e00640 1 --2- 172.21.15.159:0/2899819417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4e40039b0 0x7ff4e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:38.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.448+0000 7ff4f7e00640 1 -- 172.21.15.159:0/2899819417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4e40051c0 msgr2=0x7ff4e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:38.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.448+0000 7ff4f7e00640 1 --2- 172.21.15.159:0/2899819417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4e40051c0 0x7ff4e400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:38.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.448+0000 7ff4ffe00640 1 -- 172.21.15.159:0/2899819417 reap_dead start 2024-04-04T06:08:38.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.448+0000 7ff4f7e00640 1 -- 172.21.15.159:0/2899819417 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e4003520 con 0x7ff4e4004800 2024-04-04T06:08:38.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.450+0000 7ff4f7e00640 1 --2- 172.21.15.159:0/2899819417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4e4004800 0x7ff4e4004c80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff50006f820 tx=0x7ff4e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d572501f32107ea server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:38.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.450+0000 7ff4fea00640 1 -- 172.21.15.159:0/2899819417 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8039070 con 0x7ff4e4004800 2024-04-04T06:08:38.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.450+0000 7ff4fea00640 1 -- 172.21.15.159:0/2899819417 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4e802c660 con 0x7ff4e4004800 2024-04-04T06:08:38.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.450+0000 7ff4fea00640 1 -- 172.21.15.159:0/2899819417 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8009690 con 0x7ff4e4004800 2024-04-04T06:08:38.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.450+0000 7ff506800640 1 -- 172.21.15.159:0/2899819417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4e4004800 msgr2=0x7ff4e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:38.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.450+0000 7ff506800640 1 --2- 172.21.15.159:0/2899819417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4e4004800 0x7ff4e4004c80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff50006f820 tx=0x7ff4e4005830 comp rx=0 tx=0).stop 2024-04-04T06:08:38.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.451+0000 7ff506800640 1 -- 172.21.15.159:0/2899819417 shutdown_connections 2024-04-04T06:08:38.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.451+0000 7ff506800640 1 --2- 172.21.15.159:0/2899819417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4e4004800 0x7ff4e4004c80 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:38.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.451+0000 7ff506800640 1 -- 172.21.15.159:0/2899819417 >> 172.21.15.159:0/2899819417 conn(0x7ff50006b750 msgr2=0x7ff50006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:38.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.451+0000 7ff506800640 1 -- 172.21.15.159:0/2899819417 shutdown_connections 2024-04-04T06:08:38.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.451+0000 7ff506800640 1 -- 172.21.15.159:0/2899819417 wait complete. 2024-04-04T06:08:38.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.453+0000 7ff506800640 1 Processor -- start 2024-04-04T06:08:38.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.454+0000 7ff506800640 1 -- start start 2024-04-04T06:08:38.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.455+0000 7ff506800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4e4004800 0x7ff5000999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:38.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.455+0000 7ff506800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff50014fc10 0x7ff5000ce910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:38.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.455+0000 7ff506800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff500150090 0x7ff5000cee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:38.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.455+0000 7ff506800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff500153960 con 0x7ff4e4004800 2024-04-04T06:08:38.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.455+0000 7ff506800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff50015d4e0 con 0x7ff500150090 2024-04-04T06:08:38.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.455+0000 7ff506800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff5000af6c0 con 0x7ff50014fc10 2024-04-04T06:08:38.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.455+0000 7ff4ff400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4e4004800 0x7ff5000999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:38.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.456+0000 7ff4ff400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4e4004800 0x7ff5000999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47346/0 (socket says 172.21.15.159:47346) 2024-04-04T06:08:38.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.456+0000 7ff4ff400640 1 -- 172.21.15.159:0/1193181817 learned_addr learned my addr 172.21.15.159:0/1193181817 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:38.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.456+0000 7ff4f7e00640 1 --2- 172.21.15.159:0/1193181817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff50014fc10 0x7ff5000ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:38.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.456+0000 7ff4ffe00640 1 --2- 172.21.15.159:0/1193181817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff500150090 0x7ff5000cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:38.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.456+0000 7ff4ff400640 1 -- 172.21.15.159:0/1193181817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff50014fc10 msgr2=0x7ff5000ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:38.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.456+0000 7ff4ff400640 1 --2- 172.21.15.159:0/1193181817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff50014fc10 0x7ff5000ce910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:38.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.456+0000 7ff4ff400640 1 -- 172.21.15.159:0/1193181817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff500150090 msgr2=0x7ff5000cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:38.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.456+0000 7ff4ff400640 1 --2- 172.21.15.159:0/1193181817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff500150090 0x7ff5000cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:38.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.456+0000 7ff4ff400640 1 -- 172.21.15.159:0/1193181817 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5000cf770 con 0x7ff4e4004800 2024-04-04T06:08:38.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.456+0000 7ff4ffe00640 1 --2- 172.21.15.159:0/1193181817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff500150090 0x7ff5000cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:38.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.456+0000 7ff4ff400640 1 --2- 172.21.15.159:0/1193181817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4e4004800 0x7ff5000999e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000e4d0 tx=0x7ff4f000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:38.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.456+0000 7ff4f7400640 1 -- 172.21.15.159:0/1193181817 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f00053a0 con 0x7ff4e4004800 2024-04-04T06:08:38.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.457+0000 7ff4f7400640 1 -- 172.21.15.159:0/1193181817 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4f0018070 con 0x7ff4e4004800 2024-04-04T06:08:38.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.457+0000 7ff4f7400640 1 -- 172.21.15.159:0/1193181817 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0003810 con 0x7ff4e4004800 2024-04-04T06:08:38.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.457+0000 7ff506800640 1 -- 172.21.15.159:0/1193181817 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50014eb70 con 0x7ff4e4004800 2024-04-04T06:08:38.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.457+0000 7ff506800640 1 -- 172.21.15.159:0/1193181817 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff50014f130 con 0x7ff4e4004800 2024-04-04T06:08:38.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.458+0000 7ff506800640 1 -- 172.21.15.159:0/1193181817 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5000685c0 con 0x7ff4e4004800 2024-04-04T06:08:38.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.459+0000 7ff4f7400640 1 -- 172.21.15.159:0/1193181817 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4f00039b0 con 0x7ff4e4004800 2024-04-04T06:08:38.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.460+0000 7ff4f7400640 1 --2- 172.21.15.159:0/1193181817 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff4dc06c8f0 0x7ff4dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:38.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.460+0000 7ff4f7400640 1 -- 172.21.15.159:0/1193181817 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7ff4f008b820 con 0x7ff4e4004800 2024-04-04T06:08:38.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.460+0000 7ff4f7e00640 1 --2- 172.21.15.159:0/1193181817 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff4dc06c8f0 0x7ff4dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:38.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.461+0000 7ff4f7e00640 1 --2- 172.21.15.159:0/1193181817 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff4dc06c8f0 0x7ff4dc06edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8036000 tx=0x7ff4e8030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:38.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.464+0000 7ff4f7400640 1 -- 172.21.15.159:0/1193181817 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff5000cf770 con 0x7ff4e4004800 2024-04-04T06:08:38.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.771+0000 7ff506800640 1 -- 172.21.15.159:0/1193181817 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff50006eca0 con 0x7ff4e4004800 2024-04-04T06:08:38.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.772+0000 7ff4f7400640 1 -- 172.21.15.159:0/1193181817 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) ==== 80+0+0 (secure 0 0 0) 0x7ff4f0059ca0 con 0x7ff4e4004800 2024-04-04T06:08:38.766 INFO:teuthology.orchestra.run.smithi159.stderr:listed 0 entries 2024-04-04T06:08:38.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.774+0000 7ff506800640 1 -- 172.21.15.159:0/1193181817 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff4dc06c8f0 msgr2=0x7ff4dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:38.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.774+0000 7ff506800640 1 --2- 172.21.15.159:0/1193181817 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff4dc06c8f0 0x7ff4dc06edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8036000 tx=0x7ff4e8030000 comp rx=0 tx=0).stop 2024-04-04T06:08:38.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.774+0000 7ff506800640 1 -- 172.21.15.159:0/1193181817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4e4004800 msgr2=0x7ff5000999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:38.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.774+0000 7ff506800640 1 --2- 172.21.15.159:0/1193181817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4e4004800 0x7ff5000999e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000e4d0 tx=0x7ff4f000e990 comp rx=0 tx=0).stop 2024-04-04T06:08:38.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.775+0000 7ff506800640 1 -- 172.21.15.159:0/1193181817 shutdown_connections 2024-04-04T06:08:38.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.775+0000 7ff506800640 1 --2- 172.21.15.159:0/1193181817 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff4dc06c8f0 0x7ff4dc06edb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:38.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.775+0000 7ff506800640 1 --2- 172.21.15.159:0/1193181817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff500150090 0x7ff5000cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:38.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.775+0000 7ff506800640 1 --2- 172.21.15.159:0/1193181817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff50014fc10 0x7ff5000ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:38.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.775+0000 7ff506800640 1 --2- 172.21.15.159:0/1193181817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4e4004800 0x7ff5000999e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:38.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.775+0000 7ff506800640 1 -- 172.21.15.159:0/1193181817 >> 172.21.15.159:0/1193181817 conn(0x7ff50006b750 msgr2=0x7ff50006d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:38.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.775+0000 7ff506800640 1 -- 172.21.15.159:0/1193181817 shutdown_connections 2024-04-04T06:08:38.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:38.775+0000 7ff506800640 1 -- 172.21.15.159:0/1193181817 wait complete. 2024-04-04T06:08:38.788 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-04T06:08:38.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:08:38.789 DEBUG:teuthology.orchestra.run.smithi159:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-04T06:08:38.913 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:08:38.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:08:38.913 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-04T06:08:38.969 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:08:38.970 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:08:38.970 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:08:39.009 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:08:39.010 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:08:39.010 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:08:39.010 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:08:39.049 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:08:39.049 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:08:39.049 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:08:39.049 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:08:39.097 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:08:39.098 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:08:39.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.303+0000 7f0728600640 1 Processor -- start 2024-04-04T06:08:39.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.304+0000 7f0728600640 1 -- start start 2024-04-04T06:08:39.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.305+0000 7f0728600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f072015e540 0x7f072015f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:39.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.305+0000 7f0728600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07200b4560 0x7f07200b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:39.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.305+0000 7f0728600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07200b41e0 con 0x7f07200b00a0 2024-04-04T06:08:39.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.305+0000 7f0728600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f07200b4e80 con 0x7f072015e540 2024-04-04T06:08:39.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.305+0000 7f0728600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f07200b3c40 con 0x7f07200b4560 2024-04-04T06:08:39.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.306+0000 7f0725c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f072015e540 0x7f072015f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:39.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.306+0000 7f0725200640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f07200b00a0 0x7f0720162b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:32790/0 (socket says 172.21.15.159:32790) 2024-04-04T06:08:39.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.306+0000 7f0725200640 1 -- 172.21.15.159:0/122215836 learned_addr learned my addr 172.21.15.159:0/122215836 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:39.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.306+0000 7f0726600640 1 --2- 172.21.15.159:0/122215836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07200b4560 0x7f07200b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:39.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.306+0000 7f0725c00640 1 -- 172.21.15.159:0/122215836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07200b4560 msgr2=0x7f07200b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:39.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.306+0000 7f0725c00640 1 --2- 172.21.15.159:0/122215836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07200b4560 0x7f07200b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:39.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.306+0000 7f0725c00640 1 -- 172.21.15.159:0/122215836 >> v1:172.21.15.159:6789/0 conn(0x7f07200b00a0 legacy=0x7f0720162b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:39.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.306+0000 7f0725c00640 1 -- 172.21.15.159:0/122215836 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07200b3f00 con 0x7f072015e540 2024-04-04T06:08:39.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.306+0000 7f0726600640 1 --2- 172.21.15.159:0/122215836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07200b4560 0x7f07200b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:39.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.307+0000 7f0725c00640 1 --2- 172.21.15.159:0/122215836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015e540 0x7f072015f6e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f07140097a0 tx=0x7f071402b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c849093744d93a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:39.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.308+0000 7f071fe00640 1 -- 172.21.15.159:0/122215836 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714039070 con 0x7f072015e540 2024-04-04T06:08:39.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.308+0000 7f071fe00640 1 -- 172.21.15.159:0/122215836 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07140316f0 con 0x7f072015e540 2024-04-04T06:08:39.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.308+0000 7f071fe00640 1 -- 172.21.15.159:0/122215836 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714031ac0 con 0x7f072015e540 2024-04-04T06:08:39.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.308+0000 7f0728600640 1 -- 172.21.15.159:0/122215836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015e540 msgr2=0x7f072015f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:39.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.308+0000 7f0728600640 1 --2- 172.21.15.159:0/122215836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015e540 0x7f072015f6e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f07140097a0 tx=0x7f071402b790 comp rx=0 tx=0).stop 2024-04-04T06:08:39.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.308+0000 7f0728600640 1 -- 172.21.15.159:0/122215836 shutdown_connections 2024-04-04T06:08:39.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.308+0000 7f0728600640 1 --2- 172.21.15.159:0/122215836 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07200b4560 0x7f07200b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:39.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.309+0000 7f0728600640 1 --2- 172.21.15.159:0/122215836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015e540 0x7f072015f6e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:39.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.309+0000 7f0728600640 1 -- 172.21.15.159:0/122215836 >> 172.21.15.159:0/122215836 conn(0x7f07200ce400 msgr2=0x7f0720071e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:39.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.309+0000 7f0728600640 1 -- 172.21.15.159:0/122215836 shutdown_connections 2024-04-04T06:08:39.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.309+0000 7f0728600640 1 -- 172.21.15.159:0/122215836 wait complete. 2024-04-04T06:08:39.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.310+0000 7f0728600640 1 Processor -- start 2024-04-04T06:08:39.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.310+0000 7f0728600640 1 -- start start 2024-04-04T06:08:39.305 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.311+0000 7f0728600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07200b00a0 0x7f072014c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:39.305 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.311+0000 7f0728600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07200b4560 0x7f07201e8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:39.305 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.311+0000 7f0728600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015e540 0x7f0720072950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:39.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.311+0000 7f0728600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f072015e920 con 0x7f072015e540 2024-04-04T06:08:39.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.311+0000 7f0728600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0720162640 con 0x7f07200b4560 2024-04-04T06:08:39.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.311+0000 7f0728600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f072015eaa0 con 0x7f07200b00a0 2024-04-04T06:08:39.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.312+0000 7f0726600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015e540 0x7f0720072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:39.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.312+0000 7f0726600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015e540 0x7f0720072950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47364/0 (socket says 172.21.15.159:47364) 2024-04-04T06:08:39.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.312+0000 7f0726600640 1 -- 172.21.15.159:0/773284673 learned_addr learned my addr 172.21.15.159:0/773284673 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:39.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.312+0000 7f0725200640 1 --2- 172.21.15.159:0/773284673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07200b4560 0x7f07201e8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:39.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.312+0000 7f0725c00640 1 --2- 172.21.15.159:0/773284673 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07200b00a0 0x7f072014c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:39.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.312+0000 7f0726600640 1 -- 172.21.15.159:0/773284673 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07200b00a0 msgr2=0x7f072014c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:39.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.312+0000 7f0726600640 1 --2- 172.21.15.159:0/773284673 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07200b00a0 0x7f072014c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:39.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.312+0000 7f0726600640 1 -- 172.21.15.159:0/773284673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07200b4560 msgr2=0x7f07201e8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:39.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.312+0000 7f0726600640 1 --2- 172.21.15.159:0/773284673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07200b4560 0x7f07201e8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:39.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.312+0000 7f0726600640 1 -- 172.21.15.159:0/773284673 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072014c960 con 0x7f072015e540 2024-04-04T06:08:39.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.313+0000 7f0726600640 1 --2- 172.21.15.159:0/773284673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015e540 0x7f0720072950 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f071800cd80 tx=0x7f071800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:39.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.313+0000 7f071d600640 1 -- 172.21.15.159:0/773284673 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071800f680 con 0x7f072015e540 2024-04-04T06:08:39.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.313+0000 7f071d600640 1 -- 172.21.15.159:0/773284673 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f071800f840 con 0x7f072015e540 2024-04-04T06:08:39.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.313+0000 7f071d600640 1 -- 172.21.15.159:0/773284673 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071800fc30 con 0x7f072015e540 2024-04-04T06:08:39.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.313+0000 7f0728600640 1 -- 172.21.15.159:0/773284673 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f072015fc50 con 0x7f072015e540 2024-04-04T06:08:39.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.313+0000 7f0728600640 1 -- 172.21.15.159:0/773284673 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0720160210 con 0x7f072015e540 2024-04-04T06:08:39.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.315+0000 7f0728600640 1 -- 172.21.15.159:0/773284673 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07200705d0 con 0x7f072015e540 2024-04-04T06:08:39.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.315+0000 7f071d600640 1 -- 172.21.15.159:0/773284673 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f071800e3e0 con 0x7f072015e540 2024-04-04T06:08:39.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.316+0000 7f071d600640 1 --2- 172.21.15.159:0/773284673 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f070006c8f0 0x7f070006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:39.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.316+0000 7f071d600640 1 -- 172.21.15.159:0/773284673 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f071808c920 con 0x7f072015e540 2024-04-04T06:08:39.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.316+0000 7f0725c00640 1 --2- 172.21.15.159:0/773284673 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f070006c8f0 0x7f070006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:39.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.318+0000 7f0725c00640 1 --2- 172.21.15.159:0/773284673 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f070006c8f0 0x7f070006edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0714036000 tx=0x7f0714008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:39.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.320+0000 7f071d600640 1 -- 172.21.15.159:0/773284673 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f07200705d0 con 0x7f072015e540 2024-04-04T06:08:39.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.585+0000 7f0728600640 1 -- 172.21.15.159:0/773284673 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0720068cd0 con 0x7f072015e540 2024-04-04T06:08:39.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.586+0000 7f071d600640 1 -- 172.21.15.159:0/773284673 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5193 (secure 0 0 0) 0x7f0718014030 con 0x7f072015e540 2024-04-04T06:08:39.580 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:39.581 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3765578393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3765578393},{"type":"v1","addr":"172.21.15.178:6835","nonce":3765578393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3260412913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3260412913},{"type":"v1","addr":"172.21.15.178:6837","nonce":3260412913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3149916718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3149916718},{"type":"v1","addr":"172.21.15.159:6833","nonce":3149916718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:07:53.523212+0000","modified":"2024-04-04T06:07:56.601750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6835/1867245367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1867245367},{"type":"v1","addr":"172.21.15.159:6835","nonce":1867245367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5589,"qdb_cluster":[5589]},"id":3}]} 2024-04-04T06:08:39.581 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 25 2024-04-04T06:08:39.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.589+0000 7f0728600640 1 -- 172.21.15.159:0/773284673 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f070006c8f0 msgr2=0x7f070006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:39.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.589+0000 7f0728600640 1 --2- 172.21.15.159:0/773284673 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f070006c8f0 0x7f070006edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0714036000 tx=0x7f0714008030 comp rx=0 tx=0).stop 2024-04-04T06:08:39.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.589+0000 7f0728600640 1 -- 172.21.15.159:0/773284673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015e540 msgr2=0x7f0720072950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:39.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.589+0000 7f0728600640 1 --2- 172.21.15.159:0/773284673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015e540 0x7f0720072950 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f071800cd80 tx=0x7f071800a650 comp rx=0 tx=0).stop 2024-04-04T06:08:39.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.589+0000 7f0728600640 1 -- 172.21.15.159:0/773284673 shutdown_connections 2024-04-04T06:08:39.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.589+0000 7f0728600640 1 --2- 172.21.15.159:0/773284673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015e540 0x7f0720072950 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:39.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.589+0000 7f0728600640 1 --2- 172.21.15.159:0/773284673 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f070006c8f0 0x7f070006edb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:39.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.589+0000 7f0728600640 1 --2- 172.21.15.159:0/773284673 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07200b4560 0x7f07201e8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:39.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.589+0000 7f0728600640 1 --2- 172.21.15.159:0/773284673 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07200b00a0 0x7f072014c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:39.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.589+0000 7f0728600640 1 -- 172.21.15.159:0/773284673 >> 172.21.15.159:0/773284673 conn(0x7f07200ce400 msgr2=0x7f07200b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:39.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.589+0000 7f0728600640 1 -- 172.21.15.159:0/773284673 shutdown_connections 2024-04-04T06:08:39.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.589+0000 7f0728600640 1 -- 172.21.15.159:0/773284673 wait complete. 2024-04-04T06:08:39.603 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:08:39.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.813+0000 7f88fbe00640 1 Processor -- start 2024-04-04T06:08:39.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.813+0000 7f88fbe00640 1 -- start start 2024-04-04T06:08:39.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.814+0000 7f88fbe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88f415e700 0x7f88f40b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:39.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.814+0000 7f88fbe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f88f40d1610 0x7f88f40b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:39.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.814+0000 7f88fbe00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88f415eae0 con 0x7f88f40d1cf0 2024-04-04T06:08:39.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.814+0000 7f88fbe00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f88f40cfeb0 con 0x7f88f40d1610 2024-04-04T06:08:39.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.814+0000 7f88fbe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f88f40d0190 con 0x7f88f415e700 2024-04-04T06:08:39.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.815+0000 7f88f8a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f88f40d1610 0x7f88f40b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:39.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.815+0000 7f88f9e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f88f40d1cf0 0x7f88f4070fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:32794/0 (socket says 172.21.15.159:32794) 2024-04-04T06:08:39.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.815+0000 7f88f9e00640 1 -- 172.21.15.159:0/3413654611 learned_addr learned my addr 172.21.15.159:0/3413654611 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:39.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.815+0000 7f88f9400640 1 --2- 172.21.15.159:0/3413654611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88f415e700 0x7f88f40b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:39.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.815+0000 7f88f8a00640 1 -- 172.21.15.159:0/3413654611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88f415e700 msgr2=0x7f88f40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:39.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.815+0000 7f88f8a00640 1 --2- 172.21.15.159:0/3413654611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88f415e700 0x7f88f40b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:39.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.815+0000 7f88f8a00640 1 -- 172.21.15.159:0/3413654611 >> v1:172.21.15.159:6789/0 conn(0x7f88f40d1cf0 legacy=0x7f88f4070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:39.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.816+0000 7f88f8a00640 1 -- 172.21.15.159:0/3413654611 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f40d0470 con 0x7f88f40d1610 2024-04-04T06:08:39.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.815+0000 7f88f3e00640 1 -- 172.21.15.159:0/3413654611 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2268973831 0 0) 0x7f88f415eae0 con 0x7f88f40d1cf0 2024-04-04T06:08:39.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.816+0000 7f88f9400640 1 --2- 172.21.15.159:0/3413654611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88f415e700 0x7f88f40b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:08:39.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.817+0000 7f88f8a00640 1 --2- 172.21.15.159:0/3413654611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88f40d1610 0x7f88f40b4360 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f88e00097a0 tx=0x7f88e002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1200c07496b3934b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:39.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.817+0000 7f88f3e00640 1 -- 172.21.15.159:0/3413654611 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e0031780 con 0x7f88f40d1610 2024-04-04T06:08:39.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.817+0000 7f88f3e00640 1 -- 172.21.15.159:0/3413654611 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88e0037070 con 0x7f88f40d1610 2024-04-04T06:08:39.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.817+0000 7f88f3e00640 1 -- 172.21.15.159:0/3413654611 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e0031b40 con 0x7f88f40d1610 2024-04-04T06:08:39.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.817+0000 7f88fbe00640 1 -- 172.21.15.159:0/3413654611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88f40d1610 msgr2=0x7f88f40b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:39.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.817+0000 7f88fbe00640 1 --2- 172.21.15.159:0/3413654611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88f40d1610 0x7f88f40b4360 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f88e00097a0 tx=0x7f88e002b820 comp rx=0 tx=0).stop 2024-04-04T06:08:39.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.818+0000 7f88fbe00640 1 -- 172.21.15.159:0/3413654611 shutdown_connections 2024-04-04T06:08:39.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.818+0000 7f88fbe00640 1 --2- 172.21.15.159:0/3413654611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88f40d1610 0x7f88f40b4360 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:39.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.818+0000 7f88fbe00640 1 --2- 172.21.15.159:0/3413654611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88f415e700 0x7f88f40b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:39.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.818+0000 7f88fbe00640 1 -- 172.21.15.159:0/3413654611 >> 172.21.15.159:0/3413654611 conn(0x7f88f40d2240 msgr2=0x7f88f40d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:39.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.818+0000 7f88fbe00640 1 -- 172.21.15.159:0/3413654611 shutdown_connections 2024-04-04T06:08:39.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.818+0000 7f88fbe00640 1 -- 172.21.15.159:0/3413654611 wait complete. 2024-04-04T06:08:39.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.819+0000 7f88fbe00640 1 Processor -- start 2024-04-04T06:08:39.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.819+0000 7f88fbe00640 1 -- start start 2024-04-04T06:08:39.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.820+0000 7f88fbe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88f40d1610 0x7f88f4151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:39.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.820+0000 7f88fbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88f40d1cf0 0x7f88f40b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:39.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.820+0000 7f88fbe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88f415e700 0x7f88f41f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:39.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.820+0000 7f88fbe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f88f406d1b0 con 0x7f88f40d1cf0 2024-04-04T06:08:39.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.820+0000 7f88fbe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f88f40d0a30 con 0x7f88f40d1610 2024-04-04T06:08:39.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.820+0000 7f88fbe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f88f406d330 con 0x7f88f415e700 2024-04-04T06:08:39.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.821+0000 7f88f8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88f40d1cf0 0x7f88f40b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:39.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.821+0000 7f88f8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88f40d1cf0 0x7f88f40b6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47384/0 (socket says 172.21.15.159:47384) 2024-04-04T06:08:39.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.821+0000 7f88f8a00640 1 -- 172.21.15.159:0/2544326881 learned_addr learned my addr 172.21.15.159:0/2544326881 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:39.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.821+0000 7f88f9400640 1 --2- 172.21.15.159:0/2544326881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88f40d1610 0x7f88f4151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:39.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.821+0000 7f88f9e00640 1 --2- 172.21.15.159:0/2544326881 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88f415e700 0x7f88f41f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:39.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.821+0000 7f88f8a00640 1 -- 172.21.15.159:0/2544326881 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88f415e700 msgr2=0x7f88f41f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:39.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.821+0000 7f88f8a00640 1 --2- 172.21.15.159:0/2544326881 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88f415e700 0x7f88f41f46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:39.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.821+0000 7f88f8a00640 1 -- 172.21.15.159:0/2544326881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88f40d1610 msgr2=0x7f88f4151f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:39.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.821+0000 7f88f8a00640 1 --2- 172.21.15.159:0/2544326881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88f40d1610 0x7f88f4151f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:39.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.821+0000 7f88f8a00640 1 -- 172.21.15.159:0/2544326881 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f41f4c30 con 0x7f88f40d1cf0 2024-04-04T06:08:39.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.821+0000 7f88f9e00640 1 --2- 172.21.15.159:0/2544326881 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88f415e700 0x7f88f41f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:39.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.822+0000 7f88f8a00640 1 --2- 172.21.15.159:0/2544326881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88f40d1cf0 0x7f88f40b6af0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f88e00097a0 tx=0x7f88e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:39.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.822+0000 7f88f1600640 1 -- 172.21.15.159:0/2544326881 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e0004230 con 0x7f88f40d1cf0 2024-04-04T06:08:39.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.822+0000 7f88f1600640 1 -- 172.21.15.159:0/2544326881 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88e00043d0 con 0x7f88f40d1cf0 2024-04-04T06:08:39.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.822+0000 7f88f1600640 1 -- 172.21.15.159:0/2544326881 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e0003850 con 0x7f88f40d1cf0 2024-04-04T06:08:39.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.822+0000 7f88fbe00640 1 -- 172.21.15.159:0/2544326881 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f40b6470 con 0x7f88f40d1cf0 2024-04-04T06:08:39.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.822+0000 7f88fbe00640 1 -- 172.21.15.159:0/2544326881 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f41527d0 con 0x7f88f40d1cf0 2024-04-04T06:08:39.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.823+0000 7f88fbe00640 1 -- 172.21.15.159:0/2544326881 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88f40685c0 con 0x7f88f40d1cf0 2024-04-04T06:08:39.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.824+0000 7f88f1600640 1 -- 172.21.15.159:0/2544326881 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f88e00039f0 con 0x7f88f40d1cf0 2024-04-04T06:08:39.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.825+0000 7f88f1600640 1 --2- 172.21.15.159:0/2544326881 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f88d406c8f0 0x7f88d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:39.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.825+0000 7f88f1600640 1 -- 172.21.15.159:0/2544326881 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f88e00adb50 con 0x7f88f40d1cf0 2024-04-04T06:08:39.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.826+0000 7f88f9400640 1 --2- 172.21.15.159:0/2544326881 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f88d406c8f0 0x7f88d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:39.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.827+0000 7f88f9400640 1 --2- 172.21.15.159:0/2544326881 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f88d406c8f0 0x7f88d406edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f88f41618f0 tx=0x7f88e8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:39.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:39.830+0000 7f88f1600640 1 -- 172.21.15.159:0/2544326881 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f88e0041070 con 0x7f88f40d1cf0 2024-04-04T06:08:40.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.035+0000 7f88fbe00640 1 -- 172.21.15.159:0/2544326881 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88f415c130 con 0x7f88f40d1cf0 2024-04-04T06:08:40.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.037+0000 7f88f1600640 1 -- 172.21.15.159:0/2544326881 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16777 (secure 0 0 0) 0x7f88e002f050 con 0x7f88f40d1cf0 2024-04-04T06:08:40.031 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:40.031 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":48,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:08:34.246895+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:07:51.357366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-04T06:07:52.495884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.30","mappings":[{"from":0,"to":4}]},{"pgid":"7.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:08:40.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.039+0000 7f88fbe00640 1 -- 172.21.15.159:0/2544326881 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f88d406c8f0 msgr2=0x7f88d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:40.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.039+0000 7f88fbe00640 1 --2- 172.21.15.159:0/2544326881 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f88d406c8f0 0x7f88d406edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f88f41618f0 tx=0x7f88e8008040 comp rx=0 tx=0).stop 2024-04-04T06:08:40.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.039+0000 7f88fbe00640 1 -- 172.21.15.159:0/2544326881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88f40d1cf0 msgr2=0x7f88f40b6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:40.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.039+0000 7f88fbe00640 1 --2- 172.21.15.159:0/2544326881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88f40d1cf0 0x7f88f40b6af0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f88e00097a0 tx=0x7f88e00046c0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.040+0000 7f88fbe00640 1 -- 172.21.15.159:0/2544326881 shutdown_connections 2024-04-04T06:08:40.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.040+0000 7f88fbe00640 1 --2- 172.21.15.159:0/2544326881 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88f415e700 0x7f88f41f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.040+0000 7f88fbe00640 1 --2- 172.21.15.159:0/2544326881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88f40d1cf0 0x7f88f40b6af0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.040+0000 7f88fbe00640 1 --2- 172.21.15.159:0/2544326881 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f88d406c8f0 0x7f88d406edb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.040+0000 7f88fbe00640 1 --2- 172.21.15.159:0/2544326881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88f40d1610 0x7f88f4151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.040+0000 7f88fbe00640 1 -- 172.21.15.159:0/2544326881 >> 172.21.15.159:0/2544326881 conn(0x7f88f40d2240 msgr2=0x7f88f4070fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:40.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.040+0000 7f88fbe00640 1 -- 172.21.15.159:0/2544326881 shutdown_connections 2024-04-04T06:08:40.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.040+0000 7f88fbe00640 1 -- 172.21.15.159:0/2544326881 wait complete. 2024-04-04T06:08:40.052 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:07:51.357366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:07:52.495884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:08:40.053 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:08:40.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.279+0000 7f307a400640 1 Processor -- start 2024-04-04T06:08:40.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.279+0000 7f307a400640 1 -- start start 2024-04-04T06:08:40.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.280+0000 7f307a400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30740d43d0 0x7f30740d47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:40.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.280+0000 7f307a400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f307406dfa0 con 0x7f30740cf6b0 2024-04-04T06:08:40.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.280+0000 7f307a400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30740d4cf0 con 0x7f307415e860 2024-04-04T06:08:40.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.280+0000 7f307a400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f30740cf220 con 0x7f30740d43d0 2024-04-04T06:08:40.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.280+0000 7f3073e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f30740cf6b0 0x7f30740cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:32804/0 (socket says 172.21.15.159:32804) 2024-04-04T06:08:40.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.280+0000 7f3073e00640 1 -- 172.21.15.159:0/540911667 learned_addr learned my addr 172.21.15.159:0/540911667 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:40.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.281+0000 7f306aa00640 1 --2- 172.21.15.159:0/540911667 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30740d43d0 0x7f30740d47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:40.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.281+0000 7f3072a00640 1 -- 172.21.15.159:0/540911667 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 179850708 0 0) 0x7f307406dfa0 con 0x7f30740cf6b0 2024-04-04T06:08:40.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.281+0000 7f3072a00640 1 -- 172.21.15.159:0/540911667 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3058003520 con 0x7f30740cf6b0 2024-04-04T06:08:40.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.281+0000 7f3072a00640 1 -- 172.21.15.159:0/540911667 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 743403887 0 0) 0x7f3058003520 con 0x7f30740cf6b0 2024-04-04T06:08:40.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.281+0000 7f3072a00640 1 -- 172.21.15.159:0/540911667 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30740d43d0 msgr2=0x7f30740d47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:40.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.281+0000 7f3072a00640 1 --2- 172.21.15.159:0/540911667 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30740d43d0 0x7f30740d47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.281+0000 7f3072a00640 1 -- 172.21.15.159:0/540911667 >> v1:172.21.15.178:6789/0 conn(0x7f307415e860 legacy=0x7f307406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:40.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.281+0000 7f3072a00640 1 -- 172.21.15.159:0/540911667 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307406e240 con 0x7f30740cf6b0 2024-04-04T06:08:40.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.281+0000 7f3072a00640 1 -- 172.21.15.159:0/540911667 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f306c0019f0 con 0x7f30740cf6b0 2024-04-04T06:08:40.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.281+0000 7f3072a00640 1 -- 172.21.15.159:0/540911667 >> v1:172.21.15.159:6789/0 conn(0x7f30740cf6b0 legacy=0x7f30740cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:40.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.281+0000 7f3072a00640 1 --2- 172.21.15.159:0/540911667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30580039e0 0x7f3058003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:40.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.281+0000 7f3072a00640 1 --2- 172.21.15.159:0/540911667 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30740d43d0 0x7f3058004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:40.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.282+0000 7f3072a00640 1 --2- 172.21.15.159:0/540911667 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3058004d40 0x7f30580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:40.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.282+0000 7f3072a00640 1 -- 172.21.15.159:0/540911667 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f30740cf220 con 0x7f3058004d40 2024-04-04T06:08:40.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.282+0000 7f3072a00640 1 -- 172.21.15.159:0/540911667 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f305800bad0 con 0x7f30580039e0 2024-04-04T06:08:40.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.282+0000 7f3072a00640 1 -- 172.21.15.159:0/540911667 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f305800bcd0 con 0x7f30740d43d0 2024-04-04T06:08:40.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.282+0000 7f3073e00640 1 --2- 172.21.15.159:0/540911667 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3058004d40 0x7f30580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:40.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.282+0000 7f3073400640 1 --2- 172.21.15.159:0/540911667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30580039e0 0x7f3058003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:40.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.282+0000 7f306aa00640 1 --2- 172.21.15.159:0/540911667 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30740d43d0 0x7f3058004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:40.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.282+0000 7f3073e00640 1 -- 172.21.15.159:0/540911667 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30740d43d0 msgr2=0x7f3058004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:40.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.282+0000 7f3073e00640 1 --2- 172.21.15.159:0/540911667 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30740d43d0 0x7f3058004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.282+0000 7f3073e00640 1 -- 172.21.15.159:0/540911667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30580039e0 msgr2=0x7f3058003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:40.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.282+0000 7f3073e00640 1 --2- 172.21.15.159:0/540911667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30580039e0 0x7f3058003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.282+0000 7f3073e00640 1 -- 172.21.15.159:0/540911667 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3058003520 con 0x7f3058004d40 2024-04-04T06:08:40.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.283+0000 7f3073e00640 1 --2- 172.21.15.159:0/540911667 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3058004d40 0x7f30580051c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f306c00a760 tx=0x7f3058005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0544c359b61309e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:40.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.284+0000 7f3072a00640 1 -- 172.21.15.159:0/540911667 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c00a3b0 con 0x7f3058004d40 2024-04-04T06:08:40.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.284+0000 7f3072a00640 1 -- 172.21.15.159:0/540911667 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f306c034650 con 0x7f3058004d40 2024-04-04T06:08:40.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.284+0000 7f3072a00640 1 -- 172.21.15.159:0/540911667 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c034a40 con 0x7f3058004d40 2024-04-04T06:08:40.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.284+0000 7f307a400640 1 -- 172.21.15.159:0/540911667 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3058004d40 msgr2=0x7f30580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:40.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.284+0000 7f307a400640 1 --2- 172.21.15.159:0/540911667 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3058004d40 0x7f30580051c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f306c00a760 tx=0x7f3058005830 comp rx=0 tx=0).stop 2024-04-04T06:08:40.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.284+0000 7f3073e00640 1 -- 172.21.15.159:0/540911667 reap_dead start 2024-04-04T06:08:40.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.284+0000 7f307a400640 1 -- 172.21.15.159:0/540911667 shutdown_connections 2024-04-04T06:08:40.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.284+0000 7f307a400640 1 -- 172.21.15.159:0/540911667 >> 172.21.15.159:0/540911667 conn(0x7f307409d7d0 msgr2=0x7f307408bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:40.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.284+0000 7f307a400640 1 -- 172.21.15.159:0/540911667 shutdown_connections 2024-04-04T06:08:40.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.285+0000 7f307a400640 1 -- 172.21.15.159:0/540911667 wait complete. 2024-04-04T06:08:40.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.286+0000 7f307a400640 1 Processor -- start 2024-04-04T06:08:40.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.286+0000 7f307a400640 1 -- start start 2024-04-04T06:08:40.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.287+0000 7f307a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3058004d40 0x7f3074158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:40.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.287+0000 7f307a400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30740b8d00 0x7f30740b9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:40.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.287+0000 7f307a400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30740cd280 0x7f30740cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:40.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.287+0000 7f307a400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f307406dc00 con 0x7f3058004d40 2024-04-04T06:08:40.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.287+0000 7f307a400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f30740d2c50 con 0x7f30740b8d00 2024-04-04T06:08:40.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.287+0000 7f307a400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f307406dd80 con 0x7f30740cd280 2024-04-04T06:08:40.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.287+0000 7f3073400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3058004d40 0x7f3074158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:40.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.288+0000 7f3073400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3058004d40 0x7f3074158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47412/0 (socket says 172.21.15.159:47412) 2024-04-04T06:08:40.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.288+0000 7f3073400640 1 -- 172.21.15.159:0/1957166948 learned_addr learned my addr 172.21.15.159:0/1957166948 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:40.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.288+0000 7f306aa00640 1 --2- 172.21.15.159:0/1957166948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30740b8d00 0x7f30740b9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:40.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.288+0000 7f3073e00640 1 --2- 172.21.15.159:0/1957166948 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30740cd280 0x7f30740cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:40.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.288+0000 7f3073400640 1 -- 172.21.15.159:0/1957166948 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30740cd280 msgr2=0x7f30740cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:40.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.288+0000 7f3073400640 1 --2- 172.21.15.159:0/1957166948 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30740cd280 0x7f30740cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.288+0000 7f3073400640 1 -- 172.21.15.159:0/1957166948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30740b8d00 msgr2=0x7f30740b9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:40.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.288+0000 7f3073400640 1 --2- 172.21.15.159:0/1957166948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30740b8d00 0x7f30740b9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.288+0000 7f3073400640 1 -- 172.21.15.159:0/1957166948 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307415fa10 con 0x7f3058004d40 2024-04-04T06:08:40.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.288+0000 7f3073e00640 1 --2- 172.21.15.159:0/1957166948 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30740cd280 0x7f30740cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:40.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.288+0000 7f306aa00640 1 --2- 172.21.15.159:0/1957166948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30740b8d00 0x7f30740b9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:40.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.288+0000 7f3073400640 1 --2- 172.21.15.159:0/1957166948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3058004d40 0x7f3074158870 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f306400e4d0 tx=0x7f306400e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:40.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.289+0000 7f306be00640 1 -- 172.21.15.159:0/1957166948 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30640053a0 con 0x7f3058004d40 2024-04-04T06:08:40.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.289+0000 7f306be00640 1 -- 172.21.15.159:0/1957166948 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3064007020 con 0x7f3058004d40 2024-04-04T06:08:40.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.289+0000 7f306be00640 1 -- 172.21.15.159:0/1957166948 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30640071f0 con 0x7f3058004d40 2024-04-04T06:08:40.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.289+0000 7f307a400640 1 -- 172.21.15.159:0/1957166948 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f307415fcf0 con 0x7f3058004d40 2024-04-04T06:08:40.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.289+0000 7f307a400640 1 -- 172.21.15.159:0/1957166948 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30741602b0 con 0x7f3058004d40 2024-04-04T06:08:40.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.290+0000 7f307a400640 1 -- 172.21.15.159:0/1957166948 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30740685c0 con 0x7f3058004d40 2024-04-04T06:08:40.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.291+0000 7f306be00640 1 -- 172.21.15.159:0/1957166948 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f306401a020 con 0x7f3058004d40 2024-04-04T06:08:40.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.292+0000 7f306be00640 1 --2- 172.21.15.159:0/1957166948 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f305006c920 0x7f305006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:40.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.292+0000 7f306be00640 1 -- 172.21.15.159:0/1957166948 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f306408b150 con 0x7f3058004d40 2024-04-04T06:08:40.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.293+0000 7f306aa00640 1 --2- 172.21.15.159:0/1957166948 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f305006c920 0x7f305006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:40.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.294+0000 7f306aa00640 1 --2- 172.21.15.159:0/1957166948 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f305006c920 0x7f305006ede0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f30740b4250 tx=0x7f305c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:40.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.297+0000 7f306be00640 1 -- 172.21.15.159:0/1957166948 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3064059530 con 0x7f3058004d40 2024-04-04T06:08:40.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.498+0000 7f307a400640 1 -- 172.21.15.159:0/1957166948 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f307408c330 con 0x7f3058004d40 2024-04-04T06:08:40.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.499+0000 7f306be00640 1 -- 172.21.15.159:0/1957166948 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) ==== 93+0+10 (secure 0 0 0) 0x7f3064058e30 con 0x7f3058004d40 2024-04-04T06:08:40.493 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:08:40.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.501+0000 7f307a400640 1 -- 172.21.15.159:0/1957166948 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f305006c920 msgr2=0x7f305006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:40.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.501+0000 7f307a400640 1 --2- 172.21.15.159:0/1957166948 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f305006c920 0x7f305006ede0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f30740b4250 tx=0x7f305c008040 comp rx=0 tx=0).stop 2024-04-04T06:08:40.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.501+0000 7f307a400640 1 -- 172.21.15.159:0/1957166948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3058004d40 msgr2=0x7f3074158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:40.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.501+0000 7f307a400640 1 --2- 172.21.15.159:0/1957166948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3058004d40 0x7f3074158870 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f306400e4d0 tx=0x7f306400e990 comp rx=0 tx=0).stop 2024-04-04T06:08:40.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.501+0000 7f307a400640 1 -- 172.21.15.159:0/1957166948 shutdown_connections 2024-04-04T06:08:40.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.501+0000 7f307a400640 1 --2- 172.21.15.159:0/1957166948 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30740cd280 0x7f30740cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.501+0000 7f307a400640 1 --2- 172.21.15.159:0/1957166948 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f305006c920 0x7f305006ede0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.501+0000 7f307a400640 1 --2- 172.21.15.159:0/1957166948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30740b8d00 0x7f30740b9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.501+0000 7f307a400640 1 --2- 172.21.15.159:0/1957166948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3058004d40 0x7f3074158870 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.502+0000 7f307a400640 1 -- 172.21.15.159:0/1957166948 >> 172.21.15.159:0/1957166948 conn(0x7f307409d7d0 msgr2=0x7f30740b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:40.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.502+0000 7f307a400640 1 -- 172.21.15.159:0/1957166948 shutdown_connections 2024-04-04T06:08:40.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.502+0000 7f307a400640 1 -- 172.21.15.159:0/1957166948 wait complete. 2024-04-04T06:08:40.514 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:08:40.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.729+0000 7f0a1ec00640 1 Processor -- start 2024-04-04T06:08:40.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.729+0000 7f0a1ec00640 1 -- start start 2024-04-04T06:08:40.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.731+0000 7f0a1ec00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0a1806dd30 0x7f0a180cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:40.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.731+0000 7f0a1ec00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a180ce890 con 0x7f0a1813f190 2024-04-04T06:08:40.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.731+0000 7f0a1ec00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a180ceb50 con 0x7f0a180cc880 2024-04-04T06:08:40.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.731+0000 7f0a1ec00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0a180cd230 con 0x7f0a1806dd30 2024-04-04T06:08:40.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.731+0000 7f0a1d800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f0a1813f190 0x7f0a180cf630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:32814/0 (socket says 172.21.15.159:32814) 2024-04-04T06:08:40.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.731+0000 7f0a1d800640 1 -- 172.21.15.159:0/4240398383 learned_addr learned my addr 172.21.15.159:0/4240398383 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:40.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.732+0000 7f0a1e200640 1 --2- 172.21.15.159:0/4240398383 >> v2:172.21.15.178:3300/0 conn(0x7f0a1806dd30 0x7f0a180cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:40.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.732+0000 7f0a17e00640 1 -- 172.21.15.159:0/4240398383 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4197821477 0 0) 0x7f0a180ce890 con 0x7f0a1813f190 2024-04-04T06:08:40.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.732+0000 7f0a17e00640 1 -- 172.21.15.159:0/4240398383 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a00003520 con 0x7f0a1813f190 2024-04-04T06:08:40.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.732+0000 7f0a17e00640 1 -- 172.21.15.159:0/4240398383 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2540454138 0 0) 0x7f0a00003520 con 0x7f0a1813f190 2024-04-04T06:08:40.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.732+0000 7f0a17e00640 1 -- 172.21.15.159:0/4240398383 >> v2:172.21.15.178:3300/0 conn(0x7f0a1806dd30 msgr2=0x7f0a180cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:40.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.732+0000 7f0a17e00640 1 --2- 172.21.15.159:0/4240398383 >> v2:172.21.15.178:3300/0 conn(0x7f0a1806dd30 0x7f0a180cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.732+0000 7f0a17e00640 1 -- 172.21.15.159:0/4240398383 >> v1:172.21.15.178:6789/0 conn(0x7f0a180cc880 legacy=0x7f0a1806d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:40.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.732+0000 7f0a17e00640 1 -- 172.21.15.159:0/4240398383 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a180cee30 con 0x7f0a1813f190 2024-04-04T06:08:40.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.732+0000 7f0a17e00640 1 -- 172.21.15.159:0/4240398383 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f0a0c001670 con 0x7f0a1813f190 2024-04-04T06:08:40.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.732+0000 7f0a17e00640 1 -- 172.21.15.159:0/4240398383 >> v1:172.21.15.159:6789/0 conn(0x7f0a1813f190 legacy=0x7f0a180cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:40.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.732+0000 7f0a17e00640 1 --2- 172.21.15.159:0/4240398383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a00003a00 0x7f0a00003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:40.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.732+0000 7f0a17e00640 1 --2- 172.21.15.159:0/4240398383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a00004800 0x7f0a00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:40.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.732+0000 7f0a17e00640 1 --2- 172.21.15.159:0/4240398383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a000051c0 0x7f0a0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:40.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.732+0000 7f0a17e00640 1 -- 172.21.15.159:0/4240398383 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0a180cd230 con 0x7f0a00004800 2024-04-04T06:08:40.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.732+0000 7f0a17e00640 1 -- 172.21.15.159:0/4240398383 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0000c0c0 con 0x7f0a000051c0 2024-04-04T06:08:40.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.732+0000 7f0a17e00640 1 -- 172.21.15.159:0/4240398383 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0a0000c2c0 con 0x7f0a00003a00 2024-04-04T06:08:40.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.733+0000 7f0a1ce00640 1 --2- 172.21.15.159:0/4240398383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a00004800 0x7f0a00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:40.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.733+0000 7f0a1e200640 1 --2- 172.21.15.159:0/4240398383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a000051c0 0x7f0a0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:40.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.733+0000 7f0a1d800640 1 --2- 172.21.15.159:0/4240398383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a00003a00 0x7f0a00003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:40.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.733+0000 7f0a1ce00640 1 -- 172.21.15.159:0/4240398383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a00003a00 msgr2=0x7f0a00003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:40.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.733+0000 7f0a1ce00640 1 --2- 172.21.15.159:0/4240398383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a00003a00 0x7f0a00003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.733+0000 7f0a1ce00640 1 -- 172.21.15.159:0/4240398383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a000051c0 msgr2=0x7f0a0000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:40.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.733+0000 7f0a1ce00640 1 --2- 172.21.15.159:0/4240398383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a000051c0 0x7f0a0000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.733+0000 7f0a1ce00640 1 -- 172.21.15.159:0/4240398383 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a00003520 con 0x7f0a00004800 2024-04-04T06:08:40.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.733+0000 7f0a1e200640 1 -- 172.21.15.159:0/4240398383 reap_dead start 2024-04-04T06:08:40.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.734+0000 7f0a1ce00640 1 --2- 172.21.15.159:0/4240398383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a00004800 0x7f0a00004c80 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0a04001750 tx=0x7f0a00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9face41ba0b6b36d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:40.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.734+0000 7f0a17e00640 1 -- 172.21.15.159:0/4240398383 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a04001370 con 0x7f0a00004800 2024-04-04T06:08:40.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.735+0000 7f0a17e00640 1 -- 172.21.15.159:0/4240398383 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a040052d0 con 0x7f0a00004800 2024-04-04T06:08:40.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.735+0000 7f0a17e00640 1 -- 172.21.15.159:0/4240398383 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a04003830 con 0x7f0a00004800 2024-04-04T06:08:40.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.735+0000 7f0a1ec00640 1 -- 172.21.15.159:0/4240398383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a00004800 msgr2=0x7f0a00004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:40.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.735+0000 7f0a1ec00640 1 --2- 172.21.15.159:0/4240398383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a00004800 0x7f0a00004c80 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0a04001750 tx=0x7f0a00005830 comp rx=0 tx=0).stop 2024-04-04T06:08:40.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.735+0000 7f0a1ec00640 1 -- 172.21.15.159:0/4240398383 shutdown_connections 2024-04-04T06:08:40.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.735+0000 7f0a1ec00640 1 --2- 172.21.15.159:0/4240398383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a00004800 0x7f0a00004c80 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.735+0000 7f0a1ec00640 1 -- 172.21.15.159:0/4240398383 >> 172.21.15.159:0/4240398383 conn(0x7f0a180b1ba0 msgr2=0x7f0a180b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:40.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.735+0000 7f0a1ec00640 1 -- 172.21.15.159:0/4240398383 shutdown_connections 2024-04-04T06:08:40.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.735+0000 7f0a1ec00640 1 -- 172.21.15.159:0/4240398383 wait complete. 2024-04-04T06:08:40.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.737+0000 7f0a1ec00640 1 Processor -- start 2024-04-04T06:08:40.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.737+0000 7f0a1ec00640 1 -- start start 2024-04-04T06:08:40.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.738+0000 7f0a1ec00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a00004800 0x7f0a18155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:40.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.738+0000 7f0a1ec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a181e7890 0x7f0a181e8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:40.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.738+0000 7f0a1ec00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a181e8690 0x7f0a181e5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:40.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.738+0000 7f0a1ec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1813f570 con 0x7f0a181e7890 2024-04-04T06:08:40.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.738+0000 7f0a1ec00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0a18160c70 con 0x7f0a00004800 2024-04-04T06:08:40.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.738+0000 7f0a1ec00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0a180cccf0 con 0x7f0a181e8690 2024-04-04T06:08:40.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.738+0000 7f0a1ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a181e7890 0x7f0a181e8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:40.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.739+0000 7f0a1ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a181e7890 0x7f0a181e8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47430/0 (socket says 172.21.15.159:47430) 2024-04-04T06:08:40.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.739+0000 7f0a1ce00640 1 -- 172.21.15.159:0/236984192 learned_addr learned my addr 172.21.15.159:0/236984192 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:40.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.739+0000 7f0a1d800640 1 --2- 172.21.15.159:0/236984192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a00004800 0x7f0a18155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:40.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.739+0000 7f0a1e200640 1 --2- 172.21.15.159:0/236984192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a181e8690 0x7f0a181e5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:40.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.739+0000 7f0a1ce00640 1 -- 172.21.15.159:0/236984192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a181e8690 msgr2=0x7f0a181e5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:40.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.739+0000 7f0a1ce00640 1 --2- 172.21.15.159:0/236984192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a181e8690 0x7f0a181e5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.739+0000 7f0a1ce00640 1 -- 172.21.15.159:0/236984192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a00004800 msgr2=0x7f0a18155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:40.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.739+0000 7f0a1ce00640 1 --2- 172.21.15.159:0/236984192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a00004800 0x7f0a18155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.739+0000 7f0a1ce00640 1 -- 172.21.15.159:0/236984192 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a181e67f0 con 0x7f0a181e7890 2024-04-04T06:08:40.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.739+0000 7f0a1ce00640 1 --2- 172.21.15.159:0/236984192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a181e7890 0x7f0a181e8150 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0a04000c00 tx=0x7f0a04001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:40.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.740+0000 7f0a15600640 1 -- 172.21.15.159:0/236984192 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a04041020 con 0x7f0a181e7890 2024-04-04T06:08:40.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.740+0000 7f0a15600640 1 -- 172.21.15.159:0/236984192 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a04005480 con 0x7f0a181e7890 2024-04-04T06:08:40.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.740+0000 7f0a15600640 1 -- 172.21.15.159:0/236984192 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a04007a30 con 0x7f0a181e7890 2024-04-04T06:08:40.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.740+0000 7f0a1ec00640 1 -- 172.21.15.159:0/236984192 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a181e6ad0 con 0x7f0a181e7890 2024-04-04T06:08:40.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.740+0000 7f0a1ec00640 1 -- 172.21.15.159:0/236984192 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a181e7090 con 0x7f0a181e7890 2024-04-04T06:08:40.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.741+0000 7f0a1ec00640 1 -- 172.21.15.159:0/236984192 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a181e8aa0 con 0x7f0a181e7890 2024-04-04T06:08:40.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.742+0000 7f0a15600640 1 -- 172.21.15.159:0/236984192 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a04044020 con 0x7f0a181e7890 2024-04-04T06:08:40.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.743+0000 7f0a15600640 1 --2- 172.21.15.159:0/236984192 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f09f806c8f0 0x7f09f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:40.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.743+0000 7f0a15600640 1 -- 172.21.15.159:0/236984192 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f0a040ad6b0 con 0x7f0a181e7890 2024-04-04T06:08:40.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.744+0000 7f0a1d800640 1 --2- 172.21.15.159:0/236984192 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f09f806c8f0 0x7f09f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:40.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.745+0000 7f0a1d800640 1 --2- 172.21.15.159:0/236984192 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f09f806c8f0 0x7f09f806edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0a1806a480 tx=0x7f0a0c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:40.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.748+0000 7f0a15600640 1 -- 172.21.15.159:0/236984192 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0a181e8aa0 con 0x7f0a181e7890 2024-04-04T06:08:40.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.948+0000 7f0a1ec00640 1 -- 172.21.15.159:0/236984192 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0a180b2870 con 0x7f0a181e7890 2024-04-04T06:08:40.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.949+0000 7f0a15600640 1 -- 172.21.15.159:0/236984192 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) ==== 104+0+11 (secure 0 0 0) 0x7f0a0404b030 con 0x7f0a181e7890 2024-04-04T06:08:40.943 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:08:40.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.951+0000 7f0a1ec00640 1 -- 172.21.15.159:0/236984192 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f09f806c8f0 msgr2=0x7f09f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:40.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.951+0000 7f0a1ec00640 1 --2- 172.21.15.159:0/236984192 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f09f806c8f0 0x7f09f806edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0a1806a480 tx=0x7f0a0c009040 comp rx=0 tx=0).stop 2024-04-04T06:08:40.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.951+0000 7f0a1ec00640 1 -- 172.21.15.159:0/236984192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a181e7890 msgr2=0x7f0a181e8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:40.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.951+0000 7f0a1ec00640 1 --2- 172.21.15.159:0/236984192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a181e7890 0x7f0a181e8150 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0a04000c00 tx=0x7f0a04001120 comp rx=0 tx=0).stop 2024-04-04T06:08:40.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.951+0000 7f0a1ec00640 1 -- 172.21.15.159:0/236984192 shutdown_connections 2024-04-04T06:08:40.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.951+0000 7f0a1ec00640 1 --2- 172.21.15.159:0/236984192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a181e8690 0x7f0a181e5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.951+0000 7f0a1ec00640 1 --2- 172.21.15.159:0/236984192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a181e7890 0x7f0a181e8150 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.951+0000 7f0a1ec00640 1 --2- 172.21.15.159:0/236984192 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f09f806c8f0 0x7f09f806edb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.951+0000 7f0a1ec00640 1 --2- 172.21.15.159:0/236984192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a00004800 0x7f0a18155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:40.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.951+0000 7f0a1ec00640 1 -- 172.21.15.159:0/236984192 >> 172.21.15.159:0/236984192 conn(0x7f0a180b1ba0 msgr2=0x7f0a1815f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:40.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.952+0000 7f0a1ec00640 1 -- 172.21.15.159:0/236984192 shutdown_connections 2024-04-04T06:08:40.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:40.952+0000 7f0a1ec00640 1 -- 172.21.15.159:0/236984192 wait complete. 2024-04-04T06:08:40.966 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:08:41.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.182+0000 7f6985600640 1 Processor -- start 2024-04-04T06:08:41.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.182+0000 7f6985600640 1 -- start start 2024-04-04T06:08:41.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.184+0000 7f6985600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69800d0990 0x7f69800aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:41.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.184+0000 7f6985600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f69800af2e0 0x7f69801600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:41.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.184+0000 7f6985600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69800afab0 con 0x7f6980153580 2024-04-04T06:08:41.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.184+0000 7f6985600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f698015cc80 con 0x7f69800af2e0 2024-04-04T06:08:41.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.184+0000 7f6985600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f698015da00 con 0x7f69800d0990 2024-04-04T06:08:41.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.184+0000 7f697f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f69800af2e0 0x7f69801600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:41.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.184+0000 7f697f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f69800af2e0 0x7f69801600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47446/0 (socket says 172.21.15.159:47446) 2024-04-04T06:08:41.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.184+0000 7f697f400640 1 -- 172.21.15.159:0/61009605 learned_addr learned my addr 172.21.15.159:0/61009605 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:41.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.185+0000 7f6976000640 1 --2- 172.21.15.159:0/61009605 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69800d0990 0x7f69800aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:41.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.185+0000 7f697f400640 1 -- 172.21.15.159:0/61009605 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69800d0990 msgr2=0x7f69800aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:41.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.185+0000 7f697f400640 1 --2- 172.21.15.159:0/61009605 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69800d0990 0x7f69800aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.185+0000 7f697f400640 1 -- 172.21.15.159:0/61009605 >> v1:172.21.15.178:6789/0 conn(0x7f6980153580 legacy=0x7f69800d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:41.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.185+0000 7f697f400640 1 -- 172.21.15.159:0/61009605 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698015cf20 con 0x7f69800af2e0 2024-04-04T06:08:41.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.188+0000 7f697f400640 1 --2- 172.21.15.159:0/61009605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69800af2e0 0x7f69801600d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6978009e50 tx=0x7f697802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48bc22a9cc2012b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:41.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.188+0000 7f697e000640 1 -- 172.21.15.159:0/61009605 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697802ca70 con 0x7f69800af2e0 2024-04-04T06:08:41.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.188+0000 7f697e000640 1 -- 172.21.15.159:0/61009605 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f697802cc10 con 0x7f69800af2e0 2024-04-04T06:08:41.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.188+0000 7f697e000640 1 -- 172.21.15.159:0/61009605 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69780343f0 con 0x7f69800af2e0 2024-04-04T06:08:41.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.189+0000 7f6985600640 1 -- 172.21.15.159:0/61009605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69800af2e0 msgr2=0x7f69801600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:41.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.189+0000 7f6985600640 1 --2- 172.21.15.159:0/61009605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69800af2e0 0x7f69801600d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6978009e50 tx=0x7f697802bed0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.189+0000 7f6985600640 1 -- 172.21.15.159:0/61009605 shutdown_connections 2024-04-04T06:08:41.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.189+0000 7f6985600640 1 --2- 172.21.15.159:0/61009605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69800af2e0 0x7f69801600d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.189+0000 7f6985600640 1 --2- 172.21.15.159:0/61009605 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69800d0990 0x7f69800aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.189+0000 7f6985600640 1 -- 172.21.15.159:0/61009605 >> 172.21.15.159:0/61009605 conn(0x7f698006b750 msgr2=0x7f698006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:41.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.189+0000 7f6985600640 1 -- 172.21.15.159:0/61009605 shutdown_connections 2024-04-04T06:08:41.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.189+0000 7f6985600640 1 -- 172.21.15.159:0/61009605 wait complete. 2024-04-04T06:08:41.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.191+0000 7f6985600640 1 Processor -- start 2024-04-04T06:08:41.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.191+0000 7f6985600640 1 -- start start 2024-04-04T06:08:41.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.192+0000 7f6985600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69800af2e0 0x7f69801fc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:41.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.192+0000 7f6985600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69800d0990 0x7f698015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:41.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.192+0000 7f6985600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6980153580 0x7f698015e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:41.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.192+0000 7f6985600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6980153960 con 0x7f69800af2e0 2024-04-04T06:08:41.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.192+0000 7f6985600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f698015d4e0 con 0x7f6980153580 2024-04-04T06:08:41.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.192+0000 7f6985600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f69800af710 con 0x7f69800d0990 2024-04-04T06:08:41.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.192+0000 7f697ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69800af2e0 0x7f69801fc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:41.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.192+0000 7f697ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69800af2e0 0x7f69801fc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47450/0 (socket says 172.21.15.159:47450) 2024-04-04T06:08:41.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.193+0000 7f697ea00640 1 -- 172.21.15.159:0/3670875859 learned_addr learned my addr 172.21.15.159:0/3670875859 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:41.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.193+0000 7f697f400640 1 --2- 172.21.15.159:0/3670875859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6980153580 0x7f698015e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:41.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.193+0000 7f6976000640 1 --2- 172.21.15.159:0/3670875859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69800d0990 0x7f698015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:41.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.193+0000 7f697ea00640 1 -- 172.21.15.159:0/3670875859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69800d0990 msgr2=0x7f698015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:41.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.193+0000 7f697ea00640 1 --2- 172.21.15.159:0/3670875859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69800d0990 0x7f698015dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.193+0000 7f697ea00640 1 -- 172.21.15.159:0/3670875859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6980153580 msgr2=0x7f698015e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:41.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.193+0000 7f697ea00640 1 --2- 172.21.15.159:0/3670875859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6980153580 0x7f698015e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.193+0000 7f697ea00640 1 -- 172.21.15.159:0/3670875859 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6980155e20 con 0x7f69800af2e0 2024-04-04T06:08:41.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.193+0000 7f697f400640 1 --2- 172.21.15.159:0/3670875859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6980153580 0x7f698015e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:41.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.193+0000 7f697ea00640 1 --2- 172.21.15.159:0/3670875859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69800af2e0 0x7f69801fc850 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f696800c3c0 tx=0x7f696800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:41.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.194+0000 7f6977400640 1 -- 172.21.15.159:0/3670875859 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69680052d0 con 0x7f69800af2e0 2024-04-04T06:08:41.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.194+0000 7f6977400640 1 -- 172.21.15.159:0/3670875859 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6968005490 con 0x7f69800af2e0 2024-04-04T06:08:41.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.194+0000 7f6977400640 1 -- 172.21.15.159:0/3670875859 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6968007260 con 0x7f69800af2e0 2024-04-04T06:08:41.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.194+0000 7f6985600640 1 -- 172.21.15.159:0/3670875859 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6980156100 con 0x7f69800af2e0 2024-04-04T06:08:41.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.194+0000 7f6985600640 1 -- 172.21.15.159:0/3670875859 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69801fc330 con 0x7f69800af2e0 2024-04-04T06:08:41.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.195+0000 7f6985600640 1 -- 172.21.15.159:0/3670875859 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69801fc530 con 0x7f69800af2e0 2024-04-04T06:08:41.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.196+0000 7f6977400640 1 -- 172.21.15.159:0/3670875859 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6968007400 con 0x7f69800af2e0 2024-04-04T06:08:41.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.197+0000 7f6977400640 1 --2- 172.21.15.159:0/3670875859 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f695c06c8f0 0x7f695c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:41.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.197+0000 7f6977400640 1 -- 172.21.15.159:0/3670875859 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f696808b0d0 con 0x7f69800af2e0 2024-04-04T06:08:41.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.197+0000 7f6976000640 1 --2- 172.21.15.159:0/3670875859 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f695c06c8f0 0x7f695c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:41.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.199+0000 7f6976000640 1 --2- 172.21.15.159:0/3670875859 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f695c06c8f0 0x7f695c06edb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6980072bf0 tx=0x7f6970008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:41.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.201+0000 7f6977400640 1 -- 172.21.15.159:0/3670875859 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f69801fc530 con 0x7f69800af2e0 2024-04-04T06:08:41.332 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:08:41.340 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:08:41.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.438+0000 7f6985600640 1 -- 172.21.15.159:0/3670875859 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f698006eca0 con 0x7f69800af2e0 2024-04-04T06:08:41.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.439+0000 7f6977400640 1 -- 172.21.15.159:0/3670875859 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) ==== 100+0+11 (secure 0 0 0) 0x7f6968059420 con 0x7f69800af2e0 2024-04-04T06:08:41.432 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:08:41.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.441+0000 7f6985600640 1 -- 172.21.15.159:0/3670875859 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f695c06c8f0 msgr2=0x7f695c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:41.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.441+0000 7f6985600640 1 --2- 172.21.15.159:0/3670875859 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f695c06c8f0 0x7f695c06edb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6980072bf0 tx=0x7f6970008040 comp rx=0 tx=0).stop 2024-04-04T06:08:41.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.441+0000 7f6985600640 1 -- 172.21.15.159:0/3670875859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69800af2e0 msgr2=0x7f69801fc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:41.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.441+0000 7f6985600640 1 --2- 172.21.15.159:0/3670875859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69800af2e0 0x7f69801fc850 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f696800c3c0 tx=0x7f696800c880 comp rx=0 tx=0).stop 2024-04-04T06:08:41.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.441+0000 7f6985600640 1 -- 172.21.15.159:0/3670875859 shutdown_connections 2024-04-04T06:08:41.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.441+0000 7f6985600640 1 --2- 172.21.15.159:0/3670875859 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f695c06c8f0 0x7f695c06edb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.441+0000 7f6985600640 1 --2- 172.21.15.159:0/3670875859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6980153580 0x7f698015e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.441+0000 7f6985600640 1 --2- 172.21.15.159:0/3670875859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69800d0990 0x7f698015dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.441+0000 7f6985600640 1 --2- 172.21.15.159:0/3670875859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69800af2e0 0x7f69801fc850 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.441+0000 7f6985600640 1 -- 172.21.15.159:0/3670875859 >> 172.21.15.159:0/3670875859 conn(0x7f698006b750 msgr2=0x7f698006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:41.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.441+0000 7f6985600640 1 -- 172.21.15.159:0/3670875859 shutdown_connections 2024-04-04T06:08:41.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.441+0000 7f6985600640 1 -- 172.21.15.159:0/3670875859 wait complete. 2024-04-04T06:08:41.454 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:08:41.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.655+0000 7f58e4400640 1 Processor -- start 2024-04-04T06:08:41.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.656+0000 7f58e4400640 1 -- start start 2024-04-04T06:08:41.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.657+0000 7f58e4400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f58dc15c390 0x7f58dc073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:41.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.657+0000 7f58e4400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58dc0af830 0x7f58dc160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:41.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.657+0000 7f58e4400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f58dc160b80 0x7f58dc0d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:41.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.657+0000 7f58e4400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f58dc15c770 con 0x7f58dc15c390 2024-04-04T06:08:41.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.657+0000 7f58e4400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f58dc15fe80 con 0x7f58dc160b80 2024-04-04T06:08:41.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.657+0000 7f58e4400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f58dc0ad120 con 0x7f58dc0af830 2024-04-04T06:08:41.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.657+0000 7f58e1a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f58dc15c390 0x7f58dc073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:41.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.658+0000 7f58e1a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f58dc15c390 0x7f58dc073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47454/0 (socket says 172.21.15.159:47454) 2024-04-04T06:08:41.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.658+0000 7f58e1a00640 1 -- 172.21.15.159:0/2342345801 learned_addr learned my addr 172.21.15.159:0/2342345801 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:41.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.658+0000 7f58e1000640 1 --2- 172.21.15.159:0/2342345801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58dc0af830 0x7f58dc160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:41.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.658+0000 7f58e2400640 1 --2- 172.21.15.159:0/2342345801 >> v2:172.21.15.178:3300/0 conn(0x7f58dc160b80 0x7f58dc0d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:41.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.658+0000 7f58e1a00640 1 -- 172.21.15.159:0/2342345801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58dc0af830 msgr2=0x7f58dc160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:41.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.658+0000 7f58e1a00640 1 --2- 172.21.15.159:0/2342345801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58dc0af830 0x7f58dc160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.658+0000 7f58e1a00640 1 -- 172.21.15.159:0/2342345801 >> v2:172.21.15.178:3300/0 conn(0x7f58dc160b80 msgr2=0x7f58dc0d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:41.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.658+0000 7f58e1a00640 1 --2- 172.21.15.159:0/2342345801 >> v2:172.21.15.178:3300/0 conn(0x7f58dc160b80 0x7f58dc0d0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.658+0000 7f58e1a00640 1 -- 172.21.15.159:0/2342345801 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc0ad400 con 0x7f58dc15c390 2024-04-04T06:08:41.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.660+0000 7f58e1a00640 1 --2- 172.21.15.159:0/2342345801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58dc15c390 0x7f58dc073790 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f58dc06eaa0 tx=0x7f58d002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b77e6bc0bf3cfb server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:41.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.660+0000 7f58dbe00640 1 -- 172.21.15.159:0/2342345801 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d002ca70 con 0x7f58dc15c390 2024-04-04T06:08:41.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.660+0000 7f58dbe00640 1 -- 172.21.15.159:0/2342345801 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d002cc10 con 0x7f58dc15c390 2024-04-04T06:08:41.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.660+0000 7f58dbe00640 1 -- 172.21.15.159:0/2342345801 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d00343f0 con 0x7f58dc15c390 2024-04-04T06:08:41.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.661+0000 7f58e4400640 1 -- 172.21.15.159:0/2342345801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58dc15c390 msgr2=0x7f58dc073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:41.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.661+0000 7f58e4400640 1 --2- 172.21.15.159:0/2342345801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58dc15c390 0x7f58dc073790 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f58dc06eaa0 tx=0x7f58d002bed0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.661+0000 7f58e4400640 1 -- 172.21.15.159:0/2342345801 shutdown_connections 2024-04-04T06:08:41.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.661+0000 7f58e4400640 1 --2- 172.21.15.159:0/2342345801 >> v2:172.21.15.178:3300/0 conn(0x7f58dc160b80 0x7f58dc0d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.661+0000 7f58e4400640 1 --2- 172.21.15.159:0/2342345801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58dc0af830 0x7f58dc160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.661+0000 7f58e4400640 1 --2- 172.21.15.159:0/2342345801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58dc15c390 0x7f58dc073790 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.661+0000 7f58e4400640 1 -- 172.21.15.159:0/2342345801 >> 172.21.15.159:0/2342345801 conn(0x7f58dc06b990 msgr2=0x7f58dc06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:41.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.661+0000 7f58e4400640 1 -- 172.21.15.159:0/2342345801 shutdown_connections 2024-04-04T06:08:41.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.661+0000 7f58e4400640 1 -- 172.21.15.159:0/2342345801 wait complete. 2024-04-04T06:08:41.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.663+0000 7f58e4400640 1 Processor -- start 2024-04-04T06:08:41.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.664+0000 7f58e4400640 1 -- start start 2024-04-04T06:08:41.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.664+0000 7f58e4400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58dc0af830 0x7f58dc154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:41.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.664+0000 7f58e4400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58dc15c390 0x7f58dc14f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:41.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.664+0000 7f58e4400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58dc160b80 0x7f58dc1511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:41.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.664+0000 7f58e4400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc0cf980 con 0x7f58dc160b80 2024-04-04T06:08:41.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.664+0000 7f58e4400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc160120 con 0x7f58dc0af830 2024-04-04T06:08:41.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.664+0000 7f58e4400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f58dc0d0b90 con 0x7f58dc15c390 2024-04-04T06:08:41.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.665+0000 7f58e2400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58dc160b80 0x7f58dc1511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:41.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.665+0000 7f58e2400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58dc160b80 0x7f58dc1511a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47460/0 (socket says 172.21.15.159:47460) 2024-04-04T06:08:41.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.665+0000 7f58e2400640 1 -- 172.21.15.159:0/2460109038 learned_addr learned my addr 172.21.15.159:0/2460109038 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:41.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.665+0000 7f58e1000640 1 --2- 172.21.15.159:0/2460109038 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58dc15c390 0x7f58dc14f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:41.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.665+0000 7f58e1a00640 1 --2- 172.21.15.159:0/2460109038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58dc0af830 0x7f58dc154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:41.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.665+0000 7f58e2400640 1 -- 172.21.15.159:0/2460109038 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58dc15c390 msgr2=0x7f58dc14f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:41.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.665+0000 7f58e2400640 1 --2- 172.21.15.159:0/2460109038 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58dc15c390 0x7f58dc14f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.665+0000 7f58e2400640 1 -- 172.21.15.159:0/2460109038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58dc0af830 msgr2=0x7f58dc154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:41.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.665+0000 7f58e2400640 1 --2- 172.21.15.159:0/2460109038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58dc0af830 0x7f58dc154570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.665+0000 7f58e2400640 1 -- 172.21.15.159:0/2460109038 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc14f9d0 con 0x7f58dc160b80 2024-04-04T06:08:41.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.665+0000 7f58e2400640 1 --2- 172.21.15.159:0/2460109038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58dc160b80 0x7f58dc1511a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f58d400c540 tx=0x7f58d400ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:41.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.666+0000 7f58d9600640 1 -- 172.21.15.159:0/2460109038 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4015070 con 0x7f58dc160b80 2024-04-04T06:08:41.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.666+0000 7f58d9600640 1 -- 172.21.15.159:0/2460109038 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d4004220 con 0x7f58dc160b80 2024-04-04T06:08:41.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.666+0000 7f58e4400640 1 -- 172.21.15.159:0/2460109038 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58dc150100 con 0x7f58dc160b80 2024-04-04T06:08:41.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.666+0000 7f58d9600640 1 -- 172.21.15.159:0/2460109038 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4003610 con 0x7f58dc160b80 2024-04-04T06:08:41.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.666+0000 7f58e4400640 1 -- 172.21.15.159:0/2460109038 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58dc1506c0 con 0x7f58dc160b80 2024-04-04T06:08:41.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.667+0000 7f58e4400640 1 -- 172.21.15.159:0/2460109038 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58dc0685c0 con 0x7f58dc160b80 2024-04-04T06:08:41.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.668+0000 7f58d9600640 1 -- 172.21.15.159:0/2460109038 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58d40037b0 con 0x7f58dc160b80 2024-04-04T06:08:41.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.670+0000 7f58d9600640 1 --2- 172.21.15.159:0/2460109038 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f58bc06c8f0 0x7f58bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:41.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.670+0000 7f58d9600640 1 -- 172.21.15.159:0/2460109038 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f58d408bc80 con 0x7f58dc160b80 2024-04-04T06:08:41.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.670+0000 7f58e1a00640 1 --2- 172.21.15.159:0/2460109038 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f58bc06c8f0 0x7f58bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:41.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.671+0000 7f58e1a00640 1 --2- 172.21.15.159:0/2460109038 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f58bc06c8f0 0x7f58bc06edb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f58d0035000 tx=0x7f58d0030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:41.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.673+0000 7f58d9600640 1 -- 172.21.15.159:0/2460109038 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f58d4059fd0 con 0x7f58dc160b80 2024-04-04T06:08:41.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.914+0000 7f58e4400640 1 -- 172.21.15.159:0/2460109038 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58dc06f1b0 con 0x7f58dc160b80 2024-04-04T06:08:41.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.915+0000 7f58d9600640 1 -- 172.21.15.159:0/2460109038 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5193 (secure 0 0 0) 0x7f58d40598d0 con 0x7f58dc160b80 2024-04-04T06:08:41.909 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:41.910 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3765578393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3765578393},{"type":"v1","addr":"172.21.15.178:6835","nonce":3765578393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3260412913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3260412913},{"type":"v1","addr":"172.21.15.178:6837","nonce":3260412913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3149916718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3149916718},{"type":"v1","addr":"172.21.15.159:6833","nonce":3149916718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:07:53.523212+0000","modified":"2024-04-04T06:07:56.601750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6835/1867245367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1867245367},{"type":"v1","addr":"172.21.15.159:6835","nonce":1867245367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5589,"qdb_cluster":[5589]},"id":3}]} 2024-04-04T06:08:41.910 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 25 2024-04-04T06:08:41.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.917+0000 7f58e4400640 1 -- 172.21.15.159:0/2460109038 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f58bc06c8f0 msgr2=0x7f58bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:41.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.917+0000 7f58e4400640 1 --2- 172.21.15.159:0/2460109038 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f58bc06c8f0 0x7f58bc06edb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f58d0035000 tx=0x7f58d0030000 comp rx=0 tx=0).stop 2024-04-04T06:08:41.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.917+0000 7f58e4400640 1 -- 172.21.15.159:0/2460109038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58dc160b80 msgr2=0x7f58dc1511a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:41.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.917+0000 7f58e4400640 1 --2- 172.21.15.159:0/2460109038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58dc160b80 0x7f58dc1511a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f58d400c540 tx=0x7f58d400ca00 comp rx=0 tx=0).stop 2024-04-04T06:08:41.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.917+0000 7f58e4400640 1 -- 172.21.15.159:0/2460109038 shutdown_connections 2024-04-04T06:08:41.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.917+0000 7f58e4400640 1 --2- 172.21.15.159:0/2460109038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58dc160b80 0x7f58dc1511a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.917+0000 7f58e4400640 1 --2- 172.21.15.159:0/2460109038 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58dc15c390 0x7f58dc14f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.917+0000 7f58e4400640 1 --2- 172.21.15.159:0/2460109038 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f58bc06c8f0 0x7f58bc06edb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.917+0000 7f58e4400640 1 --2- 172.21.15.159:0/2460109038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58dc0af830 0x7f58dc154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:41.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.917+0000 7f58e4400640 1 -- 172.21.15.159:0/2460109038 >> 172.21.15.159:0/2460109038 conn(0x7f58dc06b990 msgr2=0x7f58dc0b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:41.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.917+0000 7f58e4400640 1 -- 172.21.15.159:0/2460109038 shutdown_connections 2024-04-04T06:08:41.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:41.918+0000 7f58e4400640 1 -- 172.21.15.159:0/2460109038 wait complete. 2024-04-04T06:08:41.930 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:08:42.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.136+0000 7f5d97e00640 1 Processor -- start 2024-04-04T06:08:42.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.137+0000 7f5d97e00640 1 -- start start 2024-04-04T06:08:42.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.138+0000 7f5d97e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5d980cc0b0 0x7f5d9806dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:42.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.138+0000 7f5d97e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d9815fc50 0x7f5d980cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:42.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.138+0000 7f5d97e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5d980cc490 0x7f5d980d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:42.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.138+0000 7f5d97e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f5d9815c610 con 0x7f5d980cc490 2024-04-04T06:08:42.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.138+0000 7f5d97e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f5d9806e4f0 con 0x7f5d980cc0b0 2024-04-04T06:08:42.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.138+0000 7f5d97e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5d980b3150 con 0x7f5d9815fc50 2024-04-04T06:08:42.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.138+0000 7f5d97400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5d980cc490 0x7f5d980d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:42.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.138+0000 7f5d97400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5d980cc490 0x7f5d980d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47474/0 (socket says 172.21.15.159:47474) 2024-04-04T06:08:42.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.138+0000 7f5d97400640 1 -- 172.21.15.159:0/733394298 learned_addr learned my addr 172.21.15.159:0/733394298 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:42.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.138+0000 7f5d96a00640 1 --2- 172.21.15.159:0/733394298 >> v2:172.21.15.178:3300/0 conn(0x7f5d980cc0b0 0x7f5d9806dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:42.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.138+0000 7f5d96000640 1 --2- 172.21.15.159:0/733394298 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d9815fc50 0x7f5d980cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:42.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.139+0000 7f5d97400640 1 -- 172.21.15.159:0/733394298 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d9815fc50 msgr2=0x7f5d980cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:42.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.139+0000 7f5d97400640 1 --2- 172.21.15.159:0/733394298 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d9815fc50 0x7f5d980cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.139+0000 7f5d97400640 1 -- 172.21.15.159:0/733394298 >> v2:172.21.15.178:3300/0 conn(0x7f5d980cc0b0 msgr2=0x7f5d9806dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:42.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.139+0000 7f5d97400640 1 --2- 172.21.15.159:0/733394298 >> v2:172.21.15.178:3300/0 conn(0x7f5d980cc0b0 0x7f5d9806dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.139+0000 7f5d97400640 1 -- 172.21.15.159:0/733394298 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d981600e0 con 0x7f5d980cc490 2024-04-04T06:08:42.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.140+0000 7f5d97400640 1 --2- 172.21.15.159:0/733394298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d980cc490 0x7f5d980d0300 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5d980cfd10 tx=0x7f5d9002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f87992c00b1ebf server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:42.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.140+0000 7f5d95600640 1 -- 172.21.15.159:0/733394298 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90035020 con 0x7f5d980cc490 2024-04-04T06:08:42.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.140+0000 7f5d95600640 1 -- 172.21.15.159:0/733394298 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d90031640 con 0x7f5d980cc490 2024-04-04T06:08:42.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.140+0000 7f5d95600640 1 -- 172.21.15.159:0/733394298 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9002d040 con 0x7f5d980cc490 2024-04-04T06:08:42.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.141+0000 7f5d97e00640 1 -- 172.21.15.159:0/733394298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d980cc490 msgr2=0x7f5d980d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:42.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.141+0000 7f5d97e00640 1 --2- 172.21.15.159:0/733394298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d980cc490 0x7f5d980d0300 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5d980cfd10 tx=0x7f5d9002b8f0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.141+0000 7f5d97e00640 1 -- 172.21.15.159:0/733394298 shutdown_connections 2024-04-04T06:08:42.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.141+0000 7f5d97e00640 1 --2- 172.21.15.159:0/733394298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d980cc490 0x7f5d980d0300 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.141+0000 7f5d97e00640 1 --2- 172.21.15.159:0/733394298 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d9815fc50 0x7f5d980cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.141+0000 7f5d97e00640 1 --2- 172.21.15.159:0/733394298 >> v2:172.21.15.178:3300/0 conn(0x7f5d980cc0b0 0x7f5d9806dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.141+0000 7f5d97e00640 1 -- 172.21.15.159:0/733394298 >> 172.21.15.159:0/733394298 conn(0x7f5d980ccad0 msgr2=0x7f5d980cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:42.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.141+0000 7f5d97e00640 1 -- 172.21.15.159:0/733394298 shutdown_connections 2024-04-04T06:08:42.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.141+0000 7f5d97e00640 1 -- 172.21.15.159:0/733394298 wait complete. 2024-04-04T06:08:42.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.143+0000 7f5d97e00640 1 Processor -- start 2024-04-04T06:08:42.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.143+0000 7f5d97e00640 1 -- start start 2024-04-04T06:08:42.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.144+0000 7f5d97e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d980cc0b0 0x7f5d98152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:42.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.144+0000 7f5d97e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d980cc490 0x7f5d981534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:42.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.144+0000 7f5d97e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d981550a0 0x7f5d98155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:42.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.144+0000 7f5d97e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5d98160980 con 0x7f5d980cc0b0 2024-04-04T06:08:42.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.144+0000 7f5d97e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5d981606a0 con 0x7f5d980cc490 2024-04-04T06:08:42.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.144+0000 7f5d97e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5d9815ca70 con 0x7f5d981550a0 2024-04-04T06:08:42.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.144+0000 7f5d96a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d980cc0b0 0x7f5d98152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:42.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.144+0000 7f5d96a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d980cc0b0 0x7f5d98152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47490/0 (socket says 172.21.15.159:47490) 2024-04-04T06:08:42.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.144+0000 7f5d96a00640 1 -- 172.21.15.159:0/2272603794 learned_addr learned my addr 172.21.15.159:0/2272603794 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:42.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.145+0000 7f5d96a00640 1 -- 172.21.15.159:0/2272603794 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d981550a0 msgr2=0x7f5d98155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:42.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.145+0000 7f5d97400640 1 --2- 172.21.15.159:0/2272603794 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d981550a0 0x7f5d98155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:42.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.145+0000 7f5d96000640 1 --2- 172.21.15.159:0/2272603794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d980cc490 0x7f5d981534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:42.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.145+0000 7f5d96a00640 1 --2- 172.21.15.159:0/2272603794 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d981550a0 0x7f5d98155550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.145+0000 7f5d96a00640 1 -- 172.21.15.159:0/2272603794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d980cc490 msgr2=0x7f5d981534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:42.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.145+0000 7f5d96a00640 1 --2- 172.21.15.159:0/2272603794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d980cc490 0x7f5d981534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.145+0000 7f5d96a00640 1 -- 172.21.15.159:0/2272603794 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9815e550 con 0x7f5d980cc0b0 2024-04-04T06:08:42.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.145+0000 7f5d96000640 1 --2- 172.21.15.159:0/2272603794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d980cc490 0x7f5d981534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:42.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.145+0000 7f5d96a00640 1 --2- 172.21.15.159:0/2272603794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d980cc0b0 0x7f5d98152f90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800efc0 tx=0x7f5d8800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:42.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.145+0000 7f5d8f400640 1 -- 172.21.15.159:0/2272603794 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d88018020 con 0x7f5d980cc0b0 2024-04-04T06:08:42.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.146+0000 7f5d8f400640 1 -- 172.21.15.159:0/2272603794 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d88014480 con 0x7f5d980cc0b0 2024-04-04T06:08:42.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.146+0000 7f5d97e00640 1 -- 172.21.15.159:0/2272603794 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d980b0480 con 0x7f5d980cc0b0 2024-04-04T06:08:42.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.146+0000 7f5d8f400640 1 -- 172.21.15.159:0/2272603794 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d88014850 con 0x7f5d980cc0b0 2024-04-04T06:08:42.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.146+0000 7f5d97e00640 1 -- 172.21.15.159:0/2272603794 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d98155a90 con 0x7f5d980cc0b0 2024-04-04T06:08:42.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.147+0000 7f5d97e00640 1 -- 172.21.15.159:0/2272603794 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d98075940 con 0x7f5d980cc0b0 2024-04-04T06:08:42.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.148+0000 7f5d8f400640 1 -- 172.21.15.159:0/2272603794 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d88005d10 con 0x7f5d980cc0b0 2024-04-04T06:08:42.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.149+0000 7f5d8f400640 1 --2- 172.21.15.159:0/2272603794 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d7806c920 0x7f5d7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:42.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.149+0000 7f5d8f400640 1 -- 172.21.15.159:0/2272603794 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7f5d8808ca00 con 0x7f5d980cc0b0 2024-04-04T06:08:42.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.150+0000 7f5d96000640 1 --2- 172.21.15.159:0/2272603794 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d7806c920 0x7f5d7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:42.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.151+0000 7f5d96000640 1 --2- 172.21.15.159:0/2272603794 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d7806c920 0x7f5d7806ede0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5d980af0b0 tx=0x7f5d84008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:42.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.153+0000 7f5d8f400640 1 -- 172.21.15.159:0/2272603794 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5d88022030 con 0x7f5d980cc0b0 2024-04-04T06:08:42.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.353+0000 7f5d97e00640 1 -- 172.21.15.159:0/2272603794 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d98073830 con 0x7f5d980cc0b0 2024-04-04T06:08:42.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.355+0000 7f5d8f400640 1 -- 172.21.15.159:0/2272603794 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16777 (secure 0 0 0) 0x7f5d88010040 con 0x7f5d980cc0b0 2024-04-04T06:08:42.349 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:42.349 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":48,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:08:34.246895+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:07:51.357366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-04T06:07:52.495884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.30","mappings":[{"from":0,"to":4}]},{"pgid":"7.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:08:42.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.358+0000 7f5d97e00640 1 -- 172.21.15.159:0/2272603794 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d7806c920 msgr2=0x7f5d7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:42.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.358+0000 7f5d97e00640 1 --2- 172.21.15.159:0/2272603794 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d7806c920 0x7f5d7806ede0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5d980af0b0 tx=0x7f5d84008040 comp rx=0 tx=0).stop 2024-04-04T06:08:42.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.358+0000 7f5d97e00640 1 -- 172.21.15.159:0/2272603794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d980cc0b0 msgr2=0x7f5d98152f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:42.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.358+0000 7f5d97e00640 1 --2- 172.21.15.159:0/2272603794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d980cc0b0 0x7f5d98152f90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800efc0 tx=0x7f5d8800c490 comp rx=0 tx=0).stop 2024-04-04T06:08:42.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.358+0000 7f5d97e00640 1 -- 172.21.15.159:0/2272603794 shutdown_connections 2024-04-04T06:08:42.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.358+0000 7f5d97e00640 1 --2- 172.21.15.159:0/2272603794 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d981550a0 0x7f5d98155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.358+0000 7f5d97e00640 1 --2- 172.21.15.159:0/2272603794 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d7806c920 0x7f5d7806ede0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.358+0000 7f5d97e00640 1 --2- 172.21.15.159:0/2272603794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d980cc490 0x7f5d981534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.358+0000 7f5d97e00640 1 --2- 172.21.15.159:0/2272603794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d980cc0b0 0x7f5d98152f90 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.358+0000 7f5d97e00640 1 -- 172.21.15.159:0/2272603794 >> 172.21.15.159:0/2272603794 conn(0x7f5d980ccad0 msgr2=0x7f5d98072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:42.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.358+0000 7f5d97e00640 1 -- 172.21.15.159:0/2272603794 shutdown_connections 2024-04-04T06:08:42.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.358+0000 7f5d97e00640 1 -- 172.21.15.159:0/2272603794 wait complete. 2024-04-04T06:08:42.371 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-04T06:08:42.371 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:08:42.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.573+0000 7fa038a00640 1 Processor -- start 2024-04-04T06:08:42.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.573+0000 7fa038a00640 1 -- start start 2024-04-04T06:08:42.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.575+0000 7fa038a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa0340d1610 0x7fa0340c7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:42.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.575+0000 7fa038a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa03415eae0 con 0x7fa03415e700 2024-04-04T06:08:42.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.575+0000 7fa038a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0340cfeb0 con 0x7fa0340b4530 2024-04-04T06:08:42.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.575+0000 7fa038a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa0340d0190 con 0x7fa0340d1610 2024-04-04T06:08:42.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.575+0000 7fa031e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fa03415e700 0x7fa0340b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:32826/0 (socket says 172.21.15.159:32826) 2024-04-04T06:08:42.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.575+0000 7fa031e00640 1 -- 172.21.15.159:0/3838486745 learned_addr learned my addr 172.21.15.159:0/3838486745 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:42.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.576+0000 7fa031400640 1 -- 172.21.15.159:0/3838486745 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2609511811 0 0) 0x7fa03415eae0 con 0x7fa03415e700 2024-04-04T06:08:42.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.576+0000 7fa031400640 1 -- 172.21.15.159:0/3838486745 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa018003520 con 0x7fa03415e700 2024-04-04T06:08:42.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.576+0000 7fa032800640 1 --2- 172.21.15.159:0/3838486745 >> v2:172.21.15.178:3300/0 conn(0x7fa0340d1610 0x7fa0340c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:42.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.576+0000 7fa031400640 1 -- 172.21.15.159:0/3838486745 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2718032567 0 0) 0x7fa018003520 con 0x7fa03415e700 2024-04-04T06:08:42.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.576+0000 7fa031400640 1 -- 172.21.15.159:0/3838486745 >> v2:172.21.15.178:3300/0 conn(0x7fa0340d1610 msgr2=0x7fa0340c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:42.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.576+0000 7fa031400640 1 --2- 172.21.15.159:0/3838486745 >> v2:172.21.15.178:3300/0 conn(0x7fa0340d1610 0x7fa0340c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.576+0000 7fa031400640 1 -- 172.21.15.159:0/3838486745 >> v1:172.21.15.178:6789/0 conn(0x7fa0340b4530 legacy=0x7fa0340c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:42.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.576+0000 7fa031400640 1 -- 172.21.15.159:0/3838486745 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0340d0470 con 0x7fa03415e700 2024-04-04T06:08:42.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.576+0000 7fa031400640 1 -- 172.21.15.159:0/3838486745 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fa024001670 con 0x7fa03415e700 2024-04-04T06:08:42.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.576+0000 7fa031400640 1 -- 172.21.15.159:0/3838486745 >> v1:172.21.15.159:6789/0 conn(0x7fa03415e700 legacy=0x7fa0340b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:42.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.576+0000 7fa031400640 1 --2- 172.21.15.159:0/3838486745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0180039b0 0x7fa018003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:42.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.576+0000 7fa031400640 1 --2- 172.21.15.159:0/3838486745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa018004800 0x7fa018004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:42.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.576+0000 7fa031400640 1 --2- 172.21.15.159:0/3838486745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0180052e0 0x7fa01800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:42.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.576+0000 7fa031400640 1 -- 172.21.15.159:0/3838486745 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa0340d0190 con 0x7fa018004800 2024-04-04T06:08:42.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.576+0000 7fa031400640 1 -- 172.21.15.159:0/3838486745 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa01800c1b0 con 0x7fa0180052e0 2024-04-04T06:08:42.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.576+0000 7fa031400640 1 -- 172.21.15.159:0/3838486745 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa01800c3b0 con 0x7fa0180039b0 2024-04-04T06:08:42.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.576+0000 7fa02be00640 1 --2- 172.21.15.159:0/3838486745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa018004800 0x7fa018004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:42.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.577+0000 7fa02be00640 1 -- 172.21.15.159:0/3838486745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0180039b0 msgr2=0x7fa018003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:42.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.577+0000 7fa031e00640 1 --2- 172.21.15.159:0/3838486745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0180039b0 0x7fa018003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:42.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.577+0000 7fa032800640 1 --2- 172.21.15.159:0/3838486745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0180052e0 0x7fa01800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:42.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.577+0000 7fa02be00640 1 --2- 172.21.15.159:0/3838486745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0180039b0 0x7fa018003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.577+0000 7fa02be00640 1 -- 172.21.15.159:0/3838486745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0180052e0 msgr2=0x7fa01800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:42.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.577+0000 7fa02be00640 1 --2- 172.21.15.159:0/3838486745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0180052e0 0x7fa01800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.577+0000 7fa02be00640 1 -- 172.21.15.159:0/3838486745 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa018003520 con 0x7fa018004800 2024-04-04T06:08:42.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.577+0000 7fa032800640 1 --2- 172.21.15.159:0/3838486745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0180052e0 0x7fa01800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:42.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.577+0000 7fa032800640 1 -- 172.21.15.159:0/3838486745 reap_dead start 2024-04-04T06:08:42.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.578+0000 7fa02be00640 1 --2- 172.21.15.159:0/3838486745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa018004800 0x7fa018004c80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa01c001720 tx=0x7fa018005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a709ccd84cc41bd server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:42.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.578+0000 7fa031400640 1 -- 172.21.15.159:0/3838486745 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01c001370 con 0x7fa018004800 2024-04-04T06:08:42.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.578+0000 7fa031400640 1 -- 172.21.15.159:0/3838486745 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa01c0052d0 con 0x7fa018004800 2024-04-04T06:08:42.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.578+0000 7fa031400640 1 -- 172.21.15.159:0/3838486745 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01c003830 con 0x7fa018004800 2024-04-04T06:08:42.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.579+0000 7fa038a00640 1 -- 172.21.15.159:0/3838486745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa018004800 msgr2=0x7fa018004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:42.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.579+0000 7fa038a00640 1 --2- 172.21.15.159:0/3838486745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa018004800 0x7fa018004c80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa01c001720 tx=0x7fa018005830 comp rx=0 tx=0).stop 2024-04-04T06:08:42.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.579+0000 7fa038a00640 1 -- 172.21.15.159:0/3838486745 shutdown_connections 2024-04-04T06:08:42.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.579+0000 7fa038a00640 1 --2- 172.21.15.159:0/3838486745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa018004800 0x7fa018004c80 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.579+0000 7fa038a00640 1 -- 172.21.15.159:0/3838486745 >> 172.21.15.159:0/3838486745 conn(0x7fa0340d2240 msgr2=0x7fa0340d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:42.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.579+0000 7fa038a00640 1 -- 172.21.15.159:0/3838486745 shutdown_connections 2024-04-04T06:08:42.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.579+0000 7fa038a00640 1 -- 172.21.15.159:0/3838486745 wait complete. 2024-04-04T06:08:42.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.582+0000 7fa038a00640 1 Processor -- start 2024-04-04T06:08:42.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.582+0000 7fa038a00640 1 -- start start 2024-04-04T06:08:42.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.583+0000 7fa038a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa018004800 0x7fa0340b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:42.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.583+0000 7fa038a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0340c9280 0x7fa0340c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:42.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.583+0000 7fa038a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa034160810 0x7fa034160c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:42.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.583+0000 7fa038a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa03406d1b0 con 0x7fa018004800 2024-04-04T06:08:42.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.583+0000 7fa038a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa0340d0a30 con 0x7fa0340c9280 2024-04-04T06:08:42.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.583+0000 7fa038a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa03406d330 con 0x7fa034160810 2024-04-04T06:08:42.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.584+0000 7fa031e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa018004800 0x7fa0340b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:42.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.584+0000 7fa031e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa018004800 0x7fa0340b3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47498/0 (socket says 172.21.15.159:47498) 2024-04-04T06:08:42.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.584+0000 7fa031e00640 1 -- 172.21.15.159:0/2927040658 learned_addr learned my addr 172.21.15.159:0/2927040658 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:42.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.584+0000 7fa032800640 1 --2- 172.21.15.159:0/2927040658 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa034160810 0x7fa034160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:42.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.584+0000 7fa031e00640 1 -- 172.21.15.159:0/2927040658 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa034160810 msgr2=0x7fa034160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:42.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.584+0000 7fa031e00640 1 --2- 172.21.15.159:0/2927040658 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa034160810 0x7fa034160c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.584+0000 7fa02be00640 1 --2- 172.21.15.159:0/2927040658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0340c9280 0x7fa0340c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:42.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.584+0000 7fa031e00640 1 -- 172.21.15.159:0/2927040658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0340c9280 msgr2=0x7fa0340c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:42.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.584+0000 7fa031e00640 1 --2- 172.21.15.159:0/2927040658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0340c9280 0x7fa0340c9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.584+0000 7fa031e00640 1 -- 172.21.15.159:0/2927040658 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa034161560 con 0x7fa018004800 2024-04-04T06:08:42.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.585+0000 7fa031e00640 1 --2- 172.21.15.159:0/2927040658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa018004800 0x7fa0340b3d90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa02400d7c0 tx=0x7fa02400dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:42.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.585+0000 7fa02a000640 1 -- 172.21.15.159:0/2927040658 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa024019070 con 0x7fa018004800 2024-04-04T06:08:42.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.585+0000 7fa02a000640 1 -- 172.21.15.159:0/2927040658 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0240074e0 con 0x7fa018004800 2024-04-04T06:08:42.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.585+0000 7fa02a000640 1 -- 172.21.15.159:0/2927040658 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0240049e0 con 0x7fa018004800 2024-04-04T06:08:42.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.585+0000 7fa038a00640 1 -- 172.21.15.159:0/2927040658 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0340b5ac0 con 0x7fa018004800 2024-04-04T06:08:42.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.585+0000 7fa038a00640 1 -- 172.21.15.159:0/2927040658 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0340b6080 con 0x7fa018004800 2024-04-04T06:08:42.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.586+0000 7fa038a00640 1 -- 172.21.15.159:0/2927040658 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0340685c0 con 0x7fa018004800 2024-04-04T06:08:42.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.588+0000 7fa02a000640 1 -- 172.21.15.159:0/2927040658 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa024004b80 con 0x7fa018004800 2024-04-04T06:08:42.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.589+0000 7fa02a000640 1 --2- 172.21.15.159:0/2927040658 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa01006c8f0 0x7fa01006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:42.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.589+0000 7fa02a000640 1 -- 172.21.15.159:0/2927040658 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fa02408b600 con 0x7fa018004800 2024-04-04T06:08:42.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.590+0000 7fa02be00640 1 --2- 172.21.15.159:0/2927040658 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa01006c8f0 0x7fa01006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:42.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.591+0000 7fa02be00640 1 --2- 172.21.15.159:0/2927040658 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa01006c8f0 0x7fa01006edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa01c033000 tx=0x7fa01c009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:42.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.594+0000 7fa02a000640 1 -- 172.21.15.159:0/2927040658 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa0240599f0 con 0x7fa018004800 2024-04-04T06:08:42.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.833+0000 7fa038a00640 1 -- 172.21.15.159:0/2927040658 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0340c7100 con 0x7fa018004800 2024-04-04T06:08:42.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.834+0000 7fa02a000640 1 -- 172.21.15.159:0/2927040658 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5193 (secure 0 0 0) 0x7fa0240592f0 con 0x7fa018004800 2024-04-04T06:08:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:42.828 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3765578393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3765578393},{"type":"v1","addr":"172.21.15.178:6835","nonce":3765578393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3260412913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3260412913},{"type":"v1","addr":"172.21.15.178:6837","nonce":3260412913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3149916718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3149916718},{"type":"v1","addr":"172.21.15.159:6833","nonce":3149916718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:07:53.523212+0000","modified":"2024-04-04T06:07:56.601750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6835/1867245367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1867245367},{"type":"v1","addr":"172.21.15.159:6835","nonce":1867245367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5589,"qdb_cluster":[5589]},"id":3}]} 2024-04-04T06:08:42.828 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 25 2024-04-04T06:08:42.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.836+0000 7fa038a00640 1 -- 172.21.15.159:0/2927040658 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa01006c8f0 msgr2=0x7fa01006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:42.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.836+0000 7fa038a00640 1 --2- 172.21.15.159:0/2927040658 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa01006c8f0 0x7fa01006edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa01c033000 tx=0x7fa01c009030 comp rx=0 tx=0).stop 2024-04-04T06:08:42.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.836+0000 7fa038a00640 1 -- 172.21.15.159:0/2927040658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa018004800 msgr2=0x7fa0340b3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:42.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.836+0000 7fa038a00640 1 --2- 172.21.15.159:0/2927040658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa018004800 0x7fa0340b3d90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa02400d7c0 tx=0x7fa02400dc80 comp rx=0 tx=0).stop 2024-04-04T06:08:42.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.837+0000 7fa038a00640 1 -- 172.21.15.159:0/2927040658 shutdown_connections 2024-04-04T06:08:42.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.837+0000 7fa038a00640 1 --2- 172.21.15.159:0/2927040658 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa034160810 0x7fa034160c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.837+0000 7fa038a00640 1 --2- 172.21.15.159:0/2927040658 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa01006c8f0 0x7fa01006edb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.837+0000 7fa038a00640 1 --2- 172.21.15.159:0/2927040658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0340c9280 0x7fa0340c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.837+0000 7fa038a00640 1 --2- 172.21.15.159:0/2927040658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa018004800 0x7fa0340b3d90 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:42.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.837+0000 7fa038a00640 1 -- 172.21.15.159:0/2927040658 >> 172.21.15.159:0/2927040658 conn(0x7fa0340d2240 msgr2=0x7fa0340c7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:42.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.837+0000 7fa038a00640 1 -- 172.21.15.159:0/2927040658 shutdown_connections 2024-04-04T06:08:42.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:42.837+0000 7fa038a00640 1 -- 172.21.15.159:0/2927040658 wait complete. 2024-04-04T06:08:42.848 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:08:43.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.052+0000 7fd2c1000640 1 Processor -- start 2024-04-04T06:08:43.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.052+0000 7fd2c1000640 1 -- start start 2024-04-04T06:08:43.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.053+0000 7fd2c1000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd2bc15e4a0 0x7fd2bc0b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:43.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.053+0000 7fd2c1000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2bc0d2310 0x7fd2bc0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:43.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.054+0000 7fd2c1000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2bc15e880 con 0x7fd2bc0cf530 2024-04-04T06:08:43.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.054+0000 7fd2c1000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd2bc0d34e0 con 0x7fd2bc15e4a0 2024-04-04T06:08:43.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.054+0000 7fd2c1000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd2bc0d28d0 con 0x7fd2bc0d2310 2024-04-04T06:08:43.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.054+0000 7fd2b3e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2bc0d2310 0x7fd2bc0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:43.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.054+0000 7fd2bbe00640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fd2bc0cf530 0x7fd2bc0d3c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.159:57538/0 (socket says 172.21.15.159:57538) 2024-04-04T06:08:43.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.054+0000 7fd2bbe00640 1 -- 172.21.15.159:0/4049880702 learned_addr learned my addr 172.21.15.159:0/4049880702 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:43.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.054+0000 7fd2bb400640 1 --2- 172.21.15.159:0/4049880702 >> v2:172.21.15.178:3300/0 conn(0x7fd2bc15e4a0 0x7fd2bc0b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:43.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.055+0000 7fd2bb400640 1 -- 172.21.15.159:0/4049880702 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2bc0d2310 msgr2=0x7fd2bc0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:43.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.055+0000 7fd2bb400640 1 --2- 172.21.15.159:0/4049880702 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2bc0d2310 0x7fd2bc0ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:43.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.055+0000 7fd2bb400640 1 -- 172.21.15.159:0/4049880702 >> v1:172.21.15.178:6789/0 conn(0x7fd2bc0cf530 legacy=0x7fd2bc0d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:43.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.055+0000 7fd2bb400640 1 -- 172.21.15.159:0/4049880702 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2bc0d2bb0 con 0x7fd2bc15e4a0 2024-04-04T06:08:43.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.056+0000 7fd2bb400640 1 --2- 172.21.15.159:0/4049880702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2bc15e4a0 0x7fd2bc0b9c60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0097a0 tx=0x7fd2ac02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8401878af101a3a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:43.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.057+0000 7fd2baa00640 1 -- 172.21.15.159:0/4049880702 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac035020 con 0x7fd2bc15e4a0 2024-04-04T06:08:43.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.057+0000 7fd2baa00640 1 -- 172.21.15.159:0/4049880702 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2ac031810 con 0x7fd2bc15e4a0 2024-04-04T06:08:43.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.057+0000 7fd2baa00640 1 -- 172.21.15.159:0/4049880702 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac031be0 con 0x7fd2bc15e4a0 2024-04-04T06:08:43.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.057+0000 7fd2c1000640 1 -- 172.21.15.159:0/4049880702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2bc15e4a0 msgr2=0x7fd2bc0b9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:43.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.057+0000 7fd2c1000640 1 --2- 172.21.15.159:0/4049880702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2bc15e4a0 0x7fd2bc0b9c60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0097a0 tx=0x7fd2ac02b8b0 comp rx=0 tx=0).stop 2024-04-04T06:08:43.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.057+0000 7fd2c1000640 1 -- 172.21.15.159:0/4049880702 shutdown_connections 2024-04-04T06:08:43.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.057+0000 7fd2c1000640 1 --2- 172.21.15.159:0/4049880702 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2bc0d2310 0x7fd2bc0ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:43.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.057+0000 7fd2c1000640 1 --2- 172.21.15.159:0/4049880702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2bc15e4a0 0x7fd2bc0b9c60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:43.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.057+0000 7fd2c1000640 1 -- 172.21.15.159:0/4049880702 >> 172.21.15.159:0/4049880702 conn(0x7fd2bc0676d0 msgr2=0x7fd2bc067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:43.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.057+0000 7fd2c1000640 1 -- 172.21.15.159:0/4049880702 shutdown_connections 2024-04-04T06:08:43.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.057+0000 7fd2c1000640 1 -- 172.21.15.159:0/4049880702 wait complete. 2024-04-04T06:08:43.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.059+0000 7fd2c1000640 1 Processor -- start 2024-04-04T06:08:43.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.059+0000 7fd2c1000640 1 -- start start 2024-04-04T06:08:43.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.060+0000 7fd2c1000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2bc0cf530 0x7fd2bc0c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:43.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.060+0000 7fd2c1000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2bc0d2310 0x7fd2bc159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:43.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.060+0000 7fd2c1000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2bc15e4a0 0x7fd2bc1612d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:43.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.060+0000 7fd2c1000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd2bc0d3970 con 0x7fd2bc0cf530 2024-04-04T06:08:43.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.060+0000 7fd2c1000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd2bc0d3170 con 0x7fd2bc0d2310 2024-04-04T06:08:43.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.060+0000 7fd2c1000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd2bc0ba1a0 con 0x7fd2bc15e4a0 2024-04-04T06:08:43.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.060+0000 7fd2bb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2bc0cf530 0x7fd2bc0c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:43.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.060+0000 7fd2bb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2bc0cf530 0x7fd2bc0c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47506/0 (socket says 172.21.15.159:47506) 2024-04-04T06:08:43.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.060+0000 7fd2bb400640 1 -- 172.21.15.159:0/2999863055 learned_addr learned my addr 172.21.15.159:0/2999863055 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:43.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.061+0000 7fd2bbe00640 1 --2- 172.21.15.159:0/2999863055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2bc15e4a0 0x7fd2bc1612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:43.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.061+0000 7fd2b3e00640 1 --2- 172.21.15.159:0/2999863055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2bc0d2310 0x7fd2bc159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:43.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.061+0000 7fd2bb400640 1 -- 172.21.15.159:0/2999863055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2bc15e4a0 msgr2=0x7fd2bc1612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:43.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.061+0000 7fd2bb400640 1 --2- 172.21.15.159:0/2999863055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2bc15e4a0 0x7fd2bc1612d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:43.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.061+0000 7fd2bb400640 1 -- 172.21.15.159:0/2999863055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2bc0d2310 msgr2=0x7fd2bc159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:43.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.061+0000 7fd2bb400640 1 --2- 172.21.15.159:0/2999863055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2bc0d2310 0x7fd2bc159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:43.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.061+0000 7fd2bb400640 1 -- 172.21.15.159:0/2999863055 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2bc15f8a0 con 0x7fd2bc0cf530 2024-04-04T06:08:43.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.061+0000 7fd2bb400640 1 --2- 172.21.15.159:0/2999863055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2bc0cf530 0x7fd2bc0c9fb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac034040 tx=0x7fd2ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:43.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.061+0000 7fd2b3400640 1 -- 172.21.15.159:0/2999863055 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac040070 con 0x7fd2bc0cf530 2024-04-04T06:08:43.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.061+0000 7fd2b3400640 1 -- 172.21.15.159:0/2999863055 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2ac039cf0 con 0x7fd2bc0cf530 2024-04-04T06:08:43.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.061+0000 7fd2b3400640 1 -- 172.21.15.159:0/2999863055 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac035040 con 0x7fd2bc0cf530 2024-04-04T06:08:43.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.061+0000 7fd2c1000640 1 -- 172.21.15.159:0/2999863055 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2bc15fb80 con 0x7fd2bc0cf530 2024-04-04T06:08:43.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.062+0000 7fd2c1000640 1 -- 172.21.15.159:0/2999863055 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2bc0c9a90 con 0x7fd2bc0cf530 2024-04-04T06:08:43.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.063+0000 7fd2c1000640 1 -- 172.21.15.159:0/2999863055 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2bc0c95e0 con 0x7fd2bc0cf530 2024-04-04T06:08:43.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.063+0000 7fd2b3400640 1 -- 172.21.15.159:0/2999863055 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2ac038020 con 0x7fd2bc0cf530 2024-04-04T06:08:43.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.064+0000 7fd2b3400640 1 --2- 172.21.15.159:0/2999863055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd29806c8f0 0x7fd29806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:43.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.064+0000 7fd2b3400640 1 -- 172.21.15.159:0/2999863055 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fd2ac0ad7a0 con 0x7fd2bc0cf530 2024-04-04T06:08:43.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.065+0000 7fd2b3e00640 1 --2- 172.21.15.159:0/2999863055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd29806c8f0 0x7fd29806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:43.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.066+0000 7fd2b3e00640 1 --2- 172.21.15.159:0/2999863055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd29806c8f0 0x7fd29806edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc1550e0 tx=0x7fd2a4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:43.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.069+0000 7fd2b3400640 1 -- 172.21.15.159:0/2999863055 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd2ac07bb90 con 0x7fd2bc0cf530 2024-04-04T06:08:43.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.310+0000 7fd2c1000640 1 -- 172.21.15.159:0/2999863055 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2bc075940 con 0x7fd2bc0cf530 2024-04-04T06:08:43.305 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.311+0000 7fd2b3400640 1 -- 172.21.15.159:0/2999863055 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5193 (secure 0 0 0) 0x7fd2ac07b490 con 0x7fd2bc0cf530 2024-04-04T06:08:43.305 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:43.305 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3765578393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3765578393},{"type":"v1","addr":"172.21.15.178:6835","nonce":3765578393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3260412913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3260412913},{"type":"v1","addr":"172.21.15.178:6837","nonce":3260412913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3149916718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3149916718},{"type":"v1","addr":"172.21.15.159:6833","nonce":3149916718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:07:53.523212+0000","modified":"2024-04-04T06:07:56.601750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6835/1867245367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1867245367},{"type":"v1","addr":"172.21.15.159:6835","nonce":1867245367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5589,"qdb_cluster":[5589]},"id":3}]} 2024-04-04T06:08:43.305 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 25 2024-04-04T06:08:43.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.314+0000 7fd2c1000640 1 -- 172.21.15.159:0/2999863055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd29806c8f0 msgr2=0x7fd29806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:43.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.314+0000 7fd2c1000640 1 --2- 172.21.15.159:0/2999863055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd29806c8f0 0x7fd29806edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc1550e0 tx=0x7fd2a4008040 comp rx=0 tx=0).stop 2024-04-04T06:08:43.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.314+0000 7fd2c1000640 1 -- 172.21.15.159:0/2999863055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2bc0cf530 msgr2=0x7fd2bc0c9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:43.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.314+0000 7fd2c1000640 1 --2- 172.21.15.159:0/2999863055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2bc0cf530 0x7fd2bc0c9fb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac034040 tx=0x7fd2ac0046c0 comp rx=0 tx=0).stop 2024-04-04T06:08:43.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.314+0000 7fd2c1000640 1 -- 172.21.15.159:0/2999863055 shutdown_connections 2024-04-04T06:08:43.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.314+0000 7fd2c1000640 1 --2- 172.21.15.159:0/2999863055 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2bc15e4a0 0x7fd2bc1612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:43.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.314+0000 7fd2c1000640 1 --2- 172.21.15.159:0/2999863055 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd29806c8f0 0x7fd29806edb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:43.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.314+0000 7fd2c1000640 1 --2- 172.21.15.159:0/2999863055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2bc0d2310 0x7fd2bc159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:43.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.314+0000 7fd2c1000640 1 --2- 172.21.15.159:0/2999863055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2bc0cf530 0x7fd2bc0c9fb0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:43.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.314+0000 7fd2c1000640 1 -- 172.21.15.159:0/2999863055 >> 172.21.15.159:0/2999863055 conn(0x7fd2bc0676d0 msgr2=0x7fd2bc15c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:43.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.314+0000 7fd2c1000640 1 -- 172.21.15.159:0/2999863055 shutdown_connections 2024-04-04T06:08:43.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.314+0000 7fd2c1000640 1 -- 172.21.15.159:0/2999863055 wait complete. 2024-04-04T06:08:43.327 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:08:43.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.573+0000 7fd3d0a00640 1 Processor -- start 2024-04-04T06:08:43.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.573+0000 7fd3d0a00640 1 -- start start 2024-04-04T06:08:43.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.575+0000 7fd3d0a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd3cc15e4a0 0x7fd3cc0b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:43.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.575+0000 7fd3d0a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd3cc0cf700 0x7fd3cc0b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:43.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.575+0000 7fd3d0a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3cc15e880 con 0x7fd3cc0d2310 2024-04-04T06:08:43.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.575+0000 7fd3d0a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fd3cc0d34e0 con 0x7fd3cc0cf700 2024-04-04T06:08:43.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.575+0000 7fd3d0a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd3cc0d28d0 con 0x7fd3cc15e4a0 2024-04-04T06:08:43.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.575+0000 7fd3cb200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd3cc0cf700 0x7fd3cc0b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:43.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.575+0000 7fd3cb200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd3cc0cf700 0x7fd3cc0b73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47514/0 (socket says 172.21.15.159:47514) 2024-04-04T06:08:43.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.575+0000 7fd3cb200640 1 -- 172.21.15.159:0/3421425350 learned_addr learned my addr 172.21.15.159:0/3421425350 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:43.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.575+0000 7fd3ca800640 1 --2- 172.21.15.159:0/3421425350 >> v2:172.21.15.178:3300/0 conn(0x7fd3cc15e4a0 0x7fd3cc0b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:43.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.576+0000 7fd3cb200640 1 -- 172.21.15.159:0/3421425350 >> v2:172.21.15.178:3300/0 conn(0x7fd3cc15e4a0 msgr2=0x7fd3cc0b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:43.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.576+0000 7fd3cb200640 1 --2- 172.21.15.159:0/3421425350 >> v2:172.21.15.178:3300/0 conn(0x7fd3cc15e4a0 0x7fd3cc0b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:43.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.576+0000 7fd3cb200640 1 -- 172.21.15.159:0/3421425350 >> v1:172.21.15.159:6789/0 conn(0x7fd3cc0d2310 legacy=0x7fd3cc0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:43.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.576+0000 7fd3cb200640 1 -- 172.21.15.159:0/3421425350 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3cc0d2bb0 con 0x7fd3cc0cf700 2024-04-04T06:08:43.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.576+0000 7fd3c9e00640 1 -- 172.21.15.159:0/3421425350 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3868163306 0 0) 0x7fd3cc15e880 con 0x7fd3cc0d2310 2024-04-04T06:08:43.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.577+0000 7fd3cb200640 1 --2- 172.21.15.159:0/3421425350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3cc0cf700 0x7fd3cc0b73e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0cfb20 tx=0x7fd3c402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c96adb13e851df server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:43.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.577+0000 7fd3c9e00640 1 -- 172.21.15.159:0/3421425350 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4035020 con 0x7fd3cc0cf700 2024-04-04T06:08:43.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.577+0000 7fd3c9e00640 1 -- 172.21.15.159:0/3421425350 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c4031870 con 0x7fd3cc0cf700 2024-04-04T06:08:43.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.577+0000 7fd3c9e00640 1 -- 172.21.15.159:0/3421425350 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4031c40 con 0x7fd3cc0cf700 2024-04-04T06:08:43.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.578+0000 7fd3d0a00640 1 -- 172.21.15.159:0/3421425350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3cc0cf700 msgr2=0x7fd3cc0b73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:43.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.578+0000 7fd3d0a00640 1 --2- 172.21.15.159:0/3421425350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3cc0cf700 0x7fd3cc0b73e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0cfb20 tx=0x7fd3c402b910 comp rx=0 tx=0).stop 2024-04-04T06:08:43.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.578+0000 7fd3d0a00640 1 -- 172.21.15.159:0/3421425350 shutdown_connections 2024-04-04T06:08:43.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.578+0000 7fd3d0a00640 1 --2- 172.21.15.159:0/3421425350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3cc0cf700 0x7fd3cc0b73e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:43.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.578+0000 7fd3d0a00640 1 --2- 172.21.15.159:0/3421425350 >> v2:172.21.15.178:3300/0 conn(0x7fd3cc15e4a0 0x7fd3cc0b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:43.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.578+0000 7fd3d0a00640 1 -- 172.21.15.159:0/3421425350 >> 172.21.15.159:0/3421425350 conn(0x7fd3cc0676d0 msgr2=0x7fd3cc067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:43.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.578+0000 7fd3d0a00640 1 -- 172.21.15.159:0/3421425350 shutdown_connections 2024-04-04T06:08:43.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.578+0000 7fd3d0a00640 1 -- 172.21.15.159:0/3421425350 wait complete. 2024-04-04T06:08:43.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.580+0000 7fd3d0a00640 1 Processor -- start 2024-04-04T06:08:43.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.580+0000 7fd3d0a00640 1 -- start start 2024-04-04T06:08:43.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.582+0000 7fd3d0a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3cc0cf700 0x7fd3cc149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:43.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.582+0000 7fd3d0a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3cc0d2310 0x7fd3cc0cd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:43.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.582+0000 7fd3d0a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3cc15e4a0 0x7fd3cc0c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:43.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.582+0000 7fd3d0a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc0d3970 con 0x7fd3cc0cf700 2024-04-04T06:08:43.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.582+0000 7fd3d0a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc0d3170 con 0x7fd3cc0d2310 2024-04-04T06:08:43.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.582+0000 7fd3d0a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd3cc0ba1a0 con 0x7fd3cc15e4a0 2024-04-04T06:08:43.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.582+0000 7fd3ca800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3cc0cf700 0x7fd3cc149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:43.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.582+0000 7fd3ca800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3cc0cf700 0x7fd3cc149dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47516/0 (socket says 172.21.15.159:47516) 2024-04-04T06:08:43.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.582+0000 7fd3ca800640 1 -- 172.21.15.159:0/3476613893 learned_addr learned my addr 172.21.15.159:0/3476613893 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:43.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.583+0000 7fd3ca800640 1 -- 172.21.15.159:0/3476613893 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3cc15e4a0 msgr2=0x7fd3cc0c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:43.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.583+0000 7fd3ca800640 1 --2- 172.21.15.159:0/3476613893 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3cc15e4a0 0x7fd3cc0c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:43.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.583+0000 7fd3c1e00640 1 --2- 172.21.15.159:0/3476613893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3cc0d2310 0x7fd3cc0cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:43.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.583+0000 7fd3ca800640 1 -- 172.21.15.159:0/3476613893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3cc0d2310 msgr2=0x7fd3cc0cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:43.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.583+0000 7fd3ca800640 1 --2- 172.21.15.159:0/3476613893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3cc0d2310 0x7fd3cc0cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:43.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.583+0000 7fd3ca800640 1 -- 172.21.15.159:0/3476613893 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3cc14a890 con 0x7fd3cc0cf700 2024-04-04T06:08:43.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.583+0000 7fd3c1e00640 1 --2- 172.21.15.159:0/3476613893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3cc0d2310 0x7fd3cc0cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:43.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.583+0000 7fd3ca800640 1 --2- 172.21.15.159:0/3476613893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3cc0cf700 0x7fd3cc149dc0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc007f20 tx=0x7fd3bc00e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:43.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.583+0000 7fd3c3400640 1 -- 172.21.15.159:0/3476613893 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc014620 con 0x7fd3cc0cf700 2024-04-04T06:08:43.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.583+0000 7fd3c3400640 1 -- 172.21.15.159:0/3476613893 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3bc019070 con 0x7fd3cc0cf700 2024-04-04T06:08:43.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.583+0000 7fd3c3400640 1 -- 172.21.15.159:0/3476613893 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc014a00 con 0x7fd3cc0cf700 2024-04-04T06:08:43.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.583+0000 7fd3d0a00640 1 -- 172.21.15.159:0/3476613893 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3cc0ce180 con 0x7fd3cc0cf700 2024-04-04T06:08:43.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.584+0000 7fd3d0a00640 1 -- 172.21.15.159:0/3476613893 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3cc14ae10 con 0x7fd3cc0cf700 2024-04-04T06:08:43.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.585+0000 7fd3d0a00640 1 -- 172.21.15.159:0/3476613893 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3cc075940 con 0x7fd3cc0cf700 2024-04-04T06:08:43.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.585+0000 7fd3c3400640 1 -- 172.21.15.159:0/3476613893 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3bc014ba0 con 0x7fd3cc0cf700 2024-04-04T06:08:43.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.586+0000 7fd3c3400640 1 --2- 172.21.15.159:0/3476613893 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd3a806c8f0 0x7fd3a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:43.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.586+0000 7fd3c3400640 1 -- 172.21.15.159:0/3476613893 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fd3bc08c570 con 0x7fd3cc0cf700 2024-04-04T06:08:43.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.587+0000 7fd3c1e00640 1 --2- 172.21.15.159:0/3476613893 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd3a806c8f0 0x7fd3a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:43.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.588+0000 7fd3c1e00640 1 --2- 172.21.15.159:0/3476613893 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd3a806c8f0 0x7fd3a806edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4007fd0 tx=0x7fd3b4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:43.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.591+0000 7fd3c3400640 1 -- 172.21.15.159:0/3476613893 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd3bc05a8c0 con 0x7fd3cc0cf700 2024-04-04T06:08:43.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.793+0000 7fd3d0a00640 1 -- 172.21.15.159:0/3476613893 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3cc15d890 con 0x7fd3cc0cf700 2024-04-04T06:08:43.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.794+0000 7fd3c3400640 1 -- 172.21.15.159:0/3476613893 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16777 (secure 0 0 0) 0x7fd3bc05a1c0 con 0x7fd3cc0cf700 2024-04-04T06:08:43.788 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:43.789 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":48,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:08:34.246895+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:07:51.357366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-04T06:07:52.495884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.30","mappings":[{"from":0,"to":4}]},{"pgid":"7.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:08:43.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.797+0000 7fd3d0a00640 1 -- 172.21.15.159:0/3476613893 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd3a806c8f0 msgr2=0x7fd3a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:43.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.797+0000 7fd3d0a00640 1 --2- 172.21.15.159:0/3476613893 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd3a806c8f0 0x7fd3a806edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4007fd0 tx=0x7fd3b4009040 comp rx=0 tx=0).stop 2024-04-04T06:08:43.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.797+0000 7fd3d0a00640 1 -- 172.21.15.159:0/3476613893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3cc0cf700 msgr2=0x7fd3cc149dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:43.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.797+0000 7fd3d0a00640 1 --2- 172.21.15.159:0/3476613893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3cc0cf700 0x7fd3cc149dc0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc007f20 tx=0x7fd3bc00e570 comp rx=0 tx=0).stop 2024-04-04T06:08:43.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.797+0000 7fd3d0a00640 1 -- 172.21.15.159:0/3476613893 shutdown_connections 2024-04-04T06:08:43.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.797+0000 7fd3d0a00640 1 --2- 172.21.15.159:0/3476613893 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3cc15e4a0 0x7fd3cc0c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:43.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.797+0000 7fd3d0a00640 1 --2- 172.21.15.159:0/3476613893 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd3a806c8f0 0x7fd3a806edb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:43.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.797+0000 7fd3d0a00640 1 --2- 172.21.15.159:0/3476613893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3cc0d2310 0x7fd3cc0cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:43.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.797+0000 7fd3d0a00640 1 --2- 172.21.15.159:0/3476613893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3cc0cf700 0x7fd3cc149dc0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:43.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.797+0000 7fd3d0a00640 1 -- 172.21.15.159:0/3476613893 >> 172.21.15.159:0/3476613893 conn(0x7fd3cc0676d0 msgr2=0x7fd3cc0c7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:43.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.797+0000 7fd3d0a00640 1 -- 172.21.15.159:0/3476613893 shutdown_connections 2024-04-04T06:08:43.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:43.797+0000 7fd3d0a00640 1 -- 172.21.15.159:0/3476613893 wait complete. 2024-04-04T06:08:43.809 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-04T06:08:44.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.070+0000 7fcb13000640 1 Processor -- start 2024-04-04T06:08:44.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.071+0000 7fcb13000640 1 -- start start 2024-04-04T06:08:44.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.073+0000 7fcb13000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcb0c14ba20 0x7fcb0c0b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:44.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.073+0000 7fcb13000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fcb0c14c1b0 0x7fcb0c166290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:44.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.073+0000 7fcb13000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb0c0b33e0 con 0x7fcb0c0cfda0 2024-04-04T06:08:44.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.073+0000 7fcb13000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fcb0c0cde20 con 0x7fcb0c14c1b0 2024-04-04T06:08:44.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.073+0000 7fcb13000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fcb0c166ae0 con 0x7fcb0c14ba20 2024-04-04T06:08:44.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.073+0000 7fcb11200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fcb0c14c1b0 0x7fcb0c166290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:44.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.073+0000 7fcb12600640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fcb0c0cfda0 0x7fcb0c0d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:60050/0 (socket says 172.21.15.159:60050) 2024-04-04T06:08:44.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.073+0000 7fcb12600640 1 -- 172.21.15.159:0/2493189547 learned_addr learned my addr 172.21.15.159:0/2493189547 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:44.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.073+0000 7fcb11200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fcb0c14c1b0 0x7fcb0c166290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58098/0 (socket says 172.21.15.159:58098) 2024-04-04T06:08:44.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.074+0000 7fcb11c00640 1 --2- 172.21.15.159:0/2493189547 >> v2:172.21.15.178:3300/0 conn(0x7fcb0c14ba20 0x7fcb0c0b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:44.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.074+0000 7fcb11200640 1 -- 172.21.15.159:0/2493189547 >> v2:172.21.15.178:3300/0 conn(0x7fcb0c14ba20 msgr2=0x7fcb0c0b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:44.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.074+0000 7fcb11200640 1 --2- 172.21.15.159:0/2493189547 >> v2:172.21.15.178:3300/0 conn(0x7fcb0c14ba20 0x7fcb0c0b2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:44.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.074+0000 7fcb11200640 1 -- 172.21.15.159:0/2493189547 >> v1:172.21.15.159:6789/0 conn(0x7fcb0c0cfda0 legacy=0x7fcb0c0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:44.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.074+0000 7fcb11200640 1 -- 172.21.15.159:0/2493189547 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0c166dc0 con 0x7fcb0c14c1b0 2024-04-04T06:08:44.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.074+0000 7fcb0be00640 1 -- 172.21.15.159:0/2493189547 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1613540281 0 0) 0x7fcb0c0b33e0 con 0x7fcb0c0cfda0 2024-04-04T06:08:44.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.076+0000 7fcb11200640 1 --2- 172.21.15.159:0/2493189547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb0c14c1b0 0x7fcb0c166290 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c06f160 tx=0x7fcaf802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f0cf91bdd78290 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:44.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.076+0000 7fcb0be00640 1 -- 172.21.15.159:0/2493189547 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8035020 con 0x7fcb0c14c1b0 2024-04-04T06:08:44.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.076+0000 7fcb0be00640 1 -- 172.21.15.159:0/2493189547 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaf8031830 con 0x7fcb0c14c1b0 2024-04-04T06:08:44.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.076+0000 7fcb0be00640 1 -- 172.21.15.159:0/2493189547 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8031c00 con 0x7fcb0c14c1b0 2024-04-04T06:08:44.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.076+0000 7fcb13000640 1 -- 172.21.15.159:0/2493189547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb0c14c1b0 msgr2=0x7fcb0c166290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:44.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.076+0000 7fcb13000640 1 --2- 172.21.15.159:0/2493189547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb0c14c1b0 0x7fcb0c166290 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c06f160 tx=0x7fcaf802b8d0 comp rx=0 tx=0).stop 2024-04-04T06:08:44.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.077+0000 7fcb13000640 1 -- 172.21.15.159:0/2493189547 shutdown_connections 2024-04-04T06:08:44.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.077+0000 7fcb13000640 1 --2- 172.21.15.159:0/2493189547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb0c14c1b0 0x7fcb0c166290 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:44.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.077+0000 7fcb13000640 1 --2- 172.21.15.159:0/2493189547 >> v2:172.21.15.178:3300/0 conn(0x7fcb0c14ba20 0x7fcb0c0b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:44.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.077+0000 7fcb13000640 1 -- 172.21.15.159:0/2493189547 >> 172.21.15.159:0/2493189547 conn(0x7fcb0c0ccad0 msgr2=0x7fcb0c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:44.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.077+0000 7fcb13000640 1 -- 172.21.15.159:0/2493189547 shutdown_connections 2024-04-04T06:08:44.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.077+0000 7fcb13000640 1 -- 172.21.15.159:0/2493189547 wait complete. 2024-04-04T06:08:44.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.078+0000 7fcb13000640 1 Processor -- start 2024-04-04T06:08:44.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.079+0000 7fcb13000640 1 -- start start 2024-04-04T06:08:44.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.079+0000 7fcb13000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb0c0cfda0 0x7fcb0c13b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:44.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.079+0000 7fcb13000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb0c14ba20 0x7fcb0c1f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:44.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.080+0000 7fcb13000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb0c14c1b0 0x7fcb0c1f8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:44.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.080+0000 7fcb13000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c1667d0 con 0x7fcb0c14c1b0 2024-04-04T06:08:44.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.080+0000 7fcb13000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c0ce0c0 con 0x7fcb0c0cfda0 2024-04-04T06:08:44.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.080+0000 7fcb13000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcb0c14be00 con 0x7fcb0c14ba20 2024-04-04T06:08:44.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.080+0000 7fcb12600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb0c14c1b0 0x7fcb0c1f8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:44.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.080+0000 7fcb12600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb0c14c1b0 0x7fcb0c1f8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58114/0 (socket says 172.21.15.159:58114) 2024-04-04T06:08:44.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.080+0000 7fcb12600640 1 -- 172.21.15.159:0/3943556478 learned_addr learned my addr 172.21.15.159:0/3943556478 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.080+0000 7fcb11c00640 1 --2- 172.21.15.159:0/3943556478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb0c0cfda0 0x7fcb0c13b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.080+0000 7fcb11200640 1 --2- 172.21.15.159:0/3943556478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb0c14ba20 0x7fcb0c1f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.080+0000 7fcb12600640 1 -- 172.21.15.159:0/3943556478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb0c14ba20 msgr2=0x7fcb0c1f86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.080+0000 7fcb12600640 1 --2- 172.21.15.159:0/3943556478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb0c14ba20 0x7fcb0c1f86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.080+0000 7fcb12600640 1 -- 172.21.15.159:0/3943556478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb0c0cfda0 msgr2=0x7fcb0c13b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.080+0000 7fcb12600640 1 --2- 172.21.15.159:0/3943556478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb0c0cfda0 0x7fcb0c13b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.080+0000 7fcb12600640 1 -- 172.21.15.159:0/3943556478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0c13bde0 con 0x7fcb0c14c1b0 2024-04-04T06:08:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.081+0000 7fcb12600640 1 --2- 172.21.15.159:0/3943556478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb0c14c1b0 0x7fcb0c1f8c10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400c7a0 tx=0x7fcb0400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.081+0000 7fcb09600640 1 -- 172.21.15.159:0/3943556478 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04018070 con 0x7fcb0c14c1b0 2024-04-04T06:08:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.081+0000 7fcb09600640 1 -- 172.21.15.159:0/3943556478 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb0400a640 con 0x7fcb0c14c1b0 2024-04-04T06:08:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.081+0000 7fcb09600640 1 -- 172.21.15.159:0/3943556478 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0400aab0 con 0x7fcb0c14c1b0 2024-04-04T06:08:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.081+0000 7fcb13000640 1 -- 172.21.15.159:0/3943556478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0c13b170 con 0x7fcb0c14c1b0 2024-04-04T06:08:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.081+0000 7fcb13000640 1 -- 172.21.15.159:0/3943556478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb0c1f9570 con 0x7fcb0c14c1b0 2024-04-04T06:08:44.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.082+0000 7fcb13000640 1 -- 172.21.15.159:0/3943556478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb0c075940 con 0x7fcb0c14c1b0 2024-04-04T06:08:44.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.083+0000 7fcb09600640 1 -- 172.21.15.159:0/3943556478 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb0400ac50 con 0x7fcb0c14c1b0 2024-04-04T06:08:44.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.084+0000 7fcb09600640 1 --2- 172.21.15.159:0/3943556478 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcaec06c8f0 0x7fcaec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:44.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.084+0000 7fcb09600640 1 -- 172.21.15.159:0/3943556478 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5767+0+0 (secure 0 0 0) 0x7fcb0408b2a0 con 0x7fcb0c14c1b0 2024-04-04T06:08:44.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.084+0000 7fcb11c00640 1 --2- 172.21.15.159:0/3943556478 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcaec06c8f0 0x7fcaec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:44.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.086+0000 7fcb11c00640 1 --2- 172.21.15.159:0/3943556478 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcaec06c8f0 0x7fcaec06edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcb00006fd0 tx=0x7fcb00008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:44.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.088+0000 7fcb09600640 1 -- 172.21.15.159:0/3943556478 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcb0c13bde0 con 0x7fcb0c14c1b0 2024-04-04T06:08:44.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.431+0000 7fcb13000640 1 -- 172.21.15.159:0/3943556478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcb0c162a80 con 0x7fcb0c14c1b0 2024-04-04T06:08:44.426 INFO:tasks.ceph.mon.a.smithi159.stderr:2024-04-04T06:08:44.432+0000 7f1995c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-04T06:08:44.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.435+0000 7fcb09600640 1 -- 172.21.15.159:0/3943556478 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7fcb040595f0 con 0x7fcb0c14c1b0 2024-04-04T06:08:44.429 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-04T06:08:44.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.439+0000 7fcb13000640 1 -- 172.21.15.159:0/3943556478 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcaec06c8f0 msgr2=0x7fcaec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:44.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.439+0000 7fcb13000640 1 --2- 172.21.15.159:0/3943556478 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcaec06c8f0 0x7fcaec06edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcb00006fd0 tx=0x7fcb00008040 comp rx=0 tx=0).stop 2024-04-04T06:08:44.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.439+0000 7fcb13000640 1 -- 172.21.15.159:0/3943556478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb0c14c1b0 msgr2=0x7fcb0c1f8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:44.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.439+0000 7fcb13000640 1 --2- 172.21.15.159:0/3943556478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb0c14c1b0 0x7fcb0c1f8c10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400c7a0 tx=0x7fcb0400cc60 comp rx=0 tx=0).stop 2024-04-04T06:08:44.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.439+0000 7fcb13000640 1 -- 172.21.15.159:0/3943556478 shutdown_connections 2024-04-04T06:08:44.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.439+0000 7fcb13000640 1 --2- 172.21.15.159:0/3943556478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb0c14c1b0 0x7fcb0c1f8c10 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:44.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.439+0000 7fcb13000640 1 --2- 172.21.15.159:0/3943556478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb0c14ba20 0x7fcb0c1f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:44.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.439+0000 7fcb13000640 1 --2- 172.21.15.159:0/3943556478 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcaec06c8f0 0x7fcaec06edb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:44.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.439+0000 7fcb13000640 1 --2- 172.21.15.159:0/3943556478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb0c0cfda0 0x7fcb0c13b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:44.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.439+0000 7fcb13000640 1 -- 172.21.15.159:0/3943556478 >> 172.21.15.159:0/3943556478 conn(0x7fcb0c0ccad0 msgr2=0x7fcb0c0b2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:44.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.439+0000 7fcb13000640 1 -- 172.21.15.159:0/3943556478 shutdown_connections 2024-04-04T06:08:44.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.439+0000 7fcb13000640 1 -- 172.21.15.159:0/3943556478 wait complete. 2024-04-04T06:08:44.443 INFO:tasks.ceph.mds.a.smithi159.stderr: -5570> 2024-04-04T06:07:58.450+0000 7f0f66600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:08:44.447 INFO:tasks.ceph.mds.a.smithi159.stderr: -3227> 2024-04-04T06:08:04.420+0000 7f0f61600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5751=0-4194304@1} caps={5751=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5751 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x5623139ecc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-04T06:08:44.451 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-04T06:08:44.497 INFO:tasks.ceph.mds.a.smithi159.stdout:starting mds.a at 2024-04-04T06:08:44.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.636+0000 7fc55ca00640 1 Processor -- start 2024-04-04T06:08:44.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.636+0000 7fc55ca00640 1 -- start start 2024-04-04T06:08:44.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.637+0000 7fc55ca00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5580d0990 0x7fc5580aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:44.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.638+0000 7fc55ca00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5580b40b0 con 0x7fc558155930 2024-04-04T06:08:44.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.638+0000 7fc55ca00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5580b4390 con 0x7fc558155550 2024-04-04T06:08:44.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.638+0000 7fc55ca00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc5580b4670 con 0x7fc5580d0990 2024-04-04T06:08:44.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.638+0000 7fc555400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fc558155930 0x7fc5580d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:60068/0 (socket says 172.21.15.159:60068) 2024-04-04T06:08:44.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.638+0000 7fc555400640 1 -- 172.21.15.159:0/1066268776 learned_addr learned my addr 172.21.15.159:0/1066268776 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:44.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.638+0000 7fc556800640 1 --2- 172.21.15.159:0/1066268776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5580d0990 0x7fc5580aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:44.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.638+0000 7fc554a00640 1 -- 172.21.15.159:0/1066268776 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2432844235 0 0) 0x7fc5580b40b0 con 0x7fc558155930 2024-04-04T06:08:44.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.638+0000 7fc554a00640 1 -- 172.21.15.159:0/1066268776 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc53c003520 con 0x7fc558155930 2024-04-04T06:08:44.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.639+0000 7fc554a00640 1 -- 172.21.15.159:0/1066268776 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1344921372 0 0) 0x7fc53c003520 con 0x7fc558155930 2024-04-04T06:08:44.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.639+0000 7fc554a00640 1 -- 172.21.15.159:0/1066268776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5580d0990 msgr2=0x7fc5580aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:44.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.639+0000 7fc554a00640 1 --2- 172.21.15.159:0/1066268776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5580d0990 0x7fc5580aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:44.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.639+0000 7fc554a00640 1 -- 172.21.15.159:0/1066268776 >> v1:172.21.15.178:6789/0 conn(0x7fc558155550 legacy=0x7fc55815f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:44.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.639+0000 7fc554a00640 1 -- 172.21.15.159:0/1066268776 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5580b4950 con 0x7fc558155930 2024-04-04T06:08:44.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.639+0000 7fc554a00640 1 -- 172.21.15.159:0/1066268776 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fc544001670 con 0x7fc558155930 2024-04-04T06:08:44.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.639+0000 7fc554a00640 1 -- 172.21.15.159:0/1066268776 >> v1:172.21.15.159:6789/0 conn(0x7fc558155930 legacy=0x7fc5580d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:44.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.639+0000 7fc554a00640 1 --2- 172.21.15.159:0/1066268776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc53c0039e0 0x7fc53c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:44.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.639+0000 7fc554a00640 1 --2- 172.21.15.159:0/1066268776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc53c004800 0x7fc53c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:44.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.639+0000 7fc554a00640 1 --2- 172.21.15.159:0/1066268776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5580d0990 0x7fc53c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:44.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.639+0000 7fc554a00640 1 -- 172.21.15.159:0/1066268776 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc5580b4670 con 0x7fc53c0039e0 2024-04-04T06:08:44.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.639+0000 7fc554a00640 1 -- 172.21.15.159:0/1066268776 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c00bad0 con 0x7fc53c004800 2024-04-04T06:08:44.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.639+0000 7fc554a00640 1 -- 172.21.15.159:0/1066268776 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc53c00bcd0 con 0x7fc5580d0990 2024-04-04T06:08:44.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.639+0000 7fc555e00640 1 --2- 172.21.15.159:0/1066268776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc53c0039e0 0x7fc53c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:44.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.639+0000 7fc555400640 1 --2- 172.21.15.159:0/1066268776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc53c004800 0x7fc53c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:44.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.639+0000 7fc556800640 1 --2- 172.21.15.159:0/1066268776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5580d0990 0x7fc53c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:44.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.639+0000 7fc555e00640 1 -- 172.21.15.159:0/1066268776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5580d0990 msgr2=0x7fc53c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:44.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.639+0000 7fc555e00640 1 --2- 172.21.15.159:0/1066268776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5580d0990 0x7fc53c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:44.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.639+0000 7fc555e00640 1 -- 172.21.15.159:0/1066268776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc53c004800 msgr2=0x7fc53c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:44.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.639+0000 7fc555e00640 1 --2- 172.21.15.159:0/1066268776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc53c004800 0x7fc53c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:44.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.639+0000 7fc555e00640 1 -- 172.21.15.159:0/1066268776 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53c003520 con 0x7fc53c0039e0 2024-04-04T06:08:44.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.640+0000 7fc555400640 1 --2- 172.21.15.159:0/1066268776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc53c004800 0x7fc53c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:44.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.640+0000 7fc556800640 1 --2- 172.21.15.159:0/1066268776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc5580d0990 0x7fc53c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:44.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.641+0000 7fc555e00640 1 --2- 172.21.15.159:0/1066268776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc53c0039e0 0x7fc53c003e00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc548001720 tx=0x7fc53c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515eaf557918c947 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:44.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.641+0000 7fc554a00640 1 -- 172.21.15.159:0/1066268776 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc548001370 con 0x7fc53c0039e0 2024-04-04T06:08:44.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.641+0000 7fc554a00640 1 -- 172.21.15.159:0/1066268776 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5480052d0 con 0x7fc53c0039e0 2024-04-04T06:08:44.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.641+0000 7fc554a00640 1 -- 172.21.15.159:0/1066268776 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc548003830 con 0x7fc53c0039e0 2024-04-04T06:08:44.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.641+0000 7fc55ca00640 1 -- 172.21.15.159:0/1066268776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc53c0039e0 msgr2=0x7fc53c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:44.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.641+0000 7fc55ca00640 1 --2- 172.21.15.159:0/1066268776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc53c0039e0 0x7fc53c003e00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc548001720 tx=0x7fc53c005830 comp rx=0 tx=0).stop 2024-04-04T06:08:44.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.641+0000 7fc556800640 1 -- 172.21.15.159:0/1066268776 reap_dead start 2024-04-04T06:08:44.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.641+0000 7fc55ca00640 1 -- 172.21.15.159:0/1066268776 shutdown_connections 2024-04-04T06:08:44.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.641+0000 7fc55ca00640 1 -- 172.21.15.159:0/1066268776 >> 172.21.15.159:0/1066268776 conn(0x7fc55806b750 msgr2=0x7fc55806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:44.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.642+0000 7fc55ca00640 1 -- 172.21.15.159:0/1066268776 shutdown_connections 2024-04-04T06:08:44.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.642+0000 7fc55ca00640 1 -- 172.21.15.159:0/1066268776 wait complete. 2024-04-04T06:08:44.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.643+0000 7fc55ca00640 1 Processor -- start 2024-04-04T06:08:44.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.643+0000 7fc55ca00640 1 -- start start 2024-04-04T06:08:44.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.644+0000 7fc55ca00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc53c0039e0 0x7fc558099900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:44.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.644+0000 7fc55ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc558071780 0x7fc5581f7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:44.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.644+0000 7fc55ca00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc558152430 0x7fc5581f7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:44.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.644+0000 7fc55ca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc55815bba0 con 0x7fc558071780 2024-04-04T06:08:44.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.644+0000 7fc55ca00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc55815ba20 con 0x7fc558152430 2024-04-04T06:08:44.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.644+0000 7fc55ca00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc5580b3cf0 con 0x7fc53c0039e0 2024-04-04T06:08:44.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.644+0000 7fc555400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc558071780 0x7fc5581f7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:44.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.645+0000 7fc555400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc558071780 0x7fc5581f7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58138/0 (socket says 172.21.15.159:58138) 2024-04-04T06:08:44.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.645+0000 7fc555400640 1 -- 172.21.15.159:0/2441402265 learned_addr learned my addr 172.21.15.159:0/2441402265 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:44.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.645+0000 7fc555400640 1 -- 172.21.15.159:0/2441402265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc53c0039e0 msgr2=0x7fc558099900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:44.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.645+0000 7fc555400640 1 --2- 172.21.15.159:0/2441402265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc53c0039e0 0x7fc558099900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:44.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.645+0000 7fc555400640 1 -- 172.21.15.159:0/2441402265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc558152430 msgr2=0x7fc5581f7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:44.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.645+0000 7fc555400640 1 --2- 172.21.15.159:0/2441402265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc558152430 0x7fc5581f7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:44.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.645+0000 7fc555400640 1 -- 172.21.15.159:0/2441402265 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc55809ad40 con 0x7fc558071780 2024-04-04T06:08:44.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.645+0000 7fc555400640 1 --2- 172.21.15.159:0/2441402265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc558071780 0x7fc5581f7750 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc54400d7a0 tx=0x7fc54400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:44.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.645+0000 7fc54e000640 1 -- 172.21.15.159:0/2441402265 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544019070 con 0x7fc558071780 2024-04-04T06:08:44.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.645+0000 7fc54e000640 1 -- 172.21.15.159:0/2441402265 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc54400de50 con 0x7fc558071780 2024-04-04T06:08:44.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.645+0000 7fc54e000640 1 -- 172.21.15.159:0/2441402265 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5440047e0 con 0x7fc558071780 2024-04-04T06:08:44.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.645+0000 7fc55ca00640 1 -- 172.21.15.159:0/2441402265 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5581f8420 con 0x7fc558071780 2024-04-04T06:08:44.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.646+0000 7fc55ca00640 1 -- 172.21.15.159:0/2441402265 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc558099e40 con 0x7fc558071780 2024-04-04T06:08:44.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.646+0000 7fc55ca00640 1 -- 172.21.15.159:0/2441402265 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5580685c0 con 0x7fc558071780 2024-04-04T06:08:44.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.647+0000 7fc54e000640 1 -- 172.21.15.159:0/2441402265 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc54401c070 con 0x7fc558071780 2024-04-04T06:08:44.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.648+0000 7fc54e000640 1 --2- 172.21.15.159:0/2441402265 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc53806c8f0 0x7fc53806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:44.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.648+0000 7fc54e000640 1 -- 172.21.15.159:0/2441402265 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5853+0+0 (secure 0 0 0) 0x7fc54408b6c0 con 0x7fc558071780 2024-04-04T06:08:44.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.649+0000 7fc555e00640 1 --2- 172.21.15.159:0/2441402265 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc53806c8f0 0x7fc53806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:44.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.650+0000 7fc555e00640 1 --2- 172.21.15.159:0/2441402265 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc53806c8f0 0x7fc53806edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc548033000 tx=0x7fc54802f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:44.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.653+0000 7fc54e000640 1 -- 172.21.15.159:0/2441402265 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc5440599c0 con 0x7fc558071780 2024-04-04T06:08:44.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:44.877+0000 7fc55ca00640 1 -- 172.21.15.159:0/2441402265 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc558144d70 con 0x7fc558071780 2024-04-04T06:08:45.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.438+0000 7fc54e000640 1 -- 172.21.15.159:0/2441402265 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7fc5440592c0 con 0x7fc558071780 2024-04-04T06:08:45.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.442+0000 7fc55ca00640 1 -- 172.21.15.159:0/2441402265 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc53806c8f0 msgr2=0x7fc53806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:45.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.442+0000 7fc55ca00640 1 --2- 172.21.15.159:0/2441402265 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc53806c8f0 0x7fc53806edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc548033000 tx=0x7fc54802f000 comp rx=0 tx=0).stop 2024-04-04T06:08:45.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.442+0000 7fc55ca00640 1 -- 172.21.15.159:0/2441402265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc558071780 msgr2=0x7fc5581f7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:45.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.442+0000 7fc55ca00640 1 --2- 172.21.15.159:0/2441402265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc558071780 0x7fc5581f7750 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc54400d7a0 tx=0x7fc54400dc60 comp rx=0 tx=0).stop 2024-04-04T06:08:45.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.442+0000 7fc55ca00640 1 -- 172.21.15.159:0/2441402265 shutdown_connections 2024-04-04T06:08:45.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.442+0000 7fc55ca00640 1 --2- 172.21.15.159:0/2441402265 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc53806c8f0 0x7fc53806edb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:45.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.442+0000 7fc55ca00640 1 --2- 172.21.15.159:0/2441402265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc558152430 0x7fc5581f7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:45.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.442+0000 7fc55ca00640 1 --2- 172.21.15.159:0/2441402265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc558071780 0x7fc5581f7750 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:45.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.442+0000 7fc55ca00640 1 --2- 172.21.15.159:0/2441402265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc53c0039e0 0x7fc558099900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:45.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.442+0000 7fc55ca00640 1 -- 172.21.15.159:0/2441402265 >> 172.21.15.159:0/2441402265 conn(0x7fc55806b750 msgr2=0x7fc55815d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:45.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.442+0000 7fc55ca00640 1 -- 172.21.15.159:0/2441402265 shutdown_connections 2024-04-04T06:08:45.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.442+0000 7fc55ca00640 1 -- 172.21.15.159:0/2441402265 wait complete. 2024-04-04T06:08:45.456 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-04T06:08:45.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.713+0000 7f20cfe00640 1 Processor -- start 2024-04-04T06:08:45.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.713+0000 7f20cfe00640 1 -- start start 2024-04-04T06:08:45.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.715+0000 7f20cfe00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f20d0141890 0x7f20d0141d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:45.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.715+0000 7f20cfe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f20d006db90 0x7f20d006dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:45.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.715+0000 7f20cfe00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20d00cfbf0 con 0x7f20d013f190 2024-04-04T06:08:45.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.715+0000 7f20cfe00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f20d013f570 con 0x7f20d006db90 2024-04-04T06:08:45.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.715+0000 7f20cfe00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f20d00cf350 con 0x7f20d0141890 2024-04-04T06:08:45.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.715+0000 7f20cea00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f20d013f190 0x7f20d00ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:60072/0 (socket says 172.21.15.159:60072) 2024-04-04T06:08:45.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.715+0000 7f20cea00640 1 -- 172.21.15.159:0/144563309 learned_addr learned my addr 172.21.15.159:0/144563309 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:45.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.715+0000 7f20cf400640 1 --2- 172.21.15.159:0/144563309 >> v2:172.21.15.159:3300/0 conn(0x7f20d006db90 0x7f20d006dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:45.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.715+0000 7f20c6000640 1 --2- 172.21.15.159:0/144563309 >> v2:172.21.15.178:3300/0 conn(0x7f20d0141890 0x7f20d0141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:45.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.716+0000 7f20ce000640 1 -- 172.21.15.159:0/144563309 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 349376916 0 0) 0x7f20d00cfbf0 con 0x7f20d013f190 2024-04-04T06:08:45.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.716+0000 7f20ce000640 1 -- 172.21.15.159:0/144563309 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20b4003520 con 0x7f20d013f190 2024-04-04T06:08:45.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.716+0000 7f20cf400640 1 -- 172.21.15.159:0/144563309 >> v2:172.21.15.178:3300/0 conn(0x7f20d0141890 msgr2=0x7f20d0141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:45.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.716+0000 7f20cf400640 1 --2- 172.21.15.159:0/144563309 >> v2:172.21.15.178:3300/0 conn(0x7f20d0141890 0x7f20d0141d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:45.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.716+0000 7f20cf400640 1 -- 172.21.15.159:0/144563309 >> v1:172.21.15.159:6789/0 conn(0x7f20d013f190 legacy=0x7f20d00ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:45.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.716+0000 7f20cf400640 1 -- 172.21.15.159:0/144563309 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d00cf630 con 0x7f20d006db90 2024-04-04T06:08:45.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.717+0000 7f20cf400640 1 --2- 172.21.15.159:0/144563309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20d006db90 0x7f20d006dfa0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f20d006f110 tx=0x7f20c802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931fb303bfc76f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:45.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.717+0000 7f20ce000640 1 -- 172.21.15.159:0/144563309 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8035020 con 0x7f20d006db90 2024-04-04T06:08:45.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.717+0000 7f20ce000640 1 -- 172.21.15.159:0/144563309 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c8031830 con 0x7f20d006db90 2024-04-04T06:08:45.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.717+0000 7f20ce000640 1 -- 172.21.15.159:0/144563309 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8031be0 con 0x7f20d006db90 2024-04-04T06:08:45.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.718+0000 7f20cfe00640 1 -- 172.21.15.159:0/144563309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20d006db90 msgr2=0x7f20d006dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:45.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.718+0000 7f20cfe00640 1 --2- 172.21.15.159:0/144563309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20d006db90 0x7f20d006dfa0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f20d006f110 tx=0x7f20c802b8d0 comp rx=0 tx=0).stop 2024-04-04T06:08:45.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.718+0000 7f20cfe00640 1 -- 172.21.15.159:0/144563309 shutdown_connections 2024-04-04T06:08:45.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.718+0000 7f20cfe00640 1 --2- 172.21.15.159:0/144563309 >> v2:172.21.15.178:3300/0 conn(0x7f20d0141890 0x7f20d0141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:45.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.718+0000 7f20cfe00640 1 --2- 172.21.15.159:0/144563309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20d006db90 0x7f20d006dfa0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:45.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.718+0000 7f20cfe00640 1 -- 172.21.15.159:0/144563309 >> 172.21.15.159:0/144563309 conn(0x7f20d00b1d80 msgr2=0x7f20d00b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:45.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.718+0000 7f20cfe00640 1 -- 172.21.15.159:0/144563309 shutdown_connections 2024-04-04T06:08:45.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.718+0000 7f20cfe00640 1 -- 172.21.15.159:0/144563309 wait complete. 2024-04-04T06:08:45.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.720+0000 7f20cfe00640 1 Processor -- start 2024-04-04T06:08:45.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.720+0000 7f20cfe00640 1 -- start start 2024-04-04T06:08:45.716 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.721+0000 7f20cfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20d006db90 0x7f20d0072020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:45.716 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.721+0000 7f20cfe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20d013f190 0x7f20d0138b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:45.716 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.721+0000 7f20cfe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20d0141890 0x7f20d006e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:45.716 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.721+0000 7f20cfe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f20d0142550 con 0x7f20d006db90 2024-04-04T06:08:45.716 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.721+0000 7f20cfe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f20d006d670 con 0x7f20d0141890 2024-04-04T06:08:45.716 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.721+0000 7f20cfe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f20d01426d0 con 0x7f20d013f190 2024-04-04T06:08:45.716 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.721+0000 7f20cea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20d006db90 0x7f20d0072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:45.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.721+0000 7f20cea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20d006db90 0x7f20d0072020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58162/0 (socket says 172.21.15.159:58162) 2024-04-04T06:08:45.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.721+0000 7f20cea00640 1 -- 172.21.15.159:0/350715238 learned_addr learned my addr 172.21.15.159:0/350715238 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:45.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.722+0000 7f20cea00640 1 -- 172.21.15.159:0/350715238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20d013f190 msgr2=0x7f20d0138b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:45.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.722+0000 7f20cea00640 1 --2- 172.21.15.159:0/350715238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20d013f190 0x7f20d0138b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:45.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.722+0000 7f20cea00640 1 -- 172.21.15.159:0/350715238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20d0141890 msgr2=0x7f20d006e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:45.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.722+0000 7f20cea00640 1 --2- 172.21.15.159:0/350715238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20d0141890 0x7f20d006e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:45.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.722+0000 7f20cea00640 1 -- 172.21.15.159:0/350715238 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d006e8f0 con 0x7f20d006db90 2024-04-04T06:08:45.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.722+0000 7f20cea00640 1 --2- 172.21.15.159:0/350715238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20d006db90 0x7f20d0072020 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f20c000cb70 tx=0x7f20c000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:45.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.722+0000 7f20c7e00640 1 -- 172.21.15.159:0/350715238 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0017070 con 0x7f20d006db90 2024-04-04T06:08:45.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.722+0000 7f20c7e00640 1 -- 172.21.15.159:0/350715238 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c000ad40 con 0x7f20d006db90 2024-04-04T06:08:45.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.722+0000 7f20c7e00640 1 -- 172.21.15.159:0/350715238 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0012620 con 0x7f20d006db90 2024-04-04T06:08:45.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.722+0000 7f20cfe00640 1 -- 172.21.15.159:0/350715238 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d00726b0 con 0x7f20d006db90 2024-04-04T06:08:45.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.722+0000 7f20cfe00640 1 -- 172.21.15.159:0/350715238 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d0072c70 con 0x7f20d006db90 2024-04-04T06:08:45.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.723+0000 7f20cfe00640 1 -- 172.21.15.159:0/350715238 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20d0072e70 con 0x7f20d006db90 2024-04-04T06:08:45.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.725+0000 7f20c7e00640 1 -- 172.21.15.159:0/350715238 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f20c001a070 con 0x7f20d006db90 2024-04-04T06:08:45.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.726+0000 7f20c7e00640 1 --2- 172.21.15.159:0/350715238 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f20ac06c8f0 0x7f20ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:45.720 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.726+0000 7f20c7e00640 1 -- 172.21.15.159:0/350715238 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5853+0+0 (secure 0 0 0) 0x7f20c008ba80 con 0x7f20d006db90 2024-04-04T06:08:45.720 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.727+0000 7f20c6000640 1 --2- 172.21.15.159:0/350715238 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f20ac06c8f0 0x7f20ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:45.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.727+0000 7f20c6000640 1 --2- 172.21.15.159:0/350715238 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f20ac06c8f0 0x7f20ac06edb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f20b8006fd0 tx=0x7f20b8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:45.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.730+0000 7f20c7e00640 1 -- 172.21.15.159:0/350715238 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f20d0072e70 con 0x7f20d006db90 2024-04-04T06:08:45.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:45.917+0000 7f20cfe00640 1 -- 172.21.15.159:0/350715238 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f20d015f520 con 0x7f20d006db90 2024-04-04T06:08:46.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.449+0000 7f20c7e00640 1 -- 172.21.15.159:0/350715238 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v50) ==== 181+0+0 (secure 0 0 0) 0x7f20c0059e00 con 0x7f20d006db90 2024-04-04T06:08:46.444 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_metadata' removed 2024-04-04T06:08:46.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.454+0000 7f20cfe00640 1 -- 172.21.15.159:0/350715238 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f20ac06c8f0 msgr2=0x7f20ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:46.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.454+0000 7f20cfe00640 1 --2- 172.21.15.159:0/350715238 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f20ac06c8f0 0x7f20ac06edb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f20b8006fd0 tx=0x7f20b8008040 comp rx=0 tx=0).stop 2024-04-04T06:08:46.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.454+0000 7f20cfe00640 1 -- 172.21.15.159:0/350715238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20d006db90 msgr2=0x7f20d0072020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:46.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.454+0000 7f20cfe00640 1 --2- 172.21.15.159:0/350715238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20d006db90 0x7f20d0072020 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f20c000cb70 tx=0x7f20c000a430 comp rx=0 tx=0).stop 2024-04-04T06:08:46.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.455+0000 7f20cfe00640 1 -- 172.21.15.159:0/350715238 shutdown_connections 2024-04-04T06:08:46.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.455+0000 7f20cfe00640 1 --2- 172.21.15.159:0/350715238 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f20ac06c8f0 0x7f20ac06edb0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:46.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.455+0000 7f20cfe00640 1 --2- 172.21.15.159:0/350715238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20d0141890 0x7f20d006e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:46.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.455+0000 7f20cfe00640 1 --2- 172.21.15.159:0/350715238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f20d013f190 0x7f20d0138b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:46.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.455+0000 7f20cfe00640 1 --2- 172.21.15.159:0/350715238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20d006db90 0x7f20d0072020 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:46.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.455+0000 7f20cfe00640 1 -- 172.21.15.159:0/350715238 >> 172.21.15.159:0/350715238 conn(0x7f20d00b1d80 msgr2=0x7f20d0160500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:46.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.456+0000 7f20cfe00640 1 -- 172.21.15.159:0/350715238 shutdown_connections 2024-04-04T06:08:46.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.456+0000 7f20cfe00640 1 -- 172.21.15.159:0/350715238 wait complete. 2024-04-04T06:08:46.473 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-04T06:08:46.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.692+0000 7f6ecf000640 1 Processor -- start 2024-04-04T06:08:46.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.692+0000 7f6ecf000640 1 -- start start 2024-04-04T06:08:46.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.693+0000 7f6ecf000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f6ec813f190 0x7f6ec80ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:46.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.693+0000 7f6ecf000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ec8141fa0 0x7f6ec806db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:46.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.694+0000 7f6ecf000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ec80cfbf0 con 0x7f6ec80ced40 2024-04-04T06:08:46.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.694+0000 7f6ecf000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f6ec813f570 con 0x7f6ec813f190 2024-04-04T06:08:46.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.694+0000 7f6ecf000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6ec80cf350 con 0x7f6ec8141fa0 2024-04-04T06:08:46.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.694+0000 7f6ecdc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f6ec813f190 0x7f6ec80ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:46.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.694+0000 7f6ecdc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f6ec813f190 0x7f6ec80ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58178/0 (socket says 172.21.15.159:58178) 2024-04-04T06:08:46.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.694+0000 7f6ecdc00640 1 -- 172.21.15.159:0/3446381208 learned_addr learned my addr 172.21.15.159:0/3446381208 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:46.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.694+0000 7f6ecdc00640 1 -- 172.21.15.159:0/3446381208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ec8141fa0 msgr2=0x7f6ec806db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:46.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.694+0000 7f6ecdc00640 1 --2- 172.21.15.159:0/3446381208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ec8141fa0 0x7f6ec806db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:46.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.694+0000 7f6ecdc00640 1 -- 172.21.15.159:0/3446381208 >> v1:172.21.15.178:6789/0 conn(0x7f6ec80ced40 legacy=0x7f6ec8141890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:46.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.694+0000 7f6ecdc00640 1 -- 172.21.15.159:0/3446381208 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec80cf630 con 0x7f6ec813f190 2024-04-04T06:08:46.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.695+0000 7f6ecdc00640 1 --2- 172.21.15.159:0/3446381208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ec813f190 0x7f6ec80ce800 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8073710 tx=0x7f6ebc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481d8fdc792abbcf server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:46.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.696+0000 7f6ec7e00640 1 -- 172.21.15.159:0/3446381208 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc035020 con 0x7f6ec813f190 2024-04-04T06:08:46.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.696+0000 7f6ec7e00640 1 -- 172.21.15.159:0/3446381208 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ebc031830 con 0x7f6ec813f190 2024-04-04T06:08:46.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.696+0000 7f6ec7e00640 1 -- 172.21.15.159:0/3446381208 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc031c00 con 0x7f6ec813f190 2024-04-04T06:08:46.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.696+0000 7f6ecf000640 1 -- 172.21.15.159:0/3446381208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ec813f190 msgr2=0x7f6ec80ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:46.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.696+0000 7f6ecf000640 1 --2- 172.21.15.159:0/3446381208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ec813f190 0x7f6ec80ce800 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8073710 tx=0x7f6ebc02b8d0 comp rx=0 tx=0).stop 2024-04-04T06:08:46.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.697+0000 7f6ecf000640 1 -- 172.21.15.159:0/3446381208 shutdown_connections 2024-04-04T06:08:46.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.697+0000 7f6ecf000640 1 --2- 172.21.15.159:0/3446381208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ec8141fa0 0x7f6ec806db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:46.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.697+0000 7f6ecf000640 1 --2- 172.21.15.159:0/3446381208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ec813f190 0x7f6ec80ce800 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:46.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.697+0000 7f6ecf000640 1 -- 172.21.15.159:0/3446381208 >> 172.21.15.159:0/3446381208 conn(0x7f6ec80b1d80 msgr2=0x7f6ec80b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:46.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.697+0000 7f6ecf000640 1 -- 172.21.15.159:0/3446381208 shutdown_connections 2024-04-04T06:08:46.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.697+0000 7f6ecf000640 1 -- 172.21.15.159:0/3446381208 wait complete. 2024-04-04T06:08:46.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.698+0000 7f6ecf000640 1 Processor -- start 2024-04-04T06:08:46.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.698+0000 7f6ecf000640 1 -- start start 2024-04-04T06:08:46.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.699+0000 7f6ecf000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ec80ced40 0x7f6ec813a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:46.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.699+0000 7f6ecf000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ec813f190 0x7f6ec80b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:46.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.699+0000 7f6ecf000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ec8141fa0 0x7f6ec80afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:46.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.699+0000 7f6ecf000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec8142550 con 0x7f6ec813f190 2024-04-04T06:08:46.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.699+0000 7f6ecf000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec806d670 con 0x7f6ec8141fa0 2024-04-04T06:08:46.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.699+0000 7f6ecf000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6ec81426d0 con 0x7f6ec80ced40 2024-04-04T06:08:46.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.700+0000 7f6ecd200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ec813f190 0x7f6ec80b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:46.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.700+0000 7f6ecd200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ec813f190 0x7f6ec80b3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58186/0 (socket says 172.21.15.159:58186) 2024-04-04T06:08:46.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.700+0000 7f6ecd200640 1 -- 172.21.15.159:0/3792647864 learned_addr learned my addr 172.21.15.159:0/3792647864 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:46.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.700+0000 7f6ece600640 1 --2- 172.21.15.159:0/3792647864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ec8141fa0 0x7f6ec80afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:46.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.700+0000 7f6ecd200640 1 -- 172.21.15.159:0/3792647864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ec80ced40 msgr2=0x7f6ec813a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:46.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.700+0000 7f6ecd200640 1 --2- 172.21.15.159:0/3792647864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ec80ced40 0x7f6ec813a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:46.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.700+0000 7f6ecd200640 1 -- 172.21.15.159:0/3792647864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ec8141fa0 msgr2=0x7f6ec80afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:46.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.700+0000 7f6ecd200640 1 --2- 172.21.15.159:0/3792647864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ec8141fa0 0x7f6ec80afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:46.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.700+0000 7f6ecd200640 1 -- 172.21.15.159:0/3792647864 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec813ae90 con 0x7f6ec813f190 2024-04-04T06:08:46.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.700+0000 7f6ece600640 1 --2- 172.21.15.159:0/3792647864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ec8141fa0 0x7f6ec80afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:46.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.700+0000 7f6ecd200640 1 --2- 172.21.15.159:0/3792647864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ec813f190 0x7f6ec80b3d90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6eb400c460 tx=0x7f6eb400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:46.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.701+0000 7f6ec5600640 1 -- 172.21.15.159:0/3792647864 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4016020 con 0x7f6ec813f190 2024-04-04T06:08:46.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.701+0000 7f6ec5600640 1 -- 172.21.15.159:0/3792647864 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6eb4012870 con 0x7f6ec813f190 2024-04-04T06:08:46.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.701+0000 7f6ec5600640 1 -- 172.21.15.159:0/3792647864 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4012c60 con 0x7f6ec813f190 2024-04-04T06:08:46.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.701+0000 7f6ecf000640 1 -- 172.21.15.159:0/3792647864 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec80b47e0 con 0x7f6ec813f190 2024-04-04T06:08:46.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.701+0000 7f6ecf000640 1 -- 172.21.15.159:0/3792647864 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec80b5080 con 0x7f6ec813f190 2024-04-04T06:08:46.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.702+0000 7f6ecf000640 1 -- 172.21.15.159:0/3792647864 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ec815fa80 con 0x7f6ec813f190 2024-04-04T06:08:46.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.703+0000 7f6ec5600640 1 -- 172.21.15.159:0/3792647864 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6eb4019020 con 0x7f6ec813f190 2024-04-04T06:08:46.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.704+0000 7f6ec5600640 1 --2- 172.21.15.159:0/3792647864 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6ea806c8f0 0x7f6ea806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:46.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.704+0000 7f6ec5600640 1 -- 172.21.15.159:0/3792647864 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5415+0+0 (secure 0 0 0) 0x7f6eb408b840 con 0x7f6ec813f190 2024-04-04T06:08:46.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.704+0000 7f6ecdc00640 1 --2- 172.21.15.159:0/3792647864 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6ea806c8f0 0x7f6ea806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:46.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.705+0000 7f6ecdc00640 1 --2- 172.21.15.159:0/3792647864 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6ea806c8f0 0x7f6ea806edb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc036000 tx=0x7f6ebc030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:46.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.708+0000 7f6ec5600640 1 -- 172.21.15.159:0/3792647864 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6ec815fa80 con 0x7f6ec813f190 2024-04-04T06:08:46.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:46.911+0000 7f6ecf000640 1 -- 172.21.15.159:0/3792647864 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6ec8075940 con 0x7f6ec813f190 2024-04-04T06:08:47.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.453+0000 7f6ec5600640 1 -- 172.21.15.159:0/3792647864 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v51) ==== 169+0+0 (secure 0 0 0) 0x7f6eb4020030 con 0x7f6ec813f190 2024-04-04T06:08:47.447 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_data' removed 2024-04-04T06:08:47.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.458+0000 7f6ecf000640 1 -- 172.21.15.159:0/3792647864 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6ea806c8f0 msgr2=0x7f6ea806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:47.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.458+0000 7f6ecf000640 1 --2- 172.21.15.159:0/3792647864 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6ea806c8f0 0x7f6ea806edb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc036000 tx=0x7f6ebc030000 comp rx=0 tx=0).stop 2024-04-04T06:08:47.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.458+0000 7f6ecf000640 1 -- 172.21.15.159:0/3792647864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ec813f190 msgr2=0x7f6ec80b3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:47.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.458+0000 7f6ecf000640 1 --2- 172.21.15.159:0/3792647864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ec813f190 0x7f6ec80b3d90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6eb400c460 tx=0x7f6eb400c920 comp rx=0 tx=0).stop 2024-04-04T06:08:47.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.458+0000 7f6ecf000640 1 -- 172.21.15.159:0/3792647864 shutdown_connections 2024-04-04T06:08:47.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.458+0000 7f6ecf000640 1 --2- 172.21.15.159:0/3792647864 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6ea806c8f0 0x7f6ea806edb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:47.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.458+0000 7f6ecf000640 1 --2- 172.21.15.159:0/3792647864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ec8141fa0 0x7f6ec80afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:47.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.458+0000 7f6ecf000640 1 --2- 172.21.15.159:0/3792647864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ec813f190 0x7f6ec80b3d90 secure :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6eb400c460 tx=0x7f6eb400c920 comp rx=0 tx=0).stop 2024-04-04T06:08:47.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.458+0000 7f6ecf000640 1 --2- 172.21.15.159:0/3792647864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6ec80ced40 0x7f6ec813a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:47.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.458+0000 7f6ecf000640 1 -- 172.21.15.159:0/3792647864 >> 172.21.15.159:0/3792647864 conn(0x7f6ec80b1d80 msgr2=0x7f6ec8152d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:47.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.461+0000 7f6ecf000640 1 -- 172.21.15.159:0/3792647864 shutdown_connections 2024-04-04T06:08:47.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.461+0000 7f6ecf000640 1 -- 172.21.15.159:0/3792647864 wait complete. 2024-04-04T06:08:47.480 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-04T06:08:47.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.684+0000 7f9b2a800640 1 Processor -- start 2024-04-04T06:08:47.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.684+0000 7f9b2a800640 1 -- start start 2024-04-04T06:08:47.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.685+0000 7f9b2a800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9b240d0990 0x7f9b241670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:47.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.685+0000 7f9b2a800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b24163ac0 0x7f9b24071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:47.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.685+0000 7f9b2a800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b24071d60 con 0x7f9b24162920 2024-04-04T06:08:47.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.685+0000 7f9b2a800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f9b24163f30 con 0x7f9b240d0990 2024-04-04T06:08:47.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.685+0000 7f9b2a800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9b241665d0 con 0x7f9b24163ac0 2024-04-04T06:08:47.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.686+0000 7f9b23400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f9b24162920 0x7f9b240d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:60078/0 (socket says 172.21.15.159:60078) 2024-04-04T06:08:47.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.686+0000 7f9b23400640 1 -- 172.21.15.159:0/3896826238 learned_addr learned my addr 172.21.15.159:0/3896826238 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:47.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.686+0000 7f9b1aa00640 1 --2- 172.21.15.159:0/3896826238 >> v2:172.21.15.178:3300/0 conn(0x7f9b240d0990 0x7f9b241670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:47.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.686+0000 7f9b23e00640 1 --2- 172.21.15.159:0/3896826238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b24163ac0 0x7f9b24071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:47.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.686+0000 7f9b22a00640 1 -- 172.21.15.159:0/3896826238 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2412068385 0 0) 0x7f9b24071d60 con 0x7f9b24162920 2024-04-04T06:08:47.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.686+0000 7f9b22a00640 1 -- 172.21.15.159:0/3896826238 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b08003520 con 0x7f9b24162920 2024-04-04T06:08:47.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.686+0000 7f9b22a00640 1 -- 172.21.15.159:0/3896826238 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 586777183 0 0) 0x7f9b08003520 con 0x7f9b24162920 2024-04-04T06:08:47.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.687+0000 7f9b22a00640 1 -- 172.21.15.159:0/3896826238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b24163ac0 msgr2=0x7f9b24071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:47.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.687+0000 7f9b22a00640 1 --2- 172.21.15.159:0/3896826238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b24163ac0 0x7f9b24071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:47.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.687+0000 7f9b22a00640 1 -- 172.21.15.159:0/3896826238 >> v2:172.21.15.178:3300/0 conn(0x7f9b240d0990 msgr2=0x7f9b241670b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:47.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.687+0000 7f9b22a00640 1 --2- 172.21.15.159:0/3896826238 >> v2:172.21.15.178:3300/0 conn(0x7f9b240d0990 0x7f9b241670b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:47.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.687+0000 7f9b22a00640 1 -- 172.21.15.159:0/3896826238 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b241668b0 con 0x7f9b24162920 2024-04-04T06:08:47.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.687+0000 7f9b22a00640 1 -- 172.21.15.159:0/3896826238 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f9b14001690 con 0x7f9b24162920 2024-04-04T06:08:47.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.687+0000 7f9b22a00640 1 -- 172.21.15.159:0/3896826238 >> v1:172.21.15.159:6789/0 conn(0x7f9b24162920 legacy=0x7f9b240d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:47.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.687+0000 7f9b22a00640 1 --2- 172.21.15.159:0/3896826238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b080039b0 0x7f9b08003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:47.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.687+0000 7f9b22a00640 1 --2- 172.21.15.159:0/3896826238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b08004800 0x7f9b08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:47.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.687+0000 7f9b22a00640 1 --2- 172.21.15.159:0/3896826238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b080051c0 0x7f9b0800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:47.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.687+0000 7f9b22a00640 1 -- 172.21.15.159:0/3896826238 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24163f30 con 0x7f9b080051c0 2024-04-04T06:08:47.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.687+0000 7f9b22a00640 1 -- 172.21.15.159:0/3896826238 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b241665d0 con 0x7f9b080039b0 2024-04-04T06:08:47.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.687+0000 7f9b22a00640 1 -- 172.21.15.159:0/3896826238 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9b0800c030 con 0x7f9b08004800 2024-04-04T06:08:47.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.687+0000 7f9b23e00640 1 --2- 172.21.15.159:0/3896826238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b24163ac0 0x7f9b24071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:08:47.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.687+0000 7f9b23e00640 1 --2- 172.21.15.159:0/3896826238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b080051c0 0x7f9b0800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:47.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.687+0000 7f9b23400640 1 --2- 172.21.15.159:0/3896826238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b080039b0 0x7f9b08003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:47.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.687+0000 7f9b1aa00640 1 --2- 172.21.15.159:0/3896826238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b08004800 0x7f9b08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:47.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.688+0000 7f9b23e00640 1 -- 172.21.15.159:0/3896826238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b08004800 msgr2=0x7f9b08004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:47.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.688+0000 7f9b23e00640 1 --2- 172.21.15.159:0/3896826238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b08004800 0x7f9b08004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:47.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.688+0000 7f9b23e00640 1 -- 172.21.15.159:0/3896826238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b080039b0 msgr2=0x7f9b08003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:47.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.688+0000 7f9b23e00640 1 --2- 172.21.15.159:0/3896826238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b080039b0 0x7f9b08003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:47.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.688+0000 7f9b23e00640 1 -- 172.21.15.159:0/3896826238 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b08003520 con 0x7f9b080051c0 2024-04-04T06:08:47.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.689+0000 7f9b23e00640 1 --2- 172.21.15.159:0/3896826238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b080051c0 0x7f9b0800b9c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9b24163ea0 tx=0x7f9b08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff795a1672316631 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:47.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.689+0000 7f9b22a00640 1 -- 172.21.15.159:0/3896826238 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c032070 con 0x7f9b080051c0 2024-04-04T06:08:47.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.689+0000 7f9b22a00640 1 -- 172.21.15.159:0/3896826238 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b1c030070 con 0x7f9b080051c0 2024-04-04T06:08:47.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.689+0000 7f9b22a00640 1 -- 172.21.15.159:0/3896826238 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c009b50 con 0x7f9b080051c0 2024-04-04T06:08:47.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.690+0000 7f9b2a800640 1 -- 172.21.15.159:0/3896826238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b080051c0 msgr2=0x7f9b0800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:47.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.690+0000 7f9b2a800640 1 --2- 172.21.15.159:0/3896826238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b080051c0 0x7f9b0800b9c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9b24163ea0 tx=0x7f9b08005830 comp rx=0 tx=0).stop 2024-04-04T06:08:47.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.690+0000 7f9b23e00640 1 -- 172.21.15.159:0/3896826238 reap_dead start 2024-04-04T06:08:47.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.690+0000 7f9b2a800640 1 -- 172.21.15.159:0/3896826238 shutdown_connections 2024-04-04T06:08:47.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.690+0000 7f9b2a800640 1 -- 172.21.15.159:0/3896826238 >> 172.21.15.159:0/3896826238 conn(0x7f9b2406b750 msgr2=0x7f9b2406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:47.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.690+0000 7f9b2a800640 1 -- 172.21.15.159:0/3896826238 shutdown_connections 2024-04-04T06:08:47.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.690+0000 7f9b2a800640 1 -- 172.21.15.159:0/3896826238 wait complete. 2024-04-04T06:08:47.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.692+0000 7f9b2a800640 1 Processor -- start 2024-04-04T06:08:47.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.692+0000 7f9b2a800640 1 -- start start 2024-04-04T06:08:47.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.693+0000 7f9b2a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b080051c0 0x7f9b241f8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:47.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.693+0000 7f9b2a800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b240b4ba0 0x7f9b24165eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:47.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.693+0000 7f9b2a800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b241640b0 0x7f9b24164540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:47.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.693+0000 7f9b2a800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24072040 con 0x7f9b080051c0 2024-04-04T06:08:47.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.693+0000 7f9b2a800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24162d00 con 0x7f9b240b4ba0 2024-04-04T06:08:47.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.693+0000 7f9b2a800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9b240721c0 con 0x7f9b241640b0 2024-04-04T06:08:47.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.693+0000 7f9b23400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b080051c0 0x7f9b241f8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:47.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.694+0000 7f9b23400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b080051c0 0x7f9b241f8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58204/0 (socket says 172.21.15.159:58204) 2024-04-04T06:08:47.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.694+0000 7f9b23400640 1 -- 172.21.15.159:0/1793649792 learned_addr learned my addr 172.21.15.159:0/1793649792 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:47.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.694+0000 7f9b1aa00640 1 --2- 172.21.15.159:0/1793649792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b240b4ba0 0x7f9b24165eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:47.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.694+0000 7f9b23400640 1 -- 172.21.15.159:0/1793649792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b241640b0 msgr2=0x7f9b24164540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:47.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.694+0000 7f9b23400640 1 --2- 172.21.15.159:0/1793649792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b241640b0 0x7f9b24164540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:47.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.694+0000 7f9b23400640 1 -- 172.21.15.159:0/1793649792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b240b4ba0 msgr2=0x7f9b24165eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:47.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.694+0000 7f9b23400640 1 --2- 172.21.15.159:0/1793649792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b240b4ba0 0x7f9b24165eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:47.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.694+0000 7f9b23400640 1 -- 172.21.15.159:0/1793649792 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b241663f0 con 0x7f9b080051c0 2024-04-04T06:08:47.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.694+0000 7f9b1aa00640 1 --2- 172.21.15.159:0/1793649792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b240b4ba0 0x7f9b24165eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:47.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.694+0000 7f9b23400640 1 --2- 172.21.15.159:0/1793649792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b080051c0 0x7f9b241f8520 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400e730 tx=0x7f9b1400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:47.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.694+0000 7f9b1be00640 1 -- 172.21.15.159:0/1793649792 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1400c640 con 0x7f9b080051c0 2024-04-04T06:08:47.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.695+0000 7f9b1be00640 1 -- 172.21.15.159:0/1793649792 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b14018070 con 0x7f9b080051c0 2024-04-04T06:08:47.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.695+0000 7f9b1be00640 1 -- 172.21.15.159:0/1793649792 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1400ca40 con 0x7f9b080051c0 2024-04-04T06:08:47.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.695+0000 7f9b2a800640 1 -- 172.21.15.159:0/1793649792 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b241f8c20 con 0x7f9b080051c0 2024-04-04T06:08:47.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.695+0000 7f9b2a800640 1 -- 172.21.15.159:0/1793649792 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b241f91e0 con 0x7f9b080051c0 2024-04-04T06:08:47.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.696+0000 7f9b2a800640 1 -- 172.21.15.159:0/1793649792 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b240685c0 con 0x7f9b080051c0 2024-04-04T06:08:47.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.697+0000 7f9b1be00640 1 -- 172.21.15.159:0/1793649792 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b1401d460 con 0x7f9b080051c0 2024-04-04T06:08:47.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.698+0000 7f9b1be00640 1 --2- 172.21.15.159:0/1793649792 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b0006c8f0 0x7f9b0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:47.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.698+0000 7f9b1be00640 1 -- 172.21.15.159:0/1793649792 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f9b1408b880 con 0x7f9b080051c0 2024-04-04T06:08:47.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.699+0000 7f9b1aa00640 1 --2- 172.21.15.159:0/1793649792 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b0006c8f0 0x7f9b0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:47.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.699+0000 7f9b1aa00640 1 --2- 172.21.15.159:0/1793649792 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b0006c8f0 0x7f9b0006edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009a80 tx=0x7f9b0c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:47.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.703+0000 7f9b1be00640 1 -- 172.21.15.159:0/1793649792 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9b241663f0 con 0x7f9b080051c0 2024-04-04T06:08:47.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.904+0000 7f9b2a800640 1 -- 172.21.15.159:0/1793649792 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f9b2406ef60 con 0x7f9b080051c0 2024-04-04T06:08:47.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.909+0000 7f9b1be00640 1 -- 172.21.15.159:0/1793649792 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f9b14059fa0 con 0x7f9b080051c0 2024-04-04T06:08:47.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.911+0000 7f9b2a800640 1 -- 172.21.15.159:0/1793649792 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b0006c8f0 msgr2=0x7f9b0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:47.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.911+0000 7f9b2a800640 1 --2- 172.21.15.159:0/1793649792 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b0006c8f0 0x7f9b0006edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009a80 tx=0x7f9b0c008040 comp rx=0 tx=0).stop 2024-04-04T06:08:47.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.911+0000 7f9b2a800640 1 -- 172.21.15.159:0/1793649792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b080051c0 msgr2=0x7f9b241f8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:47.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.911+0000 7f9b2a800640 1 --2- 172.21.15.159:0/1793649792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b080051c0 0x7f9b241f8520 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400e730 tx=0x7f9b1400ebf0 comp rx=0 tx=0).stop 2024-04-04T06:08:47.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.912+0000 7f9b2a800640 1 -- 172.21.15.159:0/1793649792 shutdown_connections 2024-04-04T06:08:47.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.912+0000 7f9b2a800640 1 --2- 172.21.15.159:0/1793649792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b241640b0 0x7f9b24164540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:47.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.912+0000 7f9b2a800640 1 --2- 172.21.15.159:0/1793649792 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b0006c8f0 0x7f9b0006edb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:47.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.912+0000 7f9b2a800640 1 --2- 172.21.15.159:0/1793649792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b240b4ba0 0x7f9b24165eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:47.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.912+0000 7f9b2a800640 1 --2- 172.21.15.159:0/1793649792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b080051c0 0x7f9b241f8520 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:47.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.912+0000 7f9b2a800640 1 -- 172.21.15.159:0/1793649792 >> 172.21.15.159:0/1793649792 conn(0x7f9b2406b750 msgr2=0x7f9b2415ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:47.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.912+0000 7f9b2a800640 1 -- 172.21.15.159:0/1793649792 shutdown_connections 2024-04-04T06:08:47.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:47.912+0000 7f9b2a800640 1 -- 172.21.15.159:0/1793649792 wait complete. 2024-04-04T06:08:47.924 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-04T06:08:48.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.137+0000 7f7a51600640 1 Processor -- start 2024-04-04T06:08:48.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.137+0000 7f7a51600640 1 -- start start 2024-04-04T06:08:48.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.138+0000 7f7a51600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7a4c155550 0x7f7a4c155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:48.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.138+0000 7f7a51600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a4c0d0990 0x7f7a4c0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:48.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.138+0000 7f7a51600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a4c0b40b0 con 0x7f7a4c15f800 2024-04-04T06:08:48.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.138+0000 7f7a51600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7a4c0b4390 con 0x7f7a4c155550 2024-04-04T06:08:48.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.138+0000 7f7a51600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7a4c0b4670 con 0x7f7a4c0d0990 2024-04-04T06:08:48.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.139+0000 7f7a4b400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a4c0d0990 0x7f7a4c0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:48.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.139+0000 7f7a4b400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a4c0d0990 0x7f7a4c0aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:41984/0 (socket says 172.21.15.159:41984) 2024-04-04T06:08:48.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.139+0000 7f7a4b400640 1 -- 172.21.15.159:0/2724688209 learned_addr learned my addr 172.21.15.159:0/2724688209 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:48.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.139+0000 7f7a4aa00640 1 --2- 172.21.15.159:0/2724688209 >> v2:172.21.15.178:3300/0 conn(0x7f7a4c155550 0x7f7a4c155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:48.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.140+0000 7f7a4aa00640 1 -- 172.21.15.159:0/2724688209 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a4c0d0990 msgr2=0x7f7a4c0aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:48.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.140+0000 7f7a4aa00640 1 --2- 172.21.15.159:0/2724688209 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a4c0d0990 0x7f7a4c0aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:48.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.140+0000 7f7a4aa00640 1 -- 172.21.15.159:0/2724688209 >> v1:172.21.15.178:6789/0 conn(0x7f7a4c15f800 legacy=0x7f7a4c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:48.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.140+0000 7f7a4aa00640 1 -- 172.21.15.159:0/2724688209 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4c0b4950 con 0x7f7a4c155550 2024-04-04T06:08:48.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.140+0000 7f7a4b400640 1 --2- 172.21.15.159:0/2724688209 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a4c0d0990 0x7f7a4c0aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:08:48.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.141+0000 7f7a4aa00640 1 --2- 172.21.15.159:0/2724688209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a4c155550 0x7f7a4c155930 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0097a0 tx=0x7f7a3c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19819c9d329c39d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:48.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.141+0000 7f7a4a000640 1 -- 172.21.15.159:0/2724688209 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c035020 con 0x7f7a4c155550 2024-04-04T06:08:48.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.141+0000 7f7a4a000640 1 -- 172.21.15.159:0/2724688209 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a3c031810 con 0x7f7a4c155550 2024-04-04T06:08:48.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.141+0000 7f7a4a000640 1 -- 172.21.15.159:0/2724688209 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c031be0 con 0x7f7a4c155550 2024-04-04T06:08:48.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.142+0000 7f7a51600640 1 -- 172.21.15.159:0/2724688209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a4c155550 msgr2=0x7f7a4c155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:48.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.142+0000 7f7a51600640 1 --2- 172.21.15.159:0/2724688209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a4c155550 0x7f7a4c155930 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0097a0 tx=0x7f7a3c02b8b0 comp rx=0 tx=0).stop 2024-04-04T06:08:48.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.142+0000 7f7a51600640 1 -- 172.21.15.159:0/2724688209 shutdown_connections 2024-04-04T06:08:48.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.142+0000 7f7a51600640 1 --2- 172.21.15.159:0/2724688209 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a4c0d0990 0x7f7a4c0aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:48.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.142+0000 7f7a51600640 1 --2- 172.21.15.159:0/2724688209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a4c155550 0x7f7a4c155930 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:48.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.142+0000 7f7a51600640 1 -- 172.21.15.159:0/2724688209 >> 172.21.15.159:0/2724688209 conn(0x7f7a4c06b750 msgr2=0x7f7a4c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:48.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.142+0000 7f7a51600640 1 -- 172.21.15.159:0/2724688209 shutdown_connections 2024-04-04T06:08:48.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.142+0000 7f7a51600640 1 -- 172.21.15.159:0/2724688209 wait complete. 2024-04-04T06:08:48.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.144+0000 7f7a51600640 1 Processor -- start 2024-04-04T06:08:48.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.144+0000 7f7a51600640 1 -- start start 2024-04-04T06:08:48.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.145+0000 7f7a51600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a4c0d0990 0x7f7a4c1ea3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:48.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.145+0000 7f7a51600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a4c155550 0x7f7a4c1545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:48.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.145+0000 7f7a51600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a4c15f800 0x7f7a4c154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:48.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.145+0000 7f7a51600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c15ba20 con 0x7f7a4c15f800 2024-04-04T06:08:48.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.145+0000 7f7a51600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c15fc80 con 0x7f7a4c0d0990 2024-04-04T06:08:48.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.145+0000 7f7a51600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7a4c15bba0 con 0x7f7a4c155550 2024-04-04T06:08:48.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.146+0000 7f7a4b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a4c15f800 0x7f7a4c154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:48.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.146+0000 7f7a4b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a4c15f800 0x7f7a4c154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58206/0 (socket says 172.21.15.159:58206) 2024-04-04T06:08:48.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.146+0000 7f7a4b400640 1 -- 172.21.15.159:0/1665667917 learned_addr learned my addr 172.21.15.159:0/1665667917 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:48.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.146+0000 7f7a4aa00640 1 --2- 172.21.15.159:0/1665667917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a4c0d0990 0x7f7a4c1ea3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:48.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.146+0000 7f7a43e00640 1 --2- 172.21.15.159:0/1665667917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a4c155550 0x7f7a4c1545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:48.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.146+0000 7f7a4b400640 1 -- 172.21.15.159:0/1665667917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a4c155550 msgr2=0x7f7a4c1545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:48.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.146+0000 7f7a4b400640 1 --2- 172.21.15.159:0/1665667917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a4c155550 0x7f7a4c1545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:48.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.146+0000 7f7a4b400640 1 -- 172.21.15.159:0/1665667917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a4c0d0990 msgr2=0x7f7a4c1ea3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:48.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.146+0000 7f7a4b400640 1 --2- 172.21.15.159:0/1665667917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a4c0d0990 0x7f7a4c1ea3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:48.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.146+0000 7f7a4b400640 1 -- 172.21.15.159:0/1665667917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4c0aee30 con 0x7f7a4c15f800 2024-04-04T06:08:48.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.146+0000 7f7a4aa00640 1 --2- 172.21.15.159:0/1665667917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a4c0d0990 0x7f7a4c1ea3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:48.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.146+0000 7f7a4b400640 1 --2- 172.21.15.159:0/1665667917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a4c15f800 0x7f7a4c154b10 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400ef90 tx=0x7f7a4400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:48.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.147+0000 7f7a42a00640 1 -- 172.21.15.159:0/1665667917 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4400cbe0 con 0x7f7a4c15f800 2024-04-04T06:08:48.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.147+0000 7f7a42a00640 1 -- 172.21.15.159:0/1665667917 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a4400cd80 con 0x7f7a4c15f800 2024-04-04T06:08:48.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.147+0000 7f7a51600640 1 -- 172.21.15.159:0/1665667917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4c1e6b50 con 0x7f7a4c15f800 2024-04-04T06:08:48.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.147+0000 7f7a42a00640 1 -- 172.21.15.159:0/1665667917 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44003bd0 con 0x7f7a4c15f800 2024-04-04T06:08:48.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.147+0000 7f7a51600640 1 -- 172.21.15.159:0/1665667917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a4c1e7110 con 0x7f7a4c15f800 2024-04-04T06:08:48.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.148+0000 7f7a51600640 1 -- 172.21.15.159:0/1665667917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a4c15e670 con 0x7f7a4c15f800 2024-04-04T06:08:48.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.149+0000 7f7a42a00640 1 -- 172.21.15.159:0/1665667917 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a4401b030 con 0x7f7a4c15f800 2024-04-04T06:08:48.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.150+0000 7f7a42a00640 1 --2- 172.21.15.159:0/1665667917 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a2806c920 0x7f7a2806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:48.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.150+0000 7f7a42a00640 1 -- 172.21.15.159:0/1665667917 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f7a4408bc80 con 0x7f7a4c15f800 2024-04-04T06:08:48.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.150+0000 7f7a4aa00640 1 --2- 172.21.15.159:0/1665667917 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a2806c920 0x7f7a2806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:48.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.152+0000 7f7a4aa00640 1 --2- 172.21.15.159:0/1665667917 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a2806c920 0x7f7a2806ede0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c036000 tx=0x7f7a3c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:48.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.155+0000 7f7a42a00640 1 -- 172.21.15.159:0/1665667917 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7a4c15e670 con 0x7f7a4c15f800 2024-04-04T06:08:48.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.425+0000 7f7a51600640 1 -- 172.21.15.159:0/1665667917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f7a4c0685c0 con 0x7f7a4c15f800 2024-04-04T06:08:48.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.472+0000 7f7a42a00640 1 -- 172.21.15.159:0/1665667917 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v176) ==== 149+0+0 (secure 0 0 0) 0x7f7a44022020 con 0x7f7a4c15f800 2024-04-04T06:08:48.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.475+0000 7f7a51600640 1 -- 172.21.15.159:0/1665667917 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a2806c920 msgr2=0x7f7a2806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:48.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.475+0000 7f7a51600640 1 --2- 172.21.15.159:0/1665667917 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a2806c920 0x7f7a2806ede0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c036000 tx=0x7f7a3c008030 comp rx=0 tx=0).stop 2024-04-04T06:08:48.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.475+0000 7f7a51600640 1 -- 172.21.15.159:0/1665667917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a4c15f800 msgr2=0x7f7a4c154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:48.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.475+0000 7f7a51600640 1 --2- 172.21.15.159:0/1665667917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a4c15f800 0x7f7a4c154b10 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400ef90 tx=0x7f7a4400c490 comp rx=0 tx=0).stop 2024-04-04T06:08:48.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.475+0000 7f7a51600640 1 -- 172.21.15.159:0/1665667917 shutdown_connections 2024-04-04T06:08:48.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.475+0000 7f7a51600640 1 --2- 172.21.15.159:0/1665667917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a4c15f800 0x7f7a4c154b10 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:48.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.475+0000 7f7a51600640 1 --2- 172.21.15.159:0/1665667917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a4c155550 0x7f7a4c1545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:48.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.475+0000 7f7a51600640 1 --2- 172.21.15.159:0/1665667917 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a2806c920 0x7f7a2806ede0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:48.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.475+0000 7f7a51600640 1 --2- 172.21.15.159:0/1665667917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a4c0d0990 0x7f7a4c1ea3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:48.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.475+0000 7f7a51600640 1 -- 172.21.15.159:0/1665667917 >> 172.21.15.159:0/1665667917 conn(0x7f7a4c06b750 msgr2=0x7f7a4c072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:48.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.475+0000 7f7a51600640 1 -- 172.21.15.159:0/1665667917 shutdown_connections 2024-04-04T06:08:48.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.475+0000 7f7a51600640 1 -- 172.21.15.159:0/1665667917 wait complete. 2024-04-04T06:08:48.486 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-04-04T06:08:48.486 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-04T06:08:48.487 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-04T06:08:48.487 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:08:48.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.697+0000 7fe821600640 1 Processor -- start 2024-04-04T06:08:48.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.697+0000 7fe821600640 1 -- start start 2024-04-04T06:08:48.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.699+0000 7fe821600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe81c145c10 0x7fe81c0c8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:48.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.699+0000 7fe821600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe81c0cfd80 0x7fe81c0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:48.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.699+0000 7fe821600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe81c0d0200 con 0x7fe81c145ff0 2024-04-04T06:08:48.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.699+0000 7fe821600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe81c0d0aa0 con 0x7fe81c0cfd80 2024-04-04T06:08:48.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.699+0000 7fe821600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe81c0cc800 con 0x7fe81c145c10 2024-04-04T06:08:48.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.699+0000 7fe81a000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe81c0cfd80 0x7fe81c0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:48.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.699+0000 7fe81a000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe81c0cfd80 0x7fe81c0b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58208/0 (socket says 172.21.15.159:58208) 2024-04-04T06:08:48.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.699+0000 7fe81a000640 1 -- 172.21.15.159:0/993995035 learned_addr learned my addr 172.21.15.159:0/993995035 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:48.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.699+0000 7fe81b400640 1 --1- 172.21.15.159:0/993995035 >> v1:172.21.15.159:6789/0 conn(0x7fe81c145ff0 0x7fe81c070fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:60094/0 (socket says 172.21.15.159:60094) 2024-04-04T06:08:48.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.699+0000 7fe81aa00640 1 --2- 172.21.15.159:0/993995035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe81c145c10 0x7fe81c0c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:48.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.700+0000 7fe81a000640 1 -- 172.21.15.159:0/993995035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe81c145c10 msgr2=0x7fe81c0c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:48.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.700+0000 7fe81a000640 1 --2- 172.21.15.159:0/993995035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe81c145c10 0x7fe81c0c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:48.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.700+0000 7fe81a000640 1 -- 172.21.15.159:0/993995035 >> v1:172.21.15.159:6789/0 conn(0x7fe81c145ff0 legacy=0x7fe81c070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:48.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.700+0000 7fe81a000640 1 -- 172.21.15.159:0/993995035 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81c0d04e0 con 0x7fe81c0cfd80 2024-04-04T06:08:48.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.701+0000 7fe81a000640 1 --2- 172.21.15.159:0/993995035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe81c0cfd80 0x7fe81c0b3e20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009870 tx=0x7fe80c02ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a82d3a65c25c54 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:48.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.701+0000 7fe819600640 1 -- 172.21.15.159:0/993995035 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c02f830 con 0x7fe81c0cfd80 2024-04-04T06:08:48.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.701+0000 7fe819600640 1 -- 172.21.15.159:0/993995035 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe80c02f9d0 con 0x7fe81c0cfd80 2024-04-04T06:08:48.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.701+0000 7fe819600640 1 -- 172.21.15.159:0/993995035 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c02fda0 con 0x7fe81c0cfd80 2024-04-04T06:08:48.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.702+0000 7fe821600640 1 -- 172.21.15.159:0/993995035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe81c0cfd80 msgr2=0x7fe81c0b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:48.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.702+0000 7fe821600640 1 --2- 172.21.15.159:0/993995035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe81c0cfd80 0x7fe81c0b3e20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009870 tx=0x7fe80c02ba10 comp rx=0 tx=0).stop 2024-04-04T06:08:48.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.702+0000 7fe821600640 1 -- 172.21.15.159:0/993995035 shutdown_connections 2024-04-04T06:08:48.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.702+0000 7fe821600640 1 --2- 172.21.15.159:0/993995035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe81c0cfd80 0x7fe81c0b3e20 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:48.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.702+0000 7fe821600640 1 --2- 172.21.15.159:0/993995035 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe81c145c10 0x7fe81c0c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:48.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.702+0000 7fe821600640 1 -- 172.21.15.159:0/993995035 >> 172.21.15.159:0/993995035 conn(0x7fe81c0d2240 msgr2=0x7fe81c0d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:48.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.702+0000 7fe821600640 1 -- 172.21.15.159:0/993995035 shutdown_connections 2024-04-04T06:08:48.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.702+0000 7fe821600640 1 -- 172.21.15.159:0/993995035 wait complete. 2024-04-04T06:08:48.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.703+0000 7fe821600640 1 Processor -- start 2024-04-04T06:08:48.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.704+0000 7fe821600640 1 -- start start 2024-04-04T06:08:48.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.705+0000 7fe821600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe81c0cfd80 0x7fe81c1438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:48.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.705+0000 7fe821600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe81c145c10 0x7fe81c0ab270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:48.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.705+0000 7fe821600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe81c0ab7b0 0x7fe81c0cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:48.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.705+0000 7fe821600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c0c8860 con 0x7fe81c0cfd80 2024-04-04T06:08:48.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.705+0000 7fe821600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c0ccae0 con 0x7fe81c0ab7b0 2024-04-04T06:08:48.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.705+0000 7fe821600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe81c0c89e0 con 0x7fe81c145c10 2024-04-04T06:08:48.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.705+0000 7fe81aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe81c0cfd80 0x7fe81c1438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:48.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.705+0000 7fe81aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe81c0cfd80 0x7fe81c1438a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58212/0 (socket says 172.21.15.159:58212) 2024-04-04T06:08:48.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.705+0000 7fe81aa00640 1 -- 172.21.15.159:0/3037878393 learned_addr learned my addr 172.21.15.159:0/3037878393 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:48.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.705+0000 7fe81aa00640 1 -- 172.21.15.159:0/3037878393 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe81c145c10 msgr2=0x7fe81c0ab270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:48.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.705+0000 7fe81aa00640 1 --2- 172.21.15.159:0/3037878393 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe81c145c10 0x7fe81c0ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:48.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.705+0000 7fe81aa00640 1 -- 172.21.15.159:0/3037878393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe81c0ab7b0 msgr2=0x7fe81c0cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:48.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.705+0000 7fe81aa00640 1 --2- 172.21.15.159:0/3037878393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe81c0ab7b0 0x7fe81c0cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:48.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.705+0000 7fe81aa00640 1 -- 172.21.15.159:0/3037878393 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81c0aa1f0 con 0x7fe81c0cfd80 2024-04-04T06:08:48.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.705+0000 7fe81aa00640 1 --2- 172.21.15.159:0/3037878393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe81c0cfd80 0x7fe81c1438a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe814007f20 tx=0x7fe81400e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:48.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.706+0000 7fe812a00640 1 -- 172.21.15.159:0/3037878393 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe814014620 con 0x7fe81c0cfd80 2024-04-04T06:08:48.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.706+0000 7fe812a00640 1 -- 172.21.15.159:0/3037878393 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe814019070 con 0x7fe81c0cfd80 2024-04-04T06:08:48.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.706+0000 7fe812a00640 1 -- 172.21.15.159:0/3037878393 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe814014a00 con 0x7fe81c0cfd80 2024-04-04T06:08:48.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.706+0000 7fe821600640 1 -- 172.21.15.159:0/3037878393 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe81c0aa4d0 con 0x7fe81c0cfd80 2024-04-04T06:08:48.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.706+0000 7fe821600640 1 -- 172.21.15.159:0/3037878393 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe81c0aaa90 con 0x7fe81c0cfd80 2024-04-04T06:08:48.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.707+0000 7fe821600640 1 -- 172.21.15.159:0/3037878393 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe81c0685c0 con 0x7fe81c0cfd80 2024-04-04T06:08:48.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.708+0000 7fe812a00640 1 -- 172.21.15.159:0/3037878393 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe814014ba0 con 0x7fe81c0cfd80 2024-04-04T06:08:48.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.709+0000 7fe812a00640 1 --2- 172.21.15.159:0/3037878393 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe7f806c920 0x7fe7f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:48.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.709+0000 7fe812a00640 1 -- 172.21.15.159:0/3037878393 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7fe81408c200 con 0x7fe81c0cfd80 2024-04-04T06:08:48.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.710+0000 7fe81a000640 1 --2- 172.21.15.159:0/3037878393 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe7f806c920 0x7fe7f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:48.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.711+0000 7fe81a000640 1 --2- 172.21.15.159:0/3037878393 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe7f806c920 0x7fe7f806ede0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe80c036000 tx=0x7fe80c039040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:48.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.713+0000 7fe812a00640 1 -- 172.21.15.159:0/3037878393 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe81405a890 con 0x7fe81c0cfd80 2024-04-04T06:08:48.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.912+0000 7fe821600640 1 -- 172.21.15.159:0/3037878393 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe81c0ce340 con 0x7fe81c0cfd80 2024-04-04T06:08:48.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.915+0000 7fe812a00640 1 -- 172.21.15.159:0/3037878393 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+13034 (secure 0 0 0) 0x7fe81405a190 con 0x7fe81c0cfd80 2024-04-04T06:08:48.908 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:48.909 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":51,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:08:47.450657+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6835/1867245367":"2024-04-05T06:08:44.433413+0000","172.21.15.159:6834/1867245367":"2024-04-05T06:08:44.433413+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:08:48.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.917+0000 7fe821600640 1 -- 172.21.15.159:0/3037878393 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe7f806c920 msgr2=0x7fe7f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:48.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.917+0000 7fe821600640 1 --2- 172.21.15.159:0/3037878393 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe7f806c920 0x7fe7f806ede0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe80c036000 tx=0x7fe80c039040 comp rx=0 tx=0).stop 2024-04-04T06:08:48.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.917+0000 7fe821600640 1 -- 172.21.15.159:0/3037878393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe81c0cfd80 msgr2=0x7fe81c1438a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:48.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.917+0000 7fe821600640 1 --2- 172.21.15.159:0/3037878393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe81c0cfd80 0x7fe81c1438a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe814007f20 tx=0x7fe81400e570 comp rx=0 tx=0).stop 2024-04-04T06:08:48.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.917+0000 7fe821600640 1 -- 172.21.15.159:0/3037878393 shutdown_connections 2024-04-04T06:08:48.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.917+0000 7fe821600640 1 --2- 172.21.15.159:0/3037878393 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe7f806c920 0x7fe7f806ede0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:48.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.917+0000 7fe821600640 1 --2- 172.21.15.159:0/3037878393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe81c0ab7b0 0x7fe81c0cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:48.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.917+0000 7fe821600640 1 --2- 172.21.15.159:0/3037878393 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe81c145c10 0x7fe81c0ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:48.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.917+0000 7fe821600640 1 --2- 172.21.15.159:0/3037878393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe81c0cfd80 0x7fe81c1438a0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:48.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.917+0000 7fe821600640 1 -- 172.21.15.159:0/3037878393 >> 172.21.15.159:0/3037878393 conn(0x7fe81c0d2240 msgr2=0x7fe81c0cd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:48.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.917+0000 7fe821600640 1 -- 172.21.15.159:0/3037878393 shutdown_connections 2024-04-04T06:08:48.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:48.917+0000 7fe821600640 1 -- 172.21.15.159:0/3037878393 wait complete. 2024-04-04T06:08:48.930 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:08:48.930 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:08:49.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.145+0000 7fd394e00640 1 Processor -- start 2024-04-04T06:08:49.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.145+0000 7fd394e00640 1 -- start start 2024-04-04T06:08:49.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.147+0000 7fd394e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd3900d0990 0x7fd3900aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:49.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.147+0000 7fd394e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3900b40b0 con 0x7fd390155550 2024-04-04T06:08:49.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.147+0000 7fd394e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3900b4390 con 0x7fd390155930 2024-04-04T06:08:49.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.147+0000 7fd394e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fd3900b4670 con 0x7fd3900d0990 2024-04-04T06:08:49.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.147+0000 7fd38e800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd3900d0990 0x7fd3900aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:49.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.147+0000 7fd38de00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fd390155550 0x7fd39015f800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:60098/0 (socket says 172.21.15.159:60098) 2024-04-04T06:08:49.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.147+0000 7fd38de00640 1 -- 172.21.15.159:0/2856041003 learned_addr learned my addr 172.21.15.159:0/2856041003 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:49.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.147+0000 7fd38e800640 1 -- 172.21.15.159:0/2856041003 >> v1:172.21.15.178:6789/0 conn(0x7fd390155930 legacy=0x7fd3900d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:49.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.147+0000 7fd38e800640 1 -- 172.21.15.159:0/2856041003 >> v1:172.21.15.159:6789/0 conn(0x7fd390155550 legacy=0x7fd39015f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:49.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.147+0000 7fd38e800640 1 -- 172.21.15.159:0/2856041003 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3900b4950 con 0x7fd3900d0990 2024-04-04T06:08:49.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.147+0000 7fd38d400640 1 -- 172.21.15.159:0/2856041003 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3548010348 0 0) 0x7fd3900b40b0 con 0x7fd390155550 2024-04-04T06:08:49.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.149+0000 7fd38e800640 1 --2- 172.21.15.159:0/2856041003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3900d0990 0x7fd3900aede0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd388009d90 tx=0x7fd38802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d5d0df8db5d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:49.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.149+0000 7fd38d400640 1 -- 172.21.15.159:0/2856041003 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd388039070 con 0x7fd3900d0990 2024-04-04T06:08:49.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.149+0000 7fd38d400640 1 -- 172.21.15.159:0/2856041003 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd38802cbd0 con 0x7fd3900d0990 2024-04-04T06:08:49.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.149+0000 7fd38d400640 1 -- 172.21.15.159:0/2856041003 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd388034440 con 0x7fd3900d0990 2024-04-04T06:08:49.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.149+0000 7fd394e00640 1 -- 172.21.15.159:0/2856041003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3900d0990 msgr2=0x7fd3900aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:49.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.149+0000 7fd394e00640 1 --2- 172.21.15.159:0/2856041003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3900d0990 0x7fd3900aede0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd388009d90 tx=0x7fd38802be20 comp rx=0 tx=0).stop 2024-04-04T06:08:49.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.149+0000 7fd394e00640 1 -- 172.21.15.159:0/2856041003 shutdown_connections 2024-04-04T06:08:49.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.150+0000 7fd394e00640 1 --2- 172.21.15.159:0/2856041003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3900d0990 0x7fd3900aede0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:49.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.150+0000 7fd394e00640 1 -- 172.21.15.159:0/2856041003 >> 172.21.15.159:0/2856041003 conn(0x7fd39006b750 msgr2=0x7fd39006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:49.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.150+0000 7fd394e00640 1 -- 172.21.15.159:0/2856041003 shutdown_connections 2024-04-04T06:08:49.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.150+0000 7fd394e00640 1 -- 172.21.15.159:0/2856041003 wait complete. 2024-04-04T06:08:49.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.151+0000 7fd394e00640 1 Processor -- start 2024-04-04T06:08:49.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.151+0000 7fd394e00640 1 -- start start 2024-04-04T06:08:49.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.152+0000 7fd394e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3900d0990 0x7fd39015a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:49.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.152+0000 7fd394e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd390155550 0x7fd390071880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:49.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.152+0000 7fd394e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd390155930 0x7fd3900af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:49.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.152+0000 7fd394e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd39015bba0 con 0x7fd390155930 2024-04-04T06:08:49.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.152+0000 7fd394e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd39015ba20 con 0x7fd3900d0990 2024-04-04T06:08:49.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.152+0000 7fd394e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd3900b3cf0 con 0x7fd390155550 2024-04-04T06:08:49.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.153+0000 7fd38e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd390155930 0x7fd3900af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:49.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.153+0000 7fd38e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd390155930 0x7fd3900af1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58232/0 (socket says 172.21.15.159:58232) 2024-04-04T06:08:49.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.153+0000 7fd38e800640 1 -- 172.21.15.159:0/2018550949 learned_addr learned my addr 172.21.15.159:0/2018550949 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:49.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.153+0000 7fd38de00640 1 --2- 172.21.15.159:0/2018550949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3900d0990 0x7fd39015a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:49.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.153+0000 7fd387e00640 1 --2- 172.21.15.159:0/2018550949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd390155550 0x7fd390071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:49.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.153+0000 7fd38e800640 1 -- 172.21.15.159:0/2018550949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd390155550 msgr2=0x7fd390071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:49.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.153+0000 7fd38e800640 1 --2- 172.21.15.159:0/2018550949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd390155550 0x7fd390071880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:49.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.153+0000 7fd38e800640 1 -- 172.21.15.159:0/2018550949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3900d0990 msgr2=0x7fd39015a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:49.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.153+0000 7fd38e800640 1 --2- 172.21.15.159:0/2018550949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3900d0990 0x7fd39015a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:49.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.153+0000 7fd38e800640 1 -- 172.21.15.159:0/2018550949 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39009a3e0 con 0x7fd390155930 2024-04-04T06:08:49.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.153+0000 7fd387e00640 1 --2- 172.21.15.159:0/2018550949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd390155550 0x7fd390071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:49.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.154+0000 7fd38e800640 1 --2- 172.21.15.159:0/2018550949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd390155930 0x7fd3900af1b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd388009eb0 tx=0x7fd388004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:49.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.154+0000 7fd386000640 1 -- 172.21.15.159:0/2018550949 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd388044030 con 0x7fd390155930 2024-04-04T06:08:49.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.154+0000 7fd386000640 1 -- 172.21.15.159:0/2018550949 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd38802c9d0 con 0x7fd390155930 2024-04-04T06:08:49.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.154+0000 7fd394e00640 1 -- 172.21.15.159:0/2018550949 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39009a6c0 con 0x7fd390155930 2024-04-04T06:08:49.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.154+0000 7fd386000640 1 -- 172.21.15.159:0/2018550949 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd388039040 con 0x7fd390155930 2024-04-04T06:08:49.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.154+0000 7fd394e00640 1 -- 172.21.15.159:0/2018550949 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd39009ac80 con 0x7fd390155930 2024-04-04T06:08:49.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.155+0000 7fd394e00640 1 -- 172.21.15.159:0/2018550949 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3900cf800 con 0x7fd390155930 2024-04-04T06:08:49.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.156+0000 7fd386000640 1 -- 172.21.15.159:0/2018550949 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd388047070 con 0x7fd390155930 2024-04-04T06:08:49.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.157+0000 7fd386000640 1 --2- 172.21.15.159:0/2018550949 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd36c06c8f0 0x7fd36c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:49.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.157+0000 7fd386000640 1 -- 172.21.15.159:0/2018550949 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7fd3880ae190 con 0x7fd390155930 2024-04-04T06:08:49.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.158+0000 7fd38de00640 1 --2- 172.21.15.159:0/2018550949 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd36c06c8f0 0x7fd36c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:49.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.159+0000 7fd38de00640 1 --2- 172.21.15.159:0/2018550949 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd36c06c8f0 0x7fd36c06edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd380000f60 tx=0x7fd380007b50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:49.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.161+0000 7fd386000640 1 -- 172.21.15.159:0/2018550949 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd3900cf800 con 0x7fd390155930 2024-04-04T06:08:49.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.361+0000 7fd394e00640 1 -- 172.21.15.159:0/2018550949 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd390155330 con 0x7fd390155930 2024-04-04T06:08:49.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.362+0000 7fd386000640 1 -- 172.21.15.159:0/2018550949 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) ==== 93+0+10 (secure 0 0 0) 0x7fd38804e070 con 0x7fd390155930 2024-04-04T06:08:49.356 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:08:49.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.365+0000 7fd394e00640 1 -- 172.21.15.159:0/2018550949 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd36c06c8f0 msgr2=0x7fd36c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:49.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.365+0000 7fd394e00640 1 --2- 172.21.15.159:0/2018550949 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd36c06c8f0 0x7fd36c06edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd380000f60 tx=0x7fd380007b50 comp rx=0 tx=0).stop 2024-04-04T06:08:49.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.365+0000 7fd394e00640 1 -- 172.21.15.159:0/2018550949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd390155930 msgr2=0x7fd3900af1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:49.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.365+0000 7fd394e00640 1 --2- 172.21.15.159:0/2018550949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd390155930 0x7fd3900af1b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd388009eb0 tx=0x7fd388004a40 comp rx=0 tx=0).stop 2024-04-04T06:08:49.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.365+0000 7fd394e00640 1 -- 172.21.15.159:0/2018550949 shutdown_connections 2024-04-04T06:08:49.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.365+0000 7fd394e00640 1 --2- 172.21.15.159:0/2018550949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd390155930 0x7fd3900af1b0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:49.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.365+0000 7fd394e00640 1 --2- 172.21.15.159:0/2018550949 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd390155550 0x7fd390071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:49.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.365+0000 7fd394e00640 1 --2- 172.21.15.159:0/2018550949 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd36c06c8f0 0x7fd36c06edb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:49.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.365+0000 7fd394e00640 1 --2- 172.21.15.159:0/2018550949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3900d0990 0x7fd39015a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:49.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.365+0000 7fd394e00640 1 -- 172.21.15.159:0/2018550949 >> 172.21.15.159:0/2018550949 conn(0x7fd39006b750 msgr2=0x7fd39015e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:49.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.365+0000 7fd394e00640 1 -- 172.21.15.159:0/2018550949 shutdown_connections 2024-04-04T06:08:49.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.365+0000 7fd394e00640 1 -- 172.21.15.159:0/2018550949 wait complete. 2024-04-04T06:08:49.377 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-04T06:08:49.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.590+0000 7f0d76c00640 1 Processor -- start 2024-04-04T06:08:49.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.590+0000 7f0d76c00640 1 -- start start 2024-04-04T06:08:49.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.592+0000 7f0d76c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d70155550 0x7f0d70155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:49.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.592+0000 7f0d76c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0d7015f800 0x7f0d700d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:49.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.592+0000 7f0d76c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0d700d07c0 0x7f0d700aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:49.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.592+0000 7f0d76c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f0d700d0c30 con 0x7f0d7015f800 2024-04-04T06:08:49.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.592+0000 7f0d76c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0d700b40b0 con 0x7f0d700d07c0 2024-04-04T06:08:49.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.592+0000 7f0d76c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0d700b4390 con 0x7f0d70155550 2024-04-04T06:08:49.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.592+0000 7f0d67e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0d7015f800 0x7f0d700d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:49.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.592+0000 7f0d67e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0d7015f800 0x7f0d700d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58242/0 (socket says 172.21.15.159:58242) 2024-04-04T06:08:49.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.592+0000 7f0d67e00640 1 -- 172.21.15.159:0/3154305723 learned_addr learned my addr 172.21.15.159:0/3154305723 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:49.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.593+0000 7f0d6fe00640 1 --2- 172.21.15.159:0/3154305723 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d70155550 0x7f0d70155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:49.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.593+0000 7f0d74c00640 1 --2- 172.21.15.159:0/3154305723 >> v2:172.21.15.178:3300/0 conn(0x7f0d700d07c0 0x7f0d700aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:49.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.593+0000 7f0d67e00640 1 -- 172.21.15.159:0/3154305723 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d70155550 msgr2=0x7f0d70155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:49.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.593+0000 7f0d67e00640 1 --2- 172.21.15.159:0/3154305723 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d70155550 0x7f0d70155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:49.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.593+0000 7f0d67e00640 1 -- 172.21.15.159:0/3154305723 >> v2:172.21.15.178:3300/0 conn(0x7f0d700d07c0 msgr2=0x7f0d700aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:49.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.593+0000 7f0d67e00640 1 --2- 172.21.15.159:0/3154305723 >> v2:172.21.15.178:3300/0 conn(0x7f0d700d07c0 0x7f0d700aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:49.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.593+0000 7f0d67e00640 1 -- 172.21.15.159:0/3154305723 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d700b4670 con 0x7f0d7015f800 2024-04-04T06:08:49.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.594+0000 7f0d67e00640 1 --2- 172.21.15.159:0/3154305723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d7015f800 0x7f0d700d0280 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0d700713d0 tx=0x7f0d5802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589ee419eca89f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:49.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.594+0000 7f0d6f400640 1 -- 172.21.15.159:0/3154305723 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d58035020 con 0x7f0d7015f800 2024-04-04T06:08:49.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.594+0000 7f0d6f400640 1 -- 172.21.15.159:0/3154305723 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d58031870 con 0x7f0d7015f800 2024-04-04T06:08:49.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.594+0000 7f0d6f400640 1 -- 172.21.15.159:0/3154305723 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d58031c40 con 0x7f0d7015f800 2024-04-04T06:08:49.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.595+0000 7f0d76c00640 1 -- 172.21.15.159:0/3154305723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d7015f800 msgr2=0x7f0d700d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:49.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.595+0000 7f0d76c00640 1 --2- 172.21.15.159:0/3154305723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d7015f800 0x7f0d700d0280 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0d700713d0 tx=0x7f0d5802b910 comp rx=0 tx=0).stop 2024-04-04T06:08:49.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.595+0000 7f0d76c00640 1 -- 172.21.15.159:0/3154305723 shutdown_connections 2024-04-04T06:08:49.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.595+0000 7f0d76c00640 1 --2- 172.21.15.159:0/3154305723 >> v2:172.21.15.178:3300/0 conn(0x7f0d700d07c0 0x7f0d700aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:49.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.595+0000 7f0d76c00640 1 --2- 172.21.15.159:0/3154305723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d7015f800 0x7f0d700d0280 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:49.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.595+0000 7f0d76c00640 1 --2- 172.21.15.159:0/3154305723 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d70155550 0x7f0d70155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:49.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.595+0000 7f0d76c00640 1 -- 172.21.15.159:0/3154305723 >> 172.21.15.159:0/3154305723 conn(0x7f0d7006b750 msgr2=0x7f0d7006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:49.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.595+0000 7f0d76c00640 1 -- 172.21.15.159:0/3154305723 shutdown_connections 2024-04-04T06:08:49.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.595+0000 7f0d76c00640 1 -- 172.21.15.159:0/3154305723 wait complete. 2024-04-04T06:08:49.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.597+0000 7f0d76c00640 1 Processor -- start 2024-04-04T06:08:49.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.597+0000 7f0d76c00640 1 -- start start 2024-04-04T06:08:49.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.598+0000 7f0d76c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d700d07c0 0x7f0d7009aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:49.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.598+0000 7f0d76c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d70155550 0x7f0d70154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:49.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.598+0000 7f0d76c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d7015f800 0x7f0d700adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:49.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.598+0000 7f0d76c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7015fc80 con 0x7f0d700d07c0 2024-04-04T06:08:49.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.598+0000 7f0d76c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0d700b4c30 con 0x7f0d70155550 2024-04-04T06:08:49.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.598+0000 7f0d76c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0d7015ba20 con 0x7f0d7015f800 2024-04-04T06:08:49.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.599+0000 7f0d6fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d700d07c0 0x7f0d7009aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:49.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.599+0000 7f0d6fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d700d07c0 0x7f0d7009aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58258/0 (socket says 172.21.15.159:58258) 2024-04-04T06:08:49.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.599+0000 7f0d6fe00640 1 -- 172.21.15.159:0/3377989664 learned_addr learned my addr 172.21.15.159:0/3377989664 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:49.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.599+0000 7f0d6fe00640 1 -- 172.21.15.159:0/3377989664 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d7015f800 msgr2=0x7f0d700adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:49.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.599+0000 7f0d6fe00640 1 --2- 172.21.15.159:0/3377989664 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d7015f800 0x7f0d700adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:49.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.599+0000 7f0d6fe00640 1 -- 172.21.15.159:0/3377989664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d70155550 msgr2=0x7f0d70154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:49.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.599+0000 7f0d6fe00640 1 --2- 172.21.15.159:0/3377989664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d70155550 0x7f0d70154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:49.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.599+0000 7f0d6fe00640 1 -- 172.21.15.159:0/3377989664 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d701541b0 con 0x7f0d700d07c0 2024-04-04T06:08:49.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.599+0000 7f0d6fe00640 1 --2- 172.21.15.159:0/3377989664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d700d07c0 0x7f0d7009aff0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000d5f0 tx=0x7f0d6000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:49.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.600+0000 7f0d6cc00640 1 -- 172.21.15.159:0/3377989664 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d600125f0 con 0x7f0d700d07c0 2024-04-04T06:08:49.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.600+0000 7f0d6cc00640 1 -- 172.21.15.159:0/3377989664 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d60019070 con 0x7f0d700d07c0 2024-04-04T06:08:49.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.600+0000 7f0d6cc00640 1 -- 172.21.15.159:0/3377989664 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d600129d0 con 0x7f0d700d07c0 2024-04-04T06:08:49.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.600+0000 7f0d76c00640 1 -- 172.21.15.159:0/3377989664 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7009b660 con 0x7f0d700d07c0 2024-04-04T06:08:49.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.600+0000 7f0d76c00640 1 -- 172.21.15.159:0/3377989664 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d7009bc20 con 0x7f0d700d07c0 2024-04-04T06:08:49.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.601+0000 7f0d76c00640 1 -- 172.21.15.159:0/3377989664 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d700685c0 con 0x7f0d700d07c0 2024-04-04T06:08:49.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.602+0000 7f0d6cc00640 1 -- 172.21.15.159:0/3377989664 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d60012b70 con 0x7f0d700d07c0 2024-04-04T06:08:49.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.603+0000 7f0d6cc00640 1 --2- 172.21.15.159:0/3377989664 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0d4c06c8f0 0x7f0d4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:49.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.603+0000 7f0d6cc00640 1 -- 172.21.15.159:0/3377989664 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f0d6008bb20 con 0x7f0d700d07c0 2024-04-04T06:08:49.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.604+0000 7f0d67e00640 1 --2- 172.21.15.159:0/3377989664 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0d4c06c8f0 0x7f0d4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:49.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.605+0000 7f0d67e00640 1 --2- 172.21.15.159:0/3377989664 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0d4c06c8f0 0x7f0d4c06edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0d58036000 tx=0x7f0d58030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:49.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.608+0000 7f0d6cc00640 1 -- 172.21.15.159:0/3377989664 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0d6005a2e0 con 0x7f0d700d07c0 2024-04-04T06:08:49.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:49.877+0000 7f0d76c00640 1 -- 172.21.15.159:0/3377989664 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f0d700d0370 con 0x7f0d700d07c0 2024-04-04T06:08:50.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.485+0000 7f0d6cc00640 1 -- 172.21.15.159:0/3377989664 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v178) ==== 172+0+0 (secure 0 0 0) 0x7f0d60059be0 con 0x7f0d700d07c0 2024-04-04T06:08:50.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.489+0000 7f0d76c00640 1 -- 172.21.15.159:0/3377989664 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0d4c06c8f0 msgr2=0x7f0d4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:50.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.489+0000 7f0d76c00640 1 --2- 172.21.15.159:0/3377989664 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0d4c06c8f0 0x7f0d4c06edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0d58036000 tx=0x7f0d58030000 comp rx=0 tx=0).stop 2024-04-04T06:08:50.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.489+0000 7f0d76c00640 1 -- 172.21.15.159:0/3377989664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d700d07c0 msgr2=0x7f0d7009aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:50.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.489+0000 7f0d76c00640 1 --2- 172.21.15.159:0/3377989664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d700d07c0 0x7f0d7009aff0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000d5f0 tx=0x7f0d6000dab0 comp rx=0 tx=0).stop 2024-04-04T06:08:50.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.489+0000 7f0d76c00640 1 -- 172.21.15.159:0/3377989664 shutdown_connections 2024-04-04T06:08:50.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.489+0000 7f0d76c00640 1 --2- 172.21.15.159:0/3377989664 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d7015f800 0x7f0d700adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:50.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.489+0000 7f0d76c00640 1 --2- 172.21.15.159:0/3377989664 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0d4c06c8f0 0x7f0d4c06edb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:50.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.489+0000 7f0d76c00640 1 --2- 172.21.15.159:0/3377989664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d70155550 0x7f0d70154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:50.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.489+0000 7f0d76c00640 1 --2- 172.21.15.159:0/3377989664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d700d07c0 0x7f0d7009aff0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:50.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.489+0000 7f0d76c00640 1 -- 172.21.15.159:0/3377989664 >> 172.21.15.159:0/3377989664 conn(0x7f0d7006b750 msgr2=0x7f0d7006d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:50.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.489+0000 7f0d76c00640 1 -- 172.21.15.159:0/3377989664 shutdown_connections 2024-04-04T06:08:50.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.489+0000 7f0d76c00640 1 -- 172.21.15.159:0/3377989664 wait complete. 2024-04-04T06:08:50.507 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-04T06:08:50.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.732+0000 7f3400e00640 1 Processor -- start 2024-04-04T06:08:50.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.732+0000 7f3400e00640 1 -- start start 2024-04-04T06:08:50.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.734+0000 7f3400e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f33fc13f190 0x7f33fc0cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:50.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.734+0000 7f3400e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f33fc0cc880 0x7f33fc0cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:50.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.734+0000 7f3400e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33fc0ce890 con 0x7f33fc06d620 2024-04-04T06:08:50.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.734+0000 7f3400e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f33fc0cd230 con 0x7f33fc0cc880 2024-04-04T06:08:50.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.734+0000 7f3400e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f33fc0ceb50 con 0x7f33fc13f190 2024-04-04T06:08:50.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.734+0000 7f33f2a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f33fc0cc880 0x7f33fc0cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:50.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.734+0000 7f33fbe00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f33fc06d620 0x7f33fc06da50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:60106/0 (socket says 172.21.15.159:60106) 2024-04-04T06:08:50.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.734+0000 7f33f2a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f33fc0cc880 0x7f33fc0cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58262/0 (socket says 172.21.15.159:58262) 2024-04-04T06:08:50.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.734+0000 7f33fbe00640 1 -- 172.21.15.159:0/396987579 learned_addr learned my addr 172.21.15.159:0/396987579 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:50.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.734+0000 7f33fb400640 1 --2- 172.21.15.159:0/396987579 >> v2:172.21.15.178:3300/0 conn(0x7f33fc13f190 0x7f33fc0cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:50.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.735+0000 7f33f2a00640 1 -- 172.21.15.159:0/396987579 >> v2:172.21.15.178:3300/0 conn(0x7f33fc13f190 msgr2=0x7f33fc0cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:50.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.735+0000 7f33f2a00640 1 --2- 172.21.15.159:0/396987579 >> v2:172.21.15.178:3300/0 conn(0x7f33fc13f190 0x7f33fc0cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:50.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.735+0000 7f33f2a00640 1 -- 172.21.15.159:0/396987579 >> v1:172.21.15.159:6789/0 conn(0x7f33fc06d620 legacy=0x7f33fc06da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:50.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.735+0000 7f33f2a00640 1 -- 172.21.15.159:0/396987579 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33fc0cee30 con 0x7f33fc0cc880 2024-04-04T06:08:50.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.736+0000 7f33f2a00640 1 --2- 172.21.15.159:0/396987579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33fc0cc880 0x7f33fc0cccf0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f33fc06f660 tx=0x7f33e402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c60a1f5e6d17aef server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:50.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.736+0000 7f33faa00640 1 -- 172.21.15.159:0/396987579 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e4035020 con 0x7f33fc0cc880 2024-04-04T06:08:50.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.736+0000 7f33faa00640 1 -- 172.21.15.159:0/396987579 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33e4031830 con 0x7f33fc0cc880 2024-04-04T06:08:50.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.736+0000 7f33faa00640 1 -- 172.21.15.159:0/396987579 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e4031c00 con 0x7f33fc0cc880 2024-04-04T06:08:50.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.737+0000 7f3400e00640 1 -- 172.21.15.159:0/396987579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33fc0cc880 msgr2=0x7f33fc0cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:50.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.737+0000 7f3400e00640 1 --2- 172.21.15.159:0/396987579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33fc0cc880 0x7f33fc0cccf0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f33fc06f660 tx=0x7f33e402b8d0 comp rx=0 tx=0).stop 2024-04-04T06:08:50.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.737+0000 7f3400e00640 1 -- 172.21.15.159:0/396987579 shutdown_connections 2024-04-04T06:08:50.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.737+0000 7f3400e00640 1 --2- 172.21.15.159:0/396987579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33fc0cc880 0x7f33fc0cccf0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:50.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.737+0000 7f3400e00640 1 --2- 172.21.15.159:0/396987579 >> v2:172.21.15.178:3300/0 conn(0x7f33fc13f190 0x7f33fc0cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:50.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.737+0000 7f3400e00640 1 -- 172.21.15.159:0/396987579 >> 172.21.15.159:0/396987579 conn(0x7f33fc0b1ba0 msgr2=0x7f33fc0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:50.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.737+0000 7f3400e00640 1 -- 172.21.15.159:0/396987579 shutdown_connections 2024-04-04T06:08:50.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.737+0000 7f3400e00640 1 -- 172.21.15.159:0/396987579 wait complete. 2024-04-04T06:08:50.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.739+0000 7f3400e00640 1 Processor -- start 2024-04-04T06:08:50.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.739+0000 7f3400e00640 1 -- start start 2024-04-04T06:08:50.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.740+0000 7f3400e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33fc06d620 0x7f33fc1562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:50.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.740+0000 7f3400e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33fc0cc880 0x7f33fc072430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:50.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.740+0000 7f3400e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33fc13f190 0x7f33fc072970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:50.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.740+0000 7f3400e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc13f570 con 0x7f33fc06d620 2024-04-04T06:08:50.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.740+0000 7f3400e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc160c70 con 0x7f33fc0cc880 2024-04-04T06:08:50.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.740+0000 7f3400e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f33fc0cfb70 con 0x7f33fc13f190 2024-04-04T06:08:50.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.740+0000 7f33fb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33fc06d620 0x7f33fc1562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:50.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.740+0000 7f33fb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33fc06d620 0x7f33fc1562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58274/0 (socket says 172.21.15.159:58274) 2024-04-04T06:08:50.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.740+0000 7f33fb400640 1 -- 172.21.15.159:0/1679581078 learned_addr learned my addr 172.21.15.159:0/1679581078 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:50.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.740+0000 7f33fb400640 1 -- 172.21.15.159:0/1679581078 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33fc13f190 msgr2=0x7f33fc072970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:50.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.740+0000 7f33fbe00640 1 --2- 172.21.15.159:0/1679581078 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33fc13f190 0x7f33fc072970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:50.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.740+0000 7f33f2a00640 1 --2- 172.21.15.159:0/1679581078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33fc0cc880 0x7f33fc072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:50.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.740+0000 7f33fb400640 1 --2- 172.21.15.159:0/1679581078 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33fc13f190 0x7f33fc072970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:50.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.740+0000 7f33fb400640 1 -- 172.21.15.159:0/1679581078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33fc0cc880 msgr2=0x7f33fc072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:50.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.740+0000 7f33fb400640 1 --2- 172.21.15.159:0/1679581078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33fc0cc880 0x7f33fc072430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:50.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.740+0000 7f33fb400640 1 -- 172.21.15.159:0/1679581078 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33fc1f6650 con 0x7f33fc06d620 2024-04-04T06:08:50.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.740+0000 7f33fbe00640 1 --2- 172.21.15.159:0/1679581078 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33fc13f190 0x7f33fc072970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:50.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.741+0000 7f33fb400640 1 --2- 172.21.15.159:0/1679581078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33fc06d620 0x7f33fc1562f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00d800 tx=0x7f33ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:50.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.741+0000 7f33f3e00640 1 -- 172.21.15.159:0/1679581078 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec00b6c0 con 0x7f33fc06d620 2024-04-04T06:08:50.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.741+0000 7f33f3e00640 1 -- 172.21.15.159:0/1679581078 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33ec019070 con 0x7f33fc06d620 2024-04-04T06:08:50.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.741+0000 7f3400e00640 1 -- 172.21.15.159:0/1679581078 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33fc1f6930 con 0x7f33fc06d620 2024-04-04T06:08:50.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.741+0000 7f33f3e00640 1 -- 172.21.15.159:0/1679581078 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec00ba80 con 0x7f33fc06d620 2024-04-04T06:08:50.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.741+0000 7f3400e00640 1 -- 172.21.15.159:0/1679581078 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33fc1f6ef0 con 0x7f33fc06d620 2024-04-04T06:08:50.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.742+0000 7f3400e00640 1 -- 172.21.15.159:0/1679581078 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33fc075940 con 0x7f33fc06d620 2024-04-04T06:08:50.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.743+0000 7f33f3e00640 1 -- 172.21.15.159:0/1679581078 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f33ec00bc20 con 0x7f33fc06d620 2024-04-04T06:08:50.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.744+0000 7f33f3e00640 1 --2- 172.21.15.159:0/1679581078 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f33d806c8f0 0x7f33d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:50.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.744+0000 7f33f3e00640 1 -- 172.21.15.159:0/1679581078 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f33ec08bd20 con 0x7f33fc06d620 2024-04-04T06:08:50.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.745+0000 7f33f2a00640 1 --2- 172.21.15.159:0/1679581078 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f33d806c8f0 0x7f33d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:50.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.746+0000 7f33f2a00640 1 --2- 172.21.15.159:0/1679581078 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f33d806c8f0 0x7f33d806edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f33e4036000 tx=0x7f33e4008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:50.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.749+0000 7f33f3e00640 1 -- 172.21.15.159:0/1679581078 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f33ec05a450 con 0x7f33fc06d620 2024-04-04T06:08:50.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.941+0000 7f3400e00640 1 -- 172.21.15.159:0/1679581078 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f33fc0cc2b0 con 0x7f33fc06d620 2024-04-04T06:08:50.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.946+0000 7f33f3e00640 1 -- 172.21.15.159:0/1679581078 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f33ec059d50 con 0x7f33fc06d620 2024-04-04T06:08:50.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.949+0000 7f3400e00640 1 -- 172.21.15.159:0/1679581078 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f33d806c8f0 msgr2=0x7f33d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:50.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.949+0000 7f3400e00640 1 --2- 172.21.15.159:0/1679581078 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f33d806c8f0 0x7f33d806edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f33e4036000 tx=0x7f33e4008030 comp rx=0 tx=0).stop 2024-04-04T06:08:50.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.949+0000 7f3400e00640 1 -- 172.21.15.159:0/1679581078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33fc06d620 msgr2=0x7f33fc1562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:50.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.949+0000 7f3400e00640 1 --2- 172.21.15.159:0/1679581078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33fc06d620 0x7f33fc1562f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00d800 tx=0x7f33ec00dcc0 comp rx=0 tx=0).stop 2024-04-04T06:08:50.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.949+0000 7f3400e00640 1 -- 172.21.15.159:0/1679581078 shutdown_connections 2024-04-04T06:08:50.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.949+0000 7f3400e00640 1 --2- 172.21.15.159:0/1679581078 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33fc13f190 0x7f33fc072970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:50.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.949+0000 7f3400e00640 1 --2- 172.21.15.159:0/1679581078 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f33d806c8f0 0x7f33d806edb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:50.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.949+0000 7f3400e00640 1 --2- 172.21.15.159:0/1679581078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33fc0cc880 0x7f33fc072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:50.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.949+0000 7f3400e00640 1 --2- 172.21.15.159:0/1679581078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33fc06d620 0x7f33fc1562f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:50.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.949+0000 7f3400e00640 1 -- 172.21.15.159:0/1679581078 >> 172.21.15.159:0/1679581078 conn(0x7f33fc0b1ba0 msgr2=0x7f33fc06da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:50.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.949+0000 7f3400e00640 1 -- 172.21.15.159:0/1679581078 shutdown_connections 2024-04-04T06:08:50.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:50.949+0000 7f3400e00640 1 -- 172.21.15.159:0/1679581078 wait complete. 2024-04-04T06:08:50.961 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-04T06:08:50.962 DEBUG:teuthology.orchestra.run.smithi159:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:08:51.067 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:08:51.189 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-04T06:08:51.189 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:08:51.189 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:08:51.247 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:08:51.247 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:08:51.247 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:08:51.303 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:08:51.304 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:08:51.304 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:08:51.361 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:08:51.361 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:08:51.361 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:08:51.416 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:08:51.416 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-04T06:08:51.416 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:08:51.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.640+0000 7f4ef3a00640 1 Processor -- start 2024-04-04T06:08:51.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.640+0000 7f4ef3a00640 1 -- start start 2024-04-04T06:08:51.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.642+0000 7f4ef3a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4eec0ad120 0x7f4eec0d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:51.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.642+0000 7f4ef3a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4eec0ad580 0x7f4eec0d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:51.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.642+0000 7f4ef3a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eec071b60 con 0x7f4eec161b00 2024-04-04T06:08:51.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.642+0000 7f4ef3a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4eec161ee0 con 0x7f4eec0ad580 2024-04-04T06:08:51.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.642+0000 7f4ef3a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4eec0b3450 con 0x7f4eec0ad120 2024-04-04T06:08:51.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.643+0000 7f4ef1000640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f4eec161b00 0x7f4eec162ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.159:39136/0 (socket says 172.21.15.159:39136) 2024-04-04T06:08:51.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.643+0000 7f4ef1000640 1 -- 172.21.15.159:0/2309309482 learned_addr learned my addr 172.21.15.159:0/2309309482 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:51.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.643+0000 7f4ef1a00640 1 --2- 172.21.15.159:0/2309309482 >> v2:172.21.15.178:3300/0 conn(0x7f4eec0ad580 0x7f4eec0d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:51.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.643+0000 7f4eebe00640 1 --2- 172.21.15.159:0/2309309482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4eec0ad120 0x7f4eec0d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:51.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.643+0000 7f4eeb400640 1 -- 172.21.15.159:0/2309309482 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1165549359 0 0) 0x7f4eec071b60 con 0x7f4eec161b00 2024-04-04T06:08:51.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.643+0000 7f4eeb400640 1 -- 172.21.15.159:0/2309309482 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ed0003520 con 0x7f4eec161b00 2024-04-04T06:08:51.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.644+0000 7f4ef1a00640 1 -- 172.21.15.159:0/2309309482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4eec0ad120 msgr2=0x7f4eec0d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:51.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.644+0000 7f4ef1a00640 1 --2- 172.21.15.159:0/2309309482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4eec0ad120 0x7f4eec0d0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:51.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.644+0000 7f4ef1a00640 1 -- 172.21.15.159:0/2309309482 >> v1:172.21.15.178:6789/0 conn(0x7f4eec161b00 legacy=0x7f4eec162ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:51.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.644+0000 7f4ef1a00640 1 -- 172.21.15.159:0/2309309482 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eec071e40 con 0x7f4eec0ad580 2024-04-04T06:08:51.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.645+0000 7f4ef1a00640 1 --2- 172.21.15.159:0/2309309482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eec0ad580 0x7f4eec0d0b90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40097a0 tx=0x7f4ee402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a30054b0c97180e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:51.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.645+0000 7f4eeb400640 1 -- 172.21.15.159:0/2309309482 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4031780 con 0x7f4eec0ad580 2024-04-04T06:08:51.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.646+0000 7f4eeb400640 1 -- 172.21.15.159:0/2309309482 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee4037070 con 0x7f4eec0ad580 2024-04-04T06:08:51.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.646+0000 7f4eeb400640 1 -- 172.21.15.159:0/2309309482 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4031b20 con 0x7f4eec0ad580 2024-04-04T06:08:51.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.646+0000 7f4ef3a00640 1 -- 172.21.15.159:0/2309309482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eec0ad580 msgr2=0x7f4eec0d0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:51.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.646+0000 7f4ef3a00640 1 --2- 172.21.15.159:0/2309309482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eec0ad580 0x7f4eec0d0b90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40097a0 tx=0x7f4ee402b820 comp rx=0 tx=0).stop 2024-04-04T06:08:51.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.646+0000 7f4ef3a00640 1 -- 172.21.15.159:0/2309309482 shutdown_connections 2024-04-04T06:08:51.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.646+0000 7f4ef3a00640 1 --2- 172.21.15.159:0/2309309482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eec0ad580 0x7f4eec0d0b90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:51.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.646+0000 7f4ef3a00640 1 --2- 172.21.15.159:0/2309309482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4eec0ad120 0x7f4eec0d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:51.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.646+0000 7f4ef3a00640 1 -- 172.21.15.159:0/2309309482 >> 172.21.15.159:0/2309309482 conn(0x7f4eec06b990 msgr2=0x7f4eec06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:51.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.646+0000 7f4ef3a00640 1 -- 172.21.15.159:0/2309309482 shutdown_connections 2024-04-04T06:08:51.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.646+0000 7f4ef3a00640 1 -- 172.21.15.159:0/2309309482 wait complete. 2024-04-04T06:08:51.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.648+0000 7f4ef3a00640 1 Processor -- start 2024-04-04T06:08:51.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.648+0000 7f4ef3a00640 1 -- start start 2024-04-04T06:08:51.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.649+0000 7f4ef3a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eec0ad120 0x7f4eec09c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:51.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.649+0000 7f4ef3a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4eec0ad580 0x7f4eec1f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:51.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.649+0000 7f4ef3a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4eec161b00 0x7f4eec0ade10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:51.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.649+0000 7f4ef3a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec0717f0 con 0x7f4eec161b00 2024-04-04T06:08:51.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.649+0000 7f4ef3a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec0b3730 con 0x7f4eec0ad120 2024-04-04T06:08:51.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.649+0000 7f4ef3a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4eec0b5020 con 0x7f4eec0ad580 2024-04-04T06:08:51.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.649+0000 7f4ef1a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4eec161b00 0x7f4eec0ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:51.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.649+0000 7f4ef1a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4eec161b00 0x7f4eec0ade10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58290/0 (socket says 172.21.15.159:58290) 2024-04-04T06:08:51.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.649+0000 7f4ef1a00640 1 -- 172.21.15.159:0/3615169843 learned_addr learned my addr 172.21.15.159:0/3615169843 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:51.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.650+0000 7f4eebe00640 1 --2- 172.21.15.159:0/3615169843 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4eec0ad580 0x7f4eec1f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:51.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.650+0000 7f4ef1000640 1 --2- 172.21.15.159:0/3615169843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eec0ad120 0x7f4eec09c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:51.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.650+0000 7f4ef1a00640 1 -- 172.21.15.159:0/3615169843 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4eec0ad580 msgr2=0x7f4eec1f45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:51.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.650+0000 7f4ef1a00640 1 --2- 172.21.15.159:0/3615169843 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4eec0ad580 0x7f4eec1f45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:51.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.650+0000 7f4ef1a00640 1 -- 172.21.15.159:0/3615169843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eec0ad120 msgr2=0x7f4eec09c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:51.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.650+0000 7f4ef1a00640 1 --2- 172.21.15.159:0/3615169843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eec0ad120 0x7f4eec09c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:51.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.650+0000 7f4ef1a00640 1 -- 172.21.15.159:0/3615169843 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eec0c9f70 con 0x7f4eec161b00 2024-04-04T06:08:51.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.650+0000 7f4eebe00640 1 --2- 172.21.15.159:0/3615169843 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4eec0ad580 0x7f4eec1f45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:51.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.650+0000 7f4ef1000640 1 --2- 172.21.15.159:0/3615169843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eec0ad120 0x7f4eec09c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:51.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.650+0000 7f4ef1a00640 1 --2- 172.21.15.159:0/3615169843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4eec161b00 0x7f4eec0ade10 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4034040 tx=0x7f4ee40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:51.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.650+0000 7f4ee8c00640 1 -- 172.21.15.159:0/3615169843 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4006e10 con 0x7f4eec161b00 2024-04-04T06:08:51.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.650+0000 7f4ee8c00640 1 -- 172.21.15.159:0/3615169843 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee4031780 con 0x7f4eec161b00 2024-04-04T06:08:51.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.650+0000 7f4ee8c00640 1 -- 172.21.15.159:0/3615169843 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4031b70 con 0x7f4eec161b00 2024-04-04T06:08:51.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.651+0000 7f4ef3a00640 1 -- 172.21.15.159:0/3615169843 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eec09c580 con 0x7f4eec161b00 2024-04-04T06:08:51.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.651+0000 7f4ef3a00640 1 -- 172.21.15.159:0/3615169843 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eec0c9b10 con 0x7f4eec161b00 2024-04-04T06:08:51.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.652+0000 7f4ef3a00640 1 -- 172.21.15.159:0/3615169843 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eec0685c0 con 0x7f4eec161b00 2024-04-04T06:08:51.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.653+0000 7f4ee8c00640 1 -- 172.21.15.159:0/3615169843 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ee4031d10 con 0x7f4eec161b00 2024-04-04T06:08:51.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.653+0000 7f4ee8c00640 1 --2- 172.21.15.159:0/3615169843 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ecc06c8f0 0x7f4ecc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:51.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.654+0000 7f4ee8c00640 1 -- 172.21.15.159:0/3615169843 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7f4ee40addc0 con 0x7f4eec161b00 2024-04-04T06:08:51.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.654+0000 7f4ef1000640 1 --2- 172.21.15.159:0/3615169843 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ecc06c8f0 0x7f4ecc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:51.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.655+0000 7f4ef1000640 1 --2- 172.21.15.159:0/3615169843 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ecc06c8f0 0x7f4ecc06edb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4edc005c00 tx=0x7f4edc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:51.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.658+0000 7f4ee8c00640 1 -- 172.21.15.159:0/3615169843 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4ee402f050 con 0x7f4eec161b00 2024-04-04T06:08:51.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.898+0000 7f4ef3a00640 1 -- 172.21.15.159:0/3615169843 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4eec09c7b0 con 0x7f4eec161b00 2024-04-04T06:08:51.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.900+0000 7f4ee8c00640 1 -- 172.21.15.159:0/3615169843 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+3878 (secure 0 0 0) 0x7f4ee403a380 con 0x7f4eec161b00 2024-04-04T06:08:51.894 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:51.894 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3765578393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3765578393},{"type":"v1","addr":"172.21.15.178:6835","nonce":3765578393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3260412913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3260412913},{"type":"v1","addr":"172.21.15.178:6837","nonce":3260412913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5580,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/3149916718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3149916718},{"type":"v1","addr":"172.21.15.159:6833","nonce":3149916718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5967,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/903437660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":903437660},{"type":"v1","addr":"172.21.15.159:6835","nonce":903437660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[]} 2024-04-04T06:08:51.894 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 27 2024-04-04T06:08:51.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.902+0000 7f4ef3a00640 1 -- 172.21.15.159:0/3615169843 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ecc06c8f0 msgr2=0x7f4ecc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:51.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.902+0000 7f4ef3a00640 1 --2- 172.21.15.159:0/3615169843 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ecc06c8f0 0x7f4ecc06edb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4edc005c00 tx=0x7f4edc009040 comp rx=0 tx=0).stop 2024-04-04T06:08:51.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.902+0000 7f4ef3a00640 1 -- 172.21.15.159:0/3615169843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4eec161b00 msgr2=0x7f4eec0ade10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:51.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.902+0000 7f4ef3a00640 1 --2- 172.21.15.159:0/3615169843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4eec161b00 0x7f4eec0ade10 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4034040 tx=0x7f4ee40046c0 comp rx=0 tx=0).stop 2024-04-04T06:08:51.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.902+0000 7f4ef3a00640 1 -- 172.21.15.159:0/3615169843 shutdown_connections 2024-04-04T06:08:51.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.902+0000 7f4ef3a00640 1 --2- 172.21.15.159:0/3615169843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4eec161b00 0x7f4eec0ade10 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:51.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.902+0000 7f4ef3a00640 1 --2- 172.21.15.159:0/3615169843 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4eec0ad580 0x7f4eec1f45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:51.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.903+0000 7f4ef3a00640 1 --2- 172.21.15.159:0/3615169843 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ecc06c8f0 0x7f4ecc06edb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:51.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.903+0000 7f4ef3a00640 1 --2- 172.21.15.159:0/3615169843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eec0ad120 0x7f4eec09c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:51.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.903+0000 7f4ef3a00640 1 -- 172.21.15.159:0/3615169843 >> 172.21.15.159:0/3615169843 conn(0x7f4eec06b990 msgr2=0x7f4eec0b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:51.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.903+0000 7f4ef3a00640 1 -- 172.21.15.159:0/3615169843 shutdown_connections 2024-04-04T06:08:51.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:51.903+0000 7f4ef3a00640 1 -- 172.21.15.159:0/3615169843 wait complete. 2024-04-04T06:08:51.915 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-04T06:08:51.916 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-04T06:08:51.916 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-04T06:08:51.916 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:08:51.916 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-04T06:08:51.916 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-04T06:08:51.916 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-04T06:08:51.916 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:08:51.917 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-04T06:08:51.917 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-04T06:08:51.917 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-04T06:08:51.917 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:08:51.917 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-04T06:08:51.917 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-04T06:08:51.917 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-04T06:08:51.917 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:08:51.933 INFO:tasks.ceph.mds.a:Stopped 2024-04-04T06:08:51.933 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-04T06:08:51.942 INFO:tasks.ceph.mds.c:Stopped 2024-04-04T06:08:51.942 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-04T06:08:51.944 INFO:tasks.ceph.mds.d:Stopped 2024-04-04T06:08:51.944 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-04T06:08:51.946 INFO:tasks.ceph.mds.b:Stopped 2024-04-04T06:08:51.946 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-04T06:08:51.989 INFO:tasks.ceph.mds.a:Started 2024-04-04T06:08:51.990 DEBUG:teuthology.parallel:result is None 2024-04-04T06:08:51.992 INFO:tasks.ceph.mds.d:Started 2024-04-04T06:08:51.993 DEBUG:teuthology.parallel:result is None 2024-04-04T06:08:51.996 INFO:tasks.ceph.mds.c:Started 2024-04-04T06:08:51.996 DEBUG:teuthology.parallel:result is None 2024-04-04T06:08:51.998 INFO:tasks.ceph.mds.b:Started 2024-04-04T06:08:51.999 DEBUG:teuthology.parallel:result is None 2024-04-04T06:08:51.999 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-04T06:08:52.146 INFO:tasks.ceph.mds.a.smithi159.stdout:starting mds.a at 2024-04-04T06:08:52.146 INFO:tasks.ceph.mds.c.smithi159.stdout:starting mds.c at 2024-04-04T06:08:52.154 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-04T06:08:52.157 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-04T06:08:52.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.190+0000 7fb96b600640 1 Processor -- start 2024-04-04T06:08:52.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.190+0000 7fb96b600640 1 -- start start 2024-04-04T06:08:52.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.191+0000 7fb96b600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb964155550 0x7fb964155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:52.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.191+0000 7fb96b600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb96415f800 0x7fb9640d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:52.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.191+0000 7fb96b600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9640b40b0 con 0x7fb9640d07c0 2024-04-04T06:08:52.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.191+0000 7fb96b600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb9640d0c30 con 0x7fb96415f800 2024-04-04T06:08:52.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.191+0000 7fb96b600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb9640b4390 con 0x7fb964155550 2024-04-04T06:08:52.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.192+0000 7fb969600640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fb9640d07c0 0x7fb9640aede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:60134/0 (socket says 172.21.15.159:60134) 2024-04-04T06:08:52.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.192+0000 7fb969600640 1 -- 172.21.15.159:0/1952409638 learned_addr learned my addr 172.21.15.159:0/1952409638 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:52.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.192+0000 7fb968c00640 1 --2- 172.21.15.159:0/1952409638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb964155550 0x7fb964155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:52.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.192+0000 7fb95be00640 1 --2- 172.21.15.159:0/1952409638 >> v2:172.21.15.178:3300/0 conn(0x7fb96415f800 0x7fb9640d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:52.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.192+0000 7fb95be00640 1 -- 172.21.15.159:0/1952409638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb964155550 msgr2=0x7fb964155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:52.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.192+0000 7fb95be00640 1 --2- 172.21.15.159:0/1952409638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb964155550 0x7fb964155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:52.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.192+0000 7fb95be00640 1 -- 172.21.15.159:0/1952409638 >> v1:172.21.15.159:6789/0 conn(0x7fb9640d07c0 legacy=0x7fb9640aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:52.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.192+0000 7fb95be00640 1 -- 172.21.15.159:0/1952409638 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9640b4670 con 0x7fb96415f800 2024-04-04T06:08:52.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.192+0000 7fb963e00640 1 -- 172.21.15.159:0/1952409638 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 805309660 0 0) 0x7fb9640b40b0 con 0x7fb9640d07c0 2024-04-04T06:08:52.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.194+0000 7fb95be00640 1 --2- 172.21.15.159:0/1952409638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb96415f800 0x7fb9640d0280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0097a0 tx=0x7fb94c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40af489164ce75dc server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:52.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.194+0000 7fb963e00640 1 -- 172.21.15.159:0/1952409638 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c035020 con 0x7fb96415f800 2024-04-04T06:08:52.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.194+0000 7fb963e00640 1 -- 172.21.15.159:0/1952409638 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb94c031810 con 0x7fb96415f800 2024-04-04T06:08:52.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.194+0000 7fb963e00640 1 -- 172.21.15.159:0/1952409638 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c031be0 con 0x7fb96415f800 2024-04-04T06:08:52.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.194+0000 7fb96b600640 1 -- 172.21.15.159:0/1952409638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb96415f800 msgr2=0x7fb9640d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:52.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.194+0000 7fb96b600640 1 --2- 172.21.15.159:0/1952409638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb96415f800 0x7fb9640d0280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0097a0 tx=0x7fb94c02b8b0 comp rx=0 tx=0).stop 2024-04-04T06:08:52.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.195+0000 7fb96b600640 1 -- 172.21.15.159:0/1952409638 shutdown_connections 2024-04-04T06:08:52.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.195+0000 7fb96b600640 1 --2- 172.21.15.159:0/1952409638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb96415f800 0x7fb9640d0280 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:52.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.195+0000 7fb96b600640 1 --2- 172.21.15.159:0/1952409638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb964155550 0x7fb964155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:52.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.195+0000 7fb96b600640 1 -- 172.21.15.159:0/1952409638 >> 172.21.15.159:0/1952409638 conn(0x7fb96406b750 msgr2=0x7fb96406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:52.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.195+0000 7fb96b600640 1 -- 172.21.15.159:0/1952409638 shutdown_connections 2024-04-04T06:08:52.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.195+0000 7fb96b600640 1 -- 172.21.15.159:0/1952409638 wait complete. 2024-04-04T06:08:52.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.196+0000 7fb96b600640 1 Processor -- start 2024-04-04T06:08:52.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.196+0000 7fb96b600640 1 -- start start 2024-04-04T06:08:52.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.197+0000 7fb96b600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9640d07c0 0x7fb964099990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:52.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.197+0000 7fb96b600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb964155550 0x7fb9641444b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:52.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.197+0000 7fb96b600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb96415f800 0x7fb9640af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:52.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.197+0000 7fb96b600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb96415fc80 con 0x7fb9640d07c0 2024-04-04T06:08:52.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.197+0000 7fb96b600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb9640b4c30 con 0x7fb964155550 2024-04-04T06:08:52.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.197+0000 7fb96b600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb96415ba20 con 0x7fb96415f800 2024-04-04T06:08:52.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.198+0000 7fb968c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9640d07c0 0x7fb964099990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:52.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.198+0000 7fb968c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9640d07c0 0x7fb964099990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58338/0 (socket says 172.21.15.159:58338) 2024-04-04T06:08:52.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.198+0000 7fb968c00640 1 -- 172.21.15.159:0/1717238692 learned_addr learned my addr 172.21.15.159:0/1717238692 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:52.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.198+0000 7fb969600640 1 --2- 172.21.15.159:0/1717238692 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb96415f800 0x7fb9640af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:52.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.198+0000 7fb95be00640 1 --2- 172.21.15.159:0/1717238692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb964155550 0x7fb9641444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:52.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.198+0000 7fb968c00640 1 -- 172.21.15.159:0/1717238692 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb96415f800 msgr2=0x7fb9640af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:52.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.198+0000 7fb968c00640 1 --2- 172.21.15.159:0/1717238692 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb96415f800 0x7fb9640af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:52.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.198+0000 7fb968c00640 1 -- 172.21.15.159:0/1717238692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb964155550 msgr2=0x7fb9641444b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:52.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.198+0000 7fb968c00640 1 --2- 172.21.15.159:0/1717238692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb964155550 0x7fb9641444b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:52.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.198+0000 7fb968c00640 1 -- 172.21.15.159:0/1717238692 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb964072590 con 0x7fb9640d07c0 2024-04-04T06:08:52.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.198+0000 7fb968c00640 1 --2- 172.21.15.159:0/1717238692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9640d07c0 0x7fb964099990 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb95400e9b0 tx=0x7fb95400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:52.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.199+0000 7fb961600640 1 -- 172.21.15.159:0/1717238692 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb954017070 con 0x7fb9640d07c0 2024-04-04T06:08:52.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.199+0000 7fb961600640 1 -- 172.21.15.159:0/1717238692 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb95400c9a0 con 0x7fb9640d07c0 2024-04-04T06:08:52.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.199+0000 7fb961600640 1 -- 172.21.15.159:0/1717238692 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95400cd90 con 0x7fb9640d07c0 2024-04-04T06:08:52.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.199+0000 7fb96b600640 1 -- 172.21.15.159:0/1717238692 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb964154230 con 0x7fb9640d07c0 2024-04-04T06:08:52.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.199+0000 7fb96b600640 1 -- 172.21.15.159:0/1717238692 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9641547f0 con 0x7fb9640d07c0 2024-04-04T06:08:52.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.200+0000 7fb96b600640 1 -- 172.21.15.159:0/1717238692 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9640685c0 con 0x7fb9640d07c0 2024-04-04T06:08:52.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.201+0000 7fb961600640 1 -- 172.21.15.159:0/1717238692 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb95401a070 con 0x7fb9640d07c0 2024-04-04T06:08:52.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.202+0000 7fb961600640 1 --2- 172.21.15.159:0/1717238692 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb94006c8f0 0x7fb94006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:52.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.202+0000 7fb961600640 1 -- 172.21.15.159:0/1717238692 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4943+0+0 (secure 0 0 0) 0x7fb95408bb00 con 0x7fb9640d07c0 2024-04-04T06:08:52.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.202+0000 7fb95be00640 1 --2- 172.21.15.159:0/1717238692 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb94006c8f0 0x7fb94006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:52.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.203+0000 7fb95be00640 1 --2- 172.21.15.159:0/1717238692 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb94006c8f0 0x7fb94006edb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb94c036000 tx=0x7fb94c03d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:52.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.206+0000 7fb961600640 1 -- 172.21.15.159:0/1717238692 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb954059bd0 con 0x7fb9640d07c0 2024-04-04T06:08:52.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.419+0000 7fb96b600640 1 -- 172.21.15.159:0/1717238692 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fb9640b3ad0 con 0x7fb9640d07c0 2024-04-04T06:08:52.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.515+0000 7fb961600640 1 -- 172.21.15.159:0/1717238692 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v53) ==== 97+0+0 (secure 0 0 0) 0x7fb95405ded0 con 0x7fb9640d07c0 2024-04-04T06:08:52.509 INFO:teuthology.orchestra.run.smithi159.stderr: removed all blocklist entries 2024-04-04T06:08:52.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.518+0000 7fb96b600640 1 -- 172.21.15.159:0/1717238692 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb94006c8f0 msgr2=0x7fb94006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:52.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.518+0000 7fb96b600640 1 --2- 172.21.15.159:0/1717238692 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb94006c8f0 0x7fb94006edb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb94c036000 tx=0x7fb94c03d040 comp rx=0 tx=0).stop 2024-04-04T06:08:52.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.518+0000 7fb96b600640 1 -- 172.21.15.159:0/1717238692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9640d07c0 msgr2=0x7fb964099990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:52.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.518+0000 7fb96b600640 1 --2- 172.21.15.159:0/1717238692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9640d07c0 0x7fb964099990 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb95400e9b0 tx=0x7fb95400ee70 comp rx=0 tx=0).stop 2024-04-04T06:08:52.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.518+0000 7fb96b600640 1 -- 172.21.15.159:0/1717238692 shutdown_connections 2024-04-04T06:08:52.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.518+0000 7fb96b600640 1 --2- 172.21.15.159:0/1717238692 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb96415f800 0x7fb9640af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:52.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.518+0000 7fb96b600640 1 --2- 172.21.15.159:0/1717238692 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb94006c8f0 0x7fb94006edb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:52.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.518+0000 7fb96b600640 1 --2- 172.21.15.159:0/1717238692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb964155550 0x7fb9641444b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:52.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.518+0000 7fb96b600640 1 --2- 172.21.15.159:0/1717238692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9640d07c0 0x7fb964099990 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:52.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.518+0000 7fb96b600640 1 -- 172.21.15.159:0/1717238692 >> 172.21.15.159:0/1717238692 conn(0x7fb96406b750 msgr2=0x7fb96406e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:52.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.518+0000 7fb96b600640 1 -- 172.21.15.159:0/1717238692 shutdown_connections 2024-04-04T06:08:52.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.518+0000 7fb96b600640 1 -- 172.21.15.159:0/1717238692 wait complete. 2024-04-04T06:08:52.531 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-04T06:08:52.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.737+0000 7f9109c00640 1 Processor -- start 2024-04-04T06:08:52.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.737+0000 7f9109c00640 1 -- start start 2024-04-04T06:08:52.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.738+0000 7f9109c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f91040d0990 0x7f91040aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:52.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.738+0000 7f9109c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91040af2e0 0x7f91041600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:52.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.738+0000 7f9109c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91040afab0 con 0x7f9104153580 2024-04-04T06:08:52.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.738+0000 7f9109c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f910415cc80 con 0x7f91040d0990 2024-04-04T06:08:52.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.738+0000 7f9109c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f910415da00 con 0x7f91040af2e0 2024-04-04T06:08:52.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.739+0000 7f90fa000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f91040d0990 0x7f91040aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:52.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.739+0000 7f90fa000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f91040d0990 0x7f91040aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58354/0 (socket says 172.21.15.159:58354) 2024-04-04T06:08:52.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.739+0000 7f90fa000640 1 -- 172.21.15.159:0/1503795413 learned_addr learned my addr 172.21.15.159:0/1503795413 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:52.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.739+0000 7f9103400640 1 --2- 172.21.15.159:0/1503795413 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91040af2e0 0x7f91041600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:52.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.739+0000 7f90fa000640 1 -- 172.21.15.159:0/1503795413 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91040af2e0 msgr2=0x7f91041600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:52.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.739+0000 7f90fa000640 1 --2- 172.21.15.159:0/1503795413 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91040af2e0 0x7f91041600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:52.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.739+0000 7f90fa000640 1 -- 172.21.15.159:0/1503795413 >> v1:172.21.15.178:6789/0 conn(0x7f9104153580 legacy=0x7f91040d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:52.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.739+0000 7f90fa000640 1 -- 172.21.15.159:0/1503795413 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910415cf20 con 0x7f91040d0990 2024-04-04T06:08:52.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.740+0000 7f90fa000640 1 --2- 172.21.15.159:0/1503795413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91040d0990 0x7f91040aeda0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009870 tx=0x7f90ec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aaca50f6e60fec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:52.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.741+0000 7f9102000640 1 -- 172.21.15.159:0/1503795413 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ec035020 con 0x7f91040d0990 2024-04-04T06:08:52.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.741+0000 7f9102000640 1 -- 172.21.15.159:0/1503795413 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90ec031640 con 0x7f91040d0990 2024-04-04T06:08:52.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.741+0000 7f9102000640 1 -- 172.21.15.159:0/1503795413 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ec02d040 con 0x7f91040d0990 2024-04-04T06:08:52.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.741+0000 7f9109c00640 1 -- 172.21.15.159:0/1503795413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91040d0990 msgr2=0x7f91040aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:52.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.741+0000 7f9109c00640 1 --2- 172.21.15.159:0/1503795413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91040d0990 0x7f91040aeda0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f90ec009870 tx=0x7f90ec02b8f0 comp rx=0 tx=0).stop 2024-04-04T06:08:52.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.742+0000 7f9109c00640 1 -- 172.21.15.159:0/1503795413 shutdown_connections 2024-04-04T06:08:52.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.742+0000 7f9109c00640 1 --2- 172.21.15.159:0/1503795413 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91040af2e0 0x7f91041600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:52.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.742+0000 7f9109c00640 1 --2- 172.21.15.159:0/1503795413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91040d0990 0x7f91040aeda0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:52.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.742+0000 7f9109c00640 1 -- 172.21.15.159:0/1503795413 >> 172.21.15.159:0/1503795413 conn(0x7f910406b750 msgr2=0x7f910406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:52.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.742+0000 7f9109c00640 1 -- 172.21.15.159:0/1503795413 shutdown_connections 2024-04-04T06:08:52.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.742+0000 7f9109c00640 1 -- 172.21.15.159:0/1503795413 wait complete. 2024-04-04T06:08:52.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.744+0000 7f9109c00640 1 Processor -- start 2024-04-04T06:08:52.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.744+0000 7f9109c00640 1 -- start start 2024-04-04T06:08:52.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.745+0000 7f9109c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91040af2e0 0x7f91040cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:52.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.745+0000 7f9109c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91040d0990 0x7f9104154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:52.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.745+0000 7f9109c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9104153580 0x7f9104156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:52.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.745+0000 7f9109c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9104153960 con 0x7f9104153580 2024-04-04T06:08:52.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.745+0000 7f9109c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f910415d4e0 con 0x7f91040af2e0 2024-04-04T06:08:52.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.745+0000 7f9109c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f91040af710 con 0x7f91040d0990 2024-04-04T06:08:52.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.745+0000 7f9103400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9104153580 0x7f9104156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:52.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.745+0000 7f9103400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9104153580 0x7f9104156340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58364/0 (socket says 172.21.15.159:58364) 2024-04-04T06:08:52.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.745+0000 7f9103400640 1 -- 172.21.15.159:0/475019574 learned_addr learned my addr 172.21.15.159:0/475019574 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:52.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.745+0000 7f90fa000640 1 --2- 172.21.15.159:0/475019574 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91040d0990 0x7f9104154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:52.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.745+0000 7f9102a00640 1 --2- 172.21.15.159:0/475019574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91040af2e0 0x7f91040cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:52.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.745+0000 7f9103400640 1 -- 172.21.15.159:0/475019574 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91040d0990 msgr2=0x7f9104154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:52.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.745+0000 7f9103400640 1 --2- 172.21.15.159:0/475019574 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91040d0990 0x7f9104154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:52.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.745+0000 7f9103400640 1 -- 172.21.15.159:0/475019574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91040af2e0 msgr2=0x7f91040cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:52.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.745+0000 7f9103400640 1 --2- 172.21.15.159:0/475019574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91040af2e0 0x7f91040cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:52.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.745+0000 7f9103400640 1 -- 172.21.15.159:0/475019574 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9104154cc0 con 0x7f9104153580 2024-04-04T06:08:52.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.745+0000 7f9102a00640 1 --2- 172.21.15.159:0/475019574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91040af2e0 0x7f91040cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:52.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.746+0000 7f9103400640 1 --2- 172.21.15.159:0/475019574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9104153580 0x7f9104156340 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f90fc007e60 tx=0x7f90fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:52.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.746+0000 7f90fb400640 1 -- 172.21.15.159:0/475019574 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc014420 con 0x7f9104153580 2024-04-04T06:08:52.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.746+0000 7f90fb400640 1 -- 172.21.15.159:0/475019574 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90fc019070 con 0x7f9104153580 2024-04-04T06:08:52.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.746+0000 7f9109c00640 1 -- 172.21.15.159:0/475019574 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f910414c600 con 0x7f9104153580 2024-04-04T06:08:52.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.746+0000 7f90fb400640 1 -- 172.21.15.159:0/475019574 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc014820 con 0x7f9104153580 2024-04-04T06:08:52.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.746+0000 7f9109c00640 1 -- 172.21.15.159:0/475019574 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9104150850 con 0x7f9104153580 2024-04-04T06:08:52.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.747+0000 7f9109c00640 1 -- 172.21.15.159:0/475019574 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f910406e370 con 0x7f9104153580 2024-04-04T06:08:52.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.748+0000 7f90fb400640 1 -- 172.21.15.159:0/475019574 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90fc0149c0 con 0x7f9104153580 2024-04-04T06:08:52.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.749+0000 7f90fb400640 1 --2- 172.21.15.159:0/475019574 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f90e006c8f0 0x7f90e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:52.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.749+0000 7f90fb400640 1 -- 172.21.15.159:0/475019574 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7f90fc08c070 con 0x7f9104153580 2024-04-04T06:08:52.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.750+0000 7f9102a00640 1 --2- 172.21.15.159:0/475019574 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f90e006c8f0 0x7f90e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:52.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.751+0000 7f9102a00640 1 --2- 172.21.15.159:0/475019574 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f90e006c8f0 0x7f90e006edb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f90f4000f60 tx=0x7f90f4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:52.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:52.754+0000 7f90fb400640 1 -- 172.21.15.159:0/475019574 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f910406e370 con 0x7f9104153580 2024-04-04T06:08:53.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.026+0000 7f9109c00640 1 -- 172.21.15.159:0/475019574 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f91040685c0 con 0x7f9104153580 2024-04-04T06:08:53.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.027+0000 7f90fb400640 1 -- 172.21.15.159:0/475019574 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7f90fc05a7e0 con 0x7f9104153580 2024-04-04T06:08:53.021 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:53.021 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:08:53.021 INFO:teuthology.orchestra.run.smithi159.stdout: "auth_dump": [ 2024-04-04T06:08:53.021 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:53.021 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.a", 2024-04-04T06:08:53.022 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5mvoZ2FxAAsVEQrkiICIH6T7ZzaqcWbA==", 2024-04-04T06:08:53.022 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:08:53.022 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:08:53.022 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:08:53.022 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:08:53.022 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:08:53.022 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:53.022 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:08:53.022 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:53.022 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.b", 2024-04-04T06:08:53.022 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5m+rccORAA6I/lQ+22QfWYL9+aoOFJ0w==", 2024-04-04T06:08:53.022 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:08:53.022 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:08:53.023 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:08:53.023 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:08:53.023 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:08:53.023 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:53.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:08:53.023 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:53.023 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.c", 2024-04-04T06:08:53.023 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5mjmQcKBAA8BjpQce6G7EycXOkyVECpQ==", 2024-04-04T06:08:53.023 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:08:53.023 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:08:53.023 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:08:53.023 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:08:53.023 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:08:53.023 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:53.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:08:53.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:53.024 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.d", 2024-04-04T06:08:53.024 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5m04hoDRAAKdhRI7nosZp7d4y7dVvzvg==", 2024-04-04T06:08:53.024 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:08:53.024 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:08:53.024 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:08:53.024 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:08:53.024 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:08:53.024 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:53.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:08:53.024 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:53.024 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.0", 2024-04-04T06:08:53.024 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQABQw5mCe7fNRAA6ZKxWvnMLf4jhAlpjudFRA==", 2024-04-04T06:08:53.025 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:08:53.025 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:08:53.025 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:08:53.025 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:08:53.025 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:53.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:08:53.025 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:53.025 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.1", 2024-04-04T06:08:53.025 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAFQw5mvIqTExAAU8Pm5alxFtHEL4qM9MavPw==", 2024-04-04T06:08:53.025 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:08:53.025 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:08:53.025 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:08:53.025 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:08:53.025 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:53.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:08:53.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:53.026 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.2", 2024-04-04T06:08:53.026 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAIQw5mAHblHRAAqEDe5dIGTDpdy2OsQyzJXA==", 2024-04-04T06:08:53.026 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:08:53.026 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:08:53.026 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:08:53.026 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:08:53.026 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:53.026 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:08:53.026 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:53.026 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.3", 2024-04-04T06:08:53.026 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQALQw5mwK7rJhAADQBAqE0Eej9rn91Wxhq/KA==", 2024-04-04T06:08:53.026 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:08:53.027 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:08:53.027 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:08:53.027 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:08:53.027 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:53.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:08:53.027 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:53.027 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.4", 2024-04-04T06:08:53.027 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAQQw5m/6I8GRAA2P7jI+AXYld2J11nDfhAUQ==", 2024-04-04T06:08:53.027 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:08:53.027 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:08:53.027 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:08:53.027 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:08:53.027 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:53.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:08:53.028 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:53.028 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.5", 2024-04-04T06:08:53.028 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQATQw5mv5HoMBAAn2mp2ys5KjS0UygOcX4vvg==", 2024-04-04T06:08:53.028 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:08:53.028 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:08:53.028 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:08:53.028 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:08:53.028 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:53.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:08:53.028 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:53.028 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.6", 2024-04-04T06:08:53.028 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAWQw5mx12gORAAT2LhrfEB5BJSl0ypgospzg==", 2024-04-04T06:08:53.028 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:08:53.028 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:08:53.029 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:08:53.029 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:08:53.029 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:53.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:08:53.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:53.029 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.7", 2024-04-04T06:08:53.029 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAaQw5m6HEOBxAAVegLH3bjVM7r1A6tK5eqHQ==", 2024-04-04T06:08:53.029 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:08:53.029 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:08:53.029 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:08:53.029 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:08:53.029 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:53.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:08:53.029 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:53.030 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.0", 2024-04-04T06:08:53.030 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mqNKJGxAA4jP9cadhRIyhQ3lssttWIw==", 2024-04-04T06:08:53.030 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:08:53.030 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:08:53.030 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow r", 2024-04-04T06:08:53.030 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-04T06:08:53.030 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:53.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:08:53.030 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:53.030 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.1", 2024-04-04T06:08:53.030 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mRkdOMBAA1ae1roIwrlB49DFQUoFSTg==", 2024-04-04T06:08:53.030 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:08:53.030 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:08:53.030 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:08:53.031 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:08:53.031 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:08:53.031 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:53.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:08:53.031 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:53.031 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.2", 2024-04-04T06:08:53.031 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mtW2/JRAAucxSSN9q0EQtI2a/P2yozQ==", 2024-04-04T06:08:53.031 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:08:53.031 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:08:53.031 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:08:53.031 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:08:53.031 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:08:53.031 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:53.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:08:53.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:53.032 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.3", 2024-04-04T06:08:53.032 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mgqD0OhAAov+3aDGyGEoqxep8+SBpSA==", 2024-04-04T06:08:53.032 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:08:53.032 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:08:53.032 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:08:53.032 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:08:53.032 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:08:53.032 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:53.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:08:53.032 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:53.032 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.admin", 2024-04-04T06:08:53.032 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD7Qg5m0O/iCBAAsWTzlEA0P60XYuxmJiAb4A==", 2024-04-04T06:08:53.033 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:08:53.033 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow *", 2024-04-04T06:08:53.033 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:08:53.033 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:08:53.033 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:08:53.033 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:53.033 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:08:53.033 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:53.033 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mgr.x", 2024-04-04T06:08:53.033 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5mhqbBCxAASBknMIj4uu7OiB3jn7WP8Q==", 2024-04-04T06:08:53.033 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:08:53.033 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow *", 2024-04-04T06:08:53.033 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile mgr", 2024-04-04T06:08:53.033 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:08:53.033 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:53.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:08:53.034 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:53.034 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mgr.y", 2024-04-04T06:08:53.034 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD7Qg5miXT0ORAAL6y306CPAfEGQKAJ0no4uw==", 2024-04-04T06:08:53.034 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:08:53.034 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow *", 2024-04-04T06:08:53.034 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile mgr", 2024-04-04T06:08:53.034 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:08:53.034 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:53.034 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:53.034 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-04-04T06:08:53.034 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:08:53.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.030+0000 7f9109c00640 1 -- 172.21.15.159:0/475019574 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f90e006c8f0 msgr2=0x7f90e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:53.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.030+0000 7f9109c00640 1 --2- 172.21.15.159:0/475019574 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f90e006c8f0 0x7f90e006edb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f90f4000f60 tx=0x7f90f4009040 comp rx=0 tx=0).stop 2024-04-04T06:08:53.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.030+0000 7f9109c00640 1 -- 172.21.15.159:0/475019574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9104153580 msgr2=0x7f9104156340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:53.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.030+0000 7f9109c00640 1 --2- 172.21.15.159:0/475019574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9104153580 0x7f9104156340 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f90fc007e60 tx=0x7f90fc00e4b0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.030+0000 7f9109c00640 1 -- 172.21.15.159:0/475019574 shutdown_connections 2024-04-04T06:08:53.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.030+0000 7f9109c00640 1 --2- 172.21.15.159:0/475019574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9104153580 0x7f9104156340 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.030+0000 7f9109c00640 1 --2- 172.21.15.159:0/475019574 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91040d0990 0x7f9104154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.030+0000 7f9109c00640 1 --2- 172.21.15.159:0/475019574 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f90e006c8f0 0x7f90e006edb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.030+0000 7f9109c00640 1 --2- 172.21.15.159:0/475019574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91040af2e0 0x7f91040cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.030+0000 7f9109c00640 1 -- 172.21.15.159:0/475019574 >> 172.21.15.159:0/475019574 conn(0x7f910406b750 msgr2=0x7f91040b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:53.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.030+0000 7f9109c00640 1 -- 172.21.15.159:0/475019574 shutdown_connections 2024-04-04T06:08:53.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.030+0000 7f9109c00640 1 -- 172.21.15.159:0/475019574 wait complete. 2024-04-04T06:08:53.044 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:08:53.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.266+0000 7f7063000640 1 Processor -- start 2024-04-04T06:08:53.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.267+0000 7f7063000640 1 -- start start 2024-04-04T06:08:53.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.268+0000 7f7063000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f705c0af830 0x7f705c073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:53.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.268+0000 7f7063000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f705c15fe80 con 0x7f705c15c390 2024-04-04T06:08:53.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.268+0000 7f7063000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f705c0ad120 con 0x7f705c0d0650 2024-04-04T06:08:53.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.268+0000 7f7063000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f705c15c770 con 0x7f705c0af830 2024-04-04T06:08:53.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.268+0000 7f705be00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f705c15c390 0x7f705c160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:60138/0 (socket says 172.21.15.159:60138) 2024-04-04T06:08:53.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.269+0000 7f705be00640 1 -- 172.21.15.159:0/1561988866 learned_addr learned my addr 172.21.15.159:0/1561988866 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:53.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.269+0000 7f7053400640 1 --2- 172.21.15.159:0/1561988866 >> v2:172.21.15.178:3300/0 conn(0x7f705c0af830 0x7f705c073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:53.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.269+0000 7f705b400640 1 -- 172.21.15.159:0/1561988866 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 747815393 0 0) 0x7f705c15fe80 con 0x7f705c15c390 2024-04-04T06:08:53.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.269+0000 7f705b400640 1 -- 172.21.15.159:0/1561988866 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7040003520 con 0x7f705c15c390 2024-04-04T06:08:53.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.269+0000 7f705b400640 1 -- 172.21.15.159:0/1561988866 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 154749312 0 0) 0x7f7040003520 con 0x7f705c15c390 2024-04-04T06:08:53.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.269+0000 7f705b400640 1 -- 172.21.15.159:0/1561988866 >> v2:172.21.15.178:3300/0 conn(0x7f705c0af830 msgr2=0x7f705c073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:53.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.269+0000 7f705b400640 1 --2- 172.21.15.159:0/1561988866 >> v2:172.21.15.178:3300/0 conn(0x7f705c0af830 0x7f705c073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.269+0000 7f705b400640 1 -- 172.21.15.159:0/1561988866 >> v1:172.21.15.178:6789/0 conn(0x7f705c0d0650 legacy=0x7f705c071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:53.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.269+0000 7f705b400640 1 -- 172.21.15.159:0/1561988866 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705c0ad400 con 0x7f705c15c390 2024-04-04T06:08:53.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.270+0000 7f705b400640 1 -- 172.21.15.159:0/1561988866 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f704c001690 con 0x7f705c15c390 2024-04-04T06:08:53.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.270+0000 7f705b400640 1 -- 172.21.15.159:0/1561988866 >> v1:172.21.15.159:6789/0 conn(0x7f705c15c390 legacy=0x7f705c160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:53.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.270+0000 7f705b400640 1 --2- 172.21.15.159:0/1561988866 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70400039b0 0x7f7040003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:53.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.270+0000 7f705b400640 1 --2- 172.21.15.159:0/1561988866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7040004800 0x7f7040004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:53.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.270+0000 7f705b400640 1 --2- 172.21.15.159:0/1561988866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70400051c0 0x7f704000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:53.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.270+0000 7f705b400640 1 -- 172.21.15.159:0/1561988866 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f704000c030 con 0x7f7040004800 2024-04-04T06:08:53.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.270+0000 7f705b400640 1 -- 172.21.15.159:0/1561988866 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f704000c1e0 con 0x7f70400051c0 2024-04-04T06:08:53.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.270+0000 7f705b400640 1 -- 172.21.15.159:0/1561988866 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f704000c3e0 con 0x7f70400039b0 2024-04-04T06:08:53.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.270+0000 7f7053400640 1 --2- 172.21.15.159:0/1561988866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7040004800 0x7f7040004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:53.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.270+0000 7f705be00640 1 --2- 172.21.15.159:0/1561988866 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70400039b0 0x7f7040003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:53.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.270+0000 7f7061000640 1 --2- 172.21.15.159:0/1561988866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70400051c0 0x7f704000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:53.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.271+0000 7f7053400640 1 -- 172.21.15.159:0/1561988866 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70400039b0 msgr2=0x7f7040003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:53.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.271+0000 7f7053400640 1 --2- 172.21.15.159:0/1561988866 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70400039b0 0x7f7040003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.271+0000 7f7053400640 1 -- 172.21.15.159:0/1561988866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70400051c0 msgr2=0x7f704000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:53.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.271+0000 7f7053400640 1 --2- 172.21.15.159:0/1561988866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70400051c0 0x7f704000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.271+0000 7f7053400640 1 -- 172.21.15.159:0/1561988866 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7040003520 con 0x7f7040004800 2024-04-04T06:08:53.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.271+0000 7f7061000640 1 -- 172.21.15.159:0/1561988866 reap_dead start 2024-04-04T06:08:53.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.272+0000 7f7053400640 1 --2- 172.21.15.159:0/1561988866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7040004800 0x7f7040004c80 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7044009ac0 tx=0x7f7040005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d7e25cf53fcb8c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:53.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.272+0000 7f705b400640 1 -- 172.21.15.159:0/1561988866 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7044039070 con 0x7f7040004800 2024-04-04T06:08:53.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.272+0000 7f705b400640 1 -- 172.21.15.159:0/1561988866 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f704402c6b0 con 0x7f7040004800 2024-04-04T06:08:53.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.272+0000 7f705b400640 1 -- 172.21.15.159:0/1561988866 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7044009690 con 0x7f7040004800 2024-04-04T06:08:53.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.273+0000 7f7063000640 1 -- 172.21.15.159:0/1561988866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7040004800 msgr2=0x7f7040004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:53.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.273+0000 7f7063000640 1 --2- 172.21.15.159:0/1561988866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7040004800 0x7f7040004c80 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7044009ac0 tx=0x7f7040005830 comp rx=0 tx=0).stop 2024-04-04T06:08:53.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.273+0000 7f7063000640 1 -- 172.21.15.159:0/1561988866 shutdown_connections 2024-04-04T06:08:53.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.273+0000 7f7063000640 1 --2- 172.21.15.159:0/1561988866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7040004800 0x7f7040004c80 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.273+0000 7f7063000640 1 -- 172.21.15.159:0/1561988866 >> 172.21.15.159:0/1561988866 conn(0x7f705c06b990 msgr2=0x7f705c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:53.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.273+0000 7f7063000640 1 -- 172.21.15.159:0/1561988866 shutdown_connections 2024-04-04T06:08:53.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.273+0000 7f7063000640 1 -- 172.21.15.159:0/1561988866 wait complete. 2024-04-04T06:08:53.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.275+0000 7f7063000640 1 Processor -- start 2024-04-04T06:08:53.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.275+0000 7f7063000640 1 -- start start 2024-04-04T06:08:53.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.277+0000 7f7063000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7040004800 0x7f705c1f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:53.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.277+0000 7f7063000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f705c0afce0 0x7f705c0b1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:53.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.277+0000 7f7063000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f705c15ef60 0x7f705c0b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:53.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.277+0000 7f7063000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f705c0cf980 con 0x7f705c15ef60 2024-04-04T06:08:53.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.277+0000 7f7063000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f705c160120 con 0x7f705c0afce0 2024-04-04T06:08:53.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.277+0000 7f7063000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f705c073790 con 0x7f7040004800 2024-04-04T06:08:53.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.277+0000 7f7061000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f705c15ef60 0x7f705c0b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:53.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.277+0000 7f7061000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f705c15ef60 0x7f705c0b1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58386/0 (socket says 172.21.15.159:58386) 2024-04-04T06:08:53.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.277+0000 7f7061000640 1 -- 172.21.15.159:0/34098637 learned_addr learned my addr 172.21.15.159:0/34098637 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:53.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.277+0000 7f705be00640 1 --2- 172.21.15.159:0/34098637 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7040004800 0x7f705c1f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.277+0000 7f7061000640 1 -- 172.21.15.159:0/34098637 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7040004800 msgr2=0x7f705c1f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.277+0000 7f7053400640 1 --2- 172.21.15.159:0/34098637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f705c0afce0 0x7f705c0b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.277+0000 7f7061000640 1 --2- 172.21.15.159:0/34098637 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7040004800 0x7f705c1f4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.277+0000 7f7061000640 1 -- 172.21.15.159:0/34098637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f705c0afce0 msgr2=0x7f705c0b1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.277+0000 7f7061000640 1 --2- 172.21.15.159:0/34098637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f705c0afce0 0x7f705c0b1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.277+0000 7f7061000640 1 -- 172.21.15.159:0/34098637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705c0b0170 con 0x7f705c15ef60 2024-04-04T06:08:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.278+0000 7f7061000640 1 --2- 172.21.15.159:0/34098637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f705c15ef60 0x7f705c0b1b70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7054007fd0 tx=0x7f705400e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.278+0000 7f7058c00640 1 -- 172.21.15.159:0/34098637 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054019070 con 0x7f705c15ef60 2024-04-04T06:08:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.278+0000 7f7058c00640 1 -- 172.21.15.159:0/34098637 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7054006d60 con 0x7f705c15ef60 2024-04-04T06:08:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.278+0000 7f7063000640 1 -- 172.21.15.159:0/34098637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705c0b0450 con 0x7f705c15ef60 2024-04-04T06:08:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.278+0000 7f7058c00640 1 -- 172.21.15.159:0/34098637 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054003b90 con 0x7f705c15ef60 2024-04-04T06:08:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.278+0000 7f7063000640 1 -- 172.21.15.159:0/34098637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f705c0b0a10 con 0x7f705c15ef60 2024-04-04T06:08:53.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.279+0000 7f7063000640 1 -- 172.21.15.159:0/34098637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f705c0685c0 con 0x7f705c15ef60 2024-04-04T06:08:53.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.281+0000 7f7058c00640 1 -- 172.21.15.159:0/34098637 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f705401c070 con 0x7f705c15ef60 2024-04-04T06:08:53.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.282+0000 7f7058c00640 1 --2- 172.21.15.159:0/34098637 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f703806c8f0 0x7f703806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:53.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.282+0000 7f7058c00640 1 -- 172.21.15.159:0/34098637 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7f705408afa0 con 0x7f705c15ef60 2024-04-04T06:08:53.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.282+0000 7f705be00640 1 --2- 172.21.15.159:0/34098637 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f703806c8f0 0x7f703806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:53.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.283+0000 7f705be00640 1 --2- 172.21.15.159:0/34098637 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f703806c8f0 0x7f703806edb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f704c001eb0 tx=0x7f704c000ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:53.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.286+0000 7f7058c00640 1 -- 172.21.15.159:0/34098637 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7054059680 con 0x7f705c15ef60 2024-04-04T06:08:53.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.487+0000 7f7063000640 1 -- 172.21.15.159:0/34098637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f705c06eb20 con 0x7f705c15ef60 2024-04-04T06:08:53.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.489+0000 7f7058c00640 1 -- 172.21.15.159:0/34098637 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+12903 (secure 0 0 0) 0x7f7054058f80 con 0x7f705c15ef60 2024-04-04T06:08:53.482 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:53.483 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":55,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:08:52.529186+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:08:53.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.491+0000 7f7063000640 1 -- 172.21.15.159:0/34098637 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f703806c8f0 msgr2=0x7f703806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:53.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.491+0000 7f7063000640 1 --2- 172.21.15.159:0/34098637 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f703806c8f0 0x7f703806edb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f704c001eb0 tx=0x7f704c000ed0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.491+0000 7f7063000640 1 -- 172.21.15.159:0/34098637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f705c15ef60 msgr2=0x7f705c0b1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:53.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.491+0000 7f7063000640 1 --2- 172.21.15.159:0/34098637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f705c15ef60 0x7f705c0b1b70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7054007fd0 tx=0x7f705400e9e0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.492+0000 7f7063000640 1 -- 172.21.15.159:0/34098637 shutdown_connections 2024-04-04T06:08:53.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.492+0000 7f7063000640 1 --2- 172.21.15.159:0/34098637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f705c15ef60 0x7f705c0b1b70 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.492+0000 7f7063000640 1 --2- 172.21.15.159:0/34098637 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f703806c8f0 0x7f703806edb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.492+0000 7f7063000640 1 --2- 172.21.15.159:0/34098637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f705c0afce0 0x7f705c0b1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.492+0000 7f7063000640 1 --2- 172.21.15.159:0/34098637 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7040004800 0x7f705c1f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.492+0000 7f7063000640 1 -- 172.21.15.159:0/34098637 >> 172.21.15.159:0/34098637 conn(0x7f705c06b990 msgr2=0x7f705c1575a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:53.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.492+0000 7f7063000640 1 -- 172.21.15.159:0/34098637 shutdown_connections 2024-04-04T06:08:53.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.492+0000 7f7063000640 1 -- 172.21.15.159:0/34098637 wait complete. 2024-04-04T06:08:53.505 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:08:53.506 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:08:53.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.719+0000 7f56df800640 1 Processor -- start 2024-04-04T06:08:53.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.719+0000 7f56df800640 1 -- start start 2024-04-04T06:08:53.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.720+0000 7f56df800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56d8153580 0x7f56d80d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:53.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.720+0000 7f56df800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56d80afab0 con 0x7f56d80d07c0 2024-04-04T06:08:53.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.720+0000 7f56df800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56d815cc80 con 0x7f56d80af4b0 2024-04-04T06:08:53.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.720+0000 7f56df800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f56d80d0c00 con 0x7f56d8153580 2024-04-04T06:08:53.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.721+0000 7f56d7e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f56d80d07c0 0x7f56d80aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:60148/0 (socket says 172.21.15.159:60148) 2024-04-04T06:08:53.715 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.721+0000 7f56d7e00640 1 -- 172.21.15.159:0/4149051010 learned_addr learned my addr 172.21.15.159:0/4149051010 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:53.716 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.721+0000 7f56dce00640 1 --2- 172.21.15.159:0/4149051010 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56d8153580 0x7f56d80d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:53.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.721+0000 7f56d7400640 1 -- 172.21.15.159:0/4149051010 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2414279484 0 0) 0x7f56d80afab0 con 0x7f56d80d07c0 2024-04-04T06:08:53.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.721+0000 7f56d7400640 1 -- 172.21.15.159:0/4149051010 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56c0003520 con 0x7f56d80d07c0 2024-04-04T06:08:53.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.722+0000 7f56d7400640 1 -- 172.21.15.159:0/4149051010 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3970404193 0 0) 0x7f56c0003520 con 0x7f56d80d07c0 2024-04-04T06:08:53.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.722+0000 7f56d7400640 1 -- 172.21.15.159:0/4149051010 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56d8153580 msgr2=0x7f56d80d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:53.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.722+0000 7f56d7400640 1 --2- 172.21.15.159:0/4149051010 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56d8153580 0x7f56d80d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.722+0000 7f56d7400640 1 -- 172.21.15.159:0/4149051010 >> v1:172.21.15.178:6789/0 conn(0x7f56d80af4b0 legacy=0x7f56d81600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:53.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.722+0000 7f56d7400640 1 -- 172.21.15.159:0/4149051010 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d815da00 con 0x7f56d80d07c0 2024-04-04T06:08:53.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.722+0000 7f56d7400640 1 -- 172.21.15.159:0/4149051010 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f56c4001670 con 0x7f56d80d07c0 2024-04-04T06:08:53.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.722+0000 7f56d7400640 1 -- 172.21.15.159:0/4149051010 >> v1:172.21.15.159:6789/0 conn(0x7f56d80d07c0 legacy=0x7f56d80aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:53.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.722+0000 7f56d7400640 1 --2- 172.21.15.159:0/4149051010 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56d8153580 0x7f56c00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:53.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.722+0000 7f56d7400640 1 --2- 172.21.15.159:0/4149051010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56c0003fb0 0x7f56c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:53.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.722+0000 7f56d7400640 1 --2- 172.21.15.159:0/4149051010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56c0004d40 0x7f56c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:53.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.722+0000 7f56d7400640 1 -- 172.21.15.159:0/4149051010 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56d80d0c00 con 0x7f56c0004d40 2024-04-04T06:08:53.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.722+0000 7f56d7400640 1 -- 172.21.15.159:0/4149051010 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56c000bbb0 con 0x7f56c0003fb0 2024-04-04T06:08:53.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.722+0000 7f56d7400640 1 -- 172.21.15.159:0/4149051010 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f56c000bdb0 con 0x7f56d8153580 2024-04-04T06:08:53.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.722+0000 7f56dd800640 1 --2- 172.21.15.159:0/4149051010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56c0004d40 0x7f56c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:53.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.723+0000 7f56dce00640 1 --2- 172.21.15.159:0/4149051010 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56d8153580 0x7f56c00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:53.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.723+0000 7f56d7e00640 1 --2- 172.21.15.159:0/4149051010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56c0003fb0 0x7f56c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:53.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.723+0000 7f56dd800640 1 -- 172.21.15.159:0/4149051010 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56d8153580 msgr2=0x7f56c00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:53.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.723+0000 7f56dd800640 1 --2- 172.21.15.159:0/4149051010 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56d8153580 0x7f56c00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.723+0000 7f56dd800640 1 -- 172.21.15.159:0/4149051010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56c0003fb0 msgr2=0x7f56c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:53.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.723+0000 7f56dd800640 1 --2- 172.21.15.159:0/4149051010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56c0003fb0 0x7f56c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.723+0000 7f56dd800640 1 -- 172.21.15.159:0/4149051010 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c0003520 con 0x7f56c0004d40 2024-04-04T06:08:53.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.724+0000 7f56dd800640 1 --2- 172.21.15.159:0/4149051010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56c0004d40 0x7f56c00051c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f56d0001e50 tx=0x7f56c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba44beb78579d3cc server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:53.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.724+0000 7f56d7400640 1 -- 172.21.15.159:0/4149051010 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d0001a70 con 0x7f56c0004d40 2024-04-04T06:08:53.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.724+0000 7f56d7400640 1 -- 172.21.15.159:0/4149051010 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56d0003b70 con 0x7f56c0004d40 2024-04-04T06:08:53.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.724+0000 7f56d7400640 1 -- 172.21.15.159:0/4149051010 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d00050d0 con 0x7f56c0004d40 2024-04-04T06:08:53.720 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.725+0000 7f56df800640 1 -- 172.21.15.159:0/4149051010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56c0004d40 msgr2=0x7f56c00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:53.720 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.725+0000 7f56df800640 1 --2- 172.21.15.159:0/4149051010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56c0004d40 0x7f56c00051c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f56d0001e50 tx=0x7f56c0005830 comp rx=0 tx=0).stop 2024-04-04T06:08:53.720 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.725+0000 7f56dd800640 1 -- 172.21.15.159:0/4149051010 reap_dead start 2024-04-04T06:08:53.720 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.725+0000 7f56df800640 1 -- 172.21.15.159:0/4149051010 shutdown_connections 2024-04-04T06:08:53.720 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.725+0000 7f56df800640 1 -- 172.21.15.159:0/4149051010 >> 172.21.15.159:0/4149051010 conn(0x7f56d806b750 msgr2=0x7f56d806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:53.720 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.725+0000 7f56df800640 1 -- 172.21.15.159:0/4149051010 shutdown_connections 2024-04-04T06:08:53.720 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.725+0000 7f56df800640 1 -- 172.21.15.159:0/4149051010 wait complete. 2024-04-04T06:08:53.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.726+0000 7f56df800640 1 Processor -- start 2024-04-04T06:08:53.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.727+0000 7f56df800640 1 -- start start 2024-04-04T06:08:53.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.728+0000 7f56df800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56c0004d40 0x7f56d80cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:53.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.728+0000 7f56df800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56d814e760 0x7f56d81503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:53.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.728+0000 7f56df800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56d81f6c00 0x7f56d81f7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:53.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.728+0000 7f56df800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56d815d4e0 con 0x7f56d81f6c00 2024-04-04T06:08:53.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.728+0000 7f56df800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56d815d200 con 0x7f56c0004d40 2024-04-04T06:08:53.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.728+0000 7f56df800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f56d8153960 con 0x7f56d814e760 2024-04-04T06:08:53.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.728+0000 7f56dd800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56d81f6c00 0x7f56d81f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:53.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.728+0000 7f56dd800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56d81f6c00 0x7f56d81f7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58402/0 (socket says 172.21.15.159:58402) 2024-04-04T06:08:53.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.728+0000 7f56dd800640 1 -- 172.21.15.159:0/4038553883 learned_addr learned my addr 172.21.15.159:0/4038553883 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:53.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.728+0000 7f56d7e00640 1 --2- 172.21.15.159:0/4038553883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56d814e760 0x7f56d81503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:53.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.728+0000 7f56dce00640 1 --2- 172.21.15.159:0/4038553883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56c0004d40 0x7f56d80cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:53.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.728+0000 7f56dd800640 1 -- 172.21.15.159:0/4038553883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56d814e760 msgr2=0x7f56d81503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:53.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.728+0000 7f56dd800640 1 --2- 172.21.15.159:0/4038553883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56d814e760 0x7f56d81503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.728+0000 7f56dd800640 1 -- 172.21.15.159:0/4038553883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56c0004d40 msgr2=0x7f56d80cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:53.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.728+0000 7f56dd800640 1 --2- 172.21.15.159:0/4038553883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56c0004d40 0x7f56d80cd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.728+0000 7f56dd800640 1 -- 172.21.15.159:0/4038553883 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d80cda60 con 0x7f56d81f6c00 2024-04-04T06:08:53.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.728+0000 7f56dce00640 1 --2- 172.21.15.159:0/4038553883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56c0004d40 0x7f56d80cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:53.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.729+0000 7f56dd800640 1 --2- 172.21.15.159:0/4038553883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56d81f6c00 0x7f56d81f7080 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f56d002c060 tx=0x7f56d00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:53.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.729+0000 7f56d4c00640 1 -- 172.21.15.159:0/4038553883 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d0007db0 con 0x7f56d81f6c00 2024-04-04T06:08:53.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.729+0000 7f56d4c00640 1 -- 172.21.15.159:0/4038553883 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56d0003970 con 0x7f56d81f6c00 2024-04-04T06:08:53.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.729+0000 7f56d4c00640 1 -- 172.21.15.159:0/4038553883 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d0006d70 con 0x7f56d81f6c00 2024-04-04T06:08:53.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.729+0000 7f56df800640 1 -- 172.21.15.159:0/4038553883 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d80ca6a0 con 0x7f56d81f6c00 2024-04-04T06:08:53.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.729+0000 7f56df800640 1 -- 172.21.15.159:0/4038553883 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d8144c40 con 0x7f56d81f6c00 2024-04-04T06:08:53.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.730+0000 7f56df800640 1 -- 172.21.15.159:0/4038553883 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d80685c0 con 0x7f56d81f6c00 2024-04-04T06:08:53.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.731+0000 7f56d4c00640 1 -- 172.21.15.159:0/4038553883 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f56d0007030 con 0x7f56d81f6c00 2024-04-04T06:08:53.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.732+0000 7f56d4c00640 1 --2- 172.21.15.159:0/4038553883 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f56b806c8f0 0x7f56b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:53.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.732+0000 7f56d4c00640 1 -- 172.21.15.159:0/4038553883 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7f56d00acbb0 con 0x7f56d81f6c00 2024-04-04T06:08:53.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.733+0000 7f56dce00640 1 --2- 172.21.15.159:0/4038553883 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f56b806c8f0 0x7f56b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:53.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.734+0000 7f56dce00640 1 --2- 172.21.15.159:0/4038553883 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f56b806c8f0 0x7f56b806edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009a00 tx=0x7f56cc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:53.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.737+0000 7f56d4c00640 1 -- 172.21.15.159:0/4038553883 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f56d0033020 con 0x7f56d81f6c00 2024-04-04T06:08:53.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.936+0000 7f56df800640 1 -- 172.21.15.159:0/4038553883 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f56d806eca0 con 0x7f56d81f6c00 2024-04-04T06:08:53.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.936+0000 7f56d4c00640 1 -- 172.21.15.159:0/4038553883 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) ==== 93+0+10 (secure 0 0 0) 0x7f56d007b310 con 0x7f56d81f6c00 2024-04-04T06:08:53.930 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:08:53.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.939+0000 7f56df800640 1 -- 172.21.15.159:0/4038553883 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f56b806c8f0 msgr2=0x7f56b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:53.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.939+0000 7f56df800640 1 --2- 172.21.15.159:0/4038553883 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f56b806c8f0 0x7f56b806edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009a00 tx=0x7f56cc008040 comp rx=0 tx=0).stop 2024-04-04T06:08:53.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.939+0000 7f56df800640 1 -- 172.21.15.159:0/4038553883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56d81f6c00 msgr2=0x7f56d81f7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:53.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.939+0000 7f56df800640 1 --2- 172.21.15.159:0/4038553883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56d81f6c00 0x7f56d81f7080 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f56d002c060 tx=0x7f56d00014a0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.939+0000 7f56df800640 1 -- 172.21.15.159:0/4038553883 shutdown_connections 2024-04-04T06:08:53.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.939+0000 7f56df800640 1 --2- 172.21.15.159:0/4038553883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56d81f6c00 0x7f56d81f7080 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.939+0000 7f56df800640 1 --2- 172.21.15.159:0/4038553883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56d814e760 0x7f56d81503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.939+0000 7f56df800640 1 --2- 172.21.15.159:0/4038553883 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f56b806c8f0 0x7f56b806edb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.939+0000 7f56df800640 1 --2- 172.21.15.159:0/4038553883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56c0004d40 0x7f56d80cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:53.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.939+0000 7f56df800640 1 -- 172.21.15.159:0/4038553883 >> 172.21.15.159:0/4038553883 conn(0x7f56d806b750 msgr2=0x7f56d80aeda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:53.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.939+0000 7f56df800640 1 -- 172.21.15.159:0/4038553883 shutdown_connections 2024-04-04T06:08:53.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:53.939+0000 7f56df800640 1 -- 172.21.15.159:0/4038553883 wait complete. 2024-04-04T06:08:53.951 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-04T06:08:54.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.166+0000 7fa5cb400640 1 Processor -- start 2024-04-04T06:08:54.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.166+0000 7fa5cb400640 1 -- start start 2024-04-04T06:08:54.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.167+0000 7fa5cb400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5c40b4650 0x7fa5c406c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:54.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.167+0000 7fa5cb400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa5c4158580 0x7fa5c41589d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:54.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.167+0000 7fa5cb400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5c415ede0 con 0x7fa5c40b4a30 2024-04-04T06:08:54.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.167+0000 7fa5cb400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa5c40b6320 con 0x7fa5c4158580 2024-04-04T06:08:54.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.167+0000 7fa5cb400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c4141b30 con 0x7fa5c40b4650 2024-04-04T06:08:54.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.168+0000 7fa5c3e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fa5c40b4a30 0x7fa5c40b5c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40260/0 (socket says 172.21.15.159:40260) 2024-04-04T06:08:54.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.168+0000 7fa5c3e00640 1 -- 172.21.15.159:0/2801363711 learned_addr learned my addr 172.21.15.159:0/2801363711 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:54.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.168+0000 7fa5c8a00640 1 --2- 172.21.15.159:0/2801363711 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5c40b4650 0x7fa5c406c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:54.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.168+0000 7fa5c9400640 1 --2- 172.21.15.159:0/2801363711 >> v2:172.21.15.178:3300/0 conn(0x7fa5c4158580 0x7fa5c41589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:54.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.168+0000 7fa5c3400640 1 -- 172.21.15.159:0/2801363711 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3519350016 0 0) 0x7fa5c415ede0 con 0x7fa5c40b4a30 2024-04-04T06:08:54.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.168+0000 7fa5c3400640 1 -- 172.21.15.159:0/2801363711 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5a8003520 con 0x7fa5c40b4a30 2024-04-04T06:08:54.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.168+0000 7fa5c3400640 1 -- 172.21.15.159:0/2801363711 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3377348255 0 0) 0x7fa5a8003520 con 0x7fa5c40b4a30 2024-04-04T06:08:54.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.168+0000 7fa5c3400640 1 -- 172.21.15.159:0/2801363711 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5c40b4650 msgr2=0x7fa5c406c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:54.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.168+0000 7fa5c3400640 1 --2- 172.21.15.159:0/2801363711 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5c40b4650 0x7fa5c406c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:54.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.168+0000 7fa5c3400640 1 -- 172.21.15.159:0/2801363711 >> v2:172.21.15.178:3300/0 conn(0x7fa5c4158580 msgr2=0x7fa5c41589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:54.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.168+0000 7fa5c3400640 1 --2- 172.21.15.159:0/2801363711 >> v2:172.21.15.178:3300/0 conn(0x7fa5c4158580 0x7fa5c41589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:54.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.169+0000 7fa5c3400640 1 -- 172.21.15.159:0/2801363711 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c4141d80 con 0x7fa5c40b4a30 2024-04-04T06:08:54.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.169+0000 7fa5c3400640 1 -- 172.21.15.159:0/2801363711 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fa5ac001690 con 0x7fa5c40b4a30 2024-04-04T06:08:54.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.169+0000 7fa5c3400640 1 -- 172.21.15.159:0/2801363711 >> v1:172.21.15.159:6789/0 conn(0x7fa5c40b4a30 legacy=0x7fa5c40b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:54.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.169+0000 7fa5c3400640 1 --2- 172.21.15.159:0/2801363711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5a80039b0 0x7fa5a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:54.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.169+0000 7fa5c3400640 1 --2- 172.21.15.159:0/2801363711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5a8004800 0x7fa5a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:54.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.169+0000 7fa5c3400640 1 --2- 172.21.15.159:0/2801363711 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5c40b4650 0x7fa5a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:54.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.169+0000 7fa5c3400640 1 -- 172.21.15.159:0/2801363711 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c40b6320 con 0x7fa5a8004800 2024-04-04T06:08:54.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.169+0000 7fa5c3400640 1 -- 172.21.15.159:0/2801363711 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c4141b30 con 0x7fa5a80039b0 2024-04-04T06:08:54.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.169+0000 7fa5c3400640 1 -- 172.21.15.159:0/2801363711 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a800b9c0 con 0x7fa5c40b4650 2024-04-04T06:08:54.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.169+0000 7fa5c3e00640 1 --2- 172.21.15.159:0/2801363711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5a8004800 0x7fa5a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:54.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.169+0000 7fa5c3e00640 1 -- 172.21.15.159:0/2801363711 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5c40b4650 msgr2=0x7fa5a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:54.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.169+0000 7fa5c3e00640 1 --2- 172.21.15.159:0/2801363711 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5c40b4650 0x7fa5a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:54.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.169+0000 7fa5c3e00640 1 -- 172.21.15.159:0/2801363711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5a80039b0 msgr2=0x7fa5a8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:54.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.169+0000 7fa5c3e00640 1 --2- 172.21.15.159:0/2801363711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5a80039b0 0x7fa5a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:54.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.169+0000 7fa5c3e00640 1 -- 172.21.15.159:0/2801363711 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a8003520 con 0x7fa5a8004800 2024-04-04T06:08:54.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.170+0000 7fa5c3e00640 1 --2- 172.21.15.159:0/2801363711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5a8004800 0x7fa5a8004c80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa5c415efb0 tx=0x7fa5a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae511763e2a833f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:54.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.171+0000 7fa5c3400640 1 -- 172.21.15.159:0/2801363711 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac007ca0 con 0x7fa5a8004800 2024-04-04T06:08:54.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.171+0000 7fa5c3400640 1 -- 172.21.15.159:0/2801363711 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5ac030070 con 0x7fa5a8004800 2024-04-04T06:08:54.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.171+0000 7fa5c3400640 1 -- 172.21.15.159:0/2801363711 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac005d10 con 0x7fa5a8004800 2024-04-04T06:08:54.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.171+0000 7fa5cb400640 1 -- 172.21.15.159:0/2801363711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5a8004800 msgr2=0x7fa5a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:54.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.171+0000 7fa5cb400640 1 --2- 172.21.15.159:0/2801363711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5a8004800 0x7fa5a8004c80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa5c415efb0 tx=0x7fa5a8005830 comp rx=0 tx=0).stop 2024-04-04T06:08:54.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.171+0000 7fa5c9400640 1 -- 172.21.15.159:0/2801363711 reap_dead start 2024-04-04T06:08:54.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.171+0000 7fa5cb400640 1 -- 172.21.15.159:0/2801363711 shutdown_connections 2024-04-04T06:08:54.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.171+0000 7fa5cb400640 1 -- 172.21.15.159:0/2801363711 >> 172.21.15.159:0/2801363711 conn(0x7fa5c4067670 msgr2=0x7fa5c4067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:54.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.171+0000 7fa5cb400640 1 -- 172.21.15.159:0/2801363711 shutdown_connections 2024-04-04T06:08:54.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.171+0000 7fa5cb400640 1 -- 172.21.15.159:0/2801363711 wait complete. 2024-04-04T06:08:54.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.173+0000 7fa5cb400640 1 Processor -- start 2024-04-04T06:08:54.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.173+0000 7fa5cb400640 1 -- start start 2024-04-04T06:08:54.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.174+0000 7fa5cb400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5a8004800 0x7fa5c415a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:54.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.174+0000 7fa5cb400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5c4140bf0 0x7fa5c41e69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:54.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.174+0000 7fa5cb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5c41e6f10 0x7fa5c41e81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:54.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.174+0000 7fa5cb400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c4142620 con 0x7fa5c41e6f10 2024-04-04T06:08:54.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.174+0000 7fa5cb400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c4142340 con 0x7fa5a8004800 2024-04-04T06:08:54.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.174+0000 7fa5cb400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c4142900 con 0x7fa5c4140bf0 2024-04-04T06:08:54.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.174+0000 7fa5c9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5c41e6f10 0x7fa5c41e81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:54.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.174+0000 7fa5c9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5c41e6f10 0x7fa5c41e81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37186/0 (socket says 172.21.15.159:37186) 2024-04-04T06:08:54.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.174+0000 7fa5c9400640 1 -- 172.21.15.159:0/4084507495 learned_addr learned my addr 172.21.15.159:0/4084507495 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:54.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.175+0000 7fa5c9400640 1 -- 172.21.15.159:0/4084507495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5c4140bf0 msgr2=0x7fa5c41e69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:54.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.175+0000 7fa5c9400640 1 --2- 172.21.15.159:0/4084507495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5c4140bf0 0x7fa5c41e69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:54.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.175+0000 7fa5c9400640 1 -- 172.21.15.159:0/4084507495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5a8004800 msgr2=0x7fa5c415a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:54.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.175+0000 7fa5c9400640 1 --2- 172.21.15.159:0/4084507495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5a8004800 0x7fa5c415a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:54.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.175+0000 7fa5c9400640 1 -- 172.21.15.159:0/4084507495 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c41e7870 con 0x7fa5c41e6f10 2024-04-04T06:08:54.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.175+0000 7fa5c9400640 1 --2- 172.21.15.159:0/4084507495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5c41e6f10 0x7fa5c41e81a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40711b0 tx=0x7fa5b400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:54.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.175+0000 7fa5c0c00640 1 -- 172.21.15.159:0/4084507495 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b400c630 con 0x7fa5c41e6f10 2024-04-04T06:08:54.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.175+0000 7fa5c0c00640 1 -- 172.21.15.159:0/4084507495 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5b401a070 con 0x7fa5c41e6f10 2024-04-04T06:08:54.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.175+0000 7fa5c0c00640 1 -- 172.21.15.159:0/4084507495 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b400ca10 con 0x7fa5c41e6f10 2024-04-04T06:08:54.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.175+0000 7fa5cb400640 1 -- 172.21.15.159:0/4084507495 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c41e7590 con 0x7fa5c41e6f10 2024-04-04T06:08:54.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.176+0000 7fa5cb400640 1 -- 172.21.15.159:0/4084507495 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c40cc9c0 con 0x7fa5c41e6f10 2024-04-04T06:08:54.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.177+0000 7fa5cb400640 1 -- 172.21.15.159:0/4084507495 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c4074f30 con 0x7fa5c41e6f10 2024-04-04T06:08:54.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.178+0000 7fa5c0c00640 1 -- 172.21.15.159:0/4084507495 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa5b400cbb0 con 0x7fa5c41e6f10 2024-04-04T06:08:54.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.179+0000 7fa5c0c00640 1 --2- 172.21.15.159:0/4084507495 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa5a006c8f0 0x7fa5a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:54.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.179+0000 7fa5c0c00640 1 -- 172.21.15.159:0/4084507495 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7fa5b408adf0 con 0x7fa5c41e6f10 2024-04-04T06:08:54.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.179+0000 7fa5c8a00640 1 --2- 172.21.15.159:0/4084507495 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa5a006c8f0 0x7fa5a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:54.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.180+0000 7fa5c8a00640 1 --2- 172.21.15.159:0/4084507495 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa5a006c8f0 0x7fa5a006edb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc0070b0 tx=0x7fa5bc00d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:54.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.183+0000 7fa5c0c00640 1 -- 172.21.15.159:0/4084507495 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa5b40102b0 con 0x7fa5c41e6f10 2024-04-04T06:08:54.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.386+0000 7fa5cb400640 1 -- 172.21.15.159:0/4084507495 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa5c406c910 con 0x7fa5c41e6f10 2024-04-04T06:08:54.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.387+0000 7fa5c0c00640 1 -- 172.21.15.159:0/4084507495 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v55) ==== 84+0+65 (secure 0 0 0) 0x7fa5b4059570 con 0x7fa5c41e6f10 2024-04-04T06:08:54.381 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:54.381 INFO:teuthology.orchestra.run.smithi159.stdout:[ 2024-04-04T06:08:54.381 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:08:54.381 INFO:teuthology.orchestra.run.smithi159.stdout: "poolnum": 1, 2024-04-04T06:08:54.381 INFO:teuthology.orchestra.run.smithi159.stdout: "poolname": ".mgr" 2024-04-04T06:08:54.381 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:08:54.381 INFO:teuthology.orchestra.run.smithi159.stdout:] 2024-04-04T06:08:54.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.389+0000 7fa5cb400640 1 -- 172.21.15.159:0/4084507495 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa5a006c8f0 msgr2=0x7fa5a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:54.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.389+0000 7fa5cb400640 1 --2- 172.21.15.159:0/4084507495 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa5a006c8f0 0x7fa5a006edb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc0070b0 tx=0x7fa5bc00d040 comp rx=0 tx=0).stop 2024-04-04T06:08:54.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.389+0000 7fa5cb400640 1 -- 172.21.15.159:0/4084507495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5c41e6f10 msgr2=0x7fa5c41e81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:54.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.389+0000 7fa5cb400640 1 --2- 172.21.15.159:0/4084507495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5c41e6f10 0x7fa5c41e81a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40711b0 tx=0x7fa5b400ec00 comp rx=0 tx=0).stop 2024-04-04T06:08:54.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.390+0000 7fa5cb400640 1 -- 172.21.15.159:0/4084507495 shutdown_connections 2024-04-04T06:08:54.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.390+0000 7fa5cb400640 1 --2- 172.21.15.159:0/4084507495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5c41e6f10 0x7fa5c41e81a0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:54.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.390+0000 7fa5cb400640 1 --2- 172.21.15.159:0/4084507495 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa5c4140bf0 0x7fa5c41e69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:54.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.390+0000 7fa5cb400640 1 --2- 172.21.15.159:0/4084507495 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa5a006c8f0 0x7fa5a006edb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:54.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.390+0000 7fa5cb400640 1 --2- 172.21.15.159:0/4084507495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5a8004800 0x7fa5c415a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:54.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.390+0000 7fa5cb400640 1 -- 172.21.15.159:0/4084507495 >> 172.21.15.159:0/4084507495 conn(0x7fa5c4067670 msgr2=0x7fa5c415dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:54.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.390+0000 7fa5cb400640 1 -- 172.21.15.159:0/4084507495 shutdown_connections 2024-04-04T06:08:54.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.390+0000 7fa5cb400640 1 -- 172.21.15.159:0/4084507495 wait complete. 2024-04-04T06:08:54.400 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-04T06:08:54.401 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-04T06:08:54.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.615+0000 7f4e59200640 1 Processor -- start 2024-04-04T06:08:54.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.615+0000 7f4e59200640 1 -- start start 2024-04-04T06:08:54.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.616+0000 7f4e59200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4e540b3940 0x7f4e540adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:54.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.617+0000 7f4e59200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e540ae4f0 0x7f4e540ae960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:54.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.617+0000 7f4e59200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e5406ea10 con 0x7f4e54072720 2024-04-04T06:08:54.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.617+0000 7f4e59200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f4e5406ecf0 con 0x7f4e540b3940 2024-04-04T06:08:54.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.617+0000 7f4e59200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4e5406efd0 con 0x7f4e540ae4f0 2024-04-04T06:08:54.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.617+0000 7f4e51400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4e540b3940 0x7f4e540adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:54.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.617+0000 7f4e51400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4e540b3940 0x7f4e540adfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37192/0 (socket says 172.21.15.159:37192) 2024-04-04T06:08:54.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.617+0000 7f4e51400640 1 -- 172.21.15.159:0/657987950 learned_addr learned my addr 172.21.15.159:0/657987950 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:54.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.617+0000 7f4e52800640 1 --2- 172.21.15.159:0/657987950 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e540ae4f0 0x7f4e540ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:54.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.617+0000 7f4e51400640 1 -- 172.21.15.159:0/657987950 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e540ae4f0 msgr2=0x7f4e540ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:54.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.617+0000 7f4e51400640 1 --2- 172.21.15.159:0/657987950 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e540ae4f0 0x7f4e540ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:54.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.617+0000 7f4e51400640 1 -- 172.21.15.159:0/657987950 >> v1:172.21.15.178:6789/0 conn(0x7f4e54072720 legacy=0x7f4e54072b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:54.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.617+0000 7f4e51400640 1 -- 172.21.15.159:0/657987950 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5406f2b0 con 0x7f4e540b3940 2024-04-04T06:08:54.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.617+0000 7f4e52800640 1 --2- 172.21.15.159:0/657987950 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e540ae4f0 0x7f4e540ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:54.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.619+0000 7f4e51400640 1 --2- 172.21.15.159:0/657987950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e540b3940 0x7f4e540adfb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009870 tx=0x7f4e4002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae42f71f23d30be6 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:54.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.619+0000 7f4e50a00640 1 -- 172.21.15.159:0/657987950 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e40035020 con 0x7f4e540b3940 2024-04-04T06:08:54.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.619+0000 7f4e50a00640 1 -- 172.21.15.159:0/657987950 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e40031640 con 0x7f4e540b3940 2024-04-04T06:08:54.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.619+0000 7f4e50a00640 1 -- 172.21.15.159:0/657987950 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4002d040 con 0x7f4e540b3940 2024-04-04T06:08:54.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.619+0000 7f4e59200640 1 -- 172.21.15.159:0/657987950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e540b3940 msgr2=0x7f4e540adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:54.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.619+0000 7f4e59200640 1 --2- 172.21.15.159:0/657987950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e540b3940 0x7f4e540adfb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009870 tx=0x7f4e4002b8f0 comp rx=0 tx=0).stop 2024-04-04T06:08:54.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.620+0000 7f4e59200640 1 -- 172.21.15.159:0/657987950 shutdown_connections 2024-04-04T06:08:54.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.620+0000 7f4e59200640 1 --2- 172.21.15.159:0/657987950 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e540ae4f0 0x7f4e540ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:54.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.620+0000 7f4e59200640 1 --2- 172.21.15.159:0/657987950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e540b3940 0x7f4e540adfb0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:54.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.620+0000 7f4e59200640 1 -- 172.21.15.159:0/657987950 >> 172.21.15.159:0/657987950 conn(0x7f4e5406bba0 msgr2=0x7f4e540b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:54.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.620+0000 7f4e59200640 1 -- 172.21.15.159:0/657987950 shutdown_connections 2024-04-04T06:08:54.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.620+0000 7f4e59200640 1 -- 172.21.15.159:0/657987950 wait complete. 2024-04-04T06:08:54.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.621+0000 7f4e59200640 1 Processor -- start 2024-04-04T06:08:54.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.622+0000 7f4e59200640 1 -- start start 2024-04-04T06:08:54.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.622+0000 7f4e59200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e54072720 0x7f4e5414ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:54.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.622+0000 7f4e59200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e540ae4f0 0x7f4e5409d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:54.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.622+0000 7f4e59200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e540b3940 0x7f4e540ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:54.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.623+0000 7f4e59200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5406e690 con 0x7f4e540b3940 2024-04-04T06:08:54.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.623+0000 7f4e59200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4e540b3db0 con 0x7f4e54072720 2024-04-04T06:08:54.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.623+0000 7f4e59200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4e5415eab0 con 0x7f4e540ae4f0 2024-04-04T06:08:54.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.623+0000 7f4e52800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e540b3940 0x7f4e540ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:54.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.623+0000 7f4e52800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e540b3940 0x7f4e540ae960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37198/0 (socket says 172.21.15.159:37198) 2024-04-04T06:08:54.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.623+0000 7f4e52800640 1 -- 172.21.15.159:0/439362474 learned_addr learned my addr 172.21.15.159:0/439362474 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:54.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.623+0000 7f4e51400640 1 --2- 172.21.15.159:0/439362474 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e540ae4f0 0x7f4e5409d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:54.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.623+0000 7f4e51e00640 1 --2- 172.21.15.159:0/439362474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e54072720 0x7f4e5414ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:54.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.623+0000 7f4e52800640 1 -- 172.21.15.159:0/439362474 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e540ae4f0 msgr2=0x7f4e5409d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:54.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.623+0000 7f4e52800640 1 --2- 172.21.15.159:0/439362474 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e540ae4f0 0x7f4e5409d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:54.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.623+0000 7f4e52800640 1 -- 172.21.15.159:0/439362474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e54072720 msgr2=0x7f4e5414ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:54.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.623+0000 7f4e52800640 1 --2- 172.21.15.159:0/439362474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e54072720 0x7f4e5414ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:54.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.623+0000 7f4e52800640 1 -- 172.21.15.159:0/439362474 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5415ae70 con 0x7f4e540b3940 2024-04-04T06:08:54.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.624+0000 7f4e52800640 1 --2- 172.21.15.159:0/439362474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e540b3940 0x7f4e540ae960 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00cd50 tx=0x7f4e4c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:54.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.624+0000 7f4e4a000640 1 -- 172.21.15.159:0/439362474 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c00f680 con 0x7f4e540b3940 2024-04-04T06:08:54.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.624+0000 7f4e4a000640 1 -- 172.21.15.159:0/439362474 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e4c00f840 con 0x7f4e540b3940 2024-04-04T06:08:54.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.624+0000 7f4e59200640 1 -- 172.21.15.159:0/439362474 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5415b150 con 0x7f4e540b3940 2024-04-04T06:08:54.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.624+0000 7f4e4a000640 1 -- 172.21.15.159:0/439362474 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c00fc30 con 0x7f4e540b3940 2024-04-04T06:08:54.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.624+0000 7f4e59200640 1 -- 172.21.15.159:0/439362474 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e5415b710 con 0x7f4e540b3940 2024-04-04T06:08:54.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.625+0000 7f4e59200640 1 -- 172.21.15.159:0/439362474 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e540685c0 con 0x7f4e540b3940 2024-04-04T06:08:54.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.626+0000 7f4e4a000640 1 -- 172.21.15.159:0/439362474 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e4c0213f0 con 0x7f4e540b3940 2024-04-04T06:08:54.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.627+0000 7f4e4a000640 1 --2- 172.21.15.159:0/439362474 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4e3406c8f0 0x7f4e3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:54.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.627+0000 7f4e4a000640 1 -- 172.21.15.159:0/439362474 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7f4e4c08c640 con 0x7f4e540b3940 2024-04-04T06:08:54.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.628+0000 7f4e51e00640 1 --2- 172.21.15.159:0/439362474 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4e3406c8f0 0x7f4e3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:54.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.629+0000 7f4e51e00640 1 --2- 172.21.15.159:0/439362474 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4e3406c8f0 0x7f4e3406edb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4e44006fd0 tx=0x7f4e44008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:54.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.631+0000 7f4e4a000640 1 -- 172.21.15.159:0/439362474 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4e4c014030 con 0x7f4e540b3940 2024-04-04T06:08:54.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:54.831+0000 7f4e59200640 1 -- 172.21.15.159:0/439362474 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4e5415bfa0 con 0x7f4e540b3940 2024-04-04T06:08:55.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.537+0000 7f4e4a000640 1 -- 172.21.15.159:0/439362474 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v56) ==== 138+0+0 (secure 0 0 0) 0x7f4e4c05ad20 con 0x7f4e540b3940 2024-04-04T06:08:55.531 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_metadata' created 2024-04-04T06:08:55.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.541+0000 7f4e59200640 1 -- 172.21.15.159:0/439362474 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4e3406c8f0 msgr2=0x7f4e3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:55.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.541+0000 7f4e59200640 1 --2- 172.21.15.159:0/439362474 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4e3406c8f0 0x7f4e3406edb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4e44006fd0 tx=0x7f4e44008040 comp rx=0 tx=0).stop 2024-04-04T06:08:55.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.541+0000 7f4e59200640 1 -- 172.21.15.159:0/439362474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e540b3940 msgr2=0x7f4e540ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:55.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.541+0000 7f4e59200640 1 --2- 172.21.15.159:0/439362474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e540b3940 0x7f4e540ae960 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00cd50 tx=0x7f4e4c00a650 comp rx=0 tx=0).stop 2024-04-04T06:08:55.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.541+0000 7f4e59200640 1 -- 172.21.15.159:0/439362474 shutdown_connections 2024-04-04T06:08:55.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.541+0000 7f4e59200640 1 --2- 172.21.15.159:0/439362474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e540b3940 0x7f4e540ae960 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:55.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.541+0000 7f4e59200640 1 --2- 172.21.15.159:0/439362474 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4e540ae4f0 0x7f4e5409d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:55.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.541+0000 7f4e59200640 1 --2- 172.21.15.159:0/439362474 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4e3406c8f0 0x7f4e3406edb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:55.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.541+0000 7f4e59200640 1 --2- 172.21.15.159:0/439362474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e54072720 0x7f4e5414ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:55.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.541+0000 7f4e59200640 1 -- 172.21.15.159:0/439362474 >> 172.21.15.159:0/439362474 conn(0x7f4e5406bba0 msgr2=0x7f4e5414d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:55.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.542+0000 7f4e59200640 1 -- 172.21.15.159:0/439362474 shutdown_connections 2024-04-04T06:08:55.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.542+0000 7f4e59200640 1 -- 172.21.15.159:0/439362474 wait complete. 2024-04-04T06:08:55.560 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-04T06:08:55.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.765+0000 7f3995c00640 1 Processor -- start 2024-04-04T06:08:55.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.765+0000 7f3995c00640 1 -- start start 2024-04-04T06:08:55.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.767+0000 7f3995c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3990153580 0x7f39900d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:55.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.767+0000 7f3995c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39900afab0 con 0x7f39900d07c0 2024-04-04T06:08:55.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.767+0000 7f3995c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f399015cc80 con 0x7f39900af4b0 2024-04-04T06:08:55.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.767+0000 7f3995c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f39900d0c30 con 0x7f3990153580 2024-04-04T06:08:55.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.767+0000 7f398e000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f39900d07c0 0x7f39900aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40268/0 (socket says 172.21.15.159:40268) 2024-04-04T06:08:55.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.767+0000 7f398e000640 1 -- 172.21.15.159:0/3556962342 learned_addr learned my addr 172.21.15.159:0/3556962342 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:55.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.768+0000 7f398ea00640 1 --2- 172.21.15.159:0/3556962342 >> v2:172.21.15.178:3300/0 conn(0x7f3990153580 0x7f39900d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:55.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.768+0000 7f398d600640 1 -- 172.21.15.159:0/3556962342 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4011501118 0 0) 0x7f39900afab0 con 0x7f39900d07c0 2024-04-04T06:08:55.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.768+0000 7f398d600640 1 -- 172.21.15.159:0/3556962342 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3974003520 con 0x7f39900d07c0 2024-04-04T06:08:55.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.768+0000 7f398d600640 1 -- 172.21.15.159:0/3556962342 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2114176137 0 0) 0x7f3974003520 con 0x7f39900d07c0 2024-04-04T06:08:55.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.768+0000 7f398d600640 1 -- 172.21.15.159:0/3556962342 >> v2:172.21.15.178:3300/0 conn(0x7f3990153580 msgr2=0x7f39900d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:55.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.768+0000 7f398d600640 1 --2- 172.21.15.159:0/3556962342 >> v2:172.21.15.178:3300/0 conn(0x7f3990153580 0x7f39900d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:55.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.768+0000 7f398d600640 1 -- 172.21.15.159:0/3556962342 >> v1:172.21.15.178:6789/0 conn(0x7f39900af4b0 legacy=0x7f39901600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:55.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.768+0000 7f398d600640 1 -- 172.21.15.159:0/3556962342 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f399015da00 con 0x7f39900d07c0 2024-04-04T06:08:55.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.768+0000 7f398d600640 1 -- 172.21.15.159:0/3556962342 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f3978001670 con 0x7f39900d07c0 2024-04-04T06:08:55.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.768+0000 7f398d600640 1 -- 172.21.15.159:0/3556962342 >> v1:172.21.15.159:6789/0 conn(0x7f39900d07c0 legacy=0x7f39900aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:55.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.768+0000 7f398d600640 1 --2- 172.21.15.159:0/3556962342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39740039b0 0x7f3974003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:55.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.768+0000 7f398d600640 1 --2- 172.21.15.159:0/3556962342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3974004800 0x7f3974004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:55.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.768+0000 7f398d600640 1 --2- 172.21.15.159:0/3556962342 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39740051c0 0x7f397400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:55.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.769+0000 7f398d600640 1 -- 172.21.15.159:0/3556962342 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f39900d0c30 con 0x7f39740039b0 2024-04-04T06:08:55.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.769+0000 7f398d600640 1 -- 172.21.15.159:0/3556962342 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f397400c160 con 0x7f3974004800 2024-04-04T06:08:55.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.769+0000 7f398d600640 1 -- 172.21.15.159:0/3556962342 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f397400c360 con 0x7f39740051c0 2024-04-04T06:08:55.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.769+0000 7f398ea00640 1 --2- 172.21.15.159:0/3556962342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39740039b0 0x7f3974003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:55.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.769+0000 7f398e000640 1 --2- 172.21.15.159:0/3556962342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3974004800 0x7f3974004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:55.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.769+0000 7f398ea00640 1 -- 172.21.15.159:0/3556962342 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39740051c0 msgr2=0x7f397400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:55.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.769+0000 7f398ea00640 1 --2- 172.21.15.159:0/3556962342 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39740051c0 0x7f397400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:55.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.769+0000 7f398ea00640 1 -- 172.21.15.159:0/3556962342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3974004800 msgr2=0x7f3974004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:55.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.769+0000 7f398ea00640 1 --2- 172.21.15.159:0/3556962342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3974004800 0x7f3974004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:55.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.769+0000 7f398ea00640 1 -- 172.21.15.159:0/3556962342 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3974003520 con 0x7f39740039b0 2024-04-04T06:08:55.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.769+0000 7f398f400640 1 -- 172.21.15.159:0/3556962342 reap_dead start 2024-04-04T06:08:55.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.770+0000 7f398ea00640 1 --2- 172.21.15.159:0/3556962342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39740039b0 0x7f3974003dd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3980009ac0 tx=0x7f3974005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93d382c5f74d428 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:55.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.771+0000 7f398d600640 1 -- 172.21.15.159:0/3556962342 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980039070 con 0x7f39740039b0 2024-04-04T06:08:55.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.771+0000 7f398d600640 1 -- 172.21.15.159:0/3556962342 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f398002c6d0 con 0x7f39740039b0 2024-04-04T06:08:55.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.771+0000 7f398d600640 1 -- 172.21.15.159:0/3556962342 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980009690 con 0x7f39740039b0 2024-04-04T06:08:55.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.771+0000 7f3995c00640 1 -- 172.21.15.159:0/3556962342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39740039b0 msgr2=0x7f3974003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:55.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.771+0000 7f3995c00640 1 --2- 172.21.15.159:0/3556962342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39740039b0 0x7f3974003dd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3980009ac0 tx=0x7f3974005830 comp rx=0 tx=0).stop 2024-04-04T06:08:55.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.771+0000 7f3995c00640 1 -- 172.21.15.159:0/3556962342 shutdown_connections 2024-04-04T06:08:55.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.771+0000 7f3995c00640 1 --2- 172.21.15.159:0/3556962342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39740039b0 0x7f3974003dd0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:55.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.771+0000 7f3995c00640 1 -- 172.21.15.159:0/3556962342 >> 172.21.15.159:0/3556962342 conn(0x7f399006b750 msgr2=0x7f399006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:55.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.771+0000 7f3995c00640 1 -- 172.21.15.159:0/3556962342 shutdown_connections 2024-04-04T06:08:55.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.771+0000 7f3995c00640 1 -- 172.21.15.159:0/3556962342 wait complete. 2024-04-04T06:08:55.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.773+0000 7f3995c00640 1 Processor -- start 2024-04-04T06:08:55.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.773+0000 7f3995c00640 1 -- start start 2024-04-04T06:08:55.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.774+0000 7f3995c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39740039b0 0x7f399015b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:55.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.774+0000 7f3995c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3990071610 0x7f3990148340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:55.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.774+0000 7f3995c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3990147600 0x7f3990147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:55.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.774+0000 7f3995c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f399015d4e0 con 0x7f3990147600 2024-04-04T06:08:55.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.774+0000 7f3995c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f399015d200 con 0x7f39740039b0 2024-04-04T06:08:55.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.774+0000 7f3995c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3990153960 con 0x7f3990071610 2024-04-04T06:08:55.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.775+0000 7f398f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3990147600 0x7f3990147ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:55.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.775+0000 7f398f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3990147600 0x7f3990147ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37226/0 (socket says 172.21.15.159:37226) 2024-04-04T06:08:55.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.775+0000 7f398f400640 1 -- 172.21.15.159:0/945083248 learned_addr learned my addr 172.21.15.159:0/945083248 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:55.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.775+0000 7f398ea00640 1 --2- 172.21.15.159:0/945083248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39740039b0 0x7f399015b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:55.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.775+0000 7f398e000640 1 --2- 172.21.15.159:0/945083248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3990071610 0x7f3990148340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:55.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.775+0000 7f398f400640 1 -- 172.21.15.159:0/945083248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3990071610 msgr2=0x7f3990148340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:55.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.775+0000 7f398f400640 1 --2- 172.21.15.159:0/945083248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3990071610 0x7f3990148340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:55.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.775+0000 7f398f400640 1 -- 172.21.15.159:0/945083248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39740039b0 msgr2=0x7f399015b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:55.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.775+0000 7f398f400640 1 --2- 172.21.15.159:0/945083248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39740039b0 0x7f399015b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:55.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.775+0000 7f398f400640 1 -- 172.21.15.159:0/945083248 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3990071aa0 con 0x7f3990147600 2024-04-04T06:08:55.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.775+0000 7f398e000640 1 --2- 172.21.15.159:0/945083248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3990071610 0x7f3990148340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:55.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.775+0000 7f398f400640 1 --2- 172.21.15.159:0/945083248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3990147600 0x7f3990147ab0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f399006c530 tx=0x7f398800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:55.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.776+0000 7f3986a00640 1 -- 172.21.15.159:0/945083248 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988017070 con 0x7f3990147600 2024-04-04T06:08:55.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.776+0000 7f3986a00640 1 -- 172.21.15.159:0/945083248 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3988005d40 con 0x7f3990147600 2024-04-04T06:08:55.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.776+0000 7f3986a00640 1 -- 172.21.15.159:0/945083248 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398800a850 con 0x7f3990147600 2024-04-04T06:08:55.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.776+0000 7f3995c00640 1 -- 172.21.15.159:0/945083248 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3990071d80 con 0x7f3990147600 2024-04-04T06:08:55.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.776+0000 7f3995c00640 1 -- 172.21.15.159:0/945083248 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3990072340 con 0x7f3990147600 2024-04-04T06:08:55.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.777+0000 7f3995c00640 1 -- 172.21.15.159:0/945083248 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39900ae140 con 0x7f3990147600 2024-04-04T06:08:55.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.778+0000 7f3986a00640 1 -- 172.21.15.159:0/945083248 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f398800a9f0 con 0x7f3990147600 2024-04-04T06:08:55.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.779+0000 7f3986a00640 1 --2- 172.21.15.159:0/945083248 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f396c06c8f0 0x7f396c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:55.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.779+0000 7f3986a00640 1 -- 172.21.15.159:0/945083248 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5227+0+0 (secure 0 0 0) 0x7f398808aae0 con 0x7f3990147600 2024-04-04T06:08:55.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.780+0000 7f398ea00640 1 --2- 172.21.15.159:0/945083248 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f396c06c8f0 0x7f396c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:55.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.781+0000 7f398ea00640 1 --2- 172.21.15.159:0/945083248 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f396c06c8f0 0x7f396c06edb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3980036000 tx=0x7f3980030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:55.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.785+0000 7f3986a00640 1 -- 172.21.15.159:0/945083248 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f39900ae140 con 0x7f3990147600 2024-04-04T06:08:55.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:55.985+0000 7f3995c00640 1 -- 172.21.15.159:0/945083248 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f39900685c0 con 0x7f3990147600 2024-04-04T06:08:56.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.543+0000 7f3986a00640 1 -- 172.21.15.159:0/945083248 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v57) ==== 170+0+0 (secure 0 0 0) 0x7f39880591a0 con 0x7f3990147600 2024-04-04T06:08:56.537 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_data' created 2024-04-04T06:08:56.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.547+0000 7f3995c00640 1 -- 172.21.15.159:0/945083248 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f396c06c8f0 msgr2=0x7f396c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:56.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.547+0000 7f3995c00640 1 --2- 172.21.15.159:0/945083248 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f396c06c8f0 0x7f396c06edb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3980036000 tx=0x7f3980030000 comp rx=0 tx=0).stop 2024-04-04T06:08:56.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.547+0000 7f3995c00640 1 -- 172.21.15.159:0/945083248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3990147600 msgr2=0x7f3990147ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:56.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.547+0000 7f3995c00640 1 --2- 172.21.15.159:0/945083248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3990147600 0x7f3990147ab0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f399006c530 tx=0x7f398800cb90 comp rx=0 tx=0).stop 2024-04-04T06:08:56.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.548+0000 7f3995c00640 1 -- 172.21.15.159:0/945083248 shutdown_connections 2024-04-04T06:08:56.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.548+0000 7f3995c00640 1 --2- 172.21.15.159:0/945083248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3990147600 0x7f3990147ab0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:56.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.548+0000 7f3995c00640 1 --2- 172.21.15.159:0/945083248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3990071610 0x7f3990148340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:56.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.548+0000 7f3995c00640 1 --2- 172.21.15.159:0/945083248 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f396c06c8f0 0x7f396c06edb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:56.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.548+0000 7f3995c00640 1 --2- 172.21.15.159:0/945083248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39740039b0 0x7f399015b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:56.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.548+0000 7f3995c00640 1 -- 172.21.15.159:0/945083248 >> 172.21.15.159:0/945083248 conn(0x7f399006b750 msgr2=0x7f39900b0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:56.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.549+0000 7f3995c00640 1 -- 172.21.15.159:0/945083248 shutdown_connections 2024-04-04T06:08:56.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.549+0000 7f3995c00640 1 -- 172.21.15.159:0/945083248 wait complete. 2024-04-04T06:08:56.578 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-04T06:08:56.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.766+0000 7f1385a00640 1 Processor -- start 2024-04-04T06:08:56.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.766+0000 7f1385a00640 1 -- start start 2024-04-04T06:08:56.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.768+0000 7f1385a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f138006dd30 0x7f13800cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:56.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.768+0000 7f1385a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13800ce890 con 0x7f13800cc880 2024-04-04T06:08:56.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.768+0000 7f1385a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13800ceb50 con 0x7f138013f190 2024-04-04T06:08:56.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.768+0000 7f1385a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f13800cd230 con 0x7f138006dd30 2024-04-04T06:08:56.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.768+0000 7f1385000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f138006dd30 0x7f13800cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:56.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.768+0000 7f1385000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f138006dd30 0x7f13800cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37236/0 (socket says 172.21.15.159:37236) 2024-04-04T06:08:56.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.768+0000 7f1385000640 1 -- 172.21.15.159:0/1104319296 learned_addr learned my addr 172.21.15.159:0/1104319296 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:56.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.769+0000 7f1385000640 1 -- 172.21.15.159:0/1104319296 >> v1:172.21.15.178:6789/0 conn(0x7f138013f190 legacy=0x7f13800cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:56.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.769+0000 7f1385000640 1 -- 172.21.15.159:0/1104319296 >> v1:172.21.15.159:6789/0 conn(0x7f13800cc880 legacy=0x7f138006d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:56.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.769+0000 7f1385000640 1 -- 172.21.15.159:0/1104319296 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13800cee30 con 0x7f138006dd30 2024-04-04T06:08:56.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.769+0000 7f137ea00640 1 -- 172.21.15.159:0/1104319296 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 596000832 0 0) 0x7f13800ce890 con 0x7f13800cc880 2024-04-04T06:08:56.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.770+0000 7f1385000640 1 --2- 172.21.15.159:0/1104319296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f138006dd30 0x7f13800cccf0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1378009870 tx=0x7f137802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c82e150975da30 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:56.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.770+0000 7f137ea00640 1 -- 172.21.15.159:0/1104319296 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1378034070 con 0x7f138006dd30 2024-04-04T06:08:56.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.770+0000 7f137ea00640 1 -- 172.21.15.159:0/1104319296 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1378030860 con 0x7f138006dd30 2024-04-04T06:08:56.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.770+0000 7f137ea00640 1 -- 172.21.15.159:0/1104319296 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1378030c30 con 0x7f138006dd30 2024-04-04T06:08:56.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.771+0000 7f1385a00640 1 -- 172.21.15.159:0/1104319296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f138006dd30 msgr2=0x7f13800cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:56.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.771+0000 7f1385a00640 1 --2- 172.21.15.159:0/1104319296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f138006dd30 0x7f13800cccf0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1378009870 tx=0x7f137802b9a0 comp rx=0 tx=0).stop 2024-04-04T06:08:56.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.771+0000 7f1385a00640 1 -- 172.21.15.159:0/1104319296 shutdown_connections 2024-04-04T06:08:56.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.771+0000 7f1385a00640 1 --2- 172.21.15.159:0/1104319296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f138006dd30 0x7f13800cccf0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:56.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.771+0000 7f1385a00640 1 -- 172.21.15.159:0/1104319296 >> 172.21.15.159:0/1104319296 conn(0x7f13800b1ba0 msgr2=0x7f13800b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:56.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.771+0000 7f1385a00640 1 -- 172.21.15.159:0/1104319296 shutdown_connections 2024-04-04T06:08:56.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.771+0000 7f1385a00640 1 -- 172.21.15.159:0/1104319296 wait complete. 2024-04-04T06:08:56.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.773+0000 7f1385a00640 1 Processor -- start 2024-04-04T06:08:56.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.773+0000 7f1385a00640 1 -- start start 2024-04-04T06:08:56.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.774+0000 7f1385a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f138006dd30 0x7f138013c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:56.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.774+0000 7f1385a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13800cc880 0x7f13801586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:56.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.774+0000 7f1385a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f138013f190 0x7f1380158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:56.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.774+0000 7f1385a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f138013f570 con 0x7f13800cc880 2024-04-04T06:08:56.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.774+0000 7f1385a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1380160c70 con 0x7f138006dd30 2024-04-04T06:08:56.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.774+0000 7f1385a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f138015ff10 con 0x7f138013f190 2024-04-04T06:08:56.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.774+0000 7f137f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13800cc880 0x7f13801586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:56.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.774+0000 7f137f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13800cc880 0x7f13801586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37240/0 (socket says 172.21.15.159:37240) 2024-04-04T06:08:56.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.774+0000 7f137f400640 1 -- 172.21.15.159:0/4089279269 learned_addr learned my addr 172.21.15.159:0/4089279269 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:56.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.774+0000 7f1385000640 1 --2- 172.21.15.159:0/4089279269 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f138013f190 0x7f1380158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:56.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.774+0000 7f137fe00640 1 --2- 172.21.15.159:0/4089279269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f138006dd30 0x7f138013c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:56.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.774+0000 7f137f400640 1 -- 172.21.15.159:0/4089279269 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f138013f190 msgr2=0x7f1380158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:56.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.774+0000 7f137f400640 1 --2- 172.21.15.159:0/4089279269 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f138013f190 0x7f1380158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:56.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.774+0000 7f137f400640 1 -- 172.21.15.159:0/4089279269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f138006dd30 msgr2=0x7f138013c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:56.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.774+0000 7f137f400640 1 --2- 172.21.15.159:0/4089279269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f138006dd30 0x7f138013c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:56.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.774+0000 7f137f400640 1 -- 172.21.15.159:0/4089279269 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138015b8a0 con 0x7f13800cc880 2024-04-04T06:08:56.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.775+0000 7f1385000640 1 --2- 172.21.15.159:0/4089279269 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f138013f190 0x7f1380158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:56.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.775+0000 7f137f400640 1 --2- 172.21.15.159:0/4089279269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13800cc880 0x7f13801586d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f136c00c4e0 tx=0x7f136c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:56.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.775+0000 7f1377e00640 1 -- 172.21.15.159:0/4089279269 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c017070 con 0x7f13800cc880 2024-04-04T06:08:56.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.775+0000 7f1377e00640 1 -- 172.21.15.159:0/4089279269 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f136c0035f0 con 0x7f13800cc880 2024-04-04T06:08:56.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.775+0000 7f1385a00640 1 -- 172.21.15.159:0/4089279269 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138013a390 con 0x7f13800cc880 2024-04-04T06:08:56.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.775+0000 7f1377e00640 1 -- 172.21.15.159:0/4089279269 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c0039e0 con 0x7f13800cc880 2024-04-04T06:08:56.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.775+0000 7f1385a00640 1 -- 172.21.15.159:0/4089279269 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f138015a820 con 0x7f13800cc880 2024-04-04T06:08:56.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.777+0000 7f1385a00640 1 -- 172.21.15.159:0/4089279269 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1380075940 con 0x7f13800cc880 2024-04-04T06:08:56.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.778+0000 7f1377e00640 1 -- 172.21.15.159:0/4089279269 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f136c01a070 con 0x7f13800cc880 2024-04-04T06:08:56.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.779+0000 7f1377e00640 1 --2- 172.21.15.159:0/4089279269 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f136006c8f0 0x7f136006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:56.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.779+0000 7f1377e00640 1 -- 172.21.15.159:0/4089279269 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5609+0+0 (secure 0 0 0) 0x7f136c08add0 con 0x7f13800cc880 2024-04-04T06:08:56.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.780+0000 7f137fe00640 1 --2- 172.21.15.159:0/4089279269 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f136006c8f0 0x7f136006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:56.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.781+0000 7f137fe00640 1 --2- 172.21.15.159:0/4089279269 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f136006c8f0 0x7f136006edb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1370007fd0 tx=0x7f1370009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:56.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:56.785+0000 7f1377e00640 1 -- 172.21.15.159:0/4089279269 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f136c058c10 con 0x7f13800cc880 2024-04-04T06:08:57.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.018+0000 7f1385a00640 1 -- 172.21.15.159:0/4089279269 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1380073850 con 0x7f13800cc880 2024-04-04T06:08:57.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.583+0000 7f1377e00640 1 -- 172.21.15.159:0/4089279269 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v30) ==== 172+0+0 (secure 0 0 0) 0x7f136c08c070 con 0x7f13800cc880 2024-04-04T06:08:57.578 INFO:teuthology.orchestra.run.smithi159.stderr:new fs with metadata pool 8 and data pool 9 2024-04-04T06:08:57.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.591+0000 7f1385a00640 1 -- 172.21.15.159:0/4089279269 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f136006c8f0 msgr2=0x7f136006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:57.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.591+0000 7f1385a00640 1 --2- 172.21.15.159:0/4089279269 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f136006c8f0 0x7f136006edb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1370007fd0 tx=0x7f1370009040 comp rx=0 tx=0).stop 2024-04-04T06:08:57.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.591+0000 7f1385a00640 1 -- 172.21.15.159:0/4089279269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13800cc880 msgr2=0x7f13801586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:57.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.591+0000 7f1385a00640 1 --2- 172.21.15.159:0/4089279269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13800cc880 0x7f13801586d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f136c00c4e0 tx=0x7f136c00c9a0 comp rx=0 tx=0).stop 2024-04-04T06:08:57.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.595+0000 7f1385a00640 1 -- 172.21.15.159:0/4089279269 shutdown_connections 2024-04-04T06:08:57.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.595+0000 7f1385a00640 1 --2- 172.21.15.159:0/4089279269 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f138013f190 0x7f1380158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:57.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.595+0000 7f1385a00640 1 --2- 172.21.15.159:0/4089279269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13800cc880 0x7f13801586d0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:57.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.595+0000 7f1385a00640 1 --2- 172.21.15.159:0/4089279269 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f136006c8f0 0x7f136006edb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:57.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.595+0000 7f1385a00640 1 --2- 172.21.15.159:0/4089279269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f138006dd30 0x7f138013c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:57.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.595+0000 7f1385a00640 1 -- 172.21.15.159:0/4089279269 >> 172.21.15.159:0/4089279269 conn(0x7f13800b1ba0 msgr2=0x7f138006d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:57.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.596+0000 7f1385a00640 1 -- 172.21.15.159:0/4089279269 shutdown_connections 2024-04-04T06:08:57.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.596+0000 7f1385a00640 1 -- 172.21.15.159:0/4089279269 wait complete. 2024-04-04T06:08:57.621 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:08:57.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.798+0000 7f83cfe00640 1 Processor -- start 2024-04-04T06:08:57.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.798+0000 7f83cfe00640 1 -- start start 2024-04-04T06:08:57.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.799+0000 7f83cfe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d00d0120 0x7f83d0162880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:57.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.799+0000 7f83cfe00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d00d08c0 con 0x7f83d006b570 2024-04-04T06:08:57.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.799+0000 7f83cfe00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d00d0ba0 con 0x7f83d015e6e0 2024-04-04T06:08:57.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.799+0000 7f83cfe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f83d00d0e80 con 0x7f83d00d0120 2024-04-04T06:08:57.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.799+0000 7f83cf400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f83d006b570 0x7f83d015eac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40278/0 (socket says 172.21.15.159:40278) 2024-04-04T06:08:57.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.799+0000 7f83cf400640 1 -- 172.21.15.159:0/3155795729 learned_addr learned my addr 172.21.15.159:0/3155795729 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:57.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.800+0000 7f83ce000640 1 --2- 172.21.15.159:0/3155795729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d00d0120 0x7f83d0162880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:57.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.800+0000 7f83cd600640 1 -- 172.21.15.159:0/3155795729 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4220078844 0 0) 0x7f83d00d08c0 con 0x7f83d006b570 2024-04-04T06:08:57.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.800+0000 7f83cd600640 1 -- 172.21.15.159:0/3155795729 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83b4003520 con 0x7f83d006b570 2024-04-04T06:08:57.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.800+0000 7f83cd600640 1 -- 172.21.15.159:0/3155795729 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3883873586 0 0) 0x7f83b4003520 con 0x7f83d006b570 2024-04-04T06:08:57.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.800+0000 7f83cd600640 1 -- 172.21.15.159:0/3155795729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d00d0120 msgr2=0x7f83d0162880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:57.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.800+0000 7f83cd600640 1 --2- 172.21.15.159:0/3155795729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d00d0120 0x7f83d0162880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:57.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.800+0000 7f83cd600640 1 -- 172.21.15.159:0/3155795729 >> v1:172.21.15.178:6789/0 conn(0x7f83d015e6e0 legacy=0x7f83d00b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:57.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.800+0000 7f83cd600640 1 -- 172.21.15.159:0/3155795729 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d0162080 con 0x7f83d006b570 2024-04-04T06:08:57.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.800+0000 7f83cd600640 1 -- 172.21.15.159:0/3155795729 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f83c80019f0 con 0x7f83d006b570 2024-04-04T06:08:57.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.800+0000 7f83cd600640 1 -- 172.21.15.159:0/3155795729 >> v1:172.21.15.159:6789/0 conn(0x7f83d006b570 legacy=0x7f83d015eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:57.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.800+0000 7f83cd600640 1 --2- 172.21.15.159:0/3155795729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83b40039e0 0x7f83b4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:57.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.801+0000 7f83cd600640 1 --2- 172.21.15.159:0/3155795729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d00d0120 0x7f83b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:57.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.801+0000 7f83cd600640 1 --2- 172.21.15.159:0/3155795729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83b4004d40 0x7f83b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:57.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.801+0000 7f83cd600640 1 -- 172.21.15.159:0/3155795729 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f83d00d0e80 con 0x7f83b40039e0 2024-04-04T06:08:57.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.801+0000 7f83cd600640 1 -- 172.21.15.159:0/3155795729 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f83b400ba70 con 0x7f83b4004d40 2024-04-04T06:08:57.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.801+0000 7f83cd600640 1 -- 172.21.15.159:0/3155795729 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f83b400bc70 con 0x7f83d00d0120 2024-04-04T06:08:57.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.801+0000 7f83cea00640 1 --2- 172.21.15.159:0/3155795729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83b40039e0 0x7f83b4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:57.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.801+0000 7f83ce000640 1 --2- 172.21.15.159:0/3155795729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d00d0120 0x7f83b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:57.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.801+0000 7f83cea00640 1 -- 172.21.15.159:0/3155795729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d00d0120 msgr2=0x7f83b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:57.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.801+0000 7f83cea00640 1 --2- 172.21.15.159:0/3155795729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d00d0120 0x7f83b4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:57.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.801+0000 7f83cea00640 1 -- 172.21.15.159:0/3155795729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83b4004d40 msgr2=0x7f83b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:57.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.801+0000 7f83cea00640 1 --2- 172.21.15.159:0/3155795729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83b4004d40 0x7f83b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:57.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.801+0000 7f83cea00640 1 -- 172.21.15.159:0/3155795729 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b4003520 con 0x7f83b40039e0 2024-04-04T06:08:57.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.803+0000 7f83cea00640 1 --2- 172.21.15.159:0/3155795729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83b40039e0 0x7f83b4003e00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f83c0001720 tx=0x7f83b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744dad725e377a24 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:57.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.803+0000 7f83cd600640 1 -- 172.21.15.159:0/3155795729 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0001370 con 0x7f83b40039e0 2024-04-04T06:08:57.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.803+0000 7f83cd600640 1 -- 172.21.15.159:0/3155795729 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c00052d0 con 0x7f83b40039e0 2024-04-04T06:08:57.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.803+0000 7f83cd600640 1 -- 172.21.15.159:0/3155795729 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0003830 con 0x7f83b40039e0 2024-04-04T06:08:57.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.804+0000 7f83cfe00640 1 -- 172.21.15.159:0/3155795729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83b40039e0 msgr2=0x7f83b4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:57.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.804+0000 7f83cfe00640 1 --2- 172.21.15.159:0/3155795729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83b40039e0 0x7f83b4003e00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f83c0001720 tx=0x7f83b4005830 comp rx=0 tx=0).stop 2024-04-04T06:08:57.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.804+0000 7f83cf400640 1 -- 172.21.15.159:0/3155795729 reap_dead start 2024-04-04T06:08:57.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.804+0000 7f83cfe00640 1 -- 172.21.15.159:0/3155795729 shutdown_connections 2024-04-04T06:08:57.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.804+0000 7f83cfe00640 1 -- 172.21.15.159:0/3155795729 >> 172.21.15.159:0/3155795729 conn(0x7f83d009aa20 msgr2=0x7f83d00b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:57.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.804+0000 7f83cfe00640 1 -- 172.21.15.159:0/3155795729 shutdown_connections 2024-04-04T06:08:57.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.804+0000 7f83cfe00640 1 -- 172.21.15.159:0/3155795729 wait complete. 2024-04-04T06:08:57.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.805+0000 7f83cfe00640 1 Processor -- start 2024-04-04T06:08:57.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.806+0000 7f83cfe00640 1 -- start start 2024-04-04T06:08:57.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.806+0000 7f83cfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83b40039e0 0x7f83d00c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:57.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.806+0000 7f83cfe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d01f51e0 0x7f83d00cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:57.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.806+0000 7f83cfe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d00c8c50 0x7f83d00ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:57.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.806+0000 7f83cfe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f83d015f350 con 0x7f83b40039e0 2024-04-04T06:08:57.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.806+0000 7f83cfe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f83d015f1d0 con 0x7f83d01f51e0 2024-04-04T06:08:57.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.806+0000 7f83cfe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f83d015f4d0 con 0x7f83d00c8c50 2024-04-04T06:08:57.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.807+0000 7f83cea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83b40039e0 0x7f83d00c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:57.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.807+0000 7f83cea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83b40039e0 0x7f83d00c5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37252/0 (socket says 172.21.15.159:37252) 2024-04-04T06:08:57.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.807+0000 7f83cea00640 1 -- 172.21.15.159:0/1389597970 learned_addr learned my addr 172.21.15.159:0/1389597970 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:57.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.807+0000 7f83ce000640 1 --2- 172.21.15.159:0/1389597970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d01f51e0 0x7f83d00cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:57.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.807+0000 7f83cf400640 1 --2- 172.21.15.159:0/1389597970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d00c8c50 0x7f83d00ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:57.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.807+0000 7f83cea00640 1 -- 172.21.15.159:0/1389597970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d00c8c50 msgr2=0x7f83d00ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:57.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.807+0000 7f83cea00640 1 --2- 172.21.15.159:0/1389597970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d00c8c50 0x7f83d00ae020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:57.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.807+0000 7f83cea00640 1 -- 172.21.15.159:0/1389597970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d01f51e0 msgr2=0x7f83d00cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:57.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.807+0000 7f83cea00640 1 --2- 172.21.15.159:0/1389597970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d01f51e0 0x7f83d00cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:57.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.807+0000 7f83cea00640 1 -- 172.21.15.159:0/1389597970 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d00cb3a0 con 0x7f83b40039e0 2024-04-04T06:08:57.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.807+0000 7f83cea00640 1 --2- 172.21.15.159:0/1389597970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83b40039e0 0x7f83d00c5b30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f83c0008fd0 tx=0x7f83c0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:57.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.808+0000 7f83c7400640 1 -- 172.21.15.159:0/1389597970 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0041070 con 0x7f83b40039e0 2024-04-04T06:08:57.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.808+0000 7f83c7400640 1 -- 172.21.15.159:0/1389597970 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c0005480 con 0x7f83b40039e0 2024-04-04T06:08:57.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.808+0000 7f83c7400640 1 -- 172.21.15.159:0/1389597970 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0007a30 con 0x7f83b40039e0 2024-04-04T06:08:57.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.808+0000 7f83cfe00640 1 -- 172.21.15.159:0/1389597970 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d01f5660 con 0x7f83b40039e0 2024-04-04T06:08:57.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.808+0000 7f83cfe00640 1 -- 172.21.15.159:0/1389597970 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d00c6540 con 0x7f83b40039e0 2024-04-04T06:08:57.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.809+0000 7f83cfe00640 1 -- 172.21.15.159:0/1389597970 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d00c6780 con 0x7f83b40039e0 2024-04-04T06:08:57.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.811+0000 7f83c7400640 1 -- 172.21.15.159:0/1389597970 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83c0044070 con 0x7f83b40039e0 2024-04-04T06:08:57.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.812+0000 7f83c7400640 1 --2- 172.21.15.159:0/1389597970 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f83b006c920 0x7f83b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:57.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.812+0000 7f83c7400640 1 -- 172.21.15.159:0/1389597970 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f83c00ad750 con 0x7f83b40039e0 2024-04-04T06:08:57.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.812+0000 7f83ce000640 1 --2- 172.21.15.159:0/1389597970 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f83b006c920 0x7f83b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:57.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.813+0000 7f83ce000640 1 --2- 172.21.15.159:0/1389597970 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f83b006c920 0x7f83b006ede0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f83d00aec30 tx=0x7f83bc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:57.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:57.817+0000 7f83c7400640 1 -- 172.21.15.159:0/1389597970 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f83c007bae0 con 0x7f83b40039e0 2024-04-04T06:08:58.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.016+0000 7f83cfe00640 1 -- 172.21.15.159:0/1389597970 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83d0075940 con 0x7f83b40039e0 2024-04-04T06:08:58.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.019+0000 7f83c7400640 1 -- 172.21.15.159:0/1389597970 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16690 (secure 0 0 0) 0x7f83c007b3e0 con 0x7f83b40039e0 2024-04-04T06:08:58.013 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:58.013 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":58,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:08:57.019980+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:08:54.848337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-04T06:08:56.003356+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:08:58.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.022+0000 7f83cfe00640 1 -- 172.21.15.159:0/1389597970 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f83b006c920 msgr2=0x7f83b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:58.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.022+0000 7f83cfe00640 1 --2- 172.21.15.159:0/1389597970 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f83b006c920 0x7f83b006ede0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f83d00aec30 tx=0x7f83bc008040 comp rx=0 tx=0).stop 2024-04-04T06:08:58.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.022+0000 7f83cfe00640 1 -- 172.21.15.159:0/1389597970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83b40039e0 msgr2=0x7f83d00c5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:58.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.022+0000 7f83cfe00640 1 --2- 172.21.15.159:0/1389597970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83b40039e0 0x7f83d00c5b30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f83c0008fd0 tx=0x7f83c0001120 comp rx=0 tx=0).stop 2024-04-04T06:08:58.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.022+0000 7f83cfe00640 1 -- 172.21.15.159:0/1389597970 shutdown_connections 2024-04-04T06:08:58.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.022+0000 7f83cfe00640 1 --2- 172.21.15.159:0/1389597970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d00c8c50 0x7f83d00ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:58.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.022+0000 7f83cfe00640 1 --2- 172.21.15.159:0/1389597970 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f83b006c920 0x7f83b006ede0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:58.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.022+0000 7f83cfe00640 1 --2- 172.21.15.159:0/1389597970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d01f51e0 0x7f83d00cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:58.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.022+0000 7f83cfe00640 1 --2- 172.21.15.159:0/1389597970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83b40039e0 0x7f83d00c5b30 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:58.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.022+0000 7f83cfe00640 1 -- 172.21.15.159:0/1389597970 >> 172.21.15.159:0/1389597970 conn(0x7f83d009aa20 msgr2=0x7f83d00cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:58.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.022+0000 7f83cfe00640 1 -- 172.21.15.159:0/1389597970 shutdown_connections 2024-04-04T06:08:58.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.023+0000 7f83cfe00640 1 -- 172.21.15.159:0/1389597970 wait complete. 2024-04-04T06:08:58.034 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:08:58.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.254+0000 7f671d400640 1 Processor -- start 2024-04-04T06:08:58.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.254+0000 7f671d400640 1 -- start start 2024-04-04T06:08:58.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.256+0000 7f671d400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f671815c2c0 0x7f67180d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:58.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.256+0000 7f671d400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67180cf710 0x7f67180d24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:58.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.256+0000 7f671d400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67180ccb70 con 0x7f67180d2060 2024-04-04T06:08:58.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.256+0000 7f671d400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f67180d1a80 con 0x7f671815c2c0 2024-04-04T06:08:58.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.256+0000 7f671d400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6718070110 con 0x7f67180cf710 2024-04-04T06:08:58.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.256+0000 7f6717400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f67180d2060 0x7f671815fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40288/0 (socket says 172.21.15.159:40288) 2024-04-04T06:08:58.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.256+0000 7f6717400640 1 -- 172.21.15.159:0/1199484999 learned_addr learned my addr 172.21.15.159:0/1199484999 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:58.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.256+0000 7f671ca00640 1 --2- 172.21.15.159:0/1199484999 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67180cf710 0x7f67180d24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:58.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.256+0000 7f6717e00640 1 --2- 172.21.15.159:0/1199484999 >> v2:172.21.15.178:3300/0 conn(0x7f671815c2c0 0x7f67180d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:58.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.256+0000 7f6716a00640 1 -- 172.21.15.159:0/1199484999 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1233717731 0 0) 0x7f67180ccb70 con 0x7f67180d2060 2024-04-04T06:08:58.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.257+0000 7f6716a00640 1 -- 172.21.15.159:0/1199484999 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66fc003520 con 0x7f67180d2060 2024-04-04T06:08:58.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.257+0000 7f6716a00640 1 -- 172.21.15.159:0/1199484999 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2040250469 0 0) 0x7f66fc003520 con 0x7f67180d2060 2024-04-04T06:08:58.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.257+0000 7f6716a00640 1 -- 172.21.15.159:0/1199484999 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67180cf710 msgr2=0x7f67180d24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:58.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.257+0000 7f6716a00640 1 --2- 172.21.15.159:0/1199484999 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67180cf710 0x7f67180d24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:58.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.257+0000 7f6716a00640 1 -- 172.21.15.159:0/1199484999 >> v2:172.21.15.178:3300/0 conn(0x7f671815c2c0 msgr2=0x7f67180d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:58.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.257+0000 7f6716a00640 1 --2- 172.21.15.159:0/1199484999 >> v2:172.21.15.178:3300/0 conn(0x7f671815c2c0 0x7f67180d06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:58.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.257+0000 7f6716a00640 1 -- 172.21.15.159:0/1199484999 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67180703f0 con 0x7f67180d2060 2024-04-04T06:08:58.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.257+0000 7f6716a00640 1 -- 172.21.15.159:0/1199484999 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f6704001690 con 0x7f67180d2060 2024-04-04T06:08:58.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.257+0000 7f6716a00640 1 -- 172.21.15.159:0/1199484999 >> v1:172.21.15.159:6789/0 conn(0x7f67180d2060 legacy=0x7f671815fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:58.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.257+0000 7f6716a00640 1 --2- 172.21.15.159:0/1199484999 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67180cf710 0x7f66fc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:58.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.257+0000 7f6716a00640 1 --2- 172.21.15.159:0/1199484999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc003f30 0x7f66fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:58.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.257+0000 7f6716a00640 1 --2- 172.21.15.159:0/1199484999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66fc004d40 0x7f66fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:58.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.257+0000 7f6716a00640 1 -- 172.21.15.159:0/1199484999 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6718070110 con 0x7f66fc003f30 2024-04-04T06:08:58.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.257+0000 7f6716a00640 1 -- 172.21.15.159:0/1199484999 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc00b9c0 con 0x7f66fc004d40 2024-04-04T06:08:58.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.257+0000 7f6716a00640 1 -- 172.21.15.159:0/1199484999 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f66fc00bb90 con 0x7f67180cf710 2024-04-04T06:08:58.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.257+0000 7f6717400640 1 --2- 172.21.15.159:0/1199484999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc003f30 0x7f66fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:58.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.258+0000 7f671ca00640 1 --2- 172.21.15.159:0/1199484999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66fc004d40 0x7f66fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:58.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.258+0000 7f6717400640 1 -- 172.21.15.159:0/1199484999 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67180cf710 msgr2=0x7f66fc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:58.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.258+0000 7f6717400640 1 --2- 172.21.15.159:0/1199484999 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67180cf710 0x7f66fc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:58.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.258+0000 7f6717400640 1 -- 172.21.15.159:0/1199484999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66fc004d40 msgr2=0x7f66fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:58.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.258+0000 7f6717400640 1 --2- 172.21.15.159:0/1199484999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66fc004d40 0x7f66fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:58.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.258+0000 7f6717400640 1 -- 172.21.15.159:0/1199484999 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66fc003520 con 0x7f66fc003f30 2024-04-04T06:08:58.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.258+0000 7f671ca00640 1 --2- 172.21.15.159:0/1199484999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66fc004d40 0x7f66fc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:58.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.259+0000 7f6717400640 1 --2- 172.21.15.159:0/1199484999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc003f30 0x7f66fc004800 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f670400c240 tx=0x7f66fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4cc304ebfc23113 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:58.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.259+0000 7f6716a00640 1 -- 172.21.15.159:0/1199484999 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704007d60 con 0x7f66fc003f30 2024-04-04T06:08:58.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.259+0000 7f6716a00640 1 -- 172.21.15.159:0/1199484999 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6704030070 con 0x7f66fc003f30 2024-04-04T06:08:58.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.259+0000 7f6716a00640 1 -- 172.21.15.159:0/1199484999 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704005d10 con 0x7f66fc003f30 2024-04-04T06:08:58.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.260+0000 7f671d400640 1 -- 172.21.15.159:0/1199484999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc003f30 msgr2=0x7f66fc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:58.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.260+0000 7f671d400640 1 --2- 172.21.15.159:0/1199484999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc003f30 0x7f66fc004800 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f670400c240 tx=0x7f66fc005830 comp rx=0 tx=0).stop 2024-04-04T06:08:58.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.260+0000 7f671ca00640 1 -- 172.21.15.159:0/1199484999 reap_dead start 2024-04-04T06:08:58.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.260+0000 7f671d400640 1 -- 172.21.15.159:0/1199484999 shutdown_connections 2024-04-04T06:08:58.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.260+0000 7f671d400640 1 -- 172.21.15.159:0/1199484999 >> 172.21.15.159:0/1199484999 conn(0x7f6718067280 msgr2=0x7f6718067660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:58.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.260+0000 7f671d400640 1 -- 172.21.15.159:0/1199484999 shutdown_connections 2024-04-04T06:08:58.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.260+0000 7f671d400640 1 -- 172.21.15.159:0/1199484999 wait complete. 2024-04-04T06:08:58.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.261+0000 7f671d400640 1 Processor -- start 2024-04-04T06:08:58.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.262+0000 7f671d400640 1 -- start start 2024-04-04T06:08:58.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.262+0000 7f671d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc003f30 0x7f671815ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:58.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.262+0000 7f671d400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6718152140 0x7f6718151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:58.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.263+0000 7f671d400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67180b7490 0x7f67180b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:58.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.263+0000 7f671d400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f67180d0bf0 con 0x7f66fc003f30 2024-04-04T06:08:58.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.263+0000 7f671d400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f67180cce50 con 0x7f6718152140 2024-04-04T06:08:58.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.263+0000 7f671d400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f67180cc720 con 0x7f67180b7490 2024-04-04T06:08:58.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.263+0000 7f6717e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc003f30 0x7f671815ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:58.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.263+0000 7f6717e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc003f30 0x7f671815ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37270/0 (socket says 172.21.15.159:37270) 2024-04-04T06:08:58.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.263+0000 7f6717e00640 1 -- 172.21.15.159:0/1869788553 learned_addr learned my addr 172.21.15.159:0/1869788553 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:58.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.263+0000 7f6717400640 1 --2- 172.21.15.159:0/1869788553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6718152140 0x7f6718151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:58.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.263+0000 7f6717e00640 1 -- 172.21.15.159:0/1869788553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67180b7490 msgr2=0x7f67180b7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:58.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.263+0000 7f6717e00640 1 --2- 172.21.15.159:0/1869788553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67180b7490 0x7f67180b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:58.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.263+0000 7f6717e00640 1 -- 172.21.15.159:0/1869788553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6718152140 msgr2=0x7f6718151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:58.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.263+0000 7f6717e00640 1 --2- 172.21.15.159:0/1869788553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6718152140 0x7f6718151ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:58.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.263+0000 7f6717e00640 1 -- 172.21.15.159:0/1869788553 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6718159b80 con 0x7f66fc003f30 2024-04-04T06:08:58.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.264+0000 7f6717e00640 1 --2- 172.21.15.159:0/1869788553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc003f30 0x7f671815ab40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f670800e860 tx=0x7f670800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:58.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.264+0000 7f670fe00640 1 -- 172.21.15.159:0/1869788553 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708017070 con 0x7f66fc003f30 2024-04-04T06:08:58.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.264+0000 7f670fe00640 1 -- 172.21.15.159:0/1869788553 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f670800c860 con 0x7f66fc003f30 2024-04-04T06:08:58.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.264+0000 7f670fe00640 1 -- 172.21.15.159:0/1869788553 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670800cc30 con 0x7f66fc003f30 2024-04-04T06:08:58.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.264+0000 7f671d400640 1 -- 172.21.15.159:0/1869788553 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6718159e60 con 0x7f66fc003f30 2024-04-04T06:08:58.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.264+0000 7f671d400640 1 -- 172.21.15.159:0/1869788553 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6718152890 con 0x7f66fc003f30 2024-04-04T06:08:58.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.265+0000 7f671d400640 1 -- 172.21.15.159:0/1869788553 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6718075940 con 0x7f66fc003f30 2024-04-04T06:08:58.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.266+0000 7f670fe00640 1 -- 172.21.15.159:0/1869788553 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f670801a070 con 0x7f66fc003f30 2024-04-04T06:08:58.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.267+0000 7f670fe00640 1 --2- 172.21.15.159:0/1869788553 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f66f806c8f0 0x7f66f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:58.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.268+0000 7f670fe00640 1 -- 172.21.15.159:0/1869788553 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f670808bb50 con 0x7f66fc003f30 2024-04-04T06:08:58.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.268+0000 7f6717400640 1 --2- 172.21.15.159:0/1869788553 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f66f806c8f0 0x7f66f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:58.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.269+0000 7f6717400640 1 --2- 172.21.15.159:0/1869788553 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f66f806c8f0 0x7f66f806edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6704034000 tx=0x7f6704007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:58.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.272+0000 7f670fe00640 1 -- 172.21.15.159:0/1869788553 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6708059ee0 con 0x7f66fc003f30 2024-04-04T06:08:58.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.458+0000 7f671d400640 1 -- 172.21.15.159:0/1869788553 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67180d06a0 con 0x7f66fc003f30 2024-04-04T06:08:58.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.460+0000 7f670fe00640 1 -- 172.21.15.159:0/1869788553 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16690 (secure 0 0 0) 0x7f67080597e0 con 0x7f66fc003f30 2024-04-04T06:08:58.454 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:08:58.454 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":58,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:08:57.019980+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:08:54.848337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-04T06:08:56.003356+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:08:58.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.462+0000 7f671d400640 1 -- 172.21.15.159:0/1869788553 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f66f806c8f0 msgr2=0x7f66f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:58.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.462+0000 7f671d400640 1 --2- 172.21.15.159:0/1869788553 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f66f806c8f0 0x7f66f806edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6704034000 tx=0x7f6704007000 comp rx=0 tx=0).stop 2024-04-04T06:08:58.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.462+0000 7f671d400640 1 -- 172.21.15.159:0/1869788553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc003f30 msgr2=0x7f671815ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:58.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.462+0000 7f671d400640 1 --2- 172.21.15.159:0/1869788553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc003f30 0x7f671815ab40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f670800e860 tx=0x7f670800ed20 comp rx=0 tx=0).stop 2024-04-04T06:08:58.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.462+0000 7f671d400640 1 -- 172.21.15.159:0/1869788553 shutdown_connections 2024-04-04T06:08:58.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.462+0000 7f671d400640 1 --2- 172.21.15.159:0/1869788553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67180b7490 0x7f67180b7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:58.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.462+0000 7f671d400640 1 --2- 172.21.15.159:0/1869788553 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f66f806c8f0 0x7f66f806edb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:58.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.462+0000 7f671d400640 1 --2- 172.21.15.159:0/1869788553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6718152140 0x7f6718151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:58.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.462+0000 7f671d400640 1 --2- 172.21.15.159:0/1869788553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66fc003f30 0x7f671815ab40 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:58.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.462+0000 7f671d400640 1 -- 172.21.15.159:0/1869788553 >> 172.21.15.159:0/1869788553 conn(0x7f6718067280 msgr2=0x7f67180ceae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:58.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.463+0000 7f671d400640 1 -- 172.21.15.159:0/1869788553 shutdown_connections 2024-04-04T06:08:58.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.463+0000 7f671d400640 1 -- 172.21.15.159:0/1869788553 wait complete. 2024-04-04T06:08:58.474 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-04T06:08:58.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.687+0000 7f2be7800640 1 Processor -- start 2024-04-04T06:08:58.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.688+0000 7f2be7800640 1 -- start start 2024-04-04T06:08:58.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.689+0000 7f2be7800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be01457e0 0x7f2be00b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:58.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.689+0000 7f2be7800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2be00b5ed0 con 0x7f2be00b4580 2024-04-04T06:08:58.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.689+0000 7f2be7800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2be00b61b0 con 0x7f2be006bc80 2024-04-04T06:08:58.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.689+0000 7f2be7800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2be0145bc0 con 0x7f2be01457e0 2024-04-04T06:08:58.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.690+0000 7f2bdfe00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f2be00b4580 0x7f2be006b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40292/0 (socket says 172.21.15.159:40292) 2024-04-04T06:08:58.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.690+0000 7f2bdfe00640 1 -- 172.21.15.159:0/3852705839 learned_addr learned my addr 172.21.15.159:0/3852705839 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:08:58.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.690+0000 7f2be4e00640 1 --2- 172.21.15.159:0/3852705839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be01457e0 0x7f2be00b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:58.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.690+0000 7f2bdf400640 1 -- 172.21.15.159:0/3852705839 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2746780543 0 0) 0x7f2be00b5ed0 con 0x7f2be00b4580 2024-04-04T06:08:58.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.690+0000 7f2bdf400640 1 -- 172.21.15.159:0/3852705839 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bc4003520 con 0x7f2be00b4580 2024-04-04T06:08:58.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.691+0000 7f2bdf400640 1 -- 172.21.15.159:0/3852705839 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1651266701 0 0) 0x7f2bc4003520 con 0x7f2be00b4580 2024-04-04T06:08:58.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.691+0000 7f2bdf400640 1 -- 172.21.15.159:0/3852705839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be01457e0 msgr2=0x7f2be00b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:58.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.691+0000 7f2bdf400640 1 --2- 172.21.15.159:0/3852705839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be01457e0 0x7f2be00b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:58.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.691+0000 7f2bdf400640 1 -- 172.21.15.159:0/3852705839 >> v1:172.21.15.178:6789/0 conn(0x7f2be006bc80 legacy=0x7f2be00b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:58.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.691+0000 7f2bdf400640 1 -- 172.21.15.159:0/3852705839 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be00af870 con 0x7f2be00b4580 2024-04-04T06:08:58.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.691+0000 7f2bdf400640 1 -- 172.21.15.159:0/3852705839 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f2bcc0019f0 con 0x7f2be00b4580 2024-04-04T06:08:58.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.691+0000 7f2bdf400640 1 -- 172.21.15.159:0/3852705839 >> v1:172.21.15.159:6789/0 conn(0x7f2be00b4580 legacy=0x7f2be006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:58.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.691+0000 7f2bdf400640 1 --2- 172.21.15.159:0/3852705839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be01457e0 0x7f2bc40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:58.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.691+0000 7f2bdf400640 1 --2- 172.21.15.159:0/3852705839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bc4003f60 0x7f2bc4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:58.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.691+0000 7f2bdf400640 1 --2- 172.21.15.159:0/3852705839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bc4004d40 0x7f2bc40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:58.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.691+0000 7f2bdf400640 1 -- 172.21.15.159:0/3852705839 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2be0145bc0 con 0x7f2bc4004d40 2024-04-04T06:08:58.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.691+0000 7f2bdf400640 1 -- 172.21.15.159:0/3852705839 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc400bb50 con 0x7f2bc4003f60 2024-04-04T06:08:58.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.691+0000 7f2bdf400640 1 -- 172.21.15.159:0/3852705839 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2bc400bd50 con 0x7f2be01457e0 2024-04-04T06:08:58.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.691+0000 7f2be5800640 1 --2- 172.21.15.159:0/3852705839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bc4004d40 0x7f2bc40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:58.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.692+0000 7f2bdfe00640 1 --2- 172.21.15.159:0/3852705839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bc4003f60 0x7f2bc4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:58.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.692+0000 7f2be4e00640 1 --2- 172.21.15.159:0/3852705839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be01457e0 0x7f2bc40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:58.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.692+0000 7f2be5800640 1 -- 172.21.15.159:0/3852705839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be01457e0 msgr2=0x7f2bc40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:58.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.692+0000 7f2be5800640 1 --2- 172.21.15.159:0/3852705839 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be01457e0 0x7f2bc40039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:58.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.692+0000 7f2be5800640 1 -- 172.21.15.159:0/3852705839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bc4003f60 msgr2=0x7f2bc4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:58.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.692+0000 7f2be5800640 1 --2- 172.21.15.159:0/3852705839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bc4003f60 0x7f2bc4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:58.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.692+0000 7f2be5800640 1 -- 172.21.15.159:0/3852705839 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc4003520 con 0x7f2bc4004d40 2024-04-04T06:08:58.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.693+0000 7f2be5800640 1 --2- 172.21.15.159:0/3852705839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bc4004d40 0x7f2bc40051c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8001720 tx=0x7f2bc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aedc6709548a3f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:58.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.693+0000 7f2bdf400640 1 -- 172.21.15.159:0/3852705839 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8001370 con 0x7f2bc4004d40 2024-04-04T06:08:58.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.693+0000 7f2bdf400640 1 -- 172.21.15.159:0/3852705839 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd80052d0 con 0x7f2bc4004d40 2024-04-04T06:08:58.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.693+0000 7f2bdf400640 1 -- 172.21.15.159:0/3852705839 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8003830 con 0x7f2bc4004d40 2024-04-04T06:08:58.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.694+0000 7f2be7800640 1 -- 172.21.15.159:0/3852705839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bc4004d40 msgr2=0x7f2bc40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:58.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.694+0000 7f2be7800640 1 --2- 172.21.15.159:0/3852705839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bc4004d40 0x7f2bc40051c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8001720 tx=0x7f2bc4005830 comp rx=0 tx=0).stop 2024-04-04T06:08:58.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.694+0000 7f2be5800640 1 -- 172.21.15.159:0/3852705839 reap_dead start 2024-04-04T06:08:58.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.694+0000 7f2be7800640 1 -- 172.21.15.159:0/3852705839 shutdown_connections 2024-04-04T06:08:58.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.694+0000 7f2be7800640 1 -- 172.21.15.159:0/3852705839 >> 172.21.15.159:0/3852705839 conn(0x7f2be00b65a0 msgr2=0x7f2be00b69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:58.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.694+0000 7f2be7800640 1 -- 172.21.15.159:0/3852705839 shutdown_connections 2024-04-04T06:08:58.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.694+0000 7f2be7800640 1 -- 172.21.15.159:0/3852705839 wait complete. 2024-04-04T06:08:58.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.697+0000 7f2be7800640 1 Processor -- start 2024-04-04T06:08:58.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.697+0000 7f2be7800640 1 -- start start 2024-04-04T06:08:58.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.698+0000 7f2be7800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bc4004d40 0x7f2be015bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:58.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.698+0000 7f2be7800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2be006f940 0x7f2be00b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:58.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.698+0000 7f2be7800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be006fdc0 0x7f2be00b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:58.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.698+0000 7f2be7800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2be00b0840 con 0x7f2be006f940 2024-04-04T06:08:58.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.698+0000 7f2be7800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2be00afca0 con 0x7f2bc4004d40 2024-04-04T06:08:58.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.698+0000 7f2be7800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2be00b09c0 con 0x7f2be006fdc0 2024-04-04T06:08:58.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.699+0000 7f2bdfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2be006f940 0x7f2be00b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:58.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.699+0000 7f2bdfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2be006f940 0x7f2be00b1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37288/0 (socket says 172.21.15.159:37288) 2024-04-04T06:08:58.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.699+0000 7f2bdfe00640 1 -- 172.21.15.159:0/3721625551 learned_addr learned my addr 172.21.15.159:0/3721625551 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:58.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.699+0000 7f2be5800640 1 --2- 172.21.15.159:0/3721625551 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be006fdc0 0x7f2be00b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:58.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.699+0000 7f2bdfe00640 1 -- 172.21.15.159:0/3721625551 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be006fdc0 msgr2=0x7f2be00b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:58.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.699+0000 7f2bdfe00640 1 --2- 172.21.15.159:0/3721625551 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be006fdc0 0x7f2be00b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:58.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.699+0000 7f2bdfe00640 1 -- 172.21.15.159:0/3721625551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bc4004d40 msgr2=0x7f2be015bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:58.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.699+0000 7f2be4e00640 1 --2- 172.21.15.159:0/3721625551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bc4004d40 0x7f2be015bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:58.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.699+0000 7f2bdfe00640 1 --2- 172.21.15.159:0/3721625551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bc4004d40 0x7f2be015bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:58.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.699+0000 7f2bdfe00640 1 -- 172.21.15.159:0/3721625551 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be006e8a0 con 0x7f2be006f940 2024-04-04T06:08:58.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.699+0000 7f2be5800640 1 --2- 172.21.15.159:0/3721625551 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be006fdc0 0x7f2be00b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:58.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.700+0000 7f2bdfe00640 1 --2- 172.21.15.159:0/3721625551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2be006f940 0x7f2be00b1fd0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc001580 tx=0x7f2bcc00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:58.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.700+0000 7f2bdcc00640 1 -- 172.21.15.159:0/3721625551 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc013420 con 0x7f2be006f940 2024-04-04T06:08:58.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.700+0000 7f2bdcc00640 1 -- 172.21.15.159:0/3721625551 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bcc018070 con 0x7f2be006f940 2024-04-04T06:08:58.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.700+0000 7f2bdcc00640 1 -- 172.21.15.159:0/3721625551 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc013820 con 0x7f2be006f940 2024-04-04T06:08:58.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.700+0000 7f2be7800640 1 -- 172.21.15.159:0/3721625551 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be006eb80 con 0x7f2be006f940 2024-04-04T06:08:58.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.700+0000 7f2be7800640 1 -- 172.21.15.159:0/3721625551 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be006f140 con 0x7f2be006f940 2024-04-04T06:08:58.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.702+0000 7f2be7800640 1 -- 172.21.15.159:0/3721625551 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be0068cd0 con 0x7f2be006f940 2024-04-04T06:08:58.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.702+0000 7f2bdcc00640 1 -- 172.21.15.159:0/3721625551 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2bcc0139c0 con 0x7f2be006f940 2024-04-04T06:08:58.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.703+0000 7f2bdcc00640 1 --2- 172.21.15.159:0/3721625551 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2bc006c920 0x7f2bc006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:58.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.703+0000 7f2bdcc00640 1 -- 172.21.15.159:0/3721625551 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f2bcc08c220 con 0x7f2be006f940 2024-04-04T06:08:58.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.704+0000 7f2be4e00640 1 --2- 172.21.15.159:0/3721625551 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2bc006c920 0x7f2bc006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:58.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.705+0000 7f2be4e00640 1 --2- 172.21.15.159:0/3721625551 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2bc006c920 0x7f2bc006ede0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2be01f79b0 tx=0x7f2bd0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:58.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.707+0000 7f2bdcc00640 1 -- 172.21.15.159:0/3721625551 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2bcc05a6e0 con 0x7f2be006f940 2024-04-04T06:08:58.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:58.979+0000 7f2be7800640 1 -- 172.21.15.159:0/3721625551 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2be00d0f40 con 0x7f2be006f940 2024-04-04T06:08:59.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.605+0000 7f2bdcc00640 1 -- 172.21.15.159:0/3721625551 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) ==== 118+0+0 (secure 0 0 0) 0x7f2bcc059fe0 con 0x7f2be006f940 2024-04-04T06:08:59.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.609+0000 7f2be7800640 1 -- 172.21.15.159:0/3721625551 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2bc006c920 msgr2=0x7f2bc006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:59.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.609+0000 7f2be7800640 1 --2- 172.21.15.159:0/3721625551 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2bc006c920 0x7f2bc006ede0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2be01f79b0 tx=0x7f2bd0008040 comp rx=0 tx=0).stop 2024-04-04T06:08:59.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.609+0000 7f2be7800640 1 -- 172.21.15.159:0/3721625551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2be006f940 msgr2=0x7f2be00b1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:59.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.609+0000 7f2be7800640 1 --2- 172.21.15.159:0/3721625551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2be006f940 0x7f2be00b1fd0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc001580 tx=0x7f2bcc00e530 comp rx=0 tx=0).stop 2024-04-04T06:08:59.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.609+0000 7f2be7800640 1 -- 172.21.15.159:0/3721625551 shutdown_connections 2024-04-04T06:08:59.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.609+0000 7f2be7800640 1 --2- 172.21.15.159:0/3721625551 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2be006fdc0 0x7f2be00b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:59.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.609+0000 7f2be7800640 1 --2- 172.21.15.159:0/3721625551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2be006f940 0x7f2be00b1fd0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:59.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.609+0000 7f2be7800640 1 --2- 172.21.15.159:0/3721625551 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2bc006c920 0x7f2bc006ede0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:59.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.609+0000 7f2be7800640 1 --2- 172.21.15.159:0/3721625551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bc4004d40 0x7f2be015bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:59.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.609+0000 7f2be7800640 1 -- 172.21.15.159:0/3721625551 >> 172.21.15.159:0/3721625551 conn(0x7f2be00b65a0 msgr2=0x7f2be01601d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:59.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.610+0000 7f2be7800640 1 -- 172.21.15.159:0/3721625551 shutdown_connections 2024-04-04T06:08:59.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.610+0000 7f2be7800640 1 -- 172.21.15.159:0/3721625551 wait complete. 2024-04-04T06:08:59.634 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-04T06:08:59.635 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-04T06:08:59.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.857+0000 7f4de9400640 1 Processor -- start 2024-04-04T06:08:59.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.857+0000 7f4de9400640 1 -- start start 2024-04-04T06:08:59.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.859+0000 7f4de9400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4de415e4c0 0x7f4de40b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:59.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.859+0000 7f4de9400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4de40d37b0 0x7f4de40b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:59.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.859+0000 7f4de9400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4de40b67f0 0x7f4de40d4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:59.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.860+0000 7f4de9400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f4de415e8a0 con 0x7f4de40d37b0 2024-04-04T06:08:59.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.860+0000 7f4de9400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4de40d4040 con 0x7f4de40b67f0 2024-04-04T06:08:59.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.860+0000 7f4de9400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4de40d3320 con 0x7f4de415e4c0 2024-04-04T06:08:59.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.860+0000 7f4de3400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4de40d37b0 0x7f4de40b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:59.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.860+0000 7f4de3400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4de40d37b0 0x7f4de40b62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37294/0 (socket says 172.21.15.159:37294) 2024-04-04T06:08:59.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.860+0000 7f4de3400640 1 -- 172.21.15.159:0/2764269225 learned_addr learned my addr 172.21.15.159:0/2764269225 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:59.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.860+0000 7f4de8a00640 1 --2- 172.21.15.159:0/2764269225 >> v2:172.21.15.178:3300/0 conn(0x7f4de40b67f0 0x7f4de40d4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:59.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.860+0000 7f4de3e00640 1 --2- 172.21.15.159:0/2764269225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4de415e4c0 0x7f4de40b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:59.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.860+0000 7f4de3400640 1 -- 172.21.15.159:0/2764269225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4de415e4c0 msgr2=0x7f4de40b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:59.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.860+0000 7f4de3400640 1 --2- 172.21.15.159:0/2764269225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4de415e4c0 0x7f4de40b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:59.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.860+0000 7f4de3400640 1 -- 172.21.15.159:0/2764269225 >> v2:172.21.15.178:3300/0 conn(0x7f4de40b67f0 msgr2=0x7f4de40d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:08:59.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.860+0000 7f4de3400640 1 --2- 172.21.15.159:0/2764269225 >> v2:172.21.15.178:3300/0 conn(0x7f4de40b67f0 0x7f4de40d4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:59.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.860+0000 7f4de3400640 1 -- 172.21.15.159:0/2764269225 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de40d2710 con 0x7f4de40d37b0 2024-04-04T06:08:59.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.860+0000 7f4de3e00640 1 --2- 172.21.15.159:0/2764269225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4de415e4c0 0x7f4de40b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:59.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.861+0000 7f4de3400640 1 --2- 172.21.15.159:0/2764269225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4de40d37b0 0x7f4de40b62b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4de40d1710 tx=0x7f4dcc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d24eab710c98828 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:59.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.862+0000 7f4de2a00640 1 -- 172.21.15.159:0/2764269225 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc035020 con 0x7f4de40d37b0 2024-04-04T06:08:59.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.862+0000 7f4de2a00640 1 -- 172.21.15.159:0/2764269225 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dcc031870 con 0x7f4de40d37b0 2024-04-04T06:08:59.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.862+0000 7f4de2a00640 1 -- 172.21.15.159:0/2764269225 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc031c40 con 0x7f4de40d37b0 2024-04-04T06:08:59.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.863+0000 7f4de9400640 1 -- 172.21.15.159:0/2764269225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4de40d37b0 msgr2=0x7f4de40b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:59.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.863+0000 7f4de9400640 1 --2- 172.21.15.159:0/2764269225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4de40d37b0 0x7f4de40b62b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4de40d1710 tx=0x7f4dcc02b910 comp rx=0 tx=0).stop 2024-04-04T06:08:59.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.863+0000 7f4de9400640 1 -- 172.21.15.159:0/2764269225 shutdown_connections 2024-04-04T06:08:59.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.863+0000 7f4de9400640 1 --2- 172.21.15.159:0/2764269225 >> v2:172.21.15.178:3300/0 conn(0x7f4de40b67f0 0x7f4de40d4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:59.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.863+0000 7f4de9400640 1 --2- 172.21.15.159:0/2764269225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4de40d37b0 0x7f4de40b62b0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:59.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.863+0000 7f4de9400640 1 --2- 172.21.15.159:0/2764269225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4de415e4c0 0x7f4de40b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:59.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.863+0000 7f4de9400640 1 -- 172.21.15.159:0/2764269225 >> 172.21.15.159:0/2764269225 conn(0x7f4de40676a0 msgr2=0x7f4de4067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:08:59.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.863+0000 7f4de9400640 1 -- 172.21.15.159:0/2764269225 shutdown_connections 2024-04-04T06:08:59.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.863+0000 7f4de9400640 1 -- 172.21.15.159:0/2764269225 wait complete. 2024-04-04T06:08:59.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.866+0000 7f4de9400640 1 Processor -- start 2024-04-04T06:08:59.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.866+0000 7f4de9400640 1 -- start start 2024-04-04T06:08:59.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.867+0000 7f4de9400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4de40b67f0 0x7f4de4154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:59.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.867+0000 7f4de9400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4de40d37b0 0x7f4de4160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:59.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.867+0000 7f4de9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4de415e4c0 0x7f4de415bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:59.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.867+0000 7f4de9400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4de40d2fb0 con 0x7f4de415e4c0 2024-04-04T06:08:59.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.867+0000 7f4de9400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4de40d2cd0 con 0x7f4de40b67f0 2024-04-04T06:08:59.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.867+0000 7f4de9400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4de40ba130 con 0x7f4de40d37b0 2024-04-04T06:08:59.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.868+0000 7f4de8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4de415e4c0 0x7f4de415bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:59.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.868+0000 7f4de8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4de415e4c0 0x7f4de415bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37300/0 (socket says 172.21.15.159:37300) 2024-04-04T06:08:59.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.868+0000 7f4de8a00640 1 -- 172.21.15.159:0/2267442011 learned_addr learned my addr 172.21.15.159:0/2267442011 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:08:59.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.868+0000 7f4de3e00640 1 --2- 172.21.15.159:0/2267442011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4de40b67f0 0x7f4de4154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:59.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.868+0000 7f4de3400640 1 --2- 172.21.15.159:0/2267442011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4de40d37b0 0x7f4de4160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:59.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.868+0000 7f4de8a00640 1 -- 172.21.15.159:0/2267442011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4de40d37b0 msgr2=0x7f4de4160810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:59.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.868+0000 7f4de8a00640 1 --2- 172.21.15.159:0/2267442011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4de40d37b0 0x7f4de4160810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:59.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.868+0000 7f4de8a00640 1 -- 172.21.15.159:0/2267442011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4de40b67f0 msgr2=0x7f4de4154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:08:59.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.868+0000 7f4de8a00640 1 --2- 172.21.15.159:0/2267442011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4de40b67f0 0x7f4de4154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:08:59.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.868+0000 7f4de8a00640 1 -- 172.21.15.159:0/2267442011 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de4154d90 con 0x7f4de415e4c0 2024-04-04T06:08:59.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.868+0000 7f4de3400640 1 --2- 172.21.15.159:0/2267442011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4de40d37b0 0x7f4de4160810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:08:59.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.868+0000 7f4de8a00640 1 --2- 172.21.15.159:0/2267442011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4de415e4c0 0x7f4de415bf50 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc00d830 tx=0x7f4ddc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:59.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.869+0000 7f4ddbe00640 1 -- 172.21.15.159:0/2267442011 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc00b6c0 con 0x7f4de415e4c0 2024-04-04T06:08:59.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.869+0000 7f4ddbe00640 1 -- 172.21.15.159:0/2267442011 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ddc019070 con 0x7f4de415e4c0 2024-04-04T06:08:59.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.869+0000 7f4ddbe00640 1 -- 172.21.15.159:0/2267442011 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc00bac0 con 0x7f4de415e4c0 2024-04-04T06:08:59.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.869+0000 7f4de9400640 1 -- 172.21.15.159:0/2267442011 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de4157270 con 0x7f4de415e4c0 2024-04-04T06:08:59.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.869+0000 7f4de9400640 1 -- 172.21.15.159:0/2267442011 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de4157830 con 0x7f4de415e4c0 2024-04-04T06:08:59.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.870+0000 7f4de9400640 1 -- 172.21.15.159:0/2267442011 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4de4075940 con 0x7f4de415e4c0 2024-04-04T06:08:59.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.871+0000 7f4ddbe00640 1 -- 172.21.15.159:0/2267442011 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ddc01e460 con 0x7f4de415e4c0 2024-04-04T06:08:59.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.873+0000 7f4ddbe00640 1 --2- 172.21.15.159:0/2267442011 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4dc006c8f0 0x7f4dc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:08:59.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.873+0000 7f4ddbe00640 1 -- 172.21.15.159:0/2267442011 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f4ddc08c240 con 0x7f4de415e4c0 2024-04-04T06:08:59.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.873+0000 7f4de3e00640 1 --2- 172.21.15.159:0/2267442011 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4dc006c8f0 0x7f4dc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:08:59.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.874+0000 7f4de3e00640 1 --2- 172.21.15.159:0/2267442011 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4dc006c8f0 0x7f4dc006edb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4007130 tx=0x7f4dd4007060 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:08:59.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:08:59.877+0000 7f4ddbe00640 1 -- 172.21.15.159:0/2267442011 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4ddc05a660 con 0x7f4de415e4c0 2024-04-04T06:09:00.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.115+0000 7f4de9400640 1 -- 172.21.15.159:0/2267442011 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4de406b570 con 0x7f4de415e4c0 2024-04-04T06:09:00.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.608+0000 7f4ddbe00640 1 -- 172.21.15.159:0/2267442011 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) ==== 122+0+0 (secure 0 0 0) 0x7f4ddc059f60 con 0x7f4de415e4c0 2024-04-04T06:09:00.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.612+0000 7f4de9400640 1 -- 172.21.15.159:0/2267442011 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4dc006c8f0 msgr2=0x7f4dc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:00.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.612+0000 7f4de9400640 1 --2- 172.21.15.159:0/2267442011 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4dc006c8f0 0x7f4dc006edb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4007130 tx=0x7f4dd4007060 comp rx=0 tx=0).stop 2024-04-04T06:09:00.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.612+0000 7f4de9400640 1 -- 172.21.15.159:0/2267442011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4de415e4c0 msgr2=0x7f4de415bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:00.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.612+0000 7f4de9400640 1 --2- 172.21.15.159:0/2267442011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4de415e4c0 0x7f4de415bf50 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc00d830 tx=0x7f4ddc00dcf0 comp rx=0 tx=0).stop 2024-04-04T06:09:00.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.612+0000 7f4de9400640 1 -- 172.21.15.159:0/2267442011 shutdown_connections 2024-04-04T06:09:00.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.612+0000 7f4de9400640 1 --2- 172.21.15.159:0/2267442011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4de415e4c0 0x7f4de415bf50 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:00.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.612+0000 7f4de9400640 1 --2- 172.21.15.159:0/2267442011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4de40d37b0 0x7f4de4160810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:00.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.612+0000 7f4de9400640 1 --2- 172.21.15.159:0/2267442011 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4dc006c8f0 0x7f4dc006edb0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:00.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.612+0000 7f4de9400640 1 --2- 172.21.15.159:0/2267442011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4de40b67f0 0x7f4de4154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:00.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.612+0000 7f4de9400640 1 -- 172.21.15.159:0/2267442011 >> 172.21.15.159:0/2267442011 conn(0x7f4de40676a0 msgr2=0x7f4de415cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:00.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.612+0000 7f4de9400640 1 -- 172.21.15.159:0/2267442011 shutdown_connections 2024-04-04T06:09:00.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.612+0000 7f4de9400640 1 -- 172.21.15.159:0/2267442011 wait complete. 2024-04-04T06:09:00.629 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-04T06:09:00.629 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:09:00.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.854+0000 7ff5a7a00640 1 Processor -- start 2024-04-04T06:09:00.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.855+0000 7ff5a7a00640 1 -- start start 2024-04-04T06:09:00.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.856+0000 7ff5a7a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5a015da60 0x7ff5a00cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:00.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.856+0000 7ff5a7a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff5a015fe90 0x7ff5a00d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:00.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.856+0000 7ff5a7a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5a015d060 con 0x7ff5a015f900 2024-04-04T06:09:00.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.856+0000 7ff5a7a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff5a015d340 con 0x7ff5a015fe90 2024-04-04T06:09:00.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.856+0000 7ff5a7a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a015d620 con 0x7ff5a015da60 2024-04-04T06:09:00.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.856+0000 7ff5a5000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7ff5a015f900 0x7ff5a015c850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40294/0 (socket says 172.21.15.159:40294) 2024-04-04T06:09:00.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.856+0000 7ff5a5000640 1 -- 172.21.15.159:0/1153981923 learned_addr learned my addr 172.21.15.159:0/1153981923 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:09:00.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.856+0000 7ff597e00640 1 --2- 172.21.15.159:0/1153981923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5a015da60 0x7ff5a00cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:00.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.856+0000 7ff5a5a00640 1 --2- 172.21.15.159:0/1153981923 >> v2:172.21.15.178:3300/0 conn(0x7ff5a015fe90 0x7ff5a00d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:00.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.857+0000 7ff59fe00640 1 -- 172.21.15.159:0/1153981923 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1354809196 0 0) 0x7ff5a015d060 con 0x7ff5a015f900 2024-04-04T06:09:00.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.857+0000 7ff59fe00640 1 -- 172.21.15.159:0/1153981923 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff580003520 con 0x7ff5a015f900 2024-04-04T06:09:00.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.857+0000 7ff59fe00640 1 -- 172.21.15.159:0/1153981923 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2545621486 0 0) 0x7ff580003520 con 0x7ff5a015f900 2024-04-04T06:09:00.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.857+0000 7ff59fe00640 1 -- 172.21.15.159:0/1153981923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5a015da60 msgr2=0x7ff5a00cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:00.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.857+0000 7ff59fe00640 1 --2- 172.21.15.159:0/1153981923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5a015da60 0x7ff5a00cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:00.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.857+0000 7ff59fe00640 1 -- 172.21.15.159:0/1153981923 >> v2:172.21.15.178:3300/0 conn(0x7ff5a015fe90 msgr2=0x7ff5a00d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:00.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.857+0000 7ff59fe00640 1 --2- 172.21.15.159:0/1153981923 >> v2:172.21.15.178:3300/0 conn(0x7ff5a015fe90 0x7ff5a00d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:00.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.857+0000 7ff59fe00640 1 -- 172.21.15.159:0/1153981923 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a00c74b0 con 0x7ff5a015f900 2024-04-04T06:09:00.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.857+0000 7ff59fe00640 1 -- 172.21.15.159:0/1153981923 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7ff590001670 con 0x7ff5a015f900 2024-04-04T06:09:00.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.857+0000 7ff59fe00640 1 -- 172.21.15.159:0/1153981923 >> v1:172.21.15.159:6789/0 conn(0x7ff5a015f900 legacy=0x7ff5a015c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:00.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.857+0000 7ff59fe00640 1 --2- 172.21.15.159:0/1153981923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5800039b0 0x7ff580003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:00.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.857+0000 7ff59fe00640 1 --2- 172.21.15.159:0/1153981923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff580004800 0x7ff580004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:00.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.857+0000 7ff59fe00640 1 --2- 172.21.15.159:0/1153981923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5a015da60 0x7ff5800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:00.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.857+0000 7ff59fe00640 1 -- 172.21.15.159:0/1153981923 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a015d340 con 0x7ff5800039b0 2024-04-04T06:09:00.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.857+0000 7ff59fe00640 1 -- 172.21.15.159:0/1153981923 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a015d620 con 0x7ff580004800 2024-04-04T06:09:00.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.857+0000 7ff59fe00640 1 -- 172.21.15.159:0/1153981923 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff58000b9c0 con 0x7ff5a015da60 2024-04-04T06:09:00.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.857+0000 7ff5a5000640 1 --2- 172.21.15.159:0/1153981923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5800039b0 0x7ff580003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:00.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.858+0000 7ff597e00640 1 --2- 172.21.15.159:0/1153981923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff580004800 0x7ff580004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:00.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.858+0000 7ff5a5a00640 1 --2- 172.21.15.159:0/1153981923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5a015da60 0x7ff5800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:00.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.858+0000 7ff5a5000640 1 -- 172.21.15.159:0/1153981923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5a015da60 msgr2=0x7ff5800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:00.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.858+0000 7ff5a5000640 1 --2- 172.21.15.159:0/1153981923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5a015da60 0x7ff5800051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:00.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.858+0000 7ff5a5000640 1 -- 172.21.15.159:0/1153981923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff580004800 msgr2=0x7ff580004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:00.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.858+0000 7ff5a5000640 1 --2- 172.21.15.159:0/1153981923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff580004800 0x7ff580004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:00.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.858+0000 7ff5a5000640 1 -- 172.21.15.159:0/1153981923 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff580003520 con 0x7ff5800039b0 2024-04-04T06:09:00.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.859+0000 7ff5a5000640 1 --2- 172.21.15.159:0/1153981923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5800039b0 0x7ff580003dd0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff590001050 tx=0x7ff580005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab48af31b4e35b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:00.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.859+0000 7ff59fe00640 1 -- 172.21.15.159:0/1153981923 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff590007d70 con 0x7ff5800039b0 2024-04-04T06:09:00.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.859+0000 7ff59fe00640 1 -- 172.21.15.159:0/1153981923 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff590030070 con 0x7ff5800039b0 2024-04-04T06:09:00.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.859+0000 7ff59fe00640 1 -- 172.21.15.159:0/1153981923 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59002cd80 con 0x7ff5800039b0 2024-04-04T06:09:00.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.860+0000 7ff5a7a00640 1 -- 172.21.15.159:0/1153981923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5800039b0 msgr2=0x7ff580003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:00.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.860+0000 7ff5a7a00640 1 --2- 172.21.15.159:0/1153981923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5800039b0 0x7ff580003dd0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff590001050 tx=0x7ff580005830 comp rx=0 tx=0).stop 2024-04-04T06:09:00.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.860+0000 7ff5a5a00640 1 -- 172.21.15.159:0/1153981923 reap_dead start 2024-04-04T06:09:00.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.860+0000 7ff5a7a00640 1 -- 172.21.15.159:0/1153981923 shutdown_connections 2024-04-04T06:09:00.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.860+0000 7ff5a7a00640 1 -- 172.21.15.159:0/1153981923 >> 172.21.15.159:0/1153981923 conn(0x7ff5a00b3e90 msgr2=0x7ff5a00b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:00.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.860+0000 7ff5a7a00640 1 -- 172.21.15.159:0/1153981923 shutdown_connections 2024-04-04T06:09:00.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.860+0000 7ff5a7a00640 1 -- 172.21.15.159:0/1153981923 wait complete. 2024-04-04T06:09:00.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.862+0000 7ff5a7a00640 1 Processor -- start 2024-04-04T06:09:00.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.862+0000 7ff5a7a00640 1 -- start start 2024-04-04T06:09:00.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.863+0000 7ff5a7a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5800039b0 0x7ff5a00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:00.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.863+0000 7ff5a7a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5a015be20 0x7ff5a01567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:00.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.863+0000 7ff5a7a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5a00cec60 0x7ff5a00cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:00.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.863+0000 7ff5a7a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a00d01c0 con 0x7ff5a00cec60 2024-04-04T06:09:00.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.863+0000 7ff5a7a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a00d0040 con 0x7ff5800039b0 2024-04-04T06:09:00.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.863+0000 7ff5a7a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a00ce8d0 con 0x7ff5a015be20 2024-04-04T06:09:00.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.863+0000 7ff5a5a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5a00cec60 0x7ff5a00cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:00.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.863+0000 7ff5a5a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5a00cec60 0x7ff5a00cf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37324/0 (socket says 172.21.15.159:37324) 2024-04-04T06:09:00.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.863+0000 7ff5a5a00640 1 -- 172.21.15.159:0/2008350612 learned_addr learned my addr 172.21.15.159:0/2008350612 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:00.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.864+0000 7ff5a5000640 1 --2- 172.21.15.159:0/2008350612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5800039b0 0x7ff5a00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:00.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.864+0000 7ff597e00640 1 --2- 172.21.15.159:0/2008350612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5a015be20 0x7ff5a01567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:00.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.864+0000 7ff5a5a00640 1 -- 172.21.15.159:0/2008350612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5a015be20 msgr2=0x7ff5a01567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:00.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.864+0000 7ff5a5a00640 1 --2- 172.21.15.159:0/2008350612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5a015be20 0x7ff5a01567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:00.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.864+0000 7ff5a5a00640 1 -- 172.21.15.159:0/2008350612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5800039b0 msgr2=0x7ff5a00b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:00.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.864+0000 7ff5a5a00640 1 --2- 172.21.15.159:0/2008350612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5800039b0 0x7ff5a00b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:00.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.864+0000 7ff5a5a00640 1 -- 172.21.15.159:0/2008350612 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a0156d10 con 0x7ff5a00cec60 2024-04-04T06:09:00.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.864+0000 7ff5a5000640 1 --2- 172.21.15.159:0/2008350612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5800039b0 0x7ff5a00b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:00.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.864+0000 7ff5a5a00640 1 --2- 172.21.15.159:0/2008350612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5a00cec60 0x7ff5a00cf0e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00b1d90 tx=0x7ff59800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:00.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.864+0000 7ff59d600640 1 -- 172.21.15.159:0/2008350612 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598016620 con 0x7ff5a00cec60 2024-04-04T06:09:00.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.864+0000 7ff59d600640 1 -- 172.21.15.159:0/2008350612 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff59801b070 con 0x7ff5a00cec60 2024-04-04T06:09:00.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.864+0000 7ff59d600640 1 -- 172.21.15.159:0/2008350612 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598016a20 con 0x7ff5a00cec60 2024-04-04T06:09:00.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.864+0000 7ff5a7a00640 1 -- 172.21.15.159:0/2008350612 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a00b5570 con 0x7ff5a00cec60 2024-04-04T06:09:00.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.864+0000 7ff5a7a00640 1 -- 172.21.15.159:0/2008350612 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a00b5b30 con 0x7ff5a00cec60 2024-04-04T06:09:00.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.866+0000 7ff5a7a00640 1 -- 172.21.15.159:0/2008350612 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a00c71f0 con 0x7ff5a00cec60 2024-04-04T06:09:00.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.866+0000 7ff59d600640 1 -- 172.21.15.159:0/2008350612 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff598005d10 con 0x7ff5a00cec60 2024-04-04T06:09:00.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.868+0000 7ff59d600640 1 --2- 172.21.15.159:0/2008350612 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff57c06c920 0x7ff57c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:00.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.868+0000 7ff59d600640 1 -- 172.21.15.159:0/2008350612 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7ff59808c440 con 0x7ff5a00cec60 2024-04-04T06:09:00.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.868+0000 7ff5a5000640 1 --2- 172.21.15.159:0/2008350612 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff57c06c920 0x7ff57c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:00.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.869+0000 7ff5a5000640 1 --2- 172.21.15.159:0/2008350612 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff57c06c920 0x7ff57c06ede0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff590034000 tx=0x7ff590007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:00.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:00.873+0000 7ff59d600640 1 -- 172.21.15.159:0/2008350612 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff5a00c71f0 con 0x7ff5a00cec60 2024-04-04T06:09:01.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.118+0000 7ff5a7a00640 1 -- 172.21.15.159:0/2008350612 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5a0068cd0 con 0x7ff5a00cec60 2024-04-04T06:09:01.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.119+0000 7ff59d600640 1 -- 172.21.15.159:0/2008350612 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5187 (secure 0 0 0) 0x7ff59805a860 con 0x7ff5a00cec60 2024-04-04T06:09:01.113 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:09:01.113 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5191,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3507245876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3507245876},{"type":"v1","addr":"172.21.15.159:6835","nonce":3507245876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/967262871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":967262871},{"type":"v1","addr":"172.21.15.178:6835","nonce":967262871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2969351496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2969351496},{"type":"v1","addr":"172.21.15.178:6837","nonce":2969351496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:08:57.020444+0000","modified":"2024-04-04T06:09:00.117714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6039":{"gid":6039,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6834/962475038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":962475038},{"type":"v1","addr":"172.21.15.159:6834","nonce":962475038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6039,"qdb_cluster":[6039]},"id":4}]} 2024-04-04T06:09:01.114 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 33 2024-04-04T06:09:01.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.122+0000 7ff5a7a00640 1 -- 172.21.15.159:0/2008350612 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff57c06c920 msgr2=0x7ff57c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:01.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.122+0000 7ff5a7a00640 1 --2- 172.21.15.159:0/2008350612 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff57c06c920 0x7ff57c06ede0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff590034000 tx=0x7ff590007000 comp rx=0 tx=0).stop 2024-04-04T06:09:01.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.122+0000 7ff5a7a00640 1 -- 172.21.15.159:0/2008350612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5a00cec60 msgr2=0x7ff5a00cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:01.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.122+0000 7ff5a7a00640 1 --2- 172.21.15.159:0/2008350612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5a00cec60 0x7ff5a00cf0e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00b1d90 tx=0x7ff59800f520 comp rx=0 tx=0).stop 2024-04-04T06:09:01.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.122+0000 7ff5a7a00640 1 -- 172.21.15.159:0/2008350612 shutdown_connections 2024-04-04T06:09:01.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.122+0000 7ff5a7a00640 1 --2- 172.21.15.159:0/2008350612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5a00cec60 0x7ff5a00cf0e0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:01.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.122+0000 7ff5a7a00640 1 --2- 172.21.15.159:0/2008350612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5a015be20 0x7ff5a01567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:01.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.122+0000 7ff5a7a00640 1 --2- 172.21.15.159:0/2008350612 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff57c06c920 0x7ff57c06ede0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:01.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.122+0000 7ff5a7a00640 1 --2- 172.21.15.159:0/2008350612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5800039b0 0x7ff5a00b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:01.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.122+0000 7ff5a7a00640 1 -- 172.21.15.159:0/2008350612 >> 172.21.15.159:0/2008350612 conn(0x7ff5a00b3e90 msgr2=0x7ff5a00d1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:01.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.122+0000 7ff5a7a00640 1 -- 172.21.15.159:0/2008350612 shutdown_connections 2024-04-04T06:09:01.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.122+0000 7ff5a7a00640 1 -- 172.21.15.159:0/2008350612 wait complete. 2024-04-04T06:09:01.134 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:09:01.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.339+0000 7f397f000640 1 Processor -- start 2024-04-04T06:09:01.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.339+0000 7f397f000640 1 -- start start 2024-04-04T06:09:01.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.340+0000 7f397f000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f397815c390 0x7f3978073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:01.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.341+0000 7f397f000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f397815fe80 con 0x7f39780d0650 2024-04-04T06:09:01.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.341+0000 7f397f000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39780ad120 con 0x7f39780af830 2024-04-04T06:09:01.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.341+0000 7f397f000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f397815c770 con 0x7f397815c390 2024-04-04T06:09:01.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.341+0000 7f397d000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f39780d0650 0x7f3978071cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40306/0 (socket says 172.21.15.159:40306) 2024-04-04T06:09:01.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.341+0000 7f3977e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f397815c390 0x7f3978073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:01.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.341+0000 7f397d000640 1 -- 172.21.15.159:0/3560701103 learned_addr learned my addr 172.21.15.159:0/3560701103 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:09:01.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.342+0000 7f3976a00640 1 -- 172.21.15.159:0/3560701103 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3451188708 0 0) 0x7f397815fe80 con 0x7f39780d0650 2024-04-04T06:09:01.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.342+0000 7f3976a00640 1 -- 172.21.15.159:0/3560701103 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f395c003520 con 0x7f39780d0650 2024-04-04T06:09:01.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.342+0000 7f3977e00640 1 -- 172.21.15.159:0/3560701103 >> v1:172.21.15.178:6789/0 conn(0x7f39780af830 legacy=0x7f3978160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:01.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.342+0000 7f3977e00640 1 -- 172.21.15.159:0/3560701103 >> v1:172.21.15.159:6789/0 conn(0x7f39780d0650 legacy=0x7f3978071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:01.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.342+0000 7f3976a00640 1 -- 172.21.15.159:0/3560701103 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1532699925 0 0) 0x7f395c003520 con 0x7f39780d0650 2024-04-04T06:09:01.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.342+0000 7f3977e00640 1 -- 172.21.15.159:0/3560701103 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39780ad400 con 0x7f397815c390 2024-04-04T06:09:01.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.343+0000 7f3977e00640 1 --2- 172.21.15.159:0/3560701103 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f397815c390 0x7f3978073790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f39680097d0 tx=0x7f396802b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c0691d2b685e0dd server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:01.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.344+0000 7f3976a00640 1 -- 172.21.15.159:0/3560701103 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968037070 con 0x7f397815c390 2024-04-04T06:09:01.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.344+0000 7f3976a00640 1 -- 172.21.15.159:0/3560701103 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3968030510 con 0x7f397815c390 2024-04-04T06:09:01.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.344+0000 7f3976a00640 1 -- 172.21.15.159:0/3560701103 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39700019f0 con 0x7f397815c390 2024-04-04T06:09:01.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.344+0000 7f397f000640 1 -- 172.21.15.159:0/3560701103 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f397815c390 msgr2=0x7f3978073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:01.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.344+0000 7f397f000640 1 --2- 172.21.15.159:0/3560701103 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f397815c390 0x7f3978073790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f39680097d0 tx=0x7f396802b800 comp rx=0 tx=0).stop 2024-04-04T06:09:01.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.344+0000 7f397f000640 1 -- 172.21.15.159:0/3560701103 shutdown_connections 2024-04-04T06:09:01.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.344+0000 7f397f000640 1 --2- 172.21.15.159:0/3560701103 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f397815c390 0x7f3978073790 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:01.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.344+0000 7f397f000640 1 -- 172.21.15.159:0/3560701103 >> 172.21.15.159:0/3560701103 conn(0x7f397806b990 msgr2=0x7f397806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:01.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.344+0000 7f397f000640 1 -- 172.21.15.159:0/3560701103 shutdown_connections 2024-04-04T06:09:01.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.344+0000 7f397f000640 1 -- 172.21.15.159:0/3560701103 wait complete. 2024-04-04T06:09:01.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.346+0000 7f397f000640 1 Processor -- start 2024-04-04T06:09:01.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.346+0000 7f397f000640 1 -- start start 2024-04-04T06:09:01.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.347+0000 7f397f000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39780af830 0x7f3978159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:01.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.347+0000 7f397f000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39780d0650 0x7f3978155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:01.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.347+0000 7f397f000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f397815c390 0x7f3978150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:01.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.347+0000 7f397f000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3978160d50 con 0x7f39780d0650 2024-04-04T06:09:01.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.347+0000 7f397f000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3978160120 con 0x7f39780af830 2024-04-04T06:09:01.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.347+0000 7f397f000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f39780cf980 con 0x7f397815c390 2024-04-04T06:09:01.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.348+0000 7f3977400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39780d0650 0x7f3978155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:01.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.348+0000 7f3977400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39780d0650 0x7f3978155960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37336/0 (socket says 172.21.15.159:37336) 2024-04-04T06:09:01.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.348+0000 7f3977400640 1 -- 172.21.15.159:0/4044330053 learned_addr learned my addr 172.21.15.159:0/4044330053 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:01.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.348+0000 7f397d000640 1 --2- 172.21.15.159:0/4044330053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f397815c390 0x7f3978150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:01.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.348+0000 7f3977e00640 1 --2- 172.21.15.159:0/4044330053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39780af830 0x7f3978159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:01.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.348+0000 7f3977400640 1 -- 172.21.15.159:0/4044330053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f397815c390 msgr2=0x7f3978150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:01.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.348+0000 7f3977400640 1 --2- 172.21.15.159:0/4044330053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f397815c390 0x7f3978150510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:01.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.348+0000 7f3977400640 1 -- 172.21.15.159:0/4044330053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39780af830 msgr2=0x7f3978159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:01.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.348+0000 7f3977400640 1 --2- 172.21.15.159:0/4044330053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39780af830 0x7f3978159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:01.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.348+0000 7f3977400640 1 -- 172.21.15.159:0/4044330053 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39781589c0 con 0x7f39780d0650 2024-04-04T06:09:01.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.348+0000 7f3977e00640 1 --2- 172.21.15.159:0/4044330053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39780af830 0x7f3978159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:01.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.348+0000 7f3977400640 1 --2- 172.21.15.159:0/4044330053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39780d0650 0x7f3978155960 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f396400c430 tx=0x7f396400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:01.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.349+0000 7f396fe00640 1 -- 172.21.15.159:0/4044330053 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3964017070 con 0x7f39780d0650 2024-04-04T06:09:01.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.349+0000 7f396fe00640 1 -- 172.21.15.159:0/4044330053 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39640052d0 con 0x7f39780d0650 2024-04-04T06:09:01.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.349+0000 7f397f000640 1 -- 172.21.15.159:0/4044330053 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3978158ca0 con 0x7f39780d0650 2024-04-04T06:09:01.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.349+0000 7f396fe00640 1 -- 172.21.15.159:0/4044330053 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3964003830 con 0x7f39780d0650 2024-04-04T06:09:01.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.349+0000 7f397f000640 1 -- 172.21.15.159:0/4044330053 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3978159260 con 0x7f39780d0650 2024-04-04T06:09:01.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.350+0000 7f397f000640 1 -- 172.21.15.159:0/4044330053 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39780685c0 con 0x7f39780d0650 2024-04-04T06:09:01.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.352+0000 7f396fe00640 1 -- 172.21.15.159:0/4044330053 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f396401a070 con 0x7f39780d0650 2024-04-04T06:09:01.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.353+0000 7f396fe00640 1 --2- 172.21.15.159:0/4044330053 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f395806c8f0 0x7f395806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:01.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.353+0000 7f396fe00640 1 -- 172.21.15.159:0/4044330053 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f396408ae90 con 0x7f39780d0650 2024-04-04T06:09:01.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.353+0000 7f3977e00640 1 --2- 172.21.15.159:0/4044330053 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f395806c8f0 0x7f395806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:01.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.355+0000 7f3977e00640 1 --2- 172.21.15.159:0/4044330053 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f395806c8f0 0x7f395806edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3968034000 tx=0x7f3968008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:01.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.357+0000 7f396fe00640 1 -- 172.21.15.159:0/4044330053 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f39640592c0 con 0x7f39780d0650 2024-04-04T06:09:01.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.556+0000 7f397f000640 1 -- 172.21.15.159:0/4044330053 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f397815f1e0 con 0x7f39780d0650 2024-04-04T06:09:01.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.558+0000 7f396fe00640 1 -- 172.21.15.159:0/4044330053 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16677 (secure 0 0 0) 0x7f3964058bc0 con 0x7f39780d0650 2024-04-04T06:09:01.552 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:09:01.552 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":59,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:08:58.581221+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:08:54.848337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-04T06:08:56.003356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:09:01.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.560+0000 7f397f000640 1 -- 172.21.15.159:0/4044330053 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f395806c8f0 msgr2=0x7f395806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:01.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.561+0000 7f397f000640 1 --2- 172.21.15.159:0/4044330053 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f395806c8f0 0x7f395806edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3968034000 tx=0x7f3968008030 comp rx=0 tx=0).stop 2024-04-04T06:09:01.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.561+0000 7f397f000640 1 -- 172.21.15.159:0/4044330053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39780d0650 msgr2=0x7f3978155960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:01.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.561+0000 7f397f000640 1 --2- 172.21.15.159:0/4044330053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39780d0650 0x7f3978155960 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f396400c430 tx=0x7f396400c8f0 comp rx=0 tx=0).stop 2024-04-04T06:09:01.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.561+0000 7f397f000640 1 -- 172.21.15.159:0/4044330053 shutdown_connections 2024-04-04T06:09:01.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.561+0000 7f397f000640 1 --2- 172.21.15.159:0/4044330053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f397815c390 0x7f3978150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:01.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.561+0000 7f397f000640 1 --2- 172.21.15.159:0/4044330053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39780d0650 0x7f3978155960 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:01.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.561+0000 7f397f000640 1 --2- 172.21.15.159:0/4044330053 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f395806c8f0 0x7f395806edb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:01.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.561+0000 7f397f000640 1 --2- 172.21.15.159:0/4044330053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39780af830 0x7f3978159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:01.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.561+0000 7f397f000640 1 -- 172.21.15.159:0/4044330053 >> 172.21.15.159:0/4044330053 conn(0x7f397806b990 msgr2=0x7f39780b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:01.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.561+0000 7f397f000640 1 -- 172.21.15.159:0/4044330053 shutdown_connections 2024-04-04T06:09:01.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.561+0000 7f397f000640 1 -- 172.21.15.159:0/4044330053 wait complete. 2024-04-04T06:09:01.575 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-04T06:09:01.576 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:09:01.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.787+0000 7f2dbc400640 1 Processor -- start 2024-04-04T06:09:01.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.787+0000 7f2dbc400640 1 -- start start 2024-04-04T06:09:01.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.788+0000 7f2dbc400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2db415b310 0x7f2db40727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:01.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.788+0000 7f2dbc400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2db40d0780 0x7f2db40d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:01.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.789+0000 7f2dbc400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2db415b6f0 con 0x7f2db40b4f90 2024-04-04T06:09:01.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.789+0000 7f2dbc400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f2db4072e80 con 0x7f2db40d0780 2024-04-04T06:09:01.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.789+0000 7f2dbc400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2db4073160 con 0x7f2db415b310 2024-04-04T06:09:01.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.789+0000 7f2dba400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2db40d0780 0x7f2db40d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:01.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.789+0000 7f2dba400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2db40d0780 0x7f2db40d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37346/0 (socket says 172.21.15.159:37346) 2024-04-04T06:09:01.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.789+0000 7f2dba400640 1 -- 172.21.15.159:0/778071871 learned_addr learned my addr 172.21.15.159:0/778071871 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:01.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.789+0000 7f2db9a00640 1 --2- 172.21.15.159:0/778071871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2db415b310 0x7f2db40727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:01.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.789+0000 7f2dba400640 1 -- 172.21.15.159:0/778071871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2db415b310 msgr2=0x7f2db40727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:01.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.789+0000 7f2dba400640 1 --2- 172.21.15.159:0/778071871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2db415b310 0x7f2db40727f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:01.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.789+0000 7f2dba400640 1 -- 172.21.15.159:0/778071871 >> v1:172.21.15.178:6789/0 conn(0x7f2db40b4f90 legacy=0x7f2db415f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:01.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.789+0000 7f2dba400640 1 -- 172.21.15.159:0/778071871 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db415efe0 con 0x7f2db40d0780 2024-04-04T06:09:01.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.789+0000 7f2db9a00640 1 --2- 172.21.15.159:0/778071871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2db415b310 0x7f2db40727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:01.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.790+0000 7f2dba400640 1 --2- 172.21.15.159:0/778071871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db40d0780 0x7f2db40d0c10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009e50 tx=0x7f2dac02bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfcac5d4b3dc007e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:01.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.791+0000 7f2db3e00640 1 -- 172.21.15.159:0/778071871 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac02c960 con 0x7f2db40d0780 2024-04-04T06:09:01.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.791+0000 7f2db3e00640 1 -- 172.21.15.159:0/778071871 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dac02cb00 con 0x7f2db40d0780 2024-04-04T06:09:01.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.791+0000 7f2db3e00640 1 -- 172.21.15.159:0/778071871 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac0343f0 con 0x7f2db40d0780 2024-04-04T06:09:01.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.791+0000 7f2dbc400640 1 -- 172.21.15.159:0/778071871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db40d0780 msgr2=0x7f2db40d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:01.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.791+0000 7f2dbc400640 1 --2- 172.21.15.159:0/778071871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db40d0780 0x7f2db40d0c10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009e50 tx=0x7f2dac02bdc0 comp rx=0 tx=0).stop 2024-04-04T06:09:01.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.791+0000 7f2dbc400640 1 -- 172.21.15.159:0/778071871 shutdown_connections 2024-04-04T06:09:01.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.791+0000 7f2dbc400640 1 --2- 172.21.15.159:0/778071871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db40d0780 0x7f2db40d0c10 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:01.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.791+0000 7f2dbc400640 1 --2- 172.21.15.159:0/778071871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2db415b310 0x7f2db40727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:01.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.792+0000 7f2dbc400640 1 -- 172.21.15.159:0/778071871 >> 172.21.15.159:0/778071871 conn(0x7f2db406b990 msgr2=0x7f2db406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:01.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.792+0000 7f2dbc400640 1 -- 172.21.15.159:0/778071871 shutdown_connections 2024-04-04T06:09:01.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.792+0000 7f2dbc400640 1 -- 172.21.15.159:0/778071871 wait complete. 2024-04-04T06:09:01.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.793+0000 7f2dbc400640 1 Processor -- start 2024-04-04T06:09:01.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.793+0000 7f2dbc400640 1 -- start start 2024-04-04T06:09:01.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.794+0000 7f2dbc400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2db40b4f90 0x7f2db4155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:01.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.794+0000 7f2dbc400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2db40d0780 0x7f2db414d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:01.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.794+0000 7f2dbc400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db415b310 0x7f2db415e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:01.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.794+0000 7f2dbc400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2db4073440 con 0x7f2db415b310 2024-04-04T06:09:01.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.794+0000 7f2dbc400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2db40b53f0 con 0x7f2db40b4f90 2024-04-04T06:09:01.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.794+0000 7f2dbc400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2db40adc80 con 0x7f2db40d0780 2024-04-04T06:09:01.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.795+0000 7f2dba400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db415b310 0x7f2db415e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:01.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.795+0000 7f2dba400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db415b310 0x7f2db415e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37358/0 (socket says 172.21.15.159:37358) 2024-04-04T06:09:01.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.795+0000 7f2dba400640 1 -- 172.21.15.159:0/4294582302 learned_addr learned my addr 172.21.15.159:0/4294582302 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:01.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.795+0000 7f2db9000640 1 --2- 172.21.15.159:0/4294582302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2db40d0780 0x7f2db414d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:01.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.795+0000 7f2dba400640 1 -- 172.21.15.159:0/4294582302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2db40d0780 msgr2=0x7f2db414d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:01.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.795+0000 7f2dba400640 1 --2- 172.21.15.159:0/4294582302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2db40d0780 0x7f2db414d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:01.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.795+0000 7f2db9a00640 1 --2- 172.21.15.159:0/4294582302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2db40b4f90 0x7f2db4155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:01.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.795+0000 7f2dba400640 1 -- 172.21.15.159:0/4294582302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2db40b4f90 msgr2=0x7f2db4155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:01.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.795+0000 7f2dba400640 1 --2- 172.21.15.159:0/4294582302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2db40b4f90 0x7f2db4155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:01.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.795+0000 7f2dba400640 1 -- 172.21.15.159:0/4294582302 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db41f4d20 con 0x7f2db415b310 2024-04-04T06:09:01.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.795+0000 7f2dba400640 1 --2- 172.21.15.159:0/4294582302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db415b310 0x7f2db415e080 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009e50 tx=0x7f2dac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:01.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.796+0000 7f2db1600640 1 -- 172.21.15.159:0/4294582302 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac043020 con 0x7f2db415b310 2024-04-04T06:09:01.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.796+0000 7f2db1600640 1 -- 172.21.15.159:0/4294582302 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dac03e450 con 0x7f2db415b310 2024-04-04T06:09:01.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.796+0000 7f2db1600640 1 -- 172.21.15.159:0/4294582302 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac03e820 con 0x7f2db415b310 2024-04-04T06:09:01.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.796+0000 7f2dbc400640 1 -- 172.21.15.159:0/4294582302 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db41f5000 con 0x7f2db415b310 2024-04-04T06:09:01.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.796+0000 7f2dbc400640 1 -- 172.21.15.159:0/4294582302 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db41f55c0 con 0x7f2db415b310 2024-04-04T06:09:01.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.797+0000 7f2dbc400640 1 -- 172.21.15.159:0/4294582302 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db40685c0 con 0x7f2db415b310 2024-04-04T06:09:01.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.799+0000 7f2db1600640 1 -- 172.21.15.159:0/4294582302 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2dac046020 con 0x7f2db415b310 2024-04-04T06:09:01.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.800+0000 7f2db1600640 1 --2- 172.21.15.159:0/4294582302 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d9406c8f0 0x7f2d9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:01.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.800+0000 7f2db1600640 1 -- 172.21.15.159:0/4294582302 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f2dac0ae5c0 con 0x7f2db415b310 2024-04-04T06:09:01.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.801+0000 7f2db9a00640 1 --2- 172.21.15.159:0/4294582302 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d9406c8f0 0x7f2d9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:01.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.802+0000 7f2db9a00640 1 --2- 172.21.15.159:0/4294582302 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d9406c8f0 0x7f2d9406edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2db4155790 tx=0x7f2da8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:01.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:01.805+0000 7f2db1600640 1 -- 172.21.15.159:0/4294582302 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2dac04d030 con 0x7f2db415b310 2024-04-04T06:09:02.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.018+0000 7f2dbc400640 1 -- 172.21.15.159:0/4294582302 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2db415ec20 con 0x7f2d9406c8f0 2024-04-04T06:09:02.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.036+0000 7f2db1600640 1 -- 172.21.15.159:0/4294582302 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+244908 (secure 0 0 0) 0x7f2db415ec20 con 0x7f2d9406c8f0 2024-04-04T06:09:02.033 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:09:02.035 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:09:02.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.041+0000 7f2dbc400640 1 -- 172.21.15.159:0/4294582302 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d9406c8f0 msgr2=0x7f2d9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:02.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.041+0000 7f2dbc400640 1 --2- 172.21.15.159:0/4294582302 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d9406c8f0 0x7f2d9406edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2db4155790 tx=0x7f2da8008040 comp rx=0 tx=0).stop 2024-04-04T06:09:02.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.041+0000 7f2dbc400640 1 -- 172.21.15.159:0/4294582302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db415b310 msgr2=0x7f2db415e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:02.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.041+0000 7f2dbc400640 1 --2- 172.21.15.159:0/4294582302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db415b310 0x7f2db415e080 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009e50 tx=0x7f2dac004a40 comp rx=0 tx=0).stop 2024-04-04T06:09:02.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.041+0000 7f2dbc400640 1 -- 172.21.15.159:0/4294582302 shutdown_connections 2024-04-04T06:09:02.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.041+0000 7f2dbc400640 1 --2- 172.21.15.159:0/4294582302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db415b310 0x7f2db415e080 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:02.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.041+0000 7f2dbc400640 1 --2- 172.21.15.159:0/4294582302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2db40d0780 0x7f2db414d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:02.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.041+0000 7f2dbc400640 1 --2- 172.21.15.159:0/4294582302 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d9406c8f0 0x7f2d9406edb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:02.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.041+0000 7f2dbc400640 1 --2- 172.21.15.159:0/4294582302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2db40b4f90 0x7f2db4155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:02.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.041+0000 7f2dbc400640 1 -- 172.21.15.159:0/4294582302 >> 172.21.15.159:0/4294582302 conn(0x7f2db406b990 msgr2=0x7f2db40ae0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:02.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.042+0000 7f2dbc400640 1 -- 172.21.15.159:0/4294582302 shutdown_connections 2024-04-04T06:09:02.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.042+0000 7f2dbc400640 1 -- 172.21.15.159:0/4294582302 wait complete. 2024-04-04T06:09:02.055 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-04-04T06:09:00.777798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":166,"acting":166,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243312,"kb_used_data":6332,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731408,"statfs":{"total":773094113280,"available":772844961792,"internally_reserved":0,"allocated":6483968,"data_stored":3359401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.236995"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.571500+0000","last_change":"2024-04-04T06:08:56.571500+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.571500+0000","last_undegraded":"2024-04-04T06:08:56.571500+0000","last_fullsized":"2024-04-04T06:08:56.571500+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.557520+0000","last_change":"2024-04-04T06:08:56.557520+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.557520+0000","last_undegraded":"2024-04-04T06:08:56.557520+0000","last_fullsized":"2024-04-04T06:08:56.557520+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.569551+0000","last_change":"2024-04-04T06:08:56.569551+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.569551+0000","last_undegraded":"2024-04-04T06:08:56.569551+0000","last_fullsized":"2024-04-04T06:08:56.569551+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.566616+0000","last_change":"2024-04-04T06:08:56.566616+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.566616+0000","last_undegraded":"2024-04-04T06:08:56.566616+0000","last_fullsized":"2024-04-04T06:08:56.566616+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.566130+0000","last_change":"2024-04-04T06:08:56.566130+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.566130+0000","last_undegraded":"2024-04-04T06:08:56.566130+0000","last_fullsized":"2024-04-04T06:08:56.566130+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.556089+0000","last_change":"2024-04-04T06:08:56.556089+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.556089+0000","last_undegraded":"2024-04-04T06:08:56.556089+0000","last_fullsized":"2024-04-04T06:08:56.556089+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.557291+0000","last_change":"2024-04-04T06:08:56.557291+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.557291+0000","last_undegraded":"2024-04-04T06:08:56.557291+0000","last_fullsized":"2024-04-04T06:08:56.557291+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.557121+0000","last_change":"2024-04-04T06:08:56.557121+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.557121+0000","last_undegraded":"2024-04-04T06:08:56.557121+0000","last_fullsized":"2024-04-04T06:08:56.557121+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.566615+0000","last_change":"2024-04-04T06:08:56.566615+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.566615+0000","last_undegraded":"2024-04-04T06:08:56.566615+0000","last_fullsized":"2024-04-04T06:08:56.566615+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.563685+0000","last_change":"2024-04-04T06:08:56.563685+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.563685+0000","last_undegraded":"2024-04-04T06:08:56.563685+0000","last_fullsized":"2024-04-04T06:08:56.563685+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.558504+0000","last_change":"2024-04-04T06:08:56.558504+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.558504+0000","last_undegraded":"2024-04-04T06:08:56.558504+0000","last_fullsized":"2024-04-04T06:08:56.558504+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.568936+0000","last_change":"2024-04-04T06:08:56.568936+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.568936+0000","last_undegraded":"2024-04-04T06:08:56.568936+0000","last_fullsized":"2024-04-04T06:08:56.568936+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.568905+0000","last_change":"2024-04-04T06:08:56.568905+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.568905+0000","last_undegraded":"2024-04-04T06:08:56.568905+0000","last_fullsized":"2024-04-04T06:08:56.568905+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.559296+0000","last_change":"2024-04-04T06:08:56.559296+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.559296+0000","last_undegraded":"2024-04-04T06:08:56.559296+0000","last_fullsized":"2024-04-04T06:08:56.559296+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.564535+0000","last_change":"2024-04-04T06:08:56.564535+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.564535+0000","last_undegraded":"2024-04-04T06:08:56.564535+0000","last_fullsized":"2024-04-04T06:08:56.564535+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.560679+0000","last_change":"2024-04-04T06:08:56.560679+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.560679+0000","last_undegraded":"2024-04-04T06:08:56.560679+0000","last_fullsized":"2024-04-04T06:08:56.560679+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.565507+0000","last_change":"2024-04-04T06:08:56.565507+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.565507+0000","last_undegraded":"2024-04-04T06:08:56.565507+0000","last_fullsized":"2024-04-04T06:08:56.565507+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.564719+0000","last_change":"2024-04-04T06:08:56.564719+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.564719+0000","last_undegraded":"2024-04-04T06:08:56.564719+0000","last_fullsized":"2024-04-04T06:08:56.564719+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.571019+0000","last_change":"2024-04-04T06:08:56.571019+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.571019+0000","last_undegraded":"2024-04-04T06:08:56.571019+0000","last_fullsized":"2024-04-04T06:08:56.571019+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.568258+0000","last_change":"2024-04-04T06:08:56.568258+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.568258+0000","last_undegraded":"2024-04-04T06:08:56.568258+0000","last_fullsized":"2024-04-04T06:08:56.568258+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.568844+0000","last_change":"2024-04-04T06:08:56.568844+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.568844+0000","last_undegraded":"2024-04-04T06:08:56.568844+0000","last_fullsized":"2024-04-04T06:08:56.568844+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.555962+0000","last_change":"2024-04-04T06:08:56.555962+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.555962+0000","last_undegraded":"2024-04-04T06:08:56.555962+0000","last_fullsized":"2024-04-04T06:08:56.555962+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.563532+0000","last_change":"2024-04-04T06:08:56.563532+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.563532+0000","last_undegraded":"2024-04-04T06:08:56.563532+0000","last_fullsized":"2024-04-04T06:08:56.563532+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.564880+0000","last_change":"2024-04-04T06:08:56.564880+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.564880+0000","last_undegraded":"2024-04-04T06:08:56.564880+0000","last_fullsized":"2024-04-04T06:08:56.564880+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.563141+0000","last_change":"2024-04-04T06:08:56.563141+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.563141+0000","last_undegraded":"2024-04-04T06:08:56.563141+0000","last_fullsized":"2024-04-04T06:08:56.563141+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.569809+0000","last_change":"2024-04-04T06:08:56.569809+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.569809+0000","last_undegraded":"2024-04-04T06:08:56.569809+0000","last_fullsized":"2024-04-04T06:08:56.569809+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.587031+0000","last_change":"2024-04-04T06:08:56.578928+0000","last_active":"2024-04-04T06:08:56.587031+0000","last_peered":"2024-04-04T06:08:56.587031+0000","last_clean":"2024-04-04T06:08:56.587031+0000","last_became_active":"2024-04-04T06:08:56.578602+0000","last_became_peered":"2024-04-04T06:08:56.578602+0000","last_unstale":"2024-04-04T06:08:56.587031+0000","last_undegraded":"2024-04-04T06:08:56.587031+0000","last_fullsized":"2024-04-04T06:08:56.587031+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:16:20.681483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.556001+0000","last_change":"2024-04-04T06:08:56.556001+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.556001+0000","last_undegraded":"2024-04-04T06:08:56.556001+0000","last_fullsized":"2024-04-04T06:08:56.556001+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.582255+0000","last_change":"2024-04-04T06:08:56.577361+0000","last_active":"2024-04-04T06:08:56.582255+0000","last_peered":"2024-04-04T06:08:56.582255+0000","last_clean":"2024-04-04T06:08:56.582255+0000","last_became_active":"2024-04-04T06:08:56.576619+0000","last_became_peered":"2024-04-04T06:08:56.576619+0000","last_unstale":"2024-04-04T06:08:56.582255+0000","last_undegraded":"2024-04-04T06:08:56.582255+0000","last_fullsized":"2024-04-04T06:08:56.582255+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:28:56.617090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.566480+0000","last_change":"2024-04-04T06:08:56.566480+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.566480+0000","last_undegraded":"2024-04-04T06:08:56.566480+0000","last_fullsized":"2024-04-04T06:08:56.566480+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.586821+0000","last_change":"2024-04-04T06:08:56.577805+0000","last_active":"2024-04-04T06:08:56.586821+0000","last_peered":"2024-04-04T06:08:56.586821+0000","last_clean":"2024-04-04T06:08:56.586821+0000","last_became_active":"2024-04-04T06:08:56.576393+0000","last_became_peered":"2024-04-04T06:08:56.576393+0000","last_unstale":"2024-04-04T06:08:56.586821+0000","last_undegraded":"2024-04-04T06:08:56.586821+0000","last_fullsized":"2024-04-04T06:08:56.586821+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:02:33.508277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.567741+0000","last_change":"2024-04-04T06:08:56.567741+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.567741+0000","last_undegraded":"2024-04-04T06:08:56.567741+0000","last_fullsized":"2024-04-04T06:08:56.567741+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-04T06:08:55.546828+0000","last_change":"2024-04-04T06:08:55.546828+0000","last_active":"2024-04-04T06:08:55.523759+0000","last_peered":"2024-04-04T06:08:55.523759+0000","last_clean":"2024-04-04T06:08:55.523759+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:55.546828+0000","last_undegraded":"2024-04-04T06:08:55.546828+0000","last_fullsized":"2024-04-04T06:08:55.546828+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.560203+0000","last_change":"2024-04-04T06:08:56.560203+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.560203+0000","last_undegraded":"2024-04-04T06:08:56.560203+0000","last_fullsized":"2024-04-04T06:08:56.560203+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-04T06:08:55.554250+0000","last_change":"2024-04-04T06:08:55.554250+0000","last_active":"2024-04-04T06:08:55.523759+0000","last_peered":"2024-04-04T06:08:55.523759+0000","last_clean":"2024-04-04T06:08:55.523759+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:55.554250+0000","last_undegraded":"2024-04-04T06:08:55.554250+0000","last_fullsized":"2024-04-04T06:08:55.554250+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.558303+0000","last_change":"2024-04-04T06:08:56.558303+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.558303+0000","last_undegraded":"2024-04-04T06:08:56.558303+0000","last_fullsized":"2024-04-04T06:08:56.558303+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.587052+0000","last_change":"2024-04-04T06:08:56.579024+0000","last_active":"2024-04-04T06:08:56.587052+0000","last_peered":"2024-04-04T06:08:56.587052+0000","last_clean":"2024-04-04T06:08:56.587052+0000","last_became_active":"2024-04-04T06:08:56.578301+0000","last_became_peered":"2024-04-04T06:08:56.578301+0000","last_unstale":"2024-04-04T06:08:56.587052+0000","last_undegraded":"2024-04-04T06:08:56.587052+0000","last_fullsized":"2024-04-04T06:08:56.587052+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:03:42.180669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.563882+0000","last_change":"2024-04-04T06:08:56.563882+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.563882+0000","last_undegraded":"2024-04-04T06:08:56.563882+0000","last_fullsized":"2024-04-04T06:08:56.563882+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-04T06:08:55.546818+0000","last_change":"2024-04-04T06:08:55.546818+0000","last_active":"2024-04-04T06:08:55.523759+0000","last_peered":"2024-04-04T06:08:55.523759+0000","last_clean":"2024-04-04T06:08:55.523759+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:55.546818+0000","last_undegraded":"2024-04-04T06:08:55.546818+0000","last_fullsized":"2024-04-04T06:08:55.546818+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.558924+0000","last_change":"2024-04-04T06:08:56.558924+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.558924+0000","last_undegraded":"2024-04-04T06:08:56.558924+0000","last_fullsized":"2024-04-04T06:08:56.558924+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-04T06:08:55.553932+0000","last_change":"2024-04-04T06:08:55.553932+0000","last_active":"2024-04-04T06:08:55.523759+0000","last_peered":"2024-04-04T06:08:55.523759+0000","last_clean":"2024-04-04T06:08:55.523759+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:55.553932+0000","last_undegraded":"2024-04-04T06:08:55.553932+0000","last_fullsized":"2024-04-04T06:08:55.553932+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.588221+0000","last_change":"2024-04-04T06:08:56.587466+0000","last_active":"2024-04-04T06:08:56.588221+0000","last_peered":"2024-04-04T06:08:56.588221+0000","last_clean":"2024-04-04T06:08:56.588221+0000","last_became_active":"2024-04-04T06:08:56.587163+0000","last_became_peered":"2024-04-04T06:08:56.587163+0000","last_unstale":"2024-04-04T06:08:56.588221+0000","last_undegraded":"2024-04-04T06:08:56.588221+0000","last_fullsized":"2024-04-04T06:08:56.588221+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:42:24.830758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.569960+0000","last_change":"2024-04-04T06:08:56.569960+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.569960+0000","last_undegraded":"2024-04-04T06:08:56.569960+0000","last_fullsized":"2024-04-04T06:08:56.569960+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.584371+0000","last_change":"2024-04-04T06:08:56.581539+0000","last_active":"2024-04-04T06:08:56.584371+0000","last_peered":"2024-04-04T06:08:56.584371+0000","last_clean":"2024-04-04T06:08:56.584371+0000","last_became_active":"2024-04-04T06:08:56.579490+0000","last_became_peered":"2024-04-04T06:08:56.579490+0000","last_unstale":"2024-04-04T06:08:56.584371+0000","last_undegraded":"2024-04-04T06:08:56.584371+0000","last_fullsized":"2024-04-04T06:08:56.584371+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:50:00.796043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.555714+0000","last_change":"2024-04-04T06:08:56.555714+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.555714+0000","last_undegraded":"2024-04-04T06:08:56.555714+0000","last_fullsized":"2024-04-04T06:08:56.555714+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.583461+0000","last_change":"2024-04-04T06:08:56.582280+0000","last_active":"2024-04-04T06:08:56.583461+0000","last_peered":"2024-04-04T06:08:56.583461+0000","last_clean":"2024-04-04T06:08:56.583461+0000","last_became_active":"2024-04-04T06:08:56.581968+0000","last_became_peered":"2024-04-04T06:08:56.581968+0000","last_unstale":"2024-04-04T06:08:56.583461+0000","last_undegraded":"2024-04-04T06:08:56.583461+0000","last_fullsized":"2024-04-04T06:08:56.583461+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:38:14.124440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.568800+0000","last_change":"2024-04-04T06:08:56.568800+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.568800+0000","last_undegraded":"2024-04-04T06:08:56.568800+0000","last_fullsized":"2024-04-04T06:08:56.568800+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-04T06:08:55.554325+0000","last_change":"2024-04-04T06:08:55.554325+0000","last_active":"2024-04-04T06:08:55.523759+0000","last_peered":"2024-04-04T06:08:55.523759+0000","last_clean":"2024-04-04T06:08:55.523759+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:55.554325+0000","last_undegraded":"2024-04-04T06:08:55.554325+0000","last_fullsized":"2024-04-04T06:08:55.554325+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.579674+0000","last_change":"2024-04-04T06:08:56.575082+0000","last_active":"2024-04-04T06:08:56.579674+0000","last_peered":"2024-04-04T06:08:56.579674+0000","last_clean":"2024-04-04T06:08:56.579674+0000","last_became_active":"2024-04-04T06:08:56.574805+0000","last_became_peered":"2024-04-04T06:08:56.574805+0000","last_unstale":"2024-04-04T06:08:56.579674+0000","last_undegraded":"2024-04-04T06:08:56.579674+0000","last_fullsized":"2024-04-04T06:08:56.579674+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:58:27.216190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.563960+0000","last_change":"2024-04-04T06:08:56.563960+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.563960+0000","last_undegraded":"2024-04-04T06:08:56.563960+0000","last_fullsized":"2024-04-04T06:08:56.563960+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.580524+0000","last_change":"2024-04-04T06:08:56.576046+0000","last_active":"2024-04-04T06:08:56.580524+0000","last_peered":"2024-04-04T06:08:56.580524+0000","last_clean":"2024-04-04T06:08:56.580524+0000","last_became_active":"2024-04-04T06:08:56.575726+0000","last_became_peered":"2024-04-04T06:08:56.575726+0000","last_unstale":"2024-04-04T06:08:56.580524+0000","last_undegraded":"2024-04-04T06:08:56.580524+0000","last_fullsized":"2024-04-04T06:08:56.580524+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:35:45.378741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-04T06:08:55.545600+0000","last_change":"2024-04-04T06:08:55.545600+0000","last_active":"2024-04-04T06:08:55.523759+0000","last_peered":"2024-04-04T06:08:55.523759+0000","last_clean":"2024-04-04T06:08:55.523759+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:55.545600+0000","last_undegraded":"2024-04-04T06:08:55.545600+0000","last_fullsized":"2024-04-04T06:08:55.545600+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.564501+0000","last_change":"2024-04-04T06:08:56.564501+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.564501+0000","last_undegraded":"2024-04-04T06:08:56.564501+0000","last_fullsized":"2024-04-04T06:08:56.564501+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.583872+0000","last_change":"2024-04-04T06:08:56.582998+0000","last_active":"2024-04-04T06:08:56.583872+0000","last_peered":"2024-04-04T06:08:56.583872+0000","last_clean":"2024-04-04T06:08:56.583872+0000","last_became_active":"2024-04-04T06:08:56.582720+0000","last_became_peered":"2024-04-04T06:08:56.582720+0000","last_unstale":"2024-04-04T06:08:56.583872+0000","last_undegraded":"2024-04-04T06:08:56.583872+0000","last_fullsized":"2024-04-04T06:08:56.583872+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:34:48.461720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.566434+0000","last_change":"2024-04-04T06:08:56.566434+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.566434+0000","last_undegraded":"2024-04-04T06:08:56.566434+0000","last_fullsized":"2024-04-04T06:08:56.566434+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.584260+0000","last_change":"2024-04-04T06:08:56.575119+0000","last_active":"2024-04-04T06:08:56.584260+0000","last_peered":"2024-04-04T06:08:56.584260+0000","last_clean":"2024-04-04T06:08:56.584260+0000","last_became_active":"2024-04-04T06:08:56.574832+0000","last_became_peered":"2024-04-04T06:08:56.574832+0000","last_unstale":"2024-04-04T06:08:56.584260+0000","last_undegraded":"2024-04-04T06:08:56.584260+0000","last_fullsized":"2024-04-04T06:08:56.584260+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:24:50.694749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.586722+0000","last_change":"2024-04-04T06:08:56.576721+0000","last_active":"2024-04-04T06:08:56.586722+0000","last_peered":"2024-04-04T06:08:56.586722+0000","last_clean":"2024-04-04T06:08:56.586722+0000","last_became_active":"2024-04-04T06:08:56.576404+0000","last_became_peered":"2024-04-04T06:08:56.576404+0000","last_unstale":"2024-04-04T06:08:56.586722+0000","last_undegraded":"2024-04-04T06:08:56.586722+0000","last_fullsized":"2024-04-04T06:08:56.586722+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:29:08.905489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.556399+0000","last_change":"2024-04-04T06:08:56.556399+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.556399+0000","last_undegraded":"2024-04-04T06:08:56.556399+0000","last_fullsized":"2024-04-04T06:08:56.556399+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-04T06:08:55.545610+0000","last_change":"2024-04-04T06:08:55.545610+0000","last_active":"2024-04-04T06:08:55.523759+0000","last_peered":"2024-04-04T06:08:55.523759+0000","last_clean":"2024-04-04T06:08:55.523759+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:55.545610+0000","last_undegraded":"2024-04-04T06:08:55.545610+0000","last_fullsized":"2024-04-04T06:08:55.545610+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.565011+0000","last_change":"2024-04-04T06:08:56.565011+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.565011+0000","last_undegraded":"2024-04-04T06:08:56.565011+0000","last_fullsized":"2024-04-04T06:08:56.565011+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.586828+0000","last_change":"2024-04-04T06:08:56.577972+0000","last_active":"2024-04-04T06:08:56.586828+0000","last_peered":"2024-04-04T06:08:56.586828+0000","last_clean":"2024-04-04T06:08:56.586828+0000","last_became_active":"2024-04-04T06:08:56.577700+0000","last_became_peered":"2024-04-04T06:08:56.577700+0000","last_unstale":"2024-04-04T06:08:56.586828+0000","last_undegraded":"2024-04-04T06:08:56.586828+0000","last_fullsized":"2024-04-04T06:08:56.586828+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:56.028901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.579173+0000","last_change":"2024-04-04T06:08:56.571770+0000","last_active":"2024-04-04T06:08:56.579173+0000","last_peered":"2024-04-04T06:08:56.579173+0000","last_clean":"2024-04-04T06:08:56.579173+0000","last_became_active":"2024-04-04T06:08:56.571431+0000","last_became_peered":"2024-04-04T06:08:56.571431+0000","last_unstale":"2024-04-04T06:08:56.579173+0000","last_undegraded":"2024-04-04T06:08:56.579173+0000","last_fullsized":"2024-04-04T06:08:56.579173+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:41:25.336576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.563038+0000","last_change":"2024-04-04T06:08:56.563038+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.563038+0000","last_undegraded":"2024-04-04T06:08:56.563038+0000","last_fullsized":"2024-04-04T06:08:56.563038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.581283+0000","last_change":"2024-04-04T06:08:56.574202+0000","last_active":"2024-04-04T06:08:56.581283+0000","last_peered":"2024-04-04T06:08:56.581283+0000","last_clean":"2024-04-04T06:08:56.581283+0000","last_became_active":"2024-04-04T06:08:56.573425+0000","last_became_peered":"2024-04-04T06:08:56.573425+0000","last_unstale":"2024-04-04T06:08:56.581283+0000","last_undegraded":"2024-04-04T06:08:56.581283+0000","last_fullsized":"2024-04-04T06:08:56.581283+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:55:40.756647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.564641+0000","last_change":"2024-04-04T06:08:56.564641+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.564641+0000","last_undegraded":"2024-04-04T06:08:56.564641+0000","last_fullsized":"2024-04-04T06:08:56.564641+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-04T06:08:55.545246+0000","last_change":"2024-04-04T06:08:55.545246+0000","last_active":"2024-04-04T06:08:55.523759+0000","last_peered":"2024-04-04T06:08:55.523759+0000","last_clean":"2024-04-04T06:08:55.523759+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:55.545246+0000","last_undegraded":"2024-04-04T06:08:55.545246+0000","last_fullsized":"2024-04-04T06:08:55.545246+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.581329+0000","last_change":"2024-04-04T06:08:56.574911+0000","last_active":"2024-04-04T06:08:56.581329+0000","last_peered":"2024-04-04T06:08:56.581329+0000","last_clean":"2024-04-04T06:08:56.581329+0000","last_became_active":"2024-04-04T06:08:56.573037+0000","last_became_peered":"2024-04-04T06:08:56.573037+0000","last_unstale":"2024-04-04T06:08:56.581329+0000","last_undegraded":"2024-04-04T06:08:56.581329+0000","last_fullsized":"2024-04-04T06:08:56.581329+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:52:20.881931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.586930+0000","last_change":"2024-04-04T06:08:56.578575+0000","last_active":"2024-04-04T06:08:56.586930+0000","last_peered":"2024-04-04T06:08:56.586930+0000","last_clean":"2024-04-04T06:08:56.586930+0000","last_became_active":"2024-04-04T06:08:56.578232+0000","last_became_peered":"2024-04-04T06:08:56.578232+0000","last_unstale":"2024-04-04T06:08:56.586930+0000","last_undegraded":"2024-04-04T06:08:56.586930+0000","last_fullsized":"2024-04-04T06:08:56.586930+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:25:43.268590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.554392+0000","last_change":"2024-04-04T06:08:56.554392+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.554392+0000","last_undegraded":"2024-04-04T06:08:56.554392+0000","last_fullsized":"2024-04-04T06:08:56.554392+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.578545+0000","last_change":"2024-04-04T06:08:56.573941+0000","last_active":"2024-04-04T06:08:56.578545+0000","last_peered":"2024-04-04T06:08:56.578545+0000","last_clean":"2024-04-04T06:08:56.578545+0000","last_became_active":"2024-04-04T06:08:56.573700+0000","last_became_peered":"2024-04-04T06:08:56.573700+0000","last_unstale":"2024-04-04T06:08:56.578545+0000","last_undegraded":"2024-04-04T06:08:56.578545+0000","last_fullsized":"2024-04-04T06:08:56.578545+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:17:21.845820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.567298+0000","last_change":"2024-04-04T06:08:56.567298+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.567298+0000","last_undegraded":"2024-04-04T06:08:56.567298+0000","last_fullsized":"2024-04-04T06:08:56.567298+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-04T06:08:55.545546+0000","last_change":"2024-04-04T06:08:55.545546+0000","last_active":"2024-04-04T06:08:55.523759+0000","last_peered":"2024-04-04T06:08:55.523759+0000","last_clean":"2024-04-04T06:08:55.523759+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:55.545546+0000","last_undegraded":"2024-04-04T06:08:55.545546+0000","last_fullsized":"2024-04-04T06:08:55.545546+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.555361+0000","last_change":"2024-04-04T06:08:56.555361+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.555361+0000","last_undegraded":"2024-04-04T06:08:56.555361+0000","last_fullsized":"2024-04-04T06:08:56.555361+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.574765+0000","last_change":"2024-04-04T06:08:56.569703+0000","last_active":"2024-04-04T06:08:56.574765+0000","last_peered":"2024-04-04T06:08:56.574765+0000","last_clean":"2024-04-04T06:08:56.574765+0000","last_became_active":"2024-04-04T06:08:56.569433+0000","last_became_peered":"2024-04-04T06:08:56.569433+0000","last_unstale":"2024-04-04T06:08:56.574765+0000","last_undegraded":"2024-04-04T06:08:56.574765+0000","last_fullsized":"2024-04-04T06:08:56.574765+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:20:28.512324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.563309+0000","last_change":"2024-04-04T06:08:56.563309+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.563309+0000","last_undegraded":"2024-04-04T06:08:56.563309+0000","last_fullsized":"2024-04-04T06:08:56.563309+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-04T06:08:55.553867+0000","last_change":"2024-04-04T06:08:55.553867+0000","last_active":"2024-04-04T06:08:55.523759+0000","last_peered":"2024-04-04T06:08:55.523759+0000","last_clean":"2024-04-04T06:08:55.523759+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:55.553867+0000","last_undegraded":"2024-04-04T06:08:55.553867+0000","last_fullsized":"2024-04-04T06:08:55.553867+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.554390+0000","last_change":"2024-04-04T06:08:56.554390+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.554390+0000","last_undegraded":"2024-04-04T06:08:56.554390+0000","last_fullsized":"2024-04-04T06:08:56.554390+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.581408+0000","last_change":"2024-04-04T06:08:56.576604+0000","last_active":"2024-04-04T06:08:56.581408+0000","last_peered":"2024-04-04T06:08:56.581408+0000","last_clean":"2024-04-04T06:08:56.581408+0000","last_became_active":"2024-04-04T06:08:56.576310+0000","last_became_peered":"2024-04-04T06:08:56.576310+0000","last_unstale":"2024-04-04T06:08:56.581408+0000","last_undegraded":"2024-04-04T06:08:56.581408+0000","last_fullsized":"2024-04-04T06:08:56.581408+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:00:29.103633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.555742+0000","last_change":"2024-04-04T06:08:56.555742+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.555742+0000","last_undegraded":"2024-04-04T06:08:56.555742+0000","last_fullsized":"2024-04-04T06:08:56.555742+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.575097+0000","last_change":"2024-04-04T06:08:56.568855+0000","last_active":"2024-04-04T06:08:56.575097+0000","last_peered":"2024-04-04T06:08:56.575097+0000","last_clean":"2024-04-04T06:08:56.575097+0000","last_became_active":"2024-04-04T06:08:56.568608+0000","last_became_peered":"2024-04-04T06:08:56.568608+0000","last_unstale":"2024-04-04T06:08:56.575097+0000","last_undegraded":"2024-04-04T06:08:56.575097+0000","last_fullsized":"2024-04-04T06:08:56.575097+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:42:11.384223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.555515+0000","last_change":"2024-04-04T06:08:56.555515+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.555515+0000","last_undegraded":"2024-04-04T06:08:56.555515+0000","last_fullsized":"2024-04-04T06:08:56.555515+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.582081+0000","last_change":"2024-04-04T06:08:56.578712+0000","last_active":"2024-04-04T06:08:56.582081+0000","last_peered":"2024-04-04T06:08:56.582081+0000","last_clean":"2024-04-04T06:08:56.582081+0000","last_became_active":"2024-04-04T06:08:56.578391+0000","last_became_peered":"2024-04-04T06:08:56.578391+0000","last_unstale":"2024-04-04T06:08:56.582081+0000","last_undegraded":"2024-04-04T06:08:56.582081+0000","last_fullsized":"2024-04-04T06:08:56.582081+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:44:31.471700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":233,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.574641+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:08:56.574641+0000","last_peered":"2024-04-04T06:08:56.574641+0000","last_clean":"2024-04-04T06:08:56.574641+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:08:56.574641+0000","last_undegraded":"2024-04-04T06:08:56.574641+0000","last_fullsized":"2024-04-04T06:08:56.574641+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":100,"acting":100,"num_store_stats":6},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509481,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1257472,"data_stored":863372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509483,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":663552,"data_stored":273004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509484,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":273004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509483,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":675840,"data_stored":279271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":273004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509484,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1257472,"data_stored":863372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542187,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":651264,"data_stored":267187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542185,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":651264,"data_stored":267187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:09:02.060 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:09:02.305 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.311+0000 7fa212000640 1 Processor -- start 2024-04-04T06:09:02.305 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.311+0000 7fa212000640 1 -- start start 2024-04-04T06:09:02.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.313+0000 7fa212000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa20c15c390 0x7fa20c073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:02.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.313+0000 7fa212000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa20c15fe80 con 0x7fa20c0af830 2024-04-04T06:09:02.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.313+0000 7fa212000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa20c0ad120 con 0x7fa20c0d0650 2024-04-04T06:09:02.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.313+0000 7fa212000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fa20c15c770 con 0x7fa20c15c390 2024-04-04T06:09:02.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.313+0000 7fa20b400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa20c15c390 0x7fa20c073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:02.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.313+0000 7fa20b400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa20c15c390 0x7fa20c073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37374/0 (socket says 172.21.15.159:37374) 2024-04-04T06:09:02.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.313+0000 7fa20b400640 1 -- 172.21.15.159:0/2967357091 learned_addr learned my addr 172.21.15.159:0/2967357091 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:02.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.313+0000 7fa20aa00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fa20c0af830 0x7fa20c160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40320/0 (socket says 172.21.15.159:40320) 2024-04-04T06:09:02.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.313+0000 7fa20b400640 1 -- 172.21.15.159:0/2967357091 >> v1:172.21.15.178:6789/0 conn(0x7fa20c0d0650 legacy=0x7fa20c071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:02.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.313+0000 7fa20b400640 1 -- 172.21.15.159:0/2967357091 >> v1:172.21.15.159:6789/0 conn(0x7fa20c0af830 legacy=0x7fa20c160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:02.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.314+0000 7fa20b400640 1 -- 172.21.15.159:0/2967357091 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20c0ad400 con 0x7fa20c15c390 2024-04-04T06:09:02.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.315+0000 7fa20b400640 1 --2- 172.21.15.159:0/2967357091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa20c15c390 0x7fa20c073790 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009870 tx=0x7fa1fc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab1b6dac7f26058 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:02.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.315+0000 7fa20a000640 1 -- 172.21.15.159:0/2967357091 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc034070 con 0x7fa20c15c390 2024-04-04T06:09:02.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.315+0000 7fa20a000640 1 -- 172.21.15.159:0/2967357091 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1fc030860 con 0x7fa20c15c390 2024-04-04T06:09:02.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.315+0000 7fa20a000640 1 -- 172.21.15.159:0/2967357091 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc030c30 con 0x7fa20c15c390 2024-04-04T06:09:02.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.315+0000 7fa212000640 1 -- 172.21.15.159:0/2967357091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa20c15c390 msgr2=0x7fa20c073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:02.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.315+0000 7fa212000640 1 --2- 172.21.15.159:0/2967357091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa20c15c390 0x7fa20c073790 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009870 tx=0x7fa1fc02b9a0 comp rx=0 tx=0).stop 2024-04-04T06:09:02.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.316+0000 7fa212000640 1 -- 172.21.15.159:0/2967357091 shutdown_connections 2024-04-04T06:09:02.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.316+0000 7fa212000640 1 --2- 172.21.15.159:0/2967357091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa20c15c390 0x7fa20c073790 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:02.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.316+0000 7fa212000640 1 -- 172.21.15.159:0/2967357091 >> 172.21.15.159:0/2967357091 conn(0x7fa20c06b990 msgr2=0x7fa20c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:02.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.316+0000 7fa212000640 1 -- 172.21.15.159:0/2967357091 shutdown_connections 2024-04-04T06:09:02.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.316+0000 7fa212000640 1 -- 172.21.15.159:0/2967357091 wait complete. 2024-04-04T06:09:02.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.317+0000 7fa212000640 1 Processor -- start 2024-04-04T06:09:02.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.318+0000 7fa212000640 1 -- start start 2024-04-04T06:09:02.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.318+0000 7fa212000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa20c0af830 0x7fa20c0b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:02.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.318+0000 7fa212000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa20c0d0650 0x7fa20c150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:02.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.318+0000 7fa212000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa20c15c390 0x7fa20c14f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:02.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.318+0000 7fa212000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c160d50 con 0x7fa20c0d0650 2024-04-04T06:09:02.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.319+0000 7fa212000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c160120 con 0x7fa20c15c390 2024-04-04T06:09:02.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.319+0000 7fa212000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa20c0cf980 con 0x7fa20c0af830 2024-04-04T06:09:02.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.319+0000 7fa20aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa20c0d0650 0x7fa20c150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:02.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.319+0000 7fa20aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa20c0d0650 0x7fa20c150c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37378/0 (socket says 172.21.15.159:37378) 2024-04-04T06:09:02.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.319+0000 7fa20aa00640 1 -- 172.21.15.159:0/3223615600 learned_addr learned my addr 172.21.15.159:0/3223615600 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:02.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.319+0000 7fa20b400640 1 --2- 172.21.15.159:0/3223615600 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa20c0af830 0x7fa20c0b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:02.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.319+0000 7fa20be00640 1 --2- 172.21.15.159:0/3223615600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa20c15c390 0x7fa20c14f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:02.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.319+0000 7fa20aa00640 1 -- 172.21.15.159:0/3223615600 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa20c0af830 msgr2=0x7fa20c0b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:02.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.319+0000 7fa20aa00640 1 --2- 172.21.15.159:0/3223615600 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa20c0af830 0x7fa20c0b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:02.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.319+0000 7fa20aa00640 1 -- 172.21.15.159:0/3223615600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa20c15c390 msgr2=0x7fa20c14f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:02.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.319+0000 7fa20aa00640 1 --2- 172.21.15.159:0/3223615600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa20c15c390 0x7fa20c14f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:02.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.319+0000 7fa20aa00640 1 -- 172.21.15.159:0/3223615600 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20c0b4b80 con 0x7fa20c0d0650 2024-04-04T06:09:02.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.319+0000 7fa20be00640 1 --2- 172.21.15.159:0/3223615600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa20c15c390 0x7fa20c14f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:02.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.320+0000 7fa20aa00640 1 --2- 172.21.15.159:0/3223615600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa20c0d0650 0x7fa20c150c10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800c400 tx=0x7fa1f800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:02.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.320+0000 7fa203400640 1 -- 172.21.15.159:0/3223615600 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f80074e0 con 0x7fa20c0d0650 2024-04-04T06:09:02.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.320+0000 7fa203400640 1 -- 172.21.15.159:0/3223615600 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f8007680 con 0x7fa20c0d0650 2024-04-04T06:09:02.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.320+0000 7fa212000640 1 -- 172.21.15.159:0/3223615600 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20c151150 con 0x7fa20c0d0650 2024-04-04T06:09:02.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.320+0000 7fa203400640 1 -- 172.21.15.159:0/3223615600 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8004a00 con 0x7fa20c0d0650 2024-04-04T06:09:02.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.320+0000 7fa212000640 1 -- 172.21.15.159:0/3223615600 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa20c0b45c0 con 0x7fa20c0d0650 2024-04-04T06:09:02.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.321+0000 7fa212000640 1 -- 172.21.15.159:0/3223615600 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa20c0685c0 con 0x7fa20c0d0650 2024-04-04T06:09:02.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.322+0000 7fa203400640 1 -- 172.21.15.159:0/3223615600 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1f801a020 con 0x7fa20c0d0650 2024-04-04T06:09:02.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.323+0000 7fa203400640 1 --2- 172.21.15.159:0/3223615600 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa1ec06c8f0 0x7fa1ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:02.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.323+0000 7fa203400640 1 -- 172.21.15.159:0/3223615600 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fa1f808b3c0 con 0x7fa20c0d0650 2024-04-04T06:09:02.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.324+0000 7fa20b400640 1 --2- 172.21.15.159:0/3223615600 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa1ec06c8f0 0x7fa1ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:02.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.325+0000 7fa20b400640 1 --2- 172.21.15.159:0/3223615600 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa1ec06c8f0 0x7fa1ec06edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc036000 tx=0x7fa1fc03b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:02.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.328+0000 7fa203400640 1 -- 172.21.15.159:0/3223615600 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa1f8010300 con 0x7fa20c0d0650 2024-04-04T06:09:02.507 INFO:tasks.ceph.mds.c.smithi159.stderr:2024-04-04T06:09:02.513+0000 7f193e600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:09:02.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.526+0000 7fa212000640 1 -- 172.21.15.159:0/3223615600 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa20c06f1b0 con 0x7fa1ec06c8f0 2024-04-04T06:09:02.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.545+0000 7fa203400640 1 -- 172.21.15.159:0/3223615600 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+244908 (secure 0 0 0) 0x7fa20c06f1b0 con 0x7fa1ec06c8f0 2024-04-04T06:09:02.542 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:09:02.543 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:09:02.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.550+0000 7fa212000640 1 -- 172.21.15.159:0/3223615600 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa1ec06c8f0 msgr2=0x7fa1ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:02.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.550+0000 7fa212000640 1 --2- 172.21.15.159:0/3223615600 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa1ec06c8f0 0x7fa1ec06edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc036000 tx=0x7fa1fc03b040 comp rx=0 tx=0).stop 2024-04-04T06:09:02.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.550+0000 7fa212000640 1 -- 172.21.15.159:0/3223615600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa20c0d0650 msgr2=0x7fa20c150c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:02.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.550+0000 7fa212000640 1 --2- 172.21.15.159:0/3223615600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa20c0d0650 0x7fa20c150c10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800c400 tx=0x7fa1f800c8c0 comp rx=0 tx=0).stop 2024-04-04T06:09:02.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.550+0000 7fa212000640 1 -- 172.21.15.159:0/3223615600 shutdown_connections 2024-04-04T06:09:02.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.550+0000 7fa212000640 1 --2- 172.21.15.159:0/3223615600 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa1ec06c8f0 0x7fa1ec06edb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:02.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.550+0000 7fa212000640 1 --2- 172.21.15.159:0/3223615600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa20c15c390 0x7fa20c14f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:02.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.550+0000 7fa212000640 1 --2- 172.21.15.159:0/3223615600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa20c0d0650 0x7fa20c150c10 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:02.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.550+0000 7fa212000640 1 --2- 172.21.15.159:0/3223615600 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa20c0af830 0x7fa20c0b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:02.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.550+0000 7fa212000640 1 -- 172.21.15.159:0/3223615600 >> 172.21.15.159:0/3223615600 conn(0x7fa20c06b990 msgr2=0x7fa20c073560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:02.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.550+0000 7fa212000640 1 -- 172.21.15.159:0/3223615600 shutdown_connections 2024-04-04T06:09:02.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.550+0000 7fa212000640 1 -- 172.21.15.159:0/3223615600 wait complete. 2024-04-04T06:09:02.561 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-04-04T06:09:00.777798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":166,"acting":166,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243312,"kb_used_data":6332,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731408,"statfs":{"total":773094113280,"available":772844961792,"internally_reserved":0,"allocated":6483968,"data_stored":3359401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.236995"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.571500+0000","last_change":"2024-04-04T06:08:56.571500+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.571500+0000","last_undegraded":"2024-04-04T06:08:56.571500+0000","last_fullsized":"2024-04-04T06:08:56.571500+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.557520+0000","last_change":"2024-04-04T06:08:56.557520+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.557520+0000","last_undegraded":"2024-04-04T06:08:56.557520+0000","last_fullsized":"2024-04-04T06:08:56.557520+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.569551+0000","last_change":"2024-04-04T06:08:56.569551+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.569551+0000","last_undegraded":"2024-04-04T06:08:56.569551+0000","last_fullsized":"2024-04-04T06:08:56.569551+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.566616+0000","last_change":"2024-04-04T06:08:56.566616+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.566616+0000","last_undegraded":"2024-04-04T06:08:56.566616+0000","last_fullsized":"2024-04-04T06:08:56.566616+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.566130+0000","last_change":"2024-04-04T06:08:56.566130+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.566130+0000","last_undegraded":"2024-04-04T06:08:56.566130+0000","last_fullsized":"2024-04-04T06:08:56.566130+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.556089+0000","last_change":"2024-04-04T06:08:56.556089+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.556089+0000","last_undegraded":"2024-04-04T06:08:56.556089+0000","last_fullsized":"2024-04-04T06:08:56.556089+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.557291+0000","last_change":"2024-04-04T06:08:56.557291+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.557291+0000","last_undegraded":"2024-04-04T06:08:56.557291+0000","last_fullsized":"2024-04-04T06:08:56.557291+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.557121+0000","last_change":"2024-04-04T06:08:56.557121+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.557121+0000","last_undegraded":"2024-04-04T06:08:56.557121+0000","last_fullsized":"2024-04-04T06:08:56.557121+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.566615+0000","last_change":"2024-04-04T06:08:56.566615+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.566615+0000","last_undegraded":"2024-04-04T06:08:56.566615+0000","last_fullsized":"2024-04-04T06:08:56.566615+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.563685+0000","last_change":"2024-04-04T06:08:56.563685+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.563685+0000","last_undegraded":"2024-04-04T06:08:56.563685+0000","last_fullsized":"2024-04-04T06:08:56.563685+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.558504+0000","last_change":"2024-04-04T06:08:56.558504+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.558504+0000","last_undegraded":"2024-04-04T06:08:56.558504+0000","last_fullsized":"2024-04-04T06:08:56.558504+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.568936+0000","last_change":"2024-04-04T06:08:56.568936+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.568936+0000","last_undegraded":"2024-04-04T06:08:56.568936+0000","last_fullsized":"2024-04-04T06:08:56.568936+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.568905+0000","last_change":"2024-04-04T06:08:56.568905+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.568905+0000","last_undegraded":"2024-04-04T06:08:56.568905+0000","last_fullsized":"2024-04-04T06:08:56.568905+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.559296+0000","last_change":"2024-04-04T06:08:56.559296+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.559296+0000","last_undegraded":"2024-04-04T06:08:56.559296+0000","last_fullsized":"2024-04-04T06:08:56.559296+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.564535+0000","last_change":"2024-04-04T06:08:56.564535+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.564535+0000","last_undegraded":"2024-04-04T06:08:56.564535+0000","last_fullsized":"2024-04-04T06:08:56.564535+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.560679+0000","last_change":"2024-04-04T06:08:56.560679+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.560679+0000","last_undegraded":"2024-04-04T06:08:56.560679+0000","last_fullsized":"2024-04-04T06:08:56.560679+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.565507+0000","last_change":"2024-04-04T06:08:56.565507+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.565507+0000","last_undegraded":"2024-04-04T06:08:56.565507+0000","last_fullsized":"2024-04-04T06:08:56.565507+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.564719+0000","last_change":"2024-04-04T06:08:56.564719+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.564719+0000","last_undegraded":"2024-04-04T06:08:56.564719+0000","last_fullsized":"2024-04-04T06:08:56.564719+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.571019+0000","last_change":"2024-04-04T06:08:56.571019+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.571019+0000","last_undegraded":"2024-04-04T06:08:56.571019+0000","last_fullsized":"2024-04-04T06:08:56.571019+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.568258+0000","last_change":"2024-04-04T06:08:56.568258+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.568258+0000","last_undegraded":"2024-04-04T06:08:56.568258+0000","last_fullsized":"2024-04-04T06:08:56.568258+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.568844+0000","last_change":"2024-04-04T06:08:56.568844+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.568844+0000","last_undegraded":"2024-04-04T06:08:56.568844+0000","last_fullsized":"2024-04-04T06:08:56.568844+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.555962+0000","last_change":"2024-04-04T06:08:56.555962+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.555962+0000","last_undegraded":"2024-04-04T06:08:56.555962+0000","last_fullsized":"2024-04-04T06:08:56.555962+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.563532+0000","last_change":"2024-04-04T06:08:56.563532+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.563532+0000","last_undegraded":"2024-04-04T06:08:56.563532+0000","last_fullsized":"2024-04-04T06:08:56.563532+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.564880+0000","last_change":"2024-04-04T06:08:56.564880+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.564880+0000","last_undegraded":"2024-04-04T06:08:56.564880+0000","last_fullsized":"2024-04-04T06:08:56.564880+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.563141+0000","last_change":"2024-04-04T06:08:56.563141+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.563141+0000","last_undegraded":"2024-04-04T06:08:56.563141+0000","last_fullsized":"2024-04-04T06:08:56.563141+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.569809+0000","last_change":"2024-04-04T06:08:56.569809+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.569809+0000","last_undegraded":"2024-04-04T06:08:56.569809+0000","last_fullsized":"2024-04-04T06:08:56.569809+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.587031+0000","last_change":"2024-04-04T06:08:56.578928+0000","last_active":"2024-04-04T06:08:56.587031+0000","last_peered":"2024-04-04T06:08:56.587031+0000","last_clean":"2024-04-04T06:08:56.587031+0000","last_became_active":"2024-04-04T06:08:56.578602+0000","last_became_peered":"2024-04-04T06:08:56.578602+0000","last_unstale":"2024-04-04T06:08:56.587031+0000","last_undegraded":"2024-04-04T06:08:56.587031+0000","last_fullsized":"2024-04-04T06:08:56.587031+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:16:20.681483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.556001+0000","last_change":"2024-04-04T06:08:56.556001+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.556001+0000","last_undegraded":"2024-04-04T06:08:56.556001+0000","last_fullsized":"2024-04-04T06:08:56.556001+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.582255+0000","last_change":"2024-04-04T06:08:56.577361+0000","last_active":"2024-04-04T06:08:56.582255+0000","last_peered":"2024-04-04T06:08:56.582255+0000","last_clean":"2024-04-04T06:08:56.582255+0000","last_became_active":"2024-04-04T06:08:56.576619+0000","last_became_peered":"2024-04-04T06:08:56.576619+0000","last_unstale":"2024-04-04T06:08:56.582255+0000","last_undegraded":"2024-04-04T06:08:56.582255+0000","last_fullsized":"2024-04-04T06:08:56.582255+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:28:56.617090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.566480+0000","last_change":"2024-04-04T06:08:56.566480+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.566480+0000","last_undegraded":"2024-04-04T06:08:56.566480+0000","last_fullsized":"2024-04-04T06:08:56.566480+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.586821+0000","last_change":"2024-04-04T06:08:56.577805+0000","last_active":"2024-04-04T06:08:56.586821+0000","last_peered":"2024-04-04T06:08:56.586821+0000","last_clean":"2024-04-04T06:08:56.586821+0000","last_became_active":"2024-04-04T06:08:56.576393+0000","last_became_peered":"2024-04-04T06:08:56.576393+0000","last_unstale":"2024-04-04T06:08:56.586821+0000","last_undegraded":"2024-04-04T06:08:56.586821+0000","last_fullsized":"2024-04-04T06:08:56.586821+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:02:33.508277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.567741+0000","last_change":"2024-04-04T06:08:56.567741+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.567741+0000","last_undegraded":"2024-04-04T06:08:56.567741+0000","last_fullsized":"2024-04-04T06:08:56.567741+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-04T06:08:55.546828+0000","last_change":"2024-04-04T06:08:55.546828+0000","last_active":"2024-04-04T06:08:55.523759+0000","last_peered":"2024-04-04T06:08:55.523759+0000","last_clean":"2024-04-04T06:08:55.523759+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:55.546828+0000","last_undegraded":"2024-04-04T06:08:55.546828+0000","last_fullsized":"2024-04-04T06:08:55.546828+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.560203+0000","last_change":"2024-04-04T06:08:56.560203+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.560203+0000","last_undegraded":"2024-04-04T06:08:56.560203+0000","last_fullsized":"2024-04-04T06:08:56.560203+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-04T06:08:55.554250+0000","last_change":"2024-04-04T06:08:55.554250+0000","last_active":"2024-04-04T06:08:55.523759+0000","last_peered":"2024-04-04T06:08:55.523759+0000","last_clean":"2024-04-04T06:08:55.523759+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:55.554250+0000","last_undegraded":"2024-04-04T06:08:55.554250+0000","last_fullsized":"2024-04-04T06:08:55.554250+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.558303+0000","last_change":"2024-04-04T06:08:56.558303+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.558303+0000","last_undegraded":"2024-04-04T06:08:56.558303+0000","last_fullsized":"2024-04-04T06:08:56.558303+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.587052+0000","last_change":"2024-04-04T06:08:56.579024+0000","last_active":"2024-04-04T06:08:56.587052+0000","last_peered":"2024-04-04T06:08:56.587052+0000","last_clean":"2024-04-04T06:08:56.587052+0000","last_became_active":"2024-04-04T06:08:56.578301+0000","last_became_peered":"2024-04-04T06:08:56.578301+0000","last_unstale":"2024-04-04T06:08:56.587052+0000","last_undegraded":"2024-04-04T06:08:56.587052+0000","last_fullsized":"2024-04-04T06:08:56.587052+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:03:42.180669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.563882+0000","last_change":"2024-04-04T06:08:56.563882+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.563882+0000","last_undegraded":"2024-04-04T06:08:56.563882+0000","last_fullsized":"2024-04-04T06:08:56.563882+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-04T06:08:55.546818+0000","last_change":"2024-04-04T06:08:55.546818+0000","last_active":"2024-04-04T06:08:55.523759+0000","last_peered":"2024-04-04T06:08:55.523759+0000","last_clean":"2024-04-04T06:08:55.523759+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:55.546818+0000","last_undegraded":"2024-04-04T06:08:55.546818+0000","last_fullsized":"2024-04-04T06:08:55.546818+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.558924+0000","last_change":"2024-04-04T06:08:56.558924+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.558924+0000","last_undegraded":"2024-04-04T06:08:56.558924+0000","last_fullsized":"2024-04-04T06:08:56.558924+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-04T06:08:55.553932+0000","last_change":"2024-04-04T06:08:55.553932+0000","last_active":"2024-04-04T06:08:55.523759+0000","last_peered":"2024-04-04T06:08:55.523759+0000","last_clean":"2024-04-04T06:08:55.523759+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:55.553932+0000","last_undegraded":"2024-04-04T06:08:55.553932+0000","last_fullsized":"2024-04-04T06:08:55.553932+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.588221+0000","last_change":"2024-04-04T06:08:56.587466+0000","last_active":"2024-04-04T06:08:56.588221+0000","last_peered":"2024-04-04T06:08:56.588221+0000","last_clean":"2024-04-04T06:08:56.588221+0000","last_became_active":"2024-04-04T06:08:56.587163+0000","last_became_peered":"2024-04-04T06:08:56.587163+0000","last_unstale":"2024-04-04T06:08:56.588221+0000","last_undegraded":"2024-04-04T06:08:56.588221+0000","last_fullsized":"2024-04-04T06:08:56.588221+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:42:24.830758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.569960+0000","last_change":"2024-04-04T06:08:56.569960+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.569960+0000","last_undegraded":"2024-04-04T06:08:56.569960+0000","last_fullsized":"2024-04-04T06:08:56.569960+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.584371+0000","last_change":"2024-04-04T06:08:56.581539+0000","last_active":"2024-04-04T06:08:56.584371+0000","last_peered":"2024-04-04T06:08:56.584371+0000","last_clean":"2024-04-04T06:08:56.584371+0000","last_became_active":"2024-04-04T06:08:56.579490+0000","last_became_peered":"2024-04-04T06:08:56.579490+0000","last_unstale":"2024-04-04T06:08:56.584371+0000","last_undegraded":"2024-04-04T06:08:56.584371+0000","last_fullsized":"2024-04-04T06:08:56.584371+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:50:00.796043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.555714+0000","last_change":"2024-04-04T06:08:56.555714+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.555714+0000","last_undegraded":"2024-04-04T06:08:56.555714+0000","last_fullsized":"2024-04-04T06:08:56.555714+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.583461+0000","last_change":"2024-04-04T06:08:56.582280+0000","last_active":"2024-04-04T06:08:56.583461+0000","last_peered":"2024-04-04T06:08:56.583461+0000","last_clean":"2024-04-04T06:08:56.583461+0000","last_became_active":"2024-04-04T06:08:56.581968+0000","last_became_peered":"2024-04-04T06:08:56.581968+0000","last_unstale":"2024-04-04T06:08:56.583461+0000","last_undegraded":"2024-04-04T06:08:56.583461+0000","last_fullsized":"2024-04-04T06:08:56.583461+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:38:14.124440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.568800+0000","last_change":"2024-04-04T06:08:56.568800+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.568800+0000","last_undegraded":"2024-04-04T06:08:56.568800+0000","last_fullsized":"2024-04-04T06:08:56.568800+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-04T06:08:55.554325+0000","last_change":"2024-04-04T06:08:55.554325+0000","last_active":"2024-04-04T06:08:55.523759+0000","last_peered":"2024-04-04T06:08:55.523759+0000","last_clean":"2024-04-04T06:08:55.523759+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:55.554325+0000","last_undegraded":"2024-04-04T06:08:55.554325+0000","last_fullsized":"2024-04-04T06:08:55.554325+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.579674+0000","last_change":"2024-04-04T06:08:56.575082+0000","last_active":"2024-04-04T06:08:56.579674+0000","last_peered":"2024-04-04T06:08:56.579674+0000","last_clean":"2024-04-04T06:08:56.579674+0000","last_became_active":"2024-04-04T06:08:56.574805+0000","last_became_peered":"2024-04-04T06:08:56.574805+0000","last_unstale":"2024-04-04T06:08:56.579674+0000","last_undegraded":"2024-04-04T06:08:56.579674+0000","last_fullsized":"2024-04-04T06:08:56.579674+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:58:27.216190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.563960+0000","last_change":"2024-04-04T06:08:56.563960+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.563960+0000","last_undegraded":"2024-04-04T06:08:56.563960+0000","last_fullsized":"2024-04-04T06:08:56.563960+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.580524+0000","last_change":"2024-04-04T06:08:56.576046+0000","last_active":"2024-04-04T06:08:56.580524+0000","last_peered":"2024-04-04T06:08:56.580524+0000","last_clean":"2024-04-04T06:08:56.580524+0000","last_became_active":"2024-04-04T06:08:56.575726+0000","last_became_peered":"2024-04-04T06:08:56.575726+0000","last_unstale":"2024-04-04T06:08:56.580524+0000","last_undegraded":"2024-04-04T06:08:56.580524+0000","last_fullsized":"2024-04-04T06:08:56.580524+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:35:45.378741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-04T06:08:55.545600+0000","last_change":"2024-04-04T06:08:55.545600+0000","last_active":"2024-04-04T06:08:55.523759+0000","last_peered":"2024-04-04T06:08:55.523759+0000","last_clean":"2024-04-04T06:08:55.523759+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:55.545600+0000","last_undegraded":"2024-04-04T06:08:55.545600+0000","last_fullsized":"2024-04-04T06:08:55.545600+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.564501+0000","last_change":"2024-04-04T06:08:56.564501+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.564501+0000","last_undegraded":"2024-04-04T06:08:56.564501+0000","last_fullsized":"2024-04-04T06:08:56.564501+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.583872+0000","last_change":"2024-04-04T06:08:56.582998+0000","last_active":"2024-04-04T06:08:56.583872+0000","last_peered":"2024-04-04T06:08:56.583872+0000","last_clean":"2024-04-04T06:08:56.583872+0000","last_became_active":"2024-04-04T06:08:56.582720+0000","last_became_peered":"2024-04-04T06:08:56.582720+0000","last_unstale":"2024-04-04T06:08:56.583872+0000","last_undegraded":"2024-04-04T06:08:56.583872+0000","last_fullsized":"2024-04-04T06:08:56.583872+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:34:48.461720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.566434+0000","last_change":"2024-04-04T06:08:56.566434+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.566434+0000","last_undegraded":"2024-04-04T06:08:56.566434+0000","last_fullsized":"2024-04-04T06:08:56.566434+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.584260+0000","last_change":"2024-04-04T06:08:56.575119+0000","last_active":"2024-04-04T06:08:56.584260+0000","last_peered":"2024-04-04T06:08:56.584260+0000","last_clean":"2024-04-04T06:08:56.584260+0000","last_became_active":"2024-04-04T06:08:56.574832+0000","last_became_peered":"2024-04-04T06:08:56.574832+0000","last_unstale":"2024-04-04T06:08:56.584260+0000","last_undegraded":"2024-04-04T06:08:56.584260+0000","last_fullsized":"2024-04-04T06:08:56.584260+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:24:50.694749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.586722+0000","last_change":"2024-04-04T06:08:56.576721+0000","last_active":"2024-04-04T06:08:56.586722+0000","last_peered":"2024-04-04T06:08:56.586722+0000","last_clean":"2024-04-04T06:08:56.586722+0000","last_became_active":"2024-04-04T06:08:56.576404+0000","last_became_peered":"2024-04-04T06:08:56.576404+0000","last_unstale":"2024-04-04T06:08:56.586722+0000","last_undegraded":"2024-04-04T06:08:56.586722+0000","last_fullsized":"2024-04-04T06:08:56.586722+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:29:08.905489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.556399+0000","last_change":"2024-04-04T06:08:56.556399+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.556399+0000","last_undegraded":"2024-04-04T06:08:56.556399+0000","last_fullsized":"2024-04-04T06:08:56.556399+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-04T06:08:55.545610+0000","last_change":"2024-04-04T06:08:55.545610+0000","last_active":"2024-04-04T06:08:55.523759+0000","last_peered":"2024-04-04T06:08:55.523759+0000","last_clean":"2024-04-04T06:08:55.523759+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:55.545610+0000","last_undegraded":"2024-04-04T06:08:55.545610+0000","last_fullsized":"2024-04-04T06:08:55.545610+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.565011+0000","last_change":"2024-04-04T06:08:56.565011+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.565011+0000","last_undegraded":"2024-04-04T06:08:56.565011+0000","last_fullsized":"2024-04-04T06:08:56.565011+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.586828+0000","last_change":"2024-04-04T06:08:56.577972+0000","last_active":"2024-04-04T06:08:56.586828+0000","last_peered":"2024-04-04T06:08:56.586828+0000","last_clean":"2024-04-04T06:08:56.586828+0000","last_became_active":"2024-04-04T06:08:56.577700+0000","last_became_peered":"2024-04-04T06:08:56.577700+0000","last_unstale":"2024-04-04T06:08:56.586828+0000","last_undegraded":"2024-04-04T06:08:56.586828+0000","last_fullsized":"2024-04-04T06:08:56.586828+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:56.028901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.579173+0000","last_change":"2024-04-04T06:08:56.571770+0000","last_active":"2024-04-04T06:08:56.579173+0000","last_peered":"2024-04-04T06:08:56.579173+0000","last_clean":"2024-04-04T06:08:56.579173+0000","last_became_active":"2024-04-04T06:08:56.571431+0000","last_became_peered":"2024-04-04T06:08:56.571431+0000","last_unstale":"2024-04-04T06:08:56.579173+0000","last_undegraded":"2024-04-04T06:08:56.579173+0000","last_fullsized":"2024-04-04T06:08:56.579173+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:41:25.336576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.563038+0000","last_change":"2024-04-04T06:08:56.563038+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.563038+0000","last_undegraded":"2024-04-04T06:08:56.563038+0000","last_fullsized":"2024-04-04T06:08:56.563038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.581283+0000","last_change":"2024-04-04T06:08:56.574202+0000","last_active":"2024-04-04T06:08:56.581283+0000","last_peered":"2024-04-04T06:08:56.581283+0000","last_clean":"2024-04-04T06:08:56.581283+0000","last_became_active":"2024-04-04T06:08:56.573425+0000","last_became_peered":"2024-04-04T06:08:56.573425+0000","last_unstale":"2024-04-04T06:08:56.581283+0000","last_undegraded":"2024-04-04T06:08:56.581283+0000","last_fullsized":"2024-04-04T06:08:56.581283+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:55:40.756647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.564641+0000","last_change":"2024-04-04T06:08:56.564641+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.564641+0000","last_undegraded":"2024-04-04T06:08:56.564641+0000","last_fullsized":"2024-04-04T06:08:56.564641+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-04T06:08:55.545246+0000","last_change":"2024-04-04T06:08:55.545246+0000","last_active":"2024-04-04T06:08:55.523759+0000","last_peered":"2024-04-04T06:08:55.523759+0000","last_clean":"2024-04-04T06:08:55.523759+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:55.545246+0000","last_undegraded":"2024-04-04T06:08:55.545246+0000","last_fullsized":"2024-04-04T06:08:55.545246+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.581329+0000","last_change":"2024-04-04T06:08:56.574911+0000","last_active":"2024-04-04T06:08:56.581329+0000","last_peered":"2024-04-04T06:08:56.581329+0000","last_clean":"2024-04-04T06:08:56.581329+0000","last_became_active":"2024-04-04T06:08:56.573037+0000","last_became_peered":"2024-04-04T06:08:56.573037+0000","last_unstale":"2024-04-04T06:08:56.581329+0000","last_undegraded":"2024-04-04T06:08:56.581329+0000","last_fullsized":"2024-04-04T06:08:56.581329+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:52:20.881931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:08:56.537805+0000","last_change":"2024-04-04T06:08:56.537805+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.537805+0000","last_undegraded":"2024-04-04T06:08:56.537805+0000","last_fullsized":"2024-04-04T06:08:56.537805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.586930+0000","last_change":"2024-04-04T06:08:56.578575+0000","last_active":"2024-04-04T06:08:56.586930+0000","last_peered":"2024-04-04T06:08:56.586930+0000","last_clean":"2024-04-04T06:08:56.586930+0000","last_became_active":"2024-04-04T06:08:56.578232+0000","last_became_peered":"2024-04-04T06:08:56.578232+0000","last_unstale":"2024-04-04T06:08:56.586930+0000","last_undegraded":"2024-04-04T06:08:56.586930+0000","last_fullsized":"2024-04-04T06:08:56.586930+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:25:43.268590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.554392+0000","last_change":"2024-04-04T06:08:56.554392+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.554392+0000","last_undegraded":"2024-04-04T06:08:56.554392+0000","last_fullsized":"2024-04-04T06:08:56.554392+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.578545+0000","last_change":"2024-04-04T06:08:56.573941+0000","last_active":"2024-04-04T06:08:56.578545+0000","last_peered":"2024-04-04T06:08:56.578545+0000","last_clean":"2024-04-04T06:08:56.578545+0000","last_became_active":"2024-04-04T06:08:56.573700+0000","last_became_peered":"2024-04-04T06:08:56.573700+0000","last_unstale":"2024-04-04T06:08:56.578545+0000","last_undegraded":"2024-04-04T06:08:56.578545+0000","last_fullsized":"2024-04-04T06:08:56.578545+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:17:21.845820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.567298+0000","last_change":"2024-04-04T06:08:56.567298+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.567298+0000","last_undegraded":"2024-04-04T06:08:56.567298+0000","last_fullsized":"2024-04-04T06:08:56.567298+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-04T06:08:55.545546+0000","last_change":"2024-04-04T06:08:55.545546+0000","last_active":"2024-04-04T06:08:55.523759+0000","last_peered":"2024-04-04T06:08:55.523759+0000","last_clean":"2024-04-04T06:08:55.523759+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:55.545546+0000","last_undegraded":"2024-04-04T06:08:55.545546+0000","last_fullsized":"2024-04-04T06:08:55.545546+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.555361+0000","last_change":"2024-04-04T06:08:56.555361+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.555361+0000","last_undegraded":"2024-04-04T06:08:56.555361+0000","last_fullsized":"2024-04-04T06:08:56.555361+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.574765+0000","last_change":"2024-04-04T06:08:56.569703+0000","last_active":"2024-04-04T06:08:56.574765+0000","last_peered":"2024-04-04T06:08:56.574765+0000","last_clean":"2024-04-04T06:08:56.574765+0000","last_became_active":"2024-04-04T06:08:56.569433+0000","last_became_peered":"2024-04-04T06:08:56.569433+0000","last_unstale":"2024-04-04T06:08:56.574765+0000","last_undegraded":"2024-04-04T06:08:56.574765+0000","last_fullsized":"2024-04-04T06:08:56.574765+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:20:28.512324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.563309+0000","last_change":"2024-04-04T06:08:56.563309+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.563309+0000","last_undegraded":"2024-04-04T06:08:56.563309+0000","last_fullsized":"2024-04-04T06:08:56.563309+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2024-04-04T06:08:55.553867+0000","last_change":"2024-04-04T06:08:55.553867+0000","last_active":"2024-04-04T06:08:55.523759+0000","last_peered":"2024-04-04T06:08:55.523759+0000","last_clean":"2024-04-04T06:08:55.523759+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:55.553867+0000","last_undegraded":"2024-04-04T06:08:55.553867+0000","last_fullsized":"2024-04-04T06:08:55.553867+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.554390+0000","last_change":"2024-04-04T06:08:56.554390+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.554390+0000","last_undegraded":"2024-04-04T06:08:56.554390+0000","last_fullsized":"2024-04-04T06:08:56.554390+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.581408+0000","last_change":"2024-04-04T06:08:56.576604+0000","last_active":"2024-04-04T06:08:56.581408+0000","last_peered":"2024-04-04T06:08:56.581408+0000","last_clean":"2024-04-04T06:08:56.581408+0000","last_became_active":"2024-04-04T06:08:56.576310+0000","last_became_peered":"2024-04-04T06:08:56.576310+0000","last_unstale":"2024-04-04T06:08:56.581408+0000","last_undegraded":"2024-04-04T06:08:56.581408+0000","last_fullsized":"2024-04-04T06:08:56.581408+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:00:29.103633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.555742+0000","last_change":"2024-04-04T06:08:56.555742+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.555742+0000","last_undegraded":"2024-04-04T06:08:56.555742+0000","last_fullsized":"2024-04-04T06:08:56.555742+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.575097+0000","last_change":"2024-04-04T06:08:56.568855+0000","last_active":"2024-04-04T06:08:56.575097+0000","last_peered":"2024-04-04T06:08:56.575097+0000","last_clean":"2024-04-04T06:08:56.575097+0000","last_became_active":"2024-04-04T06:08:56.568608+0000","last_became_peered":"2024-04-04T06:08:56.568608+0000","last_unstale":"2024-04-04T06:08:56.575097+0000","last_undegraded":"2024-04-04T06:08:56.575097+0000","last_fullsized":"2024-04-04T06:08:56.575097+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:42:11.384223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-04-04T06:08:56.555515+0000","last_change":"2024-04-04T06:08:56.555515+0000","last_active":"2024-04-04T06:08:56.537805+0000","last_peered":"2024-04-04T06:08:56.537805+0000","last_clean":"2024-04-04T06:08:56.537805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:08:56.555515+0000","last_undegraded":"2024-04-04T06:08:56.555515+0000","last_fullsized":"2024-04-04T06:08:56.555515+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.582081+0000","last_change":"2024-04-04T06:08:56.578712+0000","last_active":"2024-04-04T06:08:56.582081+0000","last_peered":"2024-04-04T06:08:56.582081+0000","last_clean":"2024-04-04T06:08:56.582081+0000","last_became_active":"2024-04-04T06:08:56.578391+0000","last_became_peered":"2024-04-04T06:08:56.578391+0000","last_unstale":"2024-04-04T06:08:56.582081+0000","last_undegraded":"2024-04-04T06:08:56.582081+0000","last_fullsized":"2024-04-04T06:08:56.582081+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:44:31.471700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":233,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-04T06:08:56.574641+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:08:56.574641+0000","last_peered":"2024-04-04T06:08:56.574641+0000","last_clean":"2024-04-04T06:08:56.574641+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:08:56.574641+0000","last_undegraded":"2024-04-04T06:08:56.574641+0000","last_fullsized":"2024-04-04T06:08:56.574641+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":100,"acting":100,"num_store_stats":6},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509481,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1257472,"data_stored":863372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509483,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":663552,"data_stored":273004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509484,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":273004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509483,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":675840,"data_stored":279271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":273004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509484,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1257472,"data_stored":863372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542187,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":651264,"data_stored":267187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542185,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":651264,"data_stored":267187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:09:02.565 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:09:02.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.823+0000 7f2df4c00640 1 Processor -- start 2024-04-04T06:09:02.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.823+0000 7f2df4c00640 1 -- start start 2024-04-04T06:09:02.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.824+0000 7f2df4c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df00d0d60 0x7f2df0073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:02.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.824+0000 7f2df4c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2df015fe80 con 0x7f2df015c390 2024-04-04T06:09:02.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.824+0000 7f2df4c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2df00ad120 con 0x7f2df00af830 2024-04-04T06:09:02.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.824+0000 7f2df4c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2df015c770 con 0x7f2df00d0d60 2024-04-04T06:09:02.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.825+0000 7f2dede00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f2df015c390 0x7f2df0160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40324/0 (socket says 172.21.15.159:40324) 2024-04-04T06:09:02.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.825+0000 7f2dede00640 1 -- 172.21.15.159:0/3828209742 learned_addr learned my addr 172.21.15.159:0/3828209742 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:09:02.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.825+0000 7f2dee800640 1 --2- 172.21.15.159:0/3828209742 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df00d0d60 0x7f2df0073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:02.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.825+0000 7f2deca00640 1 -- 172.21.15.159:0/3828209742 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 300823145 0 0) 0x7f2df015fe80 con 0x7f2df015c390 2024-04-04T06:09:02.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.825+0000 7f2deca00640 1 -- 172.21.15.159:0/3828209742 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2dd4003520 con 0x7f2df015c390 2024-04-04T06:09:02.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.826+0000 7f2deca00640 1 -- 172.21.15.159:0/3828209742 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2553144196 0 0) 0x7f2dd4003520 con 0x7f2df015c390 2024-04-04T06:09:02.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.826+0000 7f2deca00640 1 -- 172.21.15.159:0/3828209742 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df00d0d60 msgr2=0x7f2df0073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:02.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.826+0000 7f2deca00640 1 --2- 172.21.15.159:0/3828209742 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df00d0d60 0x7f2df0073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:02.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.826+0000 7f2deca00640 1 -- 172.21.15.159:0/3828209742 >> v1:172.21.15.178:6789/0 conn(0x7f2df00af830 legacy=0x7f2df00d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:02.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.826+0000 7f2deca00640 1 -- 172.21.15.159:0/3828209742 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df00ad400 con 0x7f2df015c390 2024-04-04T06:09:02.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.826+0000 7f2deca00640 1 -- 172.21.15.159:0/3828209742 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f2de0001670 con 0x7f2df015c390 2024-04-04T06:09:02.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.826+0000 7f2deca00640 1 -- 172.21.15.159:0/3828209742 >> v1:172.21.15.159:6789/0 conn(0x7f2df015c390 legacy=0x7f2df0160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:02.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.826+0000 7f2deca00640 1 --2- 172.21.15.159:0/3828209742 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dd40039e0 0x7f2dd4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:02.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.826+0000 7f2deca00640 1 --2- 172.21.15.159:0/3828209742 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df00d0d60 0x7f2dd4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:02.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.826+0000 7f2deca00640 1 --2- 172.21.15.159:0/3828209742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dd4004d40 0x7f2dd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:02.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.826+0000 7f2deca00640 1 -- 172.21.15.159:0/3828209742 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd400b9c0 con 0x7f2dd40039e0 2024-04-04T06:09:02.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.826+0000 7f2deca00640 1 -- 172.21.15.159:0/3828209742 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd400bb70 con 0x7f2dd4004d40 2024-04-04T06:09:02.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.826+0000 7f2deca00640 1 -- 172.21.15.159:0/3828209742 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2dd400bd70 con 0x7f2df00d0d60 2024-04-04T06:09:02.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.826+0000 7f2dede00640 1 --2- 172.21.15.159:0/3828209742 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dd40039e0 0x7f2dd4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:02.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.827+0000 7f2ded400640 1 --2- 172.21.15.159:0/3828209742 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df00d0d60 0x7f2dd4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:02.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.827+0000 7f2dee800640 1 --2- 172.21.15.159:0/3828209742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dd4004d40 0x7f2dd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:02.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.827+0000 7f2dede00640 1 -- 172.21.15.159:0/3828209742 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df00d0d60 msgr2=0x7f2dd4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:02.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.827+0000 7f2dede00640 1 --2- 172.21.15.159:0/3828209742 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df00d0d60 0x7f2dd4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:02.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.827+0000 7f2dede00640 1 -- 172.21.15.159:0/3828209742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dd4004d40 msgr2=0x7f2dd40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:02.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.827+0000 7f2dede00640 1 --2- 172.21.15.159:0/3828209742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dd4004d40 0x7f2dd40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:02.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.827+0000 7f2dede00640 1 -- 172.21.15.159:0/3828209742 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd4003520 con 0x7f2dd40039e0 2024-04-04T06:09:02.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.827+0000 7f2ded400640 1 --2- 172.21.15.159:0/3828209742 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df00d0d60 0x7f2dd4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:02.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.827+0000 7f2dee800640 1 --2- 172.21.15.159:0/3828209742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dd4004d40 0x7f2dd40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:02.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.828+0000 7f2dede00640 1 --2- 172.21.15.159:0/3828209742 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dd40039e0 0x7f2dd4003dc0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2de000c240 tx=0x7f2dd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80342993c0844c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:02.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.828+0000 7f2deca00640 1 -- 172.21.15.159:0/3828209742 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0007d30 con 0x7f2dd40039e0 2024-04-04T06:09:02.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.828+0000 7f2deca00640 1 -- 172.21.15.159:0/3828209742 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2de0030070 con 0x7f2dd40039e0 2024-04-04T06:09:02.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.828+0000 7f2deca00640 1 -- 172.21.15.159:0/3828209742 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0005d30 con 0x7f2dd40039e0 2024-04-04T06:09:02.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.829+0000 7f2df4c00640 1 -- 172.21.15.159:0/3828209742 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dd40039e0 msgr2=0x7f2dd4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:02.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.829+0000 7f2df4c00640 1 --2- 172.21.15.159:0/3828209742 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dd40039e0 0x7f2dd4003dc0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2de000c240 tx=0x7f2dd4005830 comp rx=0 tx=0).stop 2024-04-04T06:09:02.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.829+0000 7f2dee800640 1 -- 172.21.15.159:0/3828209742 reap_dead start 2024-04-04T06:09:02.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.829+0000 7f2df4c00640 1 -- 172.21.15.159:0/3828209742 shutdown_connections 2024-04-04T06:09:02.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.829+0000 7f2df4c00640 1 -- 172.21.15.159:0/3828209742 >> 172.21.15.159:0/3828209742 conn(0x7f2df006b990 msgr2=0x7f2df006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:02.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.829+0000 7f2df4c00640 1 -- 172.21.15.159:0/3828209742 shutdown_connections 2024-04-04T06:09:02.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.829+0000 7f2df4c00640 1 -- 172.21.15.159:0/3828209742 wait complete. 2024-04-04T06:09:02.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.831+0000 7f2df4c00640 1 Processor -- start 2024-04-04T06:09:02.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.831+0000 7f2df4c00640 1 -- start start 2024-04-04T06:09:02.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.832+0000 7f2df4c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dd40039e0 0x7f2df015f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:02.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.832+0000 7f2df4c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df0152790 0x7f2df01502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:02.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.832+0000 7f2df4c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2df0152070 0x7f2df015d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:02.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.832+0000 7f2df4c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0160d50 con 0x7f2df0152070 2024-04-04T06:09:02.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.832+0000 7f2df4c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0160120 con 0x7f2dd40039e0 2024-04-04T06:09:02.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.832+0000 7f2df4c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2df00cf980 con 0x7f2df0152790 2024-04-04T06:09:02.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.832+0000 7f2dee800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2df0152070 0x7f2df015d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:02.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.832+0000 7f2dee800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2df0152070 0x7f2df015d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37398/0 (socket says 172.21.15.159:37398) 2024-04-04T06:09:02.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.832+0000 7f2dee800640 1 -- 172.21.15.159:0/3198102333 learned_addr learned my addr 172.21.15.159:0/3198102333 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:02.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.833+0000 7f2dede00640 1 --2- 172.21.15.159:0/3198102333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dd40039e0 0x7f2df015f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:02.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.833+0000 7f2ded400640 1 --2- 172.21.15.159:0/3198102333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df0152790 0x7f2df01502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:02.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.833+0000 7f2dee800640 1 -- 172.21.15.159:0/3198102333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df0152790 msgr2=0x7f2df01502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:02.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.833+0000 7f2dee800640 1 --2- 172.21.15.159:0/3198102333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df0152790 0x7f2df01502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:02.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.833+0000 7f2dee800640 1 -- 172.21.15.159:0/3198102333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dd40039e0 msgr2=0x7f2df015f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:02.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.833+0000 7f2dee800640 1 --2- 172.21.15.159:0/3198102333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dd40039e0 0x7f2df015f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:02.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.833+0000 7f2dee800640 1 -- 172.21.15.159:0/3198102333 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df01507f0 con 0x7f2df0152070 2024-04-04T06:09:02.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.833+0000 7f2dee800640 1 --2- 172.21.15.159:0/3198102333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2df0152070 0x7f2df015d620 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2df00b5470 tx=0x7f2de800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:02.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.833+0000 7f2de6000640 1 -- 172.21.15.159:0/3198102333 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de801c070 con 0x7f2df0152070 2024-04-04T06:09:02.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.833+0000 7f2de6000640 1 -- 172.21.15.159:0/3198102333 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2de8017400 con 0x7f2df0152070 2024-04-04T06:09:02.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.833+0000 7f2df4c00640 1 -- 172.21.15.159:0/3198102333 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df0152ce0 con 0x7f2df0152070 2024-04-04T06:09:02.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.834+0000 7f2de6000640 1 -- 172.21.15.159:0/3198102333 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de80177d0 con 0x7f2df0152070 2024-04-04T06:09:02.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.834+0000 7f2df4c00640 1 -- 172.21.15.159:0/3198102333 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df0151590 con 0x7f2df0152070 2024-04-04T06:09:02.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.835+0000 7f2df4c00640 1 -- 172.21.15.159:0/3198102333 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df00685c0 con 0x7f2df0152070 2024-04-04T06:09:02.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.836+0000 7f2de6000640 1 -- 172.21.15.159:0/3198102333 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2de8005d10 con 0x7f2df0152070 2024-04-04T06:09:02.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.837+0000 7f2de6000640 1 --2- 172.21.15.159:0/3198102333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2dd006c8f0 0x7f2dd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:02.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.837+0000 7f2de6000640 1 -- 172.21.15.159:0/3198102333 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f2de808bda0 con 0x7f2df0152070 2024-04-04T06:09:02.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.838+0000 7f2dede00640 1 --2- 172.21.15.159:0/3198102333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2dd006c8f0 0x7f2dd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:02.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.839+0000 7f2dede00640 1 --2- 172.21.15.159:0/3198102333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2dd006c8f0 0x7f2dd006edb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2de0034000 tx=0x7f2de0009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:02.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:02.842+0000 7f2de6000640 1 -- 172.21.15.159:0/3198102333 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2de8012070 con 0x7f2df0152070 2024-04-04T06:09:03.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.061+0000 7f2df4c00640 1 -- 172.21.15.159:0/3198102333 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2df01578e0 con 0x7f2dd006c8f0 2024-04-04T06:09:03.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.080+0000 7f2de6000640 1 -- 172.21.15.159:0/3198102333 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251955 (secure 0 0 0) 0x7f2df01578e0 con 0x7f2dd006c8f0 2024-04-04T06:09:03.077 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:09:03.079 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:09:03.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.085+0000 7f2df4c00640 1 -- 172.21.15.159:0/3198102333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2dd006c8f0 msgr2=0x7f2dd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:03.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.085+0000 7f2df4c00640 1 --2- 172.21.15.159:0/3198102333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2dd006c8f0 0x7f2dd006edb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2de0034000 tx=0x7f2de0009030 comp rx=0 tx=0).stop 2024-04-04T06:09:03.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.085+0000 7f2df4c00640 1 -- 172.21.15.159:0/3198102333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2df0152070 msgr2=0x7f2df015d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:03.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.085+0000 7f2df4c00640 1 --2- 172.21.15.159:0/3198102333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2df0152070 0x7f2df015d620 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2df00b5470 tx=0x7f2de800f4a0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.085+0000 7f2df4c00640 1 -- 172.21.15.159:0/3198102333 shutdown_connections 2024-04-04T06:09:03.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.085+0000 7f2df4c00640 1 --2- 172.21.15.159:0/3198102333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2df0152070 0x7f2df015d620 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.085+0000 7f2df4c00640 1 --2- 172.21.15.159:0/3198102333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2df0152790 0x7f2df01502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.085+0000 7f2df4c00640 1 --2- 172.21.15.159:0/3198102333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2dd006c8f0 0x7f2dd006edb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.085+0000 7f2df4c00640 1 --2- 172.21.15.159:0/3198102333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2dd40039e0 0x7f2df015f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.085+0000 7f2df4c00640 1 -- 172.21.15.159:0/3198102333 >> 172.21.15.159:0/3198102333 conn(0x7f2df006b990 msgr2=0x7f2df0156c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:03.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.085+0000 7f2df4c00640 1 -- 172.21.15.159:0/3198102333 shutdown_connections 2024-04-04T06:09:03.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.085+0000 7f2df4c00640 1 -- 172.21.15.159:0/3198102333 wait complete. 2024-04-04T06:09:03.098 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-04-04T06:09:02.778415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":251996,"kb_used_data":6696,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754722724,"statfs":{"total":773094113280,"available":772836069376,"internally_reserved":0,"allocated":6856704,"data_stored":3524324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.231986"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.587764+0000","last_change":"2024-04-04T06:08:57.592232+0000","last_active":"2024-04-04T06:08:58.587764+0000","last_peered":"2024-04-04T06:08:58.587764+0000","last_clean":"2024-04-04T06:08:58.587764+0000","last_became_active":"2024-04-04T06:08:57.591849+0000","last_became_peered":"2024-04-04T06:08:57.591849+0000","last_unstale":"2024-04-04T06:08:58.587764+0000","last_undegraded":"2024-04-04T06:08:58.587764+0000","last_fullsized":"2024-04-04T06:08:58.587764+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:55:41.770693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110894+0000","last_change":"2024-04-04T06:08:57.606120+0000","last_active":"2024-04-04T06:08:59.110894+0000","last_peered":"2024-04-04T06:08:59.110894+0000","last_clean":"2024-04-04T06:08:59.110894+0000","last_became_active":"2024-04-04T06:08:57.605844+0000","last_became_peered":"2024-04-04T06:08:57.605844+0000","last_unstale":"2024-04-04T06:08:59.110894+0000","last_undegraded":"2024-04-04T06:08:59.110894+0000","last_fullsized":"2024-04-04T06:08:59.110894+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:07:04.193264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.352020+0000","last_change":"2024-04-04T06:08:57.604547+0000","last_active":"2024-04-04T06:08:59.352020+0000","last_peered":"2024-04-04T06:08:59.352020+0000","last_clean":"2024-04-04T06:08:59.352020+0000","last_became_active":"2024-04-04T06:08:57.604225+0000","last_became_peered":"2024-04-04T06:08:57.604225+0000","last_unstale":"2024-04-04T06:08:59.352020+0000","last_undegraded":"2024-04-04T06:08:59.352020+0000","last_fullsized":"2024-04-04T06:08:59.352020+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:45:43.089485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592791+0000","last_change":"2024-04-04T06:08:57.593370+0000","last_active":"2024-04-04T06:08:58.592791+0000","last_peered":"2024-04-04T06:08:58.592791+0000","last_clean":"2024-04-04T06:08:58.592791+0000","last_became_active":"2024-04-04T06:08:57.591455+0000","last_became_peered":"2024-04-04T06:08:57.591455+0000","last_unstale":"2024-04-04T06:08:58.592791+0000","last_undegraded":"2024-04-04T06:08:58.592791+0000","last_fullsized":"2024-04-04T06:08:58.592791+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:39:56.927343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.586778+0000","last_change":"2024-04-04T06:08:57.602481+0000","last_active":"2024-04-04T06:08:58.586778+0000","last_peered":"2024-04-04T06:08:58.586778+0000","last_clean":"2024-04-04T06:08:58.586778+0000","last_became_active":"2024-04-04T06:08:57.599211+0000","last_became_peered":"2024-04-04T06:08:57.599211+0000","last_unstale":"2024-04-04T06:08:58.586778+0000","last_undegraded":"2024-04-04T06:08:58.586778+0000","last_fullsized":"2024-04-04T06:08:58.586778+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:25:32.291869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604666+0000","last_change":"2024-04-04T06:08:57.614185+0000","last_active":"2024-04-04T06:08:58.604666+0000","last_peered":"2024-04-04T06:08:58.604666+0000","last_clean":"2024-04-04T06:08:58.604666+0000","last_became_active":"2024-04-04T06:08:57.611970+0000","last_became_peered":"2024-04-04T06:08:57.611970+0000","last_unstale":"2024-04-04T06:08:58.604666+0000","last_undegraded":"2024-04-04T06:08:58.604666+0000","last_fullsized":"2024-04-04T06:08:58.604666+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:47:02.001001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399802+0000","last_change":"2024-04-04T06:08:57.596174+0000","last_active":"2024-04-04T06:08:59.399802+0000","last_peered":"2024-04-04T06:08:59.399802+0000","last_clean":"2024-04-04T06:08:59.399802+0000","last_became_active":"2024-04-04T06:08:57.595899+0000","last_became_peered":"2024-04-04T06:08:57.595899+0000","last_unstale":"2024-04-04T06:08:59.399802+0000","last_undegraded":"2024-04-04T06:08:59.399802+0000","last_fullsized":"2024-04-04T06:08:59.399802+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:41:26.350622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110048+0000","last_change":"2024-04-04T06:08:57.605001+0000","last_active":"2024-04-04T06:08:59.110048+0000","last_peered":"2024-04-04T06:08:59.110048+0000","last_clean":"2024-04-04T06:08:59.110048+0000","last_became_active":"2024-04-04T06:08:57.604608+0000","last_became_peered":"2024-04-04T06:08:57.604608+0000","last_unstale":"2024-04-04T06:08:59.110048+0000","last_undegraded":"2024-04-04T06:08:59.110048+0000","last_fullsized":"2024-04-04T06:08:59.110048+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:16:53.808572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.400387+0000","last_change":"2024-04-04T06:08:57.614647+0000","last_active":"2024-04-04T06:08:59.400387+0000","last_peered":"2024-04-04T06:08:59.400387+0000","last_clean":"2024-04-04T06:08:59.400387+0000","last_became_active":"2024-04-04T06:08:57.614442+0000","last_became_peered":"2024-04-04T06:08:57.614442+0000","last_unstale":"2024-04-04T06:08:59.400387+0000","last_undegraded":"2024-04-04T06:08:59.400387+0000","last_fullsized":"2024-04-04T06:08:59.400387+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:28:57.631136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399675+0000","last_change":"2024-04-04T06:08:57.597330+0000","last_active":"2024-04-04T06:08:59.399675+0000","last_peered":"2024-04-04T06:08:59.399675+0000","last_clean":"2024-04-04T06:08:59.399675+0000","last_became_active":"2024-04-04T06:08:57.596962+0000","last_became_peered":"2024-04-04T06:08:57.596962+0000","last_unstale":"2024-04-04T06:08:59.399675+0000","last_undegraded":"2024-04-04T06:08:59.399675+0000","last_fullsized":"2024-04-04T06:08:59.399675+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:25:44.282636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399587+0000","last_change":"2024-04-04T06:08:57.604574+0000","last_active":"2024-04-04T06:08:59.399587+0000","last_peered":"2024-04-04T06:08:59.399587+0000","last_clean":"2024-04-04T06:08:59.399587+0000","last_became_active":"2024-04-04T06:08:57.603977+0000","last_became_peered":"2024-04-04T06:08:57.603977+0000","last_unstale":"2024-04-04T06:08:59.399587+0000","last_undegraded":"2024-04-04T06:08:59.399587+0000","last_fullsized":"2024-04-04T06:08:59.399587+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:57.042947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592871+0000","last_change":"2024-04-04T06:08:57.596645+0000","last_active":"2024-04-04T06:08:58.592871+0000","last_peered":"2024-04-04T06:08:58.592871+0000","last_clean":"2024-04-04T06:08:58.592871+0000","last_became_active":"2024-04-04T06:08:57.596379+0000","last_became_peered":"2024-04-04T06:08:57.596379+0000","last_unstale":"2024-04-04T06:08:58.592871+0000","last_undegraded":"2024-04-04T06:08:58.592871+0000","last_fullsized":"2024-04-04T06:08:58.592871+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:47:02.001001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.593457+0000","last_change":"2024-04-04T06:08:57.618024+0000","last_active":"2024-04-04T06:08:58.593457+0000","last_peered":"2024-04-04T06:08:58.593457+0000","last_clean":"2024-04-04T06:08:58.593457+0000","last_became_active":"2024-04-04T06:08:57.617674+0000","last_became_peered":"2024-04-04T06:08:57.617674+0000","last_unstale":"2024-04-04T06:08:58.593457+0000","last_undegraded":"2024-04-04T06:08:58.593457+0000","last_fullsized":"2024-04-04T06:08:58.593457+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:09:46.418705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399602+0000","last_change":"2024-04-04T06:08:57.592018+0000","last_active":"2024-04-04T06:08:59.399602+0000","last_peered":"2024-04-04T06:08:59.399602+0000","last_clean":"2024-04-04T06:08:59.399602+0000","last_became_active":"2024-04-04T06:08:57.591670+0000","last_became_peered":"2024-04-04T06:08:57.591670+0000","last_unstale":"2024-04-04T06:08:59.399602+0000","last_undegraded":"2024-04-04T06:08:59.399602+0000","last_fullsized":"2024-04-04T06:08:59.399602+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:42:12.398269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.400126+0000","last_change":"2024-04-04T06:08:57.589250+0000","last_active":"2024-04-04T06:08:59.400126+0000","last_peered":"2024-04-04T06:08:59.400126+0000","last_clean":"2024-04-04T06:08:59.400126+0000","last_became_active":"2024-04-04T06:08:57.588786+0000","last_became_peered":"2024-04-04T06:08:57.588786+0000","last_unstale":"2024-04-04T06:08:59.400126+0000","last_undegraded":"2024-04-04T06:08:59.400126+0000","last_fullsized":"2024-04-04T06:08:59.400126+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:58:28.230236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604781+0000","last_change":"2024-04-04T06:08:57.611129+0000","last_active":"2024-04-04T06:08:58.604781+0000","last_peered":"2024-04-04T06:08:58.604781+0000","last_clean":"2024-04-04T06:08:58.604781+0000","last_became_active":"2024-04-04T06:08:57.610825+0000","last_became_peered":"2024-04-04T06:08:57.610825+0000","last_unstale":"2024-04-04T06:08:58.604781+0000","last_undegraded":"2024-04-04T06:08:58.604781+0000","last_fullsized":"2024-04-04T06:08:58.604781+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:25:02.372768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.111430+0000","last_change":"2024-04-04T06:08:57.606441+0000","last_active":"2024-04-04T06:08:59.111430+0000","last_peered":"2024-04-04T06:08:59.111430+0000","last_clean":"2024-04-04T06:08:59.111430+0000","last_became_active":"2024-04-04T06:08:57.606119+0000","last_became_peered":"2024-04-04T06:08:57.606119+0000","last_unstale":"2024-04-04T06:08:59.111430+0000","last_undegraded":"2024-04-04T06:08:59.111430+0000","last_fullsized":"2024-04-04T06:08:59.111430+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:42:55.934487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351721+0000","last_change":"2024-04-04T06:08:57.617716+0000","last_active":"2024-04-04T06:08:59.351721+0000","last_peered":"2024-04-04T06:08:59.351721+0000","last_clean":"2024-04-04T06:08:59.351721+0000","last_became_active":"2024-04-04T06:08:57.617385+0000","last_became_peered":"2024-04-04T06:08:57.617385+0000","last_unstale":"2024-04-04T06:08:59.351721+0000","last_undegraded":"2024-04-04T06:08:59.351721+0000","last_fullsized":"2024-04-04T06:08:59.351721+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:43:00.036899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.587158+0000","last_change":"2024-04-04T06:08:57.593659+0000","last_active":"2024-04-04T06:08:58.587158+0000","last_peered":"2024-04-04T06:08:58.587158+0000","last_clean":"2024-04-04T06:08:58.587158+0000","last_became_active":"2024-04-04T06:08:57.593382+0000","last_became_peered":"2024-04-04T06:08:57.593382+0000","last_unstale":"2024-04-04T06:08:58.587158+0000","last_undegraded":"2024-04-04T06:08:58.587158+0000","last_fullsized":"2024-04-04T06:08:58.587158+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:11:49.907094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110697+0000","last_change":"2024-04-04T06:08:57.619302+0000","last_active":"2024-04-04T06:08:59.110697+0000","last_peered":"2024-04-04T06:08:59.110697+0000","last_clean":"2024-04-04T06:08:59.110697+0000","last_became_active":"2024-04-04T06:08:57.618701+0000","last_became_peered":"2024-04-04T06:08:57.618701+0000","last_unstale":"2024-04-04T06:08:59.110697+0000","last_undegraded":"2024-04-04T06:08:59.110697+0000","last_fullsized":"2024-04-04T06:08:59.110697+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:29:01.526400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351752+0000","last_change":"2024-04-04T06:08:57.602399+0000","last_active":"2024-04-04T06:08:59.351752+0000","last_peered":"2024-04-04T06:08:59.351752+0000","last_clean":"2024-04-04T06:08:59.351752+0000","last_became_active":"2024-04-04T06:08:57.602074+0000","last_became_peered":"2024-04-04T06:08:57.602074+0000","last_unstale":"2024-04-04T06:08:59.351752+0000","last_undegraded":"2024-04-04T06:08:59.351752+0000","last_fullsized":"2024-04-04T06:08:59.351752+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:38:40.159820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110320+0000","last_change":"2024-04-04T06:08:57.603092+0000","last_active":"2024-04-04T06:08:59.110320+0000","last_peered":"2024-04-04T06:08:59.110320+0000","last_clean":"2024-04-04T06:08:59.110320+0000","last_became_active":"2024-04-04T06:08:57.602639+0000","last_became_peered":"2024-04-04T06:08:57.602639+0000","last_unstale":"2024-04-04T06:08:59.110320+0000","last_undegraded":"2024-04-04T06:08:59.110320+0000","last_fullsized":"2024-04-04T06:08:59.110320+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:35:44.572604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351023+0000","last_change":"2024-04-04T06:08:57.608928+0000","last_active":"2024-04-04T06:08:59.351023+0000","last_peered":"2024-04-04T06:08:59.351023+0000","last_clean":"2024-04-04T06:08:59.351023+0000","last_became_active":"2024-04-04T06:08:57.608146+0000","last_became_peered":"2024-04-04T06:08:57.608146+0000","last_unstale":"2024-04-04T06:08:59.351023+0000","last_undegraded":"2024-04-04T06:08:59.351023+0000","last_fullsized":"2024-04-04T06:08:59.351023+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:12:26.107777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.590550+0000","last_change":"2024-04-04T06:08:57.617880+0000","last_active":"2024-04-04T06:08:58.590550+0000","last_peered":"2024-04-04T06:08:58.590550+0000","last_clean":"2024-04-04T06:08:58.590550+0000","last_became_active":"2024-04-04T06:08:57.617508+0000","last_became_peered":"2024-04-04T06:08:57.617508+0000","last_unstale":"2024-04-04T06:08:58.590550+0000","last_undegraded":"2024-04-04T06:08:58.590550+0000","last_fullsized":"2024-04-04T06:08:58.590550+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:08:22.262253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.587766+0000","last_change":"2024-04-04T06:08:57.597236+0000","last_active":"2024-04-04T06:08:58.587766+0000","last_peered":"2024-04-04T06:08:58.587766+0000","last_clean":"2024-04-04T06:08:58.587766+0000","last_became_active":"2024-04-04T06:08:57.596931+0000","last_became_peered":"2024-04-04T06:08:57.596931+0000","last_unstale":"2024-04-04T06:08:58.587766+0000","last_undegraded":"2024-04-04T06:08:58.587766+0000","last_fullsized":"2024-04-04T06:08:58.587766+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:20:29.526370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592533+0000","last_change":"2024-04-04T06:08:57.592246+0000","last_active":"2024-04-04T06:08:58.592533+0000","last_peered":"2024-04-04T06:08:58.592533+0000","last_clean":"2024-04-04T06:08:58.592533+0000","last_became_active":"2024-04-04T06:08:57.591586+0000","last_became_peered":"2024-04-04T06:08:57.591586+0000","last_unstale":"2024-04-04T06:08:58.592533+0000","last_undegraded":"2024-04-04T06:08:58.592533+0000","last_fullsized":"2024-04-04T06:08:58.592533+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:53:17.231738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604692+0000","last_change":"2024-04-04T06:08:57.614655+0000","last_active":"2024-04-04T06:08:58.604692+0000","last_peered":"2024-04-04T06:08:58.604692+0000","last_clean":"2024-04-04T06:08:58.604692+0000","last_became_active":"2024-04-04T06:08:57.611616+0000","last_became_peered":"2024-04-04T06:08:57.611616+0000","last_unstale":"2024-04-04T06:08:58.604692+0000","last_undegraded":"2024-04-04T06:08:58.604692+0000","last_fullsized":"2024-04-04T06:08:58.604692+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:22:19.057088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.587204+0000","last_change":"2024-04-04T06:08:57.603638+0000","last_active":"2024-04-04T06:08:58.587204+0000","last_peered":"2024-04-04T06:08:58.587204+0000","last_clean":"2024-04-04T06:08:58.587204+0000","last_became_active":"2024-04-04T06:08:57.600991+0000","last_became_peered":"2024-04-04T06:08:57.600991+0000","last_unstale":"2024-04-04T06:08:58.587204+0000","last_undegraded":"2024-04-04T06:08:58.587204+0000","last_fullsized":"2024-04-04T06:08:58.587204+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:20:16.128711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110731+0000","last_change":"2024-04-04T06:08:57.601986+0000","last_active":"2024-04-04T06:08:59.110731+0000","last_peered":"2024-04-04T06:08:59.110731+0000","last_clean":"2024-04-04T06:08:59.110731+0000","last_became_active":"2024-04-04T06:08:57.601677+0000","last_became_peered":"2024-04-04T06:08:57.601677+0000","last_unstale":"2024-04-04T06:08:59.110731+0000","last_undegraded":"2024-04-04T06:08:59.110731+0000","last_fullsized":"2024-04-04T06:08:59.110731+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:39:27.630288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351327+0000","last_change":"2024-04-04T06:08:57.600183+0000","last_active":"2024-04-04T06:08:59.351327+0000","last_peered":"2024-04-04T06:08:59.351327+0000","last_clean":"2024-04-04T06:08:59.351327+0000","last_became_active":"2024-04-04T06:08:57.599862+0000","last_became_peered":"2024-04-04T06:08:57.599862+0000","last_unstale":"2024-04-04T06:08:59.351327+0000","last_undegraded":"2024-04-04T06:08:59.351327+0000","last_fullsized":"2024-04-04T06:08:59.351327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:29:36.805186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.586574+0000","last_change":"2024-04-04T06:08:57.589983+0000","last_active":"2024-04-04T06:08:58.586574+0000","last_peered":"2024-04-04T06:08:58.586574+0000","last_clean":"2024-04-04T06:08:58.586574+0000","last_became_active":"2024-04-04T06:08:57.589734+0000","last_became_peered":"2024-04-04T06:08:57.589734+0000","last_unstale":"2024-04-04T06:08:58.586574+0000","last_undegraded":"2024-04-04T06:08:58.586574+0000","last_fullsized":"2024-04-04T06:08:58.586574+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:01:08.452110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.605135+0000","last_change":"2024-04-04T06:08:57.609822+0000","last_active":"2024-04-04T06:08:58.605135+0000","last_peered":"2024-04-04T06:08:58.605135+0000","last_clean":"2024-04-04T06:08:58.605135+0000","last_became_active":"2024-04-04T06:08:57.609463+0000","last_became_peered":"2024-04-04T06:08:57.609463+0000","last_unstale":"2024-04-04T06:08:58.605135+0000","last_undegraded":"2024-04-04T06:08:58.605135+0000","last_fullsized":"2024-04-04T06:08:58.605135+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:15:03.941901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.605120+0000","last_change":"2024-04-04T06:08:57.613074+0000","last_active":"2024-04-04T06:08:58.605120+0000","last_peered":"2024-04-04T06:08:58.605120+0000","last_clean":"2024-04-04T06:08:58.605120+0000","last_became_active":"2024-04-04T06:08:57.609879+0000","last_became_peered":"2024-04-04T06:08:57.609879+0000","last_unstale":"2024-04-04T06:08:58.605120+0000","last_undegraded":"2024-04-04T06:08:58.605120+0000","last_fullsized":"2024-04-04T06:08:58.605120+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:01:08.338874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.605241+0000","last_change":"2024-04-04T06:08:56.578928+0000","last_active":"2024-04-04T06:08:58.605241+0000","last_peered":"2024-04-04T06:08:58.605241+0000","last_clean":"2024-04-04T06:08:58.605241+0000","last_became_active":"2024-04-04T06:08:56.578602+0000","last_became_peered":"2024-04-04T06:08:56.578602+0000","last_unstale":"2024-04-04T06:08:58.605241+0000","last_undegraded":"2024-04-04T06:08:58.605241+0000","last_fullsized":"2024-04-04T06:08:58.605241+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:30:18.473540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399701+0000","last_change":"2024-04-04T06:08:57.599563+0000","last_active":"2024-04-04T06:08:59.399701+0000","last_peered":"2024-04-04T06:08:59.399701+0000","last_clean":"2024-04-04T06:08:59.399701+0000","last_became_active":"2024-04-04T06:08:57.599244+0000","last_became_peered":"2024-04-04T06:08:57.599244+0000","last_unstale":"2024-04-04T06:08:59.399701+0000","last_undegraded":"2024-04-04T06:08:59.399701+0000","last_fullsized":"2024-04-04T06:08:59.399701+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:16:21.695529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110705+0000","last_change":"2024-04-04T06:08:56.577361+0000","last_active":"2024-04-04T06:08:59.110705+0000","last_peered":"2024-04-04T06:08:59.110705+0000","last_clean":"2024-04-04T06:08:59.110705+0000","last_became_active":"2024-04-04T06:08:56.576619+0000","last_became_peered":"2024-04-04T06:08:56.576619+0000","last_unstale":"2024-04-04T06:08:59.110705+0000","last_undegraded":"2024-04-04T06:08:59.110705+0000","last_fullsized":"2024-04-04T06:08:59.110705+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:53:16.217692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604774+0000","last_change":"2024-04-04T06:08:57.611238+0000","last_active":"2024-04-04T06:08:58.604774+0000","last_peered":"2024-04-04T06:08:58.604774+0000","last_clean":"2024-04-04T06:08:58.604774+0000","last_became_active":"2024-04-04T06:08:57.610966+0000","last_became_peered":"2024-04-04T06:08:57.610966+0000","last_unstale":"2024-04-04T06:08:58.604774+0000","last_undegraded":"2024-04-04T06:08:58.604774+0000","last_fullsized":"2024-04-04T06:08:58.604774+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:10:11.718125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604997+0000","last_change":"2024-04-04T06:08:56.577805+0000","last_active":"2024-04-04T06:08:58.604997+0000","last_peered":"2024-04-04T06:08:58.604997+0000","last_clean":"2024-04-04T06:08:58.604997+0000","last_became_active":"2024-04-04T06:08:56.576393+0000","last_became_peered":"2024-04-04T06:08:56.576393+0000","last_unstale":"2024-04-04T06:08:58.604997+0000","last_undegraded":"2024-04-04T06:08:58.604997+0000","last_fullsized":"2024-04-04T06:08:58.604997+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:16:13.187412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604859+0000","last_change":"2024-04-04T06:08:57.614436+0000","last_active":"2024-04-04T06:08:58.604859+0000","last_peered":"2024-04-04T06:08:58.604859+0000","last_clean":"2024-04-04T06:08:58.604859+0000","last_became_active":"2024-04-04T06:08:57.609947+0000","last_became_peered":"2024-04-04T06:08:57.609947+0000","last_unstale":"2024-04-04T06:08:58.604859+0000","last_undegraded":"2024-04-04T06:08:58.604859+0000","last_fullsized":"2024-04-04T06:08:58.604859+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:57:01.275554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592942+0000","last_change":"2024-04-04T06:08:56.578999+0000","last_active":"2024-04-04T06:08:58.592942+0000","last_peered":"2024-04-04T06:08:58.592942+0000","last_clean":"2024-04-04T06:08:58.592942+0000","last_became_active":"2024-04-04T06:08:56.578689+0000","last_became_peered":"2024-04-04T06:08:56.578689+0000","last_unstale":"2024-04-04T06:08:58.592942+0000","last_undegraded":"2024-04-04T06:08:58.592942+0000","last_fullsized":"2024-04-04T06:08:58.592942+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:30:18.473540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110373+0000","last_change":"2024-04-04T06:08:57.600474+0000","last_active":"2024-04-04T06:08:59.110373+0000","last_peered":"2024-04-04T06:08:59.110373+0000","last_clean":"2024-04-04T06:08:59.110373+0000","last_became_active":"2024-04-04T06:08:57.600135+0000","last_became_peered":"2024-04-04T06:08:57.600135+0000","last_unstale":"2024-04-04T06:08:59.110373+0000","last_undegraded":"2024-04-04T06:08:59.110373+0000","last_fullsized":"2024-04-04T06:08:59.110373+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:08:50.665613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399510+0000","last_change":"2024-04-04T06:08:56.578612+0000","last_active":"2024-04-04T06:08:59.399510+0000","last_peered":"2024-04-04T06:08:59.399510+0000","last_clean":"2024-04-04T06:08:59.399510+0000","last_became_active":"2024-04-04T06:08:56.575989+0000","last_became_peered":"2024-04-04T06:08:56.575989+0000","last_unstale":"2024-04-04T06:08:59.399510+0000","last_undegraded":"2024-04-04T06:08:59.399510+0000","last_fullsized":"2024-04-04T06:08:59.399510+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:56.028901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110635+0000","last_change":"2024-04-04T06:08:57.617863+0000","last_active":"2024-04-04T06:08:59.110635+0000","last_peered":"2024-04-04T06:08:59.110635+0000","last_clean":"2024-04-04T06:08:59.110635+0000","last_became_active":"2024-04-04T06:08:57.617572+0000","last_became_peered":"2024-04-04T06:08:57.617572+0000","last_unstale":"2024-04-04T06:08:59.110635+0000","last_undegraded":"2024-04-04T06:08:59.110635+0000","last_fullsized":"2024-04-04T06:08:59.110635+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:58:22.358023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604504+0000","last_change":"2024-04-04T06:08:56.579024+0000","last_active":"2024-04-04T06:08:58.604504+0000","last_peered":"2024-04-04T06:08:58.604504+0000","last_clean":"2024-04-04T06:08:58.604504+0000","last_became_active":"2024-04-04T06:08:56.578301+0000","last_became_peered":"2024-04-04T06:08:56.578301+0000","last_unstale":"2024-04-04T06:08:58.604504+0000","last_undegraded":"2024-04-04T06:08:58.604504+0000","last_fullsized":"2024-04-04T06:08:58.604504+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:39:18.014934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.590603+0000","last_change":"2024-04-04T06:08:57.602059+0000","last_active":"2024-04-04T06:08:58.590603+0000","last_peered":"2024-04-04T06:08:58.590603+0000","last_clean":"2024-04-04T06:08:58.590603+0000","last_became_active":"2024-04-04T06:08:57.601116+0000","last_became_peered":"2024-04-04T06:08:57.601116+0000","last_unstale":"2024-04-04T06:08:58.590603+0000","last_undegraded":"2024-04-04T06:08:58.590603+0000","last_fullsized":"2024-04-04T06:08:58.590603+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:17:59.892459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591333+0000","last_change":"2024-04-04T06:08:56.571563+0000","last_active":"2024-04-04T06:08:58.591333+0000","last_peered":"2024-04-04T06:08:58.591333+0000","last_clean":"2024-04-04T06:08:58.591333+0000","last_became_active":"2024-04-04T06:08:56.571092+0000","last_became_peered":"2024-04-04T06:08:56.571092+0000","last_unstale":"2024-04-04T06:08:58.591333+0000","last_undegraded":"2024-04-04T06:08:58.591333+0000","last_fullsized":"2024-04-04T06:08:58.591333+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:03:42.180669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110248+0000","last_change":"2024-04-04T06:08:57.608674+0000","last_active":"2024-04-04T06:08:59.110248+0000","last_peered":"2024-04-04T06:08:59.110248+0000","last_clean":"2024-04-04T06:08:59.110248+0000","last_became_active":"2024-04-04T06:08:57.608433+0000","last_became_peered":"2024-04-04T06:08:57.608433+0000","last_unstale":"2024-04-04T06:08:59.110248+0000","last_undegraded":"2024-04-04T06:08:59.110248+0000","last_fullsized":"2024-04-04T06:08:59.110248+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:30:51.961748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399303+0000","last_change":"2024-04-04T06:08:56.579353+0000","last_active":"2024-04-04T06:08:59.399303+0000","last_peered":"2024-04-04T06:08:59.399303+0000","last_clean":"2024-04-04T06:08:59.399303+0000","last_became_active":"2024-04-04T06:08:56.576387+0000","last_became_peered":"2024-04-04T06:08:56.576387+0000","last_unstale":"2024-04-04T06:08:59.399303+0000","last_undegraded":"2024-04-04T06:08:59.399303+0000","last_fullsized":"2024-04-04T06:08:59.399303+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:03:42.180669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.400514+0000","last_change":"2024-04-04T06:08:57.590692+0000","last_active":"2024-04-04T06:08:59.400514+0000","last_peered":"2024-04-04T06:08:59.400514+0000","last_clean":"2024-04-04T06:08:59.400514+0000","last_became_active":"2024-04-04T06:08:57.590158+0000","last_became_peered":"2024-04-04T06:08:57.590158+0000","last_unstale":"2024-04-04T06:08:59.400514+0000","last_undegraded":"2024-04-04T06:08:59.400514+0000","last_fullsized":"2024-04-04T06:08:59.400514+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:24:51.708795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.352306+0000","last_change":"2024-04-04T06:08:56.587466+0000","last_active":"2024-04-04T06:08:59.352306+0000","last_peered":"2024-04-04T06:08:59.352306+0000","last_clean":"2024-04-04T06:08:59.352306+0000","last_became_active":"2024-04-04T06:08:56.587163+0000","last_became_peered":"2024-04-04T06:08:56.587163+0000","last_unstale":"2024-04-04T06:08:59.352306+0000","last_undegraded":"2024-04-04T06:08:59.352306+0000","last_fullsized":"2024-04-04T06:08:59.352306+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:52:42.577594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.352046+0000","last_change":"2024-04-04T06:08:57.609373+0000","last_active":"2024-04-04T06:08:59.352046+0000","last_peered":"2024-04-04T06:08:59.352046+0000","last_clean":"2024-04-04T06:08:59.352046+0000","last_became_active":"2024-04-04T06:08:57.606670+0000","last_became_peered":"2024-04-04T06:08:57.606670+0000","last_unstale":"2024-04-04T06:08:59.352046+0000","last_undegraded":"2024-04-04T06:08:59.352046+0000","last_fullsized":"2024-04-04T06:08:59.352046+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:17:58.054272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110980+0000","last_change":"2024-04-04T06:08:56.581539+0000","last_active":"2024-04-04T06:08:59.110980+0000","last_peered":"2024-04-04T06:08:59.110980+0000","last_clean":"2024-04-04T06:08:59.110980+0000","last_became_active":"2024-04-04T06:08:56.579490+0000","last_became_peered":"2024-04-04T06:08:56.579490+0000","last_unstale":"2024-04-04T06:08:59.110980+0000","last_undegraded":"2024-04-04T06:08:59.110980+0000","last_fullsized":"2024-04-04T06:08:59.110980+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:39:55.913297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399236+0000","last_change":"2024-04-04T06:08:57.588887+0000","last_active":"2024-04-04T06:08:59.399236+0000","last_peered":"2024-04-04T06:08:59.399236+0000","last_clean":"2024-04-04T06:08:59.399236+0000","last_became_active":"2024-04-04T06:08:57.588615+0000","last_became_peered":"2024-04-04T06:08:57.588615+0000","last_unstale":"2024-04-04T06:08:59.399236+0000","last_undegraded":"2024-04-04T06:08:59.399236+0000","last_fullsized":"2024-04-04T06:08:59.399236+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:29:09.919535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.352507+0000","last_change":"2024-04-04T06:08:56.582280+0000","last_active":"2024-04-04T06:08:59.352507+0000","last_peered":"2024-04-04T06:08:59.352507+0000","last_clean":"2024-04-04T06:08:59.352507+0000","last_became_active":"2024-04-04T06:08:56.581968+0000","last_became_peered":"2024-04-04T06:08:56.581968+0000","last_unstale":"2024-04-04T06:08:59.352507+0000","last_undegraded":"2024-04-04T06:08:59.352507+0000","last_fullsized":"2024-04-04T06:08:59.352507+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:34:17.865573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.352230+0000","last_change":"2024-04-04T06:08:57.604234+0000","last_active":"2024-04-04T06:08:59.352230+0000","last_peered":"2024-04-04T06:08:59.352230+0000","last_clean":"2024-04-04T06:08:59.352230+0000","last_became_active":"2024-04-04T06:08:57.602743+0000","last_became_peered":"2024-04-04T06:08:57.602743+0000","last_unstale":"2024-04-04T06:08:59.352230+0000","last_undegraded":"2024-04-04T06:08:59.352230+0000","last_fullsized":"2024-04-04T06:08:59.352230+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:52:53.825956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399994+0000","last_change":"2024-04-04T06:08:56.577639+0000","last_active":"2024-04-04T06:08:59.399994+0000","last_peered":"2024-04-04T06:08:59.399994+0000","last_clean":"2024-04-04T06:08:59.399994+0000","last_became_active":"2024-04-04T06:08:56.577372+0000","last_became_peered":"2024-04-04T06:08:56.577372+0000","last_unstale":"2024-04-04T06:08:59.399994+0000","last_undegraded":"2024-04-04T06:08:59.399994+0000","last_fullsized":"2024-04-04T06:08:59.399994+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:16:20.681483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591829+0000","last_change":"2024-04-04T06:08:57.601155+0000","last_active":"2024-04-04T06:08:58.591829+0000","last_peered":"2024-04-04T06:08:58.591829+0000","last_clean":"2024-04-04T06:08:58.591829+0000","last_became_active":"2024-04-04T06:08:57.600030+0000","last_became_peered":"2024-04-04T06:08:57.600030+0000","last_unstale":"2024-04-04T06:08:58.591829+0000","last_undegraded":"2024-04-04T06:08:58.591829+0000","last_fullsized":"2024-04-04T06:08:58.591829+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:46:59.204669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110522+0000","last_change":"2024-04-04T06:08:56.575082+0000","last_active":"2024-04-04T06:08:59.110522+0000","last_peered":"2024-04-04T06:08:59.110522+0000","last_clean":"2024-04-04T06:08:59.110522+0000","last_became_active":"2024-04-04T06:08:56.574805+0000","last_became_peered":"2024-04-04T06:08:56.574805+0000","last_unstale":"2024-04-04T06:08:59.110522+0000","last_undegraded":"2024-04-04T06:08:59.110522+0000","last_fullsized":"2024-04-04T06:08:59.110522+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:46:58.190623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351568+0000","last_change":"2024-04-04T06:08:57.607510+0000","last_active":"2024-04-04T06:08:59.351568+0000","last_peered":"2024-04-04T06:08:59.351568+0000","last_clean":"2024-04-04T06:08:59.351568+0000","last_became_active":"2024-04-04T06:08:57.606804+0000","last_became_peered":"2024-04-04T06:08:57.606804+0000","last_unstale":"2024-04-04T06:08:59.351568+0000","last_undegraded":"2024-04-04T06:08:59.351568+0000","last_fullsized":"2024-04-04T06:08:59.351568+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:01:29.815756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591001+0000","last_change":"2024-04-04T06:08:56.576046+0000","last_active":"2024-04-04T06:08:58.591001+0000","last_peered":"2024-04-04T06:08:58.591001+0000","last_clean":"2024-04-04T06:08:58.591001+0000","last_became_active":"2024-04-04T06:08:56.575726+0000","last_became_peered":"2024-04-04T06:08:56.575726+0000","last_unstale":"2024-04-04T06:08:58.591001+0000","last_undegraded":"2024-04-04T06:08:58.591001+0000","last_fullsized":"2024-04-04T06:08:58.591001+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:35:35.113120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399396+0000","last_change":"2024-04-04T06:08:57.599582+0000","last_active":"2024-04-04T06:08:59.399396+0000","last_peered":"2024-04-04T06:08:59.399396+0000","last_clean":"2024-04-04T06:08:59.399396+0000","last_became_active":"2024-04-04T06:08:57.598581+0000","last_became_peered":"2024-04-04T06:08:57.598581+0000","last_unstale":"2024-04-04T06:08:59.399396+0000","last_undegraded":"2024-04-04T06:08:59.399396+0000","last_fullsized":"2024-04-04T06:08:59.399396+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:46:55.241650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591592+0000","last_change":"2024-04-04T06:08:56.576601+0000","last_active":"2024-04-04T06:08:58.591592+0000","last_peered":"2024-04-04T06:08:58.591592+0000","last_clean":"2024-04-04T06:08:58.591592+0000","last_became_active":"2024-04-04T06:08:56.572894+0000","last_became_peered":"2024-04-04T06:08:56.572894+0000","last_unstale":"2024-04-04T06:08:58.591592+0000","last_undegraded":"2024-04-04T06:08:58.591592+0000","last_fullsized":"2024-04-04T06:08:58.591592+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:16:13.187412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351248+0000","last_change":"2024-04-04T06:08:57.610067+0000","last_active":"2024-04-04T06:08:59.351248+0000","last_peered":"2024-04-04T06:08:59.351248+0000","last_clean":"2024-04-04T06:08:59.351248+0000","last_became_active":"2024-04-04T06:08:57.609770+0000","last_became_peered":"2024-04-04T06:08:57.609770+0000","last_unstale":"2024-04-04T06:08:59.351248+0000","last_undegraded":"2024-04-04T06:08:59.351248+0000","last_fullsized":"2024-04-04T06:08:59.351248+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:43:34.312451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.350950+0000","last_change":"2024-04-04T06:08:56.582998+0000","last_active":"2024-04-04T06:08:59.350950+0000","last_peered":"2024-04-04T06:08:59.350950+0000","last_clean":"2024-04-04T06:08:59.350950+0000","last_became_active":"2024-04-04T06:08:56.582720+0000","last_became_peered":"2024-04-04T06:08:56.582720+0000","last_unstale":"2024-04-04T06:08:59.350950+0000","last_undegraded":"2024-04-04T06:08:59.350950+0000","last_fullsized":"2024-04-04T06:08:59.350950+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:48:07.678914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.352120+0000","last_change":"2024-04-04T06:08:57.600691+0000","last_active":"2024-04-04T06:08:59.352120+0000","last_peered":"2024-04-04T06:08:59.352120+0000","last_clean":"2024-04-04T06:08:59.352120+0000","last_became_active":"2024-04-04T06:08:57.600389+0000","last_became_peered":"2024-04-04T06:08:57.600389+0000","last_unstale":"2024-04-04T06:08:59.352120+0000","last_undegraded":"2024-04-04T06:08:59.352120+0000","last_fullsized":"2024-04-04T06:08:59.352120+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:49:27.426679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.111161+0000","last_change":"2024-04-04T06:08:56.575119+0000","last_active":"2024-04-04T06:08:59.111161+0000","last_peered":"2024-04-04T06:08:59.111161+0000","last_clean":"2024-04-04T06:08:59.111161+0000","last_became_active":"2024-04-04T06:08:56.574832+0000","last_became_peered":"2024-04-04T06:08:56.574832+0000","last_unstale":"2024-04-04T06:08:59.111161+0000","last_undegraded":"2024-04-04T06:08:59.111161+0000","last_fullsized":"2024-04-04T06:08:59.111161+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:47:00.986955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592375+0000","last_change":"2024-04-04T06:08:57.600113+0000","last_active":"2024-04-04T06:08:58.592375+0000","last_peered":"2024-04-04T06:08:58.592375+0000","last_clean":"2024-04-04T06:08:58.592375+0000","last_became_active":"2024-04-04T06:08:57.599658+0000","last_became_peered":"2024-04-04T06:08:57.599658+0000","last_unstale":"2024-04-04T06:08:58.592375+0000","last_undegraded":"2024-04-04T06:08:58.592375+0000","last_fullsized":"2024-04-04T06:08:58.592375+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:09:31.789910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.605123+0000","last_change":"2024-04-04T06:08:56.576721+0000","last_active":"2024-04-04T06:08:58.605123+0000","last_peered":"2024-04-04T06:08:58.605123+0000","last_clean":"2024-04-04T06:08:58.605123+0000","last_became_active":"2024-04-04T06:08:56.576404+0000","last_became_peered":"2024-04-04T06:08:56.576404+0000","last_unstale":"2024-04-04T06:08:58.605123+0000","last_undegraded":"2024-04-04T06:08:58.605123+0000","last_fullsized":"2024-04-04T06:08:58.605123+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:30:05.745690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110887+0000","last_change":"2024-04-04T06:08:57.605056+0000","last_active":"2024-04-04T06:08:59.110887+0000","last_peered":"2024-04-04T06:08:59.110887+0000","last_clean":"2024-04-04T06:08:59.110887+0000","last_became_active":"2024-04-04T06:08:57.604780+0000","last_became_peered":"2024-04-04T06:08:57.604780+0000","last_unstale":"2024-04-04T06:08:59.110887+0000","last_undegraded":"2024-04-04T06:08:59.110887+0000","last_fullsized":"2024-04-04T06:08:59.110887+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:26:29.014310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592694+0000","last_change":"2024-04-04T06:08:56.579736+0000","last_active":"2024-04-04T06:08:58.592694+0000","last_peered":"2024-04-04T06:08:58.592694+0000","last_clean":"2024-04-04T06:08:58.592694+0000","last_became_active":"2024-04-04T06:08:56.579470+0000","last_became_peered":"2024-04-04T06:08:56.579470+0000","last_unstale":"2024-04-04T06:08:58.592694+0000","last_undegraded":"2024-04-04T06:08:58.592694+0000","last_fullsized":"2024-04-04T06:08:58.592694+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:16:20.681483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.352514+0000","last_change":"2024-04-04T06:08:57.605903+0000","last_active":"2024-04-04T06:08:59.352514+0000","last_peered":"2024-04-04T06:08:59.352514+0000","last_clean":"2024-04-04T06:08:59.352514+0000","last_became_active":"2024-04-04T06:08:57.605458+0000","last_became_peered":"2024-04-04T06:08:57.605458+0000","last_unstale":"2024-04-04T06:08:59.352514+0000","last_undegraded":"2024-04-04T06:08:59.352514+0000","last_fullsized":"2024-04-04T06:08:59.352514+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:33:51.458736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604894+0000","last_change":"2024-04-04T06:08:56.577972+0000","last_active":"2024-04-04T06:08:58.604894+0000","last_peered":"2024-04-04T06:08:58.604894+0000","last_clean":"2024-04-04T06:08:58.604894+0000","last_became_active":"2024-04-04T06:08:56.577700+0000","last_became_peered":"2024-04-04T06:08:56.577700+0000","last_unstale":"2024-04-04T06:08:58.604894+0000","last_undegraded":"2024-04-04T06:08:58.604894+0000","last_fullsized":"2024-04-04T06:08:58.604894+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:37:53.831405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591961+0000","last_change":"2024-04-04T06:08:57.592739+0000","last_active":"2024-04-04T06:08:58.591961+0000","last_peered":"2024-04-04T06:08:58.591961+0000","last_clean":"2024-04-04T06:08:58.591961+0000","last_became_active":"2024-04-04T06:08:57.590596+0000","last_became_peered":"2024-04-04T06:08:57.590596+0000","last_unstale":"2024-04-04T06:08:58.591961+0000","last_undegraded":"2024-04-04T06:08:58.591961+0000","last_fullsized":"2024-04-04T06:08:58.591961+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:25:23.457982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110789+0000","last_change":"2024-04-04T06:08:56.571770+0000","last_active":"2024-04-04T06:08:59.110789+0000","last_peered":"2024-04-04T06:08:59.110789+0000","last_clean":"2024-04-04T06:08:59.110789+0000","last_became_active":"2024-04-04T06:08:56.571431+0000","last_became_peered":"2024-04-04T06:08:56.571431+0000","last_unstale":"2024-04-04T06:08:59.110789+0000","last_undegraded":"2024-04-04T06:08:59.110789+0000","last_fullsized":"2024-04-04T06:08:59.110789+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:25:22.443936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604805+0000","last_change":"2024-04-04T06:08:57.613060+0000","last_active":"2024-04-04T06:08:58.604805+0000","last_peered":"2024-04-04T06:08:58.604805+0000","last_clean":"2024-04-04T06:08:58.604805+0000","last_became_active":"2024-04-04T06:08:57.612824+0000","last_became_peered":"2024-04-04T06:08:57.612824+0000","last_unstale":"2024-04-04T06:08:58.604805+0000","last_undegraded":"2024-04-04T06:08:58.604805+0000","last_fullsized":"2024-04-04T06:08:58.604805+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:54:50.560191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.398963+0000","last_change":"2024-04-04T06:08:56.574202+0000","last_active":"2024-04-04T06:08:59.398963+0000","last_peered":"2024-04-04T06:08:59.398963+0000","last_clean":"2024-04-04T06:08:59.398963+0000","last_became_active":"2024-04-04T06:08:56.573425+0000","last_became_peered":"2024-04-04T06:08:56.573425+0000","last_unstale":"2024-04-04T06:08:59.398963+0000","last_undegraded":"2024-04-04T06:08:59.398963+0000","last_fullsized":"2024-04-04T06:08:59.398963+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:09:27.272946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604465+0000","last_change":"2024-04-04T06:08:57.612462+0000","last_active":"2024-04-04T06:08:58.604465+0000","last_peered":"2024-04-04T06:08:58.604465+0000","last_clean":"2024-04-04T06:08:58.604465+0000","last_became_active":"2024-04-04T06:08:57.611941+0000","last_became_peered":"2024-04-04T06:08:57.611941+0000","last_unstale":"2024-04-04T06:08:58.604465+0000","last_undegraded":"2024-04-04T06:08:58.604465+0000","last_fullsized":"2024-04-04T06:08:58.604465+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:39:56.927343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591348+0000","last_change":"2024-04-04T06:08:56.575741+0000","last_active":"2024-04-04T06:08:58.591348+0000","last_peered":"2024-04-04T06:08:58.591348+0000","last_clean":"2024-04-04T06:08:58.591348+0000","last_became_active":"2024-04-04T06:08:56.575438+0000","last_became_peered":"2024-04-04T06:08:56.575438+0000","last_unstale":"2024-04-04T06:08:58.591348+0000","last_undegraded":"2024-04-04T06:08:58.591348+0000","last_fullsized":"2024-04-04T06:08:58.591348+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:46:32.849014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592972+0000","last_change":"2024-04-04T06:08:57.596280+0000","last_active":"2024-04-04T06:08:58.592972+0000","last_peered":"2024-04-04T06:08:58.592972+0000","last_clean":"2024-04-04T06:08:58.592972+0000","last_became_active":"2024-04-04T06:08:57.595982+0000","last_became_peered":"2024-04-04T06:08:57.595982+0000","last_unstale":"2024-04-04T06:08:58.592972+0000","last_undegraded":"2024-04-04T06:08:58.592972+0000","last_fullsized":"2024-04-04T06:08:58.592972+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:10:11.718125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399701+0000","last_change":"2024-04-04T06:08:56.574911+0000","last_active":"2024-04-04T06:08:59.399701+0000","last_peered":"2024-04-04T06:08:59.399701+0000","last_clean":"2024-04-04T06:08:59.399701+0000","last_became_active":"2024-04-04T06:08:56.573037+0000","last_became_peered":"2024-04-04T06:08:56.573037+0000","last_unstale":"2024-04-04T06:08:59.399701+0000","last_undegraded":"2024-04-04T06:08:59.399701+0000","last_fullsized":"2024-04-04T06:08:59.399701+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:15:13.550828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592938+0000","last_change":"2024-04-04T06:08:57.602162+0000","last_active":"2024-04-04T06:08:58.592938+0000","last_peered":"2024-04-04T06:08:58.592938+0000","last_clean":"2024-04-04T06:08:58.592938+0000","last_became_active":"2024-04-04T06:08:57.601840+0000","last_became_peered":"2024-04-04T06:08:57.601840+0000","last_unstale":"2024-04-04T06:08:58.592938+0000","last_undegraded":"2024-04-04T06:08:58.592938+0000","last_fullsized":"2024-04-04T06:08:58.592938+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:22:19.057088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"59'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.605323+0000","last_change":"2024-04-04T06:08:56.578575+0000","last_active":"2024-04-04T06:08:58.605323+0000","last_peered":"2024-04-04T06:08:58.605323+0000","last_clean":"2024-04-04T06:08:58.605323+0000","last_became_active":"2024-04-04T06:08:56.578232+0000","last_became_peered":"2024-04-04T06:08:56.578232+0000","last_unstale":"2024-04-04T06:08:58.605323+0000","last_undegraded":"2024-04-04T06:08:58.605323+0000","last_fullsized":"2024-04-04T06:08:58.605323+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:46:54.227604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399457+0000","last_change":"2024-04-04T06:08:57.589470+0000","last_active":"2024-04-04T06:08:59.399457+0000","last_peered":"2024-04-04T06:08:59.399457+0000","last_clean":"2024-04-04T06:08:59.399457+0000","last_became_active":"2024-04-04T06:08:57.589193+0000","last_became_peered":"2024-04-04T06:08:57.589193+0000","last_unstale":"2024-04-04T06:08:59.399457+0000","last_undegraded":"2024-04-04T06:08:59.399457+0000","last_fullsized":"2024-04-04T06:08:59.399457+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:02:34.522323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591681+0000","last_change":"2024-04-04T06:08:56.573941+0000","last_active":"2024-04-04T06:08:58.591681+0000","last_peered":"2024-04-04T06:08:58.591681+0000","last_clean":"2024-04-04T06:08:58.591681+0000","last_became_active":"2024-04-04T06:08:56.573700+0000","last_became_peered":"2024-04-04T06:08:56.573700+0000","last_unstale":"2024-04-04T06:08:58.591681+0000","last_undegraded":"2024-04-04T06:08:58.591681+0000","last_fullsized":"2024-04-04T06:08:58.591681+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:03:55.295388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.605391+0000","last_change":"2024-04-04T06:08:57.611889+0000","last_active":"2024-04-04T06:08:58.605391+0000","last_peered":"2024-04-04T06:08:58.605391+0000","last_clean":"2024-04-04T06:08:58.605391+0000","last_became_active":"2024-04-04T06:08:57.609636+0000","last_became_peered":"2024-04-04T06:08:57.609636+0000","last_unstale":"2024-04-04T06:08:58.605391+0000","last_undegraded":"2024-04-04T06:08:58.605391+0000","last_fullsized":"2024-04-04T06:08:58.605391+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:50:39.260072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.593012+0000","last_change":"2024-04-04T06:08:56.577871+0000","last_active":"2024-04-04T06:08:58.593012+0000","last_peered":"2024-04-04T06:08:58.593012+0000","last_clean":"2024-04-04T06:08:58.593012+0000","last_became_active":"2024-04-04T06:08:56.577570+0000","last_became_peered":"2024-04-04T06:08:56.577570+0000","last_unstale":"2024-04-04T06:08:58.593012+0000","last_undegraded":"2024-04-04T06:08:58.593012+0000","last_fullsized":"2024-04-04T06:08:58.593012+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:37:53.831405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110537+0000","last_change":"2024-04-04T06:08:57.606811+0000","last_active":"2024-04-04T06:08:59.110537+0000","last_peered":"2024-04-04T06:08:59.110537+0000","last_clean":"2024-04-04T06:08:59.110537+0000","last_became_active":"2024-04-04T06:08:57.606536+0000","last_became_peered":"2024-04-04T06:08:57.606536+0000","last_unstale":"2024-04-04T06:08:59.110537+0000","last_undegraded":"2024-04-04T06:08:59.110537+0000","last_fullsized":"2024-04-04T06:08:59.110537+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:10:33.843321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.398742+0000","last_change":"2024-04-04T06:08:56.569703+0000","last_active":"2024-04-04T06:08:59.398742+0000","last_peered":"2024-04-04T06:08:59.398742+0000","last_clean":"2024-04-04T06:08:59.398742+0000","last_became_active":"2024-04-04T06:08:56.569433+0000","last_became_peered":"2024-04-04T06:08:56.569433+0000","last_unstale":"2024-04-04T06:08:59.398742+0000","last_undegraded":"2024-04-04T06:08:59.398742+0000","last_fullsized":"2024-04-04T06:08:59.398742+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:28:25.503580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351982+0000","last_change":"2024-04-04T06:08:57.611293+0000","last_active":"2024-04-04T06:08:59.351982+0000","last_peered":"2024-04-04T06:08:59.351982+0000","last_clean":"2024-04-04T06:08:59.351982+0000","last_became_active":"2024-04-04T06:08:57.610423+0000","last_became_peered":"2024-04-04T06:08:57.610423+0000","last_unstale":"2024-04-04T06:08:59.351982+0000","last_undegraded":"2024-04-04T06:08:59.351982+0000","last_fullsized":"2024-04-04T06:08:59.351982+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:02:19.696656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399524+0000","last_change":"2024-04-04T06:08:56.576880+0000","last_active":"2024-04-04T06:08:59.399524+0000","last_peered":"2024-04-04T06:08:59.399524+0000","last_clean":"2024-04-04T06:08:59.399524+0000","last_became_active":"2024-04-04T06:08:56.576600+0000","last_became_peered":"2024-04-04T06:08:56.576600+0000","last_unstale":"2024-04-04T06:08:59.399524+0000","last_undegraded":"2024-04-04T06:08:59.399524+0000","last_fullsized":"2024-04-04T06:08:59.399524+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:25:43.268590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.398678+0000","last_change":"2024-04-04T06:08:57.597335+0000","last_active":"2024-04-04T06:08:59.398678+0000","last_peered":"2024-04-04T06:08:59.398678+0000","last_clean":"2024-04-04T06:08:59.398678+0000","last_became_active":"2024-04-04T06:08:57.597051+0000","last_became_peered":"2024-04-04T06:08:57.597051+0000","last_unstale":"2024-04-04T06:08:59.398678+0000","last_undegraded":"2024-04-04T06:08:59.398678+0000","last_fullsized":"2024-04-04T06:08:59.398678+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:45:02.503911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.398912+0000","last_change":"2024-04-04T06:08:56.576604+0000","last_active":"2024-04-04T06:08:59.398912+0000","last_peered":"2024-04-04T06:08:59.398912+0000","last_clean":"2024-04-04T06:08:59.398912+0000","last_became_active":"2024-04-04T06:08:56.576310+0000","last_became_peered":"2024-04-04T06:08:56.576310+0000","last_unstale":"2024-04-04T06:08:59.398912+0000","last_undegraded":"2024-04-04T06:08:59.398912+0000","last_fullsized":"2024-04-04T06:08:59.398912+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:26:57.097292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110360+0000","last_change":"2024-04-04T06:08:57.608692+0000","last_active":"2024-04-04T06:08:59.110360+0000","last_peered":"2024-04-04T06:08:59.110360+0000","last_clean":"2024-04-04T06:08:59.110360+0000","last_became_active":"2024-04-04T06:08:57.608387+0000","last_became_peered":"2024-04-04T06:08:57.608387+0000","last_unstale":"2024-04-04T06:08:59.110360+0000","last_undegraded":"2024-04-04T06:08:59.110360+0000","last_fullsized":"2024-04-04T06:08:59.110360+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:38:04.030503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110178+0000","last_change":"2024-04-04T06:08:56.568855+0000","last_active":"2024-04-04T06:08:59.110178+0000","last_peered":"2024-04-04T06:08:59.110178+0000","last_clean":"2024-04-04T06:08:59.110178+0000","last_became_active":"2024-04-04T06:08:56.568608+0000","last_became_peered":"2024-04-04T06:08:56.568608+0000","last_unstale":"2024-04-04T06:08:59.110178+0000","last_undegraded":"2024-04-04T06:08:59.110178+0000","last_fullsized":"2024-04-04T06:08:59.110178+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:09:30.775864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110378+0000","last_change":"2024-04-04T06:08:57.618484+0000","last_active":"2024-04-04T06:08:59.110378+0000","last_peered":"2024-04-04T06:08:59.110378+0000","last_clean":"2024-04-04T06:08:59.110378+0000","last_became_active":"2024-04-04T06:08:57.618213+0000","last_became_peered":"2024-04-04T06:08:57.618213+0000","last_unstale":"2024-04-04T06:08:59.110378+0000","last_undegraded":"2024-04-04T06:08:59.110378+0000","last_fullsized":"2024-04-04T06:08:59.110378+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:09:00.433068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591002+0000","last_change":"2024-04-04T06:08:56.578712+0000","last_active":"2024-04-04T06:08:58.591002+0000","last_peered":"2024-04-04T06:08:58.591002+0000","last_clean":"2024-04-04T06:08:58.591002+0000","last_became_active":"2024-04-04T06:08:56.578391+0000","last_became_peered":"2024-04-04T06:08:56.578391+0000","last_unstale":"2024-04-04T06:08:58.591002+0000","last_undegraded":"2024-04-04T06:08:58.591002+0000","last_fullsized":"2024-04-04T06:08:58.591002+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:51:59.076333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":237,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351054+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:08:59.351054+0000","last_peered":"2024-04-04T06:08:59.351054+0000","last_clean":"2024-04-04T06:08:59.351054+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:08:59.351054+0000","last_undegraded":"2024-04-04T06:08:59.351054+0000","last_fullsized":"2024-04-04T06:08:59.351054+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1302528,"data_stored":882874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":700416,"data_stored":292346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509485,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":292402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509484,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":292402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509483,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":704512,"data_stored":292436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509485,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32460,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339380,"statfs":{"total":96636764160,"available":96603525120,"internally_reserved":0,"allocated":1310720,"data_stored":884908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542188,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":292998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542186,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31880,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339960,"statfs":{"total":96636764160,"available":96604119040,"internally_reserved":0,"allocated":712704,"data_stored":293958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:09:03.103 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-04T06:09:03.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.350+0000 7fae8be00640 1 Processor -- start 2024-04-04T06:09:03.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.351+0000 7fae8be00640 1 -- start start 2024-04-04T06:09:03.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.353+0000 7fae8be00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fae840d1580 0x7fae840cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:03.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.353+0000 7fae8be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fae840b4db0 0x7fae840cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:03.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.353+0000 7fae8be00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae840b5ce0 con 0x7fae84161f60 2024-04-04T06:09:03.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.353+0000 7fae8be00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fae840d27f0 con 0x7fae840b4db0 2024-04-04T06:09:03.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.353+0000 7fae8be00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fae840d2a90 con 0x7fae840d1580 2024-04-04T06:09:03.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.354+0000 7fae89e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fae840b4db0 0x7fae840cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:03.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.354+0000 7fae89e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fae840b4db0 0x7fae840cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37410/0 (socket says 172.21.15.159:37410) 2024-04-04T06:09:03.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.354+0000 7fae89e00640 1 -- 172.21.15.159:0/3201851733 learned_addr learned my addr 172.21.15.159:0/3201851733 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:03.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.354+0000 7fae88a00640 1 --2- 172.21.15.159:0/3201851733 >> v2:172.21.15.178:3300/0 conn(0x7fae840d1580 0x7fae840cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:03.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.354+0000 7fae89e00640 1 -- 172.21.15.159:0/3201851733 >> v2:172.21.15.178:3300/0 conn(0x7fae840d1580 msgr2=0x7fae840cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:03.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.354+0000 7fae89e00640 1 --2- 172.21.15.159:0/3201851733 >> v2:172.21.15.178:3300/0 conn(0x7fae840d1580 0x7fae840cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.354+0000 7fae89e00640 1 -- 172.21.15.159:0/3201851733 >> v1:172.21.15.178:6789/0 conn(0x7fae84161f60 legacy=0x7fae840d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:03.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.354+0000 7fae89e00640 1 -- 172.21.15.159:0/3201851733 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae84162340 con 0x7fae840b4db0 2024-04-04T06:09:03.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.355+0000 7fae89e00640 1 --2- 172.21.15.159:0/3201851733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae840b4db0 0x7fae840cf610 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fae8408c880 tx=0x7fae7c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b2f41cc0a8330e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:03.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.355+0000 7fae83e00640 1 -- 172.21.15.159:0/3201851733 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c02ca80 con 0x7fae840b4db0 2024-04-04T06:09:03.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.356+0000 7fae83e00640 1 -- 172.21.15.159:0/3201851733 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae7c02cc20 con 0x7fae840b4db0 2024-04-04T06:09:03.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.356+0000 7fae83e00640 1 -- 172.21.15.159:0/3201851733 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c0343f0 con 0x7fae840b4db0 2024-04-04T06:09:03.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.356+0000 7fae8be00640 1 -- 172.21.15.159:0/3201851733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae840b4db0 msgr2=0x7fae840cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:03.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.356+0000 7fae8be00640 1 --2- 172.21.15.159:0/3201851733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae840b4db0 0x7fae840cf610 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fae8408c880 tx=0x7fae7c02bee0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.356+0000 7fae8be00640 1 -- 172.21.15.159:0/3201851733 shutdown_connections 2024-04-04T06:09:03.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.356+0000 7fae8be00640 1 --2- 172.21.15.159:0/3201851733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae840b4db0 0x7fae840cf610 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.356+0000 7fae8be00640 1 --2- 172.21.15.159:0/3201851733 >> v2:172.21.15.178:3300/0 conn(0x7fae840d1580 0x7fae840cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.356+0000 7fae8be00640 1 -- 172.21.15.159:0/3201851733 >> 172.21.15.159:0/3201851733 conn(0x7fae8408bfb0 msgr2=0x7fae840d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:03.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.356+0000 7fae8be00640 1 -- 172.21.15.159:0/3201851733 shutdown_connections 2024-04-04T06:09:03.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.356+0000 7fae8be00640 1 -- 172.21.15.159:0/3201851733 wait complete. 2024-04-04T06:09:03.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.358+0000 7fae8be00640 1 Processor -- start 2024-04-04T06:09:03.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.358+0000 7fae8be00640 1 -- start start 2024-04-04T06:09:03.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.359+0000 7fae8be00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae840b4db0 0x7fae8415d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:03.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.359+0000 7fae8be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae840d1580 0x7fae840b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:03.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.359+0000 7fae8be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae84161f60 0x7fae840b2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:03.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.359+0000 7fae8be00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fae840d3310 con 0x7fae84161f60 2024-04-04T06:09:03.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.359+0000 7fae8be00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fae840d3030 con 0x7fae840d1580 2024-04-04T06:09:03.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.359+0000 7fae8be00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fae840b3170 con 0x7fae840b4db0 2024-04-04T06:09:03.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.359+0000 7fae89e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae84161f60 0x7fae840b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:03.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.359+0000 7fae89e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae84161f60 0x7fae840b2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37412/0 (socket says 172.21.15.159:37412) 2024-04-04T06:09:03.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.359+0000 7fae89e00640 1 -- 172.21.15.159:0/237662 learned_addr learned my addr 172.21.15.159:0/237662 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:03.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.360+0000 7fae89e00640 1 -- 172.21.15.159:0/237662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae840b4db0 msgr2=0x7fae8415d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:03.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.360+0000 7fae89e00640 1 --2- 172.21.15.159:0/237662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae840b4db0 0x7fae8415d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.360+0000 7fae89e00640 1 -- 172.21.15.159:0/237662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae840d1580 msgr2=0x7fae840b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:03.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.360+0000 7fae89e00640 1 --2- 172.21.15.159:0/237662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae840d1580 0x7fae840b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.360+0000 7fae89e00640 1 -- 172.21.15.159:0/237662 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae841f4280 con 0x7fae84161f60 2024-04-04T06:09:03.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.360+0000 7fae89e00640 1 --2- 172.21.15.159:0/237662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae84161f60 0x7fae840b2710 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fae7c036040 tx=0x7fae7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:03.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.360+0000 7fae81600640 1 -- 172.21.15.159:0/237662 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c043020 con 0x7fae84161f60 2024-04-04T06:09:03.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.360+0000 7fae81600640 1 -- 172.21.15.159:0/237662 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae7c03e450 con 0x7fae84161f60 2024-04-04T06:09:03.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.360+0000 7fae81600640 1 -- 172.21.15.159:0/237662 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c009510 con 0x7fae84161f60 2024-04-04T06:09:03.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.360+0000 7fae8be00640 1 -- 172.21.15.159:0/237662 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae841f4560 con 0x7fae84161f60 2024-04-04T06:09:03.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.361+0000 7fae8be00640 1 -- 172.21.15.159:0/237662 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae841f4b20 con 0x7fae84161f60 2024-04-04T06:09:03.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.362+0000 7fae8be00640 1 -- 172.21.15.159:0/237662 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae840685c0 con 0x7fae84161f60 2024-04-04T06:09:03.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.363+0000 7fae81600640 1 -- 172.21.15.159:0/237662 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae7c046020 con 0x7fae84161f60 2024-04-04T06:09:03.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.364+0000 7fae81600640 1 --2- 172.21.15.159:0/237662 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fae6406c8f0 0x7fae6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:03.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.364+0000 7fae81600640 1 -- 172.21.15.159:0/237662 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fae7c0ae3d0 con 0x7fae84161f60 2024-04-04T06:09:03.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.364+0000 7fae89400640 1 --2- 172.21.15.159:0/237662 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fae6406c8f0 0x7fae6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:03.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.365+0000 7fae89400640 1 --2- 172.21.15.159:0/237662 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fae6406c8f0 0x7fae6406edb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fae78006fd0 tx=0x7fae78006bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:03.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.368+0000 7fae81600640 1 -- 172.21.15.159:0/237662 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fae7c04d030 con 0x7fae84161f60 2024-04-04T06:09:03.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.648+0000 7fae8be00640 1 -- 172.21.15.159:0/237662 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fae841f4cf0 con 0x7fae84161f60 2024-04-04T06:09:03.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.649+0000 7fae81600640 1 -- 172.21.15.159:0/237662 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1175 (secure 0 0 0) 0x7fae840b5ce0 con 0x7fae84161f60 2024-04-04T06:09:03.643 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:09:03.643 INFO:teuthology.orchestra.run.smithi159.stdout:{"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":215,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":8,"num_up_osds":8,"osd_up_since":1712210741,"num_in_osds":8,"osd_in_since":1712210732,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":258043904,"bytes_avail":772836069376,"bytes_total":773094113280,"write_bytes_sec":2300,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":33,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"c","status":"up:active","gid":6039}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-04-04T06:08:48.774217+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-04T06:09:03.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.652+0000 7fae8be00640 1 -- 172.21.15.159:0/237662 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fae6406c8f0 msgr2=0x7fae6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:03.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.652+0000 7fae8be00640 1 --2- 172.21.15.159:0/237662 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fae6406c8f0 0x7fae6406edb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fae78006fd0 tx=0x7fae78006bc0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.652+0000 7fae8be00640 1 -- 172.21.15.159:0/237662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae84161f60 msgr2=0x7fae840b2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:03.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.652+0000 7fae8be00640 1 --2- 172.21.15.159:0/237662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae84161f60 0x7fae840b2710 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fae7c036040 tx=0x7fae7c004a40 comp rx=0 tx=0).stop 2024-04-04T06:09:03.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.652+0000 7fae8be00640 1 -- 172.21.15.159:0/237662 shutdown_connections 2024-04-04T06:09:03.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.652+0000 7fae8be00640 1 --2- 172.21.15.159:0/237662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae84161f60 0x7fae840b2710 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.652+0000 7fae8be00640 1 --2- 172.21.15.159:0/237662 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fae6406c8f0 0x7fae6406edb0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.652+0000 7fae8be00640 1 --2- 172.21.15.159:0/237662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae840d1580 0x7fae840b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.652+0000 7fae8be00640 1 --2- 172.21.15.159:0/237662 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae840b4db0 0x7fae8415d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.652+0000 7fae8be00640 1 -- 172.21.15.159:0/237662 >> 172.21.15.159:0/237662 conn(0x7fae8408bfb0 msgr2=0x7fae840b67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:03.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.652+0000 7fae8be00640 1 -- 172.21.15.159:0/237662 shutdown_connections 2024-04-04T06:09:03.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.652+0000 7fae8be00640 1 -- 172.21.15.159:0/237662 wait complete. 2024-04-04T06:09:03.672 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-04T06:09:03.672 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:09:03.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.880+0000 7f2bf6200640 1 Processor -- start 2024-04-04T06:09:03.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.880+0000 7f2bf6200640 1 -- start start 2024-04-04T06:09:03.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.881+0000 7f2bf6200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2bf015b310 0x7f2bf00727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:03.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.881+0000 7f2bf6200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bf00b4f90 0x7f2bf015f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:03.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.881+0000 7f2bf6200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2bf00d0780 0x7f2bf00d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:03.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.881+0000 7f2bf6200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f2bf015b6f0 con 0x7f2bf00d0780 2024-04-04T06:09:03.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.881+0000 7f2bf6200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2bf0072e80 con 0x7f2bf015b310 2024-04-04T06:09:03.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.881+0000 7f2bf6200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf0073160 con 0x7f2bf00b4f90 2024-04-04T06:09:03.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.882+0000 7f2befe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2bf00d0780 0x7f2bf00d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:03.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.882+0000 7f2befe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2bf00d0780 0x7f2bf00d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52124/0 (socket says 172.21.15.159:52124) 2024-04-04T06:09:03.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.882+0000 7f2befe00640 1 -- 172.21.15.159:0/2110125174 learned_addr learned my addr 172.21.15.159:0/2110125174 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:03.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.882+0000 7f2be7e00640 1 --2- 172.21.15.159:0/2110125174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bf00b4f90 0x7f2bf015f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:03.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.882+0000 7f2bef400640 1 --2- 172.21.15.159:0/2110125174 >> v2:172.21.15.178:3300/0 conn(0x7f2bf015b310 0x7f2bf00727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:03.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.882+0000 7f2befe00640 1 -- 172.21.15.159:0/2110125174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bf00b4f90 msgr2=0x7f2bf015f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:03.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.882+0000 7f2befe00640 1 --2- 172.21.15.159:0/2110125174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bf00b4f90 0x7f2bf015f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.882+0000 7f2befe00640 1 -- 172.21.15.159:0/2110125174 >> v2:172.21.15.178:3300/0 conn(0x7f2bf015b310 msgr2=0x7f2bf00727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:03.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.882+0000 7f2befe00640 1 --2- 172.21.15.159:0/2110125174 >> v2:172.21.15.178:3300/0 conn(0x7f2bf015b310 0x7f2bf00727f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.882+0000 7f2befe00640 1 -- 172.21.15.159:0/2110125174 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf015efe0 con 0x7f2bf00d0780 2024-04-04T06:09:03.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.884+0000 7f2befe00640 1 --2- 172.21.15.159:0/2110125174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bf00d0780 0x7f2bf00d0c10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2bf006fa40 tx=0x7f2be802dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e38d464111d95ad server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:03.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.884+0000 7f2beea00640 1 -- 172.21.15.159:0/2110125174 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be802e970 con 0x7f2bf00d0780 2024-04-04T06:09:03.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.884+0000 7f2beea00640 1 -- 172.21.15.159:0/2110125174 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be802eb10 con 0x7f2bf00d0780 2024-04-04T06:09:03.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.884+0000 7f2beea00640 1 -- 172.21.15.159:0/2110125174 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8009600 con 0x7f2bf00d0780 2024-04-04T06:09:03.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.885+0000 7f2bf6200640 1 -- 172.21.15.159:0/2110125174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bf00d0780 msgr2=0x7f2bf00d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:03.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.885+0000 7f2bf6200640 1 --2- 172.21.15.159:0/2110125174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bf00d0780 0x7f2bf00d0c10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2bf006fa40 tx=0x7f2be802dfd0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.885+0000 7f2bf6200640 1 -- 172.21.15.159:0/2110125174 shutdown_connections 2024-04-04T06:09:03.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.885+0000 7f2bf6200640 1 --2- 172.21.15.159:0/2110125174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bf00d0780 0x7f2bf00d0c10 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.885+0000 7f2bf6200640 1 --2- 172.21.15.159:0/2110125174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bf00b4f90 0x7f2bf015f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.885+0000 7f2bf6200640 1 --2- 172.21.15.159:0/2110125174 >> v2:172.21.15.178:3300/0 conn(0x7f2bf015b310 0x7f2bf00727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.885+0000 7f2bf6200640 1 -- 172.21.15.159:0/2110125174 >> 172.21.15.159:0/2110125174 conn(0x7f2bf006b990 msgr2=0x7f2bf006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:03.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.885+0000 7f2bf6200640 1 -- 172.21.15.159:0/2110125174 shutdown_connections 2024-04-04T06:09:03.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.885+0000 7f2bf6200640 1 -- 172.21.15.159:0/2110125174 wait complete. 2024-04-04T06:09:03.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.886+0000 7f2bf6200640 1 Processor -- start 2024-04-04T06:09:03.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.887+0000 7f2bf6200640 1 -- start start 2024-04-04T06:09:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.887+0000 7f2bf6200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bf00b4f90 0x7f2bf01532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.888+0000 7f2bf6200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bf00d0780 0x7f2bf01e56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.888+0000 7f2bf6200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bf015b310 0x7f2bf0152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.888+0000 7f2bf6200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf0073440 con 0x7f2bf00d0780 2024-04-04T06:09:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.888+0000 7f2bf6200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf00b53f0 con 0x7f2bf00b4f90 2024-04-04T06:09:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.888+0000 7f2bf6200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf015fd20 con 0x7f2bf015b310 2024-04-04T06:09:03.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.888+0000 7f2be7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bf00d0780 0x7f2bf01e56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.888+0000 7f2be7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bf00d0780 0x7f2bf01e56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52134/0 (socket says 172.21.15.159:52134) 2024-04-04T06:09:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.888+0000 7f2be7e00640 1 -- 172.21.15.159:0/941647145 learned_addr learned my addr 172.21.15.159:0/941647145 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.888+0000 7f2bef400640 1 --2- 172.21.15.159:0/941647145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bf00b4f90 0x7f2bf01532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.888+0000 7f2befe00640 1 --2- 172.21.15.159:0/941647145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bf015b310 0x7f2bf0152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.888+0000 7f2be7e00640 1 -- 172.21.15.159:0/941647145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bf015b310 msgr2=0x7f2bf0152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.888+0000 7f2be7e00640 1 --2- 172.21.15.159:0/941647145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bf015b310 0x7f2bf0152af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.888+0000 7f2be7e00640 1 -- 172.21.15.159:0/941647145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bf00b4f90 msgr2=0x7f2bf01532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.888+0000 7f2be7e00640 1 --2- 172.21.15.159:0/941647145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bf00b4f90 0x7f2bf01532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:03.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.888+0000 7f2be7e00640 1 -- 172.21.15.159:0/941647145 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf01e5e40 con 0x7f2bf00d0780 2024-04-04T06:09:03.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.889+0000 7f2be7e00640 1 --2- 172.21.15.159:0/941647145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bf00d0780 0x7f2bf01e56f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800d800 tx=0x7f2bd800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:03.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.889+0000 7f2be7400640 1 -- 172.21.15.159:0/941647145 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8018030 con 0x7f2bf00d0780 2024-04-04T06:09:03.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.889+0000 7f2be7400640 1 -- 172.21.15.159:0/941647145 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd80096a0 con 0x7f2bf00d0780 2024-04-04T06:09:03.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.889+0000 7f2be7400640 1 -- 172.21.15.159:0/941647145 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8009870 con 0x7f2bf00d0780 2024-04-04T06:09:03.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.889+0000 7f2bf6200640 1 -- 172.21.15.159:0/941647145 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf01e6120 con 0x7f2bf00d0780 2024-04-04T06:09:03.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.889+0000 7f2bf6200640 1 -- 172.21.15.159:0/941647145 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf0155e10 con 0x7f2bf00d0780 2024-04-04T06:09:03.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.890+0000 7f2bf6200640 1 -- 172.21.15.159:0/941647145 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bf01e6640 con 0x7f2bf00d0780 2024-04-04T06:09:03.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.892+0000 7f2be7400640 1 -- 172.21.15.159:0/941647145 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2bd801b070 con 0x7f2bf00d0780 2024-04-04T06:09:03.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.893+0000 7f2be7400640 1 --2- 172.21.15.159:0/941647145 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2bcc06c8f0 0x7f2bcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:03.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.893+0000 7f2be7400640 1 -- 172.21.15.159:0/941647145 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f2bd808bde0 con 0x7f2bf00d0780 2024-04-04T06:09:03.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.894+0000 7f2bef400640 1 --2- 172.21.15.159:0/941647145 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2bcc06c8f0 0x7f2bcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:03.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.895+0000 7f2bef400640 1 --2- 172.21.15.159:0/941647145 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2bcc06c8f0 0x7f2bcc06edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2be0006fd0 tx=0x7f2be0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:03.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:03.897+0000 7f2be7400640 1 -- 172.21.15.159:0/941647145 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2bf01e6640 con 0x7f2bf00d0780 2024-04-04T06:09:04.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:04.093+0000 7f2bf6200640 1 -- 172.21.15.159:0/941647145 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2bf00685c0 con 0x7f2bcc06c8f0 2024-04-04T06:09:04.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:04.112+0000 7f2be7400640 1 -- 172.21.15.159:0/941647145 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251955 (secure 0 0 0) 0x7f2bf00685c0 con 0x7f2bcc06c8f0 2024-04-04T06:09:04.110 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:09:04.111 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:09:04.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:04.119+0000 7f2bf6200640 1 -- 172.21.15.159:0/941647145 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2bcc06c8f0 msgr2=0x7f2bcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:04.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:04.119+0000 7f2bf6200640 1 --2- 172.21.15.159:0/941647145 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2bcc06c8f0 0x7f2bcc06edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2be0006fd0 tx=0x7f2be0008040 comp rx=0 tx=0).stop 2024-04-04T06:09:04.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:04.119+0000 7f2bf6200640 1 -- 172.21.15.159:0/941647145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bf00d0780 msgr2=0x7f2bf01e56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:04.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:04.119+0000 7f2bf6200640 1 --2- 172.21.15.159:0/941647145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bf00d0780 0x7f2bf01e56f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800d800 tx=0x7f2bd800dcc0 comp rx=0 tx=0).stop 2024-04-04T06:09:04.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:04.119+0000 7f2bf6200640 1 -- 172.21.15.159:0/941647145 shutdown_connections 2024-04-04T06:09:04.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:04.119+0000 7f2bf6200640 1 --2- 172.21.15.159:0/941647145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2bf015b310 0x7f2bf0152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:04.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:04.119+0000 7f2bf6200640 1 --2- 172.21.15.159:0/941647145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bf00d0780 0x7f2bf01e56f0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:04.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:04.119+0000 7f2bf6200640 1 --2- 172.21.15.159:0/941647145 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2bcc06c8f0 0x7f2bcc06edb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:04.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:04.119+0000 7f2bf6200640 1 --2- 172.21.15.159:0/941647145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bf00b4f90 0x7f2bf01532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:04.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:04.119+0000 7f2bf6200640 1 -- 172.21.15.159:0/941647145 >> 172.21.15.159:0/941647145 conn(0x7f2bf006b990 msgr2=0x7f2bf00b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:04.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:04.119+0000 7f2bf6200640 1 -- 172.21.15.159:0/941647145 shutdown_connections 2024-04-04T06:09:04.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:04.119+0000 7f2bf6200640 1 -- 172.21.15.159:0/941647145 wait complete. 2024-04-04T06:09:04.131 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-04-04T06:09:02.778415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":251996,"kb_used_data":6696,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754722724,"statfs":{"total":773094113280,"available":772836069376,"internally_reserved":0,"allocated":6856704,"data_stored":3524324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.231986"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.587764+0000","last_change":"2024-04-04T06:08:57.592232+0000","last_active":"2024-04-04T06:08:58.587764+0000","last_peered":"2024-04-04T06:08:58.587764+0000","last_clean":"2024-04-04T06:08:58.587764+0000","last_became_active":"2024-04-04T06:08:57.591849+0000","last_became_peered":"2024-04-04T06:08:57.591849+0000","last_unstale":"2024-04-04T06:08:58.587764+0000","last_undegraded":"2024-04-04T06:08:58.587764+0000","last_fullsized":"2024-04-04T06:08:58.587764+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:55:41.770693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110894+0000","last_change":"2024-04-04T06:08:57.606120+0000","last_active":"2024-04-04T06:08:59.110894+0000","last_peered":"2024-04-04T06:08:59.110894+0000","last_clean":"2024-04-04T06:08:59.110894+0000","last_became_active":"2024-04-04T06:08:57.605844+0000","last_became_peered":"2024-04-04T06:08:57.605844+0000","last_unstale":"2024-04-04T06:08:59.110894+0000","last_undegraded":"2024-04-04T06:08:59.110894+0000","last_fullsized":"2024-04-04T06:08:59.110894+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:07:04.193264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.352020+0000","last_change":"2024-04-04T06:08:57.604547+0000","last_active":"2024-04-04T06:08:59.352020+0000","last_peered":"2024-04-04T06:08:59.352020+0000","last_clean":"2024-04-04T06:08:59.352020+0000","last_became_active":"2024-04-04T06:08:57.604225+0000","last_became_peered":"2024-04-04T06:08:57.604225+0000","last_unstale":"2024-04-04T06:08:59.352020+0000","last_undegraded":"2024-04-04T06:08:59.352020+0000","last_fullsized":"2024-04-04T06:08:59.352020+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:45:43.089485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592791+0000","last_change":"2024-04-04T06:08:57.593370+0000","last_active":"2024-04-04T06:08:58.592791+0000","last_peered":"2024-04-04T06:08:58.592791+0000","last_clean":"2024-04-04T06:08:58.592791+0000","last_became_active":"2024-04-04T06:08:57.591455+0000","last_became_peered":"2024-04-04T06:08:57.591455+0000","last_unstale":"2024-04-04T06:08:58.592791+0000","last_undegraded":"2024-04-04T06:08:58.592791+0000","last_fullsized":"2024-04-04T06:08:58.592791+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:39:56.927343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.586778+0000","last_change":"2024-04-04T06:08:57.602481+0000","last_active":"2024-04-04T06:08:58.586778+0000","last_peered":"2024-04-04T06:08:58.586778+0000","last_clean":"2024-04-04T06:08:58.586778+0000","last_became_active":"2024-04-04T06:08:57.599211+0000","last_became_peered":"2024-04-04T06:08:57.599211+0000","last_unstale":"2024-04-04T06:08:58.586778+0000","last_undegraded":"2024-04-04T06:08:58.586778+0000","last_fullsized":"2024-04-04T06:08:58.586778+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:25:32.291869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604666+0000","last_change":"2024-04-04T06:08:57.614185+0000","last_active":"2024-04-04T06:08:58.604666+0000","last_peered":"2024-04-04T06:08:58.604666+0000","last_clean":"2024-04-04T06:08:58.604666+0000","last_became_active":"2024-04-04T06:08:57.611970+0000","last_became_peered":"2024-04-04T06:08:57.611970+0000","last_unstale":"2024-04-04T06:08:58.604666+0000","last_undegraded":"2024-04-04T06:08:58.604666+0000","last_fullsized":"2024-04-04T06:08:58.604666+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:47:02.001001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399802+0000","last_change":"2024-04-04T06:08:57.596174+0000","last_active":"2024-04-04T06:08:59.399802+0000","last_peered":"2024-04-04T06:08:59.399802+0000","last_clean":"2024-04-04T06:08:59.399802+0000","last_became_active":"2024-04-04T06:08:57.595899+0000","last_became_peered":"2024-04-04T06:08:57.595899+0000","last_unstale":"2024-04-04T06:08:59.399802+0000","last_undegraded":"2024-04-04T06:08:59.399802+0000","last_fullsized":"2024-04-04T06:08:59.399802+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:41:26.350622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110048+0000","last_change":"2024-04-04T06:08:57.605001+0000","last_active":"2024-04-04T06:08:59.110048+0000","last_peered":"2024-04-04T06:08:59.110048+0000","last_clean":"2024-04-04T06:08:59.110048+0000","last_became_active":"2024-04-04T06:08:57.604608+0000","last_became_peered":"2024-04-04T06:08:57.604608+0000","last_unstale":"2024-04-04T06:08:59.110048+0000","last_undegraded":"2024-04-04T06:08:59.110048+0000","last_fullsized":"2024-04-04T06:08:59.110048+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:16:53.808572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.400387+0000","last_change":"2024-04-04T06:08:57.614647+0000","last_active":"2024-04-04T06:08:59.400387+0000","last_peered":"2024-04-04T06:08:59.400387+0000","last_clean":"2024-04-04T06:08:59.400387+0000","last_became_active":"2024-04-04T06:08:57.614442+0000","last_became_peered":"2024-04-04T06:08:57.614442+0000","last_unstale":"2024-04-04T06:08:59.400387+0000","last_undegraded":"2024-04-04T06:08:59.400387+0000","last_fullsized":"2024-04-04T06:08:59.400387+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:28:57.631136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399675+0000","last_change":"2024-04-04T06:08:57.597330+0000","last_active":"2024-04-04T06:08:59.399675+0000","last_peered":"2024-04-04T06:08:59.399675+0000","last_clean":"2024-04-04T06:08:59.399675+0000","last_became_active":"2024-04-04T06:08:57.596962+0000","last_became_peered":"2024-04-04T06:08:57.596962+0000","last_unstale":"2024-04-04T06:08:59.399675+0000","last_undegraded":"2024-04-04T06:08:59.399675+0000","last_fullsized":"2024-04-04T06:08:59.399675+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:25:44.282636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399587+0000","last_change":"2024-04-04T06:08:57.604574+0000","last_active":"2024-04-04T06:08:59.399587+0000","last_peered":"2024-04-04T06:08:59.399587+0000","last_clean":"2024-04-04T06:08:59.399587+0000","last_became_active":"2024-04-04T06:08:57.603977+0000","last_became_peered":"2024-04-04T06:08:57.603977+0000","last_unstale":"2024-04-04T06:08:59.399587+0000","last_undegraded":"2024-04-04T06:08:59.399587+0000","last_fullsized":"2024-04-04T06:08:59.399587+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:57.042947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592871+0000","last_change":"2024-04-04T06:08:57.596645+0000","last_active":"2024-04-04T06:08:58.592871+0000","last_peered":"2024-04-04T06:08:58.592871+0000","last_clean":"2024-04-04T06:08:58.592871+0000","last_became_active":"2024-04-04T06:08:57.596379+0000","last_became_peered":"2024-04-04T06:08:57.596379+0000","last_unstale":"2024-04-04T06:08:58.592871+0000","last_undegraded":"2024-04-04T06:08:58.592871+0000","last_fullsized":"2024-04-04T06:08:58.592871+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:47:02.001001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.593457+0000","last_change":"2024-04-04T06:08:57.618024+0000","last_active":"2024-04-04T06:08:58.593457+0000","last_peered":"2024-04-04T06:08:58.593457+0000","last_clean":"2024-04-04T06:08:58.593457+0000","last_became_active":"2024-04-04T06:08:57.617674+0000","last_became_peered":"2024-04-04T06:08:57.617674+0000","last_unstale":"2024-04-04T06:08:58.593457+0000","last_undegraded":"2024-04-04T06:08:58.593457+0000","last_fullsized":"2024-04-04T06:08:58.593457+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:09:46.418705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399602+0000","last_change":"2024-04-04T06:08:57.592018+0000","last_active":"2024-04-04T06:08:59.399602+0000","last_peered":"2024-04-04T06:08:59.399602+0000","last_clean":"2024-04-04T06:08:59.399602+0000","last_became_active":"2024-04-04T06:08:57.591670+0000","last_became_peered":"2024-04-04T06:08:57.591670+0000","last_unstale":"2024-04-04T06:08:59.399602+0000","last_undegraded":"2024-04-04T06:08:59.399602+0000","last_fullsized":"2024-04-04T06:08:59.399602+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:42:12.398269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.400126+0000","last_change":"2024-04-04T06:08:57.589250+0000","last_active":"2024-04-04T06:08:59.400126+0000","last_peered":"2024-04-04T06:08:59.400126+0000","last_clean":"2024-04-04T06:08:59.400126+0000","last_became_active":"2024-04-04T06:08:57.588786+0000","last_became_peered":"2024-04-04T06:08:57.588786+0000","last_unstale":"2024-04-04T06:08:59.400126+0000","last_undegraded":"2024-04-04T06:08:59.400126+0000","last_fullsized":"2024-04-04T06:08:59.400126+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:58:28.230236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604781+0000","last_change":"2024-04-04T06:08:57.611129+0000","last_active":"2024-04-04T06:08:58.604781+0000","last_peered":"2024-04-04T06:08:58.604781+0000","last_clean":"2024-04-04T06:08:58.604781+0000","last_became_active":"2024-04-04T06:08:57.610825+0000","last_became_peered":"2024-04-04T06:08:57.610825+0000","last_unstale":"2024-04-04T06:08:58.604781+0000","last_undegraded":"2024-04-04T06:08:58.604781+0000","last_fullsized":"2024-04-04T06:08:58.604781+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:25:02.372768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.111430+0000","last_change":"2024-04-04T06:08:57.606441+0000","last_active":"2024-04-04T06:08:59.111430+0000","last_peered":"2024-04-04T06:08:59.111430+0000","last_clean":"2024-04-04T06:08:59.111430+0000","last_became_active":"2024-04-04T06:08:57.606119+0000","last_became_peered":"2024-04-04T06:08:57.606119+0000","last_unstale":"2024-04-04T06:08:59.111430+0000","last_undegraded":"2024-04-04T06:08:59.111430+0000","last_fullsized":"2024-04-04T06:08:59.111430+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:42:55.934487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351721+0000","last_change":"2024-04-04T06:08:57.617716+0000","last_active":"2024-04-04T06:08:59.351721+0000","last_peered":"2024-04-04T06:08:59.351721+0000","last_clean":"2024-04-04T06:08:59.351721+0000","last_became_active":"2024-04-04T06:08:57.617385+0000","last_became_peered":"2024-04-04T06:08:57.617385+0000","last_unstale":"2024-04-04T06:08:59.351721+0000","last_undegraded":"2024-04-04T06:08:59.351721+0000","last_fullsized":"2024-04-04T06:08:59.351721+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:43:00.036899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.587158+0000","last_change":"2024-04-04T06:08:57.593659+0000","last_active":"2024-04-04T06:08:58.587158+0000","last_peered":"2024-04-04T06:08:58.587158+0000","last_clean":"2024-04-04T06:08:58.587158+0000","last_became_active":"2024-04-04T06:08:57.593382+0000","last_became_peered":"2024-04-04T06:08:57.593382+0000","last_unstale":"2024-04-04T06:08:58.587158+0000","last_undegraded":"2024-04-04T06:08:58.587158+0000","last_fullsized":"2024-04-04T06:08:58.587158+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:11:49.907094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110697+0000","last_change":"2024-04-04T06:08:57.619302+0000","last_active":"2024-04-04T06:08:59.110697+0000","last_peered":"2024-04-04T06:08:59.110697+0000","last_clean":"2024-04-04T06:08:59.110697+0000","last_became_active":"2024-04-04T06:08:57.618701+0000","last_became_peered":"2024-04-04T06:08:57.618701+0000","last_unstale":"2024-04-04T06:08:59.110697+0000","last_undegraded":"2024-04-04T06:08:59.110697+0000","last_fullsized":"2024-04-04T06:08:59.110697+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:29:01.526400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351752+0000","last_change":"2024-04-04T06:08:57.602399+0000","last_active":"2024-04-04T06:08:59.351752+0000","last_peered":"2024-04-04T06:08:59.351752+0000","last_clean":"2024-04-04T06:08:59.351752+0000","last_became_active":"2024-04-04T06:08:57.602074+0000","last_became_peered":"2024-04-04T06:08:57.602074+0000","last_unstale":"2024-04-04T06:08:59.351752+0000","last_undegraded":"2024-04-04T06:08:59.351752+0000","last_fullsized":"2024-04-04T06:08:59.351752+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:38:40.159820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110320+0000","last_change":"2024-04-04T06:08:57.603092+0000","last_active":"2024-04-04T06:08:59.110320+0000","last_peered":"2024-04-04T06:08:59.110320+0000","last_clean":"2024-04-04T06:08:59.110320+0000","last_became_active":"2024-04-04T06:08:57.602639+0000","last_became_peered":"2024-04-04T06:08:57.602639+0000","last_unstale":"2024-04-04T06:08:59.110320+0000","last_undegraded":"2024-04-04T06:08:59.110320+0000","last_fullsized":"2024-04-04T06:08:59.110320+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:35:44.572604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351023+0000","last_change":"2024-04-04T06:08:57.608928+0000","last_active":"2024-04-04T06:08:59.351023+0000","last_peered":"2024-04-04T06:08:59.351023+0000","last_clean":"2024-04-04T06:08:59.351023+0000","last_became_active":"2024-04-04T06:08:57.608146+0000","last_became_peered":"2024-04-04T06:08:57.608146+0000","last_unstale":"2024-04-04T06:08:59.351023+0000","last_undegraded":"2024-04-04T06:08:59.351023+0000","last_fullsized":"2024-04-04T06:08:59.351023+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:12:26.107777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.590550+0000","last_change":"2024-04-04T06:08:57.617880+0000","last_active":"2024-04-04T06:08:58.590550+0000","last_peered":"2024-04-04T06:08:58.590550+0000","last_clean":"2024-04-04T06:08:58.590550+0000","last_became_active":"2024-04-04T06:08:57.617508+0000","last_became_peered":"2024-04-04T06:08:57.617508+0000","last_unstale":"2024-04-04T06:08:58.590550+0000","last_undegraded":"2024-04-04T06:08:58.590550+0000","last_fullsized":"2024-04-04T06:08:58.590550+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:08:22.262253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.587766+0000","last_change":"2024-04-04T06:08:57.597236+0000","last_active":"2024-04-04T06:08:58.587766+0000","last_peered":"2024-04-04T06:08:58.587766+0000","last_clean":"2024-04-04T06:08:58.587766+0000","last_became_active":"2024-04-04T06:08:57.596931+0000","last_became_peered":"2024-04-04T06:08:57.596931+0000","last_unstale":"2024-04-04T06:08:58.587766+0000","last_undegraded":"2024-04-04T06:08:58.587766+0000","last_fullsized":"2024-04-04T06:08:58.587766+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:20:29.526370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592533+0000","last_change":"2024-04-04T06:08:57.592246+0000","last_active":"2024-04-04T06:08:58.592533+0000","last_peered":"2024-04-04T06:08:58.592533+0000","last_clean":"2024-04-04T06:08:58.592533+0000","last_became_active":"2024-04-04T06:08:57.591586+0000","last_became_peered":"2024-04-04T06:08:57.591586+0000","last_unstale":"2024-04-04T06:08:58.592533+0000","last_undegraded":"2024-04-04T06:08:58.592533+0000","last_fullsized":"2024-04-04T06:08:58.592533+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:53:17.231738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604692+0000","last_change":"2024-04-04T06:08:57.614655+0000","last_active":"2024-04-04T06:08:58.604692+0000","last_peered":"2024-04-04T06:08:58.604692+0000","last_clean":"2024-04-04T06:08:58.604692+0000","last_became_active":"2024-04-04T06:08:57.611616+0000","last_became_peered":"2024-04-04T06:08:57.611616+0000","last_unstale":"2024-04-04T06:08:58.604692+0000","last_undegraded":"2024-04-04T06:08:58.604692+0000","last_fullsized":"2024-04-04T06:08:58.604692+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:22:19.057088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.587204+0000","last_change":"2024-04-04T06:08:57.603638+0000","last_active":"2024-04-04T06:08:58.587204+0000","last_peered":"2024-04-04T06:08:58.587204+0000","last_clean":"2024-04-04T06:08:58.587204+0000","last_became_active":"2024-04-04T06:08:57.600991+0000","last_became_peered":"2024-04-04T06:08:57.600991+0000","last_unstale":"2024-04-04T06:08:58.587204+0000","last_undegraded":"2024-04-04T06:08:58.587204+0000","last_fullsized":"2024-04-04T06:08:58.587204+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:20:16.128711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110731+0000","last_change":"2024-04-04T06:08:57.601986+0000","last_active":"2024-04-04T06:08:59.110731+0000","last_peered":"2024-04-04T06:08:59.110731+0000","last_clean":"2024-04-04T06:08:59.110731+0000","last_became_active":"2024-04-04T06:08:57.601677+0000","last_became_peered":"2024-04-04T06:08:57.601677+0000","last_unstale":"2024-04-04T06:08:59.110731+0000","last_undegraded":"2024-04-04T06:08:59.110731+0000","last_fullsized":"2024-04-04T06:08:59.110731+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:39:27.630288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351327+0000","last_change":"2024-04-04T06:08:57.600183+0000","last_active":"2024-04-04T06:08:59.351327+0000","last_peered":"2024-04-04T06:08:59.351327+0000","last_clean":"2024-04-04T06:08:59.351327+0000","last_became_active":"2024-04-04T06:08:57.599862+0000","last_became_peered":"2024-04-04T06:08:57.599862+0000","last_unstale":"2024-04-04T06:08:59.351327+0000","last_undegraded":"2024-04-04T06:08:59.351327+0000","last_fullsized":"2024-04-04T06:08:59.351327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:29:36.805186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.586574+0000","last_change":"2024-04-04T06:08:57.589983+0000","last_active":"2024-04-04T06:08:58.586574+0000","last_peered":"2024-04-04T06:08:58.586574+0000","last_clean":"2024-04-04T06:08:58.586574+0000","last_became_active":"2024-04-04T06:08:57.589734+0000","last_became_peered":"2024-04-04T06:08:57.589734+0000","last_unstale":"2024-04-04T06:08:58.586574+0000","last_undegraded":"2024-04-04T06:08:58.586574+0000","last_fullsized":"2024-04-04T06:08:58.586574+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:01:08.452110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.605135+0000","last_change":"2024-04-04T06:08:57.609822+0000","last_active":"2024-04-04T06:08:58.605135+0000","last_peered":"2024-04-04T06:08:58.605135+0000","last_clean":"2024-04-04T06:08:58.605135+0000","last_became_active":"2024-04-04T06:08:57.609463+0000","last_became_peered":"2024-04-04T06:08:57.609463+0000","last_unstale":"2024-04-04T06:08:58.605135+0000","last_undegraded":"2024-04-04T06:08:58.605135+0000","last_fullsized":"2024-04-04T06:08:58.605135+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:15:03.941901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.605120+0000","last_change":"2024-04-04T06:08:57.613074+0000","last_active":"2024-04-04T06:08:58.605120+0000","last_peered":"2024-04-04T06:08:58.605120+0000","last_clean":"2024-04-04T06:08:58.605120+0000","last_became_active":"2024-04-04T06:08:57.609879+0000","last_became_peered":"2024-04-04T06:08:57.609879+0000","last_unstale":"2024-04-04T06:08:58.605120+0000","last_undegraded":"2024-04-04T06:08:58.605120+0000","last_fullsized":"2024-04-04T06:08:58.605120+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:01:08.338874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.605241+0000","last_change":"2024-04-04T06:08:56.578928+0000","last_active":"2024-04-04T06:08:58.605241+0000","last_peered":"2024-04-04T06:08:58.605241+0000","last_clean":"2024-04-04T06:08:58.605241+0000","last_became_active":"2024-04-04T06:08:56.578602+0000","last_became_peered":"2024-04-04T06:08:56.578602+0000","last_unstale":"2024-04-04T06:08:58.605241+0000","last_undegraded":"2024-04-04T06:08:58.605241+0000","last_fullsized":"2024-04-04T06:08:58.605241+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:30:18.473540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399701+0000","last_change":"2024-04-04T06:08:57.599563+0000","last_active":"2024-04-04T06:08:59.399701+0000","last_peered":"2024-04-04T06:08:59.399701+0000","last_clean":"2024-04-04T06:08:59.399701+0000","last_became_active":"2024-04-04T06:08:57.599244+0000","last_became_peered":"2024-04-04T06:08:57.599244+0000","last_unstale":"2024-04-04T06:08:59.399701+0000","last_undegraded":"2024-04-04T06:08:59.399701+0000","last_fullsized":"2024-04-04T06:08:59.399701+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:16:21.695529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110705+0000","last_change":"2024-04-04T06:08:56.577361+0000","last_active":"2024-04-04T06:08:59.110705+0000","last_peered":"2024-04-04T06:08:59.110705+0000","last_clean":"2024-04-04T06:08:59.110705+0000","last_became_active":"2024-04-04T06:08:56.576619+0000","last_became_peered":"2024-04-04T06:08:56.576619+0000","last_unstale":"2024-04-04T06:08:59.110705+0000","last_undegraded":"2024-04-04T06:08:59.110705+0000","last_fullsized":"2024-04-04T06:08:59.110705+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:53:16.217692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604774+0000","last_change":"2024-04-04T06:08:57.611238+0000","last_active":"2024-04-04T06:08:58.604774+0000","last_peered":"2024-04-04T06:08:58.604774+0000","last_clean":"2024-04-04T06:08:58.604774+0000","last_became_active":"2024-04-04T06:08:57.610966+0000","last_became_peered":"2024-04-04T06:08:57.610966+0000","last_unstale":"2024-04-04T06:08:58.604774+0000","last_undegraded":"2024-04-04T06:08:58.604774+0000","last_fullsized":"2024-04-04T06:08:58.604774+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:10:11.718125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604997+0000","last_change":"2024-04-04T06:08:56.577805+0000","last_active":"2024-04-04T06:08:58.604997+0000","last_peered":"2024-04-04T06:08:58.604997+0000","last_clean":"2024-04-04T06:08:58.604997+0000","last_became_active":"2024-04-04T06:08:56.576393+0000","last_became_peered":"2024-04-04T06:08:56.576393+0000","last_unstale":"2024-04-04T06:08:58.604997+0000","last_undegraded":"2024-04-04T06:08:58.604997+0000","last_fullsized":"2024-04-04T06:08:58.604997+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:16:13.187412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604859+0000","last_change":"2024-04-04T06:08:57.614436+0000","last_active":"2024-04-04T06:08:58.604859+0000","last_peered":"2024-04-04T06:08:58.604859+0000","last_clean":"2024-04-04T06:08:58.604859+0000","last_became_active":"2024-04-04T06:08:57.609947+0000","last_became_peered":"2024-04-04T06:08:57.609947+0000","last_unstale":"2024-04-04T06:08:58.604859+0000","last_undegraded":"2024-04-04T06:08:58.604859+0000","last_fullsized":"2024-04-04T06:08:58.604859+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:57:01.275554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592942+0000","last_change":"2024-04-04T06:08:56.578999+0000","last_active":"2024-04-04T06:08:58.592942+0000","last_peered":"2024-04-04T06:08:58.592942+0000","last_clean":"2024-04-04T06:08:58.592942+0000","last_became_active":"2024-04-04T06:08:56.578689+0000","last_became_peered":"2024-04-04T06:08:56.578689+0000","last_unstale":"2024-04-04T06:08:58.592942+0000","last_undegraded":"2024-04-04T06:08:58.592942+0000","last_fullsized":"2024-04-04T06:08:58.592942+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:30:18.473540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110373+0000","last_change":"2024-04-04T06:08:57.600474+0000","last_active":"2024-04-04T06:08:59.110373+0000","last_peered":"2024-04-04T06:08:59.110373+0000","last_clean":"2024-04-04T06:08:59.110373+0000","last_became_active":"2024-04-04T06:08:57.600135+0000","last_became_peered":"2024-04-04T06:08:57.600135+0000","last_unstale":"2024-04-04T06:08:59.110373+0000","last_undegraded":"2024-04-04T06:08:59.110373+0000","last_fullsized":"2024-04-04T06:08:59.110373+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:08:50.665613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399510+0000","last_change":"2024-04-04T06:08:56.578612+0000","last_active":"2024-04-04T06:08:59.399510+0000","last_peered":"2024-04-04T06:08:59.399510+0000","last_clean":"2024-04-04T06:08:59.399510+0000","last_became_active":"2024-04-04T06:08:56.575989+0000","last_became_peered":"2024-04-04T06:08:56.575989+0000","last_unstale":"2024-04-04T06:08:59.399510+0000","last_undegraded":"2024-04-04T06:08:59.399510+0000","last_fullsized":"2024-04-04T06:08:59.399510+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:56.028901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110635+0000","last_change":"2024-04-04T06:08:57.617863+0000","last_active":"2024-04-04T06:08:59.110635+0000","last_peered":"2024-04-04T06:08:59.110635+0000","last_clean":"2024-04-04T06:08:59.110635+0000","last_became_active":"2024-04-04T06:08:57.617572+0000","last_became_peered":"2024-04-04T06:08:57.617572+0000","last_unstale":"2024-04-04T06:08:59.110635+0000","last_undegraded":"2024-04-04T06:08:59.110635+0000","last_fullsized":"2024-04-04T06:08:59.110635+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:58:22.358023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604504+0000","last_change":"2024-04-04T06:08:56.579024+0000","last_active":"2024-04-04T06:08:58.604504+0000","last_peered":"2024-04-04T06:08:58.604504+0000","last_clean":"2024-04-04T06:08:58.604504+0000","last_became_active":"2024-04-04T06:08:56.578301+0000","last_became_peered":"2024-04-04T06:08:56.578301+0000","last_unstale":"2024-04-04T06:08:58.604504+0000","last_undegraded":"2024-04-04T06:08:58.604504+0000","last_fullsized":"2024-04-04T06:08:58.604504+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:39:18.014934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.590603+0000","last_change":"2024-04-04T06:08:57.602059+0000","last_active":"2024-04-04T06:08:58.590603+0000","last_peered":"2024-04-04T06:08:58.590603+0000","last_clean":"2024-04-04T06:08:58.590603+0000","last_became_active":"2024-04-04T06:08:57.601116+0000","last_became_peered":"2024-04-04T06:08:57.601116+0000","last_unstale":"2024-04-04T06:08:58.590603+0000","last_undegraded":"2024-04-04T06:08:58.590603+0000","last_fullsized":"2024-04-04T06:08:58.590603+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:17:59.892459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591333+0000","last_change":"2024-04-04T06:08:56.571563+0000","last_active":"2024-04-04T06:08:58.591333+0000","last_peered":"2024-04-04T06:08:58.591333+0000","last_clean":"2024-04-04T06:08:58.591333+0000","last_became_active":"2024-04-04T06:08:56.571092+0000","last_became_peered":"2024-04-04T06:08:56.571092+0000","last_unstale":"2024-04-04T06:08:58.591333+0000","last_undegraded":"2024-04-04T06:08:58.591333+0000","last_fullsized":"2024-04-04T06:08:58.591333+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:03:42.180669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110248+0000","last_change":"2024-04-04T06:08:57.608674+0000","last_active":"2024-04-04T06:08:59.110248+0000","last_peered":"2024-04-04T06:08:59.110248+0000","last_clean":"2024-04-04T06:08:59.110248+0000","last_became_active":"2024-04-04T06:08:57.608433+0000","last_became_peered":"2024-04-04T06:08:57.608433+0000","last_unstale":"2024-04-04T06:08:59.110248+0000","last_undegraded":"2024-04-04T06:08:59.110248+0000","last_fullsized":"2024-04-04T06:08:59.110248+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:30:51.961748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399303+0000","last_change":"2024-04-04T06:08:56.579353+0000","last_active":"2024-04-04T06:08:59.399303+0000","last_peered":"2024-04-04T06:08:59.399303+0000","last_clean":"2024-04-04T06:08:59.399303+0000","last_became_active":"2024-04-04T06:08:56.576387+0000","last_became_peered":"2024-04-04T06:08:56.576387+0000","last_unstale":"2024-04-04T06:08:59.399303+0000","last_undegraded":"2024-04-04T06:08:59.399303+0000","last_fullsized":"2024-04-04T06:08:59.399303+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:03:42.180669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.400514+0000","last_change":"2024-04-04T06:08:57.590692+0000","last_active":"2024-04-04T06:08:59.400514+0000","last_peered":"2024-04-04T06:08:59.400514+0000","last_clean":"2024-04-04T06:08:59.400514+0000","last_became_active":"2024-04-04T06:08:57.590158+0000","last_became_peered":"2024-04-04T06:08:57.590158+0000","last_unstale":"2024-04-04T06:08:59.400514+0000","last_undegraded":"2024-04-04T06:08:59.400514+0000","last_fullsized":"2024-04-04T06:08:59.400514+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:24:51.708795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.352306+0000","last_change":"2024-04-04T06:08:56.587466+0000","last_active":"2024-04-04T06:08:59.352306+0000","last_peered":"2024-04-04T06:08:59.352306+0000","last_clean":"2024-04-04T06:08:59.352306+0000","last_became_active":"2024-04-04T06:08:56.587163+0000","last_became_peered":"2024-04-04T06:08:56.587163+0000","last_unstale":"2024-04-04T06:08:59.352306+0000","last_undegraded":"2024-04-04T06:08:59.352306+0000","last_fullsized":"2024-04-04T06:08:59.352306+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:52:42.577594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.352046+0000","last_change":"2024-04-04T06:08:57.609373+0000","last_active":"2024-04-04T06:08:59.352046+0000","last_peered":"2024-04-04T06:08:59.352046+0000","last_clean":"2024-04-04T06:08:59.352046+0000","last_became_active":"2024-04-04T06:08:57.606670+0000","last_became_peered":"2024-04-04T06:08:57.606670+0000","last_unstale":"2024-04-04T06:08:59.352046+0000","last_undegraded":"2024-04-04T06:08:59.352046+0000","last_fullsized":"2024-04-04T06:08:59.352046+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:17:58.054272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110980+0000","last_change":"2024-04-04T06:08:56.581539+0000","last_active":"2024-04-04T06:08:59.110980+0000","last_peered":"2024-04-04T06:08:59.110980+0000","last_clean":"2024-04-04T06:08:59.110980+0000","last_became_active":"2024-04-04T06:08:56.579490+0000","last_became_peered":"2024-04-04T06:08:56.579490+0000","last_unstale":"2024-04-04T06:08:59.110980+0000","last_undegraded":"2024-04-04T06:08:59.110980+0000","last_fullsized":"2024-04-04T06:08:59.110980+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:39:55.913297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399236+0000","last_change":"2024-04-04T06:08:57.588887+0000","last_active":"2024-04-04T06:08:59.399236+0000","last_peered":"2024-04-04T06:08:59.399236+0000","last_clean":"2024-04-04T06:08:59.399236+0000","last_became_active":"2024-04-04T06:08:57.588615+0000","last_became_peered":"2024-04-04T06:08:57.588615+0000","last_unstale":"2024-04-04T06:08:59.399236+0000","last_undegraded":"2024-04-04T06:08:59.399236+0000","last_fullsized":"2024-04-04T06:08:59.399236+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:29:09.919535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.352507+0000","last_change":"2024-04-04T06:08:56.582280+0000","last_active":"2024-04-04T06:08:59.352507+0000","last_peered":"2024-04-04T06:08:59.352507+0000","last_clean":"2024-04-04T06:08:59.352507+0000","last_became_active":"2024-04-04T06:08:56.581968+0000","last_became_peered":"2024-04-04T06:08:56.581968+0000","last_unstale":"2024-04-04T06:08:59.352507+0000","last_undegraded":"2024-04-04T06:08:59.352507+0000","last_fullsized":"2024-04-04T06:08:59.352507+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:34:17.865573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.352230+0000","last_change":"2024-04-04T06:08:57.604234+0000","last_active":"2024-04-04T06:08:59.352230+0000","last_peered":"2024-04-04T06:08:59.352230+0000","last_clean":"2024-04-04T06:08:59.352230+0000","last_became_active":"2024-04-04T06:08:57.602743+0000","last_became_peered":"2024-04-04T06:08:57.602743+0000","last_unstale":"2024-04-04T06:08:59.352230+0000","last_undegraded":"2024-04-04T06:08:59.352230+0000","last_fullsized":"2024-04-04T06:08:59.352230+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:52:53.825956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399994+0000","last_change":"2024-04-04T06:08:56.577639+0000","last_active":"2024-04-04T06:08:59.399994+0000","last_peered":"2024-04-04T06:08:59.399994+0000","last_clean":"2024-04-04T06:08:59.399994+0000","last_became_active":"2024-04-04T06:08:56.577372+0000","last_became_peered":"2024-04-04T06:08:56.577372+0000","last_unstale":"2024-04-04T06:08:59.399994+0000","last_undegraded":"2024-04-04T06:08:59.399994+0000","last_fullsized":"2024-04-04T06:08:59.399994+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:16:20.681483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591829+0000","last_change":"2024-04-04T06:08:57.601155+0000","last_active":"2024-04-04T06:08:58.591829+0000","last_peered":"2024-04-04T06:08:58.591829+0000","last_clean":"2024-04-04T06:08:58.591829+0000","last_became_active":"2024-04-04T06:08:57.600030+0000","last_became_peered":"2024-04-04T06:08:57.600030+0000","last_unstale":"2024-04-04T06:08:58.591829+0000","last_undegraded":"2024-04-04T06:08:58.591829+0000","last_fullsized":"2024-04-04T06:08:58.591829+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:46:59.204669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110522+0000","last_change":"2024-04-04T06:08:56.575082+0000","last_active":"2024-04-04T06:08:59.110522+0000","last_peered":"2024-04-04T06:08:59.110522+0000","last_clean":"2024-04-04T06:08:59.110522+0000","last_became_active":"2024-04-04T06:08:56.574805+0000","last_became_peered":"2024-04-04T06:08:56.574805+0000","last_unstale":"2024-04-04T06:08:59.110522+0000","last_undegraded":"2024-04-04T06:08:59.110522+0000","last_fullsized":"2024-04-04T06:08:59.110522+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:46:58.190623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351568+0000","last_change":"2024-04-04T06:08:57.607510+0000","last_active":"2024-04-04T06:08:59.351568+0000","last_peered":"2024-04-04T06:08:59.351568+0000","last_clean":"2024-04-04T06:08:59.351568+0000","last_became_active":"2024-04-04T06:08:57.606804+0000","last_became_peered":"2024-04-04T06:08:57.606804+0000","last_unstale":"2024-04-04T06:08:59.351568+0000","last_undegraded":"2024-04-04T06:08:59.351568+0000","last_fullsized":"2024-04-04T06:08:59.351568+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:01:29.815756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591001+0000","last_change":"2024-04-04T06:08:56.576046+0000","last_active":"2024-04-04T06:08:58.591001+0000","last_peered":"2024-04-04T06:08:58.591001+0000","last_clean":"2024-04-04T06:08:58.591001+0000","last_became_active":"2024-04-04T06:08:56.575726+0000","last_became_peered":"2024-04-04T06:08:56.575726+0000","last_unstale":"2024-04-04T06:08:58.591001+0000","last_undegraded":"2024-04-04T06:08:58.591001+0000","last_fullsized":"2024-04-04T06:08:58.591001+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:35:35.113120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399396+0000","last_change":"2024-04-04T06:08:57.599582+0000","last_active":"2024-04-04T06:08:59.399396+0000","last_peered":"2024-04-04T06:08:59.399396+0000","last_clean":"2024-04-04T06:08:59.399396+0000","last_became_active":"2024-04-04T06:08:57.598581+0000","last_became_peered":"2024-04-04T06:08:57.598581+0000","last_unstale":"2024-04-04T06:08:59.399396+0000","last_undegraded":"2024-04-04T06:08:59.399396+0000","last_fullsized":"2024-04-04T06:08:59.399396+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:46:55.241650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591592+0000","last_change":"2024-04-04T06:08:56.576601+0000","last_active":"2024-04-04T06:08:58.591592+0000","last_peered":"2024-04-04T06:08:58.591592+0000","last_clean":"2024-04-04T06:08:58.591592+0000","last_became_active":"2024-04-04T06:08:56.572894+0000","last_became_peered":"2024-04-04T06:08:56.572894+0000","last_unstale":"2024-04-04T06:08:58.591592+0000","last_undegraded":"2024-04-04T06:08:58.591592+0000","last_fullsized":"2024-04-04T06:08:58.591592+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:16:13.187412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351248+0000","last_change":"2024-04-04T06:08:57.610067+0000","last_active":"2024-04-04T06:08:59.351248+0000","last_peered":"2024-04-04T06:08:59.351248+0000","last_clean":"2024-04-04T06:08:59.351248+0000","last_became_active":"2024-04-04T06:08:57.609770+0000","last_became_peered":"2024-04-04T06:08:57.609770+0000","last_unstale":"2024-04-04T06:08:59.351248+0000","last_undegraded":"2024-04-04T06:08:59.351248+0000","last_fullsized":"2024-04-04T06:08:59.351248+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:43:34.312451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.350950+0000","last_change":"2024-04-04T06:08:56.582998+0000","last_active":"2024-04-04T06:08:59.350950+0000","last_peered":"2024-04-04T06:08:59.350950+0000","last_clean":"2024-04-04T06:08:59.350950+0000","last_became_active":"2024-04-04T06:08:56.582720+0000","last_became_peered":"2024-04-04T06:08:56.582720+0000","last_unstale":"2024-04-04T06:08:59.350950+0000","last_undegraded":"2024-04-04T06:08:59.350950+0000","last_fullsized":"2024-04-04T06:08:59.350950+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:48:07.678914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.352120+0000","last_change":"2024-04-04T06:08:57.600691+0000","last_active":"2024-04-04T06:08:59.352120+0000","last_peered":"2024-04-04T06:08:59.352120+0000","last_clean":"2024-04-04T06:08:59.352120+0000","last_became_active":"2024-04-04T06:08:57.600389+0000","last_became_peered":"2024-04-04T06:08:57.600389+0000","last_unstale":"2024-04-04T06:08:59.352120+0000","last_undegraded":"2024-04-04T06:08:59.352120+0000","last_fullsized":"2024-04-04T06:08:59.352120+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:49:27.426679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.111161+0000","last_change":"2024-04-04T06:08:56.575119+0000","last_active":"2024-04-04T06:08:59.111161+0000","last_peered":"2024-04-04T06:08:59.111161+0000","last_clean":"2024-04-04T06:08:59.111161+0000","last_became_active":"2024-04-04T06:08:56.574832+0000","last_became_peered":"2024-04-04T06:08:56.574832+0000","last_unstale":"2024-04-04T06:08:59.111161+0000","last_undegraded":"2024-04-04T06:08:59.111161+0000","last_fullsized":"2024-04-04T06:08:59.111161+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:47:00.986955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592375+0000","last_change":"2024-04-04T06:08:57.600113+0000","last_active":"2024-04-04T06:08:58.592375+0000","last_peered":"2024-04-04T06:08:58.592375+0000","last_clean":"2024-04-04T06:08:58.592375+0000","last_became_active":"2024-04-04T06:08:57.599658+0000","last_became_peered":"2024-04-04T06:08:57.599658+0000","last_unstale":"2024-04-04T06:08:58.592375+0000","last_undegraded":"2024-04-04T06:08:58.592375+0000","last_fullsized":"2024-04-04T06:08:58.592375+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:09:31.789910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.605123+0000","last_change":"2024-04-04T06:08:56.576721+0000","last_active":"2024-04-04T06:08:58.605123+0000","last_peered":"2024-04-04T06:08:58.605123+0000","last_clean":"2024-04-04T06:08:58.605123+0000","last_became_active":"2024-04-04T06:08:56.576404+0000","last_became_peered":"2024-04-04T06:08:56.576404+0000","last_unstale":"2024-04-04T06:08:58.605123+0000","last_undegraded":"2024-04-04T06:08:58.605123+0000","last_fullsized":"2024-04-04T06:08:58.605123+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:30:05.745690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110887+0000","last_change":"2024-04-04T06:08:57.605056+0000","last_active":"2024-04-04T06:08:59.110887+0000","last_peered":"2024-04-04T06:08:59.110887+0000","last_clean":"2024-04-04T06:08:59.110887+0000","last_became_active":"2024-04-04T06:08:57.604780+0000","last_became_peered":"2024-04-04T06:08:57.604780+0000","last_unstale":"2024-04-04T06:08:59.110887+0000","last_undegraded":"2024-04-04T06:08:59.110887+0000","last_fullsized":"2024-04-04T06:08:59.110887+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:26:29.014310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592694+0000","last_change":"2024-04-04T06:08:56.579736+0000","last_active":"2024-04-04T06:08:58.592694+0000","last_peered":"2024-04-04T06:08:58.592694+0000","last_clean":"2024-04-04T06:08:58.592694+0000","last_became_active":"2024-04-04T06:08:56.579470+0000","last_became_peered":"2024-04-04T06:08:56.579470+0000","last_unstale":"2024-04-04T06:08:58.592694+0000","last_undegraded":"2024-04-04T06:08:58.592694+0000","last_fullsized":"2024-04-04T06:08:58.592694+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:16:20.681483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.352514+0000","last_change":"2024-04-04T06:08:57.605903+0000","last_active":"2024-04-04T06:08:59.352514+0000","last_peered":"2024-04-04T06:08:59.352514+0000","last_clean":"2024-04-04T06:08:59.352514+0000","last_became_active":"2024-04-04T06:08:57.605458+0000","last_became_peered":"2024-04-04T06:08:57.605458+0000","last_unstale":"2024-04-04T06:08:59.352514+0000","last_undegraded":"2024-04-04T06:08:59.352514+0000","last_fullsized":"2024-04-04T06:08:59.352514+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:33:51.458736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604894+0000","last_change":"2024-04-04T06:08:56.577972+0000","last_active":"2024-04-04T06:08:58.604894+0000","last_peered":"2024-04-04T06:08:58.604894+0000","last_clean":"2024-04-04T06:08:58.604894+0000","last_became_active":"2024-04-04T06:08:56.577700+0000","last_became_peered":"2024-04-04T06:08:56.577700+0000","last_unstale":"2024-04-04T06:08:58.604894+0000","last_undegraded":"2024-04-04T06:08:58.604894+0000","last_fullsized":"2024-04-04T06:08:58.604894+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:37:53.831405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591961+0000","last_change":"2024-04-04T06:08:57.592739+0000","last_active":"2024-04-04T06:08:58.591961+0000","last_peered":"2024-04-04T06:08:58.591961+0000","last_clean":"2024-04-04T06:08:58.591961+0000","last_became_active":"2024-04-04T06:08:57.590596+0000","last_became_peered":"2024-04-04T06:08:57.590596+0000","last_unstale":"2024-04-04T06:08:58.591961+0000","last_undegraded":"2024-04-04T06:08:58.591961+0000","last_fullsized":"2024-04-04T06:08:58.591961+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:25:23.457982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110789+0000","last_change":"2024-04-04T06:08:56.571770+0000","last_active":"2024-04-04T06:08:59.110789+0000","last_peered":"2024-04-04T06:08:59.110789+0000","last_clean":"2024-04-04T06:08:59.110789+0000","last_became_active":"2024-04-04T06:08:56.571431+0000","last_became_peered":"2024-04-04T06:08:56.571431+0000","last_unstale":"2024-04-04T06:08:59.110789+0000","last_undegraded":"2024-04-04T06:08:59.110789+0000","last_fullsized":"2024-04-04T06:08:59.110789+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:25:22.443936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604805+0000","last_change":"2024-04-04T06:08:57.613060+0000","last_active":"2024-04-04T06:08:58.604805+0000","last_peered":"2024-04-04T06:08:58.604805+0000","last_clean":"2024-04-04T06:08:58.604805+0000","last_became_active":"2024-04-04T06:08:57.612824+0000","last_became_peered":"2024-04-04T06:08:57.612824+0000","last_unstale":"2024-04-04T06:08:58.604805+0000","last_undegraded":"2024-04-04T06:08:58.604805+0000","last_fullsized":"2024-04-04T06:08:58.604805+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:54:50.560191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.398963+0000","last_change":"2024-04-04T06:08:56.574202+0000","last_active":"2024-04-04T06:08:59.398963+0000","last_peered":"2024-04-04T06:08:59.398963+0000","last_clean":"2024-04-04T06:08:59.398963+0000","last_became_active":"2024-04-04T06:08:56.573425+0000","last_became_peered":"2024-04-04T06:08:56.573425+0000","last_unstale":"2024-04-04T06:08:59.398963+0000","last_undegraded":"2024-04-04T06:08:59.398963+0000","last_fullsized":"2024-04-04T06:08:59.398963+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:09:27.272946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604465+0000","last_change":"2024-04-04T06:08:57.612462+0000","last_active":"2024-04-04T06:08:58.604465+0000","last_peered":"2024-04-04T06:08:58.604465+0000","last_clean":"2024-04-04T06:08:58.604465+0000","last_became_active":"2024-04-04T06:08:57.611941+0000","last_became_peered":"2024-04-04T06:08:57.611941+0000","last_unstale":"2024-04-04T06:08:58.604465+0000","last_undegraded":"2024-04-04T06:08:58.604465+0000","last_fullsized":"2024-04-04T06:08:58.604465+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:39:56.927343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591348+0000","last_change":"2024-04-04T06:08:56.575741+0000","last_active":"2024-04-04T06:08:58.591348+0000","last_peered":"2024-04-04T06:08:58.591348+0000","last_clean":"2024-04-04T06:08:58.591348+0000","last_became_active":"2024-04-04T06:08:56.575438+0000","last_became_peered":"2024-04-04T06:08:56.575438+0000","last_unstale":"2024-04-04T06:08:58.591348+0000","last_undegraded":"2024-04-04T06:08:58.591348+0000","last_fullsized":"2024-04-04T06:08:58.591348+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:46:32.849014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592972+0000","last_change":"2024-04-04T06:08:57.596280+0000","last_active":"2024-04-04T06:08:58.592972+0000","last_peered":"2024-04-04T06:08:58.592972+0000","last_clean":"2024-04-04T06:08:58.592972+0000","last_became_active":"2024-04-04T06:08:57.595982+0000","last_became_peered":"2024-04-04T06:08:57.595982+0000","last_unstale":"2024-04-04T06:08:58.592972+0000","last_undegraded":"2024-04-04T06:08:58.592972+0000","last_fullsized":"2024-04-04T06:08:58.592972+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:10:11.718125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399701+0000","last_change":"2024-04-04T06:08:56.574911+0000","last_active":"2024-04-04T06:08:59.399701+0000","last_peered":"2024-04-04T06:08:59.399701+0000","last_clean":"2024-04-04T06:08:59.399701+0000","last_became_active":"2024-04-04T06:08:56.573037+0000","last_became_peered":"2024-04-04T06:08:56.573037+0000","last_unstale":"2024-04-04T06:08:59.399701+0000","last_undegraded":"2024-04-04T06:08:59.399701+0000","last_fullsized":"2024-04-04T06:08:59.399701+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:15:13.550828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592938+0000","last_change":"2024-04-04T06:08:57.602162+0000","last_active":"2024-04-04T06:08:58.592938+0000","last_peered":"2024-04-04T06:08:58.592938+0000","last_clean":"2024-04-04T06:08:58.592938+0000","last_became_active":"2024-04-04T06:08:57.601840+0000","last_became_peered":"2024-04-04T06:08:57.601840+0000","last_unstale":"2024-04-04T06:08:58.592938+0000","last_undegraded":"2024-04-04T06:08:58.592938+0000","last_fullsized":"2024-04-04T06:08:58.592938+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:22:19.057088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"59'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.605323+0000","last_change":"2024-04-04T06:08:56.578575+0000","last_active":"2024-04-04T06:08:58.605323+0000","last_peered":"2024-04-04T06:08:58.605323+0000","last_clean":"2024-04-04T06:08:58.605323+0000","last_became_active":"2024-04-04T06:08:56.578232+0000","last_became_peered":"2024-04-04T06:08:56.578232+0000","last_unstale":"2024-04-04T06:08:58.605323+0000","last_undegraded":"2024-04-04T06:08:58.605323+0000","last_fullsized":"2024-04-04T06:08:58.605323+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:46:54.227604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399457+0000","last_change":"2024-04-04T06:08:57.589470+0000","last_active":"2024-04-04T06:08:59.399457+0000","last_peered":"2024-04-04T06:08:59.399457+0000","last_clean":"2024-04-04T06:08:59.399457+0000","last_became_active":"2024-04-04T06:08:57.589193+0000","last_became_peered":"2024-04-04T06:08:57.589193+0000","last_unstale":"2024-04-04T06:08:59.399457+0000","last_undegraded":"2024-04-04T06:08:59.399457+0000","last_fullsized":"2024-04-04T06:08:59.399457+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:02:34.522323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591681+0000","last_change":"2024-04-04T06:08:56.573941+0000","last_active":"2024-04-04T06:08:58.591681+0000","last_peered":"2024-04-04T06:08:58.591681+0000","last_clean":"2024-04-04T06:08:58.591681+0000","last_became_active":"2024-04-04T06:08:56.573700+0000","last_became_peered":"2024-04-04T06:08:56.573700+0000","last_unstale":"2024-04-04T06:08:58.591681+0000","last_undegraded":"2024-04-04T06:08:58.591681+0000","last_fullsized":"2024-04-04T06:08:58.591681+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:03:55.295388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.605391+0000","last_change":"2024-04-04T06:08:57.611889+0000","last_active":"2024-04-04T06:08:58.605391+0000","last_peered":"2024-04-04T06:08:58.605391+0000","last_clean":"2024-04-04T06:08:58.605391+0000","last_became_active":"2024-04-04T06:08:57.609636+0000","last_became_peered":"2024-04-04T06:08:57.609636+0000","last_unstale":"2024-04-04T06:08:58.605391+0000","last_undegraded":"2024-04-04T06:08:58.605391+0000","last_fullsized":"2024-04-04T06:08:58.605391+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:50:39.260072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.593012+0000","last_change":"2024-04-04T06:08:56.577871+0000","last_active":"2024-04-04T06:08:58.593012+0000","last_peered":"2024-04-04T06:08:58.593012+0000","last_clean":"2024-04-04T06:08:58.593012+0000","last_became_active":"2024-04-04T06:08:56.577570+0000","last_became_peered":"2024-04-04T06:08:56.577570+0000","last_unstale":"2024-04-04T06:08:58.593012+0000","last_undegraded":"2024-04-04T06:08:58.593012+0000","last_fullsized":"2024-04-04T06:08:58.593012+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:37:53.831405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110537+0000","last_change":"2024-04-04T06:08:57.606811+0000","last_active":"2024-04-04T06:08:59.110537+0000","last_peered":"2024-04-04T06:08:59.110537+0000","last_clean":"2024-04-04T06:08:59.110537+0000","last_became_active":"2024-04-04T06:08:57.606536+0000","last_became_peered":"2024-04-04T06:08:57.606536+0000","last_unstale":"2024-04-04T06:08:59.110537+0000","last_undegraded":"2024-04-04T06:08:59.110537+0000","last_fullsized":"2024-04-04T06:08:59.110537+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:10:33.843321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.398742+0000","last_change":"2024-04-04T06:08:56.569703+0000","last_active":"2024-04-04T06:08:59.398742+0000","last_peered":"2024-04-04T06:08:59.398742+0000","last_clean":"2024-04-04T06:08:59.398742+0000","last_became_active":"2024-04-04T06:08:56.569433+0000","last_became_peered":"2024-04-04T06:08:56.569433+0000","last_unstale":"2024-04-04T06:08:59.398742+0000","last_undegraded":"2024-04-04T06:08:59.398742+0000","last_fullsized":"2024-04-04T06:08:59.398742+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:28:25.503580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351982+0000","last_change":"2024-04-04T06:08:57.611293+0000","last_active":"2024-04-04T06:08:59.351982+0000","last_peered":"2024-04-04T06:08:59.351982+0000","last_clean":"2024-04-04T06:08:59.351982+0000","last_became_active":"2024-04-04T06:08:57.610423+0000","last_became_peered":"2024-04-04T06:08:57.610423+0000","last_unstale":"2024-04-04T06:08:59.351982+0000","last_undegraded":"2024-04-04T06:08:59.351982+0000","last_fullsized":"2024-04-04T06:08:59.351982+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:02:19.696656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399524+0000","last_change":"2024-04-04T06:08:56.576880+0000","last_active":"2024-04-04T06:08:59.399524+0000","last_peered":"2024-04-04T06:08:59.399524+0000","last_clean":"2024-04-04T06:08:59.399524+0000","last_became_active":"2024-04-04T06:08:56.576600+0000","last_became_peered":"2024-04-04T06:08:56.576600+0000","last_unstale":"2024-04-04T06:08:59.399524+0000","last_undegraded":"2024-04-04T06:08:59.399524+0000","last_fullsized":"2024-04-04T06:08:59.399524+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:25:43.268590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.398678+0000","last_change":"2024-04-04T06:08:57.597335+0000","last_active":"2024-04-04T06:08:59.398678+0000","last_peered":"2024-04-04T06:08:59.398678+0000","last_clean":"2024-04-04T06:08:59.398678+0000","last_became_active":"2024-04-04T06:08:57.597051+0000","last_became_peered":"2024-04-04T06:08:57.597051+0000","last_unstale":"2024-04-04T06:08:59.398678+0000","last_undegraded":"2024-04-04T06:08:59.398678+0000","last_fullsized":"2024-04-04T06:08:59.398678+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:45:02.503911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.398912+0000","last_change":"2024-04-04T06:08:56.576604+0000","last_active":"2024-04-04T06:08:59.398912+0000","last_peered":"2024-04-04T06:08:59.398912+0000","last_clean":"2024-04-04T06:08:59.398912+0000","last_became_active":"2024-04-04T06:08:56.576310+0000","last_became_peered":"2024-04-04T06:08:56.576310+0000","last_unstale":"2024-04-04T06:08:59.398912+0000","last_undegraded":"2024-04-04T06:08:59.398912+0000","last_fullsized":"2024-04-04T06:08:59.398912+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:26:57.097292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110360+0000","last_change":"2024-04-04T06:08:57.608692+0000","last_active":"2024-04-04T06:08:59.110360+0000","last_peered":"2024-04-04T06:08:59.110360+0000","last_clean":"2024-04-04T06:08:59.110360+0000","last_became_active":"2024-04-04T06:08:57.608387+0000","last_became_peered":"2024-04-04T06:08:57.608387+0000","last_unstale":"2024-04-04T06:08:59.110360+0000","last_undegraded":"2024-04-04T06:08:59.110360+0000","last_fullsized":"2024-04-04T06:08:59.110360+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:38:04.030503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110178+0000","last_change":"2024-04-04T06:08:56.568855+0000","last_active":"2024-04-04T06:08:59.110178+0000","last_peered":"2024-04-04T06:08:59.110178+0000","last_clean":"2024-04-04T06:08:59.110178+0000","last_became_active":"2024-04-04T06:08:56.568608+0000","last_became_peered":"2024-04-04T06:08:56.568608+0000","last_unstale":"2024-04-04T06:08:59.110178+0000","last_undegraded":"2024-04-04T06:08:59.110178+0000","last_fullsized":"2024-04-04T06:08:59.110178+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:09:30.775864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110378+0000","last_change":"2024-04-04T06:08:57.618484+0000","last_active":"2024-04-04T06:08:59.110378+0000","last_peered":"2024-04-04T06:08:59.110378+0000","last_clean":"2024-04-04T06:08:59.110378+0000","last_became_active":"2024-04-04T06:08:57.618213+0000","last_became_peered":"2024-04-04T06:08:57.618213+0000","last_unstale":"2024-04-04T06:08:59.110378+0000","last_undegraded":"2024-04-04T06:08:59.110378+0000","last_fullsized":"2024-04-04T06:08:59.110378+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:09:00.433068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591002+0000","last_change":"2024-04-04T06:08:56.578712+0000","last_active":"2024-04-04T06:08:58.591002+0000","last_peered":"2024-04-04T06:08:58.591002+0000","last_clean":"2024-04-04T06:08:58.591002+0000","last_became_active":"2024-04-04T06:08:56.578391+0000","last_became_peered":"2024-04-04T06:08:56.578391+0000","last_unstale":"2024-04-04T06:08:58.591002+0000","last_undegraded":"2024-04-04T06:08:58.591002+0000","last_fullsized":"2024-04-04T06:08:58.591002+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:51:59.076333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":237,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351054+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:08:59.351054+0000","last_peered":"2024-04-04T06:08:59.351054+0000","last_clean":"2024-04-04T06:08:59.351054+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:08:59.351054+0000","last_undegraded":"2024-04-04T06:08:59.351054+0000","last_fullsized":"2024-04-04T06:08:59.351054+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1302528,"data_stored":882874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":700416,"data_stored":292346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509485,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":292402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509484,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":292402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509483,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":704512,"data_stored":292436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509485,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32460,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339380,"statfs":{"total":96636764160,"available":96603525120,"internally_reserved":0,"allocated":1310720,"data_stored":884908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542188,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":292998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542186,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31880,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339960,"statfs":{"total":96636764160,"available":96604119040,"internally_reserved":0,"allocated":712704,"data_stored":293958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:09:07.136 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:09:07.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.361+0000 7f3461e00640 1 Processor -- start 2024-04-04T06:09:07.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.362+0000 7f3461e00640 1 -- start start 2024-04-04T06:09:07.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.364+0000 7f3461e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f345c0af830 0x7f345c073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:07.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.364+0000 7f3461e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f345c15fe80 con 0x7f345c0d0650 2024-04-04T06:09:07.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.364+0000 7f3461e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f345c0ad120 con 0x7f345c15c390 2024-04-04T06:09:07.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.364+0000 7f3461e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f345c15c770 con 0x7f345c0af830 2024-04-04T06:09:07.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.364+0000 7f345b400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f345c0d0650 0x7f345c071cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:34226/0 (socket says 172.21.15.159:34226) 2024-04-04T06:09:07.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.364+0000 7f345b400640 1 -- 172.21.15.159:0/222060754 learned_addr learned my addr 172.21.15.159:0/222060754 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:09:07.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.364+0000 7f3452000640 1 --2- 172.21.15.159:0/222060754 >> v2:172.21.15.178:3300/0 conn(0x7f345c0af830 0x7f345c073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:07.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.364+0000 7f345a000640 1 -- 172.21.15.159:0/222060754 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2332063580 0 0) 0x7f345c15fe80 con 0x7f345c0d0650 2024-04-04T06:09:07.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.364+0000 7f345a000640 1 -- 172.21.15.159:0/222060754 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3440003520 con 0x7f345c0d0650 2024-04-04T06:09:07.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.364+0000 7f345a000640 1 -- 172.21.15.159:0/222060754 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1530089747 0 0) 0x7f3440003520 con 0x7f345c0d0650 2024-04-04T06:09:07.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.364+0000 7f345a000640 1 -- 172.21.15.159:0/222060754 >> v2:172.21.15.178:3300/0 conn(0x7f345c0af830 msgr2=0x7f345c073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:07.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.364+0000 7f345a000640 1 --2- 172.21.15.159:0/222060754 >> v2:172.21.15.178:3300/0 conn(0x7f345c0af830 0x7f345c073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:07.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.364+0000 7f345a000640 1 -- 172.21.15.159:0/222060754 >> v1:172.21.15.178:6789/0 conn(0x7f345c15c390 legacy=0x7f345c160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:07.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.364+0000 7f345a000640 1 -- 172.21.15.159:0/222060754 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345c0ad400 con 0x7f345c0d0650 2024-04-04T06:09:07.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.364+0000 7f345a000640 1 -- 172.21.15.159:0/222060754 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f3454001a10 con 0x7f345c0d0650 2024-04-04T06:09:07.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.364+0000 7f345a000640 1 -- 172.21.15.159:0/222060754 >> v1:172.21.15.159:6789/0 conn(0x7f345c0d0650 legacy=0x7f345c071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:07.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.365+0000 7f345a000640 1 --2- 172.21.15.159:0/222060754 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34400039b0 0x7f3440003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:07.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.365+0000 7f345a000640 1 --2- 172.21.15.159:0/222060754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3440004800 0x7f3440004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:07.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.365+0000 7f345a000640 1 --2- 172.21.15.159:0/222060754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34400051c0 0x7f344000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:07.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.365+0000 7f345a000640 1 -- 172.21.15.159:0/222060754 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f344000c030 con 0x7f3440004800 2024-04-04T06:09:07.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.365+0000 7f345a000640 1 -- 172.21.15.159:0/222060754 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f344000c1e0 con 0x7f34400051c0 2024-04-04T06:09:07.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.365+0000 7f345a000640 1 -- 172.21.15.159:0/222060754 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f344000c3e0 con 0x7f34400039b0 2024-04-04T06:09:07.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.365+0000 7f3452000640 1 --2- 172.21.15.159:0/222060754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3440004800 0x7f3440004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:07.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.365+0000 7f345aa00640 1 --2- 172.21.15.159:0/222060754 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34400039b0 0x7f3440003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:07.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.365+0000 7f345b400640 1 --2- 172.21.15.159:0/222060754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34400051c0 0x7f344000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:07.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.365+0000 7f3452000640 1 -- 172.21.15.159:0/222060754 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34400039b0 msgr2=0x7f3440003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:07.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.365+0000 7f3452000640 1 --2- 172.21.15.159:0/222060754 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f34400039b0 0x7f3440003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:07.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.365+0000 7f3452000640 1 -- 172.21.15.159:0/222060754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34400051c0 msgr2=0x7f344000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:07.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.365+0000 7f3452000640 1 --2- 172.21.15.159:0/222060754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34400051c0 0x7f344000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:07.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.365+0000 7f3452000640 1 -- 172.21.15.159:0/222060754 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3440003520 con 0x7f3440004800 2024-04-04T06:09:07.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.365+0000 7f345b400640 1 -- 172.21.15.159:0/222060754 reap_dead start 2024-04-04T06:09:07.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.367+0000 7f3452000640 1 --2- 172.21.15.159:0/222060754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3440004800 0x7f3440004c80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3444009ac0 tx=0x7f3440005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e8e55156abbab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:07.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.367+0000 7f345a000640 1 -- 172.21.15.159:0/222060754 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3444039070 con 0x7f3440004800 2024-04-04T06:09:07.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.367+0000 7f345a000640 1 -- 172.21.15.159:0/222060754 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f344402c6b0 con 0x7f3440004800 2024-04-04T06:09:07.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.367+0000 7f345a000640 1 -- 172.21.15.159:0/222060754 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3444009690 con 0x7f3440004800 2024-04-04T06:09:07.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.367+0000 7f3461e00640 1 -- 172.21.15.159:0/222060754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3440004800 msgr2=0x7f3440004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:07.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.367+0000 7f3461e00640 1 --2- 172.21.15.159:0/222060754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3440004800 0x7f3440004c80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3444009ac0 tx=0x7f3440005830 comp rx=0 tx=0).stop 2024-04-04T06:09:07.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.368+0000 7f3461e00640 1 -- 172.21.15.159:0/222060754 shutdown_connections 2024-04-04T06:09:07.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.368+0000 7f3461e00640 1 --2- 172.21.15.159:0/222060754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3440004800 0x7f3440004c80 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:07.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.368+0000 7f3461e00640 1 -- 172.21.15.159:0/222060754 >> 172.21.15.159:0/222060754 conn(0x7f345c06b990 msgr2=0x7f345c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:07.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.368+0000 7f3461e00640 1 -- 172.21.15.159:0/222060754 shutdown_connections 2024-04-04T06:09:07.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.368+0000 7f3461e00640 1 -- 172.21.15.159:0/222060754 wait complete. 2024-04-04T06:09:07.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.370+0000 7f3461e00640 1 Processor -- start 2024-04-04T06:09:07.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.370+0000 7f3461e00640 1 -- start start 2024-04-04T06:09:07.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.371+0000 7f3461e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3440004800 0x7f345c1f4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:07.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.371+0000 7f3461e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f345c15ecb0 0x7f345c09cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:07.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.371+0000 7f3461e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f345c09a2a0 0x7f345c09a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:07.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.371+0000 7f3461e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f345c0cf980 con 0x7f3440004800 2024-04-04T06:09:07.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.371+0000 7f3461e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f345c160120 con 0x7f345c09a2a0 2024-04-04T06:09:07.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.371+0000 7f3461e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f345c073790 con 0x7f345c15ecb0 2024-04-04T06:09:07.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.371+0000 7f345aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3440004800 0x7f345c1f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:07.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.371+0000 7f345aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3440004800 0x7f345c1f4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52148/0 (socket says 172.21.15.159:52148) 2024-04-04T06:09:07.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.371+0000 7f345aa00640 1 -- 172.21.15.159:0/1610983877 learned_addr learned my addr 172.21.15.159:0/1610983877 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:07.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.371+0000 7f3452000640 1 --2- 172.21.15.159:0/1610983877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f345c15ecb0 0x7f345c09cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:07.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.371+0000 7f345b400640 1 --2- 172.21.15.159:0/1610983877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f345c09a2a0 0x7f345c09a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:07.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.371+0000 7f345aa00640 1 -- 172.21.15.159:0/1610983877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f345c15ecb0 msgr2=0x7f345c09cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:07.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.371+0000 7f345aa00640 1 --2- 172.21.15.159:0/1610983877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f345c15ecb0 0x7f345c09cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:07.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.372+0000 7f345aa00640 1 -- 172.21.15.159:0/1610983877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f345c09a2a0 msgr2=0x7f345c09a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:07.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.372+0000 7f345aa00640 1 --2- 172.21.15.159:0/1610983877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f345c09a2a0 0x7f345c09a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:07.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.372+0000 7f345aa00640 1 -- 172.21.15.159:0/1610983877 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345c15f130 con 0x7f3440004800 2024-04-04T06:09:07.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.372+0000 7f345aa00640 1 --2- 172.21.15.159:0/1610983877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3440004800 0x7f345c1f4570 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f344c00e500 tx=0x7f344c00e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:07.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.372+0000 7f3453e00640 1 -- 172.21.15.159:0/1610983877 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c0053a0 con 0x7f3440004800 2024-04-04T06:09:07.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.372+0000 7f3453e00640 1 -- 172.21.15.159:0/1610983877 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f344c018070 con 0x7f3440004800 2024-04-04T06:09:07.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.372+0000 7f3461e00640 1 -- 172.21.15.159:0/1610983877 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345c09d3f0 con 0x7f3440004800 2024-04-04T06:09:07.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.372+0000 7f3453e00640 1 -- 172.21.15.159:0/1610983877 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c003810 con 0x7f3440004800 2024-04-04T06:09:07.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.372+0000 7f3461e00640 1 -- 172.21.15.159:0/1610983877 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f345c09d9b0 con 0x7f3440004800 2024-04-04T06:09:07.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.374+0000 7f3461e00640 1 -- 172.21.15.159:0/1610983877 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f345c0685c0 con 0x7f3440004800 2024-04-04T06:09:07.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.374+0000 7f3453e00640 1 -- 172.21.15.159:0/1610983877 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f344c0039b0 con 0x7f3440004800 2024-04-04T06:09:07.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.376+0000 7f3453e00640 1 --2- 172.21.15.159:0/1610983877 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f343806c920 0x7f343806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:07.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.376+0000 7f3453e00640 1 -- 172.21.15.159:0/1610983877 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f344c08b860 con 0x7f3440004800 2024-04-04T06:09:07.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.376+0000 7f3452000640 1 --2- 172.21.15.159:0/1610983877 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f343806c920 0x7f343806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:07.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.378+0000 7f3452000640 1 --2- 172.21.15.159:0/1610983877 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f343806c920 0x7f343806ede0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3444036000 tx=0x7f3444030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:07.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.381+0000 7f3453e00640 1 -- 172.21.15.159:0/1610983877 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f344c059ca0 con 0x7f3440004800 2024-04-04T06:09:07.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.580+0000 7f3461e00640 1 -- 172.21.15.159:0/1610983877 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f345c0cf760 con 0x7f343806c920 2024-04-04T06:09:07.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.598+0000 7f3453e00640 1 -- 172.21.15.159:0/1610983877 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251931 (secure 0 0 0) 0x7f345c0cf760 con 0x7f343806c920 2024-04-04T06:09:07.595 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:09:07.596 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:09:07.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.603+0000 7f3461e00640 1 -- 172.21.15.159:0/1610983877 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f343806c920 msgr2=0x7f343806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:07.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.604+0000 7f3461e00640 1 --2- 172.21.15.159:0/1610983877 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f343806c920 0x7f343806ede0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3444036000 tx=0x7f3444030000 comp rx=0 tx=0).stop 2024-04-04T06:09:07.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.604+0000 7f3461e00640 1 -- 172.21.15.159:0/1610983877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3440004800 msgr2=0x7f345c1f4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:07.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.604+0000 7f3461e00640 1 --2- 172.21.15.159:0/1610983877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3440004800 0x7f345c1f4570 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f344c00e500 tx=0x7f344c00e9c0 comp rx=0 tx=0).stop 2024-04-04T06:09:07.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.604+0000 7f3461e00640 1 -- 172.21.15.159:0/1610983877 shutdown_connections 2024-04-04T06:09:07.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.604+0000 7f3461e00640 1 --2- 172.21.15.159:0/1610983877 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f343806c920 0x7f343806ede0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:07.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.604+0000 7f3461e00640 1 --2- 172.21.15.159:0/1610983877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f345c09a2a0 0x7f345c09a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:07.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.604+0000 7f3461e00640 1 --2- 172.21.15.159:0/1610983877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f345c15ecb0 0x7f345c09cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:07.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.604+0000 7f3461e00640 1 --2- 172.21.15.159:0/1610983877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3440004800 0x7f345c1f4570 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:07.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.604+0000 7f3461e00640 1 -- 172.21.15.159:0/1610983877 >> 172.21.15.159:0/1610983877 conn(0x7f345c06b990 msgr2=0x7f345c0b4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:07.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.604+0000 7f3461e00640 1 -- 172.21.15.159:0/1610983877 shutdown_connections 2024-04-04T06:09:07.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.604+0000 7f3461e00640 1 -- 172.21.15.159:0/1610983877 wait complete. 2024-04-04T06:09:07.620 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-04-04T06:09:06.780090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":251996,"kb_used_data":6696,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754722724,"statfs":{"total":773094113280,"available":772836069376,"internally_reserved":0,"allocated":6856704,"data_stored":3524324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.212101"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.587764+0000","last_change":"2024-04-04T06:08:57.592232+0000","last_active":"2024-04-04T06:08:58.587764+0000","last_peered":"2024-04-04T06:08:58.587764+0000","last_clean":"2024-04-04T06:08:58.587764+0000","last_became_active":"2024-04-04T06:08:57.591849+0000","last_became_peered":"2024-04-04T06:08:57.591849+0000","last_unstale":"2024-04-04T06:08:58.587764+0000","last_undegraded":"2024-04-04T06:08:58.587764+0000","last_fullsized":"2024-04-04T06:08:58.587764+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:55:41.770693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110894+0000","last_change":"2024-04-04T06:08:57.606120+0000","last_active":"2024-04-04T06:08:59.110894+0000","last_peered":"2024-04-04T06:08:59.110894+0000","last_clean":"2024-04-04T06:08:59.110894+0000","last_became_active":"2024-04-04T06:08:57.605844+0000","last_became_peered":"2024-04-04T06:08:57.605844+0000","last_unstale":"2024-04-04T06:08:59.110894+0000","last_undegraded":"2024-04-04T06:08:59.110894+0000","last_fullsized":"2024-04-04T06:08:59.110894+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:07:04.193264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.352020+0000","last_change":"2024-04-04T06:08:57.604547+0000","last_active":"2024-04-04T06:08:59.352020+0000","last_peered":"2024-04-04T06:08:59.352020+0000","last_clean":"2024-04-04T06:08:59.352020+0000","last_became_active":"2024-04-04T06:08:57.604225+0000","last_became_peered":"2024-04-04T06:08:57.604225+0000","last_unstale":"2024-04-04T06:08:59.352020+0000","last_undegraded":"2024-04-04T06:08:59.352020+0000","last_fullsized":"2024-04-04T06:08:59.352020+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:45:43.089485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592791+0000","last_change":"2024-04-04T06:08:57.593370+0000","last_active":"2024-04-04T06:08:58.592791+0000","last_peered":"2024-04-04T06:08:58.592791+0000","last_clean":"2024-04-04T06:08:58.592791+0000","last_became_active":"2024-04-04T06:08:57.591455+0000","last_became_peered":"2024-04-04T06:08:57.591455+0000","last_unstale":"2024-04-04T06:08:58.592791+0000","last_undegraded":"2024-04-04T06:08:58.592791+0000","last_fullsized":"2024-04-04T06:08:58.592791+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:39:56.927343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.586778+0000","last_change":"2024-04-04T06:08:57.602481+0000","last_active":"2024-04-04T06:08:58.586778+0000","last_peered":"2024-04-04T06:08:58.586778+0000","last_clean":"2024-04-04T06:08:58.586778+0000","last_became_active":"2024-04-04T06:08:57.599211+0000","last_became_peered":"2024-04-04T06:08:57.599211+0000","last_unstale":"2024-04-04T06:08:58.586778+0000","last_undegraded":"2024-04-04T06:08:58.586778+0000","last_fullsized":"2024-04-04T06:08:58.586778+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:25:32.291869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604666+0000","last_change":"2024-04-04T06:08:57.614185+0000","last_active":"2024-04-04T06:08:58.604666+0000","last_peered":"2024-04-04T06:08:58.604666+0000","last_clean":"2024-04-04T06:08:58.604666+0000","last_became_active":"2024-04-04T06:08:57.611970+0000","last_became_peered":"2024-04-04T06:08:57.611970+0000","last_unstale":"2024-04-04T06:08:58.604666+0000","last_undegraded":"2024-04-04T06:08:58.604666+0000","last_fullsized":"2024-04-04T06:08:58.604666+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:47:02.001001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399802+0000","last_change":"2024-04-04T06:08:57.596174+0000","last_active":"2024-04-04T06:08:59.399802+0000","last_peered":"2024-04-04T06:08:59.399802+0000","last_clean":"2024-04-04T06:08:59.399802+0000","last_became_active":"2024-04-04T06:08:57.595899+0000","last_became_peered":"2024-04-04T06:08:57.595899+0000","last_unstale":"2024-04-04T06:08:59.399802+0000","last_undegraded":"2024-04-04T06:08:59.399802+0000","last_fullsized":"2024-04-04T06:08:59.399802+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:41:26.350622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110048+0000","last_change":"2024-04-04T06:08:57.605001+0000","last_active":"2024-04-04T06:08:59.110048+0000","last_peered":"2024-04-04T06:08:59.110048+0000","last_clean":"2024-04-04T06:08:59.110048+0000","last_became_active":"2024-04-04T06:08:57.604608+0000","last_became_peered":"2024-04-04T06:08:57.604608+0000","last_unstale":"2024-04-04T06:08:59.110048+0000","last_undegraded":"2024-04-04T06:08:59.110048+0000","last_fullsized":"2024-04-04T06:08:59.110048+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:16:53.808572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.400387+0000","last_change":"2024-04-04T06:08:57.614647+0000","last_active":"2024-04-04T06:08:59.400387+0000","last_peered":"2024-04-04T06:08:59.400387+0000","last_clean":"2024-04-04T06:08:59.400387+0000","last_became_active":"2024-04-04T06:08:57.614442+0000","last_became_peered":"2024-04-04T06:08:57.614442+0000","last_unstale":"2024-04-04T06:08:59.400387+0000","last_undegraded":"2024-04-04T06:08:59.400387+0000","last_fullsized":"2024-04-04T06:08:59.400387+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:28:57.631136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399675+0000","last_change":"2024-04-04T06:08:57.597330+0000","last_active":"2024-04-04T06:08:59.399675+0000","last_peered":"2024-04-04T06:08:59.399675+0000","last_clean":"2024-04-04T06:08:59.399675+0000","last_became_active":"2024-04-04T06:08:57.596962+0000","last_became_peered":"2024-04-04T06:08:57.596962+0000","last_unstale":"2024-04-04T06:08:59.399675+0000","last_undegraded":"2024-04-04T06:08:59.399675+0000","last_fullsized":"2024-04-04T06:08:59.399675+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:25:44.282636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399587+0000","last_change":"2024-04-04T06:08:57.604574+0000","last_active":"2024-04-04T06:08:59.399587+0000","last_peered":"2024-04-04T06:08:59.399587+0000","last_clean":"2024-04-04T06:08:59.399587+0000","last_became_active":"2024-04-04T06:08:57.603977+0000","last_became_peered":"2024-04-04T06:08:57.603977+0000","last_unstale":"2024-04-04T06:08:59.399587+0000","last_undegraded":"2024-04-04T06:08:59.399587+0000","last_fullsized":"2024-04-04T06:08:59.399587+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:57.042947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592871+0000","last_change":"2024-04-04T06:08:57.596645+0000","last_active":"2024-04-04T06:08:58.592871+0000","last_peered":"2024-04-04T06:08:58.592871+0000","last_clean":"2024-04-04T06:08:58.592871+0000","last_became_active":"2024-04-04T06:08:57.596379+0000","last_became_peered":"2024-04-04T06:08:57.596379+0000","last_unstale":"2024-04-04T06:08:58.592871+0000","last_undegraded":"2024-04-04T06:08:58.592871+0000","last_fullsized":"2024-04-04T06:08:58.592871+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:47:02.001001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.593457+0000","last_change":"2024-04-04T06:08:57.618024+0000","last_active":"2024-04-04T06:08:58.593457+0000","last_peered":"2024-04-04T06:08:58.593457+0000","last_clean":"2024-04-04T06:08:58.593457+0000","last_became_active":"2024-04-04T06:08:57.617674+0000","last_became_peered":"2024-04-04T06:08:57.617674+0000","last_unstale":"2024-04-04T06:08:58.593457+0000","last_undegraded":"2024-04-04T06:08:58.593457+0000","last_fullsized":"2024-04-04T06:08:58.593457+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:09:46.418705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399602+0000","last_change":"2024-04-04T06:08:57.592018+0000","last_active":"2024-04-04T06:08:59.399602+0000","last_peered":"2024-04-04T06:08:59.399602+0000","last_clean":"2024-04-04T06:08:59.399602+0000","last_became_active":"2024-04-04T06:08:57.591670+0000","last_became_peered":"2024-04-04T06:08:57.591670+0000","last_unstale":"2024-04-04T06:08:59.399602+0000","last_undegraded":"2024-04-04T06:08:59.399602+0000","last_fullsized":"2024-04-04T06:08:59.399602+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:42:12.398269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.400126+0000","last_change":"2024-04-04T06:08:57.589250+0000","last_active":"2024-04-04T06:08:59.400126+0000","last_peered":"2024-04-04T06:08:59.400126+0000","last_clean":"2024-04-04T06:08:59.400126+0000","last_became_active":"2024-04-04T06:08:57.588786+0000","last_became_peered":"2024-04-04T06:08:57.588786+0000","last_unstale":"2024-04-04T06:08:59.400126+0000","last_undegraded":"2024-04-04T06:08:59.400126+0000","last_fullsized":"2024-04-04T06:08:59.400126+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:58:28.230236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604781+0000","last_change":"2024-04-04T06:08:57.611129+0000","last_active":"2024-04-04T06:08:58.604781+0000","last_peered":"2024-04-04T06:08:58.604781+0000","last_clean":"2024-04-04T06:08:58.604781+0000","last_became_active":"2024-04-04T06:08:57.610825+0000","last_became_peered":"2024-04-04T06:08:57.610825+0000","last_unstale":"2024-04-04T06:08:58.604781+0000","last_undegraded":"2024-04-04T06:08:58.604781+0000","last_fullsized":"2024-04-04T06:08:58.604781+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:25:02.372768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.111430+0000","last_change":"2024-04-04T06:08:57.606441+0000","last_active":"2024-04-04T06:08:59.111430+0000","last_peered":"2024-04-04T06:08:59.111430+0000","last_clean":"2024-04-04T06:08:59.111430+0000","last_became_active":"2024-04-04T06:08:57.606119+0000","last_became_peered":"2024-04-04T06:08:57.606119+0000","last_unstale":"2024-04-04T06:08:59.111430+0000","last_undegraded":"2024-04-04T06:08:59.111430+0000","last_fullsized":"2024-04-04T06:08:59.111430+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:42:55.934487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351721+0000","last_change":"2024-04-04T06:08:57.617716+0000","last_active":"2024-04-04T06:08:59.351721+0000","last_peered":"2024-04-04T06:08:59.351721+0000","last_clean":"2024-04-04T06:08:59.351721+0000","last_became_active":"2024-04-04T06:08:57.617385+0000","last_became_peered":"2024-04-04T06:08:57.617385+0000","last_unstale":"2024-04-04T06:08:59.351721+0000","last_undegraded":"2024-04-04T06:08:59.351721+0000","last_fullsized":"2024-04-04T06:08:59.351721+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:43:00.036899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.587158+0000","last_change":"2024-04-04T06:08:57.593659+0000","last_active":"2024-04-04T06:08:58.587158+0000","last_peered":"2024-04-04T06:08:58.587158+0000","last_clean":"2024-04-04T06:08:58.587158+0000","last_became_active":"2024-04-04T06:08:57.593382+0000","last_became_peered":"2024-04-04T06:08:57.593382+0000","last_unstale":"2024-04-04T06:08:58.587158+0000","last_undegraded":"2024-04-04T06:08:58.587158+0000","last_fullsized":"2024-04-04T06:08:58.587158+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:11:49.907094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110697+0000","last_change":"2024-04-04T06:08:57.619302+0000","last_active":"2024-04-04T06:08:59.110697+0000","last_peered":"2024-04-04T06:08:59.110697+0000","last_clean":"2024-04-04T06:08:59.110697+0000","last_became_active":"2024-04-04T06:08:57.618701+0000","last_became_peered":"2024-04-04T06:08:57.618701+0000","last_unstale":"2024-04-04T06:08:59.110697+0000","last_undegraded":"2024-04-04T06:08:59.110697+0000","last_fullsized":"2024-04-04T06:08:59.110697+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:29:01.526400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351752+0000","last_change":"2024-04-04T06:08:57.602399+0000","last_active":"2024-04-04T06:08:59.351752+0000","last_peered":"2024-04-04T06:08:59.351752+0000","last_clean":"2024-04-04T06:08:59.351752+0000","last_became_active":"2024-04-04T06:08:57.602074+0000","last_became_peered":"2024-04-04T06:08:57.602074+0000","last_unstale":"2024-04-04T06:08:59.351752+0000","last_undegraded":"2024-04-04T06:08:59.351752+0000","last_fullsized":"2024-04-04T06:08:59.351752+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:38:40.159820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110320+0000","last_change":"2024-04-04T06:08:57.603092+0000","last_active":"2024-04-04T06:08:59.110320+0000","last_peered":"2024-04-04T06:08:59.110320+0000","last_clean":"2024-04-04T06:08:59.110320+0000","last_became_active":"2024-04-04T06:08:57.602639+0000","last_became_peered":"2024-04-04T06:08:57.602639+0000","last_unstale":"2024-04-04T06:08:59.110320+0000","last_undegraded":"2024-04-04T06:08:59.110320+0000","last_fullsized":"2024-04-04T06:08:59.110320+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:35:44.572604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351023+0000","last_change":"2024-04-04T06:08:57.608928+0000","last_active":"2024-04-04T06:08:59.351023+0000","last_peered":"2024-04-04T06:08:59.351023+0000","last_clean":"2024-04-04T06:08:59.351023+0000","last_became_active":"2024-04-04T06:08:57.608146+0000","last_became_peered":"2024-04-04T06:08:57.608146+0000","last_unstale":"2024-04-04T06:08:59.351023+0000","last_undegraded":"2024-04-04T06:08:59.351023+0000","last_fullsized":"2024-04-04T06:08:59.351023+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:12:26.107777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.590550+0000","last_change":"2024-04-04T06:08:57.617880+0000","last_active":"2024-04-04T06:08:58.590550+0000","last_peered":"2024-04-04T06:08:58.590550+0000","last_clean":"2024-04-04T06:08:58.590550+0000","last_became_active":"2024-04-04T06:08:57.617508+0000","last_became_peered":"2024-04-04T06:08:57.617508+0000","last_unstale":"2024-04-04T06:08:58.590550+0000","last_undegraded":"2024-04-04T06:08:58.590550+0000","last_fullsized":"2024-04-04T06:08:58.590550+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:08:22.262253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.587766+0000","last_change":"2024-04-04T06:08:57.597236+0000","last_active":"2024-04-04T06:08:58.587766+0000","last_peered":"2024-04-04T06:08:58.587766+0000","last_clean":"2024-04-04T06:08:58.587766+0000","last_became_active":"2024-04-04T06:08:57.596931+0000","last_became_peered":"2024-04-04T06:08:57.596931+0000","last_unstale":"2024-04-04T06:08:58.587766+0000","last_undegraded":"2024-04-04T06:08:58.587766+0000","last_fullsized":"2024-04-04T06:08:58.587766+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:20:29.526370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592533+0000","last_change":"2024-04-04T06:08:57.592246+0000","last_active":"2024-04-04T06:08:58.592533+0000","last_peered":"2024-04-04T06:08:58.592533+0000","last_clean":"2024-04-04T06:08:58.592533+0000","last_became_active":"2024-04-04T06:08:57.591586+0000","last_became_peered":"2024-04-04T06:08:57.591586+0000","last_unstale":"2024-04-04T06:08:58.592533+0000","last_undegraded":"2024-04-04T06:08:58.592533+0000","last_fullsized":"2024-04-04T06:08:58.592533+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:53:17.231738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604692+0000","last_change":"2024-04-04T06:08:57.614655+0000","last_active":"2024-04-04T06:08:58.604692+0000","last_peered":"2024-04-04T06:08:58.604692+0000","last_clean":"2024-04-04T06:08:58.604692+0000","last_became_active":"2024-04-04T06:08:57.611616+0000","last_became_peered":"2024-04-04T06:08:57.611616+0000","last_unstale":"2024-04-04T06:08:58.604692+0000","last_undegraded":"2024-04-04T06:08:58.604692+0000","last_fullsized":"2024-04-04T06:08:58.604692+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:22:19.057088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.587204+0000","last_change":"2024-04-04T06:08:57.603638+0000","last_active":"2024-04-04T06:08:58.587204+0000","last_peered":"2024-04-04T06:08:58.587204+0000","last_clean":"2024-04-04T06:08:58.587204+0000","last_became_active":"2024-04-04T06:08:57.600991+0000","last_became_peered":"2024-04-04T06:08:57.600991+0000","last_unstale":"2024-04-04T06:08:58.587204+0000","last_undegraded":"2024-04-04T06:08:58.587204+0000","last_fullsized":"2024-04-04T06:08:58.587204+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:20:16.128711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110731+0000","last_change":"2024-04-04T06:08:57.601986+0000","last_active":"2024-04-04T06:08:59.110731+0000","last_peered":"2024-04-04T06:08:59.110731+0000","last_clean":"2024-04-04T06:08:59.110731+0000","last_became_active":"2024-04-04T06:08:57.601677+0000","last_became_peered":"2024-04-04T06:08:57.601677+0000","last_unstale":"2024-04-04T06:08:59.110731+0000","last_undegraded":"2024-04-04T06:08:59.110731+0000","last_fullsized":"2024-04-04T06:08:59.110731+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:39:27.630288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351327+0000","last_change":"2024-04-04T06:08:57.600183+0000","last_active":"2024-04-04T06:08:59.351327+0000","last_peered":"2024-04-04T06:08:59.351327+0000","last_clean":"2024-04-04T06:08:59.351327+0000","last_became_active":"2024-04-04T06:08:57.599862+0000","last_became_peered":"2024-04-04T06:08:57.599862+0000","last_unstale":"2024-04-04T06:08:59.351327+0000","last_undegraded":"2024-04-04T06:08:59.351327+0000","last_fullsized":"2024-04-04T06:08:59.351327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:29:36.805186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.586574+0000","last_change":"2024-04-04T06:08:57.589983+0000","last_active":"2024-04-04T06:08:58.586574+0000","last_peered":"2024-04-04T06:08:58.586574+0000","last_clean":"2024-04-04T06:08:58.586574+0000","last_became_active":"2024-04-04T06:08:57.589734+0000","last_became_peered":"2024-04-04T06:08:57.589734+0000","last_unstale":"2024-04-04T06:08:58.586574+0000","last_undegraded":"2024-04-04T06:08:58.586574+0000","last_fullsized":"2024-04-04T06:08:58.586574+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:01:08.452110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.605135+0000","last_change":"2024-04-04T06:08:57.609822+0000","last_active":"2024-04-04T06:08:58.605135+0000","last_peered":"2024-04-04T06:08:58.605135+0000","last_clean":"2024-04-04T06:08:58.605135+0000","last_became_active":"2024-04-04T06:08:57.609463+0000","last_became_peered":"2024-04-04T06:08:57.609463+0000","last_unstale":"2024-04-04T06:08:58.605135+0000","last_undegraded":"2024-04-04T06:08:58.605135+0000","last_fullsized":"2024-04-04T06:08:58.605135+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:15:03.941901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.605120+0000","last_change":"2024-04-04T06:08:57.613074+0000","last_active":"2024-04-04T06:08:58.605120+0000","last_peered":"2024-04-04T06:08:58.605120+0000","last_clean":"2024-04-04T06:08:58.605120+0000","last_became_active":"2024-04-04T06:08:57.609879+0000","last_became_peered":"2024-04-04T06:08:57.609879+0000","last_unstale":"2024-04-04T06:08:58.605120+0000","last_undegraded":"2024-04-04T06:08:58.605120+0000","last_fullsized":"2024-04-04T06:08:58.605120+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:01:08.338874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.605241+0000","last_change":"2024-04-04T06:08:56.578928+0000","last_active":"2024-04-04T06:08:58.605241+0000","last_peered":"2024-04-04T06:08:58.605241+0000","last_clean":"2024-04-04T06:08:58.605241+0000","last_became_active":"2024-04-04T06:08:56.578602+0000","last_became_peered":"2024-04-04T06:08:56.578602+0000","last_unstale":"2024-04-04T06:08:58.605241+0000","last_undegraded":"2024-04-04T06:08:58.605241+0000","last_fullsized":"2024-04-04T06:08:58.605241+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:30:18.473540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399701+0000","last_change":"2024-04-04T06:08:57.599563+0000","last_active":"2024-04-04T06:08:59.399701+0000","last_peered":"2024-04-04T06:08:59.399701+0000","last_clean":"2024-04-04T06:08:59.399701+0000","last_became_active":"2024-04-04T06:08:57.599244+0000","last_became_peered":"2024-04-04T06:08:57.599244+0000","last_unstale":"2024-04-04T06:08:59.399701+0000","last_undegraded":"2024-04-04T06:08:59.399701+0000","last_fullsized":"2024-04-04T06:08:59.399701+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:16:21.695529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110705+0000","last_change":"2024-04-04T06:08:56.577361+0000","last_active":"2024-04-04T06:08:59.110705+0000","last_peered":"2024-04-04T06:08:59.110705+0000","last_clean":"2024-04-04T06:08:59.110705+0000","last_became_active":"2024-04-04T06:08:56.576619+0000","last_became_peered":"2024-04-04T06:08:56.576619+0000","last_unstale":"2024-04-04T06:08:59.110705+0000","last_undegraded":"2024-04-04T06:08:59.110705+0000","last_fullsized":"2024-04-04T06:08:59.110705+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:53:16.217692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604774+0000","last_change":"2024-04-04T06:08:57.611238+0000","last_active":"2024-04-04T06:08:58.604774+0000","last_peered":"2024-04-04T06:08:58.604774+0000","last_clean":"2024-04-04T06:08:58.604774+0000","last_became_active":"2024-04-04T06:08:57.610966+0000","last_became_peered":"2024-04-04T06:08:57.610966+0000","last_unstale":"2024-04-04T06:08:58.604774+0000","last_undegraded":"2024-04-04T06:08:58.604774+0000","last_fullsized":"2024-04-04T06:08:58.604774+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:10:11.718125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604997+0000","last_change":"2024-04-04T06:08:56.577805+0000","last_active":"2024-04-04T06:08:58.604997+0000","last_peered":"2024-04-04T06:08:58.604997+0000","last_clean":"2024-04-04T06:08:58.604997+0000","last_became_active":"2024-04-04T06:08:56.576393+0000","last_became_peered":"2024-04-04T06:08:56.576393+0000","last_unstale":"2024-04-04T06:08:58.604997+0000","last_undegraded":"2024-04-04T06:08:58.604997+0000","last_fullsized":"2024-04-04T06:08:58.604997+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:16:13.187412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604859+0000","last_change":"2024-04-04T06:08:57.614436+0000","last_active":"2024-04-04T06:08:58.604859+0000","last_peered":"2024-04-04T06:08:58.604859+0000","last_clean":"2024-04-04T06:08:58.604859+0000","last_became_active":"2024-04-04T06:08:57.609947+0000","last_became_peered":"2024-04-04T06:08:57.609947+0000","last_unstale":"2024-04-04T06:08:58.604859+0000","last_undegraded":"2024-04-04T06:08:58.604859+0000","last_fullsized":"2024-04-04T06:08:58.604859+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:57:01.275554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592942+0000","last_change":"2024-04-04T06:08:56.578999+0000","last_active":"2024-04-04T06:08:58.592942+0000","last_peered":"2024-04-04T06:08:58.592942+0000","last_clean":"2024-04-04T06:08:58.592942+0000","last_became_active":"2024-04-04T06:08:56.578689+0000","last_became_peered":"2024-04-04T06:08:56.578689+0000","last_unstale":"2024-04-04T06:08:58.592942+0000","last_undegraded":"2024-04-04T06:08:58.592942+0000","last_fullsized":"2024-04-04T06:08:58.592942+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:30:18.473540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110373+0000","last_change":"2024-04-04T06:08:57.600474+0000","last_active":"2024-04-04T06:08:59.110373+0000","last_peered":"2024-04-04T06:08:59.110373+0000","last_clean":"2024-04-04T06:08:59.110373+0000","last_became_active":"2024-04-04T06:08:57.600135+0000","last_became_peered":"2024-04-04T06:08:57.600135+0000","last_unstale":"2024-04-04T06:08:59.110373+0000","last_undegraded":"2024-04-04T06:08:59.110373+0000","last_fullsized":"2024-04-04T06:08:59.110373+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:08:50.665613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399510+0000","last_change":"2024-04-04T06:08:56.578612+0000","last_active":"2024-04-04T06:08:59.399510+0000","last_peered":"2024-04-04T06:08:59.399510+0000","last_clean":"2024-04-04T06:08:59.399510+0000","last_became_active":"2024-04-04T06:08:56.575989+0000","last_became_peered":"2024-04-04T06:08:56.575989+0000","last_unstale":"2024-04-04T06:08:59.399510+0000","last_undegraded":"2024-04-04T06:08:59.399510+0000","last_fullsized":"2024-04-04T06:08:59.399510+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:00:56.028901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110635+0000","last_change":"2024-04-04T06:08:57.617863+0000","last_active":"2024-04-04T06:08:59.110635+0000","last_peered":"2024-04-04T06:08:59.110635+0000","last_clean":"2024-04-04T06:08:59.110635+0000","last_became_active":"2024-04-04T06:08:57.617572+0000","last_became_peered":"2024-04-04T06:08:57.617572+0000","last_unstale":"2024-04-04T06:08:59.110635+0000","last_undegraded":"2024-04-04T06:08:59.110635+0000","last_fullsized":"2024-04-04T06:08:59.110635+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:58:22.358023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604504+0000","last_change":"2024-04-04T06:08:56.579024+0000","last_active":"2024-04-04T06:08:58.604504+0000","last_peered":"2024-04-04T06:08:58.604504+0000","last_clean":"2024-04-04T06:08:58.604504+0000","last_became_active":"2024-04-04T06:08:56.578301+0000","last_became_peered":"2024-04-04T06:08:56.578301+0000","last_unstale":"2024-04-04T06:08:58.604504+0000","last_undegraded":"2024-04-04T06:08:58.604504+0000","last_fullsized":"2024-04-04T06:08:58.604504+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:39:18.014934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.590603+0000","last_change":"2024-04-04T06:08:57.602059+0000","last_active":"2024-04-04T06:08:58.590603+0000","last_peered":"2024-04-04T06:08:58.590603+0000","last_clean":"2024-04-04T06:08:58.590603+0000","last_became_active":"2024-04-04T06:08:57.601116+0000","last_became_peered":"2024-04-04T06:08:57.601116+0000","last_unstale":"2024-04-04T06:08:58.590603+0000","last_undegraded":"2024-04-04T06:08:58.590603+0000","last_fullsized":"2024-04-04T06:08:58.590603+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:17:59.892459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591333+0000","last_change":"2024-04-04T06:08:56.571563+0000","last_active":"2024-04-04T06:08:58.591333+0000","last_peered":"2024-04-04T06:08:58.591333+0000","last_clean":"2024-04-04T06:08:58.591333+0000","last_became_active":"2024-04-04T06:08:56.571092+0000","last_became_peered":"2024-04-04T06:08:56.571092+0000","last_unstale":"2024-04-04T06:08:58.591333+0000","last_undegraded":"2024-04-04T06:08:58.591333+0000","last_fullsized":"2024-04-04T06:08:58.591333+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:03:42.180669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110248+0000","last_change":"2024-04-04T06:08:57.608674+0000","last_active":"2024-04-04T06:08:59.110248+0000","last_peered":"2024-04-04T06:08:59.110248+0000","last_clean":"2024-04-04T06:08:59.110248+0000","last_became_active":"2024-04-04T06:08:57.608433+0000","last_became_peered":"2024-04-04T06:08:57.608433+0000","last_unstale":"2024-04-04T06:08:59.110248+0000","last_undegraded":"2024-04-04T06:08:59.110248+0000","last_fullsized":"2024-04-04T06:08:59.110248+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:30:51.961748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399303+0000","last_change":"2024-04-04T06:08:56.579353+0000","last_active":"2024-04-04T06:08:59.399303+0000","last_peered":"2024-04-04T06:08:59.399303+0000","last_clean":"2024-04-04T06:08:59.399303+0000","last_became_active":"2024-04-04T06:08:56.576387+0000","last_became_peered":"2024-04-04T06:08:56.576387+0000","last_unstale":"2024-04-04T06:08:59.399303+0000","last_undegraded":"2024-04-04T06:08:59.399303+0000","last_fullsized":"2024-04-04T06:08:59.399303+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:03:42.180669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.400514+0000","last_change":"2024-04-04T06:08:57.590692+0000","last_active":"2024-04-04T06:08:59.400514+0000","last_peered":"2024-04-04T06:08:59.400514+0000","last_clean":"2024-04-04T06:08:59.400514+0000","last_became_active":"2024-04-04T06:08:57.590158+0000","last_became_peered":"2024-04-04T06:08:57.590158+0000","last_unstale":"2024-04-04T06:08:59.400514+0000","last_undegraded":"2024-04-04T06:08:59.400514+0000","last_fullsized":"2024-04-04T06:08:59.400514+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:24:51.708795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.352306+0000","last_change":"2024-04-04T06:08:56.587466+0000","last_active":"2024-04-04T06:08:59.352306+0000","last_peered":"2024-04-04T06:08:59.352306+0000","last_clean":"2024-04-04T06:08:59.352306+0000","last_became_active":"2024-04-04T06:08:56.587163+0000","last_became_peered":"2024-04-04T06:08:56.587163+0000","last_unstale":"2024-04-04T06:08:59.352306+0000","last_undegraded":"2024-04-04T06:08:59.352306+0000","last_fullsized":"2024-04-04T06:08:59.352306+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:52:42.577594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.352046+0000","last_change":"2024-04-04T06:08:57.609373+0000","last_active":"2024-04-04T06:08:59.352046+0000","last_peered":"2024-04-04T06:08:59.352046+0000","last_clean":"2024-04-04T06:08:59.352046+0000","last_became_active":"2024-04-04T06:08:57.606670+0000","last_became_peered":"2024-04-04T06:08:57.606670+0000","last_unstale":"2024-04-04T06:08:59.352046+0000","last_undegraded":"2024-04-04T06:08:59.352046+0000","last_fullsized":"2024-04-04T06:08:59.352046+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:17:58.054272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110980+0000","last_change":"2024-04-04T06:08:56.581539+0000","last_active":"2024-04-04T06:08:59.110980+0000","last_peered":"2024-04-04T06:08:59.110980+0000","last_clean":"2024-04-04T06:08:59.110980+0000","last_became_active":"2024-04-04T06:08:56.579490+0000","last_became_peered":"2024-04-04T06:08:56.579490+0000","last_unstale":"2024-04-04T06:08:59.110980+0000","last_undegraded":"2024-04-04T06:08:59.110980+0000","last_fullsized":"2024-04-04T06:08:59.110980+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:39:55.913297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399236+0000","last_change":"2024-04-04T06:08:57.588887+0000","last_active":"2024-04-04T06:08:59.399236+0000","last_peered":"2024-04-04T06:08:59.399236+0000","last_clean":"2024-04-04T06:08:59.399236+0000","last_became_active":"2024-04-04T06:08:57.588615+0000","last_became_peered":"2024-04-04T06:08:57.588615+0000","last_unstale":"2024-04-04T06:08:59.399236+0000","last_undegraded":"2024-04-04T06:08:59.399236+0000","last_fullsized":"2024-04-04T06:08:59.399236+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:29:09.919535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.352507+0000","last_change":"2024-04-04T06:08:56.582280+0000","last_active":"2024-04-04T06:08:59.352507+0000","last_peered":"2024-04-04T06:08:59.352507+0000","last_clean":"2024-04-04T06:08:59.352507+0000","last_became_active":"2024-04-04T06:08:56.581968+0000","last_became_peered":"2024-04-04T06:08:56.581968+0000","last_unstale":"2024-04-04T06:08:59.352507+0000","last_undegraded":"2024-04-04T06:08:59.352507+0000","last_fullsized":"2024-04-04T06:08:59.352507+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:34:17.865573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.352230+0000","last_change":"2024-04-04T06:08:57.604234+0000","last_active":"2024-04-04T06:08:59.352230+0000","last_peered":"2024-04-04T06:08:59.352230+0000","last_clean":"2024-04-04T06:08:59.352230+0000","last_became_active":"2024-04-04T06:08:57.602743+0000","last_became_peered":"2024-04-04T06:08:57.602743+0000","last_unstale":"2024-04-04T06:08:59.352230+0000","last_undegraded":"2024-04-04T06:08:59.352230+0000","last_fullsized":"2024-04-04T06:08:59.352230+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:52:53.825956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399994+0000","last_change":"2024-04-04T06:08:56.577639+0000","last_active":"2024-04-04T06:08:59.399994+0000","last_peered":"2024-04-04T06:08:59.399994+0000","last_clean":"2024-04-04T06:08:59.399994+0000","last_became_active":"2024-04-04T06:08:56.577372+0000","last_became_peered":"2024-04-04T06:08:56.577372+0000","last_unstale":"2024-04-04T06:08:59.399994+0000","last_undegraded":"2024-04-04T06:08:59.399994+0000","last_fullsized":"2024-04-04T06:08:59.399994+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:16:20.681483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591829+0000","last_change":"2024-04-04T06:08:57.601155+0000","last_active":"2024-04-04T06:08:58.591829+0000","last_peered":"2024-04-04T06:08:58.591829+0000","last_clean":"2024-04-04T06:08:58.591829+0000","last_became_active":"2024-04-04T06:08:57.600030+0000","last_became_peered":"2024-04-04T06:08:57.600030+0000","last_unstale":"2024-04-04T06:08:58.591829+0000","last_undegraded":"2024-04-04T06:08:58.591829+0000","last_fullsized":"2024-04-04T06:08:58.591829+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:46:59.204669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110522+0000","last_change":"2024-04-04T06:08:56.575082+0000","last_active":"2024-04-04T06:08:59.110522+0000","last_peered":"2024-04-04T06:08:59.110522+0000","last_clean":"2024-04-04T06:08:59.110522+0000","last_became_active":"2024-04-04T06:08:56.574805+0000","last_became_peered":"2024-04-04T06:08:56.574805+0000","last_unstale":"2024-04-04T06:08:59.110522+0000","last_undegraded":"2024-04-04T06:08:59.110522+0000","last_fullsized":"2024-04-04T06:08:59.110522+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:46:58.190623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351568+0000","last_change":"2024-04-04T06:08:57.607510+0000","last_active":"2024-04-04T06:08:59.351568+0000","last_peered":"2024-04-04T06:08:59.351568+0000","last_clean":"2024-04-04T06:08:59.351568+0000","last_became_active":"2024-04-04T06:08:57.606804+0000","last_became_peered":"2024-04-04T06:08:57.606804+0000","last_unstale":"2024-04-04T06:08:59.351568+0000","last_undegraded":"2024-04-04T06:08:59.351568+0000","last_fullsized":"2024-04-04T06:08:59.351568+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:01:29.815756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591001+0000","last_change":"2024-04-04T06:08:56.576046+0000","last_active":"2024-04-04T06:08:58.591001+0000","last_peered":"2024-04-04T06:08:58.591001+0000","last_clean":"2024-04-04T06:08:58.591001+0000","last_became_active":"2024-04-04T06:08:56.575726+0000","last_became_peered":"2024-04-04T06:08:56.575726+0000","last_unstale":"2024-04-04T06:08:58.591001+0000","last_undegraded":"2024-04-04T06:08:58.591001+0000","last_fullsized":"2024-04-04T06:08:58.591001+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:35:35.113120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399396+0000","last_change":"2024-04-04T06:08:57.599582+0000","last_active":"2024-04-04T06:08:59.399396+0000","last_peered":"2024-04-04T06:08:59.399396+0000","last_clean":"2024-04-04T06:08:59.399396+0000","last_became_active":"2024-04-04T06:08:57.598581+0000","last_became_peered":"2024-04-04T06:08:57.598581+0000","last_unstale":"2024-04-04T06:08:59.399396+0000","last_undegraded":"2024-04-04T06:08:59.399396+0000","last_fullsized":"2024-04-04T06:08:59.399396+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:46:55.241650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591592+0000","last_change":"2024-04-04T06:08:56.576601+0000","last_active":"2024-04-04T06:08:58.591592+0000","last_peered":"2024-04-04T06:08:58.591592+0000","last_clean":"2024-04-04T06:08:58.591592+0000","last_became_active":"2024-04-04T06:08:56.572894+0000","last_became_peered":"2024-04-04T06:08:56.572894+0000","last_unstale":"2024-04-04T06:08:58.591592+0000","last_undegraded":"2024-04-04T06:08:58.591592+0000","last_fullsized":"2024-04-04T06:08:58.591592+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:16:13.187412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351248+0000","last_change":"2024-04-04T06:08:57.610067+0000","last_active":"2024-04-04T06:08:59.351248+0000","last_peered":"2024-04-04T06:08:59.351248+0000","last_clean":"2024-04-04T06:08:59.351248+0000","last_became_active":"2024-04-04T06:08:57.609770+0000","last_became_peered":"2024-04-04T06:08:57.609770+0000","last_unstale":"2024-04-04T06:08:59.351248+0000","last_undegraded":"2024-04-04T06:08:59.351248+0000","last_fullsized":"2024-04-04T06:08:59.351248+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:43:34.312451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.350950+0000","last_change":"2024-04-04T06:08:56.582998+0000","last_active":"2024-04-04T06:08:59.350950+0000","last_peered":"2024-04-04T06:08:59.350950+0000","last_clean":"2024-04-04T06:08:59.350950+0000","last_became_active":"2024-04-04T06:08:56.582720+0000","last_became_peered":"2024-04-04T06:08:56.582720+0000","last_unstale":"2024-04-04T06:08:59.350950+0000","last_undegraded":"2024-04-04T06:08:59.350950+0000","last_fullsized":"2024-04-04T06:08:59.350950+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:48:07.678914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.352120+0000","last_change":"2024-04-04T06:08:57.600691+0000","last_active":"2024-04-04T06:08:59.352120+0000","last_peered":"2024-04-04T06:08:59.352120+0000","last_clean":"2024-04-04T06:08:59.352120+0000","last_became_active":"2024-04-04T06:08:57.600389+0000","last_became_peered":"2024-04-04T06:08:57.600389+0000","last_unstale":"2024-04-04T06:08:59.352120+0000","last_undegraded":"2024-04-04T06:08:59.352120+0000","last_fullsized":"2024-04-04T06:08:59.352120+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:49:27.426679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.111161+0000","last_change":"2024-04-04T06:08:56.575119+0000","last_active":"2024-04-04T06:08:59.111161+0000","last_peered":"2024-04-04T06:08:59.111161+0000","last_clean":"2024-04-04T06:08:59.111161+0000","last_became_active":"2024-04-04T06:08:56.574832+0000","last_became_peered":"2024-04-04T06:08:56.574832+0000","last_unstale":"2024-04-04T06:08:59.111161+0000","last_undegraded":"2024-04-04T06:08:59.111161+0000","last_fullsized":"2024-04-04T06:08:59.111161+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:47:00.986955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592375+0000","last_change":"2024-04-04T06:08:57.600113+0000","last_active":"2024-04-04T06:08:58.592375+0000","last_peered":"2024-04-04T06:08:58.592375+0000","last_clean":"2024-04-04T06:08:58.592375+0000","last_became_active":"2024-04-04T06:08:57.599658+0000","last_became_peered":"2024-04-04T06:08:57.599658+0000","last_unstale":"2024-04-04T06:08:58.592375+0000","last_undegraded":"2024-04-04T06:08:58.592375+0000","last_fullsized":"2024-04-04T06:08:58.592375+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:09:31.789910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.605123+0000","last_change":"2024-04-04T06:08:56.576721+0000","last_active":"2024-04-04T06:08:58.605123+0000","last_peered":"2024-04-04T06:08:58.605123+0000","last_clean":"2024-04-04T06:08:58.605123+0000","last_became_active":"2024-04-04T06:08:56.576404+0000","last_became_peered":"2024-04-04T06:08:56.576404+0000","last_unstale":"2024-04-04T06:08:58.605123+0000","last_undegraded":"2024-04-04T06:08:58.605123+0000","last_fullsized":"2024-04-04T06:08:58.605123+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:30:05.745690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110887+0000","last_change":"2024-04-04T06:08:57.605056+0000","last_active":"2024-04-04T06:08:59.110887+0000","last_peered":"2024-04-04T06:08:59.110887+0000","last_clean":"2024-04-04T06:08:59.110887+0000","last_became_active":"2024-04-04T06:08:57.604780+0000","last_became_peered":"2024-04-04T06:08:57.604780+0000","last_unstale":"2024-04-04T06:08:59.110887+0000","last_undegraded":"2024-04-04T06:08:59.110887+0000","last_fullsized":"2024-04-04T06:08:59.110887+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:26:29.014310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592694+0000","last_change":"2024-04-04T06:08:56.579736+0000","last_active":"2024-04-04T06:08:58.592694+0000","last_peered":"2024-04-04T06:08:58.592694+0000","last_clean":"2024-04-04T06:08:58.592694+0000","last_became_active":"2024-04-04T06:08:56.579470+0000","last_became_peered":"2024-04-04T06:08:56.579470+0000","last_unstale":"2024-04-04T06:08:58.592694+0000","last_undegraded":"2024-04-04T06:08:58.592694+0000","last_fullsized":"2024-04-04T06:08:58.592694+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:16:20.681483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.352514+0000","last_change":"2024-04-04T06:08:57.605903+0000","last_active":"2024-04-04T06:08:59.352514+0000","last_peered":"2024-04-04T06:08:59.352514+0000","last_clean":"2024-04-04T06:08:59.352514+0000","last_became_active":"2024-04-04T06:08:57.605458+0000","last_became_peered":"2024-04-04T06:08:57.605458+0000","last_unstale":"2024-04-04T06:08:59.352514+0000","last_undegraded":"2024-04-04T06:08:59.352514+0000","last_fullsized":"2024-04-04T06:08:59.352514+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:33:51.458736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604894+0000","last_change":"2024-04-04T06:08:56.577972+0000","last_active":"2024-04-04T06:08:58.604894+0000","last_peered":"2024-04-04T06:08:58.604894+0000","last_clean":"2024-04-04T06:08:58.604894+0000","last_became_active":"2024-04-04T06:08:56.577700+0000","last_became_peered":"2024-04-04T06:08:56.577700+0000","last_unstale":"2024-04-04T06:08:58.604894+0000","last_undegraded":"2024-04-04T06:08:58.604894+0000","last_fullsized":"2024-04-04T06:08:58.604894+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:37:53.831405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591961+0000","last_change":"2024-04-04T06:08:57.592739+0000","last_active":"2024-04-04T06:08:58.591961+0000","last_peered":"2024-04-04T06:08:58.591961+0000","last_clean":"2024-04-04T06:08:58.591961+0000","last_became_active":"2024-04-04T06:08:57.590596+0000","last_became_peered":"2024-04-04T06:08:57.590596+0000","last_unstale":"2024-04-04T06:08:58.591961+0000","last_undegraded":"2024-04-04T06:08:58.591961+0000","last_fullsized":"2024-04-04T06:08:58.591961+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:25:23.457982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110789+0000","last_change":"2024-04-04T06:08:56.571770+0000","last_active":"2024-04-04T06:08:59.110789+0000","last_peered":"2024-04-04T06:08:59.110789+0000","last_clean":"2024-04-04T06:08:59.110789+0000","last_became_active":"2024-04-04T06:08:56.571431+0000","last_became_peered":"2024-04-04T06:08:56.571431+0000","last_unstale":"2024-04-04T06:08:59.110789+0000","last_undegraded":"2024-04-04T06:08:59.110789+0000","last_fullsized":"2024-04-04T06:08:59.110789+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:25:22.443936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604805+0000","last_change":"2024-04-04T06:08:57.613060+0000","last_active":"2024-04-04T06:08:58.604805+0000","last_peered":"2024-04-04T06:08:58.604805+0000","last_clean":"2024-04-04T06:08:58.604805+0000","last_became_active":"2024-04-04T06:08:57.612824+0000","last_became_peered":"2024-04-04T06:08:57.612824+0000","last_unstale":"2024-04-04T06:08:58.604805+0000","last_undegraded":"2024-04-04T06:08:58.604805+0000","last_fullsized":"2024-04-04T06:08:58.604805+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:54:50.560191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.398963+0000","last_change":"2024-04-04T06:08:56.574202+0000","last_active":"2024-04-04T06:08:59.398963+0000","last_peered":"2024-04-04T06:08:59.398963+0000","last_clean":"2024-04-04T06:08:59.398963+0000","last_became_active":"2024-04-04T06:08:56.573425+0000","last_became_peered":"2024-04-04T06:08:56.573425+0000","last_unstale":"2024-04-04T06:08:59.398963+0000","last_undegraded":"2024-04-04T06:08:59.398963+0000","last_fullsized":"2024-04-04T06:08:59.398963+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:09:27.272946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.604465+0000","last_change":"2024-04-04T06:08:57.612462+0000","last_active":"2024-04-04T06:08:58.604465+0000","last_peered":"2024-04-04T06:08:58.604465+0000","last_clean":"2024-04-04T06:08:58.604465+0000","last_became_active":"2024-04-04T06:08:57.611941+0000","last_became_peered":"2024-04-04T06:08:57.611941+0000","last_unstale":"2024-04-04T06:08:58.604465+0000","last_undegraded":"2024-04-04T06:08:58.604465+0000","last_fullsized":"2024-04-04T06:08:58.604465+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:39:56.927343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591348+0000","last_change":"2024-04-04T06:08:56.575741+0000","last_active":"2024-04-04T06:08:58.591348+0000","last_peered":"2024-04-04T06:08:58.591348+0000","last_clean":"2024-04-04T06:08:58.591348+0000","last_became_active":"2024-04-04T06:08:56.575438+0000","last_became_peered":"2024-04-04T06:08:56.575438+0000","last_unstale":"2024-04-04T06:08:58.591348+0000","last_undegraded":"2024-04-04T06:08:58.591348+0000","last_fullsized":"2024-04-04T06:08:58.591348+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:46:32.849014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592972+0000","last_change":"2024-04-04T06:08:57.596280+0000","last_active":"2024-04-04T06:08:58.592972+0000","last_peered":"2024-04-04T06:08:58.592972+0000","last_clean":"2024-04-04T06:08:58.592972+0000","last_became_active":"2024-04-04T06:08:57.595982+0000","last_became_peered":"2024-04-04T06:08:57.595982+0000","last_unstale":"2024-04-04T06:08:58.592972+0000","last_undegraded":"2024-04-04T06:08:58.592972+0000","last_fullsized":"2024-04-04T06:08:58.592972+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:10:11.718125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399701+0000","last_change":"2024-04-04T06:08:56.574911+0000","last_active":"2024-04-04T06:08:59.399701+0000","last_peered":"2024-04-04T06:08:59.399701+0000","last_clean":"2024-04-04T06:08:59.399701+0000","last_became_active":"2024-04-04T06:08:56.573037+0000","last_became_peered":"2024-04-04T06:08:56.573037+0000","last_unstale":"2024-04-04T06:08:59.399701+0000","last_undegraded":"2024-04-04T06:08:59.399701+0000","last_fullsized":"2024-04-04T06:08:59.399701+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:15:13.550828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.592938+0000","last_change":"2024-04-04T06:08:57.602162+0000","last_active":"2024-04-04T06:08:58.592938+0000","last_peered":"2024-04-04T06:08:58.592938+0000","last_clean":"2024-04-04T06:08:58.592938+0000","last_became_active":"2024-04-04T06:08:57.601840+0000","last_became_peered":"2024-04-04T06:08:57.601840+0000","last_unstale":"2024-04-04T06:08:58.592938+0000","last_undegraded":"2024-04-04T06:08:58.592938+0000","last_fullsized":"2024-04-04T06:08:58.592938+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:22:19.057088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"59'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.605323+0000","last_change":"2024-04-04T06:08:56.578575+0000","last_active":"2024-04-04T06:08:58.605323+0000","last_peered":"2024-04-04T06:08:58.605323+0000","last_clean":"2024-04-04T06:08:58.605323+0000","last_became_active":"2024-04-04T06:08:56.578232+0000","last_became_peered":"2024-04-04T06:08:56.578232+0000","last_unstale":"2024-04-04T06:08:58.605323+0000","last_undegraded":"2024-04-04T06:08:58.605323+0000","last_fullsized":"2024-04-04T06:08:58.605323+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:46:54.227604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399457+0000","last_change":"2024-04-04T06:08:57.589470+0000","last_active":"2024-04-04T06:08:59.399457+0000","last_peered":"2024-04-04T06:08:59.399457+0000","last_clean":"2024-04-04T06:08:59.399457+0000","last_became_active":"2024-04-04T06:08:57.589193+0000","last_became_peered":"2024-04-04T06:08:57.589193+0000","last_unstale":"2024-04-04T06:08:59.399457+0000","last_undegraded":"2024-04-04T06:08:59.399457+0000","last_fullsized":"2024-04-04T06:08:59.399457+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:02:34.522323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591681+0000","last_change":"2024-04-04T06:08:56.573941+0000","last_active":"2024-04-04T06:08:58.591681+0000","last_peered":"2024-04-04T06:08:58.591681+0000","last_clean":"2024-04-04T06:08:58.591681+0000","last_became_active":"2024-04-04T06:08:56.573700+0000","last_became_peered":"2024-04-04T06:08:56.573700+0000","last_unstale":"2024-04-04T06:08:58.591681+0000","last_undegraded":"2024-04-04T06:08:58.591681+0000","last_fullsized":"2024-04-04T06:08:58.591681+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:03:55.295388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.605391+0000","last_change":"2024-04-04T06:08:57.611889+0000","last_active":"2024-04-04T06:08:58.605391+0000","last_peered":"2024-04-04T06:08:58.605391+0000","last_clean":"2024-04-04T06:08:58.605391+0000","last_became_active":"2024-04-04T06:08:57.609636+0000","last_became_peered":"2024-04-04T06:08:57.609636+0000","last_unstale":"2024-04-04T06:08:58.605391+0000","last_undegraded":"2024-04-04T06:08:58.605391+0000","last_fullsized":"2024-04-04T06:08:58.605391+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:50:39.260072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.593012+0000","last_change":"2024-04-04T06:08:56.577871+0000","last_active":"2024-04-04T06:08:58.593012+0000","last_peered":"2024-04-04T06:08:58.593012+0000","last_clean":"2024-04-04T06:08:58.593012+0000","last_became_active":"2024-04-04T06:08:56.577570+0000","last_became_peered":"2024-04-04T06:08:56.577570+0000","last_unstale":"2024-04-04T06:08:58.593012+0000","last_undegraded":"2024-04-04T06:08:58.593012+0000","last_fullsized":"2024-04-04T06:08:58.593012+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:37:53.831405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110537+0000","last_change":"2024-04-04T06:08:57.606811+0000","last_active":"2024-04-04T06:08:59.110537+0000","last_peered":"2024-04-04T06:08:59.110537+0000","last_clean":"2024-04-04T06:08:59.110537+0000","last_became_active":"2024-04-04T06:08:57.606536+0000","last_became_peered":"2024-04-04T06:08:57.606536+0000","last_unstale":"2024-04-04T06:08:59.110537+0000","last_undegraded":"2024-04-04T06:08:59.110537+0000","last_fullsized":"2024-04-04T06:08:59.110537+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:10:33.843321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.398742+0000","last_change":"2024-04-04T06:08:56.569703+0000","last_active":"2024-04-04T06:08:59.398742+0000","last_peered":"2024-04-04T06:08:59.398742+0000","last_clean":"2024-04-04T06:08:59.398742+0000","last_became_active":"2024-04-04T06:08:56.569433+0000","last_became_peered":"2024-04-04T06:08:56.569433+0000","last_unstale":"2024-04-04T06:08:59.398742+0000","last_undegraded":"2024-04-04T06:08:59.398742+0000","last_fullsized":"2024-04-04T06:08:59.398742+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:28:25.503580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351982+0000","last_change":"2024-04-04T06:08:57.611293+0000","last_active":"2024-04-04T06:08:59.351982+0000","last_peered":"2024-04-04T06:08:59.351982+0000","last_clean":"2024-04-04T06:08:59.351982+0000","last_became_active":"2024-04-04T06:08:57.610423+0000","last_became_peered":"2024-04-04T06:08:57.610423+0000","last_unstale":"2024-04-04T06:08:59.351982+0000","last_undegraded":"2024-04-04T06:08:59.351982+0000","last_fullsized":"2024-04-04T06:08:59.351982+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:02:19.696656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.399524+0000","last_change":"2024-04-04T06:08:56.576880+0000","last_active":"2024-04-04T06:08:59.399524+0000","last_peered":"2024-04-04T06:08:59.399524+0000","last_clean":"2024-04-04T06:08:59.399524+0000","last_became_active":"2024-04-04T06:08:56.576600+0000","last_became_peered":"2024-04-04T06:08:56.576600+0000","last_unstale":"2024-04-04T06:08:59.399524+0000","last_undegraded":"2024-04-04T06:08:59.399524+0000","last_fullsized":"2024-04-04T06:08:59.399524+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:25:43.268590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.398678+0000","last_change":"2024-04-04T06:08:57.597335+0000","last_active":"2024-04-04T06:08:59.398678+0000","last_peered":"2024-04-04T06:08:59.398678+0000","last_clean":"2024-04-04T06:08:59.398678+0000","last_became_active":"2024-04-04T06:08:57.597051+0000","last_became_peered":"2024-04-04T06:08:57.597051+0000","last_unstale":"2024-04-04T06:08:59.398678+0000","last_undegraded":"2024-04-04T06:08:59.398678+0000","last_fullsized":"2024-04-04T06:08:59.398678+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:45:02.503911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.398912+0000","last_change":"2024-04-04T06:08:56.576604+0000","last_active":"2024-04-04T06:08:59.398912+0000","last_peered":"2024-04-04T06:08:59.398912+0000","last_clean":"2024-04-04T06:08:59.398912+0000","last_became_active":"2024-04-04T06:08:56.576310+0000","last_became_peered":"2024-04-04T06:08:56.576310+0000","last_unstale":"2024-04-04T06:08:59.398912+0000","last_undegraded":"2024-04-04T06:08:59.398912+0000","last_fullsized":"2024-04-04T06:08:59.398912+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:26:57.097292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110360+0000","last_change":"2024-04-04T06:08:57.608692+0000","last_active":"2024-04-04T06:08:59.110360+0000","last_peered":"2024-04-04T06:08:59.110360+0000","last_clean":"2024-04-04T06:08:59.110360+0000","last_became_active":"2024-04-04T06:08:57.608387+0000","last_became_peered":"2024-04-04T06:08:57.608387+0000","last_unstale":"2024-04-04T06:08:59.110360+0000","last_undegraded":"2024-04-04T06:08:59.110360+0000","last_fullsized":"2024-04-04T06:08:59.110360+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:38:04.030503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110178+0000","last_change":"2024-04-04T06:08:56.568855+0000","last_active":"2024-04-04T06:08:59.110178+0000","last_peered":"2024-04-04T06:08:59.110178+0000","last_clean":"2024-04-04T06:08:59.110178+0000","last_became_active":"2024-04-04T06:08:56.568608+0000","last_became_peered":"2024-04-04T06:08:56.568608+0000","last_unstale":"2024-04-04T06:08:59.110178+0000","last_undegraded":"2024-04-04T06:08:59.110178+0000","last_fullsized":"2024-04-04T06:08:59.110178+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:09:30.775864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.110378+0000","last_change":"2024-04-04T06:08:57.618484+0000","last_active":"2024-04-04T06:08:59.110378+0000","last_peered":"2024-04-04T06:08:59.110378+0000","last_clean":"2024-04-04T06:08:59.110378+0000","last_became_active":"2024-04-04T06:08:57.618213+0000","last_became_peered":"2024-04-04T06:08:57.618213+0000","last_unstale":"2024-04-04T06:08:59.110378+0000","last_undegraded":"2024-04-04T06:08:59.110378+0000","last_fullsized":"2024-04-04T06:08:59.110378+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:56.537805+0000","last_clean_scrub_stamp":"2024-04-04T06:08:56.537805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:09:00.433068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:58.591002+0000","last_change":"2024-04-04T06:08:56.578712+0000","last_active":"2024-04-04T06:08:58.591002+0000","last_peered":"2024-04-04T06:08:58.591002+0000","last_clean":"2024-04-04T06:08:58.591002+0000","last_became_active":"2024-04-04T06:08:56.578391+0000","last_became_peered":"2024-04-04T06:08:56.578391+0000","last_unstale":"2024-04-04T06:08:58.591002+0000","last_undegraded":"2024-04-04T06:08:58.591002+0000","last_fullsized":"2024-04-04T06:08:58.591002+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:08:55.523759+0000","last_clean_scrub_stamp":"2024-04-04T06:08:55.523759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:51:59.076333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":237,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-04T06:08:59.351054+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:08:59.351054+0000","last_peered":"2024-04-04T06:08:59.351054+0000","last_clean":"2024-04-04T06:08:59.351054+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:08:59.351054+0000","last_undegraded":"2024-04-04T06:08:59.351054+0000","last_fullsized":"2024-04-04T06:08:59.351054+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1302528,"data_stored":882874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":700416,"data_stored":292346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509485,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":292402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509484,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":292402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509483,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":704512,"data_stored":292436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509485,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32460,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339380,"statfs":{"total":96636764160,"available":96603525120,"internally_reserved":0,"allocated":1310720,"data_stored":884908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542189,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":292998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542187,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31880,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339960,"statfs":{"total":96636764160,"available":96604119040,"internally_reserved":0,"allocated":712704,"data_stored":293958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:09:07.624 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-04T06:09:07.624 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-04T06:09:07.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.889+0000 7f6203a00640 1 Processor -- start 2024-04-04T06:09:07.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.889+0000 7f6203a00640 1 -- start start 2024-04-04T06:09:07.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.890+0000 7f6203a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61fc153580 0x7f61fc0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:07.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.890+0000 7f6203a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f61fc0d07c0 0x7f61fc0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:07.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.890+0000 7f6203a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61fc0afab0 con 0x7f61fc0af2e0 2024-04-04T06:09:07.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.890+0000 7f6203a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f61fc0d0c00 con 0x7f61fc0d07c0 2024-04-04T06:09:07.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.890+0000 7f6203a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f61fc15cc80 con 0x7f61fc153580 2024-04-04T06:09:07.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.891+0000 7f61fbe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f61fc0d07c0 0x7f61fc0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:07.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.891+0000 7f61fbe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f61fc0d07c0 0x7f61fc0aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52162/0 (socket says 172.21.15.159:52162) 2024-04-04T06:09:07.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.891+0000 7f61fbe00640 1 -- 172.21.15.159:0/2299410616 learned_addr learned my addr 172.21.15.159:0/2299410616 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:07.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.891+0000 7f6201000640 1 --2- 172.21.15.159:0/2299410616 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61fc153580 0x7f61fc0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:07.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.892+0000 7f61fbe00640 1 -- 172.21.15.159:0/2299410616 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61fc153580 msgr2=0x7f61fc0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:07.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.892+0000 7f61fbe00640 1 --2- 172.21.15.159:0/2299410616 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61fc153580 0x7f61fc0d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:07.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.892+0000 7f61fbe00640 1 -- 172.21.15.159:0/2299410616 >> v1:172.21.15.178:6789/0 conn(0x7f61fc0af2e0 legacy=0x7f61fc06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:07.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.892+0000 7f61fbe00640 1 -- 172.21.15.159:0/2299410616 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61fc15da00 con 0x7f61fc0d07c0 2024-04-04T06:09:07.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.893+0000 7f61fbe00640 1 --2- 172.21.15.159:0/2299410616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61fc0d07c0 0x7f61fc0aeda0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009870 tx=0x7f61e802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13162724da6f94ed server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:07.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.894+0000 7f61fb400640 1 -- 172.21.15.159:0/2299410616 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8035020 con 0x7f61fc0d07c0 2024-04-04T06:09:07.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.894+0000 7f61fb400640 1 -- 172.21.15.159:0/2299410616 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61e8031640 con 0x7f61fc0d07c0 2024-04-04T06:09:07.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.894+0000 7f61fb400640 1 -- 172.21.15.159:0/2299410616 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e802d040 con 0x7f61fc0d07c0 2024-04-04T06:09:07.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.894+0000 7f6203a00640 1 -- 172.21.15.159:0/2299410616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61fc0d07c0 msgr2=0x7f61fc0aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:07.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.894+0000 7f6203a00640 1 --2- 172.21.15.159:0/2299410616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61fc0d07c0 0x7f61fc0aeda0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009870 tx=0x7f61e802b8f0 comp rx=0 tx=0).stop 2024-04-04T06:09:07.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.895+0000 7f6203a00640 1 -- 172.21.15.159:0/2299410616 shutdown_connections 2024-04-04T06:09:07.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.895+0000 7f6203a00640 1 --2- 172.21.15.159:0/2299410616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61fc0d07c0 0x7f61fc0aeda0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:07.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.895+0000 7f6203a00640 1 --2- 172.21.15.159:0/2299410616 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61fc153580 0x7f61fc0d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:07.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.895+0000 7f6203a00640 1 -- 172.21.15.159:0/2299410616 >> 172.21.15.159:0/2299410616 conn(0x7f61fc06b750 msgr2=0x7f61fc06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:07.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.895+0000 7f6203a00640 1 -- 172.21.15.159:0/2299410616 shutdown_connections 2024-04-04T06:09:07.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.895+0000 7f6203a00640 1 -- 172.21.15.159:0/2299410616 wait complete. 2024-04-04T06:09:07.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.896+0000 7f6203a00640 1 Processor -- start 2024-04-04T06:09:07.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.897+0000 7f6203a00640 1 -- start start 2024-04-04T06:09:07.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.897+0000 7f6203a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61fc0af2e0 0x7f61fc1f8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:07.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.897+0000 7f6203a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61fc0d07c0 0x7f61fc154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:07.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.897+0000 7f6203a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61fc153580 0x7f61fc1ee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:07.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.897+0000 7f6203a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f61fc15d4e0 con 0x7f61fc0af2e0 2024-04-04T06:09:07.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.898+0000 7f6203a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f61fc15d200 con 0x7f61fc153580 2024-04-04T06:09:07.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.898+0000 7f6203a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f61fc153960 con 0x7f61fc0d07c0 2024-04-04T06:09:07.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.898+0000 7f6201000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61fc0af2e0 0x7f61fc1f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:07.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.898+0000 7f6201000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61fc0af2e0 0x7f61fc1f8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52174/0 (socket says 172.21.15.159:52174) 2024-04-04T06:09:07.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.898+0000 7f6201000640 1 -- 172.21.15.159:0/3180854317 learned_addr learned my addr 172.21.15.159:0/3180854317 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:07.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.898+0000 7f61fbe00640 1 --2- 172.21.15.159:0/3180854317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61fc0d07c0 0x7f61fc154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:07.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.898+0000 7f6201000640 1 -- 172.21.15.159:0/3180854317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61fc0d07c0 msgr2=0x7f61fc154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:07.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.898+0000 7f6201a00640 1 --2- 172.21.15.159:0/3180854317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61fc153580 0x7f61fc1ee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:07.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.898+0000 7f6201000640 1 --2- 172.21.15.159:0/3180854317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61fc0d07c0 0x7f61fc154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:07.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.898+0000 7f6201000640 1 -- 172.21.15.159:0/3180854317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61fc153580 msgr2=0x7f61fc1ee2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:07.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.898+0000 7f6201000640 1 --2- 172.21.15.159:0/3180854317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61fc153580 0x7f61fc1ee2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:07.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.898+0000 7f6201000640 1 -- 172.21.15.159:0/3180854317 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61fc1f4800 con 0x7f61fc0af2e0 2024-04-04T06:09:07.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.898+0000 7f61fbe00640 1 --2- 172.21.15.159:0/3180854317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61fc0d07c0 0x7f61fc154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:07.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.899+0000 7f6201000640 1 --2- 172.21.15.159:0/3180854317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61fc0af2e0 0x7f61fc1f8410 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f61f000d800 tx=0x7f61f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:07.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.899+0000 7f61f8c00640 1 -- 172.21.15.159:0/3180854317 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f000b6c0 con 0x7f61fc0af2e0 2024-04-04T06:09:07.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.899+0000 7f61f8c00640 1 -- 172.21.15.159:0/3180854317 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61f0019070 con 0x7f61fc0af2e0 2024-04-04T06:09:07.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.899+0000 7f6203a00640 1 -- 172.21.15.159:0/3180854317 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61fc1f4ae0 con 0x7f61fc0af2e0 2024-04-04T06:09:07.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.899+0000 7f61f8c00640 1 -- 172.21.15.159:0/3180854317 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f000ba80 con 0x7f61fc0af2e0 2024-04-04T06:09:07.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.899+0000 7f6203a00640 1 -- 172.21.15.159:0/3180854317 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61fc1f50a0 con 0x7f61fc0af2e0 2024-04-04T06:09:07.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.900+0000 7f6203a00640 1 -- 172.21.15.159:0/3180854317 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61fc0685c0 con 0x7f61fc0af2e0 2024-04-04T06:09:07.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.901+0000 7f61f8c00640 1 -- 172.21.15.159:0/3180854317 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61f000bc20 con 0x7f61fc0af2e0 2024-04-04T06:09:07.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.902+0000 7f61f8c00640 1 --2- 172.21.15.159:0/3180854317 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f61dc06c8f0 0x7f61dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:07.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.902+0000 7f61f8c00640 1 -- 172.21.15.159:0/3180854317 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f61f008c070 con 0x7f61fc0af2e0 2024-04-04T06:09:07.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.903+0000 7f61fbe00640 1 --2- 172.21.15.159:0/3180854317 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f61dc06c8f0 0x7f61dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:07.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.904+0000 7f61fbe00640 1 --2- 172.21.15.159:0/3180854317 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f61dc06c8f0 0x7f61dc06edb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f61e8036000 tx=0x7f61e8008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:07.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:07.911+0000 7f61f8c00640 1 -- 172.21.15.159:0/3180854317 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f61f005a400 con 0x7f61fc0af2e0 2024-04-04T06:09:08.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.186+0000 7f6203a00640 1 -- 172.21.15.159:0/3180854317 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f61fc1501d0 con 0x7f61fc0af2e0 2024-04-04T06:09:08.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.188+0000 7f61f8c00640 1 -- 172.21.15.159:0/3180854317 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f61f0059d00 con 0x7f61fc0af2e0 2024-04-04T06:09:08.182 INFO:teuthology.orchestra.run.smithi159.stdout:[client.0] 2024-04-04T06:09:08.182 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQD9Qg5mqNKJGxAA4jP9cadhRIyhQ3lssttWIw== 2024-04-04T06:09:08.182 INFO:teuthology.orchestra.run.smithi159.stdout: caps mds = "allow" 2024-04-04T06:09:08.182 INFO:teuthology.orchestra.run.smithi159.stdout: caps mon = "allow r" 2024-04-04T06:09:08.182 INFO:teuthology.orchestra.run.smithi159.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-04T06:09:08.182 INFO:teuthology.orchestra.run.smithi159.stderr:updated caps for client.0 2024-04-04T06:09:08.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.191+0000 7f6203a00640 1 -- 172.21.15.159:0/3180854317 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f61dc06c8f0 msgr2=0x7f61dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:08.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.191+0000 7f6203a00640 1 --2- 172.21.15.159:0/3180854317 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f61dc06c8f0 0x7f61dc06edb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f61e8036000 tx=0x7f61e8008030 comp rx=0 tx=0).stop 2024-04-04T06:09:08.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.191+0000 7f6203a00640 1 -- 172.21.15.159:0/3180854317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61fc0af2e0 msgr2=0x7f61fc1f8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:08.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.191+0000 7f6203a00640 1 --2- 172.21.15.159:0/3180854317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61fc0af2e0 0x7f61fc1f8410 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f61f000d800 tx=0x7f61f000dcc0 comp rx=0 tx=0).stop 2024-04-04T06:09:08.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.192+0000 7f6203a00640 1 -- 172.21.15.159:0/3180854317 shutdown_connections 2024-04-04T06:09:08.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.192+0000 7f6203a00640 1 --2- 172.21.15.159:0/3180854317 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f61dc06c8f0 0x7f61dc06edb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:08.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.192+0000 7f6203a00640 1 --2- 172.21.15.159:0/3180854317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61fc153580 0x7f61fc1ee2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:08.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.192+0000 7f6203a00640 1 --2- 172.21.15.159:0/3180854317 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61fc0d07c0 0x7f61fc154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:08.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.192+0000 7f6203a00640 1 --2- 172.21.15.159:0/3180854317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61fc0af2e0 0x7f61fc1f8410 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:08.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.192+0000 7f6203a00640 1 -- 172.21.15.159:0/3180854317 >> 172.21.15.159:0/3180854317 conn(0x7f61fc06b750 msgr2=0x7f61fc06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:08.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.192+0000 7f6203a00640 1 -- 172.21.15.159:0/3180854317 shutdown_connections 2024-04-04T06:09:08.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.192+0000 7f6203a00640 1 -- 172.21.15.159:0/3180854317 wait complete. 2024-04-04T06:09:08.204 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:09:08.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.409+0000 7f7a22000640 1 Processor -- start 2024-04-04T06:09:08.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.409+0000 7f7a22000640 1 -- start start 2024-04-04T06:09:08.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.410+0000 7f7a22000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7a1c15c390 0x7f7a1c073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:08.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.410+0000 7f7a22000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c0d0650 0x7f7a1c0d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:08.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.410+0000 7f7a22000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a1c15fe80 con 0x7f7a1c0af830 2024-04-04T06:09:08.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.410+0000 7f7a22000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7a1c15c770 con 0x7f7a1c15c390 2024-04-04T06:09:08.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.410+0000 7f7a22000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7a1c0ad120 con 0x7f7a1c0d0650 2024-04-04T06:09:08.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.411+0000 7f7a1be00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c0d0650 0x7f7a1c0d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:08.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.411+0000 7f7a1be00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c0d0650 0x7f7a1c0d0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:57164/0 (socket says 172.21.15.159:57164) 2024-04-04T06:09:08.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.411+0000 7f7a1be00640 1 -- 172.21.15.159:0/894774907 learned_addr learned my addr 172.21.15.159:0/894774907 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:08.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.411+0000 7f7a1b400640 1 --2- 172.21.15.159:0/894774907 >> v2:172.21.15.178:3300/0 conn(0x7f7a1c15c390 0x7f7a1c073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:08.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.412+0000 7f7a1aa00640 1 -- 172.21.15.159:0/894774907 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 652636027 0 0) 0x7f7a1c15fe80 con 0x7f7a1c0af830 2024-04-04T06:09:08.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.412+0000 7f7a1aa00640 1 -- 172.21.15.159:0/894774907 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a00003520 con 0x7f7a1c0af830 2024-04-04T06:09:08.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.412+0000 7f7a1be00640 1 -- 172.21.15.159:0/894774907 >> v2:172.21.15.178:3300/0 conn(0x7f7a1c15c390 msgr2=0x7f7a1c073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:08.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.412+0000 7f7a1be00640 1 --2- 172.21.15.159:0/894774907 >> v2:172.21.15.178:3300/0 conn(0x7f7a1c15c390 0x7f7a1c073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:08.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.412+0000 7f7a1be00640 1 -- 172.21.15.159:0/894774907 >> v1:172.21.15.178:6789/0 conn(0x7f7a1c0af830 legacy=0x7f7a1c160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:08.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.412+0000 7f7a1be00640 1 -- 172.21.15.159:0/894774907 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1c0ad400 con 0x7f7a1c0d0650 2024-04-04T06:09:08.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.414+0000 7f7a1be00640 1 --2- 172.21.15.159:0/894774907 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c0d0650 0x7f7a1c0d0ae0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009e50 tx=0x7f7a1402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6263b6bff758c0ab server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:08.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.414+0000 7f7a1aa00640 1 -- 172.21.15.159:0/894774907 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1402cb00 con 0x7f7a1c0d0650 2024-04-04T06:09:08.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.414+0000 7f7a1aa00640 1 -- 172.21.15.159:0/894774907 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a1402cca0 con 0x7f7a1c0d0650 2024-04-04T06:09:08.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.414+0000 7f7a1aa00640 1 -- 172.21.15.159:0/894774907 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a14034610 con 0x7f7a1c0d0650 2024-04-04T06:09:08.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.415+0000 7f7a22000640 1 -- 172.21.15.159:0/894774907 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c0d0650 msgr2=0x7f7a1c0d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:08.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.415+0000 7f7a22000640 1 --2- 172.21.15.159:0/894774907 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c0d0650 0x7f7a1c0d0ae0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009e50 tx=0x7f7a1402bf60 comp rx=0 tx=0).stop 2024-04-04T06:09:08.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.415+0000 7f7a22000640 1 -- 172.21.15.159:0/894774907 shutdown_connections 2024-04-04T06:09:08.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.415+0000 7f7a22000640 1 --2- 172.21.15.159:0/894774907 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c0d0650 0x7f7a1c0d0ae0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:08.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.415+0000 7f7a22000640 1 --2- 172.21.15.159:0/894774907 >> v2:172.21.15.178:3300/0 conn(0x7f7a1c15c390 0x7f7a1c073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:08.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.415+0000 7f7a22000640 1 -- 172.21.15.159:0/894774907 >> 172.21.15.159:0/894774907 conn(0x7f7a1c06b990 msgr2=0x7f7a1c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:08.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.415+0000 7f7a22000640 1 -- 172.21.15.159:0/894774907 shutdown_connections 2024-04-04T06:09:08.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.415+0000 7f7a22000640 1 -- 172.21.15.159:0/894774907 wait complete. 2024-04-04T06:09:08.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.417+0000 7f7a22000640 1 Processor -- start 2024-04-04T06:09:08.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.417+0000 7f7a22000640 1 -- start start 2024-04-04T06:09:08.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.418+0000 7f7a22000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c0af830 0x7f7a1c15eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:08.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.418+0000 7f7a22000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a1c0d0650 0x7f7a1c145280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:08.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.418+0000 7f7a22000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a1c15c390 0x7f7a1c1457c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:08.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.418+0000 7f7a22000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c160d50 con 0x7f7a1c15c390 2024-04-04T06:09:08.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.418+0000 7f7a22000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c160120 con 0x7f7a1c0d0650 2024-04-04T06:09:08.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.418+0000 7f7a22000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7a1c0cf980 con 0x7f7a1c0af830 2024-04-04T06:09:08.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.418+0000 7f7a1be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a1c15c390 0x7f7a1c1457c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:08.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.418+0000 7f7a1be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a1c15c390 0x7f7a1c1457c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52188/0 (socket says 172.21.15.159:52188) 2024-04-04T06:09:08.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.418+0000 7f7a1be00640 1 -- 172.21.15.159:0/2925160416 learned_addr learned my addr 172.21.15.159:0/2925160416 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:08.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.418+0000 7f7a1b400640 1 --2- 172.21.15.159:0/2925160416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c0af830 0x7f7a1c15eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:08.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.418+0000 7f7a13e00640 1 --2- 172.21.15.159:0/2925160416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a1c0d0650 0x7f7a1c145280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:08.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.418+0000 7f7a1be00640 1 -- 172.21.15.159:0/2925160416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c0af830 msgr2=0x7f7a1c15eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:08.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.418+0000 7f7a1be00640 1 --2- 172.21.15.159:0/2925160416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c0af830 0x7f7a1c15eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:08.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.418+0000 7f7a1be00640 1 -- 172.21.15.159:0/2925160416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a1c0d0650 msgr2=0x7f7a1c145280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:08.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.418+0000 7f7a1be00640 1 --2- 172.21.15.159:0/2925160416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a1c0d0650 0x7f7a1c145280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:08.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.418+0000 7f7a1be00640 1 -- 172.21.15.159:0/2925160416 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1c0b2460 con 0x7f7a1c15c390 2024-04-04T06:09:08.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.418+0000 7f7a13e00640 1 --2- 172.21.15.159:0/2925160416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a1c0d0650 0x7f7a1c145280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:08.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.419+0000 7f7a1be00640 1 --2- 172.21.15.159:0/2925160416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a1c15c390 0x7f7a1c1457c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009e50 tx=0x7f7a14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:08.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.419+0000 7f7a13400640 1 -- 172.21.15.159:0/2925160416 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a14043020 con 0x7f7a1c15c390 2024-04-04T06:09:08.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.419+0000 7f7a13400640 1 -- 172.21.15.159:0/2925160416 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a1402ce50 con 0x7f7a1c15c390 2024-04-04T06:09:08.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.419+0000 7f7a22000640 1 -- 172.21.15.159:0/2925160416 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1c0b2740 con 0x7f7a1c15c390 2024-04-04T06:09:08.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.419+0000 7f7a13400640 1 -- 172.21.15.159:0/2925160416 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a14009510 con 0x7f7a1c15c390 2024-04-04T06:09:08.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.419+0000 7f7a22000640 1 -- 172.21.15.159:0/2925160416 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a1c15f400 con 0x7f7a1c15c390 2024-04-04T06:09:08.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.420+0000 7f7a22000640 1 -- 172.21.15.159:0/2925160416 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a1c0685c0 con 0x7f7a1c15c390 2024-04-04T06:09:08.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.421+0000 7f7a13400640 1 -- 172.21.15.159:0/2925160416 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a14046020 con 0x7f7a1c15c390 2024-04-04T06:09:08.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.422+0000 7f7a13400640 1 --2- 172.21.15.159:0/2925160416 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f79f806c920 0x7f79f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:08.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.422+0000 7f7a13400640 1 -- 172.21.15.159:0/2925160416 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f7a140adde0 con 0x7f7a1c15c390 2024-04-04T06:09:08.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.423+0000 7f7a1b400640 1 --2- 172.21.15.159:0/2925160416 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f79f806c920 0x7f79f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:08.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.424+0000 7f7a1b400640 1 --2- 172.21.15.159:0/2925160416 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f79f806c920 0x7f79f806ede0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009ad0 tx=0x7f7a0c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:08.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.427+0000 7f7a13400640 1 -- 172.21.15.159:0/2925160416 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7a1404d070 con 0x7f7a1c15c390 2024-04-04T06:09:08.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.715+0000 7f7a22000640 1 -- 172.21.15.159:0/2925160416 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a1c06e1c0 con 0x7f7a1c15c390 2024-04-04T06:09:08.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.716+0000 7f7a13400640 1 -- 172.21.15.159:0/2925160416 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5187 (secure 0 0 0) 0x7f7a1407c200 con 0x7f7a1c15c390 2024-04-04T06:09:08.709 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:09:08.709 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5191,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3507245876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3507245876},{"type":"v1","addr":"172.21.15.159:6835","nonce":3507245876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/967262871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":967262871},{"type":"v1","addr":"172.21.15.178:6835","nonce":967262871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2969351496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2969351496},{"type":"v1","addr":"172.21.15.178:6837","nonce":2969351496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:08:57.020444+0000","modified":"2024-04-04T06:09:00.117714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6039":{"gid":6039,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6834/962475038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":962475038},{"type":"v1","addr":"172.21.15.159:6834","nonce":962475038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6039,"qdb_cluster":[6039]},"id":4}]} 2024-04-04T06:09:08.710 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 33 2024-04-04T06:09:08.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.719+0000 7f7a22000640 1 -- 172.21.15.159:0/2925160416 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f79f806c920 msgr2=0x7f79f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:08.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.719+0000 7f7a22000640 1 --2- 172.21.15.159:0/2925160416 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f79f806c920 0x7f79f806ede0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009ad0 tx=0x7f7a0c008040 comp rx=0 tx=0).stop 2024-04-04T06:09:08.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.719+0000 7f7a22000640 1 -- 172.21.15.159:0/2925160416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a1c15c390 msgr2=0x7f7a1c1457c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:08.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.719+0000 7f7a22000640 1 --2- 172.21.15.159:0/2925160416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a1c15c390 0x7f7a1c1457c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009e50 tx=0x7f7a14004a40 comp rx=0 tx=0).stop 2024-04-04T06:09:08.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.719+0000 7f7a22000640 1 -- 172.21.15.159:0/2925160416 shutdown_connections 2024-04-04T06:09:08.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.719+0000 7f7a22000640 1 --2- 172.21.15.159:0/2925160416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a1c15c390 0x7f7a1c1457c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:08.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.719+0000 7f7a22000640 1 --2- 172.21.15.159:0/2925160416 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f79f806c920 0x7f79f806ede0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:08.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.719+0000 7f7a22000640 1 --2- 172.21.15.159:0/2925160416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a1c0d0650 0x7f7a1c145280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:08.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.719+0000 7f7a22000640 1 --2- 172.21.15.159:0/2925160416 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a1c0af830 0x7f7a1c15eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:08.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.719+0000 7f7a22000640 1 -- 172.21.15.159:0/2925160416 >> 172.21.15.159:0/2925160416 conn(0x7f7a1c06b990 msgr2=0x7f7a1c0aece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:08.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.719+0000 7f7a22000640 1 -- 172.21.15.159:0/2925160416 shutdown_connections 2024-04-04T06:09:08.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:08.719+0000 7f7a22000640 1 -- 172.21.15.159:0/2925160416 wait complete. 2024-04-04T06:09:09.134 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:08:57.020444+0000', 'modified': '2024-04-04T06:09:00.117714+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6039}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6039': {'gid': 6039, 'name': 'c', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.159:6834/962475038', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6833', 'nonce': 962475038}, {'type': 'v1', 'addr': '172.21.15.159:6834', 'nonce': 962475038}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6039, 'qdb_cluster': [6039]} 2024-04-04T06:09:09.134 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-04T06:09:09.134 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-04T06:09:09.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.360+0000 7febdca00640 1 Processor -- start 2024-04-04T06:09:09.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.360+0000 7febdca00640 1 -- start start 2024-04-04T06:09:09.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.361+0000 7febdca00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7febd80d2770 0x7febd80cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.361+0000 7febdca00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febd80d1ba0 0x7febd80cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.361+0000 7febdca00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febd815e7e0 con 0x7febd815e400 2024-04-04T06:09:09.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.361+0000 7febdca00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7febd8162300 con 0x7febd80d2770 2024-04-04T06:09:09.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.361+0000 7febdca00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7febd81625e0 con 0x7febd80d1ba0 2024-04-04T06:09:09.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.361+0000 7febcea00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7febd80d2770 0x7febd80cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.362+0000 7febd7400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7febd815e400 0x7febd80d2060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:34232/0 (socket says 172.21.15.159:34232) 2024-04-04T06:09:09.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.362+0000 7febd7400640 1 -- 172.21.15.159:0/1901156776 learned_addr learned my addr 172.21.15.159:0/1901156776 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:09:09.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.362+0000 7febd7e00640 1 --2- 172.21.15.159:0/1901156776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febd80d1ba0 0x7febd80cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.362+0000 7febcea00640 1 -- 172.21.15.159:0/1901156776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febd80d1ba0 msgr2=0x7febd80cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:09.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.362+0000 7febcea00640 1 --2- 172.21.15.159:0/1901156776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febd80d1ba0 0x7febd80cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.362+0000 7febcea00640 1 -- 172.21.15.159:0/1901156776 >> v1:172.21.15.159:6789/0 conn(0x7febd815e400 legacy=0x7febd80d2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:09.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.362+0000 7febcea00640 1 -- 172.21.15.159:0/1901156776 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd81628c0 con 0x7febd80d2770 2024-04-04T06:09:09.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.362+0000 7febd6a00640 1 -- 172.21.15.159:0/1901156776 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 506970452 0 0) 0x7febd815e7e0 con 0x7febd815e400 2024-04-04T06:09:09.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.362+0000 7febd7e00640 1 --2- 172.21.15.159:0/1901156776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febd80d1ba0 0x7febd80cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:09.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.363+0000 7febcea00640 1 --2- 172.21.15.159:0/1901156776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febd80d2770 0x7febd80cc720 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7febc00097a0 tx=0x7febc002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d055e5812d48c07a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:09.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.363+0000 7febd6a00640 1 -- 172.21.15.159:0/1901156776 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0035020 con 0x7febd80d2770 2024-04-04T06:09:09.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.364+0000 7febd6a00640 1 -- 172.21.15.159:0/1901156776 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc0031810 con 0x7febd80d2770 2024-04-04T06:09:09.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.364+0000 7febd6a00640 1 -- 172.21.15.159:0/1901156776 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0031be0 con 0x7febd80d2770 2024-04-04T06:09:09.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.364+0000 7febdca00640 1 -- 172.21.15.159:0/1901156776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febd80d2770 msgr2=0x7febd80cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:09.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.364+0000 7febdca00640 1 --2- 172.21.15.159:0/1901156776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febd80d2770 0x7febd80cc720 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7febc00097a0 tx=0x7febc002b8b0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.364+0000 7febdca00640 1 -- 172.21.15.159:0/1901156776 shutdown_connections 2024-04-04T06:09:09.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.364+0000 7febdca00640 1 --2- 172.21.15.159:0/1901156776 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febd80d1ba0 0x7febd80cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.364+0000 7febdca00640 1 --2- 172.21.15.159:0/1901156776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febd80d2770 0x7febd80cc720 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.364+0000 7febdca00640 1 -- 172.21.15.159:0/1901156776 >> 172.21.15.159:0/1901156776 conn(0x7febd8067280 msgr2=0x7febd8067660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:09.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.364+0000 7febdca00640 1 -- 172.21.15.159:0/1901156776 shutdown_connections 2024-04-04T06:09:09.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.364+0000 7febdca00640 1 -- 172.21.15.159:0/1901156776 wait complete. 2024-04-04T06:09:09.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.367+0000 7febdca00640 1 Processor -- start 2024-04-04T06:09:09.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.367+0000 7febdca00640 1 -- start start 2024-04-04T06:09:09.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.369+0000 7febdca00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febd80d1ba0 0x7febd80c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.369+0000 7febdca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febd80d2770 0x7febd8153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.369+0000 7febdca00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febd815e400 0x7febd80c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.369+0000 7febdca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7febd81620e0 con 0x7febd80d2770 2024-04-04T06:09:09.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.369+0000 7febdca00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7febd8161f60 con 0x7febd815e400 2024-04-04T06:09:09.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.369+0000 7febdca00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7febd806e2b0 con 0x7febd80d1ba0 2024-04-04T06:09:09.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.369+0000 7febcea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febd80d2770 0x7febd8153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.369+0000 7febcea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febd80d2770 0x7febd8153450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52212/0 (socket says 172.21.15.159:52212) 2024-04-04T06:09:09.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.369+0000 7febcea00640 1 -- 172.21.15.159:0/3734728930 learned_addr learned my addr 172.21.15.159:0/3734728930 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:09.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.369+0000 7febd7400640 1 --2- 172.21.15.159:0/3734728930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febd80d1ba0 0x7febd80c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.369+0000 7febcea00640 1 -- 172.21.15.159:0/3734728930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febd80d1ba0 msgr2=0x7febd80c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:09.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.369+0000 7febcea00640 1 --2- 172.21.15.159:0/3734728930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febd80d1ba0 0x7febd80c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.369+0000 7febcea00640 1 -- 172.21.15.159:0/3734728930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febd815e400 msgr2=0x7febd80c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:09.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.369+0000 7febcea00640 1 --2- 172.21.15.159:0/3734728930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febd815e400 0x7febd80c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.369+0000 7febcea00640 1 -- 172.21.15.159:0/3734728930 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd80c6080 con 0x7febd80d2770 2024-04-04T06:09:09.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.369+0000 7febd7400640 1 --2- 172.21.15.159:0/3734728930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febd80d1ba0 0x7febd80c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:09.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.370+0000 7febcea00640 1 --2- 172.21.15.159:0/3734728930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febd80d2770 0x7febd8153450 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7febc0000c00 tx=0x7febc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:09.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.370+0000 7febcfe00640 1 -- 172.21.15.159:0/3734728930 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0035050 con 0x7febd80d2770 2024-04-04T06:09:09.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.370+0000 7febcfe00640 1 -- 172.21.15.159:0/3734728930 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc003bd70 con 0x7febd80d2770 2024-04-04T06:09:09.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.370+0000 7febdca00640 1 -- 172.21.15.159:0/3734728930 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd80c6360 con 0x7febd80d2770 2024-04-04T06:09:09.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.370+0000 7febcfe00640 1 -- 172.21.15.159:0/3734728930 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0004250 con 0x7febd80d2770 2024-04-04T06:09:09.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.370+0000 7febdca00640 1 -- 172.21.15.159:0/3734728930 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd80c6920 con 0x7febd80d2770 2024-04-04T06:09:09.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.372+0000 7febcfe00640 1 -- 172.21.15.159:0/3734728930 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7febc0038020 con 0x7febd80d2770 2024-04-04T06:09:09.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.374+0000 7febe0c81740 1 Processor -- start 2024-04-04T06:09:09.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.374+0000 7febe0c81740 1 -- start start 2024-04-04T06:09:09.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.374+0000 7febcfe00640 1 --2- 172.21.15.159:0/3734728930 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7febb406c8f0 0x7febb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.374+0000 7febcfe00640 1 -- 172.21.15.159:0/3734728930 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7febc00adc60 con 0x7febd80d2770 2024-04-04T06:09:09.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.375+0000 7febd7e00640 1 --2- 172.21.15.159:0/3734728930 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7febb406c8f0 0x7febb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.375+0000 7febe0c81740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad0876300 0x55bad08492a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.375+0000 7febe0c81740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad08766e0 0x55bad0879090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.375+0000 7febe0c81740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bad08497e0 0x55bad08795d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.375+0000 7febe0c81740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bad0810ab0 con 0x55bad08766e0 2024-04-04T06:09:09.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.375+0000 7febe0c81740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bad07b2fa0 con 0x55bad0876300 2024-04-04T06:09:09.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.375+0000 7febe0c81740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bad075b560 con 0x55bad08497e0 2024-04-04T06:09:09.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.375+0000 7febcea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad08766e0 0x55bad0879090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.375+0000 7febcea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad08766e0 0x55bad0879090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52222/0 (socket says 172.21.15.159:52222) 2024-04-04T06:09:09.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.375+0000 7febcea00640 1 -- 172.21.15.159:0/2182285310 learned_addr learned my addr 172.21.15.159:0/2182285310 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:09.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.375+0000 7febcea00640 1 -- 172.21.15.159:0/2182285310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bad08497e0 msgr2=0x55bad08795d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:09.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.375+0000 7febcea00640 1 --2- 172.21.15.159:0/2182285310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bad08497e0 0x55bad08795d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.375+0000 7febd7400640 1 --2- 172.21.15.159:0/2182285310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad0876300 0x55bad08492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.375+0000 7febcea00640 1 -- 172.21.15.159:0/2182285310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad0876300 msgr2=0x55bad08492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:09.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.376+0000 7febcea00640 1 --2- 172.21.15.159:0/2182285310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad0876300 0x55bad08492a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.376+0000 7febcea00640 1 -- 172.21.15.159:0/2182285310 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bad04c72d0 con 0x55bad08766e0 2024-04-04T06:09:09.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.376+0000 7febd7400640 1 --2- 172.21.15.159:0/2182285310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad0876300 0x55bad08492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:09.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.376+0000 7febcea00640 1 --2- 172.21.15.159:0/2182285310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad08766e0 0x55bad0879090 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7febc00ac1b0 tx=0x7febc007b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb563241c2a51a6e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:09.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.376+0000 7febd7e00640 1 --2- 172.21.15.159:0/3734728930 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7febb406c8f0 0x7febb406edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7febd0006fd0 tx=0x7febd000a330 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:09.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.376+0000 7febc7400640 1 -- 172.21.15.159:0/2182285310 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc00ab070 con 0x55bad08766e0 2024-04-04T06:09:09.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.376+0000 7febc7400640 1 -- 172.21.15.159:0/2182285310 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc007fd40 con 0x55bad08766e0 2024-04-04T06:09:09.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.376+0000 7febc7400640 1 -- 172.21.15.159:0/2182285310 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0084570 con 0x55bad08766e0 2024-04-04T06:09:09.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.377+0000 7febe0c81740 1 -- 172.21.15.159:0/2182285310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad08766e0 msgr2=0x55bad0879090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:09.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.377+0000 7febe0c81740 1 --2- 172.21.15.159:0/2182285310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad08766e0 0x55bad0879090 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7febc00ac1b0 tx=0x7febc007b3b0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.377+0000 7febe0c81740 1 -- 172.21.15.159:0/2182285310 shutdown_connections 2024-04-04T06:09:09.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.377+0000 7febe0c81740 1 --2- 172.21.15.159:0/2182285310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bad08497e0 0x55bad08795d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.377+0000 7febe0c81740 1 --2- 172.21.15.159:0/2182285310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad08766e0 0x55bad0879090 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.377+0000 7febe0c81740 1 --2- 172.21.15.159:0/2182285310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad0876300 0x55bad08492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.377+0000 7febe0c81740 1 -- 172.21.15.159:0/2182285310 >> 172.21.15.159:0/2182285310 conn(0x55bad084b510 msgr2=0x55bad0894090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:09.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.377+0000 7febe0c81740 1 -- 172.21.15.159:0/2182285310 shutdown_connections 2024-04-04T06:09:09.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.377+0000 7febe0c81740 1 -- 172.21.15.159:0/2182285310 wait complete. 2024-04-04T06:09:09.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.378+0000 7febe0c81740 1 -- start start 2024-04-04T06:09:09.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.378+0000 7febe0c81740 1 Processor -- start 2024-04-04T06:09:09.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.379+0000 7febe0c81740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad0876300 0x55bad088d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.379+0000 7febe0c81740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bad08766e0 0x55bad0867590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.379+0000 7febe0c81740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad088e6c0 0x55bad088c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.379+0000 7febe0c81740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bad076e920 con 0x55bad0876300 2024-04-04T06:09:09.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.379+0000 7febe0c81740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bad07aef50 con 0x55bad088e6c0 2024-04-04T06:09:09.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.379+0000 7febe0c81740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bad07b2fa0 con 0x55bad08766e0 2024-04-04T06:09:09.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.379+0000 7febe0c81740 10 client.0 _ll_register_callbacks cb 0x55bad076e310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:09:09.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.379+0000 7febd7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad0876300 0x55bad088d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.379+0000 7febd7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad0876300 0x55bad088d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52228/0 (socket says 172.21.15.159:52228) 2024-04-04T06:09:09.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.379+0000 7febd7400640 1 -- 172.21.15.159:0/2133352562 learned_addr learned my addr 172.21.15.159:0/2133352562 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:09.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.380+0000 7febcea00640 1 --2- 172.21.15.159:0/2133352562 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bad08766e0 0x55bad0867590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.380+0000 7febd7e00640 1 --2- 172.21.15.159:0/2133352562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad088e6c0 0x55bad088c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.380+0000 7febd7400640 1 -- 172.21.15.159:0/2133352562 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bad08766e0 msgr2=0x55bad0867590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:09.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.380+0000 7febd7400640 1 --2- 172.21.15.159:0/2133352562 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bad08766e0 0x55bad0867590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.380+0000 7febd7400640 1 -- 172.21.15.159:0/2133352562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad088e6c0 msgr2=0x55bad088c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:09.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.380+0000 7febd7400640 1 --2- 172.21.15.159:0/2133352562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad088e6c0 0x55bad088c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.380+0000 7febd7400640 1 -- 172.21.15.159:0/2133352562 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bad05916b0 con 0x55bad0876300 2024-04-04T06:09:09.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.380+0000 7febd7400640 1 --2- 172.21.15.159:0/2133352562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad0876300 0x55bad088d620 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7febc800e520 tx=0x7febc800e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1a759693d4bfc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:09.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.380+0000 7febbbe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:09:09.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.380+0000 7febbbe00640 1 -- 172.21.15.159:0/2133352562 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc8019060 con 0x55bad0876300 2024-04-04T06:09:09.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.380+0000 7febbbe00640 1 -- 172.21.15.159:0/2133352562 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7febc8014af0 con 0x55bad0876300 2024-04-04T06:09:09.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.380+0000 7febe0c81740 1 -- 172.21.15.159:0/2133352562 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febc007fd40 con 0x55bad0876300 2024-04-04T06:09:09.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.381+0000 7febbbe00640 1 -- 172.21.15.159:0/2133352562 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7febc8014cf0 con 0x55bad0876300 2024-04-04T06:09:09.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.381+0000 7febe0c81740 10 client.6201 fetch_fsmap learned FSMap version 33 2024-04-04T06:09:09.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.381+0000 7febe0c81740 1 -- 172.21.15.159:0/2133352562 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55bad088bf20 con 0x55bad0876300 2024-04-04T06:09:09.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.381+0000 7febbbe00640 1 -- 172.21.15.159:0/2133352562 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc007fd40 con 0x55bad0876300 2024-04-04T06:09:09.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.381+0000 7febbbe00640 1 -- 172.21.15.159:0/2133352562 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7febc8007150 con 0x55bad0876300 2024-04-04T06:09:09.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.381+0000 7febe0c81740 10 client.6201 fetch_fsmap finished waiting for FSMap version 33 2024-04-04T06:09:09.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.381+0000 7febe0c81740 10 client.6201 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-04T06:09:09.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.381+0000 7febe0c81740 20 client.6201 populate_metadata read hostname 'smithi159' 2024-04-04T06:09:09.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.381+0000 7febe0c81740 1 --2- 172.21.15.159:0/2133352562 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55bad08afdb0 0x55bad08b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.381+0000 7febe0c81740 4 client.6201 mds_command: new command op to 6039 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:09:09.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.381+0000 7febe0c81740 1 -- 172.21.15.159:0/2133352562 --> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bad07c04e0 con 0x55bad08afdb0 2024-04-04T06:09:09.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.381+0000 7febcea00640 1 --2- 172.21.15.159:0/2133352562 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55bad08afdb0 0x55bad08b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.382+0000 7febbbe00640 10 client.6201 ms_handle_connect on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:09.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.382+0000 7febcea00640 1 --2- 172.21.15.159:0/2133352562 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55bad08afdb0 0x55bad08b21a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b36f1d3b3a16aaf2 server_cookie=ca9c0bad6b1888f8 in_seq=0 out_seq=0 2024-04-04T06:09:09.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.383+0000 7febbbe00640 1 -- 172.21.15.159:0/2133352562 <== mds.0 v2:172.21.15.159:6833/962475038 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bad07c04e0 con 0x55bad08afdb0 2024-04-04T06:09:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.383+0000 7febbbe00640 10 client.6201 handle_command_reply: tid=0 2024-04-04T06:09:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.383+0000 7febe0c81740 1 client.6201 shutdown 2024-04-04T06:09:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.383+0000 7febe0c81740 1 -- 172.21.15.159:0/2133352562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad0876300 msgr2=0x55bad088d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.383+0000 7febe0c81740 1 --2- 172.21.15.159:0/2133352562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad0876300 0x55bad088d620 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7febc800e520 tx=0x7febc800e9e0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.384+0000 7febe0c81740 1 -- 172.21.15.159:0/2133352562 shutdown_connections 2024-04-04T06:09:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.384+0000 7febe0c81740 1 --2- 172.21.15.159:0/2133352562 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55bad08afdb0 0x55bad08b21a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.384+0000 7febe0c81740 1 --2- 172.21.15.159:0/2133352562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad088e6c0 0x55bad088c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.384+0000 7febe0c81740 1 --2- 172.21.15.159:0/2133352562 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bad08766e0 0x55bad0867590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.384+0000 7febe0c81740 1 --2- 172.21.15.159:0/2133352562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad0876300 0x55bad088d620 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.384+0000 7febe0c81740 1 -- 172.21.15.159:0/2133352562 >> 172.21.15.159:0/2133352562 conn(0x55bad08497e0 msgr2=0x55bad088def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.384+0000 7febbbe00640 0 client.6201 ms_handle_reset on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.384+0000 7febe0c81740 1 -- 172.21.15.159:0/2133352562 shutdown_connections 2024-04-04T06:09:09.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.384+0000 7febe0c81740 1 -- 172.21.15.159:0/2133352562 wait complete. 2024-04-04T06:09:09.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.384+0000 7febe0c81740 20 client.6201 trim_cache size 0 max 16384 2024-04-04T06:09:09.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.404+0000 7febe0c81740 1 Processor -- start 2024-04-04T06:09:09.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.404+0000 7febe0c81740 1 -- start start 2024-04-04T06:09:09.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.405+0000 7febe0c81740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad0876300 0x55bad0878cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.405+0000 7febe0c81740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad08766e0 0x55bad0893e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.405+0000 7febe0c81740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bad088e6c0 0x55bad08943d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.405+0000 7febe0c81740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7febd815f6d0 con 0x55bad08766e0 2024-04-04T06:09:09.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.405+0000 7febe0c81740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7febd815ea30 con 0x55bad0876300 2024-04-04T06:09:09.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.405+0000 7febe0c81740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7febd815f080 con 0x55bad088e6c0 2024-04-04T06:09:09.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.405+0000 7febcea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad08766e0 0x55bad0893e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.405+0000 7febcea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad08766e0 0x55bad0893e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52244/0 (socket says 172.21.15.159:52244) 2024-04-04T06:09:09.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.405+0000 7febcea00640 1 -- 172.21.15.159:0/4061761086 learned_addr learned my addr 172.21.15.159:0/4061761086 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:09.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.405+0000 7febd7400640 1 --2- 172.21.15.159:0/4061761086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad0876300 0x55bad0878cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.405+0000 7febd7e00640 1 --2- 172.21.15.159:0/4061761086 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bad088e6c0 0x55bad08943d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.405+0000 7febcea00640 1 -- 172.21.15.159:0/4061761086 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bad088e6c0 msgr2=0x55bad08943d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:09.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.405+0000 7febcea00640 1 --2- 172.21.15.159:0/4061761086 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bad088e6c0 0x55bad08943d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.405+0000 7febcea00640 1 -- 172.21.15.159:0/4061761086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad0876300 msgr2=0x55bad0878cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:09.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.405+0000 7febcea00640 1 --2- 172.21.15.159:0/4061761086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad0876300 0x55bad0878cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.405+0000 7febcea00640 1 -- 172.21.15.159:0/4061761086 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bad0752d00 con 0x55bad08766e0 2024-04-04T06:09:09.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.406+0000 7febcea00640 1 --2- 172.21.15.159:0/4061761086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad08766e0 0x55bad0893e90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7febc007b920 tx=0x7febc0094000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967dd6e465597176 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:09.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.406+0000 7febc6a00640 1 -- 172.21.15.159:0/4061761086 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0095040 con 0x55bad08766e0 2024-04-04T06:09:09.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.406+0000 7febc6a00640 1 -- 172.21.15.159:0/4061761086 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc007fd40 con 0x55bad08766e0 2024-04-04T06:09:09.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.406+0000 7febc6a00640 1 -- 172.21.15.159:0/4061761086 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc00ab040 con 0x55bad08766e0 2024-04-04T06:09:09.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.406+0000 7febe0c81740 1 -- 172.21.15.159:0/4061761086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad08766e0 msgr2=0x55bad0893e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:09.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.406+0000 7febe0c81740 1 --2- 172.21.15.159:0/4061761086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad08766e0 0x55bad0893e90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7febc007b920 tx=0x7febc0094000 comp rx=0 tx=0).stop 2024-04-04T06:09:09.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.407+0000 7febe0c81740 1 -- 172.21.15.159:0/4061761086 shutdown_connections 2024-04-04T06:09:09.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.407+0000 7febe0c81740 1 --2- 172.21.15.159:0/4061761086 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bad088e6c0 0x55bad08943d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.407+0000 7febe0c81740 1 --2- 172.21.15.159:0/4061761086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad08766e0 0x55bad0893e90 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.407+0000 7febe0c81740 1 --2- 172.21.15.159:0/4061761086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad0876300 0x55bad0878cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.407+0000 7febe0c81740 1 -- 172.21.15.159:0/4061761086 >> 172.21.15.159:0/4061761086 conn(0x55bad08497e0 msgr2=0x55bad088d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:09.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.407+0000 7febe0c81740 1 -- 172.21.15.159:0/4061761086 shutdown_connections 2024-04-04T06:09:09.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.407+0000 7febe0c81740 1 -- 172.21.15.159:0/4061761086 wait complete. 2024-04-04T06:09:09.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.407+0000 7febe0c81740 1 -- start start 2024-04-04T06:09:09.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.408+0000 7febe0c81740 1 Processor -- start 2024-04-04T06:09:09.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.408+0000 7febe0c81740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad0876300 0x55bad08b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.409+0000 7febe0c81740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad08766e0 0x55bad090def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.409+0000 7febe0c81740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bad088e6c0 0x55bad0878cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.409+0000 7febe0c81740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bad088cda0 con 0x55bad0876300 2024-04-04T06:09:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.409+0000 7febe0c81740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bad0810ab0 con 0x55bad08766e0 2024-04-04T06:09:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.409+0000 7febe0c81740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bad087b7e0 con 0x55bad088e6c0 2024-04-04T06:09:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.409+0000 7febe0c81740 10 client.0 _ll_register_callbacks cb 0x7febc8002090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:09:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.409+0000 7febd7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad0876300 0x55bad08b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.409+0000 7febd7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad0876300 0x55bad08b61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52246/0 (socket says 172.21.15.159:52246) 2024-04-04T06:09:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.409+0000 7febd7400640 1 -- 172.21.15.159:0/468517738 learned_addr learned my addr 172.21.15.159:0/468517738 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.409+0000 7febcea00640 1 --2- 172.21.15.159:0/468517738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad08766e0 0x55bad090def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.409+0000 7febd7400640 1 -- 172.21.15.159:0/468517738 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bad088e6c0 msgr2=0x55bad0878cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.409+0000 7febd7e00640 1 --2- 172.21.15.159:0/468517738 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bad088e6c0 0x55bad0878cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.409+0000 7febd7400640 1 --2- 172.21.15.159:0/468517738 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bad088e6c0 0x55bad0878cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.409+0000 7febd7400640 1 -- 172.21.15.159:0/468517738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad08766e0 msgr2=0x55bad090def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.409+0000 7febd7400640 1 --2- 172.21.15.159:0/468517738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad08766e0 0x55bad090def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.409+0000 7febd7400640 1 -- 172.21.15.159:0/468517738 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bad08a6f30 con 0x55bad0876300 2024-04-04T06:09:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.409+0000 7febcea00640 1 --2- 172.21.15.159:0/468517738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad08766e0 0x55bad090def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.410+0000 7febd7400640 1 --2- 172.21.15.159:0/468517738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad0876300 0x55bad08b61c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7febc8001520 tx=0x7febc8001400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1b23a1d718099d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.410+0000 7febbbe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:09:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.410+0000 7febbbe00640 1 -- 172.21.15.159:0/468517738 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc802e060 con 0x55bad0876300 2024-04-04T06:09:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.410+0000 7febbbe00640 1 -- 172.21.15.159:0/468517738 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7febc8013650 con 0x55bad0876300 2024-04-04T06:09:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.410+0000 7febe0c81740 1 -- 172.21.15.159:0/468517738 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febc007fd40 con 0x55bad0876300 2024-04-04T06:09:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.410+0000 7febbbe00640 1 -- 172.21.15.159:0/468517738 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7febc8034050 con 0x55bad0876300 2024-04-04T06:09:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.410+0000 7febe0c81740 10 client.6207 fetch_fsmap learned FSMap version 33 2024-04-04T06:09:09.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.410+0000 7febe0c81740 1 -- 172.21.15.159:0/468517738 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55bad088cfe0 con 0x55bad0876300 2024-04-04T06:09:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.410+0000 7febbbe00640 1 -- 172.21.15.159:0/468517738 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc007fd40 con 0x55bad0876300 2024-04-04T06:09:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.411+0000 7febbbe00640 1 -- 172.21.15.159:0/468517738 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7febc8037020 con 0x55bad0876300 2024-04-04T06:09:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.411+0000 7febe0c81740 10 client.6207 fetch_fsmap finished waiting for FSMap version 33 2024-04-04T06:09:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.411+0000 7febe0c81740 10 client.6207 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-04T06:09:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.411+0000 7febe0c81740 20 client.6207 populate_metadata read hostname 'smithi159' 2024-04-04T06:09:09.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.411+0000 7febe0c81740 1 --2- 172.21.15.159:0/468517738 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55bad08945d0 0x55bad084dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.411+0000 7febe0c81740 4 client.6207 mds_command: new command op to 6039 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-04T06:09:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.411+0000 7febe0c81740 1 -- 172.21.15.159:0/468517738 --> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] -- command(tid 0: {"prefix": "status"}) -- 0x55bad07ba830 con 0x55bad08945d0 2024-04-04T06:09:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.411+0000 7febcea00640 1 --2- 172.21.15.159:0/468517738 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55bad08945d0 0x55bad084dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.411+0000 7febbbe00640 10 client.6207 ms_handle_connect on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.411+0000 7febcea00640 1 --2- 172.21.15.159:0/468517738 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55bad08945d0 0x55bad084dfa0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee9d5ef101168b8f server_cookie=bbd31dc1d343adee in_seq=0 out_seq=0 2024-04-04T06:09:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.412+0000 7febbbe00640 1 -- 172.21.15.159:0/468517738 <== mds.0 v2:172.21.15.159:6833/962475038 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55bad07ba830 con 0x55bad08945d0 2024-04-04T06:09:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.412+0000 7febbbe00640 10 client.6207 handle_command_reply: tid=0 2024-04-04T06:09:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.412+0000 7febe0c81740 1 client.6207 shutdown 2024-04-04T06:09:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.412+0000 7febe0c81740 1 -- 172.21.15.159:0/468517738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad0876300 msgr2=0x55bad08b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.412+0000 7febe0c81740 1 --2- 172.21.15.159:0/468517738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad0876300 0x55bad08b61c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7febc8001520 tx=0x7febc8001400 comp rx=0 tx=0).stop 2024-04-04T06:09:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.412+0000 7febe0c81740 1 -- 172.21.15.159:0/468517738 shutdown_connections 2024-04-04T06:09:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.412+0000 7febe0c81740 1 --2- 172.21.15.159:0/468517738 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bad088e6c0 0x55bad0878cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.412+0000 7febe0c81740 1 --2- 172.21.15.159:0/468517738 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55bad08945d0 0x55bad084dfa0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.412+0000 7febe0c81740 1 --2- 172.21.15.159:0/468517738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bad08766e0 0x55bad090def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.412+0000 7febe0c81740 1 --2- 172.21.15.159:0/468517738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bad0876300 0x55bad08b61c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.412+0000 7febe0c81740 1 -- 172.21.15.159:0/468517738 >> 172.21.15.159:0/468517738 conn(0x55bad08497e0 msgr2=0x55bad08a6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:09.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.412+0000 7febbbe00640 0 client.6207 ms_handle_reset on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:09.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.412+0000 7febe0c81740 1 -- 172.21.15.159:0/468517738 shutdown_connections 2024-04-04T06:09:09.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.412+0000 7febe0c81740 1 -- 172.21.15.159:0/468517738 wait complete. 2024-04-04T06:09:09.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.412+0000 7febe0c81740 20 client.6207 trim_cache size 0 max 16384 2024-04-04T06:09:09.409 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:09:09.409 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_fsid": "30311045-f714-48fa-832b-4cb2ba8bf0bf", 2024-04-04T06:09:09.409 INFO:teuthology.orchestra.run.smithi159.stdout: "whoami": 0, 2024-04-04T06:09:09.410 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 6039, 2024-04-04T06:09:09.410 INFO:teuthology.orchestra.run.smithi159.stdout: "want_state": "up:active", 2024-04-04T06:09:09.410 INFO:teuthology.orchestra.run.smithi159.stdout: "state": "up:active", 2024-04-04T06:09:09.410 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs", 2024-04-04T06:09:09.410 INFO:teuthology.orchestra.run.smithi159.stdout: "rank_uptime": 11.826288984, 2024-04-04T06:09:09.410 INFO:teuthology.orchestra.run.smithi159.stdout: "mdsmap_epoch": 33, 2024-04-04T06:09:09.410 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch": 59, 2024-04-04T06:09:09.410 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch_barrier": 59, 2024-04-04T06:09:09.410 INFO:teuthology.orchestra.run.smithi159.stdout: "uptime": 17.254988150999999 2024-04-04T06:09:09.410 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:09:09.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.413+0000 7febdca00640 1 -- 172.21.15.159:0/3734728930 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7febb406c8f0 msgr2=0x7febb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:09.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.413+0000 7febdca00640 1 --2- 172.21.15.159:0/3734728930 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7febb406c8f0 0x7febb406edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7febd0006fd0 tx=0x7febd000a330 comp rx=0 tx=0).stop 2024-04-04T06:09:09.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.413+0000 7febdca00640 1 -- 172.21.15.159:0/3734728930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febd80d2770 msgr2=0x7febd8153450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:09.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.413+0000 7febdca00640 1 --2- 172.21.15.159:0/3734728930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febd80d2770 0x7febd8153450 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7febc0000c00 tx=0x7febc00046c0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.413+0000 7febdca00640 1 -- 172.21.15.159:0/3734728930 shutdown_connections 2024-04-04T06:09:09.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.413+0000 7febdca00640 1 --2- 172.21.15.159:0/3734728930 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7febb406c8f0 0x7febb406edb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.413+0000 7febdca00640 1 --2- 172.21.15.159:0/3734728930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febd815e400 0x7febd80c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.413+0000 7febdca00640 1 --2- 172.21.15.159:0/3734728930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febd80d2770 0x7febd8153450 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.413+0000 7febdca00640 1 --2- 172.21.15.159:0/3734728930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febd80d1ba0 0x7febd80c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.413+0000 7febdca00640 1 -- 172.21.15.159:0/3734728930 >> 172.21.15.159:0/3734728930 conn(0x7febd8067280 msgr2=0x7febd80b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:09.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.413+0000 7febdca00640 1 -- 172.21.15.159:0/3734728930 shutdown_connections 2024-04-04T06:09:09.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.413+0000 7febdca00640 1 -- 172.21.15.159:0/3734728930 wait complete. 2024-04-04T06:09:09.423 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-04T06:09:09.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.635+0000 7fa69b200640 1 Processor -- start 2024-04-04T06:09:09.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.636+0000 7fa69b200640 1 -- start start 2024-04-04T06:09:09.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.637+0000 7fa69b200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6940d0990 0x7fa6940aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.637+0000 7fa69b200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6940afab0 con 0x7fa694153580 2024-04-04T06:09:09.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.637+0000 7fa69b200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa69415cc80 con 0x7fa6940af2e0 2024-04-04T06:09:09.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.637+0000 7fa69b200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa69415da00 con 0x7fa6940d0990 2024-04-04T06:09:09.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.637+0000 7fa693e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fa694153580 0x7fa6940d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:34246/0 (socket says 172.21.15.159:34246) 2024-04-04T06:09:09.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.637+0000 7fa693e00640 1 -- 172.21.15.159:0/1618225871 learned_addr learned my addr 172.21.15.159:0/1618225871 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:09:09.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.638+0000 7fa68b400640 1 --2- 172.21.15.159:0/1618225871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6940d0990 0x7fa6940aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.638+0000 7fa693400640 1 -- 172.21.15.159:0/1618225871 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 882765482 0 0) 0x7fa6940afab0 con 0x7fa694153580 2024-04-04T06:09:09.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.638+0000 7fa693400640 1 -- 172.21.15.159:0/1618225871 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa678003520 con 0x7fa694153580 2024-04-04T06:09:09.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.638+0000 7fa693400640 1 -- 172.21.15.159:0/1618225871 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1127254025 0 0) 0x7fa678003520 con 0x7fa694153580 2024-04-04T06:09:09.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.638+0000 7fa693400640 1 -- 172.21.15.159:0/1618225871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6940d0990 msgr2=0x7fa6940aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:09.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.638+0000 7fa693400640 1 --2- 172.21.15.159:0/1618225871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6940d0990 0x7fa6940aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.638+0000 7fa693400640 1 -- 172.21.15.159:0/1618225871 >> v1:172.21.15.178:6789/0 conn(0x7fa6940af2e0 legacy=0x7fa6941600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:09.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.638+0000 7fa693400640 1 -- 172.21.15.159:0/1618225871 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69415cf20 con 0x7fa694153580 2024-04-04T06:09:09.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.638+0000 7fa693400640 1 -- 172.21.15.159:0/1618225871 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fa67c001670 con 0x7fa694153580 2024-04-04T06:09:09.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.638+0000 7fa693400640 1 -- 172.21.15.159:0/1618225871 >> v1:172.21.15.159:6789/0 conn(0x7fa694153580 legacy=0x7fa6940d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:09.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.639+0000 7fa693400640 1 --2- 172.21.15.159:0/1618225871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6940d0990 0x7fa6780039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.639+0000 7fa693400640 1 --2- 172.21.15.159:0/1618225871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa678003f60 0x7fa678004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.639+0000 7fa693400640 1 --2- 172.21.15.159:0/1618225871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa678004d40 0x7fa6780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.639+0000 7fa693400640 1 -- 172.21.15.159:0/1618225871 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa69415da00 con 0x7fa678004d40 2024-04-04T06:09:09.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.639+0000 7fa693400640 1 -- 172.21.15.159:0/1618225871 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa67800bb50 con 0x7fa678003f60 2024-04-04T06:09:09.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.639+0000 7fa693400640 1 -- 172.21.15.159:0/1618225871 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa67800bd50 con 0x7fa6940d0990 2024-04-04T06:09:09.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.639+0000 7fa699200640 1 --2- 172.21.15.159:0/1618225871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa678004d40 0x7fa6780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.639+0000 7fa693e00640 1 --2- 172.21.15.159:0/1618225871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6940d0990 0x7fa6780039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.639+0000 7fa68b400640 1 --2- 172.21.15.159:0/1618225871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa678003f60 0x7fa678004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.639+0000 7fa699200640 1 -- 172.21.15.159:0/1618225871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6940d0990 msgr2=0x7fa6780039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:09.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.639+0000 7fa699200640 1 --2- 172.21.15.159:0/1618225871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6940d0990 0x7fa6780039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.639+0000 7fa699200640 1 -- 172.21.15.159:0/1618225871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa678003f60 msgr2=0x7fa678004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:09.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.639+0000 7fa699200640 1 --2- 172.21.15.159:0/1618225871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa678003f60 0x7fa678004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.640+0000 7fa699200640 1 -- 172.21.15.159:0/1618225871 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa678003520 con 0x7fa678004d40 2024-04-04T06:09:09.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.640+0000 7fa68b400640 1 --2- 172.21.15.159:0/1618225871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa678003f60 0x7fa678004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:09.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.641+0000 7fa699200640 1 --2- 172.21.15.159:0/1618225871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa678004d40 0x7fa6780051c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa68c001e50 tx=0x7fa678005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2b882b41440d02 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:09.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.641+0000 7fa693400640 1 -- 172.21.15.159:0/1618225871 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c001a70 con 0x7fa678004d40 2024-04-04T06:09:09.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.641+0000 7fa693400640 1 -- 172.21.15.159:0/1618225871 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa68c003b70 con 0x7fa678004d40 2024-04-04T06:09:09.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.641+0000 7fa693400640 1 -- 172.21.15.159:0/1618225871 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c0050d0 con 0x7fa678004d40 2024-04-04T06:09:09.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.642+0000 7fa69b200640 1 -- 172.21.15.159:0/1618225871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa678004d40 msgr2=0x7fa6780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:09.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.642+0000 7fa69b200640 1 --2- 172.21.15.159:0/1618225871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa678004d40 0x7fa6780051c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa68c001e50 tx=0x7fa678005830 comp rx=0 tx=0).stop 2024-04-04T06:09:09.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.642+0000 7fa699200640 1 -- 172.21.15.159:0/1618225871 reap_dead start 2024-04-04T06:09:09.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.642+0000 7fa69b200640 1 -- 172.21.15.159:0/1618225871 shutdown_connections 2024-04-04T06:09:09.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.642+0000 7fa69b200640 1 -- 172.21.15.159:0/1618225871 >> 172.21.15.159:0/1618225871 conn(0x7fa69406b750 msgr2=0x7fa69406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:09.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.642+0000 7fa69b200640 1 -- 172.21.15.159:0/1618225871 shutdown_connections 2024-04-04T06:09:09.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.642+0000 7fa69b200640 1 -- 172.21.15.159:0/1618225871 wait complete. 2024-04-04T06:09:09.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.644+0000 7fa69b200640 1 Processor -- start 2024-04-04T06:09:09.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.644+0000 7fa69b200640 1 -- start start 2024-04-04T06:09:09.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.645+0000 7fa69b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa678004d40 0x7fa6941f2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.645+0000 7fa69b200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6941e52f0 0x7fa6941e5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.645+0000 7fa69b200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6940b1d60 0x7fa6940b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.645+0000 7fa69b200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa694153960 con 0x7fa678004d40 2024-04-04T06:09:09.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.645+0000 7fa69b200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa69415d4e0 con 0x7fa6940b1d60 2024-04-04T06:09:09.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.645+0000 7fa69b200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa6940af710 con 0x7fa6941e52f0 2024-04-04T06:09:09.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.645+0000 7fa693e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa678004d40 0x7fa6941f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.645+0000 7fa693e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa678004d40 0x7fa6941f2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52268/0 (socket says 172.21.15.159:52268) 2024-04-04T06:09:09.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.645+0000 7fa693e00640 1 -- 172.21.15.159:0/279350326 learned_addr learned my addr 172.21.15.159:0/279350326 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:09.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.646+0000 7fa68b400640 1 --2- 172.21.15.159:0/279350326 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6941e52f0 0x7fa6941e5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.646+0000 7fa699200640 1 --2- 172.21.15.159:0/279350326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6940b1d60 0x7fa6940b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.646+0000 7fa693e00640 1 -- 172.21.15.159:0/279350326 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6941e52f0 msgr2=0x7fa6941e5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:09.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.646+0000 7fa693e00640 1 --2- 172.21.15.159:0/279350326 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6941e52f0 0x7fa6941e5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.646+0000 7fa693e00640 1 -- 172.21.15.159:0/279350326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6940b1d60 msgr2=0x7fa6940b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:09.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.646+0000 7fa693e00640 1 --2- 172.21.15.159:0/279350326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6940b1d60 0x7fa6940b21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.646+0000 7fa693e00640 1 -- 172.21.15.159:0/279350326 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6940b0cc0 con 0x7fa678004d40 2024-04-04T06:09:09.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.646+0000 7fa693e00640 1 --2- 172.21.15.159:0/279350326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa678004d40 0x7fa6941f2330 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00e800 tx=0x7fa67c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:09.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.646+0000 7fa690c00640 1 -- 172.21.15.159:0/279350326 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c00c8d0 con 0x7fa678004d40 2024-04-04T06:09:09.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.646+0000 7fa690c00640 1 -- 172.21.15.159:0/279350326 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa67c018070 con 0x7fa678004d40 2024-04-04T06:09:09.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.647+0000 7fa69b200640 1 -- 172.21.15.159:0/279350326 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6940b0fa0 con 0x7fa678004d40 2024-04-04T06:09:09.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.647+0000 7fa690c00640 1 -- 172.21.15.159:0/279350326 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c00ccb0 con 0x7fa678004d40 2024-04-04T06:09:09.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.647+0000 7fa69b200640 1 -- 172.21.15.159:0/279350326 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6940b1560 con 0x7fa678004d40 2024-04-04T06:09:09.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.648+0000 7fa69b200640 1 -- 172.21.15.159:0/279350326 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6940685c0 con 0x7fa678004d40 2024-04-04T06:09:09.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.649+0000 7fa690c00640 1 -- 172.21.15.159:0/279350326 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa67c013420 con 0x7fa678004d40 2024-04-04T06:09:09.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.650+0000 7fa690c00640 1 --2- 172.21.15.159:0/279350326 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa67006c920 0x7fa67006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:09.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.651+0000 7fa690c00640 1 -- 172.21.15.159:0/279350326 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fa67c08b820 con 0x7fa678004d40 2024-04-04T06:09:09.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.651+0000 7fa68b400640 1 --2- 172.21.15.159:0/279350326 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa67006c920 0x7fa67006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:09.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.652+0000 7fa68b400640 1 --2- 172.21.15.159:0/279350326 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa67006c920 0x7fa67006ede0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa6940cfa50 tx=0x7fa684008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:09.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.655+0000 7fa690c00640 1 -- 172.21.15.159:0/279350326 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa67c059c40 con 0x7fa678004d40 2024-04-04T06:09:09.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.898+0000 7fa69b200640 1 -- 172.21.15.159:0/279350326 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa6940cc640 con 0x7fa678004d40 2024-04-04T06:09:09.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.899+0000 7fa690c00640 1 -- 172.21.15.159:0/279350326 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) ==== 53+0+73 (secure 0 0 0) 0x7fa67c059540 con 0x7fa678004d40 2024-04-04T06:09:09.892 INFO:teuthology.orchestra.run.smithi159.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-04T06:09:09.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.901+0000 7fa69b200640 1 -- 172.21.15.159:0/279350326 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa67006c920 msgr2=0x7fa67006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:09.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.901+0000 7fa69b200640 1 --2- 172.21.15.159:0/279350326 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa67006c920 0x7fa67006ede0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa6940cfa50 tx=0x7fa684008040 comp rx=0 tx=0).stop 2024-04-04T06:09:09.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.901+0000 7fa69b200640 1 -- 172.21.15.159:0/279350326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa678004d40 msgr2=0x7fa6941f2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:09.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.901+0000 7fa69b200640 1 --2- 172.21.15.159:0/279350326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa678004d40 0x7fa6941f2330 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00e800 tx=0x7fa67c00ecc0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.901+0000 7fa69b200640 1 -- 172.21.15.159:0/279350326 shutdown_connections 2024-04-04T06:09:09.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.901+0000 7fa69b200640 1 --2- 172.21.15.159:0/279350326 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa67006c920 0x7fa67006ede0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.901+0000 7fa69b200640 1 --2- 172.21.15.159:0/279350326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6940b1d60 0x7fa6940b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.901+0000 7fa69b200640 1 --2- 172.21.15.159:0/279350326 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa6941e52f0 0x7fa6941e5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.901+0000 7fa69b200640 1 --2- 172.21.15.159:0/279350326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa678004d40 0x7fa6941f2330 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:09.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.901+0000 7fa69b200640 1 -- 172.21.15.159:0/279350326 >> 172.21.15.159:0/279350326 conn(0x7fa69406b750 msgr2=0x7fa6940b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:09.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.902+0000 7fa69b200640 1 -- 172.21.15.159:0/279350326 shutdown_connections 2024-04-04T06:09:09.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:09.902+0000 7fa69b200640 1 -- 172.21.15.159:0/279350326 wait complete. 2024-04-04T06:09:09.914 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-04T06:09:09.914 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-04T06:09:09.914 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi159.front.sepia.ceph.com 2024-04-04T06:09:09.914 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-04T06:09:09.914 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-04T06:09:09.914 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-04T06:09:09.914 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-04T06:09:09.914 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-04T06:09:09.914 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:09:09.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:09:09.915 DEBUG:teuthology.orchestra.run.smithi159:> ip addr 2024-04-04T06:09:09.963 INFO:teuthology.orchestra.run.smithi159.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:09:09.963 INFO:teuthology.orchestra.run.smithi159.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:09:09.963 INFO:teuthology.orchestra.run.smithi159.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:09:09.963 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:09:09.963 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 ::1/128 scope host 2024-04-04T06:09:09.963 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:09:09.963 INFO:teuthology.orchestra.run.smithi159.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:09:09.963 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c6 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:09:09.963 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f0 2024-04-04T06:09:09.964 INFO:teuthology.orchestra.run.smithi159.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:09:09.964 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c7 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:09:09.964 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f1 2024-04-04T06:09:09.964 INFO:teuthology.orchestra.run.smithi159.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:09:09.964 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4e brd ff:ff:ff:ff:ff:ff 2024-04-04T06:09:09.964 INFO:teuthology.orchestra.run.smithi159.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-04T06:09:09.964 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:09:09.964 INFO:teuthology.orchestra.run.smithi159.stdout: inet 172.21.15.159/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-04T06:09:09.964 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft 42383sec preferred_lft 42383sec 2024-04-04T06:09:09.964 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::ec4:7aff:fe88:724f/64 scope link 2024-04-04T06:09:09.964 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:09:09.964 INFO:teuthology.orchestra.run.smithi159.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-04T06:09:09.965 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:09:09.965 INFO:teuthology.orchestra.run.smithi159.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-04T06:09:09.965 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:09:09.965 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::989e:e4ff:fea5:7568/64 scope link 2024-04-04T06:09:09.965 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:09:09.965 INFO:teuthology.orchestra.run.smithi159.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:09:09.965 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether a2:e3:e0:cb:0a:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-04T06:09:09.965 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::a0e3:e0ff:fecb:a73/64 scope link 2024-04-04T06:09:09.965 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:09:09.965 INFO:teuthology.orchestra.run.smithi159.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:09:09.965 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-04T06:09:09.965 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::c0a:8aff:fe54:b58f/64 scope link 2024-04-04T06:09:09.965 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:09:09.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:09:09.966 DEBUG:teuthology.orchestra.run.smithi159:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-04T06:09:10.042 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:09:10.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:09:10.048 DEBUG:teuthology.orchestra.run.smithi159:> route 2024-04-04T06:09:10.098 INFO:teuthology.orchestra.run.smithi159.stdout:Kernel IP routing table 2024-04-04T06:09:10.098 INFO:teuthology.orchestra.run.smithi159.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-04T06:09:10.098 INFO:teuthology.orchestra.run.smithi159.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-04T06:09:10.098 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-04T06:09:10.098 INFO:teuthology.orchestra.run.smithi159.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-04T06:09:10.099 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:09:10.099 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:09:10.099 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-04T06:09:10.099 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-04T06:09:10.099 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-04T06:09:10.099 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:09:10.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:09:10.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:09:10.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:09:10.338 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-04-04T06:09:10.382 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-04T06:09:10.382 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-04T06:09:10.383 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:09:10.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:09:10.383 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set brx.0 up 2024-04-04T06:09:10.509 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:09:10.509 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-04T06:09:10.559 INFO:teuthology.orchestra.run.smithi159.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:09:10.559 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:09:10.559 DEBUG:teuthology.orchestra.run.smithi159:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:09:10.607 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-04T06:09:10.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:09:10.607 DEBUG:teuthology.orchestra.run.smithi159:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:09:10.782 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:09:10.782 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:09:10.782 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:09:10.802 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:09:10.802 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:09:10.802 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:09:10.802 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:09:10.802 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:09:10.832 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:09:10.832 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:09:10.832 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:09:10.832 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:09:10.832 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:09:10.832 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-04T06:09:10.832 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-04T06:09:10.832 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30311045-f714-48fa-832b-4cb2ba8bf0bf" will pass to kernel 2024-04-04T06:09:10.835 INFO:teuthology.orchestra.run.smithi159.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-04T06:09:10.835 INFO:teuthology.orchestra.run.smithi159.stdout: You just mounted a file system that supports labels which does not 2024-04-04T06:09:10.835 INFO:teuthology.orchestra.run.smithi159.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-04T06:09:10.835 INFO:teuthology.orchestra.run.smithi159.stdout: applications will generate AVC messages and not be allowed access to 2024-04-04T06:09:10.835 INFO:teuthology.orchestra.run.smithi159.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-04T06:09:10.839 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-04T06:09:10.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:09:10.840 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:09:10.926 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:09:11.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:09:11.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:09:11.048 INFO:teuthology.orchestra.run.smithi159.stdout:client5296 2024-04-04T06:09:11.049 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:09:11.049 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:09:11.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:09:11.096 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-04-04T06:09:11.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:09:11.199 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-04-04T06:09:11.251 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:09:11.251 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:09:11.251 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:09:11.251 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:09:11.252 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-04-04T06:09:11.252 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:09:11.386 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627777 2024-04-04T06:09:11.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:09:11.399 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:09:11.461 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:09:11.491 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:09:11.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.668+0000 7efdc1800640 1 Processor -- start 2024-04-04T06:09:11.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.668+0000 7efdc1800640 1 -- start start 2024-04-04T06:09:11.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.670+0000 7efdc1800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efdbc15e4a0 0x7efdbc0b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:11.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.670+0000 7efdc1800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7efdbc0d2310 0x7efdbc0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:11.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.670+0000 7efdc1800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdbc15e880 con 0x7efdbc0cf530 2024-04-04T06:09:11.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.670+0000 7efdc1800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7efdbc0cf9a0 con 0x7efdbc0d2310 2024-04-04T06:09:11.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.670+0000 7efdc1800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7efdbc0d34e0 con 0x7efdbc15e4a0 2024-04-04T06:09:11.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.670+0000 7efdbb400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7efdbc0d2310 0x7efdbc0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:11.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.670+0000 7efdbb400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7efdbc0d2310 0x7efdbc0ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52274/0 (socket says 172.21.15.159:52274) 2024-04-04T06:09:11.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.670+0000 7efdc0e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7efdbc0cf530 0x7efdbc0d3c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:34256/0 (socket says 172.21.15.159:34256) 2024-04-04T06:09:11.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.670+0000 7efdbb400640 1 -- 172.21.15.159:0/99359436 learned_addr learned my addr 172.21.15.159:0/99359436 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:11.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.670+0000 7efdbbe00640 1 --2- 172.21.15.159:0/99359436 >> v2:172.21.15.178:3300/0 conn(0x7efdbc15e4a0 0x7efdbc0b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:11.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.671+0000 7efdbb400640 1 -- 172.21.15.159:0/99359436 >> v2:172.21.15.178:3300/0 conn(0x7efdbc15e4a0 msgr2=0x7efdbc0b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:11.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.671+0000 7efdbb400640 1 --2- 172.21.15.159:0/99359436 >> v2:172.21.15.178:3300/0 conn(0x7efdbc15e4a0 0x7efdbc0b9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:11.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.671+0000 7efdbb400640 1 -- 172.21.15.159:0/99359436 >> v1:172.21.15.159:6789/0 conn(0x7efdbc0cf530 legacy=0x7efdbc0d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:11.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.671+0000 7efdbb400640 1 -- 172.21.15.159:0/99359436 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdbc0d28d0 con 0x7efdbc0d2310 2024-04-04T06:09:11.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.672+0000 7efdbb400640 1 --2- 172.21.15.159:0/99359436 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdbc0d2310 0x7efdbc0ceff0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0d3660 tx=0x7efda802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8c760374feaaf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:11.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.672+0000 7efdbaa00640 1 -- 172.21.15.159:0/99359436 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8035020 con 0x7efdbc0d2310 2024-04-04T06:09:11.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.672+0000 7efdbaa00640 1 -- 172.21.15.159:0/99359436 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda802d040 con 0x7efdbc0d2310 2024-04-04T06:09:11.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.672+0000 7efdbaa00640 1 -- 172.21.15.159:0/99359436 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8031a80 con 0x7efdbc0d2310 2024-04-04T06:09:11.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.673+0000 7efdc1800640 1 -- 172.21.15.159:0/99359436 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdbc0d2310 msgr2=0x7efdbc0ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:11.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.673+0000 7efdc1800640 1 --2- 172.21.15.159:0/99359436 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdbc0d2310 0x7efdbc0ceff0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0d3660 tx=0x7efda802b900 comp rx=0 tx=0).stop 2024-04-04T06:09:11.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.673+0000 7efdc1800640 1 -- 172.21.15.159:0/99359436 shutdown_connections 2024-04-04T06:09:11.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.673+0000 7efdc1800640 1 --2- 172.21.15.159:0/99359436 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdbc0d2310 0x7efdbc0ceff0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:11.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.673+0000 7efdc1800640 1 --2- 172.21.15.159:0/99359436 >> v2:172.21.15.178:3300/0 conn(0x7efdbc15e4a0 0x7efdbc0b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:11.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.673+0000 7efdc1800640 1 -- 172.21.15.159:0/99359436 >> 172.21.15.159:0/99359436 conn(0x7efdbc0676d0 msgr2=0x7efdbc067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:11.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.673+0000 7efdc1800640 1 -- 172.21.15.159:0/99359436 shutdown_connections 2024-04-04T06:09:11.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.673+0000 7efdc1800640 1 -- 172.21.15.159:0/99359436 wait complete. 2024-04-04T06:09:11.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.676+0000 7efdc1800640 1 Processor -- start 2024-04-04T06:09:11.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.676+0000 7efdc1800640 1 -- start start 2024-04-04T06:09:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.677+0000 7efdc1800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdbc0cf530 0x7efdbc0c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.677+0000 7efdc1800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdbc0d2310 0x7efdbc0cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.677+0000 7efdc1800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdbc15e4a0 0x7efdbc0cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.677+0000 7efdc1800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc0d3170 con 0x7efdbc0d2310 2024-04-04T06:09:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.677+0000 7efdc1800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc0d2e90 con 0x7efdbc0cf530 2024-04-04T06:09:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.677+0000 7efdc1800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efdbc0d3970 con 0x7efdbc15e4a0 2024-04-04T06:09:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.678+0000 7efdbb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdbc0d2310 0x7efdbc0cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.678+0000 7efdbb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdbc0d2310 0x7efdbc0cd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52282/0 (socket says 172.21.15.159:52282) 2024-04-04T06:09:11.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.678+0000 7efdbb400640 1 -- 172.21.15.159:0/3927806865 learned_addr learned my addr 172.21.15.159:0/3927806865 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:11.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.678+0000 7efdbb400640 1 -- 172.21.15.159:0/3927806865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdbc15e4a0 msgr2=0x7efdbc0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:11.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.678+0000 7efdc0e00640 1 --2- 172.21.15.159:0/3927806865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdbc15e4a0 0x7efdbc0cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:11.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.678+0000 7efdbbe00640 1 --2- 172.21.15.159:0/3927806865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdbc0cf530 0x7efdbc0c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:11.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.678+0000 7efdbb400640 1 --2- 172.21.15.159:0/3927806865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdbc15e4a0 0x7efdbc0cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:11.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.678+0000 7efdbb400640 1 -- 172.21.15.159:0/3927806865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdbc0cf530 msgr2=0x7efdbc0c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:11.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.678+0000 7efdbb400640 1 --2- 172.21.15.159:0/3927806865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdbc0cf530 0x7efdbc0c6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:11.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.678+0000 7efdbb400640 1 -- 172.21.15.159:0/3927806865 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdbc15f660 con 0x7efdbc0d2310 2024-04-04T06:09:11.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.678+0000 7efdc0e00640 1 --2- 172.21.15.159:0/3927806865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdbc15e4a0 0x7efdbc0cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:11.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.678+0000 7efdbbe00640 1 --2- 172.21.15.159:0/3927806865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdbc0cf530 0x7efdbc0c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:11.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.679+0000 7efdbb400640 1 --2- 172.21.15.159:0/3927806865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdbc0d2310 0x7efdbc0cd5e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7efda802b940 tx=0x7efda80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:11.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.679+0000 7efdb3e00640 1 -- 172.21.15.159:0/3927806865 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8040070 con 0x7efdbc0d2310 2024-04-04T06:09:11.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.679+0000 7efdb3e00640 1 -- 172.21.15.159:0/3927806865 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda8039c30 con 0x7efdbc0d2310 2024-04-04T06:09:11.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.679+0000 7efdc1800640 1 -- 172.21.15.159:0/3927806865 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdbc15f940 con 0x7efdbc0d2310 2024-04-04T06:09:11.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.679+0000 7efdb3e00640 1 -- 172.21.15.159:0/3927806865 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8035040 con 0x7efdbc0d2310 2024-04-04T06:09:11.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.679+0000 7efdc1800640 1 -- 172.21.15.159:0/3927806865 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdbc15ff00 con 0x7efdbc0d2310 2024-04-04T06:09:11.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.680+0000 7efdc1800640 1 -- 172.21.15.159:0/3927806865 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdbc075940 con 0x7efdbc0d2310 2024-04-04T06:09:11.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.681+0000 7efdb3e00640 1 -- 172.21.15.159:0/3927806865 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efda8038020 con 0x7efdbc0d2310 2024-04-04T06:09:11.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.682+0000 7efdb3e00640 1 --2- 172.21.15.159:0/3927806865 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efd9c06c8f0 0x7efd9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:11.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.682+0000 7efdb3e00640 1 -- 172.21.15.159:0/3927806865 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7efda80ad970 con 0x7efdbc0d2310 2024-04-04T06:09:11.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.683+0000 7efdbbe00640 1 --2- 172.21.15.159:0/3927806865 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efd9c06c8f0 0x7efd9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:11.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.684+0000 7efdbbe00640 1 --2- 172.21.15.159:0/3927806865 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efd9c06c8f0 0x7efd9c06edb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efdb4007eb0 tx=0x7efdb4007400 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:11.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.692+0000 7efdb3e00640 1 -- 172.21.15.159:0/3927806865 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7efda807bd90 con 0x7efdbc0d2310 2024-04-04T06:09:11.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.932+0000 7efdc1800640 1 -- 172.21.15.159:0/3927806865 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdbc0d4da0 con 0x7efdbc0d2310 2024-04-04T06:09:11.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.933+0000 7efdb3e00640 1 -- 172.21.15.159:0/3927806865 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5187 (secure 0 0 0) 0x7efda807b690 con 0x7efdbc0d2310 2024-04-04T06:09:11.926 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:09:11.926 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5191,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3507245876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3507245876},{"type":"v1","addr":"172.21.15.159:6835","nonce":3507245876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/967262871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":967262871},{"type":"v1","addr":"172.21.15.178:6835","nonce":967262871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2969351496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2969351496},{"type":"v1","addr":"172.21.15.178:6837","nonce":2969351496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:08:57.020444+0000","modified":"2024-04-04T06:09:00.117714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6039":{"gid":6039,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6834/962475038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":962475038},{"type":"v1","addr":"172.21.15.159:6834","nonce":962475038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6039,"qdb_cluster":[6039]},"id":4}]} 2024-04-04T06:09:11.927 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 33 2024-04-04T06:09:11.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.935+0000 7efdc1800640 1 -- 172.21.15.159:0/3927806865 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efd9c06c8f0 msgr2=0x7efd9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:11.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.935+0000 7efdc1800640 1 --2- 172.21.15.159:0/3927806865 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efd9c06c8f0 0x7efd9c06edb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efdb4007eb0 tx=0x7efdb4007400 comp rx=0 tx=0).stop 2024-04-04T06:09:11.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.935+0000 7efdc1800640 1 -- 172.21.15.159:0/3927806865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdbc0d2310 msgr2=0x7efdbc0cd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:11.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.935+0000 7efdc1800640 1 --2- 172.21.15.159:0/3927806865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdbc0d2310 0x7efdbc0cd5e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7efda802b940 tx=0x7efda80046c0 comp rx=0 tx=0).stop 2024-04-04T06:09:11.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.935+0000 7efdc1800640 1 -- 172.21.15.159:0/3927806865 shutdown_connections 2024-04-04T06:09:11.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.935+0000 7efdc1800640 1 --2- 172.21.15.159:0/3927806865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdbc15e4a0 0x7efdbc0cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:11.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.935+0000 7efdc1800640 1 --2- 172.21.15.159:0/3927806865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdbc0d2310 0x7efdbc0cd5e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:11.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.935+0000 7efdc1800640 1 --2- 172.21.15.159:0/3927806865 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efd9c06c8f0 0x7efd9c06edb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:11.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.935+0000 7efdc1800640 1 --2- 172.21.15.159:0/3927806865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdbc0cf530 0x7efdbc0c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:11.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.935+0000 7efdc1800640 1 -- 172.21.15.159:0/3927806865 >> 172.21.15.159:0/3927806865 conn(0x7efdbc0676d0 msgr2=0x7efdbc15c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:11.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.935+0000 7efdc1800640 1 -- 172.21.15.159:0/3927806865 shutdown_connections 2024-04-04T06:09:11.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:11.935+0000 7efdc1800640 1 -- 172.21.15.159:0/3927806865 wait complete. 2024-04-04T06:09:11.952 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-04T06:09:12.194 INFO:teuthology.orchestra.run.smithi159.stdout:{"message":"","return_code":0} 2024-04-04T06:09:12.195 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-04T06:09:12.195 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-04T06:09:12.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.298+0000 7f691343bf40 1 Processor -- start 2024-04-04T06:09:12.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.298+0000 7f691343bf40 1 -- start start 2024-04-04T06:09:12.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.299+0000 7f691343bf40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559912513150 0x559912513530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.299+0000 7f691343bf40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559912518c00 0x559912517160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.299+0000 7f691343bf40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559912485b70 0x559912515620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.299+0000 7f691343bf40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55991251a560 con 0x559912485b70 2024-04-04T06:09:12.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.299+0000 7f691343bf40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5599123e72c0 con 0x559912513150 2024-04-04T06:09:12.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.299+0000 7f691343bf40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559912519820 con 0x559912518c00 2024-04-04T06:09:12.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.299+0000 7f6910800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559912485b70 0x559912515620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.299+0000 7f6910800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559912485b70 0x559912515620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52300/0 (socket says 172.21.15.159:52300) 2024-04-04T06:09:12.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.299+0000 7f6910800640 1 -- 172.21.15.159:0/1706661020 learned_addr learned my addr 172.21.15.159:0/1706661020 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:12.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.299+0000 7f690fe00640 1 --2- 172.21.15.159:0/1706661020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559912513150 0x559912513530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.299+0000 7f6907e00640 1 --2- 172.21.15.159:0/1706661020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559912518c00 0x559912517160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.299+0000 7f6910800640 1 -- 172.21.15.159:0/1706661020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559912518c00 msgr2=0x559912517160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:12.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.299+0000 7f6910800640 1 --2- 172.21.15.159:0/1706661020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559912518c00 0x559912517160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.299+0000 7f6910800640 1 -- 172.21.15.159:0/1706661020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559912513150 msgr2=0x559912513530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:12.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.299+0000 7f6910800640 1 --2- 172.21.15.159:0/1706661020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559912513150 0x559912513530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.299+0000 7f6910800640 1 -- 172.21.15.159:0/1706661020 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559912518710 con 0x559912485b70 2024-04-04T06:09:12.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.299+0000 7f6907e00640 1 --2- 172.21.15.159:0/1706661020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559912518c00 0x559912517160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:12.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.300+0000 7f6910800640 1 --2- 172.21.15.159:0/1706661020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559912485b70 0x559912515620 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x55991246f450 tx=0x7f6908006910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8279bec803d3cf31 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:12.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.300+0000 7f690ea00640 1 -- 172.21.15.159:0/1706661020 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908003510 con 0x559912485b70 2024-04-04T06:09:12.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.300+0000 7f690ea00640 1 -- 172.21.15.159:0/1706661020 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69080036b0 con 0x559912485b70 2024-04-04T06:09:12.294 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.300+0000 7f690ea00640 1 -- 172.21.15.159:0/1706661020 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908003a80 con 0x559912485b70 2024-04-04T06:09:12.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.300+0000 7f691343bf40 1 -- 172.21.15.159:0/1706661020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559912485b70 msgr2=0x559912515620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.301+0000 7f691343bf40 1 --2- 172.21.15.159:0/1706661020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559912485b70 0x559912515620 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x55991246f450 tx=0x7f6908006910 comp rx=0 tx=0).stop 2024-04-04T06:09:12.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.301+0000 7f691343bf40 1 -- 172.21.15.159:0/1706661020 shutdown_connections 2024-04-04T06:09:12.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.301+0000 7f691343bf40 1 --2- 172.21.15.159:0/1706661020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559912485b70 0x559912515620 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.301+0000 7f691343bf40 1 --2- 172.21.15.159:0/1706661020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559912518c00 0x559912517160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.301+0000 7f691343bf40 1 --2- 172.21.15.159:0/1706661020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559912513150 0x559912513530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.301+0000 7f691343bf40 1 -- 172.21.15.159:0/1706661020 >> 172.21.15.159:0/1706661020 conn(0x55991243f8a0 msgr2=0x55991246c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:12.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.301+0000 7f691343bf40 1 -- 172.21.15.159:0/1706661020 shutdown_connections 2024-04-04T06:09:12.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.301+0000 7f691343bf40 1 -- 172.21.15.159:0/1706661020 wait complete. 2024-04-04T06:09:12.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.302+0000 7f691343bf40 1 Processor -- start 2024-04-04T06:09:12.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.302+0000 7f691343bf40 1 -- start start 2024-04-04T06:09:12.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.303+0000 7f691343bf40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559912485b70 0x559912513530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.303+0000 7f691343bf40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559912513150 0x55991246b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.303+0000 7f691343bf40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559912518c00 0x55991251a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.303+0000 7f691343bf40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559912513c30 con 0x559912485b70 2024-04-04T06:09:12.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.303+0000 7f691343bf40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5599125176a0 con 0x559912513150 2024-04-04T06:09:12.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.303+0000 7f691343bf40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x559912515b60 con 0x559912518c00 2024-04-04T06:09:12.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.303+0000 7f690fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559912485b70 0x559912513530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.303+0000 7f690fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559912485b70 0x559912513530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52304/0 (socket says 172.21.15.159:52304) 2024-04-04T06:09:12.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.303+0000 7f690fe00640 1 -- 172.21.15.159:0/3351629737 learned_addr learned my addr 172.21.15.159:0/3351629737 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:12.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.303+0000 7f6907e00640 1 --2- 172.21.15.159:0/3351629737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559912513150 0x55991246b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.303+0000 7f690fe00640 1 -- 172.21.15.159:0/3351629737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559912518c00 msgr2=0x55991251a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.303+0000 7f690fe00640 1 --2- 172.21.15.159:0/3351629737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x559912518c00 0x55991251a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.303+0000 7f690fe00640 1 -- 172.21.15.159:0/3351629737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559912513150 msgr2=0x55991246b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.303+0000 7f690fe00640 1 --2- 172.21.15.159:0/3351629737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559912513150 0x55991246b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.303+0000 7f690fe00640 1 -- 172.21.15.159:0/3351629737 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55991251c0a0 con 0x559912485b70 2024-04-04T06:09:12.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.304+0000 7f690fe00640 1 --2- 172.21.15.159:0/3351629737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559912485b70 0x559912513530 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6900009fa0 tx=0x7f69000074b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:12.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.304+0000 7f690d600640 1 -- 172.21.15.159:0/3351629737 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690001b070 con 0x559912485b70 2024-04-04T06:09:12.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.304+0000 7f690d600640 1 -- 172.21.15.159:0/3351629737 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6900006790 con 0x559912485b70 2024-04-04T06:09:12.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.304+0000 7f690d600640 1 -- 172.21.15.159:0/3351629737 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900006b60 con 0x559912485b70 2024-04-04T06:09:12.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.304+0000 7f691343bf40 1 -- 172.21.15.159:0/3351629737 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55991251c310 con 0x559912485b70 2024-04-04T06:09:12.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.304+0000 7f691343bf40 1 -- 172.21.15.159:0/3351629737 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55991251c650 con 0x559912485b70 2024-04-04T06:09:12.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.306+0000 7f690d600640 1 -- 172.21.15.159:0/3351629737 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f690001e070 con 0x559912485b70 2024-04-04T06:09:12.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.307+0000 7f690d600640 1 --2- 172.21.15.159:0/3351629737 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f68ec06b2d0 0x7f68ec06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.307+0000 7f690d600640 1 -- 172.21.15.159:0/3351629737 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f690008a350 con 0x559912485b70 2024-04-04T06:09:12.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.307+0000 7f691343bf40 1 --2- 172.21.15.159:0/3351629737 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x55991246b5c0 0x55991251b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.307+0000 7f691343bf40 1 -- 172.21.15.159:0/3351629737 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59) -- 0x5599125a54f0 con 0x55991246b5c0 2024-04-04T06:09:12.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.308+0000 7f6907e00640 1 --2- 172.21.15.159:0/3351629737 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f68ec06b2d0 0x7f68ec06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.308+0000 7f6910800640 1 --2- 172.21.15.159:0/3351629737 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x55991246b5c0 0x55991251b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.309+0000 7f6910800640 1 --2- 172.21.15.159:0/3351629737 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x55991246b5c0 0x55991251b130 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:12.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.309+0000 7f6907e00640 1 --2- 172.21.15.159:0/3351629737 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f68ec06b2d0 0x7f68ec06d790 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009810 tx=0x7f68f804c070 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:12.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.310+0000 7f6910800640 1 -- 172.21.15.159:0/3351629737 <== osd.5 v2:172.21.15.178:6802/63746765 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) ==== 164+0+94 (crc 0 0 0) 0x7f690801c040 con 0x55991246b5c0 2024-04-04T06:09:12.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.310+0000 7f691343bf40 1 -- 172.21.15.159:0/3351629737 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x55991246b5c0 msgr2=0x55991251b130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.310+0000 7f691343bf40 1 --2- 172.21.15.159:0/3351629737 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x55991246b5c0 0x55991251b130 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.310+0000 7f691343bf40 1 -- 172.21.15.159:0/3351629737 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f68ec06b2d0 msgr2=0x7f68ec06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.310+0000 7f691343bf40 1 --2- 172.21.15.159:0/3351629737 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f68ec06b2d0 0x7f68ec06d790 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009810 tx=0x7f68f804c070 comp rx=0 tx=0).stop 2024-04-04T06:09:12.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.310+0000 7f691343bf40 1 -- 172.21.15.159:0/3351629737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559912485b70 msgr2=0x559912513530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.310+0000 7f691343bf40 1 --2- 172.21.15.159:0/3351629737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559912485b70 0x559912513530 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6900009fa0 tx=0x7f69000074b0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.310+0000 7f6910800640 1 -- 172.21.15.159:0/3351629737 reap_dead start 2024-04-04T06:09:12.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.310+0000 7f691343bf40 1 -- 172.21.15.159:0/3351629737 shutdown_connections 2024-04-04T06:09:12.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.310+0000 7f691343bf40 1 -- 172.21.15.159:0/3351629737 >> 172.21.15.159:0/3351629737 conn(0x55991243f8a0 msgr2=0x55991246c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:12.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.310+0000 7f691343bf40 1 -- 172.21.15.159:0/3351629737 shutdown_connections 2024-04-04T06:09:12.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.311+0000 7f691343bf40 1 -- 172.21.15.159:0/3351629737 wait complete. 2024-04-04T06:09:12.307 INFO:teuthology.orchestra.run.smithi159.stdout:Xfiledir_test 2024-04-04T06:09:12.308 DEBUG:teuthology.orchestra.run.smithi159:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-04T06:09:12.397 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:09:12.397 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 1099511627777, 2024-04-04T06:09:12.397 INFO:teuthology.orchestra.run.smithi159.stdout: "ancestors": [ 2024-04-04T06:09:12.397 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:09:12.397 INFO:teuthology.orchestra.run.smithi159.stdout: "dirino": 1099511627776, 2024-04-04T06:09:12.397 INFO:teuthology.orchestra.run.smithi159.stdout: "dname": "file", 2024-04-04T06:09:12.397 INFO:teuthology.orchestra.run.smithi159.stdout: "version": 2 2024-04-04T06:09:12.397 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:09:12.397 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:09:12.397 INFO:teuthology.orchestra.run.smithi159.stdout: "dirino": 1, 2024-04-04T06:09:12.398 INFO:teuthology.orchestra.run.smithi159.stdout: "dname": "dir_test", 2024-04-04T06:09:12.398 INFO:teuthology.orchestra.run.smithi159.stdout: "version": 4 2024-04-04T06:09:12.398 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:09:12.398 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-04-04T06:09:12.398 INFO:teuthology.orchestra.run.smithi159.stdout: "pool": 9, 2024-04-04T06:09:12.398 INFO:teuthology.orchestra.run.smithi159.stdout: "old_pools": [] 2024-04-04T06:09:12.398 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:09:12.398 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:09:12.402 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-04-04T06:09:12.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.508+0000 7febb0892f40 1 Processor -- start 2024-04-04T06:09:12.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.508+0000 7febb0892f40 1 -- start start 2024-04-04T06:09:12.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.509+0000 7febb0892f40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56330b2c5960 0x56330b38d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.509+0000 7febb0892f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56330b2f0720 0x56330b3984a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.509+0000 7febb0892f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330b399910 0x56330b3989e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.509+0000 7febb0892f40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56330b30e210 con 0x56330b2c5960 2024-04-04T06:09:12.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.509+0000 7febb0892f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56330b26b2c0 con 0x56330b399910 2024-04-04T06:09:12.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.509+0000 7febb0892f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56330b30e4b0 con 0x56330b2f0720 2024-04-04T06:09:12.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.510+0000 7febada00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56330b2c5960 0x56330b38d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.510+0000 7febada00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56330b2c5960 0x56330b38d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52318/0 (socket says 172.21.15.159:52318) 2024-04-04T06:09:12.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.510+0000 7febada00640 1 -- 172.21.15.159:0/1114312918 learned_addr learned my addr 172.21.15.159:0/1114312918 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:12.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.510+0000 7febae400640 1 --2- 172.21.15.159:0/1114312918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330b399910 0x56330b3989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.510+0000 7febad000640 1 --2- 172.21.15.159:0/1114312918 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56330b2f0720 0x56330b3984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.510+0000 7febada00640 1 -- 172.21.15.159:0/1114312918 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56330b2f0720 msgr2=0x56330b3984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:12.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.510+0000 7febada00640 1 --2- 172.21.15.159:0/1114312918 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56330b2f0720 0x56330b3984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.510+0000 7febada00640 1 -- 172.21.15.159:0/1114312918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330b399910 msgr2=0x56330b3989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:12.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.510+0000 7febada00640 1 --2- 172.21.15.159:0/1114312918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330b399910 0x56330b3989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.510+0000 7febada00640 1 -- 172.21.15.159:0/1114312918 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56330b39b6c0 con 0x56330b2c5960 2024-04-04T06:09:12.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.510+0000 7febada00640 1 --2- 172.21.15.159:0/1114312918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56330b2c5960 0x56330b38d500 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7feba000b240 tx=0x7feba000b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3baaa6c812ad8ff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:12.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.511+0000 7feba7400640 1 -- 172.21.15.159:0/1114312918 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba0008070 con 0x56330b2c5960 2024-04-04T06:09:12.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.511+0000 7feba7400640 1 -- 172.21.15.159:0/1114312918 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feba000d500 con 0x56330b2c5960 2024-04-04T06:09:12.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.511+0000 7feba7400640 1 -- 172.21.15.159:0/1114312918 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba000d8b0 con 0x56330b2c5960 2024-04-04T06:09:12.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.511+0000 7febb0892f40 1 -- 172.21.15.159:0/1114312918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56330b2c5960 msgr2=0x56330b38d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.511+0000 7febb0892f40 1 --2- 172.21.15.159:0/1114312918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56330b2c5960 0x56330b38d500 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7feba000b240 tx=0x7feba000b700 comp rx=0 tx=0).stop 2024-04-04T06:09:12.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.511+0000 7febb0892f40 1 -- 172.21.15.159:0/1114312918 shutdown_connections 2024-04-04T06:09:12.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.511+0000 7febb0892f40 1 --2- 172.21.15.159:0/1114312918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330b399910 0x56330b3989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.511+0000 7febb0892f40 1 --2- 172.21.15.159:0/1114312918 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56330b2f0720 0x56330b3984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.511+0000 7febb0892f40 1 --2- 172.21.15.159:0/1114312918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56330b2c5960 0x56330b38d500 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.511+0000 7febb0892f40 1 -- 172.21.15.159:0/1114312918 >> 172.21.15.159:0/1114312918 conn(0x56330b2eebf0 msgr2=0x56330b2c4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:12.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.511+0000 7febb0892f40 1 -- 172.21.15.159:0/1114312918 shutdown_connections 2024-04-04T06:09:12.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.511+0000 7febb0892f40 1 -- 172.21.15.159:0/1114312918 wait complete. 2024-04-04T06:09:12.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.512+0000 7febb0892f40 1 Processor -- start 2024-04-04T06:09:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.512+0000 7febb0892f40 1 -- start start 2024-04-04T06:09:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.513+0000 7febb0892f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56330b2c5960 0x56330b38f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.513+0000 7febb0892f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330b2f0720 0x56330b398bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.513+0000 7febb0892f40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56330b399910 0x56330b2f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.513+0000 7febb0892f40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56330b3a3f20 con 0x56330b399910 2024-04-04T06:09:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.513+0000 7febb0892f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56330b39c100 con 0x56330b2f0720 2024-04-04T06:09:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.513+0000 7febb0892f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56330b38da40 con 0x56330b2c5960 2024-04-04T06:09:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.513+0000 7febae400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56330b399910 0x56330b2f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.513+0000 7febae400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56330b399910 0x56330b2f3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52324/0 (socket says 172.21.15.159:52324) 2024-04-04T06:09:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.513+0000 7febae400640 1 -- 172.21.15.159:0/3979293926 learned_addr learned my addr 172.21.15.159:0/3979293926 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.514+0000 7febad000640 1 --2- 172.21.15.159:0/3979293926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330b2f0720 0x56330b398bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.514+0000 7febada00640 1 --2- 172.21.15.159:0/3979293926 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56330b2c5960 0x56330b38f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.514+0000 7febae400640 1 -- 172.21.15.159:0/3979293926 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56330b2c5960 msgr2=0x56330b38f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.514+0000 7febae400640 1 --2- 172.21.15.159:0/3979293926 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56330b2c5960 0x56330b38f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.514+0000 7febae400640 1 -- 172.21.15.159:0/3979293926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330b2f0720 msgr2=0x56330b398bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.514+0000 7febae400640 1 --2- 172.21.15.159:0/3979293926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56330b2f0720 0x56330b398bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.514+0000 7febae400640 1 -- 172.21.15.159:0/3979293926 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56330b38aa60 con 0x56330b399910 2024-04-04T06:09:12.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.514+0000 7febae400640 1 --2- 172.21.15.159:0/3979293926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56330b399910 0x56330b2f3450 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7feba80081d0 tx=0x7feba8008690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:12.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.514+0000 7feba6000640 1 -- 172.21.15.159:0/3979293926 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8016020 con 0x56330b399910 2024-04-04T06:09:12.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.515+0000 7feba6000640 1 -- 172.21.15.159:0/3979293926 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feba8012630 con 0x56330b399910 2024-04-04T06:09:12.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.515+0000 7feba6000640 1 -- 172.21.15.159:0/3979293926 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba800e040 con 0x56330b399910 2024-04-04T06:09:12.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.515+0000 7febb0892f40 1 -- 172.21.15.159:0/3979293926 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56330b38ad40 con 0x56330b399910 2024-04-04T06:09:12.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.515+0000 7febb0892f40 1 -- 172.21.15.159:0/3979293926 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56330b38b300 con 0x56330b399910 2024-04-04T06:09:12.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.517+0000 7feba6000640 1 -- 172.21.15.159:0/3979293926 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feba8019020 con 0x56330b399910 2024-04-04T06:09:12.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.518+0000 7feba6000640 1 --2- 172.21.15.159:0/3979293926 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feb9006b300 0x7feb9006d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.518+0000 7feba6000640 1 -- 172.21.15.159:0/3979293926 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7feba808a610 con 0x56330b399910 2024-04-04T06:09:12.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.519+0000 7febb0892f40 1 --2- 172.21.15.159:0/3979293926 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x56330b389780 0x56330b395270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.519+0000 7febb0892f40 1 -- 172.21.15.159:0/3979293926 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59) -- 0x56330b3957b0 con 0x56330b389780 2024-04-04T06:09:12.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.519+0000 7febada00640 1 --2- 172.21.15.159:0/3979293926 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feb9006b300 0x7feb9006d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.520+0000 7febad000640 1 --2- 172.21.15.159:0/3979293926 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x56330b389780 0x56330b395270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.520+0000 7febada00640 1 --2- 172.21.15.159:0/3979293926 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feb9006b300 0x7feb9006d7c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7feba000b740 tx=0x7feba0008000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:12.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.521+0000 7febad000640 1 --2- 172.21.15.159:0/3979293926 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x56330b389780 0x56330b395270 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.524+0000 7febad000640 1 -- 172.21.15.159:0/3979293926 <== osd.5 v2:172.21.15.178:6802/63746765 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v59'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7feb9c052610 con 0x56330b389780 2024-04-04T06:09:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.524+0000 7febb0892f40 1 -- 172.21.15.159:0/3979293926 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x56330b389780 msgr2=0x56330b395270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.524+0000 7febb0892f40 1 --2- 172.21.15.159:0/3979293926 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x56330b389780 0x56330b395270 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.524+0000 7febb0892f40 1 -- 172.21.15.159:0/3979293926 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feb9006b300 msgr2=0x7feb9006d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.524+0000 7febb0892f40 1 --2- 172.21.15.159:0/3979293926 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feb9006b300 0x7feb9006d7c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7feba000b740 tx=0x7feba0008000 comp rx=0 tx=0).stop 2024-04-04T06:09:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.524+0000 7febb0892f40 1 -- 172.21.15.159:0/3979293926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56330b399910 msgr2=0x56330b2f3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.524+0000 7febb0892f40 1 --2- 172.21.15.159:0/3979293926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56330b399910 0x56330b2f3450 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7feba80081d0 tx=0x7feba8008690 comp rx=0 tx=0).stop 2024-04-04T06:09:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.524+0000 7febae400640 1 -- 172.21.15.159:0/3979293926 reap_dead start 2024-04-04T06:09:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.524+0000 7febb0892f40 1 -- 172.21.15.159:0/3979293926 shutdown_connections 2024-04-04T06:09:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.524+0000 7febb0892f40 1 -- 172.21.15.159:0/3979293926 >> 172.21.15.159:0/3979293926 conn(0x56330b2eebf0 msgr2=0x56330b2efda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.525+0000 7febb0892f40 1 -- 172.21.15.159:0/3979293926 shutdown_connections 2024-04-04T06:09:12.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.525+0000 7febb0892f40 1 -- 172.21.15.159:0/3979293926 wait complete. 2024-04-04T06:09:12.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:09:12.522 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-04-04T06:09:12.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.734+0000 7f7b7c400640 1 Processor -- start 2024-04-04T06:09:12.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.735+0000 7f7b7c400640 1 -- start start 2024-04-04T06:09:12.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.736+0000 7f7b7c400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7406fbc0 0x7f7b74070000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.736+0000 7f7b7c400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7b7415ead0 0x7f7b7415ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.736+0000 7f7b7c400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b741481d0 con 0x7f7b74147df0 2024-04-04T06:09:12.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.736+0000 7f7b7c400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f7b74070540 con 0x7f7b7415ead0 2024-04-04T06:09:12.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.736+0000 7f7b7c400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7b7415f8a0 con 0x7f7b7406fbc0 2024-04-04T06:09:12.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.736+0000 7f7b7a400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7b7415ead0 0x7f7b7415ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.736+0000 7f7b7a400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7b7415ead0 0x7f7b7415ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52328/0 (socket says 172.21.15.159:52328) 2024-04-04T06:09:12.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.736+0000 7f7b7a400640 1 -- 172.21.15.159:0/44540551 learned_addr learned my addr 172.21.15.159:0/44540551 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.736+0000 7f7b79a00640 1 --1- 172.21.15.159:0/44540551 >> v1:172.21.15.159:6789/0 conn(0x7f7b74147df0 0x7f7b7406d800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:34290/0 (socket says 172.21.15.159:34290) 2024-04-04T06:09:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.737+0000 7f7b79000640 1 --2- 172.21.15.159:0/44540551 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7406fbc0 0x7f7b74070000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.737+0000 7f7b7a400640 1 -- 172.21.15.159:0/44540551 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7406fbc0 msgr2=0x7f7b74070000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.737+0000 7f7b7a400640 1 --2- 172.21.15.159:0/44540551 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7406fbc0 0x7f7b74070000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.737+0000 7f7b7a400640 1 -- 172.21.15.159:0/44540551 >> v1:172.21.15.159:6789/0 conn(0x7f7b74147df0 legacy=0x7f7b7406d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.737+0000 7f7b7a400640 1 -- 172.21.15.159:0/44540551 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7406cd20 con 0x7f7b7415ead0 2024-04-04T06:09:12.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.737+0000 7f7b73e00640 1 -- 172.21.15.159:0/44540551 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 28619067 0 0) 0x7f7b741481d0 con 0x7f7b74147df0 2024-04-04T06:09:12.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.738+0000 7f7b7a400640 1 --2- 172.21.15.159:0/44540551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b7415ead0 0x7f7b7415ef40 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009e50 tx=0x7f7b6c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239e0dd42cc4231d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:12.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.738+0000 7f7b73e00640 1 -- 172.21.15.159:0/44540551 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c02c9f0 con 0x7f7b7415ead0 2024-04-04T06:09:12.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.738+0000 7f7b73e00640 1 -- 172.21.15.159:0/44540551 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b6c02cb90 con 0x7f7b7415ead0 2024-04-04T06:09:12.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.738+0000 7f7b73e00640 1 -- 172.21.15.159:0/44540551 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c0343f0 con 0x7f7b7415ead0 2024-04-04T06:09:12.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.739+0000 7f7b7c400640 1 -- 172.21.15.159:0/44540551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b7415ead0 msgr2=0x7f7b7415ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.739+0000 7f7b7c400640 1 --2- 172.21.15.159:0/44540551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b7415ead0 0x7f7b7415ef40 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009e50 tx=0x7f7b6c02be50 comp rx=0 tx=0).stop 2024-04-04T06:09:12.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.739+0000 7f7b7c400640 1 -- 172.21.15.159:0/44540551 shutdown_connections 2024-04-04T06:09:12.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.739+0000 7f7b7c400640 1 --2- 172.21.15.159:0/44540551 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7406fbc0 0x7f7b74070000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.739+0000 7f7b7c400640 1 --2- 172.21.15.159:0/44540551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b7415ead0 0x7f7b7415ef40 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.739+0000 7f7b7c400640 1 -- 172.21.15.159:0/44540551 >> 172.21.15.159:0/44540551 conn(0x7f7b74067460 msgr2=0x7f7b74067870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:12.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.739+0000 7f7b7c400640 1 -- 172.21.15.159:0/44540551 shutdown_connections 2024-04-04T06:09:12.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.739+0000 7f7b7c400640 1 -- 172.21.15.159:0/44540551 wait complete. 2024-04-04T06:09:12.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.741+0000 7f7b7c400640 1 Processor -- start 2024-04-04T06:09:12.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.741+0000 7f7b7c400640 1 -- start start 2024-04-04T06:09:12.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.742+0000 7f7b7c400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b7406fbc0 0x7f7b741f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.742+0000 7f7b7c400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b74147df0 0x7f7b74153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.742+0000 7f7b7c400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7415ead0 0x7f7b7414d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.742+0000 7f7b7c400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7415f480 con 0x7f7b7406fbc0 2024-04-04T06:09:12.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.742+0000 7f7b7c400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7406d2e0 con 0x7f7b74147df0 2024-04-04T06:09:12.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.742+0000 7f7b7c400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7b7415f600 con 0x7f7b7415ead0 2024-04-04T06:09:12.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.742+0000 7f7b79a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b7406fbc0 0x7f7b741f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.742+0000 7f7b79a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b7406fbc0 0x7f7b741f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52332/0 (socket says 172.21.15.159:52332) 2024-04-04T06:09:12.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.742+0000 7f7b79a00640 1 -- 172.21.15.159:0/3507901641 learned_addr learned my addr 172.21.15.159:0/3507901641 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:12.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.742+0000 7f7b79a00640 1 -- 172.21.15.159:0/3507901641 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7415ead0 msgr2=0x7f7b7414d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.742+0000 7f7b7a400640 1 --2- 172.21.15.159:0/3507901641 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7415ead0 0x7f7b7414d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.743+0000 7f7b79000640 1 --2- 172.21.15.159:0/3507901641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b74147df0 0x7f7b74153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.743+0000 7f7b79a00640 1 --2- 172.21.15.159:0/3507901641 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7415ead0 0x7f7b7414d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.743+0000 7f7b79a00640 1 -- 172.21.15.159:0/3507901641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b74147df0 msgr2=0x7f7b74153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.743+0000 7f7b79a00640 1 --2- 172.21.15.159:0/3507901641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b74147df0 0x7f7b74153710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.743+0000 7f7b79a00640 1 -- 172.21.15.159:0/3507901641 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b741f7c20 con 0x7f7b7406fbc0 2024-04-04T06:09:12.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.743+0000 7f7b79000640 1 --2- 172.21.15.159:0/3507901641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b74147df0 0x7f7b74153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:12.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.743+0000 7f7b79a00640 1 --2- 172.21.15.159:0/3507901641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b7406fbc0 0x7f7b741f7640 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800c4d0 tx=0x7f7b6800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:12.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.743+0000 7f7b71600640 1 -- 172.21.15.159:0/3507901641 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68018070 con 0x7f7b7406fbc0 2024-04-04T06:09:12.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.743+0000 7f7b71600640 1 -- 172.21.15.159:0/3507901641 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b68007500 con 0x7f7b7406fbc0 2024-04-04T06:09:12.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.743+0000 7f7b7c400640 1 -- 172.21.15.159:0/3507901641 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b741f7f00 con 0x7f7b7406fbc0 2024-04-04T06:09:12.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.743+0000 7f7b71600640 1 -- 172.21.15.159:0/3507901641 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68004a20 con 0x7f7b7406fbc0 2024-04-04T06:09:12.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.743+0000 7f7b7c400640 1 -- 172.21.15.159:0/3507901641 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b741f84c0 con 0x7f7b7406fbc0 2024-04-04T06:09:12.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.746+0000 7f7b71600640 1 -- 172.21.15.159:0/3507901641 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b6801b070 con 0x7f7b7406fbc0 2024-04-04T06:09:12.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.746+0000 7f7b71600640 1 --2- 172.21.15.159:0/3507901641 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7b5406c8f0 0x7f7b5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.747+0000 7f7b71600640 1 -- 172.21.15.159:0/3507901641 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f7b6808aec0 con 0x7f7b7406fbc0 2024-04-04T06:09:12.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.747+0000 7f7b7f1a9740 1 Processor -- start 2024-04-04T06:09:12.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.747+0000 7f7b79000640 1 --2- 172.21.15.159:0/3507901641 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7b5406c8f0 0x7f7b5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.747+0000 7f7b7f1a9740 1 -- start start 2024-04-04T06:09:12.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.748+0000 7f7b7f1a9740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec772a70 0x55fdec745a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.748+0000 7f7b7f1a9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec772e50 0x55fdec775800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.748+0000 7f7b7f1a9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec745f50 0x55fdec775d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.748+0000 7f7b7f1a9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fdec6af350 con 0x55fdec772e50 2024-04-04T06:09:12.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.748+0000 7f7b7f1a9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fdec657530 con 0x55fdec745f50 2024-04-04T06:09:12.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.748+0000 7f7b7f1a9740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55fdec6ab300 con 0x55fdec772a70 2024-04-04T06:09:12.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.748+0000 7f7b79000640 1 --2- 172.21.15.159:0/3507901641 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7b5406c8f0 0x7f7b5406edb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7b740cdcf0 tx=0x7f7b60008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:12.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.748+0000 7f7b79000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec772e50 0x55fdec775800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.748+0000 7f7b79000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec772e50 0x55fdec775800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52342/0 (socket says 172.21.15.159:52342) 2024-04-04T06:09:12.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.749+0000 7f7b79000640 1 -- 172.21.15.159:0/3609443390 learned_addr learned my addr 172.21.15.159:0/3609443390 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:12.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.749+0000 7f7b79a00640 1 --2- 172.21.15.159:0/3609443390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec772a70 0x55fdec745a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.749+0000 7f7b7a400640 1 --2- 172.21.15.159:0/3609443390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec745f50 0x55fdec775d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.749+0000 7f7b79000640 1 -- 172.21.15.159:0/3609443390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec772a70 msgr2=0x55fdec745a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:12.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.749+0000 7f7b79000640 1 --2- 172.21.15.159:0/3609443390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec772a70 0x55fdec745a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.749+0000 7f7b79000640 1 -- 172.21.15.159:0/3609443390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec745f50 msgr2=0x55fdec775d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:12.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.749+0000 7f7b79000640 1 --2- 172.21.15.159:0/3609443390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec745f50 0x55fdec775d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.749+0000 7f7b79000640 1 -- 172.21.15.159:0/3609443390 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdec66e1d0 con 0x55fdec772e50 2024-04-04T06:09:12.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.749+0000 7f7b79a00640 1 --2- 172.21.15.159:0/3609443390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec772a70 0x55fdec745a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:12.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.749+0000 7f7b7a400640 1 --2- 172.21.15.159:0/3609443390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec745f50 0x55fdec775d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:12.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.749+0000 7f7b79000640 1 --2- 172.21.15.159:0/3609443390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec772e50 0x55fdec775800 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7b60004380 tx=0x7f7b60011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1658934748f881be server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:12.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.749+0000 7f7b66000640 1 -- 172.21.15.159:0/3609443390 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6000f040 con 0x55fdec772e50 2024-04-04T06:09:12.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.749+0000 7f7b66000640 1 -- 172.21.15.159:0/3609443390 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b60003610 con 0x55fdec772e50 2024-04-04T06:09:12.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.749+0000 7f7b66000640 1 -- 172.21.15.159:0/3609443390 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b600039c0 con 0x55fdec772e50 2024-04-04T06:09:12.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.750+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/3609443390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec772e50 msgr2=0x55fdec775800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.750+0000 7f7b7f1a9740 1 --2- 172.21.15.159:0/3609443390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec772e50 0x55fdec775800 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7b60004380 tx=0x7f7b60011000 comp rx=0 tx=0).stop 2024-04-04T06:09:12.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.750+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/3609443390 shutdown_connections 2024-04-04T06:09:12.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.750+0000 7f7b7f1a9740 1 --2- 172.21.15.159:0/3609443390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec745f50 0x55fdec775d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.750+0000 7f7b7f1a9740 1 --2- 172.21.15.159:0/3609443390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec772e50 0x55fdec775800 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.750+0000 7f7b7f1a9740 1 --2- 172.21.15.159:0/3609443390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec772a70 0x55fdec745a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.750+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/3609443390 >> 172.21.15.159:0/3609443390 conn(0x55fdec747c80 msgr2=0x55fdec790800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:12.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.750+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/3609443390 shutdown_connections 2024-04-04T06:09:12.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.750+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/3609443390 wait complete. 2024-04-04T06:09:12.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.751+0000 7f7b7f1a9740 1 -- start start 2024-04-04T06:09:12.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.751+0000 7f7b7f1a9740 1 Processor -- start 2024-04-04T06:09:12.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.752+0000 7f7b7f1a9740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec772a70 0x55fdec75fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.752+0000 7f7b7f1a9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec772e50 0x55fdec76b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.752+0000 7f7b7f1a9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec76e640 0x55fdec7767f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.752+0000 7f7b7f1a9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fdec74a220 con 0x55fdec76e640 2024-04-04T06:09:12.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.752+0000 7f7b7f1a9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fdec66a8f0 con 0x55fdec772e50 2024-04-04T06:09:12.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.752+0000 7f7b7f1a9740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55fdec749610 con 0x55fdec772a70 2024-04-04T06:09:12.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.752+0000 7f7b7f1a9740 10 client.0 _ll_register_callbacks cb 0x55fdec66f0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:09:12.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.753+0000 7f7b7a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec76e640 0x55fdec7767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.753+0000 7f7b7a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec76e640 0x55fdec7767f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52350/0 (socket says 172.21.15.159:52350) 2024-04-04T06:09:12.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.753+0000 7f7b7a400640 1 -- 172.21.15.159:0/1242145581 learned_addr learned my addr 172.21.15.159:0/1242145581 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:12.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.753+0000 7f7b79a00640 1 --2- 172.21.15.159:0/1242145581 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec772a70 0x55fdec75fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.753+0000 7f7b79000640 1 --2- 172.21.15.159:0/1242145581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec772e50 0x55fdec76b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.753+0000 7f7b7a400640 1 -- 172.21.15.159:0/1242145581 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec772a70 msgr2=0x55fdec75fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:12.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.753+0000 7f7b7a400640 1 --2- 172.21.15.159:0/1242145581 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec772a70 0x55fdec75fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.753+0000 7f7b7a400640 1 -- 172.21.15.159:0/1242145581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec772e50 msgr2=0x55fdec76b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:12.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.753+0000 7f7b7a400640 1 --2- 172.21.15.159:0/1242145581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec772e50 0x55fdec76b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.753+0000 7f7b7a400640 1 -- 172.21.15.159:0/1242145581 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdec48f7b0 con 0x55fdec76e640 2024-04-04T06:09:12.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.753+0000 7f7b79a00640 1 --2- 172.21.15.159:0/1242145581 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec772a70 0x55fdec75fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:12.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.753+0000 7f7b7a400640 1 --2- 172.21.15.159:0/1242145581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec76e640 0x55fdec7767f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009f70 tx=0x7f7b6c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af80119bc40aac4e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:12.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.753+0000 7f7b5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:09:12.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.754+0000 7f7b5aa00640 1 -- 172.21.15.159:0/1242145581 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c009510 con 0x55fdec76e640 2024-04-04T06:09:12.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.754+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/1242145581 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b60003610 con 0x55fdec76e640 2024-04-04T06:09:12.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.754+0000 7f7b5aa00640 1 -- 172.21.15.159:0/1242145581 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f7b6c03e810 con 0x55fdec76e640 2024-04-04T06:09:12.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.754+0000 7f7b5aa00640 1 -- 172.21.15.159:0/1242145581 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7b6c03ea10 con 0x55fdec76e640 2024-04-04T06:09:12.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.754+0000 7f7b7f1a9740 10 client.6258 fetch_fsmap learned FSMap version 33 2024-04-04T06:09:12.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.754+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/1242145581 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55fdec4eff40 con 0x55fdec76e640 2024-04-04T06:09:12.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.754+0000 7f7b5aa00640 1 -- 172.21.15.159:0/1242145581 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b60003610 con 0x55fdec76e640 2024-04-04T06:09:12.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.754+0000 7f7b5aa00640 1 -- 172.21.15.159:0/1242145581 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7f7b6c03ebb0 con 0x55fdec76e640 2024-04-04T06:09:12.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.754+0000 7f7b7f1a9740 10 client.6258 fetch_fsmap finished waiting for FSMap version 33 2024-04-04T06:09:12.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.754+0000 7f7b7f1a9740 10 client.6258 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-04T06:09:12.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.754+0000 7f7b7f1a9740 20 client.6258 populate_metadata read hostname 'smithi159' 2024-04-04T06:09:12.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.755+0000 7f7b7f1a9740 1 --2- 172.21.15.159:0/1242145581 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55fdec7790e0 0x55fdec7737e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.755+0000 7f7b7f1a9740 4 client.6258 mds_command: new command op to 6039 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:09:12.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.755+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/1242145581 --> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdec6bc890 con 0x55fdec7790e0 2024-04-04T06:09:12.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.755+0000 7f7b79a00640 1 --2- 172.21.15.159:0/1242145581 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55fdec7790e0 0x55fdec7737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.755+0000 7f7b79a00640 1 --2- 172.21.15.159:0/1242145581 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55fdec7790e0 0x55fdec7737e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88ba4c36dadc430b server_cookie=d56e229a0d8e39ee in_seq=0 out_seq=0 2024-04-04T06:09:12.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.755+0000 7f7b5aa00640 10 client.6258 ms_handle_connect on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:12.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.756+0000 7f7b5aa00640 1 -- 172.21.15.159:0/1242145581 <== mds.0 v2:172.21.15.159:6833/962475038 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fdec6bc890 con 0x55fdec7790e0 2024-04-04T06:09:12.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.756+0000 7f7b5aa00640 10 client.6258 handle_command_reply: tid=0 2024-04-04T06:09:12.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.756+0000 7f7b7f1a9740 1 client.6258 shutdown 2024-04-04T06:09:12.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.756+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/1242145581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec76e640 msgr2=0x55fdec7767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.756+0000 7f7b7f1a9740 1 --2- 172.21.15.159:0/1242145581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec76e640 0x55fdec7767f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009f70 tx=0x7f7b6c004ab0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.756+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/1242145581 shutdown_connections 2024-04-04T06:09:12.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.756+0000 7f7b7f1a9740 1 --2- 172.21.15.159:0/1242145581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec76e640 0x55fdec7767f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.756+0000 7f7b7f1a9740 1 --2- 172.21.15.159:0/1242145581 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55fdec7790e0 0x55fdec7737e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.756+0000 7f7b7f1a9740 1 --2- 172.21.15.159:0/1242145581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec772e50 0x55fdec76b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.756+0000 7f7b7f1a9740 1 --2- 172.21.15.159:0/1242145581 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec772a70 0x55fdec75fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.756+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/1242145581 >> 172.21.15.159:0/1242145581 conn(0x55fdec745f50 msgr2=0x55fdec793d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:12.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.756+0000 7f7b5aa00640 0 client.6258 ms_handle_reset on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:12.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.757+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/1242145581 shutdown_connections 2024-04-04T06:09:12.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.757+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/1242145581 wait complete. 2024-04-04T06:09:12.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.757+0000 7f7b7f1a9740 20 client.6258 trim_cache size 0 max 16384 2024-04-04T06:09:12.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.790+0000 7f7b7f1a9740 1 Processor -- start 2024-04-04T06:09:12.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.790+0000 7f7b7f1a9740 1 -- start start 2024-04-04T06:09:12.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.792+0000 7f7b7f1a9740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec76e640 0x55fdec775630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.792+0000 7f7b7f1a9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec772a70 0x55fdec749660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.792+0000 7f7b7f1a9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec772e50 0x55fdec76b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.792+0000 7f7b7f1a9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7b742008d0 con 0x55fdec772e50 2024-04-04T06:09:12.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.792+0000 7f7b7f1a9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741ff9b0 con 0x55fdec772a70 2024-04-04T06:09:12.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.792+0000 7f7b7f1a9740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7b74142400 con 0x55fdec76e640 2024-04-04T06:09:12.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.792+0000 7f7b7a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec772e50 0x55fdec76b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.792+0000 7f7b7a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec772e50 0x55fdec76b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52366/0 (socket says 172.21.15.159:52366) 2024-04-04T06:09:12.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.792+0000 7f7b7a400640 1 -- 172.21.15.159:0/2457137575 learned_addr learned my addr 172.21.15.159:0/2457137575 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:12.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.792+0000 7f7b79a00640 1 --2- 172.21.15.159:0/2457137575 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec76e640 0x55fdec775630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.792+0000 7f7b7a400640 1 -- 172.21.15.159:0/2457137575 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec76e640 msgr2=0x55fdec775630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:12.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.792+0000 7f7b7a400640 1 --2- 172.21.15.159:0/2457137575 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec76e640 0x55fdec775630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.792+0000 7f7b79000640 1 --2- 172.21.15.159:0/2457137575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec772a70 0x55fdec749660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.792+0000 7f7b7a400640 1 -- 172.21.15.159:0/2457137575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec772a70 msgr2=0x55fdec749660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:12.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.792+0000 7f7b7a400640 1 --2- 172.21.15.159:0/2457137575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec772a70 0x55fdec749660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.792+0000 7f7b7a400640 1 -- 172.21.15.159:0/2457137575 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdec776b60 con 0x55fdec772e50 2024-04-04T06:09:12.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.793+0000 7f7b7a400640 1 --2- 172.21.15.159:0/2457137575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec772e50 0x55fdec76b600 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c002230 tx=0x7f7b6c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3d6ecd0f78c21a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:12.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.793+0000 7f7b65600640 1 -- 172.21.15.159:0/2457137575 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c059070 con 0x55fdec772e50 2024-04-04T06:09:12.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.793+0000 7f7b65600640 1 -- 172.21.15.159:0/2457137575 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b6c02cb90 con 0x55fdec772e50 2024-04-04T06:09:12.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.793+0000 7f7b65600640 1 -- 172.21.15.159:0/2457137575 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c009510 con 0x55fdec772e50 2024-04-04T06:09:12.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.794+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/2457137575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec772e50 msgr2=0x55fdec76b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.794+0000 7f7b7f1a9740 1 --2- 172.21.15.159:0/2457137575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec772e50 0x55fdec76b600 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c002230 tx=0x7f7b6c03f000 comp rx=0 tx=0).stop 2024-04-04T06:09:12.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.794+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/2457137575 shutdown_connections 2024-04-04T06:09:12.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.794+0000 7f7b7f1a9740 1 --2- 172.21.15.159:0/2457137575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec772e50 0x55fdec76b600 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.794+0000 7f7b7f1a9740 1 --2- 172.21.15.159:0/2457137575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec772a70 0x55fdec749660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.794+0000 7f7b7f1a9740 1 --2- 172.21.15.159:0/2457137575 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec76e640 0x55fdec775630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.794+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/2457137575 >> 172.21.15.159:0/2457137575 conn(0x55fdec745f50 msgr2=0x55fdec775e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:12.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.794+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/2457137575 shutdown_connections 2024-04-04T06:09:12.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.794+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/2457137575 wait complete. 2024-04-04T06:09:12.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.795+0000 7f7b7f1a9740 1 -- start start 2024-04-04T06:09:12.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.795+0000 7f7b7f1a9740 1 Processor -- start 2024-04-04T06:09:12.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.796+0000 7f7b7f1a9740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec76e640 0x55fdec800b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.796+0000 7f7b7f1a9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec772a70 0x55fdec80e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.796+0000 7f7b7f1a9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec772e50 0x55fdec7797e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.796+0000 7f7b7f1a9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fdec76a350 con 0x55fdec772a70 2024-04-04T06:09:12.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.796+0000 7f7b7f1a9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7b740b2ae0 con 0x55fdec772e50 2024-04-04T06:09:12.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.796+0000 7f7b7f1a9740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55fdec774e60 con 0x55fdec76e640 2024-04-04T06:09:12.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.796+0000 7f7b7f1a9740 10 client.0 _ll_register_callbacks cb 0x7f7b6c009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:09:12.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.796+0000 7f7b79000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec772a70 0x55fdec80e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.796+0000 7f7b79000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec772a70 0x55fdec80e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52380/0 (socket says 172.21.15.159:52380) 2024-04-04T06:09:12.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.796+0000 7f7b79000640 1 -- 172.21.15.159:0/1874228835 learned_addr learned my addr 172.21.15.159:0/1874228835 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:12.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.797+0000 7f7b79a00640 1 --2- 172.21.15.159:0/1874228835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec76e640 0x55fdec800b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.797+0000 7f7b7a400640 1 --2- 172.21.15.159:0/1874228835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec772e50 0x55fdec7797e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.797+0000 7f7b79000640 1 -- 172.21.15.159:0/1874228835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec76e640 msgr2=0x55fdec800b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:12.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.797+0000 7f7b79000640 1 --2- 172.21.15.159:0/1874228835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec76e640 0x55fdec800b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.797+0000 7f7b79000640 1 -- 172.21.15.159:0/1874228835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec772e50 msgr2=0x55fdec7797e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:12.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.797+0000 7f7b79000640 1 --2- 172.21.15.159:0/1874228835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec772e50 0x55fdec7797e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.797+0000 7f7b79000640 1 -- 172.21.15.159:0/1874228835 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdec3830a0 con 0x55fdec772a70 2024-04-04T06:09:12.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.797+0000 7f7b7a400640 1 --2- 172.21.15.159:0/1874228835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec772e50 0x55fdec7797e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:12.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.797+0000 7f7b79000640 1 --2- 172.21.15.159:0/1874228835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec772a70 0x55fdec80e050 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7b60004690 tx=0x7f7b60013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a3289832f0d404 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:12.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.797+0000 7f7b5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:09:12.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.797+0000 7f7b5aa00640 1 -- 172.21.15.159:0/1874228835 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60009e10 con 0x55fdec772a70 2024-04-04T06:09:12.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.797+0000 7f7b5aa00640 1 -- 172.21.15.159:0/1874228835 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f7b6002ead0 con 0x55fdec772a70 2024-04-04T06:09:12.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.797+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/1874228835 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fdec6b6be0 con 0x55fdec772a70 2024-04-04T06:09:12.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.798+0000 7f7b5aa00640 1 -- 172.21.15.159:0/1874228835 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7b6002ecd0 con 0x55fdec772a70 2024-04-04T06:09:12.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.798+0000 7f7b7f1a9740 10 client.6264 fetch_fsmap learned FSMap version 33 2024-04-04T06:09:12.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.798+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/1874228835 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55fdec76b6b0 con 0x55fdec772a70 2024-04-04T06:09:12.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.798+0000 7f7b5aa00640 1 -- 172.21.15.159:0/1874228835 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55fdec6b6be0 con 0x55fdec772a70 2024-04-04T06:09:12.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.798+0000 7f7b5aa00640 1 -- 172.21.15.159:0/1874228835 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7f7b60035050 con 0x55fdec772a70 2024-04-04T06:09:12.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.798+0000 7f7b7f1a9740 10 client.6264 fetch_fsmap finished waiting for FSMap version 33 2024-04-04T06:09:12.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.798+0000 7f7b7f1a9740 10 client.6264 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-04T06:09:12.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.798+0000 7f7b7f1a9740 20 client.6264 populate_metadata read hostname 'smithi159' 2024-04-04T06:09:12.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.798+0000 7f7b7f1a9740 1 --2- 172.21.15.159:0/1874228835 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55fdec769ce0 0x55fdec740f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:12.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.798+0000 7f7b7f1a9740 4 client.6264 mds_command: new command op to 6039 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-04-04T06:09:12.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.798+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/1874228835 --> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x55fdec6346a0 con 0x55fdec769ce0 2024-04-04T06:09:12.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.798+0000 7f7b79a00640 1 --2- 172.21.15.159:0/1874228835 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55fdec769ce0 0x55fdec740f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:12.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.799+0000 7f7b79a00640 1 --2- 172.21.15.159:0/1874228835 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55fdec769ce0 0x55fdec740f90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e7814e0c180bbb server_cookie=aa8c0a6575cfadf8 in_seq=0 out_seq=0 2024-04-04T06:09:12.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.799+0000 7f7b5aa00640 10 client.6264 ms_handle_connect on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:12.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.800+0000 7f7b5aa00640 1 -- 172.21.15.159:0/1874228835 <== mds.0 v2:172.21.15.159:6833/962475038 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fdec6346a0 con 0x55fdec769ce0 2024-04-04T06:09:12.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.800+0000 7f7b5aa00640 10 client.6264 handle_command_reply: tid=0 2024-04-04T06:09:12.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.800+0000 7f7b7f1a9740 1 client.6264 shutdown 2024-04-04T06:09:12.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.800+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/1874228835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec772a70 msgr2=0x55fdec80e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.800+0000 7f7b7f1a9740 1 --2- 172.21.15.159:0/1874228835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec772a70 0x55fdec80e050 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7b60004690 tx=0x7f7b60013000 comp rx=0 tx=0).stop 2024-04-04T06:09:12.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.800+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/1874228835 shutdown_connections 2024-04-04T06:09:12.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.800+0000 7f7b7f1a9740 1 --2- 172.21.15.159:0/1874228835 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55fdec769ce0 0x55fdec740f90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.800+0000 7f7b7f1a9740 1 --2- 172.21.15.159:0/1874228835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdec772e50 0x55fdec7797e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.800+0000 7f7b7f1a9740 1 --2- 172.21.15.159:0/1874228835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdec772a70 0x55fdec80e050 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.800+0000 7f7b7f1a9740 1 --2- 172.21.15.159:0/1874228835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fdec76e640 0x55fdec800b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.800+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/1874228835 >> 172.21.15.159:0/1874228835 conn(0x55fdec745f50 msgr2=0x55fdec749660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:12.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.800+0000 7f7b5aa00640 0 client.6264 ms_handle_reset on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:12.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.800+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/1874228835 shutdown_connections 2024-04-04T06:09:12.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.800+0000 7f7b7f1a9740 1 -- 172.21.15.159:0/1874228835 wait complete. 2024-04-04T06:09:12.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.801+0000 7f7b7f1a9740 20 client.6264 trim_cache size 0 max 16384 2024-04-04T06:09:12.797 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:09:12.797 INFO:teuthology.orchestra.run.smithi159.stdout: "return_code": 0, 2024-04-04T06:09:12.797 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_tag": "02674303-2a3b-41d2-b57f-434d308ce48d", 2024-04-04T06:09:12.797 INFO:teuthology.orchestra.run.smithi159.stdout: "mode": "asynchronous" 2024-04-04T06:09:12.797 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:09:12.797 INFO:tasks.ceph.mds.c.smithi159.stderr:2024-04-04T06:09:12.801+0000 7f1939600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5296=0-4194304@1} caps={5296=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5296 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x561a275bac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-04T06:09:12.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.801+0000 7f7b7c400640 1 -- 172.21.15.159:0/3507901641 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7b5406c8f0 msgr2=0x7f7b5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.801+0000 7f7b7c400640 1 --2- 172.21.15.159:0/3507901641 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7b5406c8f0 0x7f7b5406edb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7b740cdcf0 tx=0x7f7b60008040 comp rx=0 tx=0).stop 2024-04-04T06:09:12.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.801+0000 7f7b7c400640 1 -- 172.21.15.159:0/3507901641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b7406fbc0 msgr2=0x7f7b741f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:12.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.801+0000 7f7b7c400640 1 --2- 172.21.15.159:0/3507901641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b7406fbc0 0x7f7b741f7640 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800c4d0 tx=0x7f7b6800c990 comp rx=0 tx=0).stop 2024-04-04T06:09:12.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.801+0000 7f7b7c400640 1 -- 172.21.15.159:0/3507901641 shutdown_connections 2024-04-04T06:09:12.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.801+0000 7f7b7c400640 1 --2- 172.21.15.159:0/3507901641 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7415ead0 0x7f7b7414d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.801+0000 7f7b7c400640 1 --2- 172.21.15.159:0/3507901641 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7b5406c8f0 0x7f7b5406edb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.801+0000 7f7b7c400640 1 --2- 172.21.15.159:0/3507901641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b74147df0 0x7f7b74153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.801+0000 7f7b7c400640 1 --2- 172.21.15.159:0/3507901641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b7406fbc0 0x7f7b741f7640 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:12.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.801+0000 7f7b7c400640 1 -- 172.21.15.159:0/3507901641 >> 172.21.15.159:0/3507901641 conn(0x7f7b74067460 msgr2=0x7f7b74146fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:12.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.802+0000 7f7b7c400640 1 -- 172.21.15.159:0/3507901641 shutdown_connections 2024-04-04T06:09:12.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:12.802+0000 7f7b7c400640 1 -- 172.21.15.159:0/3507901641 wait complete. 2024-04-04T06:09:12.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:09:12.812 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-04T06:09:13.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.018+0000 7f9fea200640 1 Processor -- start 2024-04-04T06:09:13.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.018+0000 7f9fea200640 1 -- start start 2024-04-04T06:09:13.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.020+0000 7f9fea200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9fe4155930 0x7f9fe40d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.020+0000 7f9fea200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fe40b40b0 con 0x7f9fe4155550 2024-04-04T06:09:13.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.020+0000 7f9fea200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fe40b4390 con 0x7f9fe40d07c0 2024-04-04T06:09:13.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.020+0000 7f9fea200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f9fe40d0c10 con 0x7f9fe4155930 2024-04-04T06:09:13.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.020+0000 7f9fe3400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f9fe4155550 0x7f9fe415f800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:34292/0 (socket says 172.21.15.159:34292) 2024-04-04T06:09:13.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.020+0000 7f9fe3400640 1 -- 172.21.15.159:0/3426672877 learned_addr learned my addr 172.21.15.159:0/3426672877 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:09:13.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.020+0000 7f9fdbe00640 1 --2- 172.21.15.159:0/3426672877 >> v2:172.21.15.178:3300/0 conn(0x7f9fe4155930 0x7f9fe40d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:13.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.021+0000 7f9fe2a00640 1 -- 172.21.15.159:0/3426672877 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3114322698 0 0) 0x7f9fe40b40b0 con 0x7f9fe4155550 2024-04-04T06:09:13.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.021+0000 7f9fe2a00640 1 -- 172.21.15.159:0/3426672877 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fc8005730 con 0x7f9fe4155550 2024-04-04T06:09:13.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.021+0000 7f9fe2a00640 1 -- 172.21.15.159:0/3426672877 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3871863772 0 0) 0x7f9fc8005730 con 0x7f9fe4155550 2024-04-04T06:09:13.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.021+0000 7f9fe2a00640 1 -- 172.21.15.159:0/3426672877 >> v2:172.21.15.178:3300/0 conn(0x7f9fe4155930 msgr2=0x7f9fe40d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:13.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.021+0000 7f9fe2a00640 1 --2- 172.21.15.159:0/3426672877 >> v2:172.21.15.178:3300/0 conn(0x7f9fe4155930 0x7f9fe40d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.021+0000 7f9fe2a00640 1 -- 172.21.15.159:0/3426672877 >> v1:172.21.15.178:6789/0 conn(0x7f9fe40d07c0 legacy=0x7f9fe40aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:13.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.021+0000 7f9fe2a00640 1 -- 172.21.15.159:0/3426672877 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe40b4670 con 0x7f9fe4155550 2024-04-04T06:09:13.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.021+0000 7f9fe2a00640 1 -- 172.21.15.159:0/3426672877 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f9fd4001670 con 0x7f9fe4155550 2024-04-04T06:09:13.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.021+0000 7f9fe2a00640 1 -- 172.21.15.159:0/3426672877 >> v1:172.21.15.159:6789/0 conn(0x7f9fe4155550 legacy=0x7f9fe415f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:13.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.021+0000 7f9fe2a00640 1 --2- 172.21.15.159:0/3426672877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9fc8005bc0 0x7f9fc8005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.021+0000 7f9fe2a00640 1 --2- 172.21.15.159:0/3426672877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fc8006a10 0x7f9fc8006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.021+0000 7f9fe2a00640 1 --2- 172.21.15.159:0/3426672877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fc80073d0 0x7f9fc800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.021+0000 7f9fe2a00640 1 -- 172.21.15.159:0/3426672877 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe40d0c10 con 0x7f9fc80073d0 2024-04-04T06:09:13.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.021+0000 7f9fe2a00640 1 -- 172.21.15.159:0/3426672877 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc800e370 con 0x7f9fc8006a10 2024-04-04T06:09:13.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.021+0000 7f9fe2a00640 1 -- 172.21.15.159:0/3426672877 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc800e570 con 0x7f9fc8005bc0 2024-04-04T06:09:13.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.022+0000 7f9fe3e00640 1 --2- 172.21.15.159:0/3426672877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fc80073d0 0x7f9fc800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:13.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.022+0000 7f9fdbe00640 1 --2- 172.21.15.159:0/3426672877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fc8006a10 0x7f9fc8006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:13.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.022+0000 7f9fe3400640 1 --2- 172.21.15.159:0/3426672877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9fc8005bc0 0x7f9fc8005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:13.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.022+0000 7f9fe3e00640 1 -- 172.21.15.159:0/3426672877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9fc8005bc0 msgr2=0x7f9fc8005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:13.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.022+0000 7f9fe3e00640 1 --2- 172.21.15.159:0/3426672877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9fc8005bc0 0x7f9fc8005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.022+0000 7f9fe3e00640 1 -- 172.21.15.159:0/3426672877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fc8006a10 msgr2=0x7f9fc8006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:13.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.022+0000 7f9fe3e00640 1 --2- 172.21.15.159:0/3426672877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fc8006a10 0x7f9fc8006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.022+0000 7f9fe3e00640 1 -- 172.21.15.159:0/3426672877 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc8005730 con 0x7f9fc80073d0 2024-04-04T06:09:13.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.022+0000 7f9fe3400640 1 --2- 172.21.15.159:0/3426672877 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9fc8005bc0 0x7f9fc8005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:13.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.023+0000 7f9fe3e00640 1 -- 172.21.15.159:0/3426672877 reap_dead start 2024-04-04T06:09:13.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.023+0000 7f9fe3e00640 1 --2- 172.21.15.159:0/3426672877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fc80073d0 0x7f9fc800dbd0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc001e50 tx=0x7f9fc8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62bbd59da89d058c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:13.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.024+0000 7f9fe2a00640 1 -- 172.21.15.159:0/3426672877 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc001a70 con 0x7f9fc80073d0 2024-04-04T06:09:13.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.024+0000 7f9fe2a00640 1 -- 172.21.15.159:0/3426672877 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fdc003bb0 con 0x7f9fc80073d0 2024-04-04T06:09:13.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.024+0000 7f9fe2a00640 1 -- 172.21.15.159:0/3426672877 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc0050d0 con 0x7f9fc80073d0 2024-04-04T06:09:13.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.024+0000 7f9fea200640 1 -- 172.21.15.159:0/3426672877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fc80073d0 msgr2=0x7f9fc800dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:13.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.024+0000 7f9fea200640 1 --2- 172.21.15.159:0/3426672877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fc80073d0 0x7f9fc800dbd0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc001e50 tx=0x7f9fc8007a40 comp rx=0 tx=0).stop 2024-04-04T06:09:13.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.024+0000 7f9fea200640 1 -- 172.21.15.159:0/3426672877 shutdown_connections 2024-04-04T06:09:13.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.024+0000 7f9fea200640 1 --2- 172.21.15.159:0/3426672877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fc80073d0 0x7f9fc800dbd0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.024+0000 7f9fea200640 1 -- 172.21.15.159:0/3426672877 >> 172.21.15.159:0/3426672877 conn(0x7f9fe406b750 msgr2=0x7f9fe406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:13.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.024+0000 7f9fea200640 1 -- 172.21.15.159:0/3426672877 shutdown_connections 2024-04-04T06:09:13.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.024+0000 7f9fea200640 1 -- 172.21.15.159:0/3426672877 wait complete. 2024-04-04T06:09:13.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.026+0000 7f9fea200640 1 Processor -- start 2024-04-04T06:09:13.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.026+0000 7f9fea200640 1 -- start start 2024-04-04T06:09:13.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.027+0000 7f9fea200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9fc80073d0 0x7f9fe415a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.027+0000 7f9fea200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fe4159080 0x7f9fe40c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.027+0000 7f9fea200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fe409db20 0x7f9fe40afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.027+0000 7f9fea200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe415ba20 con 0x7f9fe4159080 2024-04-04T06:09:13.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.027+0000 7f9fea200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe40b4c30 con 0x7f9fe409db20 2024-04-04T06:09:13.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.027+0000 7f9fea200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe415bba0 con 0x7f9fc80073d0 2024-04-04T06:09:13.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.027+0000 7f9fdbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fe4159080 0x7f9fe40c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:13.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.027+0000 7f9fdbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fe4159080 0x7f9fe40c5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52406/0 (socket says 172.21.15.159:52406) 2024-04-04T06:09:13.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.027+0000 7f9fdbe00640 1 -- 172.21.15.159:0/1861498033 learned_addr learned my addr 172.21.15.159:0/1861498033 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:13.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.028+0000 7f9fe3400640 1 --2- 172.21.15.159:0/1861498033 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9fc80073d0 0x7f9fe415a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:13.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.028+0000 7f9fe3e00640 1 --2- 172.21.15.159:0/1861498033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fe409db20 0x7f9fe40afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:13.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.028+0000 7f9fdbe00640 1 -- 172.21.15.159:0/1861498033 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9fc80073d0 msgr2=0x7f9fe415a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:13.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.028+0000 7f9fdbe00640 1 --2- 172.21.15.159:0/1861498033 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9fc80073d0 0x7f9fe415a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.028+0000 7f9fdbe00640 1 -- 172.21.15.159:0/1861498033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fe409db20 msgr2=0x7f9fe40afd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:13.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.028+0000 7f9fdbe00640 1 --2- 172.21.15.159:0/1861498033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fe409db20 0x7f9fe40afd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.028+0000 7f9fdbe00640 1 -- 172.21.15.159:0/1861498033 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe40c6320 con 0x7f9fe4159080 2024-04-04T06:09:13.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.028+0000 7f9fe3e00640 1 --2- 172.21.15.159:0/1861498033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fe409db20 0x7f9fe40afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:13.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.028+0000 7f9fdbe00640 1 --2- 172.21.15.159:0/1861498033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fe4159080 0x7f9fe40c5de0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc011570 tx=0x7f9fcc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:13.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.029+0000 7f9fdb400640 1 -- 172.21.15.159:0/1861498033 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc015930 con 0x7f9fe4159080 2024-04-04T06:09:13.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.029+0000 7f9fdb400640 1 -- 172.21.15.159:0/1861498033 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fcc015ad0 con 0x7f9fe4159080 2024-04-04T06:09:13.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.029+0000 7f9fdb400640 1 -- 172.21.15.159:0/1861498033 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc00f460 con 0x7f9fe4159080 2024-04-04T06:09:13.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.029+0000 7f9fea200640 1 -- 172.21.15.159:0/1861498033 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe4159710 con 0x7f9fe4159080 2024-04-04T06:09:13.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.029+0000 7f9fea200640 1 -- 172.21.15.159:0/1861498033 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe4159cd0 con 0x7f9fe4159080 2024-04-04T06:09:13.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.031+0000 7f9fdb400640 1 -- 172.21.15.159:0/1861498033 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9fcc00f600 con 0x7f9fe4159080 2024-04-04T06:09:13.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.032+0000 7f9fdb400640 1 --2- 172.21.15.159:0/1861498033 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9fc006c920 0x7f9fc006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.032+0000 7f9fdb400640 1 -- 172.21.15.159:0/1861498033 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f9fcc08c350 con 0x7f9fe4159080 2024-04-04T06:09:13.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.033+0000 7f9fecfdd740 1 Processor -- start 2024-04-04T06:09:13.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.033+0000 7f9fe3400640 1 --2- 172.21.15.159:0/1861498033 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9fc006c920 0x7f9fc006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:13.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.033+0000 7f9fecfdd740 1 -- start start 2024-04-04T06:09:13.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.034+0000 7f9fecfdd740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bb35f2c000 0x55bb35f29cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.034+0000 7f9fecfdd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb35f2c3e0 0x55bb35ee3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.034+0000 7f9fecfdd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f0b940 0x55bb35ee3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.034+0000 7f9fecfdd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bb35e496d0 con 0x55bb35f0b940 2024-04-04T06:09:13.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.034+0000 7f9fecfdd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bb35df1770 con 0x55bb35f2c3e0 2024-04-04T06:09:13.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.034+0000 7f9fecfdd740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bb35e45680 con 0x55bb35f2c000 2024-04-04T06:09:13.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.034+0000 7f9fe3400640 1 --2- 172.21.15.159:0/1861498033 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9fc006c920 0x7f9fc006ede0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4000f60 tx=0x7f9fd4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:13.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.034+0000 7f9fe3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f0b940 0x55bb35ee3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:13.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.034+0000 7f9fe3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f0b940 0x55bb35ee3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52416/0 (socket says 172.21.15.159:52416) 2024-04-04T06:09:13.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.034+0000 7f9fe3e00640 1 -- 172.21.15.159:0/3082992625 learned_addr learned my addr 172.21.15.159:0/3082992625 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:13.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.034+0000 7f9fe3400640 1 --2- 172.21.15.159:0/3082992625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bb35f2c000 0x55bb35f29cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:13.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.034+0000 7f9fdbe00640 1 --2- 172.21.15.159:0/3082992625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb35f2c3e0 0x55bb35ee3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:13.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.035+0000 7f9fe3e00640 1 -- 172.21.15.159:0/3082992625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bb35f2c000 msgr2=0x55bb35f29cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:13.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.035+0000 7f9fe3e00640 1 --2- 172.21.15.159:0/3082992625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bb35f2c000 0x55bb35f29cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.035+0000 7f9fe3e00640 1 -- 172.21.15.159:0/3082992625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb35f2c3e0 msgr2=0x55bb35ee3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:13.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.035+0000 7f9fe3e00640 1 --2- 172.21.15.159:0/3082992625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb35f2c3e0 0x55bb35ee3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.035+0000 7f9fe3e00640 1 -- 172.21.15.159:0/3082992625 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb35c277d0 con 0x55bb35f0b940 2024-04-04T06:09:13.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.035+0000 7f9fe3e00640 1 --2- 172.21.15.159:0/3082992625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f0b940 0x55bb35ee3b70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc001f40 tx=0x7f9fdc030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e07854a276217e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:13.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.035+0000 7f9fd3400640 1 -- 172.21.15.159:0/3082992625 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc0039f0 con 0x55bb35f0b940 2024-04-04T06:09:13.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.035+0000 7f9fd3400640 1 -- 172.21.15.159:0/3082992625 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fdc006b50 con 0x55bb35f0b940 2024-04-04T06:09:13.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.035+0000 7f9fd3400640 1 -- 172.21.15.159:0/3082992625 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc0050d0 con 0x55bb35f0b940 2024-04-04T06:09:13.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.036+0000 7f9fecfdd740 1 -- 172.21.15.159:0/3082992625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f0b940 msgr2=0x55bb35ee3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:13.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.036+0000 7f9fecfdd740 1 --2- 172.21.15.159:0/3082992625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f0b940 0x55bb35ee3b70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc001f40 tx=0x7f9fdc030000 comp rx=0 tx=0).stop 2024-04-04T06:09:13.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.036+0000 7f9fecfdd740 1 -- 172.21.15.159:0/3082992625 shutdown_connections 2024-04-04T06:09:13.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.036+0000 7f9fecfdd740 1 --2- 172.21.15.159:0/3082992625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f0b940 0x55bb35ee3b70 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.036+0000 7f9fecfdd740 1 --2- 172.21.15.159:0/3082992625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb35f2c3e0 0x55bb35ee3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.036+0000 7f9fecfdd740 1 --2- 172.21.15.159:0/3082992625 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bb35f2c000 0x55bb35f29cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.036+0000 7f9fecfdd740 1 -- 172.21.15.159:0/3082992625 >> 172.21.15.159:0/3082992625 conn(0x55bb35ee0790 msgr2=0x55bb35f29310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:13.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.036+0000 7f9fecfdd740 1 -- 172.21.15.159:0/3082992625 shutdown_connections 2024-04-04T06:09:13.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.036+0000 7f9fecfdd740 1 -- 172.21.15.159:0/3082992625 wait complete. 2024-04-04T06:09:13.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.037+0000 7f9fecfdd740 1 -- start start 2024-04-04T06:09:13.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.037+0000 7f9fecfdd740 1 Processor -- start 2024-04-04T06:09:13.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.038+0000 7f9fecfdd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f2c000 0x55bb35f24350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.038+0000 7f9fecfdd740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bb35f2c3e0 0x55bb35f07af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.038+0000 7f9fecfdd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb35f12520 0x55bb35f0c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.038+0000 7f9fecfdd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bb35ededc0 con 0x55bb35f2c000 2024-04-04T06:09:13.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.038+0000 7f9fecfdd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bb35e04aa0 con 0x55bb35f12520 2024-04-04T06:09:13.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.038+0000 7f9fecfdd740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bb35ee2bc0 con 0x55bb35f2c3e0 2024-04-04T06:09:13.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.038+0000 7f9fecfdd740 10 client.0 _ll_register_callbacks cb 0x55bb35e04490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:09:13.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.038+0000 7f9fe3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f2c000 0x55bb35f24350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:13.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.038+0000 7f9fe3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f2c000 0x55bb35f24350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52428/0 (socket says 172.21.15.159:52428) 2024-04-04T06:09:13.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.038+0000 7f9fe3400640 1 -- 172.21.15.159:0/3739041582 learned_addr learned my addr 172.21.15.159:0/3739041582 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:13.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.039+0000 7f9fe3e00640 1 --2- 172.21.15.159:0/3739041582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb35f12520 0x55bb35f0c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:13.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.039+0000 7f9fdbe00640 1 --2- 172.21.15.159:0/3739041582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bb35f2c3e0 0x55bb35f07af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:13.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.039+0000 7f9fe3400640 1 -- 172.21.15.159:0/3739041582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bb35f2c3e0 msgr2=0x55bb35f07af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:13.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.039+0000 7f9fe3400640 1 --2- 172.21.15.159:0/3739041582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bb35f2c3e0 0x55bb35f07af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.039+0000 7f9fe3400640 1 -- 172.21.15.159:0/3739041582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb35f12520 msgr2=0x55bb35f0c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:13.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.039+0000 7f9fe3400640 1 --2- 172.21.15.159:0/3739041582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb35f12520 0x55bb35f0c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.039+0000 7f9fe3400640 1 -- 172.21.15.159:0/3739041582 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb35b5d3f0 con 0x55bb35f2c000 2024-04-04T06:09:13.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.039+0000 7f9fe3e00640 1 --2- 172.21.15.159:0/3739041582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb35f12520 0x55bb35f0c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:13.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.039+0000 7f9fe3400640 1 --2- 172.21.15.159:0/3739041582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f2c000 0x55bb35f24350 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4005ad0 tx=0x7f9fe40b4560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61aa7ede7a36343e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:13.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.039+0000 7f9fc7e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:09:13.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.039+0000 7f9fc7e00640 1 -- 172.21.15.159:0/3739041582 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4027060 con 0x55bb35f2c000 2024-04-04T06:09:13.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.040+0000 7f9fc7e00640 1 -- 172.21.15.159:0/3739041582 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f9fd4022a90 con 0x55bb35f2c000 2024-04-04T06:09:13.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.040+0000 7f9fecfdd740 1 -- 172.21.15.159:0/3739041582 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fdc006b50 con 0x55bb35f2c000 2024-04-04T06:09:13.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.040+0000 7f9fc7e00640 1 -- 172.21.15.159:0/3739041582 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9fd4022460 con 0x55bb35f2c000 2024-04-04T06:09:13.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.040+0000 7f9fecfdd740 10 client.6276 fetch_fsmap learned FSMap version 33 2024-04-04T06:09:13.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.040+0000 7f9fecfdd740 1 -- 172.21.15.159:0/3739041582 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55bb35b1d0f0 con 0x55bb35f2c000 2024-04-04T06:09:13.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.040+0000 7f9fc7e00640 1 -- 172.21.15.159:0/3739041582 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fdc006b50 con 0x55bb35f2c000 2024-04-04T06:09:13.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.040+0000 7f9fc7e00640 1 -- 172.21.15.159:0/3739041582 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7f9fd4013480 con 0x55bb35f2c000 2024-04-04T06:09:13.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.040+0000 7f9fecfdd740 10 client.6276 fetch_fsmap finished waiting for FSMap version 33 2024-04-04T06:09:13.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.040+0000 7f9fecfdd740 10 client.6276 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-04T06:09:13.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.040+0000 7f9fecfdd740 20 client.6276 populate_metadata read hostname 'smithi159' 2024-04-04T06:09:13.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.040+0000 7f9fecfdd740 1 --2- 172.21.15.159:0/3739041582 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55bb35f3f500 0x55bb35f3f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.040+0000 7f9fecfdd740 4 client.6276 mds_command: new command op to 6039 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:09:13.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.040+0000 7f9fecfdd740 1 -- 172.21.15.159:0/3739041582 --> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb35e56c10 con 0x55bb35f3f500 2024-04-04T06:09:13.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.041+0000 7f9fdbe00640 1 --2- 172.21.15.159:0/3739041582 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55bb35f3f500 0x55bb35f3f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:13.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.041+0000 7f9fdbe00640 1 --2- 172.21.15.159:0/3739041582 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55bb35f3f500 0x55bb35f3f930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae84cb6166786ee3 server_cookie=e9fbf7ba929dcd8e in_seq=0 out_seq=0 2024-04-04T06:09:13.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.041+0000 7f9fc7e00640 10 client.6276 ms_handle_connect on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:13.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.042+0000 7f9fc7e00640 1 -- 172.21.15.159:0/3739041582 <== mds.0 v2:172.21.15.159:6833/962475038 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb35e56c10 con 0x55bb35f3f500 2024-04-04T06:09:13.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.042+0000 7f9fc7e00640 10 client.6276 handle_command_reply: tid=0 2024-04-04T06:09:13.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.042+0000 7f9fecfdd740 1 client.6276 shutdown 2024-04-04T06:09:13.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.042+0000 7f9fecfdd740 1 -- 172.21.15.159:0/3739041582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f2c000 msgr2=0x55bb35f24350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:13.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.042+0000 7f9fecfdd740 1 --2- 172.21.15.159:0/3739041582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f2c000 0x55bb35f24350 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4005ad0 tx=0x7f9fe40b4560 comp rx=0 tx=0).stop 2024-04-04T06:09:13.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.042+0000 7f9fecfdd740 1 -- 172.21.15.159:0/3739041582 shutdown_connections 2024-04-04T06:09:13.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.042+0000 7f9fecfdd740 1 --2- 172.21.15.159:0/3739041582 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55bb35f3f500 0x55bb35f3f930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.042+0000 7f9fecfdd740 1 --2- 172.21.15.159:0/3739041582 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb35f12520 0x55bb35f0c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.042+0000 7f9fecfdd740 1 --2- 172.21.15.159:0/3739041582 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bb35f2c3e0 0x55bb35f07af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.042+0000 7f9fecfdd740 1 --2- 172.21.15.159:0/3739041582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f2c000 0x55bb35f24350 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.042+0000 7f9fc7e00640 0 client.6276 ms_handle_reset on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:13.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.042+0000 7f9fecfdd740 1 -- 172.21.15.159:0/3739041582 >> 172.21.15.159:0/3739041582 conn(0x55bb35f0b940 msgr2=0x55bb35ee30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:13.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.042+0000 7f9fecfdd740 1 -- 172.21.15.159:0/3739041582 shutdown_connections 2024-04-04T06:09:13.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.042+0000 7f9fecfdd740 1 -- 172.21.15.159:0/3739041582 wait complete. 2024-04-04T06:09:13.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.043+0000 7f9fecfdd740 20 client.6276 trim_cache size 0 max 16384 2024-04-04T06:09:13.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.062+0000 7f9fecfdd740 1 Processor -- start 2024-04-04T06:09:13.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.062+0000 7f9fecfdd740 1 -- start start 2024-04-04T06:09:13.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.063+0000 7f9fecfdd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f12520 0x55bb35f0c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.063+0000 7f9fecfdd740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bb35f2c000 0x55bb35f9e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.063+0000 7f9fecfdd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb35f2c3e0 0x55bb35f9eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.063+0000 7f9fecfdd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe41fcda0 con 0x55bb35f12520 2024-04-04T06:09:13.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.063+0000 7f9fecfdd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe41fbe30 con 0x55bb35f2c3e0 2024-04-04T06:09:13.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.063+0000 7f9fecfdd740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe41f1070 con 0x55bb35f2c000 2024-04-04T06:09:13.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.063+0000 7f9fe3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f12520 0x55bb35f0c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:13.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.063+0000 7f9fe3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f12520 0x55bb35f0c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52430/0 (socket says 172.21.15.159:52430) 2024-04-04T06:09:13.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.063+0000 7f9fe3400640 1 -- 172.21.15.159:0/4200645963 learned_addr learned my addr 172.21.15.159:0/4200645963 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:13.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.063+0000 7f9fe3400640 1 -- 172.21.15.159:0/4200645963 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bb35f2c000 msgr2=0x55bb35f9e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:13.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.063+0000 7f9fe3400640 1 --2- 172.21.15.159:0/4200645963 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bb35f2c000 0x55bb35f9e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.063+0000 7f9fe3400640 1 -- 172.21.15.159:0/4200645963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb35f2c3e0 msgr2=0x55bb35f9eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:13.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.063+0000 7f9fe3400640 1 --2- 172.21.15.159:0/4200645963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb35f2c3e0 0x55bb35f9eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.063+0000 7f9fe3400640 1 -- 172.21.15.159:0/4200645963 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb35ed9cf0 con 0x55bb35f12520 2024-04-04T06:09:13.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.064+0000 7f9fe3400640 1 --2- 172.21.15.159:0/4200645963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f12520 0x55bb35f0c340 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009010 tx=0x7f9fd401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b1555919f18a61 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:13.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.064+0000 7f9fd2a00640 1 -- 172.21.15.159:0/4200645963 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4031030 con 0x55bb35f12520 2024-04-04T06:09:13.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.064+0000 7f9fd2a00640 1 -- 172.21.15.159:0/4200645963 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fd4013480 con 0x55bb35f12520 2024-04-04T06:09:13.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.064+0000 7f9fd2a00640 1 -- 172.21.15.159:0/4200645963 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4013650 con 0x55bb35f12520 2024-04-04T06:09:13.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.064+0000 7f9fecfdd740 1 -- 172.21.15.159:0/4200645963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f12520 msgr2=0x55bb35f0c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:13.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.064+0000 7f9fecfdd740 1 --2- 172.21.15.159:0/4200645963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f12520 0x55bb35f0c340 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009010 tx=0x7f9fd401f000 comp rx=0 tx=0).stop 2024-04-04T06:09:13.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.065+0000 7f9fecfdd740 1 -- 172.21.15.159:0/4200645963 shutdown_connections 2024-04-04T06:09:13.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.065+0000 7f9fecfdd740 1 --2- 172.21.15.159:0/4200645963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb35f2c3e0 0x55bb35f9eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.065+0000 7f9fecfdd740 1 --2- 172.21.15.159:0/4200645963 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bb35f2c000 0x55bb35f9e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.065+0000 7f9fecfdd740 1 --2- 172.21.15.159:0/4200645963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f12520 0x55bb35f0c340 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.065+0000 7f9fecfdd740 1 -- 172.21.15.159:0/4200645963 >> 172.21.15.159:0/4200645963 conn(0x55bb35f0b940 msgr2=0x55bb35f2c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:13.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.065+0000 7f9fecfdd740 1 -- 172.21.15.159:0/4200645963 shutdown_connections 2024-04-04T06:09:13.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.065+0000 7f9fecfdd740 1 -- 172.21.15.159:0/4200645963 wait complete. 2024-04-04T06:09:13.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.066+0000 7f9fecfdd740 1 -- start start 2024-04-04T06:09:13.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.066+0000 7f9fecfdd740 1 Processor -- start 2024-04-04T06:09:13.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.068+0000 7f9fecfdd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb35f12520 0x55bb35f2b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.068+0000 7f9fecfdd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f2c000 0x55bb35f9b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.068+0000 7f9fecfdd740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bb35f2c3e0 0x55bb35f9bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.068+0000 7f9fecfdd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe4157c20 con 0x55bb35f2c000 2024-04-04T06:09:13.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.068+0000 7f9fecfdd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bb35eda090 con 0x55bb35f12520 2024-04-04T06:09:13.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.068+0000 7f9fecfdd740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bb35f51090 con 0x55bb35f2c3e0 2024-04-04T06:09:13.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.068+0000 7f9fecfdd740 10 client.0 _ll_register_callbacks cb 0x7f9fd400ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:09:13.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.068+0000 7f9fdbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f2c000 0x55bb35f9b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:13.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.068+0000 7f9fdbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f2c000 0x55bb35f9b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52444/0 (socket says 172.21.15.159:52444) 2024-04-04T06:09:13.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.068+0000 7f9fdbe00640 1 -- 172.21.15.159:0/978800489 learned_addr learned my addr 172.21.15.159:0/978800489 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:13.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.068+0000 7f9fe3400640 1 --2- 172.21.15.159:0/978800489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb35f12520 0x55bb35f2b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:13.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.068+0000 7f9fe3e00640 1 --2- 172.21.15.159:0/978800489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bb35f2c3e0 0x55bb35f9bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:13.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.068+0000 7f9fdbe00640 1 -- 172.21.15.159:0/978800489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bb35f2c3e0 msgr2=0x55bb35f9bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:13.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.068+0000 7f9fdbe00640 1 --2- 172.21.15.159:0/978800489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bb35f2c3e0 0x55bb35f9bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.068+0000 7f9fdbe00640 1 -- 172.21.15.159:0/978800489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb35f12520 msgr2=0x55bb35f2b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:13.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.068+0000 7f9fdbe00640 1 --2- 172.21.15.159:0/978800489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb35f12520 0x55bb35f2b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.068+0000 7f9fdbe00640 1 -- 172.21.15.159:0/978800489 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb35f2b640 con 0x55bb35f2c000 2024-04-04T06:09:13.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.069+0000 7f9fdbe00640 1 --2- 172.21.15.159:0/978800489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f2c000 0x55bb35f9b7d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc056570 tx=0x7f9fcc05e4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718abb4b4ff38309 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:13.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.069+0000 7f9fc7e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:09:13.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.069+0000 7f9fc7e00640 1 -- 172.21.15.159:0/978800489 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc022040 con 0x55bb35f2c000 2024-04-04T06:09:13.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.069+0000 7f9fc7e00640 1 -- 172.21.15.159:0/978800489 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f9fcc056780 con 0x55bb35f2c000 2024-04-04T06:09:13.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.069+0000 7f9fecfdd740 1 -- 172.21.15.159:0/978800489 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fd4013480 con 0x55bb35f2c000 2024-04-04T06:09:13.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.069+0000 7f9fc7e00640 1 -- 172.21.15.159:0/978800489 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9fcc05e120 con 0x55bb35f2c000 2024-04-04T06:09:13.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.069+0000 7f9fecfdd740 10 client.6282 fetch_fsmap learned FSMap version 33 2024-04-04T06:09:13.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.069+0000 7f9fecfdd740 1 -- 172.21.15.159:0/978800489 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55bb35f9c950 con 0x55bb35f2c000 2024-04-04T06:09:13.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.069+0000 7f9fc7e00640 1 -- 172.21.15.159:0/978800489 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fd4013480 con 0x55bb35f2c000 2024-04-04T06:09:13.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.069+0000 7f9fc7e00640 1 -- 172.21.15.159:0/978800489 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7f9fcc062860 con 0x55bb35f2c000 2024-04-04T06:09:13.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.070+0000 7f9fecfdd740 10 client.6282 fetch_fsmap finished waiting for FSMap version 33 2024-04-04T06:09:13.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.070+0000 7f9fecfdd740 10 client.6282 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-04T06:09:13.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.070+0000 7f9fecfdd740 20 client.6282 populate_metadata read hostname 'smithi159' 2024-04-04T06:09:13.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.070+0000 7f9fecfdd740 1 --2- 172.21.15.159:0/978800489 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55bb35f9c320 0x55bb35ee2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:13.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.070+0000 7f9fecfdd740 4 client.6282 mds_command: new command op to 6039 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-04T06:09:13.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.070+0000 7f9fecfdd740 1 -- 172.21.15.159:0/978800489 --> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bb35e50f60 con 0x55bb35f9c320 2024-04-04T06:09:13.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.070+0000 7f9fe3400640 1 --2- 172.21.15.159:0/978800489 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55bb35f9c320 0x55bb35ee2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:13.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.071+0000 7f9fc7e00640 10 client.6282 ms_handle_connect on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:13.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.071+0000 7f9fe3400640 1 --2- 172.21.15.159:0/978800489 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55bb35f9c320 0x55bb35ee2af0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=790e6a225e7c19e4 server_cookie=6c16bdc161ae958d in_seq=0 out_seq=0 2024-04-04T06:09:13.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.071+0000 7f9fc7e00640 1 -- 172.21.15.159:0/978800489 <== mds.0 v2:172.21.15.159:6833/962475038 1 ==== command_reply(tid 0: 0 ) ==== 8+0+253 (crc 0 0 0) 0x55bb35e50f60 con 0x55bb35f9c320 2024-04-04T06:09:13.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.071+0000 7f9fc7e00640 10 client.6282 handle_command_reply: tid=0 2024-04-04T06:09:13.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.071+0000 7f9fecfdd740 1 client.6282 shutdown 2024-04-04T06:09:13.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.071+0000 7f9fecfdd740 1 -- 172.21.15.159:0/978800489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f2c000 msgr2=0x55bb35f9b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:13.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.071+0000 7f9fecfdd740 1 --2- 172.21.15.159:0/978800489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f2c000 0x55bb35f9b7d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc056570 tx=0x7f9fcc05e4c0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.071+0000 7f9fecfdd740 1 -- 172.21.15.159:0/978800489 shutdown_connections 2024-04-04T06:09:13.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.071+0000 7f9fecfdd740 1 --2- 172.21.15.159:0/978800489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bb35f2c3e0 0x55bb35f9bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.071+0000 7f9fecfdd740 1 --2- 172.21.15.159:0/978800489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb35f2c000 0x55bb35f9b7d0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.071+0000 7f9fecfdd740 1 --2- 172.21.15.159:0/978800489 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55bb35f9c320 0x55bb35ee2af0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.071+0000 7f9fecfdd740 1 --2- 172.21.15.159:0/978800489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb35f12520 0x55bb35f2b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.071+0000 7f9fc7e00640 0 client.6282 ms_handle_reset on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:13.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.071+0000 7f9fecfdd740 1 -- 172.21.15.159:0/978800489 >> 172.21.15.159:0/978800489 conn(0x55bb35f0b940 msgr2=0x55bb35f23ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:13.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.071+0000 7f9fecfdd740 1 -- 172.21.15.159:0/978800489 shutdown_connections 2024-04-04T06:09:13.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.071+0000 7f9fecfdd740 1 -- 172.21.15.159:0/978800489 wait complete. 2024-04-04T06:09:13.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.071+0000 7f9fecfdd740 20 client.6282 trim_cache size 0 max 16384 2024-04-04T06:09:13.067 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:09:13.068 INFO:teuthology.orchestra.run.smithi159.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-04T06:09:13.068 INFO:teuthology.orchestra.run.smithi159.stdout: "scrubs": { 2024-04-04T06:09:13.068 INFO:teuthology.orchestra.run.smithi159.stdout: "02674303-2a3b-41d2-b57f-434d308ce48d": { 2024-04-04T06:09:13.068 INFO:teuthology.orchestra.run.smithi159.stdout: "path": "/", 2024-04-04T06:09:13.068 INFO:teuthology.orchestra.run.smithi159.stdout: "tag": "02674303-2a3b-41d2-b57f-434d308ce48d", 2024-04-04T06:09:13.068 INFO:teuthology.orchestra.run.smithi159.stdout: "options": "recursive" 2024-04-04T06:09:13.068 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:09:13.068 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:09:13.068 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:09:13.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.072+0000 7f9fea200640 1 -- 172.21.15.159:0/1861498033 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9fc006c920 msgr2=0x7f9fc006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:13.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.072+0000 7f9fea200640 1 --2- 172.21.15.159:0/1861498033 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9fc006c920 0x7f9fc006ede0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4000f60 tx=0x7f9fd4009040 comp rx=0 tx=0).stop 2024-04-04T06:09:13.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.072+0000 7f9fea200640 1 -- 172.21.15.159:0/1861498033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fe4159080 msgr2=0x7f9fe40c5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:13.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.072+0000 7f9fea200640 1 --2- 172.21.15.159:0/1861498033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fe4159080 0x7f9fe40c5de0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc011570 tx=0x7f9fcc011a30 comp rx=0 tx=0).stop 2024-04-04T06:09:13.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.072+0000 7f9fea200640 1 -- 172.21.15.159:0/1861498033 shutdown_connections 2024-04-04T06:09:13.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.072+0000 7f9fea200640 1 --2- 172.21.15.159:0/1861498033 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9fc006c920 0x7f9fc006ede0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.072+0000 7f9fea200640 1 --2- 172.21.15.159:0/1861498033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fe409db20 0x7f9fe40afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.072+0000 7f9fea200640 1 --2- 172.21.15.159:0/1861498033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fe4159080 0x7f9fe40c5de0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.072+0000 7f9fea200640 1 --2- 172.21.15.159:0/1861498033 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9fc80073d0 0x7f9fe415a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:13.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.072+0000 7f9fea200640 1 -- 172.21.15.159:0/1861498033 >> 172.21.15.159:0/1861498033 conn(0x7f9fe406b750 msgr2=0x7f9fe40b15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:13.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.072+0000 7f9fea200640 1 -- 172.21.15.159:0/1861498033 shutdown_connections 2024-04-04T06:09:13.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:13.072+0000 7f9fea200640 1 -- 172.21.15.159:0/1861498033 wait complete. 2024-04-04T06:09:13.081 INFO:tasks.cephfs.filesystem:scrub status for tag:02674303-2a3b-41d2-b57f-434d308ce48d - {'path': '/', 'tag': '02674303-2a3b-41d2-b57f-434d308ce48d', 'options': 'recursive'} 2024-04-04T06:09:17.682 INFO:tasks.ceph.mon.a.smithi159.stderr:2024-04-04T06:09:17.689+0000 7f1998e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-04T06:09:41.623 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:09:41.631 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:09:43.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:09:43.083 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-04T06:09:43.305 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.311+0000 7fdae0c00640 1 Processor -- start 2024-04-04T06:09:43.305 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.311+0000 7fdae0c00640 1 -- start start 2024-04-04T06:09:43.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.313+0000 7fdae0c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fdadc15a870 0x7fdadc15e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.313+0000 7fdae0c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdadc15ac50 0x7fdadc15ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.313+0000 7fdae0c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdadc0b4c00 con 0x7fdadc0b3f10 2024-04-04T06:09:43.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.313+0000 7fdae0c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fdadc06d5b0 con 0x7fdadc15a870 2024-04-04T06:09:43.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.313+0000 7fdae0c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fdadc06db70 con 0x7fdadc15ac50 2024-04-04T06:09:43.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.313+0000 7fdada800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fdadc0b3f10 0x7fdadc0b4380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:47810/0 (socket says 172.21.15.159:47810) 2024-04-04T06:09:43.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.313+0000 7fdada800640 1 -- 172.21.15.159:0/528518317 learned_addr learned my addr 172.21.15.159:0/528518317 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:09:43.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.313+0000 7fdad9e00640 1 --2- 172.21.15.159:0/528518317 >> v2:172.21.15.159:3300/0 conn(0x7fdadc15a870 0x7fdadc15e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.314+0000 7fdad1400640 1 --2- 172.21.15.159:0/528518317 >> v2:172.21.15.178:3300/0 conn(0x7fdadc15ac50 0x7fdadc15ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.314+0000 7fdad9400640 1 -- 172.21.15.159:0/528518317 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2254856446 0 0) 0x7fdadc0b4c00 con 0x7fdadc0b3f10 2024-04-04T06:09:43.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.314+0000 7fdad9400640 1 -- 172.21.15.159:0/528518317 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdac0003520 con 0x7fdadc0b3f10 2024-04-04T06:09:43.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.314+0000 7fdad9e00640 1 -- 172.21.15.159:0/528518317 >> v2:172.21.15.178:3300/0 conn(0x7fdadc15ac50 msgr2=0x7fdadc15ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:43.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.314+0000 7fdad9e00640 1 --2- 172.21.15.159:0/528518317 >> v2:172.21.15.178:3300/0 conn(0x7fdadc15ac50 0x7fdadc15ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.314+0000 7fdad9e00640 1 -- 172.21.15.159:0/528518317 >> v1:172.21.15.159:6789/0 conn(0x7fdadc0b3f10 legacy=0x7fdadc0b4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:43.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.314+0000 7fdad9e00640 1 -- 172.21.15.159:0/528518317 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdadc0b4ee0 con 0x7fdadc15a870 2024-04-04T06:09:43.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.316+0000 7fdad9e00640 1 --2- 172.21.15.159:0/528518317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdadc15a870 0x7fdadc15e590 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fdadc15e4f0 tx=0x7fdacc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22fb557f39675ae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:43.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.316+0000 7fdad9400640 1 -- 172.21.15.159:0/528518317 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc035020 con 0x7fdadc15a870 2024-04-04T06:09:43.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.316+0000 7fdad9400640 1 -- 172.21.15.159:0/528518317 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdacc031830 con 0x7fdadc15a870 2024-04-04T06:09:43.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.316+0000 7fdad9400640 1 -- 172.21.15.159:0/528518317 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc031c00 con 0x7fdadc15a870 2024-04-04T06:09:43.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.317+0000 7fdae0c00640 1 -- 172.21.15.159:0/528518317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdadc15a870 msgr2=0x7fdadc15e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:43.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.317+0000 7fdae0c00640 1 --2- 172.21.15.159:0/528518317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdadc15a870 0x7fdadc15e590 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fdadc15e4f0 tx=0x7fdacc02b8d0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.317+0000 7fdae0c00640 1 -- 172.21.15.159:0/528518317 shutdown_connections 2024-04-04T06:09:43.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.317+0000 7fdae0c00640 1 --2- 172.21.15.159:0/528518317 >> v2:172.21.15.178:3300/0 conn(0x7fdadc15ac50 0x7fdadc15ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.317+0000 7fdae0c00640 1 --2- 172.21.15.159:0/528518317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdadc15a870 0x7fdadc15e590 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.317+0000 7fdae0c00640 1 -- 172.21.15.159:0/528518317 >> 172.21.15.159:0/528518317 conn(0x7fdadc06b750 msgr2=0x7fdadc06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:43.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.317+0000 7fdae0c00640 1 -- 172.21.15.159:0/528518317 shutdown_connections 2024-04-04T06:09:43.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.317+0000 7fdae0c00640 1 -- 172.21.15.159:0/528518317 wait complete. 2024-04-04T06:09:43.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.319+0000 7fdae0c00640 1 Processor -- start 2024-04-04T06:09:43.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.319+0000 7fdae0c00640 1 -- start start 2024-04-04T06:09:43.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.320+0000 7fdae0c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdadc0b3f10 0x7fdadc1f0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.320+0000 7fdae0c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdadc15a870 0x7fdadc159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.320+0000 7fdae0c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdadc15ac50 0x7fdadc09d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.320+0000 7fdae0c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc0d0400 con 0x7fdadc15ac50 2024-04-04T06:09:43.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.320+0000 7fdae0c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc0d0280 con 0x7fdadc15a870 2024-04-04T06:09:43.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.320+0000 7fdae0c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdadc0d0580 con 0x7fdadc0b3f10 2024-04-04T06:09:43.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.320+0000 7fdada800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdadc15ac50 0x7fdadc09d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.320+0000 7fdada800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdadc15ac50 0x7fdadc09d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38668/0 (socket says 172.21.15.159:38668) 2024-04-04T06:09:43.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.320+0000 7fdada800640 1 -- 172.21.15.159:0/1597395034 learned_addr learned my addr 172.21.15.159:0/1597395034 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:43.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.320+0000 7fdad1400640 1 --2- 172.21.15.159:0/1597395034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdadc15a870 0x7fdadc159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.320+0000 7fdad9e00640 1 --2- 172.21.15.159:0/1597395034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdadc0b3f10 0x7fdadc1f0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.320+0000 7fdada800640 1 -- 172.21.15.159:0/1597395034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdadc0b3f10 msgr2=0x7fdadc1f0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:43.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.320+0000 7fdada800640 1 --2- 172.21.15.159:0/1597395034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdadc0b3f10 0x7fdadc1f0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.320+0000 7fdada800640 1 -- 172.21.15.159:0/1597395034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdadc15a870 msgr2=0x7fdadc159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:43.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.320+0000 7fdada800640 1 --2- 172.21.15.159:0/1597395034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdadc15a870 0x7fdadc159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.320+0000 7fdada800640 1 -- 172.21.15.159:0/1597395034 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdadc0af8b0 con 0x7fdadc15ac50 2024-04-04T06:09:43.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.320+0000 7fdad1400640 1 --2- 172.21.15.159:0/1597395034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdadc15a870 0x7fdadc159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:43.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.321+0000 7fdada800640 1 --2- 172.21.15.159:0/1597395034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdadc15ac50 0x7fdadc09d480 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fdad400cb40 tx=0x7fdad400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:43.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.321+0000 7fdad2a00640 1 -- 172.21.15.159:0/1597395034 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4017070 con 0x7fdadc15ac50 2024-04-04T06:09:43.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.321+0000 7fdad2a00640 1 -- 172.21.15.159:0/1597395034 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdad400aca0 con 0x7fdadc15ac50 2024-04-04T06:09:43.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.321+0000 7fdad2a00640 1 -- 172.21.15.159:0/1597395034 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4012620 con 0x7fdadc15ac50 2024-04-04T06:09:43.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.321+0000 7fdae0c00640 1 -- 172.21.15.159:0/1597395034 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdadc1575c0 con 0x7fdadc15ac50 2024-04-04T06:09:43.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.321+0000 7fdae0c00640 1 -- 172.21.15.159:0/1597395034 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdadc157b80 con 0x7fdadc15ac50 2024-04-04T06:09:43.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.324+0000 7fdad2a00640 1 -- 172.21.15.159:0/1597395034 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdad401a070 con 0x7fdadc15ac50 2024-04-04T06:09:43.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.325+0000 7fdad2a00640 1 --2- 172.21.15.159:0/1597395034 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdab806c8f0 0x7fdab806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.325+0000 7fdad2a00640 1 -- 172.21.15.159:0/1597395034 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fdad408ba50 con 0x7fdadc15ac50 2024-04-04T06:09:43.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.326+0000 7fdae3967740 1 Processor -- start 2024-04-04T06:09:43.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.326+0000 7fdad9e00640 1 --2- 172.21.15.159:0/1597395034 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdab806c8f0 0x7fdab806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.326+0000 7fdae3967740 1 -- start start 2024-04-04T06:09:43.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.327+0000 7fdae3967740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e703000 0x556b4e700cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.327+0000 7fdae3967740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b4e7033e0 0x556b4e6ba630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.327+0000 7fdae3967740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556b4e6e2940 0x556b4e6bab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.327+0000 7fdae3967740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556b4e6206d0 con 0x556b4e703000 2024-04-04T06:09:43.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.327+0000 7fdae3967740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556b4e5c8770 con 0x556b4e7033e0 2024-04-04T06:09:43.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.327+0000 7fdae3967740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x556b4e61c680 con 0x556b4e6e2940 2024-04-04T06:09:43.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.327+0000 7fdad9e00640 1 --2- 172.21.15.159:0/1597395034 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdab806c8f0 0x7fdab806edb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdacc036000 tx=0x7fdacc008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:43.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.327+0000 7fdad9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e703000 0x556b4e700cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.327+0000 7fdad9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e703000 0x556b4e700cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38678/0 (socket says 172.21.15.159:38678) 2024-04-04T06:09:43.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.327+0000 7fdad9e00640 1 -- 172.21.15.159:0/519565880 learned_addr learned my addr 172.21.15.159:0/519565880 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:43.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.328+0000 7fdada800640 1 --2- 172.21.15.159:0/519565880 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556b4e6e2940 0x556b4e6bab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.328+0000 7fdad1400640 1 --2- 172.21.15.159:0/519565880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b4e7033e0 0x556b4e6ba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.328+0000 7fdad9e00640 1 -- 172.21.15.159:0/519565880 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556b4e6e2940 msgr2=0x556b4e6bab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:43.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.328+0000 7fdad9e00640 1 --2- 172.21.15.159:0/519565880 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556b4e6e2940 0x556b4e6bab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.328+0000 7fdad9e00640 1 -- 172.21.15.159:0/519565880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b4e7033e0 msgr2=0x556b4e6ba630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:43.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.328+0000 7fdad9e00640 1 --2- 172.21.15.159:0/519565880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b4e7033e0 0x556b4e6ba630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.328+0000 7fdad9e00640 1 -- 172.21.15.159:0/519565880 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b4e3fe7d0 con 0x556b4e703000 2024-04-04T06:09:43.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.328+0000 7fdad9e00640 1 --2- 172.21.15.159:0/519565880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e703000 0x556b4e700cc0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009860 tx=0x7fdacc030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d2a641a5fa48aa server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:43.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.328+0000 7fdaca600640 1 -- 172.21.15.159:0/519565880 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc035040 con 0x556b4e703000 2024-04-04T06:09:43.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.329+0000 7fdaca600640 1 -- 172.21.15.159:0/519565880 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdacc050630 con 0x556b4e703000 2024-04-04T06:09:43.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.329+0000 7fdaca600640 1 -- 172.21.15.159:0/519565880 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc0509e0 con 0x556b4e703000 2024-04-04T06:09:43.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.329+0000 7fdae3967740 1 -- 172.21.15.159:0/519565880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e703000 msgr2=0x556b4e700cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:43.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.329+0000 7fdae3967740 1 --2- 172.21.15.159:0/519565880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e703000 0x556b4e700cc0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009860 tx=0x7fdacc030000 comp rx=0 tx=0).stop 2024-04-04T06:09:43.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.329+0000 7fdae3967740 1 -- 172.21.15.159:0/519565880 shutdown_connections 2024-04-04T06:09:43.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.329+0000 7fdae3967740 1 --2- 172.21.15.159:0/519565880 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556b4e6e2940 0x556b4e6bab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.329+0000 7fdae3967740 1 --2- 172.21.15.159:0/519565880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b4e7033e0 0x556b4e6ba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.329+0000 7fdae3967740 1 --2- 172.21.15.159:0/519565880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e703000 0x556b4e700cc0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.329+0000 7fdae3967740 1 -- 172.21.15.159:0/519565880 >> 172.21.15.159:0/519565880 conn(0x556b4e6b7790 msgr2=0x556b4e700310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:43.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.329+0000 7fdae3967740 1 -- 172.21.15.159:0/519565880 shutdown_connections 2024-04-04T06:09:43.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.329+0000 7fdae3967740 1 -- 172.21.15.159:0/519565880 wait complete. 2024-04-04T06:09:43.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.330+0000 7fdae3967740 1 -- start start 2024-04-04T06:09:43.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.330+0000 7fdae3967740 1 Processor -- start 2024-04-04T06:09:43.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.331+0000 7fdae3967740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b4e703000 0x556b4e6ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.331+0000 7fdae3967740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e7033e0 0x556b4e6fc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.331+0000 7fdae3967740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556b4e6fa9b0 0x556b4e6e1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.331+0000 7fdae3967740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556b4e6b5dc0 con 0x556b4e7033e0 2024-04-04T06:09:43.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.331+0000 7fdae3967740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556b4e5dbaa0 con 0x556b4e703000 2024-04-04T06:09:43.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.331+0000 7fdae3967740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x556b4e6b9bc0 con 0x556b4e6fa9b0 2024-04-04T06:09:43.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.331+0000 7fdae3967740 10 client.0 _ll_register_callbacks cb 0x556b4e5db490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:09:43.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.332+0000 7fdad1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e7033e0 0x556b4e6fc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.332+0000 7fdad1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e7033e0 0x556b4e6fc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38692/0 (socket says 172.21.15.159:38692) 2024-04-04T06:09:43.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.332+0000 7fdad1400640 1 -- 172.21.15.159:0/1391001022 learned_addr learned my addr 172.21.15.159:0/1391001022 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:43.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.332+0000 7fdad9e00640 1 --2- 172.21.15.159:0/1391001022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b4e703000 0x556b4e6ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.332+0000 7fdada800640 1 --2- 172.21.15.159:0/1391001022 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556b4e6fa9b0 0x556b4e6e1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.332+0000 7fdad1400640 1 -- 172.21.15.159:0/1391001022 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556b4e6fa9b0 msgr2=0x556b4e6e1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:43.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.332+0000 7fdad1400640 1 --2- 172.21.15.159:0/1391001022 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556b4e6fa9b0 0x556b4e6e1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.332+0000 7fdad1400640 1 -- 172.21.15.159:0/1391001022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b4e703000 msgr2=0x556b4e6ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:43.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.332+0000 7fdad1400640 1 --2- 172.21.15.159:0/1391001022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b4e703000 0x556b4e6ca6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.332+0000 7fdad1400640 1 -- 172.21.15.159:0/1391001022 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b4e3343f0 con 0x556b4e7033e0 2024-04-04T06:09:43.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.332+0000 7fdad1400640 1 --2- 172.21.15.159:0/1391001022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e7033e0 0x556b4e6fc510 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdac400aee0 tx=0x7fdac40106c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6759587c9854e32 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:43.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.332+0000 7fdabea00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:09:43.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.333+0000 7fdabea00640 1 -- 172.21.15.159:0/1391001022 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4019070 con 0x556b4e7033e0 2024-04-04T06:09:43.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.333+0000 7fdabea00640 1 -- 172.21.15.159:0/1391001022 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fdac4013880 con 0x556b4e7033e0 2024-04-04T06:09:43.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.333+0000 7fdae3967740 1 -- 172.21.15.159:0/1391001022 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdacc050630 con 0x556b4e7033e0 2024-04-04T06:09:43.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.333+0000 7fdabea00640 1 -- 172.21.15.159:0/1391001022 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdac4024050 con 0x556b4e7033e0 2024-04-04T06:09:43.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.333+0000 7fdae3967740 10 client.6297 fetch_fsmap learned FSMap version 34 2024-04-04T06:09:43.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.333+0000 7fdae3967740 1 -- 172.21.15.159:0/1391001022 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x556b4e2f40f0 con 0x556b4e7033e0 2024-04-04T06:09:43.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.333+0000 7fdabea00640 1 -- 172.21.15.159:0/1391001022 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdacc050630 con 0x556b4e7033e0 2024-04-04T06:09:43.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.333+0000 7fdabea00640 1 -- 172.21.15.159:0/1391001022 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7fdac4027020 con 0x556b4e7033e0 2024-04-04T06:09:43.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.333+0000 7fdae3967740 10 client.6297 fetch_fsmap finished waiting for FSMap version 34 2024-04-04T06:09:43.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.333+0000 7fdae3967740 10 client.6297 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-04T06:09:43.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.333+0000 7fdae3967740 20 client.6297 populate_metadata read hostname 'smithi159' 2024-04-04T06:09:43.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.333+0000 7fdae3967740 1 --2- 172.21.15.159:0/1391001022 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x556b4e6dbc80 0x556b4e7224a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.333+0000 7fdae3967740 4 client.6297 mds_command: new command op to 6039 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:09:43.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.333+0000 7fdae3967740 1 -- 172.21.15.159:0/1391001022 --> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b4e62dc10 con 0x556b4e6dbc80 2024-04-04T06:09:43.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.334+0000 7fdad9e00640 1 --2- 172.21.15.159:0/1391001022 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x556b4e6dbc80 0x556b4e7224a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.334+0000 7fdad9e00640 1 --2- 172.21.15.159:0/1391001022 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x556b4e6dbc80 0x556b4e7224a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cbd33fc804d9865 server_cookie=5f8eb3eda554356a in_seq=0 out_seq=0 2024-04-04T06:09:43.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.334+0000 7fdabea00640 10 client.6297 ms_handle_connect on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:43.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.335+0000 7fdabea00640 1 -- 172.21.15.159:0/1391001022 <== mds.0 v2:172.21.15.159:6833/962475038 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b4e62dc10 con 0x556b4e6dbc80 2024-04-04T06:09:43.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.335+0000 7fdabea00640 10 client.6297 handle_command_reply: tid=0 2024-04-04T06:09:43.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.335+0000 7fdae3967740 1 client.6297 shutdown 2024-04-04T06:09:43.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.335+0000 7fdae3967740 1 -- 172.21.15.159:0/1391001022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e7033e0 msgr2=0x556b4e6fc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:43.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.335+0000 7fdae3967740 1 --2- 172.21.15.159:0/1391001022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e7033e0 0x556b4e6fc510 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdac400aee0 tx=0x7fdac40106c0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.335+0000 7fdae3967740 1 -- 172.21.15.159:0/1391001022 shutdown_connections 2024-04-04T06:09:43.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.335+0000 7fdae3967740 1 --2- 172.21.15.159:0/1391001022 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556b4e6fa9b0 0x556b4e6e1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.335+0000 7fdae3967740 1 --2- 172.21.15.159:0/1391001022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e7033e0 0x556b4e6fc510 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.335+0000 7fdae3967740 1 --2- 172.21.15.159:0/1391001022 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x556b4e6dbc80 0x556b4e7224a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.335+0000 7fdae3967740 1 --2- 172.21.15.159:0/1391001022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b4e703000 0x556b4e6ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.335+0000 7fdae3967740 1 -- 172.21.15.159:0/1391001022 >> 172.21.15.159:0/1391001022 conn(0x556b4e6e2940 msgr2=0x556b4e703860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:43.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.335+0000 7fdabea00640 0 client.6297 ms_handle_reset on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:43.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.335+0000 7fdae3967740 1 -- 172.21.15.159:0/1391001022 shutdown_connections 2024-04-04T06:09:43.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.336+0000 7fdae3967740 1 -- 172.21.15.159:0/1391001022 wait complete. 2024-04-04T06:09:43.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.336+0000 7fdae3967740 20 client.6297 trim_cache size 0 max 16384 2024-04-04T06:09:43.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.355+0000 7fdae3967740 1 Processor -- start 2024-04-04T06:09:43.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.355+0000 7fdae3967740 1 -- start start 2024-04-04T06:09:43.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.356+0000 7fdae3967740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e6fa9b0 0x556b4e6fc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.356+0000 7fdae3967740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b4e703000 0x556b4e727b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.356+0000 7fdae3967740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556b4e7033e0 0x556b4e702310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.356+0000 7fdae3967740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc1fc410 con 0x556b4e6fa9b0 2024-04-04T06:09:43.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.356+0000 7fdae3967740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc1f4490 con 0x556b4e703000 2024-04-04T06:09:43.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.356+0000 7fdae3967740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdadc1f3820 con 0x556b4e7033e0 2024-04-04T06:09:43.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.356+0000 7fdad9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e6fa9b0 0x556b4e6fc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.356+0000 7fdad9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e6fa9b0 0x556b4e6fc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38700/0 (socket says 172.21.15.159:38700) 2024-04-04T06:09:43.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.356+0000 7fdad9e00640 1 -- 172.21.15.159:0/1423587420 learned_addr learned my addr 172.21.15.159:0/1423587420 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:43.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.356+0000 7fdad9e00640 1 -- 172.21.15.159:0/1423587420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556b4e7033e0 msgr2=0x556b4e702310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:43.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.356+0000 7fdad1400640 1 --2- 172.21.15.159:0/1423587420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b4e703000 0x556b4e727b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.356+0000 7fdada800640 1 --2- 172.21.15.159:0/1423587420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556b4e7033e0 0x556b4e702310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.356+0000 7fdad9e00640 1 --2- 172.21.15.159:0/1423587420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556b4e7033e0 0x556b4e702310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.356+0000 7fdad9e00640 1 -- 172.21.15.159:0/1423587420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b4e703000 msgr2=0x556b4e727b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:43.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.356+0000 7fdad9e00640 1 --2- 172.21.15.159:0/1423587420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b4e703000 0x556b4e727b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.356+0000 7fdad9e00640 1 -- 172.21.15.159:0/1423587420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b4e6ff720 con 0x556b4e6fa9b0 2024-04-04T06:09:43.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.357+0000 7fdad9e00640 1 --2- 172.21.15.159:0/1423587420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e6fa9b0 0x556b4e6fc510 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009560 tx=0x7fdacc03d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d44e17b354cc678 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:43.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.357+0000 7fdac9c00640 1 -- 172.21.15.159:0/1423587420 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc05f070 con 0x556b4e6fa9b0 2024-04-04T06:09:43.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.357+0000 7fdac9c00640 1 -- 172.21.15.159:0/1423587420 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdacc004260 con 0x556b4e6fa9b0 2024-04-04T06:09:43.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.357+0000 7fdac9c00640 1 -- 172.21.15.159:0/1423587420 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc035040 con 0x556b4e6fa9b0 2024-04-04T06:09:43.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.358+0000 7fdae3967740 1 -- 172.21.15.159:0/1423587420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e6fa9b0 msgr2=0x556b4e6fc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:43.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.358+0000 7fdae3967740 1 --2- 172.21.15.159:0/1423587420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e6fa9b0 0x556b4e6fc510 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009560 tx=0x7fdacc03d040 comp rx=0 tx=0).stop 2024-04-04T06:09:43.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.358+0000 7fdae3967740 1 -- 172.21.15.159:0/1423587420 shutdown_connections 2024-04-04T06:09:43.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.358+0000 7fdae3967740 1 --2- 172.21.15.159:0/1423587420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556b4e7033e0 0x556b4e702310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.358+0000 7fdae3967740 1 --2- 172.21.15.159:0/1423587420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b4e703000 0x556b4e727b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.358+0000 7fdae3967740 1 --2- 172.21.15.159:0/1423587420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e6fa9b0 0x556b4e6fc510 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.358+0000 7fdae3967740 1 -- 172.21.15.159:0/1423587420 >> 172.21.15.159:0/1423587420 conn(0x556b4e6e2940 msgr2=0x556b4e6fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:43.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.358+0000 7fdae3967740 1 -- 172.21.15.159:0/1423587420 shutdown_connections 2024-04-04T06:09:43.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.358+0000 7fdae3967740 1 -- 172.21.15.159:0/1423587420 wait complete. 2024-04-04T06:09:43.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.359+0000 7fdae3967740 1 -- start start 2024-04-04T06:09:43.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.359+0000 7fdae3967740 1 Processor -- start 2024-04-04T06:09:43.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.360+0000 7fdae3967740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e6fa9b0 0x556b4e767ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.360+0000 7fdae3967740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b4e703000 0x556b4e6d7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.360+0000 7fdae3967740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556b4e7033e0 0x556b4e7037c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.360+0000 7fdae3967740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556b4e654940 con 0x556b4e6fa9b0 2024-04-04T06:09:43.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.360+0000 7fdae3967740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc1f3e40 con 0x556b4e703000 2024-04-04T06:09:43.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.360+0000 7fdae3967740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x556b4e7278e0 con 0x556b4e7033e0 2024-04-04T06:09:43.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.360+0000 7fdae3967740 10 client.0 _ll_register_callbacks cb 0x7fdac400aaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:09:43.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.360+0000 7fdad9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e6fa9b0 0x556b4e767ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.360+0000 7fdad9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e6fa9b0 0x556b4e767ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38712/0 (socket says 172.21.15.159:38712) 2024-04-04T06:09:43.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.360+0000 7fdad9e00640 1 -- 172.21.15.159:0/3448521306 learned_addr learned my addr 172.21.15.159:0/3448521306 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:43.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.360+0000 7fdad9e00640 1 -- 172.21.15.159:0/3448521306 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556b4e7033e0 msgr2=0x556b4e7037c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:43.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.360+0000 7fdada800640 1 --2- 172.21.15.159:0/3448521306 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556b4e7033e0 0x556b4e7037c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.360+0000 7fdad1400640 1 --2- 172.21.15.159:0/3448521306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b4e703000 0x556b4e6d7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.361+0000 7fdad9e00640 1 --2- 172.21.15.159:0/3448521306 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556b4e7033e0 0x556b4e7037c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.361+0000 7fdad9e00640 1 -- 172.21.15.159:0/3448521306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b4e703000 msgr2=0x556b4e6d7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:43.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.361+0000 7fdad9e00640 1 --2- 172.21.15.159:0/3448521306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b4e703000 0x556b4e6d7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.361+0000 7fdad9e00640 1 -- 172.21.15.159:0/3448521306 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b4e6e9380 con 0x556b4e6fa9b0 2024-04-04T06:09:43.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.361+0000 7fdad1400640 1 --2- 172.21.15.159:0/3448521306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b4e703000 0x556b4e6d7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:43.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.361+0000 7fdad9e00640 1 --2- 172.21.15.159:0/3448521306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e6fa9b0 0x556b4e767ca0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fdacc05a040 tx=0x7fdacc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3219041348e867 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:43.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.361+0000 7fdabea00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:09:43.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.361+0000 7fdabea00640 1 -- 172.21.15.159:0/3448521306 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc05f020 con 0x556b4e6fa9b0 2024-04-04T06:09:43.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.361+0000 7fdabea00640 1 -- 172.21.15.159:0/3448521306 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fdacc003810 con 0x556b4e6fa9b0 2024-04-04T06:09:43.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.361+0000 7fdae3967740 1 -- 172.21.15.159:0/3448521306 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdacc004260 con 0x556b4e6fa9b0 2024-04-04T06:09:43.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.361+0000 7fdabea00640 1 -- 172.21.15.159:0/3448521306 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdacc035040 con 0x556b4e6fa9b0 2024-04-04T06:09:43.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.361+0000 7fdae3967740 10 client.6303 fetch_fsmap learned FSMap version 34 2024-04-04T06:09:43.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.362+0000 7fdae3967740 1 -- 172.21.15.159:0/3448521306 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x556b4e6d7de0 con 0x556b4e6fa9b0 2024-04-04T06:09:43.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.362+0000 7fdabea00640 1 -- 172.21.15.159:0/3448521306 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdacc004260 con 0x556b4e6fa9b0 2024-04-04T06:09:43.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.362+0000 7fdabea00640 1 -- 172.21.15.159:0/3448521306 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7fdacc05ace0 con 0x556b4e6fa9b0 2024-04-04T06:09:43.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.362+0000 7fdae3967740 10 client.6303 fetch_fsmap finished waiting for FSMap version 34 2024-04-04T06:09:43.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.362+0000 7fdae3967740 10 client.6303 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-04T06:09:43.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.362+0000 7fdae3967740 20 client.6303 populate_metadata read hostname 'smithi159' 2024-04-04T06:09:43.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.362+0000 7fdae3967740 1 --2- 172.21.15.159:0/3448521306 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x556b4e703d00 0x556b4e6ddf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.362+0000 7fdae3967740 4 client.6303 mds_command: new command op to 6039 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-04T06:09:43.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.362+0000 7fdae3967740 1 -- 172.21.15.159:0/3448521306 --> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556b4e627f60 con 0x556b4e703d00 2024-04-04T06:09:43.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.362+0000 7fdad1400640 1 --2- 172.21.15.159:0/3448521306 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x556b4e703d00 0x556b4e6ddf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.363+0000 7fdad1400640 1 --2- 172.21.15.159:0/3448521306 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x556b4e703d00 0x556b4e6ddf50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1014640015113a25 server_cookie=99fec731e397dc4 in_seq=0 out_seq=0 2024-04-04T06:09:43.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.363+0000 7fdabea00640 10 client.6303 ms_handle_connect on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:43.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.363+0000 7fdabea00640 1 -- 172.21.15.159:0/3448521306 <== mds.0 v2:172.21.15.159:6833/962475038 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556b4e627f60 con 0x556b4e703d00 2024-04-04T06:09:43.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.363+0000 7fdabea00640 10 client.6303 handle_command_reply: tid=0 2024-04-04T06:09:43.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.363+0000 7fdae3967740 1 client.6303 shutdown 2024-04-04T06:09:43.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.363+0000 7fdae3967740 1 -- 172.21.15.159:0/3448521306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e6fa9b0 msgr2=0x556b4e767ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:43.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.363+0000 7fdae3967740 1 --2- 172.21.15.159:0/3448521306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e6fa9b0 0x556b4e767ca0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fdacc05a040 tx=0x7fdacc004730 comp rx=0 tx=0).stop 2024-04-04T06:09:43.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.363+0000 7fdae3967740 1 -- 172.21.15.159:0/3448521306 shutdown_connections 2024-04-04T06:09:43.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.363+0000 7fdae3967740 1 --2- 172.21.15.159:0/3448521306 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556b4e7033e0 0x556b4e7037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.363+0000 7fdae3967740 1 --2- 172.21.15.159:0/3448521306 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x556b4e703d00 0x556b4e6ddf50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.363+0000 7fdae3967740 1 --2- 172.21.15.159:0/3448521306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b4e703000 0x556b4e6d7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.363+0000 7fdae3967740 1 --2- 172.21.15.159:0/3448521306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b4e6fa9b0 0x556b4e767ca0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.363+0000 7fdabea00640 0 client.6303 ms_handle_reset on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:43.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.363+0000 7fdae3967740 1 -- 172.21.15.159:0/3448521306 >> 172.21.15.159:0/3448521306 conn(0x556b4e6e2940 msgr2=0x556b4e6fc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:43.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.363+0000 7fdae3967740 1 -- 172.21.15.159:0/3448521306 shutdown_connections 2024-04-04T06:09:43.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.364+0000 7fdae3967740 1 -- 172.21.15.159:0/3448521306 wait complete. 2024-04-04T06:09:43.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.364+0000 7fdae3967740 20 client.6303 trim_cache size 0 max 16384 2024-04-04T06:09:43.361 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:09:43.361 INFO:teuthology.orchestra.run.smithi159.stdout: "status": "no active scrubs running", 2024-04-04T06:09:43.361 INFO:teuthology.orchestra.run.smithi159.stdout: "scrubs": {} 2024-04-04T06:09:43.361 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:09:43.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.364+0000 7fdae0c00640 1 -- 172.21.15.159:0/1597395034 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdab806c8f0 msgr2=0x7fdab806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:43.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.364+0000 7fdae0c00640 1 --2- 172.21.15.159:0/1597395034 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdab806c8f0 0x7fdab806edb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdacc036000 tx=0x7fdacc008030 comp rx=0 tx=0).stop 2024-04-04T06:09:43.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.364+0000 7fdae0c00640 1 -- 172.21.15.159:0/1597395034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdadc15ac50 msgr2=0x7fdadc09d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:43.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.364+0000 7fdae0c00640 1 --2- 172.21.15.159:0/1597395034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdadc15ac50 0x7fdadc09d480 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fdad400cb40 tx=0x7fdad400a060 comp rx=0 tx=0).stop 2024-04-04T06:09:43.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.364+0000 7fdae0c00640 1 -- 172.21.15.159:0/1597395034 shutdown_connections 2024-04-04T06:09:43.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.364+0000 7fdae0c00640 1 --2- 172.21.15.159:0/1597395034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdadc15ac50 0x7fdadc09d480 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.364+0000 7fdae0c00640 1 --2- 172.21.15.159:0/1597395034 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdab806c8f0 0x7fdab806edb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.364+0000 7fdae0c00640 1 --2- 172.21.15.159:0/1597395034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdadc15a870 0x7fdadc159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.364+0000 7fdae0c00640 1 --2- 172.21.15.159:0/1597395034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdadc0b3f10 0x7fdadc1f0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.364+0000 7fdae0c00640 1 -- 172.21.15.159:0/1597395034 >> 172.21.15.159:0/1597395034 conn(0x7fdadc06b750 msgr2=0x7fdadc0b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:43.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.364+0000 7fdae0c00640 1 -- 172.21.15.159:0/1597395034 shutdown_connections 2024-04-04T06:09:43.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.365+0000 7fdae0c00640 1 -- 172.21.15.159:0/1597395034 wait complete. 2024-04-04T06:09:43.376 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-04T06:09:43.376 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:09:43.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.572+0000 7fc72f800640 1 Processor -- start 2024-04-04T06:09:43.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.572+0000 7fc72f800640 1 -- start start 2024-04-04T06:09:43.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.573+0000 7fc72f800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc7280af830 0x7fc728073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.573+0000 7fc72f800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc7280d0650 0x7fc7280d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.574+0000 7fc72f800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc72815fe80 con 0x7fc72815c390 2024-04-04T06:09:43.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.574+0000 7fc72f800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fc72815c770 con 0x7fc7280af830 2024-04-04T06:09:43.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.574+0000 7fc72f800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc7280d0fc0 con 0x7fc7280d0650 2024-04-04T06:09:43.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.574+0000 7fc727e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc7280af830 0x7fc728073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.574+0000 7fc727e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc7280af830 0x7fc728073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38726/0 (socket says 172.21.15.159:38726) 2024-04-04T06:09:43.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.574+0000 7fc727e00640 1 -- 172.21.15.159:0/375523193 learned_addr learned my addr 172.21.15.159:0/375523193 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:43.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.574+0000 7fc72d800640 1 --2- 172.21.15.159:0/375523193 >> v2:172.21.15.178:3300/0 conn(0x7fc7280d0650 0x7fc7280d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.574+0000 7fc727e00640 1 -- 172.21.15.159:0/375523193 >> v2:172.21.15.178:3300/0 conn(0x7fc7280d0650 msgr2=0x7fc7280d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:43.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.574+0000 7fc727e00640 1 --2- 172.21.15.159:0/375523193 >> v2:172.21.15.178:3300/0 conn(0x7fc7280d0650 0x7fc7280d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.574+0000 7fc727e00640 1 -- 172.21.15.159:0/375523193 >> v1:172.21.15.178:6789/0 conn(0x7fc72815c390 legacy=0x7fc728160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:43.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.574+0000 7fc727e00640 1 -- 172.21.15.159:0/375523193 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7280ad120 con 0x7fc7280af830 2024-04-04T06:09:43.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.575+0000 7fc727e00640 1 --2- 172.21.15.159:0/375523193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7280af830 0x7fc728073790 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc72806eaa0 tx=0x7fc71402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed39c4aba25a618b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:43.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.576+0000 7fc727400640 1 -- 172.21.15.159:0/375523193 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc714035020 con 0x7fc7280af830 2024-04-04T06:09:43.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.576+0000 7fc727400640 1 -- 172.21.15.159:0/375523193 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc714031830 con 0x7fc7280af830 2024-04-04T06:09:43.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.576+0000 7fc727400640 1 -- 172.21.15.159:0/375523193 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc714031c00 con 0x7fc7280af830 2024-04-04T06:09:43.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.576+0000 7fc72f800640 1 -- 172.21.15.159:0/375523193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7280af830 msgr2=0x7fc728073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:43.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.576+0000 7fc72f800640 1 --2- 172.21.15.159:0/375523193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7280af830 0x7fc728073790 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc72806eaa0 tx=0x7fc71402b8d0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.577+0000 7fc72f800640 1 -- 172.21.15.159:0/375523193 shutdown_connections 2024-04-04T06:09:43.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.577+0000 7fc72f800640 1 --2- 172.21.15.159:0/375523193 >> v2:172.21.15.178:3300/0 conn(0x7fc7280d0650 0x7fc7280d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.577+0000 7fc72f800640 1 --2- 172.21.15.159:0/375523193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7280af830 0x7fc728073790 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.577+0000 7fc72f800640 1 -- 172.21.15.159:0/375523193 >> 172.21.15.159:0/375523193 conn(0x7fc72806b990 msgr2=0x7fc72806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:43.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.577+0000 7fc72f800640 1 -- 172.21.15.159:0/375523193 shutdown_connections 2024-04-04T06:09:43.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.577+0000 7fc72f800640 1 -- 172.21.15.159:0/375523193 wait complete. 2024-04-04T06:09:43.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.578+0000 7fc72f800640 1 Processor -- start 2024-04-04T06:09:43.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.579+0000 7fc72f800640 1 -- start start 2024-04-04T06:09:43.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.579+0000 7fc72f800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7280af830 0x7fc7280b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.579+0000 7fc72f800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7280d0650 0x7fc7281f4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.580+0000 7fc72f800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc72815c390 0x7fc72813ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.580+0000 7fc72f800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc728160120 con 0x7fc7280af830 2024-04-04T06:09:43.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.580+0000 7fc72f800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc7280ad6e0 con 0x7fc7280d0650 2024-04-04T06:09:43.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.580+0000 7fc72f800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc728160d50 con 0x7fc72815c390 2024-04-04T06:09:43.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.580+0000 7fc72ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7280af830 0x7fc7280b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.580+0000 7fc72ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7280af830 0x7fc7280b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38728/0 (socket says 172.21.15.159:38728) 2024-04-04T06:09:43.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.580+0000 7fc72ce00640 1 -- 172.21.15.159:0/3006979998 learned_addr learned my addr 172.21.15.159:0/3006979998 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:43.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.580+0000 7fc727e00640 1 --2- 172.21.15.159:0/3006979998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7280d0650 0x7fc7281f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.580+0000 7fc72d800640 1 --2- 172.21.15.159:0/3006979998 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc72815c390 0x7fc72813ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.580+0000 7fc72ce00640 1 -- 172.21.15.159:0/3006979998 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc72815c390 msgr2=0x7fc72813ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:43.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.580+0000 7fc72ce00640 1 --2- 172.21.15.159:0/3006979998 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc72815c390 0x7fc72813ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.580+0000 7fc72ce00640 1 -- 172.21.15.159:0/3006979998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7280d0650 msgr2=0x7fc7281f4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:43.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.580+0000 7fc72ce00640 1 --2- 172.21.15.159:0/3006979998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7280d0650 0x7fc7281f4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.580+0000 7fc72ce00640 1 -- 172.21.15.159:0/3006979998 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7280b37d0 con 0x7fc7280af830 2024-04-04T06:09:43.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.581+0000 7fc72ce00640 1 --2- 172.21.15.159:0/3006979998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7280af830 0x7fc7280b2f10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00c430 tx=0x7fc71c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:43.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.581+0000 7fc724c00640 1 -- 172.21.15.159:0/3006979998 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c011790 con 0x7fc7280af830 2024-04-04T06:09:43.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.581+0000 7fc724c00640 1 -- 172.21.15.159:0/3006979998 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc71c017070 con 0x7fc7280af830 2024-04-04T06:09:43.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.581+0000 7fc724c00640 1 -- 172.21.15.159:0/3006979998 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c011b70 con 0x7fc7280af830 2024-04-04T06:09:43.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.581+0000 7fc72f800640 1 -- 172.21.15.159:0/3006979998 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7280b3ab0 con 0x7fc7280af830 2024-04-04T06:09:43.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.581+0000 7fc72f800640 1 -- 172.21.15.159:0/3006979998 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7280b4070 con 0x7fc7280af830 2024-04-04T06:09:43.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.582+0000 7fc72f800640 1 -- 172.21.15.159:0/3006979998 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc728141550 con 0x7fc7280af830 2024-04-04T06:09:43.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.583+0000 7fc724c00640 1 -- 172.21.15.159:0/3006979998 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc71c011d10 con 0x7fc7280af830 2024-04-04T06:09:43.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.584+0000 7fc724c00640 1 --2- 172.21.15.159:0/3006979998 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc70806c8f0 0x7fc70806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:43.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.584+0000 7fc724c00640 1 -- 172.21.15.159:0/3006979998 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fc71c08bb50 con 0x7fc7280af830 2024-04-04T06:09:43.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.585+0000 7fc727e00640 1 --2- 172.21.15.159:0/3006979998 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc70806c8f0 0x7fc70806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:43.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.586+0000 7fc727e00640 1 --2- 172.21.15.159:0/3006979998 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc70806c8f0 0x7fc70806edb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc714036000 tx=0x7fc714008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:43.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.589+0000 7fc724c00640 1 -- 172.21.15.159:0/3006979998 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc71c059e50 con 0x7fc7280af830 2024-04-04T06:09:43.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.828+0000 7fc72f800640 1 -- 172.21.15.159:0/3006979998 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7280685c0 con 0x7fc7280af830 2024-04-04T06:09:43.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.830+0000 7fc724c00640 1 -- 172.21.15.159:0/3006979998 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5187 (secure 0 0 0) 0x7fc71c059750 con 0x7fc7280af830 2024-04-04T06:09:43.823 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:09:43.823 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5191,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3507245876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3507245876},{"type":"v1","addr":"172.21.15.159:6835","nonce":3507245876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/967262871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":967262871},{"type":"v1","addr":"172.21.15.178:6835","nonce":967262871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2969351496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2969351496},{"type":"v1","addr":"172.21.15.178:6837","nonce":2969351496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:08:57.020444+0000","modified":"2024-04-04T06:09:17.627558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6039":{"gid":6039,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":8,"addr":"172.21.15.159:6834/962475038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":962475038},{"type":"v1","addr":"172.21.15.159:6834","nonce":962475038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6039,"qdb_cluster":[6039]},"id":4}]} 2024-04-04T06:09:43.824 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 34 2024-04-04T06:09:43.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.832+0000 7fc72f800640 1 -- 172.21.15.159:0/3006979998 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc70806c8f0 msgr2=0x7fc70806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:43.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.832+0000 7fc72f800640 1 --2- 172.21.15.159:0/3006979998 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc70806c8f0 0x7fc70806edb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc714036000 tx=0x7fc714008030 comp rx=0 tx=0).stop 2024-04-04T06:09:43.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.832+0000 7fc72f800640 1 -- 172.21.15.159:0/3006979998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7280af830 msgr2=0x7fc7280b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:43.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.832+0000 7fc72f800640 1 --2- 172.21.15.159:0/3006979998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7280af830 0x7fc7280b2f10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00c430 tx=0x7fc71c00c8f0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.833+0000 7fc72f800640 1 -- 172.21.15.159:0/3006979998 shutdown_connections 2024-04-04T06:09:43.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.833+0000 7fc72f800640 1 --2- 172.21.15.159:0/3006979998 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc72815c390 0x7fc72813ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.833+0000 7fc72f800640 1 --2- 172.21.15.159:0/3006979998 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc70806c8f0 0x7fc70806edb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.833+0000 7fc72f800640 1 --2- 172.21.15.159:0/3006979998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7280d0650 0x7fc7281f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.833+0000 7fc72f800640 1 --2- 172.21.15.159:0/3006979998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7280af830 0x7fc7280b2f10 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:43.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.833+0000 7fc72f800640 1 -- 172.21.15.159:0/3006979998 >> 172.21.15.159:0/3006979998 conn(0x7fc72806b990 msgr2=0x7fc728155580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:43.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.833+0000 7fc72f800640 1 -- 172.21.15.159:0/3006979998 shutdown_connections 2024-04-04T06:09:43.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:43.833+0000 7fc72f800640 1 -- 172.21.15.159:0/3006979998 wait complete. 2024-04-04T06:09:43.845 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-04T06:09:44.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.058+0000 7f532fe00640 1 Processor -- start 2024-04-04T06:09:44.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.059+0000 7f532fe00640 1 -- start start 2024-04-04T06:09:44.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.060+0000 7f532fe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53300d37b0 0x7f53300b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.060+0000 7f532fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f53300b67f0 0x7f53300b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.060+0000 7f532fe00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53300d2710 con 0x7f533015e4c0 2024-04-04T06:09:44.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.060+0000 7f532fe00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f533015e8a0 con 0x7f53300b67f0 2024-04-04T06:09:44.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.060+0000 7f532fe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f53300d4040 con 0x7f53300d37b0 2024-04-04T06:09:44.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.060+0000 7f532f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f53300b67f0 0x7f53300b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.060+0000 7f532f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f53300b67f0 0x7f53300b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44888/0 (socket says 172.21.15.159:44888) 2024-04-04T06:09:44.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.060+0000 7f532f400640 1 -- 172.21.15.159:0/3613809799 learned_addr learned my addr 172.21.15.159:0/3613809799 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:44.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.061+0000 7f532e000640 1 --2- 172.21.15.159:0/3613809799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53300d37b0 0x7f53300b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.061+0000 7f532f400640 1 -- 172.21.15.159:0/3613809799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53300d37b0 msgr2=0x7f53300b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.061+0000 7f532f400640 1 --2- 172.21.15.159:0/3613809799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53300d37b0 0x7f53300b62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.061+0000 7f532f400640 1 -- 172.21.15.159:0/3613809799 >> v1:172.21.15.178:6789/0 conn(0x7f533015e4c0 legacy=0x7f53300b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.061+0000 7f532f400640 1 -- 172.21.15.159:0/3613809799 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53300d29f0 con 0x7f53300b67f0 2024-04-04T06:09:44.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.061+0000 7f532e000640 1 --2- 172.21.15.159:0/3613809799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53300d37b0 0x7f53300b62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:44.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.062+0000 7f532f400640 1 --2- 172.21.15.159:0/3613809799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53300b67f0 0x7f53300b7400 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5328009e50 tx=0x7f532802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b57119b7558158 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:44.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.062+0000 7f532d600640 1 -- 172.21.15.159:0/3613809799 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532802ca70 con 0x7f53300b67f0 2024-04-04T06:09:44.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.062+0000 7f532d600640 1 -- 172.21.15.159:0/3613809799 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f532802cc10 con 0x7f53300b67f0 2024-04-04T06:09:44.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.062+0000 7f532d600640 1 -- 172.21.15.159:0/3613809799 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53280343f0 con 0x7f53300b67f0 2024-04-04T06:09:44.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.063+0000 7f532fe00640 1 -- 172.21.15.159:0/3613809799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53300b67f0 msgr2=0x7f53300b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.063+0000 7f532fe00640 1 --2- 172.21.15.159:0/3613809799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53300b67f0 0x7f53300b7400 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5328009e50 tx=0x7f532802bed0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.063+0000 7f532fe00640 1 -- 172.21.15.159:0/3613809799 shutdown_connections 2024-04-04T06:09:44.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.063+0000 7f532fe00640 1 --2- 172.21.15.159:0/3613809799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53300b67f0 0x7f53300b7400 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.063+0000 7f532fe00640 1 --2- 172.21.15.159:0/3613809799 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53300d37b0 0x7f53300b62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.063+0000 7f532fe00640 1 -- 172.21.15.159:0/3613809799 >> 172.21.15.159:0/3613809799 conn(0x7f53300676a0 msgr2=0x7f5330067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:44.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.063+0000 7f532fe00640 1 -- 172.21.15.159:0/3613809799 shutdown_connections 2024-04-04T06:09:44.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.063+0000 7f532fe00640 1 -- 172.21.15.159:0/3613809799 wait complete. 2024-04-04T06:09:44.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.065+0000 7f532fe00640 1 Processor -- start 2024-04-04T06:09:44.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.065+0000 7f532fe00640 1 -- start start 2024-04-04T06:09:44.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.066+0000 7f532fe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53300b67f0 0x7f53301f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.066+0000 7f532fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53300d37b0 0x7f5330155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.066+0000 7f532fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f533015e4c0 0x7f53300b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.066+0000 7f532fe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f53300d2fb0 con 0x7f533015e4c0 2024-04-04T06:09:44.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.066+0000 7f532fe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f53300d3320 con 0x7f53300d37b0 2024-04-04T06:09:44.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.066+0000 7f532fe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f53300d2350 con 0x7f53300b67f0 2024-04-04T06:09:44.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.066+0000 7f532f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f533015e4c0 0x7f53300b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.066+0000 7f532f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f533015e4c0 0x7f53300b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44900/0 (socket says 172.21.15.159:44900) 2024-04-04T06:09:44.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.066+0000 7f532f400640 1 -- 172.21.15.159:0/934752639 learned_addr learned my addr 172.21.15.159:0/934752639 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:44.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.067+0000 7f532ea00640 1 --2- 172.21.15.159:0/934752639 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53300b67f0 0x7f53301f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.067+0000 7f532e000640 1 --2- 172.21.15.159:0/934752639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53300d37b0 0x7f5330155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.067+0000 7f532f400640 1 -- 172.21.15.159:0/934752639 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53300b67f0 msgr2=0x7f53301f6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.067+0000 7f532f400640 1 --2- 172.21.15.159:0/934752639 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53300b67f0 0x7f53301f6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.067+0000 7f532f400640 1 -- 172.21.15.159:0/934752639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53300d37b0 msgr2=0x7f5330155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.067+0000 7f532f400640 1 --2- 172.21.15.159:0/934752639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53300d37b0 0x7f5330155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.067+0000 7f532f400640 1 -- 172.21.15.159:0/934752639 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5330155f50 con 0x7f533015e4c0 2024-04-04T06:09:44.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.067+0000 7f532f400640 1 --2- 172.21.15.159:0/934752639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f533015e4c0 0x7f53300b48a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5328009e50 tx=0x7f5328004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:44.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.067+0000 7f5326a00640 1 -- 172.21.15.159:0/934752639 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328043020 con 0x7f533015e4c0 2024-04-04T06:09:44.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.067+0000 7f5326a00640 1 -- 172.21.15.159:0/934752639 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f532803e450 con 0x7f533015e4c0 2024-04-04T06:09:44.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.067+0000 7f5326a00640 1 -- 172.21.15.159:0/934752639 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328009510 con 0x7f533015e4c0 2024-04-04T06:09:44.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.067+0000 7f532fe00640 1 -- 172.21.15.159:0/934752639 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f533015ec70 con 0x7f533015e4c0 2024-04-04T06:09:44.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.068+0000 7f532fe00640 1 -- 172.21.15.159:0/934752639 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53301f7900 con 0x7f533015e4c0 2024-04-04T06:09:44.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.070+0000 7f5326a00640 1 -- 172.21.15.159:0/934752639 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5328046020 con 0x7f533015e4c0 2024-04-04T06:09:44.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.071+0000 7f5326a00640 1 --2- 172.21.15.159:0/934752639 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f531006c8f0 0x7f531006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.071+0000 7f5326a00640 1 -- 172.21.15.159:0/934752639 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f53280ae450 con 0x7f533015e4c0 2024-04-04T06:09:44.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.071+0000 7f533877a740 1 Processor -- start 2024-04-04T06:09:44.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.071+0000 7f532ea00640 1 --2- 172.21.15.159:0/934752639 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f531006c8f0 0x7f531006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.071+0000 7f533877a740 1 -- start start 2024-04-04T06:09:44.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.072+0000 7f533877a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c325bd8c0 0x560c325bdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.072+0000 7f533877a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c3258d8e0 0x560c325bab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.072+0000 7f533877a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c32591440 0x560c325bb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.072+0000 7f533877a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560c324f75c0 con 0x560c325bd8c0 2024-04-04T06:09:44.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.072+0000 7f533877a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560c3249f3b0 con 0x560c3258d8e0 2024-04-04T06:09:44.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.072+0000 7f533877a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x560c324f3570 con 0x560c32591440 2024-04-04T06:09:44.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.073+0000 7f532ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c325bd8c0 0x560c325bdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.073+0000 7f532ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c325bd8c0 0x560c325bdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44904/0 (socket says 172.21.15.159:44904) 2024-04-04T06:09:44.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.073+0000 7f532ea00640 1 -- 172.21.15.159:0/2730149305 learned_addr learned my addr 172.21.15.159:0/2730149305 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:44.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.073+0000 7f532f400640 1 --2- 172.21.15.159:0/2730149305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c32591440 0x560c325bb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.073+0000 7f532e000640 1 --2- 172.21.15.159:0/2730149305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c3258d8e0 0x560c325bab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.073+0000 7f532ea00640 1 -- 172.21.15.159:0/2730149305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c32591440 msgr2=0x560c325bb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.073+0000 7f532ea00640 1 --2- 172.21.15.159:0/2730149305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c32591440 0x560c325bb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.073+0000 7f532ea00640 1 -- 172.21.15.159:0/2730149305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c3258d8e0 msgr2=0x560c325bab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.073+0000 7f532ea00640 1 --2- 172.21.15.159:0/2730149305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c3258d8e0 0x560c325bab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.073+0000 7f532ea00640 1 -- 172.21.15.159:0/2730149305 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c322d77c0 con 0x560c325bd8c0 2024-04-04T06:09:44.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.073+0000 7f532ea00640 1 --2- 172.21.15.159:0/934752639 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f531006c8f0 0x7f531006edb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f532000a750 tx=0x7f5320008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:44.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.073+0000 7f532ea00640 1 --2- 172.21.15.159:0/2730149305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c325bd8c0 0x560c325bdca0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f532000c850 tx=0x7f5320017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7e34115b23962 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:44.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.074+0000 7f531b400640 1 -- 172.21.15.159:0/2730149305 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320019040 con 0x560c325bd8c0 2024-04-04T06:09:44.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.074+0000 7f531b400640 1 -- 172.21.15.159:0/2730149305 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5320022070 con 0x560c325bd8c0 2024-04-04T06:09:44.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.074+0000 7f531b400640 1 -- 172.21.15.159:0/2730149305 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320015d60 con 0x560c325bd8c0 2024-04-04T06:09:44.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.074+0000 7f533877a740 1 -- 172.21.15.159:0/2730149305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c325bd8c0 msgr2=0x560c325bdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.074+0000 7f533877a740 1 --2- 172.21.15.159:0/2730149305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c325bd8c0 0x560c325bdca0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f532000c850 tx=0x7f5320017000 comp rx=0 tx=0).stop 2024-04-04T06:09:44.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.074+0000 7f533877a740 1 -- 172.21.15.159:0/2730149305 shutdown_connections 2024-04-04T06:09:44.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.074+0000 7f533877a740 1 --2- 172.21.15.159:0/2730149305 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c32591440 0x560c325bb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.074+0000 7f533877a740 1 --2- 172.21.15.159:0/2730149305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c3258d8e0 0x560c325bab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.074+0000 7f533877a740 1 --2- 172.21.15.159:0/2730149305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c325bd8c0 0x560c325bdca0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.074+0000 7f533877a740 1 -- 172.21.15.159:0/2730149305 >> 172.21.15.159:0/2730149305 conn(0x560c3258fb50 msgr2=0x560c325d86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:44.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.074+0000 7f533877a740 1 -- 172.21.15.159:0/2730149305 shutdown_connections 2024-04-04T06:09:44.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.074+0000 7f533877a740 1 -- 172.21.15.159:0/2730149305 wait complete. 2024-04-04T06:09:44.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.075+0000 7f533877a740 1 -- start start 2024-04-04T06:09:44.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.075+0000 7f533877a740 1 Processor -- start 2024-04-04T06:09:44.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.076+0000 7f533877a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c32591440 0x560c325af840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.076+0000 7f533877a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c325bd8c0 0x560c325c0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.076+0000 7f533877a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c325c0fa0 0x560c325b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.076+0000 7f533877a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560c325925f0 con 0x560c325bd8c0 2024-04-04T06:09:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.076+0000 7f533877a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560c324b27a0 con 0x560c32591440 2024-04-04T06:09:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.076+0000 7f533877a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x560c325919e0 con 0x560c325c0fa0 2024-04-04T06:09:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.076+0000 7f533877a740 10 client.0 _ll_register_callbacks cb 0x560c324eefd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:09:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.077+0000 7f532e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c325bd8c0 0x560c325c0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.077+0000 7f532e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c325bd8c0 0x560c325c0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44906/0 (socket says 172.21.15.159:44906) 2024-04-04T06:09:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.077+0000 7f532e000640 1 -- 172.21.15.159:0/3675287497 learned_addr learned my addr 172.21.15.159:0/3675287497 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.077+0000 7f532ea00640 1 --2- 172.21.15.159:0/3675287497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c32591440 0x560c325af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.077+0000 7f532e000640 1 -- 172.21.15.159:0/3675287497 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c325c0fa0 msgr2=0x560c325b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.077+0000 7f532e000640 1 --2- 172.21.15.159:0/3675287497 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c325c0fa0 0x560c325b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.077+0000 7f532e000640 1 -- 172.21.15.159:0/3675287497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c32591440 msgr2=0x560c325af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.077+0000 7f532e000640 1 --2- 172.21.15.159:0/3675287497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c32591440 0x560c325af840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.077+0000 7f532e000640 1 -- 172.21.15.159:0/3675287497 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c32337f50 con 0x560c325bd8c0 2024-04-04T06:09:44.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.077+0000 7f532e000640 1 --2- 172.21.15.159:0/3675287497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c325bd8c0 0x560c325c0a60 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f531c00e2e0 tx=0x7f531c00e7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6160153050a22e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:44.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.077+0000 7f530fe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:09:44.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.077+0000 7f530fe00640 1 -- 172.21.15.159:0/3675287497 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c012840 con 0x560c325bd8c0 2024-04-04T06:09:44.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.078+0000 7f530fe00640 1 -- 172.21.15.159:0/3675287497 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f531c0129e0 con 0x560c325bd8c0 2024-04-04T06:09:44.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.078+0000 7f533877a740 1 -- 172.21.15.159:0/3675287497 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5320022070 con 0x560c325bd8c0 2024-04-04T06:09:44.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.078+0000 7f530fe00640 1 -- 172.21.15.159:0/3675287497 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f531c022050 con 0x560c325bd8c0 2024-04-04T06:09:44.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.078+0000 7f533877a740 10 client.6321 fetch_fsmap learned FSMap version 34 2024-04-04T06:09:44.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.078+0000 7f533877a740 1 -- 172.21.15.159:0/3675287497 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x560c321cb0b0 con 0x560c325bd8c0 2024-04-04T06:09:44.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.078+0000 7f530fe00640 1 -- 172.21.15.159:0/3675287497 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5320022070 con 0x560c325bd8c0 2024-04-04T06:09:44.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.078+0000 7f530fe00640 1 -- 172.21.15.159:0/3675287497 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f531c025020 con 0x560c325bd8c0 2024-04-04T06:09:44.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.078+0000 7f533877a740 10 client.6321 fetch_fsmap finished waiting for FSMap version 34 2024-04-04T06:09:44.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.078+0000 7f533877a740 10 client.6321 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-04T06:09:44.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.078+0000 7f533877a740 20 client.6321 populate_metadata read hostname 'smithi159' 2024-04-04T06:09:44.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.078+0000 7f533877a740 1 --2- 172.21.15.159:0/3675287497 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x560c325ec6e0 0x560c325b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.078+0000 7f533877a740 4 client.6321 mds_command: new command op to 6039 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:09:44.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.078+0000 7f533877a740 1 -- 172.21.15.159:0/3675287497 --> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c324cb890 con 0x560c325ec6e0 2024-04-04T06:09:44.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.079+0000 7f532ea00640 1 --2- 172.21.15.159:0/3675287497 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x560c325ec6e0 0x560c325b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.079+0000 7f532ea00640 1 --2- 172.21.15.159:0/3675287497 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x560c325ec6e0 0x560c325b5070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=556992cb36a3ef89 server_cookie=f94348d9fbbe6ed5 in_seq=0 out_seq=0 2024-04-04T06:09:44.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.079+0000 7f530fe00640 10 client.6321 ms_handle_connect on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:44.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.080+0000 7f530fe00640 1 -- 172.21.15.159:0/3675287497 <== mds.0 v2:172.21.15.159:6833/962475038 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560c324cb890 con 0x560c325ec6e0 2024-04-04T06:09:44.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.080+0000 7f530fe00640 10 client.6321 handle_command_reply: tid=0 2024-04-04T06:09:44.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.080+0000 7f533877a740 1 client.6321 shutdown 2024-04-04T06:09:44.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.080+0000 7f533877a740 1 -- 172.21.15.159:0/3675287497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c325bd8c0 msgr2=0x560c325c0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.080+0000 7f533877a740 1 --2- 172.21.15.159:0/3675287497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c325bd8c0 0x560c325c0a60 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f531c00e2e0 tx=0x7f531c00e7a0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.080+0000 7f533877a740 1 -- 172.21.15.159:0/3675287497 shutdown_connections 2024-04-04T06:09:44.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.080+0000 7f533877a740 1 --2- 172.21.15.159:0/3675287497 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c325c0fa0 0x560c325b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.080+0000 7f533877a740 1 --2- 172.21.15.159:0/3675287497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c325bd8c0 0x560c325c0a60 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.080+0000 7f533877a740 1 --2- 172.21.15.159:0/3675287497 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x560c325ec6e0 0x560c325b5070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.080+0000 7f533877a740 1 --2- 172.21.15.159:0/3675287497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c32591440 0x560c325af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.080+0000 7f533877a740 1 -- 172.21.15.159:0/3675287497 >> 172.21.15.159:0/3675287497 conn(0x560c3258d8e0 msgr2=0x560c325c16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:44.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.080+0000 7f530fe00640 0 client.6321 ms_handle_reset on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:44.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.080+0000 7f533877a740 1 -- 172.21.15.159:0/3675287497 shutdown_connections 2024-04-04T06:09:44.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.080+0000 7f533877a740 1 -- 172.21.15.159:0/3675287497 wait complete. 2024-04-04T06:09:44.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.081+0000 7f533877a740 20 client.6321 trim_cache size 0 max 16384 2024-04-04T06:09:44.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.095+0000 7f533877a740 1 Processor -- start 2024-04-04T06:09:44.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.095+0000 7f533877a740 1 -- start start 2024-04-04T06:09:44.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.096+0000 7f533877a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c32591440 0x560c325b5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.096+0000 7f533877a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c325bd8c0 0x560c325b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.096+0000 7f533877a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c325c0fa0 0x560c325e8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.096+0000 7f533877a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f53301ffa20 con 0x560c32591440 2024-04-04T06:09:44.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.096+0000 7f533877a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5330158610 con 0x560c325c0fa0 2024-04-04T06:09:44.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.096+0000 7f533877a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f53301ff3d0 con 0x560c325bd8c0 2024-04-04T06:09:44.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.096+0000 7f532ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c32591440 0x560c325b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.097+0000 7f532ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c32591440 0x560c325b5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44908/0 (socket says 172.21.15.159:44908) 2024-04-04T06:09:44.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.097+0000 7f532ea00640 1 -- 172.21.15.159:0/2283303938 learned_addr learned my addr 172.21.15.159:0/2283303938 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:44.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.097+0000 7f532f400640 1 --2- 172.21.15.159:0/2283303938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c325c0fa0 0x560c325e8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.097+0000 7f532e000640 1 --2- 172.21.15.159:0/2283303938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c325bd8c0 0x560c325b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.097+0000 7f532ea00640 1 -- 172.21.15.159:0/2283303938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c325bd8c0 msgr2=0x560c325b77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.097+0000 7f532ea00640 1 --2- 172.21.15.159:0/2283303938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c325bd8c0 0x560c325b77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.097+0000 7f532ea00640 1 -- 172.21.15.159:0/2283303938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c325c0fa0 msgr2=0x560c325e8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.097+0000 7f532ea00640 1 --2- 172.21.15.159:0/2283303938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c325c0fa0 0x560c325e8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.097+0000 7f532ea00640 1 -- 172.21.15.159:0/2283303938 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c325ece00 con 0x560c32591440 2024-04-04T06:09:44.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.097+0000 7f532f400640 1 --2- 172.21.15.159:0/2283303938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c325c0fa0 0x560c325e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:44.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.097+0000 7f532ea00640 1 --2- 172.21.15.159:0/2283303938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c32591440 0x560c325b5750 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5320016000 tx=0x7f5320015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b825908a751f7c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:44.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.098+0000 7f531aa00640 1 -- 172.21.15.159:0/2283303938 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320019040 con 0x560c32591440 2024-04-04T06:09:44.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.098+0000 7f531aa00640 1 -- 172.21.15.159:0/2283303938 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5320036070 con 0x560c32591440 2024-04-04T06:09:44.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.098+0000 7f531aa00640 1 -- 172.21.15.159:0/2283303938 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53200158a0 con 0x560c32591440 2024-04-04T06:09:44.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.098+0000 7f533877a740 1 -- 172.21.15.159:0/2283303938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c32591440 msgr2=0x560c325b5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.098+0000 7f533877a740 1 --2- 172.21.15.159:0/2283303938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c32591440 0x560c325b5750 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5320016000 tx=0x7f5320015000 comp rx=0 tx=0).stop 2024-04-04T06:09:44.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.098+0000 7f533877a740 1 -- 172.21.15.159:0/2283303938 shutdown_connections 2024-04-04T06:09:44.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.098+0000 7f533877a740 1 --2- 172.21.15.159:0/2283303938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c325c0fa0 0x560c325e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.098+0000 7f533877a740 1 --2- 172.21.15.159:0/2283303938 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c325bd8c0 0x560c325b77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.098+0000 7f533877a740 1 --2- 172.21.15.159:0/2283303938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c32591440 0x560c325b5750 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.098+0000 7f533877a740 1 -- 172.21.15.159:0/2283303938 >> 172.21.15.159:0/2283303938 conn(0x560c3258d8e0 msgr2=0x560c325af650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:44.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.098+0000 7f533877a740 1 -- 172.21.15.159:0/2283303938 shutdown_connections 2024-04-04T06:09:44.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.098+0000 7f533877a740 1 -- 172.21.15.159:0/2283303938 wait complete. 2024-04-04T06:09:44.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.099+0000 7f533877a740 1 -- start start 2024-04-04T06:09:44.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.100+0000 7f533877a740 1 Processor -- start 2024-04-04T06:09:44.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.101+0000 7f533877a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c32591440 0x560c326560a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.101+0000 7f533877a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c325bd8c0 0x560c325eddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.101+0000 7f533877a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c325c0fa0 0x560c32591f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.101+0000 7f533877a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560c325ec560 con 0x560c32591440 2024-04-04T06:09:44.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.101+0000 7f533877a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560c325b4270 con 0x560c325c0fa0 2024-04-04T06:09:44.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.101+0000 7f533877a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x560c325ecbb0 con 0x560c325bd8c0 2024-04-04T06:09:44.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.101+0000 7f533877a740 10 client.0 _ll_register_callbacks cb 0x7f531c00c1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:09:44.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.102+0000 7f532ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c32591440 0x560c326560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.102+0000 7f532ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c32591440 0x560c326560a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44910/0 (socket says 172.21.15.159:44910) 2024-04-04T06:09:44.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.102+0000 7f532ea00640 1 -- 172.21.15.159:0/51504154 learned_addr learned my addr 172.21.15.159:0/51504154 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:44.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.102+0000 7f532ea00640 1 -- 172.21.15.159:0/51504154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c325bd8c0 msgr2=0x560c325eddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.102+0000 7f532ea00640 1 --2- 172.21.15.159:0/51504154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c325bd8c0 0x560c325eddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.102+0000 7f532ea00640 1 -- 172.21.15.159:0/51504154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c325c0fa0 msgr2=0x560c32591f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.102+0000 7f532ea00640 1 --2- 172.21.15.159:0/51504154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c325c0fa0 0x560c32591f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.102+0000 7f532ea00640 1 -- 172.21.15.159:0/51504154 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c325b2ec0 con 0x560c32591440 2024-04-04T06:09:44.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.102+0000 7f532ea00640 1 --2- 172.21.15.159:0/51504154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c32591440 0x560c326560a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5320006ca0 tx=0x7f5320004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231f904d4df7c495 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:44.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.102+0000 7f530fe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:09:44.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.103+0000 7f530fe00640 1 -- 172.21.15.159:0/51504154 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320019030 con 0x560c32591440 2024-04-04T06:09:44.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.103+0000 7f530fe00640 1 -- 172.21.15.159:0/51504154 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f5320005b10 con 0x560c32591440 2024-04-04T06:09:44.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.103+0000 7f533877a740 1 -- 172.21.15.159:0/51504154 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5320036070 con 0x560c32591440 2024-04-04T06:09:44.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.103+0000 7f530fe00640 1 -- 172.21.15.159:0/51504154 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5320005d10 con 0x560c32591440 2024-04-04T06:09:44.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.103+0000 7f533877a740 10 client.6327 fetch_fsmap learned FSMap version 34 2024-04-04T06:09:44.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.103+0000 7f533877a740 1 -- 172.21.15.159:0/51504154 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x560c325fcdb0 con 0x560c32591440 2024-04-04T06:09:44.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.103+0000 7f530fe00640 1 -- 172.21.15.159:0/51504154 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5320036070 con 0x560c32591440 2024-04-04T06:09:44.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.103+0000 7f530fe00640 1 -- 172.21.15.159:0/51504154 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f532001c050 con 0x560c32591440 2024-04-04T06:09:44.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.103+0000 7f533877a740 10 client.6327 fetch_fsmap finished waiting for FSMap version 34 2024-04-04T06:09:44.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.103+0000 7f533877a740 10 client.6327 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-04T06:09:44.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.103+0000 7f533877a740 20 client.6327 populate_metadata read hostname 'smithi159' 2024-04-04T06:09:44.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.103+0000 7f533877a740 1 --2- 172.21.15.159:0/51504154 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x560c325eb3c0 0x560c325bdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.104+0000 7f533877a740 4 client.6327 mds_command: new command op to 6039 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-04T06:09:44.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.104+0000 7f533877a740 1 -- 172.21.15.159:0/51504154 --> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x560c32504b00 con 0x560c325eb3c0 2024-04-04T06:09:44.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.104+0000 7f532e000640 1 --2- 172.21.15.159:0/51504154 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x560c325eb3c0 0x560c325bdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.104+0000 7f532e000640 1 --2- 172.21.15.159:0/51504154 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x560c325eb3c0 0x560c325bdca0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38d4f67267ac9231 server_cookie=e31b94e0c32ba57c in_seq=0 out_seq=0 2024-04-04T06:09:44.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.104+0000 7f530fe00640 10 client.6327 ms_handle_connect on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:44.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.105+0000 7f530fe00640 1 -- 172.21.15.159:0/51504154 <== mds.0 v2:172.21.15.159:6833/962475038 1 ==== command_reply(tid 0: 0 ) ==== 8+0+141 (crc 0 0 0) 0x560c32504b00 con 0x560c325eb3c0 2024-04-04T06:09:44.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.105+0000 7f530fe00640 10 client.6327 handle_command_reply: tid=0 2024-04-04T06:09:44.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.105+0000 7f533877a740 1 client.6327 shutdown 2024-04-04T06:09:44.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.105+0000 7f533877a740 1 -- 172.21.15.159:0/51504154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c32591440 msgr2=0x560c326560a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.105+0000 7f533877a740 1 --2- 172.21.15.159:0/51504154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c32591440 0x560c326560a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5320006ca0 tx=0x7f5320004680 comp rx=0 tx=0).stop 2024-04-04T06:09:44.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.105+0000 7f533877a740 1 -- 172.21.15.159:0/51504154 shutdown_connections 2024-04-04T06:09:44.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.105+0000 7f533877a740 1 --2- 172.21.15.159:0/51504154 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x560c325eb3c0 0x560c325bdca0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.105+0000 7f533877a740 1 --2- 172.21.15.159:0/51504154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560c325c0fa0 0x560c32591f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.105+0000 7f533877a740 1 --2- 172.21.15.159:0/51504154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560c325bd8c0 0x560c325eddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.105+0000 7f533877a740 1 --2- 172.21.15.159:0/51504154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c32591440 0x560c326560a0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.105+0000 7f530fe00640 0 client.6327 ms_handle_reset on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:44.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.105+0000 7f533877a740 1 -- 172.21.15.159:0/51504154 >> 172.21.15.159:0/51504154 conn(0x560c3258d8e0 msgr2=0x560c325e8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:44.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.105+0000 7f533877a740 1 -- 172.21.15.159:0/51504154 shutdown_connections 2024-04-04T06:09:44.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.105+0000 7f533877a740 1 -- 172.21.15.159:0/51504154 wait complete. 2024-04-04T06:09:44.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.105+0000 7f533877a740 20 client.6327 trim_cache size 0 max 16384 2024-04-04T06:09:44.106 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:09:44.106 INFO:teuthology.orchestra.run.smithi159.stdout:[ 2024-04-04T06:09:44.106 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:09:44.106 INFO:teuthology.orchestra.run.smithi159.stdout: "damage_type": "backtrace", 2024-04-04T06:09:44.106 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 2708464566, 2024-04-04T06:09:44.106 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 1099511627777, 2024-04-04T06:09:44.106 INFO:teuthology.orchestra.run.smithi159.stdout: "path": "/dir_test/file" 2024-04-04T06:09:44.107 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:09:44.107 INFO:teuthology.orchestra.run.smithi159.stdout:] 2024-04-04T06:09:44.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.106+0000 7f532fe00640 1 -- 172.21.15.159:0/934752639 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f531006c8f0 msgr2=0x7f531006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.106+0000 7f532fe00640 1 --2- 172.21.15.159:0/934752639 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f531006c8f0 0x7f531006edb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f532000a750 tx=0x7f5320008040 comp rx=0 tx=0).stop 2024-04-04T06:09:44.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.106+0000 7f532fe00640 1 -- 172.21.15.159:0/934752639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f533015e4c0 msgr2=0x7f53300b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.106+0000 7f532fe00640 1 --2- 172.21.15.159:0/934752639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f533015e4c0 0x7f53300b48a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5328009e50 tx=0x7f5328004a40 comp rx=0 tx=0).stop 2024-04-04T06:09:44.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.106+0000 7f532fe00640 1 -- 172.21.15.159:0/934752639 shutdown_connections 2024-04-04T06:09:44.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.106+0000 7f532fe00640 1 --2- 172.21.15.159:0/934752639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f533015e4c0 0x7f53300b48a0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.106+0000 7f532fe00640 1 --2- 172.21.15.159:0/934752639 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f531006c8f0 0x7f531006edb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.106+0000 7f532fe00640 1 --2- 172.21.15.159:0/934752639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53300d37b0 0x7f5330155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.106+0000 7f532fe00640 1 --2- 172.21.15.159:0/934752639 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53300b67f0 0x7f53301f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.106+0000 7f532fe00640 1 -- 172.21.15.159:0/934752639 >> 172.21.15.159:0/934752639 conn(0x7f53300676a0 msgr2=0x7f533015cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:44.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.106+0000 7f532fe00640 1 -- 172.21.15.159:0/934752639 shutdown_connections 2024-04-04T06:09:44.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.106+0000 7f532fe00640 1 -- 172.21.15.159:0/934752639 wait complete. 2024-04-04T06:09:44.116 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-04T06:09:44.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.310+0000 7ff10a800640 1 Processor -- start 2024-04-04T06:09:44.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.311+0000 7ff10a800640 1 -- start start 2024-04-04T06:09:44.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.312+0000 7ff10a800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff104153580 0x7ff1040d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.312+0000 7ff10a800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff1040d07c0 0x7ff1040aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.312+0000 7ff10a800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1040af2e0 0x7ff10406d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.312+0000 7ff10a800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7ff1040d0c30 con 0x7ff1040d07c0 2024-04-04T06:09:44.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.312+0000 7ff10a800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff1040afab0 con 0x7ff104153580 2024-04-04T06:09:44.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.312+0000 7ff10a800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff10415cc80 con 0x7ff1040af2e0 2024-04-04T06:09:44.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.313+0000 7ff102a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff1040d07c0 0x7ff1040aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.313+0000 7ff102a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff1040d07c0 0x7ff1040aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44920/0 (socket says 172.21.15.159:44920) 2024-04-04T06:09:44.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.313+0000 7ff102a00640 1 -- 172.21.15.159:0/897722395 learned_addr learned my addr 172.21.15.159:0/897722395 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:44.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.313+0000 7ff103400640 1 --2- 172.21.15.159:0/897722395 >> v2:172.21.15.178:3300/0 conn(0x7ff104153580 0x7ff1040d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.313+0000 7ff103e00640 1 --2- 172.21.15.159:0/897722395 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1040af2e0 0x7ff10406d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.313+0000 7ff102a00640 1 -- 172.21.15.159:0/897722395 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1040af2e0 msgr2=0x7ff10406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.313+0000 7ff102a00640 1 --2- 172.21.15.159:0/897722395 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1040af2e0 0x7ff10406d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.313+0000 7ff102a00640 1 -- 172.21.15.159:0/897722395 >> v2:172.21.15.178:3300/0 conn(0x7ff104153580 msgr2=0x7ff1040d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.313+0000 7ff102a00640 1 --2- 172.21.15.159:0/897722395 >> v2:172.21.15.178:3300/0 conn(0x7ff104153580 0x7ff1040d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.313+0000 7ff102a00640 1 -- 172.21.15.159:0/897722395 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10415da00 con 0x7ff1040d07c0 2024-04-04T06:09:44.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.315+0000 7ff102a00640 1 --2- 172.21.15.159:0/897722395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1040d07c0 0x7ff1040aeda0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff1040b5270 tx=0x7ff0ec02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02a01aaeb2b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:44.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.315+0000 7ff102000640 1 -- 172.21.15.159:0/897722395 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec0317d0 con 0x7ff1040d07c0 2024-04-04T06:09:44.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.315+0000 7ff102000640 1 -- 172.21.15.159:0/897722395 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0ec037070 con 0x7ff1040d07c0 2024-04-04T06:09:44.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.315+0000 7ff102000640 1 -- 172.21.15.159:0/897722395 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec031b90 con 0x7ff1040d07c0 2024-04-04T06:09:44.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.316+0000 7ff10a800640 1 -- 172.21.15.159:0/897722395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1040d07c0 msgr2=0x7ff1040aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.316+0000 7ff10a800640 1 --2- 172.21.15.159:0/897722395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1040d07c0 0x7ff1040aeda0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff1040b5270 tx=0x7ff0ec02b870 comp rx=0 tx=0).stop 2024-04-04T06:09:44.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.316+0000 7ff10a800640 1 -- 172.21.15.159:0/897722395 shutdown_connections 2024-04-04T06:09:44.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.316+0000 7ff10a800640 1 --2- 172.21.15.159:0/897722395 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1040af2e0 0x7ff10406d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.316+0000 7ff10a800640 1 --2- 172.21.15.159:0/897722395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1040d07c0 0x7ff1040aeda0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.316+0000 7ff10a800640 1 --2- 172.21.15.159:0/897722395 >> v2:172.21.15.178:3300/0 conn(0x7ff104153580 0x7ff1040d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.316+0000 7ff10a800640 1 -- 172.21.15.159:0/897722395 >> 172.21.15.159:0/897722395 conn(0x7ff10406b750 msgr2=0x7ff10406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:44.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.316+0000 7ff10a800640 1 -- 172.21.15.159:0/897722395 shutdown_connections 2024-04-04T06:09:44.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.316+0000 7ff10a800640 1 -- 172.21.15.159:0/897722395 wait complete. 2024-04-04T06:09:44.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.318+0000 7ff10a800640 1 Processor -- start 2024-04-04T06:09:44.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.318+0000 7ff10a800640 1 -- start start 2024-04-04T06:09:44.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.319+0000 7ff10a800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1040af2e0 0x7ff1040b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.319+0000 7ff10a800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1040d07c0 0x7ff104153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.319+0000 7ff10a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff104153580 0x7ff1041f8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.319+0000 7ff10a800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff10415d4e0 con 0x7ff104153580 2024-04-04T06:09:44.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.319+0000 7ff10a800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff10415d200 con 0x7ff1040af2e0 2024-04-04T06:09:44.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.319+0000 7ff10a800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff104153960 con 0x7ff1040d07c0 2024-04-04T06:09:44.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.319+0000 7ff103e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff104153580 0x7ff1041f8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.319+0000 7ff103e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff104153580 0x7ff1041f8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44934/0 (socket says 172.21.15.159:44934) 2024-04-04T06:09:44.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.320+0000 7ff103e00640 1 -- 172.21.15.159:0/1487782478 learned_addr learned my addr 172.21.15.159:0/1487782478 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:44.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.320+0000 7ff103400640 1 --2- 172.21.15.159:0/1487782478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1040af2e0 0x7ff1040b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.320+0000 7ff102a00640 1 --2- 172.21.15.159:0/1487782478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1040d07c0 0x7ff104153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.320+0000 7ff103e00640 1 -- 172.21.15.159:0/1487782478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1040d07c0 msgr2=0x7ff104153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.320+0000 7ff103e00640 1 --2- 172.21.15.159:0/1487782478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1040d07c0 0x7ff104153c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.320+0000 7ff103e00640 1 -- 172.21.15.159:0/1487782478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1040af2e0 msgr2=0x7ff1040b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.320+0000 7ff103e00640 1 --2- 172.21.15.159:0/1487782478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1040af2e0 0x7ff1040b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.320+0000 7ff103e00640 1 -- 172.21.15.159:0/1487782478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff104146d50 con 0x7ff104153580 2024-04-04T06:09:44.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.320+0000 7ff103400640 1 --2- 172.21.15.159:0/1487782478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1040af2e0 0x7ff1040b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:44.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.320+0000 7ff103e00640 1 --2- 172.21.15.159:0/1487782478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff104153580 0x7ff1041f8420 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00d5d0 tx=0x7ff0fc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:44.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.320+0000 7ff0fb400640 1 -- 172.21.15.159:0/1487782478 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc018020 con 0x7ff104153580 2024-04-04T06:09:44.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.321+0000 7ff0fb400640 1 -- 172.21.15.159:0/1487782478 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0fc014620 con 0x7ff104153580 2024-04-04T06:09:44.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.321+0000 7ff10a800640 1 -- 172.21.15.159:0/1487782478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff104154250 con 0x7ff104153580 2024-04-04T06:09:44.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.321+0000 7ff0fb400640 1 -- 172.21.15.159:0/1487782478 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc0149f0 con 0x7ff104153580 2024-04-04T06:09:44.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.321+0000 7ff10a800640 1 -- 172.21.15.159:0/1487782478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff104154810 con 0x7ff104153580 2024-04-04T06:09:44.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.322+0000 7ff10a800640 1 -- 172.21.15.159:0/1487782478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1040685c0 con 0x7ff104153580 2024-04-04T06:09:44.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.323+0000 7ff0fb400640 1 -- 172.21.15.159:0/1487782478 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff0fc01b020 con 0x7ff104153580 2024-04-04T06:09:44.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.324+0000 7ff0fb400640 1 --2- 172.21.15.159:0/1487782478 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff0e006c8f0 0x7ff0e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.324+0000 7ff0fb400640 1 -- 172.21.15.159:0/1487782478 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7ff0fc08bc90 con 0x7ff104153580 2024-04-04T06:09:44.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.324+0000 7ff103400640 1 --2- 172.21.15.159:0/1487782478 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff0e006c8f0 0x7ff0e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.326+0000 7ff103400640 1 --2- 172.21.15.159:0/1487782478 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff0e006c8f0 0x7ff0e006edb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4007330 tx=0x7ff0f400b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:44.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.329+0000 7ff0fb400640 1 -- 172.21.15.159:0/1487782478 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff0fc022030 con 0x7ff104153580 2024-04-04T06:09:44.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.573+0000 7ff10a800640 1 -- 172.21.15.159:0/1487782478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff10406eca0 con 0x7ff104153580 2024-04-04T06:09:44.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.574+0000 7ff0fb400640 1 -- 172.21.15.159:0/1487782478 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7ff0fc05a0b0 con 0x7ff104153580 2024-04-04T06:09:44.567 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:09:44.568 INFO:teuthology.orchestra.run.smithi159.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-04T06:09:44.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.577+0000 7ff10a800640 1 -- 172.21.15.159:0/1487782478 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff0e006c8f0 msgr2=0x7ff0e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.577+0000 7ff10a800640 1 --2- 172.21.15.159:0/1487782478 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff0e006c8f0 0x7ff0e006edb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4007330 tx=0x7ff0f400b040 comp rx=0 tx=0).stop 2024-04-04T06:09:44.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.577+0000 7ff10a800640 1 -- 172.21.15.159:0/1487782478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff104153580 msgr2=0x7ff1041f8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.577+0000 7ff10a800640 1 --2- 172.21.15.159:0/1487782478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff104153580 0x7ff1041f8420 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00d5d0 tx=0x7ff0fc00da90 comp rx=0 tx=0).stop 2024-04-04T06:09:44.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.577+0000 7ff10a800640 1 -- 172.21.15.159:0/1487782478 shutdown_connections 2024-04-04T06:09:44.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.577+0000 7ff10a800640 1 --2- 172.21.15.159:0/1487782478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff104153580 0x7ff1041f8420 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.577+0000 7ff10a800640 1 --2- 172.21.15.159:0/1487782478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff1040d07c0 0x7ff104153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.577+0000 7ff10a800640 1 --2- 172.21.15.159:0/1487782478 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff0e006c8f0 0x7ff0e006edb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.577+0000 7ff10a800640 1 --2- 172.21.15.159:0/1487782478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1040af2e0 0x7ff1040b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.577+0000 7ff10a800640 1 -- 172.21.15.159:0/1487782478 >> 172.21.15.159:0/1487782478 conn(0x7ff10406b750 msgr2=0x7ff10406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:44.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.577+0000 7ff10a800640 1 -- 172.21.15.159:0/1487782478 shutdown_connections 2024-04-04T06:09:44.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.577+0000 7ff10a800640 1 -- 172.21.15.159:0/1487782478 wait complete. 2024-04-04T06:09:44.594 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-04T06:09:44.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:09:44.595 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-04-04T06:09:44.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.805+0000 7f580fc00640 1 Processor -- start 2024-04-04T06:09:44.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.805+0000 7f580fc00640 1 -- start start 2024-04-04T06:09:44.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.806+0000 7f580fc00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f58080d0c90 0x7f58080d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.806+0000 7f580fc00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58080d21c0 con 0x7f58081610d0 2024-04-04T06:09:44.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.807+0000 7f580fc00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58080d24a0 con 0x7f580806e310 2024-04-04T06:09:44.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.807+0000 7f580fc00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f58080d1630 con 0x7f58080d0c90 2024-04-04T06:09:44.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.807+0000 7f580dc00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f58081610d0 0x7f5808161560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:42138/0 (socket says 172.21.15.159:42138) 2024-04-04T06:09:44.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.807+0000 7f580dc00640 1 -- 172.21.15.159:0/3585819032 learned_addr learned my addr 172.21.15.159:0/3585819032 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:09:44.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.807+0000 7f5807400640 1 -- 172.21.15.159:0/3585819032 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1459794965 0 0) 0x7f58080d21c0 con 0x7f58081610d0 2024-04-04T06:09:44.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.807+0000 7f5807400640 1 -- 172.21.15.159:0/3585819032 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57ec003520 con 0x7f58081610d0 2024-04-04T06:09:44.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.807+0000 7f5807e00640 1 --2- 172.21.15.159:0/3585819032 >> v2:172.21.15.178:3300/0 conn(0x7f58080d0c90 0x7f58080d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.808+0000 7f5807400640 1 -- 172.21.15.159:0/3585819032 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1325152089 0 0) 0x7f57ec003520 con 0x7f58081610d0 2024-04-04T06:09:44.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.808+0000 7f5807400640 1 -- 172.21.15.159:0/3585819032 >> v2:172.21.15.178:3300/0 conn(0x7f58080d0c90 msgr2=0x7f58080d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.808+0000 7f5807400640 1 --2- 172.21.15.159:0/3585819032 >> v2:172.21.15.178:3300/0 conn(0x7f58080d0c90 0x7f58080d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.808+0000 7f5807400640 1 -- 172.21.15.159:0/3585819032 >> v1:172.21.15.178:6789/0 conn(0x7f580806e310 legacy=0x7f58080cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.808+0000 7f5807400640 1 -- 172.21.15.159:0/3585819032 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58080d2780 con 0x7f58081610d0 2024-04-04T06:09:44.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.808+0000 7f5807400640 1 -- 172.21.15.159:0/3585819032 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f57f8001670 con 0x7f58081610d0 2024-04-04T06:09:44.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.808+0000 7f5807400640 1 -- 172.21.15.159:0/3585819032 >> v1:172.21.15.159:6789/0 conn(0x7f58081610d0 legacy=0x7f5808161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.808+0000 7f5807400640 1 --2- 172.21.15.159:0/3585819032 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57ec003a00 0x7f57ec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.808+0000 7f5807400640 1 --2- 172.21.15.159:0/3585819032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57ec004800 0x7f57ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.808+0000 7f5807400640 1 --2- 172.21.15.159:0/3585819032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57ec005210 0x7f57ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.808+0000 7f5807400640 1 -- 172.21.15.159:0/3585819032 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec00c110 con 0x7f57ec005210 2024-04-04T06:09:44.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.808+0000 7f5807400640 1 -- 172.21.15.159:0/3585819032 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec00c2c0 con 0x7f57ec004800 2024-04-04T06:09:44.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.808+0000 7f5807400640 1 -- 172.21.15.159:0/3585819032 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f57ec00c4c0 con 0x7f57ec003a00 2024-04-04T06:09:44.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.808+0000 7f580dc00640 1 --2- 172.21.15.159:0/3585819032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57ec005210 0x7f57ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.808+0000 7f5807e00640 1 --2- 172.21.15.159:0/3585819032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57ec004800 0x7f57ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.808+0000 7f580d200640 1 --2- 172.21.15.159:0/3585819032 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57ec003a00 0x7f57ec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.808+0000 7f580dc00640 1 -- 172.21.15.159:0/3585819032 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57ec003a00 msgr2=0x7f57ec003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.808+0000 7f580dc00640 1 --2- 172.21.15.159:0/3585819032 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57ec003a00 0x7f57ec003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.808+0000 7f580dc00640 1 -- 172.21.15.159:0/3585819032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57ec004800 msgr2=0x7f57ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.808+0000 7f580dc00640 1 --2- 172.21.15.159:0/3585819032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57ec004800 0x7f57ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.808+0000 7f580dc00640 1 -- 172.21.15.159:0/3585819032 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ec003520 con 0x7f57ec005210 2024-04-04T06:09:44.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.809+0000 7f580dc00640 1 -- 172.21.15.159:0/3585819032 reap_dead start 2024-04-04T06:09:44.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.810+0000 7f580dc00640 1 --2- 172.21.15.159:0/3585819032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57ec005210 0x7f57ec00b9c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f57f8001eb0 tx=0x7f57ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc460be8c46cafc server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:44.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.810+0000 7f5807400640 1 -- 172.21.15.159:0/3585819032 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8007c80 con 0x7f57ec005210 2024-04-04T06:09:44.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.810+0000 7f5807400640 1 -- 172.21.15.159:0/3585819032 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f8030070 con 0x7f57ec005210 2024-04-04T06:09:44.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.810+0000 7f5807400640 1 -- 172.21.15.159:0/3585819032 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8005d30 con 0x7f57ec005210 2024-04-04T06:09:44.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.810+0000 7f580fc00640 1 -- 172.21.15.159:0/3585819032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57ec005210 msgr2=0x7f57ec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.810+0000 7f580fc00640 1 --2- 172.21.15.159:0/3585819032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57ec005210 0x7f57ec00b9c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f57f8001eb0 tx=0x7f57ec005830 comp rx=0 tx=0).stop 2024-04-04T06:09:44.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.810+0000 7f580fc00640 1 -- 172.21.15.159:0/3585819032 shutdown_connections 2024-04-04T06:09:44.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.810+0000 7f580fc00640 1 --2- 172.21.15.159:0/3585819032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57ec005210 0x7f57ec00b9c0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.810+0000 7f580fc00640 1 -- 172.21.15.159:0/3585819032 >> 172.21.15.159:0/3585819032 conn(0x7f580808bfb0 msgr2=0x7f58080d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:44.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.811+0000 7f580fc00640 1 -- 172.21.15.159:0/3585819032 shutdown_connections 2024-04-04T06:09:44.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.811+0000 7f580fc00640 1 -- 172.21.15.159:0/3585819032 wait complete. 2024-04-04T06:09:44.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.813+0000 7f580fc00640 1 Processor -- start 2024-04-04T06:09:44.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.813+0000 7f580fc00640 1 -- start start 2024-04-04T06:09:44.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.814+0000 7f580fc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57ec005210 0x7f580815e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.814+0000 7f580fc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f580809e130 0x7f58080c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.814+0000 7f580fc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58080cd0a0 0x7f58080c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.814+0000 7f580fc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f58080d2d40 con 0x7f57ec005210 2024-04-04T06:09:44.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.814+0000 7f580fc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58080d53c0 con 0x7f58080cd0a0 2024-04-04T06:09:44.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.814+0000 7f580fc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f58080d3260 con 0x7f580809e130 2024-04-04T06:09:44.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.814+0000 7f580d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57ec005210 0x7f580815e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.814+0000 7f580d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57ec005210 0x7f580815e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44944/0 (socket says 172.21.15.159:44944) 2024-04-04T06:09:44.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.814+0000 7f580d200640 1 -- 172.21.15.159:0/1065649078 learned_addr learned my addr 172.21.15.159:0/1065649078 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:44.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.814+0000 7f5807e00640 1 --2- 172.21.15.159:0/1065649078 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f580809e130 0x7f58080c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.814+0000 7f580dc00640 1 --2- 172.21.15.159:0/1065649078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58080cd0a0 0x7f58080c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.814+0000 7f580d200640 1 -- 172.21.15.159:0/1065649078 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f580809e130 msgr2=0x7f58080c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.814+0000 7f580d200640 1 --2- 172.21.15.159:0/1065649078 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f580809e130 0x7f58080c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.814+0000 7f580d200640 1 -- 172.21.15.159:0/1065649078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58080cd0a0 msgr2=0x7f58080c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.814+0000 7f580d200640 1 --2- 172.21.15.159:0/1065649078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58080cd0a0 0x7f58080c9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.814+0000 7f580d200640 1 -- 172.21.15.159:0/1065649078 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58080a1f30 con 0x7f57ec005210 2024-04-04T06:09:44.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.815+0000 7f580d200640 1 --2- 172.21.15.159:0/1065649078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57ec005210 0x7f580815e7f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f580000e7d0 tx=0x7f580000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:44.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.815+0000 7f5804c00640 1 -- 172.21.15.159:0/1065649078 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800003df0 con 0x7f57ec005210 2024-04-04T06:09:44.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.815+0000 7f5804c00640 1 -- 172.21.15.159:0/1065649078 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5800019070 con 0x7f57ec005210 2024-04-04T06:09:44.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.815+0000 7f5804c00640 1 -- 172.21.15.159:0/1065649078 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580000c900 con 0x7f57ec005210 2024-04-04T06:09:44.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.815+0000 7f580fc00640 1 -- 172.21.15.159:0/1065649078 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58080cdfc0 con 0x7f57ec005210 2024-04-04T06:09:44.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.815+0000 7f580fc00640 1 -- 172.21.15.159:0/1065649078 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58080ce580 con 0x7f57ec005210 2024-04-04T06:09:44.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.818+0000 7f5804c00640 1 -- 172.21.15.159:0/1065649078 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f580000caa0 con 0x7f57ec005210 2024-04-04T06:09:44.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.819+0000 7f58129ab740 1 Processor -- start 2024-04-04T06:09:44.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.819+0000 7f5804c00640 1 --2- 172.21.15.159:0/1065649078 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f57e406c920 0x7f57e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.819+0000 7f58129ab740 1 -- start start 2024-04-04T06:09:44.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.819+0000 7f5804c00640 1 -- 172.21.15.159:0/1065649078 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f580008bea0 con 0x7f57ec005210 2024-04-04T06:09:44.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.820+0000 7f580dc00640 1 --2- 172.21.15.159:0/1065649078 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f57e406c920 0x7f57e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.820+0000 7f58129ab740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562654345500 0x5626543184a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.820+0000 7f58129ab740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626543458e0 0x562654348290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.820+0000 7f58129ab740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626543189e0 0x5626543487d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.820+0000 7f58129ab740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562654282310 con 0x5626543189e0 2024-04-04T06:09:44.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.820+0000 7f58129ab740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56265422a670 con 0x5626543458e0 2024-04-04T06:09:44.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.820+0000 7f58129ab740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56265427e2c0 con 0x562654345500 2024-04-04T06:09:44.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.821+0000 7f580dc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626543189e0 0x5626543487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.821+0000 7f580dc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626543189e0 0x5626543487d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44960/0 (socket says 172.21.15.159:44960) 2024-04-04T06:09:44.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.821+0000 7f580dc00640 1 -- 172.21.15.159:0/3383425798 learned_addr learned my addr 172.21.15.159:0/3383425798 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:44.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.821+0000 7f580d200640 1 --2- 172.21.15.159:0/3383425798 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562654345500 0x5626543184a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.821+0000 7f5807e00640 1 --2- 172.21.15.159:0/3383425798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626543458e0 0x562654348290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.821+0000 7f580dc00640 1 -- 172.21.15.159:0/3383425798 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562654345500 msgr2=0x5626543184a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.821+0000 7f580dc00640 1 --2- 172.21.15.159:0/3383425798 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562654345500 0x5626543184a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.821+0000 7f580dc00640 1 -- 172.21.15.159:0/3383425798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626543458e0 msgr2=0x562654348290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.821+0000 7f580dc00640 1 --2- 172.21.15.159:0/3383425798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626543458e0 0x562654348290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.821+0000 7f580dc00640 1 -- 172.21.15.159:0/3383425798 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562654062940 con 0x5626543189e0 2024-04-04T06:09:44.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.821+0000 7f580dc00640 1 --2- 172.21.15.159:0/1065649078 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f57e406c920 0x7f57e406ede0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00c290 tx=0x7f57f8007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:44.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.821+0000 7f580dc00640 1 --2- 172.21.15.159:0/3383425798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626543189e0 0x5626543487d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f57f8001eb0 tx=0x7f57f8037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b997f00a65c38e19 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:44.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.822+0000 7f57fd600640 1 -- 172.21.15.159:0/3383425798 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f802cca0 con 0x5626543189e0 2024-04-04T06:09:44.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.822+0000 7f57fd600640 1 -- 172.21.15.159:0/3383425798 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f8049070 con 0x5626543189e0 2024-04-04T06:09:44.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.822+0000 7f57fd600640 1 -- 172.21.15.159:0/3383425798 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8007270 con 0x5626543189e0 2024-04-04T06:09:44.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.822+0000 7f58129ab740 1 -- 172.21.15.159:0/3383425798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626543189e0 msgr2=0x5626543487d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.822+0000 7f58129ab740 1 --2- 172.21.15.159:0/3383425798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626543189e0 0x5626543487d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f57f8001eb0 tx=0x7f57f8037000 comp rx=0 tx=0).stop 2024-04-04T06:09:44.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.822+0000 7f58129ab740 1 -- 172.21.15.159:0/3383425798 shutdown_connections 2024-04-04T06:09:44.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.822+0000 7f58129ab740 1 --2- 172.21.15.159:0/3383425798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626543189e0 0x5626543487d0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.823+0000 7f58129ab740 1 --2- 172.21.15.159:0/3383425798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626543458e0 0x562654348290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.823+0000 7f58129ab740 1 --2- 172.21.15.159:0/3383425798 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562654345500 0x5626543184a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.823+0000 7f58129ab740 1 -- 172.21.15.159:0/3383425798 >> 172.21.15.159:0/3383425798 conn(0x56265431a710 msgr2=0x562654363290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:44.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.823+0000 7f58129ab740 1 -- 172.21.15.159:0/3383425798 shutdown_connections 2024-04-04T06:09:44.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.823+0000 7f58129ab740 1 -- 172.21.15.159:0/3383425798 wait complete. 2024-04-04T06:09:44.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.823+0000 7f58129ab740 1 -- start start 2024-04-04T06:09:44.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.824+0000 7f58129ab740 1 Processor -- start 2024-04-04T06:09:44.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.825+0000 7f58129ab740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562654345500 0x562654363aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.825+0000 7f58129ab740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5626543458e0 0x562654347130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.825+0000 7f58129ab740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562654347670 0x56265432e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.825+0000 7f58129ab740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56265431cdb0 con 0x562654345500 2024-04-04T06:09:44.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.825+0000 7f58129ab740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56265423da20 con 0x562654347670 2024-04-04T06:09:44.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.825+0000 7f58129ab740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56265431f5f0 con 0x5626543458e0 2024-04-04T06:09:44.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.825+0000 7f58129ab740 10 client.0 _ll_register_callbacks cb 0x562653f95d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:09:44.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.825+0000 7f580d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562654345500 0x562654363aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.825+0000 7f580d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562654345500 0x562654363aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44962/0 (socket says 172.21.15.159:44962) 2024-04-04T06:09:44.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.825+0000 7f580d200640 1 -- 172.21.15.159:0/2525300830 learned_addr learned my addr 172.21.15.159:0/2525300830 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:44.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.825+0000 7f5807e00640 1 --2- 172.21.15.159:0/2525300830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5626543458e0 0x562654347130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.825+0000 7f580dc00640 1 --2- 172.21.15.159:0/2525300830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562654347670 0x56265432e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.825+0000 7f580d200640 1 -- 172.21.15.159:0/2525300830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5626543458e0 msgr2=0x562654347130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.825+0000 7f580d200640 1 --2- 172.21.15.159:0/2525300830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5626543458e0 0x562654347130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.825+0000 7f580d200640 1 -- 172.21.15.159:0/2525300830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562654347670 msgr2=0x56265432e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.825+0000 7f580d200640 1 --2- 172.21.15.159:0/2525300830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562654347670 0x56265432e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.825+0000 7f580d200640 1 -- 172.21.15.159:0/2525300830 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626540c30d0 con 0x562654345500 2024-04-04T06:09:44.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.825+0000 7f5807e00640 1 --2- 172.21.15.159:0/2525300830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5626543458e0 0x562654347130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:44.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.826+0000 7f580d200640 1 --2- 172.21.15.159:0/2525300830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562654345500 0x562654363aa0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f580008a3c0 tx=0x7f58000124b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4608db0ac6da071 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:44.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.826+0000 7f57f6000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:09:44.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.826+0000 7f57f6000640 1 -- 172.21.15.159:0/2525300830 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800088060 con 0x562654345500 2024-04-04T06:09:44.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.826+0000 7f57f6000640 1 -- 172.21.15.159:0/2525300830 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f5800056230 con 0x562654345500 2024-04-04T06:09:44.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.826+0000 7f58129ab740 1 -- 172.21.15.159:0/2525300830 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57f8049070 con 0x562654345500 2024-04-04T06:09:44.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.826+0000 7f57f6000640 1 -- 172.21.15.159:0/2525300830 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5800072170 con 0x562654345500 2024-04-04T06:09:44.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.826+0000 7f58129ab740 10 client.6345 fetch_fsmap learned FSMap version 34 2024-04-04T06:09:44.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.826+0000 7f58129ab740 1 -- 172.21.15.159:0/2525300830 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x562653f56230 con 0x562654345500 2024-04-04T06:09:44.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.826+0000 7f57f6000640 1 -- 172.21.15.159:0/2525300830 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f8049070 con 0x562654345500 2024-04-04T06:09:44.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.827+0000 7f57f6000640 1 -- 172.21.15.159:0/2525300830 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f5800059010 con 0x562654345500 2024-04-04T06:09:44.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.827+0000 7f58129ab740 10 client.6345 fetch_fsmap finished waiting for FSMap version 34 2024-04-04T06:09:44.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.827+0000 7f58129ab740 10 client.6345 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-04T06:09:44.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.827+0000 7f58129ab740 20 client.6345 populate_metadata read hostname 'smithi159' 2024-04-04T06:09:44.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.827+0000 7f58129ab740 1 --2- 172.21.15.159:0/2525300830 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x56265433d800 0x56265433dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.827+0000 7f58129ab740 4 client.6345 mds_command: new command op to 6039 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:09:44.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.827+0000 7f58129ab740 1 -- 172.21.15.159:0/2525300830 --> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562654229d70 con 0x56265433d800 2024-04-04T06:09:44.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.827+0000 7f5807e00640 1 --2- 172.21.15.159:0/2525300830 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x56265433d800 0x56265433dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.828+0000 7f57f6000640 10 client.6345 ms_handle_connect on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:44.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.828+0000 7f5807e00640 1 --2- 172.21.15.159:0/2525300830 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x56265433d800 0x56265433dc20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47bc15867b635c8f server_cookie=267052bf96d09495 in_seq=0 out_seq=0 2024-04-04T06:09:44.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.828+0000 7f57f6000640 1 -- 172.21.15.159:0/2525300830 <== mds.0 v2:172.21.15.159:6833/962475038 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562654229d70 con 0x56265433d800 2024-04-04T06:09:44.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.828+0000 7f57f6000640 10 client.6345 handle_command_reply: tid=0 2024-04-04T06:09:44.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.828+0000 7f58129ab740 1 client.6345 shutdown 2024-04-04T06:09:44.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.829+0000 7f58129ab740 1 -- 172.21.15.159:0/2525300830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562654345500 msgr2=0x562654363aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.829+0000 7f58129ab740 1 --2- 172.21.15.159:0/2525300830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562654345500 0x562654363aa0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f580008a3c0 tx=0x7f58000124b0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.829+0000 7f58129ab740 1 -- 172.21.15.159:0/2525300830 shutdown_connections 2024-04-04T06:09:44.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.829+0000 7f58129ab740 1 --2- 172.21.15.159:0/2525300830 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x56265433d800 0x56265433dc20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.829+0000 7f58129ab740 1 --2- 172.21.15.159:0/2525300830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562654347670 0x56265432e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.829+0000 7f58129ab740 1 --2- 172.21.15.159:0/2525300830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5626543458e0 0x562654347130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.829+0000 7f57f6000640 0 client.6345 ms_handle_reset on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:44.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.829+0000 7f58129ab740 1 --2- 172.21.15.159:0/2525300830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562654345500 0x562654363aa0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.829+0000 7f58129ab740 1 -- 172.21.15.159:0/2525300830 >> 172.21.15.159:0/2525300830 conn(0x5626543189e0 msgr2=0x56265431c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:44.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.829+0000 7f58129ab740 1 -- 172.21.15.159:0/2525300830 shutdown_connections 2024-04-04T06:09:44.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.829+0000 7f58129ab740 1 -- 172.21.15.159:0/2525300830 wait complete. 2024-04-04T06:09:44.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.829+0000 7f58129ab740 20 client.6345 trim_cache size 0 max 16384 2024-04-04T06:09:44.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.848+0000 7f58129ab740 1 Processor -- start 2024-04-04T06:09:44.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.849+0000 7f58129ab740 1 -- start start 2024-04-04T06:09:44.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.849+0000 7f58129ab740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562654345500 0x56265433cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.849+0000 7f58129ab740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626543458e0 0x56265434b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.849+0000 7f58129ab740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562654347670 0x56265434bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.849+0000 7f58129ab740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5808200e30 con 0x5626543458e0 2024-04-04T06:09:44.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.849+0000 7f58129ab740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58081ffe80 con 0x562654345500 2024-04-04T06:09:44.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.849+0000 7f58129ab740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f58080cc9e0 con 0x562654347670 2024-04-04T06:09:44.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.850+0000 7f5807e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626543458e0 0x56265434b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.850+0000 7f5807e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626543458e0 0x56265434b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44976/0 (socket says 172.21.15.159:44976) 2024-04-04T06:09:44.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.850+0000 7f5807e00640 1 -- 172.21.15.159:0/4251751964 learned_addr learned my addr 172.21.15.159:0/4251751964 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:44.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.850+0000 7f580d200640 1 --2- 172.21.15.159:0/4251751964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562654345500 0x56265433cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.850+0000 7f580dc00640 1 --2- 172.21.15.159:0/4251751964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562654347670 0x56265434bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.850+0000 7f5807e00640 1 -- 172.21.15.159:0/4251751964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562654347670 msgr2=0x56265434bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.850+0000 7f5807e00640 1 --2- 172.21.15.159:0/4251751964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562654347670 0x56265434bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.850+0000 7f5807e00640 1 -- 172.21.15.159:0/4251751964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562654345500 msgr2=0x56265433cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.850+0000 7f5807e00640 1 --2- 172.21.15.159:0/4251751964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562654345500 0x56265433cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.850+0000 7f5807e00640 1 -- 172.21.15.159:0/4251751964 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562654318460 con 0x5626543458e0 2024-04-04T06:09:44.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.850+0000 7f5807e00640 1 --2- 172.21.15.159:0/4251751964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626543458e0 0x56265434b5f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f580806d150 tx=0x7f57f0018810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e763781cc97065 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:44.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.851+0000 7f57fcc00640 1 -- 172.21.15.159:0/4251751964 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0010cb0 con 0x5626543458e0 2024-04-04T06:09:44.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.851+0000 7f57fcc00640 1 -- 172.21.15.159:0/4251751964 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f0010e70 con 0x5626543458e0 2024-04-04T06:09:44.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.851+0000 7f57fcc00640 1 -- 172.21.15.159:0/4251751964 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0004240 con 0x5626543458e0 2024-04-04T06:09:44.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.851+0000 7f58129ab740 1 -- 172.21.15.159:0/4251751964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626543458e0 msgr2=0x56265434b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.851+0000 7f58129ab740 1 --2- 172.21.15.159:0/4251751964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626543458e0 0x56265434b5f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f580806d150 tx=0x7f57f0018810 comp rx=0 tx=0).stop 2024-04-04T06:09:44.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.851+0000 7f58129ab740 1 -- 172.21.15.159:0/4251751964 shutdown_connections 2024-04-04T06:09:44.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.851+0000 7f58129ab740 1 --2- 172.21.15.159:0/4251751964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562654347670 0x56265434bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.851+0000 7f58129ab740 1 --2- 172.21.15.159:0/4251751964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626543458e0 0x56265434b5f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.851+0000 7f58129ab740 1 --2- 172.21.15.159:0/4251751964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562654345500 0x56265433cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.851+0000 7f58129ab740 1 -- 172.21.15.159:0/4251751964 >> 172.21.15.159:0/4251751964 conn(0x5626543189e0 msgr2=0x56265432bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:44.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.852+0000 7f58129ab740 1 -- 172.21.15.159:0/4251751964 shutdown_connections 2024-04-04T06:09:44.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.852+0000 7f58129ab740 1 -- 172.21.15.159:0/4251751964 wait complete. 2024-04-04T06:09:44.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.852+0000 7f58129ab740 1 -- start start 2024-04-04T06:09:44.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.852+0000 7f58129ab740 1 Processor -- start 2024-04-04T06:09:44.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.853+0000 7f58129ab740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562654345500 0x562654345fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.853+0000 7f58129ab740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626543458e0 0x5626543cb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.853+0000 7f58129ab740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562654347670 0x562654349290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.853+0000 7f58129ab740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56265428cf00 con 0x562654347670 2024-04-04T06:09:44.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.853+0000 7f58129ab740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58081ff8c0 con 0x5626543458e0 2024-04-04T06:09:44.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.853+0000 7f58129ab740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5626543cb660 con 0x562654345500 2024-04-04T06:09:44.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.853+0000 7f58129ab740 10 client.0 _ll_register_callbacks cb 0x7f5800055ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:09:44.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.854+0000 7f580dc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562654347670 0x562654349290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.854+0000 7f580dc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562654347670 0x562654349290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44990/0 (socket says 172.21.15.159:44990) 2024-04-04T06:09:44.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.854+0000 7f580dc00640 1 -- 172.21.15.159:0/3354155061 learned_addr learned my addr 172.21.15.159:0/3354155061 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:44.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.854+0000 7f5807e00640 1 --2- 172.21.15.159:0/3354155061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626543458e0 0x5626543cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.854+0000 7f580d200640 1 --2- 172.21.15.159:0/3354155061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562654345500 0x562654345fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.854+0000 7f580dc00640 1 -- 172.21.15.159:0/3354155061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562654345500 msgr2=0x562654345fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.854+0000 7f580dc00640 1 --2- 172.21.15.159:0/3354155061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562654345500 0x562654345fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.854+0000 7f580dc00640 1 -- 172.21.15.159:0/3354155061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626543458e0 msgr2=0x5626543cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:44.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.854+0000 7f580dc00640 1 --2- 172.21.15.159:0/3354155061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626543458e0 0x5626543cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.854+0000 7f580dc00640 1 -- 172.21.15.159:0/3354155061 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626543e54b0 con 0x562654347670 2024-04-04T06:09:44.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.855+0000 7f580dc00640 1 --2- 172.21.15.159:0/3354155061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562654347670 0x562654349290 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f57f800c250 tx=0x7f57f802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a112f5e544657 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:44.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.855+0000 7f57f6000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:09:44.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.855+0000 7f57f6000640 1 -- 172.21.15.159:0/3354155061 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f805f060 con 0x562654347670 2024-04-04T06:09:44.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.855+0000 7f57f6000640 1 -- 172.21.15.159:0/3354155061 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f57f8007270 con 0x562654347670 2024-04-04T06:09:44.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.855+0000 7f58129ab740 1 -- 172.21.15.159:0/3354155061 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57f0010e70 con 0x562654347670 2024-04-04T06:09:44.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.855+0000 7f57f6000640 1 -- 172.21.15.159:0/3354155061 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f57f8065050 con 0x562654347670 2024-04-04T06:09:44.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.855+0000 7f58129ab740 10 client.6351 fetch_fsmap learned FSMap version 34 2024-04-04T06:09:44.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.855+0000 7f58129ab740 1 -- 172.21.15.159:0/3354155061 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x5626543497d0 con 0x562654347670 2024-04-04T06:09:44.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.856+0000 7f57f6000640 1 -- 172.21.15.159:0/3354155061 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f0010e70 con 0x562654347670 2024-04-04T06:09:44.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.856+0000 7f57f6000640 1 -- 172.21.15.159:0/3354155061 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f57f8068020 con 0x562654347670 2024-04-04T06:09:44.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.856+0000 7f58129ab740 10 client.6351 fetch_fsmap finished waiting for FSMap version 34 2024-04-04T06:09:44.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.856+0000 7f58129ab740 10 client.6351 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-04T06:09:44.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.856+0000 7f58129ab740 20 client.6351 populate_metadata read hostname 'smithi159' 2024-04-04T06:09:44.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.856+0000 7f58129ab740 1 --2- 172.21.15.159:0/3354155061 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x56265432bd40 0x56265432c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:44.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.856+0000 7f58129ab740 4 client.6351 mds_command: new command op to 6039 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-04-04T06:09:44.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.856+0000 7f58129ab740 1 -- 172.21.15.159:0/3354155061 --> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) -- 0x56265428f850 con 0x56265432bd40 2024-04-04T06:09:44.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.856+0000 7f580d200640 1 --2- 172.21.15.159:0/3354155061 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x56265432bd40 0x56265432c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:44.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.857+0000 7f57f6000640 10 client.6351 ms_handle_connect on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:44.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.857+0000 7f580d200640 1 --2- 172.21.15.159:0/3354155061 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x56265432bd40 0x56265432c150 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c62ec655b90670fd server_cookie=817661b7003b343d in_seq=0 out_seq=0 2024-04-04T06:09:44.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.857+0000 7f57f6000640 1 -- 172.21.15.159:0/3354155061 <== mds.0 v2:172.21.15.159:6833/962475038 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56265428f850 con 0x56265432bd40 2024-04-04T06:09:44.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.857+0000 7f57f6000640 10 client.6351 handle_command_reply: tid=0 2024-04-04T06:09:44.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.857+0000 7f58129ab740 1 client.6351 shutdown 2024-04-04T06:09:44.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.857+0000 7f58129ab740 1 -- 172.21.15.159:0/3354155061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562654347670 msgr2=0x562654349290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.857+0000 7f58129ab740 1 --2- 172.21.15.159:0/3354155061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562654347670 0x562654349290 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f57f800c250 tx=0x7f57f802e000 comp rx=0 tx=0).stop 2024-04-04T06:09:44.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.858+0000 7f58129ab740 1 -- 172.21.15.159:0/3354155061 shutdown_connections 2024-04-04T06:09:44.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.858+0000 7f58129ab740 1 --2- 172.21.15.159:0/3354155061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562654347670 0x562654349290 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.858+0000 7f58129ab740 1 --2- 172.21.15.159:0/3354155061 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x56265432bd40 0x56265432c150 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.858+0000 7f58129ab740 1 --2- 172.21.15.159:0/3354155061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5626543458e0 0x5626543cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.858+0000 7f58129ab740 1 --2- 172.21.15.159:0/3354155061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562654345500 0x562654345fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.858+0000 7f57f6000640 0 client.6351 ms_handle_reset on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:44.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.858+0000 7f58129ab740 1 -- 172.21.15.159:0/3354155061 >> 172.21.15.159:0/3354155061 conn(0x5626543189e0 msgr2=0x562654364e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:44.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.858+0000 7f58129ab740 1 -- 172.21.15.159:0/3354155061 shutdown_connections 2024-04-04T06:09:44.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.858+0000 7f58129ab740 1 -- 172.21.15.159:0/3354155061 wait complete. 2024-04-04T06:09:44.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.858+0000 7f58129ab740 20 client.6351 trim_cache size 0 max 16384 2024-04-04T06:09:44.855 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:09:44.855 INFO:teuthology.orchestra.run.smithi159.stdout: "return_code": 0, 2024-04-04T06:09:44.855 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_tag": "3689963f-2d46-4d29-a0eb-8adac8095d39", 2024-04-04T06:09:44.855 INFO:teuthology.orchestra.run.smithi159.stdout: "mode": "asynchronous" 2024-04-04T06:09:44.855 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:09:44.855 INFO:tasks.ceph.mds.c.smithi159.stderr:2024-04-04T06:09:44.858+0000 7f1939600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-04-04T06:09:11.257106+0000 1=1+0) caps={5296=pAsLsXsFs/-@2},l=5296(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x561a275bac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-04T06:09:44.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.859+0000 7f580fc00640 1 -- 172.21.15.159:0/1065649078 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f57e406c920 msgr2=0x7f57e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.859+0000 7f580fc00640 1 --2- 172.21.15.159:0/1065649078 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f57e406c920 0x7f57e406ede0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00c290 tx=0x7f57f8007000 comp rx=0 tx=0).stop 2024-04-04T06:09:44.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.859+0000 7f580fc00640 1 -- 172.21.15.159:0/1065649078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57ec005210 msgr2=0x7f580815e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:44.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.859+0000 7f580fc00640 1 --2- 172.21.15.159:0/1065649078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57ec005210 0x7f580815e7f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f580000e7d0 tx=0x7f580000ec90 comp rx=0 tx=0).stop 2024-04-04T06:09:44.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.859+0000 7f580fc00640 1 -- 172.21.15.159:0/1065649078 shutdown_connections 2024-04-04T06:09:44.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.859+0000 7f580fc00640 1 --2- 172.21.15.159:0/1065649078 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f57e406c920 0x7f57e406ede0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.859+0000 7f580fc00640 1 --2- 172.21.15.159:0/1065649078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58080cd0a0 0x7f58080c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.859+0000 7f580fc00640 1 --2- 172.21.15.159:0/1065649078 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f580809e130 0x7f58080c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.859+0000 7f580fc00640 1 --2- 172.21.15.159:0/1065649078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57ec005210 0x7f580815e7f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:44.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.859+0000 7f580fc00640 1 -- 172.21.15.159:0/1065649078 >> 172.21.15.159:0/1065649078 conn(0x7f580808bfb0 msgr2=0x7f580806c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:44.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.859+0000 7f580fc00640 1 -- 172.21.15.159:0/1065649078 shutdown_connections 2024-04-04T06:09:44.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:44.859+0000 7f580fc00640 1 -- 172.21.15.159:0/1065649078 wait complete. 2024-04-04T06:09:44.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:09:44.870 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-04T06:09:45.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.076+0000 7fea77e00640 1 Processor -- start 2024-04-04T06:09:45.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.077+0000 7fea77e00640 1 -- start start 2024-04-04T06:09:45.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.078+0000 7fea77e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea7806bc80 0x7fea780702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:45.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.078+0000 7fea77e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fea780707e0 0x7fea780724f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:45.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.078+0000 7fea77e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea780cfe80 con 0x7fea78147c80 2024-04-04T06:09:45.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.078+0000 7fea77e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fea78070c70 con 0x7fea780707e0 2024-04-04T06:09:45.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.078+0000 7fea77e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea780d0f20 con 0x7fea7806bc80 2024-04-04T06:09:45.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.079+0000 7fea77400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fea780707e0 0x7fea780724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:45.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.079+0000 7fea77400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fea780707e0 0x7fea780724f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45002/0 (socket says 172.21.15.159:45002) 2024-04-04T06:09:45.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.079+0000 7fea77400640 1 -- 172.21.15.159:0/879673953 learned_addr learned my addr 172.21.15.159:0/879673953 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:45.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.079+0000 7fea76a00640 1 --1- 172.21.15.159:0/879673953 >> v1:172.21.15.159:6789/0 conn(0x7fea78147c80 0x7fea7806b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:42154/0 (socket says 172.21.15.159:42154) 2024-04-04T06:09:45.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.079+0000 7fea6fe00640 1 --2- 172.21.15.159:0/879673953 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea7806bc80 0x7fea780702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:45.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.079+0000 7fea77400640 1 -- 172.21.15.159:0/879673953 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea7806bc80 msgr2=0x7fea780702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:45.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.079+0000 7fea77400640 1 --2- 172.21.15.159:0/879673953 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea7806bc80 0x7fea780702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.079+0000 7fea77400640 1 -- 172.21.15.159:0/879673953 >> v1:172.21.15.159:6789/0 conn(0x7fea78147c80 legacy=0x7fea7806b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:45.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.079+0000 7fea77400640 1 -- 172.21.15.159:0/879673953 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea780d0160 con 0x7fea780707e0 2024-04-04T06:09:45.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.081+0000 7fea77400640 1 --2- 172.21.15.159:0/879673953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea780707e0 0x7fea780724f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fea70009e50 tx=0x7fea7002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680a8556f547c31d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:45.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.081+0000 7fea76000640 1 -- 172.21.15.159:0/879673953 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7002cb00 con 0x7fea780707e0 2024-04-04T06:09:45.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.081+0000 7fea76000640 1 -- 172.21.15.159:0/879673953 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea7002cca0 con 0x7fea780707e0 2024-04-04T06:09:45.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.081+0000 7fea76000640 1 -- 172.21.15.159:0/879673953 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea70034610 con 0x7fea780707e0 2024-04-04T06:09:45.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.082+0000 7fea77e00640 1 -- 172.21.15.159:0/879673953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea780707e0 msgr2=0x7fea780724f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:45.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.082+0000 7fea77e00640 1 --2- 172.21.15.159:0/879673953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea780707e0 0x7fea780724f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fea70009e50 tx=0x7fea7002bf60 comp rx=0 tx=0).stop 2024-04-04T06:09:45.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.082+0000 7fea77e00640 1 -- 172.21.15.159:0/879673953 shutdown_connections 2024-04-04T06:09:45.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.082+0000 7fea77e00640 1 --2- 172.21.15.159:0/879673953 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea7806bc80 0x7fea780702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.082+0000 7fea77e00640 1 --2- 172.21.15.159:0/879673953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea780707e0 0x7fea780724f0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.082+0000 7fea77e00640 1 -- 172.21.15.159:0/879673953 >> 172.21.15.159:0/879673953 conn(0x7fea7806e730 msgr2=0x7fea7806eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:45.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.082+0000 7fea77e00640 1 -- 172.21.15.159:0/879673953 shutdown_connections 2024-04-04T06:09:45.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.082+0000 7fea77e00640 1 -- 172.21.15.159:0/879673953 wait complete. 2024-04-04T06:09:45.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.085+0000 7fea77e00640 1 Processor -- start 2024-04-04T06:09:45.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.085+0000 7fea77e00640 1 -- start start 2024-04-04T06:09:45.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.087+0000 7fea77e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea7806bc80 0x7fea780b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:45.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.087+0000 7fea77e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea780707e0 0x7fea780afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:45.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.087+0000 7fea77e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea78147c80 0x7fea780b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:45.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.087+0000 7fea77e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fea780d0720 con 0x7fea78147c80 2024-04-04T06:09:45.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.087+0000 7fea77e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea780d0a90 con 0x7fea780707e0 2024-04-04T06:09:45.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.087+0000 7fea77e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea780cfb50 con 0x7fea7806bc80 2024-04-04T06:09:45.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.087+0000 7fea77400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea78147c80 0x7fea780b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:45.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.087+0000 7fea77400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea78147c80 0x7fea780b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45004/0 (socket says 172.21.15.159:45004) 2024-04-04T06:09:45.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.087+0000 7fea77400640 1 -- 172.21.15.159:0/4043058058 learned_addr learned my addr 172.21.15.159:0/4043058058 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:45.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.087+0000 7fea76a00640 1 --2- 172.21.15.159:0/4043058058 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea7806bc80 0x7fea780b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:45.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.087+0000 7fea6fe00640 1 --2- 172.21.15.159:0/4043058058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea780707e0 0x7fea780afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:45.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.087+0000 7fea77400640 1 -- 172.21.15.159:0/4043058058 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea7806bc80 msgr2=0x7fea780b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:45.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.087+0000 7fea77400640 1 --2- 172.21.15.159:0/4043058058 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea7806bc80 0x7fea780b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.087+0000 7fea77400640 1 -- 172.21.15.159:0/4043058058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea780707e0 msgr2=0x7fea780afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:45.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.087+0000 7fea77400640 1 --2- 172.21.15.159:0/4043058058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea780707e0 0x7fea780afb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.087+0000 7fea77400640 1 -- 172.21.15.159:0/4043058058 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea780b1570 con 0x7fea78147c80 2024-04-04T06:09:45.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.087+0000 7fea76a00640 1 --2- 172.21.15.159:0/4043058058 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea7806bc80 0x7fea780b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:09:45.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.088+0000 7fea77400640 1 --2- 172.21.15.159:0/4043058058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea78147c80 0x7fea780b0c00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fea70009e50 tx=0x7fea70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:45.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.088+0000 7fea6ea00640 1 -- 172.21.15.159:0/4043058058 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea70043020 con 0x7fea78147c80 2024-04-04T06:09:45.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.088+0000 7fea6ea00640 1 -- 172.21.15.159:0/4043058058 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea7002ce50 con 0x7fea78147c80 2024-04-04T06:09:45.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.088+0000 7fea6ea00640 1 -- 172.21.15.159:0/4043058058 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea70009510 con 0x7fea78147c80 2024-04-04T06:09:45.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.088+0000 7fea77e00640 1 -- 172.21.15.159:0/4043058058 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea780b1850 con 0x7fea78147c80 2024-04-04T06:09:45.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.088+0000 7fea77e00640 1 -- 172.21.15.159:0/4043058058 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea780b1e10 con 0x7fea78147c80 2024-04-04T06:09:45.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.090+0000 7fea6ea00640 1 -- 172.21.15.159:0/4043058058 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea70046020 con 0x7fea78147c80 2024-04-04T06:09:45.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.091+0000 7fea6ea00640 1 --2- 172.21.15.159:0/4043058058 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea5406c8f0 0x7fea5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:45.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.091+0000 7fea6ea00640 1 -- 172.21.15.159:0/4043058058 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fea700ae260 con 0x7fea78147c80 2024-04-04T06:09:45.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.092+0000 7fea76a00640 1 --2- 172.21.15.159:0/4043058058 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea5406c8f0 0x7fea5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:45.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.093+0000 7fea809c2740 1 Processor -- start 2024-04-04T06:09:45.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.093+0000 7fea809c2740 1 -- start start 2024-04-04T06:09:45.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.093+0000 7fea76a00640 1 --2- 172.21.15.159:0/4043058058 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea5406c8f0 0x7fea5406edb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fea68000f60 tx=0x7fea68009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:45.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.094+0000 7fea809c2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556a43afb740 0x556a43ace6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:45.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.094+0000 7fea809c2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43afbb20 0x556a43afe4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:45.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.094+0000 7fea809c2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a43acec20 0x556a43afea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:45.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.094+0000 7fea809c2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556a43a396d0 con 0x556a43afbb20 2024-04-04T06:09:45.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.094+0000 7fea809c2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556a439e1770 con 0x556a43acec20 2024-04-04T06:09:45.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.094+0000 7fea809c2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x556a43a35680 con 0x556a43afb740 2024-04-04T06:09:45.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.094+0000 7fea6fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43afbb20 0x556a43afe4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:45.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.094+0000 7fea6fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43afbb20 0x556a43afe4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45010/0 (socket says 172.21.15.159:45010) 2024-04-04T06:09:45.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.094+0000 7fea6fe00640 1 -- 172.21.15.159:0/2324093330 learned_addr learned my addr 172.21.15.159:0/2324093330 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:45.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.095+0000 7fea76a00640 1 --2- 172.21.15.159:0/2324093330 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556a43afb740 0x556a43ace6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:45.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.095+0000 7fea77400640 1 --2- 172.21.15.159:0/2324093330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a43acec20 0x556a43afea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:45.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.095+0000 7fea6fe00640 1 -- 172.21.15.159:0/2324093330 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556a43afb740 msgr2=0x556a43ace6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:45.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.095+0000 7fea6fe00640 1 --2- 172.21.15.159:0/2324093330 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556a43afb740 0x556a43ace6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.095+0000 7fea6fe00640 1 -- 172.21.15.159:0/2324093330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a43acec20 msgr2=0x556a43afea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:45.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.095+0000 7fea6fe00640 1 --2- 172.21.15.159:0/2324093330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a43acec20 0x556a43afea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.095+0000 7fea6fe00640 1 -- 172.21.15.159:0/2324093330 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a438177d0 con 0x556a43afbb20 2024-04-04T06:09:45.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.095+0000 7fea6fe00640 1 --2- 172.21.15.159:0/2324093330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43afbb20 0x556a43afe4d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fea60009910 tx=0x7fea60009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863246cc04c5302a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:45.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.095+0000 7fea67400640 1 -- 172.21.15.159:0/2324093330 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea60017020 con 0x556a43afbb20 2024-04-04T06:09:45.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.095+0000 7fea67400640 1 -- 172.21.15.159:0/2324093330 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea6000e8f0 con 0x556a43afbb20 2024-04-04T06:09:45.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.095+0000 7fea67400640 1 -- 172.21.15.159:0/2324093330 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6000f040 con 0x556a43afbb20 2024-04-04T06:09:45.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.096+0000 7fea809c2740 1 -- 172.21.15.159:0/2324093330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43afbb20 msgr2=0x556a43afe4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:45.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.096+0000 7fea809c2740 1 --2- 172.21.15.159:0/2324093330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43afbb20 0x556a43afe4d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fea60009910 tx=0x7fea60009dd0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.096+0000 7fea809c2740 1 -- 172.21.15.159:0/2324093330 shutdown_connections 2024-04-04T06:09:45.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.096+0000 7fea809c2740 1 --2- 172.21.15.159:0/2324093330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a43acec20 0x556a43afea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.096+0000 7fea809c2740 1 --2- 172.21.15.159:0/2324093330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43afbb20 0x556a43afe4d0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.096+0000 7fea809c2740 1 --2- 172.21.15.159:0/2324093330 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556a43afb740 0x556a43ace6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.096+0000 7fea809c2740 1 -- 172.21.15.159:0/2324093330 >> 172.21.15.159:0/2324093330 conn(0x556a43ad0950 msgr2=0x556a43b194d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:45.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.096+0000 7fea809c2740 1 -- 172.21.15.159:0/2324093330 shutdown_connections 2024-04-04T06:09:45.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.096+0000 7fea809c2740 1 -- 172.21.15.159:0/2324093330 wait complete. 2024-04-04T06:09:45.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.097+0000 7fea809c2740 1 -- start start 2024-04-04T06:09:45.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.097+0000 7fea809c2740 1 Processor -- start 2024-04-04T06:09:45.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.098+0000 7fea809c2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a43afb740 0x556a43b059b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:45.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.098+0000 7fea809c2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556a43afbb20 0x556a43aff940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:45.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.098+0000 7fea809c2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43affe80 0x556a43b045e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:45.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.098+0000 7fea809c2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556a43ad2ff0 con 0x556a43affe80 2024-04-04T06:09:45.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.098+0000 7fea809c2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556a439f4aa0 con 0x556a43afb740 2024-04-04T06:09:45.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.098+0000 7fea809c2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x556a43ad5830 con 0x556a43afbb20 2024-04-04T06:09:45.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.098+0000 7fea809c2740 10 client.0 _ll_register_callbacks cb 0x556a439f4490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:09:45.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.098+0000 7fea77400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43affe80 0x556a43b045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:45.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.098+0000 7fea77400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43affe80 0x556a43b045e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45022/0 (socket says 172.21.15.159:45022) 2024-04-04T06:09:45.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.098+0000 7fea77400640 1 -- 172.21.15.159:0/245498493 learned_addr learned my addr 172.21.15.159:0/245498493 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:45.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.099+0000 7fea76a00640 1 --2- 172.21.15.159:0/245498493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a43afb740 0x556a43b059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:45.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.099+0000 7fea77400640 1 -- 172.21.15.159:0/245498493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556a43afbb20 msgr2=0x556a43aff940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:45.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.099+0000 7fea6fe00640 1 --2- 172.21.15.159:0/245498493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556a43afbb20 0x556a43aff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:45.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.099+0000 7fea77400640 1 --2- 172.21.15.159:0/245498493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556a43afbb20 0x556a43aff940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.099+0000 7fea77400640 1 -- 172.21.15.159:0/245498493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a43afb740 msgr2=0x556a43b059b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:45.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.099+0000 7fea77400640 1 --2- 172.21.15.159:0/245498493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a43afb740 0x556a43b059b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.099+0000 7fea77400640 1 -- 172.21.15.159:0/245498493 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a4374d3f0 con 0x556a43affe80 2024-04-04T06:09:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.099+0000 7fea77400640 1 --2- 172.21.15.159:0/245498493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43affe80 0x556a43b045e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fea70078350 tx=0x7fea7007c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8483b60334387956 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.099+0000 7fea5be00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:09:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.099+0000 7fea5be00640 1 -- 172.21.15.159:0/245498493 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea700802a0 con 0x556a43affe80 2024-04-04T06:09:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.099+0000 7fea5be00640 1 -- 172.21.15.159:0/245498493 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fea7003c350 con 0x556a43affe80 2024-04-04T06:09:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.099+0000 7fea809c2740 1 -- 172.21.15.159:0/245498493 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea6000e8f0 con 0x556a43affe80 2024-04-04T06:09:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.100+0000 7fea5be00640 1 -- 172.21.15.159:0/245498493 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fea7003c550 con 0x556a43affe80 2024-04-04T06:09:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.100+0000 7fea809c2740 10 client.6366 fetch_fsmap learned FSMap version 34 2024-04-04T06:09:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.100+0000 7fea809c2740 1 -- 172.21.15.159:0/245498493 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x556a4370d0f0 con 0x556a43affe80 2024-04-04T06:09:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.100+0000 7fea5be00640 1 -- 172.21.15.159:0/245498493 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea6000e8f0 con 0x556a43affe80 2024-04-04T06:09:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.100+0000 7fea5be00640 1 -- 172.21.15.159:0/245498493 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7fea70084640 con 0x556a43affe80 2024-04-04T06:09:45.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.100+0000 7fea809c2740 10 client.6366 fetch_fsmap finished waiting for FSMap version 34 2024-04-04T06:09:45.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.100+0000 7fea809c2740 10 client.6366 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-04T06:09:45.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.100+0000 7fea809c2740 20 client.6366 populate_metadata read hostname 'smithi159' 2024-04-04T06:09:45.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.100+0000 7fea809c2740 1 --2- 172.21.15.159:0/245498493 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x556a43b2d500 0x556a43b2d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:45.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.100+0000 7fea809c2740 4 client.6366 mds_command: new command op to 6039 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:09:45.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.100+0000 7fea809c2740 1 -- 172.21.15.159:0/245498493 --> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a43a46c10 con 0x556a43b2d500 2024-04-04T06:09:45.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.100+0000 7fea76a00640 1 --2- 172.21.15.159:0/245498493 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x556a43b2d500 0x556a43b2d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:45.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.101+0000 7fea76a00640 1 --2- 172.21.15.159:0/245498493 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x556a43b2d500 0x556a43b2d940 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4528ee796fde6929 server_cookie=b526346fe9e33f61 in_seq=0 out_seq=0 2024-04-04T06:09:45.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.101+0000 7fea5be00640 10 client.6366 ms_handle_connect on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:45.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.102+0000 7fea5be00640 1 -- 172.21.15.159:0/245498493 <== mds.0 v2:172.21.15.159:6833/962475038 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556a43a46c10 con 0x556a43b2d500 2024-04-04T06:09:45.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.102+0000 7fea5be00640 10 client.6366 handle_command_reply: tid=0 2024-04-04T06:09:45.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.102+0000 7fea809c2740 1 client.6366 shutdown 2024-04-04T06:09:45.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.102+0000 7fea809c2740 1 -- 172.21.15.159:0/245498493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43affe80 msgr2=0x556a43b045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:45.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.102+0000 7fea809c2740 1 --2- 172.21.15.159:0/245498493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43affe80 0x556a43b045e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fea70078350 tx=0x7fea7007c5f0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.102+0000 7fea809c2740 1 -- 172.21.15.159:0/245498493 shutdown_connections 2024-04-04T06:09:45.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.102+0000 7fea809c2740 1 --2- 172.21.15.159:0/245498493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43affe80 0x556a43b045e0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.102+0000 7fea809c2740 1 --2- 172.21.15.159:0/245498493 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556a43afbb20 0x556a43aff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.102+0000 7fea809c2740 1 --2- 172.21.15.159:0/245498493 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x556a43b2d500 0x556a43b2d940 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.102+0000 7fea809c2740 1 --2- 172.21.15.159:0/245498493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a43afb740 0x556a43b059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.102+0000 7fea809c2740 1 -- 172.21.15.159:0/245498493 >> 172.21.15.159:0/245498493 conn(0x556a43acec20 msgr2=0x556a43ad26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:45.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.102+0000 7fea5be00640 0 client.6366 ms_handle_reset on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:45.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.102+0000 7fea809c2740 1 -- 172.21.15.159:0/245498493 shutdown_connections 2024-04-04T06:09:45.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.102+0000 7fea809c2740 1 -- 172.21.15.159:0/245498493 wait complete. 2024-04-04T06:09:45.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.102+0000 7fea809c2740 20 client.6366 trim_cache size 0 max 16384 2024-04-04T06:09:45.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.121+0000 7fea809c2740 1 Processor -- start 2024-04-04T06:09:45.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.121+0000 7fea809c2740 1 -- start start 2024-04-04T06:09:45.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.122+0000 7fea809c2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43afb740 0x556a43b3da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:45.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.122+0000 7fea809c2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a43afbb20 0x556a43b26f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:45.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.122+0000 7fea809c2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556a43affe80 0x556a43b1c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:45.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.122+0000 7fea809c2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fea782009d0 con 0x556a43afb740 2024-04-04T06:09:45.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.122+0000 7fea809c2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea780b4b40 con 0x556a43afbb20 2024-04-04T06:09:45.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.122+0000 7fea809c2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea780b3ea0 con 0x556a43affe80 2024-04-04T06:09:45.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.122+0000 7fea76a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43afb740 0x556a43b3da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:45.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.122+0000 7fea76a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43afb740 0x556a43b3da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45028/0 (socket says 172.21.15.159:45028) 2024-04-04T06:09:45.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.123+0000 7fea76a00640 1 -- 172.21.15.159:0/169202228 learned_addr learned my addr 172.21.15.159:0/169202228 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:45.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.123+0000 7fea76a00640 1 -- 172.21.15.159:0/169202228 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556a43affe80 msgr2=0x556a43b1c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:45.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.123+0000 7fea76a00640 1 --2- 172.21.15.159:0/169202228 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556a43affe80 0x556a43b1c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.123+0000 7fea76a00640 1 -- 172.21.15.159:0/169202228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a43afbb20 msgr2=0x556a43b26f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:45.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.123+0000 7fea76a00640 1 --2- 172.21.15.159:0/169202228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a43afbb20 0x556a43b26f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.123+0000 7fea76a00640 1 -- 172.21.15.159:0/169202228 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a43a35800 con 0x556a43afb740 2024-04-04T06:09:45.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.123+0000 7fea76a00640 1 --2- 172.21.15.159:0/169202228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43afb740 0x556a43b3da10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fea68007370 tx=0x7fea68016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14c559a2fe245c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:45.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.123+0000 7fea66a00640 1 -- 172.21.15.159:0/169202228 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6801d040 con 0x556a43afb740 2024-04-04T06:09:45.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.123+0000 7fea66a00640 1 -- 172.21.15.159:0/169202228 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea68029070 con 0x556a43afb740 2024-04-04T06:09:45.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.123+0000 7fea66a00640 1 -- 172.21.15.159:0/169202228 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea68024860 con 0x556a43afb740 2024-04-04T06:09:45.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.124+0000 7fea809c2740 1 -- 172.21.15.159:0/169202228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43afb740 msgr2=0x556a43b3da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:45.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.124+0000 7fea809c2740 1 --2- 172.21.15.159:0/169202228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43afb740 0x556a43b3da10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fea68007370 tx=0x7fea68016040 comp rx=0 tx=0).stop 2024-04-04T06:09:45.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.124+0000 7fea809c2740 1 -- 172.21.15.159:0/169202228 shutdown_connections 2024-04-04T06:09:45.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.124+0000 7fea809c2740 1 --2- 172.21.15.159:0/169202228 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556a43affe80 0x556a43b1c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.124+0000 7fea809c2740 1 --2- 172.21.15.159:0/169202228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a43afbb20 0x556a43b26f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.124+0000 7fea809c2740 1 --2- 172.21.15.159:0/169202228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43afb740 0x556a43b3da10 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.124+0000 7fea809c2740 1 -- 172.21.15.159:0/169202228 >> 172.21.15.159:0/169202228 conn(0x556a43acec20 msgr2=0x556a43b04310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:45.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.124+0000 7fea809c2740 1 -- 172.21.15.159:0/169202228 shutdown_connections 2024-04-04T06:09:45.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.124+0000 7fea809c2740 1 -- 172.21.15.159:0/169202228 wait complete. 2024-04-04T06:09:45.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.125+0000 7fea809c2740 1 -- start start 2024-04-04T06:09:45.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.125+0000 7fea809c2740 1 Processor -- start 2024-04-04T06:09:45.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.126+0000 7fea809c2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556a43afb740 0x556a43ad2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:45.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.126+0000 7fea809c2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43afbb20 0x556a43acddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:45.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.126+0000 7fea809c2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a43affe80 0x556a43b19650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:45.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.126+0000 7fea809c2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556a43ad4280 con 0x556a43afbb20 2024-04-04T06:09:45.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.126+0000 7fea809c2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea780b44f0 con 0x556a43affe80 2024-04-04T06:09:45.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.126+0000 7fea809c2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x556a43ad28d0 con 0x556a43afb740 2024-04-04T06:09:45.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.126+0000 7fea809c2740 10 client.0 _ll_register_callbacks cb 0x7fea7007b850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:09:45.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.126+0000 7fea6fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43afbb20 0x556a43acddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:45.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.126+0000 7fea6fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43afbb20 0x556a43acddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45030/0 (socket says 172.21.15.159:45030) 2024-04-04T06:09:45.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.126+0000 7fea6fe00640 1 -- 172.21.15.159:0/913932465 learned_addr learned my addr 172.21.15.159:0/913932465 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:09:45.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.127+0000 7fea6fe00640 1 -- 172.21.15.159:0/913932465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556a43afb740 msgr2=0x556a43ad2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:45.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.127+0000 7fea76a00640 1 --2- 172.21.15.159:0/913932465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556a43afb740 0x556a43ad2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:45.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.127+0000 7fea6fe00640 1 --2- 172.21.15.159:0/913932465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556a43afb740 0x556a43ad2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.127+0000 7fea6fe00640 1 -- 172.21.15.159:0/913932465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a43affe80 msgr2=0x556a43b19650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:09:45.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.127+0000 7fea6fe00640 1 --2- 172.21.15.159:0/913932465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a43affe80 0x556a43b19650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.127+0000 7fea6fe00640 1 -- 172.21.15.159:0/913932465 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a43ad4a50 con 0x556a43afbb20 2024-04-04T06:09:45.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.127+0000 7fea6fe00640 1 --2- 172.21.15.159:0/913932465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43afbb20 0x556a43acddd0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fea60008010 tx=0x7fea600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9216f55cfa33e547 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:09:45.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.127+0000 7fea5be00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:09:45.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.127+0000 7fea5be00640 1 -- 172.21.15.159:0/913932465 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea60017050 con 0x556a43afbb20 2024-04-04T06:09:45.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.127+0000 7fea5be00640 1 -- 172.21.15.159:0/913932465 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fea6000eb00 con 0x556a43afbb20 2024-04-04T06:09:45.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.127+0000 7fea809c2740 1 -- 172.21.15.159:0/913932465 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea68029070 con 0x556a43afbb20 2024-04-04T06:09:45.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.128+0000 7fea5be00640 1 -- 172.21.15.159:0/913932465 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fea6002f050 con 0x556a43afbb20 2024-04-04T06:09:45.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.128+0000 7fea809c2740 10 client.6372 fetch_fsmap learned FSMap version 34 2024-04-04T06:09:45.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.128+0000 7fea809c2740 1 -- 172.21.15.159:0/913932465 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x556a43b02e00 con 0x556a43afbb20 2024-04-04T06:09:45.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.128+0000 7fea5be00640 1 -- 172.21.15.159:0/913932465 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea68029070 con 0x556a43afbb20 2024-04-04T06:09:45.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.128+0000 7fea5be00640 1 -- 172.21.15.159:0/913932465 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7fea60032020 con 0x556a43afbb20 2024-04-04T06:09:45.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.128+0000 7fea809c2740 10 client.6372 fetch_fsmap finished waiting for FSMap version 34 2024-04-04T06:09:45.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.128+0000 7fea809c2740 10 client.6372 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-04T06:09:45.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.128+0000 7fea809c2740 20 client.6372 populate_metadata read hostname 'smithi159' 2024-04-04T06:09:45.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.128+0000 7fea809c2740 1 --2- 172.21.15.159:0/913932465 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x556a43ace310 0x556a43b19b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:09:45.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.128+0000 7fea809c2740 4 client.6372 mds_command: new command op to 6039 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-04T06:09:45.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.128+0000 7fea809c2740 1 -- 172.21.15.159:0/913932465 --> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556a43a40f60 con 0x556a43ace310 2024-04-04T06:09:45.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.128+0000 7fea76a00640 1 --2- 172.21.15.159:0/913932465 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x556a43ace310 0x556a43b19b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:09:45.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.129+0000 7fea76a00640 1 --2- 172.21.15.159:0/913932465 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x556a43ace310 0x556a43b19b90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=727cc9c7f355697e server_cookie=806cfee565f32c78 in_seq=0 out_seq=0 2024-04-04T06:09:45.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.129+0000 7fea5be00640 10 client.6372 ms_handle_connect on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:45.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.129+0000 7fea5be00640 1 -- 172.21.15.159:0/913932465 <== mds.0 v2:172.21.15.159:6833/962475038 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x556a43a40f60 con 0x556a43ace310 2024-04-04T06:09:45.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.129+0000 7fea5be00640 10 client.6372 handle_command_reply: tid=0 2024-04-04T06:09:45.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.129+0000 7fea809c2740 1 client.6372 shutdown 2024-04-04T06:09:45.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.129+0000 7fea809c2740 1 -- 172.21.15.159:0/913932465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43afbb20 msgr2=0x556a43acddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:45.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.129+0000 7fea809c2740 1 --2- 172.21.15.159:0/913932465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43afbb20 0x556a43acddd0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fea60008010 tx=0x7fea600046c0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.129+0000 7fea809c2740 1 -- 172.21.15.159:0/913932465 shutdown_connections 2024-04-04T06:09:45.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.129+0000 7fea809c2740 1 --2- 172.21.15.159:0/913932465 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x556a43ace310 0x556a43b19b90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.129+0000 7fea809c2740 1 --2- 172.21.15.159:0/913932465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a43affe80 0x556a43b19650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.129+0000 7fea809c2740 1 --2- 172.21.15.159:0/913932465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a43afbb20 0x556a43acddd0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.129+0000 7fea809c2740 1 --2- 172.21.15.159:0/913932465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556a43afb740 0x556a43ad2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.129+0000 7fea5be00640 0 client.6372 ms_handle_reset on v2:172.21.15.159:6833/962475038 2024-04-04T06:09:45.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.129+0000 7fea809c2740 1 -- 172.21.15.159:0/913932465 >> 172.21.15.159:0/913932465 conn(0x556a43acec20 msgr2=0x556a43b26e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:45.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.129+0000 7fea809c2740 1 -- 172.21.15.159:0/913932465 shutdown_connections 2024-04-04T06:09:45.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.130+0000 7fea809c2740 1 -- 172.21.15.159:0/913932465 wait complete. 2024-04-04T06:09:45.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.130+0000 7fea809c2740 20 client.6372 trim_cache size 0 max 16384 2024-04-04T06:09:45.127 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:09:45.127 INFO:teuthology.orchestra.run.smithi159.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-04T06:09:45.127 INFO:teuthology.orchestra.run.smithi159.stdout: "scrubs": { 2024-04-04T06:09:45.127 INFO:teuthology.orchestra.run.smithi159.stdout: "3689963f-2d46-4d29-a0eb-8adac8095d39": { 2024-04-04T06:09:45.127 INFO:teuthology.orchestra.run.smithi159.stdout: "path": "/", 2024-04-04T06:09:45.127 INFO:teuthology.orchestra.run.smithi159.stdout: "tag": "3689963f-2d46-4d29-a0eb-8adac8095d39", 2024-04-04T06:09:45.127 INFO:teuthology.orchestra.run.smithi159.stdout: "options": "recursive,repair,force" 2024-04-04T06:09:45.127 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:09:45.127 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:09:45.127 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:09:45.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.130+0000 7fea77e00640 1 -- 172.21.15.159:0/4043058058 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea5406c8f0 msgr2=0x7fea5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:45.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.130+0000 7fea77e00640 1 --2- 172.21.15.159:0/4043058058 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea5406c8f0 0x7fea5406edb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fea68000f60 tx=0x7fea68009040 comp rx=0 tx=0).stop 2024-04-04T06:09:45.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.130+0000 7fea77e00640 1 -- 172.21.15.159:0/4043058058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea78147c80 msgr2=0x7fea780b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:09:45.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.130+0000 7fea77e00640 1 --2- 172.21.15.159:0/4043058058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea78147c80 0x7fea780b0c00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fea70009e50 tx=0x7fea70004a40 comp rx=0 tx=0).stop 2024-04-04T06:09:45.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.130+0000 7fea77e00640 1 -- 172.21.15.159:0/4043058058 shutdown_connections 2024-04-04T06:09:45.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.130+0000 7fea77e00640 1 --2- 172.21.15.159:0/4043058058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea78147c80 0x7fea780b0c00 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.130+0000 7fea77e00640 1 --2- 172.21.15.159:0/4043058058 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea5406c8f0 0x7fea5406edb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.130+0000 7fea77e00640 1 --2- 172.21.15.159:0/4043058058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea780707e0 0x7fea780afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.130+0000 7fea77e00640 1 --2- 172.21.15.159:0/4043058058 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea7806bc80 0x7fea780b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:09:45.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.130+0000 7fea77e00640 1 -- 172.21.15.159:0/4043058058 >> 172.21.15.159:0/4043058058 conn(0x7fea7806e730 msgr2=0x7fea7815fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:09:45.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.131+0000 7fea77e00640 1 -- 172.21.15.159:0/4043058058 shutdown_connections 2024-04-04T06:09:45.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:09:45.131+0000 7fea77e00640 1 -- 172.21.15.159:0/4043058058 wait complete. 2024-04-04T06:09:45.141 INFO:tasks.cephfs.filesystem:scrub status for tag:3689963f-2d46-4d29-a0eb-8adac8095d39 - {'path': '/', 'tag': '3689963f-2d46-4d29-a0eb-8adac8095d39', 'options': 'recursive,repair,force'} 2024-04-04T06:10:11.779 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:10:11.788 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:10:15.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:10:15.142 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-04T06:10:15.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.372+0000 7f023d000640 1 Processor -- start 2024-04-04T06:10:15.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.372+0000 7f023d000640 1 -- start start 2024-04-04T06:10:15.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.373+0000 7f023d000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0238155930 0x7f02380d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.373+0000 7f023d000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f02380d07c0 0x7f02380aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.373+0000 7f023d000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02380b40b0 con 0x7f0238155550 2024-04-04T06:10:15.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.373+0000 7f023d000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f02380b4390 con 0x7f02380d07c0 2024-04-04T06:10:15.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.373+0000 7f023d000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f02380b4670 con 0x7f0238155930 2024-04-04T06:10:15.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.374+0000 7f0236800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f02380d07c0 0x7f02380aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.374+0000 7f022d400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0238155930 0x7f02380d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.374+0000 7f022d400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0238155930 0x7f02380d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:56968/0 (socket says 172.21.15.159:56968) 2024-04-04T06:10:15.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.374+0000 7f022d400640 1 -- 172.21.15.159:0/1996947463 learned_addr learned my addr 172.21.15.159:0/1996947463 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:15.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.375+0000 7f022d400640 1 -- 172.21.15.159:0/1996947463 >> v2:172.21.15.178:3300/0 conn(0x7f02380d07c0 msgr2=0x7f02380aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:15.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.375+0000 7f022d400640 1 --2- 172.21.15.159:0/1996947463 >> v2:172.21.15.178:3300/0 conn(0x7f02380d07c0 0x7f02380aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.375+0000 7f022d400640 1 -- 172.21.15.159:0/1996947463 >> v1:172.21.15.178:6789/0 conn(0x7f0238155550 legacy=0x7f023815f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:15.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.375+0000 7f022d400640 1 -- 172.21.15.159:0/1996947463 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02380b4950 con 0x7f0238155930 2024-04-04T06:10:15.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.376+0000 7f022d400640 1 --2- 172.21.15.159:0/1996947463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0238155930 0x7f02380d0280 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0220009e50 tx=0x7f022002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93e46ebe24b5a02a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:15.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.377+0000 7f0235400640 1 -- 172.21.15.159:0/1996947463 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022002cb00 con 0x7f0238155930 2024-04-04T06:10:15.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.377+0000 7f0235400640 1 -- 172.21.15.159:0/1996947463 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f022002cca0 con 0x7f0238155930 2024-04-04T06:10:15.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.377+0000 7f0235400640 1 -- 172.21.15.159:0/1996947463 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220034610 con 0x7f0238155930 2024-04-04T06:10:15.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.377+0000 7f023d000640 1 -- 172.21.15.159:0/1996947463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0238155930 msgr2=0x7f02380d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:15.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.377+0000 7f023d000640 1 --2- 172.21.15.159:0/1996947463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0238155930 0x7f02380d0280 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0220009e50 tx=0x7f022002bf60 comp rx=0 tx=0).stop 2024-04-04T06:10:15.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.378+0000 7f023d000640 1 -- 172.21.15.159:0/1996947463 shutdown_connections 2024-04-04T06:10:15.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.378+0000 7f023d000640 1 --2- 172.21.15.159:0/1996947463 >> v2:172.21.15.178:3300/0 conn(0x7f02380d07c0 0x7f02380aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.378+0000 7f023d000640 1 --2- 172.21.15.159:0/1996947463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0238155930 0x7f02380d0280 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.378+0000 7f023d000640 1 -- 172.21.15.159:0/1996947463 >> 172.21.15.159:0/1996947463 conn(0x7f023806b750 msgr2=0x7f023806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:15.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.378+0000 7f023d000640 1 -- 172.21.15.159:0/1996947463 shutdown_connections 2024-04-04T06:10:15.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.378+0000 7f023d000640 1 -- 172.21.15.159:0/1996947463 wait complete. 2024-04-04T06:10:15.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.380+0000 7f023d000640 1 Processor -- start 2024-04-04T06:10:15.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.380+0000 7f023d000640 1 -- start start 2024-04-04T06:10:15.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.381+0000 7f023d000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02380d07c0 0x7f02381f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.381+0000 7f023d000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0238155550 0x7f023809aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.381+0000 7f023d000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0238155930 0x7f023809b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.381+0000 7f023d000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f023815bba0 con 0x7f0238155550 2024-04-04T06:10:15.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.381+0000 7f023d000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f023815ba20 con 0x7f02380d07c0 2024-04-04T06:10:15.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.381+0000 7f023d000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f02380b3cf0 con 0x7f0238155930 2024-04-04T06:10:15.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.381+0000 7f022d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0238155550 0x7f023809aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.381+0000 7f022d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0238155550 0x7f023809aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55258/0 (socket says 172.21.15.159:55258) 2024-04-04T06:10:15.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.381+0000 7f022d400640 1 -- 172.21.15.159:0/1830921713 learned_addr learned my addr 172.21.15.159:0/1830921713 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:15.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.381+0000 7f0235e00640 1 --2- 172.21.15.159:0/1830921713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02380d07c0 0x7f02381f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.381+0000 7f0236800640 1 --2- 172.21.15.159:0/1830921713 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0238155930 0x7f023809b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.382+0000 7f022d400640 1 -- 172.21.15.159:0/1830921713 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0238155930 msgr2=0x7f023809b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:15.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.382+0000 7f022d400640 1 --2- 172.21.15.159:0/1830921713 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0238155930 0x7f023809b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.382+0000 7f022d400640 1 -- 172.21.15.159:0/1830921713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02380d07c0 msgr2=0x7f02381f3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:15.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.382+0000 7f022d400640 1 --2- 172.21.15.159:0/1830921713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02380d07c0 0x7f02381f3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.382+0000 7f022d400640 1 -- 172.21.15.159:0/1830921713 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023809a450 con 0x7f0238155550 2024-04-04T06:10:15.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.382+0000 7f0235e00640 1 --2- 172.21.15.159:0/1830921713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02380d07c0 0x7f02381f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:10:15.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.382+0000 7f022d400640 1 --2- 172.21.15.159:0/1830921713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0238155550 0x7f023809aba0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0220036040 tx=0x7f0220004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:15.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.382+0000 7f022ea00640 1 -- 172.21.15.159:0/1830921713 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220043020 con 0x7f0238155550 2024-04-04T06:10:15.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.382+0000 7f022ea00640 1 -- 172.21.15.159:0/1830921713 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f022002ce50 con 0x7f0238155550 2024-04-04T06:10:15.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.382+0000 7f022ea00640 1 -- 172.21.15.159:0/1830921713 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220009510 con 0x7f0238155550 2024-04-04T06:10:15.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.382+0000 7f023d000640 1 -- 172.21.15.159:0/1830921713 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02381f4460 con 0x7f0238155550 2024-04-04T06:10:15.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.383+0000 7f023d000640 1 -- 172.21.15.159:0/1830921713 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02380af210 con 0x7f0238155550 2024-04-04T06:10:15.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.385+0000 7f022ea00640 1 -- 172.21.15.159:0/1830921713 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0220046020 con 0x7f0238155550 2024-04-04T06:10:15.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.386+0000 7f022ea00640 1 --2- 172.21.15.159:0/1830921713 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f021406c8f0 0x7f021406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.386+0000 7f022ea00640 1 -- 172.21.15.159:0/1830921713 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f02200ae1c0 con 0x7f0238155550 2024-04-04T06:10:15.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.386+0000 7f023fe22740 1 Processor -- start 2024-04-04T06:10:15.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.386+0000 7f0235e00640 1 --2- 172.21.15.159:0/1830921713 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f021406c8f0 0x7f021406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.386+0000 7f023fe22740 1 -- start start 2024-04-04T06:10:15.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.387+0000 7f023fe22740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a6537580 0x55a8a650a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.387+0000 7f0235e00640 1 --2- 172.21.15.159:0/1830921713 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f021406c8f0 0x7f021406edb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0228001700 tx=0x7f0228009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:15.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.387+0000 7f023fe22740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a8a6537960 0x55a8a653a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.387+0000 7f023fe22740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a8a650aa60 0x55a8a653a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.387+0000 7f023fe22740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a8a64756d0 con 0x55a8a6537580 2024-04-04T06:10:15.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.387+0000 7f023fe22740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a8a641d770 con 0x55a8a6537960 2024-04-04T06:10:15.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.387+0000 7f023fe22740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a8a6471680 con 0x55a8a650aa60 2024-04-04T06:10:15.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.388+0000 7f0235e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a6537580 0x55a8a650a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.388+0000 7f0235e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a6537580 0x55a8a650a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55260/0 (socket says 172.21.15.159:55260) 2024-04-04T06:10:15.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.388+0000 7f0235e00640 1 -- 172.21.15.159:0/4017201201 learned_addr learned my addr 172.21.15.159:0/4017201201 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:15.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.388+0000 7f0236800640 1 --2- 172.21.15.159:0/4017201201 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a8a650aa60 0x55a8a653a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.388+0000 7f022d400640 1 --2- 172.21.15.159:0/4017201201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a8a6537960 0x55a8a653a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.388+0000 7f0235e00640 1 -- 172.21.15.159:0/4017201201 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a8a650aa60 msgr2=0x55a8a653a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:15.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.388+0000 7f0235e00640 1 --2- 172.21.15.159:0/4017201201 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a8a650aa60 0x55a8a653a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.388+0000 7f0235e00640 1 -- 172.21.15.159:0/4017201201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a8a6537960 msgr2=0x55a8a653a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:15.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.388+0000 7f0235e00640 1 --2- 172.21.15.159:0/4017201201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a8a6537960 0x55a8a653a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.388+0000 7f0235e00640 1 -- 172.21.15.159:0/4017201201 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8a62537d0 con 0x55a8a6537580 2024-04-04T06:10:15.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.388+0000 7f0235e00640 1 --2- 172.21.15.159:0/4017201201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a6537580 0x55a8a650a520 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0228005120 tx=0x7f0228011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae23dbbeafe5fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:15.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.389+0000 7f0226a00640 1 -- 172.21.15.159:0/4017201201 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02280035f0 con 0x55a8a6537580 2024-04-04T06:10:15.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.389+0000 7f0226a00640 1 -- 172.21.15.159:0/4017201201 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0228003790 con 0x55a8a6537580 2024-04-04T06:10:15.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.389+0000 7f0226a00640 1 -- 172.21.15.159:0/4017201201 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228003b60 con 0x55a8a6537580 2024-04-04T06:10:15.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.389+0000 7f023fe22740 1 -- 172.21.15.159:0/4017201201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a6537580 msgr2=0x55a8a650a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:15.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.389+0000 7f023fe22740 1 --2- 172.21.15.159:0/4017201201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a6537580 0x55a8a650a520 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0228005120 tx=0x7f0228011000 comp rx=0 tx=0).stop 2024-04-04T06:10:15.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.389+0000 7f023fe22740 1 -- 172.21.15.159:0/4017201201 shutdown_connections 2024-04-04T06:10:15.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.390+0000 7f023fe22740 1 --2- 172.21.15.159:0/4017201201 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a8a650aa60 0x55a8a653a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.390+0000 7f023fe22740 1 --2- 172.21.15.159:0/4017201201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a8a6537960 0x55a8a653a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.390+0000 7f023fe22740 1 --2- 172.21.15.159:0/4017201201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a6537580 0x55a8a650a520 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.390+0000 7f023fe22740 1 -- 172.21.15.159:0/4017201201 >> 172.21.15.159:0/4017201201 conn(0x55a8a650c790 msgr2=0x55a8a6555310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:15.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.390+0000 7f023fe22740 1 -- 172.21.15.159:0/4017201201 shutdown_connections 2024-04-04T06:10:15.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.390+0000 7f023fe22740 1 -- 172.21.15.159:0/4017201201 wait complete. 2024-04-04T06:10:15.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.390+0000 7f023fe22740 1 -- start start 2024-04-04T06:10:15.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.391+0000 7f023fe22740 1 Processor -- start 2024-04-04T06:10:15.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.392+0000 7f023fe22740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a8a6537580 0x55a8a6536cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.392+0000 7f023fe22740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a8a6537960 0x55a8a653fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.392+0000 7f023fe22740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a653ffc0 0x55a8a6554800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.392+0000 7f023fe22740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a8a650ee30 con 0x55a8a653ffc0 2024-04-04T06:10:15.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.392+0000 7f023fe22740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a8a6430aa0 con 0x55a8a6537580 2024-04-04T06:10:15.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.392+0000 7f023fe22740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a8a6511670 con 0x55a8a6537960 2024-04-04T06:10:15.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.392+0000 7f023fe22740 10 client.0 _ll_register_callbacks cb 0x55a8a6430490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:10:15.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.392+0000 7f0236800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a653ffc0 0x55a8a6554800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.392+0000 7f0236800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a653ffc0 0x55a8a6554800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55272/0 (socket says 172.21.15.159:55272) 2024-04-04T06:10:15.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.392+0000 7f0236800640 1 -- 172.21.15.159:0/3138209181 learned_addr learned my addr 172.21.15.159:0/3138209181 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:15.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.392+0000 7f0235e00640 1 --2- 172.21.15.159:0/3138209181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a8a6537580 0x55a8a6536cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.392+0000 7f022d400640 1 --2- 172.21.15.159:0/3138209181 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a8a6537960 0x55a8a653fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.392+0000 7f0236800640 1 -- 172.21.15.159:0/3138209181 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a8a6537960 msgr2=0x55a8a653fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:15.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.392+0000 7f0236800640 1 --2- 172.21.15.159:0/3138209181 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a8a6537960 0x55a8a653fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.392+0000 7f0236800640 1 -- 172.21.15.159:0/3138209181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a8a6537580 msgr2=0x55a8a6536cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:15.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.392+0000 7f0236800640 1 --2- 172.21.15.159:0/3138209181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a8a6537580 0x55a8a6536cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.392+0000 7f0236800640 1 -- 172.21.15.159:0/3138209181 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8a61893f0 con 0x55a8a653ffc0 2024-04-04T06:10:15.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.393+0000 7f0236800640 1 --2- 172.21.15.159:0/3138209181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a653ffc0 0x55a8a6554800 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0230009c80 tx=0x7f02300136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291f2d04570cbca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:15.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.393+0000 7f021b400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:10:15.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.393+0000 7f021b400640 1 -- 172.21.15.159:0/3138209181 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023001a030 con 0x55a8a653ffc0 2024-04-04T06:10:15.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.393+0000 7f023fe22740 1 -- 172.21.15.159:0/3138209181 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0228003790 con 0x55a8a653ffc0 2024-04-04T06:10:15.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.393+0000 7f021b400640 1 -- 172.21.15.159:0/3138209181 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f0230016460 con 0x55a8a653ffc0 2024-04-04T06:10:15.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.393+0000 7f021b400640 1 -- 172.21.15.159:0/3138209181 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0230005990 con 0x55a8a653ffc0 2024-04-04T06:10:15.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.393+0000 7f023fe22740 10 client.6381 fetch_fsmap learned FSMap version 35 2024-04-04T06:10:15.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.393+0000 7f023fe22740 1 -- 172.21.15.159:0/3138209181 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55a8a61490f0 con 0x55a8a653ffc0 2024-04-04T06:10:15.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.394+0000 7f021b400640 1 -- 172.21.15.159:0/3138209181 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0228003790 con 0x55a8a653ffc0 2024-04-04T06:10:15.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.394+0000 7f021b400640 1 -- 172.21.15.159:0/3138209181 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f0230004020 con 0x55a8a653ffc0 2024-04-04T06:10:15.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.394+0000 7f023fe22740 10 client.6381 fetch_fsmap finished waiting for FSMap version 35 2024-04-04T06:10:15.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.394+0000 7f023fe22740 10 client.6381 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-04T06:10:15.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.394+0000 7f023fe22740 20 client.6381 populate_metadata read hostname 'smithi159' 2024-04-04T06:10:15.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.394+0000 7f023fe22740 1 --2- 172.21.15.159:0/3138209181 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55a8a653ccb0 0x55a8a653d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.394+0000 7f023fe22740 4 client.6381 mds_command: new command op to 6039 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:10:15.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.394+0000 7f023fe22740 1 -- 172.21.15.159:0/3138209181 --> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8a6482c10 con 0x55a8a653ccb0 2024-04-04T06:10:15.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.394+0000 7f0235e00640 1 --2- 172.21.15.159:0/3138209181 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55a8a653ccb0 0x55a8a653d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.395+0000 7f0235e00640 1 --2- 172.21.15.159:0/3138209181 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55a8a653ccb0 0x55a8a653d090 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16a920ef98be6ca0 server_cookie=20e5d1831ca53de9 in_seq=0 out_seq=0 2024-04-04T06:10:15.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.395+0000 7f021b400640 10 client.6381 ms_handle_connect on v2:172.21.15.159:6833/962475038 2024-04-04T06:10:15.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.395+0000 7f021b400640 1 -- 172.21.15.159:0/3138209181 <== mds.0 v2:172.21.15.159:6833/962475038 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a8a6482c10 con 0x55a8a653ccb0 2024-04-04T06:10:15.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.395+0000 7f021b400640 10 client.6381 handle_command_reply: tid=0 2024-04-04T06:10:15.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.396+0000 7f023fe22740 1 client.6381 shutdown 2024-04-04T06:10:15.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.396+0000 7f023fe22740 1 -- 172.21.15.159:0/3138209181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a653ffc0 msgr2=0x55a8a6554800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:15.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.396+0000 7f023fe22740 1 --2- 172.21.15.159:0/3138209181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a653ffc0 0x55a8a6554800 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0230009c80 tx=0x7f02300136b0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.396+0000 7f023fe22740 1 -- 172.21.15.159:0/3138209181 shutdown_connections 2024-04-04T06:10:15.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.396+0000 7f023fe22740 1 --2- 172.21.15.159:0/3138209181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a653ffc0 0x55a8a6554800 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.396+0000 7f023fe22740 1 --2- 172.21.15.159:0/3138209181 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a8a6537960 0x55a8a653fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.396+0000 7f023fe22740 1 --2- 172.21.15.159:0/3138209181 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55a8a653ccb0 0x55a8a653d090 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.396+0000 7f023fe22740 1 --2- 172.21.15.159:0/3138209181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a8a6537580 0x55a8a6536cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.396+0000 7f023fe22740 1 -- 172.21.15.159:0/3138209181 >> 172.21.15.159:0/3138209181 conn(0x55a8a650aa60 msgr2=0x55a8a650e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:15.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.396+0000 7f021b400640 0 client.6381 ms_handle_reset on v2:172.21.15.159:6833/962475038 2024-04-04T06:10:15.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.396+0000 7f023fe22740 1 -- 172.21.15.159:0/3138209181 shutdown_connections 2024-04-04T06:10:15.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.396+0000 7f023fe22740 1 -- 172.21.15.159:0/3138209181 wait complete. 2024-04-04T06:10:15.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.396+0000 7f023fe22740 20 client.6381 trim_cache size 0 max 16384 2024-04-04T06:10:15.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.415+0000 7f023fe22740 1 Processor -- start 2024-04-04T06:10:15.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.415+0000 7f023fe22740 1 -- start start 2024-04-04T06:10:15.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.416+0000 7f023fe22740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a8a6537580 0x55a8a653d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.416+0000 7f023fe22740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a6537960 0x55a8a6552220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.416+0000 7f023fe22740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a8a653ffc0 0x55a8a6552820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.416+0000 7f023fe22740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0238200cb0 con 0x55a8a6537960 2024-04-04T06:10:15.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.416+0000 7f023fe22740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f02381ffdc0 con 0x55a8a6537580 2024-04-04T06:10:15.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.416+0000 7f023fe22740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f023809d430 con 0x55a8a653ffc0 2024-04-04T06:10:15.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.416+0000 7f022d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a6537960 0x55a8a6552220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.416+0000 7f022d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a6537960 0x55a8a6552220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55282/0 (socket says 172.21.15.159:55282) 2024-04-04T06:10:15.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.416+0000 7f022d400640 1 -- 172.21.15.159:0/2272253264 learned_addr learned my addr 172.21.15.159:0/2272253264 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:15.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.417+0000 7f022d400640 1 -- 172.21.15.159:0/2272253264 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a8a653ffc0 msgr2=0x55a8a6552820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:15.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.417+0000 7f022d400640 1 --2- 172.21.15.159:0/2272253264 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a8a653ffc0 0x55a8a6552820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.417+0000 7f022d400640 1 -- 172.21.15.159:0/2272253264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a8a6537580 msgr2=0x55a8a653d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:15.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.417+0000 7f022d400640 1 --2- 172.21.15.159:0/2272253264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a8a6537580 0x55a8a653d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.417+0000 7f022d400640 1 -- 172.21.15.159:0/2272253264 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8a6511250 con 0x55a8a6537960 2024-04-04T06:10:15.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.417+0000 7f022d400640 1 --2- 172.21.15.159:0/2272253264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a6537960 0x55a8a6552220 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0220009e50 tx=0x7f0220009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8fefac3542a564 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:15.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.417+0000 7f0226000640 1 -- 172.21.15.159:0/2272253264 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022007be50 con 0x55a8a6537960 2024-04-04T06:10:15.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.417+0000 7f0226000640 1 -- 172.21.15.159:0/2272253264 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f022007e070 con 0x55a8a6537960 2024-04-04T06:10:15.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.417+0000 7f0226000640 1 -- 172.21.15.159:0/2272253264 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220080200 con 0x55a8a6537960 2024-04-04T06:10:15.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.418+0000 7f023fe22740 1 -- 172.21.15.159:0/2272253264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a6537960 msgr2=0x55a8a6552220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:15.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.418+0000 7f023fe22740 1 --2- 172.21.15.159:0/2272253264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a6537960 0x55a8a6552220 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0220009e50 tx=0x7f0220009040 comp rx=0 tx=0).stop 2024-04-04T06:10:15.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.418+0000 7f023fe22740 1 -- 172.21.15.159:0/2272253264 shutdown_connections 2024-04-04T06:10:15.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.418+0000 7f023fe22740 1 --2- 172.21.15.159:0/2272253264 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a8a653ffc0 0x55a8a6552820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.418+0000 7f023fe22740 1 --2- 172.21.15.159:0/2272253264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a6537960 0x55a8a6552220 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.418+0000 7f023fe22740 1 --2- 172.21.15.159:0/2272253264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a8a6537580 0x55a8a653d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.418+0000 7f023fe22740 1 -- 172.21.15.159:0/2272253264 >> 172.21.15.159:0/2272253264 conn(0x55a8a650aa60 msgr2=0x55a8a653eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:15.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.418+0000 7f023fe22740 1 -- 172.21.15.159:0/2272253264 shutdown_connections 2024-04-04T06:10:15.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.418+0000 7f023fe22740 1 -- 172.21.15.159:0/2272253264 wait complete. 2024-04-04T06:10:15.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.419+0000 7f023fe22740 1 -- start start 2024-04-04T06:10:15.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.419+0000 7f023fe22740 1 Processor -- start 2024-04-04T06:10:15.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.420+0000 7f023fe22740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a8a6537580 0x55a8a65ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.420+0000 7f023fe22740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a6537960 0x55a8a6556f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.420+0000 7f023fe22740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a8a653ffc0 0x55a8a6557460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.420+0000 7f023fe22740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f023809da80 con 0x55a8a6537960 2024-04-04T06:10:15.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.420+0000 7f023fe22740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a8a653fca0 con 0x55a8a6537580 2024-04-04T06:10:15.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.420+0000 7f023fe22740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a8a65cf1d0 con 0x55a8a653ffc0 2024-04-04T06:10:15.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.420+0000 7f023fe22740 10 client.0 _ll_register_callbacks cb 0x7f0230009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:10:15.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.420+0000 7f022d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a6537960 0x55a8a6556f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.420+0000 7f022d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a6537960 0x55a8a6556f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55286/0 (socket says 172.21.15.159:55286) 2024-04-04T06:10:15.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.420+0000 7f022d400640 1 -- 172.21.15.159:0/305426406 learned_addr learned my addr 172.21.15.159:0/305426406 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:15.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.420+0000 7f0235e00640 1 --2- 172.21.15.159:0/305426406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a8a6537580 0x55a8a65ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.420+0000 7f022d400640 1 -- 172.21.15.159:0/305426406 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a8a653ffc0 msgr2=0x55a8a6557460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:15.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.420+0000 7f0236800640 1 --2- 172.21.15.159:0/305426406 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a8a653ffc0 0x55a8a6557460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.420+0000 7f022d400640 1 --2- 172.21.15.159:0/305426406 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a8a653ffc0 0x55a8a6557460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.420+0000 7f022d400640 1 -- 172.21.15.159:0/305426406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a8a6537580 msgr2=0x55a8a65ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:15.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.420+0000 7f022d400640 1 --2- 172.21.15.159:0/305426406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a8a6537580 0x55a8a65ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.420+0000 7f022d400640 1 -- 172.21.15.159:0/305426406 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8a6566410 con 0x55a8a6537960 2024-04-04T06:10:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.421+0000 7f0235e00640 1 --2- 172.21.15.159:0/305426406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a8a6537580 0x55a8a65ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.421+0000 7f022d400640 1 --2- 172.21.15.159:0/305426406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a6537960 0x55a8a6556f20 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f022007bde0 tx=0x7f02200ac520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b33b2051158fc server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.421+0000 7f021b400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:10:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.421+0000 7f021b400640 1 -- 172.21.15.159:0/305426406 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02200ab050 con 0x55a8a6537960 2024-04-04T06:10:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.421+0000 7f021b400640 1 -- 172.21.15.159:0/305426406 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f02200ac290 con 0x55a8a6537960 2024-04-04T06:10:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.421+0000 7f023fe22740 1 -- 172.21.15.159:0/305426406 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f022007e070 con 0x55a8a6537960 2024-04-04T06:10:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.421+0000 7f021b400640 1 -- 172.21.15.159:0/305426406 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0220080200 con 0x55a8a6537960 2024-04-04T06:10:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.421+0000 7f023fe22740 10 client.6387 fetch_fsmap learned FSMap version 35 2024-04-04T06:10:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.421+0000 7f023fe22740 1 -- 172.21.15.159:0/305426406 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55a8a653b700 con 0x55a8a6537960 2024-04-04T06:10:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.422+0000 7f021b400640 1 -- 172.21.15.159:0/305426406 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f022007e070 con 0x55a8a6537960 2024-04-04T06:10:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.422+0000 7f021b400640 1 -- 172.21.15.159:0/305426406 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f022002cb10 con 0x55a8a6537960 2024-04-04T06:10:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.422+0000 7f023fe22740 10 client.6387 fetch_fsmap finished waiting for FSMap version 35 2024-04-04T06:10:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.422+0000 7f023fe22740 10 client.6387 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-04T06:10:15.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.422+0000 7f023fe22740 20 client.6387 populate_metadata read hostname 'smithi159' 2024-04-04T06:10:15.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.422+0000 7f023fe22740 1 --2- 172.21.15.159:0/305426406 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55a8a65794f0 0x55a8a65093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.422+0000 7f023fe22740 4 client.6387 mds_command: new command op to 6039 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-04T06:10:15.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.422+0000 7f023fe22740 1 -- 172.21.15.159:0/305426406 --> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a8a647cf60 con 0x55a8a65794f0 2024-04-04T06:10:15.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.422+0000 7f0235e00640 1 --2- 172.21.15.159:0/305426406 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55a8a65794f0 0x55a8a65093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.423+0000 7f021b400640 10 client.6387 ms_handle_connect on v2:172.21.15.159:6833/962475038 2024-04-04T06:10:15.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.423+0000 7f0235e00640 1 --2- 172.21.15.159:0/305426406 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55a8a65794f0 0x55a8a65093b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f42d888362088d1d server_cookie=e9cadafde2b7095f in_seq=0 out_seq=0 2024-04-04T06:10:15.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.423+0000 7f021b400640 1 -- 172.21.15.159:0/305426406 <== mds.0 v2:172.21.15.159:6833/962475038 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a8a647cf60 con 0x55a8a65794f0 2024-04-04T06:10:15.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.423+0000 7f021b400640 10 client.6387 handle_command_reply: tid=0 2024-04-04T06:10:15.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.423+0000 7f023fe22740 1 client.6387 shutdown 2024-04-04T06:10:15.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.423+0000 7f023fe22740 1 -- 172.21.15.159:0/305426406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a6537960 msgr2=0x55a8a6556f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:15.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.423+0000 7f023fe22740 1 --2- 172.21.15.159:0/305426406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a6537960 0x55a8a6556f20 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f022007bde0 tx=0x7f02200ac520 comp rx=0 tx=0).stop 2024-04-04T06:10:15.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.423+0000 7f023fe22740 1 -- 172.21.15.159:0/305426406 shutdown_connections 2024-04-04T06:10:15.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.423+0000 7f023fe22740 1 --2- 172.21.15.159:0/305426406 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a8a653ffc0 0x55a8a6557460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.423+0000 7f023fe22740 1 --2- 172.21.15.159:0/305426406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8a6537960 0x55a8a6556f20 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.423+0000 7f023fe22740 1 --2- 172.21.15.159:0/305426406 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55a8a65794f0 0x55a8a65093b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.423+0000 7f023fe22740 1 --2- 172.21.15.159:0/305426406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a8a6537580 0x55a8a65ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.423+0000 7f023fe22740 1 -- 172.21.15.159:0/305426406 >> 172.21.15.159:0/305426406 conn(0x55a8a650aa60 msgr2=0x55a8a6552220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:15.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.423+0000 7f021b400640 0 client.6387 ms_handle_reset on v2:172.21.15.159:6833/962475038 2024-04-04T06:10:15.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.423+0000 7f023fe22740 1 -- 172.21.15.159:0/305426406 shutdown_connections 2024-04-04T06:10:15.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.423+0000 7f023fe22740 1 -- 172.21.15.159:0/305426406 wait complete. 2024-04-04T06:10:15.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.424+0000 7f023fe22740 20 client.6387 trim_cache size 0 max 16384 2024-04-04T06:10:15.421 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:10:15.421 INFO:teuthology.orchestra.run.smithi159.stdout: "status": "no active scrubs running", 2024-04-04T06:10:15.421 INFO:teuthology.orchestra.run.smithi159.stdout: "scrubs": {} 2024-04-04T06:10:15.421 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:10:15.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.424+0000 7f023d000640 1 -- 172.21.15.159:0/1830921713 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f021406c8f0 msgr2=0x7f021406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:15.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.424+0000 7f023d000640 1 --2- 172.21.15.159:0/1830921713 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f021406c8f0 0x7f021406edb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0228001700 tx=0x7f0228009040 comp rx=0 tx=0).stop 2024-04-04T06:10:15.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.424+0000 7f023d000640 1 -- 172.21.15.159:0/1830921713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0238155550 msgr2=0x7f023809aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:15.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.424+0000 7f023d000640 1 --2- 172.21.15.159:0/1830921713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0238155550 0x7f023809aba0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0220036040 tx=0x7f0220004a40 comp rx=0 tx=0).stop 2024-04-04T06:10:15.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.424+0000 7f023d000640 1 -- 172.21.15.159:0/1830921713 shutdown_connections 2024-04-04T06:10:15.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.424+0000 7f023d000640 1 --2- 172.21.15.159:0/1830921713 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0238155930 0x7f023809b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.424+0000 7f023d000640 1 --2- 172.21.15.159:0/1830921713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0238155550 0x7f023809aba0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.424+0000 7f023d000640 1 --2- 172.21.15.159:0/1830921713 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f021406c8f0 0x7f021406edb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.424+0000 7f023d000640 1 --2- 172.21.15.159:0/1830921713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02380d07c0 0x7f02381f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.424+0000 7f023d000640 1 -- 172.21.15.159:0/1830921713 >> 172.21.15.159:0/1830921713 conn(0x7f023806b750 msgr2=0x7f02380b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:15.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.424+0000 7f023d000640 1 -- 172.21.15.159:0/1830921713 shutdown_connections 2024-04-04T06:10:15.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.424+0000 7f023d000640 1 -- 172.21.15.159:0/1830921713 wait complete. 2024-04-04T06:10:15.435 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-04T06:10:15.435 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:10:15.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.642+0000 7f0ee2e00640 1 Processor -- start 2024-04-04T06:10:15.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.643+0000 7f0ee2e00640 1 -- start start 2024-04-04T06:10:15.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.644+0000 7f0ee2e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0edc15e7e0 0x7f0edc0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.644+0000 7f0ee2e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0edc06ca00 0x7f0edc0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.644+0000 7f0ee2e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0edc0d2d60 0x7f0edc0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.644+0000 7f0ee2e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f0edc15ebc0 con 0x7f0edc0d2d60 2024-04-04T06:10:15.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.644+0000 7f0ee2e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0edc06d1c0 con 0x7f0edc15e7e0 2024-04-04T06:10:15.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.644+0000 7f0ee2e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0edc06d4a0 con 0x7f0edc06ca00 2024-04-04T06:10:15.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.644+0000 7f0ee0e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0edc0d2d60 0x7f0edc0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.644+0000 7f0ee0e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0edc0d2d60 0x7f0edc0cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55288/0 (socket says 172.21.15.159:55288) 2024-04-04T06:10:15.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.644+0000 7f0ee0e00640 1 -- 172.21.15.159:0/1166195808 learned_addr learned my addr 172.21.15.159:0/1166195808 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:15.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.645+0000 7f0ed3400640 1 --2- 172.21.15.159:0/1166195808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0edc06ca00 0x7f0edc0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.645+0000 7f0edbe00640 1 --2- 172.21.15.159:0/1166195808 >> v2:172.21.15.178:3300/0 conn(0x7f0edc15e7e0 0x7f0edc0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.645+0000 7f0ee0e00640 1 -- 172.21.15.159:0/1166195808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0edc06ca00 msgr2=0x7f0edc0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:15.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.645+0000 7f0ee0e00640 1 --2- 172.21.15.159:0/1166195808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0edc06ca00 0x7f0edc0d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.645+0000 7f0ee0e00640 1 -- 172.21.15.159:0/1166195808 >> v2:172.21.15.178:3300/0 conn(0x7f0edc15e7e0 msgr2=0x7f0edc0d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:15.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.645+0000 7f0ee0e00640 1 --2- 172.21.15.159:0/1166195808 >> v2:172.21.15.178:3300/0 conn(0x7f0edc15e7e0 0x7f0edc0d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.645+0000 7f0ee0e00640 1 -- 172.21.15.159:0/1166195808 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0edc0cf320 con 0x7f0edc0d2d60 2024-04-04T06:10:15.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.647+0000 7f0ee0e00640 1 --2- 172.21.15.159:0/1166195808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0edc0d2d60 0x7f0edc0cfb20 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0d43b0 tx=0x7f0ed402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6790bf1101c7e9d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:15.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.647+0000 7f0edb400640 1 -- 172.21.15.159:0/1166195808 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed402e070 con 0x7f0edc0d2d60 2024-04-04T06:10:15.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.647+0000 7f0edb400640 1 -- 172.21.15.159:0/1166195808 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ed4037070 con 0x7f0edc0d2d60 2024-04-04T06:10:15.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.647+0000 7f0edb400640 1 -- 172.21.15.159:0/1166195808 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4031a00 con 0x7f0edc0d2d60 2024-04-04T06:10:15.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.648+0000 7f0ee2e00640 1 -- 172.21.15.159:0/1166195808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0edc0d2d60 msgr2=0x7f0edc0cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:15.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.648+0000 7f0ee2e00640 1 --2- 172.21.15.159:0/1166195808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0edc0d2d60 0x7f0edc0cfb20 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0d43b0 tx=0x7f0ed402b880 comp rx=0 tx=0).stop 2024-04-04T06:10:15.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.648+0000 7f0ee2e00640 1 -- 172.21.15.159:0/1166195808 shutdown_connections 2024-04-04T06:10:15.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.648+0000 7f0ee2e00640 1 --2- 172.21.15.159:0/1166195808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0edc0d2d60 0x7f0edc0cfb20 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.648+0000 7f0ee2e00640 1 --2- 172.21.15.159:0/1166195808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0edc06ca00 0x7f0edc0d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.648+0000 7f0ee2e00640 1 --2- 172.21.15.159:0/1166195808 >> v2:172.21.15.178:3300/0 conn(0x7f0edc15e7e0 0x7f0edc0d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.648+0000 7f0ee2e00640 1 -- 172.21.15.159:0/1166195808 >> 172.21.15.159:0/1166195808 conn(0x7f0edc09d7c0 msgr2=0x7f0edc06bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:15.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.648+0000 7f0ee2e00640 1 -- 172.21.15.159:0/1166195808 shutdown_connections 2024-04-04T06:10:15.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.648+0000 7f0ee2e00640 1 -- 172.21.15.159:0/1166195808 wait complete. 2024-04-04T06:10:15.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.650+0000 7f0ee2e00640 1 Processor -- start 2024-04-04T06:10:15.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.650+0000 7f0ee2e00640 1 -- start start 2024-04-04T06:10:15.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.651+0000 7f0ee2e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0edc06ca00 0x7f0edc0b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.651+0000 7f0ee2e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0edc0d2d60 0x7f0edc0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.651+0000 7f0ee2e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0edc15e7e0 0x7f0edc152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.651+0000 7f0ee2e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc06ce60 con 0x7f0edc0d2d60 2024-04-04T06:10:15.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.651+0000 7f0ee2e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc0cefd0 con 0x7f0edc15e7e0 2024-04-04T06:10:15.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.651+0000 7f0ee2e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0edc0d0060 con 0x7f0edc06ca00 2024-04-04T06:10:15.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.652+0000 7f0ed3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0edc0d2d60 0x7f0edc0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.652+0000 7f0ed3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0edc0d2d60 0x7f0edc0b6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55302/0 (socket says 172.21.15.159:55302) 2024-04-04T06:10:15.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.652+0000 7f0ed3400640 1 -- 172.21.15.159:0/1909092003 learned_addr learned my addr 172.21.15.159:0/1909092003 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:15.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.652+0000 7f0edbe00640 1 --2- 172.21.15.159:0/1909092003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0edc06ca00 0x7f0edc0b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.652+0000 7f0ee0e00640 1 --2- 172.21.15.159:0/1909092003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0edc15e7e0 0x7f0edc152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.652+0000 7f0ed3400640 1 -- 172.21.15.159:0/1909092003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0edc06ca00 msgr2=0x7f0edc0b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:15.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.652+0000 7f0ed3400640 1 --2- 172.21.15.159:0/1909092003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0edc06ca00 0x7f0edc0b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.652+0000 7f0ed3400640 1 -- 172.21.15.159:0/1909092003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0edc15e7e0 msgr2=0x7f0edc152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:15.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.652+0000 7f0ed3400640 1 --2- 172.21.15.159:0/1909092003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0edc15e7e0 0x7f0edc152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.652+0000 7f0ed3400640 1 -- 172.21.15.159:0/1909092003 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0edc1538a0 con 0x7f0edc0d2d60 2024-04-04T06:10:15.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.652+0000 7f0ee0e00640 1 --2- 172.21.15.159:0/1909092003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0edc15e7e0 0x7f0edc152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:15.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.652+0000 7f0ed3400640 1 --2- 172.21.15.159:0/1909092003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0edc0d2d60 0x7f0edc0b6120 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400d830 tx=0x7f0ec400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:15.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.653+0000 7f0ed8c00640 1 -- 172.21.15.159:0/1909092003 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec400b6c0 con 0x7f0edc0d2d60 2024-04-04T06:10:15.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.653+0000 7f0ed8c00640 1 -- 172.21.15.159:0/1909092003 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ec4019070 con 0x7f0edc0d2d60 2024-04-04T06:10:15.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.653+0000 7f0ed8c00640 1 -- 172.21.15.159:0/1909092003 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec400ba80 con 0x7f0edc0d2d60 2024-04-04T06:10:15.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.653+0000 7f0ee2e00640 1 -- 172.21.15.159:0/1909092003 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0edc153b80 con 0x7f0edc0d2d60 2024-04-04T06:10:15.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.653+0000 7f0ee2e00640 1 -- 172.21.15.159:0/1909092003 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0edc154140 con 0x7f0edc0d2d60 2024-04-04T06:10:15.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.654+0000 7f0ee2e00640 1 -- 172.21.15.159:0/1909092003 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0edc0685c0 con 0x7f0edc0d2d60 2024-04-04T06:10:15.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.655+0000 7f0ed8c00640 1 -- 172.21.15.159:0/1909092003 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ec401e460 con 0x7f0edc0d2d60 2024-04-04T06:10:15.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.656+0000 7f0ed8c00640 1 --2- 172.21.15.159:0/1909092003 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0eb806c8f0 0x7f0eb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:15.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.656+0000 7f0ed8c00640 1 -- 172.21.15.159:0/1909092003 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f0ec408c290 con 0x7f0edc0d2d60 2024-04-04T06:10:15.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.656+0000 7f0edbe00640 1 --2- 172.21.15.159:0/1909092003 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0eb806c8f0 0x7f0eb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:15.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.657+0000 7f0edbe00640 1 --2- 172.21.15.159:0/1909092003 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0eb806c8f0 0x7f0eb806edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009940 tx=0x7f0ecc003cd0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:15.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.661+0000 7f0ed8c00640 1 -- 172.21.15.159:0/1909092003 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0ec405a6c0 con 0x7f0edc0d2d60 2024-04-04T06:10:15.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.901+0000 7f0ee2e00640 1 -- 172.21.15.159:0/1909092003 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0edc0d5300 con 0x7f0edc0d2d60 2024-04-04T06:10:15.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.901+0000 7f0ed8c00640 1 -- 172.21.15.159:0/1909092003 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5188 (secure 0 0 0) 0x7f0ec4059fc0 con 0x7f0edc0d2d60 2024-04-04T06:10:15.895 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:15.895 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5191,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3507245876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3507245876},{"type":"v1","addr":"172.21.15.159:6835","nonce":3507245876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/967262871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":967262871},{"type":"v1","addr":"172.21.15.178:6835","nonce":967262871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2969351496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2969351496},{"type":"v1","addr":"172.21.15.178:6837","nonce":2969351496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:08:57.020444+0000","modified":"2024-04-04T06:09:49.628589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6039":{"gid":6039,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":16,"addr":"172.21.15.159:6834/962475038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":962475038},{"type":"v1","addr":"172.21.15.159:6834","nonce":962475038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6039,"qdb_cluster":[6039]},"id":4}]} 2024-04-04T06:10:15.895 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 35 2024-04-04T06:10:15.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.904+0000 7f0ee2e00640 1 -- 172.21.15.159:0/1909092003 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0eb806c8f0 msgr2=0x7f0eb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:15.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.904+0000 7f0ee2e00640 1 --2- 172.21.15.159:0/1909092003 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0eb806c8f0 0x7f0eb806edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009940 tx=0x7f0ecc003cd0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.904+0000 7f0ee2e00640 1 -- 172.21.15.159:0/1909092003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0edc0d2d60 msgr2=0x7f0edc0b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:15.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.904+0000 7f0ee2e00640 1 --2- 172.21.15.159:0/1909092003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0edc0d2d60 0x7f0edc0b6120 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400d830 tx=0x7f0ec400dcf0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.904+0000 7f0ee2e00640 1 -- 172.21.15.159:0/1909092003 shutdown_connections 2024-04-04T06:10:15.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.904+0000 7f0ee2e00640 1 --2- 172.21.15.159:0/1909092003 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0eb806c8f0 0x7f0eb806edb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.904+0000 7f0ee2e00640 1 --2- 172.21.15.159:0/1909092003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0edc15e7e0 0x7f0edc152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.904+0000 7f0ee2e00640 1 --2- 172.21.15.159:0/1909092003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0edc0d2d60 0x7f0edc0b6120 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.904+0000 7f0ee2e00640 1 --2- 172.21.15.159:0/1909092003 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0edc06ca00 0x7f0edc0b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:15.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.904+0000 7f0ee2e00640 1 -- 172.21.15.159:0/1909092003 >> 172.21.15.159:0/1909092003 conn(0x7f0edc09d7c0 msgr2=0x7f0edc0b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:15.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.904+0000 7f0ee2e00640 1 -- 172.21.15.159:0/1909092003 shutdown_connections 2024-04-04T06:10:15.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:15.904+0000 7f0ee2e00640 1 -- 172.21.15.159:0/1909092003 wait complete. 2024-04-04T06:10:15.916 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-04T06:10:16.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.132+0000 7fd33d400640 1 Processor -- start 2024-04-04T06:10:16.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.132+0000 7fd33d400640 1 -- start start 2024-04-04T06:10:16.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.134+0000 7fd33d400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd33806dd30 0x7fd3380cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.134+0000 7fd33d400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3380ce890 con 0x7fd33813f190 2024-04-04T06:10:16.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.134+0000 7fd33d400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3380ceb50 con 0x7fd3380cc880 2024-04-04T06:10:16.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.134+0000 7fd33d400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd3380cd230 con 0x7fd33806dd30 2024-04-04T06:10:16.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.134+0000 7fd337e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fd33813f190 0x7fd3380cf630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45802/0 (socket says 172.21.15.159:45802) 2024-04-04T06:10:16.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.135+0000 7fd337e00640 1 -- 172.21.15.159:0/3322416885 learned_addr learned my addr 172.21.15.159:0/3322416885 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:16.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.135+0000 7fd33ca00640 1 --2- 172.21.15.159:0/3322416885 >> v2:172.21.15.178:3300/0 conn(0x7fd33806dd30 0x7fd3380cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.135+0000 7fd337400640 1 -- 172.21.15.159:0/3322416885 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1653865138 0 0) 0x7fd3380ce890 con 0x7fd33813f190 2024-04-04T06:10:16.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.135+0000 7fd337400640 1 -- 172.21.15.159:0/3322416885 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd318003520 con 0x7fd33813f190 2024-04-04T06:10:16.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.135+0000 7fd337400640 1 -- 172.21.15.159:0/3322416885 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3665864100 0 0) 0x7fd318003520 con 0x7fd33813f190 2024-04-04T06:10:16.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.135+0000 7fd337400640 1 -- 172.21.15.159:0/3322416885 >> v2:172.21.15.178:3300/0 conn(0x7fd33806dd30 msgr2=0x7fd3380cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:16.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.135+0000 7fd337400640 1 --2- 172.21.15.159:0/3322416885 >> v2:172.21.15.178:3300/0 conn(0x7fd33806dd30 0x7fd3380cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.135+0000 7fd337400640 1 -- 172.21.15.159:0/3322416885 >> v1:172.21.15.178:6789/0 conn(0x7fd3380cc880 legacy=0x7fd33806d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:16.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.136+0000 7fd337400640 1 -- 172.21.15.159:0/3322416885 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3380cee30 con 0x7fd33813f190 2024-04-04T06:10:16.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.136+0000 7fd337400640 1 -- 172.21.15.159:0/3322416885 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fd328001670 con 0x7fd33813f190 2024-04-04T06:10:16.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.136+0000 7fd337400640 1 -- 172.21.15.159:0/3322416885 >> v1:172.21.15.159:6789/0 conn(0x7fd33813f190 legacy=0x7fd3380cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:16.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.136+0000 7fd337400640 1 --2- 172.21.15.159:0/3322416885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3180039b0 0x7fd318003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.136+0000 7fd337400640 1 --2- 172.21.15.159:0/3322416885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd318004800 0x7fd318004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.136+0000 7fd337400640 1 --2- 172.21.15.159:0/3322416885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3180051c0 0x7fd31800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.136+0000 7fd337400640 1 -- 172.21.15.159:0/3322416885 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd3380cd230 con 0x7fd3180039b0 2024-04-04T06:10:16.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.136+0000 7fd337400640 1 -- 172.21.15.159:0/3322416885 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd31800c060 con 0x7fd318004800 2024-04-04T06:10:16.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.136+0000 7fd337400640 1 -- 172.21.15.159:0/3322416885 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd31800c260 con 0x7fd3180051c0 2024-04-04T06:10:16.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.136+0000 7fd337e00640 1 --2- 172.21.15.159:0/3322416885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3180039b0 0x7fd318003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.136+0000 7fd337e00640 1 -- 172.21.15.159:0/3322416885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3180051c0 msgr2=0x7fd31800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:16.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.136+0000 7fd337e00640 1 --2- 172.21.15.159:0/3322416885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3180051c0 0x7fd31800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.136+0000 7fd337e00640 1 -- 172.21.15.159:0/3322416885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd318004800 msgr2=0x7fd318004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:16.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.136+0000 7fd337e00640 1 --2- 172.21.15.159:0/3322416885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd318004800 0x7fd318004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.136+0000 7fd337e00640 1 -- 172.21.15.159:0/3322416885 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd318003520 con 0x7fd3180039b0 2024-04-04T06:10:16.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.136+0000 7fd33ca00640 1 -- 172.21.15.159:0/3322416885 reap_dead start 2024-04-04T06:10:16.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.137+0000 7fd337e00640 1 --2- 172.21.15.159:0/3322416885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3180039b0 0x7fd318003dd0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd32800c240 tx=0x7fd318005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613e55c881ec1173 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:16.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.138+0000 7fd337400640 1 -- 172.21.15.159:0/3322416885 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd328007d10 con 0x7fd3180039b0 2024-04-04T06:10:16.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.138+0000 7fd337400640 1 -- 172.21.15.159:0/3322416885 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd328030070 con 0x7fd3180039b0 2024-04-04T06:10:16.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.138+0000 7fd337400640 1 -- 172.21.15.159:0/3322416885 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd328005d90 con 0x7fd3180039b0 2024-04-04T06:10:16.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.138+0000 7fd33d400640 1 -- 172.21.15.159:0/3322416885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3180039b0 msgr2=0x7fd318003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:16.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.138+0000 7fd33d400640 1 --2- 172.21.15.159:0/3322416885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3180039b0 0x7fd318003dd0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd32800c240 tx=0x7fd318005830 comp rx=0 tx=0).stop 2024-04-04T06:10:16.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.138+0000 7fd33d400640 1 -- 172.21.15.159:0/3322416885 shutdown_connections 2024-04-04T06:10:16.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.138+0000 7fd33d400640 1 --2- 172.21.15.159:0/3322416885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3180039b0 0x7fd318003dd0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.138+0000 7fd33d400640 1 -- 172.21.15.159:0/3322416885 >> 172.21.15.159:0/3322416885 conn(0x7fd3380b1ba0 msgr2=0x7fd3380b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:16.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.139+0000 7fd33d400640 1 -- 172.21.15.159:0/3322416885 shutdown_connections 2024-04-04T06:10:16.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.139+0000 7fd33d400640 1 -- 172.21.15.159:0/3322416885 wait complete. 2024-04-04T06:10:16.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.140+0000 7fd33d400640 1 Processor -- start 2024-04-04T06:10:16.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.140+0000 7fd33d400640 1 -- start start 2024-04-04T06:10:16.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.141+0000 7fd33d400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3180039b0 0x7fd338155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.141+0000 7fd33d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd33815a9c0 0x7fd33815ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.141+0000 7fd33d400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3380720f0 0x7fd338072570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.141+0000 7fd33d400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd33813f570 con 0x7fd33815a9c0 2024-04-04T06:10:16.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.141+0000 7fd33d400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd338160c70 con 0x7fd3180039b0 2024-04-04T06:10:16.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.141+0000 7fd33d400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd3380cccf0 con 0x7fd3380720f0 2024-04-04T06:10:16.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.142+0000 7fd32f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd33815a9c0 0x7fd33815ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.142+0000 7fd32f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd33815a9c0 0x7fd33815ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55330/0 (socket says 172.21.15.159:55330) 2024-04-04T06:10:16.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.142+0000 7fd32f400640 1 -- 172.21.15.159:0/1057754050 learned_addr learned my addr 172.21.15.159:0/1057754050 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:16.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.142+0000 7fd32f400640 1 -- 172.21.15.159:0/1057754050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3380720f0 msgr2=0x7fd338072570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:16.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.142+0000 7fd32f400640 1 --2- 172.21.15.159:0/1057754050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3380720f0 0x7fd338072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.142+0000 7fd32f400640 1 -- 172.21.15.159:0/1057754050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3180039b0 msgr2=0x7fd338155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:16.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.142+0000 7fd32f400640 1 --2- 172.21.15.159:0/1057754050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3180039b0 0x7fd338155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.142+0000 7fd32f400640 1 -- 172.21.15.159:0/1057754050 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3380b3fb0 con 0x7fd33815a9c0 2024-04-04T06:10:16.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.142+0000 7fd32f400640 1 --2- 172.21.15.159:0/1057754050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd33815a9c0 0x7fd33815ae40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd32000c460 tx=0x7fd32000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:16.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.143+0000 7fd334c00640 1 -- 172.21.15.159:0/1057754050 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd320017070 con 0x7fd33815a9c0 2024-04-04T06:10:16.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.143+0000 7fd334c00640 1 -- 172.21.15.159:0/1057754050 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3200052f0 con 0x7fd33815a9c0 2024-04-04T06:10:16.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.143+0000 7fd334c00640 1 -- 172.21.15.159:0/1057754050 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd320003830 con 0x7fd33815a9c0 2024-04-04T06:10:16.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.143+0000 7fd33d400640 1 -- 172.21.15.159:0/1057754050 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3380b4250 con 0x7fd33815a9c0 2024-04-04T06:10:16.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.143+0000 7fd33d400640 1 -- 172.21.15.159:0/1057754050 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3380b4810 con 0x7fd33815a9c0 2024-04-04T06:10:16.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.145+0000 7fd334c00640 1 -- 172.21.15.159:0/1057754050 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd32001a070 con 0x7fd33815a9c0 2024-04-04T06:10:16.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.146+0000 7fd334c00640 1 --2- 172.21.15.159:0/1057754050 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd31406c8f0 0x7fd31406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.146+0000 7fd334c00640 1 -- 172.21.15.159:0/1057754050 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fd32008af50 con 0x7fd33815a9c0 2024-04-04T06:10:16.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.146+0000 7fd337e00640 1 --2- 172.21.15.159:0/1057754050 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd31406c8f0 0x7fd31406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.148+0000 7fd337e00640 1 --2- 172.21.15.159:0/1057754050 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd31406c8f0 0x7fd31406edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd328034000 tx=0x7fd328007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:16.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.148+0000 7fd34177e740 1 Processor -- start 2024-04-04T06:10:16.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.149+0000 7fd34177e740 1 -- start start 2024-04-04T06:10:16.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.150+0000 7fd34177e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55641d3798c0 0x55641d379ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.150+0000 7fd34177e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d3498e0 0x55641d376b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.150+0000 7fd34177e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55641d34d440 0x55641d377070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.150+0000 7fd34177e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55641d2b35c0 con 0x55641d3498e0 2024-04-04T06:10:16.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.150+0000 7fd34177e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55641d25b3b0 con 0x55641d3798c0 2024-04-04T06:10:16.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.150+0000 7fd34177e740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55641d2af570 con 0x55641d34d440 2024-04-04T06:10:16.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.150+0000 7fd32f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d3498e0 0x55641d376b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.151+0000 7fd32f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d3498e0 0x55641d376b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55332/0 (socket says 172.21.15.159:55332) 2024-04-04T06:10:16.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.151+0000 7fd32f400640 1 -- 172.21.15.159:0/2601010334 learned_addr learned my addr 172.21.15.159:0/2601010334 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:16.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.151+0000 7fd32f400640 1 -- 172.21.15.159:0/2601010334 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55641d34d440 msgr2=0x55641d377070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:16.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.151+0000 7fd33ca00640 1 --2- 172.21.15.159:0/2601010334 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55641d34d440 0x55641d377070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.151+0000 7fd32f400640 1 --2- 172.21.15.159:0/2601010334 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55641d34d440 0x55641d377070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.151+0000 7fd337e00640 1 --2- 172.21.15.159:0/2601010334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55641d3798c0 0x55641d379ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.151+0000 7fd32f400640 1 -- 172.21.15.159:0/2601010334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55641d3798c0 msgr2=0x55641d379ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:16.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.151+0000 7fd32f400640 1 --2- 172.21.15.159:0/2601010334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55641d3798c0 0x55641d379ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.151+0000 7fd32f400640 1 -- 172.21.15.159:0/2601010334 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55641d0937c0 con 0x55641d3498e0 2024-04-04T06:10:16.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.151+0000 7fd32f400640 1 --2- 172.21.15.159:0/2601010334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d3498e0 0x55641d376b30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd32000a040 tx=0x7fd320058c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d55694c4f93476a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:16.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.151+0000 7fd327e00640 1 -- 172.21.15.159:0/2601010334 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd320071180 con 0x55641d3498e0 2024-04-04T06:10:16.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.151+0000 7fd327e00640 1 -- 172.21.15.159:0/2601010334 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd320056ee0 con 0x55641d3498e0 2024-04-04T06:10:16.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.151+0000 7fd327e00640 1 -- 172.21.15.159:0/2601010334 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd320058030 con 0x55641d3498e0 2024-04-04T06:10:16.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.152+0000 7fd34177e740 1 -- 172.21.15.159:0/2601010334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d3498e0 msgr2=0x55641d376b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:16.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.152+0000 7fd34177e740 1 --2- 172.21.15.159:0/2601010334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d3498e0 0x55641d376b30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd32000a040 tx=0x7fd320058c60 comp rx=0 tx=0).stop 2024-04-04T06:10:16.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.152+0000 7fd34177e740 1 -- 172.21.15.159:0/2601010334 shutdown_connections 2024-04-04T06:10:16.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.152+0000 7fd34177e740 1 --2- 172.21.15.159:0/2601010334 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55641d34d440 0x55641d377070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.152+0000 7fd34177e740 1 --2- 172.21.15.159:0/2601010334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d3498e0 0x55641d376b30 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.152+0000 7fd34177e740 1 --2- 172.21.15.159:0/2601010334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55641d3798c0 0x55641d379ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.152+0000 7fd34177e740 1 -- 172.21.15.159:0/2601010334 >> 172.21.15.159:0/2601010334 conn(0x55641d34bb50 msgr2=0x55641d3946d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:16.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.152+0000 7fd34177e740 1 -- 172.21.15.159:0/2601010334 shutdown_connections 2024-04-04T06:10:16.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.152+0000 7fd34177e740 1 -- 172.21.15.159:0/2601010334 wait complete. 2024-04-04T06:10:16.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.153+0000 7fd34177e740 1 -- start start 2024-04-04T06:10:16.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.154+0000 7fd34177e740 1 Processor -- start 2024-04-04T06:10:16.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.155+0000 7fd34177e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55641d34d440 0x55641d365e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.155+0000 7fd34177e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55641d3798c0 0x55641d366340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.155+0000 7fd34177e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d37b330 0x55641d366880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.155+0000 7fd34177e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55641d34e5f0 con 0x55641d37b330 2024-04-04T06:10:16.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.155+0000 7fd34177e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55641d26e7a0 con 0x55641d34d440 2024-04-04T06:10:16.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.155+0000 7fd34177e740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55641d34d9e0 con 0x55641d3798c0 2024-04-04T06:10:16.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.156+0000 7fd34177e740 10 client.0 _ll_register_callbacks cb 0x55641d2aafd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:10:16.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.156+0000 7fd33ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d37b330 0x55641d366880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.156+0000 7fd33ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d37b330 0x55641d366880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55346/0 (socket says 172.21.15.159:55346) 2024-04-04T06:10:16.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.156+0000 7fd33ca00640 1 -- 172.21.15.159:0/4029122731 learned_addr learned my addr 172.21.15.159:0/4029122731 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.156+0000 7fd32f400640 1 --2- 172.21.15.159:0/4029122731 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55641d3798c0 0x55641d366340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.156+0000 7fd337e00640 1 --2- 172.21.15.159:0/4029122731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55641d34d440 0x55641d365e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.156+0000 7fd33ca00640 1 -- 172.21.15.159:0/4029122731 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55641d3798c0 msgr2=0x55641d366340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.156+0000 7fd33ca00640 1 --2- 172.21.15.159:0/4029122731 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55641d3798c0 0x55641d366340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.156+0000 7fd33ca00640 1 -- 172.21.15.159:0/4029122731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55641d34d440 msgr2=0x55641d365e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.156+0000 7fd33ca00640 1 --2- 172.21.15.159:0/4029122731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55641d34d440 0x55641d365e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.156+0000 7fd33ca00640 1 -- 172.21.15.159:0/4029122731 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55641d0f3f50 con 0x55641d37b330 2024-04-04T06:10:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.157+0000 7fd33ca00640 1 --2- 172.21.15.159:0/4029122731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d37b330 0x55641d366880 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd3380b2c40 tx=0x7fd330007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586406503c266609 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.157+0000 7fd324c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:10:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.157+0000 7fd324c00640 1 -- 172.21.15.159:0/4029122731 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33001c060 con 0x55641d37b330 2024-04-04T06:10:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.157+0000 7fd324c00640 1 -- 172.21.15.159:0/4029122731 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fd33000f930 con 0x55641d37b330 2024-04-04T06:10:16.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.157+0000 7fd34177e740 1 -- 172.21.15.159:0/4029122731 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd320056ee0 con 0x55641d37b330 2024-04-04T06:10:16.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.157+0000 7fd324c00640 1 -- 172.21.15.159:0/4029122731 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd33000fb30 con 0x55641d37b330 2024-04-04T06:10:16.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.157+0000 7fd34177e740 10 client.6405 fetch_fsmap learned FSMap version 35 2024-04-04T06:10:16.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.157+0000 7fd34177e740 1 -- 172.21.15.159:0/4029122731 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55641cf870b0 con 0x55641d37b330 2024-04-04T06:10:16.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.158+0000 7fd324c00640 1 -- 172.21.15.159:0/4029122731 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd320056ee0 con 0x55641d37b330 2024-04-04T06:10:16.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.158+0000 7fd324c00640 1 -- 172.21.15.159:0/4029122731 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7fd330025020 con 0x55641d37b330 2024-04-04T06:10:16.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.158+0000 7fd34177e740 10 client.6405 fetch_fsmap finished waiting for FSMap version 35 2024-04-04T06:10:16.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.158+0000 7fd34177e740 10 client.6405 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-04T06:10:16.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.158+0000 7fd34177e740 20 client.6405 populate_metadata read hostname 'smithi159' 2024-04-04T06:10:16.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.158+0000 7fd34177e740 1 --2- 172.21.15.159:0/4029122731 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55641d3644b0 0x55641d3a8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.158+0000 7fd34177e740 4 client.6405 mds_command: new command op to 6039 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:10:16.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.158+0000 7fd34177e740 1 -- 172.21.15.159:0/4029122731 --> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55641d287890 con 0x55641d3644b0 2024-04-04T06:10:16.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.158+0000 7fd337e00640 1 --2- 172.21.15.159:0/4029122731 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55641d3644b0 0x55641d3a8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.159+0000 7fd337e00640 1 --2- 172.21.15.159:0/4029122731 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55641d3644b0 0x55641d3a8350 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d1267d6b27723c0 server_cookie=d1cccb707150c7b6 in_seq=0 out_seq=0 2024-04-04T06:10:16.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.159+0000 7fd324c00640 10 client.6405 ms_handle_connect on v2:172.21.15.159:6833/962475038 2024-04-04T06:10:16.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.160+0000 7fd324c00640 1 -- 172.21.15.159:0/4029122731 <== mds.0 v2:172.21.15.159:6833/962475038 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55641d287890 con 0x55641d3644b0 2024-04-04T06:10:16.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.160+0000 7fd324c00640 10 client.6405 handle_command_reply: tid=0 2024-04-04T06:10:16.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.160+0000 7fd34177e740 1 client.6405 shutdown 2024-04-04T06:10:16.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.160+0000 7fd34177e740 1 -- 172.21.15.159:0/4029122731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d37b330 msgr2=0x55641d366880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:16.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.160+0000 7fd34177e740 1 --2- 172.21.15.159:0/4029122731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d37b330 0x55641d366880 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd3380b2c40 tx=0x7fd330007c90 comp rx=0 tx=0).stop 2024-04-04T06:10:16.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.160+0000 7fd34177e740 1 -- 172.21.15.159:0/4029122731 shutdown_connections 2024-04-04T06:10:16.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.160+0000 7fd34177e740 1 --2- 172.21.15.159:0/4029122731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d37b330 0x55641d366880 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.160+0000 7fd34177e740 1 --2- 172.21.15.159:0/4029122731 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55641d3798c0 0x55641d366340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.160+0000 7fd34177e740 1 --2- 172.21.15.159:0/4029122731 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55641d3644b0 0x55641d3a8350 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.160+0000 7fd34177e740 1 --2- 172.21.15.159:0/4029122731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55641d34d440 0x55641d365e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.160+0000 7fd34177e740 1 -- 172.21.15.159:0/4029122731 >> 172.21.15.159:0/4029122731 conn(0x55641d3498e0 msgr2=0x55641d397be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:16.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.160+0000 7fd324c00640 0 client.6405 ms_handle_reset on v2:172.21.15.159:6833/962475038 2024-04-04T06:10:16.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.160+0000 7fd34177e740 1 -- 172.21.15.159:0/4029122731 shutdown_connections 2024-04-04T06:10:16.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.160+0000 7fd34177e740 1 -- 172.21.15.159:0/4029122731 wait complete. 2024-04-04T06:10:16.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.161+0000 7fd34177e740 20 client.6405 trim_cache size 0 max 16384 2024-04-04T06:10:16.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.176+0000 7fd34177e740 1 Processor -- start 2024-04-04T06:10:16.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.176+0000 7fd34177e740 1 -- start start 2024-04-04T06:10:16.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.177+0000 7fd34177e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55641d34d440 0x55641d396d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.177+0000 7fd34177e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d3798c0 0x55641d3937d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.177+0000 7fd34177e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55641d37b330 0x55641d393d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.177+0000 7fd34177e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd338201080 con 0x55641d3798c0 2024-04-04T06:10:16.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.177+0000 7fd34177e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd338200080 con 0x55641d34d440 2024-04-04T06:10:16.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.177+0000 7fd34177e740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd3381ff530 con 0x55641d37b330 2024-04-04T06:10:16.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.177+0000 7fd32f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d3798c0 0x55641d3937d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.177+0000 7fd32f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d3798c0 0x55641d3937d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55362/0 (socket says 172.21.15.159:55362) 2024-04-04T06:10:16.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.177+0000 7fd32f400640 1 -- 172.21.15.159:0/2787790060 learned_addr learned my addr 172.21.15.159:0/2787790060 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:16.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.178+0000 7fd32f400640 1 -- 172.21.15.159:0/2787790060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55641d37b330 msgr2=0x55641d393d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:16.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.178+0000 7fd337e00640 1 --2- 172.21.15.159:0/2787790060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55641d34d440 0x55641d396d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.178+0000 7fd32f400640 1 --2- 172.21.15.159:0/2787790060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55641d37b330 0x55641d393d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.178+0000 7fd32f400640 1 -- 172.21.15.159:0/2787790060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55641d34d440 msgr2=0x55641d396d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:16.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.178+0000 7fd32f400640 1 --2- 172.21.15.159:0/2787790060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55641d34d440 0x55641d396d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.178+0000 7fd32f400640 1 -- 172.21.15.159:0/2787790060 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55641d34e390 con 0x55641d3798c0 2024-04-04T06:10:16.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.178+0000 7fd32f400640 1 --2- 172.21.15.159:0/2787790060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d3798c0 0x55641d3937d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd32000cfd0 tx=0x7fd32001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e224600afbad1e48 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:16.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.178+0000 7fd327400640 1 -- 172.21.15.159:0/2787790060 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3200592e0 con 0x55641d3798c0 2024-04-04T06:10:16.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.178+0000 7fd327400640 1 -- 172.21.15.159:0/2787790060 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd320056c80 con 0x55641d3798c0 2024-04-04T06:10:16.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.178+0000 7fd327400640 1 -- 172.21.15.159:0/2787790060 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd320055250 con 0x55641d3798c0 2024-04-04T06:10:16.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.179+0000 7fd34177e740 1 -- 172.21.15.159:0/2787790060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d3798c0 msgr2=0x55641d3937d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:16.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.179+0000 7fd34177e740 1 --2- 172.21.15.159:0/2787790060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d3798c0 0x55641d3937d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd32000cfd0 tx=0x7fd32001d000 comp rx=0 tx=0).stop 2024-04-04T06:10:16.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.179+0000 7fd34177e740 1 -- 172.21.15.159:0/2787790060 shutdown_connections 2024-04-04T06:10:16.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.179+0000 7fd34177e740 1 --2- 172.21.15.159:0/2787790060 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55641d37b330 0x55641d393d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.179+0000 7fd34177e740 1 --2- 172.21.15.159:0/2787790060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d3798c0 0x55641d3937d0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.179+0000 7fd34177e740 1 --2- 172.21.15.159:0/2787790060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55641d34d440 0x55641d396d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.179+0000 7fd34177e740 1 -- 172.21.15.159:0/2787790060 >> 172.21.15.159:0/2787790060 conn(0x55641d3498e0 msgr2=0x55641d38e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:16.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.179+0000 7fd34177e740 1 -- 172.21.15.159:0/2787790060 shutdown_connections 2024-04-04T06:10:16.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.179+0000 7fd34177e740 1 -- 172.21.15.159:0/2787790060 wait complete. 2024-04-04T06:10:16.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.180+0000 7fd34177e740 1 -- start start 2024-04-04T06:10:16.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.180+0000 7fd34177e740 1 Processor -- start 2024-04-04T06:10:16.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.181+0000 7fd34177e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55641d34d440 0x55641d409000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.181+0000 7fd34177e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55641d3798c0 0x55641d38e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.181+0000 7fd34177e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d37b330 0x55641d38ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.181+0000 7fd34177e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55641d364330 con 0x55641d37b330 2024-04-04T06:10:16.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.181+0000 7fd34177e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3381ffa60 con 0x55641d3798c0 2024-04-04T06:10:16.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.181+0000 7fd34177e740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55641d364950 con 0x55641d34d440 2024-04-04T06:10:16.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.181+0000 7fd34177e740 10 client.0 _ll_register_callbacks cb 0x7fd318004700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:10:16.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.181+0000 7fd33ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d37b330 0x55641d38ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.181+0000 7fd33ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d37b330 0x55641d38ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55366/0 (socket says 172.21.15.159:55366) 2024-04-04T06:10:16.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.181+0000 7fd33ca00640 1 -- 172.21.15.159:0/4123032244 learned_addr learned my addr 172.21.15.159:0/4123032244 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:16.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.181+0000 7fd337e00640 1 --2- 172.21.15.159:0/4123032244 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55641d34d440 0x55641d409000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.182+0000 7fd32f400640 1 --2- 172.21.15.159:0/4123032244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55641d3798c0 0x55641d38e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.182+0000 7fd33ca00640 1 -- 172.21.15.159:0/4123032244 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55641d34d440 msgr2=0x55641d409000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:16.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.182+0000 7fd33ca00640 1 --2- 172.21.15.159:0/4123032244 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55641d34d440 0x55641d409000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.182+0000 7fd33ca00640 1 -- 172.21.15.159:0/4123032244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55641d3798c0 msgr2=0x55641d38e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:16.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.182+0000 7fd33ca00640 1 --2- 172.21.15.159:0/4123032244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55641d3798c0 0x55641d38e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.182+0000 7fd33ca00640 1 -- 172.21.15.159:0/4123032244 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55641d3937d0 con 0x55641d37b330 2024-04-04T06:10:16.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.182+0000 7fd32f400640 1 --2- 172.21.15.159:0/4123032244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55641d3798c0 0x55641d38e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:16.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.182+0000 7fd33ca00640 1 --2- 172.21.15.159:0/4123032244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d37b330 0x55641d38ec10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fd3300096f0 tx=0x7fd330009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad0ce935a15048c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:16.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.182+0000 7fd324c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:10:16.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.182+0000 7fd324c00640 1 -- 172.21.15.159:0/4123032244 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd330017610 con 0x55641d37b330 2024-04-04T06:10:16.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.182+0000 7fd34177e740 1 -- 172.21.15.159:0/4123032244 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd320056c80 con 0x55641d37b330 2024-04-04T06:10:16.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.182+0000 7fd324c00640 1 -- 172.21.15.159:0/4123032244 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fd3300177b0 con 0x55641d37b330 2024-04-04T06:10:16.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.183+0000 7fd324c00640 1 -- 172.21.15.159:0/4123032244 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd330032050 con 0x55641d37b330 2024-04-04T06:10:16.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.183+0000 7fd34177e740 10 client.6411 fetch_fsmap learned FSMap version 35 2024-04-04T06:10:16.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.183+0000 7fd34177e740 1 -- 172.21.15.159:0/4123032244 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55641d3b8cd0 con 0x55641d37b330 2024-04-04T06:10:16.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.183+0000 7fd324c00640 1 -- 172.21.15.159:0/4123032244 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd320056c80 con 0x55641d37b330 2024-04-04T06:10:16.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.183+0000 7fd324c00640 1 -- 172.21.15.159:0/4123032244 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7fd330035020 con 0x55641d37b330 2024-04-04T06:10:16.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.183+0000 7fd34177e740 10 client.6411 fetch_fsmap finished waiting for FSMap version 35 2024-04-04T06:10:16.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.183+0000 7fd34177e740 10 client.6411 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-04T06:10:16.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.183+0000 7fd34177e740 20 client.6411 populate_metadata read hostname 'smithi159' 2024-04-04T06:10:16.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.183+0000 7fd34177e740 1 --2- 172.21.15.159:0/4123032244 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55641d396440 0x55641d379ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.183+0000 7fd34177e740 4 client.6411 mds_command: new command op to 6039 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-04T06:10:16.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.183+0000 7fd34177e740 1 -- 172.21.15.159:0/4123032244 --> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55641d2c0b00 con 0x55641d396440 2024-04-04T06:10:16.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.183+0000 7fd337e00640 1 --2- 172.21.15.159:0/4123032244 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55641d396440 0x55641d379ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.184+0000 7fd337e00640 1 --2- 172.21.15.159:0/4123032244 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55641d396440 0x55641d379ca0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f78252893c02161 server_cookie=ced08f46019da0dd in_seq=0 out_seq=0 2024-04-04T06:10:16.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.184+0000 7fd324c00640 10 client.6411 ms_handle_connect on v2:172.21.15.159:6833/962475038 2024-04-04T06:10:16.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.184+0000 7fd324c00640 1 -- 172.21.15.159:0/4123032244 <== mds.0 v2:172.21.15.159:6833/962475038 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55641d2c0b00 con 0x55641d396440 2024-04-04T06:10:16.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.184+0000 7fd324c00640 10 client.6411 handle_command_reply: tid=0 2024-04-04T06:10:16.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.184+0000 7fd34177e740 1 client.6411 shutdown 2024-04-04T06:10:16.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.184+0000 7fd34177e740 1 -- 172.21.15.159:0/4123032244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d37b330 msgr2=0x55641d38ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:16.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.184+0000 7fd34177e740 1 --2- 172.21.15.159:0/4123032244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d37b330 0x55641d38ec10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fd3300096f0 tx=0x7fd330009750 comp rx=0 tx=0).stop 2024-04-04T06:10:16.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.185+0000 7fd34177e740 1 -- 172.21.15.159:0/4123032244 shutdown_connections 2024-04-04T06:10:16.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.185+0000 7fd34177e740 1 --2- 172.21.15.159:0/4123032244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55641d37b330 0x55641d38ec10 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.185+0000 7fd34177e740 1 --2- 172.21.15.159:0/4123032244 >> [v2:172.21.15.159:6833/962475038,v1:172.21.15.159:6834/962475038] conn(0x55641d396440 0x55641d379ca0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.185+0000 7fd34177e740 1 --2- 172.21.15.159:0/4123032244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55641d3798c0 0x55641d38e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.185+0000 7fd34177e740 1 --2- 172.21.15.159:0/4123032244 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55641d34d440 0x55641d409000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.185+0000 7fd34177e740 1 -- 172.21.15.159:0/4123032244 >> 172.21.15.159:0/4123032244 conn(0x55641d3498e0 msgr2=0x55641d397be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:16.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.185+0000 7fd324c00640 0 client.6411 ms_handle_reset on v2:172.21.15.159:6833/962475038 2024-04-04T06:10:16.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.185+0000 7fd34177e740 1 -- 172.21.15.159:0/4123032244 shutdown_connections 2024-04-04T06:10:16.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.185+0000 7fd34177e740 1 -- 172.21.15.159:0/4123032244 wait complete. 2024-04-04T06:10:16.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.185+0000 7fd34177e740 20 client.6411 trim_cache size 0 max 16384 2024-04-04T06:10:16.182 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:16.182 INFO:teuthology.orchestra.run.smithi159.stdout:[] 2024-04-04T06:10:16.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.186+0000 7fd33d400640 1 -- 172.21.15.159:0/1057754050 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd31406c8f0 msgr2=0x7fd31406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:16.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.186+0000 7fd33d400640 1 --2- 172.21.15.159:0/1057754050 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd31406c8f0 0x7fd31406edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd328034000 tx=0x7fd328007000 comp rx=0 tx=0).stop 2024-04-04T06:10:16.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.186+0000 7fd33d400640 1 -- 172.21.15.159:0/1057754050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd33815a9c0 msgr2=0x7fd33815ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:16.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.186+0000 7fd33d400640 1 --2- 172.21.15.159:0/1057754050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd33815a9c0 0x7fd33815ae40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd32000c460 tx=0x7fd32000c920 comp rx=0 tx=0).stop 2024-04-04T06:10:16.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.186+0000 7fd33d400640 1 -- 172.21.15.159:0/1057754050 shutdown_connections 2024-04-04T06:10:16.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.186+0000 7fd33d400640 1 --2- 172.21.15.159:0/1057754050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3380720f0 0x7fd338072570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.186+0000 7fd33d400640 1 --2- 172.21.15.159:0/1057754050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd33815a9c0 0x7fd33815ae40 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.186+0000 7fd33d400640 1 --2- 172.21.15.159:0/1057754050 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd31406c8f0 0x7fd31406edb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.186+0000 7fd33d400640 1 --2- 172.21.15.159:0/1057754050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3180039b0 0x7fd338155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.186+0000 7fd33d400640 1 -- 172.21.15.159:0/1057754050 >> 172.21.15.159:0/1057754050 conn(0x7fd3380b1ba0 msgr2=0x7fd33806e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:16.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.186+0000 7fd33d400640 1 -- 172.21.15.159:0/1057754050 shutdown_connections 2024-04-04T06:10:16.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.186+0000 7fd33d400640 1 -- 172.21.15.159:0/1057754050 wait complete. 2024-04-04T06:10:16.195 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-04T06:10:16.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.384+0000 7f7fe4c00640 1 Processor -- start 2024-04-04T06:10:16.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.385+0000 7f7fe4c00640 1 -- start start 2024-04-04T06:10:16.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.386+0000 7f7fe4c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7fe00af4b0 0x7f7fe0167460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.386+0000 7f7fe4c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fe00afab0 con 0x7f7fe0153580 2024-04-04T06:10:16.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.386+0000 7f7fe4c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fe015cc80 con 0x7f7fe00d0990 2024-04-04T06:10:16.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.386+0000 7f7fe4c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7fe00af890 con 0x7f7fe00af4b0 2024-04-04T06:10:16.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.386+0000 7f7fdde00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f7fe0153580 0x7f7fe00d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45812/0 (socket says 172.21.15.159:45812) 2024-04-04T06:10:16.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.386+0000 7f7fdde00640 1 -- 172.21.15.159:0/3309041973 learned_addr learned my addr 172.21.15.159:0/3309041973 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:16.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.387+0000 7f7fde800640 1 --2- 172.21.15.159:0/3309041973 >> v2:172.21.15.178:3300/0 conn(0x7f7fe00af4b0 0x7f7fe0167460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.387+0000 7f7fdd400640 1 -- 172.21.15.159:0/3309041973 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3310013295 0 0) 0x7f7fe00afab0 con 0x7f7fe0153580 2024-04-04T06:10:16.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.387+0000 7f7fdd400640 1 -- 172.21.15.159:0/3309041973 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fc4003520 con 0x7f7fe0153580 2024-04-04T06:10:16.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.387+0000 7f7fdd400640 1 -- 172.21.15.159:0/3309041973 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2574692502 0 0) 0x7f7fc4003520 con 0x7f7fe0153580 2024-04-04T06:10:16.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.387+0000 7f7fdd400640 1 -- 172.21.15.159:0/3309041973 >> v2:172.21.15.178:3300/0 conn(0x7f7fe00af4b0 msgr2=0x7f7fe0167460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:16.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.387+0000 7f7fdd400640 1 --2- 172.21.15.159:0/3309041973 >> v2:172.21.15.178:3300/0 conn(0x7f7fe00af4b0 0x7f7fe0167460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.387+0000 7f7fdd400640 1 -- 172.21.15.159:0/3309041973 >> v1:172.21.15.178:6789/0 conn(0x7f7fe00d0990 legacy=0x7f7fe00aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:16.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.387+0000 7f7fdd400640 1 -- 172.21.15.159:0/3309041973 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe015da00 con 0x7f7fe0153580 2024-04-04T06:10:16.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.387+0000 7f7fdd400640 1 -- 172.21.15.159:0/3309041973 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f7fd0001670 con 0x7f7fe0153580 2024-04-04T06:10:16.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.387+0000 7f7fdd400640 1 -- 172.21.15.159:0/3309041973 >> v1:172.21.15.159:6789/0 conn(0x7f7fe0153580 legacy=0x7f7fe00d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:16.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.387+0000 7f7fdd400640 1 --2- 172.21.15.159:0/3309041973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fc40039b0 0x7f7fc4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.388+0000 7f7fdd400640 1 --2- 172.21.15.159:0/3309041973 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fc4004800 0x7f7fc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.388+0000 7f7fdd400640 1 --2- 172.21.15.159:0/3309041973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fc40051c0 0x7f7fc400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.388+0000 7f7fdd400640 1 -- 172.21.15.159:0/3309041973 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe00af890 con 0x7f7fc40039b0 2024-04-04T06:10:16.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.388+0000 7f7fdd400640 1 -- 172.21.15.159:0/3309041973 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc400c160 con 0x7f7fc40051c0 2024-04-04T06:10:16.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.388+0000 7f7fdd400640 1 -- 172.21.15.159:0/3309041973 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7fc400c360 con 0x7f7fc4004800 2024-04-04T06:10:16.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.388+0000 7f7fdde00640 1 --2- 172.21.15.159:0/3309041973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fc40039b0 0x7f7fc4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.388+0000 7f7fdde00640 1 -- 172.21.15.159:0/3309041973 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fc4004800 msgr2=0x7f7fc4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:16.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.388+0000 7f7fdde00640 1 --2- 172.21.15.159:0/3309041973 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fc4004800 0x7f7fc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.388+0000 7f7fde800640 1 --2- 172.21.15.159:0/3309041973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fc40051c0 0x7f7fc400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.388+0000 7f7fdde00640 1 -- 172.21.15.159:0/3309041973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fc40051c0 msgr2=0x7f7fc400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:16.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.388+0000 7f7fdde00640 1 --2- 172.21.15.159:0/3309041973 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fc40051c0 0x7f7fc400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.388+0000 7f7fde800640 1 -- 172.21.15.159:0/3309041973 reap_dead start 2024-04-04T06:10:16.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.388+0000 7f7fdde00640 1 -- 172.21.15.159:0/3309041973 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc4003520 con 0x7f7fc40039b0 2024-04-04T06:10:16.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.389+0000 7f7fdde00640 1 --2- 172.21.15.159:0/3309041973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fc40039b0 0x7f7fc4003dd0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000c210 tx=0x7f7fc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f1cde1c042ef19 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:16.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.390+0000 7f7fdd400640 1 -- 172.21.15.159:0/3309041973 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0007d30 con 0x7f7fc40039b0 2024-04-04T06:10:16.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.390+0000 7f7fdd400640 1 -- 172.21.15.159:0/3309041973 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fd0030070 con 0x7f7fc40039b0 2024-04-04T06:10:16.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.390+0000 7f7fdd400640 1 -- 172.21.15.159:0/3309041973 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0005d70 con 0x7f7fc40039b0 2024-04-04T06:10:16.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.390+0000 7f7fe4c00640 1 -- 172.21.15.159:0/3309041973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fc40039b0 msgr2=0x7f7fc4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:16.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.390+0000 7f7fe4c00640 1 --2- 172.21.15.159:0/3309041973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fc40039b0 0x7f7fc4003dd0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000c210 tx=0x7f7fc4005830 comp rx=0 tx=0).stop 2024-04-04T06:10:16.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.390+0000 7f7fe4c00640 1 -- 172.21.15.159:0/3309041973 shutdown_connections 2024-04-04T06:10:16.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.390+0000 7f7fe4c00640 1 --2- 172.21.15.159:0/3309041973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fc40039b0 0x7f7fc4003dd0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.390+0000 7f7fe4c00640 1 -- 172.21.15.159:0/3309041973 >> 172.21.15.159:0/3309041973 conn(0x7f7fe006b750 msgr2=0x7f7fe006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:16.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.390+0000 7f7fe4c00640 1 -- 172.21.15.159:0/3309041973 shutdown_connections 2024-04-04T06:10:16.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.390+0000 7f7fe4c00640 1 -- 172.21.15.159:0/3309041973 wait complete. 2024-04-04T06:10:16.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.393+0000 7f7fe4c00640 1 Processor -- start 2024-04-04T06:10:16.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.393+0000 7f7fe4c00640 1 -- start start 2024-04-04T06:10:16.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.395+0000 7f7fe4c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fc40039b0 0x7f7fe009db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.395+0000 7f7fe4c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fe009b8d0 0x7f7fe01ebfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.395+0000 7f7fe4c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fe01f5710 0x7f7fe01f5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.395+0000 7f7fe4c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe015d4e0 con 0x7f7fc40039b0 2024-04-04T06:10:16.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.395+0000 7f7fe4c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe015d200 con 0x7f7fe01f5710 2024-04-04T06:10:16.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.395+0000 7f7fe4c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7fe0153960 con 0x7f7fe009b8d0 2024-04-04T06:10:16.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.395+0000 7f7fdde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fc40039b0 0x7f7fe009db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.395+0000 7f7fdde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fc40039b0 0x7f7fe009db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55388/0 (socket says 172.21.15.159:55388) 2024-04-04T06:10:16.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.395+0000 7f7fdde00640 1 -- 172.21.15.159:0/1357041106 learned_addr learned my addr 172.21.15.159:0/1357041106 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:16.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.396+0000 7f7fd7e00640 1 --2- 172.21.15.159:0/1357041106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fe009b8d0 0x7f7fe01ebfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.396+0000 7f7fdde00640 1 -- 172.21.15.159:0/1357041106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fe009b8d0 msgr2=0x7f7fe01ebfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:16.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.396+0000 7f7fdde00640 1 --2- 172.21.15.159:0/1357041106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fe009b8d0 0x7f7fe01ebfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.396+0000 7f7fdde00640 1 -- 172.21.15.159:0/1357041106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fe01f5710 msgr2=0x7f7fe01f5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:16.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.396+0000 7f7fdde00640 1 --2- 172.21.15.159:0/1357041106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fe01f5710 0x7f7fe01f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.396+0000 7f7fdde00640 1 -- 172.21.15.159:0/1357041106 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe009a830 con 0x7f7fc40039b0 2024-04-04T06:10:16.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.396+0000 7f7fdde00640 1 --2- 172.21.15.159:0/1357041106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fc40039b0 0x7f7fe009db40 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000c210 tx=0x7f7fd002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:16.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.396+0000 7f7fd6000640 1 -- 172.21.15.159:0/1357041106 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0004bf0 con 0x7f7fc40039b0 2024-04-04T06:10:16.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.397+0000 7f7fd6000640 1 -- 172.21.15.159:0/1357041106 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fd0000ed0 con 0x7f7fc40039b0 2024-04-04T06:10:16.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.397+0000 7f7fd6000640 1 -- 172.21.15.159:0/1357041106 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0007d10 con 0x7f7fc40039b0 2024-04-04T06:10:16.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.397+0000 7f7fe4c00640 1 -- 172.21.15.159:0/1357041106 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe009ab10 con 0x7f7fc40039b0 2024-04-04T06:10:16.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.397+0000 7f7fe4c00640 1 -- 172.21.15.159:0/1357041106 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe009b0d0 con 0x7f7fc40039b0 2024-04-04T06:10:16.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.398+0000 7f7fe4c00640 1 -- 172.21.15.159:0/1357041106 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fe00685c0 con 0x7f7fc40039b0 2024-04-04T06:10:16.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.399+0000 7f7fd6000640 1 -- 172.21.15.159:0/1357041106 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7fd0048020 con 0x7f7fc40039b0 2024-04-04T06:10:16.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.400+0000 7f7fd6000640 1 --2- 172.21.15.159:0/1357041106 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7fbc06c8f0 0x7f7fbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:16.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.400+0000 7f7fd6000640 1 -- 172.21.15.159:0/1357041106 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f7fd00ace30 con 0x7f7fc40039b0 2024-04-04T06:10:16.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.401+0000 7f7fd7e00640 1 --2- 172.21.15.159:0/1357041106 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7fbc06c8f0 0x7f7fbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:16.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.402+0000 7f7fd7e00640 1 --2- 172.21.15.159:0/1357041106 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7fbc06c8f0 0x7f7fbc06edb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0152a20 tx=0x7f7fc8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:16.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.405+0000 7f7fd6000640 1 -- 172.21.15.159:0/1357041106 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7fd007b250 con 0x7f7fc40039b0 2024-04-04T06:10:16.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.719+0000 7f7fe4c00640 1 -- 172.21.15.159:0/1357041106 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7fe006eca0 con 0x7f7fc40039b0 2024-04-04T06:10:16.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.720+0000 7f7fd6000640 1 -- 172.21.15.159:0/1357041106 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f7fd007ab50 con 0x7f7fc40039b0 2024-04-04T06:10:16.713 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:16.713 INFO:teuthology.orchestra.run.smithi159.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-04T06:10:16.716 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.722+0000 7f7fe4c00640 1 -- 172.21.15.159:0/1357041106 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7fbc06c8f0 msgr2=0x7f7fbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:16.716 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.722+0000 7f7fe4c00640 1 --2- 172.21.15.159:0/1357041106 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7fbc06c8f0 0x7f7fbc06edb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0152a20 tx=0x7f7fc8009040 comp rx=0 tx=0).stop 2024-04-04T06:10:16.716 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.722+0000 7f7fe4c00640 1 -- 172.21.15.159:0/1357041106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fc40039b0 msgr2=0x7f7fe009db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:16.716 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.722+0000 7f7fe4c00640 1 --2- 172.21.15.159:0/1357041106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fc40039b0 0x7f7fe009db40 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000c210 tx=0x7f7fd002e070 comp rx=0 tx=0).stop 2024-04-04T06:10:16.716 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.722+0000 7f7fe4c00640 1 -- 172.21.15.159:0/1357041106 shutdown_connections 2024-04-04T06:10:16.716 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.722+0000 7f7fe4c00640 1 --2- 172.21.15.159:0/1357041106 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7fbc06c8f0 0x7f7fbc06edb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.722+0000 7f7fe4c00640 1 --2- 172.21.15.159:0/1357041106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fe01f5710 0x7f7fe01f5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.722+0000 7f7fe4c00640 1 --2- 172.21.15.159:0/1357041106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fe009b8d0 0x7f7fe01ebfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.722+0000 7f7fe4c00640 1 --2- 172.21.15.159:0/1357041106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fc40039b0 0x7f7fe009db40 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:16.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.722+0000 7f7fe4c00640 1 -- 172.21.15.159:0/1357041106 >> 172.21.15.159:0/1357041106 conn(0x7f7fe006b750 msgr2=0x7f7fe00b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:16.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.723+0000 7f7fe4c00640 1 -- 172.21.15.159:0/1357041106 shutdown_connections 2024-04-04T06:10:16.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:16.723+0000 7f7fe4c00640 1 -- 172.21.15.159:0/1357041106 wait complete. 2024-04-04T06:10:16.732 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-04T06:10:16.732 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-04T06:10:16.733 DEBUG:teuthology.orchestra.run.smithi159:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:10:16.829 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:10:16.943 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-04T06:10:16.943 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:10:16.943 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:10:16.943 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:10:16.995 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:10:16.995 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:10:17.039 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:10:17.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.324+0000 7f4884800640 1 Processor -- start 2024-04-04T06:10:17.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.324+0000 7f4884800640 1 -- start start 2024-04-04T06:10:17.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.326+0000 7f4884800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f487c15b310 0x7f487c0727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:17.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.326+0000 7f4884800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f487c0d0780 0x7f487c0d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:17.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.326+0000 7f4884800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f487c15b6f0 con 0x7f487c0b4f90 2024-04-04T06:10:17.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.326+0000 7f4884800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f487c072e80 con 0x7f487c15b310 2024-04-04T06:10:17.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.326+0000 7f4884800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f487c073160 con 0x7f487c0d0780 2024-04-04T06:10:17.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.326+0000 7f4882800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f487c0d0780 0x7f487c0d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:17.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.326+0000 7f4881e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f487c15b310 0x7f487c0727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:17.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.326+0000 7f4881e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f487c15b310 0x7f487c0727f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:58672/0 (socket says 172.21.15.159:58672) 2024-04-04T06:10:17.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.326+0000 7f4881e00640 1 -- 172.21.15.159:0/4236749789 learned_addr learned my addr 172.21.15.159:0/4236749789 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:17.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.327+0000 7f4882800640 1 -- 172.21.15.159:0/4236749789 >> v2:172.21.15.178:3300/0 conn(0x7f487c15b310 msgr2=0x7f487c0727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:17.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.327+0000 7f4882800640 1 --2- 172.21.15.159:0/4236749789 >> v2:172.21.15.178:3300/0 conn(0x7f487c15b310 0x7f487c0727f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:17.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.327+0000 7f4882800640 1 -- 172.21.15.159:0/4236749789 >> v1:172.21.15.178:6789/0 conn(0x7f487c0b4f90 legacy=0x7f487c15f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:17.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.327+0000 7f4882800640 1 -- 172.21.15.159:0/4236749789 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487c15efe0 con 0x7f487c0d0780 2024-04-04T06:10:17.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.328+0000 7f4882800640 1 --2- 172.21.15.159:0/4236749789 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f487c0d0780 0x7f487c0d0c10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4878009d90 tx=0x7f487802df90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de6e99b364a2bd09 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:17.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.329+0000 7f4880a00640 1 -- 172.21.15.159:0/4236749789 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487803b070 con 0x7f487c0d0780 2024-04-04T06:10:17.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.329+0000 7f4880a00640 1 -- 172.21.15.159:0/4236749789 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f487802ed40 con 0x7f487c0d0780 2024-04-04T06:10:17.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.329+0000 7f4880a00640 1 -- 172.21.15.159:0/4236749789 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878036660 con 0x7f487c0d0780 2024-04-04T06:10:17.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.330+0000 7f4884800640 1 -- 172.21.15.159:0/4236749789 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f487c0d0780 msgr2=0x7f487c0d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:17.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.330+0000 7f4884800640 1 --2- 172.21.15.159:0/4236749789 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f487c0d0780 0x7f487c0d0c10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4878009d90 tx=0x7f487802df90 comp rx=0 tx=0).stop 2024-04-04T06:10:17.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.330+0000 7f4884800640 1 -- 172.21.15.159:0/4236749789 shutdown_connections 2024-04-04T06:10:17.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.330+0000 7f4884800640 1 --2- 172.21.15.159:0/4236749789 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f487c0d0780 0x7f487c0d0c10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:17.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.330+0000 7f4884800640 1 --2- 172.21.15.159:0/4236749789 >> v2:172.21.15.178:3300/0 conn(0x7f487c15b310 0x7f487c0727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:17.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.330+0000 7f4884800640 1 -- 172.21.15.159:0/4236749789 >> 172.21.15.159:0/4236749789 conn(0x7f487c06b990 msgr2=0x7f487c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:17.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.330+0000 7f4884800640 1 -- 172.21.15.159:0/4236749789 shutdown_connections 2024-04-04T06:10:17.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.330+0000 7f4884800640 1 -- 172.21.15.159:0/4236749789 wait complete. 2024-04-04T06:10:17.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.333+0000 7f4884800640 1 Processor -- start 2024-04-04T06:10:17.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.333+0000 7f4884800640 1 -- start start 2024-04-04T06:10:17.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.334+0000 7f4884800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f487c0b4f90 0x7f487c15e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:17.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.334+0000 7f4884800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f487c0d0780 0x7f487c1e4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:17.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.334+0000 7f4884800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f487c15b310 0x7f487c1e60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:17.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.334+0000 7f4884800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f487c073440 con 0x7f487c0d0780 2024-04-04T06:10:17.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.334+0000 7f4884800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f487c0b53f0 con 0x7f487c15b310 2024-04-04T06:10:17.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.334+0000 7f4884800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f487c0adc80 con 0x7f487c0b4f90 2024-04-04T06:10:17.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.335+0000 7f4881400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f487c0d0780 0x7f487c1e4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:17.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.335+0000 7f4881400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f487c0d0780 0x7f487c1e4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55404/0 (socket says 172.21.15.159:55404) 2024-04-04T06:10:17.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.335+0000 7f4881400640 1 -- 172.21.15.159:0/140500189 learned_addr learned my addr 172.21.15.159:0/140500189 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:17.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.335+0000 7f4881e00640 1 --2- 172.21.15.159:0/140500189 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f487c0b4f90 0x7f487c15e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:17.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.335+0000 7f4882800640 1 --2- 172.21.15.159:0/140500189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f487c15b310 0x7f487c1e60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:17.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.335+0000 7f4881400640 1 -- 172.21.15.159:0/140500189 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f487c0b4f90 msgr2=0x7f487c15e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:17.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.335+0000 7f4881400640 1 --2- 172.21.15.159:0/140500189 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f487c0b4f90 0x7f487c15e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:17.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.335+0000 7f4881400640 1 -- 172.21.15.159:0/140500189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f487c15b310 msgr2=0x7f487c1e60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:17.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.335+0000 7f4881400640 1 --2- 172.21.15.159:0/140500189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f487c15b310 0x7f487c1e60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:17.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.335+0000 7f4881400640 1 -- 172.21.15.159:0/140500189 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487c1e5050 con 0x7f487c0d0780 2024-04-04T06:10:17.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.336+0000 7f4881400640 1 --2- 172.21.15.159:0/140500189 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f487c0d0780 0x7f487c1e4970 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f486c00c400 tx=0x7f486c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:17.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.336+0000 7f4876000640 1 -- 172.21.15.159:0/140500189 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c0052d0 con 0x7f487c0d0780 2024-04-04T06:10:17.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.336+0000 7f4876000640 1 -- 172.21.15.159:0/140500189 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f486c005490 con 0x7f487c0d0780 2024-04-04T06:10:17.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.336+0000 7f4884800640 1 -- 172.21.15.159:0/140500189 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f487c1e5330 con 0x7f487c0d0780 2024-04-04T06:10:17.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.336+0000 7f4876000640 1 -- 172.21.15.159:0/140500189 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c007260 con 0x7f487c0d0780 2024-04-04T06:10:17.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.336+0000 7f4884800640 1 -- 172.21.15.159:0/140500189 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f487c1e58f0 con 0x7f487c0d0780 2024-04-04T06:10:17.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.337+0000 7f4884800640 1 -- 172.21.15.159:0/140500189 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f487c0685c0 con 0x7f487c0d0780 2024-04-04T06:10:17.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.338+0000 7f4876000640 1 -- 172.21.15.159:0/140500189 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f486c007400 con 0x7f487c0d0780 2024-04-04T06:10:17.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.340+0000 7f4876000640 1 --2- 172.21.15.159:0/140500189 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f486006c8f0 0x7f486006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:17.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.340+0000 7f4876000640 1 -- 172.21.15.159:0/140500189 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f486c08b180 con 0x7f487c0d0780 2024-04-04T06:10:17.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.340+0000 7f4881e00640 1 --2- 172.21.15.159:0/140500189 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f486006c8f0 0x7f486006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:17.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.341+0000 7f4881e00640 1 --2- 172.21.15.159:0/140500189 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f486006c8f0 0x7f486006edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4870009ac0 tx=0x7f4870008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:17.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.344+0000 7f4876000640 1 -- 172.21.15.159:0/140500189 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f486c059510 con 0x7f487c0d0780 2024-04-04T06:10:17.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.545+0000 7f4884800640 1 -- 172.21.15.159:0/140500189 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f487c15ec20 con 0x7f487c0d0780 2024-04-04T06:10:17.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.548+0000 7f4876000640 1 -- 172.21.15.159:0/140500189 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16677 (secure 0 0 0) 0x7f486c058e10 con 0x7f487c0d0780 2024-04-04T06:10:17.542 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:17.542 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":59,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:08:58.581221+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:08:54.848337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-04T06:08:56.003356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:10:17.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.550+0000 7f4884800640 1 -- 172.21.15.159:0/140500189 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f486006c8f0 msgr2=0x7f486006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:17.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.550+0000 7f4884800640 1 --2- 172.21.15.159:0/140500189 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f486006c8f0 0x7f486006edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4870009ac0 tx=0x7f4870008040 comp rx=0 tx=0).stop 2024-04-04T06:10:17.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.550+0000 7f4884800640 1 -- 172.21.15.159:0/140500189 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f487c0d0780 msgr2=0x7f487c1e4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:17.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.550+0000 7f4884800640 1 --2- 172.21.15.159:0/140500189 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f487c0d0780 0x7f487c1e4970 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f486c00c400 tx=0x7f486c00c8c0 comp rx=0 tx=0).stop 2024-04-04T06:10:17.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.550+0000 7f4884800640 1 -- 172.21.15.159:0/140500189 shutdown_connections 2024-04-04T06:10:17.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.550+0000 7f4884800640 1 --2- 172.21.15.159:0/140500189 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f486006c8f0 0x7f486006edb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:17.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.550+0000 7f4884800640 1 --2- 172.21.15.159:0/140500189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f487c15b310 0x7f487c1e60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:17.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.550+0000 7f4884800640 1 --2- 172.21.15.159:0/140500189 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f487c0d0780 0x7f487c1e4970 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:17.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.550+0000 7f4884800640 1 --2- 172.21.15.159:0/140500189 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f487c0b4f90 0x7f487c15e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:17.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.551+0000 7f4884800640 1 -- 172.21.15.159:0/140500189 >> 172.21.15.159:0/140500189 conn(0x7f487c06b990 msgr2=0x7f487c0b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:17.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.551+0000 7f4884800640 1 -- 172.21.15.159:0/140500189 shutdown_connections 2024-04-04T06:10:17.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.551+0000 7f4884800640 1 -- 172.21.15.159:0/140500189 wait complete. 2024-04-04T06:10:17.563 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:08:54.848337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:08:56.003356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:10:17.564 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:10:17.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.772+0000 7f9629600640 1 Processor -- start 2024-04-04T06:10:17.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.772+0000 7f9629600640 1 -- start start 2024-04-04T06:10:17.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.774+0000 7f9629600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f962406ca00 0x7f962406f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:17.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.774+0000 7f9629600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f962406ceb0 con 0x7f962406c580 2024-04-04T06:10:17.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.774+0000 7f9629600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9624067900 con 0x7f96241590c0 2024-04-04T06:10:17.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.774+0000 7f9629600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f962406e5b0 con 0x7f962406ca00 2024-04-04T06:10:17.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.774+0000 7f9622000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f962406c580 0x7f962406ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45818/0 (socket says 172.21.15.159:45818) 2024-04-04T06:10:17.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.774+0000 7f9622000640 1 -- 172.21.15.159:0/1902281208 learned_addr learned my addr 172.21.15.159:0/1902281208 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:17.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.774+0000 7f9623400640 1 --2- 172.21.15.159:0/1902281208 >> v2:172.21.15.178:3300/0 conn(0x7f962406ca00 0x7f962406f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:17.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.775+0000 7f9621600640 1 -- 172.21.15.159:0/1902281208 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 5791478 0 0) 0x7f962406ceb0 con 0x7f962406c580 2024-04-04T06:10:17.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.775+0000 7f9621600640 1 -- 172.21.15.159:0/1902281208 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9608003520 con 0x7f962406c580 2024-04-04T06:10:17.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.775+0000 7f9621600640 1 -- 172.21.15.159:0/1902281208 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2538036993 0 0) 0x7f9608003520 con 0x7f962406c580 2024-04-04T06:10:17.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.775+0000 7f9621600640 1 -- 172.21.15.159:0/1902281208 >> v2:172.21.15.178:3300/0 conn(0x7f962406ca00 msgr2=0x7f962406f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:17.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.775+0000 7f9621600640 1 --2- 172.21.15.159:0/1902281208 >> v2:172.21.15.178:3300/0 conn(0x7f962406ca00 0x7f962406f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:17.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.775+0000 7f9621600640 1 -- 172.21.15.159:0/1902281208 >> v1:172.21.15.178:6789/0 conn(0x7f96241590c0 legacy=0x7f962415f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:17.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.775+0000 7f9621600640 1 -- 172.21.15.159:0/1902281208 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96240b47d0 con 0x7f962406c580 2024-04-04T06:10:17.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.775+0000 7f9621600640 1 -- 172.21.15.159:0/1902281208 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f9610001670 con 0x7f962406c580 2024-04-04T06:10:17.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.775+0000 7f9621600640 1 -- 172.21.15.159:0/1902281208 >> v1:172.21.15.159:6789/0 conn(0x7f962406c580 legacy=0x7f962406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:17.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.775+0000 7f9621600640 1 --2- 172.21.15.159:0/1902281208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96080039b0 0x7f9608003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:17.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.775+0000 7f9621600640 1 --2- 172.21.15.159:0/1902281208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9608004800 0x7f9608004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:17.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.775+0000 7f9621600640 1 --2- 172.21.15.159:0/1902281208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96080051c0 0x7f960800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:17.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.775+0000 7f9621600640 1 -- 172.21.15.159:0/1902281208 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f962406e5b0 con 0x7f96080039b0 2024-04-04T06:10:17.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.775+0000 7f9621600640 1 -- 172.21.15.159:0/1902281208 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f960800c060 con 0x7f9608004800 2024-04-04T06:10:17.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.775+0000 7f9621600640 1 -- 172.21.15.159:0/1902281208 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f960800c260 con 0x7f96080051c0 2024-04-04T06:10:17.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.776+0000 7f9622a00640 1 --2- 172.21.15.159:0/1902281208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96080039b0 0x7f9608003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:17.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.776+0000 7f9622000640 1 --2- 172.21.15.159:0/1902281208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9608004800 0x7f9608004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:17.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.776+0000 7f9622a00640 1 -- 172.21.15.159:0/1902281208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96080051c0 msgr2=0x7f960800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:17.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.776+0000 7f9622a00640 1 --2- 172.21.15.159:0/1902281208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96080051c0 0x7f960800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:17.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.776+0000 7f9622a00640 1 -- 172.21.15.159:0/1902281208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9608004800 msgr2=0x7f9608004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:17.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.776+0000 7f9622a00640 1 --2- 172.21.15.159:0/1902281208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9608004800 0x7f9608004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:17.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.776+0000 7f9622a00640 1 -- 172.21.15.159:0/1902281208 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9608003520 con 0x7f96080039b0 2024-04-04T06:10:17.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.776+0000 7f9623400640 1 -- 172.21.15.159:0/1902281208 reap_dead start 2024-04-04T06:10:17.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.776+0000 7f9622000640 1 --2- 172.21.15.159:0/1902281208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9608004800 0x7f9608004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:17.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.777+0000 7f9622a00640 1 --2- 172.21.15.159:0/1902281208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96080039b0 0x7f9608003dd0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9614001750 tx=0x7f9608005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206776ba4faa244d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:17.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.777+0000 7f9621600640 1 -- 172.21.15.159:0/1902281208 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614001370 con 0x7f96080039b0 2024-04-04T06:10:17.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.777+0000 7f9621600640 1 -- 172.21.15.159:0/1902281208 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96140052d0 con 0x7f96080039b0 2024-04-04T06:10:17.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.777+0000 7f9621600640 1 -- 172.21.15.159:0/1902281208 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614003830 con 0x7f96080039b0 2024-04-04T06:10:17.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.778+0000 7f9629600640 1 -- 172.21.15.159:0/1902281208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96080039b0 msgr2=0x7f9608003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:17.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.778+0000 7f9629600640 1 --2- 172.21.15.159:0/1902281208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96080039b0 0x7f9608003dd0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9614001750 tx=0x7f9608005830 comp rx=0 tx=0).stop 2024-04-04T06:10:17.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.778+0000 7f9629600640 1 -- 172.21.15.159:0/1902281208 shutdown_connections 2024-04-04T06:10:17.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.778+0000 7f9629600640 1 --2- 172.21.15.159:0/1902281208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96080039b0 0x7f9608003dd0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:17.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.778+0000 7f9629600640 1 -- 172.21.15.159:0/1902281208 >> 172.21.15.159:0/1902281208 conn(0x7f9624071e60 msgr2=0x7f9624072270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:17.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.778+0000 7f9629600640 1 -- 172.21.15.159:0/1902281208 shutdown_connections 2024-04-04T06:10:17.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.778+0000 7f9629600640 1 -- 172.21.15.159:0/1902281208 wait complete. 2024-04-04T06:10:17.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.781+0000 7f9629600640 1 Processor -- start 2024-04-04T06:10:17.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.781+0000 7f9629600640 1 -- start start 2024-04-04T06:10:17.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.783+0000 7f9629600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96080039b0 0x7f9624161850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:17.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.783+0000 7f9629600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96240cf920 0x7f96240cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:17.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.783+0000 7f9629600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9624150de0 0x7f9624151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:17.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.783+0000 7f9629600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f962406e240 con 0x7f9624150de0 2024-04-04T06:10:17.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.783+0000 7f9629600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f962406df60 con 0x7f96240cf920 2024-04-04T06:10:17.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.783+0000 7f9629600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f96240b4380 con 0x7f96080039b0 2024-04-04T06:10:17.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.783+0000 7f9623400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9624150de0 0x7f9624151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:17.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.783+0000 7f9623400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9624150de0 0x7f9624151260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55422/0 (socket says 172.21.15.159:55422) 2024-04-04T06:10:17.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.783+0000 7f9623400640 1 -- 172.21.15.159:0/1213728506 learned_addr learned my addr 172.21.15.159:0/1213728506 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:17.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.784+0000 7f9622000640 1 --2- 172.21.15.159:0/1213728506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96240cf920 0x7f96240cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:17.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.784+0000 7f9622a00640 1 --2- 172.21.15.159:0/1213728506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96080039b0 0x7f9624161850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:17.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.784+0000 7f9623400640 1 -- 172.21.15.159:0/1213728506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96080039b0 msgr2=0x7f9624161850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:17.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.784+0000 7f9623400640 1 --2- 172.21.15.159:0/1213728506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96080039b0 0x7f9624161850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:17.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.784+0000 7f9623400640 1 -- 172.21.15.159:0/1213728506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96240cf920 msgr2=0x7f96240cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:17.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.784+0000 7f9623400640 1 --2- 172.21.15.159:0/1213728506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96240cf920 0x7f96240cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:17.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.784+0000 7f9623400640 1 -- 172.21.15.159:0/1213728506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96241517a0 con 0x7f9624150de0 2024-04-04T06:10:17.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.784+0000 7f9623400640 1 --2- 172.21.15.159:0/1213728506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9624150de0 0x7f9624151260 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f962406d7b0 tx=0x7f961c00ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:17.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.784+0000 7f961aa00640 1 -- 172.21.15.159:0/1213728506 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c018020 con 0x7f9624150de0 2024-04-04T06:10:17.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.784+0000 7f961aa00640 1 -- 172.21.15.159:0/1213728506 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f961c00bb70 con 0x7f9624150de0 2024-04-04T06:10:17.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.784+0000 7f961aa00640 1 -- 172.21.15.159:0/1213728506 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c014400 con 0x7f9624150de0 2024-04-04T06:10:17.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.784+0000 7f9629600640 1 -- 172.21.15.159:0/1213728506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9624151a60 con 0x7f9624150de0 2024-04-04T06:10:17.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.785+0000 7f9629600640 1 -- 172.21.15.159:0/1213728506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f962413e280 con 0x7f9624150de0 2024-04-04T06:10:17.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.786+0000 7f9629600640 1 -- 172.21.15.159:0/1213728506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96240685c0 con 0x7f9624150de0 2024-04-04T06:10:17.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.787+0000 7f961aa00640 1 -- 172.21.15.159:0/1213728506 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f961c005d10 con 0x7f9624150de0 2024-04-04T06:10:17.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.788+0000 7f961aa00640 1 --2- 172.21.15.159:0/1213728506 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f960406c8f0 0x7f960406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:17.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.788+0000 7f961aa00640 1 -- 172.21.15.159:0/1213728506 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f961c08c0d0 con 0x7f9624150de0 2024-04-04T06:10:17.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.789+0000 7f9622a00640 1 --2- 172.21.15.159:0/1213728506 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f960406c8f0 0x7f960406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:17.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.790+0000 7f9622a00640 1 --2- 172.21.15.159:0/1213728506 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f960406c8f0 0x7f960406edb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9614033000 tx=0x7f9614009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:17.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.793+0000 7f961aa00640 1 -- 172.21.15.159:0/1213728506 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f961c022030 con 0x7f9624150de0 2024-04-04T06:10:17.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.993+0000 7f9629600640 1 -- 172.21.15.159:0/1213728506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9624067590 con 0x7f9624150de0 2024-04-04T06:10:17.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.993+0000 7f961aa00640 1 -- 172.21.15.159:0/1213728506 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7f961c05a4f0 con 0x7f9624150de0 2024-04-04T06:10:17.987 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:10:17.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.996+0000 7f9629600640 1 -- 172.21.15.159:0/1213728506 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f960406c8f0 msgr2=0x7f960406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:17.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.996+0000 7f9629600640 1 --2- 172.21.15.159:0/1213728506 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f960406c8f0 0x7f960406edb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9614033000 tx=0x7f9614009030 comp rx=0 tx=0).stop 2024-04-04T06:10:17.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.996+0000 7f9629600640 1 -- 172.21.15.159:0/1213728506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9624150de0 msgr2=0x7f9624151260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:17.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.996+0000 7f9629600640 1 --2- 172.21.15.159:0/1213728506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9624150de0 0x7f9624151260 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f962406d7b0 tx=0x7f961c00ef30 comp rx=0 tx=0).stop 2024-04-04T06:10:17.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.996+0000 7f9629600640 1 -- 172.21.15.159:0/1213728506 shutdown_connections 2024-04-04T06:10:17.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.996+0000 7f9629600640 1 --2- 172.21.15.159:0/1213728506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9624150de0 0x7f9624151260 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:17.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.996+0000 7f9629600640 1 --2- 172.21.15.159:0/1213728506 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f960406c8f0 0x7f960406edb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:17.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.996+0000 7f9629600640 1 --2- 172.21.15.159:0/1213728506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96240cf920 0x7f96240cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:17.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.996+0000 7f9629600640 1 --2- 172.21.15.159:0/1213728506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96080039b0 0x7f9624161850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:17.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.996+0000 7f9629600640 1 -- 172.21.15.159:0/1213728506 >> 172.21.15.159:0/1213728506 conn(0x7f9624071e60 msgr2=0x7f96240b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:17.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.996+0000 7f9629600640 1 -- 172.21.15.159:0/1213728506 shutdown_connections 2024-04-04T06:10:17.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:17.996+0000 7f9629600640 1 -- 172.21.15.159:0/1213728506 wait complete. 2024-04-04T06:10:18.007 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:10:18.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.212+0000 7f5709000640 1 Processor -- start 2024-04-04T06:10:18.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.212+0000 7f5709000640 1 -- start start 2024-04-04T06:10:18.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.215+0000 7f5709000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5704145c40 0x7f570415d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:18.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.215+0000 7f5709000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f570415e120 0x7f57040b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:18.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.215+0000 7f5709000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f570415d810 con 0x7f57040b5ac0 2024-04-04T06:10:18.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.215+0000 7f5709000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f57040b5020 con 0x7f570415e120 2024-04-04T06:10:18.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.215+0000 7f5709000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f5704073800 con 0x7f5704145c40 2024-04-04T06:10:18.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.215+0000 7f5703e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f570415e120 0x7f57040b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:18.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.215+0000 7f5703e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f570415e120 0x7f57040b5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55426/0 (socket says 172.21.15.159:55426) 2024-04-04T06:10:18.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.215+0000 7f5703e00640 1 -- 172.21.15.159:0/2695125208 learned_addr learned my addr 172.21.15.159:0/2695125208 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:18.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.216+0000 7f5703400640 1 --2- 172.21.15.159:0/2695125208 >> v2:172.21.15.178:3300/0 conn(0x7f5704145c40 0x7f570415d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:18.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.216+0000 7f5703e00640 1 -- 172.21.15.159:0/2695125208 >> v2:172.21.15.178:3300/0 conn(0x7f5704145c40 msgr2=0x7f570415d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:18.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.216+0000 7f5703e00640 1 --2- 172.21.15.159:0/2695125208 >> v2:172.21.15.178:3300/0 conn(0x7f5704145c40 0x7f570415d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:18.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.216+0000 7f5703e00640 1 -- 172.21.15.159:0/2695125208 >> v1:172.21.15.178:6789/0 conn(0x7f57040b5ac0 legacy=0x7f570415da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:18.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.216+0000 7f5703e00640 1 -- 172.21.15.159:0/2695125208 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5704070640 con 0x7f570415e120 2024-04-04T06:10:18.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.217+0000 7f5703e00640 1 --2- 172.21.15.159:0/2695125208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f570415e120 0x7f57040b5f30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f57040d0d70 tx=0x7f56fc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd38dd996c10205 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:18.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.217+0000 7f5702a00640 1 -- 172.21.15.159:0/2695125208 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc02ca80 con 0x7f570415e120 2024-04-04T06:10:18.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.217+0000 7f5702a00640 1 -- 172.21.15.159:0/2695125208 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56fc02cc20 con 0x7f570415e120 2024-04-04T06:10:18.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.217+0000 7f5702a00640 1 -- 172.21.15.159:0/2695125208 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc0343f0 con 0x7f570415e120 2024-04-04T06:10:18.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.218+0000 7f5709000640 1 -- 172.21.15.159:0/2695125208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f570415e120 msgr2=0x7f57040b5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:18.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.218+0000 7f5709000640 1 --2- 172.21.15.159:0/2695125208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f570415e120 0x7f57040b5f30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f57040d0d70 tx=0x7f56fc02bee0 comp rx=0 tx=0).stop 2024-04-04T06:10:18.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.218+0000 7f5709000640 1 -- 172.21.15.159:0/2695125208 shutdown_connections 2024-04-04T06:10:18.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.218+0000 7f5709000640 1 --2- 172.21.15.159:0/2695125208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f570415e120 0x7f57040b5f30 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:18.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.218+0000 7f5709000640 1 --2- 172.21.15.159:0/2695125208 >> v2:172.21.15.178:3300/0 conn(0x7f5704145c40 0x7f570415d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:18.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.218+0000 7f5709000640 1 -- 172.21.15.159:0/2695125208 >> 172.21.15.159:0/2695125208 conn(0x7f570406e730 msgr2=0x7f570406eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:18.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.218+0000 7f5709000640 1 -- 172.21.15.159:0/2695125208 shutdown_connections 2024-04-04T06:10:18.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.218+0000 7f5709000640 1 -- 172.21.15.159:0/2695125208 wait complete. 2024-04-04T06:10:18.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.220+0000 7f5709000640 1 Processor -- start 2024-04-04T06:10:18.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.220+0000 7f5709000640 1 -- start start 2024-04-04T06:10:18.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.221+0000 7f5709000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57040b5ac0 0x7f570413cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:18.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.221+0000 7f5709000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5704145c40 0x7f57040723f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:18.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.221+0000 7f5709000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5704072930 0x7f570413ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:18.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.221+0000 7f5709000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5704070920 con 0x7f5704072930 2024-04-04T06:10:18.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.221+0000 7f5709000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57040b55a0 con 0x7f5704145c40 2024-04-04T06:10:18.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.221+0000 7f5709000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5704070aa0 con 0x7f57040b5ac0 2024-04-04T06:10:18.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.221+0000 7f5703e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5704072930 0x7f570413ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:18.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.221+0000 7f5703e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5704072930 0x7f570413ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55436/0 (socket says 172.21.15.159:55436) 2024-04-04T06:10:18.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.221+0000 7f5703e00640 1 -- 172.21.15.159:0/3416093983 learned_addr learned my addr 172.21.15.159:0/3416093983 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:18.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.221+0000 7f5703400640 1 --2- 172.21.15.159:0/3416093983 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57040b5ac0 0x7f570413cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:18.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.221+0000 7f56fbe00640 1 --2- 172.21.15.159:0/3416093983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5704145c40 0x7f57040723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:18.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.221+0000 7f5703e00640 1 -- 172.21.15.159:0/3416093983 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57040b5ac0 msgr2=0x7f570413cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:18.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.221+0000 7f5703e00640 1 --2- 172.21.15.159:0/3416093983 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57040b5ac0 0x7f570413cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:18.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.221+0000 7f5703e00640 1 -- 172.21.15.159:0/3416093983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5704145c40 msgr2=0x7f57040723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:18.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.221+0000 7f5703e00640 1 --2- 172.21.15.159:0/3416093983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5704145c40 0x7f57040723f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:18.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.221+0000 7f5703e00640 1 -- 172.21.15.159:0/3416093983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5704159740 con 0x7f5704072930 2024-04-04T06:10:18.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.222+0000 7f5703e00640 1 --2- 172.21.15.159:0/3416093983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5704072930 0x7f570413ea70 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f56fc009f40 tx=0x7f56fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:18.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.222+0000 7f56fb400640 1 -- 172.21.15.159:0/3416093983 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc043020 con 0x7f5704072930 2024-04-04T06:10:18.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.222+0000 7f56fb400640 1 -- 172.21.15.159:0/3416093983 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56fc03e450 con 0x7f5704072930 2024-04-04T06:10:18.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.222+0000 7f5709000640 1 -- 172.21.15.159:0/3416093983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5704138450 con 0x7f5704072930 2024-04-04T06:10:18.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.222+0000 7f56fb400640 1 -- 172.21.15.159:0/3416093983 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc009510 con 0x7f5704072930 2024-04-04T06:10:18.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.222+0000 7f5709000640 1 -- 172.21.15.159:0/3416093983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57041599e0 con 0x7f5704072930 2024-04-04T06:10:18.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.224+0000 7f5709000640 1 -- 172.21.15.159:0/3416093983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5704075940 con 0x7f5704072930 2024-04-04T06:10:18.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.225+0000 7f56fb400640 1 -- 172.21.15.159:0/3416093983 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f56fc046020 con 0x7f5704072930 2024-04-04T06:10:18.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.225+0000 7f56fb400640 1 --2- 172.21.15.159:0/3416093983 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f56e006c8f0 0x7f56e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:18.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.225+0000 7f56fb400640 1 -- 172.21.15.159:0/3416093983 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f56fc0ae3f0 con 0x7f5704072930 2024-04-04T06:10:18.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.226+0000 7f5703400640 1 --2- 172.21.15.159:0/3416093983 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f56e006c8f0 0x7f56e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:18.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.227+0000 7f5703400640 1 --2- 172.21.15.159:0/3416093983 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f56e006c8f0 0x7f56e006edb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f56f4006fd0 tx=0x7f56f4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:18.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.230+0000 7f56fb400640 1 -- 172.21.15.159:0/3416093983 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f56fc04d030 con 0x7f5704072930 2024-04-04T06:10:18.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.430+0000 7f5709000640 1 -- 172.21.15.159:0/3416093983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5704159bb0 con 0x7f5704072930 2024-04-04T06:10:18.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.431+0000 7f56fb400640 1 -- 172.21.15.159:0/3416093983 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) ==== 104+0+11 (secure 0 0 0) 0x7f570415d810 con 0x7f5704072930 2024-04-04T06:10:18.424 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:10:18.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.433+0000 7f5709000640 1 -- 172.21.15.159:0/3416093983 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f56e006c8f0 msgr2=0x7f56e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:18.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.433+0000 7f5709000640 1 --2- 172.21.15.159:0/3416093983 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f56e006c8f0 0x7f56e006edb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f56f4006fd0 tx=0x7f56f4008040 comp rx=0 tx=0).stop 2024-04-04T06:10:18.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.433+0000 7f5709000640 1 -- 172.21.15.159:0/3416093983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5704072930 msgr2=0x7f570413ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:18.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.433+0000 7f5709000640 1 --2- 172.21.15.159:0/3416093983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5704072930 0x7f570413ea70 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f56fc009f40 tx=0x7f56fc004a40 comp rx=0 tx=0).stop 2024-04-04T06:10:18.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.433+0000 7f5709000640 1 -- 172.21.15.159:0/3416093983 shutdown_connections 2024-04-04T06:10:18.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.433+0000 7f5709000640 1 --2- 172.21.15.159:0/3416093983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5704072930 0x7f570413ea70 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:18.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.433+0000 7f5709000640 1 --2- 172.21.15.159:0/3416093983 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f56e006c8f0 0x7f56e006edb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:18.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.433+0000 7f5709000640 1 --2- 172.21.15.159:0/3416093983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5704145c40 0x7f57040723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:18.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.433+0000 7f5709000640 1 --2- 172.21.15.159:0/3416093983 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57040b5ac0 0x7f570413cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:18.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.433+0000 7f5709000640 1 -- 172.21.15.159:0/3416093983 >> 172.21.15.159:0/3416093983 conn(0x7f570406e730 msgr2=0x7f570415d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:18.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.433+0000 7f5709000640 1 -- 172.21.15.159:0/3416093983 shutdown_connections 2024-04-04T06:10:18.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.433+0000 7f5709000640 1 -- 172.21.15.159:0/3416093983 wait complete. 2024-04-04T06:10:18.444 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:10:18.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.650+0000 7fbd5fe00640 1 Processor -- start 2024-04-04T06:10:18.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.650+0000 7fbd5fe00640 1 -- start start 2024-04-04T06:10:18.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.651+0000 7fbd5fe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd6015e4c0 0x7fbd600b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:18.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.651+0000 7fbd5fe00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd600d2710 con 0x7fbd600d37b0 2024-04-04T06:10:18.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.651+0000 7fbd5fe00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd600d29f0 con 0x7fbd600b69c0 2024-04-04T06:10:18.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.651+0000 7fbd5fe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbd6015e8a0 con 0x7fbd6015e4c0 2024-04-04T06:10:18.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.652+0000 7fbd56000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fbd600d37b0 0x7fbd600b62b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45828/0 (socket says 172.21.15.159:45828) 2024-04-04T06:10:18.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.652+0000 7fbd56000640 1 -- 172.21.15.159:0/3829180536 learned_addr learned my addr 172.21.15.159:0/3829180536 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:18.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.652+0000 7fbd5ea00640 1 --2- 172.21.15.159:0/3829180536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd6015e4c0 0x7fbd600b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:18.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.652+0000 7fbd5e000640 1 -- 172.21.15.159:0/3829180536 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3923706948 0 0) 0x7fbd600d2710 con 0x7fbd600d37b0 2024-04-04T06:10:18.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.652+0000 7fbd5e000640 1 -- 172.21.15.159:0/3829180536 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd44003520 con 0x7fbd600d37b0 2024-04-04T06:10:18.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.653+0000 7fbd5e000640 1 -- 172.21.15.159:0/3829180536 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 575323230 0 0) 0x7fbd44003520 con 0x7fbd600d37b0 2024-04-04T06:10:18.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.653+0000 7fbd5e000640 1 -- 172.21.15.159:0/3829180536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd6015e4c0 msgr2=0x7fbd600b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:18.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.653+0000 7fbd5e000640 1 --2- 172.21.15.159:0/3829180536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd6015e4c0 0x7fbd600b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:18.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.653+0000 7fbd5e000640 1 -- 172.21.15.159:0/3829180536 >> v1:172.21.15.178:6789/0 conn(0x7fbd600b69c0 legacy=0x7fbd600b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:18.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.653+0000 7fbd5e000640 1 -- 172.21.15.159:0/3829180536 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd600d2cd0 con 0x7fbd600d37b0 2024-04-04T06:10:18.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.653+0000 7fbd5e000640 1 -- 172.21.15.159:0/3829180536 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fbd480019f0 con 0x7fbd600d37b0 2024-04-04T06:10:18.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.653+0000 7fbd5e000640 1 -- 172.21.15.159:0/3829180536 >> v1:172.21.15.159:6789/0 conn(0x7fbd600d37b0 legacy=0x7fbd600b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:18.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.653+0000 7fbd5e000640 1 --2- 172.21.15.159:0/3829180536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd440039e0 0x7fbd44003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:18.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.653+0000 7fbd5e000640 1 --2- 172.21.15.159:0/3829180536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd44004800 0x7fbd44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:18.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.653+0000 7fbd5e000640 1 --2- 172.21.15.159:0/3829180536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd440051c0 0x7fbd4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:18.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.653+0000 7fbd5e000640 1 -- 172.21.15.159:0/3829180536 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4400c090 con 0x7fbd440039e0 2024-04-04T06:10:18.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.653+0000 7fbd5e000640 1 -- 172.21.15.159:0/3829180536 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4400c240 con 0x7fbd440051c0 2024-04-04T06:10:18.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.653+0000 7fbd5e000640 1 -- 172.21.15.159:0/3829180536 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbd4400c440 con 0x7fbd44004800 2024-04-04T06:10:18.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.653+0000 7fbd5ea00640 1 --2- 172.21.15.159:0/3829180536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd6015e4c0 0x7fbd600b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:10:18.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.653+0000 7fbd5ea00640 1 --2- 172.21.15.159:0/3829180536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd440039e0 0x7fbd44003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:18.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.654+0000 7fbd56000640 1 --2- 172.21.15.159:0/3829180536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd44004800 0x7fbd44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:18.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.654+0000 7fbd5f400640 1 --2- 172.21.15.159:0/3829180536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd440051c0 0x7fbd4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:18.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.654+0000 7fbd5ea00640 1 -- 172.21.15.159:0/3829180536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd44004800 msgr2=0x7fbd44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:18.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.654+0000 7fbd5ea00640 1 --2- 172.21.15.159:0/3829180536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd44004800 0x7fbd44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:18.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.654+0000 7fbd5ea00640 1 -- 172.21.15.159:0/3829180536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd440051c0 msgr2=0x7fbd4400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:18.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.654+0000 7fbd5ea00640 1 --2- 172.21.15.159:0/3829180536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd440051c0 0x7fbd4400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:18.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.654+0000 7fbd5ea00640 1 -- 172.21.15.159:0/3829180536 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd44003520 con 0x7fbd440039e0 2024-04-04T06:10:18.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.654+0000 7fbd5f400640 1 --2- 172.21.15.159:0/3829180536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd440051c0 0x7fbd4400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:10:18.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.655+0000 7fbd5ea00640 1 --2- 172.21.15.159:0/3829180536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd440039e0 0x7fbd44003e00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fbd50006fd0 tx=0x7fbd44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9414115492ada35c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:18.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.655+0000 7fbd5e000640 1 -- 172.21.15.159:0/3829180536 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5000f040 con 0x7fbd440039e0 2024-04-04T06:10:18.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.656+0000 7fbd5e000640 1 -- 172.21.15.159:0/3829180536 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd50033600 con 0x7fbd440039e0 2024-04-04T06:10:18.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.656+0000 7fbd5e000640 1 -- 172.21.15.159:0/3829180536 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd500339d0 con 0x7fbd440039e0 2024-04-04T06:10:18.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.656+0000 7fbd5fe00640 1 -- 172.21.15.159:0/3829180536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd440039e0 msgr2=0x7fbd44003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:18.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.656+0000 7fbd5fe00640 1 --2- 172.21.15.159:0/3829180536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd440039e0 0x7fbd44003e00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fbd50006fd0 tx=0x7fbd44005830 comp rx=0 tx=0).stop 2024-04-04T06:10:18.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.656+0000 7fbd5f400640 1 -- 172.21.15.159:0/3829180536 reap_dead start 2024-04-04T06:10:18.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.656+0000 7fbd5fe00640 1 -- 172.21.15.159:0/3829180536 shutdown_connections 2024-04-04T06:10:18.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.656+0000 7fbd5fe00640 1 -- 172.21.15.159:0/3829180536 >> 172.21.15.159:0/3829180536 conn(0x7fbd600676a0 msgr2=0x7fbd60067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:18.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.656+0000 7fbd5fe00640 1 -- 172.21.15.159:0/3829180536 shutdown_connections 2024-04-04T06:10:18.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.656+0000 7fbd5fe00640 1 -- 172.21.15.159:0/3829180536 wait complete. 2024-04-04T06:10:18.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.658+0000 7fbd5fe00640 1 Processor -- start 2024-04-04T06:10:18.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.658+0000 7fbd5fe00640 1 -- start start 2024-04-04T06:10:18.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.659+0000 7fbd5fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd440039e0 0x7fbd600ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:18.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.659+0000 7fbd5fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd601553e0 0x7fbd600cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:18.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.659+0000 7fbd5fe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd600cc930 0x7fbd600ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:18.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.659+0000 7fbd5fe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbd600d3320 con 0x7fbd440039e0 2024-04-04T06:10:18.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.659+0000 7fbd5fe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbd600d4040 con 0x7fbd601553e0 2024-04-04T06:10:18.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.659+0000 7fbd5fe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbd600d2350 con 0x7fbd600cc930 2024-04-04T06:10:18.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.659+0000 7fbd5ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd440039e0 0x7fbd600ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:18.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.659+0000 7fbd5ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd440039e0 0x7fbd600ca4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55458/0 (socket says 172.21.15.159:55458) 2024-04-04T06:10:18.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.659+0000 7fbd5ea00640 1 -- 172.21.15.159:0/195800708 learned_addr learned my addr 172.21.15.159:0/195800708 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:18.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.660+0000 7fbd5ea00640 1 -- 172.21.15.159:0/195800708 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd600cc930 msgr2=0x7fbd600ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:18.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.660+0000 7fbd5ea00640 1 --2- 172.21.15.159:0/195800708 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd600cc930 0x7fbd600ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:18.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.660+0000 7fbd5ea00640 1 -- 172.21.15.159:0/195800708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd601553e0 msgr2=0x7fbd600cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:18.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.660+0000 7fbd5ea00640 1 --2- 172.21.15.159:0/195800708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd601553e0 0x7fbd600cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:18.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.660+0000 7fbd5ea00640 1 -- 172.21.15.159:0/195800708 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd600caa10 con 0x7fbd440039e0 2024-04-04T06:10:18.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.660+0000 7fbd5ea00640 1 --2- 172.21.15.159:0/195800708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd440039e0 0x7fbd600ca4d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fbd5002bdc0 tx=0x7fbd500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:18.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.660+0000 7fbd57400640 1 -- 172.21.15.159:0/195800708 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd500329e0 con 0x7fbd440039e0 2024-04-04T06:10:18.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.660+0000 7fbd57400640 1 -- 172.21.15.159:0/195800708 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd50046070 con 0x7fbd440039e0 2024-04-04T06:10:18.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.660+0000 7fbd57400640 1 -- 172.21.15.159:0/195800708 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50032da0 con 0x7fbd440039e0 2024-04-04T06:10:18.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.660+0000 7fbd5fe00640 1 -- 172.21.15.159:0/195800708 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd60157f10 con 0x7fbd440039e0 2024-04-04T06:10:18.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.660+0000 7fbd5fe00640 1 -- 172.21.15.159:0/195800708 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd601584d0 con 0x7fbd440039e0 2024-04-04T06:10:18.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.662+0000 7fbd5fe00640 1 -- 172.21.15.159:0/195800708 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd60075940 con 0x7fbd440039e0 2024-04-04T06:10:18.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.663+0000 7fbd57400640 1 -- 172.21.15.159:0/195800708 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd5003b040 con 0x7fbd440039e0 2024-04-04T06:10:18.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.664+0000 7fbd57400640 1 --2- 172.21.15.159:0/195800708 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd3c06c8f0 0x7fbd3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:18.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.664+0000 7fbd57400640 1 -- 172.21.15.159:0/195800708 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fbd500ad550 con 0x7fbd440039e0 2024-04-04T06:10:18.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.664+0000 7fbd56000640 1 --2- 172.21.15.159:0/195800708 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd3c06c8f0 0x7fbd3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:18.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.666+0000 7fbd56000640 1 --2- 172.21.15.159:0/195800708 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd3c06c8f0 0x7fbd3c06edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbd60161c30 tx=0x7fbd48001250 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:18.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.669+0000 7fbd57400640 1 -- 172.21.15.159:0/195800708 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbd50031300 con 0x7fbd440039e0 2024-04-04T06:10:18.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.868+0000 7fbd5fe00640 1 -- 172.21.15.159:0/195800708 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbd60154c50 con 0x7fbd440039e0 2024-04-04T06:10:18.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.869+0000 7fbd57400640 1 -- 172.21.15.159:0/195800708 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) ==== 100+0+11 (secure 0 0 0) 0x7fbd5007b850 con 0x7fbd440039e0 2024-04-04T06:10:18.862 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:10:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.871+0000 7fbd5fe00640 1 -- 172.21.15.159:0/195800708 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd3c06c8f0 msgr2=0x7fbd3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.871+0000 7fbd5fe00640 1 --2- 172.21.15.159:0/195800708 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd3c06c8f0 0x7fbd3c06edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbd60161c30 tx=0x7fbd48001250 comp rx=0 tx=0).stop 2024-04-04T06:10:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.871+0000 7fbd5fe00640 1 -- 172.21.15.159:0/195800708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd440039e0 msgr2=0x7fbd600ca4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.871+0000 7fbd5fe00640 1 --2- 172.21.15.159:0/195800708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd440039e0 0x7fbd600ca4d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fbd5002bdc0 tx=0x7fbd500046c0 comp rx=0 tx=0).stop 2024-04-04T06:10:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.871+0000 7fbd5fe00640 1 -- 172.21.15.159:0/195800708 shutdown_connections 2024-04-04T06:10:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.871+0000 7fbd5fe00640 1 --2- 172.21.15.159:0/195800708 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd600cc930 0x7fbd600ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.871+0000 7fbd5fe00640 1 --2- 172.21.15.159:0/195800708 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd3c06c8f0 0x7fbd3c06edb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.871+0000 7fbd5fe00640 1 --2- 172.21.15.159:0/195800708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd601553e0 0x7fbd600cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.871+0000 7fbd5fe00640 1 --2- 172.21.15.159:0/195800708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd440039e0 0x7fbd600ca4d0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.871+0000 7fbd5fe00640 1 -- 172.21.15.159:0/195800708 >> 172.21.15.159:0/195800708 conn(0x7fbd600676a0 msgr2=0x7fbd6015b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.871+0000 7fbd5fe00640 1 -- 172.21.15.159:0/195800708 shutdown_connections 2024-04-04T06:10:18.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:18.871+0000 7fbd5fe00640 1 -- 172.21.15.159:0/195800708 wait complete. 2024-04-04T06:10:18.884 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-04T06:10:19.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.099+0000 7f5d59c00640 1 Processor -- start 2024-04-04T06:10:19.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.100+0000 7f5d59c00640 1 -- start start 2024-04-04T06:10:19.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.101+0000 7f5d59c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d540cf050 0x7f5d5415f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:19.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.101+0000 7f5d59c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5d540d2110 0x7f5d540d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:19.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.101+0000 7f5d59c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d540cf6c0 con 0x7f5d540d0e70 2024-04-04T06:10:19.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.101+0000 7f5d59c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f5d5415fc20 con 0x7f5d540d2110 2024-04-04T06:10:19.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.101+0000 7f5d59c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5d540cf9a0 con 0x7f5d540cf050 2024-04-04T06:10:19.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.101+0000 7f5d53400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5d540d2110 0x7f5d540d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:19.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.101+0000 7f5d53400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5d540d2110 0x7f5d540d25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55462/0 (socket says 172.21.15.159:55462) 2024-04-04T06:10:19.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.101+0000 7f5d53400640 1 -- 172.21.15.159:0/1611456795 learned_addr learned my addr 172.21.15.159:0/1611456795 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:19.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.102+0000 7f5d4be00640 1 --2- 172.21.15.159:0/1611456795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d540cf050 0x7f5d5415f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:19.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.102+0000 7f5d53400640 1 -- 172.21.15.159:0/1611456795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d540cf050 msgr2=0x7f5d5415f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:19.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.102+0000 7f5d53400640 1 --2- 172.21.15.159:0/1611456795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d540cf050 0x7f5d5415f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:19.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.102+0000 7f5d53400640 1 -- 172.21.15.159:0/1611456795 >> v1:172.21.15.159:6789/0 conn(0x7f5d540d0e70 legacy=0x7f5d540d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:19.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.102+0000 7f5d52000640 1 -- 172.21.15.159:0/1611456795 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3609863330 0 0) 0x7f5d540cf6c0 con 0x7f5d540d0e70 2024-04-04T06:10:19.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.102+0000 7f5d53400640 1 -- 172.21.15.159:0/1611456795 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d540cfc80 con 0x7f5d540d2110 2024-04-04T06:10:19.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.104+0000 7f5d53400640 1 --2- 172.21.15.159:0/1611456795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d540d2110 0x7f5d540d25a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009e50 tx=0x7f5d4c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3fa8ee87db4d48 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:19.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.104+0000 7f5d52000640 1 -- 172.21.15.159:0/1611456795 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c02cb00 con 0x7f5d540d2110 2024-04-04T06:10:19.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.104+0000 7f5d52000640 1 -- 172.21.15.159:0/1611456795 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d4c02cca0 con 0x7f5d540d2110 2024-04-04T06:10:19.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.104+0000 7f5d52000640 1 -- 172.21.15.159:0/1611456795 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c034610 con 0x7f5d540d2110 2024-04-04T06:10:19.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.105+0000 7f5d59c00640 1 -- 172.21.15.159:0/1611456795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d540d2110 msgr2=0x7f5d540d25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:19.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.105+0000 7f5d59c00640 1 --2- 172.21.15.159:0/1611456795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d540d2110 0x7f5d540d25a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009e50 tx=0x7f5d4c02bf60 comp rx=0 tx=0).stop 2024-04-04T06:10:19.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.105+0000 7f5d59c00640 1 -- 172.21.15.159:0/1611456795 shutdown_connections 2024-04-04T06:10:19.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.105+0000 7f5d59c00640 1 --2- 172.21.15.159:0/1611456795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d540cf050 0x7f5d5415f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:19.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.105+0000 7f5d59c00640 1 --2- 172.21.15.159:0/1611456795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d540d2110 0x7f5d540d25a0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:19.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.105+0000 7f5d59c00640 1 -- 172.21.15.159:0/1611456795 >> 172.21.15.159:0/1611456795 conn(0x7f5d5408bfb0 msgr2=0x7f5d540d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:19.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.105+0000 7f5d59c00640 1 -- 172.21.15.159:0/1611456795 shutdown_connections 2024-04-04T06:10:19.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.105+0000 7f5d59c00640 1 -- 172.21.15.159:0/1611456795 wait complete. 2024-04-04T06:10:19.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.107+0000 7f5d59c00640 1 Processor -- start 2024-04-04T06:10:19.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.107+0000 7f5d59c00640 1 -- start start 2024-04-04T06:10:19.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.108+0000 7f5d59c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d540cf050 0x7f5d540ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:19.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.108+0000 7f5d59c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d540d0e70 0x7f5d540b7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:19.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.108+0000 7f5d59c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d540d2110 0x7f5d540cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:19.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.108+0000 7f5d59c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5415e910 con 0x7f5d540d0e70 2024-04-04T06:10:19.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.108+0000 7f5d59c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5d540d0240 con 0x7f5d540d2110 2024-04-04T06:10:19.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.108+0000 7f5d59c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5d5415ea90 con 0x7f5d540cf050 2024-04-04T06:10:19.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.108+0000 7f5d4be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d540d0e70 0x7f5d540b7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:19.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.108+0000 7f5d4be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d540d0e70 0x7f5d540b7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55464/0 (socket says 172.21.15.159:55464) 2024-04-04T06:10:19.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.108+0000 7f5d4be00640 1 -- 172.21.15.159:0/2696635333 learned_addr learned my addr 172.21.15.159:0/2696635333 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:19.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.108+0000 7f5d53400640 1 --2- 172.21.15.159:0/2696635333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d540d2110 0x7f5d540cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:19.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.108+0000 7f5d52a00640 1 --2- 172.21.15.159:0/2696635333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d540cf050 0x7f5d540ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:19.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.108+0000 7f5d4be00640 1 -- 172.21.15.159:0/2696635333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d540cf050 msgr2=0x7f5d540ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:19.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.108+0000 7f5d4be00640 1 --2- 172.21.15.159:0/2696635333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d540cf050 0x7f5d540ca270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:19.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.108+0000 7f5d4be00640 1 -- 172.21.15.159:0/2696635333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d540d2110 msgr2=0x7f5d540cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:19.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.108+0000 7f5d4be00640 1 --2- 172.21.15.159:0/2696635333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d540d2110 0x7f5d540cc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:19.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.108+0000 7f5d4be00640 1 -- 172.21.15.159:0/2696635333 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d540c94b0 con 0x7f5d540d0e70 2024-04-04T06:10:19.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.109+0000 7f5d4be00640 1 --2- 172.21.15.159:0/2696635333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d540d0e70 0x7f5d540b7a00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00d830 tx=0x7f5d3c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:19.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.109+0000 7f5d4aa00640 1 -- 172.21.15.159:0/2696635333 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c00b6c0 con 0x7f5d540d0e70 2024-04-04T06:10:19.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.109+0000 7f5d4aa00640 1 -- 172.21.15.159:0/2696635333 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d3c019070 con 0x7f5d540d0e70 2024-04-04T06:10:19.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.109+0000 7f5d4aa00640 1 -- 172.21.15.159:0/2696635333 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c00bac0 con 0x7f5d540d0e70 2024-04-04T06:10:19.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.109+0000 7f5d59c00640 1 -- 172.21.15.159:0/2696635333 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d540c9790 con 0x7f5d540d0e70 2024-04-04T06:10:19.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.109+0000 7f5d59c00640 1 -- 172.21.15.159:0/2696635333 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d540c9d50 con 0x7f5d540d0e70 2024-04-04T06:10:19.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.111+0000 7f5d59c00640 1 -- 172.21.15.159:0/2696635333 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d540c9f50 con 0x7f5d540d0e70 2024-04-04T06:10:19.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.111+0000 7f5d4aa00640 1 -- 172.21.15.159:0/2696635333 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d3c01e460 con 0x7f5d540d0e70 2024-04-04T06:10:19.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.113+0000 7f5d4aa00640 1 --2- 172.21.15.159:0/2696635333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d3006c8f0 0x7f5d3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:19.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.113+0000 7f5d4aa00640 1 -- 172.21.15.159:0/2696635333 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f5d3c08c1e0 con 0x7f5d540d0e70 2024-04-04T06:10:19.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.113+0000 7f5d52a00640 1 --2- 172.21.15.159:0/2696635333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d3006c8f0 0x7f5d3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:19.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.115+0000 7f5d52a00640 1 --2- 172.21.15.159:0/2696635333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d3006c8f0 0x7f5d3006edb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5d44000f60 tx=0x7f5d44009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:19.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.117+0000 7f5d4aa00640 1 -- 172.21.15.159:0/2696635333 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5d540c9f50 con 0x7f5d540d0e70 2024-04-04T06:10:19.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.455+0000 7f5d59c00640 1 -- 172.21.15.159:0/2696635333 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5d54068cd0 con 0x7f5d540d0e70 2024-04-04T06:10:19.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.456+0000 7f5d4aa00640 1 -- 172.21.15.159:0/2696635333 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v59) ==== 80+0+0 (secure 0 0 0) 0x7f5d3c05a6a0 con 0x7f5d540d0e70 2024-04-04T06:10:19.450 INFO:teuthology.orchestra.run.smithi159.stderr:listed 0 entries 2024-04-04T06:10:19.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.459+0000 7f5d59c00640 1 -- 172.21.15.159:0/2696635333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d3006c8f0 msgr2=0x7f5d3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:19.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.459+0000 7f5d59c00640 1 --2- 172.21.15.159:0/2696635333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d3006c8f0 0x7f5d3006edb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5d44000f60 tx=0x7f5d44009040 comp rx=0 tx=0).stop 2024-04-04T06:10:19.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.459+0000 7f5d59c00640 1 -- 172.21.15.159:0/2696635333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d540d0e70 msgr2=0x7f5d540b7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:19.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.459+0000 7f5d59c00640 1 --2- 172.21.15.159:0/2696635333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d540d0e70 0x7f5d540b7a00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00d830 tx=0x7f5d3c00dcf0 comp rx=0 tx=0).stop 2024-04-04T06:10:19.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.459+0000 7f5d59c00640 1 -- 172.21.15.159:0/2696635333 shutdown_connections 2024-04-04T06:10:19.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.459+0000 7f5d59c00640 1 --2- 172.21.15.159:0/2696635333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d3006c8f0 0x7f5d3006edb0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:19.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.459+0000 7f5d59c00640 1 --2- 172.21.15.159:0/2696635333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d540d2110 0x7f5d540cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:19.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.459+0000 7f5d59c00640 1 --2- 172.21.15.159:0/2696635333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d540d0e70 0x7f5d540b7a00 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:19.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.459+0000 7f5d59c00640 1 --2- 172.21.15.159:0/2696635333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d540cf050 0x7f5d540ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:19.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.459+0000 7f5d59c00640 1 -- 172.21.15.159:0/2696635333 >> 172.21.15.159:0/2696635333 conn(0x7f5d5408bfb0 msgr2=0x7f5d5406de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:19.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.459+0000 7f5d59c00640 1 -- 172.21.15.159:0/2696635333 shutdown_connections 2024-04-04T06:10:19.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.459+0000 7f5d59c00640 1 -- 172.21.15.159:0/2696635333 wait complete. 2024-04-04T06:10:19.469 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-04T06:10:19.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:10:19.470 DEBUG:teuthology.orchestra.run.smithi159:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-04T06:10:19.590 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:10:19.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:10:19.590 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-04T06:10:19.647 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:10:19.648 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:10:19.648 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:10:19.687 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:10:19.687 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:10:19.687 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:10:19.687 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:10:19.727 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:10:19.728 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:10:19.728 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:10:19.728 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:10:19.776 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:10:19.777 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:10:19.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.980+0000 7f307ea00640 1 Processor -- start 2024-04-04T06:10:19.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.980+0000 7f307ea00640 1 -- start start 2024-04-04T06:10:19.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.983+0000 7f307ea00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f307806c7f0 0x7f3078072990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:19.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.983+0000 7f307ea00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f307806cbd0 0x7f30780b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:19.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.983+0000 7f307ea00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30780afea0 con 0x7f30780d04c0 2024-04-04T06:10:19.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.983+0000 7f307ea00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f30780b0180 con 0x7f307806cbd0 2024-04-04T06:10:19.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.983+0000 7f307ea00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f30780b0460 con 0x7f307806c7f0 2024-04-04T06:10:19.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.984+0000 7f307e000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f307806cbd0 0x7f30780b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:19.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.984+0000 7f307e000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f307806cbd0 0x7f30780b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55468/0 (socket says 172.21.15.159:55468) 2024-04-04T06:10:19.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.984+0000 7f307e000640 1 -- 172.21.15.159:0/2132132443 learned_addr learned my addr 172.21.15.159:0/2132132443 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:19.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.984+0000 7f307d600640 1 --2- 172.21.15.159:0/2132132443 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f307806c7f0 0x7f3078072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:19.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.984+0000 7f307e000640 1 -- 172.21.15.159:0/2132132443 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f307806c7f0 msgr2=0x7f3078072990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:19.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.984+0000 7f307e000640 1 --2- 172.21.15.159:0/2132132443 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f307806c7f0 0x7f3078072990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:19.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.984+0000 7f307e000640 1 -- 172.21.15.159:0/2132132443 >> v1:172.21.15.178:6789/0 conn(0x7f30780d04c0 legacy=0x7f30780700b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:19.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.984+0000 7f307e000640 1 -- 172.21.15.159:0/2132132443 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307806f8b0 con 0x7f307806cbd0 2024-04-04T06:10:19.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.986+0000 7f307e000640 1 --2- 172.21.15.159:0/2132132443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f307806cbd0 0x7f30780b3ac0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f3070009e50 tx=0x7f307002bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3057aecfd7cdad03 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:19.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.986+0000 7f3077e00640 1 -- 172.21.15.159:0/2132132443 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307002cb90 con 0x7f307806cbd0 2024-04-04T06:10:19.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.986+0000 7f3077e00640 1 -- 172.21.15.159:0/2132132443 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f307002cd30 con 0x7f307806cbd0 2024-04-04T06:10:19.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.986+0000 7f3077e00640 1 -- 172.21.15.159:0/2132132443 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070034610 con 0x7f307806cbd0 2024-04-04T06:10:19.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.986+0000 7f307ea00640 1 -- 172.21.15.159:0/2132132443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f307806cbd0 msgr2=0x7f30780b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:19.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.986+0000 7f307ea00640 1 --2- 172.21.15.159:0/2132132443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f307806cbd0 0x7f30780b3ac0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f3070009e50 tx=0x7f307002bff0 comp rx=0 tx=0).stop 2024-04-04T06:10:19.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.987+0000 7f307ea00640 1 -- 172.21.15.159:0/2132132443 shutdown_connections 2024-04-04T06:10:19.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.987+0000 7f307ea00640 1 --2- 172.21.15.159:0/2132132443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f307806cbd0 0x7f30780b3ac0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:19.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.987+0000 7f307ea00640 1 --2- 172.21.15.159:0/2132132443 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f307806c7f0 0x7f3078072990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:19.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.987+0000 7f307ea00640 1 -- 172.21.15.159:0/2132132443 >> 172.21.15.159:0/2132132443 conn(0x7f3078071e70 msgr2=0x7f3078072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:19.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.987+0000 7f307ea00640 1 -- 172.21.15.159:0/2132132443 shutdown_connections 2024-04-04T06:10:19.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.987+0000 7f307ea00640 1 -- 172.21.15.159:0/2132132443 wait complete. 2024-04-04T06:10:19.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.988+0000 7f307ea00640 1 Processor -- start 2024-04-04T06:10:19.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.989+0000 7f307ea00640 1 -- start start 2024-04-04T06:10:19.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.989+0000 7f307ea00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f307806c7f0 0x7f3078141f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:19.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.989+0000 7f307ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f307806cbd0 0x7f30780b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:19.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.989+0000 7f307ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30780d04c0 0x7f30780b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:19.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.989+0000 7f307ea00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3078072ed0 con 0x7f307806cbd0 2024-04-04T06:10:19.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.989+0000 7f307ea00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f30780707c0 con 0x7f30780d04c0 2024-04-04T06:10:19.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.989+0000 7f307ea00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f30780b4000 con 0x7f307806c7f0 2024-04-04T06:10:19.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.990+0000 7f307cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f307806cbd0 0x7f30780b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:19.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.990+0000 7f307cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f307806cbd0 0x7f30780b1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55480/0 (socket says 172.21.15.159:55480) 2024-04-04T06:10:19.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.990+0000 7f307cc00640 1 -- 172.21.15.159:0/3985015934 learned_addr learned my addr 172.21.15.159:0/3985015934 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:19.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.990+0000 7f307e000640 1 --2- 172.21.15.159:0/3985015934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30780d04c0 0x7f30780b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:19.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.990+0000 7f307d600640 1 --2- 172.21.15.159:0/3985015934 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f307806c7f0 0x7f3078141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:19.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.990+0000 7f307cc00640 1 -- 172.21.15.159:0/3985015934 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f307806c7f0 msgr2=0x7f3078141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:19.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.990+0000 7f307cc00640 1 --2- 172.21.15.159:0/3985015934 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f307806c7f0 0x7f3078141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:19.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.990+0000 7f307cc00640 1 -- 172.21.15.159:0/3985015934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30780d04c0 msgr2=0x7f30780b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:19.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.990+0000 7f307cc00640 1 --2- 172.21.15.159:0/3985015934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30780d04c0 0x7f30780b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:19.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.990+0000 7f307cc00640 1 -- 172.21.15.159:0/3985015934 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3078166050 con 0x7f307806cbd0 2024-04-04T06:10:19.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.991+0000 7f307cc00640 1 --2- 172.21.15.159:0/3985015934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f307806cbd0 0x7f30780b1a90 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f306400c440 tx=0x7f306400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:19.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.991+0000 7f3075600640 1 -- 172.21.15.159:0/3985015934 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30640052d0 con 0x7f307806cbd0 2024-04-04T06:10:19.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.991+0000 7f3075600640 1 -- 172.21.15.159:0/3985015934 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3064005470 con 0x7f307806cbd0 2024-04-04T06:10:19.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.991+0000 7f3075600640 1 -- 172.21.15.159:0/3985015934 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064007260 con 0x7f307806cbd0 2024-04-04T06:10:19.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.991+0000 7f307ea00640 1 -- 172.21.15.159:0/3985015934 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f307814af00 con 0x7f307806cbd0 2024-04-04T06:10:19.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.991+0000 7f307ea00640 1 -- 172.21.15.159:0/3985015934 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f307814b4c0 con 0x7f307806cbd0 2024-04-04T06:10:19.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.992+0000 7f307ea00640 1 -- 172.21.15.159:0/3985015934 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3078075940 con 0x7f307806cbd0 2024-04-04T06:10:19.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.993+0000 7f3075600640 1 -- 172.21.15.159:0/3985015934 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3064007400 con 0x7f307806cbd0 2024-04-04T06:10:19.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.995+0000 7f3075600640 1 --2- 172.21.15.159:0/3985015934 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f305806c8f0 0x7f305806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:19.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.995+0000 7f3075600640 1 -- 172.21.15.159:0/3985015934 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f306408afc0 con 0x7f307806cbd0 2024-04-04T06:10:19.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.995+0000 7f307d600640 1 --2- 172.21.15.159:0/3985015934 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f305806c8f0 0x7f305806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:19.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.996+0000 7f307d600640 1 --2- 172.21.15.159:0/3985015934 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f305806c8f0 0x7f305806edb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f306c006fd0 tx=0x7f306c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:19.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:19.998+0000 7f3075600640 1 -- 172.21.15.159:0/3985015934 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f30640593e0 con 0x7f307806cbd0 2024-04-04T06:10:20.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.242+0000 7f307ea00640 1 -- 172.21.15.159:0/3985015934 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3078067860 con 0x7f307806cbd0 2024-04-04T06:10:20.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.243+0000 7f3075600640 1 -- 172.21.15.159:0/3985015934 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5188 (secure 0 0 0) 0x7f3064058ce0 con 0x7f307806cbd0 2024-04-04T06:10:20.236 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:20.237 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5191,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3507245876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3507245876},{"type":"v1","addr":"172.21.15.159:6835","nonce":3507245876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/967262871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":967262871},{"type":"v1","addr":"172.21.15.178:6835","nonce":967262871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2969351496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2969351496},{"type":"v1","addr":"172.21.15.178:6837","nonce":2969351496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:08:57.020444+0000","modified":"2024-04-04T06:09:49.628589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6039":{"gid":6039,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":16,"addr":"172.21.15.159:6834/962475038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":962475038},{"type":"v1","addr":"172.21.15.159:6834","nonce":962475038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6039,"qdb_cluster":[6039]},"id":4}]} 2024-04-04T06:10:20.237 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 35 2024-04-04T06:10:20.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.246+0000 7f307ea00640 1 -- 172.21.15.159:0/3985015934 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f305806c8f0 msgr2=0x7f305806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:20.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.246+0000 7f307ea00640 1 --2- 172.21.15.159:0/3985015934 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f305806c8f0 0x7f305806edb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f306c006fd0 tx=0x7f306c008040 comp rx=0 tx=0).stop 2024-04-04T06:10:20.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.246+0000 7f307ea00640 1 -- 172.21.15.159:0/3985015934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f307806cbd0 msgr2=0x7f30780b1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:20.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.246+0000 7f307ea00640 1 --2- 172.21.15.159:0/3985015934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f307806cbd0 0x7f30780b1a90 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f306400c440 tx=0x7f306400c900 comp rx=0 tx=0).stop 2024-04-04T06:10:20.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.246+0000 7f307ea00640 1 -- 172.21.15.159:0/3985015934 shutdown_connections 2024-04-04T06:10:20.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.246+0000 7f307ea00640 1 --2- 172.21.15.159:0/3985015934 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f305806c8f0 0x7f305806edb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:20.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.246+0000 7f307ea00640 1 --2- 172.21.15.159:0/3985015934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30780d04c0 0x7f30780b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:20.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.246+0000 7f307ea00640 1 --2- 172.21.15.159:0/3985015934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f307806cbd0 0x7f30780b1a90 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:20.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.246+0000 7f307ea00640 1 --2- 172.21.15.159:0/3985015934 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f307806c7f0 0x7f3078141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:20.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.246+0000 7f307ea00640 1 -- 172.21.15.159:0/3985015934 >> 172.21.15.159:0/3985015934 conn(0x7f3078071e70 msgr2=0x7f3078157970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:20.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.246+0000 7f307ea00640 1 -- 172.21.15.159:0/3985015934 shutdown_connections 2024-04-04T06:10:20.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.246+0000 7f307ea00640 1 -- 172.21.15.159:0/3985015934 wait complete. 2024-04-04T06:10:20.258 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:10:20.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.470+0000 7feed0800640 1 Processor -- start 2024-04-04T06:10:20.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.470+0000 7feed0800640 1 -- start start 2024-04-04T06:10:20.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.472+0000 7feed0800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7feec80cd300 0x7feec80ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:20.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.472+0000 7feed0800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feec80cd770 con 0x7feec80d0490 2024-04-04T06:10:20.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.472+0000 7feed0800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feec806b610 con 0x7feec80b0420 2024-04-04T06:10:20.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.472+0000 7feed0800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7feec815eca0 con 0x7feec80cd300 2024-04-04T06:10:20.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.472+0000 7feecde00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7feec80d0490 0x7feec80d0870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45854/0 (socket says 172.21.15.159:45854) 2024-04-04T06:10:20.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.472+0000 7feece800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7feec80cd300 0x7feec80ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:20.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.472+0000 7feecde00640 1 -- 172.21.15.159:0/1279681614 learned_addr learned my addr 172.21.15.159:0/1279681614 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:20.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.473+0000 7feece800640 1 -- 172.21.15.159:0/1279681614 >> v1:172.21.15.178:6789/0 conn(0x7feec80b0420 legacy=0x7feec80cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:20.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.473+0000 7feece800640 1 -- 172.21.15.159:0/1279681614 >> v1:172.21.15.159:6789/0 conn(0x7feec80d0490 legacy=0x7feec80d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:20.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.473+0000 7feecca00640 1 -- 172.21.15.159:0/1279681614 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1705160347 0 0) 0x7feec80cd770 con 0x7feec80d0490 2024-04-04T06:10:20.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.473+0000 7feece800640 1 -- 172.21.15.159:0/1279681614 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec806b8f0 con 0x7feec80cd300 2024-04-04T06:10:20.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.474+0000 7feece800640 1 --2- 172.21.15.159:0/1279681614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feec80cd300 0x7feec80ce1f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7feec4009d90 tx=0x7feec402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835128d57d5d6075 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:20.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.474+0000 7feecca00640 1 -- 172.21.15.159:0/1279681614 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4039070 con 0x7feec80cd300 2024-04-04T06:10:20.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.474+0000 7feecca00640 1 -- 172.21.15.159:0/1279681614 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec402cc60 con 0x7feec80cd300 2024-04-04T06:10:20.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.474+0000 7feecca00640 1 -- 172.21.15.159:0/1279681614 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4034660 con 0x7feec80cd300 2024-04-04T06:10:20.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.475+0000 7feed0800640 1 -- 172.21.15.159:0/1279681614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feec80cd300 msgr2=0x7feec80ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:20.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.475+0000 7feed0800640 1 --2- 172.21.15.159:0/1279681614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feec80cd300 0x7feec80ce1f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7feec4009d90 tx=0x7feec402beb0 comp rx=0 tx=0).stop 2024-04-04T06:10:20.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.475+0000 7feed0800640 1 -- 172.21.15.159:0/1279681614 shutdown_connections 2024-04-04T06:10:20.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.475+0000 7feed0800640 1 --2- 172.21.15.159:0/1279681614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feec80cd300 0x7feec80ce1f0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:20.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.475+0000 7feed0800640 1 -- 172.21.15.159:0/1279681614 >> 172.21.15.159:0/1279681614 conn(0x7feec80b2d40 msgr2=0x7feec80b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:20.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.475+0000 7feed0800640 1 -- 172.21.15.159:0/1279681614 shutdown_connections 2024-04-04T06:10:20.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.475+0000 7feed0800640 1 -- 172.21.15.159:0/1279681614 wait complete. 2024-04-04T06:10:20.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.477+0000 7feed0800640 1 Processor -- start 2024-04-04T06:10:20.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.477+0000 7feed0800640 1 -- start start 2024-04-04T06:10:20.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.478+0000 7feed0800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feec80b0420 0x7feec8156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:20.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.478+0000 7feed0800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec80cd300 0x7feec80c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:20.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.478+0000 7feed0800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feec80d0490 0x7feec80c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:20.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.478+0000 7feed0800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feec806beb0 con 0x7feec80b0420 2024-04-04T06:10:20.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.478+0000 7feed0800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feec806c220 con 0x7feec80d0490 2024-04-04T06:10:20.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.478+0000 7feed0800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feec80cb7c0 con 0x7feec80cd300 2024-04-04T06:10:20.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.478+0000 7feecde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feec80b0420 0x7feec8156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:20.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.478+0000 7feecde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feec80b0420 0x7feec8156960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55506/0 (socket says 172.21.15.159:55506) 2024-04-04T06:10:20.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.478+0000 7feecde00640 1 -- 172.21.15.159:0/2118952564 learned_addr learned my addr 172.21.15.159:0/2118952564 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:20.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.478+0000 7feece800640 1 --2- 172.21.15.159:0/2118952564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feec80d0490 0x7feec80c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:20.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.478+0000 7feecd400640 1 --2- 172.21.15.159:0/2118952564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec80cd300 0x7feec80c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:20.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.478+0000 7feecde00640 1 -- 172.21.15.159:0/2118952564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec80cd300 msgr2=0x7feec80c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:20.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.478+0000 7feecde00640 1 --2- 172.21.15.159:0/2118952564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec80cd300 0x7feec80c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:20.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.478+0000 7feecde00640 1 -- 172.21.15.159:0/2118952564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feec80d0490 msgr2=0x7feec80c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:20.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.478+0000 7feecde00640 1 --2- 172.21.15.159:0/2118952564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feec80d0490 0x7feec80c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:20.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.478+0000 7feecde00640 1 -- 172.21.15.159:0/2118952564 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec80b5370 con 0x7feec80b0420 2024-04-04T06:10:20.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.478+0000 7feecd400640 1 --2- 172.21.15.159:0/2118952564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec80cd300 0x7feec80c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:20.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.478+0000 7feece800640 1 --2- 172.21.15.159:0/2118952564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feec80d0490 0x7feec80c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:20.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.479+0000 7feecde00640 1 --2- 172.21.15.159:0/2118952564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feec80b0420 0x7feec8156960 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7feebc00c430 tx=0x7feebc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:20.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.479+0000 7feec2000640 1 -- 172.21.15.159:0/2118952564 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc0074e0 con 0x7feec80b0420 2024-04-04T06:10:20.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.479+0000 7feec2000640 1 -- 172.21.15.159:0/2118952564 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feebc0047e0 con 0x7feec80b0420 2024-04-04T06:10:20.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.479+0000 7feec2000640 1 -- 172.21.15.159:0/2118952564 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc004bd0 con 0x7feec80b0420 2024-04-04T06:10:20.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.479+0000 7feed0800640 1 -- 172.21.15.159:0/2118952564 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec80b5650 con 0x7feec80b0420 2024-04-04T06:10:20.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.479+0000 7feed0800640 1 -- 172.21.15.159:0/2118952564 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec80b5c10 con 0x7feec80b0420 2024-04-04T06:10:20.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.480+0000 7feed0800640 1 -- 172.21.15.159:0/2118952564 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec8068cd0 con 0x7feec80b0420 2024-04-04T06:10:20.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.481+0000 7feec2000640 1 -- 172.21.15.159:0/2118952564 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feebc011070 con 0x7feec80b0420 2024-04-04T06:10:20.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.482+0000 7feec2000640 1 --2- 172.21.15.159:0/2118952564 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feeac06c8f0 0x7feeac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:20.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.482+0000 7feec2000640 1 -- 172.21.15.159:0/2118952564 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7feebc08b840 con 0x7feec80b0420 2024-04-04T06:10:20.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.483+0000 7feecd400640 1 --2- 172.21.15.159:0/2118952564 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feeac06c8f0 0x7feeac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:20.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.484+0000 7feecd400640 1 --2- 172.21.15.159:0/2118952564 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feeac06c8f0 0x7feeac06edb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7feec80c8860 tx=0x7feeb8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:20.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.492+0000 7feec2000640 1 -- 172.21.15.159:0/2118952564 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7feebc059c60 con 0x7feec80b0420 2024-04-04T06:10:20.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.695+0000 7feed0800640 1 -- 172.21.15.159:0/2118952564 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feec80b0970 con 0x7feec80b0420 2024-04-04T06:10:20.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.698+0000 7feec2000640 1 -- 172.21.15.159:0/2118952564 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16677 (secure 0 0 0) 0x7feebc059560 con 0x7feec80b0420 2024-04-04T06:10:20.694 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:20.695 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":59,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:08:58.581221+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:08:54.848337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-04T06:08:56.003356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:10:20.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.700+0000 7feed0800640 1 -- 172.21.15.159:0/2118952564 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feeac06c8f0 msgr2=0x7feeac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:20.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.700+0000 7feed0800640 1 --2- 172.21.15.159:0/2118952564 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feeac06c8f0 0x7feeac06edb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7feec80c8860 tx=0x7feeb8009040 comp rx=0 tx=0).stop 2024-04-04T06:10:20.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.700+0000 7feed0800640 1 -- 172.21.15.159:0/2118952564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feec80b0420 msgr2=0x7feec8156960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:20.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.700+0000 7feed0800640 1 --2- 172.21.15.159:0/2118952564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feec80b0420 0x7feec8156960 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7feebc00c430 tx=0x7feebc00c8f0 comp rx=0 tx=0).stop 2024-04-04T06:10:20.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.700+0000 7feed0800640 1 -- 172.21.15.159:0/2118952564 shutdown_connections 2024-04-04T06:10:20.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.700+0000 7feed0800640 1 --2- 172.21.15.159:0/2118952564 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feeac06c8f0 0x7feeac06edb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:20.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.700+0000 7feed0800640 1 --2- 172.21.15.159:0/2118952564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feec80d0490 0x7feec80c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:20.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.700+0000 7feed0800640 1 --2- 172.21.15.159:0/2118952564 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feec80cd300 0x7feec80c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:20.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.700+0000 7feed0800640 1 --2- 172.21.15.159:0/2118952564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feec80b0420 0x7feec8156960 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:20.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.700+0000 7feed0800640 1 -- 172.21.15.159:0/2118952564 >> 172.21.15.159:0/2118952564 conn(0x7feec80b2d40 msgr2=0x7feec806e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:20.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.700+0000 7feed0800640 1 -- 172.21.15.159:0/2118952564 shutdown_connections 2024-04-04T06:10:20.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.701+0000 7feed0800640 1 -- 172.21.15.159:0/2118952564 wait complete. 2024-04-04T06:10:20.711 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:08:54.848337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:08:56.003356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:10:20.712 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:10:20.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.923+0000 7fac22c00640 1 Processor -- start 2024-04-04T06:10:20.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.924+0000 7fac22c00640 1 -- start start 2024-04-04T06:10:20.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.925+0000 7fac22c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fac1c15e860 0x7fac1c06d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:20.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.925+0000 7fac22c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fac1c0d4ae0 0x7fac1c0cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:20.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.925+0000 7fac22c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac1c06dfa0 con 0x7fac1c06da30 2024-04-04T06:10:20.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.925+0000 7fac22c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fac1c0cf220 con 0x7fac1c15e860 2024-04-04T06:10:20.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.925+0000 7fac22c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fac1c06e240 con 0x7fac1c0d4ae0 2024-04-04T06:10:20.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.926+0000 7fac1be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fac1c15e860 0x7fac1c06d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:20.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.926+0000 7fac1be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fac1c15e860 0x7fac1c06d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55508/0 (socket says 172.21.15.159:55508) 2024-04-04T06:10:20.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.926+0000 7fac1be00640 1 -- 172.21.15.159:0/1085913256 learned_addr learned my addr 172.21.15.159:0/1085913256 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:20.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.926+0000 7fac1b400640 1 --1- 172.21.15.159:0/1085913256 >> v1:172.21.15.159:6789/0 conn(0x7fac1c06da30 0x7fac1c0d43d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45868/0 (socket says 172.21.15.159:45868) 2024-04-04T06:10:20.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.926+0000 7fac20c00640 1 --2- 172.21.15.159:0/1085913256 >> v2:172.21.15.178:3300/0 conn(0x7fac1c0d4ae0 0x7fac1c0cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:20.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.926+0000 7fac1be00640 1 -- 172.21.15.159:0/1085913256 >> v2:172.21.15.178:3300/0 conn(0x7fac1c0d4ae0 msgr2=0x7fac1c0cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:20.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.926+0000 7fac1be00640 1 --2- 172.21.15.159:0/1085913256 >> v2:172.21.15.178:3300/0 conn(0x7fac1c0d4ae0 0x7fac1c0cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:20.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.926+0000 7fac1be00640 1 -- 172.21.15.159:0/1085913256 >> v1:172.21.15.159:6789/0 conn(0x7fac1c06da30 legacy=0x7fac1c0d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:20.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.926+0000 7fac1be00640 1 -- 172.21.15.159:0/1085913256 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c06e520 con 0x7fac1c15e860 2024-04-04T06:10:20.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.927+0000 7fac1be00640 1 --2- 172.21.15.159:0/1085913256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac1c15e860 0x7fac1c06d4f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0cf3a0 tx=0x7fac0c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc0eb091e0adfe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:20.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.928+0000 7fac1aa00640 1 -- 172.21.15.159:0/1085913256 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c02cb00 con 0x7fac1c15e860 2024-04-04T06:10:20.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.928+0000 7fac1aa00640 1 -- 172.21.15.159:0/1085913256 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac0c02cca0 con 0x7fac1c15e860 2024-04-04T06:10:20.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.928+0000 7fac1aa00640 1 -- 172.21.15.159:0/1085913256 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c034610 con 0x7fac1c15e860 2024-04-04T06:10:20.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.928+0000 7fac22c00640 1 -- 172.21.15.159:0/1085913256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac1c15e860 msgr2=0x7fac1c06d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:20.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.928+0000 7fac22c00640 1 --2- 172.21.15.159:0/1085913256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac1c15e860 0x7fac1c06d4f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0cf3a0 tx=0x7fac0c02bf60 comp rx=0 tx=0).stop 2024-04-04T06:10:20.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.928+0000 7fac22c00640 1 -- 172.21.15.159:0/1085913256 shutdown_connections 2024-04-04T06:10:20.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.928+0000 7fac22c00640 1 --2- 172.21.15.159:0/1085913256 >> v2:172.21.15.178:3300/0 conn(0x7fac1c0d4ae0 0x7fac1c0cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:20.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.928+0000 7fac22c00640 1 --2- 172.21.15.159:0/1085913256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac1c15e860 0x7fac1c06d4f0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:20.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.928+0000 7fac22c00640 1 -- 172.21.15.159:0/1085913256 >> 172.21.15.159:0/1085913256 conn(0x7fac1c09d7d0 msgr2=0x7fac1c08bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:20.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.929+0000 7fac22c00640 1 -- 172.21.15.159:0/1085913256 shutdown_connections 2024-04-04T06:10:20.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.929+0000 7fac22c00640 1 -- 172.21.15.159:0/1085913256 wait complete. 2024-04-04T06:10:20.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.930+0000 7fac22c00640 1 Processor -- start 2024-04-04T06:10:20.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.930+0000 7fac22c00640 1 -- start start 2024-04-04T06:10:20.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.931+0000 7fac22c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac1c06da30 0x7fac1c1f7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:20.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.931+0000 7fac22c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac1c0d4ae0 0x7fac1c0b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:20.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.931+0000 7fac22c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac1c15e860 0x7fac1c0b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:20.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.931+0000 7fac22c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c0cef30 con 0x7fac1c0d4ae0 2024-04-04T06:10:20.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.931+0000 7fac22c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c0cedb0 con 0x7fac1c15e860 2024-04-04T06:10:20.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.931+0000 7fac22c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fac1c0cfbf0 con 0x7fac1c06da30 2024-04-04T06:10:20.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.932+0000 7fac1b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac1c0d4ae0 0x7fac1c0b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:20.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.932+0000 7fac1b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac1c0d4ae0 0x7fac1c0b1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55516/0 (socket says 172.21.15.159:55516) 2024-04-04T06:10:20.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.932+0000 7fac1b400640 1 -- 172.21.15.159:0/2356354540 learned_addr learned my addr 172.21.15.159:0/2356354540 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:20.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.932+0000 7fac20c00640 1 --2- 172.21.15.159:0/2356354540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac1c15e860 0x7fac1c0b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:20.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.932+0000 7fac1b400640 1 -- 172.21.15.159:0/2356354540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac1c06da30 msgr2=0x7fac1c1f7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:20.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.932+0000 7fac1b400640 1 --2- 172.21.15.159:0/2356354540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac1c06da30 0x7fac1c1f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:20.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.932+0000 7fac1b400640 1 -- 172.21.15.159:0/2356354540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac1c15e860 msgr2=0x7fac1c0b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:20.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.932+0000 7fac1b400640 1 --2- 172.21.15.159:0/2356354540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac1c15e860 0x7fac1c0b17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:20.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.932+0000 7fac1b400640 1 -- 172.21.15.159:0/2356354540 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c14c0e0 con 0x7fac1c0d4ae0 2024-04-04T06:10:20.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.932+0000 7fac1b400640 1 --2- 172.21.15.159:0/2356354540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac1c0d4ae0 0x7fac1c0b1270 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fac0400c430 tx=0x7fac0400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:20.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.933+0000 7fac13e00640 1 -- 172.21.15.159:0/2356354540 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac040074e0 con 0x7fac1c0d4ae0 2024-04-04T06:10:20.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.933+0000 7fac13e00640 1 -- 172.21.15.159:0/2356354540 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac04007680 con 0x7fac1c0d4ae0 2024-04-04T06:10:20.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.933+0000 7fac13e00640 1 -- 172.21.15.159:0/2356354540 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac04004a00 con 0x7fac1c0d4ae0 2024-04-04T06:10:20.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.933+0000 7fac22c00640 1 -- 172.21.15.159:0/2356354540 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1c14ed20 con 0x7fac1c0d4ae0 2024-04-04T06:10:20.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.933+0000 7fac22c00640 1 -- 172.21.15.159:0/2356354540 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac1c1f79a0 con 0x7fac1c0d4ae0 2024-04-04T06:10:20.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.934+0000 7fac22c00640 1 -- 172.21.15.159:0/2356354540 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac1c0685c0 con 0x7fac1c0d4ae0 2024-04-04T06:10:20.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.935+0000 7fac13e00640 1 -- 172.21.15.159:0/2356354540 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac0401a020 con 0x7fac1c0d4ae0 2024-04-04T06:10:20.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.936+0000 7fac13e00640 1 --2- 172.21.15.159:0/2356354540 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fabf806c8f0 0x7fabf806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:20.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.936+0000 7fac13e00640 1 -- 172.21.15.159:0/2356354540 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fac0408b370 con 0x7fac1c0d4ae0 2024-04-04T06:10:20.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.937+0000 7fac1be00640 1 --2- 172.21.15.159:0/2356354540 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fabf806c8f0 0x7fabf806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:20.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.938+0000 7fac1be00640 1 --2- 172.21.15.159:0/2356354540 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fabf806c8f0 0x7fabf806edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fac0c035000 tx=0x7fac0c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:20.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:20.946+0000 7fac13e00640 1 -- 172.21.15.159:0/2356354540 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fac04059790 con 0x7fac1c0d4ae0 2024-04-04T06:10:21.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.148+0000 7fac22c00640 1 -- 172.21.15.159:0/2356354540 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fac1c0b28c0 con 0x7fac1c0d4ae0 2024-04-04T06:10:21.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.148+0000 7fac13e00640 1 -- 172.21.15.159:0/2356354540 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7fac04059090 con 0x7fac1c0d4ae0 2024-04-04T06:10:21.142 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:10:21.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.151+0000 7fac22c00640 1 -- 172.21.15.159:0/2356354540 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fabf806c8f0 msgr2=0x7fabf806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:21.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.151+0000 7fac22c00640 1 --2- 172.21.15.159:0/2356354540 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fabf806c8f0 0x7fabf806edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fac0c035000 tx=0x7fac0c02e000 comp rx=0 tx=0).stop 2024-04-04T06:10:21.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.151+0000 7fac22c00640 1 -- 172.21.15.159:0/2356354540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac1c0d4ae0 msgr2=0x7fac1c0b1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:21.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.151+0000 7fac22c00640 1 --2- 172.21.15.159:0/2356354540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac1c0d4ae0 0x7fac1c0b1270 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fac0400c430 tx=0x7fac0400c8f0 comp rx=0 tx=0).stop 2024-04-04T06:10:21.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.151+0000 7fac22c00640 1 -- 172.21.15.159:0/2356354540 shutdown_connections 2024-04-04T06:10:21.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.151+0000 7fac22c00640 1 --2- 172.21.15.159:0/2356354540 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fabf806c8f0 0x7fabf806edb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:21.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.151+0000 7fac22c00640 1 --2- 172.21.15.159:0/2356354540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac1c15e860 0x7fac1c0b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:21.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.151+0000 7fac22c00640 1 --2- 172.21.15.159:0/2356354540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac1c0d4ae0 0x7fac1c0b1270 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:21.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.151+0000 7fac22c00640 1 --2- 172.21.15.159:0/2356354540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac1c06da30 0x7fac1c1f7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:21.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.151+0000 7fac22c00640 1 -- 172.21.15.159:0/2356354540 >> 172.21.15.159:0/2356354540 conn(0x7fac1c09d7d0 msgr2=0x7fac1c0cff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:21.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.151+0000 7fac22c00640 1 -- 172.21.15.159:0/2356354540 shutdown_connections 2024-04-04T06:10:21.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.151+0000 7fac22c00640 1 -- 172.21.15.159:0/2356354540 wait complete. 2024-04-04T06:10:21.164 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:10:21.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.372+0000 7f5b6de00640 1 Processor -- start 2024-04-04T06:10:21.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.373+0000 7f5b6de00640 1 -- start start 2024-04-04T06:10:21.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.374+0000 7f5b6de00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b68155930 0x7f5b680d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:21.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.374+0000 7f5b6de00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b680b40b0 con 0x7f5b680d07c0 2024-04-04T06:10:21.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.374+0000 7f5b6de00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b680b4390 con 0x7f5b68155550 2024-04-04T06:10:21.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.374+0000 7f5b6de00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5b680b4670 con 0x7f5b68155930 2024-04-04T06:10:21.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.374+0000 7f5b67400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f5b680d07c0 0x7f5b680aede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45880/0 (socket says 172.21.15.159:45880) 2024-04-04T06:10:21.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.374+0000 7f5b67400640 1 -- 172.21.15.159:0/252871801 learned_addr learned my addr 172.21.15.159:0/252871801 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:21.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.375+0000 7f5b5e000640 1 --2- 172.21.15.159:0/252871801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b68155930 0x7f5b680d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:21.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.375+0000 7f5b66000640 1 -- 172.21.15.159:0/252871801 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2108791831 0 0) 0x7f5b680b40b0 con 0x7f5b680d07c0 2024-04-04T06:10:21.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.375+0000 7f5b66000640 1 -- 172.21.15.159:0/252871801 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b4c003520 con 0x7f5b680d07c0 2024-04-04T06:10:21.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.375+0000 7f5b66000640 1 -- 172.21.15.159:0/252871801 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4031866057 0 0) 0x7f5b4c003520 con 0x7f5b680d07c0 2024-04-04T06:10:21.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.375+0000 7f5b66000640 1 -- 172.21.15.159:0/252871801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b68155930 msgr2=0x7f5b680d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:21.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.375+0000 7f5b66000640 1 --2- 172.21.15.159:0/252871801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b68155930 0x7f5b680d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:21.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.376+0000 7f5b66000640 1 -- 172.21.15.159:0/252871801 >> v1:172.21.15.178:6789/0 conn(0x7f5b68155550 legacy=0x7f5b6815f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:21.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.376+0000 7f5b66000640 1 -- 172.21.15.159:0/252871801 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b680b4950 con 0x7f5b680d07c0 2024-04-04T06:10:21.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.376+0000 7f5b66000640 1 -- 172.21.15.159:0/252871801 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f5b60001670 con 0x7f5b680d07c0 2024-04-04T06:10:21.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.376+0000 7f5b66000640 1 -- 172.21.15.159:0/252871801 >> v1:172.21.15.159:6789/0 conn(0x7f5b680d07c0 legacy=0x7f5b680aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:21.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.376+0000 7f5b66000640 1 --2- 172.21.15.159:0/252871801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b4c0039e0 0x7f5b4c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:21.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.376+0000 7f5b66000640 1 --2- 172.21.15.159:0/252871801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b4c004390 0x7f5b4c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:21.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.376+0000 7f5b66000640 1 --2- 172.21.15.159:0/252871801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b68155930 0x7f5b4c0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:21.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.376+0000 7f5b66000640 1 -- 172.21.15.159:0/252871801 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5b680b4670 con 0x7f5b4c0039e0 2024-04-04T06:10:21.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.376+0000 7f5b66000640 1 -- 172.21.15.159:0/252871801 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c00dbd0 con 0x7f5b4c004390 2024-04-04T06:10:21.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.376+0000 7f5b66000640 1 -- 172.21.15.159:0/252871801 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5b4c00ddd0 con 0x7f5b68155930 2024-04-04T06:10:21.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.376+0000 7f5b66a00640 1 --2- 172.21.15.159:0/252871801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b4c0039e0 0x7f5b4c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:21.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.376+0000 7f5b67400640 1 --2- 172.21.15.159:0/252871801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b68155930 0x7f5b4c0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:21.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.376+0000 7f5b66a00640 1 -- 172.21.15.159:0/252871801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b68155930 msgr2=0x7f5b4c0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:21.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.376+0000 7f5b5e000640 1 --2- 172.21.15.159:0/252871801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b4c004390 0x7f5b4c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:21.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.376+0000 7f5b66a00640 1 --2- 172.21.15.159:0/252871801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b68155930 0x7f5b4c0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:21.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.376+0000 7f5b66a00640 1 -- 172.21.15.159:0/252871801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b4c004390 msgr2=0x7f5b4c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:21.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.376+0000 7f5b66a00640 1 --2- 172.21.15.159:0/252871801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b4c004390 0x7f5b4c004860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:21.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.376+0000 7f5b66a00640 1 -- 172.21.15.159:0/252871801 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4c003520 con 0x7f5b4c0039e0 2024-04-04T06:10:21.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.376+0000 7f5b67400640 1 --2- 172.21.15.159:0/252871801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b68155930 0x7f5b4c0050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:21.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.378+0000 7f5b66a00640 1 --2- 172.21.15.159:0/252871801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b4c0039e0 0x7f5b4c003dc0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f5b58001f60 tx=0x7f5b4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f17f75a2084f99 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:21.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.378+0000 7f5b66000640 1 -- 172.21.15.159:0/252871801 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58039070 con 0x7f5b4c0039e0 2024-04-04T06:10:21.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.378+0000 7f5b66000640 1 -- 172.21.15.159:0/252871801 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b5802c730 con 0x7f5b4c0039e0 2024-04-04T06:10:21.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.378+0000 7f5b66000640 1 -- 172.21.15.159:0/252871801 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58001b30 con 0x7f5b4c0039e0 2024-04-04T06:10:21.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.379+0000 7f5b6de00640 1 -- 172.21.15.159:0/252871801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b4c0039e0 msgr2=0x7f5b4c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:21.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.379+0000 7f5b6de00640 1 --2- 172.21.15.159:0/252871801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b4c0039e0 0x7f5b4c003dc0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f5b58001f60 tx=0x7f5b4c005830 comp rx=0 tx=0).stop 2024-04-04T06:10:21.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.379+0000 7f5b67400640 1 -- 172.21.15.159:0/252871801 reap_dead start 2024-04-04T06:10:21.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.379+0000 7f5b6de00640 1 -- 172.21.15.159:0/252871801 shutdown_connections 2024-04-04T06:10:21.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.379+0000 7f5b6de00640 1 -- 172.21.15.159:0/252871801 >> 172.21.15.159:0/252871801 conn(0x7f5b6806b750 msgr2=0x7f5b6806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:21.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.379+0000 7f5b6de00640 1 -- 172.21.15.159:0/252871801 shutdown_connections 2024-04-04T06:10:21.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.379+0000 7f5b6de00640 1 -- 172.21.15.159:0/252871801 wait complete. 2024-04-04T06:10:21.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.381+0000 7f5b6de00640 1 Processor -- start 2024-04-04T06:10:21.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.381+0000 7f5b6de00640 1 -- start start 2024-04-04T06:10:21.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.382+0000 7f5b6de00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b4c0039e0 0x7f5b6809da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:21.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.382+0000 7f5b6de00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b6809df50 0x7f5b6815a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:21.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.382+0000 7f5b6de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b681f3690 0x7f5b6815adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:21.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.382+0000 7f5b6de00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6815bba0 con 0x7f5b681f3690 2024-04-04T06:10:21.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.382+0000 7f5b6de00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6815ba20 con 0x7f5b6809df50 2024-04-04T06:10:21.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.382+0000 7f5b6de00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5b680b3cf0 con 0x7f5b4c0039e0 2024-04-04T06:10:21.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.383+0000 7f5b67400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b681f3690 0x7f5b6815adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:21.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.383+0000 7f5b67400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b681f3690 0x7f5b6815adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55528/0 (socket says 172.21.15.159:55528) 2024-04-04T06:10:21.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.383+0000 7f5b67400640 1 -- 172.21.15.159:0/2341119713 learned_addr learned my addr 172.21.15.159:0/2341119713 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:21.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.383+0000 7f5b66a00640 1 --2- 172.21.15.159:0/2341119713 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b4c0039e0 0x7f5b6809da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:21.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.383+0000 7f5b5e000640 1 --2- 172.21.15.159:0/2341119713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b6809df50 0x7f5b6815a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:21.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.383+0000 7f5b67400640 1 -- 172.21.15.159:0/2341119713 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b4c0039e0 msgr2=0x7f5b6809da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:21.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.383+0000 7f5b67400640 1 --2- 172.21.15.159:0/2341119713 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b4c0039e0 0x7f5b6809da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:21.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.383+0000 7f5b67400640 1 -- 172.21.15.159:0/2341119713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b6809df50 msgr2=0x7f5b6815a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:21.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.383+0000 7f5b67400640 1 --2- 172.21.15.159:0/2341119713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b6809df50 0x7f5b6815a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:21.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.383+0000 7f5b67400640 1 -- 172.21.15.159:0/2341119713 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6809a550 con 0x7f5b681f3690 2024-04-04T06:10:21.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.383+0000 7f5b5e000640 1 --2- 172.21.15.159:0/2341119713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b6809df50 0x7f5b6815a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:21.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.383+0000 7f5b67400640 1 --2- 172.21.15.159:0/2341119713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b681f3690 0x7f5b6815adc0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f5b60000c00 tx=0x7f5b6000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:21.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.384+0000 7f5b5fe00640 1 -- 172.21.15.159:0/2341119713 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b600072d0 con 0x7f5b681f3690 2024-04-04T06:10:21.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.384+0000 7f5b5fe00640 1 -- 172.21.15.159:0/2341119713 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b60007470 con 0x7f5b681f3690 2024-04-04T06:10:21.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.384+0000 7f5b6de00640 1 -- 172.21.15.159:0/2341119713 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6809a830 con 0x7f5b681f3690 2024-04-04T06:10:21.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.384+0000 7f5b5fe00640 1 -- 172.21.15.159:0/2341119713 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60004a20 con 0x7f5b681f3690 2024-04-04T06:10:21.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.384+0000 7f5b6de00640 1 -- 172.21.15.159:0/2341119713 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b6809adf0 con 0x7f5b681f3690 2024-04-04T06:10:21.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.385+0000 7f5b6de00640 1 -- 172.21.15.159:0/2341119713 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b6809b1d0 con 0x7f5b681f3690 2024-04-04T06:10:21.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.386+0000 7f5b5fe00640 1 -- 172.21.15.159:0/2341119713 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b6001b060 con 0x7f5b681f3690 2024-04-04T06:10:21.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.387+0000 7f5b5fe00640 1 --2- 172.21.15.159:0/2341119713 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5b4406c8f0 0x7f5b4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:21.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.387+0000 7f5b5fe00640 1 -- 172.21.15.159:0/2341119713 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f5b6008b380 con 0x7f5b681f3690 2024-04-04T06:10:21.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.388+0000 7f5b66a00640 1 --2- 172.21.15.159:0/2341119713 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5b4406c8f0 0x7f5b4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:21.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.389+0000 7f5b66a00640 1 --2- 172.21.15.159:0/2341119713 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5b4406c8f0 0x7f5b4406edb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5b58001f60 tx=0x7f5b58005040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:21.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.391+0000 7f5b5fe00640 1 -- 172.21.15.159:0/2341119713 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5b60059900 con 0x7f5b681f3690 2024-04-04T06:10:21.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.605+0000 7f5b6de00640 1 -- 172.21.15.159:0/2341119713 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5b6809b500 con 0x7f5b681f3690 2024-04-04T06:10:21.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.606+0000 7f5b5fe00640 1 -- 172.21.15.159:0/2341119713 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) ==== 104+0+11 (secure 0 0 0) 0x7f5b60059200 con 0x7f5b681f3690 2024-04-04T06:10:21.600 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:10:21.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.609+0000 7f5b6de00640 1 -- 172.21.15.159:0/2341119713 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5b4406c8f0 msgr2=0x7f5b4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:21.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.609+0000 7f5b6de00640 1 --2- 172.21.15.159:0/2341119713 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5b4406c8f0 0x7f5b4406edb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5b58001f60 tx=0x7f5b58005040 comp rx=0 tx=0).stop 2024-04-04T06:10:21.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.609+0000 7f5b6de00640 1 -- 172.21.15.159:0/2341119713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b681f3690 msgr2=0x7f5b6815adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:21.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.609+0000 7f5b6de00640 1 --2- 172.21.15.159:0/2341119713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b681f3690 0x7f5b6815adc0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f5b60000c00 tx=0x7f5b6000dc00 comp rx=0 tx=0).stop 2024-04-04T06:10:21.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.609+0000 7f5b6de00640 1 -- 172.21.15.159:0/2341119713 shutdown_connections 2024-04-04T06:10:21.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.609+0000 7f5b6de00640 1 --2- 172.21.15.159:0/2341119713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b681f3690 0x7f5b6815adc0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:21.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.609+0000 7f5b6de00640 1 --2- 172.21.15.159:0/2341119713 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5b4406c8f0 0x7f5b4406edb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:21.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.609+0000 7f5b6de00640 1 --2- 172.21.15.159:0/2341119713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b6809df50 0x7f5b6815a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:21.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.609+0000 7f5b6de00640 1 --2- 172.21.15.159:0/2341119713 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5b4c0039e0 0x7f5b6809da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:21.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.609+0000 7f5b6de00640 1 -- 172.21.15.159:0/2341119713 >> 172.21.15.159:0/2341119713 conn(0x7f5b6806b750 msgr2=0x7f5b6806d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:21.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.609+0000 7f5b6de00640 1 -- 172.21.15.159:0/2341119713 shutdown_connections 2024-04-04T06:10:21.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.610+0000 7f5b6de00640 1 -- 172.21.15.159:0/2341119713 wait complete. 2024-04-04T06:10:21.621 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:10:21.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.833+0000 7f5dbf000640 1 Processor -- start 2024-04-04T06:10:21.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.834+0000 7f5dbf000640 1 -- start start 2024-04-04T06:10:21.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.835+0000 7f5dbf000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5db806dd30 0x7f5db80cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:21.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.835+0000 7f5dbf000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5db80ce890 con 0x7f5db80cc880 2024-04-04T06:10:21.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.835+0000 7f5dbf000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5db80ceb50 con 0x7f5db813f190 2024-04-04T06:10:21.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.835+0000 7f5dbf000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f5db80cd230 con 0x7f5db806dd30 2024-04-04T06:10:21.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.836+0000 7f5dbd200640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f5db80cc880 0x7f5db806d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45886/0 (socket says 172.21.15.159:45886) 2024-04-04T06:10:21.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.836+0000 7f5dbd200640 1 -- 172.21.15.159:0/983881478 learned_addr learned my addr 172.21.15.159:0/983881478 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:21.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.836+0000 7f5dbe600640 1 --2- 172.21.15.159:0/983881478 >> v2:172.21.15.178:3300/0 conn(0x7f5db806dd30 0x7f5db80cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:21.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.836+0000 7f5db7e00640 1 -- 172.21.15.159:0/983881478 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 863318443 0 0) 0x7f5db80ce890 con 0x7f5db80cc880 2024-04-04T06:10:21.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.836+0000 7f5db7e00640 1 -- 172.21.15.159:0/983881478 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5da0003520 con 0x7f5db80cc880 2024-04-04T06:10:21.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.836+0000 7f5db7e00640 1 -- 172.21.15.159:0/983881478 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4140512391 0 0) 0x7f5da0003520 con 0x7f5db80cc880 2024-04-04T06:10:21.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.837+0000 7f5db7e00640 1 -- 172.21.15.159:0/983881478 >> v2:172.21.15.178:3300/0 conn(0x7f5db806dd30 msgr2=0x7f5db80cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:21.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.837+0000 7f5db7e00640 1 --2- 172.21.15.159:0/983881478 >> v2:172.21.15.178:3300/0 conn(0x7f5db806dd30 0x7f5db80cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:21.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.837+0000 7f5db7e00640 1 -- 172.21.15.159:0/983881478 >> v1:172.21.15.178:6789/0 conn(0x7f5db813f190 legacy=0x7f5db80cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:21.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.837+0000 7f5db7e00640 1 -- 172.21.15.159:0/983881478 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db80cee30 con 0x7f5db80cc880 2024-04-04T06:10:21.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.837+0000 7f5db7e00640 1 -- 172.21.15.159:0/983881478 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f5da4001670 con 0x7f5db80cc880 2024-04-04T06:10:21.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.837+0000 7f5db7e00640 1 -- 172.21.15.159:0/983881478 >> v1:172.21.15.159:6789/0 conn(0x7f5db80cc880 legacy=0x7f5db806d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:21.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.837+0000 7f5db7e00640 1 --2- 172.21.15.159:0/983881478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5da00039b0 0x7f5da0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:21.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.837+0000 7f5db7e00640 1 --2- 172.21.15.159:0/983881478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5da0004800 0x7f5da0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:21.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.837+0000 7f5db7e00640 1 --2- 172.21.15.159:0/983881478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da00051c0 0x7f5da000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:21.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.837+0000 7f5db7e00640 1 -- 172.21.15.159:0/983881478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5db80cd230 con 0x7f5da00051c0 2024-04-04T06:10:21.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.837+0000 7f5db7e00640 1 -- 172.21.15.159:0/983881478 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5da000c060 con 0x7f5da0004800 2024-04-04T06:10:21.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.837+0000 7f5db7e00640 1 -- 172.21.15.159:0/983881478 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5da000c260 con 0x7f5da00039b0 2024-04-04T06:10:21.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.837+0000 7f5dbe600640 1 --2- 172.21.15.159:0/983881478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da00051c0 0x7f5da000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:21.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.837+0000 7f5dbdc00640 1 --2- 172.21.15.159:0/983881478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5da00039b0 0x7f5da0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:21.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.837+0000 7f5dbd200640 1 --2- 172.21.15.159:0/983881478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5da0004800 0x7f5da0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:21.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.838+0000 7f5dbe600640 1 -- 172.21.15.159:0/983881478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5da00039b0 msgr2=0x7f5da0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:21.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.838+0000 7f5dbe600640 1 --2- 172.21.15.159:0/983881478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5da00039b0 0x7f5da0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:21.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.838+0000 7f5dbe600640 1 -- 172.21.15.159:0/983881478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5da0004800 msgr2=0x7f5da0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:21.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.838+0000 7f5dbe600640 1 --2- 172.21.15.159:0/983881478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5da0004800 0x7f5da0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:21.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.838+0000 7f5dbe600640 1 -- 172.21.15.159:0/983881478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da0003520 con 0x7f5da00051c0 2024-04-04T06:10:21.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.838+0000 7f5dbd200640 1 --2- 172.21.15.159:0/983881478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5da0004800 0x7f5da0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:21.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.838+0000 7f5dbe600640 1 -- 172.21.15.159:0/983881478 reap_dead start 2024-04-04T06:10:21.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.839+0000 7f5dbe600640 1 --2- 172.21.15.159:0/983881478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da00051c0 0x7f5da000b9c0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009fc0 tx=0x7f5da0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575495cb071b8ddb server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:21.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.839+0000 7f5db7e00640 1 -- 172.21.15.159:0/983881478 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0009c10 con 0x7f5da00051c0 2024-04-04T06:10:21.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.839+0000 7f5db7e00640 1 -- 172.21.15.159:0/983881478 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db002cca0 con 0x7f5da00051c0 2024-04-04T06:10:21.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.839+0000 7f5db7e00640 1 -- 172.21.15.159:0/983881478 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0003bd0 con 0x7f5da00051c0 2024-04-04T06:10:21.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.839+0000 7f5dbf000640 1 -- 172.21.15.159:0/983881478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da00051c0 msgr2=0x7f5da000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:21.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.839+0000 7f5dbf000640 1 --2- 172.21.15.159:0/983881478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da00051c0 0x7f5da000b9c0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009fc0 tx=0x7f5da0005830 comp rx=0 tx=0).stop 2024-04-04T06:10:21.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.840+0000 7f5dbf000640 1 -- 172.21.15.159:0/983881478 shutdown_connections 2024-04-04T06:10:21.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.840+0000 7f5dbf000640 1 --2- 172.21.15.159:0/983881478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da00051c0 0x7f5da000b9c0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:21.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.840+0000 7f5dbf000640 1 -- 172.21.15.159:0/983881478 >> 172.21.15.159:0/983881478 conn(0x7f5db80b1ba0 msgr2=0x7f5db80b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:21.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.840+0000 7f5dbf000640 1 -- 172.21.15.159:0/983881478 shutdown_connections 2024-04-04T06:10:21.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.840+0000 7f5dbf000640 1 -- 172.21.15.159:0/983881478 wait complete. 2024-04-04T06:10:21.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.842+0000 7f5dbf000640 1 Processor -- start 2024-04-04T06:10:21.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.843+0000 7f5dbf000640 1 -- start start 2024-04-04T06:10:21.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.844+0000 7f5dbf000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da00051c0 0x7f5db8138200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:21.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.844+0000 7f5dbf000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5db8157960 0x7f5db81366c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:21.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.844+0000 7f5dbf000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5db8136c00 0x7f5db815e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:21.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.844+0000 7f5dbf000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5db813f570 con 0x7f5da00051c0 2024-04-04T06:10:21.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.844+0000 7f5dbf000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5db8160c70 con 0x7f5db8136c00 2024-04-04T06:10:21.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.844+0000 7f5dbf000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5db80cccb0 con 0x7f5db8157960 2024-04-04T06:10:21.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.844+0000 7f5dbdc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da00051c0 0x7f5db8138200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:21.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.844+0000 7f5dbdc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da00051c0 0x7f5db8138200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55542/0 (socket says 172.21.15.159:55542) 2024-04-04T06:10:21.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.845+0000 7f5dbdc00640 1 -- 172.21.15.159:0/3500102595 learned_addr learned my addr 172.21.15.159:0/3500102595 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:21.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.845+0000 7f5dbdc00640 1 -- 172.21.15.159:0/3500102595 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5db8157960 msgr2=0x7f5db81366c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:21.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.845+0000 7f5dbe600640 1 --2- 172.21.15.159:0/3500102595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5db8136c00 0x7f5db815e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:21.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.845+0000 7f5dbd200640 1 --2- 172.21.15.159:0/3500102595 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5db8157960 0x7f5db81366c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:21.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.845+0000 7f5dbdc00640 1 --2- 172.21.15.159:0/3500102595 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5db8157960 0x7f5db81366c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:21.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.845+0000 7f5dbdc00640 1 -- 172.21.15.159:0/3500102595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5db8136c00 msgr2=0x7f5db815e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:21.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.845+0000 7f5dbdc00640 1 --2- 172.21.15.159:0/3500102595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5db8136c00 0x7f5db815e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:21.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.845+0000 7f5dbdc00640 1 -- 172.21.15.159:0/3500102595 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db8137160 con 0x7f5da00051c0 2024-04-04T06:10:21.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.845+0000 7f5dbe600640 1 --2- 172.21.15.159:0/3500102595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5db8136c00 0x7f5db815e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:21.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.845+0000 7f5dbdc00640 1 --2- 172.21.15.159:0/3500102595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da00051c0 0x7f5db8138200 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00e500 tx=0x7f5dac00e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:21.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.846+0000 7f5db5600640 1 -- 172.21.15.159:0/3500102595 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0052e0 con 0x7f5da00051c0 2024-04-04T06:10:21.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.846+0000 7f5db5600640 1 -- 172.21.15.159:0/3500102595 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dac018070 con 0x7f5da00051c0 2024-04-04T06:10:21.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.846+0000 7f5dbf000640 1 -- 172.21.15.159:0/3500102595 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db8137440 con 0x7f5da00051c0 2024-04-04T06:10:21.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.846+0000 7f5db5600640 1 -- 172.21.15.159:0/3500102595 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac003810 con 0x7f5da00051c0 2024-04-04T06:10:21.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.846+0000 7f5dbf000640 1 -- 172.21.15.159:0/3500102595 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db8137a00 con 0x7f5da00051c0 2024-04-04T06:10:21.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.847+0000 7f5dbf000640 1 -- 172.21.15.159:0/3500102595 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db8075940 con 0x7f5da00051c0 2024-04-04T06:10:21.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.848+0000 7f5db5600640 1 -- 172.21.15.159:0/3500102595 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5dac0039b0 con 0x7f5da00051c0 2024-04-04T06:10:21.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.849+0000 7f5db5600640 1 --2- 172.21.15.159:0/3500102595 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d9806c920 0x7f5d9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:21.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.849+0000 7f5db5600640 1 -- 172.21.15.159:0/3500102595 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f5dac08b820 con 0x7f5da00051c0 2024-04-04T06:10:21.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.849+0000 7f5dbd200640 1 --2- 172.21.15.159:0/3500102595 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d9806c920 0x7f5d9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:21.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.850+0000 7f5dbd200640 1 --2- 172.21.15.159:0/3500102595 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d9806c920 0x7f5d9806ede0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5da4001920 tx=0x7f5da4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:21.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:21.853+0000 7f5db5600640 1 -- 172.21.15.159:0/3500102595 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5dac059bc0 con 0x7f5da00051c0 2024-04-04T06:10:22.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.108+0000 7f5dbf000640 1 -- 172.21.15.159:0/3500102595 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5db806ea40 con 0x7f5da00051c0 2024-04-04T06:10:22.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.109+0000 7f5db5600640 1 -- 172.21.15.159:0/3500102595 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) ==== 100+0+11 (secure 0 0 0) 0x7f5dac0594c0 con 0x7f5da00051c0 2024-04-04T06:10:22.102 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:10:22.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.111+0000 7f5dbf000640 1 -- 172.21.15.159:0/3500102595 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d9806c920 msgr2=0x7f5d9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:22.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.111+0000 7f5dbf000640 1 --2- 172.21.15.159:0/3500102595 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d9806c920 0x7f5d9806ede0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5da4001920 tx=0x7f5da4009040 comp rx=0 tx=0).stop 2024-04-04T06:10:22.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.111+0000 7f5dbf000640 1 -- 172.21.15.159:0/3500102595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da00051c0 msgr2=0x7f5db8138200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:22.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.111+0000 7f5dbf000640 1 --2- 172.21.15.159:0/3500102595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da00051c0 0x7f5db8138200 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00e500 tx=0x7f5dac00e9c0 comp rx=0 tx=0).stop 2024-04-04T06:10:22.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.111+0000 7f5dbf000640 1 -- 172.21.15.159:0/3500102595 shutdown_connections 2024-04-04T06:10:22.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.111+0000 7f5dbf000640 1 --2- 172.21.15.159:0/3500102595 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5d9806c920 0x7f5d9806ede0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:22.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.111+0000 7f5dbf000640 1 --2- 172.21.15.159:0/3500102595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5db8136c00 0x7f5db815e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:22.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.111+0000 7f5dbf000640 1 --2- 172.21.15.159:0/3500102595 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5db8157960 0x7f5db81366c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:22.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.111+0000 7f5dbf000640 1 --2- 172.21.15.159:0/3500102595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da00051c0 0x7f5db8138200 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:22.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.111+0000 7f5dbf000640 1 -- 172.21.15.159:0/3500102595 >> 172.21.15.159:0/3500102595 conn(0x7f5db80b1ba0 msgr2=0x7f5db8138b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:22.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.112+0000 7f5dbf000640 1 -- 172.21.15.159:0/3500102595 shutdown_connections 2024-04-04T06:10:22.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.112+0000 7f5dbf000640 1 -- 172.21.15.159:0/3500102595 wait complete. 2024-04-04T06:10:22.123 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:10:22.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.334+0000 7f35eee00640 1 Processor -- start 2024-04-04T06:10:22.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.334+0000 7f35eee00640 1 -- start start 2024-04-04T06:10:22.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.336+0000 7f35eee00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e80cf1b0 0x7f35e80cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:22.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.336+0000 7f35eee00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f35e80d00d0 0x7f35e80b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:22.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.336+0000 7f35eee00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35e8071460 con 0x7f35e81442c0 2024-04-04T06:10:22.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.336+0000 7f35eee00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f35e8071d20 con 0x7f35e80d00d0 2024-04-04T06:10:22.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.336+0000 7f35eee00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f35e80721b0 con 0x7f35e80cf1b0 2024-04-04T06:10:22.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.337+0000 7f35ed000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e80cf1b0 0x7f35e80cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:22.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.337+0000 7f35ed000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e80cf1b0 0x7f35e80cfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:57326/0 (socket says 172.21.15.159:57326) 2024-04-04T06:10:22.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.337+0000 7f35ed000640 1 -- 172.21.15.159:0/75359314 learned_addr learned my addr 172.21.15.159:0/75359314 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:22.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.337+0000 7f35ee400640 1 --2- 172.21.15.159:0/75359314 >> v2:172.21.15.178:3300/0 conn(0x7f35e80d00d0 0x7f35e80b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:22.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.338+0000 7f35ed000640 1 -- 172.21.15.159:0/75359314 >> v2:172.21.15.178:3300/0 conn(0x7f35e80d00d0 msgr2=0x7f35e80b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:22.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.338+0000 7f35ed000640 1 --2- 172.21.15.159:0/75359314 >> v2:172.21.15.178:3300/0 conn(0x7f35e80d00d0 0x7f35e80b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:22.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.338+0000 7f35ed000640 1 -- 172.21.15.159:0/75359314 >> v1:172.21.15.178:6789/0 conn(0x7f35e81442c0 legacy=0x7f35e80ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:22.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.338+0000 7f35ed000640 1 -- 172.21.15.159:0/75359314 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e80716d0 con 0x7f35e80cf1b0 2024-04-04T06:10:22.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.339+0000 7f35ed000640 1 --2- 172.21.15.159:0/75359314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e80cf1b0 0x7f35e80cfb90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009870 tx=0x7f35d402ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84bbdfe001e0aa72 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:22.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.339+0000 7f35e7e00640 1 -- 172.21.15.159:0/75359314 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d402f840 con 0x7f35e80cf1b0 2024-04-04T06:10:22.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.339+0000 7f35e7e00640 1 -- 172.21.15.159:0/75359314 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d402f9e0 con 0x7f35e80cf1b0 2024-04-04T06:10:22.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.339+0000 7f35e7e00640 1 -- 172.21.15.159:0/75359314 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d402fdb0 con 0x7f35e80cf1b0 2024-04-04T06:10:22.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.340+0000 7f35eee00640 1 -- 172.21.15.159:0/75359314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e80cf1b0 msgr2=0x7f35e80cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:22.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.340+0000 7f35eee00640 1 --2- 172.21.15.159:0/75359314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e80cf1b0 0x7f35e80cfb90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009870 tx=0x7f35d402ba20 comp rx=0 tx=0).stop 2024-04-04T06:10:22.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.340+0000 7f35eee00640 1 -- 172.21.15.159:0/75359314 shutdown_connections 2024-04-04T06:10:22.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.340+0000 7f35eee00640 1 --2- 172.21.15.159:0/75359314 >> v2:172.21.15.178:3300/0 conn(0x7f35e80d00d0 0x7f35e80b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:22.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.340+0000 7f35eee00640 1 --2- 172.21.15.159:0/75359314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e80cf1b0 0x7f35e80cfb90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:22.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.340+0000 7f35eee00640 1 -- 172.21.15.159:0/75359314 >> 172.21.15.159:0/75359314 conn(0x7f35e80ccad0 msgr2=0x7f35e80cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:22.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.340+0000 7f35eee00640 1 -- 172.21.15.159:0/75359314 shutdown_connections 2024-04-04T06:10:22.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.340+0000 7f35eee00640 1 -- 172.21.15.159:0/75359314 wait complete. 2024-04-04T06:10:22.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.342+0000 7f35eee00640 1 Processor -- start 2024-04-04T06:10:22.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.342+0000 7f35eee00640 1 -- start start 2024-04-04T06:10:22.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.343+0000 7f35eee00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e80cf1b0 0x7f35e8142920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:22.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.343+0000 7f35eee00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e80d00d0 0x7f35e815c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:22.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.343+0000 7f35eee00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e81442c0 0x7f35e815d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:22.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.343+0000 7f35eee00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81446a0 con 0x7f35e81442c0 2024-04-04T06:10:22.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.343+0000 7f35eee00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35e80d0770 con 0x7f35e80d00d0 2024-04-04T06:10:22.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.343+0000 7f35eee00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f35e80ada50 con 0x7f35e80cf1b0 2024-04-04T06:10:22.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.343+0000 7f35ee400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e81442c0 0x7f35e815d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:22.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.343+0000 7f35ee400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e81442c0 0x7f35e815d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55550/0 (socket says 172.21.15.159:55550) 2024-04-04T06:10:22.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.343+0000 7f35ee400640 1 -- 172.21.15.159:0/441058429 learned_addr learned my addr 172.21.15.159:0/441058429 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:22.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.344+0000 7f35ed000640 1 --2- 172.21.15.159:0/441058429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e80d00d0 0x7f35e815c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:22.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.344+0000 7f35eda00640 1 --2- 172.21.15.159:0/441058429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e80cf1b0 0x7f35e8142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:22.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.344+0000 7f35ee400640 1 -- 172.21.15.159:0/441058429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e80cf1b0 msgr2=0x7f35e8142920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:22.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.344+0000 7f35ee400640 1 --2- 172.21.15.159:0/441058429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e80cf1b0 0x7f35e8142920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:22.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.344+0000 7f35ee400640 1 -- 172.21.15.159:0/441058429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e80d00d0 msgr2=0x7f35e815c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:22.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.344+0000 7f35ee400640 1 --2- 172.21.15.159:0/441058429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e80d00d0 0x7f35e815c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:22.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.344+0000 7f35ee400640 1 -- 172.21.15.159:0/441058429 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e81437a0 con 0x7f35e81442c0 2024-04-04T06:10:22.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.344+0000 7f35eda00640 1 --2- 172.21.15.159:0/441058429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e80cf1b0 0x7f35e8142920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:22.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.344+0000 7f35ee400640 1 --2- 172.21.15.159:0/441058429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e81442c0 0x7f35e815d610 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f35e000efc0 tx=0x7f35e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:22.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.344+0000 7f35e5600640 1 -- 172.21.15.159:0/441058429 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0018020 con 0x7f35e81442c0 2024-04-04T06:10:22.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.344+0000 7f35e5600640 1 -- 172.21.15.159:0/441058429 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e0014480 con 0x7f35e81442c0 2024-04-04T06:10:22.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.344+0000 7f35eee00640 1 -- 172.21.15.159:0/441058429 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e8142f00 con 0x7f35e81442c0 2024-04-04T06:10:22.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.344+0000 7f35e5600640 1 -- 172.21.15.159:0/441058429 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0014870 con 0x7f35e81442c0 2024-04-04T06:10:22.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.345+0000 7f35eee00640 1 -- 172.21.15.159:0/441058429 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e81434c0 con 0x7f35e81442c0 2024-04-04T06:10:22.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.346+0000 7f35eee00640 1 -- 172.21.15.159:0/441058429 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e8075940 con 0x7f35e81442c0 2024-04-04T06:10:22.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.346+0000 7f35e5600640 1 -- 172.21.15.159:0/441058429 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35e0005d10 con 0x7f35e81442c0 2024-04-04T06:10:22.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.348+0000 7f35e5600640 1 --2- 172.21.15.159:0/441058429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35c806c8f0 0x7f35c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:22.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.348+0000 7f35e5600640 1 -- 172.21.15.159:0/441058429 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f35e008c5c0 con 0x7f35e81442c0 2024-04-04T06:10:22.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.348+0000 7f35eda00640 1 --2- 172.21.15.159:0/441058429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35c806c8f0 0x7f35c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:22.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.350+0000 7f35eda00640 1 --2- 172.21.15.159:0/441058429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35c806c8f0 0x7f35c806edb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0015e0 tx=0x7f35dc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:22.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.352+0000 7f35e5600640 1 -- 172.21.15.159:0/441058429 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f35e0022070 con 0x7f35e81442c0 2024-04-04T06:10:22.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.593+0000 7f35eee00640 1 -- 172.21.15.159:0/441058429 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35e80cc140 con 0x7f35e81442c0 2024-04-04T06:10:22.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.594+0000 7f35e5600640 1 -- 172.21.15.159:0/441058429 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5188 (secure 0 0 0) 0x7f35e0010040 con 0x7f35e81442c0 2024-04-04T06:10:22.588 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:22.588 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5191,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3507245876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3507245876},{"type":"v1","addr":"172.21.15.159:6835","nonce":3507245876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/967262871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":967262871},{"type":"v1","addr":"172.21.15.178:6835","nonce":967262871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2969351496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2969351496},{"type":"v1","addr":"172.21.15.178:6837","nonce":2969351496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:08:57.020444+0000","modified":"2024-04-04T06:09:49.628589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6039":{"gid":6039,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":16,"addr":"172.21.15.159:6834/962475038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":962475038},{"type":"v1","addr":"172.21.15.159:6834","nonce":962475038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6039,"qdb_cluster":[6039]},"id":4}]} 2024-04-04T06:10:22.588 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 35 2024-04-04T06:10:22.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.598+0000 7f35eee00640 1 -- 172.21.15.159:0/441058429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35c806c8f0 msgr2=0x7f35c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:22.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.598+0000 7f35eee00640 1 --2- 172.21.15.159:0/441058429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35c806c8f0 0x7f35c806edb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0015e0 tx=0x7f35dc009040 comp rx=0 tx=0).stop 2024-04-04T06:10:22.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.598+0000 7f35eee00640 1 -- 172.21.15.159:0/441058429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e81442c0 msgr2=0x7f35e815d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:22.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.598+0000 7f35eee00640 1 --2- 172.21.15.159:0/441058429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e81442c0 0x7f35e815d610 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f35e000efc0 tx=0x7f35e000c490 comp rx=0 tx=0).stop 2024-04-04T06:10:22.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.598+0000 7f35eee00640 1 -- 172.21.15.159:0/441058429 shutdown_connections 2024-04-04T06:10:22.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.598+0000 7f35eee00640 1 --2- 172.21.15.159:0/441058429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e81442c0 0x7f35e815d610 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:22.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.598+0000 7f35eee00640 1 --2- 172.21.15.159:0/441058429 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f35c806c8f0 0x7f35c806edb0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:22.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.598+0000 7f35eee00640 1 --2- 172.21.15.159:0/441058429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35e80d00d0 0x7f35e815c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:22.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.598+0000 7f35eee00640 1 --2- 172.21.15.159:0/441058429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f35e80cf1b0 0x7f35e8142920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:22.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.598+0000 7f35eee00640 1 -- 172.21.15.159:0/441058429 >> 172.21.15.159:0/441058429 conn(0x7f35e80ccad0 msgr2=0x7f35e80b5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:22.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.598+0000 7f35eee00640 1 -- 172.21.15.159:0/441058429 shutdown_connections 2024-04-04T06:10:22.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.598+0000 7f35eee00640 1 -- 172.21.15.159:0/441058429 wait complete. 2024-04-04T06:10:22.615 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:10:22.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.819+0000 7f7c3be00640 1 Processor -- start 2024-04-04T06:10:22.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.819+0000 7f7c3be00640 1 -- start start 2024-04-04T06:10:22.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.820+0000 7f7c3be00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7c3415c390 0x7f7c34073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:22.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.821+0000 7f7c3be00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c3415fe80 con 0x7f7c340af830 2024-04-04T06:10:22.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.821+0000 7f7c3be00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c340ad120 con 0x7f7c340d0650 2024-04-04T06:10:22.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.821+0000 7f7c3be00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7c3415c770 con 0x7f7c3415c390 2024-04-04T06:10:22.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.821+0000 7f7c38a00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f7c340af830 0x7f7c34160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45902/0 (socket says 172.21.15.159:45902) 2024-04-04T06:10:22.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.821+0000 7f7c38a00640 1 -- 172.21.15.159:0/2300387210 learned_addr learned my addr 172.21.15.159:0/2300387210 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:22.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.821+0000 7f7c33e00640 1 -- 172.21.15.159:0/2300387210 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 393362597 0 0) 0x7f7c3415fe80 con 0x7f7c340af830 2024-04-04T06:10:22.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.821+0000 7f7c33e00640 1 -- 172.21.15.159:0/2300387210 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c18003520 con 0x7f7c340af830 2024-04-04T06:10:22.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.821+0000 7f7c39400640 1 --2- 172.21.15.159:0/2300387210 >> v2:172.21.15.178:3300/0 conn(0x7f7c3415c390 0x7f7c34073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:22.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.822+0000 7f7c33e00640 1 -- 172.21.15.159:0/2300387210 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1684262258 0 0) 0x7f7c18003520 con 0x7f7c340af830 2024-04-04T06:10:22.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.822+0000 7f7c33e00640 1 -- 172.21.15.159:0/2300387210 >> v2:172.21.15.178:3300/0 conn(0x7f7c3415c390 msgr2=0x7f7c34073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:22.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.822+0000 7f7c33e00640 1 --2- 172.21.15.159:0/2300387210 >> v2:172.21.15.178:3300/0 conn(0x7f7c3415c390 0x7f7c34073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:22.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.822+0000 7f7c33e00640 1 -- 172.21.15.159:0/2300387210 >> v1:172.21.15.178:6789/0 conn(0x7f7c340d0650 legacy=0x7f7c34071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:22.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.822+0000 7f7c33e00640 1 -- 172.21.15.159:0/2300387210 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c340ad400 con 0x7f7c340af830 2024-04-04T06:10:22.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.822+0000 7f7c33e00640 1 -- 172.21.15.159:0/2300387210 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f7c20001670 con 0x7f7c340af830 2024-04-04T06:10:22.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.822+0000 7f7c33e00640 1 -- 172.21.15.159:0/2300387210 >> v1:172.21.15.159:6789/0 conn(0x7f7c340af830 legacy=0x7f7c34160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:22.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.822+0000 7f7c33e00640 1 --2- 172.21.15.159:0/2300387210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c18003a00 0x7f7c18003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:22.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.822+0000 7f7c33e00640 1 --2- 172.21.15.159:0/2300387210 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c18004800 0x7f7c18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:22.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.822+0000 7f7c33e00640 1 --2- 172.21.15.159:0/2300387210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c18005210 0x7f7c1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:22.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.822+0000 7f7c33e00640 1 -- 172.21.15.159:0/2300387210 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1800c110 con 0x7f7c18005210 2024-04-04T06:10:22.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.822+0000 7f7c33e00640 1 -- 172.21.15.159:0/2300387210 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1800c2c0 con 0x7f7c18003a00 2024-04-04T06:10:22.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.822+0000 7f7c33e00640 1 -- 172.21.15.159:0/2300387210 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7c1800c4c0 con 0x7f7c18004800 2024-04-04T06:10:22.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.822+0000 7f7c39e00640 1 --2- 172.21.15.159:0/2300387210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c18005210 0x7f7c1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:22.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.822+0000 7f7c39400640 1 --2- 172.21.15.159:0/2300387210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c18003a00 0x7f7c18003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:22.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.823+0000 7f7c38a00640 1 --2- 172.21.15.159:0/2300387210 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c18004800 0x7f7c18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:22.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.823+0000 7f7c39e00640 1 -- 172.21.15.159:0/2300387210 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c18004800 msgr2=0x7f7c18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:22.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.823+0000 7f7c39e00640 1 --2- 172.21.15.159:0/2300387210 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c18004800 0x7f7c18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:22.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.823+0000 7f7c39e00640 1 -- 172.21.15.159:0/2300387210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c18003a00 msgr2=0x7f7c18003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:22.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.823+0000 7f7c39e00640 1 --2- 172.21.15.159:0/2300387210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c18003a00 0x7f7c18003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:22.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.823+0000 7f7c39e00640 1 -- 172.21.15.159:0/2300387210 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c18003520 con 0x7f7c18005210 2024-04-04T06:10:22.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.824+0000 7f7c39e00640 1 -- 172.21.15.159:0/2300387210 reap_dead start 2024-04-04T06:10:22.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.824+0000 7f7c39e00640 1 --2- 172.21.15.159:0/2300387210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c18005210 0x7f7c1800b9c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c004480 tx=0x7f7c18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda233f4517cf87a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:22.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.824+0000 7f7c33e00640 1 -- 172.21.15.159:0/2300387210 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c034070 con 0x7f7c18005210 2024-04-04T06:10:22.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.824+0000 7f7c33e00640 1 -- 172.21.15.159:0/2300387210 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c2c006b50 con 0x7f7c18005210 2024-04-04T06:10:22.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.824+0000 7f7c33e00640 1 -- 172.21.15.159:0/2300387210 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c004080 con 0x7f7c18005210 2024-04-04T06:10:22.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.825+0000 7f7c3be00640 1 -- 172.21.15.159:0/2300387210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c18005210 msgr2=0x7f7c1800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:22.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.825+0000 7f7c3be00640 1 --2- 172.21.15.159:0/2300387210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c18005210 0x7f7c1800b9c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c004480 tx=0x7f7c18005830 comp rx=0 tx=0).stop 2024-04-04T06:10:22.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.825+0000 7f7c3be00640 1 -- 172.21.15.159:0/2300387210 shutdown_connections 2024-04-04T06:10:22.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.825+0000 7f7c3be00640 1 --2- 172.21.15.159:0/2300387210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c18005210 0x7f7c1800b9c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:22.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.825+0000 7f7c3be00640 1 -- 172.21.15.159:0/2300387210 >> 172.21.15.159:0/2300387210 conn(0x7f7c3406b990 msgr2=0x7f7c3406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:22.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.825+0000 7f7c3be00640 1 -- 172.21.15.159:0/2300387210 shutdown_connections 2024-04-04T06:10:22.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.825+0000 7f7c3be00640 1 -- 172.21.15.159:0/2300387210 wait complete. 2024-04-04T06:10:22.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.826+0000 7f7c3be00640 1 Processor -- start 2024-04-04T06:10:22.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.827+0000 7f7c3be00640 1 -- start start 2024-04-04T06:10:22.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.827+0000 7f7c3be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c18005210 0x7f7c3415e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:22.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.827+0000 7f7c3be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c34150050 0x7f7c3415c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:22.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.827+0000 7f7c3be00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c341504d0 0x7f7c340afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:22.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.827+0000 7f7c3be00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7c340cf980 con 0x7f7c34150050 2024-04-04T06:10:22.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.827+0000 7f7c3be00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34160120 con 0x7f7c18005210 2024-04-04T06:10:22.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.827+0000 7f7c3be00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7c34073790 con 0x7f7c341504d0 2024-04-04T06:10:22.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.828+0000 7f7c38a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c34150050 0x7f7c3415c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:22.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.828+0000 7f7c38a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c34150050 0x7f7c3415c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55568/0 (socket says 172.21.15.159:55568) 2024-04-04T06:10:22.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.828+0000 7f7c38a00640 1 -- 172.21.15.159:0/1446813471 learned_addr learned my addr 172.21.15.159:0/1446813471 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:22.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.828+0000 7f7c39400640 1 --2- 172.21.15.159:0/1446813471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c18005210 0x7f7c3415e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:22.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.828+0000 7f7c39e00640 1 --2- 172.21.15.159:0/1446813471 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c341504d0 0x7f7c340afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:22.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.828+0000 7f7c38a00640 1 -- 172.21.15.159:0/1446813471 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c341504d0 msgr2=0x7f7c340afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:22.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.828+0000 7f7c38a00640 1 --2- 172.21.15.159:0/1446813471 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c341504d0 0x7f7c340afd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:22.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.828+0000 7f7c38a00640 1 -- 172.21.15.159:0/1446813471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c18005210 msgr2=0x7f7c3415e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:22.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.828+0000 7f7c38a00640 1 --2- 172.21.15.159:0/1446813471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c18005210 0x7f7c3415e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:22.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.828+0000 7f7c38a00640 1 -- 172.21.15.159:0/1446813471 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3415d1f0 con 0x7f7c34150050 2024-04-04T06:10:22.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.829+0000 7f7c38a00640 1 --2- 172.21.15.159:0/1446813471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c34150050 0x7f7c3415c770 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000e700 tx=0x7f7c2000ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:22.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.829+0000 7f7c31600640 1 -- 172.21.15.159:0/1446813471 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2000c660 con 0x7f7c34150050 2024-04-04T06:10:22.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.829+0000 7f7c31600640 1 -- 172.21.15.159:0/1446813471 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c20018070 con 0x7f7c34150050 2024-04-04T06:10:22.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.829+0000 7f7c31600640 1 -- 172.21.15.159:0/1446813471 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2000ca60 con 0x7f7c34150050 2024-04-04T06:10:22.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.829+0000 7f7c3be00640 1 -- 172.21.15.159:0/1446813471 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c3415d4d0 con 0x7f7c34150050 2024-04-04T06:10:22.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.829+0000 7f7c3be00640 1 -- 172.21.15.159:0/1446813471 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c3415da90 con 0x7f7c34150050 2024-04-04T06:10:22.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.830+0000 7f7c3be00640 1 -- 172.21.15.159:0/1446813471 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c340685c0 con 0x7f7c34150050 2024-04-04T06:10:22.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.831+0000 7f7c31600640 1 -- 172.21.15.159:0/1446813471 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c2000cc00 con 0x7f7c34150050 2024-04-04T06:10:22.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.832+0000 7f7c31600640 1 --2- 172.21.15.159:0/1446813471 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7c1406c8f0 0x7f7c1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:22.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.833+0000 7f7c31600640 1 -- 172.21.15.159:0/1446813471 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f7c2008ba20 con 0x7f7c34150050 2024-04-04T06:10:22.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.833+0000 7f7c39400640 1 --2- 172.21.15.159:0/1446813471 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7c1406c8f0 0x7f7c1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:22.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.834+0000 7f7c39400640 1 --2- 172.21.15.159:0/1446813471 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7c1406c8f0 0x7f7c1406edb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7c24006fd0 tx=0x7f7c24008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:22.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:22.837+0000 7f7c31600640 1 -- 172.21.15.159:0/1446813471 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7c20059db0 con 0x7f7c34150050 2024-04-04T06:10:23.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.040+0000 7f7c3be00640 1 -- 172.21.15.159:0/1446813471 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c34072920 con 0x7f7c34150050 2024-04-04T06:10:23.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.042+0000 7f7c31600640 1 -- 172.21.15.159:0/1446813471 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16677 (secure 0 0 0) 0x7f7c200596b0 con 0x7f7c34150050 2024-04-04T06:10:23.035 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:23.036 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":59,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:08:58.581221+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:08:54.848337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-04T06:08:56.003356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:10:23.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.045+0000 7f7c3be00640 1 -- 172.21.15.159:0/1446813471 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7c1406c8f0 msgr2=0x7f7c1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:23.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.045+0000 7f7c3be00640 1 --2- 172.21.15.159:0/1446813471 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7c1406c8f0 0x7f7c1406edb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7c24006fd0 tx=0x7f7c24008040 comp rx=0 tx=0).stop 2024-04-04T06:10:23.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.045+0000 7f7c3be00640 1 -- 172.21.15.159:0/1446813471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c34150050 msgr2=0x7f7c3415c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:23.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.045+0000 7f7c3be00640 1 --2- 172.21.15.159:0/1446813471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c34150050 0x7f7c3415c770 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000e700 tx=0x7f7c2000ebc0 comp rx=0 tx=0).stop 2024-04-04T06:10:23.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.045+0000 7f7c3be00640 1 -- 172.21.15.159:0/1446813471 shutdown_connections 2024-04-04T06:10:23.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.045+0000 7f7c3be00640 1 --2- 172.21.15.159:0/1446813471 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c341504d0 0x7f7c340afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:23.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.045+0000 7f7c3be00640 1 --2- 172.21.15.159:0/1446813471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c34150050 0x7f7c3415c770 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:23.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.045+0000 7f7c3be00640 1 --2- 172.21.15.159:0/1446813471 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7c1406c8f0 0x7f7c1406edb0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:23.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.045+0000 7f7c3be00640 1 --2- 172.21.15.159:0/1446813471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c18005210 0x7f7c3415e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:23.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.045+0000 7f7c3be00640 1 -- 172.21.15.159:0/1446813471 >> 172.21.15.159:0/1446813471 conn(0x7f7c3406b990 msgr2=0x7f7c34071cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:23.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.045+0000 7f7c3be00640 1 -- 172.21.15.159:0/1446813471 shutdown_connections 2024-04-04T06:10:23.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.045+0000 7f7c3be00640 1 -- 172.21.15.159:0/1446813471 wait complete. 2024-04-04T06:10:23.057 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-04T06:10:23.057 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:10:23.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.249+0000 7f3fec000640 1 Processor -- start 2024-04-04T06:10:23.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.249+0000 7f3fec000640 1 -- start start 2024-04-04T06:10:23.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.252+0000 7f3fec000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3fe40d4840 0x7f3fe40cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:23.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.252+0000 7f3fec000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3fe40d0f20 0x7f3fe40d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:23.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.252+0000 7f3fec000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fe40d1c90 con 0x7f3fe415c2c0 2024-04-04T06:10:23.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.252+0000 7f3fec000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f3fe40cf160 con 0x7f3fe40d0f20 2024-04-04T06:10:23.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.252+0000 7f3fec000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3fe40cf440 con 0x7f3fe40d4840 2024-04-04T06:10:23.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.252+0000 7f3fe9600640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f3fe415c2c0 0x7f3fe40d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45914/0 (socket says 172.21.15.159:45914) 2024-04-04T06:10:23.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.252+0000 7f3fe9600640 1 -- 172.21.15.159:0/884252773 learned_addr learned my addr 172.21.15.159:0/884252773 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:23.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.252+0000 7f3fea000640 1 --2- 172.21.15.159:0/884252773 >> v2:172.21.15.159:3300/0 conn(0x7f3fe40d0f20 0x7f3fe40d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:23.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.252+0000 7f3fe8c00640 1 --2- 172.21.15.159:0/884252773 >> v2:172.21.15.178:3300/0 conn(0x7f3fe40d4840 0x7f3fe40cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:23.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.252+0000 7f3fea000640 1 -- 172.21.15.159:0/884252773 >> v2:172.21.15.178:3300/0 conn(0x7f3fe40d4840 msgr2=0x7f3fe40cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:23.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.252+0000 7f3fea000640 1 --2- 172.21.15.159:0/884252773 >> v2:172.21.15.178:3300/0 conn(0x7f3fe40d4840 0x7f3fe40cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:23.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.252+0000 7f3fea000640 1 -- 172.21.15.159:0/884252773 >> v1:172.21.15.159:6789/0 conn(0x7f3fe415c2c0 legacy=0x7f3fe40d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:23.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.253+0000 7f3fea000640 1 -- 172.21.15.159:0/884252773 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe40cf720 con 0x7f3fe40d0f20 2024-04-04T06:10:23.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.254+0000 7f3fea000640 1 --2- 172.21.15.159:0/884252773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fe40d0f20 0x7f3fe40d1370 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f3fe40d1f70 tx=0x7f3fdc02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd47a0f0de6891 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:23.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.254+0000 7f3fe3e00640 1 -- 172.21.15.159:0/884252773 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc039070 con 0x7f3fe40d0f20 2024-04-04T06:10:23.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.254+0000 7f3fe3e00640 1 -- 172.21.15.159:0/884252773 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fdc02cc70 con 0x7f3fe40d0f20 2024-04-04T06:10:23.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.254+0000 7f3fe3e00640 1 -- 172.21.15.159:0/884252773 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc034660 con 0x7f3fe40d0f20 2024-04-04T06:10:23.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.255+0000 7f3fec000640 1 -- 172.21.15.159:0/884252773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fe40d0f20 msgr2=0x7f3fe40d1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:23.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.255+0000 7f3fec000640 1 --2- 172.21.15.159:0/884252773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fe40d0f20 0x7f3fe40d1370 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f3fe40d1f70 tx=0x7f3fdc02bec0 comp rx=0 tx=0).stop 2024-04-04T06:10:23.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.255+0000 7f3fec000640 1 -- 172.21.15.159:0/884252773 shutdown_connections 2024-04-04T06:10:23.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.255+0000 7f3fec000640 1 --2- 172.21.15.159:0/884252773 >> v2:172.21.15.178:3300/0 conn(0x7f3fe40d4840 0x7f3fe40cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:23.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.255+0000 7f3fec000640 1 --2- 172.21.15.159:0/884252773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fe40d0f20 0x7f3fe40d1370 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:23.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.255+0000 7f3fec000640 1 -- 172.21.15.159:0/884252773 >> 172.21.15.159:0/884252773 conn(0x7f3fe408bfb0 msgr2=0x7f3fe409d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:23.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.255+0000 7f3fec000640 1 -- 172.21.15.159:0/884252773 shutdown_connections 2024-04-04T06:10:23.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.255+0000 7f3fec000640 1 -- 172.21.15.159:0/884252773 wait complete. 2024-04-04T06:10:23.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.257+0000 7f3fec000640 1 Processor -- start 2024-04-04T06:10:23.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.257+0000 7f3fec000640 1 -- start start 2024-04-04T06:10:23.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.258+0000 7f3fec000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe40d0f20 0x7f3fe41593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:23.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.258+0000 7f3fec000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fe40d4840 0x7f3fe40b98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:23.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.258+0000 7f3fec000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3fe415c2c0 0x7f3fe4158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:23.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.258+0000 7f3fec000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe40d1a30 con 0x7f3fe40d4840 2024-04-04T06:10:23.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.258+0000 7f3fec000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe40d18b0 con 0x7f3fe40d0f20 2024-04-04T06:10:23.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.258+0000 7f3fec000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3fe40d1ff0 con 0x7f3fe415c2c0 2024-04-04T06:10:23.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.259+0000 7f3fe8c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fe40d4840 0x7f3fe40b98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:23.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.259+0000 7f3fe8c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fe40d4840 0x7f3fe40b98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55592/0 (socket says 172.21.15.159:55592) 2024-04-04T06:10:23.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.259+0000 7f3fe8c00640 1 -- 172.21.15.159:0/381609167 learned_addr learned my addr 172.21.15.159:0/381609167 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:23.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.259+0000 7f3fe9600640 1 --2- 172.21.15.159:0/381609167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe40d0f20 0x7f3fe41593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:23.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.259+0000 7f3fea000640 1 --2- 172.21.15.159:0/381609167 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3fe415c2c0 0x7f3fe4158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:23.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.259+0000 7f3fe8c00640 1 -- 172.21.15.159:0/381609167 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3fe415c2c0 msgr2=0x7f3fe4158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:23.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.259+0000 7f3fe8c00640 1 --2- 172.21.15.159:0/381609167 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3fe415c2c0 0x7f3fe4158c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:23.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.259+0000 7f3fe8c00640 1 -- 172.21.15.159:0/381609167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe40d0f20 msgr2=0x7f3fe41593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:23.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.259+0000 7f3fe8c00640 1 --2- 172.21.15.159:0/381609167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe40d0f20 0x7f3fe41593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:23.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.259+0000 7f3fe8c00640 1 -- 172.21.15.159:0/381609167 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe40b9e00 con 0x7f3fe40d4840 2024-04-04T06:10:23.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.260+0000 7f3fe8c00640 1 --2- 172.21.15.159:0/381609167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fe40d4840 0x7f3fe40b98c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000c820 tx=0x7f3fd000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:23.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.260+0000 7f3fe1600640 1 -- 172.21.15.159:0/381609167 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0015070 con 0x7f3fe40d4840 2024-04-04T06:10:23.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.260+0000 7f3fe1600640 1 -- 172.21.15.159:0/381609167 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fd0004220 con 0x7f3fe40d4840 2024-04-04T06:10:23.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.260+0000 7f3fec000640 1 -- 172.21.15.159:0/381609167 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe4159160 con 0x7f3fe40d4840 2024-04-04T06:10:23.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.260+0000 7f3fe1600640 1 -- 172.21.15.159:0/381609167 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd000ce10 con 0x7f3fe40d4840 2024-04-04T06:10:23.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.260+0000 7f3fec000640 1 -- 172.21.15.159:0/381609167 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe40cc5a0 con 0x7f3fe40d4840 2024-04-04T06:10:23.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.261+0000 7f3fec000640 1 -- 172.21.15.159:0/381609167 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe40685c0 con 0x7f3fe40d4840 2024-04-04T06:10:23.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.262+0000 7f3fe1600640 1 -- 172.21.15.159:0/381609167 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3fd0007830 con 0x7f3fe40d4840 2024-04-04T06:10:23.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.264+0000 7f3fe1600640 1 --2- 172.21.15.159:0/381609167 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3fc406c8f0 0x7f3fc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:23.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.264+0000 7f3fe1600640 1 -- 172.21.15.159:0/381609167 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f3fd008bd70 con 0x7f3fe40d4840 2024-04-04T06:10:23.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.264+0000 7f3fe9600640 1 --2- 172.21.15.159:0/381609167 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3fc406c8f0 0x7f3fc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:23.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.265+0000 7f3fe9600640 1 --2- 172.21.15.159:0/381609167 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3fc406c8f0 0x7f3fc406edb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4159e00 tx=0x7f3fd8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:23.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.268+0000 7f3fe1600640 1 -- 172.21.15.159:0/381609167 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3fd005a1a0 con 0x7f3fe40d4840 2024-04-04T06:10:23.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.587+0000 7f3fec000640 1 -- 172.21.15.159:0/381609167 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fe406c7c0 con 0x7f3fe40d4840 2024-04-04T06:10:23.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.588+0000 7f3fe1600640 1 -- 172.21.15.159:0/381609167 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5188 (secure 0 0 0) 0x7f3fd0059aa0 con 0x7f3fe40d4840 2024-04-04T06:10:23.582 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:23.582 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5191,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3507245876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3507245876},{"type":"v1","addr":"172.21.15.159:6835","nonce":3507245876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/967262871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":967262871},{"type":"v1","addr":"172.21.15.178:6835","nonce":967262871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2969351496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2969351496},{"type":"v1","addr":"172.21.15.178:6837","nonce":2969351496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:08:57.020444+0000","modified":"2024-04-04T06:09:49.628589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6039":{"gid":6039,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":16,"addr":"172.21.15.159:6834/962475038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":962475038},{"type":"v1","addr":"172.21.15.159:6834","nonce":962475038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6039,"qdb_cluster":[6039]},"id":4}]} 2024-04-04T06:10:23.582 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 35 2024-04-04T06:10:23.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.591+0000 7f3fec000640 1 -- 172.21.15.159:0/381609167 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3fc406c8f0 msgr2=0x7f3fc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:23.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.591+0000 7f3fec000640 1 --2- 172.21.15.159:0/381609167 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3fc406c8f0 0x7f3fc406edb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4159e00 tx=0x7f3fd8009040 comp rx=0 tx=0).stop 2024-04-04T06:10:23.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.591+0000 7f3fec000640 1 -- 172.21.15.159:0/381609167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fe40d4840 msgr2=0x7f3fe40b98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:23.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.591+0000 7f3fec000640 1 --2- 172.21.15.159:0/381609167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fe40d4840 0x7f3fe40b98c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000c820 tx=0x7f3fd000cce0 comp rx=0 tx=0).stop 2024-04-04T06:10:23.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.591+0000 7f3fec000640 1 -- 172.21.15.159:0/381609167 shutdown_connections 2024-04-04T06:10:23.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.591+0000 7f3fec000640 1 --2- 172.21.15.159:0/381609167 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3fe415c2c0 0x7f3fe4158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:23.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.591+0000 7f3fec000640 1 --2- 172.21.15.159:0/381609167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fe40d4840 0x7f3fe40b98c0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:23.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.591+0000 7f3fec000640 1 --2- 172.21.15.159:0/381609167 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3fc406c8f0 0x7f3fc406edb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:23.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.591+0000 7f3fec000640 1 --2- 172.21.15.159:0/381609167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe40d0f20 0x7f3fe41593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:23.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.591+0000 7f3fec000640 1 -- 172.21.15.159:0/381609167 >> 172.21.15.159:0/381609167 conn(0x7f3fe408bfb0 msgr2=0x7f3fe40d23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:23.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.591+0000 7f3fec000640 1 -- 172.21.15.159:0/381609167 shutdown_connections 2024-04-04T06:10:23.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.591+0000 7f3fec000640 1 -- 172.21.15.159:0/381609167 wait complete. 2024-04-04T06:10:23.604 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:10:23.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.807+0000 7fd564200640 1 Processor -- start 2024-04-04T06:10:23.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.807+0000 7fd564200640 1 -- start start 2024-04-04T06:10:23.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.808+0000 7fd564200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd55c15e700 0x7fd55c0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:23.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.808+0000 7fd564200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd55c15eae0 con 0x7fd55c0d1cf0 2024-04-04T06:10:23.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.808+0000 7fd564200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd55c071e00 con 0x7fd55c0d1610 2024-04-04T06:10:23.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.808+0000 7fd564200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fd55c0cfeb0 con 0x7fd55c15e700 2024-04-04T06:10:23.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.808+0000 7fd561800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd55c15e700 0x7fd55c0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:23.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.809+0000 7fd562200640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fd55c0d1cf0 0x7fd55c0716f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45926/0 (socket says 172.21.15.159:45926) 2024-04-04T06:10:23.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.809+0000 7fd562200640 1 -- 172.21.15.159:0/899276886 learned_addr learned my addr 172.21.15.159:0/899276886 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:23.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.809+0000 7fd561800640 1 -- 172.21.15.159:0/899276886 >> v1:172.21.15.178:6789/0 conn(0x7fd55c0d1610 legacy=0x7fd55c070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:23.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.809+0000 7fd561800640 1 -- 172.21.15.159:0/899276886 >> v1:172.21.15.159:6789/0 conn(0x7fd55c0d1cf0 legacy=0x7fd55c0716f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:23.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.809+0000 7fd561800640 1 -- 172.21.15.159:0/899276886 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55c0d0190 con 0x7fd55c15e700 2024-04-04T06:10:23.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.809+0000 7fd55be00640 1 -- 172.21.15.159:0/899276886 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4274018886 0 0) 0x7fd55c15eae0 con 0x7fd55c0d1cf0 2024-04-04T06:10:23.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.811+0000 7fd561800640 1 --2- 172.21.15.159:0/899276886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd55c15e700 0x7fd55c0b3e20 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd5500097a0 tx=0x7fd55002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf078f35967ad8c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:23.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.811+0000 7fd55be00640 1 -- 172.21.15.159:0/899276886 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5500315a0 con 0x7fd55c15e700 2024-04-04T06:10:23.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.811+0000 7fd55be00640 1 -- 172.21.15.159:0/899276886 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd550037070 con 0x7fd55c15e700 2024-04-04T06:10:23.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.811+0000 7fd55be00640 1 -- 172.21.15.159:0/899276886 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550031960 con 0x7fd55c15e700 2024-04-04T06:10:23.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.811+0000 7fd564200640 1 -- 172.21.15.159:0/899276886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd55c15e700 msgr2=0x7fd55c0b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:23.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.811+0000 7fd564200640 1 --2- 172.21.15.159:0/899276886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd55c15e700 0x7fd55c0b3e20 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd5500097a0 tx=0x7fd55002b840 comp rx=0 tx=0).stop 2024-04-04T06:10:23.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.812+0000 7fd564200640 1 -- 172.21.15.159:0/899276886 shutdown_connections 2024-04-04T06:10:23.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.812+0000 7fd564200640 1 --2- 172.21.15.159:0/899276886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd55c15e700 0x7fd55c0b3e20 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:23.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.812+0000 7fd564200640 1 -- 172.21.15.159:0/899276886 >> 172.21.15.159:0/899276886 conn(0x7fd55c0d2240 msgr2=0x7fd55c0d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:23.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.812+0000 7fd564200640 1 -- 172.21.15.159:0/899276886 shutdown_connections 2024-04-04T06:10:23.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.812+0000 7fd564200640 1 -- 172.21.15.159:0/899276886 wait complete. 2024-04-04T06:10:23.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.814+0000 7fd564200640 1 Processor -- start 2024-04-04T06:10:23.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.815+0000 7fd564200640 1 -- start start 2024-04-04T06:10:23.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.816+0000 7fd564200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd55c0d1610 0x7fd55c152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:23.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.816+0000 7fd564200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd55c0d1cf0 0x7fd55c148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:23.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.816+0000 7fd564200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd55c15e700 0x7fd55c155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:23.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.816+0000 7fd564200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c0d0a30 con 0x7fd55c0d1cf0 2024-04-04T06:10:23.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.816+0000 7fd564200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c0d0750 con 0x7fd55c15e700 2024-04-04T06:10:23.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.816+0000 7fd564200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd55c06d1b0 con 0x7fd55c0d1610 2024-04-04T06:10:23.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.816+0000 7fd560e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd55c0d1cf0 0x7fd55c148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:23.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.816+0000 7fd560e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd55c0d1cf0 0x7fd55c148ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55598/0 (socket says 172.21.15.159:55598) 2024-04-04T06:10:23.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.816+0000 7fd560e00640 1 -- 172.21.15.159:0/3553965687 learned_addr learned my addr 172.21.15.159:0/3553965687 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:23.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.817+0000 7fd561800640 1 --2- 172.21.15.159:0/3553965687 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd55c0d1610 0x7fd55c152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:23.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.817+0000 7fd560e00640 1 -- 172.21.15.159:0/3553965687 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd55c0d1610 msgr2=0x7fd55c152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:23.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.817+0000 7fd560e00640 1 --2- 172.21.15.159:0/3553965687 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd55c0d1610 0x7fd55c152020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:23.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.817+0000 7fd562200640 1 --2- 172.21.15.159:0/3553965687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd55c15e700 0x7fd55c155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:23.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.817+0000 7fd560e00640 1 -- 172.21.15.159:0/3553965687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd55c15e700 msgr2=0x7fd55c155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:23.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.817+0000 7fd560e00640 1 --2- 172.21.15.159:0/3553965687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd55c15e700 0x7fd55c155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:23.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.817+0000 7fd560e00640 1 -- 172.21.15.159:0/3553965687 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55c152be0 con 0x7fd55c0d1cf0 2024-04-04T06:10:23.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.817+0000 7fd562200640 1 --2- 172.21.15.159:0/3553965687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd55c15e700 0x7fd55c155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:23.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.817+0000 7fd560e00640 1 --2- 172.21.15.159:0/3553965687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd55c0d1cf0 0x7fd55c148ff0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd54800b3f0 tx=0x7fd54800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:23.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.817+0000 7fd559600640 1 -- 172.21.15.159:0/3553965687 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548015020 con 0x7fd55c0d1cf0 2024-04-04T06:10:23.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.817+0000 7fd559600640 1 -- 172.21.15.159:0/3553965687 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd548011830 con 0x7fd55c0d1cf0 2024-04-04T06:10:23.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.817+0000 7fd564200640 1 -- 172.21.15.159:0/3553965687 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd55c1464c0 con 0x7fd55c0d1cf0 2024-04-04T06:10:23.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.817+0000 7fd559600640 1 -- 172.21.15.159:0/3553965687 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548011c20 con 0x7fd55c0d1cf0 2024-04-04T06:10:23.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.818+0000 7fd564200640 1 -- 172.21.15.159:0/3553965687 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd55c155830 con 0x7fd55c0d1cf0 2024-04-04T06:10:23.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.819+0000 7fd564200640 1 -- 172.21.15.159:0/3553965687 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd55c0685c0 con 0x7fd55c0d1cf0 2024-04-04T06:10:23.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.820+0000 7fd559600640 1 -- 172.21.15.159:0/3553965687 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd548018020 con 0x7fd55c0d1cf0 2024-04-04T06:10:23.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.820+0000 7fd559600640 1 --2- 172.21.15.159:0/3553965687 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd53c06c8f0 0x7fd53c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:23.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.821+0000 7fd559600640 1 -- 172.21.15.159:0/3553965687 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fd54808b7a0 con 0x7fd55c0d1cf0 2024-04-04T06:10:23.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.821+0000 7fd561800640 1 --2- 172.21.15.159:0/3553965687 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd53c06c8f0 0x7fd53c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:23.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.822+0000 7fd561800640 1 --2- 172.21.15.159:0/3553965687 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd53c06c8f0 0x7fd53c06edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd550036000 tx=0x7fd550008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:23.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:23.825+0000 7fd559600640 1 -- 172.21.15.159:0/3553965687 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd54801f030 con 0x7fd55c0d1cf0 2024-04-04T06:10:24.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.252+0000 7fd564200640 1 -- 172.21.15.159:0/3553965687 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd55c0c80f0 con 0x7fd55c0d1cf0 2024-04-04T06:10:24.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.253+0000 7fd559600640 1 -- 172.21.15.159:0/3553965687 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5188 (secure 0 0 0) 0x7fd5480172c0 con 0x7fd55c0d1cf0 2024-04-04T06:10:24.246 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:24.247 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5191,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3507245876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3507245876},{"type":"v1","addr":"172.21.15.159:6835","nonce":3507245876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/967262871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":967262871},{"type":"v1","addr":"172.21.15.178:6835","nonce":967262871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2969351496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2969351496},{"type":"v1","addr":"172.21.15.178:6837","nonce":2969351496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:08:57.020444+0000","modified":"2024-04-04T06:09:49.628589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6039},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6039":{"gid":6039,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":16,"addr":"172.21.15.159:6834/962475038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":962475038},{"type":"v1","addr":"172.21.15.159:6834","nonce":962475038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6039,"qdb_cluster":[6039]},"id":4}]} 2024-04-04T06:10:24.247 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 35 2024-04-04T06:10:24.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.255+0000 7fd564200640 1 -- 172.21.15.159:0/3553965687 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd53c06c8f0 msgr2=0x7fd53c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:24.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.255+0000 7fd564200640 1 --2- 172.21.15.159:0/3553965687 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd53c06c8f0 0x7fd53c06edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd550036000 tx=0x7fd550008030 comp rx=0 tx=0).stop 2024-04-04T06:10:24.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.255+0000 7fd564200640 1 -- 172.21.15.159:0/3553965687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd55c0d1cf0 msgr2=0x7fd55c148ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:24.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.255+0000 7fd564200640 1 --2- 172.21.15.159:0/3553965687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd55c0d1cf0 0x7fd55c148ff0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd54800b3f0 tx=0x7fd54800b8b0 comp rx=0 tx=0).stop 2024-04-04T06:10:24.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.255+0000 7fd564200640 1 -- 172.21.15.159:0/3553965687 shutdown_connections 2024-04-04T06:10:24.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.255+0000 7fd564200640 1 --2- 172.21.15.159:0/3553965687 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd53c06c8f0 0x7fd53c06edb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:24.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.255+0000 7fd564200640 1 --2- 172.21.15.159:0/3553965687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd55c15e700 0x7fd55c155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:24.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.255+0000 7fd564200640 1 --2- 172.21.15.159:0/3553965687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd55c0d1cf0 0x7fd55c148ff0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:24.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.255+0000 7fd564200640 1 --2- 172.21.15.159:0/3553965687 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd55c0d1610 0x7fd55c152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:24.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.256+0000 7fd564200640 1 -- 172.21.15.159:0/3553965687 >> 172.21.15.159:0/3553965687 conn(0x7fd55c0d2240 msgr2=0x7fd55c0c7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:24.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.256+0000 7fd564200640 1 -- 172.21.15.159:0/3553965687 shutdown_connections 2024-04-04T06:10:24.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.256+0000 7fd564200640 1 -- 172.21.15.159:0/3553965687 wait complete. 2024-04-04T06:10:24.267 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:10:24.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.472+0000 7f112ec00640 1 Processor -- start 2024-04-04T06:10:24.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.472+0000 7f112ec00640 1 -- start start 2024-04-04T06:10:24.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.474+0000 7f112ec00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f112806bc80 0x7f11280cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:24.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.474+0000 7f112ec00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11280ce1c0 con 0x7f112815c390 2024-04-04T06:10:24.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.474+0000 7f112ec00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11280ce4a0 con 0x7f11280cdaf0 2024-04-04T06:10:24.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.474+0000 7f112ec00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f11280cdfa0 con 0x7f112806bc80 2024-04-04T06:10:24.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.474+0000 7f112d800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f112815c390 0x7f112806b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43970/0 (socket says 172.21.15.159:43970) 2024-04-04T06:10:24.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.474+0000 7f112d800640 1 -- 172.21.15.159:0/2993866675 learned_addr learned my addr 172.21.15.159:0/2993866675 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:24.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.475+0000 7f1127e00640 1 -- 172.21.15.159:0/2993866675 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 351350581 0 0) 0x7f11280ce1c0 con 0x7f112815c390 2024-04-04T06:10:24.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.475+0000 7f1127e00640 1 -- 172.21.15.159:0/2993866675 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1110003520 con 0x7f112815c390 2024-04-04T06:10:24.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.475+0000 7f112ce00640 1 --2- 172.21.15.159:0/2993866675 >> v2:172.21.15.178:3300/0 conn(0x7f112806bc80 0x7f11280cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:24.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.475+0000 7f1127e00640 1 -- 172.21.15.159:0/2993866675 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1285648826 0 0) 0x7f1110003520 con 0x7f112815c390 2024-04-04T06:10:24.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.475+0000 7f1127e00640 1 -- 172.21.15.159:0/2993866675 >> v2:172.21.15.178:3300/0 conn(0x7f112806bc80 msgr2=0x7f11280cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:24.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.475+0000 7f1127e00640 1 --2- 172.21.15.159:0/2993866675 >> v2:172.21.15.178:3300/0 conn(0x7f112806bc80 0x7f11280cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:24.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.475+0000 7f1127e00640 1 -- 172.21.15.159:0/2993866675 >> v1:172.21.15.178:6789/0 conn(0x7f11280cdaf0 legacy=0x7f11280cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:24.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.475+0000 7f1127e00640 1 -- 172.21.15.159:0/2993866675 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11280ce780 con 0x7f112815c390 2024-04-04T06:10:24.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.475+0000 7f1127e00640 1 -- 172.21.15.159:0/2993866675 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f111c001670 con 0x7f112815c390 2024-04-04T06:10:24.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.475+0000 7f1127e00640 1 -- 172.21.15.159:0/2993866675 >> v1:172.21.15.159:6789/0 conn(0x7f112815c390 legacy=0x7f112806b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:24.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.475+0000 7f1127e00640 1 --2- 172.21.15.159:0/2993866675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11100039b0 0x7f1110003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:24.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.475+0000 7f1127e00640 1 --2- 172.21.15.159:0/2993866675 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1110004800 0x7f1110004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:24.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.475+0000 7f1127e00640 1 --2- 172.21.15.159:0/2993866675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11100051c0 0x7f111000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:24.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.475+0000 7f1127e00640 1 -- 172.21.15.159:0/2993866675 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f11280cdfa0 con 0x7f11100051c0 2024-04-04T06:10:24.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.475+0000 7f1127e00640 1 -- 172.21.15.159:0/2993866675 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f111000c110 con 0x7f11100039b0 2024-04-04T06:10:24.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.475+0000 7f1127e00640 1 -- 172.21.15.159:0/2993866675 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f111000c310 con 0x7f1110004800 2024-04-04T06:10:24.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.476+0000 7f112e200640 1 --2- 172.21.15.159:0/2993866675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11100051c0 0x7f111000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:24.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.476+0000 7f112d800640 1 --2- 172.21.15.159:0/2993866675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11100039b0 0x7f1110003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:24.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.476+0000 7f112ce00640 1 --2- 172.21.15.159:0/2993866675 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1110004800 0x7f1110004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:24.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.476+0000 7f112e200640 1 -- 172.21.15.159:0/2993866675 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1110004800 msgr2=0x7f1110004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:24.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.476+0000 7f112e200640 1 --2- 172.21.15.159:0/2993866675 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1110004800 0x7f1110004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:24.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.476+0000 7f112e200640 1 -- 172.21.15.159:0/2993866675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11100039b0 msgr2=0x7f1110003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:24.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.476+0000 7f112e200640 1 --2- 172.21.15.159:0/2993866675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11100039b0 0x7f1110003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:24.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.476+0000 7f112e200640 1 -- 172.21.15.159:0/2993866675 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1110003520 con 0x7f11100051c0 2024-04-04T06:10:24.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.476+0000 7f112ce00640 1 --2- 172.21.15.159:0/2993866675 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1110004800 0x7f1110004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:24.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.477+0000 7f112e200640 1 -- 172.21.15.159:0/2993866675 reap_dead start 2024-04-04T06:10:24.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.477+0000 7f112e200640 1 --2- 172.21.15.159:0/2993866675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11100051c0 0x7f111000b9c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1120001e50 tx=0x7f1110005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03a8e6367ccd538 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:24.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.477+0000 7f1127e00640 1 -- 172.21.15.159:0/2993866675 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1120001a70 con 0x7f11100051c0 2024-04-04T06:10:24.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.477+0000 7f1127e00640 1 -- 172.21.15.159:0/2993866675 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1120003bb0 con 0x7f11100051c0 2024-04-04T06:10:24.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.477+0000 7f1127e00640 1 -- 172.21.15.159:0/2993866675 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11200050d0 con 0x7f11100051c0 2024-04-04T06:10:24.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.478+0000 7f112ec00640 1 -- 172.21.15.159:0/2993866675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11100051c0 msgr2=0x7f111000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:24.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.478+0000 7f112ec00640 1 --2- 172.21.15.159:0/2993866675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11100051c0 0x7f111000b9c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1120001e50 tx=0x7f1110005830 comp rx=0 tx=0).stop 2024-04-04T06:10:24.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.478+0000 7f112ec00640 1 -- 172.21.15.159:0/2993866675 shutdown_connections 2024-04-04T06:10:24.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.478+0000 7f112ec00640 1 --2- 172.21.15.159:0/2993866675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11100051c0 0x7f111000b9c0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:24.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.478+0000 7f112ec00640 1 -- 172.21.15.159:0/2993866675 >> 172.21.15.159:0/2993866675 conn(0x7f112809aa20 msgr2=0x7f11280b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:24.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.478+0000 7f112ec00640 1 -- 172.21.15.159:0/2993866675 shutdown_connections 2024-04-04T06:10:24.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.478+0000 7f112ec00640 1 -- 172.21.15.159:0/2993866675 wait complete. 2024-04-04T06:10:24.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.480+0000 7f112ec00640 1 Processor -- start 2024-04-04T06:10:24.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.480+0000 7f112ec00640 1 -- start start 2024-04-04T06:10:24.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.481+0000 7f112ec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11100051c0 0x7f11280c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:24.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.481+0000 7f112ec00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11281571d0 0x7f1128156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:24.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.481+0000 7f112ec00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11280af1a0 0x7f11280af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:24.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.481+0000 7f112ec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f11280cd940 con 0x7f11100051c0 2024-04-04T06:10:24.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.481+0000 7f112ec00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f11280cd7c0 con 0x7f11281571d0 2024-04-04T06:10:24.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.481+0000 7f112ec00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f11280cea60 con 0x7f11280af1a0 2024-04-04T06:10:24.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.481+0000 7f112d800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11100051c0 0x7f11280c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:24.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.481+0000 7f112d800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11100051c0 0x7f11280c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41852/0 (socket says 172.21.15.159:41852) 2024-04-04T06:10:24.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.481+0000 7f112d800640 1 -- 172.21.15.159:0/1631372598 learned_addr learned my addr 172.21.15.159:0/1631372598 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:24.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.482+0000 7f112d800640 1 -- 172.21.15.159:0/1631372598 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11280af1a0 msgr2=0x7f11280af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:24.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.482+0000 7f112d800640 1 --2- 172.21.15.159:0/1631372598 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11280af1a0 0x7f11280af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:24.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.482+0000 7f112d800640 1 -- 172.21.15.159:0/1631372598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11281571d0 msgr2=0x7f1128156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:24.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.482+0000 7f112d800640 1 --2- 172.21.15.159:0/1631372598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11281571d0 0x7f1128156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:24.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.482+0000 7f112d800640 1 -- 172.21.15.159:0/1631372598 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11280c7ba0 con 0x7f11100051c0 2024-04-04T06:10:24.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.482+0000 7f112d800640 1 --2- 172.21.15.159:0/1631372598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11100051c0 0x7f11280c7660 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f111c00e7b0 tx=0x7f111c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:24.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.482+0000 7f1125600640 1 -- 172.21.15.159:0/1631372598 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c00ee60 con 0x7f11100051c0 2024-04-04T06:10:24.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.482+0000 7f1125600640 1 -- 172.21.15.159:0/1631372598 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f111c018070 con 0x7f11100051c0 2024-04-04T06:10:24.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.482+0000 7f1125600640 1 -- 172.21.15.159:0/1631372598 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c00cab0 con 0x7f11100051c0 2024-04-04T06:10:24.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.482+0000 7f112ec00640 1 -- 172.21.15.159:0/1631372598 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11280b3ed0 con 0x7f11100051c0 2024-04-04T06:10:24.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.482+0000 7f112ec00640 1 -- 172.21.15.159:0/1631372598 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11280b4490 con 0x7f11100051c0 2024-04-04T06:10:24.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.483+0000 7f112ec00640 1 -- 172.21.15.159:0/1631372598 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1128075940 con 0x7f11100051c0 2024-04-04T06:10:24.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.484+0000 7f1125600640 1 -- 172.21.15.159:0/1631372598 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f111c00cc50 con 0x7f11100051c0 2024-04-04T06:10:24.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.485+0000 7f1125600640 1 --2- 172.21.15.159:0/1631372598 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f110806c920 0x7f110806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:24.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.485+0000 7f1125600640 1 -- 172.21.15.159:0/1631372598 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f111c08b950 con 0x7f11100051c0 2024-04-04T06:10:24.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.486+0000 7f112ce00640 1 --2- 172.21.15.159:0/1631372598 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f110806c920 0x7f110806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:24.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.487+0000 7f112ce00640 1 --2- 172.21.15.159:0/1631372598 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f110806c920 0x7f110806ede0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1128160cf0 tx=0x7f1114009350 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:24.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.494+0000 7f1125600640 1 -- 172.21.15.159:0/1631372598 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f111c059d80 con 0x7f11100051c0 2024-04-04T06:10:24.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.699+0000 7f112ec00640 1 -- 172.21.15.159:0/1631372598 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11280ccda0 con 0x7f11100051c0 2024-04-04T06:10:24.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.702+0000 7f1125600640 1 -- 172.21.15.159:0/1631372598 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16677 (secure 0 0 0) 0x7f111c059680 con 0x7f11100051c0 2024-04-04T06:10:24.695 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:24.695 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":59,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:08:58.581221+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:08:54.848337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-04T06:08:56.003356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:10:24.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.704+0000 7f112ec00640 1 -- 172.21.15.159:0/1631372598 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f110806c920 msgr2=0x7f110806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:24.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.704+0000 7f112ec00640 1 --2- 172.21.15.159:0/1631372598 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f110806c920 0x7f110806ede0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1128160cf0 tx=0x7f1114009350 comp rx=0 tx=0).stop 2024-04-04T06:10:24.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.704+0000 7f112ec00640 1 -- 172.21.15.159:0/1631372598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11100051c0 msgr2=0x7f11280c7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:24.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.704+0000 7f112ec00640 1 --2- 172.21.15.159:0/1631372598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11100051c0 0x7f11280c7660 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f111c00e7b0 tx=0x7f111c00ec70 comp rx=0 tx=0).stop 2024-04-04T06:10:24.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.704+0000 7f112ec00640 1 -- 172.21.15.159:0/1631372598 shutdown_connections 2024-04-04T06:10:24.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.704+0000 7f112ec00640 1 --2- 172.21.15.159:0/1631372598 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11280af1a0 0x7f11280af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:24.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.704+0000 7f112ec00640 1 --2- 172.21.15.159:0/1631372598 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f110806c920 0x7f110806ede0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:24.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.704+0000 7f112ec00640 1 --2- 172.21.15.159:0/1631372598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11281571d0 0x7f1128156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:24.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.704+0000 7f112ec00640 1 --2- 172.21.15.159:0/1631372598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11100051c0 0x7f11280c7660 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:24.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.704+0000 7f112ec00640 1 -- 172.21.15.159:0/1631372598 >> 172.21.15.159:0/1631372598 conn(0x7f112809aa20 msgr2=0x7f112815b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:24.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.704+0000 7f112ec00640 1 -- 172.21.15.159:0/1631372598 shutdown_connections 2024-04-04T06:10:24.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.705+0000 7f112ec00640 1 -- 172.21.15.159:0/1631372598 wait complete. 2024-04-04T06:10:24.718 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-04T06:10:24.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.920+0000 7f6ea1000640 1 Processor -- start 2024-04-04T06:10:24.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.920+0000 7f6ea1000640 1 -- start start 2024-04-04T06:10:24.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.923+0000 7f6ea1000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6e9c153580 0x7f6e9c0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:24.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.923+0000 7f6ea1000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f6e9c0af4b0 0x7f6e9c1600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:24.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.923+0000 7f6ea1000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e9c0afab0 con 0x7f6e9c0d07c0 2024-04-04T06:10:24.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.923+0000 7f6ea1000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f6e9c0d0c30 con 0x7f6e9c0af4b0 2024-04-04T06:10:24.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.923+0000 7f6ea1000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f6e9c15cc80 con 0x7f6e9c153580 2024-04-04T06:10:24.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.923+0000 7f6e99400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f6e9c0d07c0 0x7f6e9c0aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43976/0 (socket says 172.21.15.159:43976) 2024-04-04T06:10:24.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.923+0000 7f6e99400640 1 -- 172.21.15.159:0/1131004534 learned_addr learned my addr 172.21.15.159:0/1131004534 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:24.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.923+0000 7f6e9a800640 1 --2- 172.21.15.159:0/1131004534 >> v2:172.21.15.159:3300/0 conn(0x7f6e9c0af4b0 0x7f6e9c1600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:24.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.924+0000 7f6e99e00640 1 --2- 172.21.15.159:0/1131004534 >> v2:172.21.15.178:3300/0 conn(0x7f6e9c153580 0x7f6e9c0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:24.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.924+0000 7f6e9a800640 1 -- 172.21.15.159:0/1131004534 >> v2:172.21.15.178:3300/0 conn(0x7f6e9c153580 msgr2=0x7f6e9c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:24.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.924+0000 7f6e9a800640 1 --2- 172.21.15.159:0/1131004534 >> v2:172.21.15.178:3300/0 conn(0x7f6e9c153580 0x7f6e9c0d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:24.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.924+0000 7f6e9a800640 1 -- 172.21.15.159:0/1131004534 >> v1:172.21.15.159:6789/0 conn(0x7f6e9c0d07c0 legacy=0x7f6e9c0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:24.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.924+0000 7f6e9a800640 1 -- 172.21.15.159:0/1131004534 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9c15da00 con 0x7f6e9c0af4b0 2024-04-04T06:10:24.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.924+0000 7f6e98a00640 1 -- 172.21.15.159:0/1131004534 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 429093412 0 0) 0x7f6e9c0afab0 con 0x7f6e9c0d07c0 2024-04-04T06:10:24.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.926+0000 7f6e9a800640 1 --2- 172.21.15.159:0/1131004534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e9c0af4b0 0x7f6e9c1600d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0b5270 tx=0x7f6e9402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb9279460c838a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:24.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.926+0000 7f6e98a00640 1 -- 172.21.15.159:0/1131004534 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e94039070 con 0x7f6e9c0af4b0 2024-04-04T06:10:24.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.926+0000 7f6e98a00640 1 -- 172.21.15.159:0/1131004534 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e9402cbf0 con 0x7f6e9c0af4b0 2024-04-04T06:10:24.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.926+0000 7f6e98a00640 1 -- 172.21.15.159:0/1131004534 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e94034440 con 0x7f6e9c0af4b0 2024-04-04T06:10:24.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.926+0000 7f6ea1000640 1 -- 172.21.15.159:0/1131004534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e9c0af4b0 msgr2=0x7f6e9c1600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:24.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.926+0000 7f6ea1000640 1 --2- 172.21.15.159:0/1131004534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e9c0af4b0 0x7f6e9c1600d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0b5270 tx=0x7f6e9402be40 comp rx=0 tx=0).stop 2024-04-04T06:10:24.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.927+0000 7f6ea1000640 1 -- 172.21.15.159:0/1131004534 shutdown_connections 2024-04-04T06:10:24.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.927+0000 7f6ea1000640 1 --2- 172.21.15.159:0/1131004534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e9c0af4b0 0x7f6e9c1600d0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:24.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.927+0000 7f6ea1000640 1 --2- 172.21.15.159:0/1131004534 >> v2:172.21.15.178:3300/0 conn(0x7f6e9c153580 0x7f6e9c0d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:24.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.927+0000 7f6ea1000640 1 -- 172.21.15.159:0/1131004534 >> 172.21.15.159:0/1131004534 conn(0x7f6e9c06b750 msgr2=0x7f6e9c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:24.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.927+0000 7f6ea1000640 1 -- 172.21.15.159:0/1131004534 shutdown_connections 2024-04-04T06:10:24.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.927+0000 7f6ea1000640 1 -- 172.21.15.159:0/1131004534 wait complete. 2024-04-04T06:10:24.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.928+0000 7f6ea1000640 1 Processor -- start 2024-04-04T06:10:24.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.929+0000 7f6ea1000640 1 -- start start 2024-04-04T06:10:24.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.929+0000 7f6ea1000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e9c0af4b0 0x7f6e9c1f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:24.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.929+0000 7f6ea1000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e9c0d07c0 0x7f6e9c0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:24.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.930+0000 7f6ea1000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e9c153580 0x7f6e9c0b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:24.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.930+0000 7f6ea1000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9c15d4e0 con 0x7f6e9c0d07c0 2024-04-04T06:10:24.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.930+0000 7f6ea1000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9c15d200 con 0x7f6e9c153580 2024-04-04T06:10:24.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.930+0000 7f6ea1000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6e9c153960 con 0x7f6e9c0af4b0 2024-04-04T06:10:24.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.930+0000 7f6e99400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e9c0d07c0 0x7f6e9c0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:24.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.930+0000 7f6e99400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e9c0d07c0 0x7f6e9c0b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41880/0 (socket says 172.21.15.159:41880) 2024-04-04T06:10:24.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.930+0000 7f6e99400640 1 -- 172.21.15.159:0/1634364323 learned_addr learned my addr 172.21.15.159:0/1634364323 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.930+0000 7f6e99400640 1 -- 172.21.15.159:0/1634364323 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e9c0af4b0 msgr2=0x7f6e9c1f4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.930+0000 7f6e99e00640 1 --2- 172.21.15.159:0/1634364323 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e9c0af4b0 0x7f6e9c1f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.930+0000 7f6e9a800640 1 --2- 172.21.15.159:0/1634364323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e9c153580 0x7f6e9c0b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.930+0000 7f6e99400640 1 --2- 172.21.15.159:0/1634364323 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e9c0af4b0 0x7f6e9c1f4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.930+0000 7f6e99400640 1 -- 172.21.15.159:0/1634364323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e9c153580 msgr2=0x7f6e9c0b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.930+0000 7f6e99400640 1 --2- 172.21.15.159:0/1634364323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e9c153580 0x7f6e9c0b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.930+0000 7f6e99400640 1 -- 172.21.15.159:0/1634364323 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9c1f5400 con 0x7f6e9c0d07c0 2024-04-04T06:10:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.930+0000 7f6e99e00640 1 --2- 172.21.15.159:0/1634364323 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e9c0af4b0 0x7f6e9c1f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.930+0000 7f6e9a800640 1 --2- 172.21.15.159:0/1634364323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e9c153580 0x7f6e9c0b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.931+0000 7f6e99400640 1 --2- 172.21.15.159:0/1634364323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e9c0d07c0 0x7f6e9c0b0cf0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800c500 tx=0x7f6e8800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.931+0000 7f6e92000640 1 -- 172.21.15.159:0/1634364323 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e88018070 con 0x7f6e9c0d07c0 2024-04-04T06:10:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.931+0000 7f6e92000640 1 -- 172.21.15.159:0/1634364323 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e88007580 con 0x7f6e9c0d07c0 2024-04-04T06:10:24.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.931+0000 7f6e92000640 1 -- 172.21.15.159:0/1634364323 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e88004a20 con 0x7f6e9c0d07c0 2024-04-04T06:10:24.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.931+0000 7f6ea1000640 1 -- 172.21.15.159:0/1634364323 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e9c156250 con 0x7f6e9c0d07c0 2024-04-04T06:10:24.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.931+0000 7f6ea1000640 1 -- 172.21.15.159:0/1634364323 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e9c156810 con 0x7f6e9c0d07c0 2024-04-04T06:10:24.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.932+0000 7f6ea1000640 1 -- 172.21.15.159:0/1634364323 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e9c0685c0 con 0x7f6e9c0d07c0 2024-04-04T06:10:24.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.933+0000 7f6e92000640 1 -- 172.21.15.159:0/1634364323 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e88004bc0 con 0x7f6e9c0d07c0 2024-04-04T06:10:24.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.934+0000 7f6e92000640 1 --2- 172.21.15.159:0/1634364323 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6e7c06c920 0x7f6e7c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:24.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.934+0000 7f6e92000640 1 -- 172.21.15.159:0/1634364323 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f6e8808ae20 con 0x7f6e9c0d07c0 2024-04-04T06:10:24.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.935+0000 7f6e99e00640 1 --2- 172.21.15.159:0/1634364323 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6e7c06c920 0x7f6e7c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:24.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.936+0000 7f6e99e00640 1 --2- 172.21.15.159:0/1634364323 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6e7c06c920 0x7f6e7c06ede0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c006fd0 tx=0x7f6e8c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:24.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:24.938+0000 7f6e92000640 1 -- 172.21.15.159:0/1634364323 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6e9c1f5400 con 0x7f6e9c0d07c0 2024-04-04T06:10:25.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.181+0000 7f6ea1000640 1 -- 172.21.15.159:0/1634364323 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6e9c06eca0 con 0x7f6e9c0d07c0 2024-04-04T06:10:25.175 INFO:tasks.ceph.mon.a.smithi159.stderr:2024-04-04T06:10:25.182+0000 7f1995c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-04T06:10:25.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.185+0000 7f6e92000640 1 -- 172.21.15.159:0/1634364323 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) ==== 161+0+0 (secure 0 0 0) 0x7f6e88059240 con 0x7f6e9c0d07c0 2024-04-04T06:10:25.179 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-04T06:10:25.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.189+0000 7f6ea1000640 1 -- 172.21.15.159:0/1634364323 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6e7c06c920 msgr2=0x7f6e7c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:25.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.189+0000 7f6ea1000640 1 --2- 172.21.15.159:0/1634364323 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6e7c06c920 0x7f6e7c06ede0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c006fd0 tx=0x7f6e8c008040 comp rx=0 tx=0).stop 2024-04-04T06:10:25.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.189+0000 7f6ea1000640 1 -- 172.21.15.159:0/1634364323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e9c0d07c0 msgr2=0x7f6e9c0b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:25.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.189+0000 7f6ea1000640 1 --2- 172.21.15.159:0/1634364323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e9c0d07c0 0x7f6e9c0b0cf0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800c500 tx=0x7f6e8800c9c0 comp rx=0 tx=0).stop 2024-04-04T06:10:25.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.189+0000 7f6ea1000640 1 -- 172.21.15.159:0/1634364323 shutdown_connections 2024-04-04T06:10:25.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.189+0000 7f6ea1000640 1 --2- 172.21.15.159:0/1634364323 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6e7c06c920 0x7f6e7c06ede0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:25.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.190+0000 7f6ea1000640 1 --2- 172.21.15.159:0/1634364323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e9c153580 0x7f6e9c0b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:25.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.190+0000 7f6ea1000640 1 --2- 172.21.15.159:0/1634364323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e9c0d07c0 0x7f6e9c0b0cf0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:25.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.190+0000 7f6ea1000640 1 --2- 172.21.15.159:0/1634364323 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e9c0af4b0 0x7f6e9c1f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:25.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.190+0000 7f6ea1000640 1 -- 172.21.15.159:0/1634364323 >> 172.21.15.159:0/1634364323 conn(0x7f6e9c06b750 msgr2=0x7f6e9c06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:25.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.190+0000 7f6ea1000640 1 -- 172.21.15.159:0/1634364323 shutdown_connections 2024-04-04T06:10:25.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.190+0000 7f6ea1000640 1 -- 172.21.15.159:0/1634364323 wait complete. 2024-04-04T06:10:25.198 INFO:tasks.ceph.mds.c.smithi159.stderr: -7995> 2024-04-04T06:09:02.513+0000 7f193e600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:10:25.202 INFO:tasks.ceph.mds.c.smithi159.stderr: -4974> 2024-04-04T06:09:12.801+0000 7f1939600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5296=0-4194304@1} caps={5296=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5296 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x561a275bac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-04T06:10:25.203 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-04T06:10:25.207 INFO:tasks.ceph.mds.c.smithi159.stderr: -2984> 2024-04-04T06:09:44.858+0000 7f1939600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-04-04T06:09:11.257106+0000 1=1+0) caps={5296=pAsLsXsFs/-@2},l=5296(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x561a275bac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-04T06:10:25.258 INFO:tasks.ceph.mds.c.smithi159.stdout:starting mds.c at 2024-04-04T06:10:25.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.399+0000 7ff4ada00640 1 Processor -- start 2024-04-04T06:10:25.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.400+0000 7ff4ada00640 1 -- start start 2024-04-04T06:10:25.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.401+0000 7ff4ada00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff4a8155550 0x7ff4a8155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:25.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.401+0000 7ff4ada00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff4a80d0990 0x7ff4a80aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:25.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.401+0000 7ff4ada00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4a80b40b0 con 0x7ff4a815f800 2024-04-04T06:10:25.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.401+0000 7ff4ada00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7ff4a80b4390 con 0x7ff4a8155550 2024-04-04T06:10:25.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.401+0000 7ff4ada00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff4a80b4670 con 0x7ff4a80d0990 2024-04-04T06:10:25.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.401+0000 7ff4a6a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff4a8155550 0x7ff4a8155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:25.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.401+0000 7ff4a6a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff4a8155550 0x7ff4a8155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41896/0 (socket says 172.21.15.159:41896) 2024-04-04T06:10:25.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.401+0000 7ff4a6a00640 1 -- 172.21.15.159:0/1908653400 learned_addr learned my addr 172.21.15.159:0/1908653400 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:25.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.402+0000 7ff4a6a00640 1 -- 172.21.15.159:0/1908653400 >> v2:172.21.15.178:3300/0 conn(0x7ff4a80d0990 msgr2=0x7ff4a80aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:25.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.402+0000 7ff4a7400640 1 --2- 172.21.15.159:0/1908653400 >> v2:172.21.15.178:3300/0 conn(0x7ff4a80d0990 0x7ff4a80aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:25.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.402+0000 7ff4a6a00640 1 --2- 172.21.15.159:0/1908653400 >> v2:172.21.15.178:3300/0 conn(0x7ff4a80d0990 0x7ff4a80aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:25.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.402+0000 7ff4a6a00640 1 -- 172.21.15.159:0/1908653400 >> v1:172.21.15.178:6789/0 conn(0x7ff4a815f800 legacy=0x7ff4a80d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:25.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.402+0000 7ff4a6a00640 1 -- 172.21.15.159:0/1908653400 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a80b4950 con 0x7ff4a8155550 2024-04-04T06:10:25.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.403+0000 7ff4a6a00640 1 --2- 172.21.15.159:0/1908653400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4a8155550 0x7ff4a8155930 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8073ac0 tx=0x7ff49802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c478b32a923989 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:25.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.403+0000 7ff4a6000640 1 -- 172.21.15.159:0/1908653400 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4980317c0 con 0x7ff4a8155550 2024-04-04T06:10:25.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.403+0000 7ff4a6000640 1 -- 172.21.15.159:0/1908653400 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff498037070 con 0x7ff4a8155550 2024-04-04T06:10:25.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.403+0000 7ff4a6000640 1 -- 172.21.15.159:0/1908653400 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498031b80 con 0x7ff4a8155550 2024-04-04T06:10:25.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.404+0000 7ff4ada00640 1 -- 172.21.15.159:0/1908653400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4a8155550 msgr2=0x7ff4a8155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:25.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.404+0000 7ff4ada00640 1 --2- 172.21.15.159:0/1908653400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4a8155550 0x7ff4a8155930 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8073ac0 tx=0x7ff49802b860 comp rx=0 tx=0).stop 2024-04-04T06:10:25.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.404+0000 7ff4ada00640 1 -- 172.21.15.159:0/1908653400 shutdown_connections 2024-04-04T06:10:25.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.404+0000 7ff4ada00640 1 --2- 172.21.15.159:0/1908653400 >> v2:172.21.15.178:3300/0 conn(0x7ff4a80d0990 0x7ff4a80aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:25.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.404+0000 7ff4ada00640 1 --2- 172.21.15.159:0/1908653400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4a8155550 0x7ff4a8155930 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:25.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.404+0000 7ff4ada00640 1 -- 172.21.15.159:0/1908653400 >> 172.21.15.159:0/1908653400 conn(0x7ff4a806b750 msgr2=0x7ff4a806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:25.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.404+0000 7ff4ada00640 1 -- 172.21.15.159:0/1908653400 shutdown_connections 2024-04-04T06:10:25.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.404+0000 7ff4ada00640 1 -- 172.21.15.159:0/1908653400 wait complete. 2024-04-04T06:10:25.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.406+0000 7ff4ada00640 1 Processor -- start 2024-04-04T06:10:25.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.406+0000 7ff4ada00640 1 -- start start 2024-04-04T06:10:25.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.407+0000 7ff4ada00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4a80d0990 0x7ff4a815cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:25.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.407+0000 7ff4ada00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4a8155550 0x7ff4a814c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:25.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.407+0000 7ff4ada00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4a815f800 0x7ff4a80991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:25.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.407+0000 7ff4ada00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a815ba20 con 0x7ff4a80d0990 2024-04-04T06:10:25.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.407+0000 7ff4ada00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a815fc80 con 0x7ff4a815f800 2024-04-04T06:10:25.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.407+0000 7ff4ada00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff4a815bba0 con 0x7ff4a8155550 2024-04-04T06:10:25.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.407+0000 7ff4a6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4a80d0990 0x7ff4a815cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:25.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.407+0000 7ff4a6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4a80d0990 0x7ff4a815cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41900/0 (socket says 172.21.15.159:41900) 2024-04-04T06:10:25.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.407+0000 7ff4a6a00640 1 -- 172.21.15.159:0/2144970380 learned_addr learned my addr 172.21.15.159:0/2144970380 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:25.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.407+0000 7ff4a7400640 1 --2- 172.21.15.159:0/2144970380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4a815f800 0x7ff4a80991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:25.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.407+0000 7ff49e000640 1 --2- 172.21.15.159:0/2144970380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4a8155550 0x7ff4a814c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:25.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.407+0000 7ff4a6a00640 1 -- 172.21.15.159:0/2144970380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4a8155550 msgr2=0x7ff4a814c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:25.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.408+0000 7ff4a6a00640 1 --2- 172.21.15.159:0/2144970380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4a8155550 0x7ff4a814c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:25.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.408+0000 7ff4a6a00640 1 -- 172.21.15.159:0/2144970380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4a815f800 msgr2=0x7ff4a80991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:25.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.408+0000 7ff4a6a00640 1 --2- 172.21.15.159:0/2144970380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4a815f800 0x7ff4a80991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:25.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.408+0000 7ff4a6a00640 1 -- 172.21.15.159:0/2144970380 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a81533c0 con 0x7ff4a80d0990 2024-04-04T06:10:25.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.408+0000 7ff4a7400640 1 --2- 172.21.15.159:0/2144970380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4a815f800 0x7ff4a80991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:25.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.408+0000 7ff4a6a00640 1 --2- 172.21.15.159:0/2144970380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4a80d0990 0x7ff4a815cd70 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7ff498000c00 tx=0x7ff4980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:25.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.408+0000 7ff49f400640 1 -- 172.21.15.159:0/2144970380 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498032860 con 0x7ff4a80d0990 2024-04-04T06:10:25.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.408+0000 7ff49f400640 1 -- 172.21.15.159:0/2144970380 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff498032a00 con 0x7ff4a80d0990 2024-04-04T06:10:25.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.408+0000 7ff4ada00640 1 -- 172.21.15.159:0/2144970380 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a80b0be0 con 0x7ff4a80d0990 2024-04-04T06:10:25.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.408+0000 7ff49f400640 1 -- 172.21.15.159:0/2144970380 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498032dd0 con 0x7ff4a80d0990 2024-04-04T06:10:25.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.408+0000 7ff4ada00640 1 -- 172.21.15.159:0/2144970380 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a80cc850 con 0x7ff4a80d0990 2024-04-04T06:10:25.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.409+0000 7ff4ada00640 1 -- 172.21.15.159:0/2144970380 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a80685c0 con 0x7ff4a80d0990 2024-04-04T06:10:25.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.410+0000 7ff49f400640 1 -- 172.21.15.159:0/2144970380 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4980453f0 con 0x7ff4a80d0990 2024-04-04T06:10:25.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.411+0000 7ff49f400640 1 --2- 172.21.15.159:0/2144970380 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff48406c8f0 0x7ff48406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:25.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.411+0000 7ff49f400640 1 -- 172.21.15.159:0/2144970380 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5795+0+0 (secure 0 0 0) 0x7ff4980addc0 con 0x7ff4a80d0990 2024-04-04T06:10:25.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.412+0000 7ff49e000640 1 --2- 172.21.15.159:0/2144970380 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff48406c8f0 0x7ff48406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:25.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.413+0000 7ff49e000640 1 --2- 172.21.15.159:0/2144970380 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff48406c8f0 0x7ff48406edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff4a815af60 tx=0x7ff490009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:25.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.416+0000 7ff49f400640 1 -- 172.21.15.159:0/2144970380 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff4a81533c0 con 0x7ff4a80d0990 2024-04-04T06:10:25.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:25.663+0000 7ff4ada00640 1 -- 172.21.15.159:0/2144970380 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff4a80b3ad0 con 0x7ff4a80d0990 2024-04-04T06:10:26.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.188+0000 7ff49f400640 1 -- 172.21.15.159:0/2144970380 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) ==== 104+0+0 (secure 0 0 0) 0x7ff498040070 con 0x7ff4a80d0990 2024-04-04T06:10:26.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.192+0000 7ff4ada00640 1 -- 172.21.15.159:0/2144970380 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff48406c8f0 msgr2=0x7ff48406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:26.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.192+0000 7ff4ada00640 1 --2- 172.21.15.159:0/2144970380 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff48406c8f0 0x7ff48406edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff4a815af60 tx=0x7ff490009040 comp rx=0 tx=0).stop 2024-04-04T06:10:26.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.192+0000 7ff4ada00640 1 -- 172.21.15.159:0/2144970380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4a80d0990 msgr2=0x7ff4a815cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:26.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.192+0000 7ff4ada00640 1 --2- 172.21.15.159:0/2144970380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4a80d0990 0x7ff4a815cd70 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7ff498000c00 tx=0x7ff4980046c0 comp rx=0 tx=0).stop 2024-04-04T06:10:26.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.192+0000 7ff4ada00640 1 -- 172.21.15.159:0/2144970380 shutdown_connections 2024-04-04T06:10:26.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.192+0000 7ff4ada00640 1 --2- 172.21.15.159:0/2144970380 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff48406c8f0 0x7ff48406edb0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:26.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.192+0000 7ff4ada00640 1 --2- 172.21.15.159:0/2144970380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4a815f800 0x7ff4a80991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:26.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.192+0000 7ff4ada00640 1 --2- 172.21.15.159:0/2144970380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4a8155550 0x7ff4a814c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:26.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.192+0000 7ff4ada00640 1 --2- 172.21.15.159:0/2144970380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4a80d0990 0x7ff4a815cd70 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:26.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.192+0000 7ff4ada00640 1 -- 172.21.15.159:0/2144970380 >> 172.21.15.159:0/2144970380 conn(0x7ff4a806b750 msgr2=0x7ff4a815d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:26.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.193+0000 7ff4ada00640 1 -- 172.21.15.159:0/2144970380 shutdown_connections 2024-04-04T06:10:26.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.193+0000 7ff4ada00640 1 -- 172.21.15.159:0/2144970380 wait complete. 2024-04-04T06:10:26.207 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-04T06:10:26.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.429+0000 7fdea7800640 1 Processor -- start 2024-04-04T06:10:26.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.429+0000 7fdea7800640 1 -- start start 2024-04-04T06:10:26.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.431+0000 7fdea7800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdea0072720 0x7fdea0167300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:26.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.431+0000 7fdea7800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdea015cb90 con 0x7fdea009dcf0 2024-04-04T06:10:26.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.431+0000 7fdea7800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdea015ce70 con 0x7fdea00acf10 2024-04-04T06:10:26.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.431+0000 7fdea7800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fdea0145f60 con 0x7fdea0072720 2024-04-04T06:10:26.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.431+0000 7fde9fe00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fdea009dcf0 0x7fdea01453f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43994/0 (socket says 172.21.15.159:43994) 2024-04-04T06:10:26.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.431+0000 7fde9fe00640 1 -- 172.21.15.159:0/3965983387 learned_addr learned my addr 172.21.15.159:0/3965983387 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:26.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.431+0000 7fdea5800640 1 --2- 172.21.15.159:0/3965983387 >> v2:172.21.15.178:3300/0 conn(0x7fdea0072720 0x7fdea0167300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:26.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.431+0000 7fde9f400640 1 -- 172.21.15.159:0/3965983387 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3555734849 0 0) 0x7fdea015cb90 con 0x7fdea009dcf0 2024-04-04T06:10:26.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.432+0000 7fde9f400640 1 -- 172.21.15.159:0/3965983387 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde84003520 con 0x7fdea009dcf0 2024-04-04T06:10:26.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.432+0000 7fde9f400640 1 -- 172.21.15.159:0/3965983387 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1430158400 0 0) 0x7fde84003520 con 0x7fdea009dcf0 2024-04-04T06:10:26.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.432+0000 7fde9f400640 1 -- 172.21.15.159:0/3965983387 >> v2:172.21.15.178:3300/0 conn(0x7fdea0072720 msgr2=0x7fdea0167300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:26.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.432+0000 7fde9f400640 1 --2- 172.21.15.159:0/3965983387 >> v2:172.21.15.178:3300/0 conn(0x7fdea0072720 0x7fdea0167300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:26.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.432+0000 7fde9f400640 1 -- 172.21.15.159:0/3965983387 >> v1:172.21.15.178:6789/0 conn(0x7fdea00acf10 legacy=0x7fdea00ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:26.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.432+0000 7fde9f400640 1 -- 172.21.15.159:0/3965983387 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea015d150 con 0x7fdea009dcf0 2024-04-04T06:10:26.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.432+0000 7fde9f400640 1 -- 172.21.15.159:0/3965983387 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fde8c001670 con 0x7fdea009dcf0 2024-04-04T06:10:26.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.432+0000 7fde9f400640 1 -- 172.21.15.159:0/3965983387 >> v1:172.21.15.159:6789/0 conn(0x7fdea009dcf0 legacy=0x7fdea01453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:26.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.432+0000 7fde9f400640 1 --2- 172.21.15.159:0/3965983387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde840039b0 0x7fde84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:26.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.432+0000 7fde9f400640 1 --2- 172.21.15.159:0/3965983387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde84004800 0x7fde84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:26.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.432+0000 7fde9f400640 1 --2- 172.21.15.159:0/3965983387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde840052e0 0x7fde8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:26.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.432+0000 7fde9f400640 1 -- 172.21.15.159:0/3965983387 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdea0145f60 con 0x7fde84004800 2024-04-04T06:10:26.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.432+0000 7fde9f400640 1 -- 172.21.15.159:0/3965983387 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fde8400c1b0 con 0x7fde840052e0 2024-04-04T06:10:26.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.432+0000 7fde9f400640 1 -- 172.21.15.159:0/3965983387 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fde8400c3b0 con 0x7fde840039b0 2024-04-04T06:10:26.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.432+0000 7fde9fe00640 1 --2- 172.21.15.159:0/3965983387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde84004800 0x7fde84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:26.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.433+0000 7fdea5800640 1 --2- 172.21.15.159:0/3965983387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde840052e0 0x7fde8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:26.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.433+0000 7fdea4e00640 1 --2- 172.21.15.159:0/3965983387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde840039b0 0x7fde84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:26.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.433+0000 7fde9fe00640 1 -- 172.21.15.159:0/3965983387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde840039b0 msgr2=0x7fde84003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:26.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.433+0000 7fde9fe00640 1 --2- 172.21.15.159:0/3965983387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde840039b0 0x7fde84003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:26.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.433+0000 7fde9fe00640 1 -- 172.21.15.159:0/3965983387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde840052e0 msgr2=0x7fde8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:26.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.433+0000 7fde9fe00640 1 --2- 172.21.15.159:0/3965983387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde840052e0 0x7fde8400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:26.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.433+0000 7fde9fe00640 1 -- 172.21.15.159:0/3965983387 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde84003520 con 0x7fde84004800 2024-04-04T06:10:26.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.433+0000 7fdea5800640 1 -- 172.21.15.159:0/3965983387 reap_dead start 2024-04-04T06:10:26.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.435+0000 7fde9fe00640 1 --2- 172.21.15.159:0/3965983387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde84004800 0x7fde84004c80 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00c210 tx=0x7fde84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c90c0ea5498da5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:26.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.435+0000 7fde9f400640 1 -- 172.21.15.159:0/3965983387 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c007cf0 con 0x7fde84004800 2024-04-04T06:10:26.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.435+0000 7fde9f400640 1 -- 172.21.15.159:0/3965983387 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde8c030070 con 0x7fde84004800 2024-04-04T06:10:26.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.435+0000 7fde9f400640 1 -- 172.21.15.159:0/3965983387 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c005d70 con 0x7fde84004800 2024-04-04T06:10:26.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.435+0000 7fdea7800640 1 -- 172.21.15.159:0/3965983387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde84004800 msgr2=0x7fde84004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:26.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.435+0000 7fdea7800640 1 --2- 172.21.15.159:0/3965983387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde84004800 0x7fde84004c80 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00c210 tx=0x7fde84005830 comp rx=0 tx=0).stop 2024-04-04T06:10:26.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.436+0000 7fdea7800640 1 -- 172.21.15.159:0/3965983387 shutdown_connections 2024-04-04T06:10:26.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.436+0000 7fdea7800640 1 --2- 172.21.15.159:0/3965983387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde84004800 0x7fde84004c80 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:26.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.436+0000 7fdea7800640 1 -- 172.21.15.159:0/3965983387 >> 172.21.15.159:0/3965983387 conn(0x7fdea006bba0 msgr2=0x7fdea00b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:26.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.436+0000 7fdea7800640 1 -- 172.21.15.159:0/3965983387 shutdown_connections 2024-04-04T06:10:26.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.436+0000 7fdea7800640 1 -- 172.21.15.159:0/3965983387 wait complete. 2024-04-04T06:10:26.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.437+0000 7fdea7800640 1 Processor -- start 2024-04-04T06:10:26.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.438+0000 7fdea7800640 1 -- start start 2024-04-04T06:10:26.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.438+0000 7fdea7800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde84004800 0x7fdea009cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:26.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.438+0000 7fdea7800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdea0140890 0x7fdea015bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:26.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.438+0000 7fdea7800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea006d5d0 0x7fdea006da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:26.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.438+0000 7fdea7800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdea015d7a0 con 0x7fdea006d5d0 2024-04-04T06:10:26.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.438+0000 7fdea7800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdea015c950 con 0x7fdea0140890 2024-04-04T06:10:26.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.438+0000 7fdea7800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdea00ad2f0 con 0x7fde84004800 2024-04-04T06:10:26.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.439+0000 7fdea5800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea006d5d0 0x7fdea006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:26.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.439+0000 7fdea5800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea006d5d0 0x7fdea006da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41916/0 (socket says 172.21.15.159:41916) 2024-04-04T06:10:26.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.439+0000 7fdea5800640 1 -- 172.21.15.159:0/1331878640 learned_addr learned my addr 172.21.15.159:0/1331878640 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:26.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.439+0000 7fdea4e00640 1 --2- 172.21.15.159:0/1331878640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde84004800 0x7fdea009cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:26.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.439+0000 7fde9fe00640 1 --2- 172.21.15.159:0/1331878640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdea0140890 0x7fdea015bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:26.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.439+0000 7fdea5800640 1 -- 172.21.15.159:0/1331878640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde84004800 msgr2=0x7fdea009cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:26.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.439+0000 7fdea5800640 1 --2- 172.21.15.159:0/1331878640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde84004800 0x7fdea009cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:26.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.439+0000 7fdea5800640 1 -- 172.21.15.159:0/1331878640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdea0140890 msgr2=0x7fdea015bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:26.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.439+0000 7fdea5800640 1 --2- 172.21.15.159:0/1331878640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdea0140890 0x7fdea015bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:26.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.439+0000 7fdea5800640 1 -- 172.21.15.159:0/1331878640 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea015c1a0 con 0x7fdea006d5d0 2024-04-04T06:10:26.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.440+0000 7fdea5800640 1 --2- 172.21.15.159:0/1331878640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea006d5d0 0x7fdea006da50 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fde98007cd0 tx=0x7fde980115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:26.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.440+0000 7fde9cc00640 1 -- 172.21.15.159:0/1331878640 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde98011e40 con 0x7fdea006d5d0 2024-04-04T06:10:26.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.440+0000 7fde9cc00640 1 -- 172.21.15.159:0/1331878640 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde9801b070 con 0x7fdea006d5d0 2024-04-04T06:10:26.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.440+0000 7fde9cc00640 1 -- 172.21.15.159:0/1331878640 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde98003990 con 0x7fdea006d5d0 2024-04-04T06:10:26.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.440+0000 7fdea7800640 1 -- 172.21.15.159:0/1331878640 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea009c5b0 con 0x7fdea006d5d0 2024-04-04T06:10:26.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.440+0000 7fdea7800640 1 -- 172.21.15.159:0/1331878640 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea009cb70 con 0x7fdea006d5d0 2024-04-04T06:10:26.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.441+0000 7fdea7800640 1 -- 172.21.15.159:0/1331878640 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea006df90 con 0x7fdea006d5d0 2024-04-04T06:10:26.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.442+0000 7fde9cc00640 1 -- 172.21.15.159:0/1331878640 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde98005d10 con 0x7fdea006d5d0 2024-04-04T06:10:26.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.443+0000 7fde9cc00640 1 --2- 172.21.15.159:0/1331878640 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde8006c8f0 0x7fde8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:26.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.443+0000 7fde9cc00640 1 -- 172.21.15.159:0/1331878640 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5795+0+0 (secure 0 0 0) 0x7fde9808c0b0 con 0x7fdea006d5d0 2024-04-04T06:10:26.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.444+0000 7fdea4e00640 1 --2- 172.21.15.159:0/1331878640 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde8006c8f0 0x7fde8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:26.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.445+0000 7fdea4e00640 1 --2- 172.21.15.159:0/1331878640 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde8006c8f0 0x7fde8006edb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fde90000c00 tx=0x7fde90009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:26.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.448+0000 7fde9cc00640 1 -- 172.21.15.159:0/1331878640 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fde98014350 con 0x7fdea006d5d0 2024-04-04T06:10:26.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:26.649+0000 7fdea7800640 1 -- 172.21.15.159:0/1331878640 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdea00cf8d0 con 0x7fdea006d5d0 2024-04-04T06:10:27.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.198+0000 7fde9cc00640 1 -- 172.21.15.159:0/1331878640 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v61) ==== 181+0+0 (secure 0 0 0) 0x7fdea00cf8d0 con 0x7fdea006d5d0 2024-04-04T06:10:27.192 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_metadata' removed 2024-04-04T06:10:27.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.202+0000 7fdea7800640 1 -- 172.21.15.159:0/1331878640 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde8006c8f0 msgr2=0x7fde8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:27.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.202+0000 7fdea7800640 1 --2- 172.21.15.159:0/1331878640 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde8006c8f0 0x7fde8006edb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fde90000c00 tx=0x7fde90009040 comp rx=0 tx=0).stop 2024-04-04T06:10:27.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.202+0000 7fdea7800640 1 -- 172.21.15.159:0/1331878640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea006d5d0 msgr2=0x7fdea006da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:27.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.202+0000 7fdea7800640 1 --2- 172.21.15.159:0/1331878640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea006d5d0 0x7fdea006da50 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fde98007cd0 tx=0x7fde980115a0 comp rx=0 tx=0).stop 2024-04-04T06:10:27.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.202+0000 7fdea7800640 1 -- 172.21.15.159:0/1331878640 shutdown_connections 2024-04-04T06:10:27.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.202+0000 7fdea7800640 1 --2- 172.21.15.159:0/1331878640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea006d5d0 0x7fdea006da50 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:27.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.202+0000 7fdea7800640 1 --2- 172.21.15.159:0/1331878640 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde8006c8f0 0x7fde8006edb0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:27.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.202+0000 7fdea7800640 1 --2- 172.21.15.159:0/1331878640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdea0140890 0x7fdea015bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:27.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.202+0000 7fdea7800640 1 --2- 172.21.15.159:0/1331878640 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde84004800 0x7fdea009cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:27.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.202+0000 7fdea7800640 1 -- 172.21.15.159:0/1331878640 >> 172.21.15.159:0/1331878640 conn(0x7fdea006bba0 msgr2=0x7fdea006ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:27.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.203+0000 7fdea7800640 1 -- 172.21.15.159:0/1331878640 shutdown_connections 2024-04-04T06:10:27.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.203+0000 7fdea7800640 1 -- 172.21.15.159:0/1331878640 wait complete. 2024-04-04T06:10:27.223 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-04T06:10:27.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.436+0000 7f33bbc00640 1 Processor -- start 2024-04-04T06:10:27.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.436+0000 7f33bbc00640 1 -- start start 2024-04-04T06:10:27.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.438+0000 7f33bbc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f33b409dcf0 0x7f33b41453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:27.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.438+0000 7f33bbc00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33b415cb90 con 0x7f33b4071710 2024-04-04T06:10:27.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.438+0000 7f33bbc00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33b415ce70 con 0x7f33b40acf10 2024-04-04T06:10:27.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.438+0000 7f33bbc00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f33b4145f60 con 0x7f33b409dcf0 2024-04-04T06:10:27.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.438+0000 7f33b3e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f33b409dcf0 0x7f33b41453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:27.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.438+0000 7f33b9c00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f33b4071710 0x7f33b4071b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43996/0 (socket says 172.21.15.159:43996) 2024-04-04T06:10:27.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.438+0000 7f33b9c00640 1 -- 172.21.15.159:0/4128231687 learned_addr learned my addr 172.21.15.159:0/4128231687 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:27.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.439+0000 7f33b3400640 1 -- 172.21.15.159:0/4128231687 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 106650520 0 0) 0x7f33b415cb90 con 0x7f33b4071710 2024-04-04T06:10:27.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.439+0000 7f33b3e00640 1 -- 172.21.15.159:0/4128231687 >> v1:172.21.15.178:6789/0 conn(0x7f33b40acf10 legacy=0x7f33b40ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:27.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.439+0000 7f33b3e00640 1 -- 172.21.15.159:0/4128231687 >> v1:172.21.15.159:6789/0 conn(0x7f33b4071710 legacy=0x7f33b4071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:27.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.439+0000 7f33b3e00640 1 -- 172.21.15.159:0/4128231687 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b415d150 con 0x7f33b409dcf0 2024-04-04T06:10:27.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.440+0000 7f33b3e00640 1 --2- 172.21.15.159:0/4128231687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33b409dcf0 0x7f33b41453f0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f33a00097a0 tx=0x7f33a002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d435fe7678d59beb server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:27.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.440+0000 7f33b3400640 1 -- 172.21.15.159:0/4128231687 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a00315a0 con 0x7f33b409dcf0 2024-04-04T06:10:27.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.440+0000 7f33b3400640 1 -- 172.21.15.159:0/4128231687 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a0037070 con 0x7f33b409dcf0 2024-04-04T06:10:27.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.440+0000 7f33b3400640 1 -- 172.21.15.159:0/4128231687 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0031960 con 0x7f33b409dcf0 2024-04-04T06:10:27.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.441+0000 7f33bbc00640 1 -- 172.21.15.159:0/4128231687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33b409dcf0 msgr2=0x7f33b41453f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:27.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.441+0000 7f33bbc00640 1 --2- 172.21.15.159:0/4128231687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33b409dcf0 0x7f33b41453f0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f33a00097a0 tx=0x7f33a002b840 comp rx=0 tx=0).stop 2024-04-04T06:10:27.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.441+0000 7f33bbc00640 1 -- 172.21.15.159:0/4128231687 shutdown_connections 2024-04-04T06:10:27.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.441+0000 7f33bbc00640 1 --2- 172.21.15.159:0/4128231687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33b409dcf0 0x7f33b41453f0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:27.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.441+0000 7f33bbc00640 1 -- 172.21.15.159:0/4128231687 >> 172.21.15.159:0/4128231687 conn(0x7f33b406bba0 msgr2=0x7f33b40b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:27.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.441+0000 7f33bbc00640 1 -- 172.21.15.159:0/4128231687 shutdown_connections 2024-04-04T06:10:27.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.441+0000 7f33bbc00640 1 -- 172.21.15.159:0/4128231687 wait complete. 2024-04-04T06:10:27.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.443+0000 7f33bbc00640 1 Processor -- start 2024-04-04T06:10:27.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.443+0000 7f33bbc00640 1 -- start start 2024-04-04T06:10:27.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.444+0000 7f33bbc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33b4071710 0x7f33b413b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:27.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.444+0000 7f33bbc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33b409dcf0 0x7f33b415b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:27.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.444+0000 7f33bbc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33b40acf10 0x7f33b415fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:27.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.444+0000 7f33bbc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f33b415d7a0 con 0x7f33b40acf10 2024-04-04T06:10:27.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.444+0000 7f33bbc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f33b415c950 con 0x7f33b409dcf0 2024-04-04T06:10:27.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.444+0000 7f33bbc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f33b4145930 con 0x7f33b4071710 2024-04-04T06:10:27.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.444+0000 7f33b9c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33b40acf10 0x7f33b415fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:27.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.444+0000 7f33b9c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33b40acf10 0x7f33b415fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41946/0 (socket says 172.21.15.159:41946) 2024-04-04T06:10:27.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.444+0000 7f33b9c00640 1 -- 172.21.15.159:0/4142408508 learned_addr learned my addr 172.21.15.159:0/4142408508 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:27.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.445+0000 7f33b9c00640 1 -- 172.21.15.159:0/4142408508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33b4071710 msgr2=0x7f33b413b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:27.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.445+0000 7f33b3e00640 1 --2- 172.21.15.159:0/4142408508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33b409dcf0 0x7f33b415b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:27.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.445+0000 7f33b9200640 1 --2- 172.21.15.159:0/4142408508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33b4071710 0x7f33b413b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:27.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.445+0000 7f33b9c00640 1 --2- 172.21.15.159:0/4142408508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33b4071710 0x7f33b413b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:27.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.445+0000 7f33b9c00640 1 -- 172.21.15.159:0/4142408508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33b409dcf0 msgr2=0x7f33b415b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:27.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.445+0000 7f33b9c00640 1 --2- 172.21.15.159:0/4142408508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33b409dcf0 0x7f33b415b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:27.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.445+0000 7f33b9c00640 1 -- 172.21.15.159:0/4142408508 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b413c090 con 0x7f33b40acf10 2024-04-04T06:10:27.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.445+0000 7f33b9200640 1 --2- 172.21.15.159:0/4142408508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33b4071710 0x7f33b413b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:27.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.445+0000 7f33b3e00640 1 --2- 172.21.15.159:0/4142408508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33b409dcf0 0x7f33b415b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:27.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.445+0000 7f33b9c00640 1 --2- 172.21.15.159:0/4142408508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33b40acf10 0x7f33b415fee0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00c7d0 tx=0x7f33ac00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:27.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.445+0000 7f33b0c00640 1 -- 172.21.15.159:0/4142408508 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac018070 con 0x7f33b40acf10 2024-04-04T06:10:27.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.446+0000 7f33b0c00640 1 -- 172.21.15.159:0/4142408508 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33ac00a640 con 0x7f33b40acf10 2024-04-04T06:10:27.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.446+0000 7f33bbc00640 1 -- 172.21.15.159:0/4142408508 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b413c370 con 0x7f33b40acf10 2024-04-04T06:10:27.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.446+0000 7f33b0c00640 1 -- 172.21.15.159:0/4142408508 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac00aa70 con 0x7f33b40acf10 2024-04-04T06:10:27.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.446+0000 7f33bbc00640 1 -- 172.21.15.159:0/4142408508 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b413c930 con 0x7f33b40acf10 2024-04-04T06:10:27.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.447+0000 7f33bbc00640 1 -- 172.21.15.159:0/4142408508 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b40685c0 con 0x7f33b40acf10 2024-04-04T06:10:27.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.448+0000 7f33b0c00640 1 -- 172.21.15.159:0/4142408508 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f33ac01b070 con 0x7f33b40acf10 2024-04-04T06:10:27.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.448+0000 7f33b0c00640 1 --2- 172.21.15.159:0/4142408508 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f339406c920 0x7f339406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:27.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.449+0000 7f33b0c00640 1 -- 172.21.15.159:0/4142408508 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5357+0+0 (secure 0 0 0) 0x7f33ac08afe0 con 0x7f33b40acf10 2024-04-04T06:10:27.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.449+0000 7f33b9200640 1 --2- 172.21.15.159:0/4142408508 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f339406c920 0x7f339406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:27.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.450+0000 7f33b9200640 1 --2- 172.21.15.159:0/4142408508 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f339406c920 0x7f339406ede0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f33a800a830 tx=0x7f33a8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:27.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.453+0000 7f33b0c00640 1 -- 172.21.15.159:0/4142408508 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f33ac059580 con 0x7f33b40acf10 2024-04-04T06:10:27.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:27.654+0000 7f33bbc00640 1 -- 172.21.15.159:0/4142408508 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f33b40b2900 con 0x7f33b40acf10 2024-04-04T06:10:28.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.202+0000 7f33b0c00640 1 -- 172.21.15.159:0/4142408508 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v62) ==== 169+0+0 (secure 0 0 0) 0x7f33ac058e80 con 0x7f33b40acf10 2024-04-04T06:10:28.195 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_data' removed 2024-04-04T06:10:28.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.205+0000 7f33bbc00640 1 -- 172.21.15.159:0/4142408508 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f339406c920 msgr2=0x7f339406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:28.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.205+0000 7f33bbc00640 1 --2- 172.21.15.159:0/4142408508 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f339406c920 0x7f339406ede0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f33a800a830 tx=0x7f33a8008040 comp rx=0 tx=0).stop 2024-04-04T06:10:28.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.205+0000 7f33bbc00640 1 -- 172.21.15.159:0/4142408508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33b40acf10 msgr2=0x7f33b415fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:28.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.205+0000 7f33bbc00640 1 --2- 172.21.15.159:0/4142408508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33b40acf10 0x7f33b415fee0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00c7d0 tx=0x7f33ac00cc90 comp rx=0 tx=0).stop 2024-04-04T06:10:28.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.206+0000 7f33bbc00640 1 -- 172.21.15.159:0/4142408508 shutdown_connections 2024-04-04T06:10:28.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.206+0000 7f33bbc00640 1 --2- 172.21.15.159:0/4142408508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33b40acf10 0x7f33b415fee0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:28.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.206+0000 7f33bbc00640 1 --2- 172.21.15.159:0/4142408508 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f339406c920 0x7f339406ede0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:28.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.206+0000 7f33bbc00640 1 --2- 172.21.15.159:0/4142408508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33b409dcf0 0x7f33b415b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:28.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.206+0000 7f33bbc00640 1 --2- 172.21.15.159:0/4142408508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33b4071710 0x7f33b413b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:28.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.206+0000 7f33bbc00640 1 -- 172.21.15.159:0/4142408508 >> 172.21.15.159:0/4142408508 conn(0x7f33b406bba0 msgr2=0x7f33b4072550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:28.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.206+0000 7f33bbc00640 1 -- 172.21.15.159:0/4142408508 shutdown_connections 2024-04-04T06:10:28.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.206+0000 7f33bbc00640 1 -- 172.21.15.159:0/4142408508 wait complete. 2024-04-04T06:10:28.230 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-04T06:10:28.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.427+0000 7fa563a00640 1 Processor -- start 2024-04-04T06:10:28.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.427+0000 7fa563a00640 1 -- start start 2024-04-04T06:10:28.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.428+0000 7fa563a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa55c0d0280 0x7fa55c0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:28.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.429+0000 7fa563a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa55c141c10 0x7fa55c071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:28.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.429+0000 7fa563a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa55c15ec80 con 0x7fa55c071eb0 2024-04-04T06:10:28.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.429+0000 7fa563a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fa55c0d0ba0 con 0x7fa55c0d0280 2024-04-04T06:10:28.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.429+0000 7fa563a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa55c15ef20 con 0x7fa55c141c10 2024-04-04T06:10:28.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.429+0000 7fa561000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa55c0d0280 0x7fa55c0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:28.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.429+0000 7fa561000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa55c0d0280 0x7fa55c0d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41952/0 (socket says 172.21.15.159:41952) 2024-04-04T06:10:28.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.429+0000 7fa561000640 1 -- 172.21.15.159:0/3312085403 learned_addr learned my addr 172.21.15.159:0/3312085403 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:28.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.429+0000 7fa561000640 1 -- 172.21.15.159:0/3312085403 >> v2:172.21.15.178:3300/0 conn(0x7fa55c141c10 msgr2=0x7fa55c071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:28.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.429+0000 7fa55be00640 1 --2- 172.21.15.159:0/3312085403 >> v2:172.21.15.178:3300/0 conn(0x7fa55c141c10 0x7fa55c071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:28.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.429+0000 7fa561000640 1 --2- 172.21.15.159:0/3312085403 >> v2:172.21.15.178:3300/0 conn(0x7fa55c141c10 0x7fa55c071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:28.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.429+0000 7fa561000640 1 -- 172.21.15.159:0/3312085403 >> v1:172.21.15.178:6789/0 conn(0x7fa55c071eb0 legacy=0x7fa55c06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:28.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.429+0000 7fa561000640 1 -- 172.21.15.159:0/3312085403 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55c15f200 con 0x7fa55c0d0280 2024-04-04T06:10:28.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.431+0000 7fa561000640 1 --2- 172.21.15.159:0/3312085403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa55c0d0280 0x7fa55c0d0660 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0b5270 tx=0x7fa55002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f235f11fa1a7a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:28.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.431+0000 7fa55b400640 1 -- 172.21.15.159:0/3312085403 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550035020 con 0x7fa55c0d0280 2024-04-04T06:10:28.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.431+0000 7fa55b400640 1 -- 172.21.15.159:0/3312085403 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa550031870 con 0x7fa55c0d0280 2024-04-04T06:10:28.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.431+0000 7fa55b400640 1 -- 172.21.15.159:0/3312085403 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550031c40 con 0x7fa55c0d0280 2024-04-04T06:10:28.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.432+0000 7fa563a00640 1 -- 172.21.15.159:0/3312085403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa55c0d0280 msgr2=0x7fa55c0d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:28.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.432+0000 7fa563a00640 1 --2- 172.21.15.159:0/3312085403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa55c0d0280 0x7fa55c0d0660 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0b5270 tx=0x7fa55002b910 comp rx=0 tx=0).stop 2024-04-04T06:10:28.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.432+0000 7fa563a00640 1 -- 172.21.15.159:0/3312085403 shutdown_connections 2024-04-04T06:10:28.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.432+0000 7fa563a00640 1 --2- 172.21.15.159:0/3312085403 >> v2:172.21.15.178:3300/0 conn(0x7fa55c141c10 0x7fa55c071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:28.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.432+0000 7fa563a00640 1 --2- 172.21.15.159:0/3312085403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa55c0d0280 0x7fa55c0d0660 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:28.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.432+0000 7fa563a00640 1 -- 172.21.15.159:0/3312085403 >> 172.21.15.159:0/3312085403 conn(0x7fa55c06b750 msgr2=0x7fa55c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:28.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.432+0000 7fa563a00640 1 -- 172.21.15.159:0/3312085403 shutdown_connections 2024-04-04T06:10:28.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.432+0000 7fa563a00640 1 -- 172.21.15.159:0/3312085403 wait complete. 2024-04-04T06:10:28.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.433+0000 7fa563a00640 1 Processor -- start 2024-04-04T06:10:28.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.434+0000 7fa563a00640 1 -- start start 2024-04-04T06:10:28.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.435+0000 7fa563a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa55c071eb0 0x7fa55c09b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:28.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.435+0000 7fa563a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa55c0d0280 0x7fa55c156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:28.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.435+0000 7fa563a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa55c141c10 0x7fa55c142960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:28.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.435+0000 7fa563a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c158e90 con 0x7fa55c071eb0 2024-04-04T06:10:28.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.435+0000 7fa563a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c15f7c0 con 0x7fa55c141c10 2024-04-04T06:10:28.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.435+0000 7fa563a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa55c145200 con 0x7fa55c0d0280 2024-04-04T06:10:28.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.435+0000 7fa561000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa55c071eb0 0x7fa55c09b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:28.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.435+0000 7fa561000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa55c071eb0 0x7fa55c09b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41960/0 (socket says 172.21.15.159:41960) 2024-04-04T06:10:28.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.435+0000 7fa561000640 1 -- 172.21.15.159:0/3730612117 learned_addr learned my addr 172.21.15.159:0/3730612117 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.435+0000 7fa561000640 1 -- 172.21.15.159:0/3730612117 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa55c0d0280 msgr2=0x7fa55c156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.435+0000 7fa561000640 1 --2- 172.21.15.159:0/3730612117 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa55c0d0280 0x7fa55c156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.435+0000 7fa561000640 1 -- 172.21.15.159:0/3730612117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa55c141c10 msgr2=0x7fa55c142960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.435+0000 7fa561000640 1 --2- 172.21.15.159:0/3730612117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa55c141c10 0x7fa55c142960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.435+0000 7fa561000640 1 -- 172.21.15.159:0/3730612117 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55c1454c0 con 0x7fa55c071eb0 2024-04-04T06:10:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.436+0000 7fa561000640 1 --2- 172.21.15.159:0/3730612117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa55c071eb0 0x7fa55c09b1e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa550000c00 tx=0x7fa5500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.436+0000 7fa558c00640 1 -- 172.21.15.159:0/3730612117 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550040070 con 0x7fa55c071eb0 2024-04-04T06:10:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.436+0000 7fa558c00640 1 -- 172.21.15.159:0/3730612117 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55003a420 con 0x7fa55c071eb0 2024-04-04T06:10:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.436+0000 7fa558c00640 1 -- 172.21.15.159:0/3730612117 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550035040 con 0x7fa55c071eb0 2024-04-04T06:10:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.436+0000 7fa563a00640 1 -- 172.21.15.159:0/3730612117 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55c1457a0 con 0x7fa55c071eb0 2024-04-04T06:10:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.436+0000 7fa563a00640 1 -- 172.21.15.159:0/3730612117 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa55c145d60 con 0x7fa55c071eb0 2024-04-04T06:10:28.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.437+0000 7fa563a00640 1 -- 172.21.15.159:0/3730612117 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa55c0685c0 con 0x7fa55c071eb0 2024-04-04T06:10:28.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.438+0000 7fa558c00640 1 -- 172.21.15.159:0/3730612117 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa55004b060 con 0x7fa55c071eb0 2024-04-04T06:10:28.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.439+0000 7fa558c00640 1 --2- 172.21.15.159:0/3730612117 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa53c06c8f0 0x7fa53c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:28.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.439+0000 7fa558c00640 1 -- 172.21.15.159:0/3730612117 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7fa5500ad7c0 con 0x7fa55c071eb0 2024-04-04T06:10:28.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.440+0000 7fa55be00640 1 --2- 172.21.15.159:0/3730612117 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa53c06c8f0 0x7fa53c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:28.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.441+0000 7fa55be00640 1 --2- 172.21.15.159:0/3730612117 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa53c06c8f0 0x7fa53c06edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa55c06e850 tx=0x7fa548008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:28.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.444+0000 7fa558c00640 1 -- 172.21.15.159:0/3730612117 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa55007bee0 con 0x7fa55c071eb0 2024-04-04T06:10:28.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.658+0000 7fa563a00640 1 -- 172.21.15.159:0/3730612117 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fa55c147a80 con 0x7fa55c071eb0 2024-04-04T06:10:28.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.663+0000 7fa558c00640 1 -- 172.21.15.159:0/3730612117 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7fa55007b7e0 con 0x7fa55c071eb0 2024-04-04T06:10:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.666+0000 7fa563a00640 1 -- 172.21.15.159:0/3730612117 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa53c06c8f0 msgr2=0x7fa53c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.666+0000 7fa563a00640 1 --2- 172.21.15.159:0/3730612117 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa53c06c8f0 0x7fa53c06edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa55c06e850 tx=0x7fa548008040 comp rx=0 tx=0).stop 2024-04-04T06:10:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.666+0000 7fa563a00640 1 -- 172.21.15.159:0/3730612117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa55c071eb0 msgr2=0x7fa55c09b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.666+0000 7fa563a00640 1 --2- 172.21.15.159:0/3730612117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa55c071eb0 0x7fa55c09b1e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa550000c00 tx=0x7fa5500046c0 comp rx=0 tx=0).stop 2024-04-04T06:10:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.667+0000 7fa563a00640 1 -- 172.21.15.159:0/3730612117 shutdown_connections 2024-04-04T06:10:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.667+0000 7fa563a00640 1 --2- 172.21.15.159:0/3730612117 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa53c06c8f0 0x7fa53c06edb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.667+0000 7fa563a00640 1 --2- 172.21.15.159:0/3730612117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa55c141c10 0x7fa55c142960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:28.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.667+0000 7fa563a00640 1 --2- 172.21.15.159:0/3730612117 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa55c0d0280 0x7fa55c156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:28.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.667+0000 7fa563a00640 1 --2- 172.21.15.159:0/3730612117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa55c071eb0 0x7fa55c09b1e0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:28.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.667+0000 7fa563a00640 1 -- 172.21.15.159:0/3730612117 >> 172.21.15.159:0/3730612117 conn(0x7fa55c06b750 msgr2=0x7fa55c160c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:28.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.667+0000 7fa563a00640 1 -- 172.21.15.159:0/3730612117 shutdown_connections 2024-04-04T06:10:28.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.667+0000 7fa563a00640 1 -- 172.21.15.159:0/3730612117 wait complete. 2024-04-04T06:10:28.679 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-04T06:10:28.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.895+0000 7fcb92e00640 1 Processor -- start 2024-04-04T06:10:28.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.895+0000 7fcb92e00640 1 -- start start 2024-04-04T06:10:28.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.897+0000 7fcb92e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fcb8c144ad0 0x7fcb8c0b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:28.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.897+0000 7fcb92e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb8c15fba0 con 0x7fcb8c15f710 2024-04-04T06:10:28.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.897+0000 7fcb92e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb8c144eb0 con 0x7fcb8c072f50 2024-04-04T06:10:28.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.897+0000 7fcb92e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fcb8c0cdad0 con 0x7fcb8c144ad0 2024-04-04T06:10:28.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.897+0000 7fcb91a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fcb8c144ad0 0x7fcb8c0b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:28.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.897+0000 7fcb91a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fcb8c144ad0 0x7fcb8c0b2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41970/0 (socket says 172.21.15.159:41970) 2024-04-04T06:10:28.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.897+0000 7fcb91a00640 1 -- 172.21.15.159:0/1899886709 learned_addr learned my addr 172.21.15.159:0/1899886709 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:28.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.897+0000 7fcb91a00640 1 -- 172.21.15.159:0/1899886709 >> v1:172.21.15.178:6789/0 conn(0x7fcb8c072f50 legacy=0x7fcb8c0cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:28.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.897+0000 7fcb91a00640 1 -- 172.21.15.159:0/1899886709 >> v1:172.21.15.159:6789/0 conn(0x7fcb8c15f710 legacy=0x7fcb8c072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:28.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.897+0000 7fcb91a00640 1 -- 172.21.15.159:0/1899886709 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8c0b36f0 con 0x7fcb8c144ad0 2024-04-04T06:10:28.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.899+0000 7fcb91a00640 1 --2- 172.21.15.159:0/1899886709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb8c144ad0 0x7fcb8c0b2ff0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009870 tx=0x7fcb8002dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500d81158cf6a96d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:28.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.899+0000 7fcb8be00640 1 -- 172.21.15.159:0/1899886709 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8003b070 con 0x7fcb8c144ad0 2024-04-04T06:10:28.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.899+0000 7fcb8be00640 1 -- 172.21.15.159:0/1899886709 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb8002e960 con 0x7fcb8c144ad0 2024-04-04T06:10:28.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.899+0000 7fcb8be00640 1 -- 172.21.15.159:0/1899886709 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8002ed30 con 0x7fcb8c144ad0 2024-04-04T06:10:28.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.900+0000 7fcb92e00640 1 -- 172.21.15.159:0/1899886709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb8c144ad0 msgr2=0x7fcb8c0b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:28.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.900+0000 7fcb92e00640 1 --2- 172.21.15.159:0/1899886709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb8c144ad0 0x7fcb8c0b2ff0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009870 tx=0x7fcb8002dbb0 comp rx=0 tx=0).stop 2024-04-04T06:10:28.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.900+0000 7fcb92e00640 1 -- 172.21.15.159:0/1899886709 shutdown_connections 2024-04-04T06:10:28.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.900+0000 7fcb92e00640 1 --2- 172.21.15.159:0/1899886709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb8c144ad0 0x7fcb8c0b2ff0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:28.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.900+0000 7fcb92e00640 1 -- 172.21.15.159:0/1899886709 >> 172.21.15.159:0/1899886709 conn(0x7fcb8c06b570 msgr2=0x7fcb8c06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:28.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.900+0000 7fcb92e00640 1 -- 172.21.15.159:0/1899886709 shutdown_connections 2024-04-04T06:10:28.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.900+0000 7fcb92e00640 1 -- 172.21.15.159:0/1899886709 wait complete. 2024-04-04T06:10:28.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.902+0000 7fcb92e00640 1 Processor -- start 2024-04-04T06:10:28.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.902+0000 7fcb92e00640 1 -- start start 2024-04-04T06:10:28.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.903+0000 7fcb92e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb8c072f50 0x7fcb8c137bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:28.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.903+0000 7fcb92e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb8c144ad0 0x7fcb8c15bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:28.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.903+0000 7fcb92e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb8c15f710 0x7fcb8c0b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:28.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.903+0000 7fcb92e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c0b3f90 con 0x7fcb8c072f50 2024-04-04T06:10:28.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.903+0000 7fcb92e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c0b3cb0 con 0x7fcb8c15f710 2024-04-04T06:10:28.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.903+0000 7fcb92e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcb8c0ce670 con 0x7fcb8c144ad0 2024-04-04T06:10:28.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.903+0000 7fcb91a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb8c072f50 0x7fcb8c137bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:28.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.903+0000 7fcb91a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb8c072f50 0x7fcb8c137bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41976/0 (socket says 172.21.15.159:41976) 2024-04-04T06:10:28.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.903+0000 7fcb91a00640 1 -- 172.21.15.159:0/3481479813 learned_addr learned my addr 172.21.15.159:0/3481479813 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:28.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.903+0000 7fcb91a00640 1 -- 172.21.15.159:0/3481479813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb8c144ad0 msgr2=0x7fcb8c15bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:28.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.903+0000 7fcb91a00640 1 --2- 172.21.15.159:0/3481479813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb8c144ad0 0x7fcb8c15bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:28.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.903+0000 7fcb91a00640 1 -- 172.21.15.159:0/3481479813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb8c15f710 msgr2=0x7fcb8c0b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:28.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.903+0000 7fcb91a00640 1 --2- 172.21.15.159:0/3481479813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb8c15f710 0x7fcb8c0b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:28.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.903+0000 7fcb91a00640 1 -- 172.21.15.159:0/3481479813 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8c138130 con 0x7fcb8c072f50 2024-04-04T06:10:28.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.904+0000 7fcb91a00640 1 --2- 172.21.15.159:0/3481479813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb8c072f50 0x7fcb8c137bf0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009870 tx=0x7fcb800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:28.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.904+0000 7fcb89600640 1 -- 172.21.15.159:0/3481479813 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8003b040 con 0x7fcb8c072f50 2024-04-04T06:10:28.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.904+0000 7fcb89600640 1 -- 172.21.15.159:0/3481479813 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb80046070 con 0x7fcb8c072f50 2024-04-04T06:10:28.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.904+0000 7fcb92e00640 1 -- 172.21.15.159:0/3481479813 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb8c15c5d0 con 0x7fcb8c072f50 2024-04-04T06:10:28.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.904+0000 7fcb89600640 1 -- 172.21.15.159:0/3481479813 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb80036d90 con 0x7fcb8c072f50 2024-04-04T06:10:28.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.904+0000 7fcb92e00640 1 -- 172.21.15.159:0/3481479813 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb8c0adb70 con 0x7fcb8c072f50 2024-04-04T06:10:28.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.905+0000 7fcb92e00640 1 -- 172.21.15.159:0/3481479813 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb8c075940 con 0x7fcb8c072f50 2024-04-04T06:10:28.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.906+0000 7fcb89600640 1 -- 172.21.15.159:0/3481479813 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb8004b460 con 0x7fcb8c072f50 2024-04-04T06:10:28.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.907+0000 7fcb89600640 1 --2- 172.21.15.159:0/3481479813 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcb6c06c8f0 0x7fcb6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:28.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.907+0000 7fcb89600640 1 -- 172.21.15.159:0/3481479813 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7fcb800adcb0 con 0x7fcb8c072f50 2024-04-04T06:10:28.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.908+0000 7fcb91000640 1 --2- 172.21.15.159:0/3481479813 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcb6c06c8f0 0x7fcb6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:28.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.909+0000 7fcb91000640 1 --2- 172.21.15.159:0/3481479813 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcb6c06c8f0 0x7fcb6c06edb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcb78000f60 tx=0x7fcb780079d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:28.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:28.912+0000 7fcb89600640 1 -- 172.21.15.159:0/3481479813 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcb8007c460 con 0x7fcb8c072f50 2024-04-04T06:10:29.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.185+0000 7fcb92e00640 1 -- 172.21.15.159:0/3481479813 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7fcb8c0d0330 con 0x7fcb8c072f50 2024-04-04T06:10:29.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.218+0000 7fcb89600640 1 -- 172.21.15.159:0/3481479813 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v248) ==== 169+0+0 (secure 0 0 0) 0x7fcb8007bd60 con 0x7fcb8c072f50 2024-04-04T06:10:29.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.222+0000 7fcb92e00640 1 -- 172.21.15.159:0/3481479813 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcb6c06c8f0 msgr2=0x7fcb6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:29.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.222+0000 7fcb92e00640 1 --2- 172.21.15.159:0/3481479813 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcb6c06c8f0 0x7fcb6c06edb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcb78000f60 tx=0x7fcb780079d0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.222+0000 7fcb92e00640 1 -- 172.21.15.159:0/3481479813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb8c072f50 msgr2=0x7fcb8c137bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:29.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.222+0000 7fcb92e00640 1 --2- 172.21.15.159:0/3481479813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb8c072f50 0x7fcb8c137bf0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009870 tx=0x7fcb800046c0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.222+0000 7fcb92e00640 1 -- 172.21.15.159:0/3481479813 shutdown_connections 2024-04-04T06:10:29.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.222+0000 7fcb92e00640 1 --2- 172.21.15.159:0/3481479813 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcb6c06c8f0 0x7fcb6c06edb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.222+0000 7fcb92e00640 1 --2- 172.21.15.159:0/3481479813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb8c15f710 0x7fcb8c0b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.222+0000 7fcb92e00640 1 --2- 172.21.15.159:0/3481479813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcb8c144ad0 0x7fcb8c15bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.222+0000 7fcb92e00640 1 --2- 172.21.15.159:0/3481479813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb8c072f50 0x7fcb8c137bf0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.222+0000 7fcb92e00640 1 -- 172.21.15.159:0/3481479813 >> 172.21.15.159:0/3481479813 conn(0x7fcb8c06b570 msgr2=0x7fcb8c15eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:29.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.222+0000 7fcb92e00640 1 -- 172.21.15.159:0/3481479813 shutdown_connections 2024-04-04T06:10:29.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.222+0000 7fcb92e00640 1 -- 172.21.15.159:0/3481479813 wait complete. 2024-04-04T06:10:29.235 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-04T06:10:29.235 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-04T06:10:29.235 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-04T06:10:29.236 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:10:29.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.452+0000 7fcfc4c00640 1 Processor -- start 2024-04-04T06:10:29.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.452+0000 7fcfc4c00640 1 -- start start 2024-04-04T06:10:29.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.454+0000 7fcfc4c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfc015fc50 0x7fcfc00cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:29.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.454+0000 7fcfc4c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfc00b3150 con 0x7fcfc00cc0b0 2024-04-04T06:10:29.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.454+0000 7fcfc4c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfc01600e0 con 0x7fcfc00cc490 2024-04-04T06:10:29.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.454+0000 7fcfc4c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc015c610 con 0x7fcfc015fc50 2024-04-04T06:10:29.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.454+0000 7fcfbf400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fcfc00cc0b0 0x7fcfc006dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:44022/0 (socket says 172.21.15.159:44022) 2024-04-04T06:10:29.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.454+0000 7fcfbf400640 1 -- 172.21.15.159:0/1933732340 learned_addr learned my addr 172.21.15.159:0/1933732340 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:29.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.454+0000 7fcfb7e00640 1 --2- 172.21.15.159:0/1933732340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfc015fc50 0x7fcfc00cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:29.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.454+0000 7fcfbea00640 1 -- 172.21.15.159:0/1933732340 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3494875967 0 0) 0x7fcfc00b3150 con 0x7fcfc00cc0b0 2024-04-04T06:10:29.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.455+0000 7fcfbea00640 1 -- 172.21.15.159:0/1933732340 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcfa4003520 con 0x7fcfc00cc0b0 2024-04-04T06:10:29.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.455+0000 7fcfbea00640 1 -- 172.21.15.159:0/1933732340 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3183492 0 0) 0x7fcfa4003520 con 0x7fcfc00cc0b0 2024-04-04T06:10:29.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.455+0000 7fcfbea00640 1 -- 172.21.15.159:0/1933732340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfc015fc50 msgr2=0x7fcfc00cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:29.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.455+0000 7fcfbea00640 1 --2- 172.21.15.159:0/1933732340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfc015fc50 0x7fcfc00cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.455+0000 7fcfbea00640 1 -- 172.21.15.159:0/1933732340 >> v1:172.21.15.178:6789/0 conn(0x7fcfc00cc490 legacy=0x7fcfc015ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:29.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.455+0000 7fcfbea00640 1 -- 172.21.15.159:0/1933732340 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc01603c0 con 0x7fcfc00cc0b0 2024-04-04T06:10:29.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.455+0000 7fcfbea00640 1 -- 172.21.15.159:0/1933732340 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fcfb0001670 con 0x7fcfc00cc0b0 2024-04-04T06:10:29.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.455+0000 7fcfbea00640 1 -- 172.21.15.159:0/1933732340 >> v1:172.21.15.159:6789/0 conn(0x7fcfc00cc0b0 legacy=0x7fcfc006dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:29.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.455+0000 7fcfbea00640 1 --2- 172.21.15.159:0/1933732340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfa40039e0 0x7fcfa4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:29.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.455+0000 7fcfbea00640 1 --2- 172.21.15.159:0/1933732340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa4004340 0x7fcfa4004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:29.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.455+0000 7fcfbea00640 1 --2- 172.21.15.159:0/1933732340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfa4005020 0x7fcfa400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:29.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.455+0000 7fcfbea00640 1 -- 172.21.15.159:0/1933732340 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc015c610 con 0x7fcfa4005020 2024-04-04T06:10:29.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.455+0000 7fcfbea00640 1 -- 172.21.15.159:0/1933732340 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa400bf00 con 0x7fcfa4004340 2024-04-04T06:10:29.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.455+0000 7fcfbea00640 1 -- 172.21.15.159:0/1933732340 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa400c0b0 con 0x7fcfa40039e0 2024-04-04T06:10:29.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.455+0000 7fcfbfe00640 1 --2- 172.21.15.159:0/1933732340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfa4005020 0x7fcfa400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:29.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.456+0000 7fcfbf400640 1 --2- 172.21.15.159:0/1933732340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfa40039e0 0x7fcfa4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:29.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.456+0000 7fcfbfe00640 1 -- 172.21.15.159:0/1933732340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfa40039e0 msgr2=0x7fcfa4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:29.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.456+0000 7fcfbfe00640 1 --2- 172.21.15.159:0/1933732340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfa40039e0 0x7fcfa4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.456+0000 7fcfbfe00640 1 -- 172.21.15.159:0/1933732340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa4004340 msgr2=0x7fcfa4004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:29.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.456+0000 7fcfbfe00640 1 --2- 172.21.15.159:0/1933732340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa4004340 0x7fcfa4004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.456+0000 7fcfbfe00640 1 -- 172.21.15.159:0/1933732340 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa4003520 con 0x7fcfa4005020 2024-04-04T06:10:29.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.456+0000 7fcfbf400640 1 --2- 172.21.15.159:0/1933732340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfa40039e0 0x7fcfa4003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:29.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.457+0000 7fcfbfe00640 1 --2- 172.21.15.159:0/1933732340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfa4005020 0x7fcfa400b9c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8001e50 tx=0x7fcfa4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6deaa288d735619 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:29.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.457+0000 7fcfbea00640 1 -- 172.21.15.159:0/1933732340 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8001a70 con 0x7fcfa4005020 2024-04-04T06:10:29.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.457+0000 7fcfbea00640 1 -- 172.21.15.159:0/1933732340 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb8003b70 con 0x7fcfa4005020 2024-04-04T06:10:29.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.457+0000 7fcfbea00640 1 -- 172.21.15.159:0/1933732340 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb80050d0 con 0x7fcfa4005020 2024-04-04T06:10:29.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.458+0000 7fcfc4c00640 1 -- 172.21.15.159:0/1933732340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfa4005020 msgr2=0x7fcfa400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:29.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.458+0000 7fcfc4c00640 1 --2- 172.21.15.159:0/1933732340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfa4005020 0x7fcfa400b9c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8001e50 tx=0x7fcfa4005830 comp rx=0 tx=0).stop 2024-04-04T06:10:29.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.458+0000 7fcfbfe00640 1 -- 172.21.15.159:0/1933732340 reap_dead start 2024-04-04T06:10:29.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.458+0000 7fcfc4c00640 1 -- 172.21.15.159:0/1933732340 shutdown_connections 2024-04-04T06:10:29.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.458+0000 7fcfc4c00640 1 -- 172.21.15.159:0/1933732340 >> 172.21.15.159:0/1933732340 conn(0x7fcfc00ccad0 msgr2=0x7fcfc00cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:29.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.458+0000 7fcfc4c00640 1 -- 172.21.15.159:0/1933732340 shutdown_connections 2024-04-04T06:10:29.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.458+0000 7fcfc4c00640 1 -- 172.21.15.159:0/1933732340 wait complete. 2024-04-04T06:10:29.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.460+0000 7fcfc4c00640 1 Processor -- start 2024-04-04T06:10:29.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.460+0000 7fcfc4c00640 1 -- start start 2024-04-04T06:10:29.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.461+0000 7fcfc4c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa4005020 0x7fcfc01302c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:29.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.461+0000 7fcfc4c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfc015df70 0x7fcfc013e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:29.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.461+0000 7fcfc4c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfc015ee60 0x7fcfc0141c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:29.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.461+0000 7fcfc4c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc00b5b40 con 0x7fcfc015ee60 2024-04-04T06:10:29.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.461+0000 7fcfc4c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0160980 con 0x7fcfa4005020 2024-04-04T06:10:29.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.461+0000 7fcfc4c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc00b5cc0 con 0x7fcfc015df70 2024-04-04T06:10:29.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.461+0000 7fcfbfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfc015ee60 0x7fcfc0141c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:29.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.461+0000 7fcfbfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfc015ee60 0x7fcfc0141c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42002/0 (socket says 172.21.15.159:42002) 2024-04-04T06:10:29.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.461+0000 7fcfbfe00640 1 -- 172.21.15.159:0/3206186642 learned_addr learned my addr 172.21.15.159:0/3206186642 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:29.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.461+0000 7fcfbf400640 1 --2- 172.21.15.159:0/3206186642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa4005020 0x7fcfc01302c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:29.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.461+0000 7fcfb7e00640 1 --2- 172.21.15.159:0/3206186642 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfc015df70 0x7fcfc013e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:29.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.462+0000 7fcfbfe00640 1 -- 172.21.15.159:0/3206186642 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfc015df70 msgr2=0x7fcfc013e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:29.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.462+0000 7fcfbfe00640 1 --2- 172.21.15.159:0/3206186642 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfc015df70 0x7fcfc013e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.462+0000 7fcfbfe00640 1 -- 172.21.15.159:0/3206186642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa4005020 msgr2=0x7fcfc01302c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:29.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.462+0000 7fcfbfe00640 1 --2- 172.21.15.159:0/3206186642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa4005020 0x7fcfc01302c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.462+0000 7fcfbfe00640 1 -- 172.21.15.159:0/3206186642 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc0143bf0 con 0x7fcfc015ee60 2024-04-04T06:10:29.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.462+0000 7fcfbf400640 1 --2- 172.21.15.159:0/3206186642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa4005020 0x7fcfc01302c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:29.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.462+0000 7fcfbfe00640 1 --2- 172.21.15.159:0/3206186642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfc015ee60 0x7fcfc0141c00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8001f70 tx=0x7fcfb80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:29.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.462+0000 7fcfb7400640 1 -- 172.21.15.159:0/3206186642 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8007db0 con 0x7fcfc015ee60 2024-04-04T06:10:29.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.462+0000 7fcfb7400640 1 -- 172.21.15.159:0/3206186642 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb8003970 con 0x7fcfc015ee60 2024-04-04T06:10:29.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.462+0000 7fcfb7400640 1 -- 172.21.15.159:0/3206186642 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8006d70 con 0x7fcfc015ee60 2024-04-04T06:10:29.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.462+0000 7fcfc4c00640 1 -- 172.21.15.159:0/3206186642 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc015f6c0 con 0x7fcfc015ee60 2024-04-04T06:10:29.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.462+0000 7fcfc4c00640 1 -- 172.21.15.159:0/3206186642 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc0144150 con 0x7fcfc015ee60 2024-04-04T06:10:29.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.464+0000 7fcfc4c00640 1 -- 172.21.15.159:0/3206186642 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfc0075940 con 0x7fcfc015ee60 2024-04-04T06:10:29.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.464+0000 7fcfb7400640 1 -- 172.21.15.159:0/3206186642 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcfb8041030 con 0x7fcfc015ee60 2024-04-04T06:10:29.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.465+0000 7fcfb7400640 1 --2- 172.21.15.159:0/3206186642 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcf9c06c920 0x7fcf9c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:29.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.465+0000 7fcfb7400640 1 -- 172.21.15.159:0/3206186642 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7fcfb80ac8b0 con 0x7fcfc015ee60 2024-04-04T06:10:29.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.466+0000 7fcfbf400640 1 --2- 172.21.15.159:0/3206186642 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcf9c06c920 0x7fcf9c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:29.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.467+0000 7fcfbf400640 1 --2- 172.21.15.159:0/3206186642 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcf9c06c920 0x7fcf9c06ede0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0001200 tx=0x7fcfb0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:29.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.472+0000 7fcfb7400640 1 -- 172.21.15.159:0/3206186642 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcfc0143bf0 con 0x7fcfc015ee60 2024-04-04T06:10:29.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.674+0000 7fcfc4c00640 1 -- 172.21.15.159:0/3206186642 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcfc0068ad0 con 0x7fcfc015ee60 2024-04-04T06:10:29.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.676+0000 7fcfb7400640 1 -- 172.21.15.159:0/3206186642 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+13032 (secure 0 0 0) 0x7fcfb8033020 con 0x7fcfc015ee60 2024-04-04T06:10:29.669 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:29.669 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":62,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:10:28.199235+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6834/962475038":"2024-04-05T06:10:25.183049+0000","172.21.15.159:6833/962475038":"2024-04-05T06:10:25.183049+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:10:29.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.679+0000 7fcfc4c00640 1 -- 172.21.15.159:0/3206186642 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcf9c06c920 msgr2=0x7fcf9c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:29.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.679+0000 7fcfc4c00640 1 --2- 172.21.15.159:0/3206186642 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcf9c06c920 0x7fcf9c06ede0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0001200 tx=0x7fcfb0009040 comp rx=0 tx=0).stop 2024-04-04T06:10:29.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.679+0000 7fcfc4c00640 1 -- 172.21.15.159:0/3206186642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfc015ee60 msgr2=0x7fcfc0141c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:29.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.679+0000 7fcfc4c00640 1 --2- 172.21.15.159:0/3206186642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfc015ee60 0x7fcfc0141c00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8001f70 tx=0x7fcfb80014a0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.679+0000 7fcfc4c00640 1 -- 172.21.15.159:0/3206186642 shutdown_connections 2024-04-04T06:10:29.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.679+0000 7fcfc4c00640 1 --2- 172.21.15.159:0/3206186642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfc015ee60 0x7fcfc0141c00 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.679+0000 7fcfc4c00640 1 --2- 172.21.15.159:0/3206186642 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcfc015df70 0x7fcfc013e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.679+0000 7fcfc4c00640 1 --2- 172.21.15.159:0/3206186642 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcf9c06c920 0x7fcf9c06ede0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.679+0000 7fcfc4c00640 1 --2- 172.21.15.159:0/3206186642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcfa4005020 0x7fcfc01302c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.679+0000 7fcfc4c00640 1 -- 172.21.15.159:0/3206186642 >> 172.21.15.159:0/3206186642 conn(0x7fcfc00ccad0 msgr2=0x7fcfc00cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:29.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.679+0000 7fcfc4c00640 1 -- 172.21.15.159:0/3206186642 shutdown_connections 2024-04-04T06:10:29.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.679+0000 7fcfc4c00640 1 -- 172.21.15.159:0/3206186642 wait complete. 2024-04-04T06:10:29.692 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:10:29.692 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:10:29.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.917+0000 7ff429a00640 1 Processor -- start 2024-04-04T06:10:29.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.917+0000 7ff429a00640 1 -- start start 2024-04-04T06:10:29.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.918+0000 7ff429a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff424153580 0x7ff4240d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:29.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.918+0000 7ff429a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4240afab0 con 0x7ff4240d07c0 2024-04-04T06:10:29.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.918+0000 7ff429a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff42415cc80 con 0x7ff4240af4b0 2024-04-04T06:10:29.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.918+0000 7ff429a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff4240d0c30 con 0x7ff424153580 2024-04-04T06:10:29.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.919+0000 7ff422000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7ff4240d07c0 0x7ff4240aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:44024/0 (socket says 172.21.15.159:44024) 2024-04-04T06:10:29.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.919+0000 7ff422000640 1 -- 172.21.15.159:0/2809915904 learned_addr learned my addr 172.21.15.159:0/2809915904 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:29.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.919+0000 7ff421600640 1 -- 172.21.15.159:0/2809915904 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2207736381 0 0) 0x7ff4240afab0 con 0x7ff4240d07c0 2024-04-04T06:10:29.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.919+0000 7ff422a00640 1 --2- 172.21.15.159:0/2809915904 >> v2:172.21.15.178:3300/0 conn(0x7ff424153580 0x7ff4240d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:29.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.919+0000 7ff421600640 1 -- 172.21.15.159:0/2809915904 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff408003520 con 0x7ff4240d07c0 2024-04-04T06:10:29.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.920+0000 7ff421600640 1 -- 172.21.15.159:0/2809915904 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1970400876 0 0) 0x7ff408003520 con 0x7ff4240d07c0 2024-04-04T06:10:29.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.920+0000 7ff421600640 1 -- 172.21.15.159:0/2809915904 >> v2:172.21.15.178:3300/0 conn(0x7ff424153580 msgr2=0x7ff4240d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:29.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.920+0000 7ff421600640 1 --2- 172.21.15.159:0/2809915904 >> v2:172.21.15.178:3300/0 conn(0x7ff424153580 0x7ff4240d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.920+0000 7ff421600640 1 -- 172.21.15.159:0/2809915904 >> v1:172.21.15.178:6789/0 conn(0x7ff4240af4b0 legacy=0x7ff4241600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:29.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.920+0000 7ff421600640 1 -- 172.21.15.159:0/2809915904 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42415da00 con 0x7ff4240d07c0 2024-04-04T06:10:29.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.920+0000 7ff421600640 1 -- 172.21.15.159:0/2809915904 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7ff410001670 con 0x7ff4240d07c0 2024-04-04T06:10:29.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.920+0000 7ff421600640 1 -- 172.21.15.159:0/2809915904 >> v1:172.21.15.159:6789/0 conn(0x7ff4240d07c0 legacy=0x7ff4240aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:29.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.920+0000 7ff421600640 1 --2- 172.21.15.159:0/2809915904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff408003a00 0x7ff408003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:29.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.920+0000 7ff421600640 1 --2- 172.21.15.159:0/2809915904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff408004800 0x7ff408004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:29.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.920+0000 7ff421600640 1 --2- 172.21.15.159:0/2809915904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff408005210 0x7ff40800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:29.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.920+0000 7ff421600640 1 -- 172.21.15.159:0/2809915904 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff4240d0c30 con 0x7ff408004800 2024-04-04T06:10:29.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.920+0000 7ff421600640 1 -- 172.21.15.159:0/2809915904 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff40800c160 con 0x7ff408003a00 2024-04-04T06:10:29.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.920+0000 7ff421600640 1 -- 172.21.15.159:0/2809915904 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff40800c360 con 0x7ff408005210 2024-04-04T06:10:29.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.920+0000 7ff422000640 1 --2- 172.21.15.159:0/2809915904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff408004800 0x7ff408004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:29.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.921+0000 7ff422000640 1 -- 172.21.15.159:0/2809915904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff408005210 msgr2=0x7ff40800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:29.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.921+0000 7ff422000640 1 --2- 172.21.15.159:0/2809915904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff408005210 0x7ff40800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.921+0000 7ff422000640 1 -- 172.21.15.159:0/2809915904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff408003a00 msgr2=0x7ff408003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:29.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.921+0000 7ff422000640 1 --2- 172.21.15.159:0/2809915904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff408003a00 0x7ff408003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.921+0000 7ff422000640 1 -- 172.21.15.159:0/2809915904 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff408003520 con 0x7ff408004800 2024-04-04T06:10:29.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.921+0000 7ff423400640 1 -- 172.21.15.159:0/2809915904 reap_dead start 2024-04-04T06:10:29.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.922+0000 7ff422000640 1 --2- 172.21.15.159:0/2809915904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff408004800 0x7ff408004c80 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff41000c0e0 tx=0x7ff408005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f746f958cc42068 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:29.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.922+0000 7ff421600640 1 -- 172.21.15.159:0/2809915904 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410007c40 con 0x7ff408004800 2024-04-04T06:10:29.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.922+0000 7ff421600640 1 -- 172.21.15.159:0/2809915904 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff410030070 con 0x7ff408004800 2024-04-04T06:10:29.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.922+0000 7ff421600640 1 -- 172.21.15.159:0/2809915904 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410005d70 con 0x7ff408004800 2024-04-04T06:10:29.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.923+0000 7ff429a00640 1 -- 172.21.15.159:0/2809915904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff408004800 msgr2=0x7ff408004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:29.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.923+0000 7ff429a00640 1 --2- 172.21.15.159:0/2809915904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff408004800 0x7ff408004c80 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff41000c0e0 tx=0x7ff408005830 comp rx=0 tx=0).stop 2024-04-04T06:10:29.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.923+0000 7ff429a00640 1 -- 172.21.15.159:0/2809915904 shutdown_connections 2024-04-04T06:10:29.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.923+0000 7ff429a00640 1 --2- 172.21.15.159:0/2809915904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff408004800 0x7ff408004c80 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.923+0000 7ff429a00640 1 -- 172.21.15.159:0/2809915904 >> 172.21.15.159:0/2809915904 conn(0x7ff42406b750 msgr2=0x7ff42406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:29.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.923+0000 7ff429a00640 1 -- 172.21.15.159:0/2809915904 shutdown_connections 2024-04-04T06:10:29.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.923+0000 7ff429a00640 1 -- 172.21.15.159:0/2809915904 wait complete. 2024-04-04T06:10:29.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.925+0000 7ff429a00640 1 Processor -- start 2024-04-04T06:10:29.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.925+0000 7ff429a00640 1 -- start start 2024-04-04T06:10:29.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.926+0000 7ff429a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff408004800 0x7ff4240cf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:29.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.926+0000 7ff429a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff42414b490 0x7ff4241f79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:29.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.926+0000 7ff429a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4241f7ee0 0x7ff424154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:29.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.926+0000 7ff429a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff42415d4e0 con 0x7ff42414b490 2024-04-04T06:10:29.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.926+0000 7ff429a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff42415d200 con 0x7ff4241f7ee0 2024-04-04T06:10:29.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.926+0000 7ff429a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff424153960 con 0x7ff408004800 2024-04-04T06:10:29.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.926+0000 7ff422000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff42414b490 0x7ff4241f79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:29.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.926+0000 7ff422000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff42414b490 0x7ff4241f79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42022/0 (socket says 172.21.15.159:42022) 2024-04-04T06:10:29.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.926+0000 7ff422000640 1 -- 172.21.15.159:0/3540745759 learned_addr learned my addr 172.21.15.159:0/3540745759 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:29.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.926+0000 7ff422a00640 1 --2- 172.21.15.159:0/3540745759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff408004800 0x7ff4240cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:29.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.926+0000 7ff423400640 1 --2- 172.21.15.159:0/3540745759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4241f7ee0 0x7ff424154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:29.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.927+0000 7ff422000640 1 -- 172.21.15.159:0/3540745759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff408004800 msgr2=0x7ff4240cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:29.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.927+0000 7ff422000640 1 --2- 172.21.15.159:0/3540745759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff408004800 0x7ff4240cf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.927+0000 7ff422000640 1 -- 172.21.15.159:0/3540745759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4241f7ee0 msgr2=0x7ff424154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:29.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.927+0000 7ff422000640 1 --2- 172.21.15.159:0/3540745759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4241f7ee0 0x7ff424154d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:29.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.927+0000 7ff422000640 1 -- 172.21.15.159:0/3540745759 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42409a4c0 con 0x7ff42414b490 2024-04-04T06:10:29.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.927+0000 7ff422000640 1 --2- 172.21.15.159:0/3540745759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff42414b490 0x7ff4241f79a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff41000c200 tx=0x7ff41002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:29.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.927+0000 7ff41aa00640 1 -- 172.21.15.159:0/3540745759 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410046070 con 0x7ff42414b490 2024-04-04T06:10:29.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.927+0000 7ff41aa00640 1 -- 172.21.15.159:0/3540745759 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff410000ed0 con 0x7ff42414b490 2024-04-04T06:10:29.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.927+0000 7ff429a00640 1 -- 172.21.15.159:0/3540745759 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4240cd290 con 0x7ff42414b490 2024-04-04T06:10:29.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.927+0000 7ff41aa00640 1 -- 172.21.15.159:0/3540745759 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410004e10 con 0x7ff42414b490 2024-04-04T06:10:29.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.928+0000 7ff429a00640 1 -- 172.21.15.159:0/3540745759 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4240cd850 con 0x7ff42414b490 2024-04-04T06:10:29.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.929+0000 7ff429a00640 1 -- 172.21.15.159:0/3540745759 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4240ae140 con 0x7ff42414b490 2024-04-04T06:10:29.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.930+0000 7ff41aa00640 1 -- 172.21.15.159:0/3540745759 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff410007070 con 0x7ff42414b490 2024-04-04T06:10:29.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.930+0000 7ff41aa00640 1 --2- 172.21.15.159:0/3540745759 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff40406c920 0x7ff40406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:29.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.931+0000 7ff41aa00640 1 -- 172.21.15.159:0/3540745759 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7ff4100acc50 con 0x7ff42414b490 2024-04-04T06:10:29.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.931+0000 7ff422a00640 1 --2- 172.21.15.159:0/3540745759 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff40406c920 0x7ff40406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:29.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.932+0000 7ff422a00640 1 --2- 172.21.15.159:0/3540745759 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff40406c920 0x7ff40406ede0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff414009ad0 tx=0x7ff414008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:29.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:29.935+0000 7ff41aa00640 1 -- 172.21.15.159:0/3540745759 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff4240ae140 con 0x7ff42414b490 2024-04-04T06:10:30.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.135+0000 7ff429a00640 1 -- 172.21.15.159:0/3540745759 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff4240685c0 con 0x7ff42414b490 2024-04-04T06:10:30.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.136+0000 7ff41aa00640 1 -- 172.21.15.159:0/3540745759 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) ==== 93+0+10 (secure 0 0 0) 0x7ff410081030 con 0x7ff42414b490 2024-04-04T06:10:30.129 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:10:30.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.138+0000 7ff429a00640 1 -- 172.21.15.159:0/3540745759 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff40406c920 msgr2=0x7ff40406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:30.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.138+0000 7ff429a00640 1 --2- 172.21.15.159:0/3540745759 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff40406c920 0x7ff40406ede0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff414009ad0 tx=0x7ff414008040 comp rx=0 tx=0).stop 2024-04-04T06:10:30.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.138+0000 7ff429a00640 1 -- 172.21.15.159:0/3540745759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff42414b490 msgr2=0x7ff4241f79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:30.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.138+0000 7ff429a00640 1 --2- 172.21.15.159:0/3540745759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff42414b490 0x7ff4241f79a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff41000c200 tx=0x7ff41002e070 comp rx=0 tx=0).stop 2024-04-04T06:10:30.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.139+0000 7ff429a00640 1 -- 172.21.15.159:0/3540745759 shutdown_connections 2024-04-04T06:10:30.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.139+0000 7ff429a00640 1 --2- 172.21.15.159:0/3540745759 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff40406c920 0x7ff40406ede0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:30.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.139+0000 7ff429a00640 1 --2- 172.21.15.159:0/3540745759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4241f7ee0 0x7ff424154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:30.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.139+0000 7ff429a00640 1 --2- 172.21.15.159:0/3540745759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff42414b490 0x7ff4241f79a0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:30.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.139+0000 7ff429a00640 1 --2- 172.21.15.159:0/3540745759 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff408004800 0x7ff4240cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:30.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.139+0000 7ff429a00640 1 -- 172.21.15.159:0/3540745759 >> 172.21.15.159:0/3540745759 conn(0x7ff42406b750 msgr2=0x7ff4240b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:30.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.139+0000 7ff429a00640 1 -- 172.21.15.159:0/3540745759 shutdown_connections 2024-04-04T06:10:30.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.139+0000 7ff429a00640 1 -- 172.21.15.159:0/3540745759 wait complete. 2024-04-04T06:10:30.151 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-04T06:10:30.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.371+0000 7f6538600640 1 Processor -- start 2024-04-04T06:10:30.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.371+0000 7f6538600640 1 -- start start 2024-04-04T06:10:30.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.373+0000 7f6538600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f6530167450 0x7f6530169840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:30.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.373+0000 7f6538600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f653015ec80 con 0x7f6530141c10 2024-04-04T06:10:30.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.373+0000 7f6538600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f653015ef20 con 0x7f65300d0280 2024-04-04T06:10:30.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.373+0000 7f6538600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f653015f200 con 0x7f6530167450 2024-04-04T06:10:30.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.373+0000 7f6535200640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f6530141c10 0x7f6530071970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:44038/0 (socket says 172.21.15.159:44038) 2024-04-04T06:10:30.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.373+0000 7f6535200640 1 -- 172.21.15.159:0/1116682963 learned_addr learned my addr 172.21.15.159:0/1116682963 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:30.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.373+0000 7f6536600640 1 --2- 172.21.15.159:0/1116682963 >> v2:172.21.15.159:3300/0 conn(0x7f6530167450 0x7f6530169840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:30.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.374+0000 7f652fe00640 1 -- 172.21.15.159:0/1116682963 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2484846300 0 0) 0x7f653015ec80 con 0x7f6530141c10 2024-04-04T06:10:30.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.374+0000 7f652fe00640 1 -- 172.21.15.159:0/1116682963 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6518003520 con 0x7f6530141c10 2024-04-04T06:10:30.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.374+0000 7f6536600640 1 -- 172.21.15.159:0/1116682963 >> v1:172.21.15.178:6789/0 conn(0x7f65300d0280 legacy=0x7f65300d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:30.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.374+0000 7f6536600640 1 -- 172.21.15.159:0/1116682963 >> v1:172.21.15.159:6789/0 conn(0x7f6530141c10 legacy=0x7f6530071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:30.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.374+0000 7f6536600640 1 -- 172.21.15.159:0/1116682963 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653015f4e0 con 0x7f6530167450 2024-04-04T06:10:30.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.375+0000 7f6536600640 1 --2- 172.21.15.159:0/1116682963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6530167450 0x7f6530169840 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f6528009d90 tx=0x7f652802bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38908be0515d0db7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:30.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.375+0000 7f652fe00640 1 -- 172.21.15.159:0/1116682963 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528039070 con 0x7f6530167450 2024-04-04T06:10:30.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.375+0000 7f652fe00640 1 -- 172.21.15.159:0/1116682963 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f652802ccf0 con 0x7f6530167450 2024-04-04T06:10:30.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.375+0000 7f652fe00640 1 -- 172.21.15.159:0/1116682963 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528034660 con 0x7f6530167450 2024-04-04T06:10:30.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.376+0000 7f6538600640 1 -- 172.21.15.159:0/1116682963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6530167450 msgr2=0x7f6530169840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:30.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.376+0000 7f6538600640 1 --2- 172.21.15.159:0/1116682963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6530167450 0x7f6530169840 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f6528009d90 tx=0x7f652802bf40 comp rx=0 tx=0).stop 2024-04-04T06:10:30.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.376+0000 7f6538600640 1 -- 172.21.15.159:0/1116682963 shutdown_connections 2024-04-04T06:10:30.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.376+0000 7f6538600640 1 --2- 172.21.15.159:0/1116682963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6530167450 0x7f6530169840 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:30.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.376+0000 7f6538600640 1 -- 172.21.15.159:0/1116682963 >> 172.21.15.159:0/1116682963 conn(0x7f653006b750 msgr2=0x7f653006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:30.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.376+0000 7f6538600640 1 -- 172.21.15.159:0/1116682963 shutdown_connections 2024-04-04T06:10:30.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.376+0000 7f6538600640 1 -- 172.21.15.159:0/1116682963 wait complete. 2024-04-04T06:10:30.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.378+0000 7f6538600640 1 Processor -- start 2024-04-04T06:10:30.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.378+0000 7f6538600640 1 -- start start 2024-04-04T06:10:30.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.379+0000 7f6538600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65300d0280 0x7f6530155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:30.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.379+0000 7f6538600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6530141c10 0x7f6530152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:30.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.379+0000 7f6538600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6530167450 0x7f6530098a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:30.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.379+0000 7f6538600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6530158e90 con 0x7f65300d0280 2024-04-04T06:10:30.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.379+0000 7f6538600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f653006f340 con 0x7f6530141c10 2024-04-04T06:10:30.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.379+0000 7f6538600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f65301563b0 con 0x7f6530167450 2024-04-04T06:10:30.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.379+0000 7f6535c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65300d0280 0x7f6530155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:30.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.379+0000 7f6535c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65300d0280 0x7f6530155110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42042/0 (socket says 172.21.15.159:42042) 2024-04-04T06:10:30.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.379+0000 7f6535c00640 1 -- 172.21.15.159:0/3679153892 learned_addr learned my addr 172.21.15.159:0/3679153892 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:30.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.380+0000 7f6535c00640 1 -- 172.21.15.159:0/3679153892 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6530167450 msgr2=0x7f6530098a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:30.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.380+0000 7f6535c00640 1 --2- 172.21.15.159:0/3679153892 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6530167450 0x7f6530098a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:30.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.380+0000 7f6535c00640 1 -- 172.21.15.159:0/3679153892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6530141c10 msgr2=0x7f6530152a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:30.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.380+0000 7f6535c00640 1 --2- 172.21.15.159:0/3679153892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6530141c10 0x7f6530152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:30.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.380+0000 7f6535c00640 1 -- 172.21.15.159:0/3679153892 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65301557a0 con 0x7f65300d0280 2024-04-04T06:10:30.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.380+0000 7f6535c00640 1 --2- 172.21.15.159:0/3679153892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65300d0280 0x7f6530155110 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f652400c430 tx=0x7f652400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:30.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.380+0000 7f652d600640 1 -- 172.21.15.159:0/3679153892 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65240052d0 con 0x7f65300d0280 2024-04-04T06:10:30.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.381+0000 7f652d600640 1 -- 172.21.15.159:0/3679153892 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6524005490 con 0x7f65300d0280 2024-04-04T06:10:30.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.381+0000 7f6538600640 1 -- 172.21.15.159:0/3679153892 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6530155a80 con 0x7f65300d0280 2024-04-04T06:10:30.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.381+0000 7f652d600640 1 -- 172.21.15.159:0/3679153892 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6524007260 con 0x7f65300d0280 2024-04-04T06:10:30.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.381+0000 7f6538600640 1 -- 172.21.15.159:0/3679153892 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6530156040 con 0x7f65300d0280 2024-04-04T06:10:30.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.382+0000 7f6538600640 1 -- 172.21.15.159:0/3679153892 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65300685c0 con 0x7f65300d0280 2024-04-04T06:10:30.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.383+0000 7f652d600640 1 -- 172.21.15.159:0/3679153892 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6524007400 con 0x7f65300d0280 2024-04-04T06:10:30.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.384+0000 7f652d600640 1 --2- 172.21.15.159:0/3679153892 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f651006c8f0 0x7f651006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:30.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.384+0000 7f652d600640 1 -- 172.21.15.159:0/3679153892 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7f652408ae50 con 0x7f65300d0280 2024-04-04T06:10:30.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.385+0000 7f6535200640 1 --2- 172.21.15.159:0/3679153892 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f651006c8f0 0x7f651006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:30.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.386+0000 7f6535200640 1 --2- 172.21.15.159:0/3679153892 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f651006c8f0 0x7f651006edb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f65301f7b90 tx=0x7f651c007930 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:30.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.388+0000 7f652d600640 1 -- 172.21.15.159:0/3679153892 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f65240594e0 con 0x7f65300d0280 2024-04-04T06:10:30.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:30.649+0000 7f6538600640 1 -- 172.21.15.159:0/3679153892 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f65300cfdb0 con 0x7f65300d0280 2024-04-04T06:10:31.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.230+0000 7f652d600640 1 -- 172.21.15.159:0/3679153892 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v250) ==== 169+0+0 (secure 0 0 0) 0x7f6524058de0 con 0x7f65300d0280 2024-04-04T06:10:31.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.234+0000 7f6538600640 1 -- 172.21.15.159:0/3679153892 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f651006c8f0 msgr2=0x7f651006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:31.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.234+0000 7f6538600640 1 --2- 172.21.15.159:0/3679153892 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f651006c8f0 0x7f651006edb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f65301f7b90 tx=0x7f651c007930 comp rx=0 tx=0).stop 2024-04-04T06:10:31.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.234+0000 7f6538600640 1 -- 172.21.15.159:0/3679153892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65300d0280 msgr2=0x7f6530155110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:31.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.234+0000 7f6538600640 1 --2- 172.21.15.159:0/3679153892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65300d0280 0x7f6530155110 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f652400c430 tx=0x7f652400c8f0 comp rx=0 tx=0).stop 2024-04-04T06:10:31.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.235+0000 7f6538600640 1 -- 172.21.15.159:0/3679153892 shutdown_connections 2024-04-04T06:10:31.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.235+0000 7f6538600640 1 --2- 172.21.15.159:0/3679153892 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6530167450 0x7f6530098a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:31.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.235+0000 7f6538600640 1 --2- 172.21.15.159:0/3679153892 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f651006c8f0 0x7f651006edb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:31.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.235+0000 7f6538600640 1 --2- 172.21.15.159:0/3679153892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6530141c10 0x7f6530152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:31.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.235+0000 7f6538600640 1 --2- 172.21.15.159:0/3679153892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65300d0280 0x7f6530155110 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:31.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.235+0000 7f6538600640 1 -- 172.21.15.159:0/3679153892 >> 172.21.15.159:0/3679153892 conn(0x7f653006b750 msgr2=0x7f65300b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:31.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.235+0000 7f6538600640 1 -- 172.21.15.159:0/3679153892 shutdown_connections 2024-04-04T06:10:31.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.235+0000 7f6538600640 1 -- 172.21.15.159:0/3679153892 wait complete. 2024-04-04T06:10:31.248 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-04T06:10:31.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.467+0000 7efd01600640 1 Processor -- start 2024-04-04T06:10:31.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.467+0000 7efd01600640 1 -- start start 2024-04-04T06:10:31.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.469+0000 7efd01600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7efcfc0d0990 0x7efcfc0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:31.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.469+0000 7efd01600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcfc0b40b0 con 0x7efcfc155930 2024-04-04T06:10:31.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.469+0000 7efd01600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcfc0b4390 con 0x7efcfc155550 2024-04-04T06:10:31.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.469+0000 7efd01600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7efcfc0b4670 con 0x7efcfc0d0990 2024-04-04T06:10:31.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.469+0000 7efcfb400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7efcfc0d0990 0x7efcfc0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:31.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.469+0000 7efcfb400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7efcfc0d0990 0x7efcfc0aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42046/0 (socket says 172.21.15.159:42046) 2024-04-04T06:10:31.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.469+0000 7efcfb400640 1 -- 172.21.15.159:0/3389743256 learned_addr learned my addr 172.21.15.159:0/3389743256 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:31.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.469+0000 7efcfa000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7efcfc155930 0x7efcfc0d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:44044/0 (socket says 172.21.15.159:44044) 2024-04-04T06:10:31.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.469+0000 7efcfb400640 1 -- 172.21.15.159:0/3389743256 >> v1:172.21.15.178:6789/0 conn(0x7efcfc155550 legacy=0x7efcfc15f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:31.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.469+0000 7efcfb400640 1 -- 172.21.15.159:0/3389743256 >> v1:172.21.15.159:6789/0 conn(0x7efcfc155930 legacy=0x7efcfc0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:31.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.469+0000 7efcfb400640 1 -- 172.21.15.159:0/3389743256 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcfc0b4950 con 0x7efcfc0d0990 2024-04-04T06:10:31.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.470+0000 7efcfb400640 1 --2- 172.21.15.159:0/3389743256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcfc0d0990 0x7efcfc0aede0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009e50 tx=0x7efcf402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320e54254d3439fc server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:31.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.471+0000 7efcf9600640 1 -- 172.21.15.159:0/3389743256 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf402cb10 con 0x7efcfc0d0990 2024-04-04T06:10:31.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.471+0000 7efcf9600640 1 -- 172.21.15.159:0/3389743256 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf402ccb0 con 0x7efcfc0d0990 2024-04-04T06:10:31.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.471+0000 7efcf9600640 1 -- 172.21.15.159:0/3389743256 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf402ce60 con 0x7efcfc0d0990 2024-04-04T06:10:31.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.471+0000 7efd01600640 1 -- 172.21.15.159:0/3389743256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcfc0d0990 msgr2=0x7efcfc0aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:31.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.471+0000 7efd01600640 1 --2- 172.21.15.159:0/3389743256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcfc0d0990 0x7efcfc0aede0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009e50 tx=0x7efcf402bf70 comp rx=0 tx=0).stop 2024-04-04T06:10:31.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.471+0000 7efd01600640 1 -- 172.21.15.159:0/3389743256 shutdown_connections 2024-04-04T06:10:31.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.471+0000 7efd01600640 1 --2- 172.21.15.159:0/3389743256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcfc0d0990 0x7efcfc0aede0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:31.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.471+0000 7efd01600640 1 -- 172.21.15.159:0/3389743256 >> 172.21.15.159:0/3389743256 conn(0x7efcfc06b750 msgr2=0x7efcfc06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:31.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.472+0000 7efd01600640 1 -- 172.21.15.159:0/3389743256 shutdown_connections 2024-04-04T06:10:31.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.472+0000 7efd01600640 1 -- 172.21.15.159:0/3389743256 wait complete. 2024-04-04T06:10:31.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.473+0000 7efd01600640 1 Processor -- start 2024-04-04T06:10:31.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.473+0000 7efd01600640 1 -- start start 2024-04-04T06:10:31.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.474+0000 7efd01600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcfc0d0990 0x7efcfc0998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:31.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.474+0000 7efd01600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efcfc155550 0x7efcfc150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:31.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.474+0000 7efd01600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efcfc155930 0x7efcfc15a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:31.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.474+0000 7efd01600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc15bba0 con 0x7efcfc0d0990 2024-04-04T06:10:31.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.474+0000 7efd01600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc15ba20 con 0x7efcfc155930 2024-04-04T06:10:31.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.474+0000 7efd01600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efcfc0b3cf0 con 0x7efcfc155550 2024-04-04T06:10:31.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.474+0000 7efcfaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcfc0d0990 0x7efcfc0998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:31.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.474+0000 7efcfaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcfc0d0990 0x7efcfc0998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42052/0 (socket says 172.21.15.159:42052) 2024-04-04T06:10:31.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.474+0000 7efcfaa00640 1 -- 172.21.15.159:0/1662670245 learned_addr learned my addr 172.21.15.159:0/1662670245 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:31.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.475+0000 7efcfaa00640 1 -- 172.21.15.159:0/1662670245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efcfc155550 msgr2=0x7efcfc150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:31.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.475+0000 7efcfaa00640 1 --2- 172.21.15.159:0/1662670245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efcfc155550 0x7efcfc150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:31.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.475+0000 7efcfaa00640 1 -- 172.21.15.159:0/1662670245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efcfc155930 msgr2=0x7efcfc15a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:31.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.475+0000 7efcfaa00640 1 --2- 172.21.15.159:0/1662670245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efcfc155930 0x7efcfc15a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:31.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.475+0000 7efcfaa00640 1 -- 172.21.15.159:0/1662670245 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcfc15ca40 con 0x7efcfc0d0990 2024-04-04T06:10:31.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.475+0000 7efcfaa00640 1 --2- 172.21.15.159:0/1662670245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcfc0d0990 0x7efcfc0998e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7efcec00c480 tx=0x7efcec00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:31.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.475+0000 7efcf2a00640 1 -- 172.21.15.159:0/1662670245 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec0052d0 con 0x7efcfc0d0990 2024-04-04T06:10:31.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.475+0000 7efcf2a00640 1 -- 172.21.15.159:0/1662670245 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcec005470 con 0x7efcfc0d0990 2024-04-04T06:10:31.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.475+0000 7efd01600640 1 -- 172.21.15.159:0/1662670245 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcfc15cce0 con 0x7efcfc0d0990 2024-04-04T06:10:31.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.475+0000 7efcf2a00640 1 -- 172.21.15.159:0/1662670245 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec012870 con 0x7efcfc0d0990 2024-04-04T06:10:31.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.476+0000 7efd01600640 1 -- 172.21.15.159:0/1662670245 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcfc15d2a0 con 0x7efcfc0d0990 2024-04-04T06:10:31.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.477+0000 7efd01600640 1 -- 172.21.15.159:0/1662670245 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcfc15f800 con 0x7efcfc0d0990 2024-04-04T06:10:31.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.478+0000 7efcf2a00640 1 -- 172.21.15.159:0/1662670245 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efcec012a10 con 0x7efcfc0d0990 2024-04-04T06:10:31.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.478+0000 7efcf2a00640 1 --2- 172.21.15.159:0/1662670245 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efcdc06c920 0x7efcdc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:31.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.479+0000 7efcf2a00640 1 -- 172.21.15.159:0/1662670245 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7efcec08b470 con 0x7efcfc0d0990 2024-04-04T06:10:31.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.479+0000 7efcfa000640 1 --2- 172.21.15.159:0/1662670245 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efcdc06c920 0x7efcdc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:31.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.480+0000 7efcfa000640 1 --2- 172.21.15.159:0/1662670245 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efcdc06c920 0x7efcdc06ede0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0b38a0 tx=0x7efce80079c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:31.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.483+0000 7efcf2a00640 1 -- 172.21.15.159:0/1662670245 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7efcfc15f800 con 0x7efcfc0d0990 2024-04-04T06:10:31.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.668+0000 7efd01600640 1 -- 172.21.15.159:0/1662670245 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7efcfc0685c0 con 0x7efcfc0d0990 2024-04-04T06:10:31.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.675+0000 7efcf2a00640 1 -- 172.21.15.159:0/1662670245 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7efcec0595e0 con 0x7efcfc0d0990 2024-04-04T06:10:31.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.677+0000 7efd01600640 1 -- 172.21.15.159:0/1662670245 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efcdc06c920 msgr2=0x7efcdc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:31.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.677+0000 7efd01600640 1 --2- 172.21.15.159:0/1662670245 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efcdc06c920 0x7efcdc06ede0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0b38a0 tx=0x7efce80079c0 comp rx=0 tx=0).stop 2024-04-04T06:10:31.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.677+0000 7efd01600640 1 -- 172.21.15.159:0/1662670245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcfc0d0990 msgr2=0x7efcfc0998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:31.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.677+0000 7efd01600640 1 --2- 172.21.15.159:0/1662670245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcfc0d0990 0x7efcfc0998e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7efcec00c480 tx=0x7efcec00c940 comp rx=0 tx=0).stop 2024-04-04T06:10:31.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.677+0000 7efd01600640 1 -- 172.21.15.159:0/1662670245 shutdown_connections 2024-04-04T06:10:31.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.677+0000 7efd01600640 1 --2- 172.21.15.159:0/1662670245 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efcdc06c920 0x7efcdc06ede0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:31.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.677+0000 7efd01600640 1 --2- 172.21.15.159:0/1662670245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efcfc155930 0x7efcfc15a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:31.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.677+0000 7efd01600640 1 --2- 172.21.15.159:0/1662670245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efcfc155550 0x7efcfc150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:31.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.677+0000 7efd01600640 1 --2- 172.21.15.159:0/1662670245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcfc0d0990 0x7efcfc0998e0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:31.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.677+0000 7efd01600640 1 -- 172.21.15.159:0/1662670245 >> 172.21.15.159:0/1662670245 conn(0x7efcfc06b750 msgr2=0x7efcfc0ae1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:31.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.677+0000 7efd01600640 1 -- 172.21.15.159:0/1662670245 shutdown_connections 2024-04-04T06:10:31.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:31.677+0000 7efd01600640 1 -- 172.21.15.159:0/1662670245 wait complete. 2024-04-04T06:10:31.689 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-04T06:10:31.689 DEBUG:teuthology.orchestra.run.smithi159:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:10:31.790 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:10:31.909 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-04T06:10:31.909 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:10:31.909 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:10:31.964 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:10:31.965 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:10:31.965 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:10:32.022 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:10:32.022 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:10:32.022 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:10:32.081 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:10:32.081 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:10:32.081 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:10:32.139 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:10:32.139 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-04T06:10:32.139 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:10:32.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.365+0000 7f5ab8a00640 1 Processor -- start 2024-04-04T06:10:32.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.365+0000 7f5ab8a00640 1 -- start start 2024-04-04T06:10:32.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.366+0000 7f5ab8a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ab415fc50 0x7f5ab40cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:32.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.366+0000 7f5ab8a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5ab40cc490 0x7f5ab40d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:32.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.366+0000 7f5ab8a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ab40b3150 con 0x7f5ab40cc0b0 2024-04-04T06:10:32.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.366+0000 7f5ab8a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f5ab415c610 con 0x7f5ab40cc490 2024-04-04T06:10:32.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.366+0000 7f5ab8a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab41600e0 con 0x7f5ab415fc50 2024-04-04T06:10:32.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.367+0000 7f5ab3e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5ab40cc490 0x7f5ab40d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:32.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.367+0000 7f5ab3e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5ab40cc490 0x7f5ab40d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42066/0 (socket says 172.21.15.159:42066) 2024-04-04T06:10:32.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.367+0000 7f5ab3e00640 1 -- 172.21.15.159:0/576488499 learned_addr learned my addr 172.21.15.159:0/576488499 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:32.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.367+0000 7f5aaaa00640 1 --2- 172.21.15.159:0/576488499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ab415fc50 0x7f5ab40cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:32.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.367+0000 7f5ab3e00640 1 -- 172.21.15.159:0/576488499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ab415fc50 msgr2=0x7f5ab40cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:32.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.367+0000 7f5ab3e00640 1 --2- 172.21.15.159:0/576488499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ab415fc50 0x7f5ab40cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:32.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.367+0000 7f5ab3e00640 1 -- 172.21.15.159:0/576488499 >> v1:172.21.15.178:6789/0 conn(0x7f5ab40cc0b0 legacy=0x7f5ab406dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:32.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.367+0000 7f5ab3e00640 1 -- 172.21.15.159:0/576488499 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab41603c0 con 0x7f5ab40cc490 2024-04-04T06:10:32.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.368+0000 7f5ab3e00640 1 --2- 172.21.15.159:0/576488499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab40cc490 0x7f5ab40d0300 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5aac009e50 tx=0x7f5aac02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3344e330922c3568 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:32.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.369+0000 7f5ab2a00640 1 -- 172.21.15.159:0/576488499 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac02ca70 con 0x7f5ab40cc490 2024-04-04T06:10:32.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.369+0000 7f5ab2a00640 1 -- 172.21.15.159:0/576488499 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aac02cc10 con 0x7f5ab40cc490 2024-04-04T06:10:32.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.369+0000 7f5ab2a00640 1 -- 172.21.15.159:0/576488499 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac0343f0 con 0x7f5ab40cc490 2024-04-04T06:10:32.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.369+0000 7f5ab8a00640 1 -- 172.21.15.159:0/576488499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab40cc490 msgr2=0x7f5ab40d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:32.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.369+0000 7f5ab8a00640 1 --2- 172.21.15.159:0/576488499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab40cc490 0x7f5ab40d0300 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5aac009e50 tx=0x7f5aac02bed0 comp rx=0 tx=0).stop 2024-04-04T06:10:32.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.369+0000 7f5ab8a00640 1 -- 172.21.15.159:0/576488499 shutdown_connections 2024-04-04T06:10:32.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.369+0000 7f5ab8a00640 1 --2- 172.21.15.159:0/576488499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab40cc490 0x7f5ab40d0300 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:32.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.369+0000 7f5ab8a00640 1 --2- 172.21.15.159:0/576488499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ab415fc50 0x7f5ab40cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:32.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.369+0000 7f5ab8a00640 1 -- 172.21.15.159:0/576488499 >> 172.21.15.159:0/576488499 conn(0x7f5ab40ccad0 msgr2=0x7f5ab40cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:32.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.369+0000 7f5ab8a00640 1 -- 172.21.15.159:0/576488499 shutdown_connections 2024-04-04T06:10:32.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.369+0000 7f5ab8a00640 1 -- 172.21.15.159:0/576488499 wait complete. 2024-04-04T06:10:32.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.371+0000 7f5ab8a00640 1 Processor -- start 2024-04-04T06:10:32.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.371+0000 7f5ab8a00640 1 -- start start 2024-04-04T06:10:32.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.372+0000 7f5ab8a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ab40cc0b0 0x7f5ab415e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:32.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.372+0000 7f5ab8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab40cc490 0x7f5ab40a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:32.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.372+0000 7f5ab8a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ab41f83f0 0x7f5ab413d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:32.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.372+0000 7f5ab8a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab415ca70 con 0x7f5ab40cc490 2024-04-04T06:10:32.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.372+0000 7f5ab8a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab4160980 con 0x7f5ab40cc0b0 2024-04-04T06:10:32.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.372+0000 7f5ab8a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab415cbf0 con 0x7f5ab41f83f0 2024-04-04T06:10:32.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.372+0000 7f5aaaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab40cc490 0x7f5ab40a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:32.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.372+0000 7f5aaaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab40cc490 0x7f5ab40a9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42080/0 (socket says 172.21.15.159:42080) 2024-04-04T06:10:32.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.372+0000 7f5aaaa00640 1 -- 172.21.15.159:0/2733791297 learned_addr learned my addr 172.21.15.159:0/2733791297 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:32.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.373+0000 7f5aaaa00640 1 -- 172.21.15.159:0/2733791297 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ab41f83f0 msgr2=0x7f5ab413d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:32.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.373+0000 7f5aaaa00640 1 --2- 172.21.15.159:0/2733791297 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ab41f83f0 0x7f5ab413d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:32.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.373+0000 7f5aaaa00640 1 -- 172.21.15.159:0/2733791297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ab40cc0b0 msgr2=0x7f5ab415e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:32.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.373+0000 7f5aaaa00640 1 --2- 172.21.15.159:0/2733791297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ab40cc0b0 0x7f5ab415e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:32.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.373+0000 7f5aaaa00640 1 -- 172.21.15.159:0/2733791297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab40a9bf0 con 0x7f5ab40cc490 2024-04-04T06:10:32.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.373+0000 7f5aaaa00640 1 --2- 172.21.15.159:0/2733791297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab40cc490 0x7f5ab40a9480 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00c820 tx=0x7f5a9c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:32.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.373+0000 7f5aabe00640 1 -- 172.21.15.159:0/2733791297 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c015070 con 0x7f5ab40cc490 2024-04-04T06:10:32.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.373+0000 7f5aabe00640 1 -- 172.21.15.159:0/2733791297 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a9c004020 con 0x7f5ab40cc490 2024-04-04T06:10:32.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.373+0000 7f5aabe00640 1 -- 172.21.15.159:0/2733791297 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c0043f0 con 0x7f5ab40cc490 2024-04-04T06:10:32.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.373+0000 7f5ab8a00640 1 -- 172.21.15.159:0/2733791297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab41f7910 con 0x7f5ab40cc490 2024-04-04T06:10:32.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.374+0000 7f5ab8a00640 1 -- 172.21.15.159:0/2733791297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab41f7ed0 con 0x7f5ab40cc490 2024-04-04T06:10:32.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.374+0000 7f5ab8a00640 1 -- 172.21.15.159:0/2733791297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab41f80d0 con 0x7f5ab40cc490 2024-04-04T06:10:32.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.376+0000 7f5aabe00640 1 -- 172.21.15.159:0/2733791297 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a9c007830 con 0x7f5ab40cc490 2024-04-04T06:10:32.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.376+0000 7f5aabe00640 1 --2- 172.21.15.159:0/2733791297 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5a9006c8f0 0x7f5a9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:32.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.376+0000 7f5aabe00640 1 -- 172.21.15.159:0/2733791297 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7f5a9c08b5d0 con 0x7f5ab40cc490 2024-04-04T06:10:32.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.377+0000 7f5ab3400640 1 --2- 172.21.15.159:0/2733791297 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5a9006c8f0 0x7f5a9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:32.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.378+0000 7f5ab3400640 1 --2- 172.21.15.159:0/2733791297 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5a9006c8f0 0x7f5a9006edb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4003cc0 tx=0x7f5aa40079b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:32.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.381+0000 7f5aabe00640 1 -- 172.21.15.159:0/2733791297 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5ab41f80d0 con 0x7f5ab40cc490 2024-04-04T06:10:32.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.623+0000 7f5ab8a00640 1 -- 172.21.15.159:0/2733791297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ab4075940 con 0x7f5ab40cc490 2024-04-04T06:10:32.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.624+0000 7f5aabe00640 1 -- 172.21.15.159:0/2733791297 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+3878 (secure 0 0 0) 0x7f5a9c059d90 con 0x7f5ab40cc490 2024-04-04T06:10:32.618 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:32.618 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5191,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3507245876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3507245876},{"type":"v1","addr":"172.21.15.159:6835","nonce":3507245876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/967262871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":967262871},{"type":"v1","addr":"172.21.15.178:6835","nonce":967262871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/2969351496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":2969351496},{"type":"v1","addr":"172.21.15.178:6837","nonce":2969351496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6534,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6834/2083814478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":2083814478},{"type":"v1","addr":"172.21.15.159:6834","nonce":2083814478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[]} 2024-04-04T06:10:32.618 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 37 2024-04-04T06:10:32.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.627+0000 7f5ab8a00640 1 -- 172.21.15.159:0/2733791297 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5a9006c8f0 msgr2=0x7f5a9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:32.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.627+0000 7f5ab8a00640 1 --2- 172.21.15.159:0/2733791297 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5a9006c8f0 0x7f5a9006edb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4003cc0 tx=0x7f5aa40079b0 comp rx=0 tx=0).stop 2024-04-04T06:10:32.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.627+0000 7f5ab8a00640 1 -- 172.21.15.159:0/2733791297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab40cc490 msgr2=0x7f5ab40a9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:32.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.627+0000 7f5ab8a00640 1 --2- 172.21.15.159:0/2733791297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab40cc490 0x7f5ab40a9480 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00c820 tx=0x7f5a9c00cce0 comp rx=0 tx=0).stop 2024-04-04T06:10:32.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.627+0000 7f5ab8a00640 1 -- 172.21.15.159:0/2733791297 shutdown_connections 2024-04-04T06:10:32.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.627+0000 7f5ab8a00640 1 --2- 172.21.15.159:0/2733791297 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5ab41f83f0 0x7f5ab413d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:32.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.627+0000 7f5ab8a00640 1 --2- 172.21.15.159:0/2733791297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab40cc490 0x7f5ab40a9480 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:32.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.627+0000 7f5ab8a00640 1 --2- 172.21.15.159:0/2733791297 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5a9006c8f0 0x7f5a9006edb0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:32.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.627+0000 7f5ab8a00640 1 --2- 172.21.15.159:0/2733791297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ab40cc0b0 0x7f5ab415e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:32.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.627+0000 7f5ab8a00640 1 -- 172.21.15.159:0/2733791297 >> 172.21.15.159:0/2733791297 conn(0x7f5ab40ccad0 msgr2=0x7f5ab413e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:32.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.627+0000 7f5ab8a00640 1 -- 172.21.15.159:0/2733791297 shutdown_connections 2024-04-04T06:10:32.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:32.628+0000 7f5ab8a00640 1 -- 172.21.15.159:0/2733791297 wait complete. 2024-04-04T06:10:32.641 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-04T06:10:32.641 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-04T06:10:32.641 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-04T06:10:32.641 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:10:32.642 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-04T06:10:32.642 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-04T06:10:32.642 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-04T06:10:32.642 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:10:32.642 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-04T06:10:32.642 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-04T06:10:32.642 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-04T06:10:32.643 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:10:32.643 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-04T06:10:32.643 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-04T06:10:32.643 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-04T06:10:32.643 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:10:32.660 INFO:tasks.ceph.mds.c:Stopped 2024-04-04T06:10:32.661 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-04T06:10:32.671 INFO:tasks.ceph.mds.d:Stopped 2024-04-04T06:10:32.671 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-04T06:10:32.674 INFO:tasks.ceph.mds.b:Stopped 2024-04-04T06:10:32.674 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-04T06:10:32.708 INFO:tasks.ceph.mds.c:Started 2024-04-04T06:10:32.708 DEBUG:teuthology.parallel:result is None 2024-04-04T06:10:32.721 INFO:tasks.ceph.mds.d:Started 2024-04-04T06:10:32.721 DEBUG:teuthology.parallel:result is None 2024-04-04T06:10:32.727 INFO:tasks.ceph.mds.b:Started 2024-04-04T06:10:32.727 DEBUG:teuthology.parallel:result is None 2024-04-04T06:10:32.884 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-04T06:10:32.884 INFO:tasks.ceph.mds.c.smithi159.stdout:starting mds.c at 2024-04-04T06:10:32.886 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-04T06:10:33.402 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-04T06:10:38.661 INFO:tasks.ceph.mds.a:Stopped 2024-04-04T06:10:38.661 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-04T06:10:38.669 INFO:tasks.ceph.mds.a:Started 2024-04-04T06:10:38.669 DEBUG:teuthology.parallel:result is None 2024-04-04T06:10:38.670 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-04T06:10:38.837 INFO:tasks.ceph.mds.a.smithi159.stdout:starting mds.a at 2024-04-04T06:10:38.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.880+0000 7f656ca00640 1 Processor -- start 2024-04-04T06:10:38.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.880+0000 7f656ca00640 1 -- start start 2024-04-04T06:10:38.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.881+0000 7f656ca00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f656815e4a0 0x7f65680b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:38.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.881+0000 7f656ca00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65680d2310 0x7f65680ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:38.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.881+0000 7f656ca00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f656815e880 con 0x7f65680cf530 2024-04-04T06:10:38.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.881+0000 7f656ca00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f65680d34e0 con 0x7f656815e4a0 2024-04-04T06:10:38.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.881+0000 7f656ca00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f65680d28d0 con 0x7f65680d2310 2024-04-04T06:10:38.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.882+0000 7f6565e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65680d2310 0x7f65680ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:38.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.882+0000 7f6566800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f656815e4a0 0x7f65680b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:38.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.882+0000 7f6565e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65680d2310 0x7f65680ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:44046/0 (socket says 172.21.15.159:44046) 2024-04-04T06:10:38.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.882+0000 7f6565e00640 1 -- 172.21.15.159:0/1002598828 learned_addr learned my addr 172.21.15.159:0/1002598828 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:38.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.883+0000 7f6565e00640 1 -- 172.21.15.159:0/1002598828 >> v2:172.21.15.178:3300/0 conn(0x7f656815e4a0 msgr2=0x7f65680b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:38.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.883+0000 7f6565e00640 1 --2- 172.21.15.159:0/1002598828 >> v2:172.21.15.178:3300/0 conn(0x7f656815e4a0 0x7f65680b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:38.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.883+0000 7f6565e00640 1 -- 172.21.15.159:0/1002598828 >> v1:172.21.15.178:6789/0 conn(0x7f65680cf530 legacy=0x7f65680d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:38.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.883+0000 7f6565e00640 1 -- 172.21.15.159:0/1002598828 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65680d2bb0 con 0x7f65680d2310 2024-04-04T06:10:38.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.884+0000 7f6565e00640 1 --2- 172.21.15.159:0/1002598828 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65680d2310 0x7f65680ceff0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6554009870 tx=0x7f655402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eebbe3bda7dd944b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:38.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.884+0000 7f6565400640 1 -- 172.21.15.159:0/1002598828 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6554034070 con 0x7f65680d2310 2024-04-04T06:10:38.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.885+0000 7f6565400640 1 -- 172.21.15.159:0/1002598828 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6554030840 con 0x7f65680d2310 2024-04-04T06:10:38.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.885+0000 7f6565400640 1 -- 172.21.15.159:0/1002598828 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6554030c10 con 0x7f65680d2310 2024-04-04T06:10:38.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.885+0000 7f656ca00640 1 -- 172.21.15.159:0/1002598828 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65680d2310 msgr2=0x7f65680ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:38.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.885+0000 7f656ca00640 1 --2- 172.21.15.159:0/1002598828 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65680d2310 0x7f65680ceff0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6554009870 tx=0x7f655402b980 comp rx=0 tx=0).stop 2024-04-04T06:10:38.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.885+0000 7f656ca00640 1 -- 172.21.15.159:0/1002598828 shutdown_connections 2024-04-04T06:10:38.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.885+0000 7f656ca00640 1 --2- 172.21.15.159:0/1002598828 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65680d2310 0x7f65680ceff0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:38.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.885+0000 7f656ca00640 1 --2- 172.21.15.159:0/1002598828 >> v2:172.21.15.178:3300/0 conn(0x7f656815e4a0 0x7f65680b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:38.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.885+0000 7f656ca00640 1 -- 172.21.15.159:0/1002598828 >> 172.21.15.159:0/1002598828 conn(0x7f65680676d0 msgr2=0x7f6568067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:38.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.885+0000 7f656ca00640 1 -- 172.21.15.159:0/1002598828 shutdown_connections 2024-04-04T06:10:38.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.885+0000 7f656ca00640 1 -- 172.21.15.159:0/1002598828 wait complete. 2024-04-04T06:10:38.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.887+0000 7f656ca00640 1 Processor -- start 2024-04-04T06:10:38.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.887+0000 7f656ca00640 1 -- start start 2024-04-04T06:10:38.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.888+0000 7f656ca00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65680cf530 0x7f656815f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:38.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.888+0000 7f656ca00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65680d2310 0x7f65681573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:38.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.888+0000 7f656ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f656815e4a0 0x7f65681f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:38.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.888+0000 7f656ca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f65680d3970 con 0x7f656815e4a0 2024-04-04T06:10:38.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.888+0000 7f656ca00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f65680d3170 con 0x7f65680cf530 2024-04-04T06:10:38.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.888+0000 7f656ca00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f65680ba1a0 con 0x7f65680d2310 2024-04-04T06:10:38.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.889+0000 7f6567200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f656815e4a0 0x7f65681f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:38.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.889+0000 7f6567200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f656815e4a0 0x7f65681f78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40762/0 (socket says 172.21.15.159:40762) 2024-04-04T06:10:38.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.889+0000 7f6567200640 1 -- 172.21.15.159:0/2044602124 learned_addr learned my addr 172.21.15.159:0/2044602124 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:38.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.889+0000 7f6565e00640 1 --2- 172.21.15.159:0/2044602124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65680d2310 0x7f65681573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:38.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.889+0000 7f6566800640 1 --2- 172.21.15.159:0/2044602124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65680cf530 0x7f656815f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:38.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.889+0000 7f6567200640 1 -- 172.21.15.159:0/2044602124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65680d2310 msgr2=0x7f65681573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:38.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.889+0000 7f6567200640 1 --2- 172.21.15.159:0/2044602124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65680d2310 0x7f65681573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:38.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.889+0000 7f6567200640 1 -- 172.21.15.159:0/2044602124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65680cf530 msgr2=0x7f656815f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:38.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.889+0000 7f6567200640 1 --2- 172.21.15.159:0/2044602124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65680cf530 0x7f656815f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:38.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.889+0000 7f6567200640 1 -- 172.21.15.159:0/2044602124 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6568161960 con 0x7f656815e4a0 2024-04-04T06:10:38.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.889+0000 7f6565e00640 1 --2- 172.21.15.159:0/2044602124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65680d2310 0x7f65681573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:38.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.889+0000 7f6567200640 1 --2- 172.21.15.159:0/2044602124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f656815e4a0 0x7f65681f78f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f656000c6b0 tx=0x7f656000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:38.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.890+0000 7f655ea00640 1 -- 172.21.15.159:0/2044602124 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6560003b70 con 0x7f656815e4a0 2024-04-04T06:10:38.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.890+0000 7f655ea00640 1 -- 172.21.15.159:0/2044602124 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6560003d30 con 0x7f656815e4a0 2024-04-04T06:10:38.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.890+0000 7f656ca00640 1 -- 172.21.15.159:0/2044602124 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f656815f9a0 con 0x7f656815e4a0 2024-04-04T06:10:38.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.890+0000 7f655ea00640 1 -- 172.21.15.159:0/2044602124 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6560005310 con 0x7f656815e4a0 2024-04-04T06:10:38.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.890+0000 7f656ca00640 1 -- 172.21.15.159:0/2044602124 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65680c88f0 con 0x7f656815e4a0 2024-04-04T06:10:38.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.891+0000 7f656ca00640 1 -- 172.21.15.159:0/2044602124 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6568075940 con 0x7f656815e4a0 2024-04-04T06:10:38.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.892+0000 7f655ea00640 1 -- 172.21.15.159:0/2044602124 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f65600054b0 con 0x7f656815e4a0 2024-04-04T06:10:38.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.893+0000 7f655ea00640 1 --2- 172.21.15.159:0/2044602124 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f654806c8f0 0x7f654806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:38.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.893+0000 7f655ea00640 1 -- 172.21.15.159:0/2044602124 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4943+0+0 (secure 0 0 0) 0x7f656008b780 con 0x7f656815e4a0 2024-04-04T06:10:38.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.893+0000 7f6566800640 1 --2- 172.21.15.159:0/2044602124 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f654806c8f0 0x7f654806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:38.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.894+0000 7f6566800640 1 --2- 172.21.15.159:0/2044602124 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f654806c8f0 0x7f654806edb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6558009a10 tx=0x7f6558008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:38.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:38.898+0000 7f655ea00640 1 -- 172.21.15.159:0/2044602124 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f656815f9a0 con 0x7f656815e4a0 2024-04-04T06:10:39.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:39.108+0000 7f656ca00640 1 -- 172.21.15.159:0/2044602124 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f65680b4260 con 0x7f656815e4a0 2024-04-04T06:10:40.115 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-04T06:10:40.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.294+0000 7f655ea00640 1 -- 172.21.15.159:0/2044602124 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v67) ==== 97+0+0 (secure 0 0 0) 0x7f6568161960 con 0x7f656815e4a0 2024-04-04T06:10:40.287 INFO:teuthology.orchestra.run.smithi159.stderr: removed all blocklist entries 2024-04-04T06:10:40.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.297+0000 7f656ca00640 1 -- 172.21.15.159:0/2044602124 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f654806c8f0 msgr2=0x7f654806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:40.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.297+0000 7f656ca00640 1 --2- 172.21.15.159:0/2044602124 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f654806c8f0 0x7f654806edb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6558009a10 tx=0x7f6558008040 comp rx=0 tx=0).stop 2024-04-04T06:10:40.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.297+0000 7f656ca00640 1 -- 172.21.15.159:0/2044602124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f656815e4a0 msgr2=0x7f65681f78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:40.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.297+0000 7f656ca00640 1 --2- 172.21.15.159:0/2044602124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f656815e4a0 0x7f65681f78f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f656000c6b0 tx=0x7f656000cb70 comp rx=0 tx=0).stop 2024-04-04T06:10:40.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.298+0000 7f656ca00640 1 -- 172.21.15.159:0/2044602124 shutdown_connections 2024-04-04T06:10:40.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.298+0000 7f656ca00640 1 --2- 172.21.15.159:0/2044602124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f656815e4a0 0x7f65681f78f0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:40.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.298+0000 7f656ca00640 1 --2- 172.21.15.159:0/2044602124 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65680d2310 0x7f65681573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:40.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.298+0000 7f656ca00640 1 --2- 172.21.15.159:0/2044602124 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f654806c8f0 0x7f654806edb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:40.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.298+0000 7f656ca00640 1 --2- 172.21.15.159:0/2044602124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65680cf530 0x7f656815f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:40.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.298+0000 7f656ca00640 1 -- 172.21.15.159:0/2044602124 >> 172.21.15.159:0/2044602124 conn(0x7f65680676d0 msgr2=0x7f65680b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:40.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.298+0000 7f656ca00640 1 -- 172.21.15.159:0/2044602124 shutdown_connections 2024-04-04T06:10:40.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.298+0000 7f656ca00640 1 -- 172.21.15.159:0/2044602124 wait complete. 2024-04-04T06:10:40.312 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-04T06:10:40.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.523+0000 7fae99000640 1 Processor -- start 2024-04-04T06:10:40.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.523+0000 7fae99000640 1 -- start start 2024-04-04T06:10:40.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.524+0000 7fae99000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fae9414c360 0x7fae9406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:40.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.524+0000 7fae99000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae9406da30 0x7fae940cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:40.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.525+0000 7fae99000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fae940d0430 0x7fae940b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:40.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.525+0000 7fae99000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fae9414c740 con 0x7fae9414c360 2024-04-04T06:10:40.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.525+0000 7fae99000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fae940cc9d0 con 0x7fae940d0430 2024-04-04T06:10:40.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.525+0000 7fae99000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fae9406df70 con 0x7fae9406da30 2024-04-04T06:10:40.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.525+0000 7fae93400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fae9414c360 0x7fae9406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:40.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.525+0000 7fae93400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fae9414c360 0x7fae9406d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40764/0 (socket says 172.21.15.159:40764) 2024-04-04T06:10:40.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.525+0000 7fae93400640 1 -- 172.21.15.159:0/2547636965 learned_addr learned my addr 172.21.15.159:0/2547636965 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:40.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.525+0000 7fae93400640 1 -- 172.21.15.159:0/2547636965 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae9406da30 msgr2=0x7fae940cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:40.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.525+0000 7fae92a00640 1 --2- 172.21.15.159:0/2547636965 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae9406da30 0x7fae940cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:40.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.525+0000 7fae93e00640 1 --2- 172.21.15.159:0/2547636965 >> v2:172.21.15.178:3300/0 conn(0x7fae940d0430 0x7fae940b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:40.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.525+0000 7fae93400640 1 --2- 172.21.15.159:0/2547636965 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae9406da30 0x7fae940cfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:40.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.525+0000 7fae93400640 1 -- 172.21.15.159:0/2547636965 >> v2:172.21.15.178:3300/0 conn(0x7fae940d0430 msgr2=0x7fae940b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:40.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.525+0000 7fae93400640 1 --2- 172.21.15.159:0/2547636965 >> v2:172.21.15.178:3300/0 conn(0x7fae940d0430 0x7fae940b38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:40.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.525+0000 7fae93400640 1 -- 172.21.15.159:0/2547636965 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9406e250 con 0x7fae9414c360 2024-04-04T06:10:40.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.525+0000 7fae93e00640 1 --2- 172.21.15.159:0/2547636965 >> v2:172.21.15.178:3300/0 conn(0x7fae940d0430 0x7fae940b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:40.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.525+0000 7fae92a00640 1 --2- 172.21.15.159:0/2547636965 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae9406da30 0x7fae940cfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:40.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.527+0000 7fae93400640 1 --2- 172.21.15.159:0/2547636965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae9414c360 0x7fae9406d4f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fae940afc30 tx=0x7fae8402b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6ae3b5496ed37d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:40.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.527+0000 7fae92000640 1 -- 172.21.15.159:0/2547636965 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84035020 con 0x7fae9414c360 2024-04-04T06:10:40.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.527+0000 7fae92000640 1 -- 172.21.15.159:0/2547636965 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae84031880 con 0x7fae9414c360 2024-04-04T06:10:40.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.527+0000 7fae92000640 1 -- 172.21.15.159:0/2547636965 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84031c50 con 0x7fae9414c360 2024-04-04T06:10:40.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.527+0000 7fae99000640 1 -- 172.21.15.159:0/2547636965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae9414c360 msgr2=0x7fae9406d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:40.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.527+0000 7fae99000640 1 --2- 172.21.15.159:0/2547636965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae9414c360 0x7fae9406d4f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fae940afc30 tx=0x7fae8402b920 comp rx=0 tx=0).stop 2024-04-04T06:10:40.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.528+0000 7fae99000640 1 -- 172.21.15.159:0/2547636965 shutdown_connections 2024-04-04T06:10:40.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.528+0000 7fae99000640 1 --2- 172.21.15.159:0/2547636965 >> v2:172.21.15.178:3300/0 conn(0x7fae940d0430 0x7fae940b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:40.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.528+0000 7fae99000640 1 --2- 172.21.15.159:0/2547636965 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae9406da30 0x7fae940cfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:40.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.528+0000 7fae99000640 1 --2- 172.21.15.159:0/2547636965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae9414c360 0x7fae9406d4f0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:40.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.528+0000 7fae99000640 1 -- 172.21.15.159:0/2547636965 >> 172.21.15.159:0/2547636965 conn(0x7fae9406b570 msgr2=0x7fae9406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:40.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.528+0000 7fae99000640 1 -- 172.21.15.159:0/2547636965 shutdown_connections 2024-04-04T06:10:40.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.528+0000 7fae99000640 1 -- 172.21.15.159:0/2547636965 wait complete. 2024-04-04T06:10:40.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.529+0000 7fae99000640 1 Processor -- start 2024-04-04T06:10:40.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.530+0000 7fae99000640 1 -- start start 2024-04-04T06:10:40.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.530+0000 7fae99000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae9406da30 0x7fae940c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:40.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.530+0000 7fae99000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae940d0430 0x7fae941d6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:40.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.530+0000 7fae99000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae9414c360 0x7fae941d8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:40.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.530+0000 7fae99000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fae940b4e20 con 0x7fae9414c360 2024-04-04T06:10:40.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.530+0000 7fae99000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fae940ccf00 con 0x7fae940d0430 2024-04-04T06:10:40.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.530+0000 7fae99000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fae940b3e30 con 0x7fae9406da30 2024-04-04T06:10:40.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.531+0000 7fae93e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae9414c360 0x7fae941d8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:40.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.531+0000 7fae93e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae9414c360 0x7fae941d8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40768/0 (socket says 172.21.15.159:40768) 2024-04-04T06:10:40.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.531+0000 7fae93e00640 1 -- 172.21.15.159:0/3715058703 learned_addr learned my addr 172.21.15.159:0/3715058703 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:40.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.531+0000 7fae92a00640 1 --2- 172.21.15.159:0/3715058703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae940d0430 0x7fae941d6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:40.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.531+0000 7fae93e00640 1 -- 172.21.15.159:0/3715058703 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae9406da30 msgr2=0x7fae940c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:40.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.531+0000 7fae93e00640 1 --2- 172.21.15.159:0/3715058703 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae9406da30 0x7fae940c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:40.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.531+0000 7fae93e00640 1 -- 172.21.15.159:0/3715058703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae940d0430 msgr2=0x7fae941d6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:40.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.531+0000 7fae93e00640 1 --2- 172.21.15.159:0/3715058703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae940d0430 0x7fae941d6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:40.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.531+0000 7fae93e00640 1 -- 172.21.15.159:0/3715058703 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae941d75b0 con 0x7fae9414c360 2024-04-04T06:10:40.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.531+0000 7fae93e00640 1 --2- 172.21.15.159:0/3715058703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae9414c360 0x7fae941d8650 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00cef0 tx=0x7fae8c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:40.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.532+0000 7fae8b400640 1 -- 172.21.15.159:0/3715058703 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c01a070 con 0x7fae9414c360 2024-04-04T06:10:40.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.532+0000 7fae8b400640 1 -- 172.21.15.159:0/3715058703 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae8c00d860 con 0x7fae9414c360 2024-04-04T06:10:40.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.532+0000 7fae99000640 1 -- 172.21.15.159:0/3715058703 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae941d7890 con 0x7fae9414c360 2024-04-04T06:10:40.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.532+0000 7fae8b400640 1 -- 172.21.15.159:0/3715058703 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c00dc50 con 0x7fae9414c360 2024-04-04T06:10:40.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.532+0000 7fae99000640 1 -- 172.21.15.159:0/3715058703 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae941d7e50 con 0x7fae9414c360 2024-04-04T06:10:40.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.533+0000 7fae99000640 1 -- 172.21.15.159:0/3715058703 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae9406bb80 con 0x7fae9414c360 2024-04-04T06:10:40.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.534+0000 7fae8b400640 1 -- 172.21.15.159:0/3715058703 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae8c01d070 con 0x7fae9414c360 2024-04-04T06:10:40.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.535+0000 7fae8b400640 1 --2- 172.21.15.159:0/3715058703 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fae7406c8f0 0x7fae7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:40.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.535+0000 7fae8b400640 1 -- 172.21.15.159:0/3715058703 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4857+0+0 (secure 0 0 0) 0x7fae8c08c0a0 con 0x7fae9414c360 2024-04-04T06:10:40.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.536+0000 7fae93400640 1 --2- 172.21.15.159:0/3715058703 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fae7406c8f0 0x7fae7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:40.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.537+0000 7fae93400640 1 --2- 172.21.15.159:0/3715058703 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fae7406c8f0 0x7fae7406edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fae84036000 tx=0x7fae84030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:40.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.541+0000 7fae8b400640 1 -- 172.21.15.159:0/3715058703 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fae9406bb80 con 0x7fae9414c360 2024-04-04T06:10:40.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.806+0000 7fae99000640 1 -- 172.21.15.159:0/3715058703 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fae9406f560 con 0x7fae9414c360 2024-04-04T06:10:40.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.807+0000 7fae8b400640 1 -- 172.21.15.159:0/3715058703 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7fae9406f560 con 0x7fae9414c360 2024-04-04T06:10:40.800 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:40.800 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:10:40.800 INFO:teuthology.orchestra.run.smithi159.stdout: "auth_dump": [ 2024-04-04T06:10:40.800 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:10:40.800 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.a", 2024-04-04T06:10:40.800 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5mvoZ2FxAAsVEQrkiICIH6T7ZzaqcWbA==", 2024-04-04T06:10:40.800 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:10:40.800 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:10:40.800 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:10:40.801 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:10:40.801 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:10:40.801 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:10:40.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:10:40.801 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:10:40.801 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.b", 2024-04-04T06:10:40.801 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5m+rccORAA6I/lQ+22QfWYL9+aoOFJ0w==", 2024-04-04T06:10:40.801 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:10:40.801 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:10:40.801 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:10:40.801 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:10:40.801 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:10:40.801 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:10:40.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:10:40.802 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:10:40.802 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.c", 2024-04-04T06:10:40.802 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5mjmQcKBAA8BjpQce6G7EycXOkyVECpQ==", 2024-04-04T06:10:40.802 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:10:40.802 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:10:40.802 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:10:40.802 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:10:40.802 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:10:40.802 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:10:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:10:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:10:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.d", 2024-04-04T06:10:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5m04hoDRAAKdhRI7nosZp7d4y7dVvzvg==", 2024-04-04T06:10:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:10:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:10:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:10:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:10:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:10:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:10:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:10:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:10:40.803 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.0", 2024-04-04T06:10:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQABQw5mCe7fNRAA6ZKxWvnMLf4jhAlpjudFRA==", 2024-04-04T06:10:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:10:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:10:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:10:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:10:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:10:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:10:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:10:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.1", 2024-04-04T06:10:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAFQw5mvIqTExAAU8Pm5alxFtHEL4qM9MavPw==", 2024-04-04T06:10:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:10:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:10:40.804 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:10:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:10:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:10:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:10:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:10:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.2", 2024-04-04T06:10:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAIQw5mAHblHRAAqEDe5dIGTDpdy2OsQyzJXA==", 2024-04-04T06:10:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:10:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:10:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:10:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:10:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:10:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:10:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:10:40.805 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.3", 2024-04-04T06:10:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQALQw5mwK7rJhAADQBAqE0Eej9rn91Wxhq/KA==", 2024-04-04T06:10:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:10:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:10:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:10:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:10:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:10:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:10:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:10:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.4", 2024-04-04T06:10:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAQQw5m/6I8GRAA2P7jI+AXYld2J11nDfhAUQ==", 2024-04-04T06:10:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:10:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:10:40.806 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:10:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:10:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:10:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:10:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:10:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.5", 2024-04-04T06:10:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQATQw5mv5HoMBAAn2mp2ys5KjS0UygOcX4vvg==", 2024-04-04T06:10:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:10:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:10:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:10:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:10:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:10:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:10:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:10:40.807 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.6", 2024-04-04T06:10:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAWQw5mx12gORAAT2LhrfEB5BJSl0ypgospzg==", 2024-04-04T06:10:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:10:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:10:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:10:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:10:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:10:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:10:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:10:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.7", 2024-04-04T06:10:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAaQw5m6HEOBxAAVegLH3bjVM7r1A6tK5eqHQ==", 2024-04-04T06:10:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:10:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:10:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:10:40.808 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:10:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:10:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:10:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:10:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.0", 2024-04-04T06:10:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mqNKJGxAA4jP9cadhRIyhQ3lssttWIw==", 2024-04-04T06:10:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:10:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:10:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow r", 2024-04-04T06:10:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-04T06:10:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:10:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:10:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:10:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.1", 2024-04-04T06:10:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mRkdOMBAA1ae1roIwrlB49DFQUoFSTg==", 2024-04-04T06:10:40.809 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:10:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:10:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:10:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:10:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:10:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:10:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:10:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:10:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.2", 2024-04-04T06:10:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mtW2/JRAAucxSSN9q0EQtI2a/P2yozQ==", 2024-04-04T06:10:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:10:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:10:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:10:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:10:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:10:40.810 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:10:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:10:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:10:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.3", 2024-04-04T06:10:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mgqD0OhAAov+3aDGyGEoqxep8+SBpSA==", 2024-04-04T06:10:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:10:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:10:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:10:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:10:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:10:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:10:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:10:40.811 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:10:40.812 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.admin", 2024-04-04T06:10:40.812 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD7Qg5m0O/iCBAAsWTzlEA0P60XYuxmJiAb4A==", 2024-04-04T06:10:40.812 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:10:40.812 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow *", 2024-04-04T06:10:40.812 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:10:40.812 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:10:40.813 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:10:40.813 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:10:40.813 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:10:40.813 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:10:40.813 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mgr.x", 2024-04-04T06:10:40.813 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5mhqbBCxAASBknMIj4uu7OiB3jn7WP8Q==", 2024-04-04T06:10:40.813 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:10:40.813 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow *", 2024-04-04T06:10:40.813 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile mgr", 2024-04-04T06:10:40.813 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:10:40.813 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:10:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:10:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:10:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mgr.y", 2024-04-04T06:10:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD7Qg5miXT0ORAAL6y306CPAfEGQKAJ0no4uw==", 2024-04-04T06:10:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:10:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow *", 2024-04-04T06:10:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile mgr", 2024-04-04T06:10:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:10:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:10:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:10:40.814 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-04-04T06:10:40.815 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:10:40.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.809+0000 7fae99000640 1 -- 172.21.15.159:0/3715058703 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fae7406c8f0 msgr2=0x7fae7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:40.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.809+0000 7fae99000640 1 --2- 172.21.15.159:0/3715058703 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fae7406c8f0 0x7fae7406edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fae84036000 tx=0x7fae84030000 comp rx=0 tx=0).stop 2024-04-04T06:10:40.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.809+0000 7fae99000640 1 -- 172.21.15.159:0/3715058703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae9414c360 msgr2=0x7fae941d8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:40.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.809+0000 7fae99000640 1 --2- 172.21.15.159:0/3715058703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae9414c360 0x7fae941d8650 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00cef0 tx=0x7fae8c00a710 comp rx=0 tx=0).stop 2024-04-04T06:10:40.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.809+0000 7fae99000640 1 -- 172.21.15.159:0/3715058703 shutdown_connections 2024-04-04T06:10:40.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.809+0000 7fae99000640 1 --2- 172.21.15.159:0/3715058703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae9414c360 0x7fae941d8650 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:40.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.809+0000 7fae99000640 1 --2- 172.21.15.159:0/3715058703 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fae7406c8f0 0x7fae7406edb0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:40.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.809+0000 7fae99000640 1 --2- 172.21.15.159:0/3715058703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae940d0430 0x7fae941d6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:40.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.809+0000 7fae99000640 1 --2- 172.21.15.159:0/3715058703 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae9406da30 0x7fae940c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:40.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.809+0000 7fae99000640 1 -- 172.21.15.159:0/3715058703 >> 172.21.15.159:0/3715058703 conn(0x7fae9406b570 msgr2=0x7fae9414b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:40.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.809+0000 7fae99000640 1 -- 172.21.15.159:0/3715058703 shutdown_connections 2024-04-04T06:10:40.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:40.810+0000 7fae99000640 1 -- 172.21.15.159:0/3715058703 wait complete. 2024-04-04T06:10:40.822 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:10:41.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.037+0000 7f7e5be00640 1 Processor -- start 2024-04-04T06:10:41.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.037+0000 7f7e5be00640 1 -- start start 2024-04-04T06:10:41.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.038+0000 7f7e5be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7e5c15e4a0 0x7f7e5c0b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:41.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.039+0000 7f7e5be00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7e5c0d2310 0x7f7e5c0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:41.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.039+0000 7f7e5be00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e5c15e880 con 0x7f7e5c0cf530 2024-04-04T06:10:41.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.039+0000 7f7e5be00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f7e5c0cf9a0 con 0x7f7e5c15e4a0 2024-04-04T06:10:41.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.039+0000 7f7e5be00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7e5c0d34e0 con 0x7f7e5c0d2310 2024-04-04T06:10:41.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.039+0000 7f7e5aa00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7e5c15e4a0 0x7f7e5c0b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:41.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.039+0000 7f7e5aa00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7e5c15e4a0 0x7f7e5c0b9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40778/0 (socket says 172.21.15.159:40778) 2024-04-04T06:10:41.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.039+0000 7f7e5aa00640 1 -- 172.21.15.159:0/2124349709 learned_addr learned my addr 172.21.15.159:0/2124349709 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:41.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.039+0000 7f7e5a000640 1 --2- 172.21.15.159:0/2124349709 >> v2:172.21.15.178:3300/0 conn(0x7f7e5c0d2310 0x7f7e5c0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:41.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.040+0000 7f7e5aa00640 1 -- 172.21.15.159:0/2124349709 >> v2:172.21.15.178:3300/0 conn(0x7f7e5c0d2310 msgr2=0x7f7e5c0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:41.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.040+0000 7f7e5aa00640 1 --2- 172.21.15.159:0/2124349709 >> v2:172.21.15.178:3300/0 conn(0x7f7e5c0d2310 0x7f7e5c0ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.040+0000 7f7e5aa00640 1 -- 172.21.15.159:0/2124349709 >> v1:172.21.15.178:6789/0 conn(0x7f7e5c0cf530 legacy=0x7f7e5c0d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:41.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.040+0000 7f7e5aa00640 1 -- 172.21.15.159:0/2124349709 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5c0d28d0 con 0x7f7e5c15e4a0 2024-04-04T06:10:41.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.041+0000 7f7e5aa00640 1 --2- 172.21.15.159:0/2124349709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e5c15e4a0 0x7f7e5c0b9c60 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0d3660 tx=0x7f7e4c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf98bf860c2c021a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:41.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.041+0000 7f7e59600640 1 -- 172.21.15.159:0/2124349709 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c02e070 con 0x7f7e5c15e4a0 2024-04-04T06:10:41.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.041+0000 7f7e59600640 1 -- 172.21.15.159:0/2124349709 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e4c037070 con 0x7f7e5c15e4a0 2024-04-04T06:10:41.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.041+0000 7f7e59600640 1 -- 172.21.15.159:0/2124349709 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c031a00 con 0x7f7e5c15e4a0 2024-04-04T06:10:41.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.042+0000 7f7e5be00640 1 -- 172.21.15.159:0/2124349709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e5c15e4a0 msgr2=0x7f7e5c0b9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:41.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.042+0000 7f7e5be00640 1 --2- 172.21.15.159:0/2124349709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e5c15e4a0 0x7f7e5c0b9c60 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0d3660 tx=0x7f7e4c02b880 comp rx=0 tx=0).stop 2024-04-04T06:10:41.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.042+0000 7f7e5be00640 1 -- 172.21.15.159:0/2124349709 shutdown_connections 2024-04-04T06:10:41.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.042+0000 7f7e5be00640 1 --2- 172.21.15.159:0/2124349709 >> v2:172.21.15.178:3300/0 conn(0x7f7e5c0d2310 0x7f7e5c0ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.042+0000 7f7e5be00640 1 --2- 172.21.15.159:0/2124349709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e5c15e4a0 0x7f7e5c0b9c60 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.042+0000 7f7e5be00640 1 -- 172.21.15.159:0/2124349709 >> 172.21.15.159:0/2124349709 conn(0x7f7e5c0676d0 msgr2=0x7f7e5c067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:41.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.042+0000 7f7e5be00640 1 -- 172.21.15.159:0/2124349709 shutdown_connections 2024-04-04T06:10:41.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.042+0000 7f7e5be00640 1 -- 172.21.15.159:0/2124349709 wait complete. 2024-04-04T06:10:41.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.044+0000 7f7e5be00640 1 Processor -- start 2024-04-04T06:10:41.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.044+0000 7f7e5be00640 1 -- start start 2024-04-04T06:10:41.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.045+0000 7f7e5be00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e5c0cf530 0x7f7e5c157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:41.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.045+0000 7f7e5be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e5c0d2310 0x7f7e5c159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:41.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.045+0000 7f7e5be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e5c15e4a0 0x7f7e5c1562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:41.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.045+0000 7f7e5be00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c0d3170 con 0x7f7e5c15e4a0 2024-04-04T06:10:41.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.045+0000 7f7e5be00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c0d2e90 con 0x7f7e5c0d2310 2024-04-04T06:10:41.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.045+0000 7f7e5be00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7e5c0d3970 con 0x7f7e5c0cf530 2024-04-04T06:10:41.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.045+0000 7f7e5b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e5c15e4a0 0x7f7e5c1562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:41.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.045+0000 7f7e5b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e5c15e4a0 0x7f7e5c1562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40782/0 (socket says 172.21.15.159:40782) 2024-04-04T06:10:41.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.045+0000 7f7e5b400640 1 -- 172.21.15.159:0/74373433 learned_addr learned my addr 172.21.15.159:0/74373433 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:41.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.046+0000 7f7e5a000640 1 --2- 172.21.15.159:0/74373433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e5c0d2310 0x7f7e5c159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:41.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.046+0000 7f7e5b400640 1 -- 172.21.15.159:0/74373433 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e5c0cf530 msgr2=0x7f7e5c157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:41.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.046+0000 7f7e5aa00640 1 --2- 172.21.15.159:0/74373433 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e5c0cf530 0x7f7e5c157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:41.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.046+0000 7f7e5b400640 1 --2- 172.21.15.159:0/74373433 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e5c0cf530 0x7f7e5c157d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.046+0000 7f7e5b400640 1 -- 172.21.15.159:0/74373433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e5c0d2310 msgr2=0x7f7e5c159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:41.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.046+0000 7f7e5b400640 1 --2- 172.21.15.159:0/74373433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e5c0d2310 0x7f7e5c159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.046+0000 7f7e5b400640 1 -- 172.21.15.159:0/74373433 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5c156cd0 con 0x7f7e5c15e4a0 2024-04-04T06:10:41.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.046+0000 7f7e5a000640 1 --2- 172.21.15.159:0/74373433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e5c0d2310 0x7f7e5c159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:41.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.046+0000 7f7e5b400640 1 --2- 172.21.15.159:0/74373433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e5c15e4a0 0x7f7e5c1562f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f7e5400c780 tx=0x7f7e5400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:41.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.046+0000 7f7e53400640 1 -- 172.21.15.159:0/74373433 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e54004b60 con 0x7f7e5c15e4a0 2024-04-04T06:10:41.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.046+0000 7f7e53400640 1 -- 172.21.15.159:0/74373433 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e54004d00 con 0x7f7e5c15e4a0 2024-04-04T06:10:41.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.046+0000 7f7e53400640 1 -- 172.21.15.159:0/74373433 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5400aa90 con 0x7f7e5c15e4a0 2024-04-04T06:10:41.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.046+0000 7f7e5be00640 1 -- 172.21.15.159:0/74373433 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e5c156fb0 con 0x7f7e5c15e4a0 2024-04-04T06:10:41.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.047+0000 7f7e5be00640 1 -- 172.21.15.159:0/74373433 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e5c157570 con 0x7f7e5c15e4a0 2024-04-04T06:10:41.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.047+0000 7f7e5be00640 1 -- 172.21.15.159:0/74373433 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e5c075940 con 0x7f7e5c15e4a0 2024-04-04T06:10:41.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.048+0000 7f7e53400640 1 -- 172.21.15.159:0/74373433 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e5400ac30 con 0x7f7e5c15e4a0 2024-04-04T06:10:41.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.049+0000 7f7e53400640 1 --2- 172.21.15.159:0/74373433 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e3c06c8f0 0x7f7e3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:41.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.049+0000 7f7e53400640 1 -- 172.21.15.159:0/74373433 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4857+0+0 (secure 0 0 0) 0x7f7e5408bb00 con 0x7f7e5c15e4a0 2024-04-04T06:10:41.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.050+0000 7f7e5aa00640 1 --2- 172.21.15.159:0/74373433 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e3c06c8f0 0x7f7e3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:41.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.051+0000 7f7e5aa00640 1 --2- 172.21.15.159:0/74373433 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e3c06c8f0 0x7f7e3c06edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c036000 tx=0x7f7e4c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:41.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.054+0000 7f7e53400640 1 -- 172.21.15.159:0/74373433 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7e54020070 con 0x7f7e5c15e4a0 2024-04-04T06:10:41.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.238+0000 7f7e5be00640 1 -- 172.21.15.159:0/74373433 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e5c0d4620 con 0x7f7e5c15e4a0 2024-04-04T06:10:41.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.239+0000 7f7e53400640 1 -- 172.21.15.159:0/74373433 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+12903 (secure 0 0 0) 0x7f7e5408be20 con 0x7f7e5c15e4a0 2024-04-04T06:10:41.233 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:41.233 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":67,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:10:40.284487+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:10:41.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.242+0000 7f7e5be00640 1 -- 172.21.15.159:0/74373433 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e3c06c8f0 msgr2=0x7f7e3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:41.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.242+0000 7f7e5be00640 1 --2- 172.21.15.159:0/74373433 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e3c06c8f0 0x7f7e3c06edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c036000 tx=0x7f7e4c008030 comp rx=0 tx=0).stop 2024-04-04T06:10:41.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.242+0000 7f7e5be00640 1 -- 172.21.15.159:0/74373433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e5c15e4a0 msgr2=0x7f7e5c1562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:41.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.242+0000 7f7e5be00640 1 --2- 172.21.15.159:0/74373433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e5c15e4a0 0x7f7e5c1562f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f7e5400c780 tx=0x7f7e5400cc40 comp rx=0 tx=0).stop 2024-04-04T06:10:41.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.242+0000 7f7e5be00640 1 -- 172.21.15.159:0/74373433 shutdown_connections 2024-04-04T06:10:41.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.242+0000 7f7e5be00640 1 --2- 172.21.15.159:0/74373433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e5c15e4a0 0x7f7e5c1562f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.242+0000 7f7e5be00640 1 --2- 172.21.15.159:0/74373433 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7e3c06c8f0 0x7f7e3c06edb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.242+0000 7f7e5be00640 1 --2- 172.21.15.159:0/74373433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e5c0d2310 0x7f7e5c159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.242+0000 7f7e5be00640 1 --2- 172.21.15.159:0/74373433 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e5c0cf530 0x7f7e5c157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.242+0000 7f7e5be00640 1 -- 172.21.15.159:0/74373433 >> 172.21.15.159:0/74373433 conn(0x7f7e5c0676d0 msgr2=0x7f7e5c0b65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:41.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.242+0000 7f7e5be00640 1 -- 172.21.15.159:0/74373433 shutdown_connections 2024-04-04T06:10:41.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.242+0000 7f7e5be00640 1 -- 172.21.15.159:0/74373433 wait complete. 2024-04-04T06:10:41.252 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:10:41.252 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:10:41.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.467+0000 7f8e5fe00640 1 Processor -- start 2024-04-04T06:10:41.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.467+0000 7f8e5fe00640 1 -- start start 2024-04-04T06:10:41.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.469+0000 7f8e5fe00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8e58153580 0x7f8e580d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:41.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.469+0000 7f8e5fe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e580d07c0 0x7f8e580aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:41.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.469+0000 7f8e5fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f8e580af2e0 0x7f8e5806d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:41.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.469+0000 7f8e5fe00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f8e580d0c00 con 0x7f8e580af2e0 2024-04-04T06:10:41.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.469+0000 7f8e5fe00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8e580afab0 con 0x7f8e58153580 2024-04-04T06:10:41.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.469+0000 7f8e5fe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8e5815cc80 con 0x7f8e580d07c0 2024-04-04T06:10:41.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.469+0000 7f8e5de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f8e580af2e0 0x7f8e5806d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:41.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.469+0000 7f8e5de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f8e580af2e0 0x7f8e5806d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40790/0 (socket says 172.21.15.159:40790) 2024-04-04T06:10:41.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.469+0000 7f8e5de00640 1 -- 172.21.15.159:0/1974875522 learned_addr learned my addr 172.21.15.159:0/1974875522 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:41.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.469+0000 7f8e5d400640 1 --2- 172.21.15.159:0/1974875522 >> v2:172.21.15.178:3300/0 conn(0x7f8e58153580 0x7f8e580d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:41.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.469+0000 7f8e5ca00640 1 --2- 172.21.15.159:0/1974875522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e580d07c0 0x7f8e580aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:41.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.470+0000 7f8e5de00640 1 -- 172.21.15.159:0/1974875522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e580d07c0 msgr2=0x7f8e580aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:41.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.470+0000 7f8e5de00640 1 --2- 172.21.15.159:0/1974875522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e580d07c0 0x7f8e580aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.470+0000 7f8e5de00640 1 -- 172.21.15.159:0/1974875522 >> v2:172.21.15.178:3300/0 conn(0x7f8e58153580 msgr2=0x7f8e580d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:41.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.470+0000 7f8e5de00640 1 --2- 172.21.15.159:0/1974875522 >> v2:172.21.15.178:3300/0 conn(0x7f8e58153580 0x7f8e580d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.470+0000 7f8e5de00640 1 -- 172.21.15.159:0/1974875522 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5815da00 con 0x7f8e580af2e0 2024-04-04T06:10:41.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.471+0000 7f8e5de00640 1 --2- 172.21.15.159:0/1974875522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e580af2e0 0x7f8e5806d5b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f8e580b5270 tx=0x7f8e5002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90b21e4af92c632 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:41.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.471+0000 7f8e57e00640 1 -- 172.21.15.159:0/1974875522 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5002c9e0 con 0x7f8e580af2e0 2024-04-04T06:10:41.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.471+0000 7f8e57e00640 1 -- 172.21.15.159:0/1974875522 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e5002cb80 con 0x7f8e580af2e0 2024-04-04T06:10:41.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.471+0000 7f8e57e00640 1 -- 172.21.15.159:0/1974875522 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e500343f0 con 0x7f8e580af2e0 2024-04-04T06:10:41.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.472+0000 7f8e5fe00640 1 -- 172.21.15.159:0/1974875522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e580af2e0 msgr2=0x7f8e5806d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:41.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.472+0000 7f8e5fe00640 1 --2- 172.21.15.159:0/1974875522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e580af2e0 0x7f8e5806d5b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f8e580b5270 tx=0x7f8e5002be40 comp rx=0 tx=0).stop 2024-04-04T06:10:41.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.472+0000 7f8e5fe00640 1 -- 172.21.15.159:0/1974875522 shutdown_connections 2024-04-04T06:10:41.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.472+0000 7f8e5fe00640 1 --2- 172.21.15.159:0/1974875522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e580af2e0 0x7f8e5806d5b0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.472+0000 7f8e5fe00640 1 --2- 172.21.15.159:0/1974875522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e580d07c0 0x7f8e580aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.472+0000 7f8e5fe00640 1 --2- 172.21.15.159:0/1974875522 >> v2:172.21.15.178:3300/0 conn(0x7f8e58153580 0x7f8e580d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.472+0000 7f8e5fe00640 1 -- 172.21.15.159:0/1974875522 >> 172.21.15.159:0/1974875522 conn(0x7f8e5806b750 msgr2=0x7f8e5806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:41.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.472+0000 7f8e5fe00640 1 -- 172.21.15.159:0/1974875522 shutdown_connections 2024-04-04T06:10:41.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.472+0000 7f8e5fe00640 1 -- 172.21.15.159:0/1974875522 wait complete. 2024-04-04T06:10:41.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.474+0000 7f8e5fe00640 1 Processor -- start 2024-04-04T06:10:41.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.474+0000 7f8e5fe00640 1 -- start start 2024-04-04T06:10:41.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.475+0000 7f8e5fe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e580af2e0 0x7f8e5814d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:41.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.475+0000 7f8e5fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e580d07c0 0x7f8e581f5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:41.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.475+0000 7f8e5fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e58153580 0x7f8e5809da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:41.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.475+0000 7f8e5fe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5815d4e0 con 0x7f8e580d07c0 2024-04-04T06:10:41.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.475+0000 7f8e5fe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5815d200 con 0x7f8e58153580 2024-04-04T06:10:41.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.475+0000 7f8e5fe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8e58153960 con 0x7f8e580af2e0 2024-04-04T06:10:41.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.475+0000 7f8e5ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e580d07c0 0x7f8e581f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:41.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.475+0000 7f8e5ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e580d07c0 0x7f8e581f5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40798/0 (socket says 172.21.15.159:40798) 2024-04-04T06:10:41.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.475+0000 7f8e5ca00640 1 -- 172.21.15.159:0/3783468040 learned_addr learned my addr 172.21.15.159:0/3783468040 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:41.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.475+0000 7f8e5d400640 1 --2- 172.21.15.159:0/3783468040 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e580af2e0 0x7f8e5814d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:41.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.476+0000 7f8e5ca00640 1 -- 172.21.15.159:0/3783468040 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e580af2e0 msgr2=0x7f8e5814d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:41.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.476+0000 7f8e5de00640 1 --2- 172.21.15.159:0/3783468040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e58153580 0x7f8e5809da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:41.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.476+0000 7f8e5ca00640 1 --2- 172.21.15.159:0/3783468040 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e580af2e0 0x7f8e5814d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.476+0000 7f8e5ca00640 1 -- 172.21.15.159:0/3783468040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e58153580 msgr2=0x7f8e5809da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:41.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.476+0000 7f8e5ca00640 1 --2- 172.21.15.159:0/3783468040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e58153580 0x7f8e5809da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.476+0000 7f8e5ca00640 1 -- 172.21.15.159:0/3783468040 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5814e350 con 0x7f8e580d07c0 2024-04-04T06:10:41.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.476+0000 7f8e5ca00640 1 --2- 172.21.15.159:0/3783468040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e580d07c0 0x7f8e581f5130 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400d800 tx=0x7f8e4400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:41.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.476+0000 7f8e55600640 1 -- 172.21.15.159:0/3783468040 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44018030 con 0x7f8e580d07c0 2024-04-04T06:10:41.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.476+0000 7f8e55600640 1 -- 172.21.15.159:0/3783468040 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e440096a0 con 0x7f8e580d07c0 2024-04-04T06:10:41.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.476+0000 7f8e55600640 1 -- 172.21.15.159:0/3783468040 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44009870 con 0x7f8e580d07c0 2024-04-04T06:10:41.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.476+0000 7f8e5fe00640 1 -- 172.21.15.159:0/3783468040 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e5809df70 con 0x7f8e580d07c0 2024-04-04T06:10:41.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.476+0000 7f8e5fe00640 1 -- 172.21.15.159:0/3783468040 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e581f4650 con 0x7f8e580d07c0 2024-04-04T06:10:41.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.478+0000 7f8e5fe00640 1 -- 172.21.15.159:0/3783468040 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e580685c0 con 0x7f8e580d07c0 2024-04-04T06:10:41.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.479+0000 7f8e55600640 1 -- 172.21.15.159:0/3783468040 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e4401b070 con 0x7f8e580d07c0 2024-04-04T06:10:41.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.479+0000 7f8e55600640 1 --2- 172.21.15.159:0/3783468040 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8e3806c8f0 0x7f8e3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:41.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.480+0000 7f8e55600640 1 -- 172.21.15.159:0/3783468040 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4857+0+0 (secure 0 0 0) 0x7f8e4408ba70 con 0x7f8e580d07c0 2024-04-04T06:10:41.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.480+0000 7f8e5d400640 1 --2- 172.21.15.159:0/3783468040 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8e3806c8f0 0x7f8e3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:41.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.481+0000 7f8e5d400640 1 --2- 172.21.15.159:0/3783468040 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8e3806c8f0 0x7f8e3806edb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c006fd0 tx=0x7f8e4c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:41.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.484+0000 7f8e55600640 1 -- 172.21.15.159:0/3783468040 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8e4405a1e0 con 0x7f8e580d07c0 2024-04-04T06:10:41.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.683+0000 7f8e5fe00640 1 -- 172.21.15.159:0/3783468040 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8e5806eca0 con 0x7f8e580d07c0 2024-04-04T06:10:41.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.685+0000 7f8e55600640 1 -- 172.21.15.159:0/3783468040 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) ==== 93+0+10 (secure 0 0 0) 0x7f8e44059ae0 con 0x7f8e580d07c0 2024-04-04T06:10:41.678 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:10:41.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.687+0000 7f8e5fe00640 1 -- 172.21.15.159:0/3783468040 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8e3806c8f0 msgr2=0x7f8e3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:41.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.687+0000 7f8e5fe00640 1 --2- 172.21.15.159:0/3783468040 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8e3806c8f0 0x7f8e3806edb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c006fd0 tx=0x7f8e4c008040 comp rx=0 tx=0).stop 2024-04-04T06:10:41.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.687+0000 7f8e5fe00640 1 -- 172.21.15.159:0/3783468040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e580d07c0 msgr2=0x7f8e581f5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:41.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.687+0000 7f8e5fe00640 1 --2- 172.21.15.159:0/3783468040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e580d07c0 0x7f8e581f5130 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400d800 tx=0x7f8e4400dcc0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.687+0000 7f8e5fe00640 1 -- 172.21.15.159:0/3783468040 shutdown_connections 2024-04-04T06:10:41.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.687+0000 7f8e5fe00640 1 --2- 172.21.15.159:0/3783468040 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8e3806c8f0 0x7f8e3806edb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.687+0000 7f8e5fe00640 1 --2- 172.21.15.159:0/3783468040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e58153580 0x7f8e5809da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.687+0000 7f8e5fe00640 1 --2- 172.21.15.159:0/3783468040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e580d07c0 0x7f8e581f5130 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.687+0000 7f8e5fe00640 1 --2- 172.21.15.159:0/3783468040 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e580af2e0 0x7f8e5814d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.687+0000 7f8e5fe00640 1 -- 172.21.15.159:0/3783468040 >> 172.21.15.159:0/3783468040 conn(0x7f8e5806b750 msgr2=0x7f8e580b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:41.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.687+0000 7f8e5fe00640 1 -- 172.21.15.159:0/3783468040 shutdown_connections 2024-04-04T06:10:41.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.688+0000 7f8e5fe00640 1 -- 172.21.15.159:0/3783468040 wait complete. 2024-04-04T06:10:41.700 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-04T06:10:41.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.906+0000 7f4739200640 1 Processor -- start 2024-04-04T06:10:41.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.906+0000 7f4739200640 1 -- start start 2024-04-04T06:10:41.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.908+0000 7f4739200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4734072f50 0x7f47340b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:41.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.908+0000 7f4739200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4734144eb0 con 0x7f4734144ad0 2024-04-04T06:10:41.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.908+0000 7f4739200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47340b36f0 con 0x7f473415f710 2024-04-04T06:10:41.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.908+0000 7f4739200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f47340cdad0 con 0x7f4734072f50 2024-04-04T06:10:41.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.908+0000 7f4733400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f4734144ad0 0x7f4734072840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:38044/0 (socket says 172.21.15.159:38044) 2024-04-04T06:10:41.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.908+0000 7f4733400640 1 -- 172.21.15.159:0/2546244213 learned_addr learned my addr 172.21.15.159:0/2546244213 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:41.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.908+0000 7f472aa00640 1 --2- 172.21.15.159:0/2546244213 >> v2:172.21.15.159:3300/0 conn(0x7f4734072f50 0x7f47340b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:41.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.908+0000 7f4732a00640 1 -- 172.21.15.159:0/2546244213 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 460647934 0 0) 0x7f4734144eb0 con 0x7f4734144ad0 2024-04-04T06:10:41.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.908+0000 7f4732a00640 1 -- 172.21.15.159:0/2546244213 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4718003520 con 0x7f4734144ad0 2024-04-04T06:10:41.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.909+0000 7f4732a00640 1 -- 172.21.15.159:0/2546244213 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3192265702 0 0) 0x7f4718003520 con 0x7f4734144ad0 2024-04-04T06:10:41.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.909+0000 7f4732a00640 1 -- 172.21.15.159:0/2546244213 >> v2:172.21.15.159:3300/0 conn(0x7f4734072f50 msgr2=0x7f47340b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:41.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.909+0000 7f4732a00640 1 --2- 172.21.15.159:0/2546244213 >> v2:172.21.15.159:3300/0 conn(0x7f4734072f50 0x7f47340b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.909+0000 7f4732a00640 1 -- 172.21.15.159:0/2546244213 >> v1:172.21.15.178:6789/0 conn(0x7f473415f710 legacy=0x7f47340cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:41.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.909+0000 7f4732a00640 1 -- 172.21.15.159:0/2546244213 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47340b39d0 con 0x7f4734144ad0 2024-04-04T06:10:41.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.909+0000 7f4732a00640 1 -- 172.21.15.159:0/2546244213 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f4724001670 con 0x7f4734144ad0 2024-04-04T06:10:41.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.909+0000 7f4732a00640 1 -- 172.21.15.159:0/2546244213 >> v1:172.21.15.159:6789/0 conn(0x7f4734144ad0 legacy=0x7f4734072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:41.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.909+0000 7f4732a00640 1 --2- 172.21.15.159:0/2546244213 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47180039b0 0x7f4718003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:41.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.909+0000 7f4732a00640 1 --2- 172.21.15.159:0/2546244213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4718004800 0x7f4718004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:41.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.909+0000 7f4732a00640 1 --2- 172.21.15.159:0/2546244213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47180051c0 0x7f471800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:41.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.909+0000 7f4732a00640 1 -- 172.21.15.159:0/2546244213 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f47340cdad0 con 0x7f47180051c0 2024-04-04T06:10:41.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.909+0000 7f4732a00640 1 -- 172.21.15.159:0/2546244213 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f471800c110 con 0x7f4718004800 2024-04-04T06:10:41.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.909+0000 7f4732a00640 1 -- 172.21.15.159:0/2546244213 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f471800c310 con 0x7f47180039b0 2024-04-04T06:10:41.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.909+0000 7f4733e00640 1 --2- 172.21.15.159:0/2546244213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47180051c0 0x7f471800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:41.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.910+0000 7f4733400640 1 --2- 172.21.15.159:0/2546244213 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47180039b0 0x7f4718003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:41.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.910+0000 7f4733e00640 1 -- 172.21.15.159:0/2546244213 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47180039b0 msgr2=0x7f4718003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:41.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.910+0000 7f4733e00640 1 --2- 172.21.15.159:0/2546244213 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47180039b0 0x7f4718003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.910+0000 7f4733e00640 1 -- 172.21.15.159:0/2546244213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4718004800 msgr2=0x7f4718004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:41.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.910+0000 7f4733e00640 1 --2- 172.21.15.159:0/2546244213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4718004800 0x7f4718004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.910+0000 7f4733e00640 1 -- 172.21.15.159:0/2546244213 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4718003520 con 0x7f47180051c0 2024-04-04T06:10:41.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.910+0000 7f4733400640 1 --2- 172.21.15.159:0/2546244213 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47180039b0 0x7f4718003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:41.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.911+0000 7f4733e00640 1 -- 172.21.15.159:0/2546244213 reap_dead start 2024-04-04T06:10:41.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.911+0000 7f4733e00640 1 --2- 172.21.15.159:0/2546244213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47180051c0 0x7f471800b9c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f472c001e50 tx=0x7f4718005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463e57b5b4e8899d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:41.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.911+0000 7f4732a00640 1 -- 172.21.15.159:0/2546244213 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c001a70 con 0x7f47180051c0 2024-04-04T06:10:41.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.911+0000 7f4732a00640 1 -- 172.21.15.159:0/2546244213 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f472c003b90 con 0x7f47180051c0 2024-04-04T06:10:41.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.911+0000 7f4732a00640 1 -- 172.21.15.159:0/2546244213 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c0050d0 con 0x7f47180051c0 2024-04-04T06:10:41.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.912+0000 7f4739200640 1 -- 172.21.15.159:0/2546244213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47180051c0 msgr2=0x7f471800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:41.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.912+0000 7f4739200640 1 --2- 172.21.15.159:0/2546244213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47180051c0 0x7f471800b9c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f472c001e50 tx=0x7f4718005830 comp rx=0 tx=0).stop 2024-04-04T06:10:41.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.912+0000 7f4739200640 1 -- 172.21.15.159:0/2546244213 shutdown_connections 2024-04-04T06:10:41.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.912+0000 7f4739200640 1 --2- 172.21.15.159:0/2546244213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47180051c0 0x7f471800b9c0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.912+0000 7f4739200640 1 -- 172.21.15.159:0/2546244213 >> 172.21.15.159:0/2546244213 conn(0x7f473406b570 msgr2=0x7f473406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:41.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.912+0000 7f4739200640 1 -- 172.21.15.159:0/2546244213 shutdown_connections 2024-04-04T06:10:41.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.912+0000 7f4739200640 1 -- 172.21.15.159:0/2546244213 wait complete. 2024-04-04T06:10:41.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.914+0000 7f4739200640 1 Processor -- start 2024-04-04T06:10:41.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.914+0000 7f4739200640 1 -- start start 2024-04-04T06:10:41.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.914+0000 7f4739200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47180051c0 0x7f473415abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:41.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.915+0000 7f4739200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f473413f870 0x7f47341404e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:41.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.915+0000 7f4739200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f473413caf0 0x7f473415bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:41.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.915+0000 7f4739200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f47340b3f90 con 0x7f47180051c0 2024-04-04T06:10:41.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.915+0000 7f4739200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f473415fb40 con 0x7f473413caf0 2024-04-04T06:10:41.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.915+0000 7f4739200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f47340b2ff0 con 0x7f473413f870 2024-04-04T06:10:41.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.915+0000 7f4733400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47180051c0 0x7f473415abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:41.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.915+0000 7f4733400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47180051c0 0x7f473415abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40828/0 (socket says 172.21.15.159:40828) 2024-04-04T06:10:41.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.915+0000 7f4733400640 1 -- 172.21.15.159:0/3744193848 learned_addr learned my addr 172.21.15.159:0/3744193848 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:41.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.915+0000 7f472aa00640 1 --2- 172.21.15.159:0/3744193848 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f473413f870 0x7f47341404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:41.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.915+0000 7f4733400640 1 -- 172.21.15.159:0/3744193848 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f473413f870 msgr2=0x7f47341404e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:41.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.915+0000 7f4733400640 1 --2- 172.21.15.159:0/3744193848 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f473413f870 0x7f47341404e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.915+0000 7f4733400640 1 -- 172.21.15.159:0/3744193848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f473413caf0 msgr2=0x7f473415bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:41.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.915+0000 7f4733400640 1 --2- 172.21.15.159:0/3744193848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f473413caf0 0x7f473415bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:41.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.915+0000 7f4733400640 1 -- 172.21.15.159:0/3744193848 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473413ba50 con 0x7f47180051c0 2024-04-04T06:10:41.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.915+0000 7f472aa00640 1 --2- 172.21.15.159:0/3744193848 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f473413f870 0x7f47341404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:41.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.916+0000 7f4733400640 1 --2- 172.21.15.159:0/3744193848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47180051c0 0x7f473415abb0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f472400d8e0 tx=0x7f472400dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:41.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.916+0000 7f472be00640 1 -- 172.21.15.159:0/3744193848 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724019070 con 0x7f47180051c0 2024-04-04T06:10:41.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.916+0000 7f472be00640 1 -- 172.21.15.159:0/3744193848 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47240074e0 con 0x7f47180051c0 2024-04-04T06:10:41.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.916+0000 7f472be00640 1 -- 172.21.15.159:0/3744193848 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724004a20 con 0x7f47180051c0 2024-04-04T06:10:41.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.916+0000 7f4739200640 1 -- 172.21.15.159:0/3744193848 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f473413bd30 con 0x7f47180051c0 2024-04-04T06:10:41.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.916+0000 7f4739200640 1 -- 172.21.15.159:0/3744193848 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f473413c2f0 con 0x7f47180051c0 2024-04-04T06:10:41.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.917+0000 7f4739200640 1 -- 172.21.15.159:0/3744193848 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4734143790 con 0x7f47180051c0 2024-04-04T06:10:41.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.918+0000 7f472be00640 1 -- 172.21.15.159:0/3744193848 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f472401c070 con 0x7f47180051c0 2024-04-04T06:10:41.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.919+0000 7f472be00640 1 --2- 172.21.15.159:0/3744193848 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f471006c920 0x7f471006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:41.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.919+0000 7f472be00640 1 -- 172.21.15.159:0/3744193848 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4857+0+0 (secure 0 0 0) 0x7f472408b3f0 con 0x7f47180051c0 2024-04-04T06:10:41.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.920+0000 7f472aa00640 1 --2- 172.21.15.159:0/3744193848 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f471006c920 0x7f471006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:41.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.921+0000 7f472aa00640 1 --2- 172.21.15.159:0/3744193848 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f471006c920 0x7f471006ede0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f471c006a90 tx=0x7f471c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:41.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:41.923+0000 7f472be00640 1 -- 172.21.15.159:0/3744193848 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4734143790 con 0x7f47180051c0 2024-04-04T06:10:41.925 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:10:41.935 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:10:42.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.149+0000 7f4739200640 1 -- 172.21.15.159:0/3744193848 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f4734075940 con 0x7f47180051c0 2024-04-04T06:10:42.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.150+0000 7f472be00640 1 -- 172.21.15.159:0/3744193848 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v67) ==== 84+0+65 (secure 0 0 0) 0x7f4724059ad0 con 0x7f47180051c0 2024-04-04T06:10:42.143 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:42.143 INFO:teuthology.orchestra.run.smithi159.stdout:[ 2024-04-04T06:10:42.143 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:10:42.143 INFO:teuthology.orchestra.run.smithi159.stdout: "poolnum": 1, 2024-04-04T06:10:42.143 INFO:teuthology.orchestra.run.smithi159.stdout: "poolname": ".mgr" 2024-04-04T06:10:42.144 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:10:42.144 INFO:teuthology.orchestra.run.smithi159.stdout:] 2024-04-04T06:10:42.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.153+0000 7f4739200640 1 -- 172.21.15.159:0/3744193848 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f471006c920 msgr2=0x7f471006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:42.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.153+0000 7f4739200640 1 --2- 172.21.15.159:0/3744193848 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f471006c920 0x7f471006ede0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f471c006a90 tx=0x7f471c008040 comp rx=0 tx=0).stop 2024-04-04T06:10:42.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.153+0000 7f4739200640 1 -- 172.21.15.159:0/3744193848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47180051c0 msgr2=0x7f473415abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:42.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.153+0000 7f4739200640 1 --2- 172.21.15.159:0/3744193848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47180051c0 0x7f473415abb0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f472400d8e0 tx=0x7f472400dda0 comp rx=0 tx=0).stop 2024-04-04T06:10:42.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.153+0000 7f4739200640 1 -- 172.21.15.159:0/3744193848 shutdown_connections 2024-04-04T06:10:42.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.153+0000 7f4739200640 1 --2- 172.21.15.159:0/3744193848 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f471006c920 0x7f471006ede0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:42.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.153+0000 7f4739200640 1 --2- 172.21.15.159:0/3744193848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f473413caf0 0x7f473415bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:42.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.153+0000 7f4739200640 1 --2- 172.21.15.159:0/3744193848 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f473413f870 0x7f47341404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:42.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.153+0000 7f4739200640 1 --2- 172.21.15.159:0/3744193848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47180051c0 0x7f473415abb0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:42.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.153+0000 7f4739200640 1 -- 172.21.15.159:0/3744193848 >> 172.21.15.159:0/3744193848 conn(0x7f473406b570 msgr2=0x7f47340b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:42.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.153+0000 7f4739200640 1 -- 172.21.15.159:0/3744193848 shutdown_connections 2024-04-04T06:10:42.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.153+0000 7f4739200640 1 -- 172.21.15.159:0/3744193848 wait complete. 2024-04-04T06:10:42.167 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-04T06:10:42.167 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-04T06:10:42.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.377+0000 7f7377e00640 1 Processor -- start 2024-04-04T06:10:42.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.378+0000 7f7377e00640 1 -- start start 2024-04-04T06:10:42.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.379+0000 7f7377e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7378147ce0 0x7f73780b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:42.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.379+0000 7f7377e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7378162b80 0x7f737815f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:42.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.379+0000 7f7377e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f737806b570 0x7f737806ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:42.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.379+0000 7f7377e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f73780b3f20 con 0x7f7378147ce0 2024-04-04T06:10:42.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.379+0000 7f7377e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f73780b4200 con 0x7f737806b570 2024-04-04T06:10:42.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.379+0000 7f7377e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f737815ea70 con 0x7f7378162b80 2024-04-04T06:10:42.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.379+0000 7f7376a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7378147ce0 0x7f73780b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:42.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.379+0000 7f7376a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7378147ce0 0x7f73780b5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40836/0 (socket says 172.21.15.159:40836) 2024-04-04T06:10:42.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.379+0000 7f7376a00640 1 -- 172.21.15.159:0/3389742899 learned_addr learned my addr 172.21.15.159:0/3389742899 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:42.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.380+0000 7f7376a00640 1 -- 172.21.15.159:0/3389742899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7378162b80 msgr2=0x7f737815f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:42.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.380+0000 7f7376a00640 1 --2- 172.21.15.159:0/3389742899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7378162b80 0x7f737815f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:42.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.380+0000 7f7376a00640 1 -- 172.21.15.159:0/3389742899 >> v2:172.21.15.178:3300/0 conn(0x7f737806b570 msgr2=0x7f737806ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:42.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.380+0000 7f7376a00640 1 --2- 172.21.15.159:0/3389742899 >> v2:172.21.15.178:3300/0 conn(0x7f737806b570 0x7f737806ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:42.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.380+0000 7f7376a00640 1 -- 172.21.15.159:0/3389742899 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737815ed50 con 0x7f7378147ce0 2024-04-04T06:10:42.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.381+0000 7f7376a00640 1 --2- 172.21.15.159:0/3389742899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7378147ce0 0x7f73780b5a50 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f7378074280 tx=0x7f737002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46710039afc8cbf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:42.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.381+0000 7f7375600640 1 -- 172.21.15.159:0/3389742899 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737002c9e0 con 0x7f7378147ce0 2024-04-04T06:10:42.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.381+0000 7f7375600640 1 -- 172.21.15.159:0/3389742899 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f737002cb80 con 0x7f7378147ce0 2024-04-04T06:10:42.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.381+0000 7f7375600640 1 -- 172.21.15.159:0/3389742899 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73700343f0 con 0x7f7378147ce0 2024-04-04T06:10:42.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.382+0000 7f7377e00640 1 -- 172.21.15.159:0/3389742899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7378147ce0 msgr2=0x7f73780b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:42.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.382+0000 7f7377e00640 1 --2- 172.21.15.159:0/3389742899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7378147ce0 0x7f73780b5a50 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f7378074280 tx=0x7f737002be40 comp rx=0 tx=0).stop 2024-04-04T06:10:42.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.382+0000 7f7377e00640 1 -- 172.21.15.159:0/3389742899 shutdown_connections 2024-04-04T06:10:42.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.382+0000 7f7377e00640 1 --2- 172.21.15.159:0/3389742899 >> v2:172.21.15.178:3300/0 conn(0x7f737806b570 0x7f737806ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:42.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.382+0000 7f7377e00640 1 --2- 172.21.15.159:0/3389742899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7378162b80 0x7f737815f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:42.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.382+0000 7f7377e00640 1 --2- 172.21.15.159:0/3389742899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7378147ce0 0x7f73780b5a50 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:42.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.382+0000 7f7377e00640 1 -- 172.21.15.159:0/3389742899 >> 172.21.15.159:0/3389742899 conn(0x7f73780b2af0 msgr2=0x7f73780b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:42.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.382+0000 7f7377e00640 1 -- 172.21.15.159:0/3389742899 shutdown_connections 2024-04-04T06:10:42.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.382+0000 7f7377e00640 1 -- 172.21.15.159:0/3389742899 wait complete. 2024-04-04T06:10:42.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.383+0000 7f7377e00640 1 Processor -- start 2024-04-04T06:10:42.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.384+0000 7f7377e00640 1 -- start start 2024-04-04T06:10:42.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.384+0000 7f7377e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f737806b570 0x7f737815dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:42.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.384+0000 7f7377e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7378147ce0 0x7f7378140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:42.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.384+0000 7f7377e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7378162b80 0x7f73781406a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:42.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.385+0000 7f7377e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f73780b6110 con 0x7f7378162b80 2024-04-04T06:10:42.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.385+0000 7f7377e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73780b5f90 con 0x7f737806b570 2024-04-04T06:10:42.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.385+0000 7f7377e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f737815f7b0 con 0x7f7378147ce0 2024-04-04T06:10:42.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.385+0000 7f7377400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7378162b80 0x7f73781406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:42.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.385+0000 7f7377400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7378162b80 0x7f73781406a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40846/0 (socket says 172.21.15.159:40846) 2024-04-04T06:10:42.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.385+0000 7f7377400640 1 -- 172.21.15.159:0/2245328224 learned_addr learned my addr 172.21.15.159:0/2245328224 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:42.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.385+0000 7f7377400640 1 -- 172.21.15.159:0/2245328224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7378147ce0 msgr2=0x7f7378140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:42.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.385+0000 7f7377400640 1 --2- 172.21.15.159:0/2245328224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7378147ce0 0x7f7378140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:42.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.385+0000 7f7377400640 1 -- 172.21.15.159:0/2245328224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f737806b570 msgr2=0x7f737815dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:42.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.385+0000 7f7377400640 1 --2- 172.21.15.159:0/2245328224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f737806b570 0x7f737815dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:42.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.385+0000 7f7377400640 1 -- 172.21.15.159:0/2245328224 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7378140d30 con 0x7f7378162b80 2024-04-04T06:10:42.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.385+0000 7f7377400640 1 --2- 172.21.15.159:0/2245328224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7378162b80 0x7f73781406a0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f736800b410 tx=0x7f736800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:42.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.386+0000 7f736ea00640 1 -- 172.21.15.159:0/2245328224 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368015020 con 0x7f7378162b80 2024-04-04T06:10:42.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.386+0000 7f736ea00640 1 -- 172.21.15.159:0/2245328224 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7368011850 con 0x7f7378162b80 2024-04-04T06:10:42.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.386+0000 7f7377e00640 1 -- 172.21.15.159:0/2245328224 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7378141010 con 0x7f7378162b80 2024-04-04T06:10:42.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.386+0000 7f736ea00640 1 -- 172.21.15.159:0/2245328224 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368011c40 con 0x7f7378162b80 2024-04-04T06:10:42.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.386+0000 7f7377e00640 1 -- 172.21.15.159:0/2245328224 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73781415d0 con 0x7f7378162b80 2024-04-04T06:10:42.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.387+0000 7f7377e00640 1 -- 172.21.15.159:0/2245328224 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7378075940 con 0x7f7378162b80 2024-04-04T06:10:42.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.388+0000 7f736ea00640 1 -- 172.21.15.159:0/2245328224 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7368018020 con 0x7f7378162b80 2024-04-04T06:10:42.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.389+0000 7f736ea00640 1 --2- 172.21.15.159:0/2245328224 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f735406c8f0 0x7f735406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:42.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.389+0000 7f736ea00640 1 -- 172.21.15.159:0/2245328224 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4857+0+0 (secure 0 0 0) 0x7f736808b750 con 0x7f7378162b80 2024-04-04T06:10:42.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.389+0000 7f7376a00640 1 --2- 172.21.15.159:0/2245328224 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f735406c8f0 0x7f735406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:42.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.390+0000 7f7376a00640 1 --2- 172.21.15.159:0/2245328224 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f735406c8f0 0x7f735406edb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7370035000 tx=0x7f737002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:42.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.393+0000 7f736ea00640 1 -- 172.21.15.159:0/2245328224 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f736801f020 con 0x7f7378162b80 2024-04-04T06:10:42.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:42.582+0000 7f7377e00640 1 -- 172.21.15.159:0/2245328224 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7378068a50 con 0x7f7378162b80 2024-04-04T06:10:43.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.319+0000 7f736ea00640 1 -- 172.21.15.159:0/2245328224 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v68) ==== 138+0+0 (secure 0 0 0) 0x7f73680172c0 con 0x7f7378162b80 2024-04-04T06:10:43.313 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_metadata' created 2024-04-04T06:10:43.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.323+0000 7f7377e00640 1 -- 172.21.15.159:0/2245328224 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f735406c8f0 msgr2=0x7f735406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:43.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.323+0000 7f7377e00640 1 --2- 172.21.15.159:0/2245328224 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f735406c8f0 0x7f735406edb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7370035000 tx=0x7f737002e000 comp rx=0 tx=0).stop 2024-04-04T06:10:43.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.323+0000 7f7377e00640 1 -- 172.21.15.159:0/2245328224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7378162b80 msgr2=0x7f73781406a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:43.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.323+0000 7f7377e00640 1 --2- 172.21.15.159:0/2245328224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7378162b80 0x7f73781406a0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f736800b410 tx=0x7f736800b8d0 comp rx=0 tx=0).stop 2024-04-04T06:10:43.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.323+0000 7f7377e00640 1 -- 172.21.15.159:0/2245328224 shutdown_connections 2024-04-04T06:10:43.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.323+0000 7f7377e00640 1 --2- 172.21.15.159:0/2245328224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7378162b80 0x7f73781406a0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:43.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.323+0000 7f7377e00640 1 --2- 172.21.15.159:0/2245328224 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7378147ce0 0x7f7378140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:43.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.323+0000 7f7377e00640 1 --2- 172.21.15.159:0/2245328224 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f735406c8f0 0x7f735406edb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:43.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.323+0000 7f7377e00640 1 --2- 172.21.15.159:0/2245328224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f737806b570 0x7f737815dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:43.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.323+0000 7f7377e00640 1 -- 172.21.15.159:0/2245328224 >> 172.21.15.159:0/2245328224 conn(0x7f73780b2af0 msgr2=0x7f7378142d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:43.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.323+0000 7f7377e00640 1 -- 172.21.15.159:0/2245328224 shutdown_connections 2024-04-04T06:10:43.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.323+0000 7f7377e00640 1 -- 172.21.15.159:0/2245328224 wait complete. 2024-04-04T06:10:43.342 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-04T06:10:43.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.553+0000 7fc150400640 1 Processor -- start 2024-04-04T06:10:43.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.553+0000 7fc150400640 1 -- start start 2024-04-04T06:10:43.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.555+0000 7fc150400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc148155550 0x7fc148155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:43.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.555+0000 7fc150400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc14815f800 0x7fc1480d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:43.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.555+0000 7fc150400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1480d07c0 0x7fc1480aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:43.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.555+0000 7fc150400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fc1480d0c10 con 0x7fc14815f800 2024-04-04T06:10:43.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.555+0000 7fc150400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc1480b40b0 con 0x7fc148155550 2024-04-04T06:10:43.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.555+0000 7fc150400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc1480b4390 con 0x7fc1480d07c0 2024-04-04T06:10:43.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.555+0000 7fc14d000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc14815f800 0x7fc1480d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:43.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.555+0000 7fc14d000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc14815f800 0x7fc1480d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40852/0 (socket says 172.21.15.159:40852) 2024-04-04T06:10:43.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.555+0000 7fc14d000640 1 -- 172.21.15.159:0/2613568021 learned_addr learned my addr 172.21.15.159:0/2613568021 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:43.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.556+0000 7fc14da00640 1 --2- 172.21.15.159:0/2613568021 >> v2:172.21.15.178:3300/0 conn(0x7fc148155550 0x7fc148155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:43.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.556+0000 7fc14e400640 1 --2- 172.21.15.159:0/2613568021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1480d07c0 0x7fc1480aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:43.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.556+0000 7fc14d000640 1 -- 172.21.15.159:0/2613568021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1480d07c0 msgr2=0x7fc1480aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:43.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.556+0000 7fc14d000640 1 --2- 172.21.15.159:0/2613568021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1480d07c0 0x7fc1480aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:43.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.556+0000 7fc14d000640 1 -- 172.21.15.159:0/2613568021 >> v2:172.21.15.178:3300/0 conn(0x7fc148155550 msgr2=0x7fc148155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:43.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.556+0000 7fc14d000640 1 --2- 172.21.15.159:0/2613568021 >> v2:172.21.15.178:3300/0 conn(0x7fc148155550 0x7fc148155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:43.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.556+0000 7fc14d000640 1 -- 172.21.15.159:0/2613568021 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1480b4670 con 0x7fc14815f800 2024-04-04T06:10:43.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.556+0000 7fc14e400640 1 --2- 172.21.15.159:0/2613568021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1480d07c0 0x7fc1480aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:43.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.557+0000 7fc14d000640 1 --2- 172.21.15.159:0/2613568021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc14815f800 0x7fc1480d0280 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc148073ac0 tx=0x7fc13402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b61ef7687db46ae server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:43.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.557+0000 7fc147e00640 1 -- 172.21.15.159:0/2613568021 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13402ca80 con 0x7fc14815f800 2024-04-04T06:10:43.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.557+0000 7fc147e00640 1 -- 172.21.15.159:0/2613568021 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc13402cc20 con 0x7fc14815f800 2024-04-04T06:10:43.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.557+0000 7fc147e00640 1 -- 172.21.15.159:0/2613568021 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1340343f0 con 0x7fc14815f800 2024-04-04T06:10:43.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.558+0000 7fc150400640 1 -- 172.21.15.159:0/2613568021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc14815f800 msgr2=0x7fc1480d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:43.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.558+0000 7fc150400640 1 --2- 172.21.15.159:0/2613568021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc14815f800 0x7fc1480d0280 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc148073ac0 tx=0x7fc13402bee0 comp rx=0 tx=0).stop 2024-04-04T06:10:43.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.558+0000 7fc150400640 1 -- 172.21.15.159:0/2613568021 shutdown_connections 2024-04-04T06:10:43.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.558+0000 7fc150400640 1 --2- 172.21.15.159:0/2613568021 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1480d07c0 0x7fc1480aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:43.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.558+0000 7fc150400640 1 --2- 172.21.15.159:0/2613568021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc14815f800 0x7fc1480d0280 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:43.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.558+0000 7fc150400640 1 --2- 172.21.15.159:0/2613568021 >> v2:172.21.15.178:3300/0 conn(0x7fc148155550 0x7fc148155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:43.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.558+0000 7fc150400640 1 -- 172.21.15.159:0/2613568021 >> 172.21.15.159:0/2613568021 conn(0x7fc14806b750 msgr2=0x7fc14806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:43.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.558+0000 7fc150400640 1 -- 172.21.15.159:0/2613568021 shutdown_connections 2024-04-04T06:10:43.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.558+0000 7fc150400640 1 -- 172.21.15.159:0/2613568021 wait complete. 2024-04-04T06:10:43.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.560+0000 7fc150400640 1 Processor -- start 2024-04-04T06:10:43.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.560+0000 7fc150400640 1 -- start start 2024-04-04T06:10:43.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.561+0000 7fc150400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1480d07c0 0x7fc148145010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:43.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.561+0000 7fc150400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc148155550 0x7fc148154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:43.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.561+0000 7fc150400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc14815f800 0x7fc148154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:43.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.561+0000 7fc150400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc14815fc80 con 0x7fc148155550 2024-04-04T06:10:43.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.561+0000 7fc150400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc1480b4c30 con 0x7fc14815f800 2024-04-04T06:10:43.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.561+0000 7fc150400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc14815ba20 con 0x7fc1480d07c0 2024-04-04T06:10:43.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.561+0000 7fc14d000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc148155550 0x7fc148154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:43.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.561+0000 7fc14d000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc148155550 0x7fc148154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40860/0 (socket says 172.21.15.159:40860) 2024-04-04T06:10:43.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.561+0000 7fc14d000640 1 -- 172.21.15.159:0/16891812 learned_addr learned my addr 172.21.15.159:0/16891812 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:43.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.561+0000 7fc14da00640 1 --2- 172.21.15.159:0/16891812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1480d07c0 0x7fc148145010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:43.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.561+0000 7fc14e400640 1 --2- 172.21.15.159:0/16891812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc14815f800 0x7fc148154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:43.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.561+0000 7fc14d000640 1 -- 172.21.15.159:0/16891812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1480d07c0 msgr2=0x7fc148145010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:43.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.562+0000 7fc14d000640 1 --2- 172.21.15.159:0/16891812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1480d07c0 0x7fc148145010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:43.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.562+0000 7fc14d000640 1 -- 172.21.15.159:0/16891812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc14815f800 msgr2=0x7fc148154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:43.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.562+0000 7fc14d000640 1 --2- 172.21.15.159:0/16891812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc14815f800 0x7fc148154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:43.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.562+0000 7fc14d000640 1 -- 172.21.15.159:0/16891812 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14809b030 con 0x7fc148155550 2024-04-04T06:10:43.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.562+0000 7fc14e400640 1 --2- 172.21.15.159:0/16891812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc14815f800 0x7fc148154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:43.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.562+0000 7fc14d000640 1 --2- 172.21.15.159:0/16891812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc148155550 0x7fc148154010 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc134036040 tx=0x7fc134004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:43.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.562+0000 7fc145600640 1 -- 172.21.15.159:0/16891812 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc134043020 con 0x7fc148155550 2024-04-04T06:10:43.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.562+0000 7fc145600640 1 -- 172.21.15.159:0/16891812 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc13403e450 con 0x7fc148155550 2024-04-04T06:10:43.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.562+0000 7fc145600640 1 -- 172.21.15.159:0/16891812 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13403e820 con 0x7fc148155550 2024-04-04T06:10:43.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.562+0000 7fc150400640 1 -- 172.21.15.159:0/16891812 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc14809b310 con 0x7fc148155550 2024-04-04T06:10:43.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.562+0000 7fc150400640 1 -- 172.21.15.159:0/16891812 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc14809b8d0 con 0x7fc148155550 2024-04-04T06:10:43.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.564+0000 7fc150400640 1 -- 172.21.15.159:0/16891812 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1480685c0 con 0x7fc148155550 2024-04-04T06:10:43.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.565+0000 7fc145600640 1 -- 172.21.15.159:0/16891812 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc134046020 con 0x7fc148155550 2024-04-04T06:10:43.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.565+0000 7fc145600640 1 --2- 172.21.15.159:0/16891812 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc12806c8f0 0x7fc12806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:43.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.566+0000 7fc145600640 1 -- 172.21.15.159:0/16891812 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5227+0+0 (secure 0 0 0) 0x7fc1340ae3f0 con 0x7fc148155550 2024-04-04T06:10:43.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.566+0000 7fc14da00640 1 --2- 172.21.15.159:0/16891812 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc12806c8f0 0x7fc12806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:43.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.567+0000 7fc14da00640 1 --2- 172.21.15.159:0/16891812 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc12806c8f0 0x7fc12806edb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc13800a8f0 tx=0x7fc138008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:43.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.571+0000 7fc145600640 1 -- 172.21.15.159:0/16891812 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc13404d030 con 0x7fc148155550 2024-04-04T06:10:43.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:43.773+0000 7fc150400640 1 -- 172.21.15.159:0/16891812 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc1480b3ad0 con 0x7fc148155550 2024-04-04T06:10:44.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.325+0000 7fc145600640 1 -- 172.21.15.159:0/16891812 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v69) ==== 170+0+0 (secure 0 0 0) 0x7fc13407c970 con 0x7fc148155550 2024-04-04T06:10:44.319 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_data' created 2024-04-04T06:10:44.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.329+0000 7fc150400640 1 -- 172.21.15.159:0/16891812 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc12806c8f0 msgr2=0x7fc12806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:44.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.329+0000 7fc150400640 1 --2- 172.21.15.159:0/16891812 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc12806c8f0 0x7fc12806edb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc13800a8f0 tx=0x7fc138008040 comp rx=0 tx=0).stop 2024-04-04T06:10:44.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.329+0000 7fc150400640 1 -- 172.21.15.159:0/16891812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc148155550 msgr2=0x7fc148154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:44.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.329+0000 7fc150400640 1 --2- 172.21.15.159:0/16891812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc148155550 0x7fc148154010 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc134036040 tx=0x7fc134004a40 comp rx=0 tx=0).stop 2024-04-04T06:10:44.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.329+0000 7fc150400640 1 -- 172.21.15.159:0/16891812 shutdown_connections 2024-04-04T06:10:44.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.329+0000 7fc150400640 1 --2- 172.21.15.159:0/16891812 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc12806c8f0 0x7fc12806edb0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:44.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.329+0000 7fc150400640 1 --2- 172.21.15.159:0/16891812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc14815f800 0x7fc148154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:44.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.329+0000 7fc150400640 1 --2- 172.21.15.159:0/16891812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc148155550 0x7fc148154010 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:44.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.329+0000 7fc150400640 1 --2- 172.21.15.159:0/16891812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1480d07c0 0x7fc148145010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:44.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.329+0000 7fc150400640 1 -- 172.21.15.159:0/16891812 >> 172.21.15.159:0/16891812 conn(0x7fc14806b750 msgr2=0x7fc1480cd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:44.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.329+0000 7fc150400640 1 -- 172.21.15.159:0/16891812 shutdown_connections 2024-04-04T06:10:44.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.329+0000 7fc150400640 1 -- 172.21.15.159:0/16891812 wait complete. 2024-04-04T06:10:44.357 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-04T06:10:44.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.550+0000 7f21f7a00640 1 Processor -- start 2024-04-04T06:10:44.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.550+0000 7f21f7a00640 1 -- start start 2024-04-04T06:10:44.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.552+0000 7f21f7a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f21f0155550 0x7f21f0155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:44.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.552+0000 7f21f7a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21f00b40b0 con 0x7f21f00d0990 2024-04-04T06:10:44.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.552+0000 7f21f7a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21f00b4390 con 0x7f21f015f800 2024-04-04T06:10:44.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.552+0000 7f21f7a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f21f00b4670 con 0x7f21f0155550 2024-04-04T06:10:44.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.552+0000 7f21f5a00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f21f00d0990 0x7f21f00aede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:44362/0 (socket says 172.21.15.159:44362) 2024-04-04T06:10:44.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.552+0000 7f21f5a00640 1 -- 172.21.15.159:0/1497652972 learned_addr learned my addr 172.21.15.159:0/1497652972 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:44.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.552+0000 7f21f5000640 1 --2- 172.21.15.159:0/1497652972 >> v2:172.21.15.159:3300/0 conn(0x7f21f0155550 0x7f21f0155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:44.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.553+0000 7f21ef400640 1 -- 172.21.15.159:0/1497652972 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 530543465 0 0) 0x7f21f00b40b0 con 0x7f21f00d0990 2024-04-04T06:10:44.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.553+0000 7f21ef400640 1 -- 172.21.15.159:0/1497652972 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21d4005730 con 0x7f21f00d0990 2024-04-04T06:10:44.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.553+0000 7f21f5000640 1 -- 172.21.15.159:0/1497652972 >> v1:172.21.15.178:6789/0 conn(0x7f21f015f800 legacy=0x7f21f00d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:44.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.553+0000 7f21f5000640 1 -- 172.21.15.159:0/1497652972 >> v1:172.21.15.159:6789/0 conn(0x7f21f00d0990 legacy=0x7f21f00aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:44.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.553+0000 7f21f5000640 1 -- 172.21.15.159:0/1497652972 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f00b4950 con 0x7f21f0155550 2024-04-04T06:10:44.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.554+0000 7f21f5000640 1 --2- 172.21.15.159:0/1497652972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21f0155550 0x7f21f0155930 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009d90 tx=0x7f21e002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d096af056721ce8a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:44.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.554+0000 7f21ef400640 1 -- 172.21.15.159:0/1497652972 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0039070 con 0x7f21f0155550 2024-04-04T06:10:44.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.554+0000 7f21ef400640 1 -- 172.21.15.159:0/1497652972 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e002cbd0 con 0x7f21f0155550 2024-04-04T06:10:44.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.554+0000 7f21ef400640 1 -- 172.21.15.159:0/1497652972 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0034440 con 0x7f21f0155550 2024-04-04T06:10:44.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.555+0000 7f21f7a00640 1 -- 172.21.15.159:0/1497652972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21f0155550 msgr2=0x7f21f0155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:44.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.555+0000 7f21f7a00640 1 --2- 172.21.15.159:0/1497652972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21f0155550 0x7f21f0155930 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009d90 tx=0x7f21e002be20 comp rx=0 tx=0).stop 2024-04-04T06:10:44.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.555+0000 7f21f7a00640 1 -- 172.21.15.159:0/1497652972 shutdown_connections 2024-04-04T06:10:44.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.555+0000 7f21f7a00640 1 --2- 172.21.15.159:0/1497652972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21f0155550 0x7f21f0155930 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:44.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.555+0000 7f21f7a00640 1 -- 172.21.15.159:0/1497652972 >> 172.21.15.159:0/1497652972 conn(0x7f21f006b750 msgr2=0x7f21f006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:44.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.555+0000 7f21f7a00640 1 -- 172.21.15.159:0/1497652972 shutdown_connections 2024-04-04T06:10:44.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.555+0000 7f21f7a00640 1 -- 172.21.15.159:0/1497652972 wait complete. 2024-04-04T06:10:44.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.557+0000 7f21f7a00640 1 Processor -- start 2024-04-04T06:10:44.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.557+0000 7f21f7a00640 1 -- start start 2024-04-04T06:10:44.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.558+0000 7f21f7a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21f00d0990 0x7f21f015dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:44.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.558+0000 7f21f7a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21f0155550 0x7f21f015e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:44.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.558+0000 7f21f7a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21f015f800 0x7f21f01f0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:44.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.558+0000 7f21f7a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f21f015ba20 con 0x7f21f015f800 2024-04-04T06:10:44.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.558+0000 7f21f7a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f21f015fc80 con 0x7f21f0155550 2024-04-04T06:10:44.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.558+0000 7f21f7a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f21f015bba0 con 0x7f21f00d0990 2024-04-04T06:10:44.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.558+0000 7f21f5a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21f015f800 0x7f21f01f0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:44.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.558+0000 7f21f5a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21f015f800 0x7f21f01f0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60848/0 (socket says 172.21.15.159:60848) 2024-04-04T06:10:44.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.558+0000 7f21f5a00640 1 -- 172.21.15.159:0/3509257178 learned_addr learned my addr 172.21.15.159:0/3509257178 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:44.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.558+0000 7f21f5000640 1 --2- 172.21.15.159:0/3509257178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21f00d0990 0x7f21f015dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:44.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.558+0000 7f21efe00640 1 --2- 172.21.15.159:0/3509257178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21f0155550 0x7f21f015e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:44.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.559+0000 7f21f5a00640 1 -- 172.21.15.159:0/3509257178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21f00d0990 msgr2=0x7f21f015dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:44.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.559+0000 7f21f5a00640 1 --2- 172.21.15.159:0/3509257178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21f00d0990 0x7f21f015dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:44.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.559+0000 7f21f5a00640 1 -- 172.21.15.159:0/3509257178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21f0155550 msgr2=0x7f21f015e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:44.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.559+0000 7f21f5a00640 1 --2- 172.21.15.159:0/3509257178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21f0155550 0x7f21f015e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:44.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.559+0000 7f21f5a00640 1 -- 172.21.15.159:0/3509257178 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f015cd80 con 0x7f21f015f800 2024-04-04T06:10:44.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.559+0000 7f21f5a00640 1 --2- 172.21.15.159:0/3509257178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21f015f800 0x7f21f01f0180 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f21e800c3c0 tx=0x7f21e800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:44.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.559+0000 7f21ecc00640 1 -- 172.21.15.159:0/3509257178 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8017070 con 0x7f21f015f800 2024-04-04T06:10:44.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.559+0000 7f21ecc00640 1 -- 172.21.15.159:0/3509257178 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e8005ce0 con 0x7f21f015f800 2024-04-04T06:10:44.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.559+0000 7f21ecc00640 1 -- 172.21.15.159:0/3509257178 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e80072d0 con 0x7f21f015f800 2024-04-04T06:10:44.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.559+0000 7f21f7a00640 1 -- 172.21.15.159:0/3509257178 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f015caa0 con 0x7f21f015f800 2024-04-04T06:10:44.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.559+0000 7f21f7a00640 1 -- 172.21.15.159:0/3509257178 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f01f1a30 con 0x7f21f015f800 2024-04-04T06:10:44.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.560+0000 7f21f7a00640 1 -- 172.21.15.159:0/3509257178 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f00685c0 con 0x7f21f015f800 2024-04-04T06:10:44.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.562+0000 7f21ecc00640 1 -- 172.21.15.159:0/3509257178 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21e801a070 con 0x7f21f015f800 2024-04-04T06:10:44.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.563+0000 7f21ecc00640 1 --2- 172.21.15.159:0/3509257178 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f21d006c920 0x7f21d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:44.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.563+0000 7f21ecc00640 1 -- 172.21.15.159:0/3509257178 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5609+0+0 (secure 0 0 0) 0x7f21e808b4d0 con 0x7f21f015f800 2024-04-04T06:10:44.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.564+0000 7f21f5000640 1 --2- 172.21.15.159:0/3509257178 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f21d006c920 0x7f21d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:44.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.565+0000 7f21f5000640 1 --2- 172.21.15.159:0/3509257178 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f21d006c920 0x7f21d006ede0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f21e0036000 tx=0x7f21e0030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:44.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.567+0000 7f21ecc00640 1 -- 172.21.15.159:0/3509257178 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f21f015cd80 con 0x7f21f015f800 2024-04-04T06:10:44.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:44.803+0000 7f21f7a00640 1 -- 172.21.15.159:0/3509257178 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f21f0155330 con 0x7f21f015f800 2024-04-04T06:10:45.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.346+0000 7f21ecc00640 1 -- 172.21.15.159:0/3509257178 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v41) ==== 174+0+0 (secure 0 0 0) 0x7f21e80592f0 con 0x7f21f015f800 2024-04-04T06:10:45.339 INFO:teuthology.orchestra.run.smithi159.stderr:new fs with metadata pool 10 and data pool 11 2024-04-04T06:10:45.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.353+0000 7f21f7a00640 1 -- 172.21.15.159:0/3509257178 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f21d006c920 msgr2=0x7f21d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:45.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.354+0000 7f21f7a00640 1 --2- 172.21.15.159:0/3509257178 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f21d006c920 0x7f21d006ede0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f21e0036000 tx=0x7f21e0030000 comp rx=0 tx=0).stop 2024-04-04T06:10:45.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.354+0000 7f21f7a00640 1 -- 172.21.15.159:0/3509257178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21f015f800 msgr2=0x7f21f01f0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:45.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.354+0000 7f21f7a00640 1 --2- 172.21.15.159:0/3509257178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21f015f800 0x7f21f01f0180 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f21e800c3c0 tx=0x7f21e800c880 comp rx=0 tx=0).stop 2024-04-04T06:10:45.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.354+0000 7f21f7a00640 1 -- 172.21.15.159:0/3509257178 shutdown_connections 2024-04-04T06:10:45.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.354+0000 7f21f7a00640 1 --2- 172.21.15.159:0/3509257178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21f015f800 0x7f21f01f0180 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:45.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.354+0000 7f21f7a00640 1 --2- 172.21.15.159:0/3509257178 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f21d006c920 0x7f21d006ede0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:45.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.354+0000 7f21f7a00640 1 --2- 172.21.15.159:0/3509257178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21f0155550 0x7f21f015e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:45.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.354+0000 7f21f7a00640 1 --2- 172.21.15.159:0/3509257178 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21f00d0990 0x7f21f015dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:45.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.354+0000 7f21f7a00640 1 -- 172.21.15.159:0/3509257178 >> 172.21.15.159:0/3509257178 conn(0x7f21f006b750 msgr2=0x7f21f00b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:45.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.355+0000 7f21f7a00640 1 -- 172.21.15.159:0/3509257178 shutdown_connections 2024-04-04T06:10:45.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.355+0000 7f21f7a00640 1 -- 172.21.15.159:0/3509257178 wait complete. 2024-04-04T06:10:45.391 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:10:45.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.575+0000 7faa76200640 1 Processor -- start 2024-04-04T06:10:45.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.575+0000 7faa76200640 1 -- start start 2024-04-04T06:10:45.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.577+0000 7faa76200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7faa7006b570 0x7faa7006b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:45.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.577+0000 7faa76200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa7006bf00 con 0x7faa700726f0 2024-04-04T06:10:45.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.577+0000 7faa76200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa70148480 con 0x7faa701480a0 2024-04-04T06:10:45.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.577+0000 7faa76200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7faa700b3fa0 con 0x7faa7006b570 2024-04-04T06:10:45.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.577+0000 7faa6ea00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7faa700726f0 0x7faa700b3890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:44374/0 (socket says 172.21.15.159:44374) 2024-04-04T06:10:45.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.577+0000 7faa6ea00640 1 -- 172.21.15.159:0/964097438 learned_addr learned my addr 172.21.15.159:0/964097438 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:45.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.577+0000 7faa6fe00640 1 --2- 172.21.15.159:0/964097438 >> v2:172.21.15.178:3300/0 conn(0x7faa7006b570 0x7faa7006b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:45.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.577+0000 7faa6e000640 1 -- 172.21.15.159:0/964097438 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3574611721 0 0) 0x7faa7006bf00 con 0x7faa700726f0 2024-04-04T06:10:45.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.578+0000 7faa6e000640 1 -- 172.21.15.159:0/964097438 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa54003520 con 0x7faa700726f0 2024-04-04T06:10:45.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.578+0000 7faa6e000640 1 -- 172.21.15.159:0/964097438 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3974793689 0 0) 0x7faa54003520 con 0x7faa700726f0 2024-04-04T06:10:45.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.578+0000 7faa6e000640 1 -- 172.21.15.159:0/964097438 >> v2:172.21.15.178:3300/0 conn(0x7faa7006b570 msgr2=0x7faa7006b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:45.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.578+0000 7faa6e000640 1 --2- 172.21.15.159:0/964097438 >> v2:172.21.15.178:3300/0 conn(0x7faa7006b570 0x7faa7006b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:45.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.578+0000 7faa6e000640 1 -- 172.21.15.159:0/964097438 >> v1:172.21.15.178:6789/0 conn(0x7faa701480a0 legacy=0x7faa7006ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:45.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.578+0000 7faa6e000640 1 -- 172.21.15.159:0/964097438 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa700b41e0 con 0x7faa700726f0 2024-04-04T06:10:45.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.578+0000 7faa6e000640 1 -- 172.21.15.159:0/964097438 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7faa5c001670 con 0x7faa700726f0 2024-04-04T06:10:45.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.578+0000 7faa6e000640 1 -- 172.21.15.159:0/964097438 >> v1:172.21.15.159:6789/0 conn(0x7faa700726f0 legacy=0x7faa700b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:45.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.578+0000 7faa6e000640 1 --2- 172.21.15.159:0/964097438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa540039b0 0x7faa54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:45.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.578+0000 7faa6e000640 1 --2- 172.21.15.159:0/964097438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa54004800 0x7faa54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:45.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.578+0000 7faa6e000640 1 --2- 172.21.15.159:0/964097438 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa540051c0 0x7faa5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:45.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.578+0000 7faa6e000640 1 -- 172.21.15.159:0/964097438 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faa700b3fa0 con 0x7faa540039b0 2024-04-04T06:10:45.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.578+0000 7faa6e000640 1 -- 172.21.15.159:0/964097438 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faa5400c160 con 0x7faa54004800 2024-04-04T06:10:45.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.578+0000 7faa6e000640 1 -- 172.21.15.159:0/964097438 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7faa5400c360 con 0x7faa540051c0 2024-04-04T06:10:45.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.578+0000 7faa6f400640 1 --2- 172.21.15.159:0/964097438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa540039b0 0x7faa54003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:45.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.579+0000 7faa6ea00640 1 --2- 172.21.15.159:0/964097438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa54004800 0x7faa54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:45.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.579+0000 7faa6f400640 1 -- 172.21.15.159:0/964097438 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa540051c0 msgr2=0x7faa5400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:45.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.579+0000 7faa6f400640 1 --2- 172.21.15.159:0/964097438 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa540051c0 0x7faa5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:45.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.579+0000 7faa6f400640 1 -- 172.21.15.159:0/964097438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa54004800 msgr2=0x7faa54004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:45.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.579+0000 7faa6f400640 1 --2- 172.21.15.159:0/964097438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa54004800 0x7faa54004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:45.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.579+0000 7faa6f400640 1 -- 172.21.15.159:0/964097438 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa54003520 con 0x7faa540039b0 2024-04-04T06:10:45.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.579+0000 7faa6fe00640 1 -- 172.21.15.159:0/964097438 reap_dead start 2024-04-04T06:10:45.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.580+0000 7faa6f400640 1 --2- 172.21.15.159:0/964097438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa540039b0 0x7faa54003dd0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7faa60001750 tx=0x7faa54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb757b7e4e2b4223 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:45.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.580+0000 7faa6e000640 1 -- 172.21.15.159:0/964097438 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60001370 con 0x7faa540039b0 2024-04-04T06:10:45.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.580+0000 7faa6e000640 1 -- 172.21.15.159:0/964097438 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa600052d0 con 0x7faa540039b0 2024-04-04T06:10:45.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.580+0000 7faa6e000640 1 -- 172.21.15.159:0/964097438 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60003830 con 0x7faa540039b0 2024-04-04T06:10:45.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.581+0000 7faa76200640 1 -- 172.21.15.159:0/964097438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa540039b0 msgr2=0x7faa54003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:45.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.581+0000 7faa76200640 1 --2- 172.21.15.159:0/964097438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa540039b0 0x7faa54003dd0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7faa60001750 tx=0x7faa54005830 comp rx=0 tx=0).stop 2024-04-04T06:10:45.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.581+0000 7faa76200640 1 -- 172.21.15.159:0/964097438 shutdown_connections 2024-04-04T06:10:45.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.581+0000 7faa76200640 1 --2- 172.21.15.159:0/964097438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa540039b0 0x7faa54003dd0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:45.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.581+0000 7faa76200640 1 -- 172.21.15.159:0/964097438 >> 172.21.15.159:0/964097438 conn(0x7faa700b2230 msgr2=0x7faa700b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:45.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.581+0000 7faa76200640 1 -- 172.21.15.159:0/964097438 shutdown_connections 2024-04-04T06:10:45.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.581+0000 7faa76200640 1 -- 172.21.15.159:0/964097438 wait complete. 2024-04-04T06:10:45.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.583+0000 7faa76200640 1 Processor -- start 2024-04-04T06:10:45.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.583+0000 7faa76200640 1 -- start start 2024-04-04T06:10:45.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.584+0000 7faa76200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa540039b0 0x7faa700af560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:45.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.584+0000 7faa76200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa7013d570 0x7faa701610d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:45.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.584+0000 7faa76200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa70140950 0x7faa70140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:45.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.584+0000 7faa76200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faa7006e230 con 0x7faa70140950 2024-04-04T06:10:45.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.584+0000 7faa76200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faa7006e5a0 con 0x7faa540039b0 2024-04-04T06:10:45.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.584+0000 7faa76200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7faa700b44c0 con 0x7faa7013d570 2024-04-04T06:10:45.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.584+0000 7faa6fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa70140950 0x7faa70140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:45.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.584+0000 7faa6fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa70140950 0x7faa70140dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60870/0 (socket says 172.21.15.159:60870) 2024-04-04T06:10:45.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.584+0000 7faa6fe00640 1 -- 172.21.15.159:0/3820310034 learned_addr learned my addr 172.21.15.159:0/3820310034 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:45.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.584+0000 7faa6f400640 1 --2- 172.21.15.159:0/3820310034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa540039b0 0x7faa700af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:45.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.584+0000 7faa6ea00640 1 --2- 172.21.15.159:0/3820310034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa7013d570 0x7faa701610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:45.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.584+0000 7faa6fe00640 1 -- 172.21.15.159:0/3820310034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa7013d570 msgr2=0x7faa701610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:45.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.584+0000 7faa6fe00640 1 --2- 172.21.15.159:0/3820310034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa7013d570 0x7faa701610d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:45.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.584+0000 7faa6fe00640 1 -- 172.21.15.159:0/3820310034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa540039b0 msgr2=0x7faa700af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:45.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.584+0000 7faa6fe00640 1 --2- 172.21.15.159:0/3820310034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa540039b0 0x7faa700af560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:45.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.584+0000 7faa6fe00640 1 -- 172.21.15.159:0/3820310034 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa7013dad0 con 0x7faa70140950 2024-04-04T06:10:45.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.585+0000 7faa6fe00640 1 --2- 172.21.15.159:0/3820310034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa70140950 0x7faa70140dd0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7faa700d04d0 tx=0x7faa6800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:45.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.585+0000 7faa67400640 1 -- 172.21.15.159:0/3820310034 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6800ba80 con 0x7faa70140950 2024-04-04T06:10:45.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.585+0000 7faa67400640 1 -- 172.21.15.159:0/3820310034 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa6801a070 con 0x7faa70140950 2024-04-04T06:10:45.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.585+0000 7faa76200640 1 -- 172.21.15.159:0/3820310034 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa7013ddb0 con 0x7faa70140950 2024-04-04T06:10:45.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.585+0000 7faa67400640 1 -- 172.21.15.159:0/3820310034 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6800be40 con 0x7faa70140950 2024-04-04T06:10:45.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.585+0000 7faa76200640 1 -- 172.21.15.159:0/3820310034 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa7013e370 con 0x7faa70140950 2024-04-04T06:10:45.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.586+0000 7faa76200640 1 -- 172.21.15.159:0/3820310034 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa700685c0 con 0x7faa70140950 2024-04-04T06:10:45.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.587+0000 7faa67400640 1 -- 172.21.15.159:0/3820310034 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa68013070 con 0x7faa70140950 2024-04-04T06:10:45.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.589+0000 7faa67400640 1 --2- 172.21.15.159:0/3820310034 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faa5006c8f0 0x7faa5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:45.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.589+0000 7faa67400640 1 -- 172.21.15.159:0/3820310034 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7faa6808bbd0 con 0x7faa70140950 2024-04-04T06:10:45.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.589+0000 7faa6f400640 1 --2- 172.21.15.159:0/3820310034 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faa5006c8f0 0x7faa5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:45.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.591+0000 7faa6f400640 1 --2- 172.21.15.159:0/3820310034 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faa5006c8f0 0x7faa5006edb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7faa60033000 tx=0x7faa60009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:45.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.594+0000 7faa67400640 1 -- 172.21.15.159:0/3820310034 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7faa68059fe0 con 0x7faa70140950 2024-04-04T06:10:45.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.796+0000 7faa76200640 1 -- 172.21.15.159:0/3820310034 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa70074680 con 0x7faa70140950 2024-04-04T06:10:45.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.798+0000 7faa67400640 1 -- 172.21.15.159:0/3820310034 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16693 (secure 0 0 0) 0x7faa6805ee20 con 0x7faa70140950 2024-04-04T06:10:45.791 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:45.792 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":70,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:10:44.805077+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:10:42.595868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-04T06:10:43.788847+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:10:45.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.800+0000 7faa76200640 1 -- 172.21.15.159:0/3820310034 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faa5006c8f0 msgr2=0x7faa5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:45.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.800+0000 7faa76200640 1 --2- 172.21.15.159:0/3820310034 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faa5006c8f0 0x7faa5006edb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7faa60033000 tx=0x7faa60009030 comp rx=0 tx=0).stop 2024-04-04T06:10:45.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.800+0000 7faa76200640 1 -- 172.21.15.159:0/3820310034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa70140950 msgr2=0x7faa70140dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:45.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.800+0000 7faa76200640 1 --2- 172.21.15.159:0/3820310034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa70140950 0x7faa70140dd0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7faa700d04d0 tx=0x7faa6800b480 comp rx=0 tx=0).stop 2024-04-04T06:10:45.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.800+0000 7faa76200640 1 -- 172.21.15.159:0/3820310034 shutdown_connections 2024-04-04T06:10:45.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.800+0000 7faa76200640 1 --2- 172.21.15.159:0/3820310034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa70140950 0x7faa70140dd0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:45.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.800+0000 7faa76200640 1 --2- 172.21.15.159:0/3820310034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa7013d570 0x7faa701610d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:45.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.800+0000 7faa76200640 1 --2- 172.21.15.159:0/3820310034 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faa5006c8f0 0x7faa5006edb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:45.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.800+0000 7faa76200640 1 --2- 172.21.15.159:0/3820310034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa540039b0 0x7faa700af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:45.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.800+0000 7faa76200640 1 -- 172.21.15.159:0/3820310034 >> 172.21.15.159:0/3820310034 conn(0x7faa700b2230 msgr2=0x7faa70146ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:45.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.801+0000 7faa76200640 1 -- 172.21.15.159:0/3820310034 shutdown_connections 2024-04-04T06:10:45.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:45.801+0000 7faa76200640 1 -- 172.21.15.159:0/3820310034 wait complete. 2024-04-04T06:10:45.811 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:10:46.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.029+0000 7f0bcd600640 1 Processor -- start 2024-04-04T06:10:46.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.029+0000 7f0bcd600640 1 -- start start 2024-04-04T06:10:46.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.030+0000 7f0bcd600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0bc80cc0b0 0x7f0bc806dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:46.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.030+0000 7f0bcd600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bc815fc50 0x7f0bc80cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:46.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.030+0000 7f0bcd600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0bc80cc490 0x7f0bc80d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:46.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.030+0000 7f0bcd600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f0bc815c610 con 0x7f0bc80cc490 2024-04-04T06:10:46.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.030+0000 7f0bcd600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0bc806e4f0 con 0x7f0bc80cc0b0 2024-04-04T06:10:46.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.030+0000 7f0bcd600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc80b3150 con 0x7f0bc815fc50 2024-04-04T06:10:46.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.031+0000 7f0bccc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0bc80cc490 0x7f0bc80d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:46.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.031+0000 7f0bccc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0bc80cc490 0x7f0bc80d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60880/0 (socket says 172.21.15.159:60880) 2024-04-04T06:10:46.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.031+0000 7f0bccc00640 1 -- 172.21.15.159:0/1986592791 learned_addr learned my addr 172.21.15.159:0/1986592791 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:46.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.031+0000 7f0bbf400640 1 --2- 172.21.15.159:0/1986592791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bc815fc50 0x7f0bc80cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:46.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.031+0000 7f0bc7e00640 1 --2- 172.21.15.159:0/1986592791 >> v2:172.21.15.178:3300/0 conn(0x7f0bc80cc0b0 0x7f0bc806dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:46.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.031+0000 7f0bccc00640 1 -- 172.21.15.159:0/1986592791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bc815fc50 msgr2=0x7f0bc80cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:46.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.031+0000 7f0bccc00640 1 --2- 172.21.15.159:0/1986592791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bc815fc50 0x7f0bc80cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:46.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.031+0000 7f0bccc00640 1 -- 172.21.15.159:0/1986592791 >> v2:172.21.15.178:3300/0 conn(0x7f0bc80cc0b0 msgr2=0x7f0bc806dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:46.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.031+0000 7f0bccc00640 1 --2- 172.21.15.159:0/1986592791 >> v2:172.21.15.178:3300/0 conn(0x7f0bc80cc0b0 0x7f0bc806dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:46.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.031+0000 7f0bccc00640 1 -- 172.21.15.159:0/1986592791 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc81600e0 con 0x7f0bc80cc490 2024-04-04T06:10:46.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.032+0000 7f0bccc00640 1 --2- 172.21.15.159:0/1986592791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bc80cc490 0x7f0bc80d0300 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80cfd10 tx=0x7f0bc002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b326e4c33050ae52 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:46.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.033+0000 7f0bc7400640 1 -- 172.21.15.159:0/1986592791 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc002ca70 con 0x7f0bc80cc490 2024-04-04T06:10:46.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.033+0000 7f0bc7400640 1 -- 172.21.15.159:0/1986592791 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc002cc10 con 0x7f0bc80cc490 2024-04-04T06:10:46.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.033+0000 7f0bc7400640 1 -- 172.21.15.159:0/1986592791 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc00343f0 con 0x7f0bc80cc490 2024-04-04T06:10:46.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.033+0000 7f0bcd600640 1 -- 172.21.15.159:0/1986592791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bc80cc490 msgr2=0x7f0bc80d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:46.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.033+0000 7f0bcd600640 1 --2- 172.21.15.159:0/1986592791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bc80cc490 0x7f0bc80d0300 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80cfd10 tx=0x7f0bc002bed0 comp rx=0 tx=0).stop 2024-04-04T06:10:46.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.033+0000 7f0bcd600640 1 -- 172.21.15.159:0/1986592791 shutdown_connections 2024-04-04T06:10:46.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.033+0000 7f0bcd600640 1 --2- 172.21.15.159:0/1986592791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bc80cc490 0x7f0bc80d0300 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:46.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.033+0000 7f0bcd600640 1 --2- 172.21.15.159:0/1986592791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bc815fc50 0x7f0bc80cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:46.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.033+0000 7f0bcd600640 1 --2- 172.21.15.159:0/1986592791 >> v2:172.21.15.178:3300/0 conn(0x7f0bc80cc0b0 0x7f0bc806dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:46.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.033+0000 7f0bcd600640 1 -- 172.21.15.159:0/1986592791 >> 172.21.15.159:0/1986592791 conn(0x7f0bc80ccad0 msgr2=0x7f0bc80cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:46.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.033+0000 7f0bcd600640 1 -- 172.21.15.159:0/1986592791 shutdown_connections 2024-04-04T06:10:46.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.033+0000 7f0bcd600640 1 -- 172.21.15.159:0/1986592791 wait complete. 2024-04-04T06:10:46.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.035+0000 7f0bcd600640 1 Processor -- start 2024-04-04T06:10:46.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.035+0000 7f0bcd600640 1 -- start start 2024-04-04T06:10:46.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.036+0000 7f0bcd600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bc80cc0b0 0x7f0bc815ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:46.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.036+0000 7f0bcd600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bc80cc490 0x7f0bc80ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:46.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.036+0000 7f0bcd600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bc80ce2e0 0x7f0bc81e6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:46.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.036+0000 7f0bcd600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc8160980 con 0x7f0bc80cc490 2024-04-04T06:10:46.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.036+0000 7f0bcd600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc81606a0 con 0x7f0bc80ce2e0 2024-04-04T06:10:46.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.036+0000 7f0bcd600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc815ca70 con 0x7f0bc80cc0b0 2024-04-04T06:10:46.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.036+0000 7f0bbf400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bc80cc490 0x7f0bc80ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:46.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.036+0000 7f0bbf400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bc80cc490 0x7f0bc80ae2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60892/0 (socket says 172.21.15.159:60892) 2024-04-04T06:10:46.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.036+0000 7f0bbf400640 1 -- 172.21.15.159:0/2867772854 learned_addr learned my addr 172.21.15.159:0/2867772854 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:46.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.037+0000 7f0bc7e00640 1 --2- 172.21.15.159:0/2867772854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bc80cc0b0 0x7f0bc815ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:46.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.037+0000 7f0bccc00640 1 --2- 172.21.15.159:0/2867772854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bc80ce2e0 0x7f0bc81e6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:46.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.037+0000 7f0bbf400640 1 -- 172.21.15.159:0/2867772854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bc80cc0b0 msgr2=0x7f0bc815ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:46.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.037+0000 7f0bbf400640 1 --2- 172.21.15.159:0/2867772854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bc80cc0b0 0x7f0bc815ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:46.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.037+0000 7f0bbf400640 1 -- 172.21.15.159:0/2867772854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bc80ce2e0 msgr2=0x7f0bc81e6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:46.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.037+0000 7f0bbf400640 1 --2- 172.21.15.159:0/2867772854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bc80ce2e0 0x7f0bc81e6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:46.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.037+0000 7f0bbf400640 1 -- 172.21.15.159:0/2867772854 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc80aead0 con 0x7f0bc80cc490 2024-04-04T06:10:46.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.037+0000 7f0bccc00640 1 --2- 172.21.15.159:0/2867772854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bc80ce2e0 0x7f0bc81e6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:46.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.037+0000 7f0bbf400640 1 --2- 172.21.15.159:0/2867772854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bc80cc490 0x7f0bc80ae2a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000d5f0 tx=0x7f0bb000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:46.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.037+0000 7f0bc4c00640 1 -- 172.21.15.159:0/2867772854 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb00125f0 con 0x7f0bc80cc490 2024-04-04T06:10:46.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.037+0000 7f0bc4c00640 1 -- 172.21.15.159:0/2867772854 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb0019070 con 0x7f0bc80cc490 2024-04-04T06:10:46.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.037+0000 7f0bcd600640 1 -- 172.21.15.159:0/2867772854 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc80aedb0 con 0x7f0bc80cc490 2024-04-04T06:10:46.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.037+0000 7f0bc4c00640 1 -- 172.21.15.159:0/2867772854 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb00129d0 con 0x7f0bc80cc490 2024-04-04T06:10:46.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.037+0000 7f0bcd600640 1 -- 172.21.15.159:0/2867772854 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc80ae7e0 con 0x7f0bc80cc490 2024-04-04T06:10:46.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.039+0000 7f0bcd600640 1 -- 172.21.15.159:0/2867772854 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc8075940 con 0x7f0bc80cc490 2024-04-04T06:10:46.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.040+0000 7f0bc4c00640 1 -- 172.21.15.159:0/2867772854 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0bb0012b70 con 0x7f0bc80cc490 2024-04-04T06:10:46.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.040+0000 7f0bc4c00640 1 --2- 172.21.15.159:0/2867772854 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0ba406c8f0 0x7f0ba406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:46.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.040+0000 7f0bc4c00640 1 -- 172.21.15.159:0/2867772854 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f0bb008bd60 con 0x7f0bc80cc490 2024-04-04T06:10:46.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.041+0000 7f0bc7e00640 1 --2- 172.21.15.159:0/2867772854 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0ba406c8f0 0x7f0ba406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:46.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.042+0000 7f0bc7e00640 1 --2- 172.21.15.159:0/2867772854 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0ba406c8f0 0x7f0ba406edb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8006fd0 tx=0x7f0bb8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:46.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.045+0000 7f0bc4c00640 1 -- 172.21.15.159:0/2867772854 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0bb005a220 con 0x7f0bc80cc490 2024-04-04T06:10:46.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.236+0000 7f0bcd600640 1 -- 172.21.15.159:0/2867772854 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0bc8073830 con 0x7f0bc80cc490 2024-04-04T06:10:46.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.239+0000 7f0bc4c00640 1 -- 172.21.15.159:0/2867772854 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16693 (secure 0 0 0) 0x7f0bb0059b20 con 0x7f0bc80cc490 2024-04-04T06:10:46.233 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:46.233 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":70,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:10:44.805077+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:10:42.595868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-04T06:10:43.788847+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:10:46.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.242+0000 7f0bcd600640 1 -- 172.21.15.159:0/2867772854 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0ba406c8f0 msgr2=0x7f0ba406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:46.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.242+0000 7f0bcd600640 1 --2- 172.21.15.159:0/2867772854 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0ba406c8f0 0x7f0ba406edb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8006fd0 tx=0x7f0bb8008040 comp rx=0 tx=0).stop 2024-04-04T06:10:46.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.242+0000 7f0bcd600640 1 -- 172.21.15.159:0/2867772854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bc80cc490 msgr2=0x7f0bc80ae2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:46.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.242+0000 7f0bcd600640 1 --2- 172.21.15.159:0/2867772854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bc80cc490 0x7f0bc80ae2a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000d5f0 tx=0x7f0bb000dab0 comp rx=0 tx=0).stop 2024-04-04T06:10:46.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.242+0000 7f0bcd600640 1 -- 172.21.15.159:0/2867772854 shutdown_connections 2024-04-04T06:10:46.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.242+0000 7f0bcd600640 1 --2- 172.21.15.159:0/2867772854 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0ba406c8f0 0x7f0ba406edb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:46.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.242+0000 7f0bcd600640 1 --2- 172.21.15.159:0/2867772854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0bc80ce2e0 0x7f0bc81e6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:46.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.242+0000 7f0bcd600640 1 --2- 172.21.15.159:0/2867772854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bc80cc490 0x7f0bc80ae2a0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:46.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.242+0000 7f0bcd600640 1 --2- 172.21.15.159:0/2867772854 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0bc80cc0b0 0x7f0bc815ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:46.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.242+0000 7f0bcd600640 1 -- 172.21.15.159:0/2867772854 >> 172.21.15.159:0/2867772854 conn(0x7f0bc80ccad0 msgr2=0x7f0bc815e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:46.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.242+0000 7f0bcd600640 1 -- 172.21.15.159:0/2867772854 shutdown_connections 2024-04-04T06:10:46.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.243+0000 7f0bcd600640 1 -- 172.21.15.159:0/2867772854 wait complete. 2024-04-04T06:10:46.255 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-04T06:10:46.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.476+0000 7f996d000640 1 Processor -- start 2024-04-04T06:10:46.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.476+0000 7f996d000640 1 -- start start 2024-04-04T06:10:46.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.478+0000 7f996d000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f99680d37b0 0x7f99680b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:46.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.478+0000 7f996d000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99680b67f0 0x7f99680b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:46.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.478+0000 7f996d000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99680d2710 con 0x7f996815e4c0 2024-04-04T06:10:46.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.478+0000 7f996d000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f99680b6c60 con 0x7f99680d37b0 2024-04-04T06:10:46.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.478+0000 7f996d000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f996815e8a0 con 0x7f99680b67f0 2024-04-04T06:10:46.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.478+0000 7f9967e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99680b67f0 0x7f99680b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:46.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.478+0000 7f9967e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99680b67f0 0x7f99680b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:41776/0 (socket says 172.21.15.159:41776) 2024-04-04T06:10:46.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.478+0000 7f9967e00640 1 -- 172.21.15.159:0/834545359 learned_addr learned my addr 172.21.15.159:0/834545359 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:46.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.478+0000 7f9967400640 1 --1- 172.21.15.159:0/834545359 >> v1:172.21.15.178:6789/0 conn(0x7f996815e4c0 0x7f99680b9bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.159:39814/0 (socket says 172.21.15.159:39814) 2024-04-04T06:10:46.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.479+0000 7f995ea00640 1 --2- 172.21.15.159:0/834545359 >> v2:172.21.15.178:3300/0 conn(0x7f99680d37b0 0x7f99680b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:46.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.479+0000 7f9967e00640 1 -- 172.21.15.159:0/834545359 >> v2:172.21.15.178:3300/0 conn(0x7f99680d37b0 msgr2=0x7f99680b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:46.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.479+0000 7f9967e00640 1 --2- 172.21.15.159:0/834545359 >> v2:172.21.15.178:3300/0 conn(0x7f99680d37b0 0x7f99680b62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:46.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.479+0000 7f9967e00640 1 -- 172.21.15.159:0/834545359 >> v1:172.21.15.178:6789/0 conn(0x7f996815e4c0 legacy=0x7f99680b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:46.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.479+0000 7f9967e00640 1 -- 172.21.15.159:0/834545359 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99680d29f0 con 0x7f99680b67f0 2024-04-04T06:10:46.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.480+0000 7f9967e00640 1 --2- 172.21.15.159:0/834545359 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99680b67f0 0x7f99680b7400 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9960009870 tx=0x7f996002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=477e273bb88c19a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:46.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.481+0000 7f9966a00640 1 -- 172.21.15.159:0/834545359 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960034070 con 0x7f99680b67f0 2024-04-04T06:10:46.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.481+0000 7f9966a00640 1 -- 172.21.15.159:0/834545359 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9960030840 con 0x7f99680b67f0 2024-04-04T06:10:46.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.481+0000 7f9966a00640 1 -- 172.21.15.159:0/834545359 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960030c10 con 0x7f99680b67f0 2024-04-04T06:10:46.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.482+0000 7f996d000640 1 -- 172.21.15.159:0/834545359 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99680b67f0 msgr2=0x7f99680b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:46.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.482+0000 7f996d000640 1 --2- 172.21.15.159:0/834545359 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99680b67f0 0x7f99680b7400 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9960009870 tx=0x7f996002b980 comp rx=0 tx=0).stop 2024-04-04T06:10:46.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.482+0000 7f996d000640 1 -- 172.21.15.159:0/834545359 shutdown_connections 2024-04-04T06:10:46.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.482+0000 7f996d000640 1 --2- 172.21.15.159:0/834545359 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99680b67f0 0x7f99680b7400 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:46.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.482+0000 7f996d000640 1 --2- 172.21.15.159:0/834545359 >> v2:172.21.15.178:3300/0 conn(0x7f99680d37b0 0x7f99680b62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:46.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.482+0000 7f996d000640 1 -- 172.21.15.159:0/834545359 >> 172.21.15.159:0/834545359 conn(0x7f99680676a0 msgr2=0x7f9968067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:46.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.482+0000 7f996d000640 1 -- 172.21.15.159:0/834545359 shutdown_connections 2024-04-04T06:10:46.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.482+0000 7f996d000640 1 -- 172.21.15.159:0/834545359 wait complete. 2024-04-04T06:10:46.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.485+0000 7f996d000640 1 Processor -- start 2024-04-04T06:10:46.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.485+0000 7f996d000640 1 -- start start 2024-04-04T06:10:46.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.486+0000 7f996d000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99680b67f0 0x7f9968155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:46.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.486+0000 7f996d000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99680d37b0 0x7f99680cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:46.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.486+0000 7f996d000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f996815e4c0 0x7f99680c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:46.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.486+0000 7f996d000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f99680d3320 con 0x7f99680d37b0 2024-04-04T06:10:46.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.486+0000 7f996d000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f99680d4040 con 0x7f99680b67f0 2024-04-04T06:10:46.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.486+0000 7f996d000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f99680d2fb0 con 0x7f996815e4c0 2024-04-04T06:10:46.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.487+0000 7f995ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99680d37b0 0x7f99680cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:46.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.487+0000 7f995ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99680d37b0 0x7f99680cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60902/0 (socket says 172.21.15.159:60902) 2024-04-04T06:10:46.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.487+0000 7f995ea00640 1 -- 172.21.15.159:0/2886309341 learned_addr learned my addr 172.21.15.159:0/2886309341 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:46.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.487+0000 7f9967400640 1 --2- 172.21.15.159:0/2886309341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99680b67f0 0x7f9968155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:46.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.487+0000 7f995ea00640 1 -- 172.21.15.159:0/2886309341 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f996815e4c0 msgr2=0x7f99680c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:46.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.487+0000 7f995ea00640 1 --2- 172.21.15.159:0/2886309341 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f996815e4c0 0x7f99680c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:46.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.487+0000 7f995ea00640 1 -- 172.21.15.159:0/2886309341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99680b67f0 msgr2=0x7f9968155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:46.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.487+0000 7f995ea00640 1 --2- 172.21.15.159:0/2886309341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99680b67f0 0x7f9968155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:46.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.487+0000 7f995ea00640 1 -- 172.21.15.159:0/2886309341 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99680c7f60 con 0x7f99680d37b0 2024-04-04T06:10:46.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.487+0000 7f9967400640 1 --2- 172.21.15.159:0/2886309341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99680b67f0 0x7f9968155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:46.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.487+0000 7f995ea00640 1 --2- 172.21.15.159:0/2886309341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99680d37b0 0x7f99680cded0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f995000e9b0 tx=0x7f995000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:46.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.488+0000 7f995fe00640 1 -- 172.21.15.159:0/2886309341 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9950017070 con 0x7f99680d37b0 2024-04-04T06:10:46.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.488+0000 7f995fe00640 1 -- 172.21.15.159:0/2886309341 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f995000c9a0 con 0x7f99680d37b0 2024-04-04T06:10:46.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.488+0000 7f995fe00640 1 -- 172.21.15.159:0/2886309341 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995000cd90 con 0x7f99680d37b0 2024-04-04T06:10:46.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.488+0000 7f996d000640 1 -- 172.21.15.159:0/2886309341 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99680c8220 con 0x7f99680d37b0 2024-04-04T06:10:46.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.488+0000 7f996d000640 1 -- 172.21.15.159:0/2886309341 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99680c87e0 con 0x7f99680d37b0 2024-04-04T06:10:46.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.489+0000 7f996d000640 1 -- 172.21.15.159:0/2886309341 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9968075940 con 0x7f99680d37b0 2024-04-04T06:10:46.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.490+0000 7f995fe00640 1 -- 172.21.15.159:0/2886309341 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f995001a070 con 0x7f99680d37b0 2024-04-04T06:10:46.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.491+0000 7f995fe00640 1 --2- 172.21.15.159:0/2886309341 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f994406c8f0 0x7f994406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:46.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.491+0000 7f995fe00640 1 -- 172.21.15.159:0/2886309341 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f995008bb70 con 0x7f99680d37b0 2024-04-04T06:10:46.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.491+0000 7f9967400640 1 --2- 172.21.15.159:0/2886309341 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f994406c8f0 0x7f994406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:46.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.492+0000 7f9967400640 1 --2- 172.21.15.159:0/2886309341 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f994406c8f0 0x7f994406edb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9958007fd0 tx=0x7f9958009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:46.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.495+0000 7f995fe00640 1 -- 172.21.15.159:0/2886309341 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9950059f90 con 0x7f99680d37b0 2024-04-04T06:10:46.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:46.719+0000 7f996d000640 1 -- 172.21.15.159:0/2886309341 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f996806b570 con 0x7f99680d37b0 2024-04-04T06:10:47.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.383+0000 7f995fe00640 1 -- 172.21.15.159:0/2886309341 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v44) ==== 118+0+0 (secure 0 0 0) 0x7f9950059890 con 0x7f99680d37b0 2024-04-04T06:10:47.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.386+0000 7f996d000640 1 -- 172.21.15.159:0/2886309341 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f994406c8f0 msgr2=0x7f994406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:47.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.386+0000 7f996d000640 1 --2- 172.21.15.159:0/2886309341 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f994406c8f0 0x7f994406edb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9958007fd0 tx=0x7f9958009040 comp rx=0 tx=0).stop 2024-04-04T06:10:47.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.386+0000 7f996d000640 1 -- 172.21.15.159:0/2886309341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99680d37b0 msgr2=0x7f99680cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:47.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.386+0000 7f996d000640 1 --2- 172.21.15.159:0/2886309341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99680d37b0 0x7f99680cded0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f995000e9b0 tx=0x7f995000ee70 comp rx=0 tx=0).stop 2024-04-04T06:10:47.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.386+0000 7f996d000640 1 -- 172.21.15.159:0/2886309341 shutdown_connections 2024-04-04T06:10:47.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.386+0000 7f996d000640 1 --2- 172.21.15.159:0/2886309341 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f996815e4c0 0x7f99680c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:47.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.386+0000 7f996d000640 1 --2- 172.21.15.159:0/2886309341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99680d37b0 0x7f99680cded0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:47.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.386+0000 7f996d000640 1 --2- 172.21.15.159:0/2886309341 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f994406c8f0 0x7f994406edb0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:47.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.386+0000 7f996d000640 1 --2- 172.21.15.159:0/2886309341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99680b67f0 0x7f9968155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:47.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.386+0000 7f996d000640 1 -- 172.21.15.159:0/2886309341 >> 172.21.15.159:0/2886309341 conn(0x7f99680676a0 msgr2=0x7f996815b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:47.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.386+0000 7f996d000640 1 -- 172.21.15.159:0/2886309341 shutdown_connections 2024-04-04T06:10:47.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.387+0000 7f996d000640 1 -- 172.21.15.159:0/2886309341 wait complete. 2024-04-04T06:10:47.400 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-04T06:10:47.400 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-04T06:10:47.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.619+0000 7fa9d7200640 1 Processor -- start 2024-04-04T06:10:47.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.619+0000 7fa9d7200640 1 -- start start 2024-04-04T06:10:47.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.621+0000 7fa9d7200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa9d00b4a30 0x7fa9d00b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:47.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.621+0000 7fa9d7200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9d015ede0 con 0x7fa9d00b4650 2024-04-04T06:10:47.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.621+0000 7fa9d7200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9d0141d80 con 0x7fa9d0158580 2024-04-04T06:10:47.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.621+0000 7fa9d7200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa9d0141b30 con 0x7fa9d00b4a30 2024-04-04T06:10:47.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.621+0000 7fa9cfe00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fa9d00b4650 0x7fa9d006c680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:44388/0 (socket says 172.21.15.159:44388) 2024-04-04T06:10:47.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.621+0000 7fa9cfe00640 1 -- 172.21.15.159:0/819117845 learned_addr learned my addr 172.21.15.159:0/819117845 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:47.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.621+0000 7fa9cea00640 1 -- 172.21.15.159:0/819117845 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1544314824 0 0) 0x7fa9d015ede0 con 0x7fa9d00b4650 2024-04-04T06:10:47.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.621+0000 7fa9cea00640 1 -- 172.21.15.159:0/819117845 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9b4003520 con 0x7fa9d00b4650 2024-04-04T06:10:47.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.621+0000 7fa9cf400640 1 --2- 172.21.15.159:0/819117845 >> v2:172.21.15.178:3300/0 conn(0x7fa9d00b4a30 0x7fa9d00b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:47.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.622+0000 7fa9cea00640 1 -- 172.21.15.159:0/819117845 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4191768124 0 0) 0x7fa9b4003520 con 0x7fa9d00b4650 2024-04-04T06:10:47.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.622+0000 7fa9cea00640 1 -- 172.21.15.159:0/819117845 >> v2:172.21.15.178:3300/0 conn(0x7fa9d00b4a30 msgr2=0x7fa9d00b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:47.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.622+0000 7fa9cea00640 1 --2- 172.21.15.159:0/819117845 >> v2:172.21.15.178:3300/0 conn(0x7fa9d00b4a30 0x7fa9d00b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:47.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.622+0000 7fa9cea00640 1 -- 172.21.15.159:0/819117845 >> v1:172.21.15.178:6789/0 conn(0x7fa9d0158580 legacy=0x7fa9d01589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:47.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.622+0000 7fa9cea00640 1 -- 172.21.15.159:0/819117845 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d0142060 con 0x7fa9d00b4650 2024-04-04T06:10:47.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.622+0000 7fa9cea00640 1 -- 172.21.15.159:0/819117845 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fa9c0001690 con 0x7fa9d00b4650 2024-04-04T06:10:47.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.622+0000 7fa9cea00640 1 -- 172.21.15.159:0/819117845 >> v1:172.21.15.159:6789/0 conn(0x7fa9d00b4650 legacy=0x7fa9d006c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:47.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.622+0000 7fa9cea00640 1 --2- 172.21.15.159:0/819117845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9b4003a00 0x7fa9b4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:47.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.622+0000 7fa9cea00640 1 --2- 172.21.15.159:0/819117845 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9b4004800 0x7fa9b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:47.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.622+0000 7fa9cea00640 1 --2- 172.21.15.159:0/819117845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9b4005210 0x7fa9b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:47.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.622+0000 7fa9cea00640 1 -- 172.21.15.159:0/819117845 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d0141b30 con 0x7fa9b4003a00 2024-04-04T06:10:47.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.622+0000 7fa9cea00640 1 -- 172.21.15.159:0/819117845 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b400c140 con 0x7fa9b4005210 2024-04-04T06:10:47.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.622+0000 7fa9cea00640 1 -- 172.21.15.159:0/819117845 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa9b400c340 con 0x7fa9b4004800 2024-04-04T06:10:47.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.622+0000 7fa9cfe00640 1 --2- 172.21.15.159:0/819117845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9b4003a00 0x7fa9b4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:47.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.623+0000 7fa9d5200640 1 --2- 172.21.15.159:0/819117845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9b4005210 0x7fa9b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:47.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.623+0000 7fa9cf400640 1 --2- 172.21.15.159:0/819117845 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9b4004800 0x7fa9b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:47.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.623+0000 7fa9cfe00640 1 -- 172.21.15.159:0/819117845 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9b4004800 msgr2=0x7fa9b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:47.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.623+0000 7fa9cfe00640 1 --2- 172.21.15.159:0/819117845 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9b4004800 0x7fa9b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:47.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.623+0000 7fa9cfe00640 1 -- 172.21.15.159:0/819117845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9b4005210 msgr2=0x7fa9b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:47.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.623+0000 7fa9cfe00640 1 --2- 172.21.15.159:0/819117845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9b4005210 0x7fa9b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:47.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.623+0000 7fa9cfe00640 1 -- 172.21.15.159:0/819117845 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b4003520 con 0x7fa9b4003a00 2024-04-04T06:10:47.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.623+0000 7fa9d5200640 1 -- 172.21.15.159:0/819117845 reap_dead start 2024-04-04T06:10:47.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.625+0000 7fa9cfe00640 1 --2- 172.21.15.159:0/819117845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9b4003a00 0x7fa9b4003e00 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa9d015efb0 tx=0x7fa9b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cec7fdc61cd37bf server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:47.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.625+0000 7fa9cea00640 1 -- 172.21.15.159:0/819117845 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0007c80 con 0x7fa9b4003a00 2024-04-04T06:10:47.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.625+0000 7fa9cea00640 1 -- 172.21.15.159:0/819117845 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c0030070 con 0x7fa9b4003a00 2024-04-04T06:10:47.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.625+0000 7fa9cea00640 1 -- 172.21.15.159:0/819117845 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0005d10 con 0x7fa9b4003a00 2024-04-04T06:10:47.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.626+0000 7fa9d7200640 1 -- 172.21.15.159:0/819117845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9b4003a00 msgr2=0x7fa9b4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:47.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.626+0000 7fa9d7200640 1 --2- 172.21.15.159:0/819117845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9b4003a00 0x7fa9b4003e00 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa9d015efb0 tx=0x7fa9b4005830 comp rx=0 tx=0).stop 2024-04-04T06:10:47.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.626+0000 7fa9d7200640 1 -- 172.21.15.159:0/819117845 shutdown_connections 2024-04-04T06:10:47.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.626+0000 7fa9d7200640 1 --2- 172.21.15.159:0/819117845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9b4003a00 0x7fa9b4003e00 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:47.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.626+0000 7fa9d7200640 1 -- 172.21.15.159:0/819117845 >> 172.21.15.159:0/819117845 conn(0x7fa9d0067670 msgr2=0x7fa9d0067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:47.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.626+0000 7fa9d7200640 1 -- 172.21.15.159:0/819117845 shutdown_connections 2024-04-04T06:10:47.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.626+0000 7fa9d7200640 1 -- 172.21.15.159:0/819117845 wait complete. 2024-04-04T06:10:47.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.629+0000 7fa9d7200640 1 Processor -- start 2024-04-04T06:10:47.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.629+0000 7fa9d7200640 1 -- start start 2024-04-04T06:10:47.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.631+0000 7fa9d7200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9b4003a00 0x7fa9d00cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:47.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.631+0000 7fa9d7200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9d015b150 0x7fa9d015b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:47.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.631+0000 7fa9d7200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9d012f990 0x7fa9d012fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:47.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.631+0000 7fa9d7200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d0142900 con 0x7fa9b4003a00 2024-04-04T06:10:47.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.631+0000 7fa9d7200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d0142620 con 0x7fa9d015b150 2024-04-04T06:10:47.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.631+0000 7fa9d7200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d006c680 con 0x7fa9d012f990 2024-04-04T06:10:47.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.631+0000 7fa9cfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9b4003a00 0x7fa9d00cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:47.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.631+0000 7fa9cfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9b4003a00 0x7fa9d00cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60920/0 (socket says 172.21.15.159:60920) 2024-04-04T06:10:47.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.631+0000 7fa9cfe00640 1 -- 172.21.15.159:0/926254728 learned_addr learned my addr 172.21.15.159:0/926254728 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:47.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.631+0000 7fa9cf400640 1 --2- 172.21.15.159:0/926254728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9d015b150 0x7fa9d015b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:47.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.631+0000 7fa9cfe00640 1 -- 172.21.15.159:0/926254728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9d012f990 msgr2=0x7fa9d012fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:47.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.631+0000 7fa9cfe00640 1 --2- 172.21.15.159:0/926254728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9d012f990 0x7fa9d012fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:47.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.631+0000 7fa9cfe00640 1 -- 172.21.15.159:0/926254728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9d015b150 msgr2=0x7fa9d015b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:47.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.631+0000 7fa9cfe00640 1 --2- 172.21.15.159:0/926254728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9d015b150 0x7fa9d015b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:47.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.631+0000 7fa9cfe00640 1 -- 172.21.15.159:0/926254728 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d00ce380 con 0x7fa9b4003a00 2024-04-04T06:10:47.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.631+0000 7fa9cf400640 1 --2- 172.21.15.159:0/926254728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9d015b150 0x7fa9d015b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:47.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.632+0000 7fa9cfe00640 1 --2- 172.21.15.159:0/926254728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9b4003a00 0x7fa9d00cdd90 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000c220 tx=0x7fa9c002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:47.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.632+0000 7fa9c7e00640 1 -- 172.21.15.159:0/926254728 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0007c80 con 0x7fa9b4003a00 2024-04-04T06:10:47.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.632+0000 7fa9c7e00640 1 -- 172.21.15.159:0/926254728 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c0004bf0 con 0x7fa9b4003a00 2024-04-04T06:10:47.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.632+0000 7fa9d7200640 1 -- 172.21.15.159:0/926254728 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d015a390 con 0x7fa9b4003a00 2024-04-04T06:10:47.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.632+0000 7fa9c7e00640 1 -- 172.21.15.159:0/926254728 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0007070 con 0x7fa9b4003a00 2024-04-04T06:10:47.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.632+0000 7fa9d7200640 1 -- 172.21.15.159:0/926254728 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d015a950 con 0x7fa9b4003a00 2024-04-04T06:10:47.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.633+0000 7fa9d7200640 1 -- 172.21.15.159:0/926254728 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d0074f30 con 0x7fa9b4003a00 2024-04-04T06:10:47.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.634+0000 7fa9c7e00640 1 -- 172.21.15.159:0/926254728 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9c0048020 con 0x7fa9b4003a00 2024-04-04T06:10:47.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.635+0000 7fa9c7e00640 1 --2- 172.21.15.159:0/926254728 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa9b006c8f0 0x7fa9b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:47.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.635+0000 7fa9c7e00640 1 -- 172.21.15.159:0/926254728 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fa9c00acf00 con 0x7fa9b4003a00 2024-04-04T06:10:47.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.636+0000 7fa9cf400640 1 --2- 172.21.15.159:0/926254728 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa9b006c8f0 0x7fa9b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:47.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.637+0000 7fa9cf400640 1 --2- 172.21.15.159:0/926254728 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa9b006c8f0 0x7fa9b006edb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00d2f80 tx=0x7fa9bc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:47.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.640+0000 7fa9c7e00640 1 -- 172.21.15.159:0/926254728 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa9c007b320 con 0x7fa9b4003a00 2024-04-04T06:10:47.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:47.985+0000 7fa9d7200640 1 -- 172.21.15.159:0/926254728 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa9d00d2cd0 con 0x7fa9b4003a00 2024-04-04T06:10:48.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.385+0000 7fa9c7e00640 1 -- 172.21.15.159:0/926254728 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7fa9c007ac20 con 0x7fa9b4003a00 2024-04-04T06:10:48.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.389+0000 7fa9d7200640 1 -- 172.21.15.159:0/926254728 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa9b006c8f0 msgr2=0x7fa9b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:48.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.389+0000 7fa9d7200640 1 --2- 172.21.15.159:0/926254728 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa9b006c8f0 0x7fa9b006edb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00d2f80 tx=0x7fa9bc008040 comp rx=0 tx=0).stop 2024-04-04T06:10:48.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.389+0000 7fa9d7200640 1 -- 172.21.15.159:0/926254728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9b4003a00 msgr2=0x7fa9d00cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:48.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.389+0000 7fa9d7200640 1 --2- 172.21.15.159:0/926254728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9b4003a00 0x7fa9d00cdd90 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000c220 tx=0x7fa9c002e070 comp rx=0 tx=0).stop 2024-04-04T06:10:48.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.389+0000 7fa9d7200640 1 -- 172.21.15.159:0/926254728 shutdown_connections 2024-04-04T06:10:48.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.389+0000 7fa9d7200640 1 --2- 172.21.15.159:0/926254728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9d012f990 0x7fa9d012fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:48.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.389+0000 7fa9d7200640 1 --2- 172.21.15.159:0/926254728 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa9b006c8f0 0x7fa9b006edb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:48.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.389+0000 7fa9d7200640 1 --2- 172.21.15.159:0/926254728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9d015b150 0x7fa9d015b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:48.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.389+0000 7fa9d7200640 1 --2- 172.21.15.159:0/926254728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9b4003a00 0x7fa9d00cdd90 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:48.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.389+0000 7fa9d7200640 1 -- 172.21.15.159:0/926254728 >> 172.21.15.159:0/926254728 conn(0x7fa9d0067670 msgr2=0x7fa9d00b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:48.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.389+0000 7fa9d7200640 1 -- 172.21.15.159:0/926254728 shutdown_connections 2024-04-04T06:10:48.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.389+0000 7fa9d7200640 1 -- 172.21.15.159:0/926254728 wait complete. 2024-04-04T06:10:48.406 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-04T06:10:48.406 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:10:48.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.618+0000 7fc1d6c00640 1 Processor -- start 2024-04-04T06:10:48.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.618+0000 7fc1d6c00640 1 -- start start 2024-04-04T06:10:48.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.620+0000 7fc1d6c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1d00af830 0x7fc1d0073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:48.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.620+0000 7fc1d6c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc1d00d0650 0x7fc1d00d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:48.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.620+0000 7fc1d6c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1d015fe80 con 0x7fc1d015c390 2024-04-04T06:10:48.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.620+0000 7fc1d6c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fc1d015c770 con 0x7fc1d00d0650 2024-04-04T06:10:48.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.620+0000 7fc1d6c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d00ad120 con 0x7fc1d00af830 2024-04-04T06:10:48.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.620+0000 7fc1d4c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc1d00d0650 0x7fc1d00d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:48.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.620+0000 7fc1cfe00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fc1d015c390 0x7fc1d0160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:44404/0 (socket says 172.21.15.159:44404) 2024-04-04T06:10:48.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.620+0000 7fc1cfe00640 1 -- 172.21.15.159:0/151999258 learned_addr learned my addr 172.21.15.159:0/151999258 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:48.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.620+0000 7fc1cf400640 1 --2- 172.21.15.159:0/151999258 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1d00af830 0x7fc1d0073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:48.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.621+0000 7fc1d4c00640 1 -- 172.21.15.159:0/151999258 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1d00af830 msgr2=0x7fc1d0073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:48.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.621+0000 7fc1d4c00640 1 --2- 172.21.15.159:0/151999258 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1d00af830 0x7fc1d0073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:48.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.621+0000 7fc1d4c00640 1 -- 172.21.15.159:0/151999258 >> v1:172.21.15.159:6789/0 conn(0x7fc1d015c390 legacy=0x7fc1d0160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:48.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.621+0000 7fc1d4c00640 1 -- 172.21.15.159:0/151999258 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d00ad400 con 0x7fc1d00d0650 2024-04-04T06:10:48.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.622+0000 7fc1d4c00640 1 --2- 172.21.15.159:0/151999258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1d00d0650 0x7fc1d00d0b00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009d90 tx=0x7fc1c802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ec14ecd0f512a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:48.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.622+0000 7fc1cea00640 1 -- 172.21.15.159:0/151999258 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8039070 con 0x7fc1d00d0650 2024-04-04T06:10:48.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.622+0000 7fc1cea00640 1 -- 172.21.15.159:0/151999258 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c802cc60 con 0x7fc1d00d0650 2024-04-04T06:10:48.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.622+0000 7fc1cea00640 1 -- 172.21.15.159:0/151999258 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8034660 con 0x7fc1d00d0650 2024-04-04T06:10:48.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.623+0000 7fc1d6c00640 1 -- 172.21.15.159:0/151999258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1d00d0650 msgr2=0x7fc1d00d0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:48.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.623+0000 7fc1d6c00640 1 --2- 172.21.15.159:0/151999258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1d00d0650 0x7fc1d00d0b00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009d90 tx=0x7fc1c802beb0 comp rx=0 tx=0).stop 2024-04-04T06:10:48.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.623+0000 7fc1d6c00640 1 -- 172.21.15.159:0/151999258 shutdown_connections 2024-04-04T06:10:48.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.623+0000 7fc1d6c00640 1 --2- 172.21.15.159:0/151999258 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1d00af830 0x7fc1d0073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:48.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.623+0000 7fc1d6c00640 1 --2- 172.21.15.159:0/151999258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1d00d0650 0x7fc1d00d0b00 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:48.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.623+0000 7fc1d6c00640 1 -- 172.21.15.159:0/151999258 >> 172.21.15.159:0/151999258 conn(0x7fc1d006b990 msgr2=0x7fc1d006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:48.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.623+0000 7fc1d6c00640 1 -- 172.21.15.159:0/151999258 shutdown_connections 2024-04-04T06:10:48.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.623+0000 7fc1d6c00640 1 -- 172.21.15.159:0/151999258 wait complete. 2024-04-04T06:10:48.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.625+0000 7fc1d6c00640 1 Processor -- start 2024-04-04T06:10:48.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.625+0000 7fc1d6c00640 1 -- start start 2024-04-04T06:10:48.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.626+0000 7fc1d6c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1d00af830 0x7fc1d00b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:48.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.626+0000 7fc1d6c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1d00d0650 0x7fc1d0155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:48.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.626+0000 7fc1d6c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1d015c390 0x7fc1d01560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:48.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.626+0000 7fc1d6c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d0160d50 con 0x7fc1d015c390 2024-04-04T06:10:48.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.626+0000 7fc1d6c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d0160120 con 0x7fc1d00d0650 2024-04-04T06:10:48.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.626+0000 7fc1d6c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d00cf980 con 0x7fc1d00af830 2024-04-04T06:10:48.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.627+0000 7fc1d4c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1d015c390 0x7fc1d01560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:48.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.627+0000 7fc1d4c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1d015c390 0x7fc1d01560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60946/0 (socket says 172.21.15.159:60946) 2024-04-04T06:10:48.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.627+0000 7fc1d4c00640 1 -- 172.21.15.159:0/516668083 learned_addr learned my addr 172.21.15.159:0/516668083 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:48.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.627+0000 7fc1cfe00640 1 --2- 172.21.15.159:0/516668083 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1d00af830 0x7fc1d00b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:48.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.627+0000 7fc1cf400640 1 --2- 172.21.15.159:0/516668083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1d00d0650 0x7fc1d0155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:48.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.627+0000 7fc1d4c00640 1 -- 172.21.15.159:0/516668083 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1d00af830 msgr2=0x7fc1d00b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:48.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.627+0000 7fc1d4c00640 1 --2- 172.21.15.159:0/516668083 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1d00af830 0x7fc1d00b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:48.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.627+0000 7fc1d4c00640 1 -- 172.21.15.159:0/516668083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1d00d0650 msgr2=0x7fc1d0155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:48.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.627+0000 7fc1d4c00640 1 --2- 172.21.15.159:0/516668083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1d00d0650 0x7fc1d0155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:48.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.627+0000 7fc1d4c00640 1 -- 172.21.15.159:0/516668083 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d00b49c0 con 0x7fc1d015c390 2024-04-04T06:10:48.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.628+0000 7fc1d4c00640 1 --2- 172.21.15.159:0/516668083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1d015c390 0x7fc1d01560e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009eb0 tx=0x7fc1c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:48.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.628+0000 7fc1c7e00640 1 -- 172.21.15.159:0/516668083 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8044020 con 0x7fc1d015c390 2024-04-04T06:10:48.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.628+0000 7fc1c7e00640 1 -- 172.21.15.159:0/516668083 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c803f470 con 0x7fc1d015c390 2024-04-04T06:10:48.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.628+0000 7fc1c7e00640 1 -- 172.21.15.159:0/516668083 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8039040 con 0x7fc1d015c390 2024-04-04T06:10:48.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.628+0000 7fc1d6c00640 1 -- 172.21.15.159:0/516668083 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d00b46e0 con 0x7fc1d015c390 2024-04-04T06:10:48.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.628+0000 7fc1d6c00640 1 -- 172.21.15.159:0/516668083 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d00b35c0 con 0x7fc1d015c390 2024-04-04T06:10:48.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.629+0000 7fc1d6c00640 1 -- 172.21.15.159:0/516668083 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1d00685c0 con 0x7fc1d015c390 2024-04-04T06:10:48.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.630+0000 7fc1c7e00640 1 -- 172.21.15.159:0/516668083 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1c8047020 con 0x7fc1d015c390 2024-04-04T06:10:48.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.632+0000 7fc1c7e00640 1 --2- 172.21.15.159:0/516668083 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc1b006c8f0 0x7fc1b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:48.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.632+0000 7fc1c7e00640 1 -- 172.21.15.159:0/516668083 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fc1c80ae6e0 con 0x7fc1d015c390 2024-04-04T06:10:48.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.632+0000 7fc1cfe00640 1 --2- 172.21.15.159:0/516668083 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc1b006c8f0 0x7fc1b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:48.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.633+0000 7fc1cfe00640 1 --2- 172.21.15.159:0/516668083 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc1b006c8f0 0x7fc1b006edb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0000f60 tx=0x7fc1c0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:48.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.637+0000 7fc1c7e00640 1 -- 172.21.15.159:0/516668083 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc1c804e030 con 0x7fc1d015c390 2024-04-04T06:10:48.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.893+0000 7fc1d6c00640 1 -- 172.21.15.159:0/516668083 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1d006f1b0 con 0x7fc1d015c390 2024-04-04T06:10:48.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.894+0000 7fc1c7e00640 1 -- 172.21.15.159:0/516668083 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5192 (secure 0 0 0) 0x7fc1c807cb00 con 0x7fc1d015c390 2024-04-04T06:10:48.887 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:48.887 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3789513463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3789513463},{"type":"v1","addr":"172.21.15.178:6835","nonce":3789513463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/468875883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":468875883},{"type":"v1","addr":"172.21.15.178:6837","nonce":468875883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2546974742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2546974742},{"type":"v1","addr":"172.21.15.159:6833","nonce":2546974742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:10:44.805617+0000","modified":"2024-04-04T06:10:47.987695+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6633},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6633":{"gid":6633,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6835/1652468640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1652468640},{"type":"v1","addr":"172.21.15.159:6835","nonce":1652468640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6633,"qdb_cluster":[6633]},"id":5}]} 2024-04-04T06:10:48.887 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 45 2024-04-04T06:10:48.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.897+0000 7fc1d6c00640 1 -- 172.21.15.159:0/516668083 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc1b006c8f0 msgr2=0x7fc1b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:48.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.897+0000 7fc1d6c00640 1 --2- 172.21.15.159:0/516668083 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc1b006c8f0 0x7fc1b006edb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0000f60 tx=0x7fc1c0009040 comp rx=0 tx=0).stop 2024-04-04T06:10:48.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.897+0000 7fc1d6c00640 1 -- 172.21.15.159:0/516668083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1d015c390 msgr2=0x7fc1d01560e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:48.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.897+0000 7fc1d6c00640 1 --2- 172.21.15.159:0/516668083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1d015c390 0x7fc1d01560e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009eb0 tx=0x7fc1c8004a40 comp rx=0 tx=0).stop 2024-04-04T06:10:48.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.897+0000 7fc1d6c00640 1 -- 172.21.15.159:0/516668083 shutdown_connections 2024-04-04T06:10:48.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.897+0000 7fc1d6c00640 1 --2- 172.21.15.159:0/516668083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1d015c390 0x7fc1d01560e0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:48.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.897+0000 7fc1d6c00640 1 --2- 172.21.15.159:0/516668083 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc1b006c8f0 0x7fc1b006edb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:48.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.897+0000 7fc1d6c00640 1 --2- 172.21.15.159:0/516668083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1d00d0650 0x7fc1d0155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:48.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.897+0000 7fc1d6c00640 1 --2- 172.21.15.159:0/516668083 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1d00af830 0x7fc1d00b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:48.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.897+0000 7fc1d6c00640 1 -- 172.21.15.159:0/516668083 >> 172.21.15.159:0/516668083 conn(0x7fc1d006b990 msgr2=0x7fc1d00afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:48.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.897+0000 7fc1d6c00640 1 -- 172.21.15.159:0/516668083 shutdown_connections 2024-04-04T06:10:48.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:48.897+0000 7fc1d6c00640 1 -- 172.21.15.159:0/516668083 wait complete. 2024-04-04T06:10:48.910 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:10:49.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.118+0000 7ff055600640 1 Processor -- start 2024-04-04T06:10:49.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.118+0000 7ff055600640 1 -- start start 2024-04-04T06:10:49.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.119+0000 7ff055600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff0500d0e90 0x7ff0500b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:49.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.120+0000 7ff055600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff05015b6f0 con 0x7ff0500727f0 2024-04-04T06:10:49.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.120+0000 7ff055600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff050072e80 con 0x7ff05015b310 2024-04-04T06:10:49.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.120+0000 7ff055600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff050072c50 con 0x7ff0500d0e90 2024-04-04T06:10:49.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.120+0000 7ff047e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7ff0500727f0 0x7ff0500d0780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:44420/0 (socket says 172.21.15.159:44420) 2024-04-04T06:10:49.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.120+0000 7ff047e00640 1 -- 172.21.15.159:0/3157614641 learned_addr learned my addr 172.21.15.159:0/3157614641 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:49.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.120+0000 7ff04f400640 1 --2- 172.21.15.159:0/3157614641 >> v2:172.21.15.178:3300/0 conn(0x7ff0500d0e90 0x7ff0500b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:49.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.120+0000 7ff04e000640 1 -- 172.21.15.159:0/3157614641 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1970487384 0 0) 0x7ff05015b6f0 con 0x7ff0500727f0 2024-04-04T06:10:49.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.120+0000 7ff04e000640 1 -- 172.21.15.159:0/3157614641 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff034003520 con 0x7ff0500727f0 2024-04-04T06:10:49.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.121+0000 7ff04e000640 1 -- 172.21.15.159:0/3157614641 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2859275167 0 0) 0x7ff034003520 con 0x7ff0500727f0 2024-04-04T06:10:49.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.121+0000 7ff04e000640 1 -- 172.21.15.159:0/3157614641 >> v2:172.21.15.178:3300/0 conn(0x7ff0500d0e90 msgr2=0x7ff0500b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:49.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.121+0000 7ff04e000640 1 --2- 172.21.15.159:0/3157614641 >> v2:172.21.15.178:3300/0 conn(0x7ff0500d0e90 0x7ff0500b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:49.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.121+0000 7ff04e000640 1 -- 172.21.15.159:0/3157614641 >> v1:172.21.15.178:6789/0 conn(0x7ff05015b310 legacy=0x7ff05015f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:49.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.121+0000 7ff04e000640 1 -- 172.21.15.159:0/3157614641 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff050073160 con 0x7ff0500727f0 2024-04-04T06:10:49.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.121+0000 7ff04e000640 1 -- 172.21.15.159:0/3157614641 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7ff038001670 con 0x7ff0500727f0 2024-04-04T06:10:49.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.121+0000 7ff04e000640 1 -- 172.21.15.159:0/3157614641 >> v1:172.21.15.159:6789/0 conn(0x7ff0500727f0 legacy=0x7ff0500d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:49.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.121+0000 7ff04e000640 1 --2- 172.21.15.159:0/3157614641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0340039e0 0x7ff034003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:49.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.121+0000 7ff04e000640 1 --2- 172.21.15.159:0/3157614641 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff034004800 0x7ff034004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:49.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.121+0000 7ff04e000640 1 --2- 172.21.15.159:0/3157614641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0340052e0 0x7ff03400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:49.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.121+0000 7ff04e000640 1 -- 172.21.15.159:0/3157614641 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff03400c1b0 con 0x7ff0340052e0 2024-04-04T06:10:49.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.121+0000 7ff04e000640 1 -- 172.21.15.159:0/3157614641 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff03400c360 con 0x7ff0340039e0 2024-04-04T06:10:49.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.121+0000 7ff04e000640 1 -- 172.21.15.159:0/3157614641 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff03400c560 con 0x7ff034004800 2024-04-04T06:10:49.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.121+0000 7ff04f400640 1 --2- 172.21.15.159:0/3157614641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0340052e0 0x7ff03400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:49.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.122+0000 7ff04ea00640 1 --2- 172.21.15.159:0/3157614641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0340039e0 0x7ff034003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:49.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.122+0000 7ff047e00640 1 --2- 172.21.15.159:0/3157614641 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff034004800 0x7ff034004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:49.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.122+0000 7ff04f400640 1 -- 172.21.15.159:0/3157614641 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff034004800 msgr2=0x7ff034004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:49.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.122+0000 7ff04f400640 1 --2- 172.21.15.159:0/3157614641 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff034004800 0x7ff034004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:49.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.122+0000 7ff04f400640 1 -- 172.21.15.159:0/3157614641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0340039e0 msgr2=0x7ff034003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:49.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.122+0000 7ff04f400640 1 --2- 172.21.15.159:0/3157614641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0340039e0 0x7ff034003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:49.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.122+0000 7ff04f400640 1 -- 172.21.15.159:0/3157614641 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff034003520 con 0x7ff0340052e0 2024-04-04T06:10:49.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.123+0000 7ff04f400640 1 -- 172.21.15.159:0/3157614641 reap_dead start 2024-04-04T06:10:49.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.123+0000 7ff04f400640 1 --2- 172.21.15.159:0/3157614641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0340052e0 0x7ff03400b9c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ff0500ad9a0 tx=0x7ff034005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2cfd7b40b783aa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:49.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.123+0000 7ff04e000640 1 -- 172.21.15.159:0/3157614641 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff048009c10 con 0x7ff0340052e0 2024-04-04T06:10:49.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.123+0000 7ff04e000640 1 -- 172.21.15.159:0/3157614641 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff048003b90 con 0x7ff0340052e0 2024-04-04T06:10:49.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.123+0000 7ff04e000640 1 -- 172.21.15.159:0/3157614641 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04802c990 con 0x7ff0340052e0 2024-04-04T06:10:49.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.124+0000 7ff055600640 1 -- 172.21.15.159:0/3157614641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0340052e0 msgr2=0x7ff03400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:49.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.124+0000 7ff055600640 1 --2- 172.21.15.159:0/3157614641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0340052e0 0x7ff03400b9c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ff0500ad9a0 tx=0x7ff034005830 comp rx=0 tx=0).stop 2024-04-04T06:10:49.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.124+0000 7ff055600640 1 -- 172.21.15.159:0/3157614641 shutdown_connections 2024-04-04T06:10:49.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.124+0000 7ff055600640 1 --2- 172.21.15.159:0/3157614641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0340052e0 0x7ff03400b9c0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:49.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.124+0000 7ff055600640 1 -- 172.21.15.159:0/3157614641 >> 172.21.15.159:0/3157614641 conn(0x7ff05006b990 msgr2=0x7ff05006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:49.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.124+0000 7ff055600640 1 -- 172.21.15.159:0/3157614641 shutdown_connections 2024-04-04T06:10:49.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.124+0000 7ff055600640 1 -- 172.21.15.159:0/3157614641 wait complete. 2024-04-04T06:10:49.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.126+0000 7ff055600640 1 Processor -- start 2024-04-04T06:10:49.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.126+0000 7ff055600640 1 -- start start 2024-04-04T06:10:49.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.127+0000 7ff055600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0340052e0 0x7ff050154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:49.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.127+0000 7ff055600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0500aec90 0x7ff0500af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:49.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.127+0000 7ff055600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff05015cf30 0x7ff05015d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:49.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.127+0000 7ff055600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff050073440 con 0x7ff05015cf30 2024-04-04T06:10:49.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.127+0000 7ff055600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff05015f2c0 con 0x7ff0500aec90 2024-04-04T06:10:49.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.127+0000 7ff055600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff0500b4f90 con 0x7ff0340052e0 2024-04-04T06:10:49.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.127+0000 7ff04f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff05015cf30 0x7ff05015d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:49.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.127+0000 7ff04f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff05015cf30 0x7ff05015d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60956/0 (socket says 172.21.15.159:60956) 2024-04-04T06:10:49.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.127+0000 7ff04f400640 1 -- 172.21.15.159:0/1865949912 learned_addr learned my addr 172.21.15.159:0/1865949912 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:49.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.127+0000 7ff04ea00640 1 --2- 172.21.15.159:0/1865949912 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0340052e0 0x7ff050154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:49.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.128+0000 7ff047e00640 1 --2- 172.21.15.159:0/1865949912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0500aec90 0x7ff0500af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:49.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.128+0000 7ff04f400640 1 -- 172.21.15.159:0/1865949912 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0340052e0 msgr2=0x7ff050154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:49.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.128+0000 7ff04f400640 1 --2- 172.21.15.159:0/1865949912 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0340052e0 0x7ff050154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:49.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.128+0000 7ff04f400640 1 -- 172.21.15.159:0/1865949912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0500aec90 msgr2=0x7ff0500af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:49.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.128+0000 7ff04f400640 1 --2- 172.21.15.159:0/1865949912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0500aec90 0x7ff0500af110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:49.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.128+0000 7ff04f400640 1 -- 172.21.15.159:0/1865949912 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05015e7c0 con 0x7ff05015cf30 2024-04-04T06:10:49.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.128+0000 7ff047e00640 1 --2- 172.21.15.159:0/1865949912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0500aec90 0x7ff0500af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:49.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.128+0000 7ff04f400640 1 --2- 172.21.15.159:0/1865949912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff05015cf30 0x7ff05015d3e0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ff04802c600 tx=0x7ff048009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:49.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.128+0000 7ff046a00640 1 -- 172.21.15.159:0/1865949912 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04802ce10 con 0x7ff05015cf30 2024-04-04T06:10:49.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.128+0000 7ff046a00640 1 -- 172.21.15.159:0/1865949912 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff048044070 con 0x7ff05015cf30 2024-04-04T06:10:49.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.128+0000 7ff046a00640 1 -- 172.21.15.159:0/1865949912 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0480045e0 con 0x7ff05015cf30 2024-04-04T06:10:49.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.128+0000 7ff055600640 1 -- 172.21.15.159:0/1865949912 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05015dc40 con 0x7ff05015cf30 2024-04-04T06:10:49.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.129+0000 7ff055600640 1 -- 172.21.15.159:0/1865949912 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff05015e200 con 0x7ff05015cf30 2024-04-04T06:10:49.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.130+0000 7ff055600640 1 -- 172.21.15.159:0/1865949912 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0500685c0 con 0x7ff05015cf30 2024-04-04T06:10:49.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.131+0000 7ff046a00640 1 -- 172.21.15.159:0/1865949912 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff048005d10 con 0x7ff05015cf30 2024-04-04T06:10:49.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.132+0000 7ff046a00640 1 --2- 172.21.15.159:0/1865949912 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff02c06c8c0 0x7ff02c06ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:49.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.132+0000 7ff046a00640 1 -- 172.21.15.159:0/1865949912 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7ff0480adfb0 con 0x7ff05015cf30 2024-04-04T06:10:49.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.133+0000 7ff04ea00640 1 --2- 172.21.15.159:0/1865949912 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff02c06c8c0 0x7ff02c06ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:49.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.134+0000 7ff04ea00640 1 --2- 172.21.15.159:0/1865949912 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff02c06c8c0 0x7ff02c06ed80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff0500b4ed0 tx=0x7ff040009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:49.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.137+0000 7ff046a00640 1 -- 172.21.15.159:0/1865949912 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff04807c500 con 0x7ff05015cf30 2024-04-04T06:10:49.300 INFO:tasks.ceph.mds.a.smithi159.stderr:2024-04-04T06:10:49.307+0000 7f50bd400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:10:49.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.339+0000 7ff055600640 1 -- 172.21.15.159:0/1865949912 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0500cc8c0 con 0x7ff05015cf30 2024-04-04T06:10:49.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.341+0000 7ff046a00640 1 -- 172.21.15.159:0/1865949912 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16680 (secure 0 0 0) 0x7ff04807be00 con 0x7ff05015cf30 2024-04-04T06:10:49.335 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:49.335 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":71,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:10:46.362050+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:10:42.595868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-04T06:10:43.788847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:10:49.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.344+0000 7ff055600640 1 -- 172.21.15.159:0/1865949912 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff02c06c8c0 msgr2=0x7ff02c06ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:49.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.344+0000 7ff055600640 1 --2- 172.21.15.159:0/1865949912 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff02c06c8c0 0x7ff02c06ed80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff0500b4ed0 tx=0x7ff040009040 comp rx=0 tx=0).stop 2024-04-04T06:10:49.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.344+0000 7ff055600640 1 -- 172.21.15.159:0/1865949912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff05015cf30 msgr2=0x7ff05015d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:49.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.344+0000 7ff055600640 1 --2- 172.21.15.159:0/1865949912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff05015cf30 0x7ff05015d3e0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ff04802c600 tx=0x7ff048009750 comp rx=0 tx=0).stop 2024-04-04T06:10:49.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.344+0000 7ff055600640 1 -- 172.21.15.159:0/1865949912 shutdown_connections 2024-04-04T06:10:49.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.344+0000 7ff055600640 1 --2- 172.21.15.159:0/1865949912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff05015cf30 0x7ff05015d3e0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:49.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.344+0000 7ff055600640 1 --2- 172.21.15.159:0/1865949912 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff02c06c8c0 0x7ff02c06ed80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:49.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.344+0000 7ff055600640 1 --2- 172.21.15.159:0/1865949912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0500aec90 0x7ff0500af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:49.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.344+0000 7ff055600640 1 --2- 172.21.15.159:0/1865949912 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0340052e0 0x7ff050154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:49.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.344+0000 7ff055600640 1 -- 172.21.15.159:0/1865949912 >> 172.21.15.159:0/1865949912 conn(0x7ff05006b990 msgr2=0x7ff0500b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:49.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.344+0000 7ff055600640 1 -- 172.21.15.159:0/1865949912 shutdown_connections 2024-04-04T06:10:49.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.344+0000 7ff055600640 1 -- 172.21.15.159:0/1865949912 wait complete. 2024-04-04T06:10:49.355 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-04T06:10:49.356 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:10:49.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.560+0000 7fc4d8e00640 1 Processor -- start 2024-04-04T06:10:49.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.561+0000 7fc4d8e00640 1 -- start start 2024-04-04T06:10:49.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.563+0000 7fc4d8e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4d415e4a0 0x7fc4d40b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:49.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.563+0000 7fc4d8e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc4d40cf700 0x7fc4d40b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:49.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.563+0000 7fc4d8e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4d415e880 con 0x7fc4d40d2310 2024-04-04T06:10:49.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.563+0000 7fc4d8e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fc4d40d34e0 con 0x7fc4d40cf700 2024-04-04T06:10:49.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.563+0000 7fc4d8e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d40d28d0 con 0x7fc4d415e4a0 2024-04-04T06:10:49.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.563+0000 7fc4d3e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc4d40cf700 0x7fc4d40b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:49.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.564+0000 7fc4d2a00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fc4d40d2310 0x7fc4d40ceff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:44432/0 (socket says 172.21.15.159:44432) 2024-04-04T06:10:49.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.564+0000 7fc4d2a00640 1 -- 172.21.15.159:0/974117543 learned_addr learned my addr 172.21.15.159:0/974117543 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:49.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.564+0000 7fc4d3400640 1 --2- 172.21.15.159:0/974117543 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4d415e4a0 0x7fc4d40b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:49.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.564+0000 7fc4d3e00640 1 -- 172.21.15.159:0/974117543 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4d415e4a0 msgr2=0x7fc4d40b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:49.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.564+0000 7fc4d3e00640 1 --2- 172.21.15.159:0/974117543 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4d415e4a0 0x7fc4d40b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:49.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.564+0000 7fc4d2000640 1 -- 172.21.15.159:0/974117543 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1567501849 0 0) 0x7fc4d415e880 con 0x7fc4d40d2310 2024-04-04T06:10:49.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.564+0000 7fc4d3e00640 1 -- 172.21.15.159:0/974117543 >> v1:172.21.15.159:6789/0 conn(0x7fc4d40d2310 legacy=0x7fc4d40ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:49.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.564+0000 7fc4d3e00640 1 -- 172.21.15.159:0/974117543 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d40d2bb0 con 0x7fc4d40cf700 2024-04-04T06:10:49.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.564+0000 7fc4d3400640 1 --2- 172.21.15.159:0/974117543 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4d415e4a0 0x7fc4d40b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:49.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.565+0000 7fc4d3e00640 1 --2- 172.21.15.159:0/974117543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4d40cf700 0x7fc4d40b73e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40d3660 tx=0x7fc4cc02be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ede89c16ce5c58 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:49.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.566+0000 7fc4d2000640 1 -- 172.21.15.159:0/974117543 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc039070 con 0x7fc4d40cf700 2024-04-04T06:10:49.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.566+0000 7fc4d2000640 1 -- 172.21.15.159:0/974117543 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4cc02cc40 con 0x7fc4d40cf700 2024-04-04T06:10:49.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.566+0000 7fc4d2000640 1 -- 172.21.15.159:0/974117543 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc034440 con 0x7fc4d40cf700 2024-04-04T06:10:49.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.566+0000 7fc4d8e00640 1 -- 172.21.15.159:0/974117543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4d40cf700 msgr2=0x7fc4d40b73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:49.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.566+0000 7fc4d8e00640 1 --2- 172.21.15.159:0/974117543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4d40cf700 0x7fc4d40b73e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40d3660 tx=0x7fc4cc02be90 comp rx=0 tx=0).stop 2024-04-04T06:10:49.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.567+0000 7fc4d8e00640 1 -- 172.21.15.159:0/974117543 shutdown_connections 2024-04-04T06:10:49.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.567+0000 7fc4d8e00640 1 --2- 172.21.15.159:0/974117543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4d40cf700 0x7fc4d40b73e0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:49.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.567+0000 7fc4d8e00640 1 --2- 172.21.15.159:0/974117543 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4d415e4a0 0x7fc4d40b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:49.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.567+0000 7fc4d8e00640 1 -- 172.21.15.159:0/974117543 >> 172.21.15.159:0/974117543 conn(0x7fc4d40676d0 msgr2=0x7fc4d4067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:49.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.567+0000 7fc4d8e00640 1 -- 172.21.15.159:0/974117543 shutdown_connections 2024-04-04T06:10:49.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.567+0000 7fc4d8e00640 1 -- 172.21.15.159:0/974117543 wait complete. 2024-04-04T06:10:49.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.569+0000 7fc4d8e00640 1 Processor -- start 2024-04-04T06:10:49.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.570+0000 7fc4d8e00640 1 -- start start 2024-04-04T06:10:49.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.571+0000 7fc4d8e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4d40cf700 0x7fc4d40b87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:49.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.571+0000 7fc4d8e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4d40d2310 0x7fc4d40b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:49.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.571+0000 7fc4d8e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4d415e4a0 0x7fc4d40cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:49.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.571+0000 7fc4d8e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d40d3970 con 0x7fc4d415e4a0 2024-04-04T06:10:49.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.571+0000 7fc4d8e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d40d3170 con 0x7fc4d40d2310 2024-04-04T06:10:49.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.571+0000 7fc4d8e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d40ba1a0 con 0x7fc4d40cf700 2024-04-04T06:10:49.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.572+0000 7fc4d3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4d415e4a0 0x7fc4d40cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:49.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.572+0000 7fc4d3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4d415e4a0 0x7fc4d40cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60988/0 (socket says 172.21.15.159:60988) 2024-04-04T06:10:49.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.572+0000 7fc4d3e00640 1 -- 172.21.15.159:0/764234940 learned_addr learned my addr 172.21.15.159:0/764234940 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:49.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.572+0000 7fc4d2a00640 1 --2- 172.21.15.159:0/764234940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4d40d2310 0x7fc4d40b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:49.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.572+0000 7fc4d3e00640 1 -- 172.21.15.159:0/764234940 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4d40cf700 msgr2=0x7fc4d40b87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:49.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.572+0000 7fc4d3400640 1 --2- 172.21.15.159:0/764234940 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4d40cf700 0x7fc4d40b87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:49.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.572+0000 7fc4d3e00640 1 --2- 172.21.15.159:0/764234940 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4d40cf700 0x7fc4d40b87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:49.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.572+0000 7fc4d3e00640 1 -- 172.21.15.159:0/764234940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4d40d2310 msgr2=0x7fc4d40b8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:49.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.572+0000 7fc4d3e00640 1 --2- 172.21.15.159:0/764234940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4d40d2310 0x7fc4d40b8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:49.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.572+0000 7fc4d3e00640 1 -- 172.21.15.159:0/764234940 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d40cce80 con 0x7fc4d415e4a0 2024-04-04T06:10:49.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.572+0000 7fc4d3e00640 1 --2- 172.21.15.159:0/764234940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4d415e4a0 0x7fc4d40cb380 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009d90 tx=0x7fc4cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:49.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.573+0000 7fc4cb400640 1 -- 172.21.15.159:0/764234940 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc039050 con 0x7fc4d415e4a0 2024-04-04T06:10:49.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.573+0000 7fc4cb400640 1 -- 172.21.15.159:0/764234940 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4cc045070 con 0x7fc4d415e4a0 2024-04-04T06:10:49.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.573+0000 7fc4d8e00640 1 -- 172.21.15.159:0/764234940 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d40cd160 con 0x7fc4d415e4a0 2024-04-04T06:10:49.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.573+0000 7fc4cb400640 1 -- 172.21.15.159:0/764234940 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc03e850 con 0x7fc4d415e4a0 2024-04-04T06:10:49.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.573+0000 7fc4d8e00640 1 -- 172.21.15.159:0/764234940 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d40cd720 con 0x7fc4d415e4a0 2024-04-04T06:10:49.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.574+0000 7fc4d8e00640 1 -- 172.21.15.159:0/764234940 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d4075940 con 0x7fc4d415e4a0 2024-04-04T06:10:49.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.576+0000 7fc4cb400640 1 -- 172.21.15.159:0/764234940 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4cc03e9f0 con 0x7fc4d415e4a0 2024-04-04T06:10:49.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.577+0000 7fc4cb400640 1 --2- 172.21.15.159:0/764234940 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc4b406c8f0 0x7fc4b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:49.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.577+0000 7fc4cb400640 1 -- 172.21.15.159:0/764234940 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fc4cc0ae820 con 0x7fc4d415e4a0 2024-04-04T06:10:49.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.578+0000 7fc4d3400640 1 --2- 172.21.15.159:0/764234940 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc4b406c8f0 0x7fc4b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:49.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.579+0000 7fc4d3400640 1 --2- 172.21.15.159:0/764234940 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc4b406c8f0 0x7fc4b406edb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4008850 tx=0x7fc4c4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:49.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.582+0000 7fc4cb400640 1 -- 172.21.15.159:0/764234940 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc4cc07ce20 con 0x7fc4d415e4a0 2024-04-04T06:10:49.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.778+0000 7fc4d8e00640 1 -- 172.21.15.159:0/764234940 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc4d415da20 con 0x7fc4b406c8f0 2024-04-04T06:10:49.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.797+0000 7fc4cb400640 1 -- 172.21.15.159:0/764234940 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252046 (secure 0 0 0) 0x7fc4d415da20 con 0x7fc4b406c8f0 2024-04-04T06:10:49.794 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:49.796 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:10:49.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.804+0000 7fc4d8e00640 1 -- 172.21.15.159:0/764234940 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc4b406c8f0 msgr2=0x7fc4b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:49.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.804+0000 7fc4d8e00640 1 --2- 172.21.15.159:0/764234940 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc4b406c8f0 0x7fc4b406edb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4008850 tx=0x7fc4c4008040 comp rx=0 tx=0).stop 2024-04-04T06:10:49.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.804+0000 7fc4d8e00640 1 -- 172.21.15.159:0/764234940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4d415e4a0 msgr2=0x7fc4d40cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:49.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.804+0000 7fc4d8e00640 1 --2- 172.21.15.159:0/764234940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4d415e4a0 0x7fc4d40cb380 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009d90 tx=0x7fc4cc004a40 comp rx=0 tx=0).stop 2024-04-04T06:10:49.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.804+0000 7fc4d8e00640 1 -- 172.21.15.159:0/764234940 shutdown_connections 2024-04-04T06:10:49.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.804+0000 7fc4d8e00640 1 --2- 172.21.15.159:0/764234940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4d415e4a0 0x7fc4d40cb380 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:49.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.804+0000 7fc4d8e00640 1 --2- 172.21.15.159:0/764234940 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc4b406c8f0 0x7fc4b406edb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:49.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.804+0000 7fc4d8e00640 1 --2- 172.21.15.159:0/764234940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4d40d2310 0x7fc4d40b8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:49.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.804+0000 7fc4d8e00640 1 --2- 172.21.15.159:0/764234940 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4d40cf700 0x7fc4d40b87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:49.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.804+0000 7fc4d8e00640 1 -- 172.21.15.159:0/764234940 >> 172.21.15.159:0/764234940 conn(0x7fc4d40676d0 msgr2=0x7fc4d40ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:49.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.804+0000 7fc4d8e00640 1 -- 172.21.15.159:0/764234940 shutdown_connections 2024-04-04T06:10:49.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:49.804+0000 7fc4d8e00640 1 -- 172.21.15.159:0/764234940 wait complete. 2024-04-04T06:10:49.817 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-04-04T06:10:48.818724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257316,"kb_used_data":7836,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717404,"statfs":{"total":773094113280,"available":772830621696,"internally_reserved":0,"allocated":8024064,"data_stored":4034099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.495170"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476782+0000","last_change":"2024-04-04T06:10:45.398821+0000","last_active":"2024-04-04T06:10:46.476782+0000","last_peered":"2024-04-04T06:10:46.476782+0000","last_clean":"2024-04-04T06:10:46.476782+0000","last_became_active":"2024-04-04T06:10:45.395222+0000","last_became_peered":"2024-04-04T06:10:45.395222+0000","last_unstale":"2024-04-04T06:10:46.476782+0000","last_undegraded":"2024-04-04T06:10:46.476782+0000","last_fullsized":"2024-04-04T06:10:46.476782+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:41:23.938682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.479110+0000","last_change":"2024-04-04T06:10:45.394232+0000","last_active":"2024-04-04T06:10:46.479110+0000","last_peered":"2024-04-04T06:10:46.479110+0000","last_clean":"2024-04-04T06:10:46.479110+0000","last_became_active":"2024-04-04T06:10:45.393961+0000","last_became_peered":"2024-04-04T06:10:45.393961+0000","last_unstale":"2024-04-04T06:10:46.479110+0000","last_undegraded":"2024-04-04T06:10:46.479110+0000","last_fullsized":"2024-04-04T06:10:46.479110+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:35:45.072858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476442+0000","last_change":"2024-04-04T06:10:45.380449+0000","last_active":"2024-04-04T06:10:46.476442+0000","last_peered":"2024-04-04T06:10:46.476442+0000","last_clean":"2024-04-04T06:10:46.476442+0000","last_became_active":"2024-04-04T06:10:45.380179+0000","last_became_peered":"2024-04-04T06:10:45.380179+0000","last_unstale":"2024-04-04T06:10:46.476442+0000","last_undegraded":"2024-04-04T06:10:46.476442+0000","last_fullsized":"2024-04-04T06:10:46.476442+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:57:56.388699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.376989+0000","last_change":"2024-04-04T06:10:45.386910+0000","last_active":"2024-04-04T06:10:46.376989+0000","last_peered":"2024-04-04T06:10:46.376989+0000","last_clean":"2024-04-04T06:10:46.376989+0000","last_became_active":"2024-04-04T06:10:45.386631+0000","last_became_peered":"2024-04-04T06:10:45.386631+0000","last_unstale":"2024-04-04T06:10:46.376989+0000","last_undegraded":"2024-04-04T06:10:46.376989+0000","last_fullsized":"2024-04-04T06:10:46.376989+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:28:16.796576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474376+0000","last_change":"2024-04-04T06:10:45.389152+0000","last_active":"2024-04-04T06:10:46.474376+0000","last_peered":"2024-04-04T06:10:46.474376+0000","last_clean":"2024-04-04T06:10:46.474376+0000","last_became_active":"2024-04-04T06:10:45.388860+0000","last_became_peered":"2024-04-04T06:10:45.388860+0000","last_unstale":"2024-04-04T06:10:46.474376+0000","last_undegraded":"2024-04-04T06:10:46.474376+0000","last_fullsized":"2024-04-04T06:10:46.474376+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:18:41.590838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477421+0000","last_change":"2024-04-04T06:10:45.393878+0000","last_active":"2024-04-04T06:10:46.477421+0000","last_peered":"2024-04-04T06:10:46.477421+0000","last_clean":"2024-04-04T06:10:46.477421+0000","last_became_active":"2024-04-04T06:10:45.393399+0000","last_became_peered":"2024-04-04T06:10:45.393399+0000","last_unstale":"2024-04-04T06:10:46.477421+0000","last_undegraded":"2024-04-04T06:10:46.477421+0000","last_fullsized":"2024-04-04T06:10:46.477421+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:53:15.246653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.388340+0000","last_change":"2024-04-04T06:10:45.374167+0000","last_active":"2024-04-04T06:10:45.388340+0000","last_peered":"2024-04-04T06:10:45.388340+0000","last_clean":"2024-04-04T06:10:45.388340+0000","last_became_active":"2024-04-04T06:10:45.373899+0000","last_became_peered":"2024-04-04T06:10:45.373899+0000","last_unstale":"2024-04-04T06:10:45.388340+0000","last_undegraded":"2024-04-04T06:10:45.388340+0000","last_fullsized":"2024-04-04T06:10:45.388340+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:21:31.764115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.478192+0000","last_change":"2024-04-04T06:10:45.390552+0000","last_active":"2024-04-04T06:10:46.478192+0000","last_peered":"2024-04-04T06:10:46.478192+0000","last_clean":"2024-04-04T06:10:46.478192+0000","last_became_active":"2024-04-04T06:10:45.390232+0000","last_became_peered":"2024-04-04T06:10:45.390232+0000","last_unstale":"2024-04-04T06:10:46.478192+0000","last_undegraded":"2024-04-04T06:10:46.478192+0000","last_fullsized":"2024-04-04T06:10:46.478192+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:21:57.676061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474750+0000","last_change":"2024-04-04T06:10:45.376493+0000","last_active":"2024-04-04T06:10:46.474750+0000","last_peered":"2024-04-04T06:10:46.474750+0000","last_clean":"2024-04-04T06:10:46.474750+0000","last_became_active":"2024-04-04T06:10:45.375608+0000","last_became_peered":"2024-04-04T06:10:45.375608+0000","last_unstale":"2024-04-04T06:10:46.474750+0000","last_undegraded":"2024-04-04T06:10:46.474750+0000","last_fullsized":"2024-04-04T06:10:46.474750+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:32:39.744013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.389463+0000","last_change":"2024-04-04T06:10:45.379126+0000","last_active":"2024-04-04T06:10:45.389463+0000","last_peered":"2024-04-04T06:10:45.389463+0000","last_clean":"2024-04-04T06:10:45.389463+0000","last_became_active":"2024-04-04T06:10:45.378825+0000","last_became_peered":"2024-04-04T06:10:45.378825+0000","last_unstale":"2024-04-04T06:10:45.389463+0000","last_undegraded":"2024-04-04T06:10:45.389463+0000","last_fullsized":"2024-04-04T06:10:45.389463+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:58:36.615208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477747+0000","last_change":"2024-04-04T06:10:45.394375+0000","last_active":"2024-04-04T06:10:46.477747+0000","last_peered":"2024-04-04T06:10:46.477747+0000","last_clean":"2024-04-04T06:10:46.477747+0000","last_became_active":"2024-04-04T06:10:45.394032+0000","last_became_peered":"2024-04-04T06:10:45.394032+0000","last_unstale":"2024-04-04T06:10:46.477747+0000","last_undegraded":"2024-04-04T06:10:46.477747+0000","last_fullsized":"2024-04-04T06:10:46.477747+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:36:33.342681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.377071+0000","last_change":"2024-04-04T06:10:45.372707+0000","last_active":"2024-04-04T06:10:46.377071+0000","last_peered":"2024-04-04T06:10:46.377071+0000","last_clean":"2024-04-04T06:10:46.377071+0000","last_became_active":"2024-04-04T06:10:45.372215+0000","last_became_peered":"2024-04-04T06:10:45.372215+0000","last_unstale":"2024-04-04T06:10:46.377071+0000","last_undegraded":"2024-04-04T06:10:46.377071+0000","last_fullsized":"2024-04-04T06:10:46.377071+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:41:15.412553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474019+0000","last_change":"2024-04-04T06:10:45.393013+0000","last_active":"2024-04-04T06:10:46.474019+0000","last_peered":"2024-04-04T06:10:46.474019+0000","last_clean":"2024-04-04T06:10:46.474019+0000","last_became_active":"2024-04-04T06:10:45.391895+0000","last_became_peered":"2024-04-04T06:10:45.391895+0000","last_unstale":"2024-04-04T06:10:46.474019+0000","last_undegraded":"2024-04-04T06:10:46.474019+0000","last_fullsized":"2024-04-04T06:10:46.474019+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:19:17.088841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.473678+0000","last_change":"2024-04-04T06:10:45.382277+0000","last_active":"2024-04-04T06:10:46.473678+0000","last_peered":"2024-04-04T06:10:46.473678+0000","last_clean":"2024-04-04T06:10:46.473678+0000","last_became_active":"2024-04-04T06:10:45.381995+0000","last_became_peered":"2024-04-04T06:10:45.381995+0000","last_unstale":"2024-04-04T06:10:46.473678+0000","last_undegraded":"2024-04-04T06:10:46.473678+0000","last_fullsized":"2024-04-04T06:10:46.473678+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:59:52.200063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.473481+0000","last_change":"2024-04-04T06:10:45.370994+0000","last_active":"2024-04-04T06:10:46.473481+0000","last_peered":"2024-04-04T06:10:46.473481+0000","last_clean":"2024-04-04T06:10:46.473481+0000","last_became_active":"2024-04-04T06:10:45.369525+0000","last_became_peered":"2024-04-04T06:10:45.369525+0000","last_unstale":"2024-04-04T06:10:46.473481+0000","last_undegraded":"2024-04-04T06:10:46.473481+0000","last_fullsized":"2024-04-04T06:10:46.473481+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:30:07.102180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.389319+0000","last_change":"2024-04-04T06:10:45.379855+0000","last_active":"2024-04-04T06:10:45.389319+0000","last_peered":"2024-04-04T06:10:45.389319+0000","last_clean":"2024-04-04T06:10:45.389319+0000","last_became_active":"2024-04-04T06:10:45.379563+0000","last_became_peered":"2024-04-04T06:10:45.379563+0000","last_unstale":"2024-04-04T06:10:45.389319+0000","last_undegraded":"2024-04-04T06:10:45.389319+0000","last_fullsized":"2024-04-04T06:10:45.389319+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:12:26.070099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474772+0000","last_change":"2024-04-04T06:10:45.376194+0000","last_active":"2024-04-04T06:10:46.474772+0000","last_peered":"2024-04-04T06:10:46.474772+0000","last_clean":"2024-04-04T06:10:46.474772+0000","last_became_active":"2024-04-04T06:10:45.374933+0000","last_became_peered":"2024-04-04T06:10:45.374933+0000","last_unstale":"2024-04-04T06:10:46.474772+0000","last_undegraded":"2024-04-04T06:10:46.474772+0000","last_fullsized":"2024-04-04T06:10:46.474772+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:37:32.354870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.377218+0000","last_change":"2024-04-04T06:10:45.393074+0000","last_active":"2024-04-04T06:10:46.377218+0000","last_peered":"2024-04-04T06:10:46.377218+0000","last_clean":"2024-04-04T06:10:46.377218+0000","last_became_active":"2024-04-04T06:10:45.392451+0000","last_became_peered":"2024-04-04T06:10:45.392451+0000","last_unstale":"2024-04-04T06:10:46.377218+0000","last_undegraded":"2024-04-04T06:10:46.377218+0000","last_fullsized":"2024-04-04T06:10:46.377218+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:44:43.716753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476960+0000","last_change":"2024-04-04T06:10:45.389247+0000","last_active":"2024-04-04T06:10:46.476960+0000","last_peered":"2024-04-04T06:10:46.476960+0000","last_clean":"2024-04-04T06:10:46.476960+0000","last_became_active":"2024-04-04T06:10:45.388939+0000","last_became_peered":"2024-04-04T06:10:45.388939+0000","last_unstale":"2024-04-04T06:10:46.476960+0000","last_undegraded":"2024-04-04T06:10:46.476960+0000","last_fullsized":"2024-04-04T06:10:46.476960+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:32:56.031370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474233+0000","last_change":"2024-04-04T06:10:45.381039+0000","last_active":"2024-04-04T06:10:46.474233+0000","last_peered":"2024-04-04T06:10:46.474233+0000","last_clean":"2024-04-04T06:10:46.474233+0000","last_became_active":"2024-04-04T06:10:45.380715+0000","last_became_peered":"2024-04-04T06:10:45.380715+0000","last_unstale":"2024-04-04T06:10:46.474233+0000","last_undegraded":"2024-04-04T06:10:46.474233+0000","last_fullsized":"2024-04-04T06:10:46.474233+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:47:39.578665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.389498+0000","last_change":"2024-04-04T06:10:45.380788+0000","last_active":"2024-04-04T06:10:45.389498+0000","last_peered":"2024-04-04T06:10:45.389498+0000","last_clean":"2024-04-04T06:10:45.389498+0000","last_became_active":"2024-04-04T06:10:45.378608+0000","last_became_peered":"2024-04-04T06:10:45.378608+0000","last_unstale":"2024-04-04T06:10:45.389498+0000","last_undegraded":"2024-04-04T06:10:45.389498+0000","last_fullsized":"2024-04-04T06:10:45.389498+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:21:31.764115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.389599+0000","last_change":"2024-04-04T06:10:45.379236+0000","last_active":"2024-04-04T06:10:45.389599+0000","last_peered":"2024-04-04T06:10:45.389599+0000","last_clean":"2024-04-04T06:10:45.389599+0000","last_became_active":"2024-04-04T06:10:45.378945+0000","last_became_peered":"2024-04-04T06:10:45.378945+0000","last_unstale":"2024-04-04T06:10:45.389599+0000","last_undegraded":"2024-04-04T06:10:45.389599+0000","last_fullsized":"2024-04-04T06:10:45.389599+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:46:46.746628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476499+0000","last_change":"2024-04-04T06:10:45.399926+0000","last_active":"2024-04-04T06:10:46.476499+0000","last_peered":"2024-04-04T06:10:46.476499+0000","last_clean":"2024-04-04T06:10:46.476499+0000","last_became_active":"2024-04-04T06:10:45.399598+0000","last_became_peered":"2024-04-04T06:10:45.399598+0000","last_unstale":"2024-04-04T06:10:46.476499+0000","last_undegraded":"2024-04-04T06:10:46.476499+0000","last_fullsized":"2024-04-04T06:10:46.476499+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:08:13.120498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476368+0000","last_change":"2024-04-04T06:10:45.400192+0000","last_active":"2024-04-04T06:10:46.476368+0000","last_peered":"2024-04-04T06:10:46.476368+0000","last_clean":"2024-04-04T06:10:46.476368+0000","last_became_active":"2024-04-04T06:10:45.399888+0000","last_became_peered":"2024-04-04T06:10:45.399888+0000","last_unstale":"2024-04-04T06:10:46.476368+0000","last_undegraded":"2024-04-04T06:10:46.476368+0000","last_fullsized":"2024-04-04T06:10:46.476368+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:34:00.152224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.395931+0000","last_change":"2024-04-04T06:10:45.391386+0000","last_active":"2024-04-04T06:10:45.395931+0000","last_peered":"2024-04-04T06:10:45.395931+0000","last_clean":"2024-04-04T06:10:45.395931+0000","last_became_active":"2024-04-04T06:10:45.388814+0000","last_became_peered":"2024-04-04T06:10:45.388814+0000","last_unstale":"2024-04-04T06:10:45.395931+0000","last_undegraded":"2024-04-04T06:10:45.395931+0000","last_fullsized":"2024-04-04T06:10:45.395931+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:22:04.238317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.397409+0000","last_change":"2024-04-04T06:10:45.387727+0000","last_active":"2024-04-04T06:10:45.397409+0000","last_peered":"2024-04-04T06:10:45.397409+0000","last_clean":"2024-04-04T06:10:45.397409+0000","last_became_active":"2024-04-04T06:10:45.387394+0000","last_became_peered":"2024-04-04T06:10:45.387394+0000","last_unstale":"2024-04-04T06:10:45.397409+0000","last_undegraded":"2024-04-04T06:10:45.397409+0000","last_fullsized":"2024-04-04T06:10:45.397409+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:32:20.170224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476462+0000","last_change":"2024-04-04T06:10:45.388506+0000","last_active":"2024-04-04T06:10:46.476462+0000","last_peered":"2024-04-04T06:10:46.476462+0000","last_clean":"2024-04-04T06:10:46.476462+0000","last_became_active":"2024-04-04T06:10:45.388155+0000","last_became_peered":"2024-04-04T06:10:45.388155+0000","last_unstale":"2024-04-04T06:10:46.476462+0000","last_undegraded":"2024-04-04T06:10:46.476462+0000","last_fullsized":"2024-04-04T06:10:46.476462+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:20:43.852925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.388544+0000","last_change":"2024-04-04T06:10:45.378430+0000","last_active":"2024-04-04T06:10:45.388544+0000","last_peered":"2024-04-04T06:10:45.388544+0000","last_clean":"2024-04-04T06:10:45.388544+0000","last_became_active":"2024-04-04T06:10:45.377194+0000","last_became_peered":"2024-04-04T06:10:45.377194+0000","last_unstale":"2024-04-04T06:10:45.388544+0000","last_undegraded":"2024-04-04T06:10:45.388544+0000","last_fullsized":"2024-04-04T06:10:45.388544+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:28:13.763888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476207+0000","last_change":"2024-04-04T06:10:45.387240+0000","last_active":"2024-04-04T06:10:46.476207+0000","last_peered":"2024-04-04T06:10:46.476207+0000","last_clean":"2024-04-04T06:10:46.476207+0000","last_became_active":"2024-04-04T06:10:45.386806+0000","last_became_peered":"2024-04-04T06:10:45.386806+0000","last_unstale":"2024-04-04T06:10:46.476207+0000","last_undegraded":"2024-04-04T06:10:46.476207+0000","last_fullsized":"2024-04-04T06:10:46.476207+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:47:28.510756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.397528+0000","last_change":"2024-04-04T06:10:45.395449+0000","last_active":"2024-04-04T06:10:45.397528+0000","last_peered":"2024-04-04T06:10:45.397528+0000","last_clean":"2024-04-04T06:10:45.397528+0000","last_became_active":"2024-04-04T06:10:45.394888+0000","last_became_peered":"2024-04-04T06:10:45.394888+0000","last_unstale":"2024-04-04T06:10:45.397528+0000","last_undegraded":"2024-04-04T06:10:45.397528+0000","last_fullsized":"2024-04-04T06:10:45.397528+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:59:22.303966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.388317+0000","last_change":"2024-04-04T06:10:45.373459+0000","last_active":"2024-04-04T06:10:45.388317+0000","last_peered":"2024-04-04T06:10:45.388317+0000","last_clean":"2024-04-04T06:10:45.388317+0000","last_became_active":"2024-04-04T06:10:45.369347+0000","last_became_peered":"2024-04-04T06:10:45.369347+0000","last_unstale":"2024-04-04T06:10:45.388317+0000","last_undegraded":"2024-04-04T06:10:45.388317+0000","last_fullsized":"2024-04-04T06:10:45.388317+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:12:26.070099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477164+0000","last_change":"2024-04-04T06:10:45.380717+0000","last_active":"2024-04-04T06:10:46.477164+0000","last_peered":"2024-04-04T06:10:46.477164+0000","last_clean":"2024-04-04T06:10:46.477164+0000","last_became_active":"2024-04-04T06:10:45.380355+0000","last_became_peered":"2024-04-04T06:10:45.380355+0000","last_unstale":"2024-04-04T06:10:46.477164+0000","last_undegraded":"2024-04-04T06:10:46.477164+0000","last_fullsized":"2024-04-04T06:10:46.477164+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:14:57.446878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477515+0000","last_change":"2024-04-04T06:10:45.398637+0000","last_active":"2024-04-04T06:10:46.477515+0000","last_peered":"2024-04-04T06:10:46.477515+0000","last_clean":"2024-04-04T06:10:46.477515+0000","last_became_active":"2024-04-04T06:10:45.398372+0000","last_became_peered":"2024-04-04T06:10:45.398372+0000","last_unstale":"2024-04-04T06:10:46.477515+0000","last_undegraded":"2024-04-04T06:10:46.477515+0000","last_fullsized":"2024-04-04T06:10:46.477515+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:55:58.281630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"70'4","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477879+0000","last_change":"2024-04-04T06:10:44.366052+0000","last_active":"2024-04-04T06:10:46.477879+0000","last_peered":"2024-04-04T06:10:46.477879+0000","last_clean":"2024-04-04T06:10:46.477879+0000","last_became_active":"2024-04-04T06:10:44.365761+0000","last_became_peered":"2024-04-04T06:10:44.365761+0000","last_unstale":"2024-04-04T06:10:46.477879+0000","last_undegraded":"2024-04-04T06:10:46.477879+0000","last_fullsized":"2024-04-04T06:10:46.477879+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:54:50.005704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474829+0000","last_change":"2024-04-04T06:10:45.375889+0000","last_active":"2024-04-04T06:10:46.474829+0000","last_peered":"2024-04-04T06:10:46.474829+0000","last_clean":"2024-04-04T06:10:46.474829+0000","last_became_active":"2024-04-04T06:10:45.374560+0000","last_became_peered":"2024-04-04T06:10:45.374560+0000","last_unstale":"2024-04-04T06:10:46.474829+0000","last_undegraded":"2024-04-04T06:10:46.474829+0000","last_fullsized":"2024-04-04T06:10:46.474829+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:39:51.812769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474649+0000","last_change":"2024-04-04T06:10:44.356271+0000","last_active":"2024-04-04T06:10:46.474649+0000","last_peered":"2024-04-04T06:10:46.474649+0000","last_clean":"2024-04-04T06:10:46.474649+0000","last_became_active":"2024-04-04T06:10:44.355847+0000","last_became_peered":"2024-04-04T06:10:44.355847+0000","last_unstale":"2024-04-04T06:10:46.474649+0000","last_undegraded":"2024-04-04T06:10:46.474649+0000","last_fullsized":"2024-04-04T06:10:46.474649+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:22:03.225278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.478431+0000","last_change":"2024-04-04T06:10:45.390865+0000","last_active":"2024-04-04T06:10:46.478431+0000","last_peered":"2024-04-04T06:10:46.478431+0000","last_clean":"2024-04-04T06:10:46.478431+0000","last_became_active":"2024-04-04T06:10:45.390173+0000","last_became_peered":"2024-04-04T06:10:45.390173+0000","last_unstale":"2024-04-04T06:10:46.478431+0000","last_undegraded":"2024-04-04T06:10:46.478431+0000","last_fullsized":"2024-04-04T06:10:46.478431+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:50:10.014804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.392651+0000","last_change":"2024-04-04T06:10:44.364867+0000","last_active":"2024-04-04T06:10:45.392651+0000","last_peered":"2024-04-04T06:10:45.392651+0000","last_clean":"2024-04-04T06:10:45.392651+0000","last_became_active":"2024-04-04T06:10:44.364546+0000","last_became_peered":"2024-04-04T06:10:44.364546+0000","last_unstale":"2024-04-04T06:10:45.392651+0000","last_undegraded":"2024-04-04T06:10:45.392651+0000","last_fullsized":"2024-04-04T06:10:45.392651+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:16:58.273572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477995+0000","last_change":"2024-04-04T06:10:45.401401+0000","last_active":"2024-04-04T06:10:46.477995+0000","last_peered":"2024-04-04T06:10:46.477995+0000","last_clean":"2024-04-04T06:10:46.477995+0000","last_became_active":"2024-04-04T06:10:45.400323+0000","last_became_peered":"2024-04-04T06:10:45.400323+0000","last_unstale":"2024-04-04T06:10:46.477995+0000","last_undegraded":"2024-04-04T06:10:46.477995+0000","last_fullsized":"2024-04-04T06:10:46.477995+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:41:36.949840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.478666+0000","last_change":"2024-04-04T06:10:44.361085+0000","last_active":"2024-04-04T06:10:46.478666+0000","last_peered":"2024-04-04T06:10:46.478666+0000","last_clean":"2024-04-04T06:10:46.478666+0000","last_became_active":"2024-04-04T06:10:44.360746+0000","last_became_peered":"2024-04-04T06:10:44.360746+0000","last_unstale":"2024-04-04T06:10:46.478666+0000","last_undegraded":"2024-04-04T06:10:46.478666+0000","last_fullsized":"2024-04-04T06:10:46.478666+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:16:18.652516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.395420+0000","last_change":"2024-04-04T06:10:45.382212+0000","last_active":"2024-04-04T06:10:45.395420+0000","last_peered":"2024-04-04T06:10:45.395420+0000","last_clean":"2024-04-04T06:10:45.395420+0000","last_became_active":"2024-04-04T06:10:45.381940+0000","last_became_peered":"2024-04-04T06:10:45.381940+0000","last_unstale":"2024-04-04T06:10:45.395420+0000","last_undegraded":"2024-04-04T06:10:45.395420+0000","last_fullsized":"2024-04-04T06:10:45.395420+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:51:25.033199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476313+0000","last_change":"2024-04-04T06:10:44.362321+0000","last_active":"2024-04-04T06:10:46.476313+0000","last_peered":"2024-04-04T06:10:46.476313+0000","last_clean":"2024-04-04T06:10:46.476313+0000","last_became_active":"2024-04-04T06:10:44.362018+0000","last_became_peered":"2024-04-04T06:10:44.362018+0000","last_unstale":"2024-04-04T06:10:46.476313+0000","last_undegraded":"2024-04-04T06:10:46.476313+0000","last_fullsized":"2024-04-04T06:10:46.476313+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:44:07.842083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.391777+0000","last_change":"2024-04-04T06:10:45.389618+0000","last_active":"2024-04-04T06:10:45.391777+0000","last_peered":"2024-04-04T06:10:45.391777+0000","last_clean":"2024-04-04T06:10:45.391777+0000","last_became_active":"2024-04-04T06:10:45.389283+0000","last_became_peered":"2024-04-04T06:10:45.389283+0000","last_unstale":"2024-04-04T06:10:45.391777+0000","last_undegraded":"2024-04-04T06:10:45.391777+0000","last_fullsized":"2024-04-04T06:10:45.391777+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:51:25.033199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477707+0000","last_change":"2024-04-04T06:10:44.353411+0000","last_active":"2024-04-04T06:10:46.477707+0000","last_peered":"2024-04-04T06:10:46.477707+0000","last_clean":"2024-04-04T06:10:46.477707+0000","last_became_active":"2024-04-04T06:10:44.353138+0000","last_became_peered":"2024-04-04T06:10:44.353138+0000","last_unstale":"2024-04-04T06:10:46.477707+0000","last_undegraded":"2024-04-04T06:10:46.477707+0000","last_fullsized":"2024-04-04T06:10:46.477707+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:35:41.094196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.473117+0000","last_change":"2024-04-04T06:10:45.385118+0000","last_active":"2024-04-04T06:10:46.473117+0000","last_peered":"2024-04-04T06:10:46.473117+0000","last_clean":"2024-04-04T06:10:46.473117+0000","last_became_active":"2024-04-04T06:10:45.384528+0000","last_became_peered":"2024-04-04T06:10:45.384528+0000","last_unstale":"2024-04-04T06:10:46.473117+0000","last_undegraded":"2024-04-04T06:10:46.473117+0000","last_fullsized":"2024-04-04T06:10:46.473117+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:54:56.626282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.376801+0000","last_change":"2024-04-04T06:10:44.360848+0000","last_active":"2024-04-04T06:10:46.376801+0000","last_peered":"2024-04-04T06:10:46.376801+0000","last_clean":"2024-04-04T06:10:46.376801+0000","last_became_active":"2024-04-04T06:10:44.360401+0000","last_became_peered":"2024-04-04T06:10:44.360401+0000","last_unstale":"2024-04-04T06:10:46.376801+0000","last_undegraded":"2024-04-04T06:10:46.376801+0000","last_fullsized":"2024-04-04T06:10:46.376801+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:18:40.577799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.473625+0000","last_change":"2024-04-04T06:10:45.394395+0000","last_active":"2024-04-04T06:10:46.473625+0000","last_peered":"2024-04-04T06:10:46.473625+0000","last_clean":"2024-04-04T06:10:46.473625+0000","last_became_active":"2024-04-04T06:10:45.394106+0000","last_became_peered":"2024-04-04T06:10:45.394106+0000","last_unstale":"2024-04-04T06:10:46.473625+0000","last_undegraded":"2024-04-04T06:10:46.473625+0000","last_fullsized":"2024-04-04T06:10:46.473625+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:33:19.610246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.473059+0000","last_change":"2024-04-04T06:10:44.349363+0000","last_active":"2024-04-04T06:10:46.473059+0000","last_peered":"2024-04-04T06:10:46.473059+0000","last_clean":"2024-04-04T06:10:46.473059+0000","last_became_active":"2024-04-04T06:10:44.349069+0000","last_became_peered":"2024-04-04T06:10:44.349069+0000","last_unstale":"2024-04-04T06:10:46.473059+0000","last_undegraded":"2024-04-04T06:10:46.473059+0000","last_fullsized":"2024-04-04T06:10:46.473059+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:27:10.227209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.397664+0000","last_change":"2024-04-04T06:10:45.395504+0000","last_active":"2024-04-04T06:10:45.397664+0000","last_peered":"2024-04-04T06:10:45.397664+0000","last_clean":"2024-04-04T06:10:45.397664+0000","last_became_active":"2024-04-04T06:10:45.395190+0000","last_became_peered":"2024-04-04T06:10:45.395190+0000","last_unstale":"2024-04-04T06:10:45.397664+0000","last_undegraded":"2024-04-04T06:10:45.397664+0000","last_fullsized":"2024-04-04T06:10:45.397664+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:43:59.662259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.360723+0000","last_change":"2024-04-04T06:10:44.359478+0000","last_active":"2024-04-04T06:10:45.360723+0000","last_peered":"2024-04-04T06:10:45.360723+0000","last_clean":"2024-04-04T06:10:45.360723+0000","last_became_active":"2024-04-04T06:10:44.359189+0000","last_became_peered":"2024-04-04T06:10:44.359189+0000","last_unstale":"2024-04-04T06:10:45.360723+0000","last_undegraded":"2024-04-04T06:10:45.360723+0000","last_fullsized":"2024-04-04T06:10:45.360723+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:36:00.949780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477180+0000","last_change":"2024-04-04T06:10:45.398428+0000","last_active":"2024-04-04T06:10:46.477180+0000","last_peered":"2024-04-04T06:10:46.477180+0000","last_clean":"2024-04-04T06:10:46.477180+0000","last_became_active":"2024-04-04T06:10:45.397762+0000","last_became_peered":"2024-04-04T06:10:45.397762+0000","last_unstale":"2024-04-04T06:10:46.477180+0000","last_undegraded":"2024-04-04T06:10:46.477180+0000","last_fullsized":"2024-04-04T06:10:46.477180+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:50:38.446613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.479253+0000","last_change":"2024-04-04T06:10:44.363052+0000","last_active":"2024-04-04T06:10:46.479253+0000","last_peered":"2024-04-04T06:10:46.479253+0000","last_clean":"2024-04-04T06:10:46.479253+0000","last_became_active":"2024-04-04T06:10:44.362802+0000","last_became_peered":"2024-04-04T06:10:44.362802+0000","last_unstale":"2024-04-04T06:10:46.479253+0000","last_undegraded":"2024-04-04T06:10:46.479253+0000","last_fullsized":"2024-04-04T06:10:46.479253+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:53:08.633127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.478487+0000","last_change":"2024-04-04T06:10:45.387129+0000","last_active":"2024-04-04T06:10:46.478487+0000","last_peered":"2024-04-04T06:10:46.478487+0000","last_clean":"2024-04-04T06:10:46.478487+0000","last_became_active":"2024-04-04T06:10:45.385346+0000","last_became_peered":"2024-04-04T06:10:45.385346+0000","last_unstale":"2024-04-04T06:10:46.478487+0000","last_undegraded":"2024-04-04T06:10:46.478487+0000","last_fullsized":"2024-04-04T06:10:46.478487+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:35:26.528398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"70'3","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476556+0000","last_change":"2024-04-04T06:10:44.359306+0000","last_active":"2024-04-04T06:10:46.476556+0000","last_peered":"2024-04-04T06:10:46.476556+0000","last_clean":"2024-04-04T06:10:46.476556+0000","last_became_active":"2024-04-04T06:10:44.358978+0000","last_became_peered":"2024-04-04T06:10:44.358978+0000","last_unstale":"2024-04-04T06:10:46.476556+0000","last_undegraded":"2024-04-04T06:10:46.476556+0000","last_fullsized":"2024-04-04T06:10:46.476556+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:55:42.949268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.473976+0000","last_change":"2024-04-04T06:10:45.373368+0000","last_active":"2024-04-04T06:10:46.473976+0000","last_peered":"2024-04-04T06:10:46.473976+0000","last_clean":"2024-04-04T06:10:46.473976+0000","last_became_active":"2024-04-04T06:10:45.373013+0000","last_became_peered":"2024-04-04T06:10:45.373013+0000","last_unstale":"2024-04-04T06:10:46.473976+0000","last_undegraded":"2024-04-04T06:10:46.473976+0000","last_fullsized":"2024-04-04T06:10:46.473976+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:11:16.794252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"70'6","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.377224+0000","last_change":"2024-04-04T06:10:44.353379+0000","last_active":"2024-04-04T06:10:46.377224+0000","last_peered":"2024-04-04T06:10:46.377224+0000","last_clean":"2024-04-04T06:10:46.377224+0000","last_became_active":"2024-04-04T06:10:44.351976+0000","last_became_peered":"2024-04-04T06:10:44.351976+0000","last_unstale":"2024-04-04T06:10:46.377224+0000","last_undegraded":"2024-04-04T06:10:46.377224+0000","last_fullsized":"2024-04-04T06:10:46.377224+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:37:31.341831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477933+0000","last_change":"2024-04-04T06:10:45.396962+0000","last_active":"2024-04-04T06:10:46.477933+0000","last_peered":"2024-04-04T06:10:46.477933+0000","last_clean":"2024-04-04T06:10:46.477933+0000","last_became_active":"2024-04-04T06:10:45.394768+0000","last_became_peered":"2024-04-04T06:10:45.394768+0000","last_unstale":"2024-04-04T06:10:46.477933+0000","last_undegraded":"2024-04-04T06:10:46.477933+0000","last_fullsized":"2024-04-04T06:10:46.477933+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:55:33.843420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.377331+0000","last_change":"2024-04-04T06:10:44.360227+0000","last_active":"2024-04-04T06:10:46.377331+0000","last_peered":"2024-04-04T06:10:46.377331+0000","last_clean":"2024-04-04T06:10:46.377331+0000","last_became_active":"2024-04-04T06:10:44.359905+0000","last_became_peered":"2024-04-04T06:10:44.359905+0000","last_unstale":"2024-04-04T06:10:46.377331+0000","last_undegraded":"2024-04-04T06:10:46.377331+0000","last_fullsized":"2024-04-04T06:10:46.377331+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:07:08.850656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.394612+0000","last_change":"2024-04-04T06:10:45.383894+0000","last_active":"2024-04-04T06:10:45.394612+0000","last_peered":"2024-04-04T06:10:45.394612+0000","last_clean":"2024-04-04T06:10:45.394612+0000","last_became_active":"2024-04-04T06:10:45.383573+0000","last_became_peered":"2024-04-04T06:10:45.383573+0000","last_unstale":"2024-04-04T06:10:45.394612+0000","last_undegraded":"2024-04-04T06:10:45.394612+0000","last_fullsized":"2024-04-04T06:10:45.394612+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:22:04.238317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477951+0000","last_change":"2024-04-04T06:10:44.357439+0000","last_active":"2024-04-04T06:10:46.477951+0000","last_peered":"2024-04-04T06:10:46.477951+0000","last_clean":"2024-04-04T06:10:46.477951+0000","last_became_active":"2024-04-04T06:10:44.357159+0000","last_became_peered":"2024-04-04T06:10:44.357159+0000","last_unstale":"2024-04-04T06:10:46.477951+0000","last_undegraded":"2024-04-04T06:10:46.477951+0000","last_fullsized":"2024-04-04T06:10:46.477951+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:30:29.678930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.473117+0000","last_change":"2024-04-04T06:10:45.377697+0000","last_active":"2024-04-04T06:10:46.473117+0000","last_peered":"2024-04-04T06:10:46.473117+0000","last_clean":"2024-04-04T06:10:46.473117+0000","last_became_active":"2024-04-04T06:10:45.377297+0000","last_became_peered":"2024-04-04T06:10:45.377297+0000","last_unstale":"2024-04-04T06:10:46.473117+0000","last_undegraded":"2024-04-04T06:10:46.473117+0000","last_fullsized":"2024-04-04T06:10:46.473117+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:59:12.100267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.360174+0000","last_change":"2024-04-04T06:10:44.359666+0000","last_active":"2024-04-04T06:10:45.360174+0000","last_peered":"2024-04-04T06:10:45.360174+0000","last_clean":"2024-04-04T06:10:45.360174+0000","last_became_active":"2024-04-04T06:10:44.359382+0000","last_became_peered":"2024-04-04T06:10:44.359382+0000","last_unstale":"2024-04-04T06:10:45.360174+0000","last_undegraded":"2024-04-04T06:10:45.360174+0000","last_fullsized":"2024-04-04T06:10:45.360174+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:46:45.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.473312+0000","last_change":"2024-04-04T06:10:45.375623+0000","last_active":"2024-04-04T06:10:46.473312+0000","last_peered":"2024-04-04T06:10:46.473312+0000","last_clean":"2024-04-04T06:10:46.473312+0000","last_became_active":"2024-04-04T06:10:45.375319+0000","last_became_peered":"2024-04-04T06:10:45.375319+0000","last_unstale":"2024-04-04T06:10:46.473312+0000","last_undegraded":"2024-04-04T06:10:46.473312+0000","last_fullsized":"2024-04-04T06:10:46.473312+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:33:39.468998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"70'8","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.397256+0000","last_change":"2024-04-04T06:10:44.362714+0000","last_active":"2024-04-04T06:10:45.397256+0000","last_peered":"2024-04-04T06:10:45.397256+0000","last_clean":"2024-04-04T06:10:45.397256+0000","last_became_active":"2024-04-04T06:10:44.362399+0000","last_became_peered":"2024-04-04T06:10:44.362399+0000","last_unstale":"2024-04-04T06:10:45.397256+0000","last_undegraded":"2024-04-04T06:10:45.397256+0000","last_fullsized":"2024-04-04T06:10:45.397256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:05:03.011322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.389486+0000","last_change":"2024-04-04T06:10:45.381395+0000","last_active":"2024-04-04T06:10:45.389486+0000","last_peered":"2024-04-04T06:10:45.389486+0000","last_clean":"2024-04-04T06:10:45.389486+0000","last_became_active":"2024-04-04T06:10:45.380870+0000","last_became_peered":"2024-04-04T06:10:45.380870+0000","last_unstale":"2024-04-04T06:10:45.389486+0000","last_undegraded":"2024-04-04T06:10:45.389486+0000","last_fullsized":"2024-04-04T06:10:45.389486+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:35:54.626600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.479010+0000","last_change":"2024-04-04T06:10:44.359654+0000","last_active":"2024-04-04T06:10:46.479010+0000","last_peered":"2024-04-04T06:10:46.479010+0000","last_clean":"2024-04-04T06:10:46.479010+0000","last_became_active":"2024-04-04T06:10:44.359356+0000","last_became_peered":"2024-04-04T06:10:44.359356+0000","last_unstale":"2024-04-04T06:10:46.479010+0000","last_undegraded":"2024-04-04T06:10:46.479010+0000","last_fullsized":"2024-04-04T06:10:46.479010+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:33:55.568263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477611+0000","last_change":"2024-04-04T06:10:45.394559+0000","last_active":"2024-04-04T06:10:46.477611+0000","last_peered":"2024-04-04T06:10:46.477611+0000","last_clean":"2024-04-04T06:10:46.477611+0000","last_became_active":"2024-04-04T06:10:45.392959+0000","last_became_peered":"2024-04-04T06:10:45.392959+0000","last_unstale":"2024-04-04T06:10:46.477611+0000","last_undegraded":"2024-04-04T06:10:46.477611+0000","last_fullsized":"2024-04-04T06:10:46.477611+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:01:10.310581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.392832+0000","last_change":"2024-04-04T06:10:44.366245+0000","last_active":"2024-04-04T06:10:45.392832+0000","last_peered":"2024-04-04T06:10:45.392832+0000","last_clean":"2024-04-04T06:10:45.392832+0000","last_became_active":"2024-04-04T06:10:44.365966+0000","last_became_peered":"2024-04-04T06:10:44.365966+0000","last_unstale":"2024-04-04T06:10:45.392832+0000","last_undegraded":"2024-04-04T06:10:45.392832+0000","last_fullsized":"2024-04-04T06:10:45.392832+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:52:28.198053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.377137+0000","last_change":"2024-04-04T06:10:45.369750+0000","last_active":"2024-04-04T06:10:46.377137+0000","last_peered":"2024-04-04T06:10:46.377137+0000","last_clean":"2024-04-04T06:10:46.377137+0000","last_became_active":"2024-04-04T06:10:45.369477+0000","last_became_peered":"2024-04-04T06:10:45.369477+0000","last_unstale":"2024-04-04T06:10:46.377137+0000","last_undegraded":"2024-04-04T06:10:46.377137+0000","last_fullsized":"2024-04-04T06:10:46.377137+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:08:51.975530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.478708+0000","last_change":"2024-04-04T06:10:44.363184+0000","last_active":"2024-04-04T06:10:46.478708+0000","last_peered":"2024-04-04T06:10:46.478708+0000","last_clean":"2024-04-04T06:10:46.478708+0000","last_became_active":"2024-04-04T06:10:44.361823+0000","last_became_peered":"2024-04-04T06:10:44.361823+0000","last_unstale":"2024-04-04T06:10:46.478708+0000","last_undegraded":"2024-04-04T06:10:46.478708+0000","last_fullsized":"2024-04-04T06:10:46.478708+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:18:07.093490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474115+0000","last_change":"2024-04-04T06:10:45.382954+0000","last_active":"2024-04-04T06:10:46.474115+0000","last_peered":"2024-04-04T06:10:46.474115+0000","last_clean":"2024-04-04T06:10:46.474115+0000","last_became_active":"2024-04-04T06:10:45.382680+0000","last_became_peered":"2024-04-04T06:10:45.382680+0000","last_unstale":"2024-04-04T06:10:46.474115+0000","last_undegraded":"2024-04-04T06:10:46.474115+0000","last_fullsized":"2024-04-04T06:10:46.474115+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:07:09.863695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476828+0000","last_change":"2024-04-04T06:10:44.367018+0000","last_active":"2024-04-04T06:10:46.476828+0000","last_peered":"2024-04-04T06:10:46.476828+0000","last_clean":"2024-04-04T06:10:46.476828+0000","last_became_active":"2024-04-04T06:10:44.366681+0000","last_became_peered":"2024-04-04T06:10:44.366681+0000","last_unstale":"2024-04-04T06:10:46.476828+0000","last_undegraded":"2024-04-04T06:10:46.476828+0000","last_fullsized":"2024-04-04T06:10:46.476828+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:32:55.018331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476224+0000","last_change":"2024-04-04T06:10:45.381316+0000","last_active":"2024-04-04T06:10:46.476224+0000","last_peered":"2024-04-04T06:10:46.476224+0000","last_clean":"2024-04-04T06:10:46.476224+0000","last_became_active":"2024-04-04T06:10:45.379688+0000","last_became_peered":"2024-04-04T06:10:45.379688+0000","last_unstale":"2024-04-04T06:10:46.476224+0000","last_undegraded":"2024-04-04T06:10:46.476224+0000","last_fullsized":"2024-04-04T06:10:46.476224+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:50:02.015578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.478200+0000","last_change":"2024-04-04T06:10:44.353285+0000","last_active":"2024-04-04T06:10:46.478200+0000","last_peered":"2024-04-04T06:10:46.478200+0000","last_clean":"2024-04-04T06:10:46.478200+0000","last_became_active":"2024-04-04T06:10:44.352800+0000","last_became_peered":"2024-04-04T06:10:44.352800+0000","last_unstale":"2024-04-04T06:10:46.478200+0000","last_undegraded":"2024-04-04T06:10:46.478200+0000","last_fullsized":"2024-04-04T06:10:46.478200+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:56:38.203099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.475971+0000","last_change":"2024-04-04T06:10:45.386831+0000","last_active":"2024-04-04T06:10:46.475971+0000","last_peered":"2024-04-04T06:10:46.475971+0000","last_clean":"2024-04-04T06:10:46.475971+0000","last_became_active":"2024-04-04T06:10:45.386489+0000","last_became_peered":"2024-04-04T06:10:45.386489+0000","last_unstale":"2024-04-04T06:10:46.475971+0000","last_undegraded":"2024-04-04T06:10:46.475971+0000","last_fullsized":"2024-04-04T06:10:46.475971+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:36:08.592530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.357381+0000","last_change":"2024-04-04T06:10:44.358192+0000","last_active":"2024-04-04T06:10:45.357381+0000","last_peered":"2024-04-04T06:10:45.357381+0000","last_clean":"2024-04-04T06:10:45.357381+0000","last_became_active":"2024-04-04T06:10:44.357231+0000","last_became_peered":"2024-04-04T06:10:44.357231+0000","last_unstale":"2024-04-04T06:10:45.357381+0000","last_undegraded":"2024-04-04T06:10:45.357381+0000","last_fullsized":"2024-04-04T06:10:45.357381+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:28:12.750849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476077+0000","last_change":"2024-04-04T06:10:45.383693+0000","last_active":"2024-04-04T06:10:46.476077+0000","last_peered":"2024-04-04T06:10:46.476077+0000","last_clean":"2024-04-04T06:10:46.476077+0000","last_became_active":"2024-04-04T06:10:45.380079+0000","last_became_peered":"2024-04-04T06:10:45.380079+0000","last_unstale":"2024-04-04T06:10:46.476077+0000","last_undegraded":"2024-04-04T06:10:46.476077+0000","last_fullsized":"2024-04-04T06:10:46.476077+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:28:26.165425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.472966+0000","last_change":"2024-04-04T06:10:44.357800+0000","last_active":"2024-04-04T06:10:46.472966+0000","last_peered":"2024-04-04T06:10:46.472966+0000","last_clean":"2024-04-04T06:10:46.472966+0000","last_became_active":"2024-04-04T06:10:44.357015+0000","last_became_peered":"2024-04-04T06:10:44.357015+0000","last_unstale":"2024-04-04T06:10:46.472966+0000","last_undegraded":"2024-04-04T06:10:46.472966+0000","last_fullsized":"2024-04-04T06:10:46.472966+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:35:53.613561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474549+0000","last_change":"2024-04-04T06:10:45.369229+0000","last_active":"2024-04-04T06:10:46.474549+0000","last_peered":"2024-04-04T06:10:46.474549+0000","last_clean":"2024-04-04T06:10:46.474549+0000","last_became_active":"2024-04-04T06:10:45.368952+0000","last_became_peered":"2024-04-04T06:10:45.368952+0000","last_unstale":"2024-04-04T06:10:46.474549+0000","last_undegraded":"2024-04-04T06:10:46.474549+0000","last_fullsized":"2024-04-04T06:10:46.474549+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:50.702749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.479140+0000","last_change":"2024-04-04T06:10:44.360294+0000","last_active":"2024-04-04T06:10:46.479140+0000","last_peered":"2024-04-04T06:10:46.479140+0000","last_clean":"2024-04-04T06:10:46.479140+0000","last_became_active":"2024-04-04T06:10:44.359973+0000","last_became_peered":"2024-04-04T06:10:44.359973+0000","last_unstale":"2024-04-04T06:10:46.479140+0000","last_undegraded":"2024-04-04T06:10:46.479140+0000","last_fullsized":"2024-04-04T06:10:46.479140+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:40:18.010754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.393723+0000","last_change":"2024-04-04T06:10:45.380011+0000","last_active":"2024-04-04T06:10:45.393723+0000","last_peered":"2024-04-04T06:10:45.393723+0000","last_clean":"2024-04-04T06:10:45.393723+0000","last_became_active":"2024-04-04T06:10:45.379736+0000","last_became_peered":"2024-04-04T06:10:45.379736+0000","last_unstale":"2024-04-04T06:10:45.393723+0000","last_undegraded":"2024-04-04T06:10:45.393723+0000","last_fullsized":"2024-04-04T06:10:45.393723+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:35:54.626600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.360016+0000","last_change":"2024-04-04T06:10:44.354715+0000","last_active":"2024-04-04T06:10:45.360016+0000","last_peered":"2024-04-04T06:10:45.360016+0000","last_clean":"2024-04-04T06:10:45.360016+0000","last_became_active":"2024-04-04T06:10:44.354396+0000","last_became_peered":"2024-04-04T06:10:44.354396+0000","last_unstale":"2024-04-04T06:10:45.360016+0000","last_undegraded":"2024-04-04T06:10:45.360016+0000","last_fullsized":"2024-04-04T06:10:45.360016+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:58:35.602169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476587+0000","last_change":"2024-04-04T06:10:45.398370+0000","last_active":"2024-04-04T06:10:46.476587+0000","last_peered":"2024-04-04T06:10:46.476587+0000","last_clean":"2024-04-04T06:10:46.476587+0000","last_became_active":"2024-04-04T06:10:45.398085+0000","last_became_peered":"2024-04-04T06:10:45.398085+0000","last_unstale":"2024-04-04T06:10:46.476587+0000","last_undegraded":"2024-04-04T06:10:46.476587+0000","last_fullsized":"2024-04-04T06:10:46.476587+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:31:00.494486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474216+0000","last_change":"2024-04-04T06:10:44.352577+0000","last_active":"2024-04-04T06:10:46.474216+0000","last_peered":"2024-04-04T06:10:46.474216+0000","last_clean":"2024-04-04T06:10:46.474216+0000","last_became_active":"2024-04-04T06:10:44.352298+0000","last_became_peered":"2024-04-04T06:10:44.352298+0000","last_unstale":"2024-04-04T06:10:46.474216+0000","last_undegraded":"2024-04-04T06:10:46.474216+0000","last_fullsized":"2024-04-04T06:10:46.474216+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:55:04.000965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.478098+0000","last_change":"2024-04-04T06:10:45.401164+0000","last_active":"2024-04-04T06:10:46.478098+0000","last_peered":"2024-04-04T06:10:46.478098+0000","last_clean":"2024-04-04T06:10:46.478098+0000","last_became_active":"2024-04-04T06:10:45.400515+0000","last_became_peered":"2024-04-04T06:10:45.400515+0000","last_unstale":"2024-04-04T06:10:46.478098+0000","last_undegraded":"2024-04-04T06:10:46.478098+0000","last_fullsized":"2024-04-04T06:10:46.478098+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:15:01.516699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474224+0000","last_change":"2024-04-04T06:10:44.362280+0000","last_active":"2024-04-04T06:10:46.474224+0000","last_peered":"2024-04-04T06:10:46.474224+0000","last_clean":"2024-04-04T06:10:46.474224+0000","last_became_active":"2024-04-04T06:10:44.361114+0000","last_became_peered":"2024-04-04T06:10:44.361114+0000","last_unstale":"2024-04-04T06:10:46.474224+0000","last_undegraded":"2024-04-04T06:10:46.474224+0000","last_fullsized":"2024-04-04T06:10:46.474224+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:51:24.020161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.473820+0000","last_change":"2024-04-04T06:10:45.374897+0000","last_active":"2024-04-04T06:10:46.473820+0000","last_peered":"2024-04-04T06:10:46.473820+0000","last_clean":"2024-04-04T06:10:46.473820+0000","last_became_active":"2024-04-04T06:10:45.374594+0000","last_became_peered":"2024-04-04T06:10:45.374594+0000","last_unstale":"2024-04-04T06:10:46.473820+0000","last_undegraded":"2024-04-04T06:10:46.473820+0000","last_fullsized":"2024-04-04T06:10:46.473820+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:49:43.283191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474469+0000","last_change":"2024-04-04T06:10:44.360110+0000","last_active":"2024-04-04T06:10:46.474469+0000","last_peered":"2024-04-04T06:10:46.474469+0000","last_clean":"2024-04-04T06:10:46.474469+0000","last_became_active":"2024-04-04T06:10:44.359637+0000","last_became_peered":"2024-04-04T06:10:44.359637+0000","last_unstale":"2024-04-04T06:10:46.474469+0000","last_undegraded":"2024-04-04T06:10:46.474469+0000","last_fullsized":"2024-04-04T06:10:46.474469+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:11:18.559137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476930+0000","last_change":"2024-04-04T06:10:45.388218+0000","last_active":"2024-04-04T06:10:46.476930+0000","last_peered":"2024-04-04T06:10:46.476930+0000","last_clean":"2024-04-04T06:10:46.476930+0000","last_became_active":"2024-04-04T06:10:45.387941+0000","last_became_peered":"2024-04-04T06:10:45.387941+0000","last_unstale":"2024-04-04T06:10:46.476930+0000","last_undegraded":"2024-04-04T06:10:46.476930+0000","last_fullsized":"2024-04-04T06:10:46.476930+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:54:51.018743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.376656+0000","last_change":"2024-04-04T06:10:44.358955+0000","last_active":"2024-04-04T06:10:46.376656+0000","last_peered":"2024-04-04T06:10:46.376656+0000","last_clean":"2024-04-04T06:10:46.376656+0000","last_became_active":"2024-04-04T06:10:44.358666+0000","last_became_peered":"2024-04-04T06:10:44.358666+0000","last_unstale":"2024-04-04T06:10:46.376656+0000","last_undegraded":"2024-04-04T06:10:46.376656+0000","last_fullsized":"2024-04-04T06:10:46.376656+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:32:38.730974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.376741+0000","last_change":"2024-04-04T06:10:45.376805+0000","last_active":"2024-04-04T06:10:46.376741+0000","last_peered":"2024-04-04T06:10:46.376741+0000","last_clean":"2024-04-04T06:10:46.376741+0000","last_became_active":"2024-04-04T06:10:45.376407+0000","last_became_peered":"2024-04-04T06:10:45.376407+0000","last_unstale":"2024-04-04T06:10:46.376741+0000","last_undegraded":"2024-04-04T06:10:46.376741+0000","last_fullsized":"2024-04-04T06:10:46.376741+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:30:49.308665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.357626+0000","last_change":"2024-04-04T06:10:44.362621+0000","last_active":"2024-04-04T06:10:45.357626+0000","last_peered":"2024-04-04T06:10:45.357626+0000","last_clean":"2024-04-04T06:10:45.357626+0000","last_became_active":"2024-04-04T06:10:44.362342+0000","last_became_peered":"2024-04-04T06:10:44.362342+0000","last_unstale":"2024-04-04T06:10:45.357626+0000","last_undegraded":"2024-04-04T06:10:45.357626+0000","last_fullsized":"2024-04-04T06:10:45.357626+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:06:34.175687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476892+0000","last_change":"2024-04-04T06:10:45.381890+0000","last_active":"2024-04-04T06:10:46.476892+0000","last_peered":"2024-04-04T06:10:46.476892+0000","last_clean":"2024-04-04T06:10:46.476892+0000","last_became_active":"2024-04-04T06:10:45.381578+0000","last_became_peered":"2024-04-04T06:10:45.381578+0000","last_unstale":"2024-04-04T06:10:46.476892+0000","last_undegraded":"2024-04-04T06:10:46.476892+0000","last_fullsized":"2024-04-04T06:10:46.476892+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:05:07.220786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477445+0000","last_change":"2024-04-04T06:10:44.357737+0000","last_active":"2024-04-04T06:10:46.477445+0000","last_peered":"2024-04-04T06:10:46.477445+0000","last_clean":"2024-04-04T06:10:46.477445+0000","last_became_active":"2024-04-04T06:10:44.357480+0000","last_became_peered":"2024-04-04T06:10:44.357480+0000","last_unstale":"2024-04-04T06:10:46.477445+0000","last_undegraded":"2024-04-04T06:10:46.477445+0000","last_fullsized":"2024-04-04T06:10:46.477445+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:00:44.279660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476183+0000","last_change":"2024-04-04T06:10:45.393261+0000","last_active":"2024-04-04T06:10:46.476183+0000","last_peered":"2024-04-04T06:10:46.476183+0000","last_clean":"2024-04-04T06:10:46.476183+0000","last_became_active":"2024-04-04T06:10:45.392886+0000","last_became_peered":"2024-04-04T06:10:45.392886+0000","last_unstale":"2024-04-04T06:10:46.476183+0000","last_undegraded":"2024-04-04T06:10:46.476183+0000","last_fullsized":"2024-04-04T06:10:46.476183+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:09:18.404824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476836+0000","last_change":"2024-04-04T06:10:44.360453+0000","last_active":"2024-04-04T06:10:46.476836+0000","last_peered":"2024-04-04T06:10:46.476836+0000","last_clean":"2024-04-04T06:10:46.476836+0000","last_became_active":"2024-04-04T06:10:44.360139+0000","last_became_peered":"2024-04-04T06:10:44.360139+0000","last_unstale":"2024-04-04T06:10:46.476836+0000","last_undegraded":"2024-04-04T06:10:46.476836+0000","last_fullsized":"2024-04-04T06:10:46.476836+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:46:27.776188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":260,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476256+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:10:46.476256+0000","last_peered":"2024-04-04T06:10:46.476256+0000","last_clean":"2024-04-04T06:10:46.476256+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:10:46.476256+0000","last_undegraded":"2024-04-04T06:10:46.476256+0000","last_fullsized":"2024-04-04T06:10:46.476256+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509503,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1458176,"data_stored":951845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509505,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":864256,"data_stored":360953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":851968,"data_stored":354673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509505,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":362473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509504,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":864256,"data_stored":356789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509506,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1437696,"data_stored":944929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542209,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":847872,"data_stored":354651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542207,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":831488,"data_stored":347786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:10:49.821 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:10:50.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.071+0000 7fc9ae600640 1 Processor -- start 2024-04-04T06:10:50.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.071+0000 7fc9ae600640 1 -- start start 2024-04-04T06:10:50.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.072+0000 7fc9ae600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9a80cc0b0 0x7fc9a806dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:50.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.072+0000 7fc9ae600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9a806e4f0 con 0x7fc9a80d04d0 2024-04-04T06:10:50.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.072+0000 7fc9ae600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9a80b3150 con 0x7fc9a815fc50 2024-04-04T06:10:50.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.073+0000 7fc9ae600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc9a815c610 con 0x7fc9a80cc0b0 2024-04-04T06:10:50.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.073+0000 7fc9adc00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fc9a80d04d0 0x7fc9a815ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:44434/0 (socket says 172.21.15.159:44434) 2024-04-04T06:10:50.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.073+0000 7fc9adc00640 1 -- 172.21.15.159:0/3612180358 learned_addr learned my addr 172.21.15.159:0/3612180358 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:50.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.073+0000 7fc9ad200640 1 --2- 172.21.15.159:0/3612180358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9a80cc0b0 0x7fc9a806dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:50.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.073+0000 7fc9a7e00640 1 -- 172.21.15.159:0/3612180358 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 605438472 0 0) 0x7fc9a806e4f0 con 0x7fc9a80d04d0 2024-04-04T06:10:50.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.073+0000 7fc9a7e00640 1 -- 172.21.15.159:0/3612180358 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc98c003520 con 0x7fc9a80d04d0 2024-04-04T06:10:50.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.074+0000 7fc9a7e00640 1 -- 172.21.15.159:0/3612180358 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3067605947 0 0) 0x7fc98c003520 con 0x7fc9a80d04d0 2024-04-04T06:10:50.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.074+0000 7fc9a7e00640 1 -- 172.21.15.159:0/3612180358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9a80cc0b0 msgr2=0x7fc9a806dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:50.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.074+0000 7fc9a7e00640 1 --2- 172.21.15.159:0/3612180358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9a80cc0b0 0x7fc9a806dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:50.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.074+0000 7fc9a7e00640 1 -- 172.21.15.159:0/3612180358 >> v1:172.21.15.178:6789/0 conn(0x7fc9a815fc50 legacy=0x7fc9a80cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:50.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.074+0000 7fc9a7e00640 1 -- 172.21.15.159:0/3612180358 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a81600e0 con 0x7fc9a80d04d0 2024-04-04T06:10:50.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.074+0000 7fc9a7e00640 1 -- 172.21.15.159:0/3612180358 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fc9a00019f0 con 0x7fc9a80d04d0 2024-04-04T06:10:50.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.074+0000 7fc9a7e00640 1 -- 172.21.15.159:0/3612180358 >> v1:172.21.15.159:6789/0 conn(0x7fc9a80d04d0 legacy=0x7fc9a815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:50.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.074+0000 7fc9a7e00640 1 --2- 172.21.15.159:0/3612180358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc98c0039e0 0x7fc98c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:50.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.074+0000 7fc9a7e00640 1 --2- 172.21.15.159:0/3612180358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc98c004800 0x7fc98c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:50.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.074+0000 7fc9a7e00640 1 --2- 172.21.15.159:0/3612180358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9a80cc0b0 0x7fc98c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:50.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.074+0000 7fc9a7e00640 1 -- 172.21.15.159:0/3612180358 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a815c610 con 0x7fc98c004800 2024-04-04T06:10:50.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.074+0000 7fc9a7e00640 1 -- 172.21.15.159:0/3612180358 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc98c00ba70 con 0x7fc98c0039e0 2024-04-04T06:10:50.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.074+0000 7fc9a7e00640 1 -- 172.21.15.159:0/3612180358 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc98c00bc20 con 0x7fc9a80cc0b0 2024-04-04T06:10:50.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.074+0000 7fc99fe00640 1 --2- 172.21.15.159:0/3612180358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc98c004800 0x7fc98c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:50.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.075+0000 7fc9ad200640 1 --2- 172.21.15.159:0/3612180358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc98c0039e0 0x7fc98c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:50.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.075+0000 7fc99fe00640 1 -- 172.21.15.159:0/3612180358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9a80cc0b0 msgr2=0x7fc98c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:50.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.075+0000 7fc99fe00640 1 --2- 172.21.15.159:0/3612180358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc9a80cc0b0 0x7fc98c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:50.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.075+0000 7fc99fe00640 1 -- 172.21.15.159:0/3612180358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc98c0039e0 msgr2=0x7fc98c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:50.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.075+0000 7fc99fe00640 1 --2- 172.21.15.159:0/3612180358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc98c0039e0 0x7fc98c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:50.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.075+0000 7fc99fe00640 1 -- 172.21.15.159:0/3612180358 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98c003520 con 0x7fc98c004800 2024-04-04T06:10:50.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.075+0000 7fc9ad200640 1 --2- 172.21.15.159:0/3612180358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc98c0039e0 0x7fc98c003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:50.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.076+0000 7fc99fe00640 1 --2- 172.21.15.159:0/3612180358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc98c004800 0x7fc98c004c80 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fc990001830 tx=0x7fc98c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47740fb60b97fc80 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:50.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.076+0000 7fc9a7e00640 1 -- 172.21.15.159:0/3612180358 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc990001430 con 0x7fc98c004800 2024-04-04T06:10:50.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.076+0000 7fc9a7e00640 1 -- 172.21.15.159:0/3612180358 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc990032620 con 0x7fc98c004800 2024-04-04T06:10:50.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.076+0000 7fc9a7e00640 1 -- 172.21.15.159:0/3612180358 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc990032a10 con 0x7fc98c004800 2024-04-04T06:10:50.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.077+0000 7fc9ae600640 1 -- 172.21.15.159:0/3612180358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc98c004800 msgr2=0x7fc98c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:50.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.077+0000 7fc9ae600640 1 --2- 172.21.15.159:0/3612180358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc98c004800 0x7fc98c004c80 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fc990001830 tx=0x7fc98c005830 comp rx=0 tx=0).stop 2024-04-04T06:10:50.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.077+0000 7fc9adc00640 1 -- 172.21.15.159:0/3612180358 reap_dead start 2024-04-04T06:10:50.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.077+0000 7fc9ae600640 1 -- 172.21.15.159:0/3612180358 shutdown_connections 2024-04-04T06:10:50.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.077+0000 7fc9ae600640 1 -- 172.21.15.159:0/3612180358 >> 172.21.15.159:0/3612180358 conn(0x7fc9a80ccad0 msgr2=0x7fc9a80cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:50.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.077+0000 7fc9ae600640 1 -- 172.21.15.159:0/3612180358 shutdown_connections 2024-04-04T06:10:50.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.077+0000 7fc9ae600640 1 -- 172.21.15.159:0/3612180358 wait complete. 2024-04-04T06:10:50.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.079+0000 7fc9ae600640 1 Processor -- start 2024-04-04T06:10:50.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.079+0000 7fc9ae600640 1 -- start start 2024-04-04T06:10:50.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.080+0000 7fc9ae600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc98c004800 0x7fc9a8155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:50.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.080+0000 7fc9ae600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9a813a9a0 0x7fc9a81e5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:50.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.080+0000 7fc9ae600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9a813d700 0x7fc9a81f4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:50.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.080+0000 7fc9ae600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a8160980 con 0x7fc9a813a9a0 2024-04-04T06:10:50.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.080+0000 7fc9ae600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a81606a0 con 0x7fc9a813d700 2024-04-04T06:10:50.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.080+0000 7fc9ae600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc9a80cc490 con 0x7fc98c004800 2024-04-04T06:10:50.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.080+0000 7fc99fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9a813a9a0 0x7fc9a81e5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:50.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.080+0000 7fc99fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9a813a9a0 0x7fc9a81e5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:32770/0 (socket says 172.21.15.159:32770) 2024-04-04T06:10:50.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.080+0000 7fc99fe00640 1 -- 172.21.15.159:0/1930760965 learned_addr learned my addr 172.21.15.159:0/1930760965 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:50.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.080+0000 7fc99fe00640 1 -- 172.21.15.159:0/1930760965 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc98c004800 msgr2=0x7fc9a8155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:50.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.080+0000 7fc99fe00640 1 --2- 172.21.15.159:0/1930760965 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc98c004800 0x7fc9a8155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:50.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.080+0000 7fc99fe00640 1 -- 172.21.15.159:0/1930760965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9a813d700 msgr2=0x7fc9a81f4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:50.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.080+0000 7fc99fe00640 1 --2- 172.21.15.159:0/1930760965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9a813d700 0x7fc9a81f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:50.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.080+0000 7fc99fe00640 1 -- 172.21.15.159:0/1930760965 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a8155d50 con 0x7fc9a813a9a0 2024-04-04T06:10:50.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.081+0000 7fc99fe00640 1 --2- 172.21.15.159:0/1930760965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9a813a9a0 0x7fc9a81e5970 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc99002bdd0 tx=0x7fc990001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:50.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.081+0000 7fc9a5600640 1 -- 172.21.15.159:0/1930760965 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc990007050 con 0x7fc9a813a9a0 2024-04-04T06:10:50.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.081+0000 7fc9a5600640 1 -- 172.21.15.159:0/1930760965 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9900071f0 con 0x7fc9a813a9a0 2024-04-04T06:10:50.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.081+0000 7fc9ae600640 1 -- 172.21.15.159:0/1930760965 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a81e5eb0 con 0x7fc9a813a9a0 2024-04-04T06:10:50.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.081+0000 7fc9a5600640 1 -- 172.21.15.159:0/1930760965 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9900075c0 con 0x7fc9a813a9a0 2024-04-04T06:10:50.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.081+0000 7fc9ae600640 1 -- 172.21.15.159:0/1930760965 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a813cc20 con 0x7fc9a813a9a0 2024-04-04T06:10:50.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.083+0000 7fc9ae600640 1 -- 172.21.15.159:0/1930760965 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9a8075940 con 0x7fc9a813a9a0 2024-04-04T06:10:50.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.083+0000 7fc9a5600640 1 -- 172.21.15.159:0/1930760965 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc9900067d0 con 0x7fc9a813a9a0 2024-04-04T06:10:50.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.084+0000 7fc9a5600640 1 --2- 172.21.15.159:0/1930760965 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc98406c8f0 0x7fc98406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:50.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.084+0000 7fc9a5600640 1 -- 172.21.15.159:0/1930760965 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fc9900ad7e0 con 0x7fc9a813a9a0 2024-04-04T06:10:50.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.085+0000 7fc9ad200640 1 --2- 172.21.15.159:0/1930760965 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc98406c8f0 0x7fc98406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:50.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.086+0000 7fc9ad200640 1 --2- 172.21.15.159:0/1930760965 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc98406c8f0 0x7fc98406edb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc9980099e0 tx=0x7fc998008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:50.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.088+0000 7fc9a5600640 1 -- 172.21.15.159:0/1930760965 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc9a8155d50 con 0x7fc9a813a9a0 2024-04-04T06:10:50.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.284+0000 7fc9ae600640 1 -- 172.21.15.159:0/1930760965 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc9a8073830 con 0x7fc98406c8f0 2024-04-04T06:10:50.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.302+0000 7fc9a5600640 1 -- 172.21.15.159:0/1930760965 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252046 (secure 0 0 0) 0x7fc9a8073830 con 0x7fc98406c8f0 2024-04-04T06:10:50.321 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:50.323 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:10:50.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.307+0000 7fc9ae600640 1 -- 172.21.15.159:0/1930760965 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc98406c8f0 msgr2=0x7fc98406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:50.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.307+0000 7fc9ae600640 1 --2- 172.21.15.159:0/1930760965 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc98406c8f0 0x7fc98406edb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc9980099e0 tx=0x7fc998008040 comp rx=0 tx=0).stop 2024-04-04T06:10:50.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.307+0000 7fc9ae600640 1 -- 172.21.15.159:0/1930760965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9a813a9a0 msgr2=0x7fc9a81e5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:50.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.307+0000 7fc9ae600640 1 --2- 172.21.15.159:0/1930760965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9a813a9a0 0x7fc9a81e5970 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc99002bdd0 tx=0x7fc990001230 comp rx=0 tx=0).stop 2024-04-04T06:10:50.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.307+0000 7fc9ae600640 1 -- 172.21.15.159:0/1930760965 shutdown_connections 2024-04-04T06:10:50.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.307+0000 7fc9ae600640 1 --2- 172.21.15.159:0/1930760965 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc98406c8f0 0x7fc98406edb0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:50.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.307+0000 7fc9ae600640 1 --2- 172.21.15.159:0/1930760965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9a813d700 0x7fc9a81f4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:50.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.307+0000 7fc9ae600640 1 --2- 172.21.15.159:0/1930760965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9a813a9a0 0x7fc9a81e5970 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:50.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.307+0000 7fc9ae600640 1 --2- 172.21.15.159:0/1930760965 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc98c004800 0x7fc9a8155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:50.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.307+0000 7fc9ae600640 1 -- 172.21.15.159:0/1930760965 >> 172.21.15.159:0/1930760965 conn(0x7fc9a80ccad0 msgr2=0x7fc9a80cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:50.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.307+0000 7fc9ae600640 1 -- 172.21.15.159:0/1930760965 shutdown_connections 2024-04-04T06:10:50.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.307+0000 7fc9ae600640 1 -- 172.21.15.159:0/1930760965 wait complete. 2024-04-04T06:10:50.328 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-04-04T06:10:48.818724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257316,"kb_used_data":7836,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717404,"statfs":{"total":773094113280,"available":772830621696,"internally_reserved":0,"allocated":8024064,"data_stored":4034099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.495170"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476782+0000","last_change":"2024-04-04T06:10:45.398821+0000","last_active":"2024-04-04T06:10:46.476782+0000","last_peered":"2024-04-04T06:10:46.476782+0000","last_clean":"2024-04-04T06:10:46.476782+0000","last_became_active":"2024-04-04T06:10:45.395222+0000","last_became_peered":"2024-04-04T06:10:45.395222+0000","last_unstale":"2024-04-04T06:10:46.476782+0000","last_undegraded":"2024-04-04T06:10:46.476782+0000","last_fullsized":"2024-04-04T06:10:46.476782+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:41:23.938682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.479110+0000","last_change":"2024-04-04T06:10:45.394232+0000","last_active":"2024-04-04T06:10:46.479110+0000","last_peered":"2024-04-04T06:10:46.479110+0000","last_clean":"2024-04-04T06:10:46.479110+0000","last_became_active":"2024-04-04T06:10:45.393961+0000","last_became_peered":"2024-04-04T06:10:45.393961+0000","last_unstale":"2024-04-04T06:10:46.479110+0000","last_undegraded":"2024-04-04T06:10:46.479110+0000","last_fullsized":"2024-04-04T06:10:46.479110+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:35:45.072858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476442+0000","last_change":"2024-04-04T06:10:45.380449+0000","last_active":"2024-04-04T06:10:46.476442+0000","last_peered":"2024-04-04T06:10:46.476442+0000","last_clean":"2024-04-04T06:10:46.476442+0000","last_became_active":"2024-04-04T06:10:45.380179+0000","last_became_peered":"2024-04-04T06:10:45.380179+0000","last_unstale":"2024-04-04T06:10:46.476442+0000","last_undegraded":"2024-04-04T06:10:46.476442+0000","last_fullsized":"2024-04-04T06:10:46.476442+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:57:56.388699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.376989+0000","last_change":"2024-04-04T06:10:45.386910+0000","last_active":"2024-04-04T06:10:46.376989+0000","last_peered":"2024-04-04T06:10:46.376989+0000","last_clean":"2024-04-04T06:10:46.376989+0000","last_became_active":"2024-04-04T06:10:45.386631+0000","last_became_peered":"2024-04-04T06:10:45.386631+0000","last_unstale":"2024-04-04T06:10:46.376989+0000","last_undegraded":"2024-04-04T06:10:46.376989+0000","last_fullsized":"2024-04-04T06:10:46.376989+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:28:16.796576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474376+0000","last_change":"2024-04-04T06:10:45.389152+0000","last_active":"2024-04-04T06:10:46.474376+0000","last_peered":"2024-04-04T06:10:46.474376+0000","last_clean":"2024-04-04T06:10:46.474376+0000","last_became_active":"2024-04-04T06:10:45.388860+0000","last_became_peered":"2024-04-04T06:10:45.388860+0000","last_unstale":"2024-04-04T06:10:46.474376+0000","last_undegraded":"2024-04-04T06:10:46.474376+0000","last_fullsized":"2024-04-04T06:10:46.474376+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:18:41.590838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477421+0000","last_change":"2024-04-04T06:10:45.393878+0000","last_active":"2024-04-04T06:10:46.477421+0000","last_peered":"2024-04-04T06:10:46.477421+0000","last_clean":"2024-04-04T06:10:46.477421+0000","last_became_active":"2024-04-04T06:10:45.393399+0000","last_became_peered":"2024-04-04T06:10:45.393399+0000","last_unstale":"2024-04-04T06:10:46.477421+0000","last_undegraded":"2024-04-04T06:10:46.477421+0000","last_fullsized":"2024-04-04T06:10:46.477421+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:53:15.246653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.388340+0000","last_change":"2024-04-04T06:10:45.374167+0000","last_active":"2024-04-04T06:10:45.388340+0000","last_peered":"2024-04-04T06:10:45.388340+0000","last_clean":"2024-04-04T06:10:45.388340+0000","last_became_active":"2024-04-04T06:10:45.373899+0000","last_became_peered":"2024-04-04T06:10:45.373899+0000","last_unstale":"2024-04-04T06:10:45.388340+0000","last_undegraded":"2024-04-04T06:10:45.388340+0000","last_fullsized":"2024-04-04T06:10:45.388340+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:21:31.764115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.478192+0000","last_change":"2024-04-04T06:10:45.390552+0000","last_active":"2024-04-04T06:10:46.478192+0000","last_peered":"2024-04-04T06:10:46.478192+0000","last_clean":"2024-04-04T06:10:46.478192+0000","last_became_active":"2024-04-04T06:10:45.390232+0000","last_became_peered":"2024-04-04T06:10:45.390232+0000","last_unstale":"2024-04-04T06:10:46.478192+0000","last_undegraded":"2024-04-04T06:10:46.478192+0000","last_fullsized":"2024-04-04T06:10:46.478192+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:21:57.676061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474750+0000","last_change":"2024-04-04T06:10:45.376493+0000","last_active":"2024-04-04T06:10:46.474750+0000","last_peered":"2024-04-04T06:10:46.474750+0000","last_clean":"2024-04-04T06:10:46.474750+0000","last_became_active":"2024-04-04T06:10:45.375608+0000","last_became_peered":"2024-04-04T06:10:45.375608+0000","last_unstale":"2024-04-04T06:10:46.474750+0000","last_undegraded":"2024-04-04T06:10:46.474750+0000","last_fullsized":"2024-04-04T06:10:46.474750+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:32:39.744013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.389463+0000","last_change":"2024-04-04T06:10:45.379126+0000","last_active":"2024-04-04T06:10:45.389463+0000","last_peered":"2024-04-04T06:10:45.389463+0000","last_clean":"2024-04-04T06:10:45.389463+0000","last_became_active":"2024-04-04T06:10:45.378825+0000","last_became_peered":"2024-04-04T06:10:45.378825+0000","last_unstale":"2024-04-04T06:10:45.389463+0000","last_undegraded":"2024-04-04T06:10:45.389463+0000","last_fullsized":"2024-04-04T06:10:45.389463+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:58:36.615208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477747+0000","last_change":"2024-04-04T06:10:45.394375+0000","last_active":"2024-04-04T06:10:46.477747+0000","last_peered":"2024-04-04T06:10:46.477747+0000","last_clean":"2024-04-04T06:10:46.477747+0000","last_became_active":"2024-04-04T06:10:45.394032+0000","last_became_peered":"2024-04-04T06:10:45.394032+0000","last_unstale":"2024-04-04T06:10:46.477747+0000","last_undegraded":"2024-04-04T06:10:46.477747+0000","last_fullsized":"2024-04-04T06:10:46.477747+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:36:33.342681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.377071+0000","last_change":"2024-04-04T06:10:45.372707+0000","last_active":"2024-04-04T06:10:46.377071+0000","last_peered":"2024-04-04T06:10:46.377071+0000","last_clean":"2024-04-04T06:10:46.377071+0000","last_became_active":"2024-04-04T06:10:45.372215+0000","last_became_peered":"2024-04-04T06:10:45.372215+0000","last_unstale":"2024-04-04T06:10:46.377071+0000","last_undegraded":"2024-04-04T06:10:46.377071+0000","last_fullsized":"2024-04-04T06:10:46.377071+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:41:15.412553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474019+0000","last_change":"2024-04-04T06:10:45.393013+0000","last_active":"2024-04-04T06:10:46.474019+0000","last_peered":"2024-04-04T06:10:46.474019+0000","last_clean":"2024-04-04T06:10:46.474019+0000","last_became_active":"2024-04-04T06:10:45.391895+0000","last_became_peered":"2024-04-04T06:10:45.391895+0000","last_unstale":"2024-04-04T06:10:46.474019+0000","last_undegraded":"2024-04-04T06:10:46.474019+0000","last_fullsized":"2024-04-04T06:10:46.474019+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:19:17.088841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.473678+0000","last_change":"2024-04-04T06:10:45.382277+0000","last_active":"2024-04-04T06:10:46.473678+0000","last_peered":"2024-04-04T06:10:46.473678+0000","last_clean":"2024-04-04T06:10:46.473678+0000","last_became_active":"2024-04-04T06:10:45.381995+0000","last_became_peered":"2024-04-04T06:10:45.381995+0000","last_unstale":"2024-04-04T06:10:46.473678+0000","last_undegraded":"2024-04-04T06:10:46.473678+0000","last_fullsized":"2024-04-04T06:10:46.473678+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:59:52.200063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.473481+0000","last_change":"2024-04-04T06:10:45.370994+0000","last_active":"2024-04-04T06:10:46.473481+0000","last_peered":"2024-04-04T06:10:46.473481+0000","last_clean":"2024-04-04T06:10:46.473481+0000","last_became_active":"2024-04-04T06:10:45.369525+0000","last_became_peered":"2024-04-04T06:10:45.369525+0000","last_unstale":"2024-04-04T06:10:46.473481+0000","last_undegraded":"2024-04-04T06:10:46.473481+0000","last_fullsized":"2024-04-04T06:10:46.473481+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:30:07.102180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.389319+0000","last_change":"2024-04-04T06:10:45.379855+0000","last_active":"2024-04-04T06:10:45.389319+0000","last_peered":"2024-04-04T06:10:45.389319+0000","last_clean":"2024-04-04T06:10:45.389319+0000","last_became_active":"2024-04-04T06:10:45.379563+0000","last_became_peered":"2024-04-04T06:10:45.379563+0000","last_unstale":"2024-04-04T06:10:45.389319+0000","last_undegraded":"2024-04-04T06:10:45.389319+0000","last_fullsized":"2024-04-04T06:10:45.389319+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:12:26.070099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474772+0000","last_change":"2024-04-04T06:10:45.376194+0000","last_active":"2024-04-04T06:10:46.474772+0000","last_peered":"2024-04-04T06:10:46.474772+0000","last_clean":"2024-04-04T06:10:46.474772+0000","last_became_active":"2024-04-04T06:10:45.374933+0000","last_became_peered":"2024-04-04T06:10:45.374933+0000","last_unstale":"2024-04-04T06:10:46.474772+0000","last_undegraded":"2024-04-04T06:10:46.474772+0000","last_fullsized":"2024-04-04T06:10:46.474772+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:37:32.354870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.377218+0000","last_change":"2024-04-04T06:10:45.393074+0000","last_active":"2024-04-04T06:10:46.377218+0000","last_peered":"2024-04-04T06:10:46.377218+0000","last_clean":"2024-04-04T06:10:46.377218+0000","last_became_active":"2024-04-04T06:10:45.392451+0000","last_became_peered":"2024-04-04T06:10:45.392451+0000","last_unstale":"2024-04-04T06:10:46.377218+0000","last_undegraded":"2024-04-04T06:10:46.377218+0000","last_fullsized":"2024-04-04T06:10:46.377218+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:44:43.716753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476960+0000","last_change":"2024-04-04T06:10:45.389247+0000","last_active":"2024-04-04T06:10:46.476960+0000","last_peered":"2024-04-04T06:10:46.476960+0000","last_clean":"2024-04-04T06:10:46.476960+0000","last_became_active":"2024-04-04T06:10:45.388939+0000","last_became_peered":"2024-04-04T06:10:45.388939+0000","last_unstale":"2024-04-04T06:10:46.476960+0000","last_undegraded":"2024-04-04T06:10:46.476960+0000","last_fullsized":"2024-04-04T06:10:46.476960+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:32:56.031370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474233+0000","last_change":"2024-04-04T06:10:45.381039+0000","last_active":"2024-04-04T06:10:46.474233+0000","last_peered":"2024-04-04T06:10:46.474233+0000","last_clean":"2024-04-04T06:10:46.474233+0000","last_became_active":"2024-04-04T06:10:45.380715+0000","last_became_peered":"2024-04-04T06:10:45.380715+0000","last_unstale":"2024-04-04T06:10:46.474233+0000","last_undegraded":"2024-04-04T06:10:46.474233+0000","last_fullsized":"2024-04-04T06:10:46.474233+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:47:39.578665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.389498+0000","last_change":"2024-04-04T06:10:45.380788+0000","last_active":"2024-04-04T06:10:45.389498+0000","last_peered":"2024-04-04T06:10:45.389498+0000","last_clean":"2024-04-04T06:10:45.389498+0000","last_became_active":"2024-04-04T06:10:45.378608+0000","last_became_peered":"2024-04-04T06:10:45.378608+0000","last_unstale":"2024-04-04T06:10:45.389498+0000","last_undegraded":"2024-04-04T06:10:45.389498+0000","last_fullsized":"2024-04-04T06:10:45.389498+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:21:31.764115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.389599+0000","last_change":"2024-04-04T06:10:45.379236+0000","last_active":"2024-04-04T06:10:45.389599+0000","last_peered":"2024-04-04T06:10:45.389599+0000","last_clean":"2024-04-04T06:10:45.389599+0000","last_became_active":"2024-04-04T06:10:45.378945+0000","last_became_peered":"2024-04-04T06:10:45.378945+0000","last_unstale":"2024-04-04T06:10:45.389599+0000","last_undegraded":"2024-04-04T06:10:45.389599+0000","last_fullsized":"2024-04-04T06:10:45.389599+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:46:46.746628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476499+0000","last_change":"2024-04-04T06:10:45.399926+0000","last_active":"2024-04-04T06:10:46.476499+0000","last_peered":"2024-04-04T06:10:46.476499+0000","last_clean":"2024-04-04T06:10:46.476499+0000","last_became_active":"2024-04-04T06:10:45.399598+0000","last_became_peered":"2024-04-04T06:10:45.399598+0000","last_unstale":"2024-04-04T06:10:46.476499+0000","last_undegraded":"2024-04-04T06:10:46.476499+0000","last_fullsized":"2024-04-04T06:10:46.476499+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:08:13.120498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476368+0000","last_change":"2024-04-04T06:10:45.400192+0000","last_active":"2024-04-04T06:10:46.476368+0000","last_peered":"2024-04-04T06:10:46.476368+0000","last_clean":"2024-04-04T06:10:46.476368+0000","last_became_active":"2024-04-04T06:10:45.399888+0000","last_became_peered":"2024-04-04T06:10:45.399888+0000","last_unstale":"2024-04-04T06:10:46.476368+0000","last_undegraded":"2024-04-04T06:10:46.476368+0000","last_fullsized":"2024-04-04T06:10:46.476368+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:34:00.152224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.395931+0000","last_change":"2024-04-04T06:10:45.391386+0000","last_active":"2024-04-04T06:10:45.395931+0000","last_peered":"2024-04-04T06:10:45.395931+0000","last_clean":"2024-04-04T06:10:45.395931+0000","last_became_active":"2024-04-04T06:10:45.388814+0000","last_became_peered":"2024-04-04T06:10:45.388814+0000","last_unstale":"2024-04-04T06:10:45.395931+0000","last_undegraded":"2024-04-04T06:10:45.395931+0000","last_fullsized":"2024-04-04T06:10:45.395931+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:22:04.238317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.397409+0000","last_change":"2024-04-04T06:10:45.387727+0000","last_active":"2024-04-04T06:10:45.397409+0000","last_peered":"2024-04-04T06:10:45.397409+0000","last_clean":"2024-04-04T06:10:45.397409+0000","last_became_active":"2024-04-04T06:10:45.387394+0000","last_became_peered":"2024-04-04T06:10:45.387394+0000","last_unstale":"2024-04-04T06:10:45.397409+0000","last_undegraded":"2024-04-04T06:10:45.397409+0000","last_fullsized":"2024-04-04T06:10:45.397409+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:32:20.170224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476462+0000","last_change":"2024-04-04T06:10:45.388506+0000","last_active":"2024-04-04T06:10:46.476462+0000","last_peered":"2024-04-04T06:10:46.476462+0000","last_clean":"2024-04-04T06:10:46.476462+0000","last_became_active":"2024-04-04T06:10:45.388155+0000","last_became_peered":"2024-04-04T06:10:45.388155+0000","last_unstale":"2024-04-04T06:10:46.476462+0000","last_undegraded":"2024-04-04T06:10:46.476462+0000","last_fullsized":"2024-04-04T06:10:46.476462+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:20:43.852925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.388544+0000","last_change":"2024-04-04T06:10:45.378430+0000","last_active":"2024-04-04T06:10:45.388544+0000","last_peered":"2024-04-04T06:10:45.388544+0000","last_clean":"2024-04-04T06:10:45.388544+0000","last_became_active":"2024-04-04T06:10:45.377194+0000","last_became_peered":"2024-04-04T06:10:45.377194+0000","last_unstale":"2024-04-04T06:10:45.388544+0000","last_undegraded":"2024-04-04T06:10:45.388544+0000","last_fullsized":"2024-04-04T06:10:45.388544+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:28:13.763888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476207+0000","last_change":"2024-04-04T06:10:45.387240+0000","last_active":"2024-04-04T06:10:46.476207+0000","last_peered":"2024-04-04T06:10:46.476207+0000","last_clean":"2024-04-04T06:10:46.476207+0000","last_became_active":"2024-04-04T06:10:45.386806+0000","last_became_peered":"2024-04-04T06:10:45.386806+0000","last_unstale":"2024-04-04T06:10:46.476207+0000","last_undegraded":"2024-04-04T06:10:46.476207+0000","last_fullsized":"2024-04-04T06:10:46.476207+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:47:28.510756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.397528+0000","last_change":"2024-04-04T06:10:45.395449+0000","last_active":"2024-04-04T06:10:45.397528+0000","last_peered":"2024-04-04T06:10:45.397528+0000","last_clean":"2024-04-04T06:10:45.397528+0000","last_became_active":"2024-04-04T06:10:45.394888+0000","last_became_peered":"2024-04-04T06:10:45.394888+0000","last_unstale":"2024-04-04T06:10:45.397528+0000","last_undegraded":"2024-04-04T06:10:45.397528+0000","last_fullsized":"2024-04-04T06:10:45.397528+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:59:22.303966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.388317+0000","last_change":"2024-04-04T06:10:45.373459+0000","last_active":"2024-04-04T06:10:45.388317+0000","last_peered":"2024-04-04T06:10:45.388317+0000","last_clean":"2024-04-04T06:10:45.388317+0000","last_became_active":"2024-04-04T06:10:45.369347+0000","last_became_peered":"2024-04-04T06:10:45.369347+0000","last_unstale":"2024-04-04T06:10:45.388317+0000","last_undegraded":"2024-04-04T06:10:45.388317+0000","last_fullsized":"2024-04-04T06:10:45.388317+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:12:26.070099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477164+0000","last_change":"2024-04-04T06:10:45.380717+0000","last_active":"2024-04-04T06:10:46.477164+0000","last_peered":"2024-04-04T06:10:46.477164+0000","last_clean":"2024-04-04T06:10:46.477164+0000","last_became_active":"2024-04-04T06:10:45.380355+0000","last_became_peered":"2024-04-04T06:10:45.380355+0000","last_unstale":"2024-04-04T06:10:46.477164+0000","last_undegraded":"2024-04-04T06:10:46.477164+0000","last_fullsized":"2024-04-04T06:10:46.477164+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:14:57.446878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477515+0000","last_change":"2024-04-04T06:10:45.398637+0000","last_active":"2024-04-04T06:10:46.477515+0000","last_peered":"2024-04-04T06:10:46.477515+0000","last_clean":"2024-04-04T06:10:46.477515+0000","last_became_active":"2024-04-04T06:10:45.398372+0000","last_became_peered":"2024-04-04T06:10:45.398372+0000","last_unstale":"2024-04-04T06:10:46.477515+0000","last_undegraded":"2024-04-04T06:10:46.477515+0000","last_fullsized":"2024-04-04T06:10:46.477515+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:55:58.281630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"70'4","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477879+0000","last_change":"2024-04-04T06:10:44.366052+0000","last_active":"2024-04-04T06:10:46.477879+0000","last_peered":"2024-04-04T06:10:46.477879+0000","last_clean":"2024-04-04T06:10:46.477879+0000","last_became_active":"2024-04-04T06:10:44.365761+0000","last_became_peered":"2024-04-04T06:10:44.365761+0000","last_unstale":"2024-04-04T06:10:46.477879+0000","last_undegraded":"2024-04-04T06:10:46.477879+0000","last_fullsized":"2024-04-04T06:10:46.477879+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:54:50.005704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474829+0000","last_change":"2024-04-04T06:10:45.375889+0000","last_active":"2024-04-04T06:10:46.474829+0000","last_peered":"2024-04-04T06:10:46.474829+0000","last_clean":"2024-04-04T06:10:46.474829+0000","last_became_active":"2024-04-04T06:10:45.374560+0000","last_became_peered":"2024-04-04T06:10:45.374560+0000","last_unstale":"2024-04-04T06:10:46.474829+0000","last_undegraded":"2024-04-04T06:10:46.474829+0000","last_fullsized":"2024-04-04T06:10:46.474829+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:39:51.812769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474649+0000","last_change":"2024-04-04T06:10:44.356271+0000","last_active":"2024-04-04T06:10:46.474649+0000","last_peered":"2024-04-04T06:10:46.474649+0000","last_clean":"2024-04-04T06:10:46.474649+0000","last_became_active":"2024-04-04T06:10:44.355847+0000","last_became_peered":"2024-04-04T06:10:44.355847+0000","last_unstale":"2024-04-04T06:10:46.474649+0000","last_undegraded":"2024-04-04T06:10:46.474649+0000","last_fullsized":"2024-04-04T06:10:46.474649+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:22:03.225278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.478431+0000","last_change":"2024-04-04T06:10:45.390865+0000","last_active":"2024-04-04T06:10:46.478431+0000","last_peered":"2024-04-04T06:10:46.478431+0000","last_clean":"2024-04-04T06:10:46.478431+0000","last_became_active":"2024-04-04T06:10:45.390173+0000","last_became_peered":"2024-04-04T06:10:45.390173+0000","last_unstale":"2024-04-04T06:10:46.478431+0000","last_undegraded":"2024-04-04T06:10:46.478431+0000","last_fullsized":"2024-04-04T06:10:46.478431+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:50:10.014804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.392651+0000","last_change":"2024-04-04T06:10:44.364867+0000","last_active":"2024-04-04T06:10:45.392651+0000","last_peered":"2024-04-04T06:10:45.392651+0000","last_clean":"2024-04-04T06:10:45.392651+0000","last_became_active":"2024-04-04T06:10:44.364546+0000","last_became_peered":"2024-04-04T06:10:44.364546+0000","last_unstale":"2024-04-04T06:10:45.392651+0000","last_undegraded":"2024-04-04T06:10:45.392651+0000","last_fullsized":"2024-04-04T06:10:45.392651+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:16:58.273572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477995+0000","last_change":"2024-04-04T06:10:45.401401+0000","last_active":"2024-04-04T06:10:46.477995+0000","last_peered":"2024-04-04T06:10:46.477995+0000","last_clean":"2024-04-04T06:10:46.477995+0000","last_became_active":"2024-04-04T06:10:45.400323+0000","last_became_peered":"2024-04-04T06:10:45.400323+0000","last_unstale":"2024-04-04T06:10:46.477995+0000","last_undegraded":"2024-04-04T06:10:46.477995+0000","last_fullsized":"2024-04-04T06:10:46.477995+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:41:36.949840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.478666+0000","last_change":"2024-04-04T06:10:44.361085+0000","last_active":"2024-04-04T06:10:46.478666+0000","last_peered":"2024-04-04T06:10:46.478666+0000","last_clean":"2024-04-04T06:10:46.478666+0000","last_became_active":"2024-04-04T06:10:44.360746+0000","last_became_peered":"2024-04-04T06:10:44.360746+0000","last_unstale":"2024-04-04T06:10:46.478666+0000","last_undegraded":"2024-04-04T06:10:46.478666+0000","last_fullsized":"2024-04-04T06:10:46.478666+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:16:18.652516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.395420+0000","last_change":"2024-04-04T06:10:45.382212+0000","last_active":"2024-04-04T06:10:45.395420+0000","last_peered":"2024-04-04T06:10:45.395420+0000","last_clean":"2024-04-04T06:10:45.395420+0000","last_became_active":"2024-04-04T06:10:45.381940+0000","last_became_peered":"2024-04-04T06:10:45.381940+0000","last_unstale":"2024-04-04T06:10:45.395420+0000","last_undegraded":"2024-04-04T06:10:45.395420+0000","last_fullsized":"2024-04-04T06:10:45.395420+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:51:25.033199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476313+0000","last_change":"2024-04-04T06:10:44.362321+0000","last_active":"2024-04-04T06:10:46.476313+0000","last_peered":"2024-04-04T06:10:46.476313+0000","last_clean":"2024-04-04T06:10:46.476313+0000","last_became_active":"2024-04-04T06:10:44.362018+0000","last_became_peered":"2024-04-04T06:10:44.362018+0000","last_unstale":"2024-04-04T06:10:46.476313+0000","last_undegraded":"2024-04-04T06:10:46.476313+0000","last_fullsized":"2024-04-04T06:10:46.476313+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:44:07.842083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.391777+0000","last_change":"2024-04-04T06:10:45.389618+0000","last_active":"2024-04-04T06:10:45.391777+0000","last_peered":"2024-04-04T06:10:45.391777+0000","last_clean":"2024-04-04T06:10:45.391777+0000","last_became_active":"2024-04-04T06:10:45.389283+0000","last_became_peered":"2024-04-04T06:10:45.389283+0000","last_unstale":"2024-04-04T06:10:45.391777+0000","last_undegraded":"2024-04-04T06:10:45.391777+0000","last_fullsized":"2024-04-04T06:10:45.391777+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:51:25.033199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477707+0000","last_change":"2024-04-04T06:10:44.353411+0000","last_active":"2024-04-04T06:10:46.477707+0000","last_peered":"2024-04-04T06:10:46.477707+0000","last_clean":"2024-04-04T06:10:46.477707+0000","last_became_active":"2024-04-04T06:10:44.353138+0000","last_became_peered":"2024-04-04T06:10:44.353138+0000","last_unstale":"2024-04-04T06:10:46.477707+0000","last_undegraded":"2024-04-04T06:10:46.477707+0000","last_fullsized":"2024-04-04T06:10:46.477707+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:35:41.094196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.473117+0000","last_change":"2024-04-04T06:10:45.385118+0000","last_active":"2024-04-04T06:10:46.473117+0000","last_peered":"2024-04-04T06:10:46.473117+0000","last_clean":"2024-04-04T06:10:46.473117+0000","last_became_active":"2024-04-04T06:10:45.384528+0000","last_became_peered":"2024-04-04T06:10:45.384528+0000","last_unstale":"2024-04-04T06:10:46.473117+0000","last_undegraded":"2024-04-04T06:10:46.473117+0000","last_fullsized":"2024-04-04T06:10:46.473117+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:54:56.626282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.376801+0000","last_change":"2024-04-04T06:10:44.360848+0000","last_active":"2024-04-04T06:10:46.376801+0000","last_peered":"2024-04-04T06:10:46.376801+0000","last_clean":"2024-04-04T06:10:46.376801+0000","last_became_active":"2024-04-04T06:10:44.360401+0000","last_became_peered":"2024-04-04T06:10:44.360401+0000","last_unstale":"2024-04-04T06:10:46.376801+0000","last_undegraded":"2024-04-04T06:10:46.376801+0000","last_fullsized":"2024-04-04T06:10:46.376801+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:18:40.577799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.473625+0000","last_change":"2024-04-04T06:10:45.394395+0000","last_active":"2024-04-04T06:10:46.473625+0000","last_peered":"2024-04-04T06:10:46.473625+0000","last_clean":"2024-04-04T06:10:46.473625+0000","last_became_active":"2024-04-04T06:10:45.394106+0000","last_became_peered":"2024-04-04T06:10:45.394106+0000","last_unstale":"2024-04-04T06:10:46.473625+0000","last_undegraded":"2024-04-04T06:10:46.473625+0000","last_fullsized":"2024-04-04T06:10:46.473625+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:33:19.610246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.473059+0000","last_change":"2024-04-04T06:10:44.349363+0000","last_active":"2024-04-04T06:10:46.473059+0000","last_peered":"2024-04-04T06:10:46.473059+0000","last_clean":"2024-04-04T06:10:46.473059+0000","last_became_active":"2024-04-04T06:10:44.349069+0000","last_became_peered":"2024-04-04T06:10:44.349069+0000","last_unstale":"2024-04-04T06:10:46.473059+0000","last_undegraded":"2024-04-04T06:10:46.473059+0000","last_fullsized":"2024-04-04T06:10:46.473059+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:27:10.227209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.397664+0000","last_change":"2024-04-04T06:10:45.395504+0000","last_active":"2024-04-04T06:10:45.397664+0000","last_peered":"2024-04-04T06:10:45.397664+0000","last_clean":"2024-04-04T06:10:45.397664+0000","last_became_active":"2024-04-04T06:10:45.395190+0000","last_became_peered":"2024-04-04T06:10:45.395190+0000","last_unstale":"2024-04-04T06:10:45.397664+0000","last_undegraded":"2024-04-04T06:10:45.397664+0000","last_fullsized":"2024-04-04T06:10:45.397664+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:43:59.662259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.360723+0000","last_change":"2024-04-04T06:10:44.359478+0000","last_active":"2024-04-04T06:10:45.360723+0000","last_peered":"2024-04-04T06:10:45.360723+0000","last_clean":"2024-04-04T06:10:45.360723+0000","last_became_active":"2024-04-04T06:10:44.359189+0000","last_became_peered":"2024-04-04T06:10:44.359189+0000","last_unstale":"2024-04-04T06:10:45.360723+0000","last_undegraded":"2024-04-04T06:10:45.360723+0000","last_fullsized":"2024-04-04T06:10:45.360723+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:36:00.949780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477180+0000","last_change":"2024-04-04T06:10:45.398428+0000","last_active":"2024-04-04T06:10:46.477180+0000","last_peered":"2024-04-04T06:10:46.477180+0000","last_clean":"2024-04-04T06:10:46.477180+0000","last_became_active":"2024-04-04T06:10:45.397762+0000","last_became_peered":"2024-04-04T06:10:45.397762+0000","last_unstale":"2024-04-04T06:10:46.477180+0000","last_undegraded":"2024-04-04T06:10:46.477180+0000","last_fullsized":"2024-04-04T06:10:46.477180+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:50:38.446613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.479253+0000","last_change":"2024-04-04T06:10:44.363052+0000","last_active":"2024-04-04T06:10:46.479253+0000","last_peered":"2024-04-04T06:10:46.479253+0000","last_clean":"2024-04-04T06:10:46.479253+0000","last_became_active":"2024-04-04T06:10:44.362802+0000","last_became_peered":"2024-04-04T06:10:44.362802+0000","last_unstale":"2024-04-04T06:10:46.479253+0000","last_undegraded":"2024-04-04T06:10:46.479253+0000","last_fullsized":"2024-04-04T06:10:46.479253+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:53:08.633127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.478487+0000","last_change":"2024-04-04T06:10:45.387129+0000","last_active":"2024-04-04T06:10:46.478487+0000","last_peered":"2024-04-04T06:10:46.478487+0000","last_clean":"2024-04-04T06:10:46.478487+0000","last_became_active":"2024-04-04T06:10:45.385346+0000","last_became_peered":"2024-04-04T06:10:45.385346+0000","last_unstale":"2024-04-04T06:10:46.478487+0000","last_undegraded":"2024-04-04T06:10:46.478487+0000","last_fullsized":"2024-04-04T06:10:46.478487+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:35:26.528398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"70'3","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476556+0000","last_change":"2024-04-04T06:10:44.359306+0000","last_active":"2024-04-04T06:10:46.476556+0000","last_peered":"2024-04-04T06:10:46.476556+0000","last_clean":"2024-04-04T06:10:46.476556+0000","last_became_active":"2024-04-04T06:10:44.358978+0000","last_became_peered":"2024-04-04T06:10:44.358978+0000","last_unstale":"2024-04-04T06:10:46.476556+0000","last_undegraded":"2024-04-04T06:10:46.476556+0000","last_fullsized":"2024-04-04T06:10:46.476556+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:55:42.949268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.473976+0000","last_change":"2024-04-04T06:10:45.373368+0000","last_active":"2024-04-04T06:10:46.473976+0000","last_peered":"2024-04-04T06:10:46.473976+0000","last_clean":"2024-04-04T06:10:46.473976+0000","last_became_active":"2024-04-04T06:10:45.373013+0000","last_became_peered":"2024-04-04T06:10:45.373013+0000","last_unstale":"2024-04-04T06:10:46.473976+0000","last_undegraded":"2024-04-04T06:10:46.473976+0000","last_fullsized":"2024-04-04T06:10:46.473976+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:11:16.794252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"70'6","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.377224+0000","last_change":"2024-04-04T06:10:44.353379+0000","last_active":"2024-04-04T06:10:46.377224+0000","last_peered":"2024-04-04T06:10:46.377224+0000","last_clean":"2024-04-04T06:10:46.377224+0000","last_became_active":"2024-04-04T06:10:44.351976+0000","last_became_peered":"2024-04-04T06:10:44.351976+0000","last_unstale":"2024-04-04T06:10:46.377224+0000","last_undegraded":"2024-04-04T06:10:46.377224+0000","last_fullsized":"2024-04-04T06:10:46.377224+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:37:31.341831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477933+0000","last_change":"2024-04-04T06:10:45.396962+0000","last_active":"2024-04-04T06:10:46.477933+0000","last_peered":"2024-04-04T06:10:46.477933+0000","last_clean":"2024-04-04T06:10:46.477933+0000","last_became_active":"2024-04-04T06:10:45.394768+0000","last_became_peered":"2024-04-04T06:10:45.394768+0000","last_unstale":"2024-04-04T06:10:46.477933+0000","last_undegraded":"2024-04-04T06:10:46.477933+0000","last_fullsized":"2024-04-04T06:10:46.477933+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:55:33.843420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.377331+0000","last_change":"2024-04-04T06:10:44.360227+0000","last_active":"2024-04-04T06:10:46.377331+0000","last_peered":"2024-04-04T06:10:46.377331+0000","last_clean":"2024-04-04T06:10:46.377331+0000","last_became_active":"2024-04-04T06:10:44.359905+0000","last_became_peered":"2024-04-04T06:10:44.359905+0000","last_unstale":"2024-04-04T06:10:46.377331+0000","last_undegraded":"2024-04-04T06:10:46.377331+0000","last_fullsized":"2024-04-04T06:10:46.377331+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:07:08.850656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.394612+0000","last_change":"2024-04-04T06:10:45.383894+0000","last_active":"2024-04-04T06:10:45.394612+0000","last_peered":"2024-04-04T06:10:45.394612+0000","last_clean":"2024-04-04T06:10:45.394612+0000","last_became_active":"2024-04-04T06:10:45.383573+0000","last_became_peered":"2024-04-04T06:10:45.383573+0000","last_unstale":"2024-04-04T06:10:45.394612+0000","last_undegraded":"2024-04-04T06:10:45.394612+0000","last_fullsized":"2024-04-04T06:10:45.394612+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:22:04.238317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477951+0000","last_change":"2024-04-04T06:10:44.357439+0000","last_active":"2024-04-04T06:10:46.477951+0000","last_peered":"2024-04-04T06:10:46.477951+0000","last_clean":"2024-04-04T06:10:46.477951+0000","last_became_active":"2024-04-04T06:10:44.357159+0000","last_became_peered":"2024-04-04T06:10:44.357159+0000","last_unstale":"2024-04-04T06:10:46.477951+0000","last_undegraded":"2024-04-04T06:10:46.477951+0000","last_fullsized":"2024-04-04T06:10:46.477951+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:30:29.678930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.473117+0000","last_change":"2024-04-04T06:10:45.377697+0000","last_active":"2024-04-04T06:10:46.473117+0000","last_peered":"2024-04-04T06:10:46.473117+0000","last_clean":"2024-04-04T06:10:46.473117+0000","last_became_active":"2024-04-04T06:10:45.377297+0000","last_became_peered":"2024-04-04T06:10:45.377297+0000","last_unstale":"2024-04-04T06:10:46.473117+0000","last_undegraded":"2024-04-04T06:10:46.473117+0000","last_fullsized":"2024-04-04T06:10:46.473117+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:59:12.100267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.360174+0000","last_change":"2024-04-04T06:10:44.359666+0000","last_active":"2024-04-04T06:10:45.360174+0000","last_peered":"2024-04-04T06:10:45.360174+0000","last_clean":"2024-04-04T06:10:45.360174+0000","last_became_active":"2024-04-04T06:10:44.359382+0000","last_became_peered":"2024-04-04T06:10:44.359382+0000","last_unstale":"2024-04-04T06:10:45.360174+0000","last_undegraded":"2024-04-04T06:10:45.360174+0000","last_fullsized":"2024-04-04T06:10:45.360174+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:46:45.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.473312+0000","last_change":"2024-04-04T06:10:45.375623+0000","last_active":"2024-04-04T06:10:46.473312+0000","last_peered":"2024-04-04T06:10:46.473312+0000","last_clean":"2024-04-04T06:10:46.473312+0000","last_became_active":"2024-04-04T06:10:45.375319+0000","last_became_peered":"2024-04-04T06:10:45.375319+0000","last_unstale":"2024-04-04T06:10:46.473312+0000","last_undegraded":"2024-04-04T06:10:46.473312+0000","last_fullsized":"2024-04-04T06:10:46.473312+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:33:39.468998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"70'8","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.397256+0000","last_change":"2024-04-04T06:10:44.362714+0000","last_active":"2024-04-04T06:10:45.397256+0000","last_peered":"2024-04-04T06:10:45.397256+0000","last_clean":"2024-04-04T06:10:45.397256+0000","last_became_active":"2024-04-04T06:10:44.362399+0000","last_became_peered":"2024-04-04T06:10:44.362399+0000","last_unstale":"2024-04-04T06:10:45.397256+0000","last_undegraded":"2024-04-04T06:10:45.397256+0000","last_fullsized":"2024-04-04T06:10:45.397256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:05:03.011322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.389486+0000","last_change":"2024-04-04T06:10:45.381395+0000","last_active":"2024-04-04T06:10:45.389486+0000","last_peered":"2024-04-04T06:10:45.389486+0000","last_clean":"2024-04-04T06:10:45.389486+0000","last_became_active":"2024-04-04T06:10:45.380870+0000","last_became_peered":"2024-04-04T06:10:45.380870+0000","last_unstale":"2024-04-04T06:10:45.389486+0000","last_undegraded":"2024-04-04T06:10:45.389486+0000","last_fullsized":"2024-04-04T06:10:45.389486+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:35:54.626600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.479010+0000","last_change":"2024-04-04T06:10:44.359654+0000","last_active":"2024-04-04T06:10:46.479010+0000","last_peered":"2024-04-04T06:10:46.479010+0000","last_clean":"2024-04-04T06:10:46.479010+0000","last_became_active":"2024-04-04T06:10:44.359356+0000","last_became_peered":"2024-04-04T06:10:44.359356+0000","last_unstale":"2024-04-04T06:10:46.479010+0000","last_undegraded":"2024-04-04T06:10:46.479010+0000","last_fullsized":"2024-04-04T06:10:46.479010+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:33:55.568263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477611+0000","last_change":"2024-04-04T06:10:45.394559+0000","last_active":"2024-04-04T06:10:46.477611+0000","last_peered":"2024-04-04T06:10:46.477611+0000","last_clean":"2024-04-04T06:10:46.477611+0000","last_became_active":"2024-04-04T06:10:45.392959+0000","last_became_peered":"2024-04-04T06:10:45.392959+0000","last_unstale":"2024-04-04T06:10:46.477611+0000","last_undegraded":"2024-04-04T06:10:46.477611+0000","last_fullsized":"2024-04-04T06:10:46.477611+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:01:10.310581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.392832+0000","last_change":"2024-04-04T06:10:44.366245+0000","last_active":"2024-04-04T06:10:45.392832+0000","last_peered":"2024-04-04T06:10:45.392832+0000","last_clean":"2024-04-04T06:10:45.392832+0000","last_became_active":"2024-04-04T06:10:44.365966+0000","last_became_peered":"2024-04-04T06:10:44.365966+0000","last_unstale":"2024-04-04T06:10:45.392832+0000","last_undegraded":"2024-04-04T06:10:45.392832+0000","last_fullsized":"2024-04-04T06:10:45.392832+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:52:28.198053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.377137+0000","last_change":"2024-04-04T06:10:45.369750+0000","last_active":"2024-04-04T06:10:46.377137+0000","last_peered":"2024-04-04T06:10:46.377137+0000","last_clean":"2024-04-04T06:10:46.377137+0000","last_became_active":"2024-04-04T06:10:45.369477+0000","last_became_peered":"2024-04-04T06:10:45.369477+0000","last_unstale":"2024-04-04T06:10:46.377137+0000","last_undegraded":"2024-04-04T06:10:46.377137+0000","last_fullsized":"2024-04-04T06:10:46.377137+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:08:51.975530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.478708+0000","last_change":"2024-04-04T06:10:44.363184+0000","last_active":"2024-04-04T06:10:46.478708+0000","last_peered":"2024-04-04T06:10:46.478708+0000","last_clean":"2024-04-04T06:10:46.478708+0000","last_became_active":"2024-04-04T06:10:44.361823+0000","last_became_peered":"2024-04-04T06:10:44.361823+0000","last_unstale":"2024-04-04T06:10:46.478708+0000","last_undegraded":"2024-04-04T06:10:46.478708+0000","last_fullsized":"2024-04-04T06:10:46.478708+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:18:07.093490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474115+0000","last_change":"2024-04-04T06:10:45.382954+0000","last_active":"2024-04-04T06:10:46.474115+0000","last_peered":"2024-04-04T06:10:46.474115+0000","last_clean":"2024-04-04T06:10:46.474115+0000","last_became_active":"2024-04-04T06:10:45.382680+0000","last_became_peered":"2024-04-04T06:10:45.382680+0000","last_unstale":"2024-04-04T06:10:46.474115+0000","last_undegraded":"2024-04-04T06:10:46.474115+0000","last_fullsized":"2024-04-04T06:10:46.474115+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:07:09.863695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476828+0000","last_change":"2024-04-04T06:10:44.367018+0000","last_active":"2024-04-04T06:10:46.476828+0000","last_peered":"2024-04-04T06:10:46.476828+0000","last_clean":"2024-04-04T06:10:46.476828+0000","last_became_active":"2024-04-04T06:10:44.366681+0000","last_became_peered":"2024-04-04T06:10:44.366681+0000","last_unstale":"2024-04-04T06:10:46.476828+0000","last_undegraded":"2024-04-04T06:10:46.476828+0000","last_fullsized":"2024-04-04T06:10:46.476828+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:32:55.018331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476224+0000","last_change":"2024-04-04T06:10:45.381316+0000","last_active":"2024-04-04T06:10:46.476224+0000","last_peered":"2024-04-04T06:10:46.476224+0000","last_clean":"2024-04-04T06:10:46.476224+0000","last_became_active":"2024-04-04T06:10:45.379688+0000","last_became_peered":"2024-04-04T06:10:45.379688+0000","last_unstale":"2024-04-04T06:10:46.476224+0000","last_undegraded":"2024-04-04T06:10:46.476224+0000","last_fullsized":"2024-04-04T06:10:46.476224+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:50:02.015578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.478200+0000","last_change":"2024-04-04T06:10:44.353285+0000","last_active":"2024-04-04T06:10:46.478200+0000","last_peered":"2024-04-04T06:10:46.478200+0000","last_clean":"2024-04-04T06:10:46.478200+0000","last_became_active":"2024-04-04T06:10:44.352800+0000","last_became_peered":"2024-04-04T06:10:44.352800+0000","last_unstale":"2024-04-04T06:10:46.478200+0000","last_undegraded":"2024-04-04T06:10:46.478200+0000","last_fullsized":"2024-04-04T06:10:46.478200+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:56:38.203099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.475971+0000","last_change":"2024-04-04T06:10:45.386831+0000","last_active":"2024-04-04T06:10:46.475971+0000","last_peered":"2024-04-04T06:10:46.475971+0000","last_clean":"2024-04-04T06:10:46.475971+0000","last_became_active":"2024-04-04T06:10:45.386489+0000","last_became_peered":"2024-04-04T06:10:45.386489+0000","last_unstale":"2024-04-04T06:10:46.475971+0000","last_undegraded":"2024-04-04T06:10:46.475971+0000","last_fullsized":"2024-04-04T06:10:46.475971+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:36:08.592530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.357381+0000","last_change":"2024-04-04T06:10:44.358192+0000","last_active":"2024-04-04T06:10:45.357381+0000","last_peered":"2024-04-04T06:10:45.357381+0000","last_clean":"2024-04-04T06:10:45.357381+0000","last_became_active":"2024-04-04T06:10:44.357231+0000","last_became_peered":"2024-04-04T06:10:44.357231+0000","last_unstale":"2024-04-04T06:10:45.357381+0000","last_undegraded":"2024-04-04T06:10:45.357381+0000","last_fullsized":"2024-04-04T06:10:45.357381+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:28:12.750849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476077+0000","last_change":"2024-04-04T06:10:45.383693+0000","last_active":"2024-04-04T06:10:46.476077+0000","last_peered":"2024-04-04T06:10:46.476077+0000","last_clean":"2024-04-04T06:10:46.476077+0000","last_became_active":"2024-04-04T06:10:45.380079+0000","last_became_peered":"2024-04-04T06:10:45.380079+0000","last_unstale":"2024-04-04T06:10:46.476077+0000","last_undegraded":"2024-04-04T06:10:46.476077+0000","last_fullsized":"2024-04-04T06:10:46.476077+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:28:26.165425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.472966+0000","last_change":"2024-04-04T06:10:44.357800+0000","last_active":"2024-04-04T06:10:46.472966+0000","last_peered":"2024-04-04T06:10:46.472966+0000","last_clean":"2024-04-04T06:10:46.472966+0000","last_became_active":"2024-04-04T06:10:44.357015+0000","last_became_peered":"2024-04-04T06:10:44.357015+0000","last_unstale":"2024-04-04T06:10:46.472966+0000","last_undegraded":"2024-04-04T06:10:46.472966+0000","last_fullsized":"2024-04-04T06:10:46.472966+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:35:53.613561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474549+0000","last_change":"2024-04-04T06:10:45.369229+0000","last_active":"2024-04-04T06:10:46.474549+0000","last_peered":"2024-04-04T06:10:46.474549+0000","last_clean":"2024-04-04T06:10:46.474549+0000","last_became_active":"2024-04-04T06:10:45.368952+0000","last_became_peered":"2024-04-04T06:10:45.368952+0000","last_unstale":"2024-04-04T06:10:46.474549+0000","last_undegraded":"2024-04-04T06:10:46.474549+0000","last_fullsized":"2024-04-04T06:10:46.474549+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:50.702749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.479140+0000","last_change":"2024-04-04T06:10:44.360294+0000","last_active":"2024-04-04T06:10:46.479140+0000","last_peered":"2024-04-04T06:10:46.479140+0000","last_clean":"2024-04-04T06:10:46.479140+0000","last_became_active":"2024-04-04T06:10:44.359973+0000","last_became_peered":"2024-04-04T06:10:44.359973+0000","last_unstale":"2024-04-04T06:10:46.479140+0000","last_undegraded":"2024-04-04T06:10:46.479140+0000","last_fullsized":"2024-04-04T06:10:46.479140+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:40:18.010754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.393723+0000","last_change":"2024-04-04T06:10:45.380011+0000","last_active":"2024-04-04T06:10:45.393723+0000","last_peered":"2024-04-04T06:10:45.393723+0000","last_clean":"2024-04-04T06:10:45.393723+0000","last_became_active":"2024-04-04T06:10:45.379736+0000","last_became_peered":"2024-04-04T06:10:45.379736+0000","last_unstale":"2024-04-04T06:10:45.393723+0000","last_undegraded":"2024-04-04T06:10:45.393723+0000","last_fullsized":"2024-04-04T06:10:45.393723+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:35:54.626600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.360016+0000","last_change":"2024-04-04T06:10:44.354715+0000","last_active":"2024-04-04T06:10:45.360016+0000","last_peered":"2024-04-04T06:10:45.360016+0000","last_clean":"2024-04-04T06:10:45.360016+0000","last_became_active":"2024-04-04T06:10:44.354396+0000","last_became_peered":"2024-04-04T06:10:44.354396+0000","last_unstale":"2024-04-04T06:10:45.360016+0000","last_undegraded":"2024-04-04T06:10:45.360016+0000","last_fullsized":"2024-04-04T06:10:45.360016+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:58:35.602169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476587+0000","last_change":"2024-04-04T06:10:45.398370+0000","last_active":"2024-04-04T06:10:46.476587+0000","last_peered":"2024-04-04T06:10:46.476587+0000","last_clean":"2024-04-04T06:10:46.476587+0000","last_became_active":"2024-04-04T06:10:45.398085+0000","last_became_peered":"2024-04-04T06:10:45.398085+0000","last_unstale":"2024-04-04T06:10:46.476587+0000","last_undegraded":"2024-04-04T06:10:46.476587+0000","last_fullsized":"2024-04-04T06:10:46.476587+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:31:00.494486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474216+0000","last_change":"2024-04-04T06:10:44.352577+0000","last_active":"2024-04-04T06:10:46.474216+0000","last_peered":"2024-04-04T06:10:46.474216+0000","last_clean":"2024-04-04T06:10:46.474216+0000","last_became_active":"2024-04-04T06:10:44.352298+0000","last_became_peered":"2024-04-04T06:10:44.352298+0000","last_unstale":"2024-04-04T06:10:46.474216+0000","last_undegraded":"2024-04-04T06:10:46.474216+0000","last_fullsized":"2024-04-04T06:10:46.474216+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:55:04.000965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.478098+0000","last_change":"2024-04-04T06:10:45.401164+0000","last_active":"2024-04-04T06:10:46.478098+0000","last_peered":"2024-04-04T06:10:46.478098+0000","last_clean":"2024-04-04T06:10:46.478098+0000","last_became_active":"2024-04-04T06:10:45.400515+0000","last_became_peered":"2024-04-04T06:10:45.400515+0000","last_unstale":"2024-04-04T06:10:46.478098+0000","last_undegraded":"2024-04-04T06:10:46.478098+0000","last_fullsized":"2024-04-04T06:10:46.478098+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:15:01.516699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474224+0000","last_change":"2024-04-04T06:10:44.362280+0000","last_active":"2024-04-04T06:10:46.474224+0000","last_peered":"2024-04-04T06:10:46.474224+0000","last_clean":"2024-04-04T06:10:46.474224+0000","last_became_active":"2024-04-04T06:10:44.361114+0000","last_became_peered":"2024-04-04T06:10:44.361114+0000","last_unstale":"2024-04-04T06:10:46.474224+0000","last_undegraded":"2024-04-04T06:10:46.474224+0000","last_fullsized":"2024-04-04T06:10:46.474224+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:51:24.020161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.473820+0000","last_change":"2024-04-04T06:10:45.374897+0000","last_active":"2024-04-04T06:10:46.473820+0000","last_peered":"2024-04-04T06:10:46.473820+0000","last_clean":"2024-04-04T06:10:46.473820+0000","last_became_active":"2024-04-04T06:10:45.374594+0000","last_became_peered":"2024-04-04T06:10:45.374594+0000","last_unstale":"2024-04-04T06:10:46.473820+0000","last_undegraded":"2024-04-04T06:10:46.473820+0000","last_fullsized":"2024-04-04T06:10:46.473820+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:49:43.283191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.474469+0000","last_change":"2024-04-04T06:10:44.360110+0000","last_active":"2024-04-04T06:10:46.474469+0000","last_peered":"2024-04-04T06:10:46.474469+0000","last_clean":"2024-04-04T06:10:46.474469+0000","last_became_active":"2024-04-04T06:10:44.359637+0000","last_became_peered":"2024-04-04T06:10:44.359637+0000","last_unstale":"2024-04-04T06:10:46.474469+0000","last_undegraded":"2024-04-04T06:10:46.474469+0000","last_fullsized":"2024-04-04T06:10:46.474469+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:11:18.559137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476930+0000","last_change":"2024-04-04T06:10:45.388218+0000","last_active":"2024-04-04T06:10:46.476930+0000","last_peered":"2024-04-04T06:10:46.476930+0000","last_clean":"2024-04-04T06:10:46.476930+0000","last_became_active":"2024-04-04T06:10:45.387941+0000","last_became_peered":"2024-04-04T06:10:45.387941+0000","last_unstale":"2024-04-04T06:10:46.476930+0000","last_undegraded":"2024-04-04T06:10:46.476930+0000","last_fullsized":"2024-04-04T06:10:46.476930+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:54:51.018743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.376656+0000","last_change":"2024-04-04T06:10:44.358955+0000","last_active":"2024-04-04T06:10:46.376656+0000","last_peered":"2024-04-04T06:10:46.376656+0000","last_clean":"2024-04-04T06:10:46.376656+0000","last_became_active":"2024-04-04T06:10:44.358666+0000","last_became_peered":"2024-04-04T06:10:44.358666+0000","last_unstale":"2024-04-04T06:10:46.376656+0000","last_undegraded":"2024-04-04T06:10:46.376656+0000","last_fullsized":"2024-04-04T06:10:46.376656+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:32:38.730974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.376741+0000","last_change":"2024-04-04T06:10:45.376805+0000","last_active":"2024-04-04T06:10:46.376741+0000","last_peered":"2024-04-04T06:10:46.376741+0000","last_clean":"2024-04-04T06:10:46.376741+0000","last_became_active":"2024-04-04T06:10:45.376407+0000","last_became_peered":"2024-04-04T06:10:45.376407+0000","last_unstale":"2024-04-04T06:10:46.376741+0000","last_undegraded":"2024-04-04T06:10:46.376741+0000","last_fullsized":"2024-04-04T06:10:46.376741+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:30:49.308665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-04T06:10:45.357626+0000","last_change":"2024-04-04T06:10:44.362621+0000","last_active":"2024-04-04T06:10:45.357626+0000","last_peered":"2024-04-04T06:10:45.357626+0000","last_clean":"2024-04-04T06:10:45.357626+0000","last_became_active":"2024-04-04T06:10:44.362342+0000","last_became_peered":"2024-04-04T06:10:44.362342+0000","last_unstale":"2024-04-04T06:10:45.357626+0000","last_undegraded":"2024-04-04T06:10:45.357626+0000","last_fullsized":"2024-04-04T06:10:45.357626+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:06:34.175687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476892+0000","last_change":"2024-04-04T06:10:45.381890+0000","last_active":"2024-04-04T06:10:46.476892+0000","last_peered":"2024-04-04T06:10:46.476892+0000","last_clean":"2024-04-04T06:10:46.476892+0000","last_became_active":"2024-04-04T06:10:45.381578+0000","last_became_peered":"2024-04-04T06:10:45.381578+0000","last_unstale":"2024-04-04T06:10:46.476892+0000","last_undegraded":"2024-04-04T06:10:46.476892+0000","last_fullsized":"2024-04-04T06:10:46.476892+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:05:07.220786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.477445+0000","last_change":"2024-04-04T06:10:44.357737+0000","last_active":"2024-04-04T06:10:46.477445+0000","last_peered":"2024-04-04T06:10:46.477445+0000","last_clean":"2024-04-04T06:10:46.477445+0000","last_became_active":"2024-04-04T06:10:44.357480+0000","last_became_peered":"2024-04-04T06:10:44.357480+0000","last_unstale":"2024-04-04T06:10:46.477445+0000","last_undegraded":"2024-04-04T06:10:46.477445+0000","last_fullsized":"2024-04-04T06:10:46.477445+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:00:44.279660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476183+0000","last_change":"2024-04-04T06:10:45.393261+0000","last_active":"2024-04-04T06:10:46.476183+0000","last_peered":"2024-04-04T06:10:46.476183+0000","last_clean":"2024-04-04T06:10:46.476183+0000","last_became_active":"2024-04-04T06:10:45.392886+0000","last_became_peered":"2024-04-04T06:10:45.392886+0000","last_unstale":"2024-04-04T06:10:46.476183+0000","last_undegraded":"2024-04-04T06:10:46.476183+0000","last_fullsized":"2024-04-04T06:10:46.476183+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:44.320071+0000","last_clean_scrub_stamp":"2024-04-04T06:10:44.320071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:09:18.404824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476836+0000","last_change":"2024-04-04T06:10:44.360453+0000","last_active":"2024-04-04T06:10:46.476836+0000","last_peered":"2024-04-04T06:10:46.476836+0000","last_clean":"2024-04-04T06:10:46.476836+0000","last_became_active":"2024-04-04T06:10:44.360139+0000","last_became_peered":"2024-04-04T06:10:44.360139+0000","last_unstale":"2024-04-04T06:10:46.476836+0000","last_undegraded":"2024-04-04T06:10:46.476836+0000","last_fullsized":"2024-04-04T06:10:46.476836+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:10:43.307032+0000","last_clean_scrub_stamp":"2024-04-04T06:10:43.307032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:46:27.776188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":260,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-04T06:10:46.476256+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:10:46.476256+0000","last_peered":"2024-04-04T06:10:46.476256+0000","last_clean":"2024-04-04T06:10:46.476256+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:10:46.476256+0000","last_undegraded":"2024-04-04T06:10:46.476256+0000","last_fullsized":"2024-04-04T06:10:46.476256+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509503,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1458176,"data_stored":951845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509505,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":864256,"data_stored":360953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":851968,"data_stored":354673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509505,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":362473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509504,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":864256,"data_stored":356789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509506,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1437696,"data_stored":944929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542209,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":847872,"data_stored":354651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542207,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":831488,"data_stored":347786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:10:50.332 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-04T06:10:50.332 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-04T06:10:50.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.537+0000 7ff17c800640 1 Processor -- start 2024-04-04T06:10:50.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.537+0000 7ff17c800640 1 -- start start 2024-04-04T06:10:50.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.539+0000 7ff17c800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff17415a870 0x7ff1740d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:50.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.539+0000 7ff17c800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff17415ac50 0x7ff17415e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:50.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.539+0000 7ff17c800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1740b4c00 con 0x7ff17415ead0 2024-04-04T06:10:50.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.539+0000 7ff17c800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7ff17415ef40 con 0x7ff17415ac50 2024-04-04T06:10:50.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.539+0000 7ff17c800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff17406d5b0 con 0x7ff17415a870 2024-04-04T06:10:50.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.539+0000 7ff179400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff17415ac50 0x7ff17415e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:50.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.539+0000 7ff179400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff17415ac50 0x7ff17415e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:32778/0 (socket says 172.21.15.159:32778) 2024-04-04T06:10:50.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.539+0000 7ff179400640 1 -- 172.21.15.159:0/2365865790 learned_addr learned my addr 172.21.15.159:0/2365865790 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:50.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.539+0000 7ff179e00640 1 --2- 172.21.15.159:0/2365865790 >> v2:172.21.15.178:3300/0 conn(0x7ff17415a870 0x7ff1740d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:50.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.540+0000 7ff179400640 1 -- 172.21.15.159:0/2365865790 >> v2:172.21.15.178:3300/0 conn(0x7ff17415a870 msgr2=0x7ff1740d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:50.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.540+0000 7ff179400640 1 --2- 172.21.15.159:0/2365865790 >> v2:172.21.15.178:3300/0 conn(0x7ff17415a870 0x7ff1740d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:50.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.540+0000 7ff179400640 1 -- 172.21.15.159:0/2365865790 >> v1:172.21.15.178:6789/0 conn(0x7ff17415ead0 legacy=0x7ff1740b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:50.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.540+0000 7ff179400640 1 -- 172.21.15.159:0/2365865790 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1740b4ee0 con 0x7ff17415ac50 2024-04-04T06:10:50.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.541+0000 7ff179400640 1 --2- 172.21.15.159:0/2365865790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff17415ac50 0x7ff17415e590 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff174073ac0 tx=0x7ff16402db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27b3406a33f21a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:50.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.541+0000 7ff178a00640 1 -- 172.21.15.159:0/2365865790 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16402f8e0 con 0x7ff17415ac50 2024-04-04T06:10:50.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.541+0000 7ff178a00640 1 -- 172.21.15.159:0/2365865790 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff16402fa80 con 0x7ff17415ac50 2024-04-04T06:10:50.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.541+0000 7ff178a00640 1 -- 172.21.15.159:0/2365865790 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16402fe50 con 0x7ff17415ac50 2024-04-04T06:10:50.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.542+0000 7ff17c800640 1 -- 172.21.15.159:0/2365865790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff17415ac50 msgr2=0x7ff17415e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:50.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.542+0000 7ff17c800640 1 --2- 172.21.15.159:0/2365865790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff17415ac50 0x7ff17415e590 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff174073ac0 tx=0x7ff16402db20 comp rx=0 tx=0).stop 2024-04-04T06:10:50.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.542+0000 7ff17c800640 1 -- 172.21.15.159:0/2365865790 shutdown_connections 2024-04-04T06:10:50.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.542+0000 7ff17c800640 1 --2- 172.21.15.159:0/2365865790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff17415ac50 0x7ff17415e590 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:50.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.542+0000 7ff17c800640 1 --2- 172.21.15.159:0/2365865790 >> v2:172.21.15.178:3300/0 conn(0x7ff17415a870 0x7ff1740d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:50.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.542+0000 7ff17c800640 1 -- 172.21.15.159:0/2365865790 >> 172.21.15.159:0/2365865790 conn(0x7ff17406b750 msgr2=0x7ff17406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:50.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.542+0000 7ff17c800640 1 -- 172.21.15.159:0/2365865790 shutdown_connections 2024-04-04T06:10:50.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.542+0000 7ff17c800640 1 -- 172.21.15.159:0/2365865790 wait complete. 2024-04-04T06:10:50.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.544+0000 7ff17c800640 1 Processor -- start 2024-04-04T06:10:50.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.544+0000 7ff17c800640 1 -- start start 2024-04-04T06:10:50.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.545+0000 7ff17c800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff17415a870 0x7ff17415bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:50.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.545+0000 7ff17c800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff17415ac50 0x7ff17415c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:50.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.545+0000 7ff17c800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff17415ead0 0x7ff1741f7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:50.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.545+0000 7ff17c800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff1740d07c0 con 0x7ff17415a870 2024-04-04T06:10:50.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.545+0000 7ff17c800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff17406db70 con 0x7ff17415ac50 2024-04-04T06:10:50.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.545+0000 7ff17c800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff1740d0940 con 0x7ff17415ead0 2024-04-04T06:10:50.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.545+0000 7ff179e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff17415a870 0x7ff17415bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:50.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.545+0000 7ff179e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff17415a870 0x7ff17415bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:32780/0 (socket says 172.21.15.159:32780) 2024-04-04T06:10:50.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.545+0000 7ff179e00640 1 -- 172.21.15.159:0/406039747 learned_addr learned my addr 172.21.15.159:0/406039747 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:50.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.545+0000 7ff17a800640 1 --2- 172.21.15.159:0/406039747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff17415ead0 0x7ff1741f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:50.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.545+0000 7ff179400640 1 --2- 172.21.15.159:0/406039747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff17415ac50 0x7ff17415c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:50.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.546+0000 7ff179e00640 1 -- 172.21.15.159:0/406039747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff17415ead0 msgr2=0x7ff1741f7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:50.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.546+0000 7ff179e00640 1 --2- 172.21.15.159:0/406039747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff17415ead0 0x7ff1741f7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:50.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.546+0000 7ff179e00640 1 -- 172.21.15.159:0/406039747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff17415ac50 msgr2=0x7ff17415c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:50.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.546+0000 7ff179e00640 1 --2- 172.21.15.159:0/406039747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff17415ac50 0x7ff17415c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:50.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.546+0000 7ff179e00640 1 -- 172.21.15.159:0/406039747 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17415ced0 con 0x7ff17415a870 2024-04-04T06:10:50.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.546+0000 7ff179400640 1 --2- 172.21.15.159:0/406039747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff17415ac50 0x7ff17415c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:50.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.546+0000 7ff179e00640 1 --2- 172.21.15.159:0/406039747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff17415a870 0x7ff17415bc60 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff16800d800 tx=0x7ff16800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:50.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.546+0000 7ff16e000640 1 -- 172.21.15.159:0/406039747 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16800b6c0 con 0x7ff17415a870 2024-04-04T06:10:50.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.546+0000 7ff16e000640 1 -- 172.21.15.159:0/406039747 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff168019070 con 0x7ff17415a870 2024-04-04T06:10:50.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.546+0000 7ff16e000640 1 -- 172.21.15.159:0/406039747 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16800ba80 con 0x7ff17415a870 2024-04-04T06:10:50.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.546+0000 7ff17c800640 1 -- 172.21.15.159:0/406039747 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff174157db0 con 0x7ff17415a870 2024-04-04T06:10:50.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.546+0000 7ff17c800640 1 -- 172.21.15.159:0/406039747 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff174158370 con 0x7ff17415a870 2024-04-04T06:10:50.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.548+0000 7ff17c800640 1 -- 172.21.15.159:0/406039747 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1740685c0 con 0x7ff17415a870 2024-04-04T06:10:50.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.548+0000 7ff16e000640 1 -- 172.21.15.159:0/406039747 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff16800bc20 con 0x7ff17415a870 2024-04-04T06:10:50.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.549+0000 7ff16e000640 1 --2- 172.21.15.159:0/406039747 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff15806c8f0 0x7ff15806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:50.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.550+0000 7ff16e000640 1 -- 172.21.15.159:0/406039747 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7ff16808c0e0 con 0x7ff17415a870 2024-04-04T06:10:50.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.550+0000 7ff179400640 1 --2- 172.21.15.159:0/406039747 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff15806c8f0 0x7ff15806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:50.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.551+0000 7ff179400640 1 --2- 172.21.15.159:0/406039747 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff15806c8f0 0x7ff15806edb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff164038000 tx=0x7ff164008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:50.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.554+0000 7ff16e000640 1 -- 172.21.15.159:0/406039747 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff16801c360 con 0x7ff17415a870 2024-04-04T06:10:50.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.823+0000 7ff17c800640 1 -- 172.21.15.159:0/406039747 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff1740721e0 con 0x7ff17415a870 2024-04-04T06:10:50.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.828+0000 7ff16e000640 1 -- 172.21.15.159:0/406039747 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7ff16805a470 con 0x7ff17415a870 2024-04-04T06:10:50.821 INFO:teuthology.orchestra.run.smithi159.stdout:[client.0] 2024-04-04T06:10:50.821 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQD9Qg5mqNKJGxAA4jP9cadhRIyhQ3lssttWIw== 2024-04-04T06:10:50.821 INFO:teuthology.orchestra.run.smithi159.stdout: caps mds = "allow" 2024-04-04T06:10:50.821 INFO:teuthology.orchestra.run.smithi159.stdout: caps mon = "allow r" 2024-04-04T06:10:50.821 INFO:teuthology.orchestra.run.smithi159.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-04T06:10:50.821 INFO:teuthology.orchestra.run.smithi159.stderr:updated caps for client.0 2024-04-04T06:10:50.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.830+0000 7ff17c800640 1 -- 172.21.15.159:0/406039747 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff15806c8f0 msgr2=0x7ff15806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:50.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.830+0000 7ff17c800640 1 --2- 172.21.15.159:0/406039747 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff15806c8f0 0x7ff15806edb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff164038000 tx=0x7ff164008030 comp rx=0 tx=0).stop 2024-04-04T06:10:50.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.830+0000 7ff17c800640 1 -- 172.21.15.159:0/406039747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff17415a870 msgr2=0x7ff17415bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:50.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.830+0000 7ff17c800640 1 --2- 172.21.15.159:0/406039747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff17415a870 0x7ff17415bc60 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff16800d800 tx=0x7ff16800dcc0 comp rx=0 tx=0).stop 2024-04-04T06:10:50.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.831+0000 7ff17c800640 1 -- 172.21.15.159:0/406039747 shutdown_connections 2024-04-04T06:10:50.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.831+0000 7ff17c800640 1 --2- 172.21.15.159:0/406039747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff17415ead0 0x7ff1741f7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:50.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.831+0000 7ff17c800640 1 --2- 172.21.15.159:0/406039747 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff15806c8f0 0x7ff15806edb0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:50.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.831+0000 7ff17c800640 1 --2- 172.21.15.159:0/406039747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff17415ac50 0x7ff17415c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:50.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.831+0000 7ff17c800640 1 --2- 172.21.15.159:0/406039747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff17415a870 0x7ff17415bc60 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:50.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.831+0000 7ff17c800640 1 -- 172.21.15.159:0/406039747 >> 172.21.15.159:0/406039747 conn(0x7ff17406b750 msgr2=0x7ff1740ad560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:50.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.831+0000 7ff17c800640 1 -- 172.21.15.159:0/406039747 shutdown_connections 2024-04-04T06:10:50.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:50.831+0000 7ff17c800640 1 -- 172.21.15.159:0/406039747 wait complete. 2024-04-04T06:10:50.841 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:10:51.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.048+0000 7efdc4800640 1 Processor -- start 2024-04-04T06:10:51.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.048+0000 7efdc4800640 1 -- start start 2024-04-04T06:10:51.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.049+0000 7efdc4800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdbc15b310 0x7efdbc0727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:51.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.050+0000 7efdc4800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efdbc0b4f90 0x7efdbc15f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:51.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.050+0000 7efdc4800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdbc15b6f0 con 0x7efdbc0d0780 2024-04-04T06:10:51.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.050+0000 7efdc4800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7efdbc072e80 con 0x7efdbc0b4f90 2024-04-04T06:10:51.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.050+0000 7efdc4800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efdbc073160 con 0x7efdbc15b310 2024-04-04T06:10:51.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.050+0000 7efdc2800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7efdbc0d0780 0x7efdbc0adc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:44436/0 (socket says 172.21.15.159:44436) 2024-04-04T06:10:51.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.050+0000 7efdc2800640 1 -- 172.21.15.159:0/2186956344 learned_addr learned my addr 172.21.15.159:0/2186956344 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:51.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.050+0000 7efdc1e00640 1 --2- 172.21.15.159:0/2186956344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdbc15b310 0x7efdbc0727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:51.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.050+0000 7efdc1400640 1 --2- 172.21.15.159:0/2186956344 >> v2:172.21.15.178:3300/0 conn(0x7efdbc0b4f90 0x7efdbc15f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:51.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.050+0000 7efdc0a00640 1 -- 172.21.15.159:0/2186956344 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3967780750 0 0) 0x7efdbc15b6f0 con 0x7efdbc0d0780 2024-04-04T06:10:51.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.051+0000 7efdc0a00640 1 -- 172.21.15.159:0/2186956344 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efda4003520 con 0x7efdbc0d0780 2024-04-04T06:10:51.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.051+0000 7efdc0a00640 1 -- 172.21.15.159:0/2186956344 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3834010070 0 0) 0x7efda4003520 con 0x7efdbc0d0780 2024-04-04T06:10:51.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.051+0000 7efdc0a00640 1 -- 172.21.15.159:0/2186956344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdbc15b310 msgr2=0x7efdbc0727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:51.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.051+0000 7efdc0a00640 1 --2- 172.21.15.159:0/2186956344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdbc15b310 0x7efdbc0727f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.051+0000 7efdc0a00640 1 -- 172.21.15.159:0/2186956344 >> v2:172.21.15.178:3300/0 conn(0x7efdbc0b4f90 msgr2=0x7efdbc15f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:51.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.051+0000 7efdc0a00640 1 --2- 172.21.15.159:0/2186956344 >> v2:172.21.15.178:3300/0 conn(0x7efdbc0b4f90 0x7efdbc15f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.051+0000 7efdc0a00640 1 -- 172.21.15.159:0/2186956344 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdbc15efe0 con 0x7efdbc0d0780 2024-04-04T06:10:51.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.051+0000 7efdc0a00640 1 -- 172.21.15.159:0/2186956344 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7efdb80019f0 con 0x7efdbc0d0780 2024-04-04T06:10:51.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.051+0000 7efdc0a00640 1 -- 172.21.15.159:0/2186956344 >> v1:172.21.15.159:6789/0 conn(0x7efdbc0d0780 legacy=0x7efdbc0adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:51.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.051+0000 7efdc0a00640 1 --2- 172.21.15.159:0/2186956344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efda40039b0 0x7efda4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:51.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.051+0000 7efdc0a00640 1 --2- 172.21.15.159:0/2186956344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efda4004800 0x7efda4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:51.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.051+0000 7efdc0a00640 1 --2- 172.21.15.159:0/2186956344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdbc15b310 0x7efda40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:51.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.051+0000 7efdc0a00640 1 -- 172.21.15.159:0/2186956344 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc072e80 con 0x7efda4004800 2024-04-04T06:10:51.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.051+0000 7efdc0a00640 1 -- 172.21.15.159:0/2186956344 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc073160 con 0x7efda40039b0 2024-04-04T06:10:51.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.051+0000 7efdc0a00640 1 -- 172.21.15.159:0/2186956344 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efda400baa0 con 0x7efdbc15b310 2024-04-04T06:10:51.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.051+0000 7efdc1400640 1 --2- 172.21.15.159:0/2186956344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efda4004800 0x7efda4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:51.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.052+0000 7efdc1400640 1 -- 172.21.15.159:0/2186956344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdbc15b310 msgr2=0x7efda40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:51.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.052+0000 7efdc1400640 1 --2- 172.21.15.159:0/2186956344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdbc15b310 0x7efda40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.052+0000 7efdc1400640 1 -- 172.21.15.159:0/2186956344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efda40039b0 msgr2=0x7efda4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:51.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.052+0000 7efdc1e00640 1 --2- 172.21.15.159:0/2186956344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efda40039b0 0x7efda4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:51.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.052+0000 7efdc1400640 1 --2- 172.21.15.159:0/2186956344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efda40039b0 0x7efda4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.052+0000 7efdc1400640 1 -- 172.21.15.159:0/2186956344 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda4003520 con 0x7efda4004800 2024-04-04T06:10:51.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.052+0000 7efdc1e00640 1 --2- 172.21.15.159:0/2186956344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efda40039b0 0x7efda4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:51.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.053+0000 7efdc1400640 1 --2- 172.21.15.159:0/2186956344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efda4004800 0x7efda4004c80 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7efdac009a90 tx=0x7efda4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eec85cc38d71675 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:51.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.053+0000 7efdc0a00640 1 -- 172.21.15.159:0/2186956344 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac039070 con 0x7efda4004800 2024-04-04T06:10:51.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.053+0000 7efdc0a00640 1 -- 172.21.15.159:0/2186956344 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdac02c6b0 con 0x7efda4004800 2024-04-04T06:10:51.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.053+0000 7efdc0a00640 1 -- 172.21.15.159:0/2186956344 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac009690 con 0x7efda4004800 2024-04-04T06:10:51.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.054+0000 7efdc4800640 1 -- 172.21.15.159:0/2186956344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efda4004800 msgr2=0x7efda4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:51.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.054+0000 7efdc4800640 1 --2- 172.21.15.159:0/2186956344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efda4004800 0x7efda4004c80 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7efdac009a90 tx=0x7efda4005830 comp rx=0 tx=0).stop 2024-04-04T06:10:51.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.054+0000 7efdc2800640 1 -- 172.21.15.159:0/2186956344 reap_dead start 2024-04-04T06:10:51.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.054+0000 7efdc4800640 1 -- 172.21.15.159:0/2186956344 shutdown_connections 2024-04-04T06:10:51.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.054+0000 7efdc4800640 1 -- 172.21.15.159:0/2186956344 >> 172.21.15.159:0/2186956344 conn(0x7efdbc06b990 msgr2=0x7efdbc06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:51.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.054+0000 7efdc4800640 1 -- 172.21.15.159:0/2186956344 shutdown_connections 2024-04-04T06:10:51.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.054+0000 7efdc4800640 1 -- 172.21.15.159:0/2186956344 wait complete. 2024-04-04T06:10:51.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.056+0000 7efdc4800640 1 Processor -- start 2024-04-04T06:10:51.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.056+0000 7efdc4800640 1 -- start start 2024-04-04T06:10:51.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.057+0000 7efdc4800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efda4004800 0x7efdbc149df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:51.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.057+0000 7efdc4800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdbc0990f0 0x7efdbc15bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:51.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.057+0000 7efdc4800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdbc0cf690 0x7efdbc147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:51.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.057+0000 7efdc4800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc073440 con 0x7efda4004800 2024-04-04T06:10:51.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.057+0000 7efdc4800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc0b53f0 con 0x7efdbc0cf690 2024-04-04T06:10:51.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.057+0000 7efdc4800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efdbc15fd20 con 0x7efdbc0990f0 2024-04-04T06:10:51.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.057+0000 7efdc1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efda4004800 0x7efdbc149df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:51.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.057+0000 7efdc1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efda4004800 0x7efdbc149df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:32804/0 (socket says 172.21.15.159:32804) 2024-04-04T06:10:51.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.057+0000 7efdc1e00640 1 -- 172.21.15.159:0/2960389321 learned_addr learned my addr 172.21.15.159:0/2960389321 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:51.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.057+0000 7efdc1400640 1 --2- 172.21.15.159:0/2960389321 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdbc0990f0 0x7efdbc15bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:51.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.057+0000 7efdc1e00640 1 -- 172.21.15.159:0/2960389321 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdbc0990f0 msgr2=0x7efdbc15bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:51.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.057+0000 7efdc1e00640 1 --2- 172.21.15.159:0/2960389321 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdbc0990f0 0x7efdbc15bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.057+0000 7efdc1e00640 1 -- 172.21.15.159:0/2960389321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdbc0cf690 msgr2=0x7efdbc147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:51.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.057+0000 7efdc1e00640 1 --2- 172.21.15.159:0/2960389321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdbc0cf690 0x7efdbc147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.057+0000 7efdc1e00640 1 -- 172.21.15.159:0/2960389321 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdbc14bca0 con 0x7efda4004800 2024-04-04T06:10:51.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.058+0000 7efdc1e00640 1 --2- 172.21.15.159:0/2960389321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efda4004800 0x7efdbc149df0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7efdb00108e0 tx=0x7efdb0010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:51.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.058+0000 7efdb6000640 1 -- 172.21.15.159:0/2960389321 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb000f040 con 0x7efda4004800 2024-04-04T06:10:51.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.058+0000 7efdb6000640 1 -- 172.21.15.159:0/2960389321 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb000ec20 con 0x7efda4004800 2024-04-04T06:10:51.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.058+0000 7efdb6000640 1 -- 172.21.15.159:0/2960389321 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb001c400 con 0x7efda4004800 2024-04-04T06:10:51.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.058+0000 7efdc4800640 1 -- 172.21.15.159:0/2960389321 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdbc146910 con 0x7efda4004800 2024-04-04T06:10:51.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.058+0000 7efdc4800640 1 -- 172.21.15.159:0/2960389321 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdbc14a330 con 0x7efda4004800 2024-04-04T06:10:51.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.059+0000 7efdc4800640 1 -- 172.21.15.159:0/2960389321 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdbc0685c0 con 0x7efda4004800 2024-04-04T06:10:51.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.060+0000 7efdb6000640 1 -- 172.21.15.159:0/2960389321 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efdb001c5a0 con 0x7efda4004800 2024-04-04T06:10:51.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.061+0000 7efdb6000640 1 --2- 172.21.15.159:0/2960389321 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efda006c8f0 0x7efda006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:51.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.061+0000 7efdb6000640 1 -- 172.21.15.159:0/2960389321 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7efdb008c250 con 0x7efda4004800 2024-04-04T06:10:51.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.062+0000 7efdc1400640 1 --2- 172.21.15.159:0/2960389321 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efda006c8f0 0x7efda006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:51.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.063+0000 7efdc1400640 1 --2- 172.21.15.159:0/2960389321 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efda006c8f0 0x7efda006edb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7efdac036000 tx=0x7efdac030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:51.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.066+0000 7efdb6000640 1 -- 172.21.15.159:0/2960389321 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7efdb0018030 con 0x7efda4004800 2024-04-04T06:10:51.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.324+0000 7efdc4800640 1 -- 172.21.15.159:0/2960389321 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdbc15ec20 con 0x7efda4004800 2024-04-04T06:10:51.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.325+0000 7efdb6000640 1 -- 172.21.15.159:0/2960389321 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5192 (secure 0 0 0) 0x7efdb005a670 con 0x7efda4004800 2024-04-04T06:10:51.318 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:51.318 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3789513463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3789513463},{"type":"v1","addr":"172.21.15.178:6835","nonce":3789513463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/468875883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":468875883},{"type":"v1","addr":"172.21.15.178:6837","nonce":468875883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2546974742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2546974742},{"type":"v1","addr":"172.21.15.159:6833","nonce":2546974742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:10:44.805617+0000","modified":"2024-04-04T06:10:47.987695+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6633},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6633":{"gid":6633,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6835/1652468640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1652468640},{"type":"v1","addr":"172.21.15.159:6835","nonce":1652468640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6633,"qdb_cluster":[6633]},"id":5}]} 2024-04-04T06:10:51.319 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 45 2024-04-04T06:10:51.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.328+0000 7efdc4800640 1 -- 172.21.15.159:0/2960389321 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efda006c8f0 msgr2=0x7efda006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:51.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.328+0000 7efdc4800640 1 --2- 172.21.15.159:0/2960389321 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efda006c8f0 0x7efda006edb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7efdac036000 tx=0x7efdac030000 comp rx=0 tx=0).stop 2024-04-04T06:10:51.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.328+0000 7efdc4800640 1 -- 172.21.15.159:0/2960389321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efda4004800 msgr2=0x7efdbc149df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:51.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.328+0000 7efdc4800640 1 --2- 172.21.15.159:0/2960389321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efda4004800 0x7efdbc149df0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7efdb00108e0 tx=0x7efdb0010da0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.328+0000 7efdc4800640 1 -- 172.21.15.159:0/2960389321 shutdown_connections 2024-04-04T06:10:51.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.328+0000 7efdc4800640 1 --2- 172.21.15.159:0/2960389321 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efda006c8f0 0x7efda006edb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.329+0000 7efdc4800640 1 --2- 172.21.15.159:0/2960389321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdbc0cf690 0x7efdbc147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.329+0000 7efdc4800640 1 --2- 172.21.15.159:0/2960389321 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdbc0990f0 0x7efdbc15bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.329+0000 7efdc4800640 1 --2- 172.21.15.159:0/2960389321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efda4004800 0x7efdbc149df0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.329+0000 7efdc4800640 1 -- 172.21.15.159:0/2960389321 >> 172.21.15.159:0/2960389321 conn(0x7efdbc06b990 msgr2=0x7efdbc14c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:51.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.329+0000 7efdc4800640 1 -- 172.21.15.159:0/2960389321 shutdown_connections 2024-04-04T06:10:51.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.329+0000 7efdc4800640 1 -- 172.21.15.159:0/2960389321 wait complete. 2024-04-04T06:10:51.744 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:10:44.805617+0000', 'modified': '2024-04-04T06:10:47.987695+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6633}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6633': {'gid': 6633, 'name': 'a', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.159:6835/1652468640', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 1652468640}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 1652468640}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6633, 'qdb_cluster': [6633]} 2024-04-04T06:10:51.744 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-04T06:10:51.744 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-04T06:10:51.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.967+0000 7f48ab000640 1 Processor -- start 2024-04-04T06:10:51.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.967+0000 7f48ab000640 1 -- start start 2024-04-04T06:10:51.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.968+0000 7f48ab000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48a4155930 0x7f48a40d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:51.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.968+0000 7f48ab000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48a40b40b0 con 0x7f48a40d07c0 2024-04-04T06:10:51.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.969+0000 7f48ab000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48a40b4390 con 0x7f48a4155550 2024-04-04T06:10:51.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.969+0000 7f48ab000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f48a40b4670 con 0x7f48a4155930 2024-04-04T06:10:51.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.969+0000 7f48a9000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f48a40d07c0 0x7f48a40aede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:44440/0 (socket says 172.21.15.159:44440) 2024-04-04T06:10:51.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.969+0000 7f48a9000640 1 -- 172.21.15.159:0/3370266080 learned_addr learned my addr 172.21.15.159:0/3370266080 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:51.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.969+0000 7f48a3400640 1 --2- 172.21.15.159:0/3370266080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48a4155930 0x7f48a40d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:51.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.969+0000 7f48a2a00640 1 -- 172.21.15.159:0/3370266080 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2248032218 0 0) 0x7f48a40b40b0 con 0x7f48a40d07c0 2024-04-04T06:10:51.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.970+0000 7f48a2a00640 1 -- 172.21.15.159:0/3370266080 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4884005730 con 0x7f48a40d07c0 2024-04-04T06:10:51.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.970+0000 7f48a3400640 1 -- 172.21.15.159:0/3370266080 >> v1:172.21.15.178:6789/0 conn(0x7f48a4155550 legacy=0x7f48a415f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:51.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.970+0000 7f48a3400640 1 -- 172.21.15.159:0/3370266080 >> v1:172.21.15.159:6789/0 conn(0x7f48a40d07c0 legacy=0x7f48a40aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:51.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.970+0000 7f48a3400640 1 -- 172.21.15.159:0/3370266080 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a40b4950 con 0x7f48a4155930 2024-04-04T06:10:51.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.971+0000 7f48a3400640 1 --2- 172.21.15.159:0/3370266080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48a4155930 0x7f48a40d0280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f488c0097d0 tx=0x7f488c02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf8935c5b43100f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:51.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.972+0000 7f48a2a00640 1 -- 172.21.15.159:0/3370266080 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c034070 con 0x7f48a4155930 2024-04-04T06:10:51.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.972+0000 7f48a2a00640 1 -- 172.21.15.159:0/3370266080 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f488c030800 con 0x7f48a4155930 2024-04-04T06:10:51.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.972+0000 7f48a2a00640 1 -- 172.21.15.159:0/3370266080 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c030bb0 con 0x7f48a4155930 2024-04-04T06:10:51.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.972+0000 7f48ab000640 1 -- 172.21.15.159:0/3370266080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48a4155930 msgr2=0x7f48a40d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:51.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.972+0000 7f48ab000640 1 --2- 172.21.15.159:0/3370266080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48a4155930 0x7f48a40d0280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f488c0097d0 tx=0x7f488c02b940 comp rx=0 tx=0).stop 2024-04-04T06:10:51.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.972+0000 7f48ab000640 1 -- 172.21.15.159:0/3370266080 shutdown_connections 2024-04-04T06:10:51.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.972+0000 7f48ab000640 1 --2- 172.21.15.159:0/3370266080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48a4155930 0x7f48a40d0280 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.972+0000 7f48ab000640 1 -- 172.21.15.159:0/3370266080 >> 172.21.15.159:0/3370266080 conn(0x7f48a406b750 msgr2=0x7f48a406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:51.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.972+0000 7f48ab000640 1 -- 172.21.15.159:0/3370266080 shutdown_connections 2024-04-04T06:10:51.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.972+0000 7f48ab000640 1 -- 172.21.15.159:0/3370266080 wait complete. 2024-04-04T06:10:51.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.974+0000 7f48ab000640 1 Processor -- start 2024-04-04T06:10:51.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.974+0000 7f48ab000640 1 -- start start 2024-04-04T06:10:51.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.975+0000 7f48ab000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48a40d07c0 0x7f48a40cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:51.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.975+0000 7f48ab000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48a4155550 0x7f48a40cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:51.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.975+0000 7f48ab000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48a4155930 0x7f48a415a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:51.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.975+0000 7f48ab000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f48a415bba0 con 0x7f48a40d07c0 2024-04-04T06:10:51.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.975+0000 7f48ab000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f48a415ba20 con 0x7f48a4155550 2024-04-04T06:10:51.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.975+0000 7f48ab000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f48a40b3cf0 con 0x7f48a4155930 2024-04-04T06:10:51.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.975+0000 7f48a3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48a40d07c0 0x7f48a40cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:51.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.975+0000 7f48a3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48a40d07c0 0x7f48a40cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:32812/0 (socket says 172.21.15.159:32812) 2024-04-04T06:10:51.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.975+0000 7f48a3e00640 1 -- 172.21.15.159:0/4235169281 learned_addr learned my addr 172.21.15.159:0/4235169281 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:51.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.975+0000 7f48a3e00640 1 -- 172.21.15.159:0/4235169281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48a4155930 msgr2=0x7f48a415a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:51.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.975+0000 7f48a9000640 1 --2- 172.21.15.159:0/4235169281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48a4155930 0x7f48a415a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:51.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.976+0000 7f48a3e00640 1 --2- 172.21.15.159:0/4235169281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48a4155930 0x7f48a415a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.976+0000 7f48a3e00640 1 -- 172.21.15.159:0/4235169281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48a4155550 msgr2=0x7f48a40cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:51.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.976+0000 7f48a3e00640 1 --2- 172.21.15.159:0/4235169281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48a4155550 0x7f48a40cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.976+0000 7f48a3e00640 1 -- 172.21.15.159:0/4235169281 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a415d5b0 con 0x7f48a40d07c0 2024-04-04T06:10:51.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.976+0000 7f48a9000640 1 --2- 172.21.15.159:0/4235169281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48a4155930 0x7f48a415a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:51.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.976+0000 7f48a3e00640 1 --2- 172.21.15.159:0/4235169281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48a40d07c0 0x7f48a40cecc0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f489400c450 tx=0x7f489400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:51.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.976+0000 7f489be00640 1 -- 172.21.15.159:0/4235169281 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48940052d0 con 0x7f48a40d07c0 2024-04-04T06:10:51.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.976+0000 7f489be00640 1 -- 172.21.15.159:0/4235169281 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4894005470 con 0x7f48a40d07c0 2024-04-04T06:10:51.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.976+0000 7f489be00640 1 -- 172.21.15.159:0/4235169281 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894012870 con 0x7f48a40d07c0 2024-04-04T06:10:51.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.976+0000 7f48ab000640 1 -- 172.21.15.159:0/4235169281 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a40af940 con 0x7f48a40d07c0 2024-04-04T06:10:51.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.976+0000 7f48ab000640 1 -- 172.21.15.159:0/4235169281 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a40991a0 con 0x7f48a40d07c0 2024-04-04T06:10:51.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.978+0000 7f489be00640 1 -- 172.21.15.159:0/4235169281 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4894012a10 con 0x7f48a40d07c0 2024-04-04T06:10:51.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.979+0000 7f489be00640 1 --2- 172.21.15.159:0/4235169281 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f488006c920 0x7f488006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:51.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.980+0000 7f489be00640 1 -- 172.21.15.159:0/4235169281 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f489408b730 con 0x7f48a40d07c0 2024-04-04T06:10:51.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.980+0000 7f48a3400640 1 --2- 172.21.15.159:0/4235169281 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f488006c920 0x7f488006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:51.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.980+0000 7f48addaa740 1 Processor -- start 2024-04-04T06:10:51.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.980+0000 7f48addaa740 1 -- start start 2024-04-04T06:10:51.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.981+0000 7f48a3400640 1 --2- 172.21.15.159:0/4235169281 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f488006c920 0x7f488006ede0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f488c036000 tx=0x7f488c030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:51.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.981+0000 7f48addaa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c45894e130 0x55c4589210d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:51.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.981+0000 7f48addaa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c45894e510 0x55c458950ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:51.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.981+0000 7f48addaa740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c458921610 0x55c458951400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:51.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.981+0000 7f48addaa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c4588e8ab0 con 0x55c45894e510 2024-04-04T06:10:51.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.981+0000 7f48addaa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c45888afa0 con 0x55c45894e130 2024-04-04T06:10:51.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.981+0000 7f48addaa740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c458833560 con 0x55c458921610 2024-04-04T06:10:51.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.982+0000 7f48a3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c45894e510 0x55c458950ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:51.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.982+0000 7f48a3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c45894e510 0x55c458950ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:32820/0 (socket says 172.21.15.159:32820) 2024-04-04T06:10:51.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.982+0000 7f48a3400640 1 -- 172.21.15.159:0/2616442482 learned_addr learned my addr 172.21.15.159:0/2616442482 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:51.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.982+0000 7f48a3e00640 1 --2- 172.21.15.159:0/2616442482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c45894e130 0x55c4589210d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:51.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.982+0000 7f48a9000640 1 --2- 172.21.15.159:0/2616442482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c458921610 0x55c458951400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:51.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.982+0000 7f48a3400640 1 -- 172.21.15.159:0/2616442482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c458921610 msgr2=0x55c458951400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:51.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.982+0000 7f48a3400640 1 --2- 172.21.15.159:0/2616442482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c458921610 0x55c458951400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.982+0000 7f48a3400640 1 -- 172.21.15.159:0/2616442482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c45894e130 msgr2=0x55c4589210d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:51.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.982+0000 7f48a3400640 1 --2- 172.21.15.159:0/2616442482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c45894e130 0x55c4589210d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.982+0000 7f48a3400640 1 -- 172.21.15.159:0/2616442482 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c45859f2d0 con 0x55c45894e510 2024-04-04T06:10:51.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.982+0000 7f48a3e00640 1 --2- 172.21.15.159:0/2616442482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c45894e130 0x55c4589210d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:51.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.983+0000 7f48a3400640 1 --2- 172.21.15.159:0/2616442482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c45894e510 0x55c458950ec0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f488c0098c0 tx=0x7f488c031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0f893a8db85493 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:51.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.983+0000 7f4898c00640 1 -- 172.21.15.159:0/2616442482 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c0044b0 con 0x55c45894e510 2024-04-04T06:10:51.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.983+0000 7f4898c00640 1 -- 172.21.15.159:0/2616442482 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f488c030600 con 0x55c45894e510 2024-04-04T06:10:51.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.983+0000 7f4898c00640 1 -- 172.21.15.159:0/2616442482 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c0309f0 con 0x55c45894e510 2024-04-04T06:10:51.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.984+0000 7f48addaa740 1 -- 172.21.15.159:0/2616442482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c45894e510 msgr2=0x55c458950ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:51.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.984+0000 7f48addaa740 1 --2- 172.21.15.159:0/2616442482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c45894e510 0x55c458950ec0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f488c0098c0 tx=0x7f488c031000 comp rx=0 tx=0).stop 2024-04-04T06:10:51.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.984+0000 7f48addaa740 1 -- 172.21.15.159:0/2616442482 shutdown_connections 2024-04-04T06:10:51.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.984+0000 7f48addaa740 1 --2- 172.21.15.159:0/2616442482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c458921610 0x55c458951400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.984+0000 7f48addaa740 1 --2- 172.21.15.159:0/2616442482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c45894e510 0x55c458950ec0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.984+0000 7f48addaa740 1 --2- 172.21.15.159:0/2616442482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c45894e130 0x55c4589210d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.984+0000 7f48addaa740 1 -- 172.21.15.159:0/2616442482 >> 172.21.15.159:0/2616442482 conn(0x55c458923340 msgr2=0x55c45896bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:51.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.984+0000 7f48addaa740 1 -- 172.21.15.159:0/2616442482 shutdown_connections 2024-04-04T06:10:51.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.984+0000 7f48addaa740 1 -- 172.21.15.159:0/2616442482 wait complete. 2024-04-04T06:10:51.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.984+0000 7f48addaa740 1 -- start start 2024-04-04T06:10:51.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.985+0000 7f48addaa740 1 Processor -- start 2024-04-04T06:10:51.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.986+0000 7f48addaa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c45894e130 0x55c45893f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:51.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.986+0000 7f48addaa740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c45894e510 0x55c458953e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:51.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.986+0000 7f48addaa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4589664f0 0x55c458966950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:51.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.986+0000 7f48addaa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c458846920 con 0x55c45894e130 2024-04-04T06:10:51.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.986+0000 7f48addaa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c458886f50 con 0x55c4589664f0 2024-04-04T06:10:51.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.986+0000 7f48addaa740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c4589258e0 con 0x55c45894e510 2024-04-04T06:10:51.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.986+0000 7f48addaa740 10 client.0 _ll_register_callbacks cb 0x55c458846310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:10:51.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.986+0000 7f48a3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c45894e130 0x55c45893f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:51.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.986+0000 7f48a3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c45894e130 0x55c45893f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:32822/0 (socket says 172.21.15.159:32822) 2024-04-04T06:10:51.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.986+0000 7f48a3e00640 1 -- 172.21.15.159:0/2373338341 learned_addr learned my addr 172.21.15.159:0/2373338341 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:51.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.986+0000 7f48a3e00640 1 -- 172.21.15.159:0/2373338341 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c45894e510 msgr2=0x55c458953e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:51.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.986+0000 7f48a3e00640 1 --2- 172.21.15.159:0/2373338341 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c45894e510 0x55c458953e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.986+0000 7f48a3e00640 1 -- 172.21.15.159:0/2373338341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4589664f0 msgr2=0x55c458966950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:51.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.986+0000 7f48a3e00640 1 --2- 172.21.15.159:0/2373338341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4589664f0 0x55c458966950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.986+0000 7f48a3e00640 1 -- 172.21.15.159:0/2373338341 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4586696b0 con 0x55c45894e130 2024-04-04T06:10:51.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.986+0000 7f48a3e00640 1 --2- 172.21.15.159:0/2373338341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c45894e130 0x55c45893f460 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f4894089bf0 tx=0x7f4894059590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda5863685897ef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:51.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.987+0000 7f4891600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:10:51.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.987+0000 7f4891600640 1 -- 172.21.15.159:0/2373338341 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489406de60 con 0x55c45894e130 2024-04-04T06:10:51.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.987+0000 7f4891600640 1 -- 172.21.15.159:0/2373338341 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f4894061ba0 con 0x55c45894e130 2024-04-04T06:10:51.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.987+0000 7f48addaa740 1 -- 172.21.15.159:0/2373338341 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f488c030600 con 0x55c45894e130 2024-04-04T06:10:51.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.987+0000 7f4891600640 1 -- 172.21.15.159:0/2373338341 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4894057810 con 0x55c45894e130 2024-04-04T06:10:51.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.987+0000 7f48addaa740 10 client.6759 fetch_fsmap learned FSMap version 45 2024-04-04T06:10:51.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.987+0000 7f48addaa740 1 -- 172.21.15.159:0/2373338341 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x55c458963f60 con 0x55c45894e130 2024-04-04T06:10:51.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.987+0000 7f4891600640 1 -- 172.21.15.159:0/2373338341 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f488c030600 con 0x55c45894e130 2024-04-04T06:10:51.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.987+0000 7f4891600640 1 -- 172.21.15.159:0/2373338341 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 45) ==== 3494+0+0 (secure 0 0 0) 0x7f48940588a0 con 0x55c45894e130 2024-04-04T06:10:51.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.987+0000 7f48addaa740 10 client.6759 fetch_fsmap finished waiting for FSMap version 45 2024-04-04T06:10:51.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.987+0000 7f48addaa740 10 client.6759 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:10:51.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.987+0000 7f48addaa740 20 client.6759 populate_metadata read hostname 'smithi159' 2024-04-04T06:10:51.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.987+0000 7f48addaa740 1 --2- 172.21.15.159:0/2373338341 >> [v2:172.21.15.159:6834/1652468640,v1:172.21.15.159:6835/1652468640] conn(0x55c458964ed0 0x55c4589652b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:51.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.987+0000 7f48addaa740 4 client.6759 mds_command: new command op to 6633 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:10:51.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.987+0000 7f48addaa740 1 -- 172.21.15.159:0/2373338341 --> [v2:172.21.15.159:6834/1652468640,v1:172.21.15.159:6835/1652468640] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4588984e0 con 0x55c458964ed0 2024-04-04T06:10:51.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.988+0000 7f48a3400640 1 --2- 172.21.15.159:0/2373338341 >> [v2:172.21.15.159:6834/1652468640,v1:172.21.15.159:6835/1652468640] conn(0x55c458964ed0 0x55c4589652b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:51.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.988+0000 7f4891600640 10 client.6759 ms_handle_connect on v2:172.21.15.159:6834/1652468640 2024-04-04T06:10:51.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.988+0000 7f48a3400640 1 --2- 172.21.15.159:0/2373338341 >> [v2:172.21.15.159:6834/1652468640,v1:172.21.15.159:6835/1652468640] conn(0x55c458964ed0 0x55c4589652b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2e655cb747faa34 server_cookie=f9e0f57158a3d9c2 in_seq=0 out_seq=0 2024-04-04T06:10:51.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.989+0000 7f4891600640 1 -- 172.21.15.159:0/2373338341 <== mds.0 v2:172.21.15.159:6834/1652468640 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c4588984e0 con 0x55c458964ed0 2024-04-04T06:10:51.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.989+0000 7f4891600640 10 client.6759 handle_command_reply: tid=0 2024-04-04T06:10:51.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.989+0000 7f48addaa740 1 client.6759 shutdown 2024-04-04T06:10:51.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.989+0000 7f48addaa740 1 -- 172.21.15.159:0/2373338341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c45894e130 msgr2=0x55c45893f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:51.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.989+0000 7f48addaa740 1 --2- 172.21.15.159:0/2373338341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c45894e130 0x55c45893f460 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f4894089bf0 tx=0x7f4894059590 comp rx=0 tx=0).stop 2024-04-04T06:10:51.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.989+0000 7f48addaa740 1 -- 172.21.15.159:0/2373338341 shutdown_connections 2024-04-04T06:10:51.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.989+0000 7f48addaa740 1 --2- 172.21.15.159:0/2373338341 >> [v2:172.21.15.159:6834/1652468640,v1:172.21.15.159:6835/1652468640] conn(0x55c458964ed0 0x55c4589652b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.989+0000 7f48addaa740 1 --2- 172.21.15.159:0/2373338341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c4589664f0 0x55c458966950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.989+0000 7f48addaa740 1 --2- 172.21.15.159:0/2373338341 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c45894e510 0x55c458953e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.989+0000 7f48addaa740 1 --2- 172.21.15.159:0/2373338341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c45894e130 0x55c45893f460 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:51.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.989+0000 7f4891600640 0 client.6759 ms_handle_reset on v2:172.21.15.159:6834/1652468640 2024-04-04T06:10:51.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.989+0000 7f48addaa740 1 -- 172.21.15.159:0/2373338341 >> 172.21.15.159:0/2373338341 conn(0x55c458921610 msgr2=0x55c45891fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:51.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.990+0000 7f48addaa740 1 -- 172.21.15.159:0/2373338341 shutdown_connections 2024-04-04T06:10:51.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.990+0000 7f48addaa740 1 -- 172.21.15.159:0/2373338341 wait complete. 2024-04-04T06:10:51.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:51.990+0000 7f48addaa740 20 client.6759 trim_cache size 0 max 16384 2024-04-04T06:10:52.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.009+0000 7f48addaa740 1 Processor -- start 2024-04-04T06:10:52.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.009+0000 7f48addaa740 1 -- start start 2024-04-04T06:10:52.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.010+0000 7f48addaa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c45894e130 0x55c45898fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:52.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.010+0000 7f48addaa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c45894e510 0x55c458963190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:52.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.010+0000 7f48addaa740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c4589664f0 0x55c458950900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:52.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.010+0000 7f48addaa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f48a41ffb00 con 0x55c45894e130 2024-04-04T06:10:52.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.010+0000 7f48addaa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f48a40b0af0 con 0x55c45894e510 2024-04-04T06:10:52.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.010+0000 7f48addaa740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f48a40b1140 con 0x55c4589664f0 2024-04-04T06:10:52.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.010+0000 7f48a3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c45894e130 0x55c45898fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:52.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.010+0000 7f48a3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c45894e130 0x55c45898fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:32836/0 (socket says 172.21.15.159:32836) 2024-04-04T06:10:52.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.010+0000 7f48a3e00640 1 -- 172.21.15.159:0/1063113514 learned_addr learned my addr 172.21.15.159:0/1063113514 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:52.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.010+0000 7f48a9000640 1 --2- 172.21.15.159:0/1063113514 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c4589664f0 0x55c458950900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:52.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.010+0000 7f48a3e00640 1 -- 172.21.15.159:0/1063113514 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c4589664f0 msgr2=0x55c458950900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:52.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.010+0000 7f48a3e00640 1 --2- 172.21.15.159:0/1063113514 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c4589664f0 0x55c458950900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.010+0000 7f48a3e00640 1 -- 172.21.15.159:0/1063113514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c45894e510 msgr2=0x55c458963190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:52.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.010+0000 7f48a3e00640 1 --2- 172.21.15.159:0/1063113514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c45894e510 0x55c458963190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.010+0000 7f48a3e00640 1 -- 172.21.15.159:0/1063113514 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c458970320 con 0x55c45894e130 2024-04-04T06:10:52.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.011+0000 7f48a3e00640 1 --2- 172.21.15.159:0/1063113514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c45894e130 0x55c45898fb60 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f4894007fd0 tx=0x7f489405c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6064ba5d68a97e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:52.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.011+0000 7f4893e00640 1 -- 172.21.15.159:0/1063113514 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489406de60 con 0x55c45894e130 2024-04-04T06:10:52.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.011+0000 7f4893e00640 1 -- 172.21.15.159:0/1063113514 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48940588a0 con 0x55c45894e130 2024-04-04T06:10:52.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.011+0000 7f4893e00640 1 -- 172.21.15.159:0/1063113514 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48940551a0 con 0x55c45894e130 2024-04-04T06:10:52.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.011+0000 7f48addaa740 1 -- 172.21.15.159:0/1063113514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c45894e130 msgr2=0x55c45898fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:52.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.011+0000 7f48addaa740 1 --2- 172.21.15.159:0/1063113514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c45894e130 0x55c45898fb60 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f4894007fd0 tx=0x7f489405c000 comp rx=0 tx=0).stop 2024-04-04T06:10:52.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.011+0000 7f48addaa740 1 -- 172.21.15.159:0/1063113514 shutdown_connections 2024-04-04T06:10:52.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.011+0000 7f48addaa740 1 --2- 172.21.15.159:0/1063113514 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c4589664f0 0x55c458950900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.011+0000 7f48addaa740 1 --2- 172.21.15.159:0/1063113514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c45894e510 0x55c458963190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.011+0000 7f48addaa740 1 --2- 172.21.15.159:0/1063113514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c45894e130 0x55c45898fb60 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.012+0000 7f48addaa740 1 -- 172.21.15.159:0/1063113514 >> 172.21.15.159:0/1063113514 conn(0x55c458921610 msgr2=0x55c4589280f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:52.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.012+0000 7f48addaa740 1 -- 172.21.15.159:0/1063113514 shutdown_connections 2024-04-04T06:10:52.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.012+0000 7f48addaa740 1 -- 172.21.15.159:0/1063113514 wait complete. 2024-04-04T06:10:52.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.012+0000 7f48addaa740 1 -- start start 2024-04-04T06:10:52.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.012+0000 7f48addaa740 1 Processor -- start 2024-04-04T06:10:52.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.013+0000 7f48addaa740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c45894e130 0x55c458950900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:52.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.013+0000 7f48addaa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c45894e510 0x55c458950e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:52.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.013+0000 7f48addaa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4589664f0 0x55c45891fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:52.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.013+0000 7f48addaa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c4589530f0 con 0x55c4589664f0 2024-04-04T06:10:52.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.013+0000 7f48addaa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c458964240 con 0x55c45894e510 2024-04-04T06:10:52.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.013+0000 7f48addaa740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c45897fce0 con 0x55c45894e130 2024-04-04T06:10:52.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.013+0000 7f48addaa740 10 client.0 _ll_register_callbacks cb 0x7f4894059be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:10:52.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.014+0000 7f48a9000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4589664f0 0x55c45891fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:52.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.014+0000 7f48a9000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4589664f0 0x55c45891fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:32846/0 (socket says 172.21.15.159:32846) 2024-04-04T06:10:52.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.014+0000 7f48a9000640 1 -- 172.21.15.159:0/2106149850 learned_addr learned my addr 172.21.15.159:0/2106149850 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:52.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.014+0000 7f48a3e00640 1 --2- 172.21.15.159:0/2106149850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c45894e130 0x55c458950900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:52.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.014+0000 7f48a9000640 1 -- 172.21.15.159:0/2106149850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c45894e130 msgr2=0x55c458950900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:52.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.014+0000 7f48a3400640 1 --2- 172.21.15.159:0/2106149850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c45894e510 0x55c458950e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:52.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.014+0000 7f48a9000640 1 --2- 172.21.15.159:0/2106149850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c45894e130 0x55c458950900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.014+0000 7f48a9000640 1 -- 172.21.15.159:0/2106149850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c45894e510 msgr2=0x55c458950e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:52.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.014+0000 7f48a9000640 1 --2- 172.21.15.159:0/2106149850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c45894e510 0x55c458950e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.014+0000 7f48a9000640 1 -- 172.21.15.159:0/2106149850 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c45898ff30 con 0x55c4589664f0 2024-04-04T06:10:52.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.014+0000 7f48a3e00640 1 --2- 172.21.15.159:0/2106149850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c45894e130 0x55c458950900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:52.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.014+0000 7f48a9000640 1 --2- 172.21.15.159:0/2106149850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4589664f0 0x55c45891fea0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f489c001560 tx=0x7f489c007410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b127083a2e9241d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:52.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.014+0000 7f4891600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:10:52.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.015+0000 7f4891600640 1 -- 172.21.15.159:0/2106149850 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c01b060 con 0x55c4589664f0 2024-04-04T06:10:52.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.015+0000 7f4891600640 1 -- 172.21.15.159:0/2106149850 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f489c005bd0 con 0x55c4589664f0 2024-04-04T06:10:52.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.015+0000 7f48addaa740 1 -- 172.21.15.159:0/2106149850 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48940588a0 con 0x55c4589664f0 2024-04-04T06:10:52.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.015+0000 7f4891600640 1 -- 172.21.15.159:0/2106149850 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f489c007a90 con 0x55c4589664f0 2024-04-04T06:10:52.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.015+0000 7f48addaa740 10 client.6765 fetch_fsmap learned FSMap version 45 2024-04-04T06:10:52.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.015+0000 7f48addaa740 1 -- 172.21.15.159:0/2106149850 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x55c45897baa0 con 0x55c4589664f0 2024-04-04T06:10:52.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.015+0000 7f4891600640 1 -- 172.21.15.159:0/2106149850 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48940588a0 con 0x55c4589664f0 2024-04-04T06:10:52.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.015+0000 7f4891600640 1 -- 172.21.15.159:0/2106149850 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 45) ==== 3494+0+0 (secure 0 0 0) 0x7f489c015460 con 0x55c4589664f0 2024-04-04T06:10:52.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.015+0000 7f48addaa740 10 client.6765 fetch_fsmap finished waiting for FSMap version 45 2024-04-04T06:10:52.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.015+0000 7f48addaa740 10 client.6765 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:10:52.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.015+0000 7f48addaa740 20 client.6765 populate_metadata read hostname 'smithi159' 2024-04-04T06:10:52.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.015+0000 7f48addaa740 1 --2- 172.21.15.159:0/2106149850 >> [v2:172.21.15.159:6834/1652468640,v1:172.21.15.159:6835/1652468640] conn(0x55c45897a9c0 0x55c45897ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:52.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.015+0000 7f48addaa740 4 client.6765 mds_command: new command op to 6633 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-04T06:10:52.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.015+0000 7f48addaa740 1 -- 172.21.15.159:0/2106149850 --> [v2:172.21.15.159:6834/1652468640,v1:172.21.15.159:6835/1652468640] -- command(tid 0: {"prefix": "status"}) -- 0x55c458892830 con 0x55c45897a9c0 2024-04-04T06:10:52.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.016+0000 7f48a3e00640 1 --2- 172.21.15.159:0/2106149850 >> [v2:172.21.15.159:6834/1652468640,v1:172.21.15.159:6835/1652468640] conn(0x55c45897a9c0 0x55c45897ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:52.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.016+0000 7f48a3e00640 1 --2- 172.21.15.159:0/2106149850 >> [v2:172.21.15.159:6834/1652468640,v1:172.21.15.159:6835/1652468640] conn(0x55c45897a9c0 0x55c45897ada0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=263c46a6e5944a25 server_cookie=109c4830c61e43db in_seq=0 out_seq=0 2024-04-04T06:10:52.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.016+0000 7f4891600640 10 client.6765 ms_handle_connect on v2:172.21.15.159:6834/1652468640 2024-04-04T06:10:52.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.016+0000 7f4891600640 1 -- 172.21.15.159:0/2106149850 <== mds.0 v2:172.21.15.159:6834/1652468640 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55c458892830 con 0x55c45897a9c0 2024-04-04T06:10:52.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.016+0000 7f4891600640 10 client.6765 handle_command_reply: tid=0 2024-04-04T06:10:52.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.016+0000 7f48addaa740 1 client.6765 shutdown 2024-04-04T06:10:52.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.016+0000 7f48addaa740 1 -- 172.21.15.159:0/2106149850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4589664f0 msgr2=0x55c45891fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:52.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.016+0000 7f48addaa740 1 --2- 172.21.15.159:0/2106149850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4589664f0 0x55c45891fea0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f489c001560 tx=0x7f489c007410 comp rx=0 tx=0).stop 2024-04-04T06:10:52.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.016+0000 7f48addaa740 1 -- 172.21.15.159:0/2106149850 shutdown_connections 2024-04-04T06:10:52.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.016+0000 7f48addaa740 1 --2- 172.21.15.159:0/2106149850 >> [v2:172.21.15.159:6834/1652468640,v1:172.21.15.159:6835/1652468640] conn(0x55c45897a9c0 0x55c45897ada0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.016+0000 7f48addaa740 1 --2- 172.21.15.159:0/2106149850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4589664f0 0x55c45891fea0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.016+0000 7f48addaa740 1 --2- 172.21.15.159:0/2106149850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c45894e510 0x55c458950e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.017+0000 7f48addaa740 1 --2- 172.21.15.159:0/2106149850 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c45894e130 0x55c458950900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.017+0000 7f4891600640 0 client.6765 ms_handle_reset on v2:172.21.15.159:6834/1652468640 2024-04-04T06:10:52.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.017+0000 7f48addaa740 1 -- 172.21.15.159:0/2106149850 >> 172.21.15.159:0/2106149850 conn(0x55c458921610 msgr2=0x55c4589280f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:52.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.017+0000 7f48addaa740 1 -- 172.21.15.159:0/2106149850 shutdown_connections 2024-04-04T06:10:52.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.017+0000 7f48addaa740 1 -- 172.21.15.159:0/2106149850 wait complete. 2024-04-04T06:10:52.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.017+0000 7f48addaa740 20 client.6765 trim_cache size 0 max 16384 2024-04-04T06:10:52.014 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:10:52.015 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_fsid": "30311045-f714-48fa-832b-4cb2ba8bf0bf", 2024-04-04T06:10:52.015 INFO:teuthology.orchestra.run.smithi159.stdout: "whoami": 0, 2024-04-04T06:10:52.015 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 6633, 2024-04-04T06:10:52.015 INFO:teuthology.orchestra.run.smithi159.stdout: "want_state": "up:active", 2024-04-04T06:10:52.015 INFO:teuthology.orchestra.run.smithi159.stdout: "state": "up:active", 2024-04-04T06:10:52.015 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs", 2024-04-04T06:10:52.015 INFO:teuthology.orchestra.run.smithi159.stdout: "rank_uptime": 6.6687417770000001, 2024-04-04T06:10:52.015 INFO:teuthology.orchestra.run.smithi159.stdout: "mdsmap_epoch": 45, 2024-04-04T06:10:52.015 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch": 71, 2024-04-04T06:10:52.015 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch_barrier": 71, 2024-04-04T06:10:52.015 INFO:teuthology.orchestra.run.smithi159.stdout: "uptime": 13.169070087 2024-04-04T06:10:52.015 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:10:52.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.017+0000 7f48ab000640 1 -- 172.21.15.159:0/4235169281 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f488006c920 msgr2=0x7f488006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:52.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.017+0000 7f48ab000640 1 --2- 172.21.15.159:0/4235169281 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f488006c920 0x7f488006ede0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f488c036000 tx=0x7f488c030000 comp rx=0 tx=0).stop 2024-04-04T06:10:52.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.017+0000 7f48ab000640 1 -- 172.21.15.159:0/4235169281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48a40d07c0 msgr2=0x7f48a40cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:52.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.017+0000 7f48ab000640 1 --2- 172.21.15.159:0/4235169281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48a40d07c0 0x7f48a40cecc0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f489400c450 tx=0x7f489400c910 comp rx=0 tx=0).stop 2024-04-04T06:10:52.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.017+0000 7f48ab000640 1 -- 172.21.15.159:0/4235169281 shutdown_connections 2024-04-04T06:10:52.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.017+0000 7f48ab000640 1 --2- 172.21.15.159:0/4235169281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48a4155930 0x7f48a415a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.017+0000 7f48ab000640 1 --2- 172.21.15.159:0/4235169281 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f488006c920 0x7f488006ede0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.017+0000 7f48ab000640 1 --2- 172.21.15.159:0/4235169281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48a4155550 0x7f48a40cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.017+0000 7f48ab000640 1 --2- 172.21.15.159:0/4235169281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48a40d07c0 0x7f48a40cecc0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.017+0000 7f48ab000640 1 -- 172.21.15.159:0/4235169281 >> 172.21.15.159:0/4235169281 conn(0x7f48a406b750 msgr2=0x7f48a415d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:52.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.018+0000 7f48ab000640 1 -- 172.21.15.159:0/4235169281 shutdown_connections 2024-04-04T06:10:52.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.018+0000 7f48ab000640 1 -- 172.21.15.159:0/4235169281 wait complete. 2024-04-04T06:10:52.026 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-04T06:10:52.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.229+0000 7f7289800640 1 Processor -- start 2024-04-04T06:10:52.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.229+0000 7f7289800640 1 -- start start 2024-04-04T06:10:52.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.231+0000 7f7289800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f72840d1380 0x7f72840d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:52.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.231+0000 7f7289800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72840d2fd0 con 0x7f728415e420 2024-04-04T06:10:52.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.231+0000 7f7289800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72840cfa70 con 0x7f72840d3a70 2024-04-04T06:10:52.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.231+0000 7f7289800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f72840cf790 con 0x7f72840d1380 2024-04-04T06:10:52.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.231+0000 7f7283e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f728415e420 0x7f72840b2f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:44452/0 (socket says 172.21.15.159:44452) 2024-04-04T06:10:52.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.231+0000 7f7283e00640 1 -- 172.21.15.159:0/838051891 learned_addr learned my addr 172.21.15.159:0/838051891 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:52.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.231+0000 7f727b400640 1 --2- 172.21.15.159:0/838051891 >> v2:172.21.15.159:3300/0 conn(0x7f72840d1380 0x7f72840d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:52.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.231+0000 7f7283400640 1 -- 172.21.15.159:0/838051891 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4228640830 0 0) 0x7f72840d2fd0 con 0x7f728415e420 2024-04-04T06:10:52.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.231+0000 7f727b400640 1 -- 172.21.15.159:0/838051891 >> v1:172.21.15.178:6789/0 conn(0x7f72840d3a70 legacy=0x7f72840d3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:52.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.231+0000 7f727b400640 1 -- 172.21.15.159:0/838051891 >> v1:172.21.15.159:6789/0 conn(0x7f728415e420 legacy=0x7f72840b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:52.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.232+0000 7f727b400640 1 -- 172.21.15.159:0/838051891 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72840d3270 con 0x7f72840d1380 2024-04-04T06:10:52.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.233+0000 7f727b400640 1 --2- 172.21.15.159:0/838051891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72840d1380 0x7f72840d1810 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f72840cb440 tx=0x7f726c02b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827a748461078b89 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:52.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.233+0000 7f7283400640 1 -- 172.21.15.159:0/838051891 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c035020 con 0x7f72840d1380 2024-04-04T06:10:52.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.233+0000 7f7283400640 1 -- 172.21.15.159:0/838051891 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f726c031890 con 0x7f72840d1380 2024-04-04T06:10:52.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.233+0000 7f7283400640 1 -- 172.21.15.159:0/838051891 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c031c60 con 0x7f72840d1380 2024-04-04T06:10:52.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.233+0000 7f7289800640 1 -- 172.21.15.159:0/838051891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72840d1380 msgr2=0x7f72840d1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:52.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.233+0000 7f7289800640 1 --2- 172.21.15.159:0/838051891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72840d1380 0x7f72840d1810 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f72840cb440 tx=0x7f726c02b930 comp rx=0 tx=0).stop 2024-04-04T06:10:52.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.234+0000 7f7289800640 1 -- 172.21.15.159:0/838051891 shutdown_connections 2024-04-04T06:10:52.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.234+0000 7f7289800640 1 --2- 172.21.15.159:0/838051891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72840d1380 0x7f72840d1810 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.234+0000 7f7289800640 1 -- 172.21.15.159:0/838051891 >> 172.21.15.159:0/838051891 conn(0x7f7284067670 msgr2=0x7f7284067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:52.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.234+0000 7f7289800640 1 -- 172.21.15.159:0/838051891 shutdown_connections 2024-04-04T06:10:52.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.234+0000 7f7289800640 1 -- 172.21.15.159:0/838051891 wait complete. 2024-04-04T06:10:52.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.235+0000 7f7289800640 1 Processor -- start 2024-04-04T06:10:52.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.236+0000 7f7289800640 1 -- start start 2024-04-04T06:10:52.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.236+0000 7f7289800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72840d1380 0x7f7284155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:52.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.236+0000 7f7289800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72840d3a70 0x7f728415fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:52.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.237+0000 7f7289800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f728415e420 0x7f72840cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:52.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.237+0000 7f7289800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f72840cf420 con 0x7f728415e420 2024-04-04T06:10:52.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.237+0000 7f7289800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f728415e800 con 0x7f72840d1380 2024-04-04T06:10:52.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.237+0000 7f7289800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f72840b3630 con 0x7f72840d3a70 2024-04-04T06:10:52.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.237+0000 7f7288e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f728415e420 0x7f72840cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:52.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.237+0000 7f7288e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f728415e420 0x7f72840cb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:32862/0 (socket says 172.21.15.159:32862) 2024-04-04T06:10:52.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.237+0000 7f7288e00640 1 -- 172.21.15.159:0/2871583480 learned_addr learned my addr 172.21.15.159:0/2871583480 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:52.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.237+0000 7f727b400640 1 --2- 172.21.15.159:0/2871583480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72840d3a70 0x7f728415fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:52.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.237+0000 7f7283e00640 1 --2- 172.21.15.159:0/2871583480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72840d1380 0x7f7284155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:52.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.237+0000 7f7288e00640 1 -- 172.21.15.159:0/2871583480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72840d3a70 msgr2=0x7f728415fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:52.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.237+0000 7f7288e00640 1 --2- 172.21.15.159:0/2871583480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72840d3a70 0x7f728415fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.237+0000 7f7288e00640 1 -- 172.21.15.159:0/2871583480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72840d1380 msgr2=0x7f7284155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:52.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.237+0000 7f7288e00640 1 --2- 172.21.15.159:0/2871583480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72840d1380 0x7f7284155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.237+0000 7f7288e00640 1 -- 172.21.15.159:0/2871583480 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72840cc2f0 con 0x7f728415e420 2024-04-04T06:10:52.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.237+0000 7f727b400640 1 --2- 172.21.15.159:0/2871583480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72840d3a70 0x7f728415fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:52.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.238+0000 7f7288e00640 1 --2- 172.21.15.159:0/2871583480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f728415e420 0x7f72840cb980 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f727c00c410 tx=0x7f727c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:52.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.238+0000 7f7280c00640 1 -- 172.21.15.159:0/2871583480 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c011790 con 0x7f728415e420 2024-04-04T06:10:52.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.238+0000 7f7280c00640 1 -- 172.21.15.159:0/2871583480 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f727c017070 con 0x7f728415e420 2024-04-04T06:10:52.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.238+0000 7f7289800640 1 -- 172.21.15.159:0/2871583480 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72840cc5d0 con 0x7f728415e420 2024-04-04T06:10:52.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.238+0000 7f7280c00640 1 -- 172.21.15.159:0/2871583480 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c011bb0 con 0x7f728415e420 2024-04-04T06:10:52.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.238+0000 7f7289800640 1 -- 172.21.15.159:0/2871583480 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72840ccda0 con 0x7f728415e420 2024-04-04T06:10:52.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.239+0000 7f7289800640 1 -- 172.21.15.159:0/2871583480 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7284075940 con 0x7f728415e420 2024-04-04T06:10:52.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.241+0000 7f7280c00640 1 -- 172.21.15.159:0/2871583480 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f727c01c460 con 0x7f728415e420 2024-04-04T06:10:52.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.243+0000 7f7280c00640 1 --2- 172.21.15.159:0/2871583480 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f726006c8f0 0x7f726006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:52.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.243+0000 7f7280c00640 1 -- 172.21.15.159:0/2871583480 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f727c08b6f0 con 0x7f728415e420 2024-04-04T06:10:52.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.243+0000 7f7283e00640 1 --2- 172.21.15.159:0/2871583480 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f726006c8f0 0x7f726006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:52.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.244+0000 7f7283e00640 1 --2- 172.21.15.159:0/2871583480 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f726006c8f0 0x7f726006edb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f72740012e0 tx=0x7f7274009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:52.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.247+0000 7f7280c00640 1 -- 172.21.15.159:0/2871583480 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f727c021020 con 0x7f728415e420 2024-04-04T06:10:52.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.623+0000 7f7289800640 1 -- 172.21.15.159:0/2871583480 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f728415d9f0 con 0x7f728415e420 2024-04-04T06:10:52.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.624+0000 7f7280c00640 1 -- 172.21.15.159:0/2871583480 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) ==== 53+0+73 (secure 0 0 0) 0x7f727c059bb0 con 0x7f728415e420 2024-04-04T06:10:52.617 INFO:teuthology.orchestra.run.smithi159.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-04T06:10:52.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.627+0000 7f7289800640 1 -- 172.21.15.159:0/2871583480 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f726006c8f0 msgr2=0x7f726006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:52.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.627+0000 7f7289800640 1 --2- 172.21.15.159:0/2871583480 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f726006c8f0 0x7f726006edb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f72740012e0 tx=0x7f7274009040 comp rx=0 tx=0).stop 2024-04-04T06:10:52.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.627+0000 7f7289800640 1 -- 172.21.15.159:0/2871583480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f728415e420 msgr2=0x7f72840cb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:52.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.627+0000 7f7289800640 1 --2- 172.21.15.159:0/2871583480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f728415e420 0x7f72840cb980 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f727c00c410 tx=0x7f727c00c8d0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.627+0000 7f7289800640 1 -- 172.21.15.159:0/2871583480 shutdown_connections 2024-04-04T06:10:52.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.627+0000 7f7289800640 1 --2- 172.21.15.159:0/2871583480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f728415e420 0x7f72840cb980 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.627+0000 7f7289800640 1 --2- 172.21.15.159:0/2871583480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f72840d3a70 0x7f728415fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.627+0000 7f7289800640 1 --2- 172.21.15.159:0/2871583480 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f726006c8f0 0x7f726006edb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.627+0000 7f7289800640 1 --2- 172.21.15.159:0/2871583480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72840d1380 0x7f7284155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:52.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.627+0000 7f7289800640 1 -- 172.21.15.159:0/2871583480 >> 172.21.15.159:0/2871583480 conn(0x7f7284067670 msgr2=0x7f72840d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:52.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.627+0000 7f7289800640 1 -- 172.21.15.159:0/2871583480 shutdown_connections 2024-04-04T06:10:52.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52.627+0000 7f7289800640 1 -- 172.21.15.159:0/2871583480 wait complete. 2024-04-04T06:10:52.638 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-04T06:10:52.638 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-04T06:10:52.638 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi159.front.sepia.ceph.com 2024-04-04T06:10:52.638 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-04T06:10:52.638 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-04T06:10:52.638 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-04T06:10:52.638 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-04T06:10:52.638 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-04T06:10:52.641 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:10:52.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:10:52.641 DEBUG:teuthology.orchestra.run.smithi159:> ip addr 2024-04-04T06:10:52.684 INFO:teuthology.orchestra.run.smithi159.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:10:52.684 INFO:teuthology.orchestra.run.smithi159.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:10:52.684 INFO:teuthology.orchestra.run.smithi159.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:10:52.684 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:10:52.684 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 ::1/128 scope host 2024-04-04T06:10:52.684 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:10:52.684 INFO:teuthology.orchestra.run.smithi159.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:10:52.684 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c6 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:10:52.684 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f0 2024-04-04T06:10:52.684 INFO:teuthology.orchestra.run.smithi159.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:10:52.684 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c7 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:10:52.684 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f1 2024-04-04T06:10:52.684 INFO:teuthology.orchestra.run.smithi159.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:10:52.685 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4e brd ff:ff:ff:ff:ff:ff 2024-04-04T06:10:52.685 INFO:teuthology.orchestra.run.smithi159.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-04T06:10:52.685 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:10:52.685 INFO:teuthology.orchestra.run.smithi159.stdout: inet 172.21.15.159/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-04T06:10:52.685 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft 42280sec preferred_lft 42280sec 2024-04-04T06:10:52.685 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::ec4:7aff:fe88:724f/64 scope link 2024-04-04T06:10:52.685 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:10:52.685 INFO:teuthology.orchestra.run.smithi159.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-04T06:10:52.685 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:10:52.685 INFO:teuthology.orchestra.run.smithi159.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-04T06:10:52.685 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:10:52.685 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::989e:e4ff:fea5:7568/64 scope link 2024-04-04T06:10:52.685 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:10:52.685 INFO:teuthology.orchestra.run.smithi159.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:10:52.686 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether a2:e3:e0:cb:0a:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-04T06:10:52.686 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::a0e3:e0ff:fecb:a73/64 scope link 2024-04-04T06:10:52.686 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:10:52.686 INFO:teuthology.orchestra.run.smithi159.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:10:52.686 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-04T06:10:52.686 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::c0a:8aff:fe54:b58f/64 scope link 2024-04-04T06:10:52.686 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:10:52.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:10:52.686 DEBUG:teuthology.orchestra.run.smithi159:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-04T06:10:52.810 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:10:52.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:10:52.815 DEBUG:teuthology.orchestra.run.smithi159:> route 2024-04-04T06:10:52.865 INFO:teuthology.orchestra.run.smithi159.stdout:Kernel IP routing table 2024-04-04T06:10:52.866 INFO:teuthology.orchestra.run.smithi159.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-04T06:10:52.866 INFO:teuthology.orchestra.run.smithi159.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-04T06:10:52.866 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-04T06:10:52.866 INFO:teuthology.orchestra.run.smithi159.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-04T06:10:52.867 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:10:52.867 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:10:52.867 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-04T06:10:52.867 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-04T06:10:52.868 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-04T06:10:52.868 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:10:52.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:10:53.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:10:53.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:10:53.105 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-04-04T06:10:53.154 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-04T06:10:53.154 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-04T06:10:53.154 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:10:53.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:10:53.154 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set brx.0 up 2024-04-04T06:10:53.240 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:10:53.240 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-04T06:10:53.297 INFO:teuthology.orchestra.run.smithi159.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:10:53.297 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:10:53.297 DEBUG:teuthology.orchestra.run.smithi159:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:10:53.352 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-04T06:10:53.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:10:53.353 DEBUG:teuthology.orchestra.run.smithi159:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:10:53.484 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:10:53.484 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:10:53.484 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:10:53.509 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:10:53.509 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:10:53.510 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:10:53.510 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:10:53.510 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:10:53.532 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:10:53.533 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:10:53.533 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:10:53.533 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:10:53.533 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:10:53.533 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-04T06:10:53.533 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-04T06:10:53.533 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30311045-f714-48fa-832b-4cb2ba8bf0bf" will pass to kernel 2024-04-04T06:10:53.537 INFO:teuthology.orchestra.run.smithi159.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-04T06:10:53.537 INFO:teuthology.orchestra.run.smithi159.stdout: You just mounted a file system that supports labels which does not 2024-04-04T06:10:53.537 INFO:teuthology.orchestra.run.smithi159.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-04T06:10:53.537 INFO:teuthology.orchestra.run.smithi159.stdout: applications will generate AVC messages and not be allowed access to 2024-04-04T06:10:53.537 INFO:teuthology.orchestra.run.smithi159.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-04T06:10:53.541 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-04T06:10:53.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:10:53.541 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:10:53.622 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:10:53.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:10:53.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:10:53.763 INFO:teuthology.orchestra.run.smithi159.stdout:client6777 2024-04-04T06:10:53.763 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:10:53.763 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:10:53.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:10:53.812 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-04-04T06:10:53.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:10:53.915 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-04-04T06:10:53.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:10:53.965 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-04-04T06:10:54.009 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:10:54.009 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:10:54.009 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:10:54.009 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:10:54.009 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-04-04T06:10:54.010 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:10:54.175 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627776 2024-04-04T06:10:54.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:10:54.189 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:10:54.189 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:10:54.282 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:10:54.282 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:10:54.330 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:10:54.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.589+0000 7f7308600640 1 Processor -- start 2024-04-04T06:10:54.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.589+0000 7f7308600640 1 -- start start 2024-04-04T06:10:54.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.592+0000 7f7308600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f730015e7e0 0x7f73000d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:54.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.592+0000 7f7308600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f730006ca00 0x7f73000d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:54.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.592+0000 7f7308600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f73000d2d60 0x7f73000cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:54.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.592+0000 7f7308600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f730015ebc0 con 0x7f73000d2d60 2024-04-04T06:10:54.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.592+0000 7f7308600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f730006d1c0 con 0x7f730006ca00 2024-04-04T06:10:54.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.592+0000 7f7308600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f730006d4a0 con 0x7f730015e7e0 2024-04-04T06:10:54.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.592+0000 7f7306600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f73000d2d60 0x7f73000cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:54.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.593+0000 7f7306600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f73000d2d60 0x7f73000cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42498/0 (socket says 172.21.15.159:42498) 2024-04-04T06:10:54.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.593+0000 7f7306600640 1 -- 172.21.15.159:0/1243246907 learned_addr learned my addr 172.21.15.159:0/1243246907 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:54.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.593+0000 7f7305200640 1 --2- 172.21.15.159:0/1243246907 >> v2:172.21.15.178:3300/0 conn(0x7f730006ca00 0x7f73000d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:54.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.593+0000 7f7305c00640 1 --2- 172.21.15.159:0/1243246907 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f730015e7e0 0x7f73000d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:54.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.593+0000 7f7306600640 1 -- 172.21.15.159:0/1243246907 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f730015e7e0 msgr2=0x7f73000d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:54.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.593+0000 7f7306600640 1 --2- 172.21.15.159:0/1243246907 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f730015e7e0 0x7f73000d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:54.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.593+0000 7f7306600640 1 -- 172.21.15.159:0/1243246907 >> v2:172.21.15.178:3300/0 conn(0x7f730006ca00 msgr2=0x7f73000d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:54.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.593+0000 7f7306600640 1 --2- 172.21.15.159:0/1243246907 >> v2:172.21.15.178:3300/0 conn(0x7f730006ca00 0x7f73000d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:54.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.593+0000 7f7306600640 1 -- 172.21.15.159:0/1243246907 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73000cf320 con 0x7f73000d2d60 2024-04-04T06:10:54.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.594+0000 7f7306600640 1 --2- 172.21.15.159:0/1243246907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73000d2d60 0x7f73000cfb20 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f730008c910 tx=0x7f72f802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f78f94460887a03 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:54.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.594+0000 7f72ffe00640 1 -- 172.21.15.159:0/1243246907 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f802c9e0 con 0x7f73000d2d60 2024-04-04T06:10:54.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.594+0000 7f72ffe00640 1 -- 172.21.15.159:0/1243246907 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72f802cb80 con 0x7f73000d2d60 2024-04-04T06:10:54.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.594+0000 7f72ffe00640 1 -- 172.21.15.159:0/1243246907 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f80343f0 con 0x7f73000d2d60 2024-04-04T06:10:54.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.595+0000 7f7308600640 1 -- 172.21.15.159:0/1243246907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73000d2d60 msgr2=0x7f73000cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:54.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.595+0000 7f7308600640 1 --2- 172.21.15.159:0/1243246907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73000d2d60 0x7f73000cfb20 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f730008c910 tx=0x7f72f802be40 comp rx=0 tx=0).stop 2024-04-04T06:10:54.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.595+0000 7f7308600640 1 -- 172.21.15.159:0/1243246907 shutdown_connections 2024-04-04T06:10:54.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.595+0000 7f7308600640 1 --2- 172.21.15.159:0/1243246907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73000d2d60 0x7f73000cfb20 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:54.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.595+0000 7f7308600640 1 --2- 172.21.15.159:0/1243246907 >> v2:172.21.15.178:3300/0 conn(0x7f730006ca00 0x7f73000d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:54.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.595+0000 7f7308600640 1 --2- 172.21.15.159:0/1243246907 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f730015e7e0 0x7f73000d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:54.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.595+0000 7f7308600640 1 -- 172.21.15.159:0/1243246907 >> 172.21.15.159:0/1243246907 conn(0x7f730009d7c0 msgr2=0x7f730006bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:54.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.596+0000 7f7308600640 1 -- 172.21.15.159:0/1243246907 shutdown_connections 2024-04-04T06:10:54.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.596+0000 7f7308600640 1 -- 172.21.15.159:0/1243246907 wait complete. 2024-04-04T06:10:54.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.597+0000 7f7308600640 1 Processor -- start 2024-04-04T06:10:54.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.597+0000 7f7308600640 1 -- start start 2024-04-04T06:10:54.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.598+0000 7f7308600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f730006ca00 0x7f7300152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:54.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.598+0000 7f7308600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73000d2d60 0x7f7300157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:54.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.598+0000 7f7308600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f730015e7e0 0x7f73000b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:54.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.598+0000 7f7308600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f730006ce60 con 0x7f73000d2d60 2024-04-04T06:10:54.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.598+0000 7f7308600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73000cefd0 con 0x7f730015e7e0 2024-04-04T06:10:54.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.598+0000 7f7308600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f73000d0060 con 0x7f730006ca00 2024-04-04T06:10:54.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.599+0000 7f7305200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73000d2d60 0x7f7300157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:54.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.599+0000 7f7305200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73000d2d60 0x7f7300157eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42506/0 (socket says 172.21.15.159:42506) 2024-04-04T06:10:54.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.599+0000 7f7305200640 1 -- 172.21.15.159:0/2443063271 learned_addr learned my addr 172.21.15.159:0/2443063271 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:54.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.599+0000 7f7305c00640 1 --2- 172.21.15.159:0/2443063271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f730006ca00 0x7f7300152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:54.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.599+0000 7f7306600640 1 --2- 172.21.15.159:0/2443063271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f730015e7e0 0x7f73000b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:54.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.599+0000 7f7305200640 1 -- 172.21.15.159:0/2443063271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f730006ca00 msgr2=0x7f7300152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:54.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.599+0000 7f7305200640 1 --2- 172.21.15.159:0/2443063271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f730006ca00 0x7f7300152ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:54.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.599+0000 7f7305200640 1 -- 172.21.15.159:0/2443063271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f730015e7e0 msgr2=0x7f73000b7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:54.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.599+0000 7f7305200640 1 --2- 172.21.15.159:0/2443063271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f730015e7e0 0x7f73000b7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:54.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.599+0000 7f7305200640 1 -- 172.21.15.159:0/2443063271 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7300158950 con 0x7f73000d2d60 2024-04-04T06:10:54.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.599+0000 7f7306600640 1 --2- 172.21.15.159:0/2443063271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f730015e7e0 0x7f73000b7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:54.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.599+0000 7f7305200640 1 --2- 172.21.15.159:0/2443063271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73000d2d60 0x7f7300157eb0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00d800 tx=0x7f72ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:54.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.600+0000 7f72fd600640 1 -- 172.21.15.159:0/2443063271 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec00b6c0 con 0x7f73000d2d60 2024-04-04T06:10:54.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.600+0000 7f72fd600640 1 -- 172.21.15.159:0/2443063271 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72ec019070 con 0x7f73000d2d60 2024-04-04T06:10:54.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.600+0000 7f72fd600640 1 -- 172.21.15.159:0/2443063271 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec00ba80 con 0x7f73000d2d60 2024-04-04T06:10:54.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.600+0000 7f7308600640 1 -- 172.21.15.159:0/2443063271 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7300158c30 con 0x7f73000d2d60 2024-04-04T06:10:54.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.600+0000 7f7308600640 1 -- 172.21.15.159:0/2443063271 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73001594d0 con 0x7f73000d2d60 2024-04-04T06:10:54.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.601+0000 7f7308600640 1 -- 172.21.15.159:0/2443063271 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73000685c0 con 0x7f73000d2d60 2024-04-04T06:10:54.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.602+0000 7f72fd600640 1 -- 172.21.15.159:0/2443063271 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72ec00bc20 con 0x7f73000d2d60 2024-04-04T06:10:54.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.603+0000 7f72fd600640 1 --2- 172.21.15.159:0/2443063271 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f72e006c8f0 0x7f72e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:54.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.603+0000 7f72fd600640 1 -- 172.21.15.159:0/2443063271 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f72ec08c0e0 con 0x7f73000d2d60 2024-04-04T06:10:54.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.604+0000 7f7305c00640 1 --2- 172.21.15.159:0/2443063271 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f72e006c8f0 0x7f72e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:54.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.605+0000 7f7305c00640 1 --2- 172.21.15.159:0/2443063271 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f72e006c8f0 0x7f72e006edb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f72f4006fd0 tx=0x7f72f4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:54.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.608+0000 7f72fd600640 1 -- 172.21.15.159:0/2443063271 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f72ec05a470 con 0x7f73000d2d60 2024-04-04T06:10:54.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.809+0000 7f7308600640 1 -- 172.21.15.159:0/2443063271 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73000cb1e0 con 0x7f73000d2d60 2024-04-04T06:10:54.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.810+0000 7f72fd600640 1 -- 172.21.15.159:0/2443063271 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16680 (secure 0 0 0) 0x7f72ec059d70 con 0x7f73000d2d60 2024-04-04T06:10:54.802 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:54.803 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":71,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:10:46.362050+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:10:42.595868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-04T06:10:43.788847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:10:54.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.813+0000 7f7308600640 1 -- 172.21.15.159:0/2443063271 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f72e006c8f0 msgr2=0x7f72e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:54.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.813+0000 7f7308600640 1 --2- 172.21.15.159:0/2443063271 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f72e006c8f0 0x7f72e006edb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f72f4006fd0 tx=0x7f72f4008040 comp rx=0 tx=0).stop 2024-04-04T06:10:54.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.813+0000 7f7308600640 1 -- 172.21.15.159:0/2443063271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73000d2d60 msgr2=0x7f7300157eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:54.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.813+0000 7f7308600640 1 --2- 172.21.15.159:0/2443063271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73000d2d60 0x7f7300157eb0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00d800 tx=0x7f72ec00dcc0 comp rx=0 tx=0).stop 2024-04-04T06:10:54.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.813+0000 7f7308600640 1 -- 172.21.15.159:0/2443063271 shutdown_connections 2024-04-04T06:10:54.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.813+0000 7f7308600640 1 --2- 172.21.15.159:0/2443063271 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f72e006c8f0 0x7f72e006edb0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:54.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.813+0000 7f7308600640 1 --2- 172.21.15.159:0/2443063271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f730015e7e0 0x7f73000b7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:54.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.813+0000 7f7308600640 1 --2- 172.21.15.159:0/2443063271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73000d2d60 0x7f7300157eb0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:54.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.813+0000 7f7308600640 1 --2- 172.21.15.159:0/2443063271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f730006ca00 0x7f7300152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:54.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.813+0000 7f7308600640 1 -- 172.21.15.159:0/2443063271 >> 172.21.15.159:0/2443063271 conn(0x7f730009d7c0 msgr2=0x7f73000b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:54.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.813+0000 7f7308600640 1 -- 172.21.15.159:0/2443063271 shutdown_connections 2024-04-04T06:10:54.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:54.813+0000 7f7308600640 1 -- 172.21.15.159:0/2443063271 wait complete. 2024-04-04T06:10:54.828 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:10:42.595868+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:10:43.788847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:10:54.828 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:10:55.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.049+0000 7fa2e5400640 1 Processor -- start 2024-04-04T06:10:55.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.050+0000 7fa2e5400640 1 -- start start 2024-04-04T06:10:55.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.052+0000 7fa2e5400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa2e0072100 0x7fa2e0160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:55.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.053+0000 7fa2e5400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa2e00cd1c0 0x7fa2e006e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:55.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.053+0000 7fa2e5400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2e00cd740 con 0x7fa2e0145f00 2024-04-04T06:10:55.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.053+0000 7fa2e5400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fa2e00cda20 con 0x7fa2e00cd1c0 2024-04-04T06:10:55.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.053+0000 7fa2e5400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa2e00cdd00 con 0x7fa2e0072100 2024-04-04T06:10:55.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.053+0000 7fa2de800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fa2e0145f00 0x7fa2e00719f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43474/0 (socket says 172.21.15.159:43474) 2024-04-04T06:10:55.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.053+0000 7fa2df200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa2e00cd1c0 0x7fa2e006e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:55.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.053+0000 7fa2de800640 1 -- 172.21.15.159:0/3916419478 learned_addr learned my addr 172.21.15.159:0/3916419478 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:55.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.053+0000 7fa2dde00640 1 --2- 172.21.15.159:0/3916419478 >> v2:172.21.15.178:3300/0 conn(0x7fa2e0072100 0x7fa2e0160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:55.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.054+0000 7fa2df200640 1 -- 172.21.15.159:0/3916419478 >> v2:172.21.15.178:3300/0 conn(0x7fa2e0072100 msgr2=0x7fa2e0160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:55.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.054+0000 7fa2df200640 1 --2- 172.21.15.159:0/3916419478 >> v2:172.21.15.178:3300/0 conn(0x7fa2e0072100 0x7fa2e0160bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.054+0000 7fa2df200640 1 -- 172.21.15.159:0/3916419478 >> v1:172.21.15.159:6789/0 conn(0x7fa2e0145f00 legacy=0x7fa2e00719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:55.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.054+0000 7fa2df200640 1 -- 172.21.15.159:0/3916419478 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e006dab0 con 0x7fa2e00cd1c0 2024-04-04T06:10:55.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.054+0000 7fa2dde00640 1 --2- 172.21.15.159:0/3916419478 >> v2:172.21.15.178:3300/0 conn(0x7fa2e0072100 0x7fa2e0160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:10:55.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.055+0000 7fa2df200640 1 --2- 172.21.15.159:0/3916419478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2e00cd1c0 0x7fa2e006e2b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00705a0 tx=0x7fa2d802be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e27d9eb7c09d757 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:55.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.056+0000 7fa2dd400640 1 -- 172.21.15.159:0/3916419478 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8039070 con 0x7fa2e00cd1c0 2024-04-04T06:10:55.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.056+0000 7fa2dd400640 1 -- 172.21.15.159:0/3916419478 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d802cbe0 con 0x7fa2e00cd1c0 2024-04-04T06:10:55.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.056+0000 7fa2dd400640 1 -- 172.21.15.159:0/3916419478 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8034440 con 0x7fa2e00cd1c0 2024-04-04T06:10:55.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.056+0000 7fa2e5400640 1 -- 172.21.15.159:0/3916419478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2e00cd1c0 msgr2=0x7fa2e006e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:55.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.056+0000 7fa2e5400640 1 --2- 172.21.15.159:0/3916419478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2e00cd1c0 0x7fa2e006e2b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00705a0 tx=0x7fa2d802be30 comp rx=0 tx=0).stop 2024-04-04T06:10:55.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.056+0000 7fa2e5400640 1 -- 172.21.15.159:0/3916419478 shutdown_connections 2024-04-04T06:10:55.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.056+0000 7fa2e5400640 1 --2- 172.21.15.159:0/3916419478 >> v2:172.21.15.178:3300/0 conn(0x7fa2e0072100 0x7fa2e0160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.056+0000 7fa2e5400640 1 --2- 172.21.15.159:0/3916419478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2e00cd1c0 0x7fa2e006e2b0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.056+0000 7fa2e5400640 1 -- 172.21.15.159:0/3916419478 >> 172.21.15.159:0/3916419478 conn(0x7fa2e006bba0 msgr2=0x7fa2e00b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:55.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.056+0000 7fa2e5400640 1 -- 172.21.15.159:0/3916419478 shutdown_connections 2024-04-04T06:10:55.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.057+0000 7fa2e5400640 1 -- 172.21.15.159:0/3916419478 wait complete. 2024-04-04T06:10:55.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.058+0000 7fa2e5400640 1 Processor -- start 2024-04-04T06:10:55.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.058+0000 7fa2e5400640 1 -- start start 2024-04-04T06:10:55.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.059+0000 7fa2e5400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e0072100 0x7fa2e01e70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:55.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.059+0000 7fa2e5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2e00cd1c0 0x7fa2e01335f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:55.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.059+0000 7fa2e5400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2e0145f00 0x7fa2e0133b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:55.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.059+0000 7fa2e5400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e0146460 con 0x7fa2e00cd1c0 2024-04-04T06:10:55.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.059+0000 7fa2e5400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e01462e0 con 0x7fa2e0072100 2024-04-04T06:10:55.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.059+0000 7fa2e5400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e006e7f0 con 0x7fa2e0145f00 2024-04-04T06:10:55.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.060+0000 7fa2dde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2e00cd1c0 0x7fa2e01335f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:55.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.060+0000 7fa2dde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2e00cd1c0 0x7fa2e01335f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42532/0 (socket says 172.21.15.159:42532) 2024-04-04T06:10:55.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.060+0000 7fa2dde00640 1 -- 172.21.15.159:0/3130666056 learned_addr learned my addr 172.21.15.159:0/3130666056 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:55.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.060+0000 7fa2de800640 1 --2- 172.21.15.159:0/3130666056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e0072100 0x7fa2e01e70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:55.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.060+0000 7fa2dde00640 1 -- 172.21.15.159:0/3130666056 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2e0145f00 msgr2=0x7fa2e0133b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:55.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.060+0000 7fa2df200640 1 --2- 172.21.15.159:0/3130666056 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2e0145f00 0x7fa2e0133b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:55.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.060+0000 7fa2dde00640 1 --2- 172.21.15.159:0/3130666056 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2e0145f00 0x7fa2e0133b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.060+0000 7fa2dde00640 1 -- 172.21.15.159:0/3130666056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e0072100 msgr2=0x7fa2e01e70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:55.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.060+0000 7fa2dde00640 1 --2- 172.21.15.159:0/3130666056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e0072100 0x7fa2e01e70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.060+0000 7fa2dde00640 1 -- 172.21.15.159:0/3130666056 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e0132550 con 0x7fa2e00cd1c0 2024-04-04T06:10:55.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.061+0000 7fa2dde00640 1 --2- 172.21.15.159:0/3130666056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2e00cd1c0 0x7fa2e01335f0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00e730 tx=0x7fa2cc00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:55.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.061+0000 7fa2d6a00640 1 -- 172.21.15.159:0/3130666056 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00f040 con 0x7fa2e00cd1c0 2024-04-04T06:10:55.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.061+0000 7fa2d6a00640 1 -- 172.21.15.159:0/3130666056 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2cc004230 con 0x7fa2e00cd1c0 2024-04-04T06:10:55.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.061+0000 7fa2d6a00640 1 -- 172.21.15.159:0/3130666056 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00ee40 con 0x7fa2e00cd1c0 2024-04-04T06:10:55.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.061+0000 7fa2e5400640 1 -- 172.21.15.159:0/3130666056 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e0132830 con 0x7fa2e00cd1c0 2024-04-04T06:10:55.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.061+0000 7fa2e5400640 1 -- 172.21.15.159:0/3130666056 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e0132df0 con 0x7fa2e00cd1c0 2024-04-04T06:10:55.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.062+0000 7fa2e5400640 1 -- 172.21.15.159:0/3130666056 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e00685c0 con 0x7fa2e00cd1c0 2024-04-04T06:10:55.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.063+0000 7fa2d6a00640 1 -- 172.21.15.159:0/3130666056 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2cc00c670 con 0x7fa2e00cd1c0 2024-04-04T06:10:55.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.064+0000 7fa2d6a00640 1 --2- 172.21.15.159:0/3130666056 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa2c006c8f0 0x7fa2c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:55.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.064+0000 7fa2d6a00640 1 -- 172.21.15.159:0/3130666056 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fa2cc08b8e0 con 0x7fa2e00cd1c0 2024-04-04T06:10:55.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.065+0000 7fa2de800640 1 --2- 172.21.15.159:0/3130666056 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa2c006c8f0 0x7fa2c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:55.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.066+0000 7fa2de800640 1 --2- 172.21.15.159:0/3130666056 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa2c006c8f0 0x7fa2c006edb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0000f60 tx=0x7fa2d0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:55.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.069+0000 7fa2d6a00640 1 -- 172.21.15.159:0/3130666056 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa2cc017020 con 0x7fa2e00cd1c0 2024-04-04T06:10:55.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.269+0000 7fa2e5400640 1 -- 172.21.15.159:0/3130666056 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa2e00b2260 con 0x7fa2e00cd1c0 2024-04-04T06:10:55.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.270+0000 7fa2d6a00640 1 -- 172.21.15.159:0/3130666056 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7fa2cc059c70 con 0x7fa2e00cd1c0 2024-04-04T06:10:55.262 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:10:55.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.273+0000 7fa2e5400640 1 -- 172.21.15.159:0/3130666056 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa2c006c8f0 msgr2=0x7fa2c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:55.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.273+0000 7fa2e5400640 1 --2- 172.21.15.159:0/3130666056 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa2c006c8f0 0x7fa2c006edb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0000f60 tx=0x7fa2d0009040 comp rx=0 tx=0).stop 2024-04-04T06:10:55.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.273+0000 7fa2e5400640 1 -- 172.21.15.159:0/3130666056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2e00cd1c0 msgr2=0x7fa2e01335f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:55.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.273+0000 7fa2e5400640 1 --2- 172.21.15.159:0/3130666056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2e00cd1c0 0x7fa2e01335f0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00e730 tx=0x7fa2cc00ebf0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.273+0000 7fa2e5400640 1 -- 172.21.15.159:0/3130666056 shutdown_connections 2024-04-04T06:10:55.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.273+0000 7fa2e5400640 1 --2- 172.21.15.159:0/3130666056 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2e0145f00 0x7fa2e0133b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.273+0000 7fa2e5400640 1 --2- 172.21.15.159:0/3130666056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2e00cd1c0 0x7fa2e01335f0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.273+0000 7fa2e5400640 1 --2- 172.21.15.159:0/3130666056 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa2c006c8f0 0x7fa2c006edb0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.273+0000 7fa2e5400640 1 --2- 172.21.15.159:0/3130666056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e0072100 0x7fa2e01e70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.273+0000 7fa2e5400640 1 -- 172.21.15.159:0/3130666056 >> 172.21.15.159:0/3130666056 conn(0x7fa2e006bba0 msgr2=0x7fa2e015fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:55.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.273+0000 7fa2e5400640 1 -- 172.21.15.159:0/3130666056 shutdown_connections 2024-04-04T06:10:55.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.273+0000 7fa2e5400640 1 -- 172.21.15.159:0/3130666056 wait complete. 2024-04-04T06:10:55.283 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:10:55.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.494+0000 7f91e6800640 1 Processor -- start 2024-04-04T06:10:55.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.494+0000 7f91e6800640 1 -- start start 2024-04-04T06:10:55.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.496+0000 7f91e6800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91e0070120 0x7f91e006b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:55.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.496+0000 7f91e6800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f91e006e770 0x7f91e006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:55.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.496+0000 7f91e6800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91e00b5b10 con 0x7f91e00b0670 2024-04-04T06:10:55.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.496+0000 7f91e6800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f91e00b63e0 con 0x7f91e006e770 2024-04-04T06:10:55.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.496+0000 7f91e6800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f91e006c3f0 con 0x7f91e0070120 2024-04-04T06:10:55.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.496+0000 7f91dfe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f91e006e770 0x7f91e006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:55.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.496+0000 7f91dfe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f91e006e770 0x7f91e006bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42536/0 (socket says 172.21.15.159:42536) 2024-04-04T06:10:55.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.496+0000 7f91dfe00640 1 -- 172.21.15.159:0/2587991096 learned_addr learned my addr 172.21.15.159:0/2587991096 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:55.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.496+0000 7f91dfe00640 1 -- 172.21.15.159:0/2587991096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91e0070120 msgr2=0x7f91e006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:55.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.496+0000 7f91dfe00640 1 --2- 172.21.15.159:0/2587991096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91e0070120 0x7f91e006b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.496+0000 7f91dfe00640 1 -- 172.21.15.159:0/2587991096 >> v1:172.21.15.178:6789/0 conn(0x7f91e00b0670 legacy=0x7f91e015f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:55.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.496+0000 7f91dfe00640 1 -- 172.21.15.159:0/2587991096 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e00705e0 con 0x7f91e006e770 2024-04-04T06:10:55.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.497+0000 7f91dfe00640 1 --2- 172.21.15.159:0/2587991096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91e006e770 0x7f91e006bab0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009e80 tx=0x7f91d802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764d53cd443e237f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:55.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.498+0000 7f91de000640 1 -- 172.21.15.159:0/2587991096 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d802ca50 con 0x7f91e006e770 2024-04-04T06:10:55.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.498+0000 7f91de000640 1 -- 172.21.15.159:0/2587991096 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d802cbf0 con 0x7f91e006e770 2024-04-04T06:10:55.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.498+0000 7f91de000640 1 -- 172.21.15.159:0/2587991096 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d80343f0 con 0x7f91e006e770 2024-04-04T06:10:55.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.498+0000 7f91e6800640 1 -- 172.21.15.159:0/2587991096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91e006e770 msgr2=0x7f91e006bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:55.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.498+0000 7f91e6800640 1 --2- 172.21.15.159:0/2587991096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91e006e770 0x7f91e006bab0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009e80 tx=0x7f91d802beb0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.499+0000 7f91e6800640 1 -- 172.21.15.159:0/2587991096 shutdown_connections 2024-04-04T06:10:55.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.499+0000 7f91e6800640 1 --2- 172.21.15.159:0/2587991096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91e006e770 0x7f91e006bab0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.499+0000 7f91e6800640 1 --2- 172.21.15.159:0/2587991096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91e0070120 0x7f91e006b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.499+0000 7f91e6800640 1 -- 172.21.15.159:0/2587991096 >> 172.21.15.159:0/2587991096 conn(0x7f91e00b65a0 msgr2=0x7f91e00b69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:55.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.499+0000 7f91e6800640 1 -- 172.21.15.159:0/2587991096 shutdown_connections 2024-04-04T06:10:55.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.499+0000 7f91e6800640 1 -- 172.21.15.159:0/2587991096 wait complete. 2024-04-04T06:10:55.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.501+0000 7f91e6800640 1 Processor -- start 2024-04-04T06:10:55.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.501+0000 7f91e6800640 1 -- start start 2024-04-04T06:10:55.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.503+0000 7f91e6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91e006e770 0x7f91e01f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:55.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.503+0000 7f91e6800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91e0070120 0x7f91e00cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:55.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.503+0000 7f91e6800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91e00b0670 0x7f91e0155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:55.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.503+0000 7f91e6800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f91e00b0a50 con 0x7f91e006e770 2024-04-04T06:10:55.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.503+0000 7f91e6800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f91e00b6070 con 0x7f91e0070120 2024-04-04T06:10:55.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.503+0000 7f91e6800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f91e0168450 con 0x7f91e00b0670 2024-04-04T06:10:55.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.503+0000 7f91df400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91e006e770 0x7f91e01f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:55.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.503+0000 7f91df400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91e006e770 0x7f91e01f7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42548/0 (socket says 172.21.15.159:42548) 2024-04-04T06:10:55.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.503+0000 7f91df400640 1 -- 172.21.15.159:0/3429267978 learned_addr learned my addr 172.21.15.159:0/3429267978 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:55.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.504+0000 7f91dea00640 1 --2- 172.21.15.159:0/3429267978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91e0070120 0x7f91e00cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:55.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.504+0000 7f91dfe00640 1 --2- 172.21.15.159:0/3429267978 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91e00b0670 0x7f91e0155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:55.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.504+0000 7f91df400640 1 -- 172.21.15.159:0/3429267978 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91e00b0670 msgr2=0x7f91e0155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:55.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.504+0000 7f91df400640 1 --2- 172.21.15.159:0/3429267978 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91e00b0670 0x7f91e0155910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.504+0000 7f91df400640 1 -- 172.21.15.159:0/3429267978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91e0070120 msgr2=0x7f91e00cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:55.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.504+0000 7f91df400640 1 --2- 172.21.15.159:0/3429267978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91e0070120 0x7f91e00cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.504+0000 7f91df400640 1 -- 172.21.15.159:0/3429267978 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e0151a20 con 0x7f91e006e770 2024-04-04T06:10:55.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.504+0000 7f91dea00640 1 --2- 172.21.15.159:0/3429267978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91e0070120 0x7f91e00cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:55.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.504+0000 7f91df400640 1 --2- 172.21.15.159:0/3429267978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91e006e770 0x7f91e01f7bc0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f91d000b3e0 tx=0x7f91d000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:55.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.504+0000 7f91d7400640 1 -- 172.21.15.159:0/3429267978 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0015020 con 0x7f91e006e770 2024-04-04T06:10:55.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.504+0000 7f91d7400640 1 -- 172.21.15.159:0/3429267978 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d0011850 con 0x7f91e006e770 2024-04-04T06:10:55.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.504+0000 7f91d7400640 1 -- 172.21.15.159:0/3429267978 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0011c40 con 0x7f91e006e770 2024-04-04T06:10:55.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.504+0000 7f91e6800640 1 -- 172.21.15.159:0/3429267978 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e01f8410 con 0x7f91e006e770 2024-04-04T06:10:55.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.505+0000 7f91e6800640 1 -- 172.21.15.159:0/3429267978 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e01f8100 con 0x7f91e006e770 2024-04-04T06:10:55.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.506+0000 7f91e6800640 1 -- 172.21.15.159:0/3429267978 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e0068cd0 con 0x7f91e006e770 2024-04-04T06:10:55.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.507+0000 7f91d7400640 1 -- 172.21.15.159:0/3429267978 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91d0018020 con 0x7f91e006e770 2024-04-04T06:10:55.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.508+0000 7f91d7400640 1 --2- 172.21.15.159:0/3429267978 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f91c006c8f0 0x7f91c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:55.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.508+0000 7f91d7400640 1 -- 172.21.15.159:0/3429267978 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f91d008bb20 con 0x7f91e006e770 2024-04-04T06:10:55.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.509+0000 7f91dea00640 1 --2- 172.21.15.159:0/3429267978 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f91c006c8f0 0x7f91c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:55.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.510+0000 7f91dea00640 1 --2- 172.21.15.159:0/3429267978 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f91c006c8f0 0x7f91c006edb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f91cc006fd0 tx=0x7f91cc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:55.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.512+0000 7f91d7400640 1 -- 172.21.15.159:0/3429267978 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f91d001f020 con 0x7f91e006e770 2024-04-04T06:10:55.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.710+0000 7f91e6800640 1 -- 172.21.15.159:0/3429267978 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f91e00b30a0 con 0x7f91e006e770 2024-04-04T06:10:55.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.711+0000 7f91d7400640 1 -- 172.21.15.159:0/3429267978 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) ==== 104+0+11 (secure 0 0 0) 0x7f91d0059f40 con 0x7f91e006e770 2024-04-04T06:10:55.703 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:10:55.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.714+0000 7f91e6800640 1 -- 172.21.15.159:0/3429267978 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f91c006c8f0 msgr2=0x7f91c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:55.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.714+0000 7f91e6800640 1 --2- 172.21.15.159:0/3429267978 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f91c006c8f0 0x7f91c006edb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f91cc006fd0 tx=0x7f91cc008040 comp rx=0 tx=0).stop 2024-04-04T06:10:55.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.714+0000 7f91e6800640 1 -- 172.21.15.159:0/3429267978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91e006e770 msgr2=0x7f91e01f7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:55.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.714+0000 7f91e6800640 1 --2- 172.21.15.159:0/3429267978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91e006e770 0x7f91e01f7bc0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f91d000b3e0 tx=0x7f91d000b8a0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.714+0000 7f91e6800640 1 -- 172.21.15.159:0/3429267978 shutdown_connections 2024-04-04T06:10:55.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.714+0000 7f91e6800640 1 --2- 172.21.15.159:0/3429267978 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91e00b0670 0x7f91e0155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.714+0000 7f91e6800640 1 --2- 172.21.15.159:0/3429267978 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f91c006c8f0 0x7f91c006edb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.714+0000 7f91e6800640 1 --2- 172.21.15.159:0/3429267978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91e0070120 0x7f91e00cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.714+0000 7f91e6800640 1 --2- 172.21.15.159:0/3429267978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91e006e770 0x7f91e01f7bc0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.714+0000 7f91e6800640 1 -- 172.21.15.159:0/3429267978 >> 172.21.15.159:0/3429267978 conn(0x7f91e00b65a0 msgr2=0x7f91e00af0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:55.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.714+0000 7f91e6800640 1 -- 172.21.15.159:0/3429267978 shutdown_connections 2024-04-04T06:10:55.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.714+0000 7f91e6800640 1 -- 172.21.15.159:0/3429267978 wait complete. 2024-04-04T06:10:55.726 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:10:55.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.931+0000 7f5dfc000640 1 Processor -- start 2024-04-04T06:10:55.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.931+0000 7f5dfc000640 1 -- start start 2024-04-04T06:10:55.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.932+0000 7f5dfc000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5df40d0280 0x7f5df40d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:55.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.932+0000 7f5dfc000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5df4141c10 0x7f5df4071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:55.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.933+0000 7f5dfc000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df415ec80 con 0x7f5df4071eb0 2024-04-04T06:10:55.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.933+0000 7f5dfc000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f5df40d0ba0 con 0x7f5df4141c10 2024-04-04T06:10:55.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.933+0000 7f5dfc000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5df415ef20 con 0x7f5df40d0280 2024-04-04T06:10:55.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.933+0000 7f5df8c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5df4141c10 0x7f5df4071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:55.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.933+0000 7f5df8c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5df4141c10 0x7f5df4071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42556/0 (socket says 172.21.15.159:42556) 2024-04-04T06:10:55.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.933+0000 7f5df8c00640 1 -- 172.21.15.159:0/768105032 learned_addr learned my addr 172.21.15.159:0/768105032 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:55.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.933+0000 7f5df9600640 1 --2- 172.21.15.159:0/768105032 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5df40d0280 0x7f5df40d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:55.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.933+0000 7f5df8c00640 1 -- 172.21.15.159:0/768105032 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5df40d0280 msgr2=0x7f5df40d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:55.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.933+0000 7f5df8c00640 1 --2- 172.21.15.159:0/768105032 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5df40d0280 0x7f5df40d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.933+0000 7f5df8c00640 1 -- 172.21.15.159:0/768105032 >> v1:172.21.15.178:6789/0 conn(0x7f5df4071eb0 legacy=0x7f5df406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:55.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.933+0000 7f5df8c00640 1 -- 172.21.15.159:0/768105032 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df415f200 con 0x7f5df4141c10 2024-04-04T06:10:55.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.934+0000 7f5df8c00640 1 --2- 172.21.15.159:0/768105032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df4141c10 0x7f5df4071970 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009870 tx=0x7f5de002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5bcbd3d7d82d09 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:55.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.935+0000 7f5df3e00640 1 -- 172.21.15.159:0/768105032 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0035020 con 0x7f5df4141c10 2024-04-04T06:10:55.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.935+0000 7f5df3e00640 1 -- 172.21.15.159:0/768105032 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de0031640 con 0x7f5df4141c10 2024-04-04T06:10:55.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.935+0000 7f5df3e00640 1 -- 172.21.15.159:0/768105032 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de002d040 con 0x7f5df4141c10 2024-04-04T06:10:55.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.935+0000 7f5dfc000640 1 -- 172.21.15.159:0/768105032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df4141c10 msgr2=0x7f5df4071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:55.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.935+0000 7f5dfc000640 1 --2- 172.21.15.159:0/768105032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df4141c10 0x7f5df4071970 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009870 tx=0x7f5de002b8f0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.936+0000 7f5dfc000640 1 -- 172.21.15.159:0/768105032 shutdown_connections 2024-04-04T06:10:55.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.936+0000 7f5dfc000640 1 --2- 172.21.15.159:0/768105032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df4141c10 0x7f5df4071970 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.936+0000 7f5dfc000640 1 --2- 172.21.15.159:0/768105032 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5df40d0280 0x7f5df40d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.936+0000 7f5dfc000640 1 -- 172.21.15.159:0/768105032 >> 172.21.15.159:0/768105032 conn(0x7f5df406b750 msgr2=0x7f5df406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:55.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.936+0000 7f5dfc000640 1 -- 172.21.15.159:0/768105032 shutdown_connections 2024-04-04T06:10:55.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.936+0000 7f5dfc000640 1 -- 172.21.15.159:0/768105032 wait complete. 2024-04-04T06:10:55.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.937+0000 7f5dfc000640 1 Processor -- start 2024-04-04T06:10:55.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.937+0000 7f5dfc000640 1 -- start start 2024-04-04T06:10:55.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.938+0000 7f5dfc000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df4071eb0 0x7f5df414f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:55.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.938+0000 7f5dfc000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5df40d0280 0x7f5df4099640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:55.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.938+0000 7f5dfc000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5df4141c10 0x7f5df40ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:55.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.938+0000 7f5dfc000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5df4158e90 con 0x7f5df4071eb0 2024-04-04T06:10:55.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.938+0000 7f5dfc000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5df415f7c0 con 0x7f5df40d0280 2024-04-04T06:10:55.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.938+0000 7f5dfc000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5df414ff30 con 0x7f5df4141c10 2024-04-04T06:10:55.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.939+0000 7f5df9600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df4071eb0 0x7f5df414f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:55.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.939+0000 7f5df9600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df4071eb0 0x7f5df414f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42568/0 (socket says 172.21.15.159:42568) 2024-04-04T06:10:55.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.939+0000 7f5df9600640 1 -- 172.21.15.159:0/2971126109 learned_addr learned my addr 172.21.15.159:0/2971126109 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:55.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.939+0000 7f5df9600640 1 -- 172.21.15.159:0/2971126109 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5df4141c10 msgr2=0x7f5df40ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:55.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.939+0000 7f5df9600640 1 --2- 172.21.15.159:0/2971126109 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5df4141c10 0x7f5df40ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.939+0000 7f5df9600640 1 -- 172.21.15.159:0/2971126109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5df40d0280 msgr2=0x7f5df4099640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:55.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.939+0000 7f5df9600640 1 --2- 172.21.15.159:0/2971126109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5df40d0280 0x7f5df4099640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:55.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.939+0000 7f5df9600640 1 -- 172.21.15.159:0/2971126109 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df40af0c0 con 0x7f5df4071eb0 2024-04-04T06:10:55.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.939+0000 7f5df9600640 1 --2- 172.21.15.159:0/2971126109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df4071eb0 0x7f5df414f9f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f5de800c820 tx=0x7f5de800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:55.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.940+0000 7f5df1600640 1 -- 172.21.15.159:0/2971126109 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8015070 con 0x7f5df4071eb0 2024-04-04T06:10:55.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.940+0000 7f5df1600640 1 -- 172.21.15.159:0/2971126109 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de8004220 con 0x7f5df4071eb0 2024-04-04T06:10:55.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.940+0000 7f5df1600640 1 -- 172.21.15.159:0/2971126109 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de800ce10 con 0x7f5df4071eb0 2024-04-04T06:10:55.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.940+0000 7f5dfc000640 1 -- 172.21.15.159:0/2971126109 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df414ef10 con 0x7f5df4071eb0 2024-04-04T06:10:55.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.940+0000 7f5dfc000640 1 -- 172.21.15.159:0/2971126109 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df40b0eb0 con 0x7f5df4071eb0 2024-04-04T06:10:55.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.941+0000 7f5dfc000640 1 -- 172.21.15.159:0/2971126109 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df40685c0 con 0x7f5df4071eb0 2024-04-04T06:10:55.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.942+0000 7f5df1600640 1 -- 172.21.15.159:0/2971126109 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5de8007830 con 0x7f5df4071eb0 2024-04-04T06:10:55.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.943+0000 7f5df1600640 1 --2- 172.21.15.159:0/2971126109 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5dd406c8f0 0x7f5dd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:55.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.943+0000 7f5df1600640 1 -- 172.21.15.159:0/2971126109 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f5de808ba70 con 0x7f5df4071eb0 2024-04-04T06:10:55.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.943+0000 7f5df8c00640 1 --2- 172.21.15.159:0/2971126109 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5dd406c8f0 0x7f5dd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:55.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.944+0000 7f5df8c00640 1 --2- 172.21.15.159:0/2971126109 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5dd406c8f0 0x7f5dd406edb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5de0036000 tx=0x7f5de0030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:55.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:55.947+0000 7f5df1600640 1 -- 172.21.15.159:0/2971126109 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5de8059e90 con 0x7f5df4071eb0 2024-04-04T06:10:56.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.150+0000 7f5dfc000640 1 -- 172.21.15.159:0/2971126109 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5df40cfdb0 con 0x7f5df4071eb0 2024-04-04T06:10:56.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.151+0000 7f5df1600640 1 -- 172.21.15.159:0/2971126109 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) ==== 100+0+11 (secure 0 0 0) 0x7f5de8059790 con 0x7f5df4071eb0 2024-04-04T06:10:56.143 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:10:56.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.154+0000 7f5dfc000640 1 -- 172.21.15.159:0/2971126109 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5dd406c8f0 msgr2=0x7f5dd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:56.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.154+0000 7f5dfc000640 1 --2- 172.21.15.159:0/2971126109 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5dd406c8f0 0x7f5dd406edb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5de0036000 tx=0x7f5de0030000 comp rx=0 tx=0).stop 2024-04-04T06:10:56.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.154+0000 7f5dfc000640 1 -- 172.21.15.159:0/2971126109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df4071eb0 msgr2=0x7f5df414f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:56.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.154+0000 7f5dfc000640 1 --2- 172.21.15.159:0/2971126109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df4071eb0 0x7f5df414f9f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f5de800c820 tx=0x7f5de800cce0 comp rx=0 tx=0).stop 2024-04-04T06:10:56.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.154+0000 7f5dfc000640 1 -- 172.21.15.159:0/2971126109 shutdown_connections 2024-04-04T06:10:56.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.154+0000 7f5dfc000640 1 --2- 172.21.15.159:0/2971126109 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5df4141c10 0x7f5df40ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:56.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.154+0000 7f5dfc000640 1 --2- 172.21.15.159:0/2971126109 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5dd406c8f0 0x7f5dd406edb0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:56.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.154+0000 7f5dfc000640 1 --2- 172.21.15.159:0/2971126109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5df40d0280 0x7f5df4099640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:56.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.154+0000 7f5dfc000640 1 --2- 172.21.15.159:0/2971126109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5df4071eb0 0x7f5df414f9f0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:56.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.154+0000 7f5dfc000640 1 -- 172.21.15.159:0/2971126109 >> 172.21.15.159:0/2971126109 conn(0x7f5df406b750 msgr2=0x7f5df4144220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:56.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.154+0000 7f5dfc000640 1 -- 172.21.15.159:0/2971126109 shutdown_connections 2024-04-04T06:10:56.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.154+0000 7f5dfc000640 1 -- 172.21.15.159:0/2971126109 wait complete. 2024-04-04T06:10:56.167 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-04T06:10:56.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.367+0000 7f7a5e600640 1 Processor -- start 2024-04-04T06:10:56.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.367+0000 7f7a5e600640 1 -- start start 2024-04-04T06:10:56.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.368+0000 7f7a5e600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7a580cc0b0 0x7f7a5806dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:56.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.368+0000 7f7a5e600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7a5815fc50 0x7f7a580cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:56.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.368+0000 7f7a5e600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a5806e4f0 con 0x7f7a580cc490 2024-04-04T06:10:56.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.368+0000 7f7a5e600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f7a5815c610 con 0x7f7a5815fc50 2024-04-04T06:10:56.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.368+0000 7f7a5e600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7a580b3150 con 0x7f7a580cc0b0 2024-04-04T06:10:56.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.368+0000 7f7a4fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7a5815fc50 0x7f7a580cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:56.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.368+0000 7f7a4fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7a5815fc50 0x7f7a580cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42570/0 (socket says 172.21.15.159:42570) 2024-04-04T06:10:56.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.368+0000 7f7a4fe00640 1 -- 172.21.15.159:0/2706606950 learned_addr learned my addr 172.21.15.159:0/2706606950 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:56.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.369+0000 7f7a4fe00640 1 -- 172.21.15.159:0/2706606950 >> v2:172.21.15.178:3300/0 conn(0x7f7a580cc0b0 msgr2=0x7f7a5806dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:56.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.369+0000 7f7a4fe00640 1 --2- 172.21.15.159:0/2706606950 >> v2:172.21.15.178:3300/0 conn(0x7f7a580cc0b0 0x7f7a5806dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:56.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.369+0000 7f7a4fe00640 1 -- 172.21.15.159:0/2706606950 >> v1:172.21.15.178:6789/0 conn(0x7f7a580cc490 legacy=0x7f7a5815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:56.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.369+0000 7f7a4fe00640 1 -- 172.21.15.159:0/2706606950 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a581600e0 con 0x7f7a5815fc50 2024-04-04T06:10:56.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.370+0000 7f7a4fe00640 1 --2- 172.21.15.159:0/2706606950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a5815fc50 0x7f7a580cfdc0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f7a580cfd10 tx=0x7f7a4002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7649a8bde2a5c280 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:56.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.370+0000 7f7a57e00640 1 -- 172.21.15.159:0/2706606950 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40035020 con 0x7f7a5815fc50 2024-04-04T06:10:56.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.370+0000 7f7a57e00640 1 -- 172.21.15.159:0/2706606950 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a40031870 con 0x7f7a5815fc50 2024-04-04T06:10:56.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.370+0000 7f7a57e00640 1 -- 172.21.15.159:0/2706606950 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40031c40 con 0x7f7a5815fc50 2024-04-04T06:10:56.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.371+0000 7f7a5e600640 1 -- 172.21.15.159:0/2706606950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a5815fc50 msgr2=0x7f7a580cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:56.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.371+0000 7f7a5e600640 1 --2- 172.21.15.159:0/2706606950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a5815fc50 0x7f7a580cfdc0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f7a580cfd10 tx=0x7f7a4002b910 comp rx=0 tx=0).stop 2024-04-04T06:10:56.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.371+0000 7f7a5e600640 1 -- 172.21.15.159:0/2706606950 shutdown_connections 2024-04-04T06:10:56.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.371+0000 7f7a5e600640 1 --2- 172.21.15.159:0/2706606950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a5815fc50 0x7f7a580cfdc0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:56.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.371+0000 7f7a5e600640 1 --2- 172.21.15.159:0/2706606950 >> v2:172.21.15.178:3300/0 conn(0x7f7a580cc0b0 0x7f7a5806dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:56.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.371+0000 7f7a5e600640 1 -- 172.21.15.159:0/2706606950 >> 172.21.15.159:0/2706606950 conn(0x7f7a580ccad0 msgr2=0x7f7a580cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:56.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.371+0000 7f7a5e600640 1 -- 172.21.15.159:0/2706606950 shutdown_connections 2024-04-04T06:10:56.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.371+0000 7f7a5e600640 1 -- 172.21.15.159:0/2706606950 wait complete. 2024-04-04T06:10:56.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.373+0000 7f7a5e600640 1 Processor -- start 2024-04-04T06:10:56.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.373+0000 7f7a5e600640 1 -- start start 2024-04-04T06:10:56.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.374+0000 7f7a5e600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a580cc0b0 0x7f7a581f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:56.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.374+0000 7f7a5e600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a580cc490 0x7f7a580ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:56.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.374+0000 7f7a5e600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a581532a0 0x7f7a5815b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:56.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.374+0000 7f7a5e600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7a58160980 con 0x7f7a580cc0b0 2024-04-04T06:10:56.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.374+0000 7f7a5e600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7a581606a0 con 0x7f7a581532a0 2024-04-04T06:10:56.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.374+0000 7f7a5e600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7a580d0300 con 0x7f7a580cc490 2024-04-04T06:10:56.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.374+0000 7f7a5d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a580cc0b0 0x7f7a581f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:56.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.374+0000 7f7a5d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a580cc0b0 0x7f7a581f3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42576/0 (socket says 172.21.15.159:42576) 2024-04-04T06:10:56.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.374+0000 7f7a5d200640 1 -- 172.21.15.159:0/4148061552 learned_addr learned my addr 172.21.15.159:0/4148061552 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.375+0000 7f7a4fe00640 1 --2- 172.21.15.159:0/4148061552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a580cc490 0x7f7a580ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.375+0000 7f7a5dc00640 1 --2- 172.21.15.159:0/4148061552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a581532a0 0x7f7a5815b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.375+0000 7f7a5d200640 1 -- 172.21.15.159:0/4148061552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a580cc490 msgr2=0x7f7a580ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.375+0000 7f7a5d200640 1 --2- 172.21.15.159:0/4148061552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a580cc490 0x7f7a580ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.375+0000 7f7a5d200640 1 -- 172.21.15.159:0/4148061552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a581532a0 msgr2=0x7f7a5815b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.375+0000 7f7a5d200640 1 --2- 172.21.15.159:0/4148061552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a581532a0 0x7f7a5815b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.375+0000 7f7a5d200640 1 -- 172.21.15.159:0/4148061552 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a580ceeb0 con 0x7f7a580cc0b0 2024-04-04T06:10:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.375+0000 7f7a5d200640 1 --2- 172.21.15.159:0/4148061552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a580cc0b0 0x7f7a581f3e10 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800b4a0 tx=0x7f7a4800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.375+0000 7f7a55600640 1 -- 172.21.15.159:0/4148061552 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48014070 con 0x7f7a580cc0b0 2024-04-04T06:10:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.375+0000 7f7a55600640 1 -- 172.21.15.159:0/4148061552 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a48010610 con 0x7f7a580cc0b0 2024-04-04T06:10:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.375+0000 7f7a5e600640 1 -- 172.21.15.159:0/4148061552 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a580cf190 con 0x7f7a580cc0b0 2024-04-04T06:10:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.375+0000 7f7a55600640 1 -- 172.21.15.159:0/4148061552 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48010a00 con 0x7f7a580cc0b0 2024-04-04T06:10:56.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.376+0000 7f7a5e600640 1 -- 172.21.15.159:0/4148061552 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a58154b50 con 0x7f7a580cc0b0 2024-04-04T06:10:56.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.377+0000 7f7a5e600640 1 -- 172.21.15.159:0/4148061552 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a58075940 con 0x7f7a580cc0b0 2024-04-04T06:10:56.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.378+0000 7f7a55600640 1 -- 172.21.15.159:0/4148061552 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a48017070 con 0x7f7a580cc0b0 2024-04-04T06:10:56.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.379+0000 7f7a55600640 1 --2- 172.21.15.159:0/4148061552 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a3406c8f0 0x7f7a3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:56.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.379+0000 7f7a55600640 1 -- 172.21.15.159:0/4148061552 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f7a4808b5f0 con 0x7f7a580cc0b0 2024-04-04T06:10:56.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.380+0000 7f7a4fe00640 1 --2- 172.21.15.159:0/4148061552 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a3406c8f0 0x7f7a3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:56.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.381+0000 7f7a4fe00640 1 --2- 172.21.15.159:0/4148061552 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a3406c8f0 0x7f7a3406edb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7a40036000 tx=0x7f7a4003c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:56.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.384+0000 7f7a55600640 1 -- 172.21.15.159:0/4148061552 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7a48059a20 con 0x7f7a580cc0b0 2024-04-04T06:10:56.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.597+0000 7f7a5e600640 1 -- 172.21.15.159:0/4148061552 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7a581f4350 con 0x7f7a580cc0b0 2024-04-04T06:10:56.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.597+0000 7f7a55600640 1 -- 172.21.15.159:0/4148061552 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) ==== 80+0+0 (secure 0 0 0) 0x7f7a48059320 con 0x7f7a580cc0b0 2024-04-04T06:10:56.590 INFO:teuthology.orchestra.run.smithi159.stderr:listed 0 entries 2024-04-04T06:10:56.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.600+0000 7f7a5e600640 1 -- 172.21.15.159:0/4148061552 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a3406c8f0 msgr2=0x7f7a3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:56.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.600+0000 7f7a5e600640 1 --2- 172.21.15.159:0/4148061552 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a3406c8f0 0x7f7a3406edb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7a40036000 tx=0x7f7a4003c040 comp rx=0 tx=0).stop 2024-04-04T06:10:56.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.600+0000 7f7a5e600640 1 -- 172.21.15.159:0/4148061552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a580cc0b0 msgr2=0x7f7a581f3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:56.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.600+0000 7f7a5e600640 1 --2- 172.21.15.159:0/4148061552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a580cc0b0 0x7f7a581f3e10 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800b4a0 tx=0x7f7a4800b960 comp rx=0 tx=0).stop 2024-04-04T06:10:56.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.600+0000 7f7a5e600640 1 -- 172.21.15.159:0/4148061552 shutdown_connections 2024-04-04T06:10:56.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.600+0000 7f7a5e600640 1 --2- 172.21.15.159:0/4148061552 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a3406c8f0 0x7f7a3406edb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:56.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.600+0000 7f7a5e600640 1 --2- 172.21.15.159:0/4148061552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a581532a0 0x7f7a5815b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:56.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.600+0000 7f7a5e600640 1 --2- 172.21.15.159:0/4148061552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a580cc490 0x7f7a580ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:56.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.600+0000 7f7a5e600640 1 --2- 172.21.15.159:0/4148061552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a580cc0b0 0x7f7a581f3e10 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:56.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.600+0000 7f7a5e600640 1 -- 172.21.15.159:0/4148061552 >> 172.21.15.159:0/4148061552 conn(0x7f7a580ccad0 msgr2=0x7f7a580b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:56.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.600+0000 7f7a5e600640 1 -- 172.21.15.159:0/4148061552 shutdown_connections 2024-04-04T06:10:56.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.600+0000 7f7a5e600640 1 -- 172.21.15.159:0/4148061552 wait complete. 2024-04-04T06:10:56.610 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-04T06:10:56.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:10:56.610 DEBUG:teuthology.orchestra.run.smithi159:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-04T06:10:56.720 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:10:56.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:10:56.721 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-04T06:10:56.778 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:10:56.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.959+0000 7f479d400640 1 Processor -- start 2024-04-04T06:10:56.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.959+0000 7f479d400640 1 -- start start 2024-04-04T06:10:56.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.962+0000 7f479d400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f47980cf700 0x7f4798168fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:56.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.962+0000 7f479d400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f479815e880 con 0x7f479815e4a0 2024-04-04T06:10:56.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.962+0000 7f479d400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47980d28d0 con 0x7f47980d2310 2024-04-04T06:10:56.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.962+0000 7f479d400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f47980d34e0 con 0x7f47980cf700 2024-04-04T06:10:56.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.962+0000 7f4797e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f479815e4a0 0x7f47980b9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43480/0 (socket says 172.21.15.159:43480) 2024-04-04T06:10:56.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.962+0000 7f4797e00640 1 -- 172.21.15.159:0/3909597004 learned_addr learned my addr 172.21.15.159:0/3909597004 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:10:56.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.962+0000 7f479ca00640 1 --2- 172.21.15.159:0/3909597004 >> v2:172.21.15.159:3300/0 conn(0x7f47980cf700 0x7f4798168fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:56.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.962+0000 7f4796a00640 1 -- 172.21.15.159:0/3909597004 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 359660050 0 0) 0x7f479815e880 con 0x7f479815e4a0 2024-04-04T06:10:56.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.962+0000 7f4796a00640 1 -- 172.21.15.159:0/3909597004 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f477c003520 con 0x7f479815e4a0 2024-04-04T06:10:56.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.962+0000 7f479ca00640 1 -- 172.21.15.159:0/3909597004 >> v1:172.21.15.178:6789/0 conn(0x7f47980d2310 legacy=0x7f47980ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:56.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.963+0000 7f479ca00640 1 -- 172.21.15.159:0/3909597004 >> v1:172.21.15.159:6789/0 conn(0x7f479815e4a0 legacy=0x7f47980b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:56.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.963+0000 7f479ca00640 1 -- 172.21.15.159:0/3909597004 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47980d2bb0 con 0x7f47980cf700 2024-04-04T06:10:56.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.964+0000 7f479ca00640 1 --2- 172.21.15.159:0/3909597004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47980cf700 0x7f4798168fc0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4790009d90 tx=0x7f479002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb80d96da285f7ea server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:56.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.964+0000 7f4796a00640 1 -- 172.21.15.159:0/3909597004 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790039070 con 0x7f47980cf700 2024-04-04T06:10:56.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.964+0000 7f4796a00640 1 -- 172.21.15.159:0/3909597004 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f479002cbd0 con 0x7f47980cf700 2024-04-04T06:10:56.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.964+0000 7f4796a00640 1 -- 172.21.15.159:0/3909597004 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790034440 con 0x7f47980cf700 2024-04-04T06:10:56.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.964+0000 7f479d400640 1 -- 172.21.15.159:0/3909597004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47980cf700 msgr2=0x7f4798168fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:56.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.964+0000 7f479d400640 1 --2- 172.21.15.159:0/3909597004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47980cf700 0x7f4798168fc0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4790009d90 tx=0x7f479002be20 comp rx=0 tx=0).stop 2024-04-04T06:10:56.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.965+0000 7f479d400640 1 -- 172.21.15.159:0/3909597004 shutdown_connections 2024-04-04T06:10:56.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.965+0000 7f479d400640 1 --2- 172.21.15.159:0/3909597004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47980cf700 0x7f4798168fc0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:56.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.965+0000 7f479d400640 1 -- 172.21.15.159:0/3909597004 >> 172.21.15.159:0/3909597004 conn(0x7f47980676d0 msgr2=0x7f4798067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:56.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.965+0000 7f479d400640 1 -- 172.21.15.159:0/3909597004 shutdown_connections 2024-04-04T06:10:56.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.965+0000 7f479d400640 1 -- 172.21.15.159:0/3909597004 wait complete. 2024-04-04T06:10:56.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.966+0000 7f479d400640 1 Processor -- start 2024-04-04T06:10:56.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.966+0000 7f479d400640 1 -- start start 2024-04-04T06:10:56.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.967+0000 7f479d400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47980cf700 0x7f479815f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:56.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.967+0000 7f479d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47980d2310 0x7f47980c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:56.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.967+0000 7f479d400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f479815e4a0 0x7f47981f5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:56.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.967+0000 7f479d400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f47980d3970 con 0x7f47980d2310 2024-04-04T06:10:56.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.967+0000 7f479d400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f47980d3170 con 0x7f479815e4a0 2024-04-04T06:10:56.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.967+0000 7f479d400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f47980b83f0 con 0x7f47980cf700 2024-04-04T06:10:56.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.968+0000 7f4797400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47980d2310 0x7f47980c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:56.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.968+0000 7f4797400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47980d2310 0x7f47980c9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42592/0 (socket says 172.21.15.159:42592) 2024-04-04T06:10:56.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.968+0000 7f4797400640 1 -- 172.21.15.159:0/1226039980 learned_addr learned my addr 172.21.15.159:0/1226039980 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:56.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.968+0000 7f4797400640 1 -- 172.21.15.159:0/1226039980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47980cf700 msgr2=0x7f479815f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:56.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.968+0000 7f4797400640 1 --2- 172.21.15.159:0/1226039980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47980cf700 0x7f479815f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:56.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.968+0000 7f479ca00640 1 --2- 172.21.15.159:0/1226039980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f479815e4a0 0x7f47981f5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:56.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.968+0000 7f4797400640 1 -- 172.21.15.159:0/1226039980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f479815e4a0 msgr2=0x7f47981f5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:56.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.968+0000 7f4797400640 1 --2- 172.21.15.159:0/1226039980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f479815e4a0 0x7f47981f5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:56.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.968+0000 7f4797400640 1 -- 172.21.15.159:0/1226039980 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47980c85d0 con 0x7f47980d2310 2024-04-04T06:10:56.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.968+0000 7f4797400640 1 --2- 172.21.15.159:0/1226039980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47980d2310 0x7f47980c9640 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f478400c430 tx=0x7f478400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:56.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.969+0000 7f478fe00640 1 -- 172.21.15.159:0/1226039980 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784011790 con 0x7f47980d2310 2024-04-04T06:10:56.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.969+0000 7f478fe00640 1 -- 172.21.15.159:0/1226039980 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4784017070 con 0x7f47980d2310 2024-04-04T06:10:56.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.969+0000 7f478fe00640 1 -- 172.21.15.159:0/1226039980 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784011bb0 con 0x7f47980d2310 2024-04-04T06:10:56.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.969+0000 7f479d400640 1 -- 172.21.15.159:0/1226039980 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47980c8880 con 0x7f47980d2310 2024-04-04T06:10:56.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.969+0000 7f479d400640 1 -- 172.21.15.159:0/1226039980 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47980c8e40 con 0x7f47980d2310 2024-04-04T06:10:56.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.970+0000 7f479d400640 1 -- 172.21.15.159:0/1226039980 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4798075940 con 0x7f47980d2310 2024-04-04T06:10:56.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.971+0000 7f478fe00640 1 -- 172.21.15.159:0/1226039980 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f478401c460 con 0x7f47980d2310 2024-04-04T06:10:56.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.972+0000 7f478fe00640 1 --2- 172.21.15.159:0/1226039980 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f477806c920 0x7f477806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:56.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.972+0000 7f478fe00640 1 -- 172.21.15.159:0/1226039980 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f478408b990 con 0x7f47980d2310 2024-04-04T06:10:56.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.972+0000 7f4797e00640 1 --2- 172.21.15.159:0/1226039980 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f477806c920 0x7f477806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:56.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.973+0000 7f4797e00640 1 --2- 172.21.15.159:0/1226039980 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f477806c920 0x7f477806ede0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4788000f60 tx=0x7f47880078f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:56.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:56.976+0000 7f478fe00640 1 -- 172.21.15.159:0/1226039980 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4784059db0 con 0x7f47980d2310 2024-04-04T06:10:57.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.215+0000 7f479d400640 1 -- 172.21.15.159:0/1226039980 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47980b52f0 con 0x7f47980d2310 2024-04-04T06:10:57.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.216+0000 7f478fe00640 1 -- 172.21.15.159:0/1226039980 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5192 (secure 0 0 0) 0x7f47840596b0 con 0x7f47980d2310 2024-04-04T06:10:57.208 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:57.208 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3789513463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3789513463},{"type":"v1","addr":"172.21.15.178:6835","nonce":3789513463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/468875883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":468875883},{"type":"v1","addr":"172.21.15.178:6837","nonce":468875883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2546974742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2546974742},{"type":"v1","addr":"172.21.15.159:6833","nonce":2546974742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:10:44.805617+0000","modified":"2024-04-04T06:10:47.987695+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6633},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6633":{"gid":6633,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6835/1652468640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1652468640},{"type":"v1","addr":"172.21.15.159:6835","nonce":1652468640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6633,"qdb_cluster":[6633]},"id":5}]} 2024-04-04T06:10:57.208 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 45 2024-04-04T06:10:57.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.218+0000 7f479d400640 1 -- 172.21.15.159:0/1226039980 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f477806c920 msgr2=0x7f477806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:57.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.218+0000 7f479d400640 1 --2- 172.21.15.159:0/1226039980 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f477806c920 0x7f477806ede0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4788000f60 tx=0x7f47880078f0 comp rx=0 tx=0).stop 2024-04-04T06:10:57.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.218+0000 7f479d400640 1 -- 172.21.15.159:0/1226039980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47980d2310 msgr2=0x7f47980c9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:57.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.218+0000 7f479d400640 1 --2- 172.21.15.159:0/1226039980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47980d2310 0x7f47980c9640 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f478400c430 tx=0x7f478400c8f0 comp rx=0 tx=0).stop 2024-04-04T06:10:57.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.218+0000 7f479d400640 1 -- 172.21.15.159:0/1226039980 shutdown_connections 2024-04-04T06:10:57.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.218+0000 7f479d400640 1 --2- 172.21.15.159:0/1226039980 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f477806c920 0x7f477806ede0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:57.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.218+0000 7f479d400640 1 --2- 172.21.15.159:0/1226039980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f479815e4a0 0x7f47981f5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:57.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.218+0000 7f479d400640 1 --2- 172.21.15.159:0/1226039980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47980d2310 0x7f47980c9640 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:57.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.218+0000 7f479d400640 1 --2- 172.21.15.159:0/1226039980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47980cf700 0x7f479815f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:57.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.218+0000 7f479d400640 1 -- 172.21.15.159:0/1226039980 >> 172.21.15.159:0/1226039980 conn(0x7f47980676d0 msgr2=0x7f479815bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:57.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.219+0000 7f479d400640 1 -- 172.21.15.159:0/1226039980 shutdown_connections 2024-04-04T06:10:57.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.219+0000 7f479d400640 1 -- 172.21.15.159:0/1226039980 wait complete. 2024-04-04T06:10:57.229 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-04T06:10:57.469 INFO:teuthology.orchestra.run.smithi159.stdout:{"message":"","return_code":0} 2024-04-04T06:10:57.469 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-04T06:10:57.469 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-04T06:10:57.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.675+0000 7f64fbe00640 1 Processor -- start 2024-04-04T06:10:57.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.675+0000 7f64fbe00640 1 -- start start 2024-04-04T06:10:57.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.677+0000 7f64fbe00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f64fc0d2310 0x7f64fc0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:57.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.677+0000 7f64fbe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64fc0cf530 0x7f64fc0b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:57.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.677+0000 7f64fbe00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64fc15e880 con 0x7f64fc15e4a0 2024-04-04T06:10:57.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.677+0000 7f64fbe00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f64fc0d34e0 con 0x7f64fc0d2310 2024-04-04T06:10:57.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.677+0000 7f64fbe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f64fc0d28d0 con 0x7f64fc0cf530 2024-04-04T06:10:57.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.677+0000 7f64fb400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64fc0cf530 0x7f64fc0b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:57.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.677+0000 7f64f3e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f64fc0d2310 0x7f64fc0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:57.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.677+0000 7f64fb400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64fc0cf530 0x7f64fc0b73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:51086/0 (socket says 172.21.15.159:51086) 2024-04-04T06:10:57.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.677+0000 7f64fb400640 1 -- 172.21.15.159:0/3697589189 learned_addr learned my addr 172.21.15.159:0/3697589189 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:57.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.678+0000 7f64fb400640 1 -- 172.21.15.159:0/3697589189 >> v2:172.21.15.178:3300/0 conn(0x7f64fc0d2310 msgr2=0x7f64fc0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:57.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.678+0000 7f64fb400640 1 --2- 172.21.15.159:0/3697589189 >> v2:172.21.15.178:3300/0 conn(0x7f64fc0d2310 0x7f64fc0ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:57.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.678+0000 7f64fb400640 1 -- 172.21.15.159:0/3697589189 >> v1:172.21.15.178:6789/0 conn(0x7f64fc15e4a0 legacy=0x7f64fc0b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:57.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.678+0000 7f64fb400640 1 -- 172.21.15.159:0/3697589189 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc0d2bb0 con 0x7f64fc0cf530 2024-04-04T06:10:57.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.678+0000 7f64fa000640 1 -- 172.21.15.159:0/3697589189 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4048786307 0 0) 0x7f64fc15e880 con 0x7f64fc15e4a0 2024-04-04T06:10:57.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.678+0000 7f64f3e00640 1 --2- 172.21.15.159:0/3697589189 >> v2:172.21.15.178:3300/0 conn(0x7f64fc0d2310 0x7f64fc0ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:10:57.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.679+0000 7f64fb400640 1 --2- 172.21.15.159:0/3697589189 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64fc0cf530 0x7f64fc0b73e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009e50 tx=0x7f64f402bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2c27cac92080a19 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:57.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.680+0000 7f64fa000640 1 -- 172.21.15.159:0/3697589189 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f402ca80 con 0x7f64fc0cf530 2024-04-04T06:10:57.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.680+0000 7f64fa000640 1 -- 172.21.15.159:0/3697589189 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f402cc20 con 0x7f64fc0cf530 2024-04-04T06:10:57.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.680+0000 7f64fa000640 1 -- 172.21.15.159:0/3697589189 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f40343f0 con 0x7f64fc0cf530 2024-04-04T06:10:57.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.680+0000 7f64fbe00640 1 -- 172.21.15.159:0/3697589189 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64fc0cf530 msgr2=0x7f64fc0b73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:57.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.680+0000 7f64fbe00640 1 --2- 172.21.15.159:0/3697589189 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64fc0cf530 0x7f64fc0b73e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009e50 tx=0x7f64f402bee0 comp rx=0 tx=0).stop 2024-04-04T06:10:57.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.681+0000 7f64fbe00640 1 -- 172.21.15.159:0/3697589189 shutdown_connections 2024-04-04T06:10:57.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.681+0000 7f64fbe00640 1 --2- 172.21.15.159:0/3697589189 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64fc0cf530 0x7f64fc0b73e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:57.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.681+0000 7f64fbe00640 1 --2- 172.21.15.159:0/3697589189 >> v2:172.21.15.178:3300/0 conn(0x7f64fc0d2310 0x7f64fc0ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:57.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.681+0000 7f64fbe00640 1 -- 172.21.15.159:0/3697589189 >> 172.21.15.159:0/3697589189 conn(0x7f64fc0676d0 msgr2=0x7f64fc067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:57.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.681+0000 7f64fbe00640 1 -- 172.21.15.159:0/3697589189 shutdown_connections 2024-04-04T06:10:57.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.681+0000 7f64fbe00640 1 -- 172.21.15.159:0/3697589189 wait complete. 2024-04-04T06:10:57.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.682+0000 7f64fbe00640 1 Processor -- start 2024-04-04T06:10:57.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.682+0000 7f64fbe00640 1 -- start start 2024-04-04T06:10:57.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.683+0000 7f64fbe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64fc0cf530 0x7f64fc1547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:57.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.683+0000 7f64fbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64fc0d2310 0x7f64fc155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:57.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.683+0000 7f64fbe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64fc15e4a0 0x7f64fc1559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:57.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.683+0000 7f64fbe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc0d3970 con 0x7f64fc0d2310 2024-04-04T06:10:57.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.683+0000 7f64fbe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc0d3170 con 0x7f64fc15e4a0 2024-04-04T06:10:57.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.683+0000 7f64fbe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f64fc0b7920 con 0x7f64fc0cf530 2024-04-04T06:10:57.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.684+0000 7f64f3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64fc0d2310 0x7f64fc155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:57.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.684+0000 7f64f3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64fc0d2310 0x7f64fc155000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42596/0 (socket says 172.21.15.159:42596) 2024-04-04T06:10:57.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.684+0000 7f64f3e00640 1 -- 172.21.15.159:0/450927233 learned_addr learned my addr 172.21.15.159:0/450927233 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:57.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.684+0000 7f64fb400640 1 --2- 172.21.15.159:0/450927233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64fc15e4a0 0x7f64fc1559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:57.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.684+0000 7f64f3e00640 1 -- 172.21.15.159:0/450927233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64fc0cf530 msgr2=0x7f64fc1547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:57.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.684+0000 7f64faa00640 1 --2- 172.21.15.159:0/450927233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64fc0cf530 0x7f64fc1547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:57.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.684+0000 7f64f3e00640 1 --2- 172.21.15.159:0/450927233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64fc0cf530 0x7f64fc1547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:57.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.684+0000 7f64f3e00640 1 -- 172.21.15.159:0/450927233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64fc15e4a0 msgr2=0x7f64fc1559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:57.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.684+0000 7f64f3e00640 1 --2- 172.21.15.159:0/450927233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64fc15e4a0 0x7f64fc1559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:57.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.684+0000 7f64f3e00640 1 -- 172.21.15.159:0/450927233 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc0cb420 con 0x7f64fc0d2310 2024-04-04T06:10:57.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.684+0000 7f64faa00640 1 --2- 172.21.15.159:0/450927233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64fc0cf530 0x7f64fc1547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:57.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.684+0000 7f64fb400640 1 --2- 172.21.15.159:0/450927233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64fc15e4a0 0x7f64fc1559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:57.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.684+0000 7f64f3e00640 1 --2- 172.21.15.159:0/450927233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64fc0d2310 0x7f64fc155000 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f64e400ec10 tx=0x7f64e400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:57.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.685+0000 7f64f2a00640 1 -- 172.21.15.159:0/450927233 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4017070 con 0x7f64fc0d2310 2024-04-04T06:10:57.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.685+0000 7f64f2a00640 1 -- 172.21.15.159:0/450927233 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e400cc30 con 0x7f64fc0d2310 2024-04-04T06:10:57.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.685+0000 7f64f2a00640 1 -- 172.21.15.159:0/450927233 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4013470 con 0x7f64fc0d2310 2024-04-04T06:10:57.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.685+0000 7f64fbe00640 1 -- 172.21.15.159:0/450927233 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64fc0cb700 con 0x7f64fc0d2310 2024-04-04T06:10:57.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.685+0000 7f64fbe00640 1 -- 172.21.15.159:0/450927233 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64fc0cbcc0 con 0x7f64fc0d2310 2024-04-04T06:10:57.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.686+0000 7f64fbe00640 1 -- 172.21.15.159:0/450927233 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64fc075940 con 0x7f64fc0d2310 2024-04-04T06:10:57.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.687+0000 7f64f2a00640 1 -- 172.21.15.159:0/450927233 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64e401a070 con 0x7f64fc0d2310 2024-04-04T06:10:57.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.688+0000 7f64f2a00640 1 --2- 172.21.15.159:0/450927233 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f64d806c8f0 0x7f64d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:57.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.688+0000 7f64f2a00640 1 -- 172.21.15.159:0/450927233 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f64e408bed0 con 0x7f64fc0d2310 2024-04-04T06:10:57.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.689+0000 7f64faa00640 1 --2- 172.21.15.159:0/450927233 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f64d806c8f0 0x7f64d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:57.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.690+0000 7f64faa00640 1 --2- 172.21.15.159:0/450927233 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f64d806c8f0 0x7f64d806edb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f64ec000f60 tx=0x7f64ec009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:57.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.693+0000 7f64f2a00640 1 -- 172.21.15.159:0/450927233 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f64e405a260 con 0x7f64fc0d2310 2024-04-04T06:10:57.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.933+0000 7f64fbe00640 1 -- 172.21.15.159:0/450927233 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f64fc0d4ea0 con 0x7f64fc0d2310 2024-04-04T06:10:57.926 INFO:tasks.ceph.mon.a.smithi159.stderr:2024-04-04T06:10:57.934+0000 7f1995c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-04T06:10:57.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.937+0000 7f64f2a00640 1 -- 172.21.15.159:0/450927233 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) ==== 161+0+0 (secure 0 0 0) 0x7f64e4059b60 con 0x7f64fc0d2310 2024-04-04T06:10:57.929 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-04T06:10:57.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.941+0000 7f64fbe00640 1 -- 172.21.15.159:0/450927233 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f64d806c8f0 msgr2=0x7f64d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:57.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.941+0000 7f64fbe00640 1 --2- 172.21.15.159:0/450927233 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f64d806c8f0 0x7f64d806edb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f64ec000f60 tx=0x7f64ec009040 comp rx=0 tx=0).stop 2024-04-04T06:10:57.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.941+0000 7f64fbe00640 1 -- 172.21.15.159:0/450927233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64fc0d2310 msgr2=0x7f64fc155000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:57.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.941+0000 7f64fbe00640 1 --2- 172.21.15.159:0/450927233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64fc0d2310 0x7f64fc155000 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f64e400ec10 tx=0x7f64e400c520 comp rx=0 tx=0).stop 2024-04-04T06:10:57.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.941+0000 7f64fbe00640 1 -- 172.21.15.159:0/450927233 shutdown_connections 2024-04-04T06:10:57.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.941+0000 7f64fbe00640 1 --2- 172.21.15.159:0/450927233 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f64d806c8f0 0x7f64d806edb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:57.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.941+0000 7f64fbe00640 1 --2- 172.21.15.159:0/450927233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64fc15e4a0 0x7f64fc1559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:57.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.941+0000 7f64fbe00640 1 --2- 172.21.15.159:0/450927233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64fc0d2310 0x7f64fc155000 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:57.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.941+0000 7f64fbe00640 1 --2- 172.21.15.159:0/450927233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64fc0cf530 0x7f64fc1547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:57.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.941+0000 7f64fbe00640 1 -- 172.21.15.159:0/450927233 >> 172.21.15.159:0/450927233 conn(0x7f64fc0676d0 msgr2=0x7f64fc15c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:57.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.941+0000 7f64fbe00640 1 -- 172.21.15.159:0/450927233 shutdown_connections 2024-04-04T06:10:57.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:57.941+0000 7f64fbe00640 1 -- 172.21.15.159:0/450927233 wait complete. 2024-04-04T06:10:57.941 INFO:tasks.ceph.mds.a.smithi159.stderr: -4421> 2024-04-04T06:10:49.307+0000 7f50bd400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:10:57.953 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-04T06:10:58.002 INFO:tasks.ceph.mds.a.smithi159.stdout:starting mds.a at 2024-04-04T06:10:58.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.046+0000 7f9035b32f40 1 Processor -- start 2024-04-04T06:10:58.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.046+0000 7f9035b32f40 1 -- start start 2024-04-04T06:10:58.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.047+0000 7f9035b32f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ad21baad70 0x55ad21b91860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:58.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.047+0000 7f9035b32f40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad21c36870 0x55ad21c3fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:58.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.047+0000 7f9035b32f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ad21c36c50 0x55ad21c390a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:58.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.047+0000 7f9035b32f40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ad21ba97d0 con 0x55ad21c36870 2024-04-04T06:10:58.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.047+0000 7f9035b32f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ad21b082c0 con 0x55ad21c36c50 2024-04-04T06:10:58.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.047+0000 7f9035b32f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ad21b658d0 con 0x55ad21baad70 2024-04-04T06:10:58.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.048+0000 7f9032400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad21c36870 0x55ad21c3fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:58.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.048+0000 7f9032400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad21c36870 0x55ad21c3fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42634/0 (socket says 172.21.15.159:42634) 2024-04-04T06:10:58.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.048+0000 7f9032400640 1 -- 172.21.15.159:0/216600043 learned_addr learned my addr 172.21.15.159:0/216600043 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:58.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.048+0000 7f9033800640 1 --2- 172.21.15.159:0/216600043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ad21c36c50 0x55ad21c390a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:58.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.048+0000 7f9032e00640 1 --2- 172.21.15.159:0/216600043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ad21baad70 0x55ad21b91860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:58.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.048+0000 7f9032400640 1 -- 172.21.15.159:0/216600043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ad21baad70 msgr2=0x55ad21b91860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:58.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.048+0000 7f9032400640 1 --2- 172.21.15.159:0/216600043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ad21baad70 0x55ad21b91860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.048+0000 7f9032400640 1 -- 172.21.15.159:0/216600043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ad21c36c50 msgr2=0x55ad21c390a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:58.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.048+0000 7f9032400640 1 --2- 172.21.15.159:0/216600043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ad21c36c50 0x55ad21c390a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.048+0000 7f9032400640 1 -- 172.21.15.159:0/216600043 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad21c385a0 con 0x55ad21c36870 2024-04-04T06:10:58.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.048+0000 7f9032400640 1 --2- 172.21.15.159:0/216600043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad21c36870 0x55ad21c3fdb0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f90240096e0 tx=0x7f9024009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688de56a556b4221 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:58.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.048+0000 7f9031000640 1 -- 172.21.15.159:0/216600043 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9024014070 con 0x55ad21c36870 2024-04-04T06:10:58.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.049+0000 7f9031000640 1 -- 172.21.15.159:0/216600043 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f902400b930 con 0x55ad21c36870 2024-04-04T06:10:58.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.049+0000 7f9031000640 1 -- 172.21.15.159:0/216600043 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902400bd20 con 0x55ad21c36870 2024-04-04T06:10:58.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.049+0000 7f9035b32f40 1 -- 172.21.15.159:0/216600043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad21c36870 msgr2=0x55ad21c3fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:58.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.049+0000 7f9035b32f40 1 --2- 172.21.15.159:0/216600043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad21c36870 0x55ad21c3fdb0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f90240096e0 tx=0x7f9024009ba0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.049+0000 7f9035b32f40 1 -- 172.21.15.159:0/216600043 shutdown_connections 2024-04-04T06:10:58.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.049+0000 7f9035b32f40 1 --2- 172.21.15.159:0/216600043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ad21c36c50 0x55ad21c390a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.049+0000 7f9035b32f40 1 --2- 172.21.15.159:0/216600043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad21c36870 0x55ad21c3fdb0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.049+0000 7f9035b32f40 1 --2- 172.21.15.159:0/216600043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ad21baad70 0x55ad21b91860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.049+0000 7f9035b32f40 1 -- 172.21.15.159:0/216600043 >> 172.21.15.159:0/216600043 conn(0x55ad21b8cdc0 msgr2=0x55ad21c40e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:58.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.049+0000 7f9035b32f40 1 -- 172.21.15.159:0/216600043 shutdown_connections 2024-04-04T06:10:58.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.049+0000 7f9035b32f40 1 -- 172.21.15.159:0/216600043 wait complete. 2024-04-04T06:10:58.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.050+0000 7f9035b32f40 1 Processor -- start 2024-04-04T06:10:58.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.050+0000 7f9035b32f40 1 -- start start 2024-04-04T06:10:58.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.051+0000 7f9035b32f40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad21baad70 0x55ad21c40930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:58.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.051+0000 7f9035b32f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ad21c36870 0x55ad21cd5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:58.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.051+0000 7f9035b32f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ad21c36c50 0x55ad21cd60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:58.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.051+0000 7f9035b32f40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ad21c376a0 con 0x55ad21baad70 2024-04-04T06:10:58.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.051+0000 7f9035b32f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ad21b913d0 con 0x55ad21c36870 2024-04-04T06:10:58.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.051+0000 7f9035b32f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ad21c37940 con 0x55ad21c36c50 2024-04-04T06:10:58.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.051+0000 7f9032e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad21baad70 0x55ad21c40930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:58.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.051+0000 7f9032e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad21baad70 0x55ad21c40930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42640/0 (socket says 172.21.15.159:42640) 2024-04-04T06:10:58.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.051+0000 7f9032e00640 1 -- 172.21.15.159:0/1551384772 learned_addr learned my addr 172.21.15.159:0/1551384772 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:58.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.051+0000 7f9032e00640 1 -- 172.21.15.159:0/1551384772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ad21c36c50 msgr2=0x55ad21cd60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:58.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.051+0000 7f9032e00640 1 --2- 172.21.15.159:0/1551384772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ad21c36c50 0x55ad21cd60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.051+0000 7f9032e00640 1 -- 172.21.15.159:0/1551384772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ad21c36870 msgr2=0x55ad21cd5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:58.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.051+0000 7f9032e00640 1 --2- 172.21.15.159:0/1551384772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ad21c36870 0x55ad21cd5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.051+0000 7f9032e00640 1 -- 172.21.15.159:0/1551384772 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad21c2a4c0 con 0x55ad21baad70 2024-04-04T06:10:58.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.052+0000 7f9032e00640 1 --2- 172.21.15.159:0/1551384772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad21baad70 0x55ad21c40930 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f901c009730 tx=0x7f901c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:58.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.052+0000 7f902b400640 1 -- 172.21.15.159:0/1551384772 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c005a50 con 0x55ad21baad70 2024-04-04T06:10:58.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.052+0000 7f902b400640 1 -- 172.21.15.159:0/1551384772 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f901c005c10 con 0x55ad21baad70 2024-04-04T06:10:58.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.052+0000 7f902b400640 1 -- 172.21.15.159:0/1551384772 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c014440 con 0x55ad21baad70 2024-04-04T06:10:58.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.052+0000 7f9035b32f40 1 -- 172.21.15.159:0/1551384772 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ad21c2a1e0 con 0x55ad21baad70 2024-04-04T06:10:58.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.052+0000 7f9035b32f40 1 -- 172.21.15.159:0/1551384772 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad21cc87f0 con 0x55ad21baad70 2024-04-04T06:10:58.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.054+0000 7f902b400640 1 -- 172.21.15.159:0/1551384772 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f901c01d440 con 0x55ad21baad70 2024-04-04T06:10:58.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.055+0000 7f902b400640 1 --2- 172.21.15.159:0/1551384772 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f901006b300 0x7f901006d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:58.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.055+0000 7f902b400640 1 -- 172.21.15.159:0/1551384772 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f901c08ac60 con 0x55ad21baad70 2024-04-04T06:10:58.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.055+0000 7f9035b32f40 1 --2- 172.21.15.159:0/1551384772 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x55ad21c39ab0 0x55ad21cde390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:58.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.055+0000 7f9035b32f40 1 -- 172.21.15.159:0/1551384772 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72) -- 0x55ad21cc8ff0 con 0x55ad21c39ab0 2024-04-04T06:10:58.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.056+0000 7f9032400640 1 --2- 172.21.15.159:0/1551384772 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f901006b300 0x7f901006d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:58.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.056+0000 7f9033800640 1 --2- 172.21.15.159:0/1551384772 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x55ad21c39ab0 0x55ad21cde390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:58.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.057+0000 7f9032400640 1 --2- 172.21.15.159:0/1551384772 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f901006b300 0x7f901006d7c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55ad21b8e1f0 tx=0x7f9024010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:58.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.057+0000 7f9033800640 1 --2- 172.21.15.159:0/1551384772 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x55ad21c39ab0 0x55ad21cde390 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:58.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.061+0000 7f9033800640 1 -- 172.21.15.159:0/1551384772 <== osd.4 v2:172.21.15.178:6812/4035991429 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v72'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f902c0527f0 con 0x55ad21c39ab0 2024-04-04T06:10:58.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.061+0000 7f9035b32f40 1 -- 172.21.15.159:0/1551384772 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x55ad21c39ab0 msgr2=0x55ad21cde390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:58.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.061+0000 7f9035b32f40 1 --2- 172.21.15.159:0/1551384772 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x55ad21c39ab0 0x55ad21cde390 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.061+0000 7f9035b32f40 1 -- 172.21.15.159:0/1551384772 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f901006b300 msgr2=0x7f901006d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:58.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.061+0000 7f9035b32f40 1 --2- 172.21.15.159:0/1551384772 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f901006b300 0x7f901006d7c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55ad21b8e1f0 tx=0x7f9024010000 comp rx=0 tx=0).stop 2024-04-04T06:10:58.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.061+0000 7f9035b32f40 1 -- 172.21.15.159:0/1551384772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad21baad70 msgr2=0x55ad21c40930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:58.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.061+0000 7f9035b32f40 1 --2- 172.21.15.159:0/1551384772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad21baad70 0x55ad21c40930 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f901c009730 tx=0x7f901c009bf0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.061+0000 7f9033800640 1 -- 172.21.15.159:0/1551384772 reap_dead start 2024-04-04T06:10:58.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.061+0000 7f9035b32f40 1 -- 172.21.15.159:0/1551384772 shutdown_connections 2024-04-04T06:10:58.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.061+0000 7f9035b32f40 1 -- 172.21.15.159:0/1551384772 >> 172.21.15.159:0/1551384772 conn(0x55ad21b8cdc0 msgr2=0x55ad21b8d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:58.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.061+0000 7f9035b32f40 1 -- 172.21.15.159:0/1551384772 shutdown_connections 2024-04-04T06:10:58.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.061+0000 7f9035b32f40 1 -- 172.21.15.159:0/1551384772 wait complete. 2024-04-04T06:10:58.058 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-04T06:10:58.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.247+0000 7fa899000640 1 Processor -- start 2024-04-04T06:10:58.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.247+0000 7fa899000640 1 -- start start 2024-04-04T06:10:58.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.249+0000 7fa899000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8940b4650 0x7fa89406c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:58.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.249+0000 7fa899000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa8940b4a30 0x7fa8940b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:58.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.249+0000 7fa899000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa894158580 0x7fa8941589d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:58.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.249+0000 7fa899000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fa894141b30 con 0x7fa894158580 2024-04-04T06:10:58.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.249+0000 7fa899000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa89415ede0 con 0x7fa8940b4a30 2024-04-04T06:10:58.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.249+0000 7fa899000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa894141d80 con 0x7fa8940b4650 2024-04-04T06:10:58.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.249+0000 7fa892800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa894158580 0x7fa8941589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:58.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.249+0000 7fa892800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa894158580 0x7fa8941589d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42650/0 (socket says 172.21.15.159:42650) 2024-04-04T06:10:58.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.249+0000 7fa892800640 1 -- 172.21.15.159:0/1694598167 learned_addr learned my addr 172.21.15.159:0/1694598167 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:58.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.249+0000 7fa891e00640 1 --2- 172.21.15.159:0/1694598167 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8940b4650 0x7fa89406c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:58.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.249+0000 7fa891400640 1 --2- 172.21.15.159:0/1694598167 >> v2:172.21.15.178:3300/0 conn(0x7fa8940b4a30 0x7fa8940b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:58.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.250+0000 7fa892800640 1 -- 172.21.15.159:0/1694598167 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8940b4650 msgr2=0x7fa89406c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:58.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.250+0000 7fa892800640 1 --2- 172.21.15.159:0/1694598167 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8940b4650 0x7fa89406c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.250+0000 7fa892800640 1 -- 172.21.15.159:0/1694598167 >> v2:172.21.15.178:3300/0 conn(0x7fa8940b4a30 msgr2=0x7fa8940b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:58.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.250+0000 7fa892800640 1 --2- 172.21.15.159:0/1694598167 >> v2:172.21.15.178:3300/0 conn(0x7fa8940b4a30 0x7fa8940b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.250+0000 7fa892800640 1 -- 172.21.15.159:0/1694598167 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa894142060 con 0x7fa894158580 2024-04-04T06:10:58.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.251+0000 7fa892800640 1 --2- 172.21.15.159:0/1694598167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa894158580 0x7fa8941589d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa8940d32e0 tx=0x7fa88c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b78a7cdaed61c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:58.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.251+0000 7fa890a00640 1 -- 172.21.15.159:0/1694598167 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c02ca70 con 0x7fa894158580 2024-04-04T06:10:58.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.251+0000 7fa890a00640 1 -- 172.21.15.159:0/1694598167 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa88c02cc10 con 0x7fa894158580 2024-04-04T06:10:58.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.251+0000 7fa890a00640 1 -- 172.21.15.159:0/1694598167 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c0343f0 con 0x7fa894158580 2024-04-04T06:10:58.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.252+0000 7fa899000640 1 -- 172.21.15.159:0/1694598167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa894158580 msgr2=0x7fa8941589d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:58.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.252+0000 7fa899000640 1 --2- 172.21.15.159:0/1694598167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa894158580 0x7fa8941589d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa8940d32e0 tx=0x7fa88c02bed0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.252+0000 7fa899000640 1 -- 172.21.15.159:0/1694598167 shutdown_connections 2024-04-04T06:10:58.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.252+0000 7fa899000640 1 --2- 172.21.15.159:0/1694598167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa894158580 0x7fa8941589d0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.252+0000 7fa899000640 1 --2- 172.21.15.159:0/1694598167 >> v2:172.21.15.178:3300/0 conn(0x7fa8940b4a30 0x7fa8940b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.252+0000 7fa899000640 1 --2- 172.21.15.159:0/1694598167 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8940b4650 0x7fa89406c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.252+0000 7fa899000640 1 -- 172.21.15.159:0/1694598167 >> 172.21.15.159:0/1694598167 conn(0x7fa894067670 msgr2=0x7fa894067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:58.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.252+0000 7fa899000640 1 -- 172.21.15.159:0/1694598167 shutdown_connections 2024-04-04T06:10:58.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.252+0000 7fa899000640 1 -- 172.21.15.159:0/1694598167 wait complete. 2024-04-04T06:10:58.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.254+0000 7fa899000640 1 Processor -- start 2024-04-04T06:10:58.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.254+0000 7fa899000640 1 -- start start 2024-04-04T06:10:58.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.254+0000 7fa899000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8940b4650 0x7fa8940cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:58.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.254+0000 7fa899000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8940b4a30 0x7fa89415dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:58.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.255+0000 7fa899000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa894158580 0x7fa89415e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:58.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.255+0000 7fa899000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa894142900 con 0x7fa894158580 2024-04-04T06:10:58.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.255+0000 7fa899000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa894142620 con 0x7fa8940b4650 2024-04-04T06:10:58.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.255+0000 7fa899000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa894158f10 con 0x7fa8940b4a30 2024-04-04T06:10:58.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.255+0000 7fa892800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa894158580 0x7fa89415e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:58.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.255+0000 7fa892800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa894158580 0x7fa89415e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42666/0 (socket says 172.21.15.159:42666) 2024-04-04T06:10:58.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.255+0000 7fa892800640 1 -- 172.21.15.159:0/357985155 learned_addr learned my addr 172.21.15.159:0/357985155 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:58.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.255+0000 7fa891e00640 1 --2- 172.21.15.159:0/357985155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8940b4650 0x7fa8940cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:58.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.255+0000 7fa892800640 1 -- 172.21.15.159:0/357985155 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8940b4a30 msgr2=0x7fa89415dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:58.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.255+0000 7fa892800640 1 --2- 172.21.15.159:0/357985155 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8940b4a30 0x7fa89415dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.255+0000 7fa892800640 1 -- 172.21.15.159:0/357985155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8940b4650 msgr2=0x7fa8940cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:58.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.255+0000 7fa892800640 1 --2- 172.21.15.159:0/357985155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8940b4650 0x7fa8940cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.255+0000 7fa892800640 1 -- 172.21.15.159:0/357985155 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8940cdaa0 con 0x7fa894158580 2024-04-04T06:10:58.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.256+0000 7fa892800640 1 --2- 172.21.15.159:0/357985155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa894158580 0x7fa89415e210 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa88c036040 tx=0x7fa88c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:58.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.256+0000 7fa88a000640 1 -- 172.21.15.159:0/357985155 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c043020 con 0x7fa894158580 2024-04-04T06:10:58.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.256+0000 7fa88a000640 1 -- 172.21.15.159:0/357985155 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa88c03e450 con 0x7fa894158580 2024-04-04T06:10:58.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.256+0000 7fa88a000640 1 -- 172.21.15.159:0/357985155 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c009510 con 0x7fa894158580 2024-04-04T06:10:58.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.256+0000 7fa899000640 1 -- 172.21.15.159:0/357985155 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8941f7f40 con 0x7fa894158580 2024-04-04T06:10:58.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.256+0000 7fa899000640 1 -- 172.21.15.159:0/357985155 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8941f8500 con 0x7fa894158580 2024-04-04T06:10:58.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.257+0000 7fa899000640 1 -- 172.21.15.159:0/357985155 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa894074f30 con 0x7fa894158580 2024-04-04T06:10:58.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.258+0000 7fa88a000640 1 -- 172.21.15.159:0/357985155 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa88c046020 con 0x7fa894158580 2024-04-04T06:10:58.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.259+0000 7fa88a000640 1 --2- 172.21.15.159:0/357985155 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa87006c8f0 0x7fa87006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:58.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.259+0000 7fa88a000640 1 -- 172.21.15.159:0/357985155 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7fa88c0ae4f0 con 0x7fa894158580 2024-04-04T06:10:58.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.259+0000 7fa891e00640 1 --2- 172.21.15.159:0/357985155 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa87006c8f0 0x7fa87006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:58.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.260+0000 7fa891e00640 1 --2- 172.21.15.159:0/357985155 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa87006c8f0 0x7fa87006edb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa884006fd0 tx=0x7fa884008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:58.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.264+0000 7fa88a000640 1 -- 172.21.15.159:0/357985155 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa88c04d030 con 0x7fa894158580 2024-04-04T06:10:58.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.507+0000 7fa899000640 1 -- 172.21.15.159:0/357985155 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fa8940cc700 con 0x7fa894158580 2024-04-04T06:10:58.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.941+0000 7fa88a000640 1 -- 172.21.15.159:0/357985155 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v47) ==== 162+0+0 (secure 0 0 0) 0x7fa88c07c820 con 0x7fa894158580 2024-04-04T06:10:58.934 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-04T06:10:58.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.945+0000 7fa899000640 1 -- 172.21.15.159:0/357985155 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa87006c8f0 msgr2=0x7fa87006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:58.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.945+0000 7fa899000640 1 --2- 172.21.15.159:0/357985155 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa87006c8f0 0x7fa87006edb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa884006fd0 tx=0x7fa884008040 comp rx=0 tx=0).stop 2024-04-04T06:10:58.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.945+0000 7fa899000640 1 -- 172.21.15.159:0/357985155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa894158580 msgr2=0x7fa89415e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:58.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.945+0000 7fa899000640 1 --2- 172.21.15.159:0/357985155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa894158580 0x7fa89415e210 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa88c036040 tx=0x7fa88c004a40 comp rx=0 tx=0).stop 2024-04-04T06:10:58.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.945+0000 7fa899000640 1 -- 172.21.15.159:0/357985155 shutdown_connections 2024-04-04T06:10:58.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.945+0000 7fa899000640 1 --2- 172.21.15.159:0/357985155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa894158580 0x7fa89415e210 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.945+0000 7fa899000640 1 --2- 172.21.15.159:0/357985155 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8940b4a30 0x7fa89415dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.945+0000 7fa899000640 1 --2- 172.21.15.159:0/357985155 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa87006c8f0 0x7fa87006edb0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.945+0000 7fa899000640 1 --2- 172.21.15.159:0/357985155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8940b4650 0x7fa8940cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:58.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.945+0000 7fa899000640 1 -- 172.21.15.159:0/357985155 >> 172.21.15.159:0/357985155 conn(0x7fa894067670 msgr2=0x7fa89406d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:58.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.945+0000 7fa899000640 1 -- 172.21.15.159:0/357985155 shutdown_connections 2024-04-04T06:10:58.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:58.945+0000 7fa899000640 1 -- 172.21.15.159:0/357985155 wait complete. 2024-04-04T06:10:58.962 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:10:59.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.152+0000 7fe28c800640 1 Processor -- start 2024-04-04T06:10:59.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.152+0000 7fe28c800640 1 -- start start 2024-04-04T06:10:59.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.153+0000 7fe28c800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe28415c390 0x7fe284073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:59.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.153+0000 7fe28c800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe2840af830 0x7fe284160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:59.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.153+0000 7fe28c800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe28415fe80 con 0x7fe284160b80 2024-04-04T06:10:59.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.153+0000 7fe28c800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe28415c770 con 0x7fe2840af830 2024-04-04T06:10:59.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.153+0000 7fe28c800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe2840ad120 con 0x7fe28415c390 2024-04-04T06:10:59.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.154+0000 7fe289400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe2840af830 0x7fe284160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:59.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.154+0000 7fe289400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe2840af830 0x7fe284160640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42680/0 (socket says 172.21.15.159:42680) 2024-04-04T06:10:59.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.154+0000 7fe289400640 1 -- 172.21.15.159:0/301074761 learned_addr learned my addr 172.21.15.159:0/301074761 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:59.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.154+0000 7fe289e00640 1 --2- 172.21.15.159:0/301074761 >> v2:172.21.15.178:3300/0 conn(0x7fe28415c390 0x7fe284073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:59.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.154+0000 7fe289400640 1 -- 172.21.15.159:0/301074761 >> v2:172.21.15.178:3300/0 conn(0x7fe28415c390 msgr2=0x7fe284073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:59.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.154+0000 7fe289400640 1 --2- 172.21.15.159:0/301074761 >> v2:172.21.15.178:3300/0 conn(0x7fe28415c390 0x7fe284073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:59.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.154+0000 7fe289400640 1 -- 172.21.15.159:0/301074761 >> v1:172.21.15.178:6789/0 conn(0x7fe284160b80 legacy=0x7fe2840d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:10:59.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.154+0000 7fe289400640 1 -- 172.21.15.159:0/301074761 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2840ad400 con 0x7fe2840af830 2024-04-04T06:10:59.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.156+0000 7fe289400640 1 --2- 172.21.15.159:0/301074761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2840af830 0x7fe284160640 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fe2840b63e0 tx=0x7fe27402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f97350f81dce8af server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:59.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.156+0000 7fe288a00640 1 -- 172.21.15.159:0/301074761 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe274035020 con 0x7fe2840af830 2024-04-04T06:10:59.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.156+0000 7fe288a00640 1 -- 172.21.15.159:0/301074761 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe274031870 con 0x7fe2840af830 2024-04-04T06:10:59.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.156+0000 7fe288a00640 1 -- 172.21.15.159:0/301074761 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe274031c40 con 0x7fe2840af830 2024-04-04T06:10:59.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.157+0000 7fe28c800640 1 -- 172.21.15.159:0/301074761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2840af830 msgr2=0x7fe284160640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:59.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.157+0000 7fe28c800640 1 --2- 172.21.15.159:0/301074761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2840af830 0x7fe284160640 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fe2840b63e0 tx=0x7fe27402b910 comp rx=0 tx=0).stop 2024-04-04T06:10:59.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.157+0000 7fe28c800640 1 -- 172.21.15.159:0/301074761 shutdown_connections 2024-04-04T06:10:59.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.157+0000 7fe28c800640 1 --2- 172.21.15.159:0/301074761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2840af830 0x7fe284160640 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:59.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.157+0000 7fe28c800640 1 --2- 172.21.15.159:0/301074761 >> v2:172.21.15.178:3300/0 conn(0x7fe28415c390 0x7fe284073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:59.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.157+0000 7fe28c800640 1 -- 172.21.15.159:0/301074761 >> 172.21.15.159:0/301074761 conn(0x7fe28406b990 msgr2=0x7fe28406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:59.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.157+0000 7fe28c800640 1 -- 172.21.15.159:0/301074761 shutdown_connections 2024-04-04T06:10:59.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.157+0000 7fe28c800640 1 -- 172.21.15.159:0/301074761 wait complete. 2024-04-04T06:10:59.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.159+0000 7fe28c800640 1 Processor -- start 2024-04-04T06:10:59.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.159+0000 7fe28c800640 1 -- start start 2024-04-04T06:10:59.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.160+0000 7fe28c800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2840af830 0x7fe2840afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:59.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.160+0000 7fe28c800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe28415c390 0x7fe2841f8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:59.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.160+0000 7fe28c800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe284160b80 0x7fe2841f89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:59.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.160+0000 7fe28c800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe2840cf980 con 0x7fe2840af830 2024-04-04T06:10:59.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.160+0000 7fe28c800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe284160120 con 0x7fe284160b80 2024-04-04T06:10:59.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.160+0000 7fe28c800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe2840d0d60 con 0x7fe28415c390 2024-04-04T06:10:59.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.160+0000 7fe289e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2840af830 0x7fe2840afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:59.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.160+0000 7fe289e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2840af830 0x7fe2840afff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42686/0 (socket says 172.21.15.159:42686) 2024-04-04T06:10:59.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.160+0000 7fe289e00640 1 -- 172.21.15.159:0/2698454288 learned_addr learned my addr 172.21.15.159:0/2698454288 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:10:59.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.160+0000 7fe28a800640 1 --2- 172.21.15.159:0/2698454288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe284160b80 0x7fe2841f89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:59.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.160+0000 7fe289e00640 1 -- 172.21.15.159:0/2698454288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe28415c390 msgr2=0x7fe2841f8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:59.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.160+0000 7fe289e00640 1 --2- 172.21.15.159:0/2698454288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe28415c390 0x7fe2841f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:59.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.160+0000 7fe289e00640 1 -- 172.21.15.159:0/2698454288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe284160b80 msgr2=0x7fe2841f89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:59.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.160+0000 7fe289e00640 1 --2- 172.21.15.159:0/2698454288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe284160b80 0x7fe2841f89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:59.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.160+0000 7fe289e00640 1 -- 172.21.15.159:0/2698454288 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28415dab0 con 0x7fe2840af830 2024-04-04T06:10:59.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.160+0000 7fe28a800640 1 --2- 172.21.15.159:0/2698454288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe284160b80 0x7fe2841f89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:10:59.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.161+0000 7fe289e00640 1 --2- 172.21.15.159:0/2698454288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2840af830 0x7fe2840afff0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe27800d800 tx=0x7fe27800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:59.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.161+0000 7fe27e000640 1 -- 172.21.15.159:0/2698454288 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27800b6c0 con 0x7fe2840af830 2024-04-04T06:10:59.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.161+0000 7fe27e000640 1 -- 172.21.15.159:0/2698454288 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe278019070 con 0x7fe2840af830 2024-04-04T06:10:59.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.161+0000 7fe28c800640 1 -- 172.21.15.159:0/2698454288 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe28415dd90 con 0x7fe2840af830 2024-04-04T06:10:59.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.161+0000 7fe27e000640 1 -- 172.21.15.159:0/2698454288 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27800ba80 con 0x7fe2840af830 2024-04-04T06:10:59.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.161+0000 7fe28c800640 1 -- 172.21.15.159:0/2698454288 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2840b0530 con 0x7fe2840af830 2024-04-04T06:10:59.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.162+0000 7fe28c800640 1 -- 172.21.15.159:0/2698454288 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2840685c0 con 0x7fe2840af830 2024-04-04T06:10:59.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.163+0000 7fe27e000640 1 -- 172.21.15.159:0/2698454288 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe27800bc20 con 0x7fe2840af830 2024-04-04T06:10:59.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.164+0000 7fe27e000640 1 --2- 172.21.15.159:0/2698454288 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe26806c8f0 0x7fe26806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:10:59.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.164+0000 7fe27e000640 1 -- 172.21.15.159:0/2698454288 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7fe27808c0b0 con 0x7fe2840af830 2024-04-04T06:10:59.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.165+0000 7fe289400640 1 --2- 172.21.15.159:0/2698454288 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe26806c8f0 0x7fe26806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:10:59.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.166+0000 7fe289400640 1 --2- 172.21.15.159:0/2698454288 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe26806c8f0 0x7fe26806edb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe274036000 tx=0x7fe274030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:10:59.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.169+0000 7fe27e000640 1 -- 172.21.15.159:0/2698454288 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe27801c360 con 0x7fe2840af830 2024-04-04T06:10:59.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.407+0000 7fe28c800640 1 -- 172.21.15.159:0/2698454288 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe284166ff0 con 0x7fe2840af830 2024-04-04T06:10:59.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.408+0000 7fe27e000640 1 -- 172.21.15.159:0/2698454288 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5186 (secure 0 0 0) 0x7fe27805a470 con 0x7fe2840af830 2024-04-04T06:10:59.400 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:10:59.400 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3789513463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3789513463},{"type":"v1","addr":"172.21.15.178:6835","nonce":3789513463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/468875883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":468875883},{"type":"v1","addr":"172.21.15.178:6837","nonce":468875883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2546974742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2546974742},{"type":"v1","addr":"172.21.15.159:6833","nonce":2546974742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:10:44.805617+0000","modified":"2024-04-04T06:10:58.944955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6828":{"gid":6828,"name":"a","rank":0,"incarnation":48,"state":"up:replay","state_seq":1,"addr":"172.21.15.159:6835/2467594024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2467594024},{"type":"v1","addr":"172.21.15.159:6835","nonce":2467594024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-04-04T06:10:59.401 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 48 2024-04-04T06:10:59.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.411+0000 7fe28c800640 1 -- 172.21.15.159:0/2698454288 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe26806c8f0 msgr2=0x7fe26806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:59.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.411+0000 7fe28c800640 1 --2- 172.21.15.159:0/2698454288 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe26806c8f0 0x7fe26806edb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe274036000 tx=0x7fe274030000 comp rx=0 tx=0).stop 2024-04-04T06:10:59.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.411+0000 7fe28c800640 1 -- 172.21.15.159:0/2698454288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2840af830 msgr2=0x7fe2840afff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:10:59.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.411+0000 7fe28c800640 1 --2- 172.21.15.159:0/2698454288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2840af830 0x7fe2840afff0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe27800d800 tx=0x7fe27800dcc0 comp rx=0 tx=0).stop 2024-04-04T06:10:59.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.411+0000 7fe28c800640 1 -- 172.21.15.159:0/2698454288 shutdown_connections 2024-04-04T06:10:59.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.411+0000 7fe28c800640 1 --2- 172.21.15.159:0/2698454288 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe26806c8f0 0x7fe26806edb0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:59.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.411+0000 7fe28c800640 1 --2- 172.21.15.159:0/2698454288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe284160b80 0x7fe2841f89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:59.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.411+0000 7fe28c800640 1 --2- 172.21.15.159:0/2698454288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe28415c390 0x7fe2841f8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:59.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.411+0000 7fe28c800640 1 --2- 172.21.15.159:0/2698454288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2840af830 0x7fe2840afff0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:10:59.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.411+0000 7fe28c800640 1 -- 172.21.15.159:0/2698454288 >> 172.21.15.159:0/2698454288 conn(0x7fe28406b990 msgr2=0x7fe28415b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:10:59.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.411+0000 7fe28c800640 1 -- 172.21.15.159:0/2698454288 shutdown_connections 2024-04-04T06:10:59.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:10:59.411+0000 7fe28c800640 1 -- 172.21.15.159:0/2698454288 wait complete. 2024-04-04T06:10:59.825 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:10:44.805617+0000', 'modified': '2024-04-04T06:10:58.944955+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 72, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6828}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6828': {'gid': 6828, 'name': 'a', 'rank': 0, 'incarnation': 48, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.159:6835/2467594024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 2467594024}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 2467594024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-04T06:10:59.825 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6828:up:replay 2024-04-04T06:11:00.826 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:11:01.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.052+0000 7fd6dcc00640 1 Processor -- start 2024-04-04T06:11:01.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.052+0000 7fd6dcc00640 1 -- start start 2024-04-04T06:11:01.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.054+0000 7fd6dcc00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd6d815c390 0x7fd6d8073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:01.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.054+0000 7fd6dcc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6d80d0650 0x7fd6d80d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:01.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.054+0000 7fd6dcc00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6d815fe80 con 0x7fd6d80af830 2024-04-04T06:11:01.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.054+0000 7fd6dcc00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd6d815c770 con 0x7fd6d815c390 2024-04-04T06:11:01.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.054+0000 7fd6dcc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d80ad120 con 0x7fd6d80d0650 2024-04-04T06:11:01.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.055+0000 7fd6d5e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd6d815c390 0x7fd6d8073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:01.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.055+0000 7fd6d6800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6d80d0650 0x7fd6d80d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:01.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.055+0000 7fd6d6800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6d80d0650 0x7fd6d80d0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:51180/0 (socket says 172.21.15.159:51180) 2024-04-04T06:11:01.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.055+0000 7fd6d6800640 1 -- 172.21.15.159:0/2712107292 learned_addr learned my addr 172.21.15.159:0/2712107292 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:01.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.055+0000 7fd6d5e00640 1 -- 172.21.15.159:0/2712107292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6d80d0650 msgr2=0x7fd6d80d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:01.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.055+0000 7fd6d5e00640 1 --2- 172.21.15.159:0/2712107292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6d80d0650 0x7fd6d80d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:01.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.055+0000 7fd6d5e00640 1 -- 172.21.15.159:0/2712107292 >> v1:172.21.15.178:6789/0 conn(0x7fd6d80af830 legacy=0x7fd6d8160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:01.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.055+0000 7fd6d5e00640 1 -- 172.21.15.159:0/2712107292 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d80ad400 con 0x7fd6d815c390 2024-04-04T06:11:01.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.055+0000 7fd6d6800640 1 --2- 172.21.15.159:0/2712107292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6d80d0650 0x7fd6d80d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:11:01.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.057+0000 7fd6d5e00640 1 --2- 172.21.15.159:0/2712107292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6d815c390 0x7fd6d8073790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80097a0 tx=0x7fd6c802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8a4e5baa13b9196 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:01.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.057+0000 7fd6d5400640 1 -- 172.21.15.159:0/2712107292 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8035020 con 0x7fd6d815c390 2024-04-04T06:11:01.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.057+0000 7fd6d5400640 1 -- 172.21.15.159:0/2712107292 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6c8031830 con 0x7fd6d815c390 2024-04-04T06:11:01.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.057+0000 7fd6d5400640 1 -- 172.21.15.159:0/2712107292 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8031c00 con 0x7fd6d815c390 2024-04-04T06:11:01.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.058+0000 7fd6dcc00640 1 -- 172.21.15.159:0/2712107292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6d815c390 msgr2=0x7fd6d8073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:01.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.058+0000 7fd6dcc00640 1 --2- 172.21.15.159:0/2712107292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6d815c390 0x7fd6d8073790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80097a0 tx=0x7fd6c802b8d0 comp rx=0 tx=0).stop 2024-04-04T06:11:01.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.058+0000 7fd6dcc00640 1 -- 172.21.15.159:0/2712107292 shutdown_connections 2024-04-04T06:11:01.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.058+0000 7fd6dcc00640 1 --2- 172.21.15.159:0/2712107292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6d80d0650 0x7fd6d80d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:01.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.058+0000 7fd6dcc00640 1 --2- 172.21.15.159:0/2712107292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6d815c390 0x7fd6d8073790 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:01.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.058+0000 7fd6dcc00640 1 -- 172.21.15.159:0/2712107292 >> 172.21.15.159:0/2712107292 conn(0x7fd6d806b990 msgr2=0x7fd6d806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:01.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.058+0000 7fd6dcc00640 1 -- 172.21.15.159:0/2712107292 shutdown_connections 2024-04-04T06:11:01.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.058+0000 7fd6dcc00640 1 -- 172.21.15.159:0/2712107292 wait complete. 2024-04-04T06:11:01.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.060+0000 7fd6dcc00640 1 Processor -- start 2024-04-04T06:11:01.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.061+0000 7fd6dcc00640 1 -- start start 2024-04-04T06:11:01.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.062+0000 7fd6dcc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6d80af830 0x7fd6d814ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:01.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.062+0000 7fd6dcc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6d80d0650 0x7fd6d814c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:01.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.062+0000 7fd6dcc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6d815c390 0x7fd6d815f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:01.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.062+0000 7fd6dcc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d8160d50 con 0x7fd6d80af830 2024-04-04T06:11:01.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.062+0000 7fd6dcc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d8160120 con 0x7fd6d80d0650 2024-04-04T06:11:01.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.062+0000 7fd6dcc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d80cf980 con 0x7fd6d815c390 2024-04-04T06:11:01.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.062+0000 7fd6d5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6d80af830 0x7fd6d814ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:01.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.062+0000 7fd6d5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6d80af830 0x7fd6d814ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42698/0 (socket says 172.21.15.159:42698) 2024-04-04T06:11:01.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.062+0000 7fd6d5e00640 1 -- 172.21.15.159:0/1954042214 learned_addr learned my addr 172.21.15.159:0/1954042214 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:01.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.062+0000 7fd6cd400640 1 --2- 172.21.15.159:0/1954042214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6d80d0650 0x7fd6d814c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:01.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.063+0000 7fd6d6800640 1 --2- 172.21.15.159:0/1954042214 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6d815c390 0x7fd6d815f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:01.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.063+0000 7fd6d5e00640 1 -- 172.21.15.159:0/1954042214 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6d815c390 msgr2=0x7fd6d815f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:01.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.063+0000 7fd6d5e00640 1 --2- 172.21.15.159:0/1954042214 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6d815c390 0x7fd6d815f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:01.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.063+0000 7fd6d5e00640 1 -- 172.21.15.159:0/1954042214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6d80d0650 msgr2=0x7fd6d814c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:01.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.063+0000 7fd6d5e00640 1 --2- 172.21.15.159:0/1954042214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6d80d0650 0x7fd6d814c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:01.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.063+0000 7fd6d5e00640 1 -- 172.21.15.159:0/1954042214 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d8151340 con 0x7fd6d80af830 2024-04-04T06:11:01.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.063+0000 7fd6d6800640 1 --2- 172.21.15.159:0/1954042214 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6d815c390 0x7fd6d815f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:11:01.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.063+0000 7fd6d5e00640 1 --2- 172.21.15.159:0/1954042214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6d80af830 0x7fd6d814ac40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009890 tx=0x7fd6c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:01.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.063+0000 7fd6cea00640 1 -- 172.21.15.159:0/1954042214 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8035040 con 0x7fd6d80af830 2024-04-04T06:11:01.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.063+0000 7fd6cea00640 1 -- 172.21.15.159:0/1954042214 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6c803b920 con 0x7fd6d80af830 2024-04-04T06:11:01.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.063+0000 7fd6cea00640 1 -- 172.21.15.159:0/1954042214 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c803bd10 con 0x7fd6d80af830 2024-04-04T06:11:01.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.063+0000 7fd6dcc00640 1 -- 172.21.15.159:0/1954042214 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d815c9a0 con 0x7fd6d80af830 2024-04-04T06:11:01.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.064+0000 7fd6dcc00640 1 -- 172.21.15.159:0/1954042214 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d815cf60 con 0x7fd6d80af830 2024-04-04T06:11:01.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.065+0000 7fd6dcc00640 1 -- 172.21.15.159:0/1954042214 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d80685c0 con 0x7fd6d80af830 2024-04-04T06:11:01.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.066+0000 7fd6cea00640 1 -- 172.21.15.159:0/1954042214 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6c8038020 con 0x7fd6d80af830 2024-04-04T06:11:01.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.066+0000 7fd6cea00640 1 --2- 172.21.15.159:0/1954042214 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd6b406c8f0 0x7fd6b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:01.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.067+0000 7fd6cea00640 1 -- 172.21.15.159:0/1954042214 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7fd6c80add80 con 0x7fd6d80af830 2024-04-04T06:11:01.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.067+0000 7fd6cd400640 1 --2- 172.21.15.159:0/1954042214 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd6b406c8f0 0x7fd6b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:01.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.068+0000 7fd6cd400640 1 --2- 172.21.15.159:0/1954042214 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd6b406c8f0 0x7fd6b406edb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0007fd0 tx=0x7fd6c0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:01.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.073+0000 7fd6cea00640 1 -- 172.21.15.159:0/1954042214 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd6c807c0c0 con 0x7fd6d80af830 2024-04-04T06:11:01.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.311+0000 7fd6dcc00640 1 -- 172.21.15.159:0/1954042214 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6d806f1b0 con 0x7fd6d80af830 2024-04-04T06:11:01.305 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.313+0000 7fd6cea00640 1 -- 172.21.15.159:0/1954042214 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5186 (secure 0 0 0) 0x7fd6c8031800 con 0x7fd6d80af830 2024-04-04T06:11:01.305 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:11:01.305 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3789513463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3789513463},{"type":"v1","addr":"172.21.15.178:6835","nonce":3789513463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/468875883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":468875883},{"type":"v1","addr":"172.21.15.178:6837","nonce":468875883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2546974742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2546974742},{"type":"v1","addr":"172.21.15.159:6833","nonce":2546974742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:10:44.805617+0000","modified":"2024-04-04T06:10:59.956886+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6828":{"gid":6828,"name":"a","rank":0,"incarnation":48,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.159:6835/2467594024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2467594024},{"type":"v1","addr":"172.21.15.159:6835","nonce":2467594024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-04-04T06:11:01.305 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 50 2024-04-04T06:11:01.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.315+0000 7fd6dcc00640 1 -- 172.21.15.159:0/1954042214 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd6b406c8f0 msgr2=0x7fd6b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:01.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.315+0000 7fd6dcc00640 1 --2- 172.21.15.159:0/1954042214 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd6b406c8f0 0x7fd6b406edb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0007fd0 tx=0x7fd6c0009040 comp rx=0 tx=0).stop 2024-04-04T06:11:01.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.315+0000 7fd6dcc00640 1 -- 172.21.15.159:0/1954042214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6d80af830 msgr2=0x7fd6d814ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:01.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.315+0000 7fd6dcc00640 1 --2- 172.21.15.159:0/1954042214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6d80af830 0x7fd6d814ac40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009890 tx=0x7fd6c80046c0 comp rx=0 tx=0).stop 2024-04-04T06:11:01.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.316+0000 7fd6dcc00640 1 -- 172.21.15.159:0/1954042214 shutdown_connections 2024-04-04T06:11:01.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.316+0000 7fd6dcc00640 1 --2- 172.21.15.159:0/1954042214 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6d815c390 0x7fd6d815f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:01.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.316+0000 7fd6dcc00640 1 --2- 172.21.15.159:0/1954042214 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd6b406c8f0 0x7fd6b406edb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:01.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.316+0000 7fd6dcc00640 1 --2- 172.21.15.159:0/1954042214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6d80d0650 0x7fd6d814c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:01.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.316+0000 7fd6dcc00640 1 --2- 172.21.15.159:0/1954042214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6d80af830 0x7fd6d814ac40 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:01.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.316+0000 7fd6dcc00640 1 -- 172.21.15.159:0/1954042214 >> 172.21.15.159:0/1954042214 conn(0x7fd6d806b990 msgr2=0x7fd6d80735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:01.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.316+0000 7fd6dcc00640 1 -- 172.21.15.159:0/1954042214 shutdown_connections 2024-04-04T06:11:01.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:01.316+0000 7fd6dcc00640 1 -- 172.21.15.159:0/1954042214 wait complete. 2024-04-04T06:11:01.729 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:10:44.805617+0000', 'modified': '2024-04-04T06:10:59.956886+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 72, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6828}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6828': {'gid': 6828, 'name': 'a', 'rank': 0, 'incarnation': 48, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.159:6835/2467594024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 2467594024}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 2467594024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-04T06:11:01.729 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6828:up:rejoin 2024-04-04T06:11:02.730 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:11:02.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.950+0000 7fe1b9e00640 1 Processor -- start 2024-04-04T06:11:02.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.951+0000 7fe1b9e00640 1 -- start start 2024-04-04T06:11:02.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.952+0000 7fe1b9e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe1b415e4a0 0x7fe1b40b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:02.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.952+0000 7fe1b9e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1b415e880 con 0x7fe1b40cf700 2024-04-04T06:11:02.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.952+0000 7fe1b9e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1b40d28d0 con 0x7fe1b40d2310 2024-04-04T06:11:02.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.952+0000 7fe1b9e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe1b40d34e0 con 0x7fe1b415e4a0 2024-04-04T06:11:02.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.952+0000 7fe1b9400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fe1b40cf700 0x7fe1b40b73e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:43508/0 (socket says 172.21.15.159:43508) 2024-04-04T06:11:02.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.953+0000 7fe1b9400640 1 -- 172.21.15.159:0/1023442990 learned_addr learned my addr 172.21.15.159:0/1023442990 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:11:02.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.953+0000 7fe1b8a00640 1 --2- 172.21.15.159:0/1023442990 >> v2:172.21.15.178:3300/0 conn(0x7fe1b415e4a0 0x7fe1b40b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:02.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.953+0000 7fe1b3400640 1 -- 172.21.15.159:0/1023442990 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2654390954 0 0) 0x7fe1b415e880 con 0x7fe1b40cf700 2024-04-04T06:11:02.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.953+0000 7fe1b3400640 1 -- 172.21.15.159:0/1023442990 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe19c003520 con 0x7fe1b40cf700 2024-04-04T06:11:02.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.953+0000 7fe1b3400640 1 -- 172.21.15.159:0/1023442990 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2673790721 0 0) 0x7fe19c003520 con 0x7fe1b40cf700 2024-04-04T06:11:02.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.953+0000 7fe1b3400640 1 -- 172.21.15.159:0/1023442990 >> v2:172.21.15.178:3300/0 conn(0x7fe1b415e4a0 msgr2=0x7fe1b40b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:02.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.953+0000 7fe1b3400640 1 --2- 172.21.15.159:0/1023442990 >> v2:172.21.15.178:3300/0 conn(0x7fe1b415e4a0 0x7fe1b40b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:02.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.953+0000 7fe1b3400640 1 -- 172.21.15.159:0/1023442990 >> v1:172.21.15.178:6789/0 conn(0x7fe1b40d2310 legacy=0x7fe1b40ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:02.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.953+0000 7fe1b3400640 1 -- 172.21.15.159:0/1023442990 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b40d2bb0 con 0x7fe1b40cf700 2024-04-04T06:11:02.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.953+0000 7fe1b3400640 1 -- 172.21.15.159:0/1023442990 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fe1ac001a10 con 0x7fe1b40cf700 2024-04-04T06:11:02.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.953+0000 7fe1b3400640 1 -- 172.21.15.159:0/1023442990 >> v1:172.21.15.159:6789/0 conn(0x7fe1b40cf700 legacy=0x7fe1b40b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:02.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.954+0000 7fe1b3400640 1 --2- 172.21.15.159:0/1023442990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c003a00 0x7fe19c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:02.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.954+0000 7fe1b3400640 1 --2- 172.21.15.159:0/1023442990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe19c004800 0x7fe19c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:02.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.954+0000 7fe1b3400640 1 --2- 172.21.15.159:0/1023442990 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe19c005210 0x7fe19c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:02.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.954+0000 7fe1b3400640 1 -- 172.21.15.159:0/1023442990 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b40d34e0 con 0x7fe19c003a00 2024-04-04T06:11:02.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.954+0000 7fe1b3400640 1 -- 172.21.15.159:0/1023442990 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c00c160 con 0x7fe19c004800 2024-04-04T06:11:02.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.954+0000 7fe1b3400640 1 -- 172.21.15.159:0/1023442990 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe19c00c360 con 0x7fe19c005210 2024-04-04T06:11:02.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.954+0000 7fe1b8a00640 1 --2- 172.21.15.159:0/1023442990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c003a00 0x7fe19c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:02.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.954+0000 7fe1b8a00640 1 -- 172.21.15.159:0/1023442990 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe19c005210 msgr2=0x7fe19c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:02.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.954+0000 7fe1b3e00640 1 --2- 172.21.15.159:0/1023442990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe19c004800 0x7fe19c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:02.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.954+0000 7fe1b9400640 1 --2- 172.21.15.159:0/1023442990 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe19c005210 0x7fe19c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:02.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.954+0000 7fe1b8a00640 1 --2- 172.21.15.159:0/1023442990 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe19c005210 0x7fe19c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:02.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.954+0000 7fe1b8a00640 1 -- 172.21.15.159:0/1023442990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe19c004800 msgr2=0x7fe19c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:02.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.954+0000 7fe1b8a00640 1 --2- 172.21.15.159:0/1023442990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe19c004800 0x7fe19c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:02.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.954+0000 7fe1b8a00640 1 -- 172.21.15.159:0/1023442990 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19c003520 con 0x7fe19c003a00 2024-04-04T06:11:02.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.954+0000 7fe1b9400640 1 -- 172.21.15.159:0/1023442990 reap_dead start 2024-04-04T06:11:02.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.956+0000 7fe1b8a00640 1 --2- 172.21.15.159:0/1023442990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c003a00 0x7fe19c003e20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009a90 tx=0x7fe19c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7649d04676ec0975 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:02.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.957+0000 7fe1b3400640 1 -- 172.21.15.159:0/1023442990 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8039070 con 0x7fe19c003a00 2024-04-04T06:11:02.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.957+0000 7fe1b3400640 1 -- 172.21.15.159:0/1023442990 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1a802c6b0 con 0x7fe19c003a00 2024-04-04T06:11:02.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.957+0000 7fe1b3400640 1 -- 172.21.15.159:0/1023442990 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8009690 con 0x7fe19c003a00 2024-04-04T06:11:02.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.957+0000 7fe1b9e00640 1 -- 172.21.15.159:0/1023442990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c003a00 msgr2=0x7fe19c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:02.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.957+0000 7fe1b9e00640 1 --2- 172.21.15.159:0/1023442990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c003a00 0x7fe19c003e20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009a90 tx=0x7fe19c005830 comp rx=0 tx=0).stop 2024-04-04T06:11:02.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.957+0000 7fe1b9e00640 1 -- 172.21.15.159:0/1023442990 shutdown_connections 2024-04-04T06:11:02.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.957+0000 7fe1b9e00640 1 --2- 172.21.15.159:0/1023442990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c003a00 0x7fe19c003e20 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:02.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.957+0000 7fe1b9e00640 1 -- 172.21.15.159:0/1023442990 >> 172.21.15.159:0/1023442990 conn(0x7fe1b40676d0 msgr2=0x7fe1b4067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:02.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.958+0000 7fe1b9e00640 1 -- 172.21.15.159:0/1023442990 shutdown_connections 2024-04-04T06:11:02.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.958+0000 7fe1b9e00640 1 -- 172.21.15.159:0/1023442990 wait complete. 2024-04-04T06:11:02.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.960+0000 7fe1b9e00640 1 Processor -- start 2024-04-04T06:11:02.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.960+0000 7fe1b9e00640 1 -- start start 2024-04-04T06:11:02.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.961+0000 7fe1b9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c003a00 0x7fe1b40c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:02.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.961+0000 7fe1b9e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1b415ea80 0x7fe1b415eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:02.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.961+0000 7fe1b9e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1b40c94e0 0x7fe1b40c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:02.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.961+0000 7fe1b9e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b40d3970 con 0x7fe19c003a00 2024-04-04T06:11:02.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.961+0000 7fe1b9e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b40d3170 con 0x7fe1b415ea80 2024-04-04T06:11:02.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.961+0000 7fe1b9e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe1b40ceff0 con 0x7fe1b40c94e0 2024-04-04T06:11:02.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.961+0000 7fe1b8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c003a00 0x7fe1b40c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:02.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.961+0000 7fe1b8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c003a00 0x7fe1b40c75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42720/0 (socket says 172.21.15.159:42720) 2024-04-04T06:11:02.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.961+0000 7fe1b8a00640 1 -- 172.21.15.159:0/1875007351 learned_addr learned my addr 172.21.15.159:0/1875007351 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:02.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.961+0000 7fe1b3e00640 1 --2- 172.21.15.159:0/1875007351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1b415ea80 0x7fe1b415eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:02.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.961+0000 7fe1b9400640 1 --2- 172.21.15.159:0/1875007351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1b40c94e0 0x7fe1b40c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:02.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.962+0000 7fe1b8a00640 1 -- 172.21.15.159:0/1875007351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1b40c94e0 msgr2=0x7fe1b40c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:02.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.962+0000 7fe1b8a00640 1 --2- 172.21.15.159:0/1875007351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1b40c94e0 0x7fe1b40c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:02.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.962+0000 7fe1b8a00640 1 -- 172.21.15.159:0/1875007351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1b415ea80 msgr2=0x7fe1b415eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:02.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.962+0000 7fe1b8a00640 1 --2- 172.21.15.159:0/1875007351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1b415ea80 0x7fe1b415eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:02.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.962+0000 7fe1b8a00640 1 -- 172.21.15.159:0/1875007351 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b415f4e0 con 0x7fe19c003a00 2024-04-04T06:11:02.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.962+0000 7fe1b8a00640 1 --2- 172.21.15.159:0/1875007351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c003a00 0x7fe1b40c75a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009a90 tx=0x7fe1a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:02.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.962+0000 7fe1b0c00640 1 -- 172.21.15.159:0/1875007351 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8039050 con 0x7fe19c003a00 2024-04-04T06:11:02.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.962+0000 7fe1b0c00640 1 -- 172.21.15.159:0/1875007351 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1a8045070 con 0x7fe19c003a00 2024-04-04T06:11:02.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.962+0000 7fe1b9e00640 1 -- 172.21.15.159:0/1875007351 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b415f7c0 con 0x7fe19c003a00 2024-04-04T06:11:02.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.962+0000 7fe1b0c00640 1 -- 172.21.15.159:0/1875007351 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a80044e0 con 0x7fe19c003a00 2024-04-04T06:11:02.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.963+0000 7fe1b9e00640 1 -- 172.21.15.159:0/1875007351 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b415fd80 con 0x7fe19c003a00 2024-04-04T06:11:02.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.964+0000 7fe1b9e00640 1 -- 172.21.15.159:0/1875007351 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1b40c9ea0 con 0x7fe19c003a00 2024-04-04T06:11:02.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.965+0000 7fe1b0c00640 1 -- 172.21.15.159:0/1875007351 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe1a8003a30 con 0x7fe19c003a00 2024-04-04T06:11:02.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.966+0000 7fe1b0c00640 1 --2- 172.21.15.159:0/1875007351 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe19406c8f0 0x7fe19406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:02.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.966+0000 7fe1b0c00640 1 -- 172.21.15.159:0/1875007351 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7fe1a80ada90 con 0x7fe19c003a00 2024-04-04T06:11:02.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.966+0000 7fe1b3e00640 1 --2- 172.21.15.159:0/1875007351 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe19406c8f0 0x7fe19406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:02.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.967+0000 7fe1b3e00640 1 --2- 172.21.15.159:0/1875007351 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe19406c8f0 0x7fe19406edb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0000c00 tx=0x7fe1a0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:02.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:02.971+0000 7fe1b0c00640 1 -- 172.21.15.159:0/1875007351 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe1a804f020 con 0x7fe19c003a00 2024-04-04T06:11:03.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.210+0000 7fe1b9e00640 1 -- 172.21.15.159:0/1875007351 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1b4075940 con 0x7fe19c003a00 2024-04-04T06:11:03.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.210+0000 7fe1b0c00640 1 -- 172.21.15.159:0/1875007351 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5193 (secure 0 0 0) 0x7fe1a807bef0 con 0x7fe19c003a00 2024-04-04T06:11:03.204 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:11:03.204 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3789513463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3789513463},{"type":"v1","addr":"172.21.15.178:6835","nonce":3789513463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/468875883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":468875883},{"type":"v1","addr":"172.21.15.178:6837","nonce":468875883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2546974742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2546974742},{"type":"v1","addr":"172.21.15.159:6833","nonce":2546974742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:10:44.805617+0000","modified":"2024-04-04T06:11:01.956681+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6828":{"gid":6828,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6835/2467594024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2467594024},{"type":"v1","addr":"172.21.15.159:6835","nonce":2467594024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6828,"qdb_cluster":[6828]},"id":5}]} 2024-04-04T06:11:03.204 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 51 2024-04-04T06:11:03.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.213+0000 7fe1b9e00640 1 -- 172.21.15.159:0/1875007351 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe19406c8f0 msgr2=0x7fe19406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:03.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.213+0000 7fe1b9e00640 1 --2- 172.21.15.159:0/1875007351 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe19406c8f0 0x7fe19406edb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0000c00 tx=0x7fe1a0009040 comp rx=0 tx=0).stop 2024-04-04T06:11:03.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.213+0000 7fe1b9e00640 1 -- 172.21.15.159:0/1875007351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c003a00 msgr2=0x7fe1b40c75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:03.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.213+0000 7fe1b9e00640 1 --2- 172.21.15.159:0/1875007351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c003a00 0x7fe1b40c75a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009a90 tx=0x7fe1a8004730 comp rx=0 tx=0).stop 2024-04-04T06:11:03.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.213+0000 7fe1b9e00640 1 -- 172.21.15.159:0/1875007351 shutdown_connections 2024-04-04T06:11:03.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.213+0000 7fe1b9e00640 1 --2- 172.21.15.159:0/1875007351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe1b40c94e0 0x7fe1b40c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.213+0000 7fe1b9e00640 1 --2- 172.21.15.159:0/1875007351 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe19406c8f0 0x7fe19406edb0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.213+0000 7fe1b9e00640 1 --2- 172.21.15.159:0/1875007351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1b415ea80 0x7fe1b415eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.213+0000 7fe1b9e00640 1 --2- 172.21.15.159:0/1875007351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c003a00 0x7fe1b40c75a0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.213+0000 7fe1b9e00640 1 -- 172.21.15.159:0/1875007351 >> 172.21.15.159:0/1875007351 conn(0x7fe1b40676d0 msgr2=0x7fe1b40d3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:03.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.213+0000 7fe1b9e00640 1 -- 172.21.15.159:0/1875007351 shutdown_connections 2024-04-04T06:11:03.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.213+0000 7fe1b9e00640 1 -- 172.21.15.159:0/1875007351 wait complete. 2024-04-04T06:11:03.626 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:10:44.805617+0000', 'modified': '2024-04-04T06:11:01.956681+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 72, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6828}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6828': {'gid': 6828, 'name': 'a', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.159:6835/2467594024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 2467594024}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 2467594024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6828, 'qdb_cluster': [6828]} 2024-04-04T06:11:03.626 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-04T06:11:03.626 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-04T06:11:03.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.847+0000 7f66aac00640 1 Processor -- start 2024-04-04T06:11:03.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.847+0000 7f66aac00640 1 -- start start 2024-04-04T06:11:03.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.849+0000 7f66aac00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f66a4155550 0x7f66a4155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:03.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.849+0000 7f66aac00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f66a415f800 0x7f66a40d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:03.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.849+0000 7f66aac00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66a40b40b0 con 0x7f66a40d07c0 2024-04-04T06:11:03.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.849+0000 7f66aac00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f66a40d0bf0 con 0x7f66a4155550 2024-04-04T06:11:03.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.849+0000 7f66aac00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f66a40b4390 con 0x7f66a415f800 2024-04-04T06:11:03.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.849+0000 7f66a3e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f66a4155550 0x7f66a4155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:03.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.849+0000 7f66a3e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f66a4155550 0x7f66a4155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44612/0 (socket says 172.21.15.159:44612) 2024-04-04T06:11:03.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.849+0000 7f66a3e00640 1 -- 172.21.15.159:0/1179196557 learned_addr learned my addr 172.21.15.159:0/1179196557 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:03.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.849+0000 7f66a3400640 1 --2- 172.21.15.159:0/1179196557 >> v2:172.21.15.178:3300/0 conn(0x7f66a415f800 0x7f66a40d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:03.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.850+0000 7f66a3e00640 1 -- 172.21.15.159:0/1179196557 >> v2:172.21.15.178:3300/0 conn(0x7f66a415f800 msgr2=0x7f66a40d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:03.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.850+0000 7f66a3e00640 1 --2- 172.21.15.159:0/1179196557 >> v2:172.21.15.178:3300/0 conn(0x7f66a415f800 0x7f66a40d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.850+0000 7f66a3e00640 1 -- 172.21.15.159:0/1179196557 >> v1:172.21.15.178:6789/0 conn(0x7f66a40d07c0 legacy=0x7f66a40aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:03.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.850+0000 7f66a3e00640 1 -- 172.21.15.159:0/1179196557 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a40b4670 con 0x7f66a4155550 2024-04-04T06:11:03.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.850+0000 7f66a3400640 1 --2- 172.21.15.159:0/1179196557 >> v2:172.21.15.178:3300/0 conn(0x7f66a415f800 0x7f66a40d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:11:03.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.851+0000 7f66a3e00640 1 --2- 172.21.15.159:0/1179196557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66a4155550 0x7f66a4155930 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f66a40d0dd0 tx=0x7f669402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f78b12f1bc5668 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:03.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.851+0000 7f66a2a00640 1 -- 172.21.15.159:0/1179196557 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669402e070 con 0x7f66a4155550 2024-04-04T06:11:03.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.851+0000 7f66a2a00640 1 -- 172.21.15.159:0/1179196557 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6694037070 con 0x7f66a4155550 2024-04-04T06:11:03.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.851+0000 7f66a2a00640 1 -- 172.21.15.159:0/1179196557 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694031a00 con 0x7f66a4155550 2024-04-04T06:11:03.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.852+0000 7f66aac00640 1 -- 172.21.15.159:0/1179196557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66a4155550 msgr2=0x7f66a4155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:03.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.852+0000 7f66aac00640 1 --2- 172.21.15.159:0/1179196557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66a4155550 0x7f66a4155930 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f66a40d0dd0 tx=0x7f669402b880 comp rx=0 tx=0).stop 2024-04-04T06:11:03.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.852+0000 7f66aac00640 1 -- 172.21.15.159:0/1179196557 shutdown_connections 2024-04-04T06:11:03.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.852+0000 7f66aac00640 1 --2- 172.21.15.159:0/1179196557 >> v2:172.21.15.178:3300/0 conn(0x7f66a415f800 0x7f66a40d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.852+0000 7f66aac00640 1 --2- 172.21.15.159:0/1179196557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66a4155550 0x7f66a4155930 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.852+0000 7f66aac00640 1 -- 172.21.15.159:0/1179196557 >> 172.21.15.159:0/1179196557 conn(0x7f66a406b750 msgr2=0x7f66a406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:03.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.852+0000 7f66aac00640 1 -- 172.21.15.159:0/1179196557 shutdown_connections 2024-04-04T06:11:03.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.852+0000 7f66aac00640 1 -- 172.21.15.159:0/1179196557 wait complete. 2024-04-04T06:11:03.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.855+0000 7f66aac00640 1 Processor -- start 2024-04-04T06:11:03.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.855+0000 7f66aac00640 1 -- start start 2024-04-04T06:11:03.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.856+0000 7f66aac00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a40d07c0 0x7f66a4167190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:03.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.856+0000 7f66aac00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66a4155550 0x7f66a415a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:03.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.856+0000 7f66aac00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66a415f800 0x7f66a4168a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:03.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.856+0000 7f66aac00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f66a415fc80 con 0x7f66a4155550 2024-04-04T06:11:03.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.856+0000 7f66aac00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f66a40b4c30 con 0x7f66a40d07c0 2024-04-04T06:11:03.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.856+0000 7f66aac00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f66a415ba20 con 0x7f66a415f800 2024-04-04T06:11:03.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.857+0000 7f66a3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66a4155550 0x7f66a415a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:03.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.857+0000 7f66a3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66a4155550 0x7f66a415a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44628/0 (socket says 172.21.15.159:44628) 2024-04-04T06:11:03.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.857+0000 7f66a3400640 1 -- 172.21.15.159:0/3972043597 learned_addr learned my addr 172.21.15.159:0/3972043597 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:03.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.857+0000 7f66a3e00640 1 --2- 172.21.15.159:0/3972043597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a40d07c0 0x7f66a4167190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:03.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.857+0000 7f66a8c00640 1 --2- 172.21.15.159:0/3972043597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66a415f800 0x7f66a4168a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:03.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.857+0000 7f66a3400640 1 -- 172.21.15.159:0/3972043597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66a415f800 msgr2=0x7f66a4168a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:03.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.857+0000 7f66a3400640 1 --2- 172.21.15.159:0/3972043597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66a415f800 0x7f66a4168a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.857+0000 7f66a3400640 1 -- 172.21.15.159:0/3972043597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a40d07c0 msgr2=0x7f66a4167190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:03.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.857+0000 7f66a3400640 1 --2- 172.21.15.159:0/3972043597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a40d07c0 0x7f66a4167190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.857+0000 7f66a3400640 1 -- 172.21.15.159:0/3972043597 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a41679e0 con 0x7f66a4155550 2024-04-04T06:11:03.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.858+0000 7f66a3400640 1 --2- 172.21.15.159:0/3972043597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66a4155550 0x7f66a415a540 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f669000c7f0 tx=0x7f669000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:03.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.858+0000 7f669be00640 1 -- 172.21.15.159:0/3972043597 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690015070 con 0x7f66a4155550 2024-04-04T06:11:03.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.858+0000 7f669be00640 1 -- 172.21.15.159:0/3972043597 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6690007a30 con 0x7f66a4155550 2024-04-04T06:11:03.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.858+0000 7f669be00640 1 -- 172.21.15.159:0/3972043597 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690007e00 con 0x7f66a4155550 2024-04-04T06:11:03.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.858+0000 7f66aac00640 1 -- 172.21.15.159:0/3972043597 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a4167cc0 con 0x7f66a4155550 2024-04-04T06:11:03.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.858+0000 7f66aac00640 1 -- 172.21.15.159:0/3972043597 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a4168280 con 0x7f66a4155550 2024-04-04T06:11:03.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.861+0000 7f669be00640 1 -- 172.21.15.159:0/3972043597 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6690018070 con 0x7f66a4155550 2024-04-04T06:11:03.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.862+0000 7f669be00640 1 --2- 172.21.15.159:0/3972043597 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f668406c8f0 0x7f668406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:03.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.862+0000 7f669be00640 1 -- 172.21.15.159:0/3972043597 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f669008bb90 con 0x7f66a4155550 2024-04-04T06:11:03.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.863+0000 7f66a3e00640 1 --2- 172.21.15.159:0/3972043597 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f668406c8f0 0x7f668406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:03.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.863+0000 7f66ad9ae740 1 Processor -- start 2024-04-04T06:11:03.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.863+0000 7f66ad9ae740 1 -- start start 2024-04-04T06:11:03.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.864+0000 7f66ad9ae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa1d130 0x5600cf9f00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:03.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.864+0000 7f66a3e00640 1 --2- 172.21.15.159:0/3972043597 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f668406c8f0 0x7f668406edb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6694036000 tx=0x7f669403d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:03.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.864+0000 7f66ad9ae740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5600cfa1d510 0x5600cfa1fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:03.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.864+0000 7f66ad9ae740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600cf9f0610 0x5600cfa20400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:03.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.864+0000 7f66ad9ae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5600cf9b7ab0 con 0x5600cfa1d130 2024-04-04T06:11:03.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.864+0000 7f66ad9ae740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5600cf959fa0 con 0x5600cf9f0610 2024-04-04T06:11:03.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.864+0000 7f66ad9ae740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5600cf902560 con 0x5600cfa1d510 2024-04-04T06:11:03.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.864+0000 7f66a3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa1d130 0x5600cf9f00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:03.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.864+0000 7f66a3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa1d130 0x5600cf9f00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44638/0 (socket says 172.21.15.159:44638) 2024-04-04T06:11:03.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.864+0000 7f66a3e00640 1 -- 172.21.15.159:0/2957171461 learned_addr learned my addr 172.21.15.159:0/2957171461 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:03.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.865+0000 7f66a8c00640 1 --2- 172.21.15.159:0/2957171461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600cf9f0610 0x5600cfa20400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:03.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.865+0000 7f66a3e00640 1 -- 172.21.15.159:0/2957171461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5600cfa1d510 msgr2=0x5600cfa1fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:03.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.865+0000 7f66a3400640 1 --2- 172.21.15.159:0/2957171461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5600cfa1d510 0x5600cfa1fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:03.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.865+0000 7f66a3e00640 1 --2- 172.21.15.159:0/2957171461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5600cfa1d510 0x5600cfa1fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.865+0000 7f66a3e00640 1 -- 172.21.15.159:0/2957171461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600cf9f0610 msgr2=0x5600cfa20400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:03.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.865+0000 7f66a3e00640 1 --2- 172.21.15.159:0/2957171461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600cf9f0610 0x5600cfa20400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.865+0000 7f66a3e00640 1 -- 172.21.15.159:0/2957171461 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600cf66e2d0 con 0x5600cfa1d130 2024-04-04T06:11:03.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.865+0000 7f66a3e00640 1 --2- 172.21.15.159:0/2957171461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa1d130 0x5600cf9f00d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f669402be90 tx=0x7f6694030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d09b1397a889a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:03.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.865+0000 7f668fe00640 1 -- 172.21.15.159:0/2957171461 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694053070 con 0x5600cfa1d130 2024-04-04T06:11:03.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.865+0000 7f668fe00640 1 -- 172.21.15.159:0/2957171461 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6694037040 con 0x5600cfa1d130 2024-04-04T06:11:03.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.865+0000 7f668fe00640 1 -- 172.21.15.159:0/2957171461 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694032b40 con 0x5600cfa1d130 2024-04-04T06:11:03.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.866+0000 7f66ad9ae740 1 -- 172.21.15.159:0/2957171461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa1d130 msgr2=0x5600cf9f00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:03.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.866+0000 7f66ad9ae740 1 --2- 172.21.15.159:0/2957171461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa1d130 0x5600cf9f00d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f669402be90 tx=0x7f6694030000 comp rx=0 tx=0).stop 2024-04-04T06:11:03.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.866+0000 7f66ad9ae740 1 -- 172.21.15.159:0/2957171461 shutdown_connections 2024-04-04T06:11:03.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.866+0000 7f66ad9ae740 1 --2- 172.21.15.159:0/2957171461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600cf9f0610 0x5600cfa20400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.866+0000 7f66ad9ae740 1 --2- 172.21.15.159:0/2957171461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5600cfa1d510 0x5600cfa1fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.866+0000 7f66ad9ae740 1 --2- 172.21.15.159:0/2957171461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa1d130 0x5600cf9f00d0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.866+0000 7f66ad9ae740 1 -- 172.21.15.159:0/2957171461 >> 172.21.15.159:0/2957171461 conn(0x5600cf9f2340 msgr2=0x5600cfa3aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:03.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.866+0000 7f66ad9ae740 1 -- 172.21.15.159:0/2957171461 shutdown_connections 2024-04-04T06:11:03.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.866+0000 7f66ad9ae740 1 -- 172.21.15.159:0/2957171461 wait complete. 2024-04-04T06:11:03.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.867+0000 7f66ad9ae740 1 -- start start 2024-04-04T06:11:03.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.867+0000 7f66ad9ae740 1 Processor -- start 2024-04-04T06:11:03.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.868+0000 7f66ad9ae740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600cfa1d130 0x5600cfa4bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:03.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.868+0000 7f66ad9ae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa1d510 0x5600cfa3b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:03.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.868+0000 7f66ad9ae740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5600cfa23b70 0x5600cfa25ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:03.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.868+0000 7f66ad9ae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5600cf915920 con 0x5600cfa1d510 2024-04-04T06:11:03.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.868+0000 7f66ad9ae740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5600cf955f50 con 0x5600cfa1d130 2024-04-04T06:11:03.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.868+0000 7f66ad9ae740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5600cf9f48e0 con 0x5600cfa23b70 2024-04-04T06:11:03.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.868+0000 7f66ad9ae740 10 client.0 _ll_register_callbacks cb 0x5600cf915310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:11:03.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.868+0000 7f66a3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa1d510 0x5600cfa3b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:03.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.868+0000 7f66a3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa1d510 0x5600cfa3b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44650/0 (socket says 172.21.15.159:44650) 2024-04-04T06:11:03.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.868+0000 7f66a3400640 1 -- 172.21.15.159:0/565225421 learned_addr learned my addr 172.21.15.159:0/565225421 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:03.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.869+0000 7f66a3400640 1 -- 172.21.15.159:0/565225421 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5600cfa23b70 msgr2=0x5600cfa25ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:03.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.869+0000 7f66a3400640 1 --2- 172.21.15.159:0/565225421 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5600cfa23b70 0x5600cfa25ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.869+0000 7f66a3400640 1 -- 172.21.15.159:0/565225421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600cfa1d130 msgr2=0x5600cfa4bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:03.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.869+0000 7f66a3400640 1 --2- 172.21.15.159:0/565225421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600cfa1d130 0x5600cfa4bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.869+0000 7f66a3400640 1 -- 172.21.15.159:0/565225421 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600cf7386b0 con 0x5600cfa1d510 2024-04-04T06:11:03.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.869+0000 7f66a3400640 1 --2- 172.21.15.159:0/565225421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa1d510 0x5600cfa3b8b0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f6690061f00 tx=0x7f6690059850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36f29a020719d15 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:03.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.869+0000 7f668cc00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:11:03.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.869+0000 7f668cc00640 1 -- 172.21.15.159:0/565225421 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669001f040 con 0x5600cfa1d510 2024-04-04T06:11:03.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.869+0000 7f668cc00640 1 -- 172.21.15.159:0/565225421 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f669005ab80 con 0x5600cfa1d510 2024-04-04T06:11:03.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.869+0000 7f66ad9ae740 1 -- 172.21.15.159:0/565225421 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6694037040 con 0x5600cfa1d510 2024-04-04T06:11:03.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.869+0000 7f668cc00640 1 -- 172.21.15.159:0/565225421 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6690062430 con 0x5600cfa1d510 2024-04-04T06:11:03.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.869+0000 7f66ad9ae740 10 client.6870 fetch_fsmap learned FSMap version 51 2024-04-04T06:11:03.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.869+0000 7f66ad9ae740 1 -- 172.21.15.159:0/565225421 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x5600cfa252e0 con 0x5600cfa1d510 2024-04-04T06:11:03.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.870+0000 7f668cc00640 1 -- 172.21.15.159:0/565225421 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6694037040 con 0x5600cfa1d510 2024-04-04T06:11:03.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.870+0000 7f668cc00640 1 -- 172.21.15.159:0/565225421 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f6690058ca0 con 0x5600cfa1d510 2024-04-04T06:11:03.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.870+0000 7f66ad9ae740 10 client.6870 fetch_fsmap finished waiting for FSMap version 51 2024-04-04T06:11:03.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.870+0000 7f66ad9ae740 10 client.6870 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:11:03.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.870+0000 7f66ad9ae740 20 client.6870 populate_metadata read hostname 'smithi159' 2024-04-04T06:11:03.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.870+0000 7f66ad9ae740 1 --2- 172.21.15.159:0/565225421 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x5600cfa4ecc0 0x5600cfa4f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:03.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.870+0000 7f66ad9ae740 4 client.6870 mds_command: new command op to 6828 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:11:03.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.870+0000 7f66ad9ae740 1 -- 172.21.15.159:0/565225421 --> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600cf9674e0 con 0x5600cfa4ecc0 2024-04-04T06:11:03.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.870+0000 7f66a3e00640 1 --2- 172.21.15.159:0/565225421 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x5600cfa4ecc0 0x5600cfa4f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:03.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.871+0000 7f66a3e00640 1 --2- 172.21.15.159:0/565225421 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x5600cfa4ecc0 0x5600cfa4f0a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5417d6f081e28fe0 server_cookie=4db19f09d9bb6d2f in_seq=0 out_seq=0 2024-04-04T06:11:03.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.871+0000 7f668cc00640 10 client.6870 ms_handle_connect on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:03.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.871+0000 7f668cc00640 1 -- 172.21.15.159:0/565225421 <== mds.0 v2:172.21.15.159:6834/2467594024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600cf9674e0 con 0x5600cfa4ecc0 2024-04-04T06:11:03.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.871+0000 7f668cc00640 10 client.6870 handle_command_reply: tid=0 2024-04-04T06:11:03.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.872+0000 7f66ad9ae740 1 client.6870 shutdown 2024-04-04T06:11:03.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.872+0000 7f66ad9ae740 1 -- 172.21.15.159:0/565225421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa1d510 msgr2=0x5600cfa3b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:03.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.872+0000 7f66ad9ae740 1 --2- 172.21.15.159:0/565225421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa1d510 0x5600cfa3b8b0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f6690061f00 tx=0x7f6690059850 comp rx=0 tx=0).stop 2024-04-04T06:11:03.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.872+0000 7f66ad9ae740 1 -- 172.21.15.159:0/565225421 shutdown_connections 2024-04-04T06:11:03.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.872+0000 7f66ad9ae740 1 --2- 172.21.15.159:0/565225421 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x5600cfa4ecc0 0x5600cfa4f0a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.872+0000 7f66ad9ae740 1 --2- 172.21.15.159:0/565225421 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5600cfa23b70 0x5600cfa25ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.872+0000 7f66ad9ae740 1 --2- 172.21.15.159:0/565225421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa1d510 0x5600cfa3b8b0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.872+0000 7f66ad9ae740 1 --2- 172.21.15.159:0/565225421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600cfa1d130 0x5600cfa4bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.872+0000 7f668cc00640 0 client.6870 ms_handle_reset on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:03.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.872+0000 7f66ad9ae740 1 -- 172.21.15.159:0/565225421 >> 172.21.15.159:0/565225421 conn(0x5600cf9f0610 msgr2=0x5600cf9eeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:03.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.872+0000 7f66ad9ae740 1 -- 172.21.15.159:0/565225421 shutdown_connections 2024-04-04T06:11:03.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.872+0000 7f66ad9ae740 1 -- 172.21.15.159:0/565225421 wait complete. 2024-04-04T06:11:03.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.872+0000 7f66ad9ae740 20 client.6870 trim_cache size 0 max 16384 2024-04-04T06:11:03.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.892+0000 7f66ad9ae740 1 Processor -- start 2024-04-04T06:11:03.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.892+0000 7f66ad9ae740 1 -- start start 2024-04-04T06:11:03.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.893+0000 7f66ad9ae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa1d130 0x5600cfa269d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:03.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.893+0000 7f66ad9ae740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600cfa1d510 0x5600cfa3d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:03.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.893+0000 7f66ad9ae740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5600cfa23b70 0x5600cfa3d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:03.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.893+0000 7f66ad9ae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41ffcb0 con 0x5600cfa1d130 2024-04-04T06:11:03.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.893+0000 7f66ad9ae740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4144760 con 0x5600cfa1d510 2024-04-04T06:11:03.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.893+0000 7f66ad9ae740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f66a4142700 con 0x5600cfa23b70 2024-04-04T06:11:03.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.893+0000 7f66a3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa1d130 0x5600cfa269d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:03.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.893+0000 7f66a3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa1d130 0x5600cfa269d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44652/0 (socket says 172.21.15.159:44652) 2024-04-04T06:11:03.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.893+0000 7f66a3e00640 1 -- 172.21.15.159:0/1126696871 learned_addr learned my addr 172.21.15.159:0/1126696871 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:03.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.893+0000 7f66a3e00640 1 -- 172.21.15.159:0/1126696871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5600cfa23b70 msgr2=0x5600cfa3d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:03.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.893+0000 7f66a3e00640 1 --2- 172.21.15.159:0/1126696871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5600cfa23b70 0x5600cfa3d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.893+0000 7f66a3e00640 1 -- 172.21.15.159:0/1126696871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600cfa1d510 msgr2=0x5600cfa3d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:03.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.893+0000 7f66a3400640 1 --2- 172.21.15.159:0/1126696871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600cfa1d510 0x5600cfa3d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:03.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.893+0000 7f66a3e00640 1 --2- 172.21.15.159:0/1126696871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600cfa1d510 0x5600cfa3d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.893+0000 7f66a3e00640 1 -- 172.21.15.159:0/1126696871 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600cf9653c0 con 0x5600cfa1d130 2024-04-04T06:11:03.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.894+0000 7f66a3e00640 1 --2- 172.21.15.159:0/1126696871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa1d130 0x5600cfa269d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f6694032040 tx=0x7f669402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d38c8b9c122fc75 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:03.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.894+0000 7f668f400640 1 -- 172.21.15.159:0/1126696871 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694053070 con 0x5600cfa1d130 2024-04-04T06:11:03.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.894+0000 7f668f400640 1 -- 172.21.15.159:0/1126696871 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6694037040 con 0x5600cfa1d130 2024-04-04T06:11:03.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.894+0000 7f668f400640 1 -- 172.21.15.159:0/1126696871 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694058ae0 con 0x5600cfa1d130 2024-04-04T06:11:03.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.895+0000 7f66ad9ae740 1 -- 172.21.15.159:0/1126696871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa1d130 msgr2=0x5600cfa269d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:03.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.895+0000 7f66ad9ae740 1 --2- 172.21.15.159:0/1126696871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa1d130 0x5600cfa269d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f6694032040 tx=0x7f669402e000 comp rx=0 tx=0).stop 2024-04-04T06:11:03.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.895+0000 7f66ad9ae740 1 -- 172.21.15.159:0/1126696871 shutdown_connections 2024-04-04T06:11:03.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.895+0000 7f66ad9ae740 1 --2- 172.21.15.159:0/1126696871 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5600cfa23b70 0x5600cfa3d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.895+0000 7f66ad9ae740 1 --2- 172.21.15.159:0/1126696871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600cfa1d510 0x5600cfa3d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.895+0000 7f66ad9ae740 1 --2- 172.21.15.159:0/1126696871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa1d130 0x5600cfa269d0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.895+0000 7f66ad9ae740 1 -- 172.21.15.159:0/1126696871 >> 172.21.15.159:0/1126696871 conn(0x5600cf9f0610 msgr2=0x5600cf9f70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:03.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.895+0000 7f66ad9ae740 1 -- 172.21.15.159:0/1126696871 shutdown_connections 2024-04-04T06:11:03.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.895+0000 7f66ad9ae740 1 -- 172.21.15.159:0/1126696871 wait complete. 2024-04-04T06:11:03.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.896+0000 7f66ad9ae740 1 -- start start 2024-04-04T06:11:03.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.896+0000 7f66ad9ae740 1 Processor -- start 2024-04-04T06:11:03.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.897+0000 7f66ad9ae740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600cfa1d130 0x5600cfab8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:03.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.897+0000 7f66ad9ae740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5600cfa1d510 0x5600cfa4f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:03.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.897+0000 7f66ad9ae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa23b70 0x5600cfa51c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:03.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.897+0000 7f66ad9ae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5600cfa214c0 con 0x5600cfa23b70 2024-04-04T06:11:03.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.897+0000 7f66ad9ae740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5600cfa21830 con 0x5600cfa1d130 2024-04-04T06:11:03.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.897+0000 7f66ad9ae740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5600cfa26f20 con 0x5600cfa1d510 2024-04-04T06:11:03.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.897+0000 7f66ad9ae740 10 client.0 _ll_register_callbacks cb 0x7f669000ced0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:11:03.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.897+0000 7f66a8c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa23b70 0x5600cfa51c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:03.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.897+0000 7f66a8c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa23b70 0x5600cfa51c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44666/0 (socket says 172.21.15.159:44666) 2024-04-04T06:11:03.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.897+0000 7f66a8c00640 1 -- 172.21.15.159:0/918356125 learned_addr learned my addr 172.21.15.159:0/918356125 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:03.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.897+0000 7f66a8c00640 1 -- 172.21.15.159:0/918356125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5600cfa1d510 msgr2=0x5600cfa4f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:03.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.897+0000 7f66a8c00640 1 --2- 172.21.15.159:0/918356125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5600cfa1d510 0x5600cfa4f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.897+0000 7f66a3e00640 1 --2- 172.21.15.159:0/918356125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600cfa1d130 0x5600cfab8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:03.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.897+0000 7f66a8c00640 1 -- 172.21.15.159:0/918356125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600cfa1d130 msgr2=0x5600cfab8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:03.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.897+0000 7f66a8c00640 1 --2- 172.21.15.159:0/918356125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600cfa1d130 0x5600cfab8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.897+0000 7f66a8c00640 1 -- 172.21.15.159:0/918356125 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600cfa3d790 con 0x5600cfa23b70 2024-04-04T06:11:03.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.898+0000 7f66a8c00640 1 --2- 172.21.15.159:0/918356125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa23b70 0x5600cfa51c10 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f66a4141e00 tx=0x7f669c009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ad6b0781e82bbe server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:03.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.898+0000 7f668cc00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:11:03.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.898+0000 7f668cc00640 1 -- 172.21.15.159:0/918356125 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c007be0 con 0x5600cfa23b70 2024-04-04T06:11:03.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.898+0000 7f668cc00640 1 -- 172.21.15.159:0/918356125 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f669c007d80 con 0x5600cfa23b70 2024-04-04T06:11:03.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.898+0000 7f66ad9ae740 1 -- 172.21.15.159:0/918356125 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6694037040 con 0x5600cfa23b70 2024-04-04T06:11:03.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.898+0000 7f668cc00640 1 -- 172.21.15.159:0/918356125 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f669c021050 con 0x5600cfa23b70 2024-04-04T06:11:03.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.898+0000 7f66ad9ae740 10 client.6876 fetch_fsmap learned FSMap version 51 2024-04-04T06:11:03.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.898+0000 7f66ad9ae740 1 -- 172.21.15.159:0/918356125 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x5600cfa524f0 con 0x5600cfa23b70 2024-04-04T06:11:03.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.898+0000 7f668cc00640 1 -- 172.21.15.159:0/918356125 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6694037040 con 0x5600cfa23b70 2024-04-04T06:11:03.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.899+0000 7f668cc00640 1 -- 172.21.15.159:0/918356125 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f669c024020 con 0x5600cfa23b70 2024-04-04T06:11:03.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.899+0000 7f66ad9ae740 10 client.6876 fetch_fsmap finished waiting for FSMap version 51 2024-04-04T06:11:03.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.899+0000 7f66ad9ae740 10 client.6876 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:11:03.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.899+0000 7f66ad9ae740 20 client.6876 populate_metadata read hostname 'smithi159' 2024-04-04T06:11:03.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.899+0000 7f66ad9ae740 1 --2- 172.21.15.159:0/918356125 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x5600cfa1fbd0 0x5600cfa1ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:03.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.899+0000 7f66ad9ae740 4 client.6876 mds_command: new command op to 6828 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-04T06:11:03.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.899+0000 7f66ad9ae740 1 -- 172.21.15.159:0/918356125 --> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] -- command(tid 0: {"prefix": "status"}) -- 0x5600cf961830 con 0x5600cfa1fbd0 2024-04-04T06:11:03.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.899+0000 7f66a3e00640 1 --2- 172.21.15.159:0/918356125 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x5600cfa1fbd0 0x5600cfa1ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:03.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.899+0000 7f66a3e00640 1 --2- 172.21.15.159:0/918356125 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x5600cfa1fbd0 0x5600cfa1ffb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=995e94993b63e9bc server_cookie=9eb4954c489bbbf3 in_seq=0 out_seq=0 2024-04-04T06:11:03.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.899+0000 7f668cc00640 10 client.6876 ms_handle_connect on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:03.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.900+0000 7f668cc00640 1 -- 172.21.15.159:0/918356125 <== mds.0 v2:172.21.15.159:6834/2467594024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5600cf961830 con 0x5600cfa1fbd0 2024-04-04T06:11:03.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.900+0000 7f668cc00640 10 client.6876 handle_command_reply: tid=0 2024-04-04T06:11:03.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.900+0000 7f66ad9ae740 1 client.6876 shutdown 2024-04-04T06:11:03.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.900+0000 7f66ad9ae740 1 -- 172.21.15.159:0/918356125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa23b70 msgr2=0x5600cfa51c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:03.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.900+0000 7f66ad9ae740 1 --2- 172.21.15.159:0/918356125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa23b70 0x5600cfa51c10 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f66a4141e00 tx=0x7f669c009e50 comp rx=0 tx=0).stop 2024-04-04T06:11:03.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.900+0000 7f66ad9ae740 1 -- 172.21.15.159:0/918356125 shutdown_connections 2024-04-04T06:11:03.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.900+0000 7f66ad9ae740 1 --2- 172.21.15.159:0/918356125 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x5600cfa1fbd0 0x5600cfa1ffb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.900+0000 7f66ad9ae740 1 --2- 172.21.15.159:0/918356125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600cfa23b70 0x5600cfa51c10 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.900+0000 7f66ad9ae740 1 --2- 172.21.15.159:0/918356125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5600cfa1d510 0x5600cfa4f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.900+0000 7f668cc00640 0 client.6876 ms_handle_reset on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:03.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.900+0000 7f66ad9ae740 1 --2- 172.21.15.159:0/918356125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5600cfa1d130 0x5600cfab8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.900+0000 7f66ad9ae740 1 -- 172.21.15.159:0/918356125 >> 172.21.15.159:0/918356125 conn(0x5600cf9f0610 msgr2=0x5600cf9f3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:03.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.900+0000 7f66ad9ae740 1 -- 172.21.15.159:0/918356125 shutdown_connections 2024-04-04T06:11:03.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.900+0000 7f66ad9ae740 1 -- 172.21.15.159:0/918356125 wait complete. 2024-04-04T06:11:03.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.900+0000 7f66ad9ae740 20 client.6876 trim_cache size 0 max 16384 2024-04-04T06:11:03.897 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:11:03.897 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_fsid": "30311045-f714-48fa-832b-4cb2ba8bf0bf", 2024-04-04T06:11:03.897 INFO:teuthology.orchestra.run.smithi159.stdout: "whoami": 0, 2024-04-04T06:11:03.897 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 6828, 2024-04-04T06:11:03.897 INFO:teuthology.orchestra.run.smithi159.stdout: "want_state": "up:active", 2024-04-04T06:11:03.898 INFO:teuthology.orchestra.run.smithi159.stdout: "state": "up:active", 2024-04-04T06:11:03.898 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs", 2024-04-04T06:11:03.898 INFO:teuthology.orchestra.run.smithi159.stdout: "rank_uptime": 4.9462940079999997, 2024-04-04T06:11:03.898 INFO:teuthology.orchestra.run.smithi159.stdout: "mdsmap_epoch": 51, 2024-04-04T06:11:03.898 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch": 72, 2024-04-04T06:11:03.898 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch_barrier": 72, 2024-04-04T06:11:03.898 INFO:teuthology.orchestra.run.smithi159.stdout: "uptime": 5.8867591849999998 2024-04-04T06:11:03.898 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:11:03.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.901+0000 7f66aac00640 1 -- 172.21.15.159:0/3972043597 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f668406c8f0 msgr2=0x7f668406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:03.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.901+0000 7f66aac00640 1 --2- 172.21.15.159:0/3972043597 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f668406c8f0 0x7f668406edb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6694036000 tx=0x7f669403d040 comp rx=0 tx=0).stop 2024-04-04T06:11:03.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.901+0000 7f66aac00640 1 -- 172.21.15.159:0/3972043597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66a4155550 msgr2=0x7f66a415a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:03.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.901+0000 7f66aac00640 1 --2- 172.21.15.159:0/3972043597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66a4155550 0x7f66a415a540 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f669000c7f0 tx=0x7f669000cbf0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.901+0000 7f66aac00640 1 -- 172.21.15.159:0/3972043597 shutdown_connections 2024-04-04T06:11:03.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.901+0000 7f66aac00640 1 --2- 172.21.15.159:0/3972043597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66a415f800 0x7f66a4168a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.901+0000 7f66aac00640 1 --2- 172.21.15.159:0/3972043597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66a4155550 0x7f66a415a540 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.901+0000 7f66aac00640 1 --2- 172.21.15.159:0/3972043597 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f668406c8f0 0x7f668406edb0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.901+0000 7f66aac00640 1 --2- 172.21.15.159:0/3972043597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66a40d07c0 0x7f66a4167190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:03.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.901+0000 7f66aac00640 1 -- 172.21.15.159:0/3972043597 >> 172.21.15.159:0/3972043597 conn(0x7f66a406b750 msgr2=0x7f66a406d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:03.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.901+0000 7f66aac00640 1 -- 172.21.15.159:0/3972043597 shutdown_connections 2024-04-04T06:11:03.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:03.901+0000 7f66aac00640 1 -- 172.21.15.159:0/3972043597 wait complete. 2024-04-04T06:11:03.911 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-04T06:11:03.949 INFO:tasks.ceph.mds.a.smithi159.stderr:2024-04-04T06:11:03.956+0000 7fc061a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:11:04.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.096+0000 7f7717200640 1 Processor -- start 2024-04-04T06:11:04.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.096+0000 7f7717200640 1 -- start start 2024-04-04T06:11:04.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.097+0000 7f7717200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7718144ad0 0x7f77180b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:04.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.097+0000 7f7717200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f771815fba0 con 0x7f7718072f50 2024-04-04T06:11:04.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.097+0000 7f7717200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7718144eb0 con 0x7f771815f710 2024-04-04T06:11:04.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.097+0000 7f7717200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f77180cdad0 con 0x7f7718144ad0 2024-04-04T06:11:04.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.098+0000 7f7716800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f7718072f50 0x7f77180cdf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40262/0 (socket says 172.21.15.159:40262) 2024-04-04T06:11:04.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.098+0000 7f7716800640 1 -- 172.21.15.159:0/3077148058 learned_addr learned my addr 172.21.15.159:0/3077148058 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:11:04.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.098+0000 7f7715e00640 1 --2- 172.21.15.159:0/3077148058 >> v2:172.21.15.159:3300/0 conn(0x7f7718144ad0 0x7f77180b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:04.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.098+0000 7f7715400640 1 -- 172.21.15.159:0/3077148058 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1489311234 0 0) 0x7f771815fba0 con 0x7f7718072f50 2024-04-04T06:11:04.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.098+0000 7f7715400640 1 -- 172.21.15.159:0/3077148058 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76fc003520 con 0x7f7718072f50 2024-04-04T06:11:04.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.098+0000 7f7715e00640 1 -- 172.21.15.159:0/3077148058 >> v1:172.21.15.178:6789/0 conn(0x7f771815f710 legacy=0x7f7718072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:04.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.098+0000 7f7715e00640 1 -- 172.21.15.159:0/3077148058 >> v1:172.21.15.159:6789/0 conn(0x7f7718072f50 legacy=0x7f77180cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:04.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.098+0000 7f7715e00640 1 -- 172.21.15.159:0/3077148058 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77180b36f0 con 0x7f7718144ad0 2024-04-04T06:11:04.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.100+0000 7f7715e00640 1 --2- 172.21.15.159:0/3077148058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7718144ad0 0x7f77180b2ff0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f77080097a0 tx=0x7f770802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495ecab0b7203a60 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:04.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.100+0000 7f7715400640 1 -- 172.21.15.159:0/3077148058 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708035020 con 0x7f7718144ad0 2024-04-04T06:11:04.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.100+0000 7f7715400640 1 -- 172.21.15.159:0/3077148058 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7708031830 con 0x7f7718144ad0 2024-04-04T06:11:04.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.100+0000 7f7715400640 1 -- 172.21.15.159:0/3077148058 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708031be0 con 0x7f7718144ad0 2024-04-04T06:11:04.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.101+0000 7f7717200640 1 -- 172.21.15.159:0/3077148058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7718144ad0 msgr2=0x7f77180b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:04.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.101+0000 7f7717200640 1 --2- 172.21.15.159:0/3077148058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7718144ad0 0x7f77180b2ff0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f77080097a0 tx=0x7f770802b8d0 comp rx=0 tx=0).stop 2024-04-04T06:11:04.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.101+0000 7f7717200640 1 -- 172.21.15.159:0/3077148058 shutdown_connections 2024-04-04T06:11:04.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.101+0000 7f7717200640 1 --2- 172.21.15.159:0/3077148058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7718144ad0 0x7f77180b2ff0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:04.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.101+0000 7f7717200640 1 -- 172.21.15.159:0/3077148058 >> 172.21.15.159:0/3077148058 conn(0x7f771806b570 msgr2=0x7f771806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:04.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.101+0000 7f7717200640 1 -- 172.21.15.159:0/3077148058 shutdown_connections 2024-04-04T06:11:04.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.101+0000 7f7717200640 1 -- 172.21.15.159:0/3077148058 wait complete. 2024-04-04T06:11:04.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.103+0000 7f7717200640 1 Processor -- start 2024-04-04T06:11:04.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.103+0000 7f7717200640 1 -- start start 2024-04-04T06:11:04.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.104+0000 7f7717200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7718072f50 0x7f771813f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:04.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.104+0000 7f7717200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7718144ad0 0x7f77180b2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:04.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.104+0000 7f7717200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f771815f710 0x7f77180b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:04.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.104+0000 7f7717200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f77180b3f90 con 0x7f771815f710 2024-04-04T06:11:04.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.104+0000 7f7717200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f77180b3cb0 con 0x7f7718072f50 2024-04-04T06:11:04.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.104+0000 7f7717200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f77180ce670 con 0x7f7718144ad0 2024-04-04T06:11:04.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.104+0000 7f7716800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f771815f710 0x7f77180b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:04.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.104+0000 7f7716800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f771815f710 0x7f77180b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44690/0 (socket says 172.21.15.159:44690) 2024-04-04T06:11:04.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.104+0000 7f7716800640 1 -- 172.21.15.159:0/1111695462 learned_addr learned my addr 172.21.15.159:0/1111695462 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:04.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.104+0000 7f7715e00640 1 --2- 172.21.15.159:0/1111695462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7718072f50 0x7f771813f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:04.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.105+0000 7f770fe00640 1 --2- 172.21.15.159:0/1111695462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7718144ad0 0x7f77180b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:04.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.105+0000 7f7716800640 1 -- 172.21.15.159:0/1111695462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7718144ad0 msgr2=0x7f77180b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:04.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.105+0000 7f7716800640 1 --2- 172.21.15.159:0/1111695462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7718144ad0 0x7f77180b2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:04.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.105+0000 7f7716800640 1 -- 172.21.15.159:0/1111695462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7718072f50 msgr2=0x7f771813f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:04.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.105+0000 7f7716800640 1 --2- 172.21.15.159:0/1111695462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7718072f50 0x7f771813f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:04.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.105+0000 7f7716800640 1 -- 172.21.15.159:0/1111695462 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7718141e40 con 0x7f771815f710 2024-04-04T06:11:04.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.105+0000 7f7716800640 1 --2- 172.21.15.159:0/1111695462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f771815f710 0x7f77180b26d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f771000cb40 tx=0x7f771000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:04.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.105+0000 7f770e000640 1 -- 172.21.15.159:0/1111695462 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7710017070 con 0x7f771815f710 2024-04-04T06:11:04.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.105+0000 7f770e000640 1 -- 172.21.15.159:0/1111695462 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f771000aca0 con 0x7f771815f710 2024-04-04T06:11:04.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.105+0000 7f7717200640 1 -- 172.21.15.159:0/1111695462 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7718142120 con 0x7f771815f710 2024-04-04T06:11:04.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.105+0000 7f770e000640 1 -- 172.21.15.159:0/1111695462 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7710012620 con 0x7f771815f710 2024-04-04T06:11:04.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.105+0000 7f7717200640 1 -- 172.21.15.159:0/1111695462 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f771813ee30 con 0x7f771815f710 2024-04-04T06:11:04.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.107+0000 7f7717200640 1 -- 172.21.15.159:0/1111695462 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77180ccb30 con 0x7f771815f710 2024-04-04T06:11:04.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.108+0000 7f770e000640 1 -- 172.21.15.159:0/1111695462 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f771001a070 con 0x7f771815f710 2024-04-04T06:11:04.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.109+0000 7f770e000640 1 --2- 172.21.15.159:0/1111695462 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f76f406c8f0 0x7f76f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:04.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.109+0000 7f770e000640 1 -- 172.21.15.159:0/1111695462 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f771008ba20 con 0x7f771815f710 2024-04-04T06:11:04.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.110+0000 7f7715e00640 1 --2- 172.21.15.159:0/1111695462 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f76f406c8f0 0x7f76f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:04.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.111+0000 7f7715e00640 1 --2- 172.21.15.159:0/1111695462 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f76f406c8f0 0x7f76f406edb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7708036000 tx=0x7f7708008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:04.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.113+0000 7f770e000640 1 -- 172.21.15.159:0/1111695462 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f77180ccb30 con 0x7f771815f710 2024-04-04T06:11:04.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.353+0000 7f7717200640 1 -- 172.21.15.159:0/1111695462 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7718075940 con 0x7f771815f710 2024-04-04T06:11:04.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.354+0000 7f770e000640 1 -- 172.21.15.159:0/1111695462 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) ==== 53+0+73 (secure 0 0 0) 0x7f7710059de0 con 0x7f771815f710 2024-04-04T06:11:04.347 INFO:teuthology.orchestra.run.smithi159.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-04T06:11:04.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.356+0000 7f7717200640 1 -- 172.21.15.159:0/1111695462 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f76f406c8f0 msgr2=0x7f76f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:04.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.356+0000 7f7717200640 1 --2- 172.21.15.159:0/1111695462 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f76f406c8f0 0x7f76f406edb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7708036000 tx=0x7f7708008030 comp rx=0 tx=0).stop 2024-04-04T06:11:04.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.356+0000 7f7717200640 1 -- 172.21.15.159:0/1111695462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f771815f710 msgr2=0x7f77180b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:04.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.356+0000 7f7717200640 1 --2- 172.21.15.159:0/1111695462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f771815f710 0x7f77180b26d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f771000cb40 tx=0x7f771000a060 comp rx=0 tx=0).stop 2024-04-04T06:11:04.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.356+0000 7f7717200640 1 -- 172.21.15.159:0/1111695462 shutdown_connections 2024-04-04T06:11:04.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.356+0000 7f7717200640 1 --2- 172.21.15.159:0/1111695462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f771815f710 0x7f77180b26d0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:04.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.356+0000 7f7717200640 1 --2- 172.21.15.159:0/1111695462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7718144ad0 0x7f77180b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:04.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.356+0000 7f7717200640 1 --2- 172.21.15.159:0/1111695462 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f76f406c8f0 0x7f76f406edb0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:04.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.356+0000 7f7717200640 1 --2- 172.21.15.159:0/1111695462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7718072f50 0x7f771813f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:04.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.356+0000 7f7717200640 1 -- 172.21.15.159:0/1111695462 >> 172.21.15.159:0/1111695462 conn(0x7f771806b570 msgr2=0x7f77180cc420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:04.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.357+0000 7f7717200640 1 -- 172.21.15.159:0/1111695462 shutdown_connections 2024-04-04T06:11:04.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04.357+0000 7f7717200640 1 -- 172.21.15.159:0/1111695462 wait complete. 2024-04-04T06:11:04.368 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-04T06:11:04.368 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-04T06:11:04.368 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi159.front.sepia.ceph.com 2024-04-04T06:11:04.368 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-04T06:11:04.368 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-04T06:11:04.368 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-04T06:11:04.368 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-04T06:11:04.368 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-04T06:11:04.368 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:11:04.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:11:04.369 DEBUG:teuthology.orchestra.run.smithi159:> ip addr 2024-04-04T06:11:04.462 INFO:teuthology.orchestra.run.smithi159.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:11:04.462 INFO:teuthology.orchestra.run.smithi159.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:11:04.463 INFO:teuthology.orchestra.run.smithi159.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:11:04.463 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:11:04.463 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 ::1/128 scope host 2024-04-04T06:11:04.463 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:11:04.463 INFO:teuthology.orchestra.run.smithi159.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:11:04.463 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c6 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:11:04.463 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f0 2024-04-04T06:11:04.463 INFO:teuthology.orchestra.run.smithi159.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:11:04.463 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c7 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:11:04.463 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f1 2024-04-04T06:11:04.463 INFO:teuthology.orchestra.run.smithi159.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:11:04.464 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4e brd ff:ff:ff:ff:ff:ff 2024-04-04T06:11:04.464 INFO:teuthology.orchestra.run.smithi159.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-04T06:11:04.464 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:11:04.464 INFO:teuthology.orchestra.run.smithi159.stdout: inet 172.21.15.159/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-04T06:11:04.464 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft 42268sec preferred_lft 42268sec 2024-04-04T06:11:04.464 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::ec4:7aff:fe88:724f/64 scope link 2024-04-04T06:11:04.464 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:11:04.464 INFO:teuthology.orchestra.run.smithi159.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-04T06:11:04.464 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:11:04.464 INFO:teuthology.orchestra.run.smithi159.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-04T06:11:04.464 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:11:04.464 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::989e:e4ff:fea5:7568/64 scope link 2024-04-04T06:11:04.464 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:11:04.464 INFO:teuthology.orchestra.run.smithi159.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:11:04.465 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether a2:e3:e0:cb:0a:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-04T06:11:04.465 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::a0e3:e0ff:fecb:a73/64 scope link 2024-04-04T06:11:04.465 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:11:04.465 INFO:teuthology.orchestra.run.smithi159.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:11:04.465 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-04T06:11:04.465 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::c0a:8aff:fe54:b58f/64 scope link 2024-04-04T06:11:04.465 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:11:04.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:11:04.465 DEBUG:teuthology.orchestra.run.smithi159:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-04T06:11:04.541 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:11:04.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:11:04.545 DEBUG:teuthology.orchestra.run.smithi159:> route 2024-04-04T06:11:04.591 INFO:teuthology.orchestra.run.smithi159.stdout:Kernel IP routing table 2024-04-04T06:11:04.591 INFO:teuthology.orchestra.run.smithi159.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-04T06:11:04.591 INFO:teuthology.orchestra.run.smithi159.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-04T06:11:04.592 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-04T06:11:04.592 INFO:teuthology.orchestra.run.smithi159.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-04T06:11:04.592 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:11:04.592 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:11:04.592 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-04T06:11:04.592 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-04T06:11:04.592 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-04T06:11:04.592 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:11:04.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:11:04.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:11:04.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:11:04.823 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-04-04T06:11:04.868 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-04T06:11:04.868 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-04T06:11:04.869 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:11:04.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:11:04.870 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set brx.0 up 2024-04-04T06:11:04.958 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:11:04.958 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-04T06:11:05.010 INFO:teuthology.orchestra.run.smithi159.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:11:05.010 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:11:05.010 DEBUG:teuthology.orchestra.run.smithi159:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:11:05.057 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-04T06:11:05.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:11:05.057 DEBUG:teuthology.orchestra.run.smithi159:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:11:05.190 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:11:05.190 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:11:05.191 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:11:05.211 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:11:05.211 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:11:05.211 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:11:05.211 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:11:05.211 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:11:05.236 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:11:05.237 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:11:05.237 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:11:05.237 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:11:05.237 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:11:05.237 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-04T06:11:05.237 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-04T06:11:05.237 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30311045-f714-48fa-832b-4cb2ba8bf0bf" will pass to kernel 2024-04-04T06:11:05.238 INFO:teuthology.orchestra.run.smithi159.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-04T06:11:05.239 INFO:teuthology.orchestra.run.smithi159.stdout: You just mounted a file system that supports labels which does not 2024-04-04T06:11:05.239 INFO:teuthology.orchestra.run.smithi159.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-04T06:11:05.239 INFO:teuthology.orchestra.run.smithi159.stdout: applications will generate AVC messages and not be allowed access to 2024-04-04T06:11:05.239 INFO:teuthology.orchestra.run.smithi159.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-04T06:11:05.243 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-04T06:11:05.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:11:05.243 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:11:05.317 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:11:05.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:11:05.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:11:05.427 INFO:teuthology.orchestra.run.smithi159.stdout:client5156 2024-04-04T06:11:05.427 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:11:05.427 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:11:05.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:11:05.473 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-04-04T06:11:05.572 INFO:teuthology.orchestra.run.smithi159.stdout:file_undamaged 2024-04-04T06:11:05.574 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:11:05.574 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:11:05.574 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:11:05.574 DEBUG:teuthology.orchestra.run.smithi159:> import json 2024-04-04T06:11:05.575 DEBUG:teuthology.orchestra.run.smithi159:> import sys 2024-04-04T06:11:05.575 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:11:05.575 DEBUG:teuthology.orchestra.run.smithi159:> try: 2024-04-04T06:11:05.575 DEBUG:teuthology.orchestra.run.smithi159:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-04-04T06:11:05.575 DEBUG:teuthology.orchestra.run.smithi159:> except OSError as e: 2024-04-04T06:11:05.575 DEBUG:teuthology.orchestra.run.smithi159:> sys.exit(e.errno) 2024-04-04T06:11:05.575 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:11:05.575 DEBUG:teuthology.orchestra.run.smithi159:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-04T06:11:05.575 DEBUG:teuthology.orchestra.run.smithi159:> print(json.dumps( 2024-04-04T06:11:05.575 DEBUG:teuthology.orchestra.run.smithi159:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-04T06:11:05.575 DEBUG:teuthology.orchestra.run.smithi159:> indent=2)) 2024-04-04T06:11:05.575 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:11:05.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:05 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-04T06:11:05.718 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-04T06:11:05.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:11:05.718 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-04T06:11:05.815 INFO:teuthology.orchestra.run.smithi159.stdout:2 2024-04-04T06:11:05.815 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:11:05.815 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:11:05.866 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:11:05.866 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:11:05.917 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:11:06.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.122+0000 7f6067000640 1 Processor -- start 2024-04-04T06:11:06.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.123+0000 7f6067000640 1 -- start start 2024-04-04T06:11:06.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.125+0000 7f6067000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f606015c390 0x7f6060073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:06.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.125+0000 7f6067000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f60600d0650 0x7f60600d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:06.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.125+0000 7f6067000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f606015fe80 con 0x7f60600af830 2024-04-04T06:11:06.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.125+0000 7f6067000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f60600d1000 con 0x7f60600d0650 2024-04-04T06:11:06.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.125+0000 7f6067000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f606015c770 con 0x7f606015c390 2024-04-04T06:11:06.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.125+0000 7f6057400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f60600af830 0x7f6060160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40274/0 (socket says 172.21.15.159:40274) 2024-04-04T06:11:06.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.125+0000 7f6057400640 1 -- 172.21.15.159:0/1716344096 learned_addr learned my addr 172.21.15.159:0/1716344096 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:11:06.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.125+0000 7f6065000640 1 --2- 172.21.15.159:0/1716344096 >> v2:172.21.15.178:3300/0 conn(0x7f60600d0650 0x7f60600d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:06.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.125+0000 7f605fe00640 1 --2- 172.21.15.159:0/1716344096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f606015c390 0x7f6060073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:06.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.126+0000 7f605f400640 1 -- 172.21.15.159:0/1716344096 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3198003629 0 0) 0x7f606015fe80 con 0x7f60600af830 2024-04-04T06:11:06.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.126+0000 7f605f400640 1 -- 172.21.15.159:0/1716344096 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6040003520 con 0x7f60600af830 2024-04-04T06:11:06.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.126+0000 7f605f400640 1 -- 172.21.15.159:0/1716344096 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2956407500 0 0) 0x7f6040003520 con 0x7f60600af830 2024-04-04T06:11:06.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.126+0000 7f605f400640 1 -- 172.21.15.159:0/1716344096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f606015c390 msgr2=0x7f6060073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:06.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.126+0000 7f605f400640 1 --2- 172.21.15.159:0/1716344096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f606015c390 0x7f6060073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:06.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.126+0000 7f605f400640 1 -- 172.21.15.159:0/1716344096 >> v2:172.21.15.178:3300/0 conn(0x7f60600d0650 msgr2=0x7f60600d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:06.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.126+0000 7f605f400640 1 --2- 172.21.15.159:0/1716344096 >> v2:172.21.15.178:3300/0 conn(0x7f60600d0650 0x7f60600d0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:06.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.126+0000 7f605f400640 1 -- 172.21.15.159:0/1716344096 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60600ad120 con 0x7f60600af830 2024-04-04T06:11:06.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.126+0000 7f605f400640 1 -- 172.21.15.159:0/1716344096 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f6048001670 con 0x7f60600af830 2024-04-04T06:11:06.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.126+0000 7f605f400640 1 -- 172.21.15.159:0/1716344096 >> v1:172.21.15.159:6789/0 conn(0x7f60600af830 legacy=0x7f6060160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:06.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.126+0000 7f605f400640 1 --2- 172.21.15.159:0/1716344096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f606015c390 0x7f60400039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:06.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.126+0000 7f605f400640 1 --2- 172.21.15.159:0/1716344096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6040003f80 0x7f6040004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:06.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.126+0000 7f605f400640 1 --2- 172.21.15.159:0/1716344096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6040004d40 0x7f60400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:06.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.126+0000 7f605f400640 1 -- 172.21.15.159:0/1716344096 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f60600d1000 con 0x7f6040004d40 2024-04-04T06:11:06.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.126+0000 7f605f400640 1 -- 172.21.15.159:0/1716344096 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f604000b9c0 con 0x7f6040003f80 2024-04-04T06:11:06.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.126+0000 7f605f400640 1 -- 172.21.15.159:0/1716344096 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f604000bb40 con 0x7f606015c390 2024-04-04T06:11:06.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.127+0000 7f6065000640 1 --2- 172.21.15.159:0/1716344096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6040004d40 0x7f60400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:06.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.127+0000 7f6057400640 1 --2- 172.21.15.159:0/1716344096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6040003f80 0x7f6040004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:06.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.127+0000 7f605fe00640 1 --2- 172.21.15.159:0/1716344096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f606015c390 0x7f60400039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:06.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.127+0000 7f6065000640 1 -- 172.21.15.159:0/1716344096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f606015c390 msgr2=0x7f60400039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:06.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.127+0000 7f6065000640 1 --2- 172.21.15.159:0/1716344096 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f606015c390 0x7f60400039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:06.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.127+0000 7f6065000640 1 -- 172.21.15.159:0/1716344096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6040003f80 msgr2=0x7f6040004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:06.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.127+0000 7f6065000640 1 --2- 172.21.15.159:0/1716344096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6040003f80 0x7f6040004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:06.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.127+0000 7f6065000640 1 -- 172.21.15.159:0/1716344096 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6040003520 con 0x7f6040004d40 2024-04-04T06:11:06.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.128+0000 7f6065000640 1 --2- 172.21.15.159:0/1716344096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6040004d40 0x7f60400051c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f605800e170 tx=0x7f6040005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66f345fe3f5b032 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:06.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.128+0000 7f605f400640 1 -- 172.21.15.159:0/1716344096 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6058039070 con 0x7f6040004d40 2024-04-04T06:11:06.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.128+0000 7f605f400640 1 -- 172.21.15.159:0/1716344096 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f605802cc20 con 0x7f6040004d40 2024-04-04T06:11:06.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.128+0000 7f605f400640 1 -- 172.21.15.159:0/1716344096 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6058009c10 con 0x7f6040004d40 2024-04-04T06:11:06.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.129+0000 7f6067000640 1 -- 172.21.15.159:0/1716344096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6040004d40 msgr2=0x7f60400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:06.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.129+0000 7f6067000640 1 --2- 172.21.15.159:0/1716344096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6040004d40 0x7f60400051c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f605800e170 tx=0x7f6040005830 comp rx=0 tx=0).stop 2024-04-04T06:11:06.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.129+0000 7f6065000640 1 -- 172.21.15.159:0/1716344096 reap_dead start 2024-04-04T06:11:06.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.129+0000 7f6067000640 1 -- 172.21.15.159:0/1716344096 shutdown_connections 2024-04-04T06:11:06.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.129+0000 7f6067000640 1 -- 172.21.15.159:0/1716344096 >> 172.21.15.159:0/1716344096 conn(0x7f606006b990 msgr2=0x7f606006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:06.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.129+0000 7f6067000640 1 -- 172.21.15.159:0/1716344096 shutdown_connections 2024-04-04T06:11:06.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.129+0000 7f6067000640 1 -- 172.21.15.159:0/1716344096 wait complete. 2024-04-04T06:11:06.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.131+0000 7f6067000640 1 Processor -- start 2024-04-04T06:11:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.131+0000 7f6067000640 1 -- start start 2024-04-04T06:11:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.132+0000 7f6067000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6040004d40 0x7f606015ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.132+0000 7f6067000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f606014ffc0 0x7f6060156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.132+0000 7f6067000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60601504b0 0x7f6060156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.132+0000 7f6067000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6060160120 con 0x7f60601504b0 2024-04-04T06:11:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.132+0000 7f6067000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f60600ad6e0 con 0x7f606014ffc0 2024-04-04T06:11:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.132+0000 7f6067000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6060160d50 con 0x7f6040004d40 2024-04-04T06:11:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.132+0000 7f6065000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60601504b0 0x7f6060156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.132+0000 7f6065000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60601504b0 0x7f6060156790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44708/0 (socket says 172.21.15.159:44708) 2024-04-04T06:11:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.132+0000 7f6065000640 1 -- 172.21.15.159:0/2982440486 learned_addr learned my addr 172.21.15.159:0/2982440486 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.133+0000 7f605fe00640 1 --2- 172.21.15.159:0/2982440486 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6040004d40 0x7f606015ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.133+0000 7f6057400640 1 --2- 172.21.15.159:0/2982440486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f606014ffc0 0x7f6060156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:06.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.133+0000 7f6065000640 1 -- 172.21.15.159:0/2982440486 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6040004d40 msgr2=0x7f606015ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:06.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.133+0000 7f6065000640 1 --2- 172.21.15.159:0/2982440486 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6040004d40 0x7f606015ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:06.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.133+0000 7f6065000640 1 -- 172.21.15.159:0/2982440486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f606014ffc0 msgr2=0x7f6060156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:06.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.133+0000 7f6065000640 1 --2- 172.21.15.159:0/2982440486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f606014ffc0 0x7f6060156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:06.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.133+0000 7f6065000640 1 -- 172.21.15.159:0/2982440486 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606015c770 con 0x7f60601504b0 2024-04-04T06:11:06.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.133+0000 7f6065000640 1 --2- 172.21.15.159:0/2982440486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60601504b0 0x7f6060156790 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f6058037040 tx=0x7f6058009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:06.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.133+0000 7f605cc00640 1 -- 172.21.15.159:0/2982440486 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6058039070 con 0x7f60601504b0 2024-04-04T06:11:06.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.133+0000 7f605cc00640 1 -- 172.21.15.159:0/2982440486 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f605803a040 con 0x7f60601504b0 2024-04-04T06:11:06.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.133+0000 7f605cc00640 1 -- 172.21.15.159:0/2982440486 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6058003990 con 0x7f60601504b0 2024-04-04T06:11:06.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.134+0000 7f6067000640 1 -- 172.21.15.159:0/2982440486 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6060154fa0 con 0x7f60601504b0 2024-04-04T06:11:06.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.134+0000 7f6067000640 1 -- 172.21.15.159:0/2982440486 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6060155560 con 0x7f60601504b0 2024-04-04T06:11:06.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.135+0000 7f6067000640 1 -- 172.21.15.159:0/2982440486 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60600685c0 con 0x7f60601504b0 2024-04-04T06:11:06.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.136+0000 7f605cc00640 1 -- 172.21.15.159:0/2982440486 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6058046070 con 0x7f60601504b0 2024-04-04T06:11:06.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.137+0000 7f605cc00640 1 --2- 172.21.15.159:0/2982440486 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f603c06c8f0 0x7f603c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:06.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.137+0000 7f605cc00640 1 -- 172.21.15.159:0/2982440486 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f60580ad770 con 0x7f60601504b0 2024-04-04T06:11:06.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.137+0000 7f605fe00640 1 --2- 172.21.15.159:0/2982440486 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f603c06c8f0 0x7f603c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:06.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.138+0000 7f605fe00640 1 --2- 172.21.15.159:0/2982440486 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f603c06c8f0 0x7f603c06edb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6050009a10 tx=0x7f6050008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:06.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.141+0000 7f605cc00640 1 -- 172.21.15.159:0/2982440486 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f605807b450 con 0x7f60601504b0 2024-04-04T06:11:06.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.346+0000 7f6067000640 1 -- 172.21.15.159:0/2982440486 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60600af2b0 con 0x7f60601504b0 2024-04-04T06:11:06.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.349+0000 7f605cc00640 1 -- 172.21.15.159:0/2982440486 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16811 (secure 0 0 0) 0x7f605807f750 con 0x7f60601504b0 2024-04-04T06:11:06.342 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:11:06.342 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":72,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:10:57.934666+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:10:42.595868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-04T06:10:43.788847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6835/1652468640":"2024-04-05T06:10:57.934622+0000","172.21.15.159:6834/1652468640":"2024-04-05T06:10:57.934622+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:11:06.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.351+0000 7f6067000640 1 -- 172.21.15.159:0/2982440486 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f603c06c8f0 msgr2=0x7f603c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:06.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.351+0000 7f6067000640 1 --2- 172.21.15.159:0/2982440486 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f603c06c8f0 0x7f603c06edb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6050009a10 tx=0x7f6050008040 comp rx=0 tx=0).stop 2024-04-04T06:11:06.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.351+0000 7f6067000640 1 -- 172.21.15.159:0/2982440486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60601504b0 msgr2=0x7f6060156790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:06.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.351+0000 7f6067000640 1 --2- 172.21.15.159:0/2982440486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60601504b0 0x7f6060156790 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f6058037040 tx=0x7f6058009750 comp rx=0 tx=0).stop 2024-04-04T06:11:06.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.352+0000 7f6067000640 1 -- 172.21.15.159:0/2982440486 shutdown_connections 2024-04-04T06:11:06.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.352+0000 7f6067000640 1 --2- 172.21.15.159:0/2982440486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60601504b0 0x7f6060156790 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:06.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.352+0000 7f6067000640 1 --2- 172.21.15.159:0/2982440486 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f603c06c8f0 0x7f603c06edb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:06.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.352+0000 7f6067000640 1 --2- 172.21.15.159:0/2982440486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f606014ffc0 0x7f6060156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:06.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.352+0000 7f6067000640 1 --2- 172.21.15.159:0/2982440486 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6040004d40 0x7f606015ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:06.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.352+0000 7f6067000640 1 -- 172.21.15.159:0/2982440486 >> 172.21.15.159:0/2982440486 conn(0x7f606006b990 msgr2=0x7f606015b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:06.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.352+0000 7f6067000640 1 -- 172.21.15.159:0/2982440486 shutdown_connections 2024-04-04T06:11:06.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.352+0000 7f6067000640 1 -- 172.21.15.159:0/2982440486 wait complete. 2024-04-04T06:11:06.365 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:10:42.595868+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:10:43.788847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:11:06.365 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:11:06.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.587+0000 7f45afe00640 1 Processor -- start 2024-04-04T06:11:06.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.587+0000 7f45afe00640 1 -- start start 2024-04-04T06:11:06.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.588+0000 7f45afe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45b00b4890 0x7f45b0072570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:06.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.588+0000 7f45afe00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45b00b4c70 con 0x7f45b015f6a0 2024-04-04T06:11:06.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.588+0000 7f45afe00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45b0072200 con 0x7f45b006d620 2024-04-04T06:11:06.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.588+0000 7f45afe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f45b00b3eb0 con 0x7f45b00b4890 2024-04-04T06:11:06.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.589+0000 7f45a6000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f45b015f6a0 0x7f45b00713e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40288/0 (socket says 172.21.15.159:40288) 2024-04-04T06:11:06.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.589+0000 7f45a6000640 1 -- 172.21.15.159:0/3270028315 learned_addr learned my addr 172.21.15.159:0/3270028315 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:11:06.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.589+0000 7f45aea00640 1 --2- 172.21.15.159:0/3270028315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45b00b4890 0x7f45b0072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:06.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.589+0000 7f45ae000640 1 -- 172.21.15.159:0/3270028315 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 773987581 0 0) 0x7f45b00b4c70 con 0x7f45b015f6a0 2024-04-04T06:11:06.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.589+0000 7f45ae000640 1 -- 172.21.15.159:0/3270028315 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4590005730 con 0x7f45b015f6a0 2024-04-04T06:11:06.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.590+0000 7f45ae000640 1 -- 172.21.15.159:0/3270028315 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2341079065 0 0) 0x7f4590005730 con 0x7f45b015f6a0 2024-04-04T06:11:06.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.590+0000 7f45ae000640 1 -- 172.21.15.159:0/3270028315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45b00b4890 msgr2=0x7f45b0072570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:06.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.590+0000 7f45ae000640 1 --2- 172.21.15.159:0/3270028315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45b00b4890 0x7f45b0072570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:06.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.590+0000 7f45ae000640 1 -- 172.21.15.159:0/3270028315 >> v1:172.21.15.178:6789/0 conn(0x7f45b006d620 legacy=0x7f45b0071af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:06.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.590+0000 7f45ae000640 1 -- 172.21.15.159:0/3270028315 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b015fb30 con 0x7f45b015f6a0 2024-04-04T06:11:06.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.590+0000 7f45ae000640 1 -- 172.21.15.159:0/3270028315 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f4598001670 con 0x7f45b015f6a0 2024-04-04T06:11:06.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.590+0000 7f45ae000640 1 -- 172.21.15.159:0/3270028315 >> v1:172.21.15.159:6789/0 conn(0x7f45b015f6a0 legacy=0x7f45b00713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:06.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.590+0000 7f45ae000640 1 --2- 172.21.15.159:0/3270028315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4590005bf0 0x7f4590005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:06.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.590+0000 7f45ae000640 1 --2- 172.21.15.159:0/3270028315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4590006510 0x7f4590006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:06.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.590+0000 7f45ae000640 1 --2- 172.21.15.159:0/3270028315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4590007150 0x7f459000ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:06.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.590+0000 7f45ae000640 1 -- 172.21.15.159:0/3270028315 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f45b00b3eb0 con 0x7f4590005bf0 2024-04-04T06:11:06.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.590+0000 7f45ae000640 1 -- 172.21.15.159:0/3270028315 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4590010510 con 0x7f4590007150 2024-04-04T06:11:06.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.590+0000 7f45ae000640 1 -- 172.21.15.159:0/3270028315 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4590010690 con 0x7f4590006510 2024-04-04T06:11:06.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.590+0000 7f45aea00640 1 --2- 172.21.15.159:0/3270028315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45b00b4890 0x7f45b0072570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:11:06.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.590+0000 7f45aea00640 1 --2- 172.21.15.159:0/3270028315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4590005bf0 0x7f4590005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:06.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.591+0000 7f45a6000640 1 --2- 172.21.15.159:0/3270028315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4590006510 0x7f4590006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:06.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.591+0000 7f45aea00640 1 -- 172.21.15.159:0/3270028315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4590006510 msgr2=0x7f4590006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:06.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.591+0000 7f45af400640 1 --2- 172.21.15.159:0/3270028315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4590007150 0x7f459000ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:06.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.591+0000 7f45aea00640 1 --2- 172.21.15.159:0/3270028315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4590006510 0x7f4590006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:06.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.591+0000 7f45aea00640 1 -- 172.21.15.159:0/3270028315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4590007150 msgr2=0x7f459000ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:06.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.591+0000 7f45aea00640 1 --2- 172.21.15.159:0/3270028315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4590007150 0x7f459000ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:06.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.591+0000 7f45aea00640 1 -- 172.21.15.159:0/3270028315 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4590005730 con 0x7f4590005bf0 2024-04-04T06:11:06.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.592+0000 7f45aea00640 1 --2- 172.21.15.159:0/3270028315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4590005bf0 0x7f4590005fd0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f45b0071060 tx=0x7f4590007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45438a3a9543320e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:06.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.592+0000 7f45ae000640 1 -- 172.21.15.159:0/3270028315 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a00097d0 con 0x7f4590005bf0 2024-04-04T06:11:06.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.592+0000 7f45ae000640 1 -- 172.21.15.159:0/3270028315 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a0004030 con 0x7f4590005bf0 2024-04-04T06:11:06.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.592+0000 7f45ae000640 1 -- 172.21.15.159:0/3270028315 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0004400 con 0x7f4590005bf0 2024-04-04T06:11:06.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.593+0000 7f45afe00640 1 -- 172.21.15.159:0/3270028315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4590005bf0 msgr2=0x7f4590005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:06.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.593+0000 7f45afe00640 1 --2- 172.21.15.159:0/3270028315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4590005bf0 0x7f4590005fd0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f45b0071060 tx=0x7f4590007a40 comp rx=0 tx=0).stop 2024-04-04T06:11:06.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.593+0000 7f45af400640 1 -- 172.21.15.159:0/3270028315 reap_dead start 2024-04-04T06:11:06.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.593+0000 7f45afe00640 1 -- 172.21.15.159:0/3270028315 shutdown_connections 2024-04-04T06:11:06.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.593+0000 7f45afe00640 1 -- 172.21.15.159:0/3270028315 >> 172.21.15.159:0/3270028315 conn(0x7f45b00b1ba0 msgr2=0x7f45b00b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:06.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.593+0000 7f45afe00640 1 -- 172.21.15.159:0/3270028315 shutdown_connections 2024-04-04T06:11:06.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.593+0000 7f45afe00640 1 -- 172.21.15.159:0/3270028315 wait complete. 2024-04-04T06:11:06.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.595+0000 7f45afe00640 1 Processor -- start 2024-04-04T06:11:06.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.596+0000 7f45afe00640 1 -- start start 2024-04-04T06:11:06.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.597+0000 7f45afe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4590005bf0 0x7f45b0148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:06.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.597+0000 7f45afe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45b015c700 0x7f45b01514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:06.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.597+0000 7f45afe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45b0156200 0x7f45b0156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:06.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.597+0000 7f45afe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f45b015bd70 con 0x7f45b015c700 2024-04-04T06:11:06.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.597+0000 7f45afe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45b015ba90 con 0x7f4590005bf0 2024-04-04T06:11:06.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.597+0000 7f45afe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f45b015c050 con 0x7f45b0156200 2024-04-04T06:11:06.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.597+0000 7f45a6000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45b015c700 0x7f45b01514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:06.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.598+0000 7f45a6000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45b015c700 0x7f45b01514e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44720/0 (socket says 172.21.15.159:44720) 2024-04-04T06:11:06.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.598+0000 7f45a6000640 1 -- 172.21.15.159:0/2871108087 learned_addr learned my addr 172.21.15.159:0/2871108087 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:06.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.598+0000 7f45aea00640 1 --2- 172.21.15.159:0/2871108087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4590005bf0 0x7f45b0148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:06.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.598+0000 7f45af400640 1 --2- 172.21.15.159:0/2871108087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45b0156200 0x7f45b0156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:06.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.598+0000 7f45a6000640 1 -- 172.21.15.159:0/2871108087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45b0156200 msgr2=0x7f45b0156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:06.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.598+0000 7f45a6000640 1 --2- 172.21.15.159:0/2871108087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45b0156200 0x7f45b0156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:06.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.598+0000 7f45a6000640 1 -- 172.21.15.159:0/2871108087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4590005bf0 msgr2=0x7f45b0148950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:06.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.598+0000 7f45a6000640 1 --2- 172.21.15.159:0/2871108087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4590005bf0 0x7f45b0148950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:06.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.598+0000 7f45a6000640 1 -- 172.21.15.159:0/2871108087 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b015cc00 con 0x7f45b015c700 2024-04-04T06:11:06.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.598+0000 7f45a6000640 1 --2- 172.21.15.159:0/2871108087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45b015c700 0x7f45b01514e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f4598001200 tx=0x7f459800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:06.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.599+0000 7f45a7400640 1 -- 172.21.15.159:0/2871108087 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459800c8a0 con 0x7f45b015c700 2024-04-04T06:11:06.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.599+0000 7f45a7400640 1 -- 172.21.15.159:0/2871108087 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4598018070 con 0x7f45b015c700 2024-04-04T06:11:06.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.599+0000 7f45a7400640 1 -- 172.21.15.159:0/2871108087 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459800cc60 con 0x7f45b015c700 2024-04-04T06:11:06.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.599+0000 7f45afe00640 1 -- 172.21.15.159:0/2871108087 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b015cee0 con 0x7f45b015c700 2024-04-04T06:11:06.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.599+0000 7f45afe00640 1 -- 172.21.15.159:0/2871108087 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b015d4a0 con 0x7f45b015c700 2024-04-04T06:11:06.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.600+0000 7f45afe00640 1 -- 172.21.15.159:0/2871108087 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b0075940 con 0x7f45b015c700 2024-04-04T06:11:06.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.602+0000 7f45a7400640 1 -- 172.21.15.159:0/2871108087 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f459801d460 con 0x7f45b015c700 2024-04-04T06:11:06.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.602+0000 7f45a7400640 1 --2- 172.21.15.159:0/2871108087 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f458c06c920 0x7f458c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:06.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.603+0000 7f45a7400640 1 -- 172.21.15.159:0/2871108087 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f459808ba10 con 0x7f45b015c700 2024-04-04T06:11:06.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.603+0000 7f45aea00640 1 --2- 172.21.15.159:0/2871108087 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f458c06c920 0x7f458c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:06.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.604+0000 7f45aea00640 1 --2- 172.21.15.159:0/2871108087 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f458c06c920 0x7f458c06ede0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f45a0035000 tx=0x7f45a0030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:06.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.607+0000 7f45a7400640 1 -- 172.21.15.159:0/2871108087 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4598059e70 con 0x7f45b015c700 2024-04-04T06:11:06.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.923+0000 7f45afe00640 1 -- 172.21.15.159:0/2871108087 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f45b0153550 con 0x7f45b015c700 2024-04-04T06:11:06.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.924+0000 7f45a7400640 1 -- 172.21.15.159:0/2871108087 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f4598059770 con 0x7f45b015c700 2024-04-04T06:11:06.917 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:11:06.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.927+0000 7f45afe00640 1 -- 172.21.15.159:0/2871108087 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f458c06c920 msgr2=0x7f458c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:06.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.927+0000 7f45afe00640 1 --2- 172.21.15.159:0/2871108087 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f458c06c920 0x7f458c06ede0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f45a0035000 tx=0x7f45a0030000 comp rx=0 tx=0).stop 2024-04-04T06:11:06.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.927+0000 7f45afe00640 1 -- 172.21.15.159:0/2871108087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45b015c700 msgr2=0x7f45b01514e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:06.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.927+0000 7f45afe00640 1 --2- 172.21.15.159:0/2871108087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45b015c700 0x7f45b01514e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f4598001200 tx=0x7f459800ec80 comp rx=0 tx=0).stop 2024-04-04T06:11:06.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.927+0000 7f45afe00640 1 -- 172.21.15.159:0/2871108087 shutdown_connections 2024-04-04T06:11:06.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.927+0000 7f45afe00640 1 --2- 172.21.15.159:0/2871108087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f45b0156200 0x7f45b0156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:06.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.927+0000 7f45afe00640 1 --2- 172.21.15.159:0/2871108087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45b015c700 0x7f45b01514e0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:06.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.927+0000 7f45afe00640 1 --2- 172.21.15.159:0/2871108087 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f458c06c920 0x7f458c06ede0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:06.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.927+0000 7f45afe00640 1 --2- 172.21.15.159:0/2871108087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4590005bf0 0x7f45b0148950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:06.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.927+0000 7f45afe00640 1 -- 172.21.15.159:0/2871108087 >> 172.21.15.159:0/2871108087 conn(0x7f45b00b1ba0 msgr2=0x7f45b00b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:06.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.927+0000 7f45afe00640 1 -- 172.21.15.159:0/2871108087 shutdown_connections 2024-04-04T06:11:06.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:06.927+0000 7f45afe00640 1 -- 172.21.15.159:0/2871108087 wait complete. 2024-04-04T06:11:06.938 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:11:07.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.144+0000 7f10e9a00640 1 Processor -- start 2024-04-04T06:11:07.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.145+0000 7f10e9a00640 1 -- start start 2024-04-04T06:11:07.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.147+0000 7f10e9a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f10e4161f60 0x7f10e40d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:07.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.147+0000 7f10e9a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f10e40b3170 0x7f10e40d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:07.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.147+0000 7f10e9a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10e40b5ce0 con 0x7f10e40b3da0 2024-04-04T06:11:07.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.147+0000 7f10e9a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f10e40d27f0 con 0x7f10e4161f60 2024-04-04T06:11:07.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.147+0000 7f10e9a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f10e40d2a90 con 0x7f10e40b3170 2024-04-04T06:11:07.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.147+0000 7f10e2a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f10e4161f60 0x7f10e40d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:07.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.148+0000 7f10e3400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f10e40b3da0 0x7f10e40b4230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40296/0 (socket says 172.21.15.159:40296) 2024-04-04T06:11:07.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.148+0000 7f10e2a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f10e4161f60 0x7f10e40d0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44724/0 (socket says 172.21.15.159:44724) 2024-04-04T06:11:07.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.148+0000 7f10e3400640 1 -- 172.21.15.159:0/425886879 learned_addr learned my addr 172.21.15.159:0/425886879 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:11:07.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.148+0000 7f10da000640 1 --2- 172.21.15.159:0/425886879 >> v2:172.21.15.178:3300/0 conn(0x7f10e40b3170 0x7f10e40d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:07.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.148+0000 7f10e2a00640 1 -- 172.21.15.159:0/425886879 >> v2:172.21.15.178:3300/0 conn(0x7f10e40b3170 msgr2=0x7f10e40d13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:07.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.148+0000 7f10e2a00640 1 --2- 172.21.15.159:0/425886879 >> v2:172.21.15.178:3300/0 conn(0x7f10e40b3170 0x7f10e40d13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:07.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.148+0000 7f10e2a00640 1 -- 172.21.15.159:0/425886879 >> v1:172.21.15.159:6789/0 conn(0x7f10e40b3da0 legacy=0x7f10e40b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:07.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.148+0000 7f10e2a00640 1 -- 172.21.15.159:0/425886879 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e4162340 con 0x7f10e4161f60 2024-04-04T06:11:07.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.148+0000 7f10e2000640 1 -- 172.21.15.159:0/425886879 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4112436372 0 0) 0x7f10dc001110 con 0x7f10e40b3da0 2024-04-04T06:11:07.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.148+0000 7f10da000640 1 --2- 172.21.15.159:0/425886879 >> v2:172.21.15.178:3300/0 conn(0x7f10e40b3170 0x7f10e40d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:11:07.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.149+0000 7f10e2a00640 1 --2- 172.21.15.159:0/425886879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10e4161f60 0x7f10e40d0e70 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f10e408c880 tx=0x7f10d402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4f6d5527e91e6e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:07.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.149+0000 7f10e2000640 1 -- 172.21.15.159:0/425886879 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4039070 con 0x7f10e4161f60 2024-04-04T06:11:07.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.149+0000 7f10e2000640 1 -- 172.21.15.159:0/425886879 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d402cc80 con 0x7f10e4161f60 2024-04-04T06:11:07.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.149+0000 7f10e2000640 1 -- 172.21.15.159:0/425886879 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4034660 con 0x7f10e4161f60 2024-04-04T06:11:07.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.150+0000 7f10e9a00640 1 -- 172.21.15.159:0/425886879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10e4161f60 msgr2=0x7f10e40d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:07.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.150+0000 7f10e9a00640 1 --2- 172.21.15.159:0/425886879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10e4161f60 0x7f10e40d0e70 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f10e408c880 tx=0x7f10d402bed0 comp rx=0 tx=0).stop 2024-04-04T06:11:07.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.150+0000 7f10e9a00640 1 -- 172.21.15.159:0/425886879 shutdown_connections 2024-04-04T06:11:07.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.150+0000 7f10e9a00640 1 --2- 172.21.15.159:0/425886879 >> v2:172.21.15.178:3300/0 conn(0x7f10e40b3170 0x7f10e40d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:07.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.150+0000 7f10e9a00640 1 --2- 172.21.15.159:0/425886879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10e4161f60 0x7f10e40d0e70 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:07.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.150+0000 7f10e9a00640 1 -- 172.21.15.159:0/425886879 >> 172.21.15.159:0/425886879 conn(0x7f10e408bfb0 msgr2=0x7f10e40d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:07.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.150+0000 7f10e9a00640 1 -- 172.21.15.159:0/425886879 shutdown_connections 2024-04-04T06:11:07.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.150+0000 7f10e9a00640 1 -- 172.21.15.159:0/425886879 wait complete. 2024-04-04T06:11:07.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.152+0000 7f10e9a00640 1 Processor -- start 2024-04-04T06:11:07.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.152+0000 7f10e9a00640 1 -- start start 2024-04-04T06:11:07.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.153+0000 7f10e9a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10e40b3170 0x7f10e40cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:07.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.153+0000 7f10e9a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10e40b3da0 0x7f10e415dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:07.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.153+0000 7f10e9a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10e4161f60 0x7f10e406cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:07.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.153+0000 7f10e9a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f10e40d3310 con 0x7f10e4161f60 2024-04-04T06:11:07.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.153+0000 7f10e9a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f10e40d3030 con 0x7f10e40b3da0 2024-04-04T06:11:07.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.153+0000 7f10e9a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f10e40d00e0 con 0x7f10e40b3170 2024-04-04T06:11:07.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.153+0000 7f10e3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10e4161f60 0x7f10e406cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:07.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.153+0000 7f10e3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10e4161f60 0x7f10e406cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44730/0 (socket says 172.21.15.159:44730) 2024-04-04T06:11:07.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.153+0000 7f10e3400640 1 -- 172.21.15.159:0/1108486968 learned_addr learned my addr 172.21.15.159:0/1108486968 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:07.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.153+0000 7f10da000640 1 --2- 172.21.15.159:0/1108486968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10e40b3da0 0x7f10e415dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:07.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.153+0000 7f10e3400640 1 -- 172.21.15.159:0/1108486968 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10e40b3170 msgr2=0x7f10e40cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:07.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.153+0000 7f10e3400640 1 --2- 172.21.15.159:0/1108486968 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10e40b3170 0x7f10e40cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:07.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.153+0000 7f10e3400640 1 -- 172.21.15.159:0/1108486968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10e40b3da0 msgr2=0x7f10e415dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:07.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.154+0000 7f10e3400640 1 --2- 172.21.15.159:0/1108486968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10e40b3da0 0x7f10e415dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:07.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.154+0000 7f10e3400640 1 -- 172.21.15.159:0/1108486968 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e40cbe50 con 0x7f10e4161f60 2024-04-04T06:11:07.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.154+0000 7f10da000640 1 --2- 172.21.15.159:0/1108486968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10e40b3da0 0x7f10e415dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:11:07.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.154+0000 7f10e3400640 1 --2- 172.21.15.159:0/1108486968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10e4161f60 0x7f10e406cdb0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00c780 tx=0x7f10dc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:07.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.154+0000 7f10db400640 1 -- 172.21.15.159:0/1108486968 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc018070 con 0x7f10e4161f60 2024-04-04T06:11:07.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.154+0000 7f10db400640 1 -- 172.21.15.159:0/1108486968 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10dc0070d0 con 0x7f10e4161f60 2024-04-04T06:11:07.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.154+0000 7f10db400640 1 -- 172.21.15.159:0/1108486968 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc007500 con 0x7f10e4161f60 2024-04-04T06:11:07.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.154+0000 7f10e9a00640 1 -- 172.21.15.159:0/1108486968 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e40cc130 con 0x7f10e4161f60 2024-04-04T06:11:07.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.154+0000 7f10e9a00640 1 -- 172.21.15.159:0/1108486968 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e40c9410 con 0x7f10e4161f60 2024-04-04T06:11:07.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.155+0000 7f10e9a00640 1 -- 172.21.15.159:0/1108486968 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e40c9670 con 0x7f10e4161f60 2024-04-04T06:11:07.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.156+0000 7f10db400640 1 -- 172.21.15.159:0/1108486968 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10dc01b070 con 0x7f10e4161f60 2024-04-04T06:11:07.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.157+0000 7f10db400640 1 --2- 172.21.15.159:0/1108486968 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f10c006c8f0 0x7f10c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:07.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.157+0000 7f10db400640 1 -- 172.21.15.159:0/1108486968 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f10dc08aef0 con 0x7f10e4161f60 2024-04-04T06:11:07.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.158+0000 7f10e2a00640 1 --2- 172.21.15.159:0/1108486968 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f10c006c8f0 0x7f10c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:07.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.159+0000 7f10e2a00640 1 --2- 172.21.15.159:0/1108486968 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f10c006c8f0 0x7f10c006edb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f10d4036000 tx=0x7f10d402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:07.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.162+0000 7f10db400640 1 -- 172.21.15.159:0/1108486968 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f10e40b5ce0 con 0x7f10e4161f60 2024-04-04T06:11:07.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.364+0000 7f10e9a00640 1 -- 172.21.15.159:0/1108486968 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f10e40d3f00 con 0x7f10e4161f60 2024-04-04T06:11:07.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.365+0000 7f10db400640 1 -- 172.21.15.159:0/1108486968 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) ==== 104+0+11 (secure 0 0 0) 0x7f10dc059220 con 0x7f10e4161f60 2024-04-04T06:11:07.358 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:11:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.368+0000 7f10e9a00640 1 -- 172.21.15.159:0/1108486968 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f10c006c8f0 msgr2=0x7f10c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.368+0000 7f10e9a00640 1 --2- 172.21.15.159:0/1108486968 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f10c006c8f0 0x7f10c006edb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f10d4036000 tx=0x7f10d402e000 comp rx=0 tx=0).stop 2024-04-04T06:11:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.368+0000 7f10e9a00640 1 -- 172.21.15.159:0/1108486968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10e4161f60 msgr2=0x7f10e406cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.368+0000 7f10e9a00640 1 --2- 172.21.15.159:0/1108486968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10e4161f60 0x7f10e406cdb0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00c780 tx=0x7f10dc00cc40 comp rx=0 tx=0).stop 2024-04-04T06:11:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.369+0000 7f10e9a00640 1 -- 172.21.15.159:0/1108486968 shutdown_connections 2024-04-04T06:11:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.369+0000 7f10e9a00640 1 --2- 172.21.15.159:0/1108486968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10e4161f60 0x7f10e406cdb0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.369+0000 7f10e9a00640 1 --2- 172.21.15.159:0/1108486968 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f10c006c8f0 0x7f10c006edb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.369+0000 7f10e9a00640 1 --2- 172.21.15.159:0/1108486968 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10e40b3da0 0x7f10e415dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.369+0000 7f10e9a00640 1 --2- 172.21.15.159:0/1108486968 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10e40b3170 0x7f10e40cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.369+0000 7f10e9a00640 1 -- 172.21.15.159:0/1108486968 >> 172.21.15.159:0/1108486968 conn(0x7f10e408bfb0 msgr2=0x7f10e40cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:07.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.369+0000 7f10e9a00640 1 -- 172.21.15.159:0/1108486968 shutdown_connections 2024-04-04T06:11:07.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.369+0000 7f10e9a00640 1 -- 172.21.15.159:0/1108486968 wait complete. 2024-04-04T06:11:07.385 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:11:07.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.599+0000 7f7a7ac00640 1 Processor -- start 2024-04-04T06:11:07.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.599+0000 7f7a7ac00640 1 -- start start 2024-04-04T06:11:07.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.601+0000 7f7a7ac00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7a7415e4c0 0x7f7a740b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:07.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.601+0000 7f7a7ac00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a740b69c0 0x7f7a740b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:07.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.601+0000 7f7a7ac00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a740d2710 con 0x7f7a740d37b0 2024-04-04T06:11:07.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.601+0000 7f7a7ac00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f7a7415e8a0 con 0x7f7a7415e4c0 2024-04-04T06:11:07.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.601+0000 7f7a7ac00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7a740d4040 con 0x7f7a740b69c0 2024-04-04T06:11:07.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.601+0000 7f7a79800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7a7415e4c0 0x7f7a740b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:07.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.601+0000 7f7a79800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7a7415e4c0 0x7f7a740b9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44732/0 (socket says 172.21.15.159:44732) 2024-04-04T06:11:07.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.601+0000 7f7a79800640 1 -- 172.21.15.159:0/3907195014 learned_addr learned my addr 172.21.15.159:0/3907195014 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:07.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.602+0000 7f7a7a200640 1 --2- 172.21.15.159:0/3907195014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a740b69c0 0x7f7a740b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:07.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.602+0000 7f7a79800640 1 -- 172.21.15.159:0/3907195014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a740b69c0 msgr2=0x7f7a740b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:07.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.602+0000 7f7a79800640 1 --2- 172.21.15.159:0/3907195014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a740b69c0 0x7f7a740b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:07.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.602+0000 7f7a79800640 1 -- 172.21.15.159:0/3907195014 >> v1:172.21.15.178:6789/0 conn(0x7f7a740d37b0 legacy=0x7f7a740b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:07.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.602+0000 7f7a79800640 1 -- 172.21.15.159:0/3907195014 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a740d29f0 con 0x7f7a7415e4c0 2024-04-04T06:11:07.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.603+0000 7f7a79800640 1 --2- 172.21.15.159:0/3907195014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a7415e4c0 0x7f7a740b9bf0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009870 tx=0x7f7a6802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d92316a3b8de366 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:07.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.603+0000 7f7a73e00640 1 -- 172.21.15.159:0/3907195014 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68035020 con 0x7f7a7415e4c0 2024-04-04T06:11:07.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.603+0000 7f7a73e00640 1 -- 172.21.15.159:0/3907195014 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a68031640 con 0x7f7a7415e4c0 2024-04-04T06:11:07.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.603+0000 7f7a73e00640 1 -- 172.21.15.159:0/3907195014 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6802d040 con 0x7f7a7415e4c0 2024-04-04T06:11:07.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.604+0000 7f7a7ac00640 1 -- 172.21.15.159:0/3907195014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a7415e4c0 msgr2=0x7f7a740b9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:07.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.604+0000 7f7a7ac00640 1 --2- 172.21.15.159:0/3907195014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a7415e4c0 0x7f7a740b9bf0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009870 tx=0x7f7a6802b8f0 comp rx=0 tx=0).stop 2024-04-04T06:11:07.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.604+0000 7f7a7ac00640 1 -- 172.21.15.159:0/3907195014 shutdown_connections 2024-04-04T06:11:07.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.604+0000 7f7a7ac00640 1 --2- 172.21.15.159:0/3907195014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a740b69c0 0x7f7a740b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:07.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.604+0000 7f7a7ac00640 1 --2- 172.21.15.159:0/3907195014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a7415e4c0 0x7f7a740b9bf0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:07.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.604+0000 7f7a7ac00640 1 -- 172.21.15.159:0/3907195014 >> 172.21.15.159:0/3907195014 conn(0x7f7a740676a0 msgr2=0x7f7a74067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:07.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.604+0000 7f7a7ac00640 1 -- 172.21.15.159:0/3907195014 shutdown_connections 2024-04-04T06:11:07.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.604+0000 7f7a7ac00640 1 -- 172.21.15.159:0/3907195014 wait complete. 2024-04-04T06:11:07.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.606+0000 7f7a7ac00640 1 Processor -- start 2024-04-04T06:11:07.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.606+0000 7f7a7ac00640 1 -- start start 2024-04-04T06:11:07.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.607+0000 7f7a7ac00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a740b69c0 0x7f7a740c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:07.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.607+0000 7f7a7ac00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a740d37b0 0x7f7a740c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:07.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.607+0000 7f7a7ac00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a7415e4c0 0x7f7a741523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:07.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.607+0000 7f7a7ac00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7a740d2fb0 con 0x7f7a740b69c0 2024-04-04T06:11:07.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.607+0000 7f7a7ac00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7a740d3320 con 0x7f7a7415e4c0 2024-04-04T06:11:07.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.607+0000 7f7a7ac00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7a740ba130 con 0x7f7a740d37b0 2024-04-04T06:11:07.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.607+0000 7f7a79800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a740b69c0 0x7f7a740c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:07.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.607+0000 7f7a79800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a740b69c0 0x7f7a740c7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44736/0 (socket says 172.21.15.159:44736) 2024-04-04T06:11:07.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.607+0000 7f7a79800640 1 -- 172.21.15.159:0/2502852249 learned_addr learned my addr 172.21.15.159:0/2502852249 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:07.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.608+0000 7f7a78e00640 1 --2- 172.21.15.159:0/2502852249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a740d37b0 0x7f7a740c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:07.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.608+0000 7f7a7a200640 1 --2- 172.21.15.159:0/2502852249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a7415e4c0 0x7f7a741523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:07.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.608+0000 7f7a79800640 1 -- 172.21.15.159:0/2502852249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a740d37b0 msgr2=0x7f7a740c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:07.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.608+0000 7f7a79800640 1 --2- 172.21.15.159:0/2502852249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a740d37b0 0x7f7a740c9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:07.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.608+0000 7f7a79800640 1 -- 172.21.15.159:0/2502852249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a7415e4c0 msgr2=0x7f7a741523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:07.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.608+0000 7f7a79800640 1 --2- 172.21.15.159:0/2502852249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a7415e4c0 0x7f7a741523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:07.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.608+0000 7f7a79800640 1 -- 172.21.15.159:0/2502852249 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7415f540 con 0x7f7a740b69c0 2024-04-04T06:11:07.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.608+0000 7f7a78e00640 1 --2- 172.21.15.159:0/2502852249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a740d37b0 0x7f7a740c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:11:07.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.608+0000 7f7a79800640 1 --2- 172.21.15.159:0/2502852249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a740b69c0 0x7f7a740c7de0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f7a6802b930 tx=0x7f7a680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:07.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.609+0000 7f7a71600640 1 -- 172.21.15.159:0/2502852249 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68040070 con 0x7f7a740b69c0 2024-04-04T06:11:07.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.609+0000 7f7a71600640 1 -- 172.21.15.159:0/2502852249 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a68039c10 con 0x7f7a740b69c0 2024-04-04T06:11:07.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.609+0000 7f7a7ac00640 1 -- 172.21.15.159:0/2502852249 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a741553e0 con 0x7f7a740b69c0 2024-04-04T06:11:07.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.609+0000 7f7a71600640 1 -- 172.21.15.159:0/2502852249 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68035040 con 0x7f7a740b69c0 2024-04-04T06:11:07.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.609+0000 7f7a7ac00640 1 -- 172.21.15.159:0/2502852249 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a740c8bf0 con 0x7f7a740b69c0 2024-04-04T06:11:07.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.610+0000 7f7a7ac00640 1 -- 172.21.15.159:0/2502852249 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a74075940 con 0x7f7a740b69c0 2024-04-04T06:11:07.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.611+0000 7f7a71600640 1 -- 172.21.15.159:0/2502852249 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a6804b060 con 0x7f7a740b69c0 2024-04-04T06:11:07.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.612+0000 7f7a71600640 1 --2- 172.21.15.159:0/2502852249 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a5406c8f0 0x7f7a5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:07.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.612+0000 7f7a71600640 1 -- 172.21.15.159:0/2502852249 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f7a680ad750 con 0x7f7a740b69c0 2024-04-04T06:11:07.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.612+0000 7f7a78e00640 1 --2- 172.21.15.159:0/2502852249 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a5406c8f0 0x7f7a5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:07.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.613+0000 7f7a78e00640 1 --2- 172.21.15.159:0/2502852249 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a5406c8f0 0x7f7a5406edb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7a740d12e0 tx=0x7f7a60009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:07.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.616+0000 7f7a71600640 1 -- 172.21.15.159:0/2502852249 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7a6802f300 con 0x7f7a740b69c0 2024-04-04T06:11:07.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.868+0000 7f7a7ac00640 1 -- 172.21.15.159:0/2502852249 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7a740b4560 con 0x7f7a740b69c0 2024-04-04T06:11:07.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.869+0000 7f7a71600640 1 -- 172.21.15.159:0/2502852249 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) ==== 100+0+11 (secure 0 0 0) 0x7f7a6807ba80 con 0x7f7a740b69c0 2024-04-04T06:11:07.862 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:11:07.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.872+0000 7f7a7ac00640 1 -- 172.21.15.159:0/2502852249 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a5406c8f0 msgr2=0x7f7a5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:07.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.872+0000 7f7a7ac00640 1 --2- 172.21.15.159:0/2502852249 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a5406c8f0 0x7f7a5406edb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7a740d12e0 tx=0x7f7a60009040 comp rx=0 tx=0).stop 2024-04-04T06:11:07.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.872+0000 7f7a7ac00640 1 -- 172.21.15.159:0/2502852249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a740b69c0 msgr2=0x7f7a740c7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:07.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.872+0000 7f7a7ac00640 1 --2- 172.21.15.159:0/2502852249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a740b69c0 0x7f7a740c7de0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f7a6802b930 tx=0x7f7a680046c0 comp rx=0 tx=0).stop 2024-04-04T06:11:07.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.872+0000 7f7a7ac00640 1 -- 172.21.15.159:0/2502852249 shutdown_connections 2024-04-04T06:11:07.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.872+0000 7f7a7ac00640 1 --2- 172.21.15.159:0/2502852249 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7a5406c8f0 0x7f7a5406edb0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:07.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.872+0000 7f7a7ac00640 1 --2- 172.21.15.159:0/2502852249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a7415e4c0 0x7f7a741523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:07.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.872+0000 7f7a7ac00640 1 --2- 172.21.15.159:0/2502852249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7a740d37b0 0x7f7a740c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:07.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.872+0000 7f7a7ac00640 1 --2- 172.21.15.159:0/2502852249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a740b69c0 0x7f7a740c7de0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:07.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.872+0000 7f7a7ac00640 1 -- 172.21.15.159:0/2502852249 >> 172.21.15.159:0/2502852249 conn(0x7f7a740676a0 msgr2=0x7f7a7415d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:07.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.873+0000 7f7a7ac00640 1 -- 172.21.15.159:0/2502852249 shutdown_connections 2024-04-04T06:11:07.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:07.873+0000 7f7a7ac00640 1 -- 172.21.15.159:0/2502852249 wait complete. 2024-04-04T06:11:07.885 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-04T06:11:08.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.114+0000 7f3d64c00640 1 Processor -- start 2024-04-04T06:11:08.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.114+0000 7f3d64c00640 1 -- start start 2024-04-04T06:11:08.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.116+0000 7f3d64c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d60153580 0x7f3d600d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.116+0000 7f3d64c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3d600d07c0 0x7f3d600aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.116+0000 7f3d64c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d600afab0 con 0x7f3d600af2e0 2024-04-04T06:11:08.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.116+0000 7f3d64c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f3d600d0c00 con 0x7f3d600d07c0 2024-04-04T06:11:08.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.116+0000 7f3d64c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6015cc80 con 0x7f3d60153580 2024-04-04T06:11:08.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.116+0000 7f3d57e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3d600d07c0 0x7f3d600aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:08.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.116+0000 7f3d57e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3d600d07c0 0x7f3d600aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44742/0 (socket says 172.21.15.159:44742) 2024-04-04T06:11:08.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.116+0000 7f3d57e00640 1 -- 172.21.15.159:0/762323574 learned_addr learned my addr 172.21.15.159:0/762323574 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:08.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.117+0000 7f3d5de00640 1 --2- 172.21.15.159:0/762323574 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d60153580 0x7f3d600d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:08.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.117+0000 7f3d57e00640 1 -- 172.21.15.159:0/762323574 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d60153580 msgr2=0x7f3d600d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:08.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.117+0000 7f3d57e00640 1 --2- 172.21.15.159:0/762323574 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d60153580 0x7f3d600d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.117+0000 7f3d57e00640 1 -- 172.21.15.159:0/762323574 >> v1:172.21.15.178:6789/0 conn(0x7f3d600af2e0 legacy=0x7f3d6006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:08.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.117+0000 7f3d57e00640 1 -- 172.21.15.159:0/762323574 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6015da00 con 0x7f3d600d07c0 2024-04-04T06:11:08.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.117+0000 7f3d5de00640 1 --2- 172.21.15.159:0/762323574 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d60153580 0x7f3d600d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:11:08.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.118+0000 7f3d57e00640 1 --2- 172.21.15.159:0/762323574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d600d07c0 0x7f3d600aeda0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f3d48009870 tx=0x7f3d4802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19eed042b39ad89f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:08.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.118+0000 7f3d5d400640 1 -- 172.21.15.159:0/762323574 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d480317c0 con 0x7f3d600d07c0 2024-04-04T06:11:08.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.118+0000 7f3d5d400640 1 -- 172.21.15.159:0/762323574 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d48037070 con 0x7f3d600d07c0 2024-04-04T06:11:08.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.119+0000 7f3d5d400640 1 -- 172.21.15.159:0/762323574 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d48031b80 con 0x7f3d600d07c0 2024-04-04T06:11:08.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.119+0000 7f3d64c00640 1 -- 172.21.15.159:0/762323574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d600d07c0 msgr2=0x7f3d600aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:08.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.119+0000 7f3d64c00640 1 --2- 172.21.15.159:0/762323574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d600d07c0 0x7f3d600aeda0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f3d48009870 tx=0x7f3d4802b860 comp rx=0 tx=0).stop 2024-04-04T06:11:08.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.119+0000 7f3d64c00640 1 -- 172.21.15.159:0/762323574 shutdown_connections 2024-04-04T06:11:08.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.119+0000 7f3d64c00640 1 --2- 172.21.15.159:0/762323574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d600d07c0 0x7f3d600aeda0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.119+0000 7f3d64c00640 1 --2- 172.21.15.159:0/762323574 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d60153580 0x7f3d600d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.119+0000 7f3d64c00640 1 -- 172.21.15.159:0/762323574 >> 172.21.15.159:0/762323574 conn(0x7f3d6006b750 msgr2=0x7f3d6006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:08.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.120+0000 7f3d64c00640 1 -- 172.21.15.159:0/762323574 shutdown_connections 2024-04-04T06:11:08.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.120+0000 7f3d64c00640 1 -- 172.21.15.159:0/762323574 wait complete. 2024-04-04T06:11:08.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.122+0000 7f3d64c00640 1 Processor -- start 2024-04-04T06:11:08.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.122+0000 7f3d64c00640 1 -- start start 2024-04-04T06:11:08.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.124+0000 7f3d64c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d600af2e0 0x7f3d60144f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.124+0000 7f3d64c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d600d07c0 0x7f3d60154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.124+0000 7f3d64c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d60153580 0x7f3d601571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.124+0000 7f3d64c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6015d4e0 con 0x7f3d600d07c0 2024-04-04T06:11:08.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.124+0000 7f3d64c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6015d200 con 0x7f3d60153580 2024-04-04T06:11:08.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.124+0000 7f3d64c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3d60153960 con 0x7f3d600af2e0 2024-04-04T06:11:08.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.124+0000 7f3d57e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d600d07c0 0x7f3d60154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:08.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.124+0000 7f3d57e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d600d07c0 0x7f3d60154c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44748/0 (socket says 172.21.15.159:44748) 2024-04-04T06:11:08.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.124+0000 7f3d57e00640 1 -- 172.21.15.159:0/4136633478 learned_addr learned my addr 172.21.15.159:0/4136633478 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:08.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.125+0000 7f3d57e00640 1 -- 172.21.15.159:0/4136633478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d600af2e0 msgr2=0x7f3d60144f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:08.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.125+0000 7f3d57e00640 1 --2- 172.21.15.159:0/4136633478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d600af2e0 0x7f3d60144f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.125+0000 7f3d57e00640 1 -- 172.21.15.159:0/4136633478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d60153580 msgr2=0x7f3d601571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:08.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.125+0000 7f3d5e800640 1 --2- 172.21.15.159:0/4136633478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d60153580 0x7f3d601571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:08.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.125+0000 7f3d57e00640 1 --2- 172.21.15.159:0/4136633478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d60153580 0x7f3d601571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.125+0000 7f3d57e00640 1 -- 172.21.15.159:0/4136633478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d601555c0 con 0x7f3d600d07c0 2024-04-04T06:11:08.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.125+0000 7f3d57e00640 1 --2- 172.21.15.159:0/4136633478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d600d07c0 0x7f3d60154c50 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3d4802b8a0 tx=0x7f3d480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:08.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.125+0000 7f3d56000640 1 -- 172.21.15.159:0/4136633478 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d48032860 con 0x7f3d600d07c0 2024-04-04T06:11:08.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.126+0000 7f3d56000640 1 -- 172.21.15.159:0/4136633478 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d48032a00 con 0x7f3d600d07c0 2024-04-04T06:11:08.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.126+0000 7f3d56000640 1 -- 172.21.15.159:0/4136633478 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d48032df0 con 0x7f3d600d07c0 2024-04-04T06:11:08.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.126+0000 7f3d64c00640 1 -- 172.21.15.159:0/4136633478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d601558a0 con 0x7f3d600d07c0 2024-04-04T06:11:08.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.126+0000 7f3d64c00640 1 -- 172.21.15.159:0/4136633478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d60155e60 con 0x7f3d600d07c0 2024-04-04T06:11:08.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.127+0000 7f3d64c00640 1 -- 172.21.15.159:0/4136633478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d600685c0 con 0x7f3d600d07c0 2024-04-04T06:11:08.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.128+0000 7f3d56000640 1 -- 172.21.15.159:0/4136633478 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d4804c060 con 0x7f3d600d07c0 2024-04-04T06:11:08.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.129+0000 7f3d56000640 1 --2- 172.21.15.159:0/4136633478 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3d3c06c8f0 0x7f3d3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.130+0000 7f3d56000640 1 -- 172.21.15.159:0/4136633478 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f3d480ad950 con 0x7f3d600d07c0 2024-04-04T06:11:08.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.130+0000 7f3d5de00640 1 --2- 172.21.15.159:0/4136633478 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3d3c06c8f0 0x7f3d3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:08.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.131+0000 7f3d5de00640 1 --2- 172.21.15.159:0/4136633478 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3d3c06c8f0 0x7f3d3c06edb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3d50006fd0 tx=0x7f3d50008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:08.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.134+0000 7f3d56000640 1 -- 172.21.15.159:0/4136633478 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3d48040070 con 0x7f3d600d07c0 2024-04-04T06:11:08.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.345+0000 7f3d64c00640 1 -- 172.21.15.159:0/4136633478 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3d601454a0 con 0x7f3d600d07c0 2024-04-04T06:11:08.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.346+0000 7f3d56000640 1 -- 172.21.15.159:0/4136633478 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v72) ==== 80+0+124 (secure 0 0 0) 0x7f3d4807b6d0 con 0x7f3d600d07c0 2024-04-04T06:11:08.339 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6835/1652468640 2024-04-05T06:10:57.934622+0000 2024-04-04T06:11:08.339 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6834/1652468640 2024-04-05T06:10:57.934622+0000 2024-04-04T06:11:08.339 INFO:teuthology.orchestra.run.smithi159.stderr:listed 2 entries 2024-04-04T06:11:08.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.348+0000 7f3d64c00640 1 -- 172.21.15.159:0/4136633478 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3d3c06c8f0 msgr2=0x7f3d3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:08.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.348+0000 7f3d64c00640 1 --2- 172.21.15.159:0/4136633478 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3d3c06c8f0 0x7f3d3c06edb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3d50006fd0 tx=0x7f3d50008040 comp rx=0 tx=0).stop 2024-04-04T06:11:08.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.348+0000 7f3d64c00640 1 -- 172.21.15.159:0/4136633478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d600d07c0 msgr2=0x7f3d60154c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:08.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.348+0000 7f3d64c00640 1 --2- 172.21.15.159:0/4136633478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d600d07c0 0x7f3d60154c50 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3d4802b8a0 tx=0x7f3d480046c0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.349+0000 7f3d64c00640 1 -- 172.21.15.159:0/4136633478 shutdown_connections 2024-04-04T06:11:08.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.349+0000 7f3d64c00640 1 --2- 172.21.15.159:0/4136633478 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3d3c06c8f0 0x7f3d3c06edb0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.349+0000 7f3d64c00640 1 --2- 172.21.15.159:0/4136633478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d60153580 0x7f3d601571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.349+0000 7f3d64c00640 1 --2- 172.21.15.159:0/4136633478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d600d07c0 0x7f3d60154c50 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.349+0000 7f3d64c00640 1 --2- 172.21.15.159:0/4136633478 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d600af2e0 0x7f3d60144f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.349+0000 7f3d64c00640 1 -- 172.21.15.159:0/4136633478 >> 172.21.15.159:0/4136633478 conn(0x7f3d6006b750 msgr2=0x7f3d6006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:08.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.349+0000 7f3d64c00640 1 -- 172.21.15.159:0/4136633478 shutdown_connections 2024-04-04T06:11:08.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.349+0000 7f3d64c00640 1 -- 172.21.15.159:0/4136633478 wait complete. 2024-04-04T06:11:08.363 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-04T06:11:08.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:11:08.363 DEBUG:teuthology.orchestra.run.smithi159:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-04T06:11:08.457 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:11:08.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:11:08.458 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-04T06:11:08.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:11:08.505 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-04-04T06:11:08.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.768+0000 7facd4400640 1 Processor -- start 2024-04-04T06:11:08.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.768+0000 7facd4400640 1 -- start start 2024-04-04T06:11:08.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.770+0000 7facd4400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7faccc15e620 0x7faccc0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.770+0000 7facd4400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7faccc0d1ac0 0x7faccc0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.770+0000 7facd4400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc0b4360 0x7faccc1621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.770+0000 7facd4400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7faccc15fbc0 con 0x7faccc0d1ac0 2024-04-04T06:11:08.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.770+0000 7facd4400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7faccc0b4770 con 0x7faccc15e620 2024-04-04T06:11:08.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.770+0000 7facd4400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7faccc0c7610 con 0x7faccc0b4360 2024-04-04T06:11:08.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.770+0000 7facd1000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7faccc0d1ac0 0x7faccc0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:08.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.770+0000 7facd1000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7faccc0d1ac0 0x7faccc0b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44764/0 (socket says 172.21.15.159:44764) 2024-04-04T06:11:08.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.770+0000 7facd1000640 1 -- 172.21.15.159:0/2280204610 learned_addr learned my addr 172.21.15.159:0/2280204610 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:08.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.770+0000 7facd1a00640 1 --2- 172.21.15.159:0/2280204610 >> v2:172.21.15.178:3300/0 conn(0x7faccc15e620 0x7faccc0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:08.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.770+0000 7facd2400640 1 --2- 172.21.15.159:0/2280204610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc0b4360 0x7faccc1621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:08.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.771+0000 7facd1000640 1 -- 172.21.15.159:0/2280204610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc0b4360 msgr2=0x7faccc1621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:08.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.771+0000 7facd1000640 1 --2- 172.21.15.159:0/2280204610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc0b4360 0x7faccc1621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.771+0000 7facd1000640 1 -- 172.21.15.159:0/2280204610 >> v2:172.21.15.178:3300/0 conn(0x7faccc15e620 msgr2=0x7faccc0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:08.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.771+0000 7facd1000640 1 --2- 172.21.15.159:0/2280204610 >> v2:172.21.15.178:3300/0 conn(0x7faccc15e620 0x7faccc0d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.771+0000 7facd1000640 1 -- 172.21.15.159:0/2280204610 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faccc15ee00 con 0x7faccc0d1ac0 2024-04-04T06:11:08.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.771+0000 7facd1a00640 1 --2- 172.21.15.159:0/2280204610 >> v2:172.21.15.178:3300/0 conn(0x7faccc15e620 0x7faccc0d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:11:08.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.771+0000 7facd2400640 1 --2- 172.21.15.159:0/2280204610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc0b4360 0x7faccc1621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:11:08.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.772+0000 7facd1000640 1 --2- 172.21.15.159:0/2280204610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faccc0d1ac0 0x7faccc0b3e20 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7faccc06f2c0 tx=0x7facb802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f6abdc1340a05e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:08.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.772+0000 7faccbe00640 1 -- 172.21.15.159:0/2280204610 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb80317d0 con 0x7faccc0d1ac0 2024-04-04T06:11:08.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.772+0000 7faccbe00640 1 -- 172.21.15.159:0/2280204610 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facb8037070 con 0x7faccc0d1ac0 2024-04-04T06:11:08.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.772+0000 7faccbe00640 1 -- 172.21.15.159:0/2280204610 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8031b90 con 0x7faccc0d1ac0 2024-04-04T06:11:08.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.773+0000 7facd4400640 1 -- 172.21.15.159:0/2280204610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faccc0d1ac0 msgr2=0x7faccc0b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:08.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.773+0000 7facd4400640 1 --2- 172.21.15.159:0/2280204610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faccc0d1ac0 0x7faccc0b3e20 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7faccc06f2c0 tx=0x7facb802b870 comp rx=0 tx=0).stop 2024-04-04T06:11:08.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.773+0000 7facd4400640 1 -- 172.21.15.159:0/2280204610 shutdown_connections 2024-04-04T06:11:08.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.773+0000 7facd4400640 1 --2- 172.21.15.159:0/2280204610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc0b4360 0x7faccc1621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.773+0000 7facd4400640 1 --2- 172.21.15.159:0/2280204610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faccc0d1ac0 0x7faccc0b3e20 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.773+0000 7facd4400640 1 --2- 172.21.15.159:0/2280204610 >> v2:172.21.15.178:3300/0 conn(0x7faccc15e620 0x7faccc0d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.773+0000 7facd4400640 1 -- 172.21.15.159:0/2280204610 >> 172.21.15.159:0/2280204610 conn(0x7faccc0d2240 msgr2=0x7faccc0d2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:08.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.773+0000 7facd4400640 1 -- 172.21.15.159:0/2280204610 shutdown_connections 2024-04-04T06:11:08.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.773+0000 7facd4400640 1 -- 172.21.15.159:0/2280204610 wait complete. 2024-04-04T06:11:08.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.775+0000 7facd4400640 1 Processor -- start 2024-04-04T06:11:08.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.775+0000 7facd4400640 1 -- start start 2024-04-04T06:11:08.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.776+0000 7facd4400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc0b4360 0x7faccc146d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.776+0000 7facd4400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faccc0d1ac0 0x7faccc0b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.776+0000 7facd4400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faccc15e620 0x7faccc155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.776+0000 7facd4400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faccc15f6a0 con 0x7faccc15e620 2024-04-04T06:11:08.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.776+0000 7facd4400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faccc15f3c0 con 0x7faccc0d1ac0 2024-04-04T06:11:08.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.776+0000 7facd4400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7faccc0c72d0 con 0x7faccc0b4360 2024-04-04T06:11:08.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.776+0000 7facd2400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faccc15e620 0x7faccc155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:08.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.776+0000 7facd2400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faccc15e620 0x7faccc155f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44768/0 (socket says 172.21.15.159:44768) 2024-04-04T06:11:08.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.776+0000 7facd2400640 1 -- 172.21.15.159:0/3598259728 learned_addr learned my addr 172.21.15.159:0/3598259728 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:08.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.777+0000 7facd1000640 1 --2- 172.21.15.159:0/3598259728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faccc0d1ac0 0x7faccc0b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:08.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.777+0000 7facd2400640 1 -- 172.21.15.159:0/3598259728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc0b4360 msgr2=0x7faccc146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:08.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.777+0000 7facd2400640 1 --2- 172.21.15.159:0/3598259728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc0b4360 0x7faccc146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.777+0000 7facd2400640 1 -- 172.21.15.159:0/3598259728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faccc0d1ac0 msgr2=0x7faccc0b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:08.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.777+0000 7facd2400640 1 --2- 172.21.15.159:0/3598259728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faccc0d1ac0 0x7faccc0b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.777+0000 7facd2400640 1 -- 172.21.15.159:0/3598259728 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faccc147280 con 0x7faccc15e620 2024-04-04T06:11:08.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.777+0000 7facd1000640 1 --2- 172.21.15.159:0/3598259728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faccc0d1ac0 0x7faccc0b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:11:08.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.777+0000 7facd2400640 1 --2- 172.21.15.159:0/3598259728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faccc15e620 0x7faccc155f80 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7facc400cd40 tx=0x7facc400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:08.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.777+0000 7facc9600640 1 -- 172.21.15.159:0/3598259728 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc400f680 con 0x7faccc15e620 2024-04-04T06:11:08.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.777+0000 7facc9600640 1 -- 172.21.15.159:0/3598259728 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc400f840 con 0x7faccc15e620 2024-04-04T06:11:08.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.778+0000 7facd4400640 1 -- 172.21.15.159:0/3598259728 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faccc15bcd0 con 0x7faccc15e620 2024-04-04T06:11:08.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.778+0000 7facc9600640 1 -- 172.21.15.159:0/3598259728 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc400fc30 con 0x7faccc15e620 2024-04-04T06:11:08.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.778+0000 7facd4400640 1 -- 172.21.15.159:0/3598259728 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faccc0b6810 con 0x7faccc15e620 2024-04-04T06:11:08.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.780+0000 7facc9600640 1 -- 172.21.15.159:0/3598259728 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7facc40213f0 con 0x7faccc15e620 2024-04-04T06:11:08.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.781+0000 7facc9600640 1 --2- 172.21.15.159:0/3598259728 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7facac06c8f0 0x7facac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.781+0000 7facc9600640 1 -- 172.21.15.159:0/3598259728 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7facc408c9b0 con 0x7faccc15e620 2024-04-04T06:11:08.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.781+0000 7facd1a00640 1 --2- 172.21.15.159:0/3598259728 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7facac06c8f0 0x7facac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:08.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.782+0000 7facd71de740 1 Processor -- start 2024-04-04T06:11:08.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.782+0000 7facd71de740 1 -- start start 2024-04-04T06:11:08.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.782+0000 7facd1a00640 1 --2- 172.21.15.159:0/3598259728 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7facac06c8f0 0x7facac06edb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7facc0006fd0 tx=0x7facc0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:08.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.783+0000 7facd71de740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc75d75da0 0x55bc75d48d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.783+0000 7facd71de740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75d76180 0x55bc75d78b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.783+0000 7facd71de740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc75d49280 0x55bc75d79070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.783+0000 7facd71de740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bc75cb28f0 con 0x55bc75d76180 2024-04-04T06:11:08.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.783+0000 7facd71de740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bc75c5a500 con 0x55bc75d49280 2024-04-04T06:11:08.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.783+0000 7facd71de740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bc75cae8a0 con 0x55bc75d75da0 2024-04-04T06:11:08.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.783+0000 7facd1000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75d76180 0x55bc75d78b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:08.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.783+0000 7facd1000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75d76180 0x55bc75d78b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44770/0 (socket says 172.21.15.159:44770) 2024-04-04T06:11:08.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.783+0000 7facd1000640 1 -- 172.21.15.159:0/107190345 learned_addr learned my addr 172.21.15.159:0/107190345 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:08.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.783+0000 7facd2400640 1 --2- 172.21.15.159:0/107190345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc75d49280 0x55bc75d79070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:08.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.783+0000 7facd1a00640 1 --2- 172.21.15.159:0/107190345 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc75d75da0 0x55bc75d48d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:08.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.783+0000 7facd1000640 1 -- 172.21.15.159:0/107190345 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc75d75da0 msgr2=0x55bc75d48d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:08.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.783+0000 7facd1000640 1 --2- 172.21.15.159:0/107190345 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc75d75da0 0x55bc75d48d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.783+0000 7facd1000640 1 -- 172.21.15.159:0/107190345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc75d49280 msgr2=0x55bc75d79070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:08.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.783+0000 7facd1000640 1 --2- 172.21.15.159:0/107190345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc75d49280 0x55bc75d79070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.783+0000 7facd1000640 1 -- 172.21.15.159:0/107190345 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc75a92a20 con 0x55bc75d76180 2024-04-04T06:11:08.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.783+0000 7facd2400640 1 --2- 172.21.15.159:0/107190345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc75d49280 0x55bc75d79070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:11:08.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.784+0000 7facd1000640 1 --2- 172.21.15.159:0/107190345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75d76180 0x55bc75d78b30 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7facb8009960 tx=0x7facb8030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1917e186794f5fc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:08.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.784+0000 7facbe000640 1 -- 172.21.15.159:0/107190345 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8031a80 con 0x55bc75d76180 2024-04-04T06:11:08.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.784+0000 7facbe000640 1 -- 172.21.15.159:0/107190345 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facb8037040 con 0x55bc75d76180 2024-04-04T06:11:08.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.784+0000 7facbe000640 1 -- 172.21.15.159:0/107190345 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8031e20 con 0x55bc75d76180 2024-04-04T06:11:08.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.785+0000 7facd71de740 1 -- 172.21.15.159:0/107190345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75d76180 msgr2=0x55bc75d78b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:08.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.785+0000 7facd71de740 1 --2- 172.21.15.159:0/107190345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75d76180 0x55bc75d78b30 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7facb8009960 tx=0x7facb8030000 comp rx=0 tx=0).stop 2024-04-04T06:11:08.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.785+0000 7facd71de740 1 -- 172.21.15.159:0/107190345 shutdown_connections 2024-04-04T06:11:08.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.785+0000 7facd71de740 1 --2- 172.21.15.159:0/107190345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc75d49280 0x55bc75d79070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.785+0000 7facd71de740 1 --2- 172.21.15.159:0/107190345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75d76180 0x55bc75d78b30 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.785+0000 7facd71de740 1 --2- 172.21.15.159:0/107190345 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc75d75da0 0x55bc75d48d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.785+0000 7facd71de740 1 -- 172.21.15.159:0/107190345 >> 172.21.15.159:0/107190345 conn(0x55bc75d4afb0 msgr2=0x55bc75d93b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:08.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.785+0000 7facd71de740 1 -- 172.21.15.159:0/107190345 shutdown_connections 2024-04-04T06:11:08.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.785+0000 7facd71de740 1 -- 172.21.15.159:0/107190345 wait complete. 2024-04-04T06:11:08.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.786+0000 7facd71de740 1 -- start start 2024-04-04T06:11:08.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.786+0000 7facd71de740 1 Processor -- start 2024-04-04T06:11:08.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.787+0000 7facd71de740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75d75da0 0x55bc75d7df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.787+0000 7facd71de740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc75d76180 0x55bc75d7e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.787+0000 7facd71de740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc75da4cf0 0x55bc75d7f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.787+0000 7facd71de740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bc75d4d650 con 0x55bc75d75da0 2024-04-04T06:11:08.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.787+0000 7facd71de740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bc75c6dad0 con 0x55bc75da4cf0 2024-04-04T06:11:08.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.787+0000 7facd71de740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bc75d4fe90 con 0x55bc75d76180 2024-04-04T06:11:08.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.787+0000 7facd71de740 10 client.0 _ll_register_callbacks cb 0x55bc75c5b970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:11:08.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.787+0000 7facd1a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75d75da0 0x55bc75d7df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:08.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.787+0000 7facd1a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75d75da0 0x55bc75d7df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44774/0 (socket says 172.21.15.159:44774) 2024-04-04T06:11:08.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.787+0000 7facd1a00640 1 -- 172.21.15.159:0/3554589994 learned_addr learned my addr 172.21.15.159:0/3554589994 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:08.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.788+0000 7facd1a00640 1 -- 172.21.15.159:0/3554589994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc75d76180 msgr2=0x55bc75d7e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:08.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.788+0000 7facd1a00640 1 --2- 172.21.15.159:0/3554589994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc75d76180 0x55bc75d7e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.788+0000 7facd2400640 1 --2- 172.21.15.159:0/3554589994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc75da4cf0 0x55bc75d7f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:08.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.788+0000 7facd1a00640 1 -- 172.21.15.159:0/3554589994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc75da4cf0 msgr2=0x55bc75d7f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:08.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.788+0000 7facd1a00640 1 --2- 172.21.15.159:0/3554589994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc75da4cf0 0x55bc75d7f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.788+0000 7facd1a00640 1 -- 172.21.15.159:0/3554589994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc75986310 con 0x55bc75d75da0 2024-04-04T06:11:08.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.788+0000 7facd1a00640 1 --2- 172.21.15.159:0/3554589994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75d75da0 0x55bc75d7df70 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7facc000ceb0 tx=0x7facc0006bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f844630342733a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:08.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.788+0000 7facb2a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:11:08.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.788+0000 7facb2a00640 1 -- 172.21.15.159:0/3554589994 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc00117e0 con 0x55bc75d75da0 2024-04-04T06:11:08.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.788+0000 7facb2a00640 1 -- 172.21.15.159:0/3554589994 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7facc0011980 con 0x55bc75d75da0 2024-04-04T06:11:08.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.788+0000 7facd71de740 1 -- 172.21.15.159:0/3554589994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facb8037040 con 0x55bc75d75da0 2024-04-04T06:11:08.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.789+0000 7facb2a00640 1 -- 172.21.15.159:0/3554589994 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7facc0013e30 con 0x55bc75d75da0 2024-04-04T06:11:08.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.789+0000 7facd71de740 10 client.6930 fetch_fsmap learned FSMap version 51 2024-04-04T06:11:08.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.789+0000 7facd71de740 1 -- 172.21.15.159:0/3554589994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55bc75af31b0 con 0x55bc75d75da0 2024-04-04T06:11:08.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.789+0000 7facb2a00640 1 -- 172.21.15.159:0/3554589994 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facb8037040 con 0x55bc75d75da0 2024-04-04T06:11:08.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.789+0000 7facb2a00640 1 -- 172.21.15.159:0/3554589994 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7facc002e020 con 0x55bc75d75da0 2024-04-04T06:11:08.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.789+0000 7facd71de740 10 client.6930 fetch_fsmap finished waiting for FSMap version 51 2024-04-04T06:11:08.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.789+0000 7facd71de740 10 client.6930 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-04T06:11:08.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.789+0000 7facd71de740 20 client.6930 populate_metadata read hostname 'smithi159' 2024-04-04T06:11:08.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.789+0000 7facd71de740 1 --2- 172.21.15.159:0/3554589994 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x55bc75d7abc0 0x55bc75d7b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.789+0000 7facd71de740 4 client.6930 mds_command: new command op to 6828 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:11:08.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.789+0000 7facd71de740 1 -- 172.21.15.159:0/3554589994 --> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc75cbfe30 con 0x55bc75d7abc0 2024-04-04T06:11:08.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.789+0000 7facd1000640 1 --2- 172.21.15.159:0/3554589994 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x55bc75d7abc0 0x55bc75d7b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:08.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.790+0000 7facd1000640 1 --2- 172.21.15.159:0/3554589994 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x55bc75d7abc0 0x55bc75d7b000 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b3bca99b7f61e93 server_cookie=e537b6a2c493afba in_seq=0 out_seq=0 2024-04-04T06:11:08.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.790+0000 7facb2a00640 10 client.6930 ms_handle_connect on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:08.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.791+0000 7facb2a00640 1 -- 172.21.15.159:0/3554589994 <== mds.0 v2:172.21.15.159:6834/2467594024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc75cbfe30 con 0x55bc75d7abc0 2024-04-04T06:11:08.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.791+0000 7facb2a00640 10 client.6930 handle_command_reply: tid=0 2024-04-04T06:11:08.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.791+0000 7facd71de740 1 client.6930 shutdown 2024-04-04T06:11:08.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.791+0000 7facd71de740 1 -- 172.21.15.159:0/3554589994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75d75da0 msgr2=0x55bc75d7df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:08.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.791+0000 7facd71de740 1 --2- 172.21.15.159:0/3554589994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75d75da0 0x55bc75d7df70 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7facc000ceb0 tx=0x7facc0006bd0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.791+0000 7facd71de740 1 -- 172.21.15.159:0/3554589994 shutdown_connections 2024-04-04T06:11:08.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.791+0000 7facd71de740 1 --2- 172.21.15.159:0/3554589994 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x55bc75d7abc0 0x55bc75d7b000 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.791+0000 7facd71de740 1 --2- 172.21.15.159:0/3554589994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc75da4cf0 0x55bc75d7f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.791+0000 7facd71de740 1 --2- 172.21.15.159:0/3554589994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc75d76180 0x55bc75d7e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.791+0000 7facd71de740 1 --2- 172.21.15.159:0/3554589994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75d75da0 0x55bc75d7df70 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.791+0000 7facb2a00640 0 client.6930 ms_handle_reset on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:08.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.791+0000 7facd71de740 1 -- 172.21.15.159:0/3554589994 >> 172.21.15.159:0/3554589994 conn(0x55bc75d49280 msgr2=0x55bc75d4cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:08.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.791+0000 7facd71de740 1 -- 172.21.15.159:0/3554589994 shutdown_connections 2024-04-04T06:11:08.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.791+0000 7facd71de740 1 -- 172.21.15.159:0/3554589994 wait complete. 2024-04-04T06:11:08.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.791+0000 7facd71de740 20 client.6930 trim_cache size 0 max 16384 2024-04-04T06:11:08.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.828+0000 7facd71de740 1 Processor -- start 2024-04-04T06:11:08.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.828+0000 7facd71de740 1 -- start start 2024-04-04T06:11:08.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.829+0000 7facd71de740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75d75da0 0x55bc75d78820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.829+0000 7facd71de740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc75d76180 0x55bc75d736a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.829+0000 7facd71de740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc75da4cf0 0x55bc75d73be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.829+0000 7facd71de740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faccc200b10 con 0x55bc75d75da0 2024-04-04T06:11:08.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.829+0000 7facd71de740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1ffb10 con 0x55bc75d76180 2024-04-04T06:11:08.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.829+0000 7facd71de740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7faccc1fef60 con 0x55bc75da4cf0 2024-04-04T06:11:08.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.829+0000 7facd1a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75d75da0 0x55bc75d78820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:08.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.830+0000 7facd1a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75d75da0 0x55bc75d78820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44782/0 (socket says 172.21.15.159:44782) 2024-04-04T06:11:08.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.830+0000 7facd1a00640 1 -- 172.21.15.159:0/3051134600 learned_addr learned my addr 172.21.15.159:0/3051134600 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:08.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.830+0000 7facd1a00640 1 -- 172.21.15.159:0/3051134600 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc75da4cf0 msgr2=0x55bc75d73be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:08.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.830+0000 7facd1a00640 1 --2- 172.21.15.159:0/3051134600 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc75da4cf0 0x55bc75d73be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.830+0000 7facd1000640 1 --2- 172.21.15.159:0/3051134600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc75d76180 0x55bc75d736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:08.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.830+0000 7facd1a00640 1 -- 172.21.15.159:0/3051134600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc75d76180 msgr2=0x55bc75d736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:08.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.830+0000 7facd1a00640 1 --2- 172.21.15.159:0/3051134600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc75d76180 0x55bc75d736a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.830+0000 7facd1a00640 1 -- 172.21.15.159:0/3051134600 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc7595f490 con 0x55bc75d75da0 2024-04-04T06:11:08.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.830+0000 7facd1a00640 1 --2- 172.21.15.159:0/3051134600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75d75da0 0x55bc75d78820 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7facc000c900 tx=0x7facc001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76554a3bd9a5eaf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:08.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.831+0000 7facbd600640 1 -- 172.21.15.159:0/3051134600 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0033070 con 0x55bc75d75da0 2024-04-04T06:11:08.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.831+0000 7facbd600640 1 -- 172.21.15.159:0/3051134600 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc0023070 con 0x55bc75d75da0 2024-04-04T06:11:08.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.831+0000 7facbd600640 1 -- 172.21.15.159:0/3051134600 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0011c00 con 0x55bc75d75da0 2024-04-04T06:11:08.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.831+0000 7facd71de740 1 -- 172.21.15.159:0/3051134600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75d75da0 msgr2=0x55bc75d78820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:08.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.831+0000 7facd71de740 1 --2- 172.21.15.159:0/3051134600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75d75da0 0x55bc75d78820 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7facc000c900 tx=0x7facc001c000 comp rx=0 tx=0).stop 2024-04-04T06:11:08.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.831+0000 7facd71de740 1 -- 172.21.15.159:0/3051134600 shutdown_connections 2024-04-04T06:11:08.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.831+0000 7facd71de740 1 --2- 172.21.15.159:0/3051134600 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc75da4cf0 0x55bc75d73be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.831+0000 7facd71de740 1 --2- 172.21.15.159:0/3051134600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc75d76180 0x55bc75d736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.831+0000 7facd71de740 1 --2- 172.21.15.159:0/3051134600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75d75da0 0x55bc75d78820 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.831+0000 7facd71de740 1 -- 172.21.15.159:0/3051134600 >> 172.21.15.159:0/3051134600 conn(0x55bc75d49280 msgr2=0x55bc75d7dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:08.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.832+0000 7facd71de740 1 -- 172.21.15.159:0/3051134600 shutdown_connections 2024-04-04T06:11:08.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.832+0000 7facd71de740 1 -- 172.21.15.159:0/3051134600 wait complete. 2024-04-04T06:11:08.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.832+0000 7facd71de740 1 -- start start 2024-04-04T06:11:08.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.832+0000 7facd71de740 1 Processor -- start 2024-04-04T06:11:08.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.833+0000 7facd71de740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc75d75da0 0x55bc75d95900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.833+0000 7facd71de740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc75d76180 0x55bc75d4d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.833+0000 7facd71de740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75da4cf0 0x55bc75d4d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.833+0000 7facd71de740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bc75d802a0 con 0x55bc75da4cf0 2024-04-04T06:11:08.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.833+0000 7facd71de740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1ff4f0 con 0x55bc75d76180 2024-04-04T06:11:08.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.833+0000 7facd71de740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bc75d7bb90 con 0x55bc75d75da0 2024-04-04T06:11:08.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.833+0000 7facd71de740 10 client.0 _ll_register_callbacks cb 0x7facc000ae00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:11:08.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.834+0000 7facd2400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75da4cf0 0x55bc75d4d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:08.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.834+0000 7facd2400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75da4cf0 0x55bc75d4d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44788/0 (socket says 172.21.15.159:44788) 2024-04-04T06:11:08.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.834+0000 7facd2400640 1 -- 172.21.15.159:0/3915239384 learned_addr learned my addr 172.21.15.159:0/3915239384 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:08.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.834+0000 7facd2400640 1 -- 172.21.15.159:0/3915239384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc75d75da0 msgr2=0x55bc75d95900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:08.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.834+0000 7facd2400640 1 --2- 172.21.15.159:0/3915239384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc75d75da0 0x55bc75d95900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.834+0000 7facd2400640 1 -- 172.21.15.159:0/3915239384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc75d76180 msgr2=0x55bc75d4d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:08.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.834+0000 7facd2400640 1 --2- 172.21.15.159:0/3915239384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc75d76180 0x55bc75d4d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.834+0000 7facd2400640 1 -- 172.21.15.159:0/3915239384 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc75d75670 con 0x55bc75da4cf0 2024-04-04T06:11:08.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.834+0000 7facd2400640 1 --2- 172.21.15.159:0/3915239384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75da4cf0 0x55bc75d4d820 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7facc405a390 tx=0x7facc405a810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1e59fa1cd246fe server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:08.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.834+0000 7facb2a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:11:08.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.834+0000 7facb2a00640 1 -- 172.21.15.159:0/3915239384 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4062d30 con 0x55bc75da4cf0 2024-04-04T06:11:08.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.835+0000 7facb2a00640 1 -- 172.21.15.159:0/3915239384 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7facc4062ed0 con 0x55bc75da4cf0 2024-04-04T06:11:08.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.835+0000 7facd71de740 1 -- 172.21.15.159:0/3915239384 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facc0023070 con 0x55bc75da4cf0 2024-04-04T06:11:08.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.835+0000 7facb2a00640 1 -- 172.21.15.159:0/3915239384 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7facc4079050 con 0x55bc75da4cf0 2024-04-04T06:11:08.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.835+0000 7facd71de740 10 client.6936 fetch_fsmap learned FSMap version 51 2024-04-04T06:11:08.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.835+0000 7facd71de740 1 -- 172.21.15.159:0/3915239384 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55bc75d4de20 con 0x55bc75da4cf0 2024-04-04T06:11:08.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.835+0000 7facb2a00640 1 -- 172.21.15.159:0/3915239384 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc0023070 con 0x55bc75da4cf0 2024-04-04T06:11:08.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.835+0000 7facb2a00640 1 -- 172.21.15.159:0/3915239384 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7facc4059ae0 con 0x55bc75da4cf0 2024-04-04T06:11:08.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.835+0000 7facd71de740 10 client.6936 fetch_fsmap finished waiting for FSMap version 51 2024-04-04T06:11:08.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.835+0000 7facd71de740 10 client.6936 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-04T06:11:08.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.835+0000 7facd71de740 20 client.6936 populate_metadata read hostname 'smithi159' 2024-04-04T06:11:08.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.835+0000 7facd71de740 1 --2- 172.21.15.159:0/3915239384 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x55bc75e0db80 0x55bc75d47cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:08.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.835+0000 7facd71de740 4 client.6936 mds_command: new command op to 6828 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-04-04T06:11:08.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.835+0000 7facd71de740 1 -- 172.21.15.159:0/3915239384 --> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) -- 0x55bc75cba180 con 0x55bc75e0db80 2024-04-04T06:11:08.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.835+0000 7facd1a00640 1 --2- 172.21.15.159:0/3915239384 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x55bc75e0db80 0x55bc75d47cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:08.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.836+0000 7facd1a00640 1 --2- 172.21.15.159:0/3915239384 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x55bc75e0db80 0x55bc75d47cf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38453263acca79ed server_cookie=4a237a4a15d68a09 in_seq=0 out_seq=0 2024-04-04T06:11:08.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.836+0000 7facb2a00640 10 client.6936 ms_handle_connect on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:08.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.837+0000 7facb2a00640 1 -- 172.21.15.159:0/3915239384 <== mds.0 v2:172.21.15.159:6834/2467594024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bc75cba180 con 0x55bc75e0db80 2024-04-04T06:11:08.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.837+0000 7facb2a00640 10 client.6936 handle_command_reply: tid=0 2024-04-04T06:11:08.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.837+0000 7facd71de740 1 client.6936 shutdown 2024-04-04T06:11:08.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.837+0000 7facd71de740 1 -- 172.21.15.159:0/3915239384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75da4cf0 msgr2=0x55bc75d4d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:08.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.837+0000 7facd71de740 1 --2- 172.21.15.159:0/3915239384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75da4cf0 0x55bc75d4d820 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7facc405a390 tx=0x7facc405a810 comp rx=0 tx=0).stop 2024-04-04T06:11:08.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.837+0000 7facd71de740 1 -- 172.21.15.159:0/3915239384 shutdown_connections 2024-04-04T06:11:08.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.837+0000 7facd71de740 1 --2- 172.21.15.159:0/3915239384 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x55bc75e0db80 0x55bc75d47cf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.837+0000 7facd71de740 1 --2- 172.21.15.159:0/3915239384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc75da4cf0 0x55bc75d4d820 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.837+0000 7facd71de740 1 --2- 172.21.15.159:0/3915239384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc75d76180 0x55bc75d4d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.837+0000 7facd71de740 1 --2- 172.21.15.159:0/3915239384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc75d75da0 0x55bc75d95900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.837+0000 7facd71de740 1 -- 172.21.15.159:0/3915239384 >> 172.21.15.159:0/3915239384 conn(0x55bc75d49280 msgr2=0x55bc75dad780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:08.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.837+0000 7facb2a00640 0 client.6936 ms_handle_reset on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:08.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.837+0000 7facd71de740 1 -- 172.21.15.159:0/3915239384 shutdown_connections 2024-04-04T06:11:08.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.837+0000 7facd71de740 1 -- 172.21.15.159:0/3915239384 wait complete. 2024-04-04T06:11:08.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.837+0000 7facd71de740 20 client.6936 trim_cache size 0 max 16384 2024-04-04T06:11:08.834 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:11:08.834 INFO:teuthology.orchestra.run.smithi159.stdout: "return_code": 0, 2024-04-04T06:11:08.834 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_tag": "db4466f1-dccf-40ee-9dfb-5f2e8693acf7", 2024-04-04T06:11:08.834 INFO:teuthology.orchestra.run.smithi159.stdout: "mode": "asynchronous" 2024-04-04T06:11:08.834 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:11:08.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.838+0000 7facd4400640 1 -- 172.21.15.159:0/3598259728 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7facac06c8f0 msgr2=0x7facac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:08.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.838+0000 7facd4400640 1 --2- 172.21.15.159:0/3598259728 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7facac06c8f0 0x7facac06edb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7facc0006fd0 tx=0x7facc0008040 comp rx=0 tx=0).stop 2024-04-04T06:11:08.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.838+0000 7facd4400640 1 -- 172.21.15.159:0/3598259728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faccc15e620 msgr2=0x7faccc155f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:08.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.838+0000 7facd4400640 1 --2- 172.21.15.159:0/3598259728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faccc15e620 0x7faccc155f80 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7facc400cd40 tx=0x7facc400a650 comp rx=0 tx=0).stop 2024-04-04T06:11:08.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.838+0000 7facd4400640 1 -- 172.21.15.159:0/3598259728 shutdown_connections 2024-04-04T06:11:08.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.838+0000 7facd4400640 1 --2- 172.21.15.159:0/3598259728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faccc15e620 0x7faccc155f80 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.838+0000 7facd4400640 1 --2- 172.21.15.159:0/3598259728 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7facac06c8f0 0x7facac06edb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.838+0000 7facd4400640 1 --2- 172.21.15.159:0/3598259728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faccc0d1ac0 0x7faccc0b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.838+0000 7facd4400640 1 --2- 172.21.15.159:0/3598259728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faccc0b4360 0x7faccc146d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:08.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.838+0000 7facd4400640 1 -- 172.21.15.159:0/3598259728 >> 172.21.15.159:0/3598259728 conn(0x7faccc0d2240 msgr2=0x7faccc06cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:08.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.838+0000 7facd4400640 1 -- 172.21.15.159:0/3598259728 shutdown_connections 2024-04-04T06:11:08.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:08.838+0000 7facd4400640 1 -- 172.21.15.159:0/3598259728 wait complete. 2024-04-04T06:11:08.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:11:08.848 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-04T06:11:08.991 INFO:tasks.ceph.mon.a.smithi159.stderr:2024-04-04T06:11:08.998+0000 7f1998e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-04T06:11:09.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.056+0000 7f8b83e00640 1 Processor -- start 2024-04-04T06:11:09.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.057+0000 7f8b83e00640 1 -- start start 2024-04-04T06:11:09.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.058+0000 7f8b83e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b7c0d0990 0x7f8b7c0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:09.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.058+0000 7f8b83e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8b7c0af2e0 0x7f8b7c1600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:09.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.058+0000 7f8b83e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b7c0afab0 con 0x7f8b7c153580 2024-04-04T06:11:09.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.058+0000 7f8b83e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8b7c15cc80 con 0x7f8b7c0af2e0 2024-04-04T06:11:09.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.058+0000 7f8b83e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8b7c15da00 con 0x7f8b7c0d0990 2024-04-04T06:11:09.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.059+0000 7f8b81e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8b7c0af2e0 0x7f8b7c1600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:09.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.059+0000 7f8b81e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8b7c0af2e0 0x7f8b7c1600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:43340/0 (socket says 172.21.15.159:43340) 2024-04-04T06:11:09.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.059+0000 7f8b81e00640 1 -- 172.21.15.159:0/1716388927 learned_addr learned my addr 172.21.15.159:0/1716388927 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:09.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.059+0000 7f8b80a00640 1 --2- 172.21.15.159:0/1716388927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b7c0d0990 0x7f8b7c0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:09.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.059+0000 7f8b81e00640 1 -- 172.21.15.159:0/1716388927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b7c0d0990 msgr2=0x7f8b7c0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:09.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.059+0000 7f8b81e00640 1 --2- 172.21.15.159:0/1716388927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b7c0d0990 0x7f8b7c0aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.059+0000 7f8b81e00640 1 -- 172.21.15.159:0/1716388927 >> v1:172.21.15.178:6789/0 conn(0x7f8b7c153580 legacy=0x7f8b7c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:09.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.059+0000 7f8b81e00640 1 -- 172.21.15.159:0/1716388927 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7c15cf20 con 0x7f8b7c0af2e0 2024-04-04T06:11:09.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.059+0000 7f8b80a00640 1 --2- 172.21.15.159:0/1716388927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b7c0d0990 0x7f8b7c0aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:11:09.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.061+0000 7f8b81e00640 1 --2- 172.21.15.159:0/1716388927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b7c0af2e0 0x7f8b7c1600d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009e50 tx=0x7f8b7402bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=682c518cca8cae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:09.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.061+0000 7f8b7be00640 1 -- 172.21.15.159:0/1716388927 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7402ca70 con 0x7f8b7c0af2e0 2024-04-04T06:11:09.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.061+0000 7f8b7be00640 1 -- 172.21.15.159:0/1716388927 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b7402cc10 con 0x7f8b7c0af2e0 2024-04-04T06:11:09.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.061+0000 7f8b7be00640 1 -- 172.21.15.159:0/1716388927 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b740343f0 con 0x7f8b7c0af2e0 2024-04-04T06:11:09.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.062+0000 7f8b83e00640 1 -- 172.21.15.159:0/1716388927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b7c0af2e0 msgr2=0x7f8b7c1600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:09.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.062+0000 7f8b83e00640 1 --2- 172.21.15.159:0/1716388927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b7c0af2e0 0x7f8b7c1600d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009e50 tx=0x7f8b7402bed0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.062+0000 7f8b83e00640 1 -- 172.21.15.159:0/1716388927 shutdown_connections 2024-04-04T06:11:09.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.062+0000 7f8b83e00640 1 --2- 172.21.15.159:0/1716388927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b7c0af2e0 0x7f8b7c1600d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.062+0000 7f8b83e00640 1 --2- 172.21.15.159:0/1716388927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b7c0d0990 0x7f8b7c0aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.062+0000 7f8b83e00640 1 -- 172.21.15.159:0/1716388927 >> 172.21.15.159:0/1716388927 conn(0x7f8b7c06b750 msgr2=0x7f8b7c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:09.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.062+0000 7f8b83e00640 1 -- 172.21.15.159:0/1716388927 shutdown_connections 2024-04-04T06:11:09.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.062+0000 7f8b83e00640 1 -- 172.21.15.159:0/1716388927 wait complete. 2024-04-04T06:11:09.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.064+0000 7f8b83e00640 1 Processor -- start 2024-04-04T06:11:09.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.065+0000 7f8b83e00640 1 -- start start 2024-04-04T06:11:09.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.066+0000 7f8b83e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b7c0af2e0 0x7f8b7c0cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:09.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.066+0000 7f8b83e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b7c0d0990 0x7f8b7c1553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:09.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.066+0000 7f8b83e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b7c153580 0x7f8b7c156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:09.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.066+0000 7f8b83e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7c153960 con 0x7f8b7c153580 2024-04-04T06:11:09.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.066+0000 7f8b83e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7c15d4e0 con 0x7f8b7c0d0990 2024-04-04T06:11:09.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.066+0000 7f8b83e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8b7c0af710 con 0x7f8b7c0af2e0 2024-04-04T06:11:09.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.066+0000 7f8b81e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b7c153580 0x7f8b7c156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:09.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.066+0000 7f8b81e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b7c153580 0x7f8b7c156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44794/0 (socket says 172.21.15.159:44794) 2024-04-04T06:11:09.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.066+0000 7f8b81e00640 1 -- 172.21.15.159:0/3525672395 learned_addr learned my addr 172.21.15.159:0/3525672395 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:09.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.066+0000 7f8b80a00640 1 --2- 172.21.15.159:0/3525672395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b7c0d0990 0x7f8b7c1553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:09.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.066+0000 7f8b81400640 1 --2- 172.21.15.159:0/3525672395 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b7c0af2e0 0x7f8b7c0cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:09.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.067+0000 7f8b81e00640 1 -- 172.21.15.159:0/3525672395 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b7c0af2e0 msgr2=0x7f8b7c0cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:09.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.067+0000 7f8b81e00640 1 --2- 172.21.15.159:0/3525672395 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b7c0af2e0 0x7f8b7c0cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.067+0000 7f8b81e00640 1 -- 172.21.15.159:0/3525672395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b7c0d0990 msgr2=0x7f8b7c1553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:09.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.067+0000 7f8b81e00640 1 --2- 172.21.15.159:0/3525672395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b7c0d0990 0x7f8b7c1553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.067+0000 7f8b81e00640 1 -- 172.21.15.159:0/3525672395 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7c0ce570 con 0x7f8b7c153580 2024-04-04T06:11:09.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.067+0000 7f8b81e00640 1 --2- 172.21.15.159:0/3525672395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b7c153580 0x7f8b7c156510 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009e50 tx=0x7f8b74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:09.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.067+0000 7f8b79600640 1 -- 172.21.15.159:0/3525672395 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b74043020 con 0x7f8b7c153580 2024-04-04T06:11:09.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.067+0000 7f8b79600640 1 -- 172.21.15.159:0/3525672395 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b7403e450 con 0x7f8b7c153580 2024-04-04T06:11:09.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.067+0000 7f8b79600640 1 -- 172.21.15.159:0/3525672395 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b74009510 con 0x7f8b7c153580 2024-04-04T06:11:09.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.067+0000 7f8b83e00640 1 -- 172.21.15.159:0/3525672395 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b7c0cdfb0 con 0x7f8b7c153580 2024-04-04T06:11:09.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.068+0000 7f8b83e00640 1 -- 172.21.15.159:0/3525672395 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b7c155d10 con 0x7f8b7c153580 2024-04-04T06:11:09.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.070+0000 7f8b79600640 1 -- 172.21.15.159:0/3525672395 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b74046020 con 0x7f8b7c153580 2024-04-04T06:11:09.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.071+0000 7f8b79600640 1 --2- 172.21.15.159:0/3525672395 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8b5c06c8f0 0x7f8b5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:09.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.071+0000 7f8b79600640 1 -- 172.21.15.159:0/3525672395 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f8b740ae410 con 0x7f8b7c153580 2024-04-04T06:11:09.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.071+0000 7f8b81400640 1 --2- 172.21.15.159:0/3525672395 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8b5c06c8f0 0x7f8b5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:09.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.071+0000 7f8b86b63740 1 Processor -- start 2024-04-04T06:11:09.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.072+0000 7f8b86b63740 1 -- start start 2024-04-04T06:11:09.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.072+0000 7f8b81400640 1 --2- 172.21.15.159:0/3525672395 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8b5c06c8f0 0x7f8b5c06edb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8b70007fd0 tx=0x7f8b70009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:09.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.072+0000 7f8b86b63740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560abf4d1580 0x560abf4a4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:09.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.072+0000 7f8b86b63740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560abf4d1960 0x560abf4d4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:09.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.073+0000 7f8b86b63740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4a4a60 0x560abf4d4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:09.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.073+0000 7f8b86b63740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560abf40f6d0 con 0x560abf4a4a60 2024-04-04T06:11:09.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.073+0000 7f8b86b63740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560abf3b7770 con 0x560abf4d1580 2024-04-04T06:11:09.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.073+0000 7f8b86b63740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x560abf40b680 con 0x560abf4d1960 2024-04-04T06:11:09.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.073+0000 7f8b81e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4a4a60 0x560abf4d4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:09.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.073+0000 7f8b81e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4a4a60 0x560abf4d4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44804/0 (socket says 172.21.15.159:44804) 2024-04-04T06:11:09.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.073+0000 7f8b81e00640 1 -- 172.21.15.159:0/1591084890 learned_addr learned my addr 172.21.15.159:0/1591084890 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:09.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.073+0000 7f8b81400640 1 --2- 172.21.15.159:0/1591084890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560abf4d1580 0x560abf4a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:09.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.073+0000 7f8b80a00640 1 --2- 172.21.15.159:0/1591084890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560abf4d1960 0x560abf4d4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:09.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.073+0000 7f8b81e00640 1 -- 172.21.15.159:0/1591084890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560abf4d1960 msgr2=0x560abf4d4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:09.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.073+0000 7f8b81e00640 1 --2- 172.21.15.159:0/1591084890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560abf4d1960 0x560abf4d4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.073+0000 7f8b81e00640 1 -- 172.21.15.159:0/1591084890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560abf4d1580 msgr2=0x560abf4a4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:09.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.073+0000 7f8b81e00640 1 --2- 172.21.15.159:0/1591084890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560abf4d1580 0x560abf4a4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.073+0000 7f8b81e00640 1 -- 172.21.15.159:0/1591084890 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560abf1ed7d0 con 0x560abf4a4a60 2024-04-04T06:11:09.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.074+0000 7f8b81e00640 1 --2- 172.21.15.159:0/1591084890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4a4a60 0x560abf4d4850 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f8b7407c060 tx=0x7f8b7403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3790679ff4c5908b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:09.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.074+0000 7f8b6dc00640 1 -- 172.21.15.159:0/1591084890 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7407c0d0 con 0x560abf4a4a60 2024-04-04T06:11:09.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.074+0000 7f8b6dc00640 1 -- 172.21.15.159:0/1591084890 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b74080480 con 0x560abf4a4a60 2024-04-04T06:11:09.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.074+0000 7f8b6dc00640 1 -- 172.21.15.159:0/1591084890 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b74084cb0 con 0x560abf4a4a60 2024-04-04T06:11:09.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.074+0000 7f8b86b63740 1 -- 172.21.15.159:0/1591084890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4a4a60 msgr2=0x560abf4d4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:09.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.075+0000 7f8b86b63740 1 --2- 172.21.15.159:0/1591084890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4a4a60 0x560abf4d4850 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f8b7407c060 tx=0x7f8b7403c000 comp rx=0 tx=0).stop 2024-04-04T06:11:09.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.075+0000 7f8b86b63740 1 -- 172.21.15.159:0/1591084890 shutdown_connections 2024-04-04T06:11:09.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.075+0000 7f8b86b63740 1 --2- 172.21.15.159:0/1591084890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4a4a60 0x560abf4d4850 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.075+0000 7f8b86b63740 1 --2- 172.21.15.159:0/1591084890 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560abf4d1960 0x560abf4d4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.075+0000 7f8b86b63740 1 --2- 172.21.15.159:0/1591084890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560abf4d1580 0x560abf4a4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.075+0000 7f8b86b63740 1 -- 172.21.15.159:0/1591084890 >> 172.21.15.159:0/1591084890 conn(0x560abf4a6790 msgr2=0x560abf4ef310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:09.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.075+0000 7f8b86b63740 1 -- 172.21.15.159:0/1591084890 shutdown_connections 2024-04-04T06:11:09.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.075+0000 7f8b86b63740 1 -- 172.21.15.159:0/1591084890 wait complete. 2024-04-04T06:11:09.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.075+0000 7f8b86b63740 1 -- start start 2024-04-04T06:11:09.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.076+0000 7f8b86b63740 1 Processor -- start 2024-04-04T06:11:09.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.077+0000 7f8b86b63740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560abf4d1580 0x560abf4efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:09.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.077+0000 7f8b86b63740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4d1960 0x560abf4cb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:09.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.077+0000 7f8b86b63740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560abf4cb740 0x560abf4c03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:09.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.077+0000 7f8b86b63740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560abf40f6d0 con 0x560abf4d1960 2024-04-04T06:11:09.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.077+0000 7f8b86b63740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560abf3caaa0 con 0x560abf4cb740 2024-04-04T06:11:09.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.077+0000 7f8b86b63740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x560abf4a8e30 con 0x560abf4d1580 2024-04-04T06:11:09.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.077+0000 7f8b86b63740 10 client.0 _ll_register_callbacks cb 0x560abf3ca490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:11:09.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.077+0000 7f8b80a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4d1960 0x560abf4cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:09.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.077+0000 7f8b80a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4d1960 0x560abf4cb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44820/0 (socket says 172.21.15.159:44820) 2024-04-04T06:11:09.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.077+0000 7f8b80a00640 1 -- 172.21.15.159:0/268928976 learned_addr learned my addr 172.21.15.159:0/268928976 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:09.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.077+0000 7f8b81e00640 1 --2- 172.21.15.159:0/268928976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560abf4cb740 0x560abf4c03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:09.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.077+0000 7f8b81400640 1 --2- 172.21.15.159:0/268928976 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560abf4d1580 0x560abf4efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:09.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.077+0000 7f8b80a00640 1 -- 172.21.15.159:0/268928976 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560abf4d1580 msgr2=0x560abf4efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:09.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.077+0000 7f8b80a00640 1 --2- 172.21.15.159:0/268928976 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560abf4d1580 0x560abf4efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.077+0000 7f8b80a00640 1 -- 172.21.15.159:0/268928976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560abf4cb740 msgr2=0x560abf4c03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:09.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.077+0000 7f8b80a00640 1 --2- 172.21.15.159:0/268928976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560abf4cb740 0x560abf4c03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.078+0000 7f8b80a00640 1 -- 172.21.15.159:0/268928976 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560abf1233f0 con 0x560abf4d1960 2024-04-04T06:11:09.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.078+0000 7f8b81400640 1 --2- 172.21.15.159:0/268928976 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560abf4d1580 0x560abf4efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:11:09.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.078+0000 7f8b80a00640 1 --2- 172.21.15.159:0/268928976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4d1960 0x560abf4cb200 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f8b68006fd0 tx=0x7f8b6800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221d2efccb7b70c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:09.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.078+0000 7f8b62000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:11:09.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.078+0000 7f8b62000640 1 -- 172.21.15.159:0/268928976 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68019020 con 0x560abf4d1960 2024-04-04T06:11:09.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.078+0000 7f8b86b63740 1 -- 172.21.15.159:0/268928976 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b74080480 con 0x560abf4d1960 2024-04-04T06:11:09.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.078+0000 7f8b62000640 1 -- 172.21.15.159:0/268928976 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f8b68004240 con 0x560abf4d1960 2024-04-04T06:11:09.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.078+0000 7f8b62000640 1 -- 172.21.15.159:0/268928976 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f8b68023050 con 0x560abf4d1960 2024-04-04T06:11:09.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.078+0000 7f8b86b63740 10 client.6945 fetch_fsmap learned FSMap version 52 2024-04-04T06:11:09.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.078+0000 7f8b86b63740 1 -- 172.21.15.159:0/268928976 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x560abf0e30f0 con 0x560abf4d1960 2024-04-04T06:11:09.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.079+0000 7f8b62000640 1 -- 172.21.15.159:0/268928976 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b74080480 con 0x560abf4d1960 2024-04-04T06:11:09.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.079+0000 7f8b62000640 1 -- 172.21.15.159:0/268928976 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f8b68026020 con 0x560abf4d1960 2024-04-04T06:11:09.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.079+0000 7f8b86b63740 10 client.6945 fetch_fsmap finished waiting for FSMap version 52 2024-04-04T06:11:09.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.079+0000 7f8b86b63740 10 client.6945 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-04T06:11:09.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.079+0000 7f8b86b63740 20 client.6945 populate_metadata read hostname 'smithi159' 2024-04-04T06:11:09.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.079+0000 7f8b86b63740 1 --2- 172.21.15.159:0/268928976 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x560abf4d3d90 0x560abf4d4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:09.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.079+0000 7f8b86b63740 4 client.6945 mds_command: new command op to 6828 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:11:09.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.079+0000 7f8b86b63740 1 -- 172.21.15.159:0/268928976 --> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560abf41cc10 con 0x560abf4d3d90 2024-04-04T06:11:09.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.079+0000 7f8b81400640 1 --2- 172.21.15.159:0/268928976 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x560abf4d3d90 0x560abf4d4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:09.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.080+0000 7f8b81400640 1 --2- 172.21.15.159:0/268928976 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x560abf4d3d90 0x560abf4d4170 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a65a948ff0375974 server_cookie=9296b8f7003a3e4e in_seq=0 out_seq=0 2024-04-04T06:11:09.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.080+0000 7f8b62000640 10 client.6945 ms_handle_connect on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:09.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.080+0000 7f8b62000640 1 -- 172.21.15.159:0/268928976 <== mds.0 v2:172.21.15.159:6834/2467594024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560abf41cc10 con 0x560abf4d3d90 2024-04-04T06:11:09.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.081+0000 7f8b62000640 10 client.6945 handle_command_reply: tid=0 2024-04-04T06:11:09.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.081+0000 7f8b86b63740 1 client.6945 shutdown 2024-04-04T06:11:09.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.081+0000 7f8b86b63740 1 -- 172.21.15.159:0/268928976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4d1960 msgr2=0x560abf4cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:09.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.081+0000 7f8b86b63740 1 --2- 172.21.15.159:0/268928976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4d1960 0x560abf4cb200 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f8b68006fd0 tx=0x7f8b6800bf90 comp rx=0 tx=0).stop 2024-04-04T06:11:09.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.081+0000 7f8b86b63740 1 -- 172.21.15.159:0/268928976 shutdown_connections 2024-04-04T06:11:09.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.081+0000 7f8b86b63740 1 --2- 172.21.15.159:0/268928976 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x560abf4d3d90 0x560abf4d4170 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.081+0000 7f8b86b63740 1 --2- 172.21.15.159:0/268928976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560abf4cb740 0x560abf4c03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.081+0000 7f8b86b63740 1 --2- 172.21.15.159:0/268928976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4d1960 0x560abf4cb200 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.081+0000 7f8b62000640 0 client.6945 ms_handle_reset on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:09.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.081+0000 7f8b86b63740 1 --2- 172.21.15.159:0/268928976 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560abf4d1580 0x560abf4efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.081+0000 7f8b86b63740 1 -- 172.21.15.159:0/268928976 >> 172.21.15.159:0/268928976 conn(0x560abf4a4a60 msgr2=0x560abf4ab350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:09.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.081+0000 7f8b86b63740 1 -- 172.21.15.159:0/268928976 shutdown_connections 2024-04-04T06:11:09.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.081+0000 7f8b86b63740 1 -- 172.21.15.159:0/268928976 wait complete. 2024-04-04T06:11:09.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.081+0000 7f8b86b63740 20 client.6945 trim_cache size 0 max 16384 2024-04-04T06:11:09.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.100+0000 7f8b86b63740 1 Processor -- start 2024-04-04T06:11:09.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.100+0000 7f8b86b63740 1 -- start start 2024-04-04T06:11:09.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.101+0000 7f8b86b63740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4cb740 0x560abf4bfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:09.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.101+0000 7f8b86b63740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560abf4d1580 0x560abf5164c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:09.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.101+0000 7f8b86b63740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560abf4d1960 0x560abf516a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:09.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.101+0000 7f8b86b63740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7c154920 con 0x560abf4cb740 2024-04-04T06:11:09.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.101+0000 7f8b86b63740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7c073660 con 0x560abf4d1960 2024-04-04T06:11:09.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.101+0000 7f8b86b63740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8b7c0729c0 con 0x560abf4d1580 2024-04-04T06:11:09.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.101+0000 7f8b81400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4cb740 0x560abf4bfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:09.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.102+0000 7f8b81400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4cb740 0x560abf4bfff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44828/0 (socket says 172.21.15.159:44828) 2024-04-04T06:11:09.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.102+0000 7f8b81400640 1 -- 172.21.15.159:0/2248092652 learned_addr learned my addr 172.21.15.159:0/2248092652 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:09.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.102+0000 7f8b81400640 1 -- 172.21.15.159:0/2248092652 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560abf4d1580 msgr2=0x560abf5164c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:09.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.102+0000 7f8b81400640 1 --2- 172.21.15.159:0/2248092652 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560abf4d1580 0x560abf5164c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.102+0000 7f8b81400640 1 -- 172.21.15.159:0/2248092652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560abf4d1960 msgr2=0x560abf516a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:09.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.102+0000 7f8b81400640 1 --2- 172.21.15.159:0/2248092652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560abf4d1960 0x560abf516a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.102+0000 7f8b81400640 1 -- 172.21.15.159:0/2248092652 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560abf4ce3c0 con 0x560abf4cb740 2024-04-04T06:11:09.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.102+0000 7f8b81400640 1 --2- 172.21.15.159:0/2248092652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4cb740 0x560abf4bfff0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x560abf4d4710 tx=0x7f8b70017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692ed4b25e62873c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:09.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.102+0000 7f8b6d200640 1 -- 172.21.15.159:0/2248092652 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7001e040 con 0x560abf4cb740 2024-04-04T06:11:09.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.102+0000 7f8b6d200640 1 -- 172.21.15.159:0/2248092652 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b7002a070 con 0x560abf4cb740 2024-04-04T06:11:09.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.102+0000 7f8b6d200640 1 -- 172.21.15.159:0/2248092652 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70025660 con 0x560abf4cb740 2024-04-04T06:11:09.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.103+0000 7f8b86b63740 1 -- 172.21.15.159:0/2248092652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4cb740 msgr2=0x560abf4bfff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:09.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.103+0000 7f8b86b63740 1 --2- 172.21.15.159:0/2248092652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4cb740 0x560abf4bfff0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x560abf4d4710 tx=0x7f8b70017040 comp rx=0 tx=0).stop 2024-04-04T06:11:09.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.103+0000 7f8b86b63740 1 -- 172.21.15.159:0/2248092652 shutdown_connections 2024-04-04T06:11:09.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.103+0000 7f8b86b63740 1 --2- 172.21.15.159:0/2248092652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560abf4d1960 0x560abf516a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.103+0000 7f8b86b63740 1 --2- 172.21.15.159:0/2248092652 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560abf4d1580 0x560abf5164c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.103+0000 7f8b86b63740 1 --2- 172.21.15.159:0/2248092652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4cb740 0x560abf4bfff0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.103+0000 7f8b86b63740 1 -- 172.21.15.159:0/2248092652 >> 172.21.15.159:0/2248092652 conn(0x560abf4a4a60 msgr2=0x560abf4efa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:09.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.103+0000 7f8b86b63740 1 -- 172.21.15.159:0/2248092652 shutdown_connections 2024-04-04T06:11:09.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.103+0000 7f8b86b63740 1 -- 172.21.15.159:0/2248092652 wait complete. 2024-04-04T06:11:09.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.104+0000 7f8b86b63740 1 -- start start 2024-04-04T06:11:09.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.104+0000 7f8b86b63740 1 Processor -- start 2024-04-04T06:11:09.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.105+0000 7f8b86b63740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4cb740 0x560abf4c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:09.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.105+0000 7f8b86b63740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560abf4d1580 0x560abf4c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:09.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.105+0000 7f8b86b63740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560abf4d1960 0x560abf4d5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:09.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.105+0000 7f8b86b63740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7c073010 con 0x560abf4cb740 2024-04-04T06:11:09.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.105+0000 7f8b86b63740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560abf4ce220 con 0x560abf4d1960 2024-04-04T06:11:09.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.105+0000 7f8b86b63740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x560abf4d6340 con 0x560abf4d1580 2024-04-04T06:11:09.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.105+0000 7f8b86b63740 10 client.0 _ll_register_callbacks cb 0x7f8b6800bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:11:09.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.105+0000 7f8b81400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4cb740 0x560abf4c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:09.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.105+0000 7f8b81400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4cb740 0x560abf4c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44840/0 (socket says 172.21.15.159:44840) 2024-04-04T06:11:09.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.105+0000 7f8b81400640 1 -- 172.21.15.159:0/457767944 learned_addr learned my addr 172.21.15.159:0/457767944 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:09.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.106+0000 7f8b80a00640 1 --2- 172.21.15.159:0/457767944 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560abf4d1580 0x560abf4c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:09.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.106+0000 7f8b81e00640 1 --2- 172.21.15.159:0/457767944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560abf4d1960 0x560abf4d5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:09.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.106+0000 7f8b81400640 1 -- 172.21.15.159:0/457767944 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560abf4d1580 msgr2=0x560abf4c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:09.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.106+0000 7f8b81400640 1 --2- 172.21.15.159:0/457767944 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560abf4d1580 0x560abf4c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.106+0000 7f8b81400640 1 -- 172.21.15.159:0/457767944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560abf4d1960 msgr2=0x560abf4d5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:09.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.106+0000 7f8b81400640 1 --2- 172.21.15.159:0/457767944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560abf4d1960 0x560abf4d5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.106+0000 7f8b81400640 1 -- 172.21.15.159:0/457767944 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560abf4ca3f0 con 0x560abf4cb740 2024-04-04T06:11:09.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.106+0000 7f8b81400640 1 --2- 172.21.15.159:0/457767944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4cb740 0x560abf4c9670 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009010 tx=0x7f8b70000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecfe2fe293fe8434 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:09.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.106+0000 7f8b62000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:11:09.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.106+0000 7f8b62000640 1 -- 172.21.15.159:0/457767944 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70032020 con 0x560abf4cb740 2024-04-04T06:11:09.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.106+0000 7f8b62000640 1 -- 172.21.15.159:0/457767944 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f8b7000e450 con 0x560abf4cb740 2024-04-04T06:11:09.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.106+0000 7f8b86b63740 1 -- 172.21.15.159:0/457767944 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b7002a070 con 0x560abf4cb740 2024-04-04T06:11:09.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.107+0000 7f8b62000640 1 -- 172.21.15.159:0/457767944 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f8b7003c050 con 0x560abf4cb740 2024-04-04T06:11:09.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.107+0000 7f8b86b63740 10 client.6951 fetch_fsmap learned FSMap version 52 2024-04-04T06:11:09.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.107+0000 7f8b86b63740 1 -- 172.21.15.159:0/457767944 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x560abf4d60e0 con 0x560abf4cb740 2024-04-04T06:11:09.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.107+0000 7f8b62000640 1 -- 172.21.15.159:0/457767944 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b7002a070 con 0x560abf4cb740 2024-04-04T06:11:09.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.107+0000 7f8b62000640 1 -- 172.21.15.159:0/457767944 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f8b7003f020 con 0x560abf4cb740 2024-04-04T06:11:09.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.107+0000 7f8b86b63740 10 client.6951 fetch_fsmap finished waiting for FSMap version 52 2024-04-04T06:11:09.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.107+0000 7f8b86b63740 10 client.6951 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-04T06:11:09.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.107+0000 7f8b86b63740 20 client.6951 populate_metadata read hostname 'smithi159' 2024-04-04T06:11:09.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.107+0000 7f8b86b63740 1 --2- 172.21.15.159:0/457767944 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x560abf4d32e0 0x560abf4d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:09.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.107+0000 7f8b86b63740 4 client.6951 mds_command: new command op to 6828 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-04T06:11:09.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.107+0000 7f8b86b63740 1 -- 172.21.15.159:0/457767944 --> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560abf416f60 con 0x560abf4d32e0 2024-04-04T06:11:09.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.107+0000 7f8b80a00640 1 --2- 172.21.15.159:0/457767944 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x560abf4d32e0 0x560abf4d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:09.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.108+0000 7f8b80a00640 1 --2- 172.21.15.159:0/457767944 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x560abf4d32e0 0x560abf4d5850 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=874c7de5c08b2b2a server_cookie=1fdffe81d5fc79ed in_seq=0 out_seq=0 2024-04-04T06:11:09.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.108+0000 7f8b62000640 10 client.6951 ms_handle_connect on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:09.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.108+0000 7f8b62000640 1 -- 172.21.15.159:0/457767944 <== mds.0 v2:172.21.15.159:6834/2467594024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560abf416f60 con 0x560abf4d32e0 2024-04-04T06:11:09.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.108+0000 7f8b62000640 10 client.6951 handle_command_reply: tid=0 2024-04-04T06:11:09.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.108+0000 7f8b86b63740 1 client.6951 shutdown 2024-04-04T06:11:09.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.108+0000 7f8b86b63740 1 -- 172.21.15.159:0/457767944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4cb740 msgr2=0x560abf4c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:09.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.108+0000 7f8b86b63740 1 --2- 172.21.15.159:0/457767944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4cb740 0x560abf4c9670 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009010 tx=0x7f8b70000bc0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.109+0000 7f8b86b63740 1 -- 172.21.15.159:0/457767944 shutdown_connections 2024-04-04T06:11:09.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.109+0000 7f8b86b63740 1 --2- 172.21.15.159:0/457767944 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x560abf4d32e0 0x560abf4d5850 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.109+0000 7f8b86b63740 1 --2- 172.21.15.159:0/457767944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560abf4d1960 0x560abf4d5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.109+0000 7f8b86b63740 1 --2- 172.21.15.159:0/457767944 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560abf4d1580 0x560abf4c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.109+0000 7f8b86b63740 1 --2- 172.21.15.159:0/457767944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560abf4cb740 0x560abf4c9670 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.109+0000 7f8b62000640 0 client.6951 ms_handle_reset on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:09.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.109+0000 7f8b86b63740 1 -- 172.21.15.159:0/457767944 >> 172.21.15.159:0/457767944 conn(0x560abf4a4a60 msgr2=0x560abf4a82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:09.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.109+0000 7f8b86b63740 1 -- 172.21.15.159:0/457767944 shutdown_connections 2024-04-04T06:11:09.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.109+0000 7f8b86b63740 1 -- 172.21.15.159:0/457767944 wait complete. 2024-04-04T06:11:09.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.109+0000 7f8b86b63740 20 client.6951 trim_cache size 0 max 16384 2024-04-04T06:11:09.106 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:11:09.106 INFO:teuthology.orchestra.run.smithi159.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-04T06:11:09.106 INFO:teuthology.orchestra.run.smithi159.stdout: "scrubs": { 2024-04-04T06:11:09.106 INFO:teuthology.orchestra.run.smithi159.stdout: "db4466f1-dccf-40ee-9dfb-5f2e8693acf7": { 2024-04-04T06:11:09.106 INFO:teuthology.orchestra.run.smithi159.stdout: "path": "/subdir", 2024-04-04T06:11:09.106 INFO:teuthology.orchestra.run.smithi159.stdout: "tag": "db4466f1-dccf-40ee-9dfb-5f2e8693acf7", 2024-04-04T06:11:09.106 INFO:teuthology.orchestra.run.smithi159.stdout: "options": "recursive" 2024-04-04T06:11:09.106 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:11:09.106 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:11:09.106 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:11:09.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.109+0000 7f8b83e00640 1 -- 172.21.15.159:0/3525672395 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8b5c06c8f0 msgr2=0x7f8b5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:09.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.109+0000 7f8b83e00640 1 --2- 172.21.15.159:0/3525672395 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8b5c06c8f0 0x7f8b5c06edb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8b70007fd0 tx=0x7f8b70009040 comp rx=0 tx=0).stop 2024-04-04T06:11:09.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.109+0000 7f8b83e00640 1 -- 172.21.15.159:0/3525672395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b7c153580 msgr2=0x7f8b7c156510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:09.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.109+0000 7f8b83e00640 1 --2- 172.21.15.159:0/3525672395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b7c153580 0x7f8b7c156510 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009e50 tx=0x7f8b74004a40 comp rx=0 tx=0).stop 2024-04-04T06:11:09.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.110+0000 7f8b83e00640 1 -- 172.21.15.159:0/3525672395 shutdown_connections 2024-04-04T06:11:09.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.110+0000 7f8b83e00640 1 --2- 172.21.15.159:0/3525672395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b7c153580 0x7f8b7c156510 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.110+0000 7f8b83e00640 1 --2- 172.21.15.159:0/3525672395 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8b5c06c8f0 0x7f8b5c06edb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.110+0000 7f8b83e00640 1 --2- 172.21.15.159:0/3525672395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b7c0d0990 0x7f8b7c1553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.110+0000 7f8b83e00640 1 --2- 172.21.15.159:0/3525672395 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8b7c0af2e0 0x7f8b7c0cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:09.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.110+0000 7f8b83e00640 1 -- 172.21.15.159:0/3525672395 >> 172.21.15.159:0/3525672395 conn(0x7f8b7c06b750 msgr2=0x7f8b7c06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:09.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.110+0000 7f8b83e00640 1 -- 172.21.15.159:0/3525672395 shutdown_connections 2024-04-04T06:11:09.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:09.110+0000 7f8b83e00640 1 -- 172.21.15.159:0/3525672395 wait complete. 2024-04-04T06:11:09.121 INFO:tasks.cephfs.filesystem:scrub status for tag:db4466f1-dccf-40ee-9dfb-5f2e8693acf7 - {'path': '/subdir', 'tag': 'db4466f1-dccf-40ee-9dfb-5f2e8693acf7', 'options': 'recursive'} 2024-04-04T06:11:12.070 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:11:12.078 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:11:39.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:11:39.123 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-04T06:11:39.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.346+0000 7f8af4600640 1 Processor -- start 2024-04-04T06:11:39.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.346+0000 7f8af4600640 1 -- start start 2024-04-04T06:11:39.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.347+0000 7f8af4600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f8aec0d0e70 0x7f8aec0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.347+0000 7f8af4600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aec15f6e0 0x7f8aec0d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.347+0000 7f8af4600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8aec1621d0 0x7f8aec162660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.347+0000 7f8af4600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f8aec0d1790 con 0x7f8aec0d0e70 2024-04-04T06:11:39.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.347+0000 7f8af4600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8aec0cf6c0 con 0x7f8aec1621d0 2024-04-04T06:11:39.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.347+0000 7f8af4600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8aec0cf9a0 con 0x7f8aec15f6e0 2024-04-04T06:11:39.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.347+0000 7f8af1c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f8aec0d0e70 0x7f8aec0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.347+0000 7f8af1c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f8aec0d0e70 0x7f8aec0cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37744/0 (socket says 172.21.15.159:37744) 2024-04-04T06:11:39.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.347+0000 7f8af1c00640 1 -- 172.21.15.159:0/1359620 learned_addr learned my addr 172.21.15.159:0/1359620 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:39.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.348+0000 7f8af2600640 1 --2- 172.21.15.159:0/1359620 >> v2:172.21.15.178:3300/0 conn(0x7f8aec1621d0 0x7f8aec162660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.348+0000 7f8af1c00640 1 -- 172.21.15.159:0/1359620 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aec15f6e0 msgr2=0x7f8aec0d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:39.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.348+0000 7f8af1c00640 1 --2- 172.21.15.159:0/1359620 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aec15f6e0 0x7f8aec0d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.348+0000 7f8af1c00640 1 -- 172.21.15.159:0/1359620 >> v2:172.21.15.178:3300/0 conn(0x7f8aec1621d0 msgr2=0x7f8aec162660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:39.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.348+0000 7f8af1c00640 1 --2- 172.21.15.159:0/1359620 >> v2:172.21.15.178:3300/0 conn(0x7f8aec1621d0 0x7f8aec162660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.348+0000 7f8af1c00640 1 -- 172.21.15.159:0/1359620 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aec0cfc80 con 0x7f8aec0d0e70 2024-04-04T06:11:39.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.349+0000 7f8af1c00640 1 --2- 172.21.15.159:0/1359620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aec0d0e70 0x7f8aec0cf050 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0d3be0 tx=0x7f8ae002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea3cc9b8d0f154e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:39.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.349+0000 7f8aebe00640 1 -- 172.21.15.159:0/1359620 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae002e070 con 0x7f8aec0d0e70 2024-04-04T06:11:39.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.349+0000 7f8aebe00640 1 -- 172.21.15.159:0/1359620 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae0037070 con 0x7f8aec0d0e70 2024-04-04T06:11:39.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.349+0000 7f8aebe00640 1 -- 172.21.15.159:0/1359620 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0031a00 con 0x7f8aec0d0e70 2024-04-04T06:11:39.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.350+0000 7f8af4600640 1 -- 172.21.15.159:0/1359620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aec0d0e70 msgr2=0x7f8aec0cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:39.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.350+0000 7f8af4600640 1 --2- 172.21.15.159:0/1359620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aec0d0e70 0x7f8aec0cf050 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0d3be0 tx=0x7f8ae002b880 comp rx=0 tx=0).stop 2024-04-04T06:11:39.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.350+0000 7f8af4600640 1 -- 172.21.15.159:0/1359620 shutdown_connections 2024-04-04T06:11:39.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.350+0000 7f8af4600640 1 --2- 172.21.15.159:0/1359620 >> v2:172.21.15.178:3300/0 conn(0x7f8aec1621d0 0x7f8aec162660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.350+0000 7f8af4600640 1 --2- 172.21.15.159:0/1359620 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aec15f6e0 0x7f8aec0d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.350+0000 7f8af4600640 1 --2- 172.21.15.159:0/1359620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aec0d0e70 0x7f8aec0cf050 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.350+0000 7f8af4600640 1 -- 172.21.15.159:0/1359620 >> 172.21.15.159:0/1359620 conn(0x7f8aec08bfb0 msgr2=0x7f8aec0d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:39.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.350+0000 7f8af4600640 1 -- 172.21.15.159:0/1359620 shutdown_connections 2024-04-04T06:11:39.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.350+0000 7f8af4600640 1 -- 172.21.15.159:0/1359620 wait complete. 2024-04-04T06:11:39.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.352+0000 7f8af4600640 1 Processor -- start 2024-04-04T06:11:39.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.352+0000 7f8af4600640 1 -- start start 2024-04-04T06:11:39.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.353+0000 7f8af4600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aec0d0e70 0x7f8aec0c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.353+0000 7f8af4600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aec15f6e0 0x7f8aec1f5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.353+0000 7f8af4600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8aec1621d0 0x7f8aec1f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.353+0000 7f8af4600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec15fb40 con 0x7f8aec0d0e70 2024-04-04T06:11:39.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.353+0000 7f8af4600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec0d0240 con 0x7f8aec1621d0 2024-04-04T06:11:39.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.353+0000 7f8af4600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8aec15e910 con 0x7f8aec15f6e0 2024-04-04T06:11:39.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.353+0000 7f8af1c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aec0d0e70 0x7f8aec0c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.353+0000 7f8af1c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aec0d0e70 0x7f8aec0c9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37758/0 (socket says 172.21.15.159:37758) 2024-04-04T06:11:39.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.353+0000 7f8af1c00640 1 -- 172.21.15.159:0/1281945030 learned_addr learned my addr 172.21.15.159:0/1281945030 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:39.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.354+0000 7f8af2600640 1 --2- 172.21.15.159:0/1281945030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8aec1621d0 0x7f8aec1f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.354+0000 7f8af1c00640 1 -- 172.21.15.159:0/1281945030 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aec15f6e0 msgr2=0x7f8aec1f5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:39.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.354+0000 7f8af1200640 1 --2- 172.21.15.159:0/1281945030 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aec15f6e0 0x7f8aec1f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.354+0000 7f8af1c00640 1 --2- 172.21.15.159:0/1281945030 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aec15f6e0 0x7f8aec1f5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.354+0000 7f8af1c00640 1 -- 172.21.15.159:0/1281945030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8aec1621d0 msgr2=0x7f8aec1f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:39.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.354+0000 7f8af1c00640 1 --2- 172.21.15.159:0/1281945030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8aec1621d0 0x7f8aec1f5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.354+0000 7f8af1c00640 1 -- 172.21.15.159:0/1281945030 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aec0ca5d0 con 0x7f8aec0d0e70 2024-04-04T06:11:39.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.354+0000 7f8af2600640 1 --2- 172.21.15.159:0/1281945030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8aec1621d0 0x7f8aec1f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:11:39.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.354+0000 7f8af1c00640 1 --2- 172.21.15.159:0/1281945030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aec0d0e70 0x7f8aec0c9260 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0000c00 tx=0x7f8ae00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:39.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.354+0000 7f8ae9600640 1 -- 172.21.15.159:0/1281945030 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0032880 con 0x7f8aec0d0e70 2024-04-04T06:11:39.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.354+0000 7f8ae9600640 1 -- 172.21.15.159:0/1281945030 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae0032a20 con 0x7f8aec0d0e70 2024-04-04T06:11:39.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.354+0000 7f8ae9600640 1 -- 172.21.15.159:0/1281945030 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0032e10 con 0x7f8aec0d0e70 2024-04-04T06:11:39.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.354+0000 7f8af4600640 1 -- 172.21.15.159:0/1281945030 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aec0c81c0 con 0x7f8aec0d0e70 2024-04-04T06:11:39.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.355+0000 7f8af4600640 1 -- 172.21.15.159:0/1281945030 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aec0c8780 con 0x7f8aec0d0e70 2024-04-04T06:11:39.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.358+0000 7f8ae9600640 1 -- 172.21.15.159:0/1281945030 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ae004c060 con 0x7f8aec0d0e70 2024-04-04T06:11:39.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.358+0000 7f8ae9600640 1 --2- 172.21.15.159:0/1281945030 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8acc06c920 0x7f8acc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.359+0000 7f8ae9600640 1 -- 172.21.15.159:0/1281945030 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f8ae00ad420 con 0x7f8aec0d0e70 2024-04-04T06:11:39.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.359+0000 7f8af721a740 1 Processor -- start 2024-04-04T06:11:39.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.359+0000 7f8af1200640 1 --2- 172.21.15.159:0/1281945030 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8acc06c920 0x7f8acc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.359+0000 7f8af721a740 1 -- start start 2024-04-04T06:11:39.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.360+0000 7f8af721a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bf580 0x557fcd292520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.360+0000 7f8af721a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557fcd2bf960 0x557fcd2c2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.360+0000 7f8af721a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fcd292a60 0x557fcd2c2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.360+0000 7f8af721a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557fcd1fd6d0 con 0x557fcd2bf580 2024-04-04T06:11:39.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.360+0000 7f8af721a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557fcd1a5770 con 0x557fcd292a60 2024-04-04T06:11:39.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.360+0000 7f8af721a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x557fcd1f9680 con 0x557fcd2bf960 2024-04-04T06:11:39.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.360+0000 7f8af1c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bf580 0x557fcd292520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.361+0000 7f8af1c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bf580 0x557fcd292520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37760/0 (socket says 172.21.15.159:37760) 2024-04-04T06:11:39.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.361+0000 7f8af1c00640 1 -- 172.21.15.159:0/375612536 learned_addr learned my addr 172.21.15.159:0/375612536 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:39.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.361+0000 7f8af1200640 1 --2- 172.21.15.159:0/1281945030 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8acc06c920 0x7f8acc06ede0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0cb070 tx=0x7f8ad8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:39.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.361+0000 7f8af2600640 1 --2- 172.21.15.159:0/375612536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fcd292a60 0x557fcd2c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.361+0000 7f8af1200640 1 --2- 172.21.15.159:0/375612536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557fcd2bf960 0x557fcd2c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.361+0000 7f8af1c00640 1 -- 172.21.15.159:0/375612536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557fcd2bf960 msgr2=0x557fcd2c2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:39.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.361+0000 7f8af1c00640 1 --2- 172.21.15.159:0/375612536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557fcd2bf960 0x557fcd2c2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.361+0000 7f8af1c00640 1 -- 172.21.15.159:0/375612536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fcd292a60 msgr2=0x557fcd2c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:39.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.361+0000 7f8af1c00640 1 --2- 172.21.15.159:0/375612536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fcd292a60 0x557fcd2c2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.361+0000 7f8af1c00640 1 -- 172.21.15.159:0/375612536 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fccfdb7d0 con 0x557fcd2bf580 2024-04-04T06:11:39.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.361+0000 7f8af1c00640 1 --2- 172.21.15.159:0/375612536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bf580 0x557fcd292520 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f8ae00ab8c0 tx=0x7f8ae0031eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8417211204c65b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:39.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.361+0000 7f8ade000640 1 -- 172.21.15.159:0/375612536 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae007f500 con 0x557fcd2bf580 2024-04-04T06:11:39.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.362+0000 7f8ade000640 1 -- 172.21.15.159:0/375612536 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae0037030 con 0x557fcd2bf580 2024-04-04T06:11:39.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.362+0000 7f8ade000640 1 -- 172.21.15.159:0/375612536 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0083d30 con 0x557fcd2bf580 2024-04-04T06:11:39.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.362+0000 7f8af721a740 1 -- 172.21.15.159:0/375612536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bf580 msgr2=0x557fcd292520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:39.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.362+0000 7f8af721a740 1 --2- 172.21.15.159:0/375612536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bf580 0x557fcd292520 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f8ae00ab8c0 tx=0x7f8ae0031eb0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.362+0000 7f8af721a740 1 -- 172.21.15.159:0/375612536 shutdown_connections 2024-04-04T06:11:39.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.362+0000 7f8af721a740 1 --2- 172.21.15.159:0/375612536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fcd292a60 0x557fcd2c2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.362+0000 7f8af721a740 1 --2- 172.21.15.159:0/375612536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557fcd2bf960 0x557fcd2c2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.362+0000 7f8af721a740 1 --2- 172.21.15.159:0/375612536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bf580 0x557fcd292520 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.362+0000 7f8af721a740 1 -- 172.21.15.159:0/375612536 >> 172.21.15.159:0/375612536 conn(0x557fcd294790 msgr2=0x557fcd2dd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:39.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.362+0000 7f8af721a740 1 -- 172.21.15.159:0/375612536 shutdown_connections 2024-04-04T06:11:39.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.362+0000 7f8af721a740 1 -- 172.21.15.159:0/375612536 wait complete. 2024-04-04T06:11:39.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.363+0000 7f8af721a740 1 -- start start 2024-04-04T06:11:39.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.363+0000 7f8af721a740 1 Processor -- start 2024-04-04T06:11:39.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.364+0000 7f8af721a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fcd2bf580 0x557fcd2c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.364+0000 7f8af721a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557fcd2bf960 0x557fcd2c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.364+0000 7f8af721a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bd5c0 0x557fcd2c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.364+0000 7f8af721a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557fcd1f9680 con 0x557fcd2bd5c0 2024-04-04T06:11:39.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.364+0000 7f8af721a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557fcd1b8aa0 con 0x557fcd2bf580 2024-04-04T06:11:39.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.364+0000 7f8af721a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x557fcd1fd6d0 con 0x557fcd2bf960 2024-04-04T06:11:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.364+0000 7f8af721a740 10 client.0 _ll_register_callbacks cb 0x557fcd1b8490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:11:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.365+0000 7f8af2600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bd5c0 0x557fcd2c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.365+0000 7f8af2600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bd5c0 0x557fcd2c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37776/0 (socket says 172.21.15.159:37776) 2024-04-04T06:11:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.365+0000 7f8af2600640 1 -- 172.21.15.159:0/3760547921 learned_addr learned my addr 172.21.15.159:0/3760547921 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.365+0000 7f8af1200640 1 --2- 172.21.15.159:0/3760547921 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557fcd2bf960 0x557fcd2c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.365+0000 7f8af2600640 1 -- 172.21.15.159:0/3760547921 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557fcd2bf960 msgr2=0x557fcd2c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.365+0000 7f8af1c00640 1 --2- 172.21.15.159:0/3760547921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fcd2bf580 0x557fcd2c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.365+0000 7f8af2600640 1 --2- 172.21.15.159:0/3760547921 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557fcd2bf960 0x557fcd2c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.365+0000 7f8af2600640 1 -- 172.21.15.159:0/3760547921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fcd2bf580 msgr2=0x557fcd2c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.365+0000 7f8af2600640 1 --2- 172.21.15.159:0/3760547921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fcd2bf580 0x557fcd2c5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.365+0000 7f8af2600640 1 -- 172.21.15.159:0/3760547921 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fccf113f0 con 0x557fcd2bd5c0 2024-04-04T06:11:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.365+0000 7f8af1200640 1 --2- 172.21.15.159:0/3760547921 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557fcd2bf960 0x557fcd2c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:11:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.365+0000 7f8af2600640 1 --2- 172.21.15.159:0/3760547921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bd5c0 0x557fcd2c8a20 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8ae40079e0 tx=0x7f8ae4007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26adb9cb944b824c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:39.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.365+0000 7f8ad2a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:11:39.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.366+0000 7f8ad2a00640 1 -- 172.21.15.159:0/3760547921 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae401a030 con 0x557fcd2bd5c0 2024-04-04T06:11:39.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.366+0000 7f8ad2a00640 1 -- 172.21.15.159:0/3760547921 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f8ae4012040 con 0x557fcd2bd5c0 2024-04-04T06:11:39.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.366+0000 7f8af721a740 1 -- 172.21.15.159:0/3760547921 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ae0037030 con 0x557fcd2bd5c0 2024-04-04T06:11:39.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.366+0000 7f8ad2a00640 1 -- 172.21.15.159:0/3760547921 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f8ae4023050 con 0x557fcd2bd5c0 2024-04-04T06:11:39.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.366+0000 7f8af721a740 10 client.6963 fetch_fsmap learned FSMap version 52 2024-04-04T06:11:39.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.366+0000 7f8af721a740 1 -- 172.21.15.159:0/3760547921 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x557fcced10f0 con 0x557fcd2bd5c0 2024-04-04T06:11:39.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.366+0000 7f8ad2a00640 1 -- 172.21.15.159:0/3760547921 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae0037030 con 0x557fcd2bd5c0 2024-04-04T06:11:39.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.366+0000 7f8ad2a00640 1 -- 172.21.15.159:0/3760547921 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f8ae4026020 con 0x557fcd2bd5c0 2024-04-04T06:11:39.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.366+0000 7f8af721a740 10 client.6963 fetch_fsmap finished waiting for FSMap version 52 2024-04-04T06:11:39.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.366+0000 7f8af721a740 10 client.6963 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-04T06:11:39.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.366+0000 7f8af721a740 20 client.6963 populate_metadata read hostname 'smithi159' 2024-04-04T06:11:39.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.366+0000 7f8af721a740 1 --2- 172.21.15.159:0/3760547921 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x557fcd2c44e0 0x557fcd2c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.366+0000 7f8af721a740 4 client.6963 mds_command: new command op to 6828 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:11:39.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.366+0000 7f8af721a740 1 -- 172.21.15.159:0/3760547921 --> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fcd20ac10 con 0x557fcd2c44e0 2024-04-04T06:11:39.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.367+0000 7f8af1c00640 1 --2- 172.21.15.159:0/3760547921 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x557fcd2c44e0 0x557fcd2c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.367+0000 7f8af1c00640 1 --2- 172.21.15.159:0/3760547921 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x557fcd2c44e0 0x557fcd2c48f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd721086217069e5 server_cookie=fb3da6c1ac2b4ded in_seq=0 out_seq=0 2024-04-04T06:11:39.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.367+0000 7f8ad2a00640 10 client.6963 ms_handle_connect on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:39.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.368+0000 7f8ad2a00640 1 -- 172.21.15.159:0/3760547921 <== mds.0 v2:172.21.15.159:6834/2467594024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557fcd20ac10 con 0x557fcd2c44e0 2024-04-04T06:11:39.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.368+0000 7f8ad2a00640 10 client.6963 handle_command_reply: tid=0 2024-04-04T06:11:39.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.368+0000 7f8af721a740 1 client.6963 shutdown 2024-04-04T06:11:39.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.368+0000 7f8af721a740 1 -- 172.21.15.159:0/3760547921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bd5c0 msgr2=0x557fcd2c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:39.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.368+0000 7f8af721a740 1 --2- 172.21.15.159:0/3760547921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bd5c0 0x557fcd2c8a20 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8ae40079e0 tx=0x7f8ae4007ea0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.368+0000 7f8af721a740 1 -- 172.21.15.159:0/3760547921 shutdown_connections 2024-04-04T06:11:39.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.368+0000 7f8af721a740 1 --2- 172.21.15.159:0/3760547921 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x557fcd2c44e0 0x557fcd2c48f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.368+0000 7f8af721a740 1 --2- 172.21.15.159:0/3760547921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bd5c0 0x557fcd2c8a20 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.368+0000 7f8af721a740 1 --2- 172.21.15.159:0/3760547921 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557fcd2bf960 0x557fcd2c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.368+0000 7f8af721a740 1 --2- 172.21.15.159:0/3760547921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fcd2bf580 0x557fcd2c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.368+0000 7f8af721a740 1 -- 172.21.15.159:0/3760547921 >> 172.21.15.159:0/3760547921 conn(0x557fcd292a60 msgr2=0x557fcd299350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:39.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.368+0000 7f8ad2a00640 0 client.6963 ms_handle_reset on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:39.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.368+0000 7f8af721a740 1 -- 172.21.15.159:0/3760547921 shutdown_connections 2024-04-04T06:11:39.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.369+0000 7f8af721a740 1 -- 172.21.15.159:0/3760547921 wait complete. 2024-04-04T06:11:39.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.369+0000 7f8af721a740 20 client.6963 trim_cache size 0 max 16384 2024-04-04T06:11:39.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.388+0000 7f8af721a740 1 Processor -- start 2024-04-04T06:11:39.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.388+0000 7f8af721a740 1 -- start start 2024-04-04T06:11:39.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.389+0000 7f8af721a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bd5c0 0x557fcd2c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.389+0000 7f8af721a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557fcd2bf580 0x557fcd3512a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.389+0000 7f8af721a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fcd2bf960 0x557fcd2c1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.389+0000 7f8af721a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec200d30 con 0x557fcd2bd5c0 2024-04-04T06:11:39.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.389+0000 7f8af721a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec1ffee0 con 0x557fcd2bf960 2024-04-04T06:11:39.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.389+0000 7f8af721a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8aec1f7dd0 con 0x557fcd2bf580 2024-04-04T06:11:39.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.389+0000 7f8af1c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bd5c0 0x557fcd2c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.389+0000 7f8af1c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bd5c0 0x557fcd2c74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37790/0 (socket says 172.21.15.159:37790) 2024-04-04T06:11:39.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.389+0000 7f8af1c00640 1 -- 172.21.15.159:0/4173495804 learned_addr learned my addr 172.21.15.159:0/4173495804 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:39.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.389+0000 7f8af2600640 1 --2- 172.21.15.159:0/4173495804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fcd2bf960 0x557fcd2c1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.389+0000 7f8af1c00640 1 -- 172.21.15.159:0/4173495804 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557fcd2bf580 msgr2=0x557fcd3512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:39.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.389+0000 7f8af1c00640 1 --2- 172.21.15.159:0/4173495804 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557fcd2bf580 0x557fcd3512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.389+0000 7f8af1c00640 1 -- 172.21.15.159:0/4173495804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fcd2bf960 msgr2=0x557fcd2c1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:39.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.389+0000 7f8af1c00640 1 --2- 172.21.15.159:0/4173495804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fcd2bf960 0x557fcd2c1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.389+0000 7f8af1c00640 1 -- 172.21.15.159:0/4173495804 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fcd301240 con 0x557fcd2bd5c0 2024-04-04T06:11:39.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.390+0000 7f8af1c00640 1 --2- 172.21.15.159:0/4173495804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bd5c0 0x557fcd2c74b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f8ae002b8c0 tx=0x7f8ae0046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ecf5f7185ddd22 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:39.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.390+0000 7f8add600640 1 -- 172.21.15.159:0/4173495804 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae007f500 con 0x557fcd2bd5c0 2024-04-04T06:11:39.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.390+0000 7f8add600640 1 -- 172.21.15.159:0/4173495804 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae0037040 con 0x557fcd2bd5c0 2024-04-04T06:11:39.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.390+0000 7f8add600640 1 -- 172.21.15.159:0/4173495804 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0079450 con 0x557fcd2bd5c0 2024-04-04T06:11:39.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.391+0000 7f8af721a740 1 -- 172.21.15.159:0/4173495804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bd5c0 msgr2=0x557fcd2c74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:39.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.391+0000 7f8af721a740 1 --2- 172.21.15.159:0/4173495804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bd5c0 0x557fcd2c74b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f8ae002b8c0 tx=0x7f8ae0046000 comp rx=0 tx=0).stop 2024-04-04T06:11:39.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.391+0000 7f8af721a740 1 -- 172.21.15.159:0/4173495804 shutdown_connections 2024-04-04T06:11:39.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.391+0000 7f8af721a740 1 --2- 172.21.15.159:0/4173495804 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fcd2bf960 0x557fcd2c1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.391+0000 7f8af721a740 1 --2- 172.21.15.159:0/4173495804 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557fcd2bf580 0x557fcd3512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.391+0000 7f8af721a740 1 --2- 172.21.15.159:0/4173495804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bd5c0 0x557fcd2c74b0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.391+0000 7f8af721a740 1 -- 172.21.15.159:0/4173495804 >> 172.21.15.159:0/4173495804 conn(0x557fcd292a60 msgr2=0x557fcd2c6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:39.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.391+0000 7f8af721a740 1 -- 172.21.15.159:0/4173495804 shutdown_connections 2024-04-04T06:11:39.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.391+0000 7f8af721a740 1 -- 172.21.15.159:0/4173495804 wait complete. 2024-04-04T06:11:39.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.391+0000 7f8af721a740 1 -- start start 2024-04-04T06:11:39.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.392+0000 7f8af721a740 1 Processor -- start 2024-04-04T06:11:39.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.393+0000 7f8af721a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557fcd2bd5c0 0x557fcd2be320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.393+0000 7f8af721a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bf580 0x557fcd2c8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.393+0000 7f8af721a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fcd2bf960 0x557fcd2de800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.393+0000 7f8af721a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557fcd1a5770 con 0x557fcd2bf580 2024-04-04T06:11:39.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.393+0000 7f8af721a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec1f8420 con 0x557fcd2bf960 2024-04-04T06:11:39.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.393+0000 7f8af721a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x557fcd2c5a90 con 0x557fcd2bd5c0 2024-04-04T06:11:39.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.393+0000 7f8af721a740 10 client.0 _ll_register_callbacks cb 0x7f8ae4007540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:11:39.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.393+0000 7f8af1200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bf580 0x557fcd2c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.393+0000 7f8af1200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bf580 0x557fcd2c8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37792/0 (socket says 172.21.15.159:37792) 2024-04-04T06:11:39.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.393+0000 7f8af1200640 1 -- 172.21.15.159:0/3470755912 learned_addr learned my addr 172.21.15.159:0/3470755912 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:39.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.393+0000 7f8af1c00640 1 --2- 172.21.15.159:0/3470755912 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557fcd2bd5c0 0x557fcd2be320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.393+0000 7f8af2600640 1 --2- 172.21.15.159:0/3470755912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fcd2bf960 0x557fcd2de800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.393+0000 7f8af1200640 1 -- 172.21.15.159:0/3470755912 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557fcd2bd5c0 msgr2=0x557fcd2be320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:39.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.393+0000 7f8af1200640 1 --2- 172.21.15.159:0/3470755912 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557fcd2bd5c0 0x557fcd2be320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.393+0000 7f8af1200640 1 -- 172.21.15.159:0/3470755912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fcd2bf960 msgr2=0x557fcd2de800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:39.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.393+0000 7f8af1200640 1 --2- 172.21.15.159:0/3470755912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fcd2bf960 0x557fcd2de800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.394+0000 7f8af1200640 1 -- 172.21.15.159:0/3470755912 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fcd2c8880 con 0x557fcd2bf580 2024-04-04T06:11:39.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.394+0000 7f8af1200640 1 --2- 172.21.15.159:0/3470755912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bf580 0x557fcd2c8340 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0b5470 tx=0x7f8ad80047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad3a2b4cd7066f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:39.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.394+0000 7f8ad2a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:11:39.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.394+0000 7f8ad2a00640 1 -- 172.21.15.159:0/3470755912 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad801c040 con 0x557fcd2bf580 2024-04-04T06:11:39.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.394+0000 7f8ad2a00640 1 -- 172.21.15.159:0/3470755912 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f8ad8020430 con 0x557fcd2bf580 2024-04-04T06:11:39.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.394+0000 7f8af721a740 1 -- 172.21.15.159:0/3470755912 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ae0037040 con 0x557fcd2bf580 2024-04-04T06:11:39.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.394+0000 7f8ad2a00640 1 -- 172.21.15.159:0/3470755912 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f8ad802d050 con 0x557fcd2bf580 2024-04-04T06:11:39.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.394+0000 7f8af721a740 10 client.6969 fetch_fsmap learned FSMap version 52 2024-04-04T06:11:39.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.394+0000 7f8af721a740 1 -- 172.21.15.159:0/3470755912 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x557fcd2bdb20 con 0x557fcd2bf580 2024-04-04T06:11:39.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.395+0000 7f8ad2a00640 1 -- 172.21.15.159:0/3470755912 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae0037040 con 0x557fcd2bf580 2024-04-04T06:11:39.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.395+0000 7f8ad2a00640 1 -- 172.21.15.159:0/3470755912 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f8ad8030020 con 0x557fcd2bf580 2024-04-04T06:11:39.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.395+0000 7f8af721a740 10 client.6969 fetch_fsmap finished waiting for FSMap version 52 2024-04-04T06:11:39.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.395+0000 7f8af721a740 10 client.6969 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-04T06:11:39.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.395+0000 7f8af721a740 20 client.6969 populate_metadata read hostname 'smithi159' 2024-04-04T06:11:39.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.395+0000 7f8af721a740 1 --2- 172.21.15.159:0/3470755912 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x557fcd2ded40 0x557fcd34f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.395+0000 7f8af721a740 4 client.6969 mds_command: new command op to 6828 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-04T06:11:39.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.395+0000 7f8af721a740 1 -- 172.21.15.159:0/3470755912 --> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557fcd2ed210 con 0x557fcd2ded40 2024-04-04T06:11:39.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.395+0000 7f8af1c00640 1 --2- 172.21.15.159:0/3470755912 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x557fcd2ded40 0x557fcd34f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.396+0000 7f8ad2a00640 10 client.6969 ms_handle_connect on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:39.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.396+0000 7f8af1c00640 1 --2- 172.21.15.159:0/3470755912 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x557fcd2ded40 0x557fcd34f920 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=410bd159d98c99b0 server_cookie=29c3ab9ab360584c in_seq=0 out_seq=0 2024-04-04T06:11:39.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.396+0000 7f8ad2a00640 1 -- 172.21.15.159:0/3470755912 <== mds.0 v2:172.21.15.159:6834/2467594024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557fcd2ed210 con 0x557fcd2ded40 2024-04-04T06:11:39.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.396+0000 7f8ad2a00640 10 client.6969 handle_command_reply: tid=0 2024-04-04T06:11:39.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.396+0000 7f8af721a740 1 client.6969 shutdown 2024-04-04T06:11:39.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.396+0000 7f8af721a740 1 -- 172.21.15.159:0/3470755912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bf580 msgr2=0x557fcd2c8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:39.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.396+0000 7f8af721a740 1 --2- 172.21.15.159:0/3470755912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bf580 0x557fcd2c8340 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0b5470 tx=0x7f8ad80047d0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.396+0000 7f8af721a740 1 -- 172.21.15.159:0/3470755912 shutdown_connections 2024-04-04T06:11:39.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.396+0000 7f8af721a740 1 --2- 172.21.15.159:0/3470755912 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x557fcd2ded40 0x557fcd34f920 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.396+0000 7f8af721a740 1 --2- 172.21.15.159:0/3470755912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557fcd2bf960 0x557fcd2de800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.396+0000 7f8af721a740 1 --2- 172.21.15.159:0/3470755912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557fcd2bf580 0x557fcd2c8340 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.396+0000 7f8af721a740 1 --2- 172.21.15.159:0/3470755912 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557fcd2bd5c0 0x557fcd2be320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.396+0000 7f8af721a740 1 -- 172.21.15.159:0/3470755912 >> 172.21.15.159:0/3470755912 conn(0x557fcd292a60 msgr2=0x557fcd2c9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:39.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.396+0000 7f8ad2a00640 0 client.6969 ms_handle_reset on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:39.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.397+0000 7f8af721a740 1 -- 172.21.15.159:0/3470755912 shutdown_connections 2024-04-04T06:11:39.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.397+0000 7f8af721a740 1 -- 172.21.15.159:0/3470755912 wait complete. 2024-04-04T06:11:39.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.397+0000 7f8af721a740 20 client.6969 trim_cache size 0 max 16384 2024-04-04T06:11:39.394 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:11:39.394 INFO:teuthology.orchestra.run.smithi159.stdout: "status": "no active scrubs running", 2024-04-04T06:11:39.394 INFO:teuthology.orchestra.run.smithi159.stdout: "scrubs": {} 2024-04-04T06:11:39.394 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:11:39.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.397+0000 7f8af4600640 1 -- 172.21.15.159:0/1281945030 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8acc06c920 msgr2=0x7f8acc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:39.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.397+0000 7f8af4600640 1 --2- 172.21.15.159:0/1281945030 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8acc06c920 0x7f8acc06ede0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0cb070 tx=0x7f8ad8008040 comp rx=0 tx=0).stop 2024-04-04T06:11:39.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.397+0000 7f8af4600640 1 -- 172.21.15.159:0/1281945030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aec0d0e70 msgr2=0x7f8aec0c9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:39.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.397+0000 7f8af4600640 1 --2- 172.21.15.159:0/1281945030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aec0d0e70 0x7f8aec0c9260 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0000c00 tx=0x7f8ae00046c0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.397+0000 7f8af4600640 1 -- 172.21.15.159:0/1281945030 shutdown_connections 2024-04-04T06:11:39.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.397+0000 7f8af4600640 1 --2- 172.21.15.159:0/1281945030 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8acc06c920 0x7f8acc06ede0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.397+0000 7f8af4600640 1 --2- 172.21.15.159:0/1281945030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8aec1621d0 0x7f8aec1f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.397+0000 7f8af4600640 1 --2- 172.21.15.159:0/1281945030 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8aec15f6e0 0x7f8aec1f5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.397+0000 7f8af4600640 1 --2- 172.21.15.159:0/1281945030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aec0d0e70 0x7f8aec0c9260 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.397+0000 7f8af4600640 1 -- 172.21.15.159:0/1281945030 >> 172.21.15.159:0/1281945030 conn(0x7f8aec08bfb0 msgr2=0x7f8aec15fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:39.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.398+0000 7f8af4600640 1 -- 172.21.15.159:0/1281945030 shutdown_connections 2024-04-04T06:11:39.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.398+0000 7f8af4600640 1 -- 172.21.15.159:0/1281945030 wait complete. 2024-04-04T06:11:39.413 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-04T06:11:39.414 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:11:39.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.616+0000 7fc2e9200640 1 Processor -- start 2024-04-04T06:11:39.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.616+0000 7fc2e9200640 1 -- start start 2024-04-04T06:11:39.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.617+0000 7fc2e9200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc2e40d0d60 0x7fc2e4073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.617+0000 7fc2e9200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2e415fe80 con 0x7fc2e415c390 2024-04-04T06:11:39.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.617+0000 7fc2e9200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2e40ad120 con 0x7fc2e40af830 2024-04-04T06:11:39.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.617+0000 7fc2e9200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fc2e415c770 con 0x7fc2e40d0d60 2024-04-04T06:11:39.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.617+0000 7fc2e2800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc2e40d0d60 0x7fc2e4073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.618+0000 7fc2e2800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc2e40d0d60 0x7fc2e4073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37796/0 (socket says 172.21.15.159:37796) 2024-04-04T06:11:39.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.618+0000 7fc2e2800640 1 -- 172.21.15.159:0/634791368 learned_addr learned my addr 172.21.15.159:0/634791368 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:39.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.618+0000 7fc2e2800640 1 -- 172.21.15.159:0/634791368 >> v1:172.21.15.178:6789/0 conn(0x7fc2e40af830 legacy=0x7fc2e40d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:39.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.618+0000 7fc2e2800640 1 -- 172.21.15.159:0/634791368 >> v1:172.21.15.159:6789/0 conn(0x7fc2e415c390 legacy=0x7fc2e4160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:39.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.618+0000 7fc2e2800640 1 -- 172.21.15.159:0/634791368 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e40ad400 con 0x7fc2e40d0d60 2024-04-04T06:11:39.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.618+0000 7fc2e0a00640 1 -- 172.21.15.159:0/634791368 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1203820485 0 0) 0x7fc2e415fe80 con 0x7fc2e415c390 2024-04-04T06:11:39.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.620+0000 7fc2e2800640 1 --2- 172.21.15.159:0/634791368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2e40d0d60 0x7fc2e4073790 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4009870 tx=0x7fc2d402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820f2a91db2d1382 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:39.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.621+0000 7fc2e0a00640 1 -- 172.21.15.159:0/634791368 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4034070 con 0x7fc2e40d0d60 2024-04-04T06:11:39.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.621+0000 7fc2e0a00640 1 -- 172.21.15.159:0/634791368 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2d4030860 con 0x7fc2e40d0d60 2024-04-04T06:11:39.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.621+0000 7fc2e0a00640 1 -- 172.21.15.159:0/634791368 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4030c30 con 0x7fc2e40d0d60 2024-04-04T06:11:39.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.621+0000 7fc2e9200640 1 -- 172.21.15.159:0/634791368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2e40d0d60 msgr2=0x7fc2e4073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:39.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.621+0000 7fc2e9200640 1 --2- 172.21.15.159:0/634791368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2e40d0d60 0x7fc2e4073790 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4009870 tx=0x7fc2d402b9a0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.621+0000 7fc2e9200640 1 -- 172.21.15.159:0/634791368 shutdown_connections 2024-04-04T06:11:39.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.621+0000 7fc2e9200640 1 --2- 172.21.15.159:0/634791368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2e40d0d60 0x7fc2e4073790 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.621+0000 7fc2e9200640 1 -- 172.21.15.159:0/634791368 >> 172.21.15.159:0/634791368 conn(0x7fc2e406b990 msgr2=0x7fc2e406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:39.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.621+0000 7fc2e9200640 1 -- 172.21.15.159:0/634791368 shutdown_connections 2024-04-04T06:11:39.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.621+0000 7fc2e9200640 1 -- 172.21.15.159:0/634791368 wait complete. 2024-04-04T06:11:39.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.623+0000 7fc2e9200640 1 Processor -- start 2024-04-04T06:11:39.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.623+0000 7fc2e9200640 1 -- start start 2024-04-04T06:11:39.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.624+0000 7fc2e9200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2e40af830 0x7fc2e40b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.624+0000 7fc2e9200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2e40d0d60 0x7fc2e415f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.624+0000 7fc2e9200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2e415c390 0x7fc2e41566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.624+0000 7fc2e9200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e4160d50 con 0x7fc2e40d0d60 2024-04-04T06:11:39.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.624+0000 7fc2e9200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e4160120 con 0x7fc2e40af830 2024-04-04T06:11:39.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.624+0000 7fc2e9200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc2e40cf980 con 0x7fc2e415c390 2024-04-04T06:11:39.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.625+0000 7fc2e1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2e40d0d60 0x7fc2e415f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.625+0000 7fc2e1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2e40d0d60 0x7fc2e415f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37812/0 (socket says 172.21.15.159:37812) 2024-04-04T06:11:39.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.625+0000 7fc2e1400640 1 -- 172.21.15.159:0/2885827199 learned_addr learned my addr 172.21.15.159:0/2885827199 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:39.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.625+0000 7fc2e1e00640 1 --2- 172.21.15.159:0/2885827199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2e40af830 0x7fc2e40b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.625+0000 7fc2e1400640 1 -- 172.21.15.159:0/2885827199 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2e415c390 msgr2=0x7fc2e41566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:39.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.625+0000 7fc2e1400640 1 --2- 172.21.15.159:0/2885827199 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2e415c390 0x7fc2e41566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.625+0000 7fc2e1400640 1 -- 172.21.15.159:0/2885827199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2e40af830 msgr2=0x7fc2e40b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:39.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.625+0000 7fc2e1400640 1 --2- 172.21.15.159:0/2885827199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2e40af830 0x7fc2e40b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.625+0000 7fc2e1400640 1 -- 172.21.15.159:0/2885827199 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e415d4a0 con 0x7fc2e40d0d60 2024-04-04T06:11:39.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.625+0000 7fc2e1e00640 1 --2- 172.21.15.159:0/2885827199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2e40af830 0x7fc2e40b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:11:39.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.626+0000 7fc2e1400640 1 --2- 172.21.15.159:0/2885827199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2e40d0d60 0x7fc2e415f480 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00c7c0 tx=0x7fc2cc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:39.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.626+0000 7fc2da000640 1 -- 172.21.15.159:0/2885827199 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc016020 con 0x7fc2e40d0d60 2024-04-04T06:11:39.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.626+0000 7fc2da000640 1 -- 172.21.15.159:0/2885827199 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2cc00aa50 con 0x7fc2e40d0d60 2024-04-04T06:11:39.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.626+0000 7fc2e9200640 1 -- 172.21.15.159:0/2885827199 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e40b49a0 con 0x7fc2e40d0d60 2024-04-04T06:11:39.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.626+0000 7fc2da000640 1 -- 172.21.15.159:0/2885827199 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc012420 con 0x7fc2e40d0d60 2024-04-04T06:11:39.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.626+0000 7fc2e9200640 1 -- 172.21.15.159:0/2885827199 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e40b1c60 con 0x7fc2e40d0d60 2024-04-04T06:11:39.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.627+0000 7fc2e9200640 1 -- 172.21.15.159:0/2885827199 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e406ed00 con 0x7fc2e40d0d60 2024-04-04T06:11:39.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.628+0000 7fc2da000640 1 -- 172.21.15.159:0/2885827199 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc2cc019020 con 0x7fc2e40d0d60 2024-04-04T06:11:39.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.629+0000 7fc2da000640 1 --2- 172.21.15.159:0/2885827199 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc2c006c8f0 0x7fc2c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:39.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.629+0000 7fc2da000640 1 -- 172.21.15.159:0/2885827199 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7fc2cc08b9f0 con 0x7fc2e40d0d60 2024-04-04T06:11:39.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.630+0000 7fc2e1e00640 1 --2- 172.21.15.159:0/2885827199 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc2c006c8f0 0x7fc2c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:39.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.631+0000 7fc2e1e00640 1 --2- 172.21.15.159:0/2885827199 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc2c006c8f0 0x7fc2c006edb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc007fd0 tx=0x7fc2dc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:39.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.634+0000 7fc2da000640 1 -- 172.21.15.159:0/2885827199 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc2e406ed00 con 0x7fc2e40d0d60 2024-04-04T06:11:39.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.875+0000 7fc2e9200640 1 -- 172.21.15.159:0/2885827199 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2e40685c0 con 0x7fc2e40d0d60 2024-04-04T06:11:39.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.876+0000 7fc2da000640 1 -- 172.21.15.159:0/2885827199 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5193 (secure 0 0 0) 0x7fc2cc020070 con 0x7fc2e40d0d60 2024-04-04T06:11:39.869 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:11:39.869 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3789513463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3789513463},{"type":"v1","addr":"172.21.15.178:6835","nonce":3789513463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/468875883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":468875883},{"type":"v1","addr":"172.21.15.178:6837","nonce":468875883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2546974742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2546974742},{"type":"v1","addr":"172.21.15.159:6833","nonce":2546974742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:10:44.805617+0000","modified":"2024-04-04T06:11:08.957454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6828":{"gid":6828,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":6,"addr":"172.21.15.159:6835/2467594024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2467594024},{"type":"v1","addr":"172.21.15.159:6835","nonce":2467594024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6828,"qdb_cluster":[6828]},"id":5}]} 2024-04-04T06:11:39.870 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 52 2024-04-04T06:11:39.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.879+0000 7fc2e9200640 1 -- 172.21.15.159:0/2885827199 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc2c006c8f0 msgr2=0x7fc2c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:39.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.879+0000 7fc2e9200640 1 --2- 172.21.15.159:0/2885827199 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc2c006c8f0 0x7fc2c006edb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc007fd0 tx=0x7fc2dc009040 comp rx=0 tx=0).stop 2024-04-04T06:11:39.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.879+0000 7fc2e9200640 1 -- 172.21.15.159:0/2885827199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2e40d0d60 msgr2=0x7fc2e415f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:39.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.879+0000 7fc2e9200640 1 --2- 172.21.15.159:0/2885827199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2e40d0d60 0x7fc2e415f480 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00c7c0 tx=0x7fc2cc00cc80 comp rx=0 tx=0).stop 2024-04-04T06:11:39.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.879+0000 7fc2e9200640 1 -- 172.21.15.159:0/2885827199 shutdown_connections 2024-04-04T06:11:39.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.879+0000 7fc2e9200640 1 --2- 172.21.15.159:0/2885827199 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2e415c390 0x7fc2e41566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.879+0000 7fc2e9200640 1 --2- 172.21.15.159:0/2885827199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2e40d0d60 0x7fc2e415f480 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.879+0000 7fc2e9200640 1 --2- 172.21.15.159:0/2885827199 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc2c006c8f0 0x7fc2c006edb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.879+0000 7fc2e9200640 1 --2- 172.21.15.159:0/2885827199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2e40af830 0x7fc2e40b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:39.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.879+0000 7fc2e9200640 1 -- 172.21.15.159:0/2885827199 >> 172.21.15.159:0/2885827199 conn(0x7fc2e406b990 msgr2=0x7fc2e406bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:39.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.879+0000 7fc2e9200640 1 -- 172.21.15.159:0/2885827199 shutdown_connections 2024-04-04T06:11:39.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:39.879+0000 7fc2e9200640 1 -- 172.21.15.159:0/2885827199 wait complete. 2024-04-04T06:11:39.892 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-04T06:11:40.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.103+0000 7f2da5800640 1 Processor -- start 2024-04-04T06:11:40.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.104+0000 7f2da5800640 1 -- start start 2024-04-04T06:11:40.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.105+0000 7f2da5800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2da0153580 0x7f2da00d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.105+0000 7f2da5800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2da00d07c0 0x7f2da00aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.105+0000 7f2da5800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2da00af2e0 0x7f2da006d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.105+0000 7f2da5800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f2da00d0c30 con 0x7f2da0153580 2024-04-04T06:11:40.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.105+0000 7f2da5800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2da00afab0 con 0x7f2da00d07c0 2024-04-04T06:11:40.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.105+0000 7f2da5800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2da015cc80 con 0x7f2da00af2e0 2024-04-04T06:11:40.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.105+0000 7f2d9ea00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2da0153580 0x7f2da00d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.105+0000 7f2d9ea00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2da0153580 0x7f2da00d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37814/0 (socket says 172.21.15.159:37814) 2024-04-04T06:11:40.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.105+0000 7f2d9ea00640 1 -- 172.21.15.159:0/2941133454 learned_addr learned my addr 172.21.15.159:0/2941133454 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:40.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.106+0000 7f2d96000640 1 --2- 172.21.15.159:0/2941133454 >> v2:172.21.15.178:3300/0 conn(0x7f2da00d07c0 0x7f2da00aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.106+0000 7f2d9ea00640 1 -- 172.21.15.159:0/2941133454 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2da00af2e0 msgr2=0x7f2da006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:40.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.106+0000 7f2d9ea00640 1 --2- 172.21.15.159:0/2941133454 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2da00af2e0 0x7f2da006d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.106+0000 7f2d9ea00640 1 -- 172.21.15.159:0/2941133454 >> v2:172.21.15.178:3300/0 conn(0x7f2da00d07c0 msgr2=0x7f2da00aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:40.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.106+0000 7f2d9ea00640 1 --2- 172.21.15.159:0/2941133454 >> v2:172.21.15.178:3300/0 conn(0x7f2da00d07c0 0x7f2da00aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.106+0000 7f2d9ea00640 1 -- 172.21.15.159:0/2941133454 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da015da00 con 0x7f2da0153580 2024-04-04T06:11:40.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.107+0000 7f2d9ea00640 1 --2- 172.21.15.159:0/2941133454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2da0153580 0x7f2da00d0280 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f2da00cff70 tx=0x7f2d9002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82097b9c7100e005 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:40.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.107+0000 7f2d9e000640 1 -- 172.21.15.159:0/2941133454 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9002c9e0 con 0x7f2da0153580 2024-04-04T06:11:40.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.107+0000 7f2d9e000640 1 -- 172.21.15.159:0/2941133454 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d9002cb80 con 0x7f2da0153580 2024-04-04T06:11:40.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.107+0000 7f2d9e000640 1 -- 172.21.15.159:0/2941133454 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d900343f0 con 0x7f2da0153580 2024-04-04T06:11:40.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.108+0000 7f2da5800640 1 -- 172.21.15.159:0/2941133454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2da0153580 msgr2=0x7f2da00d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.108+0000 7f2da5800640 1 --2- 172.21.15.159:0/2941133454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2da0153580 0x7f2da00d0280 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f2da00cff70 tx=0x7f2d9002be40 comp rx=0 tx=0).stop 2024-04-04T06:11:40.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.108+0000 7f2da5800640 1 -- 172.21.15.159:0/2941133454 shutdown_connections 2024-04-04T06:11:40.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.108+0000 7f2da5800640 1 --2- 172.21.15.159:0/2941133454 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2da00af2e0 0x7f2da006d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.108+0000 7f2da5800640 1 --2- 172.21.15.159:0/2941133454 >> v2:172.21.15.178:3300/0 conn(0x7f2da00d07c0 0x7f2da00aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.108+0000 7f2da5800640 1 --2- 172.21.15.159:0/2941133454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2da0153580 0x7f2da00d0280 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.108+0000 7f2da5800640 1 -- 172.21.15.159:0/2941133454 >> 172.21.15.159:0/2941133454 conn(0x7f2da006b750 msgr2=0x7f2da006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:40.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.108+0000 7f2da5800640 1 -- 172.21.15.159:0/2941133454 shutdown_connections 2024-04-04T06:11:40.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.108+0000 7f2da5800640 1 -- 172.21.15.159:0/2941133454 wait complete. 2024-04-04T06:11:40.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.110+0000 7f2da5800640 1 Processor -- start 2024-04-04T06:11:40.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.110+0000 7f2da5800640 1 -- start start 2024-04-04T06:11:40.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.111+0000 7f2da5800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2da00af2e0 0x7f2da00cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.111+0000 7f2da5800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2da00d07c0 0x7f2da01f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.111+0000 7f2da5800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2da0153580 0x7f2da01f8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.111+0000 7f2da5800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2da015d4e0 con 0x7f2da0153580 2024-04-04T06:11:40.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.111+0000 7f2da5800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2da015d200 con 0x7f2da00af2e0 2024-04-04T06:11:40.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.111+0000 7f2da5800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2da0153960 con 0x7f2da00d07c0 2024-04-04T06:11:40.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.111+0000 7f2d9f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2da0153580 0x7f2da01f8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.111+0000 7f2d9f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2da0153580 0x7f2da01f8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37828/0 (socket says 172.21.15.159:37828) 2024-04-04T06:11:40.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.111+0000 7f2d9f400640 1 -- 172.21.15.159:0/3046679336 learned_addr learned my addr 172.21.15.159:0/3046679336 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:40.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.112+0000 7f2d9ea00640 1 --2- 172.21.15.159:0/3046679336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2da00af2e0 0x7f2da00cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.112+0000 7f2d96000640 1 --2- 172.21.15.159:0/3046679336 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2da00d07c0 0x7f2da01f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.112+0000 7f2d9f400640 1 -- 172.21.15.159:0/3046679336 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2da00d07c0 msgr2=0x7f2da01f7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.112+0000 7f2d9f400640 1 --2- 172.21.15.159:0/3046679336 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2da00d07c0 0x7f2da01f7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.112+0000 7f2d9f400640 1 -- 172.21.15.159:0/3046679336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2da00af2e0 msgr2=0x7f2da00cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.112+0000 7f2d9f400640 1 --2- 172.21.15.159:0/3046679336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2da00af2e0 0x7f2da00cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.112+0000 7f2d9f400640 1 -- 172.21.15.159:0/3046679336 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da01f8b60 con 0x7f2da0153580 2024-04-04T06:11:40.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.112+0000 7f2d9ea00640 1 --2- 172.21.15.159:0/3046679336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2da00af2e0 0x7f2da00cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:11:40.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.112+0000 7f2d9f400640 1 --2- 172.21.15.159:0/3046679336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2da0153580 0x7f2da01f8250 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800b410 tx=0x7f2d9800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:40.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.112+0000 7f2d97400640 1 -- 172.21.15.159:0/3046679336 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98015020 con 0x7f2da0153580 2024-04-04T06:11:40.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.112+0000 7f2d97400640 1 -- 172.21.15.159:0/3046679336 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d98011850 con 0x7f2da0153580 2024-04-04T06:11:40.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.112+0000 7f2da5800640 1 -- 172.21.15.159:0/3046679336 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da00cdd40 con 0x7f2da0153580 2024-04-04T06:11:40.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.113+0000 7f2d97400640 1 -- 172.21.15.159:0/3046679336 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98011c40 con 0x7f2da0153580 2024-04-04T06:11:40.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.113+0000 7f2da5800640 1 -- 172.21.15.159:0/3046679336 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da00ccf80 con 0x7f2da0153580 2024-04-04T06:11:40.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.115+0000 7f2d97400640 1 -- 172.21.15.159:0/3046679336 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d98018020 con 0x7f2da0153580 2024-04-04T06:11:40.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.116+0000 7f2d97400640 1 --2- 172.21.15.159:0/3046679336 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d7c06c8f0 0x7f2d7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.116+0000 7f2d97400640 1 -- 172.21.15.159:0/3046679336 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f2d9808ba40 con 0x7f2da0153580 2024-04-04T06:11:40.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.116+0000 7f2d9ea00640 1 --2- 172.21.15.159:0/3046679336 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d7c06c8f0 0x7f2d7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.118+0000 7f2d9ea00640 1 --2- 172.21.15.159:0/3046679336 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d7c06c8f0 0x7f2d7c06edb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2d90035000 tx=0x7f2d90030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:40.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.118+0000 7f2da85aa740 1 Processor -- start 2024-04-04T06:11:40.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.118+0000 7f2da85aa740 1 -- start start 2024-04-04T06:11:40.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.119+0000 7f2da85aa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f695276e0 0x558f69527ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.119+0000 7f2da85aa740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558f694f76b0 0x558f69524950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.119+0000 7f2da85aa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f694fb260 0x558f69524e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.119+0000 7f2da85aa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558f694615c0 con 0x558f694fb260 2024-04-04T06:11:40.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.119+0000 7f2da85aa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558f694093b0 con 0x558f695276e0 2024-04-04T06:11:40.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.119+0000 7f2da85aa740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x558f6945d570 con 0x558f694f76b0 2024-04-04T06:11:40.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.120+0000 7f2d9f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f694fb260 0x558f69524e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.120+0000 7f2d9f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f694fb260 0x558f69524e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37840/0 (socket says 172.21.15.159:37840) 2024-04-04T06:11:40.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.120+0000 7f2d9f400640 1 -- 172.21.15.159:0/1638483506 learned_addr learned my addr 172.21.15.159:0/1638483506 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:40.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.120+0000 7f2d9ea00640 1 --2- 172.21.15.159:0/1638483506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f695276e0 0x558f69527ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.120+0000 7f2d9f400640 1 -- 172.21.15.159:0/1638483506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558f694f76b0 msgr2=0x558f69524950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:40.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.120+0000 7f2d9f400640 1 --2- 172.21.15.159:0/1638483506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558f694f76b0 0x558f69524950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.120+0000 7f2d9f400640 1 -- 172.21.15.159:0/1638483506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f695276e0 msgr2=0x558f69527ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:40.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.120+0000 7f2d9f400640 1 --2- 172.21.15.159:0/1638483506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f695276e0 0x558f69527ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.120+0000 7f2d9f400640 1 -- 172.21.15.159:0/1638483506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f692417c0 con 0x558f694fb260 2024-04-04T06:11:40.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.120+0000 7f2d9ea00640 1 --2- 172.21.15.159:0/1638483506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f695276e0 0x558f69527ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:11:40.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.120+0000 7f2d9f400640 1 --2- 172.21.15.159:0/1638483506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f694fb260 0x558f69524e90 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f2d98061e20 tx=0x7f2d98059fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd5736619c1d39 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:40.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.121+0000 7f2d8f400640 1 -- 172.21.15.159:0/1638483506 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98088070 con 0x558f694fb260 2024-04-04T06:11:40.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.121+0000 7f2d8f400640 1 -- 172.21.15.159:0/1638483506 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d9806f050 con 0x558f694fb260 2024-04-04T06:11:40.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.121+0000 7f2d8f400640 1 -- 172.21.15.159:0/1638483506 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98062350 con 0x558f694fb260 2024-04-04T06:11:40.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.121+0000 7f2da85aa740 1 -- 172.21.15.159:0/1638483506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f694fb260 msgr2=0x558f69524e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.121+0000 7f2da85aa740 1 --2- 172.21.15.159:0/1638483506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f694fb260 0x558f69524e90 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f2d98061e20 tx=0x7f2d98059fb0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.121+0000 7f2da85aa740 1 -- 172.21.15.159:0/1638483506 shutdown_connections 2024-04-04T06:11:40.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.121+0000 7f2da85aa740 1 --2- 172.21.15.159:0/1638483506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f694fb260 0x558f69524e90 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.122+0000 7f2da85aa740 1 --2- 172.21.15.159:0/1638483506 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558f694f76b0 0x558f69524950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.122+0000 7f2da85aa740 1 --2- 172.21.15.159:0/1638483506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f695276e0 0x558f69527ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.122+0000 7f2da85aa740 1 -- 172.21.15.159:0/1638483506 >> 172.21.15.159:0/1638483506 conn(0x558f694f9970 msgr2=0x558f695424f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:40.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.122+0000 7f2da85aa740 1 -- 172.21.15.159:0/1638483506 shutdown_connections 2024-04-04T06:11:40.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.122+0000 7f2da85aa740 1 -- 172.21.15.159:0/1638483506 wait complete. 2024-04-04T06:11:40.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.122+0000 7f2da85aa740 1 -- start start 2024-04-04T06:11:40.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.122+0000 7f2da85aa740 1 Processor -- start 2024-04-04T06:11:40.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.123+0000 7f2da85aa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f694fb260 0x558f69514b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.123+0000 7f2da85aa740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558f695276e0 0x558f69529cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.123+0000 7f2da85aa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f6952a1f0 0x558f6953b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.123+0000 7f2da85aa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558f694fc410 con 0x558f6952a1f0 2024-04-04T06:11:40.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.124+0000 7f2da85aa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558f6941c7a0 con 0x558f694fb260 2024-04-04T06:11:40.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.124+0000 7f2da85aa740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x558f694fb800 con 0x558f695276e0 2024-04-04T06:11:40.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.124+0000 7f2da85aa740 10 client.0 _ll_register_callbacks cb 0x558f69458fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:11:40.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.124+0000 7f2d9f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f6952a1f0 0x558f6953b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.124+0000 7f2d9f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f6952a1f0 0x558f6953b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37850/0 (socket says 172.21.15.159:37850) 2024-04-04T06:11:40.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.124+0000 7f2d9f400640 1 -- 172.21.15.159:0/1994466266 learned_addr learned my addr 172.21.15.159:0/1994466266 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:40.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.124+0000 7f2d9ea00640 1 --2- 172.21.15.159:0/1994466266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f694fb260 0x558f69514b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.124+0000 7f2d9f400640 1 -- 172.21.15.159:0/1994466266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558f695276e0 msgr2=0x558f69529cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:40.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.124+0000 7f2d9f400640 1 --2- 172.21.15.159:0/1994466266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558f695276e0 0x558f69529cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.124+0000 7f2d9f400640 1 -- 172.21.15.159:0/1994466266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f694fb260 msgr2=0x558f69514b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:40.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.124+0000 7f2d9f400640 1 --2- 172.21.15.159:0/1994466266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f694fb260 0x558f69514b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.124+0000 7f2d9f400640 1 -- 172.21.15.159:0/1994466266 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f692a1f50 con 0x558f6952a1f0 2024-04-04T06:11:40.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.124+0000 7f2d9f400640 1 --2- 172.21.15.159:0/1994466266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f6952a1f0 0x558f6953b750 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f2d98089f40 tx=0x7f2d98009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83502b9e7b81098e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:40.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.124+0000 7f2d83e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:11:40.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.125+0000 7f2d83e00640 1 -- 172.21.15.159:0/1994466266 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98088030 con 0x558f6952a1f0 2024-04-04T06:11:40.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.125+0000 7f2d83e00640 1 -- 172.21.15.159:0/1994466266 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f2d98057a70 con 0x558f6952a1f0 2024-04-04T06:11:40.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.125+0000 7f2da85aa740 1 -- 172.21.15.159:0/1994466266 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d9806f050 con 0x558f6952a1f0 2024-04-04T06:11:40.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.125+0000 7f2d83e00640 1 -- 172.21.15.159:0/1994466266 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f2d98081050 con 0x558f6952a1f0 2024-04-04T06:11:40.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.125+0000 7f2da85aa740 10 client.6990 fetch_fsmap learned FSMap version 52 2024-04-04T06:11:40.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.125+0000 7f2da85aa740 1 -- 172.21.15.159:0/1994466266 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x558f691350b0 con 0x558f6952a1f0 2024-04-04T06:11:40.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.125+0000 7f2d83e00640 1 -- 172.21.15.159:0/1994466266 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d9806f050 con 0x558f6952a1f0 2024-04-04T06:11:40.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.125+0000 7f2d83e00640 1 -- 172.21.15.159:0/1994466266 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f2d98055de0 con 0x558f6952a1f0 2024-04-04T06:11:40.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.125+0000 7f2da85aa740 10 client.6990 fetch_fsmap finished waiting for FSMap version 52 2024-04-04T06:11:40.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.125+0000 7f2da85aa740 10 client.6990 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:11:40.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.125+0000 7f2da85aa740 20 client.6990 populate_metadata read hostname 'smithi159' 2024-04-04T06:11:40.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.125+0000 7f2da85aa740 1 --2- 172.21.15.159:0/1994466266 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x558f6955e4d0 0x558f695608c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.125+0000 7f2da85aa740 4 client.6990 mds_command: new command op to 6828 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:11:40.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.125+0000 7f2da85aa740 1 -- 172.21.15.159:0/1994466266 --> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f69435890 con 0x558f6955e4d0 2024-04-04T06:11:40.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.126+0000 7f2d9ea00640 1 --2- 172.21.15.159:0/1994466266 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x558f6955e4d0 0x558f695608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.126+0000 7f2d9ea00640 1 --2- 172.21.15.159:0/1994466266 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x558f6955e4d0 0x558f695608c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea5f2a0c720a4fa8 server_cookie=79e01fa5eafcdd3e in_seq=0 out_seq=0 2024-04-04T06:11:40.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.126+0000 7f2d83e00640 10 client.6990 ms_handle_connect on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:40.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.127+0000 7f2d83e00640 1 -- 172.21.15.159:0/1994466266 <== mds.0 v2:172.21.15.159:6834/2467594024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558f69435890 con 0x558f6955e4d0 2024-04-04T06:11:40.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.127+0000 7f2d83e00640 10 client.6990 handle_command_reply: tid=0 2024-04-04T06:11:40.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.127+0000 7f2da85aa740 1 client.6990 shutdown 2024-04-04T06:11:40.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.127+0000 7f2da85aa740 1 -- 172.21.15.159:0/1994466266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f6952a1f0 msgr2=0x558f6953b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.127+0000 7f2da85aa740 1 --2- 172.21.15.159:0/1994466266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f6952a1f0 0x558f6953b750 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f2d98089f40 tx=0x7f2d98009f70 comp rx=0 tx=0).stop 2024-04-04T06:11:40.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.127+0000 7f2da85aa740 1 -- 172.21.15.159:0/1994466266 shutdown_connections 2024-04-04T06:11:40.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.127+0000 7f2da85aa740 1 --2- 172.21.15.159:0/1994466266 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x558f6955e4d0 0x558f695608c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.127+0000 7f2da85aa740 1 --2- 172.21.15.159:0/1994466266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f6952a1f0 0x558f6953b750 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.127+0000 7f2da85aa740 1 --2- 172.21.15.159:0/1994466266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558f695276e0 0x558f69529cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.127+0000 7f2da85aa740 1 --2- 172.21.15.159:0/1994466266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f694fb260 0x558f69514b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.127+0000 7f2da85aa740 1 -- 172.21.15.159:0/1994466266 >> 172.21.15.159:0/1994466266 conn(0x558f694f76b0 msgr2=0x558f69545a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:40.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.127+0000 7f2d83e00640 0 client.6990 ms_handle_reset on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:40.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.128+0000 7f2da85aa740 1 -- 172.21.15.159:0/1994466266 shutdown_connections 2024-04-04T06:11:40.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.128+0000 7f2da85aa740 1 -- 172.21.15.159:0/1994466266 wait complete. 2024-04-04T06:11:40.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.128+0000 7f2da85aa740 20 client.6990 trim_cache size 0 max 16384 2024-04-04T06:11:40.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.150+0000 7f2da85aa740 1 Processor -- start 2024-04-04T06:11:40.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.151+0000 7f2da85aa740 1 -- start start 2024-04-04T06:11:40.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.151+0000 7f2da85aa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f694fb260 0x558f69537e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.151+0000 7f2da85aa740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558f695276e0 0x558f6952b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.151+0000 7f2da85aa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f6952a1f0 0x558f69544ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.151+0000 7f2da85aa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2da01ffea0 con 0x558f6952a1f0 2024-04-04T06:11:40.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.151+0000 7f2da85aa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2da01f5480 con 0x558f694fb260 2024-04-04T06:11:40.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.151+0000 7f2da85aa740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2da00cb3d0 con 0x558f695276e0 2024-04-04T06:11:40.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.152+0000 7f2d9f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f6952a1f0 0x558f69544ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.152+0000 7f2d9f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f6952a1f0 0x558f69544ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37862/0 (socket says 172.21.15.159:37862) 2024-04-04T06:11:40.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.152+0000 7f2d9f400640 1 -- 172.21.15.159:0/2087357591 learned_addr learned my addr 172.21.15.159:0/2087357591 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:40.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.152+0000 7f2d9f400640 1 -- 172.21.15.159:0/2087357591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558f695276e0 msgr2=0x558f6952b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:40.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.152+0000 7f2d9f400640 1 --2- 172.21.15.159:0/2087357591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558f695276e0 0x558f6952b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.152+0000 7f2d9f400640 1 -- 172.21.15.159:0/2087357591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f694fb260 msgr2=0x558f69537e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:40.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.152+0000 7f2d9f400640 1 --2- 172.21.15.159:0/2087357591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f694fb260 0x558f69537e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.152+0000 7f2d9f400640 1 -- 172.21.15.159:0/2087357591 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f694f9440 con 0x558f6952a1f0 2024-04-04T06:11:40.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.152+0000 7f2d9f400640 1 --2- 172.21.15.159:0/2087357591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f6952a1f0 0x558f69544ab0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f2d98089fc0 tx=0x7f2d98084000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6569c4a0705d39d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:40.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.153+0000 7f2d8ea00640 1 -- 172.21.15.159:0/2087357591 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98088070 con 0x558f6952a1f0 2024-04-04T06:11:40.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.153+0000 7f2d8ea00640 1 -- 172.21.15.159:0/2087357591 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d9806f070 con 0x558f6952a1f0 2024-04-04T06:11:40.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.153+0000 7f2d8ea00640 1 -- 172.21.15.159:0/2087357591 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9807c020 con 0x558f6952a1f0 2024-04-04T06:11:40.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.153+0000 7f2da85aa740 1 -- 172.21.15.159:0/2087357591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f6952a1f0 msgr2=0x558f69544ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.153+0000 7f2da85aa740 1 --2- 172.21.15.159:0/2087357591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f6952a1f0 0x558f69544ab0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f2d98089fc0 tx=0x7f2d98084000 comp rx=0 tx=0).stop 2024-04-04T06:11:40.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.153+0000 7f2da85aa740 1 -- 172.21.15.159:0/2087357591 shutdown_connections 2024-04-04T06:11:40.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.153+0000 7f2da85aa740 1 --2- 172.21.15.159:0/2087357591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f6952a1f0 0x558f69544ab0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.153+0000 7f2da85aa740 1 --2- 172.21.15.159:0/2087357591 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558f695276e0 0x558f6952b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.153+0000 7f2da85aa740 1 --2- 172.21.15.159:0/2087357591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f694fb260 0x558f69537e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.153+0000 7f2da85aa740 1 -- 172.21.15.159:0/2087357591 >> 172.21.15.159:0/2087357591 conn(0x558f694f76b0 msgr2=0x558f69524420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:40.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.153+0000 7f2da85aa740 1 -- 172.21.15.159:0/2087357591 shutdown_connections 2024-04-04T06:11:40.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.153+0000 7f2da85aa740 1 -- 172.21.15.159:0/2087357591 wait complete. 2024-04-04T06:11:40.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.154+0000 7f2da85aa740 1 -- start start 2024-04-04T06:11:40.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.155+0000 7f2da85aa740 1 Processor -- start 2024-04-04T06:11:40.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.156+0000 7f2da85aa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f694fb260 0x558f69524980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.156+0000 7f2da85aa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f695276e0 0x558f69565b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.156+0000 7f2da85aa740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558f6952a1f0 0x558f69566040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.156+0000 7f2da85aa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558f694fc410 con 0x558f695276e0 2024-04-04T06:11:40.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.156+0000 7f2da85aa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558f6941c7a0 con 0x558f694fb260 2024-04-04T06:11:40.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.156+0000 7f2da85aa740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x558f695562c0 con 0x558f6952a1f0 2024-04-04T06:11:40.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.156+0000 7f2da85aa740 10 client.0 _ll_register_callbacks cb 0x7f2d98086070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:11:40.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.157+0000 7f2d96000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f695276e0 0x558f69565b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.157+0000 7f2d96000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f695276e0 0x558f69565b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37870/0 (socket says 172.21.15.159:37870) 2024-04-04T06:11:40.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.157+0000 7f2d96000640 1 -- 172.21.15.159:0/1229633570 learned_addr learned my addr 172.21.15.159:0/1229633570 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:40.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.157+0000 7f2d96000640 1 -- 172.21.15.159:0/1229633570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558f6952a1f0 msgr2=0x558f69566040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:40.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.157+0000 7f2d96000640 1 --2- 172.21.15.159:0/1229633570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558f6952a1f0 0x558f69566040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.157+0000 7f2d9ea00640 1 --2- 172.21.15.159:0/1229633570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f694fb260 0x558f69524980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.157+0000 7f2d96000640 1 -- 172.21.15.159:0/1229633570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f694fb260 msgr2=0x558f69524980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:40.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.157+0000 7f2d96000640 1 --2- 172.21.15.159:0/1229633570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f694fb260 0x558f69524980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.157+0000 7f2d96000640 1 -- 172.21.15.159:0/1229633570 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f695bbc00 con 0x558f695276e0 2024-04-04T06:11:40.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.157+0000 7f2d96000640 1 --2- 172.21.15.159:0/1229633570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f695276e0 0x558f69565b00 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2d88004770 tx=0x7f2d8800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686efcb67b68239f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:40.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.157+0000 7f2d83e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:11:40.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.158+0000 7f2d83e00640 1 -- 172.21.15.159:0/1229633570 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88016070 con 0x558f695276e0 2024-04-04T06:11:40.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.158+0000 7f2d83e00640 1 -- 172.21.15.159:0/1229633570 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f2d880123f0 con 0x558f695276e0 2024-04-04T06:11:40.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.158+0000 7f2da85aa740 1 -- 172.21.15.159:0/1229633570 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d9806f070 con 0x558f695276e0 2024-04-04T06:11:40.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.158+0000 7f2d83e00640 1 -- 172.21.15.159:0/1229633570 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f2d88021050 con 0x558f695276e0 2024-04-04T06:11:40.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.158+0000 7f2da85aa740 10 client.6996 fetch_fsmap learned FSMap version 52 2024-04-04T06:11:40.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.158+0000 7f2da85aa740 1 -- 172.21.15.159:0/1229633570 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x558f69545320 con 0x558f695276e0 2024-04-04T06:11:40.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.158+0000 7f2d83e00640 1 -- 172.21.15.159:0/1229633570 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d9806f070 con 0x558f695276e0 2024-04-04T06:11:40.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.158+0000 7f2d83e00640 1 -- 172.21.15.159:0/1229633570 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f2d88024020 con 0x558f695276e0 2024-04-04T06:11:40.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.158+0000 7f2da85aa740 10 client.6996 fetch_fsmap finished waiting for FSMap version 52 2024-04-04T06:11:40.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.158+0000 7f2da85aa740 10 client.6996 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:11:40.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.158+0000 7f2da85aa740 20 client.6996 populate_metadata read hostname 'smithi159' 2024-04-04T06:11:40.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.158+0000 7f2da85aa740 1 --2- 172.21.15.159:0/1229633570 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x558f69527ac0 0x558f69527ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.158+0000 7f2da85aa740 4 client.6996 mds_command: new command op to 6828 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-04T06:11:40.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.158+0000 7f2da85aa740 1 -- 172.21.15.159:0/1229633570 --> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x558f6946eb00 con 0x558f69527ac0 2024-04-04T06:11:40.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.159+0000 7f2d9ea00640 1 --2- 172.21.15.159:0/1229633570 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x558f69527ac0 0x558f69527ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.159+0000 7f2d9ea00640 1 --2- 172.21.15.159:0/1229633570 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x558f69527ac0 0x558f69527ea0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee47157540af852 server_cookie=51ca8691c768f6b3 in_seq=0 out_seq=0 2024-04-04T06:11:40.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.159+0000 7f2d83e00640 10 client.6996 ms_handle_connect on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:40.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.159+0000 7f2d83e00640 1 -- 172.21.15.159:0/1229633570 <== mds.0 v2:172.21.15.159:6834/2467594024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+235 (crc 0 0 0) 0x558f6946eb00 con 0x558f69527ac0 2024-04-04T06:11:40.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.159+0000 7f2d83e00640 10 client.6996 handle_command_reply: tid=0 2024-04-04T06:11:40.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.159+0000 7f2da85aa740 1 client.6996 shutdown 2024-04-04T06:11:40.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.159+0000 7f2da85aa740 1 -- 172.21.15.159:0/1229633570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f695276e0 msgr2=0x558f69565b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.159+0000 7f2da85aa740 1 --2- 172.21.15.159:0/1229633570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f695276e0 0x558f69565b00 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2d88004770 tx=0x7f2d8800d520 comp rx=0 tx=0).stop 2024-04-04T06:11:40.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.160+0000 7f2da85aa740 1 -- 172.21.15.159:0/1229633570 shutdown_connections 2024-04-04T06:11:40.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.160+0000 7f2da85aa740 1 --2- 172.21.15.159:0/1229633570 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x558f69527ac0 0x558f69527ea0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.160+0000 7f2da85aa740 1 --2- 172.21.15.159:0/1229633570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558f6952a1f0 0x558f69566040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.160+0000 7f2da85aa740 1 --2- 172.21.15.159:0/1229633570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f695276e0 0x558f69565b00 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.160+0000 7f2da85aa740 1 --2- 172.21.15.159:0/1229633570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f694fb260 0x558f69524980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.160+0000 7f2d83e00640 0 client.6996 ms_handle_reset on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:40.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.160+0000 7f2da85aa740 1 -- 172.21.15.159:0/1229633570 >> 172.21.15.159:0/1229633570 conn(0x558f694f76b0 msgr2=0x558f695381a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:40.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.160+0000 7f2da85aa740 1 -- 172.21.15.159:0/1229633570 shutdown_connections 2024-04-04T06:11:40.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.160+0000 7f2da85aa740 1 -- 172.21.15.159:0/1229633570 wait complete. 2024-04-04T06:11:40.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.160+0000 7f2da85aa740 20 client.6996 trim_cache size 0 max 16384 2024-04-04T06:11:40.156 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:11:40.156 INFO:teuthology.orchestra.run.smithi159.stdout:[ 2024-04-04T06:11:40.157 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:11:40.157 INFO:teuthology.orchestra.run.smithi159.stdout: "damage_type": "dentry", 2024-04-04T06:11:40.157 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 83422231, 2024-04-04T06:11:40.157 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 1099511627776, 2024-04-04T06:11:40.157 INFO:teuthology.orchestra.run.smithi159.stdout: "frag": "*", 2024-04-04T06:11:40.157 INFO:teuthology.orchestra.run.smithi159.stdout: "dname": "file_to_be_damaged", 2024-04-04T06:11:40.157 INFO:teuthology.orchestra.run.smithi159.stdout: "snap_id": "head", 2024-04-04T06:11:40.157 INFO:teuthology.orchestra.run.smithi159.stdout: "path": "/subdir/file_to_be_damaged" 2024-04-04T06:11:40.157 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:11:40.157 INFO:teuthology.orchestra.run.smithi159.stdout:] 2024-04-04T06:11:40.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.161+0000 7f2da5800640 1 -- 172.21.15.159:0/3046679336 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d7c06c8f0 msgr2=0x7f2d7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.161+0000 7f2da5800640 1 --2- 172.21.15.159:0/3046679336 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d7c06c8f0 0x7f2d7c06edb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2d90035000 tx=0x7f2d90030000 comp rx=0 tx=0).stop 2024-04-04T06:11:40.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.161+0000 7f2da5800640 1 -- 172.21.15.159:0/3046679336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2da0153580 msgr2=0x7f2da01f8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.161+0000 7f2da5800640 1 --2- 172.21.15.159:0/3046679336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2da0153580 0x7f2da01f8250 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800b410 tx=0x7f2d9800b8d0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.161+0000 7f2da5800640 1 -- 172.21.15.159:0/3046679336 shutdown_connections 2024-04-04T06:11:40.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.161+0000 7f2da5800640 1 --2- 172.21.15.159:0/3046679336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2da0153580 0x7f2da01f8250 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.161+0000 7f2da5800640 1 --2- 172.21.15.159:0/3046679336 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2da00d07c0 0x7f2da01f7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.161+0000 7f2da5800640 1 --2- 172.21.15.159:0/3046679336 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d7c06c8f0 0x7f2d7c06edb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.161+0000 7f2da5800640 1 --2- 172.21.15.159:0/3046679336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2da00af2e0 0x7f2da00cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.161+0000 7f2da5800640 1 -- 172.21.15.159:0/3046679336 >> 172.21.15.159:0/3046679336 conn(0x7f2da006b750 msgr2=0x7f2da015e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:40.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.161+0000 7f2da5800640 1 -- 172.21.15.159:0/3046679336 shutdown_connections 2024-04-04T06:11:40.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.161+0000 7f2da5800640 1 -- 172.21.15.159:0/3046679336 wait complete. 2024-04-04T06:11:40.172 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-04T06:11:40.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.361+0000 7f82cac00640 1 Processor -- start 2024-04-04T06:11:40.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.361+0000 7f82cac00640 1 -- start start 2024-04-04T06:11:40.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.363+0000 7f82cac00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f82c413f300 0x7f82c406e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.363+0000 7f82cac00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f82c41419e0 0x7f82c415fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.363+0000 7f82cac00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82c4142880 con 0x7f82c4141e40 2024-04-04T06:11:40.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.363+0000 7f82cac00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f82c40cee90 con 0x7f82c413f300 2024-04-04T06:11:40.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.363+0000 7f82cac00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f82c41422c0 con 0x7f82c41419e0 2024-04-04T06:11:40.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.363+0000 7f82c9800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f82c413f300 0x7f82c406e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.363+0000 7f82ca200640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f82c4141e40 0x7f82c415c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:59714/0 (socket says 172.21.15.159:59714) 2024-04-04T06:11:40.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.363+0000 7f82ca200640 1 -- 172.21.15.159:0/3775855420 learned_addr learned my addr 172.21.15.159:0/3775855420 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:11:40.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.364+0000 7f82c8e00640 1 --2- 172.21.15.159:0/3775855420 >> v2:172.21.15.178:3300/0 conn(0x7f82c41419e0 0x7f82c415fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.364+0000 7f82c9800640 1 -- 172.21.15.159:0/3775855420 >> v2:172.21.15.178:3300/0 conn(0x7f82c41419e0 msgr2=0x7f82c415fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:40.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.364+0000 7f82c9800640 1 --2- 172.21.15.159:0/3775855420 >> v2:172.21.15.178:3300/0 conn(0x7f82c41419e0 0x7f82c415fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.364+0000 7f82c9800640 1 -- 172.21.15.159:0/3775855420 >> v1:172.21.15.159:6789/0 conn(0x7f82c4141e40 legacy=0x7f82c415c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.364+0000 7f82c3e00640 1 -- 172.21.15.159:0/3775855420 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2422792773 0 0) 0x7f82c4142880 con 0x7f82c4141e40 2024-04-04T06:11:40.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.364+0000 7f82c9800640 1 -- 172.21.15.159:0/3775855420 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c41425a0 con 0x7f82c413f300 2024-04-04T06:11:40.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.366+0000 7f82c9800640 1 --2- 172.21.15.159:0/3775855420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82c413f300 0x7f82c406e030 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f82c40cf010 tx=0x7f82b802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e287dbad00b593b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:40.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.366+0000 7f82c3e00640 1 -- 172.21.15.159:0/3775855420 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b8035020 con 0x7f82c413f300 2024-04-04T06:11:40.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.366+0000 7f82c3e00640 1 -- 172.21.15.159:0/3775855420 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82b8031830 con 0x7f82c413f300 2024-04-04T06:11:40.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.366+0000 7f82c3e00640 1 -- 172.21.15.159:0/3775855420 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b8031c00 con 0x7f82c413f300 2024-04-04T06:11:40.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.366+0000 7f82cac00640 1 -- 172.21.15.159:0/3775855420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82c413f300 msgr2=0x7f82c406e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.366+0000 7f82cac00640 1 --2- 172.21.15.159:0/3775855420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82c413f300 0x7f82c406e030 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f82c40cf010 tx=0x7f82b802b8d0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.367+0000 7f82cac00640 1 -- 172.21.15.159:0/3775855420 shutdown_connections 2024-04-04T06:11:40.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.367+0000 7f82cac00640 1 --2- 172.21.15.159:0/3775855420 >> v2:172.21.15.178:3300/0 conn(0x7f82c41419e0 0x7f82c415fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.367+0000 7f82cac00640 1 --2- 172.21.15.159:0/3775855420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82c413f300 0x7f82c406e030 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.367+0000 7f82cac00640 1 -- 172.21.15.159:0/3775855420 >> 172.21.15.159:0/3775855420 conn(0x7f82c406b570 msgr2=0x7f82c406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:40.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.367+0000 7f82cac00640 1 -- 172.21.15.159:0/3775855420 shutdown_connections 2024-04-04T06:11:40.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.367+0000 7f82cac00640 1 -- 172.21.15.159:0/3775855420 wait complete. 2024-04-04T06:11:40.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.368+0000 7f82cac00640 1 Processor -- start 2024-04-04T06:11:40.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.369+0000 7f82cac00640 1 -- start start 2024-04-04T06:11:40.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.370+0000 7f82cac00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c413f300 0x7f82c40b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.370+0000 7f82cac00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82c41419e0 0x7f82c41565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.370+0000 7f82cac00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82c4141e40 0x7f82c40b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.370+0000 7f82cac00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f82c40d04b0 con 0x7f82c41419e0 2024-04-04T06:11:40.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.370+0000 7f82cac00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f82c40d0330 con 0x7f82c413f300 2024-04-04T06:11:40.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.370+0000 7f82cac00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f82c40d0630 con 0x7f82c4141e40 2024-04-04T06:11:40.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.370+0000 7f82c8e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82c41419e0 0x7f82c41565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.370+0000 7f82c8e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82c41419e0 0x7f82c41565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37880/0 (socket says 172.21.15.159:37880) 2024-04-04T06:11:40.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.370+0000 7f82c8e00640 1 -- 172.21.15.159:0/4083923888 learned_addr learned my addr 172.21.15.159:0/4083923888 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:40.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.370+0000 7f82c9800640 1 --2- 172.21.15.159:0/4083923888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c413f300 0x7f82c40b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.370+0000 7f82c8e00640 1 -- 172.21.15.159:0/4083923888 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82c4141e40 msgr2=0x7f82c40b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.370+0000 7f82c8e00640 1 --2- 172.21.15.159:0/4083923888 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82c4141e40 0x7f82c40b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.370+0000 7f82c8e00640 1 -- 172.21.15.159:0/4083923888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c413f300 msgr2=0x7f82c40b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.370+0000 7f82c8e00640 1 --2- 172.21.15.159:0/4083923888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c413f300 0x7f82c40b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.370+0000 7f82c8e00640 1 -- 172.21.15.159:0/4083923888 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c40b3aa0 con 0x7f82c41419e0 2024-04-04T06:11:40.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.371+0000 7f82c8e00640 1 --2- 172.21.15.159:0/4083923888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82c41419e0 0x7f82c41565c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f82b000d800 tx=0x7f82b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:40.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.371+0000 7f82c1600640 1 -- 172.21.15.159:0/4083923888 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b0004020 con 0x7f82c41419e0 2024-04-04T06:11:40.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.371+0000 7f82c1600640 1 -- 172.21.15.159:0/4083923888 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82b00041c0 con 0x7f82c41419e0 2024-04-04T06:11:40.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.371+0000 7f82cac00640 1 -- 172.21.15.159:0/4083923888 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c40b3d80 con 0x7f82c41419e0 2024-04-04T06:11:40.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.371+0000 7f82c1600640 1 -- 172.21.15.159:0/4083923888 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b0004390 con 0x7f82c41419e0 2024-04-04T06:11:40.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.371+0000 7f82cac00640 1 -- 172.21.15.159:0/4083923888 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c40b4340 con 0x7f82c41419e0 2024-04-04T06:11:40.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.372+0000 7f82cac00640 1 -- 172.21.15.159:0/4083923888 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c4075940 con 0x7f82c41419e0 2024-04-04T06:11:40.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.373+0000 7f82c1600640 1 -- 172.21.15.159:0/4083923888 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f82b001b020 con 0x7f82c41419e0 2024-04-04T06:11:40.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.374+0000 7f82c1600640 1 --2- 172.21.15.159:0/4083923888 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f82a406c8f0 0x7f82a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.374+0000 7f82c1600640 1 -- 172.21.15.159:0/4083923888 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f82b008ba00 con 0x7f82c41419e0 2024-04-04T06:11:40.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.374+0000 7f82c9800640 1 --2- 172.21.15.159:0/4083923888 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f82a406c8f0 0x7f82a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.376+0000 7f82c9800640 1 --2- 172.21.15.159:0/4083923888 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f82a406c8f0 0x7f82a406edb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f82b8036000 tx=0x7f82b8008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:40.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.378+0000 7f82c1600640 1 -- 172.21.15.159:0/4083923888 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f82b0059d80 con 0x7f82c41419e0 2024-04-04T06:11:40.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.659+0000 7f82cac00640 1 -- 172.21.15.159:0/4083923888 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f82c406d790 con 0x7f82c41419e0 2024-04-04T06:11:40.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.659+0000 7f82c1600640 1 -- 172.21.15.159:0/4083923888 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f82b0059680 con 0x7f82c41419e0 2024-04-04T06:11:40.652 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:11:40.652 INFO:teuthology.orchestra.run.smithi159.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-04T06:11:40.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.662+0000 7f82cac00640 1 -- 172.21.15.159:0/4083923888 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f82a406c8f0 msgr2=0x7f82a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.662+0000 7f82cac00640 1 --2- 172.21.15.159:0/4083923888 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f82a406c8f0 0x7f82a406edb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f82b8036000 tx=0x7f82b8008030 comp rx=0 tx=0).stop 2024-04-04T06:11:40.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.662+0000 7f82cac00640 1 -- 172.21.15.159:0/4083923888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82c41419e0 msgr2=0x7f82c41565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.662+0000 7f82cac00640 1 --2- 172.21.15.159:0/4083923888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82c41419e0 0x7f82c41565c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f82b000d800 tx=0x7f82b000dcc0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.662+0000 7f82cac00640 1 -- 172.21.15.159:0/4083923888 shutdown_connections 2024-04-04T06:11:40.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.662+0000 7f82cac00640 1 --2- 172.21.15.159:0/4083923888 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82c4141e40 0x7f82c40b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.662+0000 7f82cac00640 1 --2- 172.21.15.159:0/4083923888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82c41419e0 0x7f82c41565c0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.662+0000 7f82cac00640 1 --2- 172.21.15.159:0/4083923888 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f82a406c8f0 0x7f82a406edb0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.662+0000 7f82cac00640 1 --2- 172.21.15.159:0/4083923888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c413f300 0x7f82c40b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.662+0000 7f82cac00640 1 -- 172.21.15.159:0/4083923888 >> 172.21.15.159:0/4083923888 conn(0x7f82c406b570 msgr2=0x7f82c4141030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:40.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.663+0000 7f82cac00640 1 -- 172.21.15.159:0/4083923888 shutdown_connections 2024-04-04T06:11:40.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.663+0000 7f82cac00640 1 -- 172.21.15.159:0/4083923888 wait complete. 2024-04-04T06:11:40.674 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-04T06:11:40.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:11:40.675 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-04-04T06:11:40.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.891+0000 7fef7fe00640 1 Processor -- start 2024-04-04T06:11:40.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.891+0000 7fef7fe00640 1 -- start start 2024-04-04T06:11:40.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.892+0000 7fef7fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fef8009ece0 0x7fef8006b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.893+0000 7fef7fe00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef800ae6d0 con 0x7fef8006bab0 2024-04-04T06:11:40.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.893+0000 7fef7fe00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef8015ff20 con 0x7fef80145bc0 2024-04-04T06:11:40.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.893+0000 7fef7fe00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fef80160200 con 0x7fef8009ece0 2024-04-04T06:11:40.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.893+0000 7fef7f400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fef8006bab0 0x7fef800723d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:59720/0 (socket says 172.21.15.159:59720) 2024-04-04T06:11:40.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.893+0000 7fef7f400640 1 -- 172.21.15.159:0/3150721760 learned_addr learned my addr 172.21.15.159:0/3150721760 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:11:40.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.893+0000 7fef77e00640 1 --2- 172.21.15.159:0/3150721760 >> v2:172.21.15.159:3300/0 conn(0x7fef8009ece0 0x7fef8006b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.893+0000 7fef7e000640 1 -- 172.21.15.159:0/3150721760 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2289615856 0 0) 0x7fef78001490 con 0x7fef8006bab0 2024-04-04T06:11:40.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.893+0000 7fef7e000640 1 -- 172.21.15.159:0/3150721760 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef64003520 con 0x7fef8006bab0 2024-04-04T06:11:40.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.894+0000 7fef77e00640 1 -- 172.21.15.159:0/3150721760 >> v1:172.21.15.178:6789/0 conn(0x7fef80145bc0 legacy=0x7fef800addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.894+0000 7fef77e00640 1 -- 172.21.15.159:0/3150721760 >> v1:172.21.15.159:6789/0 conn(0x7fef8006bab0 legacy=0x7fef800723d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.894+0000 7fef77e00640 1 -- 172.21.15.159:0/3150721760 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef801604e0 con 0x7fef8009ece0 2024-04-04T06:11:40.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.895+0000 7fef77e00640 1 --2- 172.21.15.159:0/3150721760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef8009ece0 0x7fef8006b570 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fef680097a0 tx=0x7fef6802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9874e789e480999 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:40.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.896+0000 7fef7e000640 1 -- 172.21.15.159:0/3150721760 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef680315a0 con 0x7fef8009ece0 2024-04-04T06:11:40.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.896+0000 7fef7e000640 1 -- 172.21.15.159:0/3150721760 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef68037070 con 0x7fef8009ece0 2024-04-04T06:11:40.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.896+0000 7fef7e000640 1 -- 172.21.15.159:0/3150721760 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68031960 con 0x7fef8009ece0 2024-04-04T06:11:40.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.896+0000 7fef7fe00640 1 -- 172.21.15.159:0/3150721760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef8009ece0 msgr2=0x7fef8006b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.896+0000 7fef7fe00640 1 --2- 172.21.15.159:0/3150721760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef8009ece0 0x7fef8006b570 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fef680097a0 tx=0x7fef6802b840 comp rx=0 tx=0).stop 2024-04-04T06:11:40.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.897+0000 7fef7fe00640 1 -- 172.21.15.159:0/3150721760 shutdown_connections 2024-04-04T06:11:40.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.897+0000 7fef7fe00640 1 --2- 172.21.15.159:0/3150721760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef8009ece0 0x7fef8006b570 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.897+0000 7fef7fe00640 1 -- 172.21.15.159:0/3150721760 >> 172.21.15.159:0/3150721760 conn(0x7fef800b2af0 msgr2=0x7fef800b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:40.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.897+0000 7fef7fe00640 1 -- 172.21.15.159:0/3150721760 shutdown_connections 2024-04-04T06:11:40.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.897+0000 7fef7fe00640 1 -- 172.21.15.159:0/3150721760 wait complete. 2024-04-04T06:11:40.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.898+0000 7fef7fe00640 1 Processor -- start 2024-04-04T06:11:40.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.898+0000 7fef7fe00640 1 -- start start 2024-04-04T06:11:40.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.899+0000 7fef7fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef8006bab0 0x7fef801542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.899+0000 7fef7fe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef8009ece0 0x7fef801447a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.899+0000 7fef7fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef80145bc0 0x7fef80144ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.899+0000 7fef7fe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fef80072ae0 con 0x7fef8006bab0 2024-04-04T06:11:40.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.899+0000 7fef7fe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fef8006bee0 con 0x7fef80145bc0 2024-04-04T06:11:40.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.899+0000 7fef7fe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fef80072c60 con 0x7fef8009ece0 2024-04-04T06:11:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.900+0000 7fef7ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef8006bab0 0x7fef801542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.900+0000 7fef7ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef8006bab0 0x7fef801542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37894/0 (socket says 172.21.15.159:37894) 2024-04-04T06:11:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.900+0000 7fef7ea00640 1 -- 172.21.15.159:0/2315138182 learned_addr learned my addr 172.21.15.159:0/2315138182 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.900+0000 7fef77e00640 1 --2- 172.21.15.159:0/2315138182 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef8009ece0 0x7fef801447a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.900+0000 7fef7f400640 1 --2- 172.21.15.159:0/2315138182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef80145bc0 0x7fef80144ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.900+0000 7fef7ea00640 1 -- 172.21.15.159:0/2315138182 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef8009ece0 msgr2=0x7fef801447a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.900+0000 7fef7ea00640 1 --2- 172.21.15.159:0/2315138182 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef8009ece0 0x7fef801447a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.900+0000 7fef7ea00640 1 -- 172.21.15.159:0/2315138182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef80145bc0 msgr2=0x7fef80144ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.900+0000 7fef7ea00640 1 --2- 172.21.15.159:0/2315138182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef80145bc0 0x7fef80144ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.900+0000 7fef7ea00640 1 -- 172.21.15.159:0/2315138182 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8009d270 con 0x7fef8006bab0 2024-04-04T06:11:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.900+0000 7fef7f400640 1 --2- 172.21.15.159:0/2315138182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef80145bc0 0x7fef80144ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:11:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.901+0000 7fef7ea00640 1 --2- 172.21.15.159:0/2315138182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef8006bab0 0x7fef801542b0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fef7000c400 tx=0x7fef7000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:40.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.901+0000 7fef76a00640 1 -- 172.21.15.159:0/2315138182 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70017070 con 0x7fef8006bab0 2024-04-04T06:11:40.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.901+0000 7fef76a00640 1 -- 172.21.15.159:0/2315138182 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef700052d0 con 0x7fef8006bab0 2024-04-04T06:11:40.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.901+0000 7fef7fe00640 1 -- 172.21.15.159:0/2315138182 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8009d550 con 0x7fef8006bab0 2024-04-04T06:11:40.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.901+0000 7fef76a00640 1 -- 172.21.15.159:0/2315138182 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70003830 con 0x7fef8006bab0 2024-04-04T06:11:40.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.901+0000 7fef7fe00640 1 -- 172.21.15.159:0/2315138182 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef8009db10 con 0x7fef8006bab0 2024-04-04T06:11:40.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.903+0000 7fef76a00640 1 -- 172.21.15.159:0/2315138182 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef7001a070 con 0x7fef8006bab0 2024-04-04T06:11:40.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.904+0000 7fef76a00640 1 --2- 172.21.15.159:0/2315138182 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fef5c06c8f0 0x7fef5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.905+0000 7fef76a00640 1 -- 172.21.15.159:0/2315138182 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7fef7008afe0 con 0x7fef8006bab0 2024-04-04T06:11:40.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.905+0000 7fef77e00640 1 --2- 172.21.15.159:0/2315138182 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fef5c06c8f0 0x7fef5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.905+0000 7fef88468740 1 Processor -- start 2024-04-04T06:11:40.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.906+0000 7fef88468740 1 -- start start 2024-04-04T06:11:40.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.906+0000 7fef77e00640 1 --2- 172.21.15.159:0/2315138182 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fef5c06c8f0 0x7fef5c06edb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fef68036000 tx=0x7fef6803d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:40.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.906+0000 7fef88468740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf544d5220 0x55cf544a81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.907+0000 7fef88468740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5600 0x55cf544d7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.907+0000 7fef88468740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf544a8700 0x55cf544d84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.907+0000 7fef88468740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cf544114a0 con 0x55cf544d5600 2024-04-04T06:11:40.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.907+0000 7fef88468740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cf543b9810 con 0x55cf544a8700 2024-04-04T06:11:40.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.907+0000 7fef88468740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55cf5440d450 con 0x55cf544d5220 2024-04-04T06:11:40.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.907+0000 7fef77e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5600 0x55cf544d7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.907+0000 7fef77e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5600 0x55cf544d7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37908/0 (socket says 172.21.15.159:37908) 2024-04-04T06:11:40.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.907+0000 7fef77e00640 1 -- 172.21.15.159:0/2876237818 learned_addr learned my addr 172.21.15.159:0/2876237818 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:40.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.907+0000 7fef7ea00640 1 --2- 172.21.15.159:0/2876237818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf544d5220 0x55cf544a81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.907+0000 7fef7f400640 1 --2- 172.21.15.159:0/2876237818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf544a8700 0x55cf544d84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.907+0000 7fef77e00640 1 -- 172.21.15.159:0/2876237818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf544d5220 msgr2=0x55cf544a81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:40.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.907+0000 7fef77e00640 1 --2- 172.21.15.159:0/2876237818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf544d5220 0x55cf544a81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.907+0000 7fef77e00640 1 -- 172.21.15.159:0/2876237818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf544a8700 msgr2=0x55cf544d84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:40.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.907+0000 7fef77e00640 1 --2- 172.21.15.159:0/2876237818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf544a8700 0x55cf544d84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.907+0000 7fef77e00640 1 -- 172.21.15.159:0/2876237818 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf541eda50 con 0x55cf544d5600 2024-04-04T06:11:40.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.908+0000 7fef77e00640 1 --2- 172.21.15.159:0/2876237818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5600 0x55cf544d7fb0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fef6802be50 tx=0x7fef68030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b921ddbc58445914 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:40.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.908+0000 7fef6f400640 1 -- 172.21.15.159:0/2876237818 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68032a80 con 0x55cf544d5600 2024-04-04T06:11:40.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.908+0000 7fef6f400640 1 -- 172.21.15.159:0/2876237818 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef68037040 con 0x55cf544d5600 2024-04-04T06:11:40.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.908+0000 7fef6f400640 1 -- 172.21.15.159:0/2876237818 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68032e40 con 0x55cf544d5600 2024-04-04T06:11:40.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.909+0000 7fef88468740 1 -- 172.21.15.159:0/2876237818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5600 msgr2=0x55cf544d7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.909+0000 7fef88468740 1 --2- 172.21.15.159:0/2876237818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5600 0x55cf544d7fb0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fef6802be50 tx=0x7fef68030000 comp rx=0 tx=0).stop 2024-04-04T06:11:40.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.909+0000 7fef88468740 1 -- 172.21.15.159:0/2876237818 shutdown_connections 2024-04-04T06:11:40.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.909+0000 7fef88468740 1 --2- 172.21.15.159:0/2876237818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf544a8700 0x55cf544d84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.909+0000 7fef88468740 1 --2- 172.21.15.159:0/2876237818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5600 0x55cf544d7fb0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.909+0000 7fef88468740 1 --2- 172.21.15.159:0/2876237818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf544d5220 0x55cf544a81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.909+0000 7fef88468740 1 -- 172.21.15.159:0/2876237818 >> 172.21.15.159:0/2876237818 conn(0x55cf544aa430 msgr2=0x55cf544f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:40.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.909+0000 7fef88468740 1 -- 172.21.15.159:0/2876237818 shutdown_connections 2024-04-04T06:11:40.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.909+0000 7fef88468740 1 -- 172.21.15.159:0/2876237818 wait complete. 2024-04-04T06:11:40.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.909+0000 7fef88468740 1 -- start start 2024-04-04T06:11:40.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.910+0000 7fef88468740 1 Processor -- start 2024-04-04T06:11:40.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.911+0000 7fef88468740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf544d5220 0x55cf544d9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.911+0000 7fef88468740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5600 0x55cf544f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.911+0000 7fef88468740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf54503360 0x55cf544deb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.911+0000 7fef88468740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cf544acad0 con 0x55cf544d5600 2024-04-04T06:11:40.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.911+0000 7fef88468740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cf543ccbc0 con 0x55cf544d5220 2024-04-04T06:11:40.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.911+0000 7fef88468740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55cf544af310 con 0x55cf54503360 2024-04-04T06:11:40.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.911+0000 7fef88468740 10 client.0 _ll_register_callbacks cb 0x55cf543bace0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:11:40.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.911+0000 7fef77e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5600 0x55cf544f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.911+0000 7fef77e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5600 0x55cf544f35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37916/0 (socket says 172.21.15.159:37916) 2024-04-04T06:11:40.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.911+0000 7fef77e00640 1 -- 172.21.15.159:0/1815374244 learned_addr learned my addr 172.21.15.159:0/1815374244 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:40.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.911+0000 7fef7ea00640 1 --2- 172.21.15.159:0/1815374244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf544d5220 0x55cf544d9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.911+0000 7fef77e00640 1 -- 172.21.15.159:0/1815374244 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf54503360 msgr2=0x55cf544deb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:40.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.911+0000 7fef77e00640 1 --2- 172.21.15.159:0/1815374244 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf54503360 0x55cf544deb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.911+0000 7fef77e00640 1 -- 172.21.15.159:0/1815374244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf544d5220 msgr2=0x55cf544d9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:40.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.911+0000 7fef77e00640 1 --2- 172.21.15.159:0/1815374244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf544d5220 0x55cf544d9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.911+0000 7fef77e00640 1 -- 172.21.15.159:0/1815374244 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf540e53e0 con 0x55cf544d5600 2024-04-04T06:11:40.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.912+0000 7fef77e00640 1 --2- 172.21.15.159:0/1815374244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5600 0x55cf544f35d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fef68000c00 tx=0x7fef68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6464f828b057af8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:40.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.912+0000 7fef63e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:11:40.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.912+0000 7fef63e00640 1 -- 172.21.15.159:0/1815374244 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68050890 con 0x55cf544d5600 2024-04-04T06:11:40.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.912+0000 7fef63e00640 1 -- 172.21.15.159:0/1815374244 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7fef6805cb40 con 0x55cf544d5600 2024-04-04T06:11:40.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.912+0000 7fef88468740 1 -- 172.21.15.159:0/1815374244 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef68037040 con 0x55cf544d5600 2024-04-04T06:11:40.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.912+0000 7fef63e00640 1 -- 172.21.15.159:0/1815374244 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fef68063050 con 0x55cf544d5600 2024-04-04T06:11:40.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.912+0000 7fef88468740 10 client.7020 fetch_fsmap learned FSMap version 52 2024-04-04T06:11:40.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.912+0000 7fef88468740 1 -- 172.21.15.159:0/1815374244 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55cf5424e1e0 con 0x55cf544d5600 2024-04-04T06:11:40.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.912+0000 7fef63e00640 1 -- 172.21.15.159:0/1815374244 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef68037040 con 0x55cf544d5600 2024-04-04T06:11:40.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.913+0000 7fef63e00640 1 -- 172.21.15.159:0/1815374244 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7fef68066020 con 0x55cf544d5600 2024-04-04T06:11:40.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.913+0000 7fef88468740 10 client.7020 fetch_fsmap finished waiting for FSMap version 52 2024-04-04T06:11:40.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.913+0000 7fef88468740 10 client.7020 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-04T06:11:40.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.913+0000 7fef88468740 20 client.7020 populate_metadata read hostname 'smithi159' 2024-04-04T06:11:40.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.913+0000 7fef88468740 1 --2- 172.21.15.159:0/1815374244 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x55cf54506cd0 0x55cf54507110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.913+0000 7fef88468740 4 client.7020 mds_command: new command op to 6828 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:11:40.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.913+0000 7fef88468740 1 -- 172.21.15.159:0/1815374244 --> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf543d3e10 con 0x55cf54506cd0 2024-04-04T06:11:40.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.913+0000 7fef7ea00640 1 --2- 172.21.15.159:0/1815374244 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x55cf54506cd0 0x55cf54507110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.913+0000 7fef7ea00640 1 --2- 172.21.15.159:0/1815374244 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x55cf54506cd0 0x55cf54507110 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac6b091c9c9cf5c9 server_cookie=d8b0c65c189dd95a in_seq=0 out_seq=0 2024-04-04T06:11:40.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.913+0000 7fef63e00640 10 client.7020 ms_handle_connect on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:40.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.914+0000 7fef63e00640 1 -- 172.21.15.159:0/1815374244 <== mds.0 v2:172.21.15.159:6834/2467594024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf543d3e10 con 0x55cf54506cd0 2024-04-04T06:11:40.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.914+0000 7fef63e00640 10 client.7020 handle_command_reply: tid=0 2024-04-04T06:11:40.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.914+0000 7fef88468740 1 client.7020 shutdown 2024-04-04T06:11:40.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.914+0000 7fef88468740 1 -- 172.21.15.159:0/1815374244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5600 msgr2=0x55cf544f35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.914+0000 7fef88468740 1 --2- 172.21.15.159:0/1815374244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5600 0x55cf544f35d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fef68000c00 tx=0x7fef68004730 comp rx=0 tx=0).stop 2024-04-04T06:11:40.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.914+0000 7fef88468740 1 -- 172.21.15.159:0/1815374244 shutdown_connections 2024-04-04T06:11:40.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.914+0000 7fef88468740 1 --2- 172.21.15.159:0/1815374244 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x55cf54506cd0 0x55cf54507110 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.914+0000 7fef88468740 1 --2- 172.21.15.159:0/1815374244 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf54503360 0x55cf544deb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.914+0000 7fef88468740 1 --2- 172.21.15.159:0/1815374244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5600 0x55cf544f35d0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.914+0000 7fef88468740 1 --2- 172.21.15.159:0/1815374244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf544d5220 0x55cf544d9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.914+0000 7fef63e00640 0 client.7020 ms_handle_reset on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:40.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.914+0000 7fef88468740 1 -- 172.21.15.159:0/1815374244 >> 172.21.15.159:0/1815374244 conn(0x55cf544a8700 msgr2=0x55cf544ac1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:40.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.915+0000 7fef88468740 1 -- 172.21.15.159:0/1815374244 shutdown_connections 2024-04-04T06:11:40.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.915+0000 7fef88468740 1 -- 172.21.15.159:0/1815374244 wait complete. 2024-04-04T06:11:40.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.915+0000 7fef88468740 20 client.7020 trim_cache size 0 max 16384 2024-04-04T06:11:40.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.934+0000 7fef88468740 1 Processor -- start 2024-04-04T06:11:40.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.934+0000 7fef88468740 1 -- start start 2024-04-04T06:11:40.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.934+0000 7fef88468740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5220 0x55cf544d3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.935+0000 7fef88468740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf544d5600 0x55cf544de510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.935+0000 7fef88468740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf54503360 0x55cf544dea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.935+0000 7fef88468740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fef80200a90 con 0x55cf544d5220 2024-04-04T06:11:40.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.935+0000 7fef88468740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fef801ffaf0 con 0x55cf54503360 2024-04-04T06:11:40.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.935+0000 7fef88468740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fef80155880 con 0x55cf544d5600 2024-04-04T06:11:40.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.935+0000 7fef7ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5220 0x55cf544d3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.935+0000 7fef7ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5220 0x55cf544d3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37922/0 (socket says 172.21.15.159:37922) 2024-04-04T06:11:40.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.935+0000 7fef7ea00640 1 -- 172.21.15.159:0/3222414457 learned_addr learned my addr 172.21.15.159:0/3222414457 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:40.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.935+0000 7fef7ea00640 1 -- 172.21.15.159:0/3222414457 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf544d5600 msgr2=0x55cf544de510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:40.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.935+0000 7fef7f400640 1 --2- 172.21.15.159:0/3222414457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf54503360 0x55cf544dea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.935+0000 7fef77e00640 1 --2- 172.21.15.159:0/3222414457 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf544d5600 0x55cf544de510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.935+0000 7fef7ea00640 1 --2- 172.21.15.159:0/3222414457 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf544d5600 0x55cf544de510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.935+0000 7fef7ea00640 1 -- 172.21.15.159:0/3222414457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf54503360 msgr2=0x55cf544dea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:40.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.935+0000 7fef7ea00640 1 --2- 172.21.15.159:0/3222414457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf54503360 0x55cf544dea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.935+0000 7fef7ea00640 1 -- 172.21.15.159:0/3222414457 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf540be4a0 con 0x55cf544d5220 2024-04-04T06:11:40.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.936+0000 7fef7ea00640 1 --2- 172.21.15.159:0/3222414457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5220 0x55cf544d3190 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fef700588e0 tx=0x7fef70001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e87419149ba67b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:40.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.936+0000 7fef6ea00640 1 -- 172.21.15.159:0/3222414457 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70071020 con 0x55cf544d5220 2024-04-04T06:11:40.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.936+0000 7fef6ea00640 1 -- 172.21.15.159:0/3222414457 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef700894f0 con 0x55cf544d5220 2024-04-04T06:11:40.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.936+0000 7fef6ea00640 1 -- 172.21.15.159:0/3222414457 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef700711c0 con 0x55cf544d5220 2024-04-04T06:11:40.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.936+0000 7fef88468740 1 -- 172.21.15.159:0/3222414457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5220 msgr2=0x55cf544d3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.936+0000 7fef88468740 1 --2- 172.21.15.159:0/3222414457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5220 0x55cf544d3190 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fef700588e0 tx=0x7fef70001080 comp rx=0 tx=0).stop 2024-04-04T06:11:40.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.937+0000 7fef88468740 1 -- 172.21.15.159:0/3222414457 shutdown_connections 2024-04-04T06:11:40.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.937+0000 7fef88468740 1 --2- 172.21.15.159:0/3222414457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf54503360 0x55cf544dea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.937+0000 7fef88468740 1 --2- 172.21.15.159:0/3222414457 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf544d5600 0x55cf544de510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.937+0000 7fef88468740 1 --2- 172.21.15.159:0/3222414457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5220 0x55cf544d3190 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.937+0000 7fef88468740 1 -- 172.21.15.159:0/3222414457 >> 172.21.15.159:0/3222414457 conn(0x55cf544a8700 msgr2=0x55cf544d8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:40.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.937+0000 7fef88468740 1 -- 172.21.15.159:0/3222414457 shutdown_connections 2024-04-04T06:11:40.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.937+0000 7fef88468740 1 -- 172.21.15.159:0/3222414457 wait complete. 2024-04-04T06:11:40.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.937+0000 7fef88468740 1 -- start start 2024-04-04T06:11:40.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.938+0000 7fef88468740 1 Processor -- start 2024-04-04T06:11:40.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.939+0000 7fef88468740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf544d5220 0x55cf54555de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.939+0000 7fef88468740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5600 0x55cf544de510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.939+0000 7fef88468740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf54503360 0x55cf544dea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.939+0000 7fef88468740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cf544a9d30 con 0x55cf544d5600 2024-04-04T06:11:40.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.939+0000 7fef88468740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fef80155ed0 con 0x55cf54503360 2024-04-04T06:11:40.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.939+0000 7fef88468740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55cf54504090 con 0x55cf544d5220 2024-04-04T06:11:40.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.939+0000 7fef88468740 10 client.0 _ll_register_callbacks cb 0x7fef68009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:11:40.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.939+0000 7fef77e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5600 0x55cf544de510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.939+0000 7fef77e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5600 0x55cf544de510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37938/0 (socket says 172.21.15.159:37938) 2024-04-04T06:11:40.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.939+0000 7fef77e00640 1 -- 172.21.15.159:0/1557468946 learned_addr learned my addr 172.21.15.159:0/1557468946 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:40.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.939+0000 7fef7f400640 1 --2- 172.21.15.159:0/1557468946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf54503360 0x55cf544dea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.939+0000 7fef77e00640 1 -- 172.21.15.159:0/1557468946 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf544d5220 msgr2=0x55cf54555de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:40.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.939+0000 7fef77e00640 1 --2- 172.21.15.159:0/1557468946 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf544d5220 0x55cf54555de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.939+0000 7fef77e00640 1 -- 172.21.15.159:0/1557468946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf54503360 msgr2=0x55cf544dea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:40.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.939+0000 7fef77e00640 1 --2- 172.21.15.159:0/1557468946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf54503360 0x55cf544dea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.939+0000 7fef77e00640 1 -- 172.21.15.159:0/1557468946 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf5456d4a0 con 0x55cf544d5600 2024-04-04T06:11:40.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.939+0000 7fef7f400640 1 --2- 172.21.15.159:0/1557468946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf54503360 0x55cf544dea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:11:40.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.940+0000 7fef77e00640 1 --2- 172.21.15.159:0/1557468946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5600 0x55cf544de510 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fef68009560 tx=0x7fef68008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35bfe625bc3eb36b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:40.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.940+0000 7fef63e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:11:40.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.940+0000 7fef63e00640 1 -- 172.21.15.159:0/1557468946 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68046810 con 0x55cf544d5600 2024-04-04T06:11:40.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.940+0000 7fef63e00640 1 -- 172.21.15.159:0/1557468946 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7fef680469b0 con 0x55cf544d5600 2024-04-04T06:11:40.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.940+0000 7fef88468740 1 -- 172.21.15.159:0/1557468946 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef700894f0 con 0x55cf544d5600 2024-04-04T06:11:40.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.940+0000 7fef63e00640 1 -- 172.21.15.159:0/1557468946 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fef6806f050 con 0x55cf544d5600 2024-04-04T06:11:40.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.940+0000 7fef88468740 10 client.7026 fetch_fsmap learned FSMap version 52 2024-04-04T06:11:40.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.940+0000 7fef88468740 1 -- 172.21.15.159:0/1557468946 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55cf544d4720 con 0x55cf544d5600 2024-04-04T06:11:40.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.940+0000 7fef63e00640 1 -- 172.21.15.159:0/1557468946 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef700894f0 con 0x55cf544d5600 2024-04-04T06:11:40.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.941+0000 7fef63e00640 1 -- 172.21.15.159:0/1557468946 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7fef68072020 con 0x55cf544d5600 2024-04-04T06:11:40.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.941+0000 7fef88468740 10 client.7026 fetch_fsmap finished waiting for FSMap version 52 2024-04-04T06:11:40.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.941+0000 7fef88468740 10 client.7026 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-04T06:11:40.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.941+0000 7fef88468740 20 client.7026 populate_metadata read hostname 'smithi159' 2024-04-04T06:11:40.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.941+0000 7fef88468740 1 --2- 172.21.15.159:0/1557468946 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x55cf544da370 0x55cf544d38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:40.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.941+0000 7fef88468740 4 client.7026 mds_command: new command op to 6828 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-04-04T06:11:40.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.941+0000 7fef88468740 1 -- 172.21.15.159:0/1557468946 --> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) -- 0x55cf545031b0 con 0x55cf544da370 2024-04-04T06:11:40.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.941+0000 7fef7ea00640 1 --2- 172.21.15.159:0/1557468946 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x55cf544da370 0x55cf544d38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:40.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.941+0000 7fef7ea00640 1 --2- 172.21.15.159:0/1557468946 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x55cf544da370 0x55cf544d38d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90a957fc78fc046b server_cookie=4633da5953b9f755 in_seq=0 out_seq=0 2024-04-04T06:11:40.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.941+0000 7fef63e00640 10 client.7026 ms_handle_connect on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:40.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.942+0000 7fef63e00640 1 -- 172.21.15.159:0/1557468946 <== mds.0 v2:172.21.15.159:6834/2467594024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cf545031b0 con 0x55cf544da370 2024-04-04T06:11:40.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.942+0000 7fef63e00640 10 client.7026 handle_command_reply: tid=0 2024-04-04T06:11:40.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.942+0000 7fef88468740 1 client.7026 shutdown 2024-04-04T06:11:40.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.942+0000 7fef88468740 1 -- 172.21.15.159:0/1557468946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5600 msgr2=0x55cf544de510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.942+0000 7fef88468740 1 --2- 172.21.15.159:0/1557468946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5600 0x55cf544de510 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fef68009560 tx=0x7fef68008030 comp rx=0 tx=0).stop 2024-04-04T06:11:40.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.942+0000 7fef88468740 1 -- 172.21.15.159:0/1557468946 shutdown_connections 2024-04-04T06:11:40.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.942+0000 7fef88468740 1 --2- 172.21.15.159:0/1557468946 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x55cf544da370 0x55cf544d38d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.942+0000 7fef88468740 1 --2- 172.21.15.159:0/1557468946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf54503360 0x55cf544dea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.942+0000 7fef88468740 1 --2- 172.21.15.159:0/1557468946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf544d5600 0x55cf544de510 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.942+0000 7fef88468740 1 --2- 172.21.15.159:0/1557468946 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf544d5220 0x55cf54555de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.942+0000 7fef63e00640 0 client.7026 ms_handle_reset on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:40.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.942+0000 7fef88468740 1 -- 172.21.15.159:0/1557468946 >> 172.21.15.159:0/1557468946 conn(0x55cf544a8700 msgr2=0x55cf544f64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:40.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.943+0000 7fef88468740 1 -- 172.21.15.159:0/1557468946 shutdown_connections 2024-04-04T06:11:40.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.943+0000 7fef88468740 1 -- 172.21.15.159:0/1557468946 wait complete. 2024-04-04T06:11:40.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.943+0000 7fef88468740 20 client.7026 trim_cache size 0 max 16384 2024-04-04T06:11:40.941 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:11:40.941 INFO:teuthology.orchestra.run.smithi159.stdout: "return_code": 0, 2024-04-04T06:11:40.941 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_tag": "c41e99a1-87f4-4cfc-a464-b3b1d1c16528", 2024-04-04T06:11:40.941 INFO:teuthology.orchestra.run.smithi159.stdout: "mode": "asynchronous" 2024-04-04T06:11:40.941 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:11:40.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.943+0000 7fef7fe00640 1 -- 172.21.15.159:0/2315138182 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fef5c06c8f0 msgr2=0x7fef5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.943+0000 7fef7fe00640 1 --2- 172.21.15.159:0/2315138182 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fef5c06c8f0 0x7fef5c06edb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fef68036000 tx=0x7fef6803d040 comp rx=0 tx=0).stop 2024-04-04T06:11:40.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.943+0000 7fef7fe00640 1 -- 172.21.15.159:0/2315138182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef8006bab0 msgr2=0x7fef801542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:40.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.943+0000 7fef7fe00640 1 --2- 172.21.15.159:0/2315138182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef8006bab0 0x7fef801542b0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fef7000c400 tx=0x7fef7000c8c0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.944+0000 7fef7fe00640 1 -- 172.21.15.159:0/2315138182 shutdown_connections 2024-04-04T06:11:40.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.944+0000 7fef7fe00640 1 --2- 172.21.15.159:0/2315138182 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fef5c06c8f0 0x7fef5c06edb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.944+0000 7fef7fe00640 1 --2- 172.21.15.159:0/2315138182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef80145bc0 0x7fef80144ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.944+0000 7fef7fe00640 1 --2- 172.21.15.159:0/2315138182 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef8009ece0 0x7fef801447a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.944+0000 7fef7fe00640 1 --2- 172.21.15.159:0/2315138182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef8006bab0 0x7fef801542b0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:40.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.944+0000 7fef7fe00640 1 -- 172.21.15.159:0/2315138182 >> 172.21.15.159:0/2315138182 conn(0x7fef800b2af0 msgr2=0x7fef80073020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:40.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.944+0000 7fef7fe00640 1 -- 172.21.15.159:0/2315138182 shutdown_connections 2024-04-04T06:11:40.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:40.944+0000 7fef7fe00640 1 -- 172.21.15.159:0/2315138182 wait complete. 2024-04-04T06:11:40.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:11:40.956 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-04T06:11:41.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.178+0000 7effaea00640 1 Processor -- start 2024-04-04T06:11:41.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.178+0000 7effaea00640 1 -- start start 2024-04-04T06:11:41.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.180+0000 7effaea00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7effa80d0280 0x7effa80d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:41.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.180+0000 7effaea00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7effa8141c10 0x7effa8071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:41.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.180+0000 7effaea00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effa8071eb0 0x7effa806d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:41.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.180+0000 7effaea00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7effa815ec80 con 0x7effa80d0280 2024-04-04T06:11:41.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.180+0000 7effaea00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7effa80d0ba0 con 0x7effa8141c10 2024-04-04T06:11:41.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.180+0000 7effaea00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7effa815ef20 con 0x7effa8071eb0 2024-04-04T06:11:41.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.180+0000 7effa7e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7effa80d0280 0x7effa80d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:41.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.180+0000 7effa7e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7effa80d0280 0x7effa80d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37940/0 (socket says 172.21.15.159:37940) 2024-04-04T06:11:41.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.180+0000 7effa7e00640 1 -- 172.21.15.159:0/3528114135 learned_addr learned my addr 172.21.15.159:0/3528114135 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:41.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.180+0000 7eff9fe00640 1 --2- 172.21.15.159:0/3528114135 >> v2:172.21.15.178:3300/0 conn(0x7effa8141c10 0x7effa8071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:41.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.180+0000 7effa7e00640 1 -- 172.21.15.159:0/3528114135 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effa8071eb0 msgr2=0x7effa806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:41.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.180+0000 7effaca00640 1 --2- 172.21.15.159:0/3528114135 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effa8071eb0 0x7effa806d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:41.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.181+0000 7effa7e00640 1 --2- 172.21.15.159:0/3528114135 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effa8071eb0 0x7effa806d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.181+0000 7effa7e00640 1 -- 172.21.15.159:0/3528114135 >> v2:172.21.15.178:3300/0 conn(0x7effa8141c10 msgr2=0x7effa8071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:41.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.181+0000 7effa7e00640 1 --2- 172.21.15.159:0/3528114135 >> v2:172.21.15.178:3300/0 conn(0x7effa8141c10 0x7effa8071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.181+0000 7effa7e00640 1 -- 172.21.15.159:0/3528114135 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa815f200 con 0x7effa80d0280 2024-04-04T06:11:41.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.181+0000 7eff9fe00640 1 --2- 172.21.15.159:0/3528114135 >> v2:172.21.15.178:3300/0 conn(0x7effa8141c10 0x7effa8071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:11:41.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.181+0000 7effaca00640 1 --2- 172.21.15.159:0/3528114135 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effa8071eb0 0x7effa806d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:11:41.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.182+0000 7effa7e00640 1 --2- 172.21.15.159:0/3528114135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effa80d0280 0x7effa80d0660 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7effa80b5270 tx=0x7eff9802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6895494ea12b2f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:41.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.182+0000 7effa7400640 1 -- 172.21.15.159:0/3528114135 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98035020 con 0x7effa80d0280 2024-04-04T06:11:41.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.182+0000 7effa7400640 1 -- 172.21.15.159:0/3528114135 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff98031880 con 0x7effa80d0280 2024-04-04T06:11:41.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.182+0000 7effa7400640 1 -- 172.21.15.159:0/3528114135 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98031c50 con 0x7effa80d0280 2024-04-04T06:11:41.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.183+0000 7effaea00640 1 -- 172.21.15.159:0/3528114135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effa80d0280 msgr2=0x7effa80d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:41.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.183+0000 7effaea00640 1 --2- 172.21.15.159:0/3528114135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effa80d0280 0x7effa80d0660 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7effa80b5270 tx=0x7eff9802b920 comp rx=0 tx=0).stop 2024-04-04T06:11:41.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.183+0000 7effaea00640 1 -- 172.21.15.159:0/3528114135 shutdown_connections 2024-04-04T06:11:41.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.183+0000 7effaea00640 1 --2- 172.21.15.159:0/3528114135 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effa8071eb0 0x7effa806d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.183+0000 7effaea00640 1 --2- 172.21.15.159:0/3528114135 >> v2:172.21.15.178:3300/0 conn(0x7effa8141c10 0x7effa8071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.183+0000 7effaea00640 1 --2- 172.21.15.159:0/3528114135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effa80d0280 0x7effa80d0660 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.183+0000 7effaea00640 1 -- 172.21.15.159:0/3528114135 >> 172.21.15.159:0/3528114135 conn(0x7effa806b750 msgr2=0x7effa806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:41.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.183+0000 7effaea00640 1 -- 172.21.15.159:0/3528114135 shutdown_connections 2024-04-04T06:11:41.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.183+0000 7effaea00640 1 -- 172.21.15.159:0/3528114135 wait complete. 2024-04-04T06:11:41.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.185+0000 7effaea00640 1 Processor -- start 2024-04-04T06:11:41.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.185+0000 7effaea00640 1 -- start start 2024-04-04T06:11:41.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.186+0000 7effaea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effa8071eb0 0x7effa814ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:41.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.186+0000 7effaea00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effa80d0280 0x7effa80ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:41.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.186+0000 7effaea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effa8141c10 0x7effa80ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:41.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.186+0000 7effaea00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7effa8158e90 con 0x7effa8071eb0 2024-04-04T06:11:41.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.186+0000 7effaea00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7effa815f7c0 con 0x7effa8141c10 2024-04-04T06:11:41.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.186+0000 7effaea00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7effa809ce90 con 0x7effa80d0280 2024-04-04T06:11:41.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.186+0000 7effa7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effa8071eb0 0x7effa814ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:41.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.186+0000 7effa7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effa8071eb0 0x7effa814ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37956/0 (socket says 172.21.15.159:37956) 2024-04-04T06:11:41.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.186+0000 7effa7e00640 1 -- 172.21.15.159:0/919742995 learned_addr learned my addr 172.21.15.159:0/919742995 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:41.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.186+0000 7effa7e00640 1 -- 172.21.15.159:0/919742995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effa80d0280 msgr2=0x7effa80ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:41.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.186+0000 7effa7e00640 1 --2- 172.21.15.159:0/919742995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effa80d0280 0x7effa80ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.186+0000 7effa7e00640 1 -- 172.21.15.159:0/919742995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effa8141c10 msgr2=0x7effa80ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:41.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.186+0000 7effa7e00640 1 --2- 172.21.15.159:0/919742995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effa8141c10 0x7effa80ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.186+0000 7effa7e00640 1 -- 172.21.15.159:0/919742995 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa809d170 con 0x7effa8071eb0 2024-04-04T06:11:41.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.187+0000 7effa7e00640 1 --2- 172.21.15.159:0/919742995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effa8071eb0 0x7effa814ea20 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7eff9802b960 tx=0x7eff980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:41.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.187+0000 7effa4c00640 1 -- 172.21.15.159:0/919742995 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98040070 con 0x7effa8071eb0 2024-04-04T06:11:41.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.187+0000 7effa4c00640 1 -- 172.21.15.159:0/919742995 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff9803a420 con 0x7effa8071eb0 2024-04-04T06:11:41.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.187+0000 7effa4c00640 1 -- 172.21.15.159:0/919742995 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98035040 con 0x7effa8071eb0 2024-04-04T06:11:41.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.187+0000 7effaea00640 1 -- 172.21.15.159:0/919742995 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa809d450 con 0x7effa8071eb0 2024-04-04T06:11:41.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.187+0000 7effaea00640 1 -- 172.21.15.159:0/919742995 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa809da10 con 0x7effa8071eb0 2024-04-04T06:11:41.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.190+0000 7effa4c00640 1 -- 172.21.15.159:0/919742995 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff9804b060 con 0x7effa8071eb0 2024-04-04T06:11:41.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.191+0000 7effa4c00640 1 --2- 172.21.15.159:0/919742995 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7eff8406c8f0 0x7eff8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:41.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.191+0000 7effa4c00640 1 -- 172.21.15.159:0/919742995 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7eff980adbf0 con 0x7effa8071eb0 2024-04-04T06:11:41.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.192+0000 7effb17de740 1 Processor -- start 2024-04-04T06:11:41.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.192+0000 7eff9fe00640 1 --2- 172.21.15.159:0/919742995 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7eff8406c8f0 0x7eff8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:41.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.192+0000 7effb17de740 1 -- start start 2024-04-04T06:11:41.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.193+0000 7effb17de740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560199fce580 0x560199fa1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:41.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.193+0000 7effb17de740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560199fce960 0x560199fd1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:41.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.193+0000 7effb17de740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fa1a60 0x560199fd1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:41.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.193+0000 7effb17de740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560199f0c6d0 con 0x560199fa1a60 2024-04-04T06:11:41.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.193+0000 7effb17de740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560199eb4770 con 0x560199fce960 2024-04-04T06:11:41.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.193+0000 7effb17de740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x560199f08680 con 0x560199fce580 2024-04-04T06:11:41.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.193+0000 7effaca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fa1a60 0x560199fd1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:41.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.193+0000 7eff9fe00640 1 --2- 172.21.15.159:0/919742995 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7eff8406c8f0 0x7eff8406edb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7eff90006fd0 tx=0x7eff90008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:41.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.193+0000 7effaca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fa1a60 0x560199fd1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37970/0 (socket says 172.21.15.159:37970) 2024-04-04T06:11:41.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.193+0000 7effaca00640 1 -- 172.21.15.159:0/3099578292 learned_addr learned my addr 172.21.15.159:0/3099578292 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:41.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.194+0000 7eff9fe00640 1 --2- 172.21.15.159:0/3099578292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560199fce960 0x560199fd1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:41.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.194+0000 7effa7e00640 1 --2- 172.21.15.159:0/3099578292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560199fce580 0x560199fa1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:41.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.194+0000 7effaca00640 1 -- 172.21.15.159:0/3099578292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560199fce580 msgr2=0x560199fa1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:41.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.194+0000 7effaca00640 1 --2- 172.21.15.159:0/3099578292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560199fce580 0x560199fa1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.194+0000 7effaca00640 1 -- 172.21.15.159:0/3099578292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560199fce960 msgr2=0x560199fd1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:41.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.194+0000 7effaca00640 1 --2- 172.21.15.159:0/3099578292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560199fce960 0x560199fd1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.194+0000 7effaca00640 1 -- 172.21.15.159:0/3099578292 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560199cea7d0 con 0x560199fa1a60 2024-04-04T06:11:41.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.194+0000 7effaca00640 1 --2- 172.21.15.159:0/3099578292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fa1a60 0x560199fd1850 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7effa0007e20 tx=0x7effa000c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2f21c4a3e9aba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:41.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.195+0000 7eff97e00640 1 -- 172.21.15.159:0/3099578292 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0003b90 con 0x560199fa1a60 2024-04-04T06:11:41.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.195+0000 7eff97e00640 1 -- 172.21.15.159:0/3099578292 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa0003d50 con 0x560199fa1a60 2024-04-04T06:11:41.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.195+0000 7eff97e00640 1 -- 172.21.15.159:0/3099578292 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0004270 con 0x560199fa1a60 2024-04-04T06:11:41.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.195+0000 7effb17de740 1 -- 172.21.15.159:0/3099578292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fa1a60 msgr2=0x560199fd1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:41.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.195+0000 7effb17de740 1 --2- 172.21.15.159:0/3099578292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fa1a60 0x560199fd1850 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7effa0007e20 tx=0x7effa000c6d0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.195+0000 7effb17de740 1 -- 172.21.15.159:0/3099578292 shutdown_connections 2024-04-04T06:11:41.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.195+0000 7effb17de740 1 --2- 172.21.15.159:0/3099578292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fa1a60 0x560199fd1850 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.195+0000 7effb17de740 1 --2- 172.21.15.159:0/3099578292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560199fce960 0x560199fd1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.195+0000 7effb17de740 1 --2- 172.21.15.159:0/3099578292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560199fce580 0x560199fa1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.195+0000 7effb17de740 1 -- 172.21.15.159:0/3099578292 >> 172.21.15.159:0/3099578292 conn(0x560199fa3790 msgr2=0x560199fec310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:41.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.195+0000 7effb17de740 1 -- 172.21.15.159:0/3099578292 shutdown_connections 2024-04-04T06:11:41.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.196+0000 7effb17de740 1 -- 172.21.15.159:0/3099578292 wait complete. 2024-04-04T06:11:41.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.196+0000 7effb17de740 1 -- start start 2024-04-04T06:11:41.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.196+0000 7effb17de740 1 Processor -- start 2024-04-04T06:11:41.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.197+0000 7effb17de740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560199fce580 0x560199fd1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:41.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.197+0000 7effb17de740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fce960 0x560199fc3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:41.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.197+0000 7effb17de740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560199fc7390 0x560199fd3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:41.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.197+0000 7effb17de740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560199fa5e30 con 0x560199fce960 2024-04-04T06:11:41.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.197+0000 7effb17de740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560199ec7aa0 con 0x560199fc7390 2024-04-04T06:11:41.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.197+0000 7effb17de740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x560199fa8670 con 0x560199fce580 2024-04-04T06:11:41.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.197+0000 7effb17de740 10 client.0 _ll_register_callbacks cb 0x560199ec7490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:11:41.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.198+0000 7eff9fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fce960 0x560199fc3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:41.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.198+0000 7eff9fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fce960 0x560199fc3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37984/0 (socket says 172.21.15.159:37984) 2024-04-04T06:11:41.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.198+0000 7eff9fe00640 1 -- 172.21.15.159:0/90622344 learned_addr learned my addr 172.21.15.159:0/90622344 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:41.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.198+0000 7effaca00640 1 --2- 172.21.15.159:0/90622344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560199fc7390 0x560199fd3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:41.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.198+0000 7eff9fe00640 1 -- 172.21.15.159:0/90622344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560199fce580 msgr2=0x560199fd1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:41.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.198+0000 7eff9fe00640 1 --2- 172.21.15.159:0/90622344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560199fce580 0x560199fd1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.198+0000 7eff9fe00640 1 -- 172.21.15.159:0/90622344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560199fc7390 msgr2=0x560199fd3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:41.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.198+0000 7eff9fe00640 1 --2- 172.21.15.159:0/90622344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560199fc7390 0x560199fd3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.198+0000 7eff9fe00640 1 -- 172.21.15.159:0/90622344 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560199c203f0 con 0x560199fce960 2024-04-04T06:11:41.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.198+0000 7eff9fe00640 1 --2- 172.21.15.159:0/90622344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fce960 0x560199fc3300 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7effa8148260 tx=0x7eff900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ade812ac06bcba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:41.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.198+0000 7eff94c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:11:41.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.199+0000 7eff94c00640 1 -- 172.21.15.159:0/90622344 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9001f650 con 0x560199fce960 2024-04-04T06:11:41.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.199+0000 7eff94c00640 1 -- 172.21.15.159:0/90622344 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7eff9001f7f0 con 0x560199fce960 2024-04-04T06:11:41.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.199+0000 7effb17de740 1 -- 172.21.15.159:0/90622344 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effa0003d50 con 0x560199fce960 2024-04-04T06:11:41.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.199+0000 7eff94c00640 1 -- 172.21.15.159:0/90622344 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7eff9002c050 con 0x560199fce960 2024-04-04T06:11:41.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.199+0000 7effb17de740 10 client.7038 fetch_fsmap learned FSMap version 52 2024-04-04T06:11:41.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.199+0000 7effb17de740 1 -- 172.21.15.159:0/90622344 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x560199be00f0 con 0x560199fce960 2024-04-04T06:11:41.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.199+0000 7eff94c00640 1 -- 172.21.15.159:0/90622344 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa0003d50 con 0x560199fce960 2024-04-04T06:11:41.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.200+0000 7eff94c00640 1 -- 172.21.15.159:0/90622344 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7eff9002f020 con 0x560199fce960 2024-04-04T06:11:41.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.200+0000 7effb17de740 10 client.7038 fetch_fsmap finished waiting for FSMap version 52 2024-04-04T06:11:41.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.200+0000 7effb17de740 10 client.7038 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-04T06:11:41.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.200+0000 7effb17de740 20 client.7038 populate_metadata read hostname 'smithi159' 2024-04-04T06:11:41.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.200+0000 7effb17de740 1 --2- 172.21.15.159:0/90622344 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x560199fc9b10 0x560199fc9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:41.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.200+0000 7effb17de740 4 client.7038 mds_command: new command op to 6828 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:11:41.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.200+0000 7effb17de740 1 -- 172.21.15.159:0/90622344 --> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560199f19c10 con 0x560199fc9b10 2024-04-04T06:11:41.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.200+0000 7effa7e00640 1 --2- 172.21.15.159:0/90622344 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x560199fc9b10 0x560199fc9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:41.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.201+0000 7effa7e00640 1 --2- 172.21.15.159:0/90622344 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x560199fc9b10 0x560199fc9f30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2747dce820d13ea7 server_cookie=44c1dbd413885468 in_seq=0 out_seq=0 2024-04-04T06:11:41.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.201+0000 7eff94c00640 10 client.7038 ms_handle_connect on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:41.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.201+0000 7eff94c00640 1 -- 172.21.15.159:0/90622344 <== mds.0 v2:172.21.15.159:6834/2467594024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560199f19c10 con 0x560199fc9b10 2024-04-04T06:11:41.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.201+0000 7eff94c00640 10 client.7038 handle_command_reply: tid=0 2024-04-04T06:11:41.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.201+0000 7effb17de740 1 client.7038 shutdown 2024-04-04T06:11:41.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.202+0000 7effb17de740 1 -- 172.21.15.159:0/90622344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fce960 msgr2=0x560199fc3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:41.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.202+0000 7effb17de740 1 --2- 172.21.15.159:0/90622344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fce960 0x560199fc3300 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7effa8148260 tx=0x7eff900046c0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.202+0000 7effb17de740 1 -- 172.21.15.159:0/90622344 shutdown_connections 2024-04-04T06:11:41.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.202+0000 7effb17de740 1 --2- 172.21.15.159:0/90622344 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x560199fc9b10 0x560199fc9f30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.202+0000 7effb17de740 1 --2- 172.21.15.159:0/90622344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560199fc7390 0x560199fd3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.202+0000 7effb17de740 1 --2- 172.21.15.159:0/90622344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fce960 0x560199fc3300 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.202+0000 7effb17de740 1 --2- 172.21.15.159:0/90622344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560199fce580 0x560199fd1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.202+0000 7eff94c00640 0 client.7038 ms_handle_reset on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:41.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.202+0000 7effb17de740 1 -- 172.21.15.159:0/90622344 >> 172.21.15.159:0/90622344 conn(0x560199fa1a60 msgr2=0x560199fa5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:41.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.202+0000 7effb17de740 1 -- 172.21.15.159:0/90622344 shutdown_connections 2024-04-04T06:11:41.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.202+0000 7effb17de740 1 -- 172.21.15.159:0/90622344 wait complete. 2024-04-04T06:11:41.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.202+0000 7effb17de740 20 client.7038 trim_cache size 0 max 16384 2024-04-04T06:11:41.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.222+0000 7effb17de740 1 Processor -- start 2024-04-04T06:11:41.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.222+0000 7effb17de740 1 -- start start 2024-04-04T06:11:41.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.223+0000 7effb17de740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560199fc7390 0x560199fcd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:41.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.223+0000 7effb17de740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560199fce580 0x560199fcfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:41.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.223+0000 7effb17de740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fce960 0x560199fa54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:41.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.223+0000 7effb17de740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7effa8200dc0 con 0x560199fce960 2024-04-04T06:11:41.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.223+0000 7effb17de740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7effa81ffe30 con 0x560199fc7390 2024-04-04T06:11:41.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.223+0000 7effb17de740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7effa81457c0 con 0x560199fce580 2024-04-04T06:11:41.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.223+0000 7effaca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fce960 0x560199fa54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:41.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.223+0000 7effaca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fce960 0x560199fa54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37996/0 (socket says 172.21.15.159:37996) 2024-04-04T06:11:41.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.223+0000 7effaca00640 1 -- 172.21.15.159:0/863189537 learned_addr learned my addr 172.21.15.159:0/863189537 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:41.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.224+0000 7effa7e00640 1 --2- 172.21.15.159:0/863189537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560199fc7390 0x560199fcd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:41.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.224+0000 7eff9fe00640 1 --2- 172.21.15.159:0/863189537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560199fce580 0x560199fcfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:41.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.224+0000 7effaca00640 1 -- 172.21.15.159:0/863189537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560199fce580 msgr2=0x560199fcfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:41.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.224+0000 7effaca00640 1 --2- 172.21.15.159:0/863189537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560199fce580 0x560199fcfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.224+0000 7effaca00640 1 -- 172.21.15.159:0/863189537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560199fc7390 msgr2=0x560199fcd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:41.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.224+0000 7effaca00640 1 --2- 172.21.15.159:0/863189537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560199fc7390 0x560199fcd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.224+0000 7effaca00640 1 -- 172.21.15.159:0/863189537 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560199eea500 con 0x560199fce960 2024-04-04T06:11:41.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.224+0000 7effa7e00640 1 --2- 172.21.15.159:0/863189537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560199fc7390 0x560199fcd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:11:41.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.224+0000 7effaca00640 1 --2- 172.21.15.159:0/863189537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fce960 0x560199fa54e0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7effa0007df0 tx=0x7effa000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6060e66d71ee4f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:41.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.225+0000 7eff97400640 1 -- 172.21.15.159:0/863189537 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0029070 con 0x560199fce960 2024-04-04T06:11:41.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.225+0000 7eff97400640 1 -- 172.21.15.159:0/863189537 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa0007a30 con 0x560199fce960 2024-04-04T06:11:41.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.225+0000 7eff97400640 1 -- 172.21.15.159:0/863189537 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0003bd0 con 0x560199fce960 2024-04-04T06:11:41.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.225+0000 7effb17de740 1 -- 172.21.15.159:0/863189537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fce960 msgr2=0x560199fa54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:41.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.225+0000 7effb17de740 1 --2- 172.21.15.159:0/863189537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fce960 0x560199fa54e0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7effa0007df0 tx=0x7effa000a000 comp rx=0 tx=0).stop 2024-04-04T06:11:41.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.225+0000 7effb17de740 1 -- 172.21.15.159:0/863189537 shutdown_connections 2024-04-04T06:11:41.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.225+0000 7effb17de740 1 --2- 172.21.15.159:0/863189537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fce960 0x560199fa54e0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.225+0000 7effb17de740 1 --2- 172.21.15.159:0/863189537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560199fce580 0x560199fcfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.225+0000 7effb17de740 1 --2- 172.21.15.159:0/863189537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560199fc7390 0x560199fcd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.225+0000 7effb17de740 1 -- 172.21.15.159:0/863189537 >> 172.21.15.159:0/863189537 conn(0x560199fa1a60 msgr2=0x560199fd2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:41.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.225+0000 7effb17de740 1 -- 172.21.15.159:0/863189537 shutdown_connections 2024-04-04T06:11:41.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.226+0000 7effb17de740 1 -- 172.21.15.159:0/863189537 wait complete. 2024-04-04T06:11:41.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.226+0000 7effb17de740 1 -- start start 2024-04-04T06:11:41.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.226+0000 7effb17de740 1 Processor -- start 2024-04-04T06:11:41.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.227+0000 7effb17de740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fc7390 0x560199fc32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:41.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.227+0000 7effb17de740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560199fce580 0x56019a04f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:41.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.227+0000 7effb17de740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560199fce960 0x560199fcd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:41.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.227+0000 7effb17de740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560199fa2f50 con 0x560199fc7390 2024-04-04T06:11:41.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.227+0000 7effb17de740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7effa8145e10 con 0x560199fce960 2024-04-04T06:11:41.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.227+0000 7effb17de740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x560199fee340 con 0x560199fce580 2024-04-04T06:11:41.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.227+0000 7effb17de740 10 client.0 _ll_register_callbacks cb 0x7eff9000cb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:11:41.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.228+0000 7effa7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fc7390 0x560199fc32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:41.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.228+0000 7effa7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fc7390 0x560199fc32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38000/0 (socket says 172.21.15.159:38000) 2024-04-04T06:11:41.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.228+0000 7effa7e00640 1 -- 172.21.15.159:0/138738925 learned_addr learned my addr 172.21.15.159:0/138738925 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:11:41.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.228+0000 7effaca00640 1 --2- 172.21.15.159:0/138738925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560199fce960 0x560199fcd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:41.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.228+0000 7eff9fe00640 1 --2- 172.21.15.159:0/138738925 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560199fce580 0x56019a04f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:41.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.228+0000 7effa7e00640 1 -- 172.21.15.159:0/138738925 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560199fce580 msgr2=0x56019a04f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:41.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.228+0000 7effa7e00640 1 --2- 172.21.15.159:0/138738925 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560199fce580 0x56019a04f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.228+0000 7effa7e00640 1 -- 172.21.15.159:0/138738925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560199fce960 msgr2=0x560199fcd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:11:41.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.228+0000 7effa7e00640 1 --2- 172.21.15.159:0/138738925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560199fce960 0x560199fcd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.228+0000 7effa7e00640 1 -- 172.21.15.159:0/138738925 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560199fcc6c0 con 0x560199fc7390 2024-04-04T06:11:41.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.228+0000 7effa7e00640 1 --2- 172.21.15.159:0/138738925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fc7390 0x560199fc32c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7eff9802bf30 tx=0x7eff98030e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5452d2da41d81fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:11:41.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.228+0000 7eff94c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:11:41.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.229+0000 7eff94c00640 1 -- 172.21.15.159:0/138738925 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98084470 con 0x560199fc7390 2024-04-04T06:11:41.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.229+0000 7eff94c00640 1 -- 172.21.15.159:0/138738925 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7eff980820b0 con 0x560199fc7390 2024-04-04T06:11:41.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.229+0000 7effb17de740 1 -- 172.21.15.159:0/138738925 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effa0007a30 con 0x560199fc7390 2024-04-04T06:11:41.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.229+0000 7eff94c00640 1 -- 172.21.15.159:0/138738925 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7eff98031740 con 0x560199fc7390 2024-04-04T06:11:41.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.229+0000 7effb17de740 10 client.7044 fetch_fsmap learned FSMap version 52 2024-04-04T06:11:41.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.229+0000 7effb17de740 1 -- 172.21.15.159:0/138738925 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x560199feff60 con 0x560199fc7390 2024-04-04T06:11:41.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.229+0000 7eff94c00640 1 -- 172.21.15.159:0/138738925 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa0007a30 con 0x560199fc7390 2024-04-04T06:11:41.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.229+0000 7eff94c00640 1 -- 172.21.15.159:0/138738925 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7eff98077f00 con 0x560199fc7390 2024-04-04T06:11:41.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.229+0000 7effb17de740 10 client.7044 fetch_fsmap finished waiting for FSMap version 52 2024-04-04T06:11:41.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.229+0000 7effb17de740 10 client.7044 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-04T06:11:41.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.229+0000 7effb17de740 20 client.7044 populate_metadata read hostname 'smithi159' 2024-04-04T06:11:41.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.229+0000 7effb17de740 1 --2- 172.21.15.159:0/138738925 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x560199fa52b0 0x560199fa56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:11:41.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.229+0000 7effb17de740 4 client.7044 mds_command: new command op to 6828 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-04T06:11:41.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.229+0000 7effb17de740 1 -- 172.21.15.159:0/138738925 --> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560199fc93a0 con 0x560199fa52b0 2024-04-04T06:11:41.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.230+0000 7eff9fe00640 1 --2- 172.21.15.159:0/138738925 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x560199fa52b0 0x560199fa56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:11:41.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.230+0000 7eff94c00640 10 client.7044 ms_handle_connect on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:41.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.230+0000 7eff9fe00640 1 --2- 172.21.15.159:0/138738925 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x560199fa52b0 0x560199fa56f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d50f66f6784b7a4 server_cookie=61f52f454811a5ab in_seq=0 out_seq=0 2024-04-04T06:11:41.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.230+0000 7eff94c00640 1 -- 172.21.15.159:0/138738925 <== mds.0 v2:172.21.15.159:6834/2467594024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x560199fc93a0 con 0x560199fa52b0 2024-04-04T06:11:41.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.230+0000 7eff94c00640 10 client.7044 handle_command_reply: tid=0 2024-04-04T06:11:41.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.231+0000 7effb17de740 1 client.7044 shutdown 2024-04-04T06:11:41.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.231+0000 7effb17de740 1 -- 172.21.15.159:0/138738925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fc7390 msgr2=0x560199fc32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:41.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.231+0000 7effb17de740 1 --2- 172.21.15.159:0/138738925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fc7390 0x560199fc32c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7eff9802bf30 tx=0x7eff98030e90 comp rx=0 tx=0).stop 2024-04-04T06:11:41.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.231+0000 7effb17de740 1 -- 172.21.15.159:0/138738925 shutdown_connections 2024-04-04T06:11:41.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.231+0000 7effb17de740 1 --2- 172.21.15.159:0/138738925 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x560199fa52b0 0x560199fa56f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.231+0000 7effb17de740 1 --2- 172.21.15.159:0/138738925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560199fce960 0x560199fcd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.231+0000 7effb17de740 1 --2- 172.21.15.159:0/138738925 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560199fce580 0x56019a04f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.231+0000 7effb17de740 1 --2- 172.21.15.159:0/138738925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560199fc7390 0x560199fc32c0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.231+0000 7eff94c00640 0 client.7044 ms_handle_reset on v2:172.21.15.159:6834/2467594024 2024-04-04T06:11:41.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.231+0000 7effb17de740 1 -- 172.21.15.159:0/138738925 >> 172.21.15.159:0/138738925 conn(0x560199fa1a60 msgr2=0x560199fcd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:41.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.231+0000 7effb17de740 1 -- 172.21.15.159:0/138738925 shutdown_connections 2024-04-04T06:11:41.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.231+0000 7effb17de740 1 -- 172.21.15.159:0/138738925 wait complete. 2024-04-04T06:11:41.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.231+0000 7effb17de740 20 client.7044 trim_cache size 0 max 16384 2024-04-04T06:11:41.230 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:11:41.230 INFO:teuthology.orchestra.run.smithi159.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-04T06:11:41.230 INFO:teuthology.orchestra.run.smithi159.stdout: "scrubs": { 2024-04-04T06:11:41.230 INFO:teuthology.orchestra.run.smithi159.stdout: "c41e99a1-87f4-4cfc-a464-b3b1d1c16528": { 2024-04-04T06:11:41.230 INFO:teuthology.orchestra.run.smithi159.stdout: "path": "/subdir", 2024-04-04T06:11:41.230 INFO:teuthology.orchestra.run.smithi159.stdout: "tag": "c41e99a1-87f4-4cfc-a464-b3b1d1c16528", 2024-04-04T06:11:41.230 INFO:teuthology.orchestra.run.smithi159.stdout: "options": "recursive,repair" 2024-04-04T06:11:41.230 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:11:41.230 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:11:41.230 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:11:41.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.232+0000 7effaea00640 1 -- 172.21.15.159:0/919742995 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7eff8406c8f0 msgr2=0x7eff8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:41.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.232+0000 7effaea00640 1 --2- 172.21.15.159:0/919742995 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7eff8406c8f0 0x7eff8406edb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7eff90006fd0 tx=0x7eff90008040 comp rx=0 tx=0).stop 2024-04-04T06:11:41.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.232+0000 7effaea00640 1 -- 172.21.15.159:0/919742995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effa8071eb0 msgr2=0x7effa814ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:11:41.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.232+0000 7effaea00640 1 --2- 172.21.15.159:0/919742995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effa8071eb0 0x7effa814ea20 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7eff9802b960 tx=0x7eff980046c0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.232+0000 7effaea00640 1 -- 172.21.15.159:0/919742995 shutdown_connections 2024-04-04T06:11:41.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.232+0000 7effaea00640 1 --2- 172.21.15.159:0/919742995 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7eff8406c8f0 0x7eff8406edb0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.232+0000 7effaea00640 1 --2- 172.21.15.159:0/919742995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effa8141c10 0x7effa80ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.232+0000 7effaea00640 1 --2- 172.21.15.159:0/919742995 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7effa80d0280 0x7effa80ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.232+0000 7effaea00640 1 --2- 172.21.15.159:0/919742995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effa8071eb0 0x7effa814ea20 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:11:41.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.232+0000 7effaea00640 1 -- 172.21.15.159:0/919742995 >> 172.21.15.159:0/919742995 conn(0x7effa806b750 msgr2=0x7effa80b3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:11:41.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.232+0000 7effaea00640 1 -- 172.21.15.159:0/919742995 shutdown_connections 2024-04-04T06:11:41.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:11:41.232+0000 7effaea00640 1 -- 172.21.15.159:0/919742995 wait complete. 2024-04-04T06:11:41.241 INFO:tasks.cephfs.filesystem:scrub status for tag:c41e99a1-87f4-4cfc-a464-b3b1d1c16528 - {'path': '/subdir', 'tag': 'c41e99a1-87f4-4cfc-a464-b3b1d1c16528', 'options': 'recursive,repair'} 2024-04-04T06:11:42.216 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:11:42.224 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:12:11.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:11.243 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-04T06:12:11.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.473+0000 7f2e15000640 1 Processor -- start 2024-04-04T06:12:11.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.473+0000 7f2e15000640 1 -- start start 2024-04-04T06:12:11.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.474+0000 7f2e15000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2e10153580 0x7f2e100d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.474+0000 7f2e15000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e100d07c0 0x7f2e100aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.474+0000 7f2e15000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e100afab0 con 0x7f2e100af2e0 2024-04-04T06:12:11.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.474+0000 7f2e15000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f2e100d0c00 con 0x7f2e10153580 2024-04-04T06:12:11.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.474+0000 7f2e15000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2e1015cc80 con 0x7f2e100d07c0 2024-04-04T06:12:11.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.475+0000 7f2e0de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2e10153580 0x7f2e100d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:11.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.475+0000 7f2e0de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2e10153580 0x7f2e100d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34908/0 (socket says 172.21.15.159:34908) 2024-04-04T06:12:11.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.475+0000 7f2e0de00640 1 -- 172.21.15.159:0/3474684570 learned_addr learned my addr 172.21.15.159:0/3474684570 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:11.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.475+0000 7f2e0e800640 1 --1- 172.21.15.159:0/3474684570 >> v1:172.21.15.159:6789/0 conn(0x7f2e100af2e0 0x7f2e1006d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52688/0 (socket says 172.21.15.159:52688) 2024-04-04T06:12:11.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.475+0000 7f2e0d400640 1 --2- 172.21.15.159:0/3474684570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e100d07c0 0x7f2e100aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:11.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.475+0000 7f2e0de00640 1 -- 172.21.15.159:0/3474684570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e100d07c0 msgr2=0x7f2e100aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:11.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.475+0000 7f2e0de00640 1 --2- 172.21.15.159:0/3474684570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e100d07c0 0x7f2e100aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.475+0000 7f2e0de00640 1 -- 172.21.15.159:0/3474684570 >> v1:172.21.15.159:6789/0 conn(0x7f2e100af2e0 legacy=0x7f2e1006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:11.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.475+0000 7f2e0de00640 1 -- 172.21.15.159:0/3474684570 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1015da00 con 0x7f2e10153580 2024-04-04T06:12:11.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.475+0000 7f2e0ca00640 1 -- 172.21.15.159:0/3474684570 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1905531388 0 0) 0x7f2e100afab0 con 0x7f2e100af2e0 2024-04-04T06:12:11.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.477+0000 7f2e0de00640 1 --2- 172.21.15.159:0/3474684570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e10153580 0x7f2e100d0280 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009e50 tx=0x7f2e0002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553623d655fe7d82 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:11.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.477+0000 7f2e0ca00640 1 -- 172.21.15.159:0/3474684570 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0002ca70 con 0x7f2e10153580 2024-04-04T06:12:11.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.478+0000 7f2e0ca00640 1 -- 172.21.15.159:0/3474684570 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e0002cc10 con 0x7f2e10153580 2024-04-04T06:12:11.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.478+0000 7f2e0ca00640 1 -- 172.21.15.159:0/3474684570 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e000343f0 con 0x7f2e10153580 2024-04-04T06:12:11.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.478+0000 7f2e15000640 1 -- 172.21.15.159:0/3474684570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e10153580 msgr2=0x7f2e100d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:11.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.478+0000 7f2e15000640 1 --2- 172.21.15.159:0/3474684570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e10153580 0x7f2e100d0280 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009e50 tx=0x7f2e0002bed0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.478+0000 7f2e15000640 1 -- 172.21.15.159:0/3474684570 shutdown_connections 2024-04-04T06:12:11.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.478+0000 7f2e15000640 1 --2- 172.21.15.159:0/3474684570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e100d07c0 0x7f2e100aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.478+0000 7f2e15000640 1 --2- 172.21.15.159:0/3474684570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e10153580 0x7f2e100d0280 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.478+0000 7f2e15000640 1 -- 172.21.15.159:0/3474684570 >> 172.21.15.159:0/3474684570 conn(0x7f2e1006b750 msgr2=0x7f2e1006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:11.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.478+0000 7f2e15000640 1 -- 172.21.15.159:0/3474684570 shutdown_connections 2024-04-04T06:12:11.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.478+0000 7f2e15000640 1 -- 172.21.15.159:0/3474684570 wait complete. 2024-04-04T06:12:11.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.480+0000 7f2e15000640 1 Processor -- start 2024-04-04T06:12:11.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.480+0000 7f2e15000640 1 -- start start 2024-04-04T06:12:11.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.481+0000 7f2e15000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e100af2e0 0x7f2e1014f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.481+0000 7f2e15000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e100d07c0 0x7f2e100b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.481+0000 7f2e15000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e10153580 0x7f2e100b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.481+0000 7f2e15000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1015d4e0 con 0x7f2e10153580 2024-04-04T06:12:11.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.481+0000 7f2e15000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1015d200 con 0x7f2e100af2e0 2024-04-04T06:12:11.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.481+0000 7f2e15000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2e10153960 con 0x7f2e100d07c0 2024-04-04T06:12:11.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.482+0000 7f2e0e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e10153580 0x7f2e100b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:11.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.482+0000 7f2e0e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e10153580 0x7f2e100b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34920/0 (socket says 172.21.15.159:34920) 2024-04-04T06:12:11.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.482+0000 7f2e0e800640 1 -- 172.21.15.159:0/3809946357 learned_addr learned my addr 172.21.15.159:0/3809946357 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:11.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.482+0000 7f2e0de00640 1 --2- 172.21.15.159:0/3809946357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e100af2e0 0x7f2e1014f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:11.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.482+0000 7f2e0d400640 1 --2- 172.21.15.159:0/3809946357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e100d07c0 0x7f2e100b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:11.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.482+0000 7f2e0e800640 1 -- 172.21.15.159:0/3809946357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e100d07c0 msgr2=0x7f2e100b2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:11.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.482+0000 7f2e0e800640 1 --2- 172.21.15.159:0/3809946357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e100d07c0 0x7f2e100b2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.482+0000 7f2e0e800640 1 -- 172.21.15.159:0/3809946357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e100af2e0 msgr2=0x7f2e1014f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:11.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.482+0000 7f2e0e800640 1 --2- 172.21.15.159:0/3809946357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e100af2e0 0x7f2e1014f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.482+0000 7f2e0e800640 1 -- 172.21.15.159:0/3809946357 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e100cf810 con 0x7f2e10153580 2024-04-04T06:12:11.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.482+0000 7f2e0d400640 1 --2- 172.21.15.159:0/3809946357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e100d07c0 0x7f2e100b2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:11.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.482+0000 7f2e0e800640 1 --2- 172.21.15.159:0/3809946357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e10153580 0x7f2e100b32a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800c410 tx=0x7f2e0800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:11.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.483+0000 7f2e06000640 1 -- 172.21.15.159:0/3809946357 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e080074e0 con 0x7f2e10153580 2024-04-04T06:12:11.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.483+0000 7f2e06000640 1 -- 172.21.15.159:0/3809946357 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e080047e0 con 0x7f2e10153580 2024-04-04T06:12:11.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.483+0000 7f2e15000640 1 -- 172.21.15.159:0/3809946357 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e101f2940 con 0x7f2e10153580 2024-04-04T06:12:11.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.483+0000 7f2e06000640 1 -- 172.21.15.159:0/3809946357 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e08004bd0 con 0x7f2e10153580 2024-04-04T06:12:11.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.483+0000 7f2e15000640 1 -- 172.21.15.159:0/3809946357 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e101f2f70 con 0x7f2e10153580 2024-04-04T06:12:11.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.486+0000 7f2e06000640 1 -- 172.21.15.159:0/3809946357 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e08011070 con 0x7f2e10153580 2024-04-04T06:12:11.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.487+0000 7f2e17dd2740 1 Processor -- start 2024-04-04T06:12:11.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.487+0000 7f2e17dd2740 1 -- start start 2024-04-04T06:12:11.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.487+0000 7f2e06000640 1 --2- 172.21.15.159:0/3809946357 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2dec06c8f0 0x7f2dec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.487+0000 7f2e06000640 1 -- 172.21.15.159:0/3809946357 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f2e0808b6a0 con 0x7f2e10153580 2024-04-04T06:12:11.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.488+0000 7f2e17dd2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c2580 0x557eeb895520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.488+0000 7f2e17dd2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb8c2960 0x557eeb8c5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.488+0000 7f2e17dd2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb895a60 0x557eeb8c5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.488+0000 7f2e17dd2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557eeb8006d0 con 0x557eeb8c2580 2024-04-04T06:12:11.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.488+0000 7f2e17dd2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557eeb7a8770 con 0x557eeb8c2960 2024-04-04T06:12:11.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.488+0000 7f2e17dd2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x557eeb7fc680 con 0x557eeb895a60 2024-04-04T06:12:11.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.488+0000 7f2e0d400640 1 --2- 172.21.15.159:0/3809946357 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2dec06c8f0 0x7f2dec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:11.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.488+0000 7f2e0de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c2580 0x557eeb895520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:11.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.488+0000 7f2e0de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c2580 0x557eeb895520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34932/0 (socket says 172.21.15.159:34932) 2024-04-04T06:12:11.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.488+0000 7f2e0de00640 1 -- 172.21.15.159:0/1143144860 learned_addr learned my addr 172.21.15.159:0/1143144860 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:11.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.488+0000 7f2e0d400640 1 --2- 172.21.15.159:0/1143144860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb8c2960 0x557eeb8c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:11.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.489+0000 7f2e0e800640 1 --2- 172.21.15.159:0/1143144860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb895a60 0x557eeb8c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:11.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.489+0000 7f2e0de00640 1 -- 172.21.15.159:0/1143144860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb895a60 msgr2=0x557eeb8c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:11.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.489+0000 7f2e0de00640 1 --2- 172.21.15.159:0/1143144860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb895a60 0x557eeb8c5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.489+0000 7f2e0de00640 1 -- 172.21.15.159:0/1143144860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb8c2960 msgr2=0x557eeb8c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:11.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.489+0000 7f2e0de00640 1 --2- 172.21.15.159:0/1143144860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb8c2960 0x557eeb8c5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.489+0000 7f2e0de00640 1 -- 172.21.15.159:0/1143144860 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eeb5de7d0 con 0x557eeb8c2580 2024-04-04T06:12:11.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.489+0000 7f2e0de00640 1 --2- 172.21.15.159:0/1143144860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c2580 0x557eeb895520 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f2e00036040 tx=0x7f2e0002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48dc62a5d630c76 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:11.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.489+0000 7f2dfea00640 1 -- 172.21.15.159:0/1143144860 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00044070 con 0x557eeb8c2580 2024-04-04T06:12:11.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.489+0000 7f2dfea00640 1 -- 172.21.15.159:0/1143144860 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e0002cb30 con 0x557eeb8c2580 2024-04-04T06:12:11.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.489+0000 7f2dfea00640 1 -- 172.21.15.159:0/1143144860 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00003bd0 con 0x557eeb8c2580 2024-04-04T06:12:11.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.489+0000 7f2e0d400640 1 --2- 172.21.15.159:0/3809946357 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2dec06c8f0 0x7f2dec06edb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x557eeb89c2b0 tx=0x7f2df8009530 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:11.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.490+0000 7f2e17dd2740 1 -- 172.21.15.159:0/1143144860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c2580 msgr2=0x557eeb895520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:11.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.490+0000 7f2e17dd2740 1 --2- 172.21.15.159:0/1143144860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c2580 0x557eeb895520 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f2e00036040 tx=0x7f2e0002e000 comp rx=0 tx=0).stop 2024-04-04T06:12:11.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.490+0000 7f2e17dd2740 1 -- 172.21.15.159:0/1143144860 shutdown_connections 2024-04-04T06:12:11.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.490+0000 7f2e17dd2740 1 --2- 172.21.15.159:0/1143144860 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb895a60 0x557eeb8c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.490+0000 7f2e17dd2740 1 --2- 172.21.15.159:0/1143144860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb8c2960 0x557eeb8c5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.490+0000 7f2e17dd2740 1 --2- 172.21.15.159:0/1143144860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c2580 0x557eeb895520 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.490+0000 7f2e17dd2740 1 -- 172.21.15.159:0/1143144860 >> 172.21.15.159:0/1143144860 conn(0x557eeb897790 msgr2=0x557eeb8e0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:11.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.490+0000 7f2e17dd2740 1 -- 172.21.15.159:0/1143144860 shutdown_connections 2024-04-04T06:12:11.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.490+0000 7f2e17dd2740 1 -- 172.21.15.159:0/1143144860 wait complete. 2024-04-04T06:12:11.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.491+0000 7f2e17dd2740 1 -- start start 2024-04-04T06:12:11.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.491+0000 7f2e17dd2740 1 Processor -- start 2024-04-04T06:12:11.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.492+0000 7f2e17dd2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c2580 0x557eeb8c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.492+0000 7f2e17dd2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb8c2960 0x557eeb8bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.492+0000 7f2e17dd2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb8c0590 0x557eeb8dec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.492+0000 7f2e17dd2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557eeb899e30 con 0x557eeb8c2580 2024-04-04T06:12:11.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.492+0000 7f2e17dd2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557eeb7bbaa0 con 0x557eeb8c0590 2024-04-04T06:12:11.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.492+0000 7f2e17dd2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x557eeb89c670 con 0x557eeb8c2960 2024-04-04T06:12:11.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.492+0000 7f2e17dd2740 10 client.0 _ll_register_callbacks cb 0x557eeb7bb490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:12:11.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.493+0000 7f2e0de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c2580 0x557eeb8c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:11.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.493+0000 7f2e0de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c2580 0x557eeb8c78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34944/0 (socket says 172.21.15.159:34944) 2024-04-04T06:12:11.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.493+0000 7f2e0de00640 1 -- 172.21.15.159:0/3643095574 learned_addr learned my addr 172.21.15.159:0/3643095574 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:11.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.493+0000 7f2e0de00640 1 -- 172.21.15.159:0/3643095574 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb8c2960 msgr2=0x557eeb8bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:11.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.493+0000 7f2e0de00640 1 --2- 172.21.15.159:0/3643095574 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb8c2960 0x557eeb8bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.493+0000 7f2e0de00640 1 -- 172.21.15.159:0/3643095574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb8c0590 msgr2=0x557eeb8dec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:11.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.493+0000 7f2e0de00640 1 --2- 172.21.15.159:0/3643095574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb8c0590 0x557eeb8dec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.493+0000 7f2e0de00640 1 -- 172.21.15.159:0/3643095574 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eeb5143f0 con 0x557eeb8c2580 2024-04-04T06:12:11.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.493+0000 7f2e0de00640 1 --2- 172.21.15.159:0/3643095574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c2580 0x557eeb8c78a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f2e00004260 tx=0x7f2e00004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebe03cc7ee015fb server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:11.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.493+0000 7f2df3400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:12:11.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.493+0000 7f2df3400640 1 -- 172.21.15.159:0/3643095574 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0004c030 con 0x557eeb8c2580 2024-04-04T06:12:11.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.494+0000 7f2df3400640 1 -- 172.21.15.159:0/3643095574 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f2e0002c930 con 0x557eeb8c2580 2024-04-04T06:12:11.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.494+0000 7f2e17dd2740 1 -- 172.21.15.159:0/3643095574 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e0002cb30 con 0x557eeb8c2580 2024-04-04T06:12:11.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.494+0000 7f2df3400640 1 -- 172.21.15.159:0/3643095574 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f2e00055050 con 0x557eeb8c2580 2024-04-04T06:12:11.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.494+0000 7f2e17dd2740 10 client.7059 fetch_fsmap learned FSMap version 53 2024-04-04T06:12:11.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.494+0000 7f2e17dd2740 1 -- 172.21.15.159:0/3643095574 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x557eeb4d40f0 con 0x557eeb8c2580 2024-04-04T06:12:11.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.494+0000 7f2df3400640 1 -- 172.21.15.159:0/3643095574 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e0002cb30 con 0x557eeb8c2580 2024-04-04T06:12:11.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.494+0000 7f2df3400640 1 -- 172.21.15.159:0/3643095574 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f2e00058020 con 0x557eeb8c2580 2024-04-04T06:12:11.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.494+0000 7f2e17dd2740 10 client.7059 fetch_fsmap finished waiting for FSMap version 53 2024-04-04T06:12:11.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.494+0000 7f2e17dd2740 10 client.7059 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-04T06:12:11.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.494+0000 7f2e17dd2740 20 client.7059 populate_metadata read hostname 'smithi159' 2024-04-04T06:12:11.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.494+0000 7f2e17dd2740 1 --2- 172.21.15.159:0/3643095574 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x557eeb8c6980 0x557eeb8c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.494+0000 7f2e17dd2740 4 client.7059 mds_command: new command op to 6828 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:12:11.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.494+0000 7f2e17dd2740 1 -- 172.21.15.159:0/3643095574 --> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557eeb80dc10 con 0x557eeb8c6980 2024-04-04T06:12:11.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.495+0000 7f2e0d400640 1 --2- 172.21.15.159:0/3643095574 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x557eeb8c6980 0x557eeb8c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:11.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.495+0000 7f2e0d400640 1 --2- 172.21.15.159:0/3643095574 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x557eeb8c6980 0x557eeb8c6dc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f16eda5883ca30de server_cookie=6aba11df90f03ed3 in_seq=0 out_seq=0 2024-04-04T06:12:11.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.495+0000 7f2df3400640 10 client.7059 ms_handle_connect on v2:172.21.15.159:6834/2467594024 2024-04-04T06:12:11.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.496+0000 7f2df3400640 1 -- 172.21.15.159:0/3643095574 <== mds.0 v2:172.21.15.159:6834/2467594024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557eeb80dc10 con 0x557eeb8c6980 2024-04-04T06:12:11.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.496+0000 7f2df3400640 10 client.7059 handle_command_reply: tid=0 2024-04-04T06:12:11.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.496+0000 7f2e17dd2740 1 client.7059 shutdown 2024-04-04T06:12:11.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.496+0000 7f2e17dd2740 1 -- 172.21.15.159:0/3643095574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c2580 msgr2=0x557eeb8c78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:11.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.496+0000 7f2e17dd2740 1 --2- 172.21.15.159:0/3643095574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c2580 0x557eeb8c78a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f2e00004260 tx=0x7f2e00004ab0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.496+0000 7f2e17dd2740 1 -- 172.21.15.159:0/3643095574 shutdown_connections 2024-04-04T06:12:11.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.496+0000 7f2e17dd2740 1 --2- 172.21.15.159:0/3643095574 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x557eeb8c6980 0x557eeb8c6dc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.496+0000 7f2e17dd2740 1 --2- 172.21.15.159:0/3643095574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb8c0590 0x557eeb8dec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.496+0000 7f2e17dd2740 1 --2- 172.21.15.159:0/3643095574 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb8c2960 0x557eeb8bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.496+0000 7f2e17dd2740 1 --2- 172.21.15.159:0/3643095574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c2580 0x557eeb8c78a0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.496+0000 7f2df3400640 0 client.7059 ms_handle_reset on v2:172.21.15.159:6834/2467594024 2024-04-04T06:12:11.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.496+0000 7f2e17dd2740 1 -- 172.21.15.159:0/3643095574 >> 172.21.15.159:0/3643095574 conn(0x557eeb895a60 msgr2=0x557eeb899500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:11.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.496+0000 7f2e17dd2740 1 -- 172.21.15.159:0/3643095574 shutdown_connections 2024-04-04T06:12:11.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.496+0000 7f2e17dd2740 1 -- 172.21.15.159:0/3643095574 wait complete. 2024-04-04T06:12:11.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.497+0000 7f2e17dd2740 20 client.7059 trim_cache size 0 max 16384 2024-04-04T06:12:11.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.516+0000 7f2e17dd2740 1 Processor -- start 2024-04-04T06:12:11.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.516+0000 7f2e17dd2740 1 -- start start 2024-04-04T06:12:11.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.516+0000 7f2e17dd2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c0590 0x557eeb894d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.516+0000 7f2e17dd2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb8c2580 0x557eeb8fe390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.516+0000 7f2e17dd2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb8c2960 0x557eeb8fe8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.516+0000 7f2e17dd2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101fc740 con 0x557eeb8c0590 2024-04-04T06:12:11.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.516+0000 7f2e17dd2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2e10072a90 con 0x557eeb8c2580 2024-04-04T06:12:11.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.516+0000 7f2e17dd2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2e101f44b0 con 0x557eeb8c2960 2024-04-04T06:12:11.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.517+0000 7f2e0de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c0590 0x557eeb894d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:11.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.517+0000 7f2e0de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c0590 0x557eeb894d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34954/0 (socket says 172.21.15.159:34954) 2024-04-04T06:12:11.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.517+0000 7f2e0de00640 1 -- 172.21.15.159:0/2007310827 learned_addr learned my addr 172.21.15.159:0/2007310827 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:11.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.517+0000 7f2e0de00640 1 -- 172.21.15.159:0/2007310827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb8c2960 msgr2=0x557eeb8fe8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:11.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.517+0000 7f2e0de00640 1 --2- 172.21.15.159:0/2007310827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb8c2960 0x557eeb8fe8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.517+0000 7f2e0de00640 1 -- 172.21.15.159:0/2007310827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb8c2580 msgr2=0x557eeb8fe390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:11.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.517+0000 7f2e0de00640 1 --2- 172.21.15.159:0/2007310827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb8c2580 0x557eeb8fe390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.517+0000 7f2e0de00640 1 -- 172.21.15.159:0/2007310827 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eeb8e0750 con 0x557eeb8c0590 2024-04-04T06:12:11.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.517+0000 7f2e0de00640 1 --2- 172.21.15.159:0/2007310827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c0590 0x557eeb894d70 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009350 tx=0x7f2e00042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c59aa54142bed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:11.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.517+0000 7f2dfe000640 1 -- 172.21.15.159:0/2007310827 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0004c030 con 0x557eeb8c0590 2024-04-04T06:12:11.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.518+0000 7f2dfe000640 1 -- 172.21.15.159:0/2007310827 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e00004770 con 0x557eeb8c0590 2024-04-04T06:12:11.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.518+0000 7f2dfe000640 1 -- 172.21.15.159:0/2007310827 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00042070 con 0x557eeb8c0590 2024-04-04T06:12:11.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.518+0000 7f2e17dd2740 1 -- 172.21.15.159:0/2007310827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c0590 msgr2=0x557eeb894d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:11.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.518+0000 7f2e17dd2740 1 --2- 172.21.15.159:0/2007310827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c0590 0x557eeb894d70 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009350 tx=0x7f2e00042000 comp rx=0 tx=0).stop 2024-04-04T06:12:11.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.518+0000 7f2e17dd2740 1 -- 172.21.15.159:0/2007310827 shutdown_connections 2024-04-04T06:12:11.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.518+0000 7f2e17dd2740 1 --2- 172.21.15.159:0/2007310827 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb8c2960 0x557eeb8fe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.518+0000 7f2e17dd2740 1 --2- 172.21.15.159:0/2007310827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb8c2580 0x557eeb8fe390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.518+0000 7f2e17dd2740 1 --2- 172.21.15.159:0/2007310827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c0590 0x557eeb894d70 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.518+0000 7f2e17dd2740 1 -- 172.21.15.159:0/2007310827 >> 172.21.15.159:0/2007310827 conn(0x557eeb895a60 msgr2=0x557eeb890b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:11.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.518+0000 7f2e17dd2740 1 -- 172.21.15.159:0/2007310827 shutdown_connections 2024-04-04T06:12:11.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.518+0000 7f2e17dd2740 1 -- 172.21.15.159:0/2007310827 wait complete. 2024-04-04T06:12:11.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.519+0000 7f2e17dd2740 1 -- start start 2024-04-04T06:12:11.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.519+0000 7f2e17dd2740 1 Processor -- start 2024-04-04T06:12:11.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.520+0000 7f2e17dd2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb8c0590 0x557eeb955dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.520+0000 7f2e17dd2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c2580 0x557eeb8e2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.520+0000 7f2e17dd2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb8c2960 0x557eeb8e30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.520+0000 7f2e17dd2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557eeb8952b0 con 0x557eeb8c2580 2024-04-04T06:12:11.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.520+0000 7f2e17dd2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2e101f4b00 con 0x557eeb8c0590 2024-04-04T06:12:11.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.520+0000 7f2e17dd2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x557eeb8c6e00 con 0x557eeb8c2960 2024-04-04T06:12:11.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.520+0000 7f2e17dd2740 10 client.0 _ll_register_callbacks cb 0x7f2e00009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:12:11.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.520+0000 7f2e0d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c2580 0x557eeb8e2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:11.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.520+0000 7f2e0d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c2580 0x557eeb8e2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34968/0 (socket says 172.21.15.159:34968) 2024-04-04T06:12:11.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.520+0000 7f2e0d400640 1 -- 172.21.15.159:0/1818561687 learned_addr learned my addr 172.21.15.159:0/1818561687 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:11.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.521+0000 7f2e0de00640 1 --2- 172.21.15.159:0/1818561687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb8c0590 0x557eeb955dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:11.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.521+0000 7f2e0e800640 1 --2- 172.21.15.159:0/1818561687 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb8c2960 0x557eeb8e30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:11.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.521+0000 7f2e0d400640 1 -- 172.21.15.159:0/1818561687 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb8c2960 msgr2=0x557eeb8e30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:11.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.521+0000 7f2e0d400640 1 --2- 172.21.15.159:0/1818561687 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb8c2960 0x557eeb8e30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.521+0000 7f2e0d400640 1 -- 172.21.15.159:0/1818561687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb8c0590 msgr2=0x557eeb955dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:11.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.521+0000 7f2e0d400640 1 --2- 172.21.15.159:0/1818561687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb8c0590 0x557eeb955dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.521+0000 7f2e0d400640 1 -- 172.21.15.159:0/1818561687 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eeb8e35f0 con 0x557eeb8c2580 2024-04-04T06:12:11.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.521+0000 7f2e0e800640 1 --2- 172.21.15.159:0/1818561687 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb8c2960 0x557eeb8e30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:11.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.521+0000 7f2e0d400640 1 --2- 172.21.15.159:0/1818561687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c2580 0x557eeb8e2b70 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f2df8006fd0 tx=0x7f2df8019830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9fd30c6d34d664 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:11.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.521+0000 7f2df3400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:12:11.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.522+0000 7f2df3400640 1 -- 172.21.15.159:0/1818561687 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8005da0 con 0x557eeb8c2580 2024-04-04T06:12:11.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.522+0000 7f2df3400640 1 -- 172.21.15.159:0/1818561687 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f2df8004020 con 0x557eeb8c2580 2024-04-04T06:12:11.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.522+0000 7f2e17dd2740 1 -- 172.21.15.159:0/1818561687 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e00004770 con 0x557eeb8c2580 2024-04-04T06:12:11.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.522+0000 7f2df3400640 1 -- 172.21.15.159:0/1818561687 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f2df8033050 con 0x557eeb8c2580 2024-04-04T06:12:11.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.522+0000 7f2e17dd2740 10 client.7065 fetch_fsmap learned FSMap version 53 2024-04-04T06:12:11.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.522+0000 7f2e17dd2740 1 -- 172.21.15.159:0/1818561687 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x557eeb8e22c0 con 0x557eeb8c2580 2024-04-04T06:12:11.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.522+0000 7f2df3400640 1 -- 172.21.15.159:0/1818561687 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e00004770 con 0x557eeb8c2580 2024-04-04T06:12:11.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.522+0000 7f2df3400640 1 -- 172.21.15.159:0/1818561687 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f2df8036020 con 0x557eeb8c2580 2024-04-04T06:12:11.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.522+0000 7f2e17dd2740 10 client.7065 fetch_fsmap finished waiting for FSMap version 53 2024-04-04T06:12:11.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.522+0000 7f2e17dd2740 10 client.7065 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-04T06:12:11.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.522+0000 7f2e17dd2740 20 client.7065 populate_metadata read hostname 'smithi159' 2024-04-04T06:12:11.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.522+0000 7f2e17dd2740 1 --2- 172.21.15.159:0/1818561687 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x557eeb8bf5c0 0x557eeb89a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.523+0000 7f2e17dd2740 4 client.7065 mds_command: new command op to 6828 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-04T06:12:11.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.523+0000 7f2e17dd2740 1 -- 172.21.15.159:0/1818561687 --> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557eeb807f60 con 0x557eeb8bf5c0 2024-04-04T06:12:11.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.523+0000 7f2e0de00640 1 --2- 172.21.15.159:0/1818561687 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x557eeb8bf5c0 0x557eeb89a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:11.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.523+0000 7f2e0de00640 1 --2- 172.21.15.159:0/1818561687 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x557eeb8bf5c0 0x557eeb89a080 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2feeace77d7925e2 server_cookie=14efb41fb676c861 in_seq=0 out_seq=0 2024-04-04T06:12:11.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.523+0000 7f2df3400640 10 client.7065 ms_handle_connect on v2:172.21.15.159:6834/2467594024 2024-04-04T06:12:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.523+0000 7f2df3400640 1 -- 172.21.15.159:0/1818561687 <== mds.0 v2:172.21.15.159:6834/2467594024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557eeb807f60 con 0x557eeb8bf5c0 2024-04-04T06:12:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.523+0000 7f2df3400640 10 client.7065 handle_command_reply: tid=0 2024-04-04T06:12:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.524+0000 7f2e17dd2740 1 client.7065 shutdown 2024-04-04T06:12:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.524+0000 7f2e17dd2740 1 -- 172.21.15.159:0/1818561687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c2580 msgr2=0x557eeb8e2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.524+0000 7f2e17dd2740 1 --2- 172.21.15.159:0/1818561687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c2580 0x557eeb8e2b70 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f2df8006fd0 tx=0x7f2df8019830 comp rx=0 tx=0).stop 2024-04-04T06:12:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.524+0000 7f2e17dd2740 1 -- 172.21.15.159:0/1818561687 shutdown_connections 2024-04-04T06:12:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.524+0000 7f2e17dd2740 1 --2- 172.21.15.159:0/1818561687 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x557eeb8bf5c0 0x557eeb89a080 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.524+0000 7f2e17dd2740 1 --2- 172.21.15.159:0/1818561687 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557eeb8c2960 0x557eeb8e30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.524+0000 7f2e17dd2740 1 --2- 172.21.15.159:0/1818561687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557eeb8c2580 0x557eeb8e2b70 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.524+0000 7f2e17dd2740 1 --2- 172.21.15.159:0/1818561687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557eeb8c0590 0x557eeb955dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.524+0000 7f2e17dd2740 1 -- 172.21.15.159:0/1818561687 >> 172.21.15.159:0/1818561687 conn(0x557eeb895a60 msgr2=0x557eeb8fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.524+0000 7f2df3400640 0 client.7065 ms_handle_reset on v2:172.21.15.159:6834/2467594024 2024-04-04T06:12:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.524+0000 7f2e17dd2740 1 -- 172.21.15.159:0/1818561687 shutdown_connections 2024-04-04T06:12:11.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.524+0000 7f2e17dd2740 1 -- 172.21.15.159:0/1818561687 wait complete. 2024-04-04T06:12:11.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.524+0000 7f2e17dd2740 20 client.7065 trim_cache size 0 max 16384 2024-04-04T06:12:11.521 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:12:11.521 INFO:teuthology.orchestra.run.smithi159.stdout: "status": "no active scrubs running", 2024-04-04T06:12:11.521 INFO:teuthology.orchestra.run.smithi159.stdout: "scrubs": {} 2024-04-04T06:12:11.521 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:12:11.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.525+0000 7f2e15000640 1 -- 172.21.15.159:0/3809946357 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2dec06c8f0 msgr2=0x7f2dec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:11.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.525+0000 7f2e15000640 1 --2- 172.21.15.159:0/3809946357 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2dec06c8f0 0x7f2dec06edb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x557eeb89c2b0 tx=0x7f2df8009530 comp rx=0 tx=0).stop 2024-04-04T06:12:11.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.525+0000 7f2e15000640 1 -- 172.21.15.159:0/3809946357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e10153580 msgr2=0x7f2e100b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:11.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.525+0000 7f2e15000640 1 --2- 172.21.15.159:0/3809946357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e10153580 0x7f2e100b32a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800c410 tx=0x7f2e0800c8d0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.525+0000 7f2e15000640 1 -- 172.21.15.159:0/3809946357 shutdown_connections 2024-04-04T06:12:11.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.525+0000 7f2e15000640 1 --2- 172.21.15.159:0/3809946357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e10153580 0x7f2e100b32a0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.525+0000 7f2e15000640 1 --2- 172.21.15.159:0/3809946357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e100d07c0 0x7f2e100b2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.525+0000 7f2e15000640 1 --2- 172.21.15.159:0/3809946357 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2dec06c8f0 0x7f2dec06edb0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.525+0000 7f2e15000640 1 --2- 172.21.15.159:0/3809946357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e100af2e0 0x7f2e1014f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.525+0000 7f2e15000640 1 -- 172.21.15.159:0/3809946357 >> 172.21.15.159:0/3809946357 conn(0x7f2e1006b750 msgr2=0x7f2e100afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:11.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.525+0000 7f2e15000640 1 -- 172.21.15.159:0/3809946357 shutdown_connections 2024-04-04T06:12:11.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.525+0000 7f2e15000640 1 -- 172.21.15.159:0/3809946357 wait complete. 2024-04-04T06:12:11.535 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-04T06:12:11.535 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:12:11.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.750+0000 7f8333c00640 1 Processor -- start 2024-04-04T06:12:11.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.750+0000 7f8333c00640 1 -- start start 2024-04-04T06:12:11.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.752+0000 7f8333c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f832c0d3130 0x7f832c0cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.752+0000 7f8333c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f832c0d4f10 0x7f832c0b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.752+0000 7f8333c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f832c0d53a0 con 0x7f832c15e980 2024-04-04T06:12:11.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.752+0000 7f8333c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f832c0d2530 con 0x7f832c0d3130 2024-04-04T06:12:11.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.752+0000 7f8333c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f832c0cf630 con 0x7f832c0d4f10 2024-04-04T06:12:11.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.752+0000 7f832be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f832c0d3130 0x7f832c0cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:11.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.752+0000 7f832be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f832c0d3130 0x7f832c0cefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34972/0 (socket says 172.21.15.159:34972) 2024-04-04T06:12:11.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.752+0000 7f832be00640 1 -- 172.21.15.159:0/3712448542 learned_addr learned my addr 172.21.15.159:0/3712448542 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:11.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.752+0000 7f8331c00640 1 --2- 172.21.15.159:0/3712448542 >> v2:172.21.15.178:3300/0 conn(0x7f832c0d4f10 0x7f832c0b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:11.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.753+0000 7f832be00640 1 -- 172.21.15.159:0/3712448542 >> v2:172.21.15.178:3300/0 conn(0x7f832c0d4f10 msgr2=0x7f832c0b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:11.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.753+0000 7f832be00640 1 --2- 172.21.15.159:0/3712448542 >> v2:172.21.15.178:3300/0 conn(0x7f832c0d4f10 0x7f832c0b39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.753+0000 7f832be00640 1 -- 172.21.15.159:0/3712448542 >> v1:172.21.15.178:6789/0 conn(0x7f832c15e980 legacy=0x7f832c0d2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:11.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.753+0000 7f832be00640 1 -- 172.21.15.159:0/3712448542 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832c0cf910 con 0x7f832c0d3130 2024-04-04T06:12:11.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.754+0000 7f832be00640 1 --2- 172.21.15.159:0/3712448542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f832c0d3130 0x7f832c0cefe0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f832c08cdc0 tx=0x7f831802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d24313951a90371 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:11.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.754+0000 7f832b400640 1 -- 172.21.15.159:0/3712448542 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318035020 con 0x7f832c0d3130 2024-04-04T06:12:11.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.754+0000 7f832b400640 1 -- 172.21.15.159:0/3712448542 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8318031870 con 0x7f832c0d3130 2024-04-04T06:12:11.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.754+0000 7f832b400640 1 -- 172.21.15.159:0/3712448542 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318031c40 con 0x7f832c0d3130 2024-04-04T06:12:11.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.755+0000 7f8333c00640 1 -- 172.21.15.159:0/3712448542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f832c0d3130 msgr2=0x7f832c0cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:11.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.755+0000 7f8333c00640 1 --2- 172.21.15.159:0/3712448542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f832c0d3130 0x7f832c0cefe0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f832c08cdc0 tx=0x7f831802b910 comp rx=0 tx=0).stop 2024-04-04T06:12:11.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.755+0000 7f8333c00640 1 -- 172.21.15.159:0/3712448542 shutdown_connections 2024-04-04T06:12:11.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.755+0000 7f8333c00640 1 --2- 172.21.15.159:0/3712448542 >> v2:172.21.15.178:3300/0 conn(0x7f832c0d4f10 0x7f832c0b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.755+0000 7f8333c00640 1 --2- 172.21.15.159:0/3712448542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f832c0d3130 0x7f832c0cefe0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.755+0000 7f8333c00640 1 -- 172.21.15.159:0/3712448542 >> 172.21.15.159:0/3712448542 conn(0x7f832c08bfb0 msgr2=0x7f832c0d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:11.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.755+0000 7f8333c00640 1 -- 172.21.15.159:0/3712448542 shutdown_connections 2024-04-04T06:12:11.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.755+0000 7f8333c00640 1 -- 172.21.15.159:0/3712448542 wait complete. 2024-04-04T06:12:11.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.757+0000 7f8333c00640 1 Processor -- start 2024-04-04T06:12:11.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.757+0000 7f8333c00640 1 -- start start 2024-04-04T06:12:11.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.758+0000 7f8333c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f832c0d3130 0x7f832c1f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.758+0000 7f8333c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f832c0d4f10 0x7f832c14c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.758+0000 7f8333c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f832c15e980 0x7f832c0cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.758+0000 7f8333c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f832c0d01b0 con 0x7f832c0d4f10 2024-04-04T06:12:11.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.758+0000 7f8333c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f832c0cfed0 con 0x7f832c0d3130 2024-04-04T06:12:11.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.758+0000 7f8333c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f832c0b3ee0 con 0x7f832c15e980 2024-04-04T06:12:11.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.758+0000 7f832be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f832c0d4f10 0x7f832c14c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:11.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.758+0000 7f832be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f832c0d4f10 0x7f832c14c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34976/0 (socket says 172.21.15.159:34976) 2024-04-04T06:12:11.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.758+0000 7f832be00640 1 -- 172.21.15.159:0/3003755830 learned_addr learned my addr 172.21.15.159:0/3003755830 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:11.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.758+0000 7f8331200640 1 --2- 172.21.15.159:0/3003755830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f832c0d3130 0x7f832c1f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:11.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.758+0000 7f832be00640 1 -- 172.21.15.159:0/3003755830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f832c15e980 msgr2=0x7f832c0cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:11.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.758+0000 7f832be00640 1 --2- 172.21.15.159:0/3003755830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f832c15e980 0x7f832c0cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.758+0000 7f832be00640 1 -- 172.21.15.159:0/3003755830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f832c0d3130 msgr2=0x7f832c1f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:11.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.758+0000 7f832be00640 1 --2- 172.21.15.159:0/3003755830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f832c0d3130 0x7f832c1f76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:11.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.759+0000 7f832be00640 1 -- 172.21.15.159:0/3003755830 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832c0cdda0 con 0x7f832c0d4f10 2024-04-04T06:12:11.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.759+0000 7f8331200640 1 --2- 172.21.15.159:0/3003755830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f832c0d3130 0x7f832c1f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:11.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.759+0000 7f832be00640 1 --2- 172.21.15.159:0/3003755830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f832c0d4f10 0x7f832c14c5c0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8318000c00 tx=0x7f83180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:11.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.759+0000 7f8328c00640 1 -- 172.21.15.159:0/3003755830 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318040070 con 0x7f832c0d4f10 2024-04-04T06:12:11.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.759+0000 7f8328c00640 1 -- 172.21.15.159:0/3003755830 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8318030d00 con 0x7f832c0d4f10 2024-04-04T06:12:11.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.759+0000 7f8328c00640 1 -- 172.21.15.159:0/3003755830 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318035040 con 0x7f832c0d4f10 2024-04-04T06:12:11.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.759+0000 7f8333c00640 1 -- 172.21.15.159:0/3003755830 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832c0ce080 con 0x7f832c0d4f10 2024-04-04T06:12:11.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.759+0000 7f8333c00640 1 -- 172.21.15.159:0/3003755830 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f832c14cb00 con 0x7f832c0d4f10 2024-04-04T06:12:11.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.761+0000 7f8333c00640 1 -- 172.21.15.159:0/3003755830 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f832c0685c0 con 0x7f832c0d4f10 2024-04-04T06:12:11.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.761+0000 7f8328c00640 1 -- 172.21.15.159:0/3003755830 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f831804b060 con 0x7f832c0d4f10 2024-04-04T06:12:11.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.763+0000 7f8328c00640 1 --2- 172.21.15.159:0/3003755830 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f830c06c8f0 0x7f830c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:11.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.763+0000 7f8328c00640 1 -- 172.21.15.159:0/3003755830 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f83180ad960 con 0x7f832c0d4f10 2024-04-04T06:12:11.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.763+0000 7f8331200640 1 --2- 172.21.15.159:0/3003755830 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f830c06c8f0 0x7f830c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:11.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.764+0000 7f8331200640 1 --2- 172.21.15.159:0/3003755830 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f830c06c8f0 0x7f830c06edb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f832000a830 tx=0x7f8320008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:11.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:11.768+0000 7f8328c00640 1 -- 172.21.15.159:0/3003755830 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f831807bd70 con 0x7f832c0d4f10 2024-04-04T06:12:12.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.007+0000 7f8333c00640 1 -- 172.21.15.159:0/3003755830 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f832c0cd280 con 0x7f832c0d4f10 2024-04-04T06:12:12.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.008+0000 7f8328c00640 1 -- 172.21.15.159:0/3003755830 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5194 (secure 0 0 0) 0x7f831807b670 con 0x7f832c0d4f10 2024-04-04T06:12:12.001 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:12.001 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3789513463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3789513463},{"type":"v1","addr":"172.21.15.178:6835","nonce":3789513463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/468875883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":468875883},{"type":"v1","addr":"172.21.15.178:6837","nonce":468875883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2546974742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2546974742},{"type":"v1","addr":"172.21.15.159:6833","nonce":2546974742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:10:44.805617+0000","modified":"2024-04-04T06:11:44.957858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6828":{"gid":6828,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":15,"addr":"172.21.15.159:6835/2467594024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2467594024},{"type":"v1","addr":"172.21.15.159:6835","nonce":2467594024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6828,"qdb_cluster":[6828]},"id":5}]} 2024-04-04T06:12:12.002 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 53 2024-04-04T06:12:12.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.011+0000 7f8333c00640 1 -- 172.21.15.159:0/3003755830 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f830c06c8f0 msgr2=0x7f830c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:12.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.011+0000 7f8333c00640 1 --2- 172.21.15.159:0/3003755830 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f830c06c8f0 0x7f830c06edb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f832000a830 tx=0x7f8320008040 comp rx=0 tx=0).stop 2024-04-04T06:12:12.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.011+0000 7f8333c00640 1 -- 172.21.15.159:0/3003755830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f832c0d4f10 msgr2=0x7f832c14c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:12.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.011+0000 7f8333c00640 1 --2- 172.21.15.159:0/3003755830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f832c0d4f10 0x7f832c14c5c0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8318000c00 tx=0x7f83180046c0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.011+0000 7f8333c00640 1 -- 172.21.15.159:0/3003755830 shutdown_connections 2024-04-04T06:12:12.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.011+0000 7f8333c00640 1 --2- 172.21.15.159:0/3003755830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f832c15e980 0x7f832c0cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.011+0000 7f8333c00640 1 --2- 172.21.15.159:0/3003755830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f832c0d4f10 0x7f832c14c5c0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.011+0000 7f8333c00640 1 --2- 172.21.15.159:0/3003755830 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f830c06c8f0 0x7f830c06edb0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.011+0000 7f8333c00640 1 --2- 172.21.15.159:0/3003755830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f832c0d3130 0x7f832c1f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.011+0000 7f8333c00640 1 -- 172.21.15.159:0/3003755830 >> 172.21.15.159:0/3003755830 conn(0x7f832c08bfb0 msgr2=0x7f832c06cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:12.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.011+0000 7f8333c00640 1 -- 172.21.15.159:0/3003755830 shutdown_connections 2024-04-04T06:12:12.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.011+0000 7f8333c00640 1 -- 172.21.15.159:0/3003755830 wait complete. 2024-04-04T06:12:12.022 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-04T06:12:12.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.230+0000 7feab0c00640 1 Processor -- start 2024-04-04T06:12:12.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.230+0000 7feab0c00640 1 -- start start 2024-04-04T06:12:12.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.232+0000 7feab0c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7feaac13f190 0x7feaac0cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.232+0000 7feab0c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaac0ce890 con 0x7feaac0cc880 2024-04-04T06:12:12.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.232+0000 7feab0c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaac0ceb50 con 0x7feaac06dd30 2024-04-04T06:12:12.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.232+0000 7feab0c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7feaac0cee30 con 0x7feaac13f190 2024-04-04T06:12:12.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.232+0000 7feaab400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7feaac13f190 0x7feaac0cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:12.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.232+0000 7feaa3e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7feaac0cc880 0x7feaac06d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52702/0 (socket says 172.21.15.159:52702) 2024-04-04T06:12:12.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.232+0000 7feaab400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7feaac13f190 0x7feaac0cf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34990/0 (socket says 172.21.15.159:34990) 2024-04-04T06:12:12.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.232+0000 7feaa3e00640 1 -- 172.21.15.159:0/642872168 learned_addr learned my addr 172.21.15.159:0/642872168 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:12.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.233+0000 7feaab400640 1 -- 172.21.15.159:0/642872168 >> v1:172.21.15.178:6789/0 conn(0x7feaac06dd30 legacy=0x7feaac15ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:12.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.233+0000 7feaab400640 1 -- 172.21.15.159:0/642872168 >> v1:172.21.15.159:6789/0 conn(0x7feaac0cc880 legacy=0x7feaac06d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:12.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.233+0000 7feaab400640 1 -- 172.21.15.159:0/642872168 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaac0cf110 con 0x7feaac13f190 2024-04-04T06:12:12.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.233+0000 7feaaaa00640 1 -- 172.21.15.159:0/642872168 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 188972573 0 0) 0x7feaac0ce890 con 0x7feaac0cc880 2024-04-04T06:12:12.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.234+0000 7feaab400640 1 --2- 172.21.15.159:0/642872168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feaac13f190 0x7feaac0cf630 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0097a0 tx=0x7fea9c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34432c1b7afd8b5b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:12.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.234+0000 7feaaaa00640 1 -- 172.21.15.159:0/642872168 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c035020 con 0x7feaac13f190 2024-04-04T06:12:12.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.234+0000 7feaaaa00640 1 -- 172.21.15.159:0/642872168 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea9c031830 con 0x7feaac13f190 2024-04-04T06:12:12.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.234+0000 7feaaaa00640 1 -- 172.21.15.159:0/642872168 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c031c00 con 0x7feaac13f190 2024-04-04T06:12:12.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.235+0000 7feab0c00640 1 -- 172.21.15.159:0/642872168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feaac13f190 msgr2=0x7feaac0cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:12.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.235+0000 7feab0c00640 1 --2- 172.21.15.159:0/642872168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feaac13f190 0x7feaac0cf630 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0097a0 tx=0x7fea9c02b8d0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.235+0000 7feab0c00640 1 -- 172.21.15.159:0/642872168 shutdown_connections 2024-04-04T06:12:12.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.235+0000 7feab0c00640 1 --2- 172.21.15.159:0/642872168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feaac13f190 0x7feaac0cf630 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.235+0000 7feab0c00640 1 -- 172.21.15.159:0/642872168 >> 172.21.15.159:0/642872168 conn(0x7feaac0b1ba0 msgr2=0x7feaac0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:12.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.235+0000 7feab0c00640 1 -- 172.21.15.159:0/642872168 shutdown_connections 2024-04-04T06:12:12.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.235+0000 7feab0c00640 1 -- 172.21.15.159:0/642872168 wait complete. 2024-04-04T06:12:12.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.238+0000 7feab0c00640 1 Processor -- start 2024-04-04T06:12:12.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.238+0000 7feab0c00640 1 -- start start 2024-04-04T06:12:12.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.239+0000 7feab0c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feaac06dd30 0x7feaac15b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.239+0000 7feab0c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaac0cc880 0x7feaac0b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.240+0000 7feab0c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaac13f190 0x7feaac13da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.240+0000 7feab0c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feaac0cfb70 con 0x7feaac06dd30 2024-04-04T06:12:12.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.240+0000 7feab0c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feaac13f570 con 0x7feaac0cc880 2024-04-04T06:12:12.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.240+0000 7feab0c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feaac160620 con 0x7feaac13f190 2024-04-04T06:12:12.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.240+0000 7feaab400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feaac06dd30 0x7feaac15b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:12.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.240+0000 7feaab400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feaac06dd30 0x7feaac15b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34998/0 (socket says 172.21.15.159:34998) 2024-04-04T06:12:12.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.240+0000 7feaab400640 1 -- 172.21.15.159:0/292631057 learned_addr learned my addr 172.21.15.159:0/292631057 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:12.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.240+0000 7feaab400640 1 -- 172.21.15.159:0/292631057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaac13f190 msgr2=0x7feaac13da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:12.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.240+0000 7feaab400640 1 --2- 172.21.15.159:0/292631057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaac13f190 0x7feaac13da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.240+0000 7feaab400640 1 -- 172.21.15.159:0/292631057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaac0cc880 msgr2=0x7feaac0b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:12.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.240+0000 7feaa3e00640 1 --2- 172.21.15.159:0/292631057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaac0cc880 0x7feaac0b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:12.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.240+0000 7feaab400640 1 --2- 172.21.15.159:0/292631057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaac0cc880 0x7feaac0b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.240+0000 7feaab400640 1 -- 172.21.15.159:0/292631057 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaac0b3a60 con 0x7feaac06dd30 2024-04-04T06:12:12.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.240+0000 7feaa3e00640 1 --2- 172.21.15.159:0/292631057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaac0cc880 0x7feaac0b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:12.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.241+0000 7feaab400640 1 --2- 172.21.15.159:0/292631057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feaac06dd30 0x7feaac15b4d0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fea9c02bee0 tx=0x7fea9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:12.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.241+0000 7feaa3400640 1 -- 172.21.15.159:0/292631057 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c03ba80 con 0x7feaac06dd30 2024-04-04T06:12:12.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.241+0000 7feaa3400640 1 -- 172.21.15.159:0/292631057 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea9c03bc20 con 0x7feaac06dd30 2024-04-04T06:12:12.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.241+0000 7feab0c00640 1 -- 172.21.15.159:0/292631057 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaac0b3d40 con 0x7feaac06dd30 2024-04-04T06:12:12.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.241+0000 7feaa3400640 1 -- 172.21.15.159:0/292631057 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0453f0 con 0x7feaac06dd30 2024-04-04T06:12:12.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.241+0000 7feab0c00640 1 -- 172.21.15.159:0/292631057 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaac0b45e0 con 0x7feaac06dd30 2024-04-04T06:12:12.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.243+0000 7feaa3400640 1 -- 172.21.15.159:0/292631057 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea9c038020 con 0x7feaac06dd30 2024-04-04T06:12:12.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.244+0000 7feaa3400640 1 --2- 172.21.15.159:0/292631057 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea8806c8f0 0x7fea8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.244+0000 7feaa3400640 1 -- 172.21.15.159:0/292631057 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7fea9c0ada70 con 0x7feaac06dd30 2024-04-04T06:12:12.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.245+0000 7feaa3e00640 1 --2- 172.21.15.159:0/292631057 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea8806c8f0 0x7fea8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:12.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.246+0000 7feaa3e00640 1 --2- 172.21.15.159:0/292631057 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea8806c8f0 0x7fea8806edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7feaac1e82d0 tx=0x7fea94009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:12.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.247+0000 7feab5026740 1 Processor -- start 2024-04-04T06:12:12.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.247+0000 7feab5026740 1 -- start start 2024-04-04T06:12:12.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.248+0000 7feab5026740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c39f2a8c0 0x562c39f2aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.248+0000 7feab5026740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c39efa8e0 0x562c39f27b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.249+0000 7feab5026740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39efe440 0x562c39f28070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.249+0000 7feab5026740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562c39e645c0 con 0x562c39efe440 2024-04-04T06:12:12.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.249+0000 7feab5026740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562c39e0c3b0 con 0x562c39efa8e0 2024-04-04T06:12:12.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.249+0000 7feab5026740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562c39e60570 con 0x562c39f2a8c0 2024-04-04T06:12:12.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.249+0000 7feaabe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39efe440 0x562c39f28070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:12.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.249+0000 7feaabe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39efe440 0x562c39f28070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35002/0 (socket says 172.21.15.159:35002) 2024-04-04T06:12:12.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.249+0000 7feaabe00640 1 -- 172.21.15.159:0/3121160751 learned_addr learned my addr 172.21.15.159:0/3121160751 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:12.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.249+0000 7feaa3e00640 1 --2- 172.21.15.159:0/3121160751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c39efa8e0 0x562c39f27b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:12.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.249+0000 7feaab400640 1 --2- 172.21.15.159:0/3121160751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c39f2a8c0 0x562c39f2aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:12.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.249+0000 7feaabe00640 1 -- 172.21.15.159:0/3121160751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c39f2a8c0 msgr2=0x562c39f2aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:12.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.249+0000 7feaabe00640 1 --2- 172.21.15.159:0/3121160751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c39f2a8c0 0x562c39f2aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.249+0000 7feaabe00640 1 -- 172.21.15.159:0/3121160751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c39efa8e0 msgr2=0x562c39f27b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:12.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.249+0000 7feaabe00640 1 --2- 172.21.15.159:0/3121160751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c39efa8e0 0x562c39f27b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.249+0000 7feaabe00640 1 -- 172.21.15.159:0/3121160751 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c39c447c0 con 0x562c39efe440 2024-04-04T06:12:12.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.249+0000 7feaa3e00640 1 --2- 172.21.15.159:0/3121160751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c39efa8e0 0x562c39f27b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:12.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.250+0000 7feaabe00640 1 --2- 172.21.15.159:0/3121160751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39efe440 0x562c39f28070 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009cd0 tx=0x7feaa400b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34acb5db1afc5ba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:12.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.250+0000 7fea9b400640 1 -- 172.21.15.159:0/3121160751 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa400bd40 con 0x562c39efe440 2024-04-04T06:12:12.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.250+0000 7fea9b400640 1 -- 172.21.15.159:0/3121160751 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaa4003970 con 0x562c39efe440 2024-04-04T06:12:12.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.250+0000 7fea9b400640 1 -- 172.21.15.159:0/3121160751 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa4003b60 con 0x562c39efe440 2024-04-04T06:12:12.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.250+0000 7feab5026740 1 -- 172.21.15.159:0/3121160751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39efe440 msgr2=0x562c39f28070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:12.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.250+0000 7feab5026740 1 --2- 172.21.15.159:0/3121160751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39efe440 0x562c39f28070 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009cd0 tx=0x7feaa400b420 comp rx=0 tx=0).stop 2024-04-04T06:12:12.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.251+0000 7feab5026740 1 -- 172.21.15.159:0/3121160751 shutdown_connections 2024-04-04T06:12:12.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.251+0000 7feab5026740 1 --2- 172.21.15.159:0/3121160751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39efe440 0x562c39f28070 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.251+0000 7feab5026740 1 --2- 172.21.15.159:0/3121160751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c39efa8e0 0x562c39f27b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.251+0000 7feab5026740 1 --2- 172.21.15.159:0/3121160751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c39f2a8c0 0x562c39f2aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.251+0000 7feab5026740 1 -- 172.21.15.159:0/3121160751 >> 172.21.15.159:0/3121160751 conn(0x562c39efcb50 msgr2=0x562c39f456d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:12.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.251+0000 7feab5026740 1 -- 172.21.15.159:0/3121160751 shutdown_connections 2024-04-04T06:12:12.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.251+0000 7feab5026740 1 -- 172.21.15.159:0/3121160751 wait complete. 2024-04-04T06:12:12.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.252+0000 7feab5026740 1 -- start start 2024-04-04T06:12:12.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.252+0000 7feab5026740 1 Processor -- start 2024-04-04T06:12:12.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.254+0000 7feab5026740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c39efe440 0x562c39f25640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.254+0000 7feab5026740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c39f2a8c0 0x562c39f2e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.254+0000 7feab5026740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39f566a0 0x562c39f31000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.254+0000 7feab5026740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562c39eff5f0 con 0x562c39f566a0 2024-04-04T06:12:12.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.254+0000 7feab5026740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562c39e1f7a0 con 0x562c39f2a8c0 2024-04-04T06:12:12.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.254+0000 7feab5026740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562c39efe9e0 con 0x562c39efe440 2024-04-04T06:12:12.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.254+0000 7feab5026740 10 client.0 _ll_register_callbacks cb 0x562c39e5bfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:12:12.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.254+0000 7feaabe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39f566a0 0x562c39f31000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:12.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.254+0000 7feaabe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39f566a0 0x562c39f31000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35012/0 (socket says 172.21.15.159:35012) 2024-04-04T06:12:12.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.254+0000 7feaabe00640 1 -- 172.21.15.159:0/2081173159 learned_addr learned my addr 172.21.15.159:0/2081173159 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:12.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.254+0000 7feaab400640 1 --2- 172.21.15.159:0/2081173159 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c39efe440 0x562c39f25640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:12.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.254+0000 7feaa3e00640 1 --2- 172.21.15.159:0/2081173159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c39f2a8c0 0x562c39f2e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:12.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.254+0000 7feaabe00640 1 -- 172.21.15.159:0/2081173159 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c39efe440 msgr2=0x562c39f25640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:12.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.254+0000 7feaabe00640 1 --2- 172.21.15.159:0/2081173159 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c39efe440 0x562c39f25640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.254+0000 7feaabe00640 1 -- 172.21.15.159:0/2081173159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c39f2a8c0 msgr2=0x562c39f2e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:12.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.254+0000 7feaabe00640 1 --2- 172.21.15.159:0/2081173159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c39f2a8c0 0x562c39f2e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.254+0000 7feaabe00640 1 -- 172.21.15.159:0/2081173159 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c39ca4f50 con 0x562c39f566a0 2024-04-04T06:12:12.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.255+0000 7feaabe00640 1 --2- 172.21.15.159:0/2081173159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39f566a0 0x562c39f31000 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7feaa400b920 tx=0x7feaa4004ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587c05c68acfa03d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:12.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.255+0000 7fea8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:12:12.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.255+0000 7fea8fe00640 1 -- 172.21.15.159:0/2081173159 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa4022020 con 0x562c39f566a0 2024-04-04T06:12:12.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.255+0000 7fea8fe00640 1 -- 172.21.15.159:0/2081173159 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7feaa40045d0 con 0x562c39f566a0 2024-04-04T06:12:12.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.255+0000 7feab5026740 1 -- 172.21.15.159:0/2081173159 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaa4003970 con 0x562c39f566a0 2024-04-04T06:12:12.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.255+0000 7fea8fe00640 1 -- 172.21.15.159:0/2081173159 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7feaa4004810 con 0x562c39f566a0 2024-04-04T06:12:12.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.255+0000 7feab5026740 10 client.7086 fetch_fsmap learned FSMap version 53 2024-04-04T06:12:12.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.255+0000 7feab5026740 1 -- 172.21.15.159:0/2081173159 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x562c39b380b0 con 0x562c39f566a0 2024-04-04T06:12:12.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.256+0000 7fea8fe00640 1 -- 172.21.15.159:0/2081173159 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaa4003970 con 0x562c39f566a0 2024-04-04T06:12:12.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.256+0000 7fea8fe00640 1 -- 172.21.15.159:0/2081173159 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7feaa4011070 con 0x562c39f566a0 2024-04-04T06:12:12.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.256+0000 7feab5026740 10 client.7086 fetch_fsmap finished waiting for FSMap version 53 2024-04-04T06:12:12.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.256+0000 7feab5026740 10 client.7086 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:12:12.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.256+0000 7feab5026740 20 client.7086 populate_metadata read hostname 'smithi159' 2024-04-04T06:12:12.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.256+0000 7feab5026740 1 --2- 172.21.15.159:0/2081173159 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x562c39f302f0 0x562c39f58ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.256+0000 7feab5026740 4 client.7086 mds_command: new command op to 6828 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:12:12.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.256+0000 7feab5026740 1 -- 172.21.15.159:0/2081173159 --> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c39e38890 con 0x562c39f302f0 2024-04-04T06:12:12.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.256+0000 7feaab400640 1 --2- 172.21.15.159:0/2081173159 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x562c39f302f0 0x562c39f58ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:12.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.257+0000 7fea8fe00640 10 client.7086 ms_handle_connect on v2:172.21.15.159:6834/2467594024 2024-04-04T06:12:12.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.257+0000 7feaab400640 1 --2- 172.21.15.159:0/2081173159 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x562c39f302f0 0x562c39f58ed0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa27395ce1b4fe9d server_cookie=8f42d7999da83d5d in_seq=0 out_seq=0 2024-04-04T06:12:12.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.257+0000 7fea8fe00640 1 -- 172.21.15.159:0/2081173159 <== mds.0 v2:172.21.15.159:6834/2467594024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fea9c093e50 con 0x562c39f302f0 2024-04-04T06:12:12.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.257+0000 7fea8fe00640 10 client.7086 handle_command_reply: tid=0 2024-04-04T06:12:12.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.257+0000 7feab5026740 1 client.7086 shutdown 2024-04-04T06:12:12.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.257+0000 7feab5026740 1 -- 172.21.15.159:0/2081173159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39f566a0 msgr2=0x562c39f31000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:12.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.257+0000 7feab5026740 1 --2- 172.21.15.159:0/2081173159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39f566a0 0x562c39f31000 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7feaa400b920 tx=0x7feaa4004ae0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.258+0000 7feab5026740 1 -- 172.21.15.159:0/2081173159 shutdown_connections 2024-04-04T06:12:12.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.258+0000 7feab5026740 1 --2- 172.21.15.159:0/2081173159 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x562c39f302f0 0x562c39f58ed0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.258+0000 7feab5026740 1 --2- 172.21.15.159:0/2081173159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39f566a0 0x562c39f31000 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.258+0000 7feab5026740 1 --2- 172.21.15.159:0/2081173159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c39f2a8c0 0x562c39f2e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.258+0000 7fea8fe00640 0 client.7086 ms_handle_reset on v2:172.21.15.159:6834/2467594024 2024-04-04T06:12:12.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.258+0000 7feab5026740 1 --2- 172.21.15.159:0/2081173159 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c39efe440 0x562c39f25640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.258+0000 7feab5026740 1 -- 172.21.15.159:0/2081173159 >> 172.21.15.159:0/2081173159 conn(0x562c39efa8e0 msgr2=0x562c39f00880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:12.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.258+0000 7feab5026740 1 -- 172.21.15.159:0/2081173159 shutdown_connections 2024-04-04T06:12:12.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.258+0000 7feab5026740 1 -- 172.21.15.159:0/2081173159 wait complete. 2024-04-04T06:12:12.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.258+0000 7feab5026740 20 client.7086 trim_cache size 0 max 16384 2024-04-04T06:12:12.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.272+0000 7feab5026740 1 Processor -- start 2024-04-04T06:12:12.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.272+0000 7feab5026740 1 -- start start 2024-04-04T06:12:12.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.273+0000 7feab5026740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c39efe440 0x562c39f2c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.273+0000 7feab5026740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39f2a8c0 0x562c39f48be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.273+0000 7feab5026740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c39f566a0 0x562c39f49120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.273+0000 7feab5026740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feaac1ff9a0 con 0x562c39f2a8c0 2024-04-04T06:12:12.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.273+0000 7feab5026740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feaac1e7da0 con 0x562c39f566a0 2024-04-04T06:12:12.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.273+0000 7feab5026740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feaac1f4ce0 con 0x562c39efe440 2024-04-04T06:12:12.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.273+0000 7feaa3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39f2a8c0 0x562c39f48be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:12.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.273+0000 7feaa3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39f2a8c0 0x562c39f48be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35026/0 (socket says 172.21.15.159:35026) 2024-04-04T06:12:12.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.273+0000 7feaa3e00640 1 -- 172.21.15.159:0/2277648780 learned_addr learned my addr 172.21.15.159:0/2277648780 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:12.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.273+0000 7feaa3e00640 1 -- 172.21.15.159:0/2277648780 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c39efe440 msgr2=0x562c39f2c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:12.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.273+0000 7feaa3e00640 1 --2- 172.21.15.159:0/2277648780 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c39efe440 0x562c39f2c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.273+0000 7feaa3e00640 1 -- 172.21.15.159:0/2277648780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c39f566a0 msgr2=0x562c39f49120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:12.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.273+0000 7feaa3e00640 1 --2- 172.21.15.159:0/2277648780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c39f566a0 0x562c39f49120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.273+0000 7feaa3e00640 1 -- 172.21.15.159:0/2277648780 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c39efc3f0 con 0x562c39f2a8c0 2024-04-04T06:12:12.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.274+0000 7feaa3e00640 1 --2- 172.21.15.159:0/2277648780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39f2a8c0 0x562c39f48be0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fea94007420 tx=0x7fea94011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a12d927aecf1f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:12.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.274+0000 7fea9aa00640 1 -- 172.21.15.159:0/2277648780 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9400d9b0 con 0x562c39f2a8c0 2024-04-04T06:12:12.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.274+0000 7fea9aa00640 1 -- 172.21.15.159:0/2277648780 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea9400db50 con 0x562c39f2a8c0 2024-04-04T06:12:12.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.274+0000 7fea9aa00640 1 -- 172.21.15.159:0/2277648780 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea94020430 con 0x562c39f2a8c0 2024-04-04T06:12:12.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.275+0000 7feab5026740 1 -- 172.21.15.159:0/2277648780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39f2a8c0 msgr2=0x562c39f48be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:12.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.275+0000 7feab5026740 1 --2- 172.21.15.159:0/2277648780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39f2a8c0 0x562c39f48be0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fea94007420 tx=0x7fea94011000 comp rx=0 tx=0).stop 2024-04-04T06:12:12.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.275+0000 7feab5026740 1 -- 172.21.15.159:0/2277648780 shutdown_connections 2024-04-04T06:12:12.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.275+0000 7feab5026740 1 --2- 172.21.15.159:0/2277648780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c39f566a0 0x562c39f49120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.275+0000 7feab5026740 1 --2- 172.21.15.159:0/2277648780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39f2a8c0 0x562c39f48be0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.275+0000 7feab5026740 1 --2- 172.21.15.159:0/2277648780 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c39efe440 0x562c39f2c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.275+0000 7feab5026740 1 -- 172.21.15.159:0/2277648780 >> 172.21.15.159:0/2277648780 conn(0x562c39efa8e0 msgr2=0x562c39f2aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:12.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.275+0000 7feab5026740 1 -- 172.21.15.159:0/2277648780 shutdown_connections 2024-04-04T06:12:12.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.275+0000 7feab5026740 1 -- 172.21.15.159:0/2277648780 wait complete. 2024-04-04T06:12:12.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.276+0000 7feab5026740 1 -- start start 2024-04-04T06:12:12.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.276+0000 7feab5026740 1 Processor -- start 2024-04-04T06:12:12.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.278+0000 7feab5026740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39efe440 0x562c39f007f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.278+0000 7feab5026740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c39f2a8c0 0x562c39f45720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.278+0000 7feab5026740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c39f566a0 0x562c39fc1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.278+0000 7feab5026740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562c39f2e880 con 0x562c39efe440 2024-04-04T06:12:12.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.278+0000 7feab5026740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562c39f2c200 con 0x562c39f566a0 2024-04-04T06:12:12.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.278+0000 7feab5026740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562c39f2b630 con 0x562c39f2a8c0 2024-04-04T06:12:12.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.278+0000 7feab5026740 10 client.0 _ll_register_callbacks cb 0x7feaa4009ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:12:12.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.278+0000 7feaab400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39efe440 0x562c39f007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:12.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.278+0000 7feaab400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39efe440 0x562c39f007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35028/0 (socket says 172.21.15.159:35028) 2024-04-04T06:12:12.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.278+0000 7feaab400640 1 -- 172.21.15.159:0/4230085581 learned_addr learned my addr 172.21.15.159:0/4230085581 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:12.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.278+0000 7feaab400640 1 -- 172.21.15.159:0/4230085581 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c39f2a8c0 msgr2=0x562c39f45720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:12.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.278+0000 7feaab400640 1 --2- 172.21.15.159:0/4230085581 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c39f2a8c0 0x562c39f45720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.278+0000 7feaab400640 1 -- 172.21.15.159:0/4230085581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c39f566a0 msgr2=0x562c39fc1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:12.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.278+0000 7feaab400640 1 --2- 172.21.15.159:0/4230085581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c39f566a0 0x562c39fc1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.278+0000 7feaab400640 1 -- 172.21.15.159:0/4230085581 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c39f46d40 con 0x562c39efe440 2024-04-04T06:12:12.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.279+0000 7feaab400640 1 --2- 172.21.15.159:0/4230085581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39efe440 0x562c39f007f0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fea9c07b0c0 tx=0x7fea9c07b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ddde7d22fcf560 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:12.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.279+0000 7fea8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:12:12.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.279+0000 7fea8fe00640 1 -- 172.21.15.159:0/4230085581 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c084250 con 0x562c39efe440 2024-04-04T06:12:12.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.279+0000 7fea8fe00640 1 -- 172.21.15.159:0/4230085581 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7fea9c079970 con 0x562c39efe440 2024-04-04T06:12:12.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.279+0000 7feab5026740 1 -- 172.21.15.159:0/4230085581 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea9400db50 con 0x562c39efe440 2024-04-04T06:12:12.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.279+0000 7fea8fe00640 1 -- 172.21.15.159:0/4230085581 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fea9c0aa050 con 0x562c39efe440 2024-04-04T06:12:12.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.279+0000 7feab5026740 10 client.7092 fetch_fsmap learned FSMap version 53 2024-04-04T06:12:12.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.279+0000 7feab5026740 1 -- 172.21.15.159:0/4230085581 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x562c39f30cc0 con 0x562c39efe440 2024-04-04T06:12:12.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.279+0000 7fea8fe00640 1 -- 172.21.15.159:0/4230085581 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea9400db50 con 0x562c39efe440 2024-04-04T06:12:12.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.279+0000 7fea8fe00640 1 -- 172.21.15.159:0/4230085581 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7fea9c077ce0 con 0x562c39efe440 2024-04-04T06:12:12.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.279+0000 7feab5026740 10 client.7092 fetch_fsmap finished waiting for FSMap version 53 2024-04-04T06:12:12.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.279+0000 7feab5026740 10 client.7092 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:12:12.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.279+0000 7feab5026740 20 client.7092 populate_metadata read hostname 'smithi159' 2024-04-04T06:12:12.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.279+0000 7feab5026740 1 --2- 172.21.15.159:0/4230085581 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x562c39f68ee0 0x562c39fc2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.279+0000 7feab5026740 4 client.7092 mds_command: new command op to 6828 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-04T06:12:12.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.279+0000 7feab5026740 1 -- 172.21.15.159:0/4230085581 --> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x562c39f32b20 con 0x562c39f68ee0 2024-04-04T06:12:12.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.280+0000 7feaa3e00640 1 --2- 172.21.15.159:0/4230085581 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x562c39f68ee0 0x562c39fc2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:12.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.280+0000 7fea8fe00640 10 client.7092 ms_handle_connect on v2:172.21.15.159:6834/2467594024 2024-04-04T06:12:12.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.280+0000 7feaa3e00640 1 --2- 172.21.15.159:0/4230085581 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x562c39f68ee0 0x562c39fc2a90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66cf00c9253ec6cd server_cookie=8600f4cc398d7bf7 in_seq=0 out_seq=0 2024-04-04T06:12:12.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.280+0000 7fea8fe00640 1 -- 172.21.15.159:0/4230085581 <== mds.0 v2:172.21.15.159:6834/2467594024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c39f32b20 con 0x562c39f68ee0 2024-04-04T06:12:12.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.280+0000 7fea8fe00640 10 client.7092 handle_command_reply: tid=0 2024-04-04T06:12:12.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.280+0000 7feab5026740 1 client.7092 shutdown 2024-04-04T06:12:12.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.281+0000 7feab5026740 1 -- 172.21.15.159:0/4230085581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39efe440 msgr2=0x562c39f007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:12.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.281+0000 7feab5026740 1 --2- 172.21.15.159:0/4230085581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39efe440 0x562c39f007f0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fea9c07b0c0 tx=0x7fea9c07b730 comp rx=0 tx=0).stop 2024-04-04T06:12:12.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.281+0000 7feab5026740 1 -- 172.21.15.159:0/4230085581 shutdown_connections 2024-04-04T06:12:12.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.281+0000 7feab5026740 1 --2- 172.21.15.159:0/4230085581 >> [v2:172.21.15.159:6834/2467594024,v1:172.21.15.159:6835/2467594024] conn(0x562c39f68ee0 0x562c39fc2a90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.281+0000 7feab5026740 1 --2- 172.21.15.159:0/4230085581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c39f566a0 0x562c39fc1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.281+0000 7feab5026740 1 --2- 172.21.15.159:0/4230085581 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c39f2a8c0 0x562c39f45720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.281+0000 7feab5026740 1 --2- 172.21.15.159:0/4230085581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c39efe440 0x562c39f007f0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.281+0000 7fea8fe00640 0 client.7092 ms_handle_reset on v2:172.21.15.159:6834/2467594024 2024-04-04T06:12:12.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.281+0000 7feab5026740 1 -- 172.21.15.159:0/4230085581 >> 172.21.15.159:0/4230085581 conn(0x562c39efa8e0 msgr2=0x562c39f2c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:12.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.281+0000 7feab5026740 1 -- 172.21.15.159:0/4230085581 shutdown_connections 2024-04-04T06:12:12.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.281+0000 7feab5026740 1 -- 172.21.15.159:0/4230085581 wait complete. 2024-04-04T06:12:12.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.281+0000 7feab5026740 20 client.7092 trim_cache size 0 max 16384 2024-04-04T06:12:12.277 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:12.277 INFO:teuthology.orchestra.run.smithi159.stdout:[] 2024-04-04T06:12:12.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.282+0000 7feab0c00640 1 -- 172.21.15.159:0/292631057 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea8806c8f0 msgr2=0x7fea8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:12.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.282+0000 7feab0c00640 1 --2- 172.21.15.159:0/292631057 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea8806c8f0 0x7fea8806edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7feaac1e82d0 tx=0x7fea94009040 comp rx=0 tx=0).stop 2024-04-04T06:12:12.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.282+0000 7feab0c00640 1 -- 172.21.15.159:0/292631057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feaac06dd30 msgr2=0x7feaac15b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:12.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.282+0000 7feab0c00640 1 --2- 172.21.15.159:0/292631057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feaac06dd30 0x7feaac15b4d0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fea9c02bee0 tx=0x7fea9c0046c0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.282+0000 7feab0c00640 1 -- 172.21.15.159:0/292631057 shutdown_connections 2024-04-04T06:12:12.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.282+0000 7feab0c00640 1 --2- 172.21.15.159:0/292631057 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaac13f190 0x7feaac13da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.282+0000 7feab0c00640 1 --2- 172.21.15.159:0/292631057 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea8806c8f0 0x7fea8806edb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.282+0000 7feab0c00640 1 --2- 172.21.15.159:0/292631057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaac0cc880 0x7feaac0b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.282+0000 7feab0c00640 1 --2- 172.21.15.159:0/292631057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feaac06dd30 0x7feaac15b4d0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.282+0000 7feab0c00640 1 -- 172.21.15.159:0/292631057 >> 172.21.15.159:0/292631057 conn(0x7feaac0b1ba0 msgr2=0x7feaac15fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:12.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.282+0000 7feab0c00640 1 -- 172.21.15.159:0/292631057 shutdown_connections 2024-04-04T06:12:12.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.282+0000 7feab0c00640 1 -- 172.21.15.159:0/292631057 wait complete. 2024-04-04T06:12:12.291 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-04T06:12:12.361 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:12:12.368 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:12:12.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.505+0000 7fb00b200640 1 Processor -- start 2024-04-04T06:12:12.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.505+0000 7fb00b200640 1 -- start start 2024-04-04T06:12:12.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.507+0000 7fb00b200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb00c144ad0 0x7fb00c0b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.507+0000 7fb00b200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb00c15f710 0x7fb00c072840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.507+0000 7fb00b200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb00c073190 con 0x7fb00c072d80 2024-04-04T06:12:12.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.507+0000 7fb00b200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fb00c0cdad0 con 0x7fb00c144ad0 2024-04-04T06:12:12.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.507+0000 7fb00b200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb00c15fba0 con 0x7fb00c15f710 2024-04-04T06:12:12.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.507+0000 7fb009e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb00c144ad0 0x7fb00c0b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:12.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.507+0000 7fb00a800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fb00c072d80 0x7fb00c0714e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52716/0 (socket says 172.21.15.159:52716) 2024-04-04T06:12:12.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.507+0000 7fb00a800640 1 -- 172.21.15.159:0/2512492364 learned_addr learned my addr 172.21.15.159:0/2512492364 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:12.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.508+0000 7fb001400640 1 --2- 172.21.15.159:0/2512492364 >> v2:172.21.15.178:3300/0 conn(0x7fb00c15f710 0x7fb00c072840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:12.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.508+0000 7fb009400640 1 -- 172.21.15.159:0/2512492364 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3246617418 0 0) 0x7fb004001490 con 0x7fb00c072d80 2024-04-04T06:12:12.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.508+0000 7fb009e00640 1 -- 172.21.15.159:0/2512492364 >> v2:172.21.15.178:3300/0 conn(0x7fb00c15f710 msgr2=0x7fb00c072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:12.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.508+0000 7fb009e00640 1 --2- 172.21.15.159:0/2512492364 >> v2:172.21.15.178:3300/0 conn(0x7fb00c15f710 0x7fb00c072840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.508+0000 7fb009e00640 1 -- 172.21.15.159:0/2512492364 >> v1:172.21.15.159:6789/0 conn(0x7fb00c072d80 legacy=0x7fb00c0714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:12.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.508+0000 7fb009e00640 1 -- 172.21.15.159:0/2512492364 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00c144eb0 con 0x7fb00c144ad0 2024-04-04T06:12:12.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.509+0000 7fb009e00640 1 --2- 172.21.15.159:0/2512492364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c144ad0 0x7fb00c0b2ff0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0b4d90 tx=0x7faffc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d710f308d74748b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:12.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.509+0000 7fb009400640 1 -- 172.21.15.159:0/2512492364 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc035020 con 0x7fb00c144ad0 2024-04-04T06:12:12.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.509+0000 7fb009400640 1 -- 172.21.15.159:0/2512492364 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faffc031830 con 0x7fb00c144ad0 2024-04-04T06:12:12.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.510+0000 7fb009400640 1 -- 172.21.15.159:0/2512492364 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc031c00 con 0x7fb00c144ad0 2024-04-04T06:12:12.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.510+0000 7fb00b200640 1 -- 172.21.15.159:0/2512492364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c144ad0 msgr2=0x7fb00c0b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:12.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.510+0000 7fb00b200640 1 --2- 172.21.15.159:0/2512492364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c144ad0 0x7fb00c0b2ff0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0b4d90 tx=0x7faffc02b8d0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.510+0000 7fb00b200640 1 -- 172.21.15.159:0/2512492364 shutdown_connections 2024-04-04T06:12:12.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.510+0000 7fb00b200640 1 --2- 172.21.15.159:0/2512492364 >> v2:172.21.15.178:3300/0 conn(0x7fb00c15f710 0x7fb00c072840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.510+0000 7fb00b200640 1 --2- 172.21.15.159:0/2512492364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c144ad0 0x7fb00c0b2ff0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.510+0000 7fb00b200640 1 -- 172.21.15.159:0/2512492364 >> 172.21.15.159:0/2512492364 conn(0x7fb00c06b570 msgr2=0x7fb00c06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:12.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.510+0000 7fb00b200640 1 -- 172.21.15.159:0/2512492364 shutdown_connections 2024-04-04T06:12:12.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.510+0000 7fb00b200640 1 -- 172.21.15.159:0/2512492364 wait complete. 2024-04-04T06:12:12.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.512+0000 7fb00b200640 1 Processor -- start 2024-04-04T06:12:12.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.512+0000 7fb00b200640 1 -- start start 2024-04-04T06:12:12.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.513+0000 7fb00b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c072d80 0x7fb00c131eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.513+0000 7fb00b200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb00c144ad0 0x7fb00c12f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.513+0000 7fb00b200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb00c15f710 0x7fb00c130ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.514+0000 7fb00b200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c0b3cb0 con 0x7fb00c072d80 2024-04-04T06:12:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.514+0000 7fb00b200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c0b39d0 con 0x7fb00c144ad0 2024-04-04T06:12:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.514+0000 7fb00b200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb00c0b3f90 con 0x7fb00c15f710 2024-04-04T06:12:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.514+0000 7fb009e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c072d80 0x7fb00c131eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.514+0000 7fb009e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c072d80 0x7fb00c131eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35044/0 (socket says 172.21.15.159:35044) 2024-04-04T06:12:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.514+0000 7fb009e00640 1 -- 172.21.15.159:0/4130778104 learned_addr learned my addr 172.21.15.159:0/4130778104 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.514+0000 7fb001400640 1 --2- 172.21.15.159:0/4130778104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb00c144ad0 0x7fb00c12f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.514+0000 7fb009e00640 1 -- 172.21.15.159:0/4130778104 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb00c15f710 msgr2=0x7fb00c130ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.514+0000 7fb009e00640 1 --2- 172.21.15.159:0/4130778104 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb00c15f710 0x7fb00c130ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.514+0000 7fb009e00640 1 -- 172.21.15.159:0/4130778104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb00c144ad0 msgr2=0x7fb00c12f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.514+0000 7fb009e00640 1 --2- 172.21.15.159:0/4130778104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb00c144ad0 0x7fb00c12f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.514+0000 7fb009e00640 1 -- 172.21.15.159:0/4130778104 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00c12fa40 con 0x7fb00c072d80 2024-04-04T06:12:12.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.514+0000 7fb009e00640 1 --2- 172.21.15.159:0/4130778104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c072d80 0x7fb00c131eb0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7faffc009890 tx=0x7faffc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:12.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.515+0000 7fb002a00640 1 -- 172.21.15.159:0/4130778104 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc041070 con 0x7fb00c072d80 2024-04-04T06:12:12.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.515+0000 7fb002a00640 1 -- 172.21.15.159:0/4130778104 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faffc03b920 con 0x7fb00c072d80 2024-04-04T06:12:12.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.515+0000 7fb002a00640 1 -- 172.21.15.159:0/4130778104 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc035040 con 0x7fb00c072d80 2024-04-04T06:12:12.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.515+0000 7fb00b200640 1 -- 172.21.15.159:0/4130778104 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb00c12fd20 con 0x7fb00c072d80 2024-04-04T06:12:12.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.515+0000 7fb00b200640 1 -- 172.21.15.159:0/4130778104 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb00c1302e0 con 0x7fb00c072d80 2024-04-04T06:12:12.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.516+0000 7fb00b200640 1 -- 172.21.15.159:0/4130778104 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb00c075940 con 0x7fb00c072d80 2024-04-04T06:12:12.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.517+0000 7fb002a00640 1 -- 172.21.15.159:0/4130778104 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faffc038020 con 0x7fb00c072d80 2024-04-04T06:12:12.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.518+0000 7fb002a00640 1 --2- 172.21.15.159:0/4130778104 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fafe806c8f0 0x7fafe806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:12.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.518+0000 7fb002a00640 1 -- 172.21.15.159:0/4130778104 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7faffc0ad610 con 0x7fb00c072d80 2024-04-04T06:12:12.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.519+0000 7fb001400640 1 --2- 172.21.15.159:0/4130778104 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fafe806c8f0 0x7fafe806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:12.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.520+0000 7fb001400640 1 --2- 172.21.15.159:0/4130778104 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fafe806c8f0 0x7fafe806edb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb00c1e71a0 tx=0x7faff4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:12.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.525+0000 7fb002a00640 1 -- 172.21.15.159:0/4130778104 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7faffc031800 con 0x7fb00c072d80 2024-04-04T06:12:12.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.764+0000 7fb00b200640 1 -- 172.21.15.159:0/4130778104 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb00c0b4240 con 0x7fb00c072d80 2024-04-04T06:12:12.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.764+0000 7fb002a00640 1 -- 172.21.15.159:0/4130778104 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7faffc07b9a0 con 0x7fb00c072d80 2024-04-04T06:12:12.757 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:12.758 INFO:teuthology.orchestra.run.smithi159.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-04T06:12:12.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.768+0000 7fb00b200640 1 -- 172.21.15.159:0/4130778104 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fafe806c8f0 msgr2=0x7fafe806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:12.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.768+0000 7fb00b200640 1 --2- 172.21.15.159:0/4130778104 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fafe806c8f0 0x7fafe806edb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb00c1e71a0 tx=0x7faff4008040 comp rx=0 tx=0).stop 2024-04-04T06:12:12.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.768+0000 7fb00b200640 1 -- 172.21.15.159:0/4130778104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c072d80 msgr2=0x7fb00c131eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:12.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.768+0000 7fb00b200640 1 --2- 172.21.15.159:0/4130778104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c072d80 0x7fb00c131eb0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7faffc009890 tx=0x7faffc0046c0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.768+0000 7fb00b200640 1 -- 172.21.15.159:0/4130778104 shutdown_connections 2024-04-04T06:12:12.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.768+0000 7fb00b200640 1 --2- 172.21.15.159:0/4130778104 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb00c15f710 0x7fb00c130ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.768+0000 7fb00b200640 1 --2- 172.21.15.159:0/4130778104 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fafe806c8f0 0x7fafe806edb0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.768+0000 7fb00b200640 1 --2- 172.21.15.159:0/4130778104 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb00c144ad0 0x7fb00c12f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.768+0000 7fb00b200640 1 --2- 172.21.15.159:0/4130778104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c072d80 0x7fb00c131eb0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:12.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.768+0000 7fb00b200640 1 -- 172.21.15.159:0/4130778104 >> 172.21.15.159:0/4130778104 conn(0x7fb00c06b570 msgr2=0x7fb00c0cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:12.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.768+0000 7fb00b200640 1 -- 172.21.15.159:0/4130778104 shutdown_connections 2024-04-04T06:12:12.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:12.768+0000 7fb00b200640 1 -- 172.21.15.159:0/4130778104 wait complete. 2024-04-04T06:12:12.785 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-04T06:12:12.785 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-04T06:12:13.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.051+0000 7f71a8600640 1 Processor -- start 2024-04-04T06:12:13.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.052+0000 7f71a8600640 1 -- start start 2024-04-04T06:12:13.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.053+0000 7f71a8600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f71a0155550 0x7f71a0155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:13.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.053+0000 7f71a8600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f71a00d0990 0x7f71a00aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:13.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.053+0000 7f71a8600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71a00b40b0 con 0x7f71a015f800 2024-04-04T06:12:13.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.053+0000 7f71a8600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f71a00b4390 con 0x7f71a00d0990 2024-04-04T06:12:13.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.053+0000 7f71a8600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f71a00b4670 con 0x7f71a0155550 2024-04-04T06:12:13.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.053+0000 7f71a6600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f71a00d0990 0x7f71a00aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:13.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.054+0000 7f71a6600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f71a00d0990 0x7f71a00aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35048/0 (socket says 172.21.15.159:35048) 2024-04-04T06:12:13.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.054+0000 7f71a6600640 1 -- 172.21.15.159:0/1660874540 learned_addr learned my addr 172.21.15.159:0/1660874540 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:13.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.054+0000 7f71a5c00640 1 --2- 172.21.15.159:0/1660874540 >> v2:172.21.15.178:3300/0 conn(0x7f71a0155550 0x7f71a0155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:13.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.054+0000 7f71a6600640 1 -- 172.21.15.159:0/1660874540 >> v2:172.21.15.178:3300/0 conn(0x7f71a0155550 msgr2=0x7f71a0155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:13.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.054+0000 7f71a6600640 1 --2- 172.21.15.159:0/1660874540 >> v2:172.21.15.178:3300/0 conn(0x7f71a0155550 0x7f71a0155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:13.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.054+0000 7f71a6600640 1 -- 172.21.15.159:0/1660874540 >> v1:172.21.15.178:6789/0 conn(0x7f71a015f800 legacy=0x7f71a00d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:13.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.054+0000 7f71a6600640 1 -- 172.21.15.159:0/1660874540 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a00b4950 con 0x7f71a00d0990 2024-04-04T06:12:13.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.055+0000 7f71a6600640 1 --2- 172.21.15.159:0/1660874540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a00d0990 0x7f71a00aede0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f71a00b3610 tx=0x7f719802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60244fe45f9fb8c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:13.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.056+0000 7f719fe00640 1 -- 172.21.15.159:0/1660874540 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719802c9c0 con 0x7f71a00d0990 2024-04-04T06:12:13.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.056+0000 7f719fe00640 1 -- 172.21.15.159:0/1660874540 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f719802cb60 con 0x7f71a00d0990 2024-04-04T06:12:13.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.056+0000 7f719fe00640 1 -- 172.21.15.159:0/1660874540 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71980343f0 con 0x7f71a00d0990 2024-04-04T06:12:13.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.056+0000 7f71a8600640 1 -- 172.21.15.159:0/1660874540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a00d0990 msgr2=0x7f71a00aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:13.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.056+0000 7f71a8600640 1 --2- 172.21.15.159:0/1660874540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a00d0990 0x7f71a00aede0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f71a00b3610 tx=0x7f719802be20 comp rx=0 tx=0).stop 2024-04-04T06:12:13.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.056+0000 7f71a8600640 1 -- 172.21.15.159:0/1660874540 shutdown_connections 2024-04-04T06:12:13.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.056+0000 7f71a8600640 1 --2- 172.21.15.159:0/1660874540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a00d0990 0x7f71a00aede0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:13.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.056+0000 7f71a8600640 1 --2- 172.21.15.159:0/1660874540 >> v2:172.21.15.178:3300/0 conn(0x7f71a0155550 0x7f71a0155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:13.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.056+0000 7f71a8600640 1 -- 172.21.15.159:0/1660874540 >> 172.21.15.159:0/1660874540 conn(0x7f71a006b750 msgr2=0x7f71a006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:13.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.056+0000 7f71a8600640 1 -- 172.21.15.159:0/1660874540 shutdown_connections 2024-04-04T06:12:13.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.057+0000 7f71a8600640 1 -- 172.21.15.159:0/1660874540 wait complete. 2024-04-04T06:12:13.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.058+0000 7f71a8600640 1 Processor -- start 2024-04-04T06:12:13.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.058+0000 7f71a8600640 1 -- start start 2024-04-04T06:12:13.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.059+0000 7f71a8600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71a00d0990 0x7f71a0099a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:13.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.059+0000 7f71a8600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71a0155550 0x7f71a00b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:13.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.059+0000 7f71a8600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a015f800 0x7f71a00cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:13.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.059+0000 7f71a8600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f71a015ba20 con 0x7f71a015f800 2024-04-04T06:12:13.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.059+0000 7f71a8600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f71a015fc80 con 0x7f71a00d0990 2024-04-04T06:12:13.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.059+0000 7f71a8600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f71a015bba0 con 0x7f71a0155550 2024-04-04T06:12:13.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.060+0000 7f71a6600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a015f800 0x7f71a00cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:13.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.060+0000 7f71a6600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a015f800 0x7f71a00cdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35054/0 (socket says 172.21.15.159:35054) 2024-04-04T06:12:13.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.060+0000 7f71a6600640 1 -- 172.21.15.159:0/882652204 learned_addr learned my addr 172.21.15.159:0/882652204 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:13.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.060+0000 7f71a5c00640 1 --2- 172.21.15.159:0/882652204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71a00d0990 0x7f71a0099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:13.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.060+0000 7f71a5200640 1 --2- 172.21.15.159:0/882652204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71a0155550 0x7f71a00b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:13.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.060+0000 7f71a6600640 1 -- 172.21.15.159:0/882652204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71a0155550 msgr2=0x7f71a00b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:13.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.060+0000 7f71a6600640 1 --2- 172.21.15.159:0/882652204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71a0155550 0x7f71a00b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:13.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.060+0000 7f71a6600640 1 -- 172.21.15.159:0/882652204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71a00d0990 msgr2=0x7f71a0099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:13.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.060+0000 7f71a6600640 1 --2- 172.21.15.159:0/882652204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71a00d0990 0x7f71a0099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:13.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.060+0000 7f71a6600640 1 -- 172.21.15.159:0/882652204 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a00ade00 con 0x7f71a015f800 2024-04-04T06:12:13.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.060+0000 7f71a5200640 1 --2- 172.21.15.159:0/882652204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71a0155550 0x7f71a00b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:13.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.061+0000 7f71a6600640 1 --2- 172.21.15.159:0/882652204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a015f800 0x7f71a00cdda0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f7198009f10 tx=0x7f7198004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:13.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.061+0000 7f719d600640 1 -- 172.21.15.159:0/882652204 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198043020 con 0x7f71a015f800 2024-04-04T06:12:13.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.061+0000 7f719d600640 1 -- 172.21.15.159:0/882652204 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f719803e450 con 0x7f71a015f800 2024-04-04T06:12:13.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.061+0000 7f719d600640 1 -- 172.21.15.159:0/882652204 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198009510 con 0x7f71a015f800 2024-04-04T06:12:13.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.061+0000 7f71a8600640 1 -- 172.21.15.159:0/882652204 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a00ae070 con 0x7f71a015f800 2024-04-04T06:12:13.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.061+0000 7f71a8600640 1 -- 172.21.15.159:0/882652204 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a00b0be0 con 0x7f71a015f800 2024-04-04T06:12:13.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.062+0000 7f71a8600640 1 -- 172.21.15.159:0/882652204 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a00685c0 con 0x7f71a015f800 2024-04-04T06:12:13.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.063+0000 7f719d600640 1 -- 172.21.15.159:0/882652204 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7198046020 con 0x7f71a015f800 2024-04-04T06:12:13.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.064+0000 7f719d600640 1 --2- 172.21.15.159:0/882652204 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f718006c8f0 0x7f718006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:13.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.064+0000 7f719d600640 1 -- 172.21.15.159:0/882652204 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f71980ae7e0 con 0x7f71a015f800 2024-04-04T06:12:13.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.065+0000 7f71a5c00640 1 --2- 172.21.15.159:0/882652204 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f718006c8f0 0x7f718006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:13.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.066+0000 7f71a5c00640 1 --2- 172.21.15.159:0/882652204 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f718006c8f0 0x7f718006edb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7194006fd0 tx=0x7f7194008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:13.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.074+0000 7f719d600640 1 -- 172.21.15.159:0/882652204 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f719804d020 con 0x7f71a015f800 2024-04-04T06:12:13.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.316+0000 7f71a8600640 1 -- 172.21.15.159:0/882652204 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f71a00ad2f0 con 0x7f71a015f800 2024-04-04T06:12:13.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.317+0000 7f719d600640 1 -- 172.21.15.159:0/882652204 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) ==== 53+0+73 (secure 0 0 0) 0x7f719807cae0 con 0x7f71a015f800 2024-04-04T06:12:13.309 INFO:teuthology.orchestra.run.smithi159.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-04T06:12:13.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.319+0000 7f71a8600640 1 -- 172.21.15.159:0/882652204 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f718006c8f0 msgr2=0x7f718006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:13.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.319+0000 7f71a8600640 1 --2- 172.21.15.159:0/882652204 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f718006c8f0 0x7f718006edb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7194006fd0 tx=0x7f7194008040 comp rx=0 tx=0).stop 2024-04-04T06:12:13.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.319+0000 7f71a8600640 1 -- 172.21.15.159:0/882652204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a015f800 msgr2=0x7f71a00cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:13.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.319+0000 7f71a8600640 1 --2- 172.21.15.159:0/882652204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a015f800 0x7f71a00cdda0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f7198009f10 tx=0x7f7198004a40 comp rx=0 tx=0).stop 2024-04-04T06:12:13.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.319+0000 7f71a8600640 1 -- 172.21.15.159:0/882652204 shutdown_connections 2024-04-04T06:12:13.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.319+0000 7f71a8600640 1 --2- 172.21.15.159:0/882652204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a015f800 0x7f71a00cdda0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:13.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.319+0000 7f71a8600640 1 --2- 172.21.15.159:0/882652204 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71a0155550 0x7f71a00b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:13.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.319+0000 7f71a8600640 1 --2- 172.21.15.159:0/882652204 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f718006c8f0 0x7f718006edb0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:13.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.319+0000 7f71a8600640 1 --2- 172.21.15.159:0/882652204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71a00d0990 0x7f71a0099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:13.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.319+0000 7f71a8600640 1 -- 172.21.15.159:0/882652204 >> 172.21.15.159:0/882652204 conn(0x7f71a006b750 msgr2=0x7f71a015a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:13.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.319+0000 7f71a8600640 1 -- 172.21.15.159:0/882652204 shutdown_connections 2024-04-04T06:12:13.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13.320+0000 7f71a8600640 1 -- 172.21.15.159:0/882652204 wait complete. 2024-04-04T06:12:13.331 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-04T06:12:13.331 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-04T06:12:13.331 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi159.front.sepia.ceph.com 2024-04-04T06:12:13.331 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-04T06:12:13.331 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-04T06:12:13.331 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-04T06:12:13.331 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-04T06:12:13.331 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-04T06:12:13.331 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:12:13.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:13.332 DEBUG:teuthology.orchestra.run.smithi159:> ip addr 2024-04-04T06:12:13.386 INFO:teuthology.orchestra.run.smithi159.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:12:13.386 INFO:teuthology.orchestra.run.smithi159.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:12:13.386 INFO:teuthology.orchestra.run.smithi159.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:12:13.386 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:12:13.386 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 ::1/128 scope host 2024-04-04T06:12:13.386 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:12:13.386 INFO:teuthology.orchestra.run.smithi159.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:12:13.386 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c6 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:12:13.387 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f0 2024-04-04T06:12:13.387 INFO:teuthology.orchestra.run.smithi159.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:12:13.387 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c7 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:12:13.387 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f1 2024-04-04T06:12:13.387 INFO:teuthology.orchestra.run.smithi159.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:12:13.387 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4e brd ff:ff:ff:ff:ff:ff 2024-04-04T06:12:13.387 INFO:teuthology.orchestra.run.smithi159.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-04T06:12:13.387 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:12:13.387 INFO:teuthology.orchestra.run.smithi159.stdout: inet 172.21.15.159/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-04T06:12:13.387 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft 42199sec preferred_lft 42199sec 2024-04-04T06:12:13.387 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::ec4:7aff:fe88:724f/64 scope link 2024-04-04T06:12:13.387 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:12:13.387 INFO:teuthology.orchestra.run.smithi159.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-04T06:12:13.387 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:12:13.388 INFO:teuthology.orchestra.run.smithi159.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-04T06:12:13.388 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:12:13.388 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::989e:e4ff:fea5:7568/64 scope link 2024-04-04T06:12:13.388 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:12:13.388 INFO:teuthology.orchestra.run.smithi159.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:12:13.388 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether a2:e3:e0:cb:0a:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-04T06:12:13.388 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::a0e3:e0ff:fecb:a73/64 scope link 2024-04-04T06:12:13.388 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:12:13.388 INFO:teuthology.orchestra.run.smithi159.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:12:13.388 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-04T06:12:13.388 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::c0a:8aff:fe54:b58f/64 scope link 2024-04-04T06:12:13.388 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:12:13.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:13.389 DEBUG:teuthology.orchestra.run.smithi159:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-04T06:12:13.471 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:12:13.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:13.478 DEBUG:teuthology.orchestra.run.smithi159:> route 2024-04-04T06:12:13.523 INFO:teuthology.orchestra.run.smithi159.stdout:Kernel IP routing table 2024-04-04T06:12:13.524 INFO:teuthology.orchestra.run.smithi159.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-04T06:12:13.524 INFO:teuthology.orchestra.run.smithi159.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-04T06:12:13.524 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-04T06:12:13.524 INFO:teuthology.orchestra.run.smithi159.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-04T06:12:13.524 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:12:13.524 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:12:13.524 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-04T06:12:13.525 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-04T06:12:13.525 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-04T06:12:13.525 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:12:13.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:12:13.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:12:13.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:13.800 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-04-04T06:12:13.851 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-04T06:12:13.851 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-04T06:12:13.852 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:12:13.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:13.852 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set brx.0 up 2024-04-04T06:12:13.937 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:12:13.937 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-04T06:12:13.982 INFO:teuthology.orchestra.run.smithi159.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:12:13.983 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:12:13.983 DEBUG:teuthology.orchestra.run.smithi159:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:12:14.071 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-04T06:12:14.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:14.072 DEBUG:teuthology.orchestra.run.smithi159:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:12:14.245 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:12:14.245 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:12:14.246 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:12:14.265 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:12:14.265 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:12:14.265 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:12:14.265 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:12:14.265 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:12:14.289 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:12:14.290 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:12:14.290 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:12:14.290 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:12:14.290 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:12:14.290 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-04T06:12:14.290 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-04T06:12:14.290 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30311045-f714-48fa-832b-4cb2ba8bf0bf" will pass to kernel 2024-04-04T06:12:14.291 INFO:teuthology.orchestra.run.smithi159.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-04T06:12:14.291 INFO:teuthology.orchestra.run.smithi159.stdout: You just mounted a file system that supports labels which does not 2024-04-04T06:12:14.291 INFO:teuthology.orchestra.run.smithi159.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-04T06:12:14.291 INFO:teuthology.orchestra.run.smithi159.stdout: applications will generate AVC messages and not be allowed access to 2024-04-04T06:12:14.292 INFO:teuthology.orchestra.run.smithi159.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-04T06:12:14.295 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-04T06:12:14.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:14.295 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:12:14.368 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:12:14.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:12:14.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:12:14.474 INFO:teuthology.orchestra.run.smithi159.stdout:client7110 2024-04-04T06:12:14.474 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:12:14.474 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:12:14.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:14.527 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-04T06:12:14.624 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:12:14.624 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-04T06:12:14.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.726+0000 7fa734ffef40 1 Processor -- start 2024-04-04T06:12:14.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.726+0000 7fa734ffef40 1 -- start start 2024-04-04T06:12:14.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.726+0000 7fa734ffef40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606793c9960 0x56067949fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:14.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.726+0000 7fa734ffef40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5606793f4550 0x5606793f3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:14.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.727+0000 7fa734ffef40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56067949d910 0x5606794a1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:14.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.727+0000 7fa734ffef40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5606793cb670 con 0x56067949d910 2024-04-04T06:12:14.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.727+0000 7fa734ffef40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56067936f2c0 con 0x5606793c9960 2024-04-04T06:12:14.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.727+0000 7fa734ffef40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x560679412210 con 0x5606793f4550 2024-04-04T06:12:14.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.727+0000 7fa732c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56067949d910 0x5606794a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:14.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.727+0000 7fa732c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56067949d910 0x5606794a1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58882/0 (socket says 172.21.15.159:58882) 2024-04-04T06:12:14.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.727+0000 7fa732c00640 1 -- 172.21.15.159:0/199322685 learned_addr learned my addr 172.21.15.159:0/199322685 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:14.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.727+0000 7fa732200640 1 --2- 172.21.15.159:0/199322685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606793c9960 0x56067949fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:14.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.727+0000 7fa729800640 1 --2- 172.21.15.159:0/199322685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5606793f4550 0x5606793f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:14.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.727+0000 7fa732c00640 1 -- 172.21.15.159:0/199322685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5606793f4550 msgr2=0x5606793f3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:14.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.727+0000 7fa732c00640 1 --2- 172.21.15.159:0/199322685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5606793f4550 0x5606793f3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:14.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.727+0000 7fa732c00640 1 -- 172.21.15.159:0/199322685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606793c9960 msgr2=0x56067949fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:14.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.727+0000 7fa732c00640 1 --2- 172.21.15.159:0/199322685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606793c9960 0x56067949fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:14.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.727+0000 7fa732c00640 1 -- 172.21.15.159:0/199322685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606793f3c50 con 0x56067949d910 2024-04-04T06:12:14.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.728+0000 7fa732c00640 1 --2- 172.21.15.159:0/199322685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56067949d910 0x5606794a1640 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fa72c001570 tx=0x7fa72c001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a6bf39e20eff1c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:14.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.728+0000 7fa730e00640 1 -- 172.21.15.159:0/199322685 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c017070 con 0x56067949d910 2024-04-04T06:12:14.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.728+0000 7fa730e00640 1 -- 172.21.15.159:0/199322685 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa72c00f710 con 0x56067949d910 2024-04-04T06:12:14.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.728+0000 7fa730e00640 1 -- 172.21.15.159:0/199322685 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c00fb40 con 0x56067949d910 2024-04-04T06:12:14.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.729+0000 7fa734ffef40 1 -- 172.21.15.159:0/199322685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56067949d910 msgr2=0x5606794a1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:14.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.729+0000 7fa734ffef40 1 --2- 172.21.15.159:0/199322685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56067949d910 0x5606794a1640 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fa72c001570 tx=0x7fa72c001a30 comp rx=0 tx=0).stop 2024-04-04T06:12:14.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.729+0000 7fa734ffef40 1 -- 172.21.15.159:0/199322685 shutdown_connections 2024-04-04T06:12:14.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.729+0000 7fa734ffef40 1 --2- 172.21.15.159:0/199322685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56067949d910 0x5606794a1640 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:14.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.729+0000 7fa734ffef40 1 --2- 172.21.15.159:0/199322685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5606793f4550 0x5606793f3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:14.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.729+0000 7fa734ffef40 1 --2- 172.21.15.159:0/199322685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606793c9960 0x56067949fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:14.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.729+0000 7fa734ffef40 1 -- 172.21.15.159:0/199322685 >> 172.21.15.159:0/199322685 conn(0x5606793f2bf0 msgr2=0x56067949c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:14.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.729+0000 7fa734ffef40 1 -- 172.21.15.159:0/199322685 shutdown_connections 2024-04-04T06:12:14.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.729+0000 7fa734ffef40 1 -- 172.21.15.159:0/199322685 wait complete. 2024-04-04T06:12:14.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.730+0000 7fa734ffef40 1 Processor -- start 2024-04-04T06:12:14.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.731+0000 7fa734ffef40 1 -- start start 2024-04-04T06:12:14.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.732+0000 7fa734ffef40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606793c9960 0x560679489da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:14.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.732+0000 7fa734ffef40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606793f4550 0x56067949ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:14.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.732+0000 7fa734ffef40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56067949d910 0x5606794a7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:14.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.732+0000 7fa734ffef40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5606794a06e0 con 0x5606793f4550 2024-04-04T06:12:14.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.732+0000 7fa734ffef40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5606794a0400 con 0x5606793c9960 2024-04-04T06:12:14.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.732+0000 7fa734ffef40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5606793f3ef0 con 0x56067949d910 2024-04-04T06:12:14.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.732+0000 7fa729800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606793f4550 0x56067949ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:14.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.732+0000 7fa729800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606793f4550 0x56067949ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58892/0 (socket says 172.21.15.159:58892) 2024-04-04T06:12:14.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.732+0000 7fa729800640 1 -- 172.21.15.159:0/2285246655 learned_addr learned my addr 172.21.15.159:0/2285246655 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:14.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.733+0000 7fa732c00640 1 --2- 172.21.15.159:0/2285246655 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56067949d910 0x5606794a7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:14.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.733+0000 7fa732200640 1 --2- 172.21.15.159:0/2285246655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606793c9960 0x560679489da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:14.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.733+0000 7fa729800640 1 -- 172.21.15.159:0/2285246655 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56067949d910 msgr2=0x5606794a7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:14.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.733+0000 7fa729800640 1 --2- 172.21.15.159:0/2285246655 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56067949d910 0x5606794a7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:14.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.733+0000 7fa729800640 1 -- 172.21.15.159:0/2285246655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606793c9960 msgr2=0x560679489da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:14.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.733+0000 7fa729800640 1 --2- 172.21.15.159:0/2285246655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606793c9960 0x560679489da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:14.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.733+0000 7fa729800640 1 -- 172.21.15.159:0/2285246655 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56067948d600 con 0x5606793f4550 2024-04-04T06:12:14.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.733+0000 7fa729800640 1 --2- 172.21.15.159:0/2285246655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606793f4550 0x56067949ba20 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fa71c008430 tx=0x7fa71c008520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:14.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.733+0000 7fa72b400640 1 -- 172.21.15.159:0/2285246655 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71c0145f0 con 0x5606793f4550 2024-04-04T06:12:14.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.733+0000 7fa72b400640 1 -- 172.21.15.159:0/2285246655 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa71c019070 con 0x5606793f4550 2024-04-04T06:12:14.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.733+0000 7fa734ffef40 1 -- 172.21.15.159:0/2285246655 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5606794a1650 con 0x5606793f4550 2024-04-04T06:12:14.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.733+0000 7fa72b400640 1 -- 172.21.15.159:0/2285246655 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71c0149b0 con 0x5606793f4550 2024-04-04T06:12:14.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.734+0000 7fa734ffef40 1 -- 172.21.15.159:0/2285246655 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56067948a2e0 con 0x5606793f4550 2024-04-04T06:12:14.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.736+0000 7fa72b400640 1 -- 172.21.15.159:0/2285246655 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa71c014b50 con 0x5606793f4550 2024-04-04T06:12:14.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.736+0000 7fa72b400640 1 --2- 172.21.15.159:0/2285246655 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa71006b300 0x7fa71006d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:14.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.737+0000 7fa72b400640 1 -- 172.21.15.159:0/2285246655 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7fa71c08a7c0 con 0x5606793f4550 2024-04-04T06:12:14.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.737+0000 7fa734ffef40 1 --2- 172.21.15.159:0/2285246655 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x5606793c81c0 0x56067953d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:14.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.737+0000 7fa734ffef40 1 -- 172.21.15.159:0/2285246655 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74) -- 0x56067949bf60 con 0x5606793c81c0 2024-04-04T06:12:14.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.737+0000 7fa732200640 1 --2- 172.21.15.159:0/2285246655 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa71006b300 0x7fa71006d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:14.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.738+0000 7fa732c00640 1 --2- 172.21.15.159:0/2285246655 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x5606793c81c0 0x56067953d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:14.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.738+0000 7fa732200640 1 --2- 172.21.15.159:0/2285246655 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa71006b300 0x7fa71006d7c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa724054000 tx=0x7fa72404e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:14.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.739+0000 7fa732c00640 1 --2- 172.21.15.159:0/2285246655 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x5606793c81c0 0x56067953d040 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:14.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.742+0000 7fa732c00640 1 -- 172.21.15.159:0/2285246655 <== osd.4 v2:172.21.15.178:6812/4035991429 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v74'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa72c01a040 con 0x5606793c81c0 2024-04-04T06:12:14.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.742+0000 7fa734ffef40 1 -- 172.21.15.159:0/2285246655 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x5606793c81c0 msgr2=0x56067953d040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:14.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.742+0000 7fa734ffef40 1 --2- 172.21.15.159:0/2285246655 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x5606793c81c0 0x56067953d040 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:14.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.743+0000 7fa734ffef40 1 -- 172.21.15.159:0/2285246655 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa71006b300 msgr2=0x7fa71006d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:14.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.743+0000 7fa734ffef40 1 --2- 172.21.15.159:0/2285246655 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa71006b300 0x7fa71006d7c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa724054000 tx=0x7fa72404e000 comp rx=0 tx=0).stop 2024-04-04T06:12:14.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.743+0000 7fa734ffef40 1 -- 172.21.15.159:0/2285246655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606793f4550 msgr2=0x56067949ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:14.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.743+0000 7fa734ffef40 1 --2- 172.21.15.159:0/2285246655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606793f4550 0x56067949ba20 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fa71c008430 tx=0x7fa71c008520 comp rx=0 tx=0).stop 2024-04-04T06:12:14.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.743+0000 7fa732c00640 1 -- 172.21.15.159:0/2285246655 reap_dead start 2024-04-04T06:12:14.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.743+0000 7fa734ffef40 1 -- 172.21.15.159:0/2285246655 shutdown_connections 2024-04-04T06:12:14.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.743+0000 7fa734ffef40 1 -- 172.21.15.159:0/2285246655 >> 172.21.15.159:0/2285246655 conn(0x5606793f2bf0 msgr2=0x5606793f22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:14.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.743+0000 7fa734ffef40 1 -- 172.21.15.159:0/2285246655 shutdown_connections 2024-04-04T06:12:14.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:14.743+0000 7fa734ffef40 1 -- 172.21.15.159:0/2285246655 wait complete. 2024-04-04T06:12:14.741 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-04T06:12:14.741 DEBUG:teuthology.orchestra.run.smithi159:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:12:14.842 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:12:14.961 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-04T06:12:14.961 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:12:14.961 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:12:14.961 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:12:15.015 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:12:15.015 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:12:15.065 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:12:15.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.327+0000 7f6169600640 1 Processor -- start 2024-04-04T06:12:15.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.327+0000 7f6169600640 1 -- start start 2024-04-04T06:12:15.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.328+0000 7f6169600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f616415c390 0x7f6164073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:15.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.329+0000 7f6169600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f61640d0650 0x7f61640d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:15.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.329+0000 7f6169600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f616415fe80 con 0x7f61640af830 2024-04-04T06:12:15.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.329+0000 7f6169600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f61640d1000 con 0x7f61640d0650 2024-04-04T06:12:15.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.329+0000 7f6169600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f616415c770 con 0x7f616415c390 2024-04-04T06:12:15.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.329+0000 7f6163400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f61640d0650 0x7f61640d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:15.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.329+0000 7f6163400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f61640d0650 0x7f61640d0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58894/0 (socket says 172.21.15.159:58894) 2024-04-04T06:12:15.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.329+0000 7f6163400640 1 -- 172.21.15.159:0/2008778741 learned_addr learned my addr 172.21.15.159:0/2008778741 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:15.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.329+0000 7f6162a00640 1 --2- 172.21.15.159:0/2008778741 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f616415c390 0x7f6164073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:15.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.329+0000 7f6163400640 1 -- 172.21.15.159:0/2008778741 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f616415c390 msgr2=0x7f6164073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:15.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.329+0000 7f6163400640 1 --2- 172.21.15.159:0/2008778741 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f616415c390 0x7f6164073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:15.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.330+0000 7f6163400640 1 -- 172.21.15.159:0/2008778741 >> v1:172.21.15.178:6789/0 conn(0x7f61640af830 legacy=0x7f6164160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:15.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.330+0000 7f6163400640 1 -- 172.21.15.159:0/2008778741 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61640ad120 con 0x7f61640d0650 2024-04-04T06:12:15.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.331+0000 7f6163400640 1 --2- 172.21.15.159:0/2008778741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61640d0650 0x7f61640d0ac0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f615c009870 tx=0x7f615c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1baebcbe6b481c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:15.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.331+0000 7f6161600640 1 -- 172.21.15.159:0/2008778741 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c034070 con 0x7f61640d0650 2024-04-04T06:12:15.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.331+0000 7f6161600640 1 -- 172.21.15.159:0/2008778741 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f615c030840 con 0x7f61640d0650 2024-04-04T06:12:15.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.331+0000 7f6161600640 1 -- 172.21.15.159:0/2008778741 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c030c10 con 0x7f61640d0650 2024-04-04T06:12:15.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.332+0000 7f6169600640 1 -- 172.21.15.159:0/2008778741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61640d0650 msgr2=0x7f61640d0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:15.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.332+0000 7f6169600640 1 --2- 172.21.15.159:0/2008778741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61640d0650 0x7f61640d0ac0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f615c009870 tx=0x7f615c02b980 comp rx=0 tx=0).stop 2024-04-04T06:12:15.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.332+0000 7f6169600640 1 -- 172.21.15.159:0/2008778741 shutdown_connections 2024-04-04T06:12:15.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.332+0000 7f6169600640 1 --2- 172.21.15.159:0/2008778741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61640d0650 0x7f61640d0ac0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:15.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.332+0000 7f6169600640 1 --2- 172.21.15.159:0/2008778741 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f616415c390 0x7f6164073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:15.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.332+0000 7f6169600640 1 -- 172.21.15.159:0/2008778741 >> 172.21.15.159:0/2008778741 conn(0x7f616406b990 msgr2=0x7f616406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:15.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.332+0000 7f6169600640 1 -- 172.21.15.159:0/2008778741 shutdown_connections 2024-04-04T06:12:15.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.332+0000 7f6169600640 1 -- 172.21.15.159:0/2008778741 wait complete. 2024-04-04T06:12:15.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.333+0000 7f6169600640 1 Processor -- start 2024-04-04T06:12:15.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.334+0000 7f6169600640 1 -- start start 2024-04-04T06:12:15.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.334+0000 7f6169600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61640af830 0x7f616415e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:15.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.334+0000 7f6169600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61640d0650 0x7f616415c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:15.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.335+0000 7f6169600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f616415c390 0x7f61640b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:15.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.335+0000 7f6169600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6164160120 con 0x7f61640af830 2024-04-04T06:12:15.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.335+0000 7f6169600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f61640ad6e0 con 0x7f616415c390 2024-04-04T06:12:15.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.335+0000 7f6169600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6164160d50 con 0x7f61640d0650 2024-04-04T06:12:15.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.335+0000 7f6162a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61640af830 0x7f616415e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:15.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.335+0000 7f6162a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61640af830 0x7f616415e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58908/0 (socket says 172.21.15.159:58908) 2024-04-04T06:12:15.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.335+0000 7f6162a00640 1 -- 172.21.15.159:0/750182427 learned_addr learned my addr 172.21.15.159:0/750182427 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:15.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.335+0000 7f6162a00640 1 -- 172.21.15.159:0/750182427 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61640d0650 msgr2=0x7f616415c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:15.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.335+0000 7f6162a00640 1 --2- 172.21.15.159:0/750182427 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61640d0650 0x7f616415c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:15.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.335+0000 7f6162a00640 1 -- 172.21.15.159:0/750182427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f616415c390 msgr2=0x7f61640b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:15.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.335+0000 7f6162a00640 1 --2- 172.21.15.159:0/750182427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f616415c390 0x7f61640b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:15.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.335+0000 7f6162a00640 1 -- 172.21.15.159:0/750182427 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616415d1e0 con 0x7f61640af830 2024-04-04T06:12:15.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.335+0000 7f6162a00640 1 --2- 172.21.15.159:0/750182427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61640af830 0x7f616415e060 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f615400d800 tx=0x7f615400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:15.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.336+0000 7f615aa00640 1 -- 172.21.15.159:0/750182427 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615400b6c0 con 0x7f61640af830 2024-04-04T06:12:15.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.336+0000 7f615aa00640 1 -- 172.21.15.159:0/750182427 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6154019070 con 0x7f61640af830 2024-04-04T06:12:15.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.336+0000 7f615aa00640 1 -- 172.21.15.159:0/750182427 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615400ba80 con 0x7f61640af830 2024-04-04T06:12:15.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.336+0000 7f6169600640 1 -- 172.21.15.159:0/750182427 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f616415d4c0 con 0x7f61640af830 2024-04-04T06:12:15.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.336+0000 7f6169600640 1 -- 172.21.15.159:0/750182427 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f616415d7a0 con 0x7f61640af830 2024-04-04T06:12:15.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.337+0000 7f6169600640 1 -- 172.21.15.159:0/750182427 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61640685c0 con 0x7f61640af830 2024-04-04T06:12:15.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.338+0000 7f615aa00640 1 -- 172.21.15.159:0/750182427 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f615400bc20 con 0x7f61640af830 2024-04-04T06:12:15.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.339+0000 7f615aa00640 1 --2- 172.21.15.159:0/750182427 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f614006c8f0 0x7f614006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:15.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.340+0000 7f615aa00640 1 -- 172.21.15.159:0/750182427 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f615408c0f0 con 0x7f61640af830 2024-04-04T06:12:15.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.340+0000 7f6162000640 1 --2- 172.21.15.159:0/750182427 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f614006c8f0 0x7f614006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:15.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.341+0000 7f6162000640 1 --2- 172.21.15.159:0/750182427 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f614006c8f0 0x7f614006edb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f614c0012e0 tx=0x7f614c007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:15.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.343+0000 7f615aa00640 1 -- 172.21.15.159:0/750182427 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f615405a480 con 0x7f61640af830 2024-04-04T06:12:15.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.543+0000 7f6169600640 1 -- 172.21.15.159:0/750182427 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f616406f1b0 con 0x7f61640af830 2024-04-04T06:12:15.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.545+0000 7f615aa00640 1 -- 172.21.15.159:0/750182427 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16909 (secure 0 0 0) 0x7f6154059d80 con 0x7f61640af830 2024-04-04T06:12:15.538 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:15.539 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":74,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:11:35.095911+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:10:42.595868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-04T06:10:43.788847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.20","mappings":[{"from":6,"to":0}]},{"pgid":"11.3a","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6835/1652468640":"2024-04-05T06:10:57.934622+0000","172.21.15.159:6834/1652468640":"2024-04-05T06:10:57.934622+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:12:15.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.547+0000 7f6169600640 1 -- 172.21.15.159:0/750182427 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f614006c8f0 msgr2=0x7f614006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:15.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.547+0000 7f6169600640 1 --2- 172.21.15.159:0/750182427 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f614006c8f0 0x7f614006edb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f614c0012e0 tx=0x7f614c007d50 comp rx=0 tx=0).stop 2024-04-04T06:12:15.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.547+0000 7f6169600640 1 -- 172.21.15.159:0/750182427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61640af830 msgr2=0x7f616415e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:15.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.547+0000 7f6169600640 1 --2- 172.21.15.159:0/750182427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61640af830 0x7f616415e060 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f615400d800 tx=0x7f615400dcc0 comp rx=0 tx=0).stop 2024-04-04T06:12:15.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.547+0000 7f6169600640 1 -- 172.21.15.159:0/750182427 shutdown_connections 2024-04-04T06:12:15.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.547+0000 7f6169600640 1 --2- 172.21.15.159:0/750182427 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f614006c8f0 0x7f614006edb0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:15.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.548+0000 7f6169600640 1 --2- 172.21.15.159:0/750182427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f616415c390 0x7f61640b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:15.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.548+0000 7f6169600640 1 --2- 172.21.15.159:0/750182427 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61640d0650 0x7f616415c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:15.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.548+0000 7f6169600640 1 --2- 172.21.15.159:0/750182427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61640af830 0x7f616415e060 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:15.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.548+0000 7f6169600640 1 -- 172.21.15.159:0/750182427 >> 172.21.15.159:0/750182427 conn(0x7f616406b990 msgr2=0x7f616415ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:15.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.548+0000 7f6169600640 1 -- 172.21.15.159:0/750182427 shutdown_connections 2024-04-04T06:12:15.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.548+0000 7f6169600640 1 -- 172.21.15.159:0/750182427 wait complete. 2024-04-04T06:12:15.561 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:10:42.595868+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:10:43.788847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:12:15.561 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:12:15.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.780+0000 7f5e7ec00640 1 Processor -- start 2024-04-04T06:12:15.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.780+0000 7f5e7ec00640 1 -- start start 2024-04-04T06:12:15.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.782+0000 7f5e7ec00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5e780b69c0 0x7f5e78168f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:15.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.782+0000 7f5e7ec00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e780d2710 con 0x7f5e7815e4c0 2024-04-04T06:12:15.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.782+0000 7f5e7ec00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e780d29f0 con 0x7f5e780d37b0 2024-04-04T06:12:15.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.782+0000 7f5e7ec00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f5e7815e8a0 con 0x7f5e780b69c0 2024-04-04T06:12:15.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.782+0000 7f5e7d800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f5e7815e4c0 0x7f5e780b9bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:47286/0 (socket says 172.21.15.159:47286) 2024-04-04T06:12:15.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.782+0000 7f5e7d800640 1 -- 172.21.15.159:0/1959917603 learned_addr learned my addr 172.21.15.159:0/1959917603 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:15.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.783+0000 7f5e77e00640 1 -- 172.21.15.159:0/1959917603 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4281503386 0 0) 0x7f5e780d2710 con 0x7f5e7815e4c0 2024-04-04T06:12:15.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.783+0000 7f5e7e200640 1 --2- 172.21.15.159:0/1959917603 >> v2:172.21.15.178:3300/0 conn(0x7f5e780b69c0 0x7f5e78168f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:15.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.783+0000 7f5e77e00640 1 -- 172.21.15.159:0/1959917603 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e60003520 con 0x7f5e7815e4c0 2024-04-04T06:12:15.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.783+0000 7f5e77e00640 1 -- 172.21.15.159:0/1959917603 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 965466110 0 0) 0x7f5e60003520 con 0x7f5e7815e4c0 2024-04-04T06:12:15.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.783+0000 7f5e77e00640 1 -- 172.21.15.159:0/1959917603 >> v2:172.21.15.178:3300/0 conn(0x7f5e780b69c0 msgr2=0x7f5e78168f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:15.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.783+0000 7f5e77e00640 1 --2- 172.21.15.159:0/1959917603 >> v2:172.21.15.178:3300/0 conn(0x7f5e780b69c0 0x7f5e78168f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:15.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.783+0000 7f5e77e00640 1 -- 172.21.15.159:0/1959917603 >> v1:172.21.15.178:6789/0 conn(0x7f5e780d37b0 legacy=0x7f5e780b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:15.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.783+0000 7f5e77e00640 1 -- 172.21.15.159:0/1959917603 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e780d2cd0 con 0x7f5e7815e4c0 2024-04-04T06:12:15.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.783+0000 7f5e77e00640 1 -- 172.21.15.159:0/1959917603 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f5e6c0019f0 con 0x7f5e7815e4c0 2024-04-04T06:12:15.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.783+0000 7f5e77e00640 1 -- 172.21.15.159:0/1959917603 >> v1:172.21.15.159:6789/0 conn(0x7f5e7815e4c0 legacy=0x7f5e780b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:15.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.783+0000 7f5e77e00640 1 --2- 172.21.15.159:0/1959917603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e600039b0 0x7f5e60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:15.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.783+0000 7f5e77e00640 1 --2- 172.21.15.159:0/1959917603 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e60004800 0x7f5e60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:15.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.783+0000 7f5e77e00640 1 --2- 172.21.15.159:0/1959917603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e600051c0 0x7f5e6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:15.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.784+0000 7f5e77e00640 1 -- 172.21.15.159:0/1959917603 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6000c030 con 0x7f5e600051c0 2024-04-04T06:12:15.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.784+0000 7f5e77e00640 1 -- 172.21.15.159:0/1959917603 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6000c1e0 con 0x7f5e600039b0 2024-04-04T06:12:15.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.784+0000 7f5e77e00640 1 -- 172.21.15.159:0/1959917603 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5e6000c3e0 con 0x7f5e60004800 2024-04-04T06:12:15.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.784+0000 7f5e7e200640 1 --2- 172.21.15.159:0/1959917603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e600051c0 0x7f5e6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:15.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.784+0000 7f5e7d800640 1 --2- 172.21.15.159:0/1959917603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e600039b0 0x7f5e60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:15.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.784+0000 7f5e7ce00640 1 --2- 172.21.15.159:0/1959917603 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e60004800 0x7f5e60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:15.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.784+0000 7f5e7e200640 1 -- 172.21.15.159:0/1959917603 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e60004800 msgr2=0x7f5e60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:15.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.784+0000 7f5e7e200640 1 --2- 172.21.15.159:0/1959917603 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e60004800 0x7f5e60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:15.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.784+0000 7f5e7e200640 1 -- 172.21.15.159:0/1959917603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e600039b0 msgr2=0x7f5e60003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:15.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.784+0000 7f5e7e200640 1 --2- 172.21.15.159:0/1959917603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e600039b0 0x7f5e60003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:15.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.784+0000 7f5e7e200640 1 -- 172.21.15.159:0/1959917603 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e60003520 con 0x7f5e600051c0 2024-04-04T06:12:15.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.784+0000 7f5e7d800640 1 --2- 172.21.15.159:0/1959917603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e600039b0 0x7f5e60003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:12:15.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.784+0000 7f5e7ce00640 1 --2- 172.21.15.159:0/1959917603 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e60004800 0x7f5e60004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:15.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.785+0000 7f5e7e200640 1 -- 172.21.15.159:0/1959917603 reap_dead start 2024-04-04T06:12:15.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.785+0000 7f5e7e200640 1 --2- 172.21.15.159:0/1959917603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e600051c0 0x7f5e6000b9c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009a90 tx=0x7f5e60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701c4346de9dd84 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:15.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.786+0000 7f5e77e00640 1 -- 172.21.15.159:0/1959917603 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70039070 con 0x7f5e600051c0 2024-04-04T06:12:15.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.786+0000 7f5e77e00640 1 -- 172.21.15.159:0/1959917603 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e7002c6d0 con 0x7f5e600051c0 2024-04-04T06:12:15.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.786+0000 7f5e77e00640 1 -- 172.21.15.159:0/1959917603 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70009690 con 0x7f5e600051c0 2024-04-04T06:12:15.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.786+0000 7f5e7ec00640 1 -- 172.21.15.159:0/1959917603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e600051c0 msgr2=0x7f5e6000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:15.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.786+0000 7f5e7ec00640 1 --2- 172.21.15.159:0/1959917603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e600051c0 0x7f5e6000b9c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009a90 tx=0x7f5e60005830 comp rx=0 tx=0).stop 2024-04-04T06:12:15.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.786+0000 7f5e7ec00640 1 -- 172.21.15.159:0/1959917603 shutdown_connections 2024-04-04T06:12:15.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.786+0000 7f5e7ec00640 1 --2- 172.21.15.159:0/1959917603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e600051c0 0x7f5e6000b9c0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:15.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.786+0000 7f5e7ec00640 1 -- 172.21.15.159:0/1959917603 >> 172.21.15.159:0/1959917603 conn(0x7f5e780676a0 msgr2=0x7f5e78067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:15.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.786+0000 7f5e7ec00640 1 -- 172.21.15.159:0/1959917603 shutdown_connections 2024-04-04T06:12:15.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.786+0000 7f5e7ec00640 1 -- 172.21.15.159:0/1959917603 wait complete. 2024-04-04T06:12:15.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.788+0000 7f5e7ec00640 1 Processor -- start 2024-04-04T06:12:15.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.788+0000 7f5e7ec00640 1 -- start start 2024-04-04T06:12:15.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.789+0000 7f5e7ec00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e600051c0 0x7f5e78150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:15.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.789+0000 7f5e7ec00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e78154740 0x7f5e7814fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:15.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.789+0000 7f5e7ec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e78160390 0x7f5e78160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:15.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.789+0000 7f5e7ec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5e780d3320 con 0x7f5e78160390 2024-04-04T06:12:15.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.789+0000 7f5e7ec00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5e780d4040 con 0x7f5e600051c0 2024-04-04T06:12:15.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.789+0000 7f5e7ec00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5e780d2350 con 0x7f5e78154740 2024-04-04T06:12:15.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.790+0000 7f5e7e200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e78160390 0x7f5e78160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:15.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.790+0000 7f5e7e200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e78160390 0x7f5e78160810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58928/0 (socket says 172.21.15.159:58928) 2024-04-04T06:12:15.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.790+0000 7f5e7e200640 1 -- 172.21.15.159:0/3648002004 learned_addr learned my addr 172.21.15.159:0/3648002004 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:15.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.790+0000 7f5e7d800640 1 --2- 172.21.15.159:0/3648002004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e600051c0 0x7f5e78150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:15.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.790+0000 7f5e7ce00640 1 --2- 172.21.15.159:0/3648002004 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e78154740 0x7f5e7814fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:15.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.790+0000 7f5e7e200640 1 -- 172.21.15.159:0/3648002004 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e78154740 msgr2=0x7f5e7814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:15.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.790+0000 7f5e7e200640 1 --2- 172.21.15.159:0/3648002004 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e78154740 0x7f5e7814fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:15.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.790+0000 7f5e7e200640 1 -- 172.21.15.159:0/3648002004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e600051c0 msgr2=0x7f5e78150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:15.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.790+0000 7f5e7e200640 1 --2- 172.21.15.159:0/3648002004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e600051c0 0x7f5e78150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:15.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.790+0000 7f5e7e200640 1 -- 172.21.15.159:0/3648002004 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e78150240 con 0x7f5e78160390 2024-04-04T06:12:15.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.791+0000 7f5e7e200640 1 --2- 172.21.15.159:0/3648002004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e78160390 0x7f5e78160810 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009bb0 tx=0x7f5e70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:15.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.791+0000 7f5e75600640 1 -- 172.21.15.159:0/3648002004 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70046070 con 0x7f5e78160390 2024-04-04T06:12:15.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.791+0000 7f5e75600640 1 -- 172.21.15.159:0/3648002004 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e7003e680 con 0x7f5e78160390 2024-04-04T06:12:15.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.791+0000 7f5e75600640 1 -- 172.21.15.159:0/3648002004 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70039040 con 0x7f5e78160390 2024-04-04T06:12:15.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.791+0000 7f5e7ec00640 1 -- 172.21.15.159:0/3648002004 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e781f8540 con 0x7f5e78160390 2024-04-04T06:12:15.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.791+0000 7f5e7ec00640 1 -- 172.21.15.159:0/3648002004 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e781f7a60 con 0x7f5e78160390 2024-04-04T06:12:15.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.793+0000 7f5e7ec00640 1 -- 172.21.15.159:0/3648002004 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e78154b20 con 0x7f5e78160390 2024-04-04T06:12:15.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.793+0000 7f5e75600640 1 -- 172.21.15.159:0/3648002004 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e70049070 con 0x7f5e78160390 2024-04-04T06:12:15.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.794+0000 7f5e75600640 1 --2- 172.21.15.159:0/3648002004 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5e5806c8f0 0x7f5e5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:15.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.795+0000 7f5e75600640 1 -- 172.21.15.159:0/3648002004 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f5e700ac9a0 con 0x7f5e78160390 2024-04-04T06:12:15.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.795+0000 7f5e7d800640 1 --2- 172.21.15.159:0/3648002004 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5e5806c8f0 0x7f5e5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:15.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.796+0000 7f5e7d800640 1 --2- 172.21.15.159:0/3648002004 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5e5806c8f0 0x7f5e5806edb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c005d70 tx=0x7f5e6c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:15.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:15.800+0000 7f5e75600640 1 -- 172.21.15.159:0/3648002004 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5e7007a660 con 0x7f5e78160390 2024-04-04T06:12:15.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.002+0000 7f5e7ec00640 1 -- 172.21.15.159:0/3648002004 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5e78075940 con 0x7f5e78160390 2024-04-04T06:12:15.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.003+0000 7f5e75600640 1 -- 172.21.15.159:0/3648002004 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7f5e7007e960 con 0x7f5e78160390 2024-04-04T06:12:15.996 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:12:15.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.006+0000 7f5e7ec00640 1 -- 172.21.15.159:0/3648002004 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5e5806c8f0 msgr2=0x7f5e5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:15.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.006+0000 7f5e7ec00640 1 --2- 172.21.15.159:0/3648002004 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5e5806c8f0 0x7f5e5806edb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c005d70 tx=0x7f5e6c009040 comp rx=0 tx=0).stop 2024-04-04T06:12:15.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.006+0000 7f5e7ec00640 1 -- 172.21.15.159:0/3648002004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e78160390 msgr2=0x7f5e78160810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:15.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.006+0000 7f5e7ec00640 1 --2- 172.21.15.159:0/3648002004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e78160390 0x7f5e78160810 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009bb0 tx=0x7f5e70004730 comp rx=0 tx=0).stop 2024-04-04T06:12:15.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.006+0000 7f5e7ec00640 1 -- 172.21.15.159:0/3648002004 shutdown_connections 2024-04-04T06:12:15.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.006+0000 7f5e7ec00640 1 --2- 172.21.15.159:0/3648002004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e78160390 0x7f5e78160810 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:15.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.006+0000 7f5e7ec00640 1 --2- 172.21.15.159:0/3648002004 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e78154740 0x7f5e7814fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:15.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.006+0000 7f5e7ec00640 1 --2- 172.21.15.159:0/3648002004 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5e5806c8f0 0x7f5e5806edb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:15.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.006+0000 7f5e7ec00640 1 --2- 172.21.15.159:0/3648002004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e600051c0 0x7f5e78150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:15.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.006+0000 7f5e7ec00640 1 -- 172.21.15.159:0/3648002004 >> 172.21.15.159:0/3648002004 conn(0x7f5e780676a0 msgr2=0x7f5e780b5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:15.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.006+0000 7f5e7ec00640 1 -- 172.21.15.159:0/3648002004 shutdown_connections 2024-04-04T06:12:16.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.006+0000 7f5e7ec00640 1 -- 172.21.15.159:0/3648002004 wait complete. 2024-04-04T06:12:16.017 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:12:16.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.218+0000 7f60b6800640 1 Processor -- start 2024-04-04T06:12:16.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.218+0000 7f60b6800640 1 -- start start 2024-04-04T06:12:16.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.220+0000 7f60b6800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f60b0145cd0 0x7f60b00c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:16.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.220+0000 7f60b6800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60b00c74c0 0x7f60b015fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:16.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.220+0000 7f60b6800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60b00d2400 con 0x7f60b01603d0 2024-04-04T06:12:16.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.220+0000 7f60b6800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f60b01460b0 con 0x7f60b0145cd0 2024-04-04T06:12:16.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.220+0000 7f60b6800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f60b00b5240 con 0x7f60b00c74c0 2024-04-04T06:12:16.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.220+0000 7f60b5e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f60b01603d0 0x7f60b00cffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:47300/0 (socket says 172.21.15.159:47300) 2024-04-04T06:12:16.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.220+0000 7f60b5e00640 1 -- 172.21.15.159:0/2807097466 learned_addr learned my addr 172.21.15.159:0/2807097466 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:16.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.220+0000 7f60b5400640 1 --2- 172.21.15.159:0/2807097466 >> v2:172.21.15.159:3300/0 conn(0x7f60b0145cd0 0x7f60b00c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:16.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.221+0000 7f60b4a00640 1 --2- 172.21.15.159:0/2807097466 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60b00c74c0 0x7f60b015fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:16.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.221+0000 7f60afe00640 1 -- 172.21.15.159:0/2807097466 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 394460149 0 0) 0x7f60b00d2400 con 0x7f60b01603d0 2024-04-04T06:12:16.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.221+0000 7f60afe00640 1 -- 172.21.15.159:0/2807097466 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6098003520 con 0x7f60b01603d0 2024-04-04T06:12:16.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.221+0000 7f60b5400640 1 -- 172.21.15.159:0/2807097466 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60b00c74c0 msgr2=0x7f60b015fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:16.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.221+0000 7f60b5400640 1 --2- 172.21.15.159:0/2807097466 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60b00c74c0 0x7f60b015fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:16.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.221+0000 7f60b5400640 1 -- 172.21.15.159:0/2807097466 >> v1:172.21.15.159:6789/0 conn(0x7f60b01603d0 legacy=0x7f60b00cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:16.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.221+0000 7f60b5400640 1 -- 172.21.15.159:0/2807097466 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b00d26e0 con 0x7f60b0145cd0 2024-04-04T06:12:16.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.222+0000 7f60b5400640 1 --2- 172.21.15.159:0/2807097466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b0145cd0 0x7f60b00c6f80 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f60a40097a0 tx=0x7f60a402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db42a54e7be6dc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:16.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.223+0000 7f60afe00640 1 -- 172.21.15.159:0/2807097466 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4035020 con 0x7f60b0145cd0 2024-04-04T06:12:16.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.223+0000 7f60afe00640 1 -- 172.21.15.159:0/2807097466 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a4031830 con 0x7f60b0145cd0 2024-04-04T06:12:16.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.223+0000 7f60afe00640 1 -- 172.21.15.159:0/2807097466 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4031c00 con 0x7f60b0145cd0 2024-04-04T06:12:16.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.223+0000 7f60b6800640 1 -- 172.21.15.159:0/2807097466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b0145cd0 msgr2=0x7f60b00c6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:16.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.223+0000 7f60b6800640 1 --2- 172.21.15.159:0/2807097466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b0145cd0 0x7f60b00c6f80 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f60a40097a0 tx=0x7f60a402b8d0 comp rx=0 tx=0).stop 2024-04-04T06:12:16.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.223+0000 7f60b6800640 1 -- 172.21.15.159:0/2807097466 shutdown_connections 2024-04-04T06:12:16.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.223+0000 7f60b6800640 1 --2- 172.21.15.159:0/2807097466 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60b00c74c0 0x7f60b015fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:16.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.223+0000 7f60b6800640 1 --2- 172.21.15.159:0/2807097466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b0145cd0 0x7f60b00c6f80 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:16.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.223+0000 7f60b6800640 1 -- 172.21.15.159:0/2807097466 >> 172.21.15.159:0/2807097466 conn(0x7f60b0067280 msgr2=0x7f60b0067660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:16.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.223+0000 7f60b6800640 1 -- 172.21.15.159:0/2807097466 shutdown_connections 2024-04-04T06:12:16.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.223+0000 7f60b6800640 1 -- 172.21.15.159:0/2807097466 wait complete. 2024-04-04T06:12:16.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.225+0000 7f60b6800640 1 Processor -- start 2024-04-04T06:12:16.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.225+0000 7f60b6800640 1 -- start start 2024-04-04T06:12:16.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.226+0000 7f60b6800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60b00c74c0 0x7f60b0140b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:16.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.226+0000 7f60b6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b0145cd0 0x7f60b015f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:16.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.226+0000 7f60b6800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60b01603d0 0x7f60b015f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:16.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.226+0000 7f60b6800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f60b00b66e0 con 0x7f60b0145cd0 2024-04-04T06:12:16.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.226+0000 7f60b6800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0160bc0 con 0x7f60b01603d0 2024-04-04T06:12:16.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.226+0000 7f60b6800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f60b00b6860 con 0x7f60b00c74c0 2024-04-04T06:12:16.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.227+0000 7f60b4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b0145cd0 0x7f60b015f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:16.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.227+0000 7f60b4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b0145cd0 0x7f60b015f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58934/0 (socket says 172.21.15.159:58934) 2024-04-04T06:12:16.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.227+0000 7f60b4a00640 1 -- 172.21.15.159:0/4119158806 learned_addr learned my addr 172.21.15.159:0/4119158806 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:16.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.227+0000 7f60b5e00640 1 --2- 172.21.15.159:0/4119158806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60b01603d0 0x7f60b015f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:16.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.227+0000 7f60b5400640 1 --2- 172.21.15.159:0/4119158806 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60b00c74c0 0x7f60b0140b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:16.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.227+0000 7f60b4a00640 1 -- 172.21.15.159:0/4119158806 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60b00c74c0 msgr2=0x7f60b0140b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:16.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.227+0000 7f60b4a00640 1 --2- 172.21.15.159:0/4119158806 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60b00c74c0 0x7f60b0140b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:16.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.227+0000 7f60b4a00640 1 -- 172.21.15.159:0/4119158806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60b01603d0 msgr2=0x7f60b015f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:16.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.227+0000 7f60b4a00640 1 --2- 172.21.15.159:0/4119158806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60b01603d0 0x7f60b015f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:16.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.227+0000 7f60b4a00640 1 -- 172.21.15.159:0/4119158806 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b00caa00 con 0x7f60b0145cd0 2024-04-04T06:12:16.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.227+0000 7f60b4a00640 1 --2- 172.21.15.159:0/4119158806 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b0145cd0 0x7f60b015f1e0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f609c00e870 tx=0x7f609c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:16.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.228+0000 7f60ad600640 1 -- 172.21.15.159:0/4119158806 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c017070 con 0x7f60b0145cd0 2024-04-04T06:12:16.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.228+0000 7f60ad600640 1 -- 172.21.15.159:0/4119158806 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f609c00c940 con 0x7f60b0145cd0 2024-04-04T06:12:16.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.228+0000 7f60ad600640 1 -- 172.21.15.159:0/4119158806 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c00cd30 con 0x7f60b0145cd0 2024-04-04T06:12:16.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.228+0000 7f60b6800640 1 -- 172.21.15.159:0/4119158806 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b014e800 con 0x7f60b0145cd0 2024-04-04T06:12:16.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.228+0000 7f60b6800640 1 -- 172.21.15.159:0/4119158806 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b014bec0 con 0x7f60b0145cd0 2024-04-04T06:12:16.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.229+0000 7f60b6800640 1 -- 172.21.15.159:0/4119158806 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b0075940 con 0x7f60b0145cd0 2024-04-04T06:12:16.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.230+0000 7f60ad600640 1 -- 172.21.15.159:0/4119158806 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f609c01a070 con 0x7f60b0145cd0 2024-04-04T06:12:16.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.231+0000 7f60ad600640 1 --2- 172.21.15.159:0/4119158806 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f609006c8f0 0x7f609006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:16.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.231+0000 7f60ad600640 1 -- 172.21.15.159:0/4119158806 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f609c08ba20 con 0x7f60b0145cd0 2024-04-04T06:12:16.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.232+0000 7f60b5400640 1 --2- 172.21.15.159:0/4119158806 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f609006c8f0 0x7f609006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:16.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.233+0000 7f60b5400640 1 --2- 172.21.15.159:0/4119158806 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f609006c8f0 0x7f609006edb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f60a4036000 tx=0x7f60a403c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:16.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.235+0000 7f60ad600640 1 -- 172.21.15.159:0/4119158806 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f60b00caa00 con 0x7f60b0145cd0 2024-04-04T06:12:16.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.431+0000 7f60b6800640 1 -- 172.21.15.159:0/4119158806 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f60b01601b0 con 0x7f60b0145cd0 2024-04-04T06:12:16.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.432+0000 7f60ad600640 1 -- 172.21.15.159:0/4119158806 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) ==== 104+0+11 (secure 0 0 0) 0x7f609c059e50 con 0x7f60b0145cd0 2024-04-04T06:12:16.425 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:12:16.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.434+0000 7f60b6800640 1 -- 172.21.15.159:0/4119158806 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f609006c8f0 msgr2=0x7f609006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:16.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.435+0000 7f60b6800640 1 --2- 172.21.15.159:0/4119158806 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f609006c8f0 0x7f609006edb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f60a4036000 tx=0x7f60a403c040 comp rx=0 tx=0).stop 2024-04-04T06:12:16.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.435+0000 7f60b6800640 1 -- 172.21.15.159:0/4119158806 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b0145cd0 msgr2=0x7f60b015f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:16.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.435+0000 7f60b6800640 1 --2- 172.21.15.159:0/4119158806 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b0145cd0 0x7f60b015f1e0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f609c00e870 tx=0x7f609c00ed30 comp rx=0 tx=0).stop 2024-04-04T06:12:16.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.435+0000 7f60b6800640 1 -- 172.21.15.159:0/4119158806 shutdown_connections 2024-04-04T06:12:16.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.435+0000 7f60b6800640 1 --2- 172.21.15.159:0/4119158806 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f609006c8f0 0x7f609006edb0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:16.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.435+0000 7f60b6800640 1 --2- 172.21.15.159:0/4119158806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60b01603d0 0x7f60b015f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:16.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.435+0000 7f60b6800640 1 --2- 172.21.15.159:0/4119158806 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b0145cd0 0x7f60b015f1e0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:16.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.435+0000 7f60b6800640 1 --2- 172.21.15.159:0/4119158806 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60b00c74c0 0x7f60b0140b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:16.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.435+0000 7f60b6800640 1 -- 172.21.15.159:0/4119158806 >> 172.21.15.159:0/4119158806 conn(0x7f60b0067280 msgr2=0x7f60b015ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:16.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.435+0000 7f60b6800640 1 -- 172.21.15.159:0/4119158806 shutdown_connections 2024-04-04T06:12:16.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.435+0000 7f60b6800640 1 -- 172.21.15.159:0/4119158806 wait complete. 2024-04-04T06:12:16.452 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:12:16.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.667+0000 7feeac200640 1 Processor -- start 2024-04-04T06:12:16.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.667+0000 7feeac200640 1 -- start start 2024-04-04T06:12:16.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.669+0000 7feeac200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feea4153580 0x7feea40d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:16.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.669+0000 7feeac200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feea40af4b0 0x7feea41600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:16.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.669+0000 7feeac200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feea40afab0 con 0x7feea40d07c0 2024-04-04T06:12:16.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.669+0000 7feeac200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7feea40d0c30 con 0x7feea4153580 2024-04-04T06:12:16.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.669+0000 7feeac200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feea415cc80 con 0x7feea40af4b0 2024-04-04T06:12:16.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.669+0000 7feea9800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feea4153580 0x7feea40d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:16.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.669+0000 7feeaa200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feea40af4b0 0x7feea41600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:16.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.670+0000 7feea9800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feea4153580 0x7feea40d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:43500/0 (socket says 172.21.15.159:43500) 2024-04-04T06:12:16.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.670+0000 7feea9800640 1 -- 172.21.15.159:0/2587925152 learned_addr learned my addr 172.21.15.159:0/2587925152 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:16.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.670+0000 7feea9800640 1 -- 172.21.15.159:0/2587925152 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feea40af4b0 msgr2=0x7feea41600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:16.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.670+0000 7feea9800640 1 --2- 172.21.15.159:0/2587925152 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feea40af4b0 0x7feea41600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:16.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.670+0000 7feea9800640 1 -- 172.21.15.159:0/2587925152 >> v1:172.21.15.178:6789/0 conn(0x7feea40d07c0 legacy=0x7feea40aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:16.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.670+0000 7feea9800640 1 -- 172.21.15.159:0/2587925152 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea415da00 con 0x7feea4153580 2024-04-04T06:12:16.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.672+0000 7feea9800640 1 --2- 172.21.15.159:0/2587925152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feea4153580 0x7feea40d0280 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fee98009870 tx=0x7fee9802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe2a18f202f2e2cc server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:16.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.672+0000 7feea3e00640 1 -- 172.21.15.159:0/2587925152 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee98035020 con 0x7feea4153580 2024-04-04T06:12:16.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.672+0000 7feea3e00640 1 -- 172.21.15.159:0/2587925152 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee98031640 con 0x7feea4153580 2024-04-04T06:12:16.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.672+0000 7feea3e00640 1 -- 172.21.15.159:0/2587925152 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9802d040 con 0x7feea4153580 2024-04-04T06:12:16.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.672+0000 7feeac200640 1 -- 172.21.15.159:0/2587925152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feea4153580 msgr2=0x7feea40d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:16.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.672+0000 7feeac200640 1 --2- 172.21.15.159:0/2587925152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feea4153580 0x7feea40d0280 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fee98009870 tx=0x7fee9802b8f0 comp rx=0 tx=0).stop 2024-04-04T06:12:16.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.673+0000 7feeac200640 1 -- 172.21.15.159:0/2587925152 shutdown_connections 2024-04-04T06:12:16.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.673+0000 7feeac200640 1 --2- 172.21.15.159:0/2587925152 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feea40af4b0 0x7feea41600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:16.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.673+0000 7feeac200640 1 --2- 172.21.15.159:0/2587925152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feea4153580 0x7feea40d0280 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:16.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.673+0000 7feeac200640 1 -- 172.21.15.159:0/2587925152 >> 172.21.15.159:0/2587925152 conn(0x7feea406b750 msgr2=0x7feea406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:16.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.673+0000 7feeac200640 1 -- 172.21.15.159:0/2587925152 shutdown_connections 2024-04-04T06:12:16.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.673+0000 7feeac200640 1 -- 172.21.15.159:0/2587925152 wait complete. 2024-04-04T06:12:16.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.675+0000 7feeac200640 1 Processor -- start 2024-04-04T06:12:16.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.676+0000 7feeac200640 1 -- start start 2024-04-04T06:12:16.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.677+0000 7feeac200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feea40af4b0 0x7feea41f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:16.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.677+0000 7feeac200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feea40d07c0 0x7feea40cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:16.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.677+0000 7feeac200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feea4153580 0x7feea41ee3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:16.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.677+0000 7feeac200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feea415d4e0 con 0x7feea40d07c0 2024-04-04T06:12:16.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.677+0000 7feeac200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feea415d200 con 0x7feea4153580 2024-04-04T06:12:16.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.677+0000 7feeac200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feea4153960 con 0x7feea40af4b0 2024-04-04T06:12:16.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.677+0000 7feea8e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feea40d07c0 0x7feea40cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:16.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.678+0000 7feea8e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feea40d07c0 0x7feea40cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58948/0 (socket says 172.21.15.159:58948) 2024-04-04T06:12:16.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.678+0000 7feea8e00640 1 -- 172.21.15.159:0/1535699560 learned_addr learned my addr 172.21.15.159:0/1535699560 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:16.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.678+0000 7feeaa200640 1 --2- 172.21.15.159:0/1535699560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feea4153580 0x7feea41ee3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:16.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.678+0000 7feea9800640 1 --2- 172.21.15.159:0/1535699560 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feea40af4b0 0x7feea41f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:16.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.678+0000 7feea8e00640 1 -- 172.21.15.159:0/1535699560 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feea40af4b0 msgr2=0x7feea41f4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:16.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.678+0000 7feea8e00640 1 --2- 172.21.15.159:0/1535699560 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feea40af4b0 0x7feea41f4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:16.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.678+0000 7feea8e00640 1 -- 172.21.15.159:0/1535699560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feea4153580 msgr2=0x7feea41ee3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:16.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.678+0000 7feea8e00640 1 --2- 172.21.15.159:0/1535699560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feea4153580 0x7feea41ee3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:16.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.678+0000 7feea8e00640 1 -- 172.21.15.159:0/1535699560 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea40b2e30 con 0x7feea40d07c0 2024-04-04T06:12:16.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.678+0000 7feea8e00640 1 --2- 172.21.15.159:0/1535699560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feea40d07c0 0x7feea40cca50 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fee9000c3c0 tx=0x7fee9000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:16.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.679+0000 7feea1600640 1 -- 172.21.15.159:0/1535699560 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee900052d0 con 0x7feea40d07c0 2024-04-04T06:12:16.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.679+0000 7feea1600640 1 -- 172.21.15.159:0/1535699560 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee90005490 con 0x7feea40d07c0 2024-04-04T06:12:16.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.679+0000 7feea1600640 1 -- 172.21.15.159:0/1535699560 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee90007260 con 0x7feea40d07c0 2024-04-04T06:12:16.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.679+0000 7feeac200640 1 -- 172.21.15.159:0/1535699560 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea40cd180 con 0x7feea40d07c0 2024-04-04T06:12:16.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.679+0000 7feeac200640 1 -- 172.21.15.159:0/1535699560 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea40cd740 con 0x7feea40d07c0 2024-04-04T06:12:16.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.680+0000 7feeac200640 1 -- 172.21.15.159:0/1535699560 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea40685c0 con 0x7feea40d07c0 2024-04-04T06:12:16.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.681+0000 7feea1600640 1 -- 172.21.15.159:0/1535699560 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee90007400 con 0x7feea40d07c0 2024-04-04T06:12:16.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.682+0000 7feea1600640 1 --2- 172.21.15.159:0/1535699560 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fee8406c920 0x7fee8406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:16.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.682+0000 7feea1600640 1 -- 172.21.15.159:0/1535699560 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7fee9008b130 con 0x7feea40d07c0 2024-04-04T06:12:16.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.682+0000 7feea9800640 1 --2- 172.21.15.159:0/1535699560 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fee8406c920 0x7fee8406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:16.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.683+0000 7feea9800640 1 --2- 172.21.15.159:0/1535699560 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fee8406c920 0x7fee8406ede0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fee98036000 tx=0x7fee98008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:16.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.686+0000 7feea1600640 1 -- 172.21.15.159:0/1535699560 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7feea40b2e30 con 0x7feea40d07c0 2024-04-04T06:12:16.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.885+0000 7feeac200640 1 -- 172.21.15.159:0/1535699560 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7feea406eca0 con 0x7feea40d07c0 2024-04-04T06:12:16.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.886+0000 7feea1600640 1 -- 172.21.15.159:0/1535699560 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) ==== 100+0+11 (secure 0 0 0) 0x7fee90059430 con 0x7feea40d07c0 2024-04-04T06:12:16.879 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:12:16.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.888+0000 7feeac200640 1 -- 172.21.15.159:0/1535699560 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fee8406c920 msgr2=0x7fee8406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:16.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.888+0000 7feeac200640 1 --2- 172.21.15.159:0/1535699560 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fee8406c920 0x7fee8406ede0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fee98036000 tx=0x7fee98008030 comp rx=0 tx=0).stop 2024-04-04T06:12:16.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.888+0000 7feeac200640 1 -- 172.21.15.159:0/1535699560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feea40d07c0 msgr2=0x7feea40cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:16.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.888+0000 7feeac200640 1 --2- 172.21.15.159:0/1535699560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feea40d07c0 0x7feea40cca50 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fee9000c3c0 tx=0x7fee9000c880 comp rx=0 tx=0).stop 2024-04-04T06:12:16.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.888+0000 7feeac200640 1 -- 172.21.15.159:0/1535699560 shutdown_connections 2024-04-04T06:12:16.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.888+0000 7feeac200640 1 --2- 172.21.15.159:0/1535699560 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fee8406c920 0x7fee8406ede0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:16.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.888+0000 7feeac200640 1 --2- 172.21.15.159:0/1535699560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feea4153580 0x7feea41ee3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:16.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.888+0000 7feeac200640 1 --2- 172.21.15.159:0/1535699560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feea40d07c0 0x7feea40cca50 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:16.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.888+0000 7feeac200640 1 --2- 172.21.15.159:0/1535699560 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feea40af4b0 0x7feea41f4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:16.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.888+0000 7feeac200640 1 -- 172.21.15.159:0/1535699560 >> 172.21.15.159:0/1535699560 conn(0x7feea406b750 msgr2=0x7feea415a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:16.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.889+0000 7feeac200640 1 -- 172.21.15.159:0/1535699560 shutdown_connections 2024-04-04T06:12:16.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:16.889+0000 7feeac200640 1 -- 172.21.15.159:0/1535699560 wait complete. 2024-04-04T06:12:16.900 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-04T06:12:17.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.108+0000 7f0a1ac00640 1 Processor -- start 2024-04-04T06:12:17.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.108+0000 7f0a1ac00640 1 -- start start 2024-04-04T06:12:17.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.110+0000 7f0a1ac00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a140cc0b0 0x7f0a1406dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:17.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.111+0000 7f0a1ac00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0a1415fc50 0x7f0a140cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:17.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.111+0000 7f0a1ac00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a1406e4f0 con 0x7f0a140cc490 2024-04-04T06:12:17.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.111+0000 7f0a1ac00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0a1415c610 con 0x7f0a1415fc50 2024-04-04T06:12:17.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.111+0000 7f0a1ac00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0a140b3150 con 0x7f0a140cc0b0 2024-04-04T06:12:17.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.111+0000 7f0a19800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a140cc0b0 0x7f0a1406dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:17.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.111+0000 7f0a18e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0a1415fc50 0x7f0a140cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:17.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.111+0000 7f0a19800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a140cc0b0 0x7f0a1406dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:50622/0 (socket says 172.21.15.159:50622) 2024-04-04T06:12:17.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.111+0000 7f0a18e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0a1415fc50 0x7f0a140cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:43520/0 (socket says 172.21.15.159:43520) 2024-04-04T06:12:17.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.111+0000 7f0a19800640 1 -- 172.21.15.159:0/2176077816 learned_addr learned my addr 172.21.15.159:0/2176077816 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:17.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.112+0000 7f0a19800640 1 -- 172.21.15.159:0/2176077816 >> v2:172.21.15.178:3300/0 conn(0x7f0a1415fc50 msgr2=0x7f0a140cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:17.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.112+0000 7f0a19800640 1 --2- 172.21.15.159:0/2176077816 >> v2:172.21.15.178:3300/0 conn(0x7f0a1415fc50 0x7f0a140cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:17.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.112+0000 7f0a19800640 1 -- 172.21.15.159:0/2176077816 >> v1:172.21.15.178:6789/0 conn(0x7f0a140cc490 legacy=0x7f0a1415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:17.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.112+0000 7f0a19800640 1 -- 172.21.15.159:0/2176077816 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a141600e0 con 0x7f0a140cc0b0 2024-04-04T06:12:17.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.112+0000 7f0a18e00640 1 --2- 172.21.15.159:0/2176077816 >> v2:172.21.15.178:3300/0 conn(0x7f0a1415fc50 0x7f0a140cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:12:17.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.114+0000 7f0a19800640 1 --2- 172.21.15.159:0/2176077816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a140cc0b0 0x7f0a1406dfb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009870 tx=0x7f0a0802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b824135930572e72 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:17.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.114+0000 7f0a13e00640 1 -- 172.21.15.159:0/2176077816 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a08034070 con 0x7f0a140cc0b0 2024-04-04T06:12:17.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.114+0000 7f0a13e00640 1 -- 172.21.15.159:0/2176077816 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a08030840 con 0x7f0a140cc0b0 2024-04-04T06:12:17.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.114+0000 7f0a13e00640 1 -- 172.21.15.159:0/2176077816 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a08030c10 con 0x7f0a140cc0b0 2024-04-04T06:12:17.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.115+0000 7f0a1ac00640 1 -- 172.21.15.159:0/2176077816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a140cc0b0 msgr2=0x7f0a1406dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:17.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.115+0000 7f0a1ac00640 1 --2- 172.21.15.159:0/2176077816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a140cc0b0 0x7f0a1406dfb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009870 tx=0x7f0a0802b980 comp rx=0 tx=0).stop 2024-04-04T06:12:17.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.115+0000 7f0a1ac00640 1 -- 172.21.15.159:0/2176077816 shutdown_connections 2024-04-04T06:12:17.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.115+0000 7f0a1ac00640 1 --2- 172.21.15.159:0/2176077816 >> v2:172.21.15.178:3300/0 conn(0x7f0a1415fc50 0x7f0a140cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:17.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.115+0000 7f0a1ac00640 1 --2- 172.21.15.159:0/2176077816 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a140cc0b0 0x7f0a1406dfb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:17.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.115+0000 7f0a1ac00640 1 -- 172.21.15.159:0/2176077816 >> 172.21.15.159:0/2176077816 conn(0x7f0a140ccad0 msgr2=0x7f0a140cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:17.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.115+0000 7f0a1ac00640 1 -- 172.21.15.159:0/2176077816 shutdown_connections 2024-04-04T06:12:17.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.116+0000 7f0a1ac00640 1 -- 172.21.15.159:0/2176077816 wait complete. 2024-04-04T06:12:17.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.119+0000 7f0a1ac00640 1 Processor -- start 2024-04-04T06:12:17.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.119+0000 7f0a1ac00640 1 -- start start 2024-04-04T06:12:17.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.121+0000 7f0a1ac00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a140cc0b0 0x7f0a141559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:17.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.121+0000 7f0a1ac00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a140cc490 0x7f0a14158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:17.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.121+0000 7f0a1ac00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a14158cc0 0x7f0a1415a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:17.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.121+0000 7f0a1ac00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0a14160980 con 0x7f0a140cc490 2024-04-04T06:12:17.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.121+0000 7f0a1ac00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0a141606a0 con 0x7f0a140cc0b0 2024-04-04T06:12:17.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.121+0000 7f0a1ac00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0a140d0300 con 0x7f0a14158cc0 2024-04-04T06:12:17.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.121+0000 7f0a18e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a140cc490 0x7f0a14158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:17.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.121+0000 7f0a18e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a140cc490 0x7f0a14158780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58956/0 (socket says 172.21.15.159:58956) 2024-04-04T06:12:17.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.122+0000 7f0a18e00640 1 -- 172.21.15.159:0/2917769316 learned_addr learned my addr 172.21.15.159:0/2917769316 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:17.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.122+0000 7f0a19800640 1 --2- 172.21.15.159:0/2917769316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a140cc0b0 0x7f0a141559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:17.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.122+0000 7f0a1a200640 1 --2- 172.21.15.159:0/2917769316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a14158cc0 0x7f0a1415a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:17.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.122+0000 7f0a18e00640 1 -- 172.21.15.159:0/2917769316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a14158cc0 msgr2=0x7f0a1415a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:17.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.122+0000 7f0a18e00640 1 --2- 172.21.15.159:0/2917769316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a14158cc0 0x7f0a1415a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:17.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.122+0000 7f0a18e00640 1 -- 172.21.15.159:0/2917769316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a140cc0b0 msgr2=0x7f0a141559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:17.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.122+0000 7f0a18e00640 1 --2- 172.21.15.159:0/2917769316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a140cc0b0 0x7f0a141559a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:17.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.122+0000 7f0a18e00640 1 -- 172.21.15.159:0/2917769316 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a14159240 con 0x7f0a140cc490 2024-04-04T06:12:17.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.122+0000 7f0a18e00640 1 --2- 172.21.15.159:0/2917769316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a140cc490 0x7f0a14158780 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000e990 tx=0x7f0a0000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:17.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.123+0000 7f0a11600640 1 -- 172.21.15.159:0/2917769316 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a00017070 con 0x7f0a140cc490 2024-04-04T06:12:17.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.123+0000 7f0a11600640 1 -- 172.21.15.159:0/2917769316 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a0000c980 con 0x7f0a140cc490 2024-04-04T06:12:17.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.123+0000 7f0a11600640 1 -- 172.21.15.159:0/2917769316 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0000cd50 con 0x7f0a140cc490 2024-04-04T06:12:17.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.123+0000 7f0a1ac00640 1 -- 172.21.15.159:0/2917769316 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a14159520 con 0x7f0a140cc490 2024-04-04T06:12:17.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.123+0000 7f0a1ac00640 1 -- 172.21.15.159:0/2917769316 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a140a9970 con 0x7f0a140cc490 2024-04-04T06:12:17.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.124+0000 7f0a1ac00640 1 -- 172.21.15.159:0/2917769316 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a1415f9f0 con 0x7f0a140cc490 2024-04-04T06:12:17.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.125+0000 7f0a11600640 1 -- 172.21.15.159:0/2917769316 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a0001a070 con 0x7f0a140cc490 2024-04-04T06:12:17.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.126+0000 7f0a11600640 1 --2- 172.21.15.159:0/2917769316 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f09f406c920 0x7f09f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:17.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.126+0000 7f0a11600640 1 -- 172.21.15.159:0/2917769316 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f0a0008bd50 con 0x7f0a140cc490 2024-04-04T06:12:17.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.127+0000 7f0a19800640 1 --2- 172.21.15.159:0/2917769316 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f09f406c920 0x7f09f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:17.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.128+0000 7f0a19800640 1 --2- 172.21.15.159:0/2917769316 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f09f406c920 0x7f09f406ede0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0a08036000 tx=0x7f0a0803b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:17.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.131+0000 7f0a11600640 1 -- 172.21.15.159:0/2917769316 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0a0008f030 con 0x7f0a140cc490 2024-04-04T06:12:17.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.343+0000 7f0a1ac00640 1 -- 172.21.15.159:0/2917769316 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0a140a9480 con 0x7f0a140cc490 2024-04-04T06:12:17.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.344+0000 7f0a11600640 1 -- 172.21.15.159:0/2917769316 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v74) ==== 80+0+124 (secure 0 0 0) 0x7f0a0005a0f0 con 0x7f0a140cc490 2024-04-04T06:12:17.337 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6835/1652468640 2024-04-05T06:10:57.934622+0000 2024-04-04T06:12:17.337 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6834/1652468640 2024-04-05T06:10:57.934622+0000 2024-04-04T06:12:17.337 INFO:teuthology.orchestra.run.smithi159.stderr:listed 2 entries 2024-04-04T06:12:17.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.346+0000 7f0a1ac00640 1 -- 172.21.15.159:0/2917769316 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f09f406c920 msgr2=0x7f09f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:17.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.346+0000 7f0a1ac00640 1 --2- 172.21.15.159:0/2917769316 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f09f406c920 0x7f09f406ede0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0a08036000 tx=0x7f0a0803b040 comp rx=0 tx=0).stop 2024-04-04T06:12:17.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.346+0000 7f0a1ac00640 1 -- 172.21.15.159:0/2917769316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a140cc490 msgr2=0x7f0a14158780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:17.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.346+0000 7f0a1ac00640 1 --2- 172.21.15.159:0/2917769316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a140cc490 0x7f0a14158780 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000e990 tx=0x7f0a0000ee50 comp rx=0 tx=0).stop 2024-04-04T06:12:17.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.346+0000 7f0a1ac00640 1 -- 172.21.15.159:0/2917769316 shutdown_connections 2024-04-04T06:12:17.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.346+0000 7f0a1ac00640 1 --2- 172.21.15.159:0/2917769316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a14158cc0 0x7f0a1415a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:17.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.346+0000 7f0a1ac00640 1 --2- 172.21.15.159:0/2917769316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a140cc490 0x7f0a14158780 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:17.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.346+0000 7f0a1ac00640 1 --2- 172.21.15.159:0/2917769316 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f09f406c920 0x7f09f406ede0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:17.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.346+0000 7f0a1ac00640 1 --2- 172.21.15.159:0/2917769316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a140cc0b0 0x7f0a141559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:17.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.346+0000 7f0a1ac00640 1 -- 172.21.15.159:0/2917769316 >> 172.21.15.159:0/2917769316 conn(0x7f0a140ccad0 msgr2=0x7f0a1415e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:17.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.347+0000 7f0a1ac00640 1 -- 172.21.15.159:0/2917769316 shutdown_connections 2024-04-04T06:12:17.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.347+0000 7f0a1ac00640 1 -- 172.21.15.159:0/2917769316 wait complete. 2024-04-04T06:12:17.366 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-04T06:12:17.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:17.366 DEBUG:teuthology.orchestra.run.smithi159:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-04T06:12:17.456 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:12:17.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:17.456 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-04T06:12:17.501 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:12:17.501 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:12:17.501 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:12:17.560 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:12:17.560 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:12:17.561 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:12:17.561 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:12:17.612 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:12:17.612 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:12:17.612 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:12:17.612 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:12:17.669 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:12:17.670 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:12:17.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.894+0000 7f441d800640 1 Processor -- start 2024-04-04T06:12:17.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.895+0000 7f441d800640 1 -- start start 2024-04-04T06:12:17.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.896+0000 7f441d800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f441815fc50 0x7f44180cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:17.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.896+0000 7f441d800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44180b3150 con 0x7f44180cc490 2024-04-04T06:12:17.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.896+0000 7f441d800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44181600e0 con 0x7f44180cc0b0 2024-04-04T06:12:17.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.896+0000 7f441d800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f441815c610 con 0x7f441815fc50 2024-04-04T06:12:17.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.897+0000 7f441ce00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f44180cc490 0x7f441815ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:47310/0 (socket says 172.21.15.159:47310) 2024-04-04T06:12:17.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.897+0000 7f441ce00640 1 -- 172.21.15.159:0/1852969217 learned_addr learned my addr 172.21.15.159:0/1852969217 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:17.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.897+0000 7f440f400640 1 --2- 172.21.15.159:0/1852969217 >> v2:172.21.15.159:3300/0 conn(0x7f441815fc50 0x7f44180cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:17.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.897+0000 7f4417400640 1 -- 172.21.15.159:0/1852969217 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3206801707 0 0) 0x7f44180b3150 con 0x7f44180cc490 2024-04-04T06:12:17.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.897+0000 7f4417400640 1 -- 172.21.15.159:0/1852969217 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43fc003520 con 0x7f44180cc490 2024-04-04T06:12:17.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.897+0000 7f440f400640 1 -- 172.21.15.159:0/1852969217 >> v1:172.21.15.178:6789/0 conn(0x7f44180cc0b0 legacy=0x7f441806dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:17.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.897+0000 7f440f400640 1 -- 172.21.15.159:0/1852969217 >> v1:172.21.15.159:6789/0 conn(0x7f44180cc490 legacy=0x7f441815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:17.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.897+0000 7f440f400640 1 -- 172.21.15.159:0/1852969217 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44181603c0 con 0x7f441815fc50 2024-04-04T06:12:17.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.898+0000 7f440f400640 1 --2- 172.21.15.159:0/1852969217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f441815fc50 0x7f44180cfdc0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f44000097a0 tx=0x7f440002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f83525a8f1c2a59 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:17.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.899+0000 7f4417400640 1 -- 172.21.15.159:0/1852969217 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400034070 con 0x7f441815fc50 2024-04-04T06:12:17.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.899+0000 7f4417400640 1 -- 172.21.15.159:0/1852969217 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4400030610 con 0x7f441815fc50 2024-04-04T06:12:17.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.899+0000 7f4417400640 1 -- 172.21.15.159:0/1852969217 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44000309e0 con 0x7f441815fc50 2024-04-04T06:12:17.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.899+0000 7f441d800640 1 -- 172.21.15.159:0/1852969217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f441815fc50 msgr2=0x7f44180cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:17.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.899+0000 7f441d800640 1 --2- 172.21.15.159:0/1852969217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f441815fc50 0x7f44180cfdc0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f44000097a0 tx=0x7f440002b960 comp rx=0 tx=0).stop 2024-04-04T06:12:17.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.899+0000 7f441d800640 1 -- 172.21.15.159:0/1852969217 shutdown_connections 2024-04-04T06:12:17.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.899+0000 7f441d800640 1 --2- 172.21.15.159:0/1852969217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f441815fc50 0x7f44180cfdc0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:17.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.899+0000 7f441d800640 1 -- 172.21.15.159:0/1852969217 >> 172.21.15.159:0/1852969217 conn(0x7f44180ccad0 msgr2=0x7f44180cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:17.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.899+0000 7f441d800640 1 -- 172.21.15.159:0/1852969217 shutdown_connections 2024-04-04T06:12:17.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.899+0000 7f441d800640 1 -- 172.21.15.159:0/1852969217 wait complete. 2024-04-04T06:12:17.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.902+0000 7f441d800640 1 Processor -- start 2024-04-04T06:12:17.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.902+0000 7f441d800640 1 -- start start 2024-04-04T06:12:17.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.904+0000 7f441d800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44180cc0b0 0x7f44181f4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:17.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.904+0000 7f441d800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44180cc490 0x7f44181f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:17.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.904+0000 7f441d800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f441815b130 0x7f44181f58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:17.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.904+0000 7f441d800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f44180d0300 con 0x7f44180cc490 2024-04-04T06:12:17.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.904+0000 7f441d800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4418160980 con 0x7f44180cc0b0 2024-04-04T06:12:17.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.904+0000 7f441d800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f44180d0480 con 0x7f441815b130 2024-04-04T06:12:17.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.904+0000 7f440f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44180cc490 0x7f44181f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:17.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.904+0000 7f440f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44180cc490 0x7f44181f5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58986/0 (socket says 172.21.15.159:58986) 2024-04-04T06:12:17.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.904+0000 7f440f400640 1 -- 172.21.15.159:0/4213250127 learned_addr learned my addr 172.21.15.159:0/4213250127 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:17.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.905+0000 7f4417e00640 1 --2- 172.21.15.159:0/4213250127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44180cc0b0 0x7f44181f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:17.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.905+0000 7f440f400640 1 -- 172.21.15.159:0/4213250127 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f441815b130 msgr2=0x7f44181f58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:17.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.905+0000 7f440f400640 1 --2- 172.21.15.159:0/4213250127 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f441815b130 0x7f44181f58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:17.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.905+0000 7f440f400640 1 -- 172.21.15.159:0/4213250127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44180cc0b0 msgr2=0x7f44181f4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:17.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.905+0000 7f440f400640 1 --2- 172.21.15.159:0/4213250127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44180cc0b0 0x7f44181f4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:17.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.905+0000 7f440f400640 1 -- 172.21.15.159:0/4213250127 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44180cdf30 con 0x7f44180cc490 2024-04-04T06:12:17.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.905+0000 7f4417e00640 1 --2- 172.21.15.159:0/4213250127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44180cc0b0 0x7f44181f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:17.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.905+0000 7f440f400640 1 --2- 172.21.15.159:0/4213250127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44180cc490 0x7f44181f5390 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f440002bf70 tx=0x7f44000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:17.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.905+0000 7f4414c00640 1 -- 172.21.15.159:0/4213250127 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400034040 con 0x7f44180cc490 2024-04-04T06:12:17.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.905+0000 7f4414c00640 1 -- 172.21.15.159:0/4213250127 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f440003aaa0 con 0x7f44180cc490 2024-04-04T06:12:17.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.905+0000 7f4414c00640 1 -- 172.21.15.159:0/4213250127 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44000443f0 con 0x7f44180cc490 2024-04-04T06:12:17.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.905+0000 7f441d800640 1 -- 172.21.15.159:0/4213250127 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44180ce210 con 0x7f44180cc490 2024-04-04T06:12:17.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.906+0000 7f441d800640 1 -- 172.21.15.159:0/4213250127 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44180cdbc0 con 0x7f44180cc490 2024-04-04T06:12:17.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.907+0000 7f441d800640 1 -- 172.21.15.159:0/4213250127 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4418068e80 con 0x7f44180cc490 2024-04-04T06:12:17.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.909+0000 7f4414c00640 1 -- 172.21.15.159:0/4213250127 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f440004b060 con 0x7f44180cc490 2024-04-04T06:12:17.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.909+0000 7f4414c00640 1 --2- 172.21.15.159:0/4213250127 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43f406c8f0 0x7f43f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:17.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.910+0000 7f4414c00640 1 -- 172.21.15.159:0/4213250127 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f44000ad970 con 0x7f44180cc490 2024-04-04T06:12:17.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.910+0000 7f4417e00640 1 --2- 172.21.15.159:0/4213250127 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43f406c8f0 0x7f43f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:17.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.911+0000 7f4417e00640 1 --2- 172.21.15.159:0/4213250127 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43f406c8f0 0x7f43f406edb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f441813ebb0 tx=0x7f4408009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:17.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:17.915+0000 7f4414c00640 1 -- 172.21.15.159:0/4213250127 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4418068e80 con 0x7f44180cc490 2024-04-04T06:12:18.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.153+0000 7f441d800640 1 -- 172.21.15.159:0/4213250127 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44181f41d0 con 0x7f44180cc490 2024-04-04T06:12:18.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.154+0000 7f4414c00640 1 -- 172.21.15.159:0/4213250127 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5194 (secure 0 0 0) 0x7f440003f070 con 0x7f44180cc490 2024-04-04T06:12:18.147 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:18.147 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3789513463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3789513463},{"type":"v1","addr":"172.21.15.178:6835","nonce":3789513463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/468875883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":468875883},{"type":"v1","addr":"172.21.15.178:6837","nonce":468875883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2546974742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2546974742},{"type":"v1","addr":"172.21.15.159:6833","nonce":2546974742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:10:44.805617+0000","modified":"2024-04-04T06:11:44.957858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6828":{"gid":6828,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":15,"addr":"172.21.15.159:6835/2467594024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2467594024},{"type":"v1","addr":"172.21.15.159:6835","nonce":2467594024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6828,"qdb_cluster":[6828]},"id":5}]} 2024-04-04T06:12:18.148 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 53 2024-04-04T06:12:18.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.157+0000 7f441d800640 1 -- 172.21.15.159:0/4213250127 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43f406c8f0 msgr2=0x7f43f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:18.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.157+0000 7f441d800640 1 --2- 172.21.15.159:0/4213250127 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43f406c8f0 0x7f43f406edb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f441813ebb0 tx=0x7f4408009040 comp rx=0 tx=0).stop 2024-04-04T06:12:18.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.157+0000 7f441d800640 1 -- 172.21.15.159:0/4213250127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44180cc490 msgr2=0x7f44181f5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:18.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.157+0000 7f441d800640 1 --2- 172.21.15.159:0/4213250127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44180cc490 0x7f44181f5390 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f440002bf70 tx=0x7f44000046c0 comp rx=0 tx=0).stop 2024-04-04T06:12:18.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.157+0000 7f441d800640 1 -- 172.21.15.159:0/4213250127 shutdown_connections 2024-04-04T06:12:18.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.157+0000 7f441d800640 1 --2- 172.21.15.159:0/4213250127 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f441815b130 0x7f44181f58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:18.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.157+0000 7f441d800640 1 --2- 172.21.15.159:0/4213250127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44180cc490 0x7f44181f5390 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:18.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.157+0000 7f441d800640 1 --2- 172.21.15.159:0/4213250127 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43f406c8f0 0x7f43f406edb0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:18.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.157+0000 7f441d800640 1 --2- 172.21.15.159:0/4213250127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44180cc0b0 0x7f44181f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:18.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.157+0000 7f441d800640 1 -- 172.21.15.159:0/4213250127 >> 172.21.15.159:0/4213250127 conn(0x7f44180ccad0 msgr2=0x7f441815db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:18.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.157+0000 7f441d800640 1 -- 172.21.15.159:0/4213250127 shutdown_connections 2024-04-04T06:12:18.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.157+0000 7f441d800640 1 -- 172.21.15.159:0/4213250127 wait complete. 2024-04-04T06:12:18.176 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:12:18.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.393+0000 7fc112e00640 1 Processor -- start 2024-04-04T06:12:18.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.393+0000 7fc112e00640 1 -- start start 2024-04-04T06:12:18.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.394+0000 7fc112e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc10c15e4a0 0x7fc10c0b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:18.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.394+0000 7fc112e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc10c0cf700 0x7fc10c0b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:18.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.394+0000 7fc112e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc10c15e880 con 0x7fc10c0d2310 2024-04-04T06:12:18.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.394+0000 7fc112e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc10c0d34e0 con 0x7fc10c0cf700 2024-04-04T06:12:18.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.394+0000 7fc112e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc10c0d28d0 con 0x7fc10c15e4a0 2024-04-04T06:12:18.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.395+0000 7fc112400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc10c0cf700 0x7fc10c0b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:18.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.395+0000 7fc112400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc10c0cf700 0x7fc10c0b73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:43546/0 (socket says 172.21.15.159:43546) 2024-04-04T06:12:18.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.395+0000 7fc112400640 1 -- 172.21.15.159:0/2930330390 learned_addr learned my addr 172.21.15.159:0/2930330390 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:18.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.395+0000 7fc111a00640 1 --2- 172.21.15.159:0/2930330390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc10c15e4a0 0x7fc10c0b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:18.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.396+0000 7fc112400640 1 -- 172.21.15.159:0/2930330390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc10c15e4a0 msgr2=0x7fc10c0b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:18.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.396+0000 7fc112400640 1 --2- 172.21.15.159:0/2930330390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc10c15e4a0 0x7fc10c0b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:18.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.396+0000 7fc112400640 1 -- 172.21.15.159:0/2930330390 >> v1:172.21.15.178:6789/0 conn(0x7fc10c0d2310 legacy=0x7fc10c0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:18.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.396+0000 7fc112400640 1 -- 172.21.15.159:0/2930330390 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10c0d2bb0 con 0x7fc10c0cf700 2024-04-04T06:12:18.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.397+0000 7fc112400640 1 --2- 172.21.15.159:0/2930330390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc10c0cf700 0x7fc10c0b73e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0d3660 tx=0x7fc10402bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14e045a64c78acb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:18.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.398+0000 7fc10be00640 1 -- 172.21.15.159:0/2930330390 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10402cad0 con 0x7fc10c0cf700 2024-04-04T06:12:18.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.398+0000 7fc10be00640 1 -- 172.21.15.159:0/2930330390 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc10402cc70 con 0x7fc10c0cf700 2024-04-04T06:12:18.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.398+0000 7fc10be00640 1 -- 172.21.15.159:0/2930330390 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104034610 con 0x7fc10c0cf700 2024-04-04T06:12:18.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.398+0000 7fc112e00640 1 -- 172.21.15.159:0/2930330390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc10c0cf700 msgr2=0x7fc10c0b73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:18.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.398+0000 7fc112e00640 1 --2- 172.21.15.159:0/2930330390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc10c0cf700 0x7fc10c0b73e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0d3660 tx=0x7fc10402bf30 comp rx=0 tx=0).stop 2024-04-04T06:12:18.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.398+0000 7fc112e00640 1 -- 172.21.15.159:0/2930330390 shutdown_connections 2024-04-04T06:12:18.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.398+0000 7fc112e00640 1 --2- 172.21.15.159:0/2930330390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc10c0cf700 0x7fc10c0b73e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:18.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.398+0000 7fc112e00640 1 --2- 172.21.15.159:0/2930330390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc10c15e4a0 0x7fc10c0b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:18.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.398+0000 7fc112e00640 1 -- 172.21.15.159:0/2930330390 >> 172.21.15.159:0/2930330390 conn(0x7fc10c0676d0 msgr2=0x7fc10c067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:18.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.399+0000 7fc112e00640 1 -- 172.21.15.159:0/2930330390 shutdown_connections 2024-04-04T06:12:18.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.399+0000 7fc112e00640 1 -- 172.21.15.159:0/2930330390 wait complete. 2024-04-04T06:12:18.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.400+0000 7fc112e00640 1 Processor -- start 2024-04-04T06:12:18.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.400+0000 7fc112e00640 1 -- start start 2024-04-04T06:12:18.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.401+0000 7fc112e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc10c0cf700 0x7fc10c154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:18.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.401+0000 7fc112e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc10c0d2310 0x7fc10c0c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:18.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.401+0000 7fc112e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc10c15e4a0 0x7fc10c159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:18.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.401+0000 7fc112e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c0d3970 con 0x7fc10c15e4a0 2024-04-04T06:12:18.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.401+0000 7fc112e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c0d3170 con 0x7fc10c0d2310 2024-04-04T06:12:18.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.401+0000 7fc112e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc10c0ba1a0 con 0x7fc10c0cf700 2024-04-04T06:12:18.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.402+0000 7fc112400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc10c15e4a0 0x7fc10c159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:18.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.402+0000 7fc112400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc10c15e4a0 0x7fc10c159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:58998/0 (socket says 172.21.15.159:58998) 2024-04-04T06:12:18.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.402+0000 7fc112400640 1 -- 172.21.15.159:0/2592080217 learned_addr learned my addr 172.21.15.159:0/2592080217 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:18.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.402+0000 7fc111a00640 1 --2- 172.21.15.159:0/2592080217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc10c0cf700 0x7fc10c154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:18.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.402+0000 7fc111000640 1 --2- 172.21.15.159:0/2592080217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc10c0d2310 0x7fc10c0c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:18.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.402+0000 7fc112400640 1 -- 172.21.15.159:0/2592080217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc10c0cf700 msgr2=0x7fc10c154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:18.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.402+0000 7fc112400640 1 --2- 172.21.15.159:0/2592080217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc10c0cf700 0x7fc10c154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:18.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.402+0000 7fc112400640 1 -- 172.21.15.159:0/2592080217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc10c0d2310 msgr2=0x7fc10c0c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:18.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.402+0000 7fc112400640 1 --2- 172.21.15.159:0/2592080217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc10c0d2310 0x7fc10c0c8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:18.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.402+0000 7fc112400640 1 -- 172.21.15.159:0/2592080217 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10c158d40 con 0x7fc10c15e4a0 2024-04-04T06:12:18.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.402+0000 7fc111a00640 1 --2- 172.21.15.159:0/2592080217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc10c0cf700 0x7fc10c154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:18.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.402+0000 7fc112400640 1 --2- 172.21.15.159:0/2592080217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc10c15e4a0 0x7fc10c159de0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc104036040 tx=0x7fc104004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:18.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.403+0000 7fc109600640 1 -- 172.21.15.159:0/2592080217 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104043020 con 0x7fc10c15e4a0 2024-04-04T06:12:18.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.403+0000 7fc109600640 1 -- 172.21.15.159:0/2592080217 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc10403e450 con 0x7fc10c15e4a0 2024-04-04T06:12:18.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.403+0000 7fc109600640 1 -- 172.21.15.159:0/2592080217 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104009510 con 0x7fc10c15e4a0 2024-04-04T06:12:18.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.403+0000 7fc112e00640 1 -- 172.21.15.159:0/2592080217 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc10c159020 con 0x7fc10c15e4a0 2024-04-04T06:12:18.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.403+0000 7fc112e00640 1 -- 172.21.15.159:0/2592080217 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc10c1595e0 con 0x7fc10c15e4a0 2024-04-04T06:12:18.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.404+0000 7fc112e00640 1 -- 172.21.15.159:0/2592080217 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc10c0b6290 con 0x7fc10c15e4a0 2024-04-04T06:12:18.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.405+0000 7fc109600640 1 -- 172.21.15.159:0/2592080217 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc104005d10 con 0x7fc10c15e4a0 2024-04-04T06:12:18.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.406+0000 7fc109600640 1 --2- 172.21.15.159:0/2592080217 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc0ec06c8f0 0x7fc0ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:18.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.406+0000 7fc109600640 1 -- 172.21.15.159:0/2592080217 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7fc1040ae670 con 0x7fc10c15e4a0 2024-04-04T06:12:18.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.406+0000 7fc111a00640 1 --2- 172.21.15.159:0/2592080217 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc0ec06c8f0 0x7fc0ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:18.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.408+0000 7fc111a00640 1 --2- 172.21.15.159:0/2592080217 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc0ec06c8f0 0x7fc0ec06edb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc100009a20 tx=0x7fc100008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:18.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.410+0000 7fc109600640 1 -- 172.21.15.159:0/2592080217 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc10402ce20 con 0x7fc10c15e4a0 2024-04-04T06:12:18.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.610+0000 7fc112e00640 1 -- 172.21.15.159:0/2592080217 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc10c075940 con 0x7fc10c15e4a0 2024-04-04T06:12:18.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.612+0000 7fc109600640 1 -- 172.21.15.159:0/2592080217 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16909 (secure 0 0 0) 0x7fc10407ca00 con 0x7fc10c15e4a0 2024-04-04T06:12:18.606 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:18.606 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":74,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:11:35.095911+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:10:42.595868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-04T06:10:43.788847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.20","mappings":[{"from":6,"to":0}]},{"pgid":"11.3a","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6835/1652468640":"2024-04-05T06:10:57.934622+0000","172.21.15.159:6834/1652468640":"2024-04-05T06:10:57.934622+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:12:18.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.615+0000 7fc112e00640 1 -- 172.21.15.159:0/2592080217 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc0ec06c8f0 msgr2=0x7fc0ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:18.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.615+0000 7fc112e00640 1 --2- 172.21.15.159:0/2592080217 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc0ec06c8f0 0x7fc0ec06edb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc100009a20 tx=0x7fc100008040 comp rx=0 tx=0).stop 2024-04-04T06:12:18.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.615+0000 7fc112e00640 1 -- 172.21.15.159:0/2592080217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc10c15e4a0 msgr2=0x7fc10c159de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:18.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.615+0000 7fc112e00640 1 --2- 172.21.15.159:0/2592080217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc10c15e4a0 0x7fc10c159de0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc104036040 tx=0x7fc104004a40 comp rx=0 tx=0).stop 2024-04-04T06:12:18.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.615+0000 7fc112e00640 1 -- 172.21.15.159:0/2592080217 shutdown_connections 2024-04-04T06:12:18.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.615+0000 7fc112e00640 1 --2- 172.21.15.159:0/2592080217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc10c15e4a0 0x7fc10c159de0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:18.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.615+0000 7fc112e00640 1 --2- 172.21.15.159:0/2592080217 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc0ec06c8f0 0x7fc0ec06edb0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:18.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.615+0000 7fc112e00640 1 --2- 172.21.15.159:0/2592080217 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc10c0d2310 0x7fc10c0c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:18.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.615+0000 7fc112e00640 1 --2- 172.21.15.159:0/2592080217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc10c0cf700 0x7fc10c154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:18.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.615+0000 7fc112e00640 1 -- 172.21.15.159:0/2592080217 >> 172.21.15.159:0/2592080217 conn(0x7fc10c0676d0 msgr2=0x7fc10c0b4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:18.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.615+0000 7fc112e00640 1 -- 172.21.15.159:0/2592080217 shutdown_connections 2024-04-04T06:12:18.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.615+0000 7fc112e00640 1 -- 172.21.15.159:0/2592080217 wait complete. 2024-04-04T06:12:18.625 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:10:42.595868+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:10:43.788847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:12:18.626 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:12:18.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.835+0000 7fc6ce200640 1 Processor -- start 2024-04-04T06:12:18.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.835+0000 7fc6ce200640 1 -- start start 2024-04-04T06:12:18.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.837+0000 7fc6ce200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6c815eca0 0x7fc6c80b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:18.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.837+0000 7fc6ce200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc6c815ac50 0x7fc6c80b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:18.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.837+0000 7fc6ce200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6c80b4990 con 0x7fc6c815a870 2024-04-04T06:12:18.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.837+0000 7fc6ce200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc6c806d5b0 con 0x7fc6c815ac50 2024-04-04T06:12:18.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.837+0000 7fc6ce200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc6c806db70 con 0x7fc6c815eca0 2024-04-04T06:12:18.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.838+0000 7fc6c6a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6c815eca0 0x7fc6c80b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:18.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.838+0000 7fc6c6a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6c815eca0 0x7fc6c80b3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:50664/0 (socket says 172.21.15.159:50664) 2024-04-04T06:12:18.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.838+0000 7fc6c6a00640 1 -- 172.21.15.159:0/3475096914 learned_addr learned my addr 172.21.15.159:0/3475096914 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:18.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.838+0000 7fc6c7e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc6c815ac50 0x7fc6c80b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:18.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.839+0000 7fc6c7e00640 1 -- 172.21.15.159:0/3475096914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6c815eca0 msgr2=0x7fc6c80b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:18.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.839+0000 7fc6c7e00640 1 --2- 172.21.15.159:0/3475096914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6c815eca0 0x7fc6c80b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:18.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.839+0000 7fc6c7e00640 1 -- 172.21.15.159:0/3475096914 >> v1:172.21.15.178:6789/0 conn(0x7fc6c815a870 legacy=0x7fc6c815e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:18.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.839+0000 7fc6c7e00640 1 -- 172.21.15.159:0/3475096914 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c80b4c00 con 0x7fc6c815ac50 2024-04-04T06:12:18.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.839+0000 7fc6c6a00640 1 --2- 172.21.15.159:0/3475096914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6c815eca0 0x7fc6c80b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:12:18.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.840+0000 7fc6c7e00640 1 --2- 172.21.15.159:0/3475096914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6c815ac50 0x7fc6c80b4450 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009d90 tx=0x7fc6c002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c24dda58813fbee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:18.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.840+0000 7fc6c6000640 1 -- 172.21.15.159:0/3475096914 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0039070 con 0x7fc6c815ac50 2024-04-04T06:12:18.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.841+0000 7fc6c6000640 1 -- 172.21.15.159:0/3475096914 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6c002cc60 con 0x7fc6c815ac50 2024-04-04T06:12:18.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.841+0000 7fc6c6000640 1 -- 172.21.15.159:0/3475096914 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0034660 con 0x7fc6c815ac50 2024-04-04T06:12:18.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.841+0000 7fc6ce200640 1 -- 172.21.15.159:0/3475096914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6c815ac50 msgr2=0x7fc6c80b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:18.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.841+0000 7fc6ce200640 1 --2- 172.21.15.159:0/3475096914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6c815ac50 0x7fc6c80b4450 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009d90 tx=0x7fc6c002beb0 comp rx=0 tx=0).stop 2024-04-04T06:12:18.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.841+0000 7fc6ce200640 1 -- 172.21.15.159:0/3475096914 shutdown_connections 2024-04-04T06:12:18.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.841+0000 7fc6ce200640 1 --2- 172.21.15.159:0/3475096914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6c815ac50 0x7fc6c80b4450 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:18.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.841+0000 7fc6ce200640 1 --2- 172.21.15.159:0/3475096914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6c815eca0 0x7fc6c80b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:18.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.841+0000 7fc6ce200640 1 -- 172.21.15.159:0/3475096914 >> 172.21.15.159:0/3475096914 conn(0x7fc6c806b750 msgr2=0x7fc6c806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:18.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.841+0000 7fc6ce200640 1 -- 172.21.15.159:0/3475096914 shutdown_connections 2024-04-04T06:12:18.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.841+0000 7fc6ce200640 1 -- 172.21.15.159:0/3475096914 wait complete. 2024-04-04T06:12:18.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.844+0000 7fc6ce200640 1 Processor -- start 2024-04-04T06:12:18.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.844+0000 7fc6ce200640 1 -- start start 2024-04-04T06:12:18.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.845+0000 7fc6ce200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6c815a870 0x7fc6c814e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:18.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.845+0000 7fc6ce200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6c815ac50 0x7fc6c81516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:18.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.845+0000 7fc6ce200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6c815eca0 0x7fc6c8151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:18.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.845+0000 7fc6ce200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c80d0280 con 0x7fc6c815a870 2024-04-04T06:12:18.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.845+0000 7fc6ce200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c806d800 con 0x7fc6c815eca0 2024-04-04T06:12:18.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.845+0000 7fc6ce200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc6c80d0400 con 0x7fc6c815ac50 2024-04-04T06:12:18.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.845+0000 7fc6c7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6c815a870 0x7fc6c814e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:18.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.846+0000 7fc6c7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6c815a870 0x7fc6c814e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59000/0 (socket says 172.21.15.159:59000) 2024-04-04T06:12:18.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.846+0000 7fc6c7400640 1 -- 172.21.15.159:0/3974401752 learned_addr learned my addr 172.21.15.159:0/3974401752 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:18.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.846+0000 7fc6c7e00640 1 --2- 172.21.15.159:0/3974401752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6c815eca0 0x7fc6c8151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:18.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.846+0000 7fc6c6a00640 1 --2- 172.21.15.159:0/3974401752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6c815ac50 0x7fc6c81516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:18.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.846+0000 7fc6c7400640 1 -- 172.21.15.159:0/3974401752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6c815ac50 msgr2=0x7fc6c81516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:18.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.846+0000 7fc6c7400640 1 --2- 172.21.15.159:0/3974401752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6c815ac50 0x7fc6c81516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:18.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.846+0000 7fc6c7400640 1 -- 172.21.15.159:0/3974401752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6c815eca0 msgr2=0x7fc6c8151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:18.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.846+0000 7fc6c7400640 1 --2- 172.21.15.159:0/3974401752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6c815eca0 0x7fc6c8151ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:18.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.846+0000 7fc6c7400640 1 -- 172.21.15.159:0/3974401752 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c8142ee0 con 0x7fc6c815a870 2024-04-04T06:12:18.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.846+0000 7fc6c7e00640 1 --2- 172.21.15.159:0/3974401752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6c815eca0 0x7fc6c8151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:18.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.846+0000 7fc6c7400640 1 --2- 172.21.15.159:0/3974401752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6c815a870 0x7fc6c814e8c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc6b800c430 tx=0x7fc6b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:18.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.847+0000 7fc6bf400640 1 -- 172.21.15.159:0/3974401752 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b80052d0 con 0x7fc6c815a870 2024-04-04T06:12:18.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.847+0000 7fc6bf400640 1 -- 172.21.15.159:0/3974401752 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6b8005490 con 0x7fc6c815a870 2024-04-04T06:12:18.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.847+0000 7fc6bf400640 1 -- 172.21.15.159:0/3974401752 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8007260 con 0x7fc6c815a870 2024-04-04T06:12:18.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.847+0000 7fc6ce200640 1 -- 172.21.15.159:0/3974401752 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c8099a50 con 0x7fc6c815a870 2024-04-04T06:12:18.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.847+0000 7fc6ce200640 1 -- 172.21.15.159:0/3974401752 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c8151c30 con 0x7fc6c815a870 2024-04-04T06:12:18.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.848+0000 7fc6ce200640 1 -- 172.21.15.159:0/3974401752 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c80685c0 con 0x7fc6c815a870 2024-04-04T06:12:18.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.849+0000 7fc6bf400640 1 -- 172.21.15.159:0/3974401752 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc6b8007400 con 0x7fc6c815a870 2024-04-04T06:12:18.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.850+0000 7fc6bf400640 1 --2- 172.21.15.159:0/3974401752 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc6a806c920 0x7fc6a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:18.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.850+0000 7fc6bf400640 1 -- 172.21.15.159:0/3974401752 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7fc6b808b150 con 0x7fc6c815a870 2024-04-04T06:12:18.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.851+0000 7fc6c6a00640 1 --2- 172.21.15.159:0/3974401752 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc6a806c920 0x7fc6a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:18.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.852+0000 7fc6c6a00640 1 --2- 172.21.15.159:0/3974401752 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc6a806c920 0x7fc6a806ede0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009ac0 tx=0x7fc6b4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:18.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:18.855+0000 7fc6bf400640 1 -- 172.21.15.159:0/3974401752 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc6b80594e0 con 0x7fc6c815a870 2024-04-04T06:12:19.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.060+0000 7fc6ce200640 1 -- 172.21.15.159:0/3974401752 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc6c80af280 con 0x7fc6c815a870 2024-04-04T06:12:19.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.061+0000 7fc6bf400640 1 -- 172.21.15.159:0/3974401752 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7fc6b8058de0 con 0x7fc6c815a870 2024-04-04T06:12:19.055 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:12:19.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.064+0000 7fc6ce200640 1 -- 172.21.15.159:0/3974401752 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc6a806c920 msgr2=0x7fc6a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:19.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.064+0000 7fc6ce200640 1 --2- 172.21.15.159:0/3974401752 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc6a806c920 0x7fc6a806ede0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009ac0 tx=0x7fc6b4008040 comp rx=0 tx=0).stop 2024-04-04T06:12:19.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.064+0000 7fc6ce200640 1 -- 172.21.15.159:0/3974401752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6c815a870 msgr2=0x7fc6c814e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:19.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.064+0000 7fc6ce200640 1 --2- 172.21.15.159:0/3974401752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6c815a870 0x7fc6c814e8c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc6b800c430 tx=0x7fc6b800c8f0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.064+0000 7fc6ce200640 1 -- 172.21.15.159:0/3974401752 shutdown_connections 2024-04-04T06:12:19.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.064+0000 7fc6ce200640 1 --2- 172.21.15.159:0/3974401752 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc6a806c920 0x7fc6a806ede0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.064+0000 7fc6ce200640 1 --2- 172.21.15.159:0/3974401752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6c815eca0 0x7fc6c8151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.064+0000 7fc6ce200640 1 --2- 172.21.15.159:0/3974401752 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6c815ac50 0x7fc6c81516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.064+0000 7fc6ce200640 1 --2- 172.21.15.159:0/3974401752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6c815a870 0x7fc6c814e8c0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.064+0000 7fc6ce200640 1 -- 172.21.15.159:0/3974401752 >> 172.21.15.159:0/3974401752 conn(0x7fc6c806b750 msgr2=0x7fc6c806df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:19.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.064+0000 7fc6ce200640 1 -- 172.21.15.159:0/3974401752 shutdown_connections 2024-04-04T06:12:19.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.064+0000 7fc6ce200640 1 -- 172.21.15.159:0/3974401752 wait complete. 2024-04-04T06:12:19.080 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:12:19.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.297+0000 7f8bd1200640 1 Processor -- start 2024-04-04T06:12:19.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.298+0000 7f8bd1200640 1 -- start start 2024-04-04T06:12:19.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.299+0000 7f8bd1200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8bcc155550 0x7f8bcc155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:19.291 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.299+0000 7f8bd1200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f8bcc15f800 0x7f8bcc0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:19.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.299+0000 7f8bd1200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bcc0b40b0 con 0x7f8bcc0d07c0 2024-04-04T06:12:19.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.299+0000 7f8bd1200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f8bcc0d0bf0 con 0x7f8bcc15f800 2024-04-04T06:12:19.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.299+0000 7f8bd1200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8bcc0b4390 con 0x7f8bcc155550 2024-04-04T06:12:19.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.299+0000 7f8bc1400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f8bcc15f800 0x7f8bcc0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:19.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.299+0000 7f8bca800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f8bcc0d07c0 0x7f8bcc0aede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:47316/0 (socket says 172.21.15.159:47316) 2024-04-04T06:12:19.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.299+0000 7f8bc1400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f8bcc15f800 0x7f8bcc0d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59006/0 (socket says 172.21.15.159:59006) 2024-04-04T06:12:19.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.299+0000 7f8bca800640 1 -- 172.21.15.159:0/3926154110 learned_addr learned my addr 172.21.15.159:0/3926154110 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:19.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.300+0000 7f8bc9e00640 1 --2- 172.21.15.159:0/3926154110 >> v2:172.21.15.178:3300/0 conn(0x7f8bcc155550 0x7f8bcc155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:19.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.300+0000 7f8bc1400640 1 -- 172.21.15.159:0/3926154110 >> v2:172.21.15.178:3300/0 conn(0x7f8bcc155550 msgr2=0x7f8bcc155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:19.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.300+0000 7f8bc1400640 1 --2- 172.21.15.159:0/3926154110 >> v2:172.21.15.178:3300/0 conn(0x7f8bcc155550 0x7f8bcc155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.300+0000 7f8bc1400640 1 -- 172.21.15.159:0/3926154110 >> v1:172.21.15.159:6789/0 conn(0x7f8bcc0d07c0 legacy=0x7f8bcc0aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:19.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.300+0000 7f8bc1400640 1 -- 172.21.15.159:0/3926154110 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc0b4670 con 0x7f8bcc15f800 2024-04-04T06:12:19.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.300+0000 7f8bc9400640 1 -- 172.21.15.159:0/3926154110 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1919290400 0 0) 0x7f8bcc0b40b0 con 0x7f8bcc0d07c0 2024-04-04T06:12:19.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.301+0000 7f8bc1400640 1 --2- 172.21.15.159:0/3926154110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8bcc15f800 0x7f8bcc0d0280 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc073ac0 tx=0x7f8bb402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43ffb34b80d3058 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:19.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.301+0000 7f8bc9400640 1 -- 172.21.15.159:0/3926154110 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40315a0 con 0x7f8bcc15f800 2024-04-04T06:12:19.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.301+0000 7f8bc9400640 1 -- 172.21.15.159:0/3926154110 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bb4037070 con 0x7f8bcc15f800 2024-04-04T06:12:19.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.301+0000 7f8bc9400640 1 -- 172.21.15.159:0/3926154110 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4031960 con 0x7f8bcc15f800 2024-04-04T06:12:19.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.302+0000 7f8bd1200640 1 -- 172.21.15.159:0/3926154110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8bcc15f800 msgr2=0x7f8bcc0d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:19.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.302+0000 7f8bd1200640 1 --2- 172.21.15.159:0/3926154110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8bcc15f800 0x7f8bcc0d0280 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc073ac0 tx=0x7f8bb402b840 comp rx=0 tx=0).stop 2024-04-04T06:12:19.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.302+0000 7f8bd1200640 1 -- 172.21.15.159:0/3926154110 shutdown_connections 2024-04-04T06:12:19.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.302+0000 7f8bd1200640 1 --2- 172.21.15.159:0/3926154110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8bcc15f800 0x7f8bcc0d0280 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.295 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.302+0000 7f8bd1200640 1 --2- 172.21.15.159:0/3926154110 >> v2:172.21.15.178:3300/0 conn(0x7f8bcc155550 0x7f8bcc155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.302+0000 7f8bd1200640 1 -- 172.21.15.159:0/3926154110 >> 172.21.15.159:0/3926154110 conn(0x7f8bcc06b750 msgr2=0x7f8bcc06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:19.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.302+0000 7f8bd1200640 1 -- 172.21.15.159:0/3926154110 shutdown_connections 2024-04-04T06:12:19.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.302+0000 7f8bd1200640 1 -- 172.21.15.159:0/3926154110 wait complete. 2024-04-04T06:12:19.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.304+0000 7f8bd1200640 1 Processor -- start 2024-04-04T06:12:19.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.304+0000 7f8bd1200640 1 -- start start 2024-04-04T06:12:19.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.305+0000 7f8bd1200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bcc0d07c0 0x7f8bcc15e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:19.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.305+0000 7f8bd1200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8bcc155550 0x7f8bcc097500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:19.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.305+0000 7f8bd1200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bcc15f800 0x7f8bcc0af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:19.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.305+0000 7f8bd1200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc15fc80 con 0x7f8bcc155550 2024-04-04T06:12:19.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.305+0000 7f8bd1200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc0b4c30 con 0x7f8bcc15f800 2024-04-04T06:12:19.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.305+0000 7f8bd1200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8bcc15ba20 con 0x7f8bcc0d07c0 2024-04-04T06:12:19.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.305+0000 7f8bc1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8bcc155550 0x7f8bcc097500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:19.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.305+0000 7f8bc1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8bcc155550 0x7f8bcc097500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59022/0 (socket says 172.21.15.159:59022) 2024-04-04T06:12:19.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.305+0000 7f8bc1400640 1 -- 172.21.15.159:0/2066406233 learned_addr learned my addr 172.21.15.159:0/2066406233 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:19.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.305+0000 7f8bc9e00640 1 --2- 172.21.15.159:0/2066406233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bcc0d07c0 0x7f8bcc15e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:19.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.305+0000 7f8bca800640 1 --2- 172.21.15.159:0/2066406233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bcc15f800 0x7f8bcc0af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:19.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.305+0000 7f8bc1400640 1 -- 172.21.15.159:0/2066406233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bcc0d07c0 msgr2=0x7f8bcc15e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:19.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.305+0000 7f8bc1400640 1 --2- 172.21.15.159:0/2066406233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bcc0d07c0 0x7f8bcc15e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.305+0000 7f8bc1400640 1 -- 172.21.15.159:0/2066406233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bcc15f800 msgr2=0x7f8bcc0af900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:19.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.305+0000 7f8bc1400640 1 --2- 172.21.15.159:0/2066406233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bcc15f800 0x7f8bcc0af900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.305+0000 7f8bc1400640 1 -- 172.21.15.159:0/2066406233 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc15f030 con 0x7f8bcc155550 2024-04-04T06:12:19.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.306+0000 7f8bc1400640 1 --2- 172.21.15.159:0/2066406233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8bcc155550 0x7f8bcc097500 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4000c00 tx=0x7f8bb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:19.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.306+0000 7f8bc2a00640 1 -- 172.21.15.159:0/2066406233 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4004230 con 0x7f8bcc155550 2024-04-04T06:12:19.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.306+0000 7f8bc2a00640 1 -- 172.21.15.159:0/2066406233 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bb40043d0 con 0x7f8bcc155550 2024-04-04T06:12:19.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.306+0000 7f8bd1200640 1 -- 172.21.15.159:0/2066406233 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bcc0addc0 con 0x7f8bcc155550 2024-04-04T06:12:19.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.306+0000 7f8bc2a00640 1 -- 172.21.15.159:0/2066406233 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4032860 con 0x7f8bcc155550 2024-04-04T06:12:19.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.306+0000 7f8bd1200640 1 -- 172.21.15.159:0/2066406233 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bcc152350 con 0x7f8bcc155550 2024-04-04T06:12:19.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.307+0000 7f8bd1200640 1 -- 172.21.15.159:0/2066406233 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bcc0685c0 con 0x7f8bcc155550 2024-04-04T06:12:19.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.308+0000 7f8bc2a00640 1 -- 172.21.15.159:0/2066406233 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8bb4032a00 con 0x7f8bcc155550 2024-04-04T06:12:19.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.309+0000 7f8bc2a00640 1 --2- 172.21.15.159:0/2066406233 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8ba806c8f0 0x7f8ba806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:19.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.309+0000 7f8bc2a00640 1 -- 172.21.15.159:0/2066406233 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f8bb402f030 con 0x7f8bcc155550 2024-04-04T06:12:19.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.310+0000 7f8bc9e00640 1 --2- 172.21.15.159:0/2066406233 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8ba806c8f0 0x7f8ba806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:19.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.311+0000 7f8bc9e00640 1 --2- 172.21.15.159:0/2066406233 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8ba806c8f0 0x7f8ba806edb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc008e50 tx=0x7f8bbc0083a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:19.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.314+0000 7f8bc2a00640 1 -- 172.21.15.159:0/2066406233 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8bcc15f030 con 0x7f8bcc155550 2024-04-04T06:12:19.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.516+0000 7f8bd1200640 1 -- 172.21.15.159:0/2066406233 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8bcc144d70 con 0x7f8bcc155550 2024-04-04T06:12:19.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.517+0000 7f8bc2a00640 1 -- 172.21.15.159:0/2066406233 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) ==== 104+0+11 (secure 0 0 0) 0x7f8bb4041070 con 0x7f8bcc155550 2024-04-04T06:12:19.510 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:12:19.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.519+0000 7f8bd1200640 1 -- 172.21.15.159:0/2066406233 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8ba806c8f0 msgr2=0x7f8ba806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:19.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.519+0000 7f8bd1200640 1 --2- 172.21.15.159:0/2066406233 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8ba806c8f0 0x7f8ba806edb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc008e50 tx=0x7f8bbc0083a0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.519+0000 7f8bd1200640 1 -- 172.21.15.159:0/2066406233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8bcc155550 msgr2=0x7f8bcc097500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:19.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.519+0000 7f8bd1200640 1 --2- 172.21.15.159:0/2066406233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8bcc155550 0x7f8bcc097500 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4000c00 tx=0x7f8bb40046c0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.520+0000 7f8bd1200640 1 -- 172.21.15.159:0/2066406233 shutdown_connections 2024-04-04T06:12:19.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.520+0000 7f8bd1200640 1 --2- 172.21.15.159:0/2066406233 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8ba806c8f0 0x7f8ba806edb0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.520+0000 7f8bd1200640 1 --2- 172.21.15.159:0/2066406233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bcc15f800 0x7f8bcc0af900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.520+0000 7f8bd1200640 1 --2- 172.21.15.159:0/2066406233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8bcc155550 0x7f8bcc097500 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.520+0000 7f8bd1200640 1 --2- 172.21.15.159:0/2066406233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8bcc0d07c0 0x7f8bcc15e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.520+0000 7f8bd1200640 1 -- 172.21.15.159:0/2066406233 >> 172.21.15.159:0/2066406233 conn(0x7f8bcc06b750 msgr2=0x7f8bcc071610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:19.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.520+0000 7f8bd1200640 1 -- 172.21.15.159:0/2066406233 shutdown_connections 2024-04-04T06:12:19.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.520+0000 7f8bd1200640 1 -- 172.21.15.159:0/2066406233 wait complete. 2024-04-04T06:12:19.530 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:12:19.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.743+0000 7f513fe00640 1 Processor -- start 2024-04-04T06:12:19.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.743+0000 7f513fe00640 1 -- start start 2024-04-04T06:12:19.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.744+0000 7f513fe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f514015e460 0x7f51400b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:19.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.744+0000 7f513fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f51400d3e40 0x7f51400d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:19.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.744+0000 7f513fe00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51400b8320 con 0x7f51400cef20 2024-04-04T06:12:19.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.744+0000 7f513fe00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f51400d2da0 con 0x7f51400d3e40 2024-04-04T06:12:19.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.744+0000 7f513fe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f51400d3080 con 0x7f514015e460 2024-04-04T06:12:19.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.745+0000 7f5137e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f51400d3e40 0x7f51400d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:19.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.745+0000 7f5137e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f51400d3e40 0x7f51400d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59032/0 (socket says 172.21.15.159:59032) 2024-04-04T06:12:19.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.745+0000 7f5137e00640 1 -- 172.21.15.159:0/3523646746 learned_addr learned my addr 172.21.15.159:0/3523646746 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:19.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.745+0000 7f513ea00640 1 --2- 172.21.15.159:0/3523646746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f514015e460 0x7f51400b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:19.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.745+0000 7f5137e00640 1 -- 172.21.15.159:0/3523646746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f514015e460 msgr2=0x7f51400b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:19.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.745+0000 7f5137e00640 1 --2- 172.21.15.159:0/3523646746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f514015e460 0x7f51400b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.745+0000 7f5137e00640 1 -- 172.21.15.159:0/3523646746 >> v1:172.21.15.178:6789/0 conn(0x7f51400cef20 legacy=0x7f514006c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:19.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.745+0000 7f5137e00640 1 -- 172.21.15.159:0/3523646746 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51400d3360 con 0x7f51400d3e40 2024-04-04T06:12:19.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.747+0000 7f5137e00640 1 --2- 172.21.15.159:0/3523646746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51400d3e40 0x7f51400d2550 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f5128009860 tx=0x7f512802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbdee6ca7e8bb7a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:19.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.747+0000 7f513e000640 1 -- 172.21.15.159:0/3523646746 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5128035020 con 0x7f51400d3e40 2024-04-04T06:12:19.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.747+0000 7f513e000640 1 -- 172.21.15.159:0/3523646746 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5128031640 con 0x7f51400d3e40 2024-04-04T06:12:19.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.747+0000 7f513e000640 1 -- 172.21.15.159:0/3523646746 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512802d040 con 0x7f51400d3e40 2024-04-04T06:12:19.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.747+0000 7f513fe00640 1 -- 172.21.15.159:0/3523646746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51400d3e40 msgr2=0x7f51400d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:19.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.747+0000 7f513fe00640 1 --2- 172.21.15.159:0/3523646746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51400d3e40 0x7f51400d2550 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f5128009860 tx=0x7f512802b8f0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.748+0000 7f513fe00640 1 -- 172.21.15.159:0/3523646746 shutdown_connections 2024-04-04T06:12:19.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.748+0000 7f513fe00640 1 --2- 172.21.15.159:0/3523646746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51400d3e40 0x7f51400d2550 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.748+0000 7f513fe00640 1 --2- 172.21.15.159:0/3523646746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f514015e460 0x7f51400b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.748+0000 7f513fe00640 1 -- 172.21.15.159:0/3523646746 >> 172.21.15.159:0/3523646746 conn(0x7f51400678b0 msgr2=0x7f51400d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:19.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.748+0000 7f513fe00640 1 -- 172.21.15.159:0/3523646746 shutdown_connections 2024-04-04T06:12:19.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.748+0000 7f513fe00640 1 -- 172.21.15.159:0/3523646746 wait complete. 2024-04-04T06:12:19.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.749+0000 7f513fe00640 1 Processor -- start 2024-04-04T06:12:19.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.750+0000 7f513fe00640 1 -- start start 2024-04-04T06:12:19.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.750+0000 7f513fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51400cef20 0x7f51401f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:19.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.750+0000 7f513fe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51400d3e40 0x7f51400ab410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:19.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.751+0000 7f513fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f514015e460 0x7f51400c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:19.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.751+0000 7f513fe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f51400d2a90 con 0x7f51400cef20 2024-04-04T06:12:19.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.751+0000 7f513fe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f51400d3920 con 0x7f514015e460 2024-04-04T06:12:19.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.751+0000 7f513fe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f51400cf3b0 con 0x7f51400d3e40 2024-04-04T06:12:19.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.751+0000 7f513ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51400cef20 0x7f51401f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:19.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.751+0000 7f513ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51400cef20 0x7f51401f4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59038/0 (socket says 172.21.15.159:59038) 2024-04-04T06:12:19.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.751+0000 7f513ea00640 1 -- 172.21.15.159:0/1172514138 learned_addr learned my addr 172.21.15.159:0/1172514138 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:19.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.751+0000 7f513ea00640 1 -- 172.21.15.159:0/1172514138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51400d3e40 msgr2=0x7f51400ab410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:19.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.751+0000 7f513ea00640 1 --2- 172.21.15.159:0/1172514138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51400d3e40 0x7f51400ab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.751+0000 7f513ea00640 1 -- 172.21.15.159:0/1172514138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f514015e460 msgr2=0x7f51400c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:19.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.751+0000 7f513ea00640 1 --2- 172.21.15.159:0/1172514138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f514015e460 0x7f51400c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.751+0000 7f513ea00640 1 -- 172.21.15.159:0/1172514138 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51401f4c20 con 0x7f51400cef20 2024-04-04T06:12:19.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.751+0000 7f513ea00640 1 --2- 172.21.15.159:0/1172514138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51400cef20 0x7f51401f4480 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f513000d5f0 tx=0x7f513000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:19.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.752+0000 7f5136a00640 1 -- 172.21.15.159:0/1172514138 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51300125f0 con 0x7f51400cef20 2024-04-04T06:12:19.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.752+0000 7f5136a00640 1 -- 172.21.15.159:0/1172514138 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5130019070 con 0x7f51400cef20 2024-04-04T06:12:19.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.752+0000 7f5136a00640 1 -- 172.21.15.159:0/1172514138 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51300129d0 con 0x7f51400cef20 2024-04-04T06:12:19.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.752+0000 7f513fe00640 1 -- 172.21.15.159:0/1172514138 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51401f4f00 con 0x7f51400cef20 2024-04-04T06:12:19.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.752+0000 7f513fe00640 1 -- 172.21.15.159:0/1172514138 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51400ab950 con 0x7f51400cef20 2024-04-04T06:12:19.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.753+0000 7f513fe00640 1 -- 172.21.15.159:0/1172514138 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51400c9260 con 0x7f51400cef20 2024-04-04T06:12:19.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.754+0000 7f5136a00640 1 -- 172.21.15.159:0/1172514138 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5130012b70 con 0x7f51400cef20 2024-04-04T06:12:19.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.755+0000 7f5136a00640 1 --2- 172.21.15.159:0/1172514138 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f511c06c920 0x7f511c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:19.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.755+0000 7f5136a00640 1 -- 172.21.15.159:0/1172514138 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f513008c060 con 0x7f51400cef20 2024-04-04T06:12:19.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.756+0000 7f5137e00640 1 --2- 172.21.15.159:0/1172514138 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f511c06c920 0x7f511c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:19.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.757+0000 7f5137e00640 1 --2- 172.21.15.159:0/1172514138 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f511c06c920 0x7f511c06ede0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5128036000 tx=0x7f5128030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:19.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.759+0000 7f5136a00640 1 -- 172.21.15.159:0/1172514138 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f513005a360 con 0x7f51400cef20 2024-04-04T06:12:19.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.994+0000 7f513fe00640 1 -- 172.21.15.159:0/1172514138 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5140075940 con 0x7f51400cef20 2024-04-04T06:12:19.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.995+0000 7f5136a00640 1 -- 172.21.15.159:0/1172514138 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) ==== 100+0+11 (secure 0 0 0) 0x7f5130059c60 con 0x7f51400cef20 2024-04-04T06:12:19.988 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:12:19.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.999+0000 7f513fe00640 1 -- 172.21.15.159:0/1172514138 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f511c06c920 msgr2=0x7f511c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:19.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.999+0000 7f513fe00640 1 --2- 172.21.15.159:0/1172514138 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f511c06c920 0x7f511c06ede0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5128036000 tx=0x7f5128030000 comp rx=0 tx=0).stop 2024-04-04T06:12:19.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.999+0000 7f513fe00640 1 -- 172.21.15.159:0/1172514138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51400cef20 msgr2=0x7f51401f4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:19.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.999+0000 7f513fe00640 1 --2- 172.21.15.159:0/1172514138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51400cef20 0x7f51401f4480 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f513000d5f0 tx=0x7f513000dab0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.999+0000 7f513fe00640 1 -- 172.21.15.159:0/1172514138 shutdown_connections 2024-04-04T06:12:19.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.999+0000 7f513fe00640 1 --2- 172.21.15.159:0/1172514138 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f511c06c920 0x7f511c06ede0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.999+0000 7f513fe00640 1 --2- 172.21.15.159:0/1172514138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f514015e460 0x7f51400c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.999+0000 7f513fe00640 1 --2- 172.21.15.159:0/1172514138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51400d3e40 0x7f51400ab410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.999+0000 7f513fe00640 1 --2- 172.21.15.159:0/1172514138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51400cef20 0x7f51401f4480 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:19.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.999+0000 7f513fe00640 1 -- 172.21.15.159:0/1172514138 >> 172.21.15.159:0/1172514138 conn(0x7f51400678b0 msgr2=0x7f51400d4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:19.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.999+0000 7f513fe00640 1 -- 172.21.15.159:0/1172514138 shutdown_connections 2024-04-04T06:12:19.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:19.999+0000 7f513fe00640 1 -- 172.21.15.159:0/1172514138 wait complete. 2024-04-04T06:12:20.019 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:12:20.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.224+0000 7f7e1dc00640 1 Processor -- start 2024-04-04T06:12:20.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.224+0000 7f7e1dc00640 1 -- start start 2024-04-04T06:12:20.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.226+0000 7f7e1dc00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7e180d2bf0 0x7f7e180cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:20.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.226+0000 7f7e1dc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7e180d0c30 0x7f7e180d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:20.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.226+0000 7f7e1dc00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e180d3140 con 0x7f7e1815fa10 2024-04-04T06:12:20.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.226+0000 7f7e1dc00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f7e180d15e0 con 0x7f7e180d0c30 2024-04-04T06:12:20.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.226+0000 7f7e1dc00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7e180d3420 con 0x7f7e180d2bf0 2024-04-04T06:12:20.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.226+0000 7f7e16a00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f7e1815fa10 0x7f7e180b55a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:47324/0 (socket says 172.21.15.159:47324) 2024-04-04T06:12:20.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.226+0000 7f7e16a00640 1 -- 172.21.15.159:0/1527475088 learned_addr learned my addr 172.21.15.159:0/1527475088 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:20.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.226+0000 7f7e17400640 1 --2- 172.21.15.159:0/1527475088 >> v2:172.21.15.159:3300/0 conn(0x7f7e180d0c30 0x7f7e180d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:20.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.227+0000 7f7e0fe00640 1 --2- 172.21.15.159:0/1527475088 >> v2:172.21.15.178:3300/0 conn(0x7f7e180d2bf0 0x7f7e180cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:20.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.227+0000 7f7e16000640 1 -- 172.21.15.159:0/1527475088 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 251706853 0 0) 0x7f7e180d3140 con 0x7f7e1815fa10 2024-04-04T06:12:20.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.227+0000 7f7e16000640 1 -- 172.21.15.159:0/1527475088 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7dfc003520 con 0x7f7e1815fa10 2024-04-04T06:12:20.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.227+0000 7f7e17400640 1 -- 172.21.15.159:0/1527475088 >> v2:172.21.15.178:3300/0 conn(0x7f7e180d2bf0 msgr2=0x7f7e180cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:20.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.227+0000 7f7e17400640 1 --2- 172.21.15.159:0/1527475088 >> v2:172.21.15.178:3300/0 conn(0x7f7e180d2bf0 0x7f7e180cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:20.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.227+0000 7f7e17400640 1 -- 172.21.15.159:0/1527475088 >> v1:172.21.15.159:6789/0 conn(0x7f7e1815fa10 legacy=0x7f7e180b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:20.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.227+0000 7f7e17400640 1 -- 172.21.15.159:0/1527475088 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e180cf2b0 con 0x7f7e180d0c30 2024-04-04T06:12:20.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.228+0000 7f7e17400640 1 --2- 172.21.15.159:0/1527475088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e180d0c30 0x7f7e180d10a0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7e1808c670 tx=0x7f7e1002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60969e9ffeb76592 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:20.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.228+0000 7f7e16000640 1 -- 172.21.15.159:0/1527475088 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10039070 con 0x7f7e180d0c30 2024-04-04T06:12:20.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.228+0000 7f7e16000640 1 -- 172.21.15.159:0/1527475088 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e1002cd00 con 0x7f7e180d0c30 2024-04-04T06:12:20.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.228+0000 7f7e16000640 1 -- 172.21.15.159:0/1527475088 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10034640 con 0x7f7e180d0c30 2024-04-04T06:12:20.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.229+0000 7f7e1dc00640 1 -- 172.21.15.159:0/1527475088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e180d0c30 msgr2=0x7f7e180d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:20.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.229+0000 7f7e1dc00640 1 --2- 172.21.15.159:0/1527475088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e180d0c30 0x7f7e180d10a0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7e1808c670 tx=0x7f7e1002bf50 comp rx=0 tx=0).stop 2024-04-04T06:12:20.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.229+0000 7f7e1dc00640 1 -- 172.21.15.159:0/1527475088 shutdown_connections 2024-04-04T06:12:20.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.229+0000 7f7e1dc00640 1 --2- 172.21.15.159:0/1527475088 >> v2:172.21.15.178:3300/0 conn(0x7f7e180d2bf0 0x7f7e180cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:20.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.229+0000 7f7e1dc00640 1 --2- 172.21.15.159:0/1527475088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e180d0c30 0x7f7e180d10a0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:20.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.229+0000 7f7e1dc00640 1 -- 172.21.15.159:0/1527475088 >> 172.21.15.159:0/1527475088 conn(0x7f7e1808bfb0 msgr2=0x7f7e180d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:20.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.229+0000 7f7e1dc00640 1 -- 172.21.15.159:0/1527475088 shutdown_connections 2024-04-04T06:12:20.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.229+0000 7f7e1dc00640 1 -- 172.21.15.159:0/1527475088 wait complete. 2024-04-04T06:12:20.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.232+0000 7f7e1dc00640 1 Processor -- start 2024-04-04T06:12:20.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.232+0000 7f7e1dc00640 1 -- start start 2024-04-04T06:12:20.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.233+0000 7f7e1dc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e180d0c30 0x7f7e1815e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:20.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.233+0000 7f7e1dc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e180d2bf0 0x7f7e1815a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:20.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.233+0000 7f7e1dc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e1815fa10 0x7f7e180ce300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:20.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.233+0000 7f7e1dc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7e180d02d0 con 0x7f7e180d2bf0 2024-04-04T06:12:20.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.233+0000 7f7e1dc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7e180cf870 con 0x7f7e180d0c30 2024-04-04T06:12:20.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.233+0000 7f7e1dc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7e180d0450 con 0x7f7e1815fa10 2024-04-04T06:12:20.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.234+0000 7f7e0fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e180d2bf0 0x7f7e1815a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:20.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.234+0000 7f7e0fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e180d2bf0 0x7f7e1815a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59054/0 (socket says 172.21.15.159:59054) 2024-04-04T06:12:20.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.234+0000 7f7e0fe00640 1 -- 172.21.15.159:0/3225340245 learned_addr learned my addr 172.21.15.159:0/3225340245 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:20.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.234+0000 7f7e16a00640 1 --2- 172.21.15.159:0/3225340245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e180d0c30 0x7f7e1815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:20.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.234+0000 7f7e0fe00640 1 -- 172.21.15.159:0/3225340245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e1815fa10 msgr2=0x7f7e180ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:20.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.234+0000 7f7e17400640 1 --2- 172.21.15.159:0/3225340245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e1815fa10 0x7f7e180ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:20.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.234+0000 7f7e0fe00640 1 --2- 172.21.15.159:0/3225340245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e1815fa10 0x7f7e180ce300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:20.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.234+0000 7f7e0fe00640 1 -- 172.21.15.159:0/3225340245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e180d0c30 msgr2=0x7f7e1815e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:20.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.234+0000 7f7e0fe00640 1 --2- 172.21.15.159:0/3225340245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e180d0c30 0x7f7e1815e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:20.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.234+0000 7f7e0fe00640 1 -- 172.21.15.159:0/3225340245 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e180cd260 con 0x7f7e180d2bf0 2024-04-04T06:12:20.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.234+0000 7f7e16a00640 1 --2- 172.21.15.159:0/3225340245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e180d0c30 0x7f7e1815e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:20.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.235+0000 7f7e0fe00640 1 --2- 172.21.15.159:0/3225340245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e180d2bf0 0x7f7e1815a780 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000e930 tx=0x7f7e0000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:20.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.235+0000 7f7e0ea00640 1 -- 172.21.15.159:0/3225340245 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00017070 con 0x7f7e180d2bf0 2024-04-04T06:12:20.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.235+0000 7f7e0ea00640 1 -- 172.21.15.159:0/3225340245 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e0000ca00 con 0x7f7e180d2bf0 2024-04-04T06:12:20.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.235+0000 7f7e0ea00640 1 -- 172.21.15.159:0/3225340245 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0000cdb0 con 0x7f7e180d2bf0 2024-04-04T06:12:20.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.235+0000 7f7e1dc00640 1 -- 172.21.15.159:0/3225340245 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e180cd540 con 0x7f7e180d2bf0 2024-04-04T06:12:20.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.235+0000 7f7e1dc00640 1 -- 172.21.15.159:0/3225340245 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e180cdb00 con 0x7f7e180d2bf0 2024-04-04T06:12:20.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.236+0000 7f7e1dc00640 1 -- 172.21.15.159:0/3225340245 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e18068cd0 con 0x7f7e180d2bf0 2024-04-04T06:12:20.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.238+0000 7f7e0ea00640 1 -- 172.21.15.159:0/3225340245 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e0001a070 con 0x7f7e180d2bf0 2024-04-04T06:12:20.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.239+0000 7f7e0ea00640 1 --2- 172.21.15.159:0/3225340245 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7df406c8f0 0x7f7df406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:20.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.239+0000 7f7e0ea00640 1 -- 172.21.15.159:0/3225340245 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f7e0008c090 con 0x7f7e180d2bf0 2024-04-04T06:12:20.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.240+0000 7f7e16a00640 1 --2- 172.21.15.159:0/3225340245 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7df406c8f0 0x7f7df406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:20.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.241+0000 7f7e16a00640 1 --2- 172.21.15.159:0/3225340245 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7df406c8f0 0x7f7df406edb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7e08005880 tx=0x7f7e08009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:20.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.244+0000 7f7e0ea00640 1 -- 172.21.15.159:0/3225340245 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7e0005a450 con 0x7f7e180d2bf0 2024-04-04T06:12:20.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.618+0000 7f7e1dc00640 1 -- 172.21.15.159:0/3225340245 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e180ba3d0 con 0x7f7e180d2bf0 2024-04-04T06:12:20.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.619+0000 7f7e0ea00640 1 -- 172.21.15.159:0/3225340245 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5194 (secure 0 0 0) 0x7f7e00059d50 con 0x7f7e180d2bf0 2024-04-04T06:12:20.612 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:20.612 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3789513463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3789513463},{"type":"v1","addr":"172.21.15.178:6835","nonce":3789513463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/468875883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":468875883},{"type":"v1","addr":"172.21.15.178:6837","nonce":468875883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2546974742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2546974742},{"type":"v1","addr":"172.21.15.159:6833","nonce":2546974742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:10:44.805617+0000","modified":"2024-04-04T06:11:44.957858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6828":{"gid":6828,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":15,"addr":"172.21.15.159:6835/2467594024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2467594024},{"type":"v1","addr":"172.21.15.159:6835","nonce":2467594024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6828,"qdb_cluster":[6828]},"id":5}]} 2024-04-04T06:12:20.613 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 53 2024-04-04T06:12:20.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.622+0000 7f7e1dc00640 1 -- 172.21.15.159:0/3225340245 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7df406c8f0 msgr2=0x7f7df406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:20.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.622+0000 7f7e1dc00640 1 --2- 172.21.15.159:0/3225340245 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7df406c8f0 0x7f7df406edb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7e08005880 tx=0x7f7e08009040 comp rx=0 tx=0).stop 2024-04-04T06:12:20.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.622+0000 7f7e1dc00640 1 -- 172.21.15.159:0/3225340245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e180d2bf0 msgr2=0x7f7e1815a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:20.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.622+0000 7f7e1dc00640 1 --2- 172.21.15.159:0/3225340245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e180d2bf0 0x7f7e1815a780 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000e930 tx=0x7f7e0000edf0 comp rx=0 tx=0).stop 2024-04-04T06:12:20.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.622+0000 7f7e1dc00640 1 -- 172.21.15.159:0/3225340245 shutdown_connections 2024-04-04T06:12:20.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.622+0000 7f7e1dc00640 1 --2- 172.21.15.159:0/3225340245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7e1815fa10 0x7f7e180ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:20.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.622+0000 7f7e1dc00640 1 --2- 172.21.15.159:0/3225340245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e180d2bf0 0x7f7e1815a780 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:20.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.622+0000 7f7e1dc00640 1 --2- 172.21.15.159:0/3225340245 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7df406c8f0 0x7f7df406edb0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:20.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.622+0000 7f7e1dc00640 1 --2- 172.21.15.159:0/3225340245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e180d0c30 0x7f7e1815e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:20.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.622+0000 7f7e1dc00640 1 -- 172.21.15.159:0/3225340245 >> 172.21.15.159:0/3225340245 conn(0x7f7e1808bfb0 msgr2=0x7f7e180b7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:20.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.622+0000 7f7e1dc00640 1 -- 172.21.15.159:0/3225340245 shutdown_connections 2024-04-04T06:12:20.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.622+0000 7f7e1dc00640 1 -- 172.21.15.159:0/3225340245 wait complete. 2024-04-04T06:12:20.631 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:12:20.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.876+0000 7f8be4a00640 1 Processor -- start 2024-04-04T06:12:20.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.877+0000 7f8be4a00640 1 -- start start 2024-04-04T06:12:20.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.878+0000 7f8be4a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8be00cc0b0 0x7f8be006dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:20.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.878+0000 7f8be4a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f8be00d04d0 0x7f8be015ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:20.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.878+0000 7f8be4a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8be006e4f0 con 0x7f8be015fc50 2024-04-04T06:12:20.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.878+0000 7f8be4a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f8be015c610 con 0x7f8be00d04d0 2024-04-04T06:12:20.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.878+0000 7f8be4a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8be00b3150 con 0x7f8be00cc0b0 2024-04-04T06:12:20.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.879+0000 7f8bdfe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f8be00d04d0 0x7f8be015ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:20.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.879+0000 7f8bdfe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f8be00d04d0 0x7f8be015ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59058/0 (socket says 172.21.15.159:59058) 2024-04-04T06:12:20.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.879+0000 7f8bdfe00640 1 -- 172.21.15.159:0/143904661 learned_addr learned my addr 172.21.15.159:0/143904661 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:20.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.879+0000 7f8bdf400640 1 --2- 172.21.15.159:0/143904661 >> v2:172.21.15.178:3300/0 conn(0x7f8be00cc0b0 0x7f8be006dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:20.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.879+0000 7f8bdfe00640 1 -- 172.21.15.159:0/143904661 >> v2:172.21.15.178:3300/0 conn(0x7f8be00cc0b0 msgr2=0x7f8be006dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:20.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.879+0000 7f8bdfe00640 1 --2- 172.21.15.159:0/143904661 >> v2:172.21.15.178:3300/0 conn(0x7f8be00cc0b0 0x7f8be006dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:20.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.879+0000 7f8bdfe00640 1 -- 172.21.15.159:0/143904661 >> v1:172.21.15.178:6789/0 conn(0x7f8be015fc50 legacy=0x7f8be00cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:20.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.879+0000 7f8bdfe00640 1 -- 172.21.15.159:0/143904661 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be01600e0 con 0x7f8be00d04d0 2024-04-04T06:12:20.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.880+0000 7f8bdfe00640 1 --2- 172.21.15.159:0/143904661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8be00d04d0 0x7f8be015ca70 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f8be00cfd10 tx=0x7f8bd802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e06ea1c37d974a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:20.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.880+0000 7f8bde000640 1 -- 172.21.15.159:0/143904661 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd802c9f0 con 0x7f8be00d04d0 2024-04-04T06:12:20.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.881+0000 7f8bde000640 1 -- 172.21.15.159:0/143904661 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd802cb90 con 0x7f8be00d04d0 2024-04-04T06:12:20.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.881+0000 7f8bde000640 1 -- 172.21.15.159:0/143904661 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd80343f0 con 0x7f8be00d04d0 2024-04-04T06:12:20.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.881+0000 7f8be4a00640 1 -- 172.21.15.159:0/143904661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8be00d04d0 msgr2=0x7f8be015ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:20.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.881+0000 7f8be4a00640 1 --2- 172.21.15.159:0/143904661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8be00d04d0 0x7f8be015ca70 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f8be00cfd10 tx=0x7f8bd802be50 comp rx=0 tx=0).stop 2024-04-04T06:12:20.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.881+0000 7f8be4a00640 1 -- 172.21.15.159:0/143904661 shutdown_connections 2024-04-04T06:12:20.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.881+0000 7f8be4a00640 1 --2- 172.21.15.159:0/143904661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8be00d04d0 0x7f8be015ca70 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:20.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.881+0000 7f8be4a00640 1 --2- 172.21.15.159:0/143904661 >> v2:172.21.15.178:3300/0 conn(0x7f8be00cc0b0 0x7f8be006dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:20.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.881+0000 7f8be4a00640 1 -- 172.21.15.159:0/143904661 >> 172.21.15.159:0/143904661 conn(0x7f8be00ccad0 msgr2=0x7f8be00cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:20.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.881+0000 7f8be4a00640 1 -- 172.21.15.159:0/143904661 shutdown_connections 2024-04-04T06:12:20.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.881+0000 7f8be4a00640 1 -- 172.21.15.159:0/143904661 wait complete. 2024-04-04T06:12:20.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.883+0000 7f8be4a00640 1 Processor -- start 2024-04-04T06:12:20.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.883+0000 7f8be4a00640 1 -- start start 2024-04-04T06:12:20.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.884+0000 7f8be4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8be00cc0b0 0x7f8be013e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:20.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.884+0000 7f8be4a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8be00d04d0 0x7f8be0155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:20.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.884+0000 7f8be4a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8be0154b90 0x7f8be0137d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:20.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.884+0000 7f8be4a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8be0160980 con 0x7f8be00cc0b0 2024-04-04T06:12:20.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.884+0000 7f8be4a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8be01606a0 con 0x7f8be0154b90 2024-04-04T06:12:20.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.884+0000 7f8be4a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8be00cc490 con 0x7f8be00d04d0 2024-04-04T06:12:20.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.884+0000 7f8bdf400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8be00cc0b0 0x7f8be013e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:20.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.884+0000 7f8bdf400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8be00cc0b0 0x7f8be013e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59064/0 (socket says 172.21.15.159:59064) 2024-04-04T06:12:20.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.884+0000 7f8bdf400640 1 -- 172.21.15.159:0/4165740615 learned_addr learned my addr 172.21.15.159:0/4165740615 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:20.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.885+0000 7f8bdf400640 1 -- 172.21.15.159:0/4165740615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8be00d04d0 msgr2=0x7f8be0155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:20.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.885+0000 7f8bdf400640 1 --2- 172.21.15.159:0/4165740615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8be00d04d0 0x7f8be0155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:20.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.885+0000 7f8bdf400640 1 -- 172.21.15.159:0/4165740615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8be0154b90 msgr2=0x7f8be0137d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:20.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.885+0000 7f8bdf400640 1 --2- 172.21.15.159:0/4165740615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8be0154b90 0x7f8be0137d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:20.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.885+0000 7f8bdf400640 1 -- 172.21.15.159:0/4165740615 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be0153af0 con 0x7f8be00cc0b0 2024-04-04T06:12:20.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.885+0000 7f8bdf400640 1 --2- 172.21.15.159:0/4165740615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8be00cc0b0 0x7f8be013e400 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000c7f0 tx=0x7f8bd000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:20.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.885+0000 7f8bd7400640 1 -- 172.21.15.159:0/4165740615 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0015070 con 0x7f8be00cc0b0 2024-04-04T06:12:20.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.885+0000 7f8bd7400640 1 -- 172.21.15.159:0/4165740615 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd0004240 con 0x7f8be00cc0b0 2024-04-04T06:12:20.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.886+0000 7f8bd7400640 1 -- 172.21.15.159:0/4165740615 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0007620 con 0x7f8be00cc0b0 2024-04-04T06:12:20.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.886+0000 7f8be4a00640 1 -- 172.21.15.159:0/4165740615 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be0153dd0 con 0x7f8be00cc0b0 2024-04-04T06:12:20.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.886+0000 7f8be4a00640 1 -- 172.21.15.159:0/4165740615 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be0154390 con 0x7f8be00cc0b0 2024-04-04T06:12:20.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.887+0000 7f8be4a00640 1 -- 172.21.15.159:0/4165740615 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be0075940 con 0x7f8be00cc0b0 2024-04-04T06:12:20.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.889+0000 7f8bd7400640 1 -- 172.21.15.159:0/4165740615 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8bd00079d0 con 0x7f8be00cc0b0 2024-04-04T06:12:20.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.890+0000 7f8bd7400640 1 --2- 172.21.15.159:0/4165740615 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8bc006c8f0 0x7f8bc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:20.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.890+0000 7f8bd7400640 1 -- 172.21.15.159:0/4165740615 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f8bd008bc50 con 0x7f8be00cc0b0 2024-04-04T06:12:20.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.890+0000 7f8bdea00640 1 --2- 172.21.15.159:0/4165740615 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8bc006c8f0 0x7f8bc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:20.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.891+0000 7f8bdea00640 1 --2- 172.21.15.159:0/4165740615 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8bc006c8f0 0x7f8bc006edb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8be0068a00 tx=0x7f8bcc006ba0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:20.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:20.899+0000 7f8bd7400640 1 -- 172.21.15.159:0/4165740615 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8bd0091020 con 0x7f8be00cc0b0 2024-04-04T06:12:21.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.207+0000 7f8be4a00640 1 -- 172.21.15.159:0/4165740615 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8be0073830 con 0x7f8be00cc0b0 2024-04-04T06:12:21.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.209+0000 7f8bd7400640 1 -- 172.21.15.159:0/4165740615 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16909 (secure 0 0 0) 0x7f8bd005a080 con 0x7f8be00cc0b0 2024-04-04T06:12:21.203 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:21.203 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":74,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:11:35.095911+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:10:42.595868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-04T06:10:43.788847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.20","mappings":[{"from":6,"to":0}]},{"pgid":"11.3a","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6835/1652468640":"2024-04-05T06:10:57.934622+0000","172.21.15.159:6834/1652468640":"2024-04-05T06:10:57.934622+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:12:21.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.212+0000 7f8be4a00640 1 -- 172.21.15.159:0/4165740615 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8bc006c8f0 msgr2=0x7f8bc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:21.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.212+0000 7f8be4a00640 1 --2- 172.21.15.159:0/4165740615 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8bc006c8f0 0x7f8bc006edb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8be0068a00 tx=0x7f8bcc006ba0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.212+0000 7f8be4a00640 1 -- 172.21.15.159:0/4165740615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8be00cc0b0 msgr2=0x7f8be013e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:21.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.212+0000 7f8be4a00640 1 --2- 172.21.15.159:0/4165740615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8be00cc0b0 0x7f8be013e400 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000c7f0 tx=0x7f8bd000ccb0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.212+0000 7f8be4a00640 1 -- 172.21.15.159:0/4165740615 shutdown_connections 2024-04-04T06:12:21.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.212+0000 7f8be4a00640 1 --2- 172.21.15.159:0/4165740615 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8bc006c8f0 0x7f8bc006edb0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.212+0000 7f8be4a00640 1 --2- 172.21.15.159:0/4165740615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8be0154b90 0x7f8be0137d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.212+0000 7f8be4a00640 1 --2- 172.21.15.159:0/4165740615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8be00d04d0 0x7f8be0155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.212+0000 7f8be4a00640 1 --2- 172.21.15.159:0/4165740615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8be00cc0b0 0x7f8be013e400 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.212+0000 7f8be4a00640 1 -- 172.21.15.159:0/4165740615 >> 172.21.15.159:0/4165740615 conn(0x7f8be00ccad0 msgr2=0x7f8be00b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:21.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.213+0000 7f8be4a00640 1 -- 172.21.15.159:0/4165740615 shutdown_connections 2024-04-04T06:12:21.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.213+0000 7f8be4a00640 1 -- 172.21.15.159:0/4165740615 wait complete. 2024-04-04T06:12:21.224 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-04T06:12:21.224 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:12:21.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.431+0000 7f1782e00640 1 Processor -- start 2024-04-04T06:12:21.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.431+0000 7f1782e00640 1 -- start start 2024-04-04T06:12:21.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.433+0000 7f1782e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f177c15b310 0x7f177c0727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:21.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.434+0000 7f1782e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f177c0d0780 0x7f177c0d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:21.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.434+0000 7f1782e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f177c15b6f0 con 0x7f177c0b4f90 2024-04-04T06:12:21.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.434+0000 7f1782e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f177c072e80 con 0x7f177c15b310 2024-04-04T06:12:21.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.434+0000 7f1782e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f177c073160 con 0x7f177c0d0780 2024-04-04T06:12:21.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.434+0000 7f177be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f177c15b310 0x7f177c0727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:21.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.434+0000 7f177be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f177c15b310 0x7f177c0727f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59080/0 (socket says 172.21.15.159:59080) 2024-04-04T06:12:21.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.434+0000 7f177be00640 1 -- 172.21.15.159:0/4171554126 learned_addr learned my addr 172.21.15.159:0/4171554126 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:21.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.434+0000 7f177be00640 1 -- 172.21.15.159:0/4171554126 >> v2:172.21.15.178:3300/0 conn(0x7f177c0d0780 msgr2=0x7f177c0d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:21.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.434+0000 7f177be00640 1 --2- 172.21.15.159:0/4171554126 >> v2:172.21.15.178:3300/0 conn(0x7f177c0d0780 0x7f177c0d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.434+0000 7f177be00640 1 -- 172.21.15.159:0/4171554126 >> v1:172.21.15.178:6789/0 conn(0x7f177c0b4f90 legacy=0x7f177c15f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:21.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.434+0000 7f177be00640 1 -- 172.21.15.159:0/4171554126 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c15efe0 con 0x7f177c15b310 2024-04-04T06:12:21.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.435+0000 7f177be00640 1 --2- 172.21.15.159:0/4171554126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177c15b310 0x7f177c0727f0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f177c0ad7c0 tx=0x7f176c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8edd962cb019c90 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:21.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.436+0000 7f177aa00640 1 -- 172.21.15.159:0/4171554126 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c038070 con 0x7f177c15b310 2024-04-04T06:12:21.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.436+0000 7f177aa00640 1 -- 172.21.15.159:0/4171554126 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f176c02f050 con 0x7f177c15b310 2024-04-04T06:12:21.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.436+0000 7f177aa00640 1 -- 172.21.15.159:0/4171554126 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c031970 con 0x7f177c15b310 2024-04-04T06:12:21.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.436+0000 7f1782e00640 1 -- 172.21.15.159:0/4171554126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177c15b310 msgr2=0x7f177c0727f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:21.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.436+0000 7f1782e00640 1 --2- 172.21.15.159:0/4171554126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177c15b310 0x7f177c0727f0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f177c0ad7c0 tx=0x7f176c02b7f0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.436+0000 7f1782e00640 1 -- 172.21.15.159:0/4171554126 shutdown_connections 2024-04-04T06:12:21.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.436+0000 7f1782e00640 1 --2- 172.21.15.159:0/4171554126 >> v2:172.21.15.178:3300/0 conn(0x7f177c0d0780 0x7f177c0d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.436+0000 7f1782e00640 1 --2- 172.21.15.159:0/4171554126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177c15b310 0x7f177c0727f0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.436+0000 7f1782e00640 1 -- 172.21.15.159:0/4171554126 >> 172.21.15.159:0/4171554126 conn(0x7f177c06b990 msgr2=0x7f177c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:21.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.436+0000 7f1782e00640 1 -- 172.21.15.159:0/4171554126 shutdown_connections 2024-04-04T06:12:21.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.436+0000 7f1782e00640 1 -- 172.21.15.159:0/4171554126 wait complete. 2024-04-04T06:12:21.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.439+0000 7f1782e00640 1 Processor -- start 2024-04-04T06:12:21.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.439+0000 7f1782e00640 1 -- start start 2024-04-04T06:12:21.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.441+0000 7f1782e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f177c0b4f90 0x7f177c15da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:21.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.441+0000 7f1782e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177c0d0780 0x7f177c1f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:21.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.441+0000 7f1782e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f177c15b310 0x7f177c159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:21.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.441+0000 7f1782e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f177c073440 con 0x7f177c0d0780 2024-04-04T06:12:21.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.441+0000 7f1782e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f177c0b53f0 con 0x7f177c0b4f90 2024-04-04T06:12:21.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.441+0000 7f1782e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f177c0adc80 con 0x7f177c15b310 2024-04-04T06:12:21.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.441+0000 7f177b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177c0d0780 0x7f177c1f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:21.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.441+0000 7f177b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177c0d0780 0x7f177c1f4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59096/0 (socket says 172.21.15.159:59096) 2024-04-04T06:12:21.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.441+0000 7f177b400640 1 -- 172.21.15.159:0/206524377 learned_addr learned my addr 172.21.15.159:0/206524377 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:21.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.442+0000 7f177b400640 1 -- 172.21.15.159:0/206524377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f177c15b310 msgr2=0x7f177c159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:21.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.441+0000 7f1780e00640 1 --2- 172.21.15.159:0/206524377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f177c15b310 0x7f177c159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:21.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.442+0000 7f177be00640 1 --2- 172.21.15.159:0/206524377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f177c0b4f90 0x7f177c15da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:21.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.442+0000 7f177b400640 1 --2- 172.21.15.159:0/206524377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f177c15b310 0x7f177c159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.442+0000 7f177b400640 1 -- 172.21.15.159:0/206524377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f177c0b4f90 msgr2=0x7f177c15da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:21.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.442+0000 7f177b400640 1 --2- 172.21.15.159:0/206524377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f177c0b4f90 0x7f177c15da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.442+0000 7f177b400640 1 -- 172.21.15.159:0/206524377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c1f4a60 con 0x7f177c0d0780 2024-04-04T06:12:21.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.442+0000 7f1780e00640 1 --2- 172.21.15.159:0/206524377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f177c15b310 0x7f177c159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:21.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.442+0000 7f177b400640 1 --2- 172.21.15.159:0/206524377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177c0d0780 0x7f177c1f4520 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f176800c410 tx=0x7f176800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:21.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.442+0000 7f1773e00640 1 -- 172.21.15.159:0/206524377 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1768017070 con 0x7f177c0d0780 2024-04-04T06:12:21.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.442+0000 7f1773e00640 1 -- 172.21.15.159:0/206524377 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17680035f0 con 0x7f177c0d0780 2024-04-04T06:12:21.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.442+0000 7f1773e00640 1 -- 172.21.15.159:0/206524377 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1768003a20 con 0x7f177c0d0780 2024-04-04T06:12:21.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.442+0000 7f1782e00640 1 -- 172.21.15.159:0/206524377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f177c1491e0 con 0x7f177c0d0780 2024-04-04T06:12:21.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.442+0000 7f1782e00640 1 -- 172.21.15.159:0/206524377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f177c15cc50 con 0x7f177c0d0780 2024-04-04T06:12:21.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.444+0000 7f1782e00640 1 -- 172.21.15.159:0/206524377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f177c0685c0 con 0x7f177c0d0780 2024-04-04T06:12:21.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.444+0000 7f1773e00640 1 -- 172.21.15.159:0/206524377 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1768003bc0 con 0x7f177c0d0780 2024-04-04T06:12:21.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.445+0000 7f1773e00640 1 --2- 172.21.15.159:0/206524377 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f175c06c8f0 0x7f175c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:21.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.445+0000 7f1773e00640 1 -- 172.21.15.159:0/206524377 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f176808af70 con 0x7f177c0d0780 2024-04-04T06:12:21.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.446+0000 7f177be00640 1 --2- 172.21.15.159:0/206524377 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f175c06c8f0 0x7f175c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:21.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.447+0000 7f177be00640 1 --2- 172.21.15.159:0/206524377 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f175c06c8f0 0x7f175c06edb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f176c035000 tx=0x7f176c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:21.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.450+0000 7f1773e00640 1 -- 172.21.15.159:0/206524377 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1768059270 con 0x7f177c0d0780 2024-04-04T06:12:21.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.704+0000 7f1782e00640 1 -- 172.21.15.159:0/206524377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f177c15ec20 con 0x7f177c0d0780 2024-04-04T06:12:21.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.705+0000 7f1773e00640 1 -- 172.21.15.159:0/206524377 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5194 (secure 0 0 0) 0x7f17680010c0 con 0x7f177c0d0780 2024-04-04T06:12:21.698 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:21.699 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3789513463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3789513463},{"type":"v1","addr":"172.21.15.178:6835","nonce":3789513463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/468875883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":468875883},{"type":"v1","addr":"172.21.15.178:6837","nonce":468875883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2546974742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2546974742},{"type":"v1","addr":"172.21.15.159:6833","nonce":2546974742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:10:44.805617+0000","modified":"2024-04-04T06:11:44.957858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6828":{"gid":6828,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":15,"addr":"172.21.15.159:6835/2467594024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2467594024},{"type":"v1","addr":"172.21.15.159:6835","nonce":2467594024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6828,"qdb_cluster":[6828]},"id":5}]} 2024-04-04T06:12:21.699 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 53 2024-04-04T06:12:21.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.707+0000 7f1782e00640 1 -- 172.21.15.159:0/206524377 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f175c06c8f0 msgr2=0x7f175c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:21.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.707+0000 7f1782e00640 1 --2- 172.21.15.159:0/206524377 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f175c06c8f0 0x7f175c06edb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f176c035000 tx=0x7f176c008030 comp rx=0 tx=0).stop 2024-04-04T06:12:21.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.707+0000 7f1782e00640 1 -- 172.21.15.159:0/206524377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177c0d0780 msgr2=0x7f177c1f4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:21.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.707+0000 7f1782e00640 1 --2- 172.21.15.159:0/206524377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177c0d0780 0x7f177c1f4520 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f176800c410 tx=0x7f176800c8d0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.708+0000 7f1782e00640 1 -- 172.21.15.159:0/206524377 shutdown_connections 2024-04-04T06:12:21.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.708+0000 7f1782e00640 1 --2- 172.21.15.159:0/206524377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f177c15b310 0x7f177c159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.708+0000 7f1782e00640 1 --2- 172.21.15.159:0/206524377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177c0d0780 0x7f177c1f4520 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.708+0000 7f1782e00640 1 --2- 172.21.15.159:0/206524377 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f175c06c8f0 0x7f175c06edb0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.708+0000 7f1782e00640 1 --2- 172.21.15.159:0/206524377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f177c0b4f90 0x7f177c15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.708+0000 7f1782e00640 1 -- 172.21.15.159:0/206524377 >> 172.21.15.159:0/206524377 conn(0x7f177c06b990 msgr2=0x7f177c0b1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:21.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.708+0000 7f1782e00640 1 -- 172.21.15.159:0/206524377 shutdown_connections 2024-04-04T06:12:21.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.708+0000 7f1782e00640 1 -- 172.21.15.159:0/206524377 wait complete. 2024-04-04T06:12:21.718 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:12:21.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.924+0000 7f2d71200640 1 Processor -- start 2024-04-04T06:12:21.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.925+0000 7f2d71200640 1 -- start start 2024-04-04T06:12:21.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.926+0000 7f2d71200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d6c0cfb20 0x7f2d6c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:21.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.926+0000 7f2d71200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d6c15ebc0 con 0x7f2d6c15e7e0 2024-04-04T06:12:21.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.926+0000 7f2d71200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d6c06d1c0 con 0x7f2d6c0d1950 2024-04-04T06:12:21.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.926+0000 7f2d71200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2d6c0d1db0 con 0x7f2d6c0cfb20 2024-04-04T06:12:21.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.926+0000 7f2d69e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f2d6c15e7e0 0x7f2d6c06ca00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:47330/0 (socket says 172.21.15.159:47330) 2024-04-04T06:12:21.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.926+0000 7f2d69e00640 1 -- 172.21.15.159:0/3509407637 learned_addr learned my addr 172.21.15.159:0/3509407637 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:21.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.927+0000 7f2d6a800640 1 --2- 172.21.15.159:0/3509407637 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d6c0cfb20 0x7f2d6c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:21.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.927+0000 7f2d69400640 1 -- 172.21.15.159:0/3509407637 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 759768035 0 0) 0x7f2d6c15ebc0 con 0x7f2d6c15e7e0 2024-04-04T06:12:21.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.927+0000 7f2d69400640 1 -- 172.21.15.159:0/3509407637 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d50003520 con 0x7f2d6c15e7e0 2024-04-04T06:12:21.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.927+0000 7f2d69400640 1 -- 172.21.15.159:0/3509407637 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1682755472 0 0) 0x7f2d50003520 con 0x7f2d6c15e7e0 2024-04-04T06:12:21.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.927+0000 7f2d69400640 1 -- 172.21.15.159:0/3509407637 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d6c0cfb20 msgr2=0x7f2d6c0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:21.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.927+0000 7f2d69400640 1 --2- 172.21.15.159:0/3509407637 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d6c0cfb20 0x7f2d6c0cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.927+0000 7f2d69400640 1 -- 172.21.15.159:0/3509407637 >> v1:172.21.15.178:6789/0 conn(0x7f2d6c0d1950 legacy=0x7f2d6c0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:21.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.927+0000 7f2d69400640 1 -- 172.21.15.159:0/3509407637 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6c06d4a0 con 0x7f2d6c15e7e0 2024-04-04T06:12:21.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.927+0000 7f2d69400640 1 -- 172.21.15.159:0/3509407637 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f2d5c001670 con 0x7f2d6c15e7e0 2024-04-04T06:12:21.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.927+0000 7f2d69400640 1 -- 172.21.15.159:0/3509407637 >> v1:172.21.15.159:6789/0 conn(0x7f2d6c15e7e0 legacy=0x7f2d6c06ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:21.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.927+0000 7f2d69400640 1 --2- 172.21.15.159:0/3509407637 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d6c0cfb20 0x7f2d500039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:21.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.927+0000 7f2d69400640 1 --2- 172.21.15.159:0/3509407637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d50003fb0 0x7f2d50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:21.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.927+0000 7f2d69400640 1 --2- 172.21.15.159:0/3509407637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d50004d40 0x7f2d500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:21.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.928+0000 7f2d69400640 1 -- 172.21.15.159:0/3509407637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c0d1db0 con 0x7f2d50003fb0 2024-04-04T06:12:21.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.928+0000 7f2d69400640 1 -- 172.21.15.159:0/3509407637 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5000bbb0 con 0x7f2d50004d40 2024-04-04T06:12:21.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.928+0000 7f2d69400640 1 -- 172.21.15.159:0/3509407637 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2d5000bdb0 con 0x7f2d6c0cfb20 2024-04-04T06:12:21.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.928+0000 7f2d63e00640 1 --2- 172.21.15.159:0/3509407637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d50003fb0 0x7f2d50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:21.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.928+0000 7f2d69e00640 1 --2- 172.21.15.159:0/3509407637 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d6c0cfb20 0x7f2d500039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:21.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.928+0000 7f2d63e00640 1 -- 172.21.15.159:0/3509407637 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d6c0cfb20 msgr2=0x7f2d500039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:21.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.928+0000 7f2d6a800640 1 --2- 172.21.15.159:0/3509407637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d50004d40 0x7f2d500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:21.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.928+0000 7f2d63e00640 1 --2- 172.21.15.159:0/3509407637 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d6c0cfb20 0x7f2d500039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.928+0000 7f2d63e00640 1 -- 172.21.15.159:0/3509407637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d50004d40 msgr2=0x7f2d500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:21.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.928+0000 7f2d63e00640 1 --2- 172.21.15.159:0/3509407637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d50004d40 0x7f2d500051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.928+0000 7f2d63e00640 1 -- 172.21.15.159:0/3509407637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d50003520 con 0x7f2d50003fb0 2024-04-04T06:12:21.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.929+0000 7f2d63e00640 1 --2- 172.21.15.159:0/3509407637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d50003fb0 0x7f2d50004800 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f2d54001750 tx=0x7f2d50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb2e3c7f647ee26 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:21.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.930+0000 7f2d69400640 1 -- 172.21.15.159:0/3509407637 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d54001370 con 0x7f2d50003fb0 2024-04-04T06:12:21.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.930+0000 7f2d69400640 1 -- 172.21.15.159:0/3509407637 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d540052d0 con 0x7f2d50003fb0 2024-04-04T06:12:21.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.930+0000 7f2d69400640 1 -- 172.21.15.159:0/3509407637 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d54003830 con 0x7f2d50003fb0 2024-04-04T06:12:21.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.930+0000 7f2d71200640 1 -- 172.21.15.159:0/3509407637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d50003fb0 msgr2=0x7f2d50004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:21.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.930+0000 7f2d71200640 1 --2- 172.21.15.159:0/3509407637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d50003fb0 0x7f2d50004800 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f2d54001750 tx=0x7f2d50005830 comp rx=0 tx=0).stop 2024-04-04T06:12:21.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.930+0000 7f2d6a800640 1 -- 172.21.15.159:0/3509407637 reap_dead start 2024-04-04T06:12:21.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.930+0000 7f2d71200640 1 -- 172.21.15.159:0/3509407637 shutdown_connections 2024-04-04T06:12:21.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.930+0000 7f2d71200640 1 -- 172.21.15.159:0/3509407637 >> 172.21.15.159:0/3509407637 conn(0x7f2d6c09d7c0 msgr2=0x7f2d6c06bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:21.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.930+0000 7f2d71200640 1 -- 172.21.15.159:0/3509407637 shutdown_connections 2024-04-04T06:12:21.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.930+0000 7f2d71200640 1 -- 172.21.15.159:0/3509407637 wait complete. 2024-04-04T06:12:21.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.932+0000 7f2d71200640 1 Processor -- start 2024-04-04T06:12:21.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.932+0000 7f2d71200640 1 -- start start 2024-04-04T06:12:21.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.933+0000 7f2d71200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d50003fb0 0x7f2d6c0cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:21.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.933+0000 7f2d71200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d6c0cbaf0 0x7f2d6c0cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:21.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.933+0000 7f2d71200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d6c0c99e0 0x7f2d6c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:21.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.933+0000 7f2d71200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c0cefd0 con 0x7f2d50003fb0 2024-04-04T06:12:21.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.933+0000 7f2d71200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c0cf600 con 0x7f2d6c0c99e0 2024-04-04T06:12:21.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.933+0000 7f2d71200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2d6c0d2f30 con 0x7f2d6c0cbaf0 2024-04-04T06:12:21.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.933+0000 7f2d69e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d50003fb0 0x7f2d6c0cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:21.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.933+0000 7f2d69e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d50003fb0 0x7f2d6c0cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59124/0 (socket says 172.21.15.159:59124) 2024-04-04T06:12:21.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.933+0000 7f2d69e00640 1 -- 172.21.15.159:0/1579395377 learned_addr learned my addr 172.21.15.159:0/1579395377 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:21.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.934+0000 7f2d6a800640 1 --2- 172.21.15.159:0/1579395377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d6c0c99e0 0x7f2d6c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:21.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.934+0000 7f2d69e00640 1 -- 172.21.15.159:0/1579395377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d6c0cbaf0 msgr2=0x7f2d6c0cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:21.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.934+0000 7f2d69e00640 1 --2- 172.21.15.159:0/1579395377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d6c0cbaf0 0x7f2d6c0cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.934+0000 7f2d69e00640 1 -- 172.21.15.159:0/1579395377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d6c0c99e0 msgr2=0x7f2d6c0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:21.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.934+0000 7f2d69e00640 1 --2- 172.21.15.159:0/1579395377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d6c0c99e0 0x7f2d6c0cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:21.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.934+0000 7f2d69e00640 1 -- 172.21.15.159:0/1579395377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6c0c9ee0 con 0x7f2d50003fb0 2024-04-04T06:12:21.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.934+0000 7f2d69e00640 1 --2- 172.21.15.159:0/1579395377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d50003fb0 0x7f2d6c0cb5b0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00e800 tx=0x7f2d5c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:21.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.934+0000 7f2d62000640 1 -- 172.21.15.159:0/1579395377 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c00c8d0 con 0x7f2d50003fb0 2024-04-04T06:12:21.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.935+0000 7f2d62000640 1 -- 172.21.15.159:0/1579395377 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d5c018070 con 0x7f2d50003fb0 2024-04-04T06:12:21.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.935+0000 7f2d62000640 1 -- 172.21.15.159:0/1579395377 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c00ccd0 con 0x7f2d50003fb0 2024-04-04T06:12:21.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.935+0000 7f2d71200640 1 -- 172.21.15.159:0/1579395377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d6c0ca1c0 con 0x7f2d50003fb0 2024-04-04T06:12:21.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.935+0000 7f2d71200640 1 -- 172.21.15.159:0/1579395377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d6c0ca780 con 0x7f2d50003fb0 2024-04-04T06:12:21.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.936+0000 7f2d71200640 1 -- 172.21.15.159:0/1579395377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d6c0685c0 con 0x7f2d50003fb0 2024-04-04T06:12:21.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.937+0000 7f2d62000640 1 -- 172.21.15.159:0/1579395377 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d5c013420 con 0x7f2d50003fb0 2024-04-04T06:12:21.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.938+0000 7f2d62000640 1 --2- 172.21.15.159:0/1579395377 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d4806c920 0x7f2d4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:21.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.938+0000 7f2d62000640 1 -- 172.21.15.159:0/1579395377 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f2d5c08b890 con 0x7f2d50003fb0 2024-04-04T06:12:21.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.939+0000 7f2d63e00640 1 --2- 172.21.15.159:0/1579395377 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d4806c920 0x7f2d4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:21.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.940+0000 7f2d63e00640 1 --2- 172.21.15.159:0/1579395377 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d4806c920 0x7f2d4806ede0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2d54033000 tx=0x7f2d5402f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:21.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:21.943+0000 7f2d62000640 1 -- 172.21.15.159:0/1579395377 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2d5c022250 con 0x7f2d50003fb0 2024-04-04T06:12:22.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.180+0000 7f2d71200640 1 -- 172.21.15.159:0/1579395377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d6c0d5300 con 0x7f2d50003fb0 2024-04-04T06:12:22.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.180+0000 7f2d62000640 1 -- 172.21.15.159:0/1579395377 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5194 (secure 0 0 0) 0x7f2d5c059b90 con 0x7f2d50003fb0 2024-04-04T06:12:22.175 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:22.175 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3789513463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3789513463},{"type":"v1","addr":"172.21.15.178:6835","nonce":3789513463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/468875883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":468875883},{"type":"v1","addr":"172.21.15.178:6837","nonce":468875883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2546974742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2546974742},{"type":"v1","addr":"172.21.15.159:6833","nonce":2546974742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:10:44.805617+0000","modified":"2024-04-04T06:11:44.957858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6828},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6828":{"gid":6828,"name":"a","rank":0,"incarnation":48,"state":"up:active","state_seq":15,"addr":"172.21.15.159:6835/2467594024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2467594024},{"type":"v1","addr":"172.21.15.159:6835","nonce":2467594024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6828,"qdb_cluster":[6828]},"id":5}]} 2024-04-04T06:12:22.176 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 53 2024-04-04T06:12:22.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.183+0000 7f2d71200640 1 -- 172.21.15.159:0/1579395377 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d4806c920 msgr2=0x7f2d4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:22.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.183+0000 7f2d71200640 1 --2- 172.21.15.159:0/1579395377 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d4806c920 0x7f2d4806ede0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2d54033000 tx=0x7f2d5402f000 comp rx=0 tx=0).stop 2024-04-04T06:12:22.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.183+0000 7f2d71200640 1 -- 172.21.15.159:0/1579395377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d50003fb0 msgr2=0x7f2d6c0cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:22.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.183+0000 7f2d71200640 1 --2- 172.21.15.159:0/1579395377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d50003fb0 0x7f2d6c0cb5b0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00e800 tx=0x7f2d5c00ecc0 comp rx=0 tx=0).stop 2024-04-04T06:12:22.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.183+0000 7f2d71200640 1 -- 172.21.15.159:0/1579395377 shutdown_connections 2024-04-04T06:12:22.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.183+0000 7f2d71200640 1 --2- 172.21.15.159:0/1579395377 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d4806c920 0x7f2d4806ede0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:22.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.183+0000 7f2d71200640 1 --2- 172.21.15.159:0/1579395377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d6c0c99e0 0x7f2d6c0cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:22.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.183+0000 7f2d71200640 1 --2- 172.21.15.159:0/1579395377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d6c0cbaf0 0x7f2d6c0cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:22.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.183+0000 7f2d71200640 1 --2- 172.21.15.159:0/1579395377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d50003fb0 0x7f2d6c0cb5b0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:22.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.183+0000 7f2d71200640 1 -- 172.21.15.159:0/1579395377 >> 172.21.15.159:0/1579395377 conn(0x7f2d6c09d7c0 msgr2=0x7f2d6c0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:22.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.183+0000 7f2d71200640 1 -- 172.21.15.159:0/1579395377 shutdown_connections 2024-04-04T06:12:22.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.183+0000 7f2d71200640 1 -- 172.21.15.159:0/1579395377 wait complete. 2024-04-04T06:12:22.194 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:12:22.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.397+0000 7fddf7200640 1 Processor -- start 2024-04-04T06:12:22.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.397+0000 7fddf7200640 1 -- start start 2024-04-04T06:12:22.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.399+0000 7fddf7200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fddf815e4a0 0x7fddf80b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:22.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.399+0000 7fddf7200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddf815e880 con 0x7fddf80cf700 2024-04-04T06:12:22.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.399+0000 7fddf7200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddf80d28d0 con 0x7fddf80d2310 2024-04-04T06:12:22.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.399+0000 7fddf7200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fddf80d34e0 con 0x7fddf815e4a0 2024-04-04T06:12:22.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.399+0000 7fddf6800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fddf80cf700 0x7fddf80b73e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:47342/0 (socket says 172.21.15.159:47342) 2024-04-04T06:12:22.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.399+0000 7fddf6800640 1 -- 172.21.15.159:0/3554579121 learned_addr learned my addr 172.21.15.159:0/3554579121 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:22.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.399+0000 7fddf5e00640 1 --2- 172.21.15.159:0/3554579121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fddf815e4a0 0x7fddf80b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:22.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.400+0000 7fddf5400640 1 -- 172.21.15.159:0/3554579121 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 966724999 0 0) 0x7fddf815e880 con 0x7fddf80cf700 2024-04-04T06:12:22.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.400+0000 7fddf5400640 1 -- 172.21.15.159:0/3554579121 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdddc003520 con 0x7fddf80cf700 2024-04-04T06:12:22.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.400+0000 7fddf5400640 1 -- 172.21.15.159:0/3554579121 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2165589402 0 0) 0x7fdddc003520 con 0x7fddf80cf700 2024-04-04T06:12:22.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.400+0000 7fddf5400640 1 -- 172.21.15.159:0/3554579121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fddf815e4a0 msgr2=0x7fddf80b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:22.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.400+0000 7fddf5400640 1 --2- 172.21.15.159:0/3554579121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fddf815e4a0 0x7fddf80b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:22.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.400+0000 7fddf5400640 1 -- 172.21.15.159:0/3554579121 >> v1:172.21.15.178:6789/0 conn(0x7fddf80d2310 legacy=0x7fddf80ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:22.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.400+0000 7fddf5400640 1 -- 172.21.15.159:0/3554579121 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf80d2bb0 con 0x7fddf80cf700 2024-04-04T06:12:22.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.400+0000 7fddf5400640 1 -- 172.21.15.159:0/3554579121 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fddf00019f0 con 0x7fddf80cf700 2024-04-04T06:12:22.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.400+0000 7fddf5400640 1 -- 172.21.15.159:0/3554579121 >> v1:172.21.15.159:6789/0 conn(0x7fddf80cf700 legacy=0x7fddf80b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:22.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.400+0000 7fddf5400640 1 --2- 172.21.15.159:0/3554579121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdddc0039e0 0x7fdddc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:22.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.400+0000 7fddf5400640 1 --2- 172.21.15.159:0/3554579121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fddf815e4a0 0x7fdddc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:22.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.400+0000 7fddf5400640 1 --2- 172.21.15.159:0/3554579121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdddc004d40 0x7fdddc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:22.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.400+0000 7fddf5400640 1 -- 172.21.15.159:0/3554579121 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fddf80d34e0 con 0x7fdddc004d40 2024-04-04T06:12:22.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.400+0000 7fddf5400640 1 -- 172.21.15.159:0/3554579121 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc00baa0 con 0x7fdddc0039e0 2024-04-04T06:12:22.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.400+0000 7fddf5400640 1 -- 172.21.15.159:0/3554579121 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdddc00bca0 con 0x7fddf815e4a0 2024-04-04T06:12:22.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.401+0000 7fddf6800640 1 --2- 172.21.15.159:0/3554579121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdddc004d40 0x7fdddc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:22.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.401+0000 7fddf5e00640 1 --2- 172.21.15.159:0/3554579121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdddc0039e0 0x7fdddc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:22.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.401+0000 7fddefe00640 1 --2- 172.21.15.159:0/3554579121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fddf815e4a0 0x7fdddc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:22.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.401+0000 7fddf6800640 1 -- 172.21.15.159:0/3554579121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fddf815e4a0 msgr2=0x7fdddc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:22.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.401+0000 7fddf6800640 1 --2- 172.21.15.159:0/3554579121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fddf815e4a0 0x7fdddc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:22.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.401+0000 7fddf6800640 1 -- 172.21.15.159:0/3554579121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdddc0039e0 msgr2=0x7fdddc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:22.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.401+0000 7fddf6800640 1 --2- 172.21.15.159:0/3554579121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdddc0039e0 0x7fdddc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:22.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.401+0000 7fddf6800640 1 -- 172.21.15.159:0/3554579121 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdddc003520 con 0x7fdddc004d40 2024-04-04T06:12:22.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.401+0000 7fddefe00640 1 --2- 172.21.15.159:0/3554579121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fddf815e4a0 0x7fdddc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:22.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.403+0000 7fddf6800640 1 --2- 172.21.15.159:0/3554579121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdddc004d40 0x7fdddc0051c0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fddf80d20d0 tx=0x7fdddc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f57823a97097a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:22.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.403+0000 7fddf5400640 1 -- 172.21.15.159:0/3554579121 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf000a390 con 0x7fdddc004d40 2024-04-04T06:12:22.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.403+0000 7fddf5400640 1 -- 172.21.15.159:0/3554579121 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddf0034650 con 0x7fdddc004d40 2024-04-04T06:12:22.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.403+0000 7fddf5400640 1 -- 172.21.15.159:0/3554579121 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0034a40 con 0x7fdddc004d40 2024-04-04T06:12:22.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.404+0000 7fddf7200640 1 -- 172.21.15.159:0/3554579121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdddc004d40 msgr2=0x7fdddc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:22.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.404+0000 7fddf7200640 1 --2- 172.21.15.159:0/3554579121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdddc004d40 0x7fdddc0051c0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fddf80d20d0 tx=0x7fdddc005830 comp rx=0 tx=0).stop 2024-04-04T06:12:22.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.404+0000 7fddf6800640 1 -- 172.21.15.159:0/3554579121 reap_dead start 2024-04-04T06:12:22.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.404+0000 7fddf7200640 1 -- 172.21.15.159:0/3554579121 shutdown_connections 2024-04-04T06:12:22.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.404+0000 7fddf7200640 1 -- 172.21.15.159:0/3554579121 >> 172.21.15.159:0/3554579121 conn(0x7fddf80676d0 msgr2=0x7fddf8067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:22.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.404+0000 7fddf7200640 1 -- 172.21.15.159:0/3554579121 shutdown_connections 2024-04-04T06:12:22.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.404+0000 7fddf7200640 1 -- 172.21.15.159:0/3554579121 wait complete. 2024-04-04T06:12:22.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.406+0000 7fddf7200640 1 Processor -- start 2024-04-04T06:12:22.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.406+0000 7fddf7200640 1 -- start start 2024-04-04T06:12:22.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.407+0000 7fddf7200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdddc004d40 0x7fddf8155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:22.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.407+0000 7fddf7200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddf815e880 0x7fddf815ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:22.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.407+0000 7fddf7200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddf80cc3d0 0x7fddf80cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:22.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.407+0000 7fddf7200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fddf80d3970 con 0x7fddf815e880 2024-04-04T06:12:22.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.407+0000 7fddf7200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fddf80d3170 con 0x7fddf80cc3d0 2024-04-04T06:12:22.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.407+0000 7fddf7200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fddf80b9c60 con 0x7fdddc004d40 2024-04-04T06:12:22.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.407+0000 7fddefe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddf815e880 0x7fddf815ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:22.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.407+0000 7fddefe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddf815e880 0x7fddf815ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59150/0 (socket says 172.21.15.159:59150) 2024-04-04T06:12:22.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.407+0000 7fddefe00640 1 -- 172.21.15.159:0/1458752265 learned_addr learned my addr 172.21.15.159:0/1458752265 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:22.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.407+0000 7fddf5e00640 1 --2- 172.21.15.159:0/1458752265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdddc004d40 0x7fddf8155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:22.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.407+0000 7fddf6800640 1 --2- 172.21.15.159:0/1458752265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddf80cc3d0 0x7fddf80cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:22.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.407+0000 7fddefe00640 1 -- 172.21.15.159:0/1458752265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdddc004d40 msgr2=0x7fddf8155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:22.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.407+0000 7fddefe00640 1 --2- 172.21.15.159:0/1458752265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdddc004d40 0x7fddf8155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:22.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.407+0000 7fddefe00640 1 -- 172.21.15.159:0/1458752265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddf80cc3d0 msgr2=0x7fddf80cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:22.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.407+0000 7fddefe00640 1 --2- 172.21.15.159:0/1458752265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddf80cc3d0 0x7fddf80cc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:22.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.407+0000 7fddefe00640 1 -- 172.21.15.159:0/1458752265 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf81f4010 con 0x7fddf815e880 2024-04-04T06:12:22.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.407+0000 7fddf5e00640 1 --2- 172.21.15.159:0/1458752265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdddc004d40 0x7fddf8155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:22.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.408+0000 7fddefe00640 1 --2- 172.21.15.159:0/1458752265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddf815e880 0x7fddf815ed00 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fdde000d7a0 tx=0x7fdde000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:22.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.408+0000 7fddee000640 1 -- 172.21.15.159:0/1458752265 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde0005270 con 0x7fddf815e880 2024-04-04T06:12:22.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.408+0000 7fddee000640 1 -- 172.21.15.159:0/1458752265 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde0017070 con 0x7fddf815e880 2024-04-04T06:12:22.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.408+0000 7fddf7200640 1 -- 172.21.15.159:0/1458752265 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf80cd100 con 0x7fddf815e880 2024-04-04T06:12:22.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.408+0000 7fddee000640 1 -- 172.21.15.159:0/1458752265 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde000b6b0 con 0x7fddf815e880 2024-04-04T06:12:22.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.408+0000 7fddf7200640 1 -- 172.21.15.159:0/1458752265 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf80cd6c0 con 0x7fddf815e880 2024-04-04T06:12:22.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.409+0000 7fddf7200640 1 -- 172.21.15.159:0/1458752265 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf8075940 con 0x7fddf815e880 2024-04-04T06:12:22.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.410+0000 7fddee000640 1 -- 172.21.15.159:0/1458752265 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdde000b850 con 0x7fddf815e880 2024-04-04T06:12:22.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.412+0000 7fddee000640 1 --2- 172.21.15.159:0/1458752265 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fddd406c920 0x7fddd406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:22.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.412+0000 7fddee000640 1 -- 172.21.15.159:0/1458752265 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7fdde008b630 con 0x7fddf815e880 2024-04-04T06:12:22.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.413+0000 7fddf5e00640 1 --2- 172.21.15.159:0/1458752265 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fddd406c920 0x7fddd406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:22.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.414+0000 7fddf5e00640 1 --2- 172.21.15.159:0/1458752265 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fddd406c920 0x7fddd406ede0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fddf80b5ca0 tx=0x7fdde8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:22.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.416+0000 7fddee000640 1 -- 172.21.15.159:0/1458752265 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fdde00102d0 con 0x7fddf815e880 2024-04-04T06:12:22.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.639+0000 7fddf7200640 1 -- 172.21.15.159:0/1458752265 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fddf80d4710 con 0x7fddf815e880 2024-04-04T06:12:22.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.641+0000 7fddee000640 1 -- 172.21.15.159:0/1458752265 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16909 (secure 0 0 0) 0x7fdde00599d0 con 0x7fddf815e880 2024-04-04T06:12:22.634 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:22.635 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":74,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:11:35.095911+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:10:42.595868+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-04T06:10:43.788847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.20","mappings":[{"from":6,"to":0}]},{"pgid":"11.3a","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6835/1652468640":"2024-04-05T06:10:57.934622+0000","172.21.15.159:6834/1652468640":"2024-04-05T06:10:57.934622+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:12:22.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.643+0000 7fddf7200640 1 -- 172.21.15.159:0/1458752265 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fddd406c920 msgr2=0x7fddd406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:22.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.643+0000 7fddf7200640 1 --2- 172.21.15.159:0/1458752265 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fddd406c920 0x7fddd406ede0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fddf80b5ca0 tx=0x7fdde8008040 comp rx=0 tx=0).stop 2024-04-04T06:12:22.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.643+0000 7fddf7200640 1 -- 172.21.15.159:0/1458752265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddf815e880 msgr2=0x7fddf815ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:22.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.643+0000 7fddf7200640 1 --2- 172.21.15.159:0/1458752265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddf815e880 0x7fddf815ed00 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fdde000d7a0 tx=0x7fdde000dc60 comp rx=0 tx=0).stop 2024-04-04T06:12:22.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.644+0000 7fddf7200640 1 -- 172.21.15.159:0/1458752265 shutdown_connections 2024-04-04T06:12:22.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.644+0000 7fddf7200640 1 --2- 172.21.15.159:0/1458752265 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fddd406c920 0x7fddd406ede0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:22.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.644+0000 7fddf7200640 1 --2- 172.21.15.159:0/1458752265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddf80cc3d0 0x7fddf80cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:22.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.644+0000 7fddf7200640 1 --2- 172.21.15.159:0/1458752265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddf815e880 0x7fddf815ed00 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:22.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.644+0000 7fddf7200640 1 --2- 172.21.15.159:0/1458752265 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdddc004d40 0x7fddf8155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:22.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.644+0000 7fddf7200640 1 -- 172.21.15.159:0/1458752265 >> 172.21.15.159:0/1458752265 conn(0x7fddf80676d0 msgr2=0x7fddf80d3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:22.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.644+0000 7fddf7200640 1 -- 172.21.15.159:0/1458752265 shutdown_connections 2024-04-04T06:12:22.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.644+0000 7fddf7200640 1 -- 172.21.15.159:0/1458752265 wait complete. 2024-04-04T06:12:22.655 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-04T06:12:22.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.865+0000 7f178b000640 1 Processor -- start 2024-04-04T06:12:22.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.866+0000 7f178b000640 1 -- start start 2024-04-04T06:12:22.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.867+0000 7f178b000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17840d0990 0x7f17840aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:22.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.867+0000 7f178b000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17840afab0 con 0x7f1784153580 2024-04-04T06:12:22.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.867+0000 7f178b000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f178415cc80 con 0x7f17840af2e0 2024-04-04T06:12:22.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.867+0000 7f178b000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f178415da00 con 0x7f17840d0990 2024-04-04T06:12:22.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.867+0000 7f1783e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f1784153580 0x7f17840d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:47350/0 (socket says 172.21.15.159:47350) 2024-04-04T06:12:22.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.867+0000 7f1783e00640 1 -- 172.21.15.159:0/3272830170 learned_addr learned my addr 172.21.15.159:0/3272830170 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:22.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.868+0000 7f1783400640 1 --2- 172.21.15.159:0/3272830170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17840d0990 0x7f17840aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:22.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.868+0000 7f1782a00640 1 -- 172.21.15.159:0/3272830170 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4204385605 0 0) 0x7f17840afab0 con 0x7f1784153580 2024-04-04T06:12:22.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.868+0000 7f1782a00640 1 -- 172.21.15.159:0/3272830170 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1768003520 con 0x7f1784153580 2024-04-04T06:12:22.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.868+0000 7f1782a00640 1 -- 172.21.15.159:0/3272830170 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2670341609 0 0) 0x7f1768003520 con 0x7f1784153580 2024-04-04T06:12:22.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.868+0000 7f1782a00640 1 -- 172.21.15.159:0/3272830170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17840d0990 msgr2=0x7f17840aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:22.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.868+0000 7f1782a00640 1 --2- 172.21.15.159:0/3272830170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17840d0990 0x7f17840aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:22.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.868+0000 7f1782a00640 1 -- 172.21.15.159:0/3272830170 >> v1:172.21.15.178:6789/0 conn(0x7f17840af2e0 legacy=0x7f17841600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:22.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.868+0000 7f1782a00640 1 -- 172.21.15.159:0/3272830170 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178415cf20 con 0x7f1784153580 2024-04-04T06:12:22.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.868+0000 7f1782a00640 1 -- 172.21.15.159:0/3272830170 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f177c0019f0 con 0x7f1784153580 2024-04-04T06:12:22.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.868+0000 7f1782a00640 1 -- 172.21.15.159:0/3272830170 >> v1:172.21.15.159:6789/0 conn(0x7f1784153580 legacy=0x7f17840d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:22.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.869+0000 7f1782a00640 1 --2- 172.21.15.159:0/3272830170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17840d0990 0x7f17680039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:22.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.869+0000 7f1782a00640 1 --2- 172.21.15.159:0/3272830170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1768003f60 0x7f1768004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:22.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.869+0000 7f1782a00640 1 --2- 172.21.15.159:0/3272830170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1768004d40 0x7f17680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:22.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.869+0000 7f1782a00640 1 -- 172.21.15.159:0/3272830170 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f178415da00 con 0x7f1768004d40 2024-04-04T06:12:22.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.869+0000 7f1782a00640 1 -- 172.21.15.159:0/3272830170 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f176800bb50 con 0x7f1768003f60 2024-04-04T06:12:22.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.869+0000 7f1782a00640 1 -- 172.21.15.159:0/3272830170 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f176800bd50 con 0x7f17840d0990 2024-04-04T06:12:22.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.869+0000 7f1789000640 1 --2- 172.21.15.159:0/3272830170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1768004d40 0x7f17680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:22.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.869+0000 7f1783e00640 1 --2- 172.21.15.159:0/3272830170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17840d0990 0x7f17680039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:22.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.869+0000 7f1783400640 1 --2- 172.21.15.159:0/3272830170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1768003f60 0x7f1768004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:22.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.870+0000 7f1789000640 1 -- 172.21.15.159:0/3272830170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17840d0990 msgr2=0x7f17680039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:22.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.870+0000 7f1789000640 1 --2- 172.21.15.159:0/3272830170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17840d0990 0x7f17680039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:22.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.870+0000 7f1789000640 1 -- 172.21.15.159:0/3272830170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1768003f60 msgr2=0x7f1768004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:22.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.870+0000 7f1789000640 1 --2- 172.21.15.159:0/3272830170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1768003f60 0x7f1768004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:22.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.870+0000 7f1789000640 1 -- 172.21.15.159:0/3272830170 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1768003520 con 0x7f1768004d40 2024-04-04T06:12:22.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.870+0000 7f1783400640 1 --2- 172.21.15.159:0/3272830170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1768003f60 0x7f1768004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:12:22.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.871+0000 7f1789000640 1 --2- 172.21.15.159:0/3272830170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1768004d40 0x7f17680051c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f1774001750 tx=0x7f1768005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d34b417a9a0143 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:22.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.872+0000 7f1782a00640 1 -- 172.21.15.159:0/3272830170 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774001370 con 0x7f1768004d40 2024-04-04T06:12:22.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.872+0000 7f1782a00640 1 -- 172.21.15.159:0/3272830170 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17740052d0 con 0x7f1768004d40 2024-04-04T06:12:22.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.872+0000 7f1782a00640 1 -- 172.21.15.159:0/3272830170 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774003830 con 0x7f1768004d40 2024-04-04T06:12:22.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.872+0000 7f178b000640 1 -- 172.21.15.159:0/3272830170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1768004d40 msgr2=0x7f17680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:22.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.872+0000 7f178b000640 1 --2- 172.21.15.159:0/3272830170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1768004d40 0x7f17680051c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f1774001750 tx=0x7f1768005830 comp rx=0 tx=0).stop 2024-04-04T06:12:22.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.872+0000 7f1789000640 1 -- 172.21.15.159:0/3272830170 reap_dead start 2024-04-04T06:12:22.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.872+0000 7f178b000640 1 -- 172.21.15.159:0/3272830170 shutdown_connections 2024-04-04T06:12:22.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.872+0000 7f178b000640 1 -- 172.21.15.159:0/3272830170 >> 172.21.15.159:0/3272830170 conn(0x7f178406b750 msgr2=0x7f178406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:22.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.872+0000 7f178b000640 1 -- 172.21.15.159:0/3272830170 shutdown_connections 2024-04-04T06:12:22.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.872+0000 7f178b000640 1 -- 172.21.15.159:0/3272830170 wait complete. 2024-04-04T06:12:22.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.874+0000 7f178b000640 1 Processor -- start 2024-04-04T06:12:22.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.874+0000 7f178b000640 1 -- start start 2024-04-04T06:12:22.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.875+0000 7f178b000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1768004d40 0x7f178409dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:22.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.875+0000 7f178b000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17840b2430 0x7f17841f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:22.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.875+0000 7f178b000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17840cd460 0x7f17840ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.875+0000 7f178b000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1784153960 con 0x7f17840cd460 2024-04-04T06:12:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.875+0000 7f178b000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f178415d4e0 con 0x7f1768004d40 2024-04-04T06:12:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.875+0000 7f178b000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f17840af710 con 0x7f17840b2430 2024-04-04T06:12:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.875+0000 7f1789000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17840cd460 0x7f17840ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.875+0000 7f1789000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17840cd460 0x7f17840ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59162/0 (socket says 172.21.15.159:59162) 2024-04-04T06:12:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.876+0000 7f1789000640 1 -- 172.21.15.159:0/1724940719 learned_addr learned my addr 172.21.15.159:0/1724940719 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.876+0000 7f1783e00640 1 --2- 172.21.15.159:0/1724940719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1768004d40 0x7f178409dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.876+0000 7f1783400640 1 --2- 172.21.15.159:0/1724940719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17840b2430 0x7f17841f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.876+0000 7f1789000640 1 -- 172.21.15.159:0/1724940719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17840b2430 msgr2=0x7f17841f4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.876+0000 7f1789000640 1 --2- 172.21.15.159:0/1724940719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17840b2430 0x7f17841f4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.876+0000 7f1789000640 1 -- 172.21.15.159:0/1724940719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1768004d40 msgr2=0x7f178409dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:22.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.876+0000 7f1789000640 1 --2- 172.21.15.159:0/1724940719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1768004d40 0x7f178409dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:22.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.876+0000 7f1789000640 1 -- 172.21.15.159:0/1724940719 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17840cdd60 con 0x7f17840cd460 2024-04-04T06:12:22.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.876+0000 7f1789000640 1 --2- 172.21.15.159:0/1724940719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17840cd460 0x7f17840ce6c0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f177402bd00 tx=0x7f1774001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:22.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.877+0000 7f177be00640 1 -- 172.21.15.159:0/1724940719 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774041070 con 0x7f17840cd460 2024-04-04T06:12:22.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.877+0000 7f177be00640 1 -- 172.21.15.159:0/1724940719 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1774005480 con 0x7f17840cd460 2024-04-04T06:12:22.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.877+0000 7f177be00640 1 -- 172.21.15.159:0/1724940719 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774001370 con 0x7f17840cd460 2024-04-04T06:12:22.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.877+0000 7f178b000640 1 -- 172.21.15.159:0/1724940719 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17840afc20 con 0x7f17840cd460 2024-04-04T06:12:22.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.877+0000 7f178b000640 1 -- 172.21.15.159:0/1724940719 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17841f4a90 con 0x7f17840cd460 2024-04-04T06:12:22.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.878+0000 7f178b000640 1 -- 172.21.15.159:0/1724940719 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17840685c0 con 0x7f17840cd460 2024-04-04T06:12:22.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.879+0000 7f177be00640 1 -- 172.21.15.159:0/1724940719 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1774044070 con 0x7f17840cd460 2024-04-04T06:12:22.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.880+0000 7f177be00640 1 --2- 172.21.15.159:0/1724940719 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f176006c920 0x7f176006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:22.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.880+0000 7f177be00640 1 -- 172.21.15.159:0/1724940719 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7f17740ad560 con 0x7f17840cd460 2024-04-04T06:12:22.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.880+0000 7f1783e00640 1 --2- 172.21.15.159:0/1724940719 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f176006c920 0x7f176006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:22.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.881+0000 7f1783e00640 1 --2- 172.21.15.159:0/1724940719 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f176006c920 0x7f176006ede0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f177c002230 tx=0x7f177c006040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:22.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:22.885+0000 7f177be00640 1 -- 172.21.15.159:0/1724940719 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f177407ba10 con 0x7f17840cd460 2024-04-04T06:12:23.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.125+0000 7f178b000640 1 -- 172.21.15.159:0/1724940719 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f17840727d0 con 0x7f17840cd460 2024-04-04T06:12:23.119 INFO:tasks.ceph.mon.a.smithi159.stderr:2024-04-04T06:12:23.127+0000 7f1995c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-04T06:12:23.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.130+0000 7f177be00640 1 -- 172.21.15.159:0/1724940719 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) ==== 161+0+0 (secure 0 0 0) 0x7f177407b310 con 0x7f17840cd460 2024-04-04T06:12:23.124 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-04T06:12:23.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.133+0000 7f178b000640 1 -- 172.21.15.159:0/1724940719 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f176006c920 msgr2=0x7f176006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:23.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.133+0000 7f178b000640 1 --2- 172.21.15.159:0/1724940719 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f176006c920 0x7f176006ede0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f177c002230 tx=0x7f177c006040 comp rx=0 tx=0).stop 2024-04-04T06:12:23.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.133+0000 7f178b000640 1 -- 172.21.15.159:0/1724940719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17840cd460 msgr2=0x7f17840ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:23.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.133+0000 7f178b000640 1 --2- 172.21.15.159:0/1724940719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17840cd460 0x7f17840ce6c0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f177402bd00 tx=0x7f1774001120 comp rx=0 tx=0).stop 2024-04-04T06:12:23.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.134+0000 7f178b000640 1 -- 172.21.15.159:0/1724940719 shutdown_connections 2024-04-04T06:12:23.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.134+0000 7f178b000640 1 --2- 172.21.15.159:0/1724940719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17840cd460 0x7f17840ce6c0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:23.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.134+0000 7f178b000640 1 --2- 172.21.15.159:0/1724940719 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17840b2430 0x7f17841f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:23.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.134+0000 7f178b000640 1 --2- 172.21.15.159:0/1724940719 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f176006c920 0x7f176006ede0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:23.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.134+0000 7f178b000640 1 --2- 172.21.15.159:0/1724940719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1768004d40 0x7f178409dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:23.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.134+0000 7f178b000640 1 -- 172.21.15.159:0/1724940719 >> 172.21.15.159:0/1724940719 conn(0x7f178406b750 msgr2=0x7f178415a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:23.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.134+0000 7f178b000640 1 -- 172.21.15.159:0/1724940719 shutdown_connections 2024-04-04T06:12:23.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.135+0000 7f178b000640 1 -- 172.21.15.159:0/1724940719 wait complete. 2024-04-04T06:12:23.140 INFO:tasks.ceph.mds.a.smithi159.stderr: -4438> 2024-04-04T06:11:03.956+0000 7fc061a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:12:23.146 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-04T06:12:23.198 INFO:tasks.ceph.mds.a.smithi159.stdout:starting mds.a at 2024-04-04T06:12:23.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.348+0000 7f5334c00640 1 Processor -- start 2024-04-04T06:12:23.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.349+0000 7f5334c00640 1 -- start start 2024-04-04T06:12:23.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.350+0000 7f5334c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f53300d0990 0x7f53300aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:23.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.350+0000 7f5334c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53300afab0 con 0x7f53300af2e0 2024-04-04T06:12:23.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.350+0000 7f5334c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f533015cc80 con 0x7f5330153580 2024-04-04T06:12:23.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.350+0000 7f5334c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f533015da00 con 0x7f53300d0990 2024-04-04T06:12:23.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.350+0000 7f532e800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f53300af2e0 0x7f53301600d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:47352/0 (socket says 172.21.15.159:47352) 2024-04-04T06:12:23.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.350+0000 7f532e800640 1 -- 172.21.15.159:0/3572203389 learned_addr learned my addr 172.21.15.159:0/3572203389 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:23.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.351+0000 7f5325400640 1 --2- 172.21.15.159:0/3572203389 >> v2:172.21.15.178:3300/0 conn(0x7f53300d0990 0x7f53300aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:23.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.351+0000 7f532d400640 1 -- 172.21.15.159:0/3572203389 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1741366390 0 0) 0x7f53300afab0 con 0x7f53300af2e0 2024-04-04T06:12:23.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.351+0000 7f532d400640 1 -- 172.21.15.159:0/3572203389 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5314003520 con 0x7f53300af2e0 2024-04-04T06:12:23.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.351+0000 7f532d400640 1 -- 172.21.15.159:0/3572203389 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 224750596 0 0) 0x7f5314003520 con 0x7f53300af2e0 2024-04-04T06:12:23.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.351+0000 7f532d400640 1 -- 172.21.15.159:0/3572203389 >> v2:172.21.15.178:3300/0 conn(0x7f53300d0990 msgr2=0x7f53300aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:23.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.351+0000 7f532d400640 1 --2- 172.21.15.159:0/3572203389 >> v2:172.21.15.178:3300/0 conn(0x7f53300d0990 0x7f53300aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:23.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.351+0000 7f532d400640 1 -- 172.21.15.159:0/3572203389 >> v1:172.21.15.178:6789/0 conn(0x7f5330153580 legacy=0x7f53300d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:23.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.351+0000 7f532d400640 1 -- 172.21.15.159:0/3572203389 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533015cf20 con 0x7f53300af2e0 2024-04-04T06:12:23.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.351+0000 7f532d400640 1 -- 172.21.15.159:0/3572203389 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f53280019f0 con 0x7f53300af2e0 2024-04-04T06:12:23.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.351+0000 7f532d400640 1 -- 172.21.15.159:0/3572203389 >> v1:172.21.15.159:6789/0 conn(0x7f53300af2e0 legacy=0x7f53301600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:23.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.351+0000 7f532d400640 1 --2- 172.21.15.159:0/3572203389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5314003a00 0x7f5314003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:23.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.351+0000 7f532d400640 1 --2- 172.21.15.159:0/3572203389 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5314004800 0x7f5314004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:23.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.351+0000 7f532d400640 1 --2- 172.21.15.159:0/3572203389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5314005210 0x7f531400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:23.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.351+0000 7f532d400640 1 -- 172.21.15.159:0/3572203389 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f533015da00 con 0x7f5314003a00 2024-04-04T06:12:23.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.351+0000 7f532d400640 1 -- 172.21.15.159:0/3572203389 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f531400c160 con 0x7f5314005210 2024-04-04T06:12:23.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.351+0000 7f532d400640 1 -- 172.21.15.159:0/3572203389 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f531400c360 con 0x7f5314004800 2024-04-04T06:12:23.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.352+0000 7f532de00640 1 --2- 172.21.15.159:0/3572203389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5314003a00 0x7f5314003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:23.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.352+0000 7f532de00640 1 -- 172.21.15.159:0/3572203389 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5314004800 msgr2=0x7f5314004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:23.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.352+0000 7f532de00640 1 --2- 172.21.15.159:0/3572203389 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5314004800 0x7f5314004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:23.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.352+0000 7f532e800640 1 --2- 172.21.15.159:0/3572203389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5314005210 0x7f531400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:23.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.352+0000 7f532de00640 1 -- 172.21.15.159:0/3572203389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5314005210 msgr2=0x7f531400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:23.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.352+0000 7f532de00640 1 --2- 172.21.15.159:0/3572203389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5314005210 0x7f531400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:23.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.352+0000 7f532e800640 1 -- 172.21.15.159:0/3572203389 reap_dead start 2024-04-04T06:12:23.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.352+0000 7f532de00640 1 -- 172.21.15.159:0/3572203389 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5314003520 con 0x7f5314003a00 2024-04-04T06:12:23.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.353+0000 7f532de00640 1 --2- 172.21.15.159:0/3572203389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5314003a00 0x7f5314003e20 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f5320001720 tx=0x7f5314005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599f4545687df91e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:23.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.353+0000 7f532d400640 1 -- 172.21.15.159:0/3572203389 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320001370 con 0x7f5314003a00 2024-04-04T06:12:23.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.353+0000 7f532d400640 1 -- 172.21.15.159:0/3572203389 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53200052d0 con 0x7f5314003a00 2024-04-04T06:12:23.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.353+0000 7f532d400640 1 -- 172.21.15.159:0/3572203389 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320003830 con 0x7f5314003a00 2024-04-04T06:12:23.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.354+0000 7f5334c00640 1 -- 172.21.15.159:0/3572203389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5314003a00 msgr2=0x7f5314003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:23.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.354+0000 7f5334c00640 1 --2- 172.21.15.159:0/3572203389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5314003a00 0x7f5314003e20 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f5320001720 tx=0x7f5314005830 comp rx=0 tx=0).stop 2024-04-04T06:12:23.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.354+0000 7f5334c00640 1 -- 172.21.15.159:0/3572203389 shutdown_connections 2024-04-04T06:12:23.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.354+0000 7f5334c00640 1 --2- 172.21.15.159:0/3572203389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5314003a00 0x7f5314003e20 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:23.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.354+0000 7f5334c00640 1 -- 172.21.15.159:0/3572203389 >> 172.21.15.159:0/3572203389 conn(0x7f533006b750 msgr2=0x7f533006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:23.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.354+0000 7f5334c00640 1 -- 172.21.15.159:0/3572203389 shutdown_connections 2024-04-04T06:12:23.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.354+0000 7f5334c00640 1 -- 172.21.15.159:0/3572203389 wait complete. 2024-04-04T06:12:23.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.356+0000 7f5334c00640 1 Processor -- start 2024-04-04T06:12:23.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.357+0000 7f5334c00640 1 -- start start 2024-04-04T06:12:23.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.358+0000 7f5334c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5314003a00 0x7f533014a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:23.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.358+0000 7f5334c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53300cf450 0x7f53300b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:23.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.358+0000 7f5334c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53300cf8d0 0x7f53300b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:23.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.358+0000 7f5334c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5330153960 con 0x7f53300cf8d0 2024-04-04T06:12:23.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.358+0000 7f5334c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f533015d4e0 con 0x7f53300cf450 2024-04-04T06:12:23.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.358+0000 7f5334c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f53300af6c0 con 0x7f5314003a00 2024-04-04T06:12:23.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.358+0000 7f532e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53300cf8d0 0x7f53300b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:23.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.358+0000 7f532e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53300cf8d0 0x7f53300b3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59204/0 (socket says 172.21.15.159:59204) 2024-04-04T06:12:23.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.358+0000 7f532e800640 1 -- 172.21.15.159:0/767628310 learned_addr learned my addr 172.21.15.159:0/767628310 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:23.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.358+0000 7f5325400640 1 --2- 172.21.15.159:0/767628310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53300cf450 0x7f53300b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:23.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.358+0000 7f532de00640 1 --2- 172.21.15.159:0/767628310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5314003a00 0x7f533014a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:23.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.358+0000 7f532e800640 1 -- 172.21.15.159:0/767628310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5314003a00 msgr2=0x7f533014a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:23.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.358+0000 7f532e800640 1 --2- 172.21.15.159:0/767628310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5314003a00 0x7f533014a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:23.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.358+0000 7f532e800640 1 -- 172.21.15.159:0/767628310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53300cf450 msgr2=0x7f53300b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:23.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.358+0000 7f532e800640 1 --2- 172.21.15.159:0/767628310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53300cf450 0x7f53300b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:23.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.358+0000 7f532e800640 1 -- 172.21.15.159:0/767628310 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53300cda60 con 0x7f53300cf8d0 2024-04-04T06:12:23.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.359+0000 7f532e800640 1 --2- 172.21.15.159:0/767628310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53300cf8d0 0x7f53300b3eb0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f533006c4e0 tx=0x7f532800df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:23.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.359+0000 7f5326a00640 1 -- 172.21.15.159:0/767628310 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532800b800 con 0x7f53300cf8d0 2024-04-04T06:12:23.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.359+0000 7f5326a00640 1 -- 172.21.15.159:0/767628310 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f532800b9a0 con 0x7f53300cf8d0 2024-04-04T06:12:23.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.359+0000 7f5326a00640 1 -- 172.21.15.159:0/767628310 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532800bd90 con 0x7f53300cf8d0 2024-04-04T06:12:23.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.359+0000 7f5334c00640 1 -- 172.21.15.159:0/767628310 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53300ce690 con 0x7f53300cf8d0 2024-04-04T06:12:23.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.359+0000 7f5334c00640 1 -- 172.21.15.159:0/767628310 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53300cec50 con 0x7f53300cf8d0 2024-04-04T06:12:23.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.360+0000 7f5334c00640 1 -- 172.21.15.159:0/767628310 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53300685c0 con 0x7f53300cf8d0 2024-04-04T06:12:23.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.361+0000 7f5326a00640 1 -- 172.21.15.159:0/767628310 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f532801b020 con 0x7f53300cf8d0 2024-04-04T06:12:23.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.362+0000 7f5326a00640 1 --2- 172.21.15.159:0/767628310 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f530c06c8f0 0x7f530c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:23.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.362+0000 7f5326a00640 1 -- 172.21.15.159:0/767628310 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5939+0+0 (secure 0 0 0) 0x7f532808bc90 con 0x7f53300cf8d0 2024-04-04T06:12:23.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.363+0000 7f532de00640 1 --2- 172.21.15.159:0/767628310 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f530c06c8f0 0x7f530c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:23.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.364+0000 7f532de00640 1 --2- 172.21.15.159:0/767628310 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f530c06c8f0 0x7f530c06edb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5320033000 tx=0x7f5320009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:23.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.367+0000 7f5326a00640 1 -- 172.21.15.159:0/767628310 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5328059fc0 con 0x7f53300cf8d0 2024-04-04T06:12:23.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:23.608+0000 7f5334c00640 1 -- 172.21.15.159:0/767628310 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f533006eca0 con 0x7f53300cf8d0 2024-04-04T06:12:24.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.134+0000 7f5326a00640 1 -- 172.21.15.159:0/767628310 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) ==== 104+0+0 (secure 0 0 0) 0x7f53280598c0 con 0x7f53300cf8d0 2024-04-04T06:12:24.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.137+0000 7f5334c00640 1 -- 172.21.15.159:0/767628310 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f530c06c8f0 msgr2=0x7f530c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:24.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.137+0000 7f5334c00640 1 --2- 172.21.15.159:0/767628310 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f530c06c8f0 0x7f530c06edb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5320033000 tx=0x7f5320009030 comp rx=0 tx=0).stop 2024-04-04T06:12:24.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.138+0000 7f5334c00640 1 -- 172.21.15.159:0/767628310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53300cf8d0 msgr2=0x7f53300b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:24.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.138+0000 7f5334c00640 1 --2- 172.21.15.159:0/767628310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53300cf8d0 0x7f53300b3eb0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f533006c4e0 tx=0x7f532800df70 comp rx=0 tx=0).stop 2024-04-04T06:12:24.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.138+0000 7f5334c00640 1 -- 172.21.15.159:0/767628310 shutdown_connections 2024-04-04T06:12:24.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.138+0000 7f5334c00640 1 --2- 172.21.15.159:0/767628310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53300cf8d0 0x7f53300b3eb0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:24.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.138+0000 7f5334c00640 1 --2- 172.21.15.159:0/767628310 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f530c06c8f0 0x7f530c06edb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:24.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.138+0000 7f5334c00640 1 --2- 172.21.15.159:0/767628310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53300cf450 0x7f53300b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:24.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.138+0000 7f5334c00640 1 --2- 172.21.15.159:0/767628310 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5314003a00 0x7f533014a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:24.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.138+0000 7f5334c00640 1 -- 172.21.15.159:0/767628310 >> 172.21.15.159:0/767628310 conn(0x7f533006b750 msgr2=0x7f53300b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:24.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.138+0000 7f5334c00640 1 -- 172.21.15.159:0/767628310 shutdown_connections 2024-04-04T06:12:24.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.138+0000 7f5334c00640 1 -- 172.21.15.159:0/767628310 wait complete. 2024-04-04T06:12:24.152 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-04T06:12:24.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.369+0000 7f85a7e00640 1 Processor -- start 2024-04-04T06:12:24.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.369+0000 7f85a7e00640 1 -- start start 2024-04-04T06:12:24.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.371+0000 7f85a7e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f85a8162d00 0x7f85a8072420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:24.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.371+0000 7f85a7e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85a806ea70 0x7f85a816c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:24.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.371+0000 7f85a7e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85a8072e50 con 0x7f85a8072960 2024-04-04T06:12:24.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.371+0000 7f85a7e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f85a806d5d0 con 0x7f85a8162d00 2024-04-04T06:12:24.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.371+0000 7f85a7e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f85a8073130 con 0x7f85a806ea70 2024-04-04T06:12:24.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.371+0000 7f85a6a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f85a8162d00 0x7f85a8072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:24.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.372+0000 7f85a6a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f85a8162d00 0x7f85a8072420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33950/0 (socket says 172.21.15.159:33950) 2024-04-04T06:12:24.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.372+0000 7f85a6a00640 1 -- 172.21.15.159:0/4163117909 learned_addr learned my addr 172.21.15.159:0/4163117909 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:24.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.372+0000 7f85a6a00640 1 -- 172.21.15.159:0/4163117909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85a806ea70 msgr2=0x7f85a816c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:24.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.372+0000 7f85a6a00640 1 --2- 172.21.15.159:0/4163117909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85a806ea70 0x7f85a816c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:24.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.372+0000 7f85a6a00640 1 -- 172.21.15.159:0/4163117909 >> v1:172.21.15.178:6789/0 conn(0x7f85a8072960 legacy=0x7f85a81630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:24.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.372+0000 7f85a6a00640 1 -- 172.21.15.159:0/4163117909 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a8073410 con 0x7f85a8162d00 2024-04-04T06:12:24.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.373+0000 7f85a6a00640 1 --2- 172.21.15.159:0/4163117909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a8162d00 0x7f85a8072420 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f8598009870 tx=0x7f859802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a982595533670ca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:24.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.373+0000 7f85a6000640 1 -- 172.21.15.159:0/4163117909 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85980317c0 con 0x7f85a8162d00 2024-04-04T06:12:24.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.373+0000 7f85a6000640 1 -- 172.21.15.159:0/4163117909 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8598037070 con 0x7f85a8162d00 2024-04-04T06:12:24.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.373+0000 7f85a6000640 1 -- 172.21.15.159:0/4163117909 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598031b80 con 0x7f85a8162d00 2024-04-04T06:12:24.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.374+0000 7f85a7e00640 1 -- 172.21.15.159:0/4163117909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a8162d00 msgr2=0x7f85a8072420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:24.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.374+0000 7f85a7e00640 1 --2- 172.21.15.159:0/4163117909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a8162d00 0x7f85a8072420 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f8598009870 tx=0x7f859802b860 comp rx=0 tx=0).stop 2024-04-04T06:12:24.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.374+0000 7f85a7e00640 1 -- 172.21.15.159:0/4163117909 shutdown_connections 2024-04-04T06:12:24.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.374+0000 7f85a7e00640 1 --2- 172.21.15.159:0/4163117909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85a806ea70 0x7f85a816c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:24.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.374+0000 7f85a7e00640 1 --2- 172.21.15.159:0/4163117909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a8162d00 0x7f85a8072420 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:24.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.374+0000 7f85a7e00640 1 -- 172.21.15.159:0/4163117909 >> 172.21.15.159:0/4163117909 conn(0x7f85a80b1d80 msgr2=0x7f85a80b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:24.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.374+0000 7f85a7e00640 1 -- 172.21.15.159:0/4163117909 shutdown_connections 2024-04-04T06:12:24.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.374+0000 7f85a7e00640 1 -- 172.21.15.159:0/4163117909 wait complete. 2024-04-04T06:12:24.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.376+0000 7f85a7e00640 1 Processor -- start 2024-04-04T06:12:24.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.376+0000 7f85a7e00640 1 -- start start 2024-04-04T06:12:24.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.377+0000 7f85a7e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85a806ea70 0x7f85a81461b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:24.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.377+0000 7f85a7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a8072960 0x7f85a8158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:24.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.377+0000 7f85a7e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a8162d00 0x7f85a815b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:24.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.377+0000 7f85a7e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f85a80b3bf0 con 0x7f85a8072960 2024-04-04T06:12:24.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.377+0000 7f85a7e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f85a80b3a70 con 0x7f85a8162d00 2024-04-04T06:12:24.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.377+0000 7f85a7e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f85a81637f0 con 0x7f85a806ea70 2024-04-04T06:12:24.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.377+0000 7f859fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a8072960 0x7f85a8158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:24.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.377+0000 7f859fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a8072960 0x7f85a8158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33960/0 (socket says 172.21.15.159:33960) 2024-04-04T06:12:24.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.377+0000 7f859fe00640 1 -- 172.21.15.159:0/1356733329 learned_addr learned my addr 172.21.15.159:0/1356733329 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:24.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.378+0000 7f859fe00640 1 -- 172.21.15.159:0/1356733329 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85a806ea70 msgr2=0x7f85a81461b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:24.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.378+0000 7f859fe00640 1 --2- 172.21.15.159:0/1356733329 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85a806ea70 0x7f85a81461b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:24.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.378+0000 7f859fe00640 1 -- 172.21.15.159:0/1356733329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a8162d00 msgr2=0x7f85a815b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:24.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.378+0000 7f85a7400640 1 --2- 172.21.15.159:0/1356733329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a8162d00 0x7f85a815b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:24.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.378+0000 7f859fe00640 1 --2- 172.21.15.159:0/1356733329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a8162d00 0x7f85a815b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:24.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.378+0000 7f859fe00640 1 -- 172.21.15.159:0/1356733329 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a8159190 con 0x7f85a8072960 2024-04-04T06:12:24.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.378+0000 7f859fe00640 1 --2- 172.21.15.159:0/1356733329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a8072960 0x7f85a8158710 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f859000c400 tx=0x7f859000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:24.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.378+0000 7f859f400640 1 -- 172.21.15.159:0/1356733329 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590011790 con 0x7f85a8072960 2024-04-04T06:12:24.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.378+0000 7f859f400640 1 -- 172.21.15.159:0/1356733329 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8590017070 con 0x7f85a8072960 2024-04-04T06:12:24.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.378+0000 7f859f400640 1 -- 172.21.15.159:0/1356733329 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590011bb0 con 0x7f85a8072960 2024-04-04T06:12:24.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.379+0000 7f85a7e00640 1 -- 172.21.15.159:0/1356733329 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a8159470 con 0x7f85a8072960 2024-04-04T06:12:24.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.379+0000 7f85a7e00640 1 -- 172.21.15.159:0/1356733329 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a8159a30 con 0x7f85a8072960 2024-04-04T06:12:24.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.380+0000 7f85a7e00640 1 -- 172.21.15.159:0/1356733329 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a8075940 con 0x7f85a8072960 2024-04-04T06:12:24.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.381+0000 7f859f400640 1 -- 172.21.15.159:0/1356733329 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f859001c460 con 0x7f85a8072960 2024-04-04T06:12:24.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.382+0000 7f859f400640 1 --2- 172.21.15.159:0/1356733329 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f858406c920 0x7f858406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:24.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.382+0000 7f859f400640 1 -- 172.21.15.159:0/1356733329 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5939+0+0 (secure 0 0 0) 0x7f859008ba00 con 0x7f85a8072960 2024-04-04T06:12:24.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.382+0000 7f85a6a00640 1 --2- 172.21.15.159:0/1356733329 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f858406c920 0x7f858406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:24.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.384+0000 7f85a6a00640 1 --2- 172.21.15.159:0/1356733329 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f858406c920 0x7f858406ede0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8598036000 tx=0x7f8598030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:24.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.386+0000 7f859f400640 1 -- 172.21.15.159:0/1356733329 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8590059dd0 con 0x7f85a8072960 2024-04-04T06:12:24.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:24.584+0000 7f85a7e00640 1 -- 172.21.15.159:0/1356733329 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f85a80b2570 con 0x7f85a8072960 2024-04-04T06:12:25.296 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.302+0000 7f859f400640 1 -- 172.21.15.159:0/1356733329 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v76) ==== 181+0+0 (secure 0 0 0) 0x7f85900596d0 con 0x7f85a8072960 2024-04-04T06:12:25.296 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_metadata' removed 2024-04-04T06:12:25.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.306+0000 7f85a7e00640 1 -- 172.21.15.159:0/1356733329 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f858406c920 msgr2=0x7f858406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:25.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.306+0000 7f85a7e00640 1 --2- 172.21.15.159:0/1356733329 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f858406c920 0x7f858406ede0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8598036000 tx=0x7f8598030000 comp rx=0 tx=0).stop 2024-04-04T06:12:25.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.306+0000 7f85a7e00640 1 -- 172.21.15.159:0/1356733329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a8072960 msgr2=0x7f85a8158710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:25.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.306+0000 7f85a7e00640 1 --2- 172.21.15.159:0/1356733329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a8072960 0x7f85a8158710 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f859000c400 tx=0x7f859000c8c0 comp rx=0 tx=0).stop 2024-04-04T06:12:25.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.307+0000 7f85a7e00640 1 -- 172.21.15.159:0/1356733329 shutdown_connections 2024-04-04T06:12:25.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.307+0000 7f85a7e00640 1 --2- 172.21.15.159:0/1356733329 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f858406c920 0x7f858406ede0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:25.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.307+0000 7f85a7e00640 1 --2- 172.21.15.159:0/1356733329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85a8162d00 0x7f85a815b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:25.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.307+0000 7f85a7e00640 1 --2- 172.21.15.159:0/1356733329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a8072960 0x7f85a8158710 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:25.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.307+0000 7f85a7e00640 1 --2- 172.21.15.159:0/1356733329 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85a806ea70 0x7f85a81461b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:25.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.307+0000 7f85a7e00640 1 -- 172.21.15.159:0/1356733329 >> 172.21.15.159:0/1356733329 conn(0x7f85a80b1d80 msgr2=0x7f85a80b0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:25.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.307+0000 7f85a7e00640 1 -- 172.21.15.159:0/1356733329 shutdown_connections 2024-04-04T06:12:25.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.308+0000 7f85a7e00640 1 -- 172.21.15.159:0/1356733329 wait complete. 2024-04-04T06:12:25.324 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-04T06:12:25.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.545+0000 7fc224600640 1 Processor -- start 2024-04-04T06:12:25.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.545+0000 7fc224600640 1 -- start start 2024-04-04T06:12:25.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.547+0000 7fc224600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc21c0acf10 0x7fc21c09dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:25.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.547+0000 7fc224600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc21c071710 0x7fc21c071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:25.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.547+0000 7fc224600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc21c15cb90 con 0x7fc21c0ad2f0 2024-04-04T06:12:25.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.547+0000 7fc224600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fc21c145f60 con 0x7fc21c071710 2024-04-04T06:12:25.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.547+0000 7fc224600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc21c15c950 con 0x7fc21c0acf10 2024-04-04T06:12:25.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.547+0000 7fc222600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc21c071710 0x7fc21c071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:25.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.547+0000 7fc222600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc21c071710 0x7fc21c071b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33968/0 (socket says 172.21.15.159:33968) 2024-04-04T06:12:25.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.547+0000 7fc222600640 1 -- 172.21.15.159:0/893344352 learned_addr learned my addr 172.21.15.159:0/893344352 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:25.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.547+0000 7fc221c00640 1 --2- 172.21.15.159:0/893344352 >> v2:172.21.15.178:3300/0 conn(0x7fc21c0acf10 0x7fc21c09dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:25.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.548+0000 7fc222600640 1 -- 172.21.15.159:0/893344352 >> v2:172.21.15.178:3300/0 conn(0x7fc21c0acf10 msgr2=0x7fc21c09dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:25.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.548+0000 7fc222600640 1 --2- 172.21.15.159:0/893344352 >> v2:172.21.15.178:3300/0 conn(0x7fc21c0acf10 0x7fc21c09dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:25.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.548+0000 7fc222600640 1 -- 172.21.15.159:0/893344352 >> v1:172.21.15.178:6789/0 conn(0x7fc21c0ad2f0 legacy=0x7fc21c1453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:25.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.548+0000 7fc222600640 1 -- 172.21.15.159:0/893344352 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21c15ce70 con 0x7fc21c071710 2024-04-04T06:12:25.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.549+0000 7fc222600640 1 --2- 172.21.15.159:0/893344352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc21c071710 0x7fc21c071b20 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fc21c06c580 tx=0x7fc21402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90422cca900ef18d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:25.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.549+0000 7fc21be00640 1 -- 172.21.15.159:0/893344352 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21402c9f0 con 0x7fc21c071710 2024-04-04T06:12:25.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.549+0000 7fc21be00640 1 -- 172.21.15.159:0/893344352 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc21402cb90 con 0x7fc21c071710 2024-04-04T06:12:25.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.549+0000 7fc21be00640 1 -- 172.21.15.159:0/893344352 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2140343f0 con 0x7fc21c071710 2024-04-04T06:12:25.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.550+0000 7fc224600640 1 -- 172.21.15.159:0/893344352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc21c071710 msgr2=0x7fc21c071b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:25.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.550+0000 7fc224600640 1 --2- 172.21.15.159:0/893344352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc21c071710 0x7fc21c071b20 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fc21c06c580 tx=0x7fc21402be50 comp rx=0 tx=0).stop 2024-04-04T06:12:25.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.550+0000 7fc224600640 1 -- 172.21.15.159:0/893344352 shutdown_connections 2024-04-04T06:12:25.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.550+0000 7fc224600640 1 --2- 172.21.15.159:0/893344352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc21c071710 0x7fc21c071b20 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:25.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.550+0000 7fc224600640 1 --2- 172.21.15.159:0/893344352 >> v2:172.21.15.178:3300/0 conn(0x7fc21c0acf10 0x7fc21c09dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:25.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.550+0000 7fc224600640 1 -- 172.21.15.159:0/893344352 >> 172.21.15.159:0/893344352 conn(0x7fc21c06bba0 msgr2=0x7fc21c0b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:25.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.550+0000 7fc224600640 1 -- 172.21.15.159:0/893344352 shutdown_connections 2024-04-04T06:12:25.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.550+0000 7fc224600640 1 -- 172.21.15.159:0/893344352 wait complete. 2024-04-04T06:12:25.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.552+0000 7fc224600640 1 Processor -- start 2024-04-04T06:12:25.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.553+0000 7fc224600640 1 -- start start 2024-04-04T06:12:25.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.554+0000 7fc224600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc21c071710 0x7fc21c1447c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:25.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.554+0000 7fc224600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc21c0acf10 0x7fc21c141c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:25.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.554+0000 7fc224600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc21c0ad2f0 0x7fc21c13fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:25.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.554+0000 7fc224600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c15d430 con 0x7fc21c0ad2f0 2024-04-04T06:12:25.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.554+0000 7fc224600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c15d7a0 con 0x7fc21c071710 2024-04-04T06:12:25.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.554+0000 7fc224600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc21c0ad7a0 con 0x7fc21c0acf10 2024-04-04T06:12:25.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.554+0000 7fc222600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc21c0ad2f0 0x7fc21c13fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:25.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.554+0000 7fc222600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc21c0ad2f0 0x7fc21c13fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33974/0 (socket says 172.21.15.159:33974) 2024-04-04T06:12:25.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.554+0000 7fc222600640 1 -- 172.21.15.159:0/2937800484 learned_addr learned my addr 172.21.15.159:0/2937800484 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:25.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.554+0000 7fc221c00640 1 --2- 172.21.15.159:0/2937800484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc21c071710 0x7fc21c1447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:25.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.554+0000 7fc221200640 1 --2- 172.21.15.159:0/2937800484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc21c0acf10 0x7fc21c141c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:25.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.555+0000 7fc222600640 1 -- 172.21.15.159:0/2937800484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc21c0acf10 msgr2=0x7fc21c141c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:25.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.555+0000 7fc222600640 1 --2- 172.21.15.159:0/2937800484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc21c0acf10 0x7fc21c141c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:25.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.555+0000 7fc222600640 1 -- 172.21.15.159:0/2937800484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc21c071710 msgr2=0x7fc21c1447c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:25.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.555+0000 7fc222600640 1 --2- 172.21.15.159:0/2937800484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc21c071710 0x7fc21c1447c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:25.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.555+0000 7fc222600640 1 -- 172.21.15.159:0/2937800484 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21c143a00 con 0x7fc21c0ad2f0 2024-04-04T06:12:25.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.555+0000 7fc221c00640 1 --2- 172.21.15.159:0/2937800484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc21c071710 0x7fc21c1447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:25.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.555+0000 7fc221200640 1 --2- 172.21.15.159:0/2937800484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc21c0acf10 0x7fc21c141c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:25.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.555+0000 7fc222600640 1 --2- 172.21.15.159:0/2937800484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc21c0ad2f0 0x7fc21c13fff0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fc214036040 tx=0x7fc214004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:25.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.555+0000 7fc219600640 1 -- 172.21.15.159:0/2937800484 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc214043020 con 0x7fc21c0ad2f0 2024-04-04T06:12:25.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.555+0000 7fc219600640 1 -- 172.21.15.159:0/2937800484 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc21403e450 con 0x7fc21c0ad2f0 2024-04-04T06:12:25.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.555+0000 7fc219600640 1 -- 172.21.15.159:0/2937800484 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc214009510 con 0x7fc21c0ad2f0 2024-04-04T06:12:25.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.555+0000 7fc224600640 1 -- 172.21.15.159:0/2937800484 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc21c143ce0 con 0x7fc21c0ad2f0 2024-04-04T06:12:25.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.556+0000 7fc224600640 1 -- 172.21.15.159:0/2937800484 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc21c1442a0 con 0x7fc21c0ad2f0 2024-04-04T06:12:25.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.557+0000 7fc224600640 1 -- 172.21.15.159:0/2937800484 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc21c0685c0 con 0x7fc21c0ad2f0 2024-04-04T06:12:25.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.558+0000 7fc219600640 1 -- 172.21.15.159:0/2937800484 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc214046020 con 0x7fc21c0ad2f0 2024-04-04T06:12:25.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.558+0000 7fc219600640 1 --2- 172.21.15.159:0/2937800484 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc1fc06c8f0 0x7fc1fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:25.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.558+0000 7fc219600640 1 -- 172.21.15.159:0/2937800484 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5501+0+0 (secure 0 0 0) 0x7fc2140ae3e0 con 0x7fc21c0ad2f0 2024-04-04T06:12:25.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.559+0000 7fc221c00640 1 --2- 172.21.15.159:0/2937800484 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc1fc06c8f0 0x7fc1fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:25.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.560+0000 7fc221c00640 1 --2- 172.21.15.159:0/2937800484 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc1fc06c8f0 0x7fc1fc06edb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc210006fd0 tx=0x7fc210008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:25.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.564+0000 7fc219600640 1 -- 172.21.15.159:0/2937800484 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc21404d030 con 0x7fc21c0ad2f0 2024-04-04T06:12:25.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:25.765+0000 7fc224600640 1 -- 172.21.15.159:0/2937800484 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc21c144470 con 0x7fc21c0ad2f0 2024-04-04T06:12:26.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.314+0000 7fc219600640 1 -- 172.21.15.159:0/2937800484 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v77) ==== 169+0+0 (secure 0 0 0) 0x7fc21407c7b0 con 0x7fc21c0ad2f0 2024-04-04T06:12:26.308 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_data' removed 2024-04-04T06:12:26.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.319+0000 7fc224600640 1 -- 172.21.15.159:0/2937800484 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc1fc06c8f0 msgr2=0x7fc1fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:26.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.319+0000 7fc224600640 1 --2- 172.21.15.159:0/2937800484 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc1fc06c8f0 0x7fc1fc06edb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc210006fd0 tx=0x7fc210008040 comp rx=0 tx=0).stop 2024-04-04T06:12:26.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.319+0000 7fc224600640 1 -- 172.21.15.159:0/2937800484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc21c0ad2f0 msgr2=0x7fc21c13fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:26.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.319+0000 7fc224600640 1 --2- 172.21.15.159:0/2937800484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc21c0ad2f0 0x7fc21c13fff0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fc214036040 tx=0x7fc214004a40 comp rx=0 tx=0).stop 2024-04-04T06:12:26.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.319+0000 7fc224600640 1 -- 172.21.15.159:0/2937800484 shutdown_connections 2024-04-04T06:12:26.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.319+0000 7fc224600640 1 --2- 172.21.15.159:0/2937800484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc21c0ad2f0 0x7fc21c13fff0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:26.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.319+0000 7fc224600640 1 --2- 172.21.15.159:0/2937800484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc21c0acf10 0x7fc21c141c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:26.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.319+0000 7fc224600640 1 --2- 172.21.15.159:0/2937800484 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc1fc06c8f0 0x7fc1fc06edb0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:26.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.319+0000 7fc224600640 1 --2- 172.21.15.159:0/2937800484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc21c071710 0x7fc21c1447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:26.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.319+0000 7fc224600640 1 -- 172.21.15.159:0/2937800484 >> 172.21.15.159:0/2937800484 conn(0x7fc21c06bba0 msgr2=0x7fc21c071af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:26.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.320+0000 7fc224600640 1 -- 172.21.15.159:0/2937800484 shutdown_connections 2024-04-04T06:12:26.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.320+0000 7fc224600640 1 -- 172.21.15.159:0/2937800484 wait complete. 2024-04-04T06:12:26.337 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-04T06:12:26.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.549+0000 7fd866200640 1 Processor -- start 2024-04-04T06:12:26.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.549+0000 7fd866200640 1 -- start start 2024-04-04T06:12:26.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.550+0000 7fd866200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd86015fc50 0x7fd8600cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:26.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.550+0000 7fd866200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd8600cc490 0x7fd8600d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:26.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.550+0000 7fd866200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8600b3150 con 0x7fd8600cc0b0 2024-04-04T06:12:26.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.550+0000 7fd866200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fd86015c610 con 0x7fd8600cc490 2024-04-04T06:12:26.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.550+0000 7fd866200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd8601600e0 con 0x7fd86015fc50 2024-04-04T06:12:26.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.551+0000 7fd864e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fd8600cc0b0 0x7fd86006dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:59464/0 (socket says 172.21.15.159:59464) 2024-04-04T06:12:26.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.551+0000 7fd864e00640 1 -- 172.21.15.159:0/2785283249 learned_addr learned my addr 172.21.15.159:0/2785283249 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:26.544 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.551+0000 7fd865800640 1 --2- 172.21.15.159:0/2785283249 >> v2:172.21.15.159:3300/0 conn(0x7fd8600cc490 0x7fd8600d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:26.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.551+0000 7fd857e00640 1 --2- 172.21.15.159:0/2785283249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd86015fc50 0x7fd8600cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:26.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.551+0000 7fd85fe00640 1 -- 172.21.15.159:0/2785283249 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2094638330 0 0) 0x7fd8600b3150 con 0x7fd8600cc0b0 2024-04-04T06:12:26.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.551+0000 7fd85fe00640 1 -- 172.21.15.159:0/2785283249 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd844003520 con 0x7fd8600cc0b0 2024-04-04T06:12:26.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.551+0000 7fd865800640 1 -- 172.21.15.159:0/2785283249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd86015fc50 msgr2=0x7fd8600cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:26.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.551+0000 7fd865800640 1 --2- 172.21.15.159:0/2785283249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd86015fc50 0x7fd8600cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:26.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.551+0000 7fd865800640 1 -- 172.21.15.159:0/2785283249 >> v1:172.21.15.159:6789/0 conn(0x7fd8600cc0b0 legacy=0x7fd86006dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:26.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.551+0000 7fd865800640 1 -- 172.21.15.159:0/2785283249 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8601603c0 con 0x7fd8600cc490 2024-04-04T06:12:26.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.552+0000 7fd865800640 1 --2- 172.21.15.159:0/2785283249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8600cc490 0x7fd8600d0300 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd858009d90 tx=0x7fd85802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19191b3d1a5fd5d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:26.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.553+0000 7fd85fe00640 1 -- 172.21.15.159:0/2785283249 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858039070 con 0x7fd8600cc490 2024-04-04T06:12:26.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.553+0000 7fd85fe00640 1 -- 172.21.15.159:0/2785283249 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd85802cc50 con 0x7fd8600cc490 2024-04-04T06:12:26.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.553+0000 7fd85fe00640 1 -- 172.21.15.159:0/2785283249 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858034440 con 0x7fd8600cc490 2024-04-04T06:12:26.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.553+0000 7fd866200640 1 -- 172.21.15.159:0/2785283249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8600cc490 msgr2=0x7fd8600d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:26.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.553+0000 7fd866200640 1 --2- 172.21.15.159:0/2785283249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8600cc490 0x7fd8600d0300 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd858009d90 tx=0x7fd85802bea0 comp rx=0 tx=0).stop 2024-04-04T06:12:26.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.553+0000 7fd866200640 1 -- 172.21.15.159:0/2785283249 shutdown_connections 2024-04-04T06:12:26.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.553+0000 7fd866200640 1 --2- 172.21.15.159:0/2785283249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd86015fc50 0x7fd8600cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:26.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.553+0000 7fd866200640 1 --2- 172.21.15.159:0/2785283249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8600cc490 0x7fd8600d0300 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:26.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.553+0000 7fd866200640 1 -- 172.21.15.159:0/2785283249 >> 172.21.15.159:0/2785283249 conn(0x7fd8600ccad0 msgr2=0x7fd8600cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:26.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.554+0000 7fd866200640 1 -- 172.21.15.159:0/2785283249 shutdown_connections 2024-04-04T06:12:26.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.554+0000 7fd866200640 1 -- 172.21.15.159:0/2785283249 wait complete. 2024-04-04T06:12:26.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.555+0000 7fd866200640 1 Processor -- start 2024-04-04T06:12:26.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.555+0000 7fd866200640 1 -- start start 2024-04-04T06:12:26.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.556+0000 7fd866200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8600cc0b0 0x7fd8601e5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:26.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.556+0000 7fd866200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8600cc490 0x7fd8600cf370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:26.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.556+0000 7fd866200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd860072870 0x7fd86013e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:26.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.556+0000 7fd866200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd86015ca70 con 0x7fd860072870 2024-04-04T06:12:26.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.556+0000 7fd866200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd860160980 con 0x7fd8600cc490 2024-04-04T06:12:26.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.556+0000 7fd866200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd86015cbf0 con 0x7fd8600cc0b0 2024-04-04T06:12:26.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.557+0000 7fd865800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd860072870 0x7fd86013e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:26.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.557+0000 7fd865800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd860072870 0x7fd86013e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33984/0 (socket says 172.21.15.159:33984) 2024-04-04T06:12:26.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.557+0000 7fd865800640 1 -- 172.21.15.159:0/3970809136 learned_addr learned my addr 172.21.15.159:0/3970809136 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:26.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.557+0000 7fd865800640 1 -- 172.21.15.159:0/3970809136 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8600cc0b0 msgr2=0x7fd8601e5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:26.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.557+0000 7fd865800640 1 --2- 172.21.15.159:0/3970809136 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8600cc0b0 0x7fd8601e5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:26.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.557+0000 7fd865800640 1 -- 172.21.15.159:0/3970809136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8600cc490 msgr2=0x7fd8600cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:26.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.557+0000 7fd857e00640 1 --2- 172.21.15.159:0/3970809136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8600cc490 0x7fd8600cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:26.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.557+0000 7fd865800640 1 --2- 172.21.15.159:0/3970809136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8600cc490 0x7fd8600cf370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:26.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.557+0000 7fd865800640 1 -- 172.21.15.159:0/3970809136 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8600cf8b0 con 0x7fd860072870 2024-04-04T06:12:26.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.557+0000 7fd865800640 1 --2- 172.21.15.159:0/3970809136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd860072870 0x7fd86013e6f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd858009d90 tx=0x7fd858004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:26.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.558+0000 7fd85d600640 1 -- 172.21.15.159:0/3970809136 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858039050 con 0x7fd860072870 2024-04-04T06:12:26.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.558+0000 7fd85d600640 1 -- 172.21.15.159:0/3970809136 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd858045070 con 0x7fd860072870 2024-04-04T06:12:26.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.558+0000 7fd866200640 1 -- 172.21.15.159:0/3970809136 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd86013ec30 con 0x7fd860072870 2024-04-04T06:12:26.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.558+0000 7fd85d600640 1 -- 172.21.15.159:0/3970809136 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858009450 con 0x7fd860072870 2024-04-04T06:12:26.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.558+0000 7fd866200640 1 -- 172.21.15.159:0/3970809136 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8601e6610 con 0x7fd860072870 2024-04-04T06:12:26.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.559+0000 7fd866200640 1 -- 172.21.15.159:0/3970809136 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd860075940 con 0x7fd860072870 2024-04-04T06:12:26.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.560+0000 7fd85d600640 1 -- 172.21.15.159:0/3970809136 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd85803e610 con 0x7fd860072870 2024-04-04T06:12:26.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.561+0000 7fd85d600640 1 --2- 172.21.15.159:0/3970809136 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd83c06c8f0 0x7fd83c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:26.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.561+0000 7fd85d600640 1 -- 172.21.15.159:0/3970809136 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5029+0+0 (secure 0 0 0) 0x7fd8580ae4a0 con 0x7fd860072870 2024-04-04T06:12:26.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.561+0000 7fd864e00640 1 --2- 172.21.15.159:0/3970809136 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd83c06c8f0 0x7fd83c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:26.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.562+0000 7fd864e00640 1 --2- 172.21.15.159:0/3970809136 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd83c06c8f0 0x7fd83c06edb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd850000f60 tx=0x7fd8500078f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:26.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.565+0000 7fd85d600640 1 -- 172.21.15.159:0/3970809136 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd8600cf8b0 con 0x7fd860072870 2024-04-04T06:12:26.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.765+0000 7fd866200640 1 -- 172.21.15.159:0/3970809136 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fd8601e6050 con 0x7fd860072870 2024-04-04T06:12:26.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.771+0000 7fd85d600640 1 -- 172.21.15.159:0/3970809136 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7fd85807ca40 con 0x7fd860072870 2024-04-04T06:12:26.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.774+0000 7fd866200640 1 -- 172.21.15.159:0/3970809136 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd83c06c8f0 msgr2=0x7fd83c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:26.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.774+0000 7fd866200640 1 --2- 172.21.15.159:0/3970809136 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd83c06c8f0 0x7fd83c06edb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd850000f60 tx=0x7fd8500078f0 comp rx=0 tx=0).stop 2024-04-04T06:12:26.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.774+0000 7fd866200640 1 -- 172.21.15.159:0/3970809136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd860072870 msgr2=0x7fd86013e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:26.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.774+0000 7fd866200640 1 --2- 172.21.15.159:0/3970809136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd860072870 0x7fd86013e6f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd858009d90 tx=0x7fd858004a40 comp rx=0 tx=0).stop 2024-04-04T06:12:26.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.774+0000 7fd866200640 1 -- 172.21.15.159:0/3970809136 shutdown_connections 2024-04-04T06:12:26.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.774+0000 7fd866200640 1 --2- 172.21.15.159:0/3970809136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd860072870 0x7fd86013e6f0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:26.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.774+0000 7fd866200640 1 --2- 172.21.15.159:0/3970809136 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd83c06c8f0 0x7fd83c06edb0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:26.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.774+0000 7fd866200640 1 --2- 172.21.15.159:0/3970809136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8600cc490 0x7fd8600cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:26.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.774+0000 7fd866200640 1 --2- 172.21.15.159:0/3970809136 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8600cc0b0 0x7fd8601e5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:26.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.774+0000 7fd866200640 1 -- 172.21.15.159:0/3970809136 >> 172.21.15.159:0/3970809136 conn(0x7fd8600ccad0 msgr2=0x7fd86015d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:26.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.774+0000 7fd866200640 1 -- 172.21.15.159:0/3970809136 shutdown_connections 2024-04-04T06:12:26.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.774+0000 7fd866200640 1 -- 172.21.15.159:0/3970809136 wait complete. 2024-04-04T06:12:26.787 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-04T06:12:26.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.986+0000 7f440aa00640 1 Processor -- start 2024-04-04T06:12:26.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.986+0000 7f440aa00640 1 -- start start 2024-04-04T06:12:26.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.987+0000 7f440aa00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44040af4b0 0x7f4404167460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:26.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.988+0000 7f440aa00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44040afab0 con 0x7f4404153580 2024-04-04T06:12:26.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.988+0000 7f440aa00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f440415cc80 con 0x7f44040d0990 2024-04-04T06:12:26.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.988+0000 7f440aa00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f440415da00 con 0x7f44040af4b0 2024-04-04T06:12:26.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.988+0000 7f4403e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f4404153580 0x7f44040d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:59478/0 (socket says 172.21.15.159:59478) 2024-04-04T06:12:26.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.988+0000 7f4403e00640 1 -- 172.21.15.159:0/3882354985 learned_addr learned my addr 172.21.15.159:0/3882354985 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:26.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.988+0000 7f4408a00640 1 --2- 172.21.15.159:0/3882354985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44040af4b0 0x7f4404167460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:26.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.988+0000 7f4402a00640 1 -- 172.21.15.159:0/3882354985 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1233019420 0 0) 0x7f44040afab0 con 0x7f4404153580 2024-04-04T06:12:26.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.989+0000 7f4402a00640 1 -- 172.21.15.159:0/3882354985 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43e8003520 con 0x7f4404153580 2024-04-04T06:12:26.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.989+0000 7f4402a00640 1 -- 172.21.15.159:0/3882354985 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 379880586 0 0) 0x7f43e8003520 con 0x7f4404153580 2024-04-04T06:12:26.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.989+0000 7f4402a00640 1 -- 172.21.15.159:0/3882354985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44040af4b0 msgr2=0x7f4404167460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:26.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.989+0000 7f4402a00640 1 --2- 172.21.15.159:0/3882354985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44040af4b0 0x7f4404167460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:26.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.989+0000 7f4402a00640 1 -- 172.21.15.159:0/3882354985 >> v1:172.21.15.178:6789/0 conn(0x7f44040d0990 legacy=0x7f44040aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:26.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.989+0000 7f4402a00640 1 -- 172.21.15.159:0/3882354985 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440415cf20 con 0x7f4404153580 2024-04-04T06:12:26.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.989+0000 7f4402a00640 1 -- 172.21.15.159:0/3882354985 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f43f4001670 con 0x7f4404153580 2024-04-04T06:12:26.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.989+0000 7f4402a00640 1 -- 172.21.15.159:0/3882354985 >> v1:172.21.15.159:6789/0 conn(0x7f4404153580 legacy=0x7f44040d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:26.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.989+0000 7f4402a00640 1 --2- 172.21.15.159:0/3882354985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43e80039e0 0x7f43e8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:26.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.989+0000 7f4402a00640 1 --2- 172.21.15.159:0/3882354985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44040af4b0 0x7f43e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:26.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.989+0000 7f4402a00640 1 --2- 172.21.15.159:0/3882354985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43e8004d40 0x7f43e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:26.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.989+0000 7f4402a00640 1 -- 172.21.15.159:0/3882354985 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f440415da00 con 0x7f43e80039e0 2024-04-04T06:12:26.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.989+0000 7f4402a00640 1 -- 172.21.15.159:0/3882354985 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43e800baf0 con 0x7f43e8004d40 2024-04-04T06:12:26.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.989+0000 7f4402a00640 1 -- 172.21.15.159:0/3882354985 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f43e800bcf0 con 0x7f44040af4b0 2024-04-04T06:12:26.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.989+0000 7f4403e00640 1 --2- 172.21.15.159:0/3882354985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43e80039e0 0x7f43e8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:26.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.990+0000 7f4403400640 1 --2- 172.21.15.159:0/3882354985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44040af4b0 0x7f43e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:26.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.990+0000 7f4408a00640 1 --2- 172.21.15.159:0/3882354985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43e8004d40 0x7f43e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:26.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.990+0000 7f4403e00640 1 -- 172.21.15.159:0/3882354985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44040af4b0 msgr2=0x7f43e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:26.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.990+0000 7f4403e00640 1 --2- 172.21.15.159:0/3882354985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f44040af4b0 0x7f43e8004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:26.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.990+0000 7f4403e00640 1 -- 172.21.15.159:0/3882354985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43e8004d40 msgr2=0x7f43e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:26.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.990+0000 7f4403e00640 1 --2- 172.21.15.159:0/3882354985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43e8004d40 0x7f43e80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:26.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.990+0000 7f4403e00640 1 -- 172.21.15.159:0/3882354985 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e8003520 con 0x7f43e80039e0 2024-04-04T06:12:26.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.991+0000 7f4403e00640 1 --2- 172.21.15.159:0/3882354985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43e80039e0 0x7f43e8003e00 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f43f400c240 tx=0x7f43e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1649ec77455fd98f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:26.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.991+0000 7f4402a00640 1 -- 172.21.15.159:0/3882354985 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f4007cf0 con 0x7f43e80039e0 2024-04-04T06:12:26.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.991+0000 7f4402a00640 1 -- 172.21.15.159:0/3882354985 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43f4030070 con 0x7f43e80039e0 2024-04-04T06:12:26.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.991+0000 7f4402a00640 1 -- 172.21.15.159:0/3882354985 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f4005d30 con 0x7f43e80039e0 2024-04-04T06:12:26.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.992+0000 7f440aa00640 1 -- 172.21.15.159:0/3882354985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43e80039e0 msgr2=0x7f43e8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:26.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.992+0000 7f440aa00640 1 --2- 172.21.15.159:0/3882354985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43e80039e0 0x7f43e8003e00 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f43f400c240 tx=0x7f43e8005830 comp rx=0 tx=0).stop 2024-04-04T06:12:26.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.992+0000 7f4408a00640 1 -- 172.21.15.159:0/3882354985 reap_dead start 2024-04-04T06:12:26.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.992+0000 7f440aa00640 1 -- 172.21.15.159:0/3882354985 shutdown_connections 2024-04-04T06:12:26.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.992+0000 7f440aa00640 1 -- 172.21.15.159:0/3882354985 >> 172.21.15.159:0/3882354985 conn(0x7f440406b750 msgr2=0x7f440406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:26.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.992+0000 7f440aa00640 1 -- 172.21.15.159:0/3882354985 shutdown_connections 2024-04-04T06:12:26.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.992+0000 7f440aa00640 1 -- 172.21.15.159:0/3882354985 wait complete. 2024-04-04T06:12:26.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.994+0000 7f440aa00640 1 Processor -- start 2024-04-04T06:12:26.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.994+0000 7f440aa00640 1 -- start start 2024-04-04T06:12:26.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.995+0000 7f440aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43e80039e0 0x7f440409d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:26.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.995+0000 7f440aa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44040cecd0 0x7f44040cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:26.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.995+0000 7f440aa00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f440415ecd0 0x7f440415f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:26.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.995+0000 7f440aa00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4404153960 con 0x7f43e80039e0 2024-04-04T06:12:26.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.995+0000 7f440aa00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f440415d4e0 con 0x7f44040cecd0 2024-04-04T06:12:26.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.995+0000 7f440aa00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4404165450 con 0x7f440415ecd0 2024-04-04T06:12:26.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.995+0000 7f4403e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43e80039e0 0x7f440409d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:26.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.995+0000 7f4403e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43e80039e0 0x7f440409d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33994/0 (socket says 172.21.15.159:33994) 2024-04-04T06:12:26.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.995+0000 7f4403e00640 1 -- 172.21.15.159:0/3876654132 learned_addr learned my addr 172.21.15.159:0/3876654132 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:26.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.995+0000 7f4408a00640 1 --2- 172.21.15.159:0/3876654132 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f440415ecd0 0x7f440415f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:26.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.995+0000 7f4403e00640 1 -- 172.21.15.159:0/3876654132 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f440415ecd0 msgr2=0x7f440415f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:26.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.995+0000 7f4403e00640 1 --2- 172.21.15.159:0/3876654132 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f440415ecd0 0x7f440415f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:26.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.995+0000 7f4403e00640 1 -- 172.21.15.159:0/3876654132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44040cecd0 msgr2=0x7f44040cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:26.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.995+0000 7f4403400640 1 --2- 172.21.15.159:0/3876654132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44040cecd0 0x7f44040cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:26.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.995+0000 7f4403e00640 1 --2- 172.21.15.159:0/3876654132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44040cecd0 0x7f44040cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:26.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.995+0000 7f4403e00640 1 -- 172.21.15.159:0/3876654132 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440409ddc0 con 0x7f43e80039e0 2024-04-04T06:12:26.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.995+0000 7f4408a00640 1 --2- 172.21.15.159:0/3876654132 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f440415ecd0 0x7f440415f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:26.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.995+0000 7f4403400640 1 --2- 172.21.15.159:0/3876654132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44040cecd0 0x7f44040cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:26.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.996+0000 7f4403e00640 1 --2- 172.21.15.159:0/3876654132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43e80039e0 0x7f440409d880 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f43f400c240 tx=0x7f440415d0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:26.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.996+0000 7f43fbe00640 1 -- 172.21.15.159:0/3876654132 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f4045070 con 0x7f43e80039e0 2024-04-04T06:12:26.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.996+0000 7f43fbe00640 1 -- 172.21.15.159:0/3876654132 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43f4004bf0 con 0x7f43e80039e0 2024-04-04T06:12:26.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.996+0000 7f43fbe00640 1 -- 172.21.15.159:0/3876654132 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f4007cf0 con 0x7f43e80039e0 2024-04-04T06:12:26.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.996+0000 7f440aa00640 1 -- 172.21.15.159:0/3876654132 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44040735d0 con 0x7f43e80039e0 2024-04-04T06:12:26.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.996+0000 7f440aa00640 1 -- 172.21.15.159:0/3876654132 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4404073010 con 0x7f43e80039e0 2024-04-04T06:12:26.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.998+0000 7f440aa00640 1 -- 172.21.15.159:0/3876654132 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4404067db0 con 0x7f43e80039e0 2024-04-04T06:12:26.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.998+0000 7f43fbe00640 1 -- 172.21.15.159:0/3876654132 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f43f4007070 con 0x7f43e80039e0 2024-04-04T06:12:26.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.999+0000 7f43fbe00640 1 --2- 172.21.15.159:0/3876654132 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43e406c920 0x7f43e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:26.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:26.999+0000 7f43fbe00640 1 -- 172.21.15.159:0/3876654132 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5029+0+0 (secure 0 0 0) 0x7f43f40ac7c0 con 0x7f43e80039e0 2024-04-04T06:12:26.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.000+0000 7f4403400640 1 --2- 172.21.15.159:0/3876654132 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43e406c920 0x7f43e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:26.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.001+0000 7f4403400640 1 --2- 172.21.15.159:0/3876654132 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43e406c920 0x7f43e406ede0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f44040728d0 tx=0x7f43f0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:26.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.004+0000 7f43fbe00640 1 -- 172.21.15.159:0/3876654132 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4404067db0 con 0x7f43e80039e0 2024-04-04T06:12:27.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.273+0000 7f440aa00640 1 -- 172.21.15.159:0/3876654132 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f44040685c0 con 0x7f43e80039e0 2024-04-04T06:12:27.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.317+0000 7f43fbe00640 1 -- 172.21.15.159:0/3876654132 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v335) ==== 166+0+0 (secure 0 0 0) 0x7f43f407a9b0 con 0x7f43e80039e0 2024-04-04T06:12:27.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.320+0000 7f440aa00640 1 -- 172.21.15.159:0/3876654132 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43e406c920 msgr2=0x7f43e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:27.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.320+0000 7f440aa00640 1 --2- 172.21.15.159:0/3876654132 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43e406c920 0x7f43e406ede0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f44040728d0 tx=0x7f43f0009040 comp rx=0 tx=0).stop 2024-04-04T06:12:27.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.320+0000 7f440aa00640 1 -- 172.21.15.159:0/3876654132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43e80039e0 msgr2=0x7f440409d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:27.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.320+0000 7f440aa00640 1 --2- 172.21.15.159:0/3876654132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43e80039e0 0x7f440409d880 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f43f400c240 tx=0x7f440415d0f0 comp rx=0 tx=0).stop 2024-04-04T06:12:27.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.320+0000 7f440aa00640 1 -- 172.21.15.159:0/3876654132 shutdown_connections 2024-04-04T06:12:27.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.320+0000 7f440aa00640 1 --2- 172.21.15.159:0/3876654132 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f440415ecd0 0x7f440415f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:27.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.320+0000 7f440aa00640 1 --2- 172.21.15.159:0/3876654132 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f43e406c920 0x7f43e406ede0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:27.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.320+0000 7f440aa00640 1 --2- 172.21.15.159:0/3876654132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f44040cecd0 0x7f44040cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:27.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.320+0000 7f440aa00640 1 --2- 172.21.15.159:0/3876654132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43e80039e0 0x7f440409d880 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:27.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.320+0000 7f440aa00640 1 -- 172.21.15.159:0/3876654132 >> 172.21.15.159:0/3876654132 conn(0x7f440406b750 msgr2=0x7f44040d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:27.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.320+0000 7f440aa00640 1 -- 172.21.15.159:0/3876654132 shutdown_connections 2024-04-04T06:12:27.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.320+0000 7f440aa00640 1 -- 172.21.15.159:0/3876654132 wait complete. 2024-04-04T06:12:27.331 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-04T06:12:27.332 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-04T06:12:27.332 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-04T06:12:27.332 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:12:27.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.537+0000 7f4677800640 1 Processor -- start 2024-04-04T06:12:27.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.537+0000 7f4677800640 1 -- start start 2024-04-04T06:12:27.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.538+0000 7f4677800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f467015b310 0x7f46700727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:27.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.538+0000 7f4677800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f46700b4f90 0x7f467015f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:27.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.538+0000 7f4677800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f467015b6f0 con 0x7f46700d0780 2024-04-04T06:12:27.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.538+0000 7f4677800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f4670072e80 con 0x7f46700b4f90 2024-04-04T06:12:27.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.538+0000 7f4677800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4670073160 con 0x7f467015b310 2024-04-04T06:12:27.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.538+0000 7f4667e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f46700b4f90 0x7f467015f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:27.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.539+0000 7f4667e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f46700b4f90 0x7f467015f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34010/0 (socket says 172.21.15.159:34010) 2024-04-04T06:12:27.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.539+0000 7f4667e00640 1 -- 172.21.15.159:0/2895552101 learned_addr learned my addr 172.21.15.159:0/2895552101 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:27.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.539+0000 7f4667e00640 1 -- 172.21.15.159:0/2895552101 >> v2:172.21.15.178:3300/0 conn(0x7f467015b310 msgr2=0x7f46700727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:27.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.539+0000 7f4674e00640 1 --2- 172.21.15.159:0/2895552101 >> v2:172.21.15.178:3300/0 conn(0x7f467015b310 0x7f46700727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:27.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.539+0000 7f4667e00640 1 --2- 172.21.15.159:0/2895552101 >> v2:172.21.15.178:3300/0 conn(0x7f467015b310 0x7f46700727f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:27.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.539+0000 7f4667e00640 1 -- 172.21.15.159:0/2895552101 >> v1:172.21.15.159:6789/0 conn(0x7f46700d0780 legacy=0x7f46700adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:27.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.539+0000 7f4667e00640 1 -- 172.21.15.159:0/2895552101 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467015efe0 con 0x7f46700b4f90 2024-04-04T06:12:27.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.539+0000 7f466fe00640 1 -- 172.21.15.159:0/2895552101 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1338842886 0 0) 0x7f467015b6f0 con 0x7f46700d0780 2024-04-04T06:12:27.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.539+0000 7f4674e00640 1 --2- 172.21.15.159:0/2895552101 >> v2:172.21.15.178:3300/0 conn(0x7f467015b310 0x7f46700727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:27.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.540+0000 7f4667e00640 1 --2- 172.21.15.159:0/2895552101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46700b4f90 0x7f467015f7e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f467006fa40 tx=0x7f465802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7782a9e5e8b79151 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:27.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.540+0000 7f466fe00640 1 -- 172.21.15.159:0/2895552101 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465802e070 con 0x7f46700b4f90 2024-04-04T06:12:27.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.540+0000 7f466fe00640 1 -- 172.21.15.159:0/2895552101 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4658037070 con 0x7f46700b4f90 2024-04-04T06:12:27.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.540+0000 7f466fe00640 1 -- 172.21.15.159:0/2895552101 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658031a00 con 0x7f46700b4f90 2024-04-04T06:12:27.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.541+0000 7f4677800640 1 -- 172.21.15.159:0/2895552101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46700b4f90 msgr2=0x7f467015f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:27.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.541+0000 7f4677800640 1 --2- 172.21.15.159:0/2895552101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46700b4f90 0x7f467015f7e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f467006fa40 tx=0x7f465802b880 comp rx=0 tx=0).stop 2024-04-04T06:12:27.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.541+0000 7f4677800640 1 -- 172.21.15.159:0/2895552101 shutdown_connections 2024-04-04T06:12:27.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.541+0000 7f4677800640 1 --2- 172.21.15.159:0/2895552101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46700b4f90 0x7f467015f7e0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:27.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.541+0000 7f4677800640 1 --2- 172.21.15.159:0/2895552101 >> v2:172.21.15.178:3300/0 conn(0x7f467015b310 0x7f46700727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:27.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.541+0000 7f4677800640 1 -- 172.21.15.159:0/2895552101 >> 172.21.15.159:0/2895552101 conn(0x7f467006b990 msgr2=0x7f467006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:27.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.541+0000 7f4677800640 1 -- 172.21.15.159:0/2895552101 shutdown_connections 2024-04-04T06:12:27.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.541+0000 7f4677800640 1 -- 172.21.15.159:0/2895552101 wait complete. 2024-04-04T06:12:27.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.543+0000 7f4677800640 1 Processor -- start 2024-04-04T06:12:27.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.543+0000 7f4677800640 1 -- start start 2024-04-04T06:12:27.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.544+0000 7f4677800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46700b4f90 0x7f4670155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:27.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.544+0000 7f4677800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46700d0780 0x7f46700aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:27.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.544+0000 7f4677800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f467015b310 0x7f467015da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:27.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.544+0000 7f4677800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4670073440 con 0x7f46700b4f90 2024-04-04T06:12:27.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.544+0000 7f4677800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f46700b53f0 con 0x7f46700d0780 2024-04-04T06:12:27.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.544+0000 7f4677800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f467015fd20 con 0x7f467015b310 2024-04-04T06:12:27.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.544+0000 7f4674e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46700b4f90 0x7f4670155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:27.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.544+0000 7f4674e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46700b4f90 0x7f4670155380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34018/0 (socket says 172.21.15.159:34018) 2024-04-04T06:12:27.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.544+0000 7f4674e00640 1 -- 172.21.15.159:0/3645138208 learned_addr learned my addr 172.21.15.159:0/3645138208 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:27.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.544+0000 7f4674e00640 1 -- 172.21.15.159:0/3645138208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f467015b310 msgr2=0x7f467015da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:27.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.544+0000 7f4674e00640 1 --2- 172.21.15.159:0/3645138208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f467015b310 0x7f467015da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:27.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.544+0000 7f4667e00640 1 --2- 172.21.15.159:0/3645138208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46700d0780 0x7f46700aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:27.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.544+0000 7f4674e00640 1 -- 172.21.15.159:0/3645138208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46700d0780 msgr2=0x7f46700aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:27.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.544+0000 7f4674e00640 1 --2- 172.21.15.159:0/3645138208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46700d0780 0x7f46700aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:27.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.544+0000 7f4674e00640 1 -- 172.21.15.159:0/3645138208 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4670155bd0 con 0x7f46700b4f90 2024-04-04T06:12:27.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.544+0000 7f4667e00640 1 --2- 172.21.15.159:0/3645138208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46700d0780 0x7f46700aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:27.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.545+0000 7f4674e00640 1 --2- 172.21.15.159:0/3645138208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46700b4f90 0x7f4670155380 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f466000c7f0 tx=0x7f466000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:27.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.545+0000 7f466d600640 1 -- 172.21.15.159:0/3645138208 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660015070 con 0x7f46700b4f90 2024-04-04T06:12:27.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.545+0000 7f466d600640 1 -- 172.21.15.159:0/3645138208 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4660007a30 con 0x7f46700b4f90 2024-04-04T06:12:27.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.545+0000 7f466d600640 1 -- 172.21.15.159:0/3645138208 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660007e00 con 0x7f46700b4f90 2024-04-04T06:12:27.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.545+0000 7f4677800640 1 -- 172.21.15.159:0/3645138208 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4670155eb0 con 0x7f46700b4f90 2024-04-04T06:12:27.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.545+0000 7f4677800640 1 -- 172.21.15.159:0/3645138208 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4670095750 con 0x7f46700b4f90 2024-04-04T06:12:27.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.546+0000 7f4677800640 1 -- 172.21.15.159:0/3645138208 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46700685c0 con 0x7f46700b4f90 2024-04-04T06:12:27.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.547+0000 7f466d600640 1 -- 172.21.15.159:0/3645138208 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4660018070 con 0x7f46700b4f90 2024-04-04T06:12:27.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.548+0000 7f466d600640 1 --2- 172.21.15.159:0/3645138208 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f464c06c920 0x7f464c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:27.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.548+0000 7f466d600640 1 -- 172.21.15.159:0/3645138208 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5029+0+0 (secure 0 0 0) 0x7f466008b930 con 0x7f46700b4f90 2024-04-04T06:12:27.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.549+0000 7f4667e00640 1 --2- 172.21.15.159:0/3645138208 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f464c06c920 0x7f464c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:27.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.550+0000 7f4667e00640 1 --2- 172.21.15.159:0/3645138208 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f464c06c920 0x7f464c06ede0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4658036000 tx=0x7f4658008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:27.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.553+0000 7f466d600640 1 -- 172.21.15.159:0/3645138208 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4660059f60 con 0x7f46700b4f90 2024-04-04T06:12:27.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.756+0000 7f4677800640 1 -- 172.21.15.159:0/3645138208 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f467015ec20 con 0x7f46700b4f90 2024-04-04T06:12:27.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.758+0000 7f466d600640 1 -- 172.21.15.159:0/3645138208 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+13167 (secure 0 0 0) 0x7f4660059860 con 0x7f46700b4f90 2024-04-04T06:12:27.751 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:27.751 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":77,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:12:26.304617+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6835/2467594024":"2024-04-05T06:12:23.127544+0000","172.21.15.159:6834/2467594024":"2024-04-05T06:12:23.127544+0000","172.21.15.159:6835/1652468640":"2024-04-05T06:10:57.934622+0000","172.21.15.159:6834/1652468640":"2024-04-05T06:10:57.934622+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:12:27.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.761+0000 7f4677800640 1 -- 172.21.15.159:0/3645138208 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f464c06c920 msgr2=0x7f464c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:27.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.761+0000 7f4677800640 1 --2- 172.21.15.159:0/3645138208 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f464c06c920 0x7f464c06ede0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4658036000 tx=0x7f4658008030 comp rx=0 tx=0).stop 2024-04-04T06:12:27.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.761+0000 7f4677800640 1 -- 172.21.15.159:0/3645138208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46700b4f90 msgr2=0x7f4670155380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:27.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.761+0000 7f4677800640 1 --2- 172.21.15.159:0/3645138208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46700b4f90 0x7f4670155380 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f466000c7f0 tx=0x7f466000cbf0 comp rx=0 tx=0).stop 2024-04-04T06:12:27.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.761+0000 7f4677800640 1 -- 172.21.15.159:0/3645138208 shutdown_connections 2024-04-04T06:12:27.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.761+0000 7f4677800640 1 --2- 172.21.15.159:0/3645138208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f467015b310 0x7f467015da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:27.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.761+0000 7f4677800640 1 --2- 172.21.15.159:0/3645138208 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f464c06c920 0x7f464c06ede0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:27.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.761+0000 7f4677800640 1 --2- 172.21.15.159:0/3645138208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46700d0780 0x7f46700aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:27.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.761+0000 7f4677800640 1 --2- 172.21.15.159:0/3645138208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46700b4f90 0x7f4670155380 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:27.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.761+0000 7f4677800640 1 -- 172.21.15.159:0/3645138208 >> 172.21.15.159:0/3645138208 conn(0x7f467006b990 msgr2=0x7f46700b1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:27.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.761+0000 7f4677800640 1 -- 172.21.15.159:0/3645138208 shutdown_connections 2024-04-04T06:12:27.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.761+0000 7f4677800640 1 -- 172.21.15.159:0/3645138208 wait complete. 2024-04-04T06:12:27.774 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:12:27.774 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:12:27.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.993+0000 7fe9b0a00640 1 Processor -- start 2024-04-04T06:12:27.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.993+0000 7fe9b0a00640 1 -- start start 2024-04-04T06:12:27.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.995+0000 7fe9b0a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe9ac155550 0x7fe9ac155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:27.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.995+0000 7fe9b0a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9ac0d0990 0x7fe9ac0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:27.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.995+0000 7fe9b0a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9ac0b40b0 con 0x7fe9ac15f800 2024-04-04T06:12:27.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.995+0000 7fe9b0a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe9ac0b4390 con 0x7fe9ac155550 2024-04-04T06:12:27.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.995+0000 7fe9b0a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ac0b4670 con 0x7fe9ac0d0990 2024-04-04T06:12:27.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.995+0000 7fe9a9e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe9ac155550 0x7fe9ac155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:27.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.995+0000 7fe9a9400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fe9ac15f800 0x7fe9ac0d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:59500/0 (socket says 172.21.15.159:59500) 2024-04-04T06:12:27.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.995+0000 7fe9a9e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe9ac155550 0x7fe9ac155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34028/0 (socket says 172.21.15.159:34028) 2024-04-04T06:12:27.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.995+0000 7fe9a9400640 1 -- 172.21.15.159:0/2853296249 learned_addr learned my addr 172.21.15.159:0/2853296249 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:27.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.995+0000 7fe9aa800640 1 --2- 172.21.15.159:0/2853296249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9ac0d0990 0x7fe9ac0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:27.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.996+0000 7fe9a9e00640 1 -- 172.21.15.159:0/2853296249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9ac0d0990 msgr2=0x7fe9ac0aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:27.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.996+0000 7fe9a9e00640 1 --2- 172.21.15.159:0/2853296249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9ac0d0990 0x7fe9ac0aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:27.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.996+0000 7fe9a9e00640 1 -- 172.21.15.159:0/2853296249 >> v1:172.21.15.159:6789/0 conn(0x7fe9ac15f800 legacy=0x7fe9ac0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:27.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.996+0000 7fe9a9e00640 1 -- 172.21.15.159:0/2853296249 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ac0b4950 con 0x7fe9ac155550 2024-04-04T06:12:27.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.996+0000 7fe9a8a00640 1 -- 172.21.15.159:0/2853296249 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2221618027 0 0) 0x7fe9ac0b40b0 con 0x7fe9ac15f800 2024-04-04T06:12:27.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.997+0000 7fe9a9e00640 1 --2- 172.21.15.159:0/2853296249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ac155550 0x7fe9ac155930 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fe99c0097a0 tx=0x7fe99c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2b3974c2e6dff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:27.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.997+0000 7fe9a8a00640 1 -- 172.21.15.159:0/2853296249 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c035020 con 0x7fe9ac155550 2024-04-04T06:12:27.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.998+0000 7fe9a8a00640 1 -- 172.21.15.159:0/2853296249 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe99c031830 con 0x7fe9ac155550 2024-04-04T06:12:27.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.998+0000 7fe9a8a00640 1 -- 172.21.15.159:0/2853296249 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c031c00 con 0x7fe9ac155550 2024-04-04T06:12:27.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.998+0000 7fe9b0a00640 1 -- 172.21.15.159:0/2853296249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ac155550 msgr2=0x7fe9ac155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:27.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.998+0000 7fe9b0a00640 1 --2- 172.21.15.159:0/2853296249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ac155550 0x7fe9ac155930 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fe99c0097a0 tx=0x7fe99c02b8d0 comp rx=0 tx=0).stop 2024-04-04T06:12:27.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.998+0000 7fe9b0a00640 1 -- 172.21.15.159:0/2853296249 shutdown_connections 2024-04-04T06:12:27.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.998+0000 7fe9b0a00640 1 --2- 172.21.15.159:0/2853296249 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9ac0d0990 0x7fe9ac0aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:27.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.998+0000 7fe9b0a00640 1 --2- 172.21.15.159:0/2853296249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ac155550 0x7fe9ac155930 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:27.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.998+0000 7fe9b0a00640 1 -- 172.21.15.159:0/2853296249 >> 172.21.15.159:0/2853296249 conn(0x7fe9ac06b750 msgr2=0x7fe9ac06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:27.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.998+0000 7fe9b0a00640 1 -- 172.21.15.159:0/2853296249 shutdown_connections 2024-04-04T06:12:27.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:27.998+0000 7fe9b0a00640 1 -- 172.21.15.159:0/2853296249 wait complete. 2024-04-04T06:12:27.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.000+0000 7fe9b0a00640 1 Processor -- start 2024-04-04T06:12:27.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.000+0000 7fe9b0a00640 1 -- start start 2024-04-04T06:12:27.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.001+0000 7fe9b0a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ac0d0990 0x7fe9ac14e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:27.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.001+0000 7fe9b0a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9ac155550 0x7fe9ac150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:27.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.001+0000 7fe9b0a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9ac15f800 0x7fe9ac152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:27.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.001+0000 7fe9b0a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac15ba20 con 0x7fe9ac0d0990 2024-04-04T06:12:27.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.001+0000 7fe9b0a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac15fc80 con 0x7fe9ac155550 2024-04-04T06:12:27.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.001+0000 7fe9b0a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ac15bba0 con 0x7fe9ac15f800 2024-04-04T06:12:27.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.001+0000 7fe9a9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ac0d0990 0x7fe9ac14e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:27.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.002+0000 7fe9a9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ac0d0990 0x7fe9ac14e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34032/0 (socket says 172.21.15.159:34032) 2024-04-04T06:12:27.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.002+0000 7fe9a9e00640 1 -- 172.21.15.159:0/2693561902 learned_addr learned my addr 172.21.15.159:0/2693561902 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:27.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.002+0000 7fe9aa800640 1 --2- 172.21.15.159:0/2693561902 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9ac15f800 0x7fe9ac152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:27.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.002+0000 7fe9a9400640 1 --2- 172.21.15.159:0/2693561902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9ac155550 0x7fe9ac150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:27.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.002+0000 7fe9a9e00640 1 -- 172.21.15.159:0/2693561902 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9ac15f800 msgr2=0x7fe9ac152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:27.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.002+0000 7fe9a9e00640 1 --2- 172.21.15.159:0/2693561902 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9ac15f800 0x7fe9ac152900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:27.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.002+0000 7fe9a9e00640 1 -- 172.21.15.159:0/2693561902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9ac155550 msgr2=0x7fe9ac150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:27.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.002+0000 7fe9a9e00640 1 --2- 172.21.15.159:0/2693561902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9ac155550 0x7fe9ac150f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:27.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.002+0000 7fe9a9e00640 1 -- 172.21.15.159:0/2693561902 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ac0cd210 con 0x7fe9ac0d0990 2024-04-04T06:12:27.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.002+0000 7fe9a9400640 1 --2- 172.21.15.159:0/2693561902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9ac155550 0x7fe9ac150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:27.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.002+0000 7fe9a9e00640 1 --2- 172.21.15.159:0/2693561902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ac0d0990 0x7fe9ac14e330 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fe99c000c00 tx=0x7fe99c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:27.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.003+0000 7fe9a2000640 1 -- 172.21.15.159:0/2693561902 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c035070 con 0x7fe9ac0d0990 2024-04-04T06:12:27.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.003+0000 7fe9a2000640 1 -- 172.21.15.159:0/2693561902 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe99c004230 con 0x7fe9ac0d0990 2024-04-04T06:12:27.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.003+0000 7fe9b0a00640 1 -- 172.21.15.159:0/2693561902 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ac0cd4f0 con 0x7fe9ac0d0990 2024-04-04T06:12:27.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.003+0000 7fe9a2000640 1 -- 172.21.15.159:0/2693561902 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c03a6f0 con 0x7fe9ac0d0990 2024-04-04T06:12:27.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.003+0000 7fe9b0a00640 1 -- 172.21.15.159:0/2693561902 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ac098780 con 0x7fe9ac0d0990 2024-04-04T06:12:27.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.004+0000 7fe9b0a00640 1 -- 172.21.15.159:0/2693561902 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ac06f790 con 0x7fe9ac0d0990 2024-04-04T06:12:27.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.005+0000 7fe9a2000640 1 -- 172.21.15.159:0/2693561902 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe99c038020 con 0x7fe9ac0d0990 2024-04-04T06:12:27.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.006+0000 7fe9a2000640 1 --2- 172.21.15.159:0/2693561902 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe98c06c8f0 0x7fe98c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:27.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.006+0000 7fe9a2000640 1 -- 172.21.15.159:0/2693561902 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5029+0+0 (secure 0 0 0) 0x7fe99c0ad680 con 0x7fe9ac0d0990 2024-04-04T06:12:27.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.006+0000 7fe9a9400640 1 --2- 172.21.15.159:0/2693561902 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe98c06c8f0 0x7fe98c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:28.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.007+0000 7fe9a9400640 1 --2- 172.21.15.159:0/2693561902 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe98c06c8f0 0x7fe98c06edb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0cf9b0 tx=0x7fe998009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:28.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.010+0000 7fe9a2000640 1 -- 172.21.15.159:0/2693561902 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe9ac06f790 con 0x7fe9ac0d0990 2024-04-04T06:12:28.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.202+0000 7fe9b0a00640 1 -- 172.21.15.159:0/2693561902 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe9ac0685c0 con 0x7fe9ac0d0990 2024-04-04T06:12:28.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.202+0000 7fe9a2000640 1 -- 172.21.15.159:0/2693561902 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7fe99c040070 con 0x7fe9ac0d0990 2024-04-04T06:12:28.195 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:12:28.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.205+0000 7fe9b0a00640 1 -- 172.21.15.159:0/2693561902 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe98c06c8f0 msgr2=0x7fe98c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:28.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.205+0000 7fe9b0a00640 1 --2- 172.21.15.159:0/2693561902 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe98c06c8f0 0x7fe98c06edb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0cf9b0 tx=0x7fe998009040 comp rx=0 tx=0).stop 2024-04-04T06:12:28.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.205+0000 7fe9b0a00640 1 -- 172.21.15.159:0/2693561902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ac0d0990 msgr2=0x7fe9ac14e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:28.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.205+0000 7fe9b0a00640 1 --2- 172.21.15.159:0/2693561902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ac0d0990 0x7fe9ac14e330 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fe99c000c00 tx=0x7fe99c0046c0 comp rx=0 tx=0).stop 2024-04-04T06:12:28.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.205+0000 7fe9b0a00640 1 -- 172.21.15.159:0/2693561902 shutdown_connections 2024-04-04T06:12:28.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.205+0000 7fe9b0a00640 1 --2- 172.21.15.159:0/2693561902 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9ac15f800 0x7fe9ac152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:28.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.205+0000 7fe9b0a00640 1 --2- 172.21.15.159:0/2693561902 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe98c06c8f0 0x7fe98c06edb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:28.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.205+0000 7fe9b0a00640 1 --2- 172.21.15.159:0/2693561902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9ac155550 0x7fe9ac150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:28.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.205+0000 7fe9b0a00640 1 --2- 172.21.15.159:0/2693561902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9ac0d0990 0x7fe9ac14e330 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:28.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.205+0000 7fe9b0a00640 1 -- 172.21.15.159:0/2693561902 >> 172.21.15.159:0/2693561902 conn(0x7fe9ac06b750 msgr2=0x7fe9ac15ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:28.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.205+0000 7fe9b0a00640 1 -- 172.21.15.159:0/2693561902 shutdown_connections 2024-04-04T06:12:28.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.205+0000 7fe9b0a00640 1 -- 172.21.15.159:0/2693561902 wait complete. 2024-04-04T06:12:28.217 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-04T06:12:28.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.427+0000 7f4c2ac00640 1 Processor -- start 2024-04-04T06:12:28.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.427+0000 7f4c2ac00640 1 -- start start 2024-04-04T06:12:28.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.429+0000 7f4c2ac00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4c240b4650 0x7f4c2406c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:28.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.429+0000 7f4c2ac00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c2415ede0 con 0x7f4c240b4a30 2024-04-04T06:12:28.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.429+0000 7f4c2ac00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c24141d80 con 0x7f4c24158580 2024-04-04T06:12:28.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.429+0000 7f4c2ac00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f4c240b6320 con 0x7f4c240b4650 2024-04-04T06:12:28.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.429+0000 7f4c23e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4c240b4650 0x7f4c2406c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:28.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.429+0000 7f4c23e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4c240b4650 0x7f4c2406c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34048/0 (socket says 172.21.15.159:34048) 2024-04-04T06:12:28.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.429+0000 7f4c23e00640 1 -- 172.21.15.159:0/4002562946 learned_addr learned my addr 172.21.15.159:0/4002562946 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:28.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.430+0000 7f4c23e00640 1 -- 172.21.15.159:0/4002562946 >> v1:172.21.15.178:6789/0 conn(0x7f4c24158580 legacy=0x7f4c241589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:28.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.430+0000 7f4c23e00640 1 -- 172.21.15.159:0/4002562946 >> v1:172.21.15.159:6789/0 conn(0x7f4c240b4a30 legacy=0x7f4c240b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:28.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.430+0000 7f4c23e00640 1 -- 172.21.15.159:0/4002562946 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c24142060 con 0x7f4c240b4650 2024-04-04T06:12:28.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.431+0000 7f4c23e00640 1 --2- 172.21.15.159:0/4002562946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c240b4650 0x7f4c2406c680 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009870 tx=0x7f4c1402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb793ba7dabf22d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:28.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.431+0000 7f4c23400640 1 -- 172.21.15.159:0/4002562946 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14034070 con 0x7f4c240b4650 2024-04-04T06:12:28.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.431+0000 7f4c23400640 1 -- 172.21.15.159:0/4002562946 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c14030860 con 0x7f4c240b4650 2024-04-04T06:12:28.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.431+0000 7f4c23400640 1 -- 172.21.15.159:0/4002562946 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14030c30 con 0x7f4c240b4650 2024-04-04T06:12:28.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.432+0000 7f4c2ac00640 1 -- 172.21.15.159:0/4002562946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c240b4650 msgr2=0x7f4c2406c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:28.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.432+0000 7f4c2ac00640 1 --2- 172.21.15.159:0/4002562946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c240b4650 0x7f4c2406c680 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009870 tx=0x7f4c1402b9a0 comp rx=0 tx=0).stop 2024-04-04T06:12:28.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.432+0000 7f4c2ac00640 1 -- 172.21.15.159:0/4002562946 shutdown_connections 2024-04-04T06:12:28.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.432+0000 7f4c2ac00640 1 --2- 172.21.15.159:0/4002562946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c240b4650 0x7f4c2406c680 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:28.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.432+0000 7f4c2ac00640 1 -- 172.21.15.159:0/4002562946 >> 172.21.15.159:0/4002562946 conn(0x7f4c24067670 msgr2=0x7f4c24067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:28.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.432+0000 7f4c2ac00640 1 -- 172.21.15.159:0/4002562946 shutdown_connections 2024-04-04T06:12:28.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.432+0000 7f4c2ac00640 1 -- 172.21.15.159:0/4002562946 wait complete. 2024-04-04T06:12:28.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.434+0000 7f4c2ac00640 1 Processor -- start 2024-04-04T06:12:28.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.434+0000 7f4c2ac00640 1 -- start start 2024-04-04T06:12:28.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.435+0000 7f4c2ac00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c240b4650 0x7f4c240cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:28.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.435+0000 7f4c2ac00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c240b4a30 0x7f4c240cde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:28.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.435+0000 7f4c2ac00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c24158580 0x7f4c2415ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:28.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.435+0000 7f4c2ac00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24142620 con 0x7f4c24158580 2024-04-04T06:12:28.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.435+0000 7f4c2ac00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24141b30 con 0x7f4c240b4a30 2024-04-04T06:12:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.435+0000 7f4c2ac00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4c24142900 con 0x7f4c240b4650 2024-04-04T06:12:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.435+0000 7f4c28c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c24158580 0x7f4c2415ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.435+0000 7f4c28c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c24158580 0x7f4c2415ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34058/0 (socket says 172.21.15.159:34058) 2024-04-04T06:12:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.435+0000 7f4c28c00640 1 -- 172.21.15.159:0/2901180647 learned_addr learned my addr 172.21.15.159:0/2901180647 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.436+0000 7f4c1b400640 1 --2- 172.21.15.159:0/2901180647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c240b4a30 0x7f4c240cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.436+0000 7f4c23e00640 1 --2- 172.21.15.159:0/2901180647 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c240b4650 0x7f4c240cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.436+0000 7f4c28c00640 1 -- 172.21.15.159:0/2901180647 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c240b4650 msgr2=0x7f4c240cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.436+0000 7f4c28c00640 1 --2- 172.21.15.159:0/2901180647 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c240b4650 0x7f4c240cca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.436+0000 7f4c28c00640 1 -- 172.21.15.159:0/2901180647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c240b4a30 msgr2=0x7f4c240cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.436+0000 7f4c28c00640 1 --2- 172.21.15.159:0/2901180647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c240b4a30 0x7f4c240cde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.436+0000 7f4c28c00640 1 -- 172.21.15.159:0/2901180647 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c240cb960 con 0x7f4c24158580 2024-04-04T06:12:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.436+0000 7f4c1b400640 1 --2- 172.21.15.159:0/2901180647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c240b4a30 0x7f4c240cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.436+0000 7f4c28c00640 1 --2- 172.21.15.159:0/2901180647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c24158580 0x7f4c2415ac20 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00b970 tx=0x7f4c1c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:28.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.436+0000 7f4c20c00640 1 -- 172.21.15.159:0/2901180647 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c019070 con 0x7f4c24158580 2024-04-04T06:12:28.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.436+0000 7f4c20c00640 1 -- 172.21.15.159:0/2901180647 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c1c00cb40 con 0x7f4c24158580 2024-04-04T06:12:28.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.436+0000 7f4c20c00640 1 -- 172.21.15.159:0/2901180647 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c014440 con 0x7f4c24158580 2024-04-04T06:12:28.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.436+0000 7f4c2ac00640 1 -- 172.21.15.159:0/2901180647 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c240cbc40 con 0x7f4c24158580 2024-04-04T06:12:28.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.437+0000 7f4c2ac00640 1 -- 172.21.15.159:0/2901180647 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c240cc200 con 0x7f4c24158580 2024-04-04T06:12:28.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.438+0000 7f4c2ac00640 1 -- 172.21.15.159:0/2901180647 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c24074f30 con 0x7f4c24158580 2024-04-04T06:12:28.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.439+0000 7f4c20c00640 1 -- 172.21.15.159:0/2901180647 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c1c01c070 con 0x7f4c24158580 2024-04-04T06:12:28.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.439+0000 7f4c20c00640 1 --2- 172.21.15.159:0/2901180647 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4c0006c8f0 0x7f4c0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:28.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.440+0000 7f4c20c00640 1 -- 172.21.15.159:0/2901180647 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5029+0+0 (secure 0 0 0) 0x7f4c1c08b520 con 0x7f4c24158580 2024-04-04T06:12:28.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.440+0000 7f4c23e00640 1 --2- 172.21.15.159:0/2901180647 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4c0006c8f0 0x7f4c0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:28.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.441+0000 7f4c23e00640 1 --2- 172.21.15.159:0/2901180647 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4c0006c8f0 0x7f4c0006edb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4c14036000 tx=0x7f4c1403b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:28.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.444+0000 7f4c20c00640 1 -- 172.21.15.159:0/2901180647 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4c1c059be0 con 0x7f4c24158580 2024-04-04T06:12:28.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:28.717+0000 7f4c2ac00640 1 -- 172.21.15.159:0/2901180647 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f4c24161070 con 0x7f4c24158580 2024-04-04T06:12:29.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.330+0000 7f4c20c00640 1 -- 172.21.15.159:0/2901180647 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v337) ==== 170+0+0 (secure 0 0 0) 0x7f4c1c0594e0 con 0x7f4c24158580 2024-04-04T06:12:29.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.333+0000 7f4c2ac00640 1 -- 172.21.15.159:0/2901180647 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4c0006c8f0 msgr2=0x7f4c0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:29.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.333+0000 7f4c2ac00640 1 --2- 172.21.15.159:0/2901180647 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4c0006c8f0 0x7f4c0006edb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4c14036000 tx=0x7f4c1403b040 comp rx=0 tx=0).stop 2024-04-04T06:12:29.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.333+0000 7f4c2ac00640 1 -- 172.21.15.159:0/2901180647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c24158580 msgr2=0x7f4c2415ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:29.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.333+0000 7f4c2ac00640 1 --2- 172.21.15.159:0/2901180647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c24158580 0x7f4c2415ac20 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00b970 tx=0x7f4c1c00be30 comp rx=0 tx=0).stop 2024-04-04T06:12:29.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.334+0000 7f4c2ac00640 1 -- 172.21.15.159:0/2901180647 shutdown_connections 2024-04-04T06:12:29.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.334+0000 7f4c2ac00640 1 --2- 172.21.15.159:0/2901180647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c24158580 0x7f4c2415ac20 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:29.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.334+0000 7f4c2ac00640 1 --2- 172.21.15.159:0/2901180647 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4c0006c8f0 0x7f4c0006edb0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:29.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.334+0000 7f4c2ac00640 1 --2- 172.21.15.159:0/2901180647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c240b4a30 0x7f4c240cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:29.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.334+0000 7f4c2ac00640 1 --2- 172.21.15.159:0/2901180647 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c240b4650 0x7f4c240cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:29.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.334+0000 7f4c2ac00640 1 -- 172.21.15.159:0/2901180647 >> 172.21.15.159:0/2901180647 conn(0x7f4c24067670 msgr2=0x7f4c24161460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:29.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.334+0000 7f4c2ac00640 1 -- 172.21.15.159:0/2901180647 shutdown_connections 2024-04-04T06:12:29.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.334+0000 7f4c2ac00640 1 -- 172.21.15.159:0/2901180647 wait complete. 2024-04-04T06:12:29.348 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-04T06:12:29.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.568+0000 7f919fa00640 1 Processor -- start 2024-04-04T06:12:29.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.569+0000 7f919fa00640 1 -- start start 2024-04-04T06:12:29.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.570+0000 7f919fa00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f91980d0e70 0x7f91980cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:29.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.570+0000 7f919fa00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f919815f6e0 0x7f91980d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:29.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.570+0000 7f919fa00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91980d1790 con 0x7f91981621d0 2024-04-04T06:12:29.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.570+0000 7f919fa00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f91980cf6c0 con 0x7f91980d0e70 2024-04-04T06:12:29.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.570+0000 7f919fa00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f91980cf9a0 con 0x7f919815f6e0 2024-04-04T06:12:29.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.571+0000 7f919d000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f91980d0e70 0x7f91980cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:29.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.571+0000 7f919da00640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f91981621d0 0x7f9198162660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.159:39000/0 (socket says 172.21.15.159:39000) 2024-04-04T06:12:29.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.571+0000 7f919da00640 1 -- 172.21.15.159:0/3190979170 learned_addr learned my addr 172.21.15.159:0/3190979170 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:29.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.571+0000 7f919d000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f91980d0e70 0x7f91980cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:33064/0 (socket says 172.21.15.159:33064) 2024-04-04T06:12:29.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.571+0000 7f9197e00640 1 --2- 172.21.15.159:0/3190979170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f919815f6e0 0x7f91980d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:29.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.572+0000 7f919d000640 1 -- 172.21.15.159:0/3190979170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f919815f6e0 msgr2=0x7f91980d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:29.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.572+0000 7f919d000640 1 --2- 172.21.15.159:0/3190979170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f919815f6e0 0x7f91980d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:29.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.572+0000 7f919d000640 1 -- 172.21.15.159:0/3190979170 >> v1:172.21.15.178:6789/0 conn(0x7f91981621d0 legacy=0x7f9198162660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:29.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.572+0000 7f919d000640 1 -- 172.21.15.159:0/3190979170 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91980cfc80 con 0x7f91980d0e70 2024-04-04T06:12:29.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.573+0000 7f919d000640 1 --2- 172.21.15.159:0/3190979170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91980d0e70 0x7f91980cf050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f91880097a0 tx=0x7f918802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83abee440aa1fe43 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:29.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.574+0000 7f9197400640 1 -- 172.21.15.159:0/3190979170 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188031780 con 0x7f91980d0e70 2024-04-04T06:12:29.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.574+0000 7f9197400640 1 -- 172.21.15.159:0/3190979170 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9188037070 con 0x7f91980d0e70 2024-04-04T06:12:29.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.574+0000 7f9197400640 1 -- 172.21.15.159:0/3190979170 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188031b40 con 0x7f91980d0e70 2024-04-04T06:12:29.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.574+0000 7f919fa00640 1 -- 172.21.15.159:0/3190979170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91980d0e70 msgr2=0x7f91980cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:29.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.574+0000 7f919fa00640 1 --2- 172.21.15.159:0/3190979170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91980d0e70 0x7f91980cf050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f91880097a0 tx=0x7f918802b820 comp rx=0 tx=0).stop 2024-04-04T06:12:29.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.574+0000 7f919fa00640 1 -- 172.21.15.159:0/3190979170 shutdown_connections 2024-04-04T06:12:29.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.574+0000 7f919fa00640 1 --2- 172.21.15.159:0/3190979170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f919815f6e0 0x7f91980d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:29.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.574+0000 7f919fa00640 1 --2- 172.21.15.159:0/3190979170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91980d0e70 0x7f91980cf050 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:29.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.574+0000 7f919fa00640 1 -- 172.21.15.159:0/3190979170 >> 172.21.15.159:0/3190979170 conn(0x7f919808bfb0 msgr2=0x7f91980d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:29.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.574+0000 7f919fa00640 1 -- 172.21.15.159:0/3190979170 shutdown_connections 2024-04-04T06:12:29.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.574+0000 7f919fa00640 1 -- 172.21.15.159:0/3190979170 wait complete. 2024-04-04T06:12:29.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.576+0000 7f919fa00640 1 Processor -- start 2024-04-04T06:12:29.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.576+0000 7f919fa00640 1 -- start start 2024-04-04T06:12:29.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.577+0000 7f919fa00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91980d0e70 0x7f9198159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:29.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.577+0000 7f919fa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f919815f6e0 0x7f91980c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:29.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.577+0000 7f919fa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91981621d0 0x7f91980c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:29.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.577+0000 7f919fa00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f919815fb40 con 0x7f91981621d0 2024-04-04T06:12:29.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.577+0000 7f919fa00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f91980d0240 con 0x7f919815f6e0 2024-04-04T06:12:29.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.577+0000 7f919fa00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f919815e910 con 0x7f91980d0e70 2024-04-04T06:12:29.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.577+0000 7f919da00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91981621d0 0x7f91980c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:29.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.577+0000 7f919da00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91981621d0 0x7f91980c84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34068/0 (socket says 172.21.15.159:34068) 2024-04-04T06:12:29.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.577+0000 7f919da00640 1 -- 172.21.15.159:0/3384976368 learned_addr learned my addr 172.21.15.159:0/3384976368 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:29.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.578+0000 7f9197e00640 1 --2- 172.21.15.159:0/3384976368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f919815f6e0 0x7f91980c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:29.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.578+0000 7f919d000640 1 --2- 172.21.15.159:0/3384976368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91980d0e70 0x7f9198159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:29.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.578+0000 7f919da00640 1 -- 172.21.15.159:0/3384976368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91980d0e70 msgr2=0x7f9198159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:29.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.578+0000 7f919da00640 1 --2- 172.21.15.159:0/3384976368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91980d0e70 0x7f9198159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:29.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.578+0000 7f919da00640 1 -- 172.21.15.159:0/3384976368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f919815f6e0 msgr2=0x7f91980c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:29.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.578+0000 7f919da00640 1 --2- 172.21.15.159:0/3384976368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f919815f6e0 0x7f91980c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:29.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.578+0000 7f919da00640 1 -- 172.21.15.159:0/3384976368 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9198159990 con 0x7f91981621d0 2024-04-04T06:12:29.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.578+0000 7f919da00640 1 --2- 172.21.15.159:0/3384976368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91981621d0 0x7f91980c84b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f919000c370 tx=0x7f919000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:29.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.578+0000 7f9194c00640 1 -- 172.21.15.159:0/3384976368 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9190017070 con 0x7f91981621d0 2024-04-04T06:12:29.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.578+0000 7f9194c00640 1 -- 172.21.15.159:0/3384976368 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9190005ce0 con 0x7f91981621d0 2024-04-04T06:12:29.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.578+0000 7f919fa00640 1 -- 172.21.15.159:0/3384976368 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91980cd390 con 0x7f91981621d0 2024-04-04T06:12:29.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.578+0000 7f9194c00640 1 -- 172.21.15.159:0/3384976368 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91900072d0 con 0x7f91981621d0 2024-04-04T06:12:29.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.579+0000 7f919fa00640 1 -- 172.21.15.159:0/3384976368 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91980cd910 con 0x7f91981621d0 2024-04-04T06:12:29.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.580+0000 7f919fa00640 1 -- 172.21.15.159:0/3384976368 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9198068cd0 con 0x7f91981621d0 2024-04-04T06:12:29.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.580+0000 7f9194c00640 1 -- 172.21.15.159:0/3384976368 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f919001a070 con 0x7f91981621d0 2024-04-04T06:12:29.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.581+0000 7f9194c00640 1 --2- 172.21.15.159:0/3384976368 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f917406c8f0 0x7f917406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:29.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.581+0000 7f9194c00640 1 -- 172.21.15.159:0/3384976368 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5029+0+0 (secure 0 0 0) 0x7f919008ab30 con 0x7f91981621d0 2024-04-04T06:12:29.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.582+0000 7f919d000640 1 --2- 172.21.15.159:0/3384976368 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f917406c8f0 0x7f917406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:29.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.583+0000 7f919d000640 1 --2- 172.21.15.159:0/3384976368 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f917406c8f0 0x7f917406edb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9188036000 tx=0x7f918803d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:29.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.585+0000 7f9194c00640 1 -- 172.21.15.159:0/3384976368 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9198159990 con 0x7f91981621d0 2024-04-04T06:12:29.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.786+0000 7f919fa00640 1 -- 172.21.15.159:0/3384976368 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f91980d1b80 con 0x7f91981621d0 2024-04-04T06:12:29.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.792+0000 7f9194c00640 1 -- 172.21.15.159:0/3384976368 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f9190058b90 con 0x7f91981621d0 2024-04-04T06:12:29.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.795+0000 7f919fa00640 1 -- 172.21.15.159:0/3384976368 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f917406c8f0 msgr2=0x7f917406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:29.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.795+0000 7f919fa00640 1 --2- 172.21.15.159:0/3384976368 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f917406c8f0 0x7f917406edb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9188036000 tx=0x7f918803d040 comp rx=0 tx=0).stop 2024-04-04T06:12:29.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.795+0000 7f919fa00640 1 -- 172.21.15.159:0/3384976368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91981621d0 msgr2=0x7f91980c84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:29.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.795+0000 7f919fa00640 1 --2- 172.21.15.159:0/3384976368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91981621d0 0x7f91980c84b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f919000c370 tx=0x7f919000c830 comp rx=0 tx=0).stop 2024-04-04T06:12:29.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.795+0000 7f919fa00640 1 -- 172.21.15.159:0/3384976368 shutdown_connections 2024-04-04T06:12:29.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.795+0000 7f919fa00640 1 --2- 172.21.15.159:0/3384976368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91981621d0 0x7f91980c84b0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:29.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.795+0000 7f919fa00640 1 --2- 172.21.15.159:0/3384976368 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f917406c8f0 0x7f917406edb0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:29.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.795+0000 7f919fa00640 1 --2- 172.21.15.159:0/3384976368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f919815f6e0 0x7f91980c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:29.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.795+0000 7f919fa00640 1 --2- 172.21.15.159:0/3384976368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91980d0e70 0x7f9198159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:29.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.795+0000 7f919fa00640 1 -- 172.21.15.159:0/3384976368 >> 172.21.15.159:0/3384976368 conn(0x7f919808bfb0 msgr2=0x7f919806cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:29.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.795+0000 7f919fa00640 1 -- 172.21.15.159:0/3384976368 shutdown_connections 2024-04-04T06:12:29.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:29.795+0000 7f919fa00640 1 -- 172.21.15.159:0/3384976368 wait complete. 2024-04-04T06:12:29.807 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-04T06:12:29.807 DEBUG:teuthology.orchestra.run.smithi159:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:12:29.917 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:12:30.036 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-04T06:12:30.036 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:12:30.036 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:12:30.093 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:12:30.093 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:12:30.093 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:12:30.148 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:12:30.149 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:12:30.149 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:12:30.210 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:12:30.210 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:12:30.210 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:12:30.265 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:12:30.265 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-04T06:12:30.266 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:12:30.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.492+0000 7ff001200640 1 Processor -- start 2024-04-04T06:12:30.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.492+0000 7ff001200640 1 -- start start 2024-04-04T06:12:30.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.493+0000 7ff001200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7feffc15c390 0x7feffc073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:30.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.493+0000 7ff001200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feffc0d0650 0x7feffc0d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:30.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.493+0000 7ff001200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feffc15fe80 con 0x7feffc0af830 2024-04-04T06:12:30.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.493+0000 7ff001200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7feffc15c770 con 0x7feffc15c390 2024-04-04T06:12:30.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.493+0000 7ff001200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feffc0ad120 con 0x7feffc0d0650 2024-04-04T06:12:30.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.493+0000 7feff9e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7feffc15c390 0x7feffc073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:30.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.494+0000 7feff9e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7feffc15c390 0x7feffc073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34082/0 (socket says 172.21.15.159:34082) 2024-04-04T06:12:30.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.494+0000 7feff9e00640 1 -- 172.21.15.159:0/404919443 learned_addr learned my addr 172.21.15.159:0/404919443 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:30.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.494+0000 7feff9e00640 1 -- 172.21.15.159:0/404919443 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feffc0d0650 msgr2=0x7feffc0d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:30.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.494+0000 7feff9e00640 1 --2- 172.21.15.159:0/404919443 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feffc0d0650 0x7feffc0d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:30.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.494+0000 7feff9e00640 1 -- 172.21.15.159:0/404919443 >> v1:172.21.15.178:6789/0 conn(0x7feffc0af830 legacy=0x7feffc160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:30.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.494+0000 7feff9e00640 1 -- 172.21.15.159:0/404919443 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feffc0ad400 con 0x7feffc15c390 2024-04-04T06:12:30.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.495+0000 7feff9e00640 1 --2- 172.21.15.159:0/404919443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feffc15c390 0x7feffc073790 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fefec009870 tx=0x7fefec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30248d841efc28d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:30.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.495+0000 7feff9400640 1 -- 172.21.15.159:0/404919443 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec035020 con 0x7feffc15c390 2024-04-04T06:12:30.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.495+0000 7feff9400640 1 -- 172.21.15.159:0/404919443 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefec031640 con 0x7feffc15c390 2024-04-04T06:12:30.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.495+0000 7feff9400640 1 -- 172.21.15.159:0/404919443 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec02d040 con 0x7feffc15c390 2024-04-04T06:12:30.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.496+0000 7ff001200640 1 -- 172.21.15.159:0/404919443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feffc15c390 msgr2=0x7feffc073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:30.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.496+0000 7ff001200640 1 --2- 172.21.15.159:0/404919443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feffc15c390 0x7feffc073790 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fefec009870 tx=0x7fefec02b8f0 comp rx=0 tx=0).stop 2024-04-04T06:12:30.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.496+0000 7ff001200640 1 -- 172.21.15.159:0/404919443 shutdown_connections 2024-04-04T06:12:30.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.496+0000 7ff001200640 1 --2- 172.21.15.159:0/404919443 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feffc0d0650 0x7feffc0d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:30.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.496+0000 7ff001200640 1 --2- 172.21.15.159:0/404919443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feffc15c390 0x7feffc073790 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:30.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.496+0000 7ff001200640 1 -- 172.21.15.159:0/404919443 >> 172.21.15.159:0/404919443 conn(0x7feffc06b990 msgr2=0x7feffc06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:30.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.496+0000 7ff001200640 1 -- 172.21.15.159:0/404919443 shutdown_connections 2024-04-04T06:12:30.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.496+0000 7ff001200640 1 -- 172.21.15.159:0/404919443 wait complete. 2024-04-04T06:12:30.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.498+0000 7ff001200640 1 Processor -- start 2024-04-04T06:12:30.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.498+0000 7ff001200640 1 -- start start 2024-04-04T06:12:30.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.499+0000 7ff001200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feffc0af830 0x7feffc1569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:30.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.499+0000 7ff001200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feffc0d0650 0x7feffc09c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:30.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.499+0000 7ff001200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feffc15c390 0x7feffc09c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:30.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.499+0000 7ff001200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feffc160d50 con 0x7feffc15c390 2024-04-04T06:12:30.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.499+0000 7ff001200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feffc160120 con 0x7feffc0af830 2024-04-04T06:12:30.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.499+0000 7ff001200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feffc0cf980 con 0x7feffc0d0650 2024-04-04T06:12:30.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.499+0000 7feffa800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feffc15c390 0x7feffc09c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:30.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.499+0000 7feffa800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feffc15c390 0x7feffc09c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34088/0 (socket says 172.21.15.159:34088) 2024-04-04T06:12:30.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.499+0000 7feffa800640 1 -- 172.21.15.159:0/3607424106 learned_addr learned my addr 172.21.15.159:0/3607424106 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:30.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.500+0000 7feff9e00640 1 --2- 172.21.15.159:0/3607424106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feffc0af830 0x7feffc1569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:30.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.500+0000 7feffa800640 1 -- 172.21.15.159:0/3607424106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feffc0d0650 msgr2=0x7feffc09c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:30.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.500+0000 7feff3e00640 1 --2- 172.21.15.159:0/3607424106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feffc0d0650 0x7feffc09c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:30.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.500+0000 7feffa800640 1 --2- 172.21.15.159:0/3607424106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feffc0d0650 0x7feffc09c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:30.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.500+0000 7feffa800640 1 -- 172.21.15.159:0/3607424106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feffc0af830 msgr2=0x7feffc1569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:30.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.500+0000 7feffa800640 1 --2- 172.21.15.159:0/3607424106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feffc0af830 0x7feffc1569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:30.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.500+0000 7feffa800640 1 -- 172.21.15.159:0/3607424106 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feffc15d440 con 0x7feffc15c390 2024-04-04T06:12:30.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.500+0000 7feff9e00640 1 --2- 172.21.15.159:0/3607424106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feffc0af830 0x7feffc1569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:30.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.500+0000 7feffa800640 1 --2- 172.21.15.159:0/3607424106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feffc15c390 0x7feffc09c730 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7feff400b970 tx=0x7feff400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:30.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.500+0000 7feff2000640 1 -- 172.21.15.159:0/3607424106 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4019070 con 0x7feffc15c390 2024-04-04T06:12:30.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.500+0000 7feff2000640 1 -- 172.21.15.159:0/3607424106 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff400cb60 con 0x7feffc15c390 2024-04-04T06:12:30.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.500+0000 7ff001200640 1 -- 172.21.15.159:0/3607424106 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feffc15d720 con 0x7feffc15c390 2024-04-04T06:12:30.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.500+0000 7feff2000640 1 -- 172.21.15.159:0/3607424106 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4014440 con 0x7feffc15c390 2024-04-04T06:12:30.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.500+0000 7ff001200640 1 -- 172.21.15.159:0/3607424106 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feffc15dce0 con 0x7feffc15c390 2024-04-04T06:12:30.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.501+0000 7ff001200640 1 -- 172.21.15.159:0/3607424106 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feffc0add80 con 0x7feffc15c390 2024-04-04T06:12:30.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.502+0000 7feff2000640 1 -- 172.21.15.159:0/3607424106 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feff401c070 con 0x7feffc15c390 2024-04-04T06:12:30.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.503+0000 7feff2000640 1 --2- 172.21.15.159:0/3607424106 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fefd806c8f0 0x7fefd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:30.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.503+0000 7feff2000640 1 -- 172.21.15.159:0/3607424106 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5029+0+0 (secure 0 0 0) 0x7feff408b600 con 0x7feffc15c390 2024-04-04T06:12:30.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.504+0000 7feff9e00640 1 --2- 172.21.15.159:0/3607424106 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fefd806c8f0 0x7fefd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:30.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.505+0000 7feff9e00640 1 --2- 172.21.15.159:0/3607424106 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fefd806c8f0 0x7fefd806edb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fefec036000 tx=0x7fefec03c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:30.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.508+0000 7feff2000640 1 -- 172.21.15.159:0/3607424106 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7feffc0add80 con 0x7feffc15c390 2024-04-04T06:12:30.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.745+0000 7ff001200640 1 -- 172.21.15.159:0/3607424106 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feffc06f1b0 con 0x7feffc15c390 2024-04-04T06:12:30.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.746+0000 7feff2000640 1 -- 172.21.15.159:0/3607424106 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+3875 (secure 0 0 0) 0x7feff4059cc0 con 0x7feffc15c390 2024-04-04T06:12:30.740 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:30.740 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/3789513463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":3789513463},{"type":"v1","addr":"172.21.15.178:6835","nonce":3789513463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/468875883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":468875883},{"type":"v1","addr":"172.21.15.178:6837","nonce":468875883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2546974742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2546974742},{"type":"v1","addr":"172.21.15.159:6833","nonce":2546974742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":7218,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/715962652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":715962652},{"type":"v1","addr":"172.21.15.159:6835","nonce":715962652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[]} 2024-04-04T06:12:30.740 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 55 2024-04-04T06:12:30.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.750+0000 7ff001200640 1 -- 172.21.15.159:0/3607424106 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fefd806c8f0 msgr2=0x7fefd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:30.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.750+0000 7ff001200640 1 --2- 172.21.15.159:0/3607424106 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fefd806c8f0 0x7fefd806edb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fefec036000 tx=0x7fefec03c040 comp rx=0 tx=0).stop 2024-04-04T06:12:30.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.750+0000 7ff001200640 1 -- 172.21.15.159:0/3607424106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feffc15c390 msgr2=0x7feffc09c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:30.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.750+0000 7ff001200640 1 --2- 172.21.15.159:0/3607424106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feffc15c390 0x7feffc09c730 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7feff400b970 tx=0x7feff400be30 comp rx=0 tx=0).stop 2024-04-04T06:12:30.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.750+0000 7ff001200640 1 -- 172.21.15.159:0/3607424106 shutdown_connections 2024-04-04T06:12:30.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.750+0000 7ff001200640 1 --2- 172.21.15.159:0/3607424106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feffc15c390 0x7feffc09c730 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:30.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.750+0000 7ff001200640 1 --2- 172.21.15.159:0/3607424106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feffc0d0650 0x7feffc09c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:30.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.750+0000 7ff001200640 1 --2- 172.21.15.159:0/3607424106 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fefd806c8f0 0x7fefd806edb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:30.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.750+0000 7ff001200640 1 --2- 172.21.15.159:0/3607424106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feffc0af830 0x7feffc1569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:30.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.750+0000 7ff001200640 1 -- 172.21.15.159:0/3607424106 >> 172.21.15.159:0/3607424106 conn(0x7feffc06b990 msgr2=0x7feffc15a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:30.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.750+0000 7ff001200640 1 -- 172.21.15.159:0/3607424106 shutdown_connections 2024-04-04T06:12:30.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:30.750+0000 7ff001200640 1 -- 172.21.15.159:0/3607424106 wait complete. 2024-04-04T06:12:30.761 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-04T06:12:30.761 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-04T06:12:30.761 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-04T06:12:30.762 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:12:30.762 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-04T06:12:30.762 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-04T06:12:30.762 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-04T06:12:30.762 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:12:30.762 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-04T06:12:30.765 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-04T06:12:30.765 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-04T06:12:30.765 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:12:30.765 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-04T06:12:30.765 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-04T06:12:30.766 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-04T06:12:30.766 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:12:30.783 INFO:tasks.ceph.mds.a:Stopped 2024-04-04T06:12:30.783 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-04T06:12:30.786 INFO:tasks.ceph.mds.c:Stopped 2024-04-04T06:12:30.786 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-04T06:12:30.788 INFO:tasks.ceph.mds.d:Stopped 2024-04-04T06:12:30.788 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-04T06:12:30.788 INFO:tasks.ceph.mds.b:Stopped 2024-04-04T06:12:30.789 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-04T06:12:30.833 INFO:tasks.ceph.mds.a:Started 2024-04-04T06:12:30.834 DEBUG:teuthology.parallel:result is None 2024-04-04T06:12:30.836 INFO:tasks.ceph.mds.d:Started 2024-04-04T06:12:30.836 DEBUG:teuthology.parallel:result is None 2024-04-04T06:12:30.839 INFO:tasks.ceph.mds.c:Started 2024-04-04T06:12:30.840 DEBUG:teuthology.parallel:result is None 2024-04-04T06:12:30.842 INFO:tasks.ceph.mds.b:Started 2024-04-04T06:12:30.842 DEBUG:teuthology.parallel:result is None 2024-04-04T06:12:30.842 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-04T06:12:30.984 INFO:tasks.ceph.mds.c.smithi159.stdout:starting mds.c at 2024-04-04T06:12:30.984 INFO:tasks.ceph.mds.a.smithi159.stdout:starting mds.a at 2024-04-04T06:12:31.001 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-04T06:12:31.001 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-04T06:12:31.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.028+0000 7f7d91000640 1 Processor -- start 2024-04-04T06:12:31.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.028+0000 7f7d91000640 1 -- start start 2024-04-04T06:12:31.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.030+0000 7f7d91000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d8c0b4a30 0x7f7d8c0b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:31.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.030+0000 7f7d91000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7d8c158580 0x7f7d8c1589d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:31.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.030+0000 7f7d91000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d8c15ede0 con 0x7f7d8c0b4650 2024-04-04T06:12:31.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.030+0000 7f7d91000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f7d8c141b30 con 0x7f7d8c158580 2024-04-04T06:12:31.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.030+0000 7f7d91000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8c141d80 con 0x7f7d8c0b4a30 2024-04-04T06:12:31.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.030+0000 7f7d8a800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7d8c158580 0x7f7d8c1589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:31.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.030+0000 7f7d89e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f7d8c0b4650 0x7f7d8c06c680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:59522/0 (socket says 172.21.15.159:59522) 2024-04-04T06:12:31.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.030+0000 7f7d89e00640 1 -- 172.21.15.159:0/23734065 learned_addr learned my addr 172.21.15.159:0/23734065 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:31.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.030+0000 7f7d89400640 1 --2- 172.21.15.159:0/23734065 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d8c0b4a30 0x7f7d8c0b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:31.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.030+0000 7f7d8a800640 1 -- 172.21.15.159:0/23734065 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d8c0b4a30 msgr2=0x7f7d8c0b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:31.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.030+0000 7f7d8a800640 1 --2- 172.21.15.159:0/23734065 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d8c0b4a30 0x7f7d8c0b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:31.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.030+0000 7f7d8a800640 1 -- 172.21.15.159:0/23734065 >> v1:172.21.15.159:6789/0 conn(0x7f7d8c0b4650 legacy=0x7f7d8c06c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:31.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.030+0000 7f7d88a00640 1 -- 172.21.15.159:0/23734065 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 401989553 0 0) 0x7f7d8c15ede0 con 0x7f7d8c0b4650 2024-04-04T06:12:31.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.030+0000 7f7d8a800640 1 -- 172.21.15.159:0/23734065 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8c142060 con 0x7f7d8c158580 2024-04-04T06:12:31.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.031+0000 7f7d8a800640 1 --2- 172.21.15.159:0/23734065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d8c158580 0x7f7d8c1589d0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009d90 tx=0x7f7d8402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f98bcd22ac8698a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:31.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.032+0000 7f7d88a00640 1 -- 172.21.15.159:0/23734065 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84039070 con 0x7f7d8c158580 2024-04-04T06:12:31.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.032+0000 7f7d88a00640 1 -- 172.21.15.159:0/23734065 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d8402cc50 con 0x7f7d8c158580 2024-04-04T06:12:31.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.032+0000 7f7d88a00640 1 -- 172.21.15.159:0/23734065 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84034440 con 0x7f7d8c158580 2024-04-04T06:12:31.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.032+0000 7f7d91000640 1 -- 172.21.15.159:0/23734065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d8c158580 msgr2=0x7f7d8c1589d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:31.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.032+0000 7f7d91000640 1 --2- 172.21.15.159:0/23734065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d8c158580 0x7f7d8c1589d0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009d90 tx=0x7f7d8402bea0 comp rx=0 tx=0).stop 2024-04-04T06:12:31.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.032+0000 7f7d91000640 1 -- 172.21.15.159:0/23734065 shutdown_connections 2024-04-04T06:12:31.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.032+0000 7f7d91000640 1 --2- 172.21.15.159:0/23734065 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d8c0b4a30 0x7f7d8c0b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:31.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.032+0000 7f7d91000640 1 --2- 172.21.15.159:0/23734065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d8c158580 0x7f7d8c1589d0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:31.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.032+0000 7f7d91000640 1 -- 172.21.15.159:0/23734065 >> 172.21.15.159:0/23734065 conn(0x7f7d8c067670 msgr2=0x7f7d8c067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:31.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.032+0000 7f7d91000640 1 -- 172.21.15.159:0/23734065 shutdown_connections 2024-04-04T06:12:31.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.032+0000 7f7d91000640 1 -- 172.21.15.159:0/23734065 wait complete. 2024-04-04T06:12:31.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.034+0000 7f7d91000640 1 Processor -- start 2024-04-04T06:12:31.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.034+0000 7f7d91000640 1 -- start start 2024-04-04T06:12:31.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.035+0000 7f7d91000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d8c0b4650 0x7f7d8c15b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:31.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.035+0000 7f7d91000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d8c0b4a30 0x7f7d8c0cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:31.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.035+0000 7f7d91000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d8c158580 0x7f7d8c15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:31.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.035+0000 7f7d91000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c142900 con 0x7f7d8c158580 2024-04-04T06:12:31.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.035+0000 7f7d91000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c142620 con 0x7f7d8c0b4650 2024-04-04T06:12:31.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.035+0000 7f7d91000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8c158f10 con 0x7f7d8c0b4a30 2024-04-04T06:12:31.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.035+0000 7f7d8a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d8c158580 0x7f7d8c15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:31.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.035+0000 7f7d8a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d8c158580 0x7f7d8c15d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34118/0 (socket says 172.21.15.159:34118) 2024-04-04T06:12:31.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.035+0000 7f7d8a800640 1 -- 172.21.15.159:0/2834919405 learned_addr learned my addr 172.21.15.159:0/2834919405 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:31.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.036+0000 7f7d89400640 1 --2- 172.21.15.159:0/2834919405 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d8c0b4a30 0x7f7d8c0cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:31.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.036+0000 7f7d89e00640 1 --2- 172.21.15.159:0/2834919405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d8c0b4650 0x7f7d8c15b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:31.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.036+0000 7f7d8a800640 1 -- 172.21.15.159:0/2834919405 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d8c0b4a30 msgr2=0x7f7d8c0cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:31.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.036+0000 7f7d8a800640 1 --2- 172.21.15.159:0/2834919405 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d8c0b4a30 0x7f7d8c0cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:31.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.036+0000 7f7d8a800640 1 -- 172.21.15.159:0/2834919405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d8c0b4650 msgr2=0x7f7d8c15b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:31.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.036+0000 7f7d8a800640 1 --2- 172.21.15.159:0/2834919405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d8c0b4650 0x7f7d8c15b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:31.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.036+0000 7f7d8a800640 1 -- 172.21.15.159:0/2834919405 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8c0cce10 con 0x7f7d8c158580 2024-04-04T06:12:31.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.036+0000 7f7d89400640 1 --2- 172.21.15.159:0/2834919405 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d8c0b4a30 0x7f7d8c0cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:31.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.036+0000 7f7d89e00640 1 --2- 172.21.15.159:0/2834919405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d8c0b4650 0x7f7d8c15b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:31.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.036+0000 7f7d8a800640 1 --2- 172.21.15.159:0/2834919405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d8c158580 0x7f7d8c15d5e0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009d90 tx=0x7f7d84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:31.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.036+0000 7f7d82000640 1 -- 172.21.15.159:0/2834919405 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84044020 con 0x7f7d8c158580 2024-04-04T06:12:31.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.036+0000 7f7d82000640 1 -- 172.21.15.159:0/2834919405 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d8403a040 con 0x7f7d8c158580 2024-04-04T06:12:31.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.036+0000 7f7d82000640 1 -- 172.21.15.159:0/2834919405 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84039050 con 0x7f7d8c158580 2024-04-04T06:12:31.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.037+0000 7f7d91000640 1 -- 172.21.15.159:0/2834919405 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8c0cc570 con 0x7f7d8c158580 2024-04-04T06:12:31.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.037+0000 7f7d91000640 1 -- 172.21.15.159:0/2834919405 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d8c0ccb30 con 0x7f7d8c158580 2024-04-04T06:12:31.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.038+0000 7f7d91000640 1 -- 172.21.15.159:0/2834919405 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d8c0685c0 con 0x7f7d8c158580 2024-04-04T06:12:31.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.039+0000 7f7d82000640 1 -- 172.21.15.159:0/2834919405 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d84003990 con 0x7f7d8c158580 2024-04-04T06:12:31.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.039+0000 7f7d82000640 1 --2- 172.21.15.159:0/2834919405 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7d6c06c920 0x7f7d6c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:31.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.039+0000 7f7d82000640 1 -- 172.21.15.159:0/2834919405 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5029+0+0 (secure 0 0 0) 0x7f7d840adc90 con 0x7f7d8c158580 2024-04-04T06:12:31.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.040+0000 7f7d89e00640 1 --2- 172.21.15.159:0/2834919405 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7d6c06c920 0x7f7d6c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:31.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.040+0000 7f7d89e00640 1 --2- 172.21.15.159:0/2834919405 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7d6c06c920 0x7f7d6c06ede0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c000f60 tx=0x7f7d7c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:31.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.044+0000 7f7d82000640 1 -- 172.21.15.159:0/2834919405 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7d8404e060 con 0x7f7d8c158580 2024-04-04T06:12:31.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.240+0000 7f7d91000640 1 -- 172.21.15.159:0/2834919405 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f7d8c15bc90 con 0x7f7d8c158580 2024-04-04T06:12:31.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.356+0000 7f7d82000640 1 -- 172.21.15.159:0/2834919405 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v79) ==== 97+0+0 (secure 0 0 0) 0x7f7d8407c350 con 0x7f7d8c158580 2024-04-04T06:12:31.350 INFO:teuthology.orchestra.run.smithi159.stderr: removed all blocklist entries 2024-04-04T06:12:31.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.360+0000 7f7d91000640 1 -- 172.21.15.159:0/2834919405 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7d6c06c920 msgr2=0x7f7d6c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:31.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.360+0000 7f7d91000640 1 --2- 172.21.15.159:0/2834919405 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7d6c06c920 0x7f7d6c06ede0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c000f60 tx=0x7f7d7c009040 comp rx=0 tx=0).stop 2024-04-04T06:12:31.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.360+0000 7f7d91000640 1 -- 172.21.15.159:0/2834919405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d8c158580 msgr2=0x7f7d8c15d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:31.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.360+0000 7f7d91000640 1 --2- 172.21.15.159:0/2834919405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d8c158580 0x7f7d8c15d5e0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009d90 tx=0x7f7d84004a40 comp rx=0 tx=0).stop 2024-04-04T06:12:31.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.360+0000 7f7d91000640 1 -- 172.21.15.159:0/2834919405 shutdown_connections 2024-04-04T06:12:31.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.360+0000 7f7d91000640 1 --2- 172.21.15.159:0/2834919405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d8c158580 0x7f7d8c15d5e0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:31.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.360+0000 7f7d91000640 1 --2- 172.21.15.159:0/2834919405 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d8c0b4a30 0x7f7d8c0cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:31.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.360+0000 7f7d91000640 1 --2- 172.21.15.159:0/2834919405 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7d6c06c920 0x7f7d6c06ede0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:31.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.360+0000 7f7d91000640 1 --2- 172.21.15.159:0/2834919405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d8c0b4650 0x7f7d8c15b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:31.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.360+0000 7f7d91000640 1 -- 172.21.15.159:0/2834919405 >> 172.21.15.159:0/2834919405 conn(0x7f7d8c067670 msgr2=0x7f7d8c06c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:31.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.360+0000 7f7d91000640 1 -- 172.21.15.159:0/2834919405 shutdown_connections 2024-04-04T06:12:31.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.360+0000 7f7d91000640 1 -- 172.21.15.159:0/2834919405 wait complete. 2024-04-04T06:12:31.374 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-04T06:12:31.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.581+0000 7f9d53200640 1 Processor -- start 2024-04-04T06:12:31.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.581+0000 7f9d53200640 1 -- start start 2024-04-04T06:12:31.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.582+0000 7f9d53200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9d54072f50 0x7f9d540b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:31.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.582+0000 7f9d53200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d5415f710 0x7f9d540cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:31.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.583+0000 7f9d53200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d5415fb80 con 0x7f9d54144ad0 2024-04-04T06:12:31.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.583+0000 7f9d53200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f9d540cdad0 con 0x7f9d54072f50 2024-04-04T06:12:31.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.583+0000 7f9d53200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9d54144eb0 con 0x7f9d5415f710 2024-04-04T06:12:31.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.583+0000 7f9d49400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9d54072f50 0x7f9d540b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:31.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.583+0000 7f9d49400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9d54072f50 0x7f9d540b2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34128/0 (socket says 172.21.15.159:34128) 2024-04-04T06:12:31.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.583+0000 7f9d49400640 1 -- 172.21.15.159:0/71507095 learned_addr learned my addr 172.21.15.159:0/71507095 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:31.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.583+0000 7f9d52800640 1 --2- 172.21.15.159:0/71507095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d5415f710 0x7f9d540cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:31.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.583+0000 7f9d49400640 1 -- 172.21.15.159:0/71507095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d5415f710 msgr2=0x7f9d540cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:31.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.583+0000 7f9d49400640 1 --2- 172.21.15.159:0/71507095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d5415f710 0x7f9d540cdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:31.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.583+0000 7f9d49400640 1 -- 172.21.15.159:0/71507095 >> v1:172.21.15.178:6789/0 conn(0x7f9d54144ad0 legacy=0x7f9d54072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:31.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.583+0000 7f9d49400640 1 -- 172.21.15.159:0/71507095 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d540b36f0 con 0x7f9d54072f50 2024-04-04T06:12:31.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.584+0000 7f9d49400640 1 --2- 172.21.15.159:0/71507095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d54072f50 0x7f9d540b2ff0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c009870 tx=0x7f9d3c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b93b7dd2ff1ee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:31.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.585+0000 7f9d51400640 1 -- 172.21.15.159:0/71507095 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c035020 con 0x7f9d54072f50 2024-04-04T06:12:31.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.585+0000 7f9d51400640 1 -- 172.21.15.159:0/71507095 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d3c031640 con 0x7f9d54072f50 2024-04-04T06:12:31.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.585+0000 7f9d51400640 1 -- 172.21.15.159:0/71507095 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c02d040 con 0x7f9d54072f50 2024-04-04T06:12:31.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.585+0000 7f9d53200640 1 -- 172.21.15.159:0/71507095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d54072f50 msgr2=0x7f9d540b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:31.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.585+0000 7f9d53200640 1 --2- 172.21.15.159:0/71507095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d54072f50 0x7f9d540b2ff0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c009870 tx=0x7f9d3c02b8f0 comp rx=0 tx=0).stop 2024-04-04T06:12:31.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.585+0000 7f9d53200640 1 -- 172.21.15.159:0/71507095 shutdown_connections 2024-04-04T06:12:31.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.585+0000 7f9d53200640 1 --2- 172.21.15.159:0/71507095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d5415f710 0x7f9d540cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:31.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.585+0000 7f9d53200640 1 --2- 172.21.15.159:0/71507095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d54072f50 0x7f9d540b2ff0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:31.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.585+0000 7f9d53200640 1 -- 172.21.15.159:0/71507095 >> 172.21.15.159:0/71507095 conn(0x7f9d5406b570 msgr2=0x7f9d5406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:31.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.586+0000 7f9d53200640 1 -- 172.21.15.159:0/71507095 shutdown_connections 2024-04-04T06:12:31.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.586+0000 7f9d53200640 1 -- 172.21.15.159:0/71507095 wait complete. 2024-04-04T06:12:31.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.587+0000 7f9d53200640 1 Processor -- start 2024-04-04T06:12:31.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.587+0000 7f9d53200640 1 -- start start 2024-04-04T06:12:31.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.588+0000 7f9d53200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d54072f50 0x7f9d541392c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:31.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.588+0000 7f9d53200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d54144ad0 0x7f9d5415a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:31.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.588+0000 7f9d53200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d5415f710 0x7f9d54158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:31.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.588+0000 7f9d53200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9d540b3f90 con 0x7f9d54072f50 2024-04-04T06:12:31.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.588+0000 7f9d53200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9d540b3cb0 con 0x7f9d5415f710 2024-04-04T06:12:31.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.588+0000 7f9d53200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9d540ce4a0 con 0x7f9d54144ad0 2024-04-04T06:12:31.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.589+0000 7f9d51e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d54072f50 0x7f9d541392c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:31.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.589+0000 7f9d51e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d54072f50 0x7f9d541392c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34144/0 (socket says 172.21.15.159:34144) 2024-04-04T06:12:31.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.589+0000 7f9d51e00640 1 -- 172.21.15.159:0/3867833936 learned_addr learned my addr 172.21.15.159:0/3867833936 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:31.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.589+0000 7f9d49400640 1 --2- 172.21.15.159:0/3867833936 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d54144ad0 0x7f9d5415a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:31.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.589+0000 7f9d51e00640 1 -- 172.21.15.159:0/3867833936 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d54144ad0 msgr2=0x7f9d5415a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:31.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.589+0000 7f9d51e00640 1 --2- 172.21.15.159:0/3867833936 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d54144ad0 0x7f9d5415a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:31.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.589+0000 7f9d51e00640 1 -- 172.21.15.159:0/3867833936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d5415f710 msgr2=0x7f9d54158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:31.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.589+0000 7f9d51e00640 1 --2- 172.21.15.159:0/3867833936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d5415f710 0x7f9d54158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:31.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.589+0000 7f9d51e00640 1 -- 172.21.15.159:0/3867833936 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5415a8e0 con 0x7f9d54072f50 2024-04-04T06:12:31.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.589+0000 7f9d51e00640 1 --2- 172.21.15.159:0/3867833936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d54072f50 0x7f9d541392c0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400c460 tx=0x7f9d4400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:31.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.590+0000 7f9d4aa00640 1 -- 172.21.15.159:0/3867833936 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44016020 con 0x7f9d54072f50 2024-04-04T06:12:31.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.590+0000 7f9d4aa00640 1 -- 172.21.15.159:0/3867833936 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d44012870 con 0x7f9d54072f50 2024-04-04T06:12:31.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.590+0000 7f9d53200640 1 -- 172.21.15.159:0/3867833936 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d54168b10 con 0x7f9d54072f50 2024-04-04T06:12:31.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.590+0000 7f9d4aa00640 1 -- 172.21.15.159:0/3867833936 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44012c60 con 0x7f9d54072f50 2024-04-04T06:12:31.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.590+0000 7f9d53200640 1 -- 172.21.15.159:0/3867833936 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d541671b0 con 0x7f9d54072f50 2024-04-04T06:12:31.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.591+0000 7f9d53200640 1 -- 172.21.15.159:0/3867833936 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d54075940 con 0x7f9d54072f50 2024-04-04T06:12:31.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.592+0000 7f9d4aa00640 1 -- 172.21.15.159:0/3867833936 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d44019020 con 0x7f9d54072f50 2024-04-04T06:12:31.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.592+0000 7f9d4aa00640 1 --2- 172.21.15.159:0/3867833936 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9d3006c8f0 0x7f9d3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:31.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.593+0000 7f9d4aa00640 1 -- 172.21.15.159:0/3867833936 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4857+0+0 (secure 0 0 0) 0x7f9d4408b580 con 0x7f9d54072f50 2024-04-04T06:12:31.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.593+0000 7f9d49400640 1 --2- 172.21.15.159:0/3867833936 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9d3006c8f0 0x7f9d3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:31.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.594+0000 7f9d49400640 1 --2- 172.21.15.159:0/3867833936 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9d3006c8f0 0x7f9d3006edb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c036000 tx=0x7f9d3c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:31.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.597+0000 7f9d4aa00640 1 -- 172.21.15.159:0/3867833936 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9d44020030 con 0x7f9d54072f50 2024-04-04T06:12:31.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.851+0000 7f9d53200640 1 -- 172.21.15.159:0/3867833936 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f9d540b4240 con 0x7f9d54072f50 2024-04-04T06:12:31.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.852+0000 7f9d4aa00640 1 -- 172.21.15.159:0/3867833936 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f9d44059cf0 con 0x7f9d54072f50 2024-04-04T06:12:31.845 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:31.845 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:12:31.845 INFO:teuthology.orchestra.run.smithi159.stdout: "auth_dump": [ 2024-04-04T06:12:31.846 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:12:31.846 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.a", 2024-04-04T06:12:31.846 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5mvoZ2FxAAsVEQrkiICIH6T7ZzaqcWbA==", 2024-04-04T06:12:31.846 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:12:31.846 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:12:31.846 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:12:31.846 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:12:31.846 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:12:31.846 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:31.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:12:31.846 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:12:31.846 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.b", 2024-04-04T06:12:31.846 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5m+rccORAA6I/lQ+22QfWYL9+aoOFJ0w==", 2024-04-04T06:12:31.846 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:12:31.847 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:12:31.847 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:12:31.847 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:12:31.847 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:12:31.847 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:31.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:12:31.847 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:12:31.847 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.c", 2024-04-04T06:12:31.847 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5mjmQcKBAA8BjpQce6G7EycXOkyVECpQ==", 2024-04-04T06:12:31.847 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:12:31.847 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:12:31.847 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:12:31.847 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:12:31.847 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:12:31.847 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:31.848 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:12:31.848 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:12:31.848 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.d", 2024-04-04T06:12:31.848 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5m04hoDRAAKdhRI7nosZp7d4y7dVvzvg==", 2024-04-04T06:12:31.848 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:12:31.848 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:12:31.848 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:12:31.848 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:12:31.848 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:12:31.848 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:31.848 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:12:31.848 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:12:31.848 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.0", 2024-04-04T06:12:31.848 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQABQw5mCe7fNRAA6ZKxWvnMLf4jhAlpjudFRA==", 2024-04-04T06:12:31.848 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:12:31.849 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:12:31.849 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:12:31.849 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:12:31.849 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:31.849 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:12:31.849 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:12:31.849 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.1", 2024-04-04T06:12:31.849 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAFQw5mvIqTExAAU8Pm5alxFtHEL4qM9MavPw==", 2024-04-04T06:12:31.849 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:12:31.849 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:12:31.849 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:12:31.849 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:12:31.849 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:31.849 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:12:31.849 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:12:31.850 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.2", 2024-04-04T06:12:31.850 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAIQw5mAHblHRAAqEDe5dIGTDpdy2OsQyzJXA==", 2024-04-04T06:12:31.850 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:12:31.850 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:12:31.850 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:12:31.850 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:12:31.850 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:31.850 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:12:31.850 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:12:31.850 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.3", 2024-04-04T06:12:31.850 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQALQw5mwK7rJhAADQBAqE0Eej9rn91Wxhq/KA==", 2024-04-04T06:12:31.850 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:12:31.850 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:12:31.850 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:12:31.851 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:12:31.851 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:31.851 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:12:31.851 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:12:31.851 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.4", 2024-04-04T06:12:31.851 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAQQw5m/6I8GRAA2P7jI+AXYld2J11nDfhAUQ==", 2024-04-04T06:12:31.851 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:12:31.851 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:12:31.851 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:12:31.851 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:12:31.851 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:31.851 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:12:31.851 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:12:31.851 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.5", 2024-04-04T06:12:31.852 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQATQw5mv5HoMBAAn2mp2ys5KjS0UygOcX4vvg==", 2024-04-04T06:12:31.852 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:12:31.852 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:12:31.852 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:12:31.852 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:12:31.852 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:31.852 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:12:31.852 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:12:31.852 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.6", 2024-04-04T06:12:31.852 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAWQw5mx12gORAAT2LhrfEB5BJSl0ypgospzg==", 2024-04-04T06:12:31.852 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:12:31.852 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:12:31.852 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:12:31.852 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:12:31.853 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:31.853 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:12:31.853 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:12:31.853 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.7", 2024-04-04T06:12:31.853 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAaQw5m6HEOBxAAVegLH3bjVM7r1A6tK5eqHQ==", 2024-04-04T06:12:31.853 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:12:31.853 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:12:31.853 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:12:31.853 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:12:31.853 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:31.853 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:12:31.853 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:12:31.853 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.0", 2024-04-04T06:12:31.853 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mqNKJGxAA4jP9cadhRIyhQ3lssttWIw==", 2024-04-04T06:12:31.853 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:12:31.854 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:12:31.854 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow r", 2024-04-04T06:12:31.854 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-04T06:12:31.854 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:31.854 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:12:31.854 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:12:31.854 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.1", 2024-04-04T06:12:31.854 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mRkdOMBAA1ae1roIwrlB49DFQUoFSTg==", 2024-04-04T06:12:31.854 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:12:31.854 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:12:31.854 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:12:31.854 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:12:31.854 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:12:31.854 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:31.854 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:12:31.855 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:12:31.855 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.2", 2024-04-04T06:12:31.855 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mtW2/JRAAucxSSN9q0EQtI2a/P2yozQ==", 2024-04-04T06:12:31.855 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:12:31.855 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:12:31.855 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:12:31.855 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:12:31.855 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:12:31.855 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:31.855 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:12:31.855 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:12:31.855 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.3", 2024-04-04T06:12:31.855 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mgqD0OhAAov+3aDGyGEoqxep8+SBpSA==", 2024-04-04T06:12:31.855 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:12:31.855 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:12:31.856 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:12:31.856 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:12:31.856 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:12:31.856 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:31.856 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:12:31.856 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:12:31.856 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.admin", 2024-04-04T06:12:31.856 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD7Qg5m0O/iCBAAsWTzlEA0P60XYuxmJiAb4A==", 2024-04-04T06:12:31.856 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:12:31.856 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow *", 2024-04-04T06:12:31.856 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:12:31.856 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:12:31.856 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:12:31.856 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:31.856 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:12:31.857 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:12:31.857 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mgr.x", 2024-04-04T06:12:31.857 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5mhqbBCxAASBknMIj4uu7OiB3jn7WP8Q==", 2024-04-04T06:12:31.857 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:12:31.857 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow *", 2024-04-04T06:12:31.857 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile mgr", 2024-04-04T06:12:31.857 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:12:31.857 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:31.857 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:12:31.857 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:12:31.857 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mgr.y", 2024-04-04T06:12:31.857 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD7Qg5miXT0ORAAL6y306CPAfEGQKAJ0no4uw==", 2024-04-04T06:12:31.857 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:12:31.857 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow *", 2024-04-04T06:12:31.857 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile mgr", 2024-04-04T06:12:31.858 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:12:31.858 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:31.858 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:31.858 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-04-04T06:12:31.858 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:12:31.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.854+0000 7f9d53200640 1 -- 172.21.15.159:0/3867833936 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9d3006c8f0 msgr2=0x7f9d3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:31.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.855+0000 7f9d53200640 1 --2- 172.21.15.159:0/3867833936 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9d3006c8f0 0x7f9d3006edb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c036000 tx=0x7f9d3c008030 comp rx=0 tx=0).stop 2024-04-04T06:12:31.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.855+0000 7f9d53200640 1 -- 172.21.15.159:0/3867833936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d54072f50 msgr2=0x7f9d541392c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:31.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.855+0000 7f9d53200640 1 --2- 172.21.15.159:0/3867833936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d54072f50 0x7f9d541392c0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400c460 tx=0x7f9d4400c920 comp rx=0 tx=0).stop 2024-04-04T06:12:31.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.855+0000 7f9d53200640 1 -- 172.21.15.159:0/3867833936 shutdown_connections 2024-04-04T06:12:31.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.855+0000 7f9d53200640 1 --2- 172.21.15.159:0/3867833936 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9d3006c8f0 0x7f9d3006edb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:31.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.855+0000 7f9d53200640 1 --2- 172.21.15.159:0/3867833936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d5415f710 0x7f9d54158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:31.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.855+0000 7f9d53200640 1 --2- 172.21.15.159:0/3867833936 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9d54144ad0 0x7f9d5415a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:31.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.855+0000 7f9d53200640 1 --2- 172.21.15.159:0/3867833936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d54072f50 0x7f9d541392c0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:31.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.855+0000 7f9d53200640 1 -- 172.21.15.159:0/3867833936 >> 172.21.15.159:0/3867833936 conn(0x7f9d5406b570 msgr2=0x7f9d540b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:31.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.855+0000 7f9d53200640 1 -- 172.21.15.159:0/3867833936 shutdown_connections 2024-04-04T06:12:31.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:31.855+0000 7f9d53200640 1 -- 172.21.15.159:0/3867833936 wait complete. 2024-04-04T06:12:31.866 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:12:32.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.073+0000 7f5f7de00640 1 Processor -- start 2024-04-04T06:12:32.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.073+0000 7f5f7de00640 1 -- start start 2024-04-04T06:12:32.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.074+0000 7f5f7de00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5f780d0d60 0x7f5f78073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.074+0000 7f5f7de00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f7815fe80 con 0x7f5f7815c390 2024-04-04T06:12:32.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.074+0000 7f5f7de00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f780ad120 con 0x7f5f780af830 2024-04-04T06:12:32.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.074+0000 7f5f7de00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f5f7815c770 con 0x7f5f780d0d60 2024-04-04T06:12:32.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.075+0000 7f5f76a00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f5f7815c390 0x7f5f78160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:59534/0 (socket says 172.21.15.159:59534) 2024-04-04T06:12:32.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.075+0000 7f5f76a00640 1 -- 172.21.15.159:0/4146325377 learned_addr learned my addr 172.21.15.159:0/4146325377 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:32.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.075+0000 7f5f77400640 1 --2- 172.21.15.159:0/4146325377 >> v2:172.21.15.178:3300/0 conn(0x7f5f780d0d60 0x7f5f78073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:32.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.075+0000 7f5f75600640 1 -- 172.21.15.159:0/4146325377 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2637521748 0 0) 0x7f5f7815fe80 con 0x7f5f7815c390 2024-04-04T06:12:32.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.075+0000 7f5f75600640 1 -- 172.21.15.159:0/4146325377 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f5c003520 con 0x7f5f7815c390 2024-04-04T06:12:32.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.075+0000 7f5f75600640 1 -- 172.21.15.159:0/4146325377 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3090498118 0 0) 0x7f5f5c003520 con 0x7f5f7815c390 2024-04-04T06:12:32.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.075+0000 7f5f75600640 1 -- 172.21.15.159:0/4146325377 >> v2:172.21.15.178:3300/0 conn(0x7f5f780d0d60 msgr2=0x7f5f78073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:32.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.075+0000 7f5f75600640 1 --2- 172.21.15.159:0/4146325377 >> v2:172.21.15.178:3300/0 conn(0x7f5f780d0d60 0x7f5f78073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.076+0000 7f5f75600640 1 -- 172.21.15.159:0/4146325377 >> v1:172.21.15.178:6789/0 conn(0x7f5f780af830 legacy=0x7f5f780d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:32.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.076+0000 7f5f75600640 1 -- 172.21.15.159:0/4146325377 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f780ad400 con 0x7f5f7815c390 2024-04-04T06:12:32.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.076+0000 7f5f75600640 1 -- 172.21.15.159:0/4146325377 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f5f68001690 con 0x7f5f7815c390 2024-04-04T06:12:32.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.076+0000 7f5f75600640 1 -- 172.21.15.159:0/4146325377 >> v1:172.21.15.159:6789/0 conn(0x7f5f7815c390 legacy=0x7f5f78160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:32.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.076+0000 7f5f75600640 1 --2- 172.21.15.159:0/4146325377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f5c0039b0 0x7f5f5c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.076+0000 7f5f75600640 1 --2- 172.21.15.159:0/4146325377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f5c004800 0x7f5f5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.076+0000 7f5f75600640 1 --2- 172.21.15.159:0/4146325377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f5c0051c0 0x7f5f5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.076+0000 7f5f75600640 1 -- 172.21.15.159:0/4146325377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c00c010 con 0x7f5f5c004800 2024-04-04T06:12:32.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.076+0000 7f5f75600640 1 -- 172.21.15.159:0/4146325377 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c00c1c0 con 0x7f5f5c0039b0 2024-04-04T06:12:32.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.076+0000 7f5f75600640 1 -- 172.21.15.159:0/4146325377 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5f5c00c3c0 con 0x7f5f5c0051c0 2024-04-04T06:12:32.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.076+0000 7f5f76000640 1 --2- 172.21.15.159:0/4146325377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f5c004800 0x7f5f5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:32.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.076+0000 7f5f76a00640 1 --2- 172.21.15.159:0/4146325377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f5c0039b0 0x7f5f5c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:32.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.076+0000 7f5f77400640 1 --2- 172.21.15.159:0/4146325377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f5c0051c0 0x7f5f5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:32.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.076+0000 7f5f76000640 1 -- 172.21.15.159:0/4146325377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f5c0051c0 msgr2=0x7f5f5c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:32.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.076+0000 7f5f76000640 1 --2- 172.21.15.159:0/4146325377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f5c0051c0 0x7f5f5c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.076+0000 7f5f76000640 1 -- 172.21.15.159:0/4146325377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f5c0039b0 msgr2=0x7f5f5c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:32.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.076+0000 7f5f76000640 1 --2- 172.21.15.159:0/4146325377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f5c0039b0 0x7f5f5c003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.076+0000 7f5f76000640 1 -- 172.21.15.159:0/4146325377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5c003520 con 0x7f5f5c004800 2024-04-04T06:12:32.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.076+0000 7f5f76a00640 1 --2- 172.21.15.159:0/4146325377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f5c0039b0 0x7f5f5c003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:32.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.076+0000 7f5f77400640 1 -- 172.21.15.159:0/4146325377 reap_dead start 2024-04-04T06:12:32.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.077+0000 7f5f76000640 1 --2- 172.21.15.159:0/4146325377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f5c004800 0x7f5f5c004c80 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f5f60003db0 tx=0x7f5f5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2352f7682a966398 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:32.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.078+0000 7f5f75600640 1 -- 172.21.15.159:0/4146325377 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60037070 con 0x7f5f5c004800 2024-04-04T06:12:32.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.078+0000 7f5f75600640 1 -- 172.21.15.159:0/4146325377 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f6002be50 con 0x7f5f5c004800 2024-04-04T06:12:32.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.078+0000 7f5f75600640 1 -- 172.21.15.159:0/4146325377 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60003980 con 0x7f5f5c004800 2024-04-04T06:12:32.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.078+0000 7f5f7de00640 1 -- 172.21.15.159:0/4146325377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f5c004800 msgr2=0x7f5f5c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:32.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.078+0000 7f5f7de00640 1 --2- 172.21.15.159:0/4146325377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f5c004800 0x7f5f5c004c80 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f5f60003db0 tx=0x7f5f5c005830 comp rx=0 tx=0).stop 2024-04-04T06:12:32.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.078+0000 7f5f7de00640 1 -- 172.21.15.159:0/4146325377 shutdown_connections 2024-04-04T06:12:32.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.078+0000 7f5f7de00640 1 --2- 172.21.15.159:0/4146325377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f5c004800 0x7f5f5c004c80 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.078+0000 7f5f7de00640 1 -- 172.21.15.159:0/4146325377 >> 172.21.15.159:0/4146325377 conn(0x7f5f7806b990 msgr2=0x7f5f7806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:32.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.078+0000 7f5f7de00640 1 -- 172.21.15.159:0/4146325377 shutdown_connections 2024-04-04T06:12:32.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.078+0000 7f5f7de00640 1 -- 172.21.15.159:0/4146325377 wait complete. 2024-04-04T06:12:32.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.080+0000 7f5f7de00640 1 Processor -- start 2024-04-04T06:12:32.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.080+0000 7f5f7de00640 1 -- start start 2024-04-04T06:12:32.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.081+0000 7f5f7de00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f5c004800 0x7f5f7815f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.081+0000 7f5f7de00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f7815c770 0x7f5f781423c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.081+0000 7f5f7de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f780cd8c0 0x7f5f780cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.081+0000 7f5f7de00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5f780cf980 con 0x7f5f780cd8c0 2024-04-04T06:12:32.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.081+0000 7f5f7de00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5f78160120 con 0x7f5f7815c770 2024-04-04T06:12:32.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.081+0000 7f5f7de00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5f78073790 con 0x7f5f5c004800 2024-04-04T06:12:32.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.081+0000 7f5f77400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f780cd8c0 0x7f5f780cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:32.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.081+0000 7f5f77400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f780cd8c0 0x7f5f780cdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34154/0 (socket says 172.21.15.159:34154) 2024-04-04T06:12:32.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.081+0000 7f5f77400640 1 -- 172.21.15.159:0/2318096463 learned_addr learned my addr 172.21.15.159:0/2318096463 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:32.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.082+0000 7f5f76a00640 1 --2- 172.21.15.159:0/2318096463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f5c004800 0x7f5f7815f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:32.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.082+0000 7f5f77400640 1 -- 172.21.15.159:0/2318096463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f5c004800 msgr2=0x7f5f7815f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:32.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.082+0000 7f5f77400640 1 --2- 172.21.15.159:0/2318096463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f5c004800 0x7f5f7815f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.082+0000 7f5f77400640 1 -- 172.21.15.159:0/2318096463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f7815c770 msgr2=0x7f5f781423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:32.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.082+0000 7f5f77400640 1 --2- 172.21.15.159:0/2318096463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f7815c770 0x7f5f781423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.082+0000 7f5f77400640 1 -- 172.21.15.159:0/2318096463 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7809a2a0 con 0x7f5f780cd8c0 2024-04-04T06:12:32.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.082+0000 7f5f76a00640 1 --2- 172.21.15.159:0/2318096463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f5c004800 0x7f5f7815f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:32.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.082+0000 7f5f77400640 1 --2- 172.21.15.159:0/2318096463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f780cd8c0 0x7f5f780cdd40 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c004780 tx=0x7f5f7000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:32.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.082+0000 7f5f6f400640 1 -- 172.21.15.159:0/2318096463 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f70016620 con 0x7f5f780cd8c0 2024-04-04T06:12:32.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.082+0000 7f5f6f400640 1 -- 172.21.15.159:0/2318096463 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f7001b070 con 0x7f5f780cd8c0 2024-04-04T06:12:32.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.082+0000 7f5f6f400640 1 -- 172.21.15.159:0/2318096463 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f70016a00 con 0x7f5f780cd8c0 2024-04-04T06:12:32.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.082+0000 7f5f7de00640 1 -- 172.21.15.159:0/2318096463 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f7809a820 con 0x7f5f780cd8c0 2024-04-04T06:12:32.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.083+0000 7f5f7de00640 1 -- 172.21.15.159:0/2318096463 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f7809c940 con 0x7f5f780cd8c0 2024-04-04T06:12:32.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.084+0000 7f5f7de00640 1 -- 172.21.15.159:0/2318096463 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f780685c0 con 0x7f5f780cd8c0 2024-04-04T06:12:32.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.084+0000 7f5f6f400640 1 -- 172.21.15.159:0/2318096463 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f70005d10 con 0x7f5f780cd8c0 2024-04-04T06:12:32.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.085+0000 7f5f6f400640 1 --2- 172.21.15.159:0/2318096463 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5f5406c920 0x7f5f5406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.085+0000 7f5f6f400640 1 -- 172.21.15.159:0/2318096463 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4857+0+0 (secure 0 0 0) 0x7f5f7008c2b0 con 0x7f5f780cd8c0 2024-04-04T06:12:32.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.086+0000 7f5f76a00640 1 --2- 172.21.15.159:0/2318096463 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5f5406c920 0x7f5f5406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:32.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.087+0000 7f5f76a00640 1 --2- 172.21.15.159:0/2318096463 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5f5406c920 0x7f5f5406ede0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5f68001940 tx=0x7f5f68009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:32.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.090+0000 7f5f6f400640 1 -- 172.21.15.159:0/2318096463 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f5f7005a990 con 0x7f5f780cd8c0 2024-04-04T06:12:32.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.289+0000 7f5f7de00640 1 -- 172.21.15.159:0/2318096463 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f7806f1b0 con 0x7f5f780cd8c0 2024-04-04T06:12:32.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.291+0000 7f5f6f400640 1 -- 172.21.15.159:0/2318096463 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+12904 (secure 0 0 0) 0x7f5f7005a290 con 0x7f5f780cd8c0 2024-04-04T06:12:32.284 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:32.284 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":81,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:12:31.369606+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:12:32.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.294+0000 7f5f7de00640 1 -- 172.21.15.159:0/2318096463 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5f5406c920 msgr2=0x7f5f5406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:32.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.294+0000 7f5f7de00640 1 --2- 172.21.15.159:0/2318096463 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5f5406c920 0x7f5f5406ede0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5f68001940 tx=0x7f5f68009040 comp rx=0 tx=0).stop 2024-04-04T06:12:32.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.294+0000 7f5f7de00640 1 -- 172.21.15.159:0/2318096463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f780cd8c0 msgr2=0x7f5f780cdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:32.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.294+0000 7f5f7de00640 1 --2- 172.21.15.159:0/2318096463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f780cd8c0 0x7f5f780cdd40 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c004780 tx=0x7f5f7000f520 comp rx=0 tx=0).stop 2024-04-04T06:12:32.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.294+0000 7f5f7de00640 1 -- 172.21.15.159:0/2318096463 shutdown_connections 2024-04-04T06:12:32.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.294+0000 7f5f7de00640 1 --2- 172.21.15.159:0/2318096463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f780cd8c0 0x7f5f780cdd40 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.294+0000 7f5f7de00640 1 --2- 172.21.15.159:0/2318096463 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5f5406c920 0x7f5f5406ede0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.294+0000 7f5f7de00640 1 --2- 172.21.15.159:0/2318096463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f7815c770 0x7f5f781423c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.294+0000 7f5f7de00640 1 --2- 172.21.15.159:0/2318096463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f5c004800 0x7f5f7815f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.294+0000 7f5f7de00640 1 -- 172.21.15.159:0/2318096463 >> 172.21.15.159:0/2318096463 conn(0x7f5f7806b990 msgr2=0x7f5f7815b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:32.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.294+0000 7f5f7de00640 1 -- 172.21.15.159:0/2318096463 shutdown_connections 2024-04-04T06:12:32.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.294+0000 7f5f7de00640 1 -- 172.21.15.159:0/2318096463 wait complete. 2024-04-04T06:12:32.307 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:12:32.307 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:12:32.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.530+0000 7faa97e00640 1 Processor -- start 2024-04-04T06:12:32.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.530+0000 7faa97e00640 1 -- start start 2024-04-04T06:12:32.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.532+0000 7faa97e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa90155550 0x7faa90155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.532+0000 7faa97e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7faa900d0990 0x7faa900aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.532+0000 7faa97e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa900b40b0 con 0x7faa9015f800 2024-04-04T06:12:32.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.532+0000 7faa97e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7faa900b4390 con 0x7faa900d0990 2024-04-04T06:12:32.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.532+0000 7faa97e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7faa900b4670 con 0x7faa90155550 2024-04-04T06:12:32.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.533+0000 7faa95400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa90155550 0x7faa90155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:32.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.533+0000 7faa95e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7faa900d0990 0x7faa900aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:32.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.533+0000 7faa94a00640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7faa9015f800 0x7faa900d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.159:39042/0 (socket says 172.21.15.159:39042) 2024-04-04T06:12:32.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.533+0000 7faa94a00640 1 -- 172.21.15.159:0/3122630253 learned_addr learned my addr 172.21.15.159:0/3122630253 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:32.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.533+0000 7faa95e00640 1 -- 172.21.15.159:0/3122630253 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa90155550 msgr2=0x7faa90155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:32.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.533+0000 7faa95e00640 1 --2- 172.21.15.159:0/3122630253 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa90155550 0x7faa90155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.534+0000 7faa95e00640 1 -- 172.21.15.159:0/3122630253 >> v1:172.21.15.178:6789/0 conn(0x7faa9015f800 legacy=0x7faa900d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:32.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.534+0000 7faa95e00640 1 -- 172.21.15.159:0/3122630253 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa900b4950 con 0x7faa900d0990 2024-04-04T06:12:32.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.535+0000 7faa95e00640 1 --2- 172.21.15.159:0/3122630253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa900d0990 0x7faa900aede0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faa880097a0 tx=0x7faa8802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4337a868e97981f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:32.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.536+0000 7faa8fe00640 1 -- 172.21.15.159:0/3122630253 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88031780 con 0x7faa900d0990 2024-04-04T06:12:32.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.536+0000 7faa8fe00640 1 -- 172.21.15.159:0/3122630253 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa88037070 con 0x7faa900d0990 2024-04-04T06:12:32.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.536+0000 7faa8fe00640 1 -- 172.21.15.159:0/3122630253 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88031b40 con 0x7faa900d0990 2024-04-04T06:12:32.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.537+0000 7faa97e00640 1 -- 172.21.15.159:0/3122630253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa900d0990 msgr2=0x7faa900aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:32.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.537+0000 7faa97e00640 1 --2- 172.21.15.159:0/3122630253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa900d0990 0x7faa900aede0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faa880097a0 tx=0x7faa8802b820 comp rx=0 tx=0).stop 2024-04-04T06:12:32.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.537+0000 7faa97e00640 1 -- 172.21.15.159:0/3122630253 shutdown_connections 2024-04-04T06:12:32.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.537+0000 7faa97e00640 1 --2- 172.21.15.159:0/3122630253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa900d0990 0x7faa900aede0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.537+0000 7faa97e00640 1 --2- 172.21.15.159:0/3122630253 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa90155550 0x7faa90155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.537+0000 7faa97e00640 1 -- 172.21.15.159:0/3122630253 >> 172.21.15.159:0/3122630253 conn(0x7faa9006b750 msgr2=0x7faa9006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:32.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.537+0000 7faa97e00640 1 -- 172.21.15.159:0/3122630253 shutdown_connections 2024-04-04T06:12:32.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.537+0000 7faa97e00640 1 -- 172.21.15.159:0/3122630253 wait complete. 2024-04-04T06:12:32.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.539+0000 7faa97e00640 1 Processor -- start 2024-04-04T06:12:32.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.539+0000 7faa97e00640 1 -- start start 2024-04-04T06:12:32.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.539+0000 7faa97e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa900d0990 0x7faa90151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.540+0000 7faa97e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa90155550 0x7faa90152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.540+0000 7faa97e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa9015f800 0x7faa90152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.540+0000 7faa97e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faa9015ba20 con 0x7faa90155550 2024-04-04T06:12:32.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.540+0000 7faa97e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faa9015fc80 con 0x7faa9015f800 2024-04-04T06:12:32.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.540+0000 7faa97e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7faa9015bba0 con 0x7faa900d0990 2024-04-04T06:12:32.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.540+0000 7faa94a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa90155550 0x7faa90152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:32.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.540+0000 7faa94a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa90155550 0x7faa90152990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34164/0 (socket says 172.21.15.159:34164) 2024-04-04T06:12:32.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.540+0000 7faa94a00640 1 -- 172.21.15.159:0/2415459256 learned_addr learned my addr 172.21.15.159:0/2415459256 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:32.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.540+0000 7faa95400640 1 --2- 172.21.15.159:0/2415459256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa900d0990 0x7faa90151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:32.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.540+0000 7faa95e00640 1 --2- 172.21.15.159:0/2415459256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa9015f800 0x7faa90152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:32.534 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.540+0000 7faa94a00640 1 -- 172.21.15.159:0/2415459256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa900d0990 msgr2=0x7faa90151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:32.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.540+0000 7faa94a00640 1 --2- 172.21.15.159:0/2415459256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa900d0990 0x7faa90151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.540+0000 7faa94a00640 1 -- 172.21.15.159:0/2415459256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa9015f800 msgr2=0x7faa90152ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:32.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.540+0000 7faa94a00640 1 --2- 172.21.15.159:0/2415459256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa9015f800 0x7faa90152ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.540+0000 7faa94a00640 1 -- 172.21.15.159:0/2415459256 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa900b0590 con 0x7faa90155550 2024-04-04T06:12:32.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.541+0000 7faa94a00640 1 --2- 172.21.15.159:0/2415459256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa90155550 0x7faa90152990 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00cb40 tx=0x7faa7c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:32.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.541+0000 7faa8d600640 1 -- 172.21.15.159:0/2415459256 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7c017070 con 0x7faa90155550 2024-04-04T06:12:32.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.541+0000 7faa8d600640 1 -- 172.21.15.159:0/2415459256 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa7c00acc0 con 0x7faa90155550 2024-04-04T06:12:32.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.541+0000 7faa8d600640 1 -- 172.21.15.159:0/2415459256 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7c012640 con 0x7faa90155550 2024-04-04T06:12:32.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.541+0000 7faa97e00640 1 -- 172.21.15.159:0/2415459256 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa900b0870 con 0x7faa90155550 2024-04-04T06:12:32.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.541+0000 7faa97e00640 1 -- 172.21.15.159:0/2415459256 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa9014ca20 con 0x7faa90155550 2024-04-04T06:12:32.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.542+0000 7faa97e00640 1 -- 172.21.15.159:0/2415459256 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa900685c0 con 0x7faa90155550 2024-04-04T06:12:32.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.543+0000 7faa8d600640 1 -- 172.21.15.159:0/2415459256 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa7c01a070 con 0x7faa90155550 2024-04-04T06:12:32.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.544+0000 7faa8d600640 1 --2- 172.21.15.159:0/2415459256 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faa7006c8f0 0x7faa7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.544+0000 7faa8d600640 1 -- 172.21.15.159:0/2415459256 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4857+0+0 (secure 0 0 0) 0x7faa7c08b5b0 con 0x7faa90155550 2024-04-04T06:12:32.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.545+0000 7faa95400640 1 --2- 172.21.15.159:0/2415459256 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faa7006c8f0 0x7faa7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:32.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.546+0000 7faa95400640 1 --2- 172.21.15.159:0/2415459256 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faa7006c8f0 0x7faa7006edb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7faa80009ad0 tx=0x7faa80008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:32.541 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.548+0000 7faa8d600640 1 -- 172.21.15.159:0/2415459256 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7faa7c059d20 con 0x7faa90155550 2024-04-04T06:12:32.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.732+0000 7faa97e00640 1 -- 172.21.15.159:0/2415459256 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faa900b3ad0 con 0x7faa90155550 2024-04-04T06:12:32.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.733+0000 7faa8d600640 1 -- 172.21.15.159:0/2415459256 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7faa7c08b030 con 0x7faa90155550 2024-04-04T06:12:32.726 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:12:32.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.736+0000 7faa97e00640 1 -- 172.21.15.159:0/2415459256 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faa7006c8f0 msgr2=0x7faa7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:32.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.736+0000 7faa97e00640 1 --2- 172.21.15.159:0/2415459256 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faa7006c8f0 0x7faa7006edb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7faa80009ad0 tx=0x7faa80008040 comp rx=0 tx=0).stop 2024-04-04T06:12:32.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.736+0000 7faa97e00640 1 -- 172.21.15.159:0/2415459256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa90155550 msgr2=0x7faa90152990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:32.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.736+0000 7faa97e00640 1 --2- 172.21.15.159:0/2415459256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa90155550 0x7faa90152990 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00cb40 tx=0x7faa7c00a060 comp rx=0 tx=0).stop 2024-04-04T06:12:32.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.736+0000 7faa97e00640 1 -- 172.21.15.159:0/2415459256 shutdown_connections 2024-04-04T06:12:32.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.736+0000 7faa97e00640 1 --2- 172.21.15.159:0/2415459256 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faa7006c8f0 0x7faa7006edb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.736+0000 7faa97e00640 1 --2- 172.21.15.159:0/2415459256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa9015f800 0x7faa90152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.736+0000 7faa97e00640 1 --2- 172.21.15.159:0/2415459256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa90155550 0x7faa90152990 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.736+0000 7faa97e00640 1 --2- 172.21.15.159:0/2415459256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa900d0990 0x7faa90151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.736+0000 7faa97e00640 1 -- 172.21.15.159:0/2415459256 >> 172.21.15.159:0/2415459256 conn(0x7faa9006b750 msgr2=0x7faa9006de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:32.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.736+0000 7faa97e00640 1 -- 172.21.15.159:0/2415459256 shutdown_connections 2024-04-04T06:12:32.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.736+0000 7faa97e00640 1 -- 172.21.15.159:0/2415459256 wait complete. 2024-04-04T06:12:32.749 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-04T06:12:32.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.964+0000 7f9badc00640 1 Processor -- start 2024-04-04T06:12:32.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.964+0000 7f9badc00640 1 -- start start 2024-04-04T06:12:32.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.966+0000 7f9badc00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9ba80d1560 0x7f9ba806c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.966+0000 7f9badc00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ba806cfd0 con 0x7f9ba806cbc0 2024-04-04T06:12:32.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.966+0000 7f9badc00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ba8071060 con 0x7f9ba815e590 2024-04-04T06:12:32.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.966+0000 7f9badc00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f9ba80d19d0 con 0x7f9ba80d1560 2024-04-04T06:12:32.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.966+0000 7f9ba7400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f9ba806cbc0 0x7f9ba80cfd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:59546/0 (socket says 172.21.15.159:59546) 2024-04-04T06:12:32.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.966+0000 7f9ba7400640 1 -- 172.21.15.159:0/582486110 learned_addr learned my addr 172.21.15.159:0/582486110 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:32.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.967+0000 7f9ba6000640 1 --2- 172.21.15.159:0/582486110 >> v2:172.21.15.178:3300/0 conn(0x7f9ba80d1560 0x7f9ba806c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:32.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.967+0000 7f9ba5600640 1 -- 172.21.15.159:0/582486110 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2374969110 0 0) 0x7f9ba0001490 con 0x7f9ba806cbc0 2024-04-04T06:12:32.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.967+0000 7f9ba5600640 1 -- 172.21.15.159:0/582486110 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b8c003520 con 0x7f9ba806cbc0 2024-04-04T06:12:32.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.967+0000 7f9ba5600640 1 -- 172.21.15.159:0/582486110 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2986136504 0 0) 0x7f9b8c003520 con 0x7f9ba806cbc0 2024-04-04T06:12:32.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.967+0000 7f9ba5600640 1 -- 172.21.15.159:0/582486110 >> v2:172.21.15.178:3300/0 conn(0x7f9ba80d1560 msgr2=0x7f9ba806c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:32.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.967+0000 7f9ba5600640 1 --2- 172.21.15.159:0/582486110 >> v2:172.21.15.178:3300/0 conn(0x7f9ba80d1560 0x7f9ba806c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.967+0000 7f9ba5600640 1 -- 172.21.15.159:0/582486110 >> v1:172.21.15.178:6789/0 conn(0x7f9ba815e590 legacy=0x7f9ba80b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:32.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.967+0000 7f9ba5600640 1 -- 172.21.15.159:0/582486110 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba8071340 con 0x7f9ba806cbc0 2024-04-04T06:12:32.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.967+0000 7f9ba5600640 1 -- 172.21.15.159:0/582486110 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f9ba0001bc0 con 0x7f9ba806cbc0 2024-04-04T06:12:32.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.967+0000 7f9ba5600640 1 -- 172.21.15.159:0/582486110 >> v1:172.21.15.159:6789/0 conn(0x7f9ba806cbc0 legacy=0x7f9ba80cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:32.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.967+0000 7f9ba5600640 1 --2- 172.21.15.159:0/582486110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b8c003a00 0x7f9b8c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.967+0000 7f9ba5600640 1 --2- 172.21.15.159:0/582486110 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b8c004800 0x7f9b8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.967+0000 7f9ba5600640 1 --2- 172.21.15.159:0/582486110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b8c0051c0 0x7f9b8c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.967+0000 7f9ba5600640 1 -- 172.21.15.159:0/582486110 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c00c090 con 0x7f9b8c0051c0 2024-04-04T06:12:32.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.967+0000 7f9ba5600640 1 -- 172.21.15.159:0/582486110 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c00c240 con 0x7f9b8c003a00 2024-04-04T06:12:32.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.967+0000 7f9ba5600640 1 -- 172.21.15.159:0/582486110 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8c00c440 con 0x7f9b8c004800 2024-04-04T06:12:32.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.968+0000 7f9ba7400640 1 --2- 172.21.15.159:0/582486110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b8c0051c0 0x7f9b8c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:32.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.968+0000 7f9ba6a00640 1 --2- 172.21.15.159:0/582486110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b8c003a00 0x7f9b8c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:32.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.968+0000 7f9ba6000640 1 --2- 172.21.15.159:0/582486110 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b8c004800 0x7f9b8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:32.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.968+0000 7f9ba7400640 1 -- 172.21.15.159:0/582486110 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b8c004800 msgr2=0x7f9b8c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:32.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.968+0000 7f9ba7400640 1 --2- 172.21.15.159:0/582486110 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b8c004800 0x7f9b8c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.968+0000 7f9ba7400640 1 -- 172.21.15.159:0/582486110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b8c003a00 msgr2=0x7f9b8c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:32.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.968+0000 7f9ba7400640 1 --2- 172.21.15.159:0/582486110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b8c003a00 0x7f9b8c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.968+0000 7f9ba7400640 1 -- 172.21.15.159:0/582486110 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c003520 con 0x7f9b8c0051c0 2024-04-04T06:12:32.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.968+0000 7f9ba6000640 1 --2- 172.21.15.159:0/582486110 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9b8c004800 0x7f9b8c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:32.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.969+0000 7f9ba7400640 1 -- 172.21.15.159:0/582486110 reap_dead start 2024-04-04T06:12:32.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.969+0000 7f9ba7400640 1 --2- 172.21.15.159:0/582486110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b8c0051c0 0x7f9b8c00b9c0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000a950 tx=0x7f9b8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb33dda98775373 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:32.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.969+0000 7f9ba5600640 1 -- 172.21.15.159:0/582486110 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba000a5a0 con 0x7f9b8c0051c0 2024-04-04T06:12:32.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.969+0000 7f9ba5600640 1 -- 172.21.15.159:0/582486110 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba0007290 con 0x7f9b8c0051c0 2024-04-04T06:12:32.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.969+0000 7f9ba5600640 1 -- 172.21.15.159:0/582486110 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0007680 con 0x7f9b8c0051c0 2024-04-04T06:12:32.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.970+0000 7f9badc00640 1 -- 172.21.15.159:0/582486110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b8c0051c0 msgr2=0x7f9b8c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:32.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.970+0000 7f9badc00640 1 --2- 172.21.15.159:0/582486110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b8c0051c0 0x7f9b8c00b9c0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000a950 tx=0x7f9b8c005830 comp rx=0 tx=0).stop 2024-04-04T06:12:32.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.970+0000 7f9badc00640 1 -- 172.21.15.159:0/582486110 shutdown_connections 2024-04-04T06:12:32.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.970+0000 7f9badc00640 1 --2- 172.21.15.159:0/582486110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b8c0051c0 0x7f9b8c00b9c0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.970+0000 7f9badc00640 1 -- 172.21.15.159:0/582486110 >> 172.21.15.159:0/582486110 conn(0x7f9ba80d2240 msgr2=0x7f9ba80d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:32.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.970+0000 7f9badc00640 1 -- 172.21.15.159:0/582486110 shutdown_connections 2024-04-04T06:12:32.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.970+0000 7f9badc00640 1 -- 172.21.15.159:0/582486110 wait complete. 2024-04-04T06:12:32.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.972+0000 7f9badc00640 1 Processor -- start 2024-04-04T06:12:32.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.972+0000 7f9badc00640 1 -- start start 2024-04-04T06:12:32.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.973+0000 7f9badc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b8c0051c0 0x7f9ba815eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.973+0000 7f9badc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ba8152220 0x7f9ba80b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.973+0000 7f9badc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ba80b76e0 0x7f9ba8150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.973+0000 7f9badc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba8071be0 con 0x7f9b8c0051c0 2024-04-04T06:12:32.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.973+0000 7f9badc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba8071900 con 0x7f9ba8152220 2024-04-04T06:12:32.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.973+0000 7f9badc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba815e970 con 0x7f9ba80b76e0 2024-04-04T06:12:32.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.973+0000 7f9ba6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b8c0051c0 0x7f9ba815eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:32.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.973+0000 7f9ba6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b8c0051c0 0x7f9ba815eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34178/0 (socket says 172.21.15.159:34178) 2024-04-04T06:12:32.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.973+0000 7f9ba6a00640 1 -- 172.21.15.159:0/3414770537 learned_addr learned my addr 172.21.15.159:0/3414770537 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:32.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.974+0000 7f9ba6a00640 1 -- 172.21.15.159:0/3414770537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ba80b76e0 msgr2=0x7f9ba8150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:32.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.974+0000 7f9ba6a00640 1 --2- 172.21.15.159:0/3414770537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ba80b76e0 0x7f9ba8150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.974+0000 7f9ba6a00640 1 -- 172.21.15.159:0/3414770537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ba8152220 msgr2=0x7f9ba80b71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:32.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.974+0000 7f9ba6a00640 1 --2- 172.21.15.159:0/3414770537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ba8152220 0x7f9ba80b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:32.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.974+0000 7f9ba6a00640 1 -- 172.21.15.159:0/3414770537 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba80b6100 con 0x7f9b8c0051c0 2024-04-04T06:12:32.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.974+0000 7f9ba6a00640 1 --2- 172.21.15.159:0/3414770537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b8c0051c0 0x7f9ba815eee0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800e410 tx=0x7f9b9800e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:32.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.974+0000 7f9b9ea00640 1 -- 172.21.15.159:0/3414770537 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b98019070 con 0x7f9b8c0051c0 2024-04-04T06:12:32.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.974+0000 7f9b9ea00640 1 -- 172.21.15.159:0/3414770537 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b980052e0 con 0x7f9b8c0051c0 2024-04-04T06:12:32.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.974+0000 7f9b9ea00640 1 -- 172.21.15.159:0/3414770537 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b98003810 con 0x7f9b8c0051c0 2024-04-04T06:12:32.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.974+0000 7f9badc00640 1 -- 172.21.15.159:0/3414770537 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba80b63e0 con 0x7f9b8c0051c0 2024-04-04T06:12:32.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.975+0000 7f9badc00640 1 -- 172.21.15.159:0/3414770537 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba80b69a0 con 0x7f9b8c0051c0 2024-04-04T06:12:32.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.976+0000 7f9badc00640 1 -- 172.21.15.159:0/3414770537 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba80685c0 con 0x7f9b8c0051c0 2024-04-04T06:12:32.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.977+0000 7f9b9ea00640 1 -- 172.21.15.159:0/3414770537 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b9801c070 con 0x7f9b8c0051c0 2024-04-04T06:12:32.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.978+0000 7f9b9ea00640 1 --2- 172.21.15.159:0/3414770537 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b8806c8f0 0x7f9b8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:32.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.978+0000 7f9b9ea00640 1 -- 172.21.15.159:0/3414770537 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4857+0+0 (secure 0 0 0) 0x7f9b9808ad80 con 0x7f9b8c0051c0 2024-04-04T06:12:32.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.978+0000 7f9ba6000640 1 --2- 172.21.15.159:0/3414770537 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b8806c8f0 0x7f9b8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:32.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.979+0000 7f9ba6000640 1 --2- 172.21.15.159:0/3414770537 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b8806c8f0 0x7f9b8806edb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8159960 tx=0x7f9b94008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:32.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:32.982+0000 7f9b9ea00640 1 -- 172.21.15.159:0/3414770537 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9b98058ea0 con 0x7f9b8c0051c0 2024-04-04T06:12:33.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.183+0000 7f9badc00640 1 -- 172.21.15.159:0/3414770537 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9ba80d1030 con 0x7f9b8c0051c0 2024-04-04T06:12:33.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.184+0000 7f9b9ea00640 1 -- 172.21.15.159:0/3414770537 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f9b9805d1a0 con 0x7f9b8c0051c0 2024-04-04T06:12:33.177 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:33.178 INFO:teuthology.orchestra.run.smithi159.stdout:[ 2024-04-04T06:12:33.178 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:12:33.178 INFO:teuthology.orchestra.run.smithi159.stdout: "poolnum": 1, 2024-04-04T06:12:33.178 INFO:teuthology.orchestra.run.smithi159.stdout: "poolname": ".mgr" 2024-04-04T06:12:33.178 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:33.178 INFO:teuthology.orchestra.run.smithi159.stdout:] 2024-04-04T06:12:33.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.187+0000 7f9badc00640 1 -- 172.21.15.159:0/3414770537 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b8806c8f0 msgr2=0x7f9b8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:33.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.187+0000 7f9badc00640 1 --2- 172.21.15.159:0/3414770537 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b8806c8f0 0x7f9b8806edb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8159960 tx=0x7f9b94008040 comp rx=0 tx=0).stop 2024-04-04T06:12:33.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.187+0000 7f9badc00640 1 -- 172.21.15.159:0/3414770537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b8c0051c0 msgr2=0x7f9ba815eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:33.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.187+0000 7f9badc00640 1 --2- 172.21.15.159:0/3414770537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b8c0051c0 0x7f9ba815eee0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800e410 tx=0x7f9b9800e8d0 comp rx=0 tx=0).stop 2024-04-04T06:12:33.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.187+0000 7f9badc00640 1 -- 172.21.15.159:0/3414770537 shutdown_connections 2024-04-04T06:12:33.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.187+0000 7f9badc00640 1 --2- 172.21.15.159:0/3414770537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ba80b76e0 0x7f9ba8150420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:33.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.187+0000 7f9badc00640 1 --2- 172.21.15.159:0/3414770537 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9b8806c8f0 0x7f9b8806edb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:33.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.187+0000 7f9badc00640 1 --2- 172.21.15.159:0/3414770537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ba8152220 0x7f9ba80b71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:33.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.187+0000 7f9badc00640 1 --2- 172.21.15.159:0/3414770537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b8c0051c0 0x7f9ba815eee0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:33.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.187+0000 7f9badc00640 1 -- 172.21.15.159:0/3414770537 >> 172.21.15.159:0/3414770537 conn(0x7f9ba80d2240 msgr2=0x7f9ba80cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:33.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.187+0000 7f9badc00640 1 -- 172.21.15.159:0/3414770537 shutdown_connections 2024-04-04T06:12:33.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.187+0000 7f9badc00640 1 -- 172.21.15.159:0/3414770537 wait complete. 2024-04-04T06:12:33.199 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-04T06:12:33.199 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-04T06:12:33.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.424+0000 7f5677e00640 1 Processor -- start 2024-04-04T06:12:33.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.424+0000 7f5677e00640 1 -- start start 2024-04-04T06:12:33.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.425+0000 7f5677e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678072a50 0x7f5678165fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:33.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.425+0000 7f5677e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5678071540 con 0x7f567815f320 2024-04-04T06:12:33.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.425+0000 7f5677e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5678144d60 con 0x7f5678144980 2024-04-04T06:12:33.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.425+0000 7f5677e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f56780b3b60 con 0x7f5678072a50 2024-04-04T06:12:33.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.426+0000 7f566fe00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f567815f320 0x7f56780b2ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:59556/0 (socket says 172.21.15.159:59556) 2024-04-04T06:12:33.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.426+0000 7f566fe00640 1 -- 172.21.15.159:0/3317335536 learned_addr learned my addr 172.21.15.159:0/3317335536 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:33.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.426+0000 7f5677400640 1 --2- 172.21.15.159:0/3317335536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678072a50 0x7f5678165fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:33.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.426+0000 7f5676000640 1 -- 172.21.15.159:0/3317335536 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3626240417 0 0) 0x7f5678071540 con 0x7f567815f320 2024-04-04T06:12:33.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.426+0000 7f5676000640 1 -- 172.21.15.159:0/3317335536 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f565c005730 con 0x7f567815f320 2024-04-04T06:12:33.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.427+0000 7f5676000640 1 -- 172.21.15.159:0/3317335536 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3737189669 0 0) 0x7f565c005730 con 0x7f567815f320 2024-04-04T06:12:33.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.427+0000 7f5676000640 1 -- 172.21.15.159:0/3317335536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678072a50 msgr2=0x7f5678165fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:33.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.427+0000 7f5676000640 1 --2- 172.21.15.159:0/3317335536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678072a50 0x7f5678165fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:33.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.427+0000 7f5676000640 1 -- 172.21.15.159:0/3317335536 >> v1:172.21.15.178:6789/0 conn(0x7f5678144980 legacy=0x7f567815ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:33.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.427+0000 7f5676000640 1 -- 172.21.15.159:0/3317335536 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56780b3e40 con 0x7f567815f320 2024-04-04T06:12:33.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.427+0000 7f5676000640 1 -- 172.21.15.159:0/3317335536 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f5660001670 con 0x7f567815f320 2024-04-04T06:12:33.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.427+0000 7f5676000640 1 -- 172.21.15.159:0/3317335536 >> v1:172.21.15.159:6789/0 conn(0x7f567815f320 legacy=0x7f56780b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:33.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.427+0000 7f5676000640 1 --2- 172.21.15.159:0/3317335536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678072a50 0x7f565c005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:33.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.427+0000 7f5676000640 1 --2- 172.21.15.159:0/3317335536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f565c006170 0x7f565c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:33.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.427+0000 7f5676000640 1 --2- 172.21.15.159:0/3317335536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f565c006f50 0x7f565c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:33.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.427+0000 7f5676000640 1 -- 172.21.15.159:0/3317335536 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56780b3b60 con 0x7f565c006f50 2024-04-04T06:12:33.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.427+0000 7f5676000640 1 -- 172.21.15.159:0/3317335536 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f565c00dd60 con 0x7f565c006170 2024-04-04T06:12:33.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.427+0000 7f5676000640 1 -- 172.21.15.159:0/3317335536 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f565c00df60 con 0x7f5678072a50 2024-04-04T06:12:33.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.427+0000 7f5677400640 1 --2- 172.21.15.159:0/3317335536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f565c006f50 0x7f565c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:33.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.427+0000 7f5676a00640 1 --2- 172.21.15.159:0/3317335536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678072a50 0x7f565c005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:33.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.427+0000 7f566fe00640 1 --2- 172.21.15.159:0/3317335536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f565c006170 0x7f565c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:33.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.428+0000 7f5677400640 1 -- 172.21.15.159:0/3317335536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678072a50 msgr2=0x7f565c005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:33.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.428+0000 7f5677400640 1 --2- 172.21.15.159:0/3317335536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678072a50 0x7f565c005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:33.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.428+0000 7f5677400640 1 -- 172.21.15.159:0/3317335536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f565c006170 msgr2=0x7f565c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:33.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.428+0000 7f5677400640 1 --2- 172.21.15.159:0/3317335536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f565c006170 0x7f565c006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:33.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.428+0000 7f5677400640 1 -- 172.21.15.159:0/3317335536 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565c005730 con 0x7f565c006f50 2024-04-04T06:12:33.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.428+0000 7f566fe00640 1 --2- 172.21.15.159:0/3317335536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f565c006170 0x7f565c006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:33.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.429+0000 7f5677400640 1 --2- 172.21.15.159:0/3317335536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f565c006f50 0x7f565c0073d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f5670009fa0 tx=0x7f565c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20f5c8cb4cf17f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:33.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.429+0000 7f5676000640 1 -- 172.21.15.159:0/3317335536 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670039070 con 0x7f565c006f50 2024-04-04T06:12:33.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.429+0000 7f5676000640 1 -- 172.21.15.159:0/3317335536 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f567002ccb0 con 0x7f565c006f50 2024-04-04T06:12:33.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.429+0000 7f5676000640 1 -- 172.21.15.159:0/3317335536 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670009bf0 con 0x7f565c006f50 2024-04-04T06:12:33.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.430+0000 7f5677e00640 1 -- 172.21.15.159:0/3317335536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f565c006f50 msgr2=0x7f565c0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:33.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.430+0000 7f5677e00640 1 --2- 172.21.15.159:0/3317335536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f565c006f50 0x7f565c0073d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f5670009fa0 tx=0x7f565c007a40 comp rx=0 tx=0).stop 2024-04-04T06:12:33.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.430+0000 7f5677400640 1 -- 172.21.15.159:0/3317335536 reap_dead start 2024-04-04T06:12:33.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.430+0000 7f5677e00640 1 -- 172.21.15.159:0/3317335536 shutdown_connections 2024-04-04T06:12:33.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.430+0000 7f5677e00640 1 -- 172.21.15.159:0/3317335536 >> 172.21.15.159:0/3317335536 conn(0x7f56780b1d80 msgr2=0x7f56780b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:33.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.430+0000 7f5677e00640 1 -- 172.21.15.159:0/3317335536 shutdown_connections 2024-04-04T06:12:33.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.430+0000 7f5677e00640 1 -- 172.21.15.159:0/3317335536 wait complete. 2024-04-04T06:12:33.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.432+0000 7f5677e00640 1 Processor -- start 2024-04-04T06:12:33.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.432+0000 7f5677e00640 1 -- start start 2024-04-04T06:12:33.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.433+0000 7f5677e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56781552a0 0x7f5678142c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:33.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.433+0000 7f5677e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678143140 0x7f5678071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:33.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.433+0000 7f5677e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5678071e80 0x7f56781411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:33.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.433+0000 7f5677e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56780b3730 con 0x7f5678071e80 2024-04-04T06:12:33.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.433+0000 7f5677e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56780b35b0 con 0x7f56781552a0 2024-04-04T06:12:33.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.433+0000 7f5677e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f56780b38b0 con 0x7f5678143140 2024-04-04T06:12:33.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.433+0000 7f5677400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5678071e80 0x7f56781411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:33.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.433+0000 7f5677400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5678071e80 0x7f56781411e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:34186/0 (socket says 172.21.15.159:34186) 2024-04-04T06:12:33.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.433+0000 7f5677400640 1 -- 172.21.15.159:0/2044962835 learned_addr learned my addr 172.21.15.159:0/2044962835 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:33.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.433+0000 7f5676a00640 1 --2- 172.21.15.159:0/2044962835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56781552a0 0x7f5678142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:33.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.433+0000 7f566fe00640 1 --2- 172.21.15.159:0/2044962835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678143140 0x7f5678071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:33.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.433+0000 7f5677400640 1 -- 172.21.15.159:0/2044962835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678143140 msgr2=0x7f5678071940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:33.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.433+0000 7f5677400640 1 --2- 172.21.15.159:0/2044962835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678143140 0x7f5678071940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:33.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.433+0000 7f5677400640 1 -- 172.21.15.159:0/2044962835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56781552a0 msgr2=0x7f5678142c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:33.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.433+0000 7f5677400640 1 --2- 172.21.15.159:0/2044962835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56781552a0 0x7f5678142c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:33.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.433+0000 7f5677400640 1 -- 172.21.15.159:0/2044962835 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56781f3fb0 con 0x7f5678071e80 2024-04-04T06:12:33.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.434+0000 7f5677400640 1 --2- 172.21.15.159:0/2044962835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5678071e80 0x7f56781411e0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f567000e200 tx=0x7f5670009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:33.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.434+0000 7f566f400640 1 -- 172.21.15.159:0/2044962835 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670039040 con 0x7f5678071e80 2024-04-04T06:12:33.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.434+0000 7f566f400640 1 -- 172.21.15.159:0/2044962835 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5670047070 con 0x7f5678071e80 2024-04-04T06:12:33.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.434+0000 7f566f400640 1 -- 172.21.15.159:0/2044962835 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567002ce60 con 0x7f5678071e80 2024-04-04T06:12:33.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.434+0000 7f5677e00640 1 -- 172.21.15.159:0/2044962835 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56781e6e70 con 0x7f5678071e80 2024-04-04T06:12:33.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.434+0000 7f5677e00640 1 -- 172.21.15.159:0/2044962835 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56781e7430 con 0x7f5678071e80 2024-04-04T06:12:33.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.435+0000 7f5677e00640 1 -- 172.21.15.159:0/2044962835 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5678075940 con 0x7f5678071e80 2024-04-04T06:12:33.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.436+0000 7f566f400640 1 -- 172.21.15.159:0/2044962835 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f567003f440 con 0x7f5678071e80 2024-04-04T06:12:33.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.438+0000 7f566f400640 1 --2- 172.21.15.159:0/2044962835 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f565406c920 0x7f565406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:33.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.438+0000 7f566f400640 1 -- 172.21.15.159:0/2044962835 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4857+0+0 (secure 0 0 0) 0x7f56700ad890 con 0x7f5678071e80 2024-04-04T06:12:33.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.439+0000 7f5676a00640 1 --2- 172.21.15.159:0/2044962835 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f565406c920 0x7f565406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:33.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.440+0000 7f5676a00640 1 --2- 172.21.15.159:0/2044962835 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f565406c920 0x7f565406ede0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5668005490 tx=0x7f5668009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:33.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.442+0000 7f566f400640 1 -- 172.21.15.159:0/2044962835 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f567007c110 con 0x7f5678071e80 2024-04-04T06:12:33.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:33.643+0000 7f5677e00640 1 -- 172.21.15.159:0/2044962835 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f56780b2300 con 0x7f5678071e80 2024-04-04T06:12:34.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.382+0000 7f566f400640 1 -- 172.21.15.159:0/2044962835 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f567007ba10 con 0x7f5678071e80 2024-04-04T06:12:34.376 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_metadata' created 2024-04-04T06:12:34.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.386+0000 7f5677e00640 1 -- 172.21.15.159:0/2044962835 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f565406c920 msgr2=0x7f565406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:34.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.386+0000 7f5677e00640 1 --2- 172.21.15.159:0/2044962835 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f565406c920 0x7f565406ede0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5668005490 tx=0x7f5668009040 comp rx=0 tx=0).stop 2024-04-04T06:12:34.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.386+0000 7f5677e00640 1 -- 172.21.15.159:0/2044962835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5678071e80 msgr2=0x7f56781411e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:34.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.386+0000 7f5677e00640 1 --2- 172.21.15.159:0/2044962835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5678071e80 0x7f56781411e0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f567000e200 tx=0x7f5670009750 comp rx=0 tx=0).stop 2024-04-04T06:12:34.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.387+0000 7f5677e00640 1 -- 172.21.15.159:0/2044962835 shutdown_connections 2024-04-04T06:12:34.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.387+0000 7f5677e00640 1 --2- 172.21.15.159:0/2044962835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5678071e80 0x7f56781411e0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:34.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.387+0000 7f5677e00640 1 --2- 172.21.15.159:0/2044962835 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5678143140 0x7f5678071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:34.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.387+0000 7f5677e00640 1 --2- 172.21.15.159:0/2044962835 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f565406c920 0x7f565406ede0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:34.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.387+0000 7f5677e00640 1 --2- 172.21.15.159:0/2044962835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56781552a0 0x7f5678142c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:34.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.387+0000 7f5677e00640 1 -- 172.21.15.159:0/2044962835 >> 172.21.15.159:0/2044962835 conn(0x7f56780b1d80 msgr2=0x7f567815e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:34.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.387+0000 7f5677e00640 1 -- 172.21.15.159:0/2044962835 shutdown_connections 2024-04-04T06:12:34.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.387+0000 7f5677e00640 1 -- 172.21.15.159:0/2044962835 wait complete. 2024-04-04T06:12:34.408 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-04T06:12:34.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.617+0000 7fdbc8e00640 1 Processor -- start 2024-04-04T06:12:34.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.618+0000 7fdbc8e00640 1 -- start start 2024-04-04T06:12:34.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.619+0000 7fdbc8e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdbc4158580 0x7fdbc41589a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:34.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.619+0000 7fdbc8e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbc415ede0 con 0x7fdbc40b4650 2024-04-04T06:12:34.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.619+0000 7fdbc8e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbc4141d80 con 0x7fdbc40b4a30 2024-04-04T06:12:34.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.619+0000 7fdbc8e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fdbc40b6320 con 0x7fdbc4158580 2024-04-04T06:12:34.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.620+0000 7fdbc1e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fdbc40b4650 0x7fdbc406c680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58220/0 (socket says 172.21.15.159:58220) 2024-04-04T06:12:34.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.620+0000 7fdbc1e00640 1 -- 172.21.15.159:0/3721555273 learned_addr learned my addr 172.21.15.159:0/3721555273 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:34.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.620+0000 7fdbc0a00640 1 -- 172.21.15.159:0/3721555273 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3574272815 0 0) 0x7fdbc415ede0 con 0x7fdbc40b4650 2024-04-04T06:12:34.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.620+0000 7fdbc2800640 1 --2- 172.21.15.159:0/3721555273 >> v2:172.21.15.178:3300/0 conn(0x7fdbc4158580 0x7fdbc41589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:34.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.620+0000 7fdbc0a00640 1 -- 172.21.15.159:0/3721555273 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdba8003520 con 0x7fdbc40b4650 2024-04-04T06:12:34.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.620+0000 7fdbc0a00640 1 -- 172.21.15.159:0/3721555273 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 498738015 0 0) 0x7fdba8003520 con 0x7fdbc40b4650 2024-04-04T06:12:34.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.621+0000 7fdbc0a00640 1 -- 172.21.15.159:0/3721555273 >> v2:172.21.15.178:3300/0 conn(0x7fdbc4158580 msgr2=0x7fdbc41589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:34.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.621+0000 7fdbc0a00640 1 --2- 172.21.15.159:0/3721555273 >> v2:172.21.15.178:3300/0 conn(0x7fdbc4158580 0x7fdbc41589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:34.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.621+0000 7fdbc0a00640 1 -- 172.21.15.159:0/3721555273 >> v1:172.21.15.178:6789/0 conn(0x7fdbc40b4a30 legacy=0x7fdbc40b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:34.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.621+0000 7fdbc0a00640 1 -- 172.21.15.159:0/3721555273 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc4142060 con 0x7fdbc40b4650 2024-04-04T06:12:34.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.621+0000 7fdbc0a00640 1 -- 172.21.15.159:0/3721555273 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fdbbc001a10 con 0x7fdbc40b4650 2024-04-04T06:12:34.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.621+0000 7fdbc0a00640 1 -- 172.21.15.159:0/3721555273 >> v1:172.21.15.159:6789/0 conn(0x7fdbc40b4650 legacy=0x7fdbc406c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:34.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.621+0000 7fdbc0a00640 1 --2- 172.21.15.159:0/3721555273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdba80039b0 0x7fdba8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:34.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.621+0000 7fdbc0a00640 1 --2- 172.21.15.159:0/3721555273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdba8004800 0x7fdba8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:34.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.621+0000 7fdbc0a00640 1 --2- 172.21.15.159:0/3721555273 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdba80051c0 0x7fdba800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:34.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.621+0000 7fdbc0a00640 1 -- 172.21.15.159:0/3721555273 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc40b6320 con 0x7fdba8004800 2024-04-04T06:12:34.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.621+0000 7fdbc0a00640 1 -- 172.21.15.159:0/3721555273 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdba800c060 con 0x7fdba80039b0 2024-04-04T06:12:34.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.621+0000 7fdbc0a00640 1 -- 172.21.15.159:0/3721555273 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdba800c260 con 0x7fdba80051c0 2024-04-04T06:12:34.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.621+0000 7fdbc1400640 1 --2- 172.21.15.159:0/3721555273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdba8004800 0x7fdba8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:34.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.621+0000 7fdbc1e00640 1 --2- 172.21.15.159:0/3721555273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdba80039b0 0x7fdba8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:34.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.621+0000 7fdbc2800640 1 --2- 172.21.15.159:0/3721555273 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdba80051c0 0x7fdba800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:34.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.621+0000 7fdbc1400640 1 -- 172.21.15.159:0/3721555273 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdba80051c0 msgr2=0x7fdba800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:34.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.621+0000 7fdbc1400640 1 --2- 172.21.15.159:0/3721555273 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdba80051c0 0x7fdba800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:34.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.621+0000 7fdbc1400640 1 -- 172.21.15.159:0/3721555273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdba80039b0 msgr2=0x7fdba8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:34.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.621+0000 7fdbc1400640 1 --2- 172.21.15.159:0/3721555273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdba80039b0 0x7fdba8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:34.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.621+0000 7fdbc1400640 1 -- 172.21.15.159:0/3721555273 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba8003520 con 0x7fdba8004800 2024-04-04T06:12:34.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.622+0000 7fdbc2800640 1 -- 172.21.15.159:0/3721555273 reap_dead start 2024-04-04T06:12:34.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.623+0000 7fdbc1400640 1 --2- 172.21.15.159:0/3721555273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdba8004800 0x7fdba8004c80 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fdbac001720 tx=0x7fdba8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67614765a9a94620 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:34.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.623+0000 7fdbc0a00640 1 -- 172.21.15.159:0/3721555273 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac001370 con 0x7fdba8004800 2024-04-04T06:12:34.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.623+0000 7fdbc0a00640 1 -- 172.21.15.159:0/3721555273 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbac0052d0 con 0x7fdba8004800 2024-04-04T06:12:34.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.623+0000 7fdbc0a00640 1 -- 172.21.15.159:0/3721555273 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac003830 con 0x7fdba8004800 2024-04-04T06:12:34.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.623+0000 7fdbc8e00640 1 -- 172.21.15.159:0/3721555273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdba8004800 msgr2=0x7fdba8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:34.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.623+0000 7fdbc8e00640 1 --2- 172.21.15.159:0/3721555273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdba8004800 0x7fdba8004c80 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fdbac001720 tx=0x7fdba8005830 comp rx=0 tx=0).stop 2024-04-04T06:12:34.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.623+0000 7fdbc8e00640 1 -- 172.21.15.159:0/3721555273 shutdown_connections 2024-04-04T06:12:34.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.623+0000 7fdbc8e00640 1 --2- 172.21.15.159:0/3721555273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdba8004800 0x7fdba8004c80 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:34.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.623+0000 7fdbc8e00640 1 -- 172.21.15.159:0/3721555273 >> 172.21.15.159:0/3721555273 conn(0x7fdbc4067670 msgr2=0x7fdbc4067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:34.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.624+0000 7fdbc8e00640 1 -- 172.21.15.159:0/3721555273 shutdown_connections 2024-04-04T06:12:34.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.624+0000 7fdbc8e00640 1 -- 172.21.15.159:0/3721555273 wait complete. 2024-04-04T06:12:34.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.625+0000 7fdbc8e00640 1 Processor -- start 2024-04-04T06:12:34.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.626+0000 7fdbc8e00640 1 -- start start 2024-04-04T06:12:34.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.626+0000 7fdbc8e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdba8004800 0x7fdbc41404a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:34.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.626+0000 7fdbc8e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbc40cbc90 0x7fdbc4160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:34.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.627+0000 7fdbc8e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbc40cc0d0 0x7fdbc4160980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:34.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.627+0000 7fdbc8e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc4142620 con 0x7fdbc40cbc90 2024-04-04T06:12:34.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.627+0000 7fdbc8e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc4141b30 con 0x7fdbc40cc0d0 2024-04-04T06:12:34.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.627+0000 7fdbc8e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc4142900 con 0x7fdba8004800 2024-04-04T06:12:34.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.627+0000 7fdbc1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbc40cbc90 0x7fdbc4160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:34.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.627+0000 7fdbc1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbc40cbc90 0x7fdbc4160440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56568/0 (socket says 172.21.15.159:56568) 2024-04-04T06:12:34.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.627+0000 7fdbc1400640 1 -- 172.21.15.159:0/46796604 learned_addr learned my addr 172.21.15.159:0/46796604 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:34.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.627+0000 7fdbc1e00640 1 --2- 172.21.15.159:0/46796604 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdba8004800 0x7fdbc41404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:34.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.627+0000 7fdbc2800640 1 --2- 172.21.15.159:0/46796604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbc40cc0d0 0x7fdbc4160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:34.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.627+0000 7fdbc1400640 1 -- 172.21.15.159:0/46796604 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdba8004800 msgr2=0x7fdbc41404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:34.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.627+0000 7fdbc1400640 1 --2- 172.21.15.159:0/46796604 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdba8004800 0x7fdbc41404a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:34.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.627+0000 7fdbc1400640 1 -- 172.21.15.159:0/46796604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbc40cc0d0 msgr2=0x7fdbc4160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:34.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.627+0000 7fdbc1400640 1 --2- 172.21.15.159:0/46796604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbc40cc0d0 0x7fdbc4160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:34.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.627+0000 7fdbc1400640 1 -- 172.21.15.159:0/46796604 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc41f43d0 con 0x7fdbc40cbc90 2024-04-04T06:12:34.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.628+0000 7fdbc1400640 1 --2- 172.21.15.159:0/46796604 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbc40cbc90 0x7fdbc4160440 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fdbac034040 tx=0x7fdbac001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:34.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.628+0000 7fdbba000640 1 -- 172.21.15.159:0/46796604 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac041070 con 0x7fdbc40cbc90 2024-04-04T06:12:34.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.628+0000 7fdbba000640 1 -- 172.21.15.159:0/46796604 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbac005480 con 0x7fdbc40cbc90 2024-04-04T06:12:34.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.628+0000 7fdbba000640 1 -- 172.21.15.159:0/46796604 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac007a30 con 0x7fdbc40cbc90 2024-04-04T06:12:34.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.628+0000 7fdbc8e00640 1 -- 172.21.15.159:0/46796604 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc41f46b0 con 0x7fdbc40cbc90 2024-04-04T06:12:34.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.628+0000 7fdbc8e00640 1 -- 172.21.15.159:0/46796604 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc41f4c70 con 0x7fdbc40cbc90 2024-04-04T06:12:34.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.629+0000 7fdbc8e00640 1 -- 172.21.15.159:0/46796604 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc4074f30 con 0x7fdbc40cbc90 2024-04-04T06:12:34.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.630+0000 7fdbba000640 1 -- 172.21.15.159:0/46796604 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdbac044070 con 0x7fdbc40cbc90 2024-04-04T06:12:34.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.632+0000 7fdbba000640 1 --2- 172.21.15.159:0/46796604 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdba006c920 0x7fdba006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:34.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.632+0000 7fdbba000640 1 -- 172.21.15.159:0/46796604 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5227+0+0 (secure 0 0 0) 0x7fdbac0ad4b0 con 0x7fdbc40cbc90 2024-04-04T06:12:34.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.633+0000 7fdbc1e00640 1 --2- 172.21.15.159:0/46796604 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdba006c920 0x7fdba006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:34.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.634+0000 7fdbc1e00640 1 --2- 172.21.15.159:0/46796604 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdba006c920 0x7fdba006ede0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc002230 tx=0x7fdbbc001250 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:34.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.638+0000 7fdbba000640 1 -- 172.21.15.159:0/46796604 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fdbac07bbd0 con 0x7fdbc40cbc90 2024-04-04T06:12:34.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:34.839+0000 7fdbc8e00640 1 -- 172.21.15.159:0/46796604 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fdbc40685c0 con 0x7fdbc40cbc90 2024-04-04T06:12:35.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.399+0000 7fdbba000640 1 -- 172.21.15.159:0/46796604 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7fdbac07b4d0 con 0x7fdbc40cbc90 2024-04-04T06:12:35.393 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_data' created 2024-04-04T06:12:35.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.403+0000 7fdbc8e00640 1 -- 172.21.15.159:0/46796604 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdba006c920 msgr2=0x7fdba006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:35.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.403+0000 7fdbc8e00640 1 --2- 172.21.15.159:0/46796604 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdba006c920 0x7fdba006ede0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc002230 tx=0x7fdbbc001250 comp rx=0 tx=0).stop 2024-04-04T06:12:35.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.403+0000 7fdbc8e00640 1 -- 172.21.15.159:0/46796604 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbc40cbc90 msgr2=0x7fdbc4160440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:35.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.403+0000 7fdbc8e00640 1 --2- 172.21.15.159:0/46796604 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbc40cbc90 0x7fdbc4160440 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fdbac034040 tx=0x7fdbac001120 comp rx=0 tx=0).stop 2024-04-04T06:12:35.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.404+0000 7fdbc8e00640 1 -- 172.21.15.159:0/46796604 shutdown_connections 2024-04-04T06:12:35.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.404+0000 7fdbc8e00640 1 --2- 172.21.15.159:0/46796604 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdba006c920 0x7fdba006ede0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:35.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.404+0000 7fdbc8e00640 1 --2- 172.21.15.159:0/46796604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbc40cc0d0 0x7fdbc4160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:35.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.404+0000 7fdbc8e00640 1 --2- 172.21.15.159:0/46796604 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbc40cbc90 0x7fdbc4160440 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:35.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.404+0000 7fdbc8e00640 1 --2- 172.21.15.159:0/46796604 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdba8004800 0x7fdbc41404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:35.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.404+0000 7fdbc8e00640 1 -- 172.21.15.159:0/46796604 >> 172.21.15.159:0/46796604 conn(0x7fdbc4067670 msgr2=0x7fdbc406cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:35.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.405+0000 7fdbc8e00640 1 -- 172.21.15.159:0/46796604 shutdown_connections 2024-04-04T06:12:35.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.405+0000 7fdbc8e00640 1 -- 172.21.15.159:0/46796604 wait complete. 2024-04-04T06:12:35.433 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-04T06:12:35.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.624+0000 7fbf32000640 1 Processor -- start 2024-04-04T06:12:35.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.624+0000 7fbf32000640 1 -- start start 2024-04-04T06:12:35.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.625+0000 7fbf32000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fbf2c153580 0x7fbf2c0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:35.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.625+0000 7fbf32000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbf2c0d07c0 0x7fbf2c0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:35.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.625+0000 7fbf32000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf2c0afab0 con 0x7fbf2c0af2e0 2024-04-04T06:12:35.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.625+0000 7fbf32000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fbf2c0d0c30 con 0x7fbf2c153580 2024-04-04T06:12:35.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.625+0000 7fbf32000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fbf2c15cc80 con 0x7fbf2c0d07c0 2024-04-04T06:12:35.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.626+0000 7fbf2b400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fbf2c153580 0x7fbf2c0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:35.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.626+0000 7fbf2b400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fbf2c153580 0x7fbf2c0d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56574/0 (socket says 172.21.15.159:56574) 2024-04-04T06:12:35.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.626+0000 7fbf2b400640 1 -- 172.21.15.159:0/1914092495 learned_addr learned my addr 172.21.15.159:0/1914092495 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:35.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.626+0000 7fbf2aa00640 1 --2- 172.21.15.159:0/1914092495 >> v2:172.21.15.178:3300/0 conn(0x7fbf2c0d07c0 0x7fbf2c0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:35.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.626+0000 7fbf2b400640 1 -- 172.21.15.159:0/1914092495 >> v2:172.21.15.178:3300/0 conn(0x7fbf2c0d07c0 msgr2=0x7fbf2c0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:35.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.626+0000 7fbf2b400640 1 --2- 172.21.15.159:0/1914092495 >> v2:172.21.15.178:3300/0 conn(0x7fbf2c0d07c0 0x7fbf2c0aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:35.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.626+0000 7fbf2b400640 1 -- 172.21.15.159:0/1914092495 >> v1:172.21.15.178:6789/0 conn(0x7fbf2c0af2e0 legacy=0x7fbf2c06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:35.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.626+0000 7fbf2b400640 1 -- 172.21.15.159:0/1914092495 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2c15da00 con 0x7fbf2c153580 2024-04-04T06:12:35.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.626+0000 7fbf2aa00640 1 --2- 172.21.15.159:0/1914092495 >> v2:172.21.15.178:3300/0 conn(0x7fbf2c0d07c0 0x7fbf2c0aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:35.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.627+0000 7fbf2b400640 1 --2- 172.21.15.159:0/1914092495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf2c153580 0x7fbf2c0d0280 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0cff70 tx=0x7fbf1c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22495ca1739bd001 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:35.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.627+0000 7fbf2a000640 1 -- 172.21.15.159:0/1914092495 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c02e070 con 0x7fbf2c153580 2024-04-04T06:12:35.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.628+0000 7fbf2a000640 1 -- 172.21.15.159:0/1914092495 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf1c037070 con 0x7fbf2c153580 2024-04-04T06:12:35.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.628+0000 7fbf2a000640 1 -- 172.21.15.159:0/1914092495 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c031a00 con 0x7fbf2c153580 2024-04-04T06:12:35.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.628+0000 7fbf32000640 1 -- 172.21.15.159:0/1914092495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf2c153580 msgr2=0x7fbf2c0d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:35.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.628+0000 7fbf32000640 1 --2- 172.21.15.159:0/1914092495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf2c153580 0x7fbf2c0d0280 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0cff70 tx=0x7fbf1c02b880 comp rx=0 tx=0).stop 2024-04-04T06:12:35.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.628+0000 7fbf32000640 1 -- 172.21.15.159:0/1914092495 shutdown_connections 2024-04-04T06:12:35.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.628+0000 7fbf32000640 1 --2- 172.21.15.159:0/1914092495 >> v2:172.21.15.178:3300/0 conn(0x7fbf2c0d07c0 0x7fbf2c0aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:35.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.628+0000 7fbf32000640 1 --2- 172.21.15.159:0/1914092495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf2c153580 0x7fbf2c0d0280 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:35.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.628+0000 7fbf32000640 1 -- 172.21.15.159:0/1914092495 >> 172.21.15.159:0/1914092495 conn(0x7fbf2c06b750 msgr2=0x7fbf2c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:35.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.628+0000 7fbf32000640 1 -- 172.21.15.159:0/1914092495 shutdown_connections 2024-04-04T06:12:35.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.628+0000 7fbf32000640 1 -- 172.21.15.159:0/1914092495 wait complete. 2024-04-04T06:12:35.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.630+0000 7fbf32000640 1 Processor -- start 2024-04-04T06:12:35.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.630+0000 7fbf32000640 1 -- start start 2024-04-04T06:12:35.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.631+0000 7fbf32000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf2c0af2e0 0x7fbf2c09dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:35.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.631+0000 7fbf32000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf2c0d07c0 0x7fbf2c151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:35.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.631+0000 7fbf32000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf2c153580 0x7fbf2c152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:35.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.631+0000 7fbf32000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c15d4e0 con 0x7fbf2c153580 2024-04-04T06:12:35.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.631+0000 7fbf32000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c15d200 con 0x7fbf2c0d07c0 2024-04-04T06:12:35.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.631+0000 7fbf32000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbf2c153960 con 0x7fbf2c0af2e0 2024-04-04T06:12:35.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.631+0000 7fbf2be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf2c153580 0x7fbf2c152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:35.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.631+0000 7fbf2be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf2c153580 0x7fbf2c152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56584/0 (socket says 172.21.15.159:56584) 2024-04-04T06:12:35.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.631+0000 7fbf2be00640 1 -- 172.21.15.159:0/3950332260 learned_addr learned my addr 172.21.15.159:0/3950332260 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:35.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.632+0000 7fbf2b400640 1 --2- 172.21.15.159:0/3950332260 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf2c0af2e0 0x7fbf2c09dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:35.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.632+0000 7fbf2aa00640 1 --2- 172.21.15.159:0/3950332260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf2c0d07c0 0x7fbf2c151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:35.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.632+0000 7fbf2be00640 1 -- 172.21.15.159:0/3950332260 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf2c0af2e0 msgr2=0x7fbf2c09dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:35.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.632+0000 7fbf2be00640 1 --2- 172.21.15.159:0/3950332260 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf2c0af2e0 0x7fbf2c09dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:35.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.632+0000 7fbf2be00640 1 -- 172.21.15.159:0/3950332260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf2c0d07c0 msgr2=0x7fbf2c151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:35.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.632+0000 7fbf2be00640 1 --2- 172.21.15.159:0/3950332260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf2c0d07c0 0x7fbf2c151c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:35.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.632+0000 7fbf2be00640 1 -- 172.21.15.159:0/3950332260 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2c09c550 con 0x7fbf2c153580 2024-04-04T06:12:35.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.632+0000 7fbf2b400640 1 --2- 172.21.15.159:0/3950332260 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf2c0af2e0 0x7fbf2c09dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:35.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.632+0000 7fbf2be00640 1 --2- 172.21.15.159:0/3950332260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf2c153580 0x7fbf2c152150 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400b990 tx=0x7fbf2400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:35.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.632+0000 7fbf23400640 1 -- 172.21.15.159:0/3950332260 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24019070 con 0x7fbf2c153580 2024-04-04T06:12:35.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.632+0000 7fbf23400640 1 -- 172.21.15.159:0/3950332260 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf2400cba0 con 0x7fbf2c153580 2024-04-04T06:12:35.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.632+0000 7fbf32000640 1 -- 172.21.15.159:0/3950332260 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2c09b790 con 0x7fbf2c153580 2024-04-04T06:12:35.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.632+0000 7fbf23400640 1 -- 172.21.15.159:0/3950332260 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24014440 con 0x7fbf2c153580 2024-04-04T06:12:35.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.633+0000 7fbf32000640 1 -- 172.21.15.159:0/3950332260 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf2c09bd50 con 0x7fbf2c153580 2024-04-04T06:12:35.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.634+0000 7fbf32000640 1 -- 172.21.15.159:0/3950332260 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf2c0685c0 con 0x7fbf2c153580 2024-04-04T06:12:35.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.635+0000 7fbf23400640 1 -- 172.21.15.159:0/3950332260 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf2401c070 con 0x7fbf2c153580 2024-04-04T06:12:35.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.636+0000 7fbf23400640 1 --2- 172.21.15.159:0/3950332260 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbf0c06c8f0 0x7fbf0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:35.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.636+0000 7fbf23400640 1 -- 172.21.15.159:0/3950332260 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5609+0+0 (secure 0 0 0) 0x7fbf2408b910 con 0x7fbf2c153580 2024-04-04T06:12:35.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.636+0000 7fbf2b400640 1 --2- 172.21.15.159:0/3950332260 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbf0c06c8f0 0x7fbf0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:35.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.637+0000 7fbf2b400640 1 --2- 172.21.15.159:0/3950332260 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbf0c06c8f0 0x7fbf0c06edb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c036000 tx=0x7fbf1c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:35.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.641+0000 7fbf23400640 1 -- 172.21.15.159:0/3950332260 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbf24059d00 con 0x7fbf2c153580 2024-04-04T06:12:35.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:35.877+0000 7fbf32000640 1 -- 172.21.15.159:0/3950332260 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fbf2c15b1b0 con 0x7fbf2c153580 2024-04-04T06:12:36.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.412+0000 7fbf23400640 1 -- 172.21.15.159:0/3950332260 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v58) ==== 174+0+0 (secure 0 0 0) 0x7fbf24059600 con 0x7fbf2c153580 2024-04-04T06:12:36.411 INFO:teuthology.orchestra.run.smithi159.stderr:new fs with metadata pool 12 and data pool 13 2024-04-04T06:12:36.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.425+0000 7fbf32000640 1 -- 172.21.15.159:0/3950332260 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbf0c06c8f0 msgr2=0x7fbf0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:36.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.425+0000 7fbf32000640 1 --2- 172.21.15.159:0/3950332260 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbf0c06c8f0 0x7fbf0c06edb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c036000 tx=0x7fbf1c008030 comp rx=0 tx=0).stop 2024-04-04T06:12:36.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.425+0000 7fbf32000640 1 -- 172.21.15.159:0/3950332260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf2c153580 msgr2=0x7fbf2c152150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:36.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.425+0000 7fbf32000640 1 --2- 172.21.15.159:0/3950332260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf2c153580 0x7fbf2c152150 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400b990 tx=0x7fbf2400be50 comp rx=0 tx=0).stop 2024-04-04T06:12:36.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.426+0000 7fbf32000640 1 -- 172.21.15.159:0/3950332260 shutdown_connections 2024-04-04T06:12:36.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.426+0000 7fbf32000640 1 --2- 172.21.15.159:0/3950332260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf2c153580 0x7fbf2c152150 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:36.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.426+0000 7fbf32000640 1 --2- 172.21.15.159:0/3950332260 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbf0c06c8f0 0x7fbf0c06edb0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:36.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.426+0000 7fbf32000640 1 --2- 172.21.15.159:0/3950332260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf2c0d07c0 0x7fbf2c151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:36.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.426+0000 7fbf32000640 1 --2- 172.21.15.159:0/3950332260 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf2c0af2e0 0x7fbf2c09dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:36.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.426+0000 7fbf32000640 1 -- 172.21.15.159:0/3950332260 >> 172.21.15.159:0/3950332260 conn(0x7fbf2c06b750 msgr2=0x7fbf2c06d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:36.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.426+0000 7fbf32000640 1 -- 172.21.15.159:0/3950332260 shutdown_connections 2024-04-04T06:12:36.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.426+0000 7fbf32000640 1 -- 172.21.15.159:0/3950332260 wait complete. 2024-04-04T06:12:36.452 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:12:36.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.632+0000 7fce2a200640 1 Processor -- start 2024-04-04T06:12:36.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.633+0000 7fce2a200640 1 -- start start 2024-04-04T06:12:36.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.634+0000 7fce2a200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fce24161b00 0x7fce24162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:36.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.634+0000 7fce2a200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fce240ad120 0x7fce240d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:36.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.634+0000 7fce2a200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce24071b60 con 0x7fce240ad580 2024-04-04T06:12:36.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.634+0000 7fce2a200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fce24161ee0 con 0x7fce24161b00 2024-04-04T06:12:36.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.634+0000 7fce2a200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fce240b3450 con 0x7fce240ad120 2024-04-04T06:12:36.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.634+0000 7fce23400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fce24161b00 0x7fce24162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:36.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.634+0000 7fce23400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fce24161b00 0x7fce24162ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56600/0 (socket says 172.21.15.159:56600) 2024-04-04T06:12:36.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.634+0000 7fce23400640 1 -- 172.21.15.159:0/2365948956 learned_addr learned my addr 172.21.15.159:0/2365948956 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:36.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.635+0000 7fce1be00640 1 --2- 172.21.15.159:0/2365948956 >> v2:172.21.15.178:3300/0 conn(0x7fce240ad120 0x7fce240d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:36.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.635+0000 7fce23400640 1 -- 172.21.15.159:0/2365948956 >> v2:172.21.15.178:3300/0 conn(0x7fce240ad120 msgr2=0x7fce240d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:36.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.635+0000 7fce23400640 1 --2- 172.21.15.159:0/2365948956 >> v2:172.21.15.178:3300/0 conn(0x7fce240ad120 0x7fce240d0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:36.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.635+0000 7fce23400640 1 -- 172.21.15.159:0/2365948956 >> v1:172.21.15.159:6789/0 conn(0x7fce240ad580 legacy=0x7fce240b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:36.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.635+0000 7fce23400640 1 -- 172.21.15.159:0/2365948956 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce24071e40 con 0x7fce24161b00 2024-04-04T06:12:36.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.636+0000 7fce23400640 1 --2- 172.21.15.159:0/2365948956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce24161b00 0x7fce24162ab0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fce2406ea10 tx=0x7fce1402b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e68a8c7d89f82a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:36.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.636+0000 7fce22a00640 1 -- 172.21.15.159:0/2365948956 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce14034070 con 0x7fce24161b00 2024-04-04T06:12:36.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.636+0000 7fce22a00640 1 -- 172.21.15.159:0/2365948956 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce14030870 con 0x7fce24161b00 2024-04-04T06:12:36.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.636+0000 7fce22a00640 1 -- 172.21.15.159:0/2365948956 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce14030c40 con 0x7fce24161b00 2024-04-04T06:12:36.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.637+0000 7fce2a200640 1 -- 172.21.15.159:0/2365948956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce24161b00 msgr2=0x7fce24162ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:36.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.637+0000 7fce2a200640 1 --2- 172.21.15.159:0/2365948956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce24161b00 0x7fce24162ab0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fce2406ea10 tx=0x7fce1402b9b0 comp rx=0 tx=0).stop 2024-04-04T06:12:36.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.637+0000 7fce2a200640 1 -- 172.21.15.159:0/2365948956 shutdown_connections 2024-04-04T06:12:36.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.637+0000 7fce2a200640 1 --2- 172.21.15.159:0/2365948956 >> v2:172.21.15.178:3300/0 conn(0x7fce240ad120 0x7fce240d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:36.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.637+0000 7fce2a200640 1 --2- 172.21.15.159:0/2365948956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce24161b00 0x7fce24162ab0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:36.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.637+0000 7fce2a200640 1 -- 172.21.15.159:0/2365948956 >> 172.21.15.159:0/2365948956 conn(0x7fce2406b990 msgr2=0x7fce2406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:36.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.637+0000 7fce2a200640 1 -- 172.21.15.159:0/2365948956 shutdown_connections 2024-04-04T06:12:36.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.637+0000 7fce2a200640 1 -- 172.21.15.159:0/2365948956 wait complete. 2024-04-04T06:12:36.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.638+0000 7fce2a200640 1 Processor -- start 2024-04-04T06:12:36.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.639+0000 7fce2a200640 1 -- start start 2024-04-04T06:12:36.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.639+0000 7fce2a200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce240ad120 0x7fce24152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:36.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.639+0000 7fce2a200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce240ad580 0x7fce24148530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:36.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.639+0000 7fce2a200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce24161b00 0x7fce2415dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:36.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.639+0000 7fce2a200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fce24162ff0 con 0x7fce240ad580 2024-04-04T06:12:36.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.639+0000 7fce2a200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fce240b3730 con 0x7fce240ad120 2024-04-04T06:12:36.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.639+0000 7fce2a200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fce240717f0 con 0x7fce24161b00 2024-04-04T06:12:36.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.640+0000 7fce1be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce240ad580 0x7fce24148530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:36.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.640+0000 7fce1be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce240ad580 0x7fce24148530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56604/0 (socket says 172.21.15.159:56604) 2024-04-04T06:12:36.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.640+0000 7fce1be00640 1 -- 172.21.15.159:0/1994081443 learned_addr learned my addr 172.21.15.159:0/1994081443 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:36.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.640+0000 7fce23400640 1 --2- 172.21.15.159:0/1994081443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce240ad120 0x7fce24152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:36.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.640+0000 7fce1be00640 1 -- 172.21.15.159:0/1994081443 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce24161b00 msgr2=0x7fce2415dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:36.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.640+0000 7fce1be00640 1 --2- 172.21.15.159:0/1994081443 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce24161b00 0x7fce2415dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:36.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.640+0000 7fce1be00640 1 -- 172.21.15.159:0/1994081443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce240ad120 msgr2=0x7fce24152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:36.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.640+0000 7fce1be00640 1 --2- 172.21.15.159:0/1994081443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce240ad120 0x7fce24152be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:36.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.640+0000 7fce1be00640 1 -- 172.21.15.159:0/1994081443 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2415e020 con 0x7fce240ad580 2024-04-04T06:12:36.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.640+0000 7fce1be00640 1 --2- 172.21.15.159:0/1994081443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce240ad580 0x7fce24148530 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00c7f0 tx=0x7fce0c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:36.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.641+0000 7fce1b400640 1 -- 172.21.15.159:0/1994081443 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0c015070 con 0x7fce240ad580 2024-04-04T06:12:36.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.641+0000 7fce1b400640 1 -- 172.21.15.159:0/1994081443 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce0c004220 con 0x7fce240ad580 2024-04-04T06:12:36.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.641+0000 7fce1b400640 1 -- 172.21.15.159:0/1994081443 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0c007620 con 0x7fce240ad580 2024-04-04T06:12:36.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.641+0000 7fce2a200640 1 -- 172.21.15.159:0/1994081443 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce241603c0 con 0x7fce240ad580 2024-04-04T06:12:36.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.641+0000 7fce2a200640 1 -- 172.21.15.159:0/1994081443 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce24160980 con 0x7fce240ad580 2024-04-04T06:12:36.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.642+0000 7fce2a200640 1 -- 172.21.15.159:0/1994081443 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce240685c0 con 0x7fce240ad580 2024-04-04T06:12:36.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.643+0000 7fce1b400640 1 -- 172.21.15.159:0/1994081443 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce0c0079d0 con 0x7fce240ad580 2024-04-04T06:12:36.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.643+0000 7fce1b400640 1 --2- 172.21.15.159:0/1994081443 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce0006c8f0 0x7fce0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:36.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.644+0000 7fce1b400640 1 -- 172.21.15.159:0/1994081443 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7fce0c08be70 con 0x7fce240ad580 2024-04-04T06:12:36.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.644+0000 7fce23400640 1 --2- 172.21.15.159:0/1994081443 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce0006c8f0 0x7fce0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:36.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.645+0000 7fce23400640 1 --2- 172.21.15.159:0/1994081443 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce0006c8f0 0x7fce0006edb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fce14036000 tx=0x7fce1403b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:36.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.648+0000 7fce1b400640 1 -- 172.21.15.159:0/1994081443 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fce2415e020 con 0x7fce240ad580 2024-04-04T06:12:36.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.844+0000 7fce2a200640 1 -- 172.21.15.159:0/1994081443 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce2406f3a0 con 0x7fce240ad580 2024-04-04T06:12:36.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.847+0000 7fce1b400640 1 -- 172.21.15.159:0/1994081443 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+16685 (secure 0 0 0) 0x7fce0c05a290 con 0x7fce240ad580 2024-04-04T06:12:36.841 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:36.841 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":84,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:12:35.879887+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:12:33.658405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-04T06:12:34.856266+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:12:36.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.850+0000 7fce2a200640 1 -- 172.21.15.159:0/1994081443 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce0006c8f0 msgr2=0x7fce0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:36.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.850+0000 7fce2a200640 1 --2- 172.21.15.159:0/1994081443 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce0006c8f0 0x7fce0006edb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fce14036000 tx=0x7fce1403b040 comp rx=0 tx=0).stop 2024-04-04T06:12:36.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.850+0000 7fce2a200640 1 -- 172.21.15.159:0/1994081443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce240ad580 msgr2=0x7fce24148530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:36.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.850+0000 7fce2a200640 1 --2- 172.21.15.159:0/1994081443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce240ad580 0x7fce24148530 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00c7f0 tx=0x7fce0c00ccb0 comp rx=0 tx=0).stop 2024-04-04T06:12:36.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.850+0000 7fce2a200640 1 -- 172.21.15.159:0/1994081443 shutdown_connections 2024-04-04T06:12:36.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.850+0000 7fce2a200640 1 --2- 172.21.15.159:0/1994081443 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce24161b00 0x7fce2415dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:36.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.850+0000 7fce2a200640 1 --2- 172.21.15.159:0/1994081443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce240ad580 0x7fce24148530 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:36.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.850+0000 7fce2a200640 1 --2- 172.21.15.159:0/1994081443 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce0006c8f0 0x7fce0006edb0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:36.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.850+0000 7fce2a200640 1 --2- 172.21.15.159:0/1994081443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce240ad120 0x7fce24152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:36.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.850+0000 7fce2a200640 1 -- 172.21.15.159:0/1994081443 >> 172.21.15.159:0/1994081443 conn(0x7fce2406b990 msgr2=0x7fce240afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:36.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.851+0000 7fce2a200640 1 -- 172.21.15.159:0/1994081443 shutdown_connections 2024-04-04T06:12:36.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:36.851+0000 7fce2a200640 1 -- 172.21.15.159:0/1994081443 wait complete. 2024-04-04T06:12:36.861 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:12:37.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.068+0000 7ffbd0e00640 1 Processor -- start 2024-04-04T06:12:37.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.068+0000 7ffbd0e00640 1 -- start start 2024-04-04T06:12:37.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.070+0000 7ffbd0e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ffbcc0ad120 0x7ffbcc0d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:37.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.070+0000 7ffbd0e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ffbcc0ad580 0x7ffbcc0d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:37.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.070+0000 7ffbd0e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbcc071b60 con 0x7ffbcc161b00 2024-04-04T06:12:37.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.070+0000 7ffbd0e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7ffbcc161ee0 con 0x7ffbcc0ad580 2024-04-04T06:12:37.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.070+0000 7ffbd0e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ffbcc0b3450 con 0x7ffbcc0ad120 2024-04-04T06:12:37.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.070+0000 7ffbca800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ffbcc0ad580 0x7ffbcc0d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:37.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.070+0000 7ffbc9e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7ffbcc161b00 0x7ffbcc162ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58246/0 (socket says 172.21.15.159:58246) 2024-04-04T06:12:37.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.070+0000 7ffbc9e00640 1 -- 172.21.15.159:0/971839254 learned_addr learned my addr 172.21.15.159:0/971839254 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:37.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.070+0000 7ffbca800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ffbcc0ad580 0x7ffbcc0d0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56610/0 (socket says 172.21.15.159:56610) 2024-04-04T06:12:37.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.070+0000 7ffbc1400640 1 --2- 172.21.15.159:0/971839254 >> v2:172.21.15.178:3300/0 conn(0x7ffbcc0ad120 0x7ffbcc0d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:37.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.071+0000 7ffbca800640 1 -- 172.21.15.159:0/971839254 >> v2:172.21.15.178:3300/0 conn(0x7ffbcc0ad120 msgr2=0x7ffbcc0d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:37.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.071+0000 7ffbca800640 1 --2- 172.21.15.159:0/971839254 >> v2:172.21.15.178:3300/0 conn(0x7ffbcc0ad120 0x7ffbcc0d0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:37.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.071+0000 7ffbca800640 1 -- 172.21.15.159:0/971839254 >> v1:172.21.15.159:6789/0 conn(0x7ffbcc161b00 legacy=0x7ffbcc162ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:37.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.071+0000 7ffbc9400640 1 -- 172.21.15.159:0/971839254 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2885953590 0 0) 0x7ffbcc071b60 con 0x7ffbcc161b00 2024-04-04T06:12:37.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.071+0000 7ffbca800640 1 -- 172.21.15.159:0/971839254 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbcc071e40 con 0x7ffbcc0ad580 2024-04-04T06:12:37.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.072+0000 7ffbca800640 1 --2- 172.21.15.159:0/971839254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbcc0ad580 0x7ffbcc0d0b90 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc06ea10 tx=0x7ffbc402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe1fc203b5aa6f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:37.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.072+0000 7ffbc9400640 1 -- 172.21.15.159:0/971839254 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc40315a0 con 0x7ffbcc0ad580 2024-04-04T06:12:37.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.072+0000 7ffbc9400640 1 -- 172.21.15.159:0/971839254 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbc4037070 con 0x7ffbcc0ad580 2024-04-04T06:12:37.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.072+0000 7ffbc9400640 1 -- 172.21.15.159:0/971839254 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4031960 con 0x7ffbcc0ad580 2024-04-04T06:12:37.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.073+0000 7ffbd0e00640 1 -- 172.21.15.159:0/971839254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbcc0ad580 msgr2=0x7ffbcc0d0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:37.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.073+0000 7ffbd0e00640 1 --2- 172.21.15.159:0/971839254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbcc0ad580 0x7ffbcc0d0b90 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc06ea10 tx=0x7ffbc402b840 comp rx=0 tx=0).stop 2024-04-04T06:12:37.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.073+0000 7ffbd0e00640 1 -- 172.21.15.159:0/971839254 shutdown_connections 2024-04-04T06:12:37.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.073+0000 7ffbd0e00640 1 --2- 172.21.15.159:0/971839254 >> v2:172.21.15.178:3300/0 conn(0x7ffbcc0ad120 0x7ffbcc0d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:37.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.073+0000 7ffbd0e00640 1 --2- 172.21.15.159:0/971839254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbcc0ad580 0x7ffbcc0d0b90 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:37.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.073+0000 7ffbd0e00640 1 -- 172.21.15.159:0/971839254 >> 172.21.15.159:0/971839254 conn(0x7ffbcc06b990 msgr2=0x7ffbcc06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:37.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.073+0000 7ffbd0e00640 1 -- 172.21.15.159:0/971839254 shutdown_connections 2024-04-04T06:12:37.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.073+0000 7ffbd0e00640 1 -- 172.21.15.159:0/971839254 wait complete. 2024-04-04T06:12:37.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.075+0000 7ffbd0e00640 1 Processor -- start 2024-04-04T06:12:37.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.075+0000 7ffbd0e00640 1 -- start start 2024-04-04T06:12:37.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.076+0000 7ffbd0e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbcc0ad120 0x7ffbcc151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:37.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.076+0000 7ffbd0e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbcc0ad580 0x7ffbcc15ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:37.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.076+0000 7ffbd0e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbcc161b00 0x7ffbcc158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:37.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.076+0000 7ffbd0e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc0717f0 con 0x7ffbcc161b00 2024-04-04T06:12:37.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.076+0000 7ffbd0e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc0b3730 con 0x7ffbcc0ad580 2024-04-04T06:12:37.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.076+0000 7ffbd0e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ffbcc0b5020 con 0x7ffbcc0ad120 2024-04-04T06:12:37.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.076+0000 7ffbca800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbcc161b00 0x7ffbcc158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:37.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.076+0000 7ffbca800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbcc161b00 0x7ffbcc158fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56618/0 (socket says 172.21.15.159:56618) 2024-04-04T06:12:37.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.076+0000 7ffbca800640 1 -- 172.21.15.159:0/171586192 learned_addr learned my addr 172.21.15.159:0/171586192 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:37.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.076+0000 7ffbc9e00640 1 --2- 172.21.15.159:0/171586192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbcc0ad120 0x7ffbcc151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:37.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.076+0000 7ffbc1400640 1 --2- 172.21.15.159:0/171586192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbcc0ad580 0x7ffbcc15ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:37.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.076+0000 7ffbca800640 1 -- 172.21.15.159:0/171586192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbcc0ad120 msgr2=0x7ffbcc151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:37.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.076+0000 7ffbca800640 1 --2- 172.21.15.159:0/171586192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbcc0ad120 0x7ffbcc151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:37.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.076+0000 7ffbca800640 1 -- 172.21.15.159:0/171586192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbcc0ad580 msgr2=0x7ffbcc15ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:37.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.076+0000 7ffbca800640 1 --2- 172.21.15.159:0/171586192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbcc0ad580 0x7ffbcc15ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:37.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.076+0000 7ffbca800640 1 -- 172.21.15.159:0/171586192 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbcc156a80 con 0x7ffbcc161b00 2024-04-04T06:12:37.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.077+0000 7ffbca800640 1 --2- 172.21.15.159:0/171586192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbcc161b00 0x7ffbcc158fe0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009890 tx=0x7ffbc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:37.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.077+0000 7ffbc2a00640 1 -- 172.21.15.159:0/171586192 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4004230 con 0x7ffbcc161b00 2024-04-04T06:12:37.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.077+0000 7ffbc2a00640 1 -- 172.21.15.159:0/171586192 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbc40043d0 con 0x7ffbcc161b00 2024-04-04T06:12:37.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.077+0000 7ffbc2a00640 1 -- 172.21.15.159:0/171586192 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4003850 con 0x7ffbcc161b00 2024-04-04T06:12:37.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.077+0000 7ffbd0e00640 1 -- 172.21.15.159:0/171586192 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbcc152420 con 0x7ffbcc161b00 2024-04-04T06:12:37.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.077+0000 7ffbd0e00640 1 -- 172.21.15.159:0/171586192 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbcc157000 con 0x7ffbcc161b00 2024-04-04T06:12:37.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.078+0000 7ffbd0e00640 1 -- 172.21.15.159:0/171586192 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbcc0685c0 con 0x7ffbcc161b00 2024-04-04T06:12:37.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.079+0000 7ffbc2a00640 1 -- 172.21.15.159:0/171586192 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffbc40039f0 con 0x7ffbcc161b00 2024-04-04T06:12:37.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.080+0000 7ffbc2a00640 1 --2- 172.21.15.159:0/171586192 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffba806c8f0 0x7ffba806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:37.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.080+0000 7ffbc2a00640 1 -- 172.21.15.159:0/171586192 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7ffbc402f030 con 0x7ffbcc161b00 2024-04-04T06:12:37.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.081+0000 7ffbc9e00640 1 --2- 172.21.15.159:0/171586192 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffba806c8f0 0x7ffba806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:37.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.082+0000 7ffbc9e00640 1 --2- 172.21.15.159:0/171586192 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffba806c8f0 0x7ffba806edb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc0012e0 tx=0x7ffbbc007ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:37.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.085+0000 7ffbc2a00640 1 -- 172.21.15.159:0/171586192 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ffbc4041070 con 0x7ffbcc161b00 2024-04-04T06:12:37.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.285+0000 7ffbd0e00640 1 -- 172.21.15.159:0/171586192 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffbcc06f3a0 con 0x7ffbcc161b00 2024-04-04T06:12:37.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.288+0000 7ffbc2a00640 1 -- 172.21.15.159:0/171586192 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+16685 (secure 0 0 0) 0x7ffbcc06f3a0 con 0x7ffbcc161b00 2024-04-04T06:12:37.281 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:37.282 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":84,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:12:35.879887+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:12:33.658405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-04T06:12:34.856266+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:12:37.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.290+0000 7ffbd0e00640 1 -- 172.21.15.159:0/171586192 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffba806c8f0 msgr2=0x7ffba806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:37.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.290+0000 7ffbd0e00640 1 --2- 172.21.15.159:0/171586192 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffba806c8f0 0x7ffba806edb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc0012e0 tx=0x7ffbbc007ed0 comp rx=0 tx=0).stop 2024-04-04T06:12:37.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.291+0000 7ffbd0e00640 1 -- 172.21.15.159:0/171586192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbcc161b00 msgr2=0x7ffbcc158fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:37.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.291+0000 7ffbd0e00640 1 --2- 172.21.15.159:0/171586192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbcc161b00 0x7ffbcc158fe0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009890 tx=0x7ffbc40046c0 comp rx=0 tx=0).stop 2024-04-04T06:12:37.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.291+0000 7ffbd0e00640 1 -- 172.21.15.159:0/171586192 shutdown_connections 2024-04-04T06:12:37.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.291+0000 7ffbd0e00640 1 --2- 172.21.15.159:0/171586192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbcc161b00 0x7ffbcc158fe0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:37.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.291+0000 7ffbd0e00640 1 --2- 172.21.15.159:0/171586192 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffba806c8f0 0x7ffba806edb0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:37.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.291+0000 7ffbd0e00640 1 --2- 172.21.15.159:0/171586192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbcc0ad580 0x7ffbcc15ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:37.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.291+0000 7ffbd0e00640 1 --2- 172.21.15.159:0/171586192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffbcc0ad120 0x7ffbcc151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:37.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.291+0000 7ffbd0e00640 1 -- 172.21.15.159:0/171586192 >> 172.21.15.159:0/171586192 conn(0x7ffbcc06b990 msgr2=0x7ffbcc0b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:37.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.291+0000 7ffbd0e00640 1 -- 172.21.15.159:0/171586192 shutdown_connections 2024-04-04T06:12:37.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.291+0000 7ffbd0e00640 1 -- 172.21.15.159:0/171586192 wait complete. 2024-04-04T06:12:37.302 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-04T06:12:37.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.554+0000 7f650a800640 1 Processor -- start 2024-04-04T06:12:37.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.554+0000 7f650a800640 1 -- start start 2024-04-04T06:12:37.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.556+0000 7f650a800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f650406dd30 0x7f65040cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:37.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.556+0000 7f650a800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65040ce890 con 0x7f65040cc880 2024-04-04T06:12:37.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.556+0000 7f650a800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65040ceb50 con 0x7f650413f190 2024-04-04T06:12:37.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.556+0000 7f650a800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f65040cd230 con 0x7f650406dd30 2024-04-04T06:12:37.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.556+0000 7f6508a00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f65040cc880 0x7f650406d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58252/0 (socket says 172.21.15.159:58252) 2024-04-04T06:12:37.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.556+0000 7f6508a00640 1 -- 172.21.15.159:0/1697768049 learned_addr learned my addr 172.21.15.159:0/1697768049 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:37.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.556+0000 7f6503e00640 1 -- 172.21.15.159:0/1697768049 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2026619245 0 0) 0x7f65040ce890 con 0x7f65040cc880 2024-04-04T06:12:37.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.556+0000 7f6509e00640 1 --2- 172.21.15.159:0/1697768049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f650406dd30 0x7f65040cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:37.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.557+0000 7f6503e00640 1 -- 172.21.15.159:0/1697768049 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64ec003520 con 0x7f65040cc880 2024-04-04T06:12:37.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.557+0000 7f6503e00640 1 -- 172.21.15.159:0/1697768049 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4010252558 0 0) 0x7f64ec003520 con 0x7f65040cc880 2024-04-04T06:12:37.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.557+0000 7f6503e00640 1 -- 172.21.15.159:0/1697768049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f650406dd30 msgr2=0x7f65040cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:37.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.557+0000 7f6503e00640 1 --2- 172.21.15.159:0/1697768049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f650406dd30 0x7f65040cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:37.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.557+0000 7f6503e00640 1 -- 172.21.15.159:0/1697768049 >> v1:172.21.15.178:6789/0 conn(0x7f650413f190 legacy=0x7f65040cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:37.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.557+0000 7f6503e00640 1 -- 172.21.15.159:0/1697768049 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65040cee30 con 0x7f65040cc880 2024-04-04T06:12:37.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.557+0000 7f6503e00640 1 -- 172.21.15.159:0/1697768049 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f64f0001690 con 0x7f65040cc880 2024-04-04T06:12:37.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.557+0000 7f6503e00640 1 -- 172.21.15.159:0/1697768049 >> v1:172.21.15.159:6789/0 conn(0x7f65040cc880 legacy=0x7f650406d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:37.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.557+0000 7f6503e00640 1 --2- 172.21.15.159:0/1697768049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64ec0039e0 0x7f64ec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:37.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.557+0000 7f6503e00640 1 --2- 172.21.15.159:0/1697768049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64ec004800 0x7f64ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:37.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.557+0000 7f6503e00640 1 --2- 172.21.15.159:0/1697768049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f650406dd30 0x7f64ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:37.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.557+0000 7f6503e00640 1 -- 172.21.15.159:0/1697768049 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f65040cd230 con 0x7f64ec004800 2024-04-04T06:12:37.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.557+0000 7f6503e00640 1 -- 172.21.15.159:0/1697768049 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec00bad0 con 0x7f64ec0039e0 2024-04-04T06:12:37.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.557+0000 7f6503e00640 1 -- 172.21.15.159:0/1697768049 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f64ec00bcd0 con 0x7f650406dd30 2024-04-04T06:12:37.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.557+0000 7f6508a00640 1 --2- 172.21.15.159:0/1697768049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64ec004800 0x7f64ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:37.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.558+0000 7f6509400640 1 --2- 172.21.15.159:0/1697768049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64ec0039e0 0x7f64ec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:37.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.558+0000 7f6509e00640 1 --2- 172.21.15.159:0/1697768049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f650406dd30 0x7f64ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:37.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.558+0000 7f6508a00640 1 -- 172.21.15.159:0/1697768049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f650406dd30 msgr2=0x7f64ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:37.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.558+0000 7f6508a00640 1 --2- 172.21.15.159:0/1697768049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f650406dd30 0x7f64ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:37.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.558+0000 7f6508a00640 1 -- 172.21.15.159:0/1697768049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64ec0039e0 msgr2=0x7f64ec003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:37.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.558+0000 7f6508a00640 1 --2- 172.21.15.159:0/1697768049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64ec0039e0 0x7f64ec003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:37.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.558+0000 7f6508a00640 1 -- 172.21.15.159:0/1697768049 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ec003520 con 0x7f64ec004800 2024-04-04T06:12:37.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.558+0000 7f6509400640 1 --2- 172.21.15.159:0/1697768049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64ec0039e0 0x7f64ec003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:37.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.558+0000 7f6509e00640 1 --2- 172.21.15.159:0/1697768049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f650406dd30 0x7f64ec0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:37.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.559+0000 7f6508a00640 1 --2- 172.21.15.159:0/1697768049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64ec004800 0x7f64ec004c80 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f64f0001eb0 tx=0x7f64ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699fdff4d7d28ec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:37.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.559+0000 7f6503e00640 1 -- 172.21.15.159:0/1697768049 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0007c80 con 0x7f64ec004800 2024-04-04T06:12:37.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.559+0000 7f6503e00640 1 -- 172.21.15.159:0/1697768049 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f0030070 con 0x7f64ec004800 2024-04-04T06:12:37.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.559+0000 7f6503e00640 1 -- 172.21.15.159:0/1697768049 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0005cf0 con 0x7f64ec004800 2024-04-04T06:12:37.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.560+0000 7f650a800640 1 -- 172.21.15.159:0/1697768049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64ec004800 msgr2=0x7f64ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:37.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.560+0000 7f650a800640 1 --2- 172.21.15.159:0/1697768049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64ec004800 0x7f64ec004c80 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f64f0001eb0 tx=0x7f64ec005830 comp rx=0 tx=0).stop 2024-04-04T06:12:37.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.560+0000 7f6509e00640 1 -- 172.21.15.159:0/1697768049 reap_dead start 2024-04-04T06:12:37.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.560+0000 7f650a800640 1 -- 172.21.15.159:0/1697768049 shutdown_connections 2024-04-04T06:12:37.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.560+0000 7f650a800640 1 -- 172.21.15.159:0/1697768049 >> 172.21.15.159:0/1697768049 conn(0x7f65040b1ba0 msgr2=0x7f65040b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:37.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.560+0000 7f650a800640 1 -- 172.21.15.159:0/1697768049 shutdown_connections 2024-04-04T06:12:37.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.560+0000 7f650a800640 1 -- 172.21.15.159:0/1697768049 wait complete. 2024-04-04T06:12:37.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.561+0000 7f650a800640 1 Processor -- start 2024-04-04T06:12:37.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.561+0000 7f650a800640 1 -- start start 2024-04-04T06:12:37.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.562+0000 7f650a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64ec004800 0x7f65040a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:37.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.562+0000 7f650a800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f650413db10 0x7f650413d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:37.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.562+0000 7f650a800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65040aa020 0x7f65040aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:37.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.562+0000 7f650a800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f650413f570 con 0x7f64ec004800 2024-04-04T06:12:37.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.562+0000 7f650a800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6504160c70 con 0x7f650413db10 2024-04-04T06:12:37.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.562+0000 7f650a800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f65040cccf0 con 0x7f65040aa020 2024-04-04T06:12:37.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.563+0000 7f6509400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64ec004800 0x7f65040a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:37.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.563+0000 7f6509400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64ec004800 0x7f65040a9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56636/0 (socket says 172.21.15.159:56636) 2024-04-04T06:12:37.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.563+0000 7f6509400640 1 -- 172.21.15.159:0/2980365227 learned_addr learned my addr 172.21.15.159:0/2980365227 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:37.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.563+0000 7f6509400640 1 -- 172.21.15.159:0/2980365227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65040aa020 msgr2=0x7f65040aa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:37.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.563+0000 7f6509400640 1 --2- 172.21.15.159:0/2980365227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65040aa020 0x7f65040aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:37.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.563+0000 7f6509400640 1 -- 172.21.15.159:0/2980365227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f650413db10 msgr2=0x7f650413d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:37.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.563+0000 7f6509400640 1 --2- 172.21.15.159:0/2980365227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f650413db10 0x7f650413d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:37.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.563+0000 7f6509400640 1 -- 172.21.15.159:0/2980365227 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65040a99c0 con 0x7f64ec004800 2024-04-04T06:12:37.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.563+0000 7f6509400640 1 --2- 172.21.15.159:0/2980365227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64ec004800 0x7f65040a9480 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f64f800d7d0 tx=0x7f64f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:37.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.563+0000 7f6501600640 1 -- 172.21.15.159:0/2980365227 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8005070 con 0x7f64ec004800 2024-04-04T06:12:37.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.564+0000 7f6501600640 1 -- 172.21.15.159:0/2980365227 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f8005230 con 0x7f64ec004800 2024-04-04T06:12:37.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.564+0000 7f6501600640 1 -- 172.21.15.159:0/2980365227 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8007030 con 0x7f64ec004800 2024-04-04T06:12:37.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.564+0000 7f650a800640 1 -- 172.21.15.159:0/2980365227 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f650415a200 con 0x7f64ec004800 2024-04-04T06:12:37.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.564+0000 7f650a800640 1 -- 172.21.15.159:0/2980365227 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65040aefa0 con 0x7f64ec004800 2024-04-04T06:12:37.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.565+0000 7f650a800640 1 -- 172.21.15.159:0/2980365227 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6504075940 con 0x7f64ec004800 2024-04-04T06:12:37.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.566+0000 7f6501600640 1 -- 172.21.15.159:0/2980365227 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64f801a020 con 0x7f64ec004800 2024-04-04T06:12:37.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.567+0000 7f6501600640 1 --2- 172.21.15.159:0/2980365227 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f64e406c920 0x7f64e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:37.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.567+0000 7f6501600640 1 -- 172.21.15.159:0/2980365227 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f64f808b410 con 0x7f64ec004800 2024-04-04T06:12:37.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.567+0000 7f6508a00640 1 --2- 172.21.15.159:0/2980365227 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f64e406c920 0x7f64e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:37.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.568+0000 7f6508a00640 1 --2- 172.21.15.159:0/2980365227 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f64e406c920 0x7f64e406ede0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f64f0034000 tx=0x7f64f0039040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:37.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.571+0000 7f6501600640 1 -- 172.21.15.159:0/2980365227 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f65040a99c0 con 0x7f64ec004800 2024-04-04T06:12:37.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:37.817+0000 7f650a800640 1 -- 172.21.15.159:0/2980365227 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f65040b2870 con 0x7f64ec004800 2024-04-04T06:12:38.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.434+0000 7f6501600640 1 -- 172.21.15.159:0/2980365227 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v61) ==== 118+0+0 (secure 0 0 0) 0x7f64f80597a0 con 0x7f64ec004800 2024-04-04T06:12:38.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.438+0000 7f650a800640 1 -- 172.21.15.159:0/2980365227 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f64e406c920 msgr2=0x7f64e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:38.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.438+0000 7f650a800640 1 --2- 172.21.15.159:0/2980365227 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f64e406c920 0x7f64e406ede0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f64f0034000 tx=0x7f64f0039040 comp rx=0 tx=0).stop 2024-04-04T06:12:38.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.438+0000 7f650a800640 1 -- 172.21.15.159:0/2980365227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64ec004800 msgr2=0x7f65040a9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:38.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.438+0000 7f650a800640 1 --2- 172.21.15.159:0/2980365227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64ec004800 0x7f65040a9480 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f64f800d7d0 tx=0x7f64f800dc90 comp rx=0 tx=0).stop 2024-04-04T06:12:38.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.439+0000 7f650a800640 1 -- 172.21.15.159:0/2980365227 shutdown_connections 2024-04-04T06:12:38.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.439+0000 7f650a800640 1 --2- 172.21.15.159:0/2980365227 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65040aa020 0x7f65040aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:38.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.439+0000 7f650a800640 1 --2- 172.21.15.159:0/2980365227 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f64e406c920 0x7f64e406ede0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:38.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.439+0000 7f650a800640 1 --2- 172.21.15.159:0/2980365227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f650413db10 0x7f650413d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:38.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.439+0000 7f650a800640 1 --2- 172.21.15.159:0/2980365227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64ec004800 0x7f65040a9480 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:38.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.439+0000 7f650a800640 1 -- 172.21.15.159:0/2980365227 >> 172.21.15.159:0/2980365227 conn(0x7f65040b1ba0 msgr2=0x7f650415e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:38.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.439+0000 7f650a800640 1 -- 172.21.15.159:0/2980365227 shutdown_connections 2024-04-04T06:12:38.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.439+0000 7f650a800640 1 -- 172.21.15.159:0/2980365227 wait complete. 2024-04-04T06:12:38.453 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-04T06:12:38.453 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-04T06:12:38.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.661+0000 7fafffe00640 1 Processor -- start 2024-04-04T06:12:38.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.661+0000 7fafffe00640 1 -- start start 2024-04-04T06:12:38.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.663+0000 7fafffe00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7faff815ca70 0x7faff80cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:38.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.663+0000 7fafffe00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faff80c7b40 con 0x7faff80d0370 2024-04-04T06:12:38.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.663+0000 7fafffe00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faff80c7dd0 con 0x7faff815d630 2024-04-04T06:12:38.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.663+0000 7fafffe00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7faff80c80b0 con 0x7faff815ca70 2024-04-04T06:12:38.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.663+0000 7faffde00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7faff80d0370 0x7faff80ce8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58260/0 (socket says 172.21.15.159:58260) 2024-04-04T06:12:38.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.663+0000 7faffde00640 1 -- 172.21.15.159:0/2615797632 learned_addr learned my addr 172.21.15.159:0/2615797632 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:38.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.663+0000 7faffca00640 1 --2- 172.21.15.159:0/2615797632 >> v2:172.21.15.178:3300/0 conn(0x7faff815ca70 0x7faff80cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:38.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.664+0000 7faff7e00640 1 -- 172.21.15.159:0/2615797632 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 99170752 0 0) 0x7faff80c7b40 con 0x7faff80d0370 2024-04-04T06:12:38.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.664+0000 7faff7e00640 1 -- 172.21.15.159:0/2615797632 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafe0003520 con 0x7faff80d0370 2024-04-04T06:12:38.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.664+0000 7faff7e00640 1 -- 172.21.15.159:0/2615797632 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1078336921 0 0) 0x7fafe0003520 con 0x7faff80d0370 2024-04-04T06:12:38.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.664+0000 7faff7e00640 1 -- 172.21.15.159:0/2615797632 >> v2:172.21.15.178:3300/0 conn(0x7faff815ca70 msgr2=0x7faff80cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:38.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.664+0000 7faff7e00640 1 --2- 172.21.15.159:0/2615797632 >> v2:172.21.15.178:3300/0 conn(0x7faff815ca70 0x7faff80cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:38.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.664+0000 7faff7e00640 1 -- 172.21.15.159:0/2615797632 >> v1:172.21.15.178:6789/0 conn(0x7faff815d630 legacy=0x7faff80c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:38.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.664+0000 7faff7e00640 1 -- 172.21.15.159:0/2615797632 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff80c8390 con 0x7faff80d0370 2024-04-04T06:12:38.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.664+0000 7faff7e00640 1 -- 172.21.15.159:0/2615797632 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7faff00019f0 con 0x7faff80d0370 2024-04-04T06:12:38.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.664+0000 7faff7e00640 1 -- 172.21.15.159:0/2615797632 >> v1:172.21.15.159:6789/0 conn(0x7faff80d0370 legacy=0x7faff80ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:38.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.664+0000 7faff7e00640 1 --2- 172.21.15.159:0/2615797632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fafe0003a00 0x7fafe0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:38.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.664+0000 7faff7e00640 1 --2- 172.21.15.159:0/2615797632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafe0004800 0x7fafe0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:38.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.664+0000 7faff7e00640 1 --2- 172.21.15.159:0/2615797632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafe0005210 0x7fafe000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:38.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.664+0000 7faff7e00640 1 -- 172.21.15.159:0/2615797632 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faff80c80b0 con 0x7fafe0005210 2024-04-04T06:12:38.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.664+0000 7faff7e00640 1 -- 172.21.15.159:0/2615797632 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fafe000c160 con 0x7fafe0004800 2024-04-04T06:12:38.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.664+0000 7faff7e00640 1 -- 172.21.15.159:0/2615797632 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fafe000c360 con 0x7fafe0003a00 2024-04-04T06:12:38.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.664+0000 7faffde00640 1 --2- 172.21.15.159:0/2615797632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafe0005210 0x7fafe000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:38.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.665+0000 7faffd400640 1 --2- 172.21.15.159:0/2615797632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fafe0003a00 0x7fafe0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:38.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.665+0000 7faffca00640 1 --2- 172.21.15.159:0/2615797632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafe0004800 0x7fafe0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:38.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.665+0000 7faffde00640 1 -- 172.21.15.159:0/2615797632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fafe0003a00 msgr2=0x7fafe0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:38.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.665+0000 7faffde00640 1 --2- 172.21.15.159:0/2615797632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fafe0003a00 0x7fafe0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:38.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.665+0000 7faffde00640 1 -- 172.21.15.159:0/2615797632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafe0004800 msgr2=0x7fafe0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:38.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.665+0000 7faffde00640 1 --2- 172.21.15.159:0/2615797632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafe0004800 0x7fafe0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:38.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.665+0000 7faffde00640 1 -- 172.21.15.159:0/2615797632 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe0003520 con 0x7fafe0005210 2024-04-04T06:12:38.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.665+0000 7faffd400640 1 --2- 172.21.15.159:0/2615797632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fafe0003a00 0x7fafe0003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:38.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.666+0000 7faffde00640 1 -- 172.21.15.159:0/2615797632 reap_dead start 2024-04-04T06:12:38.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.667+0000 7faffde00640 1 --2- 172.21.15.159:0/2615797632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafe0005210 0x7fafe000b9c0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7faff000a760 tx=0x7fafe0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3be48c934429a40 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:38.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.667+0000 7faff7e00640 1 -- 172.21.15.159:0/2615797632 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff000a3b0 con 0x7fafe0005210 2024-04-04T06:12:38.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.667+0000 7faff7e00640 1 -- 172.21.15.159:0/2615797632 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff0034630 con 0x7fafe0005210 2024-04-04T06:12:38.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.667+0000 7faff7e00640 1 -- 172.21.15.159:0/2615797632 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff0034a20 con 0x7fafe0005210 2024-04-04T06:12:38.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.667+0000 7fafffe00640 1 -- 172.21.15.159:0/2615797632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafe0005210 msgr2=0x7fafe000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:38.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.667+0000 7fafffe00640 1 --2- 172.21.15.159:0/2615797632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafe0005210 0x7fafe000b9c0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7faff000a760 tx=0x7fafe0005830 comp rx=0 tx=0).stop 2024-04-04T06:12:38.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.668+0000 7fafffe00640 1 -- 172.21.15.159:0/2615797632 shutdown_connections 2024-04-04T06:12:38.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.668+0000 7fafffe00640 1 --2- 172.21.15.159:0/2615797632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafe0005210 0x7fafe000b9c0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:38.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.668+0000 7fafffe00640 1 -- 172.21.15.159:0/2615797632 >> 172.21.15.159:0/2615797632 conn(0x7faff80b3e90 msgr2=0x7faff80b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:38.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.668+0000 7fafffe00640 1 -- 172.21.15.159:0/2615797632 shutdown_connections 2024-04-04T06:12:38.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.668+0000 7fafffe00640 1 -- 172.21.15.159:0/2615797632 wait complete. 2024-04-04T06:12:38.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.669+0000 7fafffe00640 1 Processor -- start 2024-04-04T06:12:38.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.670+0000 7fafffe00640 1 -- start start 2024-04-04T06:12:38.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.670+0000 7fafffe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafe0005210 0x7faff814edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:38.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.670+0000 7fafffe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faff815b3f0 0x7faff815b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:38.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.670+0000 7fafffe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faff80cea20 0x7faff80cee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:38.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.670+0000 7fafffe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faff80cffb0 con 0x7fafe0005210 2024-04-04T06:12:38.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.670+0000 7fafffe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7faff80cfe30 con 0x7faff815b3f0 2024-04-04T06:12:38.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.670+0000 7fafffe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7faff80d0130 con 0x7faff80cea20 2024-04-04T06:12:38.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.671+0000 7faffd400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafe0005210 0x7faff814edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:38.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.671+0000 7faffd400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafe0005210 0x7faff814edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56650/0 (socket says 172.21.15.159:56650) 2024-04-04T06:12:38.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.671+0000 7faffd400640 1 -- 172.21.15.159:0/2792938621 learned_addr learned my addr 172.21.15.159:0/2792938621 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:38.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.671+0000 7faffde00640 1 --2- 172.21.15.159:0/2792938621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faff80cea20 0x7faff80cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:38.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.671+0000 7faffca00640 1 --2- 172.21.15.159:0/2792938621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faff815b3f0 0x7faff815b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:38.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.671+0000 7faffd400640 1 -- 172.21.15.159:0/2792938621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faff80cea20 msgr2=0x7faff80cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:38.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.671+0000 7faffd400640 1 --2- 172.21.15.159:0/2792938621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faff80cea20 0x7faff80cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:38.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.671+0000 7faffd400640 1 -- 172.21.15.159:0/2792938621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faff815b3f0 msgr2=0x7faff815b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:38.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.671+0000 7faffd400640 1 --2- 172.21.15.159:0/2792938621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faff815b3f0 0x7faff815b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:38.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.671+0000 7faffd400640 1 -- 172.21.15.159:0/2792938621 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff814f300 con 0x7fafe0005210 2024-04-04T06:12:38.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.672+0000 7faffd400640 1 --2- 172.21.15.159:0/2792938621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafe0005210 0x7faff814edc0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fafec00d590 tx=0x7fafec00da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:38.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.672+0000 7faff5600640 1 -- 172.21.15.159:0/2792938621 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec005480 con 0x7fafe0005210 2024-04-04T06:12:38.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.672+0000 7faff5600640 1 -- 172.21.15.159:0/2792938621 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafec017070 con 0x7fafe0005210 2024-04-04T06:12:38.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.672+0000 7fafffe00640 1 -- 172.21.15.159:0/2792938621 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff80cf750 con 0x7fafe0005210 2024-04-04T06:12:38.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.672+0000 7faff5600640 1 -- 172.21.15.159:0/2792938621 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec003810 con 0x7fafe0005210 2024-04-04T06:12:38.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.672+0000 7fafffe00640 1 -- 172.21.15.159:0/2792938621 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff81f4c90 con 0x7fafe0005210 2024-04-04T06:12:38.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.673+0000 7fafffe00640 1 -- 172.21.15.159:0/2792938621 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff8068cd0 con 0x7fafe0005210 2024-04-04T06:12:38.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.674+0000 7faff5600640 1 -- 172.21.15.159:0/2792938621 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fafec0039b0 con 0x7fafe0005210 2024-04-04T06:12:38.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.675+0000 7faff5600640 1 --2- 172.21.15.159:0/2792938621 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fafd806c8f0 0x7fafd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:38.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.675+0000 7faff5600640 1 -- 172.21.15.159:0/2792938621 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fafec08b980 con 0x7fafe0005210 2024-04-04T06:12:38.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.676+0000 7faffca00640 1 --2- 172.21.15.159:0/2792938621 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fafd806c8f0 0x7fafd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:38.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.677+0000 7faffca00640 1 --2- 172.21.15.159:0/2792938621 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fafd806c8f0 0x7fafd806edb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7faff80ca960 tx=0x7fafe4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:38.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.680+0000 7faff5600640 1 -- 172.21.15.159:0/2792938621 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7faff814f300 con 0x7fafe0005210 2024-04-04T06:12:38.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:38.920+0000 7fafffe00640 1 -- 172.21.15.159:0/2792938621 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7faff80b4d60 con 0x7fafe0005210 2024-04-04T06:12:39.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.437+0000 7faff5600640 1 -- 172.21.15.159:0/2792938621 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v62) ==== 122+0+0 (secure 0 0 0) 0x7fafec019310 con 0x7fafe0005210 2024-04-04T06:12:39.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.440+0000 7fafffe00640 1 -- 172.21.15.159:0/2792938621 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fafd806c8f0 msgr2=0x7fafd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:39.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.440+0000 7fafffe00640 1 --2- 172.21.15.159:0/2792938621 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fafd806c8f0 0x7fafd806edb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7faff80ca960 tx=0x7fafe4008040 comp rx=0 tx=0).stop 2024-04-04T06:12:39.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.440+0000 7fafffe00640 1 -- 172.21.15.159:0/2792938621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafe0005210 msgr2=0x7faff814edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:39.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.440+0000 7fafffe00640 1 --2- 172.21.15.159:0/2792938621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafe0005210 0x7faff814edc0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fafec00d590 tx=0x7fafec00da50 comp rx=0 tx=0).stop 2024-04-04T06:12:39.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.441+0000 7fafffe00640 1 -- 172.21.15.159:0/2792938621 shutdown_connections 2024-04-04T06:12:39.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.441+0000 7fafffe00640 1 --2- 172.21.15.159:0/2792938621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faff80cea20 0x7faff80cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:39.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.441+0000 7fafffe00640 1 --2- 172.21.15.159:0/2792938621 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fafd806c8f0 0x7fafd806edb0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:39.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.441+0000 7fafffe00640 1 --2- 172.21.15.159:0/2792938621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faff815b3f0 0x7faff815b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:39.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.441+0000 7fafffe00640 1 --2- 172.21.15.159:0/2792938621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafe0005210 0x7faff814edc0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:39.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.441+0000 7fafffe00640 1 -- 172.21.15.159:0/2792938621 >> 172.21.15.159:0/2792938621 conn(0x7faff80b3e90 msgr2=0x7faff814f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:39.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.441+0000 7fafffe00640 1 -- 172.21.15.159:0/2792938621 shutdown_connections 2024-04-04T06:12:39.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.441+0000 7fafffe00640 1 -- 172.21.15.159:0/2792938621 wait complete. 2024-04-04T06:12:39.465 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-04T06:12:39.465 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:12:39.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.674+0000 7f51da000640 1 Processor -- start 2024-04-04T06:12:39.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.674+0000 7f51da000640 1 -- start start 2024-04-04T06:12:39.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51da000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f51d4141940 0x7f51d406fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:39.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51da000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d406ffb0 0x7f51d4161fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:39.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51da000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51d40b5f70 con 0x7f51d4141460 2024-04-04T06:12:39.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51da000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f51d415eac0 con 0x7f51d4141940 2024-04-04T06:12:39.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51da000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f51d415ed80 con 0x7f51d406ffb0 2024-04-04T06:12:39.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d8c00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f51d4141460 0x7f51d40b5860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58268/0 (socket says 172.21.15.159:58268) 2024-04-04T06:12:39.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d8c00640 1 -- 172.21.15.159:0/256020102 learned_addr learned my addr 172.21.15.159:0/256020102 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:39.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d3e00640 1 --2- 172.21.15.159:0/256020102 >> v2:172.21.15.178:3300/0 conn(0x7f51d4141940 0x7f51d406fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:39.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d3400640 1 -- 172.21.15.159:0/256020102 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 190587990 0 0) 0x7f51c8001110 con 0x7f51d4141460 2024-04-04T06:12:39.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d3400640 1 -- 172.21.15.159:0/256020102 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51bc003520 con 0x7f51d4141460 2024-04-04T06:12:39.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d9600640 1 --2- 172.21.15.159:0/256020102 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d406ffb0 0x7f51d4161fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:39.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d3400640 1 -- 172.21.15.159:0/256020102 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3884842624 0 0) 0x7f51bc003520 con 0x7f51d4141460 2024-04-04T06:12:39.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d3400640 1 -- 172.21.15.159:0/256020102 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d406ffb0 msgr2=0x7f51d4161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:39.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d3400640 1 --2- 172.21.15.159:0/256020102 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d406ffb0 0x7f51d4161fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:39.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d3400640 1 -- 172.21.15.159:0/256020102 >> v2:172.21.15.178:3300/0 conn(0x7f51d4141940 msgr2=0x7f51d406fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:39.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d3400640 1 --2- 172.21.15.159:0/256020102 >> v2:172.21.15.178:3300/0 conn(0x7f51d4141940 0x7f51d406fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:39.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d3400640 1 -- 172.21.15.159:0/256020102 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d415f060 con 0x7f51d4141460 2024-04-04T06:12:39.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d3400640 1 -- 172.21.15.159:0/256020102 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f51c8001840 con 0x7f51d4141460 2024-04-04T06:12:39.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d3400640 1 -- 172.21.15.159:0/256020102 >> v1:172.21.15.159:6789/0 conn(0x7f51d4141460 legacy=0x7f51d40b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:39.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d3400640 1 --2- 172.21.15.159:0/256020102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51bc0039b0 0x7f51bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:39.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d3400640 1 --2- 172.21.15.159:0/256020102 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51bc004800 0x7f51bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:39.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d3400640 1 --2- 172.21.15.159:0/256020102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51bc0051c0 0x7f51bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:39.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d3400640 1 -- 172.21.15.159:0/256020102 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f51d415eac0 con 0x7f51bc0039b0 2024-04-04T06:12:39.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d3400640 1 -- 172.21.15.159:0/256020102 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f51d415ed80 con 0x7f51bc0051c0 2024-04-04T06:12:39.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d3400640 1 -- 172.21.15.159:0/256020102 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f51bc00bf80 con 0x7f51bc004800 2024-04-04T06:12:39.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d9600640 1 --2- 172.21.15.159:0/256020102 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d406ffb0 0x7f51d4161fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:12:39.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.676+0000 7f51d8c00640 1 --2- 172.21.15.159:0/256020102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51bc0039b0 0x7f51bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:39.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.677+0000 7f51d8c00640 1 -- 172.21.15.159:0/256020102 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51bc004800 msgr2=0x7f51bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:39.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.677+0000 7f51d8c00640 1 --2- 172.21.15.159:0/256020102 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51bc004800 0x7f51bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:39.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.677+0000 7f51d8c00640 1 -- 172.21.15.159:0/256020102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51bc0051c0 msgr2=0x7f51bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:39.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.677+0000 7f51d8c00640 1 --2- 172.21.15.159:0/256020102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51bc0051c0 0x7f51bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:39.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.677+0000 7f51d8c00640 1 -- 172.21.15.159:0/256020102 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51bc003520 con 0x7f51bc0039b0 2024-04-04T06:12:39.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.679+0000 7f51d8c00640 1 --2- 172.21.15.159:0/256020102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51bc0039b0 0x7f51bc003dd0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f51c800c320 tx=0x7f51bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516fef4f51d2359d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:39.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.679+0000 7f51d3400640 1 -- 172.21.15.159:0/256020102 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c802cc40 con 0x7f51bc0039b0 2024-04-04T06:12:39.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.679+0000 7f51d3400640 1 -- 172.21.15.159:0/256020102 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c8030070 con 0x7f51bc0039b0 2024-04-04T06:12:39.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.679+0000 7f51d3400640 1 -- 172.21.15.159:0/256020102 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c80058d0 con 0x7f51bc0039b0 2024-04-04T06:12:39.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.680+0000 7f51da000640 1 -- 172.21.15.159:0/256020102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51bc0039b0 msgr2=0x7f51bc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:39.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.680+0000 7f51da000640 1 --2- 172.21.15.159:0/256020102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51bc0039b0 0x7f51bc003dd0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f51c800c320 tx=0x7f51bc005830 comp rx=0 tx=0).stop 2024-04-04T06:12:39.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.680+0000 7f51d9600640 1 -- 172.21.15.159:0/256020102 reap_dead start 2024-04-04T06:12:39.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.680+0000 7f51da000640 1 -- 172.21.15.159:0/256020102 shutdown_connections 2024-04-04T06:12:39.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.680+0000 7f51da000640 1 -- 172.21.15.159:0/256020102 >> 172.21.15.159:0/256020102 conn(0x7f51d4067460 msgr2=0x7f51d4067840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:39.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.680+0000 7f51da000640 1 -- 172.21.15.159:0/256020102 shutdown_connections 2024-04-04T06:12:39.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.680+0000 7f51da000640 1 -- 172.21.15.159:0/256020102 wait complete. 2024-04-04T06:12:39.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.683+0000 7f51da000640 1 Processor -- start 2024-04-04T06:12:39.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.683+0000 7f51da000640 1 -- start start 2024-04-04T06:12:39.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.684+0000 7f51da000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51bc0039b0 0x7f51d413d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:39.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.684+0000 7f51da000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51d40d01d0 0x7f51d40d0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:39.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.684+0000 7f51da000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d40cf280 0x7f51d40cf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:39.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.684+0000 7f51da000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f51d40b53b0 con 0x7f51d40d01d0 2024-04-04T06:12:39.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.684+0000 7f51da000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f51d415f620 con 0x7f51bc0039b0 2024-04-04T06:12:39.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.684+0000 7f51da000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f51d40b5530 con 0x7f51d40cf280 2024-04-04T06:12:39.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.684+0000 7f51d3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51d40d01d0 0x7f51d40d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:39.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.684+0000 7f51d3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51d40d01d0 0x7f51d40d0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56666/0 (socket says 172.21.15.159:56666) 2024-04-04T06:12:39.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.684+0000 7f51d3e00640 1 -- 172.21.15.159:0/2419104553 learned_addr learned my addr 172.21.15.159:0/2419104553 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:39.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.684+0000 7f51d8c00640 1 --2- 172.21.15.159:0/2419104553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51bc0039b0 0x7f51d413d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:39.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.684+0000 7f51d9600640 1 --2- 172.21.15.159:0/2419104553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d40cf280 0x7f51d40cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:39.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.684+0000 7f51d3e00640 1 -- 172.21.15.159:0/2419104553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d40cf280 msgr2=0x7f51d40cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:39.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.684+0000 7f51d3e00640 1 --2- 172.21.15.159:0/2419104553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d40cf280 0x7f51d40cf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:39.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.684+0000 7f51d3e00640 1 -- 172.21.15.159:0/2419104553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51bc0039b0 msgr2=0x7f51d413d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:39.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.684+0000 7f51d3e00640 1 --2- 172.21.15.159:0/2419104553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51bc0039b0 0x7f51d413d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:39.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.684+0000 7f51d3e00640 1 -- 172.21.15.159:0/2419104553 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d413db90 con 0x7f51d40d01d0 2024-04-04T06:12:39.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.684+0000 7f51d9600640 1 --2- 172.21.15.159:0/2419104553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d40cf280 0x7f51d40cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:39.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.686+0000 7f51d3e00640 1 --2- 172.21.15.159:0/2419104553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51d40d01d0 0x7f51d40d0650 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f51c000e860 tx=0x7f51c000ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:39.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.686+0000 7f51d0c00640 1 -- 172.21.15.159:0/2419104553 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0017070 con 0x7f51d40d01d0 2024-04-04T06:12:39.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.686+0000 7f51d0c00640 1 -- 172.21.15.159:0/2419104553 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c000c940 con 0x7f51d40d01d0 2024-04-04T06:12:39.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.686+0000 7f51da000640 1 -- 172.21.15.159:0/2419104553 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d40cb6e0 con 0x7f51d40d01d0 2024-04-04T06:12:39.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.686+0000 7f51d0c00640 1 -- 172.21.15.159:0/2419104553 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c000cd10 con 0x7f51d40d01d0 2024-04-04T06:12:39.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.686+0000 7f51da000640 1 -- 172.21.15.159:0/2419104553 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d40cbca0 con 0x7f51d40d01d0 2024-04-04T06:12:39.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.688+0000 7f51da000640 1 -- 172.21.15.159:0/2419104553 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51d4075940 con 0x7f51d40d01d0 2024-04-04T06:12:39.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.689+0000 7f51d0c00640 1 -- 172.21.15.159:0/2419104553 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51c001a070 con 0x7f51d40d01d0 2024-04-04T06:12:39.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.689+0000 7f51d0c00640 1 --2- 172.21.15.159:0/2419104553 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f51b406c920 0x7f51b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:39.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.690+0000 7f51d0c00640 1 -- 172.21.15.159:0/2419104553 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f51c008ba20 con 0x7f51d40d01d0 2024-04-04T06:12:39.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.690+0000 7f51d8c00640 1 --2- 172.21.15.159:0/2419104553 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f51b406c920 0x7f51b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:39.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.691+0000 7f51d8c00640 1 --2- 172.21.15.159:0/2419104553 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f51b406c920 0x7f51b406ede0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f51c800c320 tx=0x7f51c8007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:39.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.698+0000 7f51d0c00640 1 -- 172.21.15.159:0/2419104553 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f51d413db90 con 0x7f51d40d01d0 2024-04-04T06:12:39.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.966+0000 7f51da000640 1 -- 172.21.15.159:0/2419104553 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51d4071d40 con 0x7f51d40d01d0 2024-04-04T06:12:39.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.967+0000 7f51d0c00640 1 -- 172.21.15.159:0/2419104553 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5195 (secure 0 0 0) 0x7f51c0059e50 con 0x7f51d40d01d0 2024-04-04T06:12:39.960 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:39.960 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1825651864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1825651864},{"type":"v1","addr":"172.21.15.178:6836","nonce":1825651864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1824578712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1824578712},{"type":"v1","addr":"172.21.15.178:6837","nonce":1824578712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6834/3377498298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3377498298},{"type":"v1","addr":"172.21.15.159:6834","nonce":3377498298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:12:35.880454+0000","modified":"2024-04-04T06:12:38.922420+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7293":{"gid":7293,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6835/3484779950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":3484779950},{"type":"v1","addr":"172.21.15.159:6835","nonce":3484779950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7293,"qdb_cluster":[7293]},"id":6}]} 2024-04-04T06:12:39.960 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 62 2024-04-04T06:12:39.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.971+0000 7f51da000640 1 -- 172.21.15.159:0/2419104553 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f51b406c920 msgr2=0x7f51b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:39.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.971+0000 7f51da000640 1 --2- 172.21.15.159:0/2419104553 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f51b406c920 0x7f51b406ede0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f51c800c320 tx=0x7f51c8007000 comp rx=0 tx=0).stop 2024-04-04T06:12:39.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.971+0000 7f51da000640 1 -- 172.21.15.159:0/2419104553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51d40d01d0 msgr2=0x7f51d40d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:39.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.971+0000 7f51da000640 1 --2- 172.21.15.159:0/2419104553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51d40d01d0 0x7f51d40d0650 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f51c000e860 tx=0x7f51c000ed20 comp rx=0 tx=0).stop 2024-04-04T06:12:39.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.971+0000 7f51da000640 1 -- 172.21.15.159:0/2419104553 shutdown_connections 2024-04-04T06:12:39.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.971+0000 7f51da000640 1 --2- 172.21.15.159:0/2419104553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51d40cf280 0x7f51d40cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:39.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.971+0000 7f51da000640 1 --2- 172.21.15.159:0/2419104553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51d40d01d0 0x7f51d40d0650 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:39.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.971+0000 7f51da000640 1 --2- 172.21.15.159:0/2419104553 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f51b406c920 0x7f51b406ede0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:39.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.971+0000 7f51da000640 1 --2- 172.21.15.159:0/2419104553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51bc0039b0 0x7f51d413d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:39.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.971+0000 7f51da000640 1 -- 172.21.15.159:0/2419104553 >> 172.21.15.159:0/2419104553 conn(0x7f51d4067460 msgr2=0x7f51d406c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:39.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.971+0000 7f51da000640 1 -- 172.21.15.159:0/2419104553 shutdown_connections 2024-04-04T06:12:39.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:39.971+0000 7f51da000640 1 -- 172.21.15.159:0/2419104553 wait complete. 2024-04-04T06:12:39.982 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:12:40.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.188+0000 7f0627200640 1 Processor -- start 2024-04-04T06:12:40.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.188+0000 7f0627200640 1 -- start start 2024-04-04T06:12:40.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.190+0000 7f0627200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f062015c390 0x7f0620073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:40.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.190+0000 7f0627200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06200af830 0x7f0620160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:40.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.190+0000 7f0627200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f062015fe80 con 0x7f0620160b80 2024-04-04T06:12:40.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.190+0000 7f0627200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f062015c770 con 0x7f062015c390 2024-04-04T06:12:40.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.190+0000 7f0627200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f06200ad120 con 0x7f06200af830 2024-04-04T06:12:40.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.190+0000 7f061fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f062015c390 0x7f0620073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:40.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.190+0000 7f061fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f062015c390 0x7f0620073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56674/0 (socket says 172.21.15.159:56674) 2024-04-04T06:12:40.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.190+0000 7f0625200640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f0620160b80 0x7f06200d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58284/0 (socket says 172.21.15.159:58284) 2024-04-04T06:12:40.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.190+0000 7f061fe00640 1 -- 172.21.15.159:0/2707895174 learned_addr learned my addr 172.21.15.159:0/2707895174 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:40.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.191+0000 7f061f400640 1 --2- 172.21.15.159:0/2707895174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06200af830 0x7f0620160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:40.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.191+0000 7f061fe00640 1 -- 172.21.15.159:0/2707895174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06200af830 msgr2=0x7f0620160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:40.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.191+0000 7f061fe00640 1 --2- 172.21.15.159:0/2707895174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06200af830 0x7f0620160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:40.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.191+0000 7f061fe00640 1 -- 172.21.15.159:0/2707895174 >> v1:172.21.15.159:6789/0 conn(0x7f0620160b80 legacy=0x7f06200d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:40.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.191+0000 7f061fe00640 1 -- 172.21.15.159:0/2707895174 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06200ad400 con 0x7f062015c390 2024-04-04T06:12:40.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.191+0000 7f061f400640 1 --2- 172.21.15.159:0/2707895174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06200af830 0x7f0620160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:40.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.192+0000 7f061fe00640 1 --2- 172.21.15.159:0/2707895174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f062015c390 0x7f0620073790 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f0610009870 tx=0x7f061002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56295d4d92b4ffe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:40.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.192+0000 7f061ea00640 1 -- 172.21.15.159:0/2707895174 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610034070 con 0x7f062015c390 2024-04-04T06:12:40.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.192+0000 7f061ea00640 1 -- 172.21.15.159:0/2707895174 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0610030840 con 0x7f062015c390 2024-04-04T06:12:40.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.192+0000 7f061ea00640 1 -- 172.21.15.159:0/2707895174 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610030c10 con 0x7f062015c390 2024-04-04T06:12:40.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.193+0000 7f0627200640 1 -- 172.21.15.159:0/2707895174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f062015c390 msgr2=0x7f0620073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:40.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.193+0000 7f0627200640 1 --2- 172.21.15.159:0/2707895174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f062015c390 0x7f0620073790 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f0610009870 tx=0x7f061002b980 comp rx=0 tx=0).stop 2024-04-04T06:12:40.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.193+0000 7f0627200640 1 -- 172.21.15.159:0/2707895174 shutdown_connections 2024-04-04T06:12:40.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.193+0000 7f0627200640 1 --2- 172.21.15.159:0/2707895174 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06200af830 0x7f0620160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:40.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.193+0000 7f0627200640 1 --2- 172.21.15.159:0/2707895174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f062015c390 0x7f0620073790 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:40.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.193+0000 7f0627200640 1 -- 172.21.15.159:0/2707895174 >> 172.21.15.159:0/2707895174 conn(0x7f062006b990 msgr2=0x7f062006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:40.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.193+0000 7f0627200640 1 -- 172.21.15.159:0/2707895174 shutdown_connections 2024-04-04T06:12:40.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.193+0000 7f0627200640 1 -- 172.21.15.159:0/2707895174 wait complete. 2024-04-04T06:12:40.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.195+0000 7f0627200640 1 Processor -- start 2024-04-04T06:12:40.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.195+0000 7f0627200640 1 -- start start 2024-04-04T06:12:40.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.196+0000 7f0627200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06200af830 0x7f062014cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:40.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.196+0000 7f0627200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f062015c390 0x7f06200b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:40.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.196+0000 7f0627200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0620160b80 0x7f06200b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:40.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.196+0000 7f0627200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f06200cf980 con 0x7f0620160b80 2024-04-04T06:12:40.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.196+0000 7f0627200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0620160120 con 0x7f06200af830 2024-04-04T06:12:40.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.196+0000 7f0627200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f06200d0d60 con 0x7f062015c390 2024-04-04T06:12:40.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.196+0000 7f0625200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0620160b80 0x7f06200b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:40.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.196+0000 7f0625200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0620160b80 0x7f06200b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56684/0 (socket says 172.21.15.159:56684) 2024-04-04T06:12:40.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.196+0000 7f0625200640 1 -- 172.21.15.159:0/3002663766 learned_addr learned my addr 172.21.15.159:0/3002663766 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:40.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.197+0000 7f061fe00640 1 --2- 172.21.15.159:0/3002663766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06200af830 0x7f062014cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:40.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.197+0000 7f061f400640 1 --2- 172.21.15.159:0/3002663766 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f062015c390 0x7f06200b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:40.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.197+0000 7f0625200640 1 -- 172.21.15.159:0/3002663766 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f062015c390 msgr2=0x7f06200b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:40.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.197+0000 7f0625200640 1 --2- 172.21.15.159:0/3002663766 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f062015c390 0x7f06200b1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:40.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.197+0000 7f0625200640 1 -- 172.21.15.159:0/3002663766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06200af830 msgr2=0x7f062014cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:40.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.197+0000 7f0625200640 1 --2- 172.21.15.159:0/3002663766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06200af830 0x7f062014cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:40.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.197+0000 7f0625200640 1 -- 172.21.15.159:0/3002663766 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06200b4fe0 con 0x7f0620160b80 2024-04-04T06:12:40.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.197+0000 7f061fe00640 1 --2- 172.21.15.159:0/3002663766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06200af830 0x7f062014cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:40.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.197+0000 7f0625200640 1 --2- 172.21.15.159:0/3002663766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0620160b80 0x7f06200b1640 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f061800c6f0 tx=0x7f061800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:40.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.197+0000 7f0617e00640 1 -- 172.21.15.159:0/3002663766 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618017070 con 0x7f0620160b80 2024-04-04T06:12:40.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.197+0000 7f0617e00640 1 -- 172.21.15.159:0/3002663766 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0618005da0 con 0x7f0620160b80 2024-04-04T06:12:40.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.197+0000 7f0617e00640 1 -- 172.21.15.159:0/3002663766 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061800a840 con 0x7f0620160b80 2024-04-04T06:12:40.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.197+0000 7f0627200640 1 -- 172.21.15.159:0/3002663766 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06200ceb50 con 0x7f0620160b80 2024-04-04T06:12:40.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.197+0000 7f0627200640 1 -- 172.21.15.159:0/3002663766 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f062014cff0 con 0x7f0620160b80 2024-04-04T06:12:40.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.198+0000 7f0627200640 1 -- 172.21.15.159:0/3002663766 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f062006fa10 con 0x7f0620160b80 2024-04-04T06:12:40.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.199+0000 7f0617e00640 1 -- 172.21.15.159:0/3002663766 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f061801a070 con 0x7f0620160b80 2024-04-04T06:12:40.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.201+0000 7f0617e00640 1 --2- 172.21.15.159:0/3002663766 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f060006c920 0x7f060006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:40.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.201+0000 7f0617e00640 1 -- 172.21.15.159:0/3002663766 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f061808b090 con 0x7f0620160b80 2024-04-04T06:12:40.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.201+0000 7f061fe00640 1 --2- 172.21.15.159:0/3002663766 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f060006c920 0x7f060006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:40.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.202+0000 7f061fe00640 1 --2- 172.21.15.159:0/3002663766 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f060006c920 0x7f060006ede0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0610036000 tx=0x7f061003b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:40.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.204+0000 7f0617e00640 1 -- 172.21.15.159:0/3002663766 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f062006fa10 con 0x7f0620160b80 2024-04-04T06:12:40.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.401+0000 7f0627200640 1 -- 172.21.15.159:0/3002663766 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06200685c0 con 0x7f0620160b80 2024-04-04T06:12:40.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.404+0000 7f0617e00640 1 -- 172.21.15.159:0/3002663766 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16672 (secure 0 0 0) 0x7f0618058dd0 con 0x7f0620160b80 2024-04-04T06:12:40.396 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:40.396 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":85,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:12:37.412279+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:12:33.658405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-04T06:12:34.856266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:12:40.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.406+0000 7f0627200640 1 -- 172.21.15.159:0/3002663766 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f060006c920 msgr2=0x7f060006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:40.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.406+0000 7f0627200640 1 --2- 172.21.15.159:0/3002663766 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f060006c920 0x7f060006ede0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0610036000 tx=0x7f061003b040 comp rx=0 tx=0).stop 2024-04-04T06:12:40.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.406+0000 7f0627200640 1 -- 172.21.15.159:0/3002663766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0620160b80 msgr2=0x7f06200b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:40.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.406+0000 7f0627200640 1 --2- 172.21.15.159:0/3002663766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0620160b80 0x7f06200b1640 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f061800c6f0 tx=0x7f061800cbb0 comp rx=0 tx=0).stop 2024-04-04T06:12:40.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.406+0000 7f0627200640 1 -- 172.21.15.159:0/3002663766 shutdown_connections 2024-04-04T06:12:40.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.406+0000 7f0627200640 1 --2- 172.21.15.159:0/3002663766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0620160b80 0x7f06200b1640 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:40.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.406+0000 7f0627200640 1 --2- 172.21.15.159:0/3002663766 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f062015c390 0x7f06200b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:40.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.406+0000 7f0627200640 1 --2- 172.21.15.159:0/3002663766 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f060006c920 0x7f060006ede0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:40.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.406+0000 7f0627200640 1 --2- 172.21.15.159:0/3002663766 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06200af830 0x7f062014cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:40.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.406+0000 7f0627200640 1 -- 172.21.15.159:0/3002663766 >> 172.21.15.159:0/3002663766 conn(0x7f062006b990 msgr2=0x7f06200afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:40.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.407+0000 7f0627200640 1 -- 172.21.15.159:0/3002663766 shutdown_connections 2024-04-04T06:12:40.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.407+0000 7f0627200640 1 -- 172.21.15.159:0/3002663766 wait complete. 2024-04-04T06:12:40.419 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-04T06:12:40.419 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:12:40.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.625+0000 7f0175800640 1 Processor -- start 2024-04-04T06:12:40.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.625+0000 7f0175800640 1 -- start start 2024-04-04T06:12:40.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.626+0000 7f0175800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f01700d2310 0x7f01700ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:40.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.626+0000 7f0175800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f01700cf530 0x7f01700b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:40.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.626+0000 7f0175800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f017015e880 con 0x7f017015e4a0 2024-04-04T06:12:40.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.626+0000 7f0175800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f01700cf9a0 con 0x7f01700cf530 2024-04-04T06:12:40.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.626+0000 7f0175800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f01700d34e0 con 0x7f01700d2310 2024-04-04T06:12:40.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.627+0000 7f0174e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f01700cf530 0x7f01700b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:40.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.627+0000 7f0174e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f01700cf530 0x7f01700b73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56690/0 (socket says 172.21.15.159:56690) 2024-04-04T06:12:40.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.627+0000 7f0174e00640 1 -- 172.21.15.159:0/2306097967 learned_addr learned my addr 172.21.15.159:0/2306097967 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:40.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.627+0000 7f0167400640 1 --2- 172.21.15.159:0/2306097967 >> v2:172.21.15.178:3300/0 conn(0x7f01700d2310 0x7f01700ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:40.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.627+0000 7f0174e00640 1 -- 172.21.15.159:0/2306097967 >> v2:172.21.15.178:3300/0 conn(0x7f01700d2310 msgr2=0x7f01700ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:40.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.627+0000 7f0174e00640 1 --2- 172.21.15.159:0/2306097967 >> v2:172.21.15.178:3300/0 conn(0x7f01700d2310 0x7f01700ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:40.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.627+0000 7f0174e00640 1 -- 172.21.15.159:0/2306097967 >> v1:172.21.15.178:6789/0 conn(0x7f017015e4a0 legacy=0x7f01700b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:40.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.627+0000 7f0174e00640 1 -- 172.21.15.159:0/2306097967 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01700d28d0 con 0x7f01700cf530 2024-04-04T06:12:40.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.627+0000 7f0167400640 1 --2- 172.21.15.159:0/2306097967 >> v2:172.21.15.178:3300/0 conn(0x7f01700d2310 0x7f01700ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:40.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.628+0000 7f0174e00640 1 --2- 172.21.15.159:0/2306097967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01700cf530 0x7f01700b73e0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f01700d3660 tx=0x7f016802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b6d62d6276abb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:40.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.629+0000 7f016f400640 1 -- 172.21.15.159:0/2306097967 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016802e070 con 0x7f01700cf530 2024-04-04T06:12:40.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.629+0000 7f016f400640 1 -- 172.21.15.159:0/2306097967 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0168037070 con 0x7f01700cf530 2024-04-04T06:12:40.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.629+0000 7f016f400640 1 -- 172.21.15.159:0/2306097967 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0168031a00 con 0x7f01700cf530 2024-04-04T06:12:40.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.629+0000 7f0175800640 1 -- 172.21.15.159:0/2306097967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01700cf530 msgr2=0x7f01700b73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:40.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.629+0000 7f0175800640 1 --2- 172.21.15.159:0/2306097967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01700cf530 0x7f01700b73e0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f01700d3660 tx=0x7f016802b880 comp rx=0 tx=0).stop 2024-04-04T06:12:40.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.629+0000 7f0175800640 1 -- 172.21.15.159:0/2306097967 shutdown_connections 2024-04-04T06:12:40.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.629+0000 7f0175800640 1 --2- 172.21.15.159:0/2306097967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01700cf530 0x7f01700b73e0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:40.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.629+0000 7f0175800640 1 --2- 172.21.15.159:0/2306097967 >> v2:172.21.15.178:3300/0 conn(0x7f01700d2310 0x7f01700ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:40.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.629+0000 7f0175800640 1 -- 172.21.15.159:0/2306097967 >> 172.21.15.159:0/2306097967 conn(0x7f01700676d0 msgr2=0x7f0170067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:40.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.630+0000 7f0175800640 1 -- 172.21.15.159:0/2306097967 shutdown_connections 2024-04-04T06:12:40.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.630+0000 7f0175800640 1 -- 172.21.15.159:0/2306097967 wait complete. 2024-04-04T06:12:40.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.631+0000 7f0175800640 1 Processor -- start 2024-04-04T06:12:40.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.631+0000 7f0175800640 1 -- start start 2024-04-04T06:12:40.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.632+0000 7f0175800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01700cf530 0x7f01701f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:40.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.632+0000 7f0175800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01700d2310 0x7f01701530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:40.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.632+0000 7f0175800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f017015e4a0 0x7f0170153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:40.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.632+0000 7f0175800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f01700d3170 con 0x7f01700cf530 2024-04-04T06:12:40.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.632+0000 7f0175800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f01700d2e90 con 0x7f017015e4a0 2024-04-04T06:12:40.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.632+0000 7f0175800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f01700d3970 con 0x7f01700d2310 2024-04-04T06:12:40.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.632+0000 7f016fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01700cf530 0x7f01701f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:40.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.633+0000 7f016fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01700cf530 0x7f01701f4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56692/0 (socket says 172.21.15.159:56692) 2024-04-04T06:12:40.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.633+0000 7f016fe00640 1 -- 172.21.15.159:0/2179428135 learned_addr learned my addr 172.21.15.159:0/2179428135 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:40.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.633+0000 7f0167400640 1 --2- 172.21.15.159:0/2179428135 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01700d2310 0x7f01701530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:40.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.633+0000 7f0174e00640 1 --2- 172.21.15.159:0/2179428135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f017015e4a0 0x7f0170153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:40.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.633+0000 7f016fe00640 1 -- 172.21.15.159:0/2179428135 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01700d2310 msgr2=0x7f01701530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:40.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.633+0000 7f016fe00640 1 --2- 172.21.15.159:0/2179428135 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01700d2310 0x7f01701530f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:40.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.633+0000 7f016fe00640 1 -- 172.21.15.159:0/2179428135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f017015e4a0 msgr2=0x7f0170153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:40.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.633+0000 7f016fe00640 1 --2- 172.21.15.159:0/2179428135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f017015e4a0 0x7f0170153be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:40.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.633+0000 7f016fe00640 1 -- 172.21.15.159:0/2179428135 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01701f7df0 con 0x7f01700cf530 2024-04-04T06:12:40.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.633+0000 7f016fe00640 1 --2- 172.21.15.159:0/2179428135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01700cf530 0x7f01701f4e60 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f016000b3f0 tx=0x7f016000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:40.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.634+0000 7f016cc00640 1 -- 172.21.15.159:0/2179428135 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160015020 con 0x7f01700cf530 2024-04-04T06:12:40.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.634+0000 7f016cc00640 1 -- 172.21.15.159:0/2179428135 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0160011830 con 0x7f01700cf530 2024-04-04T06:12:40.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.634+0000 7f016cc00640 1 -- 172.21.15.159:0/2179428135 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160011c20 con 0x7f01700cf530 2024-04-04T06:12:40.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.634+0000 7f0175800640 1 -- 172.21.15.159:0/2179428135 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01701f80d0 con 0x7f01700cf530 2024-04-04T06:12:40.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.634+0000 7f0175800640 1 -- 172.21.15.159:0/2179428135 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01701f8690 con 0x7f01700cf530 2024-04-04T06:12:40.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.635+0000 7f0175800640 1 -- 172.21.15.159:0/2179428135 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01701f8890 con 0x7f01700cf530 2024-04-04T06:12:40.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.637+0000 7f016cc00640 1 -- 172.21.15.159:0/2179428135 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0160018020 con 0x7f01700cf530 2024-04-04T06:12:40.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.638+0000 7f016cc00640 1 --2- 172.21.15.159:0/2179428135 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f014c06c8f0 0x7f014c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:40.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.638+0000 7f016cc00640 1 -- 172.21.15.159:0/2179428135 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f016008b760 con 0x7f01700cf530 2024-04-04T06:12:40.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.638+0000 7f0167400640 1 --2- 172.21.15.159:0/2179428135 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f014c06c8f0 0x7f014c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:40.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.639+0000 7f0167400640 1 --2- 172.21.15.159:0/2179428135 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f014c06c8f0 0x7f014c06edb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f01700681c0 tx=0x7f01580084d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:40.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.642+0000 7f016cc00640 1 -- 172.21.15.159:0/2179428135 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f016001f030 con 0x7f01700cf530 2024-04-04T06:12:40.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.861+0000 7f0175800640 1 -- 172.21.15.159:0/2179428135 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0170075940 con 0x7f014c06c8f0 2024-04-04T06:12:40.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.880+0000 7f016cc00640 1 -- 172.21.15.159:0/2179428135 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252033 (secure 0 0 0) 0x7f0170075940 con 0x7f014c06c8f0 2024-04-04T06:12:40.875 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:40.877 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:12:40.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.884+0000 7f0175800640 1 -- 172.21.15.159:0/2179428135 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f014c06c8f0 msgr2=0x7f014c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:40.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.884+0000 7f0175800640 1 --2- 172.21.15.159:0/2179428135 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f014c06c8f0 0x7f014c06edb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f01700681c0 tx=0x7f01580084d0 comp rx=0 tx=0).stop 2024-04-04T06:12:40.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.884+0000 7f0175800640 1 -- 172.21.15.159:0/2179428135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01700cf530 msgr2=0x7f01701f4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:40.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.885+0000 7f0175800640 1 --2- 172.21.15.159:0/2179428135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01700cf530 0x7f01701f4e60 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f016000b3f0 tx=0x7f016000b8b0 comp rx=0 tx=0).stop 2024-04-04T06:12:40.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.885+0000 7f0175800640 1 -- 172.21.15.159:0/2179428135 shutdown_connections 2024-04-04T06:12:40.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.885+0000 7f0175800640 1 --2- 172.21.15.159:0/2179428135 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f014c06c8f0 0x7f014c06edb0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:40.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.885+0000 7f0175800640 1 --2- 172.21.15.159:0/2179428135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f017015e4a0 0x7f0170153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:40.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.885+0000 7f0175800640 1 --2- 172.21.15.159:0/2179428135 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f01700d2310 0x7f01701530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:40.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.885+0000 7f0175800640 1 --2- 172.21.15.159:0/2179428135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01700cf530 0x7f01701f4e60 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:40.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.885+0000 7f0175800640 1 -- 172.21.15.159:0/2179428135 >> 172.21.15.159:0/2179428135 conn(0x7f01700676d0 msgr2=0x7f017015cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:40.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.885+0000 7f0175800640 1 -- 172.21.15.159:0/2179428135 shutdown_connections 2024-04-04T06:12:40.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:40.885+0000 7f0175800640 1 -- 172.21.15.159:0/2179428135 wait complete. 2024-04-04T06:12:40.893 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-04-04T06:12:40.861888+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258608,"kb_used_data":9152,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716112,"statfs":{"total":773094113280,"available":772829298688,"internally_reserved":0,"allocated":9371648,"data_stored":4654413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001906"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.434230+0000","last_change":"2024-04-04T06:12:36.429361+0000","last_active":"2024-04-04T06:12:36.434230+0000","last_peered":"2024-04-04T06:12:36.434230+0000","last_clean":"2024-04-04T06:12:36.434230+0000","last_became_active":"2024-04-04T06:12:36.429065+0000","last_became_peered":"2024-04-04T06:12:36.429065+0000","last_unstale":"2024-04-04T06:12:36.434230+0000","last_undegraded":"2024-04-04T06:12:36.434230+0000","last_fullsized":"2024-04-04T06:12:36.434230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:13:39.813684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.434421+0000","last_change":"2024-04-04T06:12:36.429418+0000","last_active":"2024-04-04T06:12:36.434421+0000","last_peered":"2024-04-04T06:12:36.434421+0000","last_clean":"2024-04-04T06:12:36.434421+0000","last_became_active":"2024-04-04T06:12:36.428533+0000","last_became_peered":"2024-04-04T06:12:36.428533+0000","last_unstale":"2024-04-04T06:12:36.434421+0000","last_undegraded":"2024-04-04T06:12:36.434421+0000","last_fullsized":"2024-04-04T06:12:36.434421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:30:28.756189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.436201+0000","last_change":"2024-04-04T06:12:36.428611+0000","last_active":"2024-04-04T06:12:36.436201+0000","last_peered":"2024-04-04T06:12:36.436201+0000","last_clean":"2024-04-04T06:12:36.436201+0000","last_became_active":"2024-04-04T06:12:36.428312+0000","last_became_peered":"2024-04-04T06:12:36.428312+0000","last_unstale":"2024-04-04T06:12:36.436201+0000","last_undegraded":"2024-04-04T06:12:36.436201+0000","last_fullsized":"2024-04-04T06:12:36.436201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:13:39.813684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.442076+0000","last_change":"2024-04-04T06:12:36.431457+0000","last_active":"2024-04-04T06:12:36.442076+0000","last_peered":"2024-04-04T06:12:36.442076+0000","last_clean":"2024-04-04T06:12:36.442076+0000","last_became_active":"2024-04-04T06:12:36.431121+0000","last_became_peered":"2024-04-04T06:12:36.431121+0000","last_unstale":"2024-04-04T06:12:36.442076+0000","last_undegraded":"2024-04-04T06:12:36.442076+0000","last_fullsized":"2024-04-04T06:12:36.442076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:28:29.826082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.063342+0000","last_change":"2024-04-04T06:12:36.434305+0000","last_active":"2024-04-04T06:12:38.063342+0000","last_peered":"2024-04-04T06:12:38.063342+0000","last_clean":"2024-04-04T06:12:38.063342+0000","last_became_active":"2024-04-04T06:12:36.433949+0000","last_became_peered":"2024-04-04T06:12:36.433949+0000","last_unstale":"2024-04-04T06:12:38.063342+0000","last_undegraded":"2024-04-04T06:12:38.063342+0000","last_fullsized":"2024-04-04T06:12:38.063342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:59:19.683234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.058735+0000","last_change":"2024-04-04T06:12:36.436367+0000","last_active":"2024-04-04T06:12:38.058735+0000","last_peered":"2024-04-04T06:12:38.058735+0000","last_clean":"2024-04-04T06:12:38.058735+0000","last_became_active":"2024-04-04T06:12:36.436080+0000","last_became_peered":"2024-04-04T06:12:36.436080+0000","last_unstale":"2024-04-04T06:12:38.058735+0000","last_undegraded":"2024-04-04T06:12:38.058735+0000","last_fullsized":"2024-04-04T06:12:38.058735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:29:31.971752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.438552+0000","last_change":"2024-04-04T06:12:36.425375+0000","last_active":"2024-04-04T06:12:36.438552+0000","last_peered":"2024-04-04T06:12:36.438552+0000","last_clean":"2024-04-04T06:12:36.438552+0000","last_became_active":"2024-04-04T06:12:36.425040+0000","last_became_peered":"2024-04-04T06:12:36.425040+0000","last_unstale":"2024-04-04T06:12:36.438552+0000","last_undegraded":"2024-04-04T06:12:36.438552+0000","last_fullsized":"2024-04-04T06:12:36.438552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:19:15.808794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.058525+0000","last_change":"2024-04-04T06:12:36.420379+0000","last_active":"2024-04-04T06:12:38.058525+0000","last_peered":"2024-04-04T06:12:38.058525+0000","last_clean":"2024-04-04T06:12:38.058525+0000","last_became_active":"2024-04-04T06:12:36.420030+0000","last_became_peered":"2024-04-04T06:12:36.420030+0000","last_unstale":"2024-04-04T06:12:38.058525+0000","last_undegraded":"2024-04-04T06:12:38.058525+0000","last_fullsized":"2024-04-04T06:12:38.058525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:41:50.493586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.060594+0000","last_change":"2024-04-04T06:12:36.436125+0000","last_active":"2024-04-04T06:12:38.060594+0000","last_peered":"2024-04-04T06:12:38.060594+0000","last_clean":"2024-04-04T06:12:38.060594+0000","last_became_active":"2024-04-04T06:12:36.435403+0000","last_became_peered":"2024-04-04T06:12:36.435403+0000","last_unstale":"2024-04-04T06:12:38.060594+0000","last_undegraded":"2024-04-04T06:12:38.060594+0000","last_fullsized":"2024-04-04T06:12:38.060594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:27:36.646968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.433060+0000","last_change":"2024-04-04T06:12:36.427224+0000","last_active":"2024-04-04T06:12:36.433060+0000","last_peered":"2024-04-04T06:12:36.433060+0000","last_clean":"2024-04-04T06:12:36.433060+0000","last_became_active":"2024-04-04T06:12:36.426803+0000","last_became_peered":"2024-04-04T06:12:36.426803+0000","last_unstale":"2024-04-04T06:12:36.433060+0000","last_undegraded":"2024-04-04T06:12:36.433060+0000","last_fullsized":"2024-04-04T06:12:36.433060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:04:38.131623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.434827+0000","last_change":"2024-04-04T06:12:36.430900+0000","last_active":"2024-04-04T06:12:36.434827+0000","last_peered":"2024-04-04T06:12:36.434827+0000","last_clean":"2024-04-04T06:12:36.434827+0000","last_became_active":"2024-04-04T06:12:36.430649+0000","last_became_peered":"2024-04-04T06:12:36.430649+0000","last_unstale":"2024-04-04T06:12:36.434827+0000","last_undegraded":"2024-04-04T06:12:36.434827+0000","last_fullsized":"2024-04-04T06:12:36.434827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:37:59.657044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.432894+0000","last_change":"2024-04-04T06:12:36.426862+0000","last_active":"2024-04-04T06:12:36.432894+0000","last_peered":"2024-04-04T06:12:36.432894+0000","last_clean":"2024-04-04T06:12:36.432894+0000","last_became_active":"2024-04-04T06:12:36.426289+0000","last_became_peered":"2024-04-04T06:12:36.426289+0000","last_unstale":"2024-04-04T06:12:36.432894+0000","last_undegraded":"2024-04-04T06:12:36.432894+0000","last_fullsized":"2024-04-04T06:12:36.432894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:29:18.446245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.436663+0000","last_change":"2024-04-04T06:12:36.428368+0000","last_active":"2024-04-04T06:12:36.436663+0000","last_peered":"2024-04-04T06:12:36.436663+0000","last_clean":"2024-04-04T06:12:36.436663+0000","last_became_active":"2024-04-04T06:12:36.427313+0000","last_became_peered":"2024-04-04T06:12:36.427313+0000","last_unstale":"2024-04-04T06:12:36.436663+0000","last_undegraded":"2024-04-04T06:12:36.436663+0000","last_fullsized":"2024-04-04T06:12:36.436663+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:04:38.131623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.442036+0000","last_change":"2024-04-04T06:12:36.430884+0000","last_active":"2024-04-04T06:12:36.442036+0000","last_peered":"2024-04-04T06:12:36.442036+0000","last_clean":"2024-04-04T06:12:36.442036+0000","last_became_active":"2024-04-04T06:12:36.430628+0000","last_became_peered":"2024-04-04T06:12:36.430628+0000","last_unstale":"2024-04-04T06:12:36.442036+0000","last_undegraded":"2024-04-04T06:12:36.442036+0000","last_fullsized":"2024-04-04T06:12:36.442036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:01:44.158186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.419870+0000","last_change":"2024-04-04T06:12:36.414722+0000","last_active":"2024-04-04T06:12:36.419870+0000","last_peered":"2024-04-04T06:12:36.419870+0000","last_clean":"2024-04-04T06:12:36.419870+0000","last_became_active":"2024-04-04T06:12:36.414385+0000","last_became_peered":"2024-04-04T06:12:36.414385+0000","last_unstale":"2024-04-04T06:12:36.419870+0000","last_undegraded":"2024-04-04T06:12:36.419870+0000","last_fullsized":"2024-04-04T06:12:36.419870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:48:45.291380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.059610+0000","last_change":"2024-04-04T06:12:36.428882+0000","last_active":"2024-04-04T06:12:38.059610+0000","last_peered":"2024-04-04T06:12:38.059610+0000","last_clean":"2024-04-04T06:12:38.059610+0000","last_became_active":"2024-04-04T06:12:36.428405+0000","last_became_peered":"2024-04-04T06:12:36.428405+0000","last_unstale":"2024-04-04T06:12:38.059610+0000","last_undegraded":"2024-04-04T06:12:38.059610+0000","last_fullsized":"2024-04-04T06:12:38.059610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:54:44.714523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.063578+0000","last_change":"2024-04-04T06:12:36.441905+0000","last_active":"2024-04-04T06:12:38.063578+0000","last_peered":"2024-04-04T06:12:38.063578+0000","last_clean":"2024-04-04T06:12:38.063578+0000","last_became_active":"2024-04-04T06:12:36.441543+0000","last_became_peered":"2024-04-04T06:12:36.441543+0000","last_unstale":"2024-04-04T06:12:38.063578+0000","last_undegraded":"2024-04-04T06:12:38.063578+0000","last_fullsized":"2024-04-04T06:12:38.063578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:22:23.086185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.438410+0000","last_change":"2024-04-04T06:12:36.426213+0000","last_active":"2024-04-04T06:12:36.438410+0000","last_peered":"2024-04-04T06:12:36.438410+0000","last_clean":"2024-04-04T06:12:36.438410+0000","last_became_active":"2024-04-04T06:12:36.425915+0000","last_became_peered":"2024-04-04T06:12:36.425915+0000","last_unstale":"2024-04-04T06:12:36.438410+0000","last_undegraded":"2024-04-04T06:12:36.438410+0000","last_fullsized":"2024-04-04T06:12:36.438410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:23:30.039862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.437316+0000","last_change":"2024-04-04T06:12:36.434242+0000","last_active":"2024-04-04T06:12:36.437316+0000","last_peered":"2024-04-04T06:12:36.437316+0000","last_clean":"2024-04-04T06:12:36.437316+0000","last_became_active":"2024-04-04T06:12:36.433035+0000","last_became_peered":"2024-04-04T06:12:36.433035+0000","last_unstale":"2024-04-04T06:12:36.437316+0000","last_undegraded":"2024-04-04T06:12:36.437316+0000","last_fullsized":"2024-04-04T06:12:36.437316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:08:49.667260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.445353+0000","last_change":"2024-04-04T06:12:36.441713+0000","last_active":"2024-04-04T06:12:36.445353+0000","last_peered":"2024-04-04T06:12:36.445353+0000","last_clean":"2024-04-04T06:12:36.445353+0000","last_became_active":"2024-04-04T06:12:36.441413+0000","last_became_peered":"2024-04-04T06:12:36.441413+0000","last_unstale":"2024-04-04T06:12:36.445353+0000","last_undegraded":"2024-04-04T06:12:36.445353+0000","last_fullsized":"2024-04-04T06:12:36.445353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:34:44.792603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.443680+0000","last_change":"2024-04-04T06:12:36.436241+0000","last_active":"2024-04-04T06:12:36.443680+0000","last_peered":"2024-04-04T06:12:36.443680+0000","last_clean":"2024-04-04T06:12:36.443680+0000","last_became_active":"2024-04-04T06:12:36.435331+0000","last_became_peered":"2024-04-04T06:12:36.435331+0000","last_unstale":"2024-04-04T06:12:36.443680+0000","last_undegraded":"2024-04-04T06:12:36.443680+0000","last_fullsized":"2024-04-04T06:12:36.443680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:57:35.026821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.446667+0000","last_change":"2024-04-04T06:12:36.440344+0000","last_active":"2024-04-04T06:12:36.446667+0000","last_peered":"2024-04-04T06:12:36.446667+0000","last_clean":"2024-04-04T06:12:36.446667+0000","last_became_active":"2024-04-04T06:12:36.440027+0000","last_became_peered":"2024-04-04T06:12:36.440027+0000","last_unstale":"2024-04-04T06:12:36.446667+0000","last_undegraded":"2024-04-04T06:12:36.446667+0000","last_fullsized":"2024-04-04T06:12:36.446667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:59:47.453213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.439762+0000","last_change":"2024-04-04T06:12:36.437622+0000","last_active":"2024-04-04T06:12:36.439762+0000","last_peered":"2024-04-04T06:12:36.439762+0000","last_clean":"2024-04-04T06:12:36.439762+0000","last_became_active":"2024-04-04T06:12:36.437315+0000","last_became_peered":"2024-04-04T06:12:36.437315+0000","last_unstale":"2024-04-04T06:12:36.439762+0000","last_undegraded":"2024-04-04T06:12:36.439762+0000","last_fullsized":"2024-04-04T06:12:36.439762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:39:04.274978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.443583+0000","last_change":"2024-04-04T06:12:36.431767+0000","last_active":"2024-04-04T06:12:36.443583+0000","last_peered":"2024-04-04T06:12:36.443583+0000","last_clean":"2024-04-04T06:12:36.443583+0000","last_became_active":"2024-04-04T06:12:36.431484+0000","last_became_peered":"2024-04-04T06:12:36.431484+0000","last_unstale":"2024-04-04T06:12:36.443583+0000","last_undegraded":"2024-04-04T06:12:36.443583+0000","last_fullsized":"2024-04-04T06:12:36.443583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:45:59.919636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.432304+0000","last_change":"2024-04-04T06:12:36.427797+0000","last_active":"2024-04-04T06:12:36.432304+0000","last_peered":"2024-04-04T06:12:36.432304+0000","last_clean":"2024-04-04T06:12:36.432304+0000","last_became_active":"2024-04-04T06:12:36.427540+0000","last_became_peered":"2024-04-04T06:12:36.427540+0000","last_unstale":"2024-04-04T06:12:36.432304+0000","last_undegraded":"2024-04-04T06:12:36.432304+0000","last_fullsized":"2024-04-04T06:12:36.432304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:05:06.923430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.437086+0000","last_change":"2024-04-04T06:12:36.430661+0000","last_active":"2024-04-04T06:12:36.437086+0000","last_peered":"2024-04-04T06:12:36.437086+0000","last_clean":"2024-04-04T06:12:36.437086+0000","last_became_active":"2024-04-04T06:12:36.430311+0000","last_became_peered":"2024-04-04T06:12:36.430311+0000","last_unstale":"2024-04-04T06:12:36.437086+0000","last_undegraded":"2024-04-04T06:12:36.437086+0000","last_fullsized":"2024-04-04T06:12:36.437086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:32:08.355924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.061512+0000","last_change":"2024-04-04T06:12:36.429705+0000","last_active":"2024-04-04T06:12:38.061512+0000","last_peered":"2024-04-04T06:12:38.061512+0000","last_clean":"2024-04-04T06:12:38.061512+0000","last_became_active":"2024-04-04T06:12:36.428873+0000","last_became_peered":"2024-04-04T06:12:36.428873+0000","last_unstale":"2024-04-04T06:12:38.061512+0000","last_undegraded":"2024-04-04T06:12:38.061512+0000","last_fullsized":"2024-04-04T06:12:38.061512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:56:54.519484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.432658+0000","last_change":"2024-04-04T06:12:36.426588+0000","last_active":"2024-04-04T06:12:36.432658+0000","last_peered":"2024-04-04T06:12:36.432658+0000","last_clean":"2024-04-04T06:12:36.432658+0000","last_became_active":"2024-04-04T06:12:36.425911+0000","last_became_peered":"2024-04-04T06:12:36.425911+0000","last_unstale":"2024-04-04T06:12:36.432658+0000","last_undegraded":"2024-04-04T06:12:36.432658+0000","last_fullsized":"2024-04-04T06:12:36.432658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:41:21.034109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.440817+0000","last_change":"2024-04-04T06:12:36.432535+0000","last_active":"2024-04-04T06:12:36.440817+0000","last_peered":"2024-04-04T06:12:36.440817+0000","last_clean":"2024-04-04T06:12:36.440817+0000","last_became_active":"2024-04-04T06:12:36.429964+0000","last_became_peered":"2024-04-04T06:12:36.429964+0000","last_unstale":"2024-04-04T06:12:36.440817+0000","last_undegraded":"2024-04-04T06:12:36.440817+0000","last_fullsized":"2024-04-04T06:12:36.440817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:11:09.469338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.442505+0000","last_change":"2024-04-04T06:12:36.434144+0000","last_active":"2024-04-04T06:12:36.442505+0000","last_peered":"2024-04-04T06:12:36.442505+0000","last_clean":"2024-04-04T06:12:36.442505+0000","last_became_active":"2024-04-04T06:12:36.433845+0000","last_became_peered":"2024-04-04T06:12:36.433845+0000","last_unstale":"2024-04-04T06:12:36.442505+0000","last_undegraded":"2024-04-04T06:12:36.442505+0000","last_fullsized":"2024-04-04T06:12:36.442505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:49:19.575270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.059887+0000","last_change":"2024-04-04T06:12:36.442360+0000","last_active":"2024-04-04T06:12:38.059887+0000","last_peered":"2024-04-04T06:12:38.059887+0000","last_clean":"2024-04-04T06:12:38.059887+0000","last_became_active":"2024-04-04T06:12:36.440745+0000","last_became_peered":"2024-04-04T06:12:36.440745+0000","last_unstale":"2024-04-04T06:12:38.059887+0000","last_undegraded":"2024-04-04T06:12:38.059887+0000","last_fullsized":"2024-04-04T06:12:38.059887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:52:40.608336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.441578+0000","last_change":"2024-04-04T06:12:36.440242+0000","last_active":"2024-04-04T06:12:36.441578+0000","last_peered":"2024-04-04T06:12:36.441578+0000","last_clean":"2024-04-04T06:12:36.441578+0000","last_became_active":"2024-04-04T06:12:36.439968+0000","last_became_peered":"2024-04-04T06:12:36.439968+0000","last_unstale":"2024-04-04T06:12:36.441578+0000","last_undegraded":"2024-04-04T06:12:36.441578+0000","last_fullsized":"2024-04-04T06:12:36.441578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:32:06.609995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.063419+0000","last_change":"2024-04-04T06:12:36.427890+0000","last_active":"2024-04-04T06:12:38.063419+0000","last_peered":"2024-04-04T06:12:38.063419+0000","last_clean":"2024-04-04T06:12:38.063419+0000","last_became_active":"2024-04-04T06:12:36.427578+0000","last_became_peered":"2024-04-04T06:12:36.427578+0000","last_unstale":"2024-04-04T06:12:38.063419+0000","last_undegraded":"2024-04-04T06:12:38.063419+0000","last_fullsized":"2024-04-04T06:12:38.063419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:48:45.291380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"84'4","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.451841+0000","last_change":"2024-04-04T06:12:35.440989+0000","last_active":"2024-04-04T06:12:36.451841+0000","last_peered":"2024-04-04T06:12:36.451841+0000","last_clean":"2024-04-04T06:12:36.451841+0000","last_became_active":"2024-04-04T06:12:35.440157+0000","last_became_peered":"2024-04-04T06:12:35.440157+0000","last_unstale":"2024-04-04T06:12:36.451841+0000","last_undegraded":"2024-04-04T06:12:36.451841+0000","last_fullsized":"2024-04-04T06:12:36.451841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:12:25.822017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.438905+0000","last_change":"2024-04-04T06:12:36.435554+0000","last_active":"2024-04-04T06:12:36.438905+0000","last_peered":"2024-04-04T06:12:36.438905+0000","last_clean":"2024-04-04T06:12:36.438905+0000","last_became_active":"2024-04-04T06:12:36.435244+0000","last_became_peered":"2024-04-04T06:12:36.435244+0000","last_unstale":"2024-04-04T06:12:36.438905+0000","last_undegraded":"2024-04-04T06:12:36.438905+0000","last_fullsized":"2024-04-04T06:12:36.438905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:40:32.107666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.063747+0000","last_change":"2024-04-04T06:12:35.429074+0000","last_active":"2024-04-04T06:12:38.063747+0000","last_peered":"2024-04-04T06:12:38.063747+0000","last_clean":"2024-04-04T06:12:38.063747+0000","last_became_active":"2024-04-04T06:12:35.428641+0000","last_became_peered":"2024-04-04T06:12:35.428641+0000","last_unstale":"2024-04-04T06:12:38.063747+0000","last_undegraded":"2024-04-04T06:12:38.063747+0000","last_fullsized":"2024-04-04T06:12:38.063747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:23:56.106689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.439006+0000","last_change":"2024-04-04T06:12:36.435609+0000","last_active":"2024-04-04T06:12:36.439006+0000","last_peered":"2024-04-04T06:12:36.439006+0000","last_clean":"2024-04-04T06:12:36.439006+0000","last_became_active":"2024-04-04T06:12:36.435055+0000","last_became_peered":"2024-04-04T06:12:36.435055+0000","last_unstale":"2024-04-04T06:12:36.439006+0000","last_undegraded":"2024-04-04T06:12:36.439006+0000","last_fullsized":"2024-04-04T06:12:36.439006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:44:52.660138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"84'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.452259+0000","last_change":"2024-04-04T06:12:35.423780+0000","last_active":"2024-04-04T06:12:36.452259+0000","last_peered":"2024-04-04T06:12:36.452259+0000","last_clean":"2024-04-04T06:12:36.452259+0000","last_became_active":"2024-04-04T06:12:35.423493+0000","last_became_peered":"2024-04-04T06:12:35.423493+0000","last_unstale":"2024-04-04T06:12:36.452259+0000","last_undegraded":"2024-04-04T06:12:36.452259+0000","last_fullsized":"2024-04-04T06:12:36.452259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:11:49.111105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.439714+0000","last_change":"2024-04-04T06:12:36.437860+0000","last_active":"2024-04-04T06:12:36.439714+0000","last_peered":"2024-04-04T06:12:36.439714+0000","last_clean":"2024-04-04T06:12:36.439714+0000","last_became_active":"2024-04-04T06:12:36.437583+0000","last_became_peered":"2024-04-04T06:12:36.437583+0000","last_unstale":"2024-04-04T06:12:36.439714+0000","last_undegraded":"2024-04-04T06:12:36.439714+0000","last_fullsized":"2024-04-04T06:12:36.439714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:39:04.274978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"84'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.452281+0000","last_change":"2024-04-04T06:12:35.427149+0000","last_active":"2024-04-04T06:12:36.452281+0000","last_peered":"2024-04-04T06:12:36.452281+0000","last_clean":"2024-04-04T06:12:36.452281+0000","last_became_active":"2024-04-04T06:12:35.426845+0000","last_became_peered":"2024-04-04T06:12:35.426845+0000","last_unstale":"2024-04-04T06:12:36.452281+0000","last_undegraded":"2024-04-04T06:12:36.452281+0000","last_fullsized":"2024-04-04T06:12:36.452281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:41:49.478293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.432321+0000","last_change":"2024-04-04T06:12:36.424823+0000","last_active":"2024-04-04T06:12:36.432321+0000","last_peered":"2024-04-04T06:12:36.432321+0000","last_clean":"2024-04-04T06:12:36.432321+0000","last_became_active":"2024-04-04T06:12:36.424467+0000","last_became_peered":"2024-04-04T06:12:36.424467+0000","last_unstale":"2024-04-04T06:12:36.432321+0000","last_undegraded":"2024-04-04T06:12:36.432321+0000","last_fullsized":"2024-04-04T06:12:36.432321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:27:12.212980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.063735+0000","last_change":"2024-04-04T06:12:35.420818+0000","last_active":"2024-04-04T06:12:38.063735+0000","last_peered":"2024-04-04T06:12:38.063735+0000","last_clean":"2024-04-04T06:12:38.063735+0000","last_became_active":"2024-04-04T06:12:35.420221+0000","last_became_peered":"2024-04-04T06:12:35.420221+0000","last_unstale":"2024-04-04T06:12:38.063735+0000","last_undegraded":"2024-04-04T06:12:38.063735+0000","last_fullsized":"2024-04-04T06:12:38.063735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:59:26.700419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.063524+0000","last_change":"2024-04-04T06:12:36.443098+0000","last_active":"2024-04-04T06:12:38.063524+0000","last_peered":"2024-04-04T06:12:38.063524+0000","last_clean":"2024-04-04T06:12:38.063524+0000","last_became_active":"2024-04-04T06:12:36.441843+0000","last_became_peered":"2024-04-04T06:12:36.441843+0000","last_unstale":"2024-04-04T06:12:38.063524+0000","last_undegraded":"2024-04-04T06:12:38.063524+0000","last_fullsized":"2024-04-04T06:12:38.063524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:09:16.618599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.059902+0000","last_change":"2024-04-04T06:12:35.425083+0000","last_active":"2024-04-04T06:12:38.059902+0000","last_peered":"2024-04-04T06:12:38.059902+0000","last_clean":"2024-04-04T06:12:38.059902+0000","last_became_active":"2024-04-04T06:12:35.423680+0000","last_became_peered":"2024-04-04T06:12:35.423680+0000","last_unstale":"2024-04-04T06:12:38.059902+0000","last_undegraded":"2024-04-04T06:12:38.059902+0000","last_fullsized":"2024-04-04T06:12:38.059902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:44:49.872004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.439615+0000","last_change":"2024-04-04T06:12:36.436841+0000","last_active":"2024-04-04T06:12:36.439615+0000","last_peered":"2024-04-04T06:12:36.439615+0000","last_clean":"2024-04-04T06:12:36.439615+0000","last_became_active":"2024-04-04T06:12:36.436303+0000","last_became_peered":"2024-04-04T06:12:36.436303+0000","last_unstale":"2024-04-04T06:12:36.439615+0000","last_undegraded":"2024-04-04T06:12:36.439615+0000","last_fullsized":"2024-04-04T06:12:36.439615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:40:32.107666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.426452+0000","last_change":"2024-04-04T06:12:35.428726+0000","last_active":"2024-04-04T06:12:36.426452+0000","last_peered":"2024-04-04T06:12:36.426452+0000","last_clean":"2024-04-04T06:12:36.426452+0000","last_became_active":"2024-04-04T06:12:35.428382+0000","last_became_peered":"2024-04-04T06:12:35.428382+0000","last_unstale":"2024-04-04T06:12:36.426452+0000","last_undegraded":"2024-04-04T06:12:36.426452+0000","last_fullsized":"2024-04-04T06:12:36.426452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:59:26.700419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.419677+0000","last_change":"2024-04-04T06:12:36.413565+0000","last_active":"2024-04-04T06:12:36.419677+0000","last_peered":"2024-04-04T06:12:36.419677+0000","last_clean":"2024-04-04T06:12:36.419677+0000","last_became_active":"2024-04-04T06:12:36.412693+0000","last_became_peered":"2024-04-04T06:12:36.412693+0000","last_unstale":"2024-04-04T06:12:36.419677+0000","last_undegraded":"2024-04-04T06:12:36.419677+0000","last_fullsized":"2024-04-04T06:12:36.419677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:59:19.683234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.408892+0000","last_change":"2024-04-04T06:12:35.425315+0000","last_active":"2024-04-04T06:12:36.408892+0000","last_peered":"2024-04-04T06:12:36.408892+0000","last_clean":"2024-04-04T06:12:36.408892+0000","last_became_active":"2024-04-04T06:12:35.425003+0000","last_became_peered":"2024-04-04T06:12:35.425003+0000","last_unstale":"2024-04-04T06:12:36.408892+0000","last_undegraded":"2024-04-04T06:12:36.408892+0000","last_fullsized":"2024-04-04T06:12:36.408892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:11:16.547070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.425676+0000","last_change":"2024-04-04T06:12:36.421159+0000","last_active":"2024-04-04T06:12:36.425676+0000","last_peered":"2024-04-04T06:12:36.425676+0000","last_clean":"2024-04-04T06:12:36.425676+0000","last_became_active":"2024-04-04T06:12:36.420884+0000","last_became_peered":"2024-04-04T06:12:36.420884+0000","last_unstale":"2024-04-04T06:12:36.425676+0000","last_undegraded":"2024-04-04T06:12:36.425676+0000","last_fullsized":"2024-04-04T06:12:36.425676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:10:20.346535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.418978+0000","last_change":"2024-04-04T06:12:35.441361+0000","last_active":"2024-04-04T06:12:36.418978+0000","last_peered":"2024-04-04T06:12:36.418978+0000","last_clean":"2024-04-04T06:12:36.418978+0000","last_became_active":"2024-04-04T06:12:35.440782+0000","last_became_peered":"2024-04-04T06:12:35.440782+0000","last_unstale":"2024-04-04T06:12:36.418978+0000","last_undegraded":"2024-04-04T06:12:36.418978+0000","last_fullsized":"2024-04-04T06:12:36.418978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:29:02.762166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.060634+0000","last_change":"2024-04-04T06:12:36.428355+0000","last_active":"2024-04-04T06:12:38.060634+0000","last_peered":"2024-04-04T06:12:38.060634+0000","last_clean":"2024-04-04T06:12:38.060634+0000","last_became_active":"2024-04-04T06:12:36.428064+0000","last_became_peered":"2024-04-04T06:12:36.428064+0000","last_unstale":"2024-04-04T06:12:38.060634+0000","last_undegraded":"2024-04-04T06:12:38.060634+0000","last_fullsized":"2024-04-04T06:12:38.060634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:08:49.667260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.417173+0000","last_change":"2024-04-04T06:12:35.437893+0000","last_active":"2024-04-04T06:12:36.417173+0000","last_peered":"2024-04-04T06:12:36.417173+0000","last_clean":"2024-04-04T06:12:36.417173+0000","last_became_active":"2024-04-04T06:12:35.435814+0000","last_became_peered":"2024-04-04T06:12:35.435814+0000","last_unstale":"2024-04-04T06:12:36.417173+0000","last_undegraded":"2024-04-04T06:12:36.417173+0000","last_fullsized":"2024-04-04T06:12:36.417173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:47:59.871101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.447797+0000","last_change":"2024-04-04T06:12:36.438570+0000","last_active":"2024-04-04T06:12:36.447797+0000","last_peered":"2024-04-04T06:12:36.447797+0000","last_clean":"2024-04-04T06:12:36.447797+0000","last_became_active":"2024-04-04T06:12:36.438245+0000","last_became_peered":"2024-04-04T06:12:36.438245+0000","last_unstale":"2024-04-04T06:12:36.447797+0000","last_undegraded":"2024-04-04T06:12:36.447797+0000","last_fullsized":"2024-04-04T06:12:36.447797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:11:35.711672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"84'3","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.451596+0000","last_change":"2024-04-04T06:12:35.436052+0000","last_active":"2024-04-04T06:12:36.451596+0000","last_peered":"2024-04-04T06:12:36.451596+0000","last_clean":"2024-04-04T06:12:36.451596+0000","last_became_active":"2024-04-04T06:12:35.435387+0000","last_became_peered":"2024-04-04T06:12:35.435387+0000","last_unstale":"2024-04-04T06:12:36.451596+0000","last_undegraded":"2024-04-04T06:12:36.451596+0000","last_fullsized":"2024-04-04T06:12:36.451596+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:17:07.241351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.061894+0000","last_change":"2024-04-04T06:12:36.440179+0000","last_active":"2024-04-04T06:12:38.061894+0000","last_peered":"2024-04-04T06:12:38.061894+0000","last_clean":"2024-04-04T06:12:38.061894+0000","last_became_active":"2024-04-04T06:12:36.439899+0000","last_became_peered":"2024-04-04T06:12:36.439899+0000","last_unstale":"2024-04-04T06:12:38.061894+0000","last_undegraded":"2024-04-04T06:12:38.061894+0000","last_fullsized":"2024-04-04T06:12:38.061894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:00:29.007550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"84'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.060217+0000","last_change":"2024-04-04T06:12:35.430456+0000","last_active":"2024-04-04T06:12:38.060217+0000","last_peered":"2024-04-04T06:12:38.060217+0000","last_clean":"2024-04-04T06:12:38.060217+0000","last_became_active":"2024-04-04T06:12:35.430095+0000","last_became_peered":"2024-04-04T06:12:35.430095+0000","last_unstale":"2024-04-04T06:12:38.060217+0000","last_undegraded":"2024-04-04T06:12:38.060217+0000","last_fullsized":"2024-04-04T06:12:38.060217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:09:02.307314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.447468+0000","last_change":"2024-04-04T06:12:36.437720+0000","last_active":"2024-04-04T06:12:36.447468+0000","last_peered":"2024-04-04T06:12:36.447468+0000","last_clean":"2024-04-04T06:12:36.447468+0000","last_became_active":"2024-04-04T06:12:36.437383+0000","last_became_peered":"2024-04-04T06:12:36.437383+0000","last_unstale":"2024-04-04T06:12:36.447468+0000","last_undegraded":"2024-04-04T06:12:36.447468+0000","last_fullsized":"2024-04-04T06:12:36.447468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:27:36.646968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"84'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.456587+0000","last_change":"2024-04-04T06:12:35.417322+0000","last_active":"2024-04-04T06:12:36.456587+0000","last_peered":"2024-04-04T06:12:36.456587+0000","last_clean":"2024-04-04T06:12:36.456587+0000","last_became_active":"2024-04-04T06:12:35.417068+0000","last_became_peered":"2024-04-04T06:12:35.417068+0000","last_unstale":"2024-04-04T06:12:36.456587+0000","last_undegraded":"2024-04-04T06:12:36.456587+0000","last_fullsized":"2024-04-04T06:12:36.456587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:34:43.777310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.439678+0000","last_change":"2024-04-04T06:12:36.437309+0000","last_active":"2024-04-04T06:12:36.439678+0000","last_peered":"2024-04-04T06:12:36.439678+0000","last_clean":"2024-04-04T06:12:36.439678+0000","last_became_active":"2024-04-04T06:12:36.436858+0000","last_became_peered":"2024-04-04T06:12:36.436858+0000","last_unstale":"2024-04-04T06:12:36.439678+0000","last_undegraded":"2024-04-04T06:12:36.439678+0000","last_fullsized":"2024-04-04T06:12:36.439678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:44:52.660138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.397609+0000","last_change":"2024-04-04T06:12:35.426532+0000","last_active":"2024-04-04T06:12:36.397609+0000","last_peered":"2024-04-04T06:12:36.397609+0000","last_clean":"2024-04-04T06:12:36.397609+0000","last_became_active":"2024-04-04T06:12:35.426216+0000","last_became_peered":"2024-04-04T06:12:35.426216+0000","last_unstale":"2024-04-04T06:12:36.397609+0000","last_undegraded":"2024-04-04T06:12:36.397609+0000","last_fullsized":"2024-04-04T06:12:36.397609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:17:21.722450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.443096+0000","last_change":"2024-04-04T06:12:36.439767+0000","last_active":"2024-04-04T06:12:36.443096+0000","last_peered":"2024-04-04T06:12:36.443096+0000","last_clean":"2024-04-04T06:12:36.443096+0000","last_became_active":"2024-04-04T06:12:36.439465+0000","last_became_peered":"2024-04-04T06:12:36.439465+0000","last_unstale":"2024-04-04T06:12:36.443096+0000","last_undegraded":"2024-04-04T06:12:36.443096+0000","last_fullsized":"2024-04-04T06:12:36.443096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:51:53.080092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.058647+0000","last_change":"2024-04-04T06:12:35.435356+0000","last_active":"2024-04-04T06:12:38.058647+0000","last_peered":"2024-04-04T06:12:38.058647+0000","last_clean":"2024-04-04T06:12:38.058647+0000","last_became_active":"2024-04-04T06:12:35.433434+0000","last_became_peered":"2024-04-04T06:12:35.433434+0000","last_unstale":"2024-04-04T06:12:38.058647+0000","last_undegraded":"2024-04-04T06:12:38.058647+0000","last_fullsized":"2024-04-04T06:12:38.058647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:18:05.338746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.434660+0000","last_change":"2024-04-04T06:12:36.424564+0000","last_active":"2024-04-04T06:12:36.434660+0000","last_peered":"2024-04-04T06:12:36.434660+0000","last_clean":"2024-04-04T06:12:36.434660+0000","last_became_active":"2024-04-04T06:12:36.424262+0000","last_became_peered":"2024-04-04T06:12:36.424262+0000","last_unstale":"2024-04-04T06:12:36.434660+0000","last_undegraded":"2024-04-04T06:12:36.434660+0000","last_fullsized":"2024-04-04T06:12:36.434660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:28:47.006181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"84'8","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.451712+0000","last_change":"2024-04-04T06:12:35.432102+0000","last_active":"2024-04-04T06:12:36.451712+0000","last_peered":"2024-04-04T06:12:36.451712+0000","last_clean":"2024-04-04T06:12:36.451712+0000","last_became_active":"2024-04-04T06:12:35.431317+0000","last_became_peered":"2024-04-04T06:12:35.431317+0000","last_unstale":"2024-04-04T06:12:36.451712+0000","last_undegraded":"2024-04-04T06:12:36.451712+0000","last_fullsized":"2024-04-04T06:12:36.451712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:23:56.106689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.429988+0000","last_change":"2024-04-04T06:12:36.423474+0000","last_active":"2024-04-04T06:12:36.429988+0000","last_peered":"2024-04-04T06:12:36.429988+0000","last_clean":"2024-04-04T06:12:36.429988+0000","last_became_active":"2024-04-04T06:12:36.423174+0000","last_became_peered":"2024-04-04T06:12:36.423174+0000","last_unstale":"2024-04-04T06:12:36.429988+0000","last_undegraded":"2024-04-04T06:12:36.429988+0000","last_fullsized":"2024-04-04T06:12:36.429988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:29:18.446245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"84'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.446160+0000","last_change":"2024-04-04T06:12:35.441251+0000","last_active":"2024-04-04T06:12:36.446160+0000","last_peered":"2024-04-04T06:12:36.446160+0000","last_clean":"2024-04-04T06:12:36.446160+0000","last_became_active":"2024-04-04T06:12:35.440627+0000","last_became_peered":"2024-04-04T06:12:35.440627+0000","last_unstale":"2024-04-04T06:12:36.446160+0000","last_undegraded":"2024-04-04T06:12:36.446160+0000","last_fullsized":"2024-04-04T06:12:36.446160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:23:42.612650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.433788+0000","last_change":"2024-04-04T06:12:36.426989+0000","last_active":"2024-04-04T06:12:36.433788+0000","last_peered":"2024-04-04T06:12:36.433788+0000","last_clean":"2024-04-04T06:12:36.433788+0000","last_became_active":"2024-04-04T06:12:36.426650+0000","last_became_peered":"2024-04-04T06:12:36.426650+0000","last_unstale":"2024-04-04T06:12:36.433788+0000","last_undegraded":"2024-04-04T06:12:36.433788+0000","last_fullsized":"2024-04-04T06:12:36.433788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:12:14.379165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"84'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.449589+0000","last_change":"2024-04-04T06:12:35.432366+0000","last_active":"2024-04-04T06:12:36.449589+0000","last_peered":"2024-04-04T06:12:36.449589+0000","last_clean":"2024-04-04T06:12:36.449589+0000","last_became_active":"2024-04-04T06:12:35.431887+0000","last_became_peered":"2024-04-04T06:12:35.431887+0000","last_unstale":"2024-04-04T06:12:36.449589+0000","last_undegraded":"2024-04-04T06:12:36.449589+0000","last_fullsized":"2024-04-04T06:12:36.449589+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:29:30.956459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.062252+0000","last_change":"2024-04-04T06:12:36.433535+0000","last_active":"2024-04-04T06:12:38.062252+0000","last_peered":"2024-04-04T06:12:38.062252+0000","last_clean":"2024-04-04T06:12:38.062252+0000","last_became_active":"2024-04-04T06:12:36.433221+0000","last_became_peered":"2024-04-04T06:12:36.433221+0000","last_unstale":"2024-04-04T06:12:38.062252+0000","last_undegraded":"2024-04-04T06:12:38.062252+0000","last_fullsized":"2024-04-04T06:12:38.062252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:17:22.737743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.426913+0000","last_change":"2024-04-04T06:12:35.433632+0000","last_active":"2024-04-04T06:12:36.426913+0000","last_peered":"2024-04-04T06:12:36.426913+0000","last_clean":"2024-04-04T06:12:36.426913+0000","last_became_active":"2024-04-04T06:12:35.433367+0000","last_became_peered":"2024-04-04T06:12:35.433367+0000","last_unstale":"2024-04-04T06:12:36.426913+0000","last_undegraded":"2024-04-04T06:12:36.426913+0000","last_fullsized":"2024-04-04T06:12:36.426913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:24:50.972859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.446031+0000","last_change":"2024-04-04T06:12:36.441815+0000","last_active":"2024-04-04T06:12:36.446031+0000","last_peered":"2024-04-04T06:12:36.446031+0000","last_clean":"2024-04-04T06:12:36.446031+0000","last_became_active":"2024-04-04T06:12:36.441228+0000","last_became_peered":"2024-04-04T06:12:36.441228+0000","last_unstale":"2024-04-04T06:12:36.446031+0000","last_undegraded":"2024-04-04T06:12:36.446031+0000","last_fullsized":"2024-04-04T06:12:36.446031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:11:17.562364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"84'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.446437+0000","last_change":"2024-04-04T06:12:35.430478+0000","last_active":"2024-04-04T06:12:36.446437+0000","last_peered":"2024-04-04T06:12:36.446437+0000","last_clean":"2024-04-04T06:12:36.446437+0000","last_became_active":"2024-04-04T06:12:35.429854+0000","last_became_peered":"2024-04-04T06:12:35.429854+0000","last_unstale":"2024-04-04T06:12:36.446437+0000","last_undegraded":"2024-04-04T06:12:36.446437+0000","last_fullsized":"2024-04-04T06:12:36.446437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:58:09.090078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.058937+0000","last_change":"2024-04-04T06:12:36.422870+0000","last_active":"2024-04-04T06:12:38.058937+0000","last_peered":"2024-04-04T06:12:38.058937+0000","last_clean":"2024-04-04T06:12:38.058937+0000","last_became_active":"2024-04-04T06:12:36.422465+0000","last_became_peered":"2024-04-04T06:12:36.422465+0000","last_unstale":"2024-04-04T06:12:38.058937+0000","last_undegraded":"2024-04-04T06:12:38.058937+0000","last_fullsized":"2024-04-04T06:12:38.058937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:01:36.178837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.416819+0000","last_change":"2024-04-04T06:12:35.430066+0000","last_active":"2024-04-04T06:12:36.416819+0000","last_peered":"2024-04-04T06:12:36.416819+0000","last_clean":"2024-04-04T06:12:36.416819+0000","last_became_active":"2024-04-04T06:12:35.429780+0000","last_became_peered":"2024-04-04T06:12:35.429780+0000","last_unstale":"2024-04-04T06:12:36.416819+0000","last_undegraded":"2024-04-04T06:12:36.416819+0000","last_fullsized":"2024-04-04T06:12:36.416819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:05:40.751970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.438599+0000","last_change":"2024-04-04T06:12:36.427660+0000","last_active":"2024-04-04T06:12:36.438599+0000","last_peered":"2024-04-04T06:12:36.438599+0000","last_clean":"2024-04-04T06:12:36.438599+0000","last_became_active":"2024-04-04T06:12:36.427384+0000","last_became_peered":"2024-04-04T06:12:36.427384+0000","last_unstale":"2024-04-04T06:12:36.438599+0000","last_undegraded":"2024-04-04T06:12:36.438599+0000","last_fullsized":"2024-04-04T06:12:36.438599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:36:48.111851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.416545+0000","last_change":"2024-04-04T06:12:35.441503+0000","last_active":"2024-04-04T06:12:36.416545+0000","last_peered":"2024-04-04T06:12:36.416545+0000","last_clean":"2024-04-04T06:12:36.416545+0000","last_became_active":"2024-04-04T06:12:35.441264+0000","last_became_peered":"2024-04-04T06:12:35.441264+0000","last_unstale":"2024-04-04T06:12:36.416545+0000","last_undegraded":"2024-04-04T06:12:36.416545+0000","last_fullsized":"2024-04-04T06:12:36.416545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:49:29.627886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.058615+0000","last_change":"2024-04-04T06:12:36.424685+0000","last_active":"2024-04-04T06:12:38.058615+0000","last_peered":"2024-04-04T06:12:38.058615+0000","last_clean":"2024-04-04T06:12:38.058615+0000","last_became_active":"2024-04-04T06:12:36.423224+0000","last_became_peered":"2024-04-04T06:12:36.423224+0000","last_unstale":"2024-04-04T06:12:38.058615+0000","last_undegraded":"2024-04-04T06:12:38.058615+0000","last_fullsized":"2024-04-04T06:12:38.058615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:24:51.988152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.397454+0000","last_change":"2024-04-04T06:12:35.430335+0000","last_active":"2024-04-04T06:12:36.397454+0000","last_peered":"2024-04-04T06:12:36.397454+0000","last_clean":"2024-04-04T06:12:36.397454+0000","last_became_active":"2024-04-04T06:12:35.430048+0000","last_became_peered":"2024-04-04T06:12:35.430048+0000","last_unstale":"2024-04-04T06:12:36.397454+0000","last_undegraded":"2024-04-04T06:12:36.397454+0000","last_fullsized":"2024-04-04T06:12:36.397454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:00:27.992256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.063495+0000","last_change":"2024-04-04T06:12:36.432152+0000","last_active":"2024-04-04T06:12:38.063495+0000","last_peered":"2024-04-04T06:12:38.063495+0000","last_clean":"2024-04-04T06:12:38.063495+0000","last_became_active":"2024-04-04T06:12:36.431841+0000","last_became_peered":"2024-04-04T06:12:36.431841+0000","last_unstale":"2024-04-04T06:12:38.063495+0000","last_undegraded":"2024-04-04T06:12:38.063495+0000","last_fullsized":"2024-04-04T06:12:38.063495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:10:20.346535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"84'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.456400+0000","last_change":"2024-04-04T06:12:35.434599+0000","last_active":"2024-04-04T06:12:36.456400+0000","last_peered":"2024-04-04T06:12:36.456400+0000","last_clean":"2024-04-04T06:12:36.456400+0000","last_became_active":"2024-04-04T06:12:35.434290+0000","last_became_peered":"2024-04-04T06:12:35.434290+0000","last_unstale":"2024-04-04T06:12:36.456400+0000","last_undegraded":"2024-04-04T06:12:36.456400+0000","last_fullsized":"2024-04-04T06:12:36.456400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:39:18.902549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.433959+0000","last_change":"2024-04-04T06:12:36.427827+0000","last_active":"2024-04-04T06:12:36.433959+0000","last_peered":"2024-04-04T06:12:36.433959+0000","last_clean":"2024-04-04T06:12:36.433959+0000","last_became_active":"2024-04-04T06:12:36.426029+0000","last_became_peered":"2024-04-04T06:12:36.426029+0000","last_unstale":"2024-04-04T06:12:36.433959+0000","last_undegraded":"2024-04-04T06:12:36.433959+0000","last_fullsized":"2024-04-04T06:12:36.433959+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:12:14.379165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.410853+0000","last_change":"2024-04-04T06:12:35.417056+0000","last_active":"2024-04-04T06:12:36.410853+0000","last_peered":"2024-04-04T06:12:36.410853+0000","last_clean":"2024-04-04T06:12:36.410853+0000","last_became_active":"2024-04-04T06:12:35.416773+0000","last_became_peered":"2024-04-04T06:12:35.416773+0000","last_unstale":"2024-04-04T06:12:36.410853+0000","last_undegraded":"2024-04-04T06:12:36.410853+0000","last_fullsized":"2024-04-04T06:12:36.410853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:29:03.481599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.436680+0000","last_change":"2024-04-04T06:12:36.434451+0000","last_active":"2024-04-04T06:12:36.436680+0000","last_peered":"2024-04-04T06:12:36.436680+0000","last_clean":"2024-04-04T06:12:36.436680+0000","last_became_active":"2024-04-04T06:12:36.433588+0000","last_became_peered":"2024-04-04T06:12:36.433588+0000","last_unstale":"2024-04-04T06:12:36.436680+0000","last_undegraded":"2024-04-04T06:12:36.436680+0000","last_fullsized":"2024-04-04T06:12:36.436680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:22:56.841646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.416531+0000","last_change":"2024-04-04T06:12:35.435712+0000","last_active":"2024-04-04T06:12:36.416531+0000","last_peered":"2024-04-04T06:12:36.416531+0000","last_clean":"2024-04-04T06:12:36.416531+0000","last_became_active":"2024-04-04T06:12:35.435056+0000","last_became_peered":"2024-04-04T06:12:35.435056+0000","last_unstale":"2024-04-04T06:12:36.416531+0000","last_undegraded":"2024-04-04T06:12:36.416531+0000","last_fullsized":"2024-04-04T06:12:36.416531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:53:57.237782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.062636+0000","last_change":"2024-04-04T06:12:36.439612+0000","last_active":"2024-04-04T06:12:38.062636+0000","last_peered":"2024-04-04T06:12:38.062636+0000","last_clean":"2024-04-04T06:12:38.062636+0000","last_became_active":"2024-04-04T06:12:36.439332+0000","last_became_peered":"2024-04-04T06:12:36.439332+0000","last_unstale":"2024-04-04T06:12:38.062636+0000","last_undegraded":"2024-04-04T06:12:38.062636+0000","last_fullsized":"2024-04-04T06:12:38.062636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:58:10.105371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.414074+0000","last_change":"2024-04-04T06:12:35.444842+0000","last_active":"2024-04-04T06:12:36.414074+0000","last_peered":"2024-04-04T06:12:36.414074+0000","last_clean":"2024-04-04T06:12:36.414074+0000","last_became_active":"2024-04-04T06:12:35.444476+0000","last_became_peered":"2024-04-04T06:12:35.444476+0000","last_unstale":"2024-04-04T06:12:36.414074+0000","last_undegraded":"2024-04-04T06:12:36.414074+0000","last_fullsized":"2024-04-04T06:12:36.414074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:43:16.886563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.058760+0000","last_change":"2024-04-04T06:12:36.418565+0000","last_active":"2024-04-04T06:12:38.058760+0000","last_peered":"2024-04-04T06:12:38.058760+0000","last_clean":"2024-04-04T06:12:38.058760+0000","last_became_active":"2024-04-04T06:12:36.417986+0000","last_became_peered":"2024-04-04T06:12:36.417986+0000","last_unstale":"2024-04-04T06:12:38.058760+0000","last_undegraded":"2024-04-04T06:12:38.058760+0000","last_fullsized":"2024-04-04T06:12:38.058760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:40:24.578340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.428339+0000","last_change":"2024-04-04T06:12:35.438426+0000","last_active":"2024-04-04T06:12:36.428339+0000","last_peered":"2024-04-04T06:12:36.428339+0000","last_clean":"2024-04-04T06:12:36.428339+0000","last_became_active":"2024-04-04T06:12:35.437875+0000","last_became_peered":"2024-04-04T06:12:35.437875+0000","last_unstale":"2024-04-04T06:12:36.428339+0000","last_undegraded":"2024-04-04T06:12:36.428339+0000","last_fullsized":"2024-04-04T06:12:36.428339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:40:23.563046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.435703+0000","last_change":"2024-04-04T06:12:36.433461+0000","last_active":"2024-04-04T06:12:36.435703+0000","last_peered":"2024-04-04T06:12:36.435703+0000","last_clean":"2024-04-04T06:12:36.435703+0000","last_became_active":"2024-04-04T06:12:36.433074+0000","last_became_peered":"2024-04-04T06:12:36.433074+0000","last_unstale":"2024-04-04T06:12:36.435703+0000","last_undegraded":"2024-04-04T06:12:36.435703+0000","last_fullsized":"2024-04-04T06:12:36.435703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:01:36.178837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.060465+0000","last_change":"2024-04-04T06:12:35.427940+0000","last_active":"2024-04-04T06:12:38.060465+0000","last_peered":"2024-04-04T06:12:38.060465+0000","last_clean":"2024-04-04T06:12:38.060465+0000","last_became_active":"2024-04-04T06:12:35.424233+0000","last_became_peered":"2024-04-04T06:12:35.424233+0000","last_unstale":"2024-04-04T06:12:38.060465+0000","last_undegraded":"2024-04-04T06:12:38.060465+0000","last_fullsized":"2024-04-04T06:12:38.060465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:19:17.572260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.437040+0000","last_change":"2024-04-04T06:12:36.430526+0000","last_active":"2024-04-04T06:12:36.437040+0000","last_peered":"2024-04-04T06:12:36.437040+0000","last_clean":"2024-04-04T06:12:36.437040+0000","last_became_active":"2024-04-04T06:12:36.430094+0000","last_became_peered":"2024-04-04T06:12:36.430094+0000","last_unstale":"2024-04-04T06:12:36.437040+0000","last_undegraded":"2024-04-04T06:12:36.437040+0000","last_fullsized":"2024-04-04T06:12:36.437040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:22:56.841646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.417604+0000","last_change":"2024-04-04T06:12:35.442216+0000","last_active":"2024-04-04T06:12:36.417604+0000","last_peered":"2024-04-04T06:12:36.417604+0000","last_clean":"2024-04-04T06:12:36.417604+0000","last_became_active":"2024-04-04T06:12:35.441786+0000","last_became_peered":"2024-04-04T06:12:35.441786+0000","last_unstale":"2024-04-04T06:12:36.417604+0000","last_undegraded":"2024-04-04T06:12:36.417604+0000","last_fullsized":"2024-04-04T06:12:36.417604+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:47:34.083172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.436629+0000","last_change":"2024-04-04T06:12:36.435311+0000","last_active":"2024-04-04T06:12:36.436629+0000","last_peered":"2024-04-04T06:12:36.436629+0000","last_clean":"2024-04-04T06:12:36.436629+0000","last_became_active":"2024-04-04T06:12:36.434840+0000","last_became_peered":"2024-04-04T06:12:36.434840+0000","last_unstale":"2024-04-04T06:12:36.436629+0000","last_undegraded":"2024-04-04T06:12:36.436629+0000","last_fullsized":"2024-04-04T06:12:36.436629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:32:08.355924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.059636+0000","last_change":"2024-04-04T06:12:35.436134+0000","last_active":"2024-04-04T06:12:38.059636+0000","last_peered":"2024-04-04T06:12:38.059636+0000","last_clean":"2024-04-04T06:12:38.059636+0000","last_became_active":"2024-04-04T06:12:35.435855+0000","last_became_peered":"2024-04-04T06:12:35.435855+0000","last_unstale":"2024-04-04T06:12:38.059636+0000","last_undegraded":"2024-04-04T06:12:38.059636+0000","last_fullsized":"2024-04-04T06:12:38.059636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:07:44.485093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.441172+0000","last_change":"2024-04-04T06:12:36.426065+0000","last_active":"2024-04-04T06:12:36.441172+0000","last_peered":"2024-04-04T06:12:36.441172+0000","last_clean":"2024-04-04T06:12:36.441172+0000","last_became_active":"2024-04-04T06:12:36.425797+0000","last_became_peered":"2024-04-04T06:12:36.425797+0000","last_unstale":"2024-04-04T06:12:36.441172+0000","last_undegraded":"2024-04-04T06:12:36.441172+0000","last_fullsized":"2024-04-04T06:12:36.441172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:30:17.229939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"84'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.450024+0000","last_change":"2024-04-04T06:12:35.440311+0000","last_active":"2024-04-04T06:12:36.450024+0000","last_peered":"2024-04-04T06:12:36.450024+0000","last_clean":"2024-04-04T06:12:36.450024+0000","last_became_active":"2024-04-04T06:12:35.440019+0000","last_became_peered":"2024-04-04T06:12:35.440019+0000","last_unstale":"2024-04-04T06:12:36.450024+0000","last_undegraded":"2024-04-04T06:12:36.450024+0000","last_fullsized":"2024-04-04T06:12:36.450024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:41:20.018815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":284,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.407475+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:12:36.407475+0000","last_peered":"2024-04-04T06:12:36.407475+0000","last_clean":"2024-04-04T06:12:36.407475+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:12:36.407475+0000","last_undegraded":"2024-04-04T06:12:36.407475+0000","last_fullsized":"2024-04-04T06:12:36.407475+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":1020171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509527,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1036288,"data_stored":437272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509528,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1011712,"data_stored":429803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509527,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1011712,"data_stored":429803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509526,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1011712,"data_stored":429803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509528,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":1020171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542232,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1052672,"data_stored":444512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542230,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1036288,"data_stored":442878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:12:40.897 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:12:41.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.158+0000 7fb938200640 1 Processor -- start 2024-04-04T06:12:41.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.158+0000 7fb938200640 1 -- start start 2024-04-04T06:12:41.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.159+0000 7fb938200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9301480a0 0x7fb93006ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:41.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.159+0000 7fb938200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb9300726f0 0x7fb9300b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:41.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.160+0000 7fb938200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb930148480 con 0x7fb93006b570 2024-04-04T06:12:41.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.160+0000 7fb938200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fb93006e5a0 con 0x7fb9300726f0 2024-04-04T06:12:41.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.160+0000 7fb938200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb9300b41e0 con 0x7fb9301480a0 2024-04-04T06:12:41.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.160+0000 7fb934e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb9300726f0 0x7fb9300b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:41.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.160+0000 7fb934e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb9300726f0 0x7fb9300b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56700/0 (socket says 172.21.15.159:56700) 2024-04-04T06:12:41.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.160+0000 7fb934e00640 1 -- 172.21.15.159:0/1912915978 learned_addr learned my addr 172.21.15.159:0/1912915978 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:41.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.160+0000 7fb935800640 1 --2- 172.21.15.159:0/1912915978 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9301480a0 0x7fb93006ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:41.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.160+0000 7fb934e00640 1 -- 172.21.15.159:0/1912915978 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9301480a0 msgr2=0x7fb93006ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:41.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.161+0000 7fb934e00640 1 --2- 172.21.15.159:0/1912915978 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9301480a0 0x7fb93006ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:41.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.161+0000 7fb934e00640 1 -- 172.21.15.159:0/1912915978 >> v1:172.21.15.178:6789/0 conn(0x7fb93006b570 legacy=0x7fb93006ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:41.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.161+0000 7fb934e00640 1 -- 172.21.15.159:0/1912915978 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93006df50 con 0x7fb9300726f0 2024-04-04T06:12:41.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.162+0000 7fb934e00640 1 --2- 172.21.15.159:0/1912915978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9300726f0 0x7fb9300b3890 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009870 tx=0x7fb91c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16ca5e0a24d3bb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:41.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.162+0000 7fb92fe00640 1 -- 172.21.15.159:0/1912915978 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c034070 con 0x7fb9300726f0 2024-04-04T06:12:41.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.162+0000 7fb92fe00640 1 -- 172.21.15.159:0/1912915978 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb91c030840 con 0x7fb9300726f0 2024-04-04T06:12:41.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.162+0000 7fb92fe00640 1 -- 172.21.15.159:0/1912915978 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c030c10 con 0x7fb9300726f0 2024-04-04T06:12:41.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.163+0000 7fb938200640 1 -- 172.21.15.159:0/1912915978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9300726f0 msgr2=0x7fb9300b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:41.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.163+0000 7fb938200640 1 --2- 172.21.15.159:0/1912915978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9300726f0 0x7fb9300b3890 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009870 tx=0x7fb91c02b980 comp rx=0 tx=0).stop 2024-04-04T06:12:41.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.163+0000 7fb938200640 1 -- 172.21.15.159:0/1912915978 shutdown_connections 2024-04-04T06:12:41.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.163+0000 7fb938200640 1 --2- 172.21.15.159:0/1912915978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9300726f0 0x7fb9300b3890 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:41.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.163+0000 7fb938200640 1 --2- 172.21.15.159:0/1912915978 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9301480a0 0x7fb93006ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:41.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.163+0000 7fb938200640 1 -- 172.21.15.159:0/1912915978 >> 172.21.15.159:0/1912915978 conn(0x7fb9300b2230 msgr2=0x7fb9300b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:41.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.163+0000 7fb938200640 1 -- 172.21.15.159:0/1912915978 shutdown_connections 2024-04-04T06:12:41.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.163+0000 7fb938200640 1 -- 172.21.15.159:0/1912915978 wait complete. 2024-04-04T06:12:41.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.165+0000 7fb938200640 1 Processor -- start 2024-04-04T06:12:41.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.165+0000 7fb938200640 1 -- start start 2024-04-04T06:12:41.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.166+0000 7fb938200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb93006b570 0x7fb9301f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:41.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.166+0000 7fb938200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9300726f0 0x7fb9301e8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:41.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.166+0000 7fb938200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9301480a0 0x7fb9301610a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:41.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.166+0000 7fb938200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb9300b44c0 con 0x7fb93006b570 2024-04-04T06:12:41.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.166+0000 7fb938200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb93006ef70 con 0x7fb9300726f0 2024-04-04T06:12:41.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.166+0000 7fb938200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb9300b3dd0 con 0x7fb9301480a0 2024-04-04T06:12:41.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.166+0000 7fb935800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb93006b570 0x7fb9301f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:41.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.166+0000 7fb935800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb93006b570 0x7fb9301f4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56712/0 (socket says 172.21.15.159:56712) 2024-04-04T06:12:41.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.166+0000 7fb935800640 1 -- 172.21.15.159:0/2167131660 learned_addr learned my addr 172.21.15.159:0/2167131660 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:41.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.166+0000 7fb934e00640 1 --2- 172.21.15.159:0/2167131660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9300726f0 0x7fb9301e8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:41.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.166+0000 7fb935800640 1 -- 172.21.15.159:0/2167131660 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9301480a0 msgr2=0x7fb9301610a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:41.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.166+0000 7fb936200640 1 --2- 172.21.15.159:0/2167131660 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9301480a0 0x7fb9301610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:41.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.167+0000 7fb935800640 1 --2- 172.21.15.159:0/2167131660 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9301480a0 0x7fb9301610a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:41.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.167+0000 7fb935800640 1 -- 172.21.15.159:0/2167131660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9300726f0 msgr2=0x7fb9301e8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:41.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.167+0000 7fb935800640 1 --2- 172.21.15.159:0/2167131660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9300726f0 0x7fb9301e8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:41.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.167+0000 7fb935800640 1 -- 172.21.15.159:0/2167131660 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9301f49f0 con 0x7fb93006b570 2024-04-04T06:12:41.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.167+0000 7fb934e00640 1 --2- 172.21.15.159:0/2167131660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9300726f0 0x7fb9301e8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:41.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.167+0000 7fb936200640 1 --2- 172.21.15.159:0/2167131660 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9301480a0 0x7fb9301610a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:41.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.167+0000 7fb935800640 1 --2- 172.21.15.159:0/2167131660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb93006b570 0x7fb9301f4290 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb92400d5f0 tx=0x7fb92400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:41.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.167+0000 7fb92d600640 1 -- 172.21.15.159:0/2167131660 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9240125f0 con 0x7fb93006b570 2024-04-04T06:12:41.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.167+0000 7fb92d600640 1 -- 172.21.15.159:0/2167131660 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb924019070 con 0x7fb93006b570 2024-04-04T06:12:41.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.167+0000 7fb938200640 1 -- 172.21.15.159:0/2167131660 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9301f4cd0 con 0x7fb93006b570 2024-04-04T06:12:41.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.167+0000 7fb92d600640 1 -- 172.21.15.159:0/2167131660 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9240129d0 con 0x7fb93006b570 2024-04-04T06:12:41.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.167+0000 7fb938200640 1 -- 172.21.15.159:0/2167131660 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9301f5290 con 0x7fb93006b570 2024-04-04T06:12:41.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.169+0000 7fb938200640 1 -- 172.21.15.159:0/2167131660 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9300685c0 con 0x7fb93006b570 2024-04-04T06:12:41.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.169+0000 7fb92d600640 1 -- 172.21.15.159:0/2167131660 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb924012b70 con 0x7fb93006b570 2024-04-04T06:12:41.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.170+0000 7fb92d600640 1 --2- 172.21.15.159:0/2167131660 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb91006c8f0 0x7fb91006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:41.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.170+0000 7fb92d600640 1 -- 172.21.15.159:0/2167131660 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fb92408c080 con 0x7fb93006b570 2024-04-04T06:12:41.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.171+0000 7fb934e00640 1 --2- 172.21.15.159:0/2167131660 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb91006c8f0 0x7fb91006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:41.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.172+0000 7fb934e00640 1 --2- 172.21.15.159:0/2167131660 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb91006c8f0 0x7fb91006edb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb91c036000 tx=0x7fb91c03b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:41.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.175+0000 7fb92d600640 1 -- 172.21.15.159:0/2167131660 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb92405a380 con 0x7fb93006b570 2024-04-04T06:12:41.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.369+0000 7fb938200640 1 -- 172.21.15.159:0/2167131660 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb93006f530 con 0x7fb91006c8f0 2024-04-04T06:12:41.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.387+0000 7fb92d600640 1 -- 172.21.15.159:0/2167131660 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252033 (secure 0 0 0) 0x7fb93006f530 con 0x7fb91006c8f0 2024-04-04T06:12:41.383 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:41.384 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:12:41.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.393+0000 7fb938200640 1 -- 172.21.15.159:0/2167131660 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb91006c8f0 msgr2=0x7fb91006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:41.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.393+0000 7fb938200640 1 --2- 172.21.15.159:0/2167131660 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb91006c8f0 0x7fb91006edb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb91c036000 tx=0x7fb91c03b040 comp rx=0 tx=0).stop 2024-04-04T06:12:41.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.393+0000 7fb938200640 1 -- 172.21.15.159:0/2167131660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb93006b570 msgr2=0x7fb9301f4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:41.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.393+0000 7fb938200640 1 --2- 172.21.15.159:0/2167131660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb93006b570 0x7fb9301f4290 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb92400d5f0 tx=0x7fb92400dab0 comp rx=0 tx=0).stop 2024-04-04T06:12:41.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.393+0000 7fb938200640 1 -- 172.21.15.159:0/2167131660 shutdown_connections 2024-04-04T06:12:41.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.393+0000 7fb938200640 1 --2- 172.21.15.159:0/2167131660 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9301480a0 0x7fb9301610a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:41.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.393+0000 7fb938200640 1 --2- 172.21.15.159:0/2167131660 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb91006c8f0 0x7fb91006edb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:41.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.393+0000 7fb938200640 1 --2- 172.21.15.159:0/2167131660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9300726f0 0x7fb9301e8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:41.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.393+0000 7fb938200640 1 --2- 172.21.15.159:0/2167131660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb93006b570 0x7fb9301f4290 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:41.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.393+0000 7fb938200640 1 -- 172.21.15.159:0/2167131660 >> 172.21.15.159:0/2167131660 conn(0x7fb9300b2230 msgr2=0x7fb9301467c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:41.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.393+0000 7fb938200640 1 -- 172.21.15.159:0/2167131660 shutdown_connections 2024-04-04T06:12:41.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.393+0000 7fb938200640 1 -- 172.21.15.159:0/2167131660 wait complete. 2024-04-04T06:12:41.403 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-04-04T06:12:40.861888+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258608,"kb_used_data":9152,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716112,"statfs":{"total":773094113280,"available":772829298688,"internally_reserved":0,"allocated":9371648,"data_stored":4654413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001906"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.434230+0000","last_change":"2024-04-04T06:12:36.429361+0000","last_active":"2024-04-04T06:12:36.434230+0000","last_peered":"2024-04-04T06:12:36.434230+0000","last_clean":"2024-04-04T06:12:36.434230+0000","last_became_active":"2024-04-04T06:12:36.429065+0000","last_became_peered":"2024-04-04T06:12:36.429065+0000","last_unstale":"2024-04-04T06:12:36.434230+0000","last_undegraded":"2024-04-04T06:12:36.434230+0000","last_fullsized":"2024-04-04T06:12:36.434230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:13:39.813684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.434421+0000","last_change":"2024-04-04T06:12:36.429418+0000","last_active":"2024-04-04T06:12:36.434421+0000","last_peered":"2024-04-04T06:12:36.434421+0000","last_clean":"2024-04-04T06:12:36.434421+0000","last_became_active":"2024-04-04T06:12:36.428533+0000","last_became_peered":"2024-04-04T06:12:36.428533+0000","last_unstale":"2024-04-04T06:12:36.434421+0000","last_undegraded":"2024-04-04T06:12:36.434421+0000","last_fullsized":"2024-04-04T06:12:36.434421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:30:28.756189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.436201+0000","last_change":"2024-04-04T06:12:36.428611+0000","last_active":"2024-04-04T06:12:36.436201+0000","last_peered":"2024-04-04T06:12:36.436201+0000","last_clean":"2024-04-04T06:12:36.436201+0000","last_became_active":"2024-04-04T06:12:36.428312+0000","last_became_peered":"2024-04-04T06:12:36.428312+0000","last_unstale":"2024-04-04T06:12:36.436201+0000","last_undegraded":"2024-04-04T06:12:36.436201+0000","last_fullsized":"2024-04-04T06:12:36.436201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:13:39.813684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.442076+0000","last_change":"2024-04-04T06:12:36.431457+0000","last_active":"2024-04-04T06:12:36.442076+0000","last_peered":"2024-04-04T06:12:36.442076+0000","last_clean":"2024-04-04T06:12:36.442076+0000","last_became_active":"2024-04-04T06:12:36.431121+0000","last_became_peered":"2024-04-04T06:12:36.431121+0000","last_unstale":"2024-04-04T06:12:36.442076+0000","last_undegraded":"2024-04-04T06:12:36.442076+0000","last_fullsized":"2024-04-04T06:12:36.442076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:28:29.826082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.063342+0000","last_change":"2024-04-04T06:12:36.434305+0000","last_active":"2024-04-04T06:12:38.063342+0000","last_peered":"2024-04-04T06:12:38.063342+0000","last_clean":"2024-04-04T06:12:38.063342+0000","last_became_active":"2024-04-04T06:12:36.433949+0000","last_became_peered":"2024-04-04T06:12:36.433949+0000","last_unstale":"2024-04-04T06:12:38.063342+0000","last_undegraded":"2024-04-04T06:12:38.063342+0000","last_fullsized":"2024-04-04T06:12:38.063342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:59:19.683234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.058735+0000","last_change":"2024-04-04T06:12:36.436367+0000","last_active":"2024-04-04T06:12:38.058735+0000","last_peered":"2024-04-04T06:12:38.058735+0000","last_clean":"2024-04-04T06:12:38.058735+0000","last_became_active":"2024-04-04T06:12:36.436080+0000","last_became_peered":"2024-04-04T06:12:36.436080+0000","last_unstale":"2024-04-04T06:12:38.058735+0000","last_undegraded":"2024-04-04T06:12:38.058735+0000","last_fullsized":"2024-04-04T06:12:38.058735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:29:31.971752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.438552+0000","last_change":"2024-04-04T06:12:36.425375+0000","last_active":"2024-04-04T06:12:36.438552+0000","last_peered":"2024-04-04T06:12:36.438552+0000","last_clean":"2024-04-04T06:12:36.438552+0000","last_became_active":"2024-04-04T06:12:36.425040+0000","last_became_peered":"2024-04-04T06:12:36.425040+0000","last_unstale":"2024-04-04T06:12:36.438552+0000","last_undegraded":"2024-04-04T06:12:36.438552+0000","last_fullsized":"2024-04-04T06:12:36.438552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:19:15.808794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.058525+0000","last_change":"2024-04-04T06:12:36.420379+0000","last_active":"2024-04-04T06:12:38.058525+0000","last_peered":"2024-04-04T06:12:38.058525+0000","last_clean":"2024-04-04T06:12:38.058525+0000","last_became_active":"2024-04-04T06:12:36.420030+0000","last_became_peered":"2024-04-04T06:12:36.420030+0000","last_unstale":"2024-04-04T06:12:38.058525+0000","last_undegraded":"2024-04-04T06:12:38.058525+0000","last_fullsized":"2024-04-04T06:12:38.058525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:41:50.493586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.060594+0000","last_change":"2024-04-04T06:12:36.436125+0000","last_active":"2024-04-04T06:12:38.060594+0000","last_peered":"2024-04-04T06:12:38.060594+0000","last_clean":"2024-04-04T06:12:38.060594+0000","last_became_active":"2024-04-04T06:12:36.435403+0000","last_became_peered":"2024-04-04T06:12:36.435403+0000","last_unstale":"2024-04-04T06:12:38.060594+0000","last_undegraded":"2024-04-04T06:12:38.060594+0000","last_fullsized":"2024-04-04T06:12:38.060594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:27:36.646968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.433060+0000","last_change":"2024-04-04T06:12:36.427224+0000","last_active":"2024-04-04T06:12:36.433060+0000","last_peered":"2024-04-04T06:12:36.433060+0000","last_clean":"2024-04-04T06:12:36.433060+0000","last_became_active":"2024-04-04T06:12:36.426803+0000","last_became_peered":"2024-04-04T06:12:36.426803+0000","last_unstale":"2024-04-04T06:12:36.433060+0000","last_undegraded":"2024-04-04T06:12:36.433060+0000","last_fullsized":"2024-04-04T06:12:36.433060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:04:38.131623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.434827+0000","last_change":"2024-04-04T06:12:36.430900+0000","last_active":"2024-04-04T06:12:36.434827+0000","last_peered":"2024-04-04T06:12:36.434827+0000","last_clean":"2024-04-04T06:12:36.434827+0000","last_became_active":"2024-04-04T06:12:36.430649+0000","last_became_peered":"2024-04-04T06:12:36.430649+0000","last_unstale":"2024-04-04T06:12:36.434827+0000","last_undegraded":"2024-04-04T06:12:36.434827+0000","last_fullsized":"2024-04-04T06:12:36.434827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:37:59.657044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.432894+0000","last_change":"2024-04-04T06:12:36.426862+0000","last_active":"2024-04-04T06:12:36.432894+0000","last_peered":"2024-04-04T06:12:36.432894+0000","last_clean":"2024-04-04T06:12:36.432894+0000","last_became_active":"2024-04-04T06:12:36.426289+0000","last_became_peered":"2024-04-04T06:12:36.426289+0000","last_unstale":"2024-04-04T06:12:36.432894+0000","last_undegraded":"2024-04-04T06:12:36.432894+0000","last_fullsized":"2024-04-04T06:12:36.432894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:29:18.446245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.436663+0000","last_change":"2024-04-04T06:12:36.428368+0000","last_active":"2024-04-04T06:12:36.436663+0000","last_peered":"2024-04-04T06:12:36.436663+0000","last_clean":"2024-04-04T06:12:36.436663+0000","last_became_active":"2024-04-04T06:12:36.427313+0000","last_became_peered":"2024-04-04T06:12:36.427313+0000","last_unstale":"2024-04-04T06:12:36.436663+0000","last_undegraded":"2024-04-04T06:12:36.436663+0000","last_fullsized":"2024-04-04T06:12:36.436663+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:04:38.131623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.442036+0000","last_change":"2024-04-04T06:12:36.430884+0000","last_active":"2024-04-04T06:12:36.442036+0000","last_peered":"2024-04-04T06:12:36.442036+0000","last_clean":"2024-04-04T06:12:36.442036+0000","last_became_active":"2024-04-04T06:12:36.430628+0000","last_became_peered":"2024-04-04T06:12:36.430628+0000","last_unstale":"2024-04-04T06:12:36.442036+0000","last_undegraded":"2024-04-04T06:12:36.442036+0000","last_fullsized":"2024-04-04T06:12:36.442036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:01:44.158186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.419870+0000","last_change":"2024-04-04T06:12:36.414722+0000","last_active":"2024-04-04T06:12:36.419870+0000","last_peered":"2024-04-04T06:12:36.419870+0000","last_clean":"2024-04-04T06:12:36.419870+0000","last_became_active":"2024-04-04T06:12:36.414385+0000","last_became_peered":"2024-04-04T06:12:36.414385+0000","last_unstale":"2024-04-04T06:12:36.419870+0000","last_undegraded":"2024-04-04T06:12:36.419870+0000","last_fullsized":"2024-04-04T06:12:36.419870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:48:45.291380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.059610+0000","last_change":"2024-04-04T06:12:36.428882+0000","last_active":"2024-04-04T06:12:38.059610+0000","last_peered":"2024-04-04T06:12:38.059610+0000","last_clean":"2024-04-04T06:12:38.059610+0000","last_became_active":"2024-04-04T06:12:36.428405+0000","last_became_peered":"2024-04-04T06:12:36.428405+0000","last_unstale":"2024-04-04T06:12:38.059610+0000","last_undegraded":"2024-04-04T06:12:38.059610+0000","last_fullsized":"2024-04-04T06:12:38.059610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:54:44.714523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.063578+0000","last_change":"2024-04-04T06:12:36.441905+0000","last_active":"2024-04-04T06:12:38.063578+0000","last_peered":"2024-04-04T06:12:38.063578+0000","last_clean":"2024-04-04T06:12:38.063578+0000","last_became_active":"2024-04-04T06:12:36.441543+0000","last_became_peered":"2024-04-04T06:12:36.441543+0000","last_unstale":"2024-04-04T06:12:38.063578+0000","last_undegraded":"2024-04-04T06:12:38.063578+0000","last_fullsized":"2024-04-04T06:12:38.063578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:22:23.086185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.438410+0000","last_change":"2024-04-04T06:12:36.426213+0000","last_active":"2024-04-04T06:12:36.438410+0000","last_peered":"2024-04-04T06:12:36.438410+0000","last_clean":"2024-04-04T06:12:36.438410+0000","last_became_active":"2024-04-04T06:12:36.425915+0000","last_became_peered":"2024-04-04T06:12:36.425915+0000","last_unstale":"2024-04-04T06:12:36.438410+0000","last_undegraded":"2024-04-04T06:12:36.438410+0000","last_fullsized":"2024-04-04T06:12:36.438410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:23:30.039862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.437316+0000","last_change":"2024-04-04T06:12:36.434242+0000","last_active":"2024-04-04T06:12:36.437316+0000","last_peered":"2024-04-04T06:12:36.437316+0000","last_clean":"2024-04-04T06:12:36.437316+0000","last_became_active":"2024-04-04T06:12:36.433035+0000","last_became_peered":"2024-04-04T06:12:36.433035+0000","last_unstale":"2024-04-04T06:12:36.437316+0000","last_undegraded":"2024-04-04T06:12:36.437316+0000","last_fullsized":"2024-04-04T06:12:36.437316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:08:49.667260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.445353+0000","last_change":"2024-04-04T06:12:36.441713+0000","last_active":"2024-04-04T06:12:36.445353+0000","last_peered":"2024-04-04T06:12:36.445353+0000","last_clean":"2024-04-04T06:12:36.445353+0000","last_became_active":"2024-04-04T06:12:36.441413+0000","last_became_peered":"2024-04-04T06:12:36.441413+0000","last_unstale":"2024-04-04T06:12:36.445353+0000","last_undegraded":"2024-04-04T06:12:36.445353+0000","last_fullsized":"2024-04-04T06:12:36.445353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:34:44.792603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.443680+0000","last_change":"2024-04-04T06:12:36.436241+0000","last_active":"2024-04-04T06:12:36.443680+0000","last_peered":"2024-04-04T06:12:36.443680+0000","last_clean":"2024-04-04T06:12:36.443680+0000","last_became_active":"2024-04-04T06:12:36.435331+0000","last_became_peered":"2024-04-04T06:12:36.435331+0000","last_unstale":"2024-04-04T06:12:36.443680+0000","last_undegraded":"2024-04-04T06:12:36.443680+0000","last_fullsized":"2024-04-04T06:12:36.443680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:57:35.026821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.446667+0000","last_change":"2024-04-04T06:12:36.440344+0000","last_active":"2024-04-04T06:12:36.446667+0000","last_peered":"2024-04-04T06:12:36.446667+0000","last_clean":"2024-04-04T06:12:36.446667+0000","last_became_active":"2024-04-04T06:12:36.440027+0000","last_became_peered":"2024-04-04T06:12:36.440027+0000","last_unstale":"2024-04-04T06:12:36.446667+0000","last_undegraded":"2024-04-04T06:12:36.446667+0000","last_fullsized":"2024-04-04T06:12:36.446667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:59:47.453213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.439762+0000","last_change":"2024-04-04T06:12:36.437622+0000","last_active":"2024-04-04T06:12:36.439762+0000","last_peered":"2024-04-04T06:12:36.439762+0000","last_clean":"2024-04-04T06:12:36.439762+0000","last_became_active":"2024-04-04T06:12:36.437315+0000","last_became_peered":"2024-04-04T06:12:36.437315+0000","last_unstale":"2024-04-04T06:12:36.439762+0000","last_undegraded":"2024-04-04T06:12:36.439762+0000","last_fullsized":"2024-04-04T06:12:36.439762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:39:04.274978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.443583+0000","last_change":"2024-04-04T06:12:36.431767+0000","last_active":"2024-04-04T06:12:36.443583+0000","last_peered":"2024-04-04T06:12:36.443583+0000","last_clean":"2024-04-04T06:12:36.443583+0000","last_became_active":"2024-04-04T06:12:36.431484+0000","last_became_peered":"2024-04-04T06:12:36.431484+0000","last_unstale":"2024-04-04T06:12:36.443583+0000","last_undegraded":"2024-04-04T06:12:36.443583+0000","last_fullsized":"2024-04-04T06:12:36.443583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:45:59.919636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.432304+0000","last_change":"2024-04-04T06:12:36.427797+0000","last_active":"2024-04-04T06:12:36.432304+0000","last_peered":"2024-04-04T06:12:36.432304+0000","last_clean":"2024-04-04T06:12:36.432304+0000","last_became_active":"2024-04-04T06:12:36.427540+0000","last_became_peered":"2024-04-04T06:12:36.427540+0000","last_unstale":"2024-04-04T06:12:36.432304+0000","last_undegraded":"2024-04-04T06:12:36.432304+0000","last_fullsized":"2024-04-04T06:12:36.432304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:05:06.923430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.437086+0000","last_change":"2024-04-04T06:12:36.430661+0000","last_active":"2024-04-04T06:12:36.437086+0000","last_peered":"2024-04-04T06:12:36.437086+0000","last_clean":"2024-04-04T06:12:36.437086+0000","last_became_active":"2024-04-04T06:12:36.430311+0000","last_became_peered":"2024-04-04T06:12:36.430311+0000","last_unstale":"2024-04-04T06:12:36.437086+0000","last_undegraded":"2024-04-04T06:12:36.437086+0000","last_fullsized":"2024-04-04T06:12:36.437086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:32:08.355924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.061512+0000","last_change":"2024-04-04T06:12:36.429705+0000","last_active":"2024-04-04T06:12:38.061512+0000","last_peered":"2024-04-04T06:12:38.061512+0000","last_clean":"2024-04-04T06:12:38.061512+0000","last_became_active":"2024-04-04T06:12:36.428873+0000","last_became_peered":"2024-04-04T06:12:36.428873+0000","last_unstale":"2024-04-04T06:12:38.061512+0000","last_undegraded":"2024-04-04T06:12:38.061512+0000","last_fullsized":"2024-04-04T06:12:38.061512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:56:54.519484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.432658+0000","last_change":"2024-04-04T06:12:36.426588+0000","last_active":"2024-04-04T06:12:36.432658+0000","last_peered":"2024-04-04T06:12:36.432658+0000","last_clean":"2024-04-04T06:12:36.432658+0000","last_became_active":"2024-04-04T06:12:36.425911+0000","last_became_peered":"2024-04-04T06:12:36.425911+0000","last_unstale":"2024-04-04T06:12:36.432658+0000","last_undegraded":"2024-04-04T06:12:36.432658+0000","last_fullsized":"2024-04-04T06:12:36.432658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:41:21.034109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.440817+0000","last_change":"2024-04-04T06:12:36.432535+0000","last_active":"2024-04-04T06:12:36.440817+0000","last_peered":"2024-04-04T06:12:36.440817+0000","last_clean":"2024-04-04T06:12:36.440817+0000","last_became_active":"2024-04-04T06:12:36.429964+0000","last_became_peered":"2024-04-04T06:12:36.429964+0000","last_unstale":"2024-04-04T06:12:36.440817+0000","last_undegraded":"2024-04-04T06:12:36.440817+0000","last_fullsized":"2024-04-04T06:12:36.440817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:11:09.469338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.442505+0000","last_change":"2024-04-04T06:12:36.434144+0000","last_active":"2024-04-04T06:12:36.442505+0000","last_peered":"2024-04-04T06:12:36.442505+0000","last_clean":"2024-04-04T06:12:36.442505+0000","last_became_active":"2024-04-04T06:12:36.433845+0000","last_became_peered":"2024-04-04T06:12:36.433845+0000","last_unstale":"2024-04-04T06:12:36.442505+0000","last_undegraded":"2024-04-04T06:12:36.442505+0000","last_fullsized":"2024-04-04T06:12:36.442505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:49:19.575270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.059887+0000","last_change":"2024-04-04T06:12:36.442360+0000","last_active":"2024-04-04T06:12:38.059887+0000","last_peered":"2024-04-04T06:12:38.059887+0000","last_clean":"2024-04-04T06:12:38.059887+0000","last_became_active":"2024-04-04T06:12:36.440745+0000","last_became_peered":"2024-04-04T06:12:36.440745+0000","last_unstale":"2024-04-04T06:12:38.059887+0000","last_undegraded":"2024-04-04T06:12:38.059887+0000","last_fullsized":"2024-04-04T06:12:38.059887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:52:40.608336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.441578+0000","last_change":"2024-04-04T06:12:36.440242+0000","last_active":"2024-04-04T06:12:36.441578+0000","last_peered":"2024-04-04T06:12:36.441578+0000","last_clean":"2024-04-04T06:12:36.441578+0000","last_became_active":"2024-04-04T06:12:36.439968+0000","last_became_peered":"2024-04-04T06:12:36.439968+0000","last_unstale":"2024-04-04T06:12:36.441578+0000","last_undegraded":"2024-04-04T06:12:36.441578+0000","last_fullsized":"2024-04-04T06:12:36.441578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:32:06.609995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.063419+0000","last_change":"2024-04-04T06:12:36.427890+0000","last_active":"2024-04-04T06:12:38.063419+0000","last_peered":"2024-04-04T06:12:38.063419+0000","last_clean":"2024-04-04T06:12:38.063419+0000","last_became_active":"2024-04-04T06:12:36.427578+0000","last_became_peered":"2024-04-04T06:12:36.427578+0000","last_unstale":"2024-04-04T06:12:38.063419+0000","last_undegraded":"2024-04-04T06:12:38.063419+0000","last_fullsized":"2024-04-04T06:12:38.063419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:48:45.291380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"84'4","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.451841+0000","last_change":"2024-04-04T06:12:35.440989+0000","last_active":"2024-04-04T06:12:36.451841+0000","last_peered":"2024-04-04T06:12:36.451841+0000","last_clean":"2024-04-04T06:12:36.451841+0000","last_became_active":"2024-04-04T06:12:35.440157+0000","last_became_peered":"2024-04-04T06:12:35.440157+0000","last_unstale":"2024-04-04T06:12:36.451841+0000","last_undegraded":"2024-04-04T06:12:36.451841+0000","last_fullsized":"2024-04-04T06:12:36.451841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:12:25.822017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.438905+0000","last_change":"2024-04-04T06:12:36.435554+0000","last_active":"2024-04-04T06:12:36.438905+0000","last_peered":"2024-04-04T06:12:36.438905+0000","last_clean":"2024-04-04T06:12:36.438905+0000","last_became_active":"2024-04-04T06:12:36.435244+0000","last_became_peered":"2024-04-04T06:12:36.435244+0000","last_unstale":"2024-04-04T06:12:36.438905+0000","last_undegraded":"2024-04-04T06:12:36.438905+0000","last_fullsized":"2024-04-04T06:12:36.438905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:40:32.107666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.063747+0000","last_change":"2024-04-04T06:12:35.429074+0000","last_active":"2024-04-04T06:12:38.063747+0000","last_peered":"2024-04-04T06:12:38.063747+0000","last_clean":"2024-04-04T06:12:38.063747+0000","last_became_active":"2024-04-04T06:12:35.428641+0000","last_became_peered":"2024-04-04T06:12:35.428641+0000","last_unstale":"2024-04-04T06:12:38.063747+0000","last_undegraded":"2024-04-04T06:12:38.063747+0000","last_fullsized":"2024-04-04T06:12:38.063747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:23:56.106689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.439006+0000","last_change":"2024-04-04T06:12:36.435609+0000","last_active":"2024-04-04T06:12:36.439006+0000","last_peered":"2024-04-04T06:12:36.439006+0000","last_clean":"2024-04-04T06:12:36.439006+0000","last_became_active":"2024-04-04T06:12:36.435055+0000","last_became_peered":"2024-04-04T06:12:36.435055+0000","last_unstale":"2024-04-04T06:12:36.439006+0000","last_undegraded":"2024-04-04T06:12:36.439006+0000","last_fullsized":"2024-04-04T06:12:36.439006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:44:52.660138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"84'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.452259+0000","last_change":"2024-04-04T06:12:35.423780+0000","last_active":"2024-04-04T06:12:36.452259+0000","last_peered":"2024-04-04T06:12:36.452259+0000","last_clean":"2024-04-04T06:12:36.452259+0000","last_became_active":"2024-04-04T06:12:35.423493+0000","last_became_peered":"2024-04-04T06:12:35.423493+0000","last_unstale":"2024-04-04T06:12:36.452259+0000","last_undegraded":"2024-04-04T06:12:36.452259+0000","last_fullsized":"2024-04-04T06:12:36.452259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:11:49.111105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.439714+0000","last_change":"2024-04-04T06:12:36.437860+0000","last_active":"2024-04-04T06:12:36.439714+0000","last_peered":"2024-04-04T06:12:36.439714+0000","last_clean":"2024-04-04T06:12:36.439714+0000","last_became_active":"2024-04-04T06:12:36.437583+0000","last_became_peered":"2024-04-04T06:12:36.437583+0000","last_unstale":"2024-04-04T06:12:36.439714+0000","last_undegraded":"2024-04-04T06:12:36.439714+0000","last_fullsized":"2024-04-04T06:12:36.439714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:39:04.274978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"84'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.452281+0000","last_change":"2024-04-04T06:12:35.427149+0000","last_active":"2024-04-04T06:12:36.452281+0000","last_peered":"2024-04-04T06:12:36.452281+0000","last_clean":"2024-04-04T06:12:36.452281+0000","last_became_active":"2024-04-04T06:12:35.426845+0000","last_became_peered":"2024-04-04T06:12:35.426845+0000","last_unstale":"2024-04-04T06:12:36.452281+0000","last_undegraded":"2024-04-04T06:12:36.452281+0000","last_fullsized":"2024-04-04T06:12:36.452281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:41:49.478293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.432321+0000","last_change":"2024-04-04T06:12:36.424823+0000","last_active":"2024-04-04T06:12:36.432321+0000","last_peered":"2024-04-04T06:12:36.432321+0000","last_clean":"2024-04-04T06:12:36.432321+0000","last_became_active":"2024-04-04T06:12:36.424467+0000","last_became_peered":"2024-04-04T06:12:36.424467+0000","last_unstale":"2024-04-04T06:12:36.432321+0000","last_undegraded":"2024-04-04T06:12:36.432321+0000","last_fullsized":"2024-04-04T06:12:36.432321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:27:12.212980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.063735+0000","last_change":"2024-04-04T06:12:35.420818+0000","last_active":"2024-04-04T06:12:38.063735+0000","last_peered":"2024-04-04T06:12:38.063735+0000","last_clean":"2024-04-04T06:12:38.063735+0000","last_became_active":"2024-04-04T06:12:35.420221+0000","last_became_peered":"2024-04-04T06:12:35.420221+0000","last_unstale":"2024-04-04T06:12:38.063735+0000","last_undegraded":"2024-04-04T06:12:38.063735+0000","last_fullsized":"2024-04-04T06:12:38.063735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:59:26.700419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.063524+0000","last_change":"2024-04-04T06:12:36.443098+0000","last_active":"2024-04-04T06:12:38.063524+0000","last_peered":"2024-04-04T06:12:38.063524+0000","last_clean":"2024-04-04T06:12:38.063524+0000","last_became_active":"2024-04-04T06:12:36.441843+0000","last_became_peered":"2024-04-04T06:12:36.441843+0000","last_unstale":"2024-04-04T06:12:38.063524+0000","last_undegraded":"2024-04-04T06:12:38.063524+0000","last_fullsized":"2024-04-04T06:12:38.063524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:09:16.618599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.059902+0000","last_change":"2024-04-04T06:12:35.425083+0000","last_active":"2024-04-04T06:12:38.059902+0000","last_peered":"2024-04-04T06:12:38.059902+0000","last_clean":"2024-04-04T06:12:38.059902+0000","last_became_active":"2024-04-04T06:12:35.423680+0000","last_became_peered":"2024-04-04T06:12:35.423680+0000","last_unstale":"2024-04-04T06:12:38.059902+0000","last_undegraded":"2024-04-04T06:12:38.059902+0000","last_fullsized":"2024-04-04T06:12:38.059902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:44:49.872004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.439615+0000","last_change":"2024-04-04T06:12:36.436841+0000","last_active":"2024-04-04T06:12:36.439615+0000","last_peered":"2024-04-04T06:12:36.439615+0000","last_clean":"2024-04-04T06:12:36.439615+0000","last_became_active":"2024-04-04T06:12:36.436303+0000","last_became_peered":"2024-04-04T06:12:36.436303+0000","last_unstale":"2024-04-04T06:12:36.439615+0000","last_undegraded":"2024-04-04T06:12:36.439615+0000","last_fullsized":"2024-04-04T06:12:36.439615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:40:32.107666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.426452+0000","last_change":"2024-04-04T06:12:35.428726+0000","last_active":"2024-04-04T06:12:36.426452+0000","last_peered":"2024-04-04T06:12:36.426452+0000","last_clean":"2024-04-04T06:12:36.426452+0000","last_became_active":"2024-04-04T06:12:35.428382+0000","last_became_peered":"2024-04-04T06:12:35.428382+0000","last_unstale":"2024-04-04T06:12:36.426452+0000","last_undegraded":"2024-04-04T06:12:36.426452+0000","last_fullsized":"2024-04-04T06:12:36.426452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:59:26.700419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.419677+0000","last_change":"2024-04-04T06:12:36.413565+0000","last_active":"2024-04-04T06:12:36.419677+0000","last_peered":"2024-04-04T06:12:36.419677+0000","last_clean":"2024-04-04T06:12:36.419677+0000","last_became_active":"2024-04-04T06:12:36.412693+0000","last_became_peered":"2024-04-04T06:12:36.412693+0000","last_unstale":"2024-04-04T06:12:36.419677+0000","last_undegraded":"2024-04-04T06:12:36.419677+0000","last_fullsized":"2024-04-04T06:12:36.419677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:59:19.683234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.408892+0000","last_change":"2024-04-04T06:12:35.425315+0000","last_active":"2024-04-04T06:12:36.408892+0000","last_peered":"2024-04-04T06:12:36.408892+0000","last_clean":"2024-04-04T06:12:36.408892+0000","last_became_active":"2024-04-04T06:12:35.425003+0000","last_became_peered":"2024-04-04T06:12:35.425003+0000","last_unstale":"2024-04-04T06:12:36.408892+0000","last_undegraded":"2024-04-04T06:12:36.408892+0000","last_fullsized":"2024-04-04T06:12:36.408892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:11:16.547070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.425676+0000","last_change":"2024-04-04T06:12:36.421159+0000","last_active":"2024-04-04T06:12:36.425676+0000","last_peered":"2024-04-04T06:12:36.425676+0000","last_clean":"2024-04-04T06:12:36.425676+0000","last_became_active":"2024-04-04T06:12:36.420884+0000","last_became_peered":"2024-04-04T06:12:36.420884+0000","last_unstale":"2024-04-04T06:12:36.425676+0000","last_undegraded":"2024-04-04T06:12:36.425676+0000","last_fullsized":"2024-04-04T06:12:36.425676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:10:20.346535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.418978+0000","last_change":"2024-04-04T06:12:35.441361+0000","last_active":"2024-04-04T06:12:36.418978+0000","last_peered":"2024-04-04T06:12:36.418978+0000","last_clean":"2024-04-04T06:12:36.418978+0000","last_became_active":"2024-04-04T06:12:35.440782+0000","last_became_peered":"2024-04-04T06:12:35.440782+0000","last_unstale":"2024-04-04T06:12:36.418978+0000","last_undegraded":"2024-04-04T06:12:36.418978+0000","last_fullsized":"2024-04-04T06:12:36.418978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:29:02.762166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.060634+0000","last_change":"2024-04-04T06:12:36.428355+0000","last_active":"2024-04-04T06:12:38.060634+0000","last_peered":"2024-04-04T06:12:38.060634+0000","last_clean":"2024-04-04T06:12:38.060634+0000","last_became_active":"2024-04-04T06:12:36.428064+0000","last_became_peered":"2024-04-04T06:12:36.428064+0000","last_unstale":"2024-04-04T06:12:38.060634+0000","last_undegraded":"2024-04-04T06:12:38.060634+0000","last_fullsized":"2024-04-04T06:12:38.060634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:08:49.667260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.417173+0000","last_change":"2024-04-04T06:12:35.437893+0000","last_active":"2024-04-04T06:12:36.417173+0000","last_peered":"2024-04-04T06:12:36.417173+0000","last_clean":"2024-04-04T06:12:36.417173+0000","last_became_active":"2024-04-04T06:12:35.435814+0000","last_became_peered":"2024-04-04T06:12:35.435814+0000","last_unstale":"2024-04-04T06:12:36.417173+0000","last_undegraded":"2024-04-04T06:12:36.417173+0000","last_fullsized":"2024-04-04T06:12:36.417173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:47:59.871101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.447797+0000","last_change":"2024-04-04T06:12:36.438570+0000","last_active":"2024-04-04T06:12:36.447797+0000","last_peered":"2024-04-04T06:12:36.447797+0000","last_clean":"2024-04-04T06:12:36.447797+0000","last_became_active":"2024-04-04T06:12:36.438245+0000","last_became_peered":"2024-04-04T06:12:36.438245+0000","last_unstale":"2024-04-04T06:12:36.447797+0000","last_undegraded":"2024-04-04T06:12:36.447797+0000","last_fullsized":"2024-04-04T06:12:36.447797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:11:35.711672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"84'3","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.451596+0000","last_change":"2024-04-04T06:12:35.436052+0000","last_active":"2024-04-04T06:12:36.451596+0000","last_peered":"2024-04-04T06:12:36.451596+0000","last_clean":"2024-04-04T06:12:36.451596+0000","last_became_active":"2024-04-04T06:12:35.435387+0000","last_became_peered":"2024-04-04T06:12:35.435387+0000","last_unstale":"2024-04-04T06:12:36.451596+0000","last_undegraded":"2024-04-04T06:12:36.451596+0000","last_fullsized":"2024-04-04T06:12:36.451596+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:17:07.241351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.061894+0000","last_change":"2024-04-04T06:12:36.440179+0000","last_active":"2024-04-04T06:12:38.061894+0000","last_peered":"2024-04-04T06:12:38.061894+0000","last_clean":"2024-04-04T06:12:38.061894+0000","last_became_active":"2024-04-04T06:12:36.439899+0000","last_became_peered":"2024-04-04T06:12:36.439899+0000","last_unstale":"2024-04-04T06:12:38.061894+0000","last_undegraded":"2024-04-04T06:12:38.061894+0000","last_fullsized":"2024-04-04T06:12:38.061894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:00:29.007550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"84'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.060217+0000","last_change":"2024-04-04T06:12:35.430456+0000","last_active":"2024-04-04T06:12:38.060217+0000","last_peered":"2024-04-04T06:12:38.060217+0000","last_clean":"2024-04-04T06:12:38.060217+0000","last_became_active":"2024-04-04T06:12:35.430095+0000","last_became_peered":"2024-04-04T06:12:35.430095+0000","last_unstale":"2024-04-04T06:12:38.060217+0000","last_undegraded":"2024-04-04T06:12:38.060217+0000","last_fullsized":"2024-04-04T06:12:38.060217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:09:02.307314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.447468+0000","last_change":"2024-04-04T06:12:36.437720+0000","last_active":"2024-04-04T06:12:36.447468+0000","last_peered":"2024-04-04T06:12:36.447468+0000","last_clean":"2024-04-04T06:12:36.447468+0000","last_became_active":"2024-04-04T06:12:36.437383+0000","last_became_peered":"2024-04-04T06:12:36.437383+0000","last_unstale":"2024-04-04T06:12:36.447468+0000","last_undegraded":"2024-04-04T06:12:36.447468+0000","last_fullsized":"2024-04-04T06:12:36.447468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:27:36.646968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"84'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.456587+0000","last_change":"2024-04-04T06:12:35.417322+0000","last_active":"2024-04-04T06:12:36.456587+0000","last_peered":"2024-04-04T06:12:36.456587+0000","last_clean":"2024-04-04T06:12:36.456587+0000","last_became_active":"2024-04-04T06:12:35.417068+0000","last_became_peered":"2024-04-04T06:12:35.417068+0000","last_unstale":"2024-04-04T06:12:36.456587+0000","last_undegraded":"2024-04-04T06:12:36.456587+0000","last_fullsized":"2024-04-04T06:12:36.456587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:34:43.777310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.439678+0000","last_change":"2024-04-04T06:12:36.437309+0000","last_active":"2024-04-04T06:12:36.439678+0000","last_peered":"2024-04-04T06:12:36.439678+0000","last_clean":"2024-04-04T06:12:36.439678+0000","last_became_active":"2024-04-04T06:12:36.436858+0000","last_became_peered":"2024-04-04T06:12:36.436858+0000","last_unstale":"2024-04-04T06:12:36.439678+0000","last_undegraded":"2024-04-04T06:12:36.439678+0000","last_fullsized":"2024-04-04T06:12:36.439678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:44:52.660138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.397609+0000","last_change":"2024-04-04T06:12:35.426532+0000","last_active":"2024-04-04T06:12:36.397609+0000","last_peered":"2024-04-04T06:12:36.397609+0000","last_clean":"2024-04-04T06:12:36.397609+0000","last_became_active":"2024-04-04T06:12:35.426216+0000","last_became_peered":"2024-04-04T06:12:35.426216+0000","last_unstale":"2024-04-04T06:12:36.397609+0000","last_undegraded":"2024-04-04T06:12:36.397609+0000","last_fullsized":"2024-04-04T06:12:36.397609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:17:21.722450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.443096+0000","last_change":"2024-04-04T06:12:36.439767+0000","last_active":"2024-04-04T06:12:36.443096+0000","last_peered":"2024-04-04T06:12:36.443096+0000","last_clean":"2024-04-04T06:12:36.443096+0000","last_became_active":"2024-04-04T06:12:36.439465+0000","last_became_peered":"2024-04-04T06:12:36.439465+0000","last_unstale":"2024-04-04T06:12:36.443096+0000","last_undegraded":"2024-04-04T06:12:36.443096+0000","last_fullsized":"2024-04-04T06:12:36.443096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:51:53.080092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.058647+0000","last_change":"2024-04-04T06:12:35.435356+0000","last_active":"2024-04-04T06:12:38.058647+0000","last_peered":"2024-04-04T06:12:38.058647+0000","last_clean":"2024-04-04T06:12:38.058647+0000","last_became_active":"2024-04-04T06:12:35.433434+0000","last_became_peered":"2024-04-04T06:12:35.433434+0000","last_unstale":"2024-04-04T06:12:38.058647+0000","last_undegraded":"2024-04-04T06:12:38.058647+0000","last_fullsized":"2024-04-04T06:12:38.058647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:18:05.338746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.434660+0000","last_change":"2024-04-04T06:12:36.424564+0000","last_active":"2024-04-04T06:12:36.434660+0000","last_peered":"2024-04-04T06:12:36.434660+0000","last_clean":"2024-04-04T06:12:36.434660+0000","last_became_active":"2024-04-04T06:12:36.424262+0000","last_became_peered":"2024-04-04T06:12:36.424262+0000","last_unstale":"2024-04-04T06:12:36.434660+0000","last_undegraded":"2024-04-04T06:12:36.434660+0000","last_fullsized":"2024-04-04T06:12:36.434660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:28:47.006181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"84'8","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.451712+0000","last_change":"2024-04-04T06:12:35.432102+0000","last_active":"2024-04-04T06:12:36.451712+0000","last_peered":"2024-04-04T06:12:36.451712+0000","last_clean":"2024-04-04T06:12:36.451712+0000","last_became_active":"2024-04-04T06:12:35.431317+0000","last_became_peered":"2024-04-04T06:12:35.431317+0000","last_unstale":"2024-04-04T06:12:36.451712+0000","last_undegraded":"2024-04-04T06:12:36.451712+0000","last_fullsized":"2024-04-04T06:12:36.451712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:23:56.106689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.429988+0000","last_change":"2024-04-04T06:12:36.423474+0000","last_active":"2024-04-04T06:12:36.429988+0000","last_peered":"2024-04-04T06:12:36.429988+0000","last_clean":"2024-04-04T06:12:36.429988+0000","last_became_active":"2024-04-04T06:12:36.423174+0000","last_became_peered":"2024-04-04T06:12:36.423174+0000","last_unstale":"2024-04-04T06:12:36.429988+0000","last_undegraded":"2024-04-04T06:12:36.429988+0000","last_fullsized":"2024-04-04T06:12:36.429988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:29:18.446245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"84'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.446160+0000","last_change":"2024-04-04T06:12:35.441251+0000","last_active":"2024-04-04T06:12:36.446160+0000","last_peered":"2024-04-04T06:12:36.446160+0000","last_clean":"2024-04-04T06:12:36.446160+0000","last_became_active":"2024-04-04T06:12:35.440627+0000","last_became_peered":"2024-04-04T06:12:35.440627+0000","last_unstale":"2024-04-04T06:12:36.446160+0000","last_undegraded":"2024-04-04T06:12:36.446160+0000","last_fullsized":"2024-04-04T06:12:36.446160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:23:42.612650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.433788+0000","last_change":"2024-04-04T06:12:36.426989+0000","last_active":"2024-04-04T06:12:36.433788+0000","last_peered":"2024-04-04T06:12:36.433788+0000","last_clean":"2024-04-04T06:12:36.433788+0000","last_became_active":"2024-04-04T06:12:36.426650+0000","last_became_peered":"2024-04-04T06:12:36.426650+0000","last_unstale":"2024-04-04T06:12:36.433788+0000","last_undegraded":"2024-04-04T06:12:36.433788+0000","last_fullsized":"2024-04-04T06:12:36.433788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:12:14.379165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"84'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.449589+0000","last_change":"2024-04-04T06:12:35.432366+0000","last_active":"2024-04-04T06:12:36.449589+0000","last_peered":"2024-04-04T06:12:36.449589+0000","last_clean":"2024-04-04T06:12:36.449589+0000","last_became_active":"2024-04-04T06:12:35.431887+0000","last_became_peered":"2024-04-04T06:12:35.431887+0000","last_unstale":"2024-04-04T06:12:36.449589+0000","last_undegraded":"2024-04-04T06:12:36.449589+0000","last_fullsized":"2024-04-04T06:12:36.449589+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:29:30.956459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.062252+0000","last_change":"2024-04-04T06:12:36.433535+0000","last_active":"2024-04-04T06:12:38.062252+0000","last_peered":"2024-04-04T06:12:38.062252+0000","last_clean":"2024-04-04T06:12:38.062252+0000","last_became_active":"2024-04-04T06:12:36.433221+0000","last_became_peered":"2024-04-04T06:12:36.433221+0000","last_unstale":"2024-04-04T06:12:38.062252+0000","last_undegraded":"2024-04-04T06:12:38.062252+0000","last_fullsized":"2024-04-04T06:12:38.062252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:17:22.737743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.426913+0000","last_change":"2024-04-04T06:12:35.433632+0000","last_active":"2024-04-04T06:12:36.426913+0000","last_peered":"2024-04-04T06:12:36.426913+0000","last_clean":"2024-04-04T06:12:36.426913+0000","last_became_active":"2024-04-04T06:12:35.433367+0000","last_became_peered":"2024-04-04T06:12:35.433367+0000","last_unstale":"2024-04-04T06:12:36.426913+0000","last_undegraded":"2024-04-04T06:12:36.426913+0000","last_fullsized":"2024-04-04T06:12:36.426913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:24:50.972859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.446031+0000","last_change":"2024-04-04T06:12:36.441815+0000","last_active":"2024-04-04T06:12:36.446031+0000","last_peered":"2024-04-04T06:12:36.446031+0000","last_clean":"2024-04-04T06:12:36.446031+0000","last_became_active":"2024-04-04T06:12:36.441228+0000","last_became_peered":"2024-04-04T06:12:36.441228+0000","last_unstale":"2024-04-04T06:12:36.446031+0000","last_undegraded":"2024-04-04T06:12:36.446031+0000","last_fullsized":"2024-04-04T06:12:36.446031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:11:17.562364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"84'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.446437+0000","last_change":"2024-04-04T06:12:35.430478+0000","last_active":"2024-04-04T06:12:36.446437+0000","last_peered":"2024-04-04T06:12:36.446437+0000","last_clean":"2024-04-04T06:12:36.446437+0000","last_became_active":"2024-04-04T06:12:35.429854+0000","last_became_peered":"2024-04-04T06:12:35.429854+0000","last_unstale":"2024-04-04T06:12:36.446437+0000","last_undegraded":"2024-04-04T06:12:36.446437+0000","last_fullsized":"2024-04-04T06:12:36.446437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:58:09.090078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.058937+0000","last_change":"2024-04-04T06:12:36.422870+0000","last_active":"2024-04-04T06:12:38.058937+0000","last_peered":"2024-04-04T06:12:38.058937+0000","last_clean":"2024-04-04T06:12:38.058937+0000","last_became_active":"2024-04-04T06:12:36.422465+0000","last_became_peered":"2024-04-04T06:12:36.422465+0000","last_unstale":"2024-04-04T06:12:38.058937+0000","last_undegraded":"2024-04-04T06:12:38.058937+0000","last_fullsized":"2024-04-04T06:12:38.058937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:01:36.178837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.416819+0000","last_change":"2024-04-04T06:12:35.430066+0000","last_active":"2024-04-04T06:12:36.416819+0000","last_peered":"2024-04-04T06:12:36.416819+0000","last_clean":"2024-04-04T06:12:36.416819+0000","last_became_active":"2024-04-04T06:12:35.429780+0000","last_became_peered":"2024-04-04T06:12:35.429780+0000","last_unstale":"2024-04-04T06:12:36.416819+0000","last_undegraded":"2024-04-04T06:12:36.416819+0000","last_fullsized":"2024-04-04T06:12:36.416819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:05:40.751970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.438599+0000","last_change":"2024-04-04T06:12:36.427660+0000","last_active":"2024-04-04T06:12:36.438599+0000","last_peered":"2024-04-04T06:12:36.438599+0000","last_clean":"2024-04-04T06:12:36.438599+0000","last_became_active":"2024-04-04T06:12:36.427384+0000","last_became_peered":"2024-04-04T06:12:36.427384+0000","last_unstale":"2024-04-04T06:12:36.438599+0000","last_undegraded":"2024-04-04T06:12:36.438599+0000","last_fullsized":"2024-04-04T06:12:36.438599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:36:48.111851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.416545+0000","last_change":"2024-04-04T06:12:35.441503+0000","last_active":"2024-04-04T06:12:36.416545+0000","last_peered":"2024-04-04T06:12:36.416545+0000","last_clean":"2024-04-04T06:12:36.416545+0000","last_became_active":"2024-04-04T06:12:35.441264+0000","last_became_peered":"2024-04-04T06:12:35.441264+0000","last_unstale":"2024-04-04T06:12:36.416545+0000","last_undegraded":"2024-04-04T06:12:36.416545+0000","last_fullsized":"2024-04-04T06:12:36.416545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:49:29.627886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.058615+0000","last_change":"2024-04-04T06:12:36.424685+0000","last_active":"2024-04-04T06:12:38.058615+0000","last_peered":"2024-04-04T06:12:38.058615+0000","last_clean":"2024-04-04T06:12:38.058615+0000","last_became_active":"2024-04-04T06:12:36.423224+0000","last_became_peered":"2024-04-04T06:12:36.423224+0000","last_unstale":"2024-04-04T06:12:38.058615+0000","last_undegraded":"2024-04-04T06:12:38.058615+0000","last_fullsized":"2024-04-04T06:12:38.058615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:24:51.988152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.397454+0000","last_change":"2024-04-04T06:12:35.430335+0000","last_active":"2024-04-04T06:12:36.397454+0000","last_peered":"2024-04-04T06:12:36.397454+0000","last_clean":"2024-04-04T06:12:36.397454+0000","last_became_active":"2024-04-04T06:12:35.430048+0000","last_became_peered":"2024-04-04T06:12:35.430048+0000","last_unstale":"2024-04-04T06:12:36.397454+0000","last_undegraded":"2024-04-04T06:12:36.397454+0000","last_fullsized":"2024-04-04T06:12:36.397454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:00:27.992256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.063495+0000","last_change":"2024-04-04T06:12:36.432152+0000","last_active":"2024-04-04T06:12:38.063495+0000","last_peered":"2024-04-04T06:12:38.063495+0000","last_clean":"2024-04-04T06:12:38.063495+0000","last_became_active":"2024-04-04T06:12:36.431841+0000","last_became_peered":"2024-04-04T06:12:36.431841+0000","last_unstale":"2024-04-04T06:12:38.063495+0000","last_undegraded":"2024-04-04T06:12:38.063495+0000","last_fullsized":"2024-04-04T06:12:38.063495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:10:20.346535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"84'2","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.456400+0000","last_change":"2024-04-04T06:12:35.434599+0000","last_active":"2024-04-04T06:12:36.456400+0000","last_peered":"2024-04-04T06:12:36.456400+0000","last_clean":"2024-04-04T06:12:36.456400+0000","last_became_active":"2024-04-04T06:12:35.434290+0000","last_became_peered":"2024-04-04T06:12:35.434290+0000","last_unstale":"2024-04-04T06:12:36.456400+0000","last_undegraded":"2024-04-04T06:12:36.456400+0000","last_fullsized":"2024-04-04T06:12:36.456400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:39:18.902549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.433959+0000","last_change":"2024-04-04T06:12:36.427827+0000","last_active":"2024-04-04T06:12:36.433959+0000","last_peered":"2024-04-04T06:12:36.433959+0000","last_clean":"2024-04-04T06:12:36.433959+0000","last_became_active":"2024-04-04T06:12:36.426029+0000","last_became_peered":"2024-04-04T06:12:36.426029+0000","last_unstale":"2024-04-04T06:12:36.433959+0000","last_undegraded":"2024-04-04T06:12:36.433959+0000","last_fullsized":"2024-04-04T06:12:36.433959+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:12:14.379165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.410853+0000","last_change":"2024-04-04T06:12:35.417056+0000","last_active":"2024-04-04T06:12:36.410853+0000","last_peered":"2024-04-04T06:12:36.410853+0000","last_clean":"2024-04-04T06:12:36.410853+0000","last_became_active":"2024-04-04T06:12:35.416773+0000","last_became_peered":"2024-04-04T06:12:35.416773+0000","last_unstale":"2024-04-04T06:12:36.410853+0000","last_undegraded":"2024-04-04T06:12:36.410853+0000","last_fullsized":"2024-04-04T06:12:36.410853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:29:03.481599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.436680+0000","last_change":"2024-04-04T06:12:36.434451+0000","last_active":"2024-04-04T06:12:36.436680+0000","last_peered":"2024-04-04T06:12:36.436680+0000","last_clean":"2024-04-04T06:12:36.436680+0000","last_became_active":"2024-04-04T06:12:36.433588+0000","last_became_peered":"2024-04-04T06:12:36.433588+0000","last_unstale":"2024-04-04T06:12:36.436680+0000","last_undegraded":"2024-04-04T06:12:36.436680+0000","last_fullsized":"2024-04-04T06:12:36.436680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:22:56.841646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.416531+0000","last_change":"2024-04-04T06:12:35.435712+0000","last_active":"2024-04-04T06:12:36.416531+0000","last_peered":"2024-04-04T06:12:36.416531+0000","last_clean":"2024-04-04T06:12:36.416531+0000","last_became_active":"2024-04-04T06:12:35.435056+0000","last_became_peered":"2024-04-04T06:12:35.435056+0000","last_unstale":"2024-04-04T06:12:36.416531+0000","last_undegraded":"2024-04-04T06:12:36.416531+0000","last_fullsized":"2024-04-04T06:12:36.416531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:53:57.237782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.062636+0000","last_change":"2024-04-04T06:12:36.439612+0000","last_active":"2024-04-04T06:12:38.062636+0000","last_peered":"2024-04-04T06:12:38.062636+0000","last_clean":"2024-04-04T06:12:38.062636+0000","last_became_active":"2024-04-04T06:12:36.439332+0000","last_became_peered":"2024-04-04T06:12:36.439332+0000","last_unstale":"2024-04-04T06:12:38.062636+0000","last_undegraded":"2024-04-04T06:12:38.062636+0000","last_fullsized":"2024-04-04T06:12:38.062636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:58:10.105371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.414074+0000","last_change":"2024-04-04T06:12:35.444842+0000","last_active":"2024-04-04T06:12:36.414074+0000","last_peered":"2024-04-04T06:12:36.414074+0000","last_clean":"2024-04-04T06:12:36.414074+0000","last_became_active":"2024-04-04T06:12:35.444476+0000","last_became_peered":"2024-04-04T06:12:35.444476+0000","last_unstale":"2024-04-04T06:12:36.414074+0000","last_undegraded":"2024-04-04T06:12:36.414074+0000","last_fullsized":"2024-04-04T06:12:36.414074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:43:16.886563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.058760+0000","last_change":"2024-04-04T06:12:36.418565+0000","last_active":"2024-04-04T06:12:38.058760+0000","last_peered":"2024-04-04T06:12:38.058760+0000","last_clean":"2024-04-04T06:12:38.058760+0000","last_became_active":"2024-04-04T06:12:36.417986+0000","last_became_peered":"2024-04-04T06:12:36.417986+0000","last_unstale":"2024-04-04T06:12:38.058760+0000","last_undegraded":"2024-04-04T06:12:38.058760+0000","last_fullsized":"2024-04-04T06:12:38.058760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:40:24.578340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.428339+0000","last_change":"2024-04-04T06:12:35.438426+0000","last_active":"2024-04-04T06:12:36.428339+0000","last_peered":"2024-04-04T06:12:36.428339+0000","last_clean":"2024-04-04T06:12:36.428339+0000","last_became_active":"2024-04-04T06:12:35.437875+0000","last_became_peered":"2024-04-04T06:12:35.437875+0000","last_unstale":"2024-04-04T06:12:36.428339+0000","last_undegraded":"2024-04-04T06:12:36.428339+0000","last_fullsized":"2024-04-04T06:12:36.428339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:40:23.563046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.435703+0000","last_change":"2024-04-04T06:12:36.433461+0000","last_active":"2024-04-04T06:12:36.435703+0000","last_peered":"2024-04-04T06:12:36.435703+0000","last_clean":"2024-04-04T06:12:36.435703+0000","last_became_active":"2024-04-04T06:12:36.433074+0000","last_became_peered":"2024-04-04T06:12:36.433074+0000","last_unstale":"2024-04-04T06:12:36.435703+0000","last_undegraded":"2024-04-04T06:12:36.435703+0000","last_fullsized":"2024-04-04T06:12:36.435703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:01:36.178837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.060465+0000","last_change":"2024-04-04T06:12:35.427940+0000","last_active":"2024-04-04T06:12:38.060465+0000","last_peered":"2024-04-04T06:12:38.060465+0000","last_clean":"2024-04-04T06:12:38.060465+0000","last_became_active":"2024-04-04T06:12:35.424233+0000","last_became_peered":"2024-04-04T06:12:35.424233+0000","last_unstale":"2024-04-04T06:12:38.060465+0000","last_undegraded":"2024-04-04T06:12:38.060465+0000","last_fullsized":"2024-04-04T06:12:38.060465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:19:17.572260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.437040+0000","last_change":"2024-04-04T06:12:36.430526+0000","last_active":"2024-04-04T06:12:36.437040+0000","last_peered":"2024-04-04T06:12:36.437040+0000","last_clean":"2024-04-04T06:12:36.437040+0000","last_became_active":"2024-04-04T06:12:36.430094+0000","last_became_peered":"2024-04-04T06:12:36.430094+0000","last_unstale":"2024-04-04T06:12:36.437040+0000","last_undegraded":"2024-04-04T06:12:36.437040+0000","last_fullsized":"2024-04-04T06:12:36.437040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:22:56.841646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.417604+0000","last_change":"2024-04-04T06:12:35.442216+0000","last_active":"2024-04-04T06:12:36.417604+0000","last_peered":"2024-04-04T06:12:36.417604+0000","last_clean":"2024-04-04T06:12:36.417604+0000","last_became_active":"2024-04-04T06:12:35.441786+0000","last_became_peered":"2024-04-04T06:12:35.441786+0000","last_unstale":"2024-04-04T06:12:36.417604+0000","last_undegraded":"2024-04-04T06:12:36.417604+0000","last_fullsized":"2024-04-04T06:12:36.417604+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:47:34.083172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.436629+0000","last_change":"2024-04-04T06:12:36.435311+0000","last_active":"2024-04-04T06:12:36.436629+0000","last_peered":"2024-04-04T06:12:36.436629+0000","last_clean":"2024-04-04T06:12:36.436629+0000","last_became_active":"2024-04-04T06:12:36.434840+0000","last_became_peered":"2024-04-04T06:12:36.434840+0000","last_unstale":"2024-04-04T06:12:36.436629+0000","last_undegraded":"2024-04-04T06:12:36.436629+0000","last_fullsized":"2024-04-04T06:12:36.436629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:32:08.355924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-04T06:12:38.059636+0000","last_change":"2024-04-04T06:12:35.436134+0000","last_active":"2024-04-04T06:12:38.059636+0000","last_peered":"2024-04-04T06:12:38.059636+0000","last_clean":"2024-04-04T06:12:38.059636+0000","last_became_active":"2024-04-04T06:12:35.435855+0000","last_became_peered":"2024-04-04T06:12:35.435855+0000","last_unstale":"2024-04-04T06:12:38.059636+0000","last_undegraded":"2024-04-04T06:12:38.059636+0000","last_fullsized":"2024-04-04T06:12:38.059636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:07:44.485093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.441172+0000","last_change":"2024-04-04T06:12:36.426065+0000","last_active":"2024-04-04T06:12:36.441172+0000","last_peered":"2024-04-04T06:12:36.441172+0000","last_clean":"2024-04-04T06:12:36.441172+0000","last_became_active":"2024-04-04T06:12:36.425797+0000","last_became_peered":"2024-04-04T06:12:36.425797+0000","last_unstale":"2024-04-04T06:12:36.441172+0000","last_undegraded":"2024-04-04T06:12:36.441172+0000","last_fullsized":"2024-04-04T06:12:36.441172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:35.384585+0000","last_clean_scrub_stamp":"2024-04-04T06:12:35.384585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:30:17.229939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"84'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.450024+0000","last_change":"2024-04-04T06:12:35.440311+0000","last_active":"2024-04-04T06:12:36.450024+0000","last_peered":"2024-04-04T06:12:36.450024+0000","last_clean":"2024-04-04T06:12:36.450024+0000","last_became_active":"2024-04-04T06:12:35.440019+0000","last_became_peered":"2024-04-04T06:12:35.440019+0000","last_unstale":"2024-04-04T06:12:36.450024+0000","last_undegraded":"2024-04-04T06:12:36.450024+0000","last_fullsized":"2024-04-04T06:12:36.450024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:12:34.369291+0000","last_clean_scrub_stamp":"2024-04-04T06:12:34.369291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:41:20.018815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":284,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-04T06:12:36.407475+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:12:36.407475+0000","last_peered":"2024-04-04T06:12:36.407475+0000","last_clean":"2024-04-04T06:12:36.407475+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:12:36.407475+0000","last_undegraded":"2024-04-04T06:12:36.407475+0000","last_fullsized":"2024-04-04T06:12:36.407475+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":1020171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509527,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1036288,"data_stored":437272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509528,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1011712,"data_stored":429803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509527,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1011712,"data_stored":429803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509526,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1011712,"data_stored":429803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509528,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":1020171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542232,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1052672,"data_stored":444512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542230,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1036288,"data_stored":442878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:12:41.407 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-04T06:12:41.407 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-04T06:12:41.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.700+0000 7fdea7200640 1 Processor -- start 2024-04-04T06:12:41.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.701+0000 7fdea7200640 1 -- start start 2024-04-04T06:12:41.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.702+0000 7fdea7200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fdea80d1810 0x7fdea80d4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:41.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.702+0000 7fdea7200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdea80d2fd0 con 0x7fdea815e420 2024-04-04T06:12:41.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.702+0000 7fdea7200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdea80cfa70 con 0x7fdea80d1380 2024-04-04T06:12:41.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.702+0000 7fdea7200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fdea80cf790 con 0x7fdea80d1810 2024-04-04T06:12:41.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.702+0000 7fdea5e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fdea815e420 0x7fdea80b2f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58294/0 (socket says 172.21.15.159:58294) 2024-04-04T06:12:41.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.702+0000 7fdea5e00640 1 -- 172.21.15.159:0/237111316 learned_addr learned my addr 172.21.15.159:0/237111316 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:41.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.703+0000 7fdea6800640 1 --2- 172.21.15.159:0/237111316 >> v2:172.21.15.159:3300/0 conn(0x7fdea80d1810 0x7fdea80d4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:41.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.703+0000 7fdea5400640 1 -- 172.21.15.159:0/237111316 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2360127079 0 0) 0x7fdea80d2fd0 con 0x7fdea815e420 2024-04-04T06:12:41.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.703+0000 7fdea5400640 1 -- 172.21.15.159:0/237111316 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde8c003520 con 0x7fdea815e420 2024-04-04T06:12:41.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.703+0000 7fdea6800640 1 -- 172.21.15.159:0/237111316 >> v1:172.21.15.178:6789/0 conn(0x7fdea80d1380 legacy=0x7fdea80d3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:41.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.703+0000 7fdea6800640 1 -- 172.21.15.159:0/237111316 >> v1:172.21.15.159:6789/0 conn(0x7fdea815e420 legacy=0x7fdea80b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:41.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.703+0000 7fdea6800640 1 -- 172.21.15.159:0/237111316 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea80d3270 con 0x7fdea80d1810 2024-04-04T06:12:41.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.704+0000 7fdea6800640 1 --2- 172.21.15.159:0/237111316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea80d1810 0x7fdea80d4180 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fdea8161f30 tx=0x7fdea002bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a361c43c41d617 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:41.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.705+0000 7fdea5400640 1 -- 172.21.15.159:0/237111316 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0039070 con 0x7fdea80d1810 2024-04-04T06:12:41.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.705+0000 7fdea5400640 1 -- 172.21.15.159:0/237111316 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea002ccc0 con 0x7fdea80d1810 2024-04-04T06:12:41.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.705+0000 7fdea5400640 1 -- 172.21.15.159:0/237111316 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0034640 con 0x7fdea80d1810 2024-04-04T06:12:41.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.705+0000 7fdea7200640 1 -- 172.21.15.159:0/237111316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea80d1810 msgr2=0x7fdea80d4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:41.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.705+0000 7fdea7200640 1 --2- 172.21.15.159:0/237111316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea80d1810 0x7fdea80d4180 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fdea8161f30 tx=0x7fdea002bf10 comp rx=0 tx=0).stop 2024-04-04T06:12:41.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.705+0000 7fdea7200640 1 -- 172.21.15.159:0/237111316 shutdown_connections 2024-04-04T06:12:41.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.705+0000 7fdea7200640 1 --2- 172.21.15.159:0/237111316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea80d1810 0x7fdea80d4180 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:41.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.705+0000 7fdea7200640 1 -- 172.21.15.159:0/237111316 >> 172.21.15.159:0/237111316 conn(0x7fdea8067670 msgr2=0x7fdea8067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:41.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.706+0000 7fdea7200640 1 -- 172.21.15.159:0/237111316 shutdown_connections 2024-04-04T06:12:41.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.706+0000 7fdea7200640 1 -- 172.21.15.159:0/237111316 wait complete. 2024-04-04T06:12:41.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.707+0000 7fdea7200640 1 Processor -- start 2024-04-04T06:12:41.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.707+0000 7fdea7200640 1 -- start start 2024-04-04T06:12:41.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.708+0000 7fdea7200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdea80d1380 0x7fdea815ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:41.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.708+0000 7fdea7200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdea80d1810 0x7fdea80ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:41.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.708+0000 7fdea7200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea815e420 0x7fdea8156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:41.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.708+0000 7fdea7200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdea815e800 con 0x7fdea815e420 2024-04-04T06:12:41.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.708+0000 7fdea7200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdea80d1c20 con 0x7fdea80d1810 2024-04-04T06:12:41.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.708+0000 7fdea7200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdea80cf420 con 0x7fdea80d1380 2024-04-04T06:12:41.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.709+0000 7fdea6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea815e420 0x7fdea8156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:41.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.709+0000 7fdea6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea815e420 0x7fdea8156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56728/0 (socket says 172.21.15.159:56728) 2024-04-04T06:12:41.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.709+0000 7fdea6800640 1 -- 172.21.15.159:0/3364383219 learned_addr learned my addr 172.21.15.159:0/3364383219 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:41.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.709+0000 7fdea5e00640 1 --2- 172.21.15.159:0/3364383219 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdea80d1380 0x7fdea815ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:41.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.709+0000 7fde9fe00640 1 --2- 172.21.15.159:0/3364383219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdea80d1810 0x7fdea80ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:41.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.709+0000 7fdea6800640 1 -- 172.21.15.159:0/3364383219 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdea80d1380 msgr2=0x7fdea815ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:41.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.709+0000 7fdea6800640 1 --2- 172.21.15.159:0/3364383219 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdea80d1380 0x7fdea815ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:41.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.709+0000 7fdea6800640 1 -- 172.21.15.159:0/3364383219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdea80d1810 msgr2=0x7fdea80ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:41.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.709+0000 7fdea6800640 1 --2- 172.21.15.159:0/3364383219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdea80d1810 0x7fdea80ca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:41.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.709+0000 7fdea6800640 1 -- 172.21.15.159:0/3364383219 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea80cb4b0 con 0x7fdea815e420 2024-04-04T06:12:41.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.709+0000 7fde9fe00640 1 --2- 172.21.15.159:0/3364383219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdea80d1810 0x7fdea80ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:41.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.709+0000 7fdea5e00640 1 --2- 172.21.15.159:0/3364383219 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdea80d1380 0x7fdea815ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:41.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.709+0000 7fdea6800640 1 --2- 172.21.15.159:0/3364383219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea815e420 0x7fdea8156b80 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fdea002bf50 tx=0x7fdea0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:41.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.710+0000 7fde9e000640 1 -- 172.21.15.159:0/3364383219 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0044020 con 0x7fdea815e420 2024-04-04T06:12:41.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.710+0000 7fde9e000640 1 -- 172.21.15.159:0/3364383219 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea0004470 con 0x7fdea815e420 2024-04-04T06:12:41.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.710+0000 7fde9e000640 1 -- 172.21.15.159:0/3364383219 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0039040 con 0x7fdea815e420 2024-04-04T06:12:41.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.710+0000 7fdea7200640 1 -- 172.21.15.159:0/3364383219 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea80c93a0 con 0x7fdea815e420 2024-04-04T06:12:41.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.710+0000 7fdea7200640 1 -- 172.21.15.159:0/3364383219 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea80cba50 con 0x7fdea815e420 2024-04-04T06:12:41.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.711+0000 7fdea7200640 1 -- 172.21.15.159:0/3364383219 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea8075940 con 0x7fdea815e420 2024-04-04T06:12:41.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.712+0000 7fde9e000640 1 -- 172.21.15.159:0/3364383219 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdea0047020 con 0x7fdea815e420 2024-04-04T06:12:41.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.713+0000 7fde9e000640 1 --2- 172.21.15.159:0/3364383219 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde8406c8f0 0x7fde8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:41.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.713+0000 7fde9e000640 1 -- 172.21.15.159:0/3364383219 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fdea00ae5b0 con 0x7fdea815e420 2024-04-04T06:12:41.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.714+0000 7fdea5e00640 1 --2- 172.21.15.159:0/3364383219 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde8406c8f0 0x7fde8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:41.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.715+0000 7fdea5e00640 1 --2- 172.21.15.159:0/3364383219 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde8406c8f0 0x7fde8406edb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fde98005880 tx=0x7fde98009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:41.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.718+0000 7fde9e000640 1 -- 172.21.15.159:0/3364383219 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fdea004e030 con 0x7fdea815e420 2024-04-04T06:12:41.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:41.997+0000 7fdea7200640 1 -- 172.21.15.159:0/3364383219 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fdea80b4910 con 0x7fdea815e420 2024-04-04T06:12:41.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.002+0000 7fde9e000640 1 -- 172.21.15.159:0/3364383219 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7fdea007c940 con 0x7fdea815e420 2024-04-04T06:12:41.994 INFO:teuthology.orchestra.run.smithi159.stdout:[client.0] 2024-04-04T06:12:41.994 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQD9Qg5mqNKJGxAA4jP9cadhRIyhQ3lssttWIw== 2024-04-04T06:12:41.994 INFO:teuthology.orchestra.run.smithi159.stdout: caps mds = "allow" 2024-04-04T06:12:41.994 INFO:teuthology.orchestra.run.smithi159.stdout: caps mon = "allow r" 2024-04-04T06:12:41.994 INFO:teuthology.orchestra.run.smithi159.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-04T06:12:41.995 INFO:teuthology.orchestra.run.smithi159.stderr:updated caps for client.0 2024-04-04T06:12:41.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.004+0000 7fdea7200640 1 -- 172.21.15.159:0/3364383219 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde8406c8f0 msgr2=0x7fde8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:41.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.004+0000 7fdea7200640 1 --2- 172.21.15.159:0/3364383219 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde8406c8f0 0x7fde8406edb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fde98005880 tx=0x7fde98009040 comp rx=0 tx=0).stop 2024-04-04T06:12:41.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.004+0000 7fdea7200640 1 -- 172.21.15.159:0/3364383219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea815e420 msgr2=0x7fdea8156b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:41.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.004+0000 7fdea7200640 1 --2- 172.21.15.159:0/3364383219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea815e420 0x7fdea8156b80 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fdea002bf50 tx=0x7fdea0004a40 comp rx=0 tx=0).stop 2024-04-04T06:12:41.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.004+0000 7fdea7200640 1 -- 172.21.15.159:0/3364383219 shutdown_connections 2024-04-04T06:12:41.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.004+0000 7fdea7200640 1 --2- 172.21.15.159:0/3364383219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea815e420 0x7fdea8156b80 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:41.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.004+0000 7fdea7200640 1 --2- 172.21.15.159:0/3364383219 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde8406c8f0 0x7fde8406edb0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:41.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.004+0000 7fdea7200640 1 --2- 172.21.15.159:0/3364383219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdea80d1810 0x7fdea80ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:41.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.004+0000 7fdea7200640 1 --2- 172.21.15.159:0/3364383219 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdea80d1380 0x7fdea815ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:41.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.004+0000 7fdea7200640 1 -- 172.21.15.159:0/3364383219 >> 172.21.15.159:0/3364383219 conn(0x7fdea8067670 msgr2=0x7fdea80d3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:41.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.004+0000 7fdea7200640 1 -- 172.21.15.159:0/3364383219 shutdown_connections 2024-04-04T06:12:41.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.005+0000 7fdea7200640 1 -- 172.21.15.159:0/3364383219 wait complete. 2024-04-04T06:12:42.014 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:12:42.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.215+0000 7f4fa0200640 1 Processor -- start 2024-04-04T06:12:42.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.215+0000 7f4fa0200640 1 -- start start 2024-04-04T06:12:42.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.216+0000 7f4fa0200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4f9815c390 0x7f4f98073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:42.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.216+0000 7f4fa0200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4f980d0650 0x7f4f980d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:42.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.216+0000 7f4fa0200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f9815fe80 con 0x7f4f980af830 2024-04-04T06:12:42.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.216+0000 7f4fa0200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f4f9815c770 con 0x7f4f980d0650 2024-04-04T06:12:42.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.216+0000 7f4fa0200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4f980d0fe0 con 0x7f4f9815c390 2024-04-04T06:12:42.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.217+0000 7f4f9e200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4f980d0650 0x7f4f980d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:42.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.217+0000 7f4f9e200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4f980d0650 0x7f4f980d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56732/0 (socket says 172.21.15.159:56732) 2024-04-04T06:12:42.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.217+0000 7f4f9e200640 1 -- 172.21.15.159:0/523097989 learned_addr learned my addr 172.21.15.159:0/523097989 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:42.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.217+0000 7f4f9d800640 1 --2- 172.21.15.159:0/523097989 >> v2:172.21.15.178:3300/0 conn(0x7f4f9815c390 0x7f4f98073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:42.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.217+0000 7f4f9e200640 1 -- 172.21.15.159:0/523097989 >> v2:172.21.15.178:3300/0 conn(0x7f4f9815c390 msgr2=0x7f4f98073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:42.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.217+0000 7f4f9e200640 1 --2- 172.21.15.159:0/523097989 >> v2:172.21.15.178:3300/0 conn(0x7f4f9815c390 0x7f4f98073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:42.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.217+0000 7f4f9e200640 1 -- 172.21.15.159:0/523097989 >> v1:172.21.15.178:6789/0 conn(0x7f4f980af830 legacy=0x7f4f98160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:42.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.217+0000 7f4f9e200640 1 -- 172.21.15.159:0/523097989 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f980ad120 con 0x7f4f980d0650 2024-04-04T06:12:42.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.218+0000 7f4f9e200640 1 --2- 172.21.15.159:0/523097989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f980d0650 0x7f4f980d0aa0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f4f980b63e0 tx=0x7f4f9002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d27cc92cb4917b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:42.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.219+0000 7f4f97e00640 1 -- 172.21.15.159:0/523097989 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9002ca80 con 0x7f4f980d0650 2024-04-04T06:12:42.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.219+0000 7f4f97e00640 1 -- 172.21.15.159:0/523097989 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f9002cc20 con 0x7f4f980d0650 2024-04-04T06:12:42.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.219+0000 7f4f97e00640 1 -- 172.21.15.159:0/523097989 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f900343f0 con 0x7f4f980d0650 2024-04-04T06:12:42.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.219+0000 7f4fa0200640 1 -- 172.21.15.159:0/523097989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f980d0650 msgr2=0x7f4f980d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:42.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.219+0000 7f4fa0200640 1 --2- 172.21.15.159:0/523097989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f980d0650 0x7f4f980d0aa0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f4f980b63e0 tx=0x7f4f9002bee0 comp rx=0 tx=0).stop 2024-04-04T06:12:42.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.220+0000 7f4fa0200640 1 -- 172.21.15.159:0/523097989 shutdown_connections 2024-04-04T06:12:42.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.220+0000 7f4fa0200640 1 --2- 172.21.15.159:0/523097989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f980d0650 0x7f4f980d0aa0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:42.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.220+0000 7f4fa0200640 1 --2- 172.21.15.159:0/523097989 >> v2:172.21.15.178:3300/0 conn(0x7f4f9815c390 0x7f4f98073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:42.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.220+0000 7f4fa0200640 1 -- 172.21.15.159:0/523097989 >> 172.21.15.159:0/523097989 conn(0x7f4f9806b990 msgr2=0x7f4f9806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:42.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.220+0000 7f4fa0200640 1 -- 172.21.15.159:0/523097989 shutdown_connections 2024-04-04T06:12:42.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.220+0000 7f4fa0200640 1 -- 172.21.15.159:0/523097989 wait complete. 2024-04-04T06:12:42.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.222+0000 7f4fa0200640 1 Processor -- start 2024-04-04T06:12:42.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.222+0000 7f4fa0200640 1 -- start start 2024-04-04T06:12:42.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.223+0000 7f4fa0200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f980af830 0x7f4f9814d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:42.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.223+0000 7f4fa0200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f980d0650 0x7f4f980b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:42.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.223+0000 7f4fa0200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f9815c390 0x7f4f980b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:42.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.223+0000 7f4fa0200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4f98160120 con 0x7f4f9815c390 2024-04-04T06:12:42.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.223+0000 7f4fa0200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4f980ad6e0 con 0x7f4f980af830 2024-04-04T06:12:42.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.223+0000 7f4fa0200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4f98160d50 con 0x7f4f980d0650 2024-04-04T06:12:42.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.223+0000 7f4f9e200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f9815c390 0x7f4f980b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:42.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.223+0000 7f4f9e200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f9815c390 0x7f4f980b2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56736/0 (socket says 172.21.15.159:56736) 2024-04-04T06:12:42.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.223+0000 7f4f9e200640 1 -- 172.21.15.159:0/4214703047 learned_addr learned my addr 172.21.15.159:0/4214703047 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:42.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.223+0000 7f4f9d800640 1 --2- 172.21.15.159:0/4214703047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f980af830 0x7f4f9814d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:42.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.223+0000 7f4f9ce00640 1 --2- 172.21.15.159:0/4214703047 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f980d0650 0x7f4f980b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:42.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.224+0000 7f4f9e200640 1 -- 172.21.15.159:0/4214703047 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f980d0650 msgr2=0x7f4f980b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:42.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.224+0000 7f4f9e200640 1 --2- 172.21.15.159:0/4214703047 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f980d0650 0x7f4f980b21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:42.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.224+0000 7f4f9e200640 1 -- 172.21.15.159:0/4214703047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f980af830 msgr2=0x7f4f9814d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:42.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.224+0000 7f4f9e200640 1 --2- 172.21.15.159:0/4214703047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f980af830 0x7f4f9814d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:42.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.224+0000 7f4f9e200640 1 -- 172.21.15.159:0/4214703047 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f980b3ea0 con 0x7f4f9815c390 2024-04-04T06:12:42.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.224+0000 7f4f9e200640 1 --2- 172.21.15.159:0/4214703047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f9815c390 0x7f4f980b2730 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009f40 tx=0x7f4f90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:42.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.224+0000 7f4f95600640 1 -- 172.21.15.159:0/4214703047 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f90043020 con 0x7f4f9815c390 2024-04-04T06:12:42.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.224+0000 7f4f95600640 1 -- 172.21.15.159:0/4214703047 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f9003e450 con 0x7f4f9815c390 2024-04-04T06:12:42.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.224+0000 7f4f95600640 1 -- 172.21.15.159:0/4214703047 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f90009510 con 0x7f4f9815c390 2024-04-04T06:12:42.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.224+0000 7f4fa0200640 1 -- 172.21.15.159:0/4214703047 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f980b4180 con 0x7f4f9815c390 2024-04-04T06:12:42.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.225+0000 7f4fa0200640 1 -- 172.21.15.159:0/4214703047 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f980b4740 con 0x7f4f9815c390 2024-04-04T06:12:42.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.226+0000 7f4fa0200640 1 -- 172.21.15.159:0/4214703047 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f980685c0 con 0x7f4f9815c390 2024-04-04T06:12:42.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.227+0000 7f4f95600640 1 -- 172.21.15.159:0/4214703047 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f90046020 con 0x7f4f9815c390 2024-04-04T06:12:42.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.227+0000 7f4f95600640 1 --2- 172.21.15.159:0/4214703047 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4f7806c920 0x7f4f7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:42.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.228+0000 7f4f95600640 1 -- 172.21.15.159:0/4214703047 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f4f900ae140 con 0x7f4f9815c390 2024-04-04T06:12:42.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.228+0000 7f4f9d800640 1 --2- 172.21.15.159:0/4214703047 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4f7806c920 0x7f4f7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:42.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.229+0000 7f4f9d800640 1 --2- 172.21.15.159:0/4214703047 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4f7806c920 0x7f4f7806ede0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c006fd0 tx=0x7f4f8c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:42.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.232+0000 7f4f95600640 1 -- 172.21.15.159:0/4214703047 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4f9004d070 con 0x7f4f9815c390 2024-04-04T06:12:42.358 INFO:tasks.ceph.mds.a.smithi159.stderr:2024-04-04T06:12:42.366+0000 7ffb1c600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:12:42.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.475+0000 7f4fa0200640 1 -- 172.21.15.159:0/4214703047 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f9806f1b0 con 0x7f4f9815c390 2024-04-04T06:12:42.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.476+0000 7f4f95600640 1 -- 172.21.15.159:0/4214703047 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5195 (secure 0 0 0) 0x7f4f9007c4d0 con 0x7f4f9815c390 2024-04-04T06:12:42.468 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:42.468 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1825651864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1825651864},{"type":"v1","addr":"172.21.15.178:6836","nonce":1825651864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1824578712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1824578712},{"type":"v1","addr":"172.21.15.178:6837","nonce":1824578712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6834/3377498298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3377498298},{"type":"v1","addr":"172.21.15.159:6834","nonce":3377498298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:12:35.880454+0000","modified":"2024-04-04T06:12:38.922420+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7293":{"gid":7293,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6835/3484779950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":3484779950},{"type":"v1","addr":"172.21.15.159:6835","nonce":3484779950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7293,"qdb_cluster":[7293]},"id":6}]} 2024-04-04T06:12:42.469 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 62 2024-04-04T06:12:42.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.479+0000 7f4fa0200640 1 -- 172.21.15.159:0/4214703047 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4f7806c920 msgr2=0x7f4f7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:42.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.479+0000 7f4fa0200640 1 --2- 172.21.15.159:0/4214703047 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4f7806c920 0x7f4f7806ede0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c006fd0 tx=0x7f4f8c008040 comp rx=0 tx=0).stop 2024-04-04T06:12:42.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.479+0000 7f4fa0200640 1 -- 172.21.15.159:0/4214703047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f9815c390 msgr2=0x7f4f980b2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:42.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.479+0000 7f4fa0200640 1 --2- 172.21.15.159:0/4214703047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f9815c390 0x7f4f980b2730 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009f40 tx=0x7f4f90004a40 comp rx=0 tx=0).stop 2024-04-04T06:12:42.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.479+0000 7f4fa0200640 1 -- 172.21.15.159:0/4214703047 shutdown_connections 2024-04-04T06:12:42.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.479+0000 7f4fa0200640 1 --2- 172.21.15.159:0/4214703047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f9815c390 0x7f4f980b2730 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:42.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.479+0000 7f4fa0200640 1 --2- 172.21.15.159:0/4214703047 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f980d0650 0x7f4f980b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:42.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.479+0000 7f4fa0200640 1 --2- 172.21.15.159:0/4214703047 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4f7806c920 0x7f4f7806ede0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:42.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.479+0000 7f4fa0200640 1 --2- 172.21.15.159:0/4214703047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f980af830 0x7f4f9814d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:42.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.479+0000 7f4fa0200640 1 -- 172.21.15.159:0/4214703047 >> 172.21.15.159:0/4214703047 conn(0x7f4f9806b990 msgr2=0x7f4f98154b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:42.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.479+0000 7f4fa0200640 1 -- 172.21.15.159:0/4214703047 shutdown_connections 2024-04-04T06:12:42.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:42.479+0000 7f4fa0200640 1 -- 172.21.15.159:0/4214703047 wait complete. 2024-04-04T06:12:42.499 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:12:42.504 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:12:42.894 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:12:35.880454+0000', 'modified': '2024-04-04T06:12:38.922420+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7293}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7293': {'gid': 7293, 'name': 'a', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.159:6835/3484779950', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6833', 'nonce': 3484779950}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 3484779950}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7293, 'qdb_cluster': [7293]} 2024-04-04T06:12:42.895 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-04T06:12:42.895 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-04T06:12:43.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.126+0000 7f1780400640 1 Processor -- start 2024-04-04T06:12:43.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.126+0000 7f1780400640 1 -- start start 2024-04-04T06:12:43.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.128+0000 7f1780400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f177815b260 0x7f17780b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.128+0000 7f1780400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f17780b06b0 0x7f17780aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.128+0000 7f1780400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f177815ff50 con 0x7f177815b640 2024-04-04T06:12:43.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.128+0000 7f1780400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f177815f340 con 0x7f177815b260 2024-04-04T06:12:43.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.128+0000 7f1780400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f177815f620 con 0x7f17780b06b0 2024-04-04T06:12:43.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.128+0000 7f177da00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f177815b260 0x7f17780b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:43.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.128+0000 7f177d000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f177815b640 0x7f17780affa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:58296/0 (socket says 172.21.15.159:58296) 2024-04-04T06:12:43.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.128+0000 7f177d000640 1 -- 172.21.15.159:0/799655642 learned_addr learned my addr 172.21.15.159:0/799655642 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:43.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.129+0000 7f177e400640 1 --2- 172.21.15.159:0/799655642 >> v2:172.21.15.178:3300/0 conn(0x7f17780b06b0 0x7f17780aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:43.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.129+0000 7f177da00640 1 -- 172.21.15.159:0/799655642 >> v2:172.21.15.178:3300/0 conn(0x7f17780b06b0 msgr2=0x7f17780aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:43.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.129+0000 7f177da00640 1 --2- 172.21.15.159:0/799655642 >> v2:172.21.15.178:3300/0 conn(0x7f17780b06b0 0x7f17780aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.129+0000 7f177da00640 1 -- 172.21.15.159:0/799655642 >> v1:172.21.15.159:6789/0 conn(0x7f177815b640 legacy=0x7f17780affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:43.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.129+0000 7f177da00640 1 -- 172.21.15.159:0/799655642 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177815f900 con 0x7f177815b260 2024-04-04T06:12:43.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.129+0000 7f1777e00640 1 -- 172.21.15.159:0/799655642 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1818571028 0 0) 0x7f177815ff50 con 0x7f177815b640 2024-04-04T06:12:43.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.130+0000 7f177da00640 1 --2- 172.21.15.159:0/799655642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177815b260 0x7f17780b40b0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f17780ae020 tx=0x7f176c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd82681b030be6e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:43.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.130+0000 7f1777e00640 1 -- 172.21.15.159:0/799655642 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c035020 con 0x7f177815b260 2024-04-04T06:12:43.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.131+0000 7f1777e00640 1 -- 172.21.15.159:0/799655642 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f176c031830 con 0x7f177815b260 2024-04-04T06:12:43.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.131+0000 7f1777e00640 1 -- 172.21.15.159:0/799655642 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c031c00 con 0x7f177815b260 2024-04-04T06:12:43.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.131+0000 7f1780400640 1 -- 172.21.15.159:0/799655642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177815b260 msgr2=0x7f17780b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:43.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.131+0000 7f1780400640 1 --2- 172.21.15.159:0/799655642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177815b260 0x7f17780b40b0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f17780ae020 tx=0x7f176c02b8d0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.131+0000 7f1780400640 1 -- 172.21.15.159:0/799655642 shutdown_connections 2024-04-04T06:12:43.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.131+0000 7f1780400640 1 --2- 172.21.15.159:0/799655642 >> v2:172.21.15.178:3300/0 conn(0x7f17780b06b0 0x7f17780aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.131+0000 7f1780400640 1 --2- 172.21.15.159:0/799655642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177815b260 0x7f17780b40b0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.131+0000 7f1780400640 1 -- 172.21.15.159:0/799655642 >> 172.21.15.159:0/799655642 conn(0x7f17780b65a0 msgr2=0x7f17780b69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:43.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.132+0000 7f1780400640 1 -- 172.21.15.159:0/799655642 shutdown_connections 2024-04-04T06:12:43.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.132+0000 7f1780400640 1 -- 172.21.15.159:0/799655642 wait complete. 2024-04-04T06:12:43.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.134+0000 7f1780400640 1 Processor -- start 2024-04-04T06:12:43.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.135+0000 7f1780400640 1 -- start start 2024-04-04T06:12:43.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.136+0000 7f1780400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17780b06b0 0x7f17780ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.136+0000 7f1780400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177815b260 0x7f177814c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.136+0000 7f1780400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f177815b640 0x7f17780ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.136+0000 7f1780400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f177806e770 con 0x7f177815b260 2024-04-04T06:12:43.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.136+0000 7f1780400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f177815f010 con 0x7f17780b06b0 2024-04-04T06:12:43.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.136+0000 7f1780400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f17780b45f0 con 0x7f177815b640 2024-04-04T06:12:43.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.136+0000 7f177d000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177815b260 0x7f177814c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:43.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.136+0000 7f177d000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177815b260 0x7f177814c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56740/0 (socket says 172.21.15.159:56740) 2024-04-04T06:12:43.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.136+0000 7f177d000640 1 -- 172.21.15.159:0/3092401439 learned_addr learned my addr 172.21.15.159:0/3092401439 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:43.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.137+0000 7f177d000640 1 -- 172.21.15.159:0/3092401439 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f177815b640 msgr2=0x7f17780ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:43.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.137+0000 7f177d000640 1 --2- 172.21.15.159:0/3092401439 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f177815b640 0x7f17780ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.137+0000 7f177d000640 1 -- 172.21.15.159:0/3092401439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17780b06b0 msgr2=0x7f17780ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:43.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.137+0000 7f177d000640 1 --2- 172.21.15.159:0/3092401439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17780b06b0 0x7f17780ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.137+0000 7f177d000640 1 -- 172.21.15.159:0/3092401439 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1778158840 con 0x7f177815b260 2024-04-04T06:12:43.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.137+0000 7f177d000640 1 --2- 172.21.15.159:0/3092401439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177815b260 0x7f177814c850 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f176400c410 tx=0x7f176400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:43.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.137+0000 7f1775600640 1 -- 172.21.15.159:0/3092401439 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17640074e0 con 0x7f177815b260 2024-04-04T06:12:43.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.137+0000 7f1775600640 1 -- 172.21.15.159:0/3092401439 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17640047e0 con 0x7f177815b260 2024-04-04T06:12:43.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.137+0000 7f1775600640 1 -- 172.21.15.159:0/3092401439 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764004bd0 con 0x7f177815b260 2024-04-04T06:12:43.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.138+0000 7f1780400640 1 -- 172.21.15.159:0/3092401439 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1778158b20 con 0x7f177815b260 2024-04-04T06:12:43.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.138+0000 7f1780400640 1 -- 172.21.15.159:0/3092401439 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17780ca940 con 0x7f177815b260 2024-04-04T06:12:43.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.140+0000 7f1775600640 1 -- 172.21.15.159:0/3092401439 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1764011070 con 0x7f177815b260 2024-04-04T06:12:43.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.142+0000 7f1775600640 1 --2- 172.21.15.159:0/3092401439 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f175806c920 0x7f175806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.142+0000 7f1775600640 1 -- 172.21.15.159:0/3092401439 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f176408b750 con 0x7f177815b260 2024-04-04T06:12:43.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.142+0000 7f177da00640 1 --2- 172.21.15.159:0/3092401439 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f175806c920 0x7f175806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:43.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.142+0000 7f178302f740 1 Processor -- start 2024-04-04T06:12:43.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.142+0000 7f178302f740 1 -- start start 2024-04-04T06:12:43.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.143+0000 7f178302f740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56047ecd3130 0x56047eca60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.143+0000 7f178302f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecd3510 0x56047ecd5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.143+0000 7f178302f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56047eca6610 0x56047ecd6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.143+0000 7f178302f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56047ec6dab0 con 0x56047ecd3510 2024-04-04T06:12:43.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.143+0000 7f178302f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56047ec0ffa0 con 0x56047eca6610 2024-04-04T06:12:43.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.143+0000 7f178302f740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56047ebb8560 con 0x56047ecd3130 2024-04-04T06:12:43.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.143+0000 7f177da00640 1 --2- 172.21.15.159:0/3092401439 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f175806c920 0x7f175806ede0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f176c036000 tx=0x7f176c030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:43.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.144+0000 7f177d000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecd3510 0x56047ecd5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:43.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.144+0000 7f177d000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecd3510 0x56047ecd5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56756/0 (socket says 172.21.15.159:56756) 2024-04-04T06:12:43.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.144+0000 7f177d000640 1 -- 172.21.15.159:0/2878672327 learned_addr learned my addr 172.21.15.159:0/2878672327 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:43.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.144+0000 7f177d000640 1 -- 172.21.15.159:0/2878672327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56047ecd3130 msgr2=0x56047eca60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:43.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.144+0000 7f177e400640 1 --2- 172.21.15.159:0/2878672327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56047eca6610 0x56047ecd6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:43.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.144+0000 7f177da00640 1 --2- 172.21.15.159:0/2878672327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56047ecd3130 0x56047eca60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:43.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.144+0000 7f177d000640 1 --2- 172.21.15.159:0/2878672327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56047ecd3130 0x56047eca60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.144+0000 7f177d000640 1 -- 172.21.15.159:0/2878672327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56047eca6610 msgr2=0x56047ecd6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:43.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.144+0000 7f177d000640 1 --2- 172.21.15.159:0/2878672327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56047eca6610 0x56047ecd6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.144+0000 7f177d000640 1 -- 172.21.15.159:0/2878672327 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56047e9242d0 con 0x56047ecd3510 2024-04-04T06:12:43.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.144+0000 7f177d000640 1 --2- 172.21.15.159:0/2878672327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecd3510 0x56047ecd5ec0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f17640558d0 tx=0x7f176400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa5431de5207ce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:43.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.144+0000 7f176a000640 1 -- 172.21.15.159:0/2878672327 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176405d820 con 0x56047ecd3510 2024-04-04T06:12:43.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.144+0000 7f176a000640 1 -- 172.21.15.159:0/2878672327 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1764062050 con 0x56047ecd3510 2024-04-04T06:12:43.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.144+0000 7f176a000640 1 -- 172.21.15.159:0/2878672327 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764004d70 con 0x56047ecd3510 2024-04-04T06:12:43.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.145+0000 7f178302f740 1 -- 172.21.15.159:0/2878672327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecd3510 msgr2=0x56047ecd5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:43.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.145+0000 7f178302f740 1 --2- 172.21.15.159:0/2878672327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecd3510 0x56047ecd5ec0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f17640558d0 tx=0x7f176400f000 comp rx=0 tx=0).stop 2024-04-04T06:12:43.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.145+0000 7f178302f740 1 -- 172.21.15.159:0/2878672327 shutdown_connections 2024-04-04T06:12:43.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.145+0000 7f178302f740 1 --2- 172.21.15.159:0/2878672327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56047eca6610 0x56047ecd6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.145+0000 7f178302f740 1 --2- 172.21.15.159:0/2878672327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecd3510 0x56047ecd5ec0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.145+0000 7f178302f740 1 --2- 172.21.15.159:0/2878672327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56047ecd3130 0x56047eca60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.145+0000 7f178302f740 1 -- 172.21.15.159:0/2878672327 >> 172.21.15.159:0/2878672327 conn(0x56047eca8340 msgr2=0x56047ecf0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:43.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.145+0000 7f178302f740 1 -- 172.21.15.159:0/2878672327 shutdown_connections 2024-04-04T06:12:43.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.145+0000 7f178302f740 1 -- 172.21.15.159:0/2878672327 wait complete. 2024-04-04T06:12:43.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.146+0000 7f178302f740 1 -- start start 2024-04-04T06:12:43.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.147+0000 7f178302f740 1 Processor -- start 2024-04-04T06:12:43.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.148+0000 7f178302f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56047ecd3130 0x56047ecd8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.148+0000 7f178302f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecd3510 0x56047ecc23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.148+0000 7f178302f740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56047ecdbc10 0x56047ecc2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.148+0000 7f178302f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56047ebcb920 con 0x56047ecd3510 2024-04-04T06:12:43.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.149+0000 7f178302f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56047ec0bf50 con 0x56047ecd3130 2024-04-04T06:12:43.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.149+0000 7f178302f740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56047ec6dab0 con 0x56047ecdbc10 2024-04-04T06:12:43.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.149+0000 7f178302f740 10 client.0 _ll_register_callbacks cb 0x56047ebcb310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:12:43.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.149+0000 7f177d000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecd3510 0x56047ecc23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:43.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.149+0000 7f177d000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecd3510 0x56047ecc23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56772/0 (socket says 172.21.15.159:56772) 2024-04-04T06:12:43.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.149+0000 7f177d000640 1 -- 172.21.15.159:0/101094724 learned_addr learned my addr 172.21.15.159:0/101094724 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:43.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.149+0000 7f177d000640 1 -- 172.21.15.159:0/101094724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56047ecdbc10 msgr2=0x56047ecc2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:43.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.149+0000 7f177da00640 1 --2- 172.21.15.159:0/101094724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56047ecd3130 0x56047ecd8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:43.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.149+0000 7f177d000640 1 --2- 172.21.15.159:0/101094724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56047ecdbc10 0x56047ecc2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.149+0000 7f177d000640 1 -- 172.21.15.159:0/101094724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56047ecd3130 msgr2=0x56047ecd8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:43.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.149+0000 7f177d000640 1 --2- 172.21.15.159:0/101094724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56047ecd3130 0x56047ecd8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.149+0000 7f177d000640 1 -- 172.21.15.159:0/101094724 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56047e9ee6b0 con 0x56047ecd3510 2024-04-04T06:12:43.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.150+0000 7f177d000640 1 --2- 172.21.15.159:0/101094724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecd3510 0x56047ecc23a0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f1764059170 tx=0x7f1764000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c70b0a922fa8ecb server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:43.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.150+0000 7f175ea00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:12:43.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.150+0000 7f175ea00640 1 -- 172.21.15.159:0/101094724 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764076060 con 0x56047ecd3510 2024-04-04T06:12:43.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.150+0000 7f175ea00640 1 -- 172.21.15.159:0/101094724 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f17640588c0 con 0x56047ecd3510 2024-04-04T06:12:43.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.150+0000 7f178302f740 1 -- 172.21.15.159:0/101094724 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1764062050 con 0x56047ecd3510 2024-04-04T06:12:43.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.150+0000 7f175ea00640 1 -- 172.21.15.159:0/101094724 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f1764058ac0 con 0x56047ecd3510 2024-04-04T06:12:43.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.150+0000 7f178302f740 10 client.7443 fetch_fsmap learned FSMap version 62 2024-04-04T06:12:43.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.150+0000 7f178302f740 1 -- 172.21.15.159:0/101094724 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x56047ecd80f0 con 0x56047ecd3510 2024-04-04T06:12:43.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.150+0000 7f175ea00640 1 -- 172.21.15.159:0/101094724 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1764062050 con 0x56047ecd3510 2024-04-04T06:12:43.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.150+0000 7f175ea00640 1 -- 172.21.15.159:0/101094724 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f17640551c0 con 0x56047ecd3510 2024-04-04T06:12:43.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.150+0000 7f178302f740 10 client.7443 fetch_fsmap finished waiting for FSMap version 62 2024-04-04T06:12:43.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.150+0000 7f178302f740 10 client.7443 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:12:43.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.151+0000 7f178302f740 20 client.7443 populate_metadata read hostname 'smithi159' 2024-04-04T06:12:43.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.151+0000 7f178302f740 1 --2- 172.21.15.159:0/101094724 >> [v2:172.21.15.159:6833/3484779950,v1:172.21.15.159:6835/3484779950] conn(0x56047ecda7f0 0x56047ecdabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.151+0000 7f178302f740 4 client.7443 mds_command: new command op to 7293 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:12:43.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.151+0000 7f178302f740 1 -- 172.21.15.159:0/101094724 --> [v2:172.21.15.159:6833/3484779950,v1:172.21.15.159:6835/3484779950] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56047ec1d4e0 con 0x56047ecda7f0 2024-04-04T06:12:43.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.151+0000 7f177da00640 1 --2- 172.21.15.159:0/101094724 >> [v2:172.21.15.159:6833/3484779950,v1:172.21.15.159:6835/3484779950] conn(0x56047ecda7f0 0x56047ecdabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:43.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.151+0000 7f177da00640 1 --2- 172.21.15.159:0/101094724 >> [v2:172.21.15.159:6833/3484779950,v1:172.21.15.159:6835/3484779950] conn(0x56047ecda7f0 0x56047ecdabd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c6e2f2fda00fbc1 server_cookie=795a914eec8b328f in_seq=0 out_seq=0 2024-04-04T06:12:43.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.151+0000 7f175ea00640 10 client.7443 ms_handle_connect on v2:172.21.15.159:6833/3484779950 2024-04-04T06:12:43.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.152+0000 7f175ea00640 1 -- 172.21.15.159:0/101094724 <== mds.0 v2:172.21.15.159:6833/3484779950 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56047ec1d4e0 con 0x56047ecda7f0 2024-04-04T06:12:43.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.152+0000 7f175ea00640 10 client.7443 handle_command_reply: tid=0 2024-04-04T06:12:43.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.153+0000 7f178302f740 1 client.7443 shutdown 2024-04-04T06:12:43.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.153+0000 7f178302f740 1 -- 172.21.15.159:0/101094724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecd3510 msgr2=0x56047ecc23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:43.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.153+0000 7f178302f740 1 --2- 172.21.15.159:0/101094724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecd3510 0x56047ecc23a0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f1764059170 tx=0x7f1764000ed0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.153+0000 7f178302f740 1 -- 172.21.15.159:0/101094724 shutdown_connections 2024-04-04T06:12:43.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.153+0000 7f178302f740 1 --2- 172.21.15.159:0/101094724 >> [v2:172.21.15.159:6833/3484779950,v1:172.21.15.159:6835/3484779950] conn(0x56047ecda7f0 0x56047ecdabd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.153+0000 7f178302f740 1 --2- 172.21.15.159:0/101094724 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56047ecdbc10 0x56047ecc2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.153+0000 7f178302f740 1 --2- 172.21.15.159:0/101094724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecd3510 0x56047ecc23a0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.153+0000 7f178302f740 1 --2- 172.21.15.159:0/101094724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56047ecd3130 0x56047ecd8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.153+0000 7f178302f740 1 -- 172.21.15.159:0/101094724 >> 172.21.15.159:0/101094724 conn(0x56047eca6610 msgr2=0x56047ecf4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:43.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.153+0000 7f175ea00640 0 client.7443 ms_handle_reset on v2:172.21.15.159:6833/3484779950 2024-04-04T06:12:43.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.153+0000 7f178302f740 1 -- 172.21.15.159:0/101094724 shutdown_connections 2024-04-04T06:12:43.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.153+0000 7f178302f740 1 -- 172.21.15.159:0/101094724 wait complete. 2024-04-04T06:12:43.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.153+0000 7f178302f740 20 client.7443 trim_cache size 0 max 16384 2024-04-04T06:12:43.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.173+0000 7f178302f740 1 Processor -- start 2024-04-04T06:12:43.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.173+0000 7f178302f740 1 -- start start 2024-04-04T06:12:43.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.173+0000 7f178302f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56047ecd3130 0x56047ecf3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.173+0000 7f178302f740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56047ecd3510 0x56047ecd85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.173+0000 7f178302f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecdbc10 0x56047ecd9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.173+0000 7f178302f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1778158220 con 0x56047ecdbc10 2024-04-04T06:12:43.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.173+0000 7f178302f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f177815cc30 con 0x56047ecd3130 2024-04-04T06:12:43.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.173+0000 7f178302f740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1778157bd0 con 0x56047ecd3510 2024-04-04T06:12:43.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.174+0000 7f177e400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecdbc10 0x56047ecd9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:43.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.174+0000 7f177e400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecdbc10 0x56047ecd9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56784/0 (socket says 172.21.15.159:56784) 2024-04-04T06:12:43.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.174+0000 7f177e400640 1 -- 172.21.15.159:0/4126955588 learned_addr learned my addr 172.21.15.159:0/4126955588 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:43.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.174+0000 7f177d000640 1 --2- 172.21.15.159:0/4126955588 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56047ecd3510 0x56047ecd85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:43.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.174+0000 7f177da00640 1 --2- 172.21.15.159:0/4126955588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56047ecd3130 0x56047ecf3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:43.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.174+0000 7f177e400640 1 -- 172.21.15.159:0/4126955588 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56047ecd3510 msgr2=0x56047ecd85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:43.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.174+0000 7f177e400640 1 --2- 172.21.15.159:0/4126955588 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56047ecd3510 0x56047ecd85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.174+0000 7f177e400640 1 -- 172.21.15.159:0/4126955588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56047ecd3130 msgr2=0x56047ecf3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:43.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.174+0000 7f177e400640 1 --2- 172.21.15.159:0/4126955588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56047ecd3130 0x56047ecf3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.174+0000 7f177e400640 1 -- 172.21.15.159:0/4126955588 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56047ecd8b30 con 0x56047ecdbc10 2024-04-04T06:12:43.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.175+0000 7f177e400640 1 --2- 172.21.15.159:0/4126955588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecdbc10 0x56047ecd9140 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f17780ae590 tx=0x7f1770007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cef1e5b59aa15f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:43.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.175+0000 7f1769600640 1 -- 172.21.15.159:0/4126955588 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770007e20 con 0x56047ecdbc10 2024-04-04T06:12:43.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.175+0000 7f1769600640 1 -- 172.21.15.159:0/4126955588 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1770003990 con 0x56047ecdbc10 2024-04-04T06:12:43.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.175+0000 7f1769600640 1 -- 172.21.15.159:0/4126955588 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770003d60 con 0x56047ecdbc10 2024-04-04T06:12:43.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.176+0000 7f178302f740 1 -- 172.21.15.159:0/4126955588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecdbc10 msgr2=0x56047ecd9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:43.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.176+0000 7f178302f740 1 --2- 172.21.15.159:0/4126955588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecdbc10 0x56047ecd9140 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f17780ae590 tx=0x7f1770007700 comp rx=0 tx=0).stop 2024-04-04T06:12:43.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.176+0000 7f178302f740 1 -- 172.21.15.159:0/4126955588 shutdown_connections 2024-04-04T06:12:43.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.176+0000 7f178302f740 1 --2- 172.21.15.159:0/4126955588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecdbc10 0x56047ecd9140 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.176+0000 7f178302f740 1 --2- 172.21.15.159:0/4126955588 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56047ecd3510 0x56047ecd85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.176+0000 7f178302f740 1 --2- 172.21.15.159:0/4126955588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56047ecd3130 0x56047ecf3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.176+0000 7f178302f740 1 -- 172.21.15.159:0/4126955588 >> 172.21.15.159:0/4126955588 conn(0x56047eca6610 msgr2=0x56047ecac230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:43.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.176+0000 7f178302f740 1 -- 172.21.15.159:0/4126955588 shutdown_connections 2024-04-04T06:12:43.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.176+0000 7f178302f740 1 -- 172.21.15.159:0/4126955588 wait complete. 2024-04-04T06:12:43.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.176+0000 7f178302f740 1 -- start start 2024-04-04T06:12:43.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.177+0000 7f178302f740 1 Processor -- start 2024-04-04T06:12:43.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.178+0000 7f178302f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56047ecd3130 0x56047ed6a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.178+0000 7f178302f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecd3510 0x56047ed132a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.178+0000 7f178302f740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56047ecdbc10 0x56047ecdabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.178+0000 7f178302f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56047ebcb920 con 0x56047ecd3510 2024-04-04T06:12:43.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.178+0000 7f178302f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56047ec6dab0 con 0x56047ecd3130 2024-04-04T06:12:43.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.178+0000 7f178302f740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56047ec0bf50 con 0x56047ecdbc10 2024-04-04T06:12:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.178+0000 7f178302f740 10 client.0 _ll_register_callbacks cb 0x7f176405d820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:12:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.178+0000 7f177d000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecd3510 0x56047ed132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.178+0000 7f177d000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecd3510 0x56047ed132a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56790/0 (socket says 172.21.15.159:56790) 2024-04-04T06:12:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.178+0000 7f177d000640 1 -- 172.21.15.159:0/952783225 learned_addr learned my addr 172.21.15.159:0/952783225 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.178+0000 7f177da00640 1 --2- 172.21.15.159:0/952783225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56047ecd3130 0x56047ed6a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.178+0000 7f177e400640 1 --2- 172.21.15.159:0/952783225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56047ecdbc10 0x56047ecdabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.178+0000 7f177d000640 1 -- 172.21.15.159:0/952783225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56047ecdbc10 msgr2=0x56047ecdabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.178+0000 7f177d000640 1 --2- 172.21.15.159:0/952783225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56047ecdbc10 0x56047ecdabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.178+0000 7f177d000640 1 -- 172.21.15.159:0/952783225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56047ecd3130 msgr2=0x56047ed6a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.178+0000 7f177d000640 1 --2- 172.21.15.159:0/952783225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56047ecd3130 0x56047ed6a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.178+0000 7f177d000640 1 -- 172.21.15.159:0/952783225 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56047ecf4040 con 0x56047ecd3510 2024-04-04T06:12:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.178+0000 7f177da00640 1 --2- 172.21.15.159:0/952783225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56047ecd3130 0x56047ed6a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.178+0000 7f177e400640 1 --2- 172.21.15.159:0/952783225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56047ecdbc10 0x56047ecdabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.179+0000 7f177d000640 1 --2- 172.21.15.159:0/952783225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecd3510 0x56047ed132a0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f1764061b20 tx=0x7f17640590e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e003ed08c5d0d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:43.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.179+0000 7f175ea00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:12:43.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.179+0000 7f175ea00640 1 -- 172.21.15.159:0/952783225 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764059240 con 0x56047ecd3510 2024-04-04T06:12:43.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.179+0000 7f175ea00640 1 -- 172.21.15.159:0/952783225 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f176405ebc0 con 0x56047ecd3510 2024-04-04T06:12:43.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.179+0000 7f178302f740 1 -- 172.21.15.159:0/952783225 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1770003990 con 0x56047ecd3510 2024-04-04T06:12:43.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.179+0000 7f175ea00640 1 -- 172.21.15.159:0/952783225 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f1764062050 con 0x56047ecd3510 2024-04-04T06:12:43.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.179+0000 7f178302f740 10 client.7449 fetch_fsmap learned FSMap version 62 2024-04-04T06:12:43.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.179+0000 7f178302f740 1 -- 172.21.15.159:0/952783225 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x56047ed137e0 con 0x56047ecd3510 2024-04-04T06:12:43.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.179+0000 7f175ea00640 1 -- 172.21.15.159:0/952783225 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1770003990 con 0x56047ecd3510 2024-04-04T06:12:43.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.180+0000 7f175ea00640 1 -- 172.21.15.159:0/952783225 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 62) ==== 3494+0+0 (secure 0 0 0) 0x7f176408b0b0 con 0x56047ecd3510 2024-04-04T06:12:43.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.180+0000 7f178302f740 10 client.7449 fetch_fsmap finished waiting for FSMap version 62 2024-04-04T06:12:43.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.180+0000 7f178302f740 10 client.7449 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:12:43.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.180+0000 7f178302f740 20 client.7449 populate_metadata read hostname 'smithi159' 2024-04-04T06:12:43.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.180+0000 7f178302f740 1 --2- 172.21.15.159:0/952783225 >> [v2:172.21.15.159:6833/3484779950,v1:172.21.15.159:6835/3484779950] conn(0x56047ecd9350 0x56047ecd9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.180+0000 7f178302f740 4 client.7449 mds_command: new command op to 7293 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-04T06:12:43.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.180+0000 7f178302f740 1 -- 172.21.15.159:0/952783225 --> [v2:172.21.15.159:6833/3484779950,v1:172.21.15.159:6835/3484779950] -- command(tid 0: {"prefix": "status"}) -- 0x56047ecdc650 con 0x56047ecd9350 2024-04-04T06:12:43.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.180+0000 7f177da00640 1 --2- 172.21.15.159:0/952783225 >> [v2:172.21.15.159:6833/3484779950,v1:172.21.15.159:6835/3484779950] conn(0x56047ecd9350 0x56047ecd9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:43.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.181+0000 7f177da00640 1 --2- 172.21.15.159:0/952783225 >> [v2:172.21.15.159:6833/3484779950,v1:172.21.15.159:6835/3484779950] conn(0x56047ecd9350 0x56047ecd9730 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3aa0c309368f90 server_cookie=845c806a1875aff4 in_seq=0 out_seq=0 2024-04-04T06:12:43.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.181+0000 7f175ea00640 10 client.7449 ms_handle_connect on v2:172.21.15.159:6833/3484779950 2024-04-04T06:12:43.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.181+0000 7f175ea00640 1 -- 172.21.15.159:0/952783225 <== mds.0 v2:172.21.15.159:6833/3484779950 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x56047ecdc650 con 0x56047ecd9350 2024-04-04T06:12:43.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.181+0000 7f175ea00640 10 client.7449 handle_command_reply: tid=0 2024-04-04T06:12:43.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.181+0000 7f178302f740 1 client.7449 shutdown 2024-04-04T06:12:43.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.181+0000 7f178302f740 1 -- 172.21.15.159:0/952783225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecd3510 msgr2=0x56047ed132a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:43.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.181+0000 7f178302f740 1 --2- 172.21.15.159:0/952783225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecd3510 0x56047ed132a0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f1764061b20 tx=0x7f17640590e0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.181+0000 7f178302f740 1 -- 172.21.15.159:0/952783225 shutdown_connections 2024-04-04T06:12:43.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.181+0000 7f178302f740 1 --2- 172.21.15.159:0/952783225 >> [v2:172.21.15.159:6833/3484779950,v1:172.21.15.159:6835/3484779950] conn(0x56047ecd9350 0x56047ecd9730 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.181+0000 7f178302f740 1 --2- 172.21.15.159:0/952783225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56047ecdbc10 0x56047ecdabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.181+0000 7f178302f740 1 --2- 172.21.15.159:0/952783225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56047ecd3510 0x56047ed132a0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.181+0000 7f178302f740 1 --2- 172.21.15.159:0/952783225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56047ecd3130 0x56047ed6a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.181+0000 7f178302f740 1 -- 172.21.15.159:0/952783225 >> 172.21.15.159:0/952783225 conn(0x56047eca6610 msgr2=0x56047eca4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.181+0000 7f175ea00640 0 client.7449 ms_handle_reset on v2:172.21.15.159:6833/3484779950 2024-04-04T06:12:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.181+0000 7f178302f740 1 -- 172.21.15.159:0/952783225 shutdown_connections 2024-04-04T06:12:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.181+0000 7f178302f740 1 -- 172.21.15.159:0/952783225 wait complete. 2024-04-04T06:12:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.181+0000 7f178302f740 20 client.7449 trim_cache size 0 max 16384 2024-04-04T06:12:43.179 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:12:43.179 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_fsid": "30311045-f714-48fa-832b-4cb2ba8bf0bf", 2024-04-04T06:12:43.179 INFO:teuthology.orchestra.run.smithi159.stdout: "whoami": 0, 2024-04-04T06:12:43.179 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 7293, 2024-04-04T06:12:43.179 INFO:teuthology.orchestra.run.smithi159.stdout: "want_state": "up:active", 2024-04-04T06:12:43.180 INFO:teuthology.orchestra.run.smithi159.stdout: "state": "up:active", 2024-04-04T06:12:43.180 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs", 2024-04-04T06:12:43.180 INFO:teuthology.orchestra.run.smithi159.stdout: "rank_uptime": 6.7610599889999996, 2024-04-04T06:12:43.180 INFO:teuthology.orchestra.run.smithi159.stdout: "mdsmap_epoch": 62, 2024-04-04T06:12:43.180 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch": 85, 2024-04-04T06:12:43.180 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch_barrier": 85, 2024-04-04T06:12:43.180 INFO:teuthology.orchestra.run.smithi159.stdout: "uptime": 12.182944238999999 2024-04-04T06:12:43.180 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:12:43.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.182+0000 7f1780400640 1 -- 172.21.15.159:0/3092401439 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f175806c920 msgr2=0x7f175806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:43.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.182+0000 7f1780400640 1 --2- 172.21.15.159:0/3092401439 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f175806c920 0x7f175806ede0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f176c036000 tx=0x7f176c030000 comp rx=0 tx=0).stop 2024-04-04T06:12:43.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.182+0000 7f1780400640 1 -- 172.21.15.159:0/3092401439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177815b260 msgr2=0x7f177814c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:43.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.182+0000 7f1780400640 1 --2- 172.21.15.159:0/3092401439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177815b260 0x7f177814c850 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f176400c410 tx=0x7f176400c8d0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.182+0000 7f1780400640 1 -- 172.21.15.159:0/3092401439 shutdown_connections 2024-04-04T06:12:43.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.182+0000 7f1780400640 1 --2- 172.21.15.159:0/3092401439 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f177815b640 0x7f17780ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.182+0000 7f1780400640 1 --2- 172.21.15.159:0/3092401439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f177815b260 0x7f177814c850 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.182+0000 7f1780400640 1 --2- 172.21.15.159:0/3092401439 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f175806c920 0x7f175806ede0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.182+0000 7f1780400640 1 --2- 172.21.15.159:0/3092401439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17780b06b0 0x7f17780ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.182+0000 7f1780400640 1 -- 172.21.15.159:0/3092401439 >> 172.21.15.159:0/3092401439 conn(0x7f17780b65a0 msgr2=0x7f177815e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:43.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.182+0000 7f1780400640 1 -- 172.21.15.159:0/3092401439 shutdown_connections 2024-04-04T06:12:43.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.182+0000 7f1780400640 1 -- 172.21.15.159:0/3092401439 wait complete. 2024-04-04T06:12:43.193 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-04T06:12:43.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.395+0000 7f19ea200640 1 Processor -- start 2024-04-04T06:12:43.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.396+0000 7f19ea200640 1 -- start start 2024-04-04T06:12:43.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.397+0000 7f19ea200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f19e4144ad0 0x7f19e40b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.397+0000 7f19ea200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19e415fba0 con 0x7f19e4072f50 2024-04-04T06:12:43.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.397+0000 7f19ea200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19e4144eb0 con 0x7f19e415f710 2024-04-04T06:12:43.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.397+0000 7f19ea200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f19e40cdad0 con 0x7f19e4144ad0 2024-04-04T06:12:43.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.397+0000 7f19e8e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f19e4144ad0 0x7f19e40b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:43.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.398+0000 7f19e8e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f19e4144ad0 0x7f19e40b2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56798/0 (socket says 172.21.15.159:56798) 2024-04-04T06:12:43.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.398+0000 7f19e8e00640 1 -- 172.21.15.159:0/509205921 learned_addr learned my addr 172.21.15.159:0/509205921 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:43.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.398+0000 7f19e8e00640 1 -- 172.21.15.159:0/509205921 >> v1:172.21.15.178:6789/0 conn(0x7f19e415f710 legacy=0x7f19e4072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:43.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.398+0000 7f19e8e00640 1 -- 172.21.15.159:0/509205921 >> v1:172.21.15.159:6789/0 conn(0x7f19e4072f50 legacy=0x7f19e40cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:43.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.398+0000 7f19e8e00640 1 -- 172.21.15.159:0/509205921 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e40b36f0 con 0x7f19e4144ad0 2024-04-04T06:12:43.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.399+0000 7f19e8e00640 1 --2- 172.21.15.159:0/509205921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19e4144ad0 0x7f19e40b2ff0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009870 tx=0x7f19d402dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf71b056841ee04 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:43.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.399+0000 7f19e3e00640 1 -- 172.21.15.159:0/509205921 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d403b070 con 0x7f19e4144ad0 2024-04-04T06:12:43.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.400+0000 7f19e3e00640 1 -- 172.21.15.159:0/509205921 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19d402e960 con 0x7f19e4144ad0 2024-04-04T06:12:43.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.400+0000 7f19e3e00640 1 -- 172.21.15.159:0/509205921 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d402ed30 con 0x7f19e4144ad0 2024-04-04T06:12:43.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.400+0000 7f19ea200640 1 -- 172.21.15.159:0/509205921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19e4144ad0 msgr2=0x7f19e40b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:43.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.400+0000 7f19ea200640 1 --2- 172.21.15.159:0/509205921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19e4144ad0 0x7f19e40b2ff0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009870 tx=0x7f19d402dbb0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.400+0000 7f19ea200640 1 -- 172.21.15.159:0/509205921 shutdown_connections 2024-04-04T06:12:43.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.400+0000 7f19ea200640 1 --2- 172.21.15.159:0/509205921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19e4144ad0 0x7f19e40b2ff0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.400+0000 7f19ea200640 1 -- 172.21.15.159:0/509205921 >> 172.21.15.159:0/509205921 conn(0x7f19e406b570 msgr2=0x7f19e406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:43.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.400+0000 7f19ea200640 1 -- 172.21.15.159:0/509205921 shutdown_connections 2024-04-04T06:12:43.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.400+0000 7f19ea200640 1 -- 172.21.15.159:0/509205921 wait complete. 2024-04-04T06:12:43.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.402+0000 7f19ea200640 1 Processor -- start 2024-04-04T06:12:43.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.402+0000 7f19ea200640 1 -- start start 2024-04-04T06:12:43.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.403+0000 7f19ea200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19e4072f50 0x7f19e415a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.403+0000 7f19ea200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19e4144ad0 0x7f19e413b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.403+0000 7f19ea200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19e415f710 0x7f19e415de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.403+0000 7f19ea200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f19e40b3f90 con 0x7f19e4072f50 2024-04-04T06:12:43.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.403+0000 7f19ea200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f19e40b3cb0 con 0x7f19e4144ad0 2024-04-04T06:12:43.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.403+0000 7f19ea200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f19e40ce670 con 0x7f19e415f710 2024-04-04T06:12:43.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.403+0000 7f19e8e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19e4072f50 0x7f19e415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:43.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.403+0000 7f19e8e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19e4072f50 0x7f19e415a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:56800/0 (socket says 172.21.15.159:56800) 2024-04-04T06:12:43.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.403+0000 7f19e8e00640 1 -- 172.21.15.159:0/1588181869 learned_addr learned my addr 172.21.15.159:0/1588181869 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:43.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.404+0000 7f19e8e00640 1 -- 172.21.15.159:0/1588181869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19e415f710 msgr2=0x7f19e415de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:43.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.404+0000 7f19e8e00640 1 --2- 172.21.15.159:0/1588181869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19e415f710 0x7f19e415de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.404+0000 7f19e8e00640 1 -- 172.21.15.159:0/1588181869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19e4144ad0 msgr2=0x7f19e413b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:43.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.404+0000 7f19e8e00640 1 --2- 172.21.15.159:0/1588181869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19e4144ad0 0x7f19e413b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.404+0000 7f19e8e00640 1 -- 172.21.15.159:0/1588181869 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e415cdd0 con 0x7f19e4072f50 2024-04-04T06:12:43.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.404+0000 7f19e8e00640 1 --2- 172.21.15.159:0/1588181869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19e4072f50 0x7f19e415a670 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009870 tx=0x7f19d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:43.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.404+0000 7f19e1600640 1 -- 172.21.15.159:0/1588181869 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4045020 con 0x7f19e4072f50 2024-04-04T06:12:43.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.404+0000 7f19e1600640 1 -- 172.21.15.159:0/1588181869 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19d4036d90 con 0x7f19e4072f50 2024-04-04T06:12:43.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.404+0000 7f19e1600640 1 -- 172.21.15.159:0/1588181869 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d403b040 con 0x7f19e4072f50 2024-04-04T06:12:43.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.404+0000 7f19ea200640 1 -- 172.21.15.159:0/1588181869 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e415d0b0 con 0x7f19e4072f50 2024-04-04T06:12:43.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.405+0000 7f19ea200640 1 -- 172.21.15.159:0/1588181869 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e415d670 con 0x7f19e4072f50 2024-04-04T06:12:43.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.406+0000 7f19ea200640 1 -- 172.21.15.159:0/1588181869 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e4075940 con 0x7f19e4072f50 2024-04-04T06:12:43.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.407+0000 7f19e1600640 1 -- 172.21.15.159:0/1588181869 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19d4048020 con 0x7f19e4072f50 2024-04-04T06:12:43.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.408+0000 7f19e1600640 1 --2- 172.21.15.159:0/1588181869 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f19c006c8f0 0x7f19c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:43.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.408+0000 7f19e1600640 1 -- 172.21.15.159:0/1588181869 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f19d40afd00 con 0x7f19e4072f50 2024-04-04T06:12:43.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.409+0000 7f19dbe00640 1 --2- 172.21.15.159:0/1588181869 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f19c006c8f0 0x7f19c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:43.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.410+0000 7f19dbe00640 1 --2- 172.21.15.159:0/1588181869 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f19c006c8f0 0x7f19c006edb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f19cc006fd0 tx=0x7f19cc006bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:43.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.413+0000 7f19e1600640 1 -- 172.21.15.159:0/1588181869 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f19d404f030 con 0x7f19e4072f50 2024-04-04T06:12:43.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.674+0000 7f19ea200640 1 -- 172.21.15.159:0/1588181869 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f19e40b4240 con 0x7f19e4072f50 2024-04-04T06:12:43.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.675+0000 7f19e1600640 1 -- 172.21.15.159:0/1588181869 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) ==== 53+0+73 (secure 0 0 0) 0x7f19d407e120 con 0x7f19e4072f50 2024-04-04T06:12:43.667 INFO:teuthology.orchestra.run.smithi159.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-04T06:12:43.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.677+0000 7f19ea200640 1 -- 172.21.15.159:0/1588181869 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f19c006c8f0 msgr2=0x7f19c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:43.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.677+0000 7f19ea200640 1 --2- 172.21.15.159:0/1588181869 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f19c006c8f0 0x7f19c006edb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f19cc006fd0 tx=0x7f19cc006bc0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.677+0000 7f19ea200640 1 -- 172.21.15.159:0/1588181869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19e4072f50 msgr2=0x7f19e415a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:43.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.677+0000 7f19ea200640 1 --2- 172.21.15.159:0/1588181869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19e4072f50 0x7f19e415a670 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009870 tx=0x7f19d40046c0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.677+0000 7f19ea200640 1 -- 172.21.15.159:0/1588181869 shutdown_connections 2024-04-04T06:12:43.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.677+0000 7f19ea200640 1 --2- 172.21.15.159:0/1588181869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19e415f710 0x7f19e415de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.677+0000 7f19ea200640 1 --2- 172.21.15.159:0/1588181869 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f19c006c8f0 0x7f19c006edb0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.677+0000 7f19ea200640 1 --2- 172.21.15.159:0/1588181869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19e4144ad0 0x7f19e413b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.677+0000 7f19ea200640 1 --2- 172.21.15.159:0/1588181869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19e4072f50 0x7f19e415a670 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:43.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.677+0000 7f19ea200640 1 -- 172.21.15.159:0/1588181869 >> 172.21.15.159:0/1588181869 conn(0x7f19e406b570 msgr2=0x7f19e40b0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:43.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.678+0000 7f19ea200640 1 -- 172.21.15.159:0/1588181869 shutdown_connections 2024-04-04T06:12:43.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:43.678+0000 7f19ea200640 1 -- 172.21.15.159:0/1588181869 wait complete. 2024-04-04T06:12:43.688 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-04T06:12:43.688 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-04T06:12:43.688 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi159.front.sepia.ceph.com 2024-04-04T06:12:43.688 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-04T06:12:43.688 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-04T06:12:43.688 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-04T06:12:43.689 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-04T06:12:43.689 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-04T06:12:43.689 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:12:43.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:43.689 DEBUG:teuthology.orchestra.run.smithi159:> ip addr 2024-04-04T06:12:43.735 INFO:teuthology.orchestra.run.smithi159.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:12:43.735 INFO:teuthology.orchestra.run.smithi159.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:12:43.735 INFO:teuthology.orchestra.run.smithi159.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:12:43.735 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:12:43.735 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 ::1/128 scope host 2024-04-04T06:12:43.736 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:12:43.736 INFO:teuthology.orchestra.run.smithi159.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:12:43.736 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c6 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:12:43.736 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f0 2024-04-04T06:12:43.736 INFO:teuthology.orchestra.run.smithi159.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:12:43.736 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c7 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:12:43.736 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f1 2024-04-04T06:12:43.736 INFO:teuthology.orchestra.run.smithi159.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:12:43.736 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4e brd ff:ff:ff:ff:ff:ff 2024-04-04T06:12:43.736 INFO:teuthology.orchestra.run.smithi159.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-04T06:12:43.736 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:12:43.736 INFO:teuthology.orchestra.run.smithi159.stdout: inet 172.21.15.159/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-04T06:12:43.736 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft 42169sec preferred_lft 42169sec 2024-04-04T06:12:43.736 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::ec4:7aff:fe88:724f/64 scope link 2024-04-04T06:12:43.737 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:12:43.737 INFO:teuthology.orchestra.run.smithi159.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-04T06:12:43.737 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:12:43.737 INFO:teuthology.orchestra.run.smithi159.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-04T06:12:43.737 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:12:43.737 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::989e:e4ff:fea5:7568/64 scope link 2024-04-04T06:12:43.737 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:12:43.737 INFO:teuthology.orchestra.run.smithi159.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:12:43.737 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether a2:e3:e0:cb:0a:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-04T06:12:43.737 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::a0e3:e0ff:fecb:a73/64 scope link 2024-04-04T06:12:43.737 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:12:43.737 INFO:teuthology.orchestra.run.smithi159.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:12:43.737 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-04T06:12:43.737 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::c0a:8aff:fe54:b58f/64 scope link 2024-04-04T06:12:43.737 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:12:43.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:43.739 DEBUG:teuthology.orchestra.run.smithi159:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-04T06:12:43.868 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:12:43.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:43.873 DEBUG:teuthology.orchestra.run.smithi159:> route 2024-04-04T06:12:43.923 INFO:teuthology.orchestra.run.smithi159.stdout:Kernel IP routing table 2024-04-04T06:12:43.923 INFO:teuthology.orchestra.run.smithi159.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-04T06:12:43.923 INFO:teuthology.orchestra.run.smithi159.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-04T06:12:43.923 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-04T06:12:43.924 INFO:teuthology.orchestra.run.smithi159.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-04T06:12:43.924 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:12:43.924 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:12:43.924 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-04T06:12:43.924 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-04T06:12:43.924 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-04T06:12:43.924 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:12:44.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:12:44.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:12:44.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:44.165 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-04-04T06:12:44.213 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-04T06:12:44.214 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-04T06:12:44.214 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:12:44.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:44.214 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set brx.0 up 2024-04-04T06:12:44.313 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:12:44.313 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-04T06:12:44.365 INFO:teuthology.orchestra.run.smithi159.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:12:44.365 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:12:44.365 DEBUG:teuthology.orchestra.run.smithi159:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:12:44.413 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-04T06:12:44.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:44.413 DEBUG:teuthology.orchestra.run.smithi159:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:12:44.582 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:12:44.582 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:12:44.582 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:12:44.603 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:12:44.603 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:12:44.603 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:12:44.604 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:12:44.604 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:12:44.631 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:12:44.631 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:12:44.631 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:12:44.631 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:12:44.631 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:12:44.631 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-04T06:12:44.631 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-04T06:12:44.632 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30311045-f714-48fa-832b-4cb2ba8bf0bf" will pass to kernel 2024-04-04T06:12:44.634 INFO:teuthology.orchestra.run.smithi159.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-04T06:12:44.634 INFO:teuthology.orchestra.run.smithi159.stdout: You just mounted a file system that supports labels which does not 2024-04-04T06:12:44.634 INFO:teuthology.orchestra.run.smithi159.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-04T06:12:44.634 INFO:teuthology.orchestra.run.smithi159.stdout: applications will generate AVC messages and not be allowed access to 2024-04-04T06:12:44.634 INFO:teuthology.orchestra.run.smithi159.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-04T06:12:44.639 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-04T06:12:44.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:44.639 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:12:44.719 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:12:44.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:12:44.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:12:44.837 INFO:teuthology.orchestra.run.smithi159.stdout:client5818 2024-04-04T06:12:44.837 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:12:44.837 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:12:44.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:44.883 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-04-04T06:12:44.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:44.983 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-04-04T06:12:45.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:45.029 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-04-04T06:12:45.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:45.121 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-04-04T06:12:45.176 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:12:45.176 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:12:45.176 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:12:45.176 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:12:45.176 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-04-04T06:12:45.176 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:12:45.300 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627776 2024-04-04T06:12:45.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:12:45.313 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:12:45.314 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:12:45.406 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:12:45.406 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:12:45.457 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:12:45.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.666+0000 7f510a200640 1 Processor -- start 2024-04-04T06:12:45.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.666+0000 7f510a200640 1 -- start start 2024-04-04T06:12:45.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.669+0000 7f510a200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5104161b00 0x7f5104162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:45.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.669+0000 7f510a200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040ad120 0x7f51040d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:45.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.669+0000 7f510a200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5104071b60 con 0x7f51040ad580 2024-04-04T06:12:45.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.669+0000 7f510a200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f5104161ee0 con 0x7f5104161b00 2024-04-04T06:12:45.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.669+0000 7f510a200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f51040b3450 con 0x7f51040ad120 2024-04-04T06:12:45.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.670+0000 7f5102a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040ad120 0x7f51040d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:45.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.670+0000 7f5102a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040ad120 0x7f51040d0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:51908/0 (socket says 172.21.15.159:51908) 2024-04-04T06:12:45.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.670+0000 7f5102a00640 1 -- 172.21.15.159:0/3846101440 learned_addr learned my addr 172.21.15.159:0/3846101440 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:45.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.670+0000 7f5103400640 1 --2- 172.21.15.159:0/3846101440 >> v2:172.21.15.178:3300/0 conn(0x7f5104161b00 0x7f5104162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:45.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.670+0000 7f5102a00640 1 -- 172.21.15.159:0/3846101440 >> v2:172.21.15.178:3300/0 conn(0x7f5104161b00 msgr2=0x7f5104162ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:45.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.670+0000 7f5102a00640 1 --2- 172.21.15.159:0/3846101440 >> v2:172.21.15.178:3300/0 conn(0x7f5104161b00 0x7f5104162ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:45.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.670+0000 7f5102a00640 1 -- 172.21.15.159:0/3846101440 >> v1:172.21.15.178:6789/0 conn(0x7f51040ad580 legacy=0x7f51040b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:45.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.670+0000 7f5102a00640 1 -- 172.21.15.159:0/3846101440 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5104071e40 con 0x7f51040ad120 2024-04-04T06:12:45.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.671+0000 7f5103400640 1 --2- 172.21.15.159:0/3846101440 >> v2:172.21.15.178:3300/0 conn(0x7f5104161b00 0x7f5104162ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:12:45.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.673+0000 7f5102a00640 1 --2- 172.21.15.159:0/3846101440 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040ad120 0x7f51040d0650 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009870 tx=0x7f50f002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fb73f195043a23d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:45.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.674+0000 7f5102000640 1 -- 172.21.15.159:0/3846101440 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0034070 con 0x7f51040ad120 2024-04-04T06:12:45.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.674+0000 7f5102000640 1 -- 172.21.15.159:0/3846101440 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f0030840 con 0x7f51040ad120 2024-04-04T06:12:45.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.674+0000 7f5102000640 1 -- 172.21.15.159:0/3846101440 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0030c10 con 0x7f51040ad120 2024-04-04T06:12:45.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.674+0000 7f510a200640 1 -- 172.21.15.159:0/3846101440 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040ad120 msgr2=0x7f51040d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:45.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.674+0000 7f510a200640 1 --2- 172.21.15.159:0/3846101440 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040ad120 0x7f51040d0650 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009870 tx=0x7f50f002b980 comp rx=0 tx=0).stop 2024-04-04T06:12:45.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.674+0000 7f510a200640 1 -- 172.21.15.159:0/3846101440 shutdown_connections 2024-04-04T06:12:45.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.674+0000 7f510a200640 1 --2- 172.21.15.159:0/3846101440 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040ad120 0x7f51040d0650 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:45.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.674+0000 7f510a200640 1 --2- 172.21.15.159:0/3846101440 >> v2:172.21.15.178:3300/0 conn(0x7f5104161b00 0x7f5104162ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:45.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.674+0000 7f510a200640 1 -- 172.21.15.159:0/3846101440 >> 172.21.15.159:0/3846101440 conn(0x7f510406b990 msgr2=0x7f510406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:45.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.674+0000 7f510a200640 1 -- 172.21.15.159:0/3846101440 shutdown_connections 2024-04-04T06:12:45.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.674+0000 7f510a200640 1 -- 172.21.15.159:0/3846101440 wait complete. 2024-04-04T06:12:45.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.676+0000 7f510a200640 1 Processor -- start 2024-04-04T06:12:45.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.676+0000 7f510a200640 1 -- start start 2024-04-04T06:12:45.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.677+0000 7f510a200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51040ad120 0x7f51041573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:45.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.677+0000 7f510a200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040ad580 0x7f5104154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:45.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.677+0000 7f510a200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5104161b00 0x7f51041455b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:45.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.677+0000 7f510a200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5104162ff0 con 0x7f51040ad120 2024-04-04T06:12:45.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.677+0000 7f510a200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f51040b3730 con 0x7f5104161b00 2024-04-04T06:12:45.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.677+0000 7f510a200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f51040717f0 con 0x7f51040ad580 2024-04-04T06:12:45.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.678+0000 7f5103400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51040ad120 0x7f51041573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:45.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.678+0000 7f5103400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51040ad120 0x7f51041573f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42628/0 (socket says 172.21.15.159:42628) 2024-04-04T06:12:45.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.678+0000 7f5103400640 1 -- 172.21.15.159:0/2598300677 learned_addr learned my addr 172.21.15.159:0/2598300677 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:45.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.678+0000 7f5102a00640 1 --2- 172.21.15.159:0/2598300677 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040ad580 0x7f5104154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:45.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.678+0000 7f5103e00640 1 --2- 172.21.15.159:0/2598300677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5104161b00 0x7f51041455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:45.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.678+0000 7f5103400640 1 -- 172.21.15.159:0/2598300677 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040ad580 msgr2=0x7f5104154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:45.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.678+0000 7f5103400640 1 --2- 172.21.15.159:0/2598300677 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040ad580 0x7f5104154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:45.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.678+0000 7f5103400640 1 -- 172.21.15.159:0/2598300677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5104161b00 msgr2=0x7f51041455b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:45.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.678+0000 7f5103400640 1 --2- 172.21.15.159:0/2598300677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5104161b00 0x7f51041455b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:45.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.678+0000 7f5103400640 1 -- 172.21.15.159:0/2598300677 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5104157af0 con 0x7f51040ad120 2024-04-04T06:12:45.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.678+0000 7f5102a00640 1 --2- 172.21.15.159:0/2598300677 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040ad580 0x7f5104154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:45.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.678+0000 7f5103e00640 1 --2- 172.21.15.159:0/2598300677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5104161b00 0x7f51041455b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:45.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.678+0000 7f5103400640 1 --2- 172.21.15.159:0/2598300677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51040ad120 0x7f51041573f0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f50f400e960 tx=0x7f50f400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:45.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.679+0000 7f50fb400640 1 -- 172.21.15.159:0/2598300677 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4017070 con 0x7f51040ad120 2024-04-04T06:12:45.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.679+0000 7f50fb400640 1 -- 172.21.15.159:0/2598300677 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f400ca00 con 0x7f51040ad120 2024-04-04T06:12:45.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.679+0000 7f510a200640 1 -- 172.21.15.159:0/2598300677 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5104157dd0 con 0x7f51040ad120 2024-04-04T06:12:45.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.679+0000 7f50fb400640 1 -- 172.21.15.159:0/2598300677 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f400cdd0 con 0x7f51040ad120 2024-04-04T06:12:45.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.679+0000 7f510a200640 1 -- 172.21.15.159:0/2598300677 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5104158390 con 0x7f51040ad120 2024-04-04T06:12:45.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.680+0000 7f510a200640 1 -- 172.21.15.159:0/2598300677 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51040685c0 con 0x7f51040ad120 2024-04-04T06:12:45.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.681+0000 7f50fb400640 1 -- 172.21.15.159:0/2598300677 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f50f401a070 con 0x7f51040ad120 2024-04-04T06:12:45.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.682+0000 7f50fb400640 1 --2- 172.21.15.159:0/2598300677 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f50e406c8f0 0x7f50e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:45.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.682+0000 7f50fb400640 1 -- 172.21.15.159:0/2598300677 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f50f408be60 con 0x7f51040ad120 2024-04-04T06:12:45.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.683+0000 7f5102a00640 1 --2- 172.21.15.159:0/2598300677 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f50e406c8f0 0x7f50e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:45.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.684+0000 7f5102a00640 1 --2- 172.21.15.159:0/2598300677 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f50e406c8f0 0x7f50e406edb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f50f0036000 tx=0x7f50f003b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:45.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.687+0000 7f50fb400640 1 -- 172.21.15.159:0/2598300677 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f50f405a1f0 con 0x7f51040ad120 2024-04-04T06:12:45.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.903+0000 7f510a200640 1 -- 172.21.15.159:0/2598300677 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f510406f3a0 con 0x7f51040ad120 2024-04-04T06:12:45.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.906+0000 7f50fb400640 1 -- 172.21.15.159:0/2598300677 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16672 (secure 0 0 0) 0x7f50f4059af0 con 0x7f51040ad120 2024-04-04T06:12:45.899 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:45.899 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":85,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:12:37.412279+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:12:33.658405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-04T06:12:34.856266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:12:45.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.909+0000 7f510a200640 1 -- 172.21.15.159:0/2598300677 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f50e406c8f0 msgr2=0x7f50e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:45.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.909+0000 7f510a200640 1 --2- 172.21.15.159:0/2598300677 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f50e406c8f0 0x7f50e406edb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f50f0036000 tx=0x7f50f003b040 comp rx=0 tx=0).stop 2024-04-04T06:12:45.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.909+0000 7f510a200640 1 -- 172.21.15.159:0/2598300677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51040ad120 msgr2=0x7f51041573f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:45.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.909+0000 7f510a200640 1 --2- 172.21.15.159:0/2598300677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51040ad120 0x7f51041573f0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f50f400e960 tx=0x7f50f400ee20 comp rx=0 tx=0).stop 2024-04-04T06:12:45.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.909+0000 7f510a200640 1 -- 172.21.15.159:0/2598300677 shutdown_connections 2024-04-04T06:12:45.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.909+0000 7f510a200640 1 --2- 172.21.15.159:0/2598300677 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f50e406c8f0 0x7f50e406edb0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:45.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.909+0000 7f510a200640 1 --2- 172.21.15.159:0/2598300677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5104161b00 0x7f51041455b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:45.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.909+0000 7f510a200640 1 --2- 172.21.15.159:0/2598300677 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51040ad580 0x7f5104154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:45.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.909+0000 7f510a200640 1 --2- 172.21.15.159:0/2598300677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51040ad120 0x7f51041573f0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:45.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.909+0000 7f510a200640 1 -- 172.21.15.159:0/2598300677 >> 172.21.15.159:0/2598300677 conn(0x7f510406b990 msgr2=0x7f5104162310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:45.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.909+0000 7f510a200640 1 -- 172.21.15.159:0/2598300677 shutdown_connections 2024-04-04T06:12:45.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:45.909+0000 7f510a200640 1 -- 172.21.15.159:0/2598300677 wait complete. 2024-04-04T06:12:45.922 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:12:33.658405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:12:34.856266+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:12:45.923 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:12:46.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.140+0000 7fa365e00640 1 Processor -- start 2024-04-04T06:12:46.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.140+0000 7fa365e00640 1 -- start start 2024-04-04T06:12:46.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.141+0000 7fa365e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa360155550 0x7fa360155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:46.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.141+0000 7fa365e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa36015f800 0x7fa3600d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:46.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.141+0000 7fa365e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3600b40b0 con 0x7fa3600d07c0 2024-04-04T06:12:46.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.141+0000 7fa365e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fa3600d0bf0 con 0x7fa360155550 2024-04-04T06:12:46.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.141+0000 7fa365e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa3600b4390 con 0x7fa36015f800 2024-04-04T06:12:46.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.142+0000 7fa35ea00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa360155550 0x7fa360155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:46.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.142+0000 7fa35ea00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa360155550 0x7fa360155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42636/0 (socket says 172.21.15.159:42636) 2024-04-04T06:12:46.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.142+0000 7fa35ea00640 1 -- 172.21.15.159:0/2189458740 learned_addr learned my addr 172.21.15.159:0/2189458740 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:46.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.142+0000 7fa35ea00640 1 -- 172.21.15.159:0/2189458740 >> v2:172.21.15.178:3300/0 conn(0x7fa36015f800 msgr2=0x7fa3600d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:46.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.142+0000 7fa35ea00640 1 --2- 172.21.15.159:0/2189458740 >> v2:172.21.15.178:3300/0 conn(0x7fa36015f800 0x7fa3600d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:46.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.142+0000 7fa35ea00640 1 -- 172.21.15.159:0/2189458740 >> v1:172.21.15.178:6789/0 conn(0x7fa3600d07c0 legacy=0x7fa3600aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:46.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.142+0000 7fa35ea00640 1 -- 172.21.15.159:0/2189458740 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3600b4670 con 0x7fa360155550 2024-04-04T06:12:46.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.143+0000 7fa35ea00640 1 --2- 172.21.15.159:0/2189458740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa360155550 0x7fa360155930 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fa3600d0dd0 tx=0x7fa35002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2934db00f5597edb server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:46.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.144+0000 7fa35d600640 1 -- 172.21.15.159:0/2189458740 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35002c9f0 con 0x7fa360155550 2024-04-04T06:12:46.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.144+0000 7fa35d600640 1 -- 172.21.15.159:0/2189458740 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa35002cb90 con 0x7fa360155550 2024-04-04T06:12:46.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.144+0000 7fa35d600640 1 -- 172.21.15.159:0/2189458740 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3500343f0 con 0x7fa360155550 2024-04-04T06:12:46.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.144+0000 7fa365e00640 1 -- 172.21.15.159:0/2189458740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa360155550 msgr2=0x7fa360155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:46.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.144+0000 7fa365e00640 1 --2- 172.21.15.159:0/2189458740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa360155550 0x7fa360155930 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fa3600d0dd0 tx=0x7fa35002be50 comp rx=0 tx=0).stop 2024-04-04T06:12:46.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.145+0000 7fa365e00640 1 -- 172.21.15.159:0/2189458740 shutdown_connections 2024-04-04T06:12:46.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.145+0000 7fa365e00640 1 --2- 172.21.15.159:0/2189458740 >> v2:172.21.15.178:3300/0 conn(0x7fa36015f800 0x7fa3600d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:46.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.145+0000 7fa365e00640 1 --2- 172.21.15.159:0/2189458740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa360155550 0x7fa360155930 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:46.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.145+0000 7fa365e00640 1 -- 172.21.15.159:0/2189458740 >> 172.21.15.159:0/2189458740 conn(0x7fa36006b750 msgr2=0x7fa36006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:46.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.145+0000 7fa365e00640 1 -- 172.21.15.159:0/2189458740 shutdown_connections 2024-04-04T06:12:46.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.145+0000 7fa365e00640 1 -- 172.21.15.159:0/2189458740 wait complete. 2024-04-04T06:12:46.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.146+0000 7fa365e00640 1 Processor -- start 2024-04-04T06:12:46.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.147+0000 7fa365e00640 1 -- start start 2024-04-04T06:12:46.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.148+0000 7fa365e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3600d07c0 0x7fa36015e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:46.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.148+0000 7fa365e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa360155550 0x7fa36009b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:46.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.148+0000 7fa365e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa36015f800 0x7fa36015a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:46.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.148+0000 7fa365e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa36015fc80 con 0x7fa36015f800 2024-04-04T06:12:46.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.148+0000 7fa365e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa3600b4c30 con 0x7fa3600d07c0 2024-04-04T06:12:46.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.148+0000 7fa365e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa36015ba20 con 0x7fa360155550 2024-04-04T06:12:46.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.148+0000 7fa35f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa36015f800 0x7fa36015a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:46.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.148+0000 7fa35f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa36015f800 0x7fa36015a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42650/0 (socket says 172.21.15.159:42650) 2024-04-04T06:12:46.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.148+0000 7fa35f400640 1 -- 172.21.15.159:0/3172454555 learned_addr learned my addr 172.21.15.159:0/3172454555 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:46.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.148+0000 7fa35ea00640 1 --2- 172.21.15.159:0/3172454555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3600d07c0 0x7fa36015e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:46.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.148+0000 7fa35e000640 1 --2- 172.21.15.159:0/3172454555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa360155550 0x7fa36009b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:46.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.148+0000 7fa35f400640 1 -- 172.21.15.159:0/3172454555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa360155550 msgr2=0x7fa36009b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:46.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.148+0000 7fa35f400640 1 --2- 172.21.15.159:0/3172454555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa360155550 0x7fa36009b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:46.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.148+0000 7fa35f400640 1 -- 172.21.15.159:0/3172454555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3600d07c0 msgr2=0x7fa36015e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:46.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.148+0000 7fa35f400640 1 --2- 172.21.15.159:0/3172454555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3600d07c0 0x7fa36015e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:46.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.149+0000 7fa35f400640 1 -- 172.21.15.159:0/3172454555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3600ce950 con 0x7fa36015f800 2024-04-04T06:12:46.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.149+0000 7fa35f400640 1 --2- 172.21.15.159:0/3172454555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa36015f800 0x7fa36015a2d0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fa35800b440 tx=0x7fa35800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:46.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.149+0000 7fa357400640 1 -- 172.21.15.159:0/3172454555 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358015020 con 0x7fa36015f800 2024-04-04T06:12:46.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.149+0000 7fa357400640 1 -- 172.21.15.159:0/3172454555 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa358011830 con 0x7fa36015f800 2024-04-04T06:12:46.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.149+0000 7fa357400640 1 -- 172.21.15.159:0/3172454555 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358011c20 con 0x7fa36015f800 2024-04-04T06:12:46.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.149+0000 7fa365e00640 1 -- 172.21.15.159:0/3172454555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36015db30 con 0x7fa36015f800 2024-04-04T06:12:46.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.149+0000 7fa365e00640 1 -- 172.21.15.159:0/3172454555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa36009b720 con 0x7fa36015f800 2024-04-04T06:12:46.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.151+0000 7fa365e00640 1 -- 172.21.15.159:0/3172454555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3600685c0 con 0x7fa36015f800 2024-04-04T06:12:46.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.152+0000 7fa357400640 1 -- 172.21.15.159:0/3172454555 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa358018020 con 0x7fa36015f800 2024-04-04T06:12:46.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.153+0000 7fa357400640 1 --2- 172.21.15.159:0/3172454555 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa33c06c8f0 0x7fa33c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:46.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.153+0000 7fa357400640 1 -- 172.21.15.159:0/3172454555 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fa35808b790 con 0x7fa36015f800 2024-04-04T06:12:46.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.153+0000 7fa35ea00640 1 --2- 172.21.15.159:0/3172454555 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa33c06c8f0 0x7fa33c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:46.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.155+0000 7fa35ea00640 1 --2- 172.21.15.159:0/3172454555 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa33c06c8f0 0x7fa33c06edb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa350035000 tx=0x7fa350030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:46.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.157+0000 7fa357400640 1 -- 172.21.15.159:0/3172454555 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa35801f030 con 0x7fa36015f800 2024-04-04T06:12:46.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.359+0000 7fa365e00640 1 -- 172.21.15.159:0/3172454555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa3600b3ad0 con 0x7fa36015f800 2024-04-04T06:12:46.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.360+0000 7fa357400640 1 -- 172.21.15.159:0/3172454555 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7fa358059a90 con 0x7fa36015f800 2024-04-04T06:12:46.352 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:12:46.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.362+0000 7fa365e00640 1 -- 172.21.15.159:0/3172454555 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa33c06c8f0 msgr2=0x7fa33c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:46.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.362+0000 7fa365e00640 1 --2- 172.21.15.159:0/3172454555 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa33c06c8f0 0x7fa33c06edb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa350035000 tx=0x7fa350030000 comp rx=0 tx=0).stop 2024-04-04T06:12:46.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.362+0000 7fa365e00640 1 -- 172.21.15.159:0/3172454555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa36015f800 msgr2=0x7fa36015a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:46.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.362+0000 7fa365e00640 1 --2- 172.21.15.159:0/3172454555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa36015f800 0x7fa36015a2d0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fa35800b440 tx=0x7fa35800b900 comp rx=0 tx=0).stop 2024-04-04T06:12:46.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.362+0000 7fa365e00640 1 -- 172.21.15.159:0/3172454555 shutdown_connections 2024-04-04T06:12:46.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.362+0000 7fa365e00640 1 --2- 172.21.15.159:0/3172454555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa36015f800 0x7fa36015a2d0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:46.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.362+0000 7fa365e00640 1 --2- 172.21.15.159:0/3172454555 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa360155550 0x7fa36009b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:46.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.362+0000 7fa365e00640 1 --2- 172.21.15.159:0/3172454555 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa33c06c8f0 0x7fa33c06edb0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:46.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.362+0000 7fa365e00640 1 --2- 172.21.15.159:0/3172454555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3600d07c0 0x7fa36015e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:46.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.362+0000 7fa365e00640 1 -- 172.21.15.159:0/3172454555 >> 172.21.15.159:0/3172454555 conn(0x7fa36006b750 msgr2=0x7fa360072080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:46.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.363+0000 7fa365e00640 1 -- 172.21.15.159:0/3172454555 shutdown_connections 2024-04-04T06:12:46.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.363+0000 7fa365e00640 1 -- 172.21.15.159:0/3172454555 wait complete. 2024-04-04T06:12:46.374 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:12:46.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.586+0000 7fbf53600640 1 Processor -- start 2024-04-04T06:12:46.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.586+0000 7fbf53600640 1 -- start start 2024-04-04T06:12:46.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.588+0000 7fbf53600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fbf4c0b3cf0 0x7fbf4c071940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:46.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.588+0000 7fbf53600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbf4c071e80 0x7fbf4c0ad010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:46.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.588+0000 7fbf53600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf4c0723c0 con 0x7fbf4c0d03b0 2024-04-04T06:12:46.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.588+0000 7fbf53600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fbf4c0726a0 con 0x7fbf4c0b3cf0 2024-04-04T06:12:46.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.588+0000 7fbf53600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fbf4c0b3210 con 0x7fbf4c071e80 2024-04-04T06:12:46.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.588+0000 7fbf50c00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fbf4c0d03b0 0x7fbf4c0d0790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:39358/0 (socket says 172.21.15.159:39358) 2024-04-04T06:12:46.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.588+0000 7fbf50c00640 1 -- 172.21.15.159:0/227741029 learned_addr learned my addr 172.21.15.159:0/227741029 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:46.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.588+0000 7fbf4be00640 1 --2- 172.21.15.159:0/227741029 >> v2:172.21.15.159:3300/0 conn(0x7fbf4c0b3cf0 0x7fbf4c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:46.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.589+0000 7fbf51600640 1 --2- 172.21.15.159:0/227741029 >> v2:172.21.15.178:3300/0 conn(0x7fbf4c071e80 0x7fbf4c0ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:46.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.589+0000 7fbf4b400640 1 -- 172.21.15.159:0/227741029 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3427137630 0 0) 0x7fbf4c0723c0 con 0x7fbf4c0d03b0 2024-04-04T06:12:46.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.589+0000 7fbf4b400640 1 -- 172.21.15.159:0/227741029 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf34005730 con 0x7fbf4c0d03b0 2024-04-04T06:12:46.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.589+0000 7fbf4be00640 1 -- 172.21.15.159:0/227741029 >> v2:172.21.15.178:3300/0 conn(0x7fbf4c071e80 msgr2=0x7fbf4c0ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:46.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.589+0000 7fbf4be00640 1 --2- 172.21.15.159:0/227741029 >> v2:172.21.15.178:3300/0 conn(0x7fbf4c071e80 0x7fbf4c0ad010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:46.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.589+0000 7fbf4be00640 1 -- 172.21.15.159:0/227741029 >> v1:172.21.15.159:6789/0 conn(0x7fbf4c0d03b0 legacy=0x7fbf4c0d0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:46.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.589+0000 7fbf4be00640 1 -- 172.21.15.159:0/227741029 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf4c0b34f0 con 0x7fbf4c0b3cf0 2024-04-04T06:12:46.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.590+0000 7fbf4be00640 1 --2- 172.21.15.159:0/227741029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf4c0b3cf0 0x7fbf4c071940 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0b2db0 tx=0x7fbf3802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f00b30491d118f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:46.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.591+0000 7fbf4b400640 1 -- 172.21.15.159:0/227741029 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf380315a0 con 0x7fbf4c0b3cf0 2024-04-04T06:12:46.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.591+0000 7fbf4b400640 1 -- 172.21.15.159:0/227741029 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf38037070 con 0x7fbf4c0b3cf0 2024-04-04T06:12:46.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.591+0000 7fbf4b400640 1 -- 172.21.15.159:0/227741029 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38031960 con 0x7fbf4c0b3cf0 2024-04-04T06:12:46.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.591+0000 7fbf53600640 1 -- 172.21.15.159:0/227741029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf4c0b3cf0 msgr2=0x7fbf4c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:46.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.591+0000 7fbf53600640 1 --2- 172.21.15.159:0/227741029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf4c0b3cf0 0x7fbf4c071940 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0b2db0 tx=0x7fbf3802b840 comp rx=0 tx=0).stop 2024-04-04T06:12:46.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.591+0000 7fbf53600640 1 -- 172.21.15.159:0/227741029 shutdown_connections 2024-04-04T06:12:46.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.591+0000 7fbf53600640 1 --2- 172.21.15.159:0/227741029 >> v2:172.21.15.178:3300/0 conn(0x7fbf4c071e80 0x7fbf4c0ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:46.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.591+0000 7fbf53600640 1 --2- 172.21.15.159:0/227741029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf4c0b3cf0 0x7fbf4c071940 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:46.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.591+0000 7fbf53600640 1 -- 172.21.15.159:0/227741029 >> 172.21.15.159:0/227741029 conn(0x7fbf4c06b750 msgr2=0x7fbf4c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:46.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.591+0000 7fbf53600640 1 -- 172.21.15.159:0/227741029 shutdown_connections 2024-04-04T06:12:46.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.591+0000 7fbf53600640 1 -- 172.21.15.159:0/227741029 wait complete. 2024-04-04T06:12:46.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.593+0000 7fbf53600640 1 Processor -- start 2024-04-04T06:12:46.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.593+0000 7fbf53600640 1 -- start start 2024-04-04T06:12:46.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.594+0000 7fbf53600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf4c071e80 0x7fbf4c152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:46.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.594+0000 7fbf53600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf4c0b3cf0 0x7fbf4c15e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:46.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.594+0000 7fbf53600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf4c0d03b0 0x7fbf4c15cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:46.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.594+0000 7fbf53600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4c15a980 con 0x7fbf4c0b3cf0 2024-04-04T06:12:46.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.594+0000 7fbf53600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4c15a800 con 0x7fbf4c0d03b0 2024-04-04T06:12:46.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.594+0000 7fbf53600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbf4c06f700 con 0x7fbf4c071e80 2024-04-04T06:12:46.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.594+0000 7fbf4be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf4c0b3cf0 0x7fbf4c15e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:46.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.594+0000 7fbf4be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf4c0b3cf0 0x7fbf4c15e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42662/0 (socket says 172.21.15.159:42662) 2024-04-04T06:12:46.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.595+0000 7fbf4be00640 1 -- 172.21.15.159:0/2230205123 learned_addr learned my addr 172.21.15.159:0/2230205123 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:46.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.595+0000 7fbf4be00640 1 -- 172.21.15.159:0/2230205123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf4c071e80 msgr2=0x7fbf4c152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:46.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.595+0000 7fbf4be00640 1 --2- 172.21.15.159:0/2230205123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf4c071e80 0x7fbf4c152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:46.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.595+0000 7fbf4be00640 1 -- 172.21.15.159:0/2230205123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf4c0d03b0 msgr2=0x7fbf4c15cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:46.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.595+0000 7fbf4be00640 1 --2- 172.21.15.159:0/2230205123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf4c0d03b0 0x7fbf4c15cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:46.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.595+0000 7fbf4be00640 1 -- 172.21.15.159:0/2230205123 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf4c0b1490 con 0x7fbf4c0b3cf0 2024-04-04T06:12:46.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.595+0000 7fbf4be00640 1 --2- 172.21.15.159:0/2230205123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf4c0b3cf0 0x7fbf4c15e500 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009890 tx=0x7fbf380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:46.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.596+0000 7fbf48c00640 1 -- 172.21.15.159:0/2230205123 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38032630 con 0x7fbf4c0b3cf0 2024-04-04T06:12:46.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.596+0000 7fbf48c00640 1 -- 172.21.15.159:0/2230205123 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf380327f0 con 0x7fbf4c0b3cf0 2024-04-04T06:12:46.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.596+0000 7fbf53600640 1 -- 172.21.15.159:0/2230205123 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf4c09af60 con 0x7fbf4c0b3cf0 2024-04-04T06:12:46.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.596+0000 7fbf48c00640 1 -- 172.21.15.159:0/2230205123 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38032be0 con 0x7fbf4c0b3cf0 2024-04-04T06:12:46.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.596+0000 7fbf53600640 1 -- 172.21.15.159:0/2230205123 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf4c155060 con 0x7fbf4c0b3cf0 2024-04-04T06:12:46.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.597+0000 7fbf53600640 1 -- 172.21.15.159:0/2230205123 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf4c0685c0 con 0x7fbf4c0b3cf0 2024-04-04T06:12:46.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.598+0000 7fbf48c00640 1 -- 172.21.15.159:0/2230205123 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf3804c020 con 0x7fbf4c0b3cf0 2024-04-04T06:12:46.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.599+0000 7fbf48c00640 1 --2- 172.21.15.159:0/2230205123 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbf2c06c920 0x7fbf2c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:46.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.599+0000 7fbf48c00640 1 -- 172.21.15.159:0/2230205123 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fbf3802f020 con 0x7fbf4c0b3cf0 2024-04-04T06:12:46.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.599+0000 7fbf50c00640 1 --2- 172.21.15.159:0/2230205123 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbf2c06c920 0x7fbf2c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:46.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.600+0000 7fbf50c00640 1 --2- 172.21.15.159:0/2230205123 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbf2c06c920 0x7fbf2c06ede0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbf40000f60 tx=0x7fbf400078f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:46.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.603+0000 7fbf48c00640 1 -- 172.21.15.159:0/2230205123 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbf4c09af60 con 0x7fbf4c0b3cf0 2024-04-04T06:12:46.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.813+0000 7fbf53600640 1 -- 172.21.15.159:0/2230205123 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbf4c156a00 con 0x7fbf4c0b3cf0 2024-04-04T06:12:46.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.814+0000 7fbf48c00640 1 -- 172.21.15.159:0/2230205123 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) ==== 104+0+11 (secure 0 0 0) 0x7fbf38040070 con 0x7fbf4c0b3cf0 2024-04-04T06:12:46.806 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:12:46.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.816+0000 7fbf53600640 1 -- 172.21.15.159:0/2230205123 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbf2c06c920 msgr2=0x7fbf2c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:46.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.816+0000 7fbf53600640 1 --2- 172.21.15.159:0/2230205123 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbf2c06c920 0x7fbf2c06ede0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbf40000f60 tx=0x7fbf400078f0 comp rx=0 tx=0).stop 2024-04-04T06:12:46.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.816+0000 7fbf53600640 1 -- 172.21.15.159:0/2230205123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf4c0b3cf0 msgr2=0x7fbf4c15e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:46.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.816+0000 7fbf53600640 1 --2- 172.21.15.159:0/2230205123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf4c0b3cf0 0x7fbf4c15e500 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009890 tx=0x7fbf380046c0 comp rx=0 tx=0).stop 2024-04-04T06:12:46.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.816+0000 7fbf53600640 1 -- 172.21.15.159:0/2230205123 shutdown_connections 2024-04-04T06:12:46.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.816+0000 7fbf53600640 1 --2- 172.21.15.159:0/2230205123 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbf2c06c920 0x7fbf2c06ede0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:46.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.816+0000 7fbf53600640 1 --2- 172.21.15.159:0/2230205123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf4c0d03b0 0x7fbf4c15cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:46.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.816+0000 7fbf53600640 1 --2- 172.21.15.159:0/2230205123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf4c0b3cf0 0x7fbf4c15e500 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:46.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.817+0000 7fbf53600640 1 --2- 172.21.15.159:0/2230205123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf4c071e80 0x7fbf4c152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:46.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.817+0000 7fbf53600640 1 -- 172.21.15.159:0/2230205123 >> 172.21.15.159:0/2230205123 conn(0x7fbf4c06b750 msgr2=0x7fbf4c1574d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:46.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.817+0000 7fbf53600640 1 -- 172.21.15.159:0/2230205123 shutdown_connections 2024-04-04T06:12:46.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:46.817+0000 7fbf53600640 1 -- 172.21.15.159:0/2230205123 wait complete. 2024-04-04T06:12:46.826 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:12:47.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.035+0000 7f0da1400640 1 Processor -- start 2024-04-04T06:12:47.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.035+0000 7f0da1400640 1 -- start start 2024-04-04T06:12:47.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.036+0000 7f0da1400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d9c0d37b0 0x7f0d9c0b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:47.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.036+0000 7f0da1400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0d9c0b67f0 0x7f0d9c0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:47.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.036+0000 7f0da1400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d9c0d2710 con 0x7f0d9c15e4c0 2024-04-04T06:12:47.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.036+0000 7f0da1400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0d9c15e8a0 con 0x7f0d9c0b67f0 2024-04-04T06:12:47.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.036+0000 7f0da1400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0d9c0d4040 con 0x7f0d9c0d37b0 2024-04-04T06:12:47.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.037+0000 7f0d9be00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f0d9c15e4c0 0x7f0d9c0b9bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:39374/0 (socket says 172.21.15.159:39374) 2024-04-04T06:12:47.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.037+0000 7f0d9be00640 1 -- 172.21.15.159:0/2175885996 learned_addr learned my addr 172.21.15.159:0/2175885996 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:47.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.037+0000 7f0d9b400640 1 -- 172.21.15.159:0/2175885996 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3776390905 0 0) 0x7f0d9c0d2710 con 0x7f0d9c15e4c0 2024-04-04T06:12:47.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.037+0000 7f0da0a00640 1 --2- 172.21.15.159:0/2175885996 >> v2:172.21.15.178:3300/0 conn(0x7f0d9c0b67f0 0x7f0d9c0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:47.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.037+0000 7f0d93e00640 1 --2- 172.21.15.159:0/2175885996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d9c0d37b0 0x7f0d9c0b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:47.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.037+0000 7f0d9b400640 1 -- 172.21.15.159:0/2175885996 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d80003520 con 0x7f0d9c15e4c0 2024-04-04T06:12:47.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.038+0000 7f0d9b400640 1 -- 172.21.15.159:0/2175885996 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3834847327 0 0) 0x7f0d80003520 con 0x7f0d9c15e4c0 2024-04-04T06:12:47.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.038+0000 7f0d9b400640 1 -- 172.21.15.159:0/2175885996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d9c0d37b0 msgr2=0x7f0d9c0b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:47.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.038+0000 7f0d9b400640 1 --2- 172.21.15.159:0/2175885996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d9c0d37b0 0x7f0d9c0b62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.038+0000 7f0d9b400640 1 -- 172.21.15.159:0/2175885996 >> v2:172.21.15.178:3300/0 conn(0x7f0d9c0b67f0 msgr2=0x7f0d9c0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:47.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.038+0000 7f0d9b400640 1 --2- 172.21.15.159:0/2175885996 >> v2:172.21.15.178:3300/0 conn(0x7f0d9c0b67f0 0x7f0d9c0b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.038+0000 7f0d9b400640 1 -- 172.21.15.159:0/2175885996 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c0d29f0 con 0x7f0d9c15e4c0 2024-04-04T06:12:47.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.038+0000 7f0d9b400640 1 -- 172.21.15.159:0/2175885996 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f0d8c001670 con 0x7f0d9c15e4c0 2024-04-04T06:12:47.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.038+0000 7f0d9b400640 1 -- 172.21.15.159:0/2175885996 >> v1:172.21.15.159:6789/0 conn(0x7f0d9c15e4c0 legacy=0x7f0d9c0b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:47.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.038+0000 7f0d9b400640 1 --2- 172.21.15.159:0/2175885996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d9c0d37b0 0x7f0d800039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:47.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.038+0000 7f0d9b400640 1 --2- 172.21.15.159:0/2175885996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d80003fe0 0x7f0d80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:47.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.038+0000 7f0d9b400640 1 --2- 172.21.15.159:0/2175885996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d80004d40 0x7f0d800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:47.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.038+0000 7f0d9b400640 1 -- 172.21.15.159:0/2175885996 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c0d4040 con 0x7f0d80004d40 2024-04-04T06:12:47.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.038+0000 7f0d9b400640 1 -- 172.21.15.159:0/2175885996 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8000ba70 con 0x7f0d80003fe0 2024-04-04T06:12:47.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.038+0000 7f0d9b400640 1 -- 172.21.15.159:0/2175885996 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0d8000bc40 con 0x7f0d9c0d37b0 2024-04-04T06:12:47.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.038+0000 7f0da0a00640 1 --2- 172.21.15.159:0/2175885996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d80004d40 0x7f0d800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:47.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.038+0000 7f0d9be00640 1 --2- 172.21.15.159:0/2175885996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d9c0d37b0 0x7f0d800039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:47.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.039+0000 7f0d93e00640 1 --2- 172.21.15.159:0/2175885996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d80003fe0 0x7f0d80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:47.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.039+0000 7f0da0a00640 1 -- 172.21.15.159:0/2175885996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d9c0d37b0 msgr2=0x7f0d800039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:47.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.039+0000 7f0da0a00640 1 --2- 172.21.15.159:0/2175885996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d9c0d37b0 0x7f0d800039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.039+0000 7f0da0a00640 1 -- 172.21.15.159:0/2175885996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d80003fe0 msgr2=0x7f0d80004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:47.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.039+0000 7f0da0a00640 1 --2- 172.21.15.159:0/2175885996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d80003fe0 0x7f0d80004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.039+0000 7f0da0a00640 1 -- 172.21.15.159:0/2175885996 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d80003520 con 0x7f0d80004d40 2024-04-04T06:12:47.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.039+0000 7f0d93e00640 1 --2- 172.21.15.159:0/2175885996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d80003fe0 0x7f0d80004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:47.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.040+0000 7f0da0a00640 1 --2- 172.21.15.159:0/2175885996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d80004d40 0x7f0d800051c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f0d94007360 tx=0x7f0d80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d6215fd6fbdafb server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:47.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.040+0000 7f0d9b400640 1 -- 172.21.15.159:0/2175885996 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94039070 con 0x7f0d80004d40 2024-04-04T06:12:47.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.040+0000 7f0d9b400640 1 -- 172.21.15.159:0/2175885996 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d9402ccf0 con 0x7f0d80004d40 2024-04-04T06:12:47.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.040+0000 7f0d9b400640 1 -- 172.21.15.159:0/2175885996 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94009be0 con 0x7f0d80004d40 2024-04-04T06:12:47.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.041+0000 7f0da1400640 1 -- 172.21.15.159:0/2175885996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d80004d40 msgr2=0x7f0d800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:47.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.041+0000 7f0da1400640 1 --2- 172.21.15.159:0/2175885996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d80004d40 0x7f0d800051c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f0d94007360 tx=0x7f0d80005830 comp rx=0 tx=0).stop 2024-04-04T06:12:47.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.041+0000 7f0da0a00640 1 -- 172.21.15.159:0/2175885996 reap_dead start 2024-04-04T06:12:47.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.041+0000 7f0da1400640 1 -- 172.21.15.159:0/2175885996 shutdown_connections 2024-04-04T06:12:47.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.041+0000 7f0da1400640 1 -- 172.21.15.159:0/2175885996 >> 172.21.15.159:0/2175885996 conn(0x7f0d9c0676a0 msgr2=0x7f0d9c067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:47.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.041+0000 7f0da1400640 1 -- 172.21.15.159:0/2175885996 shutdown_connections 2024-04-04T06:12:47.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.041+0000 7f0da1400640 1 -- 172.21.15.159:0/2175885996 wait complete. 2024-04-04T06:12:47.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.043+0000 7f0da1400640 1 Processor -- start 2024-04-04T06:12:47.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.043+0000 7f0da1400640 1 -- start start 2024-04-04T06:12:47.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.044+0000 7f0da1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d80004d40 0x7f0d9c1508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:47.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.044+0000 7f0da1400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d9c0cb3a0 0x7f0d9c0c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:47.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.044+0000 7f0da1400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c0cb820 0x7f0d9c0c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:47.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.044+0000 7f0da1400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c0d2fb0 con 0x7f0d80004d40 2024-04-04T06:12:47.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.044+0000 7f0da1400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c0d3320 con 0x7f0d9c0cb820 2024-04-04T06:12:47.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.044+0000 7f0da1400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0d9c0d2350 con 0x7f0d9c0cb3a0 2024-04-04T06:12:47.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.044+0000 7f0d9be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d80004d40 0x7f0d9c1508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:47.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.044+0000 7f0d9be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d80004d40 0x7f0d9c1508a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42670/0 (socket says 172.21.15.159:42670) 2024-04-04T06:12:47.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.044+0000 7f0d9be00640 1 -- 172.21.15.159:0/3251731903 learned_addr learned my addr 172.21.15.159:0/3251731903 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:47.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.045+0000 7f0d9be00640 1 -- 172.21.15.159:0/3251731903 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d9c0cb3a0 msgr2=0x7f0d9c0c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:47.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.045+0000 7f0d9be00640 1 --2- 172.21.15.159:0/3251731903 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d9c0cb3a0 0x7f0d9c0c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.045+0000 7f0d9be00640 1 -- 172.21.15.159:0/3251731903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c0cb820 msgr2=0x7f0d9c0c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:47.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.045+0000 7f0d9be00640 1 --2- 172.21.15.159:0/3251731903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c0cb820 0x7f0d9c0c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.045+0000 7f0d9be00640 1 -- 172.21.15.159:0/3251731903 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c150de0 con 0x7f0d80004d40 2024-04-04T06:12:47.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.045+0000 7f0d9be00640 1 --2- 172.21.15.159:0/3251731903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d80004d40 0x7f0d9c1508a0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c007340 tx=0x7f0d8c007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:47.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.045+0000 7f0d98c00640 1 -- 172.21.15.159:0/3251731903 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00f8c0 con 0x7f0d80004d40 2024-04-04T06:12:47.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.045+0000 7f0d98c00640 1 -- 172.21.15.159:0/3251731903 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d8c018070 con 0x7f0d80004d40 2024-04-04T06:12:47.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.045+0000 7f0d98c00640 1 -- 172.21.15.159:0/3251731903 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00fc80 con 0x7f0d80004d40 2024-04-04T06:12:47.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.045+0000 7f0da1400640 1 -- 172.21.15.159:0/3251731903 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9c15fc60 con 0x7f0d80004d40 2024-04-04T06:12:47.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.045+0000 7f0da1400640 1 -- 172.21.15.159:0/3251731903 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9c160220 con 0x7f0d80004d40 2024-04-04T06:12:47.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.047+0000 7f0da1400640 1 -- 172.21.15.159:0/3251731903 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d9c160ce0 con 0x7f0d80004d40 2024-04-04T06:12:47.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.047+0000 7f0d98c00640 1 -- 172.21.15.159:0/3251731903 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d8c01d460 con 0x7f0d80004d40 2024-04-04T06:12:47.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.049+0000 7f0d98c00640 1 --2- 172.21.15.159:0/3251731903 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0d7806c920 0x7f0d7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:47.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.049+0000 7f0d98c00640 1 -- 172.21.15.159:0/3251731903 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f0d8c08bce0 con 0x7f0d80004d40 2024-04-04T06:12:47.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.049+0000 7f0d93e00640 1 --2- 172.21.15.159:0/3251731903 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0d7806c920 0x7f0d7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:47.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.051+0000 7f0d93e00640 1 --2- 172.21.15.159:0/3251731903 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0d7806c920 0x7f0d7806ede0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c15abf0 tx=0x7f0d84009350 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:47.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.053+0000 7f0d98c00640 1 -- 172.21.15.159:0/3251731903 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0d9c160ce0 con 0x7f0d80004d40 2024-04-04T06:12:47.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.253+0000 7f0da1400640 1 -- 172.21.15.159:0/3251731903 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0d9c075940 con 0x7f0d80004d40 2024-04-04T06:12:47.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.254+0000 7f0d98c00640 1 -- 172.21.15.159:0/3251731903 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) ==== 100+0+11 (secure 0 0 0) 0x7f0d9c150de0 con 0x7f0d80004d40 2024-04-04T06:12:47.246 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:12:47.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.256+0000 7f0da1400640 1 -- 172.21.15.159:0/3251731903 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0d7806c920 msgr2=0x7f0d7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:47.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.256+0000 7f0da1400640 1 --2- 172.21.15.159:0/3251731903 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0d7806c920 0x7f0d7806ede0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c15abf0 tx=0x7f0d84009350 comp rx=0 tx=0).stop 2024-04-04T06:12:47.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.256+0000 7f0da1400640 1 -- 172.21.15.159:0/3251731903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d80004d40 msgr2=0x7f0d9c1508a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:47.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.256+0000 7f0da1400640 1 --2- 172.21.15.159:0/3251731903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d80004d40 0x7f0d9c1508a0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c007340 tx=0x7f0d8c007800 comp rx=0 tx=0).stop 2024-04-04T06:12:47.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.257+0000 7f0da1400640 1 -- 172.21.15.159:0/3251731903 shutdown_connections 2024-04-04T06:12:47.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.257+0000 7f0da1400640 1 --2- 172.21.15.159:0/3251731903 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0d7806c920 0x7f0d7806ede0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.257+0000 7f0da1400640 1 --2- 172.21.15.159:0/3251731903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d9c0cb820 0x7f0d9c0c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.257+0000 7f0da1400640 1 --2- 172.21.15.159:0/3251731903 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d9c0cb3a0 0x7f0d9c0c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.257+0000 7f0da1400640 1 --2- 172.21.15.159:0/3251731903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d80004d40 0x7f0d9c1508a0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.257+0000 7f0da1400640 1 -- 172.21.15.159:0/3251731903 >> 172.21.15.159:0/3251731903 conn(0x7f0d9c0676a0 msgr2=0x7f0d9c06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:47.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.257+0000 7f0da1400640 1 -- 172.21.15.159:0/3251731903 shutdown_connections 2024-04-04T06:12:47.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.257+0000 7f0da1400640 1 -- 172.21.15.159:0/3251731903 wait complete. 2024-04-04T06:12:47.274 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-04T06:12:47.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.494+0000 7f26cfc00640 1 Processor -- start 2024-04-04T06:12:47.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.494+0000 7f26cfc00640 1 -- start start 2024-04-04T06:12:47.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.495+0000 7f26cfc00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f26c8155550 0x7f26c8155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:47.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.495+0000 7f26cfc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f26c815f800 0x7f26c80d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:47.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.495+0000 7f26cfc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26c80d07c0 0x7f26c80aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:47.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.495+0000 7f26cfc00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f26c80d0c10 con 0x7f26c815f800 2024-04-04T06:12:47.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.495+0000 7f26cfc00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f26c80b40b0 con 0x7f26c8155550 2024-04-04T06:12:47.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.495+0000 7f26cfc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f26c80b4390 con 0x7f26c80d07c0 2024-04-04T06:12:47.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.495+0000 7f26bfe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f26c815f800 0x7f26c80d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:47.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.496+0000 7f26bfe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f26c815f800 0x7f26c80d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42686/0 (socket says 172.21.15.159:42686) 2024-04-04T06:12:47.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.496+0000 7f26bfe00640 1 -- 172.21.15.159:0/3852406358 learned_addr learned my addr 172.21.15.159:0/3852406358 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:47.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.496+0000 7f26cd200640 1 --2- 172.21.15.159:0/3852406358 >> v2:172.21.15.178:3300/0 conn(0x7f26c8155550 0x7f26c8155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:47.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.496+0000 7f26cdc00640 1 --2- 172.21.15.159:0/3852406358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26c80d07c0 0x7f26c80aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:47.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.496+0000 7f26bfe00640 1 -- 172.21.15.159:0/3852406358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26c80d07c0 msgr2=0x7f26c80aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:47.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.496+0000 7f26bfe00640 1 --2- 172.21.15.159:0/3852406358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26c80d07c0 0x7f26c80aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.496+0000 7f26bfe00640 1 -- 172.21.15.159:0/3852406358 >> v2:172.21.15.178:3300/0 conn(0x7f26c8155550 msgr2=0x7f26c8155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:47.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.496+0000 7f26bfe00640 1 --2- 172.21.15.159:0/3852406358 >> v2:172.21.15.178:3300/0 conn(0x7f26c8155550 0x7f26c8155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.496+0000 7f26bfe00640 1 -- 172.21.15.159:0/3852406358 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c80b4670 con 0x7f26c815f800 2024-04-04T06:12:47.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.496+0000 7f26cd200640 1 --2- 172.21.15.159:0/3852406358 >> v2:172.21.15.178:3300/0 conn(0x7f26c8155550 0x7f26c8155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:47.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.496+0000 7f26cdc00640 1 --2- 172.21.15.159:0/3852406358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26c80d07c0 0x7f26c80aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:47.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.498+0000 7f26bfe00640 1 --2- 172.21.15.159:0/3852406358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26c815f800 0x7f26c80d0280 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f26c8073ac0 tx=0x7f26b002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa56a7dab4f5693a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:47.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.498+0000 7f26c7e00640 1 -- 172.21.15.159:0/3852406358 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0035020 con 0x7f26c815f800 2024-04-04T06:12:47.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.498+0000 7f26c7e00640 1 -- 172.21.15.159:0/3852406358 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b0031870 con 0x7f26c815f800 2024-04-04T06:12:47.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.498+0000 7f26c7e00640 1 -- 172.21.15.159:0/3852406358 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0031c40 con 0x7f26c815f800 2024-04-04T06:12:47.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.499+0000 7f26cfc00640 1 -- 172.21.15.159:0/3852406358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26c815f800 msgr2=0x7f26c80d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:47.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.499+0000 7f26cfc00640 1 --2- 172.21.15.159:0/3852406358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26c815f800 0x7f26c80d0280 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f26c8073ac0 tx=0x7f26b002b910 comp rx=0 tx=0).stop 2024-04-04T06:12:47.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.499+0000 7f26cfc00640 1 -- 172.21.15.159:0/3852406358 shutdown_connections 2024-04-04T06:12:47.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.499+0000 7f26cfc00640 1 --2- 172.21.15.159:0/3852406358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26c80d07c0 0x7f26c80aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.499+0000 7f26cfc00640 1 --2- 172.21.15.159:0/3852406358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26c815f800 0x7f26c80d0280 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.499+0000 7f26cfc00640 1 --2- 172.21.15.159:0/3852406358 >> v2:172.21.15.178:3300/0 conn(0x7f26c8155550 0x7f26c8155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.499+0000 7f26cfc00640 1 -- 172.21.15.159:0/3852406358 >> 172.21.15.159:0/3852406358 conn(0x7f26c806b750 msgr2=0x7f26c806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:47.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.499+0000 7f26cfc00640 1 -- 172.21.15.159:0/3852406358 shutdown_connections 2024-04-04T06:12:47.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.499+0000 7f26cfc00640 1 -- 172.21.15.159:0/3852406358 wait complete. 2024-04-04T06:12:47.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.501+0000 7f26cfc00640 1 Processor -- start 2024-04-04T06:12:47.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.501+0000 7f26cfc00640 1 -- start start 2024-04-04T06:12:47.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.502+0000 7f26cfc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26c80d07c0 0x7f26c81f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:47.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.502+0000 7f26cfc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26c8155550 0x7f26c814a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:47.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.502+0000 7f26cfc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26c815f800 0x7f26c80b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:47.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.502+0000 7f26cfc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f26c815fc80 con 0x7f26c8155550 2024-04-04T06:12:47.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.502+0000 7f26cfc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f26c80b4c30 con 0x7f26c815f800 2024-04-04T06:12:47.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.502+0000 7f26cfc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f26c815ba20 con 0x7f26c80d07c0 2024-04-04T06:12:47.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.502+0000 7f26bfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26c8155550 0x7f26c814a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:47.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.502+0000 7f26bfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26c8155550 0x7f26c814a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42700/0 (socket says 172.21.15.159:42700) 2024-04-04T06:12:47.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.502+0000 7f26bfe00640 1 -- 172.21.15.159:0/2257275859 learned_addr learned my addr 172.21.15.159:0/2257275859 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:47.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.502+0000 7f26cd200640 1 --2- 172.21.15.159:0/2257275859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26c80d07c0 0x7f26c81f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:47.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.503+0000 7f26bfe00640 1 -- 172.21.15.159:0/2257275859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26c80d07c0 msgr2=0x7f26c81f3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:47.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.503+0000 7f26bfe00640 1 --2- 172.21.15.159:0/2257275859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26c80d07c0 0x7f26c81f3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.503+0000 7f26bfe00640 1 -- 172.21.15.159:0/2257275859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26c815f800 msgr2=0x7f26c80b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:47.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.503+0000 7f26bfe00640 1 --2- 172.21.15.159:0/2257275859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26c815f800 0x7f26c80b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.503+0000 7f26bfe00640 1 -- 172.21.15.159:0/2257275859 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c80adee0 con 0x7f26c8155550 2024-04-04T06:12:47.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.503+0000 7f26bfe00640 1 --2- 172.21.15.159:0/2257275859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26c8155550 0x7f26c814a160 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f26b0000c00 tx=0x7f26b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:47.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.503+0000 7f26c5600640 1 -- 172.21.15.159:0/2257275859 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0040070 con 0x7f26c8155550 2024-04-04T06:12:47.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.503+0000 7f26c5600640 1 -- 172.21.15.159:0/2257275859 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b003a420 con 0x7f26c8155550 2024-04-04T06:12:47.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.503+0000 7f26cfc00640 1 -- 172.21.15.159:0/2257275859 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c80ae1c0 con 0x7f26c8155550 2024-04-04T06:12:47.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.503+0000 7f26c5600640 1 -- 172.21.15.159:0/2257275859 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0035040 con 0x7f26c8155550 2024-04-04T06:12:47.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.504+0000 7f26cfc00640 1 -- 172.21.15.159:0/2257275859 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c80ae780 con 0x7f26c8155550 2024-04-04T06:12:47.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.505+0000 7f26cfc00640 1 -- 172.21.15.159:0/2257275859 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c80685c0 con 0x7f26c8155550 2024-04-04T06:12:47.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.506+0000 7f26c5600640 1 -- 172.21.15.159:0/2257275859 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26b004b060 con 0x7f26c8155550 2024-04-04T06:12:47.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.506+0000 7f26c5600640 1 --2- 172.21.15.159:0/2257275859 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f26a406c8f0 0x7f26a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:47.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.507+0000 7f26c5600640 1 -- 172.21.15.159:0/2257275859 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f26b00ad9f0 con 0x7f26c8155550 2024-04-04T06:12:47.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.507+0000 7f26cd200640 1 --2- 172.21.15.159:0/2257275859 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f26a406c8f0 0x7f26a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:47.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.508+0000 7f26cd200640 1 --2- 172.21.15.159:0/2257275859 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f26a406c8f0 0x7f26a406edb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f26b800a910 tx=0x7f26b8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:47.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.511+0000 7f26c5600640 1 -- 172.21.15.159:0/2257275859 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f26b007be10 con 0x7f26c8155550 2024-04-04T06:12:47.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.789+0000 7f26cfc00640 1 -- 172.21.15.159:0/2257275859 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f26c80cebc0 con 0x7f26c8155550 2024-04-04T06:12:47.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.789+0000 7f26c5600640 1 -- 172.21.15.159:0/2257275859 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v85) ==== 80+0+0 (secure 0 0 0) 0x7f26b007b710 con 0x7f26c8155550 2024-04-04T06:12:47.844 INFO:teuthology.orchestra.run.smithi159.stderr:listed 0 entries 2024-04-04T06:12:47.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.792+0000 7f26cfc00640 1 -- 172.21.15.159:0/2257275859 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f26a406c8f0 msgr2=0x7f26a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:47.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.792+0000 7f26cfc00640 1 --2- 172.21.15.159:0/2257275859 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f26a406c8f0 0x7f26a406edb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f26b800a910 tx=0x7f26b8008040 comp rx=0 tx=0).stop 2024-04-04T06:12:47.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.792+0000 7f26cfc00640 1 -- 172.21.15.159:0/2257275859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26c8155550 msgr2=0x7f26c814a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:47.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.792+0000 7f26cfc00640 1 --2- 172.21.15.159:0/2257275859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26c8155550 0x7f26c814a160 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f26b0000c00 tx=0x7f26b00046c0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.792+0000 7f26cfc00640 1 -- 172.21.15.159:0/2257275859 shutdown_connections 2024-04-04T06:12:47.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.792+0000 7f26cfc00640 1 --2- 172.21.15.159:0/2257275859 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f26a406c8f0 0x7f26a406edb0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.792+0000 7f26cfc00640 1 --2- 172.21.15.159:0/2257275859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26c815f800 0x7f26c80b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.792+0000 7f26cfc00640 1 --2- 172.21.15.159:0/2257275859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26c8155550 0x7f26c814a160 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.792+0000 7f26cfc00640 1 --2- 172.21.15.159:0/2257275859 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26c80d07c0 0x7f26c81f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:47.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.792+0000 7f26cfc00640 1 -- 172.21.15.159:0/2257275859 >> 172.21.15.159:0/2257275859 conn(0x7f26c806b750 msgr2=0x7f26c815a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:47.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.792+0000 7f26cfc00640 1 -- 172.21.15.159:0/2257275859 shutdown_connections 2024-04-04T06:12:47.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:47.793+0000 7f26cfc00640 1 -- 172.21.15.159:0/2257275859 wait complete. 2024-04-04T06:12:47.846 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-04T06:12:47.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:47.846 DEBUG:teuthology.orchestra.run.smithi159:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-04T06:12:47.964 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:12:47.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:47.965 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-04T06:12:48.024 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:12:48.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.224+0000 7fbc3b400640 1 Processor -- start 2024-04-04T06:12:48.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.225+0000 7fbc3b400640 1 -- start start 2024-04-04T06:12:48.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.227+0000 7fbc3b400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc3415eab0 0x7fbc3406c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:48.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.227+0000 7fbc3b400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fbc340b5990 0x7fbc3406cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:48.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.227+0000 7fbc3b400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc340b53e0 con 0x7fbc34072a00 2024-04-04T06:12:48.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.227+0000 7fbc3b400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fbc340b56c0 con 0x7fbc340b5990 2024-04-04T06:12:48.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.227+0000 7fbc3b400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbc340adc80 con 0x7fbc3415eab0 2024-04-04T06:12:48.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.228+0000 7fbc2be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fbc340b5990 0x7fbc3406cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:48.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.228+0000 7fbc2be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fbc340b5990 0x7fbc3406cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42712/0 (socket says 172.21.15.159:42712) 2024-04-04T06:12:48.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.228+0000 7fbc2be00640 1 -- 172.21.15.159:0/4153432396 learned_addr learned my addr 172.21.15.159:0/4153432396 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:48.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.228+0000 7fbc38a00640 1 --2- 172.21.15.159:0/4153432396 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc3415eab0 0x7fbc3406c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:48.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.228+0000 7fbc2be00640 1 -- 172.21.15.159:0/4153432396 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc3415eab0 msgr2=0x7fbc3406c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:48.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.228+0000 7fbc2be00640 1 --2- 172.21.15.159:0/4153432396 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc3415eab0 0x7fbc3406c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:48.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.228+0000 7fbc2be00640 1 -- 172.21.15.159:0/4153432396 >> v1:172.21.15.178:6789/0 conn(0x7fbc34072a00 legacy=0x7fbc340b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:48.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.228+0000 7fbc2be00640 1 -- 172.21.15.159:0/4153432396 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc340af490 con 0x7fbc340b5990 2024-04-04T06:12:48.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.229+0000 7fbc2be00640 1 --2- 172.21.15.159:0/4153432396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc340b5990 0x7fbc3406cac0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009870 tx=0x7fbc1c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c75b348d4c8555 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:48.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.230+0000 7fbc33e00640 1 -- 172.21.15.159:0/4153432396 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c035020 con 0x7fbc340b5990 2024-04-04T06:12:48.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.230+0000 7fbc33e00640 1 -- 172.21.15.159:0/4153432396 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc1c031640 con 0x7fbc340b5990 2024-04-04T06:12:48.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.230+0000 7fbc33e00640 1 -- 172.21.15.159:0/4153432396 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c02d040 con 0x7fbc340b5990 2024-04-04T06:12:48.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.230+0000 7fbc3b400640 1 -- 172.21.15.159:0/4153432396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc340b5990 msgr2=0x7fbc3406cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:48.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.230+0000 7fbc3b400640 1 --2- 172.21.15.159:0/4153432396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc340b5990 0x7fbc3406cac0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009870 tx=0x7fbc1c02b8f0 comp rx=0 tx=0).stop 2024-04-04T06:12:48.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.230+0000 7fbc3b400640 1 -- 172.21.15.159:0/4153432396 shutdown_connections 2024-04-04T06:12:48.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.231+0000 7fbc3b400640 1 --2- 172.21.15.159:0/4153432396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc340b5990 0x7fbc3406cac0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:48.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.231+0000 7fbc3b400640 1 --2- 172.21.15.159:0/4153432396 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc3415eab0 0x7fbc3406c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:48.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.231+0000 7fbc3b400640 1 -- 172.21.15.159:0/4153432396 >> 172.21.15.159:0/4153432396 conn(0x7fbc34071df0 msgr2=0x7fbc340721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:48.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.231+0000 7fbc3b400640 1 -- 172.21.15.159:0/4153432396 shutdown_connections 2024-04-04T06:12:48.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.231+0000 7fbc3b400640 1 -- 172.21.15.159:0/4153432396 wait complete. 2024-04-04T06:12:48.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.233+0000 7fbc3b400640 1 Processor -- start 2024-04-04T06:12:48.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.234+0000 7fbc3b400640 1 -- start start 2024-04-04T06:12:48.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.235+0000 7fbc3b400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc34072a00 0x7fbc3414b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:48.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.235+0000 7fbc3b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc340b5990 0x7fbc340b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:48.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.235+0000 7fbc3b400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc3415eab0 0x7fbc340b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:48.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.235+0000 7fbc3b400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbc340b4430 con 0x7fbc340b5990 2024-04-04T06:12:48.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.235+0000 7fbc3b400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc340b42b0 con 0x7fbc3415eab0 2024-04-04T06:12:48.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.235+0000 7fbc3b400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbc340b45b0 con 0x7fbc34072a00 2024-04-04T06:12:48.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.235+0000 7fbc2be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc340b5990 0x7fbc340b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:48.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.235+0000 7fbc2be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc340b5990 0x7fbc340b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42724/0 (socket says 172.21.15.159:42724) 2024-04-04T06:12:48.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.235+0000 7fbc2be00640 1 -- 172.21.15.159:0/989536852 learned_addr learned my addr 172.21.15.159:0/989536852 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:48.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.236+0000 7fbc38a00640 1 --2- 172.21.15.159:0/989536852 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc34072a00 0x7fbc3414b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:48.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.236+0000 7fbc2be00640 1 -- 172.21.15.159:0/989536852 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc34072a00 msgr2=0x7fbc3414b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:48.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.236+0000 7fbc39400640 1 --2- 172.21.15.159:0/989536852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc3415eab0 0x7fbc340b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:48.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.236+0000 7fbc2be00640 1 --2- 172.21.15.159:0/989536852 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc34072a00 0x7fbc3414b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:48.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.236+0000 7fbc2be00640 1 -- 172.21.15.159:0/989536852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc3415eab0 msgr2=0x7fbc340b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:48.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.236+0000 7fbc2be00640 1 --2- 172.21.15.159:0/989536852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc3415eab0 0x7fbc340b0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:48.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.236+0000 7fbc2be00640 1 -- 172.21.15.159:0/989536852 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3415f030 con 0x7fbc340b5990 2024-04-04T06:12:48.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.236+0000 7fbc2be00640 1 --2- 172.21.15.159:0/989536852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc340b5990 0x7fbc340b0990 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c000c00 tx=0x7fbc1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:48.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.236+0000 7fbc31600640 1 -- 172.21.15.159:0/989536852 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c040070 con 0x7fbc340b5990 2024-04-04T06:12:48.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.236+0000 7fbc31600640 1 -- 172.21.15.159:0/989536852 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc1c0399f0 con 0x7fbc340b5990 2024-04-04T06:12:48.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.236+0000 7fbc31600640 1 -- 172.21.15.159:0/989536852 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c035040 con 0x7fbc340b5990 2024-04-04T06:12:48.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.237+0000 7fbc3b400640 1 -- 172.21.15.159:0/989536852 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc3415f2d0 con 0x7fbc340b5990 2024-04-04T06:12:48.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.237+0000 7fbc3b400640 1 -- 172.21.15.159:0/989536852 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc3415f890 con 0x7fbc340b5990 2024-04-04T06:12:48.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.238+0000 7fbc3b400640 1 -- 172.21.15.159:0/989536852 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc340685c0 con 0x7fbc340b5990 2024-04-04T06:12:48.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.239+0000 7fbc31600640 1 -- 172.21.15.159:0/989536852 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc1c04b060 con 0x7fbc340b5990 2024-04-04T06:12:48.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.240+0000 7fbc31600640 1 --2- 172.21.15.159:0/989536852 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc1006c8f0 0x7fbc1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:48.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.240+0000 7fbc31600640 1 -- 172.21.15.159:0/989536852 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fbc1c0ad4a0 con 0x7fbc340b5990 2024-04-04T06:12:48.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.240+0000 7fbc38a00640 1 --2- 172.21.15.159:0/989536852 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc1006c8f0 0x7fbc1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:48.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.241+0000 7fbc38a00640 1 --2- 172.21.15.159:0/989536852 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc1006c8f0 0x7fbc1006edb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fbc24006fd0 tx=0x7fbc24008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:48.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.245+0000 7fbc31600640 1 -- 172.21.15.159:0/989536852 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbc1c07b8d0 con 0x7fbc340b5990 2024-04-04T06:12:48.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.482+0000 7fbc3b400640 1 -- 172.21.15.159:0/989536852 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc3406b9f0 con 0x7fbc340b5990 2024-04-04T06:12:48.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.486+0000 7fbc31600640 1 -- 172.21.15.159:0/989536852 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5195 (secure 0 0 0) 0x7fbc1c07b1d0 con 0x7fbc340b5990 2024-04-04T06:12:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:48.479 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1825651864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1825651864},{"type":"v1","addr":"172.21.15.178:6836","nonce":1825651864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1824578712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1824578712},{"type":"v1","addr":"172.21.15.178:6837","nonce":1824578712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6834/3377498298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3377498298},{"type":"v1","addr":"172.21.15.159:6834","nonce":3377498298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:12:35.880454+0000","modified":"2024-04-04T06:12:38.922420+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7293":{"gid":7293,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6835/3484779950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":3484779950},{"type":"v1","addr":"172.21.15.159:6835","nonce":3484779950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7293,"qdb_cluster":[7293]},"id":6}]} 2024-04-04T06:12:48.479 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 62 2024-04-04T06:12:48.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.488+0000 7fbc3b400640 1 -- 172.21.15.159:0/989536852 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc1006c8f0 msgr2=0x7fbc1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:48.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.488+0000 7fbc3b400640 1 --2- 172.21.15.159:0/989536852 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc1006c8f0 0x7fbc1006edb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fbc24006fd0 tx=0x7fbc24008040 comp rx=0 tx=0).stop 2024-04-04T06:12:48.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.488+0000 7fbc3b400640 1 -- 172.21.15.159:0/989536852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc340b5990 msgr2=0x7fbc340b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:48.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.488+0000 7fbc3b400640 1 --2- 172.21.15.159:0/989536852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc340b5990 0x7fbc340b0990 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c000c00 tx=0x7fbc1c0046c0 comp rx=0 tx=0).stop 2024-04-04T06:12:48.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.488+0000 7fbc3b400640 1 -- 172.21.15.159:0/989536852 shutdown_connections 2024-04-04T06:12:48.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.488+0000 7fbc3b400640 1 --2- 172.21.15.159:0/989536852 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc1006c8f0 0x7fbc1006edb0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:48.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.488+0000 7fbc3b400640 1 --2- 172.21.15.159:0/989536852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc3415eab0 0x7fbc340b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:48.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.488+0000 7fbc3b400640 1 --2- 172.21.15.159:0/989536852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc340b5990 0x7fbc340b0990 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:48.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.488+0000 7fbc3b400640 1 --2- 172.21.15.159:0/989536852 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc34072a00 0x7fbc3414b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:48.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.488+0000 7fbc3b400640 1 -- 172.21.15.159:0/989536852 >> 172.21.15.159:0/989536852 conn(0x7fbc34071df0 msgr2=0x7fbc340b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:48.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.489+0000 7fbc3b400640 1 -- 172.21.15.159:0/989536852 shutdown_connections 2024-04-04T06:12:48.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.489+0000 7fbc3b400640 1 -- 172.21.15.159:0/989536852 wait complete. 2024-04-04T06:12:48.499 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-04T06:12:48.741 INFO:teuthology.orchestra.run.smithi159.stdout:{"message":"","return_code":0} 2024-04-04T06:12:48.741 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-04T06:12:48.742 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-04T06:12:48.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.943+0000 7fb133000640 1 Processor -- start 2024-04-04T06:12:48.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.944+0000 7fb133000640 1 -- start start 2024-04-04T06:12:48.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.945+0000 7fb133000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb12c0d0280 0x7fb12c0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:48.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.945+0000 7fb133000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb12c15ec80 con 0x7fb12c162160 2024-04-04T06:12:48.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.945+0000 7fb133000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb12c0d0ba0 con 0x7fb12c141c10 2024-04-04T06:12:48.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.945+0000 7fb133000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fb12c15ef20 con 0x7fb12c0d0280 2024-04-04T06:12:48.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.945+0000 7fb12be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb12c0d0280 0x7fb12c0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:48.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.946+0000 7fb12be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb12c0d0280 0x7fb12c0d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42732/0 (socket says 172.21.15.159:42732) 2024-04-04T06:12:48.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.946+0000 7fb12be00640 1 -- 172.21.15.159:0/1784733249 learned_addr learned my addr 172.21.15.159:0/1784733249 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:48.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.946+0000 7fb12be00640 1 -- 172.21.15.159:0/1784733249 >> v1:172.21.15.178:6789/0 conn(0x7fb12c141c10 legacy=0x7fb12c071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:48.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.946+0000 7fb12be00640 1 -- 172.21.15.159:0/1784733249 >> v1:172.21.15.159:6789/0 conn(0x7fb12c162160 legacy=0x7fb12c162610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:48.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.946+0000 7fb12be00640 1 -- 172.21.15.159:0/1784733249 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb12c15f200 con 0x7fb12c0d0280 2024-04-04T06:12:48.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.946+0000 7fb12b400640 1 -- 172.21.15.159:0/1784733249 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 93812864 0 0) 0x7fb12c15ec80 con 0x7fb12c162160 2024-04-04T06:12:48.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.947+0000 7fb12be00640 1 --2- 172.21.15.159:0/1784733249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb12c0d0280 0x7fb12c0d0660 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fb11c009870 tx=0x7fb11c02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e677c8ee6a75defb server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:48.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.948+0000 7fb12b400640 1 -- 172.21.15.159:0/1784733249 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c02f850 con 0x7fb12c0d0280 2024-04-04T06:12:48.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.948+0000 7fb12b400640 1 -- 172.21.15.159:0/1784733249 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb11c02f9f0 con 0x7fb12c0d0280 2024-04-04T06:12:48.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.948+0000 7fb12b400640 1 -- 172.21.15.159:0/1784733249 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c02fdc0 con 0x7fb12c0d0280 2024-04-04T06:12:48.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.949+0000 7fb133000640 1 -- 172.21.15.159:0/1784733249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb12c0d0280 msgr2=0x7fb12c0d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:48.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.949+0000 7fb133000640 1 --2- 172.21.15.159:0/1784733249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb12c0d0280 0x7fb12c0d0660 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fb11c009870 tx=0x7fb11c02ba30 comp rx=0 tx=0).stop 2024-04-04T06:12:48.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.949+0000 7fb133000640 1 -- 172.21.15.159:0/1784733249 shutdown_connections 2024-04-04T06:12:48.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.949+0000 7fb133000640 1 --2- 172.21.15.159:0/1784733249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb12c0d0280 0x7fb12c0d0660 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:48.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.949+0000 7fb133000640 1 -- 172.21.15.159:0/1784733249 >> 172.21.15.159:0/1784733249 conn(0x7fb12c06b750 msgr2=0x7fb12c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:48.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.949+0000 7fb133000640 1 -- 172.21.15.159:0/1784733249 shutdown_connections 2024-04-04T06:12:48.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.949+0000 7fb133000640 1 -- 172.21.15.159:0/1784733249 wait complete. 2024-04-04T06:12:48.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.951+0000 7fb133000640 1 Processor -- start 2024-04-04T06:12:48.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.951+0000 7fb133000640 1 -- start start 2024-04-04T06:12:48.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.952+0000 7fb133000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb12c0d0280 0x7fb12c156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:48.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.952+0000 7fb133000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb12c141c10 0x7fb12c156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:48.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.952+0000 7fb133000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb12c162160 0x7fb12c1f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:48.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.952+0000 7fb133000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c158e90 con 0x7fb12c141c10 2024-04-04T06:12:48.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.952+0000 7fb133000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c15f7c0 con 0x7fb12c162160 2024-04-04T06:12:48.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.952+0000 7fb133000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb12c145660 con 0x7fb12c0d0280 2024-04-04T06:12:48.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.952+0000 7fb123400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb12c141c10 0x7fb12c156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:48.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.952+0000 7fb123400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb12c141c10 0x7fb12c156e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42744/0 (socket says 172.21.15.159:42744) 2024-04-04T06:12:48.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.952+0000 7fb123400640 1 -- 172.21.15.159:0/2331766268 learned_addr learned my addr 172.21.15.159:0/2331766268 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:48.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.952+0000 7fb12be00640 1 --2- 172.21.15.159:0/2331766268 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb12c0d0280 0x7fb12c156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:48.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.952+0000 7fb131000640 1 --2- 172.21.15.159:0/2331766268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb12c162160 0x7fb12c1f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:48.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.953+0000 7fb123400640 1 -- 172.21.15.159:0/2331766268 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb12c0d0280 msgr2=0x7fb12c156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:48.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.953+0000 7fb123400640 1 --2- 172.21.15.159:0/2331766268 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb12c0d0280 0x7fb12c156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:48.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.953+0000 7fb123400640 1 -- 172.21.15.159:0/2331766268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb12c162160 msgr2=0x7fb12c1f4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:48.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.953+0000 7fb123400640 1 --2- 172.21.15.159:0/2331766268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb12c162160 0x7fb12c1f4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:48.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.953+0000 7fb123400640 1 -- 172.21.15.159:0/2331766268 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb12c1438f0 con 0x7fb12c141c10 2024-04-04T06:12:48.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.953+0000 7fb123400640 1 --2- 172.21.15.159:0/2331766268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb12c141c10 0x7fb12c156e70 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fb11400c400 tx=0x7fb11400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:48.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.953+0000 7fb128c00640 1 -- 172.21.15.159:0/2331766268 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb114017070 con 0x7fb12c141c10 2024-04-04T06:12:48.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.953+0000 7fb128c00640 1 -- 172.21.15.159:0/2331766268 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1140052d0 con 0x7fb12c141c10 2024-04-04T06:12:48.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.953+0000 7fb133000640 1 -- 172.21.15.159:0/2331766268 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb12c143bb0 con 0x7fb12c141c10 2024-04-04T06:12:48.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.953+0000 7fb128c00640 1 -- 172.21.15.159:0/2331766268 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb114003830 con 0x7fb12c141c10 2024-04-04T06:12:48.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.953+0000 7fb133000640 1 -- 172.21.15.159:0/2331766268 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb12c0d0880 con 0x7fb12c141c10 2024-04-04T06:12:48.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.955+0000 7fb133000640 1 -- 172.21.15.159:0/2331766268 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb12c1453c0 con 0x7fb12c141c10 2024-04-04T06:12:48.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.956+0000 7fb128c00640 1 -- 172.21.15.159:0/2331766268 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb11401a070 con 0x7fb12c141c10 2024-04-04T06:12:48.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.957+0000 7fb128c00640 1 --2- 172.21.15.159:0/2331766268 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb10806c920 0x7fb10806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:48.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.957+0000 7fb128c00640 1 -- 172.21.15.159:0/2331766268 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fb11408add0 con 0x7fb12c141c10 2024-04-04T06:12:48.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.957+0000 7fb12be00640 1 --2- 172.21.15.159:0/2331766268 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb10806c920 0x7fb10806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:48.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.958+0000 7fb12be00640 1 --2- 172.21.15.159:0/2331766268 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb10806c920 0x7fb10806ede0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb11c036000 tx=0x7fb11c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:48.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:48.966+0000 7fb128c00640 1 -- 172.21.15.159:0/2331766268 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb114059290 con 0x7fb12c141c10 2024-04-04T06:12:49.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.203+0000 7fb133000640 1 -- 172.21.15.159:0/2331766268 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb12c0685c0 con 0x7fb12c141c10 2024-04-04T06:12:49.197 INFO:tasks.ceph.mon.a.smithi159.stderr:2024-04-04T06:12:49.204+0000 7f1995c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-04T06:12:49.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.207+0000 7fb128c00640 1 -- 172.21.15.159:0/2331766268 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) ==== 161+0+0 (secure 0 0 0) 0x7fb11408c090 con 0x7fb12c141c10 2024-04-04T06:12:49.200 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-04T06:12:49.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.210+0000 7fb133000640 1 -- 172.21.15.159:0/2331766268 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb10806c920 msgr2=0x7fb10806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.210+0000 7fb133000640 1 --2- 172.21.15.159:0/2331766268 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb10806c920 0x7fb10806ede0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb11c036000 tx=0x7fb11c008030 comp rx=0 tx=0).stop 2024-04-04T06:12:49.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.210+0000 7fb133000640 1 -- 172.21.15.159:0/2331766268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb12c141c10 msgr2=0x7fb12c156e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.210+0000 7fb133000640 1 --2- 172.21.15.159:0/2331766268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb12c141c10 0x7fb12c156e70 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fb11400c400 tx=0x7fb11400c8c0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.210+0000 7fb133000640 1 -- 172.21.15.159:0/2331766268 shutdown_connections 2024-04-04T06:12:49.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.210+0000 7fb133000640 1 --2- 172.21.15.159:0/2331766268 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb10806c920 0x7fb10806ede0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.210+0000 7fb133000640 1 --2- 172.21.15.159:0/2331766268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb12c162160 0x7fb12c1f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.210+0000 7fb133000640 1 --2- 172.21.15.159:0/2331766268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb12c141c10 0x7fb12c156e70 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.210+0000 7fb133000640 1 --2- 172.21.15.159:0/2331766268 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb12c0d0280 0x7fb12c156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.210+0000 7fb133000640 1 -- 172.21.15.159:0/2331766268 >> 172.21.15.159:0/2331766268 conn(0x7fb12c06b750 msgr2=0x7fb12c0b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:49.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.210+0000 7fb133000640 1 -- 172.21.15.159:0/2331766268 shutdown_connections 2024-04-04T06:12:49.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.210+0000 7fb133000640 1 -- 172.21.15.159:0/2331766268 wait complete. 2024-04-04T06:12:49.221 INFO:tasks.ceph.mds.a.smithi159.stderr: -5017> 2024-04-04T06:12:42.366+0000 7ffb1c600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:12:49.223 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-04-04T06:12:49.302 INFO:tasks.ceph.mds.a.smithi159.stdout:starting mds.a at 2024-04-04T06:12:49.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.318+0000 7f5512140f40 1 Processor -- start 2024-04-04T06:12:49.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.318+0000 7f5512140f40 1 -- start start 2024-04-04T06:12:49.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.319+0000 7f5512140f40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d3769140 0x5651d380e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.319+0000 7f5512140f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651d3783e80 0x5651d380ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.319+0000 7f5512140f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651d3810070 0x5651d3764bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.319+0000 7f5512140f40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5651d36e02c0 con 0x5651d3769140 2024-04-04T06:12:49.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.319+0000 7f5512140f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5651d3810450 con 0x5651d3783e80 2024-04-04T06:12:49.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.319+0000 7f5512140f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5651d37385f0 con 0x5651d3810070 2024-04-04T06:12:49.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.319+0000 7f550f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d3769140 0x5651d380e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.319+0000 7f550f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d3769140 0x5651d380e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42776/0 (socket says 172.21.15.159:42776) 2024-04-04T06:12:49.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.319+0000 7f550f400640 1 -- 172.21.15.159:0/2401927476 learned_addr learned my addr 172.21.15.159:0/2401927476 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:49.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.319+0000 7f550f400640 1 -- 172.21.15.159:0/2401927476 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651d3810070 msgr2=0x5651d3764bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:49.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.319+0000 7f550f400640 1 --2- 172.21.15.159:0/2401927476 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651d3810070 0x5651d3764bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.319+0000 7f550f400640 1 -- 172.21.15.159:0/2401927476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651d3783e80 msgr2=0x5651d380ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:49.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.319+0000 7f550f400640 1 --2- 172.21.15.159:0/2401927476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651d3783e80 0x5651d380ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.319+0000 7f550f400640 1 -- 172.21.15.159:0/2401927476 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651d3738a80 con 0x5651d3769140 2024-04-04T06:12:49.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.320+0000 7f550f400640 1 --2- 172.21.15.159:0/2401927476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d3769140 0x5651d380e950 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f550000a840 tx=0x7f5500005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24dd58546c2f2d64 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:49.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.320+0000 7f550d600640 1 -- 172.21.15.159:0/2401927476 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550001a070 con 0x5651d3769140 2024-04-04T06:12:49.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.320+0000 7f550d600640 1 -- 172.21.15.159:0/2401927476 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5500008930 con 0x5651d3769140 2024-04-04T06:12:49.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.320+0000 7f550d600640 1 -- 172.21.15.159:0/2401927476 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5500008d20 con 0x5651d3769140 2024-04-04T06:12:49.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.320+0000 7f5512140f40 1 -- 172.21.15.159:0/2401927476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d3769140 msgr2=0x5651d380e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.320+0000 7f5512140f40 1 --2- 172.21.15.159:0/2401927476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d3769140 0x5651d380e950 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f550000a840 tx=0x7f5500005ac0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.320+0000 7f5512140f40 1 -- 172.21.15.159:0/2401927476 shutdown_connections 2024-04-04T06:12:49.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.320+0000 7f5512140f40 1 --2- 172.21.15.159:0/2401927476 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651d3810070 0x5651d3764bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.320+0000 7f5512140f40 1 --2- 172.21.15.159:0/2401927476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651d3783e80 0x5651d380ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.320+0000 7f5512140f40 1 --2- 172.21.15.159:0/2401927476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d3769140 0x5651d380e950 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.320+0000 7f5512140f40 1 -- 172.21.15.159:0/2401927476 >> 172.21.15.159:0/2401927476 conn(0x5651d3738210 msgr2=0x5651d373c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:49.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.320+0000 7f5512140f40 1 -- 172.21.15.159:0/2401927476 shutdown_connections 2024-04-04T06:12:49.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.321+0000 7f5512140f40 1 -- 172.21.15.159:0/2401927476 wait complete. 2024-04-04T06:12:49.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.321+0000 7f5512140f40 1 Processor -- start 2024-04-04T06:12:49.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.321+0000 7f5512140f40 1 -- start start 2024-04-04T06:12:49.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.322+0000 7f5512140f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651d3769140 0x5651d37f8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.322+0000 7f5512140f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651d3783e80 0x5651d37f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.322+0000 7f5512140f40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d3810070 0x5651d37fa270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.322+0000 7f5512140f40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5651d373d380 con 0x5651d3810070 2024-04-04T06:12:49.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.322+0000 7f5512140f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5651d373d0a0 con 0x5651d3769140 2024-04-04T06:12:49.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.322+0000 7f5512140f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5651d38106a0 con 0x5651d3783e80 2024-04-04T06:12:49.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.323+0000 7f550fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d3810070 0x5651d37fa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.323+0000 7f550fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d3810070 0x5651d37fa270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42780/0 (socket says 172.21.15.159:42780) 2024-04-04T06:12:49.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.323+0000 7f550fe00640 1 -- 172.21.15.159:0/2368343074 learned_addr learned my addr 172.21.15.159:0/2368343074 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:49.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.323+0000 7f550f400640 1 --2- 172.21.15.159:0/2368343074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651d3769140 0x5651d37f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.323+0000 7f550ea00640 1 --2- 172.21.15.159:0/2368343074 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651d3783e80 0x5651d37f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.323+0000 7f550fe00640 1 -- 172.21.15.159:0/2368343074 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651d3783e80 msgr2=0x5651d37f9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.323+0000 7f550fe00640 1 --2- 172.21.15.159:0/2368343074 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5651d3783e80 0x5651d37f9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.323+0000 7f550fe00640 1 -- 172.21.15.159:0/2368343074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651d3769140 msgr2=0x5651d37f8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.323+0000 7f550fe00640 1 --2- 172.21.15.159:0/2368343074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651d3769140 0x5651d37f8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.323+0000 7f550fe00640 1 -- 172.21.15.159:0/2368343074 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651d3809a10 con 0x5651d3810070 2024-04-04T06:12:49.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.323+0000 7f550f400640 1 --2- 172.21.15.159:0/2368343074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5651d3769140 0x5651d37f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:12:49.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.324+0000 7f550fe00640 1 --2- 172.21.15.159:0/2368343074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d3810070 0x5651d37fa270 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f5508006f70 tx=0x7f550800b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:49.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.324+0000 7f5507e00640 1 -- 172.21.15.159:0/2368343074 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508011620 con 0x5651d3810070 2024-04-04T06:12:49.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.324+0000 7f5507e00640 1 -- 172.21.15.159:0/2368343074 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55080117e0 con 0x5651d3810070 2024-04-04T06:12:49.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.324+0000 7f5507e00640 1 -- 172.21.15.159:0/2368343074 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508011bd0 con 0x5651d3810070 2024-04-04T06:12:49.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.324+0000 7f5512140f40 1 -- 172.21.15.159:0/2368343074 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5651d3809cf0 con 0x5651d3810070 2024-04-04T06:12:49.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.324+0000 7f5512140f40 1 -- 172.21.15.159:0/2368343074 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651d380a2b0 con 0x5651d3810070 2024-04-04T06:12:49.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.326+0000 7f5507e00640 1 -- 172.21.15.159:0/2368343074 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f550801a400 con 0x5651d3810070 2024-04-04T06:12:49.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.327+0000 7f5507e00640 1 --2- 172.21.15.159:0/2368343074 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f54f006b2d0 0x7f54f006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.327+0000 7f5507e00640 1 -- 172.21.15.159:0/2368343074 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f550808a6c0 con 0x5651d3810070 2024-04-04T06:12:49.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.327+0000 7f5512140f40 1 --2- 172.21.15.159:0/2368343074 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x5651d3808290 0x5651d3808e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.327+0000 7f5512140f40 1 -- 172.21.15.159:0/2368343074 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e86) -- 0x5651d38093b0 con 0x5651d3808290 2024-04-04T06:12:49.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.328+0000 7f550f400640 1 --2- 172.21.15.159:0/2368343074 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f54f006b2d0 0x7f54f006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.328+0000 7f550ea00640 1 --2- 172.21.15.159:0/2368343074 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x5651d3808290 0x5651d3808e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.328+0000 7f550ea00640 1 --2- 172.21.15.159:0/2368343074 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x5651d3808290 0x5651d3808e70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:49.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.329+0000 7f550f400640 1 --2- 172.21.15.159:0/2368343074 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f54f006b2d0 0x7f54f006d790 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f550000afd0 tx=0x7f5500012000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:49.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.333+0000 7f550ea00640 1 -- 172.21.15.159:0/2368343074 <== osd.0 v2:172.21.15.159:6800/455382518 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v86'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f54fc050550 con 0x5651d3808290 2024-04-04T06:12:49.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.333+0000 7f5512140f40 1 -- 172.21.15.159:0/2368343074 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x5651d3808290 msgr2=0x5651d3808e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.333+0000 7f5512140f40 1 --2- 172.21.15.159:0/2368343074 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x5651d3808290 0x5651d3808e70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.333+0000 7f5512140f40 1 -- 172.21.15.159:0/2368343074 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f54f006b2d0 msgr2=0x7f54f006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.333+0000 7f5512140f40 1 --2- 172.21.15.159:0/2368343074 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f54f006b2d0 0x7f54f006d790 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f550000afd0 tx=0x7f5500012000 comp rx=0 tx=0).stop 2024-04-04T06:12:49.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.333+0000 7f5512140f40 1 -- 172.21.15.159:0/2368343074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d3810070 msgr2=0x5651d37fa270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.333+0000 7f5512140f40 1 --2- 172.21.15.159:0/2368343074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d3810070 0x5651d37fa270 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f5508006f70 tx=0x7f550800b4b0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.334+0000 7f550fe00640 1 -- 172.21.15.159:0/2368343074 reap_dead start 2024-04-04T06:12:49.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.334+0000 7f5512140f40 1 -- 172.21.15.159:0/2368343074 shutdown_connections 2024-04-04T06:12:49.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.334+0000 7f5512140f40 1 -- 172.21.15.159:0/2368343074 >> 172.21.15.159:0/2368343074 conn(0x5651d3738210 msgr2=0x5651d38117f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:49.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.334+0000 7f5512140f40 1 -- 172.21.15.159:0/2368343074 shutdown_connections 2024-04-04T06:12:49.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.334+0000 7f5512140f40 1 -- 172.21.15.159:0/2368343074 wait complete. 2024-04-04T06:12:49.332 DEBUG:teuthology.orchestra.run.smithi159:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-04-04T06:12:49.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.422+0000 7f79e0e37280 1 -- start start 2024-04-04T06:12:49.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.422+0000 7f79e0e37280 1 Processor -- start 2024-04-04T06:12:49.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.423+0000 7f79e0e37280 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5218acc00 0x55c522670000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.423+0000 7f79e0e37280 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5218ad800 0x55c522670580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.423+0000 7f79e0e37280 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c5218adc00 0x55c522670b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.423+0000 7f79e0e37280 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c522610600 con 0x55c5218acc00 2024-04-04T06:12:49.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.423+0000 7f79e0e37280 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c522610480 con 0x55c5218ad800 2024-04-04T06:12:49.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.423+0000 7f79e0e37280 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c522610300 con 0x55c5218adc00 2024-04-04T06:12:49.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.423+0000 7f79de800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5218acc00 0x55c522670000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.424+0000 7f79de800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5218acc00 0x55c522670000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42798/0 (socket says 172.21.15.159:42798) 2024-04-04T06:12:49.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.424+0000 7f79de800640 1 -- 172.21.15.159:0/2197040135 learned_addr learned my addr 172.21.15.159:0/2197040135 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:49.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.424+0000 7f79de800640 1 -- 172.21.15.159:0/2197040135 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c5218adc00 msgr2=0x55c522670b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:49.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.424+0000 7f79de800640 1 --2- 172.21.15.159:0/2197040135 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c5218adc00 0x55c522670b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.424+0000 7f79de800640 1 -- 172.21.15.159:0/2197040135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5218ad800 msgr2=0x55c522670580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:49.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.424+0000 7f79dde00640 1 --2- 172.21.15.159:0/2197040135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5218ad800 0x55c522670580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.424+0000 7f79de800640 1 --2- 172.21.15.159:0/2197040135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5218ad800 0x55c522670580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.424+0000 7f79de800640 1 -- 172.21.15.159:0/2197040135 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5218e0780 con 0x55c5218acc00 2024-04-04T06:12:49.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.424+0000 7f79de800640 1 --2- 172.21.15.159:0/2197040135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5218acc00 0x55c522670000 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x55c5226c0a20 tx=0x55c52263ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e5def9ad8d6ad0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:49.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.424+0000 7f79db600640 1 -- 172.21.15.159:0/2197040135 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c5218d29c0 con 0x55c5218acc00 2024-04-04T06:12:49.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.424+0000 7f79db600640 1 -- 172.21.15.159:0/2197040135 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55c5218d8e00 con 0x55c5218acc00 2024-04-04T06:12:49.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.425+0000 7f79db600640 1 -- 172.21.15.159:0/2197040135 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c5218d2680 con 0x55c5218acc00 2024-04-04T06:12:49.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.425+0000 7f79e0e37280 1 -- 172.21.15.159:0/2197040135 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5218e14a0 con 0x55c5218acc00 2024-04-04T06:12:49.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.425+0000 7f79e0e37280 1 -- 172.21.15.159:0/2197040135 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55c5218e0000 con 0x55c5218acc00 2024-04-04T06:12:49.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.425+0000 7f79e0e37280 4 waiting for MDS map... 2024-04-04T06:12:49.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.425+0000 7f79db600640 1 -- 172.21.15.159:0/2197040135 <== mon.0 v2:172.21.15.159:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x55c522770200 con 0x55c5218acc00 2024-04-04T06:12:49.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.425+0000 7f79db600640 1 -- 172.21.15.159:0/2197040135 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 63) ==== 2885+0+0 (secure 0 0 0) 0x55c521907180 con 0x55c5218acc00 2024-04-04T06:12:49.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.425+0000 7f79e0e37280 4 Got MDS map 63 2024-04-04T06:12:49.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.425+0000 7f79e0e37280 10 main: JournalTool::main 2024-04-04T06:12:49.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.425+0000 7f79e0e37280 4 main: JournalTool: connecting to RADOS... 2024-04-04T06:12:49.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.426+0000 7f79e0e37280 1 Processor -- start 2024-04-04T06:12:49.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.426+0000 7f79e0e37280 1 -- start start 2024-04-04T06:12:49.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.427+0000 7f79e0e37280 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c5227c8000 0x55c522671080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.427+0000 7f79e0e37280 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5227c8400 0x55c522671600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.427+0000 7f79e0e37280 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5227c8800 0x55c522671b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.427+0000 7f79e0e37280 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c522610180 con 0x55c5227c8800 2024-04-04T06:12:49.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.427+0000 7f79e0e37280 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c52190e480 con 0x55c5227c8400 2024-04-04T06:12:49.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.427+0000 7f79e0e37280 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c52190e900 con 0x55c5227c8000 2024-04-04T06:12:49.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.427+0000 7f79dde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5227c8800 0x55c522671b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.427+0000 7f79dde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5227c8800 0x55c522671b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42808/0 (socket says 172.21.15.159:42808) 2024-04-04T06:12:49.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.427+0000 7f79dde00640 1 -- 172.21.15.159:0/1129664611 learned_addr learned my addr 172.21.15.159:0/1129664611 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:49.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.427+0000 7f79df200640 1 --2- 172.21.15.159:0/1129664611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c5227c8000 0x55c522671080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.427+0000 7f79de800640 1 --2- 172.21.15.159:0/1129664611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5227c8400 0x55c522671600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.428+0000 7f79dde00640 1 -- 172.21.15.159:0/1129664611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c5227c8000 msgr2=0x55c522671080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:49.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.428+0000 7f79dde00640 1 --2- 172.21.15.159:0/1129664611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c5227c8000 0x55c522671080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.428+0000 7f79dde00640 1 -- 172.21.15.159:0/1129664611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5227c8400 msgr2=0x55c522671600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:49.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.428+0000 7f79dde00640 1 --2- 172.21.15.159:0/1129664611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5227c8400 0x55c522671600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.428+0000 7f79dde00640 1 -- 172.21.15.159:0/1129664611 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5218e03c0 con 0x55c5227c8800 2024-04-04T06:12:49.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.428+0000 7f79dde00640 1 --2- 172.21.15.159:0/1129664611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5227c8800 0x55c522671b80 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x55c5227d00f0 tx=0x55c52277cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ccee709f40c7ac server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:49.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.428+0000 7f79d8e00640 1 -- 172.21.15.159:0/1129664611 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c5218d29c0 con 0x55c5227c8800 2024-04-04T06:12:49.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.428+0000 7f79d8e00640 1 -- 172.21.15.159:0/1129664611 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55c5218d8a80 con 0x55c5227c8800 2024-04-04T06:12:49.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.428+0000 7f79d8e00640 1 -- 172.21.15.159:0/1129664611 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c5218d2680 con 0x55c5227c8800 2024-04-04T06:12:49.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.428+0000 7f79e0e37280 1 -- 172.21.15.159:0/1129664611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5227c8800 msgr2=0x55c522671b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.428+0000 7f79e0e37280 1 --2- 172.21.15.159:0/1129664611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5227c8800 0x55c522671b80 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x55c5227d00f0 tx=0x55c52277cd20 comp rx=0 tx=0).stop 2024-04-04T06:12:49.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.429+0000 7f79e0e37280 1 -- 172.21.15.159:0/1129664611 shutdown_connections 2024-04-04T06:12:49.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.429+0000 7f79e0e37280 1 --2- 172.21.15.159:0/1129664611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5227c8800 0x55c522671b80 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.429+0000 7f79e0e37280 1 --2- 172.21.15.159:0/1129664611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5227c8400 0x55c522671600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.429+0000 7f79e0e37280 1 --2- 172.21.15.159:0/1129664611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c5227c8000 0x55c522671080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.429+0000 7f79e0e37280 1 -- 172.21.15.159:0/1129664611 >> 172.21.15.159:0/1129664611 conn(0x55c522681c00 msgr2=0x55c522628800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:49.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.429+0000 7f79e0e37280 1 -- 172.21.15.159:0/1129664611 shutdown_connections 2024-04-04T06:12:49.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.429+0000 7f79e0e37280 1 -- 172.21.15.159:0/1129664611 wait complete. 2024-04-04T06:12:49.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.429+0000 7f79e0e37280 1 Processor -- start 2024-04-04T06:12:49.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.429+0000 7f79e0e37280 1 -- start start 2024-04-04T06:12:49.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.430+0000 7f79e0e37280 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5227c8000 0x55c522671080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.430+0000 7f79e0e37280 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5227c8400 0x55c522671600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.430+0000 7f79e0e37280 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c5227c8800 0x55c522671b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.430+0000 7f79e0e37280 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c522611380 con 0x55c5227c8000 2024-04-04T06:12:49.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.430+0000 7f79e0e37280 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c522610180 con 0x55c5227c8400 2024-04-04T06:12:49.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.430+0000 7f79e0e37280 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c522611500 con 0x55c5227c8800 2024-04-04T06:12:49.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.430+0000 7f79df200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5227c8000 0x55c522671080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.430+0000 7f79df200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5227c8000 0x55c522671080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42822/0 (socket says 172.21.15.159:42822) 2024-04-04T06:12:49.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.430+0000 7f79df200640 1 -- 172.21.15.159:0/215104664 learned_addr learned my addr 172.21.15.159:0/215104664 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:49.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.431+0000 7f79de800640 1 --2- 172.21.15.159:0/215104664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5227c8400 0x55c522671600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.431+0000 7f79df200640 1 -- 172.21.15.159:0/215104664 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c5227c8800 msgr2=0x55c522671b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.431+0000 7f79dde00640 1 --2- 172.21.15.159:0/215104664 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c5227c8800 0x55c522671b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.431+0000 7f79df200640 1 --2- 172.21.15.159:0/215104664 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c5227c8800 0x55c522671b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.431+0000 7f79df200640 1 -- 172.21.15.159:0/215104664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5227c8400 msgr2=0x55c522671600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.431+0000 7f79df200640 1 --2- 172.21.15.159:0/215104664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5227c8400 0x55c522671600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.431+0000 7f79df200640 1 -- 172.21.15.159:0/215104664 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5218e0960 con 0x55c5227c8000 2024-04-04T06:12:49.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.431+0000 7f79de800640 1 --2- 172.21.15.159:0/215104664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5227c8400 0x55c522671600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:49.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.431+0000 7f79df200640 1 --2- 172.21.15.159:0/215104664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5227c8000 0x55c522671080 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x55c5227d13b0 tx=0x55c52277cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:49.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.431+0000 7f79d7000640 1 -- 172.21.15.159:0/215104664 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c5218d29c0 con 0x55c5227c8000 2024-04-04T06:12:49.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.431+0000 7f79d7000640 1 -- 172.21.15.159:0/215104664 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55c5218d8c40 con 0x55c5227c8000 2024-04-04T06:12:49.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.431+0000 7f79d7000640 1 -- 172.21.15.159:0/215104664 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c5218d2680 con 0x55c5227c8000 2024-04-04T06:12:49.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.431+0000 7f79e0e37280 1 -- 172.21.15.159:0/215104664 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c5218e1680 con 0x55c5227c8000 2024-04-04T06:12:49.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.432+0000 7f79e0e37280 1 -- 172.21.15.159:0/215104664 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5218e1860 con 0x55c5227c8000 2024-04-04T06:12:49.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.432+0000 7f79e0e37280 4 main: JournalTool: resolving pool 12 2024-04-04T06:12:49.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.433+0000 7f79d7000640 1 -- 172.21.15.159:0/215104664 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55c5227d6c00 con 0x55c5227c8000 2024-04-04T06:12:49.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.434+0000 7f79d7000640 1 --2- 172.21.15.159:0/215104664 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x55c5227d8c00 0x55c522672100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.434+0000 7f79d7000640 1 -- 172.21.15.159:0/215104664 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x55c5227cf800 con 0x55c5227c8000 2024-04-04T06:12:49.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.434+0000 7f79e0e37280 4 main: JournalTool: creating IoCtx.. 2024-04-04T06:12:49.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.434+0000 7f79e0e37280 4 main: Executing for rank 0 2024-04-04T06:12:49.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.434+0000 7f79de800640 1 --2- 172.21.15.159:0/215104664 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x55c5227d8c00 0x55c522672100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.435+0000 7f79e0e37280 1 -- start start 2024-04-04T06:12:49.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.435+0000 7f79e0e37280 1 Processor -- start 2024-04-04T06:12:49.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.435+0000 7f79de800640 1 --2- 172.21.15.159:0/215104664 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x55c5227d8c00 0x55c522672100 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x55c5226c17a0 tx=0x55c52263e8a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:49.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.436+0000 7f79e0e37280 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5227c9000 0x55c522672680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.436+0000 7f79e0e37280 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c521c64000 0x55c522672c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.436+0000 7f79e0e37280 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c521c64400 0x55c522673180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.436+0000 7f79e0e37280 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c522611800 con 0x55c521c64400 2024-04-04T06:12:49.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.436+0000 7f79e0e37280 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c522611980 con 0x55c5227c9000 2024-04-04T06:12:49.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.436+0000 7f79e0e37280 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c52190e900 con 0x55c521c64000 2024-04-04T06:12:49.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.436+0000 7f79dde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c521c64400 0x55c522673180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.436+0000 7f79dde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c521c64400 0x55c522673180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42836/0 (socket says 172.21.15.159:42836) 2024-04-04T06:12:49.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.436+0000 7f79dde00640 1 -- 172.21.15.159:0/840355757 learned_addr learned my addr 172.21.15.159:0/840355757 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:49.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.437+0000 7f79df200640 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5227c9000 0x55c522672680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.437+0000 7f79de800640 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c521c64000 0x55c522672c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.437+0000 7f79dde00640 1 -- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c521c64000 msgr2=0x55c522672c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:49.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.437+0000 7f79dde00640 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c521c64000 0x55c522672c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.437+0000 7f79dde00640 1 -- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5227c9000 msgr2=0x55c522672680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:49.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.437+0000 7f79dde00640 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5227c9000 0x55c522672680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.437+0000 7f79dde00640 1 -- 172.21.15.159:0/840355757 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5218e1a40 con 0x55c521c64400 2024-04-04T06:12:49.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.437+0000 7f79de800640 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c521c64000 0x55c522672c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:12:49.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.437+0000 7f79dde00640 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c521c64400 0x55c522673180 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x55c521bb9680 tx=0x55c52277cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f81c15c617698fc server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:49.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.437+0000 7f79d3400640 1 -- 172.21.15.159:0/840355757 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c5218d24e0 con 0x55c521c64400 2024-04-04T06:12:49.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.438+0000 7f79d3400640 1 -- 172.21.15.159:0/840355757 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55c5218d81c0 con 0x55c521c64400 2024-04-04T06:12:49.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.438+0000 7f79d3400640 1 -- 172.21.15.159:0/840355757 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c521b91040 con 0x55c521c64400 2024-04-04T06:12:49.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.438+0000 7f79e0e37280 1 -- 172.21.15.159:0/840355757 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5218e03c0 con 0x55c521c64400 2024-04-04T06:12:49.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.438+0000 7f79e0e37280 1 -- 172.21.15.159:0/840355757 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55c5218e1a40 con 0x55c521c64400 2024-04-04T06:12:49.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.438+0000 7f79e0e37280 4 waiting for MDS map... 2024-04-04T06:12:49.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.438+0000 7f79d3400640 1 -- 172.21.15.159:0/840355757 <== mon.0 v2:172.21.15.159:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x55c521c6a000 con 0x55c521c64400 2024-04-04T06:12:49.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.438+0000 7f79d3400640 1 -- 172.21.15.159:0/840355757 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 63) ==== 2885+0+0 (secure 0 0 0) 0x55c521907500 con 0x55c521c64400 2024-04-04T06:12:49.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.438+0000 7f79e0e37280 4 Got MDS map 63 2024-04-04T06:12:49.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.438+0000 7f79e0e37280 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x55c521c64800 0x55c522673700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.438+0000 7f79e0e37280 1 -- 172.21.15.159:0/840355757 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55c521c64c00 con 0x55c521c64800 2024-04-04T06:12:49.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.439+0000 7f79df200640 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x55c521c64800 0x55c522673700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.439+0000 7f79df200640 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x55c521c64800 0x55c522673700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=61bdf040f2da20a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:49.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.446+0000 7f79df200640 1 -- 172.21.15.159:0/840355757 <== osd.6 v2:172.21.15.178:6826/198949157 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x55c521988000 con 0x55c521c64800 2024-04-04T06:12:49.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.446+0000 7f79e0e37280 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x55c521c65000 0x55c521c8e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.446+0000 7f79e0e37280 1 -- 172.21.15.159:0/840355757 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:2 12.14 12:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55c521c65400 con 0x55c521c65000 2024-04-04T06:12:49.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.447+0000 7f79de800640 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x55c521c65000 0x55c521c8e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.448+0000 7f79de800640 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x55c521c65000 0x55c521c8e000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=353e740c5970ab07 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:49.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.449+0000 7f79de800640 1 -- 172.21.15.159:0/840355757 <== osd.5 v2:172.21.15.178:6802/63746765 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x55c521988500 con 0x55c521c65000 2024-04-04T06:12:49.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.449+0000 7f79d1600640 1 -- 172.21.15.159:0/840355757 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:3 12.14 12:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e86) -- 0x55c521c80400 con 0x55c521c65000 2024-04-04T06:12:49.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.449+0000 7f79d1600640 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x55c521c80800 0x55c521c8fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.449+0000 7f79d1600640 1 -- 172.21.15.159:0/840355757 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:4 12.1b 12:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e86) -- 0x55c521c80c00 con 0x55c521c80800 2024-04-04T06:12:49.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.450+0000 7f79df200640 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x55c521c80800 0x55c521c8fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.450+0000 7f79de800640 1 -- 172.21.15.159:0/840355757 <== osd.5 v2:172.21.15.178:6802/63746765 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x55c521988500 con 0x55c521c65000 2024-04-04T06:12:49.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.451+0000 7f79df200640 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x55c521c80800 0x55c521c8fb80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=a12d6283508a3562 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:49.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.451+0000 7f79df200640 1 -- 172.21.15.159:0/840355757 <== osd.4 v2:172.21.15.178:6812/4035991429 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55c521988000 con 0x55c521c80800 2024-04-04T06:12:49.445 INFO:teuthology.orchestra.run.smithi159.stdout:old journal was 4212746~2605 2024-04-04T06:12:49.445 INFO:teuthology.orchestra.run.smithi159.stdout:new journal start will be 8388608 (4173257 bytes past old end) 2024-04-04T06:12:49.445 INFO:teuthology.orchestra.run.smithi159.stdout:writing journal head 2024-04-04T06:12:49.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.451+0000 7f79e0e37280 1 -- 172.21.15.159:0/840355757 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:5 12.14 12:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x55c521c81000 con 0x55c521c65000 2024-04-04T06:12:49.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.455+0000 7f79de800640 1 -- 172.21.15.159:0/840355757 <== osd.5 v2:172.21.15.178:6802/63746765 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v86'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55c521988500 con 0x55c521c65000 2024-04-04T06:12:49.448 INFO:teuthology.orchestra.run.smithi159.stdout:writing EResetJournal entry 2024-04-04T06:12:49.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.455+0000 7f79e0e37280 1 -- 172.21.15.159:0/840355757 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:6 12.1b 12:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x55c521c81400 con 0x55c521c80800 2024-04-04T06:12:49.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.455+0000 7f79e0e37280 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55c521c81800 0x55c521c8e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.455+0000 7f79e0e37280 1 -- 172.21.15.159:0/840355757 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:7 12.b 12:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x55c521c81c00 con 0x55c521c81800 2024-04-04T06:12:49.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.455+0000 7f79e0e37280 1 -- 172.21.15.159:0/840355757 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:8 12.1f 12:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x55c521c9a000 con 0x55c521c64800 2024-04-04T06:12:49.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.455+0000 7f79e0e37280 1 -- 172.21.15.159:0/840355757 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:9 12.1a 12:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x55c521c9a400 con 0x55c521c80800 2024-04-04T06:12:49.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.455+0000 7f79e0e37280 1 -- 172.21.15.159:0/840355757 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:10 12.1e 12:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x55c521c9a800 con 0x55c521c80800 2024-04-04T06:12:49.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.455+0000 7f79e0e37280 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x55c521c9ac00 0x55c521c8eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.455+0000 7f79e0e37280 1 -- 172.21.15.159:0/840355757 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:11 12.1d 12:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x55c521c9b000 con 0x55c521c9ac00 2024-04-04T06:12:49.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.456+0000 7f79de800640 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55c521c81800 0x55c521c8e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.456+0000 7f79dde00640 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x55c521c9ac00 0x55c521c8eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.456+0000 7f79de800640 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55c521c81800 0x55c521c8e580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=c5cb2fa5a620d1b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:49.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.457+0000 7f79dde00640 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x55c521c9ac00 0x55c521c8eb00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=b0f47edf439bcdcf server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:49.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.459+0000 7f79df200640 1 -- 172.21.15.159:0/840355757 <== osd.4 v2:172.21.15.178:6812/4035991429 2 ==== osd_op_reply(6 200.00000002 [delete] v86'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55c521988000 con 0x55c521c80800 2024-04-04T06:12:49.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.459+0000 7f79dde00640 1 -- 172.21.15.159:0/840355757 <== osd.0 v2:172.21.15.159:6800/455382518 1 ==== osd_op_reply(11 200.00000007 [delete] v86'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55c521988780 con 0x55c521c9ac00 2024-04-04T06:12:49.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.459+0000 7f79de800640 1 -- 172.21.15.159:0/840355757 <== osd.3 v2:172.21.15.159:6820/2070760205 1 ==== osd_op_reply(7 200.00000003 [delete] v86'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55c521988500 con 0x55c521c81800 2024-04-04T06:12:49.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.459+0000 7f79d1600640 1 -- 172.21.15.159:0/840355757 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:12 12.1b 12:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x55c521c65400 con 0x55c521c80800 2024-04-04T06:12:49.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.459+0000 7f79df200640 1 -- 172.21.15.159:0/840355757 <== osd.6 v2:172.21.15.178:6826/198949157 2 ==== osd_op_reply(8 200.00000004 [delete] v86'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55c521988000 con 0x55c521c64800 2024-04-04T06:12:49.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.459+0000 7f79df200640 1 -- 172.21.15.159:0/840355757 <== osd.4 v2:172.21.15.178:6812/4035991429 3 ==== osd_op_reply(10 200.00000006 [delete] v86'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55c521988000 con 0x55c521c80800 2024-04-04T06:12:49.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.459+0000 7f79df200640 1 -- 172.21.15.159:0/840355757 <== osd.4 v2:172.21.15.178:6812/4035991429 4 ==== osd_op_reply(9 200.00000005 [delete] v86'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55c521988000 con 0x55c521c80800 2024-04-04T06:12:49.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.462+0000 7f79df200640 1 -- 172.21.15.159:0/840355757 <== osd.4 v2:172.21.15.178:6812/4035991429 5 ==== osd_op_reply(12 200.00000002 [write 0~48] v86'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55c521988000 con 0x55c521c80800 2024-04-04T06:12:49.455 INFO:teuthology.orchestra.run.smithi159.stdout:done 2024-04-04T06:12:49.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.462+0000 7f79e0e37280 1 -- 172.21.15.159:0/840355757 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x55c521c9ac00 msgr2=0x55c521c8eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.462+0000 7f79e0e37280 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x55c521c9ac00 0x55c521c8eb00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.462+0000 7f79e0e37280 1 -- 172.21.15.159:0/840355757 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55c521c81800 msgr2=0x55c521c8e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.462+0000 7f79e0e37280 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55c521c81800 0x55c521c8e580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.462+0000 7f79e0e37280 1 -- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x55c521c80800 msgr2=0x55c521c8fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79e0e37280 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x55c521c80800 0x55c521c8fb80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79e0e37280 1 -- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x55c521c65000 msgr2=0x55c521c8e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79e0e37280 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x55c521c65000 0x55c521c8e000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79e0e37280 1 -- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x55c521c64800 msgr2=0x55c522673700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79e0e37280 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x55c521c64800 0x55c522673700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79dde00640 1 -- 172.21.15.159:0/840355757 reap_dead start 2024-04-04T06:12:49.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79e0e37280 1 -- 172.21.15.159:0/840355757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c521c64400 msgr2=0x55c522673180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79e0e37280 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c521c64400 0x55c522673180 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x55c521bb9680 tx=0x55c52277cc00 comp rx=0 tx=0).stop 2024-04-04T06:12:49.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79e0e37280 1 -- 172.21.15.159:0/840355757 shutdown_connections 2024-04-04T06:12:49.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79e0e37280 1 --2- 172.21.15.159:0/840355757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c521c64400 0x55c522673180 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79e0e37280 1 -- 172.21.15.159:0/840355757 >> 172.21.15.159:0/840355757 conn(0x55c5227c8c00 msgr2=0x55c522628000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:49.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79e0e37280 1 -- 172.21.15.159:0/840355757 shutdown_connections 2024-04-04T06:12:49.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79e0e37280 1 -- 172.21.15.159:0/840355757 wait complete. 2024-04-04T06:12:49.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79e0e37280 1 -- 172.21.15.159:0/215104664 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x55c5227d8c00 msgr2=0x55c522672100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79e0e37280 1 --2- 172.21.15.159:0/215104664 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x55c5227d8c00 0x55c522672100 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x55c5226c17a0 tx=0x55c52263e8a0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79e0e37280 1 -- 172.21.15.159:0/215104664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5227c8000 msgr2=0x55c522671080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79e0e37280 1 --2- 172.21.15.159:0/215104664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5227c8000 0x55c522671080 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x55c5227d13b0 tx=0x55c52277cf00 comp rx=0 tx=0).stop 2024-04-04T06:12:49.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79e0e37280 1 -- 172.21.15.159:0/215104664 shutdown_connections 2024-04-04T06:12:49.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79e0e37280 1 --2- 172.21.15.159:0/215104664 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c5227c8800 0x55c522671b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79e0e37280 1 --2- 172.21.15.159:0/215104664 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x55c5227d8c00 0x55c522672100 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79e0e37280 1 --2- 172.21.15.159:0/215104664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5227c8400 0x55c522671600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.463+0000 7f79e0e37280 1 --2- 172.21.15.159:0/215104664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5227c8000 0x55c522671080 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.464+0000 7f79e0e37280 1 -- 172.21.15.159:0/215104664 >> 172.21.15.159:0/215104664 conn(0x55c522681c00 msgr2=0x55c522628800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:49.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.464+0000 7f79e0e37280 1 -- 172.21.15.159:0/215104664 shutdown_connections 2024-04-04T06:12:49.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.464+0000 7f79e0e37280 1 -- 172.21.15.159:0/215104664 wait complete. 2024-04-04T06:12:49.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.464+0000 7f79e0e37280 1 -- 172.21.15.159:0/2197040135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5218acc00 msgr2=0x55c522670000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.464+0000 7f79e0e37280 1 --2- 172.21.15.159:0/2197040135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5218acc00 0x55c522670000 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x55c5226c0a20 tx=0x55c52263ea20 comp rx=0 tx=0).stop 2024-04-04T06:12:49.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.464+0000 7f79e0e37280 1 -- 172.21.15.159:0/2197040135 shutdown_connections 2024-04-04T06:12:49.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.464+0000 7f79e0e37280 1 --2- 172.21.15.159:0/2197040135 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c5218adc00 0x55c522670b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.464+0000 7f79e0e37280 1 --2- 172.21.15.159:0/2197040135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c5218ad800 0x55c522670580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.464+0000 7f79e0e37280 1 --2- 172.21.15.159:0/2197040135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5218acc00 0x55c522670000 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.464+0000 7f79e0e37280 1 -- 172.21.15.159:0/2197040135 >> 172.21.15.159:0/2197040135 conn(0x55c5218ad000 msgr2=0x55c522629000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:49.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.464+0000 7f79e0e37280 1 -- 172.21.15.159:0/2197040135 shutdown_connections 2024-04-04T06:12:49.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.464+0000 7f79e0e37280 1 -- 172.21.15.159:0/2197040135 wait complete. 2024-04-04T06:12:49.462 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.131028, result: old journal was 4212746~2605 new journal start will be 8388608 (4173257 bytes past old end) writing journal head writing EResetJournal entry done 2024-04-04T06:12:49.463 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-04T06:12:49.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.673+0000 7f1069e00640 1 Processor -- start 2024-04-04T06:12:49.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.673+0000 7f1069e00640 1 -- start start 2024-04-04T06:12:49.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.675+0000 7f1069e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f10640b4890 0x7f1064072570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.675+0000 7f1069e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f106406d620 0x7f1064072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.675+0000 7f1069e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10640b4c70 con 0x7f106415f6a0 2024-04-04T06:12:49.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.675+0000 7f1069e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f10640b3eb0 con 0x7f10640b4890 2024-04-04T06:12:49.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.675+0000 7f1069e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f106415fb30 con 0x7f106406d620 2024-04-04T06:12:49.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.675+0000 7f1068a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f10640b4890 0x7f1064072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.675+0000 7f1068a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f10640b4890 0x7f1064072570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42838/0 (socket says 172.21.15.159:42838) 2024-04-04T06:12:49.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.675+0000 7f1068a00640 1 -- 172.21.15.159:0/2331875600 learned_addr learned my addr 172.21.15.159:0/2331875600 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:49.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.675+0000 7f1068a00640 1 -- 172.21.15.159:0/2331875600 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f106406d620 msgr2=0x7f1064072ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:49.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.675+0000 7f1068a00640 1 --2- 172.21.15.159:0/2331875600 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f106406d620 0x7f1064072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.675+0000 7f1068a00640 1 -- 172.21.15.159:0/2331875600 >> v1:172.21.15.178:6789/0 conn(0x7f106415f6a0 legacy=0x7f10640713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:49.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.675+0000 7f1068a00640 1 -- 172.21.15.159:0/2331875600 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106415b7b0 con 0x7f10640b4890 2024-04-04T06:12:49.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.676+0000 7f1068a00640 1 --2- 172.21.15.159:0/2331875600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10640b4890 0x7f1064072570 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f1054009870 tx=0x7f105402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d4027cf8927b60 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:49.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.677+0000 7f1063e00640 1 -- 172.21.15.159:0/2331875600 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1054035020 con 0x7f10640b4890 2024-04-04T06:12:49.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.677+0000 7f1063e00640 1 -- 172.21.15.159:0/2331875600 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1054031640 con 0x7f10640b4890 2024-04-04T06:12:49.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.677+0000 7f1063e00640 1 -- 172.21.15.159:0/2331875600 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105402d040 con 0x7f10640b4890 2024-04-04T06:12:49.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.677+0000 7f1069e00640 1 -- 172.21.15.159:0/2331875600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10640b4890 msgr2=0x7f1064072570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.677+0000 7f1069e00640 1 --2- 172.21.15.159:0/2331875600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10640b4890 0x7f1064072570 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f1054009870 tx=0x7f105402b8f0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.678+0000 7f1069e00640 1 -- 172.21.15.159:0/2331875600 shutdown_connections 2024-04-04T06:12:49.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.678+0000 7f1069e00640 1 --2- 172.21.15.159:0/2331875600 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f106406d620 0x7f1064072ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.678+0000 7f1069e00640 1 --2- 172.21.15.159:0/2331875600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f10640b4890 0x7f1064072570 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.678+0000 7f1069e00640 1 -- 172.21.15.159:0/2331875600 >> 172.21.15.159:0/2331875600 conn(0x7f10640b1ba0 msgr2=0x7f10640b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:49.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.678+0000 7f1069e00640 1 -- 172.21.15.159:0/2331875600 shutdown_connections 2024-04-04T06:12:49.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.678+0000 7f1069e00640 1 -- 172.21.15.159:0/2331875600 wait complete. 2024-04-04T06:12:49.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.680+0000 7f1069e00640 1 Processor -- start 2024-04-04T06:12:49.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.680+0000 7f1069e00640 1 -- start start 2024-04-04T06:12:49.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.681+0000 7f1069e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f106406d620 0x7f10641f7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.681+0000 7f1069e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10640b4890 0x7f1064153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.681+0000 7f1069e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10640aefa0 0x7f1064157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.681+0000 7f1069e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f106415c050 con 0x7f106406d620 2024-04-04T06:12:49.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.681+0000 7f1069e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f106415bd70 con 0x7f10640aefa0 2024-04-04T06:12:49.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.681+0000 7f1069e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f106415c570 con 0x7f10640b4890 2024-04-04T06:12:49.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.681+0000 7f1068a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f106406d620 0x7f10641f7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.681+0000 7f1068a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f106406d620 0x7f10641f7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42850/0 (socket says 172.21.15.159:42850) 2024-04-04T06:12:49.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.681+0000 7f1068a00640 1 -- 172.21.15.159:0/3843824927 learned_addr learned my addr 172.21.15.159:0/3843824927 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:49.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.681+0000 7f105be00640 1 --2- 172.21.15.159:0/3843824927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10640b4890 0x7f1064153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.681+0000 7f1068a00640 1 -- 172.21.15.159:0/3843824927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10640b4890 msgr2=0x7f1064153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.682+0000 7f1068a00640 1 --2- 172.21.15.159:0/3843824927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10640b4890 0x7f1064153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.682+0000 7f1068a00640 1 -- 172.21.15.159:0/3843824927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10640aefa0 msgr2=0x7f1064157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:49.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.682+0000 7f1068a00640 1 --2- 172.21.15.159:0/3843824927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10640aefa0 0x7f1064157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:49.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.682+0000 7f1068a00640 1 -- 172.21.15.159:0/3843824927 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1064153cb0 con 0x7f106406d620 2024-04-04T06:12:49.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.682+0000 7f1068a00640 1 --2- 172.21.15.159:0/3843824927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f106406d620 0x7f10641f7ed0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f105402b930 tx=0x7f10540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:49.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.682+0000 7f1061600640 1 -- 172.21.15.159:0/3843824927 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105403ac60 con 0x7f106406d620 2024-04-04T06:12:49.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.682+0000 7f1061600640 1 -- 172.21.15.159:0/3843824927 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f105403ae20 con 0x7f106406d620 2024-04-04T06:12:49.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.682+0000 7f1061600640 1 -- 172.21.15.159:0/3843824927 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1054044630 con 0x7f106406d620 2024-04-04T06:12:49.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.682+0000 7f1069e00640 1 -- 172.21.15.159:0/3843824927 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10641586f0 con 0x7f106406d620 2024-04-04T06:12:49.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.682+0000 7f1069e00640 1 -- 172.21.15.159:0/3843824927 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10641f76d0 con 0x7f106406d620 2024-04-04T06:12:49.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.684+0000 7f1069e00640 1 -- 172.21.15.159:0/3843824927 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10640cc290 con 0x7f106406d620 2024-04-04T06:12:49.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.684+0000 7f1061600640 1 -- 172.21.15.159:0/3843824927 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f105404b060 con 0x7f106406d620 2024-04-04T06:12:49.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.685+0000 7f1061600640 1 --2- 172.21.15.159:0/3843824927 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f104006c8f0 0x7f104006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:49.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.685+0000 7f1061600640 1 -- 172.21.15.159:0/3843824927 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f10540ad9a0 con 0x7f106406d620 2024-04-04T06:12:49.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.686+0000 7f105be00640 1 --2- 172.21.15.159:0/3843824927 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f104006c8f0 0x7f104006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:49.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.687+0000 7f105be00640 1 --2- 172.21.15.159:0/3843824927 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f104006c8f0 0x7f104006edb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f10641f7490 tx=0x7f104c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:49.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.690+0000 7f1061600640 1 -- 172.21.15.159:0/3843824927 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f105403f070 con 0x7f106406d620 2024-04-04T06:12:49.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:49.932+0000 7f1069e00640 1 -- 172.21.15.159:0/3843824927 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f1064158c10 con 0x7f106406d620 2024-04-04T06:12:50.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.210+0000 7f1061600640 1 -- 172.21.15.159:0/3843824927 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v64) ==== 162+0+0 (secure 0 0 0) 0x7f105407bcd0 con 0x7f106406d620 2024-04-04T06:12:50.204 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-04T06:12:50.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.214+0000 7f1069e00640 1 -- 172.21.15.159:0/3843824927 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f104006c8f0 msgr2=0x7f104006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:50.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.214+0000 7f1069e00640 1 --2- 172.21.15.159:0/3843824927 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f104006c8f0 0x7f104006edb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f10641f7490 tx=0x7f104c009040 comp rx=0 tx=0).stop 2024-04-04T06:12:50.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.214+0000 7f1069e00640 1 -- 172.21.15.159:0/3843824927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f106406d620 msgr2=0x7f10641f7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:50.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.214+0000 7f1069e00640 1 --2- 172.21.15.159:0/3843824927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f106406d620 0x7f10641f7ed0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f105402b930 tx=0x7f10540046c0 comp rx=0 tx=0).stop 2024-04-04T06:12:50.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.214+0000 7f1069e00640 1 -- 172.21.15.159:0/3843824927 shutdown_connections 2024-04-04T06:12:50.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.214+0000 7f1069e00640 1 --2- 172.21.15.159:0/3843824927 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f104006c8f0 0x7f104006edb0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:50.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.214+0000 7f1069e00640 1 --2- 172.21.15.159:0/3843824927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10640aefa0 0x7f1064157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:50.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.214+0000 7f1069e00640 1 --2- 172.21.15.159:0/3843824927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10640b4890 0x7f1064153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:50.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.214+0000 7f1069e00640 1 --2- 172.21.15.159:0/3843824927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f106406d620 0x7f10641f7ed0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:50.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.214+0000 7f1069e00640 1 -- 172.21.15.159:0/3843824927 >> 172.21.15.159:0/3843824927 conn(0x7f10640b1ba0 msgr2=0x7f106415d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:50.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.214+0000 7f1069e00640 1 -- 172.21.15.159:0/3843824927 shutdown_connections 2024-04-04T06:12:50.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.214+0000 7f1069e00640 1 -- 172.21.15.159:0/3843824927 wait complete. 2024-04-04T06:12:50.231 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:12:50.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.422+0000 7fe1a3000640 1 Processor -- start 2024-04-04T06:12:50.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.422+0000 7fe1a3000640 1 -- start start 2024-04-04T06:12:50.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.423+0000 7fe1a3000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe19c161b00 0x7fe19c162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:50.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.423+0000 7fe1a3000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe19c0ad120 0x7fe19c0d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:50.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.424+0000 7fe1a3000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe19c071b60 con 0x7fe19c0ad580 2024-04-04T06:12:50.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.424+0000 7fe1a3000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe19c161ee0 con 0x7fe19c0ad120 2024-04-04T06:12:50.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.424+0000 7fe1a3000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe19c0b3450 con 0x7fe19c161b00 2024-04-04T06:12:50.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.424+0000 7fe19b400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe19c0ad120 0x7fe19c0d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:50.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.424+0000 7fe1a1000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fe19c0ad580 0x7fe19c0b3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:39416/0 (socket says 172.21.15.159:39416) 2024-04-04T06:12:50.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.424+0000 7fe1a1000640 1 -- 172.21.15.159:0/3702028461 learned_addr learned my addr 172.21.15.159:0/3702028461 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:50.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.424+0000 7fe19be00640 1 --2- 172.21.15.159:0/3702028461 >> v2:172.21.15.178:3300/0 conn(0x7fe19c161b00 0x7fe19c162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:50.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.424+0000 7fe19b400640 1 -- 172.21.15.159:0/3702028461 >> v2:172.21.15.178:3300/0 conn(0x7fe19c161b00 msgr2=0x7fe19c162ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:50.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.424+0000 7fe19b400640 1 --2- 172.21.15.159:0/3702028461 >> v2:172.21.15.178:3300/0 conn(0x7fe19c161b00 0x7fe19c162ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:50.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.424+0000 7fe19b400640 1 -- 172.21.15.159:0/3702028461 >> v1:172.21.15.159:6789/0 conn(0x7fe19c0ad580 legacy=0x7fe19c0b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:50.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.425+0000 7fe19b400640 1 -- 172.21.15.159:0/3702028461 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19c071e40 con 0x7fe19c0ad120 2024-04-04T06:12:50.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.424+0000 7fe19aa00640 1 -- 172.21.15.159:0/3702028461 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1713704735 0 0) 0x7fe19c071b60 con 0x7fe19c0ad580 2024-04-04T06:12:50.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.426+0000 7fe19b400640 1 --2- 172.21.15.159:0/3702028461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c0ad120 0x7fe19c0d0650 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe19c0b63b0 tx=0x7fe18802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54dd8a4d65d1008d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:50.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.426+0000 7fe19aa00640 1 -- 172.21.15.159:0/3702028461 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188035020 con 0x7fe19c0ad120 2024-04-04T06:12:50.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.426+0000 7fe19aa00640 1 -- 172.21.15.159:0/3702028461 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe188031830 con 0x7fe19c0ad120 2024-04-04T06:12:50.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.426+0000 7fe19aa00640 1 -- 172.21.15.159:0/3702028461 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188031c00 con 0x7fe19c0ad120 2024-04-04T06:12:50.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.426+0000 7fe1a3000640 1 -- 172.21.15.159:0/3702028461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c0ad120 msgr2=0x7fe19c0d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:50.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.426+0000 7fe1a3000640 1 --2- 172.21.15.159:0/3702028461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c0ad120 0x7fe19c0d0650 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe19c0b63b0 tx=0x7fe18802b8d0 comp rx=0 tx=0).stop 2024-04-04T06:12:50.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.427+0000 7fe1a3000640 1 -- 172.21.15.159:0/3702028461 shutdown_connections 2024-04-04T06:12:50.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.427+0000 7fe1a3000640 1 --2- 172.21.15.159:0/3702028461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c0ad120 0x7fe19c0d0650 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:50.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.427+0000 7fe1a3000640 1 --2- 172.21.15.159:0/3702028461 >> v2:172.21.15.178:3300/0 conn(0x7fe19c161b00 0x7fe19c162ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:50.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.427+0000 7fe1a3000640 1 -- 172.21.15.159:0/3702028461 >> 172.21.15.159:0/3702028461 conn(0x7fe19c06b990 msgr2=0x7fe19c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:50.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.427+0000 7fe1a3000640 1 -- 172.21.15.159:0/3702028461 shutdown_connections 2024-04-04T06:12:50.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.427+0000 7fe1a3000640 1 -- 172.21.15.159:0/3702028461 wait complete. 2024-04-04T06:12:50.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.428+0000 7fe1a3000640 1 Processor -- start 2024-04-04T06:12:50.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.429+0000 7fe1a3000640 1 -- start start 2024-04-04T06:12:50.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.430+0000 7fe1a3000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe19c0ad120 0x7fe19c1f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:50.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.430+0000 7fe1a3000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe19c0ad580 0x7fe19c0adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:50.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.430+0000 7fe1a3000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c161b00 0x7fe19c0ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:50.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.430+0000 7fe1a3000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c162ff0 con 0x7fe19c161b00 2024-04-04T06:12:50.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.430+0000 7fe1a3000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c0b3730 con 0x7fe19c0ad580 2024-04-04T06:12:50.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.430+0000 7fe1a3000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe19c0717f0 con 0x7fe19c0ad120 2024-04-04T06:12:50.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.430+0000 7fe1a1000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c161b00 0x7fe19c0ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:50.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.430+0000 7fe1a1000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c161b00 0x7fe19c0ae140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42866/0 (socket says 172.21.15.159:42866) 2024-04-04T06:12:50.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.430+0000 7fe1a1000640 1 -- 172.21.15.159:0/9116989 learned_addr learned my addr 172.21.15.159:0/9116989 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:50.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.430+0000 7fe19be00640 1 --2- 172.21.15.159:0/9116989 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe19c0ad120 0x7fe19c1f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:50.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.430+0000 7fe19b400640 1 --2- 172.21.15.159:0/9116989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe19c0ad580 0x7fe19c0adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:50.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.430+0000 7fe1a1000640 1 -- 172.21.15.159:0/9116989 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe19c0ad120 msgr2=0x7fe19c1f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:50.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.430+0000 7fe1a1000640 1 --2- 172.21.15.159:0/9116989 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe19c0ad120 0x7fe19c1f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:50.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.430+0000 7fe1a1000640 1 -- 172.21.15.159:0/9116989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe19c0ad580 msgr2=0x7fe19c0adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:50.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.430+0000 7fe1a1000640 1 --2- 172.21.15.159:0/9116989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe19c0ad580 0x7fe19c0adc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:50.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.430+0000 7fe1a1000640 1 -- 172.21.15.159:0/9116989 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19c157180 con 0x7fe19c161b00 2024-04-04T06:12:50.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.430+0000 7fe19be00640 1 --2- 172.21.15.159:0/9116989 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe19c0ad120 0x7fe19c1f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:50.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.430+0000 7fe19b400640 1 --2- 172.21.15.159:0/9116989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe19c0ad580 0x7fe19c0adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:50.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.431+0000 7fe1a1000640 1 --2- 172.21.15.159:0/9116989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c161b00 0x7fe19c0ae140 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe19400c7d0 tx=0x7fe19400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:50.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.431+0000 7fe193e00640 1 -- 172.21.15.159:0/9116989 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194018070 con 0x7fe19c161b00 2024-04-04T06:12:50.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.431+0000 7fe193e00640 1 -- 172.21.15.159:0/9116989 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe19400a640 con 0x7fe19c161b00 2024-04-04T06:12:50.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.431+0000 7fe193e00640 1 -- 172.21.15.159:0/9116989 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19400aa70 con 0x7fe19c161b00 2024-04-04T06:12:50.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.431+0000 7fe1a3000640 1 -- 172.21.15.159:0/9116989 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19c1f3bc0 con 0x7fe19c161b00 2024-04-04T06:12:50.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.431+0000 7fe1a3000640 1 -- 172.21.15.159:0/9116989 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe19c1579b0 con 0x7fe19c161b00 2024-04-04T06:12:50.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.432+0000 7fe1a3000640 1 -- 172.21.15.159:0/9116989 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe19c0685c0 con 0x7fe19c161b00 2024-04-04T06:12:50.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.434+0000 7fe193e00640 1 -- 172.21.15.159:0/9116989 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe19401b070 con 0x7fe19c161b00 2024-04-04T06:12:50.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.435+0000 7fe193e00640 1 --2- 172.21.15.159:0/9116989 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe17c06c920 0x7fe17c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:50.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.435+0000 7fe193e00640 1 -- 172.21.15.159:0/9116989 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7fe19408b200 con 0x7fe19c161b00 2024-04-04T06:12:50.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.435+0000 7fe19be00640 1 --2- 172.21.15.159:0/9116989 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe17c06c920 0x7fe17c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:50.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.437+0000 7fe19be00640 1 --2- 172.21.15.159:0/9116989 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe17c06c920 0x7fe17c06ede0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe18c006fd0 tx=0x7fe18c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:50.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.439+0000 7fe193e00640 1 -- 172.21.15.159:0/9116989 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe194059530 con 0x7fe19c161b00 2024-04-04T06:12:50.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.682+0000 7fe1a3000640 1 -- 172.21.15.159:0/9116989 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe19c06f3a0 con 0x7fe19c161b00 2024-04-04T06:12:50.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.683+0000 7fe193e00640 1 -- 172.21.15.159:0/9116989 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5189 (secure 0 0 0) 0x7fe194058e30 con 0x7fe19c161b00 2024-04-04T06:12:50.676 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:50.676 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1825651864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1825651864},{"type":"v1","addr":"172.21.15.178:6836","nonce":1825651864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1824578712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1824578712},{"type":"v1","addr":"172.21.15.178:6837","nonce":1824578712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6834/3377498298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3377498298},{"type":"v1","addr":"172.21.15.159:6834","nonce":3377498298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:12:35.880454+0000","modified":"2024-04-04T06:12:50.216835+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7512":{"gid":7512,"name":"a","rank":0,"incarnation":65,"state":"up:replay","state_seq":1,"addr":"172.21.15.159:6835/3796123128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":3796123128},{"type":"v1","addr":"172.21.15.159:6835","nonce":3796123128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-04T06:12:50.677 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 65 2024-04-04T06:12:50.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.686+0000 7fe1a3000640 1 -- 172.21.15.159:0/9116989 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe17c06c920 msgr2=0x7fe17c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:50.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.686+0000 7fe1a3000640 1 --2- 172.21.15.159:0/9116989 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe17c06c920 0x7fe17c06ede0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe18c006fd0 tx=0x7fe18c008040 comp rx=0 tx=0).stop 2024-04-04T06:12:50.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.686+0000 7fe1a3000640 1 -- 172.21.15.159:0/9116989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c161b00 msgr2=0x7fe19c0ae140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:50.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.686+0000 7fe1a3000640 1 --2- 172.21.15.159:0/9116989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c161b00 0x7fe19c0ae140 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe19400c7d0 tx=0x7fe19400cc90 comp rx=0 tx=0).stop 2024-04-04T06:12:50.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.686+0000 7fe1a3000640 1 -- 172.21.15.159:0/9116989 shutdown_connections 2024-04-04T06:12:50.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.686+0000 7fe1a3000640 1 --2- 172.21.15.159:0/9116989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe19c161b00 0x7fe19c0ae140 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:50.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.686+0000 7fe1a3000640 1 --2- 172.21.15.159:0/9116989 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe17c06c920 0x7fe17c06ede0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:50.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.686+0000 7fe1a3000640 1 --2- 172.21.15.159:0/9116989 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe19c0ad580 0x7fe19c0adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:50.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.686+0000 7fe1a3000640 1 --2- 172.21.15.159:0/9116989 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe19c0ad120 0x7fe19c1f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:50.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.686+0000 7fe1a3000640 1 -- 172.21.15.159:0/9116989 >> 172.21.15.159:0/9116989 conn(0x7fe19c06b990 msgr2=0x7fe19c0b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:50.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.686+0000 7fe1a3000640 1 -- 172.21.15.159:0/9116989 shutdown_connections 2024-04-04T06:12:50.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:50.686+0000 7fe1a3000640 1 -- 172.21.15.159:0/9116989 wait complete. 2024-04-04T06:12:51.102 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:12:35.880454+0000', 'modified': '2024-04-04T06:12:50.216835+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 86, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7512}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7512': {'gid': 7512, 'name': 'a', 'rank': 0, 'incarnation': 65, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.159:6835/3796123128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6833', 'nonce': 3796123128}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 3796123128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-04T06:12:51.102 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7512:up:replay 2024-04-04T06:12:52.103 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:12:52.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.333+0000 7f5739a00640 1 Processor -- start 2024-04-04T06:12:52.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.333+0000 7f5739a00640 1 -- start start 2024-04-04T06:12:52.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.334+0000 7f5739a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57340cc0b0 0x7f573406dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:52.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.334+0000 7f5739a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f573415fc50 0x7f57340cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:52.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.334+0000 7f5739a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f573406e4f0 con 0x7f57340cc490 2024-04-04T06:12:52.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.334+0000 7f5739a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f573415c610 con 0x7f573415fc50 2024-04-04T06:12:52.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.334+0000 7f5739a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f57340b3150 con 0x7f57340cc0b0 2024-04-04T06:12:52.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.335+0000 7f5733400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f573415fc50 0x7f57340cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:52.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.335+0000 7f5739000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f57340cc490 0x7f573415ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:39428/0 (socket says 172.21.15.159:39428) 2024-04-04T06:12:52.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.335+0000 7f5739000640 1 -- 172.21.15.159:0/3808622743 learned_addr learned my addr 172.21.15.159:0/3808622743 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:52.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.335+0000 7f5733e00640 1 --2- 172.21.15.159:0/3808622743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57340cc0b0 0x7f573406dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:52.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.335+0000 7f5733400640 1 -- 172.21.15.159:0/3808622743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57340cc0b0 msgr2=0x7f573406dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:52.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.335+0000 7f5733400640 1 --2- 172.21.15.159:0/3808622743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57340cc0b0 0x7f573406dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:52.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.335+0000 7f5733400640 1 -- 172.21.15.159:0/3808622743 >> v1:172.21.15.159:6789/0 conn(0x7f57340cc490 legacy=0x7f573415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:52.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.335+0000 7f5733400640 1 -- 172.21.15.159:0/3808622743 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57341600e0 con 0x7f573415fc50 2024-04-04T06:12:52.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.335+0000 7f5732a00640 1 -- 172.21.15.159:0/3808622743 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 246466078 0 0) 0x7f573406e4f0 con 0x7f57340cc490 2024-04-04T06:12:52.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.336+0000 7f5733400640 1 --2- 172.21.15.159:0/3808622743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f573415fc50 0x7f57340cfdc0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f571c009d90 tx=0x7f571c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589230bee93535a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.337+0000 7f5732a00640 1 -- 172.21.15.159:0/3808622743 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c039070 con 0x7f573415fc50 2024-04-04T06:12:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.337+0000 7f5732a00640 1 -- 172.21.15.159:0/3808622743 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f571c02cc50 con 0x7f573415fc50 2024-04-04T06:12:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.337+0000 7f5732a00640 1 -- 172.21.15.159:0/3808622743 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c034440 con 0x7f573415fc50 2024-04-04T06:12:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.337+0000 7f5739a00640 1 -- 172.21.15.159:0/3808622743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f573415fc50 msgr2=0x7f57340cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.337+0000 7f5739a00640 1 --2- 172.21.15.159:0/3808622743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f573415fc50 0x7f57340cfdc0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f571c009d90 tx=0x7f571c02bea0 comp rx=0 tx=0).stop 2024-04-04T06:12:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.337+0000 7f5739a00640 1 -- 172.21.15.159:0/3808622743 shutdown_connections 2024-04-04T06:12:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.337+0000 7f5739a00640 1 --2- 172.21.15.159:0/3808622743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f573415fc50 0x7f57340cfdc0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.337+0000 7f5739a00640 1 --2- 172.21.15.159:0/3808622743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57340cc0b0 0x7f573406dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.337+0000 7f5739a00640 1 -- 172.21.15.159:0/3808622743 >> 172.21.15.159:0/3808622743 conn(0x7f57340ccad0 msgr2=0x7f57340cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.337+0000 7f5739a00640 1 -- 172.21.15.159:0/3808622743 shutdown_connections 2024-04-04T06:12:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.337+0000 7f5739a00640 1 -- 172.21.15.159:0/3808622743 wait complete. 2024-04-04T06:12:52.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.339+0000 7f5739a00640 1 Processor -- start 2024-04-04T06:12:52.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.339+0000 7f5739a00640 1 -- start start 2024-04-04T06:12:52.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.340+0000 7f5739a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57340cc0b0 0x7f5734154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:52.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.340+0000 7f5739a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57340cc490 0x7f57341e6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:52.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.340+0000 7f5739a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57341f6ef0 0x7f573415ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:52.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.340+0000 7f5739a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5734160980 con 0x7f57340cc0b0 2024-04-04T06:12:52.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.340+0000 7f5739a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57341606a0 con 0x7f57340cc490 2024-04-04T06:12:52.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.340+0000 7f5739a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f57340d0300 con 0x7f57341f6ef0 2024-04-04T06:12:52.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.340+0000 7f5733e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57340cc0b0 0x7f5734154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:52.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.340+0000 7f5733e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57340cc0b0 0x7f5734154920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42890/0 (socket says 172.21.15.159:42890) 2024-04-04T06:12:52.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.340+0000 7f5733e00640 1 -- 172.21.15.159:0/1307801149 learned_addr learned my addr 172.21.15.159:0/1307801149 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:52.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.341+0000 7f5733e00640 1 -- 172.21.15.159:0/1307801149 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57341f6ef0 msgr2=0x7f573415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:52.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.341+0000 7f5733e00640 1 --2- 172.21.15.159:0/1307801149 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57341f6ef0 0x7f573415ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:52.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.341+0000 7f5733e00640 1 -- 172.21.15.159:0/1307801149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57340cc490 msgr2=0x7f57341e6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:52.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.341+0000 7f5733e00640 1 --2- 172.21.15.159:0/1307801149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57340cc490 0x7f57341e6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:52.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.341+0000 7f5733e00640 1 -- 172.21.15.159:0/1307801149 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57341e5c90 con 0x7f57340cc0b0 2024-04-04T06:12:52.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.341+0000 7f5733e00640 1 --2- 172.21.15.159:0/1307801149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57340cc0b0 0x7f5734154920 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f572400d800 tx=0x7f572400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:52.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.341+0000 7f572be00640 1 -- 172.21.15.159:0/1307801149 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572400b6c0 con 0x7f57340cc0b0 2024-04-04T06:12:52.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.341+0000 7f572be00640 1 -- 172.21.15.159:0/1307801149 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5724019070 con 0x7f57340cc0b0 2024-04-04T06:12:52.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.341+0000 7f572be00640 1 -- 172.21.15.159:0/1307801149 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572400ba80 con 0x7f57340cc0b0 2024-04-04T06:12:52.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.341+0000 7f5739a00640 1 -- 172.21.15.159:0/1307801149 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5734154f00 con 0x7f57340cc0b0 2024-04-04T06:12:52.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.342+0000 7f5739a00640 1 -- 172.21.15.159:0/1307801149 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57341554c0 con 0x7f57340cc0b0 2024-04-04T06:12:52.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.343+0000 7f5739a00640 1 -- 172.21.15.159:0/1307801149 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5734075940 con 0x7f57340cc0b0 2024-04-04T06:12:52.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.344+0000 7f572be00640 1 -- 172.21.15.159:0/1307801149 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f572400bc20 con 0x7f57340cc0b0 2024-04-04T06:12:52.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.344+0000 7f572be00640 1 --2- 172.21.15.159:0/1307801149 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f571006c8f0 0x7f571006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:52.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.345+0000 7f572be00640 1 -- 172.21.15.159:0/1307801149 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f572408c140 con 0x7f57340cc0b0 2024-04-04T06:12:52.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.345+0000 7f5733400640 1 --2- 172.21.15.159:0/1307801149 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f571006c8f0 0x7f571006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:52.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.346+0000 7f5733400640 1 --2- 172.21.15.159:0/1307801149 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f571006c8f0 0x7f571006edb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f571c036000 tx=0x7f571c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:52.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.349+0000 7f572be00640 1 -- 172.21.15.159:0/1307801149 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f572401c360 con 0x7f57340cc0b0 2024-04-04T06:12:52.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.595+0000 7f5739a00640 1 -- 172.21.15.159:0/1307801149 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5734073830 con 0x7f57340cc0b0 2024-04-04T06:12:52.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.597+0000 7f572be00640 1 -- 172.21.15.159:0/1307801149 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5189 (secure 0 0 0) 0x7f572405a470 con 0x7f57340cc0b0 2024-04-04T06:12:52.590 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:52.590 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1825651864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1825651864},{"type":"v1","addr":"172.21.15.178:6836","nonce":1825651864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1824578712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1824578712},{"type":"v1","addr":"172.21.15.178:6837","nonce":1824578712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6834/3377498298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3377498298},{"type":"v1","addr":"172.21.15.159:6834","nonce":3377498298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:12:35.880454+0000","modified":"2024-04-04T06:12:51.228712+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7512":{"gid":7512,"name":"a","rank":0,"incarnation":65,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.159:6835/3796123128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":3796123128},{"type":"v1","addr":"172.21.15.159:6835","nonce":3796123128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-04T06:12:52.590 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 67 2024-04-04T06:12:52.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.600+0000 7f5739a00640 1 -- 172.21.15.159:0/1307801149 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f571006c8f0 msgr2=0x7f571006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:52.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.600+0000 7f5739a00640 1 --2- 172.21.15.159:0/1307801149 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f571006c8f0 0x7f571006edb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f571c036000 tx=0x7f571c02e000 comp rx=0 tx=0).stop 2024-04-04T06:12:52.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.600+0000 7f5739a00640 1 -- 172.21.15.159:0/1307801149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57340cc0b0 msgr2=0x7f5734154920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:52.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.600+0000 7f5739a00640 1 --2- 172.21.15.159:0/1307801149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57340cc0b0 0x7f5734154920 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f572400d800 tx=0x7f572400dcc0 comp rx=0 tx=0).stop 2024-04-04T06:12:52.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.600+0000 7f5739a00640 1 -- 172.21.15.159:0/1307801149 shutdown_connections 2024-04-04T06:12:52.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.600+0000 7f5739a00640 1 --2- 172.21.15.159:0/1307801149 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57341f6ef0 0x7f573415ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:52.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.600+0000 7f5739a00640 1 --2- 172.21.15.159:0/1307801149 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f571006c8f0 0x7f571006edb0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:52.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.600+0000 7f5739a00640 1 --2- 172.21.15.159:0/1307801149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57340cc490 0x7f57341e6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:52.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.600+0000 7f5739a00640 1 --2- 172.21.15.159:0/1307801149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57340cc0b0 0x7f5734154920 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:52.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.600+0000 7f5739a00640 1 -- 172.21.15.159:0/1307801149 >> 172.21.15.159:0/1307801149 conn(0x7f57340ccad0 msgr2=0x7f573415ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:52.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.600+0000 7f5739a00640 1 -- 172.21.15.159:0/1307801149 shutdown_connections 2024-04-04T06:12:52.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:52.600+0000 7f5739a00640 1 -- 172.21.15.159:0/1307801149 wait complete. 2024-04-04T06:12:53.017 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:12:35.880454+0000', 'modified': '2024-04-04T06:12:51.228712+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 86, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7512}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7512': {'gid': 7512, 'name': 'a', 'rank': 0, 'incarnation': 65, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.159:6835/3796123128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6833', 'nonce': 3796123128}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 3796123128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-04T06:12:53.017 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7512:up:rejoin 2024-04-04T06:12:54.021 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:12:54.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.242+0000 7f55dbe00640 1 Processor -- start 2024-04-04T06:12:54.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.242+0000 7f55dbe00640 1 -- start start 2024-04-04T06:12:54.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.243+0000 7f55dbe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f55d40d4840 0x7f55d40cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:54.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.243+0000 7f55dbe00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55d40d1c90 con 0x7f55d40d0f20 2024-04-04T06:12:54.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.243+0000 7f55dbe00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55d40cf160 con 0x7f55d415c2c0 2024-04-04T06:12:54.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.243+0000 7f55dbe00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f55d40cf440 con 0x7f55d40d4840 2024-04-04T06:12:54.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.244+0000 7f55d8a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f55d40d4840 0x7f55d40cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:54.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.244+0000 7f55d8a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f55d40d4840 0x7f55d40cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35582/0 (socket says 172.21.15.159:35582) 2024-04-04T06:12:54.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.244+0000 7f55d8a00640 1 -- 172.21.15.159:0/4064912582 learned_addr learned my addr 172.21.15.159:0/4064912582 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:54.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.244+0000 7f55d9e00640 1 --1- 172.21.15.159:0/4064912582 >> v1:172.21.15.159:6789/0 conn(0x7f55d40d0f20 0x7f55d40d1370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:46656/0 (socket says 172.21.15.159:46656) 2024-04-04T06:12:54.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.244+0000 7f55d8a00640 1 -- 172.21.15.159:0/4064912582 >> v1:172.21.15.178:6789/0 conn(0x7f55d415c2c0 legacy=0x7f55d40d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:54.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.244+0000 7f55d8a00640 1 -- 172.21.15.159:0/4064912582 >> v1:172.21.15.159:6789/0 conn(0x7f55d40d0f20 legacy=0x7f55d40d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:54.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.244+0000 7f55d8a00640 1 -- 172.21.15.159:0/4064912582 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d40cf720 con 0x7f55d40d4840 2024-04-04T06:12:54.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.244+0000 7f55d3e00640 1 -- 172.21.15.159:0/4064912582 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1728602899 0 0) 0x7f55d40d1c90 con 0x7f55d40d0f20 2024-04-04T06:12:54.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.246+0000 7f55d8a00640 1 --2- 172.21.15.159:0/4064912582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55d40d4840 0x7f55d40cff20 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009870 tx=0x7f55c002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f93ab15b17c77 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:54.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.246+0000 7f55d3e00640 1 -- 172.21.15.159:0/4064912582 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0034070 con 0x7f55d40d4840 2024-04-04T06:12:54.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.246+0000 7f55d3e00640 1 -- 172.21.15.159:0/4064912582 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c0030860 con 0x7f55d40d4840 2024-04-04T06:12:54.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.246+0000 7f55d3e00640 1 -- 172.21.15.159:0/4064912582 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0030c30 con 0x7f55d40d4840 2024-04-04T06:12:54.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.247+0000 7f55dbe00640 1 -- 172.21.15.159:0/4064912582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55d40d4840 msgr2=0x7f55d40cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:54.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.247+0000 7f55dbe00640 1 --2- 172.21.15.159:0/4064912582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55d40d4840 0x7f55d40cff20 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009870 tx=0x7f55c002b9a0 comp rx=0 tx=0).stop 2024-04-04T06:12:54.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.247+0000 7f55dbe00640 1 -- 172.21.15.159:0/4064912582 shutdown_connections 2024-04-04T06:12:54.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.247+0000 7f55dbe00640 1 --2- 172.21.15.159:0/4064912582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55d40d4840 0x7f55d40cff20 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:54.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.247+0000 7f55dbe00640 1 -- 172.21.15.159:0/4064912582 >> 172.21.15.159:0/4064912582 conn(0x7f55d408bfb0 msgr2=0x7f55d409d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:54.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.247+0000 7f55dbe00640 1 -- 172.21.15.159:0/4064912582 shutdown_connections 2024-04-04T06:12:54.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.247+0000 7f55dbe00640 1 -- 172.21.15.159:0/4064912582 wait complete. 2024-04-04T06:12:54.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.249+0000 7f55dbe00640 1 Processor -- start 2024-04-04T06:12:54.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.249+0000 7f55dbe00640 1 -- start start 2024-04-04T06:12:54.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.250+0000 7f55dbe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55d40d0f20 0x7f55d4154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:54.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.250+0000 7f55dbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55d40d4840 0x7f55d40ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:54.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.250+0000 7f55dbe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55d415c2c0 0x7f55d41e94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:54.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.250+0000 7f55dbe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f55d40d2170 con 0x7f55d40d4840 2024-04-04T06:12:54.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.250+0000 7f55dbe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f55d40d1ff0 con 0x7f55d415c2c0 2024-04-04T06:12:54.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.250+0000 7f55dbe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f55d40d22f0 con 0x7f55d40d0f20 2024-04-04T06:12:54.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.250+0000 7f55d8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55d40d4840 0x7f55d40ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:54.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.250+0000 7f55d8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55d40d4840 0x7f55d40ca440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35592/0 (socket says 172.21.15.159:35592) 2024-04-04T06:12:54.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.250+0000 7f55d8a00640 1 -- 172.21.15.159:0/2224376613 learned_addr learned my addr 172.21.15.159:0/2224376613 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:54.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.250+0000 7f55d9400640 1 --2- 172.21.15.159:0/2224376613 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55d40d0f20 0x7f55d4154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:54.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.250+0000 7f55d9e00640 1 --2- 172.21.15.159:0/2224376613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55d415c2c0 0x7f55d41e94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:54.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.251+0000 7f55d8a00640 1 -- 172.21.15.159:0/2224376613 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55d40d0f20 msgr2=0x7f55d4154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:54.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.251+0000 7f55d8a00640 1 --2- 172.21.15.159:0/2224376613 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55d40d0f20 0x7f55d4154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:54.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.251+0000 7f55d8a00640 1 -- 172.21.15.159:0/2224376613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55d415c2c0 msgr2=0x7f55d41e94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:54.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.251+0000 7f55d8a00640 1 --2- 172.21.15.159:0/2224376613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55d415c2c0 0x7f55d41e94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:54.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.251+0000 7f55d8a00640 1 -- 172.21.15.159:0/2224376613 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d41e8410 con 0x7f55d40d4840 2024-04-04T06:12:54.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.251+0000 7f55d8a00640 1 --2- 172.21.15.159:0/2224376613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55d40d4840 0x7f55d40ca440 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009870 tx=0x7f55c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:54.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.251+0000 7f55d1600640 1 -- 172.21.15.159:0/2224376613 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0034040 con 0x7f55d40d4840 2024-04-04T06:12:54.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.251+0000 7f55d1600640 1 -- 172.21.15.159:0/2224376613 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c0030660 con 0x7f55d40d4840 2024-04-04T06:12:54.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.251+0000 7f55dbe00640 1 -- 172.21.15.159:0/2224376613 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d41e86f0 con 0x7f55d40d4840 2024-04-04T06:12:54.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.251+0000 7f55d1600640 1 -- 172.21.15.159:0/2224376613 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0039d20 con 0x7f55d40d4840 2024-04-04T06:12:54.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.251+0000 7f55dbe00640 1 -- 172.21.15.159:0/2224376613 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d41e8cb0 con 0x7f55d40d4840 2024-04-04T06:12:54.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.253+0000 7f55dbe00640 1 -- 172.21.15.159:0/2224376613 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d40685c0 con 0x7f55d40d4840 2024-04-04T06:12:54.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.254+0000 7f55d1600640 1 -- 172.21.15.159:0/2224376613 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55c004b060 con 0x7f55d40d4840 2024-04-04T06:12:54.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.254+0000 7f55d1600640 1 --2- 172.21.15.159:0/2224376613 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f55b406c8f0 0x7f55b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:54.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.255+0000 7f55d1600640 1 -- 172.21.15.159:0/2224376613 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f55c00adb30 con 0x7f55d40d4840 2024-04-04T06:12:54.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.255+0000 7f55d9400640 1 --2- 172.21.15.159:0/2224376613 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f55b406c8f0 0x7f55b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:54.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.256+0000 7f55d9400640 1 --2- 172.21.15.159:0/2224376613 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f55b406c8f0 0x7f55b406edb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f55c8006fd0 tx=0x7f55c8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:54.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.264+0000 7f55d1600640 1 -- 172.21.15.159:0/2224376613 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f55c0038e90 con 0x7f55d40d4840 2024-04-04T06:12:54.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.501+0000 7f55dbe00640 1 -- 172.21.15.159:0/2224376613 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55d40b3d30 con 0x7f55d40d4840 2024-04-04T06:12:54.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.502+0000 7f55d1600640 1 -- 172.21.15.159:0/2224376613 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5196 (secure 0 0 0) 0x7f55c007be60 con 0x7f55d40d4840 2024-04-04T06:12:54.496 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:12:54.496 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1825651864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1825651864},{"type":"v1","addr":"172.21.15.178:6836","nonce":1825651864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1824578712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1824578712},{"type":"v1","addr":"172.21.15.178:6837","nonce":1824578712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6834/3377498298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3377498298},{"type":"v1","addr":"172.21.15.159:6834","nonce":3377498298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:12:35.880454+0000","modified":"2024-04-04T06:12:53.226848+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7512":{"gid":7512,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6835/3796123128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":3796123128},{"type":"v1","addr":"172.21.15.159:6835","nonce":3796123128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7512,"qdb_cluster":[7512]},"id":6}]} 2024-04-04T06:12:54.497 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 68 2024-04-04T06:12:54.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.505+0000 7f55dbe00640 1 -- 172.21.15.159:0/2224376613 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f55b406c8f0 msgr2=0x7f55b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:54.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.505+0000 7f55dbe00640 1 --2- 172.21.15.159:0/2224376613 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f55b406c8f0 0x7f55b406edb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f55c8006fd0 tx=0x7f55c8008040 comp rx=0 tx=0).stop 2024-04-04T06:12:54.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.505+0000 7f55dbe00640 1 -- 172.21.15.159:0/2224376613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55d40d4840 msgr2=0x7f55d40ca440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:54.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.505+0000 7f55dbe00640 1 --2- 172.21.15.159:0/2224376613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55d40d4840 0x7f55d40ca440 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009870 tx=0x7f55c00046c0 comp rx=0 tx=0).stop 2024-04-04T06:12:54.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.505+0000 7f55dbe00640 1 -- 172.21.15.159:0/2224376613 shutdown_connections 2024-04-04T06:12:54.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.505+0000 7f55dbe00640 1 --2- 172.21.15.159:0/2224376613 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f55b406c8f0 0x7f55b406edb0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:54.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.505+0000 7f55dbe00640 1 --2- 172.21.15.159:0/2224376613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55d415c2c0 0x7f55d41e94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:54.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.505+0000 7f55dbe00640 1 --2- 172.21.15.159:0/2224376613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55d40d4840 0x7f55d40ca440 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:54.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.505+0000 7f55dbe00640 1 --2- 172.21.15.159:0/2224376613 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55d40d0f20 0x7f55d4154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:54.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.505+0000 7f55dbe00640 1 -- 172.21.15.159:0/2224376613 >> 172.21.15.159:0/2224376613 conn(0x7f55d408bfb0 msgr2=0x7f55d40d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:54.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.505+0000 7f55dbe00640 1 -- 172.21.15.159:0/2224376613 shutdown_connections 2024-04-04T06:12:54.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:54.506+0000 7f55dbe00640 1 -- 172.21.15.159:0/2224376613 wait complete. 2024-04-04T06:12:54.923 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:12:35.880454+0000', 'modified': '2024-04-04T06:12:53.226848+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 86, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7512}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7512': {'gid': 7512, 'name': 'a', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.159:6835/3796123128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6833', 'nonce': 3796123128}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 3796123128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7512, 'qdb_cluster': [7512]} 2024-04-04T06:12:54.923 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-04T06:12:54.923 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-04T06:12:55.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.143+0000 7fe39a600640 1 Processor -- start 2024-04-04T06:12:55.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.143+0000 7fe39a600640 1 -- start start 2024-04-04T06:12:55.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.145+0000 7fe39a600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe394158ea0 0x7fe3940b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.145+0000 7fe39a600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe39415baa0 con 0x7fe39406cc90 2024-04-04T06:12:55.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.145+0000 7fe39a600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe39415bd80 con 0x7fe3940b3f90 2024-04-04T06:12:55.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.145+0000 7fe39a600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe39415c060 con 0x7fe394158ea0 2024-04-04T06:12:55.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.145+0000 7fe393e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fe39406cc90 0x7fe3940b5ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:46666/0 (socket says 172.21.15.159:46666) 2024-04-04T06:12:55.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.145+0000 7fe393e00640 1 -- 172.21.15.159:0/4136867611 learned_addr learned my addr 172.21.15.159:0/4136867611 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:12:55.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.146+0000 7fe393400640 1 --2- 172.21.15.159:0/4136867611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe394158ea0 0x7fe3940b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.146+0000 7fe392a00640 1 -- 172.21.15.159:0/4136867611 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2588783069 0 0) 0x7fe39415baa0 con 0x7fe39406cc90 2024-04-04T06:12:55.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.146+0000 7fe392a00640 1 -- 172.21.15.159:0/4136867611 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe378005730 con 0x7fe39406cc90 2024-04-04T06:12:55.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.146+0000 7fe392a00640 1 -- 172.21.15.159:0/4136867611 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 638146269 0 0) 0x7fe378005730 con 0x7fe39406cc90 2024-04-04T06:12:55.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.146+0000 7fe392a00640 1 -- 172.21.15.159:0/4136867611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe394158ea0 msgr2=0x7fe3940b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:55.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.146+0000 7fe392a00640 1 --2- 172.21.15.159:0/4136867611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe394158ea0 0x7fe3940b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.146+0000 7fe392a00640 1 -- 172.21.15.159:0/4136867611 >> v1:172.21.15.178:6789/0 conn(0x7fe3940b3f90 legacy=0x7fe39406c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:55.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.146+0000 7fe392a00640 1 -- 172.21.15.159:0/4136867611 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39415c340 con 0x7fe39406cc90 2024-04-04T06:12:55.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.146+0000 7fe392a00640 1 -- 172.21.15.159:0/4136867611 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fe38c0019f0 con 0x7fe39406cc90 2024-04-04T06:12:55.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.146+0000 7fe392a00640 1 -- 172.21.15.159:0/4136867611 >> v1:172.21.15.159:6789/0 conn(0x7fe39406cc90 legacy=0x7fe3940b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:55.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.146+0000 7fe392a00640 1 --2- 172.21.15.159:0/4136867611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe378005bf0 0x7fe378006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.146+0000 7fe392a00640 1 --2- 172.21.15.159:0/4136867611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe378006a10 0x7fe378006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.146+0000 7fe392a00640 1 --2- 172.21.15.159:0/4136867611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe394158ea0 0x7fe3780073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.146+0000 7fe392a00640 1 -- 172.21.15.159:0/4136867611 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe39415c060 con 0x7fe378005bf0 2024-04-04T06:12:55.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.146+0000 7fe392a00640 1 -- 172.21.15.159:0/4136867611 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe37800dc80 con 0x7fe378006a10 2024-04-04T06:12:55.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.146+0000 7fe392a00640 1 -- 172.21.15.159:0/4136867611 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe37800de80 con 0x7fe394158ea0 2024-04-04T06:12:55.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.147+0000 7fe393400640 1 --2- 172.21.15.159:0/4136867611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe378005bf0 0x7fe378006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.147+0000 7fe393400640 1 -- 172.21.15.159:0/4136867611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe394158ea0 msgr2=0x7fe3780073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:55.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.147+0000 7fe393400640 1 --2- 172.21.15.159:0/4136867611 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe394158ea0 0x7fe3780073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.147+0000 7fe38be00640 1 --2- 172.21.15.159:0/4136867611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe378006a10 0x7fe378006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.147+0000 7fe393400640 1 -- 172.21.15.159:0/4136867611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe378006a10 msgr2=0x7fe378006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:55.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.147+0000 7fe393400640 1 --2- 172.21.15.159:0/4136867611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe378006a10 0x7fe378006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.147+0000 7fe393400640 1 -- 172.21.15.159:0/4136867611 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39415c340 con 0x7fe378005bf0 2024-04-04T06:12:55.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.147+0000 7fe38be00640 1 --2- 172.21.15.159:0/4136867611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe378006a10 0x7fe378006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:55.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.148+0000 7fe393400640 1 --2- 172.21.15.159:0/4136867611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe378005bf0 0x7fe378006010 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fe384009a90 tx=0x7fe378007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27832c378bb6adb server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:55.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.148+0000 7fe392a00640 1 -- 172.21.15.159:0/4136867611 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384039070 con 0x7fe378005bf0 2024-04-04T06:12:55.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.148+0000 7fe392a00640 1 -- 172.21.15.159:0/4136867611 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe38402c690 con 0x7fe378005bf0 2024-04-04T06:12:55.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.148+0000 7fe392a00640 1 -- 172.21.15.159:0/4136867611 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384009690 con 0x7fe378005bf0 2024-04-04T06:12:55.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.149+0000 7fe39a600640 1 -- 172.21.15.159:0/4136867611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe378005bf0 msgr2=0x7fe378006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:55.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.149+0000 7fe39a600640 1 --2- 172.21.15.159:0/4136867611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe378005bf0 0x7fe378006010 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fe384009a90 tx=0x7fe378007a40 comp rx=0 tx=0).stop 2024-04-04T06:12:55.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.149+0000 7fe393e00640 1 -- 172.21.15.159:0/4136867611 reap_dead start 2024-04-04T06:12:55.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.149+0000 7fe39a600640 1 -- 172.21.15.159:0/4136867611 shutdown_connections 2024-04-04T06:12:55.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.149+0000 7fe39a600640 1 -- 172.21.15.159:0/4136867611 >> 172.21.15.159:0/4136867611 conn(0x7fe394071e20 msgr2=0x7fe394072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:55.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.149+0000 7fe39a600640 1 -- 172.21.15.159:0/4136867611 shutdown_connections 2024-04-04T06:12:55.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.149+0000 7fe39a600640 1 -- 172.21.15.159:0/4136867611 wait complete. 2024-04-04T06:12:55.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.151+0000 7fe39a600640 1 Processor -- start 2024-04-04T06:12:55.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.151+0000 7fe39a600640 1 -- start start 2024-04-04T06:12:55.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.152+0000 7fe39a600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe378005bf0 0x7fe39406ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.152+0000 7fe39a600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3941f8730 0x7fe394147260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.152+0000 7fe39a600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3941f7b00 0x7fe3941f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.152+0000 7fe39a600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe3940b0320 con 0x7fe378005bf0 2024-04-04T06:12:55.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.152+0000 7fe39a600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe394159280 con 0x7fe3941f8730 2024-04-04T06:12:55.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.152+0000 7fe39a600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe3940b04a0 con 0x7fe3941f7b00 2024-04-04T06:12:55.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.152+0000 7fe393400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe378005bf0 0x7fe39406ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.152+0000 7fe393400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe378005bf0 0x7fe39406ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35604/0 (socket says 172.21.15.159:35604) 2024-04-04T06:12:55.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.152+0000 7fe393400640 1 -- 172.21.15.159:0/2143547800 learned_addr learned my addr 172.21.15.159:0/2143547800 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:55.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.153+0000 7fe38be00640 1 --2- 172.21.15.159:0/2143547800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3941f8730 0x7fe394147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.153+0000 7fe393e00640 1 --2- 172.21.15.159:0/2143547800 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3941f7b00 0x7fe3941f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.153+0000 7fe393400640 1 -- 172.21.15.159:0/2143547800 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3941f7b00 msgr2=0x7fe3941f7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:55.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.153+0000 7fe393400640 1 --2- 172.21.15.159:0/2143547800 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3941f7b00 0x7fe3941f7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.153+0000 7fe393400640 1 -- 172.21.15.159:0/2143547800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3941f8730 msgr2=0x7fe394147260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:55.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.153+0000 7fe393400640 1 --2- 172.21.15.159:0/2143547800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3941f8730 0x7fe394147260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.153+0000 7fe393400640 1 -- 172.21.15.159:0/2143547800 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe394151d60 con 0x7fe378005bf0 2024-04-04T06:12:55.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.153+0000 7fe38be00640 1 --2- 172.21.15.159:0/2143547800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3941f8730 0x7fe394147260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:55.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.153+0000 7fe393e00640 1 --2- 172.21.15.159:0/2143547800 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3941f7b00 0x7fe3941f7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:55.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.153+0000 7fe393400640 1 --2- 172.21.15.159:0/2143547800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe378005bf0 0x7fe39406ebf0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fe384009bb0 tx=0x7fe384004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:55.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.153+0000 7fe38b400640 1 -- 172.21.15.159:0/2143547800 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384039050 con 0x7fe378005bf0 2024-04-04T06:12:55.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.154+0000 7fe38b400640 1 -- 172.21.15.159:0/2143547800 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe384045070 con 0x7fe378005bf0 2024-04-04T06:12:55.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.154+0000 7fe38b400640 1 -- 172.21.15.159:0/2143547800 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384003850 con 0x7fe378005bf0 2024-04-04T06:12:55.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.154+0000 7fe39a600640 1 -- 172.21.15.159:0/2143547800 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39406f440 con 0x7fe378005bf0 2024-04-04T06:12:55.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.154+0000 7fe39a600640 1 -- 172.21.15.159:0/2143547800 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe39406fa00 con 0x7fe378005bf0 2024-04-04T06:12:55.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.157+0000 7fe38b400640 1 -- 172.21.15.159:0/2143547800 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3840039f0 con 0x7fe378005bf0 2024-04-04T06:12:55.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.157+0000 7fe39d3b7740 1 Processor -- start 2024-04-04T06:12:55.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.158+0000 7fe39d3b7740 1 -- start start 2024-04-04T06:12:55.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.158+0000 7fe38b400640 1 --2- 172.21.15.159:0/2143547800 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe37006c920 0x7fe37006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.158+0000 7fe38b400640 1 -- 172.21.15.159:0/2143547800 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7fe3840adc80 con 0x7fe378005bf0 2024-04-04T06:12:55.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.158+0000 7fe39d3b7740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5596ce78b130 0x5596ce75e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.158+0000 7fe393e00640 1 --2- 172.21.15.159:0/2143547800 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe37006c920 0x7fe37006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.158+0000 7fe39d3b7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596ce78b510 0x5596ce78dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.158+0000 7fe39d3b7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce75e610 0x5596ce78e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.158+0000 7fe39d3b7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5596ce725ab0 con 0x5596ce75e610 2024-04-04T06:12:55.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.158+0000 7fe39d3b7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5596ce6c7fa0 con 0x5596ce78b510 2024-04-04T06:12:55.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.158+0000 7fe39d3b7740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5596ce670560 con 0x5596ce78b130 2024-04-04T06:12:55.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.159+0000 7fe393e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce75e610 0x5596ce78e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.159+0000 7fe393e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce75e610 0x5596ce78e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35616/0 (socket says 172.21.15.159:35616) 2024-04-04T06:12:55.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.159+0000 7fe393e00640 1 -- 172.21.15.159:0/1917242297 learned_addr learned my addr 172.21.15.159:0/1917242297 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:55.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.159+0000 7fe38be00640 1 --2- 172.21.15.159:0/1917242297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596ce78b510 0x5596ce78dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.159+0000 7fe393400640 1 --2- 172.21.15.159:0/1917242297 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5596ce78b130 0x5596ce75e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.159+0000 7fe393e00640 1 -- 172.21.15.159:0/1917242297 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5596ce78b130 msgr2=0x5596ce75e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:55.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.159+0000 7fe393e00640 1 --2- 172.21.15.159:0/1917242297 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5596ce78b130 0x5596ce75e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.159+0000 7fe393e00640 1 -- 172.21.15.159:0/1917242297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596ce78b510 msgr2=0x5596ce78dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:55.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.159+0000 7fe393e00640 1 --2- 172.21.15.159:0/1917242297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596ce78b510 0x5596ce78dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.159+0000 7fe393e00640 1 -- 172.21.15.159:0/1917242297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596ce3dc2d0 con 0x5596ce75e610 2024-04-04T06:12:55.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.159+0000 7fe393400640 1 --2- 172.21.15.159:0/1917242297 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5596ce78b130 0x5596ce75e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:55.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.160+0000 7fe393e00640 1 --2- 172.21.15.159:0/1917242297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce75e610 0x5596ce78e400 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fe3940cd2a0 tx=0x7fe38c014e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3438b6039d892060 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:55.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.160+0000 7fe383e00640 1 -- 172.21.15.159:0/1917242297 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c010040 con 0x5596ce75e610 2024-04-04T06:12:55.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.160+0000 7fe383e00640 1 -- 172.21.15.159:0/1917242297 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe38c01d030 con 0x5596ce75e610 2024-04-04T06:12:55.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.160+0000 7fe383e00640 1 -- 172.21.15.159:0/1917242297 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c019a50 con 0x5596ce75e610 2024-04-04T06:12:55.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.161+0000 7fe39d3b7740 1 -- 172.21.15.159:0/1917242297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce75e610 msgr2=0x5596ce78e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:55.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.161+0000 7fe393e00640 1 --2- 172.21.15.159:0/2143547800 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe37006c920 0x7fe37006ede0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe38c012000 tx=0x7fe38c01a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:55.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.161+0000 7fe39d3b7740 1 --2- 172.21.15.159:0/1917242297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce75e610 0x5596ce78e400 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fe3940cd2a0 tx=0x7fe38c014e30 comp rx=0 tx=0).stop 2024-04-04T06:12:55.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.161+0000 7fe39d3b7740 1 -- 172.21.15.159:0/1917242297 shutdown_connections 2024-04-04T06:12:55.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.161+0000 7fe39d3b7740 1 --2- 172.21.15.159:0/1917242297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce75e610 0x5596ce78e400 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.161+0000 7fe39d3b7740 1 --2- 172.21.15.159:0/1917242297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596ce78b510 0x5596ce78dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.161+0000 7fe39d3b7740 1 --2- 172.21.15.159:0/1917242297 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5596ce78b130 0x5596ce75e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.161+0000 7fe39d3b7740 1 -- 172.21.15.159:0/1917242297 >> 172.21.15.159:0/1917242297 conn(0x5596ce760340 msgr2=0x5596ce7a8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:55.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.161+0000 7fe39d3b7740 1 -- 172.21.15.159:0/1917242297 shutdown_connections 2024-04-04T06:12:55.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.161+0000 7fe39d3b7740 1 -- 172.21.15.159:0/1917242297 wait complete. 2024-04-04T06:12:55.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.161+0000 7fe39d3b7740 1 -- start start 2024-04-04T06:12:55.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.162+0000 7fe39d3b7740 1 Processor -- start 2024-04-04T06:12:55.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.163+0000 7fe39d3b7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596ce78b130 0x5596ce7b8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.163+0000 7fe39d3b7740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5596ce78b510 0x5596ce790910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.163+0000 7fe39d3b7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce7a34f0 0x5596ce7a3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.163+0000 7fe39d3b7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5596ce683920 con 0x5596ce7a34f0 2024-04-04T06:12:55.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.163+0000 7fe39d3b7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5596ce6c3f50 con 0x5596ce78b130 2024-04-04T06:12:55.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.163+0000 7fe39d3b7740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5596ce7628e0 con 0x5596ce78b510 2024-04-04T06:12:55.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.163+0000 7fe39d3b7740 10 client.0 _ll_register_callbacks cb 0x5596ce683310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:12:55.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.163+0000 7fe393e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce7a34f0 0x5596ce7a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.163+0000 7fe393e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce7a34f0 0x5596ce7a3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35622/0 (socket says 172.21.15.159:35622) 2024-04-04T06:12:55.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.163+0000 7fe393e00640 1 -- 172.21.15.159:0/3801641197 learned_addr learned my addr 172.21.15.159:0/3801641197 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:55.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.163+0000 7fe393400640 1 --2- 172.21.15.159:0/3801641197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596ce78b130 0x5596ce7b8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.163+0000 7fe38be00640 1 --2- 172.21.15.159:0/3801641197 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5596ce78b510 0x5596ce790910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.163+0000 7fe393e00640 1 -- 172.21.15.159:0/3801641197 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5596ce78b510 msgr2=0x5596ce790910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:55.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.163+0000 7fe393e00640 1 --2- 172.21.15.159:0/3801641197 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5596ce78b510 0x5596ce790910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.163+0000 7fe393e00640 1 -- 172.21.15.159:0/3801641197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596ce78b130 msgr2=0x5596ce7b8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:55.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.163+0000 7fe393e00640 1 --2- 172.21.15.159:0/3801641197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596ce78b130 0x5596ce7b8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.163+0000 7fe393e00640 1 -- 172.21.15.159:0/3801641197 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596ce4a66b0 con 0x5596ce7a34f0 2024-04-04T06:12:55.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.163+0000 7fe393400640 1 --2- 172.21.15.159:0/3801641197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596ce78b130 0x5596ce7b8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:55.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.164+0000 7fe393e00640 1 --2- 172.21.15.159:0/3801641197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce7a34f0 0x5596ce7a3970 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fe38c016fc0 tx=0x7fe38c0019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c724ccc4e56f5cb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:55.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.164+0000 7fe380c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:12:55.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.164+0000 7fe380c00640 1 -- 172.21.15.159:0/3801641197 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c014500 con 0x5596ce7a34f0 2024-04-04T06:12:55.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.164+0000 7fe380c00640 1 -- 172.21.15.159:0/3801641197 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7fe38c004b60 con 0x5596ce7a34f0 2024-04-04T06:12:55.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.164+0000 7fe39d3b7740 1 -- 172.21.15.159:0/3801641197 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe38c01d030 con 0x5596ce7a34f0 2024-04-04T06:12:55.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.164+0000 7fe380c00640 1 -- 172.21.15.159:0/3801641197 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fe38c036050 con 0x5596ce7a34f0 2024-04-04T06:12:55.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.164+0000 7fe39d3b7740 10 client.7581 fetch_fsmap learned FSMap version 68 2024-04-04T06:12:55.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.164+0000 7fe39d3b7740 1 -- 172.21.15.159:0/3801641197 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x5596ce7916b0 con 0x5596ce7a34f0 2024-04-04T06:12:55.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.165+0000 7fe380c00640 1 -- 172.21.15.159:0/3801641197 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe38c01d030 con 0x5596ce7a34f0 2024-04-04T06:12:55.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.165+0000 7fe380c00640 1 -- 172.21.15.159:0/3801641197 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7fe38c039020 con 0x5596ce7a34f0 2024-04-04T06:12:55.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.165+0000 7fe39d3b7740 10 client.7581 fetch_fsmap finished waiting for FSMap version 68 2024-04-04T06:12:55.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.165+0000 7fe39d3b7740 10 client.7581 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:12:55.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.165+0000 7fe39d3b7740 20 client.7581 populate_metadata read hostname 'smithi159' 2024-04-04T06:12:55.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.165+0000 7fe39d3b7740 1 --2- 172.21.15.159:0/3801641197 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x5596ce77dc70 0x5596ce77e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.165+0000 7fe39d3b7740 4 client.7581 mds_command: new command op to 7512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:12:55.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.165+0000 7fe39d3b7740 1 -- 172.21.15.159:0/3801641197 --> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596ce6d54e0 con 0x5596ce77dc70 2024-04-04T06:12:55.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.165+0000 7fe393400640 1 --2- 172.21.15.159:0/3801641197 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x5596ce77dc70 0x5596ce77e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.166+0000 7fe393400640 1 --2- 172.21.15.159:0/3801641197 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x5596ce77dc70 0x5596ce77e050 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9d5496cdc3ccc51 server_cookie=119985b425cc15bc in_seq=0 out_seq=0 2024-04-04T06:12:55.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.166+0000 7fe380c00640 10 client.7581 ms_handle_connect on v2:172.21.15.159:6833/3796123128 2024-04-04T06:12:55.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.166+0000 7fe380c00640 1 -- 172.21.15.159:0/3801641197 <== mds.0 v2:172.21.15.159:6833/3796123128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5596ce6d54e0 con 0x5596ce77dc70 2024-04-04T06:12:55.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.166+0000 7fe380c00640 10 client.7581 handle_command_reply: tid=0 2024-04-04T06:12:55.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.166+0000 7fe39d3b7740 1 client.7581 shutdown 2024-04-04T06:12:55.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.167+0000 7fe39d3b7740 1 -- 172.21.15.159:0/3801641197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce7a34f0 msgr2=0x5596ce7a3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:55.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.167+0000 7fe39d3b7740 1 --2- 172.21.15.159:0/3801641197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce7a34f0 0x5596ce7a3970 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fe38c016fc0 tx=0x7fe38c0019a0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.167+0000 7fe39d3b7740 1 -- 172.21.15.159:0/3801641197 shutdown_connections 2024-04-04T06:12:55.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.167+0000 7fe39d3b7740 1 --2- 172.21.15.159:0/3801641197 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x5596ce77dc70 0x5596ce77e050 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.167+0000 7fe39d3b7740 1 --2- 172.21.15.159:0/3801641197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce7a34f0 0x5596ce7a3970 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.167+0000 7fe39d3b7740 1 --2- 172.21.15.159:0/3801641197 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5596ce78b510 0x5596ce790910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.167+0000 7fe39d3b7740 1 --2- 172.21.15.159:0/3801641197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596ce78b130 0x5596ce7b8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.167+0000 7fe380c00640 0 client.7581 ms_handle_reset on v2:172.21.15.159:6833/3796123128 2024-04-04T06:12:55.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.167+0000 7fe39d3b7740 1 -- 172.21.15.159:0/3801641197 >> 172.21.15.159:0/3801641197 conn(0x5596ce75e610 msgr2=0x5596ce78deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:55.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.167+0000 7fe39d3b7740 1 -- 172.21.15.159:0/3801641197 shutdown_connections 2024-04-04T06:12:55.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.167+0000 7fe39d3b7740 1 -- 172.21.15.159:0/3801641197 wait complete. 2024-04-04T06:12:55.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.167+0000 7fe39d3b7740 20 client.7581 trim_cache size 0 max 16384 2024-04-04T06:12:55.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.187+0000 7fe39d3b7740 1 Processor -- start 2024-04-04T06:12:55.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.187+0000 7fe39d3b7740 1 -- start start 2024-04-04T06:12:55.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.188+0000 7fe39d3b7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce78b130 0x5596ce75ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.188+0000 7fe39d3b7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596ce78b510 0x5596ce7bb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.188+0000 7fe39d3b7740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5596ce7a34f0 0x5596ce7bb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.188+0000 7fe39d3b7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe394200330 con 0x5596ce78b130 2024-04-04T06:12:55.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.188+0000 7fe39d3b7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe394153200 con 0x5596ce78b510 2024-04-04T06:12:55.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.188+0000 7fe39d3b7740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe394153850 con 0x5596ce7a34f0 2024-04-04T06:12:55.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.188+0000 7fe393400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce78b130 0x5596ce75ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.188+0000 7fe393400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce78b130 0x5596ce75ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35628/0 (socket says 172.21.15.159:35628) 2024-04-04T06:12:55.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.188+0000 7fe393400640 1 -- 172.21.15.159:0/2566279374 learned_addr learned my addr 172.21.15.159:0/2566279374 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:55.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.188+0000 7fe393400640 1 -- 172.21.15.159:0/2566279374 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5596ce7a34f0 msgr2=0x5596ce7bb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:55.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.188+0000 7fe393400640 1 --2- 172.21.15.159:0/2566279374 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5596ce7a34f0 0x5596ce7bb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.188+0000 7fe393400640 1 -- 172.21.15.159:0/2566279374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596ce78b510 msgr2=0x5596ce7bb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:55.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.188+0000 7fe393400640 1 --2- 172.21.15.159:0/2566279374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596ce78b510 0x5596ce7bb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.188+0000 7fe393400640 1 -- 172.21.15.159:0/2566279374 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596ce77ee50 con 0x5596ce78b130 2024-04-04T06:12:55.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.188+0000 7fe393400640 1 --2- 172.21.15.159:0/2566279374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce78b130 0x5596ce75ce70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fe384009310 tx=0x7fe38402df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8235f9bf42951a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:55.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.189+0000 7fe383400640 1 -- 172.21.15.159:0/2566279374 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38407fe20 con 0x5596ce78b130 2024-04-04T06:12:55.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.189+0000 7fe383400640 1 -- 172.21.15.159:0/2566279374 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe38403a040 con 0x5596ce78b130 2024-04-04T06:12:55.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.189+0000 7fe383400640 1 -- 172.21.15.159:0/2566279374 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384084650 con 0x5596ce78b130 2024-04-04T06:12:55.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.189+0000 7fe39d3b7740 1 -- 172.21.15.159:0/2566279374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce78b130 msgr2=0x5596ce75ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:55.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.189+0000 7fe39d3b7740 1 --2- 172.21.15.159:0/2566279374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce78b130 0x5596ce75ce70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fe384009310 tx=0x7fe38402df90 comp rx=0 tx=0).stop 2024-04-04T06:12:55.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.189+0000 7fe39d3b7740 1 -- 172.21.15.159:0/2566279374 shutdown_connections 2024-04-04T06:12:55.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.189+0000 7fe39d3b7740 1 --2- 172.21.15.159:0/2566279374 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5596ce7a34f0 0x5596ce7bb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.189+0000 7fe39d3b7740 1 --2- 172.21.15.159:0/2566279374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596ce78b510 0x5596ce7bb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.189+0000 7fe39d3b7740 1 --2- 172.21.15.159:0/2566279374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce78b130 0x5596ce75ce70 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.189+0000 7fe39d3b7740 1 -- 172.21.15.159:0/2566279374 >> 172.21.15.159:0/2566279374 conn(0x5596ce75e610 msgr2=0x5596ce77e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:55.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.189+0000 7fe39d3b7740 1 -- 172.21.15.159:0/2566279374 shutdown_connections 2024-04-04T06:12:55.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.189+0000 7fe39d3b7740 1 -- 172.21.15.159:0/2566279374 wait complete. 2024-04-04T06:12:55.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.190+0000 7fe39d3b7740 1 -- start start 2024-04-04T06:12:55.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.190+0000 7fe39d3b7740 1 Processor -- start 2024-04-04T06:12:55.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.191+0000 7fe39d3b7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596ce78b130 0x5596ce7cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.191+0000 7fe39d3b7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce78b510 0x5596ce75cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.191+0000 7fe39d3b7740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5596ce7a34f0 0x5596ce80a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.191+0000 7fe39d3b7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5596ce77eaa0 con 0x5596ce78b510 2024-04-04T06:12:55.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.191+0000 7fe39d3b7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5596ce77d150 con 0x5596ce78b130 2024-04-04T06:12:55.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.191+0000 7fe39d3b7740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5596ce7bcfc0 con 0x5596ce7a34f0 2024-04-04T06:12:55.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.191+0000 7fe39d3b7740 10 client.0 _ll_register_callbacks cb 0x7fe38c001e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:12:55.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.191+0000 7fe38be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce78b510 0x5596ce75cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.192+0000 7fe38be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce78b510 0x5596ce75cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35640/0 (socket says 172.21.15.159:35640) 2024-04-04T06:12:55.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.192+0000 7fe38be00640 1 -- 172.21.15.159:0/1980150933 learned_addr learned my addr 172.21.15.159:0/1980150933 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:55.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.192+0000 7fe393e00640 1 --2- 172.21.15.159:0/1980150933 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5596ce7a34f0 0x5596ce80a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.192+0000 7fe393400640 1 --2- 172.21.15.159:0/1980150933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596ce78b130 0x5596ce7cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.192+0000 7fe38be00640 1 -- 172.21.15.159:0/1980150933 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5596ce7a34f0 msgr2=0x5596ce80a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:55.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.192+0000 7fe38be00640 1 --2- 172.21.15.159:0/1980150933 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5596ce7a34f0 0x5596ce80a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.192+0000 7fe38be00640 1 -- 172.21.15.159:0/1980150933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596ce78b130 msgr2=0x5596ce7cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:55.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.192+0000 7fe38be00640 1 --2- 172.21.15.159:0/1980150933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596ce78b130 0x5596ce7cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.192+0000 7fe38be00640 1 -- 172.21.15.159:0/1980150933 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596ce7aa420 con 0x5596ce78b510 2024-04-04T06:12:55.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.192+0000 7fe38be00640 1 --2- 172.21.15.159:0/1980150933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce78b510 0x5596ce75cf70 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fe37c001df0 tx=0x7fe37c012690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e518d01bf0a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:55.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.192+0000 7fe380c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:12:55.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.193+0000 7fe380c00640 1 -- 172.21.15.159:0/1980150933 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c019030 con 0x5596ce78b510 2024-04-04T06:12:55.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.193+0000 7fe380c00640 1 -- 172.21.15.159:0/1980150933 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7fe37c015650 con 0x5596ce78b510 2024-04-04T06:12:55.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.193+0000 7fe39d3b7740 1 -- 172.21.15.159:0/1980150933 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe38403a040 con 0x5596ce78b510 2024-04-04T06:12:55.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.193+0000 7fe380c00640 1 -- 172.21.15.159:0/1980150933 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fe37c025050 con 0x5596ce78b510 2024-04-04T06:12:55.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.193+0000 7fe39d3b7740 10 client.7587 fetch_fsmap learned FSMap version 68 2024-04-04T06:12:55.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.193+0000 7fe39d3b7740 1 -- 172.21.15.159:0/1980150933 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x5596ce7aa9e0 con 0x5596ce78b510 2024-04-04T06:12:55.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.193+0000 7fe380c00640 1 -- 172.21.15.159:0/1980150933 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe38403a040 con 0x5596ce78b510 2024-04-04T06:12:55.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.193+0000 7fe380c00640 1 -- 172.21.15.159:0/1980150933 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7fe37c028020 con 0x5596ce78b510 2024-04-04T06:12:55.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.193+0000 7fe39d3b7740 10 client.7587 fetch_fsmap finished waiting for FSMap version 68 2024-04-04T06:12:55.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.193+0000 7fe39d3b7740 10 client.7587 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:12:55.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.193+0000 7fe39d3b7740 20 client.7587 populate_metadata read hostname 'smithi159' 2024-04-04T06:12:55.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.193+0000 7fe39d3b7740 1 --2- 172.21.15.159:0/1980150933 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x5596ce77c030 0x5596ce7b7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.193+0000 7fe39d3b7740 4 client.7587 mds_command: new command op to 7512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-04T06:12:55.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.193+0000 7fe39d3b7740 1 -- 172.21.15.159:0/1980150933 --> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] -- command(tid 0: {"prefix": "status"}) -- 0x5596ce6cf830 con 0x5596ce77c030 2024-04-04T06:12:55.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.194+0000 7fe393400640 1 --2- 172.21.15.159:0/1980150933 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x5596ce77c030 0x5596ce7b7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.194+0000 7fe393400640 1 --2- 172.21.15.159:0/1980150933 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x5596ce77c030 0x5596ce7b7590 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35cd54719f9137cb server_cookie=e24b81820d1cafd6 in_seq=0 out_seq=0 2024-04-04T06:12:55.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.194+0000 7fe380c00640 10 client.7587 ms_handle_connect on v2:172.21.15.159:6833/3796123128 2024-04-04T06:12:55.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.194+0000 7fe380c00640 1 -- 172.21.15.159:0/1980150933 <== mds.0 v2:172.21.15.159:6833/3796123128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5596ce6cf830 con 0x5596ce77c030 2024-04-04T06:12:55.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.194+0000 7fe380c00640 10 client.7587 handle_command_reply: tid=0 2024-04-04T06:12:55.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.194+0000 7fe39d3b7740 1 client.7587 shutdown 2024-04-04T06:12:55.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.194+0000 7fe39d3b7740 1 -- 172.21.15.159:0/1980150933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce78b510 msgr2=0x5596ce75cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:55.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.195+0000 7fe39d3b7740 1 --2- 172.21.15.159:0/1980150933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce78b510 0x5596ce75cf70 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fe37c001df0 tx=0x7fe37c012690 comp rx=0 tx=0).stop 2024-04-04T06:12:55.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.195+0000 7fe39d3b7740 1 -- 172.21.15.159:0/1980150933 shutdown_connections 2024-04-04T06:12:55.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.195+0000 7fe39d3b7740 1 --2- 172.21.15.159:0/1980150933 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x5596ce77c030 0x5596ce7b7590 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.195+0000 7fe39d3b7740 1 --2- 172.21.15.159:0/1980150933 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5596ce7a34f0 0x5596ce80a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.195+0000 7fe39d3b7740 1 --2- 172.21.15.159:0/1980150933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596ce78b510 0x5596ce75cf70 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.195+0000 7fe39d3b7740 1 --2- 172.21.15.159:0/1980150933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596ce78b130 0x5596ce7cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.195+0000 7fe380c00640 0 client.7587 ms_handle_reset on v2:172.21.15.159:6833/3796123128 2024-04-04T06:12:55.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.195+0000 7fe39d3b7740 1 -- 172.21.15.159:0/1980150933 >> 172.21.15.159:0/1980150933 conn(0x5596ce75e610 msgr2=0x5596ce77e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:55.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.195+0000 7fe39d3b7740 1 -- 172.21.15.159:0/1980150933 shutdown_connections 2024-04-04T06:12:55.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.195+0000 7fe39d3b7740 1 -- 172.21.15.159:0/1980150933 wait complete. 2024-04-04T06:12:55.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.195+0000 7fe39d3b7740 20 client.7587 trim_cache size 0 max 16384 2024-04-04T06:12:55.193 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:12:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_fsid": "30311045-f714-48fa-832b-4cb2ba8bf0bf", 2024-04-04T06:12:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: "whoami": 0, 2024-04-04T06:12:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 7512, 2024-04-04T06:12:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: "want_state": "up:active", 2024-04-04T06:12:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: "state": "up:active", 2024-04-04T06:12:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs", 2024-04-04T06:12:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: "rank_uptime": 4.9715125139999996, 2024-04-04T06:12:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: "mdsmap_epoch": 68, 2024-04-04T06:12:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch": 86, 2024-04-04T06:12:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch_barrier": 86, 2024-04-04T06:12:55.193 INFO:teuthology.orchestra.run.smithi159.stdout: "uptime": 5.8814424120000002 2024-04-04T06:12:55.193 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:12:55.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.195+0000 7fe39a600640 1 -- 172.21.15.159:0/2143547800 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe37006c920 msgr2=0x7fe37006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:55.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.195+0000 7fe39a600640 1 --2- 172.21.15.159:0/2143547800 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe37006c920 0x7fe37006ede0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe38c012000 tx=0x7fe38c01a000 comp rx=0 tx=0).stop 2024-04-04T06:12:55.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.195+0000 7fe39a600640 1 -- 172.21.15.159:0/2143547800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe378005bf0 msgr2=0x7fe39406ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:55.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.195+0000 7fe39a600640 1 --2- 172.21.15.159:0/2143547800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe378005bf0 0x7fe39406ebf0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fe384009bb0 tx=0x7fe384004730 comp rx=0 tx=0).stop 2024-04-04T06:12:55.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.196+0000 7fe39a600640 1 -- 172.21.15.159:0/2143547800 shutdown_connections 2024-04-04T06:12:55.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.196+0000 7fe39a600640 1 --2- 172.21.15.159:0/2143547800 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3941f7b00 0x7fe3941f7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.196+0000 7fe39a600640 1 --2- 172.21.15.159:0/2143547800 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe37006c920 0x7fe37006ede0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.196+0000 7fe39a600640 1 --2- 172.21.15.159:0/2143547800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3941f8730 0x7fe394147260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.196+0000 7fe39a600640 1 --2- 172.21.15.159:0/2143547800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe378005bf0 0x7fe39406ebf0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.196+0000 7fe39a600640 1 -- 172.21.15.159:0/2143547800 >> 172.21.15.159:0/2143547800 conn(0x7fe394071e20 msgr2=0x7fe3940ae030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:55.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.196+0000 7fe39a600640 1 -- 172.21.15.159:0/2143547800 shutdown_connections 2024-04-04T06:12:55.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.196+0000 7fe39a600640 1 -- 172.21.15.159:0/2143547800 wait complete. 2024-04-04T06:12:55.205 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-04T06:12:55.219 INFO:tasks.ceph.mds.a.smithi159.stderr:2024-04-04T06:12:55.226+0000 7fb388a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:12:55.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.402+0000 7f589b200640 1 Processor -- start 2024-04-04T06:12:55.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.402+0000 7f589b200640 1 -- start start 2024-04-04T06:12:55.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.403+0000 7f589b200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f589c06b570 0x7f589c06fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.404+0000 7f589b200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f589c15fed0 0x7f589c15d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.404+0000 7f589b200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f589c145f50 con 0x7f589c06b950 2024-04-04T06:12:55.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.404+0000 7f589b200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f589c15d920 con 0x7f589c15fed0 2024-04-04T06:12:55.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.404+0000 7f589b200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f589c1603d0 con 0x7f589c06b570 2024-04-04T06:12:55.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.404+0000 7f5899e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f589c06b570 0x7f589c06fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.404+0000 7f5893e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f589c15fed0 0x7f589c15d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.404+0000 7f5899e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f589c06b570 0x7f589c06fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:41854/0 (socket says 172.21.15.159:41854) 2024-04-04T06:12:55.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.404+0000 7f5899e00640 1 -- 172.21.15.159:0/2048802782 learned_addr learned my addr 172.21.15.159:0/2048802782 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:55.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.405+0000 7f5899e00640 1 -- 172.21.15.159:0/2048802782 >> v2:172.21.15.178:3300/0 conn(0x7f589c15fed0 msgr2=0x7f589c15d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:55.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.405+0000 7f5899e00640 1 --2- 172.21.15.159:0/2048802782 >> v2:172.21.15.178:3300/0 conn(0x7f589c15fed0 0x7f589c15d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.405+0000 7f5899e00640 1 -- 172.21.15.159:0/2048802782 >> v1:172.21.15.178:6789/0 conn(0x7f589c06b950 legacy=0x7f589c0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:55.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.405+0000 7f5899e00640 1 -- 172.21.15.159:0/2048802782 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589c1606b0 con 0x7f589c06b570 2024-04-04T06:12:55.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.406+0000 7f5899e00640 1 --2- 172.21.15.159:0/2048802782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f589c06b570 0x7f589c06fa90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f588c0098a0 tx=0x7f588c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a58085d09ba316b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:55.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.407+0000 7f5899400640 1 -- 172.21.15.159:0/2048802782 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c035020 con 0x7f589c06b570 2024-04-04T06:12:55.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.407+0000 7f5899400640 1 -- 172.21.15.159:0/2048802782 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f588c031640 con 0x7f589c06b570 2024-04-04T06:12:55.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.407+0000 7f5899400640 1 -- 172.21.15.159:0/2048802782 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c02d040 con 0x7f589c06b570 2024-04-04T06:12:55.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.407+0000 7f589b200640 1 -- 172.21.15.159:0/2048802782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f589c06b570 msgr2=0x7f589c06fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:55.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.407+0000 7f589b200640 1 --2- 172.21.15.159:0/2048802782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f589c06b570 0x7f589c06fa90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f588c0098a0 tx=0x7f588c02b8f0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.407+0000 7f589b200640 1 -- 172.21.15.159:0/2048802782 shutdown_connections 2024-04-04T06:12:55.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.407+0000 7f589b200640 1 --2- 172.21.15.159:0/2048802782 >> v2:172.21.15.178:3300/0 conn(0x7f589c15fed0 0x7f589c15d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.407+0000 7f589b200640 1 --2- 172.21.15.159:0/2048802782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f589c06b570 0x7f589c06fa90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.407+0000 7f589b200640 1 -- 172.21.15.159:0/2048802782 >> 172.21.15.159:0/2048802782 conn(0x7f589c0d1090 msgr2=0x7f589c0d14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:55.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.407+0000 7f589b200640 1 -- 172.21.15.159:0/2048802782 shutdown_connections 2024-04-04T06:12:55.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.408+0000 7f589b200640 1 -- 172.21.15.159:0/2048802782 wait complete. 2024-04-04T06:12:55.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.409+0000 7f589b200640 1 Processor -- start 2024-04-04T06:12:55.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.409+0000 7f589b200640 1 -- start start 2024-04-04T06:12:55.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.410+0000 7f589b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f589c06b570 0x7f589c15b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.410+0000 7f589b200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f589c06b950 0x7f589c13ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.410+0000 7f589b200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f589c15fed0 0x7f589c13f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.410+0000 7f589b200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f589c0cfc20 con 0x7f589c06b570 2024-04-04T06:12:55.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.410+0000 7f589b200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f589c160c70 con 0x7f589c15fed0 2024-04-04T06:12:55.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.410+0000 7f589b200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f589c06bde0 con 0x7f589c06b950 2024-04-04T06:12:55.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.410+0000 7f5899e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f589c06b570 0x7f589c15b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.410+0000 7f5899e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f589c06b570 0x7f589c15b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35646/0 (socket says 172.21.15.159:35646) 2024-04-04T06:12:55.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.410+0000 7f5899e00640 1 -- 172.21.15.159:0/4166922420 learned_addr learned my addr 172.21.15.159:0/4166922420 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:55.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.411+0000 7f589a800640 1 --2- 172.21.15.159:0/4166922420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f589c15fed0 0x7f589c13f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.411+0000 7f5899e00640 1 -- 172.21.15.159:0/4166922420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f589c06b950 msgr2=0x7f589c13ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:55.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.411+0000 7f5893e00640 1 --2- 172.21.15.159:0/4166922420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f589c06b950 0x7f589c13ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.411+0000 7f5899e00640 1 --2- 172.21.15.159:0/4166922420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f589c06b950 0x7f589c13ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.411+0000 7f5899e00640 1 -- 172.21.15.159:0/4166922420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f589c15fed0 msgr2=0x7f589c13f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:55.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.411+0000 7f5899e00640 1 --2- 172.21.15.159:0/4166922420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f589c15fed0 0x7f589c13f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.411+0000 7f5899e00640 1 -- 172.21.15.159:0/4166922420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589c13d870 con 0x7f589c06b570 2024-04-04T06:12:55.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.411+0000 7f589a800640 1 --2- 172.21.15.159:0/4166922420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f589c15fed0 0x7f589c13f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:55.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.411+0000 7f5899e00640 1 --2- 172.21.15.159:0/4166922420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f589c06b570 0x7f589c15b8f0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f588c000c00 tx=0x7f588c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:55.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.412+0000 7f5892000640 1 -- 172.21.15.159:0/4166922420 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c040070 con 0x7f589c06b570 2024-04-04T06:12:55.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.412+0000 7f5892000640 1 -- 172.21.15.159:0/4166922420 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f588c0399f0 con 0x7f589c06b570 2024-04-04T06:12:55.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.412+0000 7f5892000640 1 -- 172.21.15.159:0/4166922420 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c035040 con 0x7f589c06b570 2024-04-04T06:12:55.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.412+0000 7f589b200640 1 -- 172.21.15.159:0/4166922420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589c1f8cc0 con 0x7f589c06b570 2024-04-04T06:12:55.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.412+0000 7f589b200640 1 -- 172.21.15.159:0/4166922420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f589c1f4d60 con 0x7f589c06b570 2024-04-04T06:12:55.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.413+0000 7f589b200640 1 -- 172.21.15.159:0/4166922420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f589c075940 con 0x7f589c06b570 2024-04-04T06:12:55.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.414+0000 7f5892000640 1 -- 172.21.15.159:0/4166922420 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f588c04b060 con 0x7f589c06b570 2024-04-04T06:12:55.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.415+0000 7f5892000640 1 --2- 172.21.15.159:0/4166922420 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f587806c920 0x7f587806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:55.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.415+0000 7f5892000640 1 -- 172.21.15.159:0/4166922420 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f588c0ad400 con 0x7f589c06b570 2024-04-04T06:12:55.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.416+0000 7f5893e00640 1 --2- 172.21.15.159:0/4166922420 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f587806c920 0x7f587806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:55.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.417+0000 7f5893e00640 1 --2- 172.21.15.159:0/4166922420 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f587806c920 0x7f587806ede0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f589c15c3a0 tx=0x7f5884008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:55.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.419+0000 7f5892000640 1 -- 172.21.15.159:0/4166922420 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f588c07b7c0 con 0x7f589c06b570 2024-04-04T06:12:55.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.832+0000 7f589b200640 1 -- 172.21.15.159:0/4166922420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f589c06c420 con 0x7f589c06b570 2024-04-04T06:12:55.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.833+0000 7f5892000640 1 -- 172.21.15.159:0/4166922420 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v68) ==== 53+0+73 (secure 0 0 0) 0x7f588c07b0c0 con 0x7f589c06b570 2024-04-04T06:12:55.827 INFO:teuthology.orchestra.run.smithi159.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-04T06:12:55.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.835+0000 7f589b200640 1 -- 172.21.15.159:0/4166922420 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f587806c920 msgr2=0x7f587806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:55.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.835+0000 7f589b200640 1 --2- 172.21.15.159:0/4166922420 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f587806c920 0x7f587806ede0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f589c15c3a0 tx=0x7f5884008040 comp rx=0 tx=0).stop 2024-04-04T06:12:55.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.836+0000 7f589b200640 1 -- 172.21.15.159:0/4166922420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f589c06b570 msgr2=0x7f589c15b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:55.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.836+0000 7f589b200640 1 --2- 172.21.15.159:0/4166922420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f589c06b570 0x7f589c15b8f0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f588c000c00 tx=0x7f588c0046c0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.836+0000 7f589b200640 1 -- 172.21.15.159:0/4166922420 shutdown_connections 2024-04-04T06:12:55.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.836+0000 7f589b200640 1 --2- 172.21.15.159:0/4166922420 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f587806c920 0x7f587806ede0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.836+0000 7f589b200640 1 --2- 172.21.15.159:0/4166922420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f589c15fed0 0x7f589c13f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.836+0000 7f589b200640 1 --2- 172.21.15.159:0/4166922420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f589c06b950 0x7f589c13ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.836+0000 7f589b200640 1 --2- 172.21.15.159:0/4166922420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f589c06b570 0x7f589c15b8f0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:55.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.836+0000 7f589b200640 1 -- 172.21.15.159:0/4166922420 >> 172.21.15.159:0/4166922420 conn(0x7f589c0d1090 msgr2=0x7f589c0d1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:55.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.836+0000 7f589b200640 1 -- 172.21.15.159:0/4166922420 shutdown_connections 2024-04-04T06:12:55.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:55.836+0000 7f589b200640 1 -- 172.21.15.159:0/4166922420 wait complete. 2024-04-04T06:12:55.846 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-04T06:12:55.847 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-04T06:12:55.847 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi159.front.sepia.ceph.com 2024-04-04T06:12:55.847 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-04T06:12:55.847 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-04T06:12:55.847 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-04T06:12:55.847 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-04T06:12:55.847 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-04T06:12:55.847 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:12:55.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:55.847 DEBUG:teuthology.orchestra.run.smithi159:> ip addr 2024-04-04T06:12:55.936 INFO:teuthology.orchestra.run.smithi159.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:12:55.937 INFO:teuthology.orchestra.run.smithi159.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:12:55.937 INFO:teuthology.orchestra.run.smithi159.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:12:55.937 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:12:55.937 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 ::1/128 scope host 2024-04-04T06:12:55.937 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:12:55.937 INFO:teuthology.orchestra.run.smithi159.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:12:55.937 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c6 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:12:55.937 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f0 2024-04-04T06:12:55.937 INFO:teuthology.orchestra.run.smithi159.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:12:55.937 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c7 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:12:55.937 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f1 2024-04-04T06:12:55.937 INFO:teuthology.orchestra.run.smithi159.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:12:55.937 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4e brd ff:ff:ff:ff:ff:ff 2024-04-04T06:12:55.937 INFO:teuthology.orchestra.run.smithi159.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-04T06:12:55.938 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:12:55.938 INFO:teuthology.orchestra.run.smithi159.stdout: inet 172.21.15.159/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-04T06:12:55.938 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft 42157sec preferred_lft 42157sec 2024-04-04T06:12:55.938 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::ec4:7aff:fe88:724f/64 scope link 2024-04-04T06:12:55.938 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:12:55.938 INFO:teuthology.orchestra.run.smithi159.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-04T06:12:55.938 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:12:55.938 INFO:teuthology.orchestra.run.smithi159.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-04T06:12:55.938 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:12:55.938 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::989e:e4ff:fea5:7568/64 scope link 2024-04-04T06:12:55.938 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:12:55.938 INFO:teuthology.orchestra.run.smithi159.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:12:55.938 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether a2:e3:e0:cb:0a:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-04T06:12:55.938 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::a0e3:e0ff:fecb:a73/64 scope link 2024-04-04T06:12:55.939 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:12:55.939 INFO:teuthology.orchestra.run.smithi159.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:12:55.939 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-04T06:12:55.939 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::c0a:8aff:fe54:b58f/64 scope link 2024-04-04T06:12:55.939 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:12:55.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:55.939 DEBUG:teuthology.orchestra.run.smithi159:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-04T06:12:56.017 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:12:56.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:56.021 DEBUG:teuthology.orchestra.run.smithi159:> route 2024-04-04T06:12:56.071 INFO:teuthology.orchestra.run.smithi159.stdout:Kernel IP routing table 2024-04-04T06:12:56.071 INFO:teuthology.orchestra.run.smithi159.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-04T06:12:56.071 INFO:teuthology.orchestra.run.smithi159.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-04T06:12:56.071 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-04T06:12:56.071 INFO:teuthology.orchestra.run.smithi159.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-04T06:12:56.071 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:12:56.071 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:12:56.072 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-04T06:12:56.072 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-04T06:12:56.072 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-04T06:12:56.072 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:12:56.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:12:56.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:12:56.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:56.311 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-04-04T06:12:56.358 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-04T06:12:56.359 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-04T06:12:56.360 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:12:56.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:56.360 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set brx.0 up 2024-04-04T06:12:56.455 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:12:56.456 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-04T06:12:56.504 INFO:teuthology.orchestra.run.smithi159.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:12:56.505 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:12:56.505 DEBUG:teuthology.orchestra.run.smithi159:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:12:56.548 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-04T06:12:56.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:56.548 DEBUG:teuthology.orchestra.run.smithi159:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:12:56.721 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:12:56.721 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:12:56.721 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:12:56.741 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:12:56.741 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:12:56.741 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:12:56.741 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:12:56.741 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:12:56.769 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:12:56.769 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:12:56.769 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:12:56.769 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:12:56.769 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:12:56.769 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-04T06:12:56.769 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-04T06:12:56.769 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30311045-f714-48fa-832b-4cb2ba8bf0bf" will pass to kernel 2024-04-04T06:12:56.771 INFO:teuthology.orchestra.run.smithi159.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-04T06:12:56.771 INFO:teuthology.orchestra.run.smithi159.stdout: You just mounted a file system that supports labels which does not 2024-04-04T06:12:56.771 INFO:teuthology.orchestra.run.smithi159.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-04T06:12:56.771 INFO:teuthology.orchestra.run.smithi159.stdout: applications will generate AVC messages and not be allowed access to 2024-04-04T06:12:56.772 INFO:teuthology.orchestra.run.smithi159.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-04T06:12:56.776 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-04T06:12:56.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:56.776 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:12:56.851 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:12:56.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:12:56.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:12:56.959 INFO:teuthology.orchestra.run.smithi159.stdout:client5372 2024-04-04T06:12:56.960 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:12:56.960 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:12:57.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:57.008 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-04-04T06:12:57.107 INFO:tasks.ceph.mds.a.smithi159.stderr:2024-04-04T06:12:57.114+0000 7fb383a00640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-04T06:12:57.108 INFO:teuthology.orchestra.run.smithi159.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-04-04T06:12:57.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-04T06:12:57.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:57.110 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-04-04T06:12:57.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.325+0000 7f69f5c00640 1 Processor -- start 2024-04-04T06:12:57.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.325+0000 7f69f5c00640 1 -- start start 2024-04-04T06:12:57.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.327+0000 7f69f5c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f69f0161f10 0x7f69f00d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.327+0000 7f69f5c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f69f00b3270 0x7f69f00d4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.327+0000 7f69f5c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69f006d370 con 0x7f69f006da00 2024-04-04T06:12:57.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.327+0000 7f69f5c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f69f006d650 con 0x7f69f00b3270 2024-04-04T06:12:57.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.327+0000 7f69f5c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f69f00d0f00 con 0x7f69f0161f10 2024-04-04T06:12:57.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.327+0000 7f69ee000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f69f00b3270 0x7f69f00d4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.327+0000 7f69ee000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f69f00b3270 0x7f69f00d4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35648/0 (socket says 172.21.15.159:35648) 2024-04-04T06:12:57.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.327+0000 7f69ee000640 1 -- 172.21.15.159:0/334339371 learned_addr learned my addr 172.21.15.159:0/334339371 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:57.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.328+0000 7f69eea00640 1 --2- 172.21.15.159:0/334339371 >> v2:172.21.15.178:3300/0 conn(0x7f69f0161f10 0x7f69f00d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.328+0000 7f69ee000640 1 -- 172.21.15.159:0/334339371 >> v2:172.21.15.178:3300/0 conn(0x7f69f0161f10 msgr2=0x7f69f00d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:57.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.328+0000 7f69ee000640 1 --2- 172.21.15.159:0/334339371 >> v2:172.21.15.178:3300/0 conn(0x7f69f0161f10 0x7f69f00d19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.328+0000 7f69ee000640 1 -- 172.21.15.159:0/334339371 >> v1:172.21.15.178:6789/0 conn(0x7f69f006da00 legacy=0x7f69f00d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:57.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.328+0000 7f69ee000640 1 -- 172.21.15.159:0/334339371 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f00d11e0 con 0x7f69f00b3270 2024-04-04T06:12:57.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.330+0000 7f69ee000640 1 --2- 172.21.15.159:0/334339371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69f00b3270 0x7f69f00d4b10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f69f00b6fc0 tx=0x7f69e002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378a80a3eddf61a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:57.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.330+0000 7f69ed600640 1 -- 172.21.15.159:0/334339371 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e002ca80 con 0x7f69f00b3270 2024-04-04T06:12:57.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.330+0000 7f69ed600640 1 -- 172.21.15.159:0/334339371 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e002cc20 con 0x7f69f00b3270 2024-04-04T06:12:57.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.330+0000 7f69ed600640 1 -- 172.21.15.159:0/334339371 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e00343f0 con 0x7f69f00b3270 2024-04-04T06:12:57.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.331+0000 7f69f5c00640 1 -- 172.21.15.159:0/334339371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69f00b3270 msgr2=0x7f69f00d4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:57.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.331+0000 7f69f5c00640 1 --2- 172.21.15.159:0/334339371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69f00b3270 0x7f69f00d4b10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f69f00b6fc0 tx=0x7f69e002bee0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.331+0000 7f69f5c00640 1 -- 172.21.15.159:0/334339371 shutdown_connections 2024-04-04T06:12:57.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.331+0000 7f69f5c00640 1 --2- 172.21.15.159:0/334339371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69f00b3270 0x7f69f00d4b10 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.331+0000 7f69f5c00640 1 --2- 172.21.15.159:0/334339371 >> v2:172.21.15.178:3300/0 conn(0x7f69f0161f10 0x7f69f00d19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.331+0000 7f69f5c00640 1 -- 172.21.15.159:0/334339371 >> 172.21.15.159:0/334339371 conn(0x7f69f008bfb0 msgr2=0x7f69f00d4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:57.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.331+0000 7f69f5c00640 1 -- 172.21.15.159:0/334339371 shutdown_connections 2024-04-04T06:12:57.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.331+0000 7f69f5c00640 1 -- 172.21.15.159:0/334339371 wait complete. 2024-04-04T06:12:57.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.334+0000 7f69f5c00640 1 Processor -- start 2024-04-04T06:12:57.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.334+0000 7f69f5c00640 1 -- start start 2024-04-04T06:12:57.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.336+0000 7f69f5c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69f006da00 0x7f69f01f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.336+0000 7f69f5c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69f00b3270 0x7f69f015c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.336+0000 7f69f5c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69f0161f10 0x7f69f015ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.336+0000 7f69f5c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f69f006cff0 con 0x7f69f00b3270 2024-04-04T06:12:57.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.336+0000 7f69f5c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f69f01622f0 con 0x7f69f006da00 2024-04-04T06:12:57.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.336+0000 7f69f5c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f69f006de70 con 0x7f69f0161f10 2024-04-04T06:12:57.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.336+0000 7f69ee000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69f00b3270 0x7f69f015c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.336+0000 7f69ee000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69f00b3270 0x7f69f015c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35656/0 (socket says 172.21.15.159:35656) 2024-04-04T06:12:57.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.336+0000 7f69ee000640 1 -- 172.21.15.159:0/1220769817 learned_addr learned my addr 172.21.15.159:0/1220769817 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:57.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.336+0000 7f69eea00640 1 --2- 172.21.15.159:0/1220769817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69f006da00 0x7f69f01f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.336+0000 7f69ee000640 1 -- 172.21.15.159:0/1220769817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69f0161f10 msgr2=0x7f69f015ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:57.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.336+0000 7f69ee000640 1 --2- 172.21.15.159:0/1220769817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69f0161f10 0x7f69f015ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.336+0000 7f69ee000640 1 -- 172.21.15.159:0/1220769817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69f006da00 msgr2=0x7f69f01f3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:57.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.336+0000 7f69ee000640 1 --2- 172.21.15.159:0/1220769817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69f006da00 0x7f69f01f3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.336+0000 7f69ee000640 1 -- 172.21.15.159:0/1220769817 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f00c91d0 con 0x7f69f00b3270 2024-04-04T06:12:57.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.337+0000 7f69ee000640 1 --2- 172.21.15.159:0/1220769817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69f00b3270 0x7f69f015c4e0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f69e0000f80 tx=0x7f69e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:57.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.337+0000 7f69e6a00640 1 -- 172.21.15.159:0/1220769817 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e0043020 con 0x7f69f00b3270 2024-04-04T06:12:57.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.337+0000 7f69e6a00640 1 -- 172.21.15.159:0/1220769817 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e003e450 con 0x7f69f00b3270 2024-04-04T06:12:57.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.337+0000 7f69e6a00640 1 -- 172.21.15.159:0/1220769817 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e0009510 con 0x7f69f00b3270 2024-04-04T06:12:57.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.337+0000 7f69f5c00640 1 -- 172.21.15.159:0/1220769817 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f00c94b0 con 0x7f69f00b3270 2024-04-04T06:12:57.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.337+0000 7f69f5c00640 1 -- 172.21.15.159:0/1220769817 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f00c9f90 con 0x7f69f00b3270 2024-04-04T06:12:57.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.339+0000 7f69e6a00640 1 -- 172.21.15.159:0/1220769817 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f69e0046020 con 0x7f69f00b3270 2024-04-04T06:12:57.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.340+0000 7f69e6a00640 1 --2- 172.21.15.159:0/1220769817 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f69cc06c8f0 0x7f69cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.340+0000 7f69e6a00640 1 -- 172.21.15.159:0/1220769817 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f69e00ae420 con 0x7f69f00b3270 2024-04-04T06:12:57.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.341+0000 7f69eea00640 1 --2- 172.21.15.159:0/1220769817 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f69cc06c8f0 0x7f69cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.342+0000 7f69eea00640 1 --2- 172.21.15.159:0/1220769817 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f69cc06c8f0 0x7f69cc06edb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f69d8006fd0 tx=0x7f69d8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:57.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.343+0000 7f69f89b0740 1 Processor -- start 2024-04-04T06:12:57.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.343+0000 7f69f89b0740 1 -- start start 2024-04-04T06:12:57.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.345+0000 7f69f89b0740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560d02c9ba70 0x560d02c6ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.345+0000 7f69f89b0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d02c9be50 0x560d02c9e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.345+0000 7f69f89b0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c6ef50 0x560d02c9ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.345+0000 7f69f89b0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560d02bd8350 con 0x560d02c6ef50 2024-04-04T06:12:57.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.345+0000 7f69f89b0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560d02b80530 con 0x560d02c9be50 2024-04-04T06:12:57.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.345+0000 7f69f89b0740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x560d02bd4300 con 0x560d02c9ba70 2024-04-04T06:12:57.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.345+0000 7f69ef400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c6ef50 0x560d02c9ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.345+0000 7f69ef400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c6ef50 0x560d02c9ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35672/0 (socket says 172.21.15.159:35672) 2024-04-04T06:12:57.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.345+0000 7f69ef400640 1 -- 172.21.15.159:0/363232154 learned_addr learned my addr 172.21.15.159:0/363232154 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:57.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.345+0000 7f69ee000640 1 --2- 172.21.15.159:0/363232154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d02c9be50 0x560d02c9e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.345+0000 7f69eea00640 1 --2- 172.21.15.159:0/363232154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560d02c9ba70 0x560d02c6ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.345+0000 7f69ef400640 1 -- 172.21.15.159:0/363232154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560d02c9ba70 msgr2=0x560d02c6ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:57.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.345+0000 7f69ef400640 1 --2- 172.21.15.159:0/363232154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560d02c9ba70 0x560d02c6ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.345+0000 7f69ef400640 1 -- 172.21.15.159:0/363232154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d02c9be50 msgr2=0x560d02c9e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:57.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.345+0000 7f69ef400640 1 --2- 172.21.15.159:0/363232154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d02c9be50 0x560d02c9e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.345+0000 7f69ef400640 1 -- 172.21.15.159:0/363232154 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d02b971d0 con 0x560d02c6ef50 2024-04-04T06:12:57.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.345+0000 7f69ee000640 1 --2- 172.21.15.159:0/363232154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d02c9be50 0x560d02c9e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:57.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.346+0000 7f69ef400640 1 --2- 172.21.15.159:0/363232154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c6ef50 0x560d02c9ed40 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f69e8001610 tx=0x7f69e8001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b55eb551e438761 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:57.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.346+0000 7f69df400640 1 -- 172.21.15.159:0/363232154 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8017070 con 0x560d02c6ef50 2024-04-04T06:12:57.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.346+0000 7f69df400640 1 -- 172.21.15.159:0/363232154 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e8001d20 con 0x560d02c6ef50 2024-04-04T06:12:57.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.346+0000 7f69df400640 1 -- 172.21.15.159:0/363232154 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e800c900 con 0x560d02c6ef50 2024-04-04T06:12:57.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.347+0000 7f69f89b0740 1 -- 172.21.15.159:0/363232154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c6ef50 msgr2=0x560d02c9ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:57.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.347+0000 7f69f89b0740 1 --2- 172.21.15.159:0/363232154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c6ef50 0x560d02c9ed40 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f69e8001610 tx=0x7f69e8001ad0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.347+0000 7f69f89b0740 1 -- 172.21.15.159:0/363232154 shutdown_connections 2024-04-04T06:12:57.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.347+0000 7f69f89b0740 1 --2- 172.21.15.159:0/363232154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c6ef50 0x560d02c9ed40 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.347+0000 7f69f89b0740 1 --2- 172.21.15.159:0/363232154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d02c9be50 0x560d02c9e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.347+0000 7f69f89b0740 1 --2- 172.21.15.159:0/363232154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560d02c9ba70 0x560d02c6ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.347+0000 7f69f89b0740 1 -- 172.21.15.159:0/363232154 >> 172.21.15.159:0/363232154 conn(0x560d02c70c80 msgr2=0x560d02cb9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:57.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.347+0000 7f69f89b0740 1 -- 172.21.15.159:0/363232154 shutdown_connections 2024-04-04T06:12:57.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.347+0000 7f69f89b0740 1 -- 172.21.15.159:0/363232154 wait complete. 2024-04-04T06:12:57.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.348+0000 7f69f89b0740 1 -- start start 2024-04-04T06:12:57.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.348+0000 7f69f89b0740 1 Processor -- start 2024-04-04T06:12:57.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.349+0000 7f69f89b0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c9ba70 0x560d02c931b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.349+0000 7f69f89b0740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560d02c9be50 0x560d02c9dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.349+0000 7f69f89b0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d02c938b0 0x560d02c95cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.349+0000 7f69f89b0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560d02c73220 con 0x560d02c9ba70 2024-04-04T06:12:57.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.349+0000 7f69f89b0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560d02b938f0 con 0x560d02c938b0 2024-04-04T06:12:57.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.349+0000 7f69f89b0740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x560d02c72610 con 0x560d02c9be50 2024-04-04T06:12:57.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.349+0000 7f69f89b0740 10 client.0 _ll_register_callbacks cb 0x560d02b980c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:12:57.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.349+0000 7f69eea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c9ba70 0x560d02c931b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.349+0000 7f69eea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c9ba70 0x560d02c931b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35680/0 (socket says 172.21.15.159:35680) 2024-04-04T06:12:57.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.349+0000 7f69eea00640 1 -- 172.21.15.159:0/2145681067 learned_addr learned my addr 172.21.15.159:0/2145681067 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:57.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.350+0000 7f69eea00640 1 -- 172.21.15.159:0/2145681067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560d02c9be50 msgr2=0x560d02c9dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:57.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.350+0000 7f69eea00640 1 --2- 172.21.15.159:0/2145681067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560d02c9be50 0x560d02c9dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.350+0000 7f69eea00640 1 -- 172.21.15.159:0/2145681067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d02c938b0 msgr2=0x560d02c95cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:57.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.350+0000 7f69eea00640 1 --2- 172.21.15.159:0/2145681067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d02c938b0 0x560d02c95cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.350+0000 7f69eea00640 1 -- 172.21.15.159:0/2145681067 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d029b87b0 con 0x560d02c9ba70 2024-04-04T06:12:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.350+0000 7f69eea00640 1 --2- 172.21.15.159:0/2145681067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c9ba70 0x560d02c931b0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f69d800ed00 tx=0x7f69d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ace2fbbc22f5eb server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.350+0000 7f69d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:12:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.350+0000 7f69d3e00640 1 -- 172.21.15.159:0/2145681067 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d800f040 con 0x560d02c9ba70 2024-04-04T06:12:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.350+0000 7f69d3e00640 1 -- 172.21.15.159:0/2145681067 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f69d8015dd0 con 0x560d02c9ba70 2024-04-04T06:12:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.350+0000 7f69f89b0740 1 -- 172.21.15.159:0/2145681067 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560d02be5890 con 0x560d02c9ba70 2024-04-04T06:12:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.351+0000 7f69d3e00640 1 -- 172.21.15.159:0/2145681067 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f69d802d050 con 0x560d02c9ba70 2024-04-04T06:12:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.351+0000 7f69f89b0740 10 client.7602 fetch_fsmap learned FSMap version 68 2024-04-04T06:12:57.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.351+0000 7f69f89b0740 1 -- 172.21.15.159:0/2145681067 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x560d02a18f40 con 0x560d02c9ba70 2024-04-04T06:12:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.351+0000 7f69d3e00640 1 -- 172.21.15.159:0/2145681067 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x560d02be5890 con 0x560d02c9ba70 2024-04-04T06:12:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.351+0000 7f69d3e00640 1 -- 172.21.15.159:0/2145681067 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f69d8030020 con 0x560d02c9ba70 2024-04-04T06:12:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.351+0000 7f69f89b0740 10 client.7602 fetch_fsmap finished waiting for FSMap version 68 2024-04-04T06:12:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.351+0000 7f69f89b0740 10 client.7602 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-04T06:12:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.351+0000 7f69f89b0740 20 client.7602 populate_metadata read hostname 'smithi159' 2024-04-04T06:12:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.351+0000 7f69f89b0740 1 --2- 172.21.15.159:0/2145681067 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x560d02ca1c80 0x560d02ca2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.351+0000 7f69f89b0740 4 client.7602 mds_command: new command op to 7512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:12:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.351+0000 7f69f89b0740 1 -- 172.21.15.159:0/2145681067 --> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d02bdfbe0 con 0x560d02ca1c80 2024-04-04T06:12:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.351+0000 7f69ee000640 1 --2- 172.21.15.159:0/2145681067 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x560d02ca1c80 0x560d02ca2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.352+0000 7f69ee000640 1 --2- 172.21.15.159:0/2145681067 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x560d02ca1c80 0x560d02ca2130 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a5edfef8c06a335 server_cookie=8ee296834086e8f9 in_seq=0 out_seq=0 2024-04-04T06:12:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.352+0000 7f69d3e00640 10 client.7602 ms_handle_connect on v2:172.21.15.159:6833/3796123128 2024-04-04T06:12:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.353+0000 7f69d3e00640 1 -- 172.21.15.159:0/2145681067 <== mds.0 v2:172.21.15.159:6833/3796123128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d02bdfbe0 con 0x560d02ca1c80 2024-04-04T06:12:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.353+0000 7f69d3e00640 10 client.7602 handle_command_reply: tid=0 2024-04-04T06:12:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.353+0000 7f69f89b0740 1 client.7602 shutdown 2024-04-04T06:12:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.353+0000 7f69f89b0740 1 -- 172.21.15.159:0/2145681067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c9ba70 msgr2=0x560d02c931b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.353+0000 7f69f89b0740 1 --2- 172.21.15.159:0/2145681067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c9ba70 0x560d02c931b0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f69d800ed00 tx=0x7f69d8004730 comp rx=0 tx=0).stop 2024-04-04T06:12:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.353+0000 7f69f89b0740 1 -- 172.21.15.159:0/2145681067 shutdown_connections 2024-04-04T06:12:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.353+0000 7f69f89b0740 1 --2- 172.21.15.159:0/2145681067 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x560d02ca1c80 0x560d02ca2130 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.353+0000 7f69f89b0740 1 --2- 172.21.15.159:0/2145681067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d02c938b0 0x560d02c95cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.353+0000 7f69f89b0740 1 --2- 172.21.15.159:0/2145681067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560d02c9be50 0x560d02c9dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.353+0000 7f69f89b0740 1 --2- 172.21.15.159:0/2145681067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c9ba70 0x560d02c931b0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.353+0000 7f69d3e00640 0 client.7602 ms_handle_reset on v2:172.21.15.159:6833/3796123128 2024-04-04T06:12:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.353+0000 7f69f89b0740 1 -- 172.21.15.159:0/2145681067 >> 172.21.15.159:0/2145681067 conn(0x560d02c6ef50 msgr2=0x560d02cbcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.353+0000 7f69f89b0740 1 -- 172.21.15.159:0/2145681067 shutdown_connections 2024-04-04T06:12:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.353+0000 7f69f89b0740 1 -- 172.21.15.159:0/2145681067 wait complete. 2024-04-04T06:12:57.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.353+0000 7f69f89b0740 20 client.7602 trim_cache size 0 max 16384 2024-04-04T06:12:57.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.372+0000 7f69f89b0740 1 Processor -- start 2024-04-04T06:12:57.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.373+0000 7f69f89b0740 1 -- start start 2024-04-04T06:12:57.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.373+0000 7f69f89b0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d02c938b0 0x560d02cdc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.373+0000 7f69f89b0740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560d02c9ba70 0x560d02cdcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.373+0000 7f69f89b0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c9be50 0x560d02c6d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.373+0000 7f69f89b0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f69f0200de0 con 0x560d02c9be50 2024-04-04T06:12:57.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.373+0000 7f69f89b0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f69f01ffe70 con 0x560d02c938b0 2024-04-04T06:12:57.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.373+0000 7f69f89b0740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f69f00b7cb0 con 0x560d02c9ba70 2024-04-04T06:12:57.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.374+0000 7f69ef400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c9be50 0x560d02c6d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.374+0000 7f69ef400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c9be50 0x560d02c6d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35692/0 (socket says 172.21.15.159:35692) 2024-04-04T06:12:57.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.374+0000 7f69ef400640 1 -- 172.21.15.159:0/3584322918 learned_addr learned my addr 172.21.15.159:0/3584322918 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:57.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.374+0000 7f69eea00640 1 --2- 172.21.15.159:0/3584322918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d02c938b0 0x560d02cdc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.374+0000 7f69ef400640 1 -- 172.21.15.159:0/3584322918 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560d02c9ba70 msgr2=0x560d02cdcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:57.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.374+0000 7f69ef400640 1 --2- 172.21.15.159:0/3584322918 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560d02c9ba70 0x560d02cdcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.374+0000 7f69ef400640 1 -- 172.21.15.159:0/3584322918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d02c938b0 msgr2=0x560d02cdc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:57.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.374+0000 7f69ef400640 1 --2- 172.21.15.159:0/3584322918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d02c938b0 0x560d02cdc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.374+0000 7f69ef400640 1 -- 172.21.15.159:0/3584322918 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d028ac0a0 con 0x560d02c9be50 2024-04-04T06:12:57.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.374+0000 7f69ef400640 1 --2- 172.21.15.159:0/3584322918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c9be50 0x560d02c6d8a0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f69e8011000 tx=0x7f69e8009050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02df0c54f7f1970 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:57.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.375+0000 7f69dea00640 1 -- 172.21.15.159:0/3584322918 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8023070 con 0x560d02c9be50 2024-04-04T06:12:57.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.375+0000 7f69dea00640 1 -- 172.21.15.159:0/3584322918 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e8018040 con 0x560d02c9be50 2024-04-04T06:12:57.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.375+0000 7f69dea00640 1 -- 172.21.15.159:0/3584322918 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8017070 con 0x560d02c9be50 2024-04-04T06:12:57.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.375+0000 7f69f89b0740 1 -- 172.21.15.159:0/3584322918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c9be50 msgr2=0x560d02c6d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:57.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.375+0000 7f69f89b0740 1 --2- 172.21.15.159:0/3584322918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c9be50 0x560d02c6d8a0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f69e8011000 tx=0x7f69e8009050 comp rx=0 tx=0).stop 2024-04-04T06:12:57.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.375+0000 7f69f89b0740 1 -- 172.21.15.159:0/3584322918 shutdown_connections 2024-04-04T06:12:57.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.375+0000 7f69f89b0740 1 --2- 172.21.15.159:0/3584322918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c9be50 0x560d02c6d8a0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.375+0000 7f69f89b0740 1 --2- 172.21.15.159:0/3584322918 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560d02c9ba70 0x560d02cdcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.375+0000 7f69f89b0740 1 --2- 172.21.15.159:0/3584322918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d02c938b0 0x560d02cdc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.375+0000 7f69f89b0740 1 -- 172.21.15.159:0/3584322918 >> 172.21.15.159:0/3584322918 conn(0x560d02c6ef50 msgr2=0x560d02c92fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:57.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.375+0000 7f69f89b0740 1 -- 172.21.15.159:0/3584322918 shutdown_connections 2024-04-04T06:12:57.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.375+0000 7f69f89b0740 1 -- 172.21.15.159:0/3584322918 wait complete. 2024-04-04T06:12:57.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.376+0000 7f69f89b0740 1 -- start start 2024-04-04T06:12:57.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.376+0000 7f69f89b0740 1 Processor -- start 2024-04-04T06:12:57.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.377+0000 7f69f89b0740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560d02c938b0 0x560d02c740b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.377+0000 7f69f89b0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c9ba70 0x560d02cdb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.377+0000 7f69f89b0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d02c9be50 0x560d02cbcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.377+0000 7f69f89b0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f69f00b8300 con 0x560d02c9ba70 2024-04-04T06:12:57.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.377+0000 7f69f89b0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560d02bdfbe0 con 0x560d02c9be50 2024-04-04T06:12:57.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.377+0000 7f69f89b0740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x560d02cdc5e0 con 0x560d02c938b0 2024-04-04T06:12:57.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.377+0000 7f69f89b0740 10 client.0 _ll_register_callbacks cb 0x7f69d8006eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:12:57.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.377+0000 7f69ee000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c9ba70 0x560d02cdb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.377+0000 7f69ee000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c9ba70 0x560d02cdb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35708/0 (socket says 172.21.15.159:35708) 2024-04-04T06:12:57.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.377+0000 7f69ee000640 1 -- 172.21.15.159:0/36924712 learned_addr learned my addr 172.21.15.159:0/36924712 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:57.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.378+0000 7f69ee000640 1 -- 172.21.15.159:0/36924712 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560d02c938b0 msgr2=0x560d02c740b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:57.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.378+0000 7f69ee000640 1 --2- 172.21.15.159:0/36924712 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560d02c938b0 0x560d02c740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.378+0000 7f69ef400640 1 --2- 172.21.15.159:0/36924712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d02c9be50 0x560d02cbcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.378+0000 7f69ee000640 1 -- 172.21.15.159:0/36924712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d02c9be50 msgr2=0x560d02cbcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:57.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.378+0000 7f69ee000640 1 --2- 172.21.15.159:0/36924712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d02c9be50 0x560d02cbcd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.378+0000 7f69ee000640 1 -- 172.21.15.159:0/36924712 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d02c94a60 con 0x560d02c9ba70 2024-04-04T06:12:57.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.378+0000 7f69ee000640 1 --2- 172.21.15.159:0/36924712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c9ba70 0x560d02cdb9d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f69e007c070 tx=0x7f69e0078410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca33a444e004454 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:57.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.378+0000 7f69d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:12:57.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.378+0000 7f69d3e00640 1 -- 172.21.15.159:0/36924712 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e0094030 con 0x560d02c9ba70 2024-04-04T06:12:57.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.378+0000 7f69d3e00640 1 -- 172.21.15.159:0/36924712 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f69e0080160 con 0x560d02c9ba70 2024-04-04T06:12:57.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.378+0000 7f69f89b0740 1 -- 172.21.15.159:0/36924712 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69e8018040 con 0x560d02c9ba70 2024-04-04T06:12:57.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.379+0000 7f69d3e00640 1 -- 172.21.15.159:0/36924712 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f69e0084cc0 con 0x560d02c9ba70 2024-04-04T06:12:57.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.379+0000 7f69f89b0740 10 client.7608 fetch_fsmap learned FSMap version 68 2024-04-04T06:12:57.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.379+0000 7f69f89b0740 1 -- 172.21.15.159:0/36924712 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x560d02ca2270 con 0x560d02c9ba70 2024-04-04T06:12:57.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.379+0000 7f69d3e00640 1 -- 172.21.15.159:0/36924712 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e8018040 con 0x560d02c9ba70 2024-04-04T06:12:57.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.379+0000 7f69d3e00640 1 -- 172.21.15.159:0/36924712 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f69e0078750 con 0x560d02c9ba70 2024-04-04T06:12:57.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.379+0000 7f69f89b0740 10 client.7608 fetch_fsmap finished waiting for FSMap version 68 2024-04-04T06:12:57.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.379+0000 7f69f89b0740 10 client.7608 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-04T06:12:57.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.379+0000 7f69f89b0740 20 client.7608 populate_metadata read hostname 'smithi159' 2024-04-04T06:12:57.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.379+0000 7f69f89b0740 1 --2- 172.21.15.159:0/36924712 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x560d02cdbf10 0x560d02cbd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.379+0000 7f69f89b0740 4 client.7608 mds_command: new command op to 7512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-04-04T06:12:57.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.379+0000 7f69f89b0740 1 -- 172.21.15.159:0/36924712 --> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) -- 0x560d02b5d6a0 con 0x560d02cdbf10 2024-04-04T06:12:57.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.379+0000 7f69eea00640 1 --2- 172.21.15.159:0/36924712 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x560d02cdbf10 0x560d02cbd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.380+0000 7f69eea00640 1 --2- 172.21.15.159:0/36924712 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x560d02cdbf10 0x560d02cbd250 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dcd0619e7656e83 server_cookie=d1506fc7aa76cd5c in_seq=0 out_seq=0 2024-04-04T06:12:57.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.380+0000 7f69d3e00640 10 client.7608 ms_handle_connect on v2:172.21.15.159:6833/3796123128 2024-04-04T06:12:57.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.380+0000 7f69d3e00640 1 -- 172.21.15.159:0/36924712 <== mds.0 v2:172.21.15.159:6833/3796123128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560d02b5d6a0 con 0x560d02cdbf10 2024-04-04T06:12:57.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.380+0000 7f69d3e00640 10 client.7608 handle_command_reply: tid=0 2024-04-04T06:12:57.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.380+0000 7f69f89b0740 1 client.7608 shutdown 2024-04-04T06:12:57.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.381+0000 7f69f89b0740 1 -- 172.21.15.159:0/36924712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c9ba70 msgr2=0x560d02cdb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:57.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.381+0000 7f69f89b0740 1 --2- 172.21.15.159:0/36924712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c9ba70 0x560d02cdb9d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f69e007c070 tx=0x7f69e0078410 comp rx=0 tx=0).stop 2024-04-04T06:12:57.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.381+0000 7f69f89b0740 1 -- 172.21.15.159:0/36924712 shutdown_connections 2024-04-04T06:12:57.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.381+0000 7f69f89b0740 1 --2- 172.21.15.159:0/36924712 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x560d02cdbf10 0x560d02cbd250 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.381+0000 7f69f89b0740 1 --2- 172.21.15.159:0/36924712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560d02c9be50 0x560d02cbcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.381+0000 7f69f89b0740 1 --2- 172.21.15.159:0/36924712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d02c9ba70 0x560d02cdb9d0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.381+0000 7f69f89b0740 1 --2- 172.21.15.159:0/36924712 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x560d02c938b0 0x560d02c740b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.381+0000 7f69f89b0740 1 -- 172.21.15.159:0/36924712 >> 172.21.15.159:0/36924712 conn(0x560d02c6ef50 msgr2=0x560d02c6d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:57.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.381+0000 7f69d3e00640 0 client.7608 ms_handle_reset on v2:172.21.15.159:6833/3796123128 2024-04-04T06:12:57.384 INFO:tasks.ceph.mds.a.smithi159.stderr:2024-04-04T06:12:57.381+0000 7fb383a00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-04-04T06:12:45.034063+0000 1=1+0) n(v0 rc2024-04-04T06:12:45.181060+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x55728e21c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-04T06:12:45.034063+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-04T06:12:45.181060+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-04T06:12:57.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.381+0000 7f69f89b0740 1 -- 172.21.15.159:0/36924712 shutdown_connections 2024-04-04T06:12:57.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.381+0000 7f69f89b0740 1 -- 172.21.15.159:0/36924712 wait complete. 2024-04-04T06:12:57.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.381+0000 7f69f89b0740 20 client.7608 trim_cache size 0 max 16384 2024-04-04T06:12:57.385 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:12:57.385 INFO:teuthology.orchestra.run.smithi159.stdout: "return_code": 0, 2024-04-04T06:12:57.385 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_tag": "f36edbfc-f5de-4ac5-8e8b-527a6a681789", 2024-04-04T06:12:57.385 INFO:teuthology.orchestra.run.smithi159.stdout: "mode": "asynchronous" 2024-04-04T06:12:57.385 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:12:57.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.382+0000 7f69f5c00640 1 -- 172.21.15.159:0/1220769817 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f69cc06c8f0 msgr2=0x7f69cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:57.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.382+0000 7f69f5c00640 1 --2- 172.21.15.159:0/1220769817 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f69cc06c8f0 0x7f69cc06edb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f69d8006fd0 tx=0x7f69d8008040 comp rx=0 tx=0).stop 2024-04-04T06:12:57.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.382+0000 7f69f5c00640 1 -- 172.21.15.159:0/1220769817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69f00b3270 msgr2=0x7f69f015c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:57.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.382+0000 7f69f5c00640 1 --2- 172.21.15.159:0/1220769817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69f00b3270 0x7f69f015c4e0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f69e0000f80 tx=0x7f69e0004a40 comp rx=0 tx=0).stop 2024-04-04T06:12:57.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.382+0000 7f69f5c00640 1 -- 172.21.15.159:0/1220769817 shutdown_connections 2024-04-04T06:12:57.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.382+0000 7f69f5c00640 1 --2- 172.21.15.159:0/1220769817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f69f0161f10 0x7f69f015ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.382+0000 7f69f5c00640 1 --2- 172.21.15.159:0/1220769817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69f00b3270 0x7f69f015c4e0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.382+0000 7f69f5c00640 1 --2- 172.21.15.159:0/1220769817 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f69cc06c8f0 0x7f69cc06edb0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.382+0000 7f69f5c00640 1 --2- 172.21.15.159:0/1220769817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69f006da00 0x7f69f01f3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.382+0000 7f69f5c00640 1 -- 172.21.15.159:0/1220769817 >> 172.21.15.159:0/1220769817 conn(0x7f69f008bfb0 msgr2=0x7f69f0162d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:57.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.382+0000 7f69f5c00640 1 -- 172.21.15.159:0/1220769817 shutdown_connections 2024-04-04T06:12:57.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.382+0000 7f69f5c00640 1 -- 172.21.15.159:0/1220769817 wait complete. 2024-04-04T06:12:57.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:12:57.392 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-04T06:12:57.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.594+0000 7fb7a1200640 1 Processor -- start 2024-04-04T06:12:57.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.594+0000 7fb7a1200640 1 -- start start 2024-04-04T06:12:57.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.595+0000 7fb7a1200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb79c070120 0x7fb79c06b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.596+0000 7fb7a1200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb79c06e770 0x7fb79c06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.596+0000 7fb7a1200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb79c0b5b10 con 0x7fb79c0b0670 2024-04-04T06:12:57.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.596+0000 7fb7a1200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fb79c0b63e0 con 0x7fb79c06e770 2024-04-04T06:12:57.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.596+0000 7fb7a1200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb79c06c3f0 con 0x7fb79c070120 2024-04-04T06:12:57.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.596+0000 7fb79a800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb79c06e770 0x7fb79c06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.596+0000 7fb79a800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb79c06e770 0x7fb79c06bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35718/0 (socket says 172.21.15.159:35718) 2024-04-04T06:12:57.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.596+0000 7fb79a800640 1 -- 172.21.15.159:0/1722651489 learned_addr learned my addr 172.21.15.159:0/1722651489 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:57.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.596+0000 7fb791400640 1 --2- 172.21.15.159:0/1722651489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb79c070120 0x7fb79c06b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.597+0000 7fb79a800640 1 -- 172.21.15.159:0/1722651489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb79c070120 msgr2=0x7fb79c06b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:57.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.597+0000 7fb79a800640 1 --2- 172.21.15.159:0/1722651489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb79c070120 0x7fb79c06b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.597+0000 7fb79a800640 1 -- 172.21.15.159:0/1722651489 >> v1:172.21.15.178:6789/0 conn(0x7fb79c0b0670 legacy=0x7fb79c15f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:57.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.597+0000 7fb79a800640 1 -- 172.21.15.159:0/1722651489 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79c0705e0 con 0x7fb79c06e770 2024-04-04T06:12:57.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.599+0000 7fb79a800640 1 --2- 172.21.15.159:0/1722651489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb79c06e770 0x7fb79c06bab0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fb794009e80 tx=0x7fb79402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7c7de0bc69a495 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:57.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.599+0000 7fb799400640 1 -- 172.21.15.159:0/1722651489 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79402caf0 con 0x7fb79c06e770 2024-04-04T06:12:57.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.599+0000 7fb799400640 1 -- 172.21.15.159:0/1722651489 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb79402cc90 con 0x7fb79c06e770 2024-04-04T06:12:57.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.599+0000 7fb799400640 1 -- 172.21.15.159:0/1722651489 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb794034610 con 0x7fb79c06e770 2024-04-04T06:12:57.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.600+0000 7fb7a1200640 1 -- 172.21.15.159:0/1722651489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb79c06e770 msgr2=0x7fb79c06bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:57.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.600+0000 7fb7a1200640 1 --2- 172.21.15.159:0/1722651489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb79c06e770 0x7fb79c06bab0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fb794009e80 tx=0x7fb79402bf50 comp rx=0 tx=0).stop 2024-04-04T06:12:57.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.600+0000 7fb7a1200640 1 -- 172.21.15.159:0/1722651489 shutdown_connections 2024-04-04T06:12:57.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.600+0000 7fb7a1200640 1 --2- 172.21.15.159:0/1722651489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb79c06e770 0x7fb79c06bab0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.600+0000 7fb7a1200640 1 --2- 172.21.15.159:0/1722651489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb79c070120 0x7fb79c06b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.600+0000 7fb7a1200640 1 -- 172.21.15.159:0/1722651489 >> 172.21.15.159:0/1722651489 conn(0x7fb79c0b65a0 msgr2=0x7fb79c0b69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:57.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.600+0000 7fb7a1200640 1 -- 172.21.15.159:0/1722651489 shutdown_connections 2024-04-04T06:12:57.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.600+0000 7fb7a1200640 1 -- 172.21.15.159:0/1722651489 wait complete. 2024-04-04T06:12:57.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.602+0000 7fb7a1200640 1 Processor -- start 2024-04-04T06:12:57.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.602+0000 7fb7a1200640 1 -- start start 2024-04-04T06:12:57.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.603+0000 7fb7a1200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb79c06e770 0x7fb79c14fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.603+0000 7fb7a1200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb79c070120 0x7fb79c161150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.603+0000 7fb7a1200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb79c0b0670 0x7fb79c157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.603+0000 7fb7a1200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c0b0a50 con 0x7fb79c070120 2024-04-04T06:12:57.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.603+0000 7fb7a1200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c0b6070 con 0x7fb79c0b0670 2024-04-04T06:12:57.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.603+0000 7fb7a1200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb79c168450 con 0x7fb79c06e770 2024-04-04T06:12:57.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.603+0000 7fb791400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb79c070120 0x7fb79c161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.603+0000 7fb791400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb79c070120 0x7fb79c161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35724/0 (socket says 172.21.15.159:35724) 2024-04-04T06:12:57.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.603+0000 7fb791400640 1 -- 172.21.15.159:0/2332943523 learned_addr learned my addr 172.21.15.159:0/2332943523 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:57.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.604+0000 7fb79a800640 1 --2- 172.21.15.159:0/2332943523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb79c0b0670 0x7fb79c157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.604+0000 7fb791400640 1 -- 172.21.15.159:0/2332943523 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb79c06e770 msgr2=0x7fb79c14fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:57.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.604+0000 7fb799e00640 1 --2- 172.21.15.159:0/2332943523 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb79c06e770 0x7fb79c14fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.604+0000 7fb791400640 1 --2- 172.21.15.159:0/2332943523 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb79c06e770 0x7fb79c14fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.604+0000 7fb791400640 1 -- 172.21.15.159:0/2332943523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb79c0b0670 msgr2=0x7fb79c157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:57.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.604+0000 7fb791400640 1 --2- 172.21.15.159:0/2332943523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb79c0b0670 0x7fb79c157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.604+0000 7fb791400640 1 -- 172.21.15.159:0/2332943523 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79c1500a0 con 0x7fb79c070120 2024-04-04T06:12:57.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.604+0000 7fb79a800640 1 --2- 172.21.15.159:0/2332943523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb79c0b0670 0x7fb79c157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:57.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.604+0000 7fb799e00640 1 --2- 172.21.15.159:0/2332943523 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb79c06e770 0x7fb79c14fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:57.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.604+0000 7fb791400640 1 --2- 172.21.15.159:0/2332943523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb79c070120 0x7fb79c161150 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fb78400e9b0 tx=0x7fb78400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:57.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.604+0000 7fb792a00640 1 -- 172.21.15.159:0/2332943523 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784017070 con 0x7fb79c070120 2024-04-04T06:12:57.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.604+0000 7fb792a00640 1 -- 172.21.15.159:0/2332943523 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb78400c9a0 con 0x7fb79c070120 2024-04-04T06:12:57.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.604+0000 7fb7a1200640 1 -- 172.21.15.159:0/2332943523 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb79c150380 con 0x7fb79c070120 2024-04-04T06:12:57.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.605+0000 7fb792a00640 1 -- 172.21.15.159:0/2332943523 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78400cd90 con 0x7fb79c070120 2024-04-04T06:12:57.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.605+0000 7fb7a1200640 1 -- 172.21.15.159:0/2332943523 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb79c150c20 con 0x7fb79c070120 2024-04-04T06:12:57.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.607+0000 7fb792a00640 1 -- 172.21.15.159:0/2332943523 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb78401a070 con 0x7fb79c070120 2024-04-04T06:12:57.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.608+0000 7fb792a00640 1 --2- 172.21.15.159:0/2332943523 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb77806c8f0 0x7fb77806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.608+0000 7fb792a00640 1 -- 172.21.15.159:0/2332943523 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7fb78408bfe0 con 0x7fb79c070120 2024-04-04T06:12:57.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.608+0000 7fb799e00640 1 --2- 172.21.15.159:0/2332943523 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb77806c8f0 0x7fb77806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.609+0000 7fb7a3fb3740 1 Processor -- start 2024-04-04T06:12:57.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.610+0000 7fb799e00640 1 --2- 172.21.15.159:0/2332943523 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb77806c8f0 0x7fb77806edb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb78c000f60 tx=0x7fb78c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:57.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.610+0000 7fb7a3fb3740 1 -- start start 2024-04-04T06:12:57.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.611+0000 7fb7a3fb3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88d4580 0x55a9a88a7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.611+0000 7fb7a3fb3740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88d4960 0x55a9a88d7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.611+0000 7fb7a3fb3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9a88a7a60 0x55a9a88d7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.611+0000 7fb7a3fb3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a9a88126d0 con 0x55a9a88d4580 2024-04-04T06:12:57.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.611+0000 7fb7a3fb3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a9a87ba770 con 0x55a9a88a7a60 2024-04-04T06:12:57.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.611+0000 7fb7a3fb3740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a9a880e680 con 0x55a9a88d4960 2024-04-04T06:12:57.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.611+0000 7fb799e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88d4580 0x55a9a88a7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.611+0000 7fb799e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88d4580 0x55a9a88a7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35738/0 (socket says 172.21.15.159:35738) 2024-04-04T06:12:57.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.611+0000 7fb799e00640 1 -- 172.21.15.159:0/3917801502 learned_addr learned my addr 172.21.15.159:0/3917801502 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:57.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.612+0000 7fb79a800640 1 --2- 172.21.15.159:0/3917801502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9a88a7a60 0x55a9a88d7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.612+0000 7fb791400640 1 --2- 172.21.15.159:0/3917801502 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88d4960 0x55a9a88d7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.612+0000 7fb799e00640 1 -- 172.21.15.159:0/3917801502 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88d4960 msgr2=0x55a9a88d7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:57.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.612+0000 7fb799e00640 1 --2- 172.21.15.159:0/3917801502 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88d4960 0x55a9a88d7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.612+0000 7fb799e00640 1 -- 172.21.15.159:0/3917801502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9a88a7a60 msgr2=0x55a9a88d7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:57.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.612+0000 7fb799e00640 1 --2- 172.21.15.159:0/3917801502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9a88a7a60 0x55a9a88d7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.612+0000 7fb799e00640 1 -- 172.21.15.159:0/3917801502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9a85f07d0 con 0x55a9a88d4580 2024-04-04T06:12:57.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.612+0000 7fb799e00640 1 --2- 172.21.15.159:0/3917801502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88d4580 0x55a9a88a7520 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00dac0 tx=0x7fb78c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e237ec1805469c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:57.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.612+0000 7fb78aa00640 1 -- 172.21.15.159:0/3917801502 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c024070 con 0x55a9a88d4580 2024-04-04T06:12:57.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.612+0000 7fb78aa00640 1 -- 172.21.15.159:0/3917801502 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb78c014ba0 con 0x55a9a88d4580 2024-04-04T06:12:57.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.612+0000 7fb78aa00640 1 -- 172.21.15.159:0/3917801502 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c01f400 con 0x55a9a88d4580 2024-04-04T06:12:57.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.613+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/3917801502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88d4580 msgr2=0x55a9a88a7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:57.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.613+0000 7fb7a3fb3740 1 --2- 172.21.15.159:0/3917801502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88d4580 0x55a9a88a7520 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00dac0 tx=0x7fb78c010000 comp rx=0 tx=0).stop 2024-04-04T06:12:57.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.613+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/3917801502 shutdown_connections 2024-04-04T06:12:57.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.613+0000 7fb7a3fb3740 1 --2- 172.21.15.159:0/3917801502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9a88a7a60 0x55a9a88d7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.613+0000 7fb7a3fb3740 1 --2- 172.21.15.159:0/3917801502 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88d4960 0x55a9a88d7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.613+0000 7fb7a3fb3740 1 --2- 172.21.15.159:0/3917801502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88d4580 0x55a9a88a7520 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.614+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/3917801502 >> 172.21.15.159:0/3917801502 conn(0x55a9a88a9790 msgr2=0x55a9a88f2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:57.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.614+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/3917801502 shutdown_connections 2024-04-04T06:12:57.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.614+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/3917801502 wait complete. 2024-04-04T06:12:57.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.615+0000 7fb7a3fb3740 1 -- start start 2024-04-04T06:12:57.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.615+0000 7fb7a3fb3740 1 Processor -- start 2024-04-04T06:12:57.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.617+0000 7fb7a3fb3740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88d4580 0x55a9a8903510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.617+0000 7fb7a3fb3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9a88d4960 0x55a9a88cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.617+0000 7fb7a3fb3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88cc5b0 0x55a9a88cca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.617+0000 7fb7a3fb3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a9a88abe30 con 0x55a9a88cc5b0 2024-04-04T06:12:57.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.617+0000 7fb7a3fb3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a9a87cdaa0 con 0x55a9a88d4960 2024-04-04T06:12:57.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.617+0000 7fb7a3fb3740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a9a88ae670 con 0x55a9a88d4580 2024-04-04T06:12:57.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.617+0000 7fb7a3fb3740 10 client.0 _ll_register_callbacks cb 0x55a9a87cd490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:12:57.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.617+0000 7fb79a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88cc5b0 0x55a9a88cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.617+0000 7fb79a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88cc5b0 0x55a9a88cca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35744/0 (socket says 172.21.15.159:35744) 2024-04-04T06:12:57.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.617+0000 7fb79a800640 1 -- 172.21.15.159:0/4135318773 learned_addr learned my addr 172.21.15.159:0/4135318773 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:57.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.617+0000 7fb799e00640 1 --2- 172.21.15.159:0/4135318773 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88d4580 0x55a9a8903510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.617+0000 7fb791400640 1 --2- 172.21.15.159:0/4135318773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9a88d4960 0x55a9a88cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.618+0000 7fb79a800640 1 -- 172.21.15.159:0/4135318773 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88d4580 msgr2=0x55a9a8903510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:57.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.618+0000 7fb79a800640 1 --2- 172.21.15.159:0/4135318773 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88d4580 0x55a9a8903510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.618+0000 7fb79a800640 1 -- 172.21.15.159:0/4135318773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9a88d4960 msgr2=0x55a9a88cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:57.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.618+0000 7fb79a800640 1 --2- 172.21.15.159:0/4135318773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9a88d4960 0x55a9a88cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.618+0000 7fb79a800640 1 -- 172.21.15.159:0/4135318773 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9a85263f0 con 0x55a9a88cc5b0 2024-04-04T06:12:57.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.618+0000 7fb799e00640 1 --2- 172.21.15.159:0/4135318773 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88d4580 0x55a9a8903510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:12:57.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.618+0000 7fb79a800640 1 --2- 172.21.15.159:0/4135318773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88cc5b0 0x55a9a88cca30 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fb79402bf90 tx=0x7fb794004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2eddc35507d7af server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:57.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.618+0000 7fb77f400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:12:57.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.618+0000 7fb77f400640 1 -- 172.21.15.159:0/4135318773 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb794009510 con 0x55a9a88cc5b0 2024-04-04T06:12:57.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.618+0000 7fb77f400640 1 -- 172.21.15.159:0/4135318773 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7fb794034b30 con 0x55a9a88cc5b0 2024-04-04T06:12:57.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.618+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/4135318773 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb78c014ba0 con 0x55a9a88cc5b0 2024-04-04T06:12:57.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.619+0000 7fb77f400640 1 -- 172.21.15.159:0/4135318773 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fb794034d30 con 0x55a9a88cc5b0 2024-04-04T06:12:57.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.619+0000 7fb7a3fb3740 10 client.7620 fetch_fsmap learned FSMap version 68 2024-04-04T06:12:57.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.619+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/4135318773 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55a9a84e60f0 con 0x55a9a88cc5b0 2024-04-04T06:12:57.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.619+0000 7fb77f400640 1 -- 172.21.15.159:0/4135318773 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb78c014ba0 con 0x55a9a88cc5b0 2024-04-04T06:12:57.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.619+0000 7fb77f400640 1 -- 172.21.15.159:0/4135318773 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7fb794008040 con 0x55a9a88cc5b0 2024-04-04T06:12:57.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.619+0000 7fb7a3fb3740 10 client.7620 fetch_fsmap finished waiting for FSMap version 68 2024-04-04T06:12:57.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.619+0000 7fb7a3fb3740 10 client.7620 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-04T06:12:57.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.619+0000 7fb7a3fb3740 20 client.7620 populate_metadata read hostname 'smithi159' 2024-04-04T06:12:57.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.619+0000 7fb7a3fb3740 1 --2- 172.21.15.159:0/4135318773 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55a9a8906530 0x55a9a8906940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.619+0000 7fb7a3fb3740 4 client.7620 mds_command: new command op to 7512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:12:57.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.619+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/4135318773 --> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9a881fc10 con 0x55a9a8906530 2024-04-04T06:12:57.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.619+0000 7fb799e00640 1 --2- 172.21.15.159:0/4135318773 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55a9a8906530 0x55a9a8906940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.620+0000 7fb799e00640 1 --2- 172.21.15.159:0/4135318773 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55a9a8906530 0x55a9a8906940 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fdfa7d02988a34e server_cookie=b1a459d4737c7855 in_seq=0 out_seq=0 2024-04-04T06:12:57.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.620+0000 7fb77f400640 10 client.7620 ms_handle_connect on v2:172.21.15.159:6833/3796123128 2024-04-04T06:12:57.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.621+0000 7fb77f400640 1 -- 172.21.15.159:0/4135318773 <== mds.0 v2:172.21.15.159:6833/3796123128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a9a881fc10 con 0x55a9a8906530 2024-04-04T06:12:57.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.621+0000 7fb77f400640 10 client.7620 handle_command_reply: tid=0 2024-04-04T06:12:57.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.621+0000 7fb7a3fb3740 1 client.7620 shutdown 2024-04-04T06:12:57.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.621+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/4135318773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88cc5b0 msgr2=0x55a9a88cca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:57.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.621+0000 7fb7a3fb3740 1 --2- 172.21.15.159:0/4135318773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88cc5b0 0x55a9a88cca30 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fb79402bf90 tx=0x7fb794004ab0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.621+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/4135318773 shutdown_connections 2024-04-04T06:12:57.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.621+0000 7fb7a3fb3740 1 --2- 172.21.15.159:0/4135318773 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55a9a8906530 0x55a9a8906940 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.621+0000 7fb7a3fb3740 1 --2- 172.21.15.159:0/4135318773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88cc5b0 0x55a9a88cca30 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.621+0000 7fb7a3fb3740 1 --2- 172.21.15.159:0/4135318773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9a88d4960 0x55a9a88cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.621+0000 7fb7a3fb3740 1 --2- 172.21.15.159:0/4135318773 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88d4580 0x55a9a8903510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.621+0000 7fb77f400640 0 client.7620 ms_handle_reset on v2:172.21.15.159:6833/3796123128 2024-04-04T06:12:57.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.621+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/4135318773 >> 172.21.15.159:0/4135318773 conn(0x55a9a88a7a60 msgr2=0x55a9a88ab640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:57.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.621+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/4135318773 shutdown_connections 2024-04-04T06:12:57.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.621+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/4135318773 wait complete. 2024-04-04T06:12:57.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.621+0000 7fb7a3fb3740 20 client.7620 trim_cache size 0 max 16384 2024-04-04T06:12:57.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.643+0000 7fb7a3fb3740 1 Processor -- start 2024-04-04T06:12:57.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.643+0000 7fb7a3fb3740 1 -- start start 2024-04-04T06:12:57.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.644+0000 7fb7a3fb3740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88cc5b0 0x55a9a88d4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.644+0000 7fb7a3fb3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9a88d4580 0x55a9a88d52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.644+0000 7fb7a3fb3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88d4960 0x55a9a88cae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.644+0000 7fb7a3fb3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c200d50 con 0x55a9a88d4960 2024-04-04T06:12:57.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.644+0000 7fb7a3fb3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c1ffe90 con 0x55a9a88d4580 2024-04-04T06:12:57.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.644+0000 7fb7a3fb3740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb79c160300 con 0x55a9a88cc5b0 2024-04-04T06:12:57.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.644+0000 7fb79a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88d4960 0x55a9a88cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.644+0000 7fb79a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88d4960 0x55a9a88cae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35754/0 (socket says 172.21.15.159:35754) 2024-04-04T06:12:57.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.644+0000 7fb79a800640 1 -- 172.21.15.159:0/1160018985 learned_addr learned my addr 172.21.15.159:0/1160018985 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:57.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.645+0000 7fb799e00640 1 --2- 172.21.15.159:0/1160018985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88cc5b0 0x55a9a88d4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.645+0000 7fb791400640 1 --2- 172.21.15.159:0/1160018985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9a88d4580 0x55a9a88d52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.645+0000 7fb79a800640 1 -- 172.21.15.159:0/1160018985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88cc5b0 msgr2=0x55a9a88d4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:57.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.645+0000 7fb79a800640 1 --2- 172.21.15.159:0/1160018985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88cc5b0 0x55a9a88d4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.645+0000 7fb79a800640 1 -- 172.21.15.159:0/1160018985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9a88d4580 msgr2=0x55a9a88d52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:57.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.645+0000 7fb79a800640 1 --2- 172.21.15.159:0/1160018985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9a88d4580 0x55a9a88d52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.645+0000 7fb79a800640 1 -- 172.21.15.159:0/1160018985 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9a88a82a0 con 0x55a9a88d4960 2024-04-04T06:12:57.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.645+0000 7fb799e00640 1 --2- 172.21.15.159:0/1160018985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88cc5b0 0x55a9a88d4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:57.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.645+0000 7fb79a800640 1 --2- 172.21.15.159:0/1160018985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88d4960 0x55a9a88cae40 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fb7940072a0 tx=0x7fb79403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82e5021f0686089 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:57.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.645+0000 7fb78a000640 1 -- 172.21.15.159:0/1160018985 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb794058070 con 0x55a9a88d4960 2024-04-04T06:12:57.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.645+0000 7fb78a000640 1 -- 172.21.15.159:0/1160018985 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb794034b30 con 0x55a9a88d4960 2024-04-04T06:12:57.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.646+0000 7fb78a000640 1 -- 172.21.15.159:0/1160018985 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb794054680 con 0x55a9a88d4960 2024-04-04T06:12:57.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.646+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/1160018985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88d4960 msgr2=0x55a9a88cae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:57.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.646+0000 7fb7a3fb3740 1 --2- 172.21.15.159:0/1160018985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88d4960 0x55a9a88cae40 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fb7940072a0 tx=0x7fb79403f000 comp rx=0 tx=0).stop 2024-04-04T06:12:57.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.646+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/1160018985 shutdown_connections 2024-04-04T06:12:57.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.646+0000 7fb7a3fb3740 1 --2- 172.21.15.159:0/1160018985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88d4960 0x55a9a88cae40 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.646+0000 7fb7a3fb3740 1 --2- 172.21.15.159:0/1160018985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9a88d4580 0x55a9a88d52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.646+0000 7fb7a3fb3740 1 --2- 172.21.15.159:0/1160018985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88cc5b0 0x55a9a88d4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.646+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/1160018985 >> 172.21.15.159:0/1160018985 conn(0x55a9a88a7a60 msgr2=0x55a9a88ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:57.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.647+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/1160018985 shutdown_connections 2024-04-04T06:12:57.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.647+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/1160018985 wait complete. 2024-04-04T06:12:57.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.648+0000 7fb7a3fb3740 1 -- start start 2024-04-04T06:12:57.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.648+0000 7fb7a3fb3740 1 Processor -- start 2024-04-04T06:12:57.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.649+0000 7fb7a3fb3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9a88cc5b0 0x55a9a88d8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.650+0000 7fb7a3fb3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88d4580 0x55a9a88cae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.650+0000 7fb7a3fb3740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88d4960 0x55a9a890bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.650+0000 7fb7a3fb3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a9a88a8890 con 0x55a9a88d4580 2024-04-04T06:12:57.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.650+0000 7fb7a3fb3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c160950 con 0x55a9a88cc5b0 2024-04-04T06:12:57.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.650+0000 7fb7a3fb3740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a9a88d7730 con 0x55a9a88d4960 2024-04-04T06:12:57.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.650+0000 7fb7a3fb3740 10 client.0 _ll_register_callbacks cb 0x7fb79402cc40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:12:57.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.650+0000 7fb791400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88d4580 0x55a9a88cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.650+0000 7fb791400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88d4580 0x55a9a88cae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35766/0 (socket says 172.21.15.159:35766) 2024-04-04T06:12:57.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.650+0000 7fb791400640 1 -- 172.21.15.159:0/2866473865 learned_addr learned my addr 172.21.15.159:0/2866473865 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:12:57.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.650+0000 7fb799e00640 1 --2- 172.21.15.159:0/2866473865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9a88cc5b0 0x55a9a88d8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.650+0000 7fb79a800640 1 --2- 172.21.15.159:0/2866473865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88d4960 0x55a9a890bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.650+0000 7fb791400640 1 -- 172.21.15.159:0/2866473865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88d4960 msgr2=0x55a9a890bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:57.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.650+0000 7fb791400640 1 --2- 172.21.15.159:0/2866473865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88d4960 0x55a9a890bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.650+0000 7fb791400640 1 -- 172.21.15.159:0/2866473865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9a88cc5b0 msgr2=0x55a9a88d8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:12:57.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.650+0000 7fb791400640 1 --2- 172.21.15.159:0/2866473865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9a88cc5b0 0x55a9a88d8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.650+0000 7fb791400640 1 -- 172.21.15.159:0/2866473865 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9a8902790 con 0x55a9a88d4580 2024-04-04T06:12:57.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.650+0000 7fb799e00640 1 --2- 172.21.15.159:0/2866473865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9a88cc5b0 0x55a9a88d8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:12:57.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.651+0000 7fb791400640 1 --2- 172.21.15.159:0/2866473865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88d4580 0x55a9a88cae40 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fb78408a450 tx=0x7fb784059c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6848c963641adb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:12:57.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.651+0000 7fb77f400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:12:57.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.651+0000 7fb77f400640 1 -- 172.21.15.159:0/2866473865 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784087070 con 0x55a9a88d4580 2024-04-04T06:12:57.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.651+0000 7fb77f400640 1 -- 172.21.15.159:0/2866473865 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7fb784073b40 con 0x55a9a88d4580 2024-04-04T06:12:57.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.651+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/2866473865 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb794034b30 con 0x55a9a88d4580 2024-04-04T06:12:57.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.651+0000 7fb77f400640 1 -- 172.21.15.159:0/2866473865 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fb784021040 con 0x55a9a88d4580 2024-04-04T06:12:57.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.651+0000 7fb7a3fb3740 10 client.7626 fetch_fsmap learned FSMap version 68 2024-04-04T06:12:57.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.651+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/2866473865 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55a9a88cb600 con 0x55a9a88d4580 2024-04-04T06:12:57.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.651+0000 7fb77f400640 1 -- 172.21.15.159:0/2866473865 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb794034b30 con 0x55a9a88d4580 2024-04-04T06:12:57.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.651+0000 7fb77f400640 1 -- 172.21.15.159:0/2866473865 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7fb7840590a0 con 0x55a9a88d4580 2024-04-04T06:12:57.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.651+0000 7fb7a3fb3740 10 client.7626 fetch_fsmap finished waiting for FSMap version 68 2024-04-04T06:12:57.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.651+0000 7fb7a3fb3740 10 client.7626 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-04T06:12:57.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.651+0000 7fb7a3fb3740 20 client.7626 populate_metadata read hostname 'smithi159' 2024-04-04T06:12:57.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.651+0000 7fb7a3fb3740 1 --2- 172.21.15.159:0/2866473865 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55a9a88f2b30 0x55a9a88d4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:12:57.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.651+0000 7fb7a3fb3740 4 client.7626 mds_command: new command op to 7512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-04T06:12:57.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.651+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/2866473865 --> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a9a8819f60 con 0x55a9a88f2b30 2024-04-04T06:12:57.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.652+0000 7fb799e00640 1 --2- 172.21.15.159:0/2866473865 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55a9a88f2b30 0x55a9a88d4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:12:57.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.652+0000 7fb799e00640 1 --2- 172.21.15.159:0/2866473865 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55a9a88f2b30 0x55a9a88d4d40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ab0da35f6ca7bb4 server_cookie=f0ab4abb38c5140e in_seq=0 out_seq=0 2024-04-04T06:12:57.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.652+0000 7fb77f400640 10 client.7626 ms_handle_connect on v2:172.21.15.159:6833/3796123128 2024-04-04T06:12:57.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.652+0000 7fb77f400640 1 -- 172.21.15.159:0/2866473865 <== mds.0 v2:172.21.15.159:6833/3796123128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+256 (crc 0 0 0) 0x55a9a8819f60 con 0x55a9a88f2b30 2024-04-04T06:12:57.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.652+0000 7fb77f400640 10 client.7626 handle_command_reply: tid=0 2024-04-04T06:12:57.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.652+0000 7fb7a3fb3740 1 client.7626 shutdown 2024-04-04T06:12:57.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.653+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/2866473865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88d4580 msgr2=0x55a9a88cae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:57.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.653+0000 7fb7a3fb3740 1 --2- 172.21.15.159:0/2866473865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88d4580 0x55a9a88cae40 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fb78408a450 tx=0x7fb784059c50 comp rx=0 tx=0).stop 2024-04-04T06:12:57.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.653+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/2866473865 shutdown_connections 2024-04-04T06:12:57.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.653+0000 7fb7a3fb3740 1 --2- 172.21.15.159:0/2866473865 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55a9a88f2b30 0x55a9a88d4d40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.653+0000 7fb7a3fb3740 1 --2- 172.21.15.159:0/2866473865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a9a88d4960 0x55a9a890bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.653+0000 7fb7a3fb3740 1 --2- 172.21.15.159:0/2866473865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9a88d4580 0x55a9a88cae40 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.653+0000 7fb77f400640 0 client.7626 ms_handle_reset on v2:172.21.15.159:6833/3796123128 2024-04-04T06:12:57.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.653+0000 7fb7a3fb3740 1 --2- 172.21.15.159:0/2866473865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a9a88cc5b0 0x55a9a88d8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.653+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/2866473865 >> 172.21.15.159:0/2866473865 conn(0x55a9a88a7a60 msgr2=0x55a9a88daf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:57.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.653+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/2866473865 shutdown_connections 2024-04-04T06:12:57.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.653+0000 7fb7a3fb3740 1 -- 172.21.15.159:0/2866473865 wait complete. 2024-04-04T06:12:57.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.653+0000 7fb7a3fb3740 20 client.7626 trim_cache size 0 max 16384 2024-04-04T06:12:57.651 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:12:57.651 INFO:teuthology.orchestra.run.smithi159.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-04T06:12:57.651 INFO:teuthology.orchestra.run.smithi159.stdout: "scrubs": { 2024-04-04T06:12:57.651 INFO:teuthology.orchestra.run.smithi159.stdout: "f36edbfc-f5de-4ac5-8e8b-527a6a681789": { 2024-04-04T06:12:57.651 INFO:teuthology.orchestra.run.smithi159.stdout: "path": "/dir", 2024-04-04T06:12:57.651 INFO:teuthology.orchestra.run.smithi159.stdout: "tag": "f36edbfc-f5de-4ac5-8e8b-527a6a681789", 2024-04-04T06:12:57.652 INFO:teuthology.orchestra.run.smithi159.stdout: "options": "recursive" 2024-04-04T06:12:57.652 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:57.652 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:12:57.652 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:12:57.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.654+0000 7fb7a1200640 1 -- 172.21.15.159:0/2332943523 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb77806c8f0 msgr2=0x7fb77806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:57.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.654+0000 7fb7a1200640 1 --2- 172.21.15.159:0/2332943523 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb77806c8f0 0x7fb77806edb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb78c000f60 tx=0x7fb78c009040 comp rx=0 tx=0).stop 2024-04-04T06:12:57.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.654+0000 7fb7a1200640 1 -- 172.21.15.159:0/2332943523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb79c070120 msgr2=0x7fb79c161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:12:57.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.654+0000 7fb7a1200640 1 --2- 172.21.15.159:0/2332943523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb79c070120 0x7fb79c161150 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fb78400e9b0 tx=0x7fb78400ee70 comp rx=0 tx=0).stop 2024-04-04T06:12:57.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.654+0000 7fb7a1200640 1 -- 172.21.15.159:0/2332943523 shutdown_connections 2024-04-04T06:12:57.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.654+0000 7fb7a1200640 1 --2- 172.21.15.159:0/2332943523 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb77806c8f0 0x7fb77806edb0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.654+0000 7fb7a1200640 1 --2- 172.21.15.159:0/2332943523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb79c0b0670 0x7fb79c157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.654+0000 7fb7a1200640 1 --2- 172.21.15.159:0/2332943523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb79c070120 0x7fb79c161150 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.654+0000 7fb7a1200640 1 --2- 172.21.15.159:0/2332943523 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb79c06e770 0x7fb79c14fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:12:57.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.654+0000 7fb7a1200640 1 -- 172.21.15.159:0/2332943523 >> 172.21.15.159:0/2332943523 conn(0x7fb79c0b65a0 msgr2=0x7fb79c151f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:12:57.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.654+0000 7fb7a1200640 1 -- 172.21.15.159:0/2332943523 shutdown_connections 2024-04-04T06:12:57.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:12:57.654+0000 7fb7a1200640 1 -- 172.21.15.159:0/2332943523 wait complete. 2024-04-04T06:12:57.665 INFO:tasks.cephfs.filesystem:scrub status for tag:f36edbfc-f5de-4ac5-8e8b-527a6a681789 - {'path': '/dir', 'tag': 'f36edbfc-f5de-4ac5-8e8b-527a6a681789', 'options': 'recursive'} 2024-04-04T06:13:00.543 INFO:tasks.ceph.mon.a.smithi159.stderr:2024-04-04T06:13:00.549+0000 7f1998e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-04T06:13:12.628 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:13:12.636 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:13:27.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:13:27.666 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-04T06:13:27.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.890+0000 7fdc02a00640 1 Processor -- start 2024-04-04T06:13:27.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.891+0000 7fdc02a00640 1 -- start start 2024-04-04T06:13:27.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.892+0000 7fdc02a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbfc15e6d0 0x7fdbfc0d21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:27.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.892+0000 7fdc02a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdbfc0d0cb0 0x7fdbfc0d4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:27.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.892+0000 7fdc02a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbfc0ceef0 con 0x7fdbfc0d4590 2024-04-04T06:13:27.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.892+0000 7fdc02a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fdbfc15eab0 con 0x7fdbfc0d0cb0 2024-04-04T06:13:27.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.892+0000 7fdc02a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdbfc0d1220 con 0x7fdbfc15e6d0 2024-04-04T06:13:27.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.892+0000 7fdbfbe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbfc15e6d0 0x7fdbfc0d21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:27.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.892+0000 7fdbf3400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdbfc0d0cb0 0x7fdbfc0d4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:27.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.893+0000 7fdbf3400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdbfc0d0cb0 0x7fdbfc0d4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:43378/0 (socket says 172.21.15.159:43378) 2024-04-04T06:13:27.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.893+0000 7fdbfbe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbfc15e6d0 0x7fdbfc0d21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:48470/0 (socket says 172.21.15.159:48470) 2024-04-04T06:13:27.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.893+0000 7fdbf3400640 1 -- 172.21.15.159:0/3276584183 learned_addr learned my addr 172.21.15.159:0/3276584183 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:27.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.893+0000 7fdc00a00640 1 --1- 172.21.15.159:0/3276584183 >> v1:172.21.15.178:6789/0 conn(0x7fdbfc0d4590 0x7fdbfc1621c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.159:49598/0 (socket says 172.21.15.159:49598) 2024-04-04T06:13:27.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.893+0000 7fdbf3400640 1 -- 172.21.15.159:0/3276584183 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbfc15e6d0 msgr2=0x7fdbfc0d21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:27.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.893+0000 7fdbf3400640 1 --2- 172.21.15.159:0/3276584183 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbfc15e6d0 0x7fdbfc0d21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.893+0000 7fdbf3400640 1 -- 172.21.15.159:0/3276584183 >> v1:172.21.15.178:6789/0 conn(0x7fdbfc0d4590 legacy=0x7fdbfc1621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:27.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.893+0000 7fdbf3400640 1 -- 172.21.15.159:0/3276584183 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbfc0d1500 con 0x7fdbfc0d0cb0 2024-04-04T06:13:27.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.895+0000 7fdbf3400640 1 --2- 172.21.15.159:0/3276584183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbfc0d0cb0 0x7fdbfc0d4050 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009ea0 tx=0x7fdbe402bfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4af505490cbd02a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:27.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.895+0000 7fdbfb400640 1 -- 172.21.15.159:0/3276584183 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe402cb40 con 0x7fdbfc0d0cb0 2024-04-04T06:13:27.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.895+0000 7fdbfb400640 1 -- 172.21.15.159:0/3276584183 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbe402cce0 con 0x7fdbfc0d0cb0 2024-04-04T06:13:27.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.895+0000 7fdbfb400640 1 -- 172.21.15.159:0/3276584183 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4034610 con 0x7fdbfc0d0cb0 2024-04-04T06:13:27.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.896+0000 7fdc02a00640 1 -- 172.21.15.159:0/3276584183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbfc0d0cb0 msgr2=0x7fdbfc0d4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:27.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.896+0000 7fdc02a00640 1 --2- 172.21.15.159:0/3276584183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbfc0d0cb0 0x7fdbfc0d4050 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009ea0 tx=0x7fdbe402bfa0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.896+0000 7fdc02a00640 1 -- 172.21.15.159:0/3276584183 shutdown_connections 2024-04-04T06:13:27.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.896+0000 7fdc02a00640 1 --2- 172.21.15.159:0/3276584183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbfc0d0cb0 0x7fdbfc0d4050 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.896+0000 7fdc02a00640 1 --2- 172.21.15.159:0/3276584183 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbfc15e6d0 0x7fdbfc0d21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.896+0000 7fdc02a00640 1 -- 172.21.15.159:0/3276584183 >> 172.21.15.159:0/3276584183 conn(0x7fdbfc09d7b0 msgr2=0x7fdbfc08bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:27.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.896+0000 7fdc02a00640 1 -- 172.21.15.159:0/3276584183 shutdown_connections 2024-04-04T06:13:27.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.896+0000 7fdc02a00640 1 -- 172.21.15.159:0/3276584183 wait complete. 2024-04-04T06:13:27.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.898+0000 7fdc02a00640 1 Processor -- start 2024-04-04T06:13:27.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.898+0000 7fdc02a00640 1 -- start start 2024-04-04T06:13:27.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.899+0000 7fdc02a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbfc0d0cb0 0x7fdbfc1f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:27.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.899+0000 7fdc02a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbfc0d4590 0x7fdbfc1f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:27.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.899+0000 7fdc02a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbfc15e6d0 0x7fdbfc1f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:27.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.899+0000 7fdc02a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc0d4a20 con 0x7fdbfc0d4590 2024-04-04T06:13:27.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.899+0000 7fdc02a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc0d1ac0 con 0x7fdbfc15e6d0 2024-04-04T06:13:27.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.899+0000 7fdc02a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdbfc0d2720 con 0x7fdbfc0d0cb0 2024-04-04T06:13:27.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.899+0000 7fdbf3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbfc0d4590 0x7fdbfc1f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:27.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.899+0000 7fdbf3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbfc0d4590 0x7fdbfc1f5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38718/0 (socket says 172.21.15.159:38718) 2024-04-04T06:13:27.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.899+0000 7fdbf3400640 1 -- 172.21.15.159:0/2063269821 learned_addr learned my addr 172.21.15.159:0/2063269821 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:27.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.899+0000 7fdc00a00640 1 --2- 172.21.15.159:0/2063269821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbfc15e6d0 0x7fdbfc1f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:27.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.899+0000 7fdbfbe00640 1 --2- 172.21.15.159:0/2063269821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbfc0d0cb0 0x7fdbfc1f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:27.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.899+0000 7fdbf3400640 1 -- 172.21.15.159:0/2063269821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbfc0d0cb0 msgr2=0x7fdbfc1f1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:27.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.899+0000 7fdbf3400640 1 --2- 172.21.15.159:0/2063269821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbfc0d0cb0 0x7fdbfc1f1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.900+0000 7fdbf3400640 1 -- 172.21.15.159:0/2063269821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbfc15e6d0 msgr2=0x7fdbfc1f5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:27.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.900+0000 7fdbf3400640 1 --2- 172.21.15.159:0/2063269821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbfc15e6d0 0x7fdbfc1f5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.900+0000 7fdbf3400640 1 -- 172.21.15.159:0/2063269821 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbfc1f22f0 con 0x7fdbfc0d4590 2024-04-04T06:13:27.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.900+0000 7fdbf3400640 1 --2- 172.21.15.159:0/2063269821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbfc0d4590 0x7fdbfc1f5320 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4000f80 tx=0x7fdbe4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:27.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.900+0000 7fdbf8c00640 1 -- 172.21.15.159:0/2063269821 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4043020 con 0x7fdbfc0d4590 2024-04-04T06:13:27.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.900+0000 7fdbf8c00640 1 -- 172.21.15.159:0/2063269821 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbe403e450 con 0x7fdbfc0d4590 2024-04-04T06:13:27.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.900+0000 7fdc02a00640 1 -- 172.21.15.159:0/2063269821 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbfc1f5860 con 0x7fdbfc0d4590 2024-04-04T06:13:27.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.900+0000 7fdbf8c00640 1 -- 172.21.15.159:0/2063269821 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4009510 con 0x7fdbfc0d4590 2024-04-04T06:13:27.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.900+0000 7fdc02a00640 1 -- 172.21.15.159:0/2063269821 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbfc1e75e0 con 0x7fdbfc0d4590 2024-04-04T06:13:27.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.903+0000 7fdbf8c00640 1 -- 172.21.15.159:0/2063269821 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdbe4046020 con 0x7fdbfc0d4590 2024-04-04T06:13:27.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.904+0000 7fdc05666740 1 Processor -- start 2024-04-04T06:13:27.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.904+0000 7fdbf8c00640 1 --2- 172.21.15.159:0/2063269821 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdbd806c8f0 0x7fdbd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:27.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.904+0000 7fdc05666740 1 -- start start 2024-04-04T06:13:27.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.904+0000 7fdbf8c00640 1 -- 172.21.15.159:0/2063269821 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7fdbe40ae3c0 con 0x7fdbfc0d4590 2024-04-04T06:13:27.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.905+0000 7fdc00a00640 1 --2- 172.21.15.159:0/2063269821 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdbd806c8f0 0x7fdbd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:27.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.905+0000 7fdc05666740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fd62143580 0x55fd62116520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:27.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.905+0000 7fdc05666740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd62143960 0x55fd62146310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:27.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.905+0000 7fdc05666740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd62116a60 0x55fd62146850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:27.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.905+0000 7fdc05666740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fd620816d0 con 0x55fd62116a60 2024-04-04T06:13:27.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.905+0000 7fdc05666740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fd62029770 con 0x55fd62143960 2024-04-04T06:13:27.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.905+0000 7fdc05666740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55fd6207d680 con 0x55fd62143580 2024-04-04T06:13:27.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.906+0000 7fdc00a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd62116a60 0x55fd62146850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:27.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.906+0000 7fdc00a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd62116a60 0x55fd62146850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38726/0 (socket says 172.21.15.159:38726) 2024-04-04T06:13:27.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.906+0000 7fdc00a00640 1 -- 172.21.15.159:0/1130274894 learned_addr learned my addr 172.21.15.159:0/1130274894 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:27.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.906+0000 7fdbfbe00640 1 --2- 172.21.15.159:0/1130274894 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fd62143580 0x55fd62116520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:27.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.906+0000 7fdbf3400640 1 --2- 172.21.15.159:0/1130274894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd62143960 0x55fd62146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:27.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.906+0000 7fdc00a00640 1 -- 172.21.15.159:0/1130274894 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fd62143580 msgr2=0x55fd62116520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:27.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.906+0000 7fdc00a00640 1 --2- 172.21.15.159:0/1130274894 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fd62143580 0x55fd62116520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.906+0000 7fdc00a00640 1 -- 172.21.15.159:0/1130274894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd62143960 msgr2=0x55fd62146310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:27.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.906+0000 7fdc00a00640 1 --2- 172.21.15.159:0/1130274894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd62143960 0x55fd62146310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.906+0000 7fdc00a00640 1 -- 172.21.15.159:0/1130274894 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd61e5f7d0 con 0x55fd62116a60 2024-04-04T06:13:27.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.906+0000 7fdc00a00640 1 --2- 172.21.15.159:0/2063269821 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdbd806c8f0 0x7fdbd806edb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4007320 tx=0x7fdbf4007720 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:27.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.907+0000 7fdc00a00640 1 --2- 172.21.15.159:0/1130274894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd62116a60 0x55fd62146850 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4001560 tx=0x7fdbf401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b07d97354a4333 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:27.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.907+0000 7fdbebe00640 1 -- 172.21.15.159:0/1130274894 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf401c070 con 0x55fd62116a60 2024-04-04T06:13:27.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.907+0000 7fdbebe00640 1 -- 172.21.15.159:0/1130274894 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbf4018ba0 con 0x55fd62116a60 2024-04-04T06:13:27.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.907+0000 7fdbebe00640 1 -- 172.21.15.159:0/1130274894 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4021410 con 0x55fd62116a60 2024-04-04T06:13:27.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.908+0000 7fdc05666740 1 -- 172.21.15.159:0/1130274894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd62116a60 msgr2=0x55fd62146850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:27.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.908+0000 7fdc05666740 1 --2- 172.21.15.159:0/1130274894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd62116a60 0x55fd62146850 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4001560 tx=0x7fdbf401b000 comp rx=0 tx=0).stop 2024-04-04T06:13:27.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.908+0000 7fdc05666740 1 -- 172.21.15.159:0/1130274894 shutdown_connections 2024-04-04T06:13:27.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.908+0000 7fdc05666740 1 --2- 172.21.15.159:0/1130274894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd62116a60 0x55fd62146850 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.908+0000 7fdc05666740 1 --2- 172.21.15.159:0/1130274894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd62143960 0x55fd62146310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.908+0000 7fdc05666740 1 --2- 172.21.15.159:0/1130274894 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fd62143580 0x55fd62116520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.908+0000 7fdc05666740 1 -- 172.21.15.159:0/1130274894 >> 172.21.15.159:0/1130274894 conn(0x55fd62118790 msgr2=0x55fd62161310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:27.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.908+0000 7fdc05666740 1 -- 172.21.15.159:0/1130274894 shutdown_connections 2024-04-04T06:13:27.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.908+0000 7fdc05666740 1 -- 172.21.15.159:0/1130274894 wait complete. 2024-04-04T06:13:27.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.909+0000 7fdc05666740 1 -- start start 2024-04-04T06:13:27.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.909+0000 7fdc05666740 1 Processor -- start 2024-04-04T06:13:27.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.910+0000 7fdc05666740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd62143580 0x55fd62138300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:27.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.910+0000 7fdc05666740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fd62143960 0x55fd62146490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:27.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.910+0000 7fdc05666740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd6215ef30 0x55fd62148440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:27.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.910+0000 7fdc05666740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fd6211ae30 con 0x55fd62143580 2024-04-04T06:13:27.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.910+0000 7fdc05666740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fd6203caa0 con 0x55fd6215ef30 2024-04-04T06:13:27.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.910+0000 7fdc05666740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55fd6211d670 con 0x55fd62143960 2024-04-04T06:13:27.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.910+0000 7fdc05666740 10 client.0 _ll_register_callbacks cb 0x55fd6203c490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:13:27.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.910+0000 7fdbfbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd62143580 0x55fd62138300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:27.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.910+0000 7fdbfbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd62143580 0x55fd62138300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38732/0 (socket says 172.21.15.159:38732) 2024-04-04T06:13:27.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.910+0000 7fdbfbe00640 1 -- 172.21.15.159:0/2396048867 learned_addr learned my addr 172.21.15.159:0/2396048867 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:27.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.910+0000 7fdc00a00640 1 --2- 172.21.15.159:0/2396048867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd6215ef30 0x55fd62148440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:27.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.910+0000 7fdbf3400640 1 --2- 172.21.15.159:0/2396048867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fd62143960 0x55fd62146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:27.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.911+0000 7fdbfbe00640 1 -- 172.21.15.159:0/2396048867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fd62143960 msgr2=0x55fd62146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:27.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.911+0000 7fdbfbe00640 1 --2- 172.21.15.159:0/2396048867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fd62143960 0x55fd62146490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.911+0000 7fdbfbe00640 1 -- 172.21.15.159:0/2396048867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd6215ef30 msgr2=0x55fd62148440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:27.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.911+0000 7fdbfbe00640 1 --2- 172.21.15.159:0/2396048867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd6215ef30 0x55fd62148440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.911+0000 7fdbfbe00640 1 -- 172.21.15.159:0/2396048867 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd61d953f0 con 0x55fd62143580 2024-04-04T06:13:27.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.911+0000 7fdc00a00640 1 --2- 172.21.15.159:0/2396048867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd6215ef30 0x55fd62148440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:13:27.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.911+0000 7fdbfbe00640 1 --2- 172.21.15.159:0/2396048867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd62143580 0x55fd62138300 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009dd0 tx=0x7fdbec0136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3140420a1e5eeab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:27.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.911+0000 7fdbe8c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:13:27.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.911+0000 7fdbe8c00640 1 -- 172.21.15.159:0/2396048867 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec01a030 con 0x55fd62143580 2024-04-04T06:13:27.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.911+0000 7fdbe8c00640 1 -- 172.21.15.159:0/2396048867 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7fdbec016460 con 0x55fd62143580 2024-04-04T06:13:27.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.911+0000 7fdc05666740 1 -- 172.21.15.159:0/2396048867 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbf4018ba0 con 0x55fd62143580 2024-04-04T06:13:27.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.911+0000 7fdbe8c00640 1 -- 172.21.15.159:0/2396048867 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fdbec005990 con 0x55fd62143580 2024-04-04T06:13:27.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.911+0000 7fdc05666740 10 client.7635 fetch_fsmap learned FSMap version 69 2024-04-04T06:13:27.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.911+0000 7fdc05666740 1 -- 172.21.15.159:0/2396048867 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x55fd61d550f0 con 0x55fd62143580 2024-04-04T06:13:27.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.912+0000 7fdbe8c00640 1 -- 172.21.15.159:0/2396048867 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbf4018ba0 con 0x55fd62143580 2024-04-04T06:13:27.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.912+0000 7fdbe8c00640 1 -- 172.21.15.159:0/2396048867 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 69) ==== 3494+0+0 (secure 0 0 0) 0x7fdbec029080 con 0x55fd62143580 2024-04-04T06:13:27.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.912+0000 7fdc05666740 10 client.7635 fetch_fsmap finished waiting for FSMap version 69 2024-04-04T06:13:27.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.912+0000 7fdc05666740 10 client.7635 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-04T06:13:27.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.912+0000 7fdc05666740 20 client.7635 populate_metadata read hostname 'smithi159' 2024-04-04T06:13:27.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.912+0000 7fdc05666740 1 --2- 172.21.15.159:0/2396048867 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55fd6215fd10 0x55fd62174d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:27.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.912+0000 7fdc05666740 4 client.7635 mds_command: new command op to 7512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:13:27.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.912+0000 7fdc05666740 1 -- 172.21.15.159:0/2396048867 --> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd6208ec10 con 0x55fd6215fd10 2024-04-04T06:13:27.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.912+0000 7fdbf3400640 1 --2- 172.21.15.159:0/2396048867 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55fd6215fd10 0x55fd62174d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:27.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.913+0000 7fdbf3400640 1 --2- 172.21.15.159:0/2396048867 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55fd6215fd10 0x55fd62174d40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c65a1943f0fc759b server_cookie=2ce105201eab8c95 in_seq=0 out_seq=0 2024-04-04T06:13:27.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.913+0000 7fdbe8c00640 10 client.7635 ms_handle_connect on v2:172.21.15.159:6833/3796123128 2024-04-04T06:13:27.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.913+0000 7fdbe8c00640 1 -- 172.21.15.159:0/2396048867 <== mds.0 v2:172.21.15.159:6833/3796123128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fd6208ec10 con 0x55fd6215fd10 2024-04-04T06:13:27.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.913+0000 7fdbe8c00640 10 client.7635 handle_command_reply: tid=0 2024-04-04T06:13:27.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.913+0000 7fdc05666740 1 client.7635 shutdown 2024-04-04T06:13:27.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.914+0000 7fdc05666740 1 -- 172.21.15.159:0/2396048867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd62143580 msgr2=0x55fd62138300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:27.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.914+0000 7fdc05666740 1 --2- 172.21.15.159:0/2396048867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd62143580 0x55fd62138300 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009dd0 tx=0x7fdbec0136b0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.914+0000 7fdc05666740 1 -- 172.21.15.159:0/2396048867 shutdown_connections 2024-04-04T06:13:27.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.914+0000 7fdc05666740 1 --2- 172.21.15.159:0/2396048867 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55fd6215fd10 0x55fd62174d40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.914+0000 7fdc05666740 1 --2- 172.21.15.159:0/2396048867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd6215ef30 0x55fd62148440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.914+0000 7fdc05666740 1 --2- 172.21.15.159:0/2396048867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fd62143960 0x55fd62146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.914+0000 7fdc05666740 1 --2- 172.21.15.159:0/2396048867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd62143580 0x55fd62138300 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.914+0000 7fdc05666740 1 -- 172.21.15.159:0/2396048867 >> 172.21.15.159:0/2396048867 conn(0x55fd62116a60 msgr2=0x55fd62164790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:27.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.914+0000 7fdbe8c00640 0 client.7635 ms_handle_reset on v2:172.21.15.159:6833/3796123128 2024-04-04T06:13:27.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.914+0000 7fdc05666740 1 -- 172.21.15.159:0/2396048867 shutdown_connections 2024-04-04T06:13:27.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.914+0000 7fdc05666740 1 -- 172.21.15.159:0/2396048867 wait complete. 2024-04-04T06:13:27.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.914+0000 7fdc05666740 20 client.7635 trim_cache size 0 max 16384 2024-04-04T06:13:27.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.933+0000 7fdc05666740 1 Processor -- start 2024-04-04T06:13:27.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.933+0000 7fdc05666740 1 -- start start 2024-04-04T06:13:27.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.934+0000 7fdc05666740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd62143580 0x55fd621153b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:27.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.934+0000 7fdc05666740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fd62143960 0x55fd62147f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:27.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.934+0000 7fdc05666740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd6215ef30 0x55fd621640b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:27.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.934+0000 7fdc05666740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc200b20 con 0x55fd6215ef30 2024-04-04T06:13:27.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.934+0000 7fdc05666740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc0b7d70 con 0x55fd62143580 2024-04-04T06:13:27.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.934+0000 7fdc05666740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdbfc1e8fe0 con 0x55fd62143960 2024-04-04T06:13:27.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.934+0000 7fdc00a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd6215ef30 0x55fd621640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:27.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.934+0000 7fdc00a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd6215ef30 0x55fd621640b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38736/0 (socket says 172.21.15.159:38736) 2024-04-04T06:13:27.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.934+0000 7fdc00a00640 1 -- 172.21.15.159:0/3936167390 learned_addr learned my addr 172.21.15.159:0/3936167390 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:27.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.935+0000 7fdbf3400640 1 --2- 172.21.15.159:0/3936167390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fd62143960 0x55fd62147f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:27.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.935+0000 7fdc00a00640 1 -- 172.21.15.159:0/3936167390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fd62143960 msgr2=0x55fd62147f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:27.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.935+0000 7fdc00a00640 1 --2- 172.21.15.159:0/3936167390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fd62143960 0x55fd62147f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.935+0000 7fdbfbe00640 1 --2- 172.21.15.159:0/3936167390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd62143580 0x55fd621153b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:27.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.935+0000 7fdc00a00640 1 -- 172.21.15.159:0/3936167390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd62143580 msgr2=0x55fd621153b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:27.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.935+0000 7fdc00a00640 1 --2- 172.21.15.159:0/3936167390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd62143580 0x55fd621153b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.935+0000 7fdc00a00640 1 -- 172.21.15.159:0/3936167390 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd61da21a0 con 0x55fd6215ef30 2024-04-04T06:13:27.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.935+0000 7fdc00a00640 1 --2- 172.21.15.159:0/3936167390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd6215ef30 0x55fd621640b0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fdbf401e000 tx=0x7fdbf4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb3e4784deb0ba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:27.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.935+0000 7fdbeb400640 1 -- 172.21.15.159:0/3936167390 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4018ba0 con 0x55fd6215ef30 2024-04-04T06:13:27.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.935+0000 7fdbeb400640 1 -- 172.21.15.159:0/3936167390 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbf4032070 con 0x55fd6215ef30 2024-04-04T06:13:27.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.935+0000 7fdbeb400640 1 -- 172.21.15.159:0/3936167390 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf402d480 con 0x55fd6215ef30 2024-04-04T06:13:27.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.936+0000 7fdc05666740 1 -- 172.21.15.159:0/3936167390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd6215ef30 msgr2=0x55fd621640b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:27.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.936+0000 7fdc05666740 1 --2- 172.21.15.159:0/3936167390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd6215ef30 0x55fd621640b0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fdbf401e000 tx=0x7fdbf4019000 comp rx=0 tx=0).stop 2024-04-04T06:13:27.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.936+0000 7fdc05666740 1 -- 172.21.15.159:0/3936167390 shutdown_connections 2024-04-04T06:13:27.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.936+0000 7fdc05666740 1 --2- 172.21.15.159:0/3936167390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd6215ef30 0x55fd621640b0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.936+0000 7fdc05666740 1 --2- 172.21.15.159:0/3936167390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fd62143960 0x55fd62147f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.936+0000 7fdc05666740 1 --2- 172.21.15.159:0/3936167390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd62143580 0x55fd621153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.936+0000 7fdc05666740 1 -- 172.21.15.159:0/3936167390 >> 172.21.15.159:0/3936167390 conn(0x55fd62116a60 msgr2=0x55fd62146250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:27.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.936+0000 7fdc05666740 1 -- 172.21.15.159:0/3936167390 shutdown_connections 2024-04-04T06:13:27.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.936+0000 7fdc05666740 1 -- 172.21.15.159:0/3936167390 wait complete. 2024-04-04T06:13:27.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.937+0000 7fdc05666740 1 -- start start 2024-04-04T06:13:27.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.937+0000 7fdc05666740 1 Processor -- start 2024-04-04T06:13:27.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.938+0000 7fdc05666740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd62143580 0x55fd621c4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:27.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.938+0000 7fdc05666740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fd62143960 0x55fd621634e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:27.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.938+0000 7fdc05666740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd6215ef30 0x55fd621497c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:27.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.938+0000 7fdc05666740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fd6211ca70 con 0x55fd6215ef30 2024-04-04T06:13:27.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.938+0000 7fdc05666740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc0b7720 con 0x55fd62143580 2024-04-04T06:13:27.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.938+0000 7fdc05666740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55fd6211d060 con 0x55fd62143960 2024-04-04T06:13:27.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.938+0000 7fdc05666740 10 client.0 _ll_register_callbacks cb 0x7fdbec0098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:13:27.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.938+0000 7fdc00a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd6215ef30 0x55fd621497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:27.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.938+0000 7fdc00a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd6215ef30 0x55fd621497c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38746/0 (socket says 172.21.15.159:38746) 2024-04-04T06:13:27.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.938+0000 7fdc00a00640 1 -- 172.21.15.159:0/174145665 learned_addr learned my addr 172.21.15.159:0/174145665 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:27.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.939+0000 7fdbfbe00640 1 --2- 172.21.15.159:0/174145665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd62143580 0x55fd621c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:27.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.939+0000 7fdc00a00640 1 -- 172.21.15.159:0/174145665 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fd62143960 msgr2=0x55fd621634e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:27.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.939+0000 7fdc00a00640 1 --2- 172.21.15.159:0/174145665 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fd62143960 0x55fd621634e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.939+0000 7fdc00a00640 1 -- 172.21.15.159:0/174145665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd62143580 msgr2=0x55fd621c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:27.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.939+0000 7fdc00a00640 1 --2- 172.21.15.159:0/174145665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd62143580 0x55fd621c4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.939+0000 7fdc00a00640 1 -- 172.21.15.159:0/174145665 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd62162720 con 0x55fd6215ef30 2024-04-04T06:13:27.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.939+0000 7fdc00a00640 1 --2- 172.21.15.159:0/174145665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd6215ef30 0x55fd621497c0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4000c00 tx=0x7fdbf4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278731881b6d9528 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:27.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.939+0000 7fdbe8c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:13:27.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.939+0000 7fdbe8c00640 1 -- 172.21.15.159:0/174145665 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf402d890 con 0x55fd6215ef30 2024-04-04T06:13:27.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.939+0000 7fdc05666740 1 -- 172.21.15.159:0/174145665 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbf4032070 con 0x55fd6215ef30 2024-04-04T06:13:27.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.939+0000 7fdbe8c00640 1 -- 172.21.15.159:0/174145665 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7fdbf403fb20 con 0x55fd6215ef30 2024-04-04T06:13:27.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.940+0000 7fdbe8c00640 1 -- 172.21.15.159:0/174145665 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fdbf403fd20 con 0x55fd6215ef30 2024-04-04T06:13:27.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.940+0000 7fdc05666740 10 client.7641 fetch_fsmap learned FSMap version 69 2024-04-04T06:13:27.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.940+0000 7fdc05666740 1 -- 172.21.15.159:0/174145665 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x55fd62163a80 con 0x55fd6215ef30 2024-04-04T06:13:27.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.940+0000 7fdbe8c00640 1 -- 172.21.15.159:0/174145665 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbf4032070 con 0x55fd6215ef30 2024-04-04T06:13:27.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.940+0000 7fdbe8c00640 1 -- 172.21.15.159:0/174145665 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 69) ==== 3494+0+0 (secure 0 0 0) 0x7fdbf4012030 con 0x55fd6215ef30 2024-04-04T06:13:27.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.940+0000 7fdc05666740 10 client.7641 fetch_fsmap finished waiting for FSMap version 69 2024-04-04T06:13:27.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.940+0000 7fdc05666740 10 client.7641 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-04T06:13:27.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.940+0000 7fdc05666740 20 client.7641 populate_metadata read hostname 'smithi159' 2024-04-04T06:13:27.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.940+0000 7fdc05666740 1 --2- 172.21.15.159:0/174145665 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55fd621c3bf0 0x55fd62149d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:27.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.940+0000 7fdc05666740 4 client.7641 mds_command: new command op to 7512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-04T06:13:27.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.940+0000 7fdc05666740 1 -- 172.21.15.159:0/174145665 --> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fd62088f60 con 0x55fd621c3bf0 2024-04-04T06:13:27.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.940+0000 7fdbfbe00640 1 --2- 172.21.15.159:0/174145665 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55fd621c3bf0 0x55fd62149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:27.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.941+0000 7fdbfbe00640 1 --2- 172.21.15.159:0/174145665 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55fd621c3bf0 0x55fd62149d00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66eb3d38a0554da5 server_cookie=a6afaf1c9566ebe8 in_seq=0 out_seq=0 2024-04-04T06:13:27.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.941+0000 7fdbe8c00640 10 client.7641 ms_handle_connect on v2:172.21.15.159:6833/3796123128 2024-04-04T06:13:27.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.941+0000 7fdbe8c00640 1 -- 172.21.15.159:0/174145665 <== mds.0 v2:172.21.15.159:6833/3796123128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fd62088f60 con 0x55fd621c3bf0 2024-04-04T06:13:27.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.941+0000 7fdbe8c00640 10 client.7641 handle_command_reply: tid=0 2024-04-04T06:13:27.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.941+0000 7fdc05666740 1 client.7641 shutdown 2024-04-04T06:13:27.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.941+0000 7fdc05666740 1 -- 172.21.15.159:0/174145665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd6215ef30 msgr2=0x55fd621497c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:27.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.941+0000 7fdc05666740 1 --2- 172.21.15.159:0/174145665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd6215ef30 0x55fd621497c0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4000c00 tx=0x7fdbf4000ed0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.941+0000 7fdc05666740 1 -- 172.21.15.159:0/174145665 shutdown_connections 2024-04-04T06:13:27.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.941+0000 7fdc05666740 1 --2- 172.21.15.159:0/174145665 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55fd621c3bf0 0x55fd62149d00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.941+0000 7fdc05666740 1 --2- 172.21.15.159:0/174145665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd6215ef30 0x55fd621497c0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.941+0000 7fdc05666740 1 --2- 172.21.15.159:0/174145665 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fd62143960 0x55fd621634e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.941+0000 7fdc05666740 1 --2- 172.21.15.159:0/174145665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fd62143580 0x55fd621c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.941+0000 7fdbe8c00640 0 client.7641 ms_handle_reset on v2:172.21.15.159:6833/3796123128 2024-04-04T06:13:27.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.941+0000 7fdc05666740 1 -- 172.21.15.159:0/174145665 >> 172.21.15.159:0/174145665 conn(0x55fd62116a60 msgr2=0x55fd6213e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:27.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.942+0000 7fdc05666740 1 -- 172.21.15.159:0/174145665 shutdown_connections 2024-04-04T06:13:27.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.942+0000 7fdc05666740 1 -- 172.21.15.159:0/174145665 wait complete. 2024-04-04T06:13:27.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.942+0000 7fdc05666740 20 client.7641 trim_cache size 0 max 16384 2024-04-04T06:13:27.939 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:13:27.939 INFO:teuthology.orchestra.run.smithi159.stdout: "status": "no active scrubs running", 2024-04-04T06:13:27.939 INFO:teuthology.orchestra.run.smithi159.stdout: "scrubs": {} 2024-04-04T06:13:27.939 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:13:27.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.943+0000 7fdc02a00640 1 -- 172.21.15.159:0/2063269821 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdbd806c8f0 msgr2=0x7fdbd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:27.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.943+0000 7fdc02a00640 1 --2- 172.21.15.159:0/2063269821 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdbd806c8f0 0x7fdbd806edb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4007320 tx=0x7fdbf4007720 comp rx=0 tx=0).stop 2024-04-04T06:13:27.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.943+0000 7fdc02a00640 1 -- 172.21.15.159:0/2063269821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbfc0d4590 msgr2=0x7fdbfc1f5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:27.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.943+0000 7fdc02a00640 1 --2- 172.21.15.159:0/2063269821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbfc0d4590 0x7fdbfc1f5320 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4000f80 tx=0x7fdbe4004a40 comp rx=0 tx=0).stop 2024-04-04T06:13:27.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.943+0000 7fdc02a00640 1 -- 172.21.15.159:0/2063269821 shutdown_connections 2024-04-04T06:13:27.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.943+0000 7fdc02a00640 1 --2- 172.21.15.159:0/2063269821 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdbd806c8f0 0x7fdbd806edb0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.943+0000 7fdc02a00640 1 --2- 172.21.15.159:0/2063269821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbfc15e6d0 0x7fdbfc1f5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.943+0000 7fdc02a00640 1 --2- 172.21.15.159:0/2063269821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbfc0d4590 0x7fdbfc1f5320 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.943+0000 7fdc02a00640 1 --2- 172.21.15.159:0/2063269821 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbfc0d0cb0 0x7fdbfc1f1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:27.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.943+0000 7fdc02a00640 1 -- 172.21.15.159:0/2063269821 >> 172.21.15.159:0/2063269821 conn(0x7fdbfc09d7b0 msgr2=0x7fdbfc06d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:27.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.943+0000 7fdc02a00640 1 -- 172.21.15.159:0/2063269821 shutdown_connections 2024-04-04T06:13:27.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:27.943+0000 7fdc02a00640 1 -- 172.21.15.159:0/2063269821 wait complete. 2024-04-04T06:13:27.953 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-04T06:13:27.953 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:13:28.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.142+0000 7f4d36800640 1 Processor -- start 2024-04-04T06:13:28.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.142+0000 7f4d36800640 1 -- start start 2024-04-04T06:13:28.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.143+0000 7f4d36800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4d3015c390 0x7f4d30073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.143+0000 7f4d36800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d300af830 0x7f4d30160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.143+0000 7f4d36800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4d30160b80 0x7f4d300d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.143+0000 7f4d36800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f4d3015c770 con 0x7f4d30160b80 2024-04-04T06:13:28.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.143+0000 7f4d36800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4d3015fe80 con 0x7f4d3015c390 2024-04-04T06:13:28.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.143+0000 7f4d36800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4d300ad120 con 0x7f4d300af830 2024-04-04T06:13:28.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.144+0000 7f4d2fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4d30160b80 0x7f4d300d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.144+0000 7f4d2fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4d30160b80 0x7f4d300d0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38748/0 (socket says 172.21.15.159:38748) 2024-04-04T06:13:28.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.144+0000 7f4d2fe00640 1 -- 172.21.15.159:0/1549354575 learned_addr learned my addr 172.21.15.159:0/1549354575 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:28.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.144+0000 7f4d2ea00640 1 --2- 172.21.15.159:0/1549354575 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d300af830 0x7f4d30160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.144+0000 7f4d2f400640 1 --2- 172.21.15.159:0/1549354575 >> v2:172.21.15.178:3300/0 conn(0x7f4d3015c390 0x7f4d30073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.144+0000 7f4d2fe00640 1 -- 172.21.15.159:0/1549354575 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d300af830 msgr2=0x7f4d30160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:28.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.144+0000 7f4d2fe00640 1 --2- 172.21.15.159:0/1549354575 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d300af830 0x7f4d30160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.144+0000 7f4d2fe00640 1 -- 172.21.15.159:0/1549354575 >> v2:172.21.15.178:3300/0 conn(0x7f4d3015c390 msgr2=0x7f4d30073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:28.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.144+0000 7f4d2fe00640 1 --2- 172.21.15.159:0/1549354575 >> v2:172.21.15.178:3300/0 conn(0x7f4d3015c390 0x7f4d30073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.144+0000 7f4d2fe00640 1 -- 172.21.15.159:0/1549354575 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d300ad400 con 0x7f4d30160b80 2024-04-04T06:13:28.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.144+0000 7f4d2f400640 1 --2- 172.21.15.159:0/1549354575 >> v2:172.21.15.178:3300/0 conn(0x7f4d3015c390 0x7f4d30073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:13:28.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.144+0000 7f4d2ea00640 1 --2- 172.21.15.159:0/1549354575 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d300af830 0x7f4d30160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:13:28.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.145+0000 7f4d2fe00640 1 --2- 172.21.15.159:0/1549354575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d30160b80 0x7f4d300d0650 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f4d300b63e0 tx=0x7f4d2802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115f9c1df286ce25 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:28.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.146+0000 7f4d2e000640 1 -- 172.21.15.159:0/1549354575 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2802ca80 con 0x7f4d30160b80 2024-04-04T06:13:28.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.146+0000 7f4d2e000640 1 -- 172.21.15.159:0/1549354575 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d2802cc20 con 0x7f4d30160b80 2024-04-04T06:13:28.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.146+0000 7f4d2e000640 1 -- 172.21.15.159:0/1549354575 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d280343f0 con 0x7f4d30160b80 2024-04-04T06:13:28.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.146+0000 7f4d36800640 1 -- 172.21.15.159:0/1549354575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d30160b80 msgr2=0x7f4d300d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:28.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.146+0000 7f4d36800640 1 --2- 172.21.15.159:0/1549354575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d30160b80 0x7f4d300d0650 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f4d300b63e0 tx=0x7f4d2802bee0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.147+0000 7f4d36800640 1 -- 172.21.15.159:0/1549354575 shutdown_connections 2024-04-04T06:13:28.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.147+0000 7f4d36800640 1 --2- 172.21.15.159:0/1549354575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d30160b80 0x7f4d300d0650 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.147+0000 7f4d36800640 1 --2- 172.21.15.159:0/1549354575 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d300af830 0x7f4d30160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.147+0000 7f4d36800640 1 --2- 172.21.15.159:0/1549354575 >> v2:172.21.15.178:3300/0 conn(0x7f4d3015c390 0x7f4d30073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.147+0000 7f4d36800640 1 -- 172.21.15.159:0/1549354575 >> 172.21.15.159:0/1549354575 conn(0x7f4d3006b990 msgr2=0x7f4d3006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:28.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.147+0000 7f4d36800640 1 -- 172.21.15.159:0/1549354575 shutdown_connections 2024-04-04T06:13:28.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.147+0000 7f4d36800640 1 -- 172.21.15.159:0/1549354575 wait complete. 2024-04-04T06:13:28.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.148+0000 7f4d36800640 1 Processor -- start 2024-04-04T06:13:28.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.149+0000 7f4d36800640 1 -- start start 2024-04-04T06:13:28.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.149+0000 7f4d36800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d300af830 0x7f4d300b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.149+0000 7f4d36800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d3015c390 0x7f4d3014aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.149+0000 7f4d36800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d30160b80 0x7f4d3015eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.149+0000 7f4d36800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4d300cf980 con 0x7f4d3015c390 2024-04-04T06:13:28.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.149+0000 7f4d36800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4d30160120 con 0x7f4d30160b80 2024-04-04T06:13:28.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.149+0000 7f4d36800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4d300d0b90 con 0x7f4d300af830 2024-04-04T06:13:28.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.150+0000 7f4d2ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d3015c390 0x7f4d3014aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.150+0000 7f4d2ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d3015c390 0x7f4d3014aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38760/0 (socket says 172.21.15.159:38760) 2024-04-04T06:13:28.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.150+0000 7f4d2ea00640 1 -- 172.21.15.159:0/2799442173 learned_addr learned my addr 172.21.15.159:0/2799442173 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:28.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.150+0000 7f4d2f400640 1 --2- 172.21.15.159:0/2799442173 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d300af830 0x7f4d300b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.150+0000 7f4d2ea00640 1 -- 172.21.15.159:0/2799442173 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d300af830 msgr2=0x7f4d300b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:28.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.150+0000 7f4d2fe00640 1 --2- 172.21.15.159:0/2799442173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d30160b80 0x7f4d3015eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.150+0000 7f4d2ea00640 1 --2- 172.21.15.159:0/2799442173 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d300af830 0x7f4d300b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.150+0000 7f4d2ea00640 1 -- 172.21.15.159:0/2799442173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d30160b80 msgr2=0x7f4d3015eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:28.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.150+0000 7f4d2ea00640 1 --2- 172.21.15.159:0/2799442173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d30160b80 0x7f4d3015eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.150+0000 7f4d2ea00640 1 -- 172.21.15.159:0/2799442173 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d3015dfa0 con 0x7f4d3015c390 2024-04-04T06:13:28.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.150+0000 7f4d2f400640 1 --2- 172.21.15.159:0/2799442173 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d300af830 0x7f4d300b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:13:28.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.151+0000 7f4d2ea00640 1 --2- 172.21.15.159:0/2799442173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d3015c390 0x7f4d3014aa90 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c00c820 tx=0x7f4d1c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:28.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.151+0000 7f4d27400640 1 -- 172.21.15.159:0/2799442173 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c015070 con 0x7f4d3015c390 2024-04-04T06:13:28.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.151+0000 7f4d27400640 1 -- 172.21.15.159:0/2799442173 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d1c007a30 con 0x7f4d3015c390 2024-04-04T06:13:28.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.151+0000 7f4d36800640 1 -- 172.21.15.159:0/2799442173 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d3014b590 con 0x7f4d3015c390 2024-04-04T06:13:28.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.151+0000 7f4d27400640 1 -- 172.21.15.159:0/2799442173 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c007e00 con 0x7f4d3015c390 2024-04-04T06:13:28.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.151+0000 7f4d36800640 1 -- 172.21.15.159:0/2799442173 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d300b0b80 con 0x7f4d3015c390 2024-04-04T06:13:28.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.152+0000 7f4d36800640 1 -- 172.21.15.159:0/2799442173 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d300685c0 con 0x7f4d3015c390 2024-04-04T06:13:28.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.153+0000 7f4d27400640 1 -- 172.21.15.159:0/2799442173 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d1c018070 con 0x7f4d3015c390 2024-04-04T06:13:28.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.154+0000 7f4d27400640 1 --2- 172.21.15.159:0/2799442173 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4d1006c8f0 0x7f4d1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.154+0000 7f4d27400640 1 -- 172.21.15.159:0/2799442173 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f4d1c08bc20 con 0x7f4d3015c390 2024-04-04T06:13:28.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.154+0000 7f4d2f400640 1 --2- 172.21.15.159:0/2799442173 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4d1006c8f0 0x7f4d1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.156+0000 7f4d2f400640 1 --2- 172.21.15.159:0/2799442173 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4d1006c8f0 0x7f4d1006edb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4d200088b0 tx=0x7f4d20008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:28.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.158+0000 7f4d27400640 1 -- 172.21.15.159:0/2799442173 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4d1c059f50 con 0x7f4d3015c390 2024-04-04T06:13:28.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.397+0000 7f4d36800640 1 -- 172.21.15.159:0/2799442173 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d3006f1b0 con 0x7f4d3015c390 2024-04-04T06:13:28.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.398+0000 7f4d27400640 1 -- 172.21.15.159:0/2799442173 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5196 (secure 0 0 0) 0x7f4d1c059850 con 0x7f4d3015c390 2024-04-04T06:13:28.392 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:13:28.392 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1825651864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1825651864},{"type":"v1","addr":"172.21.15.178:6836","nonce":1825651864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1824578712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1824578712},{"type":"v1","addr":"172.21.15.178:6837","nonce":1824578712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6834/3377498298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3377498298},{"type":"v1","addr":"172.21.15.159:6834","nonce":3377498298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:12:35.880454+0000","modified":"2024-04-04T06:13:00.228456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7512":{"gid":7512,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":6,"addr":"172.21.15.159:6835/3796123128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":3796123128},{"type":"v1","addr":"172.21.15.159:6835","nonce":3796123128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7512,"qdb_cluster":[7512]},"id":6}]} 2024-04-04T06:13:28.392 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 69 2024-04-04T06:13:28.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.401+0000 7f4d36800640 1 -- 172.21.15.159:0/2799442173 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4d1006c8f0 msgr2=0x7f4d1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:28.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.401+0000 7f4d36800640 1 --2- 172.21.15.159:0/2799442173 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4d1006c8f0 0x7f4d1006edb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4d200088b0 tx=0x7f4d20008040 comp rx=0 tx=0).stop 2024-04-04T06:13:28.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.401+0000 7f4d36800640 1 -- 172.21.15.159:0/2799442173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d3015c390 msgr2=0x7f4d3014aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:28.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.401+0000 7f4d36800640 1 --2- 172.21.15.159:0/2799442173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d3015c390 0x7f4d3014aa90 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c00c820 tx=0x7f4d1c00cc20 comp rx=0 tx=0).stop 2024-04-04T06:13:28.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.401+0000 7f4d36800640 1 -- 172.21.15.159:0/2799442173 shutdown_connections 2024-04-04T06:13:28.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.401+0000 7f4d36800640 1 --2- 172.21.15.159:0/2799442173 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4d1006c8f0 0x7f4d1006edb0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.401+0000 7f4d36800640 1 --2- 172.21.15.159:0/2799442173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d30160b80 0x7f4d3015eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.401+0000 7f4d36800640 1 --2- 172.21.15.159:0/2799442173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d3015c390 0x7f4d3014aa90 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.401+0000 7f4d36800640 1 --2- 172.21.15.159:0/2799442173 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d300af830 0x7f4d300b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.401+0000 7f4d36800640 1 -- 172.21.15.159:0/2799442173 >> 172.21.15.159:0/2799442173 conn(0x7f4d3006b990 msgr2=0x7f4d3006e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:28.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.401+0000 7f4d36800640 1 -- 172.21.15.159:0/2799442173 shutdown_connections 2024-04-04T06:13:28.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.401+0000 7f4d36800640 1 -- 172.21.15.159:0/2799442173 wait complete. 2024-04-04T06:13:28.412 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-04T06:13:28.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.610+0000 7f1234400640 1 Processor -- start 2024-04-04T06:13:28.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.611+0000 7f1234400640 1 -- start start 2024-04-04T06:13:28.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.612+0000 7f1234400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f122c153580 0x7f122c0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.612+0000 7f1234400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f122c0d07c0 0x7f122c0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.612+0000 7f1234400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f122c0afab0 con 0x7f122c0af2e0 2024-04-04T06:13:28.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.612+0000 7f1234400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f122c0d0c00 con 0x7f122c153580 2024-04-04T06:13:28.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.612+0000 7f1234400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f122c15cc80 con 0x7f122c0d07c0 2024-04-04T06:13:28.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.612+0000 7f1231a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f122c153580 0x7f122c0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.613+0000 7f1231a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f122c153580 0x7f122c0d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38764/0 (socket says 172.21.15.159:38764) 2024-04-04T06:13:28.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.613+0000 7f1231a00640 1 -- 172.21.15.159:0/4097466972 learned_addr learned my addr 172.21.15.159:0/4097466972 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:28.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.613+0000 7f1231000640 1 --2- 172.21.15.159:0/4097466972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f122c0d07c0 0x7f122c0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.613+0000 7f1231a00640 1 -- 172.21.15.159:0/4097466972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f122c0d07c0 msgr2=0x7f122c0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:28.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.613+0000 7f1231a00640 1 --2- 172.21.15.159:0/4097466972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f122c0d07c0 0x7f122c0aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.613+0000 7f1231a00640 1 -- 172.21.15.159:0/4097466972 >> v1:172.21.15.178:6789/0 conn(0x7f122c0af2e0 legacy=0x7f122c06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:28.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.613+0000 7f1231a00640 1 -- 172.21.15.159:0/4097466972 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122c15da00 con 0x7f122c153580 2024-04-04T06:13:28.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.614+0000 7f1231a00640 1 --2- 172.21.15.159:0/4097466972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f122c153580 0x7f122c0d0280 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f1220009870 tx=0x7f122002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68ff28d0ebd9fec server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:28.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.615+0000 7f122be00640 1 -- 172.21.15.159:0/4097466972 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220035020 con 0x7f122c153580 2024-04-04T06:13:28.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.615+0000 7f122be00640 1 -- 172.21.15.159:0/4097466972 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1220031640 con 0x7f122c153580 2024-04-04T06:13:28.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.615+0000 7f122be00640 1 -- 172.21.15.159:0/4097466972 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122002d040 con 0x7f122c153580 2024-04-04T06:13:28.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.615+0000 7f1234400640 1 -- 172.21.15.159:0/4097466972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f122c153580 msgr2=0x7f122c0d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:28.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.615+0000 7f1234400640 1 --2- 172.21.15.159:0/4097466972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f122c153580 0x7f122c0d0280 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f1220009870 tx=0x7f122002b8f0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.615+0000 7f1234400640 1 -- 172.21.15.159:0/4097466972 shutdown_connections 2024-04-04T06:13:28.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.615+0000 7f1234400640 1 --2- 172.21.15.159:0/4097466972 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f122c0d07c0 0x7f122c0aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.615+0000 7f1234400640 1 --2- 172.21.15.159:0/4097466972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f122c153580 0x7f122c0d0280 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.615+0000 7f1234400640 1 -- 172.21.15.159:0/4097466972 >> 172.21.15.159:0/4097466972 conn(0x7f122c06b750 msgr2=0x7f122c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:28.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.615+0000 7f1234400640 1 -- 172.21.15.159:0/4097466972 shutdown_connections 2024-04-04T06:13:28.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.615+0000 7f1234400640 1 -- 172.21.15.159:0/4097466972 wait complete. 2024-04-04T06:13:28.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.617+0000 7f1234400640 1 Processor -- start 2024-04-04T06:13:28.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.617+0000 7f1234400640 1 -- start start 2024-04-04T06:13:28.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.618+0000 7f1234400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f122c0af2e0 0x7f122c1f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.618+0000 7f1234400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f122c0d07c0 0x7f122c158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.618+0000 7f1234400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f122c153580 0x7f122c073190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.618+0000 7f1234400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f122c15d4e0 con 0x7f122c153580 2024-04-04T06:13:28.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.618+0000 7f1234400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f122c15d200 con 0x7f122c0d07c0 2024-04-04T06:13:28.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.618+0000 7f1234400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f122c153960 con 0x7f122c0af2e0 2024-04-04T06:13:28.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.618+0000 7f1232400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f122c153580 0x7f122c073190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.618+0000 7f1232400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f122c153580 0x7f122c073190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38768/0 (socket says 172.21.15.159:38768) 2024-04-04T06:13:28.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.618+0000 7f1232400640 1 -- 172.21.15.159:0/2576135401 learned_addr learned my addr 172.21.15.159:0/2576135401 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:28.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.619+0000 7f1231a00640 1 --2- 172.21.15.159:0/2576135401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f122c0af2e0 0x7f122c1f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.619+0000 7f1231000640 1 --2- 172.21.15.159:0/2576135401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f122c0d07c0 0x7f122c158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.619+0000 7f1232400640 1 -- 172.21.15.159:0/2576135401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f122c0af2e0 msgr2=0x7f122c1f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:28.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.619+0000 7f1232400640 1 --2- 172.21.15.159:0/2576135401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f122c0af2e0 0x7f122c1f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.619+0000 7f1232400640 1 -- 172.21.15.159:0/2576135401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f122c0d07c0 msgr2=0x7f122c158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:28.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.619+0000 7f1232400640 1 --2- 172.21.15.159:0/2576135401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f122c0d07c0 0x7f122c158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.619+0000 7f1232400640 1 -- 172.21.15.159:0/2576135401 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122c1f4310 con 0x7f122c153580 2024-04-04T06:13:28.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.619+0000 7f1232400640 1 --2- 172.21.15.159:0/2576135401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f122c153580 0x7f122c073190 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f122400c6b0 tx=0x7f122400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:28.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.619+0000 7f1229600640 1 -- 172.21.15.159:0/2576135401 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224003b70 con 0x7f122c153580 2024-04-04T06:13:28.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.619+0000 7f1229600640 1 -- 172.21.15.159:0/2576135401 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1224003d30 con 0x7f122c153580 2024-04-04T06:13:28.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.619+0000 7f1234400640 1 -- 172.21.15.159:0/2576135401 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122c159570 con 0x7f122c153580 2024-04-04T06:13:28.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.620+0000 7f1229600640 1 -- 172.21.15.159:0/2576135401 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224005310 con 0x7f122c153580 2024-04-04T06:13:28.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.620+0000 7f1234400640 1 -- 172.21.15.159:0/2576135401 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f122c1f45d0 con 0x7f122c153580 2024-04-04T06:13:28.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.622+0000 7f1229600640 1 -- 172.21.15.159:0/2576135401 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f12240054b0 con 0x7f122c153580 2024-04-04T06:13:28.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.622+0000 7f1229600640 1 --2- 172.21.15.159:0/2576135401 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f120c06c8f0 0x7f120c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.623+0000 7f1229600640 1 -- 172.21.15.159:0/2576135401 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f122408ba50 con 0x7f122c153580 2024-04-04T06:13:28.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.623+0000 7f1231a00640 1 --2- 172.21.15.159:0/2576135401 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f120c06c8f0 0x7f120c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.623+0000 7f1237171740 1 Processor -- start 2024-04-04T06:13:28.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.624+0000 7f1237171740 1 -- start start 2024-04-04T06:13:28.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.624+0000 7f1231a00640 1 --2- 172.21.15.159:0/2576135401 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f120c06c8f0 0x7f120c06edb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1220036000 tx=0x7f122003c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:28.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.624+0000 7f1237171740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c685cb6e0 0x563c685cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.624+0000 7f1237171740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c6859b6b0 0x563c685c8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.624+0000 7f1237171740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c6859f260 0x563c685c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.624+0000 7f1237171740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563c685055c0 con 0x563c685cb6e0 2024-04-04T06:13:28.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.624+0000 7f1237171740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563c684ad3b0 con 0x563c6859b6b0 2024-04-04T06:13:28.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.624+0000 7f1237171740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x563c68501570 con 0x563c6859f260 2024-04-04T06:13:28.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.625+0000 7f1231a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c685cb6e0 0x563c685cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.625+0000 7f1231a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c685cb6e0 0x563c685cbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38776/0 (socket says 172.21.15.159:38776) 2024-04-04T06:13:28.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.625+0000 7f1231a00640 1 -- 172.21.15.159:0/3666308615 learned_addr learned my addr 172.21.15.159:0/3666308615 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:28.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.625+0000 7f1231a00640 1 -- 172.21.15.159:0/3666308615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c6859f260 msgr2=0x563c685c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:28.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.625+0000 7f1231a00640 1 --2- 172.21.15.159:0/3666308615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c6859f260 0x563c685c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.625+0000 7f1231000640 1 --2- 172.21.15.159:0/3666308615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c6859b6b0 0x563c685c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.625+0000 7f1231a00640 1 -- 172.21.15.159:0/3666308615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c6859b6b0 msgr2=0x563c685c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:28.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.625+0000 7f1231a00640 1 --2- 172.21.15.159:0/3666308615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c6859b6b0 0x563c685c8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.625+0000 7f1231a00640 1 -- 172.21.15.159:0/3666308615 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c682e57c0 con 0x563c685cb6e0 2024-04-04T06:13:28.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.625+0000 7f1231a00640 1 --2- 172.21.15.159:0/3666308615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c685cb6e0 0x563c685cbac0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f1220009630 tx=0x7f1220030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ebf53e88b63688 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:28.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.626+0000 7f121e000640 1 -- 172.21.15.159:0/3666308615 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220035040 con 0x563c685cb6e0 2024-04-04T06:13:28.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.626+0000 7f121e000640 1 -- 172.21.15.159:0/3666308615 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f122003a5a0 con 0x563c685cb6e0 2024-04-04T06:13:28.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.626+0000 7f121e000640 1 -- 172.21.15.159:0/3666308615 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122003a950 con 0x563c685cb6e0 2024-04-04T06:13:28.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.626+0000 7f1237171740 1 -- 172.21.15.159:0/3666308615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c685cb6e0 msgr2=0x563c685cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:28.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.626+0000 7f1237171740 1 --2- 172.21.15.159:0/3666308615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c685cb6e0 0x563c685cbac0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f1220009630 tx=0x7f1220030000 comp rx=0 tx=0).stop 2024-04-04T06:13:28.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.626+0000 7f1237171740 1 -- 172.21.15.159:0/3666308615 shutdown_connections 2024-04-04T06:13:28.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.626+0000 7f1237171740 1 --2- 172.21.15.159:0/3666308615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c6859f260 0x563c685c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.626+0000 7f1237171740 1 --2- 172.21.15.159:0/3666308615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c6859b6b0 0x563c685c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.626+0000 7f1237171740 1 --2- 172.21.15.159:0/3666308615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c685cb6e0 0x563c685cbac0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.626+0000 7f1237171740 1 -- 172.21.15.159:0/3666308615 >> 172.21.15.159:0/3666308615 conn(0x563c6859d970 msgr2=0x563c685e64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:28.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.627+0000 7f1237171740 1 -- 172.21.15.159:0/3666308615 shutdown_connections 2024-04-04T06:13:28.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.627+0000 7f1237171740 1 -- 172.21.15.159:0/3666308615 wait complete. 2024-04-04T06:13:28.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.628+0000 7f1237171740 1 -- start start 2024-04-04T06:13:28.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.628+0000 7f1237171740 1 Processor -- start 2024-04-04T06:13:28.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.629+0000 7f1237171740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c6859f260 0x563c685cc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.630+0000 7f1237171740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c685cb6e0 0x563c685c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.630+0000 7f1237171740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c685ba050 0x563c685c7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.630+0000 7f1237171740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563c685a0410 con 0x563c6859f260 2024-04-04T06:13:28.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.630+0000 7f1237171740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563c684c07a0 con 0x563c685ba050 2024-04-04T06:13:28.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.630+0000 7f1237171740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x563c6859f800 con 0x563c685cb6e0 2024-04-04T06:13:28.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.630+0000 7f1237171740 10 client.0 _ll_register_callbacks cb 0x563c684fcfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:13:28.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.630+0000 7f1231a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c6859f260 0x563c685cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.630+0000 7f1231a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c6859f260 0x563c685cc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38780/0 (socket says 172.21.15.159:38780) 2024-04-04T06:13:28.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.630+0000 7f1231a00640 1 -- 172.21.15.159:0/2129965536 learned_addr learned my addr 172.21.15.159:0/2129965536 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:28.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.630+0000 7f1231a00640 1 -- 172.21.15.159:0/2129965536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c685cb6e0 msgr2=0x563c685c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:28.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.630+0000 7f1232400640 1 --2- 172.21.15.159:0/2129965536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c685ba050 0x563c685c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.630+0000 7f1231000640 1 --2- 172.21.15.159:0/2129965536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c685cb6e0 0x563c685c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.630+0000 7f1231a00640 1 --2- 172.21.15.159:0/2129965536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c685cb6e0 0x563c685c6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.630+0000 7f1231a00640 1 -- 172.21.15.159:0/2129965536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c685ba050 msgr2=0x563c685c7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:28.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.630+0000 7f1231a00640 1 --2- 172.21.15.159:0/2129965536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c685ba050 0x563c685c7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.630+0000 7f1231a00640 1 -- 172.21.15.159:0/2129965536 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c68345f50 con 0x563c6859f260 2024-04-04T06:13:28.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.630+0000 7f1232400640 1 --2- 172.21.15.159:0/2129965536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c685ba050 0x563c685c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:13:28.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.631+0000 7f1231a00640 1 --2- 172.21.15.159:0/2129965536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c6859f260 0x563c685cc020 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f122002bf00 tx=0x7f1220004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40c68ffa554825a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:28.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.631+0000 7f1212a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:13:28.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.631+0000 7f1212a00640 1 -- 172.21.15.159:0/2129965536 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220050680 con 0x563c6859f260 2024-04-04T06:13:28.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.631+0000 7f1212a00640 1 -- 172.21.15.159:0/2129965536 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f1220050820 con 0x563c6859f260 2024-04-04T06:13:28.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.631+0000 7f1237171740 1 -- 172.21.15.159:0/2129965536 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f122003a5a0 con 0x563c6859f260 2024-04-04T06:13:28.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.631+0000 7f1212a00640 1 -- 172.21.15.159:0/2129965536 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f1220050a20 con 0x563c6859f260 2024-04-04T06:13:28.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.631+0000 7f1237171740 10 client.7659 fetch_fsmap learned FSMap version 69 2024-04-04T06:13:28.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.631+0000 7f1237171740 1 -- 172.21.15.159:0/2129965536 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x563c681d90b0 con 0x563c6859f260 2024-04-04T06:13:28.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.632+0000 7f1212a00640 1 -- 172.21.15.159:0/2129965536 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f122003a5a0 con 0x563c6859f260 2024-04-04T06:13:28.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.632+0000 7f1212a00640 1 -- 172.21.15.159:0/2129965536 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 69) ==== 3494+0+0 (secure 0 0 0) 0x7f1220050bc0 con 0x563c6859f260 2024-04-04T06:13:28.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.632+0000 7f1237171740 10 client.7659 fetch_fsmap finished waiting for FSMap version 69 2024-04-04T06:13:28.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.632+0000 7f1237171740 10 client.7659 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:13:28.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.632+0000 7f1237171740 20 client.7659 populate_metadata read hostname 'smithi159' 2024-04-04T06:13:28.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.632+0000 7f1237171740 1 --2- 172.21.15.159:0/2129965536 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x563c685c3060 0x563c685c3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.632+0000 7f1237171740 4 client.7659 mds_command: new command op to 7512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:13:28.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.632+0000 7f1237171740 1 -- 172.21.15.159:0/2129965536 --> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c684d9890 con 0x563c685c3060 2024-04-04T06:13:28.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.632+0000 7f1231000640 1 --2- 172.21.15.159:0/2129965536 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x563c685c3060 0x563c685c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.633+0000 7f1231000640 1 --2- 172.21.15.159:0/2129965536 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x563c685c3060 0x563c685c3440 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10f82f91e815c943 server_cookie=7b6dbd40d592bea4 in_seq=0 out_seq=0 2024-04-04T06:13:28.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.633+0000 7f1212a00640 10 client.7659 ms_handle_connect on v2:172.21.15.159:6833/3796123128 2024-04-04T06:13:28.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.633+0000 7f1212a00640 1 -- 172.21.15.159:0/2129965536 <== mds.0 v2:172.21.15.159:6833/3796123128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c684d9890 con 0x563c685c3060 2024-04-04T06:13:28.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.633+0000 7f1212a00640 10 client.7659 handle_command_reply: tid=0 2024-04-04T06:13:28.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.633+0000 7f1237171740 1 client.7659 shutdown 2024-04-04T06:13:28.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.634+0000 7f1237171740 1 -- 172.21.15.159:0/2129965536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c6859f260 msgr2=0x563c685cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:28.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.634+0000 7f1237171740 1 --2- 172.21.15.159:0/2129965536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c6859f260 0x563c685cc020 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f122002bf00 tx=0x7f1220004730 comp rx=0 tx=0).stop 2024-04-04T06:13:28.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.634+0000 7f1237171740 1 -- 172.21.15.159:0/2129965536 shutdown_connections 2024-04-04T06:13:28.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.634+0000 7f1237171740 1 --2- 172.21.15.159:0/2129965536 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x563c685c3060 0x563c685c3440 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.634+0000 7f1237171740 1 --2- 172.21.15.159:0/2129965536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c685ba050 0x563c685c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.634+0000 7f1237171740 1 --2- 172.21.15.159:0/2129965536 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c685cb6e0 0x563c685c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.634+0000 7f1237171740 1 --2- 172.21.15.159:0/2129965536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c6859f260 0x563c685cc020 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.634+0000 7f1237171740 1 -- 172.21.15.159:0/2129965536 >> 172.21.15.159:0/2129965536 conn(0x563c6859b6b0 msgr2=0x563c685e7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:28.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.634+0000 7f1212a00640 0 client.7659 ms_handle_reset on v2:172.21.15.159:6833/3796123128 2024-04-04T06:13:28.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.634+0000 7f1237171740 1 -- 172.21.15.159:0/2129965536 shutdown_connections 2024-04-04T06:13:28.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.634+0000 7f1237171740 1 -- 172.21.15.159:0/2129965536 wait complete. 2024-04-04T06:13:28.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.634+0000 7f1237171740 20 client.7659 trim_cache size 0 max 16384 2024-04-04T06:13:28.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.657+0000 7f1237171740 1 Processor -- start 2024-04-04T06:13:28.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.657+0000 7f1237171740 1 -- start start 2024-04-04T06:13:28.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.658+0000 7f1237171740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c6859f260 0x563c685c3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.658+0000 7f1237171740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c685ba050 0x563c685c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.658+0000 7f1237171740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c685cb6e0 0x563c685c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.658+0000 7f1237171740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f122c1ffe30 con 0x563c685cb6e0 2024-04-04T06:13:28.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.658+0000 7f1237171740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f122c1f86c0 con 0x563c685ba050 2024-04-04T06:13:28.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.658+0000 7f1237171740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f122c072000 con 0x563c6859f260 2024-04-04T06:13:28.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.658+0000 7f1232400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c685cb6e0 0x563c685c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.658+0000 7f1232400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c685cb6e0 0x563c685c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38786/0 (socket says 172.21.15.159:38786) 2024-04-04T06:13:28.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.658+0000 7f1232400640 1 -- 172.21.15.159:0/3190157817 learned_addr learned my addr 172.21.15.159:0/3190157817 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:28.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.659+0000 7f1232400640 1 -- 172.21.15.159:0/3190157817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c6859f260 msgr2=0x563c685c3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:28.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.659+0000 7f1232400640 1 --2- 172.21.15.159:0/3190157817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c6859f260 0x563c685c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.659+0000 7f1232400640 1 -- 172.21.15.159:0/3190157817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c685ba050 msgr2=0x563c685c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:28.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.659+0000 7f1232400640 1 --2- 172.21.15.159:0/3190157817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c685ba050 0x563c685c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.659+0000 7f1232400640 1 -- 172.21.15.159:0/3190157817 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c685cea30 con 0x563c685cb6e0 2024-04-04T06:13:28.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.659+0000 7f1232400640 1 --2- 172.21.15.159:0/3190157817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c685cb6e0 0x563c685c83c0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f1224061dc0 tx=0x7f1224011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f216d7f1ab246f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:28.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.659+0000 7f121d600640 1 -- 172.21.15.159:0/3190157817 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224059710 con 0x563c685cb6e0 2024-04-04T06:13:28.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.659+0000 7f121d600640 1 -- 172.21.15.159:0/3190157817 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f122405dac0 con 0x563c685cb6e0 2024-04-04T06:13:28.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.659+0000 7f121d600640 1 -- 172.21.15.159:0/3190157817 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12240622f0 con 0x563c685cb6e0 2024-04-04T06:13:28.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.660+0000 7f1237171740 1 -- 172.21.15.159:0/3190157817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c685cb6e0 msgr2=0x563c685c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:28.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.660+0000 7f1237171740 1 --2- 172.21.15.159:0/3190157817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c685cb6e0 0x563c685c83c0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f1224061dc0 tx=0x7f1224011000 comp rx=0 tx=0).stop 2024-04-04T06:13:28.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.660+0000 7f1237171740 1 -- 172.21.15.159:0/3190157817 shutdown_connections 2024-04-04T06:13:28.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.660+0000 7f1237171740 1 --2- 172.21.15.159:0/3190157817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c685cb6e0 0x563c685c83c0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.660+0000 7f1237171740 1 --2- 172.21.15.159:0/3190157817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c685ba050 0x563c685c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.660+0000 7f1237171740 1 --2- 172.21.15.159:0/3190157817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c6859f260 0x563c685c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.660+0000 7f1237171740 1 -- 172.21.15.159:0/3190157817 >> 172.21.15.159:0/3190157817 conn(0x563c6859b6b0 msgr2=0x563c6859a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:28.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.660+0000 7f1237171740 1 -- 172.21.15.159:0/3190157817 shutdown_connections 2024-04-04T06:13:28.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.660+0000 7f1237171740 1 -- 172.21.15.159:0/3190157817 wait complete. 2024-04-04T06:13:28.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.660+0000 7f1237171740 1 -- start start 2024-04-04T06:13:28.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.661+0000 7f1237171740 1 Processor -- start 2024-04-04T06:13:28.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.662+0000 7f1237171740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c6859f260 0x563c6865f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.662+0000 7f1237171740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c685ba050 0x563c685c1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.662+0000 7f1237171740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c685cb6e0 0x563c685c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.662+0000 7f1237171740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563c685c3f70 con 0x563c6859f260 2024-04-04T06:13:28.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.662+0000 7f1237171740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563c685cd310 con 0x563c685ba050 2024-04-04T06:13:28.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.662+0000 7f1237171740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x563c685cf0d0 con 0x563c685cb6e0 2024-04-04T06:13:28.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.662+0000 7f1237171740 10 client.0 _ll_register_callbacks cb 0x7f12200318d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:13:28.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.662+0000 7f1231a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c6859f260 0x563c6865f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.662+0000 7f1231a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c6859f260 0x563c6865f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38788/0 (socket says 172.21.15.159:38788) 2024-04-04T06:13:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.662+0000 7f1231a00640 1 -- 172.21.15.159:0/1176070913 learned_addr learned my addr 172.21.15.159:0/1176070913 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.662+0000 7f1231000640 1 --2- 172.21.15.159:0/1176070913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c685ba050 0x563c685c1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.662+0000 7f1232400640 1 --2- 172.21.15.159:0/1176070913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c685cb6e0 0x563c685c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.662+0000 7f1231a00640 1 -- 172.21.15.159:0/1176070913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c685cb6e0 msgr2=0x563c685c1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.662+0000 7f1231a00640 1 --2- 172.21.15.159:0/1176070913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c685cb6e0 0x563c685c1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.662+0000 7f1231a00640 1 -- 172.21.15.159:0/1176070913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c685ba050 msgr2=0x563c685c1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.662+0000 7f1231a00640 1 --2- 172.21.15.159:0/1176070913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c685ba050 0x563c685c1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.662+0000 7f1231a00640 1 -- 172.21.15.159:0/1176070913 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c685cbd10 con 0x563c6859f260 2024-04-04T06:13:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.663+0000 7f1231a00640 1 --2- 172.21.15.159:0/1176070913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c6859f260 0x563c6865f810 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f1220009770 tx=0x7f12200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da8f28206eeffa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.663+0000 7f1212a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:13:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.663+0000 7f1212a00640 1 -- 172.21.15.159:0/1176070913 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220031e60 con 0x563c6859f260 2024-04-04T06:13:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.663+0000 7f1212a00640 1 -- 172.21.15.159:0/1176070913 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f1220050680 con 0x563c6859f260 2024-04-04T06:13:28.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.663+0000 7f1237171740 1 -- 172.21.15.159:0/1176070913 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f122405dac0 con 0x563c6859f260 2024-04-04T06:13:28.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.663+0000 7f1212a00640 1 -- 172.21.15.159:0/1176070913 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f1220066080 con 0x563c6859f260 2024-04-04T06:13:28.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.663+0000 7f1237171740 10 client.7665 fetch_fsmap learned FSMap version 69 2024-04-04T06:13:28.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.663+0000 7f1237171740 1 -- 172.21.15.159:0/1176070913 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x563c685bdc70 con 0x563c6859f260 2024-04-04T06:13:28.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.663+0000 7f1212a00640 1 -- 172.21.15.159:0/1176070913 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f122405dac0 con 0x563c6859f260 2024-04-04T06:13:28.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.664+0000 7f1212a00640 1 -- 172.21.15.159:0/1176070913 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 69) ==== 3494+0+0 (secure 0 0 0) 0x7f122005b780 con 0x563c6859f260 2024-04-04T06:13:28.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.664+0000 7f1237171740 10 client.7665 fetch_fsmap finished waiting for FSMap version 69 2024-04-04T06:13:28.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.664+0000 7f1237171740 10 client.7665 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:13:28.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.664+0000 7f1237171740 20 client.7665 populate_metadata read hostname 'smithi159' 2024-04-04T06:13:28.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.664+0000 7f1237171740 1 --2- 172.21.15.159:0/1176070913 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x563c685b9530 0x563c685a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.664+0000 7f1237171740 4 client.7665 mds_command: new command op to 7512 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-04T06:13:28.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.664+0000 7f1237171740 1 -- 172.21.15.159:0/1176070913 --> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x563c68512b00 con 0x563c685b9530 2024-04-04T06:13:28.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.664+0000 7f1231000640 1 --2- 172.21.15.159:0/1176070913 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x563c685b9530 0x563c685a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.665+0000 7f1212a00640 10 client.7665 ms_handle_connect on v2:172.21.15.159:6833/3796123128 2024-04-04T06:13:28.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.665+0000 7f1231000640 1 --2- 172.21.15.159:0/1176070913 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x563c685b9530 0x563c685a04d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0e2be95891449ef server_cookie=53b2b3b9a3a13c92 in_seq=0 out_seq=0 2024-04-04T06:13:28.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.665+0000 7f1212a00640 1 -- 172.21.15.159:0/1176070913 <== mds.0 v2:172.21.15.159:6833/3796123128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+419 (crc 0 0 0) 0x563c68512b00 con 0x563c685b9530 2024-04-04T06:13:28.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.665+0000 7f1212a00640 10 client.7665 handle_command_reply: tid=0 2024-04-04T06:13:28.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.665+0000 7f1237171740 1 client.7665 shutdown 2024-04-04T06:13:28.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.665+0000 7f1237171740 1 -- 172.21.15.159:0/1176070913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c6859f260 msgr2=0x563c6865f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:28.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.665+0000 7f1237171740 1 --2- 172.21.15.159:0/1176070913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c6859f260 0x563c6865f810 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f1220009770 tx=0x7f12200046c0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.665+0000 7f1237171740 1 -- 172.21.15.159:0/1176070913 shutdown_connections 2024-04-04T06:13:28.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.665+0000 7f1237171740 1 --2- 172.21.15.159:0/1176070913 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x563c685b9530 0x563c685a04d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.665+0000 7f1237171740 1 --2- 172.21.15.159:0/1176070913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563c685cb6e0 0x563c685c1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.665+0000 7f1237171740 1 --2- 172.21.15.159:0/1176070913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c685ba050 0x563c685c1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.665+0000 7f1237171740 1 --2- 172.21.15.159:0/1176070913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c6859f260 0x563c6865f810 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.665+0000 7f1212a00640 0 client.7665 ms_handle_reset on v2:172.21.15.159:6833/3796123128 2024-04-04T06:13:28.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.665+0000 7f1237171740 1 -- 172.21.15.159:0/1176070913 >> 172.21.15.159:0/1176070913 conn(0x563c6859b6b0 msgr2=0x563c685e7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:28.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.665+0000 7f1237171740 1 -- 172.21.15.159:0/1176070913 shutdown_connections 2024-04-04T06:13:28.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.665+0000 7f1237171740 1 -- 172.21.15.159:0/1176070913 wait complete. 2024-04-04T06:13:28.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.665+0000 7f1237171740 20 client.7665 trim_cache size 0 max 16384 2024-04-04T06:13:28.663 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:13:28.663 INFO:teuthology.orchestra.run.smithi159.stdout:[ 2024-04-04T06:13:28.663 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:13:28.663 INFO:teuthology.orchestra.run.smithi159.stdout: "damage_type": "backtrace", 2024-04-04T06:13:28.663 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 465334627, 2024-04-04T06:13:28.663 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 1099511627777, 2024-04-04T06:13:28.663 INFO:teuthology.orchestra.run.smithi159.stdout: "path": "/testdir/hardlink" 2024-04-04T06:13:28.663 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:13:28.664 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:13:28.664 INFO:teuthology.orchestra.run.smithi159.stdout: "damage_type": "dir_frag", 2024-04-04T06:13:28.664 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 2568503133, 2024-04-04T06:13:28.664 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 1099511627776, 2024-04-04T06:13:28.664 INFO:teuthology.orchestra.run.smithi159.stdout: "frag": "*", 2024-04-04T06:13:28.664 INFO:teuthology.orchestra.run.smithi159.stdout: "path": "/dir" 2024-04-04T06:13:28.664 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:13:28.664 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:13:28.664 INFO:teuthology.orchestra.run.smithi159.stdout: "damage_type": "backtrace", 2024-04-04T06:13:28.664 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 3265784033, 2024-04-04T06:13:28.664 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 1099511627776, 2024-04-04T06:13:28.664 INFO:teuthology.orchestra.run.smithi159.stdout: "path": "/dir" 2024-04-04T06:13:28.664 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:13:28.665 INFO:teuthology.orchestra.run.smithi159.stdout:] 2024-04-04T06:13:28.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.666+0000 7f1234400640 1 -- 172.21.15.159:0/2576135401 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f120c06c8f0 msgr2=0x7f120c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:28.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.666+0000 7f1234400640 1 --2- 172.21.15.159:0/2576135401 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f120c06c8f0 0x7f120c06edb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1220036000 tx=0x7f122003c040 comp rx=0 tx=0).stop 2024-04-04T06:13:28.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.666+0000 7f1234400640 1 -- 172.21.15.159:0/2576135401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f122c153580 msgr2=0x7f122c073190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:28.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.666+0000 7f1234400640 1 --2- 172.21.15.159:0/2576135401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f122c153580 0x7f122c073190 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f122400c6b0 tx=0x7f122400cb70 comp rx=0 tx=0).stop 2024-04-04T06:13:28.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.666+0000 7f1234400640 1 -- 172.21.15.159:0/2576135401 shutdown_connections 2024-04-04T06:13:28.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.666+0000 7f1234400640 1 --2- 172.21.15.159:0/2576135401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f122c153580 0x7f122c073190 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.666+0000 7f1234400640 1 --2- 172.21.15.159:0/2576135401 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f120c06c8f0 0x7f120c06edb0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.666+0000 7f1234400640 1 --2- 172.21.15.159:0/2576135401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f122c0d07c0 0x7f122c158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.666+0000 7f1234400640 1 --2- 172.21.15.159:0/2576135401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f122c0af2e0 0x7f122c1f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.666+0000 7f1234400640 1 -- 172.21.15.159:0/2576135401 >> 172.21.15.159:0/2576135401 conn(0x7f122c06b750 msgr2=0x7f122c0b3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:28.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.666+0000 7f1234400640 1 -- 172.21.15.159:0/2576135401 shutdown_connections 2024-04-04T06:13:28.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.666+0000 7f1234400640 1 -- 172.21.15.159:0/2576135401 wait complete. 2024-04-04T06:13:28.675 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-04T06:13:28.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.861+0000 7fde07200640 1 Processor -- start 2024-04-04T06:13:28.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.861+0000 7fde07200640 1 -- start start 2024-04-04T06:13:28.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.863+0000 7fde07200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fde001590c0 0x7fde0015f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.863+0000 7fde07200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde0006c580 0x7fde0006ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.863+0000 7fde07200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde0015fe10 con 0x7fde00162440 2024-04-04T06:13:28.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.863+0000 7fde07200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fde0006e5b0 con 0x7fde001590c0 2024-04-04T06:13:28.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.863+0000 7fde07200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fde00067900 con 0x7fde0006c580 2024-04-04T06:13:28.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.863+0000 7fddffe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fde001590c0 0x7fde0015f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.863+0000 7fde05200640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fde00162440 0x7fde001628c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:46672/0 (socket says 172.21.15.159:46672) 2024-04-04T06:13:28.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.863+0000 7fde05200640 1 -- 172.21.15.159:0/212970844 learned_addr learned my addr 172.21.15.159:0/212970844 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:13:28.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.863+0000 7fddf7400640 1 --2- 172.21.15.159:0/212970844 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde0006c580 0x7fde0006ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.863+0000 7fddffe00640 1 -- 172.21.15.159:0/212970844 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde0006c580 msgr2=0x7fde0006ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:28.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.864+0000 7fddffe00640 1 --2- 172.21.15.159:0/212970844 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde0006c580 0x7fde0006ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.864+0000 7fddffe00640 1 -- 172.21.15.159:0/212970844 >> v1:172.21.15.159:6789/0 conn(0x7fde00162440 legacy=0x7fde001628c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:28.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.864+0000 7fddffe00640 1 -- 172.21.15.159:0/212970844 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde000b47d0 con 0x7fde001590c0 2024-04-04T06:13:28.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.865+0000 7fddffe00640 1 --2- 172.21.15.159:0/212970844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde001590c0 0x7fde0015f8d0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009d90 tx=0x7fddf002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47418a612322dc server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:28.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.865+0000 7fddff400640 1 -- 172.21.15.159:0/212970844 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0039070 con 0x7fde001590c0 2024-04-04T06:13:28.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.865+0000 7fddff400640 1 -- 172.21.15.159:0/212970844 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddf002cc60 con 0x7fde001590c0 2024-04-04T06:13:28.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.865+0000 7fddff400640 1 -- 172.21.15.159:0/212970844 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0034660 con 0x7fde001590c0 2024-04-04T06:13:28.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.865+0000 7fde07200640 1 -- 172.21.15.159:0/212970844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde001590c0 msgr2=0x7fde0015f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:28.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.865+0000 7fde07200640 1 --2- 172.21.15.159:0/212970844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde001590c0 0x7fde0015f8d0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009d90 tx=0x7fddf002beb0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.866+0000 7fde07200640 1 -- 172.21.15.159:0/212970844 shutdown_connections 2024-04-04T06:13:28.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.866+0000 7fde07200640 1 --2- 172.21.15.159:0/212970844 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde0006c580 0x7fde0006ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.866+0000 7fde07200640 1 --2- 172.21.15.159:0/212970844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde001590c0 0x7fde0015f8d0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.866+0000 7fde07200640 1 -- 172.21.15.159:0/212970844 >> 172.21.15.159:0/212970844 conn(0x7fde00071e60 msgr2=0x7fde00072270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:28.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.866+0000 7fde07200640 1 -- 172.21.15.159:0/212970844 shutdown_connections 2024-04-04T06:13:28.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.866+0000 7fde07200640 1 -- 172.21.15.159:0/212970844 wait complete. 2024-04-04T06:13:28.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.867+0000 7fde07200640 1 Processor -- start 2024-04-04T06:13:28.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.867+0000 7fde07200640 1 -- start start 2024-04-04T06:13:28.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.868+0000 7fde07200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde0006c580 0x7fde000aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.868+0000 7fde07200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde001590c0 0x7fde00157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.868+0000 7fde07200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde00162440 0x7fde00145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.868+0000 7fde07200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fde0006e240 con 0x7fde00162440 2024-04-04T06:13:28.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.868+0000 7fde07200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fde0006df60 con 0x7fde0006c580 2024-04-04T06:13:28.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.868+0000 7fde07200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fde000b4380 con 0x7fde001590c0 2024-04-04T06:13:28.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.869+0000 7fde05200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde00162440 0x7fde00145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.869+0000 7fde05200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde00162440 0x7fde00145e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38808/0 (socket says 172.21.15.159:38808) 2024-04-04T06:13:28.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.869+0000 7fde05200640 1 -- 172.21.15.159:0/885435252 learned_addr learned my addr 172.21.15.159:0/885435252 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:28.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.869+0000 7fddf7400640 1 --2- 172.21.15.159:0/885435252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde001590c0 0x7fde00157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.869+0000 7fddffe00640 1 --2- 172.21.15.159:0/885435252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde0006c580 0x7fde000aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.869+0000 7fde05200640 1 -- 172.21.15.159:0/885435252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde001590c0 msgr2=0x7fde00157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:28.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.869+0000 7fde05200640 1 --2- 172.21.15.159:0/885435252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde001590c0 0x7fde00157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.869+0000 7fde05200640 1 -- 172.21.15.159:0/885435252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde0006c580 msgr2=0x7fde000aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:28.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.869+0000 7fde05200640 1 --2- 172.21.15.159:0/885435252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde0006c580 0x7fde000aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:28.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.869+0000 7fde05200640 1 -- 172.21.15.159:0/885435252 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde000cd760 con 0x7fde00162440 2024-04-04T06:13:28.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.870+0000 7fde05200640 1 --2- 172.21.15.159:0/885435252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde00162440 0x7fde00145e40 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fddf800c370 tx=0x7fddf800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:28.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.870+0000 7fddfcc00640 1 -- 172.21.15.159:0/885435252 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8017070 con 0x7fde00162440 2024-04-04T06:13:28.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.870+0000 7fddfcc00640 1 -- 172.21.15.159:0/885435252 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddf8005ce0 con 0x7fde00162440 2024-04-04T06:13:28.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.870+0000 7fddfcc00640 1 -- 172.21.15.159:0/885435252 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf80072d0 con 0x7fde00162440 2024-04-04T06:13:28.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.870+0000 7fde07200640 1 -- 172.21.15.159:0/885435252 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde000cda40 con 0x7fde00162440 2024-04-04T06:13:28.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.870+0000 7fde07200640 1 -- 172.21.15.159:0/885435252 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde000ce000 con 0x7fde00162440 2024-04-04T06:13:28.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.871+0000 7fde07200640 1 -- 172.21.15.159:0/885435252 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde000ce7e0 con 0x7fde00162440 2024-04-04T06:13:28.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.873+0000 7fddfcc00640 1 -- 172.21.15.159:0/885435252 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fddf801a070 con 0x7fde00162440 2024-04-04T06:13:28.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.874+0000 7fddfcc00640 1 --2- 172.21.15.159:0/885435252 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdddc06c8f0 0x7fdddc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:28.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.874+0000 7fddfcc00640 1 -- 172.21.15.159:0/885435252 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7fddf808ae00 con 0x7fde00162440 2024-04-04T06:13:28.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.875+0000 7fddffe00640 1 --2- 172.21.15.159:0/885435252 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdddc06c8f0 0x7fdddc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:28.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.876+0000 7fddffe00640 1 --2- 172.21.15.159:0/885435252 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdddc06c8f0 0x7fdddc06edb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fddf0036000 tx=0x7fddf0030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:28.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:28.879+0000 7fddfcc00640 1 -- 172.21.15.159:0/885435252 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fde000ce7e0 con 0x7fde00162440 2024-04-04T06:13:29.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.117+0000 7fde07200640 1 -- 172.21.15.159:0/885435252 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fde000685c0 con 0x7fde00162440 2024-04-04T06:13:29.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.118+0000 7fddfcc00640 1 -- 172.21.15.159:0/885435252 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7fde0015fe10 con 0x7fde00162440 2024-04-04T06:13:29.112 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:13:29.112 INFO:teuthology.orchestra.run.smithi159.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-04T06:13:29.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.120+0000 7fde07200640 1 -- 172.21.15.159:0/885435252 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdddc06c8f0 msgr2=0x7fdddc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:29.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.120+0000 7fde07200640 1 --2- 172.21.15.159:0/885435252 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdddc06c8f0 0x7fdddc06edb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fddf0036000 tx=0x7fddf0030000 comp rx=0 tx=0).stop 2024-04-04T06:13:29.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.120+0000 7fde07200640 1 -- 172.21.15.159:0/885435252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde00162440 msgr2=0x7fde00145e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:29.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.120+0000 7fde07200640 1 --2- 172.21.15.159:0/885435252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde00162440 0x7fde00145e40 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fddf800c370 tx=0x7fddf800c830 comp rx=0 tx=0).stop 2024-04-04T06:13:29.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.120+0000 7fde07200640 1 -- 172.21.15.159:0/885435252 shutdown_connections 2024-04-04T06:13:29.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.120+0000 7fde07200640 1 --2- 172.21.15.159:0/885435252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde00162440 0x7fde00145e40 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.120+0000 7fde07200640 1 --2- 172.21.15.159:0/885435252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde001590c0 0x7fde00157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.120+0000 7fde07200640 1 --2- 172.21.15.159:0/885435252 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdddc06c8f0 0x7fdddc06edb0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.120+0000 7fde07200640 1 --2- 172.21.15.159:0/885435252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde0006c580 0x7fde000aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.120+0000 7fde07200640 1 -- 172.21.15.159:0/885435252 >> 172.21.15.159:0/885435252 conn(0x7fde00071e60 msgr2=0x7fde000b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:29.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.121+0000 7fde07200640 1 -- 172.21.15.159:0/885435252 shutdown_connections 2024-04-04T06:13:29.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.121+0000 7fde07200640 1 -- 172.21.15.159:0/885435252 wait complete. 2024-04-04T06:13:29.133 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-04T06:13:29.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:13:29.133 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-04T06:13:29.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.358+0000 7f6c25e00640 1 Processor -- start 2024-04-04T06:13:29.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.358+0000 7f6c25e00640 1 -- start start 2024-04-04T06:13:29.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.359+0000 7f6c25e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6c2013f190 0x7f6c200ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.359+0000 7f6c25e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c200ced40 0x7f6c20141890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.359+0000 7f6c25e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f6c20141dd0 0x7f6c2015fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.359+0000 7f6c25e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f6c20142240 con 0x7f6c20141dd0 2024-04-04T06:13:29.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.359+0000 7f6c25e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f6c2013f570 con 0x7f6c2013f190 2024-04-04T06:13:29.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.359+0000 7f6c25e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6c200cfbf0 con 0x7f6c200ced40 2024-04-04T06:13:29.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.360+0000 7f6c25400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f6c20141dd0 0x7f6c2015fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.360+0000 7f6c25400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f6c20141dd0 0x7f6c2015fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38822/0 (socket says 172.21.15.159:38822) 2024-04-04T06:13:29.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.360+0000 7f6c25400640 1 -- 172.21.15.159:0/822098147 learned_addr learned my addr 172.21.15.159:0/822098147 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:29.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.360+0000 7f6c24a00640 1 --2- 172.21.15.159:0/822098147 >> v2:172.21.15.178:3300/0 conn(0x7f6c2013f190 0x7f6c200ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.360+0000 7f6c1fe00640 1 --2- 172.21.15.159:0/822098147 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c200ced40 0x7f6c20141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.360+0000 7f6c25400640 1 -- 172.21.15.159:0/822098147 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c200ced40 msgr2=0x7f6c20141890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:29.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.360+0000 7f6c25400640 1 --2- 172.21.15.159:0/822098147 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c200ced40 0x7f6c20141890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.360+0000 7f6c25400640 1 -- 172.21.15.159:0/822098147 >> v2:172.21.15.178:3300/0 conn(0x7f6c2013f190 msgr2=0x7f6c200ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:29.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.360+0000 7f6c25400640 1 --2- 172.21.15.159:0/822098147 >> v2:172.21.15.178:3300/0 conn(0x7f6c2013f190 0x7f6c200ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.360+0000 7f6c25400640 1 -- 172.21.15.159:0/822098147 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c200cf350 con 0x7f6c20141dd0 2024-04-04T06:13:29.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.361+0000 7f6c25400640 1 --2- 172.21.15.159:0/822098147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c20141dd0 0x7f6c2015fd00 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f6c20073710 tx=0x7f6c1802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0dea624ef895a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:29.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.362+0000 7f6c1f400640 1 -- 172.21.15.159:0/822098147 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1802ca70 con 0x7f6c20141dd0 2024-04-04T06:13:29.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.362+0000 7f6c1f400640 1 -- 172.21.15.159:0/822098147 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c1802cc10 con 0x7f6c20141dd0 2024-04-04T06:13:29.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.362+0000 7f6c1f400640 1 -- 172.21.15.159:0/822098147 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c180343f0 con 0x7f6c20141dd0 2024-04-04T06:13:29.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.362+0000 7f6c25e00640 1 -- 172.21.15.159:0/822098147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c20141dd0 msgr2=0x7f6c2015fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:29.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.362+0000 7f6c25e00640 1 --2- 172.21.15.159:0/822098147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c20141dd0 0x7f6c2015fd00 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f6c20073710 tx=0x7f6c1802bed0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.362+0000 7f6c25e00640 1 -- 172.21.15.159:0/822098147 shutdown_connections 2024-04-04T06:13:29.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.362+0000 7f6c25e00640 1 --2- 172.21.15.159:0/822098147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c20141dd0 0x7f6c2015fd00 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.362+0000 7f6c25e00640 1 --2- 172.21.15.159:0/822098147 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c200ced40 0x7f6c20141890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.362+0000 7f6c25e00640 1 --2- 172.21.15.159:0/822098147 >> v2:172.21.15.178:3300/0 conn(0x7f6c2013f190 0x7f6c200ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.362+0000 7f6c25e00640 1 -- 172.21.15.159:0/822098147 >> 172.21.15.159:0/822098147 conn(0x7f6c200b1d80 msgr2=0x7f6c200b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:29.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.363+0000 7f6c25e00640 1 -- 172.21.15.159:0/822098147 shutdown_connections 2024-04-04T06:13:29.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.363+0000 7f6c25e00640 1 -- 172.21.15.159:0/822098147 wait complete. 2024-04-04T06:13:29.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.364+0000 7f6c25e00640 1 Processor -- start 2024-04-04T06:13:29.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.364+0000 7f6c25e00640 1 -- start start 2024-04-04T06:13:29.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.365+0000 7f6c25e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c200ced40 0x7f6c20156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.365+0000 7f6c25e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c2013f190 0x7f6c201416e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.365+0000 7f6c25e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c20141dd0 0x7f6c201526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.365+0000 7f6c25e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2006d670 con 0x7f6c200ced40 2024-04-04T06:13:29.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.365+0000 7f6c25e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6c200cf910 con 0x7f6c20141dd0 2024-04-04T06:13:29.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.365+0000 7f6c25e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6c20142550 con 0x7f6c2013f190 2024-04-04T06:13:29.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.365+0000 7f6c24a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c200ced40 0x7f6c20156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.366+0000 7f6c24a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c200ced40 0x7f6c20156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38830/0 (socket says 172.21.15.159:38830) 2024-04-04T06:13:29.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.366+0000 7f6c24a00640 1 -- 172.21.15.159:0/3139295066 learned_addr learned my addr 172.21.15.159:0/3139295066 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:29.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.366+0000 7f6c25400640 1 --2- 172.21.15.159:0/3139295066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c20141dd0 0x7f6c201526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.366+0000 7f6c24a00640 1 -- 172.21.15.159:0/3139295066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c2013f190 msgr2=0x7f6c201416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:29.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.366+0000 7f6c24a00640 1 --2- 172.21.15.159:0/3139295066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c2013f190 0x7f6c201416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.366+0000 7f6c24a00640 1 -- 172.21.15.159:0/3139295066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c20141dd0 msgr2=0x7f6c201526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:29.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.366+0000 7f6c24a00640 1 --2- 172.21.15.159:0/3139295066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c20141dd0 0x7f6c201526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.366+0000 7f6c24a00640 1 -- 172.21.15.159:0/3139295066 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c20152c10 con 0x7f6c200ced40 2024-04-04T06:13:29.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.366+0000 7f6c25400640 1 --2- 172.21.15.159:0/3139295066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c20141dd0 0x7f6c201526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:13:29.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.366+0000 7f6c24a00640 1 --2- 172.21.15.159:0/3139295066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c200ced40 0x7f6c20156db0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400d620 tx=0x7f6c1400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:29.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.367+0000 7f6c1cc00640 1 -- 172.21.15.159:0/3139295066 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1400b620 con 0x7f6c200ced40 2024-04-04T06:13:29.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.367+0000 7f6c1cc00640 1 -- 172.21.15.159:0/3139295066 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c14019070 con 0x7f6c200ced40 2024-04-04T06:13:29.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.367+0000 7f6c25e00640 1 -- 172.21.15.159:0/3139295066 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c2013f700 con 0x7f6c200ced40 2024-04-04T06:13:29.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.367+0000 7f6c1cc00640 1 -- 172.21.15.159:0/3139295066 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1400ba00 con 0x7f6c200ced40 2024-04-04T06:13:29.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.367+0000 7f6c25e00640 1 -- 172.21.15.159:0/3139295066 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c2013fcc0 con 0x7f6c200ced40 2024-04-04T06:13:29.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.370+0000 7f6c1cc00640 1 -- 172.21.15.159:0/3139295066 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c1400bba0 con 0x7f6c200ced40 2024-04-04T06:13:29.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.371+0000 7f6c2a200740 1 Processor -- start 2024-04-04T06:13:29.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.371+0000 7f6c2a200740 1 -- start start 2024-04-04T06:13:29.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.371+0000 7f6c1cc00640 1 --2- 172.21.15.159:0/3139295066 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6c0006c8f0 0x7f6c0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.371+0000 7f6c1cc00640 1 -- 172.21.15.159:0/3139295066 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f6c1408bf10 con 0x7f6c200ced40 2024-04-04T06:13:29.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.372+0000 7f6c25400640 1 --2- 172.21.15.159:0/3139295066 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6c0006c8f0 0x7f6c0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.372+0000 7f6c2a200740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d258580180 0x55d258580560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.372+0000 7f6c2a200740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d258585fa0 0x55d2585841d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.372+0000 7f6c2a200740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2585cd890 0x55d2585cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.372+0000 7f6c2a200740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d2584ea280 con 0x55d2585cd890 2024-04-04T06:13:29.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.372+0000 7f6c2a200740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d2584926f0 con 0x55d258580180 2024-04-04T06:13:29.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.372+0000 7f6c2a200740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55d2584e6230 con 0x55d258585fa0 2024-04-04T06:13:29.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.372+0000 7f6c25400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2585cd890 0x55d2585cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.372+0000 7f6c25400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2585cd890 0x55d2585cdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38832/0 (socket says 172.21.15.159:38832) 2024-04-04T06:13:29.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.372+0000 7f6c25400640 1 -- 172.21.15.159:0/1302429691 learned_addr learned my addr 172.21.15.159:0/1302429691 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:29.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.372+0000 7f6c24a00640 1 --2- 172.21.15.159:0/1302429691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d258580180 0x55d258580560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.372+0000 7f6c1fe00640 1 --2- 172.21.15.159:0/1302429691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d258585fa0 0x55d2585841d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.373+0000 7f6c25400640 1 -- 172.21.15.159:0/1302429691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d258585fa0 msgr2=0x55d2585841d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:29.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.373+0000 7f6c25400640 1 --2- 172.21.15.159:0/1302429691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d258585fa0 0x55d2585841d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.373+0000 7f6c25400640 1 -- 172.21.15.159:0/1302429691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d258580180 msgr2=0x55d258580560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:29.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.373+0000 7f6c25400640 1 --2- 172.21.15.159:0/1302429691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d258580180 0x55d258580560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.373+0000 7f6c25400640 1 -- 172.21.15.159:0/1302429691 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2582c67e0 con 0x55d2585cd890 2024-04-04T06:13:29.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.373+0000 7f6c25400640 1 --2- 172.21.15.159:0/3139295066 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6c0006c8f0 0x7f6c0006edb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6c18009c90 tx=0x7f6c1802e070 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:29.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.373+0000 7f6c25400640 1 --2- 172.21.15.159:0/1302429691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2585cd890 0x55d2585cdd10 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f6c18036040 tx=0x7f6c18048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9651ea0560223e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:29.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.373+0000 7f6c11200640 1 -- 172.21.15.159:0/1302429691 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18043820 con 0x55d2585cd890 2024-04-04T06:13:29.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.373+0000 7f6c11200640 1 -- 172.21.15.159:0/1302429691 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c180439c0 con 0x55d2585cd890 2024-04-04T06:13:29.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.373+0000 7f6c11200640 1 -- 172.21.15.159:0/1302429691 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18043d70 con 0x55d2585cd890 2024-04-04T06:13:29.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.374+0000 7f6c2a200740 1 -- 172.21.15.159:0/1302429691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2585cd890 msgr2=0x55d2585cdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:29.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.374+0000 7f6c2a200740 1 --2- 172.21.15.159:0/1302429691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2585cd890 0x55d2585cdd10 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f6c18036040 tx=0x7f6c18048000 comp rx=0 tx=0).stop 2024-04-04T06:13:29.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.374+0000 7f6c2a200740 1 -- 172.21.15.159:0/1302429691 shutdown_connections 2024-04-04T06:13:29.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.374+0000 7f6c2a200740 1 --2- 172.21.15.159:0/1302429691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2585cd890 0x55d2585cdd10 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.374+0000 7f6c2a200740 1 --2- 172.21.15.159:0/1302429691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d258585fa0 0x55d2585841d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.374+0000 7f6c2a200740 1 --2- 172.21.15.159:0/1302429691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d258580180 0x55d258580560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.374+0000 7f6c2a200740 1 -- 172.21.15.159:0/1302429691 >> 172.21.15.159:0/1302429691 conn(0x55d258582200 msgr2=0x55d258583ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:29.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.374+0000 7f6c2a200740 1 -- 172.21.15.159:0/1302429691 shutdown_connections 2024-04-04T06:13:29.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.374+0000 7f6c2a200740 1 -- 172.21.15.159:0/1302429691 wait complete. 2024-04-04T06:13:29.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.375+0000 7f6c2a200740 1 -- start start 2024-04-04T06:13:29.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.375+0000 7f6c2a200740 1 Processor -- start 2024-04-04T06:13:29.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.376+0000 7f6c2a200740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d258585fa0 0x55d2585dbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.376+0000 7f6c2a200740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d2585cd890 0x55d2585affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.376+0000 7f6c2a200740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2585c71b0 0x55d2585b0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.376+0000 7f6c2a200740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d2585ac8f0 con 0x55d258585fa0 2024-04-04T06:13:29.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.376+0000 7f6c2a200740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d2584a5aa0 con 0x55d2585c71b0 2024-04-04T06:13:29.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.376+0000 7f6c2a200740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55d2585acb40 con 0x55d2585cd890 2024-04-04T06:13:29.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.376+0000 7f6c2a200740 10 client.0 _ll_register_callbacks cb 0x55d2584a5490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:13:29.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.376+0000 7f6c24a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d258585fa0 0x55d2585dbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.377+0000 7f6c24a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d258585fa0 0x55d2585dbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38840/0 (socket says 172.21.15.159:38840) 2024-04-04T06:13:29.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.377+0000 7f6c24a00640 1 -- 172.21.15.159:0/1703431119 learned_addr learned my addr 172.21.15.159:0/1703431119 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:29.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.377+0000 7f6c24a00640 1 -- 172.21.15.159:0/1703431119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d2585cd890 msgr2=0x55d2585affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:29.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.377+0000 7f6c24a00640 1 --2- 172.21.15.159:0/1703431119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d2585cd890 0x55d2585affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.377+0000 7f6c24a00640 1 -- 172.21.15.159:0/1703431119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2585c71b0 msgr2=0x55d2585b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:29.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.377+0000 7f6c24a00640 1 --2- 172.21.15.159:0/1703431119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2585c71b0 0x55d2585b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.377+0000 7f6c24a00640 1 -- 172.21.15.159:0/1703431119 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d258326f70 con 0x55d258585fa0 2024-04-04T06:13:29.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.377+0000 7f6c24a00640 1 --2- 172.21.15.159:0/1703431119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d258585fa0 0x55d2585dbf30 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400bfd0 tx=0x7f6c1405a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471742ad73d99fe server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:29.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.377+0000 7f6c05600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:13:29.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.377+0000 7f6c05600640 1 -- 172.21.15.159:0/1703431119 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14088060 con 0x55d258585fa0 2024-04-04T06:13:29.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.377+0000 7f6c05600640 1 -- 172.21.15.159:0/1703431119 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f6c14062330 con 0x55d258585fa0 2024-04-04T06:13:29.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.377+0000 7f6c2a200740 1 -- 172.21.15.159:0/1703431119 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c180439c0 con 0x55d258585fa0 2024-04-04T06:13:29.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.378+0000 7f6c05600640 1 -- 172.21.15.159:0/1703431119 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f6c14072070 con 0x55d258585fa0 2024-04-04T06:13:29.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.378+0000 7f6c2a200740 10 client.7686 fetch_fsmap learned FSMap version 69 2024-04-04T06:13:29.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.378+0000 7f6c2a200740 1 -- 172.21.15.159:0/1703431119 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x55d2581be170 con 0x55d258585fa0 2024-04-04T06:13:29.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.378+0000 7f6c05600640 1 -- 172.21.15.159:0/1703431119 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c180439c0 con 0x55d258585fa0 2024-04-04T06:13:29.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.378+0000 7f6c05600640 1 -- 172.21.15.159:0/1703431119 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 69) ==== 3494+0+0 (secure 0 0 0) 0x7f6c14013070 con 0x55d258585fa0 2024-04-04T06:13:29.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.378+0000 7f6c2a200740 10 client.7686 fetch_fsmap finished waiting for FSMap version 69 2024-04-04T06:13:29.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.378+0000 7f6c2a200740 10 client.7686 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-04T06:13:29.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.378+0000 7f6c2a200740 20 client.7686 populate_metadata read hostname 'smithi159' 2024-04-04T06:13:29.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.378+0000 7f6c2a200740 1 --2- 172.21.15.159:0/1703431119 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55d2585c5a10 0x55d2585c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.378+0000 7f6c2a200740 4 client.7686 mds_command: new command op to 7512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:13:29.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.378+0000 7f6c2a200740 1 -- 172.21.15.159:0/1703431119 --> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2584f77c0 con 0x55d2585c5a10 2024-04-04T06:13:29.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.378+0000 7f6c1fe00640 1 --2- 172.21.15.159:0/1703431119 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55d2585c5a10 0x55d2585c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.379+0000 7f6c1fe00640 1 --2- 172.21.15.159:0/1703431119 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55d2585c5a10 0x55d2585c5e70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8f46167647fdf82 server_cookie=d82f53eeb5a70b9 in_seq=0 out_seq=0 2024-04-04T06:13:29.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.379+0000 7f6c05600640 10 client.7686 ms_handle_connect on v2:172.21.15.159:6833/3796123128 2024-04-04T06:13:29.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.380+0000 7f6c05600640 1 -- 172.21.15.159:0/1703431119 <== mds.0 v2:172.21.15.159:6833/3796123128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2584f77c0 con 0x55d2585c5a10 2024-04-04T06:13:29.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.380+0000 7f6c05600640 10 client.7686 handle_command_reply: tid=0 2024-04-04T06:13:29.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.380+0000 7f6c2a200740 1 client.7686 shutdown 2024-04-04T06:13:29.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.380+0000 7f6c2a200740 1 -- 172.21.15.159:0/1703431119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d258585fa0 msgr2=0x55d2585dbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:29.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.380+0000 7f6c2a200740 1 --2- 172.21.15.159:0/1703431119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d258585fa0 0x55d2585dbf30 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400bfd0 tx=0x7f6c1405a420 comp rx=0 tx=0).stop 2024-04-04T06:13:29.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.380+0000 7f6c2a200740 1 -- 172.21.15.159:0/1703431119 shutdown_connections 2024-04-04T06:13:29.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.380+0000 7f6c2a200740 1 --2- 172.21.15.159:0/1703431119 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55d2585c5a10 0x55d2585c5e70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.380+0000 7f6c2a200740 1 --2- 172.21.15.159:0/1703431119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2585c71b0 0x55d2585b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.380+0000 7f6c2a200740 1 --2- 172.21.15.159:0/1703431119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d2585cd890 0x55d2585affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.380+0000 7f6c2a200740 1 --2- 172.21.15.159:0/1703431119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d258585fa0 0x55d2585dbf30 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.380+0000 7f6c2a200740 1 -- 172.21.15.159:0/1703431119 >> 172.21.15.159:0/1703431119 conn(0x55d258580180 msgr2=0x55d2585cc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:29.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.380+0000 7f6c05600640 0 client.7686 ms_handle_reset on v2:172.21.15.159:6833/3796123128 2024-04-04T06:13:29.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.380+0000 7f6c2a200740 1 -- 172.21.15.159:0/1703431119 shutdown_connections 2024-04-04T06:13:29.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.380+0000 7f6c2a200740 1 -- 172.21.15.159:0/1703431119 wait complete. 2024-04-04T06:13:29.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.381+0000 7f6c2a200740 20 client.7686 trim_cache size 0 max 16384 2024-04-04T06:13:29.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.399+0000 7f6c2a200740 1 Processor -- start 2024-04-04T06:13:29.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.399+0000 7f6c2a200740 1 -- start start 2024-04-04T06:13:29.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.400+0000 7f6c2a200740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d258585fa0 0x55d2585c6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.400+0000 7f6c2a200740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d2585c71b0 0x55d2585aff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.400+0000 7f6c2a200740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2585cd890 0x55d2585b0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.400+0000 7f6c2a200740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201fc560 con 0x55d2585cd890 2024-04-04T06:13:29.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.400+0000 7f6c2a200740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201fb5e0 con 0x55d258585fa0 2024-04-04T06:13:29.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.400+0000 7f6c2a200740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6c201fa940 con 0x55d2585c71b0 2024-04-04T06:13:29.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.400+0000 7f6c25400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2585cd890 0x55d2585b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.400+0000 7f6c25400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2585cd890 0x55d2585b0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38844/0 (socket says 172.21.15.159:38844) 2024-04-04T06:13:29.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.400+0000 7f6c25400640 1 -- 172.21.15.159:0/4194009303 learned_addr learned my addr 172.21.15.159:0/4194009303 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:29.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.401+0000 7f6c24a00640 1 --2- 172.21.15.159:0/4194009303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d258585fa0 0x55d2585c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.401+0000 7f6c1fe00640 1 --2- 172.21.15.159:0/4194009303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d2585c71b0 0x55d2585aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.401+0000 7f6c25400640 1 -- 172.21.15.159:0/4194009303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d2585c71b0 msgr2=0x55d2585aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:29.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.401+0000 7f6c25400640 1 --2- 172.21.15.159:0/4194009303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d2585c71b0 0x55d2585aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.401+0000 7f6c25400640 1 -- 172.21.15.159:0/4194009303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d258585fa0 msgr2=0x55d2585c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:29.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.401+0000 7f6c25400640 1 --2- 172.21.15.159:0/4194009303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d258585fa0 0x55d2585c6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.401+0000 7f6c25400640 1 -- 172.21.15.159:0/4194009303 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2585c6fc0 con 0x55d2585cd890 2024-04-04T06:13:29.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.401+0000 7f6c25400640 1 --2- 172.21.15.159:0/4194009303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2585cd890 0x55d2585b0440 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f6c18009f40 tx=0x7f6c18009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704d0a9c62b4e22d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:29.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.401+0000 7f6c07e00640 1 -- 172.21.15.159:0/4194009303 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1805b070 con 0x55d2585cd890 2024-04-04T06:13:29.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.401+0000 7f6c07e00640 1 -- 172.21.15.159:0/4194009303 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c18003990 con 0x55d2585cd890 2024-04-04T06:13:29.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.401+0000 7f6c07e00640 1 -- 172.21.15.159:0/4194009303 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18003d40 con 0x55d2585cd890 2024-04-04T06:13:29.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.402+0000 7f6c2a200740 1 -- 172.21.15.159:0/4194009303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2585cd890 msgr2=0x55d2585b0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:29.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.402+0000 7f6c2a200740 1 --2- 172.21.15.159:0/4194009303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2585cd890 0x55d2585b0440 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f6c18009f40 tx=0x7f6c18009000 comp rx=0 tx=0).stop 2024-04-04T06:13:29.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.402+0000 7f6c2a200740 1 -- 172.21.15.159:0/4194009303 shutdown_connections 2024-04-04T06:13:29.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.402+0000 7f6c2a200740 1 --2- 172.21.15.159:0/4194009303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2585cd890 0x55d2585b0440 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.402+0000 7f6c2a200740 1 --2- 172.21.15.159:0/4194009303 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d2585c71b0 0x55d2585aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.402+0000 7f6c2a200740 1 --2- 172.21.15.159:0/4194009303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d258585fa0 0x55d2585c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.402+0000 7f6c2a200740 1 -- 172.21.15.159:0/4194009303 >> 172.21.15.159:0/4194009303 conn(0x55d258580180 msgr2=0x55d2585cc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:29.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.402+0000 7f6c2a200740 1 -- 172.21.15.159:0/4194009303 shutdown_connections 2024-04-04T06:13:29.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.402+0000 7f6c2a200740 1 -- 172.21.15.159:0/4194009303 wait complete. 2024-04-04T06:13:29.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.403+0000 7f6c2a200740 1 -- start start 2024-04-04T06:13:29.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.403+0000 7f6c2a200740 1 Processor -- start 2024-04-04T06:13:29.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.405+0000 7f6c2a200740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d258585fa0 0x55d2585c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.405+0000 7f6c2a200740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2585c71b0 0x55d2585ab840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.405+0000 7f6c2a200740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2585cd890 0x55d258645ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.405+0000 7f6c2a200740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d2585de5b0 con 0x55d2585cd890 2024-04-04T06:13:29.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.405+0000 7f6c2a200740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6c201faf90 con 0x55d2585c71b0 2024-04-04T06:13:29.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.405+0000 7f6c2a200740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55d25857bf50 con 0x55d258585fa0 2024-04-04T06:13:29.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.405+0000 7f6c2a200740 10 client.0 _ll_register_callbacks cb 0x7f6c14059a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:13:29.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.405+0000 7f6c25400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2585cd890 0x55d258645ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.405+0000 7f6c25400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2585cd890 0x55d258645ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38852/0 (socket says 172.21.15.159:38852) 2024-04-04T06:13:29.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.405+0000 7f6c25400640 1 -- 172.21.15.159:0/1272591474 learned_addr learned my addr 172.21.15.159:0/1272591474 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:29.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.406+0000 7f6c1fe00640 1 --2- 172.21.15.159:0/1272591474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2585c71b0 0x55d2585ab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.406+0000 7f6c24a00640 1 --2- 172.21.15.159:0/1272591474 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d258585fa0 0x55d2585c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.406+0000 7f6c25400640 1 -- 172.21.15.159:0/1272591474 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d258585fa0 msgr2=0x55d2585c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:29.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.406+0000 7f6c25400640 1 --2- 172.21.15.159:0/1272591474 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d258585fa0 0x55d2585c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.406+0000 7f6c25400640 1 -- 172.21.15.159:0/1272591474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2585c71b0 msgr2=0x55d2585ab840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:29.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.406+0000 7f6c25400640 1 --2- 172.21.15.159:0/1272591474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2585c71b0 0x55d2585ab840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.406+0000 7f6c25400640 1 -- 172.21.15.159:0/1272591474 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2585c60d0 con 0x55d2585cd890 2024-04-04T06:13:29.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.406+0000 7f6c25400640 1 --2- 172.21.15.159:0/1272591474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2585cd890 0x55d258645ed0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f6c1802bf10 tx=0x7f6c18004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938c9015945f099b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:29.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.406+0000 7f6c05600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:13:29.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.406+0000 7f6c05600640 1 -- 172.21.15.159:0/1272591474 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1805b050 con 0x55d2585cd890 2024-04-04T06:13:29.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.406+0000 7f6c05600640 1 -- 172.21.15.159:0/1272591474 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f6c18043c30 con 0x55d2585cd890 2024-04-04T06:13:29.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.406+0000 7f6c2a200740 1 -- 172.21.15.159:0/1272591474 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c18003990 con 0x55d2585cd890 2024-04-04T06:13:29.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.407+0000 7f6c05600640 1 -- 172.21.15.159:0/1272591474 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f6c18043e30 con 0x55d2585cd890 2024-04-04T06:13:29.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.407+0000 7f6c2a200740 10 client.7692 fetch_fsmap learned FSMap version 69 2024-04-04T06:13:29.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.407+0000 7f6c2a200740 1 -- 172.21.15.159:0/1272591474 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x55d2585aa440 con 0x55d2585cd890 2024-04-04T06:13:29.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.407+0000 7f6c05600640 1 -- 172.21.15.159:0/1272591474 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c18070070 con 0x55d2585cd890 2024-04-04T06:13:29.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.407+0000 7f6c05600640 1 -- 172.21.15.159:0/1272591474 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 69) ==== 3494+0+0 (secure 0 0 0) 0x7f6c18008040 con 0x55d2585cd890 2024-04-04T06:13:29.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.407+0000 7f6c2a200740 10 client.7692 fetch_fsmap finished waiting for FSMap version 69 2024-04-04T06:13:29.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.407+0000 7f6c2a200740 10 client.7692 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-04T06:13:29.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.407+0000 7f6c2a200740 20 client.7692 populate_metadata read hostname 'smithi159' 2024-04-04T06:13:29.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.407+0000 7f6c2a200740 1 --2- 172.21.15.159:0/1272591474 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55d2585a9af0 0x55d2585aff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.407+0000 7f6c2a200740 4 client.7692 mds_command: new command op to 7512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-04T06:13:29.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.407+0000 7f6c2a200740 1 -- 172.21.15.159:0/1272591474 --> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x55d2585b3c90 con 0x55d2585a9af0 2024-04-04T06:13:29.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.407+0000 7f6c24a00640 1 --2- 172.21.15.159:0/1272591474 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55d2585a9af0 0x55d2585aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.408+0000 7f6c24a00640 1 --2- 172.21.15.159:0/1272591474 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55d2585a9af0 0x55d2585aff00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec50d3c02bbc3a09 server_cookie=9b62a837e69bb31e in_seq=0 out_seq=0 2024-04-04T06:13:29.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.408+0000 7f6c05600640 10 client.7692 ms_handle_connect on v2:172.21.15.159:6833/3796123128 2024-04-04T06:13:29.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.408+0000 7f6c05600640 1 -- 172.21.15.159:0/1272591474 <== mds.0 v2:172.21.15.159:6833/3796123128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d2585b3c90 con 0x55d2585a9af0 2024-04-04T06:13:29.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.408+0000 7f6c05600640 10 client.7692 handle_command_reply: tid=0 2024-04-04T06:13:29.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.408+0000 7f6c2a200740 1 client.7692 shutdown 2024-04-04T06:13:29.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.409+0000 7f6c2a200740 1 -- 172.21.15.159:0/1272591474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2585cd890 msgr2=0x55d258645ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:29.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.409+0000 7f6c2a200740 1 --2- 172.21.15.159:0/1272591474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2585cd890 0x55d258645ed0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f6c1802bf10 tx=0x7f6c18004ab0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.409+0000 7f6c2a200740 1 -- 172.21.15.159:0/1272591474 shutdown_connections 2024-04-04T06:13:29.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.409+0000 7f6c2a200740 1 --2- 172.21.15.159:0/1272591474 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55d2585a9af0 0x55d2585aff00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.409+0000 7f6c2a200740 1 --2- 172.21.15.159:0/1272591474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2585cd890 0x55d258645ed0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.409+0000 7f6c2a200740 1 --2- 172.21.15.159:0/1272591474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2585c71b0 0x55d2585ab840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.409+0000 7f6c2a200740 1 --2- 172.21.15.159:0/1272591474 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55d258585fa0 0x55d2585c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.409+0000 7f6c2a200740 1 -- 172.21.15.159:0/1272591474 >> 172.21.15.159:0/1272591474 conn(0x55d258580180 msgr2=0x55d2585cc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:29.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.409+0000 7f6c05600640 0 client.7692 ms_handle_reset on v2:172.21.15.159:6833/3796123128 2024-04-04T06:13:29.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.409+0000 7f6c2a200740 1 -- 172.21.15.159:0/1272591474 shutdown_connections 2024-04-04T06:13:29.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.409+0000 7f6c2a200740 1 -- 172.21.15.159:0/1272591474 wait complete. 2024-04-04T06:13:29.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.409+0000 7f6c2a200740 20 client.7692 trim_cache size 0 max 16384 2024-04-04T06:13:29.407 INFO:tasks.ceph.mds.a.smithi159.stderr:2024-04-04T06:13:29.409+0000 7fb383a00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-04-04T06:12:45.034063+0000 1=1+0)->f(v1 m2024-04-04T06:12:45.034063+0000) n(v0 rc2024-04-04T06:12:45.181060+0000 2=1+1)->n(v0 rc2024-04-04T06:12:45.181060+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55728e21c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-04T06:12:45.034063+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-04T06:12:45.181060+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-04T06:13:29.408 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:13:29.408 INFO:teuthology.orchestra.run.smithi159.stdout: "return_code": 0, 2024-04-04T06:13:29.408 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_tag": "3be18bbd-ff6a-4ca3-aedf-c1550fc49ad7", 2024-04-04T06:13:29.408 INFO:teuthology.orchestra.run.smithi159.stdout: "mode": "asynchronous" 2024-04-04T06:13:29.408 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:13:29.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.410+0000 7f6c25e00640 1 -- 172.21.15.159:0/3139295066 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6c0006c8f0 msgr2=0x7f6c0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:29.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.410+0000 7f6c25e00640 1 --2- 172.21.15.159:0/3139295066 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6c0006c8f0 0x7f6c0006edb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6c18009c90 tx=0x7f6c1802e070 comp rx=0 tx=0).stop 2024-04-04T06:13:29.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.410+0000 7f6c25e00640 1 -- 172.21.15.159:0/3139295066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c200ced40 msgr2=0x7f6c20156db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:29.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.410+0000 7f6c25e00640 1 --2- 172.21.15.159:0/3139295066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c200ced40 0x7f6c20156db0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400d620 tx=0x7f6c1400dae0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.410+0000 7f6c25e00640 1 -- 172.21.15.159:0/3139295066 shutdown_connections 2024-04-04T06:13:29.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.410+0000 7f6c25e00640 1 --2- 172.21.15.159:0/3139295066 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6c0006c8f0 0x7f6c0006edb0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.410+0000 7f6c25e00640 1 --2- 172.21.15.159:0/3139295066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c20141dd0 0x7f6c201526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.410+0000 7f6c25e00640 1 --2- 172.21.15.159:0/3139295066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c2013f190 0x7f6c201416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.410+0000 7f6c25e00640 1 --2- 172.21.15.159:0/3139295066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c200ced40 0x7f6c20156db0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.410+0000 7f6c25e00640 1 -- 172.21.15.159:0/3139295066 >> 172.21.15.159:0/3139295066 conn(0x7f6c200b1d80 msgr2=0x7f6c2006d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:29.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.410+0000 7f6c25e00640 1 -- 172.21.15.159:0/3139295066 shutdown_connections 2024-04-04T06:13:29.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.410+0000 7f6c25e00640 1 -- 172.21.15.159:0/3139295066 wait complete. 2024-04-04T06:13:29.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:13:29.422 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-04T06:13:29.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.619+0000 7ff511800640 1 Processor -- start 2024-04-04T06:13:29.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.619+0000 7ff511800640 1 -- start start 2024-04-04T06:13:29.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.620+0000 7ff511800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff50c0d0990 0x7ff50c0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.620+0000 7ff511800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff50c0af2e0 0x7ff50c1600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.620+0000 7ff511800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff50c0afab0 con 0x7ff50c153580 2024-04-04T06:13:29.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.620+0000 7ff511800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7ff50c15cc80 con 0x7ff50c0d0990 2024-04-04T06:13:29.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.620+0000 7ff511800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff50c15da00 con 0x7ff50c0af2e0 2024-04-04T06:13:29.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.621+0000 7ff503e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff50c0d0990 0x7ff50c0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.621+0000 7ff503e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff50c0d0990 0x7ff50c0aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38868/0 (socket says 172.21.15.159:38868) 2024-04-04T06:13:29.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.621+0000 7ff503e00640 1 -- 172.21.15.159:0/1694450945 learned_addr learned my addr 172.21.15.159:0/1694450945 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:29.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.621+0000 7ff50b400640 1 --2- 172.21.15.159:0/1694450945 >> v2:172.21.15.178:3300/0 conn(0x7ff50c0af2e0 0x7ff50c1600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.621+0000 7ff503e00640 1 -- 172.21.15.159:0/1694450945 >> v2:172.21.15.178:3300/0 conn(0x7ff50c0af2e0 msgr2=0x7ff50c1600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:29.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.621+0000 7ff503e00640 1 --2- 172.21.15.159:0/1694450945 >> v2:172.21.15.178:3300/0 conn(0x7ff50c0af2e0 0x7ff50c1600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.621+0000 7ff503e00640 1 -- 172.21.15.159:0/1694450945 >> v1:172.21.15.178:6789/0 conn(0x7ff50c153580 legacy=0x7ff50c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:29.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.621+0000 7ff503e00640 1 -- 172.21.15.159:0/1694450945 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c15cf20 con 0x7ff50c0d0990 2024-04-04T06:13:29.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.623+0000 7ff503e00640 1 --2- 172.21.15.159:0/1694450945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff50c0d0990 0x7ff50c0aeda0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7ff50c06efa0 tx=0x7ff4f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effaec40c5715616 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:29.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.623+0000 7ff50a000640 1 -- 172.21.15.159:0/1694450945 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4035020 con 0x7ff50c0d0990 2024-04-04T06:13:29.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.623+0000 7ff50a000640 1 -- 172.21.15.159:0/1694450945 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4f4031870 con 0x7ff50c0d0990 2024-04-04T06:13:29.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.623+0000 7ff50a000640 1 -- 172.21.15.159:0/1694450945 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4031c40 con 0x7ff50c0d0990 2024-04-04T06:13:29.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.624+0000 7ff511800640 1 -- 172.21.15.159:0/1694450945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff50c0d0990 msgr2=0x7ff50c0aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:29.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.624+0000 7ff511800640 1 --2- 172.21.15.159:0/1694450945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff50c0d0990 0x7ff50c0aeda0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7ff50c06efa0 tx=0x7ff4f402b910 comp rx=0 tx=0).stop 2024-04-04T06:13:29.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.624+0000 7ff511800640 1 -- 172.21.15.159:0/1694450945 shutdown_connections 2024-04-04T06:13:29.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.624+0000 7ff511800640 1 --2- 172.21.15.159:0/1694450945 >> v2:172.21.15.178:3300/0 conn(0x7ff50c0af2e0 0x7ff50c1600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.624+0000 7ff511800640 1 --2- 172.21.15.159:0/1694450945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff50c0d0990 0x7ff50c0aeda0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.624+0000 7ff511800640 1 -- 172.21.15.159:0/1694450945 >> 172.21.15.159:0/1694450945 conn(0x7ff50c06b750 msgr2=0x7ff50c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:29.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.624+0000 7ff511800640 1 -- 172.21.15.159:0/1694450945 shutdown_connections 2024-04-04T06:13:29.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.624+0000 7ff511800640 1 -- 172.21.15.159:0/1694450945 wait complete. 2024-04-04T06:13:29.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.626+0000 7ff511800640 1 Processor -- start 2024-04-04T06:13:29.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.626+0000 7ff511800640 1 -- start start 2024-04-04T06:13:29.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.627+0000 7ff511800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff50c0af2e0 0x7ff50c151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.627+0000 7ff511800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff50c0d0990 0x7ff50c072d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.627+0000 7ff511800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff50c153580 0x7ff50c0ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.627+0000 7ff511800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c153960 con 0x7ff50c0d0990 2024-04-04T06:13:29.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.627+0000 7ff511800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c15d4e0 con 0x7ff50c153580 2024-04-04T06:13:29.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.627+0000 7ff511800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff50c0af6c0 con 0x7ff50c0af2e0 2024-04-04T06:13:29.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.627+0000 7ff503e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff50c0d0990 0x7ff50c072d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.627+0000 7ff503e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff50c0d0990 0x7ff50c072d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38884/0 (socket says 172.21.15.159:38884) 2024-04-04T06:13:29.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.627+0000 7ff503e00640 1 -- 172.21.15.159:0/3617114737 learned_addr learned my addr 172.21.15.159:0/3617114737 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:29.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.627+0000 7ff50aa00640 1 --2- 172.21.15.159:0/3617114737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff50c0af2e0 0x7ff50c151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.627+0000 7ff503e00640 1 -- 172.21.15.159:0/3617114737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff50c0af2e0 msgr2=0x7ff50c151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:29.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.627+0000 7ff503e00640 1 --2- 172.21.15.159:0/3617114737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff50c0af2e0 0x7ff50c151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.627+0000 7ff503e00640 1 -- 172.21.15.159:0/3617114737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff50c153580 msgr2=0x7ff50c0ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:29.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.627+0000 7ff503e00640 1 --2- 172.21.15.159:0/3617114737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff50c153580 0x7ff50c0ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.627+0000 7ff503e00640 1 -- 172.21.15.159:0/3617114737 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c0cd2b0 con 0x7ff50c0d0990 2024-04-04T06:13:29.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.628+0000 7ff503e00640 1 --2- 172.21.15.159:0/3617114737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff50c0d0990 0x7ff50c072d20 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4000c00 tx=0x7ff4f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:29.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.628+0000 7ff502a00640 1 -- 172.21.15.159:0/3617114737 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4035040 con 0x7ff50c0d0990 2024-04-04T06:13:29.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.628+0000 7ff502a00640 1 -- 172.21.15.159:0/3617114737 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4f4030d00 con 0x7ff50c0d0990 2024-04-04T06:13:29.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.628+0000 7ff502a00640 1 -- 172.21.15.159:0/3617114737 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f40456a0 con 0x7ff50c0d0990 2024-04-04T06:13:29.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.628+0000 7ff511800640 1 -- 172.21.15.159:0/3617114737 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50c073490 con 0x7ff50c0d0990 2024-04-04T06:13:29.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.628+0000 7ff511800640 1 -- 172.21.15.159:0/3617114737 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff50c1586e0 con 0x7ff50c0d0990 2024-04-04T06:13:29.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.630+0000 7ff502a00640 1 -- 172.21.15.159:0/3617114737 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4f4038020 con 0x7ff50c0d0990 2024-04-04T06:13:29.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.631+0000 7ff502a00640 1 --2- 172.21.15.159:0/3617114737 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff4e806c8f0 0x7ff4e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.631+0000 7ff502a00640 1 -- 172.21.15.159:0/3617114737 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7ff4f40ade20 con 0x7ff50c0d0990 2024-04-04T06:13:29.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.632+0000 7ff50aa00640 1 --2- 172.21.15.159:0/3617114737 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff4e806c8f0 0x7ff4e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.633+0000 7ff50aa00640 1 --2- 172.21.15.159:0/3617114737 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff4e806c8f0 0x7ff4e806edb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc001640 tx=0x7ff4fc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:29.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.634+0000 7ff514604740 1 Processor -- start 2024-04-04T06:13:29.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.634+0000 7ff514604740 1 -- start start 2024-04-04T06:13:29.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.635+0000 7ff514604740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f5f580 0x5563f9f32520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.635+0000 7ff514604740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5563f9f5f960 0x5563f9f62310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.635+0000 7ff514604740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563f9f32a60 0x5563f9f62850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.636+0000 7ff514604740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5563f9e9d6d0 con 0x5563f9f5f580 2024-04-04T06:13:29.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.636+0000 7ff514604740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5563f9e45770 con 0x5563f9f32a60 2024-04-04T06:13:29.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.636+0000 7ff514604740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5563f9e99680 con 0x5563f9f5f960 2024-04-04T06:13:29.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.636+0000 7ff50aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f5f580 0x5563f9f32520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.636+0000 7ff50aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f5f580 0x5563f9f32520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38890/0 (socket says 172.21.15.159:38890) 2024-04-04T06:13:29.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.636+0000 7ff50aa00640 1 -- 172.21.15.159:0/3526366494 learned_addr learned my addr 172.21.15.159:0/3526366494 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:29.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.636+0000 7ff50b400640 1 --2- 172.21.15.159:0/3526366494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563f9f32a60 0x5563f9f62850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.636+0000 7ff503e00640 1 --2- 172.21.15.159:0/3526366494 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5563f9f5f960 0x5563f9f62310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.636+0000 7ff50aa00640 1 -- 172.21.15.159:0/3526366494 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5563f9f5f960 msgr2=0x5563f9f62310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:29.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.636+0000 7ff50aa00640 1 --2- 172.21.15.159:0/3526366494 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5563f9f5f960 0x5563f9f62310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.636+0000 7ff50aa00640 1 -- 172.21.15.159:0/3526366494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563f9f32a60 msgr2=0x5563f9f62850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:29.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.636+0000 7ff50aa00640 1 --2- 172.21.15.159:0/3526366494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563f9f32a60 0x5563f9f62850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.636+0000 7ff50aa00640 1 -- 172.21.15.159:0/3526366494 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563f9c7b7d0 con 0x5563f9f5f580 2024-04-04T06:13:29.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.637+0000 7ff50aa00640 1 --2- 172.21.15.159:0/3526366494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f5f580 0x5563f9f32520 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0052f0 tx=0x7ff4fc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ed7148cf9c8526 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:29.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.637+0000 7ff4fb400640 1 -- 172.21.15.159:0/3526366494 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc00e530 con 0x5563f9f5f580 2024-04-04T06:13:29.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.637+0000 7ff4fb400640 1 -- 172.21.15.159:0/3526366494 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4fc00e6d0 con 0x5563f9f5f580 2024-04-04T06:13:29.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.637+0000 7ff4fb400640 1 -- 172.21.15.159:0/3526366494 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc00ea80 con 0x5563f9f5f580 2024-04-04T06:13:29.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.638+0000 7ff514604740 1 -- 172.21.15.159:0/3526366494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f5f580 msgr2=0x5563f9f32520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:29.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.638+0000 7ff514604740 1 --2- 172.21.15.159:0/3526366494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f5f580 0x5563f9f32520 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0052f0 tx=0x7ff4fc011000 comp rx=0 tx=0).stop 2024-04-04T06:13:29.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.638+0000 7ff514604740 1 -- 172.21.15.159:0/3526366494 shutdown_connections 2024-04-04T06:13:29.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.638+0000 7ff514604740 1 --2- 172.21.15.159:0/3526366494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563f9f32a60 0x5563f9f62850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.638+0000 7ff514604740 1 --2- 172.21.15.159:0/3526366494 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5563f9f5f960 0x5563f9f62310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.638+0000 7ff514604740 1 --2- 172.21.15.159:0/3526366494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f5f580 0x5563f9f32520 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.638+0000 7ff514604740 1 -- 172.21.15.159:0/3526366494 >> 172.21.15.159:0/3526366494 conn(0x5563f9f34790 msgr2=0x5563f9f7d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:29.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.638+0000 7ff514604740 1 -- 172.21.15.159:0/3526366494 shutdown_connections 2024-04-04T06:13:29.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.638+0000 7ff514604740 1 -- 172.21.15.159:0/3526366494 wait complete. 2024-04-04T06:13:29.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.639+0000 7ff514604740 1 -- start start 2024-04-04T06:13:29.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.639+0000 7ff514604740 1 Processor -- start 2024-04-04T06:13:29.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.640+0000 7ff514604740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563f9f5f580 0x5563f9f61fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.640+0000 7ff514604740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f5f960 0x5563f9f8d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.640+0000 7ff514604740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5563f9f58d10 0x5563f9f641a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.640+0000 7ff514604740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5563f9f36e30 con 0x5563f9f5f960 2024-04-04T06:13:29.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.640+0000 7ff514604740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5563f9e58aa0 con 0x5563f9f5f580 2024-04-04T06:13:29.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.640+0000 7ff514604740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5563f9f39670 con 0x5563f9f58d10 2024-04-04T06:13:29.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.640+0000 7ff514604740 10 client.0 _ll_register_callbacks cb 0x5563f9e58490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:13:29.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.640+0000 7ff503e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f5f960 0x5563f9f8d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.640+0000 7ff503e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f5f960 0x5563f9f8d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38906/0 (socket says 172.21.15.159:38906) 2024-04-04T06:13:29.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.640+0000 7ff503e00640 1 -- 172.21.15.159:0/688454241 learned_addr learned my addr 172.21.15.159:0/688454241 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:29.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.640+0000 7ff50aa00640 1 --2- 172.21.15.159:0/688454241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563f9f5f580 0x5563f9f61fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.640+0000 7ff50b400640 1 --2- 172.21.15.159:0/688454241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5563f9f58d10 0x5563f9f641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.640+0000 7ff503e00640 1 -- 172.21.15.159:0/688454241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5563f9f58d10 msgr2=0x5563f9f641a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:29.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.641+0000 7ff503e00640 1 --2- 172.21.15.159:0/688454241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5563f9f58d10 0x5563f9f641a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.641+0000 7ff503e00640 1 -- 172.21.15.159:0/688454241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563f9f5f580 msgr2=0x5563f9f61fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:29.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.641+0000 7ff503e00640 1 --2- 172.21.15.159:0/688454241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563f9f5f580 0x5563f9f61fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.641+0000 7ff503e00640 1 -- 172.21.15.159:0/688454241 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563f9bb13f0 con 0x5563f9f5f960 2024-04-04T06:13:29.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.641+0000 7ff50b400640 1 --2- 172.21.15.159:0/688454241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5563f9f58d10 0x5563f9f641a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:13:29.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.641+0000 7ff503e00640 1 --2- 172.21.15.159:0/688454241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f5f960 0x5563f9f8d1c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4044000 tx=0x7ff4f4031870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8fccda6d652b237 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:29.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.641+0000 7ff4efe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:13:29.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.641+0000 7ff4efe00640 1 -- 172.21.15.159:0/688454241 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f40aa060 con 0x5563f9f5f960 2024-04-04T06:13:29.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.641+0000 7ff4efe00640 1 -- 172.21.15.159:0/688454241 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7ff4f4084230 con 0x5563f9f5f960 2024-04-04T06:13:29.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.641+0000 7ff514604740 1 -- 172.21.15.159:0/688454241 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4fc00e6d0 con 0x5563f9f5f960 2024-04-04T06:13:29.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.641+0000 7ff4efe00640 1 -- 172.21.15.159:0/688454241 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7ff4f4094070 con 0x5563f9f5f960 2024-04-04T06:13:29.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.641+0000 7ff514604740 10 client.7704 fetch_fsmap learned FSMap version 69 2024-04-04T06:13:29.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.641+0000 7ff514604740 1 -- 172.21.15.159:0/688454241 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x5563f9b710f0 con 0x5563f9f5f960 2024-04-04T06:13:29.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.642+0000 7ff4efe00640 1 -- 172.21.15.159:0/688454241 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4fc00e6d0 con 0x5563f9f5f960 2024-04-04T06:13:29.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.642+0000 7ff4efe00640 1 -- 172.21.15.159:0/688454241 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 69) ==== 3494+0+0 (secure 0 0 0) 0x7ff4f407af30 con 0x5563f9f5f960 2024-04-04T06:13:29.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.642+0000 7ff514604740 10 client.7704 fetch_fsmap finished waiting for FSMap version 69 2024-04-04T06:13:29.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.642+0000 7ff514604740 10 client.7704 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-04T06:13:29.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.642+0000 7ff514604740 20 client.7704 populate_metadata read hostname 'smithi159' 2024-04-04T06:13:29.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.642+0000 7ff514604740 1 --2- 172.21.15.159:0/688454241 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x5563f9f5a900 0x5563f9f5ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.642+0000 7ff514604740 4 client.7704 mds_command: new command op to 7512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:13:29.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.642+0000 7ff514604740 1 -- 172.21.15.159:0/688454241 --> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563f9eaac10 con 0x5563f9f5a900 2024-04-04T06:13:29.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.642+0000 7ff50aa00640 1 --2- 172.21.15.159:0/688454241 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x5563f9f5a900 0x5563f9f5ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.643+0000 7ff50aa00640 1 --2- 172.21.15.159:0/688454241 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x5563f9f5a900 0x5563f9f5ad40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0ff0739aa6e4c75 server_cookie=479a0b7b06689299 in_seq=0 out_seq=0 2024-04-04T06:13:29.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.643+0000 7ff4efe00640 10 client.7704 ms_handle_connect on v2:172.21.15.159:6833/3796123128 2024-04-04T06:13:29.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.643+0000 7ff4efe00640 1 -- 172.21.15.159:0/688454241 <== mds.0 v2:172.21.15.159:6833/3796123128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5563f9eaac10 con 0x5563f9f5a900 2024-04-04T06:13:29.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.643+0000 7ff4efe00640 10 client.7704 handle_command_reply: tid=0 2024-04-04T06:13:29.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.643+0000 7ff514604740 1 client.7704 shutdown 2024-04-04T06:13:29.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.644+0000 7ff514604740 1 -- 172.21.15.159:0/688454241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f5f960 msgr2=0x5563f9f8d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:29.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.644+0000 7ff514604740 1 --2- 172.21.15.159:0/688454241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f5f960 0x5563f9f8d1c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4044000 tx=0x7ff4f4031870 comp rx=0 tx=0).stop 2024-04-04T06:13:29.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.644+0000 7ff514604740 1 -- 172.21.15.159:0/688454241 shutdown_connections 2024-04-04T06:13:29.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.644+0000 7ff514604740 1 --2- 172.21.15.159:0/688454241 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x5563f9f5a900 0x5563f9f5ad40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.644+0000 7ff514604740 1 --2- 172.21.15.159:0/688454241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5563f9f58d10 0x5563f9f641a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.644+0000 7ff514604740 1 --2- 172.21.15.159:0/688454241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f5f960 0x5563f9f8d1c0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.644+0000 7ff514604740 1 --2- 172.21.15.159:0/688454241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563f9f5f580 0x5563f9f61fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.644+0000 7ff514604740 1 -- 172.21.15.159:0/688454241 >> 172.21.15.159:0/688454241 conn(0x5563f9f32a60 msgr2=0x5563f9f37420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:29.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.644+0000 7ff4efe00640 0 client.7704 ms_handle_reset on v2:172.21.15.159:6833/3796123128 2024-04-04T06:13:29.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.644+0000 7ff514604740 1 -- 172.21.15.159:0/688454241 shutdown_connections 2024-04-04T06:13:29.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.644+0000 7ff514604740 1 -- 172.21.15.159:0/688454241 wait complete. 2024-04-04T06:13:29.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.644+0000 7ff514604740 20 client.7704 trim_cache size 0 max 16384 2024-04-04T06:13:29.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.681+0000 7ff514604740 1 Processor -- start 2024-04-04T06:13:29.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.681+0000 7ff514604740 1 -- start start 2024-04-04T06:13:29.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.682+0000 7ff514604740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f58d10 0x5563f9f364e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.682+0000 7ff514604740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563f9f5f580 0x5563f9f80a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.682+0000 7ff514604740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5563f9f5f960 0x5563f9fa1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.682+0000 7ff514604740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c200d90 con 0x5563f9f58d10 2024-04-04T06:13:29.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.682+0000 7ff514604740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1ffdc0 con 0x5563f9f5f580 2024-04-04T06:13:29.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.682+0000 7ff514604740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff50c159b90 con 0x5563f9f5f960 2024-04-04T06:13:29.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.682+0000 7ff50aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f58d10 0x5563f9f364e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.682+0000 7ff50aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f58d10 0x5563f9f364e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38908/0 (socket says 172.21.15.159:38908) 2024-04-04T06:13:29.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.682+0000 7ff50aa00640 1 -- 172.21.15.159:0/67192266 learned_addr learned my addr 172.21.15.159:0/67192266 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:29.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.683+0000 7ff50aa00640 1 -- 172.21.15.159:0/67192266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5563f9f5f960 msgr2=0x5563f9fa1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:29.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.683+0000 7ff50aa00640 1 --2- 172.21.15.159:0/67192266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5563f9f5f960 0x5563f9fa1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.683+0000 7ff50aa00640 1 -- 172.21.15.159:0/67192266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563f9f5f580 msgr2=0x5563f9f80a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:29.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.683+0000 7ff50aa00640 1 --2- 172.21.15.159:0/67192266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563f9f5f580 0x5563f9f80a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.683+0000 7ff50aa00640 1 -- 172.21.15.159:0/67192266 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563f9f35420 con 0x5563f9f58d10 2024-04-04T06:13:29.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.683+0000 7ff50aa00640 1 --2- 172.21.15.159:0/67192266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f58d10 0x5563f9f364e0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc014000 tx=0x7ff4fc005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a800aa0ee8f21f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:29.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.683+0000 7ff4faa00640 1 -- 172.21.15.159:0/67192266 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc0100d0 con 0x5563f9f58d10 2024-04-04T06:13:29.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.683+0000 7ff4faa00640 1 -- 172.21.15.159:0/67192266 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4fc027040 con 0x5563f9f58d10 2024-04-04T06:13:29.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.683+0000 7ff4faa00640 1 -- 172.21.15.159:0/67192266 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc0104b0 con 0x5563f9f58d10 2024-04-04T06:13:29.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.684+0000 7ff514604740 1 -- 172.21.15.159:0/67192266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f58d10 msgr2=0x5563f9f364e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:29.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.684+0000 7ff514604740 1 --2- 172.21.15.159:0/67192266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f58d10 0x5563f9f364e0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc014000 tx=0x7ff4fc005000 comp rx=0 tx=0).stop 2024-04-04T06:13:29.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.684+0000 7ff514604740 1 -- 172.21.15.159:0/67192266 shutdown_connections 2024-04-04T06:13:29.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.684+0000 7ff514604740 1 --2- 172.21.15.159:0/67192266 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5563f9f5f960 0x5563f9fa1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.684+0000 7ff514604740 1 --2- 172.21.15.159:0/67192266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563f9f5f580 0x5563f9f80a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.684+0000 7ff514604740 1 --2- 172.21.15.159:0/67192266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f58d10 0x5563f9f364e0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.684+0000 7ff514604740 1 -- 172.21.15.159:0/67192266 >> 172.21.15.159:0/67192266 conn(0x5563f9f32a60 msgr2=0x5563f9f5fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:29.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.684+0000 7ff514604740 1 -- 172.21.15.159:0/67192266 shutdown_connections 2024-04-04T06:13:29.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.684+0000 7ff514604740 1 -- 172.21.15.159:0/67192266 wait complete. 2024-04-04T06:13:29.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.685+0000 7ff514604740 1 -- start start 2024-04-04T06:13:29.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.685+0000 7ff514604740 1 Processor -- start 2024-04-04T06:13:29.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.687+0000 7ff514604740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f58d10 0x5563f9ff7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.687+0000 7ff514604740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5563f9f5f580 0x5563f9f5e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.687+0000 7ff514604740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563f9f5f960 0x5563f9f63310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.687+0000 7ff514604740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5563f9f62ed0 con 0x5563f9f58d10 2024-04-04T06:13:29.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.687+0000 7ff514604740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1ff770 con 0x5563f9f5f960 2024-04-04T06:13:29.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.687+0000 7ff514604740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5563f9f7f710 con 0x5563f9f5f580 2024-04-04T06:13:29.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.687+0000 7ff514604740 10 client.0 _ll_register_callbacks cb 0x7ff4f4006cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:13:29.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.687+0000 7ff50aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f58d10 0x5563f9ff7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.687+0000 7ff50aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f58d10 0x5563f9ff7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38924/0 (socket says 172.21.15.159:38924) 2024-04-04T06:13:29.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.687+0000 7ff50aa00640 1 -- 172.21.15.159:0/2671235920 learned_addr learned my addr 172.21.15.159:0/2671235920 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:29.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.688+0000 7ff50b400640 1 --2- 172.21.15.159:0/2671235920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563f9f5f960 0x5563f9f63310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.688+0000 7ff50aa00640 1 -- 172.21.15.159:0/2671235920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5563f9f5f580 msgr2=0x5563f9f5e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:29.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.688+0000 7ff50aa00640 1 --2- 172.21.15.159:0/2671235920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5563f9f5f580 0x5563f9f5e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.688+0000 7ff50aa00640 1 -- 172.21.15.159:0/2671235920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563f9f5f960 msgr2=0x5563f9f63310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:29.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.688+0000 7ff50aa00640 1 --2- 172.21.15.159:0/2671235920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563f9f5f960 0x5563f9f63310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.688+0000 7ff50aa00640 1 -- 172.21.15.159:0/2671235920 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563f9f5edc0 con 0x5563f9f58d10 2024-04-04T06:13:29.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.688+0000 7ff50b400640 1 --2- 172.21.15.159:0/2671235920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563f9f5f960 0x5563f9f63310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:13:29.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.688+0000 7ff50aa00640 1 --2- 172.21.15.159:0/2671235920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f58d10 0x5563f9ff7020 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc003f50 tx=0x7ff4fc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4811944cd3f95422 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:29.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.688+0000 7ff4efe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:13:29.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.688+0000 7ff4efe00640 1 -- 172.21.15.159:0/2671235920 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc0102d0 con 0x5563f9f58d10 2024-04-04T06:13:29.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.688+0000 7ff4efe00640 1 -- 172.21.15.159:0/2671235920 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7ff4fc010470 con 0x5563f9f58d10 2024-04-04T06:13:29.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.688+0000 7ff514604740 1 -- 172.21.15.159:0/2671235920 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4fc027040 con 0x5563f9f58d10 2024-04-04T06:13:29.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.689+0000 7ff4efe00640 1 -- 172.21.15.159:0/2671235920 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7ff4fc010670 con 0x5563f9f58d10 2024-04-04T06:13:29.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.689+0000 7ff514604740 10 client.7710 fetch_fsmap learned FSMap version 69 2024-04-04T06:13:29.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.689+0000 7ff514604740 1 -- 172.21.15.159:0/2671235920 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x5563f9fa0780 con 0x5563f9f58d10 2024-04-04T06:13:29.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.689+0000 7ff4efe00640 1 -- 172.21.15.159:0/2671235920 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4fc027040 con 0x5563f9f58d10 2024-04-04T06:13:29.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.689+0000 7ff4efe00640 1 -- 172.21.15.159:0/2671235920 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 69) ==== 3494+0+0 (secure 0 0 0) 0x7ff4fc0300d0 con 0x5563f9f58d10 2024-04-04T06:13:29.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.689+0000 7ff514604740 10 client.7710 fetch_fsmap finished waiting for FSMap version 69 2024-04-04T06:13:29.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.689+0000 7ff514604740 10 client.7710 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-04T06:13:29.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.689+0000 7ff514604740 20 client.7710 populate_metadata read hostname 'smithi159' 2024-04-04T06:13:29.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.689+0000 7ff514604740 1 --2- 172.21.15.159:0/2671235920 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x5563f9fa0340 0x5563f9f63850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:29.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.689+0000 7ff514604740 4 client.7710 mds_command: new command op to 7512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-04T06:13:29.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.689+0000 7ff514604740 1 -- 172.21.15.159:0/2671235920 --> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5563f9ea4f60 con 0x5563f9fa0340 2024-04-04T06:13:29.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.690+0000 7ff503e00640 1 --2- 172.21.15.159:0/2671235920 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x5563f9fa0340 0x5563f9f63850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:29.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.690+0000 7ff503e00640 1 --2- 172.21.15.159:0/2671235920 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x5563f9fa0340 0x5563f9f63850 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=997d476af9c4ceea server_cookie=3dc1007e1d0ba2b6 in_seq=0 out_seq=0 2024-04-04T06:13:29.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.690+0000 7ff4efe00640 10 client.7710 ms_handle_connect on v2:172.21.15.159:6833/3796123128 2024-04-04T06:13:29.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.690+0000 7ff4efe00640 1 -- 172.21.15.159:0/2671235920 <== mds.0 v2:172.21.15.159:6833/3796123128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x5563f9ea4f60 con 0x5563f9fa0340 2024-04-04T06:13:29.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.690+0000 7ff4efe00640 10 client.7710 handle_command_reply: tid=0 2024-04-04T06:13:29.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.690+0000 7ff514604740 1 client.7710 shutdown 2024-04-04T06:13:29.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.691+0000 7ff514604740 1 -- 172.21.15.159:0/2671235920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f58d10 msgr2=0x5563f9ff7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:29.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.691+0000 7ff514604740 1 --2- 172.21.15.159:0/2671235920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f58d10 0x5563f9ff7020 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc003f50 tx=0x7ff4fc000bc0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.691+0000 7ff514604740 1 -- 172.21.15.159:0/2671235920 shutdown_connections 2024-04-04T06:13:29.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.691+0000 7ff514604740 1 --2- 172.21.15.159:0/2671235920 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x5563f9fa0340 0x5563f9f63850 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.691+0000 7ff514604740 1 --2- 172.21.15.159:0/2671235920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563f9f5f960 0x5563f9f63310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.691+0000 7ff514604740 1 --2- 172.21.15.159:0/2671235920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5563f9f5f580 0x5563f9f5e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.691+0000 7ff514604740 1 --2- 172.21.15.159:0/2671235920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563f9f58d10 0x5563f9ff7020 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.691+0000 7ff514604740 1 -- 172.21.15.159:0/2671235920 >> 172.21.15.159:0/2671235920 conn(0x5563f9f32a60 msgr2=0x5563f9fa1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:29.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.691+0000 7ff4efe00640 0 client.7710 ms_handle_reset on v2:172.21.15.159:6833/3796123128 2024-04-04T06:13:29.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.691+0000 7ff514604740 1 -- 172.21.15.159:0/2671235920 shutdown_connections 2024-04-04T06:13:29.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.691+0000 7ff514604740 1 -- 172.21.15.159:0/2671235920 wait complete. 2024-04-04T06:13:29.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.691+0000 7ff514604740 20 client.7710 trim_cache size 0 max 16384 2024-04-04T06:13:29.688 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:13:29.688 INFO:teuthology.orchestra.run.smithi159.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-04T06:13:29.688 INFO:teuthology.orchestra.run.smithi159.stdout: "scrubs": { 2024-04-04T06:13:29.688 INFO:teuthology.orchestra.run.smithi159.stdout: "3be18bbd-ff6a-4ca3-aedf-c1550fc49ad7": { 2024-04-04T06:13:29.688 INFO:teuthology.orchestra.run.smithi159.stdout: "path": "/dir", 2024-04-04T06:13:29.688 INFO:teuthology.orchestra.run.smithi159.stdout: "tag": "3be18bbd-ff6a-4ca3-aedf-c1550fc49ad7", 2024-04-04T06:13:29.688 INFO:teuthology.orchestra.run.smithi159.stdout: "options": "recursive,repair" 2024-04-04T06:13:29.689 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:13:29.689 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:13:29.689 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:13:29.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.692+0000 7ff511800640 1 -- 172.21.15.159:0/3617114737 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff4e806c8f0 msgr2=0x7ff4e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:29.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.692+0000 7ff511800640 1 --2- 172.21.15.159:0/3617114737 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff4e806c8f0 0x7ff4e806edb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc001640 tx=0x7ff4fc009040 comp rx=0 tx=0).stop 2024-04-04T06:13:29.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.692+0000 7ff511800640 1 -- 172.21.15.159:0/3617114737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff50c0d0990 msgr2=0x7ff50c072d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:29.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.692+0000 7ff511800640 1 --2- 172.21.15.159:0/3617114737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff50c0d0990 0x7ff50c072d20 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4000c00 tx=0x7ff4f40046c0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.692+0000 7ff511800640 1 -- 172.21.15.159:0/3617114737 shutdown_connections 2024-04-04T06:13:29.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.692+0000 7ff511800640 1 --2- 172.21.15.159:0/3617114737 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff4e806c8f0 0x7ff4e806edb0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.692+0000 7ff511800640 1 --2- 172.21.15.159:0/3617114737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff50c153580 0x7ff50c0ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.692+0000 7ff511800640 1 --2- 172.21.15.159:0/3617114737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff50c0d0990 0x7ff50c072d20 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.692+0000 7ff511800640 1 --2- 172.21.15.159:0/3617114737 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff50c0af2e0 0x7ff50c151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:29.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.692+0000 7ff511800640 1 -- 172.21.15.159:0/3617114737 >> 172.21.15.159:0/3617114737 conn(0x7ff50c06b750 msgr2=0x7ff50c15ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:29.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.692+0000 7ff511800640 1 -- 172.21.15.159:0/3617114737 shutdown_connections 2024-04-04T06:13:29.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:29.692+0000 7ff511800640 1 -- 172.21.15.159:0/3617114737 wait complete. 2024-04-04T06:13:29.702 INFO:tasks.cephfs.filesystem:scrub status for tag:3be18bbd-ff6a-4ca3-aedf-c1550fc49ad7 - {'path': '/dir', 'tag': '3be18bbd-ff6a-4ca3-aedf-c1550fc49ad7', 'options': 'recursive,repair'} 2024-04-04T06:13:42.791 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:13:42.799 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:13:59.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:13:59.704 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-04T06:13:59.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.928+0000 7fea78200640 1 Processor -- start 2024-04-04T06:13:59.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.928+0000 7fea78200640 1 -- start start 2024-04-04T06:13:59.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.930+0000 7fea78200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fea700d0280 0x7fea700d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:59.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.930+0000 7fea78200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fea70141c10 0x7fea70071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:59.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.930+0000 7fea78200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea70071eb0 0x7fea7006d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:59.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.930+0000 7fea78200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fea7015ec80 con 0x7fea70141c10 2024-04-04T06:13:59.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.930+0000 7fea78200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fea700d0ba0 con 0x7fea700d0280 2024-04-04T06:13:59.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.930+0000 7fea78200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea7015ef20 con 0x7fea70071eb0 2024-04-04T06:13:59.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.930+0000 7fea74e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fea70141c10 0x7fea70071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:59.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.930+0000 7fea74e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fea70141c10 0x7fea70071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60352/0 (socket says 172.21.15.159:60352) 2024-04-04T06:13:59.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.930+0000 7fea74e00640 1 -- 172.21.15.159:0/2528603948 learned_addr learned my addr 172.21.15.159:0/2528603948 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:59.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.931+0000 7fea76200640 1 --2- 172.21.15.159:0/2528603948 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea70071eb0 0x7fea7006d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:59.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.931+0000 7fea75800640 1 --2- 172.21.15.159:0/2528603948 >> v2:172.21.15.178:3300/0 conn(0x7fea700d0280 0x7fea700d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:59.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.931+0000 7fea74e00640 1 -- 172.21.15.159:0/2528603948 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea70071eb0 msgr2=0x7fea7006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:59.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.931+0000 7fea74e00640 1 --2- 172.21.15.159:0/2528603948 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea70071eb0 0x7fea7006d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.931+0000 7fea74e00640 1 -- 172.21.15.159:0/2528603948 >> v2:172.21.15.178:3300/0 conn(0x7fea700d0280 msgr2=0x7fea700d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:59.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.931+0000 7fea74e00640 1 --2- 172.21.15.159:0/2528603948 >> v2:172.21.15.178:3300/0 conn(0x7fea700d0280 0x7fea700d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.931+0000 7fea74e00640 1 -- 172.21.15.159:0/2528603948 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7015f200 con 0x7fea70141c10 2024-04-04T06:13:59.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.931+0000 7fea76200640 1 --2- 172.21.15.159:0/2528603948 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea70071eb0 0x7fea7006d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:13:59.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.931+0000 7fea75800640 1 --2- 172.21.15.159:0/2528603948 >> v2:172.21.15.178:3300/0 conn(0x7fea700d0280 0x7fea700d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:13:59.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.932+0000 7fea74e00640 1 --2- 172.21.15.159:0/2528603948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea70141c10 0x7fea70071970 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fea700d1050 tx=0x7fea5c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69b476f3e737d3b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:59.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.932+0000 7fea6fe00640 1 -- 172.21.15.159:0/2528603948 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c035020 con 0x7fea70141c10 2024-04-04T06:13:59.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.932+0000 7fea6fe00640 1 -- 172.21.15.159:0/2528603948 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea5c031870 con 0x7fea70141c10 2024-04-04T06:13:59.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.932+0000 7fea6fe00640 1 -- 172.21.15.159:0/2528603948 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c031c40 con 0x7fea70141c10 2024-04-04T06:13:59.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.933+0000 7fea78200640 1 -- 172.21.15.159:0/2528603948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea70141c10 msgr2=0x7fea70071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:59.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.933+0000 7fea78200640 1 --2- 172.21.15.159:0/2528603948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea70141c10 0x7fea70071970 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fea700d1050 tx=0x7fea5c02b910 comp rx=0 tx=0).stop 2024-04-04T06:13:59.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.933+0000 7fea78200640 1 -- 172.21.15.159:0/2528603948 shutdown_connections 2024-04-04T06:13:59.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.933+0000 7fea78200640 1 --2- 172.21.15.159:0/2528603948 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea70071eb0 0x7fea7006d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.933+0000 7fea78200640 1 --2- 172.21.15.159:0/2528603948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea70141c10 0x7fea70071970 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.933+0000 7fea78200640 1 --2- 172.21.15.159:0/2528603948 >> v2:172.21.15.178:3300/0 conn(0x7fea700d0280 0x7fea700d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.933+0000 7fea78200640 1 -- 172.21.15.159:0/2528603948 >> 172.21.15.159:0/2528603948 conn(0x7fea7006b750 msgr2=0x7fea7006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:59.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.933+0000 7fea78200640 1 -- 172.21.15.159:0/2528603948 shutdown_connections 2024-04-04T06:13:59.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.933+0000 7fea78200640 1 -- 172.21.15.159:0/2528603948 wait complete. 2024-04-04T06:13:59.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.935+0000 7fea78200640 1 Processor -- start 2024-04-04T06:13:59.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.935+0000 7fea78200640 1 -- start start 2024-04-04T06:13:59.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.936+0000 7fea78200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea70071eb0 0x7fea7009bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:59.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.936+0000 7fea78200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea700d0280 0x7fea7009a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:59.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.936+0000 7fea78200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea70141c10 0x7fea7009a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:59.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.936+0000 7fea78200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fea70158e90 con 0x7fea70141c10 2024-04-04T06:13:59.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.936+0000 7fea78200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea7015f7c0 con 0x7fea70071eb0 2024-04-04T06:13:59.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.936+0000 7fea78200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea700aeb40 con 0x7fea700d0280 2024-04-04T06:13:59.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.936+0000 7fea76200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea70141c10 0x7fea7009a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:59.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.936+0000 7fea76200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea70141c10 0x7fea7009a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60356/0 (socket says 172.21.15.159:60356) 2024-04-04T06:13:59.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.936+0000 7fea76200640 1 -- 172.21.15.159:0/544813454 learned_addr learned my addr 172.21.15.159:0/544813454 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:59.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.936+0000 7fea74e00640 1 --2- 172.21.15.159:0/544813454 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea700d0280 0x7fea7009a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:59.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.936+0000 7fea75800640 1 --2- 172.21.15.159:0/544813454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea70071eb0 0x7fea7009bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:59.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.937+0000 7fea76200640 1 -- 172.21.15.159:0/544813454 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea700d0280 msgr2=0x7fea7009a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:59.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.937+0000 7fea76200640 1 --2- 172.21.15.159:0/544813454 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea700d0280 0x7fea7009a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.937+0000 7fea76200640 1 -- 172.21.15.159:0/544813454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea70071eb0 msgr2=0x7fea7009bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:59.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.937+0000 7fea76200640 1 --2- 172.21.15.159:0/544813454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea70071eb0 0x7fea7009bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.937+0000 7fea76200640 1 -- 172.21.15.159:0/544813454 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea700adf30 con 0x7fea70141c10 2024-04-04T06:13:59.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.937+0000 7fea75800640 1 --2- 172.21.15.159:0/544813454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea70071eb0 0x7fea7009bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:13:59.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.937+0000 7fea76200640 1 --2- 172.21.15.159:0/544813454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea70141c10 0x7fea7009a6d0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fea6800cf20 tx=0x7fea6800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:59.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.937+0000 7fea6d600640 1 -- 172.21.15.159:0/544813454 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6801a070 con 0x7fea70141c10 2024-04-04T06:13:59.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.937+0000 7fea6d600640 1 -- 172.21.15.159:0/544813454 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea6800d840 con 0x7fea70141c10 2024-04-04T06:13:59.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.937+0000 7fea6d600640 1 -- 172.21.15.159:0/544813454 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6800dc30 con 0x7fea70141c10 2024-04-04T06:13:59.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.938+0000 7fea78200640 1 -- 172.21.15.159:0/544813454 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea700ae210 con 0x7fea70141c10 2024-04-04T06:13:59.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.938+0000 7fea78200640 1 -- 172.21.15.159:0/544813454 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea700ae7d0 con 0x7fea70141c10 2024-04-04T06:13:59.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.941+0000 7fea6d600640 1 -- 172.21.15.159:0/544813454 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea6801d070 con 0x7fea70141c10 2024-04-04T06:13:59.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.942+0000 7fea7af87740 1 Processor -- start 2024-04-04T06:13:59.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.942+0000 7fea7af87740 1 -- start start 2024-04-04T06:13:59.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.942+0000 7fea6d600640 1 --2- 172.21.15.159:0/544813454 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea5006c8f0 0x7fea5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:59.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.942+0000 7fea6d600640 1 -- 172.21.15.159:0/544813454 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5853+0+0 (secure 0 0 0) 0x7fea6808c480 con 0x7fea70141c10 2024-04-04T06:13:59.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.943+0000 7fea74e00640 1 --2- 172.21.15.159:0/544813454 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea5006c8f0 0x7fea5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:59.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.943+0000 7fea7af87740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962726580 0x5639626f9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:59.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.943+0000 7fea7af87740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563962726960 0x563962729310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:59.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.943+0000 7fea7af87740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639626f9a60 0x563962729850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:59.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.943+0000 7fea7af87740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5639626646d0 con 0x563962726580 2024-04-04T06:13:59.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.943+0000 7fea7af87740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56396260c770 con 0x563962726960 2024-04-04T06:13:59.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.943+0000 7fea7af87740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x563962660680 con 0x5639626f9a60 2024-04-04T06:13:59.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.943+0000 7fea75800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962726580 0x5639626f9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:59.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.943+0000 7fea75800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962726580 0x5639626f9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60372/0 (socket says 172.21.15.159:60372) 2024-04-04T06:13:59.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.943+0000 7fea75800640 1 -- 172.21.15.159:0/223826043 learned_addr learned my addr 172.21.15.159:0/223826043 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:59.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.944+0000 7fea74e00640 1 --2- 172.21.15.159:0/223826043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563962726960 0x563962729310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:59.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.944+0000 7fea76200640 1 --2- 172.21.15.159:0/223826043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639626f9a60 0x563962729850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:59.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.944+0000 7fea75800640 1 -- 172.21.15.159:0/223826043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639626f9a60 msgr2=0x563962729850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:59.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.944+0000 7fea75800640 1 --2- 172.21.15.159:0/223826043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639626f9a60 0x563962729850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.944+0000 7fea75800640 1 -- 172.21.15.159:0/223826043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563962726960 msgr2=0x563962729310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:59.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.944+0000 7fea75800640 1 --2- 172.21.15.159:0/223826043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563962726960 0x563962729310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.944+0000 7fea75800640 1 -- 172.21.15.159:0/223826043 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639624427d0 con 0x563962726580 2024-04-04T06:13:59.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.944+0000 7fea75800640 1 --2- 172.21.15.159:0/223826043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962726580 0x5639626f9520 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fea6400bae0 tx=0x7fea6400d1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b48bad27ac1e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:59.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.944+0000 7fea74e00640 1 --2- 172.21.15.159:0/544813454 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea5006c8f0 0x7fea5006edb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fea5c036000 tx=0x7fea5c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:59.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.944+0000 7fea62000640 1 -- 172.21.15.159:0/223826043 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6400dd40 con 0x563962726580 2024-04-04T06:13:59.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.944+0000 7fea62000640 1 -- 172.21.15.159:0/223826043 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea6400b030 con 0x563962726580 2024-04-04T06:13:59.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.944+0000 7fea62000640 1 -- 172.21.15.159:0/223826043 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea64012660 con 0x563962726580 2024-04-04T06:13:59.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.945+0000 7fea7af87740 1 -- 172.21.15.159:0/223826043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962726580 msgr2=0x5639626f9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:59.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.945+0000 7fea7af87740 1 --2- 172.21.15.159:0/223826043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962726580 0x5639626f9520 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fea6400bae0 tx=0x7fea6400d1c0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.945+0000 7fea7af87740 1 -- 172.21.15.159:0/223826043 shutdown_connections 2024-04-04T06:13:59.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.945+0000 7fea7af87740 1 --2- 172.21.15.159:0/223826043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5639626f9a60 0x563962729850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.945+0000 7fea7af87740 1 --2- 172.21.15.159:0/223826043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563962726960 0x563962729310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.945+0000 7fea7af87740 1 --2- 172.21.15.159:0/223826043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962726580 0x5639626f9520 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.945+0000 7fea7af87740 1 -- 172.21.15.159:0/223826043 >> 172.21.15.159:0/223826043 conn(0x5639626fb790 msgr2=0x563962744310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:59.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.945+0000 7fea7af87740 1 -- 172.21.15.159:0/223826043 shutdown_connections 2024-04-04T06:13:59.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.945+0000 7fea7af87740 1 -- 172.21.15.159:0/223826043 wait complete. 2024-04-04T06:13:59.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.946+0000 7fea7af87740 1 -- start start 2024-04-04T06:13:59.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.946+0000 7fea7af87740 1 Processor -- start 2024-04-04T06:13:59.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.947+0000 7fea7af87740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962726580 0x56396271b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:59.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.947+0000 7fea7af87740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563962726960 0x56396272a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:59.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.947+0000 7fea7af87740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563962757db0 0x563962758230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:59.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.947+0000 7fea7af87740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5639626fde30 con 0x563962726580 2024-04-04T06:13:59.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.947+0000 7fea7af87740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56396261faa0 con 0x563962757db0 2024-04-04T06:13:59.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.947+0000 7fea7af87740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x563962700670 con 0x563962726960 2024-04-04T06:13:59.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.947+0000 7fea7af87740 10 client.0 _ll_register_callbacks cb 0x56396261f490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:13:59.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.947+0000 7fea75800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962726580 0x56396271b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:59.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.948+0000 7fea75800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962726580 0x56396271b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60388/0 (socket says 172.21.15.159:60388) 2024-04-04T06:13:59.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.948+0000 7fea75800640 1 -- 172.21.15.159:0/53678123 learned_addr learned my addr 172.21.15.159:0/53678123 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:59.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.948+0000 7fea74e00640 1 --2- 172.21.15.159:0/53678123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563962726960 0x56396272a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:59.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.948+0000 7fea76200640 1 --2- 172.21.15.159:0/53678123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563962757db0 0x563962758230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:59.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.948+0000 7fea75800640 1 -- 172.21.15.159:0/53678123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563962726960 msgr2=0x56396272a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:59.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.948+0000 7fea75800640 1 --2- 172.21.15.159:0/53678123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563962726960 0x56396272a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.948+0000 7fea75800640 1 -- 172.21.15.159:0/53678123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563962757db0 msgr2=0x563962758230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:59.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.948+0000 7fea75800640 1 --2- 172.21.15.159:0/53678123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563962757db0 0x563962758230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.948+0000 7fea75800640 1 -- 172.21.15.159:0/53678123 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639623783f0 con 0x563962726580 2024-04-04T06:13:59.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.948+0000 7fea76200640 1 --2- 172.21.15.159:0/53678123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563962757db0 0x563962758230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:13:59.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.948+0000 7fea75800640 1 --2- 172.21.15.159:0/53678123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962726580 0x56396271b170 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fea64012690 tx=0x7fea64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b59077d7691c07b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:59.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.948+0000 7fea56a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:13:59.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.949+0000 7fea56a00640 1 -- 172.21.15.159:0/53678123 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6402a060 con 0x563962726580 2024-04-04T06:13:59.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.949+0000 7fea56a00640 1 -- 172.21.15.159:0/53678123 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5853+0+0 (secure 0 0 0) 0x7fea64012a80 con 0x563962726580 2024-04-04T06:13:59.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.949+0000 7fea7af87740 1 -- 172.21.15.159:0/53678123 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea6400b030 con 0x563962726580 2024-04-04T06:13:59.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.949+0000 7fea56a00640 1 -- 172.21.15.159:0/53678123 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fea64030050 con 0x563962726580 2024-04-04T06:13:59.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.949+0000 7fea7af87740 10 client.7722 fetch_fsmap learned FSMap version 69 2024-04-04T06:13:59.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.949+0000 7fea7af87740 1 -- 172.21.15.159:0/53678123 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x5639623380f0 con 0x563962726580 2024-04-04T06:13:59.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.949+0000 7fea56a00640 1 -- 172.21.15.159:0/53678123 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea6400b030 con 0x563962726580 2024-04-04T06:13:59.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.949+0000 7fea56a00640 1 -- 172.21.15.159:0/53678123 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 69) ==== 3494+0+0 (secure 0 0 0) 0x7fea64033020 con 0x563962726580 2024-04-04T06:13:59.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.949+0000 7fea7af87740 10 client.7722 fetch_fsmap finished waiting for FSMap version 69 2024-04-04T06:13:59.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.949+0000 7fea7af87740 10 client.7722 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-04T06:13:59.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.949+0000 7fea7af87740 20 client.7722 populate_metadata read hostname 'smithi159' 2024-04-04T06:13:59.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.949+0000 7fea7af87740 1 --2- 172.21.15.159:0/53678123 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x563962743970 0x563962727e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:59.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.949+0000 7fea7af87740 4 client.7722 mds_command: new command op to 7512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:13:59.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.949+0000 7fea7af87740 1 -- 172.21.15.159:0/53678123 --> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563962671c10 con 0x563962743970 2024-04-04T06:13:59.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.950+0000 7fea74e00640 1 --2- 172.21.15.159:0/53678123 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x563962743970 0x563962727e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:59.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.950+0000 7fea74e00640 1 --2- 172.21.15.159:0/53678123 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x563962743970 0x563962727e00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b2c2910d3c9ec37 server_cookie=1ebcf3c5135ce8c6 in_seq=0 out_seq=0 2024-04-04T06:13:59.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.950+0000 7fea56a00640 10 client.7722 ms_handle_connect on v2:172.21.15.159:6833/3796123128 2024-04-04T06:13:59.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.951+0000 7fea56a00640 1 -- 172.21.15.159:0/53678123 <== mds.0 v2:172.21.15.159:6833/3796123128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563962671c10 con 0x563962743970 2024-04-04T06:13:59.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.951+0000 7fea56a00640 10 client.7722 handle_command_reply: tid=0 2024-04-04T06:13:59.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.951+0000 7fea7af87740 1 client.7722 shutdown 2024-04-04T06:13:59.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.952+0000 7fea7af87740 1 -- 172.21.15.159:0/53678123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962726580 msgr2=0x56396271b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:59.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.952+0000 7fea7af87740 1 --2- 172.21.15.159:0/53678123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962726580 0x56396271b170 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fea64012690 tx=0x7fea64004730 comp rx=0 tx=0).stop 2024-04-04T06:13:59.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.952+0000 7fea7af87740 1 -- 172.21.15.159:0/53678123 shutdown_connections 2024-04-04T06:13:59.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.952+0000 7fea7af87740 1 --2- 172.21.15.159:0/53678123 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x563962743970 0x563962727e00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.952+0000 7fea7af87740 1 --2- 172.21.15.159:0/53678123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563962757db0 0x563962758230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.952+0000 7fea7af87740 1 --2- 172.21.15.159:0/53678123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563962726960 0x56396272a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.952+0000 7fea7af87740 1 --2- 172.21.15.159:0/53678123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962726580 0x56396271b170 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.952+0000 7fea56a00640 0 client.7722 ms_handle_reset on v2:172.21.15.159:6833/3796123128 2024-04-04T06:13:59.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.952+0000 7fea7af87740 1 -- 172.21.15.159:0/53678123 >> 172.21.15.159:0/53678123 conn(0x5639626f9a60 msgr2=0x563962724c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:59.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.952+0000 7fea7af87740 1 -- 172.21.15.159:0/53678123 shutdown_connections 2024-04-04T06:13:59.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.952+0000 7fea7af87740 1 -- 172.21.15.159:0/53678123 wait complete. 2024-04-04T06:13:59.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.952+0000 7fea7af87740 20 client.7722 trim_cache size 0 max 16384 2024-04-04T06:13:59.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.971+0000 7fea7af87740 1 Processor -- start 2024-04-04T06:13:59.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.971+0000 7fea7af87740 1 -- start start 2024-04-04T06:13:59.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.972+0000 7fea7af87740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563962726580 0x56396275e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:59.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.972+0000 7fea7af87740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563962726960 0x56396272a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:59.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.972+0000 7fea7af87740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962757db0 0x5639626f9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:59.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.972+0000 7fea7af87740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fea70200c00 con 0x563962757db0 2024-04-04T06:13:59.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.972+0000 7fea7af87740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea701ffc50 con 0x563962726960 2024-04-04T06:13:59.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.972+0000 7fea7af87740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea70160e40 con 0x563962726580 2024-04-04T06:13:59.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.972+0000 7fea76200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962757db0 0x5639626f9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:59.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.972+0000 7fea76200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962757db0 0x5639626f9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60394/0 (socket says 172.21.15.159:60394) 2024-04-04T06:13:59.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.972+0000 7fea76200640 1 -- 172.21.15.159:0/3954460793 learned_addr learned my addr 172.21.15.159:0/3954460793 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:59.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.973+0000 7fea76200640 1 -- 172.21.15.159:0/3954460793 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563962726580 msgr2=0x56396275e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:59.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.973+0000 7fea76200640 1 --2- 172.21.15.159:0/3954460793 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563962726580 0x56396275e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.973+0000 7fea76200640 1 -- 172.21.15.159:0/3954460793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563962726960 msgr2=0x56396272a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:59.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.973+0000 7fea76200640 1 --2- 172.21.15.159:0/3954460793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563962726960 0x56396272a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.973+0000 7fea76200640 1 -- 172.21.15.159:0/3954460793 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56396272c040 con 0x563962757db0 2024-04-04T06:13:59.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.973+0000 7fea76200640 1 --2- 172.21.15.159:0/3954460793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962757db0 0x5639626f9ea0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fea6800cfd0 tx=0x7fea6805a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f8e6f4ae22db75 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:59.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.973+0000 7fea61600640 1 -- 172.21.15.159:0/3954460793 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6805e430 con 0x563962757db0 2024-04-04T06:13:59.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.973+0000 7fea61600640 1 -- 172.21.15.159:0/3954460793 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea68062c60 con 0x563962757db0 2024-04-04T06:13:59.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.973+0000 7fea61600640 1 -- 172.21.15.159:0/3954460793 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea68058380 con 0x563962757db0 2024-04-04T06:13:59.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.974+0000 7fea7af87740 1 -- 172.21.15.159:0/3954460793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962757db0 msgr2=0x5639626f9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:59.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.974+0000 7fea7af87740 1 --2- 172.21.15.159:0/3954460793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962757db0 0x5639626f9ea0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fea6800cfd0 tx=0x7fea6805a780 comp rx=0 tx=0).stop 2024-04-04T06:13:59.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.974+0000 7fea7af87740 1 -- 172.21.15.159:0/3954460793 shutdown_connections 2024-04-04T06:13:59.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.974+0000 7fea7af87740 1 --2- 172.21.15.159:0/3954460793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962757db0 0x5639626f9ea0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.974+0000 7fea7af87740 1 --2- 172.21.15.159:0/3954460793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563962726960 0x56396272a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.974+0000 7fea7af87740 1 --2- 172.21.15.159:0/3954460793 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563962726580 0x56396275e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.974+0000 7fea7af87740 1 -- 172.21.15.159:0/3954460793 >> 172.21.15.159:0/3954460793 conn(0x5639626f9a60 msgr2=0x563962721230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:59.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.974+0000 7fea7af87740 1 -- 172.21.15.159:0/3954460793 shutdown_connections 2024-04-04T06:13:59.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.974+0000 7fea7af87740 1 -- 172.21.15.159:0/3954460793 wait complete. 2024-04-04T06:13:59.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.974+0000 7fea7af87740 1 -- start start 2024-04-04T06:13:59.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.975+0000 7fea7af87740 1 Processor -- start 2024-04-04T06:13:59.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.976+0000 7fea7af87740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563962726580 0x56396275e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:59.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.976+0000 7fea7af87740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563962726960 0x5639627568d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:59.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.976+0000 7fea7af87740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962757db0 0x563962756e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:59.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.976+0000 7fea7af87740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563962385210 con 0x563962757db0 2024-04-04T06:13:59.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.976+0000 7fea7af87740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea701ff680 con 0x563962726960 2024-04-04T06:13:59.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.976+0000 7fea7af87740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x563962742fb0 con 0x563962726580 2024-04-04T06:13:59.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.976+0000 7fea7af87740 10 client.0 _ll_register_callbacks cb 0x7fea6400df20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:13:59.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.976+0000 7fea76200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962757db0 0x563962756e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:59.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.976+0000 7fea76200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962757db0 0x563962756e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60408/0 (socket says 172.21.15.159:60408) 2024-04-04T06:13:59.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.976+0000 7fea76200640 1 -- 172.21.15.159:0/576195134 learned_addr learned my addr 172.21.15.159:0/576195134 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:13:59.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.976+0000 7fea75800640 1 --2- 172.21.15.159:0/576195134 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563962726580 0x56396275e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:59.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.976+0000 7fea74e00640 1 --2- 172.21.15.159:0/576195134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563962726960 0x5639627568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:59.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.976+0000 7fea76200640 1 -- 172.21.15.159:0/576195134 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563962726580 msgr2=0x56396275e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:59.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.976+0000 7fea76200640 1 --2- 172.21.15.159:0/576195134 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563962726580 0x56396275e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.976+0000 7fea76200640 1 -- 172.21.15.159:0/576195134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563962726960 msgr2=0x5639627568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:13:59.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.976+0000 7fea76200640 1 --2- 172.21.15.159:0/576195134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563962726960 0x5639627568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.976+0000 7fea76200640 1 -- 172.21.15.159:0/576195134 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639626f9e60 con 0x563962757db0 2024-04-04T06:13:59.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.977+0000 7fea76200640 1 --2- 172.21.15.159:0/576195134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962757db0 0x563962756e10 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fea6805a240 tx=0x7fea68007440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137218dc01b41931 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:13:59.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.977+0000 7fea56a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:13:59.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.977+0000 7fea56a00640 1 -- 172.21.15.159:0/576195134 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6800de50 con 0x563962757db0 2024-04-04T06:13:59.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.977+0000 7fea56a00640 1 -- 172.21.15.159:0/576195134 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5853+0+0 (secure 0 0 0) 0x7fea6805e0d0 con 0x563962757db0 2024-04-04T06:13:59.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.977+0000 7fea7af87740 1 -- 172.21.15.159:0/576195134 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea68062c60 con 0x563962757db0 2024-04-04T06:13:59.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.977+0000 7fea56a00640 1 -- 172.21.15.159:0/576195134 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fea68081050 con 0x563962757db0 2024-04-04T06:13:59.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.977+0000 7fea7af87740 10 client.7728 fetch_fsmap learned FSMap version 69 2024-04-04T06:13:59.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.977+0000 7fea7af87740 1 -- 172.21.15.159:0/576195134 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x563962744f80 con 0x563962757db0 2024-04-04T06:13:59.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.977+0000 7fea56a00640 1 -- 172.21.15.159:0/576195134 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea68062c60 con 0x563962757db0 2024-04-04T06:13:59.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.977+0000 7fea56a00640 1 -- 172.21.15.159:0/576195134 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 69) ==== 3494+0+0 (secure 0 0 0) 0x7fea6808ace0 con 0x563962757db0 2024-04-04T06:13:59.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.978+0000 7fea7af87740 10 client.7728 fetch_fsmap finished waiting for FSMap version 69 2024-04-04T06:13:59.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.978+0000 7fea7af87740 10 client.7728 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-04T06:13:59.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.978+0000 7fea7af87740 20 client.7728 populate_metadata read hostname 'smithi159' 2024-04-04T06:13:59.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.978+0000 7fea7af87740 1 --2- 172.21.15.159:0/576195134 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x5639627c9700 0x563962746ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:13:59.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.978+0000 7fea7af87740 4 client.7728 mds_command: new command op to 7512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-04T06:13:59.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.978+0000 7fea7af87740 1 -- 172.21.15.159:0/576195134 --> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56396266bf60 con 0x5639627c9700 2024-04-04T06:13:59.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.978+0000 7fea75800640 1 --2- 172.21.15.159:0/576195134 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x5639627c9700 0x563962746ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:13:59.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.978+0000 7fea75800640 1 --2- 172.21.15.159:0/576195134 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x5639627c9700 0x563962746ec0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f3c059fde3c3a98 server_cookie=dbb77d4a682ac8e6 in_seq=0 out_seq=0 2024-04-04T06:13:59.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.978+0000 7fea56a00640 10 client.7728 ms_handle_connect on v2:172.21.15.159:6833/3796123128 2024-04-04T06:13:59.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.978+0000 7fea56a00640 1 -- 172.21.15.159:0/576195134 <== mds.0 v2:172.21.15.159:6833/3796123128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56396266bf60 con 0x5639627c9700 2024-04-04T06:13:59.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.979+0000 7fea56a00640 10 client.7728 handle_command_reply: tid=0 2024-04-04T06:13:59.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.979+0000 7fea7af87740 1 client.7728 shutdown 2024-04-04T06:13:59.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.979+0000 7fea7af87740 1 -- 172.21.15.159:0/576195134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962757db0 msgr2=0x563962756e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:59.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.979+0000 7fea7af87740 1 --2- 172.21.15.159:0/576195134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962757db0 0x563962756e10 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fea6805a240 tx=0x7fea68007440 comp rx=0 tx=0).stop 2024-04-04T06:13:59.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.979+0000 7fea7af87740 1 -- 172.21.15.159:0/576195134 shutdown_connections 2024-04-04T06:13:59.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.979+0000 7fea7af87740 1 --2- 172.21.15.159:0/576195134 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x5639627c9700 0x563962746ec0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.979+0000 7fea7af87740 1 --2- 172.21.15.159:0/576195134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563962757db0 0x563962756e10 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.979+0000 7fea7af87740 1 --2- 172.21.15.159:0/576195134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563962726960 0x5639627568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.979+0000 7fea7af87740 1 --2- 172.21.15.159:0/576195134 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563962726580 0x56396275e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.979+0000 7fea56a00640 0 client.7728 ms_handle_reset on v2:172.21.15.159:6833/3796123128 2024-04-04T06:13:59.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.979+0000 7fea7af87740 1 -- 172.21.15.159:0/576195134 >> 172.21.15.159:0/576195134 conn(0x5639626f9a60 msgr2=0x563962728010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:59.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.979+0000 7fea7af87740 1 -- 172.21.15.159:0/576195134 shutdown_connections 2024-04-04T06:13:59.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.979+0000 7fea7af87740 1 -- 172.21.15.159:0/576195134 wait complete. 2024-04-04T06:13:59.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.979+0000 7fea7af87740 20 client.7728 trim_cache size 0 max 16384 2024-04-04T06:13:59.978 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:13:59.978 INFO:teuthology.orchestra.run.smithi159.stdout: "status": "no active scrubs running", 2024-04-04T06:13:59.978 INFO:teuthology.orchestra.run.smithi159.stdout: "scrubs": {} 2024-04-04T06:13:59.978 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:13:59.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.980+0000 7fea78200640 1 -- 172.21.15.159:0/544813454 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea5006c8f0 msgr2=0x7fea5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:59.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.980+0000 7fea78200640 1 --2- 172.21.15.159:0/544813454 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea5006c8f0 0x7fea5006edb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fea5c036000 tx=0x7fea5c02e000 comp rx=0 tx=0).stop 2024-04-04T06:13:59.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.980+0000 7fea78200640 1 -- 172.21.15.159:0/544813454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea70141c10 msgr2=0x7fea7009a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:13:59.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.980+0000 7fea78200640 1 --2- 172.21.15.159:0/544813454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea70141c10 0x7fea7009a6d0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fea6800cf20 tx=0x7fea6800a710 comp rx=0 tx=0).stop 2024-04-04T06:13:59.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.980+0000 7fea78200640 1 -- 172.21.15.159:0/544813454 shutdown_connections 2024-04-04T06:13:59.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.980+0000 7fea78200640 1 --2- 172.21.15.159:0/544813454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea70141c10 0x7fea7009a6d0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.980+0000 7fea78200640 1 --2- 172.21.15.159:0/544813454 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea700d0280 0x7fea7009a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.980+0000 7fea78200640 1 --2- 172.21.15.159:0/544813454 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea5006c8f0 0x7fea5006edb0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.980+0000 7fea78200640 1 --2- 172.21.15.159:0/544813454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea70071eb0 0x7fea7009bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:13:59.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.980+0000 7fea78200640 1 -- 172.21.15.159:0/544813454 >> 172.21.15.159:0/544813454 conn(0x7fea7006b750 msgr2=0x7fea700b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:13:59.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.980+0000 7fea78200640 1 -- 172.21.15.159:0/544813454 shutdown_connections 2024-04-04T06:13:59.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:13:59.980+0000 7fea78200640 1 -- 172.21.15.159:0/544813454 wait complete. 2024-04-04T06:13:59.991 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-04T06:13:59.992 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:14:00.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.192+0000 7f24df800640 1 Processor -- start 2024-04-04T06:14:00.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.193+0000 7f24df800640 1 -- start start 2024-04-04T06:14:00.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.195+0000 7f24df800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f24d80b4530 0x7f24d80c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.195+0000 7f24df800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24d815eae0 con 0x7f24d815e700 2024-04-04T06:14:00.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.195+0000 7f24df800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24d80cfeb0 con 0x7f24d80d1610 2024-04-04T06:14:00.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.195+0000 7f24df800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f24d80d0190 con 0x7f24d80b4530 2024-04-04T06:14:00.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.196+0000 7f24dce00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f24d815e700 0x7f24d80b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:48314/0 (socket says 172.21.15.159:48314) 2024-04-04T06:14:00.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.196+0000 7f24dce00640 1 -- 172.21.15.159:0/2996552384 learned_addr learned my addr 172.21.15.159:0/2996552384 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:00.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.196+0000 7f24cfe00640 1 --2- 172.21.15.159:0/2996552384 >> v2:172.21.15.178:3300/0 conn(0x7f24d80b4530 0x7f24d80c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.196+0000 7f24d7e00640 1 -- 172.21.15.159:0/2996552384 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3323604894 0 0) 0x7f24d815eae0 con 0x7f24d815e700 2024-04-04T06:14:00.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.196+0000 7f24d7e00640 1 -- 172.21.15.159:0/2996552384 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24b8003520 con 0x7f24d815e700 2024-04-04T06:14:00.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.197+0000 7f24d7e00640 1 -- 172.21.15.159:0/2996552384 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1480000009 0 0) 0x7f24b8003520 con 0x7f24d815e700 2024-04-04T06:14:00.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.197+0000 7f24d7e00640 1 -- 172.21.15.159:0/2996552384 >> v2:172.21.15.178:3300/0 conn(0x7f24d80b4530 msgr2=0x7f24d80c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:00.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.197+0000 7f24d7e00640 1 --2- 172.21.15.159:0/2996552384 >> v2:172.21.15.178:3300/0 conn(0x7f24d80b4530 0x7f24d80c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.197+0000 7f24d7e00640 1 -- 172.21.15.159:0/2996552384 >> v1:172.21.15.178:6789/0 conn(0x7f24d80d1610 legacy=0x7f24d80c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:00.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.197+0000 7f24d7e00640 1 -- 172.21.15.159:0/2996552384 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d80d0470 con 0x7f24d815e700 2024-04-04T06:14:00.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.197+0000 7f24d7e00640 1 -- 172.21.15.159:0/2996552384 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f24c0001670 con 0x7f24d815e700 2024-04-04T06:14:00.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.197+0000 7f24d7e00640 1 -- 172.21.15.159:0/2996552384 >> v1:172.21.15.159:6789/0 conn(0x7f24d815e700 legacy=0x7f24d80b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:00.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.197+0000 7f24d7e00640 1 --2- 172.21.15.159:0/2996552384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24b8003a00 0x7f24b8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.197+0000 7f24d7e00640 1 --2- 172.21.15.159:0/2996552384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24b8004800 0x7f24b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.197+0000 7f24d7e00640 1 --2- 172.21.15.159:0/2996552384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24b8005210 0x7f24b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.197+0000 7f24d7e00640 1 -- 172.21.15.159:0/2996552384 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f24d80d0190 con 0x7f24b8004800 2024-04-04T06:14:00.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.197+0000 7f24d7e00640 1 -- 172.21.15.159:0/2996552384 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f24b800c160 con 0x7f24b8005210 2024-04-04T06:14:00.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.197+0000 7f24d7e00640 1 -- 172.21.15.159:0/2996552384 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f24b800c360 con 0x7f24b8003a00 2024-04-04T06:14:00.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.197+0000 7f24cfe00640 1 --2- 172.21.15.159:0/2996552384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24b8004800 0x7f24b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.197+0000 7f24dce00640 1 --2- 172.21.15.159:0/2996552384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24b8003a00 0x7f24b8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.197+0000 7f24cfe00640 1 -- 172.21.15.159:0/2996552384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24b8003a00 msgr2=0x7f24b8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:00.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.197+0000 7f24dd800640 1 --2- 172.21.15.159:0/2996552384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24b8005210 0x7f24b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.198+0000 7f24cfe00640 1 --2- 172.21.15.159:0/2996552384 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24b8003a00 0x7f24b8003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.198+0000 7f24cfe00640 1 -- 172.21.15.159:0/2996552384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24b8005210 msgr2=0x7f24b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:00.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.198+0000 7f24cfe00640 1 --2- 172.21.15.159:0/2996552384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24b8005210 0x7f24b800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.198+0000 7f24cfe00640 1 -- 172.21.15.159:0/2996552384 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b8003520 con 0x7f24b8004800 2024-04-04T06:14:00.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.198+0000 7f24dd800640 1 -- 172.21.15.159:0/2996552384 reap_dead start 2024-04-04T06:14:00.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.199+0000 7f24cfe00640 1 --2- 172.21.15.159:0/2996552384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24b8004800 0x7f24b8004c80 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009a90 tx=0x7f24b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6a202caf25685e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:00.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.199+0000 7f24d7e00640 1 -- 172.21.15.159:0/2996552384 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8039070 con 0x7f24b8004800 2024-04-04T06:14:00.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.199+0000 7f24d7e00640 1 -- 172.21.15.159:0/2996552384 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c802c680 con 0x7f24b8004800 2024-04-04T06:14:00.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.199+0000 7f24d7e00640 1 -- 172.21.15.159:0/2996552384 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8009690 con 0x7f24b8004800 2024-04-04T06:14:00.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.199+0000 7f24df800640 1 -- 172.21.15.159:0/2996552384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24b8004800 msgr2=0x7f24b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:00.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.199+0000 7f24df800640 1 --2- 172.21.15.159:0/2996552384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24b8004800 0x7f24b8004c80 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009a90 tx=0x7f24b8005830 comp rx=0 tx=0).stop 2024-04-04T06:14:00.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.200+0000 7f24df800640 1 -- 172.21.15.159:0/2996552384 shutdown_connections 2024-04-04T06:14:00.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.200+0000 7f24df800640 1 --2- 172.21.15.159:0/2996552384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24b8004800 0x7f24b8004c80 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.200+0000 7f24df800640 1 -- 172.21.15.159:0/2996552384 >> 172.21.15.159:0/2996552384 conn(0x7f24d80d2240 msgr2=0x7f24d80d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:00.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.200+0000 7f24df800640 1 -- 172.21.15.159:0/2996552384 shutdown_connections 2024-04-04T06:14:00.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.200+0000 7f24df800640 1 -- 172.21.15.159:0/2996552384 wait complete. 2024-04-04T06:14:00.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.201+0000 7f24df800640 1 Processor -- start 2024-04-04T06:14:00.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.202+0000 7f24df800640 1 -- start start 2024-04-04T06:14:00.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.202+0000 7f24df800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24b8004800 0x7f24d80ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.203+0000 7f24df800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24d814c520 0x7f24d814bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.203+0000 7f24df800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24d80cbc40 0x7f24d80c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.203+0000 7f24df800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f24d806d1b0 con 0x7f24d814c520 2024-04-04T06:14:00.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.203+0000 7f24df800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f24d80d0a30 con 0x7f24d80cbc40 2024-04-04T06:14:00.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.203+0000 7f24df800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f24d806d330 con 0x7f24b8004800 2024-04-04T06:14:00.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.203+0000 7f24cfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24d814c520 0x7f24d814bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.203+0000 7f24cfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24d814c520 0x7f24d814bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60426/0 (socket says 172.21.15.159:60426) 2024-04-04T06:14:00.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.203+0000 7f24cfe00640 1 -- 172.21.15.159:0/542314994 learned_addr learned my addr 172.21.15.159:0/542314994 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:00.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.203+0000 7f24dce00640 1 --2- 172.21.15.159:0/542314994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24b8004800 0x7f24d80ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.203+0000 7f24dd800640 1 --2- 172.21.15.159:0/542314994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24d80cbc40 0x7f24d80c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.203+0000 7f24cfe00640 1 -- 172.21.15.159:0/542314994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24b8004800 msgr2=0x7f24d80ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:00.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.203+0000 7f24cfe00640 1 --2- 172.21.15.159:0/542314994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24b8004800 0x7f24d80ca380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.203+0000 7f24cfe00640 1 -- 172.21.15.159:0/542314994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24d80cbc40 msgr2=0x7f24d80c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:00.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.203+0000 7f24cfe00640 1 --2- 172.21.15.159:0/542314994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24d80cbc40 0x7f24d80c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.203+0000 7f24cfe00640 1 -- 172.21.15.159:0/542314994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d814c240 con 0x7f24d814c520 2024-04-04T06:14:00.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.203+0000 7f24dce00640 1 --2- 172.21.15.159:0/542314994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24b8004800 0x7f24d80ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:00.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.203+0000 7f24dd800640 1 --2- 172.21.15.159:0/542314994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24d80cbc40 0x7f24d80c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:00.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.204+0000 7f24cfe00640 1 --2- 172.21.15.159:0/542314994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24d814c520 0x7f24d814bd00 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f24c8037040 tx=0x7f24c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:00.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.204+0000 7f24d5600640 1 -- 172.21.15.159:0/542314994 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8039050 con 0x7f24d814c520 2024-04-04T06:14:00.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.204+0000 7f24d5600640 1 -- 172.21.15.159:0/542314994 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c8045070 con 0x7f24d814c520 2024-04-04T06:14:00.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.204+0000 7f24d5600640 1 -- 172.21.15.159:0/542314994 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80044e0 con 0x7f24d814c520 2024-04-04T06:14:00.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.204+0000 7f24df800640 1 -- 172.21.15.159:0/542314994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d80cc570 con 0x7f24d814c520 2024-04-04T06:14:00.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.204+0000 7f24df800640 1 -- 172.21.15.159:0/542314994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d80ccb30 con 0x7f24d814c520 2024-04-04T06:14:00.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.205+0000 7f24df800640 1 -- 172.21.15.159:0/542314994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24d80685c0 con 0x7f24d814c520 2024-04-04T06:14:00.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.206+0000 7f24d5600640 1 -- 172.21.15.159:0/542314994 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24c8003a50 con 0x7f24d814c520 2024-04-04T06:14:00.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.207+0000 7f24d5600640 1 --2- 172.21.15.159:0/542314994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f24b406c8f0 0x7f24b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.208+0000 7f24d5600640 1 -- 172.21.15.159:0/542314994 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5853+0+0 (secure 0 0 0) 0x7f24c80ad720 con 0x7f24d814c520 2024-04-04T06:14:00.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.208+0000 7f24dce00640 1 --2- 172.21.15.159:0/542314994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f24b406c8f0 0x7f24b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.209+0000 7f24dce00640 1 --2- 172.21.15.159:0/542314994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f24b406c8f0 0x7f24b406edb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f24c0000c00 tx=0x7f24c0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:00.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.213+0000 7f24d5600640 1 -- 172.21.15.159:0/542314994 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f24c804e020 con 0x7f24d814c520 2024-04-04T06:14:00.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.486+0000 7f24df800640 1 -- 172.21.15.159:0/542314994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24d80008d0 con 0x7f24d814c520 2024-04-04T06:14:00.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.487+0000 7f24d5600640 1 -- 172.21.15.159:0/542314994 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5196 (secure 0 0 0) 0x7f24d814c240 con 0x7f24d814c520 2024-04-04T06:14:00.481 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:14:00.481 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1825651864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1825651864},{"type":"v1","addr":"172.21.15.178:6836","nonce":1825651864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1824578712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1824578712},{"type":"v1","addr":"172.21.15.178:6837","nonce":1824578712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6834/3377498298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3377498298},{"type":"v1","addr":"172.21.15.159:6834","nonce":3377498298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:12:35.880454+0000","modified":"2024-04-04T06:13:00.228456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7512":{"gid":7512,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":6,"addr":"172.21.15.159:6835/3796123128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":3796123128},{"type":"v1","addr":"172.21.15.159:6835","nonce":3796123128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7512,"qdb_cluster":[7512]},"id":6}]} 2024-04-04T06:14:00.482 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 69 2024-04-04T06:14:00.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.489+0000 7f24df800640 1 -- 172.21.15.159:0/542314994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f24b406c8f0 msgr2=0x7f24b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:00.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.489+0000 7f24df800640 1 --2- 172.21.15.159:0/542314994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f24b406c8f0 0x7f24b406edb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f24c0000c00 tx=0x7f24c0009040 comp rx=0 tx=0).stop 2024-04-04T06:14:00.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.489+0000 7f24df800640 1 -- 172.21.15.159:0/542314994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24d814c520 msgr2=0x7f24d814bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:00.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.489+0000 7f24df800640 1 --2- 172.21.15.159:0/542314994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24d814c520 0x7f24d814bd00 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f24c8037040 tx=0x7f24c8004730 comp rx=0 tx=0).stop 2024-04-04T06:14:00.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.490+0000 7f24df800640 1 -- 172.21.15.159:0/542314994 shutdown_connections 2024-04-04T06:14:00.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.490+0000 7f24df800640 1 --2- 172.21.15.159:0/542314994 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f24b406c8f0 0x7f24b406edb0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.490+0000 7f24df800640 1 --2- 172.21.15.159:0/542314994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24d80cbc40 0x7f24d80c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.490+0000 7f24df800640 1 --2- 172.21.15.159:0/542314994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24d814c520 0x7f24d814bd00 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.490+0000 7f24df800640 1 --2- 172.21.15.159:0/542314994 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24b8004800 0x7f24d80ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.490+0000 7f24df800640 1 -- 172.21.15.159:0/542314994 >> 172.21.15.159:0/542314994 conn(0x7f24d80d2240 msgr2=0x7f24d815ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:00.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.490+0000 7f24df800640 1 -- 172.21.15.159:0/542314994 shutdown_connections 2024-04-04T06:14:00.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.490+0000 7f24df800640 1 -- 172.21.15.159:0/542314994 wait complete. 2024-04-04T06:14:00.503 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-04T06:14:00.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.749+0000 7f9aae000640 1 Processor -- start 2024-04-04T06:14:00.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.749+0000 7f9aae000640 1 -- start start 2024-04-04T06:14:00.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.751+0000 7f9aae000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9aa80b4890 0x7f9aa8072570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.751+0000 7f9aae000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aa806d620 0x7f9aa8072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.751+0000 7f9aae000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aa80b4c70 con 0x7f9aa815f6a0 2024-04-04T06:14:00.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.751+0000 7f9aae000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f9aa80b3eb0 con 0x7f9aa80b4890 2024-04-04T06:14:00.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.751+0000 7f9aae000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa815fb30 con 0x7f9aa806d620 2024-04-04T06:14:00.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.751+0000 7f9aacc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9aa80b4890 0x7f9aa8072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.751+0000 7f9aacc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9aa80b4890 0x7f9aa8072570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60438/0 (socket says 172.21.15.159:60438) 2024-04-04T06:14:00.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.751+0000 7f9aa7e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f9aa815f6a0 0x7f9aa80713e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:48330/0 (socket says 172.21.15.159:48330) 2024-04-04T06:14:00.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.751+0000 7f9aacc00640 1 -- 172.21.15.159:0/4148216076 learned_addr learned my addr 172.21.15.159:0/4148216076 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:00.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.752+0000 7f9aad600640 1 --2- 172.21.15.159:0/4148216076 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aa806d620 0x7f9aa8072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.752+0000 7f9aacc00640 1 -- 172.21.15.159:0/4148216076 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aa806d620 msgr2=0x7f9aa8072ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:00.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.752+0000 7f9aacc00640 1 --2- 172.21.15.159:0/4148216076 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aa806d620 0x7f9aa8072ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.752+0000 7f9aacc00640 1 -- 172.21.15.159:0/4148216076 >> v1:172.21.15.159:6789/0 conn(0x7f9aa815f6a0 legacy=0x7f9aa80713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:00.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.752+0000 7f9aacc00640 1 -- 172.21.15.159:0/4148216076 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa815b7b0 con 0x7f9aa80b4890 2024-04-04T06:14:00.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.752+0000 7f9aad600640 1 --2- 172.21.15.159:0/4148216076 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aa806d620 0x7f9aa8072ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:00.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.753+0000 7f9aacc00640 1 --2- 172.21.15.159:0/4148216076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa80b4890 0x7f9aa8072570 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009870 tx=0x7f9a9c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a820b62d56b60df server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:00.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.753+0000 7f9aa7400640 1 -- 172.21.15.159:0/4148216076 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c034070 con 0x7f9aa80b4890 2024-04-04T06:14:00.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.753+0000 7f9aa7400640 1 -- 172.21.15.159:0/4148216076 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a9c030840 con 0x7f9aa80b4890 2024-04-04T06:14:00.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.753+0000 7f9aa7400640 1 -- 172.21.15.159:0/4148216076 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c030c10 con 0x7f9aa80b4890 2024-04-04T06:14:00.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.754+0000 7f9aae000640 1 -- 172.21.15.159:0/4148216076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa80b4890 msgr2=0x7f9aa8072570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:00.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.754+0000 7f9aae000640 1 --2- 172.21.15.159:0/4148216076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa80b4890 0x7f9aa8072570 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009870 tx=0x7f9a9c02b980 comp rx=0 tx=0).stop 2024-04-04T06:14:00.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.754+0000 7f9aae000640 1 -- 172.21.15.159:0/4148216076 shutdown_connections 2024-04-04T06:14:00.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.754+0000 7f9aae000640 1 --2- 172.21.15.159:0/4148216076 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aa806d620 0x7f9aa8072ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.754+0000 7f9aae000640 1 --2- 172.21.15.159:0/4148216076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa80b4890 0x7f9aa8072570 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.754+0000 7f9aae000640 1 -- 172.21.15.159:0/4148216076 >> 172.21.15.159:0/4148216076 conn(0x7f9aa80b1ba0 msgr2=0x7f9aa80b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:00.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.754+0000 7f9aae000640 1 -- 172.21.15.159:0/4148216076 shutdown_connections 2024-04-04T06:14:00.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.754+0000 7f9aae000640 1 -- 172.21.15.159:0/4148216076 wait complete. 2024-04-04T06:14:00.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.756+0000 7f9aae000640 1 Processor -- start 2024-04-04T06:14:00.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.756+0000 7f9aae000640 1 -- start start 2024-04-04T06:14:00.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.757+0000 7f9aae000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9aa806d620 0x7f9aa814e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.757+0000 7f9aae000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aa80b4890 0x7f9aa814b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.757+0000 7f9aae000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa814cd90 0x7f9aa81f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.757+0000 7f9aae000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa815c050 con 0x7f9aa814cd90 2024-04-04T06:14:00.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.757+0000 7f9aae000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa815bd70 con 0x7f9aa806d620 2024-04-04T06:14:00.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.757+0000 7f9aae000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa815c570 con 0x7f9aa80b4890 2024-04-04T06:14:00.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.757+0000 7f9aad600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa814cd90 0x7f9aa81f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.757+0000 7f9aad600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa814cd90 0x7f9aa81f35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60446/0 (socket says 172.21.15.159:60446) 2024-04-04T06:14:00.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.757+0000 7f9aad600640 1 -- 172.21.15.159:0/3330556626 learned_addr learned my addr 172.21.15.159:0/3330556626 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:00.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.757+0000 7f9aacc00640 1 --2- 172.21.15.159:0/3330556626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9aa806d620 0x7f9aa814e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.757+0000 7f9aa7e00640 1 --2- 172.21.15.159:0/3330556626 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aa80b4890 0x7f9aa814b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.758+0000 7f9aad600640 1 -- 172.21.15.159:0/3330556626 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aa80b4890 msgr2=0x7f9aa814b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:00.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.758+0000 7f9aad600640 1 --2- 172.21.15.159:0/3330556626 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aa80b4890 0x7f9aa814b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.758+0000 7f9aad600640 1 -- 172.21.15.159:0/3330556626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9aa806d620 msgr2=0x7f9aa814e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:00.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.758+0000 7f9aad600640 1 --2- 172.21.15.159:0/3330556626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9aa806d620 0x7f9aa814e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.758+0000 7f9aad600640 1 -- 172.21.15.159:0/3330556626 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa80b18b0 con 0x7f9aa814cd90 2024-04-04T06:14:00.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.758+0000 7f9aa7e00640 1 --2- 172.21.15.159:0/3330556626 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aa80b4890 0x7f9aa814b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:14:00.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.758+0000 7f9aad600640 1 --2- 172.21.15.159:0/3330556626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa814cd90 0x7f9aa81f35a0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000cd50 tx=0x7f9aa000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:00.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.758+0000 7f9aa4c00640 1 -- 172.21.15.159:0/3330556626 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa000f680 con 0x7f9aa814cd90 2024-04-04T06:14:00.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.759+0000 7f9aa4c00640 1 -- 172.21.15.159:0/3330556626 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa000f840 con 0x7f9aa814cd90 2024-04-04T06:14:00.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.759+0000 7f9aae000640 1 -- 172.21.15.159:0/3330556626 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa81f25d0 con 0x7f9aa814cd90 2024-04-04T06:14:00.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.759+0000 7f9aa4c00640 1 -- 172.21.15.159:0/3330556626 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa000fc30 con 0x7f9aa814cd90 2024-04-04T06:14:00.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.759+0000 7f9aae000640 1 -- 172.21.15.159:0/3330556626 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa814f200 con 0x7f9aa814cd90 2024-04-04T06:14:00.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.761+0000 7f9aa4c00640 1 -- 172.21.15.159:0/3330556626 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9aa00213f0 con 0x7f9aa814cd90 2024-04-04T06:14:00.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.762+0000 7f9aa4c00640 1 --2- 172.21.15.159:0/3330556626 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9a8806c8f0 0x7f9a8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.762+0000 7f9aa4c00640 1 -- 172.21.15.159:0/3330556626 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5853+0+0 (secure 0 0 0) 0x7f9aa008c9a0 con 0x7f9aa814cd90 2024-04-04T06:14:00.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.763+0000 7f9ab23e2740 1 Processor -- start 2024-04-04T06:14:00.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.763+0000 7f9aacc00640 1 --2- 172.21.15.159:0/3330556626 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9a8806c8f0 0x7f9a8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.763+0000 7f9ab23e2740 1 -- start start 2024-04-04T06:14:00.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.764+0000 7f9ab23e2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aba1e2c8c0 0x55aba1e2cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.764+0000 7f9ab23e2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55aba1dfc8e0 0x55aba1e29b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.764+0000 7f9ab23e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e00440 0x55aba1e2a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.764+0000 7f9ab23e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55aba1d665c0 con 0x55aba1e00440 2024-04-04T06:14:00.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.764+0000 7f9ab23e2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55aba1d0e3b0 con 0x55aba1e2c8c0 2024-04-04T06:14:00.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.764+0000 7f9ab23e2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55aba1d62570 con 0x55aba1dfc8e0 2024-04-04T06:14:00.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.764+0000 7f9aacc00640 1 --2- 172.21.15.159:0/3330556626 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9a8806c8f0 0x7f9a8806edb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c036000 tx=0x7f9a9c03b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:00.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.764+0000 7f9aad600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e00440 0x55aba1e2a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.764+0000 7f9aad600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e00440 0x55aba1e2a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60456/0 (socket says 172.21.15.159:60456) 2024-04-04T06:14:00.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.764+0000 7f9aad600640 1 -- 172.21.15.159:0/493703686 learned_addr learned my addr 172.21.15.159:0/493703686 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:00.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.764+0000 7f9aacc00640 1 --2- 172.21.15.159:0/493703686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aba1e2c8c0 0x55aba1e2cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.764+0000 7f9aad600640 1 -- 172.21.15.159:0/493703686 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55aba1dfc8e0 msgr2=0x55aba1e29b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:00.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.764+0000 7f9aad600640 1 --2- 172.21.15.159:0/493703686 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55aba1dfc8e0 0x55aba1e29b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.764+0000 7f9aad600640 1 -- 172.21.15.159:0/493703686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aba1e2c8c0 msgr2=0x55aba1e2cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:00.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.764+0000 7f9aad600640 1 --2- 172.21.15.159:0/493703686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aba1e2c8c0 0x55aba1e2cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.764+0000 7f9aad600640 1 -- 172.21.15.159:0/493703686 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aba1b467c0 con 0x55aba1e00440 2024-04-04T06:14:00.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.765+0000 7f9aad600640 1 --2- 172.21.15.159:0/493703686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e00440 0x55aba1e2a070 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0056a00 tx=0x7f9aa005a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba211e5ec034313 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:00.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.765+0000 7f9a99200640 1 -- 172.21.15.159:0/493703686 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0058640 con 0x55aba1e00440 2024-04-04T06:14:00.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.765+0000 7f9a99200640 1 -- 172.21.15.159:0/493703686 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa00588a0 con 0x55aba1e00440 2024-04-04T06:14:00.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.765+0000 7f9a99200640 1 -- 172.21.15.159:0/493703686 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0062cf0 con 0x55aba1e00440 2024-04-04T06:14:00.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.766+0000 7f9ab23e2740 1 -- 172.21.15.159:0/493703686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e00440 msgr2=0x55aba1e2a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:00.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.766+0000 7f9ab23e2740 1 --2- 172.21.15.159:0/493703686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e00440 0x55aba1e2a070 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0056a00 tx=0x7f9aa005a660 comp rx=0 tx=0).stop 2024-04-04T06:14:00.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.766+0000 7f9ab23e2740 1 -- 172.21.15.159:0/493703686 shutdown_connections 2024-04-04T06:14:00.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.766+0000 7f9ab23e2740 1 --2- 172.21.15.159:0/493703686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e00440 0x55aba1e2a070 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.766+0000 7f9ab23e2740 1 --2- 172.21.15.159:0/493703686 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55aba1dfc8e0 0x55aba1e29b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.766+0000 7f9ab23e2740 1 --2- 172.21.15.159:0/493703686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aba1e2c8c0 0x55aba1e2cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.766+0000 7f9ab23e2740 1 -- 172.21.15.159:0/493703686 >> 172.21.15.159:0/493703686 conn(0x55aba1dfeb50 msgr2=0x55aba1e476d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:00.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.766+0000 7f9ab23e2740 1 -- 172.21.15.159:0/493703686 shutdown_connections 2024-04-04T06:14:00.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.766+0000 7f9ab23e2740 1 -- 172.21.15.159:0/493703686 wait complete. 2024-04-04T06:14:00.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.767+0000 7f9ab23e2740 1 -- start start 2024-04-04T06:14:00.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.767+0000 7f9ab23e2740 1 Processor -- start 2024-04-04T06:14:00.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.769+0000 7f9ab23e2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aba1e00440 0x55aba1e21b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.769+0000 7f9ab23e2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55aba1e2c8c0 0x55aba1e2ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.769+0000 7f9ab23e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e22040 0x55aba1e2b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.769+0000 7f9ab23e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55aba1e015f0 con 0x55aba1e22040 2024-04-04T06:14:00.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.769+0000 7f9ab23e2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55aba1d217a0 con 0x55aba1e00440 2024-04-04T06:14:00.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.769+0000 7f9ab23e2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55aba1e009e0 con 0x55aba1e2c8c0 2024-04-04T06:14:00.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.769+0000 7f9ab23e2740 10 client.0 _ll_register_callbacks cb 0x55aba1d5dfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:14:00.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.769+0000 7f9aad600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e22040 0x55aba1e2b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.769+0000 7f9aad600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e22040 0x55aba1e2b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60458/0 (socket says 172.21.15.159:60458) 2024-04-04T06:14:00.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.769+0000 7f9aad600640 1 -- 172.21.15.159:0/419346837 learned_addr learned my addr 172.21.15.159:0/419346837 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:00.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.770+0000 7f9aacc00640 1 --2- 172.21.15.159:0/419346837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aba1e00440 0x55aba1e21b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.770+0000 7f9aad600640 1 -- 172.21.15.159:0/419346837 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55aba1e2c8c0 msgr2=0x55aba1e2ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:00.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.770+0000 7f9aad600640 1 --2- 172.21.15.159:0/419346837 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55aba1e2c8c0 0x55aba1e2ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.770+0000 7f9aad600640 1 -- 172.21.15.159:0/419346837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aba1e00440 msgr2=0x55aba1e21b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:00.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.770+0000 7f9aad600640 1 --2- 172.21.15.159:0/419346837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aba1e00440 0x55aba1e21b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.770+0000 7f9aad600640 1 -- 172.21.15.159:0/419346837 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aba1ba6f50 con 0x55aba1e22040 2024-04-04T06:14:00.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.770+0000 7f9aacc00640 1 --2- 172.21.15.159:0/419346837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aba1e00440 0x55aba1e21b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:00.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.770+0000 7f9aad600640 1 --2- 172.21.15.159:0/419346837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e22040 0x55aba1e2b3a0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0062c50 tx=0x7f9aa000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102bc75313a7693b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:00.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.770+0000 7f9a8d600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:14:00.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.770+0000 7f9a8d600640 1 -- 172.21.15.159:0/419346837 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa00599f0 con 0x55aba1e22040 2024-04-04T06:14:00.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.770+0000 7f9a8d600640 1 -- 172.21.15.159:0/419346837 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5853+0+0 (secure 0 0 0) 0x7f9aa008a9a0 con 0x55aba1e22040 2024-04-04T06:14:00.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.770+0000 7f9ab23e2740 1 -- 172.21.15.159:0/419346837 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aa00588a0 con 0x55aba1e22040 2024-04-04T06:14:00.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.771+0000 7f9a8d600640 1 -- 172.21.15.159:0/419346837 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f9aa0077050 con 0x55aba1e22040 2024-04-04T06:14:00.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.771+0000 7f9ab23e2740 10 client.7749 fetch_fsmap learned FSMap version 69 2024-04-04T06:14:00.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.771+0000 7f9ab23e2740 1 -- 172.21.15.159:0/419346837 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x55aba1a3a0b0 con 0x55aba1e22040 2024-04-04T06:14:00.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.771+0000 7f9a8d600640 1 -- 172.21.15.159:0/419346837 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa00588a0 con 0x55aba1e22040 2024-04-04T06:14:00.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.771+0000 7f9a8d600640 1 -- 172.21.15.159:0/419346837 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 69) ==== 3494+0+0 (secure 0 0 0) 0x7f9aa0059b90 con 0x55aba1e22040 2024-04-04T06:14:00.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.771+0000 7f9ab23e2740 10 client.7749 fetch_fsmap finished waiting for FSMap version 69 2024-04-04T06:14:00.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.771+0000 7f9ab23e2740 10 client.7749 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:14:00.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.771+0000 7f9ab23e2740 20 client.7749 populate_metadata read hostname 'smithi159' 2024-04-04T06:14:00.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.771+0000 7f9ab23e2740 1 --2- 172.21.15.159:0/419346837 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55aba1e63600 0x55aba1e659f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.771+0000 7f9ab23e2740 4 client.7749 mds_command: new command op to 7512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:14:00.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.771+0000 7f9ab23e2740 1 -- 172.21.15.159:0/419346837 --> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aba1d3a890 con 0x55aba1e63600 2024-04-04T06:14:00.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.771+0000 7f9aacc00640 1 --2- 172.21.15.159:0/419346837 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55aba1e63600 0x55aba1e659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.772+0000 7f9a8d600640 10 client.7749 ms_handle_connect on v2:172.21.15.159:6833/3796123128 2024-04-04T06:14:00.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.772+0000 7f9aacc00640 1 --2- 172.21.15.159:0/419346837 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55aba1e63600 0x55aba1e659f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4544598dff96dab2 server_cookie=cd07c8a93ff1dea6 in_seq=0 out_seq=0 2024-04-04T06:14:00.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.773+0000 7f9a8d600640 1 -- 172.21.15.159:0/419346837 <== mds.0 v2:172.21.15.159:6833/3796123128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aba1d3a890 con 0x55aba1e63600 2024-04-04T06:14:00.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.773+0000 7f9a8d600640 10 client.7749 handle_command_reply: tid=0 2024-04-04T06:14:00.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.773+0000 7f9ab23e2740 1 client.7749 shutdown 2024-04-04T06:14:00.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.773+0000 7f9ab23e2740 1 -- 172.21.15.159:0/419346837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e22040 msgr2=0x55aba1e2b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:00.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.773+0000 7f9ab23e2740 1 --2- 172.21.15.159:0/419346837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e22040 0x55aba1e2b3a0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0062c50 tx=0x7f9aa000cf60 comp rx=0 tx=0).stop 2024-04-04T06:14:00.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.773+0000 7f9ab23e2740 1 -- 172.21.15.159:0/419346837 shutdown_connections 2024-04-04T06:14:00.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.773+0000 7f9ab23e2740 1 --2- 172.21.15.159:0/419346837 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55aba1e63600 0x55aba1e659f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.773+0000 7f9ab23e2740 1 --2- 172.21.15.159:0/419346837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e22040 0x55aba1e2b3a0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.773+0000 7f9ab23e2740 1 --2- 172.21.15.159:0/419346837 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55aba1e2c8c0 0x55aba1e2ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.773+0000 7f9ab23e2740 1 --2- 172.21.15.159:0/419346837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aba1e00440 0x55aba1e21b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.773+0000 7f9ab23e2740 1 -- 172.21.15.159:0/419346837 >> 172.21.15.159:0/419346837 conn(0x55aba1dfc8e0 msgr2=0x55aba1e48be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:00.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.773+0000 7f9a8d600640 0 client.7749 ms_handle_reset on v2:172.21.15.159:6833/3796123128 2024-04-04T06:14:00.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.773+0000 7f9ab23e2740 1 -- 172.21.15.159:0/419346837 shutdown_connections 2024-04-04T06:14:00.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.773+0000 7f9ab23e2740 1 -- 172.21.15.159:0/419346837 wait complete. 2024-04-04T06:14:00.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.774+0000 7f9ab23e2740 20 client.7749 trim_cache size 0 max 16384 2024-04-04T06:14:00.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.788+0000 7f9ab23e2740 1 Processor -- start 2024-04-04T06:14:00.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.788+0000 7f9ab23e2740 1 -- start start 2024-04-04T06:14:00.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.789+0000 7f9ab23e2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aba1e00440 0x55aba1e576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.789+0000 7f9ab23e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e2c8c0 0x55aba1e48960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.789+0000 7f9ab23e2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55aba1e21c20 0x55aba1e48ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.789+0000 7f9ab23e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa815b0d0 con 0x55aba1e2c8c0 2024-04-04T06:14:00.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.789+0000 7f9ab23e2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa815a430 con 0x55aba1e00440 2024-04-04T06:14:00.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.789+0000 7f9ab23e2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa815aa80 con 0x55aba1e21c20 2024-04-04T06:14:00.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.789+0000 7f9aa7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e2c8c0 0x55aba1e48960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.789+0000 7f9aa7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e2c8c0 0x55aba1e48960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60470/0 (socket says 172.21.15.159:60470) 2024-04-04T06:14:00.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.789+0000 7f9aa7e00640 1 -- 172.21.15.159:0/128814438 learned_addr learned my addr 172.21.15.159:0/128814438 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:00.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.789+0000 7f9aacc00640 1 --2- 172.21.15.159:0/128814438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aba1e00440 0x55aba1e576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.789+0000 7f9aa7e00640 1 -- 172.21.15.159:0/128814438 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55aba1e21c20 msgr2=0x55aba1e48ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:00.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.789+0000 7f9aa7e00640 1 --2- 172.21.15.159:0/128814438 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55aba1e21c20 0x55aba1e48ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.789+0000 7f9aa7e00640 1 -- 172.21.15.159:0/128814438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aba1e00440 msgr2=0x55aba1e576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:00.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.790+0000 7f9aa7e00640 1 --2- 172.21.15.159:0/128814438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aba1e00440 0x55aba1e576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.790+0000 7f9aa7e00640 1 -- 172.21.15.159:0/128814438 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aba1dfe3f0 con 0x55aba1e2c8c0 2024-04-04T06:14:00.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.790+0000 7f9aa7e00640 1 --2- 172.21.15.159:0/128814438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e2c8c0 0x55aba1e48960 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f9a94005bc0 tx=0x7f9a94007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26bdb2fac56c96f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:00.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.790+0000 7f9a8fe00640 1 -- 172.21.15.159:0/128814438 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a94019070 con 0x55aba1e2c8c0 2024-04-04T06:14:00.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.790+0000 7f9a8fe00640 1 -- 172.21.15.159:0/128814438 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a94005cb0 con 0x55aba1e2c8c0 2024-04-04T06:14:00.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.790+0000 7f9a8fe00640 1 -- 172.21.15.159:0/128814438 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a94005e60 con 0x55aba1e2c8c0 2024-04-04T06:14:00.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.791+0000 7f9ab23e2740 1 -- 172.21.15.159:0/128814438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e2c8c0 msgr2=0x55aba1e48960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:00.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.791+0000 7f9ab23e2740 1 --2- 172.21.15.159:0/128814438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e2c8c0 0x55aba1e48960 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f9a94005bc0 tx=0x7f9a94007420 comp rx=0 tx=0).stop 2024-04-04T06:14:00.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.791+0000 7f9ab23e2740 1 -- 172.21.15.159:0/128814438 shutdown_connections 2024-04-04T06:14:00.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.791+0000 7f9ab23e2740 1 --2- 172.21.15.159:0/128814438 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55aba1e21c20 0x55aba1e48ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.791+0000 7f9ab23e2740 1 --2- 172.21.15.159:0/128814438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e2c8c0 0x55aba1e48960 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.791+0000 7f9ab23e2740 1 --2- 172.21.15.159:0/128814438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aba1e00440 0x55aba1e576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.791+0000 7f9ab23e2740 1 -- 172.21.15.159:0/128814438 >> 172.21.15.159:0/128814438 conn(0x55aba1dfc8e0 msgr2=0x55aba1e2b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:00.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.791+0000 7f9ab23e2740 1 -- 172.21.15.159:0/128814438 shutdown_connections 2024-04-04T06:14:00.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.791+0000 7f9ab23e2740 1 -- 172.21.15.159:0/128814438 wait complete. 2024-04-04T06:14:00.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.792+0000 7f9ab23e2740 1 -- start start 2024-04-04T06:14:00.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.792+0000 7f9ab23e2740 1 Processor -- start 2024-04-04T06:14:00.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.793+0000 7f9ab23e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e00440 0x55aba1e20d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.793+0000 7f9ab23e2740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55aba1e21c20 0x55aba1e5a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.793+0000 7f9ab23e2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aba1e2c8c0 0x55aba1e48960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.793+0000 7f9ab23e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55aba1e015f0 con 0x55aba1e00440 2024-04-04T06:14:00.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.793+0000 7f9ab23e2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55aba1d217a0 con 0x55aba1e2c8c0 2024-04-04T06:14:00.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.793+0000 7f9ab23e2740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55aba1e61770 con 0x55aba1e21c20 2024-04-04T06:14:00.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.793+0000 7f9ab23e2740 10 client.0 _ll_register_callbacks cb 0x7f9aa0072070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:14:00.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.793+0000 7f9aacc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e00440 0x55aba1e20d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.793+0000 7f9aacc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e00440 0x55aba1e20d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60484/0 (socket says 172.21.15.159:60484) 2024-04-04T06:14:00.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.793+0000 7f9aacc00640 1 -- 172.21.15.159:0/4175543116 learned_addr learned my addr 172.21.15.159:0/4175543116 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:00.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.793+0000 7f9aacc00640 1 -- 172.21.15.159:0/4175543116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55aba1e21c20 msgr2=0x55aba1e5a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:00.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.793+0000 7f9aacc00640 1 --2- 172.21.15.159:0/4175543116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55aba1e21c20 0x55aba1e5a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.793+0000 7f9aacc00640 1 -- 172.21.15.159:0/4175543116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aba1e2c8c0 msgr2=0x55aba1e48960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:00.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.793+0000 7f9aacc00640 1 --2- 172.21.15.159:0/4175543116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aba1e2c8c0 0x55aba1e48960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.793+0000 7f9aacc00640 1 -- 172.21.15.159:0/4175543116 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aba1dfbdf0 con 0x55aba1e00440 2024-04-04T06:14:00.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.794+0000 7f9aacc00640 1 --2- 172.21.15.159:0/4175543116 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e00440 0x55aba1e20d60 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009630 tx=0x7f9a9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3541ad1402489b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:00.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.794+0000 7f9a8d600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:14:00.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.794+0000 7f9a8d600640 1 -- 172.21.15.159:0/4175543116 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c05a060 con 0x55aba1e00440 2024-04-04T06:14:00.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.794+0000 7f9a8d600640 1 -- 172.21.15.159:0/4175543116 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5853+0+0 (secure 0 0 0) 0x7f9a9c0545a0 con 0x55aba1e00440 2024-04-04T06:14:00.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.794+0000 7f9ab23e2740 1 -- 172.21.15.159:0/4175543116 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a94005cb0 con 0x55aba1e00440 2024-04-04T06:14:00.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.794+0000 7f9a8d600640 1 -- 172.21.15.159:0/4175543116 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f9a9c034040 con 0x55aba1e00440 2024-04-04T06:14:00.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.794+0000 7f9ab23e2740 10 client.7755 fetch_fsmap learned FSMap version 69 2024-04-04T06:14:00.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.794+0000 7f9ab23e2740 1 -- 172.21.15.159:0/4175543116 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x55aba1e5ad20 con 0x55aba1e00440 2024-04-04T06:14:00.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.794+0000 7f9a8d600640 1 -- 172.21.15.159:0/4175543116 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a94005cb0 con 0x55aba1e00440 2024-04-04T06:14:00.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.795+0000 7f9a8d600640 1 -- 172.21.15.159:0/4175543116 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 69) ==== 3494+0+0 (secure 0 0 0) 0x7f9a9c063020 con 0x55aba1e00440 2024-04-04T06:14:00.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.795+0000 7f9ab23e2740 10 client.7755 fetch_fsmap finished waiting for FSMap version 69 2024-04-04T06:14:00.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.795+0000 7f9ab23e2740 10 client.7755 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:14:00.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.795+0000 7f9ab23e2740 20 client.7755 populate_metadata read hostname 'smithi159' 2024-04-04T06:14:00.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.795+0000 7f9ab23e2740 1 --2- 172.21.15.159:0/4175543116 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55aba1e27fc0 0x55aba1e283a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:00.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.795+0000 7f9ab23e2740 4 client.7755 mds_command: new command op to 7512 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-04T06:14:00.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.795+0000 7f9ab23e2740 1 -- 172.21.15.159:0/4175543116 --> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55aba1d73b00 con 0x55aba1e27fc0 2024-04-04T06:14:00.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.795+0000 7f9aa7e00640 1 --2- 172.21.15.159:0/4175543116 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55aba1e27fc0 0x55aba1e283a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:00.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.796+0000 7f9aa7e00640 1 --2- 172.21.15.159:0/4175543116 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55aba1e27fc0 0x55aba1e283a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a1c0eef87c294d1 server_cookie=15bc1000048cf7f2 in_seq=0 out_seq=0 2024-04-04T06:14:00.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.796+0000 7f9a8d600640 10 client.7755 ms_handle_connect on v2:172.21.15.159:6833/3796123128 2024-04-04T06:14:00.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.796+0000 7f9a8d600640 1 -- 172.21.15.159:0/4175543116 <== mds.0 v2:172.21.15.159:6833/3796123128 1 ==== command_reply(tid 0: 0 ) ==== 8+0+143 (crc 0 0 0) 0x7f9a9401a0a0 con 0x55aba1e27fc0 2024-04-04T06:14:00.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.796+0000 7f9a8d600640 10 client.7755 handle_command_reply: tid=0 2024-04-04T06:14:00.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.796+0000 7f9ab23e2740 1 client.7755 shutdown 2024-04-04T06:14:00.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.796+0000 7f9ab23e2740 1 -- 172.21.15.159:0/4175543116 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e00440 msgr2=0x55aba1e20d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:00.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.796+0000 7f9ab23e2740 1 --2- 172.21.15.159:0/4175543116 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e00440 0x55aba1e20d60 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009630 tx=0x7f9a9c004730 comp rx=0 tx=0).stop 2024-04-04T06:14:00.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.796+0000 7f9ab23e2740 1 -- 172.21.15.159:0/4175543116 shutdown_connections 2024-04-04T06:14:00.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.796+0000 7f9ab23e2740 1 --2- 172.21.15.159:0/4175543116 >> [v2:172.21.15.159:6833/3796123128,v1:172.21.15.159:6835/3796123128] conn(0x55aba1e27fc0 0x55aba1e283a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.796+0000 7f9ab23e2740 1 --2- 172.21.15.159:0/4175543116 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aba1e2c8c0 0x55aba1e48960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.796+0000 7f9ab23e2740 1 --2- 172.21.15.159:0/4175543116 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55aba1e21c20 0x55aba1e5a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.796+0000 7f9ab23e2740 1 --2- 172.21.15.159:0/4175543116 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55aba1e00440 0x55aba1e20d60 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.796+0000 7f9ab23e2740 1 -- 172.21.15.159:0/4175543116 >> 172.21.15.159:0/4175543116 conn(0x55aba1dfc8e0 msgr2=0x55aba1e23820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:00.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.796+0000 7f9a8d600640 0 client.7755 ms_handle_reset on v2:172.21.15.159:6833/3796123128 2024-04-04T06:14:00.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.796+0000 7f9ab23e2740 1 -- 172.21.15.159:0/4175543116 shutdown_connections 2024-04-04T06:14:00.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.796+0000 7f9ab23e2740 1 -- 172.21.15.159:0/4175543116 wait complete. 2024-04-04T06:14:00.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.796+0000 7f9ab23e2740 20 client.7755 trim_cache size 0 max 16384 2024-04-04T06:14:00.794 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:14:00.794 INFO:teuthology.orchestra.run.smithi159.stdout:[ 2024-04-04T06:14:00.794 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:14:00.794 INFO:teuthology.orchestra.run.smithi159.stdout: "damage_type": "backtrace", 2024-04-04T06:14:00.794 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 465334627, 2024-04-04T06:14:00.794 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 1099511627777, 2024-04-04T06:14:00.794 INFO:teuthology.orchestra.run.smithi159.stdout: "path": "/testdir/hardlink" 2024-04-04T06:14:00.795 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:14:00.795 INFO:teuthology.orchestra.run.smithi159.stdout:] 2024-04-04T06:14:00.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.797+0000 7f9aae000640 1 -- 172.21.15.159:0/3330556626 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9a8806c8f0 msgr2=0x7f9a8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:00.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.797+0000 7f9aae000640 1 --2- 172.21.15.159:0/3330556626 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9a8806c8f0 0x7f9a8806edb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c036000 tx=0x7f9a9c03b040 comp rx=0 tx=0).stop 2024-04-04T06:14:00.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.797+0000 7f9aae000640 1 -- 172.21.15.159:0/3330556626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa814cd90 msgr2=0x7f9aa81f35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:00.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.797+0000 7f9aae000640 1 --2- 172.21.15.159:0/3330556626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa814cd90 0x7f9aa81f35a0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000cd50 tx=0x7f9aa000a650 comp rx=0 tx=0).stop 2024-04-04T06:14:00.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.797+0000 7f9aae000640 1 -- 172.21.15.159:0/3330556626 shutdown_connections 2024-04-04T06:14:00.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.797+0000 7f9aae000640 1 --2- 172.21.15.159:0/3330556626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa814cd90 0x7f9aa81f35a0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.797+0000 7f9aae000640 1 --2- 172.21.15.159:0/3330556626 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aa80b4890 0x7f9aa814b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.797+0000 7f9aae000640 1 --2- 172.21.15.159:0/3330556626 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9a8806c8f0 0x7f9a8806edb0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.797+0000 7f9aae000640 1 --2- 172.21.15.159:0/3330556626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9aa806d620 0x7f9aa814e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:00.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.797+0000 7f9aae000640 1 -- 172.21.15.159:0/3330556626 >> 172.21.15.159:0/3330556626 conn(0x7f9aa80b1ba0 msgr2=0x7f9aa8072570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:00.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.797+0000 7f9aae000640 1 -- 172.21.15.159:0/3330556626 shutdown_connections 2024-04-04T06:14:00.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:00.798+0000 7f9aae000640 1 -- 172.21.15.159:0/3330556626 wait complete. 2024-04-04T06:14:00.816 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:14:00.816 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:14:00.864 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:14:00.864 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:14:00.953 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:14:01.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.201+0000 7f0726400640 1 Processor -- start 2024-04-04T06:14:01.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.201+0000 7f0726400640 1 -- start start 2024-04-04T06:14:01.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.203+0000 7f0726400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f072015eab0 0x7f072006c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:01.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.203+0000 7f0726400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07200af490 0x7f072006cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:01.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.203+0000 7f0726400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07200b53e0 con 0x7f07200b5990 2024-04-04T06:14:01.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.203+0000 7f0726400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f07200b56c0 con 0x7f072015eab0 2024-04-04T06:14:01.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.203+0000 7f0726400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f07200adc80 con 0x7f07200af490 2024-04-04T06:14:01.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.203+0000 7f071f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f072015eab0 0x7f072006c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:01.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.203+0000 7f071f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f072015eab0 0x7f072006c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60498/0 (socket says 172.21.15.159:60498) 2024-04-04T06:14:01.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.203+0000 7f071f400640 1 -- 172.21.15.159:0/2633992243 learned_addr learned my addr 172.21.15.159:0/2633992243 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:01.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.204+0000 7f071fe00640 1 --2- 172.21.15.159:0/2633992243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07200af490 0x7f072006cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:01.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.204+0000 7f071f400640 1 -- 172.21.15.159:0/2633992243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07200af490 msgr2=0x7f072006cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:01.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.204+0000 7f071f400640 1 --2- 172.21.15.159:0/2633992243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07200af490 0x7f072006cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:01.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.204+0000 7f071f400640 1 -- 172.21.15.159:0/2633992243 >> v1:172.21.15.178:6789/0 conn(0x7f07200b5990 legacy=0x7f07200b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:01.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.204+0000 7f071f400640 1 -- 172.21.15.159:0/2633992243 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0720072a00 con 0x7f072015eab0 2024-04-04T06:14:01.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.205+0000 7f071f400640 1 --2- 172.21.15.159:0/2633992243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015eab0 0x7f072006c580 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f0710009870 tx=0x7f071002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8588ecabaae4407 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:01.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.205+0000 7f071ea00640 1 -- 172.21.15.159:0/2633992243 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710035020 con 0x7f072015eab0 2024-04-04T06:14:01.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.205+0000 7f071ea00640 1 -- 172.21.15.159:0/2633992243 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0710031640 con 0x7f072015eab0 2024-04-04T06:14:01.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.205+0000 7f071ea00640 1 -- 172.21.15.159:0/2633992243 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071002d040 con 0x7f072015eab0 2024-04-04T06:14:01.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.206+0000 7f0726400640 1 -- 172.21.15.159:0/2633992243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015eab0 msgr2=0x7f072006c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:01.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.206+0000 7f0726400640 1 --2- 172.21.15.159:0/2633992243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015eab0 0x7f072006c580 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f0710009870 tx=0x7f071002b8f0 comp rx=0 tx=0).stop 2024-04-04T06:14:01.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.206+0000 7f0726400640 1 -- 172.21.15.159:0/2633992243 shutdown_connections 2024-04-04T06:14:01.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.206+0000 7f0726400640 1 --2- 172.21.15.159:0/2633992243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07200af490 0x7f072006cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:01.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.206+0000 7f0726400640 1 --2- 172.21.15.159:0/2633992243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015eab0 0x7f072006c580 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:01.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.206+0000 7f0726400640 1 -- 172.21.15.159:0/2633992243 >> 172.21.15.159:0/2633992243 conn(0x7f0720071df0 msgr2=0x7f07200721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:01.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.206+0000 7f0726400640 1 -- 172.21.15.159:0/2633992243 shutdown_connections 2024-04-04T06:14:01.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.206+0000 7f0726400640 1 -- 172.21.15.159:0/2633992243 wait complete. 2024-04-04T06:14:01.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.208+0000 7f0726400640 1 Processor -- start 2024-04-04T06:14:01.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.208+0000 7f0726400640 1 -- start start 2024-04-04T06:14:01.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.209+0000 7f0726400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07200af490 0x7f0720147f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:01.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.209+0000 7f0726400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07200b5990 0x7f0720147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:01.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.209+0000 7f0726400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015eab0 0x7f07201611e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:01.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.209+0000 7f0726400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f07200b4430 con 0x7f072015eab0 2024-04-04T06:14:01.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.209+0000 7f0726400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f07200b42b0 con 0x7f07200af490 2024-04-04T06:14:01.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.209+0000 7f0726400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f07200b45b0 con 0x7f07200b5990 2024-04-04T06:14:01.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.209+0000 7f071fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015eab0 0x7f07201611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:01.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.209+0000 7f071fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015eab0 0x7f07201611e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60502/0 (socket says 172.21.15.159:60502) 2024-04-04T06:14:01.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.209+0000 7f071fe00640 1 -- 172.21.15.159:0/4123217166 learned_addr learned my addr 172.21.15.159:0/4123217166 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:01.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.209+0000 7f071f400640 1 --2- 172.21.15.159:0/4123217166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07200af490 0x7f0720147f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:01.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.209+0000 7f071fe00640 1 -- 172.21.15.159:0/4123217166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07200b5990 msgr2=0x7f0720147380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:01.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.209+0000 7f071fe00640 1 --2- 172.21.15.159:0/4123217166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07200b5990 0x7f0720147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:01.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.209+0000 7f071fe00640 1 -- 172.21.15.159:0/4123217166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07200af490 msgr2=0x7f0720147f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:01.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.209+0000 7f071fe00640 1 --2- 172.21.15.159:0/4123217166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07200af490 0x7f0720147f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:01.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.209+0000 7f071fe00640 1 -- 172.21.15.159:0/4123217166 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0720154930 con 0x7f072015eab0 2024-04-04T06:14:01.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.210+0000 7f071fe00640 1 --2- 172.21.15.159:0/4123217166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015eab0 0x7f07201611e0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f071800cd70 tx=0x7f071800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:01.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.210+0000 7f0717400640 1 -- 172.21.15.159:0/4123217166 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0718003b90 con 0x7f072015eab0 2024-04-04T06:14:01.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.210+0000 7f0717400640 1 -- 172.21.15.159:0/4123217166 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0718003d30 con 0x7f072015eab0 2024-04-04T06:14:01.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.210+0000 7f0717400640 1 -- 172.21.15.159:0/4123217166 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0718004240 con 0x7f072015eab0 2024-04-04T06:14:01.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.210+0000 7f0726400640 1 -- 172.21.15.159:0/4123217166 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0720154c10 con 0x7f072015eab0 2024-04-04T06:14:01.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.210+0000 7f0726400640 1 -- 172.21.15.159:0/4123217166 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07201478c0 con 0x7f072015eab0 2024-04-04T06:14:01.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.212+0000 7f0726400640 1 -- 172.21.15.159:0/4123217166 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07200685c0 con 0x7f072015eab0 2024-04-04T06:14:01.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.212+0000 7f0717400640 1 -- 172.21.15.159:0/4123217166 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07180043e0 con 0x7f072015eab0 2024-04-04T06:14:01.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.214+0000 7f0717400640 1 --2- 172.21.15.159:0/4123217166 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f06fc06c940 0x7f06fc06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:01.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.214+0000 7f0717400640 1 -- 172.21.15.159:0/4123217166 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5853+0+0 (secure 0 0 0) 0x7f071808c680 con 0x7f072015eab0 2024-04-04T06:14:01.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.214+0000 7f071f400640 1 --2- 172.21.15.159:0/4123217166 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f06fc06c940 0x7f06fc06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:01.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.216+0000 7f071f400640 1 --2- 172.21.15.159:0/4123217166 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f06fc06c940 0x7f06fc06ee00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0710036000 tx=0x7f0710008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:01.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.218+0000 7f0717400640 1 -- 172.21.15.159:0/4123217166 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f071805aa90 con 0x7f072015eab0 2024-04-04T06:14:01.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.420+0000 7f0726400640 1 -- 172.21.15.159:0/4123217166 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0720074a50 con 0x7f072015eab0 2024-04-04T06:14:01.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.423+0000 7f0717400640 1 -- 172.21.15.159:0/4123217166 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+16901 (secure 0 0 0) 0x7f071805a390 con 0x7f072015eab0 2024-04-04T06:14:01.417 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:14:01.417 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":88,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:13:35.801126+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:12:33.658405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-04T06:12:34.856266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.3d","mappings":[{"from":6,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6835/3484779950":"2024-04-05T06:12:49.205338+0000","172.21.15.159:6833/3484779950":"2024-04-05T06:12:49.205338+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:14:01.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.425+0000 7f0726400640 1 -- 172.21.15.159:0/4123217166 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f06fc06c940 msgr2=0x7f06fc06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:01.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.425+0000 7f0726400640 1 --2- 172.21.15.159:0/4123217166 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f06fc06c940 0x7f06fc06ee00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0710036000 tx=0x7f0710008030 comp rx=0 tx=0).stop 2024-04-04T06:14:01.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.425+0000 7f0726400640 1 -- 172.21.15.159:0/4123217166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015eab0 msgr2=0x7f07201611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:01.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.425+0000 7f0726400640 1 --2- 172.21.15.159:0/4123217166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015eab0 0x7f07201611e0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f071800cd70 tx=0x7f071800a660 comp rx=0 tx=0).stop 2024-04-04T06:14:01.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.426+0000 7f0726400640 1 -- 172.21.15.159:0/4123217166 shutdown_connections 2024-04-04T06:14:01.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.426+0000 7f0726400640 1 --2- 172.21.15.159:0/4123217166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f072015eab0 0x7f07201611e0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:01.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.426+0000 7f0726400640 1 --2- 172.21.15.159:0/4123217166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07200b5990 0x7f0720147380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:01.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.426+0000 7f0726400640 1 --2- 172.21.15.159:0/4123217166 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f06fc06c940 0x7f06fc06ee00 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:01.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.426+0000 7f0726400640 1 --2- 172.21.15.159:0/4123217166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07200af490 0x7f0720147f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:01.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.426+0000 7f0726400640 1 -- 172.21.15.159:0/4123217166 >> 172.21.15.159:0/4123217166 conn(0x7f0720071df0 msgr2=0x7f07200b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:01.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.426+0000 7f0726400640 1 -- 172.21.15.159:0/4123217166 shutdown_connections 2024-04-04T06:14:01.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.426+0000 7f0726400640 1 -- 172.21.15.159:0/4123217166 wait complete. 2024-04-04T06:14:01.439 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:12:33.658405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:12:34.856266+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:14:01.440 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:14:01.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.660+0000 7f33f8200640 1 Processor -- start 2024-04-04T06:14:01.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.661+0000 7f33f8200640 1 -- start start 2024-04-04T06:14:01.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.662+0000 7f33f8200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f33f0155550 0x7f33f0155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:01.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.662+0000 7f33f8200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f33f015f800 0x7f33f00d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:01.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.662+0000 7f33f8200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33f00b40b0 con 0x7f33f00d07c0 2024-04-04T06:14:01.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.662+0000 7f33f8200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f33f00d0bf0 con 0x7f33f0155550 2024-04-04T06:14:01.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.662+0000 7f33f8200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f33f00b4390 con 0x7f33f015f800 2024-04-04T06:14:01.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.662+0000 7f33f5800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f33f0155550 0x7f33f0155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:01.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.662+0000 7f33f6200640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f33f00d07c0 0x7f33f00aede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:48332/0 (socket says 172.21.15.159:48332) 2024-04-04T06:14:01.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.662+0000 7f33f6200640 1 -- 172.21.15.159:0/448092143 learned_addr learned my addr 172.21.15.159:0/448092143 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:01.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.663+0000 7f33f4e00640 1 --2- 172.21.15.159:0/448092143 >> v2:172.21.15.178:3300/0 conn(0x7f33f015f800 0x7f33f00d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:01.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.663+0000 7f33f5800640 1 -- 172.21.15.159:0/448092143 >> v2:172.21.15.178:3300/0 conn(0x7f33f015f800 msgr2=0x7f33f00d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:01.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.663+0000 7f33f5800640 1 --2- 172.21.15.159:0/448092143 >> v2:172.21.15.178:3300/0 conn(0x7f33f015f800 0x7f33f00d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:01.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.663+0000 7f33f5800640 1 -- 172.21.15.159:0/448092143 >> v1:172.21.15.159:6789/0 conn(0x7f33f00d07c0 legacy=0x7f33f00aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:01.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.663+0000 7f33efe00640 1 -- 172.21.15.159:0/448092143 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1104637404 0 0) 0x7f33f00b40b0 con 0x7f33f00d07c0 2024-04-04T06:14:01.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.663+0000 7f33f5800640 1 -- 172.21.15.159:0/448092143 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f00b4670 con 0x7f33f0155550 2024-04-04T06:14:01.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.664+0000 7f33f5800640 1 --2- 172.21.15.159:0/448092143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33f0155550 0x7f33f0155930 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f33f00d0dd0 tx=0x7f33e402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a4faf5f61f2850 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:01.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.664+0000 7f33efe00640 1 -- 172.21.15.159:0/448092143 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e4039070 con 0x7f33f0155550 2024-04-04T06:14:01.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.664+0000 7f33efe00640 1 -- 172.21.15.159:0/448092143 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33e402cbf0 con 0x7f33f0155550 2024-04-04T06:14:01.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.665+0000 7f33efe00640 1 -- 172.21.15.159:0/448092143 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e4034440 con 0x7f33f0155550 2024-04-04T06:14:01.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.665+0000 7f33f8200640 1 -- 172.21.15.159:0/448092143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33f0155550 msgr2=0x7f33f0155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:01.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.665+0000 7f33f8200640 1 --2- 172.21.15.159:0/448092143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33f0155550 0x7f33f0155930 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f33f00d0dd0 tx=0x7f33e402be40 comp rx=0 tx=0).stop 2024-04-04T06:14:01.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.665+0000 7f33f8200640 1 -- 172.21.15.159:0/448092143 shutdown_connections 2024-04-04T06:14:01.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.665+0000 7f33f8200640 1 --2- 172.21.15.159:0/448092143 >> v2:172.21.15.178:3300/0 conn(0x7f33f015f800 0x7f33f00d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:01.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.665+0000 7f33f8200640 1 --2- 172.21.15.159:0/448092143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33f0155550 0x7f33f0155930 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:01.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.665+0000 7f33f8200640 1 -- 172.21.15.159:0/448092143 >> 172.21.15.159:0/448092143 conn(0x7f33f006b750 msgr2=0x7f33f006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:01.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.665+0000 7f33f8200640 1 -- 172.21.15.159:0/448092143 shutdown_connections 2024-04-04T06:14:01.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.665+0000 7f33f8200640 1 -- 172.21.15.159:0/448092143 wait complete. 2024-04-04T06:14:01.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.667+0000 7f33f8200640 1 Processor -- start 2024-04-04T06:14:01.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.667+0000 7f33f8200640 1 -- start start 2024-04-04T06:14:01.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.668+0000 7f33f8200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33f00d07c0 0x7f33f014ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:01.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.668+0000 7f33f8200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33f0155550 0x7f33f00afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:01.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.668+0000 7f33f8200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33f015f800 0x7f33f0099900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:01.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.668+0000 7f33f8200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f33f015fc80 con 0x7f33f0155550 2024-04-04T06:14:01.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.668+0000 7f33f8200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f33f00b4c30 con 0x7f33f015f800 2024-04-04T06:14:01.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.668+0000 7f33f8200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f33f015ba20 con 0x7f33f00d07c0 2024-04-04T06:14:01.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.668+0000 7f33f4e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33f0155550 0x7f33f00afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:01.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.668+0000 7f33f4e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33f0155550 0x7f33f00afc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60514/0 (socket says 172.21.15.159:60514) 2024-04-04T06:14:01.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.668+0000 7f33f4e00640 1 -- 172.21.15.159:0/1611682335 learned_addr learned my addr 172.21.15.159:0/1611682335 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:01.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.669+0000 7f33f4e00640 1 -- 172.21.15.159:0/1611682335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33f00d07c0 msgr2=0x7f33f014ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:01.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.669+0000 7f33f4e00640 1 --2- 172.21.15.159:0/1611682335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33f00d07c0 0x7f33f014ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:01.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.669+0000 7f33f4e00640 1 -- 172.21.15.159:0/1611682335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33f015f800 msgr2=0x7f33f0099900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:01.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.669+0000 7f33f6200640 1 --2- 172.21.15.159:0/1611682335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33f015f800 0x7f33f0099900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:01.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.669+0000 7f33f4e00640 1 --2- 172.21.15.159:0/1611682335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33f015f800 0x7f33f0099900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:01.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.669+0000 7f33f4e00640 1 -- 172.21.15.159:0/1611682335 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f00b0180 con 0x7f33f0155550 2024-04-04T06:14:01.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.669+0000 7f33f4e00640 1 --2- 172.21.15.159:0/1611682335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33f0155550 0x7f33f00afc40 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00d800 tx=0x7f33dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:01.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.669+0000 7f33ed600640 1 -- 172.21.15.159:0/1611682335 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc00b6c0 con 0x7f33f0155550 2024-04-04T06:14:01.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.669+0000 7f33ed600640 1 -- 172.21.15.159:0/1611682335 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33dc019070 con 0x7f33f0155550 2024-04-04T06:14:01.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.669+0000 7f33ed600640 1 -- 172.21.15.159:0/1611682335 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc00ba80 con 0x7f33f0155550 2024-04-04T06:14:01.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.670+0000 7f33f8200640 1 -- 172.21.15.159:0/1611682335 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f00b0450 con 0x7f33f0155550 2024-04-04T06:14:01.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.670+0000 7f33f8200640 1 -- 172.21.15.159:0/1611682335 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f0153270 con 0x7f33f0155550 2024-04-04T06:14:01.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.671+0000 7f33f8200640 1 -- 172.21.15.159:0/1611682335 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33f006f520 con 0x7f33f0155550 2024-04-04T06:14:01.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.672+0000 7f33ed600640 1 -- 172.21.15.159:0/1611682335 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f33dc00bc20 con 0x7f33f0155550 2024-04-04T06:14:01.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.673+0000 7f33ed600640 1 --2- 172.21.15.159:0/1611682335 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f33d006c920 0x7f33d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:01.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.673+0000 7f33ed600640 1 -- 172.21.15.159:0/1611682335 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5853+0+0 (secure 0 0 0) 0x7f33dc08c110 con 0x7f33f0155550 2024-04-04T06:14:01.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.673+0000 7f33f5800640 1 --2- 172.21.15.159:0/1611682335 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f33d006c920 0x7f33d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:01.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.674+0000 7f33f5800640 1 --2- 172.21.15.159:0/1611682335 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f33d006c920 0x7f33d006ede0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f33e4036000 tx=0x7f33e402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:01.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.677+0000 7f33ed600640 1 -- 172.21.15.159:0/1611682335 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f33f006f520 con 0x7f33f0155550 2024-04-04T06:14:01.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.877+0000 7f33f8200640 1 -- 172.21.15.159:0/1611682335 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f33f0144db0 con 0x7f33f0155550 2024-04-04T06:14:01.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.878+0000 7f33ed600640 1 -- 172.21.15.159:0/1611682335 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f33f0144db0 con 0x7f33f0155550 2024-04-04T06:14:01.872 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:14:01.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.880+0000 7f33f8200640 1 -- 172.21.15.159:0/1611682335 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f33d006c920 msgr2=0x7f33d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:01.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.880+0000 7f33f8200640 1 --2- 172.21.15.159:0/1611682335 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f33d006c920 0x7f33d006ede0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f33e4036000 tx=0x7f33e402e000 comp rx=0 tx=0).stop 2024-04-04T06:14:01.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.880+0000 7f33f8200640 1 -- 172.21.15.159:0/1611682335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33f0155550 msgr2=0x7f33f00afc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:01.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.880+0000 7f33f8200640 1 --2- 172.21.15.159:0/1611682335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33f0155550 0x7f33f00afc40 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00d800 tx=0x7f33dc00dcc0 comp rx=0 tx=0).stop 2024-04-04T06:14:01.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.880+0000 7f33f8200640 1 -- 172.21.15.159:0/1611682335 shutdown_connections 2024-04-04T06:14:01.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.881+0000 7f33f8200640 1 --2- 172.21.15.159:0/1611682335 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f33d006c920 0x7f33d006ede0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:01.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.881+0000 7f33f8200640 1 --2- 172.21.15.159:0/1611682335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33f015f800 0x7f33f0099900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:01.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.881+0000 7f33f8200640 1 --2- 172.21.15.159:0/1611682335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33f0155550 0x7f33f00afc40 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:01.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.881+0000 7f33f8200640 1 --2- 172.21.15.159:0/1611682335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f33f00d07c0 0x7f33f014ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:01.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.881+0000 7f33f8200640 1 -- 172.21.15.159:0/1611682335 >> 172.21.15.159:0/1611682335 conn(0x7f33f006b750 msgr2=0x7f33f006e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:01.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.881+0000 7f33f8200640 1 -- 172.21.15.159:0/1611682335 shutdown_connections 2024-04-04T06:14:01.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:01.881+0000 7f33f8200640 1 -- 172.21.15.159:0/1611682335 wait complete. 2024-04-04T06:14:01.897 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:14:02.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.152+0000 7f3b7ac00640 1 Processor -- start 2024-04-04T06:14:02.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.152+0000 7f3b7ac00640 1 -- start start 2024-04-04T06:14:02.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.154+0000 7f3b7ac00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b740ccc40 0x7f3b7406c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:02.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.154+0000 7f3b7ac00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b740b4710 con 0x7f3b740afd80 2024-04-04T06:14:02.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.154+0000 7f3b7ac00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b740b4980 con 0x7f3b7406cac0 2024-04-04T06:14:02.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.154+0000 7f3b7ac00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3b740b5740 con 0x7f3b740ccc40 2024-04-04T06:14:02.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.154+0000 7f3b73e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f3b740afd80 0x7f3b7415d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:48340/0 (socket says 172.21.15.159:48340) 2024-04-04T06:14:02.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.154+0000 7f3b73e00640 1 -- 172.21.15.159:0/3271326476 learned_addr learned my addr 172.21.15.159:0/3271326476 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:02.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.155+0000 7f3b73400640 1 -- 172.21.15.159:0/3271326476 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4002954953 0 0) 0x7f3b740b4710 con 0x7f3b740afd80 2024-04-04T06:14:02.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.155+0000 7f3b73400640 1 -- 172.21.15.159:0/3271326476 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b58003520 con 0x7f3b740afd80 2024-04-04T06:14:02.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.155+0000 7f3b6b400640 1 --2- 172.21.15.159:0/3271326476 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b740ccc40 0x7f3b7406c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:02.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.155+0000 7f3b73400640 1 -- 172.21.15.159:0/3271326476 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2197560232 0 0) 0x7f3b58003520 con 0x7f3b740afd80 2024-04-04T06:14:02.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.155+0000 7f3b73400640 1 -- 172.21.15.159:0/3271326476 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b740ccc40 msgr2=0x7f3b7406c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:02.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.155+0000 7f3b73400640 1 --2- 172.21.15.159:0/3271326476 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b740ccc40 0x7f3b7406c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:02.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.155+0000 7f3b73400640 1 -- 172.21.15.159:0/3271326476 >> v1:172.21.15.178:6789/0 conn(0x7f3b7406cac0 legacy=0x7f3b740ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:02.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.155+0000 7f3b73400640 1 -- 172.21.15.159:0/3271326476 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b740b4c60 con 0x7f3b740afd80 2024-04-04T06:14:02.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.155+0000 7f3b73400640 1 -- 172.21.15.159:0/3271326476 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f3b64001690 con 0x7f3b740afd80 2024-04-04T06:14:02.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.155+0000 7f3b73400640 1 -- 172.21.15.159:0/3271326476 >> v1:172.21.15.159:6789/0 conn(0x7f3b740afd80 legacy=0x7f3b7415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:02.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.155+0000 7f3b73400640 1 --2- 172.21.15.159:0/3271326476 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b740ccc40 0x7f3b580039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:02.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.155+0000 7f3b73400640 1 --2- 172.21.15.159:0/3271326476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b58003fb0 0x7f3b58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:02.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.155+0000 7f3b73400640 1 --2- 172.21.15.159:0/3271326476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b58004d40 0x7f3b580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:02.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.156+0000 7f3b73400640 1 -- 172.21.15.159:0/3271326476 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3b740b5740 con 0x7f3b58003fb0 2024-04-04T06:14:02.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.156+0000 7f3b73400640 1 -- 172.21.15.159:0/3271326476 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5800bb00 con 0x7f3b58004d40 2024-04-04T06:14:02.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.156+0000 7f3b73400640 1 -- 172.21.15.159:0/3271326476 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3b5800bd00 con 0x7f3b740ccc40 2024-04-04T06:14:02.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.156+0000 7f3b6b400640 1 --2- 172.21.15.159:0/3271326476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b58003fb0 0x7f3b58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:02.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.156+0000 7f3b78c00640 1 --2- 172.21.15.159:0/3271326476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b58004d40 0x7f3b580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:02.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.156+0000 7f3b73e00640 1 --2- 172.21.15.159:0/3271326476 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b740ccc40 0x7f3b580039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:02.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.156+0000 7f3b6b400640 1 -- 172.21.15.159:0/3271326476 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b740ccc40 msgr2=0x7f3b580039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:02.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.156+0000 7f3b6b400640 1 --2- 172.21.15.159:0/3271326476 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b740ccc40 0x7f3b580039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:02.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.156+0000 7f3b6b400640 1 -- 172.21.15.159:0/3271326476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b58004d40 msgr2=0x7f3b580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:02.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.156+0000 7f3b6b400640 1 --2- 172.21.15.159:0/3271326476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b58004d40 0x7f3b580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:02.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.156+0000 7f3b6b400640 1 -- 172.21.15.159:0/3271326476 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b58003520 con 0x7f3b58003fb0 2024-04-04T06:14:02.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.156+0000 7f3b78c00640 1 --2- 172.21.15.159:0/3271326476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b58004d40 0x7f3b580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:14:02.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.158+0000 7f3b6b400640 1 --2- 172.21.15.159:0/3271326476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b58003fb0 0x7f3b58004800 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009ac0 tx=0x7f3b58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378dd47822b9d09e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:02.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.158+0000 7f3b73400640 1 -- 172.21.15.159:0/3271326476 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c039070 con 0x7f3b58003fb0 2024-04-04T06:14:02.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.158+0000 7f3b73400640 1 -- 172.21.15.159:0/3271326476 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b5c02c600 con 0x7f3b58003fb0 2024-04-04T06:14:02.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.158+0000 7f3b73400640 1 -- 172.21.15.159:0/3271326476 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c009690 con 0x7f3b58003fb0 2024-04-04T06:14:02.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.159+0000 7f3b7ac00640 1 -- 172.21.15.159:0/3271326476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b58003fb0 msgr2=0x7f3b58004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:02.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.159+0000 7f3b7ac00640 1 --2- 172.21.15.159:0/3271326476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b58003fb0 0x7f3b58004800 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009ac0 tx=0x7f3b58005830 comp rx=0 tx=0).stop 2024-04-04T06:14:02.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.159+0000 7f3b78c00640 1 -- 172.21.15.159:0/3271326476 reap_dead start 2024-04-04T06:14:02.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.159+0000 7f3b7ac00640 1 -- 172.21.15.159:0/3271326476 shutdown_connections 2024-04-04T06:14:02.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.159+0000 7f3b7ac00640 1 -- 172.21.15.159:0/3271326476 >> 172.21.15.159:0/3271326476 conn(0x7f3b74071e20 msgr2=0x7f3b74072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:02.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.159+0000 7f3b7ac00640 1 -- 172.21.15.159:0/3271326476 shutdown_connections 2024-04-04T06:14:02.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.159+0000 7f3b7ac00640 1 -- 172.21.15.159:0/3271326476 wait complete. 2024-04-04T06:14:02.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.161+0000 7f3b7ac00640 1 Processor -- start 2024-04-04T06:14:02.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.161+0000 7f3b7ac00640 1 -- start start 2024-04-04T06:14:02.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.162+0000 7f3b7ac00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b58003fb0 0x7f3b74157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:02.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.162+0000 7f3b7ac00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b741424c0 0x7f3b7414f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:02.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.162+0000 7f3b7ac00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b7415b6b0 0x7f3b7415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:02.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.162+0000 7f3b7ac00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3b740b5220 con 0x7f3b741424c0 2024-04-04T06:14:02.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.162+0000 7f3b7ac00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7406cf30 con 0x7f3b58003fb0 2024-04-04T06:14:02.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.162+0000 7f3b7ac00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3b740b0160 con 0x7f3b7415b6b0 2024-04-04T06:14:02.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.162+0000 7f3b6b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b741424c0 0x7f3b7414f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:02.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.162+0000 7f3b6b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b741424c0 0x7f3b7414f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60534/0 (socket says 172.21.15.159:60534) 2024-04-04T06:14:02.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.162+0000 7f3b6b400640 1 -- 172.21.15.159:0/3465792464 learned_addr learned my addr 172.21.15.159:0/3465792464 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:02.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.163+0000 7f3b73e00640 1 --2- 172.21.15.159:0/3465792464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b58003fb0 0x7f3b74157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:02.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.163+0000 7f3b78c00640 1 --2- 172.21.15.159:0/3465792464 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b7415b6b0 0x7f3b7415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:02.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.163+0000 7f3b6b400640 1 -- 172.21.15.159:0/3465792464 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b7415b6b0 msgr2=0x7f3b7415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:02.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.163+0000 7f3b6b400640 1 --2- 172.21.15.159:0/3465792464 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b7415b6b0 0x7f3b7415bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:02.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.163+0000 7f3b6b400640 1 -- 172.21.15.159:0/3465792464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b58003fb0 msgr2=0x7f3b74157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:02.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.163+0000 7f3b6b400640 1 --2- 172.21.15.159:0/3465792464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b58003fb0 0x7f3b74157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:02.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.163+0000 7f3b6b400640 1 -- 172.21.15.159:0/3465792464 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7414fd50 con 0x7f3b741424c0 2024-04-04T06:14:02.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.163+0000 7f3b6b400640 1 --2- 172.21.15.159:0/3465792464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b741424c0 0x7f3b7414f600 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c037040 tx=0x7f3b5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:02.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.163+0000 7f3b70c00640 1 -- 172.21.15.159:0/3465792464 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c039050 con 0x7f3b741424c0 2024-04-04T06:14:02.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.163+0000 7f3b70c00640 1 -- 172.21.15.159:0/3465792464 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b5c03a040 con 0x7f3b741424c0 2024-04-04T06:14:02.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.163+0000 7f3b70c00640 1 -- 172.21.15.159:0/3465792464 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c0044e0 con 0x7f3b741424c0 2024-04-04T06:14:02.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.163+0000 7f3b7ac00640 1 -- 172.21.15.159:0/3465792464 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b74150030 con 0x7f3b741424c0 2024-04-04T06:14:02.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.164+0000 7f3b7ac00640 1 -- 172.21.15.159:0/3465792464 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b741505f0 con 0x7f3b741424c0 2024-04-04T06:14:02.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.165+0000 7f3b7ac00640 1 -- 172.21.15.159:0/3465792464 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b74068cd0 con 0x7f3b741424c0 2024-04-04T06:14:02.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.166+0000 7f3b70c00640 1 -- 172.21.15.159:0/3465792464 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b5c047020 con 0x7f3b741424c0 2024-04-04T06:14:02.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.167+0000 7f3b70c00640 1 --2- 172.21.15.159:0/3465792464 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3b5006c920 0x7f3b5006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:02.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.167+0000 7f3b70c00640 1 -- 172.21.15.159:0/3465792464 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5853+0+0 (secure 0 0 0) 0x7f3b5c0ad340 con 0x7f3b741424c0 2024-04-04T06:14:02.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.167+0000 7f3b73e00640 1 --2- 172.21.15.159:0/3465792464 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3b5006c920 0x7f3b5006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:02.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.168+0000 7f3b73e00640 1 --2- 172.21.15.159:0/3465792464 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3b5006c920 0x7f3b5006ede0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3b7415a630 tx=0x7f3b64009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:02.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.175+0000 7f3b70c00640 1 -- 172.21.15.159:0/3465792464 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3b5c04e020 con 0x7f3b741424c0 2024-04-04T06:14:02.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.376+0000 7f3b7ac00640 1 -- 172.21.15.159:0/3465792464 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3b74074bf0 con 0x7f3b741424c0 2024-04-04T06:14:02.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.377+0000 7f3b70c00640 1 -- 172.21.15.159:0/3465792464 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f3b5c07b6d0 con 0x7f3b741424c0 2024-04-04T06:14:02.375 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:14:02.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.380+0000 7f3b7ac00640 1 -- 172.21.15.159:0/3465792464 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3b5006c920 msgr2=0x7f3b5006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:02.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.380+0000 7f3b7ac00640 1 --2- 172.21.15.159:0/3465792464 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3b5006c920 0x7f3b5006ede0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3b7415a630 tx=0x7f3b64009040 comp rx=0 tx=0).stop 2024-04-04T06:14:02.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.380+0000 7f3b7ac00640 1 -- 172.21.15.159:0/3465792464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b741424c0 msgr2=0x7f3b7414f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:02.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.380+0000 7f3b7ac00640 1 --2- 172.21.15.159:0/3465792464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b741424c0 0x7f3b7414f600 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c037040 tx=0x7f3b5c004730 comp rx=0 tx=0).stop 2024-04-04T06:14:02.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.380+0000 7f3b7ac00640 1 -- 172.21.15.159:0/3465792464 shutdown_connections 2024-04-04T06:14:02.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.380+0000 7f3b7ac00640 1 --2- 172.21.15.159:0/3465792464 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3b7415b6b0 0x7f3b7415bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:02.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.380+0000 7f3b7ac00640 1 --2- 172.21.15.159:0/3465792464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b741424c0 0x7f3b7414f600 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:02.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.380+0000 7f3b7ac00640 1 --2- 172.21.15.159:0/3465792464 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3b5006c920 0x7f3b5006ede0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:02.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.380+0000 7f3b7ac00640 1 --2- 172.21.15.159:0/3465792464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b58003fb0 0x7f3b74157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:02.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.380+0000 7f3b7ac00640 1 -- 172.21.15.159:0/3465792464 >> 172.21.15.159:0/3465792464 conn(0x7f3b74071e20 msgr2=0x7f3b740ae6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:02.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.380+0000 7f3b7ac00640 1 -- 172.21.15.159:0/3465792464 shutdown_connections 2024-04-04T06:14:02.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.380+0000 7f3b7ac00640 1 -- 172.21.15.159:0/3465792464 wait complete. 2024-04-04T06:14:02.393 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:14:02.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.600+0000 7f1ecd200640 1 Processor -- start 2024-04-04T06:14:02.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.600+0000 7f1ecd200640 1 -- start start 2024-04-04T06:14:02.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.601+0000 7f1ecd200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ec815e860 0x7f1ec806d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:02.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.601+0000 7f1ecd200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ec806dfa0 con 0x7f1ec806da30 2024-04-04T06:14:02.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.601+0000 7f1ecd200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ec806e240 con 0x7f1ec80d4ae0 2024-04-04T06:14:02.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.601+0000 7f1ecd200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec80cf220 con 0x7f1ec815e860 2024-04-04T06:14:02.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.602+0000 7f1ebd400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f1ec806da30 0x7f1ec80d43d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:48344/0 (socket says 172.21.15.159:48344) 2024-04-04T06:14:02.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.602+0000 7f1ebd400640 1 -- 172.21.15.159:0/2092970862 learned_addr learned my addr 172.21.15.159:0/2092970862 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:02.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.602+0000 7f1ec5e00640 1 --2- 172.21.15.159:0/2092970862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ec815e860 0x7f1ec806d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:02.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.602+0000 7f1ec5400640 1 -- 172.21.15.159:0/2092970862 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1235042989 0 0) 0x7f1ec806dfa0 con 0x7f1ec806da30 2024-04-04T06:14:02.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.602+0000 7f1ec5400640 1 -- 172.21.15.159:0/2092970862 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1eac003520 con 0x7f1ec806da30 2024-04-04T06:14:02.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.603+0000 7f1ec5400640 1 -- 172.21.15.159:0/2092970862 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 872707217 0 0) 0x7f1eac003520 con 0x7f1ec806da30 2024-04-04T06:14:02.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.603+0000 7f1ec5400640 1 -- 172.21.15.159:0/2092970862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ec815e860 msgr2=0x7f1ec806d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:02.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.603+0000 7f1ec5400640 1 --2- 172.21.15.159:0/2092970862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ec815e860 0x7f1ec806d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:02.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.603+0000 7f1ec5400640 1 -- 172.21.15.159:0/2092970862 >> v1:172.21.15.178:6789/0 conn(0x7f1ec80d4ae0 legacy=0x7f1ec80cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:02.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.603+0000 7f1ec5400640 1 -- 172.21.15.159:0/2092970862 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec806e520 con 0x7f1ec806da30 2024-04-04T06:14:02.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.603+0000 7f1ec5400640 1 -- 172.21.15.159:0/2092970862 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f1eb0001690 con 0x7f1ec806da30 2024-04-04T06:14:02.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.603+0000 7f1ec5400640 1 -- 172.21.15.159:0/2092970862 >> v1:172.21.15.159:6789/0 conn(0x7f1ec806da30 legacy=0x7f1ec80d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:02.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.603+0000 7f1ec5400640 1 --2- 172.21.15.159:0/2092970862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ec815e860 0x7f1eac0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:02.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.603+0000 7f1ec5400640 1 --2- 172.21.15.159:0/2092970862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1eac003f60 0x7f1eac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:02.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.603+0000 7f1ec5400640 1 --2- 172.21.15.159:0/2092970862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1eac004d40 0x7f1eac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:02.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.603+0000 7f1ec5400640 1 -- 172.21.15.159:0/2092970862 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec80cf220 con 0x7f1eac004d40 2024-04-04T06:14:02.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.603+0000 7f1ec5400640 1 -- 172.21.15.159:0/2092970862 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac00baa0 con 0x7f1eac003f60 2024-04-04T06:14:02.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.603+0000 7f1ec5400640 1 -- 172.21.15.159:0/2092970862 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1eac00bca0 con 0x7f1ec815e860 2024-04-04T06:14:02.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.603+0000 7f1ec6800640 1 --2- 172.21.15.159:0/2092970862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1eac004d40 0x7f1eac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:02.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.604+0000 7f1ebd400640 1 --2- 172.21.15.159:0/2092970862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1eac003f60 0x7f1eac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:02.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.604+0000 7f1ec6800640 1 -- 172.21.15.159:0/2092970862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ec815e860 msgr2=0x7f1eac0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:02.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.604+0000 7f1ec6800640 1 --2- 172.21.15.159:0/2092970862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ec815e860 0x7f1eac0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:02.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.604+0000 7f1ec6800640 1 -- 172.21.15.159:0/2092970862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1eac003f60 msgr2=0x7f1eac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:02.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.604+0000 7f1ec6800640 1 --2- 172.21.15.159:0/2092970862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1eac003f60 0x7f1eac004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:02.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.604+0000 7f1ec6800640 1 -- 172.21.15.159:0/2092970862 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eac003520 con 0x7f1eac004d40 2024-04-04T06:14:02.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.605+0000 7f1ec6800640 1 --2- 172.21.15.159:0/2092970862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1eac004d40 0x7f1eac0051c0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0001720 tx=0x7f1eac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b117c4a05a7fbabd server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:02.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.605+0000 7f1ec5400640 1 -- 172.21.15.159:0/2092970862 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0001370 con 0x7f1eac004d40 2024-04-04T06:14:02.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.605+0000 7f1ec5400640 1 -- 172.21.15.159:0/2092970862 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec00052d0 con 0x7f1eac004d40 2024-04-04T06:14:02.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.605+0000 7f1ec5400640 1 -- 172.21.15.159:0/2092970862 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0003830 con 0x7f1eac004d40 2024-04-04T06:14:02.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.605+0000 7f1ecd200640 1 -- 172.21.15.159:0/2092970862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1eac004d40 msgr2=0x7f1eac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:02.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.605+0000 7f1ecd200640 1 --2- 172.21.15.159:0/2092970862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1eac004d40 0x7f1eac0051c0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0001720 tx=0x7f1eac005830 comp rx=0 tx=0).stop 2024-04-04T06:14:02.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.605+0000 7f1ec6800640 1 -- 172.21.15.159:0/2092970862 reap_dead start 2024-04-04T06:14:02.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.606+0000 7f1ecd200640 1 -- 172.21.15.159:0/2092970862 shutdown_connections 2024-04-04T06:14:02.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.606+0000 7f1ecd200640 1 -- 172.21.15.159:0/2092970862 >> 172.21.15.159:0/2092970862 conn(0x7f1ec809d7d0 msgr2=0x7f1ec808bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:02.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.606+0000 7f1ecd200640 1 -- 172.21.15.159:0/2092970862 shutdown_connections 2024-04-04T06:14:02.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.606+0000 7f1ecd200640 1 -- 172.21.15.159:0/2092970862 wait complete. 2024-04-04T06:14:02.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.607+0000 7f1ecd200640 1 Processor -- start 2024-04-04T06:14:02.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.607+0000 7f1ecd200640 1 -- start start 2024-04-04T06:14:02.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.608+0000 7f1ecd200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1eac004d40 0x7f1ec8150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:02.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.608+0000 7f1ecd200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ec80cb510 0x7f1ec80b9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:02.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.608+0000 7f1ecd200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ec80c8f00 0x7f1ec80c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:02.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.608+0000 7f1ecd200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec80cef30 con 0x7f1ec80c8f00 2024-04-04T06:14:02.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.608+0000 7f1ecd200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec80cedb0 con 0x7f1eac004d40 2024-04-04T06:14:02.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.608+0000 7f1ecd200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec806d4f0 con 0x7f1ec80cb510 2024-04-04T06:14:02.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.609+0000 7f1ec6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ec80c8f00 0x7f1ec80c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:02.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.609+0000 7f1ec6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ec80c8f00 0x7f1ec80c9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60548/0 (socket says 172.21.15.159:60548) 2024-04-04T06:14:02.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.609+0000 7f1ec6800640 1 -- 172.21.15.159:0/412846419 learned_addr learned my addr 172.21.15.159:0/412846419 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:02.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.609+0000 7f1ec5e00640 1 --2- 172.21.15.159:0/412846419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1eac004d40 0x7f1ec8150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.609+0000 7f1ec6800640 1 -- 172.21.15.159:0/412846419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ec80cb510 msgr2=0x7f1ec80b9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.609+0000 7f1ec6800640 1 --2- 172.21.15.159:0/412846419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ec80cb510 0x7f1ec80b9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.609+0000 7f1ec6800640 1 -- 172.21.15.159:0/412846419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1eac004d40 msgr2=0x7f1ec8150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.609+0000 7f1ec6800640 1 --2- 172.21.15.159:0/412846419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1eac004d40 0x7f1ec8150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.609+0000 7f1ec6800640 1 -- 172.21.15.159:0/412846419 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec80cbb80 con 0x7f1ec80c8f00 2024-04-04T06:14:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.609+0000 7f1ec5e00640 1 --2- 172.21.15.159:0/412846419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1eac004d40 0x7f1ec8150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.609+0000 7f1ec6800640 1 --2- 172.21.15.159:0/412846419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ec80c8f00 0x7f1ec80c9380 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f1ec002bcb0 tx=0x7f1ec0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:02.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.610+0000 7f1ebea00640 1 -- 172.21.15.159:0/412846419 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0001370 con 0x7f1ec80c8f00 2024-04-04T06:14:02.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.610+0000 7f1ebea00640 1 -- 172.21.15.159:0/412846419 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec0044070 con 0x7f1ec80c8f00 2024-04-04T06:14:02.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.610+0000 7f1ebea00640 1 -- 172.21.15.159:0/412846419 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0007a30 con 0x7f1ec80c8f00 2024-04-04T06:14:02.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.610+0000 7f1ecd200640 1 -- 172.21.15.159:0/412846419 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec80cbe60 con 0x7f1ec80c8f00 2024-04-04T06:14:02.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.610+0000 7f1ecd200640 1 -- 172.21.15.159:0/412846419 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec80cc420 con 0x7f1ec80c8f00 2024-04-04T06:14:02.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.611+0000 7f1ecd200640 1 -- 172.21.15.159:0/412846419 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec80685c0 con 0x7f1ec80c8f00 2024-04-04T06:14:02.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.612+0000 7f1ebea00640 1 -- 172.21.15.159:0/412846419 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ec003b040 con 0x7f1ec80c8f00 2024-04-04T06:14:02.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.613+0000 7f1ebea00640 1 --2- 172.21.15.159:0/412846419 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1ea406c8f0 0x7f1ea406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:02.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.613+0000 7f1ebea00640 1 -- 172.21.15.159:0/412846419 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5853+0+0 (secure 0 0 0) 0x7f1ec00ad150 con 0x7f1ec80c8f00 2024-04-04T06:14:02.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.613+0000 7f1ec5e00640 1 --2- 172.21.15.159:0/412846419 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1ea406c8f0 0x7f1ea406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:02.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.614+0000 7f1ec5e00640 1 --2- 172.21.15.159:0/412846419 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1ea406c8f0 0x7f1ea406edb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8004b30 tx=0x7f1eb800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:02.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.617+0000 7f1ebea00640 1 -- 172.21.15.159:0/412846419 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1ec007b3c0 con 0x7f1ec80c8f00 2024-04-04T06:14:02.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.871+0000 7f1ecd200640 1 -- 172.21.15.159:0/412846419 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1ec80d1c70 con 0x7f1ec80c8f00 2024-04-04T06:14:02.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.872+0000 7f1ebea00640 1 -- 172.21.15.159:0/412846419 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f1ec007acc0 con 0x7f1ec80c8f00 2024-04-04T06:14:02.866 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:14:02.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.874+0000 7f1ecd200640 1 -- 172.21.15.159:0/412846419 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1ea406c8f0 msgr2=0x7f1ea406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:02.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.874+0000 7f1ecd200640 1 --2- 172.21.15.159:0/412846419 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1ea406c8f0 0x7f1ea406edb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8004b30 tx=0x7f1eb800b040 comp rx=0 tx=0).stop 2024-04-04T06:14:02.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.874+0000 7f1ecd200640 1 -- 172.21.15.159:0/412846419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ec80c8f00 msgr2=0x7f1ec80c9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:02.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.874+0000 7f1ecd200640 1 --2- 172.21.15.159:0/412846419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ec80c8f00 0x7f1ec80c9380 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f1ec002bcb0 tx=0x7f1ec0001120 comp rx=0 tx=0).stop 2024-04-04T06:14:02.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.874+0000 7f1ecd200640 1 -- 172.21.15.159:0/412846419 shutdown_connections 2024-04-04T06:14:02.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.875+0000 7f1ecd200640 1 --2- 172.21.15.159:0/412846419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ec80c8f00 0x7f1ec80c9380 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:02.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.875+0000 7f1ecd200640 1 --2- 172.21.15.159:0/412846419 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ec80cb510 0x7f1ec80b9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:02.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.875+0000 7f1ecd200640 1 --2- 172.21.15.159:0/412846419 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1ea406c8f0 0x7f1ea406edb0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:02.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.875+0000 7f1ecd200640 1 --2- 172.21.15.159:0/412846419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1eac004d40 0x7f1ec8150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:02.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.875+0000 7f1ecd200640 1 -- 172.21.15.159:0/412846419 >> 172.21.15.159:0/412846419 conn(0x7f1ec809d7d0 msgr2=0x7f1ec80b52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:02.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.875+0000 7f1ecd200640 1 -- 172.21.15.159:0/412846419 shutdown_connections 2024-04-04T06:14:02.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:02.875+0000 7f1ecd200640 1 -- 172.21.15.159:0/412846419 wait complete. 2024-04-04T06:14:02.888 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-04T06:14:03.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.104+0000 7fe503200640 1 Processor -- start 2024-04-04T06:14:03.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.104+0000 7fe503200640 1 -- start start 2024-04-04T06:14:03.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.105+0000 7fe503200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe4fc15e4a0 0x7fe4fc0b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:03.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.105+0000 7fe503200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4fc15e880 con 0x7fe4fc0d2310 2024-04-04T06:14:03.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.105+0000 7fe503200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4fc0d28d0 con 0x7fe4fc0cf700 2024-04-04T06:14:03.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.105+0000 7fe503200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe4fc0d34e0 con 0x7fe4fc15e4a0 2024-04-04T06:14:03.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.106+0000 7fe501e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe4fc15e4a0 0x7fe4fc0b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:03.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.106+0000 7fe501400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fe4fc0d2310 0x7fe4fc0ceff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:48350/0 (socket says 172.21.15.159:48350) 2024-04-04T06:14:03.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.106+0000 7fe501400640 1 -- 172.21.15.159:0/762071419 learned_addr learned my addr 172.21.15.159:0/762071419 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:03.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.106+0000 7fe501e00640 1 -- 172.21.15.159:0/762071419 >> v1:172.21.15.178:6789/0 conn(0x7fe4fc0cf700 legacy=0x7fe4fc0b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:03.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.106+0000 7fe501e00640 1 -- 172.21.15.159:0/762071419 >> v1:172.21.15.159:6789/0 conn(0x7fe4fc0d2310 legacy=0x7fe4fc0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:03.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.106+0000 7fe501e00640 1 -- 172.21.15.159:0/762071419 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4fc0d2bb0 con 0x7fe4fc15e4a0 2024-04-04T06:14:03.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.106+0000 7fe500a00640 1 -- 172.21.15.159:0/762071419 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1246715698 0 0) 0x7fe4fc15e880 con 0x7fe4fc0d2310 2024-04-04T06:14:03.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.107+0000 7fe501e00640 1 --2- 172.21.15.159:0/762071419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4fc15e4a0 0x7fe4fc0b9c60 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009d90 tx=0x7fe4f402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d962d9456bc4ce10 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:03.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.107+0000 7fe500a00640 1 -- 172.21.15.159:0/762071419 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4039070 con 0x7fe4fc15e4a0 2024-04-04T06:14:03.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.107+0000 7fe500a00640 1 -- 172.21.15.159:0/762071419 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4f402cbd0 con 0x7fe4fc15e4a0 2024-04-04T06:14:03.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.107+0000 7fe500a00640 1 -- 172.21.15.159:0/762071419 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4034440 con 0x7fe4fc15e4a0 2024-04-04T06:14:03.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.108+0000 7fe503200640 1 -- 172.21.15.159:0/762071419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4fc15e4a0 msgr2=0x7fe4fc0b9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:03.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.108+0000 7fe503200640 1 --2- 172.21.15.159:0/762071419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4fc15e4a0 0x7fe4fc0b9c60 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009d90 tx=0x7fe4f402be20 comp rx=0 tx=0).stop 2024-04-04T06:14:03.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.108+0000 7fe503200640 1 -- 172.21.15.159:0/762071419 shutdown_connections 2024-04-04T06:14:03.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.108+0000 7fe503200640 1 --2- 172.21.15.159:0/762071419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4fc15e4a0 0x7fe4fc0b9c60 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:03.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.108+0000 7fe503200640 1 -- 172.21.15.159:0/762071419 >> 172.21.15.159:0/762071419 conn(0x7fe4fc0676d0 msgr2=0x7fe4fc067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:03.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.108+0000 7fe503200640 1 -- 172.21.15.159:0/762071419 shutdown_connections 2024-04-04T06:14:03.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.108+0000 7fe503200640 1 -- 172.21.15.159:0/762071419 wait complete. 2024-04-04T06:14:03.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.110+0000 7fe503200640 1 Processor -- start 2024-04-04T06:14:03.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.110+0000 7fe503200640 1 -- start start 2024-04-04T06:14:03.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.111+0000 7fe503200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4fc0cf700 0x7fe4fc0b9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:03.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.111+0000 7fe503200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4fc0d2310 0x7fe4fc14f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:03.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.111+0000 7fe503200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4fc15e4a0 0x7fe4fc0b8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:03.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.111+0000 7fe503200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc0d3970 con 0x7fe4fc0cf700 2024-04-04T06:14:03.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.111+0000 7fe503200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc0d3170 con 0x7fe4fc15e4a0 2024-04-04T06:14:03.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.111+0000 7fe503200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe4fc0ba1a0 con 0x7fe4fc0d2310 2024-04-04T06:14:03.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.111+0000 7fe501e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4fc0cf700 0x7fe4fc0b9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:03.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.111+0000 7fe501e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4fc0cf700 0x7fe4fc0b9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60570/0 (socket says 172.21.15.159:60570) 2024-04-04T06:14:03.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.111+0000 7fe501e00640 1 -- 172.21.15.159:0/327835633 learned_addr learned my addr 172.21.15.159:0/327835633 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:03.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.112+0000 7fe501e00640 1 -- 172.21.15.159:0/327835633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4fc0d2310 msgr2=0x7fe4fc14f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:03.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.112+0000 7fe501e00640 1 --2- 172.21.15.159:0/327835633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4fc0d2310 0x7fe4fc14f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:03.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.112+0000 7fe501e00640 1 -- 172.21.15.159:0/327835633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4fc15e4a0 msgr2=0x7fe4fc0b8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:03.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.112+0000 7fe501e00640 1 --2- 172.21.15.159:0/327835633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4fc15e4a0 0x7fe4fc0b8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:03.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.112+0000 7fe501e00640 1 -- 172.21.15.159:0/327835633 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4fc1f7570 con 0x7fe4fc0cf700 2024-04-04T06:14:03.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.112+0000 7fe501e00640 1 --2- 172.21.15.159:0/327835633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4fc0cf700 0x7fe4fc0b9270 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009d90 tx=0x7fe4f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:03.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.112+0000 7fe4f2000640 1 -- 172.21.15.159:0/327835633 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4044020 con 0x7fe4fc0cf700 2024-04-04T06:14:03.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.112+0000 7fe4f2000640 1 -- 172.21.15.159:0/327835633 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4f403f670 con 0x7fe4fc0cf700 2024-04-04T06:14:03.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.112+0000 7fe4f2000640 1 -- 172.21.15.159:0/327835633 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4039040 con 0x7fe4fc0cf700 2024-04-04T06:14:03.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.112+0000 7fe503200640 1 -- 172.21.15.159:0/327835633 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4fc1f7810 con 0x7fe4fc0cf700 2024-04-04T06:14:03.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.112+0000 7fe503200640 1 -- 172.21.15.159:0/327835633 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4fc1f7dd0 con 0x7fe4fc0cf700 2024-04-04T06:14:03.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.114+0000 7fe503200640 1 -- 172.21.15.159:0/327835633 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4fc075940 con 0x7fe4fc0cf700 2024-04-04T06:14:03.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.114+0000 7fe4f2000640 1 -- 172.21.15.159:0/327835633 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4f4005d10 con 0x7fe4fc0cf700 2024-04-04T06:14:03.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.115+0000 7fe4f2000640 1 --2- 172.21.15.159:0/327835633 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe4e006c8f0 0x7fe4e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:03.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.115+0000 7fe4f2000640 1 -- 172.21.15.159:0/327835633 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5853+0+0 (secure 0 0 0) 0x7fe4f40ae7b0 con 0x7fe4fc0cf700 2024-04-04T06:14:03.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.116+0000 7fe501400640 1 --2- 172.21.15.159:0/327835633 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe4e006c8f0 0x7fe4e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:03.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.117+0000 7fe501400640 1 --2- 172.21.15.159:0/327835633 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe4e006c8f0 0x7fe4e006edb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0011e0 tx=0x7fe4ec007950 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:03.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.120+0000 7fe4f2000640 1 -- 172.21.15.159:0/327835633 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe4f404e070 con 0x7fe4fc0cf700 2024-04-04T06:14:03.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.334+0000 7fe503200640 1 -- 172.21.15.159:0/327835633 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe4fc0b5950 con 0x7fe4fc0cf700 2024-04-04T06:14:03.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.335+0000 7fe4f2000640 1 -- 172.21.15.159:0/327835633 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v88) ==== 80+0+124 (secure 0 0 0) 0x7fe4f407cbe0 con 0x7fe4fc0cf700 2024-04-04T06:14:03.329 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6835/3484779950 2024-04-05T06:12:49.205338+0000 2024-04-04T06:14:03.329 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6833/3484779950 2024-04-05T06:12:49.205338+0000 2024-04-04T06:14:03.329 INFO:teuthology.orchestra.run.smithi159.stderr:listed 2 entries 2024-04-04T06:14:03.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.338+0000 7fe503200640 1 -- 172.21.15.159:0/327835633 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe4e006c8f0 msgr2=0x7fe4e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:03.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.338+0000 7fe503200640 1 --2- 172.21.15.159:0/327835633 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe4e006c8f0 0x7fe4e006edb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0011e0 tx=0x7fe4ec007950 comp rx=0 tx=0).stop 2024-04-04T06:14:03.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.338+0000 7fe503200640 1 -- 172.21.15.159:0/327835633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4fc0cf700 msgr2=0x7fe4fc0b9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:03.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.338+0000 7fe503200640 1 --2- 172.21.15.159:0/327835633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4fc0cf700 0x7fe4fc0b9270 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009d90 tx=0x7fe4f4004a40 comp rx=0 tx=0).stop 2024-04-04T06:14:03.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.338+0000 7fe503200640 1 -- 172.21.15.159:0/327835633 shutdown_connections 2024-04-04T06:14:03.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.338+0000 7fe503200640 1 --2- 172.21.15.159:0/327835633 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe4e006c8f0 0x7fe4e006edb0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:03.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.338+0000 7fe503200640 1 --2- 172.21.15.159:0/327835633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4fc15e4a0 0x7fe4fc0b8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:03.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.338+0000 7fe503200640 1 --2- 172.21.15.159:0/327835633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4fc0d2310 0x7fe4fc14f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:03.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.338+0000 7fe503200640 1 --2- 172.21.15.159:0/327835633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4fc0cf700 0x7fe4fc0b9270 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:03.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.338+0000 7fe503200640 1 -- 172.21.15.159:0/327835633 >> 172.21.15.159:0/327835633 conn(0x7fe4fc0676d0 msgr2=0x7fe4fc15b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:03.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.338+0000 7fe503200640 1 -- 172.21.15.159:0/327835633 shutdown_connections 2024-04-04T06:14:03.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.338+0000 7fe503200640 1 -- 172.21.15.159:0/327835633 wait complete. 2024-04-04T06:14:03.351 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-04T06:14:03.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:14:03.352 DEBUG:teuthology.orchestra.run.smithi159:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-04T06:14:03.443 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:14:03.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:14:03.443 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-04T06:14:03.480 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:14:03.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.719+0000 7f8d5fe00640 1 Processor -- start 2024-04-04T06:14:03.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.719+0000 7f8d5fe00640 1 -- start start 2024-04-04T06:14:03.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.720+0000 7f8d5fe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d6015e4a0 0x7f8d600b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:03.715 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.720+0000 7f8d5fe00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8d600d2310 0x7f8d600ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:03.715 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.720+0000 7f8d5fe00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d6015e880 con 0x7f8d600cf530 2024-04-04T06:14:03.715 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.720+0000 7f8d5fe00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f8d600d34e0 con 0x7f8d600d2310 2024-04-04T06:14:03.715 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.720+0000 7f8d5fe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8d600d28d0 con 0x7f8d6015e4a0 2024-04-04T06:14:03.715 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.721+0000 7f8d5f400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f8d600cf530 0x7f8d600d3c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:48354/0 (socket says 172.21.15.159:48354) 2024-04-04T06:14:03.715 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.721+0000 7f8d5f400640 1 -- 172.21.15.159:0/691408014 learned_addr learned my addr 172.21.15.159:0/691408014 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:03.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.721+0000 7f8d5ea00640 1 --2- 172.21.15.159:0/691408014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d6015e4a0 0x7f8d600b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:03.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.721+0000 7f8d5e000640 1 --2- 172.21.15.159:0/691408014 >> v2:172.21.15.178:3300/0 conn(0x7f8d600d2310 0x7f8d600ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:03.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.721+0000 7f8d5d600640 1 -- 172.21.15.159:0/691408014 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3438736764 0 0) 0x7f8d6015e880 con 0x7f8d600cf530 2024-04-04T06:14:03.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.722+0000 7f8d5d600640 1 -- 172.21.15.159:0/691408014 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d44003520 con 0x7f8d600cf530 2024-04-04T06:14:03.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.722+0000 7f8d5d600640 1 -- 172.21.15.159:0/691408014 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 187860540 0 0) 0x7f8d44003520 con 0x7f8d600cf530 2024-04-04T06:14:03.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.722+0000 7f8d5d600640 1 -- 172.21.15.159:0/691408014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d6015e4a0 msgr2=0x7f8d600b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:03.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.722+0000 7f8d5d600640 1 --2- 172.21.15.159:0/691408014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d6015e4a0 0x7f8d600b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:03.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.722+0000 7f8d5d600640 1 -- 172.21.15.159:0/691408014 >> v2:172.21.15.178:3300/0 conn(0x7f8d600d2310 msgr2=0x7f8d600ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:03.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.722+0000 7f8d5d600640 1 --2- 172.21.15.159:0/691408014 >> v2:172.21.15.178:3300/0 conn(0x7f8d600d2310 0x7f8d600ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:03.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.722+0000 7f8d5d600640 1 -- 172.21.15.159:0/691408014 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d600d2bb0 con 0x7f8d600cf530 2024-04-04T06:14:03.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.722+0000 7f8d5d600640 1 -- 172.21.15.159:0/691408014 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f8d580019f0 con 0x7f8d600cf530 2024-04-04T06:14:03.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.722+0000 7f8d5d600640 1 -- 172.21.15.159:0/691408014 >> v1:172.21.15.159:6789/0 conn(0x7f8d600cf530 legacy=0x7f8d600d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:03.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.722+0000 7f8d5d600640 1 --2- 172.21.15.159:0/691408014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d440039b0 0x7f8d44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:03.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.722+0000 7f8d5d600640 1 --2- 172.21.15.159:0/691408014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d44004800 0x7f8d44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:03.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.722+0000 7f8d5d600640 1 --2- 172.21.15.159:0/691408014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d6015e4a0 0x7f8d440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:03.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.722+0000 7f8d5d600640 1 -- 172.21.15.159:0/691408014 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8d600d34e0 con 0x7f8d440039b0 2024-04-04T06:14:03.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.722+0000 7f8d5d600640 1 -- 172.21.15.159:0/691408014 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8d600d28d0 con 0x7f8d44004800 2024-04-04T06:14:03.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.722+0000 7f8d5d600640 1 -- 172.21.15.159:0/691408014 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8d4400b9c0 con 0x7f8d6015e4a0 2024-04-04T06:14:03.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.722+0000 7f8d5ea00640 1 --2- 172.21.15.159:0/691408014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d440039b0 0x7f8d44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:03.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.723+0000 7f8d5e000640 1 --2- 172.21.15.159:0/691408014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d44004800 0x7f8d44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:03.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.723+0000 7f8d5f400640 1 --2- 172.21.15.159:0/691408014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d6015e4a0 0x7f8d440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:03.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.723+0000 7f8d5ea00640 1 -- 172.21.15.159:0/691408014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d6015e4a0 msgr2=0x7f8d440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:03.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.723+0000 7f8d5ea00640 1 --2- 172.21.15.159:0/691408014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d6015e4a0 0x7f8d440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:03.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.723+0000 7f8d5ea00640 1 -- 172.21.15.159:0/691408014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d44004800 msgr2=0x7f8d44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:03.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.723+0000 7f8d5ea00640 1 --2- 172.21.15.159:0/691408014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d44004800 0x7f8d44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:03.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.723+0000 7f8d5ea00640 1 -- 172.21.15.159:0/691408014 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d44003520 con 0x7f8d440039b0 2024-04-04T06:14:03.720 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.724+0000 7f8d5ea00640 1 --2- 172.21.15.159:0/691408014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d440039b0 0x7f8d44003dd0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f8d50009ac0 tx=0x7f8d44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cd099971322d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:03.720 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.725+0000 7f8d5d600640 1 -- 172.21.15.159:0/691408014 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d50039070 con 0x7f8d440039b0 2024-04-04T06:14:03.720 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.725+0000 7f8d5d600640 1 -- 172.21.15.159:0/691408014 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d5002c6b0 con 0x7f8d440039b0 2024-04-04T06:14:03.720 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.725+0000 7f8d5d600640 1 -- 172.21.15.159:0/691408014 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d50009690 con 0x7f8d440039b0 2024-04-04T06:14:03.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.725+0000 7f8d5fe00640 1 -- 172.21.15.159:0/691408014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d440039b0 msgr2=0x7f8d44003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:03.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.725+0000 7f8d5fe00640 1 --2- 172.21.15.159:0/691408014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d440039b0 0x7f8d44003dd0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f8d50009ac0 tx=0x7f8d44005830 comp rx=0 tx=0).stop 2024-04-04T06:14:03.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.725+0000 7f8d5f400640 1 -- 172.21.15.159:0/691408014 reap_dead start 2024-04-04T06:14:03.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.725+0000 7f8d5fe00640 1 -- 172.21.15.159:0/691408014 shutdown_connections 2024-04-04T06:14:03.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.725+0000 7f8d5fe00640 1 -- 172.21.15.159:0/691408014 >> 172.21.15.159:0/691408014 conn(0x7f8d600676d0 msgr2=0x7f8d60067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:03.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.725+0000 7f8d5fe00640 1 -- 172.21.15.159:0/691408014 shutdown_connections 2024-04-04T06:14:03.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.725+0000 7f8d5fe00640 1 -- 172.21.15.159:0/691408014 wait complete. 2024-04-04T06:14:03.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.727+0000 7f8d5fe00640 1 Processor -- start 2024-04-04T06:14:03.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.727+0000 7f8d5fe00640 1 -- start start 2024-04-04T06:14:03.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.728+0000 7f8d5fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d440039b0 0x7f8d601609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:03.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.728+0000 7f8d5fe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d60160f10 0x7f8d601416d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:03.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.728+0000 7f8d5fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d60141c10 0x7f8d60151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:03.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.728+0000 7f8d5fe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8d600d3970 con 0x7f8d440039b0 2024-04-04T06:14:03.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.728+0000 7f8d5fe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8d600d3170 con 0x7f8d60141c10 2024-04-04T06:14:03.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.728+0000 7f8d5fe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8d600b83f0 con 0x7f8d60160f10 2024-04-04T06:14:03.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.728+0000 7f8d5ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d440039b0 0x7f8d601609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:03.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.728+0000 7f8d5ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d440039b0 0x7f8d601609d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60576/0 (socket says 172.21.15.159:60576) 2024-04-04T06:14:03.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.728+0000 7f8d5ea00640 1 -- 172.21.15.159:0/1354299456 learned_addr learned my addr 172.21.15.159:0/1354299456 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:03.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.729+0000 7f8d5ea00640 1 -- 172.21.15.159:0/1354299456 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d60160f10 msgr2=0x7f8d601416d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:03.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.729+0000 7f8d5ea00640 1 --2- 172.21.15.159:0/1354299456 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d60160f10 0x7f8d601416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:03.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.729+0000 7f8d5ea00640 1 -- 172.21.15.159:0/1354299456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d60141c10 msgr2=0x7f8d60151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:03.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.729+0000 7f8d5ea00640 1 --2- 172.21.15.159:0/1354299456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d60141c10 0x7f8d60151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:03.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.729+0000 7f8d5ea00640 1 -- 172.21.15.159:0/1354299456 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6015f970 con 0x7f8d440039b0 2024-04-04T06:14:03.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.729+0000 7f8d5ea00640 1 --2- 172.21.15.159:0/1354299456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d440039b0 0x7f8d601609d0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f8d50000c00 tx=0x7f8d50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:03.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.729+0000 7f8d57400640 1 -- 172.21.15.159:0/1354299456 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d50042020 con 0x7f8d440039b0 2024-04-04T06:14:03.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.730+0000 7f8d57400640 1 -- 172.21.15.159:0/1354299456 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d5003a040 con 0x7f8d440039b0 2024-04-04T06:14:03.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.730+0000 7f8d57400640 1 -- 172.21.15.159:0/1354299456 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d50039070 con 0x7f8d440039b0 2024-04-04T06:14:03.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.730+0000 7f8d5fe00640 1 -- 172.21.15.159:0/1354299456 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d6015fc10 con 0x7f8d440039b0 2024-04-04T06:14:03.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.730+0000 7f8d5fe00640 1 -- 172.21.15.159:0/1354299456 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d601601d0 con 0x7f8d440039b0 2024-04-04T06:14:03.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.731+0000 7f8d5fe00640 1 -- 172.21.15.159:0/1354299456 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d60075940 con 0x7f8d440039b0 2024-04-04T06:14:03.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.733+0000 7f8d57400640 1 -- 172.21.15.159:0/1354299456 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8d50045020 con 0x7f8d440039b0 2024-04-04T06:14:03.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.734+0000 7f8d57400640 1 --2- 172.21.15.159:0/1354299456 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8d4006c8f0 0x7f8d4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:03.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.735+0000 7f8d57400640 1 -- 172.21.15.159:0/1354299456 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5853+0+0 (secure 0 0 0) 0x7f8d500ad120 con 0x7f8d440039b0 2024-04-04T06:14:03.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.735+0000 7f8d5e000640 1 --2- 172.21.15.159:0/1354299456 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8d4006c8f0 0x7f8d4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:03.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.736+0000 7f8d5e000640 1 --2- 172.21.15.159:0/1354299456 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8d4006c8f0 0x7f8d4006edb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c006b10 tx=0x7f8d4c006aa0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:03.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.739+0000 7f8d57400640 1 -- 172.21.15.159:0/1354299456 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8d5004c070 con 0x7f8d440039b0 2024-04-04T06:14:03.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.983+0000 7f8d5fe00640 1 -- 172.21.15.159:0/1354299456 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d60158420 con 0x7f8d440039b0 2024-04-04T06:14:03.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.984+0000 7f8d57400640 1 -- 172.21.15.159:0/1354299456 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5196 (secure 0 0 0) 0x7f8d5007b420 con 0x7f8d440039b0 2024-04-04T06:14:03.978 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:14:03.978 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1825651864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1825651864},{"type":"v1","addr":"172.21.15.178:6836","nonce":1825651864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1824578712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1824578712},{"type":"v1","addr":"172.21.15.178:6837","nonce":1824578712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6834/3377498298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3377498298},{"type":"v1","addr":"172.21.15.159:6834","nonce":3377498298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:12:35.880454+0000","modified":"2024-04-04T06:13:00.228456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7512":{"gid":7512,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":6,"addr":"172.21.15.159:6835/3796123128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":3796123128},{"type":"v1","addr":"172.21.15.159:6835","nonce":3796123128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7512,"qdb_cluster":[7512]},"id":6}]} 2024-04-04T06:14:03.978 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 69 2024-04-04T06:14:03.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.986+0000 7f8d5fe00640 1 -- 172.21.15.159:0/1354299456 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8d4006c8f0 msgr2=0x7f8d4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:03.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.986+0000 7f8d5fe00640 1 --2- 172.21.15.159:0/1354299456 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8d4006c8f0 0x7f8d4006edb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c006b10 tx=0x7f8d4c006aa0 comp rx=0 tx=0).stop 2024-04-04T06:14:03.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.987+0000 7f8d5fe00640 1 -- 172.21.15.159:0/1354299456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d440039b0 msgr2=0x7f8d601609d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:03.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.987+0000 7f8d5fe00640 1 --2- 172.21.15.159:0/1354299456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d440039b0 0x7f8d601609d0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f8d50000c00 tx=0x7f8d50004730 comp rx=0 tx=0).stop 2024-04-04T06:14:03.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.987+0000 7f8d5fe00640 1 -- 172.21.15.159:0/1354299456 shutdown_connections 2024-04-04T06:14:03.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.987+0000 7f8d5fe00640 1 --2- 172.21.15.159:0/1354299456 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8d4006c8f0 0x7f8d4006edb0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:03.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.987+0000 7f8d5fe00640 1 --2- 172.21.15.159:0/1354299456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d60141c10 0x7f8d60151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:03.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.987+0000 7f8d5fe00640 1 --2- 172.21.15.159:0/1354299456 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d60160f10 0x7f8d601416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:03.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.987+0000 7f8d5fe00640 1 --2- 172.21.15.159:0/1354299456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d440039b0 0x7f8d601609d0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:03.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.987+0000 7f8d5fe00640 1 -- 172.21.15.159:0/1354299456 >> 172.21.15.159:0/1354299456 conn(0x7f8d600676d0 msgr2=0x7f8d6015bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:03.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.987+0000 7f8d5fe00640 1 -- 172.21.15.159:0/1354299456 shutdown_connections 2024-04-04T06:14:03.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:03.987+0000 7f8d5fe00640 1 -- 172.21.15.159:0/1354299456 wait complete. 2024-04-04T06:14:04.001 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-04T06:14:04.287 INFO:teuthology.orchestra.run.smithi159.stdout:{"message":"","return_code":0} 2024-04-04T06:14:04.287 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-04T06:14:04.287 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-04T06:14:04.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.490+0000 7f24cc400640 1 Processor -- start 2024-04-04T06:14:04.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.490+0000 7f24cc400640 1 -- start start 2024-04-04T06:14:04.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.492+0000 7f24cc400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24c4155550 0x7f24c4155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.492+0000 7f24cc400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f24c415f800 0x7f24c40d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.492+0000 7f24cc400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24c40b40b0 con 0x7f24c40d07c0 2024-04-04T06:14:04.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.492+0000 7f24cc400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f24c40d0c30 con 0x7f24c415f800 2024-04-04T06:14:04.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.492+0000 7f24cc400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f24c40b4390 con 0x7f24c4155550 2024-04-04T06:14:04.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.492+0000 7f24c9000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f24c415f800 0x7f24c40d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:04.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.492+0000 7f24c9000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f24c415f800 0x7f24c40d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33546/0 (socket says 172.21.15.159:33546) 2024-04-04T06:14:04.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.492+0000 7f24c9000640 1 -- 172.21.15.159:0/628366795 learned_addr learned my addr 172.21.15.159:0/628366795 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:04.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.492+0000 7f24c9000640 1 -- 172.21.15.159:0/628366795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24c4155550 msgr2=0x7f24c4155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:04.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.492+0000 7f24c9a00640 1 --2- 172.21.15.159:0/628366795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24c4155550 0x7f24c4155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:04.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.492+0000 7f24c9000640 1 --2- 172.21.15.159:0/628366795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24c4155550 0x7f24c4155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.492+0000 7f24c9000640 1 -- 172.21.15.159:0/628366795 >> v1:172.21.15.178:6789/0 conn(0x7f24c40d07c0 legacy=0x7f24c40aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:04.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.492+0000 7f24c9000640 1 -- 172.21.15.159:0/628366795 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c40b4670 con 0x7f24c415f800 2024-04-04T06:14:04.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.494+0000 7f24c9000640 1 --2- 172.21.15.159:0/628366795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24c415f800 0x7f24c40d0280 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009870 tx=0x7f24b002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a63a1003c3038c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:04.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.494+0000 7f24c3e00640 1 -- 172.21.15.159:0/628366795 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0035020 con 0x7f24c415f800 2024-04-04T06:14:04.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.494+0000 7f24c3e00640 1 -- 172.21.15.159:0/628366795 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24b0031640 con 0x7f24c415f800 2024-04-04T06:14:04.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.494+0000 7f24c3e00640 1 -- 172.21.15.159:0/628366795 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b002d040 con 0x7f24c415f800 2024-04-04T06:14:04.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.494+0000 7f24cc400640 1 -- 172.21.15.159:0/628366795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24c415f800 msgr2=0x7f24c40d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:04.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.494+0000 7f24cc400640 1 --2- 172.21.15.159:0/628366795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24c415f800 0x7f24c40d0280 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009870 tx=0x7f24b002b8f0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.495+0000 7f24cc400640 1 -- 172.21.15.159:0/628366795 shutdown_connections 2024-04-04T06:14:04.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.495+0000 7f24cc400640 1 --2- 172.21.15.159:0/628366795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24c415f800 0x7f24c40d0280 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.495+0000 7f24cc400640 1 --2- 172.21.15.159:0/628366795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24c4155550 0x7f24c4155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.495+0000 7f24cc400640 1 -- 172.21.15.159:0/628366795 >> 172.21.15.159:0/628366795 conn(0x7f24c406b750 msgr2=0x7f24c406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:04.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.495+0000 7f24cc400640 1 -- 172.21.15.159:0/628366795 shutdown_connections 2024-04-04T06:14:04.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.495+0000 7f24cc400640 1 -- 172.21.15.159:0/628366795 wait complete. 2024-04-04T06:14:04.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.497+0000 7f24cc400640 1 Processor -- start 2024-04-04T06:14:04.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.498+0000 7f24cc400640 1 -- start start 2024-04-04T06:14:04.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.499+0000 7f24cc400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24c40d07c0 0x7f24c415cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.499+0000 7f24cc400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24c4155550 0x7f24c41f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.499+0000 7f24cc400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24c415f800 0x7f24c409dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.499+0000 7f24cc400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f24c415fc80 con 0x7f24c40d07c0 2024-04-04T06:14:04.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.499+0000 7f24cc400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f24c40b4c30 con 0x7f24c4155550 2024-04-04T06:14:04.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.499+0000 7f24cc400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f24c415ba20 con 0x7f24c415f800 2024-04-04T06:14:04.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.500+0000 7f24c9a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24c40d07c0 0x7f24c415cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:04.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.500+0000 7f24c9a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24c40d07c0 0x7f24c415cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33550/0 (socket says 172.21.15.159:33550) 2024-04-04T06:14:04.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.500+0000 7f24c9a00640 1 -- 172.21.15.159:0/1273993090 learned_addr learned my addr 172.21.15.159:0/1273993090 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:04.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.500+0000 7f24c9000640 1 --2- 172.21.15.159:0/1273993090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24c4155550 0x7f24c41f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:04.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.500+0000 7f24ca400640 1 --2- 172.21.15.159:0/1273993090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24c415f800 0x7f24c409dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:04.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.500+0000 7f24c9a00640 1 -- 172.21.15.159:0/1273993090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24c415f800 msgr2=0x7f24c409dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:04.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.500+0000 7f24c9a00640 1 --2- 172.21.15.159:0/1273993090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24c415f800 0x7f24c409dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.500+0000 7f24c9a00640 1 -- 172.21.15.159:0/1273993090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24c4155550 msgr2=0x7f24c41f23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:04.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.500+0000 7f24c9a00640 1 --2- 172.21.15.159:0/1273993090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24c4155550 0x7f24c41f23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.500+0000 7f24c9a00640 1 -- 172.21.15.159:0/1273993090 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c4151470 con 0x7f24c40d07c0 2024-04-04T06:14:04.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.500+0000 7f24c9a00640 1 --2- 172.21.15.159:0/1273993090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24c40d07c0 0x7f24c415cb00 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f24b800c7f0 tx=0x7f24b800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:04.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.501+0000 7f24c1600640 1 -- 172.21.15.159:0/1273993090 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8015070 con 0x7f24c40d07c0 2024-04-04T06:14:04.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.501+0000 7f24c1600640 1 -- 172.21.15.159:0/1273993090 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24b8004240 con 0x7f24c40d07c0 2024-04-04T06:14:04.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.501+0000 7f24c1600640 1 -- 172.21.15.159:0/1273993090 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8007620 con 0x7f24c40d07c0 2024-04-04T06:14:04.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.501+0000 7f24cc400640 1 -- 172.21.15.159:0/1273993090 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c4151750 con 0x7f24c40d07c0 2024-04-04T06:14:04.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.501+0000 7f24cc400640 1 -- 172.21.15.159:0/1273993090 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c4151d10 con 0x7f24c40d07c0 2024-04-04T06:14:04.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.502+0000 7f24cc400640 1 -- 172.21.15.159:0/1273993090 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24c40685c0 con 0x7f24c40d07c0 2024-04-04T06:14:04.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.504+0000 7f24c1600640 1 -- 172.21.15.159:0/1273993090 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24b80079d0 con 0x7f24c40d07c0 2024-04-04T06:14:04.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.505+0000 7f24c1600640 1 --2- 172.21.15.159:0/1273993090 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f24a406c8f0 0x7f24a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.505+0000 7f24c1600640 1 -- 172.21.15.159:0/1273993090 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5853+0+0 (secure 0 0 0) 0x7f24b808bca0 con 0x7f24c40d07c0 2024-04-04T06:14:04.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.506+0000 7f24c9000640 1 --2- 172.21.15.159:0/1273993090 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f24a406c8f0 0x7f24a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:04.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.507+0000 7f24c9000640 1 --2- 172.21.15.159:0/1273993090 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f24a406c8f0 0x7f24a406edb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f24b0036000 tx=0x7f24b003c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:04.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.510+0000 7f24c1600640 1 -- 172.21.15.159:0/1273993090 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f24b805a040 con 0x7f24c40d07c0 2024-04-04T06:14:04.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.755+0000 7f24cc400640 1 -- 172.21.15.159:0/1273993090 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f24c4166f70 con 0x7f24c40d07c0 2024-04-04T06:14:04.749 INFO:tasks.ceph.mon.a.smithi159.stderr:2024-04-04T06:14:04.756+0000 7f1995c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-04T06:14:04.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.759+0000 7f24c1600640 1 -- 172.21.15.159:0/1273993090 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v70) ==== 161+0+0 (secure 0 0 0) 0x7f24b8059940 con 0x7f24c40d07c0 2024-04-04T06:14:04.753 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-04T06:14:04.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.763+0000 7f24cc400640 1 -- 172.21.15.159:0/1273993090 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f24a406c8f0 msgr2=0x7f24a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:04.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.763+0000 7f24cc400640 1 --2- 172.21.15.159:0/1273993090 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f24a406c8f0 0x7f24a406edb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f24b0036000 tx=0x7f24b003c040 comp rx=0 tx=0).stop 2024-04-04T06:14:04.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.763+0000 7f24cc400640 1 -- 172.21.15.159:0/1273993090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24c40d07c0 msgr2=0x7f24c415cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:04.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.763+0000 7f24cc400640 1 --2- 172.21.15.159:0/1273993090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24c40d07c0 0x7f24c415cb00 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f24b800c7f0 tx=0x7f24b800ccb0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.763+0000 7f24cc400640 1 -- 172.21.15.159:0/1273993090 shutdown_connections 2024-04-04T06:14:04.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.763+0000 7f24cc400640 1 --2- 172.21.15.159:0/1273993090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24c415f800 0x7f24c409dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.763+0000 7f24cc400640 1 --2- 172.21.15.159:0/1273993090 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f24a406c8f0 0x7f24a406edb0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.763+0000 7f24cc400640 1 --2- 172.21.15.159:0/1273993090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24c4155550 0x7f24c41f23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.763+0000 7f24cc400640 1 --2- 172.21.15.159:0/1273993090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24c40d07c0 0x7f24c415cb00 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.763+0000 7f24cc400640 1 -- 172.21.15.159:0/1273993090 >> 172.21.15.159:0/1273993090 conn(0x7f24c406b750 msgr2=0x7f24c415d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:04.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.764+0000 7f24cc400640 1 -- 172.21.15.159:0/1273993090 shutdown_connections 2024-04-04T06:14:04.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.764+0000 7f24cc400640 1 -- 172.21.15.159:0/1273993090 wait complete. 2024-04-04T06:14:04.772 INFO:tasks.ceph.mds.a.smithi159.stderr: -4807> 2024-04-04T06:12:55.226+0000 7fb388a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:14:04.773 INFO:tasks.ceph.mds.a.smithi159.stderr: -4381> 2024-04-04T06:12:57.114+0000 7fb383a00640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-04T06:14:04.774 INFO:tasks.ceph.mds.a.smithi159.stderr: -4179> 2024-04-04T06:12:57.381+0000 7fb383a00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-04-04T06:12:45.034063+0000 1=1+0) n(v0 rc2024-04-04T06:12:45.181060+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x55728e21c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-04T06:12:45.034063+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-04T06:12:45.181060+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-04T06:14:04.776 DEBUG:teuthology.orchestra.run.smithi159:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-04T06:14:04.782 INFO:tasks.ceph.mds.a.smithi159.stderr: -2304> 2024-04-04T06:13:29.409+0000 7fb383a00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-04-04T06:12:45.034063+0000 1=1+0)->f(v1 m2024-04-04T06:12:45.034063+0000) n(v0 rc2024-04-04T06:12:45.181060+0000 2=1+1)->n(v0 rc2024-04-04T06:12:45.181060+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55728e21c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-04T06:12:45.034063+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-04T06:12:45.181060+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-04T06:14:04.841 INFO:tasks.ceph.mds.a.smithi159.stdout:starting mds.a at 2024-04-04T06:14:04.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.869+0000 7f6b0c23e740 1 -- start start 2024-04-04T06:14:04.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.869+0000 7f6b0c23e740 1 Processor -- start 2024-04-04T06:14:04.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.870+0000 7f6b0c23e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc18102400 0x55bc17509600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.870+0000 7f6b0c23e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc18102800 0x55bc17508b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.870+0000 7f6b0c23e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc18103c00 0x55bc17509b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.870+0000 7f6b0c23e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bc1817e480 con 0x55bc18102800 2024-04-04T06:14:04.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.870+0000 7f6b0c23e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bc1817e600 con 0x55bc18103c00 2024-04-04T06:14:04.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.870+0000 7f6b0c23e740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bc1817e780 con 0x55bc18102400 2024-04-04T06:14:04.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.870+0000 7f6b09200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc18102800 0x55bc17508b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:04.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.870+0000 7f6b09200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc18102800 0x55bc17508b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33586/0 (socket says 172.21.15.159:33586) 2024-04-04T06:14:04.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.870+0000 7f6b09200640 1 -- 172.21.15.159:0/3289886741 learned_addr learned my addr 172.21.15.159:0/3289886741 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:04.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.871+0000 7f6b09c00640 1 --2- 172.21.15.159:0/3289886741 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc18102400 0x55bc17509600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:04.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.871+0000 7f6b0a600640 1 --2- 172.21.15.159:0/3289886741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc18103c00 0x55bc17509b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:04.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.871+0000 7f6b09200640 1 -- 172.21.15.159:0/3289886741 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc18102400 msgr2=0x55bc17509600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:04.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.871+0000 7f6b09200640 1 --2- 172.21.15.159:0/3289886741 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc18102400 0x55bc17509600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.871+0000 7f6b09200640 1 -- 172.21.15.159:0/3289886741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc18103c00 msgr2=0x55bc17509b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:04.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.871+0000 7f6b09200640 1 --2- 172.21.15.159:0/3289886741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc18103c00 0x55bc17509b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.871+0000 7f6b09200640 1 -- 172.21.15.159:0/3289886741 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc173bd680 con 0x55bc18102800 2024-04-04T06:14:04.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.871+0000 7f6b09c00640 1 --2- 172.21.15.159:0/3289886741 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc18102400 0x55bc17509600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:04.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.871+0000 7f6b0a600640 1 --2- 172.21.15.159:0/3289886741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc18103c00 0x55bc17509b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:04.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.871+0000 7f6b09200640 1 --2- 172.21.15.159:0/3289886741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc18102800 0x55bc17508b00 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x55bc181b58f0 tx=0x55bc1812a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e01624281d284c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:04.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.871+0000 7f6b06a00640 1 -- 172.21.15.159:0/3289886741 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bc173b6d00 con 0x55bc18102800 2024-04-04T06:14:04.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.871+0000 7f6b06a00640 1 -- 172.21.15.159:0/3289886741 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55bc173da1c0 con 0x55bc18102800 2024-04-04T06:14:04.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.871+0000 7f6b06a00640 1 -- 172.21.15.159:0/3289886741 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bc173b6680 con 0x55bc18102800 2024-04-04T06:14:04.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.872+0000 7f6b0c23e740 1 -- 172.21.15.159:0/3289886741 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc173bd2c0 con 0x55bc18102800 2024-04-04T06:14:04.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.872+0000 7f6b0c23e740 1 -- 172.21.15.159:0/3289886741 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55bc173bcb40 con 0x55bc18102800 2024-04-04T06:14:04.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.872+0000 7f6b0c23e740 4 waiting for MDS map... 2024-04-04T06:14:04.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.872+0000 7f6b06a00640 1 -- 172.21.15.159:0/3289886741 <== mon.0 v2:172.21.15.159:3300/0 4 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x55bc1825c600 con 0x55bc18102800 2024-04-04T06:14:04.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.872+0000 7f6b06a00640 1 -- 172.21.15.159:0/3289886741 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 70) ==== 2885+0+0 (secure 0 0 0) 0x55bc173e9180 con 0x55bc18102800 2024-04-04T06:14:04.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.872+0000 7f6b0c23e740 4 Got MDS map 70 2024-04-04T06:14:04.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.872+0000 7f6b0c23e740 4 datascan.main: Using metadata pool output 2024-04-04T06:14:04.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.872+0000 7f6b0c23e740 4 datascan.main: connecting to RADOS... 2024-04-04T06:14:04.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.873+0000 7f6b0c23e740 1 Processor -- start 2024-04-04T06:14:04.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.873+0000 7f6b0c23e740 1 -- start start 2024-04-04T06:14:04.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.874+0000 7f6b0c23e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc18103800 0x55bc17509080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.874+0000 7f6b0c23e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc1825f000 0x55bc17508580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.874+0000 7f6b0c23e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc1825f400 0x55bc1750a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.874+0000 7f6b0c23e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bc173f0180 con 0x55bc1825f400 2024-04-04T06:14:04.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.874+0000 7f6b0c23e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bc1817ef00 con 0x55bc1825f000 2024-04-04T06:14:04.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.874+0000 7f6b0c23e740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bc1817f080 con 0x55bc18103800 2024-04-04T06:14:04.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.874+0000 7f6b09200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc1825f400 0x55bc1750a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:04.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.874+0000 7f6b09200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc1825f400 0x55bc1750a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33600/0 (socket says 172.21.15.159:33600) 2024-04-04T06:14:04.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.874+0000 7f6b09200640 1 -- 172.21.15.159:0/3137144813 learned_addr learned my addr 172.21.15.159:0/3137144813 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:04.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.874+0000 7f6b09c00640 1 --2- 172.21.15.159:0/3137144813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc1825f000 0x55bc17508580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:04.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.874+0000 7f6b09200640 1 -- 172.21.15.159:0/3137144813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc18103800 msgr2=0x55bc17509080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:04.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.875+0000 7f6b09200640 1 --2- 172.21.15.159:0/3137144813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc18103800 0x55bc17509080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.875+0000 7f6b09200640 1 -- 172.21.15.159:0/3137144813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc1825f000 msgr2=0x55bc17508580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:04.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.875+0000 7f6b09200640 1 --2- 172.21.15.159:0/3137144813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc1825f000 0x55bc17508580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.875+0000 7f6b09200640 1 -- 172.21.15.159:0/3137144813 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc173bc3c0 con 0x55bc1825f400 2024-04-04T06:14:04.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.875+0000 7f6b09200640 1 --2- 172.21.15.159:0/3137144813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc1825f400 0x55bc1750a100 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x55bc18272c00 tx=0x55bc18128a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6642fb75d642e5b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:04.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.875+0000 7f6b04200640 1 -- 172.21.15.159:0/3137144813 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bc173b6820 con 0x55bc1825f400 2024-04-04T06:14:04.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.875+0000 7f6b04200640 1 -- 172.21.15.159:0/3137144813 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55bc173dae00 con 0x55bc1825f400 2024-04-04T06:14:04.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.875+0000 7f6b04200640 1 -- 172.21.15.159:0/3137144813 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bc173b6d00 con 0x55bc1825f400 2024-04-04T06:14:04.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.876+0000 7f6b0c23e740 1 -- 172.21.15.159:0/3137144813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc1825f400 msgr2=0x55bc1750a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:04.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.876+0000 7f6b0c23e740 1 --2- 172.21.15.159:0/3137144813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc1825f400 0x55bc1750a100 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x55bc18272c00 tx=0x55bc18128a80 comp rx=0 tx=0).stop 2024-04-04T06:14:04.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.876+0000 7f6b0c23e740 1 -- 172.21.15.159:0/3137144813 shutdown_connections 2024-04-04T06:14:04.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.876+0000 7f6b0c23e740 1 --2- 172.21.15.159:0/3137144813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc1825f400 0x55bc1750a100 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.876+0000 7f6b0c23e740 1 --2- 172.21.15.159:0/3137144813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc1825f000 0x55bc17508580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.876+0000 7f6b0c23e740 1 --2- 172.21.15.159:0/3137144813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc18103800 0x55bc17509080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.876+0000 7f6b0c23e740 1 -- 172.21.15.159:0/3137144813 >> 172.21.15.159:0/3137144813 conn(0x55bc18103400 msgr2=0x55bc1747d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:04.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.876+0000 7f6b0c23e740 1 -- 172.21.15.159:0/3137144813 shutdown_connections 2024-04-04T06:14:04.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.876+0000 7f6b0c23e740 1 -- 172.21.15.159:0/3137144813 wait complete. 2024-04-04T06:14:04.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.876+0000 7f6b0c23e740 1 Processor -- start 2024-04-04T06:14:04.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.877+0000 7f6b0c23e740 1 -- start start 2024-04-04T06:14:04.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.877+0000 7f6b0c23e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc18103800 0x55bc1750b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.877+0000 7f6b0c23e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc1825f000 0x55bc1750ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.877+0000 7f6b0c23e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc1825f400 0x55bc1750a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.877+0000 7f6b0c23e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bc1817f380 con 0x55bc1825f400 2024-04-04T06:14:04.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.877+0000 7f6b0c23e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bc1817f200 con 0x55bc18103800 2024-04-04T06:14:04.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.877+0000 7f6b0c23e740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bc1817f500 con 0x55bc1825f000 2024-04-04T06:14:04.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.877+0000 7f6b09200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc1825f400 0x55bc1750a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:04.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.878+0000 7f6b09200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc1825f400 0x55bc1750a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33616/0 (socket says 172.21.15.159:33616) 2024-04-04T06:14:04.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.878+0000 7f6b09200640 1 -- 172.21.15.159:0/2690490501 learned_addr learned my addr 172.21.15.159:0/2690490501 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:04.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.878+0000 7f6b09200640 1 -- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc1825f000 msgr2=0x55bc1750ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:04.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.878+0000 7f6b09200640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc1825f000 0x55bc1750ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.878+0000 7f6b09200640 1 -- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc18103800 msgr2=0x55bc1750b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:04.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.878+0000 7f6b09200640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc18103800 0x55bc1750b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.878+0000 7f6b09200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc173bd680 con 0x55bc1825f400 2024-04-04T06:14:04.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.878+0000 7f6b09200640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc1825f400 0x55bc1750a680 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x55bc182738c0 tx=0x55bc18276c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:04.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.878+0000 7f6b02400640 1 -- 172.21.15.159:0/2690490501 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bc173b6680 con 0x55bc1825f400 2024-04-04T06:14:04.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.878+0000 7f6b02400640 1 -- 172.21.15.159:0/2690490501 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55bc173dac40 con 0x55bc1825f400 2024-04-04T06:14:04.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.878+0000 7f6b02400640 1 -- 172.21.15.159:0/2690490501 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bc173b64e0 con 0x55bc1825f400 2024-04-04T06:14:04.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.878+0000 7f6b0c23e740 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bc173bd2c0 con 0x55bc1825f400 2024-04-04T06:14:04.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.879+0000 7f6b0c23e740 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc173bda40 con 0x55bc1825f400 2024-04-04T06:14:04.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.879+0000 7f6b0c23e740 4 datascan.init: resolving metadata pool 12 2024-04-04T06:14:04.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.880+0000 7f6b02400640 1 -- 172.21.15.159:0/2690490501 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55bc182cac00 con 0x55bc1825f400 2024-04-04T06:14:04.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.881+0000 7f6b02400640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x55bc182c9000 0x55bc1750b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.881+0000 7f6b02400640 1 -- 172.21.15.159:0/2690490501 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x55bc182d8600 con 0x55bc1825f400 2024-04-04T06:14:04.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.881+0000 7f6b0c23e740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-04T06:14:04.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.881+0000 7f6b0c23e740 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-04-04T06:14:04.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.881+0000 7f6b0c23e740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-04T06:14:04.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.881+0000 7f6b0c23e740 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x55bc1825f800 0x55bc1750a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.881+0000 7f6b0c23e740 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc1825fc00 con 0x55bc1825f800 2024-04-04T06:14:04.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.881+0000 7f6b0a600640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x55bc182c9000 0x55bc1750b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:04.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.882+0000 7f6b09c00640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x55bc1825f800 0x55bc1750a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:04.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.883+0000 7f6b0a600640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x55bc182c9000 0x55bc1750b700 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x55bc18110240 tx=0x55bc18128300 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:04.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.883+0000 7f6b09c00640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x55bc1825f800 0x55bc1750a100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:04.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.889+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.7 v2:172.21.15.178:6810/1066817938 1 ==== osd_op_reply(1 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc1825f800 2024-04-04T06:14:04.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.889+0000 7f6b03800640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x55bc182c9400 0x55bc1771a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.889+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:2 13.20 13:04000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc182c9800 con 0x55bc182c9400 2024-04-04T06:14:04.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.889+0000 7f6b0a600640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x55bc182c9400 0x55bc1771a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:04.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.890+0000 7f6b0a600640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x55bc182c9400 0x55bc1771a000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:04.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.896+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.6 v2:172.21.15.178:6826/198949157 1 ==== osd_op_reply(2 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc182c9400 2024-04-04T06:14:04.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.897+0000 7f6b04200640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x55bc17724000 0x55bc1771a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.897+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:3 13.10 13:08000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17724400 con 0x55bc17724000 2024-04-04T06:14:04.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.897+0000 7f6b09c00640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x55bc17724000 0x55bc1771a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:04.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.898+0000 7f6b09c00640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x55bc17724000 0x55bc1771a580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:04.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.900+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.1 v2:172.21.15.159:6816/3765338297 1 ==== osd_op_reply(3 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc17724000 2024-04-04T06:14:04.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.900+0000 7f6b03800640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x55bc182c9c00 0x55bc1771ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.900+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:4 13.30 13:0c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17724400 con 0x55bc182c9c00 2024-04-04T06:14:04.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.901+0000 7f6b09200640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x55bc182c9c00 0x55bc1771ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:04.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.902+0000 7f6b09200640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x55bc182c9c00 0x55bc1771ab00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:04.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.902+0000 7f6b09200640 1 -- 172.21.15.159:0/2690490501 <== osd.4 v2:172.21.15.178:6812/4035991429 1 ==== osd_op_reply(4 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112780 con 0x55bc182c9c00 2024-04-04T06:14:04.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.902+0000 7f6b04200640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x55bc17724800 0x55bc1771b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.902+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:5 13.8 13:10000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17725000 con 0x55bc17724800 2024-04-04T06:14:04.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.903+0000 7f6b0a600640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x55bc17724800 0x55bc1771b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:04.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.903+0000 7f6b0a600640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x55bc17724800 0x55bc1771b600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:04.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.904+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.2 v2:172.21.15.159:6808/2619731047 1 ==== osd_op_reply(5 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc17724800 2024-04-04T06:14:04.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.904+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:6 13.28 13:14000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17724c00 con 0x55bc17724800 2024-04-04T06:14:04.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.904+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.2 v2:172.21.15.159:6808/2619731047 2 ==== osd_op_reply(6 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc17724800 2024-04-04T06:14:04.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.904+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:7 13.18 13:18000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17725400 con 0x55bc1825f800 2024-04-04T06:14:04.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.905+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.7 v2:172.21.15.178:6810/1066817938 2 ==== osd_op_reply(7 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc1825f800 2024-04-04T06:14:04.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.905+0000 7f6b03800640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x55bc1825fc00 0x55bc1771b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.905+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:8 13.38 13:1c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17725c00 con 0x55bc1825fc00 2024-04-04T06:14:04.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.905+0000 7f6b09c00640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x55bc1825fc00 0x55bc1771b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:04.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.906+0000 7f6b09c00640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x55bc1825fc00 0x55bc1771b080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:04.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.907+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.5 v2:172.21.15.178:6802/63746765 1 ==== osd_op_reply(8 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc1825fc00 2024-04-04T06:14:04.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.907+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:9 13.4 13:20000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17725800 con 0x55bc182c9400 2024-04-04T06:14:04.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.908+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.6 v2:172.21.15.178:6826/198949157 2 ==== osd_op_reply(9 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc182c9400 2024-04-04T06:14:04.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.908+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:10 13.24 13:24000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17738000 con 0x55bc182c9400 2024-04-04T06:14:04.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.909+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.6 v2:172.21.15.178:6826/198949157 3 ==== osd_op_reply(10 [pgnls start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc182c9400 2024-04-04T06:14:04.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.909+0000 7f6b04200640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x55bc17739400 0x55bc1771bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.909+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:11 13.14 13:28000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17739800 con 0x55bc17739400 2024-04-04T06:14:04.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.910+0000 7f6b09200640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x55bc17739400 0x55bc1771bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:04.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.910+0000 7f6b09200640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x55bc17739400 0x55bc1771bb80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:04.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.913+0000 7f6b09200640 1 -- 172.21.15.159:0/2690490501 <== osd.0 v2:172.21.15.159:6800/455382518 1 ==== osd_op_reply(11 [pgnls start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112780 con 0x55bc17739400 2024-04-04T06:14:04.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.913+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:12 13.34 13:2c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17738400 con 0x55bc182c9400 2024-04-04T06:14:04.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.914+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.6 v2:172.21.15.178:6826/198949157 4 ==== osd_op_reply(12 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc182c9400 2024-04-04T06:14:04.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.914+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:13 13.c 13:30000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17739c00 con 0x55bc17724000 2024-04-04T06:14:04.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.914+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.1 v2:172.21.15.159:6816/3765338297 2 ==== osd_op_reply(13 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc17724000 2024-04-04T06:14:04.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.915+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:14 13.2c 13:34000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17738800 con 0x55bc17724800 2024-04-04T06:14:04.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.915+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.2 v2:172.21.15.159:6808/2619731047 3 ==== osd_op_reply(14 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc17724800 2024-04-04T06:14:04.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.915+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:15 13.1c 13:38000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17725400 con 0x55bc182c9400 2024-04-04T06:14:04.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.916+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.6 v2:172.21.15.178:6826/198949157 5 ==== osd_op_reply(15 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc182c9400 2024-04-04T06:14:04.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.916+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:16 13.3c 13:3c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17738c00 con 0x55bc17724800 2024-04-04T06:14:04.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.916+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.2 v2:172.21.15.159:6808/2619731047 4 ==== osd_op_reply(16 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc17724800 2024-04-04T06:14:04.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.916+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:17 13.2 13:40000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17725c00 con 0x55bc1825fc00 2024-04-04T06:14:04.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.917+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.5 v2:172.21.15.178:6802/63746765 2 ==== osd_op_reply(17 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc1825fc00 2024-04-04T06:14:04.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.917+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:18 13.22 13:44000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17739000 con 0x55bc17724000 2024-04-04T06:14:04.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.918+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.1 v2:172.21.15.159:6816/3765338297 3 ==== osd_op_reply(18 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc17724000 2024-04-04T06:14:04.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.918+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:19 13.12 13:48000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17725000 con 0x55bc17739400 2024-04-04T06:14:04.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.918+0000 7f6b09200640 1 -- 172.21.15.159:0/2690490501 <== osd.0 v2:172.21.15.159:6800/455382518 2 ==== osd_op_reply(19 [pgnls start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112780 con 0x55bc17739400 2024-04-04T06:14:04.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.918+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:20 13.32 13:4c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17739c00 con 0x55bc17724800 2024-04-04T06:14:04.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.919+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.2 v2:172.21.15.159:6808/2619731047 5 ==== osd_op_reply(20 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc17724800 2024-04-04T06:14:04.913 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.919+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:21 13.a 13:50000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17724c00 con 0x55bc17724800 2024-04-04T06:14:04.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.919+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.2 v2:172.21.15.159:6808/2619731047 6 ==== osd_op_reply(21 [pgnls start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc17724800 2024-04-04T06:14:04.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.919+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:22 13.2a 13:54000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17738000 con 0x55bc17724000 2024-04-04T06:14:04.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.919+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.1 v2:172.21.15.159:6816/3765338297 4 ==== osd_op_reply(22 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc17724000 2024-04-04T06:14:04.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.919+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:23 13.1a 13:58000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc182c9800 con 0x55bc17724000 2024-04-04T06:14:04.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.920+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.1 v2:172.21.15.159:6816/3765338297 5 ==== osd_op_reply(23 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc17724000 2024-04-04T06:14:04.914 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.920+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:24 13.3a 13:5c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17738400 con 0x55bc1825fc00 2024-04-04T06:14:04.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.921+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.5 v2:172.21.15.178:6802/63746765 3 ==== osd_op_reply(24 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc1825fc00 2024-04-04T06:14:04.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.921+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:25 13.6 13:60000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc182c9800 con 0x55bc182c9c00 2024-04-04T06:14:04.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.922+0000 7f6b09200640 1 -- 172.21.15.159:0/2690490501 <== osd.4 v2:172.21.15.178:6812/4035991429 2 ==== osd_op_reply(25 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112780 con 0x55bc182c9c00 2024-04-04T06:14:04.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.922+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:26 13.26 13:64000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17725800 con 0x55bc1825f800 2024-04-04T06:14:04.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.923+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.7 v2:172.21.15.178:6810/1066817938 3 ==== osd_op_reply(26 [pgnls start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc1825f800 2024-04-04T06:14:04.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.923+0000 7f6b04200640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55bc17739000 0x55bc1771c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:04.916 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.923+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:27 13.16 13:68000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17738000 con 0x55bc17739000 2024-04-04T06:14:04.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.923+0000 7f6b0a600640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55bc17739000 0x55bc1771c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:04.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.924+0000 7f6b0a600640 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55bc17739000 0x55bc1771c100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:04.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.927+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.3 v2:172.21.15.159:6820/2070760205 1 ==== osd_op_reply(27 [pgnls start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc17739000 2024-04-04T06:14:04.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.927+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:28 13.36 13:6c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc182c8c00 con 0x55bc182c9400 2024-04-04T06:14:04.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.928+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.6 v2:172.21.15.178:6826/198949157 6 ==== osd_op_reply(28 [pgnls start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc182c9400 2024-04-04T06:14:04.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.928+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:29 13.e 13:70000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17725c00 con 0x55bc1825f800 2024-04-04T06:14:04.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.929+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.7 v2:172.21.15.178:6810/1066817938 4 ==== osd_op_reply(29 [pgnls start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc1825f800 2024-04-04T06:14:04.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.929+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:30 13.2e 13:74000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc182c8800 con 0x55bc17724800 2024-04-04T06:14:04.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.929+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.2 v2:172.21.15.159:6808/2619731047 7 ==== osd_op_reply(30 [pgnls start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc17724800 2024-04-04T06:14:04.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.929+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:31 13.1e 13:78000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc182c8c00 con 0x55bc182c9c00 2024-04-04T06:14:04.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.930+0000 7f6b09200640 1 -- 172.21.15.159:0/2690490501 <== osd.4 v2:172.21.15.178:6812/4035991429 3 ==== osd_op_reply(31 [pgnls start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112780 con 0x55bc182c9c00 2024-04-04T06:14:04.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.930+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:32 13.3e 13:7c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc182c9800 con 0x55bc17724000 2024-04-04T06:14:04.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.931+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.1 v2:172.21.15.159:6816/3765338297 6 ==== osd_op_reply(32 [pgnls start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc17724000 2024-04-04T06:14:04.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.931+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:33 13.1 13:80000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17738800 con 0x55bc182c9400 2024-04-04T06:14:04.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.932+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.6 v2:172.21.15.178:6826/198949157 7 ==== osd_op_reply(33 [pgnls start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc182c9400 2024-04-04T06:14:04.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.932+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:34 13.21 13:84000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17724400 con 0x55bc182c9c00 2024-04-04T06:14:04.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.933+0000 7f6b09200640 1 -- 172.21.15.159:0/2690490501 <== osd.4 v2:172.21.15.178:6812/4035991429 4 ==== osd_op_reply(34 [pgnls start_epoch 89 out=49b] v85'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112780 con 0x55bc182c9c00 2024-04-04T06:14:04.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.933+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:35 13.11 13:88000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17738c00 con 0x55bc17724000 2024-04-04T06:14:04.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.934+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.1 v2:172.21.15.159:6816/3765338297 7 ==== osd_op_reply(35 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc17724000 2024-04-04T06:14:04.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.934+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:36 13.31 13:8c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17725000 con 0x55bc17739000 2024-04-04T06:14:04.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.934+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.3 v2:172.21.15.159:6820/2070760205 2 ==== osd_op_reply(36 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc17739000 2024-04-04T06:14:04.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.934+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:37 13.9 13:90000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17739c00 con 0x55bc1825fc00 2024-04-04T06:14:04.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.935+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.5 v2:172.21.15.178:6802/63746765 4 ==== osd_op_reply(37 [pgnls start_epoch 89 out=49b] v85'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc1825fc00 2024-04-04T06:14:04.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.935+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:38 13.29 13:94000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17739800 con 0x55bc182c9400 2024-04-04T06:14:04.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.936+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.6 v2:172.21.15.178:6826/198949157 8 ==== osd_op_reply(38 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc182c9400 2024-04-04T06:14:04.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.936+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:39 13.19 13:98000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17738000 con 0x55bc17739000 2024-04-04T06:14:04.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.937+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.3 v2:172.21.15.159:6820/2070760205 3 ==== osd_op_reply(39 [pgnls start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc17739000 2024-04-04T06:14:04.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.937+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:40 13.39 13:9c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17738400 con 0x55bc17724800 2024-04-04T06:14:04.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.937+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.2 v2:172.21.15.159:6808/2619731047 8 ==== osd_op_reply(40 [pgnls start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc17724800 2024-04-04T06:14:04.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.938+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:41 13.5 13:a0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17725400 con 0x55bc182c9c00 2024-04-04T06:14:04.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.939+0000 7f6b09200640 1 -- 172.21.15.159:0/2690490501 <== osd.4 v2:172.21.15.178:6812/4035991429 5 ==== osd_op_reply(41 [pgnls start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112780 con 0x55bc182c9c00 2024-04-04T06:14:04.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.939+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:42 13.25 13:a4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17738c00 con 0x55bc182c9c00 2024-04-04T06:14:04.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.940+0000 7f6b09200640 1 -- 172.21.15.159:0/2690490501 <== osd.4 v2:172.21.15.178:6812/4035991429 6 ==== osd_op_reply(42 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112780 con 0x55bc182c9c00 2024-04-04T06:14:04.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.940+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:43 13.15 13:a8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17724c00 con 0x55bc182c9c00 2024-04-04T06:14:04.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.941+0000 7f6b09200640 1 -- 172.21.15.159:0/2690490501 <== osd.4 v2:172.21.15.178:6812/4035991429 7 ==== osd_op_reply(43 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112780 con 0x55bc182c9c00 2024-04-04T06:14:04.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.941+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:44 13.35 13:ac000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc182c9800 con 0x55bc17724000 2024-04-04T06:14:04.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.942+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.1 v2:172.21.15.159:6816/3765338297 8 ==== osd_op_reply(44 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc17724000 2024-04-04T06:14:04.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.942+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:45 13.d 13:b0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17746000 con 0x55bc182c9c00 2024-04-04T06:14:04.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.943+0000 7f6b09200640 1 -- 172.21.15.159:0/2690490501 <== osd.4 v2:172.21.15.178:6812/4035991429 8 ==== osd_op_reply(45 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112780 con 0x55bc182c9c00 2024-04-04T06:14:04.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.943+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:46 13.2d 13:b4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17725800 con 0x55bc17739400 2024-04-04T06:14:04.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.944+0000 7f6b09200640 1 -- 172.21.15.159:0/2690490501 <== osd.0 v2:172.21.15.159:6800/455382518 3 ==== osd_op_reply(46 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112780 con 0x55bc17739400 2024-04-04T06:14:04.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.944+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:47 13.1d 13:b8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17746400 con 0x55bc1825f800 2024-04-04T06:14:04.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.945+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.7 v2:172.21.15.178:6810/1066817938 5 ==== osd_op_reply(47 [pgnls start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc1825f800 2024-04-04T06:14:04.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.945+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:48 13.3d 13:bc000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17725c00 con 0x55bc1825f800 2024-04-04T06:14:04.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.946+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.7 v2:172.21.15.178:6810/1066817938 6 ==== osd_op_reply(48 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc1825f800 2024-04-04T06:14:04.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.946+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:49 13.3 13:c0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17746800 con 0x55bc17739400 2024-04-04T06:14:04.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.947+0000 7f6b09200640 1 -- 172.21.15.159:0/2690490501 <== osd.0 v2:172.21.15.159:6800/455382518 4 ==== osd_op_reply(49 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112780 con 0x55bc17739400 2024-04-04T06:14:04.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.947+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:50 13.23 13:c4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17724400 con 0x55bc17724000 2024-04-04T06:14:04.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.948+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.1 v2:172.21.15.159:6816/3765338297 9 ==== osd_op_reply(50 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc17724000 2024-04-04T06:14:04.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.948+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:51 13.13 13:c8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17746c00 con 0x55bc1825f800 2024-04-04T06:14:04.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.949+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.7 v2:172.21.15.178:6810/1066817938 7 ==== osd_op_reply(51 [pgnls start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc1825f800 2024-04-04T06:14:04.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.949+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:52 13.33 13:cc000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17725400 con 0x55bc1825f800 2024-04-04T06:14:04.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.950+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.7 v2:172.21.15.178:6810/1066817938 8 ==== osd_op_reply(52 [pgnls start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc1825f800 2024-04-04T06:14:04.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.950+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:53 13.b 13:d0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17747000 con 0x55bc1825fc00 2024-04-04T06:14:04.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.951+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.5 v2:172.21.15.178:6802/63746765 5 ==== osd_op_reply(53 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc1825fc00 2024-04-04T06:14:04.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.952+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:54 13.2b 13:d4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17724c00 con 0x55bc17724800 2024-04-04T06:14:04.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.952+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.2 v2:172.21.15.159:6808/2619731047 9 ==== osd_op_reply(54 [pgnls start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc17724800 2024-04-04T06:14:04.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.952+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:55 13.1b 13:d8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17747400 con 0x55bc182c9c00 2024-04-04T06:14:04.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.953+0000 7f6b09200640 1 -- 172.21.15.159:0/2690490501 <== osd.4 v2:172.21.15.178:6812/4035991429 9 ==== osd_op_reply(55 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112780 con 0x55bc182c9c00 2024-04-04T06:14:04.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.953+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:56 13.3b 13:dc000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17725800 con 0x55bc182c9c00 2024-04-04T06:14:04.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.954+0000 7f6b09200640 1 -- 172.21.15.159:0/2690490501 <== osd.4 v2:172.21.15.178:6812/4035991429 10 ==== osd_op_reply(56 [pgnls start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112780 con 0x55bc182c9c00 2024-04-04T06:14:04.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.954+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:57 13.7 13:e0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17747800 con 0x55bc1825f800 2024-04-04T06:14:04.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.956+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.7 v2:172.21.15.178:6810/1066817938 9 ==== osd_op_reply(57 [pgnls start_epoch 89 out=81b] v85'6 uv6 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55bc18112000 con 0x55bc1825f800 2024-04-04T06:14:04.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.956+0000 7f6b0c23e740 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:58 13.7 13:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x55bc17746400 con 0x55bc1825f800 2024-04-04T06:14:04.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.957+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.7 v2:172.21.15.178:6810/1066817938 10 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55bc18112000 con 0x55bc1825f800 2024-04-04T06:14:04.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.957+0000 7f6b0c23e740 10 datascan.operator(): handling object 1099511627777.0 2024-04-04T06:14:04.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.957+0000 7f6b0c23e740 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:59 13.7 13:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x55bc17746c00 con 0x55bc1825f800 2024-04-04T06:14:04.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.961+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.7 v2:172.21.15.178:6810/1066817938 11 ==== osd_op_reply(59 10000000001.00000000 [call] v89'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55bc18112000 con 0x55bc1825f800 2024-04-04T06:14:04.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.962+0000 7f6b0c23e740 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:60 13.27 13:e4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17725c00 con 0x55bc17724000 2024-04-04T06:14:04.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.962+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.1 v2:172.21.15.159:6816/3765338297 10 ==== osd_op_reply(60 [pgnls start_epoch 89 out=49b] v85'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc17724000 2024-04-04T06:14:04.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.962+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:61 13.17 13:e8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17747c00 con 0x55bc1825fc00 2024-04-04T06:14:04.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.963+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.5 v2:172.21.15.178:6802/63746765 6 ==== osd_op_reply(61 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc1825fc00 2024-04-04T06:14:04.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.963+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:62 13.37 13:ec000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17746000 con 0x55bc1825fc00 2024-04-04T06:14:04.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.964+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 <== osd.5 v2:172.21.15.178:6802/63746765 7 ==== osd_op_reply(62 [pgnls start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112000 con 0x55bc1825fc00 2024-04-04T06:14:04.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.964+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:63 13.f 13:f0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17747000 con 0x55bc182c9400 2024-04-04T06:14:04.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.965+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.6 v2:172.21.15.178:6826/198949157 9 ==== osd_op_reply(63 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc182c9400 2024-04-04T06:14:04.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.965+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:64 13.2f 13:f4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17738c00 con 0x55bc182c9c00 2024-04-04T06:14:04.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.966+0000 7f6b09200640 1 -- 172.21.15.159:0/2690490501 <== osd.4 v2:172.21.15.178:6812/4035991429 11 ==== osd_op_reply(64 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112780 con 0x55bc182c9c00 2024-04-04T06:14:04.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.966+0000 7f6b04200640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:65 13.1f 13:f8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc17747800 con 0x55bc182c9400 2024-04-04T06:14:04.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.968+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.6 v2:172.21.15.178:6826/198949157 10 ==== osd_op_reply(65 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc182c9400 2024-04-04T06:14:04.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.968+0000 7f6b03800640 1 -- 172.21.15.159:0/2690490501 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:66 13.3f 13:fc000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55bc182c8c00 con 0x55bc182c9400 2024-04-04T06:14:04.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b0a600640 1 -- 172.21.15.159:0/2690490501 <== osd.6 v2:172.21.15.178:6826/198949157 11 ==== osd_op_reply(66 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bc18112500 con 0x55bc182c9400 2024-04-04T06:14:04.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b0c23e740 1 -- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x55bc17739400 msgr2=0x55bc1771bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:04.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b0c23e740 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x55bc17739400 0x55bc1771bb80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b0c23e740 1 -- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x55bc17724000 msgr2=0x55bc1771a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:04.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b0c23e740 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x55bc17724000 0x55bc1771a580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b0c23e740 1 -- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x55bc17724800 msgr2=0x55bc1771b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:04.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b0c23e740 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x55bc17724800 0x55bc1771b600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b0c23e740 1 -- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55bc17739000 msgr2=0x55bc1771c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:04.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b0c23e740 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55bc17739000 0x55bc1771c100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b0c23e740 1 -- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x55bc182c9c00 msgr2=0x55bc1771ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:04.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b0c23e740 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x55bc182c9c00 0x55bc1771ab00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b0c23e740 1 -- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x55bc1825fc00 msgr2=0x55bc1771b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:04.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b0c23e740 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x55bc1825fc00 0x55bc1771b080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b0c23e740 1 -- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x55bc182c9400 msgr2=0x55bc1771a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:04.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b0c23e740 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x55bc182c9400 0x55bc1771a000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b0c23e740 1 -- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x55bc1825f800 msgr2=0x55bc1750a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:04.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b0c23e740 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x55bc1825f800 0x55bc1750a100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 reap_dead start 2024-04-04T06:14:04.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 reap_dead start 2024-04-04T06:14:04.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 reap_dead start 2024-04-04T06:14:04.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 reap_dead start 2024-04-04T06:14:04.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.969+0000 7f6b09c00640 1 -- 172.21.15.159:0/2690490501 reap_dead start 2024-04-04T06:14:04.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.970+0000 7f6b0c23e740 1 -- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x55bc182c9000 msgr2=0x55bc1750b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:04.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.970+0000 7f6b0c23e740 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x55bc182c9000 0x55bc1750b700 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x55bc18110240 tx=0x55bc18128300 comp rx=0 tx=0).stop 2024-04-04T06:14:04.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.970+0000 7f6b0c23e740 1 -- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc1825f400 msgr2=0x55bc1750a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:04.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.970+0000 7f6b0c23e740 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc1825f400 0x55bc1750a680 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x55bc182738c0 tx=0x55bc18276c00 comp rx=0 tx=0).stop 2024-04-04T06:14:04.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.970+0000 7f6b0c23e740 1 -- 172.21.15.159:0/2690490501 shutdown_connections 2024-04-04T06:14:04.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.970+0000 7f6b0c23e740 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc1825f400 0x55bc1750a680 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.970+0000 7f6b0c23e740 1 --2- 172.21.15.159:0/2690490501 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x55bc182c9000 0x55bc1750b700 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.970+0000 7f6b0c23e740 1 -- 172.21.15.159:0/2690490501 >> 172.21.15.159:0/2690490501 conn(0x55bc18103400 msgr2=0x55bc1747d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:04.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.970+0000 7f6b0c23e740 1 -- 172.21.15.159:0/2690490501 shutdown_connections 2024-04-04T06:14:04.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.970+0000 7f6b0c23e740 1 -- 172.21.15.159:0/2690490501 wait complete. 2024-04-04T06:14:04.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.971+0000 7f6b0c23e740 1 -- 172.21.15.159:0/3289886741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc18102800 msgr2=0x55bc17508b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:04.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.971+0000 7f6b0c23e740 1 --2- 172.21.15.159:0/3289886741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc18102800 0x55bc17508b00 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x55bc181b58f0 tx=0x55bc1812a120 comp rx=0 tx=0).stop 2024-04-04T06:14:04.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.971+0000 7f6b0c23e740 1 -- 172.21.15.159:0/3289886741 shutdown_connections 2024-04-04T06:14:04.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.971+0000 7f6b0c23e740 1 --2- 172.21.15.159:0/3289886741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bc18103c00 0x55bc17509b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.971+0000 7f6b0c23e740 1 --2- 172.21.15.159:0/3289886741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc18102800 0x55bc17508b00 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.971+0000 7f6b0c23e740 1 --2- 172.21.15.159:0/3289886741 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bc18102400 0x55bc17509600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:04.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.971+0000 7f6b0c23e740 1 -- 172.21.15.159:0/3289886741 >> 172.21.15.159:0/3289886741 conn(0x55bc17391000 msgr2=0x55bc1747d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:04.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.972+0000 7f6b0c23e740 1 -- 172.21.15.159:0/3289886741 shutdown_connections 2024-04-04T06:14:04.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:04.972+0000 7f6b0c23e740 1 -- 172.21.15.159:0/3289886741 wait complete. 2024-04-04T06:14:04.975 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.199115, result: 2024-04-04T06:14:04.975 DEBUG:teuthology.orchestra.run.smithi159:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-04-04T06:14:05.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.075+0000 7fde34c49740 1 -- start start 2024-04-04T06:14:05.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.076+0000 7fde34c49740 1 Processor -- start 2024-04-04T06:14:05.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.076+0000 7fde34c49740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0e1b28800 0x55a0e0f35080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.077+0000 7fde34c49740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e1b29c00 0x55a0e0f34580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.077+0000 7fde34c49740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a0e1b29400 0x55a0e0f34000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.077+0000 7fde34c49740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a0e1bc8000 con 0x55a0e1b29c00 2024-04-04T06:14:05.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.077+0000 7fde34c49740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a0e1bc8180 con 0x55a0e1b28800 2024-04-04T06:14:05.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.077+0000 7fde34c49740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a0e1bc8300 con 0x55a0e1b29400 2024-04-04T06:14:05.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.077+0000 7fde31c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e1b29c00 0x55a0e0f34580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.077+0000 7fde31c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e1b29c00 0x55a0e0f34580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33624/0 (socket says 172.21.15.159:33624) 2024-04-04T06:14:05.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.077+0000 7fde31c00640 1 -- 172.21.15.159:0/2932260390 learned_addr learned my addr 172.21.15.159:0/2932260390 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:05.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.077+0000 7fde33000640 1 --2- 172.21.15.159:0/2932260390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a0e1b29400 0x55a0e0f34000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.077+0000 7fde31c00640 1 -- 172.21.15.159:0/2932260390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a0e1b29400 msgr2=0x55a0e0f34000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:05.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.077+0000 7fde32600640 1 --2- 172.21.15.159:0/2932260390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0e1b28800 0x55a0e0f35080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.077+0000 7fde31c00640 1 --2- 172.21.15.159:0/2932260390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a0e1b29400 0x55a0e0f34000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.077+0000 7fde31c00640 1 -- 172.21.15.159:0/2932260390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0e1b28800 msgr2=0x55a0e0f35080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:05.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.077+0000 7fde31c00640 1 --2- 172.21.15.159:0/2932260390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0e1b28800 0x55a0e0f35080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.077+0000 7fde31c00640 1 -- 172.21.15.159:0/2932260390 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0e0ded680 con 0x55a0e1b29c00 2024-04-04T06:14:05.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.078+0000 7fde31c00640 1 --2- 172.21.15.159:0/2932260390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e1b29c00 0x55a0e0f34580 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x55a0e1be1950 tx=0x55a0e1b5a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e93d5fe3d8212 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.078+0000 7fde2f400640 1 -- 172.21.15.159:0/2932260390 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a0e0de69c0 con 0x55a0e1b29c00 2024-04-04T06:14:05.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.078+0000 7fde2f400640 1 -- 172.21.15.159:0/2932260390 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a0e0e0a540 con 0x55a0e1b29c00 2024-04-04T06:14:05.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.078+0000 7fde34c49740 1 -- 172.21.15.159:0/2932260390 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0e0ded4a0 con 0x55a0e1b29c00 2024-04-04T06:14:05.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.078+0000 7fde34c49740 1 -- 172.21.15.159:0/2932260390 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55a0e0dec3c0 con 0x55a0e1b29c00 2024-04-04T06:14:05.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.078+0000 7fde34c49740 4 waiting for MDS map... 2024-04-04T06:14:05.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.078+0000 7fde2f400640 1 -- 172.21.15.159:0/2932260390 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a0e0de6b60 con 0x55a0e1b29c00 2024-04-04T06:14:05.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.078+0000 7fde2f400640 1 -- 172.21.15.159:0/2932260390 <== mon.0 v2:172.21.15.159:3300/0 4 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x55a0e1bca400 con 0x55a0e1b29c00 2024-04-04T06:14:05.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.078+0000 7fde2f400640 1 -- 172.21.15.159:0/2932260390 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 70) ==== 2885+0+0 (secure 0 0 0) 0x55a0e0e19180 con 0x55a0e1b29c00 2024-04-04T06:14:05.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.078+0000 7fde34c49740 4 Got MDS map 70 2024-04-04T06:14:05.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.079+0000 7fde34c49740 4 datascan.main: Using metadata pool output 2024-04-04T06:14:05.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.079+0000 7fde34c49740 4 datascan.main: connecting to RADOS... 2024-04-04T06:14:05.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.079+0000 7fde34c49740 1 Processor -- start 2024-04-04T06:14:05.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.079+0000 7fde34c49740 1 -- start start 2024-04-04T06:14:05.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.080+0000 7fde34c49740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e1c8b000 0x55a0e0f35600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.080+0000 7fde34c49740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a0e1c8b400 0x55a0e0f35b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.080+0000 7fde34c49740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0e1c8b800 0x55a0e0f36100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.080+0000 7fde34c49740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a0e1bc8480 con 0x55a0e1c8b000 2024-04-04T06:14:05.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.080+0000 7fde34c49740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a0e1bc8600 con 0x55a0e1c8b800 2024-04-04T06:14:05.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.080+0000 7fde34c49740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a0e1bc8780 con 0x55a0e1c8b400 2024-04-04T06:14:05.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.080+0000 7fde33000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e1c8b000 0x55a0e0f35600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.081+0000 7fde33000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e1c8b000 0x55a0e0f35600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33640/0 (socket says 172.21.15.159:33640) 2024-04-04T06:14:05.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.081+0000 7fde33000640 1 -- 172.21.15.159:0/3079545774 learned_addr learned my addr 172.21.15.159:0/3079545774 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:05.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.081+0000 7fde33000640 1 -- 172.21.15.159:0/3079545774 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a0e1c8b400 msgr2=0x55a0e0f35b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:05.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.081+0000 7fde33000640 1 --2- 172.21.15.159:0/3079545774 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a0e1c8b400 0x55a0e0f35b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.081+0000 7fde33000640 1 -- 172.21.15.159:0/3079545774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0e1c8b800 msgr2=0x55a0e0f36100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:05.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.081+0000 7fde33000640 1 --2- 172.21.15.159:0/3079545774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0e1c8b800 0x55a0e0f36100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.081+0000 7fde33000640 1 -- 172.21.15.159:0/3079545774 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0e0decb40 con 0x55a0e1c8b000 2024-04-04T06:14:05.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.081+0000 7fde33000640 1 --2- 172.21.15.159:0/3079545774 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e1c8b000 0x55a0e0f35600 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x55a0e1b404b0 tx=0x55a0e1b5a240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22bae2ebac5df12 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.081+0000 7fde2cc00640 1 -- 172.21.15.159:0/3079545774 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a0e0de6680 con 0x55a0e1c8b000 2024-04-04T06:14:05.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.081+0000 7fde2cc00640 1 -- 172.21.15.159:0/3079545774 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a0e0e0aa80 con 0x55a0e1c8b000 2024-04-04T06:14:05.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.081+0000 7fde2cc00640 1 -- 172.21.15.159:0/3079545774 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a0e0de69c0 con 0x55a0e1c8b000 2024-04-04T06:14:05.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.082+0000 7fde34c49740 1 -- 172.21.15.159:0/3079545774 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e1c8b000 msgr2=0x55a0e0f35600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.082+0000 7fde34c49740 1 --2- 172.21.15.159:0/3079545774 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e1c8b000 0x55a0e0f35600 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x55a0e1b404b0 tx=0x55a0e1b5a240 comp rx=0 tx=0).stop 2024-04-04T06:14:05.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.082+0000 7fde34c49740 1 -- 172.21.15.159:0/3079545774 shutdown_connections 2024-04-04T06:14:05.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.082+0000 7fde34c49740 1 --2- 172.21.15.159:0/3079545774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0e1c8b800 0x55a0e0f36100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.082+0000 7fde34c49740 1 --2- 172.21.15.159:0/3079545774 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a0e1c8b400 0x55a0e0f35b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.082+0000 7fde34c49740 1 --2- 172.21.15.159:0/3079545774 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e1c8b000 0x55a0e0f35600 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.082+0000 7fde34c49740 1 -- 172.21.15.159:0/3079545774 >> 172.21.15.159:0/3079545774 conn(0x55a0e1b29800 msgr2=0x55a0e1b5f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:05.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.082+0000 7fde34c49740 1 -- 172.21.15.159:0/3079545774 shutdown_connections 2024-04-04T06:14:05.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.082+0000 7fde34c49740 1 -- 172.21.15.159:0/3079545774 wait complete. 2024-04-04T06:14:05.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.083+0000 7fde34c49740 1 Processor -- start 2024-04-04T06:14:05.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.083+0000 7fde34c49740 1 -- start start 2024-04-04T06:14:05.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.084+0000 7fde34c49740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a0e1c8b000 0x55a0e0f37180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.084+0000 7fde34c49740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0e1c8b400 0x55a0e0f36c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.084+0000 7fde34c49740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e1c8b800 0x55a0e0f36680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.084+0000 7fde34c49740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a0e1bc9c80 con 0x55a0e1c8b800 2024-04-04T06:14:05.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.084+0000 7fde34c49740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a0e1bc9b00 con 0x55a0e1c8b400 2024-04-04T06:14:05.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.084+0000 7fde34c49740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a0e1bc9980 con 0x55a0e1c8b000 2024-04-04T06:14:05.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.084+0000 7fde31c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e1c8b800 0x55a0e0f36680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.084+0000 7fde31c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e1c8b800 0x55a0e0f36680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33644/0 (socket says 172.21.15.159:33644) 2024-04-04T06:14:05.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.084+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 learned_addr learned my addr 172.21.15.159:0/1477335233 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:05.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.084+0000 7fde33000640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a0e1c8b000 0x55a0e0f37180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.081 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.084+0000 7fde32600640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0e1c8b400 0x55a0e0f36c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.084+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a0e1c8b000 msgr2=0x55a0e0f37180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.084+0000 7fde31c00640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a0e1c8b000 0x55a0e0f37180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.084+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0e1c8b400 msgr2=0x55a0e0f36c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.084+0000 7fde31c00640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0e1c8b400 0x55a0e0f36c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.085+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0e0ded860 con 0x55a0e1c8b800 2024-04-04T06:14:05.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.085+0000 7fde33000640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a0e1c8b000 0x55a0e0f37180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:05.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.085+0000 7fde32600640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0e1c8b400 0x55a0e0f36c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:05.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.085+0000 7fde31c00640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e1c8b800 0x55a0e0f36680 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x55a0e1be1d40 tx=0x55a0e1b49bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.085+0000 7fde2ae00640 1 -- 172.21.15.159:0/1477335233 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a0e0de6680 con 0x55a0e1c8b800 2024-04-04T06:14:05.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.085+0000 7fde2ae00640 1 -- 172.21.15.159:0/1477335233 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a0e0e0ac40 con 0x55a0e1c8b800 2024-04-04T06:14:05.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.085+0000 7fde2ae00640 1 -- 172.21.15.159:0/1477335233 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a0e0de69c0 con 0x55a0e1c8b800 2024-04-04T06:14:05.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.085+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a0e0ded0e0 con 0x55a0e1c8b800 2024-04-04T06:14:05.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.085+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0e0ded680 con 0x55a0e1c8b800 2024-04-04T06:14:05.082 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.085+0000 7fde34c49740 4 datascan.init: resolving metadata pool 12 2024-04-04T06:14:05.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.087+0000 7fde2ae00640 1 -- 172.21.15.159:0/1477335233 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55a0e1cf8c00 con 0x55a0e1c8b800 2024-04-04T06:14:05.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.088+0000 7fde2ae00640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x55a0e1cff000 0x55a0e0f37700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.088+0000 7fde2ae00640 1 -- 172.21.15.159:0/1477335233 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x55a0e0f3c600 con 0x55a0e1c8b800 2024-04-04T06:14:05.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.088+0000 7fde34c49740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-04T06:14:05.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.088+0000 7fde34c49740 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-04-04T06:14:05.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.088+0000 7fde34c49740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-04T06:14:05.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.088+0000 7fde34c49740 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x55a0e1c8bc00 0x55a0e0f36100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.088+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1cff400 con 0x55a0e1c8bc00 2024-04-04T06:14:05.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.088+0000 7fde33000640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x55a0e1cff000 0x55a0e0f37700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.089+0000 7fde32600640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x55a0e1c8bc00 0x55a0e0f36100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.090+0000 7fde33000640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x55a0e1cff000 0x55a0e0f37700 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x55a0e1b404b0 tx=0x55a0e1b5a180 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.090+0000 7fde32600640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x55a0e1c8bc00 0x55a0e0f36100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.091+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.5 v2:172.21.15.178:6802/63746765 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv25 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1c8bc00 2024-04-04T06:14:05.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.091+0000 7fde34c49740 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x55a0e1cff800 0x55a0e0f35b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.091+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:2 12.1f 12:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1cffc00 con 0x55a0e1cff800 2024-04-04T06:14:05.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.092+0000 7fde33000640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x55a0e1cff800 0x55a0e0f35b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.093+0000 7fde33000640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x55a0e1cff800 0x55a0e0f35b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.093+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.6 v2:172.21.15.178:6826/198949157 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1cff800 2024-04-04T06:14:05.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.094+0000 7fde34c49740 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x55a0e1d62000 0x55a0e0f35600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.094+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:3 13.0 13:00000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1d62400 con 0x55a0e1d62000 2024-04-04T06:14:05.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.094+0000 7fde32600640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x55a0e1d62000 0x55a0e0f35600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.095+0000 7fde32600640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x55a0e1d62000 0x55a0e0f35600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.096+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.7 v2:172.21.15.178:6810/1066817938 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1d62000 2024-04-04T06:14:05.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.096+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:4 13.20 13:04000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1cff400 con 0x55a0e1cff800 2024-04-04T06:14:05.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.097+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.6 v2:172.21.15.178:6826/198949157 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1cff800 2024-04-04T06:14:05.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.097+0000 7fde2cc00640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x55a0e1d62400 0x55a0e1152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.097+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:5 13.10 13:08000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1d63c00 con 0x55a0e1d62400 2024-04-04T06:14:05.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.098+0000 7fde31c00640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x55a0e1d62400 0x55a0e1152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.098+0000 7fde31c00640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x55a0e1d62400 0x55a0e1152000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.099+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.1 v2:172.21.15.159:6816/3765338297 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e1d62400 2024-04-04T06:14:05.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.099+0000 7fde2c200640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x55a0e115a000 0x55a0e1152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.099+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:6 13.30 13:0c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115a400 con 0x55a0e115a000 2024-04-04T06:14:05.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.099+0000 7fde33000640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x55a0e115a000 0x55a0e1152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.100+0000 7fde33000640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x55a0e115a000 0x55a0e1152580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.101+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.4 v2:172.21.15.178:6812/4035991429 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e115a000 2024-04-04T06:14:05.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.101+0000 7fde2cc00640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x55a0e115a800 0x55a0e1152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.101+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:7 13.8 13:10000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115ac00 con 0x55a0e115a800 2024-04-04T06:14:05.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.102+0000 7fde32600640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x55a0e115a800 0x55a0e1152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.102+0000 7fde32600640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x55a0e115a800 0x55a0e1152b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.102+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.2 v2:172.21.15.159:6808/2619731047 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e115a800 2024-04-04T06:14:05.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.102+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:8 13.28 13:14000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115b400 con 0x55a0e115a800 2024-04-04T06:14:05.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.103+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.2 v2:172.21.15.159:6808/2619731047 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e115a800 2024-04-04T06:14:05.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.103+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:9 13.18 13:18000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115b000 con 0x55a0e1d62000 2024-04-04T06:14:05.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.104+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.7 v2:172.21.15.178:6810/1066817938 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1d62000 2024-04-04T06:14:05.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.104+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:10 13.38 13:1c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115b800 con 0x55a0e1c8bc00 2024-04-04T06:14:05.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.105+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.5 v2:172.21.15.178:6802/63746765 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1c8bc00 2024-04-04T06:14:05.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.105+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:11 13.4 13:20000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115ac00 con 0x55a0e1cff800 2024-04-04T06:14:05.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.105+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.6 v2:172.21.15.178:6826/198949157 3 ==== osd_op_reply(11 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1cff800 2024-04-04T06:14:05.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.106+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:12 13.24 13:24000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115bc00 con 0x55a0e1cff800 2024-04-04T06:14:05.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.107+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.6 v2:172.21.15.178:6826/198949157 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1cff800 2024-04-04T06:14:05.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.107+0000 7fde2cc00640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x55a0e115b400 0x55a0e1153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.107+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:13 13.14 13:28000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115b000 con 0x55a0e115b400 2024-04-04T06:14:05.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.107+0000 7fde31c00640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x55a0e115b400 0x55a0e1153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.107+0000 7fde31c00640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x55a0e115b400 0x55a0e1153080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.108+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.0 v2:172.21.15.159:6800/455382518 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e115b400 2024-04-04T06:14:05.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.108+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:14 13.34 13:2c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115a400 con 0x55a0e1cff800 2024-04-04T06:14:05.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.109+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.6 v2:172.21.15.178:6826/198949157 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1cff800 2024-04-04T06:14:05.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.109+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:15 13.c 13:30000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115ac00 con 0x55a0e1d62400 2024-04-04T06:14:05.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.109+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.1 v2:172.21.15.159:6816/3765338297 2 ==== osd_op_reply(15 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e1d62400 2024-04-04T06:14:05.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.109+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:16 13.2c 13:34000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1cffc00 con 0x55a0e115a800 2024-04-04T06:14:05.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.110+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.2 v2:172.21.15.159:6808/2619731047 3 ==== osd_op_reply(16 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e115a800 2024-04-04T06:14:05.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.110+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:17 13.1c 13:38000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115bc00 con 0x55a0e1cff800 2024-04-04T06:14:05.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.111+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.6 v2:172.21.15.178:6826/198949157 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1cff800 2024-04-04T06:14:05.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.111+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:18 13.3c 13:3c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1cff400 con 0x55a0e115a800 2024-04-04T06:14:05.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.111+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.2 v2:172.21.15.159:6808/2619731047 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e115a800 2024-04-04T06:14:05.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.111+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:19 13.2 13:40000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1170000 con 0x55a0e1c8bc00 2024-04-04T06:14:05.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.112+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.5 v2:172.21.15.178:6802/63746765 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1c8bc00 2024-04-04T06:14:05.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.113+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:20 13.22 13:44000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e0dc1800 con 0x55a0e1d62400 2024-04-04T06:14:05.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.113+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.1 v2:172.21.15.159:6816/3765338297 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e1d62400 2024-04-04T06:14:05.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.113+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:21 13.12 13:48000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1170400 con 0x55a0e115b400 2024-04-04T06:14:05.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.113+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.0 v2:172.21.15.159:6800/455382518 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e115b400 2024-04-04T06:14:05.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.113+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:22 13.32 13:4c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115b000 con 0x55a0e115a800 2024-04-04T06:14:05.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.114+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.2 v2:172.21.15.159:6808/2619731047 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e115a800 2024-04-04T06:14:05.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.114+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:23 13.a 13:50000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1170800 con 0x55a0e115a800 2024-04-04T06:14:05.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.114+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.2 v2:172.21.15.159:6808/2619731047 6 ==== osd_op_reply(23 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e115a800 2024-04-04T06:14:05.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.114+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:24 13.2a 13:54000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115ac00 con 0x55a0e1d62400 2024-04-04T06:14:05.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.115+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.1 v2:172.21.15.159:6816/3765338297 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e1d62400 2024-04-04T06:14:05.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.115+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:25 13.1a 13:58000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1170c00 con 0x55a0e1d62400 2024-04-04T06:14:05.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.115+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.1 v2:172.21.15.159:6816/3765338297 5 ==== osd_op_reply(25 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e1d62400 2024-04-04T06:14:05.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.115+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:26 13.3a 13:5c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115b800 con 0x55a0e1c8bc00 2024-04-04T06:14:05.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.116+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.5 v2:172.21.15.178:6802/63746765 4 ==== osd_op_reply(26 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1c8bc00 2024-04-04T06:14:05.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.116+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:27 13.6 13:60000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1171000 con 0x55a0e115a000 2024-04-04T06:14:05.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.117+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.4 v2:172.21.15.178:6812/4035991429 2 ==== osd_op_reply(27 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e115a000 2024-04-04T06:14:05.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.117+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:28 13.26 13:64000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e0dc1800 con 0x55a0e1d62000 2024-04-04T06:14:05.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.118+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.7 v2:172.21.15.178:6810/1066817938 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1d62000 2024-04-04T06:14:05.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.119+0000 7fde2cc00640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55a0e1171400 0x55a0e1153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.119+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:29 13.16 13:68000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1170000 con 0x55a0e1171400 2024-04-04T06:14:05.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.119+0000 7fde33000640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55a0e1171400 0x55a0e1153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.119+0000 7fde33000640 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55a0e1171400 0x55a0e1153600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.120+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.3 v2:172.21.15.159:6820/2070760205 1 ==== osd_op_reply(29 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1171400 2024-04-04T06:14:05.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.120+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:30 13.36 13:6c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1d63c00 con 0x55a0e1cff800 2024-04-04T06:14:05.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.121+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.6 v2:172.21.15.178:6826/198949157 7 ==== osd_op_reply(30 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1cff800 2024-04-04T06:14:05.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.121+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:31 13.e 13:70000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1171800 con 0x55a0e1d62000 2024-04-04T06:14:05.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.122+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.7 v2:172.21.15.178:6810/1066817938 4 ==== osd_op_reply(31 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1d62000 2024-04-04T06:14:05.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.122+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:32 13.2e 13:74000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1170800 con 0x55a0e115a800 2024-04-04T06:14:05.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.123+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.2 v2:172.21.15.159:6808/2619731047 7 ==== osd_op_reply(32 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e115a800 2024-04-04T06:14:05.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.123+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:33 13.1e 13:78000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1171c00 con 0x55a0e115a000 2024-04-04T06:14:05.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.124+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.4 v2:172.21.15.178:6812/4035991429 3 ==== osd_op_reply(33 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e115a000 2024-04-04T06:14:05.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.124+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:34 13.3e 13:7c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1171800 con 0x55a0e1d62400 2024-04-04T06:14:05.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.124+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.1 v2:172.21.15.159:6816/3765338297 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e1d62400 2024-04-04T06:14:05.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.124+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:35 13.1 13:80000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1cfec00 con 0x55a0e1cff800 2024-04-04T06:14:05.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.125+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.6 v2:172.21.15.178:6826/198949157 8 ==== osd_op_reply(35 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1cff800 2024-04-04T06:14:05.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.125+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:36 13.21 13:84000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e0dc1800 con 0x55a0e115a000 2024-04-04T06:14:05.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.126+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.4 v2:172.21.15.178:6812/4035991429 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 89 out=49b] v85'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e115a000 2024-04-04T06:14:05.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.126+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:37 13.11 13:88000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1cfe800 con 0x55a0e1d62400 2024-04-04T06:14:05.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.127+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.1 v2:172.21.15.159:6816/3765338297 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e1d62400 2024-04-04T06:14:05.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.127+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:38 13.31 13:8c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115b000 con 0x55a0e1171400 2024-04-04T06:14:05.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.128+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.3 v2:172.21.15.159:6820/2070760205 2 ==== osd_op_reply(38 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1171400 2024-04-04T06:14:05.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.128+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:39 13.9 13:90000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1cffc00 con 0x55a0e1c8bc00 2024-04-04T06:14:05.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.129+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.5 v2:172.21.15.178:6802/63746765 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 89 out=49b] v85'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1c8bc00 2024-04-04T06:14:05.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.129+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:40 13.29 13:94000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115b800 con 0x55a0e1cff800 2024-04-04T06:14:05.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.130+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.6 v2:172.21.15.178:6826/198949157 9 ==== osd_op_reply(40 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1cff800 2024-04-04T06:14:05.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.130+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:41 13.19 13:98000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1cff400 con 0x55a0e1171400 2024-04-04T06:14:05.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.131+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.3 v2:172.21.15.159:6820/2070760205 3 ==== osd_op_reply(41 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1171400 2024-04-04T06:14:05.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.131+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:42 13.39 13:9c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1cfec00 con 0x55a0e115a800 2024-04-04T06:14:05.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.131+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.2 v2:172.21.15.159:6808/2619731047 8 ==== osd_op_reply(42 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e115a800 2024-04-04T06:14:05.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.131+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:43 13.5 13:a0000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1178000 con 0x55a0e115a000 2024-04-04T06:14:05.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.132+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.4 v2:172.21.15.178:6812/4035991429 5 ==== osd_op_reply(43 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e115a000 2024-04-04T06:14:05.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.132+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:44 13.25 13:a4000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1171000 con 0x55a0e115a000 2024-04-04T06:14:05.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.133+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.4 v2:172.21.15.178:6812/4035991429 6 ==== osd_op_reply(44 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e115a000 2024-04-04T06:14:05.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.133+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:45 13.15 13:a8000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1178400 con 0x55a0e115a000 2024-04-04T06:14:05.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.134+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.4 v2:172.21.15.178:6812/4035991429 7 ==== osd_op_reply(45 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e115a000 2024-04-04T06:14:05.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.135+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:46 13.35 13:ac000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1170000 con 0x55a0e1d62400 2024-04-04T06:14:05.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.135+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.1 v2:172.21.15.159:6816/3765338297 8 ==== osd_op_reply(46 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e1d62400 2024-04-04T06:14:05.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.135+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:47 13.d 13:b0000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1178800 con 0x55a0e115a000 2024-04-04T06:14:05.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.136+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.4 v2:172.21.15.178:6812/4035991429 8 ==== osd_op_reply(47 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e115a000 2024-04-04T06:14:05.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.136+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:48 13.2d 13:b4000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1171c00 con 0x55a0e115b400 2024-04-04T06:14:05.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.137+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.0 v2:172.21.15.159:6800/455382518 3 ==== osd_op_reply(48 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e115b400 2024-04-04T06:14:05.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.137+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:49 13.1d 13:b8000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1178c00 con 0x55a0e1d62000 2024-04-04T06:14:05.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.138+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.7 v2:172.21.15.178:6810/1066817938 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1d62000 2024-04-04T06:14:05.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.138+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:50 13.3d 13:bc000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1d63c00 con 0x55a0e1d62000 2024-04-04T06:14:05.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.139+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.7 v2:172.21.15.178:6810/1066817938 6 ==== osd_op_reply(50 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1d62000 2024-04-04T06:14:05.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.139+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:51 13.3 13:c0000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1179000 con 0x55a0e115b400 2024-04-04T06:14:05.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.140+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.0 v2:172.21.15.159:6800/455382518 4 ==== osd_op_reply(51 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e115b400 2024-04-04T06:14:05.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.140+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:52 13.23 13:c4000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115a400 con 0x55a0e1d62400 2024-04-04T06:14:05.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.141+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.1 v2:172.21.15.159:6816/3765338297 9 ==== osd_op_reply(52 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e1d62400 2024-04-04T06:14:05.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.141+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:53 13.13 13:c8000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1179400 con 0x55a0e1d62000 2024-04-04T06:14:05.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.142+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.7 v2:172.21.15.178:6810/1066817938 7 ==== osd_op_reply(53 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1d62000 2024-04-04T06:14:05.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.142+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:54 13.33 13:cc000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115bc00 con 0x55a0e1d62000 2024-04-04T06:14:05.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.143+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.7 v2:172.21.15.178:6810/1066817938 8 ==== osd_op_reply(54 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1d62000 2024-04-04T06:14:05.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.143+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:55 13.b 13:d0000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1179800 con 0x55a0e1c8bc00 2024-04-04T06:14:05.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.144+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.5 v2:172.21.15.178:6802/63746765 6 ==== osd_op_reply(55 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1c8bc00 2024-04-04T06:14:05.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.144+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:56 13.2b 13:d4000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1cffc00 con 0x55a0e115a800 2024-04-04T06:14:05.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.145+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.2 v2:172.21.15.159:6808/2619731047 9 ==== osd_op_reply(56 [pgnls-filter start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e115a800 2024-04-04T06:14:05.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.145+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:57 13.1b 13:d8000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1179c00 con 0x55a0e115a000 2024-04-04T06:14:05.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.146+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.4 v2:172.21.15.178:6812/4035991429 9 ==== osd_op_reply(57 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e115a000 2024-04-04T06:14:05.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.146+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:58 13.3b 13:dc000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1cfec00 con 0x55a0e115a000 2024-04-04T06:14:05.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.147+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.4 v2:172.21.15.178:6812/4035991429 10 ==== osd_op_reply(58 [pgnls-filter start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e115a000 2024-04-04T06:14:05.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.147+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:59 13.7 13:e0000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1178c00 con 0x55a0e1d62000 2024-04-04T06:14:05.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.148+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.7 v2:172.21.15.178:6810/1066817938 9 ==== osd_op_reply(59 [pgnls-filter start_epoch 89 out=81b] v89'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1d62000 2024-04-04T06:14:05.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.148+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:60 13.0 13:00000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1d62800 con 0x55a0e1d62000 2024-04-04T06:14:05.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.149+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.7 v2:172.21.15.178:6810/1066817938 10 ==== osd_op_reply(60 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1d62000 2024-04-04T06:14:05.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.150+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:61 13.20 13:04000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1170800 con 0x55a0e1cff800 2024-04-04T06:14:05.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.151+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.6 v2:172.21.15.178:6826/198949157 10 ==== osd_op_reply(61 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1cff800 2024-04-04T06:14:05.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.151+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:62 13.10 13:08000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1cfe800 con 0x55a0e1d62400 2024-04-04T06:14:05.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.152+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.1 v2:172.21.15.159:6816/3765338297 10 ==== osd_op_reply(62 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e1d62400 2024-04-04T06:14:05.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.152+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:63 13.30 13:0c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1170400 con 0x55a0e115a000 2024-04-04T06:14:05.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.153+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.4 v2:172.21.15.178:6812/4035991429 11 ==== osd_op_reply(63 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e115a000 2024-04-04T06:14:05.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.153+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:64 13.8 13:10000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115ac00 con 0x55a0e115a800 2024-04-04T06:14:05.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.153+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.2 v2:172.21.15.159:6808/2619731047 10 ==== osd_op_reply(64 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e115a800 2024-04-04T06:14:05.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.153+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:65 13.28 13:14000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1170c00 con 0x55a0e115a800 2024-04-04T06:14:05.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.154+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.2 v2:172.21.15.159:6808/2619731047 11 ==== osd_op_reply(65 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e115a800 2024-04-04T06:14:05.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.154+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:66 13.18 13:18000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115b000 con 0x55a0e1d62000 2024-04-04T06:14:05.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.155+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.7 v2:172.21.15.178:6810/1066817938 11 ==== osd_op_reply(66 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1d62000 2024-04-04T06:14:05.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.155+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:67 13.38 13:1c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1171800 con 0x55a0e1c8bc00 2024-04-04T06:14:05.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.156+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.5 v2:172.21.15.178:6802/63746765 7 ==== osd_op_reply(67 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1c8bc00 2024-04-04T06:14:05.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.156+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:68 13.4 13:20000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115b800 con 0x55a0e1cff800 2024-04-04T06:14:05.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.157+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.6 v2:172.21.15.178:6826/198949157 11 ==== osd_op_reply(68 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1cff800 2024-04-04T06:14:05.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.157+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:69 13.24 13:24000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1170000 con 0x55a0e1cff800 2024-04-04T06:14:05.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.158+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.6 v2:172.21.15.178:6826/198949157 12 ==== osd_op_reply(69 [pgnls-filter start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1cff800 2024-04-04T06:14:05.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.159+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:70 13.14 13:28000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e0dc1800 con 0x55a0e115b400 2024-04-04T06:14:05.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.159+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.0 v2:172.21.15.159:6800/455382518 5 ==== osd_op_reply(70 [pgnls-filter start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e115b400 2024-04-04T06:14:05.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.159+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:71 13.34 13:2c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1171c00 con 0x55a0e1cff800 2024-04-04T06:14:05.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.160+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.6 v2:172.21.15.178:6826/198949157 13 ==== osd_op_reply(71 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1cff800 2024-04-04T06:14:05.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.161+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:72 13.c 13:30000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1d63c00 con 0x55a0e1d62400 2024-04-04T06:14:05.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.161+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.1 v2:172.21.15.159:6816/3765338297 11 ==== osd_op_reply(72 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e1d62400 2024-04-04T06:14:05.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.161+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:73 13.2c 13:34000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1171000 con 0x55a0e115a800 2024-04-04T06:14:05.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.162+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.2 v2:172.21.15.159:6808/2619731047 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e115a800 2024-04-04T06:14:05.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.162+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:74 13.1c 13:38000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1179800 con 0x55a0e1cff800 2024-04-04T06:14:05.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.163+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.6 v2:172.21.15.178:6826/198949157 14 ==== osd_op_reply(74 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1cff800 2024-04-04T06:14:05.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.163+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:75 13.3c 13:3c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1cff400 con 0x55a0e115a800 2024-04-04T06:14:05.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.163+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.2 v2:172.21.15.159:6808/2619731047 13 ==== osd_op_reply(75 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e115a800 2024-04-04T06:14:05.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.164+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:76 13.2 13:40000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1178c00 con 0x55a0e1c8bc00 2024-04-04T06:14:05.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.165+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.5 v2:172.21.15.178:6802/63746765 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1c8bc00 2024-04-04T06:14:05.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.165+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:77 13.22 13:44000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1178400 con 0x55a0e1d62400 2024-04-04T06:14:05.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.165+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.1 v2:172.21.15.159:6816/3765338297 12 ==== osd_op_reply(77 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e1d62400 2024-04-04T06:14:05.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.166+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:78 13.12 13:48000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1179400 con 0x55a0e115b400 2024-04-04T06:14:05.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.166+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.0 v2:172.21.15.159:6800/455382518 6 ==== osd_op_reply(78 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e115b400 2024-04-04T06:14:05.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.166+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:79 13.32 13:4c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1178800 con 0x55a0e115a800 2024-04-04T06:14:05.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.167+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.2 v2:172.21.15.159:6808/2619731047 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e115a800 2024-04-04T06:14:05.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.167+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:80 13.a 13:50000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1178000 con 0x55a0e115a800 2024-04-04T06:14:05.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.168+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.2 v2:172.21.15.159:6808/2619731047 15 ==== osd_op_reply(80 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e115a800 2024-04-04T06:14:05.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.168+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:81 13.2a 13:54000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e117e000 con 0x55a0e1d62400 2024-04-04T06:14:05.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.168+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.1 v2:172.21.15.159:6816/3765338297 13 ==== osd_op_reply(81 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e1d62400 2024-04-04T06:14:05.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.169+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:82 13.1a 13:58000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1170c00 con 0x55a0e1d62400 2024-04-04T06:14:05.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.169+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.1 v2:172.21.15.159:6816/3765338297 14 ==== osd_op_reply(82 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e1d62400 2024-04-04T06:14:05.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.169+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:83 13.3a 13:5c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e117e400 con 0x55a0e1c8bc00 2024-04-04T06:14:05.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.170+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.5 v2:172.21.15.178:6802/63746765 9 ==== osd_op_reply(83 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1c8bc00 2024-04-04T06:14:05.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.170+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:84 13.6 13:60000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115bc00 con 0x55a0e115a000 2024-04-04T06:14:05.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.172+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.4 v2:172.21.15.178:6812/4035991429 12 ==== osd_op_reply(84 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e115a000 2024-04-04T06:14:05.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.172+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:85 13.26 13:64000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e117e800 con 0x55a0e1d62000 2024-04-04T06:14:05.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.173+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.7 v2:172.21.15.178:6810/1066817938 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1d62000 2024-04-04T06:14:05.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.173+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:86 13.16 13:68000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115ac00 con 0x55a0e1171400 2024-04-04T06:14:05.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.174+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.3 v2:172.21.15.159:6820/2070760205 4 ==== osd_op_reply(86 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1171400 2024-04-04T06:14:05.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.174+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:87 13.36 13:6c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e117ec00 con 0x55a0e1cff800 2024-04-04T06:14:05.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.175+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.6 v2:172.21.15.178:6826/198949157 15 ==== osd_op_reply(87 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1cff800 2024-04-04T06:14:05.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.175+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:88 13.e 13:70000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1d62800 con 0x55a0e1d62000 2024-04-04T06:14:05.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.176+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.7 v2:172.21.15.178:6810/1066817938 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1d62000 2024-04-04T06:14:05.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.176+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:89 13.2e 13:74000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e117f000 con 0x55a0e115a800 2024-04-04T06:14:05.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.177+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.2 v2:172.21.15.159:6808/2619731047 16 ==== osd_op_reply(89 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e115a800 2024-04-04T06:14:05.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.177+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:90 13.1e 13:78000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1cffc00 con 0x55a0e115a000 2024-04-04T06:14:05.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.178+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.4 v2:172.21.15.178:6812/4035991429 13 ==== osd_op_reply(90 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e115a000 2024-04-04T06:14:05.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.178+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:91 13.3e 13:7c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e117f400 con 0x55a0e1d62400 2024-04-04T06:14:05.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.179+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.1 v2:172.21.15.159:6816/3765338297 15 ==== osd_op_reply(91 [pgnls-filter start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e1d62400 2024-04-04T06:14:05.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.180+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:92 13.1 13:80000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115b800 con 0x55a0e1cff800 2024-04-04T06:14:05.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.181+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.6 v2:172.21.15.178:6826/198949157 16 ==== osd_op_reply(92 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1cff800 2024-04-04T06:14:05.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.181+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:93 13.21 13:84000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e117f800 con 0x55a0e115a000 2024-04-04T06:14:05.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.182+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.4 v2:172.21.15.178:6812/4035991429 14 ==== osd_op_reply(93 [pgnls-filter start_epoch 89 out=49b] v85'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e115a000 2024-04-04T06:14:05.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.182+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:94 13.11 13:88000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115bc00 con 0x55a0e1d62400 2024-04-04T06:14:05.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.183+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.1 v2:172.21.15.159:6816/3765338297 16 ==== osd_op_reply(94 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e1d62400 2024-04-04T06:14:05.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.183+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:95 13.31 13:8c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e117fc00 con 0x55a0e1171400 2024-04-04T06:14:05.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.184+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.3 v2:172.21.15.159:6820/2070760205 5 ==== osd_op_reply(95 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1171400 2024-04-04T06:14:05.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.184+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:96 13.9 13:90000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115b000 con 0x55a0e1c8bc00 2024-04-04T06:14:05.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.185+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.5 v2:172.21.15.178:6802/63746765 10 ==== osd_op_reply(96 [pgnls-filter start_epoch 89 out=49b] v85'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1c8bc00 2024-04-04T06:14:05.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.185+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:97 13.29 13:94000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e117e400 con 0x55a0e1cff800 2024-04-04T06:14:05.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.186+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.6 v2:172.21.15.178:6826/198949157 17 ==== osd_op_reply(97 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1cff800 2024-04-04T06:14:05.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.186+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:98 13.19 13:98000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115a400 con 0x55a0e1171400 2024-04-04T06:14:05.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.187+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.3 v2:172.21.15.159:6820/2070760205 6 ==== osd_op_reply(98 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1171400 2024-04-04T06:14:05.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.187+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:99 13.39 13:9c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e117e000 con 0x55a0e115a800 2024-04-04T06:14:05.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.188+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.2 v2:172.21.15.159:6808/2619731047 17 ==== osd_op_reply(99 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e115a800 2024-04-04T06:14:05.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.188+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:100 13.5 13:a0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1170800 con 0x55a0e115a000 2024-04-04T06:14:05.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.189+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.4 v2:172.21.15.178:6812/4035991429 15 ==== osd_op_reply(100 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e115a000 2024-04-04T06:14:05.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.189+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:101 13.25 13:a4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1d63c00 con 0x55a0e115a000 2024-04-04T06:14:05.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.190+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.4 v2:172.21.15.178:6812/4035991429 16 ==== osd_op_reply(101 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e115a000 2024-04-04T06:14:05.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.190+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:102 13.15 13:a8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1170400 con 0x55a0e115a000 2024-04-04T06:14:05.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.192+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.4 v2:172.21.15.178:6812/4035991429 17 ==== osd_op_reply(102 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e115a000 2024-04-04T06:14:05.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.192+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:103 13.35 13:ac000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e0dc1800 con 0x55a0e1d62400 2024-04-04T06:14:05.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.193+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.1 v2:172.21.15.159:6816/3765338297 17 ==== osd_op_reply(103 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e1d62400 2024-04-04T06:14:05.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.193+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:104 13.d 13:b0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1170000 con 0x55a0e115a000 2024-04-04T06:14:05.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.194+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.4 v2:172.21.15.178:6812/4035991429 18 ==== osd_op_reply(104 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e115a000 2024-04-04T06:14:05.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.194+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:105 13.2d 13:b4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1cfec00 con 0x55a0e115b400 2024-04-04T06:14:05.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.195+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.0 v2:172.21.15.159:6800/455382518 7 ==== osd_op_reply(105 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e115b400 2024-04-04T06:14:05.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.196+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:106 13.1d 13:b8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1171c00 con 0x55a0e1d62000 2024-04-04T06:14:05.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.197+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.7 v2:172.21.15.178:6810/1066817938 14 ==== osd_op_reply(106 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1d62000 2024-04-04T06:14:05.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.197+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:107 13.3d 13:bc000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1cff400 con 0x55a0e1d62000 2024-04-04T06:14:05.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.198+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.7 v2:172.21.15.178:6810/1066817938 15 ==== osd_op_reply(107 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1d62000 2024-04-04T06:14:05.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.198+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:108 13.3 13:c0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1171800 con 0x55a0e115b400 2024-04-04T06:14:05.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.199+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.0 v2:172.21.15.159:6800/455382518 8 ==== osd_op_reply(108 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e115b400 2024-04-04T06:14:05.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.199+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:109 13.23 13:c4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1cfe800 con 0x55a0e1d62400 2024-04-04T06:14:05.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.200+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.1 v2:172.21.15.159:6816/3765338297 18 ==== osd_op_reply(109 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e1d62400 2024-04-04T06:14:05.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.200+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:110 13.13 13:c8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1171000 con 0x55a0e1d62000 2024-04-04T06:14:05.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.201+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.7 v2:172.21.15.178:6810/1066817938 16 ==== osd_op_reply(110 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1d62000 2024-04-04T06:14:05.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.201+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:111 13.33 13:cc000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1d63c00 con 0x55a0e1d62000 2024-04-04T06:14:05.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.203+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.7 v2:172.21.15.178:6810/1066817938 17 ==== osd_op_reply(111 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1d62000 2024-04-04T06:14:05.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.203+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:112 13.b 13:d0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1d62800 con 0x55a0e1c8bc00 2024-04-04T06:14:05.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.204+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.5 v2:172.21.15.178:6802/63746765 11 ==== osd_op_reply(112 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1c8bc00 2024-04-04T06:14:05.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.204+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:113 13.2b 13:d4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1170800 con 0x55a0e115a800 2024-04-04T06:14:05.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.205+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.2 v2:172.21.15.159:6808/2619731047 18 ==== osd_op_reply(113 [pgnls-filter start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e115a800 2024-04-04T06:14:05.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.205+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:114 13.1b 13:d8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1d63c00 con 0x55a0e115a000 2024-04-04T06:14:05.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.206+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.4 v2:172.21.15.178:6812/4035991429 19 ==== osd_op_reply(114 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e115a000 2024-04-04T06:14:05.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.206+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:115 13.3b 13:dc000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115ac00 con 0x55a0e115a000 2024-04-04T06:14:05.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.207+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.4 v2:172.21.15.178:6812/4035991429 20 ==== osd_op_reply(115 [pgnls-filter start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e115a000 2024-04-04T06:14:05.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.207+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:116 13.7 13:e0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1171c00 con 0x55a0e1d62000 2024-04-04T06:14:05.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.209+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.7 v2:172.21.15.178:6810/1066817938 18 ==== osd_op_reply(116 [pgnls-filter start_epoch 89 out=81b] v89'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1d62000 2024-04-04T06:14:05.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.209+0000 7fde34c49740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-04-04T06:14:05.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.209+0000 7fde34c49740 10 datascan.operator(): handling object 10000000001.0 2024-04-04T06:14:05.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.209+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:117 13.7 13:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1d63000 con 0x55a0e1d62000 2024-04-04T06:14:05.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.210+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.7 v2:172.21.15.178:6810/1066817938 19 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) ==== 416+0+157 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1d62000 2024-04-04T06:14:05.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.210+0000 7fde34c49740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-04-04T06:14:05.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.210+0000 7fde34c49740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-04-04T06:14:05.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.210+0000 7fde34c49740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-04-04T06:14:05.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.210+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:118 12.10 12:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1d63400 con 0x55a0e115b400 2024-04-04T06:14:05.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.211+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.0 v2:172.21.15.159:6800/455382518 9 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) ==== 164+0+59 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e115b400 2024-04-04T06:14:05.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.211+0000 7fde34c49740 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-04-04T06:14:05.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.211+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1d63800 con 0x55a0e1cff800 2024-04-04T06:14:05.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.212+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.6 v2:172.21.15.178:6826/198949157 18 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=581b] v0'0 uv9 ondisk = 0) ==== 154+0+581 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1cff800 2024-04-04T06:14:05.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.213+0000 7fde34c49740 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-04-04T06:14:05.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.213+0000 7fde34c49740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-04-04T06:14:05.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.213+0000 7fde34c49740 20 datascan.get_frag_of: frag is * 2024-04-04T06:14:05.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.213+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:120 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1168000 con 0x55a0e115b400 2024-04-04T06:14:05.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.213+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.0 v2:172.21.15.159:6800/455382518 10 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e115b400 2024-04-04T06:14:05.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.213+0000 7fde34c49740 10 datascan.find_or_create_dirfrag: read_version = 17 2024-04-04T06:14:05.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.213+0000 7fde34c49740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-04-04T06:14:05.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.213+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:121 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1168400 con 0x55a0e115b400 2024-04-04T06:14:05.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.214+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.0 v2:172.21.15.159:6800/455382518 11 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e115b400 2024-04-04T06:14:05.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.214+0000 7fde34c49740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-04T06:14:05.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.214+0000 7fde34c49740 20 datascan.read_dentry: file_head not found in result 2024-04-04T06:14:05.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.214+0000 7fde34c49740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-04-04T06:14:05.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.214+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:122 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1168800 con 0x55a0e115b400 2024-04-04T06:14:05.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.218+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.0 v2:172.21.15.159:6800/455382518 12 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v89'18 uv18 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e115b400 2024-04-04T06:14:05.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.218+0000 7fde34c49740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-04T06:14:05.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.218+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:123 13.27 13:e4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1168c00 con 0x55a0e1d62400 2024-04-04T06:14:05.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.219+0000 7fde31c00640 1 -- 172.21.15.159:0/1477335233 <== osd.1 v2:172.21.15.159:6816/3765338297 19 ==== osd_op_reply(123 [pgnls-filter start_epoch 89 out=49b] v85'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42780 con 0x55a0e1d62400 2024-04-04T06:14:05.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.219+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:124 13.17 13:e8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1171000 con 0x55a0e1c8bc00 2024-04-04T06:14:05.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.220+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.5 v2:172.21.15.178:6802/63746765 12 ==== osd_op_reply(124 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1c8bc00 2024-04-04T06:14:05.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.220+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:125 13.37 13:ec000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115b800 con 0x55a0e1c8bc00 2024-04-04T06:14:05.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.221+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 <== osd.5 v2:172.21.15.178:6802/63746765 13 ==== osd_op_reply(125 [pgnls-filter start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42000 con 0x55a0e1c8bc00 2024-04-04T06:14:05.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.221+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:126 13.f 13:f0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e1cff400 con 0x55a0e1cff800 2024-04-04T06:14:05.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.222+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.6 v2:172.21.15.178:6826/198949157 19 ==== osd_op_reply(126 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1cff800 2024-04-04T06:14:05.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.223+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:127 13.2f 13:f4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e115a400 con 0x55a0e115a000 2024-04-04T06:14:05.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.224+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.4 v2:172.21.15.178:6812/4035991429 21 ==== osd_op_reply(127 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e115a000 2024-04-04T06:14:05.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.224+0000 7fde2c200640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:128 13.1f 13:f8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e117e800 con 0x55a0e1cff800 2024-04-04T06:14:05.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.225+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.6 v2:172.21.15.178:6826/198949157 20 ==== osd_op_reply(128 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1cff800 2024-04-04T06:14:05.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.225+0000 7fde2cc00640 1 -- 172.21.15.159:0/1477335233 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:129 13.3f 13:fc000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55a0e117ec00 con 0x55a0e1cff800 2024-04-04T06:14:05.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.226+0000 7fde33000640 1 -- 172.21.15.159:0/1477335233 <== osd.6 v2:172.21.15.178:6826/198949157 21 ==== osd_op_reply(129 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a0e1b42500 con 0x55a0e1cff800 2024-04-04T06:14:05.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.226+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x55a0e115b400 msgr2=0x55a0e1153080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.226+0000 7fde34c49740 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x55a0e115b400 0x55a0e1153080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.226+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x55a0e1d62400 msgr2=0x55a0e1152000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.226+0000 7fde34c49740 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x55a0e1d62400 0x55a0e1152000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.226+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x55a0e115a800 msgr2=0x55a0e1152b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.226+0000 7fde34c49740 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x55a0e115a800 0x55a0e1152b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.226+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55a0e1171400 msgr2=0x55a0e1153600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.226+0000 7fde34c49740 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55a0e1171400 0x55a0e1153600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.226+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x55a0e115a000 msgr2=0x55a0e1152580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.226+0000 7fde34c49740 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x55a0e115a000 0x55a0e1152580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.226+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 reap_dead start 2024-04-04T06:14:05.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.226+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x55a0e1c8bc00 msgr2=0x55a0e0f36100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.226+0000 7fde34c49740 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x55a0e1c8bc00 0x55a0e0f36100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.226+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x55a0e1cff800 msgr2=0x55a0e0f35b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.226+0000 7fde34c49740 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x55a0e1cff800 0x55a0e0f35b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.226+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x55a0e1d62000 msgr2=0x55a0e0f35600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.226+0000 7fde34c49740 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x55a0e1d62000 0x55a0e0f35600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.226+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 reap_dead start 2024-04-04T06:14:05.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.226+0000 7fde32600640 1 -- 172.21.15.159:0/1477335233 reap_dead start 2024-04-04T06:14:05.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.227+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x55a0e1cff000 msgr2=0x55a0e0f37700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.227+0000 7fde34c49740 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x55a0e1cff000 0x55a0e0f37700 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x55a0e1b404b0 tx=0x55a0e1b5a180 comp rx=0 tx=0).stop 2024-04-04T06:14:05.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.227+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e1c8b800 msgr2=0x55a0e0f36680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.227+0000 7fde34c49740 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e1c8b800 0x55a0e0f36680 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x55a0e1be1d40 tx=0x55a0e1b49bc0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.227+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 shutdown_connections 2024-04-04T06:14:05.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.227+0000 7fde34c49740 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e1c8b800 0x55a0e0f36680 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.227+0000 7fde34c49740 1 --2- 172.21.15.159:0/1477335233 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x55a0e1cff000 0x55a0e0f37700 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.227+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 >> 172.21.15.159:0/1477335233 conn(0x55a0e1b29800 msgr2=0x55a0e1b5f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:05.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.227+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 shutdown_connections 2024-04-04T06:14:05.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.228+0000 7fde34c49740 1 -- 172.21.15.159:0/1477335233 wait complete. 2024-04-04T06:14:05.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.229+0000 7fde34c49740 1 -- 172.21.15.159:0/2932260390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e1b29c00 msgr2=0x55a0e0f34580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.229+0000 7fde34c49740 1 --2- 172.21.15.159:0/2932260390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e1b29c00 0x55a0e0f34580 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x55a0e1be1950 tx=0x55a0e1b5a3c0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.229+0000 7fde34c49740 1 -- 172.21.15.159:0/2932260390 shutdown_connections 2024-04-04T06:14:05.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.229+0000 7fde34c49740 1 --2- 172.21.15.159:0/2932260390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a0e1b29400 0x55a0e0f34000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.229+0000 7fde34c49740 1 --2- 172.21.15.159:0/2932260390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e1b29c00 0x55a0e0f34580 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.229+0000 7fde34c49740 1 --2- 172.21.15.159:0/2932260390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a0e1b28800 0x55a0e0f35080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.229+0000 7fde34c49740 1 -- 172.21.15.159:0/2932260390 >> 172.21.15.159:0/2932260390 conn(0x55a0e0dc1000 msgr2=0x55a0e1b5e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:05.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.229+0000 7fde34c49740 1 -- 172.21.15.159:0/2932260390 shutdown_connections 2024-04-04T06:14:05.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.229+0000 7fde34c49740 1 -- 172.21.15.159:0/2932260390 wait complete. 2024-04-04T06:14:05.238 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.263350, result: 2024-04-04T06:14:05.239 DEBUG:teuthology.orchestra.run.smithi159:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-04T06:14:05.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.350+0000 7fa2de831740 1 -- start start 2024-04-04T06:14:05.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.350+0000 7fa2de831740 1 Processor -- start 2024-04-04T06:14:05.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.351+0000 7fa2de831740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ed355cc00 0x556ed432e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.351+0000 7fa2de831740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556ed355d800 0x556ed432e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.351+0000 7fa2de831740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556ed355dc00 0x556ed432eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.351+0000 7fa2de831740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556ed36d8180 con 0x556ed355cc00 2024-04-04T06:14:05.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.351+0000 7fa2de831740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556ed35bc900 con 0x556ed355d800 2024-04-04T06:14:05.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.351+0000 7fa2de831740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x556ed35bc480 con 0x556ed355dc00 2024-04-04T06:14:05.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.352+0000 7fa2dba00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ed355cc00 0x556ed432e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.352+0000 7fa2dba00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ed355cc00 0x556ed432e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33652/0 (socket says 172.21.15.159:33652) 2024-04-04T06:14:05.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.352+0000 7fa2db000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556ed355d800 0x556ed432e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.352+0000 7fa2dba00640 1 -- 172.21.15.159:0/3586335996 learned_addr learned my addr 172.21.15.159:0/3586335996 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:05.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.352+0000 7fa2dc400640 1 --2- 172.21.15.159:0/3586335996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556ed355dc00 0x556ed432eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.352+0000 7fa2dba00640 1 -- 172.21.15.159:0/3586335996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556ed355dc00 msgr2=0x556ed432eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:05.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.352+0000 7fa2dba00640 1 --2- 172.21.15.159:0/3586335996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556ed355dc00 0x556ed432eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.352+0000 7fa2dba00640 1 -- 172.21.15.159:0/3586335996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556ed355d800 msgr2=0x556ed432e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:05.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.352+0000 7fa2dba00640 1 --2- 172.21.15.159:0/3586335996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556ed355d800 0x556ed432e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.352+0000 7fa2dba00640 1 -- 172.21.15.159:0/3586335996 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ed3589680 con 0x556ed355cc00 2024-04-04T06:14:05.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.352+0000 7fa2db000640 1 --2- 172.21.15.159:0/3586335996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556ed355d800 0x556ed432e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:14:05.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.352+0000 7fa2dc400640 1 --2- 172.21.15.159:0/3586335996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556ed355dc00 0x556ed432eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:14:05.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.352+0000 7fa2dba00640 1 --2- 172.21.15.159:0/3586335996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ed355cc00 0x556ed432e000 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x556ed43810e0 tx=0x556ed42f6120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6206f909ea3b59 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.353+0000 7fa2d8800640 1 -- 172.21.15.159:0/3586335996 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556ed35829c0 con 0x556ed355cc00 2024-04-04T06:14:05.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.353+0000 7fa2d8800640 1 -- 172.21.15.159:0/3586335996 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556ed35a61c0 con 0x556ed355cc00 2024-04-04T06:14:05.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.353+0000 7fa2d8800640 1 -- 172.21.15.159:0/3586335996 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556ed3582680 con 0x556ed355cc00 2024-04-04T06:14:05.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.353+0000 7fa2de831740 1 -- 172.21.15.159:0/3586335996 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ed3589860 con 0x556ed355cc00 2024-04-04T06:14:05.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.353+0000 7fa2de831740 1 -- 172.21.15.159:0/3586335996 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({fsmap=0}) -- 0x556ed35892c0 con 0x556ed355cc00 2024-04-04T06:14:05.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.353+0000 7fa2de831740 4 waiting for MDS map... 2024-04-04T06:14:05.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.353+0000 7fa2d8800640 1 -- 172.21.15.159:0/3586335996 <== mon.0 v2:172.21.15.159:3300/0 4 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x556ed442a200 con 0x556ed355cc00 2024-04-04T06:14:05.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.353+0000 7fa2d8800640 1 -- 172.21.15.159:0/3586335996 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 70) ==== 2885+0+0 (secure 0 0 0) 0x556ed35b5180 con 0x556ed355cc00 2024-04-04T06:14:05.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.353+0000 7fa2de831740 4 Got MDS map 70 2024-04-04T06:14:05.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.353+0000 7fa2de831740 4 datascan.main: Using metadata pool output 2024-04-04T06:14:05.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.353+0000 7fa2de831740 4 datascan.main: connecting to RADOS... 2024-04-04T06:14:05.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.354+0000 7fa2de831740 1 Processor -- start 2024-04-04T06:14:05.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.354+0000 7fa2de831740 1 -- start start 2024-04-04T06:14:05.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.355+0000 7fa2de831740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556ed4488000 0x556ed432f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.355+0000 7fa2de831740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556ed4488400 0x556ed432f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.355+0000 7fa2de831740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ed4488800 0x556ed432fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.355+0000 7fa2de831740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556ed35bc180 con 0x556ed4488800 2024-04-04T06:14:05.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.355+0000 7fa2de831740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556ed36d8d80 con 0x556ed4488000 2024-04-04T06:14:05.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.355+0000 7fa2de831740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x556ed36d8f00 con 0x556ed4488400 2024-04-04T06:14:05.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.356+0000 7fa2db000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ed4488800 0x556ed432fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.356+0000 7fa2db000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ed4488800 0x556ed432fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33668/0 (socket says 172.21.15.159:33668) 2024-04-04T06:14:05.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.356+0000 7fa2db000640 1 -- 172.21.15.159:0/480163313 learned_addr learned my addr 172.21.15.159:0/480163313 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:05.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.356+0000 7fa2dba00640 1 --2- 172.21.15.159:0/480163313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556ed4488400 0x556ed432f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.356+0000 7fa2db000640 1 -- 172.21.15.159:0/480163313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556ed4488400 msgr2=0x556ed432f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:05.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.356+0000 7fa2dc400640 1 --2- 172.21.15.159:0/480163313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556ed4488000 0x556ed432f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.356+0000 7fa2db000640 1 --2- 172.21.15.159:0/480163313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556ed4488400 0x556ed432f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.356+0000 7fa2db000640 1 -- 172.21.15.159:0/480163313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556ed4488000 msgr2=0x556ed432f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:05.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.356+0000 7fa2db000640 1 --2- 172.21.15.159:0/480163313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556ed4488000 0x556ed432f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.356+0000 7fa2db000640 1 -- 172.21.15.159:0/480163313 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ed35894a0 con 0x556ed4488800 2024-04-04T06:14:05.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.356+0000 7fa2db000640 1 --2- 172.21.15.159:0/480163313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ed4488800 0x556ed432fb80 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x556ed4346ea0 tx=0x556ed4338c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0482bd7e382403 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.357+0000 7fa2d6000640 1 -- 172.21.15.159:0/480163313 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556ed35829c0 con 0x556ed4488800 2024-04-04T06:14:05.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.357+0000 7fa2d6000640 1 -- 172.21.15.159:0/480163313 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556ed35a6c40 con 0x556ed4488800 2024-04-04T06:14:05.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.357+0000 7fa2d6000640 1 -- 172.21.15.159:0/480163313 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556ed3582680 con 0x556ed4488800 2024-04-04T06:14:05.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.357+0000 7fa2de831740 1 -- 172.21.15.159:0/480163313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ed4488800 msgr2=0x556ed432fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.357+0000 7fa2de831740 1 --2- 172.21.15.159:0/480163313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ed4488800 0x556ed432fb80 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x556ed4346ea0 tx=0x556ed4338c00 comp rx=0 tx=0).stop 2024-04-04T06:14:05.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.357+0000 7fa2de831740 1 -- 172.21.15.159:0/480163313 shutdown_connections 2024-04-04T06:14:05.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.357+0000 7fa2de831740 1 --2- 172.21.15.159:0/480163313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ed4488800 0x556ed432fb80 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.357+0000 7fa2de831740 1 --2- 172.21.15.159:0/480163313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556ed4488400 0x556ed432f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.357+0000 7fa2de831740 1 --2- 172.21.15.159:0/480163313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556ed4488000 0x556ed432f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.357+0000 7fa2de831740 1 -- 172.21.15.159:0/480163313 >> 172.21.15.159:0/480163313 conn(0x556ed433bc00 msgr2=0x556ed3649000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:05.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.357+0000 7fa2de831740 1 -- 172.21.15.159:0/480163313 shutdown_connections 2024-04-04T06:14:05.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.357+0000 7fa2de831740 1 -- 172.21.15.159:0/480163313 wait complete. 2024-04-04T06:14:05.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.358+0000 7fa2de831740 1 Processor -- start 2024-04-04T06:14:05.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.358+0000 7fa2de831740 1 -- start start 2024-04-04T06:14:05.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.359+0000 7fa2de831740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556ed4488000 0x556ed432f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.359+0000 7fa2de831740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556ed4488400 0x556ed432f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.359+0000 7fa2de831740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ed4488800 0x556ed432fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.359+0000 7fa2de831740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556ed36d9200 con 0x556ed4488800 2024-04-04T06:14:05.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.359+0000 7fa2de831740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556ed36d9080 con 0x556ed4488400 2024-04-04T06:14:05.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.359+0000 7fa2de831740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x556ed36d9380 con 0x556ed4488000 2024-04-04T06:14:05.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.359+0000 7fa2db000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ed4488800 0x556ed432fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.360+0000 7fa2db000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ed4488800 0x556ed432fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33682/0 (socket says 172.21.15.159:33682) 2024-04-04T06:14:05.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.360+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 learned_addr learned my addr 172.21.15.159:0/578846420 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:05.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.360+0000 7fa2dc400640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556ed4488000 0x556ed432f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.360+0000 7fa2dba00640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556ed4488400 0x556ed432f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.360+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556ed4488000 msgr2=0x556ed432f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.360+0000 7fa2db000640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556ed4488000 0x556ed432f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.360+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556ed4488400 msgr2=0x556ed432f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.360+0000 7fa2db000640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556ed4488400 0x556ed432f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.360+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ed35890e0 con 0x556ed4488800 2024-04-04T06:14:05.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.360+0000 7fa2dba00640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556ed4488400 0x556ed432f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:05.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.360+0000 7fa2db000640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ed4488800 0x556ed432fb80 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x556ed4347b60 tx=0x556ed42f4ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.360+0000 7fa2d4200640 1 -- 172.21.15.159:0/578846420 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556ed35824e0 con 0x556ed4488800 2024-04-04T06:14:05.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.361+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556ed3589a40 con 0x556ed4488800 2024-04-04T06:14:05.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.361+0000 7fa2d4200640 1 -- 172.21.15.159:0/578846420 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556ed35a6540 con 0x556ed4488800 2024-04-04T06:14:05.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.361+0000 7fa2d4200640 1 -- 172.21.15.159:0/578846420 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556ed35829c0 con 0x556ed4488800 2024-04-04T06:14:05.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.361+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ed35883c0 con 0x556ed4488800 2024-04-04T06:14:05.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.361+0000 7fa2de831740 4 datascan.init: resolving metadata pool 12 2024-04-04T06:14:05.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.362+0000 7fa2d4200640 1 -- 172.21.15.159:0/578846420 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x556ed4426c00 con 0x556ed4488800 2024-04-04T06:14:05.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.363+0000 7fa2d4200640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x556ed38c8000 0x556ed4330100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.363+0000 7fa2d4200640 1 -- 172.21.15.159:0/578846420 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x556ed448f400 con 0x556ed4488800 2024-04-04T06:14:05.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.363+0000 7fa2de831740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-04T06:14:05.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.363+0000 7fa2de831740 4 datascan.main: resolving metadata pool 12 2024-04-04T06:14:05.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.363+0000 7fa2de831740 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x556ed4488c00 0x556ed4330680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.363+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:1 12.0 12:00000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed4489000 con 0x556ed4488c00 2024-04-04T06:14:05.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.364+0000 7fa2dc400640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x556ed38c8000 0x556ed4330100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.364+0000 7fa2dba00640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x556ed4488c00 0x556ed4330680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.365+0000 7fa2dba00640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x556ed4488c00 0x556ed4330680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.366+0000 7fa2dc400640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x556ed38c8000 0x556ed4330100 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x556ed42dccf0 tx=0x556ed4338f00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.366+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.4 v2:172.21.15.178:6812/4035991429 1 ==== osd_op_reply(1 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de000 con 0x556ed4488c00 2024-04-04T06:14:05.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.366+0000 7fa2dba00640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x556ed4489000 0x556ed38e4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.366+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:2 12.10 12:08000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38c9000 con 0x556ed4489000 2024-04-04T06:14:05.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.367+0000 7fa2dc400640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x556ed4489000 0x556ed38e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.368+0000 7fa2dc400640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x556ed4489000 0x556ed38e4000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.368+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 1 ==== osd_op_reply(2 [pgnls start_epoch 89 out=177b] v89'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.368+0000 7fa2dc400640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x556ed38c9000 0x556ed38e4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.368+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:3 12.8 12:10000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38c8c00 con 0x556ed38c9000 2024-04-04T06:14:05.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.368+0000 7fa2dba00640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x556ed38c9000 0x556ed38e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.369+0000 7fa2dba00640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x556ed38c9000 0x556ed38e4580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.369+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.3 v2:172.21.15.159:6820/2070760205 1 ==== osd_op_reply(3 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de000 con 0x556ed38c9000 2024-04-04T06:14:05.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.369+0000 7fa2dba00640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x556ed38c8c00 0x556ed38e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.369+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:4 12.18 12:18000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38c9400 con 0x556ed38c8c00 2024-04-04T06:14:05.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.370+0000 7fa2db000640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x556ed38c8c00 0x556ed38e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.370+0000 7fa2db000640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x556ed38c8c00 0x556ed38e4b00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.370+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 <== osd.1 v2:172.21.15.159:6816/3765338297 1 ==== osd_op_reply(4 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de780 con 0x556ed38c8c00 2024-04-04T06:14:05.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.370+0000 7fa2db000640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x556ed38c9400 0x556ed38e5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.370+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:5 12.4 12:20000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ee000 con 0x556ed38c9400 2024-04-04T06:14:05.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.371+0000 7fa2dc400640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x556ed38c9400 0x556ed38e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.372+0000 7fa2dc400640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x556ed38c9400 0x556ed38e5600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.373+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.6 v2:172.21.15.178:6826/198949157 1 ==== osd_op_reply(5 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9400 2024-04-04T06:14:05.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.373+0000 7fa2dc400640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x556ed38ee000 0x556ed38e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.373+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:6 12.14 12:28000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38eec00 con 0x556ed38ee000 2024-04-04T06:14:05.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.374+0000 7fa2dba00640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x556ed38ee000 0x556ed38e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.375+0000 7fa2dba00640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x556ed38ee000 0x556ed38e5b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.376+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.5 v2:172.21.15.178:6802/63746765 1 ==== osd_op_reply(6 [pgnls start_epoch 89 out=149b] v88'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x556ed42de000 con 0x556ed38ee000 2024-04-04T06:14:05.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.376+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:7 12.c 12:30000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38eec00 con 0x556ed4489000 2024-04-04T06:14:05.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.376+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 2 ==== osd_op_reply(7 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.376+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:8 12.1c 12:38000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38eec00 con 0x556ed4489000 2024-04-04T06:14:05.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.377+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 3 ==== osd_op_reply(8 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.377+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:9 12.2 12:40000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38eec00 con 0x556ed4488c00 2024-04-04T06:14:05.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.378+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.4 v2:172.21.15.178:6812/4035991429 2 ==== osd_op_reply(9 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de000 con 0x556ed4488c00 2024-04-04T06:14:05.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.378+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:10 12.12 12:48000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38eec00 con 0x556ed38c9000 2024-04-04T06:14:05.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.378+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.3 v2:172.21.15.159:6820/2070760205 2 ==== osd_op_reply(10 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de000 con 0x556ed38c9000 2024-04-04T06:14:05.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.378+0000 7fa2dba00640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x556ed38eec00 0x556ed38e5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.378+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:11 12.a 12:50000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38c9800 con 0x556ed38eec00 2024-04-04T06:14:05.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.379+0000 7fa2db000640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x556ed38eec00 0x556ed38e5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.379+0000 7fa2db000640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x556ed38eec00 0x556ed38e5080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.380+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 <== osd.2 v2:172.21.15.159:6808/2619731047 1 ==== osd_op_reply(11 [pgnls start_epoch 89 out=81b] v88'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x556ed42de780 con 0x556ed38eec00 2024-04-04T06:14:05.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.380+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:12 12.1a 12:58000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38c9800 con 0x556ed4488c00 2024-04-04T06:14:05.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.381+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.4 v2:172.21.15.178:6812/4035991429 3 ==== osd_op_reply(12 [pgnls start_epoch 89 out=49b] v86'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de000 con 0x556ed4488c00 2024-04-04T06:14:05.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.381+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:13 12.6 12:60000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38c9800 con 0x556ed38c8c00 2024-04-04T06:14:05.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.381+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 <== osd.1 v2:172.21.15.159:6816/3765338297 2 ==== osd_op_reply(13 [pgnls start_epoch 89 out=77b] v88'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x556ed42de780 con 0x556ed38c8c00 2024-04-04T06:14:05.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.381+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:14 12.16 12:68000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38c9800 con 0x556ed38c9400 2024-04-04T06:14:05.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.382+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.6 v2:172.21.15.178:6826/198949157 2 ==== osd_op_reply(14 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9400 2024-04-04T06:14:05.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.382+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:15 12.e 12:70000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38c9800 con 0x556ed4489000 2024-04-04T06:14:05.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.383+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 4 ==== osd_op_reply(15 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.383+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:16 12.1e 12:78000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38c9800 con 0x556ed4488c00 2024-04-04T06:14:05.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.384+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.4 v2:172.21.15.178:6812/4035991429 4 ==== osd_op_reply(16 [pgnls start_epoch 89 out=49b] v86'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de000 con 0x556ed4488c00 2024-04-04T06:14:05.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.384+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:17 12.1 12:80000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38c9800 con 0x556ed38c9400 2024-04-04T06:14:05.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.385+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.6 v2:172.21.15.178:6826/198949157 3 ==== osd_op_reply(17 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9400 2024-04-04T06:14:05.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.385+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:18 12.11 12:88000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38c9800 con 0x556ed38ee000 2024-04-04T06:14:05.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.386+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.5 v2:172.21.15.178:6802/63746765 2 ==== osd_op_reply(18 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de000 con 0x556ed38ee000 2024-04-04T06:14:05.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.386+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:19 12.9 12:90000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38c9800 con 0x556ed38eec00 2024-04-04T06:14:05.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.386+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 <== osd.2 v2:172.21.15.159:6808/2619731047 2 ==== osd_op_reply(19 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de780 con 0x556ed38eec00 2024-04-04T06:14:05.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.386+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:20 12.19 12:98000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38c9800 con 0x556ed4489000 2024-04-04T06:14:05.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.387+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 5 ==== osd_op_reply(20 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.387+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:21 12.5 12:a0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38c9800 con 0x556ed38c9400 2024-04-04T06:14:05.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.388+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.6 v2:172.21.15.178:6826/198949157 4 ==== osd_op_reply(21 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9400 2024-04-04T06:14:05.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.388+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:22 12.15 12:a8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38c9800 con 0x556ed38c9400 2024-04-04T06:14:05.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.389+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.6 v2:172.21.15.178:6826/198949157 5 ==== osd_op_reply(22 [pgnls start_epoch 89 out=97b] v88'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9400 2024-04-04T06:14:05.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.389+0000 7fa2dc400640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x556ed38c9800 0x556ed38e6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.389+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:23 12.d 12:b0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ef000 con 0x556ed38c9800 2024-04-04T06:14:05.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.390+0000 7fa2dc400640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x556ed38c9800 0x556ed38e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.390+0000 7fa2dc400640 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x556ed38c9800 0x556ed38e6100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.391+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.7 v2:172.21.15.178:6810/1066817938 1 ==== osd_op_reply(23 [pgnls start_epoch 89 out=74b] v84'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9800 2024-04-04T06:14:05.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.391+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:24 12.1d 12:b8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ef000 con 0x556ed4489000 2024-04-04T06:14:05.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.392+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 6 ==== osd_op_reply(24 [pgnls start_epoch 89 out=73b] v88'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.392+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:25 12.3 12:c0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ef000 con 0x556ed4489000 2024-04-04T06:14:05.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.393+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 7 ==== osd_op_reply(25 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.393+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:26 12.13 12:c8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ef000 con 0x556ed38c8c00 2024-04-04T06:14:05.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.393+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 <== osd.1 v2:172.21.15.159:6816/3765338297 3 ==== osd_op_reply(26 [pgnls start_epoch 89 out=98b] v88'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x556ed42de780 con 0x556ed38c8c00 2024-04-04T06:14:05.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.393+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:27 12.b 12:d0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ef000 con 0x556ed38c9000 2024-04-04T06:14:05.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.394+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.3 v2:172.21.15.159:6820/2070760205 3 ==== osd_op_reply(27 [pgnls start_epoch 89 out=76b] v88'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x556ed42de000 con 0x556ed38c9000 2024-04-04T06:14:05.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.394+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:28 12.1b 12:d8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ef000 con 0x556ed4488c00 2024-04-04T06:14:05.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.395+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.4 v2:172.21.15.178:6812/4035991429 5 ==== osd_op_reply(28 [pgnls start_epoch 89 out=73b] v88'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556ed42de000 con 0x556ed4488c00 2024-04-04T06:14:05.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.395+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:29 12.7 12:e0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ef000 con 0x556ed38c8c00 2024-04-04T06:14:05.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.396+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 <== osd.1 v2:172.21.15.159:6816/3765338297 4 ==== osd_op_reply(29 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556ed42de780 con 0x556ed38c8c00 2024-04-04T06:14:05.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.396+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:30 12.17 12:e8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ef000 con 0x556ed38c9400 2024-04-04T06:14:05.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.397+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.6 v2:172.21.15.178:6826/198949157 6 ==== osd_op_reply(30 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9400 2024-04-04T06:14:05.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.397+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:31 12.f 12:f0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ef000 con 0x556ed38c9400 2024-04-04T06:14:05.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.398+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.6 v2:172.21.15.178:6826/198949157 7 ==== osd_op_reply(31 [pgnls start_epoch 89 out=73b] v84'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9400 2024-04-04T06:14:05.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.398+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:32 12.1f 12:f8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ef000 con 0x556ed38c9400 2024-04-04T06:14:05.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.399+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.6 v2:172.21.15.178:6826/198949157 8 ==== osd_op_reply(32 [pgnls start_epoch 89 out=101b] v88'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9400 2024-04-04T06:14:05.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.399+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-04T06:14:05.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.399+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:33 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ef800 con 0x556ed4489000 2024-04-04T06:14:05.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.400+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.400+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-04T06:14:05.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.400+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:34 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed4489000 2024-04-04T06:14:05.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.400+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.400+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-04T06:14:05.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.400+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:35 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed3906000 con 0x556ed4489000 2024-04-04T06:14:05.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.401+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.401+0000 7fa2de831740 20 datascan.scan_links: injected ino detected 2024-04-04T06:14:05.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.401+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-04T06:14:05.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.401+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:36 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed3906400 con 0x556ed4489000 2024-04-04T06:14:05.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.401+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.401+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-04T06:14:05.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.401+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:37 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed3906800 con 0x556ed4489000 2024-04-04T06:14:05.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.402+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.402+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-04T06:14:05.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.402+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-04T06:14:05.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.402+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-04T06:14:05.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.402+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-04T06:14:05.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.402+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-04T06:14:05.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.402+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-04T06:14:05.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.402+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-04T06:14:05.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.402+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:38 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed3906c00 con 0x556ed38ee000 2024-04-04T06:14:05.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.403+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.5 v2:172.21.15.178:6802/63746765 3 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556ed42de000 con 0x556ed38ee000 2024-04-04T06:14:05.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.403+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-04T06:14:05.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.403+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:39 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed3907000 con 0x556ed4489000 2024-04-04T06:14:05.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.404+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.404+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-04-04T06:14:05.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.404+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:40 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed3907400 con 0x556ed38eec00 2024-04-04T06:14:05.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.404+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 <== osd.2 v2:172.21.15.159:6808/2619731047 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x556ed42de780 con 0x556ed38eec00 2024-04-04T06:14:05.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.405+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-04T06:14:05.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.405+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-04T06:14:05.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.405+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-04T06:14:05.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.405+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:41 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed3907800 con 0x556ed4489000 2024-04-04T06:14:05.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.405+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 14 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.405+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-04T06:14:05.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.405+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:42 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed3906000 con 0x556ed38c9400 2024-04-04T06:14:05.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.406+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.6 v2:172.21.15.178:6826/198949157 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9400 2024-04-04T06:14:05.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.407+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-04T06:14:05.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.407+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-04T06:14:05.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.407+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-04T06:14:05.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.407+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-04T06:14:05.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.407+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-04T06:14:05.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.407+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:43 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed3907c00 con 0x556ed4489000 2024-04-04T06:14:05.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.407+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.407+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-04T06:14:05.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.407+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-04T06:14:05.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.407+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-04T06:14:05.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.407+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:44 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ef400 con 0x556ed38c8c00 2024-04-04T06:14:05.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.408+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 <== osd.1 v2:172.21.15.159:6816/3765338297 5 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x556ed42de780 con 0x556ed38c8c00 2024-04-04T06:14:05.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.408+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-04T06:14:05.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.408+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-04T06:14:05.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.408+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object 200.00000002 2024-04-04T06:14:05.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.408+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-04T06:14:05.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.408+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-04T06:14:05.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.409+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:45 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ef000 con 0x556ed38c8c00 2024-04-04T06:14:05.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.409+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 <== osd.1 v2:172.21.15.159:6816/3765338297 6 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556ed42de780 con 0x556ed38c8c00 2024-04-04T06:14:05.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.409+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-04T06:14:05.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.409+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-04T06:14:05.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.409+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-04T06:14:05.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.409+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-04T06:14:05.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.409+0000 7fa2de831740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-04T06:14:05.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.409+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:46 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ef800 con 0x556ed38c9400 2024-04-04T06:14:05.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.411+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.6 v2:172.21.15.178:6826/198949157 10 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) ==== 154+0+1163 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9400 2024-04-04T06:14:05.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.411+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:47 12.0 12:00000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed4488c00 2024-04-04T06:14:05.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.412+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.4 v2:172.21.15.178:6812/4035991429 6 ==== osd_op_reply(47 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de000 con 0x556ed4488c00 2024-04-04T06:14:05.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.412+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:48 12.10 12:08000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed4489000 2024-04-04T06:14:05.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.413+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 16 ==== osd_op_reply(48 [pgnls start_epoch 89 out=177b] v89'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.413+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:49 12.8 12:10000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed38c9000 2024-04-04T06:14:05.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.414+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.3 v2:172.21.15.159:6820/2070760205 4 ==== osd_op_reply(49 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de000 con 0x556ed38c9000 2024-04-04T06:14:05.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.414+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:50 12.18 12:18000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed38c8c00 2024-04-04T06:14:05.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.414+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 <== osd.1 v2:172.21.15.159:6816/3765338297 7 ==== osd_op_reply(50 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de780 con 0x556ed38c8c00 2024-04-04T06:14:05.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.414+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:51 12.4 12:20000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed38c9400 2024-04-04T06:14:05.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.416+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.6 v2:172.21.15.178:6826/198949157 11 ==== osd_op_reply(51 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9400 2024-04-04T06:14:05.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.416+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:52 12.14 12:28000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed38ee000 2024-04-04T06:14:05.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.417+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.5 v2:172.21.15.178:6802/63746765 4 ==== osd_op_reply(52 [pgnls start_epoch 89 out=149b] v88'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x556ed42de000 con 0x556ed38ee000 2024-04-04T06:14:05.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.417+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:53 12.c 12:30000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed4489000 2024-04-04T06:14:05.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.418+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 17 ==== osd_op_reply(53 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.418+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:54 12.1c 12:38000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed4489000 2024-04-04T06:14:05.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.418+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 18 ==== osd_op_reply(54 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.418+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:55 12.2 12:40000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed4488c00 2024-04-04T06:14:05.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.419+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.4 v2:172.21.15.178:6812/4035991429 7 ==== osd_op_reply(55 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de000 con 0x556ed4488c00 2024-04-04T06:14:05.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.420+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:56 12.12 12:48000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed38c9000 2024-04-04T06:14:05.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.420+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.3 v2:172.21.15.159:6820/2070760205 5 ==== osd_op_reply(56 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de000 con 0x556ed38c9000 2024-04-04T06:14:05.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.420+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:57 12.a 12:50000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed38eec00 2024-04-04T06:14:05.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.421+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 <== osd.2 v2:172.21.15.159:6808/2619731047 4 ==== osd_op_reply(57 [pgnls start_epoch 89 out=81b] v88'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x556ed42de780 con 0x556ed38eec00 2024-04-04T06:14:05.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.421+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:58 12.1a 12:58000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed4488c00 2024-04-04T06:14:05.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.422+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.4 v2:172.21.15.178:6812/4035991429 8 ==== osd_op_reply(58 [pgnls start_epoch 89 out=49b] v86'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de000 con 0x556ed4488c00 2024-04-04T06:14:05.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.422+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:59 12.6 12:60000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed38c8c00 2024-04-04T06:14:05.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.423+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 <== osd.1 v2:172.21.15.159:6816/3765338297 8 ==== osd_op_reply(59 [pgnls start_epoch 89 out=77b] v88'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x556ed42de780 con 0x556ed38c8c00 2024-04-04T06:14:05.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.423+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:60 12.16 12:68000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed38c9400 2024-04-04T06:14:05.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.424+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.6 v2:172.21.15.178:6826/198949157 12 ==== osd_op_reply(60 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9400 2024-04-04T06:14:05.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.424+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:61 12.e 12:70000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed4489000 2024-04-04T06:14:05.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.425+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 19 ==== osd_op_reply(61 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.425+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:62 12.1e 12:78000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed4488c00 2024-04-04T06:14:05.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.426+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.4 v2:172.21.15.178:6812/4035991429 9 ==== osd_op_reply(62 [pgnls start_epoch 89 out=49b] v86'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de000 con 0x556ed4488c00 2024-04-04T06:14:05.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.426+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:63 12.1 12:80000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed38c9400 2024-04-04T06:14:05.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.427+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.6 v2:172.21.15.178:6826/198949157 13 ==== osd_op_reply(63 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9400 2024-04-04T06:14:05.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.427+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:64 12.11 12:88000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed38ee000 2024-04-04T06:14:05.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.428+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.5 v2:172.21.15.178:6802/63746765 5 ==== osd_op_reply(64 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de000 con 0x556ed38ee000 2024-04-04T06:14:05.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.429+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:65 12.9 12:90000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed38eec00 2024-04-04T06:14:05.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.429+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 <== osd.2 v2:172.21.15.159:6808/2619731047 5 ==== osd_op_reply(65 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de780 con 0x556ed38eec00 2024-04-04T06:14:05.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.429+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:66 12.19 12:98000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed4489000 2024-04-04T06:14:05.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.430+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 20 ==== osd_op_reply(66 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.430+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:67 12.5 12:a0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed38c9400 2024-04-04T06:14:05.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.431+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.6 v2:172.21.15.178:6826/198949157 14 ==== osd_op_reply(67 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9400 2024-04-04T06:14:05.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.431+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:68 12.15 12:a8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed38c9400 2024-04-04T06:14:05.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.432+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.6 v2:172.21.15.178:6826/198949157 15 ==== osd_op_reply(68 [pgnls start_epoch 89 out=97b] v88'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9400 2024-04-04T06:14:05.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.432+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:69 12.d 12:b0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed38c9800 2024-04-04T06:14:05.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.433+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.7 v2:172.21.15.178:6810/1066817938 2 ==== osd_op_reply(69 [pgnls start_epoch 89 out=74b] v84'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9800 2024-04-04T06:14:05.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.433+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:70 12.1d 12:b8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed4489000 2024-04-04T06:14:05.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.434+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 21 ==== osd_op_reply(70 [pgnls start_epoch 89 out=73b] v88'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.434+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:71 12.3 12:c0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed4489000 2024-04-04T06:14:05.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.435+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 22 ==== osd_op_reply(71 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.435+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:72 12.13 12:c8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed38c8c00 2024-04-04T06:14:05.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.436+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 <== osd.1 v2:172.21.15.159:6816/3765338297 9 ==== osd_op_reply(72 [pgnls start_epoch 89 out=98b] v88'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x556ed42de780 con 0x556ed38c8c00 2024-04-04T06:14:05.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.437+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:73 12.b 12:d0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed38c9000 2024-04-04T06:14:05.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.437+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.3 v2:172.21.15.159:6820/2070760205 6 ==== osd_op_reply(73 [pgnls start_epoch 89 out=76b] v88'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x556ed42de000 con 0x556ed38c9000 2024-04-04T06:14:05.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.437+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:74 12.1b 12:d8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed4488c00 2024-04-04T06:14:05.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.438+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.4 v2:172.21.15.178:6812/4035991429 10 ==== osd_op_reply(74 [pgnls start_epoch 89 out=73b] v88'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556ed42de000 con 0x556ed4488c00 2024-04-04T06:14:05.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.438+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:75 12.7 12:e0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed38c8c00 2024-04-04T06:14:05.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.439+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 <== osd.1 v2:172.21.15.159:6816/3765338297 10 ==== osd_op_reply(75 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556ed42de780 con 0x556ed38c8c00 2024-04-04T06:14:05.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.439+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:76 12.17 12:e8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed38c9400 2024-04-04T06:14:05.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.440+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.6 v2:172.21.15.178:6826/198949157 16 ==== osd_op_reply(76 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9400 2024-04-04T06:14:05.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.440+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:77 12.f 12:f0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed38c9400 2024-04-04T06:14:05.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.441+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.6 v2:172.21.15.178:6826/198949157 17 ==== osd_op_reply(77 [pgnls start_epoch 89 out=73b] v84'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9400 2024-04-04T06:14:05.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.442+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:78 12.1f 12:f8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed38c9400 2024-04-04T06:14:05.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.442+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.6 v2:172.21.15.178:6826/198949157 18 ==== osd_op_reply(78 [pgnls start_epoch 89 out=101b] v88'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9400 2024-04-04T06:14:05.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.442+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-04T06:14:05.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.443+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:79 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed3907400 con 0x556ed4489000 2024-04-04T06:14:05.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.443+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 23 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.444+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-04T06:14:05.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.444+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:80 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed3906400 con 0x556ed4489000 2024-04-04T06:14:05.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.444+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 24 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.444+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-04T06:14:05.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.445+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:81 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed3906800 con 0x556ed4489000 2024-04-04T06:14:05.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.445+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 25 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.445+0000 7fa2de831740 20 datascan.scan_links: injected ino detected 2024-04-04T06:14:05.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.446+0000 7fa2de831740 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-04-04T06:14:05.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.446+0000 7fa2de831740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-04-04T06:14:05.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.446+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-04T06:14:05.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.446+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:82 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed3907000 con 0x556ed4489000 2024-04-04T06:14:05.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.446+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 26 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.446+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-04T06:14:05.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.446+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:83 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed3907800 con 0x556ed4489000 2024-04-04T06:14:05.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.447+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 27 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.447+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-04T06:14:05.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.447+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-04T06:14:05.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.447+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-04T06:14:05.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.447+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-04T06:14:05.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.447+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-04T06:14:05.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.447+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-04T06:14:05.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.447+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-04T06:14:05.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.447+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:84 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ee800 con 0x556ed38ee000 2024-04-04T06:14:05.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.449+0000 7fa2dba00640 1 -- 172.21.15.159:0/578846420 <== osd.5 v2:172.21.15.178:6802/63746765 6 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556ed42de000 con 0x556ed38ee000 2024-04-04T06:14:05.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.449+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-04T06:14:05.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.449+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:85 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ee400 con 0x556ed4489000 2024-04-04T06:14:05.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.450+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 28 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.450+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-04-04T06:14:05.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.450+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:86 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ef400 con 0x556ed38eec00 2024-04-04T06:14:05.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.451+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 <== osd.2 v2:172.21.15.159:6808/2619731047 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x556ed42de780 con 0x556ed38eec00 2024-04-04T06:14:05.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.451+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-04T06:14:05.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.451+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-04T06:14:05.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.451+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-04T06:14:05.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.451+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:87 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ef000 con 0x556ed4489000 2024-04-04T06:14:05.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.452+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 29 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.452+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-04T06:14:05.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.452+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:88 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed3907400 con 0x556ed38c9400 2024-04-04T06:14:05.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.453+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.6 v2:172.21.15.178:6826/198949157 19 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9400 2024-04-04T06:14:05.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.453+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-04T06:14:05.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.453+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-04T06:14:05.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.453+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-04T06:14:05.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.453+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-04T06:14:05.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.453+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-04T06:14:05.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.453+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed3906400 con 0x556ed4489000 2024-04-04T06:14:05.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.454+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 30 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.454+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-04T06:14:05.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.454+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-04T06:14:05.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.454+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-04T06:14:05.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.454+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:90 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed3906800 con 0x556ed38c8c00 2024-04-04T06:14:05.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.455+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 <== osd.1 v2:172.21.15.159:6816/3765338297 11 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x556ed42de780 con 0x556ed38c8c00 2024-04-04T06:14:05.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.456+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-04T06:14:05.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.456+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-04T06:14:05.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.456+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object 200.00000002 2024-04-04T06:14:05.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.456+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-04T06:14:05.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.456+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-04T06:14:05.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.456+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:91 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed3907000 con 0x556ed38c8c00 2024-04-04T06:14:05.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.457+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 <== osd.1 v2:172.21.15.159:6816/3765338297 12 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556ed42de780 con 0x556ed38c8c00 2024-04-04T06:14:05.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.457+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-04T06:14:05.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.457+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-04T06:14:05.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.457+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-04T06:14:05.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.457+0000 7fa2de831740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-04T06:14:05.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.457+0000 7fa2de831740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-04T06:14:05.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.457+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:92 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed3907800 con 0x556ed38c9400 2024-04-04T06:14:05.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.458+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.6 v2:172.21.15.178:6826/198949157 20 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) ==== 154+0+1163 (crc 0 0 0) 0x556ed42de500 con 0x556ed38c9400 2024-04-04T06:14:05.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.458+0000 7fa2de831740 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-04-04T06:14:05.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.458+0000 7fa2de831740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-04T06:14:05.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.458+0000 7fa2de831740 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-04-04T06:14:05.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.458+0000 7fa2de831740 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-04-04T06:14:05.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.458+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:93 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed3906000 con 0x556ed4489000 2024-04-04T06:14:05.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.459+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 31 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.459+0000 7fa2de831740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-04T06:14:05.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.459+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:94 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x556ed3907c00 con 0x556ed4489000 2024-04-04T06:14:05.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.463+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 32 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v89'19 uv19 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.463+0000 7fa2de831740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-04T06:14:05.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.463+0000 7fa2de831740 10 datascan.scan_links: processing 1 injected_inos 2024-04-04T06:14:05.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.463+0000 7fa2de831740 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-04-04T06:14:05.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.463+0000 7fa2de831740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-04-04T06:14:05.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.463+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:95 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ef800 con 0x556ed4489000 2024-04-04T06:14:05.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.464+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 33 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.464+0000 7fa2de831740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-04T06:14:05.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.464+0000 7fa2de831740 20 datascan.scan_links: first is now 2 2024-04-04T06:14:05.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.464+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:96 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x556ed38efc00 con 0x556ed4489000 2024-04-04T06:14:05.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.469+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.0 v2:172.21.15.159:6800/455382518 34 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v89'20 uv20 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x556ed42de500 con 0x556ed4489000 2024-04-04T06:14:05.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.469+0000 7fa2de831740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-04T06:14:05.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.469+0000 7fa2de831740 10 datascan.scan_links: updating inotable 2024-04-04T06:14:05.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.469+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:97 12.13 12:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ee400 con 0x556ed38c8c00 2024-04-04T06:14:05.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.470+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 <== osd.1 v2:172.21.15.159:6816/3765338297 13 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x556ed42de780 con 0x556ed38c8c00 2024-04-04T06:14:05.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.470+0000 7fa2de831740 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-04-04T06:14:05.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.470+0000 7fa2de831740 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-04-04T06:14:05.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.470+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:98 12.13 12:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ef000 con 0x556ed38c8c00 2024-04-04T06:14:05.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.474+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 <== osd.1 v2:172.21.15.159:6816/3765338297 14 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v89'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x556ed42de780 con 0x556ed38c8c00 2024-04-04T06:14:05.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.474+0000 7fa2de831740 10 datascan.scan_links: updating snaptable 2024-04-04T06:14:05.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.474+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:99 12.d 12:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556ed38ef800 con 0x556ed38c9800 2024-04-04T06:14:05.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.475+0000 7fa2dc400640 1 -- 172.21.15.159:0/578846420 <== osd.7 v2:172.21.15.178:6810/1066817938 3 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x556ed42de280 con 0x556ed38c9800 2024-04-04T06:14:05.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x556ed4489000 msgr2=0x556ed38e4000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2de831740 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x556ed4489000 0x556ed38e4000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x556ed38c8c00 msgr2=0x556ed38e4b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2de831740 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x556ed38c8c00 0x556ed38e4b00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x556ed38eec00 msgr2=0x556ed38e5080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2de831740 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x556ed38eec00 0x556ed38e5080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x556ed38c9000 msgr2=0x556ed38e4580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2de831740 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x556ed38c9000 0x556ed38e4580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x556ed4488c00 msgr2=0x556ed4330680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2de831740 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x556ed4488c00 0x556ed4330680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x556ed38ee000 msgr2=0x556ed38e5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2de831740 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x556ed38ee000 0x556ed38e5b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x556ed38c9400 msgr2=0x556ed38e5600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2de831740 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x556ed38c9400 0x556ed38e5600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 reap_dead start 2024-04-04T06:14:05.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x556ed38c9800 msgr2=0x556ed38e6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2de831740 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x556ed38c9800 0x556ed38e6100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2db000640 1 -- 172.21.15.159:0/578846420 reap_dead start 2024-04-04T06:14:05.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x556ed38c8000 msgr2=0x556ed4330100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2de831740 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x556ed38c8000 0x556ed4330100 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x556ed42dccf0 tx=0x556ed4338f00 comp rx=0 tx=0).stop 2024-04-04T06:14:05.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ed4488800 msgr2=0x556ed432fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.476+0000 7fa2de831740 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ed4488800 0x556ed432fb80 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x556ed4347b60 tx=0x556ed42f4ae0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.477+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 shutdown_connections 2024-04-04T06:14:05.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.477+0000 7fa2de831740 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ed4488800 0x556ed432fb80 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.477+0000 7fa2de831740 1 --2- 172.21.15.159:0/578846420 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x556ed38c8000 0x556ed4330100 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.477+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 >> 172.21.15.159:0/578846420 conn(0x556ed433bc00 msgr2=0x556ed3649000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:05.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.477+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 shutdown_connections 2024-04-04T06:14:05.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.477+0000 7fa2de831740 1 -- 172.21.15.159:0/578846420 wait complete. 2024-04-04T06:14:05.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.478+0000 7fa2de831740 1 -- 172.21.15.159:0/3586335996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ed355cc00 msgr2=0x556ed432e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.478+0000 7fa2de831740 1 --2- 172.21.15.159:0/3586335996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ed355cc00 0x556ed432e000 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x556ed43810e0 tx=0x556ed42f6120 comp rx=0 tx=0).stop 2024-04-04T06:14:05.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.478+0000 7fa2de831740 1 -- 172.21.15.159:0/3586335996 shutdown_connections 2024-04-04T06:14:05.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.478+0000 7fa2de831740 1 --2- 172.21.15.159:0/3586335996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x556ed355dc00 0x556ed432eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.478+0000 7fa2de831740 1 --2- 172.21.15.159:0/3586335996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556ed355d800 0x556ed432e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.478+0000 7fa2de831740 1 --2- 172.21.15.159:0/3586335996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ed355cc00 0x556ed432e000 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.478+0000 7fa2de831740 1 -- 172.21.15.159:0/3586335996 >> 172.21.15.159:0/3586335996 conn(0x556ed355d000 msgr2=0x556ed3648800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:05.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.479+0000 7fa2de831740 1 -- 172.21.15.159:0/3586335996 shutdown_connections 2024-04-04T06:14:05.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.479+0000 7fa2de831740 1 -- 172.21.15.159:0/3586335996 wait complete. 2024-04-04T06:14:05.484 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.244997, result: 2024-04-04T06:14:05.484 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-04T06:14:05.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.709+0000 7f4091800640 1 Processor -- start 2024-04-04T06:14:05.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.710+0000 7f4091800640 1 -- start start 2024-04-04T06:14:05.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.711+0000 7f4091800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f408c161f60 0x7f408c0d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.711+0000 7f4091800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f408c0b3170 0x7f408c0d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.711+0000 7f4091800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f408c0b3da0 0x7f408c0b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.711+0000 7f4091800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f408c0b5ce0 con 0x7f408c0b3da0 2024-04-04T06:14:05.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.711+0000 7f4091800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f408c0d27f0 con 0x7f408c0b3170 2024-04-04T06:14:05.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.711+0000 7f4091800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f408c0d2a90 con 0x7f408c161f60 2024-04-04T06:14:05.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.711+0000 7f408b400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f408c0b3da0 0x7f408c0b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.711+0000 7f408b400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f408c0b3da0 0x7f408c0b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33690/0 (socket says 172.21.15.159:33690) 2024-04-04T06:14:05.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.711+0000 7f408b400640 1 -- 172.21.15.159:0/599024049 learned_addr learned my addr 172.21.15.159:0/599024049 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:05.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.712+0000 7f408b400640 1 -- 172.21.15.159:0/599024049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f408c161f60 msgr2=0x7f408c0d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:05.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.712+0000 7f408b400640 1 --2- 172.21.15.159:0/599024049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f408c161f60 0x7f408c0d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.712+0000 7f408b400640 1 -- 172.21.15.159:0/599024049 >> v2:172.21.15.178:3300/0 conn(0x7f408c0b3170 msgr2=0x7f408c0d13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:05.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.712+0000 7f408a000640 1 --2- 172.21.15.159:0/599024049 >> v2:172.21.15.178:3300/0 conn(0x7f408c0b3170 0x7f408c0d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.712+0000 7f408b400640 1 --2- 172.21.15.159:0/599024049 >> v2:172.21.15.178:3300/0 conn(0x7f408c0b3170 0x7f408c0d13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.712+0000 7f408b400640 1 -- 172.21.15.159:0/599024049 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408c162340 con 0x7f408c0b3da0 2024-04-04T06:14:05.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.712+0000 7f408a000640 1 --2- 172.21.15.159:0/599024049 >> v2:172.21.15.178:3300/0 conn(0x7f408c0b3170 0x7f408c0d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:05.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.713+0000 7f408b400640 1 --2- 172.21.15.159:0/599024049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f408c0b3da0 0x7f408c0b4220 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f408c08c980 tx=0x7f408402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1c9550caad64fe server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.713+0000 7f4089600640 1 -- 172.21.15.159:0/599024049 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408402ca80 con 0x7f408c0b3da0 2024-04-04T06:14:05.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.713+0000 7f4089600640 1 -- 172.21.15.159:0/599024049 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f408402cc20 con 0x7f408c0b3da0 2024-04-04T06:14:05.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.713+0000 7f4089600640 1 -- 172.21.15.159:0/599024049 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40840343f0 con 0x7f408c0b3da0 2024-04-04T06:14:05.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.714+0000 7f4091800640 1 -- 172.21.15.159:0/599024049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f408c0b3da0 msgr2=0x7f408c0b4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.714+0000 7f4091800640 1 --2- 172.21.15.159:0/599024049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f408c0b3da0 0x7f408c0b4220 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f408c08c980 tx=0x7f408402bee0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.714+0000 7f4091800640 1 -- 172.21.15.159:0/599024049 shutdown_connections 2024-04-04T06:14:05.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.714+0000 7f4091800640 1 --2- 172.21.15.159:0/599024049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f408c0b3da0 0x7f408c0b4220 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.714+0000 7f4091800640 1 --2- 172.21.15.159:0/599024049 >> v2:172.21.15.178:3300/0 conn(0x7f408c0b3170 0x7f408c0d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.714+0000 7f4091800640 1 --2- 172.21.15.159:0/599024049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f408c161f60 0x7f408c0d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.714+0000 7f4091800640 1 -- 172.21.15.159:0/599024049 >> 172.21.15.159:0/599024049 conn(0x7f408c08bfb0 msgr2=0x7f408c0d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:05.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.714+0000 7f4091800640 1 -- 172.21.15.159:0/599024049 shutdown_connections 2024-04-04T06:14:05.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.714+0000 7f4091800640 1 -- 172.21.15.159:0/599024049 wait complete. 2024-04-04T06:14:05.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.716+0000 7f4091800640 1 Processor -- start 2024-04-04T06:14:05.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.716+0000 7f4091800640 1 -- start start 2024-04-04T06:14:05.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.717+0000 7f4091800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f408c0b3170 0x7f408c0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.717+0000 7f4091800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f408c0b3da0 0x7f408c1544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.717+0000 7f4091800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f408c161f60 0x7f408c0b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.717+0000 7f4091800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f408c0d3310 con 0x7f408c161f60 2024-04-04T06:14:05.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.717+0000 7f4091800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f408c0d3030 con 0x7f408c0b3da0 2024-04-04T06:14:05.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.717+0000 7f4091800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f408c0b4760 con 0x7f408c0b3170 2024-04-04T06:14:05.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.717+0000 7f408b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f408c161f60 0x7f408c0b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.717+0000 7f408b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f408c161f60 0x7f408c0b20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33704/0 (socket says 172.21.15.159:33704) 2024-04-04T06:14:05.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.717+0000 7f408b400640 1 -- 172.21.15.159:0/1143703544 learned_addr learned my addr 172.21.15.159:0/1143703544 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:05.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.717+0000 7f408a000640 1 --2- 172.21.15.159:0/1143703544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f408c0b3da0 0x7f408c1544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.717+0000 7f408b400640 1 -- 172.21.15.159:0/1143703544 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f408c0b3170 msgr2=0x7f408c0cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.717+0000 7f408aa00640 1 --2- 172.21.15.159:0/1143703544 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f408c0b3170 0x7f408c0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.718+0000 7f408b400640 1 --2- 172.21.15.159:0/1143703544 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f408c0b3170 0x7f408c0cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.718+0000 7f408b400640 1 -- 172.21.15.159:0/1143703544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f408c0b3da0 msgr2=0x7f408c1544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:05.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.718+0000 7f408b400640 1 --2- 172.21.15.159:0/1143703544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f408c0b3da0 0x7f408c1544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:05.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.718+0000 7f408b400640 1 -- 172.21.15.159:0/1143703544 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408c0cdc60 con 0x7f408c161f60 2024-04-04T06:14:05.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.718+0000 7f408a000640 1 --2- 172.21.15.159:0/1143703544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f408c0b3da0 0x7f408c1544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:05.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.718+0000 7f408b400640 1 --2- 172.21.15.159:0/1143703544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f408c161f60 0x7f408c0b20a0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f4084009f40 tx=0x7f4084004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.718+0000 7f4082a00640 1 -- 172.21.15.159:0/1143703544 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084043020 con 0x7f408c161f60 2024-04-04T06:14:05.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.718+0000 7f4082a00640 1 -- 172.21.15.159:0/1143703544 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f408403e450 con 0x7f408c161f60 2024-04-04T06:14:05.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.718+0000 7f4082a00640 1 -- 172.21.15.159:0/1143703544 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084009510 con 0x7f408c161f60 2024-04-04T06:14:05.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.718+0000 7f4091800640 1 -- 172.21.15.159:0/1143703544 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408c0cdf00 con 0x7f408c161f60 2024-04-04T06:14:05.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.718+0000 7f4091800640 1 -- 172.21.15.159:0/1143703544 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f408c0cadc0 con 0x7f408c161f60 2024-04-04T06:14:05.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.720+0000 7f4091800640 1 -- 172.21.15.159:0/1143703544 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f408c0685c0 con 0x7f408c161f60 2024-04-04T06:14:05.715 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.720+0000 7f4082a00640 1 -- 172.21.15.159:0/1143703544 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4084046020 con 0x7f408c161f60 2024-04-04T06:14:05.715 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.721+0000 7f4082a00640 1 --2- 172.21.15.159:0/1143703544 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f406806c8f0 0x7f406806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:05.715 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.721+0000 7f4082a00640 1 -- 172.21.15.159:0/1143703544 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f40840ae950 con 0x7f408c161f60 2024-04-04T06:14:05.715 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.722+0000 7f408aa00640 1 --2- 172.21.15.159:0/1143703544 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f406806c8f0 0x7f406806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:05.717 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.723+0000 7f408aa00640 1 --2- 172.21.15.159:0/1143703544 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f406806c8f0 0x7f406806edb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f407c00a770 tx=0x7f407c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:05.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.726+0000 7f4082a00640 1 -- 172.21.15.159:0/1143703544 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f408404d020 con 0x7f408c161f60 2024-04-04T06:14:05.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:05.972+0000 7f4091800640 1 -- 172.21.15.159:0/1143703544 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f408c1616e0 con 0x7f408c161f60 2024-04-04T06:14:06.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:06.780+0000 7f4082a00640 1 -- 172.21.15.159:0/1143703544 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v72) ==== 162+0+0 (secure 0 0 0) 0x7f408407cb60 con 0x7f408c161f60 2024-04-04T06:14:06.775 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-04T06:14:06.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:06.784+0000 7f4091800640 1 -- 172.21.15.159:0/1143703544 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f406806c8f0 msgr2=0x7f406806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:06.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:06.784+0000 7f4091800640 1 --2- 172.21.15.159:0/1143703544 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f406806c8f0 0x7f406806edb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f407c00a770 tx=0x7f407c008040 comp rx=0 tx=0).stop 2024-04-04T06:14:06.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:06.784+0000 7f4091800640 1 -- 172.21.15.159:0/1143703544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f408c161f60 msgr2=0x7f408c0b20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:06.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:06.784+0000 7f4091800640 1 --2- 172.21.15.159:0/1143703544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f408c161f60 0x7f408c0b20a0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f4084009f40 tx=0x7f4084004a40 comp rx=0 tx=0).stop 2024-04-04T06:14:06.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:06.784+0000 7f4091800640 1 -- 172.21.15.159:0/1143703544 shutdown_connections 2024-04-04T06:14:06.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:06.784+0000 7f4091800640 1 --2- 172.21.15.159:0/1143703544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f408c161f60 0x7f408c0b20a0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:06.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:06.784+0000 7f4091800640 1 --2- 172.21.15.159:0/1143703544 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f406806c8f0 0x7f406806edb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:06.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:06.784+0000 7f4091800640 1 --2- 172.21.15.159:0/1143703544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f408c0b3da0 0x7f408c1544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:06.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:06.784+0000 7f4091800640 1 --2- 172.21.15.159:0/1143703544 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f408c0b3170 0x7f408c0cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:06.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:06.784+0000 7f4091800640 1 -- 172.21.15.159:0/1143703544 >> 172.21.15.159:0/1143703544 conn(0x7f408c08bfb0 msgr2=0x7f408c1607a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:06.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:06.785+0000 7f4091800640 1 -- 172.21.15.159:0/1143703544 shutdown_connections 2024-04-04T06:14:06.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:06.785+0000 7f4091800640 1 -- 172.21.15.159:0/1143703544 wait complete. 2024-04-04T06:14:06.801 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:14:06.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.003+0000 7f9832c00640 1 Processor -- start 2024-04-04T06:14:06.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.003+0000 7f9832c00640 1 -- start start 2024-04-04T06:14:06.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.004+0000 7f9832c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f982c15c390 0x7f982c073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:06.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.004+0000 7f9832c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f982c15fe80 con 0x7f982c0d0650 2024-04-04T06:14:06.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.004+0000 7f9832c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f982c0ad120 con 0x7f982c0af830 2024-04-04T06:14:06.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.004+0000 7f9832c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f982c15c770 con 0x7f982c15c390 2024-04-04T06:14:06.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.005+0000 7f982be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f982c15c390 0x7f982c073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:06.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.005+0000 7f982be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f982c15c390 0x7f982c073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33716/0 (socket says 172.21.15.159:33716) 2024-04-04T06:14:06.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.005+0000 7f982be00640 1 -- 172.21.15.159:0/470693508 learned_addr learned my addr 172.21.15.159:0/470693508 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:06.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.005+0000 7f982be00640 1 -- 172.21.15.159:0/470693508 >> v1:172.21.15.178:6789/0 conn(0x7f982c0af830 legacy=0x7f982c160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:06.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.005+0000 7f982be00640 1 -- 172.21.15.159:0/470693508 >> v1:172.21.15.159:6789/0 conn(0x7f982c0d0650 legacy=0x7f982c071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:06.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.005+0000 7f982be00640 1 -- 172.21.15.159:0/470693508 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c0ad400 con 0x7f982c15c390 2024-04-04T06:14:06.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.005+0000 7f982aa00640 1 -- 172.21.15.159:0/470693508 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3853649772 0 0) 0x7f982c15fe80 con 0x7f982c0d0650 2024-04-04T06:14:07.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.006+0000 7f982be00640 1 --2- 172.21.15.159:0/470693508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f982c15c390 0x7f982c073790 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f981c009870 tx=0x7f981c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30759cb8315dc27 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:07.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.007+0000 7f982aa00640 1 -- 172.21.15.159:0/470693508 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c034070 con 0x7f982c15c390 2024-04-04T06:14:07.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.007+0000 7f982aa00640 1 -- 172.21.15.159:0/470693508 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f981c030860 con 0x7f982c15c390 2024-04-04T06:14:07.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.007+0000 7f982aa00640 1 -- 172.21.15.159:0/470693508 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c030c30 con 0x7f982c15c390 2024-04-04T06:14:07.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.007+0000 7f9832c00640 1 -- 172.21.15.159:0/470693508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f982c15c390 msgr2=0x7f982c073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:07.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.007+0000 7f9832c00640 1 --2- 172.21.15.159:0/470693508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f982c15c390 0x7f982c073790 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f981c009870 tx=0x7f981c02b9a0 comp rx=0 tx=0).stop 2024-04-04T06:14:07.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.007+0000 7f9832c00640 1 -- 172.21.15.159:0/470693508 shutdown_connections 2024-04-04T06:14:07.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.007+0000 7f9832c00640 1 --2- 172.21.15.159:0/470693508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f982c15c390 0x7f982c073790 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:07.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.007+0000 7f9832c00640 1 -- 172.21.15.159:0/470693508 >> 172.21.15.159:0/470693508 conn(0x7f982c06b990 msgr2=0x7f982c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:07.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.007+0000 7f9832c00640 1 -- 172.21.15.159:0/470693508 shutdown_connections 2024-04-04T06:14:07.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.007+0000 7f9832c00640 1 -- 172.21.15.159:0/470693508 wait complete. 2024-04-04T06:14:07.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.009+0000 7f9832c00640 1 Processor -- start 2024-04-04T06:14:07.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.009+0000 7f9832c00640 1 -- start start 2024-04-04T06:14:07.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.010+0000 7f9832c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f982c0af830 0x7f982c0998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:07.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.010+0000 7f9832c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f982c0d0650 0x7f982c15de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:07.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.010+0000 7f9832c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f982c15c390 0x7f982c159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:07.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.010+0000 7f9832c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f982c160d50 con 0x7f982c0d0650 2024-04-04T06:14:07.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.010+0000 7f9832c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f982c160120 con 0x7f982c0af830 2024-04-04T06:14:07.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.010+0000 7f9832c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f982c0cf980 con 0x7f982c15c390 2024-04-04T06:14:07.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.010+0000 7f982b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f982c0d0650 0x7f982c15de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:07.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.010+0000 7f982b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f982c0d0650 0x7f982c15de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33730/0 (socket says 172.21.15.159:33730) 2024-04-04T06:14:07.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.010+0000 7f982b400640 1 -- 172.21.15.159:0/405822753 learned_addr learned my addr 172.21.15.159:0/405822753 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:07.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.011+0000 7f982be00640 1 --2- 172.21.15.159:0/405822753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f982c0af830 0x7f982c0998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:07.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.011+0000 7f9830c00640 1 --2- 172.21.15.159:0/405822753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f982c15c390 0x7f982c159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:07.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.011+0000 7f982b400640 1 -- 172.21.15.159:0/405822753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f982c15c390 msgr2=0x7f982c159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:07.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.011+0000 7f982b400640 1 --2- 172.21.15.159:0/405822753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f982c15c390 0x7f982c159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:07.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.011+0000 7f982b400640 1 -- 172.21.15.159:0/405822753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f982c0af830 msgr2=0x7f982c0998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:07.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.011+0000 7f982b400640 1 --2- 172.21.15.159:0/405822753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f982c0af830 0x7f982c0998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:07.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.011+0000 7f982b400640 1 -- 172.21.15.159:0/405822753 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c15e7e0 con 0x7f982c0d0650 2024-04-04T06:14:07.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.011+0000 7f982be00640 1 --2- 172.21.15.159:0/405822753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f982c0af830 0x7f982c0998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:07.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.011+0000 7f982b400640 1 --2- 172.21.15.159:0/405822753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f982c0d0650 0x7f982c15de90 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f981800c430 tx=0x7f981800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:07.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.011+0000 7f9823e00640 1 -- 172.21.15.159:0/405822753 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9818011790 con 0x7f982c0d0650 2024-04-04T06:14:07.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.012+0000 7f9823e00640 1 -- 172.21.15.159:0/405822753 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9818017070 con 0x7f982c0d0650 2024-04-04T06:14:07.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.012+0000 7f9832c00640 1 -- 172.21.15.159:0/405822753 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f982c15eac0 con 0x7f982c0d0650 2024-04-04T06:14:07.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.012+0000 7f9823e00640 1 -- 172.21.15.159:0/405822753 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9818011bb0 con 0x7f982c0d0650 2024-04-04T06:14:07.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.012+0000 7f9832c00640 1 -- 172.21.15.159:0/405822753 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f982c15f080 con 0x7f982c0d0650 2024-04-04T06:14:07.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.013+0000 7f9832c00640 1 -- 172.21.15.159:0/405822753 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f982c0685c0 con 0x7f982c0d0650 2024-04-04T06:14:07.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.014+0000 7f9823e00640 1 -- 172.21.15.159:0/405822753 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f981801c460 con 0x7f982c0d0650 2024-04-04T06:14:07.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.014+0000 7f9823e00640 1 --2- 172.21.15.159:0/405822753 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f980c06c940 0x7f980c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:07.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.015+0000 7f9823e00640 1 -- 172.21.15.159:0/405822753 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f981808bb10 con 0x7f982c0d0650 2024-04-04T06:14:07.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.015+0000 7f982be00640 1 --2- 172.21.15.159:0/405822753 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f980c06c940 0x7f980c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:07.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.016+0000 7f982be00640 1 --2- 172.21.15.159:0/405822753 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f980c06c940 0x7f980c06ee00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f981c036000 tx=0x7f981c03b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:07.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.019+0000 7f9823e00640 1 -- 172.21.15.159:0/405822753 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9818059e30 con 0x7f982c0d0650 2024-04-04T06:14:07.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.257+0000 7f9832c00640 1 -- 172.21.15.159:0/405822753 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f982c158010 con 0x7f982c0d0650 2024-04-04T06:14:07.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.258+0000 7f9823e00640 1 -- 172.21.15.159:0/405822753 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5189 (secure 0 0 0) 0x7f9818059730 con 0x7f982c0d0650 2024-04-04T06:14:07.253 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:14:07.253 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1825651864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1825651864},{"type":"v1","addr":"172.21.15.178:6836","nonce":1825651864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1824578712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1824578712},{"type":"v1","addr":"172.21.15.178:6837","nonce":1824578712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6834/3377498298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3377498298},{"type":"v1","addr":"172.21.15.159:6834","nonce":3377498298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:12:35.880454+0000","modified":"2024-04-04T06:14:06.785421+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7809},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7809":{"gid":7809,"name":"a","rank":0,"incarnation":73,"state":"up:replay","state_seq":1,"addr":"172.21.15.159:6835/3555825575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":3555825575},{"type":"v1","addr":"172.21.15.159:6835","nonce":3555825575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-04T06:14:07.253 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 73 2024-04-04T06:14:07.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.261+0000 7f9832c00640 1 -- 172.21.15.159:0/405822753 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f980c06c940 msgr2=0x7f980c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:07.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.261+0000 7f9832c00640 1 --2- 172.21.15.159:0/405822753 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f980c06c940 0x7f980c06ee00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f981c036000 tx=0x7f981c03b040 comp rx=0 tx=0).stop 2024-04-04T06:14:07.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.261+0000 7f9832c00640 1 -- 172.21.15.159:0/405822753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f982c0d0650 msgr2=0x7f982c15de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:07.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.261+0000 7f9832c00640 1 --2- 172.21.15.159:0/405822753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f982c0d0650 0x7f982c15de90 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f981800c430 tx=0x7f981800c8f0 comp rx=0 tx=0).stop 2024-04-04T06:14:07.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.261+0000 7f9832c00640 1 -- 172.21.15.159:0/405822753 shutdown_connections 2024-04-04T06:14:07.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.261+0000 7f9832c00640 1 --2- 172.21.15.159:0/405822753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f982c15c390 0x7f982c159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:07.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.261+0000 7f9832c00640 1 --2- 172.21.15.159:0/405822753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f982c0d0650 0x7f982c15de90 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:07.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.261+0000 7f9832c00640 1 --2- 172.21.15.159:0/405822753 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f980c06c940 0x7f980c06ee00 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:07.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.261+0000 7f9832c00640 1 --2- 172.21.15.159:0/405822753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f982c0af830 0x7f982c0998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:07.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.261+0000 7f9832c00640 1 -- 172.21.15.159:0/405822753 >> 172.21.15.159:0/405822753 conn(0x7f982c06b990 msgr2=0x7f982c0b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:07.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.261+0000 7f9832c00640 1 -- 172.21.15.159:0/405822753 shutdown_connections 2024-04-04T06:14:07.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:07.261+0000 7f9832c00640 1 -- 172.21.15.159:0/405822753 wait complete. 2024-04-04T06:14:07.675 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:12:35.880454+0000', 'modified': '2024-04-04T06:14:06.785421+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 89, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7809}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7809': {'gid': 7809, 'name': 'a', 'rank': 0, 'incarnation': 73, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.159:6835/3555825575', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6833', 'nonce': 3555825575}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 3555825575}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-04T06:14:07.676 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7809:up:replay 2024-04-04T06:14:08.677 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:14:08.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.889+0000 7f0d08a00640 1 Processor -- start 2024-04-04T06:14:08.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.889+0000 7f0d08a00640 1 -- start start 2024-04-04T06:14:08.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.891+0000 7f0d08a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0d041442c0 0x7f0d040ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:08.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.891+0000 7f0d08a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0d040d02a0 0x7f0d040b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:08.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.891+0000 7f0d08a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d04071460 con 0x7f0d040cefe0 2024-04-04T06:14:08.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.891+0000 7f0d08a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f0d04071d20 con 0x7f0d040d02a0 2024-04-04T06:14:08.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.891+0000 7f0d08a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0d040cf420 con 0x7f0d041442c0 2024-04-04T06:14:08.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.891+0000 7f0d03e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0d040d02a0 0x7f0d040b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:08.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.891+0000 7f0d03e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0d040d02a0 0x7f0d040b3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33738/0 (socket says 172.21.15.159:33738) 2024-04-04T06:14:08.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.891+0000 7f0d03e00640 1 -- 172.21.15.159:0/1835000107 learned_addr learned my addr 172.21.15.159:0/1835000107 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:08.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.891+0000 7f0d03400640 1 --2- 172.21.15.159:0/1835000107 >> v2:172.21.15.178:3300/0 conn(0x7f0d041442c0 0x7f0d040ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:08.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.891+0000 7f0d03e00640 1 -- 172.21.15.159:0/1835000107 >> v2:172.21.15.178:3300/0 conn(0x7f0d041442c0 msgr2=0x7f0d040ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:08.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.891+0000 7f0d03e00640 1 --2- 172.21.15.159:0/1835000107 >> v2:172.21.15.178:3300/0 conn(0x7f0d041442c0 0x7f0d040ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:08.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.892+0000 7f0d03e00640 1 -- 172.21.15.159:0/1835000107 >> v1:172.21.15.178:6789/0 conn(0x7f0d040cefe0 legacy=0x7f0d040cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:08.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.892+0000 7f0d03e00640 1 -- 172.21.15.159:0/1835000107 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d040721b0 con 0x7f0d040d02a0 2024-04-04T06:14:08.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.893+0000 7f0d03e00640 1 --2- 172.21.15.159:0/1835000107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d040d02a0 0x7f0d040b3b20 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f0d040d0ae0 tx=0x7f0cfc02e190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be23e7a155eb6969 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:08.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.893+0000 7f0d02000640 1 -- 172.21.15.159:0/1835000107 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc02ed30 con 0x7f0d040d02a0 2024-04-04T06:14:08.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.893+0000 7f0d02000640 1 -- 172.21.15.159:0/1835000107 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cfc0363f0 con 0x7f0d040d02a0 2024-04-04T06:14:08.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.893+0000 7f0d02000640 1 -- 172.21.15.159:0/1835000107 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0367c0 con 0x7f0d040d02a0 2024-04-04T06:14:08.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.894+0000 7f0d08a00640 1 -- 172.21.15.159:0/1835000107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d040d02a0 msgr2=0x7f0d040b3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:08.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.894+0000 7f0d08a00640 1 --2- 172.21.15.159:0/1835000107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d040d02a0 0x7f0d040b3b20 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f0d040d0ae0 tx=0x7f0cfc02e190 comp rx=0 tx=0).stop 2024-04-04T06:14:08.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.894+0000 7f0d08a00640 1 -- 172.21.15.159:0/1835000107 shutdown_connections 2024-04-04T06:14:08.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.894+0000 7f0d08a00640 1 --2- 172.21.15.159:0/1835000107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d040d02a0 0x7f0d040b3b20 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:08.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.894+0000 7f0d08a00640 1 --2- 172.21.15.159:0/1835000107 >> v2:172.21.15.178:3300/0 conn(0x7f0d041442c0 0x7f0d040ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:08.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.894+0000 7f0d08a00640 1 -- 172.21.15.159:0/1835000107 >> 172.21.15.159:0/1835000107 conn(0x7f0d040ccad0 msgr2=0x7f0d040cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:08.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.894+0000 7f0d08a00640 1 -- 172.21.15.159:0/1835000107 shutdown_connections 2024-04-04T06:14:08.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.894+0000 7f0d08a00640 1 -- 172.21.15.159:0/1835000107 wait complete. 2024-04-04T06:14:08.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.896+0000 7f0d08a00640 1 Processor -- start 2024-04-04T06:14:08.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.896+0000 7f0d08a00640 1 -- start start 2024-04-04T06:14:08.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.897+0000 7f0d08a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d040cefe0 0x7f0d0415a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:08.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.898+0000 7f0d08a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d040d02a0 0x7f0d0413d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:08.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.898+0000 7f0d08a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d041442c0 0x7f0d0413fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:08.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.898+0000 7f0d08a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0d040d0770 con 0x7f0d040cefe0 2024-04-04T06:14:08.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.898+0000 7f0d08a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0d040719b0 con 0x7f0d041442c0 2024-04-04T06:14:08.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.898+0000 7f0d08a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0d041446a0 con 0x7f0d040d02a0 2024-04-04T06:14:08.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.898+0000 7f0d03400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d040cefe0 0x7f0d0415a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:08.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.898+0000 7f0d03400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d040cefe0 0x7f0d0415a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33744/0 (socket says 172.21.15.159:33744) 2024-04-04T06:14:08.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.898+0000 7f0d03400640 1 -- 172.21.15.159:0/3792449484 learned_addr learned my addr 172.21.15.159:0/3792449484 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:08.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.898+0000 7f0d03e00640 1 --2- 172.21.15.159:0/3792449484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d041442c0 0x7f0d0413fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:08.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.898+0000 7f0d02a00640 1 --2- 172.21.15.159:0/3792449484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d040d02a0 0x7f0d0413d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:08.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.898+0000 7f0d03400640 1 -- 172.21.15.159:0/3792449484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d040d02a0 msgr2=0x7f0d0413d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:08.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.898+0000 7f0d03400640 1 --2- 172.21.15.159:0/3792449484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d040d02a0 0x7f0d0413d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:08.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.898+0000 7f0d03400640 1 -- 172.21.15.159:0/3792449484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d041442c0 msgr2=0x7f0d0413fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:08.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.898+0000 7f0d03400640 1 --2- 172.21.15.159:0/3792449484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d041442c0 0x7f0d0413fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:08.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.898+0000 7f0d03400640 1 -- 172.21.15.159:0/3792449484 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0413d8c0 con 0x7f0d040cefe0 2024-04-04T06:14:08.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.899+0000 7f0d03400640 1 --2- 172.21.15.159:0/3792449484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d040cefe0 0x7f0d0415a8b0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400d830 tx=0x7f0cf400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:08.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.899+0000 7f0cfb400640 1 -- 172.21.15.159:0/3792449484 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf400b6c0 con 0x7f0d040cefe0 2024-04-04T06:14:08.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.899+0000 7f0cfb400640 1 -- 172.21.15.159:0/3792449484 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf4019070 con 0x7f0d040cefe0 2024-04-04T06:14:08.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.899+0000 7f0cfb400640 1 -- 172.21.15.159:0/3792449484 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf400ba80 con 0x7f0d040cefe0 2024-04-04T06:14:08.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.899+0000 7f0d08a00640 1 -- 172.21.15.159:0/3792449484 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0413dba0 con 0x7f0d040cefe0 2024-04-04T06:14:08.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.899+0000 7f0d08a00640 1 -- 172.21.15.159:0/3792449484 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d0415c900 con 0x7f0d040cefe0 2024-04-04T06:14:08.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.900+0000 7f0d08a00640 1 -- 172.21.15.159:0/3792449484 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d04075940 con 0x7f0d040cefe0 2024-04-04T06:14:08.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.901+0000 7f0cfb400640 1 -- 172.21.15.159:0/3792449484 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0cf401e460 con 0x7f0d040cefe0 2024-04-04T06:14:08.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.902+0000 7f0cfb400640 1 --2- 172.21.15.159:0/3792449484 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0ce406c8f0 0x7f0ce406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:08.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.902+0000 7f0cfb400640 1 -- 172.21.15.159:0/3792449484 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f0cf408c450 con 0x7f0d040cefe0 2024-04-04T06:14:08.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.903+0000 7f0d02a00640 1 --2- 172.21.15.159:0/3792449484 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0ce406c8f0 0x7f0ce406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:08.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.904+0000 7f0d02a00640 1 --2- 172.21.15.159:0/3792449484 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0ce406c8f0 0x7f0ce406edb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0006fd0 tx=0x7f0cf0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:08.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:08.907+0000 7f0cfb400640 1 -- 172.21.15.159:0/3792449484 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0cf405a660 con 0x7f0d040cefe0 2024-04-04T06:14:09.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:09.150+0000 7f0d08a00640 1 -- 172.21.15.159:0/3792449484 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d040add10 con 0x7f0d040cefe0 2024-04-04T06:14:09.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:09.152+0000 7f0cfb400640 1 -- 172.21.15.159:0/3792449484 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5189 (secure 0 0 0) 0x7f0cf4059f60 con 0x7f0d040cefe0 2024-04-04T06:14:09.146 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:14:09.146 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":75,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1825651864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1825651864},{"type":"v1","addr":"172.21.15.178:6836","nonce":1825651864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1824578712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1824578712},{"type":"v1","addr":"172.21.15.178:6837","nonce":1824578712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6834/3377498298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3377498298},{"type":"v1","addr":"172.21.15.159:6834","nonce":3377498298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:12:35.880454+0000","modified":"2024-04-04T06:14:07.803085+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7809},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7809":{"gid":7809,"name":"a","rank":0,"incarnation":73,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.159:6835/3555825575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":3555825575},{"type":"v1","addr":"172.21.15.159:6835","nonce":3555825575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-04T06:14:09.146 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 75 2024-04-04T06:14:09.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:09.155+0000 7f0d08a00640 1 -- 172.21.15.159:0/3792449484 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0ce406c8f0 msgr2=0x7f0ce406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:09.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:09.155+0000 7f0d08a00640 1 --2- 172.21.15.159:0/3792449484 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0ce406c8f0 0x7f0ce406edb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0006fd0 tx=0x7f0cf0008040 comp rx=0 tx=0).stop 2024-04-04T06:14:09.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:09.155+0000 7f0d08a00640 1 -- 172.21.15.159:0/3792449484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d040cefe0 msgr2=0x7f0d0415a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:09.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:09.155+0000 7f0d08a00640 1 --2- 172.21.15.159:0/3792449484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d040cefe0 0x7f0d0415a8b0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400d830 tx=0x7f0cf400dcf0 comp rx=0 tx=0).stop 2024-04-04T06:14:09.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:09.155+0000 7f0d08a00640 1 -- 172.21.15.159:0/3792449484 shutdown_connections 2024-04-04T06:14:09.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:09.155+0000 7f0d08a00640 1 --2- 172.21.15.159:0/3792449484 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0ce406c8f0 0x7f0ce406edb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:09.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:09.155+0000 7f0d08a00640 1 --2- 172.21.15.159:0/3792449484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d041442c0 0x7f0d0413fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:09.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:09.155+0000 7f0d08a00640 1 --2- 172.21.15.159:0/3792449484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0d040d02a0 0x7f0d0413d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:09.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:09.155+0000 7f0d08a00640 1 --2- 172.21.15.159:0/3792449484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d040cefe0 0x7f0d0415a8b0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:09.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:09.155+0000 7f0d08a00640 1 -- 172.21.15.159:0/3792449484 >> 172.21.15.159:0/3792449484 conn(0x7f0d040ccad0 msgr2=0x7f0d040b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:09.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:09.155+0000 7f0d08a00640 1 -- 172.21.15.159:0/3792449484 shutdown_connections 2024-04-04T06:14:09.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:09.155+0000 7f0d08a00640 1 -- 172.21.15.159:0/3792449484 wait complete. 2024-04-04T06:14:09.570 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:12:35.880454+0000', 'modified': '2024-04-04T06:14:07.803085+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 89, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7809}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7809': {'gid': 7809, 'name': 'a', 'rank': 0, 'incarnation': 73, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.159:6835/3555825575', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6833', 'nonce': 3555825575}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 3555825575}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-04T06:14:09.570 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7809:up:rejoin 2024-04-04T06:14:10.571 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:14:10.773 INFO:tasks.ceph.mds.a.smithi159.stderr:2024-04-04T06:14:10.779+0000 7fce9e800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:14:10.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.781+0000 7f1377600640 1 Processor -- start 2024-04-04T06:14:10.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.781+0000 7f1377600640 1 -- start start 2024-04-04T06:14:10.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.783+0000 7f1377600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f137015c390 0x7f1370073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:10.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.783+0000 7f1377600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13700af830 0x7f1370160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:10.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.783+0000 7f1377600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1370160b80 0x7f13700d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:10.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.783+0000 7f1377600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f137015c770 con 0x7f137015c390 2024-04-04T06:14:10.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.783+0000 7f1377600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f137015fe80 con 0x7f1370160b80 2024-04-04T06:14:10.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.783+0000 7f1377600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f13700ad120 con 0x7f13700af830 2024-04-04T06:14:10.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.783+0000 7f1374c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f137015c390 0x7f1370073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:10.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.783+0000 7f1374c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f137015c390 0x7f1370073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33754/0 (socket says 172.21.15.159:33754) 2024-04-04T06:14:10.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.783+0000 7f1374c00640 1 -- 172.21.15.159:0/619133786 learned_addr learned my addr 172.21.15.159:0/619133786 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:10.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.783+0000 7f136fe00640 1 --2- 172.21.15.159:0/619133786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13700af830 0x7f1370160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:10.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.783+0000 7f1375600640 1 --2- 172.21.15.159:0/619133786 >> v2:172.21.15.178:3300/0 conn(0x7f1370160b80 0x7f13700d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:10.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.784+0000 7f1374c00640 1 -- 172.21.15.159:0/619133786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13700af830 msgr2=0x7f1370160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:10.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.784+0000 7f1374c00640 1 --2- 172.21.15.159:0/619133786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13700af830 0x7f1370160640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:10.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.784+0000 7f1374c00640 1 -- 172.21.15.159:0/619133786 >> v2:172.21.15.178:3300/0 conn(0x7f1370160b80 msgr2=0x7f13700d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:10.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.784+0000 7f1374c00640 1 --2- 172.21.15.159:0/619133786 >> v2:172.21.15.178:3300/0 conn(0x7f1370160b80 0x7f13700d0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:10.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.784+0000 7f1374c00640 1 -- 172.21.15.159:0/619133786 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13700ad400 con 0x7f137015c390 2024-04-04T06:14:10.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.784+0000 7f1375600640 1 --2- 172.21.15.159:0/619133786 >> v2:172.21.15.178:3300/0 conn(0x7f1370160b80 0x7f13700d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:10.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.785+0000 7f1374c00640 1 --2- 172.21.15.159:0/619133786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f137015c390 0x7f1370073790 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f137006eaa0 tx=0x7f136402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5664e1dc9ca346 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:10.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.785+0000 7f136f400640 1 -- 172.21.15.159:0/619133786 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364035020 con 0x7f137015c390 2024-04-04T06:14:10.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.785+0000 7f136f400640 1 -- 172.21.15.159:0/619133786 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1364031870 con 0x7f137015c390 2024-04-04T06:14:10.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.785+0000 7f136f400640 1 -- 172.21.15.159:0/619133786 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364031c40 con 0x7f137015c390 2024-04-04T06:14:10.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.785+0000 7f1377600640 1 -- 172.21.15.159:0/619133786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f137015c390 msgr2=0x7f1370073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:10.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.786+0000 7f1377600640 1 --2- 172.21.15.159:0/619133786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f137015c390 0x7f1370073790 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f137006eaa0 tx=0x7f136402b910 comp rx=0 tx=0).stop 2024-04-04T06:14:10.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.786+0000 7f1377600640 1 -- 172.21.15.159:0/619133786 shutdown_connections 2024-04-04T06:14:10.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.786+0000 7f1377600640 1 --2- 172.21.15.159:0/619133786 >> v2:172.21.15.178:3300/0 conn(0x7f1370160b80 0x7f13700d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:10.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.786+0000 7f1377600640 1 --2- 172.21.15.159:0/619133786 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13700af830 0x7f1370160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:10.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.786+0000 7f1377600640 1 --2- 172.21.15.159:0/619133786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f137015c390 0x7f1370073790 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:10.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.786+0000 7f1377600640 1 -- 172.21.15.159:0/619133786 >> 172.21.15.159:0/619133786 conn(0x7f137006b990 msgr2=0x7f137006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:10.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.786+0000 7f1377600640 1 -- 172.21.15.159:0/619133786 shutdown_connections 2024-04-04T06:14:10.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.786+0000 7f1377600640 1 -- 172.21.15.159:0/619133786 wait complete. 2024-04-04T06:14:10.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.787+0000 7f1377600640 1 Processor -- start 2024-04-04T06:14:10.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.788+0000 7f1377600640 1 -- start start 2024-04-04T06:14:10.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.789+0000 7f1377600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13700af830 0x7f13700b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:10.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.789+0000 7f1377600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f137015c390 0x7f137015e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:10.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.789+0000 7f1377600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1370160b80 0x7f13701e5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:10.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.789+0000 7f1377600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f13700cf980 con 0x7f13700af830 2024-04-04T06:14:10.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.789+0000 7f1377600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1370160120 con 0x7f137015c390 2024-04-04T06:14:10.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.789+0000 7f1377600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f13700d0b90 con 0x7f1370160b80 2024-04-04T06:14:10.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.789+0000 7f1374c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13700af830 0x7f13700b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:10.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.789+0000 7f1374c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13700af830 0x7f13700b4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33764/0 (socket says 172.21.15.159:33764) 2024-04-04T06:14:10.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.789+0000 7f1374c00640 1 -- 172.21.15.159:0/1112999802 learned_addr learned my addr 172.21.15.159:0/1112999802 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:10.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.789+0000 7f136fe00640 1 --2- 172.21.15.159:0/1112999802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f137015c390 0x7f137015e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:10.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.789+0000 7f1375600640 1 --2- 172.21.15.159:0/1112999802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1370160b80 0x7f13701e5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:10.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.789+0000 7f1374c00640 1 -- 172.21.15.159:0/1112999802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1370160b80 msgr2=0x7f13701e5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:10.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.789+0000 7f1374c00640 1 --2- 172.21.15.159:0/1112999802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1370160b80 0x7f13701e5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:10.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.789+0000 7f1374c00640 1 -- 172.21.15.159:0/1112999802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f137015c390 msgr2=0x7f137015e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:10.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.789+0000 7f1374c00640 1 --2- 172.21.15.159:0/1112999802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f137015c390 0x7f137015e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:10.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.789+0000 7f1374c00640 1 -- 172.21.15.159:0/1112999802 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13700b3420 con 0x7f13700af830 2024-04-04T06:14:10.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.789+0000 7f136fe00640 1 --2- 172.21.15.159:0/1112999802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f137015c390 0x7f137015e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:10.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.790+0000 7f1374c00640 1 --2- 172.21.15.159:0/1112999802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13700af830 0x7f13700b4b90 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f136402b950 tx=0x7f13640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:10.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.790+0000 7f136cc00640 1 -- 172.21.15.159:0/1112999802 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364040070 con 0x7f13700af830 2024-04-04T06:14:10.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.790+0000 7f136cc00640 1 -- 172.21.15.159:0/1112999802 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f136403a420 con 0x7f13700af830 2024-04-04T06:14:10.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.790+0000 7f136cc00640 1 -- 172.21.15.159:0/1112999802 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364035040 con 0x7f13700af830 2024-04-04T06:14:10.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.790+0000 7f1377600640 1 -- 172.21.15.159:0/1112999802 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13700b3700 con 0x7f13700af830 2024-04-04T06:14:10.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.790+0000 7f1377600640 1 -- 172.21.15.159:0/1112999802 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13700b4720 con 0x7f13700af830 2024-04-04T06:14:10.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.792+0000 7f1377600640 1 -- 172.21.15.159:0/1112999802 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13700759e0 con 0x7f13700af830 2024-04-04T06:14:10.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.793+0000 7f136cc00640 1 -- 172.21.15.159:0/1112999802 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f136404b060 con 0x7f13700af830 2024-04-04T06:14:10.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.795+0000 7f136cc00640 1 --2- 172.21.15.159:0/1112999802 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f135006c8f0 0x7f135006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:10.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.795+0000 7f136cc00640 1 -- 172.21.15.159:0/1112999802 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f13640adbe0 con 0x7f13700af830 2024-04-04T06:14:10.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.796+0000 7f136fe00640 1 --2- 172.21.15.159:0/1112999802 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f135006c8f0 0x7f135006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:10.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.797+0000 7f136fe00640 1 --2- 172.21.15.159:0/1112999802 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f135006c8f0 0x7f135006edb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f13700b0800 tx=0x7f135c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:10.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:10.799+0000 7f136cc00640 1 -- 172.21.15.159:0/1112999802 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f13700759e0 con 0x7f13700af830 2024-04-04T06:14:11.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.038+0000 7f1377600640 1 -- 172.21.15.159:0/1112999802 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13701593d0 con 0x7f13700af830 2024-04-04T06:14:11.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.039+0000 7f136cc00640 1 -- 172.21.15.159:0/1112999802 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5196 (secure 0 0 0) 0x7f13701593d0 con 0x7f13700af830 2024-04-04T06:14:11.033 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:14:11.033 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":76,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1825651864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1825651864},{"type":"v1","addr":"172.21.15.178:6836","nonce":1825651864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1824578712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1824578712},{"type":"v1","addr":"172.21.15.178:6837","nonce":1824578712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6834/3377498298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3377498298},{"type":"v1","addr":"172.21.15.159:6834","nonce":3377498298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:12:35.880454+0000","modified":"2024-04-04T06:14:09.810244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7809},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7809":{"gid":7809,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6835/3555825575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":3555825575},{"type":"v1","addr":"172.21.15.159:6835","nonce":3555825575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7809,"qdb_cluster":[7809]},"id":6}]} 2024-04-04T06:14:11.034 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 76 2024-04-04T06:14:11.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.042+0000 7f1377600640 1 -- 172.21.15.159:0/1112999802 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f135006c8f0 msgr2=0x7f135006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:11.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.042+0000 7f1377600640 1 --2- 172.21.15.159:0/1112999802 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f135006c8f0 0x7f135006edb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f13700b0800 tx=0x7f135c008040 comp rx=0 tx=0).stop 2024-04-04T06:14:11.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.042+0000 7f1377600640 1 -- 172.21.15.159:0/1112999802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13700af830 msgr2=0x7f13700b4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:11.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.042+0000 7f1377600640 1 --2- 172.21.15.159:0/1112999802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13700af830 0x7f13700b4b90 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f136402b950 tx=0x7f13640046c0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.042+0000 7f1377600640 1 -- 172.21.15.159:0/1112999802 shutdown_connections 2024-04-04T06:14:11.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.042+0000 7f1377600640 1 --2- 172.21.15.159:0/1112999802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1370160b80 0x7f13701e5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.042+0000 7f1377600640 1 --2- 172.21.15.159:0/1112999802 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f135006c8f0 0x7f135006edb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.042+0000 7f1377600640 1 --2- 172.21.15.159:0/1112999802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f137015c390 0x7f137015e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.042+0000 7f1377600640 1 --2- 172.21.15.159:0/1112999802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13700af830 0x7f13700b4b90 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.042+0000 7f1377600640 1 -- 172.21.15.159:0/1112999802 >> 172.21.15.159:0/1112999802 conn(0x7f137006b990 msgr2=0x7f13700afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:11.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.042+0000 7f1377600640 1 -- 172.21.15.159:0/1112999802 shutdown_connections 2024-04-04T06:14:11.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.042+0000 7f1377600640 1 -- 172.21.15.159:0/1112999802 wait complete. 2024-04-04T06:14:11.458 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:12:35.880454+0000', 'modified': '2024-04-04T06:14:09.810244+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 89, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7809}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7809': {'gid': 7809, 'name': 'a', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.159:6835/3555825575', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6833', 'nonce': 3555825575}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 3555825575}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7809, 'qdb_cluster': [7809]} 2024-04-04T06:14:11.458 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-04T06:14:11.458 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-04T06:14:11.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.678+0000 7fc092a00640 1 Processor -- start 2024-04-04T06:14:11.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.678+0000 7fc092a00640 1 -- start start 2024-04-04T06:14:11.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.680+0000 7fc092a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc08c153580 0x7fc08c0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.680+0000 7fc092a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc08c0d07c0 0x7fc08c0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.680+0000 7fc092a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc08c0afab0 con 0x7fc08c0af2e0 2024-04-04T06:14:11.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.680+0000 7fc092a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc08c0d0c00 con 0x7fc08c0d07c0 2024-04-04T06:14:11.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.680+0000 7fc092a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc08c15cc80 con 0x7fc08c153580 2024-04-04T06:14:11.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.681+0000 7fc083e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc08c0d07c0 0x7fc08c0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:11.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.681+0000 7fc08be00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc08c153580 0x7fc08c0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:11.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.681+0000 7fc083e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc08c0d07c0 0x7fc08c0aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:46602/0 (socket says 172.21.15.159:46602) 2024-04-04T06:14:11.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.681+0000 7fc083e00640 1 -- 172.21.15.159:0/754573461 learned_addr learned my addr 172.21.15.159:0/754573461 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:11.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.681+0000 7fc08be00640 1 -- 172.21.15.159:0/754573461 >> v2:172.21.15.178:3300/0 conn(0x7fc08c0d07c0 msgr2=0x7fc08c0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:11.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.681+0000 7fc08be00640 1 --2- 172.21.15.159:0/754573461 >> v2:172.21.15.178:3300/0 conn(0x7fc08c0d07c0 0x7fc08c0aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.681+0000 7fc08be00640 1 -- 172.21.15.159:0/754573461 >> v1:172.21.15.178:6789/0 conn(0x7fc08c0af2e0 legacy=0x7fc08c06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:11.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.681+0000 7fc08be00640 1 -- 172.21.15.159:0/754573461 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08c15da00 con 0x7fc08c153580 2024-04-04T06:14:11.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.683+0000 7fc08be00640 1 --2- 172.21.15.159:0/754573461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc08c153580 0x7fc08c0d0280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0097a0 tx=0x7fc07c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13ad8c7a7f0da0bf server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:11.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.683+0000 7fc08b400640 1 -- 172.21.15.159:0/754573461 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c0315a0 con 0x7fc08c153580 2024-04-04T06:14:11.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.683+0000 7fc08b400640 1 -- 172.21.15.159:0/754573461 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc07c037070 con 0x7fc08c153580 2024-04-04T06:14:11.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.684+0000 7fc08b400640 1 -- 172.21.15.159:0/754573461 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c031960 con 0x7fc08c153580 2024-04-04T06:14:11.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.684+0000 7fc092a00640 1 -- 172.21.15.159:0/754573461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc08c153580 msgr2=0x7fc08c0d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:11.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.684+0000 7fc092a00640 1 --2- 172.21.15.159:0/754573461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc08c153580 0x7fc08c0d0280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0097a0 tx=0x7fc07c02b840 comp rx=0 tx=0).stop 2024-04-04T06:14:11.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.684+0000 7fc092a00640 1 -- 172.21.15.159:0/754573461 shutdown_connections 2024-04-04T06:14:11.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.684+0000 7fc092a00640 1 --2- 172.21.15.159:0/754573461 >> v2:172.21.15.178:3300/0 conn(0x7fc08c0d07c0 0x7fc08c0aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.684+0000 7fc092a00640 1 --2- 172.21.15.159:0/754573461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc08c153580 0x7fc08c0d0280 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.684+0000 7fc092a00640 1 -- 172.21.15.159:0/754573461 >> 172.21.15.159:0/754573461 conn(0x7fc08c06b750 msgr2=0x7fc08c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:11.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.684+0000 7fc092a00640 1 -- 172.21.15.159:0/754573461 shutdown_connections 2024-04-04T06:14:11.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.684+0000 7fc092a00640 1 -- 172.21.15.159:0/754573461 wait complete. 2024-04-04T06:14:11.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.686+0000 7fc092a00640 1 Processor -- start 2024-04-04T06:14:11.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.686+0000 7fc092a00640 1 -- start start 2024-04-04T06:14:11.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.687+0000 7fc092a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc08c0af2e0 0x7fc08c152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.687+0000 7fc092a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc08c0d07c0 0x7fc08c0cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.687+0000 7fc092a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc08c153580 0x7fc08c1f54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.687+0000 7fc092a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c15d4e0 con 0x7fc08c153580 2024-04-04T06:14:11.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.687+0000 7fc092a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c15d200 con 0x7fc08c0af2e0 2024-04-04T06:14:11.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.687+0000 7fc092a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc08c153960 con 0x7fc08c0d07c0 2024-04-04T06:14:11.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.687+0000 7fc090a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc08c153580 0x7fc08c1f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:11.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.687+0000 7fc090a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc08c153580 0x7fc08c1f54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33778/0 (socket says 172.21.15.159:33778) 2024-04-04T06:14:11.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.687+0000 7fc090a00640 1 -- 172.21.15.159:0/2879130748 learned_addr learned my addr 172.21.15.159:0/2879130748 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:11.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.687+0000 7fc083e00640 1 --2- 172.21.15.159:0/2879130748 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc08c0d07c0 0x7fc08c0cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:11.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.687+0000 7fc08be00640 1 --2- 172.21.15.159:0/2879130748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc08c0af2e0 0x7fc08c152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:11.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.688+0000 7fc090a00640 1 -- 172.21.15.159:0/2879130748 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc08c0d07c0 msgr2=0x7fc08c0cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:11.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.688+0000 7fc090a00640 1 --2- 172.21.15.159:0/2879130748 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc08c0d07c0 0x7fc08c0cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.688+0000 7fc090a00640 1 -- 172.21.15.159:0/2879130748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc08c0af2e0 msgr2=0x7fc08c152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:11.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.688+0000 7fc090a00640 1 --2- 172.21.15.159:0/2879130748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc08c0af2e0 0x7fc08c152120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.688+0000 7fc090a00640 1 -- 172.21.15.159:0/2879130748 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08c0b0e20 con 0x7fc08c153580 2024-04-04T06:14:11.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.688+0000 7fc083e00640 1 --2- 172.21.15.159:0/2879130748 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc08c0d07c0 0x7fc08c0cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:11.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.688+0000 7fc090a00640 1 --2- 172.21.15.159:0/2879130748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc08c153580 0x7fc08c1f54f0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fc08400c6b0 tx=0x7fc08400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:11.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.688+0000 7fc088c00640 1 -- 172.21.15.159:0/2879130748 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc084003b70 con 0x7fc08c153580 2024-04-04T06:14:11.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.688+0000 7fc088c00640 1 -- 172.21.15.159:0/2879130748 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc084003d30 con 0x7fc08c153580 2024-04-04T06:14:11.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.688+0000 7fc088c00640 1 -- 172.21.15.159:0/2879130748 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc084005310 con 0x7fc08c153580 2024-04-04T06:14:11.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.688+0000 7fc092a00640 1 -- 172.21.15.159:0/2879130748 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08c0b1100 con 0x7fc08c153580 2024-04-04T06:14:11.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.688+0000 7fc092a00640 1 -- 172.21.15.159:0/2879130748 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08c0b16c0 con 0x7fc08c153580 2024-04-04T06:14:11.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.692+0000 7fc088c00640 1 -- 172.21.15.159:0/2879130748 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0840054b0 con 0x7fc08c153580 2024-04-04T06:14:11.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.692+0000 7fc09564d740 1 Processor -- start 2024-04-04T06:14:11.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.692+0000 7fc09564d740 1 -- start start 2024-04-04T06:14:11.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.693+0000 7fc088c00640 1 --2- 172.21.15.159:0/2879130748 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc06806c8f0 0x7fc06806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.693+0000 7fc088c00640 1 -- 172.21.15.159:0/2879130748 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7fc08408ba70 con 0x7fc08c153580 2024-04-04T06:14:11.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.693+0000 7fc09564d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf8fcd4130 0x55cf8fca70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.693+0000 7fc09564d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcd4510 0x55cf8fcd6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.693+0000 7fc09564d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf8fca7610 0x55cf8fcd7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.693+0000 7fc09564d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cf8fc6eab0 con 0x55cf8fcd4510 2024-04-04T06:14:11.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.693+0000 7fc09564d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cf8fc10fa0 con 0x55cf8fca7610 2024-04-04T06:14:11.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.693+0000 7fc09564d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55cf8fbb9560 con 0x55cf8fcd4130 2024-04-04T06:14:11.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.693+0000 7fc083e00640 1 --2- 172.21.15.159:0/2879130748 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc06806c8f0 0x7fc06806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:11.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.693+0000 7fc083e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcd4510 0x55cf8fcd6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:11.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.693+0000 7fc083e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcd4510 0x55cf8fcd6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33784/0 (socket says 172.21.15.159:33784) 2024-04-04T06:14:11.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.693+0000 7fc083e00640 1 -- 172.21.15.159:0/2008674358 learned_addr learned my addr 172.21.15.159:0/2008674358 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:11.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.694+0000 7fc08be00640 1 --2- 172.21.15.159:0/2008674358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf8fcd4130 0x55cf8fca70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:11.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.694+0000 7fc090a00640 1 --2- 172.21.15.159:0/2008674358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf8fca7610 0x55cf8fcd7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:11.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.694+0000 7fc083e00640 1 -- 172.21.15.159:0/2008674358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf8fcd4130 msgr2=0x55cf8fca70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:11.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.694+0000 7fc083e00640 1 --2- 172.21.15.159:0/2008674358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf8fcd4130 0x55cf8fca70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.694+0000 7fc083e00640 1 -- 172.21.15.159:0/2008674358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf8fca7610 msgr2=0x55cf8fcd7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:11.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.694+0000 7fc083e00640 1 --2- 172.21.15.159:0/2008674358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf8fca7610 0x55cf8fcd7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.694+0000 7fc083e00640 1 -- 172.21.15.159:0/2008674358 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf8f9252d0 con 0x55cf8fcd4510 2024-04-04T06:14:11.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.694+0000 7fc090a00640 1 --2- 172.21.15.159:0/2008674358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf8fca7610 0x55cf8fcd7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:11.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.694+0000 7fc08be00640 1 --2- 172.21.15.159:0/2008674358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf8fcd4130 0x55cf8fca70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:11.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.694+0000 7fc083e00640 1 --2- 172.21.15.159:0/2008674358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcd4510 0x55cf8fcd6ec0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fc0740165e0 tx=0x7fc074009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5932377d7aeaa4de server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:11.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.694+0000 7fc07be00640 1 -- 172.21.15.159:0/2008674358 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc074018450 con 0x55cf8fcd4510 2024-04-04T06:14:11.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.694+0000 7fc07be00640 1 -- 172.21.15.159:0/2008674358 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0740185f0 con 0x55cf8fcd4510 2024-04-04T06:14:11.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.694+0000 7fc07be00640 1 -- 172.21.15.159:0/2008674358 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0740189a0 con 0x55cf8fcd4510 2024-04-04T06:14:11.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.695+0000 7fc09564d740 1 -- 172.21.15.159:0/2008674358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcd4510 msgr2=0x55cf8fcd6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:11.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.695+0000 7fc083e00640 1 --2- 172.21.15.159:0/2879130748 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc06806c8f0 0x7fc06806edb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc074020000 tx=0x7fc07401e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:11.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.695+0000 7fc09564d740 1 --2- 172.21.15.159:0/2008674358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcd4510 0x55cf8fcd6ec0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fc0740165e0 tx=0x7fc074009f90 comp rx=0 tx=0).stop 2024-04-04T06:14:11.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.695+0000 7fc09564d740 1 -- 172.21.15.159:0/2008674358 shutdown_connections 2024-04-04T06:14:11.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.695+0000 7fc09564d740 1 --2- 172.21.15.159:0/2008674358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf8fca7610 0x55cf8fcd7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.695+0000 7fc09564d740 1 --2- 172.21.15.159:0/2008674358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcd4510 0x55cf8fcd6ec0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.695+0000 7fc09564d740 1 --2- 172.21.15.159:0/2008674358 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf8fcd4130 0x55cf8fca70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.695+0000 7fc09564d740 1 -- 172.21.15.159:0/2008674358 >> 172.21.15.159:0/2008674358 conn(0x55cf8fca9340 msgr2=0x55cf8fcf1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:11.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.695+0000 7fc09564d740 1 -- 172.21.15.159:0/2008674358 shutdown_connections 2024-04-04T06:14:11.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.695+0000 7fc09564d740 1 -- 172.21.15.159:0/2008674358 wait complete. 2024-04-04T06:14:11.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.696+0000 7fc09564d740 1 -- start start 2024-04-04T06:14:11.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.696+0000 7fc09564d740 1 Processor -- start 2024-04-04T06:14:11.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.697+0000 7fc09564d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf8fcd4130 0x55cf8fce8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.697+0000 7fc09564d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcd4510 0x55cf8fcd6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.697+0000 7fc09564d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf8fcc1520 0x55cf8fd02e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.697+0000 7fc09564d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cf8fbcc920 con 0x55cf8fcd4510 2024-04-04T06:14:11.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.697+0000 7fc09564d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cf8fc0cf50 con 0x55cf8fcd4130 2024-04-04T06:14:11.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.697+0000 7fc09564d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55cf8fcab8e0 con 0x55cf8fcc1520 2024-04-04T06:14:11.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.697+0000 7fc09564d740 10 client.0 _ll_register_callbacks cb 0x55cf8fbcc310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:14:11.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.697+0000 7fc083e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcd4510 0x55cf8fcd6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:11.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.697+0000 7fc083e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcd4510 0x55cf8fcd6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33792/0 (socket says 172.21.15.159:33792) 2024-04-04T06:14:11.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.697+0000 7fc083e00640 1 -- 172.21.15.159:0/321763872 learned_addr learned my addr 172.21.15.159:0/321763872 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:11.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.698+0000 7fc083e00640 1 -- 172.21.15.159:0/321763872 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf8fcc1520 msgr2=0x55cf8fd02e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:11.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.698+0000 7fc090a00640 1 --2- 172.21.15.159:0/321763872 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf8fcc1520 0x55cf8fd02e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:11.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.698+0000 7fc08be00640 1 --2- 172.21.15.159:0/321763872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf8fcd4130 0x55cf8fce8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:11.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.698+0000 7fc083e00640 1 --2- 172.21.15.159:0/321763872 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf8fcc1520 0x55cf8fd02e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.698+0000 7fc083e00640 1 -- 172.21.15.159:0/321763872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf8fcd4130 msgr2=0x55cf8fce8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:11.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.698+0000 7fc083e00640 1 --2- 172.21.15.159:0/321763872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf8fcd4130 0x55cf8fce8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.698+0000 7fc083e00640 1 -- 172.21.15.159:0/321763872 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf8f9ef6b0 con 0x55cf8fcd4510 2024-04-04T06:14:11.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.698+0000 7fc083e00640 1 --2- 172.21.15.159:0/321763872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcd4510 0x55cf8fcd6390 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fc074016580 tx=0x7fc074006eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8502b18b53a3c728 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:11.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.698+0000 7fc078c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:14:11.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.699+0000 7fc078c00640 1 -- 172.21.15.159:0/321763872 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc074018450 con 0x55cf8fcd4510 2024-04-04T06:14:11.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.699+0000 7fc078c00640 1 -- 172.21.15.159:0/321763872 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7fc074033be0 con 0x55cf8fcd4510 2024-04-04T06:14:11.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.699+0000 7fc09564d740 1 -- 172.21.15.159:0/321763872 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0740185f0 con 0x55cf8fcd4510 2024-04-04T06:14:11.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.699+0000 7fc078c00640 1 -- 172.21.15.159:0/321763872 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fc07403a050 con 0x55cf8fcd4510 2024-04-04T06:14:11.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.699+0000 7fc09564d740 10 client.7881 fetch_fsmap learned FSMap version 76 2024-04-04T06:14:11.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.699+0000 7fc09564d740 1 -- 172.21.15.159:0/321763872 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55cf8fcd5860 con 0x55cf8fcd4510 2024-04-04T06:14:11.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.699+0000 7fc078c00640 1 -- 172.21.15.159:0/321763872 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0740185f0 con 0x55cf8fcd4510 2024-04-04T06:14:11.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.699+0000 7fc078c00640 1 -- 172.21.15.159:0/321763872 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 76) ==== 3494+0+0 (secure 0 0 0) 0x7fc07403d020 con 0x55cf8fcd4510 2024-04-04T06:14:11.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.699+0000 7fc09564d740 10 client.7881 fetch_fsmap finished waiting for FSMap version 76 2024-04-04T06:14:11.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.699+0000 7fc09564d740 10 client.7881 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:14:11.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.699+0000 7fc09564d740 20 client.7881 populate_metadata read hostname 'smithi159' 2024-04-04T06:14:11.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.699+0000 7fc09564d740 1 --2- 172.21.15.159:0/321763872 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55cf8fd05420 0x55cf8fd01fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.699+0000 7fc09564d740 4 client.7881 mds_command: new command op to 7809 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:14:11.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.699+0000 7fc09564d740 1 -- 172.21.15.159:0/321763872 --> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf8fc1e4e0 con 0x55cf8fd05420 2024-04-04T06:14:11.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.700+0000 7fc08be00640 1 --2- 172.21.15.159:0/321763872 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55cf8fd05420 0x55cf8fd01fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:11.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.700+0000 7fc08be00640 1 --2- 172.21.15.159:0/321763872 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55cf8fd05420 0x55cf8fd01fe0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67941aba3a7e9a41 server_cookie=5c217baec3f9b2b0 in_seq=0 out_seq=0 2024-04-04T06:14:11.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.700+0000 7fc078c00640 10 client.7881 ms_handle_connect on v2:172.21.15.159:6833/3555825575 2024-04-04T06:14:11.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.701+0000 7fc078c00640 1 -- 172.21.15.159:0/321763872 <== mds.0 v2:172.21.15.159:6833/3555825575 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf8fc1e4e0 con 0x55cf8fd05420 2024-04-04T06:14:11.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.701+0000 7fc078c00640 10 client.7881 handle_command_reply: tid=0 2024-04-04T06:14:11.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.701+0000 7fc09564d740 1 client.7881 shutdown 2024-04-04T06:14:11.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.701+0000 7fc09564d740 1 -- 172.21.15.159:0/321763872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcd4510 msgr2=0x55cf8fcd6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:11.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.701+0000 7fc09564d740 1 --2- 172.21.15.159:0/321763872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcd4510 0x55cf8fcd6390 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fc074016580 tx=0x7fc074006eb0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.702+0000 7fc09564d740 1 -- 172.21.15.159:0/321763872 shutdown_connections 2024-04-04T06:14:11.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.702+0000 7fc09564d740 1 --2- 172.21.15.159:0/321763872 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55cf8fd05420 0x55cf8fd01fe0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.702+0000 7fc09564d740 1 --2- 172.21.15.159:0/321763872 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf8fcc1520 0x55cf8fd02e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.702+0000 7fc09564d740 1 --2- 172.21.15.159:0/321763872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcd4510 0x55cf8fcd6390 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.702+0000 7fc09564d740 1 --2- 172.21.15.159:0/321763872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf8fcd4130 0x55cf8fce8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.702+0000 7fc078c00640 0 client.7881 ms_handle_reset on v2:172.21.15.159:6833/3555825575 2024-04-04T06:14:11.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.702+0000 7fc09564d740 1 -- 172.21.15.159:0/321763872 >> 172.21.15.159:0/321763872 conn(0x55cf8fca7610 msgr2=0x55cf8fcf5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:11.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.702+0000 7fc09564d740 1 -- 172.21.15.159:0/321763872 shutdown_connections 2024-04-04T06:14:11.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.702+0000 7fc09564d740 1 -- 172.21.15.159:0/321763872 wait complete. 2024-04-04T06:14:11.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.702+0000 7fc09564d740 20 client.7881 trim_cache size 0 max 16384 2024-04-04T06:14:11.715 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.721+0000 7fc09564d740 1 Processor -- start 2024-04-04T06:14:11.715 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.722+0000 7fc09564d740 1 -- start start 2024-04-04T06:14:11.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.722+0000 7fc09564d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcc1520 0x55cf8fcc04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.722+0000 7fc09564d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf8fcd4130 0x55cf8fcceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.722+0000 7fc09564d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf8fcd4510 0x55cf8fca6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.722+0000 7fc09564d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1fffc0 con 0x55cf8fcc1520 2024-04-04T06:14:11.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.722+0000 7fc09564d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1ff470 con 0x55cf8fcd4510 2024-04-04T06:14:11.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.722+0000 7fc09564d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc08c1ff970 con 0x55cf8fcd4130 2024-04-04T06:14:11.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.723+0000 7fc08be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcc1520 0x55cf8fcc04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:11.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.723+0000 7fc08be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcc1520 0x55cf8fcc04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33794/0 (socket says 172.21.15.159:33794) 2024-04-04T06:14:11.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.723+0000 7fc08be00640 1 -- 172.21.15.159:0/3472971138 learned_addr learned my addr 172.21.15.159:0/3472971138 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:11.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.723+0000 7fc08be00640 1 -- 172.21.15.159:0/3472971138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf8fcd4130 msgr2=0x55cf8fcceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:11.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.723+0000 7fc08be00640 1 --2- 172.21.15.159:0/3472971138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf8fcd4130 0x55cf8fcceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.723+0000 7fc08be00640 1 -- 172.21.15.159:0/3472971138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf8fcd4510 msgr2=0x55cf8fca6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:11.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.723+0000 7fc08be00640 1 --2- 172.21.15.159:0/3472971138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf8fcd4510 0x55cf8fca6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.723+0000 7fc08be00640 1 -- 172.21.15.159:0/3472971138 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf8fcc1330 con 0x55cf8fcc1520 2024-04-04T06:14:11.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.723+0000 7fc08be00640 1 --2- 172.21.15.159:0/3472971138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcc1520 0x55cf8fcc04d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fc07c000c00 tx=0x7fc07c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190966cb0c51f303 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:11.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.723+0000 7fc07b400640 1 -- 172.21.15.159:0/3472971138 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c031ab0 con 0x55cf8fcc1520 2024-04-04T06:14:11.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.724+0000 7fc07b400640 1 -- 172.21.15.159:0/3472971138 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc07c037040 con 0x55cf8fcc1520 2024-04-04T06:14:11.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.724+0000 7fc07b400640 1 -- 172.21.15.159:0/3472971138 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c031e50 con 0x55cf8fcc1520 2024-04-04T06:14:11.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.724+0000 7fc09564d740 1 -- 172.21.15.159:0/3472971138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcc1520 msgr2=0x55cf8fcc04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:11.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.724+0000 7fc09564d740 1 --2- 172.21.15.159:0/3472971138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcc1520 0x55cf8fcc04d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fc07c000c00 tx=0x7fc07c008030 comp rx=0 tx=0).stop 2024-04-04T06:14:11.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.724+0000 7fc09564d740 1 -- 172.21.15.159:0/3472971138 shutdown_connections 2024-04-04T06:14:11.719 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.724+0000 7fc09564d740 1 --2- 172.21.15.159:0/3472971138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf8fcd4510 0x55cf8fca6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.720 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.724+0000 7fc09564d740 1 --2- 172.21.15.159:0/3472971138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf8fcd4130 0x55cf8fcceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.720 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.724+0000 7fc09564d740 1 --2- 172.21.15.159:0/3472971138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcc1520 0x55cf8fcc04d0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.720 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.724+0000 7fc09564d740 1 -- 172.21.15.159:0/3472971138 >> 172.21.15.159:0/3472971138 conn(0x55cf8fca7610 msgr2=0x55cf8fca23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:11.720 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.724+0000 7fc09564d740 1 -- 172.21.15.159:0/3472971138 shutdown_connections 2024-04-04T06:14:11.720 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.724+0000 7fc09564d740 1 -- 172.21.15.159:0/3472971138 wait complete. 2024-04-04T06:14:11.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.725+0000 7fc09564d740 1 -- start start 2024-04-04T06:14:11.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.725+0000 7fc09564d740 1 Processor -- start 2024-04-04T06:14:11.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.726+0000 7fc09564d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcc1520 0x55cf8fd6bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.726+0000 7fc09564d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf8fcd4130 0x55cf8fcce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.726+0000 7fc09564d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf8fcd4510 0x55cf8fcd2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.726+0000 7fc09564d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cf8fd052a0 con 0x55cf8fcc1520 2024-04-04T06:14:11.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.726+0000 7fc09564d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cf8fbcc920 con 0x55cf8fcd4130 2024-04-04T06:14:11.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.726+0000 7fc09564d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55cf8fd058f0 con 0x55cf8fcd4510 2024-04-04T06:14:11.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.726+0000 7fc09564d740 10 client.0 _ll_register_callbacks cb 0x7fc074016400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:14:11.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.726+0000 7fc08be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcc1520 0x55cf8fd6bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:11.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.726+0000 7fc08be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcc1520 0x55cf8fd6bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33800/0 (socket says 172.21.15.159:33800) 2024-04-04T06:14:11.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.726+0000 7fc08be00640 1 -- 172.21.15.159:0/887563482 learned_addr learned my addr 172.21.15.159:0/887563482 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:11.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.727+0000 7fc08be00640 1 -- 172.21.15.159:0/887563482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf8fcd4510 msgr2=0x55cf8fcd2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:11.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.727+0000 7fc08be00640 1 --2- 172.21.15.159:0/887563482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf8fcd4510 0x55cf8fcd2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.727+0000 7fc08be00640 1 -- 172.21.15.159:0/887563482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf8fcd4130 msgr2=0x55cf8fcce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:11.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.727+0000 7fc08be00640 1 --2- 172.21.15.159:0/887563482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf8fcd4130 0x55cf8fcce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.727+0000 7fc08be00640 1 -- 172.21.15.159:0/887563482 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf8fca7b90 con 0x55cf8fcc1520 2024-04-04T06:14:11.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.727+0000 7fc08be00640 1 --2- 172.21.15.159:0/887563482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcc1520 0x55cf8fd6bb70 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fc07c04b040 tx=0x7fc07c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df9c33b591c3ee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:11.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.727+0000 7fc078c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:14:11.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.727+0000 7fc078c00640 1 -- 172.21.15.159:0/887563482 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c031e40 con 0x55cf8fcc1520 2024-04-04T06:14:11.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.727+0000 7fc078c00640 1 -- 172.21.15.159:0/887563482 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7fc07c053bb0 con 0x55cf8fcc1520 2024-04-04T06:14:11.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.727+0000 7fc09564d740 1 -- 172.21.15.159:0/887563482 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc07c037040 con 0x55cf8fcc1520 2024-04-04T06:14:11.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.727+0000 7fc078c00640 1 -- 172.21.15.159:0/887563482 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fc07c053db0 con 0x55cf8fcc1520 2024-04-04T06:14:11.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.728+0000 7fc09564d740 10 client.7887 fetch_fsmap learned FSMap version 76 2024-04-04T06:14:11.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.728+0000 7fc09564d740 1 -- 172.21.15.159:0/887563482 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55cf8fcd2e20 con 0x55cf8fcc1520 2024-04-04T06:14:11.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.728+0000 7fc078c00640 1 -- 172.21.15.159:0/887563482 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc07c037040 con 0x55cf8fcc1520 2024-04-04T06:14:11.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.728+0000 7fc078c00640 1 -- 172.21.15.159:0/887563482 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 76) ==== 3494+0+0 (secure 0 0 0) 0x7fc07c045070 con 0x55cf8fcc1520 2024-04-04T06:14:11.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.728+0000 7fc09564d740 10 client.7887 fetch_fsmap finished waiting for FSMap version 76 2024-04-04T06:14:11.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.728+0000 7fc09564d740 10 client.7887 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:14:11.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.728+0000 7fc09564d740 20 client.7887 populate_metadata read hostname 'smithi159' 2024-04-04T06:14:11.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.728+0000 7fc09564d740 1 --2- 172.21.15.159:0/887563482 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55cf8fcd78f0 0x55cf8fcf3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.728+0000 7fc09564d740 4 client.7887 mds_command: new command op to 7809 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-04T06:14:11.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.728+0000 7fc09564d740 1 -- 172.21.15.159:0/887563482 --> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] -- command(tid 0: {"prefix": "status"}) -- 0x55cf8fcda6a0 con 0x55cf8fcd78f0 2024-04-04T06:14:11.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.728+0000 7fc083e00640 1 --2- 172.21.15.159:0/887563482 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55cf8fcd78f0 0x55cf8fcf3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:11.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.729+0000 7fc083e00640 1 --2- 172.21.15.159:0/887563482 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55cf8fcd78f0 0x55cf8fcf3060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=156e9fe94dcf0da2 server_cookie=68a116132cddce0f in_seq=0 out_seq=0 2024-04-04T06:14:11.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.729+0000 7fc078c00640 10 client.7887 ms_handle_connect on v2:172.21.15.159:6833/3555825575 2024-04-04T06:14:11.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.729+0000 7fc078c00640 1 -- 172.21.15.159:0/887563482 <== mds.0 v2:172.21.15.159:6833/3555825575 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55cf8fcda6a0 con 0x55cf8fcd78f0 2024-04-04T06:14:11.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.729+0000 7fc078c00640 10 client.7887 handle_command_reply: tid=0 2024-04-04T06:14:11.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.729+0000 7fc09564d740 1 client.7887 shutdown 2024-04-04T06:14:11.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.729+0000 7fc09564d740 1 -- 172.21.15.159:0/887563482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcc1520 msgr2=0x55cf8fd6bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:11.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.729+0000 7fc09564d740 1 --2- 172.21.15.159:0/887563482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcc1520 0x55cf8fd6bb70 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fc07c04b040 tx=0x7fc07c004730 comp rx=0 tx=0).stop 2024-04-04T06:14:11.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.729+0000 7fc09564d740 1 -- 172.21.15.159:0/887563482 shutdown_connections 2024-04-04T06:14:11.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.729+0000 7fc09564d740 1 --2- 172.21.15.159:0/887563482 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55cf8fcd78f0 0x55cf8fcf3060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.729+0000 7fc09564d740 1 --2- 172.21.15.159:0/887563482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cf8fcd4510 0x55cf8fcd2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.729+0000 7fc09564d740 1 --2- 172.21.15.159:0/887563482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cf8fcd4130 0x55cf8fcce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.729+0000 7fc09564d740 1 --2- 172.21.15.159:0/887563482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf8fcc1520 0x55cf8fd6bb70 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.729+0000 7fc09564d740 1 -- 172.21.15.159:0/887563482 >> 172.21.15.159:0/887563482 conn(0x55cf8fca7610 msgr2=0x55cf8fcc04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:11.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.729+0000 7fc078c00640 0 client.7887 ms_handle_reset on v2:172.21.15.159:6833/3555825575 2024-04-04T06:14:11.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.729+0000 7fc09564d740 1 -- 172.21.15.159:0/887563482 shutdown_connections 2024-04-04T06:14:11.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.730+0000 7fc09564d740 1 -- 172.21.15.159:0/887563482 wait complete. 2024-04-04T06:14:11.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.730+0000 7fc09564d740 20 client.7887 trim_cache size 0 max 16384 2024-04-04T06:14:11.728 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:14:11.728 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_fsid": "30311045-f714-48fa-832b-4cb2ba8bf0bf", 2024-04-04T06:14:11.728 INFO:teuthology.orchestra.run.smithi159.stdout: "whoami": 0, 2024-04-04T06:14:11.728 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 7809, 2024-04-04T06:14:11.728 INFO:teuthology.orchestra.run.smithi159.stdout: "want_state": "up:active", 2024-04-04T06:14:11.728 INFO:teuthology.orchestra.run.smithi159.stdout: "state": "up:active", 2024-04-04T06:14:11.728 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs", 2024-04-04T06:14:11.728 INFO:teuthology.orchestra.run.smithi159.stdout: "rank_uptime": 4.9411926810000004, 2024-04-04T06:14:11.728 INFO:teuthology.orchestra.run.smithi159.stdout: "mdsmap_epoch": 76, 2024-04-04T06:14:11.728 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch": 89, 2024-04-04T06:14:11.728 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch_barrier": 89, 2024-04-04T06:14:11.728 INFO:teuthology.orchestra.run.smithi159.stdout: "uptime": 6.8775500330000003 2024-04-04T06:14:11.729 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:14:11.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.730+0000 7fc092a00640 1 -- 172.21.15.159:0/2879130748 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc06806c8f0 msgr2=0x7fc06806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:11.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.730+0000 7fc092a00640 1 --2- 172.21.15.159:0/2879130748 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc06806c8f0 0x7fc06806edb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc074020000 tx=0x7fc07401e000 comp rx=0 tx=0).stop 2024-04-04T06:14:11.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.730+0000 7fc092a00640 1 -- 172.21.15.159:0/2879130748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc08c153580 msgr2=0x7fc08c1f54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:11.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.730+0000 7fc092a00640 1 --2- 172.21.15.159:0/2879130748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc08c153580 0x7fc08c1f54f0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fc08400c6b0 tx=0x7fc08400cb70 comp rx=0 tx=0).stop 2024-04-04T06:14:11.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.730+0000 7fc092a00640 1 -- 172.21.15.159:0/2879130748 shutdown_connections 2024-04-04T06:14:11.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.730+0000 7fc092a00640 1 --2- 172.21.15.159:0/2879130748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc08c153580 0x7fc08c1f54f0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.730+0000 7fc092a00640 1 --2- 172.21.15.159:0/2879130748 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc08c0d07c0 0x7fc08c0cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.730+0000 7fc092a00640 1 --2- 172.21.15.159:0/2879130748 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc06806c8f0 0x7fc06806edb0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.730+0000 7fc092a00640 1 --2- 172.21.15.159:0/2879130748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc08c0af2e0 0x7fc08c152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.730+0000 7fc092a00640 1 -- 172.21.15.159:0/2879130748 >> 172.21.15.159:0/2879130748 conn(0x7fc08c06b750 msgr2=0x7fc08c158010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:11.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.731+0000 7fc092a00640 1 -- 172.21.15.159:0/2879130748 shutdown_connections 2024-04-04T06:14:11.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.731+0000 7fc092a00640 1 -- 172.21.15.159:0/2879130748 wait complete. 2024-04-04T06:14:11.740 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-04T06:14:11.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.926+0000 7f46dc800640 1 Processor -- start 2024-04-04T06:14:11.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.927+0000 7f46dc800640 1 -- start start 2024-04-04T06:14:11.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.928+0000 7f46dc800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46d40d0990 0x7f46d40aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.928+0000 7f46dc800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f46d40af2e0 0x7f46d41600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.928+0000 7f46dc800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46d40afab0 con 0x7f46d4153580 2024-04-04T06:14:11.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.928+0000 7f46dc800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f46d415cc80 con 0x7f46d40af2e0 2024-04-04T06:14:11.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.928+0000 7f46dc800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f46d415da00 con 0x7f46d40d0990 2024-04-04T06:14:11.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.928+0000 7f46d9e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f46d4153580 0x7f46d40d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:54316/0 (socket says 172.21.15.159:54316) 2024-04-04T06:14:11.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.928+0000 7f46d9e00640 1 -- 172.21.15.159:0/425007517 learned_addr learned my addr 172.21.15.159:0/425007517 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:11.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.928+0000 7f46da800640 1 --2- 172.21.15.159:0/425007517 >> v2:172.21.15.159:3300/0 conn(0x7f46d40af2e0 0x7f46d41600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:11.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.929+0000 7f46d9400640 1 --2- 172.21.15.159:0/425007517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46d40d0990 0x7f46d40aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:11.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.929+0000 7f46d8a00640 1 -- 172.21.15.159:0/425007517 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3608473922 0 0) 0x7f46d40afab0 con 0x7f46d4153580 2024-04-04T06:14:11.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.929+0000 7f46da800640 1 -- 172.21.15.159:0/425007517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46d40d0990 msgr2=0x7f46d40aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:11.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.929+0000 7f46da800640 1 --2- 172.21.15.159:0/425007517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46d40d0990 0x7f46d40aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.929+0000 7f46da800640 1 -- 172.21.15.159:0/425007517 >> v1:172.21.15.159:6789/0 conn(0x7f46d4153580 legacy=0x7f46d40d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:11.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.929+0000 7f46da800640 1 -- 172.21.15.159:0/425007517 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d415cf20 con 0x7f46d40af2e0 2024-04-04T06:14:11.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.931+0000 7f46da800640 1 --2- 172.21.15.159:0/425007517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46d40af2e0 0x7f46d41600d0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009d90 tx=0x7f46d002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3470ba1857d10dac server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:11.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.931+0000 7f46d8a00640 1 -- 172.21.15.159:0/425007517 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0039070 con 0x7f46d40af2e0 2024-04-04T06:14:11.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.931+0000 7f46d8a00640 1 -- 172.21.15.159:0/425007517 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d002cbc0 con 0x7f46d40af2e0 2024-04-04T06:14:11.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.931+0000 7f46d8a00640 1 -- 172.21.15.159:0/425007517 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0034440 con 0x7f46d40af2e0 2024-04-04T06:14:11.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.931+0000 7f46dc800640 1 -- 172.21.15.159:0/425007517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46d40af2e0 msgr2=0x7f46d41600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:11.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.931+0000 7f46dc800640 1 --2- 172.21.15.159:0/425007517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46d40af2e0 0x7f46d41600d0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009d90 tx=0x7f46d002be10 comp rx=0 tx=0).stop 2024-04-04T06:14:11.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.932+0000 7f46dc800640 1 -- 172.21.15.159:0/425007517 shutdown_connections 2024-04-04T06:14:11.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.932+0000 7f46dc800640 1 --2- 172.21.15.159:0/425007517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46d40d0990 0x7f46d40aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.932+0000 7f46dc800640 1 --2- 172.21.15.159:0/425007517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46d40af2e0 0x7f46d41600d0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.932+0000 7f46dc800640 1 -- 172.21.15.159:0/425007517 >> 172.21.15.159:0/425007517 conn(0x7f46d406b750 msgr2=0x7f46d406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:11.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.932+0000 7f46dc800640 1 -- 172.21.15.159:0/425007517 shutdown_connections 2024-04-04T06:14:11.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.932+0000 7f46dc800640 1 -- 172.21.15.159:0/425007517 wait complete. 2024-04-04T06:14:11.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.933+0000 7f46dc800640 1 Processor -- start 2024-04-04T06:14:11.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.933+0000 7f46dc800640 1 -- start start 2024-04-04T06:14:11.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.934+0000 7f46dc800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46d40af2e0 0x7f46d40cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.934+0000 7f46dc800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46d40d0990 0x7f46d414e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.934+0000 7f46dc800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46d4153580 0x7f46d4157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.934+0000 7f46dc800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f46d4153960 con 0x7f46d40af2e0 2024-04-04T06:14:11.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.934+0000 7f46dc800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f46d415d4e0 con 0x7f46d40d0990 2024-04-04T06:14:11.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.934+0000 7f46dc800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f46d40af710 con 0x7f46d4153580 2024-04-04T06:14:11.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.934+0000 7f46d9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46d40af2e0 0x7f46d40cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:11.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.934+0000 7f46d9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46d40af2e0 0x7f46d40cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33822/0 (socket says 172.21.15.159:33822) 2024-04-04T06:14:11.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.934+0000 7f46d9e00640 1 -- 172.21.15.159:0/2758365718 learned_addr learned my addr 172.21.15.159:0/2758365718 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:11.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.935+0000 7f46d9e00640 1 -- 172.21.15.159:0/2758365718 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46d4153580 msgr2=0x7f46d4157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:11.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.935+0000 7f46d9e00640 1 --2- 172.21.15.159:0/2758365718 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46d4153580 0x7f46d4157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.935+0000 7f46d9e00640 1 -- 172.21.15.159:0/2758365718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46d40d0990 msgr2=0x7f46d414e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:11.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.935+0000 7f46d9e00640 1 --2- 172.21.15.159:0/2758365718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46d40d0990 0x7f46d414e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:11.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.935+0000 7f46d9e00640 1 -- 172.21.15.159:0/2758365718 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d414ea70 con 0x7f46d40af2e0 2024-04-04T06:14:11.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.935+0000 7f46d9e00640 1 --2- 172.21.15.159:0/2758365718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46d40af2e0 0x7f46d40cca50 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f46c800c500 tx=0x7f46c800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:11.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.935+0000 7f46ce000640 1 -- 172.21.15.159:0/2758365718 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8018070 con 0x7f46d40af2e0 2024-04-04T06:14:11.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.935+0000 7f46ce000640 1 -- 172.21.15.159:0/2758365718 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c8007580 con 0x7f46d40af2e0 2024-04-04T06:14:11.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.935+0000 7f46ce000640 1 -- 172.21.15.159:0/2758365718 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8004a20 con 0x7f46d40af2e0 2024-04-04T06:14:11.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.935+0000 7f46dc800640 1 -- 172.21.15.159:0/2758365718 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d40cd030 con 0x7f46d40af2e0 2024-04-04T06:14:11.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.936+0000 7f46dc800640 1 -- 172.21.15.159:0/2758365718 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d40cd5f0 con 0x7f46d40af2e0 2024-04-04T06:14:11.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.937+0000 7f46dc800640 1 -- 172.21.15.159:0/2758365718 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d40685c0 con 0x7f46d40af2e0 2024-04-04T06:14:11.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.938+0000 7f46ce000640 1 -- 172.21.15.159:0/2758365718 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46c801b070 con 0x7f46d40af2e0 2024-04-04T06:14:11.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.939+0000 7f46ce000640 1 --2- 172.21.15.159:0/2758365718 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f46b806c8f0 0x7f46b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:11.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.939+0000 7f46ce000640 1 -- 172.21.15.159:0/2758365718 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f46c808afb0 con 0x7f46d40af2e0 2024-04-04T06:14:11.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.940+0000 7f46d9400640 1 --2- 172.21.15.159:0/2758365718 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f46b806c8f0 0x7f46b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:11.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.941+0000 7f46d9400640 1 --2- 172.21.15.159:0/2758365718 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f46b806c8f0 0x7f46b806edb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f46c4006fd0 tx=0x7f46c4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:11.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:11.942+0000 7f46ce000640 1 -- 172.21.15.159:0/2758365718 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f46d414ea70 con 0x7f46d40af2e0 2024-04-04T06:14:12.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:12.180+0000 7f46dc800640 1 -- 172.21.15.159:0/2758365718 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f46d406eca0 con 0x7f46d40af2e0 2024-04-04T06:14:12.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:12.181+0000 7f46ce000640 1 -- 172.21.15.159:0/2758365718 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) ==== 53+0+73 (secure 0 0 0) 0x7f46c80592e0 con 0x7f46d40af2e0 2024-04-04T06:14:12.175 INFO:teuthology.orchestra.run.smithi159.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-04T06:14:12.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:12.183+0000 7f46dc800640 1 -- 172.21.15.159:0/2758365718 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f46b806c8f0 msgr2=0x7f46b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:12.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:12.183+0000 7f46dc800640 1 --2- 172.21.15.159:0/2758365718 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f46b806c8f0 0x7f46b806edb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f46c4006fd0 tx=0x7f46c4008040 comp rx=0 tx=0).stop 2024-04-04T06:14:12.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:12.183+0000 7f46dc800640 1 -- 172.21.15.159:0/2758365718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46d40af2e0 msgr2=0x7f46d40cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:12.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:12.183+0000 7f46dc800640 1 --2- 172.21.15.159:0/2758365718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46d40af2e0 0x7f46d40cca50 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f46c800c500 tx=0x7f46c800c9c0 comp rx=0 tx=0).stop 2024-04-04T06:14:12.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:12.183+0000 7f46dc800640 1 -- 172.21.15.159:0/2758365718 shutdown_connections 2024-04-04T06:14:12.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:12.183+0000 7f46dc800640 1 --2- 172.21.15.159:0/2758365718 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46d4153580 0x7f46d4157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:12.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:12.183+0000 7f46dc800640 1 --2- 172.21.15.159:0/2758365718 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f46b806c8f0 0x7f46b806edb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:12.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:12.183+0000 7f46dc800640 1 --2- 172.21.15.159:0/2758365718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46d40d0990 0x7f46d414e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:12.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:12.183+0000 7f46dc800640 1 --2- 172.21.15.159:0/2758365718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46d40af2e0 0x7f46d40cca50 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:12.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:12.183+0000 7f46dc800640 1 -- 172.21.15.159:0/2758365718 >> 172.21.15.159:0/2758365718 conn(0x7f46d406b750 msgr2=0x7f46d415e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:12.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:12.183+0000 7f46dc800640 1 -- 172.21.15.159:0/2758365718 shutdown_connections 2024-04-04T06:14:12.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:12.183+0000 7f46dc800640 1 -- 172.21.15.159:0/2758365718 wait complete. 2024-04-04T06:14:12.197 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-04T06:14:12.197 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-04T06:14:12.197 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi159.front.sepia.ceph.com 2024-04-04T06:14:12.197 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-04T06:14:12.197 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-04T06:14:12.197 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-04T06:14:12.197 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-04T06:14:12.197 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-04T06:14:12.197 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:14:12.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:14:12.198 DEBUG:teuthology.orchestra.run.smithi159:> ip addr 2024-04-04T06:14:12.250 INFO:teuthology.orchestra.run.smithi159.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:14:12.250 INFO:teuthology.orchestra.run.smithi159.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:14:12.251 INFO:teuthology.orchestra.run.smithi159.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:14:12.251 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:14:12.251 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 ::1/128 scope host 2024-04-04T06:14:12.251 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:14:12.251 INFO:teuthology.orchestra.run.smithi159.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:14:12.251 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c6 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:14:12.251 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f0 2024-04-04T06:14:12.251 INFO:teuthology.orchestra.run.smithi159.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:14:12.251 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c7 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:14:12.251 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f1 2024-04-04T06:14:12.251 INFO:teuthology.orchestra.run.smithi159.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:14:12.251 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4e brd ff:ff:ff:ff:ff:ff 2024-04-04T06:14:12.252 INFO:teuthology.orchestra.run.smithi159.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-04T06:14:12.252 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:14:12.252 INFO:teuthology.orchestra.run.smithi159.stdout: inet 172.21.15.159/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-04T06:14:12.252 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft 42081sec preferred_lft 42081sec 2024-04-04T06:14:12.252 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::ec4:7aff:fe88:724f/64 scope link 2024-04-04T06:14:12.252 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:14:12.252 INFO:teuthology.orchestra.run.smithi159.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-04T06:14:12.252 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:14:12.252 INFO:teuthology.orchestra.run.smithi159.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-04T06:14:12.252 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:14:12.252 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::989e:e4ff:fea5:7568/64 scope link 2024-04-04T06:14:12.252 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:14:12.252 INFO:teuthology.orchestra.run.smithi159.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:14:12.253 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether a2:e3:e0:cb:0a:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-04T06:14:12.253 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::a0e3:e0ff:fecb:a73/64 scope link 2024-04-04T06:14:12.253 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:14:12.253 INFO:teuthology.orchestra.run.smithi159.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:14:12.253 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-04T06:14:12.253 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::c0a:8aff:fe54:b58f/64 scope link 2024-04-04T06:14:12.253 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:14:12.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:14:12.254 DEBUG:teuthology.orchestra.run.smithi159:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-04T06:14:12.346 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:14:12.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:14:12.349 DEBUG:teuthology.orchestra.run.smithi159:> route 2024-04-04T06:14:12.396 INFO:teuthology.orchestra.run.smithi159.stdout:Kernel IP routing table 2024-04-04T06:14:12.396 INFO:teuthology.orchestra.run.smithi159.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-04T06:14:12.396 INFO:teuthology.orchestra.run.smithi159.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-04T06:14:12.396 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-04T06:14:12.396 INFO:teuthology.orchestra.run.smithi159.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-04T06:14:12.397 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:14:12.397 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:14:12.397 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-04T06:14:12.397 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-04T06:14:12.397 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-04T06:14:12.397 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:14:12.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:14:12.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:14:12.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:14:12.609 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-04-04T06:14:12.657 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-04T06:14:12.657 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-04T06:14:12.657 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:14:12.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:14:12.658 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set brx.0 up 2024-04-04T06:14:12.738 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:14:12.738 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-04T06:14:12.793 INFO:teuthology.orchestra.run.smithi159.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:14:12.793 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:14:12.793 DEBUG:teuthology.orchestra.run.smithi159:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:14:12.840 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-04T06:14:12.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:14:12.841 DEBUG:teuthology.orchestra.run.smithi159:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:14:12.929 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:14:12.936 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:14:13.008 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:14:13.008 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:14:13.008 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:14:13.027 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:14:13.027 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:14:13.027 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:14:13.027 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:14:13.027 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:14:13.052 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:14:13.052 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:14:13.052 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:14:13.052 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:14:13.052 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:14:13.052 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-04T06:14:13.052 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-04T06:14:13.052 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30311045-f714-48fa-832b-4cb2ba8bf0bf" will pass to kernel 2024-04-04T06:14:13.055 INFO:teuthology.orchestra.run.smithi159.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-04T06:14:13.055 INFO:teuthology.orchestra.run.smithi159.stdout: You just mounted a file system that supports labels which does not 2024-04-04T06:14:13.055 INFO:teuthology.orchestra.run.smithi159.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-04T06:14:13.055 INFO:teuthology.orchestra.run.smithi159.stdout: applications will generate AVC messages and not be allowed access to 2024-04-04T06:14:13.055 INFO:teuthology.orchestra.run.smithi159.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-04T06:14:13.059 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-04T06:14:13.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:14:13.059 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:14:13.185 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:14:13.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:14:13.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:14:13.298 INFO:teuthology.orchestra.run.smithi159.stdout:client5468 2024-04-04T06:14:13.298 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:14:13.298 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:14:13.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:14:13.344 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-04T06:14:13.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.603+0000 7fd5d3c00640 1 Processor -- start 2024-04-04T06:14:13.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.604+0000 7fd5d3c00640 1 -- start start 2024-04-04T06:14:13.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.605+0000 7fd5d3c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5cc161f10 0x7fd5cc0d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.605+0000 7fd5d3c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd5cc06e110 0x7fd5cc0d4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.605+0000 7fd5d3c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5cc06d370 con 0x7fd5cc0b3270 2024-04-04T06:14:13.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.605+0000 7fd5d3c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fd5cc06d650 con 0x7fd5cc06e110 2024-04-04T06:14:13.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.605+0000 7fd5d3c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd5cc0d0f00 con 0x7fd5cc161f10 2024-04-04T06:14:13.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.605+0000 7fd5d1c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd5cc06e110 0x7fd5cc0d4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.606+0000 7fd5d1c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd5cc06e110 0x7fd5cc0d4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33832/0 (socket says 172.21.15.159:33832) 2024-04-04T06:14:13.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.606+0000 7fd5d1c00640 1 -- 172.21.15.159:0/2397040567 learned_addr learned my addr 172.21.15.159:0/2397040567 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:13.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.606+0000 7fd5d1200640 1 --2- 172.21.15.159:0/2397040567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5cc161f10 0x7fd5cc0d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.606+0000 7fd5d1c00640 1 -- 172.21.15.159:0/2397040567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5cc161f10 msgr2=0x7fd5cc0d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:13.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.606+0000 7fd5d1c00640 1 --2- 172.21.15.159:0/2397040567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5cc161f10 0x7fd5cc0d19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.606+0000 7fd5d1c00640 1 -- 172.21.15.159:0/2397040567 >> v1:172.21.15.178:6789/0 conn(0x7fd5cc0b3270 legacy=0x7fd5cc06da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:13.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.606+0000 7fd5d1c00640 1 -- 172.21.15.159:0/2397040567 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5cc0d11e0 con 0x7fd5cc06e110 2024-04-04T06:14:13.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.608+0000 7fd5d1c00640 1 --2- 172.21.15.159:0/2397040567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5cc06e110 0x7fd5cc0d4b10 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009e50 tx=0x7fd5c402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f97a33f7a5f7362 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:13.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.609+0000 7fd5cb400640 1 -- 172.21.15.159:0/2397040567 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c402ca70 con 0x7fd5cc06e110 2024-04-04T06:14:13.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.609+0000 7fd5cb400640 1 -- 172.21.15.159:0/2397040567 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c402cc10 con 0x7fd5cc06e110 2024-04-04T06:14:13.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.609+0000 7fd5cb400640 1 -- 172.21.15.159:0/2397040567 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c40343f0 con 0x7fd5cc06e110 2024-04-04T06:14:13.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.609+0000 7fd5d3c00640 1 -- 172.21.15.159:0/2397040567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5cc06e110 msgr2=0x7fd5cc0d4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:13.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.609+0000 7fd5d3c00640 1 --2- 172.21.15.159:0/2397040567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5cc06e110 0x7fd5cc0d4b10 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009e50 tx=0x7fd5c402bed0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.609+0000 7fd5d3c00640 1 -- 172.21.15.159:0/2397040567 shutdown_connections 2024-04-04T06:14:13.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.609+0000 7fd5d3c00640 1 --2- 172.21.15.159:0/2397040567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5cc06e110 0x7fd5cc0d4b10 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.609+0000 7fd5d3c00640 1 --2- 172.21.15.159:0/2397040567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5cc161f10 0x7fd5cc0d19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.609+0000 7fd5d3c00640 1 -- 172.21.15.159:0/2397040567 >> 172.21.15.159:0/2397040567 conn(0x7fd5cc08bfb0 msgr2=0x7fd5cc0d4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:13.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.610+0000 7fd5d3c00640 1 -- 172.21.15.159:0/2397040567 shutdown_connections 2024-04-04T06:14:13.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.610+0000 7fd5d3c00640 1 -- 172.21.15.159:0/2397040567 wait complete. 2024-04-04T06:14:13.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.611+0000 7fd5d3c00640 1 Processor -- start 2024-04-04T06:14:13.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.611+0000 7fd5d3c00640 1 -- start start 2024-04-04T06:14:13.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.612+0000 7fd5d3c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5cc06e110 0x7fd5cc1f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.612+0000 7fd5d3c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5cc0b3270 0x7fd5cc1f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.612+0000 7fd5d3c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5cc161f10 0x7fd5cc0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.612+0000 7fd5d3c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc06cff0 con 0x7fd5cc0b3270 2024-04-04T06:14:13.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.612+0000 7fd5d3c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc1622f0 con 0x7fd5cc06e110 2024-04-04T06:14:13.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.612+0000 7fd5d3c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd5cc0d2300 con 0x7fd5cc161f10 2024-04-04T06:14:13.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.612+0000 7fd5cbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5cc0b3270 0x7fd5cc1f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.613+0000 7fd5cbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5cc0b3270 0x7fd5cc1f5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33844/0 (socket says 172.21.15.159:33844) 2024-04-04T06:14:13.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.613+0000 7fd5cbe00640 1 -- 172.21.15.159:0/3550569755 learned_addr learned my addr 172.21.15.159:0/3550569755 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:13.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.613+0000 7fd5cbe00640 1 -- 172.21.15.159:0/3550569755 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5cc161f10 msgr2=0x7fd5cc0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:13.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.613+0000 7fd5d1c00640 1 --2- 172.21.15.159:0/3550569755 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5cc161f10 0x7fd5cc0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.613+0000 7fd5d1200640 1 --2- 172.21.15.159:0/3550569755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5cc06e110 0x7fd5cc1f16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.613+0000 7fd5cbe00640 1 --2- 172.21.15.159:0/3550569755 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5cc161f10 0x7fd5cc0cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.613+0000 7fd5cbe00640 1 -- 172.21.15.159:0/3550569755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5cc06e110 msgr2=0x7fd5cc1f16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:13.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.613+0000 7fd5cbe00640 1 --2- 172.21.15.159:0/3550569755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5cc06e110 0x7fd5cc1f16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.613+0000 7fd5cbe00640 1 -- 172.21.15.159:0/3550569755 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5cc1f1c00 con 0x7fd5cc0b3270 2024-04-04T06:14:13.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.613+0000 7fd5cbe00640 1 --2- 172.21.15.159:0/3550569755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5cc0b3270 0x7fd5cc1f5390 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800c430 tx=0x7fd5b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:13.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.614+0000 7fd5c8c00640 1 -- 172.21.15.159:0/3550569755 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8011790 con 0x7fd5cc0b3270 2024-04-04T06:14:13.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.614+0000 7fd5c8c00640 1 -- 172.21.15.159:0/3550569755 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5b8017070 con 0x7fd5cc0b3270 2024-04-04T06:14:13.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.614+0000 7fd5d3c00640 1 -- 172.21.15.159:0/3550569755 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5cc0b50c0 con 0x7fd5cc0b3270 2024-04-04T06:14:13.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.614+0000 7fd5c8c00640 1 -- 172.21.15.159:0/3550569755 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8011bb0 con 0x7fd5cc0b3270 2024-04-04T06:14:13.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.614+0000 7fd5d3c00640 1 -- 172.21.15.159:0/3550569755 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5cc0b5680 con 0x7fd5cc0b3270 2024-04-04T06:14:13.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.616+0000 7fd5c8c00640 1 -- 172.21.15.159:0/3550569755 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd5b801c460 con 0x7fd5cc0b3270 2024-04-04T06:14:13.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.617+0000 7fd5c8c00640 1 --2- 172.21.15.159:0/3550569755 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd5ac06c8f0 0x7fd5ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.617+0000 7fd5c8c00640 1 -- 172.21.15.159:0/3550569755 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7fd5b808bad0 con 0x7fd5cc0b3270 2024-04-04T06:14:13.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.618+0000 7fd5d1200640 1 --2- 172.21.15.159:0/3550569755 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd5ac06c8f0 0x7fd5ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.618+0000 7fd5d683d740 1 Processor -- start 2024-04-04T06:14:13.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.618+0000 7fd5d683d740 1 -- start start 2024-04-04T06:14:13.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.619+0000 7fd5d683d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16603fc0 0x55ba166043a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.619+0000 7fd5d683d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba16609de0 0x55ba16608010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.619+0000 7fd5d683d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba166516d0 0x55ba16651b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.619+0000 7fd5d683d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ba1656e280 con 0x55ba16603fc0 2024-04-04T06:14:13.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.619+0000 7fd5d683d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ba165166f0 con 0x55ba166516d0 2024-04-04T06:14:13.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.619+0000 7fd5d683d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ba1656a230 con 0x55ba16609de0 2024-04-04T06:14:13.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.619+0000 7fd5d1200640 1 --2- 172.21.15.159:0/3550569755 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd5ac06c8f0 0x7fd5ac06edb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0006fd0 tx=0x7fd5c0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:13.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.619+0000 7fd5d1200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16603fc0 0x55ba166043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.619+0000 7fd5d1200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16603fc0 0x55ba166043a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33858/0 (socket says 172.21.15.159:33858) 2024-04-04T06:14:13.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.619+0000 7fd5d1200640 1 -- 172.21.15.159:0/3991396761 learned_addr learned my addr 172.21.15.159:0/3991396761 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:13.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.619+0000 7fd5cbe00640 1 --2- 172.21.15.159:0/3991396761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba16609de0 0x55ba16608010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.619+0000 7fd5d1c00640 1 --2- 172.21.15.159:0/3991396761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba166516d0 0x55ba16651b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.619+0000 7fd5d1200640 1 -- 172.21.15.159:0/3991396761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba16609de0 msgr2=0x55ba16608010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:13.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.619+0000 7fd5d1200640 1 --2- 172.21.15.159:0/3991396761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba16609de0 0x55ba16608010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.619+0000 7fd5d1200640 1 -- 172.21.15.159:0/3991396761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba166516d0 msgr2=0x55ba16651b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:13.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.619+0000 7fd5d1200640 1 --2- 172.21.15.159:0/3991396761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba166516d0 0x55ba16651b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.619+0000 7fd5d1200640 1 -- 172.21.15.159:0/3991396761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba1634a7e0 con 0x55ba16603fc0 2024-04-04T06:14:13.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.619+0000 7fd5cbe00640 1 --2- 172.21.15.159:0/3991396761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba16609de0 0x55ba16608010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:13.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.620+0000 7fd5d1c00640 1 --2- 172.21.15.159:0/3991396761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba166516d0 0x55ba16651b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:13.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.620+0000 7fd5d1200640 1 --2- 172.21.15.159:0/3991396761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16603fc0 0x55ba166043a0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000e830 tx=0x7fd5c0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752dc22dc9996bbd server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:13.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.620+0000 7fd5bd600640 1 -- 172.21.15.159:0/3991396761 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0025070 con 0x55ba16603fc0 2024-04-04T06:14:13.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.620+0000 7fd5bd600640 1 -- 172.21.15.159:0/3991396761 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c0015640 con 0x55ba16603fc0 2024-04-04T06:14:13.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.620+0000 7fd5bd600640 1 -- 172.21.15.159:0/3991396761 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c000f040 con 0x55ba16603fc0 2024-04-04T06:14:13.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.621+0000 7fd5d683d740 1 -- 172.21.15.159:0/3991396761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16603fc0 msgr2=0x55ba166043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.621+0000 7fd5d683d740 1 --2- 172.21.15.159:0/3991396761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16603fc0 0x55ba166043a0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000e830 tx=0x7fd5c0011000 comp rx=0 tx=0).stop 2024-04-04T06:14:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.621+0000 7fd5d683d740 1 -- 172.21.15.159:0/3991396761 shutdown_connections 2024-04-04T06:14:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.621+0000 7fd5d683d740 1 --2- 172.21.15.159:0/3991396761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba166516d0 0x55ba16651b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.621+0000 7fd5d683d740 1 --2- 172.21.15.159:0/3991396761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba16609de0 0x55ba16608010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.621+0000 7fd5d683d740 1 --2- 172.21.15.159:0/3991396761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16603fc0 0x55ba166043a0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.621+0000 7fd5d683d740 1 -- 172.21.15.159:0/3991396761 >> 172.21.15.159:0/3991396761 conn(0x55ba16606040 msgr2=0x55ba16607900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.621+0000 7fd5d683d740 1 -- 172.21.15.159:0/3991396761 shutdown_connections 2024-04-04T06:14:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.621+0000 7fd5d683d740 1 -- 172.21.15.159:0/3991396761 wait complete. 2024-04-04T06:14:13.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.622+0000 7fd5d683d740 1 -- start start 2024-04-04T06:14:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.622+0000 7fd5d683d740 1 Processor -- start 2024-04-04T06:14:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.623+0000 7fd5d683d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba16609de0 0x55ba1665fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.623+0000 7fd5d683d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba166516d0 0x55ba16633e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.623+0000 7fd5d683d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba166358d0 0x55ba16637870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.623+0000 7fd5d683d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ba16630730 con 0x55ba166516d0 2024-04-04T06:14:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.623+0000 7fd5d683d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ba16529aa0 con 0x55ba166358d0 2024-04-04T06:14:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.623+0000 7fd5d683d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ba16630980 con 0x55ba16609de0 2024-04-04T06:14:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.623+0000 7fd5d683d740 10 client.0 _ll_register_callbacks cb 0x55ba16529490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:14:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.623+0000 7fd5cbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba166516d0 0x55ba16633e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.623+0000 7fd5cbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba166516d0 0x55ba16633e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33872/0 (socket says 172.21.15.159:33872) 2024-04-04T06:14:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.623+0000 7fd5cbe00640 1 -- 172.21.15.159:0/2824174922 learned_addr learned my addr 172.21.15.159:0/2824174922 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.624+0000 7fd5d1c00640 1 --2- 172.21.15.159:0/2824174922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba166358d0 0x55ba16637870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.624+0000 7fd5cbe00640 1 -- 172.21.15.159:0/2824174922 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba16609de0 msgr2=0x55ba1665fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:13.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.624+0000 7fd5cbe00640 1 --2- 172.21.15.159:0/2824174922 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba16609de0 0x55ba1665fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.624+0000 7fd5cbe00640 1 -- 172.21.15.159:0/2824174922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba166358d0 msgr2=0x55ba16637870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:13.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.624+0000 7fd5cbe00640 1 --2- 172.21.15.159:0/2824174922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba166358d0 0x55ba16637870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.624+0000 7fd5cbe00640 1 -- 172.21.15.159:0/2824174922 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba163aaf70 con 0x55ba166516d0 2024-04-04T06:14:13.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.624+0000 7fd5d1c00640 1 --2- 172.21.15.159:0/2824174922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba166358d0 0x55ba16637870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:13.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.624+0000 7fd5cbe00640 1 --2- 172.21.15.159:0/2824174922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba166516d0 0x55ba16633e30 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8089f10 tx=0x7fd5b8059fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78bfd24b1010eec8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:13.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.624+0000 7fd5b2000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:14:13.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.624+0000 7fd5b2000640 1 -- 172.21.15.159:0/2824174922 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8062380 con 0x55ba166516d0 2024-04-04T06:14:13.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.624+0000 7fd5b2000640 1 -- 172.21.15.159:0/2824174922 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7fd5b8055e10 con 0x55ba166516d0 2024-04-04T06:14:13.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.624+0000 7fd5d683d740 1 -- 172.21.15.159:0/2824174922 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5c0015640 con 0x55ba166516d0 2024-04-04T06:14:13.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.624+0000 7fd5b2000640 1 -- 172.21.15.159:0/2824174922 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fd5b8061ef0 con 0x55ba166516d0 2024-04-04T06:14:13.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.625+0000 7fd5d683d740 10 client.7908 fetch_fsmap learned FSMap version 76 2024-04-04T06:14:13.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.625+0000 7fd5d683d740 1 -- 172.21.15.159:0/2824174922 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55ba16242170 con 0x55ba166516d0 2024-04-04T06:14:13.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.625+0000 7fd5b2000640 1 -- 172.21.15.159:0/2824174922 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c0015640 con 0x55ba166516d0 2024-04-04T06:14:13.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.625+0000 7fd5b2000640 1 -- 172.21.15.159:0/2824174922 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 76) ==== 3494+0+0 (secure 0 0 0) 0x7fd5b805ece0 con 0x55ba166516d0 2024-04-04T06:14:13.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.625+0000 7fd5d683d740 10 client.7908 fetch_fsmap finished waiting for FSMap version 76 2024-04-04T06:14:13.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.625+0000 7fd5d683d740 10 client.7908 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-04T06:14:13.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.625+0000 7fd5d683d740 20 client.7908 populate_metadata read hostname 'smithi159' 2024-04-04T06:14:13.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.625+0000 7fd5d683d740 1 --2- 172.21.15.159:0/2824174922 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55ba16646d70 0x55ba166471c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.625+0000 7fd5d683d740 4 client.7908 mds_command: new command op to 7809 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:14:13.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.625+0000 7fd5d683d740 1 -- 172.21.15.159:0/2824174922 --> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba1657b7c0 con 0x55ba16646d70 2024-04-04T06:14:13.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.625+0000 7fd5d1200640 1 --2- 172.21.15.159:0/2824174922 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55ba16646d70 0x55ba166471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.626+0000 7fd5d1200640 1 --2- 172.21.15.159:0/2824174922 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55ba16646d70 0x55ba166471c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab7828640d6995fb server_cookie=dfd4e2a0a3855ba0 in_seq=0 out_seq=0 2024-04-04T06:14:13.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.626+0000 7fd5b2000640 10 client.7908 ms_handle_connect on v2:172.21.15.159:6833/3555825575 2024-04-04T06:14:13.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.626+0000 7fd5b2000640 1 -- 172.21.15.159:0/2824174922 <== mds.0 v2:172.21.15.159:6833/3555825575 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba1657b7c0 con 0x55ba16646d70 2024-04-04T06:14:13.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.626+0000 7fd5b2000640 10 client.7908 handle_command_reply: tid=0 2024-04-04T06:14:13.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.627+0000 7fd5d683d740 1 client.7908 shutdown 2024-04-04T06:14:13.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.627+0000 7fd5d683d740 1 -- 172.21.15.159:0/2824174922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba166516d0 msgr2=0x55ba16633e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:13.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.627+0000 7fd5d683d740 1 --2- 172.21.15.159:0/2824174922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba166516d0 0x55ba16633e30 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8089f10 tx=0x7fd5b8059fe0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.627+0000 7fd5d683d740 1 -- 172.21.15.159:0/2824174922 shutdown_connections 2024-04-04T06:14:13.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.627+0000 7fd5d683d740 1 --2- 172.21.15.159:0/2824174922 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55ba16646d70 0x55ba166471c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.627+0000 7fd5d683d740 1 --2- 172.21.15.159:0/2824174922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba166358d0 0x55ba16637870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.627+0000 7fd5d683d740 1 --2- 172.21.15.159:0/2824174922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba166516d0 0x55ba16633e30 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.627+0000 7fd5d683d740 1 --2- 172.21.15.159:0/2824174922 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba16609de0 0x55ba1665fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.627+0000 7fd5b2000640 0 client.7908 ms_handle_reset on v2:172.21.15.159:6833/3555825575 2024-04-04T06:14:13.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.627+0000 7fd5d683d740 1 -- 172.21.15.159:0/2824174922 >> 172.21.15.159:0/2824174922 conn(0x55ba16603fc0 msgr2=0x55ba16650140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:13.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.627+0000 7fd5d683d740 1 -- 172.21.15.159:0/2824174922 shutdown_connections 2024-04-04T06:14:13.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.627+0000 7fd5d683d740 1 -- 172.21.15.159:0/2824174922 wait complete. 2024-04-04T06:14:13.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.627+0000 7fd5d683d740 20 client.7908 trim_cache size 0 max 16384 2024-04-04T06:14:13.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.646+0000 7fd5d683d740 1 Processor -- start 2024-04-04T06:14:13.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.646+0000 7fd5d683d740 1 -- start start 2024-04-04T06:14:13.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.647+0000 7fd5d683d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba16609de0 0x55ba16630d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.647+0000 7fd5d683d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba166358d0 0x55ba166312c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.647+0000 7fd5d683d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba166516d0 0x55ba1664f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.647+0000 7fd5d683d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc200d50 con 0x55ba166516d0 2024-04-04T06:14:13.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.647+0000 7fd5d683d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc1ffda0 con 0x55ba16609de0 2024-04-04T06:14:13.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.647+0000 7fd5d683d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd5cc0b6740 con 0x55ba166358d0 2024-04-04T06:14:13.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.648+0000 7fd5d1c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba166516d0 0x55ba1664f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.648+0000 7fd5d1c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba166516d0 0x55ba1664f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33886/0 (socket says 172.21.15.159:33886) 2024-04-04T06:14:13.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.648+0000 7fd5d1c00640 1 -- 172.21.15.159:0/3352832487 learned_addr learned my addr 172.21.15.159:0/3352832487 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:13.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.648+0000 7fd5d1200640 1 --2- 172.21.15.159:0/3352832487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba16609de0 0x55ba16630d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.648+0000 7fd5d1c00640 1 -- 172.21.15.159:0/3352832487 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba166358d0 msgr2=0x55ba166312c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:13.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.648+0000 7fd5cbe00640 1 --2- 172.21.15.159:0/3352832487 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba166358d0 0x55ba166312c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.648+0000 7fd5d1c00640 1 --2- 172.21.15.159:0/3352832487 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba166358d0 0x55ba166312c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.648+0000 7fd5d1c00640 1 -- 172.21.15.159:0/3352832487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba16609de0 msgr2=0x55ba16630d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:13.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.648+0000 7fd5d1c00640 1 --2- 172.21.15.159:0/3352832487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba16609de0 0x55ba16630d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.648+0000 7fd5d1c00640 1 -- 172.21.15.159:0/3352832487 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba16637be0 con 0x55ba166516d0 2024-04-04T06:14:13.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.648+0000 7fd5d1200640 1 --2- 172.21.15.159:0/3352832487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba16609de0 0x55ba16630d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:13.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.648+0000 7fd5d1c00640 1 --2- 172.21.15.159:0/3352832487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba166516d0 0x55ba1664f130 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009f40 tx=0x7fd5c402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b9298591cb6f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:13.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.649+0000 7fd5bcc00640 1 -- 172.21.15.159:0/3352832487 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4009510 con 0x55ba166516d0 2024-04-04T06:14:13.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.649+0000 7fd5bcc00640 1 -- 172.21.15.159:0/3352832487 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c402ca70 con 0x55ba166516d0 2024-04-04T06:14:13.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.649+0000 7fd5bcc00640 1 -- 172.21.15.159:0/3352832487 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c402cc20 con 0x55ba166516d0 2024-04-04T06:14:13.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.649+0000 7fd5d683d740 1 -- 172.21.15.159:0/3352832487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba166516d0 msgr2=0x55ba1664f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:13.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.649+0000 7fd5d683d740 1 --2- 172.21.15.159:0/3352832487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba166516d0 0x55ba1664f130 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009f40 tx=0x7fd5c402e000 comp rx=0 tx=0).stop 2024-04-04T06:14:13.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.649+0000 7fd5d683d740 1 -- 172.21.15.159:0/3352832487 shutdown_connections 2024-04-04T06:14:13.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.649+0000 7fd5d683d740 1 --2- 172.21.15.159:0/3352832487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba166516d0 0x55ba1664f130 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.649+0000 7fd5d683d740 1 --2- 172.21.15.159:0/3352832487 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba166358d0 0x55ba166312c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.649+0000 7fd5d683d740 1 --2- 172.21.15.159:0/3352832487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba16609de0 0x55ba16630d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.649+0000 7fd5d683d740 1 -- 172.21.15.159:0/3352832487 >> 172.21.15.159:0/3352832487 conn(0x55ba16603fc0 msgr2=0x55ba16650140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:13.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.649+0000 7fd5d683d740 1 -- 172.21.15.159:0/3352832487 shutdown_connections 2024-04-04T06:14:13.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.649+0000 7fd5d683d740 1 -- 172.21.15.159:0/3352832487 wait complete. 2024-04-04T06:14:13.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.650+0000 7fd5d683d740 1 -- start start 2024-04-04T06:14:13.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.650+0000 7fd5d683d740 1 Processor -- start 2024-04-04T06:14:13.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.651+0000 7fd5d683d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba16609de0 0x55ba166733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.651+0000 7fd5d683d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba166358d0 0x55ba16630d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.651+0000 7fd5d683d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba166516d0 0x55ba166312c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.651+0000 7fd5d683d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ba166081e0 con 0x55ba166516d0 2024-04-04T06:14:13.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.651+0000 7fd5d683d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc1f4ed0 con 0x55ba166358d0 2024-04-04T06:14:13.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.651+0000 7fd5d683d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ba16633de0 con 0x55ba16609de0 2024-04-04T06:14:13.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.651+0000 7fd5d683d740 10 client.0 _ll_register_callbacks cb 0x7fd5b8059570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:14:13.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.652+0000 7fd5d1c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba166516d0 0x55ba166312c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.652+0000 7fd5d1c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba166516d0 0x55ba166312c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33898/0 (socket says 172.21.15.159:33898) 2024-04-04T06:14:13.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.652+0000 7fd5d1c00640 1 -- 172.21.15.159:0/1204237926 learned_addr learned my addr 172.21.15.159:0/1204237926 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:13.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.652+0000 7fd5cbe00640 1 --2- 172.21.15.159:0/1204237926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba166358d0 0x55ba16630d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.652+0000 7fd5d1200640 1 --2- 172.21.15.159:0/1204237926 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba16609de0 0x55ba166733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.652+0000 7fd5d1c00640 1 -- 172.21.15.159:0/1204237926 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba16609de0 msgr2=0x55ba166733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:13.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.652+0000 7fd5d1c00640 1 --2- 172.21.15.159:0/1204237926 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba16609de0 0x55ba166733f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.652+0000 7fd5d1c00640 1 -- 172.21.15.159:0/1204237926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba166358d0 msgr2=0x55ba16630d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:13.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.652+0000 7fd5d1c00640 1 --2- 172.21.15.159:0/1204237926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba166358d0 0x55ba16630d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.652+0000 7fd5d1c00640 1 -- 172.21.15.159:0/1204237926 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba16606630 con 0x55ba166516d0 2024-04-04T06:14:13.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.652+0000 7fd5d1200640 1 --2- 172.21.15.159:0/1204237926 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba16609de0 0x55ba166733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:13.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.652+0000 7fd5d1c00640 1 --2- 172.21.15.159:0/1204237926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba166516d0 0x55ba166312c0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009c90 tx=0x7fd5c4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1e15f87bf249dd server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:13.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.652+0000 7fd5b2000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:14:13.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.653+0000 7fd5b2000640 1 -- 172.21.15.159:0/1204237926 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c40041d0 con 0x55ba166516d0 2024-04-04T06:14:13.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.653+0000 7fd5b2000640 1 -- 172.21.15.159:0/1204237926 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7fd5c4004370 con 0x55ba166516d0 2024-04-04T06:14:13.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.653+0000 7fd5d683d740 1 -- 172.21.15.159:0/1204237926 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5c402ca70 con 0x55ba166516d0 2024-04-04T06:14:13.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.653+0000 7fd5b2000640 1 -- 172.21.15.159:0/1204237926 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fd5c4058050 con 0x55ba166516d0 2024-04-04T06:14:13.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.653+0000 7fd5d683d740 10 client.7914 fetch_fsmap learned FSMap version 76 2024-04-04T06:14:13.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.653+0000 7fd5d683d740 1 -- 172.21.15.159:0/1204237926 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55ba16609680 con 0x55ba166516d0 2024-04-04T06:14:13.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.653+0000 7fd5b2000640 1 -- 172.21.15.159:0/1204237926 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c402ca70 con 0x55ba166516d0 2024-04-04T06:14:13.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.653+0000 7fd5b2000640 1 -- 172.21.15.159:0/1204237926 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 76) ==== 3494+0+0 (secure 0 0 0) 0x7fd5c405b020 con 0x55ba166516d0 2024-04-04T06:14:13.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.653+0000 7fd5d683d740 10 client.7914 fetch_fsmap finished waiting for FSMap version 76 2024-04-04T06:14:13.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.653+0000 7fd5d683d740 10 client.7914 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-04T06:14:13.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.653+0000 7fd5d683d740 20 client.7914 populate_metadata read hostname 'smithi159' 2024-04-04T06:14:13.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.653+0000 7fd5d683d740 1 --2- 172.21.15.159:0/1204237926 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55ba16669860 0x55ba166c9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.653+0000 7fd5d683d740 4 client.7914 mds_command: new command op to 7809 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-04T06:14:13.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.653+0000 7fd5d683d740 1 -- 172.21.15.159:0/1204237926 --> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x55ba16575b10 con 0x55ba16669860 2024-04-04T06:14:13.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.654+0000 7fd5d1200640 1 --2- 172.21.15.159:0/1204237926 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55ba16669860 0x55ba166c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.654+0000 7fd5d1200640 1 --2- 172.21.15.159:0/1204237926 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55ba16669860 0x55ba166c9240 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37e84bcb87efc7f8 server_cookie=f186814a250e0745 in_seq=0 out_seq=0 2024-04-04T06:14:13.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.654+0000 7fd5b2000640 10 client.7914 ms_handle_connect on v2:172.21.15.159:6833/3555825575 2024-04-04T06:14:13.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.655+0000 7fd5b2000640 1 -- 172.21.15.159:0/1204237926 <== mds.0 v2:172.21.15.159:6833/3555825575 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ba16575b10 con 0x55ba16669860 2024-04-04T06:14:13.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.655+0000 7fd5b2000640 10 client.7914 handle_command_reply: tid=0 2024-04-04T06:14:13.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.655+0000 7fd5d683d740 1 client.7914 shutdown 2024-04-04T06:14:13.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.655+0000 7fd5d683d740 1 -- 172.21.15.159:0/1204237926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba166516d0 msgr2=0x55ba166312c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:13.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.655+0000 7fd5d683d740 1 --2- 172.21.15.159:0/1204237926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba166516d0 0x55ba166312c0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009c90 tx=0x7fd5c4004ab0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.655+0000 7fd5d683d740 1 -- 172.21.15.159:0/1204237926 shutdown_connections 2024-04-04T06:14:13.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.655+0000 7fd5d683d740 1 --2- 172.21.15.159:0/1204237926 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55ba16669860 0x55ba166c9240 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.655+0000 7fd5d683d740 1 --2- 172.21.15.159:0/1204237926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba166516d0 0x55ba166312c0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.655+0000 7fd5d683d740 1 --2- 172.21.15.159:0/1204237926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba166358d0 0x55ba16630d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.655+0000 7fd5b2000640 0 client.7914 ms_handle_reset on v2:172.21.15.159:6833/3555825575 2024-04-04T06:14:13.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.655+0000 7fd5d683d740 1 --2- 172.21.15.159:0/1204237926 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ba16609de0 0x55ba166733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.655+0000 7fd5d683d740 1 -- 172.21.15.159:0/1204237926 >> 172.21.15.159:0/1204237926 conn(0x55ba16603fc0 msgr2=0x55ba1662e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:13.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.655+0000 7fd5d683d740 1 -- 172.21.15.159:0/1204237926 shutdown_connections 2024-04-04T06:14:13.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.655+0000 7fd5d683d740 1 -- 172.21.15.159:0/1204237926 wait complete. 2024-04-04T06:14:13.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.655+0000 7fd5d683d740 20 client.7914 trim_cache size 0 max 16384 2024-04-04T06:14:13.655 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:14:13.655 INFO:teuthology.orchestra.run.smithi159.stdout: "return_code": 0, 2024-04-04T06:14:13.655 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_tag": "0f011e90-4f0b-4b8a-a640-42ff9588111b", 2024-04-04T06:14:13.655 INFO:teuthology.orchestra.run.smithi159.stdout: "mode": "asynchronous" 2024-04-04T06:14:13.655 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:14:13.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.656+0000 7fd5d3c00640 1 -- 172.21.15.159:0/3550569755 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd5ac06c8f0 msgr2=0x7fd5ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:13.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.656+0000 7fd5d3c00640 1 --2- 172.21.15.159:0/3550569755 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd5ac06c8f0 0x7fd5ac06edb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0006fd0 tx=0x7fd5c0008040 comp rx=0 tx=0).stop 2024-04-04T06:14:13.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.656+0000 7fd5d3c00640 1 -- 172.21.15.159:0/3550569755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5cc0b3270 msgr2=0x7fd5cc1f5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:13.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.656+0000 7fd5d3c00640 1 --2- 172.21.15.159:0/3550569755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5cc0b3270 0x7fd5cc1f5390 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800c430 tx=0x7fd5b800c8f0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.656+0000 7fd5d3c00640 1 -- 172.21.15.159:0/3550569755 shutdown_connections 2024-04-04T06:14:13.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.656+0000 7fd5d3c00640 1 --2- 172.21.15.159:0/3550569755 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd5cc161f10 0x7fd5cc0cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.656+0000 7fd5d3c00640 1 --2- 172.21.15.159:0/3550569755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5cc0b3270 0x7fd5cc1f5390 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.656+0000 7fd5d3c00640 1 --2- 172.21.15.159:0/3550569755 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd5ac06c8f0 0x7fd5ac06edb0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.656+0000 7fd5d3c00640 1 --2- 172.21.15.159:0/3550569755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5cc06e110 0x7fd5cc1f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.656+0000 7fd5d3c00640 1 -- 172.21.15.159:0/3550569755 >> 172.21.15.159:0/3550569755 conn(0x7fd5cc08bfb0 msgr2=0x7fd5cc0d2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:13.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.656+0000 7fd5d3c00640 1 -- 172.21.15.159:0/3550569755 shutdown_connections 2024-04-04T06:14:13.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.656+0000 7fd5d3c00640 1 -- 172.21.15.159:0/3550569755 wait complete. 2024-04-04T06:14:13.657 INFO:tasks.ceph.mds.a.smithi159.stderr:2024-04-04T06:14:13.657+0000 7fce99800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x560faf987180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-04T06:14:13.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:14:13.669 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-04T06:14:13.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.879+0000 7fb97be00640 1 Processor -- start 2024-04-04T06:14:13.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.879+0000 7fb97be00640 1 -- start start 2024-04-04T06:14:13.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.880+0000 7fb97be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb97c13f190 0x7fb97c0cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.881+0000 7fb97be00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb97c06dd30 0x7fb97c0cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.881+0000 7fb97be00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb97c0ce890 con 0x7fb97c0cc880 2024-04-04T06:14:13.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.881+0000 7fb97be00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fb97c0cd230 con 0x7fb97c13f190 2024-04-04T06:14:13.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.881+0000 7fb97be00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb97c0ceb50 con 0x7fb97c06dd30 2024-04-04T06:14:13.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.881+0000 7fb97aa00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb97c13f190 0x7fb97c0cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.881+0000 7fb97aa00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb97c13f190 0x7fb97c0cf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48444/0 (socket says 172.21.15.159:48444) 2024-04-04T06:14:13.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.881+0000 7fb97aa00640 1 -- 172.21.15.159:0/2465332087 learned_addr learned my addr 172.21.15.159:0/2465332087 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:13.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.881+0000 7fb97b400640 1 --2- 172.21.15.159:0/2465332087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb97c06dd30 0x7fb97c0cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.881+0000 7fb97aa00640 1 -- 172.21.15.159:0/2465332087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb97c06dd30 msgr2=0x7fb97c0cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:13.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.881+0000 7fb97aa00640 1 --2- 172.21.15.159:0/2465332087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb97c06dd30 0x7fb97c0cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.881+0000 7fb97aa00640 1 -- 172.21.15.159:0/2465332087 >> v1:172.21.15.159:6789/0 conn(0x7fb97c0cc880 legacy=0x7fb97c06d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:13.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.881+0000 7fb97aa00640 1 -- 172.21.15.159:0/2465332087 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97c0cee30 con 0x7fb97c13f190 2024-04-04T06:14:13.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.881+0000 7fb97b400640 1 --2- 172.21.15.159:0/2465332087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb97c06dd30 0x7fb97c0cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:13.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.883+0000 7fb97aa00640 1 --2- 172.21.15.159:0/2465332087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb97c13f190 0x7fb97c0cf630 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009870 tx=0x7fb96c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f244d7dd43db8e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:13.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.883+0000 7fb979600640 1 -- 172.21.15.159:0/2465332087 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c034070 con 0x7fb97c13f190 2024-04-04T06:14:13.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.883+0000 7fb979600640 1 -- 172.21.15.159:0/2465332087 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96c030840 con 0x7fb97c13f190 2024-04-04T06:14:13.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.883+0000 7fb979600640 1 -- 172.21.15.159:0/2465332087 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c030c10 con 0x7fb97c13f190 2024-04-04T06:14:13.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.884+0000 7fb97be00640 1 -- 172.21.15.159:0/2465332087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb97c13f190 msgr2=0x7fb97c0cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:13.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.884+0000 7fb97be00640 1 --2- 172.21.15.159:0/2465332087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb97c13f190 0x7fb97c0cf630 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009870 tx=0x7fb96c02b980 comp rx=0 tx=0).stop 2024-04-04T06:14:13.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.884+0000 7fb97be00640 1 -- 172.21.15.159:0/2465332087 shutdown_connections 2024-04-04T06:14:13.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.884+0000 7fb97be00640 1 --2- 172.21.15.159:0/2465332087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb97c06dd30 0x7fb97c0cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.884+0000 7fb97be00640 1 --2- 172.21.15.159:0/2465332087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb97c13f190 0x7fb97c0cf630 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.884+0000 7fb97be00640 1 -- 172.21.15.159:0/2465332087 >> 172.21.15.159:0/2465332087 conn(0x7fb97c0b1ba0 msgr2=0x7fb97c0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:13.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.884+0000 7fb97be00640 1 -- 172.21.15.159:0/2465332087 shutdown_connections 2024-04-04T06:14:13.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.884+0000 7fb97be00640 1 -- 172.21.15.159:0/2465332087 wait complete. 2024-04-04T06:14:13.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.886+0000 7fb97be00640 1 Processor -- start 2024-04-04T06:14:13.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.886+0000 7fb97be00640 1 -- start start 2024-04-04T06:14:13.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.887+0000 7fb97be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb97c06dd30 0x7fb97c138ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.887+0000 7fb97be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb97c0cc880 0x7fb97c139fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.887+0000 7fb97be00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb97c13f190 0x7fb97c13a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.887+0000 7fb97be00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c13f570 con 0x7fb97c0cc880 2024-04-04T06:14:13.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.887+0000 7fb97be00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c160c70 con 0x7fb97c06dd30 2024-04-04T06:14:13.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.887+0000 7fb97be00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb97c0cfb70 con 0x7fb97c13f190 2024-04-04T06:14:13.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.887+0000 7fb97a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb97c0cc880 0x7fb97c139fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.887+0000 7fb97a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb97c0cc880 0x7fb97c139fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48452/0 (socket says 172.21.15.159:48452) 2024-04-04T06:14:13.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.887+0000 7fb97a000640 1 -- 172.21.15.159:0/1773184847 learned_addr learned my addr 172.21.15.159:0/1773184847 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:13.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.888+0000 7fb97aa00640 1 --2- 172.21.15.159:0/1773184847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb97c06dd30 0x7fb97c138ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.888+0000 7fb97b400640 1 --2- 172.21.15.159:0/1773184847 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb97c13f190 0x7fb97c13a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.888+0000 7fb97a000640 1 -- 172.21.15.159:0/1773184847 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb97c13f190 msgr2=0x7fb97c13a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:13.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.888+0000 7fb97a000640 1 --2- 172.21.15.159:0/1773184847 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb97c13f190 0x7fb97c13a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.888+0000 7fb97a000640 1 -- 172.21.15.159:0/1773184847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb97c06dd30 msgr2=0x7fb97c138ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:13.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.888+0000 7fb97a000640 1 --2- 172.21.15.159:0/1773184847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb97c06dd30 0x7fb97c138ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.888+0000 7fb97a000640 1 -- 172.21.15.159:0/1773184847 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97c1f2d50 con 0x7fb97c0cc880 2024-04-04T06:14:13.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.888+0000 7fb97aa00640 1 --2- 172.21.15.159:0/1773184847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb97c06dd30 0x7fb97c138ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:13.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.888+0000 7fb97a000640 1 --2- 172.21.15.159:0/1773184847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb97c0cc880 0x7fb97c139fa0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fb96400c7a0 tx=0x7fb96400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:13.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.888+0000 7fb973400640 1 -- 172.21.15.159:0/1773184847 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964003b90 con 0x7fb97c0cc880 2024-04-04T06:14:13.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.888+0000 7fb973400640 1 -- 172.21.15.159:0/1773184847 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb964003d50 con 0x7fb97c0cc880 2024-04-04T06:14:13.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.888+0000 7fb973400640 1 -- 172.21.15.159:0/1773184847 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96400aa90 con 0x7fb97c0cc880 2024-04-04T06:14:13.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.888+0000 7fb97be00640 1 -- 172.21.15.159:0/1773184847 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97c1f1f90 con 0x7fb97c0cc880 2024-04-04T06:14:13.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.889+0000 7fb97be00640 1 -- 172.21.15.159:0/1773184847 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97c1f2550 con 0x7fb97c0cc880 2024-04-04T06:14:13.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.891+0000 7fb973400640 1 -- 172.21.15.159:0/1773184847 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb96400ac30 con 0x7fb97c0cc880 2024-04-04T06:14:13.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.892+0000 7fb973400640 1 --2- 172.21.15.159:0/1773184847 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb95806c8f0 0x7fb95806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.892+0000 7fb973400640 1 -- 172.21.15.159:0/1773184847 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7fb96408bc30 con 0x7fb97c0cc880 2024-04-04T06:14:13.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.892+0000 7fb97aa00640 1 --2- 172.21.15.159:0/1773184847 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb95806c8f0 0x7fb95806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.894+0000 7fb97aa00640 1 --2- 172.21.15.159:0/1773184847 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb95806c8f0 0x7fb95806edb0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb96c036000 tx=0x7fb96c03b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:13.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.894+0000 7fb984722740 1 Processor -- start 2024-04-04T06:14:13.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.895+0000 7fb984722740 1 -- start start 2024-04-04T06:14:13.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.895+0000 7fb984722740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cb468591c0 0x55cb46856e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.895+0000 7fb984722740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468595a0 0x55cb468107f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.895+0000 7fb984722740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cb46838b00 0x55cb46810d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.895+0000 7fb984722740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cb467766d0 con 0x55cb468595a0 2024-04-04T06:14:13.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.895+0000 7fb984722740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cb4671e770 con 0x55cb46838b00 2024-04-04T06:14:13.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.895+0000 7fb984722740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55cb46772680 con 0x55cb468591c0 2024-04-04T06:14:13.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.896+0000 7fb97a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468595a0 0x55cb468107f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.896+0000 7fb97a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468595a0 0x55cb468107f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48458/0 (socket says 172.21.15.159:48458) 2024-04-04T06:14:13.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.896+0000 7fb97a000640 1 -- 172.21.15.159:0/1266717482 learned_addr learned my addr 172.21.15.159:0/1266717482 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:13.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.896+0000 7fb97aa00640 1 --2- 172.21.15.159:0/1266717482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cb468591c0 0x55cb46856e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.896+0000 7fb97b400640 1 --2- 172.21.15.159:0/1266717482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cb46838b00 0x55cb46810d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.896+0000 7fb97a000640 1 -- 172.21.15.159:0/1266717482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cb468591c0 msgr2=0x55cb46856e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:13.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.896+0000 7fb97a000640 1 --2- 172.21.15.159:0/1266717482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cb468591c0 0x55cb46856e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.896+0000 7fb97a000640 1 -- 172.21.15.159:0/1266717482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cb46838b00 msgr2=0x55cb46810d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:13.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.896+0000 7fb97a000640 1 --2- 172.21.15.159:0/1266717482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cb46838b00 0x55cb46810d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.896+0000 7fb97a000640 1 -- 172.21.15.159:0/1266717482 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb465547d0 con 0x55cb468595a0 2024-04-04T06:14:13.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.896+0000 7fb97aa00640 1 --2- 172.21.15.159:0/1266717482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cb468591c0 0x55cb46856e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:13.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.896+0000 7fb97b400640 1 --2- 172.21.15.159:0/1266717482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cb46838b00 0x55cb46810d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:13.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.896+0000 7fb97a000640 1 --2- 172.21.15.159:0/1266717482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468595a0 0x55cb468107f0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fb964061e80 tx=0x7fb9640591f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbeb409bd413e2c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:13.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.897+0000 7fb96b400640 1 -- 172.21.15.159:0/1266717482 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9640600c0 con 0x55cb468595a0 2024-04-04T06:14:13.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.897+0000 7fb96b400640 1 -- 172.21.15.159:0/1266717482 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96405db80 con 0x55cb468595a0 2024-04-04T06:14:13.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.897+0000 7fb96b400640 1 -- 172.21.15.159:0/1266717482 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9640623b0 con 0x55cb468595a0 2024-04-04T06:14:13.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.897+0000 7fb984722740 1 -- 172.21.15.159:0/1266717482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468595a0 msgr2=0x55cb468107f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:13.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.897+0000 7fb984722740 1 --2- 172.21.15.159:0/1266717482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468595a0 0x55cb468107f0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fb964061e80 tx=0x7fb9640591f0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.897+0000 7fb984722740 1 -- 172.21.15.159:0/1266717482 shutdown_connections 2024-04-04T06:14:13.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.897+0000 7fb984722740 1 --2- 172.21.15.159:0/1266717482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cb46838b00 0x55cb46810d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.897+0000 7fb984722740 1 --2- 172.21.15.159:0/1266717482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468595a0 0x55cb468107f0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.897+0000 7fb984722740 1 --2- 172.21.15.159:0/1266717482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cb468591c0 0x55cb46856e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.897+0000 7fb984722740 1 -- 172.21.15.159:0/1266717482 >> 172.21.15.159:0/1266717482 conn(0x55cb4680d950 msgr2=0x55cb468564d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:13.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.898+0000 7fb984722740 1 -- 172.21.15.159:0/1266717482 shutdown_connections 2024-04-04T06:14:13.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.898+0000 7fb984722740 1 -- 172.21.15.159:0/1266717482 wait complete. 2024-04-04T06:14:13.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.899+0000 7fb984722740 1 -- start start 2024-04-04T06:14:13.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.899+0000 7fb984722740 1 Processor -- start 2024-04-04T06:14:13.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.901+0000 7fb984722740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cb468591c0 0x55cb46833f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.901+0000 7fb984722740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cb468595a0 0x55cb4685be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.901+0000 7fb984722740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468344c0 0x55cb4683d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.901+0000 7fb984722740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cb4680bf80 con 0x55cb468344c0 2024-04-04T06:14:13.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.901+0000 7fb984722740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55cb46731aa0 con 0x55cb468591c0 2024-04-04T06:14:13.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.901+0000 7fb984722740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55cb4680fd80 con 0x55cb468595a0 2024-04-04T06:14:13.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.901+0000 7fb984722740 10 client.0 _ll_register_callbacks cb 0x55cb46731490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:14:13.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.901+0000 7fb97b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468344c0 0x55cb4683d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.901+0000 7fb97b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468344c0 0x55cb4683d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48474/0 (socket says 172.21.15.159:48474) 2024-04-04T06:14:13.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.901+0000 7fb97b400640 1 -- 172.21.15.159:0/17390051 learned_addr learned my addr 172.21.15.159:0/17390051 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:13.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.901+0000 7fb97aa00640 1 --2- 172.21.15.159:0/17390051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cb468591c0 0x55cb46833f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.901+0000 7fb97a000640 1 --2- 172.21.15.159:0/17390051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cb468595a0 0x55cb4685be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.901+0000 7fb97b400640 1 -- 172.21.15.159:0/17390051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cb468595a0 msgr2=0x55cb4685be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:13.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.901+0000 7fb97b400640 1 --2- 172.21.15.159:0/17390051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cb468595a0 0x55cb4685be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.901+0000 7fb97b400640 1 -- 172.21.15.159:0/17390051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cb468591c0 msgr2=0x55cb46833f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:13.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.901+0000 7fb97b400640 1 --2- 172.21.15.159:0/17390051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cb468591c0 0x55cb46833f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.901+0000 7fb97b400640 1 -- 172.21.15.159:0/17390051 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb4648a3f0 con 0x55cb468344c0 2024-04-04T06:14:13.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.902+0000 7fb97b400640 1 --2- 172.21.15.159:0/17390051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468344c0 0x55cb4683d010 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fb97400e2e0 tx=0x7fb97400e7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6899563cb70914 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:13.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.902+0000 7fb963e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:14:13.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.902+0000 7fb963e00640 1 -- 172.21.15.159:0/17390051 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974012840 con 0x55cb468344c0 2024-04-04T06:14:13.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.902+0000 7fb963e00640 1 -- 172.21.15.159:0/17390051 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7fb9740129e0 con 0x55cb468344c0 2024-04-04T06:14:13.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.902+0000 7fb984722740 1 -- 172.21.15.159:0/17390051 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb96405db80 con 0x55cb468344c0 2024-04-04T06:14:13.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.902+0000 7fb963e00640 1 -- 172.21.15.159:0/17390051 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fb974022050 con 0x55cb468344c0 2024-04-04T06:14:13.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.902+0000 7fb984722740 10 client.7929 fetch_fsmap learned FSMap version 76 2024-04-04T06:14:13.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.902+0000 7fb984722740 1 -- 172.21.15.159:0/17390051 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55cb4644a0f0 con 0x55cb468344c0 2024-04-04T06:14:13.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.903+0000 7fb963e00640 1 -- 172.21.15.159:0/17390051 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96405db80 con 0x55cb468344c0 2024-04-04T06:14:13.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.903+0000 7fb963e00640 1 -- 172.21.15.159:0/17390051 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 76) ==== 3494+0+0 (secure 0 0 0) 0x7fb974025020 con 0x55cb468344c0 2024-04-04T06:14:13.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.903+0000 7fb984722740 10 client.7929 fetch_fsmap finished waiting for FSMap version 76 2024-04-04T06:14:13.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.903+0000 7fb984722740 10 client.7929 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-04T06:14:13.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.903+0000 7fb984722740 20 client.7929 populate_metadata read hostname 'smithi159' 2024-04-04T06:14:13.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.903+0000 7fb984722740 1 --2- 172.21.15.159:0/17390051 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55cb4683e7c0 0x55cb4683ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.903+0000 7fb984722740 4 client.7929 mds_command: new command op to 7809 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:14:13.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.903+0000 7fb984722740 1 -- 172.21.15.159:0/17390051 --> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb46783c10 con 0x55cb4683e7c0 2024-04-04T06:14:13.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.903+0000 7fb97aa00640 1 --2- 172.21.15.159:0/17390051 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55cb4683e7c0 0x55cb4683ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.904+0000 7fb97aa00640 1 --2- 172.21.15.159:0/17390051 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55cb4683e7c0 0x55cb4683ebf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b38534ef0ea41e1 server_cookie=1038fbd2d9561649 in_seq=0 out_seq=0 2024-04-04T06:14:13.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.904+0000 7fb963e00640 10 client.7929 ms_handle_connect on v2:172.21.15.159:6833/3555825575 2024-04-04T06:14:13.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.904+0000 7fb963e00640 1 -- 172.21.15.159:0/17390051 <== mds.0 v2:172.21.15.159:6833/3555825575 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb46783c10 con 0x55cb4683e7c0 2024-04-04T06:14:13.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.904+0000 7fb963e00640 10 client.7929 handle_command_reply: tid=0 2024-04-04T06:14:13.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.905+0000 7fb984722740 1 client.7929 shutdown 2024-04-04T06:14:13.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.905+0000 7fb984722740 1 -- 172.21.15.159:0/17390051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468344c0 msgr2=0x55cb4683d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:13.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.905+0000 7fb984722740 1 --2- 172.21.15.159:0/17390051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468344c0 0x55cb4683d010 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fb97400e2e0 tx=0x7fb97400e7a0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.905+0000 7fb984722740 1 -- 172.21.15.159:0/17390051 shutdown_connections 2024-04-04T06:14:13.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.905+0000 7fb984722740 1 --2- 172.21.15.159:0/17390051 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55cb4683e7c0 0x55cb4683ebf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.905+0000 7fb984722740 1 --2- 172.21.15.159:0/17390051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468344c0 0x55cb4683d010 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.905+0000 7fb984722740 1 --2- 172.21.15.159:0/17390051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cb468595a0 0x55cb4685be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.905+0000 7fb984722740 1 --2- 172.21.15.159:0/17390051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cb468591c0 0x55cb46833f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.905+0000 7fb984722740 1 -- 172.21.15.159:0/17390051 >> 172.21.15.159:0/17390051 conn(0x55cb46838b00 msgr2=0x55cb468562f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:13.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.905+0000 7fb963e00640 0 client.7929 ms_handle_reset on v2:172.21.15.159:6833/3555825575 2024-04-04T06:14:13.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.905+0000 7fb984722740 1 -- 172.21.15.159:0/17390051 shutdown_connections 2024-04-04T06:14:13.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.905+0000 7fb984722740 1 -- 172.21.15.159:0/17390051 wait complete. 2024-04-04T06:14:13.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.905+0000 7fb984722740 20 client.7929 trim_cache size 0 max 16384 2024-04-04T06:14:13.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.924+0000 7fb984722740 1 Processor -- start 2024-04-04T06:14:13.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.924+0000 7fb984722740 1 -- start start 2024-04-04T06:14:13.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.925+0000 7fb984722740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cb468344c0 0x55cb4680fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.925+0000 7fb984722740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468591c0 0x55cb4683da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.925+0000 7fb984722740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cb468595a0 0x55cb4683dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.925+0000 7fb984722740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1fc7f0 con 0x55cb468591c0 2024-04-04T06:14:13.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.925+0000 7fb984722740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c0b4560 con 0x55cb468344c0 2024-04-04T06:14:13.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.925+0000 7fb984722740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb97c1e6410 con 0x55cb468595a0 2024-04-04T06:14:13.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.925+0000 7fb97a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468591c0 0x55cb4683da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.926+0000 7fb97a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468591c0 0x55cb4683da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48478/0 (socket says 172.21.15.159:48478) 2024-04-04T06:14:13.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.926+0000 7fb97a000640 1 -- 172.21.15.159:0/2718658189 learned_addr learned my addr 172.21.15.159:0/2718658189 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:13.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.926+0000 7fb97aa00640 1 --2- 172.21.15.159:0/2718658189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cb468344c0 0x55cb4680fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.926+0000 7fb97a000640 1 -- 172.21.15.159:0/2718658189 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cb468595a0 msgr2=0x55cb4683dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:13.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.926+0000 7fb97a000640 1 --2- 172.21.15.159:0/2718658189 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cb468595a0 0x55cb4683dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.926+0000 7fb97a000640 1 -- 172.21.15.159:0/2718658189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cb468344c0 msgr2=0x55cb4680fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:13.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.926+0000 7fb97a000640 1 --2- 172.21.15.159:0/2718658189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cb468344c0 0x55cb4680fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.926+0000 7fb97a000640 1 -- 172.21.15.159:0/2718658189 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb46810350 con 0x55cb468591c0 2024-04-04T06:14:13.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.926+0000 7fb97a000640 1 --2- 172.21.15.159:0/2718658189 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468591c0 0x55cb4683da80 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb964059710 tx=0x7fb964060050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8923baf36d00fd1a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:13.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.926+0000 7fb96aa00640 1 -- 172.21.15.159:0/2718658189 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9640623b0 con 0x55cb468591c0 2024-04-04T06:14:13.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.927+0000 7fb96aa00640 1 -- 172.21.15.159:0/2718658189 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96405db80 con 0x55cb468591c0 2024-04-04T06:14:13.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.927+0000 7fb96aa00640 1 -- 172.21.15.159:0/2718658189 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964010dc0 con 0x55cb468591c0 2024-04-04T06:14:13.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.927+0000 7fb984722740 1 -- 172.21.15.159:0/2718658189 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468591c0 msgr2=0x55cb4683da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:13.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.927+0000 7fb984722740 1 --2- 172.21.15.159:0/2718658189 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468591c0 0x55cb4683da80 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb964059710 tx=0x7fb964060050 comp rx=0 tx=0).stop 2024-04-04T06:14:13.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.927+0000 7fb984722740 1 -- 172.21.15.159:0/2718658189 shutdown_connections 2024-04-04T06:14:13.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.927+0000 7fb984722740 1 --2- 172.21.15.159:0/2718658189 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cb468595a0 0x55cb4683dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.927+0000 7fb984722740 1 --2- 172.21.15.159:0/2718658189 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468591c0 0x55cb4683da80 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.927+0000 7fb984722740 1 --2- 172.21.15.159:0/2718658189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cb468344c0 0x55cb4680fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.927+0000 7fb984722740 1 -- 172.21.15.159:0/2718658189 >> 172.21.15.159:0/2718658189 conn(0x55cb46838b00 msgr2=0x55cb4687eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:13.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.927+0000 7fb984722740 1 -- 172.21.15.159:0/2718658189 shutdown_connections 2024-04-04T06:14:13.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.927+0000 7fb984722740 1 -- 172.21.15.159:0/2718658189 wait complete. 2024-04-04T06:14:13.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.928+0000 7fb984722740 1 -- start start 2024-04-04T06:14:13.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.928+0000 7fb984722740 1 Processor -- start 2024-04-04T06:14:13.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.929+0000 7fb984722740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cb468344c0 0x55cb468d21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.929+0000 7fb984722740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468591c0 0x55cb468d8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.929+0000 7fb984722740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cb468595a0 0x55cb46832ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.929+0000 7fb984722740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cb468553c0 con 0x55cb468591c0 2024-04-04T06:14:13.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.929+0000 7fb984722740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1e6a60 con 0x55cb468595a0 2024-04-04T06:14:13.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.929+0000 7fb984722740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55cb4683bd20 con 0x55cb468344c0 2024-04-04T06:14:13.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.929+0000 7fb984722740 10 client.0 _ll_register_callbacks cb 0x7fb97400cc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:14:13.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.929+0000 7fb97a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468591c0 0x55cb468d8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.930+0000 7fb97a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468591c0 0x55cb468d8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48482/0 (socket says 172.21.15.159:48482) 2024-04-04T06:14:13.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.930+0000 7fb97a000640 1 -- 172.21.15.159:0/3373040918 learned_addr learned my addr 172.21.15.159:0/3373040918 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:13.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.930+0000 7fb97a000640 1 -- 172.21.15.159:0/3373040918 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cb468344c0 msgr2=0x55cb468d21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:13.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.930+0000 7fb97a000640 1 --2- 172.21.15.159:0/3373040918 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cb468344c0 0x55cb468d21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.930+0000 7fb97b400640 1 --2- 172.21.15.159:0/3373040918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cb468595a0 0x55cb46832ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.930+0000 7fb97a000640 1 -- 172.21.15.159:0/3373040918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cb468595a0 msgr2=0x55cb46832ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:13.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.930+0000 7fb97a000640 1 --2- 172.21.15.159:0/3373040918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cb468595a0 0x55cb46832ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.930+0000 7fb97a000640 1 -- 172.21.15.159:0/3373040918 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb468d8690 con 0x55cb468591c0 2024-04-04T06:14:13.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.930+0000 7fb97a000640 1 --2- 172.21.15.159:0/3373040918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468591c0 0x55cb468d8150 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb964001330 tx=0x7fb964000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26160a7fc9bef1b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:13.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.930+0000 7fb963e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:14:13.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.930+0000 7fb963e00640 1 -- 172.21.15.159:0/3373040918 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964076060 con 0x55cb468591c0 2024-04-04T06:14:13.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.930+0000 7fb963e00640 1 -- 172.21.15.159:0/3373040918 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7fb964010dc0 con 0x55cb468591c0 2024-04-04T06:14:13.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.930+0000 7fb984722740 1 -- 172.21.15.159:0/3373040918 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb96405db80 con 0x55cb468591c0 2024-04-04T06:14:13.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.931+0000 7fb963e00640 1 -- 172.21.15.159:0/3373040918 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fb9640600c0 con 0x55cb468591c0 2024-04-04T06:14:13.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.931+0000 7fb984722740 10 client.7935 fetch_fsmap learned FSMap version 76 2024-04-04T06:14:13.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.931+0000 7fb984722740 1 -- 172.21.15.159:0/3373040918 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55cb468d8960 con 0x55cb468591c0 2024-04-04T06:14:13.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.931+0000 7fb963e00640 1 -- 172.21.15.159:0/3373040918 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96405db80 con 0x55cb468591c0 2024-04-04T06:14:13.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.931+0000 7fb963e00640 1 -- 172.21.15.159:0/3373040918 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 76) ==== 3494+0+0 (secure 0 0 0) 0x7fb964055e40 con 0x55cb468591c0 2024-04-04T06:14:13.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.931+0000 7fb984722740 10 client.7935 fetch_fsmap finished waiting for FSMap version 76 2024-04-04T06:14:13.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.931+0000 7fb984722740 10 client.7935 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-04T06:14:13.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.931+0000 7fb984722740 20 client.7935 populate_metadata read hostname 'smithi159' 2024-04-04T06:14:13.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.931+0000 7fb984722740 1 --2- 172.21.15.159:0/3373040918 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55cb46854a90 0x55cb46854ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:13.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.931+0000 7fb984722740 4 client.7935 mds_command: new command op to 7809 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-04T06:14:13.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.931+0000 7fb984722740 1 -- 172.21.15.159:0/3373040918 --> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cb4677df60 con 0x55cb46854a90 2024-04-04T06:14:13.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.931+0000 7fb97aa00640 1 --2- 172.21.15.159:0/3373040918 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55cb46854a90 0x55cb46854ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:13.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.932+0000 7fb97aa00640 1 --2- 172.21.15.159:0/3373040918 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55cb46854a90 0x55cb46854ef0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5a797c5651f092d server_cookie=e9f2147f007aabd1 in_seq=0 out_seq=0 2024-04-04T06:14:13.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.932+0000 7fb963e00640 10 client.7935 ms_handle_connect on v2:172.21.15.159:6833/3555825575 2024-04-04T06:14:13.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.932+0000 7fb963e00640 1 -- 172.21.15.159:0/3373040918 <== mds.0 v2:172.21.15.159:6833/3555825575 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x55cb4677df60 con 0x55cb46854a90 2024-04-04T06:14:13.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.932+0000 7fb963e00640 10 client.7935 handle_command_reply: tid=0 2024-04-04T06:14:13.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.932+0000 7fb984722740 1 client.7935 shutdown 2024-04-04T06:14:13.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.932+0000 7fb984722740 1 -- 172.21.15.159:0/3373040918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468591c0 msgr2=0x55cb468d8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:13.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.932+0000 7fb984722740 1 --2- 172.21.15.159:0/3373040918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468591c0 0x55cb468d8150 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb964001330 tx=0x7fb964000f40 comp rx=0 tx=0).stop 2024-04-04T06:14:13.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.932+0000 7fb984722740 1 -- 172.21.15.159:0/3373040918 shutdown_connections 2024-04-04T06:14:13.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.932+0000 7fb984722740 1 --2- 172.21.15.159:0/3373040918 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55cb46854a90 0x55cb46854ef0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.932+0000 7fb984722740 1 --2- 172.21.15.159:0/3373040918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55cb468595a0 0x55cb46832ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.932+0000 7fb984722740 1 --2- 172.21.15.159:0/3373040918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb468591c0 0x55cb468d8150 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.932+0000 7fb984722740 1 --2- 172.21.15.159:0/3373040918 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55cb468344c0 0x55cb468d21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.932+0000 7fb963e00640 0 client.7935 ms_handle_reset on v2:172.21.15.159:6833/3555825575 2024-04-04T06:14:13.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.932+0000 7fb984722740 1 -- 172.21.15.159:0/3373040918 >> 172.21.15.159:0/3373040918 conn(0x55cb46838b00 msgr2=0x55cb46811300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:13.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.933+0000 7fb984722740 1 -- 172.21.15.159:0/3373040918 shutdown_connections 2024-04-04T06:14:13.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.933+0000 7fb984722740 1 -- 172.21.15.159:0/3373040918 wait complete. 2024-04-04T06:14:13.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.933+0000 7fb984722740 20 client.7935 trim_cache size 0 max 16384 2024-04-04T06:14:13.932 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:14:13.932 INFO:teuthology.orchestra.run.smithi159.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-04T06:14:13.932 INFO:teuthology.orchestra.run.smithi159.stdout: "scrubs": { 2024-04-04T06:14:13.932 INFO:teuthology.orchestra.run.smithi159.stdout: "0f011e90-4f0b-4b8a-a640-42ff9588111b": { 2024-04-04T06:14:13.932 INFO:teuthology.orchestra.run.smithi159.stdout: "path": "/dir", 2024-04-04T06:14:13.932 INFO:teuthology.orchestra.run.smithi159.stdout: "tag": "0f011e90-4f0b-4b8a-a640-42ff9588111b", 2024-04-04T06:14:13.932 INFO:teuthology.orchestra.run.smithi159.stdout: "options": "recursive,repair" 2024-04-04T06:14:13.933 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:14:13.933 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:14:13.933 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:14:13.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.933+0000 7fb97be00640 1 -- 172.21.15.159:0/1773184847 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb95806c8f0 msgr2=0x7fb95806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:13.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.933+0000 7fb97be00640 1 --2- 172.21.15.159:0/1773184847 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb95806c8f0 0x7fb95806edb0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb96c036000 tx=0x7fb96c03b040 comp rx=0 tx=0).stop 2024-04-04T06:14:13.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.933+0000 7fb97be00640 1 -- 172.21.15.159:0/1773184847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb97c0cc880 msgr2=0x7fb97c139fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:13.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.933+0000 7fb97be00640 1 --2- 172.21.15.159:0/1773184847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb97c0cc880 0x7fb97c139fa0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fb96400c7a0 tx=0x7fb96400cc60 comp rx=0 tx=0).stop 2024-04-04T06:14:13.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.933+0000 7fb97be00640 1 -- 172.21.15.159:0/1773184847 shutdown_connections 2024-04-04T06:14:13.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.933+0000 7fb97be00640 1 --2- 172.21.15.159:0/1773184847 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb97c13f190 0x7fb97c13a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.933+0000 7fb97be00640 1 --2- 172.21.15.159:0/1773184847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb97c0cc880 0x7fb97c139fa0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.933+0000 7fb97be00640 1 --2- 172.21.15.159:0/1773184847 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb95806c8f0 0x7fb95806edb0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.933+0000 7fb97be00640 1 --2- 172.21.15.159:0/1773184847 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb97c06dd30 0x7fb97c138ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:13.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.933+0000 7fb97be00640 1 -- 172.21.15.159:0/1773184847 >> 172.21.15.159:0/1773184847 conn(0x7fb97c0b1ba0 msgr2=0x7fb97c06d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:13.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.934+0000 7fb97be00640 1 -- 172.21.15.159:0/1773184847 shutdown_connections 2024-04-04T06:14:13.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:13.934+0000 7fb97be00640 1 -- 172.21.15.159:0/1773184847 wait complete. 2024-04-04T06:14:13.944 INFO:tasks.cephfs.filesystem:scrub status for tag:0f011e90-4f0b-4b8a-a640-42ff9588111b - {'path': '/dir', 'tag': '0f011e90-4f0b-4b8a-a640-42ff9588111b', 'options': 'recursive,repair'} 2024-04-04T06:14:43.075 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:14:43.086 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:14:43.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:14:43.945 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-04T06:14:44.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.168+0000 7f2fa5c00640 1 Processor -- start 2024-04-04T06:14:44.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.168+0000 7f2fa5c00640 1 -- start start 2024-04-04T06:14:44.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.169+0000 7f2fa5c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa00c7810 0x7f2fa00b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.169+0000 7f2fa5c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fa00d01a0 con 0x7f2fa015e7b0 2024-04-04T06:14:44.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.170+0000 7f2fa5c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fa00d0480 con 0x7f2fa00d1d70 2024-04-04T06:14:44.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.170+0000 7f2fa5c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa00d1240 con 0x7f2fa00c7810 2024-04-04T06:14:44.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.170+0000 7f2f9ea00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f2fa015e7b0 0x7f2fa00c7100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:36450/0 (socket says 172.21.15.159:36450) 2024-04-04T06:14:44.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.170+0000 7f2f9ea00640 1 -- 172.21.15.159:0/2021305288 learned_addr learned my addr 172.21.15.159:0/2021305288 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:44.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.170+0000 7f2f9e000640 1 --2- 172.21.15.159:0/2021305288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa00c7810 0x7f2fa00b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.170+0000 7f2f9d600640 1 -- 172.21.15.159:0/2021305288 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3568186737 0 0) 0x7f2fa00d01a0 con 0x7f2fa015e7b0 2024-04-04T06:14:44.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.170+0000 7f2f9d600640 1 -- 172.21.15.159:0/2021305288 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f80003520 con 0x7f2fa015e7b0 2024-04-04T06:14:44.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.171+0000 7f2f9d600640 1 -- 172.21.15.159:0/2021305288 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4208611200 0 0) 0x7f2f80003520 con 0x7f2fa015e7b0 2024-04-04T06:14:44.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.171+0000 7f2f9d600640 1 -- 172.21.15.159:0/2021305288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa00c7810 msgr2=0x7f2fa00b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.171+0000 7f2f9d600640 1 --2- 172.21.15.159:0/2021305288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa00c7810 0x7f2fa00b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.171+0000 7f2f9d600640 1 -- 172.21.15.159:0/2021305288 >> v1:172.21.15.178:6789/0 conn(0x7f2fa00d1d70 legacy=0x7f2fa006ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.171+0000 7f2f9d600640 1 -- 172.21.15.159:0/2021305288 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa00d0760 con 0x7f2fa015e7b0 2024-04-04T06:14:44.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.171+0000 7f2f9d600640 1 -- 172.21.15.159:0/2021305288 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f2f90001670 con 0x7f2fa015e7b0 2024-04-04T06:14:44.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.171+0000 7f2f9d600640 1 -- 172.21.15.159:0/2021305288 >> v1:172.21.15.159:6789/0 conn(0x7f2fa015e7b0 legacy=0x7f2fa00c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.171+0000 7f2f9d600640 1 --2- 172.21.15.159:0/2021305288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f800039e0 0x7f2f80003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.171+0000 7f2f9d600640 1 --2- 172.21.15.159:0/2021305288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f80004800 0x7f2f80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.171+0000 7f2f9d600640 1 --2- 172.21.15.159:0/2021305288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa00c7810 0x7f2f800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.171+0000 7f2f9d600640 1 -- 172.21.15.159:0/2021305288 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa00d1240 con 0x7f2f80004800 2024-04-04T06:14:44.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.171+0000 7f2f9d600640 1 -- 172.21.15.159:0/2021305288 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8000ba70 con 0x7f2f800039e0 2024-04-04T06:14:44.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.171+0000 7f2f9d600640 1 -- 172.21.15.159:0/2021305288 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2f8000bc70 con 0x7f2fa00c7810 2024-04-04T06:14:44.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.171+0000 7f2f9e000640 1 --2- 172.21.15.159:0/2021305288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f80004800 0x7f2f80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.171+0000 7f2f9ea00640 1 --2- 172.21.15.159:0/2021305288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f800039e0 0x7f2f80003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.172+0000 7f2f9e000640 1 -- 172.21.15.159:0/2021305288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa00c7810 msgr2=0x7f2f800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.172+0000 7f2f9e000640 1 --2- 172.21.15.159:0/2021305288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa00c7810 0x7f2f800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.172+0000 7f2f9e000640 1 -- 172.21.15.159:0/2021305288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f800039e0 msgr2=0x7f2f80003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.172+0000 7f2f9e000640 1 --2- 172.21.15.159:0/2021305288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f800039e0 0x7f2f80003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.172+0000 7f2f9e000640 1 -- 172.21.15.159:0/2021305288 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f80003520 con 0x7f2f80004800 2024-04-04T06:14:44.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.172+0000 7f2f9ea00640 1 --2- 172.21.15.159:0/2021305288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f800039e0 0x7f2f80003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:44.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.173+0000 7f2f9e000640 1 --2- 172.21.15.159:0/2021305288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f80004800 0x7f2f80004c80 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009f90 tx=0x7f2f80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662dfc8a91879097 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:44.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.173+0000 7f2f9d600640 1 -- 172.21.15.159:0/2021305288 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f88039070 con 0x7f2f80004800 2024-04-04T06:14:44.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.173+0000 7f2f9d600640 1 -- 172.21.15.159:0/2021305288 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f8802cbf0 con 0x7f2f80004800 2024-04-04T06:14:44.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.173+0000 7f2f9d600640 1 -- 172.21.15.159:0/2021305288 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f88009c10 con 0x7f2f80004800 2024-04-04T06:14:44.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.173+0000 7f2fa5c00640 1 -- 172.21.15.159:0/2021305288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f80004800 msgr2=0x7f2f80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.173+0000 7f2fa5c00640 1 --2- 172.21.15.159:0/2021305288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f80004800 0x7f2f80004c80 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009f90 tx=0x7f2f80005830 comp rx=0 tx=0).stop 2024-04-04T06:14:44.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.174+0000 7f2f9f400640 1 -- 172.21.15.159:0/2021305288 reap_dead start 2024-04-04T06:14:44.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.174+0000 7f2fa5c00640 1 -- 172.21.15.159:0/2021305288 shutdown_connections 2024-04-04T06:14:44.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.174+0000 7f2fa5c00640 1 -- 172.21.15.159:0/2021305288 >> 172.21.15.159:0/2021305288 conn(0x7f2fa00b3e90 msgr2=0x7f2fa00b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:44.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.174+0000 7f2fa5c00640 1 -- 172.21.15.159:0/2021305288 shutdown_connections 2024-04-04T06:14:44.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.174+0000 7f2fa5c00640 1 -- 172.21.15.159:0/2021305288 wait complete. 2024-04-04T06:14:44.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.175+0000 7f2fa5c00640 1 Processor -- start 2024-04-04T06:14:44.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.176+0000 7f2fa5c00640 1 -- start start 2024-04-04T06:14:44.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.176+0000 7f2fa5c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f80004800 0x7f2fa0153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.176+0000 7f2fa5c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fa00cac30 0x7f2fa00cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.177+0000 7f2fa5c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa00cb900 0x7f2fa00cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.177+0000 7f2fa5c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa00cfe30 con 0x7f2fa00cac30 2024-04-04T06:14:44.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.177+0000 7f2fa5c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa00d0d20 con 0x7f2f80004800 2024-04-04T06:14:44.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.177+0000 7f2fa5c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa00b4980 con 0x7f2fa00cb900 2024-04-04T06:14:44.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.177+0000 7f2f9e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fa00cac30 0x7f2fa00cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.177+0000 7f2f9e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fa00cac30 0x7f2fa00cb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36928/0 (socket says 172.21.15.159:36928) 2024-04-04T06:14:44.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.177+0000 7f2f9e000640 1 -- 172.21.15.159:0/1895269832 learned_addr learned my addr 172.21.15.159:0/1895269832 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:44.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.177+0000 7f2f9ea00640 1 --2- 172.21.15.159:0/1895269832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f80004800 0x7f2fa0153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.177+0000 7f2f9e000640 1 -- 172.21.15.159:0/1895269832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa00cb900 msgr2=0x7f2fa00cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.177+0000 7f2f9f400640 1 --2- 172.21.15.159:0/1895269832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa00cb900 0x7f2fa00cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.177+0000 7f2f9e000640 1 --2- 172.21.15.159:0/1895269832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa00cb900 0x7f2fa00cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.177+0000 7f2f9e000640 1 -- 172.21.15.159:0/1895269832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f80004800 msgr2=0x7f2fa0153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.177+0000 7f2f9e000640 1 --2- 172.21.15.159:0/1895269832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f80004800 0x7f2fa0153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.177+0000 7f2f9e000640 1 -- 172.21.15.159:0/1895269832 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa015fe80 con 0x7f2fa00cac30 2024-04-04T06:14:44.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.177+0000 7f2f9ea00640 1 --2- 172.21.15.159:0/1895269832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f80004800 0x7f2fa0153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:44.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.178+0000 7f2f9e000640 1 --2- 172.21.15.159:0/1895269832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fa00cac30 0x7f2fa00cb090 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009f90 tx=0x7f2f88009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:44.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.178+0000 7f2f96a00640 1 -- 172.21.15.159:0/1895269832 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f88039050 con 0x7f2fa00cac30 2024-04-04T06:14:44.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.178+0000 7f2f96a00640 1 -- 172.21.15.159:0/1895269832 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f88045070 con 0x7f2fa00cac30 2024-04-04T06:14:44.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.178+0000 7f2f96a00640 1 -- 172.21.15.159:0/1895269832 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f88009c10 con 0x7f2fa00cac30 2024-04-04T06:14:44.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.178+0000 7f2fa5c00640 1 -- 172.21.15.159:0/1895269832 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa015ede0 con 0x7f2fa00cac30 2024-04-04T06:14:44.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.178+0000 7f2fa5c00640 1 -- 172.21.15.159:0/1895269832 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa015f3a0 con 0x7f2fa00cac30 2024-04-04T06:14:44.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.180+0000 7f2f96a00640 1 -- 172.21.15.159:0/1895269832 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f88005d10 con 0x7f2fa00cac30 2024-04-04T06:14:44.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.182+0000 7f2f96a00640 1 --2- 172.21.15.159:0/1895269832 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2f7c06c8f0 0x7f2f7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.182+0000 7f2f96a00640 1 -- 172.21.15.159:0/1895269832 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f2f880ae4c0 con 0x7f2fa00cac30 2024-04-04T06:14:44.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.182+0000 7f2fa887d740 1 Processor -- start 2024-04-04T06:14:44.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.182+0000 7f2f9ea00640 1 --2- 172.21.15.159:0/1895269832 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2f7c06c8f0 0x7f2f7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.183+0000 7f2fa887d740 1 -- start start 2024-04-04T06:14:44.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.183+0000 7f2fa887d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ed3b70580 0x562ed3b43520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.183+0000 7f2fa887d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ed3b70960 0x562ed3b73310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.184+0000 7f2fa887d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b43a60 0x562ed3b73850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.184+0000 7f2fa887d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562ed3aae6d0 con 0x562ed3b43a60 2024-04-04T06:14:44.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.184+0000 7f2fa887d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562ed3a56770 con 0x562ed3b70580 2024-04-04T06:14:44.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.184+0000 7f2fa887d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562ed3aaa680 con 0x562ed3b70960 2024-04-04T06:14:44.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.184+0000 7f2f9ea00640 1 --2- 172.21.15.159:0/1895269832 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2f7c06c8f0 0x7f2f7c06edb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2f90001eb0 tx=0x7f2f90009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:44.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.184+0000 7f2f9f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b43a60 0x562ed3b73850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.184+0000 7f2f9f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b43a60 0x562ed3b73850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36942/0 (socket says 172.21.15.159:36942) 2024-04-04T06:14:44.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.184+0000 7f2f9f400640 1 -- 172.21.15.159:0/2855291094 learned_addr learned my addr 172.21.15.159:0/2855291094 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:44.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.184+0000 7f2f9ea00640 1 --2- 172.21.15.159:0/2855291094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ed3b70580 0x562ed3b43520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.184+0000 7f2f9e000640 1 --2- 172.21.15.159:0/2855291094 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ed3b70960 0x562ed3b73310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.184+0000 7f2f9f400640 1 -- 172.21.15.159:0/2855291094 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ed3b70960 msgr2=0x562ed3b73310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.184+0000 7f2f9f400640 1 --2- 172.21.15.159:0/2855291094 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ed3b70960 0x562ed3b73310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.184+0000 7f2f9f400640 1 -- 172.21.15.159:0/2855291094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ed3b70580 msgr2=0x562ed3b43520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.184+0000 7f2f9f400640 1 --2- 172.21.15.159:0/2855291094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ed3b70580 0x562ed3b43520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.184+0000 7f2f9f400640 1 -- 172.21.15.159:0/2855291094 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ed388c7d0 con 0x562ed3b43a60 2024-04-04T06:14:44.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.185+0000 7f2f9f400640 1 --2- 172.21.15.159:0/2855291094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b43a60 0x562ed3b73850 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f2fa00d2190 tx=0x7f2f980019d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95da5fee1038ea0d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:44.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.185+0000 7f2f8f400640 1 -- 172.21.15.159:0/2855291094 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f980074e0 con 0x562ed3b43a60 2024-04-04T06:14:44.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.185+0000 7f2f8f400640 1 -- 172.21.15.159:0/2855291094 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f98004020 con 0x562ed3b43a60 2024-04-04T06:14:44.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.185+0000 7f2f8f400640 1 -- 172.21.15.159:0/2855291094 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f980043d0 con 0x562ed3b43a60 2024-04-04T06:14:44.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.185+0000 7f2fa887d740 1 -- 172.21.15.159:0/2855291094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b43a60 msgr2=0x562ed3b73850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.185+0000 7f2fa887d740 1 --2- 172.21.15.159:0/2855291094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b43a60 0x562ed3b73850 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f2fa00d2190 tx=0x7f2f980019d0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.186+0000 7f2fa887d740 1 -- 172.21.15.159:0/2855291094 shutdown_connections 2024-04-04T06:14:44.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.186+0000 7f2fa887d740 1 --2- 172.21.15.159:0/2855291094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b43a60 0x562ed3b73850 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.186+0000 7f2fa887d740 1 --2- 172.21.15.159:0/2855291094 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ed3b70960 0x562ed3b73310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.186+0000 7f2fa887d740 1 --2- 172.21.15.159:0/2855291094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ed3b70580 0x562ed3b43520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.186+0000 7f2fa887d740 1 -- 172.21.15.159:0/2855291094 >> 172.21.15.159:0/2855291094 conn(0x562ed3b45790 msgr2=0x562ed3b8e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:44.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.186+0000 7f2fa887d740 1 -- 172.21.15.159:0/2855291094 shutdown_connections 2024-04-04T06:14:44.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.186+0000 7f2fa887d740 1 -- 172.21.15.159:0/2855291094 wait complete. 2024-04-04T06:14:44.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.186+0000 7f2fa887d740 1 -- start start 2024-04-04T06:14:44.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.187+0000 7f2fa887d740 1 Processor -- start 2024-04-04T06:14:44.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.188+0000 7f2fa887d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b70580 0x562ed3b8c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.188+0000 7f2fa887d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ed3b70960 0x562ed3b68b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.188+0000 7f2fa887d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ed3b8bf30 0x562ed3b71e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.188+0000 7f2fa887d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562ed3b47e30 con 0x562ed3b70580 2024-04-04T06:14:44.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.188+0000 7f2fa887d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562ed3a69aa0 con 0x562ed3b70960 2024-04-04T06:14:44.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.188+0000 7f2fa887d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562ed3b4a670 con 0x562ed3b8bf30 2024-04-04T06:14:44.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.188+0000 7f2fa887d740 10 client.0 _ll_register_callbacks cb 0x562ed3a69490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:14:44.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.188+0000 7f2f9ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b70580 0x562ed3b8c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.188+0000 7f2f9ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b70580 0x562ed3b8c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36950/0 (socket says 172.21.15.159:36950) 2024-04-04T06:14:44.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.188+0000 7f2f9ea00640 1 -- 172.21.15.159:0/638899529 learned_addr learned my addr 172.21.15.159:0/638899529 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:44.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.188+0000 7f2f9f400640 1 --2- 172.21.15.159:0/638899529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ed3b8bf30 0x562ed3b71e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.188+0000 7f2f9e000640 1 --2- 172.21.15.159:0/638899529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ed3b70960 0x562ed3b68b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.188+0000 7f2f9ea00640 1 -- 172.21.15.159:0/638899529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ed3b8bf30 msgr2=0x562ed3b71e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.188+0000 7f2f9ea00640 1 --2- 172.21.15.159:0/638899529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ed3b8bf30 0x562ed3b71e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.188+0000 7f2f9ea00640 1 -- 172.21.15.159:0/638899529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ed3b70960 msgr2=0x562ed3b68b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.188+0000 7f2f9ea00640 1 --2- 172.21.15.159:0/638899529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ed3b70960 0x562ed3b68b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.189+0000 7f2f9ea00640 1 -- 172.21.15.159:0/638899529 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ed37c23f0 con 0x562ed3b70580 2024-04-04T06:14:44.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.189+0000 7f2f9e000640 1 --2- 172.21.15.159:0/638899529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ed3b70960 0x562ed3b68b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:44.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.189+0000 7f2f9ea00640 1 --2- 172.21.15.159:0/638899529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b70580 0x562ed3b8c8d0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f2f900076e0 tx=0x7f2f90001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dfb93a5c748bb85 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:44.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.189+0000 7f2f87e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:14:44.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.189+0000 7f2f87e00640 1 -- 172.21.15.159:0/638899529 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90027060 con 0x562ed3b70580 2024-04-04T06:14:44.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.189+0000 7f2f87e00640 1 -- 172.21.15.159:0/638899529 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f2f9000ee00 con 0x562ed3b70580 2024-04-04T06:14:44.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.189+0000 7f2fa887d740 1 -- 172.21.15.159:0/638899529 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f98004020 con 0x562ed3b70580 2024-04-04T06:14:44.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.189+0000 7f2f87e00640 1 -- 172.21.15.159:0/638899529 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f2f90015110 con 0x562ed3b70580 2024-04-04T06:14:44.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.189+0000 7f2fa887d740 10 client.7947 fetch_fsmap learned FSMap version 76 2024-04-04T06:14:44.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.189+0000 7f2fa887d740 1 -- 172.21.15.159:0/638899529 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x562ed37820f0 con 0x562ed3b70580 2024-04-04T06:14:44.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.190+0000 7f2f87e00640 1 -- 172.21.15.159:0/638899529 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f98004020 con 0x562ed3b70580 2024-04-04T06:14:44.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.190+0000 7f2f87e00640 1 -- 172.21.15.159:0/638899529 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 76) ==== 3494+0+0 (secure 0 0 0) 0x7f2f90015a20 con 0x562ed3b70580 2024-04-04T06:14:44.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.190+0000 7f2fa887d740 10 client.7947 fetch_fsmap finished waiting for FSMap version 76 2024-04-04T06:14:44.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.190+0000 7f2fa887d740 10 client.7947 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-04T06:14:44.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.190+0000 7f2fa887d740 20 client.7947 populate_metadata read hostname 'smithi159' 2024-04-04T06:14:44.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.190+0000 7f2fa887d740 1 --2- 172.21.15.159:0/638899529 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x562ed3b8ead0 0x562ed3ba1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.190+0000 7f2fa887d740 4 client.7947 mds_command: new command op to 7809 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:14:44.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.190+0000 7f2fa887d740 1 -- 172.21.15.159:0/638899529 --> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ed3abbc10 con 0x562ed3b8ead0 2024-04-04T06:14:44.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.190+0000 7f2f9e000640 1 --2- 172.21.15.159:0/638899529 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x562ed3b8ead0 0x562ed3ba1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.191+0000 7f2f9e000640 1 --2- 172.21.15.159:0/638899529 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x562ed3b8ead0 0x562ed3ba1bb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aaae833ca3a2873 server_cookie=bf792db2db6ca056 in_seq=0 out_seq=0 2024-04-04T06:14:44.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.191+0000 7f2f87e00640 10 client.7947 ms_handle_connect on v2:172.21.15.159:6833/3555825575 2024-04-04T06:14:44.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.191+0000 7f2f87e00640 1 -- 172.21.15.159:0/638899529 <== mds.0 v2:172.21.15.159:6833/3555825575 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ed3abbc10 con 0x562ed3b8ead0 2024-04-04T06:14:44.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.191+0000 7f2f87e00640 10 client.7947 handle_command_reply: tid=0 2024-04-04T06:14:44.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.191+0000 7f2fa887d740 1 client.7947 shutdown 2024-04-04T06:14:44.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.192+0000 7f2fa887d740 1 -- 172.21.15.159:0/638899529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b70580 msgr2=0x562ed3b8c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.192+0000 7f2fa887d740 1 --2- 172.21.15.159:0/638899529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b70580 0x562ed3b8c8d0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f2f900076e0 tx=0x7f2f90001a40 comp rx=0 tx=0).stop 2024-04-04T06:14:44.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.192+0000 7f2fa887d740 1 -- 172.21.15.159:0/638899529 shutdown_connections 2024-04-04T06:14:44.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.192+0000 7f2fa887d740 1 --2- 172.21.15.159:0/638899529 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x562ed3b8ead0 0x562ed3ba1bb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.192+0000 7f2fa887d740 1 --2- 172.21.15.159:0/638899529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ed3b8bf30 0x562ed3b71e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.192+0000 7f2fa887d740 1 --2- 172.21.15.159:0/638899529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ed3b70960 0x562ed3b68b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.192+0000 7f2f87e00640 0 client.7947 ms_handle_reset on v2:172.21.15.159:6833/3555825575 2024-04-04T06:14:44.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.192+0000 7f2fa887d740 1 --2- 172.21.15.159:0/638899529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b70580 0x562ed3b8c8d0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.192+0000 7f2fa887d740 1 -- 172.21.15.159:0/638899529 >> 172.21.15.159:0/638899529 conn(0x562ed3b43a60 msgr2=0x562ed3b47430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:44.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.192+0000 7f2fa887d740 1 -- 172.21.15.159:0/638899529 shutdown_connections 2024-04-04T06:14:44.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.192+0000 7f2fa887d740 1 -- 172.21.15.159:0/638899529 wait complete. 2024-04-04T06:14:44.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.192+0000 7f2fa887d740 20 client.7947 trim_cache size 0 max 16384 2024-04-04T06:14:44.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.211+0000 7f2fa887d740 1 Processor -- start 2024-04-04T06:14:44.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.211+0000 7f2fa887d740 1 -- start start 2024-04-04T06:14:44.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.212+0000 7f2fa887d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ed3b70580 0x562ed3b42a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.212+0000 7f2fa887d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b70960 0x562ed3b717e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.212+0000 7f2fa887d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ed3b8bf30 0x562ed3b71d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.212+0000 7f2fa887d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa0201530 con 0x562ed3b70960 2024-04-04T06:14:44.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.212+0000 7f2fa887d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa0200550 con 0x562ed3b8bf30 2024-04-04T06:14:44.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.212+0000 7f2fa887d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa01ff9b0 con 0x562ed3b70580 2024-04-04T06:14:44.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.212+0000 7f2f9e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b70960 0x562ed3b717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.212+0000 7f2f9e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b70960 0x562ed3b717e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36956/0 (socket says 172.21.15.159:36956) 2024-04-04T06:14:44.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.212+0000 7f2f9e000640 1 -- 172.21.15.159:0/3516355840 learned_addr learned my addr 172.21.15.159:0/3516355840 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:44.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.213+0000 7f2f9e000640 1 -- 172.21.15.159:0/3516355840 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ed3b70580 msgr2=0x562ed3b42a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.213+0000 7f2f9e000640 1 --2- 172.21.15.159:0/3516355840 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ed3b70580 0x562ed3b42a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.213+0000 7f2f9e000640 1 -- 172.21.15.159:0/3516355840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ed3b8bf30 msgr2=0x562ed3b71d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.213+0000 7f2f9e000640 1 --2- 172.21.15.159:0/3516355840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ed3b8bf30 0x562ed3b71d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.213+0000 7f2f9e000640 1 -- 172.21.15.159:0/3516355840 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ed3b8d8b0 con 0x562ed3b70960 2024-04-04T06:14:44.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.213+0000 7f2f9e000640 1 --2- 172.21.15.159:0/3516355840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b70960 0x562ed3b717e0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f2f8807bff0 tx=0x7f2f880ac730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d4c9bd46bfa9f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:44.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.213+0000 7f2f8ea00640 1 -- 172.21.15.159:0/3516355840 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f88080410 con 0x562ed3b70960 2024-04-04T06:14:44.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.213+0000 7f2f8ea00640 1 -- 172.21.15.159:0/3516355840 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f8803a040 con 0x562ed3b70960 2024-04-04T06:14:44.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.213+0000 7f2f8ea00640 1 -- 172.21.15.159:0/3516355840 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f88084c40 con 0x562ed3b70960 2024-04-04T06:14:44.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.214+0000 7f2fa887d740 1 -- 172.21.15.159:0/3516355840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b70960 msgr2=0x562ed3b717e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.214+0000 7f2fa887d740 1 --2- 172.21.15.159:0/3516355840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b70960 0x562ed3b717e0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f2f8807bff0 tx=0x7f2f880ac730 comp rx=0 tx=0).stop 2024-04-04T06:14:44.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.214+0000 7f2fa887d740 1 -- 172.21.15.159:0/3516355840 shutdown_connections 2024-04-04T06:14:44.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.214+0000 7f2fa887d740 1 --2- 172.21.15.159:0/3516355840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ed3b8bf30 0x562ed3b71d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.214+0000 7f2fa887d740 1 --2- 172.21.15.159:0/3516355840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b70960 0x562ed3b717e0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.214+0000 7f2fa887d740 1 --2- 172.21.15.159:0/3516355840 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ed3b70580 0x562ed3b42a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.214+0000 7f2fa887d740 1 -- 172.21.15.159:0/3516355840 >> 172.21.15.159:0/3516355840 conn(0x562ed3b43a60 msgr2=0x562ed3b3eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:44.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.214+0000 7f2fa887d740 1 -- 172.21.15.159:0/3516355840 shutdown_connections 2024-04-04T06:14:44.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.214+0000 7f2fa887d740 1 -- 172.21.15.159:0/3516355840 wait complete. 2024-04-04T06:14:44.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.215+0000 7f2fa887d740 1 -- start start 2024-04-04T06:14:44.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.215+0000 7f2fa887d740 1 Processor -- start 2024-04-04T06:14:44.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.216+0000 7f2fa887d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b70580 0x562ed3c05680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.216+0000 7f2fa887d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ed3b70960 0x562ed3b9c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.216+0000 7f2fa887d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ed3b8bf30 0x562ed3b9ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.216+0000 7f2fa887d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562ed3b91e90 con 0x562ed3b70580 2024-04-04T06:14:44.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.216+0000 7f2fa887d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa01fff00 con 0x562ed3b70960 2024-04-04T06:14:44.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.216+0000 7f2fa887d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562ed3b43ff0 con 0x562ed3b8bf30 2024-04-04T06:14:44.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.216+0000 7f2fa887d740 10 client.0 _ll_register_callbacks cb 0x7f2f90007300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:14:44.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.216+0000 7f2f9ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b70580 0x562ed3c05680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.216+0000 7f2f9ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b70580 0x562ed3c05680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36968/0 (socket says 172.21.15.159:36968) 2024-04-04T06:14:44.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.216+0000 7f2f9ea00640 1 -- 172.21.15.159:0/2226180377 learned_addr learned my addr 172.21.15.159:0/2226180377 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:44.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.216+0000 7f2f9ea00640 1 -- 172.21.15.159:0/2226180377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ed3b8bf30 msgr2=0x562ed3b9ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.216+0000 7f2f9ea00640 1 --2- 172.21.15.159:0/2226180377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ed3b8bf30 0x562ed3b9ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.216+0000 7f2f9ea00640 1 -- 172.21.15.159:0/2226180377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ed3b70960 msgr2=0x562ed3b9c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.216+0000 7f2f9ea00640 1 --2- 172.21.15.159:0/2226180377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ed3b70960 0x562ed3b9c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.216+0000 7f2f9ea00640 1 -- 172.21.15.159:0/2226180377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ed3b70f30 con 0x562ed3b70580 2024-04-04T06:14:44.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.217+0000 7f2f9ea00640 1 --2- 172.21.15.159:0/2226180377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b70580 0x562ed3c05680 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f2f90001d30 tx=0x7f2f90000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef3e0311e3b45ef server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:44.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.217+0000 7f2f87e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:14:44.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.217+0000 7f2f87e00640 1 -- 172.21.15.159:0/2226180377 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f900152b0 con 0x562ed3b70580 2024-04-04T06:14:44.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.217+0000 7f2f87e00640 1 -- 172.21.15.159:0/2226180377 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f2f9001e040 con 0x562ed3b70580 2024-04-04T06:14:44.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.217+0000 7f2fa887d740 1 -- 172.21.15.159:0/2226180377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f8803a040 con 0x562ed3b70580 2024-04-04T06:14:44.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.217+0000 7f2f87e00640 1 -- 172.21.15.159:0/2226180377 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f2f90015a80 con 0x562ed3b70580 2024-04-04T06:14:44.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.217+0000 7f2fa887d740 10 client.7953 fetch_fsmap learned FSMap version 76 2024-04-04T06:14:44.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.217+0000 7f2fa887d740 1 -- 172.21.15.159:0/2226180377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x562ed3b71550 con 0x562ed3b70580 2024-04-04T06:14:44.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.218+0000 7f2f87e00640 1 -- 172.21.15.159:0/2226180377 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f8803a040 con 0x562ed3b70580 2024-04-04T06:14:44.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.218+0000 7f2f87e00640 1 -- 172.21.15.159:0/2226180377 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 76) ==== 3494+0+0 (secure 0 0 0) 0x7f2f90041080 con 0x562ed3b70580 2024-04-04T06:14:44.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.218+0000 7f2fa887d740 10 client.7953 fetch_fsmap finished waiting for FSMap version 76 2024-04-04T06:14:44.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.218+0000 7f2fa887d740 10 client.7953 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-04T06:14:44.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.218+0000 7f2fa887d740 20 client.7953 populate_metadata read hostname 'smithi159' 2024-04-04T06:14:44.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.218+0000 7f2fa887d740 1 --2- 172.21.15.159:0/2226180377 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x562ed3b3f020 0x562ed3c08020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.218+0000 7f2fa887d740 4 client.7953 mds_command: new command op to 7809 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-04T06:14:44.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.218+0000 7f2fa887d740 1 -- 172.21.15.159:0/2226180377 --> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562ed3b8de90 con 0x562ed3b3f020 2024-04-04T06:14:44.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.218+0000 7f2f9e000640 1 --2- 172.21.15.159:0/2226180377 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x562ed3b3f020 0x562ed3c08020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.218+0000 7f2f87e00640 10 client.7953 ms_handle_connect on v2:172.21.15.159:6833/3555825575 2024-04-04T06:14:44.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.218+0000 7f2f9e000640 1 --2- 172.21.15.159:0/2226180377 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x562ed3b3f020 0x562ed3c08020 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=752dcd734f613e54 server_cookie=3d2f12f4464f5a3c in_seq=0 out_seq=0 2024-04-04T06:14:44.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.219+0000 7f2f87e00640 1 -- 172.21.15.159:0/2226180377 <== mds.0 v2:172.21.15.159:6833/3555825575 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562ed3b8de90 con 0x562ed3b3f020 2024-04-04T06:14:44.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.219+0000 7f2f87e00640 10 client.7953 handle_command_reply: tid=0 2024-04-04T06:14:44.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.219+0000 7f2fa887d740 1 client.7953 shutdown 2024-04-04T06:14:44.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.219+0000 7f2fa887d740 1 -- 172.21.15.159:0/2226180377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b70580 msgr2=0x562ed3c05680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.219+0000 7f2fa887d740 1 --2- 172.21.15.159:0/2226180377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b70580 0x562ed3c05680 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f2f90001d30 tx=0x7f2f90000ed0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.219+0000 7f2fa887d740 1 -- 172.21.15.159:0/2226180377 shutdown_connections 2024-04-04T06:14:44.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.219+0000 7f2fa887d740 1 --2- 172.21.15.159:0/2226180377 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x562ed3b3f020 0x562ed3c08020 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.219+0000 7f2fa887d740 1 --2- 172.21.15.159:0/2226180377 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ed3b8bf30 0x562ed3b9ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.219+0000 7f2fa887d740 1 --2- 172.21.15.159:0/2226180377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ed3b70960 0x562ed3b9c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.219+0000 7f2fa887d740 1 --2- 172.21.15.159:0/2226180377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed3b70580 0x562ed3c05680 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.219+0000 7f2f87e00640 0 client.7953 ms_handle_reset on v2:172.21.15.159:6833/3555825575 2024-04-04T06:14:44.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.219+0000 7f2fa887d740 1 -- 172.21.15.159:0/2226180377 >> 172.21.15.159:0/2226180377 conn(0x562ed3b43a60 msgr2=0x562ed3b8c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:44.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.219+0000 7f2fa887d740 1 -- 172.21.15.159:0/2226180377 shutdown_connections 2024-04-04T06:14:44.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.219+0000 7f2fa887d740 1 -- 172.21.15.159:0/2226180377 wait complete. 2024-04-04T06:14:44.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.219+0000 7f2fa887d740 20 client.7953 trim_cache size 0 max 16384 2024-04-04T06:14:44.218 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:14:44.218 INFO:teuthology.orchestra.run.smithi159.stdout: "status": "no active scrubs running", 2024-04-04T06:14:44.218 INFO:teuthology.orchestra.run.smithi159.stdout: "scrubs": {} 2024-04-04T06:14:44.218 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:14:44.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.220+0000 7f2fa5c00640 1 -- 172.21.15.159:0/1895269832 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2f7c06c8f0 msgr2=0x7f2f7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.220+0000 7f2fa5c00640 1 --2- 172.21.15.159:0/1895269832 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2f7c06c8f0 0x7f2f7c06edb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2f90001eb0 tx=0x7f2f90009040 comp rx=0 tx=0).stop 2024-04-04T06:14:44.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.220+0000 7f2fa5c00640 1 -- 172.21.15.159:0/1895269832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fa00cac30 msgr2=0x7f2fa00cb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.220+0000 7f2fa5c00640 1 --2- 172.21.15.159:0/1895269832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fa00cac30 0x7f2fa00cb090 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009f90 tx=0x7f2f88009750 comp rx=0 tx=0).stop 2024-04-04T06:14:44.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.220+0000 7f2fa5c00640 1 -- 172.21.15.159:0/1895269832 shutdown_connections 2024-04-04T06:14:44.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.220+0000 7f2fa5c00640 1 --2- 172.21.15.159:0/1895269832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2fa00cb900 0x7f2fa00cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.220+0000 7f2fa5c00640 1 --2- 172.21.15.159:0/1895269832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fa00cac30 0x7f2fa00cb090 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.220+0000 7f2fa5c00640 1 --2- 172.21.15.159:0/1895269832 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2f7c06c8f0 0x7f2f7c06edb0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.220+0000 7f2fa5c00640 1 --2- 172.21.15.159:0/1895269832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f80004800 0x7f2fa0153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.220+0000 7f2fa5c00640 1 -- 172.21.15.159:0/1895269832 >> 172.21.15.159:0/1895269832 conn(0x7f2fa00b3e90 msgr2=0x7f2fa015b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:44.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.220+0000 7f2fa5c00640 1 -- 172.21.15.159:0/1895269832 shutdown_connections 2024-04-04T06:14:44.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.220+0000 7f2fa5c00640 1 -- 172.21.15.159:0/1895269832 wait complete. 2024-04-04T06:14:44.233 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-04T06:14:44.233 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:14:44.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.449+0000 7f534a600640 1 Processor -- start 2024-04-04T06:14:44.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.449+0000 7f534a600640 1 -- start start 2024-04-04T06:14:44.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.450+0000 7f534a600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f53440d0120 0x7f5344162880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.450+0000 7f534a600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53440d08c0 con 0x7f534415e6e0 2024-04-04T06:14:44.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.450+0000 7f534a600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53440d0ba0 con 0x7f534406b570 2024-04-04T06:14:44.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.451+0000 7f534a600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f53440d0e80 con 0x7f53440d0120 2024-04-04T06:14:44.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.451+0000 7f5349200640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f534415e6e0 0x7f53440b0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:36466/0 (socket says 172.21.15.159:36466) 2024-04-04T06:14:44.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.451+0000 7f5349200640 1 -- 172.21.15.159:0/2583324061 learned_addr learned my addr 172.21.15.159:0/2583324061 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:44.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.451+0000 7f5343e00640 1 --2- 172.21.15.159:0/2583324061 >> v2:172.21.15.178:3300/0 conn(0x7f53440d0120 0x7f5344162880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.451+0000 7f5343400640 1 -- 172.21.15.159:0/2583324061 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2455826082 0 0) 0x7f53440d08c0 con 0x7f534415e6e0 2024-04-04T06:14:44.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.451+0000 7f5343400640 1 -- 172.21.15.159:0/2583324061 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f532c003520 con 0x7f534415e6e0 2024-04-04T06:14:44.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.451+0000 7f5343400640 1 -- 172.21.15.159:0/2583324061 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 466027317 0 0) 0x7f532c003520 con 0x7f534415e6e0 2024-04-04T06:14:44.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.451+0000 7f5343400640 1 -- 172.21.15.159:0/2583324061 >> v2:172.21.15.178:3300/0 conn(0x7f53440d0120 msgr2=0x7f5344162880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.451+0000 7f5343400640 1 --2- 172.21.15.159:0/2583324061 >> v2:172.21.15.178:3300/0 conn(0x7f53440d0120 0x7f5344162880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.451+0000 7f5343400640 1 -- 172.21.15.159:0/2583324061 >> v1:172.21.15.178:6789/0 conn(0x7f534406b570 legacy=0x7f534415eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.451+0000 7f5343400640 1 -- 172.21.15.159:0/2583324061 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5344162080 con 0x7f534415e6e0 2024-04-04T06:14:44.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.452+0000 7f5343400640 1 -- 172.21.15.159:0/2583324061 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f5338001670 con 0x7f534415e6e0 2024-04-04T06:14:44.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.452+0000 7f5343400640 1 -- 172.21.15.159:0/2583324061 >> v1:172.21.15.159:6789/0 conn(0x7f534415e6e0 legacy=0x7f53440b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.452+0000 7f5343400640 1 --2- 172.21.15.159:0/2583324061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f532c0039b0 0x7f532c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.452+0000 7f5343400640 1 --2- 172.21.15.159:0/2583324061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f532c004800 0x7f532c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.452+0000 7f5343400640 1 --2- 172.21.15.159:0/2583324061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f532c0051c0 0x7f532c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.452+0000 7f5343400640 1 -- 172.21.15.159:0/2583324061 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f53440d0e80 con 0x7f532c0039b0 2024-04-04T06:14:44.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.452+0000 7f5343400640 1 -- 172.21.15.159:0/2583324061 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f532c00c060 con 0x7f532c004800 2024-04-04T06:14:44.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.452+0000 7f5343400640 1 -- 172.21.15.159:0/2583324061 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f532c00c260 con 0x7f532c0051c0 2024-04-04T06:14:44.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.452+0000 7f5349200640 1 --2- 172.21.15.159:0/2583324061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f532c0039b0 0x7f532c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.452+0000 7f5343e00640 1 --2- 172.21.15.159:0/2583324061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f532c004800 0x7f532c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.452+0000 7f5349c00640 1 --2- 172.21.15.159:0/2583324061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f532c0051c0 0x7f532c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.452+0000 7f5349200640 1 -- 172.21.15.159:0/2583324061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f532c0051c0 msgr2=0x7f532c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.453+0000 7f5349200640 1 --2- 172.21.15.159:0/2583324061 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f532c0051c0 0x7f532c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.453+0000 7f5349200640 1 -- 172.21.15.159:0/2583324061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f532c004800 msgr2=0x7f532c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.453+0000 7f5349200640 1 --2- 172.21.15.159:0/2583324061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f532c004800 0x7f532c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.453+0000 7f5349200640 1 -- 172.21.15.159:0/2583324061 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f532c003520 con 0x7f532c0039b0 2024-04-04T06:14:44.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.453+0000 7f5349c00640 1 -- 172.21.15.159:0/2583324061 reap_dead start 2024-04-04T06:14:44.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.453+0000 7f5343e00640 1 --2- 172.21.15.159:0/2583324061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f532c004800 0x7f532c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:14:44.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.454+0000 7f5349200640 1 --2- 172.21.15.159:0/2583324061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f532c0039b0 0x7f532c003dd0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f533800c290 tx=0x7f532c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d79b7658e204de server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:44.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.454+0000 7f5343400640 1 -- 172.21.15.159:0/2583324061 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338007d30 con 0x7f532c0039b0 2024-04-04T06:14:44.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.454+0000 7f5343400640 1 -- 172.21.15.159:0/2583324061 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5338030070 con 0x7f532c0039b0 2024-04-04T06:14:44.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.454+0000 7f5343400640 1 -- 172.21.15.159:0/2583324061 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338005d30 con 0x7f532c0039b0 2024-04-04T06:14:44.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.455+0000 7f534a600640 1 -- 172.21.15.159:0/2583324061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f532c0039b0 msgr2=0x7f532c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.455+0000 7f534a600640 1 --2- 172.21.15.159:0/2583324061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f532c0039b0 0x7f532c003dd0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f533800c290 tx=0x7f532c005830 comp rx=0 tx=0).stop 2024-04-04T06:14:44.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.455+0000 7f534a600640 1 -- 172.21.15.159:0/2583324061 shutdown_connections 2024-04-04T06:14:44.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.455+0000 7f534a600640 1 --2- 172.21.15.159:0/2583324061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f532c0039b0 0x7f532c003dd0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.455+0000 7f534a600640 1 -- 172.21.15.159:0/2583324061 >> 172.21.15.159:0/2583324061 conn(0x7f534409aa20 msgr2=0x7f53440b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:44.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.455+0000 7f534a600640 1 -- 172.21.15.159:0/2583324061 shutdown_connections 2024-04-04T06:14:44.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.455+0000 7f534a600640 1 -- 172.21.15.159:0/2583324061 wait complete. 2024-04-04T06:14:44.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.458+0000 7f534a600640 1 Processor -- start 2024-04-04T06:14:44.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.458+0000 7f534a600640 1 -- start start 2024-04-04T06:14:44.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.460+0000 7f534a600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f532c0039b0 0x7f53441f8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.460+0000 7f534a600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53440c8ee0 0x7f53440b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.460+0000 7f534a600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5344160800 0x7f53440c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.460+0000 7f534a600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f534406bb40 con 0x7f532c0039b0 2024-04-04T06:14:44.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.460+0000 7f534a600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f534406b9c0 con 0x7f5344160800 2024-04-04T06:14:44.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.460+0000 7f534a600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f534406bcc0 con 0x7f53440c8ee0 2024-04-04T06:14:44.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.460+0000 7f5349200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f532c0039b0 0x7f53441f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.460+0000 7f5349200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f532c0039b0 0x7f53441f8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36994/0 (socket says 172.21.15.159:36994) 2024-04-04T06:14:44.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.460+0000 7f5349200640 1 -- 172.21.15.159:0/384980920 learned_addr learned my addr 172.21.15.159:0/384980920 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:44.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.460+0000 7f5343e00640 1 --2- 172.21.15.159:0/384980920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53440c8ee0 0x7f53440b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.460+0000 7f5349c00640 1 --2- 172.21.15.159:0/384980920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5344160800 0x7f53440c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.461+0000 7f5349200640 1 -- 172.21.15.159:0/384980920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53440c8ee0 msgr2=0x7f53440b1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.461+0000 7f5349200640 1 --2- 172.21.15.159:0/384980920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53440c8ee0 0x7f53440b1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.461+0000 7f5349200640 1 -- 172.21.15.159:0/384980920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5344160800 msgr2=0x7f53440c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.461+0000 7f5349200640 1 --2- 172.21.15.159:0/384980920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5344160800 0x7f53440c6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.461+0000 7f5349200640 1 -- 172.21.15.159:0/384980920 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53440b0d80 con 0x7f532c0039b0 2024-04-04T06:14:44.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.461+0000 7f5343e00640 1 --2- 172.21.15.159:0/384980920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53440c8ee0 0x7f53440b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:44.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.461+0000 7f5349200640 1 --2- 172.21.15.159:0/384980920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f532c0039b0 0x7f53441f8260 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f533800c3b0 tx=0x7f5338001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:44.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.461+0000 7f5340c00640 1 -- 172.21.15.159:0/384980920 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338007d30 con 0x7f532c0039b0 2024-04-04T06:14:44.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.461+0000 7f5340c00640 1 -- 172.21.15.159:0/384980920 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5338004c10 con 0x7f532c0039b0 2024-04-04T06:14:44.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.461+0000 7f5340c00640 1 -- 172.21.15.159:0/384980920 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338007070 con 0x7f532c0039b0 2024-04-04T06:14:44.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.461+0000 7f534a600640 1 -- 172.21.15.159:0/384980920 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53440b1020 con 0x7f532c0039b0 2024-04-04T06:14:44.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.461+0000 7f534a600640 1 -- 172.21.15.159:0/384980920 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53440b15e0 con 0x7f532c0039b0 2024-04-04T06:14:44.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.462+0000 7f534a600640 1 -- 172.21.15.159:0/384980920 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5344075940 con 0x7f532c0039b0 2024-04-04T06:14:44.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.463+0000 7f5340c00640 1 -- 172.21.15.159:0/384980920 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5338048020 con 0x7f532c0039b0 2024-04-04T06:14:44.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.464+0000 7f5340c00640 1 --2- 172.21.15.159:0/384980920 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f532406c8f0 0x7f532406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.464+0000 7f5340c00640 1 -- 172.21.15.159:0/384980920 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f53380ad140 con 0x7f532c0039b0 2024-04-04T06:14:44.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.465+0000 7f5343e00640 1 --2- 172.21.15.159:0/384980920 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f532406c8f0 0x7f532406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.466+0000 7f5343e00640 1 --2- 172.21.15.159:0/384980920 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f532406c8f0 0x7f532406edb0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5344063110 tx=0x7f5330008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:44.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.471+0000 7f5340c00640 1 -- 172.21.15.159:0/384980920 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f533807b470 con 0x7f532c0039b0 2024-04-04T06:14:44.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.709+0000 7f534a600640 1 -- 172.21.15.159:0/384980920 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f534406de30 con 0x7f532c0039b0 2024-04-04T06:14:44.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.710+0000 7f5340c00640 1 -- 172.21.15.159:0/384980920 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5196 (secure 0 0 0) 0x7f533807ad70 con 0x7f532c0039b0 2024-04-04T06:14:44.705 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:14:44.705 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":76,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1825651864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1825651864},{"type":"v1","addr":"172.21.15.178:6836","nonce":1825651864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1824578712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1824578712},{"type":"v1","addr":"172.21.15.178:6837","nonce":1824578712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6834/3377498298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3377498298},{"type":"v1","addr":"172.21.15.159:6834","nonce":3377498298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:12:35.880454+0000","modified":"2024-04-04T06:14:09.810244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7809},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7809":{"gid":7809,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6835/3555825575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":3555825575},{"type":"v1","addr":"172.21.15.159:6835","nonce":3555825575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7809,"qdb_cluster":[7809]},"id":6}]} 2024-04-04T06:14:44.705 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 76 2024-04-04T06:14:44.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.713+0000 7f534a600640 1 -- 172.21.15.159:0/384980920 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f532406c8f0 msgr2=0x7f532406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.713+0000 7f534a600640 1 --2- 172.21.15.159:0/384980920 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f532406c8f0 0x7f532406edb0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5344063110 tx=0x7f5330008040 comp rx=0 tx=0).stop 2024-04-04T06:14:44.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.713+0000 7f534a600640 1 -- 172.21.15.159:0/384980920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f532c0039b0 msgr2=0x7f53441f8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.713+0000 7f534a600640 1 --2- 172.21.15.159:0/384980920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f532c0039b0 0x7f53441f8260 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f533800c3b0 tx=0x7f5338001a40 comp rx=0 tx=0).stop 2024-04-04T06:14:44.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.713+0000 7f534a600640 1 -- 172.21.15.159:0/384980920 shutdown_connections 2024-04-04T06:14:44.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.713+0000 7f534a600640 1 --2- 172.21.15.159:0/384980920 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f532406c8f0 0x7f532406edb0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.713+0000 7f534a600640 1 --2- 172.21.15.159:0/384980920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5344160800 0x7f53440c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.713+0000 7f534a600640 1 --2- 172.21.15.159:0/384980920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53440c8ee0 0x7f53440b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.713+0000 7f534a600640 1 --2- 172.21.15.159:0/384980920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f532c0039b0 0x7f53441f8260 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.713+0000 7f534a600640 1 -- 172.21.15.159:0/384980920 >> 172.21.15.159:0/384980920 conn(0x7f534409aa20 msgr2=0x7f534406fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:44.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.713+0000 7f534a600640 1 -- 172.21.15.159:0/384980920 shutdown_connections 2024-04-04T06:14:44.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.713+0000 7f534a600640 1 -- 172.21.15.159:0/384980920 wait complete. 2024-04-04T06:14:44.726 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-04T06:14:44.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.927+0000 7f2d5a800640 1 Processor -- start 2024-04-04T06:14:44.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.927+0000 7f2d5a800640 1 -- start start 2024-04-04T06:14:44.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.929+0000 7f2d5a800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2d54146f70 0x7f2d54070430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.929+0000 7f2d5a800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54147350 0x7f2d540d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.929+0000 7f2d5a800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d5406cbf0 con 0x7f2d5406c680 2024-04-04T06:14:44.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.929+0000 7f2d5a800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2d540cffe0 con 0x7f2d54146f70 2024-04-04T06:14:44.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.929+0000 7f2d5a800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2d540d0230 con 0x7f2d54147350 2024-04-04T06:14:44.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.929+0000 7f2d59400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2d54146f70 0x7f2d54070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.930+0000 7f2d58a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54147350 0x7f2d540d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.930+0000 7f2d58a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54147350 0x7f2d540d0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:53674/0 (socket says 172.21.15.159:53674) 2024-04-04T06:14:44.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.930+0000 7f2d58a00640 1 -- 172.21.15.159:0/2905373367 learned_addr learned my addr 172.21.15.159:0/2905373367 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:44.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.930+0000 7f2d58a00640 1 -- 172.21.15.159:0/2905373367 >> v2:172.21.15.178:3300/0 conn(0x7f2d54146f70 msgr2=0x7f2d54070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.930+0000 7f2d58a00640 1 --2- 172.21.15.159:0/2905373367 >> v2:172.21.15.178:3300/0 conn(0x7f2d54146f70 0x7f2d54070430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.930+0000 7f2d58a00640 1 -- 172.21.15.159:0/2905373367 >> v1:172.21.15.178:6789/0 conn(0x7f2d5406c680 legacy=0x7f2d541620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.930+0000 7f2d58a00640 1 -- 172.21.15.159:0/2905373367 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d540d0510 con 0x7f2d54147350 2024-04-04T06:14:44.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.930+0000 7f2d59400640 1 --2- 172.21.15.159:0/2905373367 >> v2:172.21.15.178:3300/0 conn(0x7f2d54146f70 0x7f2d54070430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:14:44.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.932+0000 7f2d58a00640 1 --2- 172.21.15.159:0/2905373367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54147350 0x7f2d540d0d10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009870 tx=0x7f2d4002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=850c771bd68fd477 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:44.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.933+0000 7f2d53e00640 1 -- 172.21.15.159:0/2905373367 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40035020 con 0x7f2d54147350 2024-04-04T06:14:44.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.933+0000 7f2d53e00640 1 -- 172.21.15.159:0/2905373367 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d40031640 con 0x7f2d54147350 2024-04-04T06:14:44.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.933+0000 7f2d53e00640 1 -- 172.21.15.159:0/2905373367 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4002d040 con 0x7f2d54147350 2024-04-04T06:14:44.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.933+0000 7f2d5a800640 1 -- 172.21.15.159:0/2905373367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54147350 msgr2=0x7f2d540d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.933+0000 7f2d5a800640 1 --2- 172.21.15.159:0/2905373367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54147350 0x7f2d540d0d10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009870 tx=0x7f2d4002b8f0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.933+0000 7f2d5a800640 1 -- 172.21.15.159:0/2905373367 shutdown_connections 2024-04-04T06:14:44.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.933+0000 7f2d5a800640 1 --2- 172.21.15.159:0/2905373367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54147350 0x7f2d540d0d10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.933+0000 7f2d5a800640 1 --2- 172.21.15.159:0/2905373367 >> v2:172.21.15.178:3300/0 conn(0x7f2d54146f70 0x7f2d54070430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.933+0000 7f2d5a800640 1 -- 172.21.15.159:0/2905373367 >> 172.21.15.159:0/2905373367 conn(0x7f2d54067280 msgr2=0x7f2d54067660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:44.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.933+0000 7f2d5a800640 1 -- 172.21.15.159:0/2905373367 shutdown_connections 2024-04-04T06:14:44.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.933+0000 7f2d5a800640 1 -- 172.21.15.159:0/2905373367 wait complete. 2024-04-04T06:14:44.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.935+0000 7f2d5a800640 1 Processor -- start 2024-04-04T06:14:44.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.935+0000 7f2d5a800640 1 -- start start 2024-04-04T06:14:44.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.936+0000 7f2d5a800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d5406c680 0x7f2d541429c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.936+0000 7f2d5a800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54146f70 0x7f2d540cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.936+0000 7f2d5a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d54147350 0x7f2d540c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.936+0000 7f2d5a800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d540d1250 con 0x7f2d54147350 2024-04-04T06:14:44.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.936+0000 7f2d5a800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2d54070970 con 0x7f2d5406c680 2024-04-04T06:14:44.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.936+0000 7f2d5a800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2d540ca100 con 0x7f2d54146f70 2024-04-04T06:14:44.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.936+0000 7f2d59e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d54147350 0x7f2d540c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.937+0000 7f2d59e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d54147350 0x7f2d540c9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37010/0 (socket says 172.21.15.159:37010) 2024-04-04T06:14:44.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.937+0000 7f2d59e00640 1 -- 172.21.15.159:0/1626171209 learned_addr learned my addr 172.21.15.159:0/1626171209 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:44.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.937+0000 7f2d58a00640 1 --2- 172.21.15.159:0/1626171209 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54146f70 0x7f2d540cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.937+0000 7f2d59400640 1 --2- 172.21.15.159:0/1626171209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d5406c680 0x7f2d541429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.937+0000 7f2d59e00640 1 -- 172.21.15.159:0/1626171209 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54146f70 msgr2=0x7f2d540cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.937+0000 7f2d59e00640 1 --2- 172.21.15.159:0/1626171209 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54146f70 0x7f2d540cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.937+0000 7f2d59e00640 1 -- 172.21.15.159:0/1626171209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d5406c680 msgr2=0x7f2d541429c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.937+0000 7f2d59e00640 1 --2- 172.21.15.159:0/1626171209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d5406c680 0x7f2d541429c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.937+0000 7f2d59e00640 1 -- 172.21.15.159:0/1626171209 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5415f390 con 0x7f2d54147350 2024-04-04T06:14:44.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.937+0000 7f2d59e00640 1 --2- 172.21.15.159:0/1626171209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d54147350 0x7f2d540c9bc0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00c7d0 tx=0x7f2d4c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:44.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.938+0000 7f2d51600640 1 -- 172.21.15.159:0/1626171209 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c003b70 con 0x7f2d54147350 2024-04-04T06:14:44.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.938+0000 7f2d51600640 1 -- 172.21.15.159:0/1626171209 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d4c003d30 con 0x7f2d54147350 2024-04-04T06:14:44.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.938+0000 7f2d5a800640 1 -- 172.21.15.159:0/1626171209 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d540ca330 con 0x7f2d54147350 2024-04-04T06:14:44.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.938+0000 7f2d51600640 1 -- 172.21.15.159:0/1626171209 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c00aa90 con 0x7f2d54147350 2024-04-04T06:14:44.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.938+0000 7f2d5a800640 1 -- 172.21.15.159:0/1626171209 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d540ca8f0 con 0x7f2d54147350 2024-04-04T06:14:44.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.940+0000 7f2d51600640 1 -- 172.21.15.159:0/1626171209 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d4c00ac30 con 0x7f2d54147350 2024-04-04T06:14:44.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.941+0000 7f2d51600640 1 --2- 172.21.15.159:0/1626171209 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d3406c8f0 0x7f2d3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.941+0000 7f2d51600640 1 -- 172.21.15.159:0/1626171209 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f2d4c08bc80 con 0x7f2d54147350 2024-04-04T06:14:44.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.941+0000 7f2d59400640 1 --2- 172.21.15.159:0/1626171209 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d3406c8f0 0x7f2d3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.941+0000 7f2d5eb99740 1 Processor -- start 2024-04-04T06:14:44.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.941+0000 7f2d5eb99740 1 -- start start 2024-04-04T06:14:44.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.942+0000 7f2d5eb99740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ee254078c0 0x55ee25407ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.942+0000 7f2d5eb99740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253d78e0 0x55ee25404b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.942+0000 7f2d5eb99740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee253db440 0x55ee25405070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.942+0000 7f2d5eb99740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ee253415c0 con 0x55ee253d78e0 2024-04-04T06:14:44.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.942+0000 7f2d5eb99740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ee252e93b0 con 0x55ee253db440 2024-04-04T06:14:44.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.942+0000 7f2d5eb99740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ee2533d570 con 0x55ee254078c0 2024-04-04T06:14:44.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.942+0000 7f2d59400640 1 --2- 172.21.15.159:0/1626171209 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d3406c8f0 0x7f2d3406edb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009560 tx=0x7f2d48008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:44.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.943+0000 7f2d58a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253d78e0 0x55ee25404b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.943+0000 7f2d58a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253d78e0 0x55ee25404b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37026/0 (socket says 172.21.15.159:37026) 2024-04-04T06:14:44.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.943+0000 7f2d58a00640 1 -- 172.21.15.159:0/714135867 learned_addr learned my addr 172.21.15.159:0/714135867 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:44.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.943+0000 7f2d59400640 1 --2- 172.21.15.159:0/714135867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ee254078c0 0x55ee25407ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.943+0000 7f2d59e00640 1 --2- 172.21.15.159:0/714135867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee253db440 0x55ee25405070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.943+0000 7f2d58a00640 1 -- 172.21.15.159:0/714135867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ee254078c0 msgr2=0x55ee25407ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.943+0000 7f2d58a00640 1 --2- 172.21.15.159:0/714135867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ee254078c0 0x55ee25407ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.943+0000 7f2d58a00640 1 -- 172.21.15.159:0/714135867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee253db440 msgr2=0x55ee25405070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.943+0000 7f2d58a00640 1 --2- 172.21.15.159:0/714135867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee253db440 0x55ee25405070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.943+0000 7f2d58a00640 1 -- 172.21.15.159:0/714135867 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee251217c0 con 0x55ee253d78e0 2024-04-04T06:14:44.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.943+0000 7f2d59400640 1 --2- 172.21.15.159:0/714135867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ee254078c0 0x55ee25407ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:44.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.943+0000 7f2d58a00640 1 --2- 172.21.15.159:0/714135867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253d78e0 0x55ee25404b30 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f2d40000c00 tx=0x7f2d40008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b09504494c4e055 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:44.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.943+0000 7f2d45c00640 1 -- 172.21.15.159:0/714135867 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40035040 con 0x55ee253d78e0 2024-04-04T06:14:44.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.944+0000 7f2d45c00640 1 -- 172.21.15.159:0/714135867 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d400317d0 con 0x55ee253d78e0 2024-04-04T06:14:44.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.944+0000 7f2d45c00640 1 -- 172.21.15.159:0/714135867 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40031b80 con 0x55ee253d78e0 2024-04-04T06:14:44.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.944+0000 7f2d5eb99740 1 -- 172.21.15.159:0/714135867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253d78e0 msgr2=0x55ee25404b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.944+0000 7f2d5eb99740 1 --2- 172.21.15.159:0/714135867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253d78e0 0x55ee25404b30 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f2d40000c00 tx=0x7f2d40008030 comp rx=0 tx=0).stop 2024-04-04T06:14:44.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.944+0000 7f2d5eb99740 1 -- 172.21.15.159:0/714135867 shutdown_connections 2024-04-04T06:14:44.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.944+0000 7f2d5eb99740 1 --2- 172.21.15.159:0/714135867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee253db440 0x55ee25405070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.944+0000 7f2d5eb99740 1 --2- 172.21.15.159:0/714135867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253d78e0 0x55ee25404b30 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.944+0000 7f2d5eb99740 1 --2- 172.21.15.159:0/714135867 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ee254078c0 0x55ee25407ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.944+0000 7f2d5eb99740 1 -- 172.21.15.159:0/714135867 >> 172.21.15.159:0/714135867 conn(0x55ee253d9b50 msgr2=0x55ee254226d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:44.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.944+0000 7f2d5eb99740 1 -- 172.21.15.159:0/714135867 shutdown_connections 2024-04-04T06:14:44.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.944+0000 7f2d5eb99740 1 -- 172.21.15.159:0/714135867 wait complete. 2024-04-04T06:14:44.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.945+0000 7f2d5eb99740 1 -- start start 2024-04-04T06:14:44.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.945+0000 7f2d5eb99740 1 Processor -- start 2024-04-04T06:14:44.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.946+0000 7f2d5eb99740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee253db440 0x55ee254012e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.946+0000 7f2d5eb99740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ee254078c0 0x55ee25401820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.946+0000 7f2d5eb99740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253f97c0 0x55ee253fd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.946+0000 7f2d5eb99740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ee253dc5f0 con 0x55ee253f97c0 2024-04-04T06:14:44.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.946+0000 7f2d5eb99740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ee252fc7a0 con 0x55ee253db440 2024-04-04T06:14:44.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.946+0000 7f2d5eb99740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ee253db9e0 con 0x55ee254078c0 2024-04-04T06:14:44.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.946+0000 7f2d5eb99740 10 client.0 _ll_register_callbacks cb 0x55ee25338fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:14:44.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.946+0000 7f2d59e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253f97c0 0x55ee253fd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.946+0000 7f2d59e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253f97c0 0x55ee253fd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37036/0 (socket says 172.21.15.159:37036) 2024-04-04T06:14:44.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.946+0000 7f2d59e00640 1 -- 172.21.15.159:0/4151085524 learned_addr learned my addr 172.21.15.159:0/4151085524 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:44.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.947+0000 7f2d59400640 1 --2- 172.21.15.159:0/4151085524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee253db440 0x55ee254012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.947+0000 7f2d58a00640 1 --2- 172.21.15.159:0/4151085524 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ee254078c0 0x55ee25401820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.947+0000 7f2d59e00640 1 -- 172.21.15.159:0/4151085524 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ee254078c0 msgr2=0x55ee25401820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.947+0000 7f2d59e00640 1 --2- 172.21.15.159:0/4151085524 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ee254078c0 0x55ee25401820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.947+0000 7f2d59e00640 1 -- 172.21.15.159:0/4151085524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee253db440 msgr2=0x55ee254012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.947+0000 7f2d59e00640 1 --2- 172.21.15.159:0/4151085524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee253db440 0x55ee254012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.947+0000 7f2d59e00640 1 -- 172.21.15.159:0/4151085524 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee25181f50 con 0x55ee253f97c0 2024-04-04T06:14:44.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.947+0000 7f2d59e00640 1 --2- 172.21.15.159:0/4151085524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253f97c0 0x55ee253fd520 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00c7a0 tx=0x7f2d4c0600b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e5ed5d3f0011 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:44.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.947+0000 7f2d3a000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:14:44.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.947+0000 7f2d3a000640 1 -- 172.21.15.159:0/4151085524 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c05db50 con 0x55ee253f97c0 2024-04-04T06:14:44.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.947+0000 7f2d3a000640 1 -- 172.21.15.159:0/4151085524 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f2d4c055e10 con 0x55ee253f97c0 2024-04-04T06:14:44.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.947+0000 7f2d5eb99740 1 -- 172.21.15.159:0/4151085524 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d400317d0 con 0x55ee253f97c0 2024-04-04T06:14:44.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.947+0000 7f2d3a000640 1 -- 172.21.15.159:0/4151085524 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f2d4c062380 con 0x55ee253f97c0 2024-04-04T06:14:44.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.948+0000 7f2d5eb99740 10 client.7968 fetch_fsmap learned FSMap version 76 2024-04-04T06:14:44.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.948+0000 7f2d5eb99740 1 -- 172.21.15.159:0/4151085524 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55ee250150b0 con 0x55ee253f97c0 2024-04-04T06:14:44.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.948+0000 7f2d3a000640 1 -- 172.21.15.159:0/4151085524 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d400317d0 con 0x55ee253f97c0 2024-04-04T06:14:44.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.948+0000 7f2d3a000640 1 -- 172.21.15.159:0/4151085524 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 76) ==== 3494+0+0 (secure 0 0 0) 0x7f2d4c05ece0 con 0x55ee253f97c0 2024-04-04T06:14:44.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.948+0000 7f2d5eb99740 10 client.7968 fetch_fsmap finished waiting for FSMap version 76 2024-04-04T06:14:44.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.948+0000 7f2d5eb99740 10 client.7968 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:14:44.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.948+0000 7f2d5eb99740 20 client.7968 populate_metadata read hostname 'smithi159' 2024-04-04T06:14:44.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.948+0000 7f2d5eb99740 1 --2- 172.21.15.159:0/4151085524 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55ee253ff030 0x55ee253ff410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.948+0000 7f2d5eb99740 4 client.7968 mds_command: new command op to 7809 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:14:44.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.948+0000 7f2d5eb99740 1 -- 172.21.15.159:0/4151085524 --> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee25315890 con 0x55ee253ff030 2024-04-04T06:14:44.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.948+0000 7f2d59400640 1 --2- 172.21.15.159:0/4151085524 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55ee253ff030 0x55ee253ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.949+0000 7f2d59400640 1 --2- 172.21.15.159:0/4151085524 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55ee253ff030 0x55ee253ff410 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fc60b2686f69db5 server_cookie=b00b64f537f74f96 in_seq=0 out_seq=0 2024-04-04T06:14:44.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.949+0000 7f2d3a000640 10 client.7968 ms_handle_connect on v2:172.21.15.159:6833/3555825575 2024-04-04T06:14:44.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.949+0000 7f2d3a000640 1 -- 172.21.15.159:0/4151085524 <== mds.0 v2:172.21.15.159:6833/3555825575 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee25315890 con 0x55ee253ff030 2024-04-04T06:14:44.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.949+0000 7f2d3a000640 10 client.7968 handle_command_reply: tid=0 2024-04-04T06:14:44.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.949+0000 7f2d5eb99740 1 client.7968 shutdown 2024-04-04T06:14:44.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.950+0000 7f2d5eb99740 1 -- 172.21.15.159:0/4151085524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253f97c0 msgr2=0x55ee253fd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.950+0000 7f2d5eb99740 1 --2- 172.21.15.159:0/4151085524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253f97c0 0x55ee253fd520 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00c7a0 tx=0x7f2d4c0600b0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.950+0000 7f2d5eb99740 1 -- 172.21.15.159:0/4151085524 shutdown_connections 2024-04-04T06:14:44.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.950+0000 7f2d5eb99740 1 --2- 172.21.15.159:0/4151085524 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55ee253ff030 0x55ee253ff410 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.950+0000 7f2d5eb99740 1 --2- 172.21.15.159:0/4151085524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253f97c0 0x55ee253fd520 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.950+0000 7f2d5eb99740 1 --2- 172.21.15.159:0/4151085524 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ee254078c0 0x55ee25401820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.950+0000 7f2d5eb99740 1 --2- 172.21.15.159:0/4151085524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee253db440 0x55ee254012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.950+0000 7f2d5eb99740 1 -- 172.21.15.159:0/4151085524 >> 172.21.15.159:0/4151085524 conn(0x55ee253d78e0 msgr2=0x55ee25425be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:44.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.950+0000 7f2d3a000640 0 client.7968 ms_handle_reset on v2:172.21.15.159:6833/3555825575 2024-04-04T06:14:44.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.950+0000 7f2d5eb99740 1 -- 172.21.15.159:0/4151085524 shutdown_connections 2024-04-04T06:14:44.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.950+0000 7f2d5eb99740 1 -- 172.21.15.159:0/4151085524 wait complete. 2024-04-04T06:14:44.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.950+0000 7f2d5eb99740 20 client.7968 trim_cache size 0 max 16384 2024-04-04T06:14:44.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.965+0000 7f2d5eb99740 1 Processor -- start 2024-04-04T06:14:44.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.965+0000 7f2d5eb99740 1 -- start start 2024-04-04T06:14:44.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.965+0000 7f2d5eb99740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee253db440 0x55ee25422a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.965+0000 7f2d5eb99740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253f97c0 0x55ee25426020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.965+0000 7f2d5eb99740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ee254078c0 0x55ee254091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.965+0000 7f2d5eb99740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541ffba0 con 0x55ee253f97c0 2024-04-04T06:14:44.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.965+0000 7f2d5eb99740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541e8990 con 0x55ee253db440 2024-04-04T06:14:44.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.965+0000 7f2d5eb99740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2d5413fa40 con 0x55ee254078c0 2024-04-04T06:14:44.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.966+0000 7f2d58a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253f97c0 0x55ee25426020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.966+0000 7f2d58a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253f97c0 0x55ee25426020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37046/0 (socket says 172.21.15.159:37046) 2024-04-04T06:14:44.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.966+0000 7f2d58a00640 1 -- 172.21.15.159:0/3862758402 learned_addr learned my addr 172.21.15.159:0/3862758402 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:44.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.966+0000 7f2d58a00640 1 -- 172.21.15.159:0/3862758402 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ee254078c0 msgr2=0x55ee254091e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.966+0000 7f2d58a00640 1 --2- 172.21.15.159:0/3862758402 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ee254078c0 0x55ee254091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.966+0000 7f2d58a00640 1 -- 172.21.15.159:0/3862758402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee253db440 msgr2=0x55ee25422a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.966+0000 7f2d58a00640 1 --2- 172.21.15.159:0/3862758402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee253db440 0x55ee25422a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.966+0000 7f2d58a00640 1 -- 172.21.15.159:0/3862758402 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee253fde20 con 0x55ee253f97c0 2024-04-04T06:14:44.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.966+0000 7f2d58a00640 1 --2- 172.21.15.159:0/3862758402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253f97c0 0x55ee25426020 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009960 tx=0x7f2d40031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e61809f4d41de server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:44.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.967+0000 7f2d45200640 1 -- 172.21.15.159:0/3862758402 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40035070 con 0x55ee253f97c0 2024-04-04T06:14:44.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.967+0000 7f2d45200640 1 -- 172.21.15.159:0/3862758402 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d40004040 con 0x55ee253f97c0 2024-04-04T06:14:44.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.967+0000 7f2d45200640 1 -- 172.21.15.159:0/3862758402 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40004430 con 0x55ee253f97c0 2024-04-04T06:14:44.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.967+0000 7f2d5eb99740 1 -- 172.21.15.159:0/3862758402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253f97c0 msgr2=0x55ee25426020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.967+0000 7f2d5eb99740 1 --2- 172.21.15.159:0/3862758402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253f97c0 0x55ee25426020 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009960 tx=0x7f2d40031000 comp rx=0 tx=0).stop 2024-04-04T06:14:44.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.967+0000 7f2d5eb99740 1 -- 172.21.15.159:0/3862758402 shutdown_connections 2024-04-04T06:14:44.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.967+0000 7f2d5eb99740 1 --2- 172.21.15.159:0/3862758402 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ee254078c0 0x55ee254091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.967+0000 7f2d5eb99740 1 --2- 172.21.15.159:0/3862758402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253f97c0 0x55ee25426020 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.967+0000 7f2d5eb99740 1 --2- 172.21.15.159:0/3862758402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee253db440 0x55ee25422a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.967+0000 7f2d5eb99740 1 -- 172.21.15.159:0/3862758402 >> 172.21.15.159:0/3862758402 conn(0x55ee253d78e0 msgr2=0x55ee25446920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:44.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.967+0000 7f2d5eb99740 1 -- 172.21.15.159:0/3862758402 shutdown_connections 2024-04-04T06:14:44.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.967+0000 7f2d5eb99740 1 -- 172.21.15.159:0/3862758402 wait complete. 2024-04-04T06:14:44.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.968+0000 7f2d5eb99740 1 -- start start 2024-04-04T06:14:44.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.968+0000 7f2d5eb99740 1 Processor -- start 2024-04-04T06:14:44.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.969+0000 7f2d5eb99740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee253db440 0x55ee25484f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.969+0000 7f2d5eb99740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253f97c0 0x55ee254858a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.969+0000 7f2d5eb99740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ee254078c0 0x55ee253ff310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.969+0000 7f2d5eb99740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ee2540a4d0 con 0x55ee253f97c0 2024-04-04T06:14:44.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.969+0000 7f2d5eb99740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ee2540b400 con 0x55ee253db440 2024-04-04T06:14:44.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.969+0000 7f2d5eb99740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ee253fdbd0 con 0x55ee254078c0 2024-04-04T06:14:44.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.969+0000 7f2d5eb99740 10 client.0 _ll_register_callbacks cb 0x7f2d4c001830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:14:44.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.969+0000 7f2d58a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253f97c0 0x55ee254858a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.969+0000 7f2d58a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253f97c0 0x55ee254858a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37052/0 (socket says 172.21.15.159:37052) 2024-04-04T06:14:44.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.969+0000 7f2d58a00640 1 -- 172.21.15.159:0/1798555822 learned_addr learned my addr 172.21.15.159:0/1798555822 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:44.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.970+0000 7f2d59400640 1 --2- 172.21.15.159:0/1798555822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee253db440 0x55ee25484f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.970+0000 7f2d58a00640 1 -- 172.21.15.159:0/1798555822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ee254078c0 msgr2=0x55ee253ff310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.970+0000 7f2d58a00640 1 --2- 172.21.15.159:0/1798555822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ee254078c0 0x55ee253ff310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.970+0000 7f2d58a00640 1 -- 172.21.15.159:0/1798555822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee253db440 msgr2=0x55ee25484f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:44.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.970+0000 7f2d58a00640 1 --2- 172.21.15.159:0/1798555822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee253db440 0x55ee25484f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.970+0000 7f2d58a00640 1 -- 172.21.15.159:0/1798555822 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee25409950 con 0x55ee253f97c0 2024-04-04T06:14:44.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.970+0000 7f2d58a00640 1 --2- 172.21.15.159:0/1798555822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253f97c0 0x55ee254858a0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009990 tx=0x7f2d40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439cfc3960735f6d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:44.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.970+0000 7f2d3a000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:14:44.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.970+0000 7f2d3a000640 1 -- 172.21.15.159:0/1798555822 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40035040 con 0x55ee253f97c0 2024-04-04T06:14:44.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.970+0000 7f2d3a000640 1 -- 172.21.15.159:0/1798555822 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f2d40031960 con 0x55ee253f97c0 2024-04-04T06:14:44.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.970+0000 7f2d5eb99740 1 -- 172.21.15.159:0/1798555822 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d40004040 con 0x55ee253f97c0 2024-04-04T06:14:44.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.971+0000 7f2d3a000640 1 -- 172.21.15.159:0/1798555822 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f2d4005d050 con 0x55ee253f97c0 2024-04-04T06:14:44.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.971+0000 7f2d5eb99740 10 client.7974 fetch_fsmap learned FSMap version 76 2024-04-04T06:14:44.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.971+0000 7f2d5eb99740 1 -- 172.21.15.159:0/1798555822 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55ee253fc120 con 0x55ee253f97c0 2024-04-04T06:14:44.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.971+0000 7f2d3a000640 1 -- 172.21.15.159:0/1798555822 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d40004040 con 0x55ee253f97c0 2024-04-04T06:14:44.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.971+0000 7f2d3a000640 1 -- 172.21.15.159:0/1798555822 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 76) ==== 3494+0+0 (secure 0 0 0) 0x7f2d40060020 con 0x55ee253f97c0 2024-04-04T06:14:44.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.971+0000 7f2d5eb99740 10 client.7974 fetch_fsmap finished waiting for FSMap version 76 2024-04-04T06:14:44.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.971+0000 7f2d5eb99740 10 client.7974 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:14:44.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.971+0000 7f2d5eb99740 20 client.7974 populate_metadata read hostname 'smithi159' 2024-04-04T06:14:44.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.971+0000 7f2d5eb99740 1 --2- 172.21.15.159:0/1798555822 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55ee25431a90 0x55ee25408180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:44.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.971+0000 7f2d5eb99740 4 client.7974 mds_command: new command op to 7809 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-04T06:14:44.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.971+0000 7f2d5eb99740 1 -- 172.21.15.159:0/1798555822 --> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55ee2534eb00 con 0x55ee25431a90 2024-04-04T06:14:44.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.971+0000 7f2d59400640 1 --2- 172.21.15.159:0/1798555822 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55ee25431a90 0x55ee25408180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:44.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.972+0000 7f2d59400640 1 --2- 172.21.15.159:0/1798555822 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55ee25431a90 0x55ee25408180 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a27b524712c2d161 server_cookie=b4840ccb40ba9bae in_seq=0 out_seq=0 2024-04-04T06:14:44.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.972+0000 7f2d3a000640 10 client.7974 ms_handle_connect on v2:172.21.15.159:6833/3555825575 2024-04-04T06:14:44.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.972+0000 7f2d3a000640 1 -- 172.21.15.159:0/1798555822 <== mds.0 v2:172.21.15.159:6833/3555825575 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee2534eb00 con 0x55ee25431a90 2024-04-04T06:14:44.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.972+0000 7f2d3a000640 10 client.7974 handle_command_reply: tid=0 2024-04-04T06:14:44.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.972+0000 7f2d5eb99740 1 client.7974 shutdown 2024-04-04T06:14:44.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.972+0000 7f2d5eb99740 1 -- 172.21.15.159:0/1798555822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253f97c0 msgr2=0x55ee254858a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.972+0000 7f2d5eb99740 1 --2- 172.21.15.159:0/1798555822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253f97c0 0x55ee254858a0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009990 tx=0x7f2d40004730 comp rx=0 tx=0).stop 2024-04-04T06:14:44.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.972+0000 7f2d5eb99740 1 -- 172.21.15.159:0/1798555822 shutdown_connections 2024-04-04T06:14:44.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.972+0000 7f2d5eb99740 1 --2- 172.21.15.159:0/1798555822 >> [v2:172.21.15.159:6833/3555825575,v1:172.21.15.159:6835/3555825575] conn(0x55ee25431a90 0x55ee25408180 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.972+0000 7f2d5eb99740 1 --2- 172.21.15.159:0/1798555822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ee254078c0 0x55ee253ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.972+0000 7f2d5eb99740 1 --2- 172.21.15.159:0/1798555822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee253f97c0 0x55ee254858a0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.972+0000 7f2d5eb99740 1 --2- 172.21.15.159:0/1798555822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ee253db440 0x55ee25484f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.972+0000 7f2d3a000640 0 client.7974 ms_handle_reset on v2:172.21.15.159:6833/3555825575 2024-04-04T06:14:44.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.972+0000 7f2d5eb99740 1 -- 172.21.15.159:0/1798555822 >> 172.21.15.159:0/1798555822 conn(0x55ee253d78e0 msgr2=0x55ee254220b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:44.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.972+0000 7f2d5eb99740 1 -- 172.21.15.159:0/1798555822 shutdown_connections 2024-04-04T06:14:44.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.972+0000 7f2d5eb99740 1 -- 172.21.15.159:0/1798555822 wait complete. 2024-04-04T06:14:44.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.973+0000 7f2d5eb99740 20 client.7974 trim_cache size 0 max 16384 2024-04-04T06:14:44.970 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:14:44.971 INFO:teuthology.orchestra.run.smithi159.stdout:[] 2024-04-04T06:14:44.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.973+0000 7f2d5a800640 1 -- 172.21.15.159:0/1626171209 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d3406c8f0 msgr2=0x7f2d3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.973+0000 7f2d5a800640 1 --2- 172.21.15.159:0/1626171209 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d3406c8f0 0x7f2d3406edb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009560 tx=0x7f2d48008040 comp rx=0 tx=0).stop 2024-04-04T06:14:44.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.973+0000 7f2d5a800640 1 -- 172.21.15.159:0/1626171209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d54147350 msgr2=0x7f2d540c9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:44.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.973+0000 7f2d5a800640 1 --2- 172.21.15.159:0/1626171209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d54147350 0x7f2d540c9bc0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00c7d0 tx=0x7f2d4c00cc90 comp rx=0 tx=0).stop 2024-04-04T06:14:44.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.973+0000 7f2d5a800640 1 -- 172.21.15.159:0/1626171209 shutdown_connections 2024-04-04T06:14:44.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.973+0000 7f2d5a800640 1 --2- 172.21.15.159:0/1626171209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d54147350 0x7f2d540c9bc0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.973+0000 7f2d5a800640 1 --2- 172.21.15.159:0/1626171209 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2d54146f70 0x7f2d540cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.973+0000 7f2d5a800640 1 --2- 172.21.15.159:0/1626171209 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2d3406c8f0 0x7f2d3406edb0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.973+0000 7f2d5a800640 1 --2- 172.21.15.159:0/1626171209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d5406c680 0x7f2d541429c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:44.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.973+0000 7f2d5a800640 1 -- 172.21.15.159:0/1626171209 >> 172.21.15.159:0/1626171209 conn(0x7f2d54067280 msgr2=0x7f2d540d1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:44.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.973+0000 7f2d5a800640 1 -- 172.21.15.159:0/1626171209 shutdown_connections 2024-04-04T06:14:44.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:44.973+0000 7f2d5a800640 1 -- 172.21.15.159:0/1626171209 wait complete. 2024-04-04T06:14:44.984 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-04T06:14:45.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.180+0000 7f92b1e00640 1 Processor -- start 2024-04-04T06:14:45.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.180+0000 7f92b1e00640 1 -- start start 2024-04-04T06:14:45.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.181+0000 7f92b1e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f92ac153580 0x7f92ac0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:45.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.181+0000 7f92b1e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92ac0afab0 con 0x7f92ac0d07c0 2024-04-04T06:14:45.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.182+0000 7f92b1e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92ac15cc80 con 0x7f92ac0af4b0 2024-04-04T06:14:45.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.182+0000 7f92b1e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f92ac0d0c30 con 0x7f92ac153580 2024-04-04T06:14:45.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.182+0000 7f92aa000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f92ac0d07c0 0x7f92ac0aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:36478/0 (socket says 172.21.15.159:36478) 2024-04-04T06:14:45.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.182+0000 7f92aa000640 1 -- 172.21.15.159:0/2867491014 learned_addr learned my addr 172.21.15.159:0/2867491014 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:45.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.182+0000 7f92aaa00640 1 --2- 172.21.15.159:0/2867491014 >> v2:172.21.15.159:3300/0 conn(0x7f92ac153580 0x7f92ac0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:45.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.182+0000 7f92a9600640 1 -- 172.21.15.159:0/2867491014 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2498302087 0 0) 0x7f92ac0afab0 con 0x7f92ac0d07c0 2024-04-04T06:14:45.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.182+0000 7f92a9600640 1 -- 172.21.15.159:0/2867491014 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9290003520 con 0x7f92ac0d07c0 2024-04-04T06:14:45.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.182+0000 7f92aaa00640 1 -- 172.21.15.159:0/2867491014 >> v1:172.21.15.178:6789/0 conn(0x7f92ac0af4b0 legacy=0x7f92ac1600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:45.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.182+0000 7f92aaa00640 1 -- 172.21.15.159:0/2867491014 >> v1:172.21.15.159:6789/0 conn(0x7f92ac0d07c0 legacy=0x7f92ac0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:45.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.182+0000 7f92aaa00640 1 -- 172.21.15.159:0/2867491014 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ac15da00 con 0x7f92ac153580 2024-04-04T06:14:45.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.184+0000 7f92aaa00640 1 --2- 172.21.15.159:0/2867491014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92ac153580 0x7f92ac0d0280 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f929c0097a0 tx=0x7f929c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3a025aebfcb3be server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:45.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.184+0000 7f92a9600640 1 -- 172.21.15.159:0/2867491014 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c0315a0 con 0x7f92ac153580 2024-04-04T06:14:45.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.184+0000 7f92a9600640 1 -- 172.21.15.159:0/2867491014 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f929c037070 con 0x7f92ac153580 2024-04-04T06:14:45.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.184+0000 7f92a9600640 1 -- 172.21.15.159:0/2867491014 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c031960 con 0x7f92ac153580 2024-04-04T06:14:45.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.184+0000 7f92b1e00640 1 -- 172.21.15.159:0/2867491014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92ac153580 msgr2=0x7f92ac0d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:45.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.184+0000 7f92b1e00640 1 --2- 172.21.15.159:0/2867491014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92ac153580 0x7f92ac0d0280 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f929c0097a0 tx=0x7f929c02b840 comp rx=0 tx=0).stop 2024-04-04T06:14:45.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.185+0000 7f92b1e00640 1 -- 172.21.15.159:0/2867491014 shutdown_connections 2024-04-04T06:14:45.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.185+0000 7f92b1e00640 1 --2- 172.21.15.159:0/2867491014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92ac153580 0x7f92ac0d0280 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:45.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.185+0000 7f92b1e00640 1 -- 172.21.15.159:0/2867491014 >> 172.21.15.159:0/2867491014 conn(0x7f92ac06b750 msgr2=0x7f92ac06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:45.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.185+0000 7f92b1e00640 1 -- 172.21.15.159:0/2867491014 shutdown_connections 2024-04-04T06:14:45.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.185+0000 7f92b1e00640 1 -- 172.21.15.159:0/2867491014 wait complete. 2024-04-04T06:14:45.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.186+0000 7f92b1e00640 1 Processor -- start 2024-04-04T06:14:45.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.187+0000 7f92b1e00640 1 -- start start 2024-04-04T06:14:45.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.188+0000 7f92b1e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92ac0af4b0 0x7f92ac153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:45.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.188+0000 7f92b1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92ac0d07c0 0x7f92ac072f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:45.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.188+0000 7f92b1e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92ac153580 0x7f92ac1549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:45.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.188+0000 7f92b1e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac15d4e0 con 0x7f92ac0d07c0 2024-04-04T06:14:45.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.188+0000 7f92b1e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac15d200 con 0x7f92ac153580 2024-04-04T06:14:45.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.188+0000 7f92b1e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f92ac153960 con 0x7f92ac0af4b0 2024-04-04T06:14:45.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.188+0000 7f92aa000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92ac0d07c0 0x7f92ac072f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:45.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.188+0000 7f92aa000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92ac0d07c0 0x7f92ac072f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37068/0 (socket says 172.21.15.159:37068) 2024-04-04T06:14:45.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.188+0000 7f92aa000640 1 -- 172.21.15.159:0/457787659 learned_addr learned my addr 172.21.15.159:0/457787659 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:45.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.188+0000 7f92aa000640 1 -- 172.21.15.159:0/457787659 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92ac0af4b0 msgr2=0x7f92ac153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:45.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.188+0000 7f92aaa00640 1 --2- 172.21.15.159:0/457787659 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92ac0af4b0 0x7f92ac153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:45.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.188+0000 7f92aa000640 1 --2- 172.21.15.159:0/457787659 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92ac0af4b0 0x7f92ac153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:45.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.188+0000 7f92aa000640 1 -- 172.21.15.159:0/457787659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92ac153580 msgr2=0x7f92ac1549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:45.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.188+0000 7f92ab400640 1 --2- 172.21.15.159:0/457787659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92ac153580 0x7f92ac1549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:45.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.188+0000 7f92aa000640 1 --2- 172.21.15.159:0/457787659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92ac153580 0x7f92ac1549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:45.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.188+0000 7f92aa000640 1 -- 172.21.15.159:0/457787659 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ac071ec0 con 0x7f92ac0d07c0 2024-04-04T06:14:45.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.188+0000 7f92aaa00640 1 --2- 172.21.15.159:0/457787659 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92ac0af4b0 0x7f92ac153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:45.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.189+0000 7f92aa000640 1 --2- 172.21.15.159:0/457787659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92ac0d07c0 0x7f92ac072f60 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f929800c430 tx=0x7f929800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:45.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.189+0000 7f92a3400640 1 -- 172.21.15.159:0/457787659 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298017070 con 0x7f92ac0d07c0 2024-04-04T06:14:45.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.189+0000 7f92a3400640 1 -- 172.21.15.159:0/457787659 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9298005ce0 con 0x7f92ac0d07c0 2024-04-04T06:14:45.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.189+0000 7f92b1e00640 1 -- 172.21.15.159:0/457787659 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92ac0721a0 con 0x7f92ac0d07c0 2024-04-04T06:14:45.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.189+0000 7f92a3400640 1 -- 172.21.15.159:0/457787659 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92980072f0 con 0x7f92ac0d07c0 2024-04-04T06:14:45.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.189+0000 7f92b1e00640 1 -- 172.21.15.159:0/457787659 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92ac072760 con 0x7f92ac0d07c0 2024-04-04T06:14:45.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.190+0000 7f92b1e00640 1 -- 172.21.15.159:0/457787659 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92ac0685c0 con 0x7f92ac0d07c0 2024-04-04T06:14:45.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.191+0000 7f92a3400640 1 -- 172.21.15.159:0/457787659 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f929801a070 con 0x7f92ac0d07c0 2024-04-04T06:14:45.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.192+0000 7f92a3400640 1 --2- 172.21.15.159:0/457787659 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f928c06c920 0x7f928c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:45.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.192+0000 7f92a3400640 1 -- 172.21.15.159:0/457787659 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f929808b200 con 0x7f92ac0d07c0 2024-04-04T06:14:45.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.193+0000 7f92aaa00640 1 --2- 172.21.15.159:0/457787659 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f928c06c920 0x7f928c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:45.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.194+0000 7f92aaa00640 1 --2- 172.21.15.159:0/457787659 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f928c06c920 0x7f928c06ede0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f929c036000 tx=0x7f929c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:45.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.196+0000 7f92a3400640 1 -- 172.21.15.159:0/457787659 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9298059410 con 0x7f92ac0d07c0 2024-04-04T06:14:45.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.428+0000 7f92b1e00640 1 -- 172.21.15.159:0/457787659 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f92ac15e7f0 con 0x7f92ac0d07c0 2024-04-04T06:14:45.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.428+0000 7f92a3400640 1 -- 172.21.15.159:0/457787659 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f9298058d10 con 0x7f92ac0d07c0 2024-04-04T06:14:45.422 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:14:45.423 INFO:teuthology.orchestra.run.smithi159.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-04T06:14:45.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.431+0000 7f92b1e00640 1 -- 172.21.15.159:0/457787659 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f928c06c920 msgr2=0x7f928c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:45.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.431+0000 7f92b1e00640 1 --2- 172.21.15.159:0/457787659 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f928c06c920 0x7f928c06ede0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f929c036000 tx=0x7f929c008030 comp rx=0 tx=0).stop 2024-04-04T06:14:45.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.431+0000 7f92b1e00640 1 -- 172.21.15.159:0/457787659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92ac0d07c0 msgr2=0x7f92ac072f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:45.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.431+0000 7f92b1e00640 1 --2- 172.21.15.159:0/457787659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92ac0d07c0 0x7f92ac072f60 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f929800c430 tx=0x7f929800c8f0 comp rx=0 tx=0).stop 2024-04-04T06:14:45.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.431+0000 7f92b1e00640 1 -- 172.21.15.159:0/457787659 shutdown_connections 2024-04-04T06:14:45.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.431+0000 7f92b1e00640 1 --2- 172.21.15.159:0/457787659 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f928c06c920 0x7f928c06ede0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:45.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.431+0000 7f92b1e00640 1 --2- 172.21.15.159:0/457787659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92ac153580 0x7f92ac1549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:45.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.431+0000 7f92b1e00640 1 --2- 172.21.15.159:0/457787659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92ac0d07c0 0x7f92ac072f60 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:45.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.431+0000 7f92b1e00640 1 --2- 172.21.15.159:0/457787659 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f92ac0af4b0 0x7f92ac153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:45.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.432+0000 7f92b1e00640 1 -- 172.21.15.159:0/457787659 >> 172.21.15.159:0/457787659 conn(0x7f92ac06b750 msgr2=0x7f92ac0afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:45.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.432+0000 7f92b1e00640 1 -- 172.21.15.159:0/457787659 shutdown_connections 2024-04-04T06:14:45.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.432+0000 7f92b1e00640 1 -- 172.21.15.159:0/457787659 wait complete. 2024-04-04T06:14:45.442 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-04T06:14:45.442 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-04T06:14:45.443 DEBUG:teuthology.orchestra.run.smithi159:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:14:45.557 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:14:45.677 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-04T06:14:45.677 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:14:45.677 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:14:45.677 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:14:45.731 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:14:45.732 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:14:45.783 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:14:45.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.990+0000 7f00e7600640 1 Processor -- start 2024-04-04T06:14:45.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.990+0000 7f00e7600640 1 -- start start 2024-04-04T06:14:45.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.991+0000 7f00e7600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00e015c390 0x7f00e0073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:45.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.991+0000 7f00e7600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f00e00af830 0x7f00e0160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:45.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.992+0000 7f00e7600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00e015fe80 con 0x7f00e0160b80 2024-04-04T06:14:45.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.992+0000 7f00e7600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f00e015c770 con 0x7f00e00af830 2024-04-04T06:14:45.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.992+0000 7f00e7600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f00e00ad120 con 0x7f00e015c390 2024-04-04T06:14:45.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.992+0000 7f00dfe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f00e00af830 0x7f00e0160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:45.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.992+0000 7f00dfe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f00e00af830 0x7f00e0160640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37070/0 (socket says 172.21.15.159:37070) 2024-04-04T06:14:45.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.992+0000 7f00dfe00640 1 -- 172.21.15.159:0/2556106542 learned_addr learned my addr 172.21.15.159:0/2556106542 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:45.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.992+0000 7f00e4c00640 1 --2- 172.21.15.159:0/2556106542 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00e015c390 0x7f00e0073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:45.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.992+0000 7f00dfe00640 1 -- 172.21.15.159:0/2556106542 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00e015c390 msgr2=0x7f00e0073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:45.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.992+0000 7f00dfe00640 1 --2- 172.21.15.159:0/2556106542 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00e015c390 0x7f00e0073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:45.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.992+0000 7f00dfe00640 1 -- 172.21.15.159:0/2556106542 >> v1:172.21.15.178:6789/0 conn(0x7f00e0160b80 legacy=0x7f00e00d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:45.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.992+0000 7f00dfe00640 1 -- 172.21.15.159:0/2556106542 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e00ad400 con 0x7f00e00af830 2024-04-04T06:14:45.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.994+0000 7f00dfe00640 1 --2- 172.21.15.159:0/2556106542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00e00af830 0x7f00e0160640 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f00cc009870 tx=0x7f00cc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9c77846066297c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:45.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.994+0000 7f00df400640 1 -- 172.21.15.159:0/2556106542 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc035020 con 0x7f00e00af830 2024-04-04T06:14:45.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.994+0000 7f00df400640 1 -- 172.21.15.159:0/2556106542 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00cc031640 con 0x7f00e00af830 2024-04-04T06:14:45.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.994+0000 7f00df400640 1 -- 172.21.15.159:0/2556106542 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc02d040 con 0x7f00e00af830 2024-04-04T06:14:45.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.994+0000 7f00e7600640 1 -- 172.21.15.159:0/2556106542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00e00af830 msgr2=0x7f00e0160640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:45.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.994+0000 7f00e7600640 1 --2- 172.21.15.159:0/2556106542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00e00af830 0x7f00e0160640 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f00cc009870 tx=0x7f00cc02b8f0 comp rx=0 tx=0).stop 2024-04-04T06:14:45.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.995+0000 7f00e7600640 1 -- 172.21.15.159:0/2556106542 shutdown_connections 2024-04-04T06:14:45.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.995+0000 7f00e7600640 1 --2- 172.21.15.159:0/2556106542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00e00af830 0x7f00e0160640 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:45.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.995+0000 7f00e7600640 1 --2- 172.21.15.159:0/2556106542 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00e015c390 0x7f00e0073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:45.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.995+0000 7f00e7600640 1 -- 172.21.15.159:0/2556106542 >> 172.21.15.159:0/2556106542 conn(0x7f00e006b990 msgr2=0x7f00e006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:45.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.995+0000 7f00e7600640 1 -- 172.21.15.159:0/2556106542 shutdown_connections 2024-04-04T06:14:45.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.995+0000 7f00e7600640 1 -- 172.21.15.159:0/2556106542 wait complete. 2024-04-04T06:14:45.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.996+0000 7f00e7600640 1 Processor -- start 2024-04-04T06:14:45.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.997+0000 7f00e7600640 1 -- start start 2024-04-04T06:14:45.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.997+0000 7f00e7600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00e00af830 0x7f00e01f8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:45.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.997+0000 7f00e7600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00e015c390 0x7f00e01e84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:45.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.997+0000 7f00e7600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00e0160b80 0x7f00e015cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:45.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.997+0000 7f00e7600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f00e00cf980 con 0x7f00e00af830 2024-04-04T06:14:45.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.997+0000 7f00e7600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f00e0160120 con 0x7f00e015c390 2024-04-04T06:14:45.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.997+0000 7f00e7600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f00e00d0d60 con 0x7f00e0160b80 2024-04-04T06:14:45.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.998+0000 7f00e4c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00e00af830 0x7f00e01f8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:45.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.998+0000 7f00e4c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00e00af830 0x7f00e01f8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37078/0 (socket says 172.21.15.159:37078) 2024-04-04T06:14:45.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.998+0000 7f00e4c00640 1 -- 172.21.15.159:0/451660788 learned_addr learned my addr 172.21.15.159:0/451660788 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:45.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.998+0000 7f00dfe00640 1 --2- 172.21.15.159:0/451660788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00e015c390 0x7f00e01e84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:45.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.998+0000 7f00e4c00640 1 -- 172.21.15.159:0/451660788 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00e0160b80 msgr2=0x7f00e015cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:45.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.998+0000 7f00e4c00640 1 --2- 172.21.15.159:0/451660788 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00e0160b80 0x7f00e015cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:45.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.998+0000 7f00e4c00640 1 -- 172.21.15.159:0/451660788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00e015c390 msgr2=0x7f00e01e84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:45.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.998+0000 7f00e4c00640 1 --2- 172.21.15.159:0/451660788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00e015c390 0x7f00e01e84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:45.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.998+0000 7f00e4c00640 1 -- 172.21.15.159:0/451660788 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e015d050 con 0x7f00e00af830 2024-04-04T06:14:45.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.998+0000 7f00dfe00640 1 --2- 172.21.15.159:0/451660788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00e015c390 0x7f00e01e84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:45.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.999+0000 7f00e4c00640 1 --2- 172.21.15.159:0/451660788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00e00af830 0x7f00e01f8d70 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f00d400d800 tx=0x7f00d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:45.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.999+0000 7f00dcc00640 1 -- 172.21.15.159:0/451660788 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d400b660 con 0x7f00e00af830 2024-04-04T06:14:45.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.999+0000 7f00dcc00640 1 -- 172.21.15.159:0/451660788 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d4019070 con 0x7f00e00af830 2024-04-04T06:14:45.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.999+0000 7f00dcc00640 1 -- 172.21.15.159:0/451660788 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d400ba20 con 0x7f00e00af830 2024-04-04T06:14:45.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.999+0000 7f00e7600640 1 -- 172.21.15.159:0/451660788 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e015d2f0 con 0x7f00e00af830 2024-04-04T06:14:45.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:45.999+0000 7f00e7600640 1 -- 172.21.15.159:0/451660788 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e015d8b0 con 0x7f00e00af830 2024-04-04T06:14:45.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.000+0000 7f00e7600640 1 -- 172.21.15.159:0/451660788 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e00685c0 con 0x7f00e00af830 2024-04-04T06:14:45.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.001+0000 7f00dcc00640 1 -- 172.21.15.159:0/451660788 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f00d400bbc0 con 0x7f00e00af830 2024-04-04T06:14:45.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.002+0000 7f00dcc00640 1 --2- 172.21.15.159:0/451660788 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f00c006c8f0 0x7f00c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:45.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.002+0000 7f00dcc00640 1 -- 172.21.15.159:0/451660788 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f00d408c070 con 0x7f00e00af830 2024-04-04T06:14:45.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.003+0000 7f00dfe00640 1 --2- 172.21.15.159:0/451660788 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f00c006c8f0 0x7f00c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:45.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.004+0000 7f00dfe00640 1 --2- 172.21.15.159:0/451660788 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f00c006c8f0 0x7f00c006edb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f00cc036000 tx=0x7f00cc008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:46.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.006+0000 7f00dcc00640 1 -- 172.21.15.159:0/451660788 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f00d401c360 con 0x7f00e00af830 2024-04-04T06:14:46.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.209+0000 7f00e7600640 1 -- 172.21.15.159:0/451660788 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00e015b190 con 0x7f00e00af830 2024-04-04T06:14:46.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.212+0000 7f00dcc00640 1 -- 172.21.15.159:0/451660788 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+17033 (secure 0 0 0) 0x7f00d405a3b0 con 0x7f00e00af830 2024-04-04T06:14:46.207 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:14:46.207 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":89,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:14:04.757031+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:12:33.658405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-04T06:12:34.856266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.3d","mappings":[{"from":6,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6835/3796123128":"2024-04-05T06:14:04.756982+0000","172.21.15.159:6833/3796123128":"2024-04-05T06:14:04.756982+0000","172.21.15.159:6835/3484779950":"2024-04-05T06:12:49.205338+0000","172.21.15.159:6833/3484779950":"2024-04-05T06:12:49.205338+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:14:46.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.215+0000 7f00e7600640 1 -- 172.21.15.159:0/451660788 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f00c006c8f0 msgr2=0x7f00c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:46.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.215+0000 7f00e7600640 1 --2- 172.21.15.159:0/451660788 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f00c006c8f0 0x7f00c006edb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f00cc036000 tx=0x7f00cc008030 comp rx=0 tx=0).stop 2024-04-04T06:14:46.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.215+0000 7f00e7600640 1 -- 172.21.15.159:0/451660788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00e00af830 msgr2=0x7f00e01f8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:46.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.215+0000 7f00e7600640 1 --2- 172.21.15.159:0/451660788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00e00af830 0x7f00e01f8d70 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f00d400d800 tx=0x7f00d400dcc0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.215+0000 7f00e7600640 1 -- 172.21.15.159:0/451660788 shutdown_connections 2024-04-04T06:14:46.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.215+0000 7f00e7600640 1 --2- 172.21.15.159:0/451660788 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00e0160b80 0x7f00e015cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.215+0000 7f00e7600640 1 --2- 172.21.15.159:0/451660788 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f00c006c8f0 0x7f00c006edb0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.215+0000 7f00e7600640 1 --2- 172.21.15.159:0/451660788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00e015c390 0x7f00e01e84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.215+0000 7f00e7600640 1 --2- 172.21.15.159:0/451660788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00e00af830 0x7f00e01f8d70 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.215+0000 7f00e7600640 1 -- 172.21.15.159:0/451660788 >> 172.21.15.159:0/451660788 conn(0x7f00e006b990 msgr2=0x7f00e01554f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:46.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.215+0000 7f00e7600640 1 -- 172.21.15.159:0/451660788 shutdown_connections 2024-04-04T06:14:46.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.215+0000 7f00e7600640 1 -- 172.21.15.159:0/451660788 wait complete. 2024-04-04T06:14:46.229 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:12:33.658405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:12:34.856266+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:14:46.229 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:14:46.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.445+0000 7f591b200640 1 Processor -- start 2024-04-04T06:14:46.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.445+0000 7f591b200640 1 -- start start 2024-04-04T06:14:46.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.447+0000 7f591b200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f591c15e4c0 0x7f591c0b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:46.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.447+0000 7f591b200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f591c0b69c0 0x7f591c0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:46.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.447+0000 7f591b200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f591c0d2710 con 0x7f591c0d37b0 2024-04-04T06:14:46.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.447+0000 7f591b200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f591c15e8a0 con 0x7f591c15e4c0 2024-04-04T06:14:46.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.447+0000 7f591b200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f591c0d4040 con 0x7f591c0b69c0 2024-04-04T06:14:46.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.447+0000 7f5919e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f591c15e4c0 0x7f591c0b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:46.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.447+0000 7f591a800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f591c0b69c0 0x7f591c0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:46.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.447+0000 7f591a800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f591c0b69c0 0x7f591c0b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:53776/0 (socket says 172.21.15.159:53776) 2024-04-04T06:14:46.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.447+0000 7f591a800640 1 -- 172.21.15.159:0/2525650129 learned_addr learned my addr 172.21.15.159:0/2525650129 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:46.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.448+0000 7f5919e00640 1 -- 172.21.15.159:0/2525650129 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f591c0b69c0 msgr2=0x7f591c0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:46.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.448+0000 7f5919e00640 1 --2- 172.21.15.159:0/2525650129 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f591c0b69c0 0x7f591c0b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.448+0000 7f5919e00640 1 -- 172.21.15.159:0/2525650129 >> v1:172.21.15.178:6789/0 conn(0x7f591c0d37b0 legacy=0x7f591c0b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:46.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.448+0000 7f5919e00640 1 -- 172.21.15.159:0/2525650129 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f591c0d29f0 con 0x7f591c15e4c0 2024-04-04T06:14:46.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.449+0000 7f5919e00640 1 --2- 172.21.15.159:0/2525650129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f591c15e4c0 0x7f591c0b9bf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f59040097a0 tx=0x7f590402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f10816f6d01da3f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:46.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.450+0000 7f5918a00640 1 -- 172.21.15.159:0/2525650129 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5904035020 con 0x7f591c15e4c0 2024-04-04T06:14:46.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.450+0000 7f5918a00640 1 -- 172.21.15.159:0/2525650129 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5904031830 con 0x7f591c15e4c0 2024-04-04T06:14:46.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.450+0000 7f5918a00640 1 -- 172.21.15.159:0/2525650129 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5904031c00 con 0x7f591c15e4c0 2024-04-04T06:14:46.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.450+0000 7f591b200640 1 -- 172.21.15.159:0/2525650129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f591c15e4c0 msgr2=0x7f591c0b9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:46.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.450+0000 7f591b200640 1 --2- 172.21.15.159:0/2525650129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f591c15e4c0 0x7f591c0b9bf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f59040097a0 tx=0x7f590402b8d0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.451+0000 7f591b200640 1 -- 172.21.15.159:0/2525650129 shutdown_connections 2024-04-04T06:14:46.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.451+0000 7f591b200640 1 --2- 172.21.15.159:0/2525650129 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f591c0b69c0 0x7f591c0b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.451+0000 7f591b200640 1 --2- 172.21.15.159:0/2525650129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f591c15e4c0 0x7f591c0b9bf0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.451+0000 7f591b200640 1 -- 172.21.15.159:0/2525650129 >> 172.21.15.159:0/2525650129 conn(0x7f591c0676a0 msgr2=0x7f591c067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:46.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.451+0000 7f591b200640 1 -- 172.21.15.159:0/2525650129 shutdown_connections 2024-04-04T06:14:46.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.451+0000 7f591b200640 1 -- 172.21.15.159:0/2525650129 wait complete. 2024-04-04T06:14:46.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.452+0000 7f591b200640 1 Processor -- start 2024-04-04T06:14:46.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.453+0000 7f591b200640 1 -- start start 2024-04-04T06:14:46.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.454+0000 7f591b200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f591c0b69c0 0x7f591c151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:46.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.454+0000 7f591b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f591c0d37b0 0x7f591c15f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:46.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.454+0000 7f591b200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f591c15e4c0 0x7f591c1518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:46.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.454+0000 7f591b200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f591c0d2fb0 con 0x7f591c0d37b0 2024-04-04T06:14:46.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.454+0000 7f591b200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f591c0d3320 con 0x7f591c15e4c0 2024-04-04T06:14:46.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.454+0000 7f591b200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f591c0ba130 con 0x7f591c0b69c0 2024-04-04T06:14:46.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.454+0000 7f5919400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f591c0d37b0 0x7f591c15f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:46.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.454+0000 7f5919400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f591c0d37b0 0x7f591c15f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37086/0 (socket says 172.21.15.159:37086) 2024-04-04T06:14:46.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.454+0000 7f5919400640 1 -- 172.21.15.159:0/1973363560 learned_addr learned my addr 172.21.15.159:0/1973363560 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:46.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.454+0000 7f591a800640 1 --2- 172.21.15.159:0/1973363560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f591c15e4c0 0x7f591c1518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:46.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.454+0000 7f5919e00640 1 --2- 172.21.15.159:0/1973363560 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f591c0b69c0 0x7f591c151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:46.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.454+0000 7f5919400640 1 -- 172.21.15.159:0/1973363560 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f591c0b69c0 msgr2=0x7f591c151f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:46.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.454+0000 7f5919400640 1 --2- 172.21.15.159:0/1973363560 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f591c0b69c0 0x7f591c151f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.454+0000 7f5919400640 1 -- 172.21.15.159:0/1973363560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f591c15e4c0 msgr2=0x7f591c1518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:46.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.454+0000 7f5919400640 1 --2- 172.21.15.159:0/1973363560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f591c15e4c0 0x7f591c1518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.455+0000 7f5919400640 1 -- 172.21.15.159:0/1973363560 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f591c160070 con 0x7f591c0d37b0 2024-04-04T06:14:46.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.455+0000 7f5919400640 1 --2- 172.21.15.159:0/1973363560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f591c0d37b0 0x7f591c15f440 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f590c00c430 tx=0x7f590c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:46.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.455+0000 7f5912000640 1 -- 172.21.15.159:0/1973363560 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c0052d0 con 0x7f591c0d37b0 2024-04-04T06:14:46.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.455+0000 7f5912000640 1 -- 172.21.15.159:0/1973363560 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f590c005490 con 0x7f591c0d37b0 2024-04-04T06:14:46.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.455+0000 7f5912000640 1 -- 172.21.15.159:0/1973363560 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c007260 con 0x7f591c0d37b0 2024-04-04T06:14:46.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.455+0000 7f591b200640 1 -- 172.21.15.159:0/1973363560 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f591c1619d0 con 0x7f591c0d37b0 2024-04-04T06:14:46.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.455+0000 7f591b200640 1 -- 172.21.15.159:0/1973363560 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f591c15fd90 con 0x7f591c0d37b0 2024-04-04T06:14:46.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.457+0000 7f591b200640 1 -- 172.21.15.159:0/1973363560 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f591c075940 con 0x7f591c0d37b0 2024-04-04T06:14:46.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.457+0000 7f5912000640 1 -- 172.21.15.159:0/1973363560 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f590c007400 con 0x7f591c0d37b0 2024-04-04T06:14:46.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.458+0000 7f5912000640 1 --2- 172.21.15.159:0/1973363560 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f58f806c8f0 0x7f58f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:46.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.458+0000 7f5912000640 1 -- 172.21.15.159:0/1973363560 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f590c08b1b0 con 0x7f591c0d37b0 2024-04-04T06:14:46.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.459+0000 7f5919e00640 1 --2- 172.21.15.159:0/1973363560 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f58f806c8f0 0x7f58f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:46.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.460+0000 7f5919e00640 1 --2- 172.21.15.159:0/1973363560 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f58f806c8f0 0x7f58f806edb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5904036000 tx=0x7f590403c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:46.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.463+0000 7f5912000640 1 -- 172.21.15.159:0/1973363560 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f590c0594e0 con 0x7f591c0d37b0 2024-04-04T06:14:46.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.663+0000 7f591b200640 1 -- 172.21.15.159:0/1973363560 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f591c0b3ef0 con 0x7f591c0d37b0 2024-04-04T06:14:46.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.664+0000 7f5912000640 1 -- 172.21.15.159:0/1973363560 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f590c058de0 con 0x7f591c0d37b0 2024-04-04T06:14:46.658 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:14:46.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.667+0000 7f591b200640 1 -- 172.21.15.159:0/1973363560 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f58f806c8f0 msgr2=0x7f58f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:46.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.667+0000 7f591b200640 1 --2- 172.21.15.159:0/1973363560 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f58f806c8f0 0x7f58f806edb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5904036000 tx=0x7f590403c040 comp rx=0 tx=0).stop 2024-04-04T06:14:46.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.667+0000 7f591b200640 1 -- 172.21.15.159:0/1973363560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f591c0d37b0 msgr2=0x7f591c15f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:46.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.667+0000 7f591b200640 1 --2- 172.21.15.159:0/1973363560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f591c0d37b0 0x7f591c15f440 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f590c00c430 tx=0x7f590c00c8f0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.667+0000 7f591b200640 1 -- 172.21.15.159:0/1973363560 shutdown_connections 2024-04-04T06:14:46.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.667+0000 7f591b200640 1 --2- 172.21.15.159:0/1973363560 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f58f806c8f0 0x7f58f806edb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.667+0000 7f591b200640 1 --2- 172.21.15.159:0/1973363560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f591c15e4c0 0x7f591c1518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.667+0000 7f591b200640 1 --2- 172.21.15.159:0/1973363560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f591c0d37b0 0x7f591c15f440 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.667+0000 7f591b200640 1 --2- 172.21.15.159:0/1973363560 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f591c0b69c0 0x7f591c151f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.667+0000 7f591b200640 1 -- 172.21.15.159:0/1973363560 >> 172.21.15.159:0/1973363560 conn(0x7f591c0676a0 msgr2=0x7f591c06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:46.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.667+0000 7f591b200640 1 -- 172.21.15.159:0/1973363560 shutdown_connections 2024-04-04T06:14:46.662 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.667+0000 7f591b200640 1 -- 172.21.15.159:0/1973363560 wait complete. 2024-04-04T06:14:46.678 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:14:46.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.883+0000 7f9599600640 1 Processor -- start 2024-04-04T06:14:46.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.883+0000 7f9599600640 1 -- start start 2024-04-04T06:14:46.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.885+0000 7f9599600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95940d43d0 0x7f95940d47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:46.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.885+0000 7f9599600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f95940cf6b0 0x7f95940cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:46.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.885+0000 7f9599600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f959406dfa0 con 0x7f959415e860 2024-04-04T06:14:46.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.885+0000 7f9599600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f95940cf220 con 0x7f95940cf6b0 2024-04-04T06:14:46.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.885+0000 7f9599600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f95940d4cf0 con 0x7f95940d43d0 2024-04-04T06:14:46.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.885+0000 7f9592a00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f959415e860 0x7f959406d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:36480/0 (socket says 172.21.15.159:36480) 2024-04-04T06:14:46.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.885+0000 7f9592a00640 1 -- 172.21.15.159:0/2491235364 learned_addr learned my addr 172.21.15.159:0/2491235364 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:46.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.886+0000 7f9593400640 1 --2- 172.21.15.159:0/2491235364 >> v2:172.21.15.178:3300/0 conn(0x7f95940cf6b0 0x7f95940cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:46.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.886+0000 7f9592000640 1 --2- 172.21.15.159:0/2491235364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95940d43d0 0x7f95940d47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:46.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.886+0000 7f9591600640 1 -- 172.21.15.159:0/2491235364 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3454540098 0 0) 0x7f959406dfa0 con 0x7f959415e860 2024-04-04T06:14:46.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.886+0000 7f9591600640 1 -- 172.21.15.159:0/2491235364 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9578003520 con 0x7f959415e860 2024-04-04T06:14:46.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.886+0000 7f9591600640 1 -- 172.21.15.159:0/2491235364 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3707030107 0 0) 0x7f9578003520 con 0x7f959415e860 2024-04-04T06:14:46.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.886+0000 7f9591600640 1 -- 172.21.15.159:0/2491235364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95940d43d0 msgr2=0x7f95940d47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:46.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.886+0000 7f9591600640 1 --2- 172.21.15.159:0/2491235364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95940d43d0 0x7f95940d47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.886+0000 7f9591600640 1 -- 172.21.15.159:0/2491235364 >> v2:172.21.15.178:3300/0 conn(0x7f95940cf6b0 msgr2=0x7f95940cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:46.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.886+0000 7f9591600640 1 --2- 172.21.15.159:0/2491235364 >> v2:172.21.15.178:3300/0 conn(0x7f95940cf6b0 0x7f95940cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.886+0000 7f9591600640 1 -- 172.21.15.159:0/2491235364 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959406e240 con 0x7f959415e860 2024-04-04T06:14:46.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.886+0000 7f9591600640 1 -- 172.21.15.159:0/2491235364 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f9584001670 con 0x7f959415e860 2024-04-04T06:14:46.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.886+0000 7f9591600640 1 -- 172.21.15.159:0/2491235364 >> v1:172.21.15.159:6789/0 conn(0x7f959415e860 legacy=0x7f959406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:46.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.886+0000 7f9591600640 1 --2- 172.21.15.159:0/2491235364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95940d43d0 0x7f95780039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:46.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.887+0000 7f9591600640 1 --2- 172.21.15.159:0/2491235364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9578003f50 0x7f9578004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:46.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.887+0000 7f9591600640 1 --2- 172.21.15.159:0/2491235364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9578004d40 0x7f95780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:46.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.887+0000 7f9591600640 1 -- 172.21.15.159:0/2491235364 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f95940cf220 con 0x7f9578003f50 2024-04-04T06:14:46.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.887+0000 7f9591600640 1 -- 172.21.15.159:0/2491235364 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95940d4cf0 con 0x7f9578004d40 2024-04-04T06:14:46.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.887+0000 7f9591600640 1 -- 172.21.15.159:0/2491235364 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f957800b9c0 con 0x7f95940d43d0 2024-04-04T06:14:46.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.887+0000 7f9592000640 1 --2- 172.21.15.159:0/2491235364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9578003f50 0x7f9578004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:46.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.887+0000 7f9592a00640 1 --2- 172.21.15.159:0/2491235364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95940d43d0 0x7f95780039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:46.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.887+0000 7f9593400640 1 --2- 172.21.15.159:0/2491235364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9578004d40 0x7f95780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:46.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.887+0000 7f9592000640 1 -- 172.21.15.159:0/2491235364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95940d43d0 msgr2=0x7f95780039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:46.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.887+0000 7f9592000640 1 --2- 172.21.15.159:0/2491235364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95940d43d0 0x7f95780039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.887+0000 7f9592000640 1 -- 172.21.15.159:0/2491235364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9578004d40 msgr2=0x7f95780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:46.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.887+0000 7f9592000640 1 --2- 172.21.15.159:0/2491235364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9578004d40 0x7f95780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.887+0000 7f9592000640 1 -- 172.21.15.159:0/2491235364 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9578003520 con 0x7f9578003f50 2024-04-04T06:14:46.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.887+0000 7f9593400640 1 --2- 172.21.15.159:0/2491235364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9578004d40 0x7f95780051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:14:46.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.890+0000 7f9592000640 1 --2- 172.21.15.159:0/2491235364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9578003f50 0x7f9578004800 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f9580009af0 tx=0x7f9578005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df816b3055464345 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:46.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.890+0000 7f9591600640 1 -- 172.21.15.159:0/2491235364 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580039070 con 0x7f9578003f50 2024-04-04T06:14:46.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.891+0000 7f9591600640 1 -- 172.21.15.159:0/2491235364 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f958002c6b0 con 0x7f9578003f50 2024-04-04T06:14:46.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.891+0000 7f9591600640 1 -- 172.21.15.159:0/2491235364 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580009690 con 0x7f9578003f50 2024-04-04T06:14:46.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.891+0000 7f9599600640 1 -- 172.21.15.159:0/2491235364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9578003f50 msgr2=0x7f9578004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:46.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.891+0000 7f9599600640 1 --2- 172.21.15.159:0/2491235364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9578003f50 0x7f9578004800 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f9580009af0 tx=0x7f9578005830 comp rx=0 tx=0).stop 2024-04-04T06:14:46.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.891+0000 7f9593400640 1 -- 172.21.15.159:0/2491235364 reap_dead start 2024-04-04T06:14:46.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.891+0000 7f9599600640 1 -- 172.21.15.159:0/2491235364 shutdown_connections 2024-04-04T06:14:46.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.891+0000 7f9599600640 1 -- 172.21.15.159:0/2491235364 >> 172.21.15.159:0/2491235364 conn(0x7f959409d7d0 msgr2=0x7f959408bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:46.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.891+0000 7f9599600640 1 -- 172.21.15.159:0/2491235364 shutdown_connections 2024-04-04T06:14:46.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.891+0000 7f9599600640 1 -- 172.21.15.159:0/2491235364 wait complete. 2024-04-04T06:14:46.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.893+0000 7f9599600640 1 Processor -- start 2024-04-04T06:14:46.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.893+0000 7f9599600640 1 -- start start 2024-04-04T06:14:46.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.894+0000 7f9599600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9578003f50 0x7f95940b8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:46.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.894+0000 7f9599600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9594150980 0x7f95940b6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:46.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.894+0000 7f9599600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95940b7500 0x7f95940d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:46.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.894+0000 7f9599600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f959406dc00 con 0x7f9578003f50 2024-04-04T06:14:46.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.894+0000 7f9599600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95940d2c50 con 0x7f95940b7500 2024-04-04T06:14:46.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.894+0000 7f9599600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f959406dd80 con 0x7f9594150980 2024-04-04T06:14:46.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.894+0000 7f9592a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9578003f50 0x7f95940b8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:46.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.894+0000 7f9592a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9578003f50 0x7f95940b8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37106/0 (socket says 172.21.15.159:37106) 2024-04-04T06:14:46.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.894+0000 7f9592a00640 1 -- 172.21.15.159:0/1218100693 learned_addr learned my addr 172.21.15.159:0/1218100693 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:46.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.895+0000 7f9592a00640 1 -- 172.21.15.159:0/1218100693 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9594150980 msgr2=0x7f95940b6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:46.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.895+0000 7f9592a00640 1 --2- 172.21.15.159:0/1218100693 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9594150980 0x7f95940b6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.895+0000 7f9593400640 1 --2- 172.21.15.159:0/1218100693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95940b7500 0x7f95940d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:46.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.895+0000 7f9592a00640 1 -- 172.21.15.159:0/1218100693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95940b7500 msgr2=0x7f95940d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:46.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.895+0000 7f9592a00640 1 --2- 172.21.15.159:0/1218100693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95940b7500 0x7f95940d0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:46.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.895+0000 7f9592a00640 1 -- 172.21.15.159:0/1218100693 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95940b7b30 con 0x7f9578003f50 2024-04-04T06:14:46.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.895+0000 7f9592a00640 1 --2- 172.21.15.159:0/1218100693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9578003f50 0x7f95940b8bd0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f9584001860 tx=0x7f958400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:46.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.895+0000 7f958aa00640 1 -- 172.21.15.159:0/1218100693 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958400c8d0 con 0x7f9578003f50 2024-04-04T06:14:46.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.895+0000 7f958aa00640 1 -- 172.21.15.159:0/1218100693 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9584018070 con 0x7f9578003f50 2024-04-04T06:14:46.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.895+0000 7f9599600640 1 -- 172.21.15.159:0/1218100693 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95940b7e10 con 0x7f9578003f50 2024-04-04T06:14:46.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.895+0000 7f958aa00640 1 -- 172.21.15.159:0/1218100693 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958400ccd0 con 0x7f9578003f50 2024-04-04T06:14:46.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.896+0000 7f9599600640 1 -- 172.21.15.159:0/1218100693 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95940b83d0 con 0x7f9578003f50 2024-04-04T06:14:46.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.897+0000 7f9599600640 1 -- 172.21.15.159:0/1218100693 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95940685c0 con 0x7f9578003f50 2024-04-04T06:14:46.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.898+0000 7f958aa00640 1 -- 172.21.15.159:0/1218100693 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f958401d460 con 0x7f9578003f50 2024-04-04T06:14:46.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.898+0000 7f958aa00640 1 --2- 172.21.15.159:0/1218100693 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f957406c8f0 0x7f957406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:46.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.899+0000 7f958aa00640 1 -- 172.21.15.159:0/1218100693 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f958408baf0 con 0x7f9578003f50 2024-04-04T06:14:46.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.899+0000 7f9592000640 1 --2- 172.21.15.159:0/1218100693 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f957406c8f0 0x7f957406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:46.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.900+0000 7f9592000640 1 --2- 172.21.15.159:0/1218100693 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f957406c8f0 0x7f957406edb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9580036000 tx=0x7f958002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:46.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:46.903+0000 7f958aa00640 1 -- 172.21.15.159:0/1218100693 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9584059ec0 con 0x7f9578003f50 2024-04-04T06:14:47.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.103+0000 7f9599600640 1 -- 172.21.15.159:0/1218100693 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9594161f20 con 0x7f9578003f50 2024-04-04T06:14:47.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.103+0000 7f958aa00640 1 -- 172.21.15.159:0/1218100693 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f95840597c0 con 0x7f9578003f50 2024-04-04T06:14:47.098 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:14:47.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.106+0000 7f9599600640 1 -- 172.21.15.159:0/1218100693 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f957406c8f0 msgr2=0x7f957406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:47.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.106+0000 7f9599600640 1 --2- 172.21.15.159:0/1218100693 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f957406c8f0 0x7f957406edb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9580036000 tx=0x7f958002e000 comp rx=0 tx=0).stop 2024-04-04T06:14:47.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.106+0000 7f9599600640 1 -- 172.21.15.159:0/1218100693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9578003f50 msgr2=0x7f95940b8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:47.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.106+0000 7f9599600640 1 --2- 172.21.15.159:0/1218100693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9578003f50 0x7f95940b8bd0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f9584001860 tx=0x7f958400ecd0 comp rx=0 tx=0).stop 2024-04-04T06:14:47.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.106+0000 7f9599600640 1 -- 172.21.15.159:0/1218100693 shutdown_connections 2024-04-04T06:14:47.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.106+0000 7f9599600640 1 --2- 172.21.15.159:0/1218100693 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f957406c8f0 0x7f957406edb0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:47.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.106+0000 7f9599600640 1 --2- 172.21.15.159:0/1218100693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95940b7500 0x7f95940d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:47.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.106+0000 7f9599600640 1 --2- 172.21.15.159:0/1218100693 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9594150980 0x7f95940b6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:47.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.106+0000 7f9599600640 1 --2- 172.21.15.159:0/1218100693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9578003f50 0x7f95940b8bd0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:47.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.106+0000 7f9599600640 1 -- 172.21.15.159:0/1218100693 >> 172.21.15.159:0/1218100693 conn(0x7f959409d7d0 msgr2=0x7f95940cfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:47.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.106+0000 7f9599600640 1 -- 172.21.15.159:0/1218100693 shutdown_connections 2024-04-04T06:14:47.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.106+0000 7f9599600640 1 -- 172.21.15.159:0/1218100693 wait complete. 2024-04-04T06:14:47.119 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:14:47.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.321+0000 7f5787a00640 1 Processor -- start 2024-04-04T06:14:47.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.322+0000 7f5787a00640 1 -- start start 2024-04-04T06:14:47.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.323+0000 7f5787a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f57800d0990 0x7f57800aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:47.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.323+0000 7f5787a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f57800af2e0 0x7f57801600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:47.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.323+0000 7f5787a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57800afab0 con 0x7f5780153580 2024-04-04T06:14:47.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.323+0000 7f5787a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f578015cc80 con 0x7f57800d0990 2024-04-04T06:14:47.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.323+0000 7f5787a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f578015da00 con 0x7f57800af2e0 2024-04-04T06:14:47.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.323+0000 7f577fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f57800d0990 0x7f57800aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:47.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.324+0000 7f577fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f57800d0990 0x7f57800aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37118/0 (socket says 172.21.15.159:37118) 2024-04-04T06:14:47.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.324+0000 7f577fe00640 1 -- 172.21.15.159:0/3829759108 learned_addr learned my addr 172.21.15.159:0/3829759108 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:47.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.324+0000 7f5785a00640 1 --2- 172.21.15.159:0/3829759108 >> v2:172.21.15.178:3300/0 conn(0x7f57800af2e0 0x7f57801600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:47.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.324+0000 7f577fe00640 1 -- 172.21.15.159:0/3829759108 >> v2:172.21.15.178:3300/0 conn(0x7f57800af2e0 msgr2=0x7f57801600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:47.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.324+0000 7f577fe00640 1 --2- 172.21.15.159:0/3829759108 >> v2:172.21.15.178:3300/0 conn(0x7f57800af2e0 0x7f57801600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:47.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.324+0000 7f577fe00640 1 -- 172.21.15.159:0/3829759108 >> v1:172.21.15.178:6789/0 conn(0x7f5780153580 legacy=0x7f57800d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:47.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.324+0000 7f577fe00640 1 -- 172.21.15.159:0/3829759108 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578015cf20 con 0x7f57800d0990 2024-04-04T06:14:47.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.325+0000 7f577fe00640 1 --2- 172.21.15.159:0/3829759108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57800d0990 0x7f57800aeda0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f578006efa0 tx=0x7f576c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d84798ab27f5e3d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:47.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.325+0000 7f577f400640 1 -- 172.21.15.159:0/3829759108 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c02e070 con 0x7f57800d0990 2024-04-04T06:14:47.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.325+0000 7f577f400640 1 -- 172.21.15.159:0/3829759108 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f576c037070 con 0x7f57800d0990 2024-04-04T06:14:47.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.325+0000 7f577f400640 1 -- 172.21.15.159:0/3829759108 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c031a00 con 0x7f57800d0990 2024-04-04T06:14:47.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.326+0000 7f5787a00640 1 -- 172.21.15.159:0/3829759108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57800d0990 msgr2=0x7f57800aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:47.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.326+0000 7f5787a00640 1 --2- 172.21.15.159:0/3829759108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57800d0990 0x7f57800aeda0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f578006efa0 tx=0x7f576c02b880 comp rx=0 tx=0).stop 2024-04-04T06:14:47.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.326+0000 7f5787a00640 1 -- 172.21.15.159:0/3829759108 shutdown_connections 2024-04-04T06:14:47.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.326+0000 7f5787a00640 1 --2- 172.21.15.159:0/3829759108 >> v2:172.21.15.178:3300/0 conn(0x7f57800af2e0 0x7f57801600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:47.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.326+0000 7f5787a00640 1 --2- 172.21.15.159:0/3829759108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57800d0990 0x7f57800aeda0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:47.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.326+0000 7f5787a00640 1 -- 172.21.15.159:0/3829759108 >> 172.21.15.159:0/3829759108 conn(0x7f578006b750 msgr2=0x7f578006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:47.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.326+0000 7f5787a00640 1 -- 172.21.15.159:0/3829759108 shutdown_connections 2024-04-04T06:14:47.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.327+0000 7f5787a00640 1 -- 172.21.15.159:0/3829759108 wait complete. 2024-04-04T06:14:47.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.328+0000 7f5787a00640 1 Processor -- start 2024-04-04T06:14:47.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.328+0000 7f5787a00640 1 -- start start 2024-04-04T06:14:47.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.329+0000 7f5787a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57800af2e0 0x7f57800ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:47.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.329+0000 7f5787a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57800d0990 0x7f578013fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:47.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.329+0000 7f5787a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5780153580 0x7f5780141510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:47.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.329+0000 7f5787a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5780153960 con 0x7f57800d0990 2024-04-04T06:14:47.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.329+0000 7f5787a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f578015d4e0 con 0x7f57800af2e0 2024-04-04T06:14:47.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.329+0000 7f5787a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f57800af6c0 con 0x7f5780153580 2024-04-04T06:14:47.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.330+0000 7f577fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57800d0990 0x7f578013fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:47.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.330+0000 7f577fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57800d0990 0x7f578013fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37122/0 (socket says 172.21.15.159:37122) 2024-04-04T06:14:47.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.330+0000 7f577fe00640 1 -- 172.21.15.159:0/2520393826 learned_addr learned my addr 172.21.15.159:0/2520393826 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:47.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.330+0000 7f5785000640 1 --2- 172.21.15.159:0/2520393826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57800af2e0 0x7f57800ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:47.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.330+0000 7f577fe00640 1 -- 172.21.15.159:0/2520393826 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5780153580 msgr2=0x7f5780141510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:47.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.330+0000 7f577fe00640 1 --2- 172.21.15.159:0/2520393826 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5780153580 0x7f5780141510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:47.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.330+0000 7f577fe00640 1 -- 172.21.15.159:0/2520393826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57800af2e0 msgr2=0x7f57800ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:47.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.330+0000 7f577fe00640 1 --2- 172.21.15.159:0/2520393826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57800af2e0 0x7f57800ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:47.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.330+0000 7f577fe00640 1 -- 172.21.15.159:0/2520393826 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5780140600 con 0x7f57800d0990 2024-04-04T06:14:47.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.330+0000 7f5785000640 1 --2- 172.21.15.159:0/2520393826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57800af2e0 0x7f57800ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:47.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.330+0000 7f577fe00640 1 --2- 172.21.15.159:0/2520393826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57800d0990 0x7f578013fd30 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f576c000c00 tx=0x7f576c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:47.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.331+0000 7f577cc00640 1 -- 172.21.15.159:0/2520393826 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c032880 con 0x7f57800d0990 2024-04-04T06:14:47.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.331+0000 7f577cc00640 1 -- 172.21.15.159:0/2520393826 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f576c032a20 con 0x7f57800d0990 2024-04-04T06:14:47.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.331+0000 7f5787a00640 1 -- 172.21.15.159:0/2520393826 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57801408e0 con 0x7f57800d0990 2024-04-04T06:14:47.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.331+0000 7f577cc00640 1 -- 172.21.15.159:0/2520393826 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c032e10 con 0x7f57800d0990 2024-04-04T06:14:47.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.331+0000 7f5787a00640 1 -- 172.21.15.159:0/2520393826 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5780141180 con 0x7f57800d0990 2024-04-04T06:14:47.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.332+0000 7f5787a00640 1 -- 172.21.15.159:0/2520393826 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5780140cd0 con 0x7f57800d0990 2024-04-04T06:14:47.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.333+0000 7f577cc00640 1 -- 172.21.15.159:0/2520393826 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f576c04c060 con 0x7f57800d0990 2024-04-04T06:14:47.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.334+0000 7f577cc00640 1 --2- 172.21.15.159:0/2520393826 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f576006c8f0 0x7f576006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:47.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.334+0000 7f577cc00640 1 -- 172.21.15.159:0/2520393826 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f576c0ad420 con 0x7f57800d0990 2024-04-04T06:14:47.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.335+0000 7f5785000640 1 --2- 172.21.15.159:0/2520393826 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f576006c8f0 0x7f576006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:47.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.336+0000 7f5785000640 1 --2- 172.21.15.159:0/2520393826 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f576006c8f0 0x7f576006edb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f57800cba20 tx=0x7f5774009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:47.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.338+0000 7f577cc00640 1 -- 172.21.15.159:0/2520393826 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f576c040070 con 0x7f57800d0990 2024-04-04T06:14:47.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.629+0000 7f5787a00640 1 -- 172.21.15.159:0/2520393826 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f578015fa70 con 0x7f57800d0990 2024-04-04T06:14:47.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.630+0000 7f577cc00640 1 -- 172.21.15.159:0/2520393826 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f578015fa70 con 0x7f57800d0990 2024-04-04T06:14:47.624 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:14:47.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.633+0000 7f5787a00640 1 -- 172.21.15.159:0/2520393826 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f576006c8f0 msgr2=0x7f576006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:47.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.633+0000 7f5787a00640 1 --2- 172.21.15.159:0/2520393826 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f576006c8f0 0x7f576006edb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f57800cba20 tx=0x7f5774009040 comp rx=0 tx=0).stop 2024-04-04T06:14:47.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.633+0000 7f5787a00640 1 -- 172.21.15.159:0/2520393826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57800d0990 msgr2=0x7f578013fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:47.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.633+0000 7f5787a00640 1 --2- 172.21.15.159:0/2520393826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57800d0990 0x7f578013fd30 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f576c000c00 tx=0x7f576c0046c0 comp rx=0 tx=0).stop 2024-04-04T06:14:47.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.633+0000 7f5787a00640 1 -- 172.21.15.159:0/2520393826 shutdown_connections 2024-04-04T06:14:47.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.633+0000 7f5787a00640 1 --2- 172.21.15.159:0/2520393826 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5780153580 0x7f5780141510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:47.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.633+0000 7f5787a00640 1 --2- 172.21.15.159:0/2520393826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57800d0990 0x7f578013fd30 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:47.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.633+0000 7f5787a00640 1 --2- 172.21.15.159:0/2520393826 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f576006c8f0 0x7f576006edb0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:47.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.633+0000 7f5787a00640 1 --2- 172.21.15.159:0/2520393826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57800af2e0 0x7f57800ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:47.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.633+0000 7f5787a00640 1 -- 172.21.15.159:0/2520393826 >> 172.21.15.159:0/2520393826 conn(0x7f578006b750 msgr2=0x7f578015ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:47.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.633+0000 7f5787a00640 1 -- 172.21.15.159:0/2520393826 shutdown_connections 2024-04-04T06:14:47.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.633+0000 7f5787a00640 1 -- 172.21.15.159:0/2520393826 wait complete. 2024-04-04T06:14:47.647 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-04T06:14:47.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.857+0000 7fe3af200640 1 Processor -- start 2024-04-04T06:14:47.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.857+0000 7fe3af200640 1 -- start start 2024-04-04T06:14:47.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.859+0000 7fe3af200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe3a8153580 0x7fe3a80d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:47.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.859+0000 7fe3af200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3a80afab0 con 0x7fe3a80af4b0 2024-04-04T06:14:47.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.859+0000 7fe3af200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3a815cc80 con 0x7fe3a80d07c0 2024-04-04T06:14:47.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.859+0000 7fe3af200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe3a80d0c30 con 0x7fe3a8153580 2024-04-04T06:14:47.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.859+0000 7fe3a7e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe3a8153580 0x7fe3a80d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:47.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.859+0000 7fe3a7e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe3a8153580 0x7fe3a80d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37130/0 (socket says 172.21.15.159:37130) 2024-04-04T06:14:47.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.859+0000 7fe3a7e00640 1 -- 172.21.15.159:0/77630921 learned_addr learned my addr 172.21.15.159:0/77630921 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:47.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.859+0000 7fe3ad200640 1 --1- 172.21.15.159:0/77630921 >> v1:172.21.15.159:6789/0 conn(0x7fe3a80af4b0 0x7fe3a81600d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:36488/0 (socket says 172.21.15.159:36488) 2024-04-04T06:14:47.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.859+0000 7fe3a7e00640 1 -- 172.21.15.159:0/77630921 >> v1:172.21.15.178:6789/0 conn(0x7fe3a80d07c0 legacy=0x7fe3a80aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:47.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.859+0000 7fe3a7e00640 1 -- 172.21.15.159:0/77630921 >> v1:172.21.15.159:6789/0 conn(0x7fe3a80af4b0 legacy=0x7fe3a81600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:47.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.860+0000 7fe3a7e00640 1 -- 172.21.15.159:0/77630921 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a815da00 con 0x7fe3a8153580 2024-04-04T06:14:47.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.860+0000 7fe3a6a00640 1 -- 172.21.15.159:0/77630921 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 522788408 0 0) 0x7fe3a80afab0 con 0x7fe3a80af4b0 2024-04-04T06:14:47.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.861+0000 7fe3a7e00640 1 --2- 172.21.15.159:0/77630921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3a8153580 0x7fe3a80d0280 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fe398009870 tx=0x7fe39802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b521cb0f26b02652 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:47.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.861+0000 7fe3a6a00640 1 -- 172.21.15.159:0/77630921 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe398035020 con 0x7fe3a8153580 2024-04-04T06:14:47.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.861+0000 7fe3a6a00640 1 -- 172.21.15.159:0/77630921 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe398031870 con 0x7fe3a8153580 2024-04-04T06:14:47.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.861+0000 7fe3a6a00640 1 -- 172.21.15.159:0/77630921 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe398031c40 con 0x7fe3a8153580 2024-04-04T06:14:47.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.862+0000 7fe3af200640 1 -- 172.21.15.159:0/77630921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3a8153580 msgr2=0x7fe3a80d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:47.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.862+0000 7fe3af200640 1 --2- 172.21.15.159:0/77630921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3a8153580 0x7fe3a80d0280 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fe398009870 tx=0x7fe39802b910 comp rx=0 tx=0).stop 2024-04-04T06:14:47.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.862+0000 7fe3af200640 1 -- 172.21.15.159:0/77630921 shutdown_connections 2024-04-04T06:14:47.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.862+0000 7fe3af200640 1 --2- 172.21.15.159:0/77630921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3a8153580 0x7fe3a80d0280 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:47.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.862+0000 7fe3af200640 1 -- 172.21.15.159:0/77630921 >> 172.21.15.159:0/77630921 conn(0x7fe3a806b750 msgr2=0x7fe3a806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:47.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.862+0000 7fe3af200640 1 -- 172.21.15.159:0/77630921 shutdown_connections 2024-04-04T06:14:47.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.862+0000 7fe3af200640 1 -- 172.21.15.159:0/77630921 wait complete. 2024-04-04T06:14:47.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.863+0000 7fe3af200640 1 Processor -- start 2024-04-04T06:14:47.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.864+0000 7fe3af200640 1 -- start start 2024-04-04T06:14:47.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.864+0000 7fe3af200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3a80af4b0 0x7fe3a80cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:47.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.864+0000 7fe3af200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3a80d07c0 0x7fe3a814a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:47.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.865+0000 7fe3af200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3a8153580 0x7fe3a814a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:47.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.865+0000 7fe3af200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a815d4e0 con 0x7fe3a8153580 2024-04-04T06:14:47.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.865+0000 7fe3af200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a815d200 con 0x7fe3a80d07c0 2024-04-04T06:14:47.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.865+0000 7fe3af200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a8153960 con 0x7fe3a80af4b0 2024-04-04T06:14:47.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.865+0000 7fe3ad200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3a8153580 0x7fe3a814a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:47.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.865+0000 7fe3ad200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3a8153580 0x7fe3a814a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37144/0 (socket says 172.21.15.159:37144) 2024-04-04T06:14:47.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.865+0000 7fe3ad200640 1 -- 172.21.15.159:0/1395003254 learned_addr learned my addr 172.21.15.159:0/1395003254 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:47.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.865+0000 7fe3a7400640 1 --2- 172.21.15.159:0/1395003254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3a80d07c0 0x7fe3a814a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:47.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.865+0000 7fe3a7e00640 1 --2- 172.21.15.159:0/1395003254 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3a80af4b0 0x7fe3a80cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:47.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.865+0000 7fe3ad200640 1 -- 172.21.15.159:0/1395003254 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3a80af4b0 msgr2=0x7fe3a80cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:47.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.865+0000 7fe3ad200640 1 --2- 172.21.15.159:0/1395003254 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3a80af4b0 0x7fe3a80cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:47.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.865+0000 7fe3ad200640 1 -- 172.21.15.159:0/1395003254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3a80d07c0 msgr2=0x7fe3a814a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:47.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.865+0000 7fe3ad200640 1 --2- 172.21.15.159:0/1395003254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3a80d07c0 0x7fe3a814a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:47.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.865+0000 7fe3ad200640 1 -- 172.21.15.159:0/1395003254 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a81f4e80 con 0x7fe3a8153580 2024-04-04T06:14:47.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.866+0000 7fe3ad200640 1 --2- 172.21.15.159:0/1395003254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3a8153580 0x7fe3a814a570 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fe3a000c700 tx=0x7fe3a000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:47.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.866+0000 7fe39fe00640 1 -- 172.21.15.159:0/1395003254 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0018070 con 0x7fe3a8153580 2024-04-04T06:14:47.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.866+0000 7fe39fe00640 1 -- 172.21.15.159:0/1395003254 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3a000ce10 con 0x7fe3a8153580 2024-04-04T06:14:47.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.866+0000 7fe3af200640 1 -- 172.21.15.159:0/1395003254 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a81f4ba0 con 0x7fe3a8153580 2024-04-04T06:14:47.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.866+0000 7fe39fe00640 1 -- 172.21.15.159:0/1395003254 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a000a860 con 0x7fe3a8153580 2024-04-04T06:14:47.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.866+0000 7fe3af200640 1 -- 172.21.15.159:0/1395003254 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a8071ca0 con 0x7fe3a8153580 2024-04-04T06:14:47.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.867+0000 7fe3af200640 1 -- 172.21.15.159:0/1395003254 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a80685c0 con 0x7fe3a8153580 2024-04-04T06:14:47.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.868+0000 7fe39fe00640 1 -- 172.21.15.159:0/1395003254 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3a001b070 con 0x7fe3a8153580 2024-04-04T06:14:47.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.869+0000 7fe39fe00640 1 --2- 172.21.15.159:0/1395003254 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe38806c8f0 0x7fe38806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:47.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.869+0000 7fe39fe00640 1 -- 172.21.15.159:0/1395003254 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7fe3a008b1f0 con 0x7fe3a8153580 2024-04-04T06:14:47.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.870+0000 7fe3a7e00640 1 --2- 172.21.15.159:0/1395003254 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe38806c8f0 0x7fe38806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:47.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.871+0000 7fe3a7e00640 1 --2- 172.21.15.159:0/1395003254 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe38806c8f0 0x7fe38806edb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe398036000 tx=0x7fe398008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:47.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:47.879+0000 7fe39fe00640 1 -- 172.21.15.159:0/1395003254 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe3a0059520 con 0x7fe3a8153580 2024-04-04T06:14:48.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.093+0000 7fe3af200640 1 -- 172.21.15.159:0/1395003254 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe3a806eca0 con 0x7fe3a8153580 2024-04-04T06:14:48.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.094+0000 7fe39fe00640 1 -- 172.21.15.159:0/1395003254 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v89) ==== 80+0+248 (secure 0 0 0) 0x7fe3a0058e20 con 0x7fe3a8153580 2024-04-04T06:14:48.088 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6835/3796123128 2024-04-05T06:14:04.756982+0000 2024-04-04T06:14:48.088 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6833/3796123128 2024-04-05T06:14:04.756982+0000 2024-04-04T06:14:48.088 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6835/3484779950 2024-04-05T06:12:49.205338+0000 2024-04-04T06:14:48.088 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6833/3484779950 2024-04-05T06:12:49.205338+0000 2024-04-04T06:14:48.089 INFO:teuthology.orchestra.run.smithi159.stderr:listed 4 entries 2024-04-04T06:14:48.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.097+0000 7fe3af200640 1 -- 172.21.15.159:0/1395003254 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe38806c8f0 msgr2=0x7fe38806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:48.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.097+0000 7fe3af200640 1 --2- 172.21.15.159:0/1395003254 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe38806c8f0 0x7fe38806edb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe398036000 tx=0x7fe398008030 comp rx=0 tx=0).stop 2024-04-04T06:14:48.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.097+0000 7fe3af200640 1 -- 172.21.15.159:0/1395003254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3a8153580 msgr2=0x7fe3a814a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:48.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.097+0000 7fe3af200640 1 --2- 172.21.15.159:0/1395003254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3a8153580 0x7fe3a814a570 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fe3a000c700 tx=0x7fe3a000cbc0 comp rx=0 tx=0).stop 2024-04-04T06:14:48.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.097+0000 7fe3af200640 1 -- 172.21.15.159:0/1395003254 shutdown_connections 2024-04-04T06:14:48.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.097+0000 7fe3af200640 1 --2- 172.21.15.159:0/1395003254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3a8153580 0x7fe3a814a570 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:48.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.097+0000 7fe3af200640 1 --2- 172.21.15.159:0/1395003254 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe38806c8f0 0x7fe38806edb0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:48.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.097+0000 7fe3af200640 1 --2- 172.21.15.159:0/1395003254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3a80d07c0 0x7fe3a814a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:48.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.097+0000 7fe3af200640 1 --2- 172.21.15.159:0/1395003254 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3a80af4b0 0x7fe3a80cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:48.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.097+0000 7fe3af200640 1 -- 172.21.15.159:0/1395003254 >> 172.21.15.159:0/1395003254 conn(0x7fe3a806b750 msgr2=0x7fe3a80afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:48.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.097+0000 7fe3af200640 1 -- 172.21.15.159:0/1395003254 shutdown_connections 2024-04-04T06:14:48.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.097+0000 7fe3af200640 1 -- 172.21.15.159:0/1395003254 wait complete. 2024-04-04T06:14:48.113 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-04T06:14:48.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:14:48.113 DEBUG:teuthology.orchestra.run.smithi159:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-04T06:14:48.209 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:14:48.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:14:48.209 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-04T06:14:48.258 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:14:48.258 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:14:48.258 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:14:48.316 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:14:48.316 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:14:48.316 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:14:48.316 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:14:48.368 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:14:48.369 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:14:48.369 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:14:48.369 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:14:48.427 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:14:48.427 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:14:48.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.645+0000 7f4071600640 1 Processor -- start 2024-04-04T06:14:48.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.646+0000 7f4071600640 1 -- start start 2024-04-04T06:14:48.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.647+0000 7f4071600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f406c15f660 0x7f406c0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:48.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.647+0000 7f4071600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f406c0b3890 0x7f406c06b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:48.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.647+0000 7f4071600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f406c06bed0 con 0x7f406c06bab0 2024-04-04T06:14:48.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.647+0000 7f4071600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f406c0cde40 con 0x7f406c15f660 2024-04-04T06:14:48.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.647+0000 7f4071600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f406c0ce120 con 0x7f406c0b3890 2024-04-04T06:14:48.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.648+0000 7f406be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f406c15f660 0x7f406c0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:48.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.648+0000 7f4070c00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f406c06bab0 0x7f406c15fdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:36504/0 (socket says 172.21.15.159:36504) 2024-04-04T06:14:48.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.648+0000 7f406be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f406c15f660 0x7f406c0cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37150/0 (socket says 172.21.15.159:37150) 2024-04-04T06:14:48.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.648+0000 7f4070c00640 1 -- 172.21.15.159:0/3621239865 learned_addr learned my addr 172.21.15.159:0/3621239865 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:48.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.648+0000 7f4063e00640 1 --2- 172.21.15.159:0/3621239865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f406c0b3890 0x7f406c06b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:48.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.648+0000 7f406be00640 1 -- 172.21.15.159:0/3621239865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f406c0b3890 msgr2=0x7f406c06b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:48.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.648+0000 7f406be00640 1 --2- 172.21.15.159:0/3621239865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f406c0b3890 0x7f406c06b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:48.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.648+0000 7f406be00640 1 -- 172.21.15.159:0/3621239865 >> v1:172.21.15.159:6789/0 conn(0x7f406c06bab0 legacy=0x7f406c15fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:48.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.648+0000 7f406b400640 1 -- 172.21.15.159:0/3621239865 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1126970278 0 0) 0x7f406c06bed0 con 0x7f406c06bab0 2024-04-04T06:14:48.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.648+0000 7f406be00640 1 -- 172.21.15.159:0/3621239865 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c0ce400 con 0x7f406c15f660 2024-04-04T06:14:48.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.650+0000 7f406be00640 1 --2- 172.21.15.159:0/3621239865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f406c15f660 0x7f406c0cd5f0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f405c0097a0 tx=0x7f405c02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dacc38f7c035612 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:48.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.650+0000 7f406b400640 1 -- 172.21.15.159:0/3621239865 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c038070 con 0x7f406c15f660 2024-04-04T06:14:48.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.650+0000 7f406b400640 1 -- 172.21.15.159:0/3621239865 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f405c031700 con 0x7f406c15f660 2024-04-04T06:14:48.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.650+0000 7f406b400640 1 -- 172.21.15.159:0/3621239865 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c031ad0 con 0x7f406c15f660 2024-04-04T06:14:48.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.651+0000 7f4071600640 1 -- 172.21.15.159:0/3621239865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f406c15f660 msgr2=0x7f406c0cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:48.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.651+0000 7f4071600640 1 --2- 172.21.15.159:0/3621239865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f406c15f660 0x7f406c0cd5f0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f405c0097a0 tx=0x7f405c02b7a0 comp rx=0 tx=0).stop 2024-04-04T06:14:48.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.651+0000 7f4071600640 1 -- 172.21.15.159:0/3621239865 shutdown_connections 2024-04-04T06:14:48.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.651+0000 7f4071600640 1 --2- 172.21.15.159:0/3621239865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f406c0b3890 0x7f406c06b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:48.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.651+0000 7f4071600640 1 --2- 172.21.15.159:0/3621239865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f406c15f660 0x7f406c0cd5f0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:48.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.651+0000 7f4071600640 1 -- 172.21.15.159:0/3621239865 >> 172.21.15.159:0/3621239865 conn(0x7f406c09aa20 msgr2=0x7f406c0b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:48.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.651+0000 7f4071600640 1 -- 172.21.15.159:0/3621239865 shutdown_connections 2024-04-04T06:14:48.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.651+0000 7f4071600640 1 -- 172.21.15.159:0/3621239865 wait complete. 2024-04-04T06:14:48.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.654+0000 7f4071600640 1 Processor -- start 2024-04-04T06:14:48.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.654+0000 7f4071600640 1 -- start start 2024-04-04T06:14:48.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.656+0000 7f4071600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f406c06bab0 0x7f406c1539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:48.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.656+0000 7f4071600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f406c0b3890 0x7f406c0d0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:48.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.656+0000 7f4071600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f406c15f660 0x7f406c0af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:48.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.656+0000 7f4071600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f406c15fa40 con 0x7f406c0b3890 2024-04-04T06:14:48.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.656+0000 7f4071600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f406c0cdb30 con 0x7f406c06bab0 2024-04-04T06:14:48.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.656+0000 7f4071600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f406c15fbc0 con 0x7f406c15f660 2024-04-04T06:14:48.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.656+0000 7f4063e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f406c0b3890 0x7f406c0d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:48.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.656+0000 7f4063e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f406c0b3890 0x7f406c0d0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37160/0 (socket says 172.21.15.159:37160) 2024-04-04T06:14:48.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.656+0000 7f4063e00640 1 -- 172.21.15.159:0/3542459761 learned_addr learned my addr 172.21.15.159:0/3542459761 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:48.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.656+0000 7f406be00640 1 --2- 172.21.15.159:0/3542459761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f406c06bab0 0x7f406c1539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:48.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.656+0000 7f4070c00640 1 --2- 172.21.15.159:0/3542459761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f406c15f660 0x7f406c0af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:48.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.656+0000 7f4063e00640 1 -- 172.21.15.159:0/3542459761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f406c15f660 msgr2=0x7f406c0af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:48.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.656+0000 7f4063e00640 1 --2- 172.21.15.159:0/3542459761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f406c15f660 0x7f406c0af340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:48.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.656+0000 7f4063e00640 1 -- 172.21.15.159:0/3542459761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f406c06bab0 msgr2=0x7f406c1539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:48.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.656+0000 7f4063e00640 1 --2- 172.21.15.159:0/3542459761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f406c06bab0 0x7f406c1539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:48.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.656+0000 7f4063e00640 1 -- 172.21.15.159:0/3542459761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c0d0c90 con 0x7f406c0b3890 2024-04-04T06:14:48.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.656+0000 7f406be00640 1 --2- 172.21.15.159:0/3542459761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f406c06bab0 0x7f406c1539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:48.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.657+0000 7f4063e00640 1 --2- 172.21.15.159:0/3542459761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f406c0b3890 0x7f406c0d0340 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f405400c7f0 tx=0x7f405400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:48.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.657+0000 7f4068c00640 1 -- 172.21.15.159:0/3542459761 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054015070 con 0x7f406c0b3890 2024-04-04T06:14:48.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.657+0000 7f4068c00640 1 -- 172.21.15.159:0/3542459761 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4054004240 con 0x7f406c0b3890 2024-04-04T06:14:48.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.657+0000 7f4068c00640 1 -- 172.21.15.159:0/3542459761 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054007620 con 0x7f406c0b3890 2024-04-04T06:14:48.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.657+0000 7f4071600640 1 -- 172.21.15.159:0/3542459761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406c0d0f70 con 0x7f406c0b3890 2024-04-04T06:14:48.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.657+0000 7f4071600640 1 -- 172.21.15.159:0/3542459761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f406c0d1530 con 0x7f406c0b3890 2024-04-04T06:14:48.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.658+0000 7f4071600640 1 -- 172.21.15.159:0/3542459761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f406c0ce920 con 0x7f406c0b3890 2024-04-04T06:14:48.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.659+0000 7f4068c00640 1 -- 172.21.15.159:0/3542459761 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f40540079d0 con 0x7f406c0b3890 2024-04-04T06:14:48.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.660+0000 7f4068c00640 1 --2- 172.21.15.159:0/3542459761 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f404806c8f0 0x7f404806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:48.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.660+0000 7f4068c00640 1 -- 172.21.15.159:0/3542459761 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f405408bdc0 con 0x7f406c0b3890 2024-04-04T06:14:48.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.661+0000 7f406be00640 1 --2- 172.21.15.159:0/3542459761 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f404806c8f0 0x7f404806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:48.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.662+0000 7f406be00640 1 --2- 172.21.15.159:0/3542459761 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f404806c8f0 0x7f404806edb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f405c035000 tx=0x7f405c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:48.659 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.665+0000 7f4068c00640 1 -- 172.21.15.159:0/3542459761 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f406c0ce920 con 0x7f406c0b3890 2024-04-04T06:14:48.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.907+0000 7f4071600640 1 -- 172.21.15.159:0/3542459761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f406c0732e0 con 0x7f406c0b3890 2024-04-04T06:14:48.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.908+0000 7f4068c00640 1 -- 172.21.15.159:0/3542459761 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5196 (secure 0 0 0) 0x7f405405a060 con 0x7f406c0b3890 2024-04-04T06:14:48.902 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:14:48.902 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":76,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1825651864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1825651864},{"type":"v1","addr":"172.21.15.178:6836","nonce":1825651864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1824578712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1824578712},{"type":"v1","addr":"172.21.15.178:6837","nonce":1824578712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6834/3377498298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3377498298},{"type":"v1","addr":"172.21.15.159:6834","nonce":3377498298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:12:35.880454+0000","modified":"2024-04-04T06:14:09.810244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7809},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7809":{"gid":7809,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6835/3555825575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":3555825575},{"type":"v1","addr":"172.21.15.159:6835","nonce":3555825575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7809,"qdb_cluster":[7809]},"id":6}]} 2024-04-04T06:14:48.903 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 76 2024-04-04T06:14:48.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.910+0000 7f4071600640 1 -- 172.21.15.159:0/3542459761 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f404806c8f0 msgr2=0x7f404806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:48.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.910+0000 7f4071600640 1 --2- 172.21.15.159:0/3542459761 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f404806c8f0 0x7f404806edb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f405c035000 tx=0x7f405c008030 comp rx=0 tx=0).stop 2024-04-04T06:14:48.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.910+0000 7f4071600640 1 -- 172.21.15.159:0/3542459761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f406c0b3890 msgr2=0x7f406c0d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:48.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.910+0000 7f4071600640 1 --2- 172.21.15.159:0/3542459761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f406c0b3890 0x7f406c0d0340 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f405400c7f0 tx=0x7f405400ccb0 comp rx=0 tx=0).stop 2024-04-04T06:14:48.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.910+0000 7f4071600640 1 -- 172.21.15.159:0/3542459761 shutdown_connections 2024-04-04T06:14:48.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.910+0000 7f4071600640 1 --2- 172.21.15.159:0/3542459761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f406c15f660 0x7f406c0af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:48.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.910+0000 7f4071600640 1 --2- 172.21.15.159:0/3542459761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f406c0b3890 0x7f406c0d0340 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:48.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.910+0000 7f4071600640 1 --2- 172.21.15.159:0/3542459761 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f404806c8f0 0x7f404806edb0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:48.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.910+0000 7f4071600640 1 --2- 172.21.15.159:0/3542459761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f406c06bab0 0x7f406c1539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:48.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.911+0000 7f4071600640 1 -- 172.21.15.159:0/3542459761 >> 172.21.15.159:0/3542459761 conn(0x7f406c09aa20 msgr2=0x7f406c15fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:48.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.911+0000 7f4071600640 1 -- 172.21.15.159:0/3542459761 shutdown_connections 2024-04-04T06:14:48.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:48.911+0000 7f4071600640 1 -- 172.21.15.159:0/3542459761 wait complete. 2024-04-04T06:14:48.924 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:14:49.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.134+0000 7fd9d6600640 1 Processor -- start 2024-04-04T06:14:49.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.135+0000 7fd9d6600640 1 -- start start 2024-04-04T06:14:49.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.136+0000 7fd9d6600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9d015c310 0x7fd9d00d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:49.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.136+0000 7fd9d6600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd9d00cf670 0x7fd9d006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:49.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.136+0000 7fd9d6600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9d00cfdf0 con 0x7fd9d006f010 2024-04-04T06:14:49.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.136+0000 7fd9d6600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd9d0072f00 con 0x7fd9d00cf670 2024-04-04T06:14:49.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.136+0000 7fd9d6600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd9d00731e0 con 0x7fd9d015c310 2024-04-04T06:14:49.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.137+0000 7fd9c7e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd9d00cf670 0x7fd9d006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:49.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.137+0000 7fd9d5200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9d015c310 0x7fd9d00d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:49.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.137+0000 7fd9c7e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd9d00cf670 0x7fd9d006ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:58724/0 (socket says 172.21.15.159:58724) 2024-04-04T06:14:49.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.137+0000 7fd9c7e00640 1 -- 172.21.15.159:0/3413398243 learned_addr learned my addr 172.21.15.159:0/3413398243 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:49.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.137+0000 7fd9d5200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9d015c310 0x7fd9d00d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:53856/0 (socket says 172.21.15.159:53856) 2024-04-04T06:14:49.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.138+0000 7fd9d5200640 1 -- 172.21.15.159:0/3413398243 >> v2:172.21.15.178:3300/0 conn(0x7fd9d00cf670 msgr2=0x7fd9d006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:49.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.138+0000 7fd9d5200640 1 --2- 172.21.15.159:0/3413398243 >> v2:172.21.15.178:3300/0 conn(0x7fd9d00cf670 0x7fd9d006ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.138+0000 7fd9d5200640 1 -- 172.21.15.159:0/3413398243 >> v1:172.21.15.178:6789/0 conn(0x7fd9d006f010 legacy=0x7fd9d015c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:49.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.138+0000 7fd9d5200640 1 -- 172.21.15.159:0/3413398243 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d00b04d0 con 0x7fd9d015c310 2024-04-04T06:14:49.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.138+0000 7fd9c7e00640 1 --2- 172.21.15.159:0/3413398243 >> v2:172.21.15.178:3300/0 conn(0x7fd9d00cf670 0x7fd9d006ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:14:49.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.139+0000 7fd9d5200640 1 --2- 172.21.15.159:0/3413398243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9d015c310 0x7fd9d00d03c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80097a0 tx=0x7fd9b802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa4658a137b2bd63 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:49.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.139+0000 7fd9cfe00640 1 -- 172.21.15.159:0/3413398243 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8035020 con 0x7fd9d015c310 2024-04-04T06:14:49.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.140+0000 7fd9cfe00640 1 -- 172.21.15.159:0/3413398243 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b8031810 con 0x7fd9d015c310 2024-04-04T06:14:49.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.140+0000 7fd9cfe00640 1 -- 172.21.15.159:0/3413398243 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8031be0 con 0x7fd9d015c310 2024-04-04T06:14:49.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.140+0000 7fd9d6600640 1 -- 172.21.15.159:0/3413398243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9d015c310 msgr2=0x7fd9d00d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:49.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.140+0000 7fd9d6600640 1 --2- 172.21.15.159:0/3413398243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9d015c310 0x7fd9d00d03c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80097a0 tx=0x7fd9b802b8b0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.140+0000 7fd9d6600640 1 -- 172.21.15.159:0/3413398243 shutdown_connections 2024-04-04T06:14:49.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.140+0000 7fd9d6600640 1 --2- 172.21.15.159:0/3413398243 >> v2:172.21.15.178:3300/0 conn(0x7fd9d00cf670 0x7fd9d006ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.140+0000 7fd9d6600640 1 --2- 172.21.15.159:0/3413398243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9d015c310 0x7fd9d00d03c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.140+0000 7fd9d6600640 1 -- 172.21.15.159:0/3413398243 >> 172.21.15.159:0/3413398243 conn(0x7fd9d0071e70 msgr2=0x7fd9d0072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:49.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.140+0000 7fd9d6600640 1 -- 172.21.15.159:0/3413398243 shutdown_connections 2024-04-04T06:14:49.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.140+0000 7fd9d6600640 1 -- 172.21.15.159:0/3413398243 wait complete. 2024-04-04T06:14:49.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.142+0000 7fd9d6600640 1 Processor -- start 2024-04-04T06:14:49.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.142+0000 7fd9d6600640 1 -- start start 2024-04-04T06:14:49.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.143+0000 7fd9d6600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9d006f010 0x7fd9d00c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:49.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.143+0000 7fd9d6600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9d00cf670 0x7fd9d014f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:49.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.143+0000 7fd9d6600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9d015c310 0x7fd9d00c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:49.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.143+0000 7fd9d6600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d015cf80 con 0x7fd9d006f010 2024-04-04T06:14:49.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.143+0000 7fd9d6600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d015ce00 con 0x7fd9d015c310 2024-04-04T06:14:49.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.143+0000 7fd9d6600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd9d00afcc0 con 0x7fd9d00cf670 2024-04-04T06:14:49.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.143+0000 7fd9d5200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9d006f010 0x7fd9d00c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:49.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.143+0000 7fd9d5200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9d006f010 0x7fd9d00c5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37162/0 (socket says 172.21.15.159:37162) 2024-04-04T06:14:49.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.143+0000 7fd9d5200640 1 -- 172.21.15.159:0/3547473264 learned_addr learned my addr 172.21.15.159:0/3547473264 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:49.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.144+0000 7fd9c7e00640 1 --2- 172.21.15.159:0/3547473264 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9d00cf670 0x7fd9d014f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:49.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.144+0000 7fd9d5c00640 1 --2- 172.21.15.159:0/3547473264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9d015c310 0x7fd9d00c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:49.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.144+0000 7fd9d5200640 1 -- 172.21.15.159:0/3547473264 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9d00cf670 msgr2=0x7fd9d014f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:49.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.144+0000 7fd9d5200640 1 --2- 172.21.15.159:0/3547473264 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9d00cf670 0x7fd9d014f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.144+0000 7fd9d5200640 1 -- 172.21.15.159:0/3547473264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9d015c310 msgr2=0x7fd9d00c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:49.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.144+0000 7fd9d5200640 1 --2- 172.21.15.159:0/3547473264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9d015c310 0x7fd9d00c6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.144+0000 7fd9d5200640 1 -- 172.21.15.159:0/3547473264 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d00c5f70 con 0x7fd9d006f010 2024-04-04T06:14:49.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.144+0000 7fd9d5c00640 1 --2- 172.21.15.159:0/3547473264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9d015c310 0x7fd9d00c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:49.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.144+0000 7fd9d5200640 1 --2- 172.21.15.159:0/3547473264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9d006f010 0x7fd9d00c5a30 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8034040 tx=0x7fd9b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:49.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.144+0000 7fd9cd600640 1 -- 172.21.15.159:0/3547473264 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8006e10 con 0x7fd9d006f010 2024-04-04T06:14:49.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.144+0000 7fd9cd600640 1 -- 172.21.15.159:0/3547473264 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b803bd70 con 0x7fd9d006f010 2024-04-04T06:14:49.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.145+0000 7fd9d6600640 1 -- 172.21.15.159:0/3547473264 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d00af0b0 con 0x7fd9d006f010 2024-04-04T06:14:49.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.145+0000 7fd9cd600640 1 -- 172.21.15.159:0/3547473264 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8045630 con 0x7fd9d006f010 2024-04-04T06:14:49.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.145+0000 7fd9d6600640 1 -- 172.21.15.159:0/3547473264 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d00af670 con 0x7fd9d006f010 2024-04-04T06:14:49.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.146+0000 7fd9d6600640 1 -- 172.21.15.159:0/3547473264 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d0158a70 con 0x7fd9d006f010 2024-04-04T06:14:49.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.147+0000 7fd9cd600640 1 -- 172.21.15.159:0/3547473264 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd9b804c060 con 0x7fd9d006f010 2024-04-04T06:14:49.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.148+0000 7fd9cd600640 1 --2- 172.21.15.159:0/3547473264 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd9ac06c8f0 0x7fd9ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:49.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.148+0000 7fd9cd600640 1 -- 172.21.15.159:0/3547473264 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7fd9b80adb50 con 0x7fd9d006f010 2024-04-04T06:14:49.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.148+0000 7fd9c7e00640 1 --2- 172.21.15.159:0/3547473264 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd9ac06c8f0 0x7fd9ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:49.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.150+0000 7fd9c7e00640 1 --2- 172.21.15.159:0/3547473264 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd9ac06c8f0 0x7fd9ac06edb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00cd3f0 tx=0x7fd9c0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:49.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.157+0000 7fd9cd600640 1 -- 172.21.15.159:0/3547473264 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd9b8040070 con 0x7fd9d006f010 2024-04-04T06:14:49.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.359+0000 7fd9d6600640 1 -- 172.21.15.159:0/3547473264 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9d006fa40 con 0x7fd9d006f010 2024-04-04T06:14:49.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.361+0000 7fd9cd600640 1 -- 172.21.15.159:0/3547473264 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+17033 (secure 0 0 0) 0x7fd9b807be80 con 0x7fd9d006f010 2024-04-04T06:14:49.355 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:14:49.355 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":89,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:14:04.757031+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:12:33.658405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-04T06:12:34.856266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.3d","mappings":[{"from":6,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6835/3796123128":"2024-04-05T06:14:04.756982+0000","172.21.15.159:6833/3796123128":"2024-04-05T06:14:04.756982+0000","172.21.15.159:6835/3484779950":"2024-04-05T06:12:49.205338+0000","172.21.15.159:6833/3484779950":"2024-04-05T06:12:49.205338+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:14:49.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.363+0000 7fd9d6600640 1 -- 172.21.15.159:0/3547473264 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd9ac06c8f0 msgr2=0x7fd9ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:49.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.363+0000 7fd9d6600640 1 --2- 172.21.15.159:0/3547473264 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd9ac06c8f0 0x7fd9ac06edb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00cd3f0 tx=0x7fd9c0008040 comp rx=0 tx=0).stop 2024-04-04T06:14:49.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.363+0000 7fd9d6600640 1 -- 172.21.15.159:0/3547473264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9d006f010 msgr2=0x7fd9d00c5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:49.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.363+0000 7fd9d6600640 1 --2- 172.21.15.159:0/3547473264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9d006f010 0x7fd9d00c5a30 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8034040 tx=0x7fd9b80046c0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.363+0000 7fd9d6600640 1 -- 172.21.15.159:0/3547473264 shutdown_connections 2024-04-04T06:14:49.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.363+0000 7fd9d6600640 1 --2- 172.21.15.159:0/3547473264 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd9ac06c8f0 0x7fd9ac06edb0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.363+0000 7fd9d6600640 1 --2- 172.21.15.159:0/3547473264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9d015c310 0x7fd9d00c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.363+0000 7fd9d6600640 1 --2- 172.21.15.159:0/3547473264 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9d00cf670 0x7fd9d014f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.363+0000 7fd9d6600640 1 --2- 172.21.15.159:0/3547473264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9d006f010 0x7fd9d00c5a30 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.363+0000 7fd9d6600640 1 -- 172.21.15.159:0/3547473264 >> 172.21.15.159:0/3547473264 conn(0x7fd9d0071e70 msgr2=0x7fd9d006e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:49.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.363+0000 7fd9d6600640 1 -- 172.21.15.159:0/3547473264 shutdown_connections 2024-04-04T06:14:49.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.364+0000 7fd9d6600640 1 -- 172.21.15.159:0/3547473264 wait complete. 2024-04-04T06:14:49.376 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:12:33.658405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:12:34.856266+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:14:49.376 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:14:49.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.596+0000 7f520c600640 1 Processor -- start 2024-04-04T06:14:49.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.596+0000 7f520c600640 1 -- start start 2024-04-04T06:14:49.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.597+0000 7f520c600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f52040d1ce0 0x7f52040b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:49.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.598+0000 7f520c600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52040d0080 con 0x7f520415e7b0 2024-04-04T06:14:49.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.598+0000 7f520c600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52040d0360 con 0x7f52040c7810 2024-04-04T06:14:49.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.598+0000 7f520c600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f52040cfe30 con 0x7f52040d1ce0 2024-04-04T06:14:49.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.598+0000 7f5209c00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f520415e7b0 0x7f52040c7100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:36520/0 (socket says 172.21.15.159:36520) 2024-04-04T06:14:49.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.598+0000 7f5209c00640 1 -- 172.21.15.159:0/3900256565 learned_addr learned my addr 172.21.15.159:0/3900256565 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:49.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.598+0000 7f520a600640 1 --2- 172.21.15.159:0/3900256565 >> v2:172.21.15.178:3300/0 conn(0x7f52040d1ce0 0x7f52040b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:49.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.598+0000 7f5203e00640 1 -- 172.21.15.159:0/3900256565 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 593603434 0 0) 0x7f52040d0080 con 0x7f520415e7b0 2024-04-04T06:14:49.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.598+0000 7f5203e00640 1 -- 172.21.15.159:0/3900256565 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51ec003520 con 0x7f520415e7b0 2024-04-04T06:14:49.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.599+0000 7f5203e00640 1 -- 172.21.15.159:0/3900256565 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 26142533 0 0) 0x7f51ec003520 con 0x7f520415e7b0 2024-04-04T06:14:49.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.599+0000 7f5203e00640 1 -- 172.21.15.159:0/3900256565 >> v2:172.21.15.178:3300/0 conn(0x7f52040d1ce0 msgr2=0x7f52040b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:49.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.599+0000 7f5203e00640 1 --2- 172.21.15.159:0/3900256565 >> v2:172.21.15.178:3300/0 conn(0x7f52040d1ce0 0x7f52040b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.599+0000 7f5203e00640 1 -- 172.21.15.159:0/3900256565 >> v1:172.21.15.178:6789/0 conn(0x7f52040c7810 legacy=0x7f520406ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:49.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.599+0000 7f5203e00640 1 -- 172.21.15.159:0/3900256565 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52040d0640 con 0x7f520415e7b0 2024-04-04T06:14:49.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.599+0000 7f5203e00640 1 -- 172.21.15.159:0/3900256565 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f51f8001670 con 0x7f520415e7b0 2024-04-04T06:14:49.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.599+0000 7f5203e00640 1 -- 172.21.15.159:0/3900256565 >> v1:172.21.15.159:6789/0 conn(0x7f520415e7b0 legacy=0x7f52040c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:49.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.599+0000 7f5203e00640 1 --2- 172.21.15.159:0/3900256565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51ec003a00 0x7f51ec003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:49.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.599+0000 7f5203e00640 1 --2- 172.21.15.159:0/3900256565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51ec004800 0x7f51ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:49.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.599+0000 7f5203e00640 1 --2- 172.21.15.159:0/3900256565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51ec0051c0 0x7f51ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:49.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.599+0000 7f5203e00640 1 -- 172.21.15.159:0/3900256565 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f52040cfe30 con 0x7f51ec0051c0 2024-04-04T06:14:49.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.599+0000 7f5203e00640 1 -- 172.21.15.159:0/3900256565 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec00c0c0 con 0x7f51ec003a00 2024-04-04T06:14:49.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.599+0000 7f5203e00640 1 -- 172.21.15.159:0/3900256565 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f51ec00c2c0 con 0x7f51ec004800 2024-04-04T06:14:49.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.600+0000 7f520a600640 1 --2- 172.21.15.159:0/3900256565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51ec0051c0 0x7f51ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:49.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.600+0000 7f5209c00640 1 --2- 172.21.15.159:0/3900256565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51ec003a00 0x7f51ec003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:49.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.600+0000 7f5209200640 1 --2- 172.21.15.159:0/3900256565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51ec004800 0x7f51ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:49.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.600+0000 7f520a600640 1 -- 172.21.15.159:0/3900256565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51ec004800 msgr2=0x7f51ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:49.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.600+0000 7f520a600640 1 --2- 172.21.15.159:0/3900256565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51ec004800 0x7f51ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.600+0000 7f520a600640 1 -- 172.21.15.159:0/3900256565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51ec003a00 msgr2=0x7f51ec003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:49.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.600+0000 7f520a600640 1 --2- 172.21.15.159:0/3900256565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51ec003a00 0x7f51ec003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.600+0000 7f520a600640 1 -- 172.21.15.159:0/3900256565 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ec003520 con 0x7f51ec0051c0 2024-04-04T06:14:49.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.600+0000 7f5209200640 1 --2- 172.21.15.159:0/3900256565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f51ec004800 0x7f51ec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:14:49.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.602+0000 7f520a600640 1 -- 172.21.15.159:0/3900256565 reap_dead start 2024-04-04T06:14:49.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.602+0000 7f520a600640 1 --2- 172.21.15.159:0/3900256565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51ec0051c0 0x7f51ec00b9c0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009fc0 tx=0x7f51ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da4b1821872e059 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:49.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.603+0000 7f5203e00640 1 -- 172.21.15.159:0/3900256565 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc009c10 con 0x7f51ec0051c0 2024-04-04T06:14:49.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.603+0000 7f5203e00640 1 -- 172.21.15.159:0/3900256565 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51fc02cca0 con 0x7f51ec0051c0 2024-04-04T06:14:49.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.603+0000 7f5203e00640 1 -- 172.21.15.159:0/3900256565 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc003bd0 con 0x7f51ec0051c0 2024-04-04T06:14:49.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.603+0000 7f520c600640 1 -- 172.21.15.159:0/3900256565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51ec0051c0 msgr2=0x7f51ec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:49.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.603+0000 7f520c600640 1 --2- 172.21.15.159:0/3900256565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51ec0051c0 0x7f51ec00b9c0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009fc0 tx=0x7f51ec005830 comp rx=0 tx=0).stop 2024-04-04T06:14:49.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.604+0000 7f520c600640 1 -- 172.21.15.159:0/3900256565 shutdown_connections 2024-04-04T06:14:49.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.604+0000 7f520c600640 1 --2- 172.21.15.159:0/3900256565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51ec0051c0 0x7f51ec00b9c0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.604+0000 7f520c600640 1 -- 172.21.15.159:0/3900256565 >> 172.21.15.159:0/3900256565 conn(0x7f52040b3e90 msgr2=0x7f52040b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:49.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.604+0000 7f520c600640 1 -- 172.21.15.159:0/3900256565 shutdown_connections 2024-04-04T06:14:49.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.604+0000 7f520c600640 1 -- 172.21.15.159:0/3900256565 wait complete. 2024-04-04T06:14:49.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.605+0000 7f520c600640 1 Processor -- start 2024-04-04T06:14:49.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.606+0000 7f520c600640 1 -- start start 2024-04-04T06:14:49.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.606+0000 7f520c600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51ec0051c0 0x7f52040ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:49.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.606+0000 7f520c600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52040af460 0x7f5204159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:49.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.606+0000 7f520c600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52040af8e0 0x7f520415a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:49.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.606+0000 7f520c600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f52040d1120 con 0x7f51ec0051c0 2024-04-04T06:14:49.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.606+0000 7f520c600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f52040d0c00 con 0x7f52040af8e0 2024-04-04T06:14:49.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.606+0000 7f520c600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f52040b4980 con 0x7f52040af460 2024-04-04T06:14:49.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.607+0000 7f5209c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51ec0051c0 0x7f52040ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:49.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.607+0000 7f5209c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51ec0051c0 0x7f52040ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37182/0 (socket says 172.21.15.159:37182) 2024-04-04T06:14:49.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.607+0000 7f5209c00640 1 -- 172.21.15.159:0/153735598 learned_addr learned my addr 172.21.15.159:0/153735598 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:49.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.607+0000 7f5209c00640 1 -- 172.21.15.159:0/153735598 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52040af460 msgr2=0x7f5204159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:49.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.607+0000 7f5209c00640 1 --2- 172.21.15.159:0/153735598 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52040af460 0x7f5204159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.607+0000 7f5209c00640 1 -- 172.21.15.159:0/153735598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52040af8e0 msgr2=0x7f520415a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:49.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.607+0000 7f5209c00640 1 --2- 172.21.15.159:0/153735598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52040af8e0 0x7f520415a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.607+0000 7f5209c00640 1 -- 172.21.15.159:0/153735598 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52040afeb0 con 0x7f51ec0051c0 2024-04-04T06:14:49.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.607+0000 7f5209c00640 1 --2- 172.21.15.159:0/153735598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51ec0051c0 0x7f52040ceb90 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f51f80072c0 tx=0x7f51f80076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:49.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.608+0000 7f5201600640 1 -- 172.21.15.159:0/153735598 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f800f9c0 con 0x7f51ec0051c0 2024-04-04T06:14:49.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.608+0000 7f5201600640 1 -- 172.21.15.159:0/153735598 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f8018070 con 0x7f51ec0051c0 2024-04-04T06:14:49.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.608+0000 7f5201600640 1 -- 172.21.15.159:0/153735598 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f800fdc0 con 0x7f51ec0051c0 2024-04-04T06:14:49.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.608+0000 7f520c600640 1 -- 172.21.15.159:0/153735598 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52040b0190 con 0x7f51ec0051c0 2024-04-04T06:14:49.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.608+0000 7f520c600640 1 -- 172.21.15.159:0/153735598 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52040b0750 con 0x7f51ec0051c0 2024-04-04T06:14:49.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.609+0000 7f520c600640 1 -- 172.21.15.159:0/153735598 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5204068cd0 con 0x7f51ec0051c0 2024-04-04T06:14:49.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.610+0000 7f5201600640 1 -- 172.21.15.159:0/153735598 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51f801d460 con 0x7f51ec0051c0 2024-04-04T06:14:49.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.611+0000 7f5201600640 1 --2- 172.21.15.159:0/153735598 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f51e406c920 0x7f51e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:49.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.611+0000 7f5201600640 1 -- 172.21.15.159:0/153735598 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f51f808bc90 con 0x7f51ec0051c0 2024-04-04T06:14:49.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.612+0000 7f5209200640 1 --2- 172.21.15.159:0/153735598 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f51e406c920 0x7f51e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:49.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.613+0000 7f5209200640 1 --2- 172.21.15.159:0/153735598 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f51e406c920 0x7f51e406ede0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f520415b3a0 tx=0x7f51f0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:49.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.615+0000 7f5201600640 1 -- 172.21.15.159:0/153735598 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f51f8059fc0 con 0x7f51ec0051c0 2024-04-04T06:14:49.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.816+0000 7f520c600640 1 -- 172.21.15.159:0/153735598 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f520415b5b0 con 0x7f51ec0051c0 2024-04-04T06:14:49.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.817+0000 7f5201600640 1 -- 172.21.15.159:0/153735598 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f51f80598c0 con 0x7f51ec0051c0 2024-04-04T06:14:49.811 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:14:49.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.819+0000 7f520c600640 1 -- 172.21.15.159:0/153735598 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f51e406c920 msgr2=0x7f51e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:49.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.819+0000 7f520c600640 1 --2- 172.21.15.159:0/153735598 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f51e406c920 0x7f51e406ede0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f520415b3a0 tx=0x7f51f0009040 comp rx=0 tx=0).stop 2024-04-04T06:14:49.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.819+0000 7f520c600640 1 -- 172.21.15.159:0/153735598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51ec0051c0 msgr2=0x7f52040ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:49.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.819+0000 7f520c600640 1 --2- 172.21.15.159:0/153735598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51ec0051c0 0x7f52040ceb90 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f51f80072c0 tx=0x7f51f80076c0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.819+0000 7f520c600640 1 -- 172.21.15.159:0/153735598 shutdown_connections 2024-04-04T06:14:49.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.819+0000 7f520c600640 1 --2- 172.21.15.159:0/153735598 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f51e406c920 0x7f51e406ede0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.819+0000 7f520c600640 1 --2- 172.21.15.159:0/153735598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52040af8e0 0x7f520415a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.819+0000 7f520c600640 1 --2- 172.21.15.159:0/153735598 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52040af460 0x7f5204159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.819+0000 7f520c600640 1 --2- 172.21.15.159:0/153735598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51ec0051c0 0x7f52040ceb90 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:49.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.819+0000 7f520c600640 1 -- 172.21.15.159:0/153735598 >> 172.21.15.159:0/153735598 conn(0x7f52040b3e90 msgr2=0x7f5204071c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:49.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.819+0000 7f520c600640 1 -- 172.21.15.159:0/153735598 shutdown_connections 2024-04-04T06:14:49.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:49.819+0000 7f520c600640 1 -- 172.21.15.159:0/153735598 wait complete. 2024-04-04T06:14:49.832 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:14:50.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.045+0000 7f2e6ca00640 1 Processor -- start 2024-04-04T06:14:50.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.045+0000 7f2e6ca00640 1 -- start start 2024-04-04T06:14:50.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.047+0000 7f2e6ca00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2e680afaa0 0x7f2e6815d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:50.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.047+0000 7f2e6ca00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e680b3ee0 con 0x7f2e680cd180 2024-04-04T06:14:50.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.047+0000 7f2e6ca00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e680b41c0 con 0x7f2e680cd630 2024-04-04T06:14:50.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.047+0000 7f2e6ca00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f2e680b44a0 con 0x7f2e680afaa0 2024-04-04T06:14:50.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.047+0000 7f2e67400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2e680afaa0 0x7f2e6815d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:50.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.047+0000 7f2e5ea00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f2e680cd180 0x7f2e681529d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:36534/0 (socket says 172.21.15.159:36534) 2024-04-04T06:14:50.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.048+0000 7f2e67400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2e680afaa0 0x7f2e6815d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37198/0 (socket says 172.21.15.159:37198) 2024-04-04T06:14:50.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.048+0000 7f2e5ea00640 1 -- 172.21.15.159:0/522573682 learned_addr learned my addr 172.21.15.159:0/522573682 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:50.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.048+0000 7f2e67400640 1 -- 172.21.15.159:0/522573682 >> v1:172.21.15.178:6789/0 conn(0x7f2e680cd630 legacy=0x7f2e681530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:50.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.048+0000 7f2e67400640 1 -- 172.21.15.159:0/522573682 >> v1:172.21.15.159:6789/0 conn(0x7f2e680cd180 legacy=0x7f2e681529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:50.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.048+0000 7f2e67400640 1 -- 172.21.15.159:0/522573682 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e680b4780 con 0x7f2e680afaa0 2024-04-04T06:14:50.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.048+0000 7f2e66a00640 1 -- 172.21.15.159:0/522573682 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4206806147 0 0) 0x7f2e680b3ee0 con 0x7f2e680cd180 2024-04-04T06:14:50.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.049+0000 7f2e67400640 1 --2- 172.21.15.159:0/522573682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e680afaa0 0x7f2e6815d230 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f2e580097a0 tx=0x7f2e5802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6deb0445094ca12 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:50.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.050+0000 7f2e66a00640 1 -- 172.21.15.159:0/522573682 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e580315a0 con 0x7f2e680afaa0 2024-04-04T06:14:50.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.050+0000 7f2e66a00640 1 -- 172.21.15.159:0/522573682 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e58037070 con 0x7f2e680afaa0 2024-04-04T06:14:50.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.050+0000 7f2e66a00640 1 -- 172.21.15.159:0/522573682 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e58031960 con 0x7f2e680afaa0 2024-04-04T06:14:50.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.050+0000 7f2e6ca00640 1 -- 172.21.15.159:0/522573682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e680afaa0 msgr2=0x7f2e6815d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:50.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.050+0000 7f2e6ca00640 1 --2- 172.21.15.159:0/522573682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e680afaa0 0x7f2e6815d230 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f2e580097a0 tx=0x7f2e5802b840 comp rx=0 tx=0).stop 2024-04-04T06:14:50.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.050+0000 7f2e6ca00640 1 -- 172.21.15.159:0/522573682 shutdown_connections 2024-04-04T06:14:50.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.050+0000 7f2e6ca00640 1 --2- 172.21.15.159:0/522573682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e680afaa0 0x7f2e6815d230 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:50.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.050+0000 7f2e6ca00640 1 -- 172.21.15.159:0/522573682 >> 172.21.15.159:0/522573682 conn(0x7f2e680d0770 msgr2=0x7f2e680d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:50.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.050+0000 7f2e6ca00640 1 -- 172.21.15.159:0/522573682 shutdown_connections 2024-04-04T06:14:50.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.050+0000 7f2e6ca00640 1 -- 172.21.15.159:0/522573682 wait complete. 2024-04-04T06:14:50.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.053+0000 7f2e6ca00640 1 Processor -- start 2024-04-04T06:14:50.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.053+0000 7f2e6ca00640 1 -- start start 2024-04-04T06:14:50.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.055+0000 7f2e6ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e680afaa0 0x7f2e68142be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:50.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.055+0000 7f2e6ca00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e680cd180 0x7f2e68155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:50.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.055+0000 7f2e6ca00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e680cd630 0x7f2e6813f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:50.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.055+0000 7f2e6ca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2e680b4a60 con 0x7f2e680afaa0 2024-04-04T06:14:50.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.055+0000 7f2e6ca00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2e680b3b90 con 0x7f2e680cd180 2024-04-04T06:14:50.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.055+0000 7f2e6ca00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2e680afe80 con 0x7f2e680cd630 2024-04-04T06:14:50.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.055+0000 7f2e67400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e680afaa0 0x7f2e68142be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:50.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.055+0000 7f2e67400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e680afaa0 0x7f2e68142be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37200/0 (socket says 172.21.15.159:37200) 2024-04-04T06:14:50.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.055+0000 7f2e67400640 1 -- 172.21.15.159:0/1197881282 learned_addr learned my addr 172.21.15.159:0/1197881282 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:50.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.056+0000 7f2e5ea00640 1 --2- 172.21.15.159:0/1197881282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e680cd180 0x7f2e68155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:50.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.056+0000 7f2e67e00640 1 --2- 172.21.15.159:0/1197881282 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e680cd630 0x7f2e6813f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:50.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.056+0000 7f2e67400640 1 -- 172.21.15.159:0/1197881282 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e680cd630 msgr2=0x7f2e6813f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:50.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.056+0000 7f2e67400640 1 --2- 172.21.15.159:0/1197881282 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e680cd630 0x7f2e6813f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:50.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.056+0000 7f2e67400640 1 -- 172.21.15.159:0/1197881282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e680cd180 msgr2=0x7f2e68155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:50.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.056+0000 7f2e67400640 1 --2- 172.21.15.159:0/1197881282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e680cd180 0x7f2e68155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:50.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.056+0000 7f2e67400640 1 -- 172.21.15.159:0/1197881282 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e68156300 con 0x7f2e680afaa0 2024-04-04T06:14:50.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.056+0000 7f2e5ea00640 1 --2- 172.21.15.159:0/1197881282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e680cd180 0x7f2e68155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:50.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.056+0000 7f2e67400640 1 --2- 172.21.15.159:0/1197881282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e680afaa0 0x7f2e68142be0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f2e580097a0 tx=0x7f2e580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:50.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.056+0000 7f2e5fe00640 1 -- 172.21.15.159:0/1197881282 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e58004030 con 0x7f2e680afaa0 2024-04-04T06:14:50.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.056+0000 7f2e5fe00640 1 -- 172.21.15.159:0/1197881282 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e580041f0 con 0x7f2e680afaa0 2024-04-04T06:14:50.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.056+0000 7f2e6ca00640 1 -- 172.21.15.159:0/1197881282 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e6814f0f0 con 0x7f2e680afaa0 2024-04-04T06:14:50.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.056+0000 7f2e5fe00640 1 -- 172.21.15.159:0/1197881282 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e58032640 con 0x7f2e680afaa0 2024-04-04T06:14:50.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.057+0000 7f2e6ca00640 1 -- 172.21.15.159:0/1197881282 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e6814f6b0 con 0x7f2e680afaa0 2024-04-04T06:14:50.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.058+0000 7f2e6ca00640 1 -- 172.21.15.159:0/1197881282 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e68075940 con 0x7f2e680afaa0 2024-04-04T06:14:50.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.059+0000 7f2e5fe00640 1 -- 172.21.15.159:0/1197881282 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e580327e0 con 0x7f2e680afaa0 2024-04-04T06:14:50.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.060+0000 7f2e5fe00640 1 --2- 172.21.15.159:0/1197881282 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2e4406c8f0 0x7f2e4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:50.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.060+0000 7f2e5fe00640 1 -- 172.21.15.159:0/1197881282 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f2e5802f030 con 0x7f2e680afaa0 2024-04-04T06:14:50.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.061+0000 7f2e5ea00640 1 --2- 172.21.15.159:0/1197881282 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2e4406c8f0 0x7f2e4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:50.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.062+0000 7f2e5ea00640 1 --2- 172.21.15.159:0/1197881282 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2e4406c8f0 0x7f2e4406edb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2e681f7460 tx=0x7f2e50009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:50.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.064+0000 7f2e5fe00640 1 -- 172.21.15.159:0/1197881282 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2e58041070 con 0x7f2e680afaa0 2024-04-04T06:14:50.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.264+0000 7f2e6ca00640 1 -- 172.21.15.159:0/1197881282 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2e680b1650 con 0x7f2e680afaa0 2024-04-04T06:14:50.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.265+0000 7f2e5fe00640 1 -- 172.21.15.159:0/1197881282 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f2e5807c270 con 0x7f2e680afaa0 2024-04-04T06:14:50.259 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:14:50.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.267+0000 7f2e6ca00640 1 -- 172.21.15.159:0/1197881282 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2e4406c8f0 msgr2=0x7f2e4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:50.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.267+0000 7f2e6ca00640 1 --2- 172.21.15.159:0/1197881282 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2e4406c8f0 0x7f2e4406edb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2e681f7460 tx=0x7f2e50009040 comp rx=0 tx=0).stop 2024-04-04T06:14:50.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.267+0000 7f2e6ca00640 1 -- 172.21.15.159:0/1197881282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e680afaa0 msgr2=0x7f2e68142be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:50.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.267+0000 7f2e6ca00640 1 --2- 172.21.15.159:0/1197881282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e680afaa0 0x7f2e68142be0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f2e580097a0 tx=0x7f2e580046c0 comp rx=0 tx=0).stop 2024-04-04T06:14:50.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.267+0000 7f2e6ca00640 1 -- 172.21.15.159:0/1197881282 shutdown_connections 2024-04-04T06:14:50.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.267+0000 7f2e6ca00640 1 --2- 172.21.15.159:0/1197881282 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e680cd630 0x7f2e6813f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:50.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.267+0000 7f2e6ca00640 1 --2- 172.21.15.159:0/1197881282 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2e4406c8f0 0x7f2e4406edb0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:50.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.267+0000 7f2e6ca00640 1 --2- 172.21.15.159:0/1197881282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e680cd180 0x7f2e68155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:50.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.267+0000 7f2e6ca00640 1 --2- 172.21.15.159:0/1197881282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e680afaa0 0x7f2e68142be0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:50.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.267+0000 7f2e6ca00640 1 -- 172.21.15.159:0/1197881282 >> 172.21.15.159:0/1197881282 conn(0x7f2e680d0770 msgr2=0x7f2e681530b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:50.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.267+0000 7f2e6ca00640 1 -- 172.21.15.159:0/1197881282 shutdown_connections 2024-04-04T06:14:50.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.267+0000 7f2e6ca00640 1 -- 172.21.15.159:0/1197881282 wait complete. 2024-04-04T06:14:50.282 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:14:50.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.551+0000 7f7ed2a00640 1 Processor -- start 2024-04-04T06:14:50.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.551+0000 7f7ed2a00640 1 -- start start 2024-04-04T06:14:50.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.552+0000 7f7ed2a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ecc15e4c0 0x7f7ecc0b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:50.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.552+0000 7f7ed2a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7ecc0d37b0 0x7f7ecc0b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:50.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.552+0000 7f7ed2a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ecc0d2710 con 0x7f7ecc0b67f0 2024-04-04T06:14:50.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.552+0000 7f7ed2a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f7ecc15e8a0 con 0x7f7ecc0d37b0 2024-04-04T06:14:50.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.552+0000 7f7ed2a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7ecc0d4040 con 0x7f7ecc15e4c0 2024-04-04T06:14:50.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.553+0000 7f7ed0c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7ecc0d37b0 0x7f7ecc0b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:50.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.553+0000 7f7ed2000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f7ecc0b67f0 0x7f7ecc0d4260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:36542/0 (socket says 172.21.15.159:36542) 2024-04-04T06:14:50.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.553+0000 7f7ed2000640 1 -- 172.21.15.159:0/2900392761 learned_addr learned my addr 172.21.15.159:0/2900392761 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:50.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.553+0000 7f7ed1600640 1 --2- 172.21.15.159:0/2900392761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ecc15e4c0 0x7f7ecc0b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:50.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.553+0000 7f7ed0c00640 1 -- 172.21.15.159:0/2900392761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ecc15e4c0 msgr2=0x7f7ecc0b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:50.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.553+0000 7f7ed0c00640 1 --2- 172.21.15.159:0/2900392761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ecc15e4c0 0x7f7ecc0b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:50.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.553+0000 7f7ed0c00640 1 -- 172.21.15.159:0/2900392761 >> v1:172.21.15.159:6789/0 conn(0x7f7ecc0b67f0 legacy=0x7f7ecc0d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:50.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.553+0000 7f7ed0c00640 1 -- 172.21.15.159:0/2900392761 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc0d29f0 con 0x7f7ecc0d37b0 2024-04-04T06:14:50.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.553+0000 7f7ecbe00640 1 -- 172.21.15.159:0/2900392761 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 179927715 0 0) 0x7f7ecc0d2710 con 0x7f7ecc0b67f0 2024-04-04T06:14:50.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.554+0000 7f7ed0c00640 1 --2- 172.21.15.159:0/2900392761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ecc0d37b0 0x7f7ecc0b62b0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80097a0 tx=0x7f7eb802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b855b0fd2273929 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:50.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.555+0000 7f7ecbe00640 1 -- 172.21.15.159:0/2900392761 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8035020 con 0x7f7ecc0d37b0 2024-04-04T06:14:50.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.555+0000 7f7ecbe00640 1 -- 172.21.15.159:0/2900392761 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eb8031830 con 0x7f7ecc0d37b0 2024-04-04T06:14:50.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.555+0000 7f7ecbe00640 1 -- 172.21.15.159:0/2900392761 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8031c00 con 0x7f7ecc0d37b0 2024-04-04T06:14:50.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.555+0000 7f7ed2a00640 1 -- 172.21.15.159:0/2900392761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ecc0d37b0 msgr2=0x7f7ecc0b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:50.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.556+0000 7f7ed2a00640 1 --2- 172.21.15.159:0/2900392761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ecc0d37b0 0x7f7ecc0b62b0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80097a0 tx=0x7f7eb802b8d0 comp rx=0 tx=0).stop 2024-04-04T06:14:50.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.556+0000 7f7ed2a00640 1 -- 172.21.15.159:0/2900392761 shutdown_connections 2024-04-04T06:14:50.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.556+0000 7f7ed2a00640 1 --2- 172.21.15.159:0/2900392761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ecc0d37b0 0x7f7ecc0b62b0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:50.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.556+0000 7f7ed2a00640 1 --2- 172.21.15.159:0/2900392761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ecc15e4c0 0x7f7ecc0b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:50.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.556+0000 7f7ed2a00640 1 -- 172.21.15.159:0/2900392761 >> 172.21.15.159:0/2900392761 conn(0x7f7ecc0676a0 msgr2=0x7f7ecc067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:50.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.556+0000 7f7ed2a00640 1 -- 172.21.15.159:0/2900392761 shutdown_connections 2024-04-04T06:14:50.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.556+0000 7f7ed2a00640 1 -- 172.21.15.159:0/2900392761 wait complete. 2024-04-04T06:14:50.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.558+0000 7f7ed2a00640 1 Processor -- start 2024-04-04T06:14:50.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.559+0000 7f7ed2a00640 1 -- start start 2024-04-04T06:14:50.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.560+0000 7f7ed2a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ecc0b67f0 0x7f7ecc1547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:50.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.560+0000 7f7ed2a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ecc0d37b0 0x7f7ecc0ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:50.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.560+0000 7f7ed2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ecc15e4c0 0x7f7ecc0b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:50.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.560+0000 7f7ed2a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc0d2fb0 con 0x7f7ecc15e4c0 2024-04-04T06:14:50.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.560+0000 7f7ed2a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc0d3320 con 0x7f7ecc0d37b0 2024-04-04T06:14:50.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.560+0000 7f7ed2a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7ecc0ba130 con 0x7f7ecc0b67f0 2024-04-04T06:14:50.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.561+0000 7f7ed2000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ecc15e4c0 0x7f7ecc0b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:50.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.561+0000 7f7ed2000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ecc15e4c0 0x7f7ecc0b8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37220/0 (socket says 172.21.15.159:37220) 2024-04-04T06:14:50.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.561+0000 7f7ed2000640 1 -- 172.21.15.159:0/4206605472 learned_addr learned my addr 172.21.15.159:0/4206605472 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:50.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.561+0000 7f7ed0c00640 1 --2- 172.21.15.159:0/4206605472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ecc0d37b0 0x7f7ecc0ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:50.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.561+0000 7f7ed1600640 1 --2- 172.21.15.159:0/4206605472 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ecc0b67f0 0x7f7ecc1547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:50.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.561+0000 7f7ed2000640 1 -- 172.21.15.159:0/4206605472 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ecc0b67f0 msgr2=0x7f7ecc1547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:50.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.561+0000 7f7ed2000640 1 --2- 172.21.15.159:0/4206605472 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ecc0b67f0 0x7f7ecc1547e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:50.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.561+0000 7f7ed2000640 1 -- 172.21.15.159:0/4206605472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ecc0d37b0 msgr2=0x7f7ecc0ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:50.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.561+0000 7f7ed2000640 1 --2- 172.21.15.159:0/4206605472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ecc0d37b0 0x7f7ecc0ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:50.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.561+0000 7f7ed2000640 1 -- 172.21.15.159:0/4206605472 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc154d20 con 0x7f7ecc15e4c0 2024-04-04T06:14:50.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.561+0000 7f7ed0c00640 1 --2- 172.21.15.159:0/4206605472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ecc0d37b0 0x7f7ecc0ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:50.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.561+0000 7f7ed1600640 1 --2- 172.21.15.159:0/4206605472 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ecc0b67f0 0x7f7ecc1547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:50.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.561+0000 7f7ed2000640 1 --2- 172.21.15.159:0/4206605472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ecc15e4c0 0x7f7ecc0b8e40 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400c7a0 tx=0x7f7ec400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:50.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.562+0000 7f7ec9600640 1 -- 172.21.15.159:0/4206605472 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4018070 con 0x7f7ecc15e4c0 2024-04-04T06:14:50.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.562+0000 7f7ec9600640 1 -- 172.21.15.159:0/4206605472 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec400a640 con 0x7f7ecc15e4c0 2024-04-04T06:14:50.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.562+0000 7f7ec9600640 1 -- 172.21.15.159:0/4206605472 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec400aab0 con 0x7f7ecc15e4c0 2024-04-04T06:14:50.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.562+0000 7f7ed2a00640 1 -- 172.21.15.159:0/4206605472 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ecc0cb860 con 0x7f7ecc15e4c0 2024-04-04T06:14:50.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.562+0000 7f7ed2a00640 1 -- 172.21.15.159:0/4206605472 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ecc0cbe20 con 0x7f7ecc15e4c0 2024-04-04T06:14:50.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.563+0000 7f7ed2a00640 1 -- 172.21.15.159:0/4206605472 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ecc073830 con 0x7f7ecc15e4c0 2024-04-04T06:14:50.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.564+0000 7f7ec9600640 1 -- 172.21.15.159:0/4206605472 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ec400ac50 con 0x7f7ecc15e4c0 2024-04-04T06:14:50.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.566+0000 7f7ec9600640 1 --2- 172.21.15.159:0/4206605472 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7eac06c8f0 0x7f7eac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:50.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.566+0000 7f7ec9600640 1 -- 172.21.15.159:0/4206605472 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f7ec408b360 con 0x7f7ecc15e4c0 2024-04-04T06:14:50.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.566+0000 7f7ed1600640 1 --2- 172.21.15.159:0/4206605472 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7eac06c8f0 0x7f7eac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:50.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.567+0000 7f7ed1600640 1 --2- 172.21.15.159:0/4206605472 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7eac06c8f0 0x7f7eac06edb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0006fd0 tx=0x7f7ec0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:50.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.570+0000 7f7ec9600640 1 -- 172.21.15.159:0/4206605472 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7ecc073830 con 0x7f7ecc15e4c0 2024-04-04T06:14:50.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.770+0000 7f7ed2a00640 1 -- 172.21.15.159:0/4206605472 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7ecc075940 con 0x7f7ecc15e4c0 2024-04-04T06:14:50.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.771+0000 7f7ec9600640 1 -- 172.21.15.159:0/4206605472 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f7ecc154d20 con 0x7f7ecc15e4c0 2024-04-04T06:14:50.765 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:14:50.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.773+0000 7f7ed2a00640 1 -- 172.21.15.159:0/4206605472 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7eac06c8f0 msgr2=0x7f7eac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:50.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.773+0000 7f7ed2a00640 1 --2- 172.21.15.159:0/4206605472 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7eac06c8f0 0x7f7eac06edb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0006fd0 tx=0x7f7ec0008040 comp rx=0 tx=0).stop 2024-04-04T06:14:50.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.773+0000 7f7ed2a00640 1 -- 172.21.15.159:0/4206605472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ecc15e4c0 msgr2=0x7f7ecc0b8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:50.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.773+0000 7f7ed2a00640 1 --2- 172.21.15.159:0/4206605472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ecc15e4c0 0x7f7ecc0b8e40 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400c7a0 tx=0x7f7ec400cc60 comp rx=0 tx=0).stop 2024-04-04T06:14:50.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.773+0000 7f7ed2a00640 1 -- 172.21.15.159:0/4206605472 shutdown_connections 2024-04-04T06:14:50.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.773+0000 7f7ed2a00640 1 --2- 172.21.15.159:0/4206605472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ecc15e4c0 0x7f7ecc0b8e40 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:50.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.773+0000 7f7ed2a00640 1 --2- 172.21.15.159:0/4206605472 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7eac06c8f0 0x7f7eac06edb0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:50.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.773+0000 7f7ed2a00640 1 --2- 172.21.15.159:0/4206605472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ecc0d37b0 0x7f7ecc0ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:50.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.773+0000 7f7ed2a00640 1 --2- 172.21.15.159:0/4206605472 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7ecc0b67f0 0x7f7ecc1547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:50.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.773+0000 7f7ed2a00640 1 -- 172.21.15.159:0/4206605472 >> 172.21.15.159:0/4206605472 conn(0x7f7ecc0676a0 msgr2=0x7f7ecc0b3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:50.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.773+0000 7f7ed2a00640 1 -- 172.21.15.159:0/4206605472 shutdown_connections 2024-04-04T06:14:50.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:50.773+0000 7f7ed2a00640 1 -- 172.21.15.159:0/4206605472 wait complete. 2024-04-04T06:14:50.785 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:14:50.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.003+0000 7fc810e00640 1 Processor -- start 2024-04-04T06:14:50.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.003+0000 7fc810e00640 1 -- start start 2024-04-04T06:14:50.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.005+0000 7fc810e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc80c15c390 0x7fc80c073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:50.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.005+0000 7fc810e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc80c0d0650 0x7fc80c0d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:50.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.005+0000 7fc810e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc80c15fe80 con 0x7fc80c0af830 2024-04-04T06:14:50.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.005+0000 7fc810e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fc80c0d1000 con 0x7fc80c0d0650 2024-04-04T06:14:50.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.005+0000 7fc810e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc80c15c770 con 0x7fc80c15c390 2024-04-04T06:14:50.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.005+0000 7fc803e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fc80c0af830 0x7fc80c160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:36548/0 (socket says 172.21.15.159:36548) 2024-04-04T06:14:50.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.005+0000 7fc803e00640 1 -- 172.21.15.159:0/2565485946 learned_addr learned my addr 172.21.15.159:0/2565485946 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:50.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.005+0000 7fc80a800640 1 --2- 172.21.15.159:0/2565485946 >> v2:172.21.15.159:3300/0 conn(0x7fc80c0d0650 0x7fc80c0d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:51.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.005+0000 7fc809e00640 1 --2- 172.21.15.159:0/2565485946 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc80c15c390 0x7fc80c073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:51.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.006+0000 7fc809400640 1 -- 172.21.15.159:0/2565485946 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3119001643 0 0) 0x7fc80c15fe80 con 0x7fc80c0af830 2024-04-04T06:14:51.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.006+0000 7fc809400640 1 -- 172.21.15.159:0/2565485946 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7f0003520 con 0x7fc80c0af830 2024-04-04T06:14:51.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.006+0000 7fc80a800640 1 -- 172.21.15.159:0/2565485946 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc80c15c390 msgr2=0x7fc80c073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:51.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.006+0000 7fc80a800640 1 --2- 172.21.15.159:0/2565485946 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc80c15c390 0x7fc80c073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:51.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.006+0000 7fc80a800640 1 -- 172.21.15.159:0/2565485946 >> v1:172.21.15.159:6789/0 conn(0x7fc80c0af830 legacy=0x7fc80c160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:51.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.006+0000 7fc80a800640 1 -- 172.21.15.159:0/2565485946 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80c0ad120 con 0x7fc80c0d0650 2024-04-04T06:14:51.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.006+0000 7fc809e00640 1 --2- 172.21.15.159:0/2565485946 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc80c15c390 0x7fc80c073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:14:51.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.007+0000 7fc80a800640 1 --2- 172.21.15.159:0/2565485946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc80c0d0650 0x7fc80c0d0ac0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fc804009d90 tx=0x7fc80402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66351106b349eba server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:51.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.007+0000 7fc809400640 1 -- 172.21.15.159:0/2565485946 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804039070 con 0x7fc80c0d0650 2024-04-04T06:14:51.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.007+0000 7fc809400640 1 -- 172.21.15.159:0/2565485946 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc80402cc60 con 0x7fc80c0d0650 2024-04-04T06:14:51.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.007+0000 7fc809400640 1 -- 172.21.15.159:0/2565485946 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804034640 con 0x7fc80c0d0650 2024-04-04T06:14:51.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.008+0000 7fc810e00640 1 -- 172.21.15.159:0/2565485946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc80c0d0650 msgr2=0x7fc80c0d0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:51.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.008+0000 7fc810e00640 1 --2- 172.21.15.159:0/2565485946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc80c0d0650 0x7fc80c0d0ac0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fc804009d90 tx=0x7fc80402beb0 comp rx=0 tx=0).stop 2024-04-04T06:14:51.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.008+0000 7fc810e00640 1 -- 172.21.15.159:0/2565485946 shutdown_connections 2024-04-04T06:14:51.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.008+0000 7fc810e00640 1 --2- 172.21.15.159:0/2565485946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc80c0d0650 0x7fc80c0d0ac0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:51.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.008+0000 7fc810e00640 1 --2- 172.21.15.159:0/2565485946 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc80c15c390 0x7fc80c073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:51.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.008+0000 7fc810e00640 1 -- 172.21.15.159:0/2565485946 >> 172.21.15.159:0/2565485946 conn(0x7fc80c06b990 msgr2=0x7fc80c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:51.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.008+0000 7fc810e00640 1 -- 172.21.15.159:0/2565485946 shutdown_connections 2024-04-04T06:14:51.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.008+0000 7fc810e00640 1 -- 172.21.15.159:0/2565485946 wait complete. 2024-04-04T06:14:51.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.011+0000 7fc810e00640 1 Processor -- start 2024-04-04T06:14:51.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.011+0000 7fc810e00640 1 -- start start 2024-04-04T06:14:51.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.012+0000 7fc810e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc80c0af830 0x7fc80c14a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:51.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.013+0000 7fc810e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc80c0d0650 0x7fc80c15cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:51.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.013+0000 7fc810e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc80c15c390 0x7fc80c0b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:51.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.013+0000 7fc810e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c160120 con 0x7fc80c0d0650 2024-04-04T06:14:51.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.013+0000 7fc810e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c0ad6e0 con 0x7fc80c0af830 2024-04-04T06:14:51.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.013+0000 7fc810e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc80c160d50 con 0x7fc80c15c390 2024-04-04T06:14:51.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.013+0000 7fc803e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc80c0d0650 0x7fc80c15cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:51.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.013+0000 7fc803e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc80c0d0650 0x7fc80c15cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37238/0 (socket says 172.21.15.159:37238) 2024-04-04T06:14:51.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.013+0000 7fc803e00640 1 -- 172.21.15.159:0/9945152 learned_addr learned my addr 172.21.15.159:0/9945152 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:51.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.013+0000 7fc809e00640 1 --2- 172.21.15.159:0/9945152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc80c0af830 0x7fc80c14a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:51.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.013+0000 7fc80a800640 1 --2- 172.21.15.159:0/9945152 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc80c15c390 0x7fc80c0b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:51.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.013+0000 7fc803e00640 1 -- 172.21.15.159:0/9945152 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc80c15c390 msgr2=0x7fc80c0b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:51.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.013+0000 7fc803e00640 1 --2- 172.21.15.159:0/9945152 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc80c15c390 0x7fc80c0b2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:51.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.013+0000 7fc803e00640 1 -- 172.21.15.159:0/9945152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc80c0af830 msgr2=0x7fc80c14a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:51.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.013+0000 7fc803e00640 1 --2- 172.21.15.159:0/9945152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc80c0af830 0x7fc80c14a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:51.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.013+0000 7fc803e00640 1 -- 172.21.15.159:0/9945152 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80c14bb50 con 0x7fc80c0d0650 2024-04-04T06:14:51.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.014+0000 7fc803e00640 1 --2- 172.21.15.159:0/9945152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc80c0d0650 0x7fc80c15cc90 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400c390 tx=0x7fc7f400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:51.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.014+0000 7fc802000640 1 -- 172.21.15.159:0/9945152 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4017070 con 0x7fc80c0d0650 2024-04-04T06:14:51.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.014+0000 7fc802000640 1 -- 172.21.15.159:0/9945152 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7f4005ce0 con 0x7fc80c0d0650 2024-04-04T06:14:51.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.014+0000 7fc810e00640 1 -- 172.21.15.159:0/9945152 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc80c14be30 con 0x7fc80c0d0650 2024-04-04T06:14:51.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.014+0000 7fc802000640 1 -- 172.21.15.159:0/9945152 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f40072f0 con 0x7fc80c0d0650 2024-04-04T06:14:51.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.014+0000 7fc810e00640 1 -- 172.21.15.159:0/9945152 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc80c14c3f0 con 0x7fc80c0d0650 2024-04-04T06:14:51.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.016+0000 7fc810e00640 1 -- 172.21.15.159:0/9945152 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc80c0685c0 con 0x7fc80c0d0650 2024-04-04T06:14:51.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.016+0000 7fc802000640 1 -- 172.21.15.159:0/9945152 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7f401a070 con 0x7fc80c0d0650 2024-04-04T06:14:51.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.018+0000 7fc802000640 1 --2- 172.21.15.159:0/9945152 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc7e806c8f0 0x7fc7e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:51.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.018+0000 7fc802000640 1 -- 172.21.15.159:0/9945152 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7fc7f408b100 con 0x7fc80c0d0650 2024-04-04T06:14:51.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.018+0000 7fc809e00640 1 --2- 172.21.15.159:0/9945152 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc7e806c8f0 0x7fc7e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:51.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.019+0000 7fc809e00640 1 --2- 172.21.15.159:0/9945152 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc7e806c8f0 0x7fc7e806edb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc006fd0 tx=0x7fc7fc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:51.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.021+0000 7fc802000640 1 -- 172.21.15.159:0/9945152 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc7f40593a0 con 0x7fc80c0d0650 2024-04-04T06:14:51.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.405+0000 7fc810e00640 1 -- 172.21.15.159:0/9945152 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc80c06f1b0 con 0x7fc80c0d0650 2024-04-04T06:14:51.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.406+0000 7fc802000640 1 -- 172.21.15.159:0/9945152 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5196 (secure 0 0 0) 0x7fc7f4058ca0 con 0x7fc80c0d0650 2024-04-04T06:14:51.401 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:14:51.401 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":76,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1825651864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1825651864},{"type":"v1","addr":"172.21.15.178:6836","nonce":1825651864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1824578712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1824578712},{"type":"v1","addr":"172.21.15.178:6837","nonce":1824578712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6834/3377498298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3377498298},{"type":"v1","addr":"172.21.15.159:6834","nonce":3377498298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:12:35.880454+0000","modified":"2024-04-04T06:14:09.810244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7809},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7809":{"gid":7809,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6835/3555825575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":3555825575},{"type":"v1","addr":"172.21.15.159:6835","nonce":3555825575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7809,"qdb_cluster":[7809]},"id":6}]} 2024-04-04T06:14:51.401 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 76 2024-04-04T06:14:51.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.409+0000 7fc810e00640 1 -- 172.21.15.159:0/9945152 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc7e806c8f0 msgr2=0x7fc7e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:51.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.409+0000 7fc810e00640 1 --2- 172.21.15.159:0/9945152 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc7e806c8f0 0x7fc7e806edb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc006fd0 tx=0x7fc7fc008040 comp rx=0 tx=0).stop 2024-04-04T06:14:51.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.409+0000 7fc810e00640 1 -- 172.21.15.159:0/9945152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc80c0d0650 msgr2=0x7fc80c15cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:51.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.409+0000 7fc810e00640 1 --2- 172.21.15.159:0/9945152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc80c0d0650 0x7fc80c15cc90 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400c390 tx=0x7fc7f400c850 comp rx=0 tx=0).stop 2024-04-04T06:14:51.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.409+0000 7fc810e00640 1 -- 172.21.15.159:0/9945152 shutdown_connections 2024-04-04T06:14:51.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.409+0000 7fc810e00640 1 --2- 172.21.15.159:0/9945152 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc80c15c390 0x7fc80c0b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:51.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.409+0000 7fc810e00640 1 --2- 172.21.15.159:0/9945152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc80c0d0650 0x7fc80c15cc90 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:51.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.409+0000 7fc810e00640 1 --2- 172.21.15.159:0/9945152 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc7e806c8f0 0x7fc7e806edb0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:51.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.409+0000 7fc810e00640 1 --2- 172.21.15.159:0/9945152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc80c0af830 0x7fc80c14a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:51.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.409+0000 7fc810e00640 1 -- 172.21.15.159:0/9945152 >> 172.21.15.159:0/9945152 conn(0x7fc80c06b990 msgr2=0x7fc80c0aece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:51.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.409+0000 7fc810e00640 1 -- 172.21.15.159:0/9945152 shutdown_connections 2024-04-04T06:14:51.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.409+0000 7fc810e00640 1 -- 172.21.15.159:0/9945152 wait complete. 2024-04-04T06:14:51.423 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:14:51.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.633+0000 7ff709800640 1 Processor -- start 2024-04-04T06:14:51.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.633+0000 7ff709800640 1 -- start start 2024-04-04T06:14:51.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.635+0000 7ff709800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff70415e700 0x7ff7040b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:51.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.635+0000 7ff709800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff70415eae0 con 0x7ff7040d1610 2024-04-04T06:14:51.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.635+0000 7ff709800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff704071e00 con 0x7ff7040d1cf0 2024-04-04T06:14:51.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.635+0000 7ff709800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7ff7040cfeb0 con 0x7ff70415e700 2024-04-04T06:14:51.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.635+0000 7ff702a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff70415e700 0x7ff7040b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:51.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.635+0000 7ff6fa000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7ff7040d1610 0x7ff704070fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:36562/0 (socket says 172.21.15.159:36562) 2024-04-04T06:14:51.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.635+0000 7ff6fa000640 1 -- 172.21.15.159:0/2697198161 learned_addr learned my addr 172.21.15.159:0/2697198161 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:51.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.636+0000 7ff702a00640 1 -- 172.21.15.159:0/2697198161 >> v1:172.21.15.178:6789/0 conn(0x7ff7040d1cf0 legacy=0x7ff7040716f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:51.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.636+0000 7ff702a00640 1 -- 172.21.15.159:0/2697198161 >> v1:172.21.15.159:6789/0 conn(0x7ff7040d1610 legacy=0x7ff704070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:51.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.636+0000 7ff702a00640 1 -- 172.21.15.159:0/2697198161 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7040d0190 con 0x7ff70415e700 2024-04-04T06:14:51.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.636+0000 7ff702000640 1 -- 172.21.15.159:0/2697198161 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2716881265 0 0) 0x7ff70415eae0 con 0x7ff7040d1610 2024-04-04T06:14:51.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.637+0000 7ff702a00640 1 --2- 172.21.15.159:0/2697198161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff70415e700 0x7ff7040b3e20 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009d90 tx=0x7ff6f402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a84dbd526e72faa server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.637+0000 7ff702000640 1 -- 172.21.15.159:0/2697198161 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4039070 con 0x7ff70415e700 2024-04-04T06:14:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.637+0000 7ff702000640 1 -- 172.21.15.159:0/2697198161 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f402cbd0 con 0x7ff70415e700 2024-04-04T06:14:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.637+0000 7ff702000640 1 -- 172.21.15.159:0/2697198161 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4034440 con 0x7ff70415e700 2024-04-04T06:14:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.638+0000 7ff709800640 1 -- 172.21.15.159:0/2697198161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff70415e700 msgr2=0x7ff7040b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.638+0000 7ff709800640 1 --2- 172.21.15.159:0/2697198161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff70415e700 0x7ff7040b3e20 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009d90 tx=0x7ff6f402be20 comp rx=0 tx=0).stop 2024-04-04T06:14:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.638+0000 7ff709800640 1 -- 172.21.15.159:0/2697198161 shutdown_connections 2024-04-04T06:14:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.638+0000 7ff709800640 1 --2- 172.21.15.159:0/2697198161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff70415e700 0x7ff7040b3e20 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.638+0000 7ff709800640 1 -- 172.21.15.159:0/2697198161 >> 172.21.15.159:0/2697198161 conn(0x7ff7040d2240 msgr2=0x7ff7040d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.638+0000 7ff709800640 1 -- 172.21.15.159:0/2697198161 shutdown_connections 2024-04-04T06:14:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.638+0000 7ff709800640 1 -- 172.21.15.159:0/2697198161 wait complete. 2024-04-04T06:14:51.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.639+0000 7ff709800640 1 Processor -- start 2024-04-04T06:14:51.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.640+0000 7ff709800640 1 -- start start 2024-04-04T06:14:51.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.640+0000 7ff709800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7040d1610 0x7ff7041f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:51.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.640+0000 7ff709800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7040d1cf0 0x7ff704157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:51.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.640+0000 7ff709800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff70415e700 0x7ff7041f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:51.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.640+0000 7ff709800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff7040d0a30 con 0x7ff7040d1610 2024-04-04T06:14:51.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.640+0000 7ff709800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff7040d0750 con 0x7ff70415e700 2024-04-04T06:14:51.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.640+0000 7ff709800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff70406d1b0 con 0x7ff7040d1cf0 2024-04-04T06:14:51.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.641+0000 7ff702a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7040d1610 0x7ff7041f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:51.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.641+0000 7ff702a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7040d1610 0x7ff7041f50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37262/0 (socket says 172.21.15.159:37262) 2024-04-04T06:14:51.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.641+0000 7ff702a00640 1 -- 172.21.15.159:0/1954645810 learned_addr learned my addr 172.21.15.159:0/1954645810 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:51.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.641+0000 7ff703400640 1 --2- 172.21.15.159:0/1954645810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff70415e700 0x7ff7041f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:51.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.641+0000 7ff702a00640 1 -- 172.21.15.159:0/1954645810 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7040d1cf0 msgr2=0x7ff704157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:51.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.641+0000 7ff702a00640 1 --2- 172.21.15.159:0/1954645810 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7040d1cf0 0x7ff704157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:51.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.641+0000 7ff702a00640 1 -- 172.21.15.159:0/1954645810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff70415e700 msgr2=0x7ff7041f7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:51.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.641+0000 7ff702a00640 1 --2- 172.21.15.159:0/1954645810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff70415e700 0x7ff7041f7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:51.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.641+0000 7ff702a00640 1 -- 172.21.15.159:0/1954645810 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7041f5610 con 0x7ff7040d1610 2024-04-04T06:14:51.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.642+0000 7ff702a00640 1 --2- 172.21.15.159:0/1954645810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7040d1610 0x7ff7041f50d0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4037040 tx=0x7ff6f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:51.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.642+0000 7ff6fb400640 1 -- 172.21.15.159:0/1954645810 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4039040 con 0x7ff7040d1610 2024-04-04T06:14:51.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.642+0000 7ff6fb400640 1 -- 172.21.15.159:0/1954645810 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f4003b90 con 0x7ff7040d1610 2024-04-04T06:14:51.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.642+0000 7ff6fb400640 1 -- 172.21.15.159:0/1954645810 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4004030 con 0x7ff7040d1610 2024-04-04T06:14:51.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.642+0000 7ff709800640 1 -- 172.21.15.159:0/1954645810 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7041561e0 con 0x7ff7040d1610 2024-04-04T06:14:51.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.642+0000 7ff709800640 1 -- 172.21.15.159:0/1954645810 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7041567a0 con 0x7ff7040d1610 2024-04-04T06:14:51.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.644+0000 7ff709800640 1 -- 172.21.15.159:0/1954645810 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7040685c0 con 0x7ff7040d1610 2024-04-04T06:14:51.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.644+0000 7ff6fb400640 1 -- 172.21.15.159:0/1954645810 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff6f402ca90 con 0x7ff7040d1610 2024-04-04T06:14:51.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.645+0000 7ff6fb400640 1 --2- 172.21.15.159:0/1954645810 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff6e006c8f0 0x7ff6e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:51.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.645+0000 7ff6fb400640 1 -- 172.21.15.159:0/1954645810 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7ff6f40aed90 con 0x7ff7040d1610 2024-04-04T06:14:51.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.645+0000 7ff6fa000640 1 --2- 172.21.15.159:0/1954645810 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff6e006c8f0 0x7ff6e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:51.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.646+0000 7ff6fa000640 1 --2- 172.21.15.159:0/1954645810 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff6e006c8f0 0x7ff6e006edb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec007fd0 tx=0x7ff6ec007960 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:51.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.654+0000 7ff6fb400640 1 -- 172.21.15.159:0/1954645810 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff7041f5610 con 0x7ff7040d1610 2024-04-04T06:14:51.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.855+0000 7ff709800640 1 -- 172.21.15.159:0/1954645810 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff70415cb80 con 0x7ff7040d1610 2024-04-04T06:14:51.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.857+0000 7ff6fb400640 1 -- 172.21.15.159:0/1954645810 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+17033 (secure 0 0 0) 0x7ff6f407d160 con 0x7ff7040d1610 2024-04-04T06:14:51.852 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:14:51.852 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":89,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:14:04.757031+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:12:33.658405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-04T06:12:34.856266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.3d","mappings":[{"from":6,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6835/3796123128":"2024-04-05T06:14:04.756982+0000","172.21.15.159:6833/3796123128":"2024-04-05T06:14:04.756982+0000","172.21.15.159:6835/3484779950":"2024-04-05T06:12:49.205338+0000","172.21.15.159:6833/3484779950":"2024-04-05T06:12:49.205338+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:14:51.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.860+0000 7ff709800640 1 -- 172.21.15.159:0/1954645810 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff6e006c8f0 msgr2=0x7ff6e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:51.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.860+0000 7ff709800640 1 --2- 172.21.15.159:0/1954645810 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff6e006c8f0 0x7ff6e006edb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec007fd0 tx=0x7ff6ec007960 comp rx=0 tx=0).stop 2024-04-04T06:14:51.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.860+0000 7ff709800640 1 -- 172.21.15.159:0/1954645810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7040d1610 msgr2=0x7ff7041f50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:51.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.860+0000 7ff709800640 1 --2- 172.21.15.159:0/1954645810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7040d1610 0x7ff7041f50d0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4037040 tx=0x7ff6f4004a40 comp rx=0 tx=0).stop 2024-04-04T06:14:51.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.860+0000 7ff709800640 1 -- 172.21.15.159:0/1954645810 shutdown_connections 2024-04-04T06:14:51.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.860+0000 7ff709800640 1 --2- 172.21.15.159:0/1954645810 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff6e006c8f0 0x7ff6e006edb0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:51.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.860+0000 7ff709800640 1 --2- 172.21.15.159:0/1954645810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff70415e700 0x7ff7041f7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:51.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.860+0000 7ff709800640 1 --2- 172.21.15.159:0/1954645810 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7040d1cf0 0x7ff704157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:51.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.860+0000 7ff709800640 1 --2- 172.21.15.159:0/1954645810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7040d1610 0x7ff7041f50d0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:51.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.860+0000 7ff709800640 1 -- 172.21.15.159:0/1954645810 >> 172.21.15.159:0/1954645810 conn(0x7ff7040d2240 msgr2=0x7ff70415b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:51.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.860+0000 7ff709800640 1 -- 172.21.15.159:0/1954645810 shutdown_connections 2024-04-04T06:14:51.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:51.860+0000 7ff709800640 1 -- 172.21.15.159:0/1954645810 wait complete. 2024-04-04T06:14:51.872 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-04T06:14:51.872 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:14:52.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.090+0000 7f855ca00640 1 Processor -- start 2024-04-04T06:14:52.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.090+0000 7f855ca00640 1 -- start start 2024-04-04T06:14:52.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.091+0000 7f855ca00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f85580d2310 0x7f85580ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:52.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.091+0000 7f855ca00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f85580cf530 0x7f85580b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:52.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.091+0000 7f855ca00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f855815e880 con 0x7f855815e4a0 2024-04-04T06:14:52.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.091+0000 7f855ca00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f85580cf9a0 con 0x7f85580d2310 2024-04-04T06:14:52.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.091+0000 7f855ca00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f85580d34e0 con 0x7f85580cf530 2024-04-04T06:14:52.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.092+0000 7f854de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f85580d2310 0x7f85580ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:52.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.092+0000 7f8556800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f855815e4a0 0x7f85580b9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:36576/0 (socket says 172.21.15.159:36576) 2024-04-04T06:14:52.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.092+0000 7f8556800640 1 -- 172.21.15.159:0/2109267751 learned_addr learned my addr 172.21.15.159:0/2109267751 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:52.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.092+0000 7f8557200640 1 --2- 172.21.15.159:0/2109267751 >> v2:172.21.15.178:3300/0 conn(0x7f85580cf530 0x7f85580b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:52.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.092+0000 7f8555e00640 1 -- 172.21.15.159:0/2109267751 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3620203662 0 0) 0x7f855815e880 con 0x7f855815e4a0 2024-04-04T06:14:52.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.092+0000 7f8555e00640 1 -- 172.21.15.159:0/2109267751 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f853c003520 con 0x7f855815e4a0 2024-04-04T06:14:52.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.092+0000 7f854de00640 1 -- 172.21.15.159:0/2109267751 >> v2:172.21.15.178:3300/0 conn(0x7f85580cf530 msgr2=0x7f85580b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:52.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.092+0000 7f854de00640 1 --2- 172.21.15.159:0/2109267751 >> v2:172.21.15.178:3300/0 conn(0x7f85580cf530 0x7f85580b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:52.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.093+0000 7f854de00640 1 -- 172.21.15.159:0/2109267751 >> v1:172.21.15.159:6789/0 conn(0x7f855815e4a0 legacy=0x7f85580b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:52.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.093+0000 7f854de00640 1 -- 172.21.15.159:0/2109267751 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85580d28d0 con 0x7f85580d2310 2024-04-04T06:14:52.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.094+0000 7f854de00640 1 --2- 172.21.15.159:0/2109267751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85580d2310 0x7f85580ceff0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f85580d3660 tx=0x7f854002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87864ade6523d660 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:52.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.094+0000 7f8555e00640 1 -- 172.21.15.159:0/2109267751 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85400315a0 con 0x7f85580d2310 2024-04-04T06:14:52.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.094+0000 7f8555e00640 1 -- 172.21.15.159:0/2109267751 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8540037070 con 0x7f85580d2310 2024-04-04T06:14:52.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.094+0000 7f8555e00640 1 -- 172.21.15.159:0/2109267751 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540031940 con 0x7f85580d2310 2024-04-04T06:14:52.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.095+0000 7f855ca00640 1 -- 172.21.15.159:0/2109267751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85580d2310 msgr2=0x7f85580ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:52.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.095+0000 7f855ca00640 1 --2- 172.21.15.159:0/2109267751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85580d2310 0x7f85580ceff0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f85580d3660 tx=0x7f854002b840 comp rx=0 tx=0).stop 2024-04-04T06:14:52.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.095+0000 7f855ca00640 1 -- 172.21.15.159:0/2109267751 shutdown_connections 2024-04-04T06:14:52.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.095+0000 7f855ca00640 1 --2- 172.21.15.159:0/2109267751 >> v2:172.21.15.178:3300/0 conn(0x7f85580cf530 0x7f85580b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:52.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.095+0000 7f855ca00640 1 --2- 172.21.15.159:0/2109267751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85580d2310 0x7f85580ceff0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:52.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.095+0000 7f855ca00640 1 -- 172.21.15.159:0/2109267751 >> 172.21.15.159:0/2109267751 conn(0x7f85580676d0 msgr2=0x7f8558067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:52.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.095+0000 7f855ca00640 1 -- 172.21.15.159:0/2109267751 shutdown_connections 2024-04-04T06:14:52.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.095+0000 7f855ca00640 1 -- 172.21.15.159:0/2109267751 wait complete. 2024-04-04T06:14:52.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.097+0000 7f855ca00640 1 Processor -- start 2024-04-04T06:14:52.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.097+0000 7f855ca00640 1 -- start start 2024-04-04T06:14:52.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.098+0000 7f855ca00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85580cf530 0x7f85580c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:52.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.098+0000 7f855ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85580d2310 0x7f8558160290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:52.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.098+0000 7f855ca00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f855815e4a0 0x7f855815ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:52.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.098+0000 7f855ca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f85580d3170 con 0x7f85580d2310 2024-04-04T06:14:52.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.098+0000 7f855ca00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f85580d2e90 con 0x7f855815e4a0 2024-04-04T06:14:52.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.098+0000 7f855ca00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f85580d3970 con 0x7f85580cf530 2024-04-04T06:14:52.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.098+0000 7f854de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85580d2310 0x7f8558160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:52.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.098+0000 7f854de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85580d2310 0x7f8558160290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37284/0 (socket says 172.21.15.159:37284) 2024-04-04T06:14:52.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.098+0000 7f854de00640 1 -- 172.21.15.159:0/2838751145 learned_addr learned my addr 172.21.15.159:0/2838751145 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:52.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.098+0000 7f854de00640 1 -- 172.21.15.159:0/2838751145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85580cf530 msgr2=0x7f85580c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:52.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.098+0000 7f854de00640 1 --2- 172.21.15.159:0/2838751145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85580cf530 0x7f85580c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:52.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.098+0000 7f854de00640 1 -- 172.21.15.159:0/2838751145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f855815e4a0 msgr2=0x7f855815ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:52.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.098+0000 7f854de00640 1 --2- 172.21.15.159:0/2838751145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f855815e4a0 0x7f855815ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:52.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.098+0000 7f854de00640 1 -- 172.21.15.159:0/2838751145 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855815f1f0 con 0x7f85580d2310 2024-04-04T06:14:52.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.099+0000 7f854de00640 1 --2- 172.21.15.159:0/2838751145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85580d2310 0x7f8558160290 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f8540000c00 tx=0x7f85400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:52.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.099+0000 7f854f400640 1 -- 172.21.15.159:0/2838751145 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540032630 con 0x7f85580d2310 2024-04-04T06:14:52.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.099+0000 7f854f400640 1 -- 172.21.15.159:0/2838751145 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85400327d0 con 0x7f85580d2310 2024-04-04T06:14:52.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.099+0000 7f855ca00640 1 -- 172.21.15.159:0/2838751145 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f855815f4d0 con 0x7f85580d2310 2024-04-04T06:14:52.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.099+0000 7f854f400640 1 -- 172.21.15.159:0/2838751145 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540032ba0 con 0x7f85580d2310 2024-04-04T06:14:52.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.099+0000 7f855ca00640 1 -- 172.21.15.159:0/2838751145 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f855815fa90 con 0x7f85580d2310 2024-04-04T06:14:52.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.100+0000 7f855ca00640 1 -- 172.21.15.159:0/2838751145 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8558075940 con 0x7f85580d2310 2024-04-04T06:14:52.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.101+0000 7f854f400640 1 -- 172.21.15.159:0/2838751145 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f854004e430 con 0x7f85580d2310 2024-04-04T06:14:52.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.102+0000 7f854f400640 1 --2- 172.21.15.159:0/2838751145 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f853406c8f0 0x7f853406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:52.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.102+0000 7f854f400640 1 -- 172.21.15.159:0/2838751145 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f854002f030 con 0x7f85580d2310 2024-04-04T06:14:52.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.103+0000 7f8556800640 1 --2- 172.21.15.159:0/2838751145 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f853406c8f0 0x7f853406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:52.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.104+0000 7f8556800640 1 --2- 172.21.15.159:0/2838751145 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f853406c8f0 0x7f853406edb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8548000f60 tx=0x7f85480078f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:52.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.106+0000 7f854f400640 1 -- 172.21.15.159:0/2838751145 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8540040070 con 0x7f85580d2310 2024-04-04T06:14:52.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.351+0000 7f855ca00640 1 -- 172.21.15.159:0/2838751145 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8558143d60 con 0x7f85580d2310 2024-04-04T06:14:52.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.352+0000 7f854f400640 1 -- 172.21.15.159:0/2838751145 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5196 (secure 0 0 0) 0x7f854007bf20 con 0x7f85580d2310 2024-04-04T06:14:52.346 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:14:52.347 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":76,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1825651864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1825651864},{"type":"v1","addr":"172.21.15.178:6836","nonce":1825651864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1824578712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1824578712},{"type":"v1","addr":"172.21.15.178:6837","nonce":1824578712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6834/3377498298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3377498298},{"type":"v1","addr":"172.21.15.159:6834","nonce":3377498298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:12:35.880454+0000","modified":"2024-04-04T06:14:09.810244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7809},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7809":{"gid":7809,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6835/3555825575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":3555825575},{"type":"v1","addr":"172.21.15.159:6835","nonce":3555825575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7809,"qdb_cluster":[7809]},"id":6}]} 2024-04-04T06:14:52.347 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 76 2024-04-04T06:14:52.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.355+0000 7f855ca00640 1 -- 172.21.15.159:0/2838751145 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f853406c8f0 msgr2=0x7f853406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:52.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.355+0000 7f855ca00640 1 --2- 172.21.15.159:0/2838751145 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f853406c8f0 0x7f853406edb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8548000f60 tx=0x7f85480078f0 comp rx=0 tx=0).stop 2024-04-04T06:14:52.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.355+0000 7f855ca00640 1 -- 172.21.15.159:0/2838751145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85580d2310 msgr2=0x7f8558160290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:52.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.355+0000 7f855ca00640 1 --2- 172.21.15.159:0/2838751145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85580d2310 0x7f8558160290 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f8540000c00 tx=0x7f85400046c0 comp rx=0 tx=0).stop 2024-04-04T06:14:52.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.355+0000 7f855ca00640 1 -- 172.21.15.159:0/2838751145 shutdown_connections 2024-04-04T06:14:52.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.355+0000 7f855ca00640 1 --2- 172.21.15.159:0/2838751145 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f853406c8f0 0x7f853406edb0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:52.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.355+0000 7f855ca00640 1 --2- 172.21.15.159:0/2838751145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f855815e4a0 0x7f855815ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:52.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.355+0000 7f855ca00640 1 --2- 172.21.15.159:0/2838751145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85580d2310 0x7f8558160290 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:52.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.355+0000 7f855ca00640 1 --2- 172.21.15.159:0/2838751145 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85580cf530 0x7f85580c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:52.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.355+0000 7f855ca00640 1 -- 172.21.15.159:0/2838751145 >> 172.21.15.159:0/2838751145 conn(0x7f85580676d0 msgr2=0x7f8558067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:52.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.355+0000 7f855ca00640 1 -- 172.21.15.159:0/2838751145 shutdown_connections 2024-04-04T06:14:52.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.355+0000 7f855ca00640 1 -- 172.21.15.159:0/2838751145 wait complete. 2024-04-04T06:14:52.369 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:14:52.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.577+0000 7f9776200640 1 Processor -- start 2024-04-04T06:14:52.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.577+0000 7f9776200640 1 -- start start 2024-04-04T06:14:52.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.578+0000 7f9776200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f977015aa90 0x7f977015b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:52.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.578+0000 7f9776200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f977015bee0 con 0x7f97700717f0 2024-04-04T06:14:52.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.578+0000 7f9776200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97700b3910 con 0x7f9770072100 2024-04-04T06:14:52.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.578+0000 7f9776200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f97700b5190 con 0x7f977015aa90 2024-04-04T06:14:52.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.578+0000 7f976f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f977015aa90 0x7f977015b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:52.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.578+0000 7f976f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f977015aa90 0x7f977015b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37296/0 (socket says 172.21.15.159:37296) 2024-04-04T06:14:52.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.578+0000 7f976f400640 1 -- 172.21.15.159:0/95817956 learned_addr learned my addr 172.21.15.159:0/95817956 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:52.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.579+0000 7f976f400640 1 -- 172.21.15.159:0/95817956 >> v1:172.21.15.178:6789/0 conn(0x7f9770072100 legacy=0x7f9770072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:52.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.579+0000 7f976f400640 1 -- 172.21.15.159:0/95817956 >> v1:172.21.15.159:6789/0 conn(0x7f97700717f0 legacy=0x7f97700b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:52.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.579+0000 7f976e000640 1 -- 172.21.15.159:0/95817956 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1004519335 0 0) 0x7f975c001110 con 0x7f97700717f0 2024-04-04T06:14:52.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.579+0000 7f976f400640 1 -- 172.21.15.159:0/95817956 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97700b3bf0 con 0x7f977015aa90 2024-04-04T06:14:52.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.580+0000 7f976f400640 1 --2- 172.21.15.159:0/95817956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f977015aa90 0x7f977015b9a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f9760009870 tx=0x7f976002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39962c93fe395d26 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:52.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.581+0000 7f976e000640 1 -- 172.21.15.159:0/95817956 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760034070 con 0x7f977015aa90 2024-04-04T06:14:52.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.581+0000 7f976e000640 1 -- 172.21.15.159:0/95817956 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9760030860 con 0x7f977015aa90 2024-04-04T06:14:52.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.581+0000 7f976e000640 1 -- 172.21.15.159:0/95817956 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760030c30 con 0x7f977015aa90 2024-04-04T06:14:52.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.581+0000 7f9776200640 1 -- 172.21.15.159:0/95817956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f977015aa90 msgr2=0x7f977015b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:52.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.581+0000 7f9776200640 1 --2- 172.21.15.159:0/95817956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f977015aa90 0x7f977015b9a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f9760009870 tx=0x7f976002b9a0 comp rx=0 tx=0).stop 2024-04-04T06:14:52.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.581+0000 7f9776200640 1 -- 172.21.15.159:0/95817956 shutdown_connections 2024-04-04T06:14:52.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.581+0000 7f9776200640 1 --2- 172.21.15.159:0/95817956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f977015aa90 0x7f977015b9a0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:52.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.581+0000 7f9776200640 1 -- 172.21.15.159:0/95817956 >> 172.21.15.159:0/95817956 conn(0x7f977006b990 msgr2=0x7f977006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:52.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.581+0000 7f9776200640 1 -- 172.21.15.159:0/95817956 shutdown_connections 2024-04-04T06:14:52.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.581+0000 7f9776200640 1 -- 172.21.15.159:0/95817956 wait complete. 2024-04-04T06:14:52.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.583+0000 7f9776200640 1 Processor -- start 2024-04-04T06:14:52.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.584+0000 7f9776200640 1 -- start start 2024-04-04T06:14:52.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.585+0000 7f9776200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97700717f0 0x7f977006e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:52.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.585+0000 7f9776200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9770072100 0x7f9770145d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:52.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.585+0000 7f9776200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f977015aa90 0x7f977014ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:52.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.585+0000 7f9776200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9770071cc0 con 0x7f9770072100 2024-04-04T06:14:52.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.585+0000 7f9776200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97700b4240 con 0x7f97700717f0 2024-04-04T06:14:52.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.585+0000 7f9776200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f97700d0650 con 0x7f977015aa90 2024-04-04T06:14:52.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.585+0000 7f976ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9770072100 0x7f9770145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:52.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.585+0000 7f976ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9770072100 0x7f9770145d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37308/0 (socket says 172.21.15.159:37308) 2024-04-04T06:14:52.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.585+0000 7f976ea00640 1 -- 172.21.15.159:0/9102920 learned_addr learned my addr 172.21.15.159:0/9102920 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:52.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.585+0000 7f976fe00640 1 --2- 172.21.15.159:0/9102920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f977015aa90 0x7f977014ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:52.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.585+0000 7f976f400640 1 --2- 172.21.15.159:0/9102920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97700717f0 0x7f977006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:52.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.585+0000 7f976ea00640 1 -- 172.21.15.159:0/9102920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f977015aa90 msgr2=0x7f977014ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:52.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.585+0000 7f976ea00640 1 --2- 172.21.15.159:0/9102920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f977015aa90 0x7f977014ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:52.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.586+0000 7f976ea00640 1 -- 172.21.15.159:0/9102920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97700717f0 msgr2=0x7f977006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:52.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.586+0000 7f976ea00640 1 --2- 172.21.15.159:0/9102920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97700717f0 0x7f977006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:52.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.586+0000 7f976ea00640 1 -- 172.21.15.159:0/9102920 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977014c840 con 0x7f9770072100 2024-04-04T06:14:52.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.586+0000 7f976f400640 1 --2- 172.21.15.159:0/9102920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97700717f0 0x7f977006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:52.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.586+0000 7f976ea00640 1 --2- 172.21.15.159:0/9102920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9770072100 0x7f9770145d40 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f975c00c780 tx=0x7f975c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:52.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.586+0000 7f9767400640 1 -- 172.21.15.159:0/9102920 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c017070 con 0x7f9770072100 2024-04-04T06:14:52.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.586+0000 7f9767400640 1 -- 172.21.15.159:0/9102920 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f975c005070 con 0x7f9770072100 2024-04-04T06:14:52.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.586+0000 7f9767400640 1 -- 172.21.15.159:0/9102920 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c00a640 con 0x7f9770072100 2024-04-04T06:14:52.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.586+0000 7f9776200640 1 -- 172.21.15.159:0/9102920 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977014cb20 con 0x7f9770072100 2024-04-04T06:14:52.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.586+0000 7f9776200640 1 -- 172.21.15.159:0/9102920 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9770146280 con 0x7f9770072100 2024-04-04T06:14:52.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.588+0000 7f9776200640 1 -- 172.21.15.159:0/9102920 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97700685c0 con 0x7f9770072100 2024-04-04T06:14:52.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.588+0000 7f9767400640 1 -- 172.21.15.159:0/9102920 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f975c01a070 con 0x7f9770072100 2024-04-04T06:14:52.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.589+0000 7f9767400640 1 --2- 172.21.15.159:0/9102920 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f975006c8f0 0x7f975006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:52.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.589+0000 7f9767400640 1 -- 172.21.15.159:0/9102920 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7f975c08afc0 con 0x7f9770072100 2024-04-04T06:14:52.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.590+0000 7f976f400640 1 --2- 172.21.15.159:0/9102920 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f975006c8f0 0x7f975006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:52.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.591+0000 7f976f400640 1 --2- 172.21.15.159:0/9102920 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f975006c8f0 0x7f975006edb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9760036000 tx=0x7f976003b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:52.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.594+0000 7f9767400640 1 -- 172.21.15.159:0/9102920 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f977015bee0 con 0x7f9770072100 2024-04-04T06:14:52.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.834+0000 7f9776200640 1 -- 172.21.15.159:0/9102920 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97700b3650 con 0x7f9770072100 2024-04-04T06:14:52.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.835+0000 7f9767400640 1 -- 172.21.15.159:0/9102920 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5196 (secure 0 0 0) 0x7f975c059390 con 0x7f9770072100 2024-04-04T06:14:52.829 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:14:52.829 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":76,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1825651864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1825651864},{"type":"v1","addr":"172.21.15.178:6836","nonce":1825651864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1824578712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1824578712},{"type":"v1","addr":"172.21.15.178:6837","nonce":1824578712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6834/3377498298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3377498298},{"type":"v1","addr":"172.21.15.159:6834","nonce":3377498298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:12:35.880454+0000","modified":"2024-04-04T06:14:09.810244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7809},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7809":{"gid":7809,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6835/3555825575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":3555825575},{"type":"v1","addr":"172.21.15.159:6835","nonce":3555825575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7809,"qdb_cluster":[7809]},"id":6}]} 2024-04-04T06:14:52.829 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 76 2024-04-04T06:14:52.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.837+0000 7f9776200640 1 -- 172.21.15.159:0/9102920 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f975006c8f0 msgr2=0x7f975006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:52.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.837+0000 7f9776200640 1 --2- 172.21.15.159:0/9102920 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f975006c8f0 0x7f975006edb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9760036000 tx=0x7f976003b040 comp rx=0 tx=0).stop 2024-04-04T06:14:52.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.837+0000 7f9776200640 1 -- 172.21.15.159:0/9102920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9770072100 msgr2=0x7f9770145d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:52.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.837+0000 7f9776200640 1 --2- 172.21.15.159:0/9102920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9770072100 0x7f9770145d40 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f975c00c780 tx=0x7f975c00cc40 comp rx=0 tx=0).stop 2024-04-04T06:14:52.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.838+0000 7f9776200640 1 -- 172.21.15.159:0/9102920 shutdown_connections 2024-04-04T06:14:52.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.838+0000 7f9776200640 1 --2- 172.21.15.159:0/9102920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f977015aa90 0x7f977014ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:52.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.838+0000 7f9776200640 1 --2- 172.21.15.159:0/9102920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9770072100 0x7f9770145d40 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:52.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.838+0000 7f9776200640 1 --2- 172.21.15.159:0/9102920 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f975006c8f0 0x7f975006edb0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:52.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.838+0000 7f9776200640 1 --2- 172.21.15.159:0/9102920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97700717f0 0x7f977006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:52.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.838+0000 7f9776200640 1 -- 172.21.15.159:0/9102920 >> 172.21.15.159:0/9102920 conn(0x7f977006b990 msgr2=0x7f9770153800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:52.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.838+0000 7f9776200640 1 -- 172.21.15.159:0/9102920 shutdown_connections 2024-04-04T06:14:52.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:52.838+0000 7f9776200640 1 -- 172.21.15.159:0/9102920 wait complete. 2024-04-04T06:14:52.851 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:14:53.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.054+0000 7fac01e00640 1 Processor -- start 2024-04-04T06:14:53.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.054+0000 7fac01e00640 1 -- start start 2024-04-04T06:14:53.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.056+0000 7fac01e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fabfc15fc50 0x7fabfc0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:53.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.056+0000 7fac01e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fabfc0cc490 0x7fabfc0d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:53.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.056+0000 7fac01e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabfc0b3150 con 0x7fabfc0cc0b0 2024-04-04T06:14:53.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.056+0000 7fac01e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fabfc15c610 con 0x7fabfc0cc490 2024-04-04T06:14:53.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.056+0000 7fac01e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fabfc1600e0 con 0x7fabfc15fc50 2024-04-04T06:14:53.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.056+0000 7fac01400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fabfc0cc490 0x7fabfc0d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:53.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.056+0000 7fac00a00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fabfc0cc0b0 0x7fabfc06dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:36602/0 (socket says 172.21.15.159:36602) 2024-04-04T06:14:53.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.056+0000 7fac00a00640 1 -- 172.21.15.159:0/2580311958 learned_addr learned my addr 172.21.15.159:0/2580311958 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:53.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.056+0000 7fabfbe00640 1 --2- 172.21.15.159:0/2580311958 >> v2:172.21.15.178:3300/0 conn(0x7fabfc15fc50 0x7fabfc0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:53.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.057+0000 7fac01400640 1 -- 172.21.15.159:0/2580311958 >> v2:172.21.15.178:3300/0 conn(0x7fabfc15fc50 msgr2=0x7fabfc0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:53.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.057+0000 7fac01400640 1 --2- 172.21.15.159:0/2580311958 >> v2:172.21.15.178:3300/0 conn(0x7fabfc15fc50 0x7fabfc0cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.057+0000 7fac01400640 1 -- 172.21.15.159:0/2580311958 >> v1:172.21.15.159:6789/0 conn(0x7fabfc0cc0b0 legacy=0x7fabfc06dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:53.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.057+0000 7fac01400640 1 -- 172.21.15.159:0/2580311958 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabfc1603c0 con 0x7fabfc0cc490 2024-04-04T06:14:53.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.057+0000 7fabfb400640 1 -- 172.21.15.159:0/2580311958 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3990493825 0 0) 0x7fabfc0b3150 con 0x7fabfc0cc0b0 2024-04-04T06:14:53.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.058+0000 7fac01400640 1 --2- 172.21.15.159:0/2580311958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabfc0cc490 0x7fabfc0d0300 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fabfc06ee00 tx=0x7fabf402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02d1f2203fd2e53 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:53.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.058+0000 7fabfb400640 1 -- 172.21.15.159:0/2580311958 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4039070 con 0x7fabfc0cc490 2024-04-04T06:14:53.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.058+0000 7fabfb400640 1 -- 172.21.15.159:0/2580311958 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabf402cc80 con 0x7fabfc0cc490 2024-04-04T06:14:53.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.058+0000 7fabfb400640 1 -- 172.21.15.159:0/2580311958 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4034660 con 0x7fabfc0cc490 2024-04-04T06:14:53.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.059+0000 7fac01e00640 1 -- 172.21.15.159:0/2580311958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabfc0cc490 msgr2=0x7fabfc0d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:53.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.059+0000 7fac01e00640 1 --2- 172.21.15.159:0/2580311958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabfc0cc490 0x7fabfc0d0300 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fabfc06ee00 tx=0x7fabf402bed0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.059+0000 7fac01e00640 1 -- 172.21.15.159:0/2580311958 shutdown_connections 2024-04-04T06:14:53.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.059+0000 7fac01e00640 1 --2- 172.21.15.159:0/2580311958 >> v2:172.21.15.178:3300/0 conn(0x7fabfc15fc50 0x7fabfc0cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.059+0000 7fac01e00640 1 --2- 172.21.15.159:0/2580311958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabfc0cc490 0x7fabfc0d0300 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.059+0000 7fac01e00640 1 -- 172.21.15.159:0/2580311958 >> 172.21.15.159:0/2580311958 conn(0x7fabfc0ccad0 msgr2=0x7fabfc0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:53.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.059+0000 7fac01e00640 1 -- 172.21.15.159:0/2580311958 shutdown_connections 2024-04-04T06:14:53.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.059+0000 7fac01e00640 1 -- 172.21.15.159:0/2580311958 wait complete. 2024-04-04T06:14:53.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.061+0000 7fac01e00640 1 Processor -- start 2024-04-04T06:14:53.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.062+0000 7fac01e00640 1 -- start start 2024-04-04T06:14:53.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.063+0000 7fac01e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fabfc0cc0b0 0x7fabfc138d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:53.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.063+0000 7fac01e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabfc0cc490 0x7fabfc135660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:53.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.063+0000 7fac01e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabfc1527f0 0x7fabfc0b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:53.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.063+0000 7fac01e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc15ca70 con 0x7fabfc0cc490 2024-04-04T06:14:53.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.063+0000 7fac01e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc160980 con 0x7fabfc1527f0 2024-04-04T06:14:53.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.063+0000 7fac01e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fabfc15cbf0 con 0x7fabfc0cc0b0 2024-04-04T06:14:53.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.063+0000 7fabfbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabfc0cc490 0x7fabfc135660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:53.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.064+0000 7fabfbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabfc0cc490 0x7fabfc135660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37330/0 (socket says 172.21.15.159:37330) 2024-04-04T06:14:53.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.064+0000 7fabfbe00640 1 -- 172.21.15.159:0/2263892761 learned_addr learned my addr 172.21.15.159:0/2263892761 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:53.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.064+0000 7fac00a00640 1 --2- 172.21.15.159:0/2263892761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fabfc0cc0b0 0x7fabfc138d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:53.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.064+0000 7fabfbe00640 1 -- 172.21.15.159:0/2263892761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fabfc0cc0b0 msgr2=0x7fabfc138d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:53.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.064+0000 7fabfbe00640 1 --2- 172.21.15.159:0/2263892761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fabfc0cc0b0 0x7fabfc138d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.064+0000 7fabfbe00640 1 -- 172.21.15.159:0/2263892761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabfc1527f0 msgr2=0x7fabfc0b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:53.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.064+0000 7fabfbe00640 1 --2- 172.21.15.159:0/2263892761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabfc1527f0 0x7fabfc0b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.064+0000 7fabfbe00640 1 -- 172.21.15.159:0/2263892761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabfc135ba0 con 0x7fabfc0cc490 2024-04-04T06:14:53.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.064+0000 7fabfbe00640 1 --2- 172.21.15.159:0/2263892761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabfc0cc490 0x7fabfc135660 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fabe800d620 tx=0x7fabe800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:53.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.064+0000 7fabf8c00640 1 -- 172.21.15.159:0/2263892761 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe800b620 con 0x7fabfc0cc490 2024-04-04T06:14:53.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.065+0000 7fabf8c00640 1 -- 172.21.15.159:0/2263892761 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabe8019070 con 0x7fabfc0cc490 2024-04-04T06:14:53.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.065+0000 7fac01e00640 1 -- 172.21.15.159:0/2263892761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabfc139710 con 0x7fabfc0cc490 2024-04-04T06:14:53.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.065+0000 7fabf8c00640 1 -- 172.21.15.159:0/2263892761 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe800ba00 con 0x7fabfc0cc490 2024-04-04T06:14:53.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.065+0000 7fac01e00640 1 -- 172.21.15.159:0/2263892761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabfc139d70 con 0x7fabfc0cc490 2024-04-04T06:14:53.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.066+0000 7fac01e00640 1 -- 172.21.15.159:0/2263892761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabfc075940 con 0x7fabfc0cc490 2024-04-04T06:14:53.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.067+0000 7fabf8c00640 1 -- 172.21.15.159:0/2263892761 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fabe800bba0 con 0x7fabfc0cc490 2024-04-04T06:14:53.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.068+0000 7fabf8c00640 1 --2- 172.21.15.159:0/2263892761 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fabdc06c8f0 0x7fabdc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:53.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.068+0000 7fabf8c00640 1 -- 172.21.15.159:0/2263892761 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7fabe808c0d0 con 0x7fabfc0cc490 2024-04-04T06:14:53.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.068+0000 7fac00a00640 1 --2- 172.21.15.159:0/2263892761 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fabdc06c8f0 0x7fabdc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:53.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.069+0000 7fac00a00640 1 --2- 172.21.15.159:0/2263892761 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fabdc06c8f0 0x7fabdc06edb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fabf0007fd0 tx=0x7fabf0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:53.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.072+0000 7fabf8c00640 1 -- 172.21.15.159:0/2263892761 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fabe805a370 con 0x7fabfc0cc490 2024-04-04T06:14:53.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.284+0000 7fac01e00640 1 -- 172.21.15.159:0/2263892761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fabfc06f2a0 con 0x7fabfc0cc490 2024-04-04T06:14:53.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.285+0000 7fabf8c00640 1 -- 172.21.15.159:0/2263892761 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+17033 (secure 0 0 0) 0x7fabe8060030 con 0x7fabfc0cc490 2024-04-04T06:14:53.280 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:14:53.280 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":89,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:14:04.757031+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:12:33.658405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-04T06:12:34.856266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.3d","mappings":[{"from":6,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6835/3796123128":"2024-04-05T06:14:04.756982+0000","172.21.15.159:6833/3796123128":"2024-04-05T06:14:04.756982+0000","172.21.15.159:6835/3484779950":"2024-04-05T06:12:49.205338+0000","172.21.15.159:6833/3484779950":"2024-04-05T06:12:49.205338+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:14:53.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.288+0000 7fac01e00640 1 -- 172.21.15.159:0/2263892761 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fabdc06c8f0 msgr2=0x7fabdc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:53.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.288+0000 7fac01e00640 1 --2- 172.21.15.159:0/2263892761 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fabdc06c8f0 0x7fabdc06edb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fabf0007fd0 tx=0x7fabf0009040 comp rx=0 tx=0).stop 2024-04-04T06:14:53.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.288+0000 7fac01e00640 1 -- 172.21.15.159:0/2263892761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabfc0cc490 msgr2=0x7fabfc135660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:53.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.288+0000 7fac01e00640 1 --2- 172.21.15.159:0/2263892761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabfc0cc490 0x7fabfc135660 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fabe800d620 tx=0x7fabe800dae0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.288+0000 7fac01e00640 1 -- 172.21.15.159:0/2263892761 shutdown_connections 2024-04-04T06:14:53.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.288+0000 7fac01e00640 1 --2- 172.21.15.159:0/2263892761 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fabdc06c8f0 0x7fabdc06edb0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.288+0000 7fac01e00640 1 --2- 172.21.15.159:0/2263892761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabfc1527f0 0x7fabfc0b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.288+0000 7fac01e00640 1 --2- 172.21.15.159:0/2263892761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabfc0cc490 0x7fabfc135660 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.288+0000 7fac01e00640 1 --2- 172.21.15.159:0/2263892761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fabfc0cc0b0 0x7fabfc138d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.288+0000 7fac01e00640 1 -- 172.21.15.159:0/2263892761 >> 172.21.15.159:0/2263892761 conn(0x7fabfc0ccad0 msgr2=0x7fabfc072790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:53.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.288+0000 7fac01e00640 1 -- 172.21.15.159:0/2263892761 shutdown_connections 2024-04-04T06:14:53.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.288+0000 7fac01e00640 1 -- 172.21.15.159:0/2263892761 wait complete. 2024-04-04T06:14:53.304 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-04T06:14:53.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.579+0000 7fac7d800640 1 Processor -- start 2024-04-04T06:14:53.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.579+0000 7fac7d800640 1 -- start start 2024-04-04T06:14:53.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.581+0000 7fac7d800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fac7814ba20 0x7fac780b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:53.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.581+0000 7fac7d800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fac7814c1b0 0x7fac78166290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:53.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.581+0000 7fac7d800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac780b33e0 con 0x7fac780cfda0 2024-04-04T06:14:53.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.581+0000 7fac7d800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fac780cde20 con 0x7fac7814ba20 2024-04-04T06:14:53.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.581+0000 7fac7d800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fac78166ae0 con 0x7fac7814c1b0 2024-04-04T06:14:53.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.581+0000 7fac7ce00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fac780cfda0 0x7fac780d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:36612/0 (socket says 172.21.15.159:36612) 2024-04-04T06:14:53.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.581+0000 7fac7ce00640 1 -- 172.21.15.159:0/3769701172 learned_addr learned my addr 172.21.15.159:0/3769701172 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:53.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.582+0000 7fac77e00640 1 --2- 172.21.15.159:0/3769701172 >> v2:172.21.15.159:3300/0 conn(0x7fac7814ba20 0x7fac780b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:53.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.582+0000 7fac6f400640 1 --2- 172.21.15.159:0/3769701172 >> v2:172.21.15.178:3300/0 conn(0x7fac7814c1b0 0x7fac78166290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:53.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.582+0000 7fac77400640 1 -- 172.21.15.159:0/3769701172 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3175834389 0 0) 0x7fac780b33e0 con 0x7fac780cfda0 2024-04-04T06:14:53.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.582+0000 7fac77e00640 1 -- 172.21.15.159:0/3769701172 >> v2:172.21.15.178:3300/0 conn(0x7fac7814c1b0 msgr2=0x7fac78166290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:53.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.582+0000 7fac77e00640 1 --2- 172.21.15.159:0/3769701172 >> v2:172.21.15.178:3300/0 conn(0x7fac7814c1b0 0x7fac78166290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.582+0000 7fac77e00640 1 -- 172.21.15.159:0/3769701172 >> v1:172.21.15.159:6789/0 conn(0x7fac780cfda0 legacy=0x7fac780d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:53.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.582+0000 7fac77e00640 1 -- 172.21.15.159:0/3769701172 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac78166dc0 con 0x7fac7814ba20 2024-04-04T06:14:53.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.583+0000 7fac77e00640 1 --2- 172.21.15.159:0/3769701172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac7814ba20 0x7fac780b2ea0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fac7814c640 tx=0x7fac6802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac25d44b1e947c73 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:53.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.584+0000 7fac77400640 1 -- 172.21.15.159:0/3769701172 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac68035020 con 0x7fac7814ba20 2024-04-04T06:14:53.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.584+0000 7fac77400640 1 -- 172.21.15.159:0/3769701172 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac68031830 con 0x7fac7814ba20 2024-04-04T06:14:53.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.584+0000 7fac77400640 1 -- 172.21.15.159:0/3769701172 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac68031c00 con 0x7fac7814ba20 2024-04-04T06:14:53.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.584+0000 7fac7d800640 1 -- 172.21.15.159:0/3769701172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac7814ba20 msgr2=0x7fac780b2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:53.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.584+0000 7fac7d800640 1 --2- 172.21.15.159:0/3769701172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac7814ba20 0x7fac780b2ea0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fac7814c640 tx=0x7fac6802b8d0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.584+0000 7fac7d800640 1 -- 172.21.15.159:0/3769701172 shutdown_connections 2024-04-04T06:14:53.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.584+0000 7fac7d800640 1 --2- 172.21.15.159:0/3769701172 >> v2:172.21.15.178:3300/0 conn(0x7fac7814c1b0 0x7fac78166290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.584+0000 7fac7d800640 1 --2- 172.21.15.159:0/3769701172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac7814ba20 0x7fac780b2ea0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.584+0000 7fac7d800640 1 -- 172.21.15.159:0/3769701172 >> 172.21.15.159:0/3769701172 conn(0x7fac780ccad0 msgr2=0x7fac780cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:53.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.584+0000 7fac7d800640 1 -- 172.21.15.159:0/3769701172 shutdown_connections 2024-04-04T06:14:53.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.584+0000 7fac7d800640 1 -- 172.21.15.159:0/3769701172 wait complete. 2024-04-04T06:14:53.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.586+0000 7fac7d800640 1 Processor -- start 2024-04-04T06:14:53.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.586+0000 7fac7d800640 1 -- start start 2024-04-04T06:14:53.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.587+0000 7fac7d800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac780cfda0 0x7fac780ab360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:53.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.587+0000 7fac7d800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac7814ba20 0x7fac780ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:53.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.587+0000 7fac7d800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac7814c1b0 0x7fac7813f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:53.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.587+0000 7fac7d800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fac781667d0 con 0x7fac780cfda0 2024-04-04T06:14:53.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.587+0000 7fac7d800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fac780ce0c0 con 0x7fac7814c1b0 2024-04-04T06:14:53.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.587+0000 7fac7d800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fac7814be00 con 0x7fac7814ba20 2024-04-04T06:14:53.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.587+0000 7fac77e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac780cfda0 0x7fac780ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:53.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.587+0000 7fac77e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac780cfda0 0x7fac780ab360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37346/0 (socket says 172.21.15.159:37346) 2024-04-04T06:14:53.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.587+0000 7fac77e00640 1 -- 172.21.15.159:0/2758116330 learned_addr learned my addr 172.21.15.159:0/2758116330 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:53.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.588+0000 7fac77e00640 1 -- 172.21.15.159:0/2758116330 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac7814ba20 msgr2=0x7fac780ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:53.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.588+0000 7fac6f400640 1 --2- 172.21.15.159:0/2758116330 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac7814ba20 0x7fac780ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:53.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.588+0000 7fac7ce00640 1 --2- 172.21.15.159:0/2758116330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac7814c1b0 0x7fac7813f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:53.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.588+0000 7fac77e00640 1 --2- 172.21.15.159:0/2758116330 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac7814ba20 0x7fac780ad7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.588+0000 7fac77e00640 1 -- 172.21.15.159:0/2758116330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac7814c1b0 msgr2=0x7fac7813f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:53.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.588+0000 7fac77e00640 1 --2- 172.21.15.159:0/2758116330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac7814c1b0 0x7fac7813f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.588+0000 7fac77e00640 1 -- 172.21.15.159:0/2758116330 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac780ab940 con 0x7fac780cfda0 2024-04-04T06:14:53.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.588+0000 7fac7ce00640 1 --2- 172.21.15.159:0/2758116330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac7814c1b0 0x7fac7813f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:53.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.588+0000 7fac77e00640 1 --2- 172.21.15.159:0/2758116330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac780cfda0 0x7fac780ab360 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fac68000c00 tx=0x7fac680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:53.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.589+0000 7fac74c00640 1 -- 172.21.15.159:0/2758116330 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac68035070 con 0x7fac780cfda0 2024-04-04T06:14:53.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.589+0000 7fac74c00640 1 -- 172.21.15.159:0/2758116330 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac68004250 con 0x7fac780cfda0 2024-04-04T06:14:53.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.589+0000 7fac7d800640 1 -- 172.21.15.159:0/2758116330 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac780abc20 con 0x7fac780cfda0 2024-04-04T06:14:53.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.589+0000 7fac74c00640 1 -- 172.21.15.159:0/2758116330 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6803a630 con 0x7fac780cfda0 2024-04-04T06:14:53.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.589+0000 7fac7d800640 1 -- 172.21.15.159:0/2758116330 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac780ac4c0 con 0x7fac780cfda0 2024-04-04T06:14:53.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.590+0000 7fac7d800640 1 -- 172.21.15.159:0/2758116330 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac78075940 con 0x7fac780cfda0 2024-04-04T06:14:53.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.591+0000 7fac74c00640 1 -- 172.21.15.159:0/2758116330 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac68038020 con 0x7fac780cfda0 2024-04-04T06:14:53.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.592+0000 7fac74c00640 1 --2- 172.21.15.159:0/2758116330 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fac5406c8f0 0x7fac5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:53.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.592+0000 7fac74c00640 1 -- 172.21.15.159:0/2758116330 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5939+0+0 (secure 0 0 0) 0x7fac680ada40 con 0x7fac780cfda0 2024-04-04T06:14:53.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.593+0000 7fac6f400640 1 --2- 172.21.15.159:0/2758116330 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fac5406c8f0 0x7fac5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:53.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.594+0000 7fac6f400640 1 --2- 172.21.15.159:0/2758116330 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fac5406c8f0 0x7fac5406edb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fac780af4b0 tx=0x7fac60008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:53.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.596+0000 7fac74c00640 1 -- 172.21.15.159:0/2758116330 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fac68040070 con 0x7fac780cfda0 2024-04-04T06:14:53.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.838+0000 7fac7d800640 1 -- 172.21.15.159:0/2758116330 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fac78162a80 con 0x7fac780cfda0 2024-04-04T06:14:53.833 INFO:tasks.ceph.mon.a.smithi159.stderr:2024-04-04T06:14:53.839+0000 7f1995c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-04T06:14:53.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.842+0000 7fac74c00640 1 -- 172.21.15.159:0/2758116330 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v77) ==== 161+0+0 (secure 0 0 0) 0x7fac6807bc50 con 0x7fac780cfda0 2024-04-04T06:14:53.836 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-04T06:14:53.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.846+0000 7fac7d800640 1 -- 172.21.15.159:0/2758116330 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fac5406c8f0 msgr2=0x7fac5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:53.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.846+0000 7fac7d800640 1 --2- 172.21.15.159:0/2758116330 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fac5406c8f0 0x7fac5406edb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fac780af4b0 tx=0x7fac60008040 comp rx=0 tx=0).stop 2024-04-04T06:14:53.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.846+0000 7fac7d800640 1 -- 172.21.15.159:0/2758116330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac780cfda0 msgr2=0x7fac780ab360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:53.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.846+0000 7fac7d800640 1 --2- 172.21.15.159:0/2758116330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac780cfda0 0x7fac780ab360 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fac68000c00 tx=0x7fac680046c0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.847+0000 7fac7d800640 1 -- 172.21.15.159:0/2758116330 shutdown_connections 2024-04-04T06:14:53.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.847+0000 7fac7d800640 1 --2- 172.21.15.159:0/2758116330 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fac5406c8f0 0x7fac5406edb0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.847+0000 7fac7d800640 1 --2- 172.21.15.159:0/2758116330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac7814c1b0 0x7fac7813f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.847+0000 7fac7d800640 1 --2- 172.21.15.159:0/2758116330 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac7814ba20 0x7fac780ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.847+0000 7fac7d800640 1 --2- 172.21.15.159:0/2758116330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac780cfda0 0x7fac780ab360 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:53.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.847+0000 7fac7d800640 1 -- 172.21.15.159:0/2758116330 >> 172.21.15.159:0/2758116330 conn(0x7fac780ccad0 msgr2=0x7fac780b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:53.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.847+0000 7fac7d800640 1 -- 172.21.15.159:0/2758116330 shutdown_connections 2024-04-04T06:14:53.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:53.847+0000 7fac7d800640 1 -- 172.21.15.159:0/2758116330 wait complete. 2024-04-04T06:14:53.851 INFO:tasks.ceph.mds.a.smithi159.stderr: -2600> 2024-04-04T06:14:10.779+0000 7fce9e800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:14:53.853 INFO:tasks.ceph.mds.a.smithi159.stderr: -2050> 2024-04-04T06:14:13.657+0000 7fce99800640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x560faf987180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-04T06:14:53.861 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-04T06:14:53.906 INFO:tasks.ceph.mds.a.smithi159.stdout:starting mds.a at 2024-04-04T06:14:54.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.049+0000 7f6048400640 1 Processor -- start 2024-04-04T06:14:54.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.049+0000 7f6048400640 1 -- start start 2024-04-04T06:14:54.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.051+0000 7f6048400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f60401452f0 0x7f60401456d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:54.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.051+0000 7f6048400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60400ae4a0 con 0x7f604006bc80 2024-04-04T06:14:54.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.051+0000 7f6048400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f604015c990 con 0x7f60400ae020 2024-04-04T06:14:54.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.051+0000 7f6048400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f60400afcf0 con 0x7f60401452f0 2024-04-04T06:14:54.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.051+0000 7f6045000640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f60400ae020 0x7f604006b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.159:40328/0 (socket says 172.21.15.159:40328) 2024-04-04T06:14:54.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.051+0000 7f6046400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f604006bc80 0x7f60400b4140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:39966/0 (socket says 172.21.15.159:39966) 2024-04-04T06:14:54.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.051+0000 7f6045a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f60401452f0 0x7f60401456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:54.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.051+0000 7f6045000640 1 -- 172.21.15.159:0/1983692818 learned_addr learned my addr 172.21.15.159:0/1983692818 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:54.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.052+0000 7f6045a00640 1 -- 172.21.15.159:0/1983692818 >> v1:172.21.15.178:6789/0 conn(0x7f60400ae020 legacy=0x7f604006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:54.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.052+0000 7f6045a00640 1 -- 172.21.15.159:0/1983692818 >> v1:172.21.15.159:6789/0 conn(0x7f604006bc80 legacy=0x7f60400b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:54.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.052+0000 7f603fe00640 1 -- 172.21.15.159:0/1983692818 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3122401379 0 0) 0x7f60400ae4a0 con 0x7f604006bc80 2024-04-04T06:14:54.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.052+0000 7f6045a00640 1 -- 172.21.15.159:0/1983692818 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60400affd0 con 0x7f60401452f0 2024-04-04T06:14:54.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.054+0000 7f6045a00640 1 --2- 172.21.15.159:0/1983692818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60401452f0 0x7f60401456d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f60300097a0 tx=0x7f603002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=487f682d272474f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:54.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.054+0000 7f603fe00640 1 -- 172.21.15.159:0/1983692818 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030035020 con 0x7f60401452f0 2024-04-04T06:14:54.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.054+0000 7f603fe00640 1 -- 172.21.15.159:0/1983692818 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6030031830 con 0x7f60401452f0 2024-04-04T06:14:54.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.054+0000 7f603fe00640 1 -- 172.21.15.159:0/1983692818 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030031c00 con 0x7f60401452f0 2024-04-04T06:14:54.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.055+0000 7f6048400640 1 -- 172.21.15.159:0/1983692818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60401452f0 msgr2=0x7f60401456d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:54.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.055+0000 7f6048400640 1 --2- 172.21.15.159:0/1983692818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60401452f0 0x7f60401456d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f60300097a0 tx=0x7f603002b8d0 comp rx=0 tx=0).stop 2024-04-04T06:14:54.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.055+0000 7f6048400640 1 -- 172.21.15.159:0/1983692818 shutdown_connections 2024-04-04T06:14:54.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.055+0000 7f6048400640 1 --2- 172.21.15.159:0/1983692818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60401452f0 0x7f60401456d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:54.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.055+0000 7f6048400640 1 -- 172.21.15.159:0/1983692818 >> 172.21.15.159:0/1983692818 conn(0x7f60400b6560 msgr2=0x7f60400b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:54.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.055+0000 7f6048400640 1 -- 172.21.15.159:0/1983692818 shutdown_connections 2024-04-04T06:14:54.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.055+0000 7f6048400640 1 -- 172.21.15.159:0/1983692818 wait complete. 2024-04-04T06:14:54.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.057+0000 7f6048400640 1 Processor -- start 2024-04-04T06:14:54.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.057+0000 7f6048400640 1 -- start start 2024-04-04T06:14:54.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.058+0000 7f6048400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f604006bc80 0x7f604013adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:54.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.058+0000 7f6048400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60400ae020 0x7f604015e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:54.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.058+0000 7f6048400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60401452f0 0x7f60400b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:54.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.058+0000 7f6048400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f60400b0870 con 0x7f60401452f0 2024-04-04T06:14:54.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.058+0000 7f6048400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f60400b0590 con 0x7f604006bc80 2024-04-04T06:14:54.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.058+0000 7f6048400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f60400af910 con 0x7f60400ae020 2024-04-04T06:14:54.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.058+0000 7f6046400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60401452f0 0x7f60400b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:54.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.058+0000 7f6046400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60401452f0 0x7f60400b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37494/0 (socket says 172.21.15.159:37494) 2024-04-04T06:14:54.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.058+0000 7f6046400640 1 -- 172.21.15.159:0/3466140501 learned_addr learned my addr 172.21.15.159:0/3466140501 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:54.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.059+0000 7f6045a00640 1 --2- 172.21.15.159:0/3466140501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f604006bc80 0x7f604013adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:54.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.059+0000 7f6046400640 1 -- 172.21.15.159:0/3466140501 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60400ae020 msgr2=0x7f604015e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:54.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.059+0000 7f6045000640 1 --2- 172.21.15.159:0/3466140501 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60400ae020 0x7f604015e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:54.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.059+0000 7f6046400640 1 --2- 172.21.15.159:0/3466140501 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60400ae020 0x7f604015e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:54.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.059+0000 7f6046400640 1 -- 172.21.15.159:0/3466140501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f604006bc80 msgr2=0x7f604013adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:54.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.059+0000 7f6046400640 1 --2- 172.21.15.159:0/3466140501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f604006bc80 0x7f604013adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:54.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.059+0000 7f6046400640 1 -- 172.21.15.159:0/3466140501 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604015f7b0 con 0x7f60401452f0 2024-04-04T06:14:54.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.059+0000 7f6046400640 1 --2- 172.21.15.159:0/3466140501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60401452f0 0x7f60400b4f60 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f603800c700 tx=0x7f603800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:54.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.059+0000 7f603d600640 1 -- 172.21.15.159:0/3466140501 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038018070 con 0x7f60401452f0 2024-04-04T06:14:54.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.059+0000 7f603d600640 1 -- 172.21.15.159:0/3466140501 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f603800ce10 con 0x7f60401452f0 2024-04-04T06:14:54.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.059+0000 7f603d600640 1 -- 172.21.15.159:0/3466140501 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603800a880 con 0x7f60401452f0 2024-04-04T06:14:54.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.059+0000 7f6048400640 1 -- 172.21.15.159:0/3466140501 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604015fa90 con 0x7f60401452f0 2024-04-04T06:14:54.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.060+0000 7f6048400640 1 -- 172.21.15.159:0/3466140501 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60401589d0 con 0x7f60401452f0 2024-04-04T06:14:54.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.061+0000 7f6048400640 1 -- 172.21.15.159:0/3466140501 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60400685c0 con 0x7f60401452f0 2024-04-04T06:14:54.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.062+0000 7f603d600640 1 -- 172.21.15.159:0/3466140501 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f603800aa20 con 0x7f60401452f0 2024-04-04T06:14:54.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.062+0000 7f603d600640 1 --2- 172.21.15.159:0/3466140501 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f602006c920 0x7f602006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:54.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.063+0000 7f603d600640 1 -- 172.21.15.159:0/3466140501 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6025+0+0 (secure 0 0 0) 0x7f603808b200 con 0x7f60401452f0 2024-04-04T06:14:54.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.063+0000 7f6045a00640 1 --2- 172.21.15.159:0/3466140501 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f602006c920 0x7f602006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:54.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.064+0000 7f6045a00640 1 --2- 172.21.15.159:0/3466140501 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f602006c920 0x7f602006ede0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6030036000 tx=0x7f6030008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:54.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.067+0000 7f603d600640 1 -- 172.21.15.159:0/3466140501 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6038059450 con 0x7f60401452f0 2024-04-04T06:14:54.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.307+0000 7f6048400640 1 -- 172.21.15.159:0/3466140501 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f60400b6280 con 0x7f60401452f0 2024-04-04T06:14:54.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.845+0000 7f603d600640 1 -- 172.21.15.159:0/3466140501 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v78) ==== 104+0+0 (secure 0 0 0) 0x7f6038058d50 con 0x7f60401452f0 2024-04-04T06:14:54.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.849+0000 7f6048400640 1 -- 172.21.15.159:0/3466140501 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f602006c920 msgr2=0x7f602006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:54.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.849+0000 7f6048400640 1 --2- 172.21.15.159:0/3466140501 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f602006c920 0x7f602006ede0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6030036000 tx=0x7f6030008030 comp rx=0 tx=0).stop 2024-04-04T06:14:54.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.849+0000 7f6048400640 1 -- 172.21.15.159:0/3466140501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60401452f0 msgr2=0x7f60400b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:54.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.849+0000 7f6048400640 1 --2- 172.21.15.159:0/3466140501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60401452f0 0x7f60400b4f60 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f603800c700 tx=0x7f603800cbc0 comp rx=0 tx=0).stop 2024-04-04T06:14:54.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.849+0000 7f6048400640 1 -- 172.21.15.159:0/3466140501 shutdown_connections 2024-04-04T06:14:54.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.849+0000 7f6048400640 1 --2- 172.21.15.159:0/3466140501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60401452f0 0x7f60400b4f60 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:54.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.849+0000 7f6048400640 1 --2- 172.21.15.159:0/3466140501 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60400ae020 0x7f604015e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:54.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.849+0000 7f6048400640 1 --2- 172.21.15.159:0/3466140501 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f602006c920 0x7f602006ede0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:54.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.849+0000 7f6048400640 1 --2- 172.21.15.159:0/3466140501 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f604006bc80 0x7f604013adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:54.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.849+0000 7f6048400640 1 -- 172.21.15.159:0/3466140501 >> 172.21.15.159:0/3466140501 conn(0x7f60400b6560 msgr2=0x7f60401401f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:54.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.850+0000 7f6048400640 1 -- 172.21.15.159:0/3466140501 shutdown_connections 2024-04-04T06:14:54.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:54.850+0000 7f6048400640 1 -- 172.21.15.159:0/3466140501 wait complete. 2024-04-04T06:14:54.866 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-04T06:14:55.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.127+0000 7fab4ec00640 1 Processor -- start 2024-04-04T06:14:55.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.127+0000 7fab4ec00640 1 -- start start 2024-04-04T06:14:55.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.128+0000 7fab4ec00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fab48072720 0x7fab480b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:55.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.128+0000 7fab4ec00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fab48072b00 0x7fab4806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:55.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.128+0000 7fab4ec00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab4806db10 0x7fab4806df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:55.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.128+0000 7fab4ec00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fab480b3e80 con 0x7fab48072b00 2024-04-04T06:14:55.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.129+0000 7fab4ec00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fab4806ea10 con 0x7fab48072720 2024-04-04T06:14:55.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.129+0000 7fab4ec00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fab4806ecf0 con 0x7fab4806db10 2024-04-04T06:14:55.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.129+0000 7fab47400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fab48072b00 0x7fab4806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:55.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.129+0000 7fab47400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fab48072b00 0x7fab4806d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37506/0 (socket says 172.21.15.159:37506) 2024-04-04T06:14:55.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.129+0000 7fab47400640 1 -- 172.21.15.159:0/4115988457 learned_addr learned my addr 172.21.15.159:0/4115988457 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:55.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.129+0000 7fab47e00640 1 --2- 172.21.15.159:0/4115988457 >> v2:172.21.15.178:3300/0 conn(0x7fab48072720 0x7fab480b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:55.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.129+0000 7fab4cc00640 1 --2- 172.21.15.159:0/4115988457 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab4806db10 0x7fab4806df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:55.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.129+0000 7fab47400640 1 -- 172.21.15.159:0/4115988457 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab4806db10 msgr2=0x7fab4806df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:55.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.129+0000 7fab47400640 1 --2- 172.21.15.159:0/4115988457 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab4806db10 0x7fab4806df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:55.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.129+0000 7fab47400640 1 -- 172.21.15.159:0/4115988457 >> v2:172.21.15.178:3300/0 conn(0x7fab48072720 msgr2=0x7fab480b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:55.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.129+0000 7fab47400640 1 --2- 172.21.15.159:0/4115988457 >> v2:172.21.15.178:3300/0 conn(0x7fab48072720 0x7fab480b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:55.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.129+0000 7fab47400640 1 -- 172.21.15.159:0/4115988457 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4806efd0 con 0x7fab48072b00 2024-04-04T06:14:55.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.129+0000 7fab47e00640 1 --2- 172.21.15.159:0/4115988457 >> v2:172.21.15.178:3300/0 conn(0x7fab48072720 0x7fab480b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:55.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.130+0000 7fab47400640 1 --2- 172.21.15.159:0/4115988457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab48072b00 0x7fab4806d5d0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fab48073390 tx=0x7fab3402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43e78ae51d39824 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:55.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.131+0000 7fab46a00640 1 -- 172.21.15.159:0/4115988457 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab34035020 con 0x7fab48072b00 2024-04-04T06:14:55.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.131+0000 7fab46a00640 1 -- 172.21.15.159:0/4115988457 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab34031870 con 0x7fab48072b00 2024-04-04T06:14:55.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.131+0000 7fab46a00640 1 -- 172.21.15.159:0/4115988457 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab34031c40 con 0x7fab48072b00 2024-04-04T06:14:55.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.131+0000 7fab4ec00640 1 -- 172.21.15.159:0/4115988457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab48072b00 msgr2=0x7fab4806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:55.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.131+0000 7fab4ec00640 1 --2- 172.21.15.159:0/4115988457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab48072b00 0x7fab4806d5d0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fab48073390 tx=0x7fab3402b910 comp rx=0 tx=0).stop 2024-04-04T06:14:55.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.131+0000 7fab4ec00640 1 -- 172.21.15.159:0/4115988457 shutdown_connections 2024-04-04T06:14:55.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.131+0000 7fab4ec00640 1 --2- 172.21.15.159:0/4115988457 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab4806db10 0x7fab4806df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:55.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.131+0000 7fab4ec00640 1 --2- 172.21.15.159:0/4115988457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab48072b00 0x7fab4806d5d0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:55.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.131+0000 7fab4ec00640 1 --2- 172.21.15.159:0/4115988457 >> v2:172.21.15.178:3300/0 conn(0x7fab48072720 0x7fab480b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:55.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.131+0000 7fab4ec00640 1 -- 172.21.15.159:0/4115988457 >> 172.21.15.159:0/4115988457 conn(0x7fab4806bba0 msgr2=0x7fab480b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:55.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.131+0000 7fab4ec00640 1 -- 172.21.15.159:0/4115988457 shutdown_connections 2024-04-04T06:14:55.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.131+0000 7fab4ec00640 1 -- 172.21.15.159:0/4115988457 wait complete. 2024-04-04T06:14:55.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.133+0000 7fab4ec00640 1 Processor -- start 2024-04-04T06:14:55.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.133+0000 7fab4ec00640 1 -- start start 2024-04-04T06:14:55.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.134+0000 7fab4ec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab4806db10 0x7fab480cf430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:55.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.134+0000 7fab4ec00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab48072720 0x7fab48152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:55.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.134+0000 7fab4ec00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab48072b00 0x7fab481f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:55.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.134+0000 7fab4ec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fab4806e690 con 0x7fab4806db10 2024-04-04T06:14:55.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.134+0000 7fab4ec00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fab4806f590 con 0x7fab48072720 2024-04-04T06:14:55.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.134+0000 7fab4ec00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fab48072f70 con 0x7fab48072b00 2024-04-04T06:14:55.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.134+0000 7fab47e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab4806db10 0x7fab480cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:55.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.134+0000 7fab47e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab4806db10 0x7fab480cf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37514/0 (socket says 172.21.15.159:37514) 2024-04-04T06:14:55.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.134+0000 7fab47e00640 1 -- 172.21.15.159:0/998061289 learned_addr learned my addr 172.21.15.159:0/998061289 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:55.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.135+0000 7fab47e00640 1 -- 172.21.15.159:0/998061289 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab48072b00 msgr2=0x7fab481f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:55.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.135+0000 7fab47e00640 1 --2- 172.21.15.159:0/998061289 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab48072b00 0x7fab481f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:55.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.135+0000 7fab47400640 1 --2- 172.21.15.159:0/998061289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab48072720 0x7fab48152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:55.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.135+0000 7fab47e00640 1 -- 172.21.15.159:0/998061289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab48072720 msgr2=0x7fab48152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:55.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.135+0000 7fab47e00640 1 --2- 172.21.15.159:0/998061289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab48072720 0x7fab48152450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:55.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.135+0000 7fab47e00640 1 -- 172.21.15.159:0/998061289 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4809bea0 con 0x7fab4806db10 2024-04-04T06:14:55.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.135+0000 7fab47e00640 1 --2- 172.21.15.159:0/998061289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab4806db10 0x7fab480cf430 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fab3800c7f0 tx=0x7fab3800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:55.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.135+0000 7fab3fe00640 1 -- 172.21.15.159:0/998061289 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab38015070 con 0x7fab4806db10 2024-04-04T06:14:55.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.135+0000 7fab3fe00640 1 -- 172.21.15.159:0/998061289 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab38007820 con 0x7fab4806db10 2024-04-04T06:14:55.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.135+0000 7fab3fe00640 1 -- 172.21.15.159:0/998061289 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab38007bd0 con 0x7fab4806db10 2024-04-04T06:14:55.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.135+0000 7fab4ec00640 1 -- 172.21.15.159:0/998061289 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab4809c180 con 0x7fab4806db10 2024-04-04T06:14:55.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.136+0000 7fab4ec00640 1 -- 172.21.15.159:0/998061289 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab4809ca20 con 0x7fab4806db10 2024-04-04T06:14:55.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.137+0000 7fab4ec00640 1 -- 172.21.15.159:0/998061289 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab4809c350 con 0x7fab4806db10 2024-04-04T06:14:55.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.137+0000 7fab3fe00640 1 -- 172.21.15.159:0/998061289 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab38018070 con 0x7fab4806db10 2024-04-04T06:14:55.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.138+0000 7fab3fe00640 1 --2- 172.21.15.159:0/998061289 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fab2806c920 0x7fab2806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:55.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.138+0000 7fab3fe00640 1 -- 172.21.15.159:0/998061289 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6025+0+0 (secure 0 0 0) 0x7fab3808be00 con 0x7fab4806db10 2024-04-04T06:14:55.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.139+0000 7fab47400640 1 --2- 172.21.15.159:0/998061289 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fab2806c920 0x7fab2806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:55.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.140+0000 7fab47400640 1 --2- 172.21.15.159:0/998061289 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fab2806c920 0x7fab2806ede0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fab34036000 tx=0x7fab34008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:55.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.143+0000 7fab3fe00640 1 -- 172.21.15.159:0/998061289 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fab3805a050 con 0x7fab4806db10 2024-04-04T06:14:55.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:55.340+0000 7fab4ec00640 1 -- 172.21.15.159:0/998061289 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fab480b2720 con 0x7fab4806db10 2024-04-04T06:14:56.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.115+0000 7fab3fe00640 1 -- 172.21.15.159:0/998061289 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) ==== 181+0+0 (secure 0 0 0) 0x7fab480b2720 con 0x7fab4806db10 2024-04-04T06:14:56.110 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_metadata' removed 2024-04-04T06:14:56.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.119+0000 7fab4ec00640 1 -- 172.21.15.159:0/998061289 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fab2806c920 msgr2=0x7fab2806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:56.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.119+0000 7fab4ec00640 1 --2- 172.21.15.159:0/998061289 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fab2806c920 0x7fab2806ede0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fab34036000 tx=0x7fab34008030 comp rx=0 tx=0).stop 2024-04-04T06:14:56.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.119+0000 7fab4ec00640 1 -- 172.21.15.159:0/998061289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab4806db10 msgr2=0x7fab480cf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:56.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.119+0000 7fab4ec00640 1 --2- 172.21.15.159:0/998061289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab4806db10 0x7fab480cf430 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fab3800c7f0 tx=0x7fab3800ccb0 comp rx=0 tx=0).stop 2024-04-04T06:14:56.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.119+0000 7fab4ec00640 1 -- 172.21.15.159:0/998061289 shutdown_connections 2024-04-04T06:14:56.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.119+0000 7fab4ec00640 1 --2- 172.21.15.159:0/998061289 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab48072b00 0x7fab481f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:56.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.119+0000 7fab4ec00640 1 --2- 172.21.15.159:0/998061289 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fab2806c920 0x7fab2806ede0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:56.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.119+0000 7fab4ec00640 1 --2- 172.21.15.159:0/998061289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab48072720 0x7fab48152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:56.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.119+0000 7fab4ec00640 1 --2- 172.21.15.159:0/998061289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab4806db10 0x7fab480cf430 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:56.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.119+0000 7fab4ec00640 1 -- 172.21.15.159:0/998061289 >> 172.21.15.159:0/998061289 conn(0x7fab4806bba0 msgr2=0x7fab481539a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:56.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.119+0000 7fab4ec00640 1 -- 172.21.15.159:0/998061289 shutdown_connections 2024-04-04T06:14:56.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.119+0000 7fab4ec00640 1 -- 172.21.15.159:0/998061289 wait complete. 2024-04-04T06:14:56.136 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-04T06:14:56.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.348+0000 7fadbe800640 1 Processor -- start 2024-04-04T06:14:56.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.348+0000 7fadbe800640 1 -- start start 2024-04-04T06:14:56.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.349+0000 7fadbe800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb806e7f0 0x7fadb806efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:56.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.349+0000 7fadbe800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb8071e40 con 0x7fadb806f510 2024-04-04T06:14:56.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.349+0000 7fadbe800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb8072120 con 0x7fadb815e4d0 2024-04-04T06:14:56.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.349+0000 7fadbe800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fadb806f960 con 0x7fadb806e7f0 2024-04-04T06:14:56.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.350+0000 7fadb7e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fadb806f510 0x7fadb80acf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:39968/0 (socket says 172.21.15.159:39968) 2024-04-04T06:14:56.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.350+0000 7fadb7e00640 1 -- 172.21.15.159:0/1554400001 learned_addr learned my addr 172.21.15.159:0/1554400001 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:56.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.350+0000 7fadafe00640 1 --2- 172.21.15.159:0/1554400001 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb806e7f0 0x7fadb806efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:56.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.350+0000 7fadb6a00640 1 -- 172.21.15.159:0/1554400001 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 975209351 0 0) 0x7fadb8071e40 con 0x7fadb806f510 2024-04-04T06:14:56.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.350+0000 7fadb6a00640 1 -- 172.21.15.159:0/1554400001 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad9c003520 con 0x7fadb806f510 2024-04-04T06:14:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.351+0000 7fadb6a00640 1 -- 172.21.15.159:0/1554400001 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2804263767 0 0) 0x7fad9c003520 con 0x7fadb806f510 2024-04-04T06:14:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.351+0000 7fadb6a00640 1 -- 172.21.15.159:0/1554400001 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb806e7f0 msgr2=0x7fadb806efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.351+0000 7fadb6a00640 1 --2- 172.21.15.159:0/1554400001 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb806e7f0 0x7fadb806efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.351+0000 7fadb6a00640 1 -- 172.21.15.159:0/1554400001 >> v1:172.21.15.178:6789/0 conn(0x7fadb815e4d0 legacy=0x7fadb8071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.351+0000 7fadb6a00640 1 -- 172.21.15.159:0/1554400001 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb8072400 con 0x7fadb806f510 2024-04-04T06:14:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.351+0000 7fadb6a00640 1 -- 172.21.15.159:0/1554400001 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fadb00019f0 con 0x7fadb806f510 2024-04-04T06:14:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.351+0000 7fadb6a00640 1 -- 172.21.15.159:0/1554400001 >> v1:172.21.15.159:6789/0 conn(0x7fadb806f510 legacy=0x7fadb80acf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.351+0000 7fadb6a00640 1 --2- 172.21.15.159:0/1554400001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad9c0039e0 0x7fad9c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.351+0000 7fadb6a00640 1 --2- 172.21.15.159:0/1554400001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad9c004800 0x7fad9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.351+0000 7fadb6a00640 1 --2- 172.21.15.159:0/1554400001 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb806e7f0 0x7fad9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.351+0000 7fadb6a00640 1 -- 172.21.15.159:0/1554400001 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fadb806f960 con 0x7fad9c0039e0 2024-04-04T06:14:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.351+0000 7fadb6a00640 1 -- 172.21.15.159:0/1554400001 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c00baa0 con 0x7fad9c004800 2024-04-04T06:14:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.351+0000 7fadb6a00640 1 -- 172.21.15.159:0/1554400001 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fad9c00bca0 con 0x7fadb806e7f0 2024-04-04T06:14:56.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.351+0000 7fadb7400640 1 --2- 172.21.15.159:0/1554400001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad9c0039e0 0x7fad9c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.351+0000 7fadafe00640 1 --2- 172.21.15.159:0/1554400001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad9c004800 0x7fad9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.351+0000 7fadb7e00640 1 --2- 172.21.15.159:0/1554400001 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb806e7f0 0x7fad9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.352+0000 7fadb7400640 1 -- 172.21.15.159:0/1554400001 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb806e7f0 msgr2=0x7fad9c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.352+0000 7fadb7400640 1 --2- 172.21.15.159:0/1554400001 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fadb806e7f0 0x7fad9c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.352+0000 7fadb7400640 1 -- 172.21.15.159:0/1554400001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad9c004800 msgr2=0x7fad9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.352+0000 7fadb7400640 1 --2- 172.21.15.159:0/1554400001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad9c004800 0x7fad9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.352+0000 7fadb7400640 1 -- 172.21.15.159:0/1554400001 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9c003520 con 0x7fad9c0039e0 2024-04-04T06:14:56.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.353+0000 7fadb7400640 1 --2- 172.21.15.159:0/1554400001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad9c0039e0 0x7fad9c003e00 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fada8001720 tx=0x7fad9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d2f5d116c6c491 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:56.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.353+0000 7fadb6a00640 1 -- 172.21.15.159:0/1554400001 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8001370 con 0x7fad9c0039e0 2024-04-04T06:14:56.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.353+0000 7fadb6a00640 1 -- 172.21.15.159:0/1554400001 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada80052d0 con 0x7fad9c0039e0 2024-04-04T06:14:56.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.353+0000 7fadb6a00640 1 -- 172.21.15.159:0/1554400001 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8003830 con 0x7fad9c0039e0 2024-04-04T06:14:56.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.353+0000 7fadbe800640 1 -- 172.21.15.159:0/1554400001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad9c0039e0 msgr2=0x7fad9c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:56.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.353+0000 7fadbe800640 1 --2- 172.21.15.159:0/1554400001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad9c0039e0 0x7fad9c003e00 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fada8001720 tx=0x7fad9c005830 comp rx=0 tx=0).stop 2024-04-04T06:14:56.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.354+0000 7fadb7e00640 1 -- 172.21.15.159:0/1554400001 reap_dead start 2024-04-04T06:14:56.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.354+0000 7fadbe800640 1 -- 172.21.15.159:0/1554400001 shutdown_connections 2024-04-04T06:14:56.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.354+0000 7fadbe800640 1 -- 172.21.15.159:0/1554400001 >> 172.21.15.159:0/1554400001 conn(0x7fadb806bba0 msgr2=0x7fadb80b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:56.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.354+0000 7fadbe800640 1 -- 172.21.15.159:0/1554400001 shutdown_connections 2024-04-04T06:14:56.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.354+0000 7fadbe800640 1 -- 172.21.15.159:0/1554400001 wait complete. 2024-04-04T06:14:56.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.356+0000 7fadbe800640 1 Processor -- start 2024-04-04T06:14:56.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.356+0000 7fadbe800640 1 -- start start 2024-04-04T06:14:56.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.357+0000 7fadbe800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad9c0039e0 0x7fadb81e6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:56.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.357+0000 7fadbe800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadb81f8310 0x7fadb809ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:56.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.357+0000 7fadbe800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fadb8156200 0x7fadb8156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:56.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.357+0000 7fadbe800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fadb815e8b0 con 0x7fadb8156200 2024-04-04T06:14:56.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.357+0000 7fadbe800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fadb80729c0 con 0x7fadb81f8310 2024-04-04T06:14:56.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.357+0000 7fadbe800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fadb8072ee0 con 0x7fad9c0039e0 2024-04-04T06:14:56.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.357+0000 7fadb7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fadb8156200 0x7fadb8156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:56.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.357+0000 7fadb7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fadb8156200 0x7fadb8156690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37528/0 (socket says 172.21.15.159:37528) 2024-04-04T06:14:56.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.357+0000 7fadb7e00640 1 -- 172.21.15.159:0/518585390 learned_addr learned my addr 172.21.15.159:0/518585390 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:56.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.357+0000 7fadb7400640 1 --2- 172.21.15.159:0/518585390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad9c0039e0 0x7fadb81e6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:56.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.357+0000 7fadafe00640 1 --2- 172.21.15.159:0/518585390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadb81f8310 0x7fadb809ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:56.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.357+0000 7fadb7e00640 1 -- 172.21.15.159:0/518585390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad9c0039e0 msgr2=0x7fadb81e6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:56.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.357+0000 7fadb7e00640 1 --2- 172.21.15.159:0/518585390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad9c0039e0 0x7fadb81e6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:56.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.357+0000 7fadb7e00640 1 -- 172.21.15.159:0/518585390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadb81f8310 msgr2=0x7fadb809ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:56.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.357+0000 7fadb7e00640 1 --2- 172.21.15.159:0/518585390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadb81f8310 0x7fadb809ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:56.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.357+0000 7fadb7e00640 1 -- 172.21.15.159:0/518585390 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb809c3e0 con 0x7fadb8156200 2024-04-04T06:14:56.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.357+0000 7fadb7400640 1 --2- 172.21.15.159:0/518585390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad9c0039e0 0x7fadb81e6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:56.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.357+0000 7fadafe00640 1 --2- 172.21.15.159:0/518585390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadb81f8310 0x7fadb809ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:56.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.358+0000 7fadb7e00640 1 --2- 172.21.15.159:0/518585390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fadb8156200 0x7fadb8156690 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fadb806c580 tx=0x7fadb000e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:56.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.358+0000 7fadaf400640 1 -- 172.21.15.159:0/518585390 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb000ec30 con 0x7fadb8156200 2024-04-04T06:14:56.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.358+0000 7fadaf400640 1 -- 172.21.15.159:0/518585390 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb0018070 con 0x7fadb8156200 2024-04-04T06:14:56.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.358+0000 7fadbe800640 1 -- 172.21.15.159:0/518585390 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb809c6c0 con 0x7fadb8156200 2024-04-04T06:14:56.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.358+0000 7fadaf400640 1 -- 172.21.15.159:0/518585390 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0013420 con 0x7fadb8156200 2024-04-04T06:14:56.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.358+0000 7fadbe800640 1 -- 172.21.15.159:0/518585390 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb809cc80 con 0x7fadb8156200 2024-04-04T06:14:56.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.359+0000 7fadbe800640 1 -- 172.21.15.159:0/518585390 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb80685c0 con 0x7fadb8156200 2024-04-04T06:14:56.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.360+0000 7fadaf400640 1 -- 172.21.15.159:0/518585390 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fadb00135c0 con 0x7fadb8156200 2024-04-04T06:14:56.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.361+0000 7fadaf400640 1 --2- 172.21.15.159:0/518585390 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fad9406c920 0x7fad9406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:56.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.361+0000 7fadaf400640 1 -- 172.21.15.159:0/518585390 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5587+0+0 (secure 0 0 0) 0x7fadb008be30 con 0x7fadb8156200 2024-04-04T06:14:56.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.362+0000 7fadb7400640 1 --2- 172.21.15.159:0/518585390 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fad9406c920 0x7fad9406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:56.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.363+0000 7fadb7400640 1 --2- 172.21.15.159:0/518585390 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fad9406c920 0x7fad9406ede0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fada8033000 tx=0x7fada8009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:56.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.366+0000 7fadaf400640 1 -- 172.21.15.159:0/518585390 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fadb005a2d0 con 0x7fadb8156200 2024-04-04T06:14:56.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:56.570+0000 7fadbe800640 1 -- 172.21.15.159:0/518585390 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fadb8157390 con 0x7fadb8156200 2024-04-04T06:14:57.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.119+0000 7fadaf400640 1 -- 172.21.15.159:0/518585390 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) ==== 169+0+0 (secure 0 0 0) 0x7fadb0059bd0 con 0x7fadb8156200 2024-04-04T06:14:57.114 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_data' removed 2024-04-04T06:14:57.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.122+0000 7fadbe800640 1 -- 172.21.15.159:0/518585390 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fad9406c920 msgr2=0x7fad9406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:57.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.122+0000 7fadbe800640 1 --2- 172.21.15.159:0/518585390 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fad9406c920 0x7fad9406ede0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fada8033000 tx=0x7fada8009030 comp rx=0 tx=0).stop 2024-04-04T06:14:57.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.122+0000 7fadbe800640 1 -- 172.21.15.159:0/518585390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fadb8156200 msgr2=0x7fadb8156690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:57.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.122+0000 7fadbe800640 1 --2- 172.21.15.159:0/518585390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fadb8156200 0x7fadb8156690 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fadb806c580 tx=0x7fadb000e160 comp rx=0 tx=0).stop 2024-04-04T06:14:57.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.122+0000 7fadbe800640 1 -- 172.21.15.159:0/518585390 shutdown_connections 2024-04-04T06:14:57.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.122+0000 7fadbe800640 1 --2- 172.21.15.159:0/518585390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fadb8156200 0x7fadb8156690 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:57.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.122+0000 7fadbe800640 1 --2- 172.21.15.159:0/518585390 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fad9406c920 0x7fad9406ede0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:57.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.122+0000 7fadbe800640 1 --2- 172.21.15.159:0/518585390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadb81f8310 0x7fadb809ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:57.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.122+0000 7fadbe800640 1 --2- 172.21.15.159:0/518585390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad9c0039e0 0x7fadb81e6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:57.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.123+0000 7fadbe800640 1 -- 172.21.15.159:0/518585390 >> 172.21.15.159:0/518585390 conn(0x7fadb806bba0 msgr2=0x7fadb80b2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:57.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.123+0000 7fadbe800640 1 -- 172.21.15.159:0/518585390 shutdown_connections 2024-04-04T06:14:57.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.123+0000 7fadbe800640 1 -- 172.21.15.159:0/518585390 wait complete. 2024-04-04T06:14:57.144 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-04T06:14:57.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.355+0000 7facb8000640 1 Processor -- start 2024-04-04T06:14:57.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.355+0000 7facb8000640 1 -- start start 2024-04-04T06:14:57.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.356+0000 7facb8000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7facb0153580 0x7facb00d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:57.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.357+0000 7facb8000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facb00afab0 con 0x7facb00af4b0 2024-04-04T06:14:57.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.357+0000 7facb8000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facb015cc80 con 0x7facb00d07c0 2024-04-04T06:14:57.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.357+0000 7facb8000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7facb00d0c00 con 0x7facb0153580 2024-04-04T06:14:57.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.357+0000 7facb6000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7facb00af4b0 0x7facb01600d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:39980/0 (socket says 172.21.15.159:39980) 2024-04-04T06:14:57.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.357+0000 7facb6000640 1 -- 172.21.15.159:0/2458499584 learned_addr learned my addr 172.21.15.159:0/2458499584 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.357+0000 7facafe00640 1 -- 172.21.15.159:0/2458499584 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 619822835 0 0) 0x7facb00afab0 con 0x7facb00af4b0 2024-04-04T06:14:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.357+0000 7facafe00640 1 -- 172.21.15.159:0/2458499584 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac94003520 con 0x7facb00af4b0 2024-04-04T06:14:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.357+0000 7facb5600640 1 --2- 172.21.15.159:0/2458499584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7facb0153580 0x7facb00d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.358+0000 7facafe00640 1 -- 172.21.15.159:0/2458499584 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2330810804 0 0) 0x7fac94003520 con 0x7facb00af4b0 2024-04-04T06:14:57.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.358+0000 7facafe00640 1 -- 172.21.15.159:0/2458499584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7facb0153580 msgr2=0x7facb00d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.358+0000 7facafe00640 1 --2- 172.21.15.159:0/2458499584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7facb0153580 0x7facb00d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.358+0000 7facafe00640 1 -- 172.21.15.159:0/2458499584 >> v1:172.21.15.178:6789/0 conn(0x7facb00d07c0 legacy=0x7facb00aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.358+0000 7facafe00640 1 -- 172.21.15.159:0/2458499584 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb015da00 con 0x7facb00af4b0 2024-04-04T06:14:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.358+0000 7facafe00640 1 -- 172.21.15.159:0/2458499584 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7faca80019f0 con 0x7facb00af4b0 2024-04-04T06:14:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.358+0000 7facafe00640 1 -- 172.21.15.159:0/2458499584 >> v1:172.21.15.159:6789/0 conn(0x7facb00af4b0 legacy=0x7facb01600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.358+0000 7facafe00640 1 --2- 172.21.15.159:0/2458499584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac940039e0 0x7fac94003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.358+0000 7facafe00640 1 --2- 172.21.15.159:0/2458499584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7facb0153580 0x7fac94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.358+0000 7facafe00640 1 --2- 172.21.15.159:0/2458499584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac94004d40 0x7fac940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.358+0000 7facafe00640 1 -- 172.21.15.159:0/2458499584 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7facb00d0c00 con 0x7fac940039e0 2024-04-04T06:14:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.358+0000 7facafe00640 1 -- 172.21.15.159:0/2458499584 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fac9400bbd0 con 0x7fac94004d40 2024-04-04T06:14:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.358+0000 7facafe00640 1 -- 172.21.15.159:0/2458499584 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fac9400bdd0 con 0x7facb0153580 2024-04-04T06:14:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.358+0000 7facb5600640 1 --2- 172.21.15.159:0/2458499584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac940039e0 0x7fac94003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:57.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.359+0000 7facb5600640 1 -- 172.21.15.159:0/2458499584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7facb0153580 msgr2=0x7fac94004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.359+0000 7facb4c00640 1 --2- 172.21.15.159:0/2458499584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7facb0153580 0x7fac94004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.359+0000 7facb5600640 1 --2- 172.21.15.159:0/2458499584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7facb0153580 0x7fac94004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.359+0000 7facb5600640 1 -- 172.21.15.159:0/2458499584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac94004d40 msgr2=0x7fac940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.359+0000 7facb5600640 1 --2- 172.21.15.159:0/2458499584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac94004d40 0x7fac940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:57.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.359+0000 7facb5600640 1 -- 172.21.15.159:0/2458499584 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac94003520 con 0x7fac940039e0 2024-04-04T06:14:57.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.360+0000 7facb5600640 1 --2- 172.21.15.159:0/2458499584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac940039e0 0x7fac94003de0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7faca000ba00 tx=0x7fac94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0a1dcfe82fc531 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:57.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.360+0000 7facafe00640 1 -- 172.21.15.159:0/2458499584 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca0036020 con 0x7fac940039e0 2024-04-04T06:14:57.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.360+0000 7facafe00640 1 -- 172.21.15.159:0/2458499584 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faca0032620 con 0x7fac940039e0 2024-04-04T06:14:57.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.360+0000 7facafe00640 1 -- 172.21.15.159:0/2458499584 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca000b600 con 0x7fac940039e0 2024-04-04T06:14:57.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.361+0000 7facb8000640 1 -- 172.21.15.159:0/2458499584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac940039e0 msgr2=0x7fac94003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:57.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.361+0000 7facb8000640 1 --2- 172.21.15.159:0/2458499584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac940039e0 0x7fac94003de0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7faca000ba00 tx=0x7fac94005830 comp rx=0 tx=0).stop 2024-04-04T06:14:57.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.361+0000 7facb6000640 1 -- 172.21.15.159:0/2458499584 reap_dead start 2024-04-04T06:14:57.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.361+0000 7facb8000640 1 -- 172.21.15.159:0/2458499584 shutdown_connections 2024-04-04T06:14:57.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.361+0000 7facb8000640 1 -- 172.21.15.159:0/2458499584 >> 172.21.15.159:0/2458499584 conn(0x7facb006b750 msgr2=0x7facb006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:57.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.361+0000 7facb8000640 1 -- 172.21.15.159:0/2458499584 shutdown_connections 2024-04-04T06:14:57.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.361+0000 7facb8000640 1 -- 172.21.15.159:0/2458499584 wait complete. 2024-04-04T06:14:57.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.363+0000 7facb8000640 1 Processor -- start 2024-04-04T06:14:57.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.363+0000 7facb8000640 1 -- start start 2024-04-04T06:14:57.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.364+0000 7facb8000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac940039e0 0x7facb01f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:57.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.364+0000 7facb8000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7facb014d080 0x7facb014d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:57.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.364+0000 7facb8000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facb0072790 0x7facb0072c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:57.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.364+0000 7facb8000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7facb015d4e0 con 0x7facb014d080 2024-04-04T06:14:57.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.364+0000 7facb8000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7facb015d200 con 0x7facb0072790 2024-04-04T06:14:57.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.364+0000 7facb8000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7facb0153960 con 0x7fac940039e0 2024-04-04T06:14:57.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.364+0000 7facb4c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7facb014d080 0x7facb014d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:57.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.364+0000 7facb4c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7facb014d080 0x7facb014d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37550/0 (socket says 172.21.15.159:37550) 2024-04-04T06:14:57.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.364+0000 7facb4c00640 1 -- 172.21.15.159:0/470245004 learned_addr learned my addr 172.21.15.159:0/470245004 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:57.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.365+0000 7facb5600640 1 --2- 172.21.15.159:0/470245004 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac940039e0 0x7facb01f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:57.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.365+0000 7facb6000640 1 --2- 172.21.15.159:0/470245004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facb0072790 0x7facb0072c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:57.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.365+0000 7facb4c00640 1 -- 172.21.15.159:0/470245004 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac940039e0 msgr2=0x7facb01f3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:57.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.365+0000 7facb4c00640 1 --2- 172.21.15.159:0/470245004 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac940039e0 0x7facb01f3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:57.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.365+0000 7facb4c00640 1 -- 172.21.15.159:0/470245004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facb0072790 msgr2=0x7facb0072c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:57.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.365+0000 7facb4c00640 1 --2- 172.21.15.159:0/470245004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facb0072790 0x7facb0072c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:57.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.365+0000 7facb4c00640 1 -- 172.21.15.159:0/470245004 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb014db40 con 0x7facb014d080 2024-04-04T06:14:57.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.365+0000 7facb5600640 1 --2- 172.21.15.159:0/470245004 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac940039e0 0x7facb01f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:57.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.365+0000 7facb4c00640 1 --2- 172.21.15.159:0/470245004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7facb014d080 0x7facb014d500 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00d510 tx=0x7fac9c00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:57.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.365+0000 7facad600640 1 -- 172.21.15.159:0/470245004 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c016070 con 0x7facb014d080 2024-04-04T06:14:57.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.365+0000 7facad600640 1 -- 172.21.15.159:0/470245004 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac9c012800 con 0x7facb014d080 2024-04-04T06:14:57.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.365+0000 7facb8000640 1 -- 172.21.15.159:0/470245004 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb014de20 con 0x7facb014d080 2024-04-04T06:14:57.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.365+0000 7facad600640 1 -- 172.21.15.159:0/470245004 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c012bf0 con 0x7facb014d080 2024-04-04T06:14:57.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.366+0000 7facb8000640 1 -- 172.21.15.159:0/470245004 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb014e3e0 con 0x7facb014d080 2024-04-04T06:14:57.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.367+0000 7facb8000640 1 -- 172.21.15.159:0/470245004 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facb00685c0 con 0x7facb014d080 2024-04-04T06:14:57.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.368+0000 7facad600640 1 -- 172.21.15.159:0/470245004 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac9c019070 con 0x7facb014d080 2024-04-04T06:14:57.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.369+0000 7facad600640 1 --2- 172.21.15.159:0/470245004 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fac9006c8f0 0x7fac9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:57.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.369+0000 7facad600640 1 -- 172.21.15.159:0/470245004 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7fac9c08b830 con 0x7facb014d080 2024-04-04T06:14:57.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.369+0000 7facb5600640 1 --2- 172.21.15.159:0/470245004 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fac9006c8f0 0x7fac9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:57.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.370+0000 7facb5600640 1 --2- 172.21.15.159:0/470245004 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fac9006c8f0 0x7fac9006edb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fac94004780 tx=0x7faca0030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:57.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.373+0000 7facad600640 1 -- 172.21.15.159:0/470245004 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fac9c059e10 con 0x7facb014d080 2024-04-04T06:14:57.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.574+0000 7facb8000640 1 -- 172.21.15.159:0/470245004 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7facb0151f60 con 0x7facb014d080 2024-04-04T06:14:57.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.581+0000 7facad600640 1 -- 172.21.15.159:0/470245004 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7fac9c059710 con 0x7facb014d080 2024-04-04T06:14:57.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.584+0000 7facb8000640 1 -- 172.21.15.159:0/470245004 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fac9006c8f0 msgr2=0x7fac9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:57.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.584+0000 7facb8000640 1 --2- 172.21.15.159:0/470245004 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fac9006c8f0 0x7fac9006edb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fac94004780 tx=0x7faca0030000 comp rx=0 tx=0).stop 2024-04-04T06:14:57.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.584+0000 7facb8000640 1 -- 172.21.15.159:0/470245004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7facb014d080 msgr2=0x7facb014d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:57.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.584+0000 7facb8000640 1 --2- 172.21.15.159:0/470245004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7facb014d080 0x7facb014d500 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00d510 tx=0x7fac9c00d9d0 comp rx=0 tx=0).stop 2024-04-04T06:14:57.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.584+0000 7facb8000640 1 -- 172.21.15.159:0/470245004 shutdown_connections 2024-04-04T06:14:57.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.584+0000 7facb8000640 1 --2- 172.21.15.159:0/470245004 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fac9006c8f0 0x7fac9006edb0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:57.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.584+0000 7facb8000640 1 --2- 172.21.15.159:0/470245004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facb0072790 0x7facb0072c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:57.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.584+0000 7facb8000640 1 --2- 172.21.15.159:0/470245004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7facb014d080 0x7facb014d500 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:57.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.584+0000 7facb8000640 1 --2- 172.21.15.159:0/470245004 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac940039e0 0x7facb01f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:57.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.584+0000 7facb8000640 1 -- 172.21.15.159:0/470245004 >> 172.21.15.159:0/470245004 conn(0x7facb006b750 msgr2=0x7facb00aeda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:57.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.585+0000 7facb8000640 1 -- 172.21.15.159:0/470245004 shutdown_connections 2024-04-04T06:14:57.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.585+0000 7facb8000640 1 -- 172.21.15.159:0/470245004 wait complete. 2024-04-04T06:14:57.599 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-04T06:14:57.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.814+0000 7fb2c6600640 1 Processor -- start 2024-04-04T06:14:57.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.814+0000 7fb2c6600640 1 -- start start 2024-04-04T06:14:57.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.816+0000 7fb2c6600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb2c015d490 0x7fb2c00cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:57.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.816+0000 7fb2c6600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2c0071c80 con 0x7fb2c00b4a30 2024-04-04T06:14:57.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.816+0000 7fb2c6600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2c00b3f50 con 0x7fb2c00716d0 2024-04-04T06:14:57.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.816+0000 7fb2c6600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fb2c0071f60 con 0x7fb2c015d490 2024-04-04T06:14:57.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.816+0000 7fb2bf400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb2c015d490 0x7fb2c00cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:57.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.816+0000 7fb2b6a00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fb2c00b4a30 0x7fb2c0070fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:39990/0 (socket says 172.21.15.159:39990) 2024-04-04T06:14:57.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.816+0000 7fb2b6a00640 1 -- 172.21.15.159:0/2645919102 learned_addr learned my addr 172.21.15.159:0/2645919102 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:57.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.817+0000 7fb2bf400640 1 -- 172.21.15.159:0/2645919102 >> v1:172.21.15.178:6789/0 conn(0x7fb2c00716d0 legacy=0x7fb2c00c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:57.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.817+0000 7fb2bf400640 1 -- 172.21.15.159:0/2645919102 >> v1:172.21.15.159:6789/0 conn(0x7fb2c00b4a30 legacy=0x7fb2c0070fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:57.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.817+0000 7fb2bf400640 1 -- 172.21.15.159:0/2645919102 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c00b4230 con 0x7fb2c015d490 2024-04-04T06:14:57.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.818+0000 7fb2bf400640 1 --2- 172.21.15.159:0/2645919102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2c015d490 0x7fb2c00cfd80 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009d90 tx=0x7fb2b002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae64eee78f70457e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:57.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.819+0000 7fb2bea00640 1 -- 172.21.15.159:0/2645919102 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0039070 con 0x7fb2c015d490 2024-04-04T06:14:57.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.819+0000 7fb2bea00640 1 -- 172.21.15.159:0/2645919102 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b002cbd0 con 0x7fb2c015d490 2024-04-04T06:14:57.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.819+0000 7fb2bea00640 1 -- 172.21.15.159:0/2645919102 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0034440 con 0x7fb2c015d490 2024-04-04T06:14:57.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.819+0000 7fb2c6600640 1 -- 172.21.15.159:0/2645919102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2c015d490 msgr2=0x7fb2c00cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:57.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.819+0000 7fb2c6600640 1 --2- 172.21.15.159:0/2645919102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2c015d490 0x7fb2c00cfd80 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009d90 tx=0x7fb2b002be20 comp rx=0 tx=0).stop 2024-04-04T06:14:57.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.820+0000 7fb2c6600640 1 -- 172.21.15.159:0/2645919102 shutdown_connections 2024-04-04T06:14:57.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.820+0000 7fb2c6600640 1 --2- 172.21.15.159:0/2645919102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2c015d490 0x7fb2c00cfd80 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:57.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.820+0000 7fb2c6600640 1 -- 172.21.15.159:0/2645919102 >> 172.21.15.159:0/2645919102 conn(0x7fb2c00d2240 msgr2=0x7fb2c00d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:57.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.820+0000 7fb2c6600640 1 -- 172.21.15.159:0/2645919102 shutdown_connections 2024-04-04T06:14:57.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.820+0000 7fb2c6600640 1 -- 172.21.15.159:0/2645919102 wait complete. 2024-04-04T06:14:57.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.822+0000 7fb2c6600640 1 Processor -- start 2024-04-04T06:14:57.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.823+0000 7fb2c6600640 1 -- start start 2024-04-04T06:14:57.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.824+0000 7fb2c6600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2c00716d0 0x7fb2c014b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:57.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.824+0000 7fb2c6600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2c00b4a30 0x7fb2c014d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:57.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.824+0000 7fb2c6600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2c015d490 0x7fb2c014dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:57.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.824+0000 7fb2c6600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c015d9f0 con 0x7fb2c00716d0 2024-04-04T06:14:57.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.824+0000 7fb2c6600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c015d870 con 0x7fb2c015d490 2024-04-04T06:14:57.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.824+0000 7fb2c6600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c00d02c0 con 0x7fb2c00b4a30 2024-04-04T06:14:57.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.824+0000 7fb2bf400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2c00716d0 0x7fb2c014b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:57.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.824+0000 7fb2bf400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2c00716d0 0x7fb2c014b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37574/0 (socket says 172.21.15.159:37574) 2024-04-04T06:14:57.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.824+0000 7fb2bf400640 1 -- 172.21.15.159:0/2278383983 learned_addr learned my addr 172.21.15.159:0/2278383983 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:57.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.825+0000 7fb2b6a00640 1 --2- 172.21.15.159:0/2278383983 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2c00b4a30 0x7fb2c014d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:57.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.825+0000 7fb2bfe00640 1 --2- 172.21.15.159:0/2278383983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2c015d490 0x7fb2c014dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:57.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.825+0000 7fb2bf400640 1 -- 172.21.15.159:0/2278383983 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2c00b4a30 msgr2=0x7fb2c014d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:57.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.825+0000 7fb2bf400640 1 --2- 172.21.15.159:0/2278383983 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2c00b4a30 0x7fb2c014d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:57.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.825+0000 7fb2bf400640 1 -- 172.21.15.159:0/2278383983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2c015d490 msgr2=0x7fb2c014dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:57.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.825+0000 7fb2bf400640 1 --2- 172.21.15.159:0/2278383983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2c015d490 0x7fb2c014dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:57.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.825+0000 7fb2bf400640 1 -- 172.21.15.159:0/2278383983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c014c6f0 con 0x7fb2c00716d0 2024-04-04T06:14:57.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.825+0000 7fb2bfe00640 1 --2- 172.21.15.159:0/2278383983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2c015d490 0x7fb2c014dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:14:57.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.826+0000 7fb2bf400640 1 --2- 172.21.15.159:0/2278383983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2c00716d0 0x7fb2c014b030 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0037040 tx=0x7fb2b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:57.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.826+0000 7fb2b7e00640 1 -- 172.21.15.159:0/2278383983 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0044020 con 0x7fb2c00716d0 2024-04-04T06:14:57.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.826+0000 7fb2b7e00640 1 -- 172.21.15.159:0/2278383983 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b003f670 con 0x7fb2c00716d0 2024-04-04T06:14:57.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.826+0000 7fb2b7e00640 1 -- 172.21.15.159:0/2278383983 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0039040 con 0x7fb2c00716d0 2024-04-04T06:14:57.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.826+0000 7fb2c6600640 1 -- 172.21.15.159:0/2278383983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c014c9d0 con 0x7fb2c00716d0 2024-04-04T06:14:57.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.826+0000 7fb2c6600640 1 -- 172.21.15.159:0/2278383983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c014cf90 con 0x7fb2c00716d0 2024-04-04T06:14:57.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.828+0000 7fb2c6600640 1 -- 172.21.15.159:0/2278383983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c00685c0 con 0x7fb2c00716d0 2024-04-04T06:14:57.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.829+0000 7fb2b7e00640 1 -- 172.21.15.159:0/2278383983 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2b0047020 con 0x7fb2c00716d0 2024-04-04T06:14:57.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.831+0000 7fb2b7e00640 1 --2- 172.21.15.159:0/2278383983 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb29c06c8f0 0x7fb29c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:57.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.831+0000 7fb2b7e00640 1 -- 172.21.15.159:0/2278383983 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7fb2b00ae610 con 0x7fb2c00716d0 2024-04-04T06:14:57.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.831+0000 7fb2b6a00640 1 --2- 172.21.15.159:0/2278383983 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb29c06c8f0 0x7fb29c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:57.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.832+0000 7fb2b6a00640 1 --2- 172.21.15.159:0/2278383983 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb29c06c8f0 0x7fb29c06edb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb2c01610c0 tx=0x7fb2a8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:57.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:57.836+0000 7fb2b7e00640 1 -- 172.21.15.159:0/2278383983 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb2b004e030 con 0x7fb2c00716d0 2024-04-04T06:14:58.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.111+0000 7fb2c6600640 1 -- 172.21.15.159:0/2278383983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7fb2c0160820 con 0x7fb2c00716d0 2024-04-04T06:14:58.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.140+0000 7fb2b7e00640 1 -- 172.21.15.159:0/2278383983 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v440) ==== 167+0+0 (secure 0 0 0) 0x7fb2b007cbf0 con 0x7fb2c00716d0 2024-04-04T06:14:58.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.143+0000 7fb2c6600640 1 -- 172.21.15.159:0/2278383983 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb29c06c8f0 msgr2=0x7fb29c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:58.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.143+0000 7fb2c6600640 1 --2- 172.21.15.159:0/2278383983 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb29c06c8f0 0x7fb29c06edb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb2c01610c0 tx=0x7fb2a8009040 comp rx=0 tx=0).stop 2024-04-04T06:14:58.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.143+0000 7fb2c6600640 1 -- 172.21.15.159:0/2278383983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2c00716d0 msgr2=0x7fb2c014b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:58.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.143+0000 7fb2c6600640 1 --2- 172.21.15.159:0/2278383983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2c00716d0 0x7fb2c014b030 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0037040 tx=0x7fb2b0004a40 comp rx=0 tx=0).stop 2024-04-04T06:14:58.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.143+0000 7fb2c6600640 1 -- 172.21.15.159:0/2278383983 shutdown_connections 2024-04-04T06:14:58.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.143+0000 7fb2c6600640 1 --2- 172.21.15.159:0/2278383983 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb29c06c8f0 0x7fb29c06edb0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:58.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.143+0000 7fb2c6600640 1 --2- 172.21.15.159:0/2278383983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2c015d490 0x7fb2c014dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:58.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.143+0000 7fb2c6600640 1 --2- 172.21.15.159:0/2278383983 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2c00b4a30 0x7fb2c014d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:58.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.143+0000 7fb2c6600640 1 --2- 172.21.15.159:0/2278383983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2c00716d0 0x7fb2c014b030 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:58.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.143+0000 7fb2c6600640 1 -- 172.21.15.159:0/2278383983 >> 172.21.15.159:0/2278383983 conn(0x7fb2c00d2240 msgr2=0x7fb2c015fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:58.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.143+0000 7fb2c6600640 1 -- 172.21.15.159:0/2278383983 shutdown_connections 2024-04-04T06:14:58.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.143+0000 7fb2c6600640 1 -- 172.21.15.159:0/2278383983 wait complete. 2024-04-04T06:14:58.157 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-04T06:14:58.157 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-04T06:14:58.158 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:14:58.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.372+0000 7fa672600640 1 Processor -- start 2024-04-04T06:14:58.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.372+0000 7fa672600640 1 -- start start 2024-04-04T06:14:58.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.374+0000 7fa672600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa66c15b620 0x7fa66c0b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:58.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.374+0000 7fa672600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa66c0ccdb0 0x7fa66c15efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:58.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.374+0000 7fa672600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa66c0ceb50 con 0x7fa66c0cc970 2024-04-04T06:14:58.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.374+0000 7fa672600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fa66c0ce590 con 0x7fa66c15b620 2024-04-04T06:14:58.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.374+0000 7fa672600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa66c0ce870 con 0x7fa66c0ccdb0 2024-04-04T06:14:58.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.374+0000 7fa66b400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa66c15b620 0x7fa66c0b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:58.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.374+0000 7fa66b400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa66c15b620 0x7fa66c0b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37576/0 (socket says 172.21.15.159:37576) 2024-04-04T06:14:58.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.374+0000 7fa66b400640 1 -- 172.21.15.159:0/557821038 learned_addr learned my addr 172.21.15.159:0/557821038 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:58.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.375+0000 7fa66b400640 1 -- 172.21.15.159:0/557821038 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa66c0ccdb0 msgr2=0x7fa66c15efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:58.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.375+0000 7fa66b400640 1 --2- 172.21.15.159:0/557821038 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa66c0ccdb0 0x7fa66c15efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:58.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.375+0000 7fa66b400640 1 -- 172.21.15.159:0/557821038 >> v1:172.21.15.178:6789/0 conn(0x7fa66c0cc970 legacy=0x7fa66c06e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:58.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.375+0000 7fa66b400640 1 -- 172.21.15.159:0/557821038 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66c0cbe90 con 0x7fa66c15b620 2024-04-04T06:14:58.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.376+0000 7fa66b400640 1 --2- 172.21.15.159:0/557821038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa66c15b620 0x7fa66c0b3860 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fa654009870 tx=0x7fa65402db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d435e092e99d3fb server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:58.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.376+0000 7fa66a000640 1 -- 172.21.15.159:0/557821038 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65402f8c0 con 0x7fa66c15b620 2024-04-04T06:14:58.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.376+0000 7fa66a000640 1 -- 172.21.15.159:0/557821038 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa65402fa60 con 0x7fa66c15b620 2024-04-04T06:14:58.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.376+0000 7fa66a000640 1 -- 172.21.15.159:0/557821038 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65402fe30 con 0x7fa66c15b620 2024-04-04T06:14:58.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.377+0000 7fa672600640 1 -- 172.21.15.159:0/557821038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa66c15b620 msgr2=0x7fa66c0b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:58.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.377+0000 7fa672600640 1 --2- 172.21.15.159:0/557821038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa66c15b620 0x7fa66c0b3860 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fa654009870 tx=0x7fa65402db00 comp rx=0 tx=0).stop 2024-04-04T06:14:58.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.377+0000 7fa672600640 1 -- 172.21.15.159:0/557821038 shutdown_connections 2024-04-04T06:14:58.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.377+0000 7fa672600640 1 --2- 172.21.15.159:0/557821038 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa66c0ccdb0 0x7fa66c15efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:58.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.377+0000 7fa672600640 1 --2- 172.21.15.159:0/557821038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa66c15b620 0x7fa66c0b3860 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:58.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.377+0000 7fa672600640 1 -- 172.21.15.159:0/557821038 >> 172.21.15.159:0/557821038 conn(0x7fa66c0b2d40 msgr2=0x7fa66c0b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:58.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.377+0000 7fa672600640 1 -- 172.21.15.159:0/557821038 shutdown_connections 2024-04-04T06:14:58.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.377+0000 7fa672600640 1 -- 172.21.15.159:0/557821038 wait complete. 2024-04-04T06:14:58.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.378+0000 7fa672600640 1 Processor -- start 2024-04-04T06:14:58.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.379+0000 7fa672600640 1 -- start start 2024-04-04T06:14:58.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.379+0000 7fa672600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa66c0cc970 0x7fa66c1565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:58.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.379+0000 7fa672600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa66c0ccdb0 0x7fa66c0ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:58.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.379+0000 7fa672600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa66c15b620 0x7fa66c153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:58.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.379+0000 7fa672600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c06f000 con 0x7fa66c15b620 2024-04-04T06:14:58.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.379+0000 7fa672600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c06ee80 con 0x7fa66c0cc970 2024-04-04T06:14:58.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.379+0000 7fa672600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa66c06f180 con 0x7fa66c0ccdb0 2024-04-04T06:14:58.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.380+0000 7fa66be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa66c15b620 0x7fa66c153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:58.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.380+0000 7fa66be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa66c15b620 0x7fa66c153000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37588/0 (socket says 172.21.15.159:37588) 2024-04-04T06:14:58.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.380+0000 7fa66be00640 1 -- 172.21.15.159:0/1127877874 learned_addr learned my addr 172.21.15.159:0/1127877874 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:58.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.380+0000 7fa66aa00640 1 --2- 172.21.15.159:0/1127877874 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa66c0ccdb0 0x7fa66c0ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:58.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.380+0000 7fa66b400640 1 --2- 172.21.15.159:0/1127877874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa66c0cc970 0x7fa66c1565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:58.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.380+0000 7fa66be00640 1 -- 172.21.15.159:0/1127877874 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa66c0ccdb0 msgr2=0x7fa66c0ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:58.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.380+0000 7fa66be00640 1 --2- 172.21.15.159:0/1127877874 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa66c0ccdb0 0x7fa66c0ca990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:58.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.380+0000 7fa66be00640 1 -- 172.21.15.159:0/1127877874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa66c0cc970 msgr2=0x7fa66c1565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:58.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.380+0000 7fa66be00640 1 --2- 172.21.15.159:0/1127877874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa66c0cc970 0x7fa66c1565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:58.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.380+0000 7fa66be00640 1 -- 172.21.15.159:0/1127877874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66c156e20 con 0x7fa66c15b620 2024-04-04T06:14:58.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.381+0000 7fa66be00640 1 --2- 172.21.15.159:0/1127877874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa66c15b620 0x7fa66c153000 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fa66400b3f0 tx=0x7fa66400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:58.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.381+0000 7fa663400640 1 -- 172.21.15.159:0/1127877874 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa664015020 con 0x7fa66c15b620 2024-04-04T06:14:58.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.381+0000 7fa663400640 1 -- 172.21.15.159:0/1127877874 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa664011830 con 0x7fa66c15b620 2024-04-04T06:14:58.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.381+0000 7fa663400640 1 -- 172.21.15.159:0/1127877874 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa664011c20 con 0x7fa66c15b620 2024-04-04T06:14:58.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.381+0000 7fa672600640 1 -- 172.21.15.159:0/1127877874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa66c157160 con 0x7fa66c15b620 2024-04-04T06:14:58.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.381+0000 7fa672600640 1 -- 172.21.15.159:0/1127877874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa66c15d2c0 con 0x7fa66c15b620 2024-04-04T06:14:58.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.382+0000 7fa672600640 1 -- 172.21.15.159:0/1127877874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa66c0b3bc0 con 0x7fa66c15b620 2024-04-04T06:14:58.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.383+0000 7fa663400640 1 -- 172.21.15.159:0/1127877874 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa664018020 con 0x7fa66c15b620 2024-04-04T06:14:58.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.384+0000 7fa663400640 1 --2- 172.21.15.159:0/1127877874 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa64806c8f0 0x7fa64806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:58.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.384+0000 7fa663400640 1 -- 172.21.15.159:0/1127877874 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7fa66408b940 con 0x7fa66c15b620 2024-04-04T06:14:58.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.385+0000 7fa66b400640 1 --2- 172.21.15.159:0/1127877874 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa64806c8f0 0x7fa64806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:58.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.386+0000 7fa66b400640 1 --2- 172.21.15.159:0/1127877874 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa64806c8f0 0x7fa64806edb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa654038000 tx=0x7fa654008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:58.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.388+0000 7fa663400640 1 -- 172.21.15.159:0/1127877874 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa66c0b3bc0 con 0x7fa66c15b620 2024-04-04T06:14:58.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.589+0000 7fa672600640 1 -- 172.21.15.159:0/1127877874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa66c068cd0 con 0x7fa66c15b620 2024-04-04T06:14:58.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.591+0000 7fa663400640 1 -- 172.21.15.159:0/1127877874 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+13299 (secure 0 0 0) 0x7fa66401f020 con 0x7fa66c15b620 2024-04-04T06:14:58.585 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:14:58.586 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":92,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:14:57.116515+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6833/3555825575":"2024-04-05T06:14:53.840586+0000","172.21.15.159:6835/3796123128":"2024-04-05T06:14:04.756982+0000","172.21.15.159:6833/3796123128":"2024-04-05T06:14:04.756982+0000","172.21.15.159:6835/3555825575":"2024-04-05T06:14:53.840586+0000","172.21.15.159:6835/3484779950":"2024-04-05T06:12:49.205338+0000","172.21.15.159:6833/3484779950":"2024-04-05T06:12:49.205338+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:14:58.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.593+0000 7fa672600640 1 -- 172.21.15.159:0/1127877874 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa64806c8f0 msgr2=0x7fa64806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:58.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.593+0000 7fa672600640 1 --2- 172.21.15.159:0/1127877874 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa64806c8f0 0x7fa64806edb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa654038000 tx=0x7fa654008030 comp rx=0 tx=0).stop 2024-04-04T06:14:58.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.593+0000 7fa672600640 1 -- 172.21.15.159:0/1127877874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa66c15b620 msgr2=0x7fa66c153000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:58.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.593+0000 7fa672600640 1 --2- 172.21.15.159:0/1127877874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa66c15b620 0x7fa66c153000 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fa66400b3f0 tx=0x7fa66400b8b0 comp rx=0 tx=0).stop 2024-04-04T06:14:58.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.594+0000 7fa672600640 1 -- 172.21.15.159:0/1127877874 shutdown_connections 2024-04-04T06:14:58.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.594+0000 7fa672600640 1 --2- 172.21.15.159:0/1127877874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa66c15b620 0x7fa66c153000 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:58.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.594+0000 7fa672600640 1 --2- 172.21.15.159:0/1127877874 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa66c0ccdb0 0x7fa66c0ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:58.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.594+0000 7fa672600640 1 --2- 172.21.15.159:0/1127877874 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa64806c8f0 0x7fa64806edb0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:58.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.594+0000 7fa672600640 1 --2- 172.21.15.159:0/1127877874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa66c0cc970 0x7fa66c1565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:58.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.594+0000 7fa672600640 1 -- 172.21.15.159:0/1127877874 >> 172.21.15.159:0/1127877874 conn(0x7fa66c0b2d40 msgr2=0x7fa66c0b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:58.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.594+0000 7fa672600640 1 -- 172.21.15.159:0/1127877874 shutdown_connections 2024-04-04T06:14:58.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.594+0000 7fa672600640 1 -- 172.21.15.159:0/1127877874 wait complete. 2024-04-04T06:14:58.606 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:14:58.606 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:14:58.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.829+0000 7f58b5200640 1 Processor -- start 2024-04-04T06:14:58.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.830+0000 7f58b5200640 1 -- start start 2024-04-04T06:14:58.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.831+0000 7f58b5200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f58b0153580 0x7f58b00d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:58.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.831+0000 7f58b5200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b00af4b0 0x7f58b01600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:58.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.831+0000 7f58b5200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58b00afab0 con 0x7f58b00d07c0 2024-04-04T06:14:58.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.831+0000 7f58b5200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f58b00d0c30 con 0x7f58b0153580 2024-04-04T06:14:58.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.831+0000 7f58b5200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f58b015cc80 con 0x7f58b00af4b0 2024-04-04T06:14:58.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.831+0000 7f58a5400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f58b00d07c0 0x7f58b00aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:39998/0 (socket says 172.21.15.159:39998) 2024-04-04T06:14:58.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.831+0000 7f58a5400640 1 -- 172.21.15.159:0/2239099447 learned_addr learned my addr 172.21.15.159:0/2239099447 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:58.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.832+0000 7f58ad400640 1 -- 172.21.15.159:0/2239099447 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2179868029 0 0) 0x7f58b00afab0 con 0x7f58b00d07c0 2024-04-04T06:14:58.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.832+0000 7f58ade00640 1 --2- 172.21.15.159:0/2239099447 >> v2:172.21.15.178:3300/0 conn(0x7f58b0153580 0x7f58b00d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:58.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.832+0000 7f58ad400640 1 -- 172.21.15.159:0/2239099447 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5894003520 con 0x7f58b00d07c0 2024-04-04T06:14:58.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.832+0000 7f58ae800640 1 --2- 172.21.15.159:0/2239099447 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b00af4b0 0x7f58b01600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:58.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.832+0000 7f58ad400640 1 -- 172.21.15.159:0/2239099447 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3800044913 0 0) 0x7f5894003520 con 0x7f58b00d07c0 2024-04-04T06:14:58.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.832+0000 7f58ad400640 1 -- 172.21.15.159:0/2239099447 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b00af4b0 msgr2=0x7f58b01600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:58.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.832+0000 7f58ad400640 1 --2- 172.21.15.159:0/2239099447 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b00af4b0 0x7f58b01600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:58.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.832+0000 7f58ad400640 1 -- 172.21.15.159:0/2239099447 >> v2:172.21.15.178:3300/0 conn(0x7f58b0153580 msgr2=0x7f58b00d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:58.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.832+0000 7f58ad400640 1 --2- 172.21.15.159:0/2239099447 >> v2:172.21.15.178:3300/0 conn(0x7f58b0153580 0x7f58b00d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:58.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.832+0000 7f58ad400640 1 -- 172.21.15.159:0/2239099447 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b015da00 con 0x7f58b00d07c0 2024-04-04T06:14:58.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.832+0000 7f58ad400640 1 -- 172.21.15.159:0/2239099447 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f5898001670 con 0x7f58b00d07c0 2024-04-04T06:14:58.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.832+0000 7f58ad400640 1 -- 172.21.15.159:0/2239099447 >> v1:172.21.15.159:6789/0 conn(0x7f58b00d07c0 legacy=0x7f58b00aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:58.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.832+0000 7f58ad400640 1 --2- 172.21.15.159:0/2239099447 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58940039b0 0x7f5894003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:58.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.832+0000 7f58ad400640 1 --2- 172.21.15.159:0/2239099447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5894004800 0x7f5894004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:58.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.832+0000 7f58ad400640 1 --2- 172.21.15.159:0/2239099447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58940051c0 0x7f589400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:58.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.832+0000 7f58ad400640 1 -- 172.21.15.159:0/2239099447 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f58b00d0c30 con 0x7f5894004800 2024-04-04T06:14:58.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.832+0000 7f58ad400640 1 -- 172.21.15.159:0/2239099447 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58b015cc80 con 0x7f58940051c0 2024-04-04T06:14:58.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.832+0000 7f58ad400640 1 -- 172.21.15.159:0/2239099447 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f589400c090 con 0x7f58940039b0 2024-04-04T06:14:58.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.832+0000 7f58ae800640 1 --2- 172.21.15.159:0/2239099447 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b00af4b0 0x7f58b01600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:14:58.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.832+0000 7f58ade00640 1 --2- 172.21.15.159:0/2239099447 >> v2:172.21.15.178:3300/0 conn(0x7f58b0153580 0x7f58b00d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:14:58.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.833+0000 7f58a5400640 1 --2- 172.21.15.159:0/2239099447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5894004800 0x7f5894004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:58.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.833+0000 7f58a5400640 1 -- 172.21.15.159:0/2239099447 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58940039b0 msgr2=0x7f5894003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:58.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.833+0000 7f58a5400640 1 --2- 172.21.15.159:0/2239099447 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58940039b0 0x7f5894003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:58.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.833+0000 7f58a5400640 1 -- 172.21.15.159:0/2239099447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58940051c0 msgr2=0x7f589400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:58.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.833+0000 7f58a5400640 1 --2- 172.21.15.159:0/2239099447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58940051c0 0x7f589400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:58.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.833+0000 7f58a5400640 1 -- 172.21.15.159:0/2239099447 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5894003520 con 0x7f5894004800 2024-04-04T06:14:58.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.834+0000 7f58a5400640 1 --2- 172.21.15.159:0/2239099447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5894004800 0x7f5894004c80 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f589800c210 tx=0x7f5894005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809e831acaeac4a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:58.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.834+0000 7f58ad400640 1 -- 172.21.15.159:0/2239099447 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898007d50 con 0x7f5894004800 2024-04-04T06:14:58.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.834+0000 7f58ad400640 1 -- 172.21.15.159:0/2239099447 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5898030070 con 0x7f5894004800 2024-04-04T06:14:58.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.834+0000 7f58ad400640 1 -- 172.21.15.159:0/2239099447 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589802cdc0 con 0x7f5894004800 2024-04-04T06:14:58.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.835+0000 7f58b5200640 1 -- 172.21.15.159:0/2239099447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5894004800 msgr2=0x7f5894004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:58.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.835+0000 7f58b5200640 1 --2- 172.21.15.159:0/2239099447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5894004800 0x7f5894004c80 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f589800c210 tx=0x7f5894005830 comp rx=0 tx=0).stop 2024-04-04T06:14:58.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.835+0000 7f58ae800640 1 -- 172.21.15.159:0/2239099447 reap_dead start 2024-04-04T06:14:58.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.835+0000 7f58b5200640 1 -- 172.21.15.159:0/2239099447 shutdown_connections 2024-04-04T06:14:58.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.835+0000 7f58b5200640 1 -- 172.21.15.159:0/2239099447 >> 172.21.15.159:0/2239099447 conn(0x7f58b006b750 msgr2=0x7f58b006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:58.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.835+0000 7f58b5200640 1 -- 172.21.15.159:0/2239099447 shutdown_connections 2024-04-04T06:14:58.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.835+0000 7f58b5200640 1 -- 172.21.15.159:0/2239099447 wait complete. 2024-04-04T06:14:58.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.837+0000 7f58b5200640 1 Processor -- start 2024-04-04T06:14:58.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.837+0000 7f58b5200640 1 -- start start 2024-04-04T06:14:58.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.838+0000 7f58b5200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58b0148780 0x7f58b00cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:58.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.838+0000 7f58b5200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b00ce270 0x7f58b00ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:58.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.838+0000 7f58b5200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58b00ca720 0x7f58b00cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:58.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.838+0000 7f58b5200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f58b015d4e0 con 0x7f58b0148780 2024-04-04T06:14:58.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.838+0000 7f58b5200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58b015d200 con 0x7f58b00ca720 2024-04-04T06:14:58.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.838+0000 7f58b5200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f58b0153960 con 0x7f58b00ce270 2024-04-04T06:14:58.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.838+0000 7f58ade00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58b0148780 0x7f58b00cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:58.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.838+0000 7f58ade00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58b0148780 0x7f58b00cd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37612/0 (socket says 172.21.15.159:37612) 2024-04-04T06:14:58.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.838+0000 7f58ade00640 1 -- 172.21.15.159:0/3362841469 learned_addr learned my addr 172.21.15.159:0/3362841469 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:58.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.838+0000 7f58ade00640 1 -- 172.21.15.159:0/3362841469 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b00ce270 msgr2=0x7f58b00ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:58.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.838+0000 7f58ade00640 1 --2- 172.21.15.159:0/3362841469 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b00ce270 0x7f58b00ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:58.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.838+0000 7f58ade00640 1 -- 172.21.15.159:0/3362841469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58b00ca720 msgr2=0x7f58b00cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:58.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.838+0000 7f58ade00640 1 --2- 172.21.15.159:0/3362841469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58b00ca720 0x7f58b00cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:58.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.838+0000 7f58ade00640 1 -- 172.21.15.159:0/3362841469 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b00cdba0 con 0x7f58b0148780 2024-04-04T06:14:58.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.839+0000 7f58ade00640 1 --2- 172.21.15.159:0/3362841469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58b0148780 0x7f58b00cd660 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f58a000e810 tx=0x7f58a000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:58.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.839+0000 7f58a6a00640 1 -- 172.21.15.159:0/3362841469 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a000f040 con 0x7f58b0148780 2024-04-04T06:14:58.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.839+0000 7f58a6a00640 1 -- 172.21.15.159:0/3362841469 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a0004230 con 0x7f58b0148780 2024-04-04T06:14:58.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.839+0000 7f58a6a00640 1 -- 172.21.15.159:0/3362841469 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a000c650 con 0x7f58b0148780 2024-04-04T06:14:58.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.839+0000 7f58b5200640 1 -- 172.21.15.159:0/3362841469 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b00cb3a0 con 0x7f58b0148780 2024-04-04T06:14:58.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.839+0000 7f58b5200640 1 -- 172.21.15.159:0/3362841469 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b00cce60 con 0x7f58b0148780 2024-04-04T06:14:58.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.840+0000 7f58b5200640 1 -- 172.21.15.159:0/3362841469 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b00685c0 con 0x7f58b0148780 2024-04-04T06:14:58.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.841+0000 7f58a6a00640 1 -- 172.21.15.159:0/3362841469 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58a000c7f0 con 0x7f58b0148780 2024-04-04T06:14:58.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.842+0000 7f58a6a00640 1 --2- 172.21.15.159:0/3362841469 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f588c06c8f0 0x7f588c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:58.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.842+0000 7f58a6a00640 1 -- 172.21.15.159:0/3362841469 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7f58a008b6c0 con 0x7f58b0148780 2024-04-04T06:14:58.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.842+0000 7f58a5400640 1 --2- 172.21.15.159:0/3362841469 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f588c06c8f0 0x7f588c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:58.837 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.844+0000 7f58a5400640 1 --2- 172.21.15.159:0/3362841469 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f588c06c8f0 0x7f588c06edb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5898034000 tx=0x7f589802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:58.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:58.846+0000 7f58a6a00640 1 -- 172.21.15.159:0/3362841469 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f58a0017020 con 0x7f58b0148780 2024-04-04T06:14:59.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.030+0000 7f58b5200640 1 -- 172.21.15.159:0/3362841469 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f58b006eca0 con 0x7f58b0148780 2024-04-04T06:14:59.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.031+0000 7f58a6a00640 1 -- 172.21.15.159:0/3362841469 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f58a0059d30 con 0x7f58b0148780 2024-04-04T06:14:59.025 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:14:59.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.033+0000 7f58b5200640 1 -- 172.21.15.159:0/3362841469 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f588c06c8f0 msgr2=0x7f588c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:59.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.033+0000 7f58b5200640 1 --2- 172.21.15.159:0/3362841469 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f588c06c8f0 0x7f588c06edb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5898034000 tx=0x7f589802e000 comp rx=0 tx=0).stop 2024-04-04T06:14:59.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.033+0000 7f58b5200640 1 -- 172.21.15.159:0/3362841469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58b0148780 msgr2=0x7f58b00cd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:59.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.033+0000 7f58b5200640 1 --2- 172.21.15.159:0/3362841469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58b0148780 0x7f58b00cd660 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f58a000e810 tx=0x7f58a000ecd0 comp rx=0 tx=0).stop 2024-04-04T06:14:59.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.033+0000 7f58b5200640 1 -- 172.21.15.159:0/3362841469 shutdown_connections 2024-04-04T06:14:59.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.033+0000 7f58b5200640 1 --2- 172.21.15.159:0/3362841469 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f588c06c8f0 0x7f588c06edb0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:59.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.033+0000 7f58b5200640 1 --2- 172.21.15.159:0/3362841469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58b00ca720 0x7f58b00cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:59.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.033+0000 7f58b5200640 1 --2- 172.21.15.159:0/3362841469 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f58b00ce270 0x7f58b00ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:59.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.033+0000 7f58b5200640 1 --2- 172.21.15.159:0/3362841469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58b0148780 0x7f58b00cd660 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:59.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.033+0000 7f58b5200640 1 -- 172.21.15.159:0/3362841469 >> 172.21.15.159:0/3362841469 conn(0x7f58b006b750 msgr2=0x7f58b006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:59.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.034+0000 7f58b5200640 1 -- 172.21.15.159:0/3362841469 shutdown_connections 2024-04-04T06:14:59.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.034+0000 7f58b5200640 1 -- 172.21.15.159:0/3362841469 wait complete. 2024-04-04T06:14:59.047 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-04T06:14:59.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.259+0000 7fab11e00640 1 Processor -- start 2024-04-04T06:14:59.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.260+0000 7fab11e00640 1 -- start start 2024-04-04T06:14:59.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.261+0000 7fab11e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab0c153580 0x7fab0c0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:59.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.261+0000 7fab11e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fab0c0d07c0 0x7fab0c0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:59.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.261+0000 7fab11e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab0c0afab0 con 0x7fab0c0af2e0 2024-04-04T06:14:59.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.261+0000 7fab11e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fab0c0d0c00 con 0x7fab0c0d07c0 2024-04-04T06:14:59.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.261+0000 7fab11e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fab0c15cc80 con 0x7fab0c153580 2024-04-04T06:14:59.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.262+0000 7fab0b400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fab0c0af2e0 0x7fab0c06d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40014/0 (socket says 172.21.15.159:40014) 2024-04-04T06:14:59.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.262+0000 7fab0b400640 1 -- 172.21.15.159:0/68509192 learned_addr learned my addr 172.21.15.159:0/68509192 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:14:59.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.262+0000 7fab0a000640 1 --2- 172.21.15.159:0/68509192 >> v2:172.21.15.178:3300/0 conn(0x7fab0c0d07c0 0x7fab0c0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:59.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.262+0000 7fab0aa00640 1 --2- 172.21.15.159:0/68509192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab0c153580 0x7fab0c0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:59.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.262+0000 7fab09600640 1 -- 172.21.15.159:0/68509192 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1859792279 0 0) 0x7fab0c0afab0 con 0x7fab0c0af2e0 2024-04-04T06:14:59.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.262+0000 7fab09600640 1 -- 172.21.15.159:0/68509192 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faaf0003520 con 0x7fab0c0af2e0 2024-04-04T06:14:59.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.262+0000 7fab09600640 1 -- 172.21.15.159:0/68509192 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4019354358 0 0) 0x7faaf0003520 con 0x7fab0c0af2e0 2024-04-04T06:14:59.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.262+0000 7fab09600640 1 -- 172.21.15.159:0/68509192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab0c153580 msgr2=0x7fab0c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:59.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.262+0000 7fab09600640 1 --2- 172.21.15.159:0/68509192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab0c153580 0x7fab0c0d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:59.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.262+0000 7fab09600640 1 -- 172.21.15.159:0/68509192 >> v2:172.21.15.178:3300/0 conn(0x7fab0c0d07c0 msgr2=0x7fab0c0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:59.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.262+0000 7fab09600640 1 --2- 172.21.15.159:0/68509192 >> v2:172.21.15.178:3300/0 conn(0x7fab0c0d07c0 0x7fab0c0aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:59.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.262+0000 7fab09600640 1 -- 172.21.15.159:0/68509192 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c15da00 con 0x7fab0c0af2e0 2024-04-04T06:14:59.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.262+0000 7fab09600640 1 -- 172.21.15.159:0/68509192 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fab040019f0 con 0x7fab0c0af2e0 2024-04-04T06:14:59.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.262+0000 7fab09600640 1 -- 172.21.15.159:0/68509192 >> v1:172.21.15.159:6789/0 conn(0x7fab0c0af2e0 legacy=0x7fab0c06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:59.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.262+0000 7fab09600640 1 --2- 172.21.15.159:0/68509192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faaf00039b0 0x7faaf0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:59.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.263+0000 7fab09600640 1 --2- 172.21.15.159:0/68509192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faaf0004800 0x7faaf0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:59.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.263+0000 7fab09600640 1 --2- 172.21.15.159:0/68509192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab0c153580 0x7faaf00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:59.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.263+0000 7fab09600640 1 -- 172.21.15.159:0/68509192 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c0d0c00 con 0x7faaf0004800 2024-04-04T06:14:59.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.263+0000 7fab09600640 1 -- 172.21.15.159:0/68509192 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c15cc80 con 0x7faaf00039b0 2024-04-04T06:14:59.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.263+0000 7fab09600640 1 -- 172.21.15.159:0/68509192 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7faaf000ba10 con 0x7fab0c153580 2024-04-04T06:14:59.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.263+0000 7fab0a000640 1 --2- 172.21.15.159:0/68509192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faaf0004800 0x7faaf0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:59.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.263+0000 7fab0aa00640 1 --2- 172.21.15.159:0/68509192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faaf00039b0 0x7faaf0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:59.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.263+0000 7fab0b400640 1 --2- 172.21.15.159:0/68509192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab0c153580 0x7faaf00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:59.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.263+0000 7fab0a000640 1 -- 172.21.15.159:0/68509192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab0c153580 msgr2=0x7faaf00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:59.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.263+0000 7fab0a000640 1 --2- 172.21.15.159:0/68509192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fab0c153580 0x7faaf00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:59.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.263+0000 7fab0a000640 1 -- 172.21.15.159:0/68509192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faaf00039b0 msgr2=0x7faaf0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:14:59.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.263+0000 7fab0a000640 1 --2- 172.21.15.159:0/68509192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faaf00039b0 0x7faaf0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:59.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.263+0000 7fab0a000640 1 -- 172.21.15.159:0/68509192 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf0003520 con 0x7faaf0004800 2024-04-04T06:14:59.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.264+0000 7fab0a000640 1 --2- 172.21.15.159:0/68509192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faaf0004800 0x7faaf0004c80 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009af0 tx=0x7faaf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf961005cbba7cb server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:59.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.265+0000 7fab09600640 1 -- 172.21.15.159:0/68509192 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8039070 con 0x7faaf0004800 2024-04-04T06:14:59.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.265+0000 7fab09600640 1 -- 172.21.15.159:0/68509192 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf802c710 con 0x7faaf0004800 2024-04-04T06:14:59.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.265+0000 7fab09600640 1 -- 172.21.15.159:0/68509192 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8009690 con 0x7faaf0004800 2024-04-04T06:14:59.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.265+0000 7fab11e00640 1 -- 172.21.15.159:0/68509192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faaf0004800 msgr2=0x7faaf0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:59.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.265+0000 7fab11e00640 1 --2- 172.21.15.159:0/68509192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faaf0004800 0x7faaf0004c80 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009af0 tx=0x7faaf0005830 comp rx=0 tx=0).stop 2024-04-04T06:14:59.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.265+0000 7fab0b400640 1 -- 172.21.15.159:0/68509192 reap_dead start 2024-04-04T06:14:59.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.265+0000 7fab11e00640 1 -- 172.21.15.159:0/68509192 shutdown_connections 2024-04-04T06:14:59.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.265+0000 7fab11e00640 1 -- 172.21.15.159:0/68509192 >> 172.21.15.159:0/68509192 conn(0x7fab0c06b750 msgr2=0x7fab0c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:14:59.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.265+0000 7fab11e00640 1 -- 172.21.15.159:0/68509192 shutdown_connections 2024-04-04T06:14:59.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.265+0000 7fab11e00640 1 -- 172.21.15.159:0/68509192 wait complete. 2024-04-04T06:14:59.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.267+0000 7fab11e00640 1 Processor -- start 2024-04-04T06:14:59.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.267+0000 7fab11e00640 1 -- start start 2024-04-04T06:14:59.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.268+0000 7fab11e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faaf0004800 0x7fab0c0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:59.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.268+0000 7fab11e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab0c14f670 0x7fab0c1483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:59.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.268+0000 7fab11e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab0c072660 0x7fab0c072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:59.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.268+0000 7fab11e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c15d4e0 con 0x7fab0c14f670 2024-04-04T06:14:59.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.268+0000 7fab11e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c15d200 con 0x7fab0c072660 2024-04-04T06:14:59.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.268+0000 7fab11e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fab0c153960 con 0x7faaf0004800 2024-04-04T06:14:59.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.268+0000 7fab0a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab0c14f670 0x7fab0c1483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:59.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.268+0000 7fab0a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab0c14f670 0x7fab0c1483f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37630/0 (socket says 172.21.15.159:37630) 2024-04-04T06:14:59.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.269+0000 7fab0a000640 1 -- 172.21.15.159:0/631432125 learned_addr learned my addr 172.21.15.159:0/631432125 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:14:59.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.269+0000 7fab0aa00640 1 --2- 172.21.15.159:0/631432125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faaf0004800 0x7fab0c0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:59.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.269+0000 7fab0a000640 1 -- 172.21.15.159:0/631432125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faaf0004800 msgr2=0x7fab0c0b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:59.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.269+0000 7fab0a000640 1 --2- 172.21.15.159:0/631432125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faaf0004800 0x7fab0c0b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:59.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.269+0000 7fab0a000640 1 -- 172.21.15.159:0/631432125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab0c072660 msgr2=0x7fab0c072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:14:59.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.269+0000 7fab0a000640 1 --2- 172.21.15.159:0/631432125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab0c072660 0x7fab0c072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:14:59.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.269+0000 7fab0a000640 1 -- 172.21.15.159:0/631432125 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c0b5040 con 0x7fab0c14f670 2024-04-04T06:14:59.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.269+0000 7fab0a000640 1 --2- 172.21.15.159:0/631432125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab0c14f670 0x7fab0c1483f0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009af0 tx=0x7faaf8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:59.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.270+0000 7fab03400640 1 -- 172.21.15.159:0/631432125 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8042020 con 0x7fab0c14f670 2024-04-04T06:14:59.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.270+0000 7fab03400640 1 -- 172.21.15.159:0/631432125 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf803a040 con 0x7fab0c14f670 2024-04-04T06:14:59.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.270+0000 7fab03400640 1 -- 172.21.15.159:0/631432125 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8039070 con 0x7fab0c14f670 2024-04-04T06:14:59.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.270+0000 7fab11e00640 1 -- 172.21.15.159:0/631432125 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0c0ce7b0 con 0x7fab0c14f670 2024-04-04T06:14:59.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.270+0000 7fab11e00640 1 -- 172.21.15.159:0/631432125 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab0c0ced70 con 0x7fab0c14f670 2024-04-04T06:14:59.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.271+0000 7fab11e00640 1 -- 172.21.15.159:0/631432125 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab0c0685c0 con 0x7fab0c14f670 2024-04-04T06:14:59.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.272+0000 7fab03400640 1 -- 172.21.15.159:0/631432125 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faaf8045020 con 0x7fab0c14f670 2024-04-04T06:14:59.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.273+0000 7fab03400640 1 --2- 172.21.15.159:0/631432125 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faaec06c920 0x7faaec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:14:59.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.273+0000 7fab03400640 1 -- 172.21.15.159:0/631432125 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7faaf80acbc0 con 0x7fab0c14f670 2024-04-04T06:14:59.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.273+0000 7fab0aa00640 1 --2- 172.21.15.159:0/631432125 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faaec06c920 0x7faaec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:14:59.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.274+0000 7fab0aa00640 1 --2- 172.21.15.159:0/631432125 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faaec06c920 0x7faaec06ede0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7faafc009930 tx=0x7faafc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:14:59.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.277+0000 7fab03400640 1 -- 172.21.15.159:0/631432125 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7faaf804c020 con 0x7fab0c14f670 2024-04-04T06:14:59.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:14:59.550+0000 7fab11e00640 1 -- 172.21.15.159:0/631432125 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7fab0c06eca0 con 0x7fab0c14f670 2024-04-04T06:15:00.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.151+0000 7fab03400640 1 -- 172.21.15.159:0/631432125 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v442) ==== 149+0+0 (secure 0 0 0) 0x7fab0c0b5040 con 0x7fab0c14f670 2024-04-04T06:15:00.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.154+0000 7fab11e00640 1 -- 172.21.15.159:0/631432125 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faaec06c920 msgr2=0x7faaec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:00.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.154+0000 7fab11e00640 1 --2- 172.21.15.159:0/631432125 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faaec06c920 0x7faaec06ede0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7faafc009930 tx=0x7faafc008040 comp rx=0 tx=0).stop 2024-04-04T06:15:00.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.154+0000 7fab11e00640 1 -- 172.21.15.159:0/631432125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab0c14f670 msgr2=0x7fab0c1483f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:00.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.154+0000 7fab11e00640 1 --2- 172.21.15.159:0/631432125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab0c14f670 0x7fab0c1483f0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009af0 tx=0x7faaf8004730 comp rx=0 tx=0).stop 2024-04-04T06:15:00.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.155+0000 7fab11e00640 1 -- 172.21.15.159:0/631432125 shutdown_connections 2024-04-04T06:15:00.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.155+0000 7fab11e00640 1 --2- 172.21.15.159:0/631432125 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faaec06c920 0x7faaec06ede0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:00.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.155+0000 7fab11e00640 1 --2- 172.21.15.159:0/631432125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab0c072660 0x7fab0c072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:00.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.155+0000 7fab11e00640 1 --2- 172.21.15.159:0/631432125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab0c14f670 0x7fab0c1483f0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:00.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.155+0000 7fab11e00640 1 --2- 172.21.15.159:0/631432125 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faaf0004800 0x7fab0c0b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:00.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.155+0000 7fab11e00640 1 -- 172.21.15.159:0/631432125 >> 172.21.15.159:0/631432125 conn(0x7fab0c06b750 msgr2=0x7fab0c0ad820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:00.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.155+0000 7fab11e00640 1 -- 172.21.15.159:0/631432125 shutdown_connections 2024-04-04T06:15:00.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.155+0000 7fab11e00640 1 -- 172.21.15.159:0/631432125 wait complete. 2024-04-04T06:15:00.173 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-04T06:15:00.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.394+0000 7f8d6d800640 1 Processor -- start 2024-04-04T06:15:00.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.395+0000 7f8d6d800640 1 -- start start 2024-04-04T06:15:00.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.396+0000 7f8d6d800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f8d680b4db0 0x7f8d680cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:00.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.396+0000 7f8d6d800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d680b5ce0 con 0x7f8d680d1580 2024-04-04T06:15:00.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.396+0000 7f8d6d800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d680d27f0 con 0x7f8d68161f60 2024-04-04T06:15:00.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.396+0000 7f8d6d800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f8d680d2a90 con 0x7f8d680b4db0 2024-04-04T06:15:00.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.396+0000 7f8d5e000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f8d680d1580 0x7f8d680cf0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40020/0 (socket says 172.21.15.159:40020) 2024-04-04T06:15:00.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.396+0000 7f8d67400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f8d680b4db0 0x7f8d680cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:00.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.396+0000 7f8d5e000640 1 -- 172.21.15.159:0/1057874468 learned_addr learned my addr 172.21.15.159:0/1057874468 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:00.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.397+0000 7f8d67400640 1 -- 172.21.15.159:0/1057874468 >> v1:172.21.15.178:6789/0 conn(0x7f8d68161f60 legacy=0x7f8d680d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:00.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.397+0000 7f8d67400640 1 -- 172.21.15.159:0/1057874468 >> v1:172.21.15.159:6789/0 conn(0x7f8d680d1580 legacy=0x7f8d680cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:00.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.397+0000 7f8d67400640 1 -- 172.21.15.159:0/1057874468 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d68162340 con 0x7f8d680b4db0 2024-04-04T06:15:00.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.397+0000 7f8d66000640 1 -- 172.21.15.159:0/1057874468 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 199100370 0 0) 0x7f8d50001110 con 0x7f8d680d1580 2024-04-04T06:15:00.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.398+0000 7f8d67400640 1 --2- 172.21.15.159:0/1057874468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d680b4db0 0x7f8d680cf7e0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f8d600097a0 tx=0x7f8d6002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5213e6c568d92061 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:00.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.398+0000 7f8d66000640 1 -- 172.21.15.159:0/1057874468 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60035020 con 0x7f8d680b4db0 2024-04-04T06:15:00.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.398+0000 7f8d66000640 1 -- 172.21.15.159:0/1057874468 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d60031830 con 0x7f8d680b4db0 2024-04-04T06:15:00.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.398+0000 7f8d66000640 1 -- 172.21.15.159:0/1057874468 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60031c00 con 0x7f8d680b4db0 2024-04-04T06:15:00.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.399+0000 7f8d6d800640 1 -- 172.21.15.159:0/1057874468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d680b4db0 msgr2=0x7f8d680cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:00.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.399+0000 7f8d6d800640 1 --2- 172.21.15.159:0/1057874468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d680b4db0 0x7f8d680cf7e0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f8d600097a0 tx=0x7f8d6002b8d0 comp rx=0 tx=0).stop 2024-04-04T06:15:00.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.399+0000 7f8d6d800640 1 -- 172.21.15.159:0/1057874468 shutdown_connections 2024-04-04T06:15:00.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.399+0000 7f8d6d800640 1 --2- 172.21.15.159:0/1057874468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d680b4db0 0x7f8d680cf7e0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:00.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.399+0000 7f8d6d800640 1 -- 172.21.15.159:0/1057874468 >> 172.21.15.159:0/1057874468 conn(0x7f8d6808bfb0 msgr2=0x7f8d680d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:00.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.399+0000 7f8d6d800640 1 -- 172.21.15.159:0/1057874468 shutdown_connections 2024-04-04T06:15:00.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.399+0000 7f8d6d800640 1 -- 172.21.15.159:0/1057874468 wait complete. 2024-04-04T06:15:00.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.401+0000 7f8d6d800640 1 Processor -- start 2024-04-04T06:15:00.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.401+0000 7f8d6d800640 1 -- start start 2024-04-04T06:15:00.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.402+0000 7f8d6d800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d680b4db0 0x7f8d6814e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:00.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.402+0000 7f8d6d800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d680d1580 0x7f8d6814eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:00.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.402+0000 7f8d6d800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d68161f60 0x7f8d6814f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:00.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.402+0000 7f8d6d800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8d680d3310 con 0x7f8d680b4db0 2024-04-04T06:15:00.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.402+0000 7f8d6d800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8d680d3030 con 0x7f8d68161f60 2024-04-04T06:15:00.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.402+0000 7f8d6d800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8d680b3170 con 0x7f8d680d1580 2024-04-04T06:15:00.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.402+0000 7f8d66a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d680b4db0 0x7f8d6814e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:00.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.402+0000 7f8d66a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d680b4db0 0x7f8d6814e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37636/0 (socket says 172.21.15.159:37636) 2024-04-04T06:15:00.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.402+0000 7f8d66a00640 1 -- 172.21.15.159:0/1506260996 learned_addr learned my addr 172.21.15.159:0/1506260996 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:00.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.403+0000 7f8d5e000640 1 --2- 172.21.15.159:0/1506260996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d680d1580 0x7f8d6814eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:00.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.403+0000 7f8d66a00640 1 -- 172.21.15.159:0/1506260996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d680d1580 msgr2=0x7f8d6814eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:00.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.403+0000 7f8d66a00640 1 --2- 172.21.15.159:0/1506260996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d680d1580 0x7f8d6814eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:00.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.403+0000 7f8d66a00640 1 -- 172.21.15.159:0/1506260996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d68161f60 msgr2=0x7f8d6814f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:00.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.403+0000 7f8d66a00640 1 --2- 172.21.15.159:0/1506260996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d68161f60 0x7f8d6814f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:00.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.403+0000 7f8d66a00640 1 -- 172.21.15.159:0/1506260996 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d680c9600 con 0x7f8d680b4db0 2024-04-04T06:15:00.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.403+0000 7f8d66a00640 1 --2- 172.21.15.159:0/1506260996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d680b4db0 0x7f8d6814e220 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800ba50 tx=0x7f8d5800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:00.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.403+0000 7f8d5f400640 1 -- 172.21.15.159:0/1506260996 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58019070 con 0x7f8d680b4db0 2024-04-04T06:15:00.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.403+0000 7f8d5f400640 1 -- 172.21.15.159:0/1506260996 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d5800cc20 con 0x7f8d680b4db0 2024-04-04T06:15:00.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.403+0000 7f8d5f400640 1 -- 172.21.15.159:0/1506260996 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58014440 con 0x7f8d680b4db0 2024-04-04T06:15:00.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.403+0000 7f8d6d800640 1 -- 172.21.15.159:0/1506260996 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d680c98e0 con 0x7f8d680b4db0 2024-04-04T06:15:00.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.404+0000 7f8d6d800640 1 -- 172.21.15.159:0/1506260996 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d680c9ea0 con 0x7f8d680b4db0 2024-04-04T06:15:00.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.405+0000 7f8d6d800640 1 -- 172.21.15.159:0/1506260996 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d680685c0 con 0x7f8d680b4db0 2024-04-04T06:15:00.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.406+0000 7f8d5f400640 1 -- 172.21.15.159:0/1506260996 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8d5801c070 con 0x7f8d680b4db0 2024-04-04T06:15:00.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.406+0000 7f8d5f400640 1 --2- 172.21.15.159:0/1506260996 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8d4406c8f0 0x7f8d4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:00.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.407+0000 7f8d5f400640 1 -- 172.21.15.159:0/1506260996 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7f8d5808b550 con 0x7f8d680b4db0 2024-04-04T06:15:00.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.407+0000 7f8d5e000640 1 --2- 172.21.15.159:0/1506260996 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8d4406c8f0 0x7f8d4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:00.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.408+0000 7f8d5e000640 1 --2- 172.21.15.159:0/1506260996 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8d4406c8f0 0x7f8d4406edb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8d680d0be0 tx=0x7f8d50009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:00.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.411+0000 7f8d5f400640 1 -- 172.21.15.159:0/1506260996 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8d58059610 con 0x7f8d680b4db0 2024-04-04T06:15:00.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.612+0000 7f8d6d800640 1 -- 172.21.15.159:0/1506260996 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f8d680cecd0 con 0x7f8d680b4db0 2024-04-04T06:15:00.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.618+0000 7f8d5f400640 1 -- 172.21.15.159:0/1506260996 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f8d5805d910 con 0x7f8d680b4db0 2024-04-04T06:15:00.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.620+0000 7f8d6d800640 1 -- 172.21.15.159:0/1506260996 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8d4406c8f0 msgr2=0x7f8d4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:00.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.620+0000 7f8d6d800640 1 --2- 172.21.15.159:0/1506260996 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8d4406c8f0 0x7f8d4406edb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8d680d0be0 tx=0x7f8d50009040 comp rx=0 tx=0).stop 2024-04-04T06:15:00.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.620+0000 7f8d6d800640 1 -- 172.21.15.159:0/1506260996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d680b4db0 msgr2=0x7f8d6814e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:00.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.620+0000 7f8d6d800640 1 --2- 172.21.15.159:0/1506260996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d680b4db0 0x7f8d6814e220 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800ba50 tx=0x7f8d5800bf10 comp rx=0 tx=0).stop 2024-04-04T06:15:00.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.621+0000 7f8d6d800640 1 -- 172.21.15.159:0/1506260996 shutdown_connections 2024-04-04T06:15:00.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.621+0000 7f8d6d800640 1 --2- 172.21.15.159:0/1506260996 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8d4406c8f0 0x7f8d4406edb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:00.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.621+0000 7f8d6d800640 1 --2- 172.21.15.159:0/1506260996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d68161f60 0x7f8d6814f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:00.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.621+0000 7f8d6d800640 1 --2- 172.21.15.159:0/1506260996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8d680d1580 0x7f8d6814eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:00.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.621+0000 7f8d6d800640 1 --2- 172.21.15.159:0/1506260996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d680b4db0 0x7f8d6814e220 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:00.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.621+0000 7f8d6d800640 1 -- 172.21.15.159:0/1506260996 >> 172.21.15.159:0/1506260996 conn(0x7f8d6808bfb0 msgr2=0x7f8d680b5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:00.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.621+0000 7f8d6d800640 1 -- 172.21.15.159:0/1506260996 shutdown_connections 2024-04-04T06:15:00.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:00.621+0000 7f8d6d800640 1 -- 172.21.15.159:0/1506260996 wait complete. 2024-04-04T06:15:00.633 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-04T06:15:00.633 DEBUG:teuthology.orchestra.run.smithi159:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:15:00.743 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:15:00.868 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-04T06:15:00.868 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:15:00.868 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:15:00.921 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:15:00.922 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:15:00.922 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:15:00.978 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:15:00.978 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:15:00.979 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:15:01.037 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:15:01.038 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:15:01.038 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:15:01.092 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:15:01.093 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-04T06:15:01.093 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:15:01.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.317+0000 7f1b48c00640 1 Processor -- start 2024-04-04T06:15:01.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.317+0000 7f1b48c00640 1 -- start start 2024-04-04T06:15:01.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.318+0000 7f1b48c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f1b4415b310 0x7f1b440727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:01.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.318+0000 7f1b48c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1b440b4f90 0x7f1b4415f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:01.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.319+0000 7f1b48c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b4415b6f0 con 0x7f1b440d0780 2024-04-04T06:15:01.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.319+0000 7f1b48c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f1b44072e80 con 0x7f1b4415b310 2024-04-04T06:15:01.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.319+0000 7f1b48c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f1b44073160 con 0x7f1b440b4f90 2024-04-04T06:15:01.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.319+0000 7f1b42800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f1b440d0780 0x7f1b440adc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40026/0 (socket says 172.21.15.159:40026) 2024-04-04T06:15:01.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.319+0000 7f1b42800640 1 -- 172.21.15.159:0/525807369 learned_addr learned my addr 172.21.15.159:0/525807369 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:01.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.319+0000 7f1b41e00640 1 --2- 172.21.15.159:0/525807369 >> v2:172.21.15.159:3300/0 conn(0x7f1b4415b310 0x7f1b440727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:01.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.319+0000 7f1b41400640 1 --2- 172.21.15.159:0/525807369 >> v2:172.21.15.178:3300/0 conn(0x7f1b440b4f90 0x7f1b4415f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:01.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.319+0000 7f1b40a00640 1 -- 172.21.15.159:0/525807369 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3806449002 0 0) 0x7f1b4415b6f0 con 0x7f1b440d0780 2024-04-04T06:15:01.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.320+0000 7f1b40a00640 1 -- 172.21.15.159:0/525807369 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b28003520 con 0x7f1b440d0780 2024-04-04T06:15:01.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.320+0000 7f1b41e00640 1 -- 172.21.15.159:0/525807369 >> v2:172.21.15.178:3300/0 conn(0x7f1b440b4f90 msgr2=0x7f1b4415f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:01.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.320+0000 7f1b41e00640 1 --2- 172.21.15.159:0/525807369 >> v2:172.21.15.178:3300/0 conn(0x7f1b440b4f90 0x7f1b4415f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:01.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.320+0000 7f1b41e00640 1 -- 172.21.15.159:0/525807369 >> v1:172.21.15.159:6789/0 conn(0x7f1b440d0780 legacy=0x7f1b440adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:01.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.320+0000 7f1b41e00640 1 -- 172.21.15.159:0/525807369 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4415efe0 con 0x7f1b4415b310 2024-04-04T06:15:01.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.321+0000 7f1b41e00640 1 --2- 172.21.15.159:0/525807369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b4415b310 0x7f1b440727f0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f1b44071200 tx=0x7f1b3402b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383243d4773e15de server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:01.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.321+0000 7f1b40a00640 1 -- 172.21.15.159:0/525807369 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34038070 con 0x7f1b4415b310 2024-04-04T06:15:01.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.321+0000 7f1b40a00640 1 -- 172.21.15.159:0/525807369 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b34031520 con 0x7f1b4415b310 2024-04-04T06:15:01.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.321+0000 7f1b40a00640 1 -- 172.21.15.159:0/525807369 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b340318f0 con 0x7f1b4415b310 2024-04-04T06:15:01.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.321+0000 7f1b48c00640 1 -- 172.21.15.159:0/525807369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b4415b310 msgr2=0x7f1b440727f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:01.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.321+0000 7f1b48c00640 1 --2- 172.21.15.159:0/525807369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b4415b310 0x7f1b440727f0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f1b44071200 tx=0x7f1b3402b7c0 comp rx=0 tx=0).stop 2024-04-04T06:15:01.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.322+0000 7f1b48c00640 1 -- 172.21.15.159:0/525807369 shutdown_connections 2024-04-04T06:15:01.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.322+0000 7f1b48c00640 1 --2- 172.21.15.159:0/525807369 >> v2:172.21.15.178:3300/0 conn(0x7f1b440b4f90 0x7f1b4415f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:01.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.322+0000 7f1b48c00640 1 --2- 172.21.15.159:0/525807369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b4415b310 0x7f1b440727f0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:01.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.322+0000 7f1b48c00640 1 -- 172.21.15.159:0/525807369 >> 172.21.15.159:0/525807369 conn(0x7f1b4406b990 msgr2=0x7f1b4406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:01.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.322+0000 7f1b48c00640 1 -- 172.21.15.159:0/525807369 shutdown_connections 2024-04-04T06:15:01.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.322+0000 7f1b48c00640 1 -- 172.21.15.159:0/525807369 wait complete. 2024-04-04T06:15:01.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.323+0000 7f1b48c00640 1 Processor -- start 2024-04-04T06:15:01.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.324+0000 7f1b48c00640 1 -- start start 2024-04-04T06:15:01.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.325+0000 7f1b48c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b440b4f90 0x7f1b44150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:01.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.325+0000 7f1b48c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b440d0780 0x7f1b4414e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:01.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.325+0000 7f1b48c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b4415b310 0x7f1b4414ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:01.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.325+0000 7f1b48c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1b44073440 con 0x7f1b4415b310 2024-04-04T06:15:01.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.325+0000 7f1b48c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1b440b53f0 con 0x7f1b440b4f90 2024-04-04T06:15:01.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.325+0000 7f1b48c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1b4415fd20 con 0x7f1b440d0780 2024-04-04T06:15:01.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.325+0000 7f1b42800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b4415b310 0x7f1b4414ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:01.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.325+0000 7f1b42800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b4415b310 0x7f1b4414ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37658/0 (socket says 172.21.15.159:37658) 2024-04-04T06:15:01.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.325+0000 7f1b42800640 1 -- 172.21.15.159:0/4101080680 learned_addr learned my addr 172.21.15.159:0/4101080680 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:01.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.325+0000 7f1b41e00640 1 --2- 172.21.15.159:0/4101080680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b440b4f90 0x7f1b44150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:01.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.325+0000 7f1b41400640 1 --2- 172.21.15.159:0/4101080680 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b440d0780 0x7f1b4414e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:01.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.325+0000 7f1b42800640 1 -- 172.21.15.159:0/4101080680 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b440d0780 msgr2=0x7f1b4414e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:01.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.325+0000 7f1b42800640 1 --2- 172.21.15.159:0/4101080680 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b440d0780 0x7f1b4414e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:01.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.325+0000 7f1b42800640 1 -- 172.21.15.159:0/4101080680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b440b4f90 msgr2=0x7f1b44150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:01.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.325+0000 7f1b42800640 1 --2- 172.21.15.159:0/4101080680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b440b4f90 0x7f1b44150a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:01.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.325+0000 7f1b42800640 1 -- 172.21.15.159:0/4101080680 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b44151370 con 0x7f1b4415b310 2024-04-04T06:15:01.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.325+0000 7f1b41400640 1 --2- 172.21.15.159:0/4101080680 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b440d0780 0x7f1b4414e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:01.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.326+0000 7f1b42800640 1 --2- 172.21.15.159:0/4101080680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b4415b310 0x7f1b4414ea60 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00cad0 tx=0x7f1b3c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:01.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.326+0000 7f1b3a000640 1 -- 172.21.15.159:0/4101080680 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c017070 con 0x7f1b4415b310 2024-04-04T06:15:01.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.326+0000 7f1b3a000640 1 -- 172.21.15.159:0/4101080680 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3c00ac30 con 0x7f1b4415b310 2024-04-04T06:15:01.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.326+0000 7f1b48c00640 1 -- 172.21.15.159:0/4101080680 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4415c540 con 0x7f1b4415b310 2024-04-04T06:15:01.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.326+0000 7f1b3a000640 1 -- 172.21.15.159:0/4101080680 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c012400 con 0x7f1b4415b310 2024-04-04T06:15:01.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.326+0000 7f1b48c00640 1 -- 172.21.15.159:0/4101080680 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b4415cb00 con 0x7f1b4415b310 2024-04-04T06:15:01.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.327+0000 7f1b48c00640 1 -- 172.21.15.159:0/4101080680 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b440685c0 con 0x7f1b4415b310 2024-04-04T06:15:01.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.328+0000 7f1b3a000640 1 -- 172.21.15.159:0/4101080680 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b3c01a070 con 0x7f1b4415b310 2024-04-04T06:15:01.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.329+0000 7f1b3a000640 1 --2- 172.21.15.159:0/4101080680 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1b2406c8f0 0x7f1b2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:01.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.329+0000 7f1b3a000640 1 -- 172.21.15.159:0/4101080680 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7f1b3c08b680 con 0x7f1b4415b310 2024-04-04T06:15:01.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.330+0000 7f1b41e00640 1 --2- 172.21.15.159:0/4101080680 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1b2406c8f0 0x7f1b2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:01.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.331+0000 7f1b41e00640 1 --2- 172.21.15.159:0/4101080680 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1b2406c8f0 0x7f1b2406edb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1b34035000 tx=0x7f1b34030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:01.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.333+0000 7f1b3a000640 1 -- 172.21.15.159:0/4101080680 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1b3c059d00 con 0x7f1b4415b310 2024-04-04T06:15:01.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.573+0000 7f1b48c00640 1 -- 172.21.15.159:0/4101080680 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b4415ec20 con 0x7f1b4415b310 2024-04-04T06:15:01.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.574+0000 7f1b3a000640 1 -- 172.21.15.159:0/4101080680 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+3881 (secure 0 0 0) 0x7f1b3c059600 con 0x7f1b4415b310 2024-04-04T06:15:01.568 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:15:01.568 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":78,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5752,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1825651864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1825651864},{"type":"v1","addr":"172.21.15.178:6836","nonce":1825651864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5755,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1824578712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1824578712},{"type":"v1","addr":"172.21.15.178:6837","nonce":1824578712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6834/3377498298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3377498298},{"type":"v1","addr":"172.21.15.159:6834","nonce":3377498298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":8109,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2897299316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":2897299316},{"type":"v1","addr":"172.21.15.159:6835","nonce":2897299316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[]} 2024-04-04T06:15:01.568 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 78 2024-04-04T06:15:01.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.576+0000 7f1b48c00640 1 -- 172.21.15.159:0/4101080680 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1b2406c8f0 msgr2=0x7f1b2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:01.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.576+0000 7f1b48c00640 1 --2- 172.21.15.159:0/4101080680 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1b2406c8f0 0x7f1b2406edb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1b34035000 tx=0x7f1b34030000 comp rx=0 tx=0).stop 2024-04-04T06:15:01.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.576+0000 7f1b48c00640 1 -- 172.21.15.159:0/4101080680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b4415b310 msgr2=0x7f1b4414ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:01.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.576+0000 7f1b48c00640 1 --2- 172.21.15.159:0/4101080680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b4415b310 0x7f1b4414ea60 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00cad0 tx=0x7f1b3c00cf90 comp rx=0 tx=0).stop 2024-04-04T06:15:01.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.576+0000 7f1b48c00640 1 -- 172.21.15.159:0/4101080680 shutdown_connections 2024-04-04T06:15:01.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.576+0000 7f1b48c00640 1 --2- 172.21.15.159:0/4101080680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b4415b310 0x7f1b4414ea60 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:01.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.576+0000 7f1b48c00640 1 --2- 172.21.15.159:0/4101080680 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1b440d0780 0x7f1b4414e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:01.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.576+0000 7f1b48c00640 1 --2- 172.21.15.159:0/4101080680 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1b2406c8f0 0x7f1b2406edb0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:01.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.576+0000 7f1b48c00640 1 --2- 172.21.15.159:0/4101080680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b440b4f90 0x7f1b44150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:01.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.577+0000 7f1b48c00640 1 -- 172.21.15.159:0/4101080680 >> 172.21.15.159:0/4101080680 conn(0x7f1b4406b990 msgr2=0x7f1b4406d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:01.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.577+0000 7f1b48c00640 1 -- 172.21.15.159:0/4101080680 shutdown_connections 2024-04-04T06:15:01.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.577+0000 7f1b48c00640 1 -- 172.21.15.159:0/4101080680 wait complete. 2024-04-04T06:15:01.593 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-04T06:15:01.593 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-04T06:15:01.594 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-04T06:15:01.594 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:15:01.594 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-04T06:15:01.594 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-04T06:15:01.594 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-04T06:15:01.594 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:15:01.594 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-04T06:15:01.595 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-04T06:15:01.595 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-04T06:15:01.595 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:15:01.596 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-04T06:15:01.596 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-04T06:15:01.596 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-04T06:15:01.596 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:15:01.611 INFO:tasks.ceph.mds.a:Stopped 2024-04-04T06:15:01.611 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-04T06:15:01.612 INFO:tasks.ceph.mds.c:Stopped 2024-04-04T06:15:01.612 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-04T06:15:01.614 INFO:tasks.ceph.mds.b:Stopped 2024-04-04T06:15:01.614 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-04T06:15:01.620 INFO:tasks.ceph.mds.d:Stopped 2024-04-04T06:15:01.621 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-04T06:15:01.658 INFO:tasks.ceph.mds.a:Started 2024-04-04T06:15:01.659 DEBUG:teuthology.parallel:result is None 2024-04-04T06:15:01.665 INFO:tasks.ceph.mds.c:Started 2024-04-04T06:15:01.665 DEBUG:teuthology.parallel:result is None 2024-04-04T06:15:01.667 INFO:tasks.ceph.mds.b:Started 2024-04-04T06:15:01.668 DEBUG:teuthology.parallel:result is None 2024-04-04T06:15:01.674 INFO:tasks.ceph.mds.d:Started 2024-04-04T06:15:01.674 DEBUG:teuthology.parallel:result is None 2024-04-04T06:15:01.674 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-04T06:15:01.812 INFO:tasks.ceph.mds.c.smithi159.stdout:starting mds.c at 2024-04-04T06:15:01.815 INFO:tasks.ceph.mds.a.smithi159.stdout:starting mds.a at 2024-04-04T06:15:01.823 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-04T06:15:01.823 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-04T06:15:01.828 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-04T06:15:01.829 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-04T06:15:01.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.862+0000 7f26d8a00640 1 Processor -- start 2024-04-04T06:15:01.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.862+0000 7f26d8a00640 1 -- start start 2024-04-04T06:15:01.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.864+0000 7f26d8a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26d4070170 0x7f26d406b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:01.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.864+0000 7f26d8a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f26d40705f0 0x7f26d406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:01.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.864+0000 7f26d8a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d415edc0 con 0x7f26d40d1bf0 2024-04-04T06:15:01.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.864+0000 7f26d8a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f26d40d1fd0 con 0x7f26d40705f0 2024-04-04T06:15:01.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.864+0000 7f26d8a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f26d415eb80 con 0x7f26d4070170 2024-04-04T06:15:01.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.864+0000 7f26d1e00640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f26d40d1bf0 0x7f26d415f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.159:40410/0 (socket says 172.21.15.159:40410) 2024-04-04T06:15:01.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.864+0000 7f26d1e00640 1 -- 172.21.15.159:0/2822516897 learned_addr learned my addr 172.21.15.159:0/2822516897 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:01.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.864+0000 7f26d2800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f26d40705f0 0x7f26d406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:01.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.864+0000 7f26d1400640 1 --2- 172.21.15.159:0/2822516897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26d4070170 0x7f26d406b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:01.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.864+0000 7f26d0a00640 1 -- 172.21.15.159:0/2822516897 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 399402626 0 0) 0x7f26d415edc0 con 0x7f26d40d1bf0 2024-04-04T06:15:01.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.864+0000 7f26d0a00640 1 -- 172.21.15.159:0/2822516897 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26b8003520 con 0x7f26d40d1bf0 2024-04-04T06:15:01.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.864+0000 7f26d1400640 1 -- 172.21.15.159:0/2822516897 >> v2:172.21.15.159:3300/0 conn(0x7f26d40705f0 msgr2=0x7f26d406bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:01.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.864+0000 7f26d1400640 1 --2- 172.21.15.159:0/2822516897 >> v2:172.21.15.159:3300/0 conn(0x7f26d40705f0 0x7f26d406bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:01.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.864+0000 7f26d1400640 1 -- 172.21.15.159:0/2822516897 >> v1:172.21.15.178:6789/0 conn(0x7f26d40d1bf0 legacy=0x7f26d415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:01.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.864+0000 7f26d1400640 1 -- 172.21.15.159:0/2822516897 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d40b5e60 con 0x7f26d4070170 2024-04-04T06:15:01.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.865+0000 7f26d2800640 1 --2- 172.21.15.159:0/2822516897 >> v2:172.21.15.159:3300/0 conn(0x7f26d40705f0 0x7f26d406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:15:01.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.866+0000 7f26d1400640 1 --2- 172.21.15.159:0/2822516897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26d4070170 0x7f26d406b570 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f26c00097a0 tx=0x7f26c002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90ed2979c0c9cb78 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:01.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.866+0000 7f26d0a00640 1 -- 172.21.15.159:0/2822516897 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0031780 con 0x7f26d4070170 2024-04-04T06:15:01.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.866+0000 7f26d0a00640 1 -- 172.21.15.159:0/2822516897 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c0037070 con 0x7f26d4070170 2024-04-04T06:15:01.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.866+0000 7f26d0a00640 1 -- 172.21.15.159:0/2822516897 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0031b40 con 0x7f26d4070170 2024-04-04T06:15:01.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.867+0000 7f26d8a00640 1 -- 172.21.15.159:0/2822516897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26d4070170 msgr2=0x7f26d406b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:01.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.867+0000 7f26d8a00640 1 --2- 172.21.15.159:0/2822516897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26d4070170 0x7f26d406b570 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f26c00097a0 tx=0x7f26c002b820 comp rx=0 tx=0).stop 2024-04-04T06:15:01.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.867+0000 7f26d8a00640 1 -- 172.21.15.159:0/2822516897 shutdown_connections 2024-04-04T06:15:01.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.867+0000 7f26d8a00640 1 --2- 172.21.15.159:0/2822516897 >> v2:172.21.15.159:3300/0 conn(0x7f26d40705f0 0x7f26d406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:01.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.867+0000 7f26d8a00640 1 --2- 172.21.15.159:0/2822516897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26d4070170 0x7f26d406b570 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:01.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.867+0000 7f26d8a00640 1 -- 172.21.15.159:0/2822516897 >> 172.21.15.159:0/2822516897 conn(0x7f26d40b6560 msgr2=0x7f26d40b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:01.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.867+0000 7f26d8a00640 1 -- 172.21.15.159:0/2822516897 shutdown_connections 2024-04-04T06:15:01.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.867+0000 7f26d8a00640 1 -- 172.21.15.159:0/2822516897 wait complete. 2024-04-04T06:15:01.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.869+0000 7f26d8a00640 1 Processor -- start 2024-04-04T06:15:01.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.869+0000 7f26d8a00640 1 -- start start 2024-04-04T06:15:01.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.870+0000 7f26d8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26d4070170 0x7f26d41f3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:01.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.870+0000 7f26d8a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26d40705f0 0x7f26d41f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:01.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.870+0000 7f26d8a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26d40d1bf0 0x7f26d4147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:01.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.870+0000 7f26d8a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f26d40b5b10 con 0x7f26d4070170 2024-04-04T06:15:01.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.870+0000 7f26d8a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f26d415f080 con 0x7f26d40705f0 2024-04-04T06:15:01.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.870+0000 7f26d8a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f26d40b4220 con 0x7f26d40d1bf0 2024-04-04T06:15:01.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.870+0000 7f26d1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26d4070170 0x7f26d41f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:01.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.870+0000 7f26d1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26d4070170 0x7f26d41f3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37690/0 (socket says 172.21.15.159:37690) 2024-04-04T06:15:01.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.870+0000 7f26d1e00640 1 -- 172.21.15.159:0/355529810 learned_addr learned my addr 172.21.15.159:0/355529810 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:01.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.870+0000 7f26d1400640 1 --2- 172.21.15.159:0/355529810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26d40705f0 0x7f26d41f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:01.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.870+0000 7f26d2800640 1 --2- 172.21.15.159:0/355529810 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26d40d1bf0 0x7f26d4147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:01.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.870+0000 7f26d1e00640 1 -- 172.21.15.159:0/355529810 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26d40d1bf0 msgr2=0x7f26d4147b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:01.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.870+0000 7f26d1e00640 1 --2- 172.21.15.159:0/355529810 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26d40d1bf0 0x7f26d4147b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:01.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.870+0000 7f26d1e00640 1 -- 172.21.15.159:0/355529810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26d40705f0 msgr2=0x7f26d41f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:01.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.870+0000 7f26d1e00640 1 --2- 172.21.15.159:0/355529810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26d40705f0 0x7f26d41f3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:01.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.870+0000 7f26d1e00640 1 -- 172.21.15.159:0/355529810 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d415abd0 con 0x7f26d4070170 2024-04-04T06:15:01.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.870+0000 7f26d2800640 1 --2- 172.21.15.159:0/355529810 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26d40d1bf0 0x7f26d4147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:01.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.871+0000 7f26d1e00640 1 --2- 172.21.15.159:0/355529810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26d4070170 0x7f26d41f3250 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f26c400c390 tx=0x7f26c400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:01.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.871+0000 7f26ca000640 1 -- 172.21.15.159:0/355529810 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4017070 con 0x7f26d4070170 2024-04-04T06:15:01.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.871+0000 7f26ca000640 1 -- 172.21.15.159:0/355529810 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c4005ce0 con 0x7f26d4070170 2024-04-04T06:15:01.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.871+0000 7f26ca000640 1 -- 172.21.15.159:0/355529810 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c40072f0 con 0x7f26d4070170 2024-04-04T06:15:01.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.871+0000 7f26d8a00640 1 -- 172.21.15.159:0/355529810 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d415aeb0 con 0x7f26d4070170 2024-04-04T06:15:01.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.871+0000 7f26d8a00640 1 -- 172.21.15.159:0/355529810 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d415b470 con 0x7f26d4070170 2024-04-04T06:15:01.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.872+0000 7f26d8a00640 1 -- 172.21.15.159:0/355529810 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26d40685c0 con 0x7f26d4070170 2024-04-04T06:15:01.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.873+0000 7f26ca000640 1 -- 172.21.15.159:0/355529810 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26c401a070 con 0x7f26d4070170 2024-04-04T06:15:01.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.874+0000 7f26ca000640 1 --2- 172.21.15.159:0/355529810 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f26b406c8f0 0x7f26b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:01.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.874+0000 7f26ca000640 1 -- 172.21.15.159:0/355529810 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7f26c408adc0 con 0x7f26d4070170 2024-04-04T06:15:01.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.874+0000 7f26d1400640 1 --2- 172.21.15.159:0/355529810 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f26b406c8f0 0x7f26b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:01.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.875+0000 7f26d1400640 1 --2- 172.21.15.159:0/355529810 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f26b406c8f0 0x7f26b406edb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f26c0036000 tx=0x7f26c003d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:01.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:01.878+0000 7f26ca000640 1 -- 172.21.15.159:0/355529810 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f26c40593a0 con 0x7f26d4070170 2024-04-04T06:15:02.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:02.089+0000 7f26d8a00640 1 -- 172.21.15.159:0/355529810 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f26d40afad0 con 0x7f26d4070170 2024-04-04T06:15:03.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.187+0000 7f26ca000640 1 -- 172.21.15.159:0/355529810 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v97) ==== 97+0+0 (secure 0 0 0) 0x7f26c4058ca0 con 0x7f26d4070170 2024-04-04T06:15:03.182 INFO:teuthology.orchestra.run.smithi159.stderr: removed all blocklist entries 2024-04-04T06:15:03.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.191+0000 7f26d8a00640 1 -- 172.21.15.159:0/355529810 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f26b406c8f0 msgr2=0x7f26b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:03.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.192+0000 7f26d8a00640 1 --2- 172.21.15.159:0/355529810 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f26b406c8f0 0x7f26b406edb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f26c0036000 tx=0x7f26c003d040 comp rx=0 tx=0).stop 2024-04-04T06:15:03.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.192+0000 7f26d8a00640 1 -- 172.21.15.159:0/355529810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26d4070170 msgr2=0x7f26d41f3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:03.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.192+0000 7f26d8a00640 1 --2- 172.21.15.159:0/355529810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26d4070170 0x7f26d41f3250 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f26c400c390 tx=0x7f26c400c850 comp rx=0 tx=0).stop 2024-04-04T06:15:03.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.192+0000 7f26d8a00640 1 -- 172.21.15.159:0/355529810 shutdown_connections 2024-04-04T06:15:03.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.192+0000 7f26d8a00640 1 --2- 172.21.15.159:0/355529810 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f26d40d1bf0 0x7f26d4147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:03.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.192+0000 7f26d8a00640 1 --2- 172.21.15.159:0/355529810 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f26b406c8f0 0x7f26b406edb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:03.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.192+0000 7f26d8a00640 1 --2- 172.21.15.159:0/355529810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26d40705f0 0x7f26d41f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:03.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.192+0000 7f26d8a00640 1 --2- 172.21.15.159:0/355529810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26d4070170 0x7f26d41f3250 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:03.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.192+0000 7f26d8a00640 1 -- 172.21.15.159:0/355529810 >> 172.21.15.159:0/355529810 conn(0x7f26d40b6560 msgr2=0x7f26d415f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:03.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.192+0000 7f26d8a00640 1 -- 172.21.15.159:0/355529810 shutdown_connections 2024-04-04T06:15:03.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.192+0000 7f26d8a00640 1 -- 172.21.15.159:0/355529810 wait complete. 2024-04-04T06:15:03.207 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-04T06:15:03.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.423+0000 7f1e5ea00640 1 Processor -- start 2024-04-04T06:15:03.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.423+0000 7f1e5ea00640 1 -- start start 2024-04-04T06:15:03.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.424+0000 7f1e5ea00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1e580d0990 0x7f1e580aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:03.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.424+0000 7f1e5ea00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f1e580af2e0 0x7f1e581600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:03.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.424+0000 7f1e5ea00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e580afab0 con 0x7f1e58153580 2024-04-04T06:15:03.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.424+0000 7f1e5ea00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f1e5815cc80 con 0x7f1e580af2e0 2024-04-04T06:15:03.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.424+0000 7f1e5ea00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f1e5815da00 con 0x7f1e580d0990 2024-04-04T06:15:03.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.425+0000 7f1e57e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f1e58153580 0x7f1e580d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40032/0 (socket says 172.21.15.159:40032) 2024-04-04T06:15:03.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.425+0000 7f1e57e00640 1 -- 172.21.15.159:0/3416006545 learned_addr learned my addr 172.21.15.159:0/3416006545 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:03.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.425+0000 7f1e5ca00640 1 --2- 172.21.15.159:0/3416006545 >> v2:172.21.15.159:3300/0 conn(0x7f1e580af2e0 0x7f1e581600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:03.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.425+0000 7f1e56a00640 1 -- 172.21.15.159:0/3416006545 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 650736718 0 0) 0x7f1e580afab0 con 0x7f1e58153580 2024-04-04T06:15:03.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.425+0000 7f1e56a00640 1 -- 172.21.15.159:0/3416006545 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e3c003520 con 0x7f1e58153580 2024-04-04T06:15:03.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.425+0000 7f1e57400640 1 --2- 172.21.15.159:0/3416006545 >> v2:172.21.15.178:3300/0 conn(0x7f1e580d0990 0x7f1e580aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:03.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.425+0000 7f1e5ca00640 1 -- 172.21.15.159:0/3416006545 >> v2:172.21.15.178:3300/0 conn(0x7f1e580d0990 msgr2=0x7f1e580aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:03.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.425+0000 7f1e5ca00640 1 --2- 172.21.15.159:0/3416006545 >> v2:172.21.15.178:3300/0 conn(0x7f1e580d0990 0x7f1e580aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:03.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.425+0000 7f1e5ca00640 1 -- 172.21.15.159:0/3416006545 >> v1:172.21.15.159:6789/0 conn(0x7f1e58153580 legacy=0x7f1e580d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:03.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.425+0000 7f1e5ca00640 1 -- 172.21.15.159:0/3416006545 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5815cf20 con 0x7f1e580af2e0 2024-04-04T06:15:03.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.426+0000 7f1e5ca00640 1 --2- 172.21.15.159:0/3416006545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e580af2e0 0x7f1e581600d0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f1e5815a6b0 tx=0x7f1e5002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c5323c5db68888 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:03.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.427+0000 7f1e56a00640 1 -- 172.21.15.159:0/3416006545 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e500315a0 con 0x7f1e580af2e0 2024-04-04T06:15:03.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.427+0000 7f1e56a00640 1 -- 172.21.15.159:0/3416006545 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e50037070 con 0x7f1e580af2e0 2024-04-04T06:15:03.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.427+0000 7f1e56a00640 1 -- 172.21.15.159:0/3416006545 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50031960 con 0x7f1e580af2e0 2024-04-04T06:15:03.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.427+0000 7f1e5ea00640 1 -- 172.21.15.159:0/3416006545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e580af2e0 msgr2=0x7f1e581600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:03.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.427+0000 7f1e5ea00640 1 --2- 172.21.15.159:0/3416006545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e580af2e0 0x7f1e581600d0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f1e5815a6b0 tx=0x7f1e5002b840 comp rx=0 tx=0).stop 2024-04-04T06:15:03.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.427+0000 7f1e5ea00640 1 -- 172.21.15.159:0/3416006545 shutdown_connections 2024-04-04T06:15:03.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.427+0000 7f1e5ea00640 1 --2- 172.21.15.159:0/3416006545 >> v2:172.21.15.178:3300/0 conn(0x7f1e580d0990 0x7f1e580aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:03.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.427+0000 7f1e5ea00640 1 --2- 172.21.15.159:0/3416006545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e580af2e0 0x7f1e581600d0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:03.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.427+0000 7f1e5ea00640 1 -- 172.21.15.159:0/3416006545 >> 172.21.15.159:0/3416006545 conn(0x7f1e5806b750 msgr2=0x7f1e5806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:03.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.428+0000 7f1e5ea00640 1 -- 172.21.15.159:0/3416006545 shutdown_connections 2024-04-04T06:15:03.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.428+0000 7f1e5ea00640 1 -- 172.21.15.159:0/3416006545 wait complete. 2024-04-04T06:15:03.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.429+0000 7f1e5ea00640 1 Processor -- start 2024-04-04T06:15:03.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.429+0000 7f1e5ea00640 1 -- start start 2024-04-04T06:15:03.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.430+0000 7f1e5ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e580af2e0 0x7f1e58099820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:03.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.430+0000 7f1e5ea00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e580d0990 0x7f1e5809ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:03.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.430+0000 7f1e5ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e58153580 0x7f1e580cf220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:03.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.430+0000 7f1e5ea00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1e58153960 con 0x7f1e58153580 2024-04-04T06:15:03.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.430+0000 7f1e5ea00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5815d4e0 con 0x7f1e580af2e0 2024-04-04T06:15:03.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.430+0000 7f1e5ea00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1e580af6c0 con 0x7f1e580d0990 2024-04-04T06:15:03.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.431+0000 7f1e5ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e58153580 0x7f1e580cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:03.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.431+0000 7f1e5ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e58153580 0x7f1e580cf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37712/0 (socket says 172.21.15.159:37712) 2024-04-04T06:15:03.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.431+0000 7f1e5ca00640 1 -- 172.21.15.159:0/1596458089 learned_addr learned my addr 172.21.15.159:0/1596458089 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:03.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.431+0000 7f1e5ca00640 1 -- 172.21.15.159:0/1596458089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e580d0990 msgr2=0x7f1e5809ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:03.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.431+0000 7f1e5ca00640 1 --2- 172.21.15.159:0/1596458089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e580d0990 0x7f1e5809ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:03.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.431+0000 7f1e5ca00640 1 -- 172.21.15.159:0/1596458089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e580af2e0 msgr2=0x7f1e58099820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:03.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.431+0000 7f1e57e00640 1 --2- 172.21.15.159:0/1596458089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e580af2e0 0x7f1e58099820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:03.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.431+0000 7f1e5ca00640 1 --2- 172.21.15.159:0/1596458089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e580af2e0 0x7f1e58099820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:03.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.431+0000 7f1e5ca00640 1 -- 172.21.15.159:0/1596458089 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5809d460 con 0x7f1e58153580 2024-04-04T06:15:03.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.431+0000 7f1e5ca00640 1 --2- 172.21.15.159:0/1596458089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e58153580 0x7f1e580cf220 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f1e50034040 tx=0x7f1e500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:03.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.432+0000 7f1e4fe00640 1 -- 172.21.15.159:0/1596458089 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50032630 con 0x7f1e58153580 2024-04-04T06:15:03.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.432+0000 7f1e4fe00640 1 -- 172.21.15.159:0/1596458089 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e500327f0 con 0x7f1e58153580 2024-04-04T06:15:03.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.432+0000 7f1e4fe00640 1 -- 172.21.15.159:0/1596458089 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50032be0 con 0x7f1e58153580 2024-04-04T06:15:03.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.432+0000 7f1e5ea00640 1 -- 172.21.15.159:0/1596458089 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5809d740 con 0x7f1e58153580 2024-04-04T06:15:03.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.432+0000 7f1e5ea00640 1 -- 172.21.15.159:0/1596458089 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e5809dd00 con 0x7f1e58153580 2024-04-04T06:15:03.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.433+0000 7f1e5ea00640 1 -- 172.21.15.159:0/1596458089 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e580685c0 con 0x7f1e58153580 2024-04-04T06:15:03.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.434+0000 7f1e4fe00640 1 -- 172.21.15.159:0/1596458089 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e5004c020 con 0x7f1e58153580 2024-04-04T06:15:03.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.435+0000 7f1e4fe00640 1 --2- 172.21.15.159:0/1596458089 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1e3406c8f0 0x7f1e3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:03.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.435+0000 7f1e4fe00640 1 -- 172.21.15.159:0/1596458089 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4857+0+0 (secure 0 0 0) 0x7f1e5002f020 con 0x7f1e58153580 2024-04-04T06:15:03.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.436+0000 7f1e57e00640 1 --2- 172.21.15.159:0/1596458089 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1e3406c8f0 0x7f1e3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:03.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.437+0000 7f1e57e00640 1 --2- 172.21.15.159:0/1596458089 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1e3406c8f0 0x7f1e3406edb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1e480012e0 tx=0x7f1e48009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:03.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.440+0000 7f1e4fe00640 1 -- 172.21.15.159:0/1596458089 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1e50040070 con 0x7f1e58153580 2024-04-04T06:15:03.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.713+0000 7f1e5ea00640 1 -- 172.21.15.159:0/1596458089 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1e580aeda0 con 0x7f1e58153580 2024-04-04T06:15:03.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.714+0000 7f1e4fe00640 1 -- 172.21.15.159:0/1596458089 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7f1e5007bb40 con 0x7f1e58153580 2024-04-04T06:15:03.708 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:15:03.708 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:15:03.708 INFO:teuthology.orchestra.run.smithi159.stdout: "auth_dump": [ 2024-04-04T06:15:03.708 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:15:03.708 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.a", 2024-04-04T06:15:03.708 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5mvoZ2FxAAsVEQrkiICIH6T7ZzaqcWbA==", 2024-04-04T06:15:03.708 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:15:03.708 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:15:03.708 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:15:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:15:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:15:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:15:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:15:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.b", 2024-04-04T06:15:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5m+rccORAA6I/lQ+22QfWYL9+aoOFJ0w==", 2024-04-04T06:15:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:15:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:15:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:15:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:15:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:15:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:15:03.709 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:15:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.c", 2024-04-04T06:15:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5mjmQcKBAA8BjpQce6G7EycXOkyVECpQ==", 2024-04-04T06:15:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:15:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:15:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:15:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:15:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:15:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:15:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:15:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.d", 2024-04-04T06:15:03.710 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5m04hoDRAAKdhRI7nosZp7d4y7dVvzvg==", 2024-04-04T06:15:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:15:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:15:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:15:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:15:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:15:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:15:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:15:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.0", 2024-04-04T06:15:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQABQw5mCe7fNRAA6ZKxWvnMLf4jhAlpjudFRA==", 2024-04-04T06:15:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:15:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:15:03.711 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:15:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:15:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:15:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:15:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.1", 2024-04-04T06:15:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAFQw5mvIqTExAAU8Pm5alxFtHEL4qM9MavPw==", 2024-04-04T06:15:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:15:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:15:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:15:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:15:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:15:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:15:03.712 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.2", 2024-04-04T06:15:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAIQw5mAHblHRAAqEDe5dIGTDpdy2OsQyzJXA==", 2024-04-04T06:15:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:15:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:15:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:15:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:15:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:15:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:15:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.3", 2024-04-04T06:15:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQALQw5mwK7rJhAADQBAqE0Eej9rn91Wxhq/KA==", 2024-04-04T06:15:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:15:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:15:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:15:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:15:03.713 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:15:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:15:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.4", 2024-04-04T06:15:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAQQw5m/6I8GRAA2P7jI+AXYld2J11nDfhAUQ==", 2024-04-04T06:15:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:15:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:15:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:15:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:15:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:15:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:15:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.5", 2024-04-04T06:15:03.714 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQATQw5mv5HoMBAAn2mp2ys5KjS0UygOcX4vvg==", 2024-04-04T06:15:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:15:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:15:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:15:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:15:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:15:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:15:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.6", 2024-04-04T06:15:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAWQw5mx12gORAAT2LhrfEB5BJSl0ypgospzg==", 2024-04-04T06:15:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:15:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:15:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:15:03.715 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:15:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:15:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:15:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.7", 2024-04-04T06:15:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAaQw5m6HEOBxAAVegLH3bjVM7r1A6tK5eqHQ==", 2024-04-04T06:15:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:15:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:15:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:15:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:15:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:15:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:15:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.0", 2024-04-04T06:15:03.716 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mqNKJGxAA4jP9cadhRIyhQ3lssttWIw==", 2024-04-04T06:15:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:15:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:15:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow r", 2024-04-04T06:15:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-04T06:15:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:15:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:15:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.1", 2024-04-04T06:15:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mRkdOMBAA1ae1roIwrlB49DFQUoFSTg==", 2024-04-04T06:15:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:15:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:15:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:15:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:15:03.717 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:15:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:15:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:15:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.2", 2024-04-04T06:15:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mtW2/JRAAucxSSN9q0EQtI2a/P2yozQ==", 2024-04-04T06:15:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:15:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:15:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:15:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:15:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:15:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:15:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:15:03.718 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.3", 2024-04-04T06:15:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mgqD0OhAAov+3aDGyGEoqxep8+SBpSA==", 2024-04-04T06:15:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:15:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:15:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:15:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:15:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:15:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:15:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:15:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.admin", 2024-04-04T06:15:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD7Qg5m0O/iCBAAsWTzlEA0P60XYuxmJiAb4A==", 2024-04-04T06:15:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:15:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow *", 2024-04-04T06:15:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:15:03.719 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:15:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:15:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:15:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:15:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mgr.x", 2024-04-04T06:15:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5mhqbBCxAASBknMIj4uu7OiB3jn7WP8Q==", 2024-04-04T06:15:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:15:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow *", 2024-04-04T06:15:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile mgr", 2024-04-04T06:15:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:15:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:15:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:15:03.720 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mgr.y", 2024-04-04T06:15:03.721 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD7Qg5miXT0ORAAL6y306CPAfEGQKAJ0no4uw==", 2024-04-04T06:15:03.721 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:15:03.721 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow *", 2024-04-04T06:15:03.721 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile mgr", 2024-04-04T06:15:03.721 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:15:03.721 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:03.721 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:03.721 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-04-04T06:15:03.721 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:15:03.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.716+0000 7f1e5ea00640 1 -- 172.21.15.159:0/1596458089 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1e3406c8f0 msgr2=0x7f1e3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:03.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.716+0000 7f1e5ea00640 1 --2- 172.21.15.159:0/1596458089 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1e3406c8f0 0x7f1e3406edb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1e480012e0 tx=0x7f1e48009040 comp rx=0 tx=0).stop 2024-04-04T06:15:03.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.716+0000 7f1e5ea00640 1 -- 172.21.15.159:0/1596458089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e58153580 msgr2=0x7f1e580cf220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:03.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.716+0000 7f1e5ea00640 1 --2- 172.21.15.159:0/1596458089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e58153580 0x7f1e580cf220 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f1e50034040 tx=0x7f1e500046c0 comp rx=0 tx=0).stop 2024-04-04T06:15:03.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.716+0000 7f1e5ea00640 1 -- 172.21.15.159:0/1596458089 shutdown_connections 2024-04-04T06:15:03.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.716+0000 7f1e5ea00640 1 --2- 172.21.15.159:0/1596458089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e58153580 0x7f1e580cf220 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:03.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.716+0000 7f1e5ea00640 1 --2- 172.21.15.159:0/1596458089 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e580d0990 0x7f1e5809ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:03.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.716+0000 7f1e5ea00640 1 --2- 172.21.15.159:0/1596458089 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1e3406c8f0 0x7f1e3406edb0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:03.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.716+0000 7f1e5ea00640 1 --2- 172.21.15.159:0/1596458089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e580af2e0 0x7f1e58099820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:03.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.716+0000 7f1e5ea00640 1 -- 172.21.15.159:0/1596458089 >> 172.21.15.159:0/1596458089 conn(0x7f1e5806b750 msgr2=0x7f1e5815e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:03.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.716+0000 7f1e5ea00640 1 -- 172.21.15.159:0/1596458089 shutdown_connections 2024-04-04T06:15:03.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.716+0000 7f1e5ea00640 1 -- 172.21.15.159:0/1596458089 wait complete. 2024-04-04T06:15:03.731 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:15:03.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.948+0000 7f00b6c00640 1 Processor -- start 2024-04-04T06:15:03.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.948+0000 7f00b6c00640 1 -- start start 2024-04-04T06:15:03.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.950+0000 7f00b6c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f00b00727f0 0x7f00b00b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:03.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.950+0000 7f00b6c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00b015b6f0 con 0x7f00b015b310 2024-04-04T06:15:03.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.950+0000 7f00b6c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00b0072e80 con 0x7f00b00d0780 2024-04-04T06:15:03.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.950+0000 7f00b6c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f00b0072c50 con 0x7f00b00727f0 2024-04-04T06:15:03.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.950+0000 7f00afe00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f00b015b310 0x7f00b015f7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:42920/0 (socket says 172.21.15.159:42920) 2024-04-04T06:15:03.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.950+0000 7f00afe00640 1 -- 172.21.15.159:0/3891406878 learned_addr learned my addr 172.21.15.159:0/3891406878 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:03.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.951+0000 7f00aea00640 1 -- 172.21.15.159:0/3891406878 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1358863877 0 0) 0x7f00b015b6f0 con 0x7f00b015b310 2024-04-04T06:15:03.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.950+0000 7f00af400640 1 --2- 172.21.15.159:0/3891406878 >> v2:172.21.15.178:3300/0 conn(0x7f00b00727f0 0x7f00b00b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:03.946 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.951+0000 7f00aea00640 1 -- 172.21.15.159:0/3891406878 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0094003520 con 0x7f00b015b310 2024-04-04T06:15:03.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.951+0000 7f00aea00640 1 -- 172.21.15.159:0/3891406878 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1129710492 0 0) 0x7f0094003520 con 0x7f00b015b310 2024-04-04T06:15:03.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.951+0000 7f00aea00640 1 -- 172.21.15.159:0/3891406878 >> v2:172.21.15.178:3300/0 conn(0x7f00b00727f0 msgr2=0x7f00b00b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:03.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.951+0000 7f00aea00640 1 --2- 172.21.15.159:0/3891406878 >> v2:172.21.15.178:3300/0 conn(0x7f00b00727f0 0x7f00b00b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:03.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.951+0000 7f00aea00640 1 -- 172.21.15.159:0/3891406878 >> v1:172.21.15.178:6789/0 conn(0x7f00b00d0780 legacy=0x7f00b00adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:03.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.951+0000 7f00aea00640 1 -- 172.21.15.159:0/3891406878 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b0073160 con 0x7f00b015b310 2024-04-04T06:15:03.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.951+0000 7f00aea00640 1 -- 172.21.15.159:0/3891406878 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f00a0001670 con 0x7f00b015b310 2024-04-04T06:15:03.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.951+0000 7f00aea00640 1 -- 172.21.15.159:0/3891406878 >> v1:172.21.15.159:6789/0 conn(0x7f00b015b310 legacy=0x7f00b015f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:03.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.951+0000 7f00aea00640 1 --2- 172.21.15.159:0/3891406878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00940039b0 0x7f0094003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:03.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.951+0000 7f00aea00640 1 --2- 172.21.15.159:0/3891406878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0094004800 0x7f0094004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:03.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.951+0000 7f00aea00640 1 --2- 172.21.15.159:0/3891406878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00940053c0 0x7f009400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:03.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.951+0000 7f00aea00640 1 -- 172.21.15.159:0/3891406878 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f009400e430 con 0x7f0094004800 2024-04-04T06:15:03.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.951+0000 7f00aea00640 1 -- 172.21.15.159:0/3891406878 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f009400e5e0 con 0x7f00940053c0 2024-04-04T06:15:03.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.951+0000 7f00aea00640 1 -- 172.21.15.159:0/3891406878 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f009400e7e0 con 0x7f00940039b0 2024-04-04T06:15:03.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.951+0000 7f00af400640 1 --2- 172.21.15.159:0/3891406878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0094004800 0x7f0094004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:03.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.952+0000 7f00af400640 1 -- 172.21.15.159:0/3891406878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00940039b0 msgr2=0x7f0094003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:03.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.952+0000 7f00afe00640 1 --2- 172.21.15.159:0/3891406878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00940039b0 0x7f0094003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:03.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.952+0000 7f00b4c00640 1 --2- 172.21.15.159:0/3891406878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00940053c0 0x7f009400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:03.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.952+0000 7f00af400640 1 --2- 172.21.15.159:0/3891406878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00940039b0 0x7f0094003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:03.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.952+0000 7f00af400640 1 -- 172.21.15.159:0/3891406878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00940053c0 msgr2=0x7f009400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:03.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.952+0000 7f00af400640 1 --2- 172.21.15.159:0/3891406878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00940053c0 0x7f009400dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:03.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.952+0000 7f00b4c00640 1 -- 172.21.15.159:0/3891406878 reap_dead start 2024-04-04T06:15:03.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.952+0000 7f00af400640 1 -- 172.21.15.159:0/3891406878 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0094003520 con 0x7f0094004800 2024-04-04T06:15:03.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.953+0000 7f00af400640 1 --2- 172.21.15.159:0/3891406878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0094004800 0x7f0094004c80 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f009c00ba30 tx=0x7f0094005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9dbd4fc978b405 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:03.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.953+0000 7f00aea00640 1 -- 172.21.15.159:0/3891406878 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c036020 con 0x7f0094004800 2024-04-04T06:15:03.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.953+0000 7f00aea00640 1 -- 172.21.15.159:0/3891406878 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f009c032640 con 0x7f0094004800 2024-04-04T06:15:03.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.953+0000 7f00aea00640 1 -- 172.21.15.159:0/3891406878 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c00b600 con 0x7f0094004800 2024-04-04T06:15:03.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.954+0000 7f00b6c00640 1 -- 172.21.15.159:0/3891406878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0094004800 msgr2=0x7f0094004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:03.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.954+0000 7f00b6c00640 1 --2- 172.21.15.159:0/3891406878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0094004800 0x7f0094004c80 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f009c00ba30 tx=0x7f0094005830 comp rx=0 tx=0).stop 2024-04-04T06:15:03.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.954+0000 7f00b6c00640 1 -- 172.21.15.159:0/3891406878 shutdown_connections 2024-04-04T06:15:03.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.954+0000 7f00b6c00640 1 --2- 172.21.15.159:0/3891406878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0094004800 0x7f0094004c80 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:03.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.954+0000 7f00b6c00640 1 -- 172.21.15.159:0/3891406878 >> 172.21.15.159:0/3891406878 conn(0x7f00b006b990 msgr2=0x7f00b006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:03.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.954+0000 7f00b6c00640 1 -- 172.21.15.159:0/3891406878 shutdown_connections 2024-04-04T06:15:03.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.954+0000 7f00b6c00640 1 -- 172.21.15.159:0/3891406878 wait complete. 2024-04-04T06:15:03.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.956+0000 7f00b6c00640 1 Processor -- start 2024-04-04T06:15:03.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.956+0000 7f00b6c00640 1 -- start start 2024-04-04T06:15:03.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.957+0000 7f00b6c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0094004800 0x7f00b0147480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:03.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.957+0000 7f00b6c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00b0159db0 0x7f00b015a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:03.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.957+0000 7f00b6c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00b01503c0 0x7f00b0150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:03.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.957+0000 7f00b6c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f00b0073440 con 0x7f00b0159db0 2024-04-04T06:15:03.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.957+0000 7f00b6c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f00b015f2c0 con 0x7f0094004800 2024-04-04T06:15:03.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.957+0000 7f00b6c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f00b00d0c10 con 0x7f00b01503c0 2024-04-04T06:15:03.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.957+0000 7f00af400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00b0159db0 0x7f00b015a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:03.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.957+0000 7f00af400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00b0159db0 0x7f00b015a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35456/0 (socket says 172.21.15.159:35456) 2024-04-04T06:15:03.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.957+0000 7f00af400640 1 -- 172.21.15.159:0/1141387172 learned_addr learned my addr 172.21.15.159:0/1141387172 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:03.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.957+0000 7f00af400640 1 -- 172.21.15.159:0/1141387172 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00b01503c0 msgr2=0x7f00b0150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:03.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.957+0000 7f00af400640 1 --2- 172.21.15.159:0/1141387172 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00b01503c0 0x7f00b0150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:03.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.957+0000 7f00af400640 1 -- 172.21.15.159:0/1141387172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0094004800 msgr2=0x7f00b0147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:03.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.957+0000 7f00af400640 1 --2- 172.21.15.159:0/1141387172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0094004800 0x7f00b0147480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:03.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.957+0000 7f00af400640 1 -- 172.21.15.159:0/1141387172 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b0150d80 con 0x7f00b0159db0 2024-04-04T06:15:03.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.958+0000 7f00af400640 1 --2- 172.21.15.159:0/1141387172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00b0159db0 0x7f00b015a230 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f009c00bb50 tx=0x7f009c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:03.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.958+0000 7f00a7e00640 1 -- 172.21.15.159:0/1141387172 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c041070 con 0x7f00b0159db0 2024-04-04T06:15:03.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.958+0000 7f00a7e00640 1 -- 172.21.15.159:0/1141387172 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f009c030a50 con 0x7f00b0159db0 2024-04-04T06:15:03.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.958+0000 7f00a7e00640 1 -- 172.21.15.159:0/1141387172 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c036040 con 0x7f00b0159db0 2024-04-04T06:15:03.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.958+0000 7f00b6c00640 1 -- 172.21.15.159:0/1141387172 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b0151040 con 0x7f00b0159db0 2024-04-04T06:15:03.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.958+0000 7f00b6c00640 1 -- 172.21.15.159:0/1141387172 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b0151600 con 0x7f00b0159db0 2024-04-04T06:15:03.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.959+0000 7f00b6c00640 1 -- 172.21.15.159:0/1141387172 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b00685c0 con 0x7f00b0159db0 2024-04-04T06:15:03.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.960+0000 7f00a7e00640 1 -- 172.21.15.159:0/1141387172 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f009c04c060 con 0x7f00b0159db0 2024-04-04T06:15:03.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.962+0000 7f00a7e00640 1 --2- 172.21.15.159:0/1141387172 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f009006c8f0 0x7f009006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:03.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.962+0000 7f00a7e00640 1 -- 172.21.15.159:0/1141387172 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4857+0+0 (secure 0 0 0) 0x7f009c0ad650 con 0x7f00b0159db0 2024-04-04T06:15:03.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.962+0000 7f00afe00640 1 --2- 172.21.15.159:0/1141387172 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f009006c8f0 0x7f009006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:03.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.963+0000 7f00afe00640 1 --2- 172.21.15.159:0/1141387172 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f009006c8f0 0x7f009006edb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f00a00017e0 tx=0x7f00a0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:03.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:03.966+0000 7f00a7e00640 1 -- 172.21.15.159:0/1141387172 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f009c07bd30 con 0x7f00b0159db0 2024-04-04T06:15:04.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.165+0000 7f00b6c00640 1 -- 172.21.15.159:0/1141387172 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00b015ec20 con 0x7f00b0159db0 2024-04-04T06:15:04.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.167+0000 7f00a7e00640 1 -- 172.21.15.159:0/1141387172 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+12904 (secure 0 0 0) 0x7f009c07b630 con 0x7f00b0159db0 2024-04-04T06:15:04.161 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:15:04.161 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":97,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:15:03.184897+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:15:04.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.169+0000 7f00b6c00640 1 -- 172.21.15.159:0/1141387172 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f009006c8f0 msgr2=0x7f009006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:04.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.169+0000 7f00b6c00640 1 --2- 172.21.15.159:0/1141387172 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f009006c8f0 0x7f009006edb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f00a00017e0 tx=0x7f00a0009040 comp rx=0 tx=0).stop 2024-04-04T06:15:04.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.169+0000 7f00b6c00640 1 -- 172.21.15.159:0/1141387172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00b0159db0 msgr2=0x7f00b015a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:04.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.169+0000 7f00b6c00640 1 --2- 172.21.15.159:0/1141387172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00b0159db0 0x7f00b015a230 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f009c00bb50 tx=0x7f009c004730 comp rx=0 tx=0).stop 2024-04-04T06:15:04.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.169+0000 7f00b6c00640 1 -- 172.21.15.159:0/1141387172 shutdown_connections 2024-04-04T06:15:04.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.169+0000 7f00b6c00640 1 --2- 172.21.15.159:0/1141387172 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f00b01503c0 0x7f00b0150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:04.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.169+0000 7f00b6c00640 1 --2- 172.21.15.159:0/1141387172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00b0159db0 0x7f00b015a230 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:04.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.169+0000 7f00b6c00640 1 --2- 172.21.15.159:0/1141387172 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f009006c8f0 0x7f009006edb0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:04.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.169+0000 7f00b6c00640 1 --2- 172.21.15.159:0/1141387172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0094004800 0x7f00b0147480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:04.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.170+0000 7f00b6c00640 1 -- 172.21.15.159:0/1141387172 >> 172.21.15.159:0/1141387172 conn(0x7f00b006b990 msgr2=0x7f00b00aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:04.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.170+0000 7f00b6c00640 1 -- 172.21.15.159:0/1141387172 shutdown_connections 2024-04-04T06:15:04.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.170+0000 7f00b6c00640 1 -- 172.21.15.159:0/1141387172 wait complete. 2024-04-04T06:15:04.182 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:15:04.182 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:15:04.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.397+0000 7f6e7d600640 1 Processor -- start 2024-04-04T06:15:04.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.398+0000 7f6e7d600640 1 -- start start 2024-04-04T06:15:04.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.399+0000 7f6e7d600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f6e7815d490 0x7f6e780cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:04.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.399+0000 7f6e7d600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e780716d0 0x7f6e780d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:04.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.399+0000 7f6e7d600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e78071c80 con 0x7f6e780b4a30 2024-04-04T06:15:04.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.399+0000 7f6e7d600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f6e78071f60 con 0x7f6e7815d490 2024-04-04T06:15:04.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.399+0000 7f6e7d600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6e780b3f50 con 0x7f6e780716d0 2024-04-04T06:15:04.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.400+0000 7f6e76000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f6e780b4a30 0x7f6e78070fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:42928/0 (socket says 172.21.15.159:42928) 2024-04-04T06:15:04.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.400+0000 7f6e76000640 1 -- 172.21.15.159:0/127172728 learned_addr learned my addr 172.21.15.159:0/127172728 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:04.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.400+0000 7f6e76a00640 1 --2- 172.21.15.159:0/127172728 >> v2:172.21.15.159:3300/0 conn(0x7f6e7815d490 0x7f6e780cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:04.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.400+0000 7f6e75600640 1 -- 172.21.15.159:0/127172728 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3445862460 0 0) 0x7f6e78071c80 con 0x7f6e780b4a30 2024-04-04T06:15:04.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.400+0000 7f6e75600640 1 -- 172.21.15.159:0/127172728 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e58003520 con 0x7f6e780b4a30 2024-04-04T06:15:04.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.400+0000 7f6e77400640 1 --2- 172.21.15.159:0/127172728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e780716d0 0x7f6e780d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:04.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.400+0000 7f6e76a00640 1 -- 172.21.15.159:0/127172728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e780716d0 msgr2=0x7f6e780d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:04.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.400+0000 7f6e76a00640 1 --2- 172.21.15.159:0/127172728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e780716d0 0x7f6e780d02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:04.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.400+0000 7f6e76a00640 1 -- 172.21.15.159:0/127172728 >> v1:172.21.15.159:6789/0 conn(0x7f6e780b4a30 legacy=0x7f6e78070fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:04.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.400+0000 7f6e76a00640 1 -- 172.21.15.159:0/127172728 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e780b4230 con 0x7f6e7815d490 2024-04-04T06:15:04.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.400+0000 7f6e77400640 1 --2- 172.21.15.159:0/127172728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e780716d0 0x7f6e780d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:04.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.401+0000 7f6e76a00640 1 --2- 172.21.15.159:0/127172728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e7815d490 0x7f6e780cfd80 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f6e680097a0 tx=0x7f6e6802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfe75ee81e8db83 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:04.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.402+0000 7f6e75600640 1 -- 172.21.15.159:0/127172728 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e68031780 con 0x7f6e7815d490 2024-04-04T06:15:04.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.402+0000 7f6e75600640 1 -- 172.21.15.159:0/127172728 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e68037070 con 0x7f6e7815d490 2024-04-04T06:15:04.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.402+0000 7f6e75600640 1 -- 172.21.15.159:0/127172728 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e68031b40 con 0x7f6e7815d490 2024-04-04T06:15:04.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.402+0000 7f6e7d600640 1 -- 172.21.15.159:0/127172728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e7815d490 msgr2=0x7f6e780cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:04.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.402+0000 7f6e7d600640 1 --2- 172.21.15.159:0/127172728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e7815d490 0x7f6e780cfd80 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f6e680097a0 tx=0x7f6e6802b820 comp rx=0 tx=0).stop 2024-04-04T06:15:04.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.403+0000 7f6e7d600640 1 -- 172.21.15.159:0/127172728 shutdown_connections 2024-04-04T06:15:04.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.403+0000 7f6e7d600640 1 --2- 172.21.15.159:0/127172728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e780716d0 0x7f6e780d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:04.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.403+0000 7f6e7d600640 1 --2- 172.21.15.159:0/127172728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e7815d490 0x7f6e780cfd80 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:04.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.403+0000 7f6e7d600640 1 -- 172.21.15.159:0/127172728 >> 172.21.15.159:0/127172728 conn(0x7f6e780d2240 msgr2=0x7f6e780d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:04.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.403+0000 7f6e7d600640 1 -- 172.21.15.159:0/127172728 shutdown_connections 2024-04-04T06:15:04.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.403+0000 7f6e7d600640 1 -- 172.21.15.159:0/127172728 wait complete. 2024-04-04T06:15:04.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.404+0000 7f6e7d600640 1 Processor -- start 2024-04-04T06:15:04.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.405+0000 7f6e7d600640 1 -- start start 2024-04-04T06:15:04.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.405+0000 7f6e7d600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e780716d0 0x7f6e7806e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:04.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.406+0000 7f6e7d600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e780b4a30 0x7f6e78142be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:04.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.406+0000 7f6e7d600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e7815d490 0x7f6e7815b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:04.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.406+0000 7f6e7d600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7815d9f0 con 0x7f6e780716d0 2024-04-04T06:15:04.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.406+0000 7f6e7d600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7815d870 con 0x7f6e780b4a30 2024-04-04T06:15:04.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.406+0000 7f6e7d600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6e780c74a0 con 0x7f6e7815d490 2024-04-04T06:15:04.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.406+0000 7f6e76a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e780716d0 0x7f6e7806e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:04.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.406+0000 7f6e76a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e780716d0 0x7f6e7806e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35474/0 (socket says 172.21.15.159:35474) 2024-04-04T06:15:04.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.406+0000 7f6e76a00640 1 -- 172.21.15.159:0/3443813612 learned_addr learned my addr 172.21.15.159:0/3443813612 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:04.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.406+0000 7f6e76a00640 1 -- 172.21.15.159:0/3443813612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e7815d490 msgr2=0x7f6e7815b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:04.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.406+0000 7f6e76a00640 1 --2- 172.21.15.159:0/3443813612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e7815d490 0x7f6e7815b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:04.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.406+0000 7f6e76a00640 1 -- 172.21.15.159:0/3443813612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e780b4a30 msgr2=0x7f6e78142be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:04.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.406+0000 7f6e76a00640 1 --2- 172.21.15.159:0/3443813612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e780b4a30 0x7f6e78142be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:04.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.406+0000 7f6e76a00640 1 -- 172.21.15.159:0/3443813612 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7814c570 con 0x7f6e780716d0 2024-04-04T06:15:04.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.407+0000 7f6e76a00640 1 --2- 172.21.15.159:0/3443813612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e780716d0 0x7f6e7806e0d0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f6e680098c0 tx=0x7f6e680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:04.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.407+0000 7f6e6ea00640 1 -- 172.21.15.159:0/3443813612 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e68032610 con 0x7f6e780716d0 2024-04-04T06:15:04.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.407+0000 7f6e6ea00640 1 -- 172.21.15.159:0/3443813612 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e680327b0 con 0x7f6e780716d0 2024-04-04T06:15:04.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.407+0000 7f6e6ea00640 1 -- 172.21.15.159:0/3443813612 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e68032ba0 con 0x7f6e780716d0 2024-04-04T06:15:04.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.407+0000 7f6e7d600640 1 -- 172.21.15.159:0/3443813612 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e7814c850 con 0x7f6e780716d0 2024-04-04T06:15:04.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.407+0000 7f6e7d600640 1 -- 172.21.15.159:0/3443813612 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e7814ce10 con 0x7f6e780716d0 2024-04-04T06:15:04.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.408+0000 7f6e7d600640 1 -- 172.21.15.159:0/3443813612 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e780685c0 con 0x7f6e780716d0 2024-04-04T06:15:04.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.409+0000 7f6e6ea00640 1 -- 172.21.15.159:0/3443813612 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e6804e440 con 0x7f6e780716d0 2024-04-04T06:15:04.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.410+0000 7f6e6ea00640 1 --2- 172.21.15.159:0/3443813612 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6e5406c8f0 0x7f6e5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:04.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.410+0000 7f6e6ea00640 1 -- 172.21.15.159:0/3443813612 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4857+0+0 (secure 0 0 0) 0x7f6e680ad680 con 0x7f6e780716d0 2024-04-04T06:15:04.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.411+0000 7f6e76000640 1 --2- 172.21.15.159:0/3443813612 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6e5406c8f0 0x7f6e5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:04.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.412+0000 7f6e76000640 1 --2- 172.21.15.159:0/3443813612 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6e5406c8f0 0x7f6e5406edb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6e600012e0 tx=0x7f6e60001670 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:04.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.414+0000 7f6e6ea00640 1 -- 172.21.15.159:0/3443813612 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6e68040070 con 0x7f6e780716d0 2024-04-04T06:15:04.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.612+0000 7f6e7d600640 1 -- 172.21.15.159:0/3443813612 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6e780b5140 con 0x7f6e780716d0 2024-04-04T06:15:04.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.613+0000 7f6e6ea00640 1 -- 172.21.15.159:0/3443813612 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f6e6802f050 con 0x7f6e780716d0 2024-04-04T06:15:04.607 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:15:04.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.615+0000 7f6e7d600640 1 -- 172.21.15.159:0/3443813612 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6e5406c8f0 msgr2=0x7f6e5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:04.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.615+0000 7f6e7d600640 1 --2- 172.21.15.159:0/3443813612 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6e5406c8f0 0x7f6e5406edb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6e600012e0 tx=0x7f6e60001670 comp rx=0 tx=0).stop 2024-04-04T06:15:04.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.615+0000 7f6e7d600640 1 -- 172.21.15.159:0/3443813612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e780716d0 msgr2=0x7f6e7806e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:04.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.615+0000 7f6e7d600640 1 --2- 172.21.15.159:0/3443813612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e780716d0 0x7f6e7806e0d0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f6e680098c0 tx=0x7f6e680046c0 comp rx=0 tx=0).stop 2024-04-04T06:15:04.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.615+0000 7f6e7d600640 1 -- 172.21.15.159:0/3443813612 shutdown_connections 2024-04-04T06:15:04.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.615+0000 7f6e7d600640 1 --2- 172.21.15.159:0/3443813612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e7815d490 0x7f6e7815b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:04.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.615+0000 7f6e7d600640 1 --2- 172.21.15.159:0/3443813612 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6e5406c8f0 0x7f6e5406edb0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:04.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.615+0000 7f6e7d600640 1 --2- 172.21.15.159:0/3443813612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e780b4a30 0x7f6e78142be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:04.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.615+0000 7f6e7d600640 1 --2- 172.21.15.159:0/3443813612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e780716d0 0x7f6e7806e0d0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:04.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.616+0000 7f6e7d600640 1 -- 172.21.15.159:0/3443813612 >> 172.21.15.159:0/3443813612 conn(0x7f6e780d2240 msgr2=0x7f6e780b68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:04.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.616+0000 7f6e7d600640 1 -- 172.21.15.159:0/3443813612 shutdown_connections 2024-04-04T06:15:04.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.616+0000 7f6e7d600640 1 -- 172.21.15.159:0/3443813612 wait complete. 2024-04-04T06:15:04.630 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-04T06:15:04.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.846+0000 7fae8fe00640 1 Processor -- start 2024-04-04T06:15:04.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.846+0000 7fae8fe00640 1 -- start start 2024-04-04T06:15:04.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.848+0000 7fae8fe00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fae9006e030 0x7fae9015fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:04.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.848+0000 7fae8fe00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae90142880 con 0x7fae9013f300 2024-04-04T06:15:04.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.848+0000 7fae8fe00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae901422c0 con 0x7fae901602d0 2024-04-04T06:15:04.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.848+0000 7fae8fe00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fae90160720 con 0x7fae9006e030 2024-04-04T06:15:04.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.848+0000 7fae8ea00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fae9013f300 0x7fae901419e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:42938/0 (socket says 172.21.15.159:42938) 2024-04-04T06:15:04.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.848+0000 7fae8ea00640 1 -- 172.21.15.159:0/2229607152 learned_addr learned my addr 172.21.15.159:0/2229607152 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:04.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.848+0000 7fae8e000640 1 --2- 172.21.15.159:0/2229607152 >> v2:172.21.15.178:3300/0 conn(0x7fae9006e030 0x7fae9015fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:04.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.848+0000 7fae8d600640 1 -- 172.21.15.159:0/2229607152 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3913748421 0 0) 0x7fae90142880 con 0x7fae9013f300 2024-04-04T06:15:04.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.848+0000 7fae8d600640 1 -- 172.21.15.159:0/2229607152 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae74003520 con 0x7fae9013f300 2024-04-04T06:15:04.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.849+0000 7fae8d600640 1 -- 172.21.15.159:0/2229607152 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2698514066 0 0) 0x7fae74003520 con 0x7fae9013f300 2024-04-04T06:15:04.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.849+0000 7fae8d600640 1 -- 172.21.15.159:0/2229607152 >> v2:172.21.15.178:3300/0 conn(0x7fae9006e030 msgr2=0x7fae9015fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:04.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.849+0000 7fae8d600640 1 --2- 172.21.15.159:0/2229607152 >> v2:172.21.15.178:3300/0 conn(0x7fae9006e030 0x7fae9015fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:04.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.849+0000 7fae8d600640 1 -- 172.21.15.159:0/2229607152 >> v1:172.21.15.178:6789/0 conn(0x7fae901602d0 legacy=0x7fae900cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:04.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.849+0000 7fae8d600640 1 -- 172.21.15.159:0/2229607152 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae901425a0 con 0x7fae9013f300 2024-04-04T06:15:04.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.849+0000 7fae8d600640 1 -- 172.21.15.159:0/2229607152 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fae80001670 con 0x7fae9013f300 2024-04-04T06:15:04.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.849+0000 7fae8d600640 1 -- 172.21.15.159:0/2229607152 >> v1:172.21.15.159:6789/0 conn(0x7fae9013f300 legacy=0x7fae901419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:04.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.849+0000 7fae8d600640 1 --2- 172.21.15.159:0/2229607152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae740039b0 0x7fae74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:04.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.849+0000 7fae8d600640 1 --2- 172.21.15.159:0/2229607152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae74004800 0x7fae74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:04.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.849+0000 7fae8d600640 1 --2- 172.21.15.159:0/2229607152 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae740051c0 0x7fae7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:04.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.849+0000 7fae8d600640 1 -- 172.21.15.159:0/2229607152 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fae90160720 con 0x7fae740039b0 2024-04-04T06:15:04.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.849+0000 7fae8d600640 1 -- 172.21.15.159:0/2229607152 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fae7400c110 con 0x7fae74004800 2024-04-04T06:15:04.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.849+0000 7fae8d600640 1 -- 172.21.15.159:0/2229607152 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fae7400c310 con 0x7fae740051c0 2024-04-04T06:15:04.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.849+0000 7fae8ea00640 1 --2- 172.21.15.159:0/2229607152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae740039b0 0x7fae74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:04.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.850+0000 7fae8e000640 1 --2- 172.21.15.159:0/2229607152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae74004800 0x7fae74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:04.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.850+0000 7fae8ea00640 1 -- 172.21.15.159:0/2229607152 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae740051c0 msgr2=0x7fae7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:04.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.850+0000 7fae8ea00640 1 --2- 172.21.15.159:0/2229607152 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae740051c0 0x7fae7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:04.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.850+0000 7fae8ea00640 1 -- 172.21.15.159:0/2229607152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae74004800 msgr2=0x7fae74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:04.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.850+0000 7fae8ea00640 1 --2- 172.21.15.159:0/2229607152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae74004800 0x7fae74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:04.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.850+0000 7fae8ea00640 1 -- 172.21.15.159:0/2229607152 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae74003520 con 0x7fae740039b0 2024-04-04T06:15:04.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.850+0000 7fae8f400640 1 -- 172.21.15.159:0/2229607152 reap_dead start 2024-04-04T06:15:04.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.850+0000 7fae8e000640 1 --2- 172.21.15.159:0/2229607152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae74004800 0x7fae74004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:04.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.851+0000 7fae8ea00640 1 --2- 172.21.15.159:0/2229607152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae740039b0 0x7fae74003dd0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fae8000c240 tx=0x7fae74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb3a0574afa8788 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:04.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.851+0000 7fae8d600640 1 -- 172.21.15.159:0/2229607152 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80007cf0 con 0x7fae740039b0 2024-04-04T06:15:04.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.851+0000 7fae8d600640 1 -- 172.21.15.159:0/2229607152 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae80030070 con 0x7fae740039b0 2024-04-04T06:15:04.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.851+0000 7fae8d600640 1 -- 172.21.15.159:0/2229607152 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80005d70 con 0x7fae740039b0 2024-04-04T06:15:04.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.851+0000 7fae8fe00640 1 -- 172.21.15.159:0/2229607152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae740039b0 msgr2=0x7fae74003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:04.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.851+0000 7fae8fe00640 1 --2- 172.21.15.159:0/2229607152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae740039b0 0x7fae74003dd0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fae8000c240 tx=0x7fae74005830 comp rx=0 tx=0).stop 2024-04-04T06:15:04.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.852+0000 7fae8fe00640 1 -- 172.21.15.159:0/2229607152 shutdown_connections 2024-04-04T06:15:04.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.852+0000 7fae8fe00640 1 --2- 172.21.15.159:0/2229607152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae740039b0 0x7fae74003dd0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:04.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.852+0000 7fae8fe00640 1 -- 172.21.15.159:0/2229607152 >> 172.21.15.159:0/2229607152 conn(0x7fae9006b570 msgr2=0x7fae9006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:04.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.852+0000 7fae8fe00640 1 -- 172.21.15.159:0/2229607152 shutdown_connections 2024-04-04T06:15:04.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.852+0000 7fae8fe00640 1 -- 172.21.15.159:0/2229607152 wait complete. 2024-04-04T06:15:04.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.853+0000 7fae8fe00640 1 Processor -- start 2024-04-04T06:15:04.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.854+0000 7fae8fe00640 1 -- start start 2024-04-04T06:15:04.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.854+0000 7fae8fe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae740039b0 0x7fae9013d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:04.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.854+0000 7fae8fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae901f7250 0x7fae900b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:04.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.855+0000 7fae8fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae901f4440 0x7fae9015df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:04.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.855+0000 7fae8fe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fae9015fd90 con 0x7fae901f7250 2024-04-04T06:15:04.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.855+0000 7fae8fe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fae900cee90 con 0x7fae901f4440 2024-04-04T06:15:04.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.855+0000 7fae8fe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fae9015ff10 con 0x7fae740039b0 2024-04-04T06:15:04.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.855+0000 7fae8e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae901f7250 0x7fae900b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:04.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.855+0000 7fae8e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae901f7250 0x7fae900b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35484/0 (socket says 172.21.15.159:35484) 2024-04-04T06:15:04.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.855+0000 7fae8e000640 1 -- 172.21.15.159:0/4126773137 learned_addr learned my addr 172.21.15.159:0/4126773137 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:04.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.855+0000 7fae8ea00640 1 --2- 172.21.15.159:0/4126773137 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae740039b0 0x7fae9013d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:04.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.855+0000 7fae8e000640 1 -- 172.21.15.159:0/4126773137 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae740039b0 msgr2=0x7fae9013d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:04.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.855+0000 7fae8e000640 1 --2- 172.21.15.159:0/4126773137 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae740039b0 0x7fae9013d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:04.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.855+0000 7fae8f400640 1 --2- 172.21.15.159:0/4126773137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae901f4440 0x7fae9015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:04.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.855+0000 7fae8e000640 1 -- 172.21.15.159:0/4126773137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae901f4440 msgr2=0x7fae9015df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:04.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.855+0000 7fae8e000640 1 --2- 172.21.15.159:0/4126773137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae901f4440 0x7fae9015df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:04.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.855+0000 7fae8e000640 1 -- 172.21.15.159:0/4126773137 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae901f7a60 con 0x7fae901f7250 2024-04-04T06:15:04.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.856+0000 7fae8e000640 1 --2- 172.21.15.159:0/4126773137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae901f7250 0x7fae900b2070 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00e5a0 tx=0x7fae7c00ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:04.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.856+0000 7fae86a00640 1 -- 172.21.15.159:0/4126773137 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c017070 con 0x7fae901f7250 2024-04-04T06:15:04.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.856+0000 7fae86a00640 1 -- 172.21.15.159:0/4126773137 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae7c013610 con 0x7fae901f7250 2024-04-04T06:15:04.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.856+0000 7fae86a00640 1 -- 172.21.15.159:0/4126773137 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c00f040 con 0x7fae901f7250 2024-04-04T06:15:04.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.856+0000 7fae8fe00640 1 -- 172.21.15.159:0/4126773137 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae901f7d40 con 0x7fae901f7250 2024-04-04T06:15:04.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.856+0000 7fae8fe00640 1 -- 172.21.15.159:0/4126773137 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae901f76b0 con 0x7fae901f7250 2024-04-04T06:15:04.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.857+0000 7fae8fe00640 1 -- 172.21.15.159:0/4126773137 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae90075940 con 0x7fae901f7250 2024-04-04T06:15:04.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.858+0000 7fae86a00640 1 -- 172.21.15.159:0/4126773137 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae7c01a070 con 0x7fae901f7250 2024-04-04T06:15:04.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.860+0000 7fae86a00640 1 --2- 172.21.15.159:0/4126773137 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fae7006c8f0 0x7fae7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:04.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.860+0000 7fae86a00640 1 -- 172.21.15.159:0/4126773137 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4857+0+0 (secure 0 0 0) 0x7fae7c08b2c0 con 0x7fae901f7250 2024-04-04T06:15:04.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.860+0000 7fae8ea00640 1 --2- 172.21.15.159:0/4126773137 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fae7006c8f0 0x7fae7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:04.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.861+0000 7fae8ea00640 1 --2- 172.21.15.159:0/4126773137 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fae7006c8f0 0x7fae7006edb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fae80034000 tx=0x7fae80039040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:04.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:04.864+0000 7fae86a00640 1 -- 172.21.15.159:0/4126773137 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fae7c0599a0 con 0x7fae901f7250 2024-04-04T06:15:05.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.066+0000 7fae8fe00640 1 -- 172.21.15.159:0/4126773137 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fae900b3f00 con 0x7fae901f7250 2024-04-04T06:15:05.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.067+0000 7fae86a00640 1 -- 172.21.15.159:0/4126773137 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v97) ==== 84+0+65 (secure 0 0 0) 0x7fae7c0592a0 con 0x7fae901f7250 2024-04-04T06:15:05.062 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:15:05.063 INFO:teuthology.orchestra.run.smithi159.stdout:[ 2024-04-04T06:15:05.063 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:15:05.063 INFO:teuthology.orchestra.run.smithi159.stdout: "poolnum": 1, 2024-04-04T06:15:05.063 INFO:teuthology.orchestra.run.smithi159.stdout: "poolname": ".mgr" 2024-04-04T06:15:05.063 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:05.063 INFO:teuthology.orchestra.run.smithi159.stdout:] 2024-04-04T06:15:05.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.069+0000 7fae8fe00640 1 -- 172.21.15.159:0/4126773137 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fae7006c8f0 msgr2=0x7fae7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:05.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.069+0000 7fae8fe00640 1 --2- 172.21.15.159:0/4126773137 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fae7006c8f0 0x7fae7006edb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fae80034000 tx=0x7fae80039040 comp rx=0 tx=0).stop 2024-04-04T06:15:05.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.069+0000 7fae8fe00640 1 -- 172.21.15.159:0/4126773137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae901f7250 msgr2=0x7fae900b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:05.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.069+0000 7fae8fe00640 1 --2- 172.21.15.159:0/4126773137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae901f7250 0x7fae900b2070 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00e5a0 tx=0x7fae7c00ea60 comp rx=0 tx=0).stop 2024-04-04T06:15:05.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.069+0000 7fae8fe00640 1 -- 172.21.15.159:0/4126773137 shutdown_connections 2024-04-04T06:15:05.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.069+0000 7fae8fe00640 1 --2- 172.21.15.159:0/4126773137 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fae7006c8f0 0x7fae7006edb0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:05.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.069+0000 7fae8fe00640 1 --2- 172.21.15.159:0/4126773137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae901f4440 0x7fae9015df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:05.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.069+0000 7fae8fe00640 1 --2- 172.21.15.159:0/4126773137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae901f7250 0x7fae900b2070 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:05.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.069+0000 7fae8fe00640 1 --2- 172.21.15.159:0/4126773137 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae740039b0 0x7fae9013d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:05.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.070+0000 7fae8fe00640 1 -- 172.21.15.159:0/4126773137 >> 172.21.15.159:0/4126773137 conn(0x7fae9006b570 msgr2=0x7fae900b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:05.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.070+0000 7fae8fe00640 1 -- 172.21.15.159:0/4126773137 shutdown_connections 2024-04-04T06:15:05.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.070+0000 7fae8fe00640 1 -- 172.21.15.159:0/4126773137 wait complete. 2024-04-04T06:15:05.083 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-04T06:15:05.083 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-04T06:15:05.298 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.304+0000 7f3c4a200640 1 Processor -- start 2024-04-04T06:15:05.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.304+0000 7f3c4a200640 1 -- start start 2024-04-04T06:15:05.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.306+0000 7f3c4a200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3c4413f190 0x7f3c440ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:05.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.306+0000 7f3c4a200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3c440ced40 0x7f3c44141890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:05.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.306+0000 7f3c4a200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c441421e0 con 0x7f3c44141dd0 2024-04-04T06:15:05.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.306+0000 7f3c4a200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f3c4413f570 con 0x7f3c440ced40 2024-04-04T06:15:05.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.306+0000 7f3c4a200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3c440cfbf0 con 0x7f3c4413f190 2024-04-04T06:15:05.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.306+0000 7f3c43e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3c440ced40 0x7f3c44141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:05.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.306+0000 7f3c43e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3c440ced40 0x7f3c44141890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35490/0 (socket says 172.21.15.159:35490) 2024-04-04T06:15:05.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.306+0000 7f3c43e00640 1 -- 172.21.15.159:0/1049300827 learned_addr learned my addr 172.21.15.159:0/1049300827 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:05.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.306+0000 7f3c48e00640 1 --2- 172.21.15.159:0/1049300827 >> v2:172.21.15.178:3300/0 conn(0x7f3c4413f190 0x7f3c440ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:05.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.307+0000 7f3c43e00640 1 -- 172.21.15.159:0/1049300827 >> v2:172.21.15.178:3300/0 conn(0x7f3c4413f190 msgr2=0x7f3c440ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:05.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.307+0000 7f3c43e00640 1 --2- 172.21.15.159:0/1049300827 >> v2:172.21.15.178:3300/0 conn(0x7f3c4413f190 0x7f3c440ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:05.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.307+0000 7f3c43e00640 1 -- 172.21.15.159:0/1049300827 >> v1:172.21.15.178:6789/0 conn(0x7f3c44141dd0 legacy=0x7f3c4415fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:05.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.307+0000 7f3c43e00640 1 -- 172.21.15.159:0/1049300827 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c440cf350 con 0x7f3c440ced40 2024-04-04T06:15:05.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.308+0000 7f3c43e00640 1 --2- 172.21.15.159:0/1049300827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c440ced40 0x7f3c44141890 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f3c44073710 tx=0x7f3c3002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0d1dd5920fe2c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:05.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.308+0000 7f3c43400640 1 -- 172.21.15.159:0/1049300827 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30035020 con 0x7f3c440ced40 2024-04-04T06:15:05.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.308+0000 7f3c43400640 1 -- 172.21.15.159:0/1049300827 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c30031870 con 0x7f3c440ced40 2024-04-04T06:15:05.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.308+0000 7f3c43400640 1 -- 172.21.15.159:0/1049300827 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30031c40 con 0x7f3c440ced40 2024-04-04T06:15:05.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.309+0000 7f3c4a200640 1 -- 172.21.15.159:0/1049300827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c440ced40 msgr2=0x7f3c44141890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:05.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.309+0000 7f3c4a200640 1 --2- 172.21.15.159:0/1049300827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c440ced40 0x7f3c44141890 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f3c44073710 tx=0x7f3c3002b910 comp rx=0 tx=0).stop 2024-04-04T06:15:05.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.309+0000 7f3c4a200640 1 -- 172.21.15.159:0/1049300827 shutdown_connections 2024-04-04T06:15:05.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.309+0000 7f3c4a200640 1 --2- 172.21.15.159:0/1049300827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c440ced40 0x7f3c44141890 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:05.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.309+0000 7f3c4a200640 1 --2- 172.21.15.159:0/1049300827 >> v2:172.21.15.178:3300/0 conn(0x7f3c4413f190 0x7f3c440ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:05.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.309+0000 7f3c4a200640 1 -- 172.21.15.159:0/1049300827 >> 172.21.15.159:0/1049300827 conn(0x7f3c440b1d80 msgr2=0x7f3c440b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:05.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.310+0000 7f3c4a200640 1 -- 172.21.15.159:0/1049300827 shutdown_connections 2024-04-04T06:15:05.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.310+0000 7f3c4a200640 1 -- 172.21.15.159:0/1049300827 wait complete. 2024-04-04T06:15:05.305 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.312+0000 7f3c4a200640 1 Processor -- start 2024-04-04T06:15:05.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.312+0000 7f3c4a200640 1 -- start start 2024-04-04T06:15:05.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.313+0000 7f3c4a200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c440ced40 0x7f3c441f7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:05.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.313+0000 7f3c4a200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c4413f190 0x7f3c4413fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:05.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.313+0000 7f3c4a200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c44141dd0 0x7f3c4415b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:05.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.313+0000 7f3c4a200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4406d670 con 0x7f3c44141dd0 2024-04-04T06:15:05.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.313+0000 7f3c4a200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3c440cf910 con 0x7f3c4413f190 2024-04-04T06:15:05.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.313+0000 7f3c4a200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3c44142550 con 0x7f3c440ced40 2024-04-04T06:15:05.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.313+0000 7f3c49800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c44141dd0 0x7f3c4415b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:05.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.313+0000 7f3c49800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c44141dd0 0x7f3c4415b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35506/0 (socket says 172.21.15.159:35506) 2024-04-04T06:15:05.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.313+0000 7f3c49800640 1 -- 172.21.15.159:0/3580735590 learned_addr learned my addr 172.21.15.159:0/3580735590 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:05.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.314+0000 7f3c48e00640 1 --2- 172.21.15.159:0/3580735590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c440ced40 0x7f3c441f7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:05.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.314+0000 7f3c43e00640 1 --2- 172.21.15.159:0/3580735590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c4413f190 0x7f3c4413fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:05.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.314+0000 7f3c49800640 1 -- 172.21.15.159:0/3580735590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c440ced40 msgr2=0x7f3c441f7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:05.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.314+0000 7f3c49800640 1 --2- 172.21.15.159:0/3580735590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c440ced40 0x7f3c441f7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:05.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.314+0000 7f3c49800640 1 -- 172.21.15.159:0/3580735590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c4413f190 msgr2=0x7f3c4413fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:05.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.314+0000 7f3c49800640 1 --2- 172.21.15.159:0/3580735590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c4413f190 0x7f3c4413fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:05.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.314+0000 7f3c49800640 1 -- 172.21.15.159:0/3580735590 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c441406c0 con 0x7f3c44141dd0 2024-04-04T06:15:05.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.314+0000 7f3c43e00640 1 --2- 172.21.15.159:0/3580735590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c4413f190 0x7f3c4413fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:05.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.314+0000 7f3c49800640 1 --2- 172.21.15.159:0/3580735590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c44141dd0 0x7f3c4415b5d0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00b970 tx=0x7f3c3c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:05.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.314+0000 7f3c40c00640 1 -- 172.21.15.159:0/3580735590 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c019070 con 0x7f3c44141dd0 2024-04-04T06:15:05.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.315+0000 7f3c40c00640 1 -- 172.21.15.159:0/3580735590 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c3c00cb40 con 0x7f3c44141dd0 2024-04-04T06:15:05.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.315+0000 7f3c40c00640 1 -- 172.21.15.159:0/3580735590 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c014440 con 0x7f3c44141dd0 2024-04-04T06:15:05.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.315+0000 7f3c4a200640 1 -- 172.21.15.159:0/3580735590 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c441409a0 con 0x7f3c44141dd0 2024-04-04T06:15:05.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.315+0000 7f3c4a200640 1 -- 172.21.15.159:0/3580735590 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c44140f60 con 0x7f3c44141dd0 2024-04-04T06:15:05.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.316+0000 7f3c4a200640 1 -- 172.21.15.159:0/3580735590 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c44075940 con 0x7f3c44141dd0 2024-04-04T06:15:05.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.317+0000 7f3c40c00640 1 -- 172.21.15.159:0/3580735590 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c3c01c070 con 0x7f3c44141dd0 2024-04-04T06:15:05.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.318+0000 7f3c40c00640 1 --2- 172.21.15.159:0/3580735590 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c2406c8f0 0x7f3c2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:05.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.318+0000 7f3c40c00640 1 -- 172.21.15.159:0/3580735590 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4857+0+0 (secure 0 0 0) 0x7f3c3c08b440 con 0x7f3c44141dd0 2024-04-04T06:15:05.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.318+0000 7f3c48e00640 1 --2- 172.21.15.159:0/3580735590 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c2406c8f0 0x7f3c2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:05.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.319+0000 7f3c48e00640 1 --2- 172.21.15.159:0/3580735590 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c2406c8f0 0x7f3c2406edb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3c38006fd0 tx=0x7f3c38008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:05.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.322+0000 7f3c40c00640 1 -- 172.21.15.159:0/3580735590 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3c3c059bb0 con 0x7f3c44141dd0 2024-04-04T06:15:05.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:05.522+0000 7f3c4a200640 1 -- 172.21.15.159:0/3580735590 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3c440b3710 con 0x7f3c44141dd0 2024-04-04T06:15:06.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.213+0000 7f3c40c00640 1 -- 172.21.15.159:0/3580735590 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v98) ==== 138+0+0 (secure 0 0 0) 0x7f3c3c0594b0 con 0x7f3c44141dd0 2024-04-04T06:15:06.208 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_metadata' created 2024-04-04T06:15:06.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.218+0000 7f3c4a200640 1 -- 172.21.15.159:0/3580735590 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c2406c8f0 msgr2=0x7f3c2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:06.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.218+0000 7f3c4a200640 1 --2- 172.21.15.159:0/3580735590 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c2406c8f0 0x7f3c2406edb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3c38006fd0 tx=0x7f3c38008040 comp rx=0 tx=0).stop 2024-04-04T06:15:06.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.218+0000 7f3c4a200640 1 -- 172.21.15.159:0/3580735590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c44141dd0 msgr2=0x7f3c4415b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:06.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.218+0000 7f3c4a200640 1 --2- 172.21.15.159:0/3580735590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c44141dd0 0x7f3c4415b5d0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00b970 tx=0x7f3c3c00be30 comp rx=0 tx=0).stop 2024-04-04T06:15:06.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.218+0000 7f3c4a200640 1 -- 172.21.15.159:0/3580735590 shutdown_connections 2024-04-04T06:15:06.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.218+0000 7f3c4a200640 1 --2- 172.21.15.159:0/3580735590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c44141dd0 0x7f3c4415b5d0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:06.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.218+0000 7f3c4a200640 1 --2- 172.21.15.159:0/3580735590 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3c2406c8f0 0x7f3c2406edb0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:06.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.218+0000 7f3c4a200640 1 --2- 172.21.15.159:0/3580735590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c4413f190 0x7f3c4413fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:06.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.218+0000 7f3c4a200640 1 --2- 172.21.15.159:0/3580735590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3c440ced40 0x7f3c441f7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:06.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.218+0000 7f3c4a200640 1 -- 172.21.15.159:0/3580735590 >> 172.21.15.159:0/3580735590 conn(0x7f3c440b1d80 msgr2=0x7f3c44152d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:06.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.222+0000 7f3c4a200640 1 -- 172.21.15.159:0/3580735590 shutdown_connections 2024-04-04T06:15:06.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.222+0000 7f3c4a200640 1 -- 172.21.15.159:0/3580735590 wait complete. 2024-04-04T06:15:06.241 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-04T06:15:06.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.454+0000 7f4d06a00640 1 Processor -- start 2024-04-04T06:15:06.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.454+0000 7f4d06a00640 1 -- start start 2024-04-04T06:15:06.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.456+0000 7f4d06a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4d0015e440 0x7f4d000cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:06.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.456+0000 7f4d06a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4d000b64e0 0x7f4d000d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:06.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.456+0000 7f4d06a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d000b7580 con 0x7f4d000cf0f0 2024-04-04T06:15:06.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.456+0000 7f4d06a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f4d000cf670 con 0x7f4d000b64e0 2024-04-04T06:15:06.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.456+0000 7f4d06a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4d000cf950 con 0x7f4d0015e440 2024-04-04T06:15:06.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.456+0000 7f4d06000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4d000b64e0 0x7f4d000d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:06.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.456+0000 7f4d06000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4d000b64e0 0x7f4d000d2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35510/0 (socket says 172.21.15.159:35510) 2024-04-04T06:15:06.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.456+0000 7f4d06000640 1 -- 172.21.15.159:0/3783153963 learned_addr learned my addr 172.21.15.159:0/3783153963 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:06.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.456+0000 7f4d05600640 1 --2- 172.21.15.159:0/3783153963 >> v2:172.21.15.178:3300/0 conn(0x7f4d0015e440 0x7f4d000cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:06.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.457+0000 7f4d06000640 1 -- 172.21.15.159:0/3783153963 >> v2:172.21.15.178:3300/0 conn(0x7f4d0015e440 msgr2=0x7f4d000cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:06.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.457+0000 7f4d06000640 1 --2- 172.21.15.159:0/3783153963 >> v2:172.21.15.178:3300/0 conn(0x7f4d0015e440 0x7f4d000cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:06.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.457+0000 7f4d06000640 1 -- 172.21.15.159:0/3783153963 >> v1:172.21.15.178:6789/0 conn(0x7f4d000cf0f0 legacy=0x7f4d000b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:06.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.457+0000 7f4d06000640 1 -- 172.21.15.159:0/3783153963 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d000d2670 con 0x7f4d000b64e0 2024-04-04T06:15:06.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.458+0000 7f4d06000640 1 --2- 172.21.15.159:0/3783153963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d000b64e0 0x7f4d000d2e70 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f4d000d1ed0 tx=0x7f4cf802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b073b5e8378e231 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:06.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.458+0000 7f4cffe00640 1 -- 172.21.15.159:0/3783153963 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf802ca80 con 0x7f4d000b64e0 2024-04-04T06:15:06.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.458+0000 7f4cffe00640 1 -- 172.21.15.159:0/3783153963 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf802cc20 con 0x7f4d000b64e0 2024-04-04T06:15:06.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.458+0000 7f4cffe00640 1 -- 172.21.15.159:0/3783153963 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf80343f0 con 0x7f4d000b64e0 2024-04-04T06:15:06.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.459+0000 7f4d06a00640 1 -- 172.21.15.159:0/3783153963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d000b64e0 msgr2=0x7f4d000d2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:06.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.459+0000 7f4d06a00640 1 --2- 172.21.15.159:0/3783153963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d000b64e0 0x7f4d000d2e70 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f4d000d1ed0 tx=0x7f4cf802bee0 comp rx=0 tx=0).stop 2024-04-04T06:15:06.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.459+0000 7f4d06a00640 1 -- 172.21.15.159:0/3783153963 shutdown_connections 2024-04-04T06:15:06.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.459+0000 7f4d06a00640 1 --2- 172.21.15.159:0/3783153963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d000b64e0 0x7f4d000d2e70 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:06.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.459+0000 7f4d06a00640 1 --2- 172.21.15.159:0/3783153963 >> v2:172.21.15.178:3300/0 conn(0x7f4d0015e440 0x7f4d000cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:06.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.459+0000 7f4d06a00640 1 -- 172.21.15.159:0/3783153963 >> 172.21.15.159:0/3783153963 conn(0x7f4d000676a0 msgr2=0x7f4d00067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:06.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.459+0000 7f4d06a00640 1 -- 172.21.15.159:0/3783153963 shutdown_connections 2024-04-04T06:15:06.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.459+0000 7f4d06a00640 1 -- 172.21.15.159:0/3783153963 wait complete. 2024-04-04T06:15:06.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.461+0000 7f4d06a00640 1 Processor -- start 2024-04-04T06:15:06.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.461+0000 7f4d06a00640 1 -- start start 2024-04-04T06:15:06.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.462+0000 7f4d06a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d000b64e0 0x7f4d001f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:06.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.462+0000 7f4d06a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d000cf0f0 0x7f4d00161150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:06.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.462+0000 7f4d06a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d0015e440 0x7f4d000c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:06.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.462+0000 7f4d06a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4d00162560 con 0x7f4d0015e440 2024-04-04T06:15:06.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.462+0000 7f4d06a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001623e0 con 0x7f4d000cf0f0 2024-04-04T06:15:06.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.462+0000 7f4d06a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4d0015e820 con 0x7f4d000b64e0 2024-04-04T06:15:06.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.462+0000 7f4d06000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d0015e440 0x7f4d000c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:06.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.462+0000 7f4d06000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d0015e440 0x7f4d000c9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35526/0 (socket says 172.21.15.159:35526) 2024-04-04T06:15:06.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.462+0000 7f4d06000640 1 -- 172.21.15.159:0/4166837544 learned_addr learned my addr 172.21.15.159:0/4166837544 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:06.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.462+0000 7f4d04c00640 1 --2- 172.21.15.159:0/4166837544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d000cf0f0 0x7f4d00161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:06.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.462+0000 7f4d05600640 1 --2- 172.21.15.159:0/4166837544 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d000b64e0 0x7f4d001f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:06.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.463+0000 7f4d06000640 1 -- 172.21.15.159:0/4166837544 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d000b64e0 msgr2=0x7f4d001f4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:06.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.463+0000 7f4d06000640 1 --2- 172.21.15.159:0/4166837544 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d000b64e0 0x7f4d001f4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:06.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.463+0000 7f4d06000640 1 -- 172.21.15.159:0/4166837544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d000cf0f0 msgr2=0x7f4d00161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:06.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.463+0000 7f4d06000640 1 --2- 172.21.15.159:0/4166837544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d000cf0f0 0x7f4d00161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:06.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.463+0000 7f4d06000640 1 -- 172.21.15.159:0/4166837544 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d000c8e90 con 0x7f4d0015e440 2024-04-04T06:15:06.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.463+0000 7f4d06000640 1 --2- 172.21.15.159:0/4166837544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d0015e440 0x7f4d000c9a70 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8009f40 tx=0x7f4cf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:06.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.463+0000 7f4cfd600640 1 -- 172.21.15.159:0/4166837544 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8043020 con 0x7f4d0015e440 2024-04-04T06:15:06.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.463+0000 7f4cfd600640 1 -- 172.21.15.159:0/4166837544 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf803e450 con 0x7f4d0015e440 2024-04-04T06:15:06.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.463+0000 7f4d06a00640 1 -- 172.21.15.159:0/4166837544 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d000c7e30 con 0x7f4d0015e440 2024-04-04T06:15:06.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.463+0000 7f4cfd600640 1 -- 172.21.15.159:0/4166837544 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf803e820 con 0x7f4d0015e440 2024-04-04T06:15:06.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.463+0000 7f4d06a00640 1 -- 172.21.15.159:0/4166837544 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d000c83b0 con 0x7f4d0015e440 2024-04-04T06:15:06.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.465+0000 7f4d06a00640 1 -- 172.21.15.159:0/4166837544 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d00075940 con 0x7f4d0015e440 2024-04-04T06:15:06.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.466+0000 7f4cfd600640 1 -- 172.21.15.159:0/4166837544 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4cf8046020 con 0x7f4d0015e440 2024-04-04T06:15:06.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.467+0000 7f4cfd600640 1 --2- 172.21.15.159:0/4166837544 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ce006c8f0 0x7f4ce006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:06.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.467+0000 7f4cfd600640 1 -- 172.21.15.159:0/4166837544 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5227+0+0 (secure 0 0 0) 0x7f4cf80ae2c0 con 0x7f4d0015e440 2024-04-04T06:15:06.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.467+0000 7f4d05600640 1 --2- 172.21.15.159:0/4166837544 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ce006c8f0 0x7f4ce006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:06.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.469+0000 7f4d05600640 1 --2- 172.21.15.159:0/4166837544 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ce006c8f0 0x7f4ce006edb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4006fd0 tx=0x7f4cf4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:06.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.472+0000 7f4cfd600640 1 -- 172.21.15.159:0/4166837544 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4cf804d030 con 0x7f4d0015e440 2024-04-04T06:15:06.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:06.671+0000 7f4d06a00640 1 -- 172.21.15.159:0/4166837544 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4d000c7650 con 0x7f4d0015e440 2024-04-04T06:15:07.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.228+0000 7f4cfd600640 1 -- 172.21.15.159:0/4166837544 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v99) ==== 170+0+0 (secure 0 0 0) 0x7f4d000b7580 con 0x7f4d0015e440 2024-04-04T06:15:07.223 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_data' created 2024-04-04T06:15:07.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.232+0000 7f4d06a00640 1 -- 172.21.15.159:0/4166837544 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ce006c8f0 msgr2=0x7f4ce006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:07.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.232+0000 7f4d06a00640 1 --2- 172.21.15.159:0/4166837544 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ce006c8f0 0x7f4ce006edb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4006fd0 tx=0x7f4cf4008040 comp rx=0 tx=0).stop 2024-04-04T06:15:07.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.232+0000 7f4d06a00640 1 -- 172.21.15.159:0/4166837544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d0015e440 msgr2=0x7f4d000c9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:07.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.232+0000 7f4d06a00640 1 --2- 172.21.15.159:0/4166837544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d0015e440 0x7f4d000c9a70 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8009f40 tx=0x7f4cf8004a40 comp rx=0 tx=0).stop 2024-04-04T06:15:07.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.233+0000 7f4d06a00640 1 -- 172.21.15.159:0/4166837544 shutdown_connections 2024-04-04T06:15:07.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.233+0000 7f4d06a00640 1 --2- 172.21.15.159:0/4166837544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d0015e440 0x7f4d000c9a70 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:07.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.233+0000 7f4d06a00640 1 --2- 172.21.15.159:0/4166837544 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ce006c8f0 0x7f4ce006edb0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:07.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.233+0000 7f4d06a00640 1 --2- 172.21.15.159:0/4166837544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d000cf0f0 0x7f4d00161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:07.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.233+0000 7f4d06a00640 1 --2- 172.21.15.159:0/4166837544 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d000b64e0 0x7f4d001f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:07.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.233+0000 7f4d06a00640 1 -- 172.21.15.159:0/4166837544 >> 172.21.15.159:0/4166837544 conn(0x7f4d000676a0 msgr2=0x7f4d000b5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:07.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.235+0000 7f4d06a00640 1 -- 172.21.15.159:0/4166837544 shutdown_connections 2024-04-04T06:15:07.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.235+0000 7f4d06a00640 1 -- 172.21.15.159:0/4166837544 wait complete. 2024-04-04T06:15:07.261 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-04T06:15:07.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.451+0000 7f7d68200640 1 Processor -- start 2024-04-04T06:15:07.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.451+0000 7f7d68200640 1 -- start start 2024-04-04T06:15:07.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.452+0000 7f7d68200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7d60153580 0x7f7d600d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:07.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.452+0000 7f7d68200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7d600af4b0 0x7f7d601600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:07.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.452+0000 7f7d68200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d600afab0 con 0x7f7d600d07c0 2024-04-04T06:15:07.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.453+0000 7f7d68200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f7d600d0c30 con 0x7f7d600af4b0 2024-04-04T06:15:07.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.453+0000 7f7d68200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7d6015cc80 con 0x7f7d60153580 2024-04-04T06:15:07.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.453+0000 7f7d66200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7d600af4b0 0x7f7d601600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:07.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.453+0000 7f7d66200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7d600af4b0 0x7f7d601600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35536/0 (socket says 172.21.15.159:35536) 2024-04-04T06:15:07.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.453+0000 7f7d66200640 1 -- 172.21.15.159:0/3426797277 learned_addr learned my addr 172.21.15.159:0/3426797277 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:07.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.453+0000 7f7d65800640 1 --2- 172.21.15.159:0/3426797277 >> v2:172.21.15.178:3300/0 conn(0x7f7d60153580 0x7f7d600d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:07.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.453+0000 7f7d66200640 1 -- 172.21.15.159:0/3426797277 >> v2:172.21.15.178:3300/0 conn(0x7f7d60153580 msgr2=0x7f7d600d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:07.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.453+0000 7f7d66200640 1 --2- 172.21.15.159:0/3426797277 >> v2:172.21.15.178:3300/0 conn(0x7f7d60153580 0x7f7d600d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:07.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.453+0000 7f7d66200640 1 -- 172.21.15.159:0/3426797277 >> v1:172.21.15.178:6789/0 conn(0x7f7d600d07c0 legacy=0x7f7d600aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:07.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.453+0000 7f7d66200640 1 -- 172.21.15.159:0/3426797277 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6015da00 con 0x7f7d600af4b0 2024-04-04T06:15:07.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.455+0000 7f7d66200640 1 --2- 172.21.15.159:0/3426797277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d600af4b0 0x7f7d601600d0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f7d600b5270 tx=0x7f7d5802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ad23aca75143b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:07.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.455+0000 7f7d5fe00640 1 -- 172.21.15.159:0/3426797277 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5802c9f0 con 0x7f7d600af4b0 2024-04-04T06:15:07.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.455+0000 7f7d5fe00640 1 -- 172.21.15.159:0/3426797277 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d5802cb90 con 0x7f7d600af4b0 2024-04-04T06:15:07.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.455+0000 7f7d5fe00640 1 -- 172.21.15.159:0/3426797277 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d580343f0 con 0x7f7d600af4b0 2024-04-04T06:15:07.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.455+0000 7f7d68200640 1 -- 172.21.15.159:0/3426797277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d600af4b0 msgr2=0x7f7d601600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:07.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.455+0000 7f7d68200640 1 --2- 172.21.15.159:0/3426797277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d600af4b0 0x7f7d601600d0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f7d600b5270 tx=0x7f7d5802be50 comp rx=0 tx=0).stop 2024-04-04T06:15:07.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.456+0000 7f7d68200640 1 -- 172.21.15.159:0/3426797277 shutdown_connections 2024-04-04T06:15:07.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.456+0000 7f7d68200640 1 --2- 172.21.15.159:0/3426797277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d600af4b0 0x7f7d601600d0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:07.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.456+0000 7f7d68200640 1 --2- 172.21.15.159:0/3426797277 >> v2:172.21.15.178:3300/0 conn(0x7f7d60153580 0x7f7d600d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:07.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.456+0000 7f7d68200640 1 -- 172.21.15.159:0/3426797277 >> 172.21.15.159:0/3426797277 conn(0x7f7d6006b750 msgr2=0x7f7d6006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:07.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.456+0000 7f7d68200640 1 -- 172.21.15.159:0/3426797277 shutdown_connections 2024-04-04T06:15:07.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.456+0000 7f7d68200640 1 -- 172.21.15.159:0/3426797277 wait complete. 2024-04-04T06:15:07.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.457+0000 7f7d68200640 1 Processor -- start 2024-04-04T06:15:07.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.457+0000 7f7d68200640 1 -- start start 2024-04-04T06:15:07.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.458+0000 7f7d68200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d600af4b0 0x7f7d6014c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:07.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.458+0000 7f7d68200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d600d07c0 0x7f7d600b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:07.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.458+0000 7f7d68200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d60153580 0x7f7d600b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:07.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.458+0000 7f7d68200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6015d4e0 con 0x7f7d60153580 2024-04-04T06:15:07.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.458+0000 7f7d68200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6015d200 con 0x7f7d600af4b0 2024-04-04T06:15:07.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.458+0000 7f7d68200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7d60153960 con 0x7f7d600d07c0 2024-04-04T06:15:07.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.459+0000 7f7d66200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d60153580 0x7f7d600b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:07.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.459+0000 7f7d66200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d60153580 0x7f7d600b2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35544/0 (socket says 172.21.15.159:35544) 2024-04-04T06:15:07.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.459+0000 7f7d66200640 1 -- 172.21.15.159:0/1111072951 learned_addr learned my addr 172.21.15.159:0/1111072951 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:07.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.459+0000 7f7d66200640 1 -- 172.21.15.159:0/1111072951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d600d07c0 msgr2=0x7f7d600b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:07.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.459+0000 7f7d66200640 1 --2- 172.21.15.159:0/1111072951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d600d07c0 0x7f7d600b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:07.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.459+0000 7f7d66200640 1 -- 172.21.15.159:0/1111072951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d600af4b0 msgr2=0x7f7d6014c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:07.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.459+0000 7f7d66200640 1 --2- 172.21.15.159:0/1111072951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d600af4b0 0x7f7d6014c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:07.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.459+0000 7f7d66200640 1 -- 172.21.15.159:0/1111072951 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d600ce680 con 0x7f7d60153580 2024-04-04T06:15:07.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.459+0000 7f7d66200640 1 --2- 172.21.15.159:0/1111072951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d60153580 0x7f7d600b2250 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f7d58036040 tx=0x7f7d58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:07.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.460+0000 7f7d5d600640 1 -- 172.21.15.159:0/1111072951 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58043020 con 0x7f7d60153580 2024-04-04T06:15:07.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.460+0000 7f7d5d600640 1 -- 172.21.15.159:0/1111072951 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d5803e450 con 0x7f7d60153580 2024-04-04T06:15:07.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.460+0000 7f7d5d600640 1 -- 172.21.15.159:0/1111072951 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58009510 con 0x7f7d60153580 2024-04-04T06:15:07.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.460+0000 7f7d68200640 1 -- 172.21.15.159:0/1111072951 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d600ce960 con 0x7f7d60153580 2024-04-04T06:15:07.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.460+0000 7f7d68200640 1 -- 172.21.15.159:0/1111072951 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d600cef20 con 0x7f7d60153580 2024-04-04T06:15:07.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.461+0000 7f7d68200640 1 -- 172.21.15.159:0/1111072951 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d600685c0 con 0x7f7d60153580 2024-04-04T06:15:07.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.462+0000 7f7d5d600640 1 -- 172.21.15.159:0/1111072951 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d58046020 con 0x7f7d60153580 2024-04-04T06:15:07.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.463+0000 7f7d5d600640 1 --2- 172.21.15.159:0/1111072951 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7d4406c8f0 0x7f7d4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:07.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.463+0000 7f7d5d600640 1 -- 172.21.15.159:0/1111072951 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5609+0+0 (secure 0 0 0) 0x7f7d580ae410 con 0x7f7d60153580 2024-04-04T06:15:07.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.464+0000 7f7d65800640 1 --2- 172.21.15.159:0/1111072951 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7d4406c8f0 0x7f7d4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:07.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.465+0000 7f7d65800640 1 --2- 172.21.15.159:0/1111072951 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7d4406c8f0 0x7f7d4406edb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7d60153d20 tx=0x7f7d54008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:07.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.469+0000 7f7d5d600640 1 -- 172.21.15.159:0/1111072951 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7d5804d030 con 0x7f7d60153580 2024-04-04T06:15:07.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:07.702+0000 7f7d68200640 1 -- 172.21.15.159:0/1111072951 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7d600ad010 con 0x7f7d60153580 2024-04-04T06:15:08.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.232+0000 7f7d5d600640 1 -- 172.21.15.159:0/1111072951 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v80) ==== 174+0+0 (secure 0 0 0) 0x7f7d5807c800 con 0x7f7d60153580 2024-04-04T06:15:08.227 INFO:teuthology.orchestra.run.smithi159.stderr:new fs with metadata pool 14 and data pool 15 2024-04-04T06:15:08.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.236+0000 7f7d68200640 1 -- 172.21.15.159:0/1111072951 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7d4406c8f0 msgr2=0x7f7d4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:08.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.236+0000 7f7d68200640 1 --2- 172.21.15.159:0/1111072951 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7d4406c8f0 0x7f7d4406edb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7d60153d20 tx=0x7f7d54008040 comp rx=0 tx=0).stop 2024-04-04T06:15:08.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.236+0000 7f7d68200640 1 -- 172.21.15.159:0/1111072951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d60153580 msgr2=0x7f7d600b2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:08.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.236+0000 7f7d68200640 1 --2- 172.21.15.159:0/1111072951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d60153580 0x7f7d600b2250 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f7d58036040 tx=0x7f7d58004a40 comp rx=0 tx=0).stop 2024-04-04T06:15:08.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.236+0000 7f7d68200640 1 -- 172.21.15.159:0/1111072951 shutdown_connections 2024-04-04T06:15:08.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.236+0000 7f7d68200640 1 --2- 172.21.15.159:0/1111072951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d60153580 0x7f7d600b2250 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:08.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.236+0000 7f7d68200640 1 --2- 172.21.15.159:0/1111072951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7d600d07c0 0x7f7d600b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:08.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.236+0000 7f7d68200640 1 --2- 172.21.15.159:0/1111072951 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7d4406c8f0 0x7f7d4406edb0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:08.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.236+0000 7f7d68200640 1 --2- 172.21.15.159:0/1111072951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d600af4b0 0x7f7d6014c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:08.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.236+0000 7f7d68200640 1 -- 172.21.15.159:0/1111072951 >> 172.21.15.159:0/1111072951 conn(0x7f7d6006b750 msgr2=0x7f7d6015e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:08.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.239+0000 7f7d68200640 1 -- 172.21.15.159:0/1111072951 shutdown_connections 2024-04-04T06:15:08.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.239+0000 7f7d68200640 1 -- 172.21.15.159:0/1111072951 wait complete. 2024-04-04T06:15:08.276 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:15:08.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.460+0000 7f741f600640 1 Processor -- start 2024-04-04T06:15:08.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.460+0000 7f741f600640 1 -- start start 2024-04-04T06:15:08.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.461+0000 7f741f600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f741815c390 0x7f7418073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:08.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.461+0000 7f741f600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f741815fe80 con 0x7f74180af830 2024-04-04T06:15:08.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.461+0000 7f741f600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74180ad120 con 0x7f74180d0650 2024-04-04T06:15:08.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.461+0000 7f741f600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f741815c770 con 0x7f741815c390 2024-04-04T06:15:08.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.462+0000 7f7417e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f74180af830 0x7f7418160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:42950/0 (socket says 172.21.15.159:42950) 2024-04-04T06:15:08.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.462+0000 7f7417e00640 1 -- 172.21.15.159:0/1516906111 learned_addr learned my addr 172.21.15.159:0/1516906111 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:08.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.462+0000 7f741cc00640 1 --2- 172.21.15.159:0/1516906111 >> v2:172.21.15.178:3300/0 conn(0x7f741815c390 0x7f7418073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:08.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.462+0000 7f7417400640 1 -- 172.21.15.159:0/1516906111 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3531503694 0 0) 0x7f741815fe80 con 0x7f74180af830 2024-04-04T06:15:08.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.462+0000 7f7417400640 1 -- 172.21.15.159:0/1516906111 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73fc003520 con 0x7f74180af830 2024-04-04T06:15:08.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.462+0000 7f7417400640 1 -- 172.21.15.159:0/1516906111 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3005098523 0 0) 0x7f73fc003520 con 0x7f74180af830 2024-04-04T06:15:08.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.462+0000 7f7417400640 1 -- 172.21.15.159:0/1516906111 >> v2:172.21.15.178:3300/0 conn(0x7f741815c390 msgr2=0x7f7418073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:08.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.462+0000 7f7417400640 1 --2- 172.21.15.159:0/1516906111 >> v2:172.21.15.178:3300/0 conn(0x7f741815c390 0x7f7418073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:08.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.463+0000 7f7417400640 1 -- 172.21.15.159:0/1516906111 >> v1:172.21.15.178:6789/0 conn(0x7f74180d0650 legacy=0x7f7418071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:08.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.463+0000 7f7417400640 1 -- 172.21.15.159:0/1516906111 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74180ad400 con 0x7f74180af830 2024-04-04T06:15:08.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.463+0000 7f7417400640 1 -- 172.21.15.159:0/1516906111 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f7404001670 con 0x7f74180af830 2024-04-04T06:15:08.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.463+0000 7f7417400640 1 -- 172.21.15.159:0/1516906111 >> v1:172.21.15.159:6789/0 conn(0x7f74180af830 legacy=0x7f7418160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:08.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.463+0000 7f7417400640 1 --2- 172.21.15.159:0/1516906111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73fc0039b0 0x7f73fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:08.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.463+0000 7f7417400640 1 --2- 172.21.15.159:0/1516906111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73fc004800 0x7f73fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:08.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.463+0000 7f7417400640 1 --2- 172.21.15.159:0/1516906111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73fc0051c0 0x7f73fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:08.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.463+0000 7f7417400640 1 -- 172.21.15.159:0/1516906111 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f73fc00c030 con 0x7f73fc0051c0 2024-04-04T06:15:08.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.463+0000 7f7417400640 1 -- 172.21.15.159:0/1516906111 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73fc00c1e0 con 0x7f73fc0039b0 2024-04-04T06:15:08.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.463+0000 7f7417400640 1 -- 172.21.15.159:0/1516906111 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f73fc00c3e0 con 0x7f73fc004800 2024-04-04T06:15:08.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.463+0000 7f741d600640 1 --2- 172.21.15.159:0/1516906111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73fc0051c0 0x7f73fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:08.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.463+0000 7f7417e00640 1 --2- 172.21.15.159:0/1516906111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73fc004800 0x7f73fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:08.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.463+0000 7f741cc00640 1 --2- 172.21.15.159:0/1516906111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73fc0039b0 0x7f73fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:08.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.463+0000 7f741d600640 1 -- 172.21.15.159:0/1516906111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73fc004800 msgr2=0x7f73fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:08.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.463+0000 7f741d600640 1 --2- 172.21.15.159:0/1516906111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73fc004800 0x7f73fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:08.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.463+0000 7f741d600640 1 -- 172.21.15.159:0/1516906111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73fc0039b0 msgr2=0x7f73fc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:08.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.463+0000 7f741d600640 1 --2- 172.21.15.159:0/1516906111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73fc0039b0 0x7f73fc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:08.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.463+0000 7f741d600640 1 -- 172.21.15.159:0/1516906111 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73fc003520 con 0x7f73fc0051c0 2024-04-04T06:15:08.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.464+0000 7f741d600640 1 -- 172.21.15.159:0/1516906111 reap_dead start 2024-04-04T06:15:08.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.464+0000 7f741d600640 1 --2- 172.21.15.159:0/1516906111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73fc0051c0 0x7f73fc00b9c0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f7410001e50 tx=0x7f73fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21e349ed1ab04cc server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:08.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.465+0000 7f7417400640 1 -- 172.21.15.159:0/1516906111 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410001a70 con 0x7f73fc0051c0 2024-04-04T06:15:08.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.465+0000 7f7417400640 1 -- 172.21.15.159:0/1516906111 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7410003bb0 con 0x7f73fc0051c0 2024-04-04T06:15:08.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.465+0000 7f7417400640 1 -- 172.21.15.159:0/1516906111 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74100050d0 con 0x7f73fc0051c0 2024-04-04T06:15:08.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.465+0000 7f741f600640 1 -- 172.21.15.159:0/1516906111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73fc0051c0 msgr2=0x7f73fc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:08.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.465+0000 7f741f600640 1 --2- 172.21.15.159:0/1516906111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73fc0051c0 0x7f73fc00b9c0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f7410001e50 tx=0x7f73fc005830 comp rx=0 tx=0).stop 2024-04-04T06:15:08.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.465+0000 7f741f600640 1 -- 172.21.15.159:0/1516906111 shutdown_connections 2024-04-04T06:15:08.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.465+0000 7f741f600640 1 --2- 172.21.15.159:0/1516906111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73fc0051c0 0x7f73fc00b9c0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:08.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.465+0000 7f741f600640 1 -- 172.21.15.159:0/1516906111 >> 172.21.15.159:0/1516906111 conn(0x7f741806b990 msgr2=0x7f741806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:08.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.465+0000 7f741f600640 1 -- 172.21.15.159:0/1516906111 shutdown_connections 2024-04-04T06:15:08.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.465+0000 7f741f600640 1 -- 172.21.15.159:0/1516906111 wait complete. 2024-04-04T06:15:08.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.467+0000 7f741f600640 1 Processor -- start 2024-04-04T06:15:08.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.467+0000 7f741f600640 1 -- start start 2024-04-04T06:15:08.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.468+0000 7f741f600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73fc0051c0 0x7f7418150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:08.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.468+0000 7f741f600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7418152750 0x7f7418152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:08.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.468+0000 7f741f600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7418157660 0x7f7418157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:08.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.468+0000 7f741f600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f74180cf980 con 0x7f7418152750 2024-04-04T06:15:08.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.468+0000 7f741f600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7418160120 con 0x7f7418157660 2024-04-04T06:15:08.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.468+0000 7f741f600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7418073790 con 0x7f73fc0051c0 2024-04-04T06:15:08.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.468+0000 7f7417e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7418152750 0x7f7418152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:08.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.469+0000 7f7417e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7418152750 0x7f7418152bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35562/0 (socket says 172.21.15.159:35562) 2024-04-04T06:15:08.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.469+0000 7f7417e00640 1 -- 172.21.15.159:0/667153830 learned_addr learned my addr 172.21.15.159:0/667153830 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:08.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.469+0000 7f741cc00640 1 --2- 172.21.15.159:0/667153830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73fc0051c0 0x7f7418150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:08.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.469+0000 7f741d600640 1 --2- 172.21.15.159:0/667153830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7418157660 0x7f7418157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:08.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.469+0000 7f7417e00640 1 -- 172.21.15.159:0/667153830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73fc0051c0 msgr2=0x7f7418150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:08.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.469+0000 7f7417e00640 1 --2- 172.21.15.159:0/667153830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73fc0051c0 0x7f7418150860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:08.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.469+0000 7f7417e00640 1 -- 172.21.15.159:0/667153830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7418157660 msgr2=0x7f7418157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:08.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.469+0000 7f7417e00640 1 --2- 172.21.15.159:0/667153830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7418157660 0x7f7418157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:08.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.469+0000 7f7417e00640 1 -- 172.21.15.159:0/667153830 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7418150da0 con 0x7f7418152750 2024-04-04T06:15:08.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.469+0000 7f7417e00640 1 --2- 172.21.15.159:0/667153830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7418152750 0x7f7418152bd0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f740400e860 tx=0x7f740400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:08.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.469+0000 7f7414c00640 1 -- 172.21.15.159:0/667153830 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740400c8d0 con 0x7f7418152750 2024-04-04T06:15:08.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.469+0000 7f7414c00640 1 -- 172.21.15.159:0/667153830 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7404018070 con 0x7f7418152750 2024-04-04T06:15:08.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.469+0000 7f7414c00640 1 -- 172.21.15.159:0/667153830 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740400ccb0 con 0x7f7418152750 2024-04-04T06:15:08.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.469+0000 7f741f600640 1 -- 172.21.15.159:0/667153830 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7418159310 con 0x7f7418152750 2024-04-04T06:15:08.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.470+0000 7f741f600640 1 -- 172.21.15.159:0/667153830 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74181598d0 con 0x7f7418152750 2024-04-04T06:15:08.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.471+0000 7f741f600640 1 -- 172.21.15.159:0/667153830 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74180685c0 con 0x7f7418152750 2024-04-04T06:15:08.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.472+0000 7f7414c00640 1 -- 172.21.15.159:0/667153830 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f740401d460 con 0x7f7418152750 2024-04-04T06:15:08.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.473+0000 7f7414c00640 1 --2- 172.21.15.159:0/667153830 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f73f806c8f0 0x7f73f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:08.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.473+0000 7f7414c00640 1 -- 172.21.15.159:0/667153830 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f740408b960 con 0x7f7418152750 2024-04-04T06:15:08.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.474+0000 7f741cc00640 1 --2- 172.21.15.159:0/667153830 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f73f806c8f0 0x7f73f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:08.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.475+0000 7f741cc00640 1 --2- 172.21.15.159:0/667153830 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f73f806c8f0 0x7f73f806edb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f74080098d0 tx=0x7f7408008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:08.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.478+0000 7f7414c00640 1 -- 172.21.15.159:0/667153830 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7418150da0 con 0x7f7418152750 2024-04-04T06:15:08.536 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-04T06:15:08.536 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-04T06:15:08.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.674+0000 7f741f600640 1 -- 172.21.15.159:0/667153830 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f741806f1b0 con 0x7f7418152750 2024-04-04T06:15:08.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.677+0000 7f7414c00640 1 -- 172.21.15.159:0/667153830 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16636 (secure 0 0 0) 0x7f7404059e20 con 0x7f7418152750 2024-04-04T06:15:08.672 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:15:08.672 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":100,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:15:07.704490+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:15:05.537357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-04T06:15:06.689304+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:15:08.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.679+0000 7f741f600640 1 -- 172.21.15.159:0/667153830 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f73f806c8f0 msgr2=0x7f73f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:08.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.679+0000 7f741f600640 1 --2- 172.21.15.159:0/667153830 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f73f806c8f0 0x7f73f806edb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f74080098d0 tx=0x7f7408008040 comp rx=0 tx=0).stop 2024-04-04T06:15:08.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.679+0000 7f741f600640 1 -- 172.21.15.159:0/667153830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7418152750 msgr2=0x7f7418152bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:08.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.679+0000 7f741f600640 1 --2- 172.21.15.159:0/667153830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7418152750 0x7f7418152bd0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f740400e860 tx=0x7f740400ec60 comp rx=0 tx=0).stop 2024-04-04T06:15:08.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.680+0000 7f741f600640 1 -- 172.21.15.159:0/667153830 shutdown_connections 2024-04-04T06:15:08.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.680+0000 7f741f600640 1 --2- 172.21.15.159:0/667153830 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f73f806c8f0 0x7f73f806edb0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:08.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.680+0000 7f741f600640 1 --2- 172.21.15.159:0/667153830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7418157660 0x7f7418157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:08.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.680+0000 7f741f600640 1 --2- 172.21.15.159:0/667153830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7418152750 0x7f7418152bd0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:08.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.680+0000 7f741f600640 1 --2- 172.21.15.159:0/667153830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f73fc0051c0 0x7f7418150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:08.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.680+0000 7f741f600640 1 -- 172.21.15.159:0/667153830 >> 172.21.15.159:0/667153830 conn(0x7f741806b990 msgr2=0x7f7418154a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:08.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.680+0000 7f741f600640 1 -- 172.21.15.159:0/667153830 shutdown_connections 2024-04-04T06:15:08.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.680+0000 7f741f600640 1 -- 172.21.15.159:0/667153830 wait complete. 2024-04-04T06:15:08.691 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:15:08.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.898+0000 7efc4e600640 1 Processor -- start 2024-04-04T06:15:08.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.898+0000 7efc4e600640 1 -- start start 2024-04-04T06:15:08.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.899+0000 7efc4e600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc480af830 0x7efc48073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:08.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.899+0000 7efc4e600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7efc480d0650 0x7efc480d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:08.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.899+0000 7efc4e600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc4815fe80 con 0x7efc4815c390 2024-04-04T06:15:08.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.899+0000 7efc4e600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7efc4815c770 con 0x7efc480d0650 2024-04-04T06:15:08.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.899+0000 7efc4e600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efc480ad120 con 0x7efc480af830 2024-04-04T06:15:08.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.899+0000 7efc47e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7efc480d0650 0x7efc480d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:08.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.899+0000 7efc47e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7efc480d0650 0x7efc480d0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35568/0 (socket says 172.21.15.159:35568) 2024-04-04T06:15:08.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.899+0000 7efc47e00640 1 -- 172.21.15.159:0/2598937088 learned_addr learned my addr 172.21.15.159:0/2598937088 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:08.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.900+0000 7efc47e00640 1 -- 172.21.15.159:0/2598937088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc480af830 msgr2=0x7efc48073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:08.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.900+0000 7efc47e00640 1 --2- 172.21.15.159:0/2598937088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc480af830 0x7efc48073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:08.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.900+0000 7efc47e00640 1 -- 172.21.15.159:0/2598937088 >> v1:172.21.15.178:6789/0 conn(0x7efc4815c390 legacy=0x7efc48160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:08.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.900+0000 7efc47e00640 1 -- 172.21.15.159:0/2598937088 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc480ad400 con 0x7efc480d0650 2024-04-04T06:15:08.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.901+0000 7efc47e00640 1 --2- 172.21.15.159:0/2598937088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc480d0650 0x7efc480d0b00 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7efc40009e50 tx=0x7efc4002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4471aa13f31c16b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:08.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.901+0000 7efc46a00640 1 -- 172.21.15.159:0/2598937088 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4002ca70 con 0x7efc480d0650 2024-04-04T06:15:08.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.901+0000 7efc46a00640 1 -- 172.21.15.159:0/2598937088 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc4002cc10 con 0x7efc480d0650 2024-04-04T06:15:08.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.901+0000 7efc46a00640 1 -- 172.21.15.159:0/2598937088 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc400343f0 con 0x7efc480d0650 2024-04-04T06:15:08.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.902+0000 7efc4e600640 1 -- 172.21.15.159:0/2598937088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc480d0650 msgr2=0x7efc480d0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:08.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.902+0000 7efc4e600640 1 --2- 172.21.15.159:0/2598937088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc480d0650 0x7efc480d0b00 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7efc40009e50 tx=0x7efc4002bed0 comp rx=0 tx=0).stop 2024-04-04T06:15:08.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.902+0000 7efc4e600640 1 -- 172.21.15.159:0/2598937088 shutdown_connections 2024-04-04T06:15:08.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.902+0000 7efc4e600640 1 --2- 172.21.15.159:0/2598937088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc480d0650 0x7efc480d0b00 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:08.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.902+0000 7efc4e600640 1 --2- 172.21.15.159:0/2598937088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc480af830 0x7efc48073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:08.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.902+0000 7efc4e600640 1 -- 172.21.15.159:0/2598937088 >> 172.21.15.159:0/2598937088 conn(0x7efc4806b990 msgr2=0x7efc4806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:08.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.902+0000 7efc4e600640 1 -- 172.21.15.159:0/2598937088 shutdown_connections 2024-04-04T06:15:08.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.902+0000 7efc4e600640 1 -- 172.21.15.159:0/2598937088 wait complete. 2024-04-04T06:15:08.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.903+0000 7efc4e600640 1 Processor -- start 2024-04-04T06:15:08.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.904+0000 7efc4e600640 1 -- start start 2024-04-04T06:15:08.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.904+0000 7efc4e600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc480af830 0x7efc4814b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:08.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.904+0000 7efc4e600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc480d0650 0x7efc48150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:08.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.904+0000 7efc4e600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc4815c390 0x7efc4814e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:08.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.904+0000 7efc4e600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efc48160d50 con 0x7efc480af830 2024-04-04T06:15:08.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.904+0000 7efc4e600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efc48160120 con 0x7efc480d0650 2024-04-04T06:15:08.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.904+0000 7efc4e600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efc480cf980 con 0x7efc4815c390 2024-04-04T06:15:08.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.905+0000 7efc47400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc480af830 0x7efc4814b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:08.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.905+0000 7efc47400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc480af830 0x7efc4814b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35580/0 (socket says 172.21.15.159:35580) 2024-04-04T06:15:08.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.905+0000 7efc47400640 1 -- 172.21.15.159:0/3169783485 learned_addr learned my addr 172.21.15.159:0/3169783485 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:08.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.905+0000 7efc47e00640 1 --2- 172.21.15.159:0/3169783485 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc4815c390 0x7efc4814e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:08.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.905+0000 7efc47400640 1 -- 172.21.15.159:0/3169783485 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc4815c390 msgr2=0x7efc4814e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:08.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.905+0000 7efc47400640 1 --2- 172.21.15.159:0/3169783485 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc4815c390 0x7efc4814e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:08.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.905+0000 7efc3fe00640 1 --2- 172.21.15.159:0/3169783485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc480d0650 0x7efc48150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:08.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.905+0000 7efc47400640 1 -- 172.21.15.159:0/3169783485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc480d0650 msgr2=0x7efc48150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:08.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.905+0000 7efc47400640 1 --2- 172.21.15.159:0/3169783485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc480d0650 0x7efc48150650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:08.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.905+0000 7efc47400640 1 -- 172.21.15.159:0/3169783485 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4814f5b0 con 0x7efc480af830 2024-04-04T06:15:08.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.905+0000 7efc47400640 1 --2- 172.21.15.159:0/3169783485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc480af830 0x7efc4814b930 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7efc3800b410 tx=0x7efc3800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:08.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.906+0000 7efc3f400640 1 -- 172.21.15.159:0/3169783485 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc38015020 con 0x7efc480af830 2024-04-04T06:15:08.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.906+0000 7efc3f400640 1 -- 172.21.15.159:0/3169783485 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc38011830 con 0x7efc480af830 2024-04-04T06:15:08.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.906+0000 7efc3f400640 1 -- 172.21.15.159:0/3169783485 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc38011c20 con 0x7efc480af830 2024-04-04T06:15:08.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.906+0000 7efc4e600640 1 -- 172.21.15.159:0/3169783485 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4814f890 con 0x7efc480af830 2024-04-04T06:15:08.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.906+0000 7efc4e600640 1 -- 172.21.15.159:0/3169783485 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc4814fe50 con 0x7efc480af830 2024-04-04T06:15:08.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.907+0000 7efc4e600640 1 -- 172.21.15.159:0/3169783485 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc480685c0 con 0x7efc480af830 2024-04-04T06:15:08.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.908+0000 7efc3f400640 1 -- 172.21.15.159:0/3169783485 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc38018020 con 0x7efc480af830 2024-04-04T06:15:08.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.909+0000 7efc3f400640 1 --2- 172.21.15.159:0/3169783485 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efc2406c8f0 0x7efc2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:08.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.909+0000 7efc3f400640 1 -- 172.21.15.159:0/3169783485 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7efc3808bb20 con 0x7efc480af830 2024-04-04T06:15:08.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.910+0000 7efc3fe00640 1 --2- 172.21.15.159:0/3169783485 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efc2406c8f0 0x7efc2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:08.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.911+0000 7efc3fe00640 1 --2- 172.21.15.159:0/3169783485 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efc2406c8f0 0x7efc2406edb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7efc48158820 tx=0x7efc30008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:08.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:08.914+0000 7efc3f400640 1 -- 172.21.15.159:0/3169783485 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7efc3801f020 con 0x7efc480af830 2024-04-04T06:15:09.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.100+0000 7efc4e600640 1 -- 172.21.15.159:0/3169783485 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc48071f80 con 0x7efc480af830 2024-04-04T06:15:09.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.102+0000 7efc3f400640 1 -- 172.21.15.159:0/3169783485 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16636 (secure 0 0 0) 0x7efc380172c0 con 0x7efc480af830 2024-04-04T06:15:09.096 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:15:09.096 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":100,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:15:07.704490+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:15:05.537357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-04T06:15:06.689304+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:15:09.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.104+0000 7efc4e600640 1 -- 172.21.15.159:0/3169783485 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efc2406c8f0 msgr2=0x7efc2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:09.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.104+0000 7efc4e600640 1 --2- 172.21.15.159:0/3169783485 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efc2406c8f0 0x7efc2406edb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7efc48158820 tx=0x7efc30008040 comp rx=0 tx=0).stop 2024-04-04T06:15:09.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.104+0000 7efc4e600640 1 -- 172.21.15.159:0/3169783485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc480af830 msgr2=0x7efc4814b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:09.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.104+0000 7efc4e600640 1 --2- 172.21.15.159:0/3169783485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc480af830 0x7efc4814b930 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7efc3800b410 tx=0x7efc3800b8d0 comp rx=0 tx=0).stop 2024-04-04T06:15:09.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.104+0000 7efc4e600640 1 -- 172.21.15.159:0/3169783485 shutdown_connections 2024-04-04T06:15:09.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.104+0000 7efc4e600640 1 --2- 172.21.15.159:0/3169783485 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc4815c390 0x7efc4814e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:09.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.104+0000 7efc4e600640 1 --2- 172.21.15.159:0/3169783485 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efc2406c8f0 0x7efc2406edb0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:09.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.104+0000 7efc4e600640 1 --2- 172.21.15.159:0/3169783485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc480d0650 0x7efc48150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:09.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.104+0000 7efc4e600640 1 --2- 172.21.15.159:0/3169783485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc480af830 0x7efc4814b930 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:09.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.105+0000 7efc4e600640 1 -- 172.21.15.159:0/3169783485 >> 172.21.15.159:0/3169783485 conn(0x7efc4806b990 msgr2=0x7efc480735a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:09.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.105+0000 7efc4e600640 1 -- 172.21.15.159:0/3169783485 shutdown_connections 2024-04-04T06:15:09.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.105+0000 7efc4e600640 1 -- 172.21.15.159:0/3169783485 wait complete. 2024-04-04T06:15:09.117 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-04T06:15:09.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.330+0000 7f1292400640 1 Processor -- start 2024-04-04T06:15:09.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.330+0000 7f1292400640 1 -- start start 2024-04-04T06:15:09.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.331+0000 7f1292400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c153580 0x7f128c0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:09.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.331+0000 7f1292400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f128c0d07c0 0x7f128c0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:09.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.331+0000 7f1292400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f128c0afab0 con 0x7f128c0af2e0 2024-04-04T06:15:09.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.331+0000 7f1292400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f128c0d0c00 con 0x7f128c0d07c0 2024-04-04T06:15:09.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.331+0000 7f1292400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f128c15cc80 con 0x7f128c153580 2024-04-04T06:15:09.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.332+0000 7f128aa00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f128c0d07c0 0x7f128c0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:09.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.332+0000 7f128aa00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f128c0d07c0 0x7f128c0aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35584/0 (socket says 172.21.15.159:35584) 2024-04-04T06:15:09.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.332+0000 7f128aa00640 1 -- 172.21.15.159:0/1326978490 learned_addr learned my addr 172.21.15.159:0/1326978490 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:09.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.332+0000 7f128b400640 1 --2- 172.21.15.159:0/1326978490 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c153580 0x7f128c0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:09.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.332+0000 7f128aa00640 1 -- 172.21.15.159:0/1326978490 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c153580 msgr2=0x7f128c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:09.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.332+0000 7f128aa00640 1 --2- 172.21.15.159:0/1326978490 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c153580 0x7f128c0d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:09.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.332+0000 7f128aa00640 1 -- 172.21.15.159:0/1326978490 >> v1:172.21.15.159:6789/0 conn(0x7f128c0af2e0 legacy=0x7f128c06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:09.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.332+0000 7f128aa00640 1 -- 172.21.15.159:0/1326978490 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128c15da00 con 0x7f128c0d07c0 2024-04-04T06:15:09.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.332+0000 7f128a000640 1 -- 172.21.15.159:0/1326978490 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4263012769 0 0) 0x7f128c0afab0 con 0x7f128c0af2e0 2024-04-04T06:15:09.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.334+0000 7f128aa00640 1 --2- 172.21.15.159:0/1326978490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f128c0d07c0 0x7f128c0aeda0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f1278009870 tx=0x7f127802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bc6ab69d15fb11 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:09.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.334+0000 7f128a000640 1 -- 172.21.15.159:0/1326978490 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278035020 con 0x7f128c0d07c0 2024-04-04T06:15:09.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.334+0000 7f128a000640 1 -- 172.21.15.159:0/1326978490 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1278031640 con 0x7f128c0d07c0 2024-04-04T06:15:09.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.334+0000 7f128a000640 1 -- 172.21.15.159:0/1326978490 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127802d040 con 0x7f128c0d07c0 2024-04-04T06:15:09.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.334+0000 7f1292400640 1 -- 172.21.15.159:0/1326978490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f128c0d07c0 msgr2=0x7f128c0aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:09.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.334+0000 7f1292400640 1 --2- 172.21.15.159:0/1326978490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f128c0d07c0 0x7f128c0aeda0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f1278009870 tx=0x7f127802b8f0 comp rx=0 tx=0).stop 2024-04-04T06:15:09.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.334+0000 7f1292400640 1 -- 172.21.15.159:0/1326978490 shutdown_connections 2024-04-04T06:15:09.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.334+0000 7f1292400640 1 --2- 172.21.15.159:0/1326978490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f128c0d07c0 0x7f128c0aeda0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:09.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.334+0000 7f1292400640 1 --2- 172.21.15.159:0/1326978490 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c153580 0x7f128c0d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:09.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.334+0000 7f1292400640 1 -- 172.21.15.159:0/1326978490 >> 172.21.15.159:0/1326978490 conn(0x7f128c06b750 msgr2=0x7f128c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:09.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.335+0000 7f1292400640 1 -- 172.21.15.159:0/1326978490 shutdown_connections 2024-04-04T06:15:09.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.335+0000 7f1292400640 1 -- 172.21.15.159:0/1326978490 wait complete. 2024-04-04T06:15:09.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.336+0000 7f1292400640 1 Processor -- start 2024-04-04T06:15:09.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.336+0000 7f1292400640 1 -- start start 2024-04-04T06:15:09.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.337+0000 7f1292400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f128c0af2e0 0x7f128c159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:09.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.337+0000 7f1292400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f128c0d07c0 0x7f128c0cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:09.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.337+0000 7f1292400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c153580 0x7f128c0cf110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:09.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.337+0000 7f1292400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f128c15d4e0 con 0x7f128c0d07c0 2024-04-04T06:15:09.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.337+0000 7f1292400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f128c15d200 con 0x7f128c0af2e0 2024-04-04T06:15:09.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.337+0000 7f1292400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f128c153960 con 0x7f128c153580 2024-04-04T06:15:09.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.338+0000 7f128aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f128c0d07c0 0x7f128c0cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:09.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.338+0000 7f128aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f128c0d07c0 0x7f128c0cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35596/0 (socket says 172.21.15.159:35596) 2024-04-04T06:15:09.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.338+0000 7f128aa00640 1 -- 172.21.15.159:0/2513994974 learned_addr learned my addr 172.21.15.159:0/2513994974 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:09.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.338+0000 7f128b400640 1 --2- 172.21.15.159:0/2513994974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f128c0af2e0 0x7f128c159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:09.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.338+0000 7f128aa00640 1 -- 172.21.15.159:0/2513994974 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c153580 msgr2=0x7f128c0cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:09.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.338+0000 7f128aa00640 1 --2- 172.21.15.159:0/2513994974 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c153580 0x7f128c0cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:09.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.338+0000 7f128aa00640 1 -- 172.21.15.159:0/2513994974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f128c0af2e0 msgr2=0x7f128c159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:09.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.338+0000 7f128aa00640 1 --2- 172.21.15.159:0/2513994974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f128c0af2e0 0x7f128c159880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:09.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.338+0000 7f128aa00640 1 -- 172.21.15.159:0/2513994974 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128c1f4240 con 0x7f128c0d07c0 2024-04-04T06:15:09.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.338+0000 7f128aa00640 1 --2- 172.21.15.159:0/2513994974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f128c0d07c0 0x7f128c0cebd0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f12780090b0 tx=0x7f12780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:09.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.339+0000 7f1283400640 1 -- 172.21.15.159:0/2513994974 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278040070 con 0x7f128c0d07c0 2024-04-04T06:15:09.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.339+0000 7f1283400640 1 -- 172.21.15.159:0/2513994974 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12780399f0 con 0x7f128c0d07c0 2024-04-04T06:15:09.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.339+0000 7f1292400640 1 -- 172.21.15.159:0/2513994974 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f128c159dc0 con 0x7f128c0d07c0 2024-04-04T06:15:09.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.339+0000 7f1283400640 1 -- 172.21.15.159:0/2513994974 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278035040 con 0x7f128c0d07c0 2024-04-04T06:15:09.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.339+0000 7f1292400640 1 -- 172.21.15.159:0/2513994974 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f128c0c9ff0 con 0x7f128c0d07c0 2024-04-04T06:15:09.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.340+0000 7f1292400640 1 -- 172.21.15.159:0/2513994974 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f128c0685c0 con 0x7f128c0d07c0 2024-04-04T06:15:09.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.342+0000 7f1283400640 1 -- 172.21.15.159:0/2513994974 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1278038020 con 0x7f128c0d07c0 2024-04-04T06:15:09.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.343+0000 7f1283400640 1 --2- 172.21.15.159:0/2513994974 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f126c06c8f0 0x7f126c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:09.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.343+0000 7f1283400640 1 -- 172.21.15.159:0/2513994974 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f12780ad500 con 0x7f128c0d07c0 2024-04-04T06:15:09.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.343+0000 7f128b400640 1 --2- 172.21.15.159:0/2513994974 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f126c06c8f0 0x7f126c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:09.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.344+0000 7f128b400640 1 --2- 172.21.15.159:0/2513994974 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f126c06c8f0 0x7f126c06edb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f127c006fd0 tx=0x7f127c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:09.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.348+0000 7f1283400640 1 -- 172.21.15.159:0/2513994974 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f127807b930 con 0x7f128c0d07c0 2024-04-04T06:15:09.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:09.586+0000 7f1292400640 1 -- 172.21.15.159:0/2513994974 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f128c06ec40 con 0x7f128c0d07c0 2024-04-04T06:15:10.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.277+0000 7f1283400640 1 -- 172.21.15.159:0/2513994974 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v83) ==== 118+0+0 (secure 0 0 0) 0x7f127807b230 con 0x7f128c0d07c0 2024-04-04T06:15:10.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.281+0000 7f1292400640 1 -- 172.21.15.159:0/2513994974 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f126c06c8f0 msgr2=0x7f126c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:10.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.281+0000 7f1292400640 1 --2- 172.21.15.159:0/2513994974 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f126c06c8f0 0x7f126c06edb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f127c006fd0 tx=0x7f127c008040 comp rx=0 tx=0).stop 2024-04-04T06:15:10.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.281+0000 7f1292400640 1 -- 172.21.15.159:0/2513994974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f128c0d07c0 msgr2=0x7f128c0cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:10.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.281+0000 7f1292400640 1 --2- 172.21.15.159:0/2513994974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f128c0d07c0 0x7f128c0cebd0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f12780090b0 tx=0x7f12780046c0 comp rx=0 tx=0).stop 2024-04-04T06:15:10.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.281+0000 7f1292400640 1 -- 172.21.15.159:0/2513994974 shutdown_connections 2024-04-04T06:15:10.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.281+0000 7f1292400640 1 --2- 172.21.15.159:0/2513994974 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f128c153580 0x7f128c0cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:10.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.281+0000 7f1292400640 1 --2- 172.21.15.159:0/2513994974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f128c0d07c0 0x7f128c0cebd0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:10.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.281+0000 7f1292400640 1 --2- 172.21.15.159:0/2513994974 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f126c06c8f0 0x7f126c06edb0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:10.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.281+0000 7f1292400640 1 --2- 172.21.15.159:0/2513994974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f128c0af2e0 0x7f128c159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:10.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.281+0000 7f1292400640 1 -- 172.21.15.159:0/2513994974 >> 172.21.15.159:0/2513994974 conn(0x7f128c06b750 msgr2=0x7f128c06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:10.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.281+0000 7f1292400640 1 -- 172.21.15.159:0/2513994974 shutdown_connections 2024-04-04T06:15:10.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.282+0000 7f1292400640 1 -- 172.21.15.159:0/2513994974 wait complete. 2024-04-04T06:15:10.300 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-04T06:15:10.301 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-04T06:15:10.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.548+0000 7f9ea3200640 1 Processor -- start 2024-04-04T06:15:10.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.548+0000 7f9ea3200640 1 -- start start 2024-04-04T06:15:10.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.549+0000 7f9ea3200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9ea415f6a0 0x7f9ea40b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:10.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.549+0000 7f9ea3200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ea4072c80 0x7f9ea40b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:10.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.549+0000 7f9ea3200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ea40b4c70 con 0x7f9ea40b4890 2024-04-04T06:15:10.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.549+0000 7f9ea3200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f9ea415fb30 con 0x7f9ea415f6a0 2024-04-04T06:15:10.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.549+0000 7f9ea3200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9ea415b7b0 con 0x7f9ea4072c80 2024-04-04T06:15:10.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.550+0000 7f9ea1e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f9ea40b4890 0x7f9ea4072570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:42968/0 (socket says 172.21.15.159:42968) 2024-04-04T06:15:10.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.550+0000 7f9ea1e00640 1 -- 172.21.15.159:0/3745928858 learned_addr learned my addr 172.21.15.159:0/3745928858 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:10.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.550+0000 7f9e99400640 1 --2- 172.21.15.159:0/3745928858 >> v2:172.21.15.159:3300/0 conn(0x7f9ea415f6a0 0x7f9ea40b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:10.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.550+0000 7f9ea2800640 1 --2- 172.21.15.159:0/3745928858 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ea4072c80 0x7f9ea40b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:10.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.550+0000 7f9ea1400640 1 -- 172.21.15.159:0/3745928858 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 7368608 0 0) 0x7f9ea40b4c70 con 0x7f9ea40b4890 2024-04-04T06:15:10.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.550+0000 7f9ea1400640 1 -- 172.21.15.159:0/3745928858 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e88003520 con 0x7f9ea40b4890 2024-04-04T06:15:10.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.550+0000 7f9e99400640 1 -- 172.21.15.159:0/3745928858 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ea4072c80 msgr2=0x7f9ea40b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:10.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.550+0000 7f9e99400640 1 --2- 172.21.15.159:0/3745928858 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ea4072c80 0x7f9ea40b33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:10.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.550+0000 7f9e99400640 1 -- 172.21.15.159:0/3745928858 >> v1:172.21.15.159:6789/0 conn(0x7f9ea40b4890 legacy=0x7f9ea4072570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:10.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.550+0000 7f9e99400640 1 -- 172.21.15.159:0/3745928858 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea415ba90 con 0x7f9ea415f6a0 2024-04-04T06:15:10.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.550+0000 7f9ea2800640 1 --2- 172.21.15.159:0/3745928858 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ea4072c80 0x7f9ea40b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:10.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.551+0000 7f9e99400640 1 --2- 172.21.15.159:0/3745928858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ea415f6a0 0x7f9ea40b2ea0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0097a0 tx=0x7f9e8c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd649c3fbb0cb9f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:10.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.552+0000 7f9ea1400640 1 -- 172.21.15.159:0/3745928858 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c035020 con 0x7f9ea415f6a0 2024-04-04T06:15:10.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.552+0000 7f9ea1400640 1 -- 172.21.15.159:0/3745928858 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e8c031810 con 0x7f9ea415f6a0 2024-04-04T06:15:10.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.552+0000 7f9ea1400640 1 -- 172.21.15.159:0/3745928858 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c031be0 con 0x7f9ea415f6a0 2024-04-04T06:15:10.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.552+0000 7f9ea3200640 1 -- 172.21.15.159:0/3745928858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ea415f6a0 msgr2=0x7f9ea40b2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:10.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.552+0000 7f9ea3200640 1 --2- 172.21.15.159:0/3745928858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ea415f6a0 0x7f9ea40b2ea0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0097a0 tx=0x7f9e8c02b8b0 comp rx=0 tx=0).stop 2024-04-04T06:15:10.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.552+0000 7f9ea3200640 1 -- 172.21.15.159:0/3745928858 shutdown_connections 2024-04-04T06:15:10.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.552+0000 7f9ea3200640 1 --2- 172.21.15.159:0/3745928858 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ea4072c80 0x7f9ea40b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:10.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.552+0000 7f9ea3200640 1 --2- 172.21.15.159:0/3745928858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ea415f6a0 0x7f9ea40b2ea0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:10.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.552+0000 7f9ea3200640 1 -- 172.21.15.159:0/3745928858 >> 172.21.15.159:0/3745928858 conn(0x7f9ea40b1ba0 msgr2=0x7f9ea40b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:10.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.553+0000 7f9ea3200640 1 -- 172.21.15.159:0/3745928858 shutdown_connections 2024-04-04T06:15:10.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.553+0000 7f9ea3200640 1 -- 172.21.15.159:0/3745928858 wait complete. 2024-04-04T06:15:10.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.554+0000 7f9ea3200640 1 Processor -- start 2024-04-04T06:15:10.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.554+0000 7f9ea3200640 1 -- start start 2024-04-04T06:15:10.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.555+0000 7f9ea3200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ea4072c80 0x7f9ea4158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:10.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.555+0000 7f9ea3200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ea40b4890 0x7f9ea40ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:10.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.555+0000 7f9ea3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ea415a140 0x7f9ea40ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:10.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.555+0000 7f9ea3200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea415c570 con 0x7f9ea415a140 2024-04-04T06:15:10.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.555+0000 7f9ea3200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea415c050 con 0x7f9ea40b4890 2024-04-04T06:15:10.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.555+0000 7f9ea3200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9ea40b2900 con 0x7f9ea4072c80 2024-04-04T06:15:10.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.556+0000 7f9ea2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ea415a140 0x7f9ea40ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:10.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.556+0000 7f9ea2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ea415a140 0x7f9ea40ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35608/0 (socket says 172.21.15.159:35608) 2024-04-04T06:15:10.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.556+0000 7f9ea2800640 1 -- 172.21.15.159:0/3487342278 learned_addr learned my addr 172.21.15.159:0/3487342278 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:10.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.556+0000 7f9e99400640 1 --2- 172.21.15.159:0/3487342278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ea40b4890 0x7f9ea40ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:10.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.556+0000 7f9ea1e00640 1 --2- 172.21.15.159:0/3487342278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ea4072c80 0x7f9ea4158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:10.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.556+0000 7f9ea2800640 1 -- 172.21.15.159:0/3487342278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ea4072c80 msgr2=0x7f9ea4158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:10.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.556+0000 7f9ea2800640 1 --2- 172.21.15.159:0/3487342278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ea4072c80 0x7f9ea4158c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:10.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.556+0000 7f9ea2800640 1 -- 172.21.15.159:0/3487342278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ea40b4890 msgr2=0x7f9ea40ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:10.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.556+0000 7f9ea2800640 1 --2- 172.21.15.159:0/3487342278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ea40b4890 0x7f9ea40ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:10.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.556+0000 7f9ea2800640 1 -- 172.21.15.159:0/3487342278 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea40cefc0 con 0x7f9ea415a140 2024-04-04T06:15:10.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.556+0000 7f9e99400640 1 --2- 172.21.15.159:0/3487342278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ea40b4890 0x7f9ea40ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:10.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.556+0000 7f9ea2800640 1 --2- 172.21.15.159:0/3487342278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ea415a140 0x7f9ea40ad890 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00cd80 tx=0x7f9e9c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:10.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.557+0000 7f9e9aa00640 1 -- 172.21.15.159:0/3487342278 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00f680 con 0x7f9ea415a140 2024-04-04T06:15:10.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.557+0000 7f9e9aa00640 1 -- 172.21.15.159:0/3487342278 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e9c00f840 con 0x7f9ea415a140 2024-04-04T06:15:10.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.557+0000 7f9e9aa00640 1 -- 172.21.15.159:0/3487342278 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00fc30 con 0x7f9ea415a140 2024-04-04T06:15:10.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.557+0000 7f9ea3200640 1 -- 172.21.15.159:0/3487342278 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea40cf2a0 con 0x7f9ea415a140 2024-04-04T06:15:10.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.557+0000 7f9ea3200640 1 -- 172.21.15.159:0/3487342278 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea4158980 con 0x7f9ea415a140 2024-04-04T06:15:10.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.558+0000 7f9ea3200640 1 -- 172.21.15.159:0/3487342278 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea4075940 con 0x7f9ea415a140 2024-04-04T06:15:10.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.559+0000 7f9e9aa00640 1 -- 172.21.15.159:0/3487342278 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e9c0223f0 con 0x7f9ea415a140 2024-04-04T06:15:10.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.560+0000 7f9e9aa00640 1 --2- 172.21.15.159:0/3487342278 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9e8006c8f0 0x7f9e8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:10.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.560+0000 7f9e9aa00640 1 -- 172.21.15.159:0/3487342278 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f9e9c08c860 con 0x7f9ea415a140 2024-04-04T06:15:10.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.561+0000 7f9ea1e00640 1 --2- 172.21.15.159:0/3487342278 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9e8006c8f0 0x7f9e8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:10.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.562+0000 7f9ea1e00640 1 --2- 172.21.15.159:0/3487342278 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9e8006c8f0 0x7f9e8006edb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9e94000f60 tx=0x7f9e94009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:10.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.565+0000 7f9e9aa00640 1 -- 172.21.15.159:0/3487342278 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9e9c014030 con 0x7f9ea415a140 2024-04-04T06:15:10.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:10.809+0000 7f9ea3200640 1 -- 172.21.15.159:0/3487342278 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9ea414f300 con 0x7f9ea415a140 2024-04-04T06:15:11.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.288+0000 7f9e9aa00640 1 -- 172.21.15.159:0/3487342278 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v84) ==== 122+0+0 (secure 0 0 0) 0x7f9e9c05ad00 con 0x7f9ea415a140 2024-04-04T06:15:11.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.293+0000 7f9ea3200640 1 -- 172.21.15.159:0/3487342278 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9e8006c8f0 msgr2=0x7f9e8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:11.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.293+0000 7f9ea3200640 1 --2- 172.21.15.159:0/3487342278 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9e8006c8f0 0x7f9e8006edb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9e94000f60 tx=0x7f9e94009040 comp rx=0 tx=0).stop 2024-04-04T06:15:11.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.293+0000 7f9ea3200640 1 -- 172.21.15.159:0/3487342278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ea415a140 msgr2=0x7f9ea40ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:11.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.293+0000 7f9ea3200640 1 --2- 172.21.15.159:0/3487342278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ea415a140 0x7f9ea40ad890 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00cd80 tx=0x7f9e9c00a650 comp rx=0 tx=0).stop 2024-04-04T06:15:11.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.293+0000 7f9ea3200640 1 -- 172.21.15.159:0/3487342278 shutdown_connections 2024-04-04T06:15:11.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.293+0000 7f9ea3200640 1 --2- 172.21.15.159:0/3487342278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ea415a140 0x7f9ea40ad890 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:11.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.293+0000 7f9ea3200640 1 --2- 172.21.15.159:0/3487342278 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9e8006c8f0 0x7f9e8006edb0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:11.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.293+0000 7f9ea3200640 1 --2- 172.21.15.159:0/3487342278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ea40b4890 0x7f9ea40ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:11.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.293+0000 7f9ea3200640 1 --2- 172.21.15.159:0/3487342278 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ea4072c80 0x7f9ea4158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:11.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.293+0000 7f9ea3200640 1 -- 172.21.15.159:0/3487342278 >> 172.21.15.159:0/3487342278 conn(0x7f9ea40b1ba0 msgr2=0x7f9ea4155580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:11.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.293+0000 7f9ea3200640 1 -- 172.21.15.159:0/3487342278 shutdown_connections 2024-04-04T06:15:11.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.293+0000 7f9ea3200640 1 -- 172.21.15.159:0/3487342278 wait complete. 2024-04-04T06:15:11.309 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-04T06:15:11.309 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:15:11.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.513+0000 7f97cda00640 1 Processor -- start 2024-04-04T06:15:11.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.513+0000 7f97cda00640 1 -- start start 2024-04-04T06:15:11.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.515+0000 7f97cda00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f97c815aa90 0x7f97c815b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:11.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.515+0000 7f97cda00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f97c80717f0 0x7f97c80b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:11.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.515+0000 7f97cda00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c8072100 0x7f97c8072550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:11.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.515+0000 7f97cda00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f97c80b5190 con 0x7f97c80717f0 2024-04-04T06:15:11.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.515+0000 7f97cda00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f97c8072a90 con 0x7f97c815aa90 2024-04-04T06:15:11.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.515+0000 7f97cda00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f97c80b4240 con 0x7f97c8072100 2024-04-04T06:15:11.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.515+0000 7f97c6000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f97c80717f0 0x7f97c80b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:11.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.515+0000 7f97c6000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f97c80717f0 0x7f97c80b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35616/0 (socket says 172.21.15.159:35616) 2024-04-04T06:15:11.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.515+0000 7f97c6000640 1 -- 172.21.15.159:0/20391891 learned_addr learned my addr 172.21.15.159:0/20391891 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:11.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.515+0000 7f97c6a00640 1 --2- 172.21.15.159:0/20391891 >> v2:172.21.15.178:3300/0 conn(0x7f97c815aa90 0x7f97c815b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:11.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.515+0000 7f97c6000640 1 -- 172.21.15.159:0/20391891 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c8072100 msgr2=0x7f97c8072550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:11.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.515+0000 7f97c7400640 1 --2- 172.21.15.159:0/20391891 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c8072100 0x7f97c8072550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:11.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.516+0000 7f97c6000640 1 --2- 172.21.15.159:0/20391891 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c8072100 0x7f97c8072550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:11.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.516+0000 7f97c6000640 1 -- 172.21.15.159:0/20391891 >> v2:172.21.15.178:3300/0 conn(0x7f97c815aa90 msgr2=0x7f97c815b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:11.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.516+0000 7f97c6000640 1 --2- 172.21.15.159:0/20391891 >> v2:172.21.15.178:3300/0 conn(0x7f97c815aa90 0x7f97c815b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:11.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.516+0000 7f97c6000640 1 -- 172.21.15.159:0/20391891 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c815bee0 con 0x7f97c80717f0 2024-04-04T06:15:11.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.516+0000 7f97c7400640 1 --2- 172.21.15.159:0/20391891 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c8072100 0x7f97c8072550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:11.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.517+0000 7f97c6000640 1 --2- 172.21.15.159:0/20391891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97c80717f0 0x7f97c80b46d0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f97c8071200 tx=0x7f97b802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1dba51f4f19c56 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:11.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.517+0000 7f97c5600640 1 -- 172.21.15.159:0/20391891 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8035020 con 0x7f97c80717f0 2024-04-04T06:15:11.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.517+0000 7f97c5600640 1 -- 172.21.15.159:0/20391891 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b802d040 con 0x7f97c80717f0 2024-04-04T06:15:11.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.517+0000 7f97c5600640 1 -- 172.21.15.159:0/20391891 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8031a80 con 0x7f97c80717f0 2024-04-04T06:15:11.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.518+0000 7f97cda00640 1 -- 172.21.15.159:0/20391891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97c80717f0 msgr2=0x7f97c80b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:11.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.518+0000 7f97cda00640 1 --2- 172.21.15.159:0/20391891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97c80717f0 0x7f97c80b46d0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f97c8071200 tx=0x7f97b802b900 comp rx=0 tx=0).stop 2024-04-04T06:15:11.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.518+0000 7f97cda00640 1 -- 172.21.15.159:0/20391891 shutdown_connections 2024-04-04T06:15:11.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.518+0000 7f97cda00640 1 --2- 172.21.15.159:0/20391891 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c8072100 0x7f97c8072550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:11.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.518+0000 7f97cda00640 1 --2- 172.21.15.159:0/20391891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97c80717f0 0x7f97c80b46d0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:11.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.518+0000 7f97cda00640 1 --2- 172.21.15.159:0/20391891 >> v2:172.21.15.178:3300/0 conn(0x7f97c815aa90 0x7f97c815b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:11.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.518+0000 7f97cda00640 1 -- 172.21.15.159:0/20391891 >> 172.21.15.159:0/20391891 conn(0x7f97c806b990 msgr2=0x7f97c806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:11.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.518+0000 7f97cda00640 1 -- 172.21.15.159:0/20391891 shutdown_connections 2024-04-04T06:15:11.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.518+0000 7f97cda00640 1 -- 172.21.15.159:0/20391891 wait complete. 2024-04-04T06:15:11.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.521+0000 7f97cda00640 1 Processor -- start 2024-04-04T06:15:11.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.521+0000 7f97cda00640 1 -- start start 2024-04-04T06:15:11.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.523+0000 7f97cda00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97c80717f0 0x7f97c814e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:11.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.523+0000 7f97cda00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c8072100 0x7f97c80cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.523+0000 7f97cda00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97c815aa90 0x7f97c80ad3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.523+0000 7f97cda00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f97c80b3ed0 con 0x7f97c815aa90 2024-04-04T06:15:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.523+0000 7f97cda00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f97c80b3bf0 con 0x7f97c80717f0 2024-04-04T06:15:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.523+0000 7f97cda00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f97c80b4c10 con 0x7f97c8072100 2024-04-04T06:15:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.524+0000 7f97c7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97c815aa90 0x7f97c80ad3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.524+0000 7f97c7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97c815aa90 0x7f97c80ad3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35622/0 (socket says 172.21.15.159:35622) 2024-04-04T06:15:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.524+0000 7f97c7400640 1 -- 172.21.15.159:0/3845598801 learned_addr learned my addr 172.21.15.159:0/3845598801 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.524+0000 7f97c6a00640 1 --2- 172.21.15.159:0/3845598801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97c80717f0 0x7f97c814e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.524+0000 7f97c6000640 1 --2- 172.21.15.159:0/3845598801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c8072100 0x7f97c80cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.524+0000 7f97c7400640 1 -- 172.21.15.159:0/3845598801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c8072100 msgr2=0x7f97c80cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.524+0000 7f97c7400640 1 --2- 172.21.15.159:0/3845598801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c8072100 0x7f97c80cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.524+0000 7f97c7400640 1 -- 172.21.15.159:0/3845598801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97c80717f0 msgr2=0x7f97c814e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:11.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.524+0000 7f97c7400640 1 --2- 172.21.15.159:0/3845598801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97c80717f0 0x7f97c814e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:11.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.524+0000 7f97c7400640 1 -- 172.21.15.159:0/3845598801 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c80ade30 con 0x7f97c815aa90 2024-04-04T06:15:11.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.524+0000 7f97c6000640 1 --2- 172.21.15.159:0/3845598801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c8072100 0x7f97c80cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:11.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.524+0000 7f97c7400640 1 --2- 172.21.15.159:0/3845598801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97c815aa90 0x7f97c80ad3e0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f97c000cef0 tx=0x7f97c000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:11.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.525+0000 7f97bea00640 1 -- 172.21.15.159:0/3845598801 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c001a070 con 0x7f97c815aa90 2024-04-04T06:15:11.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.525+0000 7f97bea00640 1 -- 172.21.15.159:0/3845598801 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97c000d840 con 0x7f97c815aa90 2024-04-04T06:15:11.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.525+0000 7f97cda00640 1 -- 172.21.15.159:0/3845598801 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c80ae110 con 0x7f97c815aa90 2024-04-04T06:15:11.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.525+0000 7f97bea00640 1 -- 172.21.15.159:0/3845598801 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c000dc30 con 0x7f97c815aa90 2024-04-04T06:15:11.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.525+0000 7f97cda00640 1 -- 172.21.15.159:0/3845598801 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c814ec40 con 0x7f97c815aa90 2024-04-04T06:15:11.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.526+0000 7f97cda00640 1 -- 172.21.15.159:0/3845598801 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97c80685c0 con 0x7f97c815aa90 2024-04-04T06:15:11.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.528+0000 7f97bea00640 1 -- 172.21.15.159:0/3845598801 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f97c001d070 con 0x7f97c815aa90 2024-04-04T06:15:11.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.529+0000 7f97bea00640 1 --2- 172.21.15.159:0/3845598801 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f97a406c8f0 0x7f97a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:11.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.529+0000 7f97bea00640 1 -- 172.21.15.159:0/3845598801 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f97c008c3b0 con 0x7f97c815aa90 2024-04-04T06:15:11.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.530+0000 7f97c6a00640 1 --2- 172.21.15.159:0/3845598801 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f97a406c8f0 0x7f97a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:11.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.531+0000 7f97c6a00640 1 --2- 172.21.15.159:0/3845598801 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f97a406c8f0 0x7f97a406edb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f97b0006fd0 tx=0x7f97b0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:11.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.534+0000 7f97bea00640 1 -- 172.21.15.159:0/3845598801 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f97c005a7d0 con 0x7f97c815aa90 2024-04-04T06:15:11.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.816+0000 7f97cda00640 1 -- 172.21.15.159:0/3845598801 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97c80cfc30 con 0x7f97c815aa90 2024-04-04T06:15:11.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.817+0000 7f97bea00640 1 -- 172.21.15.159:0/3845598801 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5189 (secure 0 0 0) 0x7f97c005a0d0 con 0x7f97c815aa90 2024-04-04T06:15:11.812 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:15:11.812 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":84,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5546,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/40219476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":40219476},{"type":"v1","addr":"172.21.15.178:6837","nonce":40219476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/4060483382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4060483382},{"type":"v1","addr":"172.21.15.178:6836","nonce":4060483382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8187,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1945665739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1945665739},{"type":"v1","addr":"172.21.15.159:6833","nonce":1945665739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:15:07.704835+0000","modified":"2024-04-04T06:15:10.811654+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8190":{"gid":8190,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6835/2253383212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2253383212},{"type":"v1","addr":"172.21.15.159:6835","nonce":2253383212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8190,"qdb_cluster":[8190]},"id":7}]} 2024-04-04T06:15:11.812 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 84 2024-04-04T06:15:11.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.820+0000 7f97cda00640 1 -- 172.21.15.159:0/3845598801 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f97a406c8f0 msgr2=0x7f97a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:11.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.820+0000 7f97cda00640 1 --2- 172.21.15.159:0/3845598801 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f97a406c8f0 0x7f97a406edb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f97b0006fd0 tx=0x7f97b0008040 comp rx=0 tx=0).stop 2024-04-04T06:15:11.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.820+0000 7f97cda00640 1 -- 172.21.15.159:0/3845598801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97c815aa90 msgr2=0x7f97c80ad3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:11.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.820+0000 7f97cda00640 1 --2- 172.21.15.159:0/3845598801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97c815aa90 0x7f97c80ad3e0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f97c000cef0 tx=0x7f97c000a710 comp rx=0 tx=0).stop 2024-04-04T06:15:11.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.820+0000 7f97cda00640 1 -- 172.21.15.159:0/3845598801 shutdown_connections 2024-04-04T06:15:11.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.820+0000 7f97cda00640 1 --2- 172.21.15.159:0/3845598801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f97c815aa90 0x7f97c80ad3e0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:11.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.820+0000 7f97cda00640 1 --2- 172.21.15.159:0/3845598801 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f97c8072100 0x7f97c80cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:11.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.820+0000 7f97cda00640 1 --2- 172.21.15.159:0/3845598801 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f97a406c8f0 0x7f97a406edb0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:11.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.820+0000 7f97cda00640 1 --2- 172.21.15.159:0/3845598801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97c80717f0 0x7f97c814e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:11.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.820+0000 7f97cda00640 1 -- 172.21.15.159:0/3845598801 >> 172.21.15.159:0/3845598801 conn(0x7f97c806b990 msgr2=0x7f97c815ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:11.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.821+0000 7f97cda00640 1 -- 172.21.15.159:0/3845598801 shutdown_connections 2024-04-04T06:15:11.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:11.821+0000 7f97cda00640 1 -- 172.21.15.159:0/3845598801 wait complete. 2024-04-04T06:15:11.834 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:15:12.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.047+0000 7ff574a00640 1 Processor -- start 2024-04-04T06:15:12.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.047+0000 7ff574a00640 1 -- start start 2024-04-04T06:15:12.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.048+0000 7ff574a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff5700af830 0x7ff570073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:12.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.048+0000 7ff574a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5700d0650 0x7ff5700d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:12.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.048+0000 7ff574a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff57015fe80 con 0x7ff57015c390 2024-04-04T06:15:12.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.048+0000 7ff574a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7ff5700d1000 con 0x7ff5700af830 2024-04-04T06:15:12.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.048+0000 7ff574a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff57015c770 con 0x7ff5700d0650 2024-04-04T06:15:12.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.049+0000 7ff565400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff5700af830 0x7ff570073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:12.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.049+0000 7ff56de00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7ff57015c390 0x7ff570160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:42984/0 (socket says 172.21.15.159:42984) 2024-04-04T06:15:12.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.049+0000 7ff56de00640 1 -- 172.21.15.159:0/453454316 learned_addr learned my addr 172.21.15.159:0/453454316 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:12.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.049+0000 7ff56e800640 1 --2- 172.21.15.159:0/453454316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5700d0650 0x7ff5700d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:12.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.049+0000 7ff565400640 1 -- 172.21.15.159:0/453454316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5700d0650 msgr2=0x7ff5700d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:12.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.049+0000 7ff565400640 1 --2- 172.21.15.159:0/453454316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5700d0650 0x7ff5700d0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.049+0000 7ff565400640 1 -- 172.21.15.159:0/453454316 >> v1:172.21.15.159:6789/0 conn(0x7ff57015c390 legacy=0x7ff570160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:12.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.049+0000 7ff565400640 1 -- 172.21.15.159:0/453454316 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5700ad120 con 0x7ff5700af830 2024-04-04T06:15:12.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.049+0000 7ff56d400640 1 -- 172.21.15.159:0/453454316 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 457862670 0 0) 0x7ff57015fe80 con 0x7ff57015c390 2024-04-04T06:15:12.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.050+0000 7ff565400640 1 --2- 172.21.15.159:0/453454316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5700af830 0x7ff570073790 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7ff5580097a0 tx=0x7ff55802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41234ff0661e34c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:12.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.051+0000 7ff56d400640 1 -- 172.21.15.159:0/453454316 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff558035020 con 0x7ff5700af830 2024-04-04T06:15:12.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.051+0000 7ff56d400640 1 -- 172.21.15.159:0/453454316 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff558031830 con 0x7ff5700af830 2024-04-04T06:15:12.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.051+0000 7ff56d400640 1 -- 172.21.15.159:0/453454316 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff558031c00 con 0x7ff5700af830 2024-04-04T06:15:12.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.052+0000 7ff574a00640 1 -- 172.21.15.159:0/453454316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5700af830 msgr2=0x7ff570073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:12.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.052+0000 7ff574a00640 1 --2- 172.21.15.159:0/453454316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5700af830 0x7ff570073790 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7ff5580097a0 tx=0x7ff55802b8d0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.052+0000 7ff574a00640 1 -- 172.21.15.159:0/453454316 shutdown_connections 2024-04-04T06:15:12.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.052+0000 7ff574a00640 1 --2- 172.21.15.159:0/453454316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5700d0650 0x7ff5700d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.052+0000 7ff574a00640 1 --2- 172.21.15.159:0/453454316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5700af830 0x7ff570073790 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.052+0000 7ff574a00640 1 -- 172.21.15.159:0/453454316 >> 172.21.15.159:0/453454316 conn(0x7ff57006b990 msgr2=0x7ff57006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:12.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.052+0000 7ff574a00640 1 -- 172.21.15.159:0/453454316 shutdown_connections 2024-04-04T06:15:12.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.052+0000 7ff574a00640 1 -- 172.21.15.159:0/453454316 wait complete. 2024-04-04T06:15:12.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.053+0000 7ff574a00640 1 Processor -- start 2024-04-04T06:15:12.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.054+0000 7ff574a00640 1 -- start start 2024-04-04T06:15:12.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.054+0000 7ff574a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5700af830 0x7ff5701f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:12.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.054+0000 7ff574a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5700d0650 0x7ff5701525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:12.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.054+0000 7ff574a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff57015c390 0x7ff5700b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:12.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.054+0000 7ff574a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff570160120 con 0x7ff5700af830 2024-04-04T06:15:12.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.054+0000 7ff574a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff5700ad6e0 con 0x7ff5700d0650 2024-04-04T06:15:12.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.054+0000 7ff574a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff570160d50 con 0x7ff57015c390 2024-04-04T06:15:12.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.055+0000 7ff56de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5700af830 0x7ff5701f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:12.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.055+0000 7ff56de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5700af830 0x7ff5701f4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35634/0 (socket says 172.21.15.159:35634) 2024-04-04T06:15:12.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.055+0000 7ff56de00640 1 -- 172.21.15.159:0/1544124985 learned_addr learned my addr 172.21.15.159:0/1544124985 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:12.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.055+0000 7ff56e800640 1 --2- 172.21.15.159:0/1544124985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff57015c390 0x7ff5700b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:12.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.055+0000 7ff565400640 1 --2- 172.21.15.159:0/1544124985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5700d0650 0x7ff5701525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:12.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.055+0000 7ff56de00640 1 -- 172.21.15.159:0/1544124985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff57015c390 msgr2=0x7ff5700b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:12.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.055+0000 7ff56de00640 1 --2- 172.21.15.159:0/1544124985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff57015c390 0x7ff5700b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.055+0000 7ff56de00640 1 -- 172.21.15.159:0/1544124985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5700d0650 msgr2=0x7ff5701525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:12.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.055+0000 7ff56de00640 1 --2- 172.21.15.159:0/1544124985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5700d0650 0x7ff5701525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.055+0000 7ff56de00640 1 -- 172.21.15.159:0/1544124985 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5701f49b0 con 0x7ff5700af830 2024-04-04T06:15:12.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.055+0000 7ff565400640 1 --2- 172.21.15.159:0/1544124985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5700d0650 0x7ff5701525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:12.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.056+0000 7ff56de00640 1 --2- 172.21.15.159:0/1544124985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5700af830 0x7ff5701f4470 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7ff56000c7c0 tx=0x7ff56000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:12.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.056+0000 7ff566a00640 1 -- 172.21.15.159:0/1544124985 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560018070 con 0x7ff5700af830 2024-04-04T06:15:12.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.056+0000 7ff566a00640 1 -- 172.21.15.159:0/1544124985 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff56000a640 con 0x7ff5700af830 2024-04-04T06:15:12.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.056+0000 7ff566a00640 1 -- 172.21.15.159:0/1544124985 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56000aa30 con 0x7ff5700af830 2024-04-04T06:15:12.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.056+0000 7ff574a00640 1 -- 172.21.15.159:0/1544124985 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff57015dc10 con 0x7ff5700af830 2024-04-04T06:15:12.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.056+0000 7ff574a00640 1 -- 172.21.15.159:0/1544124985 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff57015e1d0 con 0x7ff5700af830 2024-04-04T06:15:12.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.057+0000 7ff574a00640 1 -- 172.21.15.159:0/1544124985 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5700685c0 con 0x7ff5700af830 2024-04-04T06:15:12.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.058+0000 7ff566a00640 1 -- 172.21.15.159:0/1544124985 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff56001b070 con 0x7ff5700af830 2024-04-04T06:15:12.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.059+0000 7ff566a00640 1 --2- 172.21.15.159:0/1544124985 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff54c06c8f0 0x7ff54c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:12.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.059+0000 7ff566a00640 1 -- 172.21.15.159:0/1544124985 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7ff56008b170 con 0x7ff5700af830 2024-04-04T06:15:12.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.060+0000 7ff565400640 1 --2- 172.21.15.159:0/1544124985 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff54c06c8f0 0x7ff54c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:12.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.061+0000 7ff565400640 1 --2- 172.21.15.159:0/1544124985 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff54c06c8f0 0x7ff54c06edb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff558036000 tx=0x7ff558008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:12.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.064+0000 7ff566a00640 1 -- 172.21.15.159:0/1544124985 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff560059590 con 0x7ff5700af830 2024-04-04T06:15:12.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.265+0000 7ff574a00640 1 -- 172.21.15.159:0/1544124985 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff57006f1b0 con 0x7ff5700af830 2024-04-04T06:15:12.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.267+0000 7ff566a00640 1 -- 172.21.15.159:0/1544124985 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16623 (secure 0 0 0) 0x7ff560058e90 con 0x7ff5700af830 2024-04-04T06:15:12.262 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:15:12.262 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":101,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:15:09.233029+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:15:05.537357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-04T06:15:06.689304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:15:12.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.270+0000 7ff574a00640 1 -- 172.21.15.159:0/1544124985 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff54c06c8f0 msgr2=0x7ff54c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:12.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.270+0000 7ff574a00640 1 --2- 172.21.15.159:0/1544124985 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff54c06c8f0 0x7ff54c06edb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff558036000 tx=0x7ff558008030 comp rx=0 tx=0).stop 2024-04-04T06:15:12.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.271+0000 7ff574a00640 1 -- 172.21.15.159:0/1544124985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5700af830 msgr2=0x7ff5701f4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:12.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.271+0000 7ff574a00640 1 --2- 172.21.15.159:0/1544124985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5700af830 0x7ff5701f4470 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7ff56000c7c0 tx=0x7ff56000cc80 comp rx=0 tx=0).stop 2024-04-04T06:15:12.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.271+0000 7ff574a00640 1 -- 172.21.15.159:0/1544124985 shutdown_connections 2024-04-04T06:15:12.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.271+0000 7ff574a00640 1 --2- 172.21.15.159:0/1544124985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff57015c390 0x7ff5700b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.271+0000 7ff574a00640 1 --2- 172.21.15.159:0/1544124985 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff54c06c8f0 0x7ff54c06edb0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.271+0000 7ff574a00640 1 --2- 172.21.15.159:0/1544124985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5700d0650 0x7ff5701525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.271+0000 7ff574a00640 1 --2- 172.21.15.159:0/1544124985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5700af830 0x7ff5701f4470 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.271+0000 7ff574a00640 1 -- 172.21.15.159:0/1544124985 >> 172.21.15.159:0/1544124985 conn(0x7ff57006b990 msgr2=0x7ff5700aece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:12.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.271+0000 7ff574a00640 1 -- 172.21.15.159:0/1544124985 shutdown_connections 2024-04-04T06:15:12.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.271+0000 7ff574a00640 1 -- 172.21.15.159:0/1544124985 wait complete. 2024-04-04T06:15:12.289 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-04T06:15:12.289 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:15:12.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.494+0000 7fa0fe400640 1 Processor -- start 2024-04-04T06:15:12.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.494+0000 7fa0fe400640 1 -- start start 2024-04-04T06:15:12.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.496+0000 7fa0fe400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa0f8141460 0x7fa0f80b5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:12.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.496+0000 7fa0fe400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0f8141940 0x7fa0f806fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:12.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.496+0000 7fa0fe400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa0f806ffb0 0x7fa0f80d2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:12.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.496+0000 7fa0fe400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fa0f815eac0 con 0x7fa0f806ffb0 2024-04-04T06:15:12.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.496+0000 7fa0fe400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa0f815ed80 con 0x7fa0f8141460 2024-04-04T06:15:12.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.496+0000 7fa0fe400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa0f815f060 con 0x7fa0f8141940 2024-04-04T06:15:12.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.496+0000 7fa0fda00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa0f806ffb0 0x7fa0f80d2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:12.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.496+0000 7fa0fda00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa0f806ffb0 0x7fa0f80d2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35636/0 (socket says 172.21.15.159:35636) 2024-04-04T06:15:12.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.496+0000 7fa0fda00640 1 -- 172.21.15.159:0/1002568552 learned_addr learned my addr 172.21.15.159:0/1002568552 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:12.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.496+0000 7fa0fd000640 1 --2- 172.21.15.159:0/1002568552 >> v2:172.21.15.178:3300/0 conn(0x7fa0f8141460 0x7fa0f80b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:12.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.496+0000 7fa0f7e00640 1 --2- 172.21.15.159:0/1002568552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0f8141940 0x7fa0f806fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:12.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.496+0000 7fa0fda00640 1 -- 172.21.15.159:0/1002568552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0f8141940 msgr2=0x7fa0f806fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:12.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.497+0000 7fa0fda00640 1 --2- 172.21.15.159:0/1002568552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0f8141940 0x7fa0f806fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.497+0000 7fa0fda00640 1 -- 172.21.15.159:0/1002568552 >> v2:172.21.15.178:3300/0 conn(0x7fa0f8141460 msgr2=0x7fa0f80b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:12.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.497+0000 7fa0fda00640 1 --2- 172.21.15.159:0/1002568552 >> v2:172.21.15.178:3300/0 conn(0x7fa0f8141460 0x7fa0f80b5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.497+0000 7fa0fda00640 1 -- 172.21.15.159:0/1002568552 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f815f340 con 0x7fa0f806ffb0 2024-04-04T06:15:12.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.497+0000 7fa0fd000640 1 --2- 172.21.15.159:0/1002568552 >> v2:172.21.15.178:3300/0 conn(0x7fa0f8141460 0x7fa0f80b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:12.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.497+0000 7fa0f7e00640 1 --2- 172.21.15.159:0/1002568552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0f8141940 0x7fa0f806fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:12.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.498+0000 7fa0fda00640 1 --2- 172.21.15.159:0/1002568552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0f806ffb0 0x7fa0f80d2010 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80713a0 tx=0x7fa0f002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e257a80d4858f510 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:12.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.498+0000 7fa0f7400640 1 -- 172.21.15.159:0/1002568552 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f002ca70 con 0x7fa0f806ffb0 2024-04-04T06:15:12.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.498+0000 7fa0f7400640 1 -- 172.21.15.159:0/1002568552 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f002cc10 con 0x7fa0f806ffb0 2024-04-04T06:15:12.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.498+0000 7fa0f7400640 1 -- 172.21.15.159:0/1002568552 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f00343f0 con 0x7fa0f806ffb0 2024-04-04T06:15:12.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.499+0000 7fa0fe400640 1 -- 172.21.15.159:0/1002568552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0f806ffb0 msgr2=0x7fa0f80d2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:12.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.499+0000 7fa0fe400640 1 --2- 172.21.15.159:0/1002568552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0f806ffb0 0x7fa0f80d2010 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80713a0 tx=0x7fa0f002bed0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.499+0000 7fa0fe400640 1 -- 172.21.15.159:0/1002568552 shutdown_connections 2024-04-04T06:15:12.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.499+0000 7fa0fe400640 1 --2- 172.21.15.159:0/1002568552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0f806ffb0 0x7fa0f80d2010 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.499+0000 7fa0fe400640 1 --2- 172.21.15.159:0/1002568552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0f8141940 0x7fa0f806fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.499+0000 7fa0fe400640 1 --2- 172.21.15.159:0/1002568552 >> v2:172.21.15.178:3300/0 conn(0x7fa0f8141460 0x7fa0f80b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.499+0000 7fa0fe400640 1 -- 172.21.15.159:0/1002568552 >> 172.21.15.159:0/1002568552 conn(0x7fa0f8067460 msgr2=0x7fa0f8067840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:12.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.499+0000 7fa0fe400640 1 -- 172.21.15.159:0/1002568552 shutdown_connections 2024-04-04T06:15:12.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.499+0000 7fa0fe400640 1 -- 172.21.15.159:0/1002568552 wait complete. 2024-04-04T06:15:12.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.500+0000 7fa0fe400640 1 Processor -- start 2024-04-04T06:15:12.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.501+0000 7fa0fe400640 1 -- start start 2024-04-04T06:15:12.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.501+0000 7fa0fe400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0f806ffb0 0x7fa0f8155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:12.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.501+0000 7fa0fe400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0f8141460 0x7fa0f81540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:12.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.501+0000 7fa0fe400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0f8141940 0x7fa0f8157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:12.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.501+0000 7fa0fe400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f80b5530 con 0x7fa0f806ffb0 2024-04-04T06:15:12.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.501+0000 7fa0fe400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f80b53b0 con 0x7fa0f8141460 2024-04-04T06:15:12.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.501+0000 7fa0fe400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa0f80b5da0 con 0x7fa0f8141940 2024-04-04T06:15:12.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.502+0000 7fa0fd000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0f806ffb0 0x7fa0f8155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:12.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.502+0000 7fa0fd000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0f806ffb0 0x7fa0f8155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35644/0 (socket says 172.21.15.159:35644) 2024-04-04T06:15:12.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.502+0000 7fa0fd000640 1 -- 172.21.15.159:0/430602046 learned_addr learned my addr 172.21.15.159:0/430602046 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:12.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.502+0000 7fa0fda00640 1 --2- 172.21.15.159:0/430602046 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0f8141940 0x7fa0f8157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:12.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.502+0000 7fa0f7e00640 1 --2- 172.21.15.159:0/430602046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0f8141460 0x7fa0f81540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:12.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.502+0000 7fa0fd000640 1 -- 172.21.15.159:0/430602046 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0f8141940 msgr2=0x7fa0f8157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:12.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.502+0000 7fa0fd000640 1 --2- 172.21.15.159:0/430602046 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0f8141940 0x7fa0f8157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.502+0000 7fa0fd000640 1 -- 172.21.15.159:0/430602046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0f8141460 msgr2=0x7fa0f81540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:12.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.502+0000 7fa0fd000640 1 --2- 172.21.15.159:0/430602046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0f8141460 0x7fa0f81540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.502+0000 7fa0fd000640 1 -- 172.21.15.159:0/430602046 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f8159710 con 0x7fa0f806ffb0 2024-04-04T06:15:12.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.502+0000 7fa0f7e00640 1 --2- 172.21.15.159:0/430602046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0f8141460 0x7fa0f81540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:12.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.503+0000 7fa0fd000640 1 --2- 172.21.15.159:0/430602046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0f806ffb0 0x7fa0f8155c90 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00c820 tx=0x7fa0ec00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:12.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.503+0000 7fa0f4c00640 1 -- 172.21.15.159:0/430602046 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec015070 con 0x7fa0f806ffb0 2024-04-04T06:15:12.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.503+0000 7fa0f4c00640 1 -- 172.21.15.159:0/430602046 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0ec007a30 con 0x7fa0f806ffb0 2024-04-04T06:15:12.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.503+0000 7fa0f4c00640 1 -- 172.21.15.159:0/430602046 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec007e00 con 0x7fa0f806ffb0 2024-04-04T06:15:12.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.503+0000 7fa0fe400640 1 -- 172.21.15.159:0/430602046 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f8159150 con 0x7fa0f806ffb0 2024-04-04T06:15:12.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.503+0000 7fa0fe400640 1 -- 172.21.15.159:0/430602046 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f8155490 con 0x7fa0f806ffb0 2024-04-04T06:15:12.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.504+0000 7fa0fe400640 1 -- 172.21.15.159:0/430602046 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f8075940 con 0x7fa0f806ffb0 2024-04-04T06:15:12.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.505+0000 7fa0f4c00640 1 -- 172.21.15.159:0/430602046 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0ec018070 con 0x7fa0f806ffb0 2024-04-04T06:15:12.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.506+0000 7fa0f4c00640 1 --2- 172.21.15.159:0/430602046 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa0d806c8f0 0x7fa0d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:12.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.506+0000 7fa0f4c00640 1 -- 172.21.15.159:0/430602046 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fa0ec08bbc0 con 0x7fa0f806ffb0 2024-04-04T06:15:12.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.507+0000 7fa0f7e00640 1 --2- 172.21.15.159:0/430602046 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa0d806c8f0 0x7fa0d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:12.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.508+0000 7fa0f7e00640 1 --2- 172.21.15.159:0/430602046 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa0d806c8f0 0x7fa0d806edb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4006fd0 tx=0x7fa0e4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:12.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.511+0000 7fa0f4c00640 1 -- 172.21.15.159:0/430602046 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa0ec059f50 con 0x7fa0f806ffb0 2024-04-04T06:15:12.630 INFO:tasks.ceph.mds.a.smithi159.stderr:2024-04-04T06:15:12.635+0000 7f1bda400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:15:12.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.707+0000 7fa0fe400640 1 -- 172.21.15.159:0/430602046 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa0f8071d40 con 0x7fa0d806c8f0 2024-04-04T06:15:12.722 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.728+0000 7fa0f4c00640 1 -- 172.21.15.159:0/430602046 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242356 (secure 0 0 0) 0x7fa0f8071d40 con 0x7fa0d806c8f0 2024-04-04T06:15:12.727 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:15:12.728 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:15:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.734+0000 7fa0fe400640 1 -- 172.21.15.159:0/430602046 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa0d806c8f0 msgr2=0x7fa0d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.734+0000 7fa0fe400640 1 --2- 172.21.15.159:0/430602046 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa0d806c8f0 0x7fa0d806edb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4006fd0 tx=0x7fa0e4008040 comp rx=0 tx=0).stop 2024-04-04T06:15:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.734+0000 7fa0fe400640 1 -- 172.21.15.159:0/430602046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0f806ffb0 msgr2=0x7fa0f8155c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.734+0000 7fa0fe400640 1 --2- 172.21.15.159:0/430602046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0f806ffb0 0x7fa0f8155c90 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00c820 tx=0x7fa0ec00cc20 comp rx=0 tx=0).stop 2024-04-04T06:15:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.734+0000 7fa0fe400640 1 -- 172.21.15.159:0/430602046 shutdown_connections 2024-04-04T06:15:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.734+0000 7fa0fe400640 1 --2- 172.21.15.159:0/430602046 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa0f8141940 0x7fa0f8157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.734+0000 7fa0fe400640 1 --2- 172.21.15.159:0/430602046 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa0d806c8f0 0x7fa0d806edb0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.734+0000 7fa0fe400640 1 --2- 172.21.15.159:0/430602046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0f8141460 0x7fa0f81540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.734+0000 7fa0fe400640 1 --2- 172.21.15.159:0/430602046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0f806ffb0 0x7fa0f8155c90 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:12.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.734+0000 7fa0fe400640 1 -- 172.21.15.159:0/430602046 >> 172.21.15.159:0/430602046 conn(0x7fa0f8067460 msgr2=0x7fa0f815df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:12.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.734+0000 7fa0fe400640 1 -- 172.21.15.159:0/430602046 shutdown_connections 2024-04-04T06:15:12.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:12.734+0000 7fa0fe400640 1 -- 172.21.15.159:0/430602046 wait complete. 2024-04-04T06:15:12.746 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-04-04T06:15:10.920717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":86,"acting":86,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276752,"kb_used_data":10652,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754697968,"statfs":{"total":773094113280,"available":772810719232,"internally_reserved":0,"allocated":10907648,"data_stored":5359609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.704049"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.246160+0000","last_change":"2024-04-04T06:15:08.265768+0000","last_active":"2024-04-04T06:15:09.246160+0000","last_peered":"2024-04-04T06:15:09.246160+0000","last_clean":"2024-04-04T06:15:09.246160+0000","last_became_active":"2024-04-04T06:15:08.265463+0000","last_became_peered":"2024-04-04T06:15:08.265463+0000","last_unstale":"2024-04-04T06:15:09.246160+0000","last_undegraded":"2024-04-04T06:15:09.246160+0000","last_fullsized":"2024-04-04T06:15:09.246160+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:09:00.735586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.245920+0000","last_change":"2024-04-04T06:15:08.258651+0000","last_active":"2024-04-04T06:15:09.245920+0000","last_peered":"2024-04-04T06:15:09.245920+0000","last_clean":"2024-04-04T06:15:09.245920+0000","last_became_active":"2024-04-04T06:15:08.258208+0000","last_became_peered":"2024-04-04T06:15:08.258208+0000","last_unstale":"2024-04-04T06:15:09.245920+0000","last_undegraded":"2024-04-04T06:15:09.245920+0000","last_fullsized":"2024-04-04T06:15:09.245920+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:00:19.132428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.245941+0000","last_change":"2024-04-04T06:15:08.268155+0000","last_active":"2024-04-04T06:15:09.245941+0000","last_peered":"2024-04-04T06:15:09.245941+0000","last_clean":"2024-04-04T06:15:09.245941+0000","last_became_active":"2024-04-04T06:15:08.267743+0000","last_became_peered":"2024-04-04T06:15:08.267743+0000","last_unstale":"2024-04-04T06:15:09.245941+0000","last_undegraded":"2024-04-04T06:15:09.245941+0000","last_fullsized":"2024-04-04T06:15:09.245941+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:20:33.154788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.220509+0000","last_change":"2024-04-04T06:15:06.220509+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.220509+0000","last_undegraded":"2024-04-04T06:15:06.220509+0000","last_fullsized":"2024-04-04T06:15:06.220509+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.229554+0000","last_change":"2024-04-04T06:15:06.229554+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.229554+0000","last_undegraded":"2024-04-04T06:15:06.229554+0000","last_fullsized":"2024-04-04T06:15:06.229554+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.222091+0000","last_change":"2024-04-04T06:15:06.222091+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.222091+0000","last_undegraded":"2024-04-04T06:15:06.222091+0000","last_fullsized":"2024-04-04T06:15:06.222091+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.246443+0000","last_change":"2024-04-04T06:15:08.261392+0000","last_active":"2024-04-04T06:15:09.246443+0000","last_peered":"2024-04-04T06:15:09.246443+0000","last_clean":"2024-04-04T06:15:09.246443+0000","last_became_active":"2024-04-04T06:15:08.260157+0000","last_became_peered":"2024-04-04T06:15:08.260157+0000","last_unstale":"2024-04-04T06:15:09.246443+0000","last_undegraded":"2024-04-04T06:15:09.246443+0000","last_fullsized":"2024-04-04T06:15:09.246443+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:51:03.617173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.223399+0000","last_change":"2024-04-04T06:15:06.223399+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.223399+0000","last_undegraded":"2024-04-04T06:15:06.223399+0000","last_fullsized":"2024-04-04T06:15:06.223399+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.243828+0000","last_change":"2024-04-04T06:15:08.269814+0000","last_active":"2024-04-04T06:15:09.243828+0000","last_peered":"2024-04-04T06:15:09.243828+0000","last_clean":"2024-04-04T06:15:09.243828+0000","last_became_active":"2024-04-04T06:15:08.269194+0000","last_became_peered":"2024-04-04T06:15:08.269194+0000","last_unstale":"2024-04-04T06:15:09.243828+0000","last_undegraded":"2024-04-04T06:15:09.243828+0000","last_fullsized":"2024-04-04T06:15:09.243828+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:33:18.163742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.222665+0000","last_change":"2024-04-04T06:15:06.222665+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.222665+0000","last_undegraded":"2024-04-04T06:15:06.222665+0000","last_fullsized":"2024-04-04T06:15:06.222665+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.220718+0000","last_change":"2024-04-04T06:15:06.220718+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.220718+0000","last_undegraded":"2024-04-04T06:15:06.220718+0000","last_fullsized":"2024-04-04T06:15:06.220718+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.227526+0000","last_change":"2024-04-04T06:15:06.227526+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.227526+0000","last_undegraded":"2024-04-04T06:15:06.227526+0000","last_fullsized":"2024-04-04T06:15:06.227526+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.245158+0000","last_change":"2024-04-04T06:15:08.269364+0000","last_active":"2024-04-04T06:15:09.245158+0000","last_peered":"2024-04-04T06:15:09.245158+0000","last_clean":"2024-04-04T06:15:09.245158+0000","last_became_active":"2024-04-04T06:15:08.269046+0000","last_became_peered":"2024-04-04T06:15:08.269046+0000","last_unstale":"2024-04-04T06:15:09.245158+0000","last_undegraded":"2024-04-04T06:15:09.245158+0000","last_fullsized":"2024-04-04T06:15:09.245158+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:54:39.934757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.223160+0000","last_change":"2024-04-04T06:15:06.223160+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.223160+0000","last_undegraded":"2024-04-04T06:15:06.223160+0000","last_fullsized":"2024-04-04T06:15:06.223160+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.221505+0000","last_change":"2024-04-04T06:15:06.221505+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.221505+0000","last_undegraded":"2024-04-04T06:15:06.221505+0000","last_fullsized":"2024-04-04T06:15:06.221505+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.246325+0000","last_change":"2024-04-04T06:15:08.266337+0000","last_active":"2024-04-04T06:15:09.246325+0000","last_peered":"2024-04-04T06:15:09.246325+0000","last_clean":"2024-04-04T06:15:09.246325+0000","last_became_active":"2024-04-04T06:15:08.265918+0000","last_became_peered":"2024-04-04T06:15:08.265918+0000","last_unstale":"2024-04-04T06:15:09.246325+0000","last_undegraded":"2024-04-04T06:15:09.246325+0000","last_fullsized":"2024-04-04T06:15:09.246325+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:35:25.232360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.229285+0000","last_change":"2024-04-04T06:15:06.229285+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.229285+0000","last_undegraded":"2024-04-04T06:15:06.229285+0000","last_fullsized":"2024-04-04T06:15:06.229285+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.229193+0000","last_change":"2024-04-04T06:15:06.229193+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.229193+0000","last_undegraded":"2024-04-04T06:15:06.229193+0000","last_fullsized":"2024-04-04T06:15:06.229193+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"100'6","reported_seq":20,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.246148+0000","last_change":"2024-04-04T06:15:07.268705+0000","last_active":"2024-04-04T06:15:09.246148+0000","last_peered":"2024-04-04T06:15:09.246148+0000","last_clean":"2024-04-04T06:15:09.246148+0000","last_became_active":"2024-04-04T06:15:07.267294+0000","last_became_peered":"2024-04-04T06:15:07.267294+0000","last_unstale":"2024-04-04T06:15:09.246148+0000","last_undegraded":"2024-04-04T06:15:09.246148+0000","last_fullsized":"2024-04-04T06:15:09.246148+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:43:17.923204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.226721+0000","last_change":"2024-04-04T06:15:06.226721+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.226721+0000","last_undegraded":"2024-04-04T06:15:06.226721+0000","last_fullsized":"2024-04-04T06:15:06.226721+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.227054+0000","last_change":"2024-04-04T06:15:06.227054+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.227054+0000","last_undegraded":"2024-04-04T06:15:06.227054+0000","last_fullsized":"2024-04-04T06:15:06.227054+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.244442+0000","last_change":"2024-04-04T06:15:08.258098+0000","last_active":"2024-04-04T06:15:09.244442+0000","last_peered":"2024-04-04T06:15:09.244442+0000","last_clean":"2024-04-04T06:15:09.244442+0000","last_became_active":"2024-04-04T06:15:08.256314+0000","last_became_peered":"2024-04-04T06:15:08.256314+0000","last_unstale":"2024-04-04T06:15:09.244442+0000","last_undegraded":"2024-04-04T06:15:09.244442+0000","last_fullsized":"2024-04-04T06:15:09.244442+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:27:01.114506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.245031+0000","last_change":"2024-04-04T06:15:07.258294+0000","last_active":"2024-04-04T06:15:09.245031+0000","last_peered":"2024-04-04T06:15:09.245031+0000","last_clean":"2024-04-04T06:15:09.245031+0000","last_became_active":"2024-04-04T06:15:07.257999+0000","last_became_peered":"2024-04-04T06:15:07.257999+0000","last_unstale":"2024-04-04T06:15:09.245031+0000","last_undegraded":"2024-04-04T06:15:09.245031+0000","last_fullsized":"2024-04-04T06:15:09.245031+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:30:52.738137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.220750+0000","last_change":"2024-04-04T06:15:06.220750+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.220750+0000","last_undegraded":"2024-04-04T06:15:06.220750+0000","last_fullsized":"2024-04-04T06:15:06.220750+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.245638+0000","last_change":"2024-04-04T06:15:07.264124+0000","last_active":"2024-04-04T06:15:09.245638+0000","last_peered":"2024-04-04T06:15:09.245638+0000","last_clean":"2024-04-04T06:15:09.245638+0000","last_became_active":"2024-04-04T06:15:07.263806+0000","last_became_peered":"2024-04-04T06:15:07.263806+0000","last_unstale":"2024-04-04T06:15:09.245638+0000","last_undegraded":"2024-04-04T06:15:09.245638+0000","last_fullsized":"2024-04-04T06:15:09.245638+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:54:38.124794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.243789+0000","last_change":"2024-04-04T06:15:08.272101+0000","last_active":"2024-04-04T06:15:09.243789+0000","last_peered":"2024-04-04T06:15:09.243789+0000","last_clean":"2024-04-04T06:15:09.243789+0000","last_became_active":"2024-04-04T06:15:08.271815+0000","last_became_peered":"2024-04-04T06:15:08.271815+0000","last_unstale":"2024-04-04T06:15:09.243789+0000","last_undegraded":"2024-04-04T06:15:09.243789+0000","last_fullsized":"2024-04-04T06:15:09.243789+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:03:35.933763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.227497+0000","last_change":"2024-04-04T06:15:06.227497+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.227497+0000","last_undegraded":"2024-04-04T06:15:06.227497+0000","last_fullsized":"2024-04-04T06:15:06.227497+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.220984+0000","last_change":"2024-04-04T06:15:06.220984+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.220984+0000","last_undegraded":"2024-04-04T06:15:06.220984+0000","last_fullsized":"2024-04-04T06:15:06.220984+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.221781+0000","last_change":"2024-04-04T06:15:06.221781+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.221781+0000","last_undegraded":"2024-04-04T06:15:06.221781+0000","last_fullsized":"2024-04-04T06:15:06.221781+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.244051+0000","last_change":"2024-04-04T06:15:07.253311+0000","last_active":"2024-04-04T06:15:09.244051+0000","last_peered":"2024-04-04T06:15:09.244051+0000","last_clean":"2024-04-04T06:15:09.244051+0000","last_became_active":"2024-04-04T06:15:07.252771+0000","last_became_peered":"2024-04-04T06:15:07.252771+0000","last_unstale":"2024-04-04T06:15:09.244051+0000","last_undegraded":"2024-04-04T06:15:09.244051+0000","last_fullsized":"2024-04-04T06:15:09.244051+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:32:17.201608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.242980+0000","last_change":"2024-04-04T06:15:07.258451+0000","last_active":"2024-04-04T06:15:09.242980+0000","last_peered":"2024-04-04T06:15:09.242980+0000","last_clean":"2024-04-04T06:15:09.242980+0000","last_became_active":"2024-04-04T06:15:07.256375+0000","last_became_peered":"2024-04-04T06:15:07.256375+0000","last_unstale":"2024-04-04T06:15:09.242980+0000","last_undegraded":"2024-04-04T06:15:09.242980+0000","last_fullsized":"2024-04-04T06:15:09.242980+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:08:38.240680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.220554+0000","last_change":"2024-04-04T06:15:06.220554+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.220554+0000","last_undegraded":"2024-04-04T06:15:06.220554+0000","last_fullsized":"2024-04-04T06:15:06.220554+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.244834+0000","last_change":"2024-04-04T06:15:07.267441+0000","last_active":"2024-04-04T06:15:09.244834+0000","last_peered":"2024-04-04T06:15:09.244834+0000","last_clean":"2024-04-04T06:15:09.244834+0000","last_became_active":"2024-04-04T06:15:07.266558+0000","last_became_peered":"2024-04-04T06:15:07.266558+0000","last_unstale":"2024-04-04T06:15:09.244834+0000","last_undegraded":"2024-04-04T06:15:09.244834+0000","last_fullsized":"2024-04-04T06:15:09.244834+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:23:47.803575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.220650+0000","last_change":"2024-04-04T06:15:06.220650+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.220650+0000","last_undegraded":"2024-04-04T06:15:06.220650+0000","last_fullsized":"2024-04-04T06:15:06.220650+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.221565+0000","last_change":"2024-04-04T06:15:06.221565+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.221565+0000","last_undegraded":"2024-04-04T06:15:06.221565+0000","last_fullsized":"2024-04-04T06:15:06.221565+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.232112+0000","last_change":"2024-04-04T06:15:06.232112+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.232112+0000","last_undegraded":"2024-04-04T06:15:06.232112+0000","last_fullsized":"2024-04-04T06:15:06.232112+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.220714+0000","last_change":"2024-04-04T06:15:06.220714+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.220714+0000","last_undegraded":"2024-04-04T06:15:06.220714+0000","last_fullsized":"2024-04-04T06:15:06.220714+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.246124+0000","last_change":"2024-04-04T06:15:08.273883+0000","last_active":"2024-04-04T06:15:09.246124+0000","last_peered":"2024-04-04T06:15:09.246124+0000","last_clean":"2024-04-04T06:15:09.246124+0000","last_became_active":"2024-04-04T06:15:08.272764+0000","last_became_peered":"2024-04-04T06:15:08.272764+0000","last_unstale":"2024-04-04T06:15:09.246124+0000","last_undegraded":"2024-04-04T06:15:09.246124+0000","last_fullsized":"2024-04-04T06:15:09.246124+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:11:50.935948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.227539+0000","last_change":"2024-04-04T06:15:06.227539+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.227539+0000","last_undegraded":"2024-04-04T06:15:06.227539+0000","last_fullsized":"2024-04-04T06:15:06.227539+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.227430+0000","last_change":"2024-04-04T06:15:06.227430+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.227430+0000","last_undegraded":"2024-04-04T06:15:06.227430+0000","last_fullsized":"2024-04-04T06:15:06.227430+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.226657+0000","last_change":"2024-04-04T06:15:06.226657+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.226657+0000","last_undegraded":"2024-04-04T06:15:06.226657+0000","last_fullsized":"2024-04-04T06:15:06.226657+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-04T06:15:07.215356+0000","last_change":"2024-04-04T06:15:07.215356+0000","last_active":"2024-04-04T06:15:07.215356+0000","last_peered":"2024-04-04T06:15:07.215356+0000","last_clean":"2024-04-04T06:15:07.215356+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:07.215356+0000","last_undegraded":"2024-04-04T06:15:07.215356+0000","last_fullsized":"2024-04-04T06:15:07.215356+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-04T06:15:06.229736+0000","last_change":"2024-04-04T06:15:06.229736+0000","last_active":"2024-04-04T06:15:06.200397+0000","last_peered":"2024-04-04T06:15:06.200397+0000","last_clean":"2024-04-04T06:15:06.200397+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:15:06.229736+0000","last_undegraded":"2024-04-04T06:15:06.229736+0000","last_fullsized":"2024-04-04T06:15:06.229736+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":311,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-04T06:15:06.229471+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:15:06.229471+0000","last_peered":"2024-04-04T06:15:06.229471+0000","last_clean":"2024-04-04T06:15:06.229471+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:15:06.229471+0000","last_undegraded":"2024-04-04T06:15:06.229471+0000","last_fullsized":"2024-04-04T06:15:06.229471+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":2},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509555,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37096,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334744,"statfs":{"total":96636764160,"available":96598777856,"internally_reserved":0,"allocated":1798144,"data_stored":1108391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36516,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335324,"statfs":{"total":96636764160,"available":96599371776,"internally_reserved":0,"allocated":1204224,"data_stored":518023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1191936,"data_stored":512206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1204224,"data_stored":518023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509556,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1204224,"data_stored":518023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509558,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32936,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338904,"statfs":{"total":96636764160,"available":96603037696,"internally_reserved":0,"allocated":1798144,"data_stored":1108391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542262,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36560,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335280,"statfs":{"total":96636764160,"available":96599326720,"internally_reserved":0,"allocated":1245184,"data_stored":537443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542260,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36576,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335264,"statfs":{"total":96636764160,"available":96599310336,"internally_reserved":0,"allocated":1261568,"data_stored":539109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:15:12.753 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:15:13.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.011+0000 7f3d5c600640 1 Processor -- start 2024-04-04T06:15:13.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.011+0000 7f3d5c600640 1 -- start start 2024-04-04T06:15:13.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.013+0000 7f3d5c600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3d54161b00 0x7f3d54162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:13.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.013+0000 7f3d5c600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3d540ad120 0x7f3d540d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:13.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.013+0000 7f3d5c600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d540ad580 0x7f3d540d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:13.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.013+0000 7f3d5c600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f3d54161ee0 con 0x7f3d54161b00 2024-04-04T06:15:13.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.013+0000 7f3d5c600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3d540b3450 con 0x7f3d540ad120 2024-04-04T06:15:13.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.013+0000 7f3d5c600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3d540b3730 con 0x7f3d540ad580 2024-04-04T06:15:13.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.013+0000 7f3d59c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3d54161b00 0x7f3d54162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:13.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.013+0000 7f3d59c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3d54161b00 0x7f3d54162ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35648/0 (socket says 172.21.15.159:35648) 2024-04-04T06:15:13.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.013+0000 7f3d59c00640 1 -- 172.21.15.159:0/3499629701 learned_addr learned my addr 172.21.15.159:0/3499629701 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:13.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.014+0000 7f3d5a600640 1 --2- 172.21.15.159:0/3499629701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d540ad580 0x7f3d540d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:13.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.014+0000 7f3d59200640 1 --2- 172.21.15.159:0/3499629701 >> v2:172.21.15.178:3300/0 conn(0x7f3d540ad120 0x7f3d540d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:13.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.014+0000 7f3d59c00640 1 -- 172.21.15.159:0/3499629701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d540ad580 msgr2=0x7f3d540d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:13.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.014+0000 7f3d59c00640 1 --2- 172.21.15.159:0/3499629701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d540ad580 0x7f3d540d0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.014+0000 7f3d59c00640 1 -- 172.21.15.159:0/3499629701 >> v2:172.21.15.178:3300/0 conn(0x7f3d540ad120 msgr2=0x7f3d540d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:13.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.014+0000 7f3d59c00640 1 --2- 172.21.15.159:0/3499629701 >> v2:172.21.15.178:3300/0 conn(0x7f3d540ad120 0x7f3d540d0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.014+0000 7f3d59c00640 1 -- 172.21.15.159:0/3499629701 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d54071b60 con 0x7f3d54161b00 2024-04-04T06:15:13.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.015+0000 7f3d59c00640 1 --2- 172.21.15.159:0/3499629701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d54161b00 0x7f3d54162ab0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f3d5406ea10 tx=0x7f3d4802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b941a6f8256e388d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:13.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.015+0000 7f3d53e00640 1 -- 172.21.15.159:0/3499629701 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d48035020 con 0x7f3d54161b00 2024-04-04T06:15:13.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.015+0000 7f3d53e00640 1 -- 172.21.15.159:0/3499629701 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d48031870 con 0x7f3d54161b00 2024-04-04T06:15:13.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.015+0000 7f3d53e00640 1 -- 172.21.15.159:0/3499629701 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d48031c40 con 0x7f3d54161b00 2024-04-04T06:15:13.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.016+0000 7f3d5c600640 1 -- 172.21.15.159:0/3499629701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d54161b00 msgr2=0x7f3d54162ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:13.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.016+0000 7f3d5c600640 1 --2- 172.21.15.159:0/3499629701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d54161b00 0x7f3d54162ab0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f3d5406ea10 tx=0x7f3d4802b910 comp rx=0 tx=0).stop 2024-04-04T06:15:13.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.016+0000 7f3d5c600640 1 -- 172.21.15.159:0/3499629701 shutdown_connections 2024-04-04T06:15:13.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.016+0000 7f3d5c600640 1 --2- 172.21.15.159:0/3499629701 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d540ad580 0x7f3d540d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.016+0000 7f3d5c600640 1 --2- 172.21.15.159:0/3499629701 >> v2:172.21.15.178:3300/0 conn(0x7f3d540ad120 0x7f3d540d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.016+0000 7f3d5c600640 1 --2- 172.21.15.159:0/3499629701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d54161b00 0x7f3d54162ab0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.016+0000 7f3d5c600640 1 -- 172.21.15.159:0/3499629701 >> 172.21.15.159:0/3499629701 conn(0x7f3d5406b990 msgr2=0x7f3d5406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:13.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.016+0000 7f3d5c600640 1 -- 172.21.15.159:0/3499629701 shutdown_connections 2024-04-04T06:15:13.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.016+0000 7f3d5c600640 1 -- 172.21.15.159:0/3499629701 wait complete. 2024-04-04T06:15:13.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.019+0000 7f3d5c600640 1 Processor -- start 2024-04-04T06:15:13.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.019+0000 7f3d5c600640 1 -- start start 2024-04-04T06:15:13.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.021+0000 7f3d5c600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d540ad120 0x7f3d54152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:13.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.021+0000 7f3d5c600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d540ad580 0x7f3d54159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:13.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.021+0000 7f3d5c600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d54161b00 0x7f3d54158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:13.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.021+0000 7f3d5c600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3d54162ff0 con 0x7f3d540ad580 2024-04-04T06:15:13.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.021+0000 7f3d5c600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3d54072120 con 0x7f3d540ad120 2024-04-04T06:15:13.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.021+0000 7f3d5c600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3d540717f0 con 0x7f3d54161b00 2024-04-04T06:15:13.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.021+0000 7f3d59200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d540ad580 0x7f3d54159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:13.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.021+0000 7f3d59200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d540ad580 0x7f3d54159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35664/0 (socket says 172.21.15.159:35664) 2024-04-04T06:15:13.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.021+0000 7f3d59200640 1 -- 172.21.15.159:0/1069823842 learned_addr learned my addr 172.21.15.159:0/1069823842 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:13.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.021+0000 7f3d59c00640 1 --2- 172.21.15.159:0/1069823842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d540ad120 0x7f3d54152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:13.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.022+0000 7f3d5a600640 1 --2- 172.21.15.159:0/1069823842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d54161b00 0x7f3d54158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:13.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.022+0000 7f3d59200640 1 -- 172.21.15.159:0/1069823842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d54161b00 msgr2=0x7f3d54158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:13.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.022+0000 7f3d59200640 1 --2- 172.21.15.159:0/1069823842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d54161b00 0x7f3d54158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.022+0000 7f3d59200640 1 -- 172.21.15.159:0/1069823842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d540ad120 msgr2=0x7f3d54152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:13.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.022+0000 7f3d59200640 1 --2- 172.21.15.159:0/1069823842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d540ad120 0x7f3d54152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.022+0000 7f3d59200640 1 -- 172.21.15.159:0/1069823842 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d54159ec0 con 0x7f3d540ad580 2024-04-04T06:15:13.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.022+0000 7f3d5a600640 1 --2- 172.21.15.159:0/1069823842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d54161b00 0x7f3d54158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:13.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.022+0000 7f3d59200640 1 --2- 172.21.15.159:0/1069823842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d540ad580 0x7f3d54159980 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000d830 tx=0x7f3d4000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:13.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.022+0000 7f3d51600640 1 -- 172.21.15.159:0/1069823842 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40018030 con 0x7f3d540ad580 2024-04-04T06:15:13.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.022+0000 7f3d51600640 1 -- 172.21.15.159:0/1069823842 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d4000be10 con 0x7f3d540ad580 2024-04-04T06:15:13.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.022+0000 7f3d5c600640 1 -- 172.21.15.159:0/1069823842 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d54158330 con 0x7f3d540ad580 2024-04-04T06:15:13.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.022+0000 7f3d51600640 1 -- 172.21.15.159:0/1069823842 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d400098b0 con 0x7f3d540ad580 2024-04-04T06:15:13.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.023+0000 7f3d5c600640 1 -- 172.21.15.159:0/1069823842 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d54154ae0 con 0x7f3d540ad580 2024-04-04T06:15:13.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.024+0000 7f3d5c600640 1 -- 172.21.15.159:0/1069823842 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d540685c0 con 0x7f3d540ad580 2024-04-04T06:15:13.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.025+0000 7f3d51600640 1 -- 172.21.15.159:0/1069823842 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d4001b070 con 0x7f3d540ad580 2024-04-04T06:15:13.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.026+0000 7f3d51600640 1 --2- 172.21.15.159:0/1069823842 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3d3406c8f0 0x7f3d3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:13.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.026+0000 7f3d51600640 1 -- 172.21.15.159:0/1069823842 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f3d4008bda0 con 0x7f3d540ad580 2024-04-04T06:15:13.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.027+0000 7f3d59c00640 1 --2- 172.21.15.159:0/1069823842 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3d3406c8f0 0x7f3d3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:13.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.028+0000 7f3d59c00640 1 --2- 172.21.15.159:0/1069823842 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3d3406c8f0 0x7f3d3406edb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3d48036000 tx=0x7f3d4803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:13.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.031+0000 7f3d51600640 1 -- 172.21.15.159:0/1069823842 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3d4005a130 con 0x7f3d540ad580 2024-04-04T06:15:13.219 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:15:13.227 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:15:13.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.227+0000 7f3d5c600640 1 -- 172.21.15.159:0/1069823842 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d54159530 con 0x7f3d3406c8f0 2024-04-04T06:15:13.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.251+0000 7f3d51600640 1 -- 172.21.15.159:0/1069823842 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252262 (secure 0 0 0) 0x7f3d54159530 con 0x7f3d3406c8f0 2024-04-04T06:15:13.248 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:15:13.249 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:15:13.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.255+0000 7f3d5c600640 1 -- 172.21.15.159:0/1069823842 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3d3406c8f0 msgr2=0x7f3d3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:13.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.255+0000 7f3d5c600640 1 --2- 172.21.15.159:0/1069823842 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3d3406c8f0 0x7f3d3406edb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3d48036000 tx=0x7f3d4803c040 comp rx=0 tx=0).stop 2024-04-04T06:15:13.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.255+0000 7f3d5c600640 1 -- 172.21.15.159:0/1069823842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d540ad580 msgr2=0x7f3d54159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:13.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.255+0000 7f3d5c600640 1 --2- 172.21.15.159:0/1069823842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d540ad580 0x7f3d54159980 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000d830 tx=0x7f3d4000dcf0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.255+0000 7f3d5c600640 1 -- 172.21.15.159:0/1069823842 shutdown_connections 2024-04-04T06:15:13.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.255+0000 7f3d5c600640 1 --2- 172.21.15.159:0/1069823842 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d54161b00 0x7f3d54158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.255+0000 7f3d5c600640 1 --2- 172.21.15.159:0/1069823842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d540ad580 0x7f3d54159980 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.255+0000 7f3d5c600640 1 --2- 172.21.15.159:0/1069823842 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3d3406c8f0 0x7f3d3406edb0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.255+0000 7f3d5c600640 1 --2- 172.21.15.159:0/1069823842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d540ad120 0x7f3d54152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.255+0000 7f3d5c600640 1 -- 172.21.15.159:0/1069823842 >> 172.21.15.159:0/1069823842 conn(0x7f3d5406b990 msgr2=0x7f3d540b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:13.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.255+0000 7f3d5c600640 1 -- 172.21.15.159:0/1069823842 shutdown_connections 2024-04-04T06:15:13.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.255+0000 7f3d5c600640 1 -- 172.21.15.159:0/1069823842 wait complete. 2024-04-04T06:15:13.267 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2024-04-04T06:15:12.921367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":277020,"kb_used_data":10920,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754697700,"statfs":{"total":773094113280,"available":772810444800,"internally_reserved":0,"allocated":11182080,"data_stored":5484540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001946"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.898718+0000","last_change":"2024-04-04T06:15:08.262853+0000","last_active":"2024-04-04T06:15:10.898718+0000","last_peered":"2024-04-04T06:15:10.898718+0000","last_clean":"2024-04-04T06:15:10.898718+0000","last_became_active":"2024-04-04T06:15:08.262554+0000","last_became_peered":"2024-04-04T06:15:08.262554+0000","last_unstale":"2024-04-04T06:15:10.898718+0000","last_undegraded":"2024-04-04T06:15:10.898718+0000","last_fullsized":"2024-04-04T06:15:10.898718+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:27:37.898480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.252123+0000","last_change":"2024-04-04T06:15:08.262573+0000","last_active":"2024-04-04T06:15:10.252123+0000","last_peered":"2024-04-04T06:15:10.252123+0000","last_clean":"2024-04-04T06:15:10.252123+0000","last_became_active":"2024-04-04T06:15:08.262315+0000","last_became_peered":"2024-04-04T06:15:08.262315+0000","last_unstale":"2024-04-04T06:15:10.252123+0000","last_undegraded":"2024-04-04T06:15:10.252123+0000","last_fullsized":"2024-04-04T06:15:10.252123+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:00:20.941105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.898803+0000","last_change":"2024-04-04T06:15:08.263096+0000","last_active":"2024-04-04T06:15:10.898803+0000","last_peered":"2024-04-04T06:15:10.898803+0000","last_clean":"2024-04-04T06:15:10.898803+0000","last_became_active":"2024-04-04T06:15:08.262512+0000","last_became_peered":"2024-04-04T06:15:08.262512+0000","last_unstale":"2024-04-04T06:15:10.898803+0000","last_undegraded":"2024-04-04T06:15:10.898803+0000","last_fullsized":"2024-04-04T06:15:10.898803+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:31:59.346028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.395695+0000","last_change":"2024-04-04T06:15:08.273179+0000","last_active":"2024-04-04T06:15:10.395695+0000","last_peered":"2024-04-04T06:15:10.395695+0000","last_clean":"2024-04-04T06:15:10.395695+0000","last_became_active":"2024-04-04T06:15:08.272569+0000","last_became_peered":"2024-04-04T06:15:08.272569+0000","last_unstale":"2024-04-04T06:15:10.395695+0000","last_undegraded":"2024-04-04T06:15:10.395695+0000","last_fullsized":"2024-04-04T06:15:10.395695+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:09:00.735586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.800344+0000","last_change":"2024-04-04T06:15:08.277971+0000","last_active":"2024-04-04T06:15:10.800344+0000","last_peered":"2024-04-04T06:15:10.800344+0000","last_clean":"2024-04-04T06:15:10.800344+0000","last_became_active":"2024-04-04T06:15:08.277677+0000","last_became_peered":"2024-04-04T06:15:08.277677+0000","last_unstale":"2024-04-04T06:15:10.800344+0000","last_undegraded":"2024-04-04T06:15:10.800344+0000","last_fullsized":"2024-04-04T06:15:10.800344+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:47:43.808817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.675912+0000","last_change":"2024-04-04T06:15:08.273114+0000","last_active":"2024-04-04T06:15:10.675912+0000","last_peered":"2024-04-04T06:15:10.675912+0000","last_clean":"2024-04-04T06:15:10.675912+0000","last_became_active":"2024-04-04T06:15:08.272763+0000","last_became_peered":"2024-04-04T06:15:08.272763+0000","last_unstale":"2024-04-04T06:15:10.675912+0000","last_undegraded":"2024-04-04T06:15:10.675912+0000","last_fullsized":"2024-04-04T06:15:10.675912+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:17:07.553582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.675718+0000","last_change":"2024-04-04T06:15:08.279724+0000","last_active":"2024-04-04T06:15:10.675718+0000","last_peered":"2024-04-04T06:15:10.675718+0000","last_clean":"2024-04-04T06:15:10.675718+0000","last_became_active":"2024-04-04T06:15:08.278321+0000","last_became_peered":"2024-04-04T06:15:08.278321+0000","last_unstale":"2024-04-04T06:15:10.675718+0000","last_undegraded":"2024-04-04T06:15:10.675718+0000","last_fullsized":"2024-04-04T06:15:10.675718+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:55:31.733177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.251330+0000","last_change":"2024-04-04T06:15:08.279775+0000","last_active":"2024-04-04T06:15:10.251330+0000","last_peered":"2024-04-04T06:15:10.251330+0000","last_clean":"2024-04-04T06:15:10.251330+0000","last_became_active":"2024-04-04T06:15:08.279495+0000","last_became_peered":"2024-04-04T06:15:08.279495+0000","last_unstale":"2024-04-04T06:15:10.251330+0000","last_undegraded":"2024-04-04T06:15:10.251330+0000","last_fullsized":"2024-04-04T06:15:10.251330+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:09:14.565560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.246160+0000","last_change":"2024-04-04T06:15:08.265768+0000","last_active":"2024-04-04T06:15:09.246160+0000","last_peered":"2024-04-04T06:15:09.246160+0000","last_clean":"2024-04-04T06:15:09.246160+0000","last_became_active":"2024-04-04T06:15:08.265463+0000","last_became_peered":"2024-04-04T06:15:08.265463+0000","last_unstale":"2024-04-04T06:15:09.246160+0000","last_undegraded":"2024-04-04T06:15:09.246160+0000","last_fullsized":"2024-04-04T06:15:09.246160+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:09:00.735586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.800897+0000","last_change":"2024-04-04T06:15:08.273105+0000","last_active":"2024-04-04T06:15:10.800897+0000","last_peered":"2024-04-04T06:15:10.800897+0000","last_clean":"2024-04-04T06:15:10.800897+0000","last_became_active":"2024-04-04T06:15:08.272804+0000","last_became_peered":"2024-04-04T06:15:08.272804+0000","last_unstale":"2024-04-04T06:15:10.800897+0000","last_undegraded":"2024-04-04T06:15:10.800897+0000","last_fullsized":"2024-04-04T06:15:10.800897+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:56:01.942867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.675915+0000","last_change":"2024-04-04T06:15:08.279190+0000","last_active":"2024-04-04T06:15:10.675915+0000","last_peered":"2024-04-04T06:15:10.675915+0000","last_clean":"2024-04-04T06:15:10.675915+0000","last_became_active":"2024-04-04T06:15:08.278612+0000","last_became_peered":"2024-04-04T06:15:08.278612+0000","last_unstale":"2024-04-04T06:15:10.675915+0000","last_undegraded":"2024-04-04T06:15:10.675915+0000","last_fullsized":"2024-04-04T06:15:10.675915+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:11:44.816887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.245920+0000","last_change":"2024-04-04T06:15:08.258651+0000","last_active":"2024-04-04T06:15:09.245920+0000","last_peered":"2024-04-04T06:15:09.245920+0000","last_clean":"2024-04-04T06:15:09.245920+0000","last_became_active":"2024-04-04T06:15:08.258208+0000","last_became_peered":"2024-04-04T06:15:08.258208+0000","last_unstale":"2024-04-04T06:15:09.245920+0000","last_undegraded":"2024-04-04T06:15:09.245920+0000","last_fullsized":"2024-04-04T06:15:09.245920+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:00:19.132428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.395274+0000","last_change":"2024-04-04T06:15:08.267704+0000","last_active":"2024-04-04T06:15:10.395274+0000","last_peered":"2024-04-04T06:15:10.395274+0000","last_clean":"2024-04-04T06:15:10.395274+0000","last_became_active":"2024-04-04T06:15:08.266430+0000","last_became_peered":"2024-04-04T06:15:08.266430+0000","last_unstale":"2024-04-04T06:15:10.395274+0000","last_undegraded":"2024-04-04T06:15:10.395274+0000","last_fullsized":"2024-04-04T06:15:10.395274+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:20:33.154788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.241465+0000","last_change":"2024-04-04T06:15:08.272960+0000","last_active":"2024-04-04T06:15:09.241465+0000","last_peered":"2024-04-04T06:15:09.241465+0000","last_clean":"2024-04-04T06:15:09.241465+0000","last_became_active":"2024-04-04T06:15:08.272675+0000","last_became_peered":"2024-04-04T06:15:08.272675+0000","last_unstale":"2024-04-04T06:15:09.241465+0000","last_undegraded":"2024-04-04T06:15:09.241465+0000","last_fullsized":"2024-04-04T06:15:09.241465+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:36:44.977532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.394930+0000","last_change":"2024-04-04T06:15:08.252612+0000","last_active":"2024-04-04T06:15:10.394930+0000","last_peered":"2024-04-04T06:15:10.394930+0000","last_clean":"2024-04-04T06:15:10.394930+0000","last_became_active":"2024-04-04T06:15:08.252329+0000","last_became_peered":"2024-04-04T06:15:08.252329+0000","last_unstale":"2024-04-04T06:15:10.394930+0000","last_undegraded":"2024-04-04T06:15:10.394930+0000","last_fullsized":"2024-04-04T06:15:10.394930+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:27:01.114506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.675868+0000","last_change":"2024-04-04T06:15:08.284192+0000","last_active":"2024-04-04T06:15:10.675868+0000","last_peered":"2024-04-04T06:15:10.675868+0000","last_clean":"2024-04-04T06:15:10.675868+0000","last_became_active":"2024-04-04T06:15:08.283848+0000","last_became_peered":"2024-04-04T06:15:08.283848+0000","last_unstale":"2024-04-04T06:15:10.675868+0000","last_undegraded":"2024-04-04T06:15:10.675868+0000","last_fullsized":"2024-04-04T06:15:10.675868+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:37:46.111687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.252423+0000","last_change":"2024-04-04T06:15:08.259109+0000","last_active":"2024-04-04T06:15:10.252423+0000","last_peered":"2024-04-04T06:15:10.252423+0000","last_clean":"2024-04-04T06:15:10.252423+0000","last_became_active":"2024-04-04T06:15:08.258651+0000","last_became_peered":"2024-04-04T06:15:08.258651+0000","last_unstale":"2024-04-04T06:15:10.252423+0000","last_undegraded":"2024-04-04T06:15:10.252423+0000","last_fullsized":"2024-04-04T06:15:10.252423+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:02:38.063803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.395668+0000","last_change":"2024-04-04T06:15:08.265106+0000","last_active":"2024-04-04T06:15:10.395668+0000","last_peered":"2024-04-04T06:15:10.395668+0000","last_clean":"2024-04-04T06:15:10.395668+0000","last_became_active":"2024-04-04T06:15:08.264801+0000","last_became_peered":"2024-04-04T06:15:08.264801+0000","last_unstale":"2024-04-04T06:15:10.395668+0000","last_undegraded":"2024-04-04T06:15:10.395668+0000","last_fullsized":"2024-04-04T06:15:10.395668+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:11:50.935948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.243039+0000","last_change":"2024-04-04T06:15:08.269650+0000","last_active":"2024-04-04T06:15:09.243039+0000","last_peered":"2024-04-04T06:15:09.243039+0000","last_clean":"2024-04-04T06:15:09.243039+0000","last_became_active":"2024-04-04T06:15:08.268124+0000","last_became_peered":"2024-04-04T06:15:08.268124+0000","last_unstale":"2024-04-04T06:15:09.243039+0000","last_undegraded":"2024-04-04T06:15:09.243039+0000","last_fullsized":"2024-04-04T06:15:09.243039+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:06:47.930415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.245941+0000","last_change":"2024-04-04T06:15:08.268155+0000","last_active":"2024-04-04T06:15:09.245941+0000","last_peered":"2024-04-04T06:15:09.245941+0000","last_clean":"2024-04-04T06:15:09.245941+0000","last_became_active":"2024-04-04T06:15:08.267743+0000","last_became_peered":"2024-04-04T06:15:08.267743+0000","last_unstale":"2024-04-04T06:15:09.245941+0000","last_undegraded":"2024-04-04T06:15:09.245941+0000","last_fullsized":"2024-04-04T06:15:09.245941+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:20:33.154788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.242639+0000","last_change":"2024-04-04T06:15:08.261620+0000","last_active":"2024-04-04T06:15:09.242639+0000","last_peered":"2024-04-04T06:15:09.242639+0000","last_clean":"2024-04-04T06:15:09.242639+0000","last_became_active":"2024-04-04T06:15:08.261354+0000","last_became_peered":"2024-04-04T06:15:08.261354+0000","last_unstale":"2024-04-04T06:15:09.242639+0000","last_undegraded":"2024-04-04T06:15:09.242639+0000","last_fullsized":"2024-04-04T06:15:09.242639+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:44:18.215763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.251145+0000","last_change":"2024-04-04T06:15:08.262867+0000","last_active":"2024-04-04T06:15:10.251145+0000","last_peered":"2024-04-04T06:15:10.251145+0000","last_clean":"2024-04-04T06:15:10.251145+0000","last_became_active":"2024-04-04T06:15:08.259277+0000","last_became_peered":"2024-04-04T06:15:08.259277+0000","last_unstale":"2024-04-04T06:15:10.251145+0000","last_undegraded":"2024-04-04T06:15:10.251145+0000","last_fullsized":"2024-04-04T06:15:10.251145+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:51:03.617173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.251198+0000","last_change":"2024-04-04T06:15:08.260120+0000","last_active":"2024-04-04T06:15:10.251198+0000","last_peered":"2024-04-04T06:15:10.251198+0000","last_clean":"2024-04-04T06:15:10.251198+0000","last_became_active":"2024-04-04T06:15:08.259790+0000","last_became_peered":"2024-04-04T06:15:08.259790+0000","last_unstale":"2024-04-04T06:15:10.251198+0000","last_undegraded":"2024-04-04T06:15:10.251198+0000","last_fullsized":"2024-04-04T06:15:10.251198+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:59:44.940587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.676445+0000","last_change":"2024-04-04T06:15:08.273054+0000","last_active":"2024-04-04T06:15:10.676445+0000","last_peered":"2024-04-04T06:15:10.676445+0000","last_clean":"2024-04-04T06:15:10.676445+0000","last_became_active":"2024-04-04T06:15:08.272752+0000","last_became_peered":"2024-04-04T06:15:08.272752+0000","last_unstale":"2024-04-04T06:15:10.676445+0000","last_undegraded":"2024-04-04T06:15:10.676445+0000","last_fullsized":"2024-04-04T06:15:10.676445+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:56:34.343122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.800957+0000","last_change":"2024-04-04T06:15:08.281343+0000","last_active":"2024-04-04T06:15:10.800957+0000","last_peered":"2024-04-04T06:15:10.800957+0000","last_clean":"2024-04-04T06:15:10.800957+0000","last_became_active":"2024-04-04T06:15:08.281013+0000","last_became_peered":"2024-04-04T06:15:08.281013+0000","last_unstale":"2024-04-04T06:15:10.800957+0000","last_undegraded":"2024-04-04T06:15:10.800957+0000","last_fullsized":"2024-04-04T06:15:10.800957+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:33:55.827705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.252227+0000","last_change":"2024-04-04T06:15:08.261625+0000","last_active":"2024-04-04T06:15:10.252227+0000","last_peered":"2024-04-04T06:15:10.252227+0000","last_clean":"2024-04-04T06:15:10.252227+0000","last_became_active":"2024-04-04T06:15:08.261329+0000","last_became_peered":"2024-04-04T06:15:08.261329+0000","last_unstale":"2024-04-04T06:15:10.252227+0000","last_undegraded":"2024-04-04T06:15:10.252227+0000","last_fullsized":"2024-04-04T06:15:10.252227+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:35:00.104264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.675995+0000","last_change":"2024-04-04T06:15:08.269918+0000","last_active":"2024-04-04T06:15:10.675995+0000","last_peered":"2024-04-04T06:15:10.675995+0000","last_clean":"2024-04-04T06:15:10.675995+0000","last_became_active":"2024-04-04T06:15:08.269588+0000","last_became_peered":"2024-04-04T06:15:08.269588+0000","last_unstale":"2024-04-04T06:15:10.675995+0000","last_undegraded":"2024-04-04T06:15:10.675995+0000","last_fullsized":"2024-04-04T06:15:10.675995+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:25:59.625904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.800514+0000","last_change":"2024-04-04T06:15:08.284460+0000","last_active":"2024-04-04T06:15:10.800514+0000","last_peered":"2024-04-04T06:15:10.800514+0000","last_clean":"2024-04-04T06:15:10.800514+0000","last_became_active":"2024-04-04T06:15:08.283223+0000","last_became_peered":"2024-04-04T06:15:08.283223+0000","last_unstale":"2024-04-04T06:15:10.800514+0000","last_undegraded":"2024-04-04T06:15:10.800514+0000","last_fullsized":"2024-04-04T06:15:10.800514+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:21:44.177449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.251317+0000","last_change":"2024-04-04T06:15:08.254036+0000","last_active":"2024-04-04T06:15:10.251317+0000","last_peered":"2024-04-04T06:15:10.251317+0000","last_clean":"2024-04-04T06:15:10.251317+0000","last_became_active":"2024-04-04T06:15:08.253741+0000","last_became_peered":"2024-04-04T06:15:08.253741+0000","last_unstale":"2024-04-04T06:15:10.251317+0000","last_undegraded":"2024-04-04T06:15:10.251317+0000","last_fullsized":"2024-04-04T06:15:10.251317+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:23:53.950787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.241602+0000","last_change":"2024-04-04T06:15:08.261507+0000","last_active":"2024-04-04T06:15:09.241602+0000","last_peered":"2024-04-04T06:15:09.241602+0000","last_clean":"2024-04-04T06:15:09.241602+0000","last_became_active":"2024-04-04T06:15:08.260266+0000","last_became_peered":"2024-04-04T06:15:08.260266+0000","last_unstale":"2024-04-04T06:15:09.241602+0000","last_undegraded":"2024-04-04T06:15:09.241602+0000","last_fullsized":"2024-04-04T06:15:09.241602+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:14:15.628307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.394890+0000","last_change":"2024-04-04T06:15:08.276932+0000","last_active":"2024-04-04T06:15:10.394890+0000","last_peered":"2024-04-04T06:15:10.394890+0000","last_clean":"2024-04-04T06:15:10.394890+0000","last_became_active":"2024-04-04T06:15:08.276611+0000","last_became_peered":"2024-04-04T06:15:08.276611+0000","last_unstale":"2024-04-04T06:15:10.394890+0000","last_undegraded":"2024-04-04T06:15:10.394890+0000","last_fullsized":"2024-04-04T06:15:10.394890+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:04:25.022092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.898852+0000","last_change":"2024-04-04T06:15:08.260198+0000","last_active":"2024-04-04T06:15:10.898852+0000","last_peered":"2024-04-04T06:15:10.898852+0000","last_clean":"2024-04-04T06:15:10.898852+0000","last_became_active":"2024-04-04T06:15:08.259928+0000","last_became_peered":"2024-04-04T06:15:08.259928+0000","last_unstale":"2024-04-04T06:15:10.898852+0000","last_undegraded":"2024-04-04T06:15:10.898852+0000","last_fullsized":"2024-04-04T06:15:10.898852+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:35:18.112013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.243230+0000","last_change":"2024-04-04T06:15:08.269686+0000","last_active":"2024-04-04T06:15:09.243230+0000","last_peered":"2024-04-04T06:15:09.243230+0000","last_clean":"2024-04-04T06:15:09.243230+0000","last_became_active":"2024-04-04T06:15:08.268771+0000","last_became_peered":"2024-04-04T06:15:08.268771+0000","last_unstale":"2024-04-04T06:15:09.243230+0000","last_undegraded":"2024-04-04T06:15:09.243230+0000","last_fullsized":"2024-04-04T06:15:09.243230+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:59:08.205218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"100'4","reported_seq":18,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.252296+0000","last_change":"2024-04-04T06:15:07.268012+0000","last_active":"2024-04-04T06:15:10.252296+0000","last_peered":"2024-04-04T06:15:10.252296+0000","last_clean":"2024-04-04T06:15:10.252296+0000","last_became_active":"2024-04-04T06:15:07.267038+0000","last_became_peered":"2024-04-04T06:15:07.267038+0000","last_unstale":"2024-04-04T06:15:10.252296+0000","last_undegraded":"2024-04-04T06:15:10.252296+0000","last_fullsized":"2024-04-04T06:15:10.252296+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:51:03.634795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.395857+0000","last_change":"2024-04-04T06:15:08.277442+0000","last_active":"2024-04-04T06:15:10.395857+0000","last_peered":"2024-04-04T06:15:10.395857+0000","last_clean":"2024-04-04T06:15:10.395857+0000","last_became_active":"2024-04-04T06:15:08.276835+0000","last_became_peered":"2024-04-04T06:15:08.276835+0000","last_unstale":"2024-04-04T06:15:10.395857+0000","last_undegraded":"2024-04-04T06:15:10.395857+0000","last_fullsized":"2024-04-04T06:15:10.395857+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:35:25.232360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"100'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.675827+0000","last_change":"2024-04-04T06:15:07.270895+0000","last_active":"2024-04-04T06:15:10.675827+0000","last_peered":"2024-04-04T06:15:10.675827+0000","last_clean":"2024-04-04T06:15:10.675827+0000","last_became_active":"2024-04-04T06:15:07.269974+0000","last_became_peered":"2024-04-04T06:15:07.269974+0000","last_unstale":"2024-04-04T06:15:10.675827+0000","last_undegraded":"2024-04-04T06:15:10.675827+0000","last_fullsized":"2024-04-04T06:15:10.675827+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:23:26.162589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.899344+0000","last_change":"2024-04-04T06:15:08.266765+0000","last_active":"2024-04-04T06:15:10.899344+0000","last_peered":"2024-04-04T06:15:10.899344+0000","last_clean":"2024-04-04T06:15:10.899344+0000","last_became_active":"2024-04-04T06:15:08.266461+0000","last_became_peered":"2024-04-04T06:15:08.266461+0000","last_unstale":"2024-04-04T06:15:10.899344+0000","last_undegraded":"2024-04-04T06:15:10.899344+0000","last_fullsized":"2024-04-04T06:15:10.899344+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:09:35.570287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.251711+0000","last_change":"2024-04-04T06:15:07.268310+0000","last_active":"2024-04-04T06:15:10.251711+0000","last_peered":"2024-04-04T06:15:10.251711+0000","last_clean":"2024-04-04T06:15:10.251711+0000","last_became_active":"2024-04-04T06:15:07.267994+0000","last_became_peered":"2024-04-04T06:15:07.267994+0000","last_unstale":"2024-04-04T06:15:10.251711+0000","last_undegraded":"2024-04-04T06:15:10.251711+0000","last_fullsized":"2024-04-04T06:15:10.251711+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:54:38.124794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.246443+0000","last_change":"2024-04-04T06:15:08.261392+0000","last_active":"2024-04-04T06:15:09.246443+0000","last_peered":"2024-04-04T06:15:09.246443+0000","last_clean":"2024-04-04T06:15:09.246443+0000","last_became_active":"2024-04-04T06:15:08.260157+0000","last_became_peered":"2024-04-04T06:15:08.260157+0000","last_unstale":"2024-04-04T06:15:09.246443+0000","last_undegraded":"2024-04-04T06:15:09.246443+0000","last_fullsized":"2024-04-04T06:15:09.246443+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:51:03.617173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.251617+0000","last_change":"2024-04-04T06:15:07.269953+0000","last_active":"2024-04-04T06:15:10.251617+0000","last_peered":"2024-04-04T06:15:10.251617+0000","last_clean":"2024-04-04T06:15:10.251617+0000","last_became_active":"2024-04-04T06:15:07.269361+0000","last_became_peered":"2024-04-04T06:15:07.269361+0000","last_unstale":"2024-04-04T06:15:10.251617+0000","last_undegraded":"2024-04-04T06:15:10.251617+0000","last_fullsized":"2024-04-04T06:15:10.251617+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:51:37.208145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.243828+0000","last_change":"2024-04-04T06:15:08.269814+0000","last_active":"2024-04-04T06:15:09.243828+0000","last_peered":"2024-04-04T06:15:09.243828+0000","last_clean":"2024-04-04T06:15:09.243828+0000","last_became_active":"2024-04-04T06:15:08.269194+0000","last_became_peered":"2024-04-04T06:15:08.269194+0000","last_unstale":"2024-04-04T06:15:09.243828+0000","last_undegraded":"2024-04-04T06:15:09.243828+0000","last_fullsized":"2024-04-04T06:15:09.243828+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:33:18.163742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"100'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.898971+0000","last_change":"2024-04-04T06:15:07.258857+0000","last_active":"2024-04-04T06:15:10.898971+0000","last_peered":"2024-04-04T06:15:10.898971+0000","last_clean":"2024-04-04T06:15:10.898971+0000","last_became_active":"2024-04-04T06:15:07.255538+0000","last_became_peered":"2024-04-04T06:15:07.255538+0000","last_unstale":"2024-04-04T06:15:10.898971+0000","last_undegraded":"2024-04-04T06:15:10.898971+0000","last_fullsized":"2024-04-04T06:15:10.898971+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:36:43.962574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.675947+0000","last_change":"2024-04-04T06:15:08.274446+0000","last_active":"2024-04-04T06:15:10.675947+0000","last_peered":"2024-04-04T06:15:10.675947+0000","last_clean":"2024-04-04T06:15:10.675947+0000","last_became_active":"2024-04-04T06:15:08.274113+0000","last_became_peered":"2024-04-04T06:15:08.274113+0000","last_unstale":"2024-04-04T06:15:10.675947+0000","last_undegraded":"2024-04-04T06:15:10.675947+0000","last_fullsized":"2024-04-04T06:15:10.675947+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:33:24.924218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"100'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.250949+0000","last_change":"2024-04-04T06:15:07.263219+0000","last_active":"2024-04-04T06:15:10.250949+0000","last_peered":"2024-04-04T06:15:10.250949+0000","last_clean":"2024-04-04T06:15:10.250949+0000","last_became_active":"2024-04-04T06:15:07.262938+0000","last_became_peered":"2024-04-04T06:15:07.262938+0000","last_unstale":"2024-04-04T06:15:10.250949+0000","last_undegraded":"2024-04-04T06:15:10.250949+0000","last_fullsized":"2024-04-04T06:15:10.250949+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:21:54.529112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.395044+0000","last_change":"2024-04-04T06:15:08.256782+0000","last_active":"2024-04-04T06:15:10.395044+0000","last_peered":"2024-04-04T06:15:10.395044+0000","last_clean":"2024-04-04T06:15:10.395044+0000","last_became_active":"2024-04-04T06:15:08.256493+0000","last_became_peered":"2024-04-04T06:15:08.256493+0000","last_unstale":"2024-04-04T06:15:10.395044+0000","last_undegraded":"2024-04-04T06:15:10.395044+0000","last_fullsized":"2024-04-04T06:15:10.395044+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:00:19.132428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.800226+0000","last_change":"2024-04-04T06:15:07.262226+0000","last_active":"2024-04-04T06:15:10.800226+0000","last_peered":"2024-04-04T06:15:10.800226+0000","last_clean":"2024-04-04T06:15:10.800226+0000","last_became_active":"2024-04-04T06:15:07.261171+0000","last_became_peered":"2024-04-04T06:15:07.261171+0000","last_unstale":"2024-04-04T06:15:10.800226+0000","last_undegraded":"2024-04-04T06:15:10.800226+0000","last_fullsized":"2024-04-04T06:15:10.800226+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:01:55.357763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.245158+0000","last_change":"2024-04-04T06:15:08.269364+0000","last_active":"2024-04-04T06:15:09.245158+0000","last_peered":"2024-04-04T06:15:09.245158+0000","last_clean":"2024-04-04T06:15:09.245158+0000","last_became_active":"2024-04-04T06:15:08.269046+0000","last_became_peered":"2024-04-04T06:15:08.269046+0000","last_unstale":"2024-04-04T06:15:09.245158+0000","last_undegraded":"2024-04-04T06:15:09.245158+0000","last_fullsized":"2024-04-04T06:15:09.245158+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:54:39.934757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.898445+0000","last_change":"2024-04-04T06:15:07.251630+0000","last_active":"2024-04-04T06:15:10.898445+0000","last_peered":"2024-04-04T06:15:10.898445+0000","last_clean":"2024-04-04T06:15:10.898445+0000","last_became_active":"2024-04-04T06:15:07.251234+0000","last_became_peered":"2024-04-04T06:15:07.251234+0000","last_unstale":"2024-04-04T06:15:10.898445+0000","last_undegraded":"2024-04-04T06:15:10.898445+0000","last_fullsized":"2024-04-04T06:15:10.898445+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:50:18.750254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.252450+0000","last_change":"2024-04-04T06:15:08.265757+0000","last_active":"2024-04-04T06:15:10.252450+0000","last_peered":"2024-04-04T06:15:10.252450+0000","last_clean":"2024-04-04T06:15:10.252450+0000","last_became_active":"2024-04-04T06:15:08.265461+0000","last_became_peered":"2024-04-04T06:15:08.265461+0000","last_unstale":"2024-04-04T06:15:10.252450+0000","last_undegraded":"2024-04-04T06:15:10.252450+0000","last_fullsized":"2024-04-04T06:15:10.252450+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:25:49.999586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.898856+0000","last_change":"2024-04-04T06:15:07.256559+0000","last_active":"2024-04-04T06:15:10.898856+0000","last_peered":"2024-04-04T06:15:10.898856+0000","last_clean":"2024-04-04T06:15:10.898856+0000","last_became_active":"2024-04-04T06:15:07.256285+0000","last_became_peered":"2024-04-04T06:15:07.256285+0000","last_unstale":"2024-04-04T06:15:10.898856+0000","last_undegraded":"2024-04-04T06:15:10.898856+0000","last_fullsized":"2024-04-04T06:15:10.898856+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:14:14.613349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.246325+0000","last_change":"2024-04-04T06:15:08.266337+0000","last_active":"2024-04-04T06:15:09.246325+0000","last_peered":"2024-04-04T06:15:09.246325+0000","last_clean":"2024-04-04T06:15:09.246325+0000","last_became_active":"2024-04-04T06:15:08.265918+0000","last_became_peered":"2024-04-04T06:15:08.265918+0000","last_unstale":"2024-04-04T06:15:09.246325+0000","last_undegraded":"2024-04-04T06:15:09.246325+0000","last_fullsized":"2024-04-04T06:15:09.246325+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:35:25.232360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.395589+0000","last_change":"2024-04-04T06:15:07.263591+0000","last_active":"2024-04-04T06:15:10.395589+0000","last_peered":"2024-04-04T06:15:10.395589+0000","last_clean":"2024-04-04T06:15:10.395589+0000","last_became_active":"2024-04-04T06:15:07.261952+0000","last_became_peered":"2024-04-04T06:15:07.261952+0000","last_unstale":"2024-04-04T06:15:10.395589+0000","last_undegraded":"2024-04-04T06:15:10.395589+0000","last_fullsized":"2024-04-04T06:15:10.395589+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:54:38.124794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.800594+0000","last_change":"2024-04-04T06:15:08.284145+0000","last_active":"2024-04-04T06:15:10.800594+0000","last_peered":"2024-04-04T06:15:10.800594+0000","last_clean":"2024-04-04T06:15:10.800594+0000","last_became_active":"2024-04-04T06:15:08.282239+0000","last_became_peered":"2024-04-04T06:15:08.282239+0000","last_unstale":"2024-04-04T06:15:10.800594+0000","last_undegraded":"2024-04-04T06:15:10.800594+0000","last_fullsized":"2024-04-04T06:15:10.800594+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:17:32.640170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"100'3","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.676406+0000","last_change":"2024-04-04T06:15:07.270748+0000","last_active":"2024-04-04T06:15:10.676406+0000","last_peered":"2024-04-04T06:15:10.676406+0000","last_clean":"2024-04-04T06:15:10.676406+0000","last_became_active":"2024-04-04T06:15:07.270455+0000","last_became_peered":"2024-04-04T06:15:07.270455+0000","last_unstale":"2024-04-04T06:15:10.676406+0000","last_undegraded":"2024-04-04T06:15:10.676406+0000","last_fullsized":"2024-04-04T06:15:10.676406+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:59:41.884223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.676339+0000","last_change":"2024-04-04T06:15:08.282706+0000","last_active":"2024-04-04T06:15:10.676339+0000","last_peered":"2024-04-04T06:15:10.676339+0000","last_clean":"2024-04-04T06:15:10.676339+0000","last_became_active":"2024-04-04T06:15:08.282329+0000","last_became_peered":"2024-04-04T06:15:08.282329+0000","last_unstale":"2024-04-04T06:15:10.676339+0000","last_undegraded":"2024-04-04T06:15:10.676339+0000","last_fullsized":"2024-04-04T06:15:10.676339+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:02:29.925598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"100'6","reported_seq":20,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.246148+0000","last_change":"2024-04-04T06:15:07.268705+0000","last_active":"2024-04-04T06:15:09.246148+0000","last_peered":"2024-04-04T06:15:09.246148+0000","last_clean":"2024-04-04T06:15:09.246148+0000","last_became_active":"2024-04-04T06:15:07.267294+0000","last_became_peered":"2024-04-04T06:15:07.267294+0000","last_unstale":"2024-04-04T06:15:09.246148+0000","last_undegraded":"2024-04-04T06:15:09.246148+0000","last_fullsized":"2024-04-04T06:15:09.246148+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:43:17.923204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.394960+0000","last_change":"2024-04-04T06:15:08.277031+0000","last_active":"2024-04-04T06:15:10.394960+0000","last_peered":"2024-04-04T06:15:10.394960+0000","last_clean":"2024-04-04T06:15:10.394960+0000","last_became_active":"2024-04-04T06:15:08.276715+0000","last_became_peered":"2024-04-04T06:15:08.276715+0000","last_unstale":"2024-04-04T06:15:10.394960+0000","last_undegraded":"2024-04-04T06:15:10.394960+0000","last_fullsized":"2024-04-04T06:15:10.394960+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:54:39.934757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.242523+0000","last_change":"2024-04-04T06:15:07.271882+0000","last_active":"2024-04-04T06:15:09.242523+0000","last_peered":"2024-04-04T06:15:09.242523+0000","last_clean":"2024-04-04T06:15:09.242523+0000","last_became_active":"2024-04-04T06:15:07.271593+0000","last_became_peered":"2024-04-04T06:15:07.271593+0000","last_unstale":"2024-04-04T06:15:09.242523+0000","last_undegraded":"2024-04-04T06:15:09.242523+0000","last_fullsized":"2024-04-04T06:15:09.242523+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:24:53.901998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.898871+0000","last_change":"2024-04-04T06:15:08.263589+0000","last_active":"2024-04-04T06:15:10.898871+0000","last_peered":"2024-04-04T06:15:10.898871+0000","last_clean":"2024-04-04T06:15:10.898871+0000","last_became_active":"2024-04-04T06:15:08.263296+0000","last_became_peered":"2024-04-04T06:15:08.263296+0000","last_unstale":"2024-04-04T06:15:10.898871+0000","last_undegraded":"2024-04-04T06:15:10.898871+0000","last_fullsized":"2024-04-04T06:15:10.898871+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:53:27.543073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.242356+0000","last_change":"2024-04-04T06:15:07.263121+0000","last_active":"2024-04-04T06:15:09.242356+0000","last_peered":"2024-04-04T06:15:09.242356+0000","last_clean":"2024-04-04T06:15:09.242356+0000","last_became_active":"2024-04-04T06:15:07.262809+0000","last_became_peered":"2024-04-04T06:15:07.262809+0000","last_unstale":"2024-04-04T06:15:09.242356+0000","last_undegraded":"2024-04-04T06:15:09.242356+0000","last_fullsized":"2024-04-04T06:15:09.242356+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:11:47.434411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.244442+0000","last_change":"2024-04-04T06:15:08.258098+0000","last_active":"2024-04-04T06:15:09.244442+0000","last_peered":"2024-04-04T06:15:09.244442+0000","last_clean":"2024-04-04T06:15:09.244442+0000","last_became_active":"2024-04-04T06:15:08.256314+0000","last_became_peered":"2024-04-04T06:15:08.256314+0000","last_unstale":"2024-04-04T06:15:09.244442+0000","last_undegraded":"2024-04-04T06:15:09.244442+0000","last_fullsized":"2024-04-04T06:15:09.244442+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:27:01.114506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.245031+0000","last_change":"2024-04-04T06:15:07.258294+0000","last_active":"2024-04-04T06:15:09.245031+0000","last_peered":"2024-04-04T06:15:09.245031+0000","last_clean":"2024-04-04T06:15:09.245031+0000","last_became_active":"2024-04-04T06:15:07.257999+0000","last_became_peered":"2024-04-04T06:15:07.257999+0000","last_unstale":"2024-04-04T06:15:09.245031+0000","last_undegraded":"2024-04-04T06:15:09.245031+0000","last_fullsized":"2024-04-04T06:15:09.245031+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:30:52.738137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.240954+0000","last_change":"2024-04-04T06:15:08.269518+0000","last_active":"2024-04-04T06:15:09.240954+0000","last_peered":"2024-04-04T06:15:09.240954+0000","last_clean":"2024-04-04T06:15:09.240954+0000","last_became_active":"2024-04-04T06:15:08.269310+0000","last_became_peered":"2024-04-04T06:15:08.269310+0000","last_unstale":"2024-04-04T06:15:09.240954+0000","last_undegraded":"2024-04-04T06:15:09.240954+0000","last_fullsized":"2024-04-04T06:15:09.240954+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:37:44.650424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"100'8","reported_seq":22,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.898723+0000","last_change":"2024-04-04T06:15:07.263997+0000","last_active":"2024-04-04T06:15:10.898723+0000","last_peered":"2024-04-04T06:15:10.898723+0000","last_clean":"2024-04-04T06:15:10.898723+0000","last_became_active":"2024-04-04T06:15:07.263688+0000","last_became_peered":"2024-04-04T06:15:07.263688+0000","last_unstale":"2024-04-04T06:15:10.898723+0000","last_undegraded":"2024-04-04T06:15:10.898723+0000","last_fullsized":"2024-04-04T06:15:10.898723+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:39:59.258339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.801038+0000","last_change":"2024-04-04T06:15:08.281603+0000","last_active":"2024-04-04T06:15:10.801038+0000","last_peered":"2024-04-04T06:15:10.801038+0000","last_clean":"2024-04-04T06:15:10.801038+0000","last_became_active":"2024-04-04T06:15:08.280298+0000","last_became_peered":"2024-04-04T06:15:08.280298+0000","last_unstale":"2024-04-04T06:15:10.801038+0000","last_undegraded":"2024-04-04T06:15:10.801038+0000","last_fullsized":"2024-04-04T06:15:10.801038+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:01:50.443019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.245638+0000","last_change":"2024-04-04T06:15:07.264124+0000","last_active":"2024-04-04T06:15:09.245638+0000","last_peered":"2024-04-04T06:15:09.245638+0000","last_clean":"2024-04-04T06:15:09.245638+0000","last_became_active":"2024-04-04T06:15:07.263806+0000","last_became_peered":"2024-04-04T06:15:07.263806+0000","last_unstale":"2024-04-04T06:15:09.245638+0000","last_undegraded":"2024-04-04T06:15:09.245638+0000","last_fullsized":"2024-04-04T06:15:09.245638+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:54:38.124794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.243789+0000","last_change":"2024-04-04T06:15:08.272101+0000","last_active":"2024-04-04T06:15:09.243789+0000","last_peered":"2024-04-04T06:15:09.243789+0000","last_clean":"2024-04-04T06:15:09.243789+0000","last_became_active":"2024-04-04T06:15:08.271815+0000","last_became_peered":"2024-04-04T06:15:08.271815+0000","last_unstale":"2024-04-04T06:15:09.243789+0000","last_undegraded":"2024-04-04T06:15:09.243789+0000","last_fullsized":"2024-04-04T06:15:09.243789+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:03:35.933763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"100'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.800583+0000","last_change":"2024-04-04T06:15:07.265307+0000","last_active":"2024-04-04T06:15:10.800583+0000","last_peered":"2024-04-04T06:15:10.800583+0000","last_clean":"2024-04-04T06:15:10.800583+0000","last_became_active":"2024-04-04T06:15:07.265004+0000","last_became_peered":"2024-04-04T06:15:07.265004+0000","last_unstale":"2024-04-04T06:15:10.800583+0000","last_undegraded":"2024-04-04T06:15:10.800583+0000","last_fullsized":"2024-04-04T06:15:10.800583+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:40:16.433390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.676255+0000","last_change":"2024-04-04T06:15:08.278891+0000","last_active":"2024-04-04T06:15:10.676255+0000","last_peered":"2024-04-04T06:15:10.676255+0000","last_clean":"2024-04-04T06:15:10.676255+0000","last_became_active":"2024-04-04T06:15:08.278081+0000","last_became_peered":"2024-04-04T06:15:08.278081+0000","last_unstale":"2024-04-04T06:15:10.676255+0000","last_undegraded":"2024-04-04T06:15:10.676255+0000","last_fullsized":"2024-04-04T06:15:10.676255+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:44:56.362287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.251828+0000","last_change":"2024-04-04T06:15:07.268246+0000","last_active":"2024-04-04T06:15:10.251828+0000","last_peered":"2024-04-04T06:15:10.251828+0000","last_clean":"2024-04-04T06:15:10.251828+0000","last_became_active":"2024-04-04T06:15:07.267669+0000","last_became_peered":"2024-04-04T06:15:07.267669+0000","last_unstale":"2024-04-04T06:15:10.251828+0000","last_undegraded":"2024-04-04T06:15:10.251828+0000","last_fullsized":"2024-04-04T06:15:10.251828+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:46:51.748295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.241800+0000","last_change":"2024-04-04T06:15:08.265865+0000","last_active":"2024-04-04T06:15:09.241800+0000","last_peered":"2024-04-04T06:15:09.241800+0000","last_clean":"2024-04-04T06:15:09.241800+0000","last_became_active":"2024-04-04T06:15:08.265566+0000","last_became_peered":"2024-04-04T06:15:08.265566+0000","last_unstale":"2024-04-04T06:15:09.241800+0000","last_undegraded":"2024-04-04T06:15:09.241800+0000","last_fullsized":"2024-04-04T06:15:09.241800+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:59:59.080884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.251108+0000","last_change":"2024-04-04T06:15:07.269230+0000","last_active":"2024-04-04T06:15:10.251108+0000","last_peered":"2024-04-04T06:15:10.251108+0000","last_clean":"2024-04-04T06:15:10.251108+0000","last_became_active":"2024-04-04T06:15:07.268809+0000","last_became_peered":"2024-04-04T06:15:07.268809+0000","last_unstale":"2024-04-04T06:15:10.251108+0000","last_undegraded":"2024-04-04T06:15:10.251108+0000","last_fullsized":"2024-04-04T06:15:10.251108+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:09:23.845988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.242459+0000","last_change":"2024-04-04T06:15:08.275916+0000","last_active":"2024-04-04T06:15:09.242459+0000","last_peered":"2024-04-04T06:15:09.242459+0000","last_clean":"2024-04-04T06:15:09.242459+0000","last_became_active":"2024-04-04T06:15:08.275533+0000","last_became_peered":"2024-04-04T06:15:08.275533+0000","last_unstale":"2024-04-04T06:15:09.242459+0000","last_undegraded":"2024-04-04T06:15:09.242459+0000","last_fullsized":"2024-04-04T06:15:09.242459+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:54:20.852059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.244051+0000","last_change":"2024-04-04T06:15:07.253311+0000","last_active":"2024-04-04T06:15:09.244051+0000","last_peered":"2024-04-04T06:15:09.244051+0000","last_clean":"2024-04-04T06:15:09.244051+0000","last_became_active":"2024-04-04T06:15:07.252771+0000","last_became_peered":"2024-04-04T06:15:07.252771+0000","last_unstale":"2024-04-04T06:15:09.244051+0000","last_undegraded":"2024-04-04T06:15:09.244051+0000","last_fullsized":"2024-04-04T06:15:09.244051+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:32:17.201608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.241035+0000","last_change":"2024-04-04T06:15:08.258416+0000","last_active":"2024-04-04T06:15:09.241035+0000","last_peered":"2024-04-04T06:15:09.241035+0000","last_clean":"2024-04-04T06:15:09.241035+0000","last_became_active":"2024-04-04T06:15:08.257704+0000","last_became_peered":"2024-04-04T06:15:08.257704+0000","last_unstale":"2024-04-04T06:15:09.241035+0000","last_undegraded":"2024-04-04T06:15:09.241035+0000","last_fullsized":"2024-04-04T06:15:09.241035+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:50:19.765212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.242980+0000","last_change":"2024-04-04T06:15:07.258451+0000","last_active":"2024-04-04T06:15:09.242980+0000","last_peered":"2024-04-04T06:15:09.242980+0000","last_clean":"2024-04-04T06:15:09.242980+0000","last_became_active":"2024-04-04T06:15:07.256375+0000","last_became_peered":"2024-04-04T06:15:07.256375+0000","last_unstale":"2024-04-04T06:15:09.242980+0000","last_undegraded":"2024-04-04T06:15:09.242980+0000","last_fullsized":"2024-04-04T06:15:09.242980+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:08:38.240680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.799903+0000","last_change":"2024-04-04T06:15:08.276303+0000","last_active":"2024-04-04T06:15:10.799903+0000","last_peered":"2024-04-04T06:15:10.799903+0000","last_clean":"2024-04-04T06:15:10.799903+0000","last_became_active":"2024-04-04T06:15:08.275588+0000","last_became_peered":"2024-04-04T06:15:08.275588+0000","last_unstale":"2024-04-04T06:15:10.799903+0000","last_undegraded":"2024-04-04T06:15:10.799903+0000","last_fullsized":"2024-04-04T06:15:10.799903+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:56:34.343122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.251050+0000","last_change":"2024-04-04T06:15:07.255589+0000","last_active":"2024-04-04T06:15:10.251050+0000","last_peered":"2024-04-04T06:15:10.251050+0000","last_clean":"2024-04-04T06:15:10.251050+0000","last_became_active":"2024-04-04T06:15:07.255303+0000","last_became_peered":"2024-04-04T06:15:07.255303+0000","last_unstale":"2024-04-04T06:15:10.251050+0000","last_undegraded":"2024-04-04T06:15:10.251050+0000","last_fullsized":"2024-04-04T06:15:10.251050+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:30:52.738137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.252506+0000","last_change":"2024-04-04T06:15:08.261088+0000","last_active":"2024-04-04T06:15:10.252506+0000","last_peered":"2024-04-04T06:15:10.252506+0000","last_clean":"2024-04-04T06:15:10.252506+0000","last_became_active":"2024-04-04T06:15:08.260822+0000","last_became_peered":"2024-04-04T06:15:08.260822+0000","last_unstale":"2024-04-04T06:15:10.252506+0000","last_undegraded":"2024-04-04T06:15:10.252506+0000","last_fullsized":"2024-04-04T06:15:10.252506+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:07:44.019917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.244834+0000","last_change":"2024-04-04T06:15:07.267441+0000","last_active":"2024-04-04T06:15:09.244834+0000","last_peered":"2024-04-04T06:15:09.244834+0000","last_clean":"2024-04-04T06:15:09.244834+0000","last_became_active":"2024-04-04T06:15:07.266558+0000","last_became_peered":"2024-04-04T06:15:07.266558+0000","last_unstale":"2024-04-04T06:15:09.244834+0000","last_undegraded":"2024-04-04T06:15:09.244834+0000","last_fullsized":"2024-04-04T06:15:09.244834+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:23:47.803575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.898741+0000","last_change":"2024-04-04T06:15:08.265830+0000","last_active":"2024-04-04T06:15:10.898741+0000","last_peered":"2024-04-04T06:15:10.898741+0000","last_clean":"2024-04-04T06:15:10.898741+0000","last_became_active":"2024-04-04T06:15:08.265564+0000","last_became_peered":"2024-04-04T06:15:08.265564+0000","last_unstale":"2024-04-04T06:15:10.898741+0000","last_undegraded":"2024-04-04T06:15:10.898741+0000","last_fullsized":"2024-04-04T06:15:10.898741+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:03:19.930665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"101'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.252634+0000","last_change":"2024-04-04T06:15:07.267912+0000","last_active":"2024-04-04T06:15:10.252634+0000","last_peered":"2024-04-04T06:15:10.252634+0000","last_clean":"2024-04-04T06:15:10.252634+0000","last_became_active":"2024-04-04T06:15:07.267635+0000","last_became_peered":"2024-04-04T06:15:07.267635+0000","last_unstale":"2024-04-04T06:15:10.252634+0000","last_undegraded":"2024-04-04T06:15:10.252634+0000","last_fullsized":"2024-04-04T06:15:10.252634+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:09:55.652202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.899403+0000","last_change":"2024-04-04T06:15:08.267506+0000","last_active":"2024-04-04T06:15:10.899403+0000","last_peered":"2024-04-04T06:15:10.899403+0000","last_clean":"2024-04-04T06:15:10.899403+0000","last_became_active":"2024-04-04T06:15:08.267214+0000","last_became_peered":"2024-04-04T06:15:08.267214+0000","last_unstale":"2024-04-04T06:15:10.899403+0000","last_undegraded":"2024-04-04T06:15:10.899403+0000","last_fullsized":"2024-04-04T06:15:10.899403+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:34:11.549476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.251938+0000","last_change":"2024-04-04T06:15:07.268766+0000","last_active":"2024-04-04T06:15:10.251938+0000","last_peered":"2024-04-04T06:15:10.251938+0000","last_clean":"2024-04-04T06:15:10.251938+0000","last_became_active":"2024-04-04T06:15:07.268469+0000","last_became_peered":"2024-04-04T06:15:07.268469+0000","last_unstale":"2024-04-04T06:15:10.251938+0000","last_undegraded":"2024-04-04T06:15:10.251938+0000","last_fullsized":"2024-04-04T06:15:10.251938+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:43:17.923204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.800570+0000","last_change":"2024-04-04T06:15:08.280460+0000","last_active":"2024-04-04T06:15:10.800570+0000","last_peered":"2024-04-04T06:15:10.800570+0000","last_clean":"2024-04-04T06:15:10.800570+0000","last_became_active":"2024-04-04T06:15:08.279353+0000","last_became_peered":"2024-04-04T06:15:08.279353+0000","last_unstale":"2024-04-04T06:15:10.800570+0000","last_undegraded":"2024-04-04T06:15:10.800570+0000","last_fullsized":"2024-04-04T06:15:10.800570+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:43:07.963103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.800659+0000","last_change":"2024-04-04T06:15:07.264701+0000","last_active":"2024-04-04T06:15:10.800659+0000","last_peered":"2024-04-04T06:15:10.800659+0000","last_clean":"2024-04-04T06:15:10.800659+0000","last_became_active":"2024-04-04T06:15:07.264414+0000","last_became_peered":"2024-04-04T06:15:07.264414+0000","last_unstale":"2024-04-04T06:15:10.800659+0000","last_undegraded":"2024-04-04T06:15:10.800659+0000","last_fullsized":"2024-04-04T06:15:10.800659+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:23:49.464407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.242397+0000","last_change":"2024-04-04T06:15:08.270418+0000","last_active":"2024-04-04T06:15:09.242397+0000","last_peered":"2024-04-04T06:15:09.242397+0000","last_clean":"2024-04-04T06:15:09.242397+0000","last_became_active":"2024-04-04T06:15:08.269981+0000","last_became_peered":"2024-04-04T06:15:08.269981+0000","last_unstale":"2024-04-04T06:15:09.242397+0000","last_undegraded":"2024-04-04T06:15:09.242397+0000","last_fullsized":"2024-04-04T06:15:09.242397+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:18:54.307450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.251541+0000","last_change":"2024-04-04T06:15:07.261409+0000","last_active":"2024-04-04T06:15:10.251541+0000","last_peered":"2024-04-04T06:15:10.251541+0000","last_clean":"2024-04-04T06:15:10.251541+0000","last_became_active":"2024-04-04T06:15:07.259078+0000","last_became_peered":"2024-04-04T06:15:07.259078+0000","last_unstale":"2024-04-04T06:15:10.251541+0000","last_undegraded":"2024-04-04T06:15:10.251541+0000","last_fullsized":"2024-04-04T06:15:10.251541+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:21:42.936913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.246124+0000","last_change":"2024-04-04T06:15:08.273883+0000","last_active":"2024-04-04T06:15:09.246124+0000","last_peered":"2024-04-04T06:15:09.246124+0000","last_clean":"2024-04-04T06:15:09.246124+0000","last_became_active":"2024-04-04T06:15:08.272764+0000","last_became_peered":"2024-04-04T06:15:08.272764+0000","last_unstale":"2024-04-04T06:15:09.246124+0000","last_undegraded":"2024-04-04T06:15:09.246124+0000","last_fullsized":"2024-04-04T06:15:09.246124+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:11:50.935948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.799753+0000","last_change":"2024-04-04T06:15:07.257195+0000","last_active":"2024-04-04T06:15:10.799753+0000","last_peered":"2024-04-04T06:15:10.799753+0000","last_clean":"2024-04-04T06:15:10.799753+0000","last_became_active":"2024-04-04T06:15:07.256677+0000","last_became_peered":"2024-04-04T06:15:07.256677+0000","last_unstale":"2024-04-04T06:15:10.799753+0000","last_undegraded":"2024-04-04T06:15:10.799753+0000","last_fullsized":"2024-04-04T06:15:10.799753+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:50:00.815666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.241158+0000","last_change":"2024-04-04T06:15:08.280508+0000","last_active":"2024-04-04T06:15:09.241158+0000","last_peered":"2024-04-04T06:15:09.241158+0000","last_clean":"2024-04-04T06:15:09.241158+0000","last_became_active":"2024-04-04T06:15:08.278743+0000","last_became_peered":"2024-04-04T06:15:08.278743+0000","last_unstale":"2024-04-04T06:15:09.241158+0000","last_undegraded":"2024-04-04T06:15:09.241158+0000","last_fullsized":"2024-04-04T06:15:09.241158+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:40:00.273297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.394953+0000","last_change":"2024-04-04T06:15:07.251778+0000","last_active":"2024-04-04T06:15:10.394953+0000","last_peered":"2024-04-04T06:15:10.394953+0000","last_clean":"2024-04-04T06:15:10.394953+0000","last_became_active":"2024-04-04T06:15:07.251491+0000","last_became_peered":"2024-04-04T06:15:07.251491+0000","last_unstale":"2024-04-04T06:15:10.394953+0000","last_undegraded":"2024-04-04T06:15:10.394953+0000","last_fullsized":"2024-04-04T06:15:10.394953+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:46:51.748295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.800679+0000","last_change":"2024-04-04T06:15:08.280856+0000","last_active":"2024-04-04T06:15:10.800679+0000","last_peered":"2024-04-04T06:15:10.800679+0000","last_clean":"2024-04-04T06:15:10.800679+0000","last_became_active":"2024-04-04T06:15:08.280549+0000","last_became_peered":"2024-04-04T06:15:08.280549+0000","last_unstale":"2024-04-04T06:15:10.800679+0000","last_undegraded":"2024-04-04T06:15:10.800679+0000","last_fullsized":"2024-04-04T06:15:10.800679+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:32:12.108888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.241190+0000","last_change":"2024-04-04T06:15:07.270123+0000","last_active":"2024-04-04T06:15:09.241190+0000","last_peered":"2024-04-04T06:15:09.241190+0000","last_clean":"2024-04-04T06:15:09.241190+0000","last_became_active":"2024-04-04T06:15:07.269847+0000","last_became_peered":"2024-04-04T06:15:07.269847+0000","last_unstale":"2024-04-04T06:15:09.241190+0000","last_undegraded":"2024-04-04T06:15:09.241190+0000","last_fullsized":"2024-04-04T06:15:09.241190+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:40:55.844990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:09.243287+0000","last_change":"2024-04-04T06:15:08.264637+0000","last_active":"2024-04-04T06:15:09.243287+0000","last_peered":"2024-04-04T06:15:09.243287+0000","last_clean":"2024-04-04T06:15:09.243287+0000","last_became_active":"2024-04-04T06:15:08.264326+0000","last_became_peered":"2024-04-04T06:15:08.264326+0000","last_unstale":"2024-04-04T06:15:09.243287+0000","last_undegraded":"2024-04-04T06:15:09.243287+0000","last_fullsized":"2024-04-04T06:15:09.243287+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:07.215356+0000","last_clean_scrub_stamp":"2024-04-04T06:15:07.215356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:29:52.936549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"100'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.675827+0000","last_change":"2024-04-04T06:15:07.268956+0000","last_active":"2024-04-04T06:15:10.675827+0000","last_peered":"2024-04-04T06:15:10.675827+0000","last_clean":"2024-04-04T06:15:10.675827+0000","last_became_active":"2024-04-04T06:15:07.268652+0000","last_became_peered":"2024-04-04T06:15:07.268652+0000","last_unstale":"2024-04-04T06:15:10.675827+0000","last_undegraded":"2024-04-04T06:15:10.675827+0000","last_fullsized":"2024-04-04T06:15:10.675827+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:15:06.200397+0000","last_clean_scrub_stamp":"2024-04-04T06:15:06.200397+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:44:29.297710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":317,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-04-04T06:15:10.675148+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:15:10.675148+0000","last_peered":"2024-04-04T06:15:10.675148+0000","last_clean":"2024-04-04T06:15:10.675148+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:15:10.675148+0000","last_undegraded":"2024-04-04T06:15:10.675148+0000","last_fullsized":"2024-04-04T06:15:10.675148+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509556,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37140,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334700,"statfs":{"total":96636764160,"available":96598732800,"internally_reserved":0,"allocated":1843200,"data_stored":1127853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509558,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36560,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335280,"statfs":{"total":96636764160,"available":96599326720,"internally_reserved":0,"allocated":1249280,"data_stored":538025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509559,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1257472,"data_stored":539007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509558,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1245184,"data_stored":537407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509557,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1241088,"data_stored":537373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509559,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32980,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338860,"statfs":{"total":96636764160,"available":96602992640,"internally_reserved":0,"allocated":1839104,"data_stored":1128323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542262,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36560,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335280,"statfs":{"total":96636764160,"available":96599326720,"internally_reserved":0,"allocated":1245184,"data_stored":537443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542260,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36576,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335264,"statfs":{"total":96636764160,"available":96599310336,"internally_reserved":0,"allocated":1261568,"data_stored":539109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:15:13.271 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-04T06:15:13.272 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-04T06:15:13.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.521+0000 7f4866a00640 1 Processor -- start 2024-04-04T06:15:13.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.522+0000 7f4866a00640 1 -- start start 2024-04-04T06:15:13.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.523+0000 7f4866a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48600d0e70 0x7f48600cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:13.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.523+0000 7f4866a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f486015f6e0 0x7f48600d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:13.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.523+0000 7f4866a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48600d1790 con 0x7f48601621d0 2024-04-04T06:15:13.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.523+0000 7f4866a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f48600cf6c0 con 0x7f486015f6e0 2024-04-04T06:15:13.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.523+0000 7f4866a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f48600cf9a0 con 0x7f48600d0e70 2024-04-04T06:15:13.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.524+0000 7f4857e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f486015f6e0 0x7f48600d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:13.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.524+0000 7f4857e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f486015f6e0 0x7f48600d1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:41202/0 (socket says 172.21.15.159:41202) 2024-04-04T06:15:13.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.524+0000 7f4857e00640 1 -- 172.21.15.159:0/978110216 learned_addr learned my addr 172.21.15.159:0/978110216 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:13.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.524+0000 7f485fe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48600d0e70 0x7f48600cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:13.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.525+0000 7f485fe00640 1 -- 172.21.15.159:0/978110216 >> v2:172.21.15.178:3300/0 conn(0x7f486015f6e0 msgr2=0x7f48600d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:13.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.525+0000 7f485f400640 1 -- 172.21.15.159:0/978110216 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 315284640 0 0) 0x7f48600d1790 con 0x7f48601621d0 2024-04-04T06:15:13.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.525+0000 7f485fe00640 1 --2- 172.21.15.159:0/978110216 >> v2:172.21.15.178:3300/0 conn(0x7f486015f6e0 0x7f48600d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.525+0000 7f485fe00640 1 -- 172.21.15.159:0/978110216 >> v1:172.21.15.178:6789/0 conn(0x7f48601621d0 legacy=0x7f4860162660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:13.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.525+0000 7f485fe00640 1 -- 172.21.15.159:0/978110216 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48600cfc80 con 0x7f48600d0e70 2024-04-04T06:15:13.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.527+0000 7f485fe00640 1 --2- 172.21.15.159:0/978110216 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48600d0e70 0x7f48600cf050 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f48500097a0 tx=0x7f485002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1952f59ac5d3fb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:13.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.527+0000 7f485f400640 1 -- 172.21.15.159:0/978110216 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48500315a0 con 0x7f48600d0e70 2024-04-04T06:15:13.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.527+0000 7f485f400640 1 -- 172.21.15.159:0/978110216 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4850037070 con 0x7f48600d0e70 2024-04-04T06:15:13.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.527+0000 7f485f400640 1 -- 172.21.15.159:0/978110216 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4850031960 con 0x7f48600d0e70 2024-04-04T06:15:13.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.528+0000 7f4866a00640 1 -- 172.21.15.159:0/978110216 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48600d0e70 msgr2=0x7f48600cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:13.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.528+0000 7f4866a00640 1 --2- 172.21.15.159:0/978110216 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48600d0e70 0x7f48600cf050 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f48500097a0 tx=0x7f485002b840 comp rx=0 tx=0).stop 2024-04-04T06:15:13.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.528+0000 7f4866a00640 1 -- 172.21.15.159:0/978110216 shutdown_connections 2024-04-04T06:15:13.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.528+0000 7f4866a00640 1 --2- 172.21.15.159:0/978110216 >> v2:172.21.15.178:3300/0 conn(0x7f486015f6e0 0x7f48600d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.528+0000 7f4866a00640 1 --2- 172.21.15.159:0/978110216 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48600d0e70 0x7f48600cf050 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.528+0000 7f4866a00640 1 -- 172.21.15.159:0/978110216 >> 172.21.15.159:0/978110216 conn(0x7f486008bfb0 msgr2=0x7f48600d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:13.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.528+0000 7f4866a00640 1 -- 172.21.15.159:0/978110216 shutdown_connections 2024-04-04T06:15:13.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.528+0000 7f4866a00640 1 -- 172.21.15.159:0/978110216 wait complete. 2024-04-04T06:15:13.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.530+0000 7f4866a00640 1 Processor -- start 2024-04-04T06:15:13.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.530+0000 7f4866a00640 1 -- start start 2024-04-04T06:15:13.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.531+0000 7f4866a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48600d0e70 0x7f4860158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:13.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.531+0000 7f4866a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f486015f6e0 0x7f486015af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:13.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.531+0000 7f4866a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48601621d0 0x7f486014e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:13.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.531+0000 7f4866a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f486015fb40 con 0x7f48600d0e70 2024-04-04T06:15:13.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.531+0000 7f4866a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f48600d0240 con 0x7f486015f6e0 2024-04-04T06:15:13.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.531+0000 7f4866a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f486015e910 con 0x7f48601621d0 2024-04-04T06:15:13.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.532+0000 7f485fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48600d0e70 0x7f4860158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:13.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.532+0000 7f485fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48600d0e70 0x7f4860158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35666/0 (socket says 172.21.15.159:35666) 2024-04-04T06:15:13.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.532+0000 7f485fe00640 1 -- 172.21.15.159:0/2940552833 learned_addr learned my addr 172.21.15.159:0/2940552833 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:13.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.532+0000 7f4857e00640 1 --2- 172.21.15.159:0/2940552833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f486015f6e0 0x7f486015af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:13.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.532+0000 7f4864a00640 1 --2- 172.21.15.159:0/2940552833 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48601621d0 0x7f486014e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:13.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.532+0000 7f485fe00640 1 -- 172.21.15.159:0/2940552833 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48601621d0 msgr2=0x7f486014e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:13.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.532+0000 7f485fe00640 1 --2- 172.21.15.159:0/2940552833 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48601621d0 0x7f486014e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.532+0000 7f485fe00640 1 -- 172.21.15.159:0/2940552833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f486015f6e0 msgr2=0x7f486015af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:13.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.532+0000 7f485fe00640 1 --2- 172.21.15.159:0/2940552833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f486015f6e0 0x7f486015af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.532+0000 7f485fe00640 1 -- 172.21.15.159:0/2940552833 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48600b8660 con 0x7f48600d0e70 2024-04-04T06:15:13.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.532+0000 7f4857e00640 1 --2- 172.21.15.159:0/2940552833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f486015f6e0 0x7f486015af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:13.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.532+0000 7f4864a00640 1 --2- 172.21.15.159:0/2940552833 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48601621d0 0x7f486014e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:13.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.533+0000 7f485fe00640 1 --2- 172.21.15.159:0/2940552833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48600d0e70 0x7f4860158150 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f48500097a0 tx=0x7f48500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:13.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.533+0000 7f485cc00640 1 -- 172.21.15.159:0/2940552833 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4850004230 con 0x7f48600d0e70 2024-04-04T06:15:13.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.533+0000 7f485cc00640 1 -- 172.21.15.159:0/2940552833 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48500043d0 con 0x7f48600d0e70 2024-04-04T06:15:13.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.533+0000 7f4866a00640 1 -- 172.21.15.159:0/2940552833 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48600b8940 con 0x7f48600d0e70 2024-04-04T06:15:13.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.533+0000 7f485cc00640 1 -- 172.21.15.159:0/2940552833 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4850003850 con 0x7f48600d0e70 2024-04-04T06:15:13.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.533+0000 7f4866a00640 1 -- 172.21.15.159:0/2940552833 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48600b8f00 con 0x7f48600d0e70 2024-04-04T06:15:13.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.534+0000 7f4866a00640 1 -- 172.21.15.159:0/2940552833 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4860068cd0 con 0x7f48600d0e70 2024-04-04T06:15:13.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.535+0000 7f485cc00640 1 -- 172.21.15.159:0/2940552833 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f48500039f0 con 0x7f48600d0e70 2024-04-04T06:15:13.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.537+0000 7f485cc00640 1 --2- 172.21.15.159:0/2940552833 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f483c06c8f0 0x7f483c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:13.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.537+0000 7f485cc00640 1 -- 172.21.15.159:0/2940552833 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f485002f030 con 0x7f48600d0e70 2024-04-04T06:15:13.531 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.537+0000 7f4857e00640 1 --2- 172.21.15.159:0/2940552833 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f483c06c8f0 0x7f483c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:13.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.539+0000 7f4857e00640 1 --2- 172.21.15.159:0/2940552833 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f483c06c8f0 0x7f483c06edb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f486015ac30 tx=0x7f4848008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:13.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.541+0000 7f485cc00640 1 -- 172.21.15.159:0/2940552833 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4850041070 con 0x7f48600d0e70 2024-04-04T06:15:13.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.821+0000 7f4866a00640 1 -- 172.21.15.159:0/2940552833 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f486008e580 con 0x7f48600d0e70 2024-04-04T06:15:13.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.824+0000 7f485cc00640 1 -- 172.21.15.159:0/2940552833 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f485007be50 con 0x7f48600d0e70 2024-04-04T06:15:13.819 INFO:teuthology.orchestra.run.smithi159.stdout:[client.0] 2024-04-04T06:15:13.819 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQD9Qg5mqNKJGxAA4jP9cadhRIyhQ3lssttWIw== 2024-04-04T06:15:13.819 INFO:teuthology.orchestra.run.smithi159.stdout: caps mds = "allow" 2024-04-04T06:15:13.819 INFO:teuthology.orchestra.run.smithi159.stdout: caps mon = "allow r" 2024-04-04T06:15:13.819 INFO:teuthology.orchestra.run.smithi159.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-04T06:15:13.819 INFO:teuthology.orchestra.run.smithi159.stderr:updated caps for client.0 2024-04-04T06:15:13.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.827+0000 7f4866a00640 1 -- 172.21.15.159:0/2940552833 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f483c06c8f0 msgr2=0x7f483c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:13.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.827+0000 7f4866a00640 1 --2- 172.21.15.159:0/2940552833 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f483c06c8f0 0x7f483c06edb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f486015ac30 tx=0x7f4848008040 comp rx=0 tx=0).stop 2024-04-04T06:15:13.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.827+0000 7f4866a00640 1 -- 172.21.15.159:0/2940552833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48600d0e70 msgr2=0x7f4860158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:13.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.827+0000 7f4866a00640 1 --2- 172.21.15.159:0/2940552833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48600d0e70 0x7f4860158150 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f48500097a0 tx=0x7f48500046c0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.827+0000 7f4866a00640 1 -- 172.21.15.159:0/2940552833 shutdown_connections 2024-04-04T06:15:13.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.827+0000 7f4866a00640 1 --2- 172.21.15.159:0/2940552833 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48601621d0 0x7f486014e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.827+0000 7f4866a00640 1 --2- 172.21.15.159:0/2940552833 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f483c06c8f0 0x7f483c06edb0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.827+0000 7f4866a00640 1 --2- 172.21.15.159:0/2940552833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f486015f6e0 0x7f486015af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.827+0000 7f4866a00640 1 --2- 172.21.15.159:0/2940552833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48600d0e70 0x7f4860158150 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:13.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.827+0000 7f4866a00640 1 -- 172.21.15.159:0/2940552833 >> 172.21.15.159:0/2940552833 conn(0x7f486008bfb0 msgr2=0x7f486015d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:13.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.828+0000 7f4866a00640 1 -- 172.21.15.159:0/2940552833 shutdown_connections 2024-04-04T06:15:13.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:13.828+0000 7f4866a00640 1 -- 172.21.15.159:0/2940552833 wait complete. 2024-04-04T06:15:13.843 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:15:14.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.052+0000 7f2942e00640 1 Processor -- start 2024-04-04T06:15:14.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.052+0000 7f2942e00640 1 -- start start 2024-04-04T06:15:14.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.054+0000 7f2942e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f293c15eab0 0x7f293c06c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:14.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.054+0000 7f2942e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f293c0b5990 0x7f293c06cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:14.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.054+0000 7f2942e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f293c0b53e0 con 0x7f293c072a00 2024-04-04T06:15:14.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.054+0000 7f2942e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f293c0b56c0 con 0x7f293c0b5990 2024-04-04T06:15:14.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.054+0000 7f2942e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f293c0adc80 con 0x7f293c15eab0 2024-04-04T06:15:14.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.054+0000 7f2933e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f293c0b5990 0x7f293c06cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:14.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.054+0000 7f2940e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f293c072a00 0x7f293c0b3990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41118/0 (socket says 172.21.15.159:41118) 2024-04-04T06:15:14.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.054+0000 7f2940e00640 1 -- 172.21.15.159:0/349976749 learned_addr learned my addr 172.21.15.159:0/349976749 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:14.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.055+0000 7f293be00640 1 --2- 172.21.15.159:0/349976749 >> v2:172.21.15.178:3300/0 conn(0x7f293c15eab0 0x7f293c06c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:14.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.055+0000 7f2933e00640 1 -- 172.21.15.159:0/349976749 >> v2:172.21.15.178:3300/0 conn(0x7f293c15eab0 msgr2=0x7f293c06c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:14.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.055+0000 7f2933e00640 1 --2- 172.21.15.159:0/349976749 >> v2:172.21.15.178:3300/0 conn(0x7f293c15eab0 0x7f293c06c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.055+0000 7f2933e00640 1 -- 172.21.15.159:0/349976749 >> v1:172.21.15.159:6789/0 conn(0x7f293c072a00 legacy=0x7f293c0b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:14.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.055+0000 7f2933e00640 1 -- 172.21.15.159:0/349976749 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293c0af490 con 0x7f293c0b5990 2024-04-04T06:15:14.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.056+0000 7f2933e00640 1 --2- 172.21.15.159:0/349976749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f293c0b5990 0x7f293c06cac0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f293c071aa0 tx=0x7f292402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dde0baee22a9fef server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:14.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.056+0000 7f293b400640 1 -- 172.21.15.159:0/349976749 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2924035020 con 0x7f293c0b5990 2024-04-04T06:15:14.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.056+0000 7f293b400640 1 -- 172.21.15.159:0/349976749 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2924031830 con 0x7f293c0b5990 2024-04-04T06:15:14.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.056+0000 7f293b400640 1 -- 172.21.15.159:0/349976749 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2924031c00 con 0x7f293c0b5990 2024-04-04T06:15:14.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.057+0000 7f2942e00640 1 -- 172.21.15.159:0/349976749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f293c0b5990 msgr2=0x7f293c06cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:14.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.057+0000 7f2942e00640 1 --2- 172.21.15.159:0/349976749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f293c0b5990 0x7f293c06cac0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f293c071aa0 tx=0x7f292402b8d0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.057+0000 7f2942e00640 1 -- 172.21.15.159:0/349976749 shutdown_connections 2024-04-04T06:15:14.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.057+0000 7f2942e00640 1 --2- 172.21.15.159:0/349976749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f293c0b5990 0x7f293c06cac0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.057+0000 7f2942e00640 1 --2- 172.21.15.159:0/349976749 >> v2:172.21.15.178:3300/0 conn(0x7f293c15eab0 0x7f293c06c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.057+0000 7f2942e00640 1 -- 172.21.15.159:0/349976749 >> 172.21.15.159:0/349976749 conn(0x7f293c071df0 msgr2=0x7f293c0721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:14.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.057+0000 7f2942e00640 1 -- 172.21.15.159:0/349976749 shutdown_connections 2024-04-04T06:15:14.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.057+0000 7f2942e00640 1 -- 172.21.15.159:0/349976749 wait complete. 2024-04-04T06:15:14.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.059+0000 7f2942e00640 1 Processor -- start 2024-04-04T06:15:14.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.059+0000 7f2942e00640 1 -- start start 2024-04-04T06:15:14.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.060+0000 7f2942e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f293c072a00 0x7f293c142d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:14.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.060+0000 7f2942e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f293c0b5990 0x7f293c155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:14.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.060+0000 7f2942e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f293c15eab0 0x7f293c15ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:14.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.060+0000 7f2942e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f293c0b4470 con 0x7f293c072a00 2024-04-04T06:15:14.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.060+0000 7f2942e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f293c0b42f0 con 0x7f293c0b5990 2024-04-04T06:15:14.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.060+0000 7f2942e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f293c0b45f0 con 0x7f293c15eab0 2024-04-04T06:15:14.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.060+0000 7f293be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f293c072a00 0x7f293c142d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:14.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.060+0000 7f293be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f293c072a00 0x7f293c142d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49424/0 (socket says 172.21.15.159:49424) 2024-04-04T06:15:14.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.060+0000 7f293be00640 1 -- 172.21.15.159:0/3458395208 learned_addr learned my addr 172.21.15.159:0/3458395208 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:14.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.060+0000 7f293be00640 1 -- 172.21.15.159:0/3458395208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f293c15eab0 msgr2=0x7f293c15ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:14.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.060+0000 7f293be00640 1 --2- 172.21.15.159:0/3458395208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f293c15eab0 0x7f293c15ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.060+0000 7f2933e00640 1 --2- 172.21.15.159:0/3458395208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f293c0b5990 0x7f293c155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:14.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.060+0000 7f293be00640 1 -- 172.21.15.159:0/3458395208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f293c0b5990 msgr2=0x7f293c155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:14.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.061+0000 7f293be00640 1 --2- 172.21.15.159:0/3458395208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f293c0b5990 0x7f293c155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.061+0000 7f293be00640 1 -- 172.21.15.159:0/3458395208 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293c1562f0 con 0x7f293c072a00 2024-04-04T06:15:14.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.061+0000 7f293be00640 1 --2- 172.21.15.159:0/3458395208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f293c072a00 0x7f293c142d30 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f292c00c7f0 tx=0x7f292c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:14.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.061+0000 7f2938c00640 1 -- 172.21.15.159:0/3458395208 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c015070 con 0x7f293c072a00 2024-04-04T06:15:14.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.061+0000 7f2938c00640 1 -- 172.21.15.159:0/3458395208 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f292c004220 con 0x7f293c072a00 2024-04-04T06:15:14.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.061+0000 7f2938c00640 1 -- 172.21.15.159:0/3458395208 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c007620 con 0x7f293c072a00 2024-04-04T06:15:14.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.061+0000 7f2942e00640 1 -- 172.21.15.159:0/3458395208 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f293c1565d0 con 0x7f293c072a00 2024-04-04T06:15:14.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.061+0000 7f2942e00640 1 -- 172.21.15.159:0/3458395208 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f293c156b90 con 0x7f293c072a00 2024-04-04T06:15:14.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.063+0000 7f2942e00640 1 -- 172.21.15.159:0/3458395208 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f293c0685c0 con 0x7f293c072a00 2024-04-04T06:15:14.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.063+0000 7f2938c00640 1 -- 172.21.15.159:0/3458395208 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f292c0079d0 con 0x7f293c072a00 2024-04-04T06:15:14.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.065+0000 7f2938c00640 1 --2- 172.21.15.159:0/3458395208 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f291806c8f0 0x7f291806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:14.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.065+0000 7f2938c00640 1 -- 172.21.15.159:0/3458395208 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f292c08bfb0 con 0x7f293c072a00 2024-04-04T06:15:14.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.065+0000 7f2933e00640 1 --2- 172.21.15.159:0/3458395208 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f291806c8f0 0x7f291806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:14.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.067+0000 7f2933e00640 1 --2- 172.21.15.159:0/3458395208 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f291806c8f0 0x7f291806edb0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2924036000 tx=0x7f2924008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:14.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.069+0000 7f2938c00640 1 -- 172.21.15.159:0/3458395208 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f292c05a340 con 0x7f293c072a00 2024-04-04T06:15:14.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.327+0000 7f2942e00640 1 -- 172.21.15.159:0/3458395208 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f293c0b2120 con 0x7f293c072a00 2024-04-04T06:15:14.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.328+0000 7f2938c00640 1 -- 172.21.15.159:0/3458395208 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5189 (secure 0 0 0) 0x7f292c059c40 con 0x7f293c072a00 2024-04-04T06:15:14.323 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:15:14.323 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":84,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5546,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/40219476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":40219476},{"type":"v1","addr":"172.21.15.178:6837","nonce":40219476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/4060483382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4060483382},{"type":"v1","addr":"172.21.15.178:6836","nonce":4060483382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8187,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1945665739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1945665739},{"type":"v1","addr":"172.21.15.159:6833","nonce":1945665739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:15:07.704835+0000","modified":"2024-04-04T06:15:10.811654+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8190":{"gid":8190,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6835/2253383212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2253383212},{"type":"v1","addr":"172.21.15.159:6835","nonce":2253383212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8190,"qdb_cluster":[8190]},"id":7}]} 2024-04-04T06:15:14.323 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 84 2024-04-04T06:15:14.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.331+0000 7f2942e00640 1 -- 172.21.15.159:0/3458395208 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f291806c8f0 msgr2=0x7f291806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:14.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.331+0000 7f2942e00640 1 --2- 172.21.15.159:0/3458395208 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f291806c8f0 0x7f291806edb0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2924036000 tx=0x7f2924008030 comp rx=0 tx=0).stop 2024-04-04T06:15:14.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.331+0000 7f2942e00640 1 -- 172.21.15.159:0/3458395208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f293c072a00 msgr2=0x7f293c142d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:14.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.331+0000 7f2942e00640 1 --2- 172.21.15.159:0/3458395208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f293c072a00 0x7f293c142d30 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f292c00c7f0 tx=0x7f292c00ccb0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.331+0000 7f2942e00640 1 -- 172.21.15.159:0/3458395208 shutdown_connections 2024-04-04T06:15:14.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.331+0000 7f2942e00640 1 --2- 172.21.15.159:0/3458395208 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f293c15eab0 0x7f293c15ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.331+0000 7f2942e00640 1 --2- 172.21.15.159:0/3458395208 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f291806c8f0 0x7f291806edb0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.331+0000 7f2942e00640 1 --2- 172.21.15.159:0/3458395208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f293c0b5990 0x7f293c155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.331+0000 7f2942e00640 1 --2- 172.21.15.159:0/3458395208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f293c072a00 0x7f293c142d30 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.331+0000 7f2942e00640 1 -- 172.21.15.159:0/3458395208 >> 172.21.15.159:0/3458395208 conn(0x7f293c071df0 msgr2=0x7f293c15dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:14.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.331+0000 7f2942e00640 1 -- 172.21.15.159:0/3458395208 shutdown_connections 2024-04-04T06:15:14.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.331+0000 7f2942e00640 1 -- 172.21.15.159:0/3458395208 wait complete. 2024-04-04T06:15:14.754 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 84, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:15:07.704835+0000', 'modified': '2024-04-04T06:15:10.811654+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8190}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8190': {'gid': 8190, 'name': 'a', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.159:6835/2253383212', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 2253383212}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 2253383212}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8190, 'qdb_cluster': [8190]} 2024-04-04T06:15:14.754 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-04T06:15:14.754 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-04T06:15:14.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.969+0000 7f9e47e00640 1 Processor -- start 2024-04-04T06:15:14.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.969+0000 7f9e47e00640 1 -- start start 2024-04-04T06:15:14.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.970+0000 7f9e47e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e40153580 0x7f9e400d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:14.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.970+0000 7f9e47e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9e400d07c0 0x7f9e400aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:14.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.970+0000 7f9e47e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9e400af2e0 0x7f9e4006d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:14.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.970+0000 7f9e47e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f9e400d0c00 con 0x7f9e400d07c0 2024-04-04T06:15:14.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.970+0000 7f9e47e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f9e400afab0 con 0x7f9e400af2e0 2024-04-04T06:15:14.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.970+0000 7f9e47e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9e4015cc80 con 0x7f9e40153580 2024-04-04T06:15:14.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.971+0000 7f9e44a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9e400d07c0 0x7f9e400aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:14.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.971+0000 7f9e44a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9e400d07c0 0x7f9e400aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49432/0 (socket says 172.21.15.159:49432) 2024-04-04T06:15:14.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.971+0000 7f9e44a00640 1 -- 172.21.15.159:0/3517863765 learned_addr learned my addr 172.21.15.159:0/3517863765 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:14.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.971+0000 7f9e45400640 1 --2- 172.21.15.159:0/3517863765 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e40153580 0x7f9e400d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:14.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.971+0000 7f9e44a00640 1 -- 172.21.15.159:0/3517863765 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e40153580 msgr2=0x7f9e400d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:14.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.971+0000 7f9e44a00640 1 --2- 172.21.15.159:0/3517863765 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e40153580 0x7f9e400d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.971+0000 7f9e44a00640 1 -- 172.21.15.159:0/3517863765 >> v2:172.21.15.178:3300/0 conn(0x7f9e400af2e0 msgr2=0x7f9e4006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:14.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.971+0000 7f9e45e00640 1 --2- 172.21.15.159:0/3517863765 >> v2:172.21.15.178:3300/0 conn(0x7f9e400af2e0 0x7f9e4006d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:14.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.971+0000 7f9e44a00640 1 --2- 172.21.15.159:0/3517863765 >> v2:172.21.15.178:3300/0 conn(0x7f9e400af2e0 0x7f9e4006d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.971+0000 7f9e44a00640 1 -- 172.21.15.159:0/3517863765 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4015da00 con 0x7f9e400d07c0 2024-04-04T06:15:14.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.972+0000 7f9e44a00640 1 --2- 172.21.15.159:0/3517863765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e400d07c0 0x7f9e400aeda0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f9e400cff70 tx=0x7f9e2c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b126e61f1d0324dd server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:14.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.973+0000 7f9e3fe00640 1 -- 172.21.15.159:0/3517863765 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c02e070 con 0x7f9e400d07c0 2024-04-04T06:15:14.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.973+0000 7f9e3fe00640 1 -- 172.21.15.159:0/3517863765 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e2c037070 con 0x7f9e400d07c0 2024-04-04T06:15:14.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.973+0000 7f9e3fe00640 1 -- 172.21.15.159:0/3517863765 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c031a00 con 0x7f9e400d07c0 2024-04-04T06:15:14.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.974+0000 7f9e47e00640 1 -- 172.21.15.159:0/3517863765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e400d07c0 msgr2=0x7f9e400aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:14.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.974+0000 7f9e47e00640 1 --2- 172.21.15.159:0/3517863765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e400d07c0 0x7f9e400aeda0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f9e400cff70 tx=0x7f9e2c02b880 comp rx=0 tx=0).stop 2024-04-04T06:15:14.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.974+0000 7f9e47e00640 1 -- 172.21.15.159:0/3517863765 shutdown_connections 2024-04-04T06:15:14.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.974+0000 7f9e47e00640 1 --2- 172.21.15.159:0/3517863765 >> v2:172.21.15.178:3300/0 conn(0x7f9e400af2e0 0x7f9e4006d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.974+0000 7f9e47e00640 1 --2- 172.21.15.159:0/3517863765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e400d07c0 0x7f9e400aeda0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.974+0000 7f9e47e00640 1 --2- 172.21.15.159:0/3517863765 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e40153580 0x7f9e400d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.974+0000 7f9e47e00640 1 -- 172.21.15.159:0/3517863765 >> 172.21.15.159:0/3517863765 conn(0x7f9e4006b750 msgr2=0x7f9e4006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:14.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.974+0000 7f9e47e00640 1 -- 172.21.15.159:0/3517863765 shutdown_connections 2024-04-04T06:15:14.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.974+0000 7f9e47e00640 1 -- 172.21.15.159:0/3517863765 wait complete. 2024-04-04T06:15:14.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.976+0000 7f9e47e00640 1 Processor -- start 2024-04-04T06:15:14.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.976+0000 7f9e47e00640 1 -- start start 2024-04-04T06:15:14.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.977+0000 7f9e47e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e400af2e0 0x7f9e40146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:14.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.977+0000 7f9e47e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e400d07c0 0x7f9e40145da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:14.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.977+0000 7f9e47e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e40153580 0x7f9e40147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:14.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.977+0000 7f9e47e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4015d4e0 con 0x7f9e400d07c0 2024-04-04T06:15:14.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.977+0000 7f9e47e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4015d200 con 0x7f9e400af2e0 2024-04-04T06:15:14.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.977+0000 7f9e47e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9e40153960 con 0x7f9e40153580 2024-04-04T06:15:14.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.977+0000 7f9e44a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e400d07c0 0x7f9e40145da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:14.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.977+0000 7f9e44a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e400d07c0 0x7f9e40145da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49434/0 (socket says 172.21.15.159:49434) 2024-04-04T06:15:14.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.978+0000 7f9e44a00640 1 -- 172.21.15.159:0/696303721 learned_addr learned my addr 172.21.15.159:0/696303721 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:14.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.978+0000 7f9e45400640 1 --2- 172.21.15.159:0/696303721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e400af2e0 0x7f9e40146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:14.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.978+0000 7f9e45e00640 1 --2- 172.21.15.159:0/696303721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e40153580 0x7f9e40147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:14.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.978+0000 7f9e44a00640 1 -- 172.21.15.159:0/696303721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e40153580 msgr2=0x7f9e40147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:14.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.978+0000 7f9e44a00640 1 --2- 172.21.15.159:0/696303721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e40153580 0x7f9e40147f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.978+0000 7f9e44a00640 1 -- 172.21.15.159:0/696303721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e400af2e0 msgr2=0x7f9e40146f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:14.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.978+0000 7f9e44a00640 1 --2- 172.21.15.159:0/696303721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e400af2e0 0x7f9e40146f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.978+0000 7f9e44a00640 1 -- 172.21.15.159:0/696303721 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e40147550 con 0x7f9e400d07c0 2024-04-04T06:15:14.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.978+0000 7f9e44a00640 1 --2- 172.21.15.159:0/696303721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e400d07c0 0x7f9e40145da0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c02b8c0 tx=0x7f9e2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:14.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.979+0000 7f9e3d600640 1 -- 172.21.15.159:0/696303721 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c032880 con 0x7f9e400d07c0 2024-04-04T06:15:14.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.979+0000 7f9e3d600640 1 -- 172.21.15.159:0/696303721 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e2c032a20 con 0x7f9e400d07c0 2024-04-04T06:15:14.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.979+0000 7f9e3d600640 1 -- 172.21.15.159:0/696303721 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c032e10 con 0x7f9e400d07c0 2024-04-04T06:15:14.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.979+0000 7f9e47e00640 1 -- 172.21.15.159:0/696303721 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e40147830 con 0x7f9e400d07c0 2024-04-04T06:15:14.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.979+0000 7f9e47e00640 1 -- 172.21.15.159:0/696303721 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e4014b850 con 0x7f9e400d07c0 2024-04-04T06:15:14.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.981+0000 7f9e3d600640 1 -- 172.21.15.159:0/696303721 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e2c0453f0 con 0x7f9e400d07c0 2024-04-04T06:15:14.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.982+0000 7f9e3d600640 1 --2- 172.21.15.159:0/696303721 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9e2006c8f0 0x7f9e2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:14.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.982+0000 7f9e3d600640 1 -- 172.21.15.159:0/696303721 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f9e2c0adbc0 con 0x7f9e400d07c0 2024-04-04T06:15:14.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.983+0000 7f9e4aa38740 1 Processor -- start 2024-04-04T06:15:14.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.983+0000 7f9e45400640 1 --2- 172.21.15.159:0/696303721 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9e2006c8f0 0x7f9e2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:14.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.983+0000 7f9e4aa38740 1 -- start start 2024-04-04T06:15:14.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.984+0000 7f9e4aa38740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931fe130 0x55dd931d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:14.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.984+0000 7f9e4aa38740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931fe510 0x55dd93200ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:14.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.984+0000 7f9e4aa38740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd931d1610 0x55dd93201400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:14.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.984+0000 7f9e4aa38740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dd93198ab0 con 0x55dd931fe510 2024-04-04T06:15:14.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.984+0000 7f9e4aa38740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dd9313afa0 con 0x55dd931fe130 2024-04-04T06:15:14.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.984+0000 7f9e4aa38740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55dd930e3560 con 0x55dd931d1610 2024-04-04T06:15:14.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.984+0000 7f9e45400640 1 --2- 172.21.15.159:0/696303721 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9e2006c8f0 0x7f9e2006edb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400a970 tx=0x7f9e34008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:14.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.984+0000 7f9e44a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931fe510 0x55dd93200ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:14.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.984+0000 7f9e44a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931fe510 0x55dd93200ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49446/0 (socket says 172.21.15.159:49446) 2024-04-04T06:15:14.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.984+0000 7f9e44a00640 1 -- 172.21.15.159:0/1666165587 learned_addr learned my addr 172.21.15.159:0/1666165587 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:14.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.984+0000 7f9e45400640 1 --2- 172.21.15.159:0/1666165587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931fe130 0x55dd931d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:14.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.984+0000 7f9e44a00640 1 -- 172.21.15.159:0/1666165587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd931d1610 msgr2=0x55dd93201400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:14.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.984+0000 7f9e45e00640 1 --2- 172.21.15.159:0/1666165587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd931d1610 0x55dd93201400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:14.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.984+0000 7f9e44a00640 1 --2- 172.21.15.159:0/1666165587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd931d1610 0x55dd93201400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.984+0000 7f9e44a00640 1 -- 172.21.15.159:0/1666165587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931fe130 msgr2=0x55dd931d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:14.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.984+0000 7f9e44a00640 1 --2- 172.21.15.159:0/1666165587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931fe130 0x55dd931d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.984+0000 7f9e44a00640 1 -- 172.21.15.159:0/1666165587 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd92e4f2d0 con 0x55dd931fe510 2024-04-04T06:15:14.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.984+0000 7f9e45400640 1 --2- 172.21.15.159:0/1666165587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931fe130 0x55dd931d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:14.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.985+0000 7f9e44a00640 1 --2- 172.21.15.159:0/1666165587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931fe510 0x55dd93200ec0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c006da0 tx=0x7f9e2c07b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96057a802b499d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:14.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.985+0000 7f9e31c00640 1 -- 172.21.15.159:0/1666165587 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c07fd20 con 0x55dd931fe510 2024-04-04T06:15:14.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.985+0000 7f9e31c00640 1 -- 172.21.15.159:0/1666165587 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e2c037040 con 0x55dd931fe510 2024-04-04T06:15:14.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.985+0000 7f9e31c00640 1 -- 172.21.15.159:0/1666165587 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c084550 con 0x55dd931fe510 2024-04-04T06:15:14.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.985+0000 7f9e4aa38740 1 -- 172.21.15.159:0/1666165587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931fe510 msgr2=0x55dd93200ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:14.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.985+0000 7f9e4aa38740 1 --2- 172.21.15.159:0/1666165587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931fe510 0x55dd93200ec0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c006da0 tx=0x7f9e2c07b390 comp rx=0 tx=0).stop 2024-04-04T06:15:14.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.986+0000 7f9e4aa38740 1 -- 172.21.15.159:0/1666165587 shutdown_connections 2024-04-04T06:15:14.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.986+0000 7f9e4aa38740 1 --2- 172.21.15.159:0/1666165587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd931d1610 0x55dd93201400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.986+0000 7f9e4aa38740 1 --2- 172.21.15.159:0/1666165587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931fe510 0x55dd93200ec0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.986+0000 7f9e4aa38740 1 --2- 172.21.15.159:0/1666165587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931fe130 0x55dd931d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.986+0000 7f9e4aa38740 1 -- 172.21.15.159:0/1666165587 >> 172.21.15.159:0/1666165587 conn(0x55dd931d3340 msgr2=0x55dd9321bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:14.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.986+0000 7f9e4aa38740 1 -- 172.21.15.159:0/1666165587 shutdown_connections 2024-04-04T06:15:14.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.986+0000 7f9e4aa38740 1 -- 172.21.15.159:0/1666165587 wait complete. 2024-04-04T06:15:14.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.986+0000 7f9e4aa38740 1 -- start start 2024-04-04T06:15:14.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.987+0000 7f9e4aa38740 1 Processor -- start 2024-04-04T06:15:14.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.987+0000 7f9e4aa38740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd931fe130 0x55dd931efe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:14.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.987+0000 7f9e4aa38740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931fe510 0x55dd93216e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:14.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.988+0000 7f9e4aa38740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931ef680 0x55dd93215b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:14.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.988+0000 7f9e4aa38740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dd930f6920 con 0x55dd931fe510 2024-04-04T06:15:14.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.988+0000 7f9e4aa38740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dd93136f50 con 0x55dd931ef680 2024-04-04T06:15:14.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.988+0000 7f9e4aa38740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55dd930e3560 con 0x55dd931fe130 2024-04-04T06:15:14.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.988+0000 7f9e4aa38740 10 client.0 _ll_register_callbacks cb 0x55dd930f6310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:15:14.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.988+0000 7f9e44a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931fe510 0x55dd93216e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:14.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.988+0000 7f9e44a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931fe510 0x55dd93216e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49456/0 (socket says 172.21.15.159:49456) 2024-04-04T06:15:14.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.988+0000 7f9e44a00640 1 -- 172.21.15.159:0/1324862465 learned_addr learned my addr 172.21.15.159:0/1324862465 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:14.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.988+0000 7f9e45400640 1 --2- 172.21.15.159:0/1324862465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd931fe130 0x55dd931efe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:14.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.988+0000 7f9e44a00640 1 -- 172.21.15.159:0/1324862465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd931fe130 msgr2=0x55dd931efe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:14.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.988+0000 7f9e45e00640 1 --2- 172.21.15.159:0/1324862465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931ef680 0x55dd93215b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:14.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.988+0000 7f9e44a00640 1 --2- 172.21.15.159:0/1324862465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd931fe130 0x55dd931efe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.988+0000 7f9e44a00640 1 -- 172.21.15.159:0/1324862465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931ef680 msgr2=0x55dd93215b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:14.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.988+0000 7f9e44a00640 1 --2- 172.21.15.159:0/1324862465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931ef680 0x55dd93215b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.988+0000 7f9e44a00640 1 -- 172.21.15.159:0/1324862465 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd92f196b0 con 0x55dd931fe510 2024-04-04T06:15:14.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.988+0000 7f9e45400640 1 --2- 172.21.15.159:0/1324862465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd931fe130 0x55dd931efe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:14.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.988+0000 7f9e44a00640 1 --2- 172.21.15.159:0/1324862465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931fe510 0x55dd93216e70 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c077dd0 tx=0x7f9e2c006cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d055fe2a3a046a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:14.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.988+0000 7f9e26000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:15:14.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.989+0000 7f9e26000640 1 -- 172.21.15.159:0/1324862465 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c07fd20 con 0x55dd931fe510 2024-04-04T06:15:14.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.989+0000 7f9e26000640 1 -- 172.21.15.159:0/1324862465 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f9e2c085e50 con 0x55dd931fe510 2024-04-04T06:15:14.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.989+0000 7f9e4aa38740 1 -- 172.21.15.159:0/1324862465 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e2c037040 con 0x55dd931fe510 2024-04-04T06:15:14.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.989+0000 7f9e26000640 1 -- 172.21.15.159:0/1324862465 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f9e2c0840c0 con 0x55dd931fe510 2024-04-04T06:15:14.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.989+0000 7f9e4aa38740 10 client.8337 fetch_fsmap learned FSMap version 84 2024-04-04T06:15:14.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.989+0000 7f9e4aa38740 1 -- 172.21.15.159:0/1324862465 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x55dd93215030 con 0x55dd931fe510 2024-04-04T06:15:14.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.989+0000 7f9e26000640 1 -- 172.21.15.159:0/1324862465 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e2c037040 con 0x55dd931fe510 2024-04-04T06:15:14.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.990+0000 7f9e26000640 1 -- 172.21.15.159:0/1324862465 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7f9e2c07adc0 con 0x55dd931fe510 2024-04-04T06:15:14.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.990+0000 7f9e4aa38740 10 client.8337 fetch_fsmap finished waiting for FSMap version 84 2024-04-04T06:15:14.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.990+0000 7f9e4aa38740 10 client.8337 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:15:14.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.990+0000 7f9e4aa38740 20 client.8337 populate_metadata read hostname 'smithi159' 2024-04-04T06:15:14.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.990+0000 7f9e4aa38740 1 --2- 172.21.15.159:0/1324862465 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x55dd93203d10 0x55dd932129e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:14.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.990+0000 7f9e4aa38740 4 client.8337 mds_command: new command op to 8190 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:15:14.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.990+0000 7f9e4aa38740 1 -- 172.21.15.159:0/1324862465 --> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd931484e0 con 0x55dd93203d10 2024-04-04T06:15:14.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.990+0000 7f9e45400640 1 --2- 172.21.15.159:0/1324862465 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x55dd93203d10 0x55dd932129e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:14.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.991+0000 7f9e26000640 10 client.8337 ms_handle_connect on v2:172.21.15.159:6834/2253383212 2024-04-04T06:15:14.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.991+0000 7f9e45400640 1 --2- 172.21.15.159:0/1324862465 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x55dd93203d10 0x55dd932129e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ca44c2c3a79ddee server_cookie=398c3473e15f43dc in_seq=0 out_seq=0 2024-04-04T06:15:14.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.992+0000 7f9e26000640 1 -- 172.21.15.159:0/1324862465 <== mds.0 v2:172.21.15.159:6834/2253383212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd931484e0 con 0x55dd93203d10 2024-04-04T06:15:14.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.992+0000 7f9e26000640 10 client.8337 handle_command_reply: tid=0 2024-04-04T06:15:14.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.992+0000 7f9e4aa38740 1 client.8337 shutdown 2024-04-04T06:15:14.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.992+0000 7f9e4aa38740 1 -- 172.21.15.159:0/1324862465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931fe510 msgr2=0x55dd93216e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:14.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.992+0000 7f9e4aa38740 1 --2- 172.21.15.159:0/1324862465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931fe510 0x55dd93216e70 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c077dd0 tx=0x7f9e2c006cf0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.992+0000 7f9e4aa38740 1 -- 172.21.15.159:0/1324862465 shutdown_connections 2024-04-04T06:15:14.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.992+0000 7f9e4aa38740 1 --2- 172.21.15.159:0/1324862465 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x55dd93203d10 0x55dd932129e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.992+0000 7f9e4aa38740 1 --2- 172.21.15.159:0/1324862465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931ef680 0x55dd93215b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.992+0000 7f9e4aa38740 1 --2- 172.21.15.159:0/1324862465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931fe510 0x55dd93216e70 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.992+0000 7f9e4aa38740 1 --2- 172.21.15.159:0/1324862465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd931fe130 0x55dd931efe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:14.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.992+0000 7f9e26000640 0 client.8337 ms_handle_reset on v2:172.21.15.159:6834/2253383212 2024-04-04T06:15:14.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.992+0000 7f9e4aa38740 1 -- 172.21.15.159:0/1324862465 >> 172.21.15.159:0/1324862465 conn(0x55dd931d1610 msgr2=0x55dd932009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:14.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.992+0000 7f9e4aa38740 1 -- 172.21.15.159:0/1324862465 shutdown_connections 2024-04-04T06:15:14.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.992+0000 7f9e4aa38740 1 -- 172.21.15.159:0/1324862465 wait complete. 2024-04-04T06:15:14.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:14.993+0000 7f9e4aa38740 20 client.8337 trim_cache size 0 max 16384 2024-04-04T06:15:15.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.012+0000 7f9e4aa38740 1 Processor -- start 2024-04-04T06:15:15.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.012+0000 7f9e4aa38740 1 -- start start 2024-04-04T06:15:15.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.013+0000 7f9e4aa38740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931ef680 0x55dd93204f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:15.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.013+0000 7f9e4aa38740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931fe130 0x55dd93200b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:15.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.013+0000 7f9e4aa38740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd931fe510 0x55dd931fc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:15.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.013+0000 7f9e4aa38740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9e401ffd10 con 0x55dd931ef680 2024-04-04T06:15:15.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.013+0000 7f9e4aa38740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9e40073430 con 0x55dd931fe130 2024-04-04T06:15:15.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.013+0000 7f9e4aa38740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9e4014cd70 con 0x55dd931fe510 2024-04-04T06:15:15.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.014+0000 7f9e45400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931ef680 0x55dd93204f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:15.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.014+0000 7f9e45400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931ef680 0x55dd93204f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49472/0 (socket says 172.21.15.159:49472) 2024-04-04T06:15:15.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.014+0000 7f9e45400640 1 -- 172.21.15.159:0/326323944 learned_addr learned my addr 172.21.15.159:0/326323944 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:15.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.014+0000 7f9e44a00640 1 --2- 172.21.15.159:0/326323944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931fe130 0x55dd93200b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:15.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.014+0000 7f9e45e00640 1 --2- 172.21.15.159:0/326323944 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd931fe510 0x55dd931fc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:15.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.014+0000 7f9e45400640 1 -- 172.21.15.159:0/326323944 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd931fe510 msgr2=0x55dd931fc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:15.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.014+0000 7f9e45400640 1 --2- 172.21.15.159:0/326323944 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd931fe510 0x55dd931fc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.014+0000 7f9e45400640 1 -- 172.21.15.159:0/326323944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931fe130 msgr2=0x55dd93200b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:15.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.014+0000 7f9e45400640 1 --2- 172.21.15.159:0/326323944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931fe130 0x55dd93200b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.014+0000 7f9e45400640 1 -- 172.21.15.159:0/326323944 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd930dacc0 con 0x55dd931ef680 2024-04-04T06:15:15.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.014+0000 7f9e45400640 1 --2- 172.21.15.159:0/326323944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931ef680 0x55dd93204f80 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400e5a0 tx=0x7f9e34014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edebaba65f1d7ea server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:15.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.015+0000 7f9e31200640 1 -- 172.21.15.159:0/326323944 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3402c070 con 0x55dd931ef680 2024-04-04T06:15:15.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.015+0000 7f9e31200640 1 -- 172.21.15.159:0/326323944 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e3400ed00 con 0x55dd931ef680 2024-04-04T06:15:15.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.015+0000 7f9e31200640 1 -- 172.21.15.159:0/326323944 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e340263f0 con 0x55dd931ef680 2024-04-04T06:15:15.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.015+0000 7f9e4aa38740 1 -- 172.21.15.159:0/326323944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931ef680 msgr2=0x55dd93204f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:15.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.015+0000 7f9e4aa38740 1 --2- 172.21.15.159:0/326323944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931ef680 0x55dd93204f80 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400e5a0 tx=0x7f9e34014000 comp rx=0 tx=0).stop 2024-04-04T06:15:15.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.015+0000 7f9e4aa38740 1 -- 172.21.15.159:0/326323944 shutdown_connections 2024-04-04T06:15:15.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.015+0000 7f9e4aa38740 1 --2- 172.21.15.159:0/326323944 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd931fe510 0x55dd931fc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.015+0000 7f9e4aa38740 1 --2- 172.21.15.159:0/326323944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931fe130 0x55dd93200b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.015+0000 7f9e4aa38740 1 --2- 172.21.15.159:0/326323944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931ef680 0x55dd93204f80 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.015+0000 7f9e4aa38740 1 -- 172.21.15.159:0/326323944 >> 172.21.15.159:0/326323944 conn(0x55dd931d1610 msgr2=0x55dd931cc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:15.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.016+0000 7f9e4aa38740 1 -- 172.21.15.159:0/326323944 shutdown_connections 2024-04-04T06:15:15.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.016+0000 7f9e4aa38740 1 -- 172.21.15.159:0/326323944 wait complete. 2024-04-04T06:15:15.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.016+0000 7f9e4aa38740 1 -- start start 2024-04-04T06:15:15.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.016+0000 7f9e4aa38740 1 Processor -- start 2024-04-04T06:15:15.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.017+0000 7f9e4aa38740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd931ef680 0x55dd93200b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:15.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.017+0000 7f9e4aa38740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931fe130 0x55dd931d5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:15.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.017+0000 7f9e4aa38740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931fe510 0x55dd9321d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:15.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.017+0000 7f9e4aa38740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dd9313afa0 con 0x55dd931fe130 2024-04-04T06:15:15.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.017+0000 7f9e4aa38740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dd93198ab0 con 0x55dd931fe510 2024-04-04T06:15:15.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.017+0000 7f9e4aa38740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55dd931f0590 con 0x55dd931ef680 2024-04-04T06:15:15.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.017+0000 7f9e4aa38740 10 client.0 _ll_register_callbacks cb 0x7f9e2c094020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:15:15.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.018+0000 7f9e44a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931fe130 0x55dd931d5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:15.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.018+0000 7f9e44a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931fe130 0x55dd931d5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49488/0 (socket says 172.21.15.159:49488) 2024-04-04T06:15:15.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.018+0000 7f9e44a00640 1 -- 172.21.15.159:0/3624054944 learned_addr learned my addr 172.21.15.159:0/3624054944 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:15.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.018+0000 7f9e45e00640 1 --2- 172.21.15.159:0/3624054944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931fe510 0x55dd9321d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:15.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.018+0000 7f9e45400640 1 --2- 172.21.15.159:0/3624054944 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd931ef680 0x55dd93200b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:15.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.018+0000 7f9e44a00640 1 -- 172.21.15.159:0/3624054944 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd931ef680 msgr2=0x55dd93200b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:15.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.018+0000 7f9e44a00640 1 --2- 172.21.15.159:0/3624054944 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd931ef680 0x55dd93200b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.018+0000 7f9e44a00640 1 -- 172.21.15.159:0/3624054944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931fe510 msgr2=0x55dd9321d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:15.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.018+0000 7f9e44a00640 1 --2- 172.21.15.159:0/3624054944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931fe510 0x55dd9321d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.018+0000 7f9e44a00640 1 -- 172.21.15.159:0/3624054944 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd931d0250 con 0x55dd931fe130 2024-04-04T06:15:15.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.018+0000 7f9e45e00640 1 --2- 172.21.15.159:0/3624054944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931fe510 0x55dd9321d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:15.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.018+0000 7f9e44a00640 1 --2- 172.21.15.159:0/3624054944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931fe130 0x55dd931d5f40 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c032fd0 tx=0x7f9e2c082040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c39d0b21e850d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:15.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.018+0000 7f9e26000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:15:15.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.019+0000 7f9e26000640 1 -- 172.21.15.159:0/3624054944 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c07fd20 con 0x55dd931fe130 2024-04-04T06:15:15.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.019+0000 7f9e26000640 1 -- 172.21.15.159:0/3624054944 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f9e2c084260 con 0x55dd931fe130 2024-04-04T06:15:15.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.019+0000 7f9e4aa38740 1 -- 172.21.15.159:0/3624054944 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e3400ed00 con 0x55dd931fe130 2024-04-04T06:15:15.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.019+0000 7f9e26000640 1 -- 172.21.15.159:0/3624054944 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f9e2c07b4b0 con 0x55dd931fe130 2024-04-04T06:15:15.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.019+0000 7f9e4aa38740 10 client.8343 fetch_fsmap learned FSMap version 84 2024-04-04T06:15:15.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.019+0000 7f9e4aa38740 1 -- 172.21.15.159:0/3624054944 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x55dd931d0250 con 0x55dd931fe130 2024-04-04T06:15:15.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.019+0000 7f9e26000640 1 -- 172.21.15.159:0/3624054944 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e3400ed00 con 0x55dd931fe130 2024-04-04T06:15:15.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.019+0000 7f9e26000640 1 -- 172.21.15.159:0/3624054944 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7f9e2c0776c0 con 0x55dd931fe130 2024-04-04T06:15:15.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.019+0000 7f9e4aa38740 10 client.8343 fetch_fsmap finished waiting for FSMap version 84 2024-04-04T06:15:15.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.019+0000 7f9e4aa38740 10 client.8343 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:15:15.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.019+0000 7f9e4aa38740 20 client.8343 populate_metadata read hostname 'smithi159' 2024-04-04T06:15:15.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.019+0000 7f9e4aa38740 1 --2- 172.21.15.159:0/3624054944 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x55dd9321d600 0x55dd931fc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:15.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.019+0000 7f9e4aa38740 4 client.8343 mds_command: new command op to 8190 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-04T06:15:15.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.019+0000 7f9e4aa38740 1 -- 172.21.15.159:0/3624054944 --> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] -- command(tid 0: {"prefix": "status"}) -- 0x55dd93204af0 con 0x55dd9321d600 2024-04-04T06:15:15.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.020+0000 7f9e45400640 1 --2- 172.21.15.159:0/3624054944 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x55dd9321d600 0x55dd931fc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:15.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.020+0000 7f9e45400640 1 --2- 172.21.15.159:0/3624054944 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x55dd9321d600 0x55dd931fc840 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4620f812a47a7b61 server_cookie=8635cea9d450ceb4 in_seq=0 out_seq=0 2024-04-04T06:15:15.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.020+0000 7f9e26000640 10 client.8343 ms_handle_connect on v2:172.21.15.159:6834/2253383212 2024-04-04T06:15:15.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.020+0000 7f9e26000640 1 -- 172.21.15.159:0/3624054944 <== mds.0 v2:172.21.15.159:6834/2253383212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55dd93204af0 con 0x55dd9321d600 2024-04-04T06:15:15.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.020+0000 7f9e26000640 10 client.8343 handle_command_reply: tid=0 2024-04-04T06:15:15.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.020+0000 7f9e4aa38740 1 client.8343 shutdown 2024-04-04T06:15:15.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.020+0000 7f9e4aa38740 1 -- 172.21.15.159:0/3624054944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931fe130 msgr2=0x55dd931d5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:15.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.020+0000 7f9e4aa38740 1 --2- 172.21.15.159:0/3624054944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931fe130 0x55dd931d5f40 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c032fd0 tx=0x7f9e2c082040 comp rx=0 tx=0).stop 2024-04-04T06:15:15.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.021+0000 7f9e4aa38740 1 -- 172.21.15.159:0/3624054944 shutdown_connections 2024-04-04T06:15:15.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.021+0000 7f9e4aa38740 1 --2- 172.21.15.159:0/3624054944 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x55dd9321d600 0x55dd931fc840 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.021+0000 7f9e4aa38740 1 --2- 172.21.15.159:0/3624054944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dd931fe510 0x55dd9321d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.021+0000 7f9e4aa38740 1 --2- 172.21.15.159:0/3624054944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd931fe130 0x55dd931d5f40 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.021+0000 7f9e26000640 0 client.8343 ms_handle_reset on v2:172.21.15.159:6834/2253383212 2024-04-04T06:15:15.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.021+0000 7f9e4aa38740 1 --2- 172.21.15.159:0/3624054944 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55dd931ef680 0x55dd93200b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.021+0000 7f9e4aa38740 1 -- 172.21.15.159:0/3624054944 >> 172.21.15.159:0/3624054944 conn(0x55dd931d1610 msgr2=0x55dd93216cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:15.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.021+0000 7f9e4aa38740 1 -- 172.21.15.159:0/3624054944 shutdown_connections 2024-04-04T06:15:15.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.021+0000 7f9e4aa38740 1 -- 172.21.15.159:0/3624054944 wait complete. 2024-04-04T06:15:15.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.021+0000 7f9e4aa38740 20 client.8343 trim_cache size 0 max 16384 2024-04-04T06:15:15.020 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:15:15.020 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_fsid": "30311045-f714-48fa-832b-4cb2ba8bf0bf", 2024-04-04T06:15:15.020 INFO:teuthology.orchestra.run.smithi159.stdout: "whoami": 0, 2024-04-04T06:15:15.020 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 8190, 2024-04-04T06:15:15.020 INFO:teuthology.orchestra.run.smithi159.stdout: "want_state": "up:active", 2024-04-04T06:15:15.020 INFO:teuthology.orchestra.run.smithi159.stdout: "state": "up:active", 2024-04-04T06:15:15.020 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs", 2024-04-04T06:15:15.020 INFO:teuthology.orchestra.run.smithi159.stdout: "rank_uptime": 6.7864813399999999, 2024-04-04T06:15:15.020 INFO:teuthology.orchestra.run.smithi159.stdout: "mdsmap_epoch": 84, 2024-04-04T06:15:15.021 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch": 101, 2024-04-04T06:15:15.021 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch_barrier": 101, 2024-04-04T06:15:15.021 INFO:teuthology.orchestra.run.smithi159.stdout: "uptime": 13.195232535000001 2024-04-04T06:15:15.021 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:15:15.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.022+0000 7f9e47e00640 1 -- 172.21.15.159:0/696303721 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9e2006c8f0 msgr2=0x7f9e2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:15.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.022+0000 7f9e47e00640 1 --2- 172.21.15.159:0/696303721 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9e2006c8f0 0x7f9e2006edb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400a970 tx=0x7f9e34008040 comp rx=0 tx=0).stop 2024-04-04T06:15:15.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.022+0000 7f9e47e00640 1 -- 172.21.15.159:0/696303721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e400d07c0 msgr2=0x7f9e40145da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:15.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.022+0000 7f9e47e00640 1 --2- 172.21.15.159:0/696303721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e400d07c0 0x7f9e40145da0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c02b8c0 tx=0x7f9e2c0046c0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.022+0000 7f9e47e00640 1 -- 172.21.15.159:0/696303721 shutdown_connections 2024-04-04T06:15:15.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.022+0000 7f9e47e00640 1 --2- 172.21.15.159:0/696303721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e40153580 0x7f9e40147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.022+0000 7f9e47e00640 1 --2- 172.21.15.159:0/696303721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e400d07c0 0x7f9e40145da0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.022+0000 7f9e47e00640 1 --2- 172.21.15.159:0/696303721 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9e2006c8f0 0x7f9e2006edb0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.022+0000 7f9e47e00640 1 --2- 172.21.15.159:0/696303721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e400af2e0 0x7f9e40146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.022+0000 7f9e47e00640 1 -- 172.21.15.159:0/696303721 >> 172.21.15.159:0/696303721 conn(0x7f9e4006b750 msgr2=0x7f9e4015dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:15.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.022+0000 7f9e47e00640 1 -- 172.21.15.159:0/696303721 shutdown_connections 2024-04-04T06:15:15.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.022+0000 7f9e47e00640 1 -- 172.21.15.159:0/696303721 wait complete. 2024-04-04T06:15:15.039 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-04T06:15:15.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.236+0000 7fd670000640 1 Processor -- start 2024-04-04T06:15:15.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.236+0000 7fd670000640 1 -- start start 2024-04-04T06:15:15.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.237+0000 7fd670000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd668153580 0x7fd6680d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:15.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.238+0000 7fd670000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6680afab0 con 0x7fd6680af4b0 2024-04-04T06:15:15.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.238+0000 7fd670000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd66815cc80 con 0x7fd6680d07c0 2024-04-04T06:15:15.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.238+0000 7fd670000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd6680d0c30 con 0x7fd668153580 2024-04-04T06:15:15.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.238+0000 7fd66e000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fd6680af4b0 0x7fd6681600d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41128/0 (socket says 172.21.15.159:41128) 2024-04-04T06:15:15.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.238+0000 7fd66e000640 1 -- 172.21.15.159:0/1087417095 learned_addr learned my addr 172.21.15.159:0/1087417095 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:15.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.238+0000 7fd66d600640 1 --2- 172.21.15.159:0/1087417095 >> v2:172.21.15.178:3300/0 conn(0x7fd668153580 0x7fd6680d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:15.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.238+0000 7fd667e00640 1 -- 172.21.15.159:0/1087417095 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1173394040 0 0) 0x7fd6680afab0 con 0x7fd6680af4b0 2024-04-04T06:15:15.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.238+0000 7fd667e00640 1 -- 172.21.15.159:0/1087417095 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd650003520 con 0x7fd6680af4b0 2024-04-04T06:15:15.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.239+0000 7fd667e00640 1 -- 172.21.15.159:0/1087417095 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3249081051 0 0) 0x7fd650003520 con 0x7fd6680af4b0 2024-04-04T06:15:15.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.239+0000 7fd667e00640 1 -- 172.21.15.159:0/1087417095 >> v2:172.21.15.178:3300/0 conn(0x7fd668153580 msgr2=0x7fd6680d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:15.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.239+0000 7fd667e00640 1 --2- 172.21.15.159:0/1087417095 >> v2:172.21.15.178:3300/0 conn(0x7fd668153580 0x7fd6680d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.239+0000 7fd667e00640 1 -- 172.21.15.159:0/1087417095 >> v1:172.21.15.178:6789/0 conn(0x7fd6680d07c0 legacy=0x7fd6680aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:15.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.239+0000 7fd667e00640 1 -- 172.21.15.159:0/1087417095 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66815da00 con 0x7fd6680af4b0 2024-04-04T06:15:15.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.239+0000 7fd667e00640 1 -- 172.21.15.159:0/1087417095 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fd6600019f0 con 0x7fd6680af4b0 2024-04-04T06:15:15.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.239+0000 7fd667e00640 1 -- 172.21.15.159:0/1087417095 >> v1:172.21.15.159:6789/0 conn(0x7fd6680af4b0 legacy=0x7fd6681600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:15.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.239+0000 7fd667e00640 1 --2- 172.21.15.159:0/1087417095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd650003a00 0x7fd650003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:15.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.239+0000 7fd667e00640 1 --2- 172.21.15.159:0/1087417095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd650004800 0x7fd650004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:15.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.239+0000 7fd667e00640 1 --2- 172.21.15.159:0/1087417095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd650005210 0x7fd65000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:15.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.239+0000 7fd667e00640 1 -- 172.21.15.159:0/1087417095 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd6680d0c30 con 0x7fd650005210 2024-04-04T06:15:15.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.239+0000 7fd667e00640 1 -- 172.21.15.159:0/1087417095 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd65000c160 con 0x7fd650004800 2024-04-04T06:15:15.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.239+0000 7fd667e00640 1 -- 172.21.15.159:0/1087417095 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd65000c360 con 0x7fd650003a00 2024-04-04T06:15:15.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.239+0000 7fd66e000640 1 --2- 172.21.15.159:0/1087417095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd650005210 0x7fd65000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:15.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.240+0000 7fd66e000640 1 -- 172.21.15.159:0/1087417095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd650003a00 msgr2=0x7fd650003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:15.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.240+0000 7fd66e000640 1 --2- 172.21.15.159:0/1087417095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd650003a00 0x7fd650003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.240+0000 7fd66cc00640 1 --2- 172.21.15.159:0/1087417095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd650004800 0x7fd650004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:15.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.240+0000 7fd66e000640 1 -- 172.21.15.159:0/1087417095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd650004800 msgr2=0x7fd650004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:15.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.240+0000 7fd66e000640 1 --2- 172.21.15.159:0/1087417095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd650004800 0x7fd650004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.240+0000 7fd66e000640 1 -- 172.21.15.159:0/1087417095 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd650003520 con 0x7fd650005210 2024-04-04T06:15:15.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.240+0000 7fd66cc00640 1 --2- 172.21.15.159:0/1087417095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd650004800 0x7fd650004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:15.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.241+0000 7fd66e000640 1 -- 172.21.15.159:0/1087417095 reap_dead start 2024-04-04T06:15:15.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.241+0000 7fd66e000640 1 --2- 172.21.15.159:0/1087417095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd650005210 0x7fd65000b9c0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fd66000a760 tx=0x7fd650005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55dd46398ce717cb server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:15.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.241+0000 7fd667e00640 1 -- 172.21.15.159:0/1087417095 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66000a3b0 con 0x7fd650005210 2024-04-04T06:15:15.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.241+0000 7fd667e00640 1 -- 172.21.15.159:0/1087417095 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd660034630 con 0x7fd650005210 2024-04-04T06:15:15.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.241+0000 7fd667e00640 1 -- 172.21.15.159:0/1087417095 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660034a20 con 0x7fd650005210 2024-04-04T06:15:15.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.242+0000 7fd670000640 1 -- 172.21.15.159:0/1087417095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd650005210 msgr2=0x7fd65000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:15.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.242+0000 7fd670000640 1 --2- 172.21.15.159:0/1087417095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd650005210 0x7fd65000b9c0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fd66000a760 tx=0x7fd650005830 comp rx=0 tx=0).stop 2024-04-04T06:15:15.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.242+0000 7fd670000640 1 -- 172.21.15.159:0/1087417095 shutdown_connections 2024-04-04T06:15:15.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.242+0000 7fd670000640 1 --2- 172.21.15.159:0/1087417095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd650005210 0x7fd65000b9c0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.242+0000 7fd670000640 1 -- 172.21.15.159:0/1087417095 >> 172.21.15.159:0/1087417095 conn(0x7fd66806b750 msgr2=0x7fd66806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:15.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.242+0000 7fd670000640 1 -- 172.21.15.159:0/1087417095 shutdown_connections 2024-04-04T06:15:15.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.242+0000 7fd670000640 1 -- 172.21.15.159:0/1087417095 wait complete. 2024-04-04T06:15:15.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.245+0000 7fd670000640 1 Processor -- start 2024-04-04T06:15:15.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.245+0000 7fd670000640 1 -- start start 2024-04-04T06:15:15.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.246+0000 7fd670000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd650005210 0x7fd66814d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:15.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.246+0000 7fd670000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6680716a0 0x7fd668072500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:15.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.246+0000 7fd670000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd668156060 0x7fd6681564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:15.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.246+0000 7fd670000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd66815d4e0 con 0x7fd668156060 2024-04-04T06:15:15.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.246+0000 7fd670000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd66815d200 con 0x7fd6680716a0 2024-04-04T06:15:15.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.246+0000 7fd670000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd668153960 con 0x7fd650005210 2024-04-04T06:15:15.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.247+0000 7fd66e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd668156060 0x7fd6681564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:15.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.247+0000 7fd66e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd668156060 0x7fd6681564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49494/0 (socket says 172.21.15.159:49494) 2024-04-04T06:15:15.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.247+0000 7fd66e000640 1 -- 172.21.15.159:0/2397028802 learned_addr learned my addr 172.21.15.159:0/2397028802 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:15.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.247+0000 7fd66d600640 1 --2- 172.21.15.159:0/2397028802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd650005210 0x7fd66814d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:15.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.247+0000 7fd66e000640 1 -- 172.21.15.159:0/2397028802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd650005210 msgr2=0x7fd66814d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:15.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.247+0000 7fd66e000640 1 --2- 172.21.15.159:0/2397028802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd650005210 0x7fd66814d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.247+0000 7fd66e000640 1 -- 172.21.15.159:0/2397028802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6680716a0 msgr2=0x7fd668072500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:15.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.247+0000 7fd66e000640 1 --2- 172.21.15.159:0/2397028802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6680716a0 0x7fd668072500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.247+0000 7fd66e000640 1 -- 172.21.15.159:0/2397028802 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66814e0b0 con 0x7fd668156060 2024-04-04T06:15:15.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.247+0000 7fd66e000640 1 --2- 172.21.15.159:0/2397028802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd668156060 0x7fd6681564e0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd66002cb80 tx=0x7fd6680b46b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:15.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.248+0000 7fd665600640 1 -- 172.21.15.159:0/2397028802 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660001250 con 0x7fd668156060 2024-04-04T06:15:15.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.248+0000 7fd665600640 1 -- 172.21.15.159:0/2397028802 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd660044070 con 0x7fd668156060 2024-04-04T06:15:15.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.248+0000 7fd670000640 1 -- 172.21.15.159:0/2397028802 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd668072ea0 con 0x7fd668156060 2024-04-04T06:15:15.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.248+0000 7fd665600640 1 -- 172.21.15.159:0/2397028802 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6600347e0 con 0x7fd668156060 2024-04-04T06:15:15.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.248+0000 7fd670000640 1 -- 172.21.15.159:0/2397028802 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd66814e650 con 0x7fd668156060 2024-04-04T06:15:15.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.249+0000 7fd670000640 1 -- 172.21.15.159:0/2397028802 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd668067db0 con 0x7fd668156060 2024-04-04T06:15:15.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.250+0000 7fd665600640 1 -- 172.21.15.159:0/2397028802 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6600349a0 con 0x7fd668156060 2024-04-04T06:15:15.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.251+0000 7fd665600640 1 --2- 172.21.15.159:0/2397028802 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd64806ca90 0x7fd64806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:15.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.251+0000 7fd665600640 1 -- 172.21.15.159:0/2397028802 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fd6600ae0b0 con 0x7fd668156060 2024-04-04T06:15:15.245 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.251+0000 7fd66d600640 1 --2- 172.21.15.159:0/2397028802 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd64806ca90 0x7fd64806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:15.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.253+0000 7fd66d600640 1 --2- 172.21.15.159:0/2397028802 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd64806ca90 0x7fd64806ef50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009970 tx=0x7fd65c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:15.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.256+0000 7fd665600640 1 -- 172.21.15.159:0/2397028802 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd668067db0 con 0x7fd668156060 2024-04-04T06:15:15.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.538+0000 7fd670000640 1 -- 172.21.15.159:0/2397028802 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd668075a30 con 0x7fd668156060 2024-04-04T06:15:15.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.539+0000 7fd665600640 1 -- 172.21.15.159:0/2397028802 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v84) ==== 53+0+73 (secure 0 0 0) 0x7fd668075a30 con 0x7fd668156060 2024-04-04T06:15:15.533 INFO:teuthology.orchestra.run.smithi159.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-04T06:15:15.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.541+0000 7fd670000640 1 -- 172.21.15.159:0/2397028802 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd64806ca90 msgr2=0x7fd64806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:15.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.541+0000 7fd670000640 1 --2- 172.21.15.159:0/2397028802 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd64806ca90 0x7fd64806ef50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009970 tx=0x7fd65c008040 comp rx=0 tx=0).stop 2024-04-04T06:15:15.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.541+0000 7fd670000640 1 -- 172.21.15.159:0/2397028802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd668156060 msgr2=0x7fd6681564e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:15.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.541+0000 7fd670000640 1 --2- 172.21.15.159:0/2397028802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd668156060 0x7fd6681564e0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd66002cb80 tx=0x7fd6680b46b0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.542+0000 7fd670000640 1 -- 172.21.15.159:0/2397028802 shutdown_connections 2024-04-04T06:15:15.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.542+0000 7fd670000640 1 --2- 172.21.15.159:0/2397028802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd668156060 0x7fd6681564e0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.542+0000 7fd670000640 1 --2- 172.21.15.159:0/2397028802 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd64806ca90 0x7fd64806ef50 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.542+0000 7fd670000640 1 --2- 172.21.15.159:0/2397028802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6680716a0 0x7fd668072500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.542+0000 7fd670000640 1 --2- 172.21.15.159:0/2397028802 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd650005210 0x7fd66814d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:15.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.542+0000 7fd670000640 1 -- 172.21.15.159:0/2397028802 >> 172.21.15.159:0/2397028802 conn(0x7fd66806b750 msgr2=0x7fd6680afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:15.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.542+0000 7fd670000640 1 -- 172.21.15.159:0/2397028802 shutdown_connections 2024-04-04T06:15:15.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15.542+0000 7fd670000640 1 -- 172.21.15.159:0/2397028802 wait complete. 2024-04-04T06:15:15.554 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-04T06:15:15.554 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-04T06:15:15.554 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi159.front.sepia.ceph.com 2024-04-04T06:15:15.554 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-04T06:15:15.554 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-04T06:15:15.554 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-04T06:15:15.554 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-04T06:15:15.554 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-04T06:15:15.554 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:15:15.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:15.555 DEBUG:teuthology.orchestra.run.smithi159:> ip addr 2024-04-04T06:15:15.599 INFO:teuthology.orchestra.run.smithi159.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:15:15.599 INFO:teuthology.orchestra.run.smithi159.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:15:15.599 INFO:teuthology.orchestra.run.smithi159.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:15:15.599 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:15:15.599 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 ::1/128 scope host 2024-04-04T06:15:15.599 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:15:15.599 INFO:teuthology.orchestra.run.smithi159.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:15:15.599 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c6 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:15:15.599 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f0 2024-04-04T06:15:15.599 INFO:teuthology.orchestra.run.smithi159.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:15:15.599 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c7 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:15:15.599 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f1 2024-04-04T06:15:15.600 INFO:teuthology.orchestra.run.smithi159.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:15:15.600 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4e brd ff:ff:ff:ff:ff:ff 2024-04-04T06:15:15.600 INFO:teuthology.orchestra.run.smithi159.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-04T06:15:15.600 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:15:15.600 INFO:teuthology.orchestra.run.smithi159.stdout: inet 172.21.15.159/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-04T06:15:15.600 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft 42017sec preferred_lft 42017sec 2024-04-04T06:15:15.600 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::ec4:7aff:fe88:724f/64 scope link 2024-04-04T06:15:15.600 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:15:15.600 INFO:teuthology.orchestra.run.smithi159.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-04T06:15:15.600 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:15:15.600 INFO:teuthology.orchestra.run.smithi159.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-04T06:15:15.600 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:15:15.600 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::989e:e4ff:fea5:7568/64 scope link 2024-04-04T06:15:15.601 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:15:15.601 INFO:teuthology.orchestra.run.smithi159.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:15:15.601 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether a2:e3:e0:cb:0a:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-04T06:15:15.601 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::a0e3:e0ff:fecb:a73/64 scope link 2024-04-04T06:15:15.601 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:15:15.601 INFO:teuthology.orchestra.run.smithi159.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:15:15.601 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-04T06:15:15.601 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::c0a:8aff:fe54:b58f/64 scope link 2024-04-04T06:15:15.601 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:15:15.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:15.602 DEBUG:teuthology.orchestra.run.smithi159:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-04T06:15:15.725 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:15:15.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:15.730 DEBUG:teuthology.orchestra.run.smithi159:> route 2024-04-04T06:15:15.777 INFO:teuthology.orchestra.run.smithi159.stdout:Kernel IP routing table 2024-04-04T06:15:15.777 INFO:teuthology.orchestra.run.smithi159.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-04T06:15:15.778 INFO:teuthology.orchestra.run.smithi159.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-04T06:15:15.778 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-04T06:15:15.778 INFO:teuthology.orchestra.run.smithi159.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-04T06:15:15.778 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:15:15.778 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:15:15.778 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-04T06:15:15.778 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-04T06:15:15.778 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-04T06:15:15.778 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:15:15.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:15:16.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:15:16.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:16.050 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-04-04T06:15:16.095 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-04T06:15:16.095 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-04T06:15:16.096 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:15:16.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:16.096 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set brx.0 up 2024-04-04T06:15:16.236 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:15:16.236 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-04T06:15:16.288 INFO:teuthology.orchestra.run.smithi159.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:15:16.288 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:15:16.289 DEBUG:teuthology.orchestra.run.smithi159:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:15:16.335 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-04T06:15:16.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:16.335 DEBUG:teuthology.orchestra.run.smithi159:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:15:16.507 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:15:16.508 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:15:16.508 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:15:16.530 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:15:16.530 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:15:16.530 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:15:16.530 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:15:16.530 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:15:16.555 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:15:16.555 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:15:16.555 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:15:16.556 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:15:16.556 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:15:16.556 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-04T06:15:16.556 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-04T06:15:16.556 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30311045-f714-48fa-832b-4cb2ba8bf0bf" will pass to kernel 2024-04-04T06:15:16.557 INFO:teuthology.orchestra.run.smithi159.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-04T06:15:16.557 INFO:teuthology.orchestra.run.smithi159.stdout: You just mounted a file system that supports labels which does not 2024-04-04T06:15:16.557 INFO:teuthology.orchestra.run.smithi159.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-04T06:15:16.557 INFO:teuthology.orchestra.run.smithi159.stdout: applications will generate AVC messages and not be allowed access to 2024-04-04T06:15:16.557 INFO:teuthology.orchestra.run.smithi159.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-04T06:15:16.560 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-04T06:15:16.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:16.561 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:15:16.633 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:15:16.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:15:16.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:15:16.760 INFO:teuthology.orchestra.run.smithi159.stdout:client6154 2024-04-04T06:15:16.761 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:15:16.761 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:15:16.811 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:15:16.811 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:15:16.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:16.902 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-04T06:15:17.154 INFO:teuthology.orchestra.run.smithi159.stderr:6+0 records in 2024-04-04T06:15:17.154 INFO:teuthology.orchestra.run.smithi159.stderr:6+0 records out 2024-04-04T06:15:17.155 INFO:teuthology.orchestra.run.smithi159.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.154494 s, 40.7 MB/s 2024-04-04T06:15:17.156 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:15:17.156 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:15:17.206 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:15:17.206 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:15:17.299 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:15:17.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.513+0000 7fb639000640 1 Processor -- start 2024-04-04T06:15:17.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.513+0000 7fb639000640 1 -- start start 2024-04-04T06:15:17.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.515+0000 7fb639000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb6340d0d60 0x7fb634073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:17.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.515+0000 7fb639000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb63415fe80 con 0x7fb6340af830 2024-04-04T06:15:17.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.515+0000 7fb639000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6340ad120 con 0x7fb63415c390 2024-04-04T06:15:17.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.515+0000 7fb639000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fb63415c770 con 0x7fb6340d0d60 2024-04-04T06:15:17.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.515+0000 7fb62be00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fb6340af830 0x7fb6340d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41136/0 (socket says 172.21.15.159:41136) 2024-04-04T06:15:17.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.515+0000 7fb62be00640 1 -- 172.21.15.159:0/2932194510 learned_addr learned my addr 172.21.15.159:0/2932194510 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:17.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.515+0000 7fb632800640 1 --2- 172.21.15.159:0/2932194510 >> v2:172.21.15.159:3300/0 conn(0x7fb6340d0d60 0x7fb634073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:17.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.516+0000 7fb631400640 1 -- 172.21.15.159:0/2932194510 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3701475136 0 0) 0x7fb63415fe80 con 0x7fb6340af830 2024-04-04T06:15:17.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.516+0000 7fb631400640 1 -- 172.21.15.159:0/2932194510 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb618003520 con 0x7fb6340af830 2024-04-04T06:15:17.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.516+0000 7fb632800640 1 -- 172.21.15.159:0/2932194510 >> v1:172.21.15.178:6789/0 conn(0x7fb63415c390 legacy=0x7fb634160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:17.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.516+0000 7fb632800640 1 -- 172.21.15.159:0/2932194510 >> v1:172.21.15.159:6789/0 conn(0x7fb6340af830 legacy=0x7fb6340d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:17.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.516+0000 7fb632800640 1 -- 172.21.15.159:0/2932194510 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6340ad400 con 0x7fb6340d0d60 2024-04-04T06:15:17.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.517+0000 7fb632800640 1 --2- 172.21.15.159:0/2932194510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6340d0d60 0x7fb634073790 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009d90 tx=0x7fb62c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee804bcc17e313bf server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:17.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.517+0000 7fb631400640 1 -- 172.21.15.159:0/2932194510 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c039070 con 0x7fb6340d0d60 2024-04-04T06:15:17.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.517+0000 7fb631400640 1 -- 172.21.15.159:0/2932194510 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb62c02cd00 con 0x7fb6340d0d60 2024-04-04T06:15:17.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.517+0000 7fb631400640 1 -- 172.21.15.159:0/2932194510 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c034640 con 0x7fb6340d0d60 2024-04-04T06:15:17.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.518+0000 7fb639000640 1 -- 172.21.15.159:0/2932194510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6340d0d60 msgr2=0x7fb634073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:17.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.518+0000 7fb639000640 1 --2- 172.21.15.159:0/2932194510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6340d0d60 0x7fb634073790 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009d90 tx=0x7fb62c02bf50 comp rx=0 tx=0).stop 2024-04-04T06:15:17.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.518+0000 7fb639000640 1 -- 172.21.15.159:0/2932194510 shutdown_connections 2024-04-04T06:15:17.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.518+0000 7fb639000640 1 --2- 172.21.15.159:0/2932194510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6340d0d60 0x7fb634073790 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:17.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.518+0000 7fb639000640 1 -- 172.21.15.159:0/2932194510 >> 172.21.15.159:0/2932194510 conn(0x7fb63406b990 msgr2=0x7fb63406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:17.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.518+0000 7fb639000640 1 -- 172.21.15.159:0/2932194510 shutdown_connections 2024-04-04T06:15:17.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.518+0000 7fb639000640 1 -- 172.21.15.159:0/2932194510 wait complete. 2024-04-04T06:15:17.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.520+0000 7fb639000640 1 Processor -- start 2024-04-04T06:15:17.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.520+0000 7fb639000640 1 -- start start 2024-04-04T06:15:17.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.521+0000 7fb639000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6340af830 0x7fb634154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:17.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.521+0000 7fb639000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6340d0d60 0x7fb6340b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:17.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.521+0000 7fb639000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb63415c390 0x7fb6340b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:17.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.521+0000 7fb639000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb634160d50 con 0x7fb63415c390 2024-04-04T06:15:17.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.521+0000 7fb639000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb634160120 con 0x7fb6340d0d60 2024-04-04T06:15:17.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.521+0000 7fb639000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb6340cf980 con 0x7fb6340af830 2024-04-04T06:15:17.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.521+0000 7fb632800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb63415c390 0x7fb6340b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:17.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.521+0000 7fb632800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb63415c390 0x7fb6340b1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49518/0 (socket says 172.21.15.159:49518) 2024-04-04T06:15:17.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.521+0000 7fb632800640 1 -- 172.21.15.159:0/3006034119 learned_addr learned my addr 172.21.15.159:0/3006034119 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:17.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.521+0000 7fb62be00640 1 --2- 172.21.15.159:0/3006034119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6340d0d60 0x7fb6340b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:17.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.522+0000 7fb631e00640 1 --2- 172.21.15.159:0/3006034119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6340af830 0x7fb634154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:17.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.522+0000 7fb632800640 1 -- 172.21.15.159:0/3006034119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6340af830 msgr2=0x7fb634154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:17.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.522+0000 7fb632800640 1 --2- 172.21.15.159:0/3006034119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6340af830 0x7fb634154430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:17.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.522+0000 7fb632800640 1 -- 172.21.15.159:0/3006034119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6340d0d60 msgr2=0x7fb6340b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:17.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.522+0000 7fb632800640 1 --2- 172.21.15.159:0/3006034119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6340d0d60 0x7fb6340b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:17.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.522+0000 7fb632800640 1 -- 172.21.15.159:0/3006034119 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6340b4020 con 0x7fb63415c390 2024-04-04T06:15:17.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.522+0000 7fb632800640 1 --2- 172.21.15.159:0/3006034119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb63415c390 0x7fb6340b1390 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009eb0 tx=0x7fb62c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:17.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.522+0000 7fb62a000640 1 -- 172.21.15.159:0/3006034119 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c044020 con 0x7fb63415c390 2024-04-04T06:15:17.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.522+0000 7fb62a000640 1 -- 172.21.15.159:0/3006034119 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb62c03a040 con 0x7fb63415c390 2024-04-04T06:15:17.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.522+0000 7fb62a000640 1 -- 172.21.15.159:0/3006034119 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c039050 con 0x7fb63415c390 2024-04-04T06:15:17.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.522+0000 7fb639000640 1 -- 172.21.15.159:0/3006034119 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb63415cae0 con 0x7fb63415c390 2024-04-04T06:15:17.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.523+0000 7fb639000640 1 -- 172.21.15.159:0/3006034119 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb63415d0a0 con 0x7fb63415c390 2024-04-04T06:15:17.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.524+0000 7fb639000640 1 -- 172.21.15.159:0/3006034119 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb63415d2a0 con 0x7fb63415c390 2024-04-04T06:15:17.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.525+0000 7fb62a000640 1 -- 172.21.15.159:0/3006034119 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb62c003b90 con 0x7fb63415c390 2024-04-04T06:15:17.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.526+0000 7fb62a000640 1 --2- 172.21.15.159:0/3006034119 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb61006c8f0 0x7fb61006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:17.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.526+0000 7fb62a000640 1 -- 172.21.15.159:0/3006034119 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fb62c0ae2a0 con 0x7fb63415c390 2024-04-04T06:15:17.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.527+0000 7fb631e00640 1 --2- 172.21.15.159:0/3006034119 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb61006c8f0 0x7fb61006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:17.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.528+0000 7fb631e00640 1 --2- 172.21.15.159:0/3006034119 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb61006c8f0 0x7fb61006edb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb63415f900 tx=0x7fb624009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:17.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.530+0000 7fb62a000640 1 -- 172.21.15.159:0/3006034119 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb63415d2a0 con 0x7fb63415c390 2024-04-04T06:15:17.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.730+0000 7fb639000640 1 -- 172.21.15.159:0/3006034119 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6340685c0 con 0x7fb63415c390 2024-04-04T06:15:17.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.732+0000 7fb62a000640 1 -- 172.21.15.159:0/3006034119 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16623 (secure 0 0 0) 0x7fb62c04e020 con 0x7fb63415c390 2024-04-04T06:15:17.727 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:15:17.727 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":101,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:15:09.233029+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:15:05.537357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-04T06:15:06.689304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:15:17.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.735+0000 7fb639000640 1 -- 172.21.15.159:0/3006034119 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb61006c8f0 msgr2=0x7fb61006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:17.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.735+0000 7fb639000640 1 --2- 172.21.15.159:0/3006034119 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb61006c8f0 0x7fb61006edb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb63415f900 tx=0x7fb624009040 comp rx=0 tx=0).stop 2024-04-04T06:15:17.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.735+0000 7fb639000640 1 -- 172.21.15.159:0/3006034119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb63415c390 msgr2=0x7fb6340b1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:17.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.735+0000 7fb639000640 1 --2- 172.21.15.159:0/3006034119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb63415c390 0x7fb6340b1390 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009eb0 tx=0x7fb62c004a40 comp rx=0 tx=0).stop 2024-04-04T06:15:17.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.735+0000 7fb639000640 1 -- 172.21.15.159:0/3006034119 shutdown_connections 2024-04-04T06:15:17.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.735+0000 7fb639000640 1 --2- 172.21.15.159:0/3006034119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb63415c390 0x7fb6340b1390 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:17.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.735+0000 7fb639000640 1 --2- 172.21.15.159:0/3006034119 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb61006c8f0 0x7fb61006edb0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:17.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.735+0000 7fb639000640 1 --2- 172.21.15.159:0/3006034119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6340d0d60 0x7fb6340b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:17.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.735+0000 7fb639000640 1 --2- 172.21.15.159:0/3006034119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6340af830 0x7fb634154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:17.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.735+0000 7fb639000640 1 -- 172.21.15.159:0/3006034119 >> 172.21.15.159:0/3006034119 conn(0x7fb63406b990 msgr2=0x7fb63406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:17.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.736+0000 7fb639000640 1 -- 172.21.15.159:0/3006034119 shutdown_connections 2024-04-04T06:15:17.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.736+0000 7fb639000640 1 -- 172.21.15.159:0/3006034119 wait complete. 2024-04-04T06:15:17.750 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:15:05.537357+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:15:06.689304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:15:17.750 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:15:17.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.970+0000 7fe961a00640 1 Processor -- start 2024-04-04T06:15:17.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.971+0000 7fe961a00640 1 -- start start 2024-04-04T06:15:17.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.972+0000 7fe961a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe95c155550 0x7fe95c155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:17.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.972+0000 7fe961a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe95c15f800 0x7fe95c0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:17.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.972+0000 7fe961a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe95c0d07c0 0x7fe95c0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:17.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.972+0000 7fe961a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe95c0d0c10 con 0x7fe95c155550 2024-04-04T06:15:17.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.972+0000 7fe961a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe95c0b40b0 con 0x7fe95c15f800 2024-04-04T06:15:17.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.972+0000 7fe961a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe95c0b4390 con 0x7fe95c0d07c0 2024-04-04T06:15:17.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.972+0000 7fe95aa00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe95c155550 0x7fe95c155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:17.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.972+0000 7fe95aa00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe95c155550 0x7fe95c155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49530/0 (socket says 172.21.15.159:49530) 2024-04-04T06:15:17.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.972+0000 7fe95aa00640 1 -- 172.21.15.159:0/4214382748 learned_addr learned my addr 172.21.15.159:0/4214382748 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:17.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.973+0000 7fe95aa00640 1 -- 172.21.15.159:0/4214382748 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe95c0d07c0 msgr2=0x7fe95c0aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:17.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.973+0000 7fe95aa00640 1 --2- 172.21.15.159:0/4214382748 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe95c0d07c0 0x7fe95c0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:17.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.973+0000 7fe95a000640 1 --2- 172.21.15.159:0/4214382748 >> v2:172.21.15.178:3300/0 conn(0x7fe95c15f800 0x7fe95c0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:17.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.973+0000 7fe95aa00640 1 -- 172.21.15.159:0/4214382748 >> v2:172.21.15.178:3300/0 conn(0x7fe95c15f800 msgr2=0x7fe95c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:17.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.973+0000 7fe95aa00640 1 --2- 172.21.15.159:0/4214382748 >> v2:172.21.15.178:3300/0 conn(0x7fe95c15f800 0x7fe95c0d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:17.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.973+0000 7fe95aa00640 1 -- 172.21.15.159:0/4214382748 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95c0b4670 con 0x7fe95c155550 2024-04-04T06:15:17.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.973+0000 7fe95a000640 1 --2- 172.21.15.159:0/4214382748 >> v2:172.21.15.178:3300/0 conn(0x7fe95c15f800 0x7fe95c0d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:17.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.974+0000 7fe95aa00640 1 --2- 172.21.15.159:0/4214382748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe95c155550 0x7fe95c155930 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0d0dd0 tx=0x7fe94c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b6c58be1ed1f0a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:17.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.974+0000 7fe959600640 1 -- 172.21.15.159:0/4214382748 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c035020 con 0x7fe95c155550 2024-04-04T06:15:17.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.975+0000 7fe959600640 1 -- 172.21.15.159:0/4214382748 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe94c02d040 con 0x7fe95c155550 2024-04-04T06:15:17.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.975+0000 7fe959600640 1 -- 172.21.15.159:0/4214382748 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c031a80 con 0x7fe95c155550 2024-04-04T06:15:17.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.975+0000 7fe961a00640 1 -- 172.21.15.159:0/4214382748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe95c155550 msgr2=0x7fe95c155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:17.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.975+0000 7fe961a00640 1 --2- 172.21.15.159:0/4214382748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe95c155550 0x7fe95c155930 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0d0dd0 tx=0x7fe94c02b900 comp rx=0 tx=0).stop 2024-04-04T06:15:17.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.976+0000 7fe961a00640 1 -- 172.21.15.159:0/4214382748 shutdown_connections 2024-04-04T06:15:17.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.976+0000 7fe961a00640 1 --2- 172.21.15.159:0/4214382748 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe95c0d07c0 0x7fe95c0aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:17.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.976+0000 7fe961a00640 1 --2- 172.21.15.159:0/4214382748 >> v2:172.21.15.178:3300/0 conn(0x7fe95c15f800 0x7fe95c0d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:17.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.976+0000 7fe961a00640 1 --2- 172.21.15.159:0/4214382748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe95c155550 0x7fe95c155930 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:17.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.976+0000 7fe961a00640 1 -- 172.21.15.159:0/4214382748 >> 172.21.15.159:0/4214382748 conn(0x7fe95c06b750 msgr2=0x7fe95c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:17.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.976+0000 7fe961a00640 1 -- 172.21.15.159:0/4214382748 shutdown_connections 2024-04-04T06:15:17.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.976+0000 7fe961a00640 1 -- 172.21.15.159:0/4214382748 wait complete. 2024-04-04T06:15:17.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.977+0000 7fe961a00640 1 Processor -- start 2024-04-04T06:15:17.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.978+0000 7fe961a00640 1 -- start start 2024-04-04T06:15:17.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.979+0000 7fe961a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe95c0d07c0 0x7fe95c099c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:17.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.979+0000 7fe961a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe95c155550 0x7fe95c06d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:17.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.979+0000 7fe961a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe95c15f800 0x7fe95c06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:17.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.979+0000 7fe961a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c15fc80 con 0x7fe95c15f800 2024-04-04T06:15:17.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.979+0000 7fe961a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c0b4c30 con 0x7fe95c0d07c0 2024-04-04T06:15:17.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.979+0000 7fe961a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe95c15ba20 con 0x7fe95c155550 2024-04-04T06:15:17.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.979+0000 7fe95b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe95c15f800 0x7fe95c06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:17.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.979+0000 7fe95b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe95c15f800 0x7fe95c06dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49538/0 (socket says 172.21.15.159:49538) 2024-04-04T06:15:17.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.979+0000 7fe95b400640 1 -- 172.21.15.159:0/968868215 learned_addr learned my addr 172.21.15.159:0/968868215 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:17.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.979+0000 7fe95aa00640 1 --2- 172.21.15.159:0/968868215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe95c0d07c0 0x7fe95c099c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:17.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.979+0000 7fe95a000640 1 --2- 172.21.15.159:0/968868215 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe95c155550 0x7fe95c06d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:17.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.979+0000 7fe95b400640 1 -- 172.21.15.159:0/968868215 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe95c155550 msgr2=0x7fe95c06d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:17.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.979+0000 7fe95b400640 1 --2- 172.21.15.159:0/968868215 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe95c155550 0x7fe95c06d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:17.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.979+0000 7fe95b400640 1 -- 172.21.15.159:0/968868215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe95c0d07c0 msgr2=0x7fe95c099c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:17.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.979+0000 7fe95b400640 1 --2- 172.21.15.159:0/968868215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe95c0d07c0 0x7fe95c099c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:17.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.979+0000 7fe95b400640 1 -- 172.21.15.159:0/968868215 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95c09a1d0 con 0x7fe95c15f800 2024-04-04T06:15:17.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.980+0000 7fe95b400640 1 --2- 172.21.15.159:0/968868215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe95c15f800 0x7fe95c06dd60 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe95400b970 tx=0x7fe95400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:17.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.980+0000 7fe952a00640 1 -- 172.21.15.159:0/968868215 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe954019070 con 0x7fe95c15f800 2024-04-04T06:15:17.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.980+0000 7fe952a00640 1 -- 172.21.15.159:0/968868215 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe95400cb60 con 0x7fe95c15f800 2024-04-04T06:15:17.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.980+0000 7fe952a00640 1 -- 172.21.15.159:0/968868215 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe954014440 con 0x7fe95c15f800 2024-04-04T06:15:17.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.980+0000 7fe961a00640 1 -- 172.21.15.159:0/968868215 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe95c09c860 con 0x7fe95c15f800 2024-04-04T06:15:17.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.980+0000 7fe961a00640 1 -- 172.21.15.159:0/968868215 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe95c0aed10 con 0x7fe95c15f800 2024-04-04T06:15:17.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.982+0000 7fe961a00640 1 -- 172.21.15.159:0/968868215 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe95c0aef50 con 0x7fe95c15f800 2024-04-04T06:15:17.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.983+0000 7fe952a00640 1 -- 172.21.15.159:0/968868215 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe95401c070 con 0x7fe95c15f800 2024-04-04T06:15:17.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.984+0000 7fe952a00640 1 --2- 172.21.15.159:0/968868215 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe93c06c8f0 0x7fe93c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:17.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.984+0000 7fe952a00640 1 -- 172.21.15.159:0/968868215 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fe95408b880 con 0x7fe95c15f800 2024-04-04T06:15:17.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.985+0000 7fe95aa00640 1 --2- 172.21.15.159:0/968868215 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe93c06c8f0 0x7fe93c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:17.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.986+0000 7fe95aa00640 1 --2- 172.21.15.159:0/968868215 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe93c06c8f0 0x7fe93c06edb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe94c036000 tx=0x7fe94c03c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:17.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:17.989+0000 7fe952a00640 1 -- 172.21.15.159:0/968868215 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe95c0aef50 con 0x7fe95c15f800 2024-04-04T06:15:18.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.187+0000 7fe961a00640 1 -- 172.21.15.159:0/968868215 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe95c0685c0 con 0x7fe95c15f800 2024-04-04T06:15:18.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.188+0000 7fe952a00640 1 -- 172.21.15.159:0/968868215 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7fe954059cb0 con 0x7fe95c15f800 2024-04-04T06:15:18.182 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:15:18.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.191+0000 7fe961a00640 1 -- 172.21.15.159:0/968868215 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe93c06c8f0 msgr2=0x7fe93c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:18.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.191+0000 7fe961a00640 1 --2- 172.21.15.159:0/968868215 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe93c06c8f0 0x7fe93c06edb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe94c036000 tx=0x7fe94c03c040 comp rx=0 tx=0).stop 2024-04-04T06:15:18.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.191+0000 7fe961a00640 1 -- 172.21.15.159:0/968868215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe95c15f800 msgr2=0x7fe95c06dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:18.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.191+0000 7fe961a00640 1 --2- 172.21.15.159:0/968868215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe95c15f800 0x7fe95c06dd60 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe95400b970 tx=0x7fe95400be30 comp rx=0 tx=0).stop 2024-04-04T06:15:18.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.191+0000 7fe961a00640 1 -- 172.21.15.159:0/968868215 shutdown_connections 2024-04-04T06:15:18.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.191+0000 7fe961a00640 1 --2- 172.21.15.159:0/968868215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe95c15f800 0x7fe95c06dd60 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:18.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.191+0000 7fe961a00640 1 --2- 172.21.15.159:0/968868215 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe95c155550 0x7fe95c06d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:18.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.191+0000 7fe961a00640 1 --2- 172.21.15.159:0/968868215 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe93c06c8f0 0x7fe93c06edb0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:18.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.191+0000 7fe961a00640 1 --2- 172.21.15.159:0/968868215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe95c0d07c0 0x7fe95c099c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:18.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.191+0000 7fe961a00640 1 -- 172.21.15.159:0/968868215 >> 172.21.15.159:0/968868215 conn(0x7fe95c06b750 msgr2=0x7fe95c15d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:18.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.191+0000 7fe961a00640 1 -- 172.21.15.159:0/968868215 shutdown_connections 2024-04-04T06:15:18.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.191+0000 7fe961a00640 1 -- 172.21.15.159:0/968868215 wait complete. 2024-04-04T06:15:18.211 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:15:18.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.408+0000 7fbc89000640 1 Processor -- start 2024-04-04T06:15:18.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.408+0000 7fbc89000640 1 -- start start 2024-04-04T06:15:18.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.410+0000 7fbc89000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbc8415e590 0x7fbc840d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:18.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.410+0000 7fbc89000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fbc840b47d0 0x7fbc8406c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:18.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.410+0000 7fbc89000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc8406cfd0 con 0x7fbc8406cbc0 2024-04-04T06:15:18.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.410+0000 7fbc89000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fbc84071060 con 0x7fbc840b47d0 2024-04-04T06:15:18.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.410+0000 7fbc89000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fbc84071340 con 0x7fbc8415e590 2024-04-04T06:15:18.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.410+0000 7fbc82800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fbc8406cbc0 0x7fbc84162230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41140/0 (socket says 172.21.15.159:41140) 2024-04-04T06:15:18.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.410+0000 7fbc82800640 1 -- 172.21.15.159:0/440459888 learned_addr learned my addr 172.21.15.159:0/440459888 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:18.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.410+0000 7fbc81400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fbc840b47d0 0x7fbc8406c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:18.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.410+0000 7fbc81e00640 1 --2- 172.21.15.159:0/440459888 >> v2:172.21.15.178:3300/0 conn(0x7fbc8415e590 0x7fbc840d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:18.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.411+0000 7fbc80a00640 1 -- 172.21.15.159:0/440459888 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1687453755 0 0) 0x7fbc7c001490 con 0x7fbc8406cbc0 2024-04-04T06:15:18.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.411+0000 7fbc80a00640 1 -- 172.21.15.159:0/440459888 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc68003520 con 0x7fbc8406cbc0 2024-04-04T06:15:18.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.411+0000 7fbc81400640 1 -- 172.21.15.159:0/440459888 >> v2:172.21.15.178:3300/0 conn(0x7fbc8415e590 msgr2=0x7fbc840d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:18.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.411+0000 7fbc81400640 1 --2- 172.21.15.159:0/440459888 >> v2:172.21.15.178:3300/0 conn(0x7fbc8415e590 0x7fbc840d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:18.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.411+0000 7fbc81400640 1 -- 172.21.15.159:0/440459888 >> v1:172.21.15.159:6789/0 conn(0x7fbc8406cbc0 legacy=0x7fbc84162230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:18.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.411+0000 7fbc81400640 1 -- 172.21.15.159:0/440459888 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc84071620 con 0x7fbc840b47d0 2024-04-04T06:15:18.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.414+0000 7fbc81400640 1 --2- 172.21.15.159:0/440459888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc840b47d0 0x7fbc8406c680 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fbc840691c0 tx=0x7fbc7002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa169bfabc7e999 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:18.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.414+0000 7fbc80a00640 1 -- 172.21.15.159:0/440459888 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70035020 con 0x7fbc840b47d0 2024-04-04T06:15:18.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.414+0000 7fbc80a00640 1 -- 172.21.15.159:0/440459888 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc70031830 con 0x7fbc840b47d0 2024-04-04T06:15:18.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.414+0000 7fbc80a00640 1 -- 172.21.15.159:0/440459888 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70031be0 con 0x7fbc840b47d0 2024-04-04T06:15:18.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.414+0000 7fbc89000640 1 -- 172.21.15.159:0/440459888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc840b47d0 msgr2=0x7fbc8406c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:18.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.414+0000 7fbc89000640 1 --2- 172.21.15.159:0/440459888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc840b47d0 0x7fbc8406c680 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fbc840691c0 tx=0x7fbc7002b8d0 comp rx=0 tx=0).stop 2024-04-04T06:15:18.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.414+0000 7fbc89000640 1 -- 172.21.15.159:0/440459888 shutdown_connections 2024-04-04T06:15:18.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.414+0000 7fbc89000640 1 --2- 172.21.15.159:0/440459888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc840b47d0 0x7fbc8406c680 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:18.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.414+0000 7fbc89000640 1 --2- 172.21.15.159:0/440459888 >> v2:172.21.15.178:3300/0 conn(0x7fbc8415e590 0x7fbc840d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:18.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.414+0000 7fbc89000640 1 -- 172.21.15.159:0/440459888 >> 172.21.15.159:0/440459888 conn(0x7fbc840d2240 msgr2=0x7fbc840d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:18.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.414+0000 7fbc89000640 1 -- 172.21.15.159:0/440459888 shutdown_connections 2024-04-04T06:15:18.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.414+0000 7fbc89000640 1 -- 172.21.15.159:0/440459888 wait complete. 2024-04-04T06:15:18.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.416+0000 7fbc89000640 1 Processor -- start 2024-04-04T06:15:18.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.416+0000 7fbc89000640 1 -- start start 2024-04-04T06:15:18.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.417+0000 7fbc89000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc8406cbc0 0x7fbc84149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:18.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.417+0000 7fbc89000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc840b47d0 0x7fbc8415ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:18.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.417+0000 7fbc89000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc8415e590 0x7fbc841556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:18.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.417+0000 7fbc89000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbc840b4c60 con 0x7fbc840b47d0 2024-04-04T06:15:18.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.417+0000 7fbc89000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc84071be0 con 0x7fbc8406cbc0 2024-04-04T06:15:18.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.417+0000 7fbc89000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbc8415e970 con 0x7fbc8415e590 2024-04-04T06:15:18.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.417+0000 7fbc81400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc840b47d0 0x7fbc8415ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:18.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.417+0000 7fbc81400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc840b47d0 0x7fbc8415ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49558/0 (socket says 172.21.15.159:49558) 2024-04-04T06:15:18.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.417+0000 7fbc81400640 1 -- 172.21.15.159:0/3834934114 learned_addr learned my addr 172.21.15.159:0/3834934114 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:18.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.417+0000 7fbc81e00640 1 --2- 172.21.15.159:0/3834934114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc8406cbc0 0x7fbc84149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:18.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.417+0000 7fbc81400640 1 -- 172.21.15.159:0/3834934114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc8415e590 msgr2=0x7fbc841556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:18.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.417+0000 7fbc81400640 1 --2- 172.21.15.159:0/3834934114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc8415e590 0x7fbc841556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:18.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.417+0000 7fbc81400640 1 -- 172.21.15.159:0/3834934114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc8406cbc0 msgr2=0x7fbc84149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:18.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.417+0000 7fbc81400640 1 --2- 172.21.15.159:0/3834934114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc8406cbc0 0x7fbc84149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:18.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.417+0000 7fbc81400640 1 -- 172.21.15.159:0/3834934114 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc840b51d0 con 0x7fbc840b47d0 2024-04-04T06:15:18.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.418+0000 7fbc81400640 1 --2- 172.21.15.159:0/3834934114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc840b47d0 0x7fbc8415ce10 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fbc70000c00 tx=0x7fbc700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:18.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.418+0000 7fbc7a000640 1 -- 172.21.15.159:0/3834934114 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70041070 con 0x7fbc840b47d0 2024-04-04T06:15:18.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.418+0000 7fbc7a000640 1 -- 172.21.15.159:0/3834934114 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc7003b860 con 0x7fbc840b47d0 2024-04-04T06:15:18.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.418+0000 7fbc7a000640 1 -- 172.21.15.159:0/3834934114 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70035040 con 0x7fbc840b47d0 2024-04-04T06:15:18.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.418+0000 7fbc89000640 1 -- 172.21.15.159:0/3834934114 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc840b54b0 con 0x7fbc840b47d0 2024-04-04T06:15:18.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.418+0000 7fbc89000640 1 -- 172.21.15.159:0/3834934114 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc840b5a70 con 0x7fbc840b47d0 2024-04-04T06:15:18.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.419+0000 7fbc89000640 1 -- 172.21.15.159:0/3834934114 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc84153990 con 0x7fbc840b47d0 2024-04-04T06:15:18.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.421+0000 7fbc7a000640 1 -- 172.21.15.159:0/3834934114 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc70038020 con 0x7fbc840b47d0 2024-04-04T06:15:18.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.422+0000 7fbc7a000640 1 --2- 172.21.15.159:0/3834934114 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc6406c8f0 0x7fbc6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:18.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.422+0000 7fbc7a000640 1 -- 172.21.15.159:0/3834934114 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fbc700ad3f0 con 0x7fbc840b47d0 2024-04-04T06:15:18.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.423+0000 7fbc81e00640 1 --2- 172.21.15.159:0/3834934114 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc6406c8f0 0x7fbc6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:18.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.424+0000 7fbc81e00640 1 --2- 172.21.15.159:0/3834934114 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc6406c8f0 0x7fbc6406edb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbc74006fd0 tx=0x7fbc74008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:18.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.427+0000 7fbc7a000640 1 -- 172.21.15.159:0/3834934114 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbc70031800 con 0x7fbc840b47d0 2024-04-04T06:15:18.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.627+0000 7fbc89000640 1 -- 172.21.15.159:0/3834934114 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbc840685c0 con 0x7fbc840b47d0 2024-04-04T06:15:18.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.627+0000 7fbc7a000640 1 -- 172.21.15.159:0/3834934114 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7fbc7007b8b0 con 0x7fbc840b47d0 2024-04-04T06:15:18.622 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:15:18.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.631+0000 7fbc89000640 1 -- 172.21.15.159:0/3834934114 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc6406c8f0 msgr2=0x7fbc6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:18.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.631+0000 7fbc89000640 1 --2- 172.21.15.159:0/3834934114 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc6406c8f0 0x7fbc6406edb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbc74006fd0 tx=0x7fbc74008040 comp rx=0 tx=0).stop 2024-04-04T06:15:18.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.631+0000 7fbc89000640 1 -- 172.21.15.159:0/3834934114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc840b47d0 msgr2=0x7fbc8415ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:18.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.631+0000 7fbc89000640 1 --2- 172.21.15.159:0/3834934114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc840b47d0 0x7fbc8415ce10 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fbc70000c00 tx=0x7fbc700046c0 comp rx=0 tx=0).stop 2024-04-04T06:15:18.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.631+0000 7fbc89000640 1 -- 172.21.15.159:0/3834934114 shutdown_connections 2024-04-04T06:15:18.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.631+0000 7fbc89000640 1 --2- 172.21.15.159:0/3834934114 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc8415e590 0x7fbc841556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:18.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.631+0000 7fbc89000640 1 --2- 172.21.15.159:0/3834934114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc840b47d0 0x7fbc8415ce10 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:18.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.631+0000 7fbc89000640 1 --2- 172.21.15.159:0/3834934114 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc6406c8f0 0x7fbc6406edb0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:18.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.631+0000 7fbc89000640 1 --2- 172.21.15.159:0/3834934114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc8406cbc0 0x7fbc84149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:18.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.631+0000 7fbc89000640 1 -- 172.21.15.159:0/3834934114 >> 172.21.15.159:0/3834934114 conn(0x7fbc840d2240 msgr2=0x7fbc840d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:18.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.631+0000 7fbc89000640 1 -- 172.21.15.159:0/3834934114 shutdown_connections 2024-04-04T06:15:18.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.631+0000 7fbc89000640 1 -- 172.21.15.159:0/3834934114 wait complete. 2024-04-04T06:15:18.651 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:15:18.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.861+0000 7f2f8be00640 1 Processor -- start 2024-04-04T06:15:18.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.861+0000 7f2f8be00640 1 -- start start 2024-04-04T06:15:18.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.864+0000 7f2f8be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2f8c0d37b0 0x7f2f8c0b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:18.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.864+0000 7f2f8be00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f8c0d2710 con 0x7f2f8c0b67f0 2024-04-04T06:15:18.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.864+0000 7f2f8be00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f8c0d29f0 con 0x7f2f8c15e4c0 2024-04-04T06:15:18.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.864+0000 7f2f8be00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f2f8c15e8a0 con 0x7f2f8c0d37b0 2024-04-04T06:15:18.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.864+0000 7f2f8a000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f2f8c0d37b0 0x7f2f8c0b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:18.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.864+0000 7f2f8b400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f2f8c0b67f0 0x7f2f8c0b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41152/0 (socket says 172.21.15.159:41152) 2024-04-04T06:15:18.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.864+0000 7f2f8b400640 1 -- 172.21.15.159:0/1434698954 learned_addr learned my addr 172.21.15.159:0/1434698954 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:18.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.865+0000 7f2f8a000640 1 -- 172.21.15.159:0/1434698954 >> v1:172.21.15.178:6789/0 conn(0x7f2f8c15e4c0 legacy=0x7f2f8c0b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:18.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.865+0000 7f2f8a000640 1 -- 172.21.15.159:0/1434698954 >> v1:172.21.15.159:6789/0 conn(0x7f2f8c0b67f0 legacy=0x7f2f8c0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:18.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.865+0000 7f2f8a000640 1 -- 172.21.15.159:0/1434698954 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8c0d2cd0 con 0x7f2f8c0d37b0 2024-04-04T06:15:18.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.865+0000 7f2f89600640 1 -- 172.21.15.159:0/1434698954 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 685024067 0 0) 0x7f2f8c0d2710 con 0x7f2f8c0b67f0 2024-04-04T06:15:18.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.866+0000 7f2f8a000640 1 --2- 172.21.15.159:0/1434698954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f8c0d37b0 0x7f2f8c0b62b0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f2f780097a0 tx=0x7f2f7802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6af73ca76fad034 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:18.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.866+0000 7f2f89600640 1 -- 172.21.15.159:0/1434698954 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f78035020 con 0x7f2f8c0d37b0 2024-04-04T06:15:18.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.866+0000 7f2f89600640 1 -- 172.21.15.159:0/1434698954 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f78031830 con 0x7f2f8c0d37b0 2024-04-04T06:15:18.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.866+0000 7f2f89600640 1 -- 172.21.15.159:0/1434698954 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f78031c00 con 0x7f2f8c0d37b0 2024-04-04T06:15:18.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.867+0000 7f2f8be00640 1 -- 172.21.15.159:0/1434698954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f8c0d37b0 msgr2=0x7f2f8c0b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:18.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.867+0000 7f2f8be00640 1 --2- 172.21.15.159:0/1434698954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f8c0d37b0 0x7f2f8c0b62b0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f2f780097a0 tx=0x7f2f7802b8d0 comp rx=0 tx=0).stop 2024-04-04T06:15:18.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.867+0000 7f2f8be00640 1 -- 172.21.15.159:0/1434698954 shutdown_connections 2024-04-04T06:15:18.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.867+0000 7f2f8be00640 1 --2- 172.21.15.159:0/1434698954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f8c0d37b0 0x7f2f8c0b62b0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:18.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.867+0000 7f2f8be00640 1 -- 172.21.15.159:0/1434698954 >> 172.21.15.159:0/1434698954 conn(0x7f2f8c0676a0 msgr2=0x7f2f8c067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:18.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.867+0000 7f2f8be00640 1 -- 172.21.15.159:0/1434698954 shutdown_connections 2024-04-04T06:15:18.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.867+0000 7f2f8be00640 1 -- 172.21.15.159:0/1434698954 wait complete. 2024-04-04T06:15:18.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.869+0000 7f2f8be00640 1 Processor -- start 2024-04-04T06:15:18.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.869+0000 7f2f8be00640 1 -- start start 2024-04-04T06:15:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.870+0000 7f2f8be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f8c0b67f0 0x7f2f8c154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.870+0000 7f2f8be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f8c0d37b0 0x7f2f8c155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.870+0000 7f2f8be00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f8c15e4c0 0x7f2f8c1602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.870+0000 7f2f8be00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c0d3320 con 0x7f2f8c0b67f0 2024-04-04T06:15:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.870+0000 7f2f8be00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c0d4040 con 0x7f2f8c0d37b0 2024-04-04T06:15:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.870+0000 7f2f8be00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2f8c0b6c20 con 0x7f2f8c15e4c0 2024-04-04T06:15:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.870+0000 7f2f8aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f8c0b67f0 0x7f2f8c154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.870+0000 7f2f8aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f8c0b67f0 0x7f2f8c154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49584/0 (socket says 172.21.15.159:49584) 2024-04-04T06:15:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.870+0000 7f2f8aa00640 1 -- 172.21.15.159:0/675887755 learned_addr learned my addr 172.21.15.159:0/675887755 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.870+0000 7f2f8a000640 1 --2- 172.21.15.159:0/675887755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f8c0d37b0 0x7f2f8c155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:18.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.871+0000 7f2f8b400640 1 --2- 172.21.15.159:0/675887755 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f8c15e4c0 0x7f2f8c1602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:18.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.871+0000 7f2f8aa00640 1 -- 172.21.15.159:0/675887755 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f8c15e4c0 msgr2=0x7f2f8c1602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:18.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.871+0000 7f2f8aa00640 1 --2- 172.21.15.159:0/675887755 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f8c15e4c0 0x7f2f8c1602f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:18.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.871+0000 7f2f8aa00640 1 -- 172.21.15.159:0/675887755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f8c0d37b0 msgr2=0x7f2f8c155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:18.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.871+0000 7f2f8aa00640 1 --2- 172.21.15.159:0/675887755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f8c0d37b0 0x7f2f8c155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:18.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.871+0000 7f2f8aa00640 1 -- 172.21.15.159:0/675887755 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8c160d40 con 0x7f2f8c0b67f0 2024-04-04T06:15:18.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.871+0000 7f2f8aa00640 1 --2- 172.21.15.159:0/675887755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f8c0b67f0 0x7f2f8c154a20 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00b3f0 tx=0x7f2f7c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:18.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.871+0000 7f2f82a00640 1 -- 172.21.15.159:0/675887755 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c015020 con 0x7f2f8c0b67f0 2024-04-04T06:15:18.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.871+0000 7f2f82a00640 1 -- 172.21.15.159:0/675887755 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f7c011830 con 0x7f2f8c0b67f0 2024-04-04T06:15:18.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.871+0000 7f2f8be00640 1 -- 172.21.15.159:0/675887755 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f8c0c91a0 con 0x7f2f8c0b67f0 2024-04-04T06:15:18.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.871+0000 7f2f82a00640 1 -- 172.21.15.159:0/675887755 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c011c20 con 0x7f2f8c0b67f0 2024-04-04T06:15:18.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.872+0000 7f2f8be00640 1 -- 172.21.15.159:0/675887755 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f8c0c9760 con 0x7f2f8c0b67f0 2024-04-04T06:15:18.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.873+0000 7f2f8be00640 1 -- 172.21.15.159:0/675887755 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f8c075940 con 0x7f2f8c0b67f0 2024-04-04T06:15:18.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.873+0000 7f2f82a00640 1 -- 172.21.15.159:0/675887755 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f7c018020 con 0x7f2f8c0b67f0 2024-04-04T06:15:18.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.874+0000 7f2f82a00640 1 --2- 172.21.15.159:0/675887755 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2f6c06c8f0 0x7f2f6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:18.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.875+0000 7f2f82a00640 1 -- 172.21.15.159:0/675887755 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f2f7c08b7d0 con 0x7f2f8c0b67f0 2024-04-04T06:15:18.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.875+0000 7f2f8a000640 1 --2- 172.21.15.159:0/675887755 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2f6c06c8f0 0x7f2f6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:18.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.876+0000 7f2f8a000640 1 --2- 172.21.15.159:0/675887755 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2f6c06c8f0 0x7f2f6c06edb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2f78036000 tx=0x7f2f7803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:18.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:18.879+0000 7f2f82a00640 1 -- 172.21.15.159:0/675887755 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2f7c01f030 con 0x7f2f8c0b67f0 2024-04-04T06:15:19.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.079+0000 7f2f8be00640 1 -- 172.21.15.159:0/675887755 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2f8c157cc0 con 0x7f2f8c0b67f0 2024-04-04T06:15:19.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.079+0000 7f2f82a00640 1 -- 172.21.15.159:0/675887755 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7f2f7c059bf0 con 0x7f2f8c0b67f0 2024-04-04T06:15:19.073 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:15:19.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.082+0000 7f2f8be00640 1 -- 172.21.15.159:0/675887755 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2f6c06c8f0 msgr2=0x7f2f6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:19.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.082+0000 7f2f8be00640 1 --2- 172.21.15.159:0/675887755 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2f6c06c8f0 0x7f2f6c06edb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2f78036000 tx=0x7f2f7803c040 comp rx=0 tx=0).stop 2024-04-04T06:15:19.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.082+0000 7f2f8be00640 1 -- 172.21.15.159:0/675887755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f8c0b67f0 msgr2=0x7f2f8c154a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:19.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.082+0000 7f2f8be00640 1 --2- 172.21.15.159:0/675887755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f8c0b67f0 0x7f2f8c154a20 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00b3f0 tx=0x7f2f7c00b8b0 comp rx=0 tx=0).stop 2024-04-04T06:15:19.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.082+0000 7f2f8be00640 1 -- 172.21.15.159:0/675887755 shutdown_connections 2024-04-04T06:15:19.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.082+0000 7f2f8be00640 1 --2- 172.21.15.159:0/675887755 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f8c15e4c0 0x7f2f8c1602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:19.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.082+0000 7f2f8be00640 1 --2- 172.21.15.159:0/675887755 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2f6c06c8f0 0x7f2f6c06edb0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:19.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.082+0000 7f2f8be00640 1 --2- 172.21.15.159:0/675887755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f8c0d37b0 0x7f2f8c155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:19.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.082+0000 7f2f8be00640 1 --2- 172.21.15.159:0/675887755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f8c0b67f0 0x7f2f8c154a20 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:19.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.082+0000 7f2f8be00640 1 -- 172.21.15.159:0/675887755 >> 172.21.15.159:0/675887755 conn(0x7f2f8c0676a0 msgr2=0x7f2f8c0b5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:19.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.082+0000 7f2f8be00640 1 -- 172.21.15.159:0/675887755 shutdown_connections 2024-04-04T06:15:19.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.082+0000 7f2f8be00640 1 -- 172.21.15.159:0/675887755 wait complete. 2024-04-04T06:15:19.093 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-04T06:15:19.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.306+0000 7fd99ac00640 1 Processor -- start 2024-04-04T06:15:19.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.306+0000 7fd99ac00640 1 -- start start 2024-04-04T06:15:19.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.307+0000 7fd99ac00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9940d1580 0x7fd9940cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:19.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.307+0000 7fd99ac00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9940b5ce0 con 0x7fd994161f60 2024-04-04T06:15:19.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.308+0000 7fd99ac00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9940d27f0 con 0x7fd9940b4db0 2024-04-04T06:15:19.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.308+0000 7fd99ac00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd9940d2a90 con 0x7fd9940d1580 2024-04-04T06:15:19.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.308+0000 7fd993e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fd994161f60 0x7fd9940d0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41160/0 (socket says 172.21.15.159:41160) 2024-04-04T06:15:19.301 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.308+0000 7fd993e00640 1 -- 172.21.15.159:0/2270631292 learned_addr learned my addr 172.21.15.159:0/2270631292 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:19.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.308+0000 7fd993400640 1 --2- 172.21.15.159:0/2270631292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9940d1580 0x7fd9940cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:19.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.308+0000 7fd992a00640 1 -- 172.21.15.159:0/2270631292 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4103603246 0 0) 0x7fd984001110 con 0x7fd994161f60 2024-04-04T06:15:19.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.308+0000 7fd992a00640 1 -- 172.21.15.159:0/2270631292 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd978003520 con 0x7fd994161f60 2024-04-04T06:15:19.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.309+0000 7fd992a00640 1 -- 172.21.15.159:0/2270631292 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1210162011 0 0) 0x7fd978003520 con 0x7fd994161f60 2024-04-04T06:15:19.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.309+0000 7fd992a00640 1 -- 172.21.15.159:0/2270631292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9940d1580 msgr2=0x7fd9940cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:19.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.309+0000 7fd992a00640 1 --2- 172.21.15.159:0/2270631292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9940d1580 0x7fd9940cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:19.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.309+0000 7fd992a00640 1 -- 172.21.15.159:0/2270631292 >> v1:172.21.15.178:6789/0 conn(0x7fd9940b4db0 legacy=0x7fd9940cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:19.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.309+0000 7fd992a00640 1 -- 172.21.15.159:0/2270631292 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd994162340 con 0x7fd994161f60 2024-04-04T06:15:19.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.309+0000 7fd992a00640 1 -- 172.21.15.159:0/2270631292 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fd984001840 con 0x7fd994161f60 2024-04-04T06:15:19.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.309+0000 7fd992a00640 1 -- 172.21.15.159:0/2270631292 >> v1:172.21.15.159:6789/0 conn(0x7fd994161f60 legacy=0x7fd9940d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:19.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.309+0000 7fd992a00640 1 --2- 172.21.15.159:0/2270631292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9780039e0 0x7fd978003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:19.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.309+0000 7fd992a00640 1 --2- 172.21.15.159:0/2270631292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd978004800 0x7fd978004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:19.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.309+0000 7fd992a00640 1 --2- 172.21.15.159:0/2270631292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9940d1580 0x7fd9780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:19.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.309+0000 7fd992a00640 1 -- 172.21.15.159:0/2270631292 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9940d2a90 con 0x7fd978004800 2024-04-04T06:15:19.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.309+0000 7fd992a00640 1 -- 172.21.15.159:0/2270631292 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd97800ba70 con 0x7fd9780039e0 2024-04-04T06:15:19.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.309+0000 7fd992a00640 1 -- 172.21.15.159:0/2270631292 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd97800bc70 con 0x7fd9940d1580 2024-04-04T06:15:19.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.309+0000 7fd993400640 1 --2- 172.21.15.159:0/2270631292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd978004800 0x7fd978004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:19.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.310+0000 7fd993e00640 1 --2- 172.21.15.159:0/2270631292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9780039e0 0x7fd978003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:19.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.310+0000 7fd993400640 1 -- 172.21.15.159:0/2270631292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9940d1580 msgr2=0x7fd9780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:19.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.310+0000 7fd993400640 1 --2- 172.21.15.159:0/2270631292 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9940d1580 0x7fd9780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:19.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.310+0000 7fd993400640 1 -- 172.21.15.159:0/2270631292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9780039e0 msgr2=0x7fd978003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:19.304 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.310+0000 7fd993400640 1 --2- 172.21.15.159:0/2270631292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9780039e0 0x7fd978003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:19.305 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.310+0000 7fd993400640 1 -- 172.21.15.159:0/2270631292 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd978003520 con 0x7fd978004800 2024-04-04T06:15:19.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.311+0000 7fd993400640 1 --2- 172.21.15.159:0/2270631292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd978004800 0x7fd978004c80 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fd980009a90 tx=0x7fd978005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509b8fc267896f01 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:19.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.311+0000 7fd992a00640 1 -- 172.21.15.159:0/2270631292 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980039070 con 0x7fd978004800 2024-04-04T06:15:19.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.311+0000 7fd992a00640 1 -- 172.21.15.159:0/2270631292 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd98002c6b0 con 0x7fd978004800 2024-04-04T06:15:19.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.311+0000 7fd992a00640 1 -- 172.21.15.159:0/2270631292 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980009690 con 0x7fd978004800 2024-04-04T06:15:19.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.312+0000 7fd99ac00640 1 -- 172.21.15.159:0/2270631292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd978004800 msgr2=0x7fd978004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:19.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.312+0000 7fd99ac00640 1 --2- 172.21.15.159:0/2270631292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd978004800 0x7fd978004c80 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fd980009a90 tx=0x7fd978005830 comp rx=0 tx=0).stop 2024-04-04T06:15:19.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.312+0000 7fd998c00640 1 -- 172.21.15.159:0/2270631292 reap_dead start 2024-04-04T06:15:19.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.312+0000 7fd99ac00640 1 -- 172.21.15.159:0/2270631292 shutdown_connections 2024-04-04T06:15:19.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.312+0000 7fd99ac00640 1 -- 172.21.15.159:0/2270631292 >> 172.21.15.159:0/2270631292 conn(0x7fd99408bfb0 msgr2=0x7fd9940d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:19.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.312+0000 7fd99ac00640 1 -- 172.21.15.159:0/2270631292 shutdown_connections 2024-04-04T06:15:19.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.312+0000 7fd99ac00640 1 -- 172.21.15.159:0/2270631292 wait complete. 2024-04-04T06:15:19.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.314+0000 7fd99ac00640 1 Processor -- start 2024-04-04T06:15:19.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.314+0000 7fd99ac00640 1 -- start start 2024-04-04T06:15:19.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.315+0000 7fd99ac00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd978004800 0x7fd9940b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:19.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.315+0000 7fd99ac00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9940cd400 0x7fd9940a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:19.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.315+0000 7fd99ac00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9940cbe60 0x7fd9940b95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:19.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.315+0000 7fd99ac00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9940d3310 con 0x7fd9940cbe60 2024-04-04T06:15:19.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.315+0000 7fd99ac00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd9940d3030 con 0x7fd978004800 2024-04-04T06:15:19.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.315+0000 7fd99ac00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd9940b3170 con 0x7fd9940cd400 2024-04-04T06:15:19.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.315+0000 7fd998c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9940cbe60 0x7fd9940b95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:19.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.315+0000 7fd998c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9940cbe60 0x7fd9940b95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49612/0 (socket says 172.21.15.159:49612) 2024-04-04T06:15:19.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.315+0000 7fd998c00640 1 -- 172.21.15.159:0/3339186885 learned_addr learned my addr 172.21.15.159:0/3339186885 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:19.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.316+0000 7fd993e00640 1 --2- 172.21.15.159:0/3339186885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd978004800 0x7fd9940b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:19.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.316+0000 7fd998c00640 1 -- 172.21.15.159:0/3339186885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9940cd400 msgr2=0x7fd9940a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:19.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.316+0000 7fd998c00640 1 --2- 172.21.15.159:0/3339186885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9940cd400 0x7fd9940a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:19.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.316+0000 7fd998c00640 1 -- 172.21.15.159:0/3339186885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd978004800 msgr2=0x7fd9940b7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:19.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.316+0000 7fd998c00640 1 --2- 172.21.15.159:0/3339186885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd978004800 0x7fd9940b7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:19.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.316+0000 7fd998c00640 1 -- 172.21.15.159:0/3339186885 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9940cc360 con 0x7fd9940cbe60 2024-04-04T06:15:19.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.316+0000 7fd998c00640 1 --2- 172.21.15.159:0/3339186885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9940cbe60 0x7fd9940b95a0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fd9940b6fc0 tx=0x7fd98c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:19.310 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.316+0000 7fd98be00640 1 -- 172.21.15.159:0/3339186885 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c017070 con 0x7fd9940cbe60 2024-04-04T06:15:19.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.316+0000 7fd98be00640 1 -- 172.21.15.159:0/3339186885 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd98c005d60 con 0x7fd9940cbe60 2024-04-04T06:15:19.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.316+0000 7fd98be00640 1 -- 172.21.15.159:0/3339186885 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c00a850 con 0x7fd9940cbe60 2024-04-04T06:15:19.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.316+0000 7fd99ac00640 1 -- 172.21.15.159:0/3339186885 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9940cc640 con 0x7fd9940cbe60 2024-04-04T06:15:19.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.317+0000 7fd99ac00640 1 -- 172.21.15.159:0/3339186885 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9940ccc00 con 0x7fd9940cbe60 2024-04-04T06:15:19.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.318+0000 7fd99ac00640 1 -- 172.21.15.159:0/3339186885 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9940685c0 con 0x7fd9940cbe60 2024-04-04T06:15:19.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.319+0000 7fd98be00640 1 -- 172.21.15.159:0/3339186885 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd98c00a9f0 con 0x7fd9940cbe60 2024-04-04T06:15:19.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.320+0000 7fd98be00640 1 --2- 172.21.15.159:0/3339186885 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd97406c8f0 0x7fd97406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:19.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.320+0000 7fd98be00640 1 -- 172.21.15.159:0/3339186885 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fd98c08ad00 con 0x7fd9940cbe60 2024-04-04T06:15:19.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.320+0000 7fd993e00640 1 --2- 172.21.15.159:0/3339186885 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd97406c8f0 0x7fd97406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:19.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.321+0000 7fd993e00640 1 --2- 172.21.15.159:0/3339186885 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd97406c8f0 0x7fd97406edb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd9840013d0 tx=0x7fd984001320 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:19.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.324+0000 7fd98be00640 1 -- 172.21.15.159:0/3339186885 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd98c0591c0 con 0x7fd9940cbe60 2024-04-04T06:15:19.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.640+0000 7fd99ac00640 1 -- 172.21.15.159:0/3339186885 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd9940b5270 con 0x7fd9940cbe60 2024-04-04T06:15:19.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.641+0000 7fd98be00640 1 -- 172.21.15.159:0/3339186885 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v101) ==== 80+0+0 (secure 0 0 0) 0x7fd98c058ac0 con 0x7fd9940cbe60 2024-04-04T06:15:19.634 INFO:teuthology.orchestra.run.smithi159.stderr:listed 0 entries 2024-04-04T06:15:19.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.643+0000 7fd99ac00640 1 -- 172.21.15.159:0/3339186885 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd97406c8f0 msgr2=0x7fd97406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:19.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.643+0000 7fd99ac00640 1 --2- 172.21.15.159:0/3339186885 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd97406c8f0 0x7fd97406edb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd9840013d0 tx=0x7fd984001320 comp rx=0 tx=0).stop 2024-04-04T06:15:19.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.644+0000 7fd99ac00640 1 -- 172.21.15.159:0/3339186885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9940cbe60 msgr2=0x7fd9940b95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:19.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.644+0000 7fd99ac00640 1 --2- 172.21.15.159:0/3339186885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9940cbe60 0x7fd9940b95a0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fd9940b6fc0 tx=0x7fd98c00cb90 comp rx=0 tx=0).stop 2024-04-04T06:15:19.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.644+0000 7fd99ac00640 1 -- 172.21.15.159:0/3339186885 shutdown_connections 2024-04-04T06:15:19.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.644+0000 7fd99ac00640 1 --2- 172.21.15.159:0/3339186885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9940cbe60 0x7fd9940b95a0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:19.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.644+0000 7fd99ac00640 1 --2- 172.21.15.159:0/3339186885 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9940cd400 0x7fd9940a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:19.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.644+0000 7fd99ac00640 1 --2- 172.21.15.159:0/3339186885 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd97406c8f0 0x7fd97406edb0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:19.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.644+0000 7fd99ac00640 1 --2- 172.21.15.159:0/3339186885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd978004800 0x7fd9940b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:19.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.644+0000 7fd99ac00640 1 -- 172.21.15.159:0/3339186885 >> 172.21.15.159:0/3339186885 conn(0x7fd99408bfb0 msgr2=0x7fd9940cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:19.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.644+0000 7fd99ac00640 1 -- 172.21.15.159:0/3339186885 shutdown_connections 2024-04-04T06:15:19.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:19.644+0000 7fd99ac00640 1 -- 172.21.15.159:0/3339186885 wait complete. 2024-04-04T06:15:19.655 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-04T06:15:19.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:19.656 DEBUG:teuthology.orchestra.run.smithi159:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-04T06:15:19.745 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:15:19.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:19.745 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-04T06:15:19.789 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:15:20.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.035+0000 7f70fee00640 1 Processor -- start 2024-04-04T06:15:20.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.035+0000 7f70fee00640 1 -- start start 2024-04-04T06:15:20.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.036+0000 7f70fee00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70f80d0490 0x7f70f80cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:20.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.036+0000 7f70fee00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f70f80cdae0 0x7f70f80cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:20.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.036+0000 7f70fee00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70f806b610 con 0x7f70f80b0420 2024-04-04T06:15:20.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.036+0000 7f70fee00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f70f815eca0 con 0x7f70f80cdae0 2024-04-04T06:15:20.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.036+0000 7f70fee00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f70f806c220 con 0x7f70f80d0490 2024-04-04T06:15:20.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.037+0000 7f70f7400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f70f80b0420 0x7f70f80d0870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41168/0 (socket says 172.21.15.159:41168) 2024-04-04T06:15:20.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.037+0000 7f70f7400640 1 -- 172.21.15.159:0/1798741908 learned_addr learned my addr 172.21.15.159:0/1798741908 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:20.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.037+0000 7f70f7e00640 1 --2- 172.21.15.159:0/1798741908 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70f80d0490 0x7f70f80cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:20.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.037+0000 7f70fce00640 1 --2- 172.21.15.159:0/1798741908 >> v2:172.21.15.178:3300/0 conn(0x7f70f80cdae0 0x7f70f80cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:20.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.037+0000 7f70f6a00640 1 -- 172.21.15.159:0/1798741908 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 127880285 0 0) 0x7f70f806b610 con 0x7f70f80b0420 2024-04-04T06:15:20.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.037+0000 7f70f6a00640 1 -- 172.21.15.159:0/1798741908 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70dc003520 con 0x7f70f80b0420 2024-04-04T06:15:20.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.038+0000 7f70f6a00640 1 -- 172.21.15.159:0/1798741908 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2067901213 0 0) 0x7f70dc003520 con 0x7f70f80b0420 2024-04-04T06:15:20.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.038+0000 7f70f6a00640 1 -- 172.21.15.159:0/1798741908 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70f80d0490 msgr2=0x7f70f80cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:20.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.038+0000 7f70f6a00640 1 --2- 172.21.15.159:0/1798741908 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70f80d0490 0x7f70f80cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:20.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.038+0000 7f70f6a00640 1 -- 172.21.15.159:0/1798741908 >> v2:172.21.15.178:3300/0 conn(0x7f70f80cdae0 msgr2=0x7f70f80cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:20.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.038+0000 7f70f6a00640 1 --2- 172.21.15.159:0/1798741908 >> v2:172.21.15.178:3300/0 conn(0x7f70f80cdae0 0x7f70f80cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:20.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.038+0000 7f70f6a00640 1 -- 172.21.15.159:0/1798741908 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f806b8f0 con 0x7f70f80b0420 2024-04-04T06:15:20.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.038+0000 7f70f6a00640 1 -- 172.21.15.159:0/1798741908 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f70e4001670 con 0x7f70f80b0420 2024-04-04T06:15:20.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.038+0000 7f70f6a00640 1 -- 172.21.15.159:0/1798741908 >> v1:172.21.15.159:6789/0 conn(0x7f70f80b0420 legacy=0x7f70f80d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:20.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.038+0000 7f70f6a00640 1 --2- 172.21.15.159:0/1798741908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70dc0039b0 0x7f70dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:20.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.038+0000 7f70f6a00640 1 --2- 172.21.15.159:0/1798741908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70dc004800 0x7f70dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:20.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.038+0000 7f70f6a00640 1 --2- 172.21.15.159:0/1798741908 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70f80d0490 0x7f70dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:20.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.038+0000 7f70f6a00640 1 -- 172.21.15.159:0/1798741908 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f70f815eca0 con 0x7f70dc0039b0 2024-04-04T06:15:20.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.038+0000 7f70f6a00640 1 -- 172.21.15.159:0/1798741908 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f70f806c220 con 0x7f70dc004800 2024-04-04T06:15:20.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.038+0000 7f70f6a00640 1 -- 172.21.15.159:0/1798741908 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f70dc00b9c0 con 0x7f70f80d0490 2024-04-04T06:15:20.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.038+0000 7f70f7e00640 1 --2- 172.21.15.159:0/1798741908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70dc0039b0 0x7f70dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:20.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.039+0000 7f70f7400640 1 --2- 172.21.15.159:0/1798741908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70dc004800 0x7f70dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:20.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.039+0000 7f70f7e00640 1 -- 172.21.15.159:0/1798741908 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70f80d0490 msgr2=0x7f70dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:20.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.039+0000 7f70f7e00640 1 --2- 172.21.15.159:0/1798741908 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70f80d0490 0x7f70dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:20.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.039+0000 7f70f7e00640 1 -- 172.21.15.159:0/1798741908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70dc004800 msgr2=0x7f70dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:20.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.039+0000 7f70f7e00640 1 --2- 172.21.15.159:0/1798741908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70dc004800 0x7f70dc004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:20.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.039+0000 7f70f7e00640 1 -- 172.21.15.159:0/1798741908 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70dc003520 con 0x7f70dc0039b0 2024-04-04T06:15:20.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.040+0000 7f70f7e00640 1 --2- 172.21.15.159:0/1798741908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70dc0039b0 0x7f70dc003dd0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009ac0 tx=0x7f70dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934733f7022ecd03 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:20.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.040+0000 7f70f6a00640 1 -- 172.21.15.159:0/1798741908 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e8039070 con 0x7f70dc0039b0 2024-04-04T06:15:20.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.040+0000 7f70f6a00640 1 -- 172.21.15.159:0/1798741908 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e802c710 con 0x7f70dc0039b0 2024-04-04T06:15:20.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.040+0000 7f70f6a00640 1 -- 172.21.15.159:0/1798741908 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e8009690 con 0x7f70dc0039b0 2024-04-04T06:15:20.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.041+0000 7f70fee00640 1 -- 172.21.15.159:0/1798741908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70dc0039b0 msgr2=0x7f70dc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:20.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.041+0000 7f70fee00640 1 --2- 172.21.15.159:0/1798741908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70dc0039b0 0x7f70dc003dd0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009ac0 tx=0x7f70dc005830 comp rx=0 tx=0).stop 2024-04-04T06:15:20.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.041+0000 7f70fce00640 1 -- 172.21.15.159:0/1798741908 reap_dead start 2024-04-04T06:15:20.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.041+0000 7f70fee00640 1 -- 172.21.15.159:0/1798741908 shutdown_connections 2024-04-04T06:15:20.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.041+0000 7f70fee00640 1 -- 172.21.15.159:0/1798741908 >> 172.21.15.159:0/1798741908 conn(0x7f70f80b2d40 msgr2=0x7f70f80b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:20.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.041+0000 7f70fee00640 1 -- 172.21.15.159:0/1798741908 shutdown_connections 2024-04-04T06:15:20.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.041+0000 7f70fee00640 1 -- 172.21.15.159:0/1798741908 wait complete. 2024-04-04T06:15:20.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.043+0000 7f70fee00640 1 Processor -- start 2024-04-04T06:15:20.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.043+0000 7f70fee00640 1 -- start start 2024-04-04T06:15:20.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.044+0000 7f70fee00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70dc0039b0 0x7f70f81f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:20.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.044+0000 7f70fee00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70f80c6ba0 0x7f70f8156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:20.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.044+0000 7f70fee00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70f81f77c0 0x7f70f815c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:20.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.044+0000 7f70fee00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f70f80cb7c0 con 0x7f70f81f77c0 2024-04-04T06:15:20.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.044+0000 7f70fee00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f70f806beb0 con 0x7f70f80c6ba0 2024-04-04T06:15:20.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.044+0000 7f70fee00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f70f80cb940 con 0x7f70dc0039b0 2024-04-04T06:15:20.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.044+0000 7f70fce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70f81f77c0 0x7f70f815c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:20.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.044+0000 7f70fce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70f81f77c0 0x7f70f815c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49636/0 (socket says 172.21.15.159:49636) 2024-04-04T06:15:20.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.044+0000 7f70fce00640 1 -- 172.21.15.159:0/3545090844 learned_addr learned my addr 172.21.15.159:0/3545090844 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:20.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.045+0000 7f70f7400640 1 --2- 172.21.15.159:0/3545090844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70f80c6ba0 0x7f70f8156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:20.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.045+0000 7f70fce00640 1 -- 172.21.15.159:0/3545090844 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70dc0039b0 msgr2=0x7f70f81f4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:20.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.045+0000 7f70fce00640 1 --2- 172.21.15.159:0/3545090844 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70dc0039b0 0x7f70f81f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:20.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.045+0000 7f70fce00640 1 -- 172.21.15.159:0/3545090844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70f80c6ba0 msgr2=0x7f70f8156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:20.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.045+0000 7f70fce00640 1 --2- 172.21.15.159:0/3545090844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70f80c6ba0 0x7f70f8156ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:20.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.045+0000 7f70fce00640 1 -- 172.21.15.159:0/3545090844 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f81ee6a0 con 0x7f70f81f77c0 2024-04-04T06:15:20.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.045+0000 7f70fce00640 1 --2- 172.21.15.159:0/3545090844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70f81f77c0 0x7f70f815c3d0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f70f806d5a0 tx=0x7f70f000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:20.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.045+0000 7f70efe00640 1 -- 172.21.15.159:0/3545090844 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f000bce0 con 0x7f70f81f77c0 2024-04-04T06:15:20.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.045+0000 7f70efe00640 1 -- 172.21.15.159:0/3545090844 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f001a070 con 0x7f70f81f77c0 2024-04-04T06:15:20.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.045+0000 7f70fee00640 1 -- 172.21.15.159:0/3545090844 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f81f81a0 con 0x7f70f81f77c0 2024-04-04T06:15:20.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.045+0000 7f70efe00640 1 -- 172.21.15.159:0/3545090844 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0014640 con 0x7f70f81f77c0 2024-04-04T06:15:20.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.045+0000 7f70fee00640 1 -- 172.21.15.159:0/3545090844 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f81f43e0 con 0x7f70f81f77c0 2024-04-04T06:15:20.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.047+0000 7f70fee00640 1 -- 172.21.15.159:0/3545090844 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70f81f41d0 con 0x7f70f81f77c0 2024-04-04T06:15:20.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.047+0000 7f70efe00640 1 -- 172.21.15.159:0/3545090844 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f70f0005d10 con 0x7f70f81f77c0 2024-04-04T06:15:20.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.048+0000 7f70efe00640 1 --2- 172.21.15.159:0/3545090844 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f70d806c920 0x7f70d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:20.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.048+0000 7f70efe00640 1 -- 172.21.15.159:0/3545090844 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f70f008bbe0 con 0x7f70f81f77c0 2024-04-04T06:15:20.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.049+0000 7f70f7e00640 1 --2- 172.21.15.159:0/3545090844 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f70d806c920 0x7f70d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:20.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.050+0000 7f70f7e00640 1 --2- 172.21.15.159:0/3545090844 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f70d806c920 0x7f70d806ede0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f70e8036000 tx=0x7f70e802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:20.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.052+0000 7f70efe00640 1 -- 172.21.15.159:0/3545090844 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f70f005a010 con 0x7f70f81f77c0 2024-04-04T06:15:20.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.333+0000 7f70fee00640 1 -- 172.21.15.159:0/3545090844 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70f8068cd0 con 0x7f70f81f77c0 2024-04-04T06:15:20.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.337+0000 7f70efe00640 1 -- 172.21.15.159:0/3545090844 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5189 (secure 0 0 0) 0x7f70f0059910 con 0x7f70f81f77c0 2024-04-04T06:15:20.330 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:15:20.330 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":84,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5546,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/40219476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":40219476},{"type":"v1","addr":"172.21.15.178:6837","nonce":40219476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/4060483382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4060483382},{"type":"v1","addr":"172.21.15.178:6836","nonce":4060483382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8187,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1945665739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1945665739},{"type":"v1","addr":"172.21.15.159:6833","nonce":1945665739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:15:07.704835+0000","modified":"2024-04-04T06:15:10.811654+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8190":{"gid":8190,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6835/2253383212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2253383212},{"type":"v1","addr":"172.21.15.159:6835","nonce":2253383212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8190,"qdb_cluster":[8190]},"id":7}]} 2024-04-04T06:15:20.331 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 84 2024-04-04T06:15:20.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.339+0000 7f70fee00640 1 -- 172.21.15.159:0/3545090844 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f70d806c920 msgr2=0x7f70d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:20.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.339+0000 7f70fee00640 1 --2- 172.21.15.159:0/3545090844 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f70d806c920 0x7f70d806ede0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f70e8036000 tx=0x7f70e802e000 comp rx=0 tx=0).stop 2024-04-04T06:15:20.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.339+0000 7f70fee00640 1 -- 172.21.15.159:0/3545090844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70f81f77c0 msgr2=0x7f70f815c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:20.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.339+0000 7f70fee00640 1 --2- 172.21.15.159:0/3545090844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70f81f77c0 0x7f70f815c3d0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f70f806d5a0 tx=0x7f70f000b480 comp rx=0 tx=0).stop 2024-04-04T06:15:20.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.339+0000 7f70fee00640 1 -- 172.21.15.159:0/3545090844 shutdown_connections 2024-04-04T06:15:20.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.339+0000 7f70fee00640 1 --2- 172.21.15.159:0/3545090844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70f81f77c0 0x7f70f815c3d0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:20.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.339+0000 7f70fee00640 1 --2- 172.21.15.159:0/3545090844 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f70d806c920 0x7f70d806ede0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:20.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.339+0000 7f70fee00640 1 --2- 172.21.15.159:0/3545090844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70f80c6ba0 0x7f70f8156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:20.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.339+0000 7f70fee00640 1 --2- 172.21.15.159:0/3545090844 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70dc0039b0 0x7f70f81f4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:20.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.339+0000 7f70fee00640 1 -- 172.21.15.159:0/3545090844 >> 172.21.15.159:0/3545090844 conn(0x7f70f80b2d40 msgr2=0x7f70f806f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:20.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.340+0000 7f70fee00640 1 -- 172.21.15.159:0/3545090844 shutdown_connections 2024-04-04T06:15:20.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.340+0000 7f70fee00640 1 -- 172.21.15.159:0/3545090844 wait complete. 2024-04-04T06:15:20.352 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-04T06:15:20.590 INFO:teuthology.orchestra.run.smithi159.stdout:{"message":"","return_code":0} 2024-04-04T06:15:20.591 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-04T06:15:20.591 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:15:20.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.808+0000 7f02c7200640 1 Processor -- start 2024-04-04T06:15:20.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.808+0000 7f02c7200640 1 -- start start 2024-04-04T06:15:20.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.810+0000 7f02c7200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02c015c390 0x7f02c0073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:20.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.810+0000 7f02c7200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02c015fe80 con 0x7f02c00af830 2024-04-04T06:15:20.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.810+0000 7f02c7200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02c00ad120 con 0x7f02c00d0650 2024-04-04T06:15:20.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.810+0000 7f02c7200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f02c015c770 con 0x7f02c015c390 2024-04-04T06:15:20.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.810+0000 7f02bf400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f02c00af830 0x7f02c0160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41172/0 (socket says 172.21.15.159:41172) 2024-04-04T06:15:20.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.810+0000 7f02bf400640 1 -- 172.21.15.159:0/512575531 learned_addr learned my addr 172.21.15.159:0/512575531 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:20.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.811+0000 7f02bfe00640 1 --2- 172.21.15.159:0/512575531 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02c015c390 0x7f02c0073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:20.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.811+0000 7f02bea00640 1 -- 172.21.15.159:0/512575531 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1413574530 0 0) 0x7f02c015fe80 con 0x7f02c00af830 2024-04-04T06:15:20.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.811+0000 7f02bea00640 1 -- 172.21.15.159:0/512575531 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02a4003520 con 0x7f02c00af830 2024-04-04T06:15:20.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.811+0000 7f02bea00640 1 -- 172.21.15.159:0/512575531 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 182496920 0 0) 0x7f02a4003520 con 0x7f02c00af830 2024-04-04T06:15:20.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.811+0000 7f02bea00640 1 -- 172.21.15.159:0/512575531 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02c015c390 msgr2=0x7f02c0073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:20.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.811+0000 7f02bea00640 1 --2- 172.21.15.159:0/512575531 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02c015c390 0x7f02c0073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:20.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.811+0000 7f02bea00640 1 -- 172.21.15.159:0/512575531 >> v1:172.21.15.178:6789/0 conn(0x7f02c00d0650 legacy=0x7f02c0071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:20.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.811+0000 7f02bea00640 1 -- 172.21.15.159:0/512575531 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c00ad400 con 0x7f02c00af830 2024-04-04T06:15:20.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.811+0000 7f02bea00640 1 -- 172.21.15.159:0/512575531 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f02ac001670 con 0x7f02c00af830 2024-04-04T06:15:20.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.811+0000 7f02bea00640 1 -- 172.21.15.159:0/512575531 >> v1:172.21.15.159:6789/0 conn(0x7f02c00af830 legacy=0x7f02c0160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:20.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.811+0000 7f02bea00640 1 --2- 172.21.15.159:0/512575531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02a40039e0 0x7f02a4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:20.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.811+0000 7f02bea00640 1 --2- 172.21.15.159:0/512575531 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02c015c390 0x7f02a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:20.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.811+0000 7f02bea00640 1 --2- 172.21.15.159:0/512575531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02a4004d40 0x7f02a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:20.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.811+0000 7f02bea00640 1 -- 172.21.15.159:0/512575531 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f02a400ba70 con 0x7f02a40039e0 2024-04-04T06:15:20.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.811+0000 7f02bea00640 1 -- 172.21.15.159:0/512575531 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f02a400bc20 con 0x7f02a4004d40 2024-04-04T06:15:20.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.811+0000 7f02bea00640 1 -- 172.21.15.159:0/512575531 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f02a400be20 con 0x7f02c015c390 2024-04-04T06:15:20.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.811+0000 7f02bfe00640 1 --2- 172.21.15.159:0/512575531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02a40039e0 0x7f02a4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:20.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.812+0000 7f02bf400640 1 --2- 172.21.15.159:0/512575531 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02c015c390 0x7f02a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:20.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.812+0000 7f02bfe00640 1 -- 172.21.15.159:0/512575531 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02c015c390 msgr2=0x7f02a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:20.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.812+0000 7f02c5200640 1 --2- 172.21.15.159:0/512575531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02a4004d40 0x7f02a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:20.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.812+0000 7f02bfe00640 1 --2- 172.21.15.159:0/512575531 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02c015c390 0x7f02a4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:20.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.812+0000 7f02bfe00640 1 -- 172.21.15.159:0/512575531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02a4004d40 msgr2=0x7f02a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:20.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.812+0000 7f02bfe00640 1 --2- 172.21.15.159:0/512575531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02a4004d40 0x7f02a40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:20.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.812+0000 7f02bfe00640 1 -- 172.21.15.159:0/512575531 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a4003520 con 0x7f02a40039e0 2024-04-04T06:15:20.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.812+0000 7f02bf400640 1 --2- 172.21.15.159:0/512575531 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02c015c390 0x7f02a4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:20.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.813+0000 7f02bfe00640 1 --2- 172.21.15.159:0/512575531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02a40039e0 0x7f02a4003e00 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009a90 tx=0x7f02a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f117f5ae73a91961 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:20.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.813+0000 7f02bea00640 1 -- 172.21.15.159:0/512575531 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0039070 con 0x7f02a40039e0 2024-04-04T06:15:20.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.813+0000 7f02bea00640 1 -- 172.21.15.159:0/512575531 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b002c680 con 0x7f02a40039e0 2024-04-04T06:15:20.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.813+0000 7f02bea00640 1 -- 172.21.15.159:0/512575531 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0009690 con 0x7f02a40039e0 2024-04-04T06:15:20.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.814+0000 7f02c7200640 1 -- 172.21.15.159:0/512575531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02a40039e0 msgr2=0x7f02a4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:20.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.814+0000 7f02c7200640 1 --2- 172.21.15.159:0/512575531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02a40039e0 0x7f02a4003e00 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009a90 tx=0x7f02a4005830 comp rx=0 tx=0).stop 2024-04-04T06:15:20.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.814+0000 7f02c5200640 1 -- 172.21.15.159:0/512575531 reap_dead start 2024-04-04T06:15:20.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.814+0000 7f02c7200640 1 -- 172.21.15.159:0/512575531 shutdown_connections 2024-04-04T06:15:20.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.814+0000 7f02c7200640 1 -- 172.21.15.159:0/512575531 >> 172.21.15.159:0/512575531 conn(0x7f02c006b990 msgr2=0x7f02c006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:20.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.814+0000 7f02c7200640 1 -- 172.21.15.159:0/512575531 shutdown_connections 2024-04-04T06:15:20.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.814+0000 7f02c7200640 1 -- 172.21.15.159:0/512575531 wait complete. 2024-04-04T06:15:20.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.816+0000 7f02c7200640 1 Processor -- start 2024-04-04T06:15:20.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.816+0000 7f02c7200640 1 -- start start 2024-04-04T06:15:20.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.817+0000 7f02c7200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02a40039e0 0x7f02c00ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:20.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.817+0000 7f02c7200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02c01f4160 0x7f02c015eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:20.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.817+0000 7f02c7200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02c01f45e0 0x7f02c00b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:20.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.817+0000 7f02c7200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f02c0160d50 con 0x7f02a40039e0 2024-04-04T06:15:20.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.817+0000 7f02c7200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f02c0160120 con 0x7f02c01f4160 2024-04-04T06:15:20.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.817+0000 7f02c7200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f02c00cf980 con 0x7f02c01f45e0 2024-04-04T06:15:20.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.817+0000 7f02bfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02a40039e0 0x7f02c00ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:20.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.817+0000 7f02bfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02a40039e0 0x7f02c00ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49654/0 (socket says 172.21.15.159:49654) 2024-04-04T06:15:20.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.817+0000 7f02bfe00640 1 -- 172.21.15.159:0/3210359034 learned_addr learned my addr 172.21.15.159:0/3210359034 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:20.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.818+0000 7f02bfe00640 1 -- 172.21.15.159:0/3210359034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02c01f45e0 msgr2=0x7f02c00b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:20.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.818+0000 7f02bfe00640 1 --2- 172.21.15.159:0/3210359034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02c01f45e0 0x7f02c00b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:20.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.818+0000 7f02bfe00640 1 -- 172.21.15.159:0/3210359034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02c01f4160 msgr2=0x7f02c015eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:20.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.818+0000 7f02bfe00640 1 --2- 172.21.15.159:0/3210359034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02c01f4160 0x7f02c015eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:20.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.818+0000 7f02bfe00640 1 -- 172.21.15.159:0/3210359034 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c015ba10 con 0x7f02a40039e0 2024-04-04T06:15:20.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.818+0000 7f02bfe00640 1 --2- 172.21.15.159:0/3210359034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02a40039e0 0x7f02c00ce940 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f02b0037040 tx=0x7f02b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:20.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.818+0000 7f02b7e00640 1 -- 172.21.15.159:0/3210359034 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0039050 con 0x7f02a40039e0 2024-04-04T06:15:20.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.818+0000 7f02b7e00640 1 -- 172.21.15.159:0/3210359034 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b0045070 con 0x7f02a40039e0 2024-04-04T06:15:20.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.818+0000 7f02c7200640 1 -- 172.21.15.159:0/3210359034 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c0158ce0 con 0x7f02a40039e0 2024-04-04T06:15:20.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.819+0000 7f02b7e00640 1 -- 172.21.15.159:0/3210359034 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0003850 con 0x7f02a40039e0 2024-04-04T06:15:20.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.819+0000 7f02c7200640 1 -- 172.21.15.159:0/3210359034 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c00cee80 con 0x7f02a40039e0 2024-04-04T06:15:20.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.820+0000 7f02c7200640 1 -- 172.21.15.159:0/3210359034 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c00ae620 con 0x7f02a40039e0 2024-04-04T06:15:20.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.821+0000 7f02b7e00640 1 -- 172.21.15.159:0/3210359034 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f02b00039f0 con 0x7f02a40039e0 2024-04-04T06:15:20.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.822+0000 7f02b7e00640 1 --2- 172.21.15.159:0/3210359034 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f02a006c8f0 0x7f02a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:20.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.822+0000 7f02b7e00640 1 -- 172.21.15.159:0/3210359034 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f02b00ada60 con 0x7f02a40039e0 2024-04-04T06:15:20.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.822+0000 7f02bf400640 1 --2- 172.21.15.159:0/3210359034 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f02a006c8f0 0x7f02a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:20.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.823+0000 7f02bf400640 1 --2- 172.21.15.159:0/3210359034 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f02a006c8f0 0x7f02a006edb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f02ac000ff0 tx=0x7f02ac009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:20.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:20.826+0000 7f02b7e00640 1 -- 172.21.15.159:0/3210359034 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f02c00ae620 con 0x7f02a40039e0 2024-04-04T06:15:21.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.140+0000 7f02c7200640 1 -- 172.21.15.159:0/3210359034 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02c00685c0 con 0x7f02a40039e0 2024-04-04T06:15:21.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.142+0000 7f02b7e00640 1 -- 172.21.15.159:0/3210359034 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5189 (secure 0 0 0) 0x7f02b004f020 con 0x7f02a40039e0 2024-04-04T06:15:21.135 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:15:21.135 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":84,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5546,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/40219476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":40219476},{"type":"v1","addr":"172.21.15.178:6837","nonce":40219476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/4060483382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4060483382},{"type":"v1","addr":"172.21.15.178:6836","nonce":4060483382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8187,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1945665739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1945665739},{"type":"v1","addr":"172.21.15.159:6833","nonce":1945665739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:15:07.704835+0000","modified":"2024-04-04T06:15:10.811654+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8190":{"gid":8190,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6835/2253383212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2253383212},{"type":"v1","addr":"172.21.15.159:6835","nonce":2253383212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8190,"qdb_cluster":[8190]},"id":7}]} 2024-04-04T06:15:21.135 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 84 2024-04-04T06:15:21.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.144+0000 7f02c7200640 1 -- 172.21.15.159:0/3210359034 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f02a006c8f0 msgr2=0x7f02a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:21.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.144+0000 7f02c7200640 1 --2- 172.21.15.159:0/3210359034 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f02a006c8f0 0x7f02a006edb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f02ac000ff0 tx=0x7f02ac009040 comp rx=0 tx=0).stop 2024-04-04T06:15:21.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.144+0000 7f02c7200640 1 -- 172.21.15.159:0/3210359034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02a40039e0 msgr2=0x7f02c00ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:21.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.144+0000 7f02c7200640 1 --2- 172.21.15.159:0/3210359034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02a40039e0 0x7f02c00ce940 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f02b0037040 tx=0x7f02b0004730 comp rx=0 tx=0).stop 2024-04-04T06:15:21.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.144+0000 7f02c7200640 1 -- 172.21.15.159:0/3210359034 shutdown_connections 2024-04-04T06:15:21.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.144+0000 7f02c7200640 1 --2- 172.21.15.159:0/3210359034 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f02c01f45e0 0x7f02c00b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.144+0000 7f02c7200640 1 --2- 172.21.15.159:0/3210359034 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f02a006c8f0 0x7f02a006edb0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.145+0000 7f02c7200640 1 --2- 172.21.15.159:0/3210359034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02c01f4160 0x7f02c015eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.145+0000 7f02c7200640 1 --2- 172.21.15.159:0/3210359034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02a40039e0 0x7f02c00ce940 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.145+0000 7f02c7200640 1 -- 172.21.15.159:0/3210359034 >> 172.21.15.159:0/3210359034 conn(0x7f02c006b990 msgr2=0x7f02c00aece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:21.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.145+0000 7f02c7200640 1 -- 172.21.15.159:0/3210359034 shutdown_connections 2024-04-04T06:15:21.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.145+0000 7f02c7200640 1 -- 172.21.15.159:0/3210359034 wait complete. 2024-04-04T06:15:21.155 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-04-04T06:15:21.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.253+0000 7effe003bf40 1 Processor -- start 2024-04-04T06:15:21.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.253+0000 7effe003bf40 1 -- start start 2024-04-04T06:15:21.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.254+0000 7effe003bf40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56168e7204c0 0x56168e705120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:21.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.254+0000 7effe003bf40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56168e721990 0x56168e7aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:21.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.254+0000 7effe003bf40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56168e7adad0 0x56168e6d78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:21.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.254+0000 7effe003bf40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56168e7adeb0 con 0x56168e7adad0 2024-04-04T06:15:21.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.254+0000 7effe003bf40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56168e67f2c0 con 0x56168e721990 2024-04-04T06:15:21.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.254+0000 7effe003bf40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56168e720cb0 con 0x56168e7204c0 2024-04-04T06:15:21.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.255+0000 7effdd400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56168e7adad0 0x56168e6d78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:21.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.255+0000 7effdd400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56168e7adad0 0x56168e6d78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49674/0 (socket says 172.21.15.159:49674) 2024-04-04T06:15:21.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.255+0000 7effdd400640 1 -- 172.21.15.159:0/1157088504 learned_addr learned my addr 172.21.15.159:0/1157088504 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:21.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.255+0000 7effdc000640 1 --2- 172.21.15.159:0/1157088504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56168e721990 0x56168e7aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:21.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.255+0000 7effdca00640 1 --2- 172.21.15.159:0/1157088504 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56168e7204c0 0x56168e705120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:21.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.255+0000 7effdd400640 1 -- 172.21.15.159:0/1157088504 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56168e7204c0 msgr2=0x56168e705120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:21.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.255+0000 7effdd400640 1 --2- 172.21.15.159:0/1157088504 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56168e7204c0 0x56168e705120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.255+0000 7effdd400640 1 -- 172.21.15.159:0/1157088504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56168e721990 msgr2=0x56168e7aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:21.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.255+0000 7effdd400640 1 --2- 172.21.15.159:0/1157088504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56168e721990 0x56168e7aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.255+0000 7effdd400640 1 -- 172.21.15.159:0/1157088504 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56168e7b1cc0 con 0x56168e7adad0 2024-04-04T06:15:21.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.255+0000 7effdc000640 1 --2- 172.21.15.159:0/1157088504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56168e721990 0x56168e7aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:21.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.255+0000 7effdd400640 1 --2- 172.21.15.159:0/1157088504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56168e7adad0 0x56168e6d78a0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7effd40015f0 tx=0x7effd4001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2afba74336f7a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:21.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.256+0000 7effdac00640 1 -- 172.21.15.159:0/1157088504 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd4003650 con 0x56168e7adad0 2024-04-04T06:15:21.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.256+0000 7effdac00640 1 -- 172.21.15.159:0/1157088504 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd40037f0 con 0x56168e7adad0 2024-04-04T06:15:21.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.256+0000 7effdac00640 1 -- 172.21.15.159:0/1157088504 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd4003c20 con 0x56168e7adad0 2024-04-04T06:15:21.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.256+0000 7effe003bf40 1 -- 172.21.15.159:0/1157088504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56168e7adad0 msgr2=0x56168e6d78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:21.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.256+0000 7effe003bf40 1 --2- 172.21.15.159:0/1157088504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56168e7adad0 0x56168e6d78a0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7effd40015f0 tx=0x7effd4001ab0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.256+0000 7effe003bf40 1 -- 172.21.15.159:0/1157088504 shutdown_connections 2024-04-04T06:15:21.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.256+0000 7effe003bf40 1 --2- 172.21.15.159:0/1157088504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56168e7adad0 0x56168e6d78a0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.256+0000 7effe003bf40 1 --2- 172.21.15.159:0/1157088504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56168e721990 0x56168e7aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.256+0000 7effe003bf40 1 --2- 172.21.15.159:0/1157088504 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56168e7204c0 0x56168e705120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.256+0000 7effe003bf40 1 -- 172.21.15.159:0/1157088504 >> 172.21.15.159:0/1157088504 conn(0x56168e703cf0 msgr2=0x56168e6d9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:21.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.256+0000 7effe003bf40 1 -- 172.21.15.159:0/1157088504 shutdown_connections 2024-04-04T06:15:21.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.256+0000 7effe003bf40 1 -- 172.21.15.159:0/1157088504 wait complete. 2024-04-04T06:15:21.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.257+0000 7effe003bf40 1 Processor -- start 2024-04-04T06:15:21.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.257+0000 7effe003bf40 1 -- start start 2024-04-04T06:15:21.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.258+0000 7effe003bf40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56168e7204c0 0x56168e7aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:21.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.258+0000 7effe003bf40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56168e721990 0x56168e7aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:21.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.258+0000 7effe003bf40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56168e7adad0 0x56168e7af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:21.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.258+0000 7effe003bf40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56168e7aeb00 con 0x56168e721990 2024-04-04T06:15:21.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.258+0000 7effe003bf40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56168e7ae8b0 con 0x56168e7204c0 2024-04-04T06:15:21.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.258+0000 7effe003bf40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x56168e7b6b60 con 0x56168e7adad0 2024-04-04T06:15:21.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.258+0000 7effdc000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56168e721990 0x56168e7aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:21.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.258+0000 7effdc000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56168e721990 0x56168e7aed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49684/0 (socket says 172.21.15.159:49684) 2024-04-04T06:15:21.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.258+0000 7effdc000640 1 -- 172.21.15.159:0/155995569 learned_addr learned my addr 172.21.15.159:0/155995569 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:21.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.259+0000 7effdca00640 1 --2- 172.21.15.159:0/155995569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56168e7204c0 0x56168e7aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:21.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.259+0000 7effdc000640 1 -- 172.21.15.159:0/155995569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56168e7adad0 msgr2=0x56168e7af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:21.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.259+0000 7effdc000640 1 --2- 172.21.15.159:0/155995569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56168e7adad0 0x56168e7af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.259+0000 7effdc000640 1 -- 172.21.15.159:0/155995569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56168e7204c0 msgr2=0x56168e7aa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:21.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.259+0000 7effdc000640 1 --2- 172.21.15.159:0/155995569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56168e7204c0 0x56168e7aa4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.259+0000 7effdc000640 1 -- 172.21.15.159:0/155995569 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56168e7aac10 con 0x56168e721990 2024-04-04T06:15:21.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.259+0000 7effdc000640 1 --2- 172.21.15.159:0/155995569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56168e721990 0x56168e7aed40 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7effc8007d80 tx=0x7effc8006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:21.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.259+0000 7effd9800640 1 -- 172.21.15.159:0/155995569 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8006e60 con 0x56168e721990 2024-04-04T06:15:21.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.259+0000 7effd9800640 1 -- 172.21.15.159:0/155995569 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effc8010710 con 0x56168e721990 2024-04-04T06:15:21.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.259+0000 7effe003bf40 1 -- 172.21.15.159:0/155995569 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56168e7b0530 con 0x56168e721990 2024-04-04T06:15:21.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.260+0000 7effd9800640 1 -- 172.21.15.159:0/155995569 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8010b00 con 0x56168e721990 2024-04-04T06:15:21.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.260+0000 7effe003bf40 1 -- 172.21.15.159:0/155995569 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56168e6d88d0 con 0x56168e721990 2024-04-04T06:15:21.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.262+0000 7effd9800640 1 -- 172.21.15.159:0/155995569 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7effc8010ca0 con 0x56168e721990 2024-04-04T06:15:21.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.263+0000 7effd9800640 1 --2- 172.21.15.159:0/155995569 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7effbc06b2d0 0x7effbc06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:21.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.263+0000 7effd9800640 1 -- 172.21.15.159:0/155995569 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7effc8089f70 con 0x56168e721990 2024-04-04T06:15:21.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.264+0000 7effe003bf40 1 --2- 172.21.15.159:0/155995569 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x56168e6da1a0 0x56168e71dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:21.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.264+0000 7effe003bf40 1 -- 172.21.15.159:0/155995569 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) -- 0x56168e71e2c0 con 0x56168e6da1a0 2024-04-04T06:15:21.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.264+0000 7effdca00640 1 --2- 172.21.15.159:0/155995569 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7effbc06b2d0 0x7effbc06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:21.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.264+0000 7effdd400640 1 --2- 172.21.15.159:0/155995569 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x56168e6da1a0 0x56168e71dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:21.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.265+0000 7effdd400640 1 --2- 172.21.15.159:0/155995569 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x56168e6da1a0 0x56168e71dd80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:21.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.265+0000 7effdca00640 1 --2- 172.21.15.159:0/155995569 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7effbc06b2d0 0x7effbc06d790 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7effcc054000 tx=0x7effcc04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:21.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.265+0000 7effdd400640 1 -- 172.21.15.159:0/155995569 <== osd.2 v2:172.21.15.159:6808/2619731047 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7effd400f030 con 0x56168e6da1a0 2024-04-04T06:15:21.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.265+0000 7effe003bf40 1 -- 172.21.15.159:0/155995569 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x56168e6da1a0 msgr2=0x56168e71dd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:21.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.265+0000 7effe003bf40 1 --2- 172.21.15.159:0/155995569 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x56168e6da1a0 0x56168e71dd80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.265+0000 7effe003bf40 1 -- 172.21.15.159:0/155995569 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7effbc06b2d0 msgr2=0x7effbc06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:21.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.265+0000 7effe003bf40 1 --2- 172.21.15.159:0/155995569 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7effbc06b2d0 0x7effbc06d790 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7effcc054000 tx=0x7effcc04e000 comp rx=0 tx=0).stop 2024-04-04T06:15:21.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.265+0000 7effe003bf40 1 -- 172.21.15.159:0/155995569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56168e721990 msgr2=0x56168e7aed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:21.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.265+0000 7effe003bf40 1 --2- 172.21.15.159:0/155995569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56168e721990 0x56168e7aed40 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7effc8007d80 tx=0x7effc8006750 comp rx=0 tx=0).stop 2024-04-04T06:15:21.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.265+0000 7effdd400640 1 -- 172.21.15.159:0/155995569 reap_dead start 2024-04-04T06:15:21.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.265+0000 7effe003bf40 1 -- 172.21.15.159:0/155995569 shutdown_connections 2024-04-04T06:15:21.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.265+0000 7effe003bf40 1 -- 172.21.15.159:0/155995569 >> 172.21.15.159:0/155995569 conn(0x56168e703cf0 msgr2=0x56168e705120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:21.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.266+0000 7effe003bf40 1 -- 172.21.15.159:0/155995569 shutdown_connections 2024-04-04T06:15:21.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.266+0000 7effe003bf40 1 -- 172.21.15.159:0/155995569 wait complete. 2024-04-04T06:15:21.263 INFO:teuthology.orchestra.run.smithi159.stdout:����� 2024-04-04T06:15:21.263 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-04T06:15:21.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.476+0000 7fc754c00640 1 Processor -- start 2024-04-04T06:15:21.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.476+0000 7fc754c00640 1 -- start start 2024-04-04T06:15:21.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.478+0000 7fc754c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc750161f60 0x7fc7500d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:21.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.478+0000 7fc754c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7500b5ce0 con 0x7fc7500d13b0 2024-04-04T06:15:21.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.478+0000 7fc754c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7500d27f0 con 0x7fc7500b3170 2024-04-04T06:15:21.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.478+0000 7fc754c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc7500d2a90 con 0x7fc750161f60 2024-04-04T06:15:21.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.479+0000 7fc74e800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fc7500d13b0 0x7fc7500b44b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41196/0 (socket says 172.21.15.159:41196) 2024-04-04T06:15:21.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.479+0000 7fc74e800640 1 -- 172.21.15.159:0/3556750508 learned_addr learned my addr 172.21.15.159:0/3556750508 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:21.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.479+0000 7fc74de00640 1 --2- 172.21.15.159:0/3556750508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc750161f60 0x7fc7500d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:21.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.479+0000 7fc74d400640 1 -- 172.21.15.159:0/3556750508 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3818601490 0 0) 0x7fc748001490 con 0x7fc7500d13b0 2024-04-04T06:15:21.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.479+0000 7fc74d400640 1 -- 172.21.15.159:0/3556750508 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc734003520 con 0x7fc7500d13b0 2024-04-04T06:15:21.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.479+0000 7fc74d400640 1 -- 172.21.15.159:0/3556750508 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1816468835 0 0) 0x7fc734003520 con 0x7fc7500d13b0 2024-04-04T06:15:21.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.479+0000 7fc74d400640 1 -- 172.21.15.159:0/3556750508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc750161f60 msgr2=0x7fc7500d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:21.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.479+0000 7fc74d400640 1 --2- 172.21.15.159:0/3556750508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc750161f60 0x7fc7500d0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.479+0000 7fc74d400640 1 -- 172.21.15.159:0/3556750508 >> v1:172.21.15.178:6789/0 conn(0x7fc7500b3170 legacy=0x7fc7500b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:21.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.479+0000 7fc74d400640 1 -- 172.21.15.159:0/3556750508 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc750162340 con 0x7fc7500d13b0 2024-04-04T06:15:21.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.480+0000 7fc74d400640 1 -- 172.21.15.159:0/3556750508 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fc748001bc0 con 0x7fc7500d13b0 2024-04-04T06:15:21.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.480+0000 7fc74d400640 1 -- 172.21.15.159:0/3556750508 >> v1:172.21.15.159:6789/0 conn(0x7fc7500d13b0 legacy=0x7fc7500b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:21.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.480+0000 7fc74d400640 1 --2- 172.21.15.159:0/3556750508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7340039e0 0x7fc734003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:21.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.480+0000 7fc74d400640 1 --2- 172.21.15.159:0/3556750508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc734004340 0x7fc734004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:21.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.480+0000 7fc74d400640 1 --2- 172.21.15.159:0/3556750508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc750161f60 0x7fc734005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:21.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.480+0000 7fc74d400640 1 -- 172.21.15.159:0/3556750508 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc7500d2a90 con 0x7fc734004340 2024-04-04T06:15:21.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.480+0000 7fc74d400640 1 -- 172.21.15.159:0/3556750508 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc73400b9c0 con 0x7fc7340039e0 2024-04-04T06:15:21.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.480+0000 7fc74d400640 1 -- 172.21.15.159:0/3556750508 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc73400bb70 con 0x7fc750161f60 2024-04-04T06:15:21.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.480+0000 7fc747e00640 1 --2- 172.21.15.159:0/3556750508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc734004340 0x7fc734004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:21.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.480+0000 7fc74de00640 1 --2- 172.21.15.159:0/3556750508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7340039e0 0x7fc734003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:21.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.480+0000 7fc74e800640 1 --2- 172.21.15.159:0/3556750508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc750161f60 0x7fc734005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:21.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.481+0000 7fc747e00640 1 -- 172.21.15.159:0/3556750508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc750161f60 msgr2=0x7fc734005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:21.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.481+0000 7fc747e00640 1 --2- 172.21.15.159:0/3556750508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc750161f60 0x7fc734005080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.481+0000 7fc747e00640 1 -- 172.21.15.159:0/3556750508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7340039e0 msgr2=0x7fc734003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:21.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.481+0000 7fc747e00640 1 --2- 172.21.15.159:0/3556750508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7340039e0 0x7fc734003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.481+0000 7fc747e00640 1 -- 172.21.15.159:0/3556750508 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc750162340 con 0x7fc734004340 2024-04-04T06:15:21.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.481+0000 7fc74de00640 1 --2- 172.21.15.159:0/3556750508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7340039e0 0x7fc734003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:15:21.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.482+0000 7fc747e00640 1 --2- 172.21.15.159:0/3556750508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc734004340 0x7fc734004820 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fc738001910 tx=0x7fc734005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2d85b58e98a07f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:21.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.483+0000 7fc74d400640 1 -- 172.21.15.159:0/3556750508 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7380014e0 con 0x7fc734004340 2024-04-04T06:15:21.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.483+0000 7fc74d400640 1 -- 172.21.15.159:0/3556750508 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc738037070 con 0x7fc734004340 2024-04-04T06:15:21.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.483+0000 7fc74d400640 1 -- 172.21.15.159:0/3556750508 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc738032840 con 0x7fc734004340 2024-04-04T06:15:21.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.483+0000 7fc754c00640 1 -- 172.21.15.159:0/3556750508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc734004340 msgr2=0x7fc734004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:21.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.483+0000 7fc754c00640 1 --2- 172.21.15.159:0/3556750508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc734004340 0x7fc734004820 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fc738001910 tx=0x7fc734005830 comp rx=0 tx=0).stop 2024-04-04T06:15:21.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.483+0000 7fc74e800640 1 -- 172.21.15.159:0/3556750508 reap_dead start 2024-04-04T06:15:21.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.483+0000 7fc754c00640 1 -- 172.21.15.159:0/3556750508 shutdown_connections 2024-04-04T06:15:21.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.483+0000 7fc754c00640 1 -- 172.21.15.159:0/3556750508 >> 172.21.15.159:0/3556750508 conn(0x7fc75008bfb0 msgr2=0x7fc7500d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:21.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.483+0000 7fc754c00640 1 -- 172.21.15.159:0/3556750508 shutdown_connections 2024-04-04T06:15:21.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.483+0000 7fc754c00640 1 -- 172.21.15.159:0/3556750508 wait complete. 2024-04-04T06:15:21.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.485+0000 7fc754c00640 1 Processor -- start 2024-04-04T06:15:21.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.485+0000 7fc754c00640 1 -- start start 2024-04-04T06:15:21.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.487+0000 7fc754c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc734004340 0x7fc7500b8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:21.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.487+0000 7fc754c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7500b7310 0x7fc7500cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:21.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.487+0000 7fc754c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc75015cc90 0x7fc750153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:21.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.487+0000 7fc754c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc7500d3310 con 0x7fc7500b7310 2024-04-04T06:15:21.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.487+0000 7fc754c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc7500d3030 con 0x7fc75015cc90 2024-04-04T06:15:21.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.487+0000 7fc754c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc7500d0e70 con 0x7fc734004340 2024-04-04T06:15:21.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.487+0000 7fc747e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7500b7310 0x7fc7500cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:21.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.487+0000 7fc747e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7500b7310 0x7fc7500cd9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49704/0 (socket says 172.21.15.159:49704) 2024-04-04T06:15:21.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.487+0000 7fc747e00640 1 -- 172.21.15.159:0/2166026925 learned_addr learned my addr 172.21.15.159:0/2166026925 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:21.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.487+0000 7fc74e800640 1 --2- 172.21.15.159:0/2166026925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc75015cc90 0x7fc750153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:21.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.488+0000 7fc74de00640 1 --2- 172.21.15.159:0/2166026925 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc734004340 0x7fc7500b8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:21.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.488+0000 7fc747e00640 1 -- 172.21.15.159:0/2166026925 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc734004340 msgr2=0x7fc7500b8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:21.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.488+0000 7fc747e00640 1 --2- 172.21.15.159:0/2166026925 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc734004340 0x7fc7500b8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.488+0000 7fc747e00640 1 -- 172.21.15.159:0/2166026925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc75015cc90 msgr2=0x7fc750153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:21.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.488+0000 7fc747e00640 1 --2- 172.21.15.159:0/2166026925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc75015cc90 0x7fc750153af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.488+0000 7fc747e00640 1 -- 172.21.15.159:0/2166026925 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7500b77a0 con 0x7fc7500b7310 2024-04-04T06:15:21.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.488+0000 7fc74e800640 1 --2- 172.21.15.159:0/2166026925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc75015cc90 0x7fc750153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:21.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.488+0000 7fc747e00640 1 --2- 172.21.15.159:0/2166026925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7500b7310 0x7fc7500cd9b0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fc738001a30 tx=0x7fc7380012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:21.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.488+0000 7fc746000640 1 -- 172.21.15.159:0/2166026925 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc738006e50 con 0x7fc7500b7310 2024-04-04T06:15:21.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.488+0000 7fc746000640 1 -- 172.21.15.159:0/2166026925 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc738040070 con 0x7fc7500b7310 2024-04-04T06:15:21.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.488+0000 7fc754c00640 1 -- 172.21.15.159:0/2166026925 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7500b7a80 con 0x7fc7500b7310 2024-04-04T06:15:21.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.488+0000 7fc746000640 1 -- 172.21.15.159:0/2166026925 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc738007260 con 0x7fc7500b7310 2024-04-04T06:15:21.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.489+0000 7fc754c00640 1 -- 172.21.15.159:0/2166026925 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7500b8040 con 0x7fc7500b7310 2024-04-04T06:15:21.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.490+0000 7fc754c00640 1 -- 172.21.15.159:0/2166026925 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7500685c0 con 0x7fc7500b7310 2024-04-04T06:15:21.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.491+0000 7fc746000640 1 -- 172.21.15.159:0/2166026925 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc738007400 con 0x7fc7500b7310 2024-04-04T06:15:21.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.492+0000 7fc746000640 1 --2- 172.21.15.159:0/2166026925 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc72c06cb10 0x7fc72c06efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:21.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.492+0000 7fc746000640 1 -- 172.21.15.159:0/2166026925 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fc7380ad950 con 0x7fc7500b7310 2024-04-04T06:15:21.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.493+0000 7fc74de00640 1 --2- 172.21.15.159:0/2166026925 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc72c06cb10 0x7fc72c06efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:21.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.494+0000 7fc74de00640 1 --2- 172.21.15.159:0/2166026925 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc72c06cb10 0x7fc72c06efd0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc7400099b0 tx=0x7fc740008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:21.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.497+0000 7fc746000640 1 -- 172.21.15.159:0/2166026925 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc73807bd60 con 0x7fc7500b7310 2024-04-04T06:15:21.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.740+0000 7fc754c00640 1 -- 172.21.15.159:0/2166026925 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc7500d0460 con 0x7fc7500b7310 2024-04-04T06:15:21.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.741+0000 7fc746000640 1 -- 172.21.15.159:0/2166026925 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v84) ==== 53+0+73 (secure 0 0 0) 0x7fc73807b660 con 0x7fc7500b7310 2024-04-04T06:15:21.734 INFO:teuthology.orchestra.run.smithi159.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-04T06:15:21.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.743+0000 7fc754c00640 1 -- 172.21.15.159:0/2166026925 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc72c06cb10 msgr2=0x7fc72c06efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:21.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.743+0000 7fc754c00640 1 --2- 172.21.15.159:0/2166026925 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc72c06cb10 0x7fc72c06efd0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc7400099b0 tx=0x7fc740008040 comp rx=0 tx=0).stop 2024-04-04T06:15:21.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.743+0000 7fc754c00640 1 -- 172.21.15.159:0/2166026925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7500b7310 msgr2=0x7fc7500cd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:21.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.744+0000 7fc754c00640 1 --2- 172.21.15.159:0/2166026925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7500b7310 0x7fc7500cd9b0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fc738001a30 tx=0x7fc7380012b0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.744+0000 7fc754c00640 1 -- 172.21.15.159:0/2166026925 shutdown_connections 2024-04-04T06:15:21.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.744+0000 7fc754c00640 1 --2- 172.21.15.159:0/2166026925 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc72c06cb10 0x7fc72c06efd0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.744+0000 7fc754c00640 1 --2- 172.21.15.159:0/2166026925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc75015cc90 0x7fc750153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.744+0000 7fc754c00640 1 --2- 172.21.15.159:0/2166026925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7500b7310 0x7fc7500cd9b0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.744+0000 7fc754c00640 1 --2- 172.21.15.159:0/2166026925 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc734004340 0x7fc7500b8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:21.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.744+0000 7fc754c00640 1 -- 172.21.15.159:0/2166026925 >> 172.21.15.159:0/2166026925 conn(0x7fc75008bfb0 msgr2=0x7fc7500cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:21.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.744+0000 7fc754c00640 1 -- 172.21.15.159:0/2166026925 shutdown_connections 2024-04-04T06:15:21.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:21.744+0000 7fc754c00640 1 -- 172.21.15.159:0/2166026925 wait complete. 2024-04-04T06:15:21.763 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-04T06:15:21.764 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-04T06:15:21.764 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi159.front.sepia.ceph.com 2024-04-04T06:15:21.764 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-04T06:15:21.764 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-04T06:15:21.764 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-04T06:15:21.764 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-04T06:15:21.764 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-04T06:15:21.764 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:15:21.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:21.764 DEBUG:teuthology.orchestra.run.smithi159:> ip addr 2024-04-04T06:15:21.813 INFO:teuthology.orchestra.run.smithi159.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:15:21.814 INFO:teuthology.orchestra.run.smithi159.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:15:21.814 INFO:teuthology.orchestra.run.smithi159.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:15:21.814 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:15:21.814 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 ::1/128 scope host 2024-04-04T06:15:21.814 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:15:21.814 INFO:teuthology.orchestra.run.smithi159.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:15:21.814 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c6 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:15:21.814 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f0 2024-04-04T06:15:21.814 INFO:teuthology.orchestra.run.smithi159.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:15:21.814 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c7 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:15:21.814 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f1 2024-04-04T06:15:21.815 INFO:teuthology.orchestra.run.smithi159.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:15:21.815 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4e brd ff:ff:ff:ff:ff:ff 2024-04-04T06:15:21.815 INFO:teuthology.orchestra.run.smithi159.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-04T06:15:21.815 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:15:21.815 INFO:teuthology.orchestra.run.smithi159.stdout: inet 172.21.15.159/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-04T06:15:21.815 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft 42011sec preferred_lft 42011sec 2024-04-04T06:15:21.815 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::ec4:7aff:fe88:724f/64 scope link 2024-04-04T06:15:21.815 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:15:21.815 INFO:teuthology.orchestra.run.smithi159.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-04T06:15:21.815 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:15:21.815 INFO:teuthology.orchestra.run.smithi159.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-04T06:15:21.815 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:15:21.815 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::989e:e4ff:fea5:7568/64 scope link 2024-04-04T06:15:21.816 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:15:21.816 INFO:teuthology.orchestra.run.smithi159.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:15:21.816 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether a2:e3:e0:cb:0a:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-04T06:15:21.816 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::a0e3:e0ff:fecb:a73/64 scope link 2024-04-04T06:15:21.816 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:15:21.816 INFO:teuthology.orchestra.run.smithi159.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:15:21.816 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-04T06:15:21.816 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::c0a:8aff:fe54:b58f/64 scope link 2024-04-04T06:15:21.816 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:15:21.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:21.816 DEBUG:teuthology.orchestra.run.smithi159:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-04T06:15:21.943 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:15:21.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:21.947 DEBUG:teuthology.orchestra.run.smithi159:> route 2024-04-04T06:15:21.995 INFO:teuthology.orchestra.run.smithi159.stdout:Kernel IP routing table 2024-04-04T06:15:21.996 INFO:teuthology.orchestra.run.smithi159.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-04T06:15:21.996 INFO:teuthology.orchestra.run.smithi159.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-04T06:15:21.996 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-04T06:15:21.996 INFO:teuthology.orchestra.run.smithi159.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-04T06:15:21.996 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:15:21.996 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:15:21.996 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-04T06:15:21.996 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-04T06:15:21.997 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-04T06:15:21.997 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:15:22.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:15:22.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:15:22.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:22.237 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-04-04T06:15:22.282 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-04T06:15:22.283 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-04T06:15:22.284 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:15:22.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:22.284 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set brx.0 up 2024-04-04T06:15:22.420 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:15:22.420 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-04T06:15:22.470 INFO:teuthology.orchestra.run.smithi159.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:15:22.470 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:15:22.470 DEBUG:teuthology.orchestra.run.smithi159:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:15:22.520 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-04T06:15:22.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:22.521 DEBUG:teuthology.orchestra.run.smithi159:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:15:22.650 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:15:22.651 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:15:22.651 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:15:22.669 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:15:22.669 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:15:22.669 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:15:22.669 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:15:22.670 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:15:22.695 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:15:22.696 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:15:22.696 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:15:22.696 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:15:22.696 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:15:22.696 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-04T06:15:22.696 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-04T06:15:22.696 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30311045-f714-48fa-832b-4cb2ba8bf0bf" will pass to kernel 2024-04-04T06:15:22.698 INFO:teuthology.orchestra.run.smithi159.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-04T06:15:22.699 INFO:teuthology.orchestra.run.smithi159.stdout: You just mounted a file system that supports labels which does not 2024-04-04T06:15:22.699 INFO:teuthology.orchestra.run.smithi159.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-04T06:15:22.699 INFO:teuthology.orchestra.run.smithi159.stdout: applications will generate AVC messages and not be allowed access to 2024-04-04T06:15:22.699 INFO:teuthology.orchestra.run.smithi159.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-04T06:15:22.702 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-04T06:15:22.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:22.702 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:15:22.779 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:15:22.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:15:22.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:15:22.896 INFO:teuthology.orchestra.run.smithi159.stdout:client5594 2024-04-04T06:15:22.896 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:15:22.896 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:15:22.941 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:15:22.942 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:15:23.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:23.036 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-04T06:15:23.283 INFO:teuthology.orchestra.run.smithi159.stderr:6+0 records in 2024-04-04T06:15:23.283 INFO:teuthology.orchestra.run.smithi159.stderr:6+0 records out 2024-04-04T06:15:23.283 INFO:teuthology.orchestra.run.smithi159.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.15185 s, 41.4 MB/s 2024-04-04T06:15:23.285 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:15:23.285 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:15:23.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:23.329 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-04T06:15:23.554 INFO:teuthology.orchestra.run.smithi159.stderr:6+0 records in 2024-04-04T06:15:23.554 INFO:teuthology.orchestra.run.smithi159.stderr:6+0 records out 2024-04-04T06:15:23.555 INFO:teuthology.orchestra.run.smithi159.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.131938 s, 47.7 MB/s 2024-04-04T06:15:23.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:23.556 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-04T06:15:23.604 INFO:teuthology.orchestra.run.smithi159.stdout:./ 2024-04-04T06:15:23.604 INFO:teuthology.orchestra.run.smithi159.stdout:./file2_sixmegs 2024-04-04T06:15:23.604 INFO:teuthology.orchestra.run.smithi159.stdout:./file3_sixmegs 2024-04-04T06:15:23.604 INFO:teuthology.orchestra.run.smithi159.stdout:./file1_sixmegs 2024-04-04T06:15:23.605 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:15:23.605 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:15:23.605 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:15:23.605 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:15:23.605 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-04T06:15:23.605 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:15:23.724 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:15:23.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:15:23.737 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:15:23.738 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:15:23.738 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:15:23.738 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:15:23.738 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-04-04T06:15:23.738 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:15:23.940 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627777 2024-04-04T06:15:23.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:15:23.957 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:15:23.957 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:15:23.957 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:15:23.957 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:15:23.957 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-04-04T06:15:23.957 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:15:24.137 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627778 2024-04-04T06:15:24.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:15:24.151 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:15:24.151 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:15:24.151 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:15:24.151 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:15:24.151 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-04-04T06:15:24.151 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:15:24.329 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627776 2024-04-04T06:15:24.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:15:24.342 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:15:24.342 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:15:24.436 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:15:24.436 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:15:24.487 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:15:24.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.695+0000 7f1793000640 1 Processor -- start 2024-04-04T06:15:24.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.695+0000 7f1793000640 1 -- start start 2024-04-04T06:15:24.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.697+0000 7f1793000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f178c0d2310 0x7f178c0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:24.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.697+0000 7f1793000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f178c15e880 con 0x7f178c15e4a0 2024-04-04T06:15:24.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.697+0000 7f1793000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f178c0d28d0 con 0x7f178c0cf530 2024-04-04T06:15:24.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.697+0000 7f1793000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f178c0cf9a0 con 0x7f178c0d2310 2024-04-04T06:15:24.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.697+0000 7f1791200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f178c0d2310 0x7f178c0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:24.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.697+0000 7f1791200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f178c0d2310 0x7f178c0ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39424/0 (socket says 172.21.15.159:39424) 2024-04-04T06:15:24.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.697+0000 7f1791200640 1 -- 172.21.15.159:0/3279007158 learned_addr learned my addr 172.21.15.159:0/3279007158 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:24.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.697+0000 7f1791200640 1 -- 172.21.15.159:0/3279007158 >> v1:172.21.15.178:6789/0 conn(0x7f178c0cf530 legacy=0x7f178c0b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:24.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.697+0000 7f1791200640 1 -- 172.21.15.159:0/3279007158 >> v1:172.21.15.159:6789/0 conn(0x7f178c15e4a0 legacy=0x7f178c0b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:24.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.697+0000 7f1791200640 1 -- 172.21.15.159:0/3279007158 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178c0d2bb0 con 0x7f178c0d2310 2024-04-04T06:15:24.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.699+0000 7f1791200640 1 --2- 172.21.15.159:0/3279007158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f178c0d2310 0x7f178c0ceff0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f1778009870 tx=0x7f177802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b4ed980fe42f27 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:24.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.699+0000 7f178be00640 1 -- 172.21.15.159:0/3279007158 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778035020 con 0x7f178c0d2310 2024-04-04T06:15:24.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.699+0000 7f178be00640 1 -- 172.21.15.159:0/3279007158 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1778031870 con 0x7f178c0d2310 2024-04-04T06:15:24.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.699+0000 7f178be00640 1 -- 172.21.15.159:0/3279007158 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778031c40 con 0x7f178c0d2310 2024-04-04T06:15:24.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.700+0000 7f1793000640 1 -- 172.21.15.159:0/3279007158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f178c0d2310 msgr2=0x7f178c0ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:24.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.700+0000 7f1793000640 1 --2- 172.21.15.159:0/3279007158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f178c0d2310 0x7f178c0ceff0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f1778009870 tx=0x7f177802b910 comp rx=0 tx=0).stop 2024-04-04T06:15:24.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.700+0000 7f1793000640 1 -- 172.21.15.159:0/3279007158 shutdown_connections 2024-04-04T06:15:24.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.700+0000 7f1793000640 1 --2- 172.21.15.159:0/3279007158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f178c0d2310 0x7f178c0ceff0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:24.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.700+0000 7f1793000640 1 -- 172.21.15.159:0/3279007158 >> 172.21.15.159:0/3279007158 conn(0x7f178c0676d0 msgr2=0x7f178c067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:24.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.700+0000 7f1793000640 1 -- 172.21.15.159:0/3279007158 shutdown_connections 2024-04-04T06:15:24.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.700+0000 7f1793000640 1 -- 172.21.15.159:0/3279007158 wait complete. 2024-04-04T06:15:24.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.701+0000 7f1793000640 1 Processor -- start 2024-04-04T06:15:24.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.702+0000 7f1793000640 1 -- start start 2024-04-04T06:15:24.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.703+0000 7f1793000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f178c0cf530 0x7f178c159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:24.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.703+0000 7f1793000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f178c0d2310 0x7f178c1f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:24.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.703+0000 7f1793000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f178c15e4a0 0x7f178c0ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:24.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.703+0000 7f1793000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f178c0d3170 con 0x7f178c15e4a0 2024-04-04T06:15:24.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.703+0000 7f1793000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f178c0d34e0 con 0x7f178c0d2310 2024-04-04T06:15:24.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.703+0000 7f1793000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f178c0d3970 con 0x7f178c0cf530 2024-04-04T06:15:24.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.703+0000 7f1792600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f178c15e4a0 0x7f178c0ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:24.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.703+0000 7f1792600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f178c15e4a0 0x7f178c0ccf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39440/0 (socket says 172.21.15.159:39440) 2024-04-04T06:15:24.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.703+0000 7f1792600640 1 -- 172.21.15.159:0/2776734185 learned_addr learned my addr 172.21.15.159:0/2776734185 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:24.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.703+0000 7f1791200640 1 --2- 172.21.15.159:0/2776734185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f178c0d2310 0x7f178c1f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:24.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.703+0000 7f1791c00640 1 --2- 172.21.15.159:0/2776734185 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f178c0cf530 0x7f178c159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:24.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.703+0000 7f1792600640 1 -- 172.21.15.159:0/2776734185 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f178c0cf530 msgr2=0x7f178c159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:24.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.703+0000 7f1792600640 1 --2- 172.21.15.159:0/2776734185 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f178c0cf530 0x7f178c159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:24.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.703+0000 7f1792600640 1 -- 172.21.15.159:0/2776734185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f178c0d2310 msgr2=0x7f178c1f4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:24.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.703+0000 7f1792600640 1 --2- 172.21.15.159:0/2776734185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f178c0d2310 0x7f178c1f4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:24.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.704+0000 7f1792600640 1 -- 172.21.15.159:0/2776734185 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178c15ff70 con 0x7f178c15e4a0 2024-04-04T06:15:24.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.704+0000 7f1792600640 1 --2- 172.21.15.159:0/2776734185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f178c15e4a0 0x7f178c0ccf20 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f178400c780 tx=0x7f178400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:24.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.704+0000 7f1789600640 1 -- 172.21.15.159:0/2776734185 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1784017070 con 0x7f178c15e4a0 2024-04-04T06:15:24.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.704+0000 7f1789600640 1 -- 172.21.15.159:0/2776734185 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1784003da0 con 0x7f178c15e4a0 2024-04-04T06:15:24.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.704+0000 7f1789600640 1 -- 172.21.15.159:0/2776734185 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17840052b0 con 0x7f178c15e4a0 2024-04-04T06:15:24.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.704+0000 7f1793000640 1 -- 172.21.15.159:0/2776734185 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f178c160250 con 0x7f178c15e4a0 2024-04-04T06:15:24.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.704+0000 7f1793000640 1 -- 172.21.15.159:0/2776734185 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f178c160810 con 0x7f178c15e4a0 2024-04-04T06:15:24.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.706+0000 7f1793000640 1 -- 172.21.15.159:0/2776734185 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f178c075940 con 0x7f178c15e4a0 2024-04-04T06:15:24.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.707+0000 7f1789600640 1 -- 172.21.15.159:0/2776734185 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f178401a070 con 0x7f178c15e4a0 2024-04-04T06:15:24.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.707+0000 7f1789600640 1 --2- 172.21.15.159:0/2776734185 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f176c06c8f0 0x7f176c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:24.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.708+0000 7f1789600640 1 -- 172.21.15.159:0/2776734185 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f178408b1d0 con 0x7f178c15e4a0 2024-04-04T06:15:24.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.708+0000 7f1791c00640 1 --2- 172.21.15.159:0/2776734185 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f176c06c8f0 0x7f176c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:24.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.709+0000 7f1791c00640 1 --2- 172.21.15.159:0/2776734185 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f176c06c8f0 0x7f176c06edb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1780007fd0 tx=0x7f1780009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:24.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.712+0000 7f1789600640 1 -- 172.21.15.159:0/2776734185 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f17840595f0 con 0x7f178c15e4a0 2024-04-04T06:15:24.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.992+0000 7f1793000640 1 -- 172.21.15.159:0/2776734185 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f178c06b570 con 0x7f178c15e4a0 2024-04-04T06:15:24.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.995+0000 7f1789600640 1 -- 172.21.15.159:0/2776734185 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16623 (secure 0 0 0) 0x7f1784058ef0 con 0x7f178c15e4a0 2024-04-04T06:15:24.988 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:15:24.989 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":101,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:15:09.233029+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:15:05.537357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-04T06:15:06.689304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:15:24.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.998+0000 7f1793000640 1 -- 172.21.15.159:0/2776734185 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f176c06c8f0 msgr2=0x7f176c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:24.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.998+0000 7f1793000640 1 --2- 172.21.15.159:0/2776734185 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f176c06c8f0 0x7f176c06edb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1780007fd0 tx=0x7f1780009040 comp rx=0 tx=0).stop 2024-04-04T06:15:24.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.998+0000 7f1793000640 1 -- 172.21.15.159:0/2776734185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f178c15e4a0 msgr2=0x7f178c0ccf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:24.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.998+0000 7f1793000640 1 --2- 172.21.15.159:0/2776734185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f178c15e4a0 0x7f178c0ccf20 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f178400c780 tx=0x7f178400cc40 comp rx=0 tx=0).stop 2024-04-04T06:15:24.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.998+0000 7f1793000640 1 -- 172.21.15.159:0/2776734185 shutdown_connections 2024-04-04T06:15:24.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.998+0000 7f1793000640 1 --2- 172.21.15.159:0/2776734185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f178c15e4a0 0x7f178c0ccf20 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:24.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.998+0000 7f1793000640 1 --2- 172.21.15.159:0/2776734185 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f176c06c8f0 0x7f176c06edb0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:24.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.998+0000 7f1793000640 1 --2- 172.21.15.159:0/2776734185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f178c0d2310 0x7f178c1f4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:24.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.998+0000 7f1793000640 1 --2- 172.21.15.159:0/2776734185 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f178c0cf530 0x7f178c159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:24.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.998+0000 7f1793000640 1 -- 172.21.15.159:0/2776734185 >> 172.21.15.159:0/2776734185 conn(0x7f178c0676d0 msgr2=0x7f178c067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:24.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.998+0000 7f1793000640 1 -- 172.21.15.159:0/2776734185 shutdown_connections 2024-04-04T06:15:24.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:24.998+0000 7f1793000640 1 -- 172.21.15.159:0/2776734185 wait complete. 2024-04-04T06:15:25.011 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:15:05.537357+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:15:06.689304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:15:25.012 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:15:25.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.228+0000 7ff2a6400640 1 Processor -- start 2024-04-04T06:15:25.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.229+0000 7ff2a6400640 1 -- start start 2024-04-04T06:15:25.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.230+0000 7ff2a6400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff2a00d1bf0 0x7ff2a015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:25.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.230+0000 7ff2a6400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2a0070170 0x7ff2a006b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:25.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.231+0000 7ff2a6400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2a015edc0 con 0x7ff2a00705f0 2024-04-04T06:15:25.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.231+0000 7ff2a6400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7ff2a00d1fd0 con 0x7ff2a00d1bf0 2024-04-04T06:15:25.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.231+0000 7ff2a6400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a015eb80 con 0x7ff2a0070170 2024-04-04T06:15:25.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.231+0000 7ff29f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff2a00d1bf0 0x7ff2a015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:25.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.231+0000 7ff29f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff2a00d1bf0 0x7ff2a015f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39452/0 (socket says 172.21.15.159:39452) 2024-04-04T06:15:25.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.231+0000 7ff29f400640 1 -- 172.21.15.159:0/708349607 learned_addr learned my addr 172.21.15.159:0/708349607 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:25.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.231+0000 7ff29ea00640 1 --2- 172.21.15.159:0/708349607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2a0070170 0x7ff2a006b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:25.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.231+0000 7ff29f400640 1 -- 172.21.15.159:0/708349607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2a0070170 msgr2=0x7ff2a006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:25.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.231+0000 7ff29f400640 1 --2- 172.21.15.159:0/708349607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2a0070170 0x7ff2a006b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:25.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.231+0000 7ff29f400640 1 -- 172.21.15.159:0/708349607 >> v1:172.21.15.178:6789/0 conn(0x7ff2a00705f0 legacy=0x7ff2a006e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:25.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.231+0000 7ff29f400640 1 -- 172.21.15.159:0/708349607 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a00b5e60 con 0x7ff2a00d1bf0 2024-04-04T06:15:25.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.233+0000 7ff29f400640 1 --2- 172.21.15.159:0/708349607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2a00d1bf0 0x7ff2a015f5a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7ff290009e50 tx=0x7ff29002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268592287c67a405 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:25.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.233+0000 7ff29e000640 1 -- 172.21.15.159:0/708349607 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29002ca80 con 0x7ff2a00d1bf0 2024-04-04T06:15:25.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.233+0000 7ff29e000640 1 -- 172.21.15.159:0/708349607 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff29002cc20 con 0x7ff2a00d1bf0 2024-04-04T06:15:25.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.233+0000 7ff29e000640 1 -- 172.21.15.159:0/708349607 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2900343f0 con 0x7ff2a00d1bf0 2024-04-04T06:15:25.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.233+0000 7ff2a6400640 1 -- 172.21.15.159:0/708349607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2a00d1bf0 msgr2=0x7ff2a015f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:25.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.233+0000 7ff2a6400640 1 --2- 172.21.15.159:0/708349607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2a00d1bf0 0x7ff2a015f5a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7ff290009e50 tx=0x7ff29002bee0 comp rx=0 tx=0).stop 2024-04-04T06:15:25.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.234+0000 7ff2a6400640 1 -- 172.21.15.159:0/708349607 shutdown_connections 2024-04-04T06:15:25.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.234+0000 7ff2a6400640 1 --2- 172.21.15.159:0/708349607 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2a0070170 0x7ff2a006b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:25.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.234+0000 7ff2a6400640 1 --2- 172.21.15.159:0/708349607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2a00d1bf0 0x7ff2a015f5a0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:25.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.234+0000 7ff2a6400640 1 -- 172.21.15.159:0/708349607 >> 172.21.15.159:0/708349607 conn(0x7ff2a00b6560 msgr2=0x7ff2a00b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:25.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.234+0000 7ff2a6400640 1 -- 172.21.15.159:0/708349607 shutdown_connections 2024-04-04T06:15:25.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.234+0000 7ff2a6400640 1 -- 172.21.15.159:0/708349607 wait complete. 2024-04-04T06:15:25.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.235+0000 7ff2a6400640 1 Processor -- start 2024-04-04T06:15:25.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.236+0000 7ff2a6400640 1 -- start start 2024-04-04T06:15:25.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.236+0000 7ff2a6400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2a0070170 0x7ff2a0158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:25.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.236+0000 7ff2a6400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2a00705f0 0x7ff2a0158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:25.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.236+0000 7ff2a6400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2a00d1bf0 0x7ff2a0159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:25.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.237+0000 7ff2a6400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a015fae0 con 0x7ff2a00d1bf0 2024-04-04T06:15:25.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.237+0000 7ff2a6400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a015f080 con 0x7ff2a00705f0 2024-04-04T06:15:25.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.237+0000 7ff2a6400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a00b5b10 con 0x7ff2a0070170 2024-04-04T06:15:25.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.237+0000 7ff29fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2a00d1bf0 0x7ff2a0159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:25.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.237+0000 7ff29fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2a00d1bf0 0x7ff2a0159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39466/0 (socket says 172.21.15.159:39466) 2024-04-04T06:15:25.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.237+0000 7ff29fe00640 1 -- 172.21.15.159:0/3677139730 learned_addr learned my addr 172.21.15.159:0/3677139730 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:25.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.237+0000 7ff29ea00640 1 --2- 172.21.15.159:0/3677139730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2a00705f0 0x7ff2a0158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:25.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.237+0000 7ff29f400640 1 --2- 172.21.15.159:0/3677139730 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2a0070170 0x7ff2a0158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:25.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.237+0000 7ff29fe00640 1 -- 172.21.15.159:0/3677139730 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2a0070170 msgr2=0x7ff2a0158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:25.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.237+0000 7ff29fe00640 1 --2- 172.21.15.159:0/3677139730 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2a0070170 0x7ff2a0158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:25.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.237+0000 7ff29fe00640 1 -- 172.21.15.159:0/3677139730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2a00705f0 msgr2=0x7ff2a0158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:25.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.237+0000 7ff29fe00640 1 --2- 172.21.15.159:0/3677139730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2a00705f0 0x7ff2a0158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:25.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.237+0000 7ff29fe00640 1 -- 172.21.15.159:0/3677139730 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a015abe0 con 0x7ff2a00d1bf0 2024-04-04T06:15:25.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.238+0000 7ff29fe00640 1 --2- 172.21.15.159:0/3677139730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2a00d1bf0 0x7ff2a0159000 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7ff29800c440 tx=0x7ff29800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:25.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.238+0000 7ff297400640 1 -- 172.21.15.159:0/3677139730 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298011790 con 0x7ff2a00d1bf0 2024-04-04T06:15:25.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.238+0000 7ff297400640 1 -- 172.21.15.159:0/3677139730 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff298017070 con 0x7ff2a00d1bf0 2024-04-04T06:15:25.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.238+0000 7ff2a6400640 1 -- 172.21.15.159:0/3677139730 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a015aec0 con 0x7ff2a00d1bf0 2024-04-04T06:15:25.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.238+0000 7ff297400640 1 -- 172.21.15.159:0/3677139730 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298011b70 con 0x7ff2a00d1bf0 2024-04-04T06:15:25.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.238+0000 7ff2a6400640 1 -- 172.21.15.159:0/3677139730 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a01f6590 con 0x7ff2a00d1bf0 2024-04-04T06:15:25.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.240+0000 7ff2a6400640 1 -- 172.21.15.159:0/3677139730 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a00685c0 con 0x7ff2a00d1bf0 2024-04-04T06:15:25.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.240+0000 7ff297400640 1 -- 172.21.15.159:0/3677139730 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff298011d10 con 0x7ff2a00d1bf0 2024-04-04T06:15:25.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.242+0000 7ff297400640 1 --2- 172.21.15.159:0/3677139730 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff28006c8f0 0x7ff28006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:25.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.242+0000 7ff297400640 1 -- 172.21.15.159:0/3677139730 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7ff29808b8e0 con 0x7ff2a00d1bf0 2024-04-04T06:15:25.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.242+0000 7ff29f400640 1 --2- 172.21.15.159:0/3677139730 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff28006c8f0 0x7ff28006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:25.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.243+0000 7ff29f400640 1 --2- 172.21.15.159:0/3677139730 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff28006c8f0 0x7ff28006edb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff290035000 tx=0x7ff290030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:25.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.247+0000 7ff297400640 1 -- 172.21.15.159:0/3677139730 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff298059dc0 con 0x7ff2a00d1bf0 2024-04-04T06:15:25.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.513+0000 7ff2a6400640 1 -- 172.21.15.159:0/3677139730 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff2a00afad0 con 0x7ff2a00d1bf0 2024-04-04T06:15:25.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.514+0000 7ff297400640 1 -- 172.21.15.159:0/3677139730 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7ff2980596c0 con 0x7ff2a00d1bf0 2024-04-04T06:15:25.508 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:15:25.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.516+0000 7ff2a6400640 1 -- 172.21.15.159:0/3677139730 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff28006c8f0 msgr2=0x7ff28006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:25.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.516+0000 7ff2a6400640 1 --2- 172.21.15.159:0/3677139730 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff28006c8f0 0x7ff28006edb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff290035000 tx=0x7ff290030000 comp rx=0 tx=0).stop 2024-04-04T06:15:25.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.516+0000 7ff2a6400640 1 -- 172.21.15.159:0/3677139730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2a00d1bf0 msgr2=0x7ff2a0159000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:25.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.516+0000 7ff2a6400640 1 --2- 172.21.15.159:0/3677139730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2a00d1bf0 0x7ff2a0159000 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7ff29800c440 tx=0x7ff29800c840 comp rx=0 tx=0).stop 2024-04-04T06:15:25.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.517+0000 7ff2a6400640 1 -- 172.21.15.159:0/3677139730 shutdown_connections 2024-04-04T06:15:25.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.517+0000 7ff2a6400640 1 --2- 172.21.15.159:0/3677139730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2a00d1bf0 0x7ff2a0159000 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:25.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.517+0000 7ff2a6400640 1 --2- 172.21.15.159:0/3677139730 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff28006c8f0 0x7ff28006edb0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:25.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.517+0000 7ff2a6400640 1 --2- 172.21.15.159:0/3677139730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2a00705f0 0x7ff2a0158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:25.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.517+0000 7ff2a6400640 1 --2- 172.21.15.159:0/3677139730 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2a0070170 0x7ff2a0158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:25.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.517+0000 7ff2a6400640 1 -- 172.21.15.159:0/3677139730 >> 172.21.15.159:0/3677139730 conn(0x7ff2a00b6560 msgr2=0x7ff2a00afcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:25.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.517+0000 7ff2a6400640 1 -- 172.21.15.159:0/3677139730 shutdown_connections 2024-04-04T06:15:25.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.517+0000 7ff2a6400640 1 -- 172.21.15.159:0/3677139730 wait complete. 2024-04-04T06:15:25.529 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:15:25.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.741+0000 7ff9bc000640 1 Processor -- start 2024-04-04T06:15:25.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.741+0000 7ff9bc000640 1 -- start start 2024-04-04T06:15:25.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.742+0000 7ff9bc000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff9b4153580 0x7ff9b40d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:25.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.742+0000 7ff9bc000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9b40d07c0 0x7ff9b40aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:25.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.742+0000 7ff9bc000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9b40afab0 con 0x7ff9b40af2e0 2024-04-04T06:15:25.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.742+0000 7ff9bc000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7ff9b40d0c00 con 0x7ff9b4153580 2024-04-04T06:15:25.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.742+0000 7ff9bc000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b415cc80 con 0x7ff9b40d07c0 2024-04-04T06:15:25.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.743+0000 7ff9b9600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff9b4153580 0x7ff9b40d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:25.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.743+0000 7ff9b9600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff9b4153580 0x7ff9b40d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39474/0 (socket says 172.21.15.159:39474) 2024-04-04T06:15:25.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.743+0000 7ff9b9600640 1 -- 172.21.15.159:0/2048348565 learned_addr learned my addr 172.21.15.159:0/2048348565 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:25.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.743+0000 7ff9b8c00640 1 --2- 172.21.15.159:0/2048348565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9b40d07c0 0x7ff9b40aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:25.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.743+0000 7ff9b9600640 1 -- 172.21.15.159:0/2048348565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9b40d07c0 msgr2=0x7ff9b40aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:25.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.743+0000 7ff9b9600640 1 --2- 172.21.15.159:0/2048348565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9b40d07c0 0x7ff9b40aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:25.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.743+0000 7ff9b9600640 1 -- 172.21.15.159:0/2048348565 >> v1:172.21.15.178:6789/0 conn(0x7ff9b40af2e0 legacy=0x7ff9b406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:25.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.743+0000 7ff9b9600640 1 -- 172.21.15.159:0/2048348565 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b415da00 con 0x7ff9b4153580 2024-04-04T06:15:25.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.745+0000 7ff9b9600640 1 --2- 172.21.15.159:0/2048348565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b4153580 0x7ff9b40d0280 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009870 tx=0x7ff9a802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40eb93d050bd7350 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:25.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.745+0000 7ff9b3e00640 1 -- 172.21.15.159:0/2048348565 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8035020 con 0x7ff9b4153580 2024-04-04T06:15:25.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.745+0000 7ff9b3e00640 1 -- 172.21.15.159:0/2048348565 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9a8031640 con 0x7ff9b4153580 2024-04-04T06:15:25.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.745+0000 7ff9b3e00640 1 -- 172.21.15.159:0/2048348565 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a802d040 con 0x7ff9b4153580 2024-04-04T06:15:25.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.746+0000 7ff9bc000640 1 -- 172.21.15.159:0/2048348565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b4153580 msgr2=0x7ff9b40d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:25.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.746+0000 7ff9bc000640 1 --2- 172.21.15.159:0/2048348565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b4153580 0x7ff9b40d0280 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009870 tx=0x7ff9a802b8f0 comp rx=0 tx=0).stop 2024-04-04T06:15:25.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.746+0000 7ff9bc000640 1 -- 172.21.15.159:0/2048348565 shutdown_connections 2024-04-04T06:15:25.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.746+0000 7ff9bc000640 1 --2- 172.21.15.159:0/2048348565 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9b40d07c0 0x7ff9b40aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:25.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.746+0000 7ff9bc000640 1 --2- 172.21.15.159:0/2048348565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b4153580 0x7ff9b40d0280 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:25.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.746+0000 7ff9bc000640 1 -- 172.21.15.159:0/2048348565 >> 172.21.15.159:0/2048348565 conn(0x7ff9b406b750 msgr2=0x7ff9b406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:25.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.746+0000 7ff9bc000640 1 -- 172.21.15.159:0/2048348565 shutdown_connections 2024-04-04T06:15:25.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.746+0000 7ff9bc000640 1 -- 172.21.15.159:0/2048348565 wait complete. 2024-04-04T06:15:25.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.749+0000 7ff9bc000640 1 Processor -- start 2024-04-04T06:15:25.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.749+0000 7ff9bc000640 1 -- start start 2024-04-04T06:15:25.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.751+0000 7ff9bc000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9b40af2e0 0x7ff9b40cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:25.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.751+0000 7ff9bc000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9b40d07c0 0x7ff9b415ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:25.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.751+0000 7ff9bc000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b4153580 0x7ff9b415f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:25.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.751+0000 7ff9bc000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b415d4e0 con 0x7ff9b4153580 2024-04-04T06:15:25.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.751+0000 7ff9bc000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b415d200 con 0x7ff9b40d07c0 2024-04-04T06:15:25.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.751+0000 7ff9bc000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b4153960 con 0x7ff9b40af2e0 2024-04-04T06:15:25.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.751+0000 7ff9ba000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b4153580 0x7ff9b415f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:25.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.751+0000 7ff9ba000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b4153580 0x7ff9b415f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39482/0 (socket says 172.21.15.159:39482) 2024-04-04T06:15:25.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.751+0000 7ff9ba000640 1 -- 172.21.15.159:0/1865101882 learned_addr learned my addr 172.21.15.159:0/1865101882 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:25.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.751+0000 7ff9b9600640 1 --2- 172.21.15.159:0/1865101882 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9b40af2e0 0x7ff9b40cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:25.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.751+0000 7ff9b8c00640 1 --2- 172.21.15.159:0/1865101882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9b40d07c0 0x7ff9b415ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:25.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.752+0000 7ff9ba000640 1 -- 172.21.15.159:0/1865101882 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9b40af2e0 msgr2=0x7ff9b40cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:25.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.752+0000 7ff9ba000640 1 --2- 172.21.15.159:0/1865101882 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9b40af2e0 0x7ff9b40cf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:25.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.752+0000 7ff9ba000640 1 -- 172.21.15.159:0/1865101882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9b40d07c0 msgr2=0x7ff9b415ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:25.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.752+0000 7ff9ba000640 1 --2- 172.21.15.159:0/1865101882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9b40d07c0 0x7ff9b415ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:25.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.752+0000 7ff9ba000640 1 -- 172.21.15.159:0/1865101882 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b40cfa30 con 0x7ff9b4153580 2024-04-04T06:15:25.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.752+0000 7ff9ba000640 1 --2- 172.21.15.159:0/1865101882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b4153580 0x7ff9b415f290 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00c6b0 tx=0x7ff9ac00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:25.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.752+0000 7ff9b1600640 1 -- 172.21.15.159:0/1865101882 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac003b70 con 0x7ff9b4153580 2024-04-04T06:15:25.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.752+0000 7ff9b1600640 1 -- 172.21.15.159:0/1865101882 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9ac003d30 con 0x7ff9b4153580 2024-04-04T06:15:25.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.752+0000 7ff9bc000640 1 -- 172.21.15.159:0/1865101882 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b414cde0 con 0x7ff9b4153580 2024-04-04T06:15:25.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.752+0000 7ff9b1600640 1 -- 172.21.15.159:0/1865101882 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac005310 con 0x7ff9b4153580 2024-04-04T06:15:25.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.752+0000 7ff9bc000640 1 -- 172.21.15.159:0/1865101882 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b414d3a0 con 0x7ff9b4153580 2024-04-04T06:15:25.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.753+0000 7ff9bc000640 1 -- 172.21.15.159:0/1865101882 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b40685c0 con 0x7ff9b4153580 2024-04-04T06:15:25.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.755+0000 7ff9b1600640 1 -- 172.21.15.159:0/1865101882 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9ac0054b0 con 0x7ff9b4153580 2024-04-04T06:15:25.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.755+0000 7ff9b1600640 1 --2- 172.21.15.159:0/1865101882 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff99406c8f0 0x7ff99406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:25.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.756+0000 7ff9b1600640 1 -- 172.21.15.159:0/1865101882 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7ff9ac08b920 con 0x7ff9b4153580 2024-04-04T06:15:25.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.756+0000 7ff9b9600640 1 --2- 172.21.15.159:0/1865101882 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff99406c8f0 0x7ff99406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:25.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.757+0000 7ff9b9600640 1 --2- 172.21.15.159:0/1865101882 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff99406c8f0 0x7ff99406edb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8036000 tx=0x7ff9a803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:25.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.760+0000 7ff9b1600640 1 -- 172.21.15.159:0/1865101882 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff9ac059de0 con 0x7ff9b4153580 2024-04-04T06:15:25.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.961+0000 7ff9bc000640 1 -- 172.21.15.159:0/1865101882 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff9b406eca0 con 0x7ff9b4153580 2024-04-04T06:15:25.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.962+0000 7ff9b1600640 1 -- 172.21.15.159:0/1865101882 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7ff9ac0596e0 con 0x7ff9b4153580 2024-04-04T06:15:25.955 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:15:25.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.964+0000 7ff9bc000640 1 -- 172.21.15.159:0/1865101882 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff99406c8f0 msgr2=0x7ff99406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:25.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.964+0000 7ff9bc000640 1 --2- 172.21.15.159:0/1865101882 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff99406c8f0 0x7ff99406edb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8036000 tx=0x7ff9a803c040 comp rx=0 tx=0).stop 2024-04-04T06:15:25.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.964+0000 7ff9bc000640 1 -- 172.21.15.159:0/1865101882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b4153580 msgr2=0x7ff9b415f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:25.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.964+0000 7ff9bc000640 1 --2- 172.21.15.159:0/1865101882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b4153580 0x7ff9b415f290 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00c6b0 tx=0x7ff9ac00cb70 comp rx=0 tx=0).stop 2024-04-04T06:15:25.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.964+0000 7ff9bc000640 1 -- 172.21.15.159:0/1865101882 shutdown_connections 2024-04-04T06:15:25.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.964+0000 7ff9bc000640 1 --2- 172.21.15.159:0/1865101882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b4153580 0x7ff9b415f290 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:25.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.964+0000 7ff9bc000640 1 --2- 172.21.15.159:0/1865101882 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff99406c8f0 0x7ff99406edb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:25.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.964+0000 7ff9bc000640 1 --2- 172.21.15.159:0/1865101882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9b40d07c0 0x7ff9b415ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:25.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.964+0000 7ff9bc000640 1 --2- 172.21.15.159:0/1865101882 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff9b40af2e0 0x7ff9b40cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:25.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.964+0000 7ff9bc000640 1 -- 172.21.15.159:0/1865101882 >> 172.21.15.159:0/1865101882 conn(0x7ff9b406b750 msgr2=0x7ff9b415a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:25.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.964+0000 7ff9bc000640 1 -- 172.21.15.159:0/1865101882 shutdown_connections 2024-04-04T06:15:25.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:25.965+0000 7ff9bc000640 1 -- 172.21.15.159:0/1865101882 wait complete. 2024-04-04T06:15:25.978 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:15:26.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.194+0000 7fe063000640 1 Processor -- start 2024-04-04T06:15:26.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.194+0000 7fe063000640 1 -- start start 2024-04-04T06:15:26.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.196+0000 7fe063000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe05c155930 0x7fe05c0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:26.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.196+0000 7fe063000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe05c0d07c0 0x7fe05c0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:26.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.196+0000 7fe063000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe05c0b40b0 con 0x7fe05c155550 2024-04-04T06:15:26.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.196+0000 7fe063000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe05c0d0c30 con 0x7fe05c155930 2024-04-04T06:15:26.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.196+0000 7fe063000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe05c0b4390 con 0x7fe05c0d07c0 2024-04-04T06:15:26.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.196+0000 7fe05be00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fe05c155550 0x7fe05c15f800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:53532/0 (socket says 172.21.15.159:53532) 2024-04-04T06:15:26.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.196+0000 7fe05be00640 1 -- 172.21.15.159:0/401553277 learned_addr learned my addr 172.21.15.159:0/401553277 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:26.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.197+0000 7fe053400640 1 --2- 172.21.15.159:0/401553277 >> v2:172.21.15.178:3300/0 conn(0x7fe05c155930 0x7fe05c0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:26.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.197+0000 7fe061000640 1 --2- 172.21.15.159:0/401553277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe05c0d07c0 0x7fe05c0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:26.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.197+0000 7fe05b400640 1 -- 172.21.15.159:0/401553277 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2025996691 0 0) 0x7fe05c0b40b0 con 0x7fe05c155550 2024-04-04T06:15:26.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.197+0000 7fe05b400640 1 -- 172.21.15.159:0/401553277 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe040003520 con 0x7fe05c155550 2024-04-04T06:15:26.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.197+0000 7fe05b400640 1 -- 172.21.15.159:0/401553277 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 886280727 0 0) 0x7fe040003520 con 0x7fe05c155550 2024-04-04T06:15:26.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.197+0000 7fe05b400640 1 -- 172.21.15.159:0/401553277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe05c0d07c0 msgr2=0x7fe05c0aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:26.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.197+0000 7fe05b400640 1 --2- 172.21.15.159:0/401553277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe05c0d07c0 0x7fe05c0aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:26.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.197+0000 7fe05b400640 1 -- 172.21.15.159:0/401553277 >> v2:172.21.15.178:3300/0 conn(0x7fe05c155930 msgr2=0x7fe05c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:26.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.197+0000 7fe05b400640 1 --2- 172.21.15.159:0/401553277 >> v2:172.21.15.178:3300/0 conn(0x7fe05c155930 0x7fe05c0d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:26.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.197+0000 7fe05b400640 1 -- 172.21.15.159:0/401553277 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05c0b4670 con 0x7fe05c155550 2024-04-04T06:15:26.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.197+0000 7fe05b400640 1 -- 172.21.15.159:0/401553277 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fe04c0019f0 con 0x7fe05c155550 2024-04-04T06:15:26.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.197+0000 7fe05b400640 1 -- 172.21.15.159:0/401553277 >> v1:172.21.15.159:6789/0 conn(0x7fe05c155550 legacy=0x7fe05c15f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:26.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.197+0000 7fe05b400640 1 --2- 172.21.15.159:0/401553277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0400039b0 0x7fe040003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:26.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.197+0000 7fe05b400640 1 --2- 172.21.15.159:0/401553277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe05c0d07c0 0x7fe040004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:26.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.197+0000 7fe05b400640 1 --2- 172.21.15.159:0/401553277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe040004d40 0x7fe0400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:26.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.197+0000 7fe05b400640 1 -- 172.21.15.159:0/401553277 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c0d0c30 con 0x7fe0400039b0 2024-04-04T06:15:26.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.198+0000 7fe05b400640 1 -- 172.21.15.159:0/401553277 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c0b4390 con 0x7fe040004d40 2024-04-04T06:15:26.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.198+0000 7fe05b400640 1 -- 172.21.15.159:0/401553277 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe04000b9c0 con 0x7fe05c0d07c0 2024-04-04T06:15:26.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.198+0000 7fe05be00640 1 --2- 172.21.15.159:0/401553277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0400039b0 0x7fe040003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:26.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.198+0000 7fe053400640 1 --2- 172.21.15.159:0/401553277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe05c0d07c0 0x7fe040004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:26.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.198+0000 7fe061000640 1 --2- 172.21.15.159:0/401553277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe040004d40 0x7fe0400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:26.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.198+0000 7fe05be00640 1 -- 172.21.15.159:0/401553277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe05c0d07c0 msgr2=0x7fe040004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:26.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.198+0000 7fe05be00640 1 --2- 172.21.15.159:0/401553277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe05c0d07c0 0x7fe040004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:26.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.198+0000 7fe05be00640 1 -- 172.21.15.159:0/401553277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe040004d40 msgr2=0x7fe0400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:26.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.198+0000 7fe05be00640 1 --2- 172.21.15.159:0/401553277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe040004d40 0x7fe0400051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:26.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.198+0000 7fe05be00640 1 -- 172.21.15.159:0/401553277 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe040003520 con 0x7fe0400039b0 2024-04-04T06:15:26.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.200+0000 7fe05be00640 1 --2- 172.21.15.159:0/401553277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0400039b0 0x7fe040003dd0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00a880 tx=0x7fe040005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4cf8cda3915f171 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:26.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.200+0000 7fe05b400640 1 -- 172.21.15.159:0/401553277 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c00a4a0 con 0x7fe0400039b0 2024-04-04T06:15:26.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.200+0000 7fe05b400640 1 -- 172.21.15.159:0/401553277 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe04c034650 con 0x7fe0400039b0 2024-04-04T06:15:26.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.200+0000 7fe05b400640 1 -- 172.21.15.159:0/401553277 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c034a40 con 0x7fe0400039b0 2024-04-04T06:15:26.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.201+0000 7fe063000640 1 -- 172.21.15.159:0/401553277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0400039b0 msgr2=0x7fe040003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:26.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.201+0000 7fe063000640 1 --2- 172.21.15.159:0/401553277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0400039b0 0x7fe040003dd0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00a880 tx=0x7fe040005830 comp rx=0 tx=0).stop 2024-04-04T06:15:26.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.201+0000 7fe061000640 1 -- 172.21.15.159:0/401553277 reap_dead start 2024-04-04T06:15:26.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.201+0000 7fe063000640 1 -- 172.21.15.159:0/401553277 shutdown_connections 2024-04-04T06:15:26.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.201+0000 7fe063000640 1 -- 172.21.15.159:0/401553277 >> 172.21.15.159:0/401553277 conn(0x7fe05c06b750 msgr2=0x7fe05c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:26.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.201+0000 7fe063000640 1 -- 172.21.15.159:0/401553277 shutdown_connections 2024-04-04T06:15:26.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.201+0000 7fe063000640 1 -- 172.21.15.159:0/401553277 wait complete. 2024-04-04T06:15:26.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.203+0000 7fe063000640 1 Processor -- start 2024-04-04T06:15:26.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.203+0000 7fe063000640 1 -- start start 2024-04-04T06:15:26.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.204+0000 7fe063000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0400039b0 0x7fe05c145010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:26.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.204+0000 7fe063000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe05c1f8bd0 0x7fe05c0ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:26.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.204+0000 7fe063000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe05c14cc80 0x7fe05c1f7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:26.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.204+0000 7fe063000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c15ba20 con 0x7fe05c1f8bd0 2024-04-04T06:15:26.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.204+0000 7fe063000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c0b4c30 con 0x7fe0400039b0 2024-04-04T06:15:26.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.204+0000 7fe063000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe05c15bba0 con 0x7fe05c14cc80 2024-04-04T06:15:26.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.204+0000 7fe053400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe05c1f8bd0 0x7fe05c0ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:26.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.204+0000 7fe053400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe05c1f8bd0 0x7fe05c0ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39494/0 (socket says 172.21.15.159:39494) 2024-04-04T06:15:26.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.204+0000 7fe053400640 1 -- 172.21.15.159:0/853103698 learned_addr learned my addr 172.21.15.159:0/853103698 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:26.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.204+0000 7fe061000640 1 --2- 172.21.15.159:0/853103698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe05c14cc80 0x7fe05c1f7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:26.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.205+0000 7fe05be00640 1 --2- 172.21.15.159:0/853103698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0400039b0 0x7fe05c145010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:26.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.205+0000 7fe053400640 1 -- 172.21.15.159:0/853103698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe05c14cc80 msgr2=0x7fe05c1f7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:26.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.205+0000 7fe053400640 1 --2- 172.21.15.159:0/853103698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe05c14cc80 0x7fe05c1f7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:26.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.205+0000 7fe053400640 1 -- 172.21.15.159:0/853103698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0400039b0 msgr2=0x7fe05c145010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:26.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.205+0000 7fe053400640 1 --2- 172.21.15.159:0/853103698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0400039b0 0x7fe05c145010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:26.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.205+0000 7fe053400640 1 -- 172.21.15.159:0/853103698 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05c0cc8e0 con 0x7fe05c1f8bd0 2024-04-04T06:15:26.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.205+0000 7fe053400640 1 --2- 172.21.15.159:0/853103698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe05c1f8bd0 0x7fe05c0ced30 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fe044010940 tx=0x7fe044010e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:26.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.205+0000 7fe058c00640 1 -- 172.21.15.159:0/853103698 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04400f040 con 0x7fe05c1f8bd0 2024-04-04T06:15:26.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.205+0000 7fe058c00640 1 -- 172.21.15.159:0/853103698 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe04400ebc0 con 0x7fe05c1f8bd0 2024-04-04T06:15:26.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.205+0000 7fe063000640 1 -- 172.21.15.159:0/853103698 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe05c0ccbc0 con 0x7fe05c1f8bd0 2024-04-04T06:15:26.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.205+0000 7fe058c00640 1 -- 172.21.15.159:0/853103698 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04401c400 con 0x7fe05c1f8bd0 2024-04-04T06:15:26.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.206+0000 7fe063000640 1 -- 172.21.15.159:0/853103698 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe05c0cd180 con 0x7fe05c1f8bd0 2024-04-04T06:15:26.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.207+0000 7fe063000640 1 -- 172.21.15.159:0/853103698 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe05c0685c0 con 0x7fe05c1f8bd0 2024-04-04T06:15:26.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.207+0000 7fe058c00640 1 -- 172.21.15.159:0/853103698 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe04401c5a0 con 0x7fe05c1f8bd0 2024-04-04T06:15:26.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.208+0000 7fe058c00640 1 --2- 172.21.15.159:0/853103698 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe03806c8f0 0x7fe03806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:26.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.208+0000 7fe058c00640 1 -- 172.21.15.159:0/853103698 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fe04408bfe0 con 0x7fe05c1f8bd0 2024-04-04T06:15:26.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.209+0000 7fe05be00640 1 --2- 172.21.15.159:0/853103698 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe03806c8f0 0x7fe03806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:26.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.210+0000 7fe05be00640 1 --2- 172.21.15.159:0/853103698 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe03806c8f0 0x7fe03806edb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe04c034000 tx=0x7fe04c009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:26.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.213+0000 7fe058c00640 1 -- 172.21.15.159:0/853103698 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe044018030 con 0x7fe05c1f8bd0 2024-04-04T06:15:26.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.414+0000 7fe063000640 1 -- 172.21.15.159:0/853103698 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe05c0cc560 con 0x7fe05c1f8bd0 2024-04-04T06:15:26.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.414+0000 7fe058c00640 1 -- 172.21.15.159:0/853103698 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7fe04405a440 con 0x7fe05c1f8bd0 2024-04-04T06:15:26.408 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:15:26.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.417+0000 7fe063000640 1 -- 172.21.15.159:0/853103698 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe03806c8f0 msgr2=0x7fe03806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:26.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.417+0000 7fe063000640 1 --2- 172.21.15.159:0/853103698 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe03806c8f0 0x7fe03806edb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe04c034000 tx=0x7fe04c009030 comp rx=0 tx=0).stop 2024-04-04T06:15:26.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.417+0000 7fe063000640 1 -- 172.21.15.159:0/853103698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe05c1f8bd0 msgr2=0x7fe05c0ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:26.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.417+0000 7fe063000640 1 --2- 172.21.15.159:0/853103698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe05c1f8bd0 0x7fe05c0ced30 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fe044010940 tx=0x7fe044010e00 comp rx=0 tx=0).stop 2024-04-04T06:15:26.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.417+0000 7fe063000640 1 -- 172.21.15.159:0/853103698 shutdown_connections 2024-04-04T06:15:26.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.417+0000 7fe063000640 1 --2- 172.21.15.159:0/853103698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe05c14cc80 0x7fe05c1f7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:26.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.417+0000 7fe063000640 1 --2- 172.21.15.159:0/853103698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe05c1f8bd0 0x7fe05c0ced30 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:26.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.417+0000 7fe063000640 1 --2- 172.21.15.159:0/853103698 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe03806c8f0 0x7fe03806edb0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:26.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.417+0000 7fe063000640 1 --2- 172.21.15.159:0/853103698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0400039b0 0x7fe05c145010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:26.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.417+0000 7fe063000640 1 -- 172.21.15.159:0/853103698 >> 172.21.15.159:0/853103698 conn(0x7fe05c06b750 msgr2=0x7fe05c15d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:26.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.417+0000 7fe063000640 1 -- 172.21.15.159:0/853103698 shutdown_connections 2024-04-04T06:15:26.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.417+0000 7fe063000640 1 -- 172.21.15.159:0/853103698 wait complete. 2024-04-04T06:15:26.436 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-04T06:15:26.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.643+0000 7fe590800640 1 Processor -- start 2024-04-04T06:15:26.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.643+0000 7fe590800640 1 -- start start 2024-04-04T06:15:26.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.645+0000 7fe590800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe5880d1560 0x7fe58806c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:26.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.645+0000 7fe590800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe58806cbc0 0x7fe5880cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:26.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.645+0000 7fe590800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe58806cfd0 con 0x7fe58815e590 2024-04-04T06:15:26.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.645+0000 7fe590800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe5880d19d0 con 0x7fe5880d1560 2024-04-04T06:15:26.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.645+0000 7fe590800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe588071060 con 0x7fe58806cbc0 2024-04-04T06:15:26.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.645+0000 7fe58de00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fe58815e590 0x7fe5880b47d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:53544/0 (socket says 172.21.15.159:53544) 2024-04-04T06:15:26.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.645+0000 7fe58de00640 1 -- 172.21.15.159:0/2664964281 learned_addr learned my addr 172.21.15.159:0/2664964281 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:26.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.646+0000 7fe58e800640 1 --2- 172.21.15.159:0/2664964281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe58806cbc0 0x7fe5880cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:26.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.646+0000 7fe58d400640 1 --2- 172.21.15.159:0/2664964281 >> v2:172.21.15.178:3300/0 conn(0x7fe5880d1560 0x7fe58806c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:26.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.646+0000 7fe58ca00640 1 -- 172.21.15.159:0/2664964281 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 286207566 0 0) 0x7fe57c001110 con 0x7fe58815e590 2024-04-04T06:15:26.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.646+0000 7fe58ca00640 1 -- 172.21.15.159:0/2664964281 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe570003520 con 0x7fe58815e590 2024-04-04T06:15:26.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.646+0000 7fe58ca00640 1 -- 172.21.15.159:0/2664964281 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2545571395 0 0) 0x7fe570003520 con 0x7fe58815e590 2024-04-04T06:15:26.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.646+0000 7fe58ca00640 1 -- 172.21.15.159:0/2664964281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe58806cbc0 msgr2=0x7fe5880cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:26.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.646+0000 7fe58ca00640 1 --2- 172.21.15.159:0/2664964281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe58806cbc0 0x7fe5880cfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:26.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.646+0000 7fe58ca00640 1 -- 172.21.15.159:0/2664964281 >> v2:172.21.15.178:3300/0 conn(0x7fe5880d1560 msgr2=0x7fe58806c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:26.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.646+0000 7fe58ca00640 1 --2- 172.21.15.159:0/2664964281 >> v2:172.21.15.178:3300/0 conn(0x7fe5880d1560 0x7fe58806c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:26.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.646+0000 7fe58ca00640 1 -- 172.21.15.159:0/2664964281 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe588071340 con 0x7fe58815e590 2024-04-04T06:15:26.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.646+0000 7fe58ca00640 1 -- 172.21.15.159:0/2664964281 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fe57c001840 con 0x7fe58815e590 2024-04-04T06:15:26.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.646+0000 7fe58ca00640 1 -- 172.21.15.159:0/2664964281 >> v1:172.21.15.159:6789/0 conn(0x7fe58815e590 legacy=0x7fe5880b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:26.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.646+0000 7fe58ca00640 1 --2- 172.21.15.159:0/2664964281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5700039b0 0x7fe570003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:26.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.646+0000 7fe58ca00640 1 --2- 172.21.15.159:0/2664964281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe58806cbc0 0x7fe570004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:26.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.646+0000 7fe58ca00640 1 --2- 172.21.15.159:0/2664964281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe570004d40 0x7fe5700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:26.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.646+0000 7fe58ca00640 1 -- 172.21.15.159:0/2664964281 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe588071060 con 0x7fe570004d40 2024-04-04T06:15:26.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.646+0000 7fe58ca00640 1 -- 172.21.15.159:0/2664964281 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe57000b9c0 con 0x7fe5700039b0 2024-04-04T06:15:26.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.646+0000 7fe58ca00640 1 -- 172.21.15.159:0/2664964281 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe57000bb40 con 0x7fe58806cbc0 2024-04-04T06:15:26.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.647+0000 7fe58e800640 1 --2- 172.21.15.159:0/2664964281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe570004d40 0x7fe5700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:26.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.647+0000 7fe58d400640 1 --2- 172.21.15.159:0/2664964281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe58806cbc0 0x7fe570004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:26.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.647+0000 7fe58e800640 1 -- 172.21.15.159:0/2664964281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe58806cbc0 msgr2=0x7fe570004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:26.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.647+0000 7fe58de00640 1 --2- 172.21.15.159:0/2664964281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5700039b0 0x7fe570003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:26.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.647+0000 7fe58e800640 1 --2- 172.21.15.159:0/2664964281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe58806cbc0 0x7fe570004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:26.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.647+0000 7fe58e800640 1 -- 172.21.15.159:0/2664964281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5700039b0 msgr2=0x7fe570003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:26.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.647+0000 7fe58e800640 1 --2- 172.21.15.159:0/2664964281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5700039b0 0x7fe570003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:26.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.647+0000 7fe58e800640 1 -- 172.21.15.159:0/2664964281 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe570003520 con 0x7fe570004d40 2024-04-04T06:15:26.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.648+0000 7fe58e800640 1 --2- 172.21.15.159:0/2664964281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe570004d40 0x7fe5700051c0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fe58400e170 tx=0x7fe570005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c706fcaa1b74ba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:26.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.648+0000 7fe58ca00640 1 -- 172.21.15.159:0/2664964281 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584039070 con 0x7fe570004d40 2024-04-04T06:15:26.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.648+0000 7fe58ca00640 1 -- 172.21.15.159:0/2664964281 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe58402cbf0 con 0x7fe570004d40 2024-04-04T06:15:26.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.648+0000 7fe58ca00640 1 -- 172.21.15.159:0/2664964281 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584009c10 con 0x7fe570004d40 2024-04-04T06:15:26.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.649+0000 7fe590800640 1 -- 172.21.15.159:0/2664964281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe570004d40 msgr2=0x7fe5700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:26.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.649+0000 7fe590800640 1 --2- 172.21.15.159:0/2664964281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe570004d40 0x7fe5700051c0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fe58400e170 tx=0x7fe570005830 comp rx=0 tx=0).stop 2024-04-04T06:15:26.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.649+0000 7fe58e800640 1 -- 172.21.15.159:0/2664964281 reap_dead start 2024-04-04T06:15:26.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.649+0000 7fe590800640 1 -- 172.21.15.159:0/2664964281 shutdown_connections 2024-04-04T06:15:26.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.649+0000 7fe590800640 1 -- 172.21.15.159:0/2664964281 >> 172.21.15.159:0/2664964281 conn(0x7fe5880d2240 msgr2=0x7fe5880d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:26.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.649+0000 7fe590800640 1 -- 172.21.15.159:0/2664964281 shutdown_connections 2024-04-04T06:15:26.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.649+0000 7fe590800640 1 -- 172.21.15.159:0/2664964281 wait complete. 2024-04-04T06:15:26.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.651+0000 7fe590800640 1 Processor -- start 2024-04-04T06:15:26.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.651+0000 7fe590800640 1 -- start start 2024-04-04T06:15:26.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.652+0000 7fe590800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe570004d40 0x7fe5881e91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:26.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.652+0000 7fe590800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5880b1de0 0x7fe5881e9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:26.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.652+0000 7fe590800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5881e9c70 0x7fe5880b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:26.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.652+0000 7fe590800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe588071be0 con 0x7fe5880b1de0 2024-04-04T06:15:26.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.652+0000 7fe590800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe588071900 con 0x7fe5881e9c70 2024-04-04T06:15:26.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.652+0000 7fe590800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe58815e970 con 0x7fe570004d40 2024-04-04T06:15:26.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.652+0000 7fe58d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5880b1de0 0x7fe5881e9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:26.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.652+0000 7fe58d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5880b1de0 0x7fe5881e9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39518/0 (socket says 172.21.15.159:39518) 2024-04-04T06:15:26.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.652+0000 7fe58d400640 1 -- 172.21.15.159:0/1997241909 learned_addr learned my addr 172.21.15.159:0/1997241909 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:26.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.653+0000 7fe58de00640 1 --2- 172.21.15.159:0/1997241909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe570004d40 0x7fe5881e91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:26.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.653+0000 7fe58e800640 1 --2- 172.21.15.159:0/1997241909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5881e9c70 0x7fe5880b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:26.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.653+0000 7fe58d400640 1 -- 172.21.15.159:0/1997241909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe570004d40 msgr2=0x7fe5881e91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:26.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.653+0000 7fe58d400640 1 --2- 172.21.15.159:0/1997241909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe570004d40 0x7fe5881e91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:26.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.653+0000 7fe58d400640 1 -- 172.21.15.159:0/1997241909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5881e9c70 msgr2=0x7fe5880b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:26.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.653+0000 7fe58d400640 1 --2- 172.21.15.159:0/1997241909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5881e9c70 0x7fe5880b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:26.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.653+0000 7fe58d400640 1 -- 172.21.15.159:0/1997241909 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5880b22a0 con 0x7fe5880b1de0 2024-04-04T06:15:26.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.653+0000 7fe58de00640 1 --2- 172.21.15.159:0/1997241909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe570004d40 0x7fe5881e91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:26.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.653+0000 7fe58e800640 1 --2- 172.21.15.159:0/1997241909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5881e9c70 0x7fe5880b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:26.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.653+0000 7fe58d400640 1 --2- 172.21.15.159:0/1997241909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5880b1de0 0x7fe5881e9730 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fe578011700 tx=0x7fe578011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:26.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.653+0000 7fe582000640 1 -- 172.21.15.159:0/1997241909 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57800f9c0 con 0x7fe5880b1de0 2024-04-04T06:15:26.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.653+0000 7fe582000640 1 -- 172.21.15.159:0/1997241909 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe57800fb60 con 0x7fe5880b1de0 2024-04-04T06:15:26.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.653+0000 7fe582000640 1 -- 172.21.15.159:0/1997241909 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe578015460 con 0x7fe5880b1de0 2024-04-04T06:15:26.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.653+0000 7fe590800640 1 -- 172.21.15.159:0/1997241909 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe58815f0c0 con 0x7fe5880b1de0 2024-04-04T06:15:26.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.654+0000 7fe590800640 1 -- 172.21.15.159:0/1997241909 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5880b2580 con 0x7fe5880b1de0 2024-04-04T06:15:26.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.655+0000 7fe590800640 1 -- 172.21.15.159:0/1997241909 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5880685c0 con 0x7fe5880b1de0 2024-04-04T06:15:26.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.656+0000 7fe582000640 1 -- 172.21.15.159:0/1997241909 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe578015600 con 0x7fe5880b1de0 2024-04-04T06:15:26.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.657+0000 7fe582000640 1 --2- 172.21.15.159:0/1997241909 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe56c06c920 0x7fe56c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:26.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.657+0000 7fe582000640 1 -- 172.21.15.159:0/1997241909 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fe57808c560 con 0x7fe5880b1de0 2024-04-04T06:15:26.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.658+0000 7fe58de00640 1 --2- 172.21.15.159:0/1997241909 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe56c06c920 0x7fe56c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:26.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.659+0000 7fe58de00640 1 --2- 172.21.15.159:0/1997241909 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe56c06c920 0x7fe56c06ede0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe57c001eb0 tx=0x7fe57c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:26.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.661+0000 7fe582000640 1 -- 172.21.15.159:0/1997241909 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe578019030 con 0x7fe5880b1de0 2024-04-04T06:15:26.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.873+0000 7fe590800640 1 -- 172.21.15.159:0/1997241909 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe5880b1960 con 0x7fe5880b1de0 2024-04-04T06:15:26.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.874+0000 7fe582000640 1 -- 172.21.15.159:0/1997241909 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v101) ==== 80+0+0 (secure 0 0 0) 0x7fe57805a980 con 0x7fe5880b1de0 2024-04-04T06:15:26.867 INFO:teuthology.orchestra.run.smithi159.stderr:listed 0 entries 2024-04-04T06:15:26.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.876+0000 7fe590800640 1 -- 172.21.15.159:0/1997241909 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe56c06c920 msgr2=0x7fe56c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:26.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.876+0000 7fe590800640 1 --2- 172.21.15.159:0/1997241909 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe56c06c920 0x7fe56c06ede0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe57c001eb0 tx=0x7fe57c009040 comp rx=0 tx=0).stop 2024-04-04T06:15:26.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.876+0000 7fe590800640 1 -- 172.21.15.159:0/1997241909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5880b1de0 msgr2=0x7fe5881e9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:26.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.876+0000 7fe590800640 1 --2- 172.21.15.159:0/1997241909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5880b1de0 0x7fe5881e9730 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fe578011700 tx=0x7fe578011bc0 comp rx=0 tx=0).stop 2024-04-04T06:15:26.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.876+0000 7fe590800640 1 -- 172.21.15.159:0/1997241909 shutdown_connections 2024-04-04T06:15:26.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.876+0000 7fe590800640 1 --2- 172.21.15.159:0/1997241909 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe56c06c920 0x7fe56c06ede0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:26.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.876+0000 7fe590800640 1 --2- 172.21.15.159:0/1997241909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5881e9c70 0x7fe5880b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:26.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.876+0000 7fe590800640 1 --2- 172.21.15.159:0/1997241909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5880b1de0 0x7fe5881e9730 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:26.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.876+0000 7fe590800640 1 --2- 172.21.15.159:0/1997241909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe570004d40 0x7fe5881e91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:26.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.876+0000 7fe590800640 1 -- 172.21.15.159:0/1997241909 >> 172.21.15.159:0/1997241909 conn(0x7fe5880d2240 msgr2=0x7fe5881570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:26.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.876+0000 7fe590800640 1 -- 172.21.15.159:0/1997241909 shutdown_connections 2024-04-04T06:15:26.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:26.876+0000 7fe590800640 1 -- 172.21.15.159:0/1997241909 wait complete. 2024-04-04T06:15:26.890 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-04T06:15:26.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:26.890 DEBUG:teuthology.orchestra.run.smithi159:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-04T06:15:27.008 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:15:27.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:27.008 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-04T06:15:27.073 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:15:27.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.258+0000 7fdebbe00640 1 Processor -- start 2024-04-04T06:15:27.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.258+0000 7fdebbe00640 1 -- start start 2024-04-04T06:15:27.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.259+0000 7fdebbe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fdebc0d2310 0x7fdebc0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:27.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.259+0000 7fdebbe00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdebc15e880 con 0x7fdebc15e4a0 2024-04-04T06:15:27.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.259+0000 7fdebbe00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdebc0d28d0 con 0x7fdebc0cf530 2024-04-04T06:15:27.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.259+0000 7fdebbe00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fdebc0cf9a0 con 0x7fdebc0d2310 2024-04-04T06:15:27.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.260+0000 7fdeba000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fdebc0d2310 0x7fdebc0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:27.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.260+0000 7fdeba000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fdebc0d2310 0x7fdebc0ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39520/0 (socket says 172.21.15.159:39520) 2024-04-04T06:15:27.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.260+0000 7fdeba000640 1 -- 172.21.15.159:0/4174655226 learned_addr learned my addr 172.21.15.159:0/4174655226 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:27.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.260+0000 7fdebaa00640 1 --1- 172.21.15.159:0/4174655226 >> v1:172.21.15.159:6789/0 conn(0x7fdebc15e4a0 0x7fdebc0b9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:53558/0 (socket says 172.21.15.159:53558) 2024-04-04T06:15:27.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.260+0000 7fdeba000640 1 -- 172.21.15.159:0/4174655226 >> v1:172.21.15.178:6789/0 conn(0x7fdebc0cf530 legacy=0x7fdebc0b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:27.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.260+0000 7fdeba000640 1 -- 172.21.15.159:0/4174655226 >> v1:172.21.15.159:6789/0 conn(0x7fdebc15e4a0 legacy=0x7fdebc0b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:27.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.260+0000 7fdeba000640 1 -- 172.21.15.159:0/4174655226 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdebc0d2bb0 con 0x7fdebc0d2310 2024-04-04T06:15:27.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.261+0000 7fdeba000640 1 --2- 172.21.15.159:0/4174655226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdebc0d2310 0x7fdebc0ceff0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fdea8009870 tx=0x7fdea802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fc5108b06a1d6d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:27.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.262+0000 7fdeb9600640 1 -- 172.21.15.159:0/4174655226 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea8035020 con 0x7fdebc0d2310 2024-04-04T06:15:27.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.262+0000 7fdeb9600640 1 -- 172.21.15.159:0/4174655226 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea802d040 con 0x7fdebc0d2310 2024-04-04T06:15:27.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.262+0000 7fdeb9600640 1 -- 172.21.15.159:0/4174655226 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea8031a80 con 0x7fdebc0d2310 2024-04-04T06:15:27.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.262+0000 7fdebbe00640 1 -- 172.21.15.159:0/4174655226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdebc0d2310 msgr2=0x7fdebc0ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:27.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.262+0000 7fdebbe00640 1 --2- 172.21.15.159:0/4174655226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdebc0d2310 0x7fdebc0ceff0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fdea8009870 tx=0x7fdea802b900 comp rx=0 tx=0).stop 2024-04-04T06:15:27.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.262+0000 7fdebbe00640 1 -- 172.21.15.159:0/4174655226 shutdown_connections 2024-04-04T06:15:27.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.262+0000 7fdebbe00640 1 --2- 172.21.15.159:0/4174655226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdebc0d2310 0x7fdebc0ceff0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:27.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.262+0000 7fdebbe00640 1 -- 172.21.15.159:0/4174655226 >> 172.21.15.159:0/4174655226 conn(0x7fdebc0676d0 msgr2=0x7fdebc067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:27.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.263+0000 7fdebbe00640 1 -- 172.21.15.159:0/4174655226 shutdown_connections 2024-04-04T06:15:27.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.263+0000 7fdebbe00640 1 -- 172.21.15.159:0/4174655226 wait complete. 2024-04-04T06:15:27.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.264+0000 7fdebbe00640 1 Processor -- start 2024-04-04T06:15:27.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.264+0000 7fdebbe00640 1 -- start start 2024-04-04T06:15:27.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.265+0000 7fdebbe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdebc0cf530 0x7fdebc14b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:27.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.265+0000 7fdebbe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdebc0d2310 0x7fdebc155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:27.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.265+0000 7fdebbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdebc15e4a0 0x7fdebc0ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:27.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.265+0000 7fdebbe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc0d3170 con 0x7fdebc15e4a0 2024-04-04T06:15:27.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.265+0000 7fdebbe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc0d34e0 con 0x7fdebc0cf530 2024-04-04T06:15:27.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.265+0000 7fdebbe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdebc0d3970 con 0x7fdebc0d2310 2024-04-04T06:15:27.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.266+0000 7fdebb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdebc15e4a0 0x7fdebc0ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:27.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.266+0000 7fdebb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdebc15e4a0 0x7fdebc0ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39534/0 (socket says 172.21.15.159:39534) 2024-04-04T06:15:27.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.266+0000 7fdebb400640 1 -- 172.21.15.159:0/1730167451 learned_addr learned my addr 172.21.15.159:0/1730167451 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:27.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.266+0000 7fdebaa00640 1 --2- 172.21.15.159:0/1730167451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdebc0cf530 0x7fdebc14b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:27.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.266+0000 7fdeba000640 1 --2- 172.21.15.159:0/1730167451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdebc0d2310 0x7fdebc155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:27.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.266+0000 7fdebb400640 1 -- 172.21.15.159:0/1730167451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdebc0d2310 msgr2=0x7fdebc155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:27.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.266+0000 7fdebb400640 1 --2- 172.21.15.159:0/1730167451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdebc0d2310 0x7fdebc155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:27.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.266+0000 7fdebb400640 1 -- 172.21.15.159:0/1730167451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdebc0cf530 msgr2=0x7fdebc14b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:27.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.266+0000 7fdebb400640 1 --2- 172.21.15.159:0/1730167451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdebc0cf530 0x7fdebc14b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:27.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.266+0000 7fdebb400640 1 -- 172.21.15.159:0/1730167451 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdebc0cd790 con 0x7fdebc15e4a0 2024-04-04T06:15:27.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.267+0000 7fdebb400640 1 --2- 172.21.15.159:0/1730167451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdebc15e4a0 0x7fdebc0ccdf0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400c780 tx=0x7fdeb400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:27.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.267+0000 7fdeb2a00640 1 -- 172.21.15.159:0/1730167451 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4004b60 con 0x7fdebc15e4a0 2024-04-04T06:15:27.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.267+0000 7fdeb2a00640 1 -- 172.21.15.159:0/1730167451 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb4004d00 con 0x7fdebc15e4a0 2024-04-04T06:15:27.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.267+0000 7fdeb2a00640 1 -- 172.21.15.159:0/1730167451 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb400aa90 con 0x7fdebc15e4a0 2024-04-04T06:15:27.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.267+0000 7fdebbe00640 1 -- 172.21.15.159:0/1730167451 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdebc1552c0 con 0x7fdebc15e4a0 2024-04-04T06:15:27.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.267+0000 7fdebbe00640 1 -- 172.21.15.159:0/1730167451 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdebc155880 con 0x7fdebc15e4a0 2024-04-04T06:15:27.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.268+0000 7fdebbe00640 1 -- 172.21.15.159:0/1730167451 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdebc075940 con 0x7fdebc15e4a0 2024-04-04T06:15:27.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.269+0000 7fdeb2a00640 1 -- 172.21.15.159:0/1730167451 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdeb400ac30 con 0x7fdebc15e4a0 2024-04-04T06:15:27.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.271+0000 7fdeb2a00640 1 --2- 172.21.15.159:0/1730167451 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde9c06c920 0x7fde9c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:27.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.271+0000 7fdeb2a00640 1 -- 172.21.15.159:0/1730167451 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fdeb408bde0 con 0x7fdebc15e4a0 2024-04-04T06:15:27.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.271+0000 7fdebaa00640 1 --2- 172.21.15.159:0/1730167451 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde9c06c920 0x7fde9c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:27.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.272+0000 7fdebaa00640 1 --2- 172.21.15.159:0/1730167451 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde9c06c920 0x7fde9c06ede0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdeac000f60 tx=0x7fdeac007ce0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:27.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.274+0000 7fdeb2a00640 1 -- 172.21.15.159:0/1730167451 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fdeb4020070 con 0x7fdebc15e4a0 2024-04-04T06:15:27.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.517+0000 7fdebbe00640 1 -- 172.21.15.159:0/1730167451 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdebc0d4ba0 con 0x7fdebc15e4a0 2024-04-04T06:15:27.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.518+0000 7fdeb2a00640 1 -- 172.21.15.159:0/1730167451 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5189 (secure 0 0 0) 0x7fdeb405a200 con 0x7fdebc15e4a0 2024-04-04T06:15:27.513 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:15:27.513 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":84,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5546,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/40219476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":40219476},{"type":"v1","addr":"172.21.15.178:6837","nonce":40219476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/4060483382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4060483382},{"type":"v1","addr":"172.21.15.178:6836","nonce":4060483382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8187,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1945665739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1945665739},{"type":"v1","addr":"172.21.15.159:6833","nonce":1945665739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:15:07.704835+0000","modified":"2024-04-04T06:15:10.811654+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8190":{"gid":8190,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6835/2253383212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":2253383212},{"type":"v1","addr":"172.21.15.159:6835","nonce":2253383212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8190,"qdb_cluster":[8190]},"id":7}]} 2024-04-04T06:15:27.513 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 84 2024-04-04T06:15:27.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.521+0000 7fdebbe00640 1 -- 172.21.15.159:0/1730167451 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde9c06c920 msgr2=0x7fde9c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:27.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.521+0000 7fdebbe00640 1 --2- 172.21.15.159:0/1730167451 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde9c06c920 0x7fde9c06ede0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdeac000f60 tx=0x7fdeac007ce0 comp rx=0 tx=0).stop 2024-04-04T06:15:27.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.521+0000 7fdebbe00640 1 -- 172.21.15.159:0/1730167451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdebc15e4a0 msgr2=0x7fdebc0ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:27.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.521+0000 7fdebbe00640 1 --2- 172.21.15.159:0/1730167451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdebc15e4a0 0x7fdebc0ccdf0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400c780 tx=0x7fdeb400cc40 comp rx=0 tx=0).stop 2024-04-04T06:15:27.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.521+0000 7fdebbe00640 1 -- 172.21.15.159:0/1730167451 shutdown_connections 2024-04-04T06:15:27.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.521+0000 7fdebbe00640 1 --2- 172.21.15.159:0/1730167451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdebc15e4a0 0x7fdebc0ccdf0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:27.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.521+0000 7fdebbe00640 1 --2- 172.21.15.159:0/1730167451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdebc0d2310 0x7fdebc155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:27.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.521+0000 7fdebbe00640 1 --2- 172.21.15.159:0/1730167451 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde9c06c920 0x7fde9c06ede0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:27.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.521+0000 7fdebbe00640 1 --2- 172.21.15.159:0/1730167451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdebc0cf530 0x7fdebc14b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:27.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.521+0000 7fdebbe00640 1 -- 172.21.15.159:0/1730167451 >> 172.21.15.159:0/1730167451 conn(0x7fdebc0676d0 msgr2=0x7fdebc15d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:27.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.521+0000 7fdebbe00640 1 -- 172.21.15.159:0/1730167451 shutdown_connections 2024-04-04T06:15:27.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:27.521+0000 7fdebbe00640 1 -- 172.21.15.159:0/1730167451 wait complete. 2024-04-04T06:15:27.534 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-04T06:15:27.804 INFO:teuthology.orchestra.run.smithi159.stdout:{"message":"","return_code":0} 2024-04-04T06:15:27.806 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-04T06:15:27.806 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:15:27.806 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:15:27.851 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:15:27.851 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:15:27.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:27.851 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-04T06:15:27.943 INFO:teuthology.orchestra.run.smithi159.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-04T06:15:27.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-04T06:15:27.945 DEBUG:teuthology.orchestra.run.smithi159:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-04T06:15:27.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:27.950 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-04T06:15:28.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.165+0000 7f3aa5000640 1 Processor -- start 2024-04-04T06:15:28.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.165+0000 7f3aa5000640 1 -- start start 2024-04-04T06:15:28.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.167+0000 7f3aa5000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3aa0073810 0x7f3aa015ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.167+0000 7f3aa5000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aa00b3310 con 0x7f3aa00716a0 2024-04-04T06:15:28.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.167+0000 7f3aa5000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aa00b35f0 con 0x7f3aa0147fb0 2024-04-04T06:15:28.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.167+0000 7f3aa5000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3aa00b3f20 con 0x7f3aa0073810 2024-04-04T06:15:28.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.167+0000 7f3a9e800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f3aa00716a0 0x7f3aa006d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:53572/0 (socket says 172.21.15.159:53572) 2024-04-04T06:15:28.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.167+0000 7f3a9e800640 1 -- 172.21.15.159:0/3304623455 learned_addr learned my addr 172.21.15.159:0/3304623455 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:28.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.168+0000 7f3a97e00640 1 --2- 172.21.15.159:0/3304623455 >> v2:172.21.15.178:3300/0 conn(0x7f3aa0073810 0x7f3aa015ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.168+0000 7f3a9d400640 1 -- 172.21.15.159:0/3304623455 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4212735975 0 0) 0x7f3aa00b3310 con 0x7f3aa00716a0 2024-04-04T06:15:28.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.168+0000 7f3a9d400640 1 -- 172.21.15.159:0/3304623455 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a84003520 con 0x7f3aa00716a0 2024-04-04T06:15:28.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.168+0000 7f3a9d400640 1 -- 172.21.15.159:0/3304623455 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 362930865 0 0) 0x7f3a84003520 con 0x7f3aa00716a0 2024-04-04T06:15:28.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.168+0000 7f3a9d400640 1 -- 172.21.15.159:0/3304623455 >> v2:172.21.15.178:3300/0 conn(0x7f3aa0073810 msgr2=0x7f3aa015ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.168+0000 7f3a9d400640 1 --2- 172.21.15.159:0/3304623455 >> v2:172.21.15.178:3300/0 conn(0x7f3aa0073810 0x7f3aa015ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.168+0000 7f3a9d400640 1 -- 172.21.15.159:0/3304623455 >> v1:172.21.15.178:6789/0 conn(0x7f3aa0147fb0 legacy=0x7f3aa00b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.168+0000 7f3a9d400640 1 -- 172.21.15.159:0/3304623455 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa00b38d0 con 0x7f3aa00716a0 2024-04-04T06:15:28.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.168+0000 7f3a9d400640 1 -- 172.21.15.159:0/3304623455 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f3a980019f0 con 0x7f3aa00716a0 2024-04-04T06:15:28.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.168+0000 7f3a9d400640 1 -- 172.21.15.159:0/3304623455 >> v1:172.21.15.159:6789/0 conn(0x7f3aa00716a0 legacy=0x7f3aa006d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.168+0000 7f3a9d400640 1 --2- 172.21.15.159:0/3304623455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a84003a00 0x7f3a84003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.168+0000 7f3a9d400640 1 --2- 172.21.15.159:0/3304623455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a84004800 0x7f3a84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.169+0000 7f3a9d400640 1 --2- 172.21.15.159:0/3304623455 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a84005210 0x7f3a8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.169+0000 7f3a9d400640 1 -- 172.21.15.159:0/3304623455 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa00b3f20 con 0x7f3a84004800 2024-04-04T06:15:28.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.169+0000 7f3a9d400640 1 -- 172.21.15.159:0/3304623455 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8400c160 con 0x7f3a84003a00 2024-04-04T06:15:28.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.169+0000 7f3a9d400640 1 -- 172.21.15.159:0/3304623455 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3a8400c360 con 0x7f3a84005210 2024-04-04T06:15:28.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.169+0000 7f3a97e00640 1 --2- 172.21.15.159:0/3304623455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a84004800 0x7f3a84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.169+0000 7f3a9e800640 1 --2- 172.21.15.159:0/3304623455 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a84005210 0x7f3a8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.169+0000 7f3a97e00640 1 -- 172.21.15.159:0/3304623455 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a84005210 msgr2=0x7f3a8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.169+0000 7f3a9de00640 1 --2- 172.21.15.159:0/3304623455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a84003a00 0x7f3a84003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.169+0000 7f3a97e00640 1 --2- 172.21.15.159:0/3304623455 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a84005210 0x7f3a8400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.169+0000 7f3a97e00640 1 -- 172.21.15.159:0/3304623455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a84003a00 msgr2=0x7f3a84003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.169+0000 7f3a97e00640 1 --2- 172.21.15.159:0/3304623455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a84003a00 0x7f3a84003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.169+0000 7f3a97e00640 1 -- 172.21.15.159:0/3304623455 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a84003520 con 0x7f3a84004800 2024-04-04T06:15:28.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.169+0000 7f3a9e800640 1 --2- 172.21.15.159:0/3304623455 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a84005210 0x7f3a8400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:28.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.169+0000 7f3a9de00640 1 --2- 172.21.15.159:0/3304623455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a84003a00 0x7f3a84003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:28.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.169+0000 7f3a9e800640 1 -- 172.21.15.159:0/3304623455 reap_dead start 2024-04-04T06:15:28.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.171+0000 7f3a97e00640 1 --2- 172.21.15.159:0/3304623455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a84004800 0x7f3a84004c80 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009a90 tx=0x7f3a84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95a7b8d416398a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:28.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.171+0000 7f3a9d400640 1 -- 172.21.15.159:0/3304623455 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a88039070 con 0x7f3a84004800 2024-04-04T06:15:28.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.171+0000 7f3a9d400640 1 -- 172.21.15.159:0/3304623455 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a8802c680 con 0x7f3a84004800 2024-04-04T06:15:28.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.171+0000 7f3a9d400640 1 -- 172.21.15.159:0/3304623455 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a88009690 con 0x7f3a84004800 2024-04-04T06:15:28.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.171+0000 7f3aa5000640 1 -- 172.21.15.159:0/3304623455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a84004800 msgr2=0x7f3a84004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.171+0000 7f3aa5000640 1 --2- 172.21.15.159:0/3304623455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a84004800 0x7f3a84004c80 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009a90 tx=0x7f3a84005830 comp rx=0 tx=0).stop 2024-04-04T06:15:28.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.172+0000 7f3aa5000640 1 -- 172.21.15.159:0/3304623455 shutdown_connections 2024-04-04T06:15:28.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.172+0000 7f3aa5000640 1 --2- 172.21.15.159:0/3304623455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a84004800 0x7f3a84004c80 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.172+0000 7f3aa5000640 1 -- 172.21.15.159:0/3304623455 >> 172.21.15.159:0/3304623455 conn(0x7f3aa006b570 msgr2=0x7f3aa006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:28.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.172+0000 7f3aa5000640 1 -- 172.21.15.159:0/3304623455 shutdown_connections 2024-04-04T06:15:28.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.172+0000 7f3aa5000640 1 -- 172.21.15.159:0/3304623455 wait complete. 2024-04-04T06:15:28.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.174+0000 7f3aa5000640 1 Processor -- start 2024-04-04T06:15:28.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.174+0000 7f3aa5000640 1 -- start start 2024-04-04T06:15:28.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.174+0000 7f629f200640 1 Processor -- start 2024-04-04T06:15:28.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.174+0000 7f629f200640 1 -- start start 2024-04-04T06:15:28.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.176+0000 7f629f200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f62a00b6310 0x7f62a00b6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.176+0000 7f629f200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62a00d23b0 con 0x7f62a00d3d50 2024-04-04T06:15:28.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.176+0000 7f629f200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62a00d2690 con 0x7f62a015e420 2024-04-04T06:15:28.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.176+0000 7f629f200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f62a00d2fc0 con 0x7f62a00b6310 2024-04-04T06:15:28.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.176+0000 7f3aa5000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a84004800 0x7f3aa0141d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.176+0000 7f3aa5000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3aa009b3e0 0x7f3aa009b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.176+0000 7f3aa5000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3aa00adf60 0x7f3aa00ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.176+0000 7f3aa5000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa00b2f30 con 0x7f3aa00adf60 2024-04-04T06:15:28.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.176+0000 7f3aa5000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa00b3bb0 con 0x7f3a84004800 2024-04-04T06:15:28.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.176+0000 7f3aa5000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa00b30b0 con 0x7f3aa009b3e0 2024-04-04T06:15:28.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.176+0000 7f629d400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f62a00d3d50 0x7f62a00d3450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:53576/0 (socket says 172.21.15.159:53576) 2024-04-04T06:15:28.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.176+0000 7f629d400640 1 -- 172.21.15.159:0/673130345 learned_addr learned my addr 172.21.15.159:0/673130345 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:28.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.176+0000 7f629e800640 1 --2- 172.21.15.159:0/673130345 >> v2:172.21.15.159:3300/0 conn(0x7f62a00b6310 0x7f62a00b6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.176+0000 7f3a9e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3aa00adf60 0x7f3aa00ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.176+0000 7f3a9de00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a84004800 0x7f3aa0141d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.176+0000 7f3a9e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3aa00adf60 0x7f3aa00ae3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39576/0 (socket says 172.21.15.159:39576) 2024-04-04T06:15:28.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.177+0000 7f3a9e800640 1 -- 172.21.15.159:0/293948300 learned_addr learned my addr 172.21.15.159:0/293948300 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:28.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.176+0000 7f629ca00640 1 -- 172.21.15.159:0/673130345 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2765012291 0 0) 0x7f62a00d23b0 con 0x7f62a00d3d50 2024-04-04T06:15:28.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.177+0000 7f629ca00640 1 -- 172.21.15.159:0/673130345 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6284003520 con 0x7f62a00d3d50 2024-04-04T06:15:28.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.177+0000 7f629e800640 1 -- 172.21.15.159:0/673130345 >> v1:172.21.15.178:6789/0 conn(0x7f62a015e420 legacy=0x7f62a00b9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.177+0000 7f629e800640 1 -- 172.21.15.159:0/673130345 >> v1:172.21.15.159:6789/0 conn(0x7f62a00d3d50 legacy=0x7f62a00d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.177+0000 7f629e800640 1 -- 172.21.15.159:0/673130345 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a00d2970 con 0x7f62a00b6310 2024-04-04T06:15:28.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.177+0000 7f3a97e00640 1 --2- 172.21.15.159:0/293948300 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3aa009b3e0 0x7f3aa009b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.177+0000 7f3a9e800640 1 -- 172.21.15.159:0/293948300 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3aa009b3e0 msgr2=0x7f3aa009b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.177+0000 7f3a9e800640 1 --2- 172.21.15.159:0/293948300 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3aa009b3e0 0x7f3aa009b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.177+0000 7f3a9e800640 1 -- 172.21.15.159:0/293948300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a84004800 msgr2=0x7f3aa0141d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.177+0000 7f3a9e800640 1 --2- 172.21.15.159:0/293948300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a84004800 0x7f3aa0141d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.177+0000 7f3a9e800640 1 -- 172.21.15.159:0/293948300 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa009bef0 con 0x7f3aa00adf60 2024-04-04T06:15:28.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.177+0000 7f3a9e800640 1 --2- 172.21.15.159:0/293948300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3aa00adf60 0x7f3aa00ae3e0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800db50 tx=0x7f3a9800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:28.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.178+0000 7f3a96000640 1 -- 172.21.15.159:0/293948300 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98004d60 con 0x7f3aa00adf60 2024-04-04T06:15:28.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.178+0000 7f3a96000640 1 -- 172.21.15.159:0/293948300 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a9800bb50 con 0x7f3aa00adf60 2024-04-04T06:15:28.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.178+0000 7f3a96000640 1 -- 172.21.15.159:0/293948300 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98007070 con 0x7f3aa00adf60 2024-04-04T06:15:28.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.178+0000 7f3aa5000640 1 -- 172.21.15.159:0/293948300 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa009c1d0 con 0x7f3aa00adf60 2024-04-04T06:15:28.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.178+0000 7f3aa5000640 1 -- 172.21.15.159:0/293948300 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa0142250 con 0x7f3aa00adf60 2024-04-04T06:15:28.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.178+0000 7f629e800640 1 --2- 172.21.15.159:0/673130345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62a00b6310 0x7f62a00b6740 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f62a00d3140 tx=0x7f629802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5216315115a80f36 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:28.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.179+0000 7f3aa5000640 1 -- 172.21.15.159:0/293948300 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f3aa00ae920 con 0x7f3aa00adf60 2024-04-04T06:15:28.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.179+0000 7f629ca00640 1 -- 172.21.15.159:0/673130345 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298039070 con 0x7f62a00b6310 2024-04-04T06:15:28.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.179+0000 7f629ca00640 1 -- 172.21.15.159:0/673130345 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f629802cbc0 con 0x7f62a00b6310 2024-04-04T06:15:28.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.179+0000 7f629ca00640 1 -- 172.21.15.159:0/673130345 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298034440 con 0x7f62a00b6310 2024-04-04T06:15:28.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.180+0000 7f629f200640 1 -- 172.21.15.159:0/673130345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62a00b6310 msgr2=0x7f62a00b6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.180+0000 7f629f200640 1 --2- 172.21.15.159:0/673130345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62a00b6310 0x7f62a00b6740 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f62a00d3140 tx=0x7f629802be10 comp rx=0 tx=0).stop 2024-04-04T06:15:28.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.180+0000 7f629f200640 1 -- 172.21.15.159:0/673130345 shutdown_connections 2024-04-04T06:15:28.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.180+0000 7f629f200640 1 --2- 172.21.15.159:0/673130345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62a00b6310 0x7f62a00b6740 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.180+0000 7f629f200640 1 -- 172.21.15.159:0/673130345 >> 172.21.15.159:0/673130345 conn(0x7f62a00676a0 msgr2=0x7f62a0067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:28.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.180+0000 7f629f200640 1 -- 172.21.15.159:0/673130345 shutdown_connections 2024-04-04T06:15:28.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.180+0000 7f629f200640 1 -- 172.21.15.159:0/673130345 wait complete. 2024-04-04T06:15:28.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.180+0000 7f3a96000640 1 -- 172.21.15.159:0/293948300 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a9801b020 con 0x7f3aa00adf60 2024-04-04T06:15:28.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.181+0000 7f3a96000640 1 --2- 172.21.15.159:0/293948300 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3a7c06c8f0 0x7f3a7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.181+0000 7f3a96000640 1 -- 172.21.15.159:0/293948300 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f3a9808c2a0 con 0x7f3aa00adf60 2024-04-04T06:15:28.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.181+0000 7f3a96000640 1 -- 172.21.15.159:0/293948300 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+512 (secure 0 0 0) 0x7f3a98022030 con 0x7f3aa00adf60 2024-04-04T06:15:28.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.182+0000 7f3aa5000640 1 -- 172.21.15.159:0/293948300 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f3aa00685c0 con 0x7f3aa00adf60 2024-04-04T06:15:28.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.182+0000 7f3a9de00640 1 --2- 172.21.15.159:0/293948300 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3a7c06c8f0 0x7f3a7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.182+0000 7f629f200640 1 Processor -- start 2024-04-04T06:15:28.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.182+0000 7f629f200640 1 -- start start 2024-04-04T06:15:28.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.183+0000 7f3a9de00640 1 --2- 172.21.15.159:0/293948300 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3a7c06c8f0 0x7f3a7c06edb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3a90000f60 tx=0x7f3a90009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:28.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.183+0000 7f629f200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62a00b6310 0x7f62a014b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.183+0000 7f629f200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62a00d3d50 0x7f62a00c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.183+0000 7f629f200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f62a015e420 0x7f62a01597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.183+0000 7f629f200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f62a015e800 con 0x7f62a00d3d50 2024-04-04T06:15:28.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.183+0000 7f629f200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f62a00b6c80 con 0x7f62a00b6310 2024-04-04T06:15:28.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.183+0000 7f629f200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f62a00b7460 con 0x7f62a015e420 2024-04-04T06:15:28.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.183+0000 7f629d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62a00d3d50 0x7f62a00c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.183+0000 7f629d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62a00d3d50 0x7f62a00c9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39584/0 (socket says 172.21.15.159:39584) 2024-04-04T06:15:28.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.183+0000 7f629d400640 1 -- 172.21.15.159:0/3081536533 learned_addr learned my addr 172.21.15.159:0/3081536533 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:28.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.184+0000 7f629e800640 1 --2- 172.21.15.159:0/3081536533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f62a015e420 0x7f62a01597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.184+0000 7f629d400640 1 -- 172.21.15.159:0/3081536533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f62a015e420 msgr2=0x7f62a01597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.184+0000 7f629d400640 1 --2- 172.21.15.159:0/3081536533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f62a015e420 0x7f62a01597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.184+0000 7f629d400640 1 -- 172.21.15.159:0/3081536533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62a00b6310 msgr2=0x7f62a014b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.184+0000 7f629d400640 1 --2- 172.21.15.159:0/3081536533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62a00b6310 0x7f62a014b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.184+0000 7f629d400640 1 -- 172.21.15.159:0/3081536533 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a0158470 con 0x7f62a00d3d50 2024-04-04T06:15:28.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.184+0000 7f629d400640 1 --2- 172.21.15.159:0/3081536533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62a00d3d50 0x7f62a00c9540 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f628c00c550 tx=0x7f628c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:28.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.184+0000 7f6296000640 1 -- 172.21.15.159:0/3081536533 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c018070 con 0x7f62a00d3d50 2024-04-04T06:15:28.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.184+0000 7f6296000640 1 -- 172.21.15.159:0/3081536533 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f628c0074e0 con 0x7f62a00d3d50 2024-04-04T06:15:28.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.184+0000 7f6296000640 1 -- 172.21.15.159:0/3081536533 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c004a20 con 0x7f62a00d3d50 2024-04-04T06:15:28.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.184+0000 7f629f200640 1 -- 172.21.15.159:0/3081536533 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a0158750 con 0x7f62a00d3d50 2024-04-04T06:15:28.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.184+0000 7f629f200640 1 -- 172.21.15.159:0/3081536533 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a0158d10 con 0x7f62a00d3d50 2024-04-04T06:15:28.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.186+0000 7f6296000640 1 -- 172.21.15.159:0/3081536533 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f628c004bc0 con 0x7f62a00d3d50 2024-04-04T06:15:28.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.187+0000 7f6296000640 1 --2- 172.21.15.159:0/3081536533 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f628006c8f0 0x7f628006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.187+0000 7f6296000640 1 -- 172.21.15.159:0/3081536533 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f628c08afb0 con 0x7f62a00d3d50 2024-04-04T06:15:28.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.188+0000 7f629de00640 1 --2- 172.21.15.159:0/3081536533 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f628006c8f0 0x7f628006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.188+0000 7f62a7e70740 1 Processor -- start 2024-04-04T06:15:28.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.189+0000 7f62a7e70740 1 -- start start 2024-04-04T06:15:28.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.189+0000 7f629de00640 1 --2- 172.21.15.159:0/3081536533 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f628006c8f0 0x7f628006edb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6290006fd0 tx=0x7f6290006bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:28.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.189+0000 7f62a7e70740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559dbb5e180 0x5559dbb5e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.190+0000 7f62a7e70740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559dbb650a0 0x5559dbb63970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.190+0000 7f62a7e70740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb63260 0x5559dbb63eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.190+0000 7f62a7e70740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5559dbac8280 con 0x5559dbb63260 2024-04-04T06:15:28.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.190+0000 7f62a7e70740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5559dba706f0 con 0x5559dbb5e180 2024-04-04T06:15:28.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.190+0000 7f62a7e70740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5559dbac4230 con 0x5559dbb650a0 2024-04-04T06:15:28.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.190+0000 7f629e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb63260 0x5559dbb63eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.190+0000 7f629e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb63260 0x5559dbb63eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39596/0 (socket says 172.21.15.159:39596) 2024-04-04T06:15:28.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.190+0000 7f629e800640 1 -- 172.21.15.159:0/530537259 learned_addr learned my addr 172.21.15.159:0/530537259 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:28.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.190+0000 7f629de00640 1 --2- 172.21.15.159:0/530537259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559dbb5e180 0x5559dbb5e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.190+0000 7f629d400640 1 --2- 172.21.15.159:0/530537259 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559dbb650a0 0x5559dbb63970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.190+0000 7f629e800640 1 -- 172.21.15.159:0/530537259 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559dbb650a0 msgr2=0x5559dbb63970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.190+0000 7f629e800640 1 --2- 172.21.15.159:0/530537259 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559dbb650a0 0x5559dbb63970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.190+0000 7f629e800640 1 -- 172.21.15.159:0/530537259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559dbb5e180 msgr2=0x5559dbb5e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.190+0000 7f629e800640 1 --2- 172.21.15.159:0/530537259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559dbb5e180 0x5559dbb5e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.190+0000 7f629e800640 1 -- 172.21.15.159:0/530537259 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559db8a47e0 con 0x5559dbb63260 2024-04-04T06:15:28.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.191+0000 7f629e800640 1 --2- 172.21.15.159:0/530537259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb63260 0x5559dbb63eb0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f6298009e80 tx=0x7f629802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c4f2275adbf7d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:28.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.191+0000 7f628aa00640 1 -- 172.21.15.159:0/530537259 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298039040 con 0x5559dbb63260 2024-04-04T06:15:28.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.191+0000 7f628aa00640 1 -- 172.21.15.159:0/530537259 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f629802cbc0 con 0x5559dbb63260 2024-04-04T06:15:28.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.191+0000 7f628aa00640 1 -- 172.21.15.159:0/530537259 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629803f660 con 0x5559dbb63260 2024-04-04T06:15:28.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.192+0000 7f62a7e70740 1 -- 172.21.15.159:0/530537259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb63260 msgr2=0x5559dbb63eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.192+0000 7f62a7e70740 1 --2- 172.21.15.159:0/530537259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb63260 0x5559dbb63eb0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f6298009e80 tx=0x7f629802e000 comp rx=0 tx=0).stop 2024-04-04T06:15:28.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.192+0000 7f62a7e70740 1 -- 172.21.15.159:0/530537259 shutdown_connections 2024-04-04T06:15:28.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.192+0000 7f62a7e70740 1 --2- 172.21.15.159:0/530537259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb63260 0x5559dbb63eb0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.192+0000 7f62a7e70740 1 --2- 172.21.15.159:0/530537259 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559dbb650a0 0x5559dbb63970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.192+0000 7f62a7e70740 1 --2- 172.21.15.159:0/530537259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559dbb5e180 0x5559dbb5e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.192+0000 7f62a7e70740 1 -- 172.21.15.159:0/530537259 >> 172.21.15.159:0/530537259 conn(0x5559dbb60200 msgr2=0x5559dbb61ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:28.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.192+0000 7f62a7e70740 1 -- 172.21.15.159:0/530537259 shutdown_connections 2024-04-04T06:15:28.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.192+0000 7f62a7e70740 1 -- 172.21.15.159:0/530537259 wait complete. 2024-04-04T06:15:28.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.193+0000 7f62a7e70740 1 -- start start 2024-04-04T06:15:28.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.193+0000 7f62a7e70740 1 Processor -- start 2024-04-04T06:15:28.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.194+0000 7f62a7e70740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559dbb63260 0x5559dbbc2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.194+0000 7f62a7e70740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559dbb650a0 0x5559dbb8b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.194+0000 7f62a7e70740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb8d570 0x5559dbb8d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.194+0000 7f62a7e70740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5559dbb9faa0 con 0x5559dbb8d570 2024-04-04T06:15:28.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.194+0000 7f62a7e70740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5559dba83aa0 con 0x5559dbb650a0 2024-04-04T06:15:28.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.194+0000 7f62a7e70740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5559dbbb7000 con 0x5559dbb63260 2024-04-04T06:15:28.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.194+0000 7f62a7e70740 10 client.0 _ll_register_callbacks cb 0x5559dba83490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:15:28.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.194+0000 7f629e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb8d570 0x5559dbb8d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.194+0000 7f629e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb8d570 0x5559dbb8d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39612/0 (socket says 172.21.15.159:39612) 2024-04-04T06:15:28.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.194+0000 7f629e800640 1 -- 172.21.15.159:0/2553540510 learned_addr learned my addr 172.21.15.159:0/2553540510 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:28.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.195+0000 7f629e800640 1 -- 172.21.15.159:0/2553540510 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559dbb63260 msgr2=0x5559dbbc2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.195+0000 7f629e800640 1 --2- 172.21.15.159:0/2553540510 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559dbb63260 0x5559dbbc2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.195+0000 7f629e800640 1 -- 172.21.15.159:0/2553540510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559dbb650a0 msgr2=0x5559dbb8b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.195+0000 7f629d400640 1 --2- 172.21.15.159:0/2553540510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559dbb650a0 0x5559dbb8b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.195+0000 7f629e800640 1 --2- 172.21.15.159:0/2553540510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559dbb650a0 0x5559dbb8b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.195+0000 7f629e800640 1 -- 172.21.15.159:0/2553540510 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559db904f70 con 0x5559dbb8d570 2024-04-04T06:15:28.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.195+0000 7f629e800640 1 --2- 172.21.15.159:0/2553540510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb8d570 0x5559dbb8d9f0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f629802c420 tx=0x7f6298004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24551eed0f6ee013 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:28.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.195+0000 7f627f400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:15:28.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.195+0000 7f627f400640 1 -- 172.21.15.159:0/2553540510 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298039040 con 0x5559dbb8d570 2024-04-04T06:15:28.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.195+0000 7f62a7e70740 1 -- 172.21.15.159:0/2553540510 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f629802cbc0 con 0x5559dbb8d570 2024-04-04T06:15:28.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.195+0000 7f627f400640 1 -- 172.21.15.159:0/2553540510 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f6298003ba0 con 0x5559dbb8d570 2024-04-04T06:15:28.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.196+0000 7f627f400640 1 -- 172.21.15.159:0/2553540510 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f6298003da0 con 0x5559dbb8d570 2024-04-04T06:15:28.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.196+0000 7f62a7e70740 10 client.8481 fetch_fsmap learned FSMap version 84 2024-04-04T06:15:28.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.196+0000 7f62a7e70740 1 -- 172.21.15.159:0/2553540510 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x5559db79c170 con 0x5559dbb8d570 2024-04-04T06:15:28.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.196+0000 7f627f400640 1 -- 172.21.15.159:0/2553540510 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f629802cbc0 con 0x5559dbb8d570 2024-04-04T06:15:28.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.196+0000 7f627f400640 1 -- 172.21.15.159:0/2553540510 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7f629803f980 con 0x5559dbb8d570 2024-04-04T06:15:28.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.196+0000 7f62a7e70740 10 client.8481 fetch_fsmap finished waiting for FSMap version 84 2024-04-04T06:15:28.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.196+0000 7f62a7e70740 10 client.8481 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-04T06:15:28.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.196+0000 7f62a7e70740 20 client.8481 populate_metadata read hostname 'smithi159' 2024-04-04T06:15:28.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.196+0000 7f62a7e70740 1 --2- 172.21.15.159:0/2553540510 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x5559dbb8c0b0 0x5559dbb8c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.196+0000 7f62a7e70740 4 client.8481 mds_command: new command op to 8190 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:15:28.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.196+0000 7f62a7e70740 1 -- 172.21.15.159:0/2553540510 --> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559dbad57c0 con 0x5559dbb8c0b0 2024-04-04T06:15:28.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.196+0000 7f629de00640 1 --2- 172.21.15.159:0/2553540510 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x5559dbb8c0b0 0x5559dbb8c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.197+0000 7f629de00640 1 --2- 172.21.15.159:0/2553540510 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x5559dbb8c0b0 0x5559dbb8c4c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae8dc8be82b842ce server_cookie=ed8c5dc97e94fe8c in_seq=0 out_seq=0 2024-04-04T06:15:28.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.197+0000 7f627f400640 10 client.8481 ms_handle_connect on v2:172.21.15.159:6834/2253383212 2024-04-04T06:15:28.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.198+0000 7f627f400640 1 -- 172.21.15.159:0/2553540510 <== mds.0 v2:172.21.15.159:6834/2253383212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5559dbad57c0 con 0x5559dbb8c0b0 2024-04-04T06:15:28.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.198+0000 7f627f400640 10 client.8481 handle_command_reply: tid=0 2024-04-04T06:15:28.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.198+0000 7f62a7e70740 1 client.8481 shutdown 2024-04-04T06:15:28.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.198+0000 7f62a7e70740 1 -- 172.21.15.159:0/2553540510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb8d570 msgr2=0x5559dbb8d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.198+0000 7f62a7e70740 1 --2- 172.21.15.159:0/2553540510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb8d570 0x5559dbb8d9f0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f629802c420 tx=0x7f6298004ab0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.198+0000 7f62a7e70740 1 -- 172.21.15.159:0/2553540510 shutdown_connections 2024-04-04T06:15:28.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.198+0000 7f62a7e70740 1 --2- 172.21.15.159:0/2553540510 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x5559dbb8c0b0 0x5559dbb8c4c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.198+0000 7f62a7e70740 1 --2- 172.21.15.159:0/2553540510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb8d570 0x5559dbb8d9f0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.198+0000 7f62a7e70740 1 --2- 172.21.15.159:0/2553540510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559dbb650a0 0x5559dbb8b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.198+0000 7f62a7e70740 1 --2- 172.21.15.159:0/2553540510 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559dbb63260 0x5559dbbc2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.198+0000 7f62a7e70740 1 -- 172.21.15.159:0/2553540510 >> 172.21.15.159:0/2553540510 conn(0x5559dbb5e180 msgr2=0x5559dbbac570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:28.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.198+0000 7f627f400640 0 client.8481 ms_handle_reset on v2:172.21.15.159:6834/2253383212 2024-04-04T06:15:28.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.198+0000 7f62a7e70740 1 -- 172.21.15.159:0/2553540510 shutdown_connections 2024-04-04T06:15:28.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.198+0000 7f62a7e70740 1 -- 172.21.15.159:0/2553540510 wait complete. 2024-04-04T06:15:28.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.198+0000 7f62a7e70740 20 client.8481 trim_cache size 0 max 16384 2024-04-04T06:15:28.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.217+0000 7f62a7e70740 1 Processor -- start 2024-04-04T06:15:28.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.217+0000 7f62a7e70740 1 -- start start 2024-04-04T06:15:28.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.218+0000 7f62a7e70740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559dbb63260 0x5559dbb9ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.218+0000 7f62a7e70740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559dbb650a0 0x5559dbb916f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.218+0000 7f62a7e70740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb8d570 0x5559dbb91c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.218+0000 7f62a7e70740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f62a0070ab0 con 0x5559dbb8d570 2024-04-04T06:15:28.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.218+0000 7f62a7e70740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f62a006faf0 con 0x5559dbb650a0 2024-04-04T06:15:28.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.218+0000 7f62a7e70740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f62a00c7fe0 con 0x5559dbb63260 2024-04-04T06:15:28.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.218+0000 7f629e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb8d570 0x5559dbb91c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.219+0000 7f629e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb8d570 0x5559dbb91c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39628/0 (socket says 172.21.15.159:39628) 2024-04-04T06:15:28.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.219+0000 7f629e800640 1 -- 172.21.15.159:0/564761433 learned_addr learned my addr 172.21.15.159:0/564761433 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:28.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.219+0000 7f629e800640 1 -- 172.21.15.159:0/564761433 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559dbb63260 msgr2=0x5559dbb9ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.219+0000 7f629e800640 1 --2- 172.21.15.159:0/564761433 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559dbb63260 0x5559dbb9ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.219+0000 7f629e800640 1 -- 172.21.15.159:0/564761433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559dbb650a0 msgr2=0x5559dbb916f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.219+0000 7f629e800640 1 --2- 172.21.15.159:0/564761433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559dbb650a0 0x5559dbb916f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.219+0000 7f629e800640 1 -- 172.21.15.159:0/564761433 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559dbbb7000 con 0x5559dbb8d570 2024-04-04T06:15:28.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.219+0000 7f629e800640 1 --2- 172.21.15.159:0/564761433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb8d570 0x5559dbb91c30 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f6298003d30 tx=0x7f6298049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421b42090eacfc02 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:28.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.219+0000 7f628a000640 1 -- 172.21.15.159:0/564761433 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629805f070 con 0x5559dbb8d570 2024-04-04T06:15:28.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.219+0000 7f628a000640 1 -- 172.21.15.159:0/564761433 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f629802cac0 con 0x5559dbb8d570 2024-04-04T06:15:28.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.220+0000 7f628a000640 1 -- 172.21.15.159:0/564761433 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298039040 con 0x5559dbb8d570 2024-04-04T06:15:28.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.220+0000 7f62a7e70740 1 -- 172.21.15.159:0/564761433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb8d570 msgr2=0x5559dbb91c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.220+0000 7f62a7e70740 1 --2- 172.21.15.159:0/564761433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb8d570 0x5559dbb91c30 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f6298003d30 tx=0x7f6298049000 comp rx=0 tx=0).stop 2024-04-04T06:15:28.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.220+0000 7f62a7e70740 1 -- 172.21.15.159:0/564761433 shutdown_connections 2024-04-04T06:15:28.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.220+0000 7f62a7e70740 1 --2- 172.21.15.159:0/564761433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb8d570 0x5559dbb91c30 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.220+0000 7f62a7e70740 1 --2- 172.21.15.159:0/564761433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559dbb650a0 0x5559dbb916f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.220+0000 7f62a7e70740 1 --2- 172.21.15.159:0/564761433 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559dbb63260 0x5559dbb9ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.220+0000 7f62a7e70740 1 -- 172.21.15.159:0/564761433 >> 172.21.15.159:0/564761433 conn(0x5559dbb5e180 msgr2=0x5559dbb8af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:28.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.221+0000 7f62a7e70740 1 -- 172.21.15.159:0/564761433 shutdown_connections 2024-04-04T06:15:28.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.221+0000 7f62a7e70740 1 -- 172.21.15.159:0/564761433 wait complete. 2024-04-04T06:15:28.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.221+0000 7f62a7e70740 1 -- start start 2024-04-04T06:15:28.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.221+0000 7f62a7e70740 1 Processor -- start 2024-04-04T06:15:28.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.222+0000 7f62a7e70740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb63260 0x5559dbc2bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.222+0000 7f62a7e70740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559dbb650a0 0x5559dbbac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.222+0000 7f62a7e70740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559dbb8d570 0x5559dbbc22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.222+0000 7f62a7e70740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5559dbb59f50 con 0x5559dbb63260 2024-04-04T06:15:28.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.222+0000 7f62a7e70740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f62a00c8630 con 0x5559dbb8d570 2024-04-04T06:15:28.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.222+0000 7f62a7e70740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5559dbba9600 con 0x5559dbb650a0 2024-04-04T06:15:28.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.222+0000 7f62a7e70740 10 client.0 _ll_register_callbacks cb 0x7f62840036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:15:28.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.223+0000 7f629de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb63260 0x5559dbc2bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.223+0000 7f629de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb63260 0x5559dbc2bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39634/0 (socket says 172.21.15.159:39634) 2024-04-04T06:15:28.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.223+0000 7f629de00640 1 -- 172.21.15.159:0/25043401 learned_addr learned my addr 172.21.15.159:0/25043401 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:28.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.223+0000 7f629d400640 1 --2- 172.21.15.159:0/25043401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559dbb650a0 0x5559dbbac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.223+0000 7f629e800640 1 --2- 172.21.15.159:0/25043401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559dbb8d570 0x5559dbbc22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.223+0000 7f629de00640 1 -- 172.21.15.159:0/25043401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559dbb650a0 msgr2=0x5559dbbac4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.223+0000 7f629de00640 1 --2- 172.21.15.159:0/25043401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559dbb650a0 0x5559dbbac4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.223+0000 7f629de00640 1 -- 172.21.15.159:0/25043401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559dbb8d570 msgr2=0x5559dbbc22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.223+0000 7f629de00640 1 --2- 172.21.15.159:0/25043401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559dbb8d570 0x5559dbbc22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.223+0000 7f629de00640 1 -- 172.21.15.159:0/25043401 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559dbb5dc30 con 0x5559dbb63260 2024-04-04T06:15:28.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.223+0000 7f629de00640 1 --2- 172.21.15.159:0/25043401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb63260 0x5559dbc2bea0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f6290013970 tx=0x7f62900046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc81d5d6edf8aa4b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:28.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.223+0000 7f627f400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:15:28.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.224+0000 7f627f400640 1 -- 172.21.15.159:0/25043401 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6290012ca0 con 0x5559dbb63260 2024-04-04T06:15:28.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.224+0000 7f627f400640 1 -- 172.21.15.159:0/25043401 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f62900230d0 con 0x5559dbb63260 2024-04-04T06:15:28.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.224+0000 7f62a7e70740 1 -- 172.21.15.159:0/25043401 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f629802cac0 con 0x5559dbb63260 2024-04-04T06:15:28.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.224+0000 7f627f400640 1 -- 172.21.15.159:0/25043401 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f6290032050 con 0x5559dbb63260 2024-04-04T06:15:28.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.224+0000 7f62a7e70740 10 client.8487 fetch_fsmap learned FSMap version 84 2024-04-04T06:15:28.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.224+0000 7f62a7e70740 1 -- 172.21.15.159:0/25043401 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x5559dbbacd60 con 0x5559dbb63260 2024-04-04T06:15:28.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.224+0000 7f627f400640 1 -- 172.21.15.159:0/25043401 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f629802cac0 con 0x5559dbb63260 2024-04-04T06:15:28.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.224+0000 7f627f400640 1 -- 172.21.15.159:0/25043401 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7f6290035020 con 0x5559dbb63260 2024-04-04T06:15:28.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.224+0000 7f62a7e70740 10 client.8487 fetch_fsmap finished waiting for FSMap version 84 2024-04-04T06:15:28.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.224+0000 7f62a7e70740 10 client.8487 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-04T06:15:28.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.224+0000 7f62a7e70740 20 client.8487 populate_metadata read hostname 'smithi159' 2024-04-04T06:15:28.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.224+0000 7f62a7e70740 1 --2- 172.21.15.159:0/25043401 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x5559dbbc0620 0x5559dbbc0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.224+0000 7f62a7e70740 4 client.8487 mds_command: new command op to 8190 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-04T06:15:28.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.224+0000 7f62a7e70740 1 -- 172.21.15.159:0/25043401 --> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x5559dbbc5360 con 0x5559dbbc0620 2024-04-04T06:15:28.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.225+0000 7f629d400640 1 --2- 172.21.15.159:0/25043401 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x5559dbbc0620 0x5559dbbc0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.225+0000 7f629d400640 1 --2- 172.21.15.159:0/25043401 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x5559dbbc0620 0x5559dbbc0a30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca4ca6beb37f0e90 server_cookie=cd79be42b19069fa in_seq=0 out_seq=0 2024-04-04T06:15:28.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.225+0000 7f627f400640 10 client.8487 ms_handle_connect on v2:172.21.15.159:6834/2253383212 2024-04-04T06:15:28.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.226+0000 7f627f400640 1 -- 172.21.15.159:0/25043401 <== mds.0 v2:172.21.15.159:6834/2253383212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5559dbbc5360 con 0x5559dbbc0620 2024-04-04T06:15:28.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.226+0000 7f627f400640 10 client.8487 handle_command_reply: tid=0 2024-04-04T06:15:28.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.226+0000 7f62a7e70740 1 client.8487 shutdown 2024-04-04T06:15:28.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.226+0000 7f62a7e70740 1 -- 172.21.15.159:0/25043401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb63260 msgr2=0x5559dbc2bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.226+0000 7f62a7e70740 1 --2- 172.21.15.159:0/25043401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb63260 0x5559dbc2bea0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f6290013970 tx=0x7f62900046f0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.226+0000 7f62a7e70740 1 -- 172.21.15.159:0/25043401 shutdown_connections 2024-04-04T06:15:28.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.226+0000 7f62a7e70740 1 --2- 172.21.15.159:0/25043401 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x5559dbbc0620 0x5559dbbc0a30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.226+0000 7f62a7e70740 1 --2- 172.21.15.159:0/25043401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559dbb8d570 0x5559dbbc22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.226+0000 7f62a7e70740 1 --2- 172.21.15.159:0/25043401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5559dbb650a0 0x5559dbbac4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.226+0000 7f62a7e70740 1 --2- 172.21.15.159:0/25043401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559dbb63260 0x5559dbc2bea0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.226+0000 7f62a7e70740 1 -- 172.21.15.159:0/25043401 >> 172.21.15.159:0/25043401 conn(0x5559dbb5e180 msgr2=0x5559dbbb74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:28.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.226+0000 7f627f400640 0 client.8487 ms_handle_reset on v2:172.21.15.159:6834/2253383212 2024-04-04T06:15:28.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.227+0000 7f62a7e70740 1 -- 172.21.15.159:0/25043401 shutdown_connections 2024-04-04T06:15:28.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.227+0000 7f62a7e70740 1 -- 172.21.15.159:0/25043401 wait complete. 2024-04-04T06:15:28.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.227+0000 7f62a7e70740 20 client.8487 trim_cache size 0 max 16384 2024-04-04T06:15:28.225 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:15:28.225 INFO:teuthology.orchestra.run.smithi159.stdout: "return_code": 0, 2024-04-04T06:15:28.225 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_tag": "9ba77e50-6c91-42a6-acc0-4c82a2673c3f", 2024-04-04T06:15:28.225 INFO:teuthology.orchestra.run.smithi159.stdout: "mode": "asynchronous" 2024-04-04T06:15:28.225 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:15:28.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.227+0000 7f629f200640 1 -- 172.21.15.159:0/3081536533 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f628006c8f0 msgr2=0x7f628006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.227+0000 7f629f200640 1 --2- 172.21.15.159:0/3081536533 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f628006c8f0 0x7f628006edb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6290006fd0 tx=0x7f6290006bc0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.227+0000 7f629f200640 1 -- 172.21.15.159:0/3081536533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62a00d3d50 msgr2=0x7f62a00c9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.227+0000 7f629f200640 1 --2- 172.21.15.159:0/3081536533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62a00d3d50 0x7f62a00c9540 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f628c00c550 tx=0x7f628c00ca10 comp rx=0 tx=0).stop 2024-04-04T06:15:28.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.228+0000 7f629f200640 1 -- 172.21.15.159:0/3081536533 shutdown_connections 2024-04-04T06:15:28.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.228+0000 7f629f200640 1 --2- 172.21.15.159:0/3081536533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f62a015e420 0x7f62a01597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.228+0000 7f629f200640 1 --2- 172.21.15.159:0/3081536533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62a00d3d50 0x7f62a00c9540 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.228+0000 7f629f200640 1 --2- 172.21.15.159:0/3081536533 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f628006c8f0 0x7f628006edb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.228+0000 7f629f200640 1 --2- 172.21.15.159:0/3081536533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62a00b6310 0x7f62a014b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.228+0000 7f629f200640 1 -- 172.21.15.159:0/3081536533 >> 172.21.15.159:0/3081536533 conn(0x7f62a00676a0 msgr2=0x7f62a00cecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:28.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.228+0000 7f629f200640 1 -- 172.21.15.159:0/3081536533 shutdown_connections 2024-04-04T06:15:28.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.228+0000 7f629f200640 1 -- 172.21.15.159:0/3081536533 wait complete. 2024-04-04T06:15:28.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:28.239 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-04T06:15:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.436+0000 7f5bc9000640 1 Processor -- start 2024-04-04T06:15:28.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.436+0000 7f5bc9000640 1 -- start start 2024-04-04T06:15:28.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.437+0000 7f5bc9000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bc4072100 0x7f5bc406f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.438+0000 7f5bc9000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5bc4162d90 0x7f5bc406efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.438+0000 7f5bc9000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bc4163270 con 0x7f5bc415e960 2024-04-04T06:15:28.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.438+0000 7f5bc9000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f5bc406e220 con 0x7f5bc4162d90 2024-04-04T06:15:28.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.438+0000 7f5bc9000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5bc406e500 con 0x7f5bc4072100 2024-04-04T06:15:28.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.438+0000 7f5bc1e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f5bc415e960 0x7f5bc40719f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:53580/0 (socket says 172.21.15.159:53580) 2024-04-04T06:15:28.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.438+0000 7f5bc1e00640 1 -- 172.21.15.159:0/295333772 learned_addr learned my addr 172.21.15.159:0/295333772 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:28.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.438+0000 7f5bc1400640 1 --2- 172.21.15.159:0/295333772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bc4072100 0x7f5bc406f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.438+0000 7f5bc2800640 1 --2- 172.21.15.159:0/295333772 >> v2:172.21.15.178:3300/0 conn(0x7f5bc4162d90 0x7f5bc406efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.438+0000 7f5bc0a00640 1 -- 172.21.15.159:0/295333772 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4008131114 0 0) 0x7f5bc4163270 con 0x7f5bc415e960 2024-04-04T06:15:28.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.438+0000 7f5bc0a00640 1 -- 172.21.15.159:0/295333772 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ba8003520 con 0x7f5bc415e960 2024-04-04T06:15:28.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.439+0000 7f5bc0a00640 1 -- 172.21.15.159:0/295333772 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3962520595 0 0) 0x7f5ba8003520 con 0x7f5bc415e960 2024-04-04T06:15:28.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.439+0000 7f5bc0a00640 1 -- 172.21.15.159:0/295333772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bc4072100 msgr2=0x7f5bc406f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.439+0000 7f5bc0a00640 1 --2- 172.21.15.159:0/295333772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bc4072100 0x7f5bc406f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.439+0000 7f5bc0a00640 1 -- 172.21.15.159:0/295333772 >> v2:172.21.15.178:3300/0 conn(0x7f5bc4162d90 msgr2=0x7f5bc406efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.439+0000 7f5bc0a00640 1 --2- 172.21.15.159:0/295333772 >> v2:172.21.15.178:3300/0 conn(0x7f5bc4162d90 0x7f5bc406efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.439+0000 7f5bc0a00640 1 -- 172.21.15.159:0/295333772 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc406e7e0 con 0x7f5bc415e960 2024-04-04T06:15:28.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.439+0000 7f5bc0a00640 1 -- 172.21.15.159:0/295333772 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f5bb40019f0 con 0x7f5bc415e960 2024-04-04T06:15:28.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.439+0000 7f5bc0a00640 1 -- 172.21.15.159:0/295333772 >> v1:172.21.15.159:6789/0 conn(0x7f5bc415e960 legacy=0x7f5bc40719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.439+0000 7f5bc0a00640 1 --2- 172.21.15.159:0/295333772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bc4072100 0x7f5ba80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.439+0000 7f5bc0a00640 1 --2- 172.21.15.159:0/295333772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ba8003f30 0x7f5ba8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.439+0000 7f5bc0a00640 1 --2- 172.21.15.159:0/295333772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ba8004d40 0x7f5ba80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.439+0000 7f5bc0a00640 1 -- 172.21.15.159:0/295333772 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc406e220 con 0x7f5ba8004d40 2024-04-04T06:15:28.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.439+0000 7f5bc0a00640 1 -- 172.21.15.159:0/295333772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc406e500 con 0x7f5ba8003f30 2024-04-04T06:15:28.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.439+0000 7f5bc0a00640 1 -- 172.21.15.159:0/295333772 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba800ba10 con 0x7f5bc4072100 2024-04-04T06:15:28.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.439+0000 7f5bc2800640 1 --2- 172.21.15.159:0/295333772 >> v2:172.21.15.178:3300/0 conn(0x7f5bc4162d90 0x7f5bc406efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:15:28.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.439+0000 7f5bc2800640 1 --2- 172.21.15.159:0/295333772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ba8004d40 0x7f5ba80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.440+0000 7f5bc1e00640 1 --2- 172.21.15.159:0/295333772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bc4072100 0x7f5ba80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.440+0000 7f5bc1400640 1 --2- 172.21.15.159:0/295333772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ba8003f30 0x7f5ba8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.440+0000 7f5bc2800640 1 -- 172.21.15.159:0/295333772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bc4072100 msgr2=0x7f5ba80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.440+0000 7f5bc2800640 1 --2- 172.21.15.159:0/295333772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bc4072100 0x7f5ba80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.440+0000 7f5bc2800640 1 -- 172.21.15.159:0/295333772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ba8003f30 msgr2=0x7f5ba8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.440+0000 7f5bc2800640 1 --2- 172.21.15.159:0/295333772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ba8003f30 0x7f5ba8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.440+0000 7f5bc2800640 1 -- 172.21.15.159:0/295333772 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba8003520 con 0x7f5ba8004d40 2024-04-04T06:15:28.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.440+0000 7f5bc1e00640 1 --2- 172.21.15.159:0/295333772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bc4072100 0x7f5ba80039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:28.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.440+0000 7f5bc1400640 1 --2- 172.21.15.159:0/295333772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ba8003f30 0x7f5ba8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:28.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.441+0000 7f5bc2800640 1 --2- 172.21.15.159:0/295333772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ba8004d40 0x7f5ba80051c0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f5bc41631a0 tx=0x7f5ba8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7e626e26e91c87 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:28.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.441+0000 7f5bc0a00640 1 -- 172.21.15.159:0/295333772 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc00f040 con 0x7f5ba8004d40 2024-04-04T06:15:28.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.441+0000 7f5bc0a00640 1 -- 172.21.15.159:0/295333772 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bbc004230 con 0x7f5ba8004d40 2024-04-04T06:15:28.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.441+0000 7f5bc0a00640 1 -- 172.21.15.159:0/295333772 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc003610 con 0x7f5ba8004d40 2024-04-04T06:15:28.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.442+0000 7f5bc9000640 1 -- 172.21.15.159:0/295333772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ba8004d40 msgr2=0x7f5ba80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.442+0000 7f5bc9000640 1 --2- 172.21.15.159:0/295333772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ba8004d40 0x7f5ba80051c0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f5bc41631a0 tx=0x7f5ba8005830 comp rx=0 tx=0).stop 2024-04-04T06:15:28.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.442+0000 7f5bc2800640 1 -- 172.21.15.159:0/295333772 reap_dead start 2024-04-04T06:15:28.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.442+0000 7f5bc9000640 1 -- 172.21.15.159:0/295333772 shutdown_connections 2024-04-04T06:15:28.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.442+0000 7f5bc9000640 1 -- 172.21.15.159:0/295333772 >> 172.21.15.159:0/295333772 conn(0x7f5bc406bba0 msgr2=0x7f5bc40b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:28.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.442+0000 7f5bc9000640 1 -- 172.21.15.159:0/295333772 shutdown_connections 2024-04-04T06:15:28.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.442+0000 7f5bc9000640 1 -- 172.21.15.159:0/295333772 wait complete. 2024-04-04T06:15:28.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.444+0000 7f5bc9000640 1 Processor -- start 2024-04-04T06:15:28.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.444+0000 7f5bc9000640 1 -- start start 2024-04-04T06:15:28.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.445+0000 7f5bc9000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ba8004d40 0x7f5bc4152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.445+0000 7f5bc9000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bc414f120 0x7f5bc40b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.445+0000 7f5bc9000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bc415c330 0x7f5bc40b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.445+0000 7f5bc9000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc406f860 con 0x7f5bc414f120 2024-04-04T06:15:28.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.445+0000 7f5bc9000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc40701f0 con 0x7f5ba8004d40 2024-04-04T06:15:28.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.445+0000 7f5bc9000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5bc406f9e0 con 0x7f5bc415c330 2024-04-04T06:15:28.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.445+0000 7f5bc1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bc414f120 0x7f5bc40b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.445+0000 7f5bc1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bc414f120 0x7f5bc40b1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39648/0 (socket says 172.21.15.159:39648) 2024-04-04T06:15:28.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.445+0000 7f5bc1400640 1 -- 172.21.15.159:0/411305746 learned_addr learned my addr 172.21.15.159:0/411305746 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:28.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.445+0000 7f5bc1e00640 1 --2- 172.21.15.159:0/411305746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ba8004d40 0x7f5bc4152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.446+0000 7f5bc1400640 1 -- 172.21.15.159:0/411305746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bc415c330 msgr2=0x7f5bc40b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.446+0000 7f5bc1400640 1 --2- 172.21.15.159:0/411305746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bc415c330 0x7f5bc40b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.446+0000 7f5bc1400640 1 -- 172.21.15.159:0/411305746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ba8004d40 msgr2=0x7f5bc4152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.446+0000 7f5bc1400640 1 --2- 172.21.15.159:0/411305746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ba8004d40 0x7f5bc4152d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.446+0000 7f5bc1400640 1 -- 172.21.15.159:0/411305746 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc415fec0 con 0x7f5bc414f120 2024-04-04T06:15:28.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.446+0000 7f5bc1400640 1 --2- 172.21.15.159:0/411305746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bc414f120 0x7f5bc40b1220 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0010830 tx=0x7f5bb0010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:28.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.446+0000 7f5bba000640 1 -- 172.21.15.159:0/411305746 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb000f040 con 0x7f5bc414f120 2024-04-04T06:15:28.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.446+0000 7f5bba000640 1 -- 172.21.15.159:0/411305746 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb000eb60 con 0x7f5bc414f120 2024-04-04T06:15:28.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.446+0000 7f5bba000640 1 -- 172.21.15.159:0/411305746 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb001c400 con 0x7f5bc414f120 2024-04-04T06:15:28.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.446+0000 7f5bc9000640 1 -- 172.21.15.159:0/411305746 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc41601a0 con 0x7f5bc414f120 2024-04-04T06:15:28.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.446+0000 7f5bc9000640 1 -- 172.21.15.159:0/411305746 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc4160760 con 0x7f5bc414f120 2024-04-04T06:15:28.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.450+0000 7f5bba000640 1 -- 172.21.15.159:0/411305746 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5bb001c5a0 con 0x7f5bc414f120 2024-04-04T06:15:28.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.450+0000 7f5bcbdaa740 1 Processor -- start 2024-04-04T06:15:28.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.450+0000 7f5bcbdaa740 1 -- start start 2024-04-04T06:15:28.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.450+0000 7f5bba000640 1 --2- 172.21.15.159:0/411305746 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5ba406c8f0 0x7f5ba406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.451+0000 7f5bba000640 1 -- 172.21.15.159:0/411305746 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f5bb008c190 con 0x7f5bc414f120 2024-04-04T06:15:28.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.451+0000 7f5bc2800640 1 --2- 172.21.15.159:0/411305746 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5ba406c8f0 0x7f5ba406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.451+0000 7f5bcbdaa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfb580 0x564697cce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.451+0000 7f5bcbdaa740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564697cfb960 0x564697cfe310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.451+0000 7f5bcbdaa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697ccea60 0x564697cfe850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.451+0000 7f5bcbdaa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564697c396d0 con 0x564697cfb580 2024-04-04T06:15:28.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.451+0000 7f5bcbdaa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564697be1770 con 0x564697ccea60 2024-04-04T06:15:28.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.451+0000 7f5bcbdaa740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x564697c35680 con 0x564697cfb960 2024-04-04T06:15:28.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.451+0000 7f5bc1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfb580 0x564697cce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.452+0000 7f5bc1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfb580 0x564697cce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39654/0 (socket says 172.21.15.159:39654) 2024-04-04T06:15:28.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.452+0000 7f5bc1e00640 1 -- 172.21.15.159:0/2360660000 learned_addr learned my addr 172.21.15.159:0/2360660000 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:28.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.452+0000 7f5bc1e00640 1 -- 172.21.15.159:0/2360660000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564697cfb960 msgr2=0x564697cfe310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.452+0000 7f5bc1e00640 1 --2- 172.21.15.159:0/2360660000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564697cfb960 0x564697cfe310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.452+0000 7f5bc2800640 1 --2- 172.21.15.159:0/2360660000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697ccea60 0x564697cfe850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.452+0000 7f5bc1e00640 1 -- 172.21.15.159:0/2360660000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697ccea60 msgr2=0x564697cfe850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.452+0000 7f5bc1e00640 1 --2- 172.21.15.159:0/2360660000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697ccea60 0x564697cfe850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.452+0000 7f5bc1e00640 1 -- 172.21.15.159:0/2360660000 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564697a177d0 con 0x564697cfb580 2024-04-04T06:15:28.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.452+0000 7f5bc1e00640 1 --2- 172.21.15.159:0/2360660000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfb580 0x564697cce520 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400e360 tx=0x7f5bb400e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f9d19a727b18e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:28.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.452+0000 7f5bc2800640 1 --2- 172.21.15.159:0/411305746 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5ba406c8f0 0x7f5ba406edb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc035000 tx=0x7f5bbc0046c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:28.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.452+0000 7f5baea00640 1 -- 172.21.15.159:0/2360660000 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4019070 con 0x564697cfb580 2024-04-04T06:15:28.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.453+0000 7f5baea00640 1 -- 172.21.15.159:0/2360660000 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb40070d0 con 0x564697cfb580 2024-04-04T06:15:28.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.453+0000 7f5baea00640 1 -- 172.21.15.159:0/2360660000 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4007480 con 0x564697cfb580 2024-04-04T06:15:28.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.453+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/2360660000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfb580 msgr2=0x564697cce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.453+0000 7f5bcbdaa740 1 --2- 172.21.15.159:0/2360660000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfb580 0x564697cce520 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400e360 tx=0x7f5bb400e820 comp rx=0 tx=0).stop 2024-04-04T06:15:28.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.453+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/2360660000 shutdown_connections 2024-04-04T06:15:28.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.453+0000 7f5bcbdaa740 1 --2- 172.21.15.159:0/2360660000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697ccea60 0x564697cfe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.453+0000 7f5bcbdaa740 1 --2- 172.21.15.159:0/2360660000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564697cfb960 0x564697cfe310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.453+0000 7f5bcbdaa740 1 --2- 172.21.15.159:0/2360660000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfb580 0x564697cce520 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.453+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/2360660000 >> 172.21.15.159:0/2360660000 conn(0x564697cd0790 msgr2=0x564697d19310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:28.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.453+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/2360660000 shutdown_connections 2024-04-04T06:15:28.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.453+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/2360660000 wait complete. 2024-04-04T06:15:28.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.455+0000 7f5bcbdaa740 1 -- start start 2024-04-04T06:15:28.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.455+0000 7f5bcbdaa740 1 Processor -- start 2024-04-04T06:15:28.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.456+0000 7f5bcbdaa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfb580 0x564697ceb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.457+0000 7f5bcbdaa740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564697cfb960 0x564697d199e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.457+0000 7f5bcbdaa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697d00c80 0x564697ce8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.457+0000 7f5bcbdaa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564697cd2e30 con 0x564697cfb580 2024-04-04T06:15:28.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.457+0000 7f5bcbdaa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564697bf4aa0 con 0x564697d00c80 2024-04-04T06:15:28.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.457+0000 7f5bcbdaa740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x564697cd5670 con 0x564697cfb960 2024-04-04T06:15:28.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.457+0000 7f5bcbdaa740 10 client.0 _ll_register_callbacks cb 0x564697bf4490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:15:28.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.457+0000 7f5bc1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfb580 0x564697ceb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.457+0000 7f5bc1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfb580 0x564697ceb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39660/0 (socket says 172.21.15.159:39660) 2024-04-04T06:15:28.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.457+0000 7f5bc1e00640 1 -- 172.21.15.159:0/1015347382 learned_addr learned my addr 172.21.15.159:0/1015347382 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:28.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.457+0000 7f5bc1400640 1 --2- 172.21.15.159:0/1015347382 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564697cfb960 0x564697d199e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.457+0000 7f5bc2800640 1 --2- 172.21.15.159:0/1015347382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697d00c80 0x564697ce8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.457+0000 7f5bc1e00640 1 -- 172.21.15.159:0/1015347382 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564697cfb960 msgr2=0x564697d199e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.457+0000 7f5bc1e00640 1 --2- 172.21.15.159:0/1015347382 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564697cfb960 0x564697d199e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.457+0000 7f5bc1e00640 1 -- 172.21.15.159:0/1015347382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697d00c80 msgr2=0x564697ce8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.457+0000 7f5bc1e00640 1 --2- 172.21.15.159:0/1015347382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697d00c80 0x564697ce8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.457+0000 7f5bc1e00640 1 -- 172.21.15.159:0/1015347382 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56469794d3f0 con 0x564697cfb580 2024-04-04T06:15:28.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.457+0000 7f5bc2800640 1 --2- 172.21.15.159:0/1015347382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697d00c80 0x564697ce8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:28.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.457+0000 7f5bc1400640 1 --2- 172.21.15.159:0/1015347382 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564697cfb960 0x564697d199e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:28.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.458+0000 7f5bc1e00640 1 --2- 172.21.15.159:0/1015347382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfb580 0x564697ceb410 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4007c40 tx=0x7f5bb4001e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff79af4b1deeebc server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:28.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.458+0000 7f5ba3400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:15:28.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.458+0000 7f5ba3400640 1 -- 172.21.15.159:0/1015347382 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4019040 con 0x564697cfb580 2024-04-04T06:15:28.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.458+0000 7f5ba3400640 1 -- 172.21.15.159:0/1015347382 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f5bb4007840 con 0x564697cfb580 2024-04-04T06:15:28.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.458+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/1015347382 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bb40070d0 con 0x564697cfb580 2024-04-04T06:15:28.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.458+0000 7f5ba3400640 1 -- 172.21.15.159:0/1015347382 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f5bb401f470 con 0x564697cfb580 2024-04-04T06:15:28.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.458+0000 7f5bcbdaa740 10 client.8499 fetch_fsmap learned FSMap version 84 2024-04-04T06:15:28.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.458+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/1015347382 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x56469790d0f0 con 0x564697cfb580 2024-04-04T06:15:28.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.459+0000 7f5ba3400640 1 -- 172.21.15.159:0/1015347382 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb40070d0 con 0x564697cfb580 2024-04-04T06:15:28.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.459+0000 7f5ba3400640 1 -- 172.21.15.159:0/1015347382 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7f5bb401f610 con 0x564697cfb580 2024-04-04T06:15:28.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.459+0000 7f5bcbdaa740 10 client.8499 fetch_fsmap finished waiting for FSMap version 84 2024-04-04T06:15:28.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.459+0000 7f5bcbdaa740 10 client.8499 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-04T06:15:28.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.459+0000 7f5bcbdaa740 20 client.8499 populate_metadata read hostname 'smithi159' 2024-04-04T06:15:28.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.459+0000 7f5bcbdaa740 1 --2- 172.21.15.159:0/1015347382 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x564697d29510 0x564697d29930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.459+0000 7f5bcbdaa740 4 client.8499 mds_command: new command op to 8190 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:15:28.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.459+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/1015347382 --> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564697c46c10 con 0x564697d29510 2024-04-04T06:15:28.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.459+0000 7f5bc1400640 1 --2- 172.21.15.159:0/1015347382 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x564697d29510 0x564697d29930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.460+0000 7f5bc1400640 1 --2- 172.21.15.159:0/1015347382 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x564697d29510 0x564697d29930 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5d2e64adad22d5 server_cookie=e868fcd0e7bbc7f2 in_seq=0 out_seq=0 2024-04-04T06:15:28.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.460+0000 7f5ba3400640 10 client.8499 ms_handle_connect on v2:172.21.15.159:6834/2253383212 2024-04-04T06:15:28.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.460+0000 7f5ba3400640 1 -- 172.21.15.159:0/1015347382 <== mds.0 v2:172.21.15.159:6834/2253383212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564697c46c10 con 0x564697d29510 2024-04-04T06:15:28.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.460+0000 7f5ba3400640 10 client.8499 handle_command_reply: tid=0 2024-04-04T06:15:28.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.460+0000 7f5bcbdaa740 1 client.8499 shutdown 2024-04-04T06:15:28.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.461+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/1015347382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfb580 msgr2=0x564697ceb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.461+0000 7f5bcbdaa740 1 --2- 172.21.15.159:0/1015347382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfb580 0x564697ceb410 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4007c40 tx=0x7f5bb4001e30 comp rx=0 tx=0).stop 2024-04-04T06:15:28.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.461+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/1015347382 shutdown_connections 2024-04-04T06:15:28.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.461+0000 7f5bcbdaa740 1 --2- 172.21.15.159:0/1015347382 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x564697d29510 0x564697d29930 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.461+0000 7f5bcbdaa740 1 --2- 172.21.15.159:0/1015347382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697d00c80 0x564697ce8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.461+0000 7f5bcbdaa740 1 --2- 172.21.15.159:0/1015347382 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564697cfb960 0x564697d199e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.461+0000 7f5bcbdaa740 1 --2- 172.21.15.159:0/1015347382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfb580 0x564697ceb410 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.461+0000 7f5ba3400640 0 client.8499 ms_handle_reset on v2:172.21.15.159:6834/2253383212 2024-04-04T06:15:28.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.461+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/1015347382 >> 172.21.15.159:0/1015347382 conn(0x564697ccea60 msgr2=0x564697cd2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:28.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.461+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/1015347382 shutdown_connections 2024-04-04T06:15:28.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.461+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/1015347382 wait complete. 2024-04-04T06:15:28.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.461+0000 7f5bcbdaa740 20 client.8499 trim_cache size 0 max 16384 2024-04-04T06:15:28.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.480+0000 7f5bcbdaa740 1 Processor -- start 2024-04-04T06:15:28.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.480+0000 7f5bcbdaa740 1 -- start start 2024-04-04T06:15:28.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.481+0000 7f5bcbdaa740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564697cfb580 0x564697cf9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.481+0000 7f5bcbdaa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697cfb960 0x564697d1c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.481+0000 7f5bcbdaa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfa7a0 0x564697d1cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.481+0000 7f5bcbdaa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc4201380 con 0x564697cfa7a0 2024-04-04T06:15:28.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.481+0000 7f5bcbdaa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc42003e0 con 0x564697cfb960 2024-04-04T06:15:28.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.481+0000 7f5bcbdaa740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5bc41ff740 con 0x564697cfb580 2024-04-04T06:15:28.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.481+0000 7f5bc2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfa7a0 0x564697d1cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.481+0000 7f5bc2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfa7a0 0x564697d1cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39670/0 (socket says 172.21.15.159:39670) 2024-04-04T06:15:28.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.481+0000 7f5bc2800640 1 -- 172.21.15.159:0/3112085016 learned_addr learned my addr 172.21.15.159:0/3112085016 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:28.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.481+0000 7f5bc1400640 1 --2- 172.21.15.159:0/3112085016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697cfb960 0x564697d1c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.481+0000 7f5bc1e00640 1 --2- 172.21.15.159:0/3112085016 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564697cfb580 0x564697cf9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.482+0000 7f5bc2800640 1 -- 172.21.15.159:0/3112085016 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564697cfb580 msgr2=0x564697cf9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.482+0000 7f5bc2800640 1 --2- 172.21.15.159:0/3112085016 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564697cfb580 0x564697cf9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.482+0000 7f5bc2800640 1 -- 172.21.15.159:0/3112085016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697cfb960 msgr2=0x564697d1c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.482+0000 7f5bc2800640 1 --2- 172.21.15.159:0/3112085016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697cfb960 0x564697d1c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.482+0000 7f5bc2800640 1 -- 172.21.15.159:0/3112085016 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564697c35800 con 0x564697cfa7a0 2024-04-04T06:15:28.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.482+0000 7f5bc1400640 1 --2- 172.21.15.159:0/3112085016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697cfb960 0x564697d1c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:28.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.482+0000 7f5bc1e00640 1 --2- 172.21.15.159:0/3112085016 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564697cfb580 0x564697cf9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:28.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.482+0000 7f5bc2800640 1 --2- 172.21.15.159:0/3112085016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfa7a0 0x564697d1cca0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc02bfc0 tx=0x7f5bbc044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a5e46447d3c755 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:28.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.482+0000 7f5bae000640 1 -- 172.21.15.159:0/3112085016 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc041030 con 0x564697cfa7a0 2024-04-04T06:15:28.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.482+0000 7f5bae000640 1 -- 172.21.15.159:0/3112085016 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bbc042040 con 0x564697cfa7a0 2024-04-04T06:15:28.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.482+0000 7f5bae000640 1 -- 172.21.15.159:0/3112085016 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc0038d0 con 0x564697cfa7a0 2024-04-04T06:15:28.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.483+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/3112085016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfa7a0 msgr2=0x564697d1cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.483+0000 7f5bcbdaa740 1 --2- 172.21.15.159:0/3112085016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfa7a0 0x564697d1cca0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc02bfc0 tx=0x7f5bbc044000 comp rx=0 tx=0).stop 2024-04-04T06:15:28.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.483+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/3112085016 shutdown_connections 2024-04-04T06:15:28.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.483+0000 7f5bcbdaa740 1 --2- 172.21.15.159:0/3112085016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfa7a0 0x564697d1cca0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.483+0000 7f5bcbdaa740 1 --2- 172.21.15.159:0/3112085016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697cfb960 0x564697d1c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.483+0000 7f5bcbdaa740 1 --2- 172.21.15.159:0/3112085016 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564697cfb580 0x564697cf9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.483+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/3112085016 >> 172.21.15.159:0/3112085016 conn(0x564697ccea60 msgr2=0x564697d01fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:28.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.483+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/3112085016 shutdown_connections 2024-04-04T06:15:28.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.483+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/3112085016 wait complete. 2024-04-04T06:15:28.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.484+0000 7f5bcbdaa740 1 -- start start 2024-04-04T06:15:28.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.484+0000 7f5bcbdaa740 1 Processor -- start 2024-04-04T06:15:28.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.485+0000 7f5bcbdaa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697cfa7a0 0x564697d9b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.485+0000 7f5bcbdaa740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564697cfb580 0x564697ccd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.485+0000 7f5bcbdaa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfb960 0x564697ccdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.485+0000 7f5bcbdaa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc41ffd90 con 0x564697cfb960 2024-04-04T06:15:28.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.485+0000 7f5bcbdaa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564697d01250 con 0x564697cfa7a0 2024-04-04T06:15:28.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.485+0000 7f5bcbdaa740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x564697d1d050 con 0x564697cfb580 2024-04-04T06:15:28.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.485+0000 7f5bcbdaa740 10 client.0 _ll_register_callbacks cb 0x7f5bb40075d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:15:28.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.485+0000 7f5bc2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfb960 0x564697ccdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.485+0000 7f5bc2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfb960 0x564697ccdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39672/0 (socket says 172.21.15.159:39672) 2024-04-04T06:15:28.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.485+0000 7f5bc2800640 1 -- 172.21.15.159:0/4179430898 learned_addr learned my addr 172.21.15.159:0/4179430898 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:28.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.486+0000 7f5bc1400640 1 --2- 172.21.15.159:0/4179430898 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564697cfb580 0x564697ccd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.486+0000 7f5bc2800640 1 -- 172.21.15.159:0/4179430898 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564697cfb580 msgr2=0x564697ccd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.486+0000 7f5bc1e00640 1 --2- 172.21.15.159:0/4179430898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697cfa7a0 0x564697d9b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.486+0000 7f5bc2800640 1 --2- 172.21.15.159:0/4179430898 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564697cfb580 0x564697ccd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.486+0000 7f5bc2800640 1 -- 172.21.15.159:0/4179430898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697cfa7a0 msgr2=0x564697d9b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:28.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.486+0000 7f5bc2800640 1 --2- 172.21.15.159:0/4179430898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697cfa7a0 0x564697d9b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.486+0000 7f5bc2800640 1 -- 172.21.15.159:0/4179430898 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564697d04d80 con 0x564697cfb960 2024-04-04T06:15:28.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.486+0000 7f5bc2800640 1 --2- 172.21.15.159:0/4179430898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfb960 0x564697ccdd90 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc006fd0 tx=0x7f5bbc032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808bba8028d5b866 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:28.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.486+0000 7f5ba3400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:15:28.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.486+0000 7f5ba3400640 1 -- 172.21.15.159:0/4179430898 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc041070 con 0x564697cfb960 2024-04-04T06:15:28.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.486+0000 7f5ba3400640 1 -- 172.21.15.159:0/4179430898 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f5bbc05eaa0 con 0x564697cfb960 2024-04-04T06:15:28.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.486+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/4179430898 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bbc042040 con 0x564697cfb960 2024-04-04T06:15:28.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.487+0000 7f5ba3400640 1 -- 172.21.15.159:0/4179430898 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f5bbc003fe0 con 0x564697cfb960 2024-04-04T06:15:28.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.487+0000 7f5bcbdaa740 10 client.8505 fetch_fsmap learned FSMap version 84 2024-04-04T06:15:28.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.487+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/4179430898 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x564697d1ce20 con 0x564697cfb960 2024-04-04T06:15:28.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.487+0000 7f5ba3400640 1 -- 172.21.15.159:0/4179430898 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bbc042040 con 0x564697cfb960 2024-04-04T06:15:28.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.487+0000 7f5ba3400640 1 -- 172.21.15.159:0/4179430898 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7f5bbc05eca0 con 0x564697cfb960 2024-04-04T06:15:28.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.487+0000 7f5bcbdaa740 10 client.8505 fetch_fsmap finished waiting for FSMap version 84 2024-04-04T06:15:28.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.487+0000 7f5bcbdaa740 10 client.8505 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-04T06:15:28.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.487+0000 7f5bcbdaa740 20 client.8505 populate_metadata read hostname 'smithi159' 2024-04-04T06:15:28.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.487+0000 7f5bcbdaa740 1 --2- 172.21.15.159:0/4179430898 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x564697d3d640 0x564697d3c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:28.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.487+0000 7f5bcbdaa740 4 client.8505 mds_command: new command op to 8190 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-04T06:15:28.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.487+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/4179430898 --> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564697c40f60 con 0x564697d3d640 2024-04-04T06:15:28.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.487+0000 7f5bc1e00640 1 --2- 172.21.15.159:0/4179430898 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x564697d3d640 0x564697d3c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:28.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.488+0000 7f5bc1e00640 1 --2- 172.21.15.159:0/4179430898 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x564697d3d640 0x564697d3c0e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2f211ed992bd16 server_cookie=6cee461f6ac41b96 in_seq=0 out_seq=0 2024-04-04T06:15:28.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.488+0000 7f5ba3400640 10 client.8505 ms_handle_connect on v2:172.21.15.159:6834/2253383212 2024-04-04T06:15:28.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.488+0000 7f5ba3400640 1 -- 172.21.15.159:0/4179430898 <== mds.0 v2:172.21.15.159:6834/2253383212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x564697c40f60 con 0x564697d3d640 2024-04-04T06:15:28.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.488+0000 7f5ba3400640 10 client.8505 handle_command_reply: tid=0 2024-04-04T06:15:28.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.488+0000 7f5bcbdaa740 1 client.8505 shutdown 2024-04-04T06:15:28.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.488+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/4179430898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfb960 msgr2=0x564697ccdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.488+0000 7f5bcbdaa740 1 --2- 172.21.15.159:0/4179430898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfb960 0x564697ccdd90 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc006fd0 tx=0x7f5bbc032000 comp rx=0 tx=0).stop 2024-04-04T06:15:28.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.489+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/4179430898 shutdown_connections 2024-04-04T06:15:28.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.489+0000 7f5bcbdaa740 1 --2- 172.21.15.159:0/4179430898 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x564697d3d640 0x564697d3c0e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.489+0000 7f5bcbdaa740 1 --2- 172.21.15.159:0/4179430898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564697cfb960 0x564697ccdd90 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.489+0000 7f5bcbdaa740 1 --2- 172.21.15.159:0/4179430898 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x564697cfb580 0x564697ccd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.489+0000 7f5ba3400640 0 client.8505 ms_handle_reset on v2:172.21.15.159:6834/2253383212 2024-04-04T06:15:28.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.489+0000 7f5bcbdaa740 1 --2- 172.21.15.159:0/4179430898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564697cfa7a0 0x564697d9b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.489+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/4179430898 >> 172.21.15.159:0/4179430898 conn(0x564697ccea60 msgr2=0x564697ce7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:28.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.489+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/4179430898 shutdown_connections 2024-04-04T06:15:28.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.489+0000 7f5bcbdaa740 1 -- 172.21.15.159:0/4179430898 wait complete. 2024-04-04T06:15:28.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.489+0000 7f5bcbdaa740 20 client.8505 trim_cache size 0 max 16384 2024-04-04T06:15:28.487 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:15:28.487 INFO:teuthology.orchestra.run.smithi159.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-04T06:15:28.487 INFO:teuthology.orchestra.run.smithi159.stdout: "scrubs": { 2024-04-04T06:15:28.487 INFO:teuthology.orchestra.run.smithi159.stdout: "9ba77e50-6c91-42a6-acc0-4c82a2673c3f": { 2024-04-04T06:15:28.487 INFO:teuthology.orchestra.run.smithi159.stdout: "path": "/", 2024-04-04T06:15:28.487 INFO:teuthology.orchestra.run.smithi159.stdout: "tag": "9ba77e50-6c91-42a6-acc0-4c82a2673c3f", 2024-04-04T06:15:28.487 INFO:teuthology.orchestra.run.smithi159.stdout: "options": "recursive,repair" 2024-04-04T06:15:28.487 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:28.487 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:15:28.487 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:15:28.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.489+0000 7f5bc9000640 1 -- 172.21.15.159:0/411305746 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5ba406c8f0 msgr2=0x7f5ba406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.489+0000 7f5bc9000640 1 --2- 172.21.15.159:0/411305746 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5ba406c8f0 0x7f5ba406edb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc035000 tx=0x7f5bbc0046c0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.489+0000 7f5bc9000640 1 -- 172.21.15.159:0/411305746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bc414f120 msgr2=0x7f5bc40b1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:28.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.489+0000 7f5bc9000640 1 --2- 172.21.15.159:0/411305746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bc414f120 0x7f5bc40b1220 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0010830 tx=0x7f5bb0010cf0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.490+0000 7f5bc9000640 1 -- 172.21.15.159:0/411305746 shutdown_connections 2024-04-04T06:15:28.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.490+0000 7f5bc9000640 1 --2- 172.21.15.159:0/411305746 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5bc415c330 0x7f5bc40b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.490+0000 7f5bc9000640 1 --2- 172.21.15.159:0/411305746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bc414f120 0x7f5bc40b1220 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.490+0000 7f5bc9000640 1 --2- 172.21.15.159:0/411305746 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f5ba406c8f0 0x7f5ba406edb0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.490+0000 7f5bc9000640 1 --2- 172.21.15.159:0/411305746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ba8004d40 0x7f5bc4152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:28.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.490+0000 7f5bc9000640 1 -- 172.21.15.159:0/411305746 >> 172.21.15.159:0/411305746 conn(0x7f5bc406bba0 msgr2=0x7f5bc40b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:28.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.490+0000 7f5bc9000640 1 -- 172.21.15.159:0/411305746 shutdown_connections 2024-04-04T06:15:28.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:28.490+0000 7f5bc9000640 1 -- 172.21.15.159:0/411305746 wait complete. 2024-04-04T06:15:28.509 INFO:tasks.cephfs.filesystem:scrub status for tag:9ba77e50-6c91-42a6-acc0-4c82a2673c3f - {'path': '/', 'tag': '9ba77e50-6c91-42a6-acc0-4c82a2673c3f', 'options': 'recursive,repair'} 2024-04-04T06:15:29.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:29.387+0000 7f3a96000640 1 -- 172.21.15.159:0/293948300 <== mon.0 v2:172.21.15.159:3300/0 7 ==== log(4 entries from seq 1 at 2024-04-04T06:15:28.227125+0000) ==== 695+0+0 (secure 0 0 0) 0x7f3a98057fd0 con 0x7f3aa00adf60 2024-04-04T06:15:29.382 INFO:teuthology.orchestra.run.smithi159.stdout: cluster: 2024-04-04T06:15:29.382 INFO:teuthology.orchestra.run.smithi159.stdout: id: 30311045-f714-48fa-832b-4cb2ba8bf0bf 2024-04-04T06:15:29.382 INFO:teuthology.orchestra.run.smithi159.stdout: health: HEALTH_OK 2024-04-04T06:15:29.382 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:15:29.382 INFO:teuthology.orchestra.run.smithi159.stdout: services: 2024-04-04T06:15:29.382 INFO:teuthology.orchestra.run.smithi159.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-04-04T06:15:29.382 INFO:teuthology.orchestra.run.smithi159.stdout: mgr: x(active, since 9m), standbys: y 2024-04-04T06:15:29.383 INFO:teuthology.orchestra.run.smithi159.stdout: mds: 1/1 daemons up, 3 standby 2024-04-04T06:15:29.383 INFO:teuthology.orchestra.run.smithi159.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 9m) 2024-04-04T06:15:29.383 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:15:29.383 INFO:teuthology.orchestra.run.smithi159.stdout: data: 2024-04-04T06:15:29.383 INFO:teuthology.orchestra.run.smithi159.stdout: volumes: 1/1 healthy 2024-04-04T06:15:29.383 INFO:teuthology.orchestra.run.smithi159.stdout: pools: 3 pools, 97 pgs 2024-04-04T06:15:29.383 INFO:teuthology.orchestra.run.smithi159.stdout: objects: 30 objects, 19 MiB 2024-04-04T06:15:29.383 INFO:teuthology.orchestra.run.smithi159.stdout: usage: 307 MiB used, 720 GiB / 720 GiB avail 2024-04-04T06:15:29.383 INFO:teuthology.orchestra.run.smithi159.stdout: pgs: 97 active+clean 2024-04-04T06:15:29.383 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:15:29.383 INFO:teuthology.orchestra.run.smithi159.stdout: io: 2024-04-04T06:15:29.383 INFO:teuthology.orchestra.run.smithi159.stdout: client: 85 B/s rd, 1.5 MiB/s wr, 0 op/s rd, 3 op/s wr 2024-04-04T06:15:29.383 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:15:29.384 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:15:29.384 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:15:28.227125+0000 mds.a [INF] scrub queued for path: / 2024-04-04T06:15:29.384 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:15:28.227129+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-04T06:15:29.384 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:15:28.227133+0000 mds.a [INF] scrub summary: active paths [/] 2024-04-04T06:15:29.384 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:15:28.229511+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-04T06:15:32.390 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:15:31.830600+0000 mds.a [INF] scrub summary: idle 2024-04-04T06:15:32.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:32.395+0000 7f3a96000640 1 -- 172.21.15.159:0/293948300 <== mon.0 v2:172.21.15.159:3300/0 8 ==== log(1 entries from seq 5 at 2024-04-04T06:15:31.830600+0000) ==== 189+0+0 (secure 0 0 0) 0x7f3a98058230 con 0x7f3aa00adf60 2024-04-04T06:15:43.368 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:15:43.376 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:15:58.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:15:58.511 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-04T06:15:58.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.733+0000 7fb115e00640 1 Processor -- start 2024-04-04T06:15:58.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.733+0000 7fb115e00640 1 -- start start 2024-04-04T06:15:58.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.735+0000 7fb115e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb1100b69c0 0x7fb110168f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:58.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.735+0000 7fb115e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1100d2710 con 0x7fb11015e4c0 2024-04-04T06:15:58.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.735+0000 7fb115e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1100d29f0 con 0x7fb1100d37b0 2024-04-04T06:15:58.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.735+0000 7fb115e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fb11015e8a0 con 0x7fb1100b69c0 2024-04-04T06:15:58.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.735+0000 7fb114a00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fb11015e4c0 0x7fb1100b9bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52656/0 (socket says 172.21.15.159:52656) 2024-04-04T06:15:58.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.735+0000 7fb114a00640 1 -- 172.21.15.159:0/2091314848 learned_addr learned my addr 172.21.15.159:0/2091314848 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:15:58.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.735+0000 7fb115400640 1 --2- 172.21.15.159:0/2091314848 >> v2:172.21.15.159:3300/0 conn(0x7fb1100b69c0 0x7fb110168f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:58.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.736+0000 7fb10fe00640 1 -- 172.21.15.159:0/2091314848 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2931761575 0 0) 0x7fb1100d2710 con 0x7fb11015e4c0 2024-04-04T06:15:58.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.736+0000 7fb10fe00640 1 -- 172.21.15.159:0/2091314848 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0f4003520 con 0x7fb11015e4c0 2024-04-04T06:15:58.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.736+0000 7fb115400640 1 -- 172.21.15.159:0/2091314848 >> v1:172.21.15.178:6789/0 conn(0x7fb1100d37b0 legacy=0x7fb1100b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:58.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.736+0000 7fb115400640 1 -- 172.21.15.159:0/2091314848 >> v1:172.21.15.159:6789/0 conn(0x7fb11015e4c0 legacy=0x7fb1100b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:58.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.736+0000 7fb115400640 1 -- 172.21.15.159:0/2091314848 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1100d2cd0 con 0x7fb1100b69c0 2024-04-04T06:15:58.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.737+0000 7fb115400640 1 --2- 172.21.15.159:0/2091314848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1100b69c0 0x7fb110168f70 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fb108009d90 tx=0x7fb10802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e535a508b4f8cb34 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:58.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.737+0000 7fb10fe00640 1 -- 172.21.15.159:0/2091314848 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb108039070 con 0x7fb1100b69c0 2024-04-04T06:15:58.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.737+0000 7fb10fe00640 1 -- 172.21.15.159:0/2091314848 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb10802cc60 con 0x7fb1100b69c0 2024-04-04T06:15:58.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.737+0000 7fb10fe00640 1 -- 172.21.15.159:0/2091314848 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb108034660 con 0x7fb1100b69c0 2024-04-04T06:15:58.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.738+0000 7fb115e00640 1 -- 172.21.15.159:0/2091314848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1100b69c0 msgr2=0x7fb110168f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:58.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.738+0000 7fb115e00640 1 --2- 172.21.15.159:0/2091314848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1100b69c0 0x7fb110168f70 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fb108009d90 tx=0x7fb10802beb0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.738+0000 7fb115e00640 1 -- 172.21.15.159:0/2091314848 shutdown_connections 2024-04-04T06:15:58.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.738+0000 7fb115e00640 1 --2- 172.21.15.159:0/2091314848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1100b69c0 0x7fb110168f70 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.738+0000 7fb115e00640 1 -- 172.21.15.159:0/2091314848 >> 172.21.15.159:0/2091314848 conn(0x7fb1100676a0 msgr2=0x7fb110067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:58.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.738+0000 7fb115e00640 1 -- 172.21.15.159:0/2091314848 shutdown_connections 2024-04-04T06:15:58.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.738+0000 7fb115e00640 1 -- 172.21.15.159:0/2091314848 wait complete. 2024-04-04T06:15:58.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.740+0000 7fb115e00640 1 Processor -- start 2024-04-04T06:15:58.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.740+0000 7fb115e00640 1 -- start start 2024-04-04T06:15:58.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.741+0000 7fb115e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1100b69c0 0x7fb110154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:58.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.741+0000 7fb115e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1100d37b0 0x7fb110160d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:58.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.741+0000 7fb115e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb11015e4c0 0x7fb1100cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:58.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.741+0000 7fb115e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb1100d3320 con 0x7fb1100b69c0 2024-04-04T06:15:58.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.741+0000 7fb115e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb1100d4040 con 0x7fb11015e4c0 2024-04-04T06:15:58.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.741+0000 7fb115e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb1100d2350 con 0x7fb1100d37b0 2024-04-04T06:15:58.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.741+0000 7fb114a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1100b69c0 0x7fb110154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:58.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.741+0000 7fb114a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1100b69c0 0x7fb110154390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52454/0 (socket says 172.21.15.159:52454) 2024-04-04T06:15:58.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.741+0000 7fb114a00640 1 -- 172.21.15.159:0/2673745318 learned_addr learned my addr 172.21.15.159:0/2673745318 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:58.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.741+0000 7fb114a00640 1 -- 172.21.15.159:0/2673745318 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1100d37b0 msgr2=0x7fb110160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:58.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.741+0000 7fb114a00640 1 --2- 172.21.15.159:0/2673745318 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1100d37b0 0x7fb110160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.741+0000 7fb114a00640 1 -- 172.21.15.159:0/2673745318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb11015e4c0 msgr2=0x7fb1100cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:58.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.741+0000 7fb114a00640 1 --2- 172.21.15.159:0/2673745318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb11015e4c0 0x7fb1100cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.741+0000 7fb114a00640 1 -- 172.21.15.159:0/2673745318 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1101612c0 con 0x7fb1100b69c0 2024-04-04T06:15:58.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.742+0000 7fb114a00640 1 --2- 172.21.15.159:0/2673745318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1100b69c0 0x7fb110154390 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fb10000c460 tx=0x7fb10000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:58.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.742+0000 7fb10d600640 1 -- 172.21.15.159:0/2673745318 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100017070 con 0x7fb1100b69c0 2024-04-04T06:15:58.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.742+0000 7fb10d600640 1 -- 172.21.15.159:0/2673745318 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb100005ce0 con 0x7fb1100b69c0 2024-04-04T06:15:58.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.742+0000 7fb10d600640 1 -- 172.21.15.159:0/2673745318 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100007290 con 0x7fb1100b69c0 2024-04-04T06:15:58.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.742+0000 7fb115e00640 1 -- 172.21.15.159:0/2673745318 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11015fce0 con 0x7fb1100b69c0 2024-04-04T06:15:58.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.742+0000 7fb115e00640 1 -- 172.21.15.159:0/2673745318 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1101602a0 con 0x7fb1100b69c0 2024-04-04T06:15:58.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.744+0000 7fb10d600640 1 -- 172.21.15.159:0/2673745318 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb10001a070 con 0x7fb1100b69c0 2024-04-04T06:15:58.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.746+0000 7fb10d600640 1 --2- 172.21.15.159:0/2673745318 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb0ec06c8f0 0x7fb0ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:58.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.746+0000 7fb10d600640 1 -- 172.21.15.159:0/2673745318 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fb10008afd0 con 0x7fb1100b69c0 2024-04-04T06:15:58.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.747+0000 7fb11a236740 1 Processor -- start 2024-04-04T06:15:58.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.747+0000 7fb107e00640 1 --2- 172.21.15.159:0/2673745318 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb0ec06c8f0 0x7fb0ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:58.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.747+0000 7fb11a236740 1 -- start start 2024-04-04T06:15:58.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.747+0000 7fb11a236740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d63a740 0x55b59d60d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:58.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.748+0000 7fb11a236740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b59d63ab20 0x55b59d63d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:58.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.748+0000 7fb11a236740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d60dc20 0x55b59d63da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:58.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.748+0000 7fb11a236740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b59d5786d0 con 0x55b59d63a740 2024-04-04T06:15:58.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.748+0000 7fb11a236740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b59d520770 con 0x55b59d60dc20 2024-04-04T06:15:58.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.748+0000 7fb11a236740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55b59d574680 con 0x55b59d63ab20 2024-04-04T06:15:58.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.748+0000 7fb107e00640 1 --2- 172.21.15.159:0/2673745318 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb0ec06c8f0 0x7fb0ec06edb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb1100b6570 tx=0x7fb0f80079b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:58.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.748+0000 7fb114a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d63a740 0x55b59d60d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:58.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.748+0000 7fb114a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d63a740 0x55b59d60d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52470/0 (socket says 172.21.15.159:52470) 2024-04-04T06:15:58.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.748+0000 7fb114a00640 1 -- 172.21.15.159:0/697284864 learned_addr learned my addr 172.21.15.159:0/697284864 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:58.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.748+0000 7fb115400640 1 --2- 172.21.15.159:0/697284864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d60dc20 0x55b59d63da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:58.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.748+0000 7fb107e00640 1 --2- 172.21.15.159:0/697284864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b59d63ab20 0x55b59d63d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:58.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.748+0000 7fb114a00640 1 -- 172.21.15.159:0/697284864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b59d63ab20 msgr2=0x55b59d63d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:58.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.748+0000 7fb114a00640 1 --2- 172.21.15.159:0/697284864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b59d63ab20 0x55b59d63d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.748+0000 7fb114a00640 1 -- 172.21.15.159:0/697284864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d60dc20 msgr2=0x55b59d63da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:58.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.748+0000 7fb114a00640 1 --2- 172.21.15.159:0/697284864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d60dc20 0x55b59d63da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.748+0000 7fb114a00640 1 -- 172.21.15.159:0/697284864 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b59d3567d0 con 0x55b59d63a740 2024-04-04T06:15:58.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.748+0000 7fb115400640 1 --2- 172.21.15.159:0/697284864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d60dc20 0x55b59d63da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:58.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.749+0000 7fb114a00640 1 --2- 172.21.15.159:0/697284864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d63a740 0x55b59d60d6e0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fb1000613d0 tx=0x7fb10005d0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a0b6dcfe084ab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:58.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.749+0000 7fb105200640 1 -- 172.21.15.159:0/697284864 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100057020 con 0x55b59d63a740 2024-04-04T06:15:58.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.749+0000 7fb105200640 1 -- 172.21.15.159:0/697284864 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb100055390 con 0x55b59d63a740 2024-04-04T06:15:58.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.749+0000 7fb105200640 1 -- 172.21.15.159:0/697284864 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1000616b0 con 0x55b59d63a740 2024-04-04T06:15:58.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.749+0000 7fb11a236740 1 -- 172.21.15.159:0/697284864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d63a740 msgr2=0x55b59d60d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:58.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.749+0000 7fb11a236740 1 --2- 172.21.15.159:0/697284864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d63a740 0x55b59d60d6e0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fb1000613d0 tx=0x7fb10005d0d0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.750+0000 7fb11a236740 1 -- 172.21.15.159:0/697284864 shutdown_connections 2024-04-04T06:15:58.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.750+0000 7fb11a236740 1 --2- 172.21.15.159:0/697284864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d60dc20 0x55b59d63da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.750+0000 7fb11a236740 1 --2- 172.21.15.159:0/697284864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b59d63ab20 0x55b59d63d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.750+0000 7fb11a236740 1 --2- 172.21.15.159:0/697284864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d63a740 0x55b59d60d6e0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.750+0000 7fb11a236740 1 -- 172.21.15.159:0/697284864 >> 172.21.15.159:0/697284864 conn(0x55b59d60f950 msgr2=0x55b59d6584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:58.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.750+0000 7fb11a236740 1 -- 172.21.15.159:0/697284864 shutdown_connections 2024-04-04T06:15:58.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.750+0000 7fb11a236740 1 -- 172.21.15.159:0/697284864 wait complete. 2024-04-04T06:15:58.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.750+0000 7fb11a236740 1 -- start start 2024-04-04T06:15:58.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.751+0000 7fb11a236740 1 Processor -- start 2024-04-04T06:15:58.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.752+0000 7fb11a236740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d63a740 0x55b59d6289b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:58.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.752+0000 7fb11a236740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b59d63ab20 0x55b59d635e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:58.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.752+0000 7fb11a236740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d6363b0 0x55b59d633f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:58.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.752+0000 7fb11a236740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b59d611ff0 con 0x55b59d6363b0 2024-04-04T06:15:58.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.752+0000 7fb11a236740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b59d533aa0 con 0x55b59d63a740 2024-04-04T06:15:58.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.752+0000 7fb11a236740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55b59d614830 con 0x55b59d63ab20 2024-04-04T06:15:58.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.752+0000 7fb11a236740 10 client.0 _ll_register_callbacks cb 0x55b59d533490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:15:58.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.752+0000 7fb115400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d6363b0 0x55b59d633f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:58.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.752+0000 7fb115400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d6363b0 0x55b59d633f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52486/0 (socket says 172.21.15.159:52486) 2024-04-04T06:15:58.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.752+0000 7fb115400640 1 -- 172.21.15.159:0/3472622046 learned_addr learned my addr 172.21.15.159:0/3472622046 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:58.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.752+0000 7fb114a00640 1 --2- 172.21.15.159:0/3472622046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d63a740 0x55b59d6289b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:58.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.752+0000 7fb107e00640 1 --2- 172.21.15.159:0/3472622046 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b59d63ab20 0x55b59d635e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:58.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.752+0000 7fb115400640 1 -- 172.21.15.159:0/3472622046 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b59d63ab20 msgr2=0x55b59d635e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:58.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.752+0000 7fb115400640 1 --2- 172.21.15.159:0/3472622046 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b59d63ab20 0x55b59d635e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.752+0000 7fb115400640 1 -- 172.21.15.159:0/3472622046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d63a740 msgr2=0x55b59d6289b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:58.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.752+0000 7fb115400640 1 --2- 172.21.15.159:0/3472622046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d63a740 0x55b59d6289b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.752+0000 7fb115400640 1 -- 172.21.15.159:0/3472622046 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b59d28c3f0 con 0x55b59d6363b0 2024-04-04T06:15:58.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.753+0000 7fb115400640 1 --2- 172.21.15.159:0/3472622046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d6363b0 0x55b59d633f80 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fb108009eb0 tx=0x7fb108004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fae15419d3798e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:58.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.753+0000 7fb0fd600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:15:58.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.753+0000 7fb0fd600640 1 -- 172.21.15.159:0/3472622046 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb108044020 con 0x55b59d6363b0 2024-04-04T06:15:58.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.753+0000 7fb11a236740 1 -- 172.21.15.159:0/3472622046 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb100055390 con 0x55b59d6363b0 2024-04-04T06:15:58.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.753+0000 7fb0fd600640 1 -- 172.21.15.159:0/3472622046 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fb10804aaa0 con 0x55b59d6363b0 2024-04-04T06:15:58.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.753+0000 7fb0fd600640 1 -- 172.21.15.159:0/3472622046 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fb108039040 con 0x55b59d6363b0 2024-04-04T06:15:58.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.753+0000 7fb11a236740 10 client.8520 fetch_fsmap learned FSMap version 84 2024-04-04T06:15:58.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.753+0000 7fb11a236740 1 -- 172.21.15.159:0/3472622046 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x55b59d24c0f0 con 0x55b59d6363b0 2024-04-04T06:15:58.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.754+0000 7fb0fd600640 1 -- 172.21.15.159:0/3472622046 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb100055390 con 0x55b59d6363b0 2024-04-04T06:15:58.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.754+0000 7fb0fd600640 1 -- 172.21.15.159:0/3472622046 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7fb108051080 con 0x55b59d6363b0 2024-04-04T06:15:58.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.754+0000 7fb11a236740 10 client.8520 fetch_fsmap finished waiting for FSMap version 84 2024-04-04T06:15:58.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.754+0000 7fb11a236740 10 client.8520 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-04T06:15:58.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.754+0000 7fb11a236740 20 client.8520 populate_metadata read hostname 'smithi159' 2024-04-04T06:15:58.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.754+0000 7fb11a236740 1 --2- 172.21.15.159:0/3472622046 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x55b59d6690c0 0x55b59d640f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:58.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.754+0000 7fb11a236740 4 client.8520 mds_command: new command op to 8190 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:15:58.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.754+0000 7fb11a236740 1 -- 172.21.15.159:0/3472622046 --> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b59d585c10 con 0x55b59d6690c0 2024-04-04T06:15:58.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.754+0000 7fb114a00640 1 --2- 172.21.15.159:0/3472622046 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x55b59d6690c0 0x55b59d640f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:58.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.755+0000 7fb114a00640 1 --2- 172.21.15.159:0/3472622046 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x55b59d6690c0 0x55b59d640f90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40407aa61167ae7d server_cookie=61b54f2a05ed0c9c in_seq=0 out_seq=0 2024-04-04T06:15:58.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.755+0000 7fb0fd600640 10 client.8520 ms_handle_connect on v2:172.21.15.159:6834/2253383212 2024-04-04T06:15:58.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.755+0000 7fb0fd600640 1 -- 172.21.15.159:0/3472622046 <== mds.0 v2:172.21.15.159:6834/2253383212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b59d585c10 con 0x55b59d6690c0 2024-04-04T06:15:58.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.755+0000 7fb0fd600640 10 client.8520 handle_command_reply: tid=0 2024-04-04T06:15:58.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.755+0000 7fb11a236740 1 client.8520 shutdown 2024-04-04T06:15:58.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.756+0000 7fb11a236740 1 -- 172.21.15.159:0/3472622046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d6363b0 msgr2=0x55b59d633f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:58.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.756+0000 7fb11a236740 1 --2- 172.21.15.159:0/3472622046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d6363b0 0x55b59d633f80 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fb108009eb0 tx=0x7fb108004a40 comp rx=0 tx=0).stop 2024-04-04T06:15:58.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.756+0000 7fb11a236740 1 -- 172.21.15.159:0/3472622046 shutdown_connections 2024-04-04T06:15:58.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.756+0000 7fb11a236740 1 --2- 172.21.15.159:0/3472622046 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x55b59d6690c0 0x55b59d640f90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.756+0000 7fb11a236740 1 --2- 172.21.15.159:0/3472622046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d6363b0 0x55b59d633f80 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.756+0000 7fb11a236740 1 --2- 172.21.15.159:0/3472622046 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b59d63ab20 0x55b59d635e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.756+0000 7fb11a236740 1 --2- 172.21.15.159:0/3472622046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d63a740 0x55b59d6289b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.756+0000 7fb0fd600640 0 client.8520 ms_handle_reset on v2:172.21.15.159:6834/2253383212 2024-04-04T06:15:58.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.756+0000 7fb11a236740 1 -- 172.21.15.159:0/3472622046 >> 172.21.15.159:0/3472622046 conn(0x55b59d60dc20 msgr2=0x55b59d6115f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:58.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.756+0000 7fb11a236740 1 -- 172.21.15.159:0/3472622046 shutdown_connections 2024-04-04T06:15:58.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.756+0000 7fb11a236740 1 -- 172.21.15.159:0/3472622046 wait complete. 2024-04-04T06:15:58.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.756+0000 7fb11a236740 20 client.8520 trim_cache size 0 max 16384 2024-04-04T06:15:58.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.775+0000 7fb11a236740 1 Processor -- start 2024-04-04T06:15:58.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.775+0000 7fb11a236740 1 -- start start 2024-04-04T06:15:58.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.776+0000 7fb11a236740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d6363b0 0x55b59d633f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:58.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.776+0000 7fb11a236740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d63a740 0x55b59d65ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:58.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.776+0000 7fb11a236740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b59d63ab20 0x55b59d6c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:58.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.776+0000 7fb11a236740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb1102008b0 con 0x55b59d6363b0 2024-04-04T06:15:58.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.776+0000 7fb11a236740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb11015f820 con 0x55b59d63a740 2024-04-04T06:15:58.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.776+0000 7fb11a236740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb11015eb80 con 0x55b59d63ab20 2024-04-04T06:15:58.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.776+0000 7fb114a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d6363b0 0x55b59d633f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:58.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.776+0000 7fb114a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d6363b0 0x55b59d633f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52498/0 (socket says 172.21.15.159:52498) 2024-04-04T06:15:58.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.776+0000 7fb114a00640 1 -- 172.21.15.159:0/4205804534 learned_addr learned my addr 172.21.15.159:0/4205804534 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:58.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.777+0000 7fb107e00640 1 --2- 172.21.15.159:0/4205804534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d63a740 0x55b59d65ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:58.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.777+0000 7fb114a00640 1 -- 172.21.15.159:0/4205804534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b59d63ab20 msgr2=0x55b59d6c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:58.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.777+0000 7fb114a00640 1 --2- 172.21.15.159:0/4205804534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b59d63ab20 0x55b59d6c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.777+0000 7fb114a00640 1 -- 172.21.15.159:0/4205804534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d63a740 msgr2=0x55b59d65ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:58.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.777+0000 7fb114a00640 1 --2- 172.21.15.159:0/4205804534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d63a740 0x55b59d65ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.777+0000 7fb114a00640 1 -- 172.21.15.159:0/4205804534 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b59d63e4d0 con 0x55b59d6363b0 2024-04-04T06:15:58.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.777+0000 7fb107e00640 1 --2- 172.21.15.159:0/4205804534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d63a740 0x55b59d65ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:58.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.777+0000 7fb114a00640 1 --2- 172.21.15.159:0/4205804534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d6363b0 0x55b59d633f80 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fb1000589d0 tx=0x7fb100014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582fe5760045172a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:58.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.777+0000 7fb0ffe00640 1 -- 172.21.15.159:0/4205804534 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100058af0 con 0x55b59d6363b0 2024-04-04T06:15:58.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.777+0000 7fb0ffe00640 1 -- 172.21.15.159:0/4205804534 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb100055390 con 0x55b59d6363b0 2024-04-04T06:15:58.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.777+0000 7fb0ffe00640 1 -- 172.21.15.159:0/4205804534 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1000782c0 con 0x55b59d6363b0 2024-04-04T06:15:58.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.778+0000 7fb11a236740 1 -- 172.21.15.159:0/4205804534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d6363b0 msgr2=0x55b59d633f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:58.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.778+0000 7fb11a236740 1 --2- 172.21.15.159:0/4205804534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d6363b0 0x55b59d633f80 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fb1000589d0 tx=0x7fb100014000 comp rx=0 tx=0).stop 2024-04-04T06:15:58.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.778+0000 7fb11a236740 1 -- 172.21.15.159:0/4205804534 shutdown_connections 2024-04-04T06:15:58.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.778+0000 7fb11a236740 1 --2- 172.21.15.159:0/4205804534 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b59d63ab20 0x55b59d6c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.778+0000 7fb11a236740 1 --2- 172.21.15.159:0/4205804534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d63a740 0x55b59d65ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.778+0000 7fb11a236740 1 --2- 172.21.15.159:0/4205804534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d6363b0 0x55b59d633f80 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.778+0000 7fb11a236740 1 -- 172.21.15.159:0/4205804534 >> 172.21.15.159:0/4205804534 conn(0x55b59d60dc20 msgr2=0x55b59d6287c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:58.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.778+0000 7fb11a236740 1 -- 172.21.15.159:0/4205804534 shutdown_connections 2024-04-04T06:15:58.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.778+0000 7fb11a236740 1 -- 172.21.15.159:0/4205804534 wait complete. 2024-04-04T06:15:58.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.779+0000 7fb11a236740 1 -- start start 2024-04-04T06:15:58.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.780+0000 7fb11a236740 1 Processor -- start 2024-04-04T06:15:58.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.781+0000 7fb11a236740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d6363b0 0x55b59d659cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:58.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.781+0000 7fb11a236740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b59d63a740 0x55b59d65a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:58.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.781+0000 7fb11a236740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d63ab20 0x55b59d626fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:58.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.781+0000 7fb11a236740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb11015f1d0 con 0x55b59d63ab20 2024-04-04T06:15:58.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.781+0000 7fb11a236740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b59d60f400 con 0x55b59d6363b0 2024-04-04T06:15:58.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.781+0000 7fb11a236740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55b59d6280c0 con 0x55b59d63a740 2024-04-04T06:15:58.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.781+0000 7fb11a236740 10 client.0 _ll_register_callbacks cb 0x7fb108009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:15:58.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.782+0000 7fb115400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d63ab20 0x55b59d626fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:58.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.782+0000 7fb115400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d63ab20 0x55b59d626fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52504/0 (socket says 172.21.15.159:52504) 2024-04-04T06:15:58.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.782+0000 7fb115400640 1 -- 172.21.15.159:0/3896214221 learned_addr learned my addr 172.21.15.159:0/3896214221 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:58.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.782+0000 7fb107e00640 1 --2- 172.21.15.159:0/3896214221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b59d63a740 0x55b59d65a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:58.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.782+0000 7fb115400640 1 -- 172.21.15.159:0/3896214221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b59d63a740 msgr2=0x55b59d65a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:58.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.782+0000 7fb114a00640 1 --2- 172.21.15.159:0/3896214221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d6363b0 0x55b59d659cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:58.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.782+0000 7fb115400640 1 --2- 172.21.15.159:0/3896214221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b59d63a740 0x55b59d65a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.782+0000 7fb115400640 1 -- 172.21.15.159:0/3896214221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d6363b0 msgr2=0x55b59d659cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:58.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.782+0000 7fb115400640 1 --2- 172.21.15.159:0/3896214221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d6363b0 0x55b59d659cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.782+0000 7fb115400640 1 -- 172.21.15.159:0/3896214221 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b59d632df0 con 0x55b59d63ab20 2024-04-04T06:15:58.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.782+0000 7fb114a00640 1 --2- 172.21.15.159:0/3896214221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d6363b0 0x55b59d659cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:58.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.782+0000 7fb115400640 1 --2- 172.21.15.159:0/3896214221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d63ab20 0x55b59d626fb0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb108009d90 tx=0x7fb108030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35536bbf1a7d214 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:58.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.782+0000 7fb0fd600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:15:58.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.783+0000 7fb0fd600640 1 -- 172.21.15.159:0/3896214221 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb108044070 con 0x55b59d63ab20 2024-04-04T06:15:58.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.783+0000 7fb0fd600640 1 -- 172.21.15.159:0/3896214221 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fb108003990 con 0x55b59d63ab20 2024-04-04T06:15:58.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.783+0000 7fb11a236740 1 -- 172.21.15.159:0/3896214221 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb100055390 con 0x55b59d63ab20 2024-04-04T06:15:58.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.783+0000 7fb0fd600640 1 -- 172.21.15.159:0/3896214221 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fb10805e050 con 0x55b59d63ab20 2024-04-04T06:15:58.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.783+0000 7fb11a236740 10 client.8526 fetch_fsmap learned FSMap version 84 2024-04-04T06:15:58.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.783+0000 7fb11a236740 1 -- 172.21.15.159:0/3896214221 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x55b59d627e00 con 0x55b59d63ab20 2024-04-04T06:15:58.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.783+0000 7fb0fd600640 1 -- 172.21.15.159:0/3896214221 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb100055390 con 0x55b59d63ab20 2024-04-04T06:15:58.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.783+0000 7fb0fd600640 1 -- 172.21.15.159:0/3896214221 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7fb108061020 con 0x55b59d63ab20 2024-04-04T06:15:58.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.783+0000 7fb11a236740 10 client.8526 fetch_fsmap finished waiting for FSMap version 84 2024-04-04T06:15:58.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.783+0000 7fb11a236740 10 client.8526 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-04T06:15:58.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.783+0000 7fb11a236740 20 client.8526 populate_metadata read hostname 'smithi159' 2024-04-04T06:15:58.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.783+0000 7fb11a236740 1 --2- 172.21.15.159:0/3896214221 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x55b59d6588c0 0x55b59d6274f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:58.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.783+0000 7fb11a236740 4 client.8526 mds_command: new command op to 8190 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-04T06:15:58.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.783+0000 7fb11a236740 1 -- 172.21.15.159:0/3896214221 --> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b59d57ff60 con 0x55b59d6588c0 2024-04-04T06:15:58.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.784+0000 7fb114a00640 1 --2- 172.21.15.159:0/3896214221 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x55b59d6588c0 0x55b59d6274f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:58.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.784+0000 7fb114a00640 1 --2- 172.21.15.159:0/3896214221 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x55b59d6588c0 0x55b59d6274f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=404f019984a94b6f server_cookie=7685b33f2d6110ad in_seq=0 out_seq=0 2024-04-04T06:15:58.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.784+0000 7fb0fd600640 10 client.8526 ms_handle_connect on v2:172.21.15.159:6834/2253383212 2024-04-04T06:15:58.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.784+0000 7fb0fd600640 1 -- 172.21.15.159:0/3896214221 <== mds.0 v2:172.21.15.159:6834/2253383212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b59d57ff60 con 0x55b59d6588c0 2024-04-04T06:15:58.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.784+0000 7fb0fd600640 10 client.8526 handle_command_reply: tid=0 2024-04-04T06:15:58.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.784+0000 7fb11a236740 1 client.8526 shutdown 2024-04-04T06:15:58.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.785+0000 7fb11a236740 1 -- 172.21.15.159:0/3896214221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d63ab20 msgr2=0x55b59d626fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:58.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.785+0000 7fb11a236740 1 --2- 172.21.15.159:0/3896214221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d63ab20 0x55b59d626fb0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb108009d90 tx=0x7fb108030000 comp rx=0 tx=0).stop 2024-04-04T06:15:58.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.785+0000 7fb11a236740 1 -- 172.21.15.159:0/3896214221 shutdown_connections 2024-04-04T06:15:58.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.785+0000 7fb11a236740 1 --2- 172.21.15.159:0/3896214221 >> [v2:172.21.15.159:6834/2253383212,v1:172.21.15.159:6835/2253383212] conn(0x55b59d6588c0 0x55b59d6274f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.785+0000 7fb11a236740 1 --2- 172.21.15.159:0/3896214221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b59d63ab20 0x55b59d626fb0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.785+0000 7fb11a236740 1 --2- 172.21.15.159:0/3896214221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b59d63a740 0x55b59d65a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.785+0000 7fb11a236740 1 --2- 172.21.15.159:0/3896214221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b59d6363b0 0x55b59d659cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.785+0000 7fb0fd600640 0 client.8526 ms_handle_reset on v2:172.21.15.159:6834/2253383212 2024-04-04T06:15:58.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.785+0000 7fb11a236740 1 -- 172.21.15.159:0/3896214221 >> 172.21.15.159:0/3896214221 conn(0x55b59d60dc20 msgr2=0x55b59d65b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:58.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.785+0000 7fb11a236740 1 -- 172.21.15.159:0/3896214221 shutdown_connections 2024-04-04T06:15:58.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.785+0000 7fb11a236740 1 -- 172.21.15.159:0/3896214221 wait complete. 2024-04-04T06:15:58.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.785+0000 7fb11a236740 20 client.8526 trim_cache size 0 max 16384 2024-04-04T06:15:58.786 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:15:58.786 INFO:teuthology.orchestra.run.smithi159.stdout: "status": "no active scrubs running", 2024-04-04T06:15:58.786 INFO:teuthology.orchestra.run.smithi159.stdout: "scrubs": {} 2024-04-04T06:15:58.786 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:15:58.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.786+0000 7fb115e00640 1 -- 172.21.15.159:0/2673745318 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb0ec06c8f0 msgr2=0x7fb0ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:58.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.786+0000 7fb115e00640 1 --2- 172.21.15.159:0/2673745318 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb0ec06c8f0 0x7fb0ec06edb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb1100b6570 tx=0x7fb0f80079b0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.786+0000 7fb115e00640 1 -- 172.21.15.159:0/2673745318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1100b69c0 msgr2=0x7fb110154390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:58.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.786+0000 7fb115e00640 1 --2- 172.21.15.159:0/2673745318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1100b69c0 0x7fb110154390 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fb10000c460 tx=0x7fb10000c920 comp rx=0 tx=0).stop 2024-04-04T06:15:58.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.786+0000 7fb115e00640 1 -- 172.21.15.159:0/2673745318 shutdown_connections 2024-04-04T06:15:58.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.786+0000 7fb115e00640 1 --2- 172.21.15.159:0/2673745318 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb0ec06c8f0 0x7fb0ec06edb0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.786+0000 7fb115e00640 1 --2- 172.21.15.159:0/2673745318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb11015e4c0 0x7fb1100cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.786+0000 7fb115e00640 1 --2- 172.21.15.159:0/2673745318 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1100d37b0 0x7fb110160d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.786+0000 7fb115e00640 1 --2- 172.21.15.159:0/2673745318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1100b69c0 0x7fb110154390 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:58.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.786+0000 7fb115e00640 1 -- 172.21.15.159:0/2673745318 >> 172.21.15.159:0/2673745318 conn(0x7fb1100676a0 msgr2=0x7fb11015cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:58.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.786+0000 7fb115e00640 1 -- 172.21.15.159:0/2673745318 shutdown_connections 2024-04-04T06:15:58.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:58.786+0000 7fb115e00640 1 -- 172.21.15.159:0/2673745318 wait complete. 2024-04-04T06:15:58.798 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-04T06:15:58.899 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-04T06:15:59.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.125+0000 7f9141a00640 1 Processor -- start 2024-04-04T06:15:59.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.126+0000 7f9141a00640 1 -- start start 2024-04-04T06:15:59.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.127+0000 7f9141a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f913c15da60 0x7f913c0cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.127+0000 7f9141a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f913c15fe90 0x7f913c0d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.127+0000 7f9141a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f913c15d060 con 0x7f913c15f900 2024-04-04T06:15:59.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.127+0000 7f9141a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f913c15d340 con 0x7f913c15da60 2024-04-04T06:15:59.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.127+0000 7f9141a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f913c15d620 con 0x7f913c15fe90 2024-04-04T06:15:59.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.127+0000 7f9132000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f913c15da60 0x7f913c0cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.128+0000 7f9132000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f913c15da60 0x7f913c0cfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52520/0 (socket says 172.21.15.159:52520) 2024-04-04T06:15:59.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.128+0000 7f9132000640 1 -- 172.21.15.159:0/820083351 learned_addr learned my addr 172.21.15.159:0/820083351 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:59.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.128+0000 7f913b400640 1 --2- 172.21.15.159:0/820083351 >> v2:172.21.15.178:3300/0 conn(0x7f913c15fe90 0x7f913c0d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.128+0000 7f9132000640 1 -- 172.21.15.159:0/820083351 >> v2:172.21.15.178:3300/0 conn(0x7f913c15fe90 msgr2=0x7f913c0d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:59.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.128+0000 7f9132000640 1 --2- 172.21.15.159:0/820083351 >> v2:172.21.15.178:3300/0 conn(0x7f913c15fe90 0x7f913c0d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.128+0000 7f9132000640 1 -- 172.21.15.159:0/820083351 >> v1:172.21.15.178:6789/0 conn(0x7f913c15f900 legacy=0x7f913c15c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:59.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.128+0000 7f9132000640 1 -- 172.21.15.159:0/820083351 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913c0c74b0 con 0x7f913c15da60 2024-04-04T06:15:59.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.128+0000 7f913b400640 1 --2- 172.21.15.159:0/820083351 >> v2:172.21.15.178:3300/0 conn(0x7f913c15fe90 0x7f913c0d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:59.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.129+0000 7f9132000640 1 --2- 172.21.15.159:0/820083351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f913c15da60 0x7f913c0cfe30 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f913c0b1d90 tx=0x7f912402b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d585d2f333cab2c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:59.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.129+0000 7f913a000640 1 -- 172.21.15.159:0/820083351 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9124035020 con 0x7f913c15da60 2024-04-04T06:15:59.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.129+0000 7f913a000640 1 -- 172.21.15.159:0/820083351 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9124031890 con 0x7f913c15da60 2024-04-04T06:15:59.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.129+0000 7f913a000640 1 -- 172.21.15.159:0/820083351 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9124031c60 con 0x7f913c15da60 2024-04-04T06:15:59.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.130+0000 7f9141a00640 1 -- 172.21.15.159:0/820083351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f913c15da60 msgr2=0x7f913c0cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.130+0000 7f9141a00640 1 --2- 172.21.15.159:0/820083351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f913c15da60 0x7f913c0cfe30 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f913c0b1d90 tx=0x7f912402b930 comp rx=0 tx=0).stop 2024-04-04T06:15:59.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.130+0000 7f9141a00640 1 -- 172.21.15.159:0/820083351 shutdown_connections 2024-04-04T06:15:59.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.130+0000 7f9141a00640 1 --2- 172.21.15.159:0/820083351 >> v2:172.21.15.178:3300/0 conn(0x7f913c15fe90 0x7f913c0d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.130+0000 7f9141a00640 1 --2- 172.21.15.159:0/820083351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f913c15da60 0x7f913c0cfe30 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.130+0000 7f9141a00640 1 -- 172.21.15.159:0/820083351 >> 172.21.15.159:0/820083351 conn(0x7f913c0b3e90 msgr2=0x7f913c0b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:59.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.130+0000 7f9141a00640 1 -- 172.21.15.159:0/820083351 shutdown_connections 2024-04-04T06:15:59.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.130+0000 7f9141a00640 1 -- 172.21.15.159:0/820083351 wait complete. 2024-04-04T06:15:59.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.133+0000 7f9141a00640 1 Processor -- start 2024-04-04T06:15:59.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.133+0000 7f9141a00640 1 -- start start 2024-04-04T06:15:59.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.135+0000 7f9141a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f913c15da60 0x7f913c14ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.135+0000 7f9141a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f913c15f900 0x7f913c0c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.135+0000 7f9141a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f913c15fe90 0x7f913c0ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.135+0000 7f9141a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f913c0cece0 con 0x7f913c15f900 2024-04-04T06:15:59.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.135+0000 7f9141a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f913c0ceb60 con 0x7f913c15da60 2024-04-04T06:15:59.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.135+0000 7f9141a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f913c0cee60 con 0x7f913c15fe90 2024-04-04T06:15:59.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.135+0000 7f9132000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f913c15f900 0x7f913c0c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.135+0000 7f9132000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f913c15f900 0x7f913c0c9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52526/0 (socket says 172.21.15.159:52526) 2024-04-04T06:15:59.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.135+0000 7f9132000640 1 -- 172.21.15.159:0/1892171288 learned_addr learned my addr 172.21.15.159:0/1892171288 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:59.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.135+0000 7f913aa00640 1 --2- 172.21.15.159:0/1892171288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f913c15da60 0x7f913c14ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.135+0000 7f9132000640 1 -- 172.21.15.159:0/1892171288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f913c15fe90 msgr2=0x7f913c0ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.135+0000 7f913b400640 1 --2- 172.21.15.159:0/1892171288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f913c15fe90 0x7f913c0ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.135+0000 7f9132000640 1 --2- 172.21.15.159:0/1892171288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f913c15fe90 0x7f913c0ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.135+0000 7f9132000640 1 -- 172.21.15.159:0/1892171288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f913c15da60 msgr2=0x7f913c14ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.135+0000 7f9132000640 1 --2- 172.21.15.159:0/1892171288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f913c15da60 0x7f913c14ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.135+0000 7f9132000640 1 -- 172.21.15.159:0/1892171288 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913c1f2010 con 0x7f913c15f900 2024-04-04T06:15:59.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.135+0000 7f913aa00640 1 --2- 172.21.15.159:0/1892171288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f913c15da60 0x7f913c14ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:59.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.135+0000 7f913b400640 1 --2- 172.21.15.159:0/1892171288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f913c15fe90 0x7f913c0ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:59.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.136+0000 7f9132000640 1 --2- 172.21.15.159:0/1892171288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f913c15f900 0x7f913c0c9ca0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f9124000c00 tx=0x7f91240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:59.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.136+0000 7f9133400640 1 -- 172.21.15.159:0/1892171288 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9124040070 con 0x7f913c15f900 2024-04-04T06:15:59.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.136+0000 7f9133400640 1 -- 172.21.15.159:0/1892171288 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f912403a420 con 0x7f913c15f900 2024-04-04T06:15:59.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.136+0000 7f9133400640 1 -- 172.21.15.159:0/1892171288 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9124035040 con 0x7f913c15f900 2024-04-04T06:15:59.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.136+0000 7f9141a00640 1 -- 172.21.15.159:0/1892171288 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f913c1f22f0 con 0x7f913c15f900 2024-04-04T06:15:59.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.136+0000 7f9141a00640 1 -- 172.21.15.159:0/1892171288 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f913c1f28b0 con 0x7f913c15f900 2024-04-04T06:15:59.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.137+0000 7f9141a00640 1 -- 172.21.15.159:0/1892171288 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f913c068cd0 con 0x7f913c15f900 2024-04-04T06:15:59.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.138+0000 7f9133400640 1 -- 172.21.15.159:0/1892171288 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f912404b060 con 0x7f913c15f900 2024-04-04T06:15:59.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.139+0000 7f9133400640 1 --2- 172.21.15.159:0/1892171288 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f911806c8f0 0x7f911806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.139+0000 7f9133400640 1 -- 172.21.15.159:0/1892171288 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f91240adc90 con 0x7f913c15f900 2024-04-04T06:15:59.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.140+0000 7f913aa00640 1 --2- 172.21.15.159:0/1892171288 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f911806c8f0 0x7f911806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.141+0000 7f913aa00640 1 --2- 172.21.15.159:0/1892171288 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f911806c8f0 0x7f911806edb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f912c007fd0 tx=0x7f912c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:59.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.144+0000 7f9133400640 1 -- 172.21.15.159:0/1892171288 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f912402d040 con 0x7f913c15f900 2024-04-04T06:15:59.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.383+0000 7f9141a00640 1 -- 172.21.15.159:0/1892171288 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f913c0cf430 con 0x7f913c15f900 2024-04-04T06:15:59.378 INFO:tasks.ceph.mon.a.smithi159.stderr:2024-04-04T06:15:59.384+0000 7f1995c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-04T06:15:59.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.387+0000 7f9133400640 1 -- 172.21.15.159:0/1892171288 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v85) ==== 161+0+0 (secure 0 0 0) 0x7f912407c020 con 0x7f913c15f900 2024-04-04T06:15:59.382 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-04T06:15:59.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.390+0000 7f9141a00640 1 -- 172.21.15.159:0/1892171288 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f911806c8f0 msgr2=0x7f911806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.390+0000 7f9141a00640 1 --2- 172.21.15.159:0/1892171288 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f911806c8f0 0x7f911806edb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f912c007fd0 tx=0x7f912c009040 comp rx=0 tx=0).stop 2024-04-04T06:15:59.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.391+0000 7f9141a00640 1 -- 172.21.15.159:0/1892171288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f913c15f900 msgr2=0x7f913c0c9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.391+0000 7f9141a00640 1 --2- 172.21.15.159:0/1892171288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f913c15f900 0x7f913c0c9ca0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f9124000c00 tx=0x7f91240046c0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.391+0000 7f9141a00640 1 -- 172.21.15.159:0/1892171288 shutdown_connections 2024-04-04T06:15:59.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.391+0000 7f9141a00640 1 --2- 172.21.15.159:0/1892171288 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f913c15fe90 0x7f913c0ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.391+0000 7f9141a00640 1 --2- 172.21.15.159:0/1892171288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f913c15f900 0x7f913c0c9ca0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.391+0000 7f9141a00640 1 --2- 172.21.15.159:0/1892171288 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f911806c8f0 0x7f911806edb0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.391+0000 7f9141a00640 1 --2- 172.21.15.159:0/1892171288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f913c15da60 0x7f913c14ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.391+0000 7f9141a00640 1 -- 172.21.15.159:0/1892171288 >> 172.21.15.159:0/1892171288 conn(0x7f913c0b3e90 msgr2=0x7f913c0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:59.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.392+0000 7f9141a00640 1 -- 172.21.15.159:0/1892171288 shutdown_connections 2024-04-04T06:15:59.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.392+0000 7f9141a00640 1 -- 172.21.15.159:0/1892171288 wait complete. 2024-04-04T06:15:59.406 DEBUG:teuthology.orchestra.run.smithi159:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-04-04T06:15:59.406 INFO:tasks.ceph.mds.a.smithi159.stderr: -7066> 2024-04-04T06:15:12.635+0000 7f1bda400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:15:59.478 INFO:tasks.ceph.mds.a.smithi159.stdout:starting mds.a at 2024-04-04T06:15:59.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.499+0000 7f4ae9639280 1 -- start start 2024-04-04T06:15:59.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.499+0000 7f4ae9639280 1 Processor -- start 2024-04-04T06:15:59.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.500+0000 7f4ae9639280 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fae932c00 0x563faf6fa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.500+0000 7f4ae9639280 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fae933800 0x563faf6fa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.500+0000 7f4ae9639280 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563fae933c00 0x563faf6fab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.500+0000 7f4ae9639280 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563faf70c180 con 0x563fae932c00 2024-04-04T06:15:59.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.500+0000 7f4ae9639280 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563faf70c300 con 0x563fae933800 2024-04-04T06:15:59.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.500+0000 7f4ae9639280 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x563faf70c480 con 0x563fae933c00 2024-04-04T06:15:59.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.500+0000 7f4ae7000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fae932c00 0x563faf6fa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.501+0000 7f4ae7000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fae932c00 0x563faf6fa000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52560/0 (socket says 172.21.15.159:52560) 2024-04-04T06:15:59.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.501+0000 7f4ae7000640 1 -- 172.21.15.159:0/3425401056 learned_addr learned my addr 172.21.15.159:0/3425401056 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:59.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.501+0000 7f4ae7000640 1 -- 172.21.15.159:0/3425401056 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563fae933c00 msgr2=0x563faf6fab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:59.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.501+0000 7f4ae7000640 1 --2- 172.21.15.159:0/3425401056 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563fae933c00 0x563faf6fab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.501+0000 7f4ae7000640 1 -- 172.21.15.159:0/3425401056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fae933800 msgr2=0x563faf6fa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:59.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.501+0000 7f4ae7000640 1 --2- 172.21.15.159:0/3425401056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fae933800 0x563faf6fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.501+0000 7f4ae7000640 1 -- 172.21.15.159:0/3425401056 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fae966780 con 0x563fae932c00 2024-04-04T06:15:59.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.501+0000 7f4ae7000640 1 --2- 172.21.15.159:0/3425401056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fae932c00 0x563faf6fa000 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x563faf74f3e0 tx=0x563faf6c44e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1142a2bc168991b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:59.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.502+0000 7f4ae3e00640 1 -- 172.21.15.159:0/3425401056 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563fae958680 con 0x563fae932c00 2024-04-04T06:15:59.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.502+0000 7f4ae3e00640 1 -- 172.21.15.159:0/3425401056 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563fae95e700 con 0x563fae932c00 2024-04-04T06:15:59.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.502+0000 7f4ae3e00640 1 -- 172.21.15.159:0/3425401056 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563fae958b60 con 0x563fae932c00 2024-04-04T06:15:59.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.502+0000 7f4ae9639280 1 -- 172.21.15.159:0/3425401056 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fae9674a0 con 0x563fae932c00 2024-04-04T06:15:59.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.502+0000 7f4ae9639280 1 -- 172.21.15.159:0/3425401056 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({fsmap=0}) -- 0x563fae966000 con 0x563fae932c00 2024-04-04T06:15:59.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.502+0000 7f4ae9639280 4 waiting for MDS map... 2024-04-04T06:15:59.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.502+0000 7f4ae3e00640 1 -- 172.21.15.159:0/3425401056 <== mon.0 v2:172.21.15.159:3300/0 4 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x563faf7f8600 con 0x563fae932c00 2024-04-04T06:15:59.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.502+0000 7f4ae3e00640 1 -- 172.21.15.159:0/3425401056 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 85) ==== 2885+0+0 (secure 0 0 0) 0x563fae98d180 con 0x563fae932c00 2024-04-04T06:15:59.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.502+0000 7f4ae9639280 4 Got MDS map 85 2024-04-04T06:15:59.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.502+0000 7f4ae9639280 10 main: JournalTool::main 2024-04-04T06:15:59.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.502+0000 7f4ae9639280 4 main: JournalTool: connecting to RADOS... 2024-04-04T06:15:59.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.503+0000 7f4ae9639280 1 Processor -- start 2024-04-04T06:15:59.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.503+0000 7f4ae9639280 1 -- start start 2024-04-04T06:15:59.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.504+0000 7f4ae9639280 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563faf7fb400 0x563faf6fb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.504+0000 7f4ae9639280 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563faf7fb800 0x563faf6fb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.504+0000 7f4ae9639280 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563faf7fbc00 0x563faf6fbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.504+0000 7f4ae9639280 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563faf70c600 con 0x563faf7fbc00 2024-04-04T06:15:59.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.504+0000 7f4ae9639280 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563faf70c780 con 0x563faf7fb400 2024-04-04T06:15:59.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.504+0000 7f4ae9639280 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x563faf70c900 con 0x563faf7fb800 2024-04-04T06:15:59.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.504+0000 7f4ae6600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563faf7fbc00 0x563faf6fbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.504+0000 7f4ae6600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563faf7fbc00 0x563faf6fbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52570/0 (socket says 172.21.15.159:52570) 2024-04-04T06:15:59.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.504+0000 7f4ae6600640 1 -- 172.21.15.159:0/1023065293 learned_addr learned my addr 172.21.15.159:0/1023065293 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:59.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.505+0000 7f4ae6600640 1 -- 172.21.15.159:0/1023065293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563faf7fb800 msgr2=0x563faf6fb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:59.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.505+0000 7f4ae6600640 1 --2- 172.21.15.159:0/1023065293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563faf7fb800 0x563faf6fb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.505+0000 7f4ae6600640 1 -- 172.21.15.159:0/1023065293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563faf7fb400 msgr2=0x563faf6fb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:59.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.505+0000 7f4ae6600640 1 --2- 172.21.15.159:0/1023065293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563faf7fb400 0x563faf6fb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.505+0000 7f4ae6600640 1 -- 172.21.15.159:0/1023065293 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fae9663c0 con 0x563faf7fbc00 2024-04-04T06:15:59.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.505+0000 7f4ae6600640 1 --2- 172.21.15.159:0/1023065293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563faf7fbc00 0x563faf6fbb80 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x563faf812660 tx=0x563faf816cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5e9d2ce9c8c8e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:59.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.505+0000 7f4ae1600640 1 -- 172.21.15.159:0/1023065293 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563fae958680 con 0x563faf7fbc00 2024-04-04T06:15:59.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.505+0000 7f4ae1600640 1 -- 172.21.15.159:0/1023065293 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563fae95efc0 con 0x563faf7fbc00 2024-04-04T06:15:59.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.505+0000 7f4ae1600640 1 -- 172.21.15.159:0/1023065293 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563fae958b60 con 0x563faf7fbc00 2024-04-04T06:15:59.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.506+0000 7f4ae9639280 1 -- 172.21.15.159:0/1023065293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563faf7fbc00 msgr2=0x563faf6fbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.506+0000 7f4ae9639280 1 --2- 172.21.15.159:0/1023065293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563faf7fbc00 0x563faf6fbb80 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x563faf812660 tx=0x563faf816cc0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.506+0000 7f4ae9639280 1 -- 172.21.15.159:0/1023065293 shutdown_connections 2024-04-04T06:15:59.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.506+0000 7f4ae9639280 1 --2- 172.21.15.159:0/1023065293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563faf7fbc00 0x563faf6fbb80 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.506+0000 7f4ae9639280 1 --2- 172.21.15.159:0/1023065293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563faf7fb800 0x563faf6fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.506+0000 7f4ae9639280 1 --2- 172.21.15.159:0/1023065293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563faf7fb400 0x563faf6fb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.506+0000 7f4ae9639280 1 -- 172.21.15.159:0/1023065293 >> 172.21.15.159:0/1023065293 conn(0x563faf7fb000 msgr2=0x563faea1b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:59.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.506+0000 7f4ae9639280 1 -- 172.21.15.159:0/1023065293 shutdown_connections 2024-04-04T06:15:59.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.506+0000 7f4ae9639280 1 -- 172.21.15.159:0/1023065293 wait complete. 2024-04-04T06:15:59.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.506+0000 7f4ae9639280 1 Processor -- start 2024-04-04T06:15:59.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.507+0000 7f4ae9639280 1 -- start start 2024-04-04T06:15:59.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.507+0000 7f4ae9639280 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563faf7fb400 0x563faf6fb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.507+0000 7f4ae9639280 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563faf7fb800 0x563faf6fb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.507+0000 7f4ae9639280 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563faf7fbc00 0x563faf6fbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.507+0000 7f4ae9639280 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563faf70dc80 con 0x563faf7fb800 2024-04-04T06:15:59.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.507+0000 7f4ae9639280 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563faf70db00 con 0x563faf7fbc00 2024-04-04T06:15:59.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.507+0000 7f4ae9639280 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x563faf70d980 con 0x563faf7fb400 2024-04-04T06:15:59.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.508+0000 7f4ae7000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563faf7fb800 0x563faf6fb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.508+0000 7f4ae7000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563faf7fb800 0x563faf6fb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52580/0 (socket says 172.21.15.159:52580) 2024-04-04T06:15:59.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.508+0000 7f4ae7000640 1 -- 172.21.15.159:0/2534127941 learned_addr learned my addr 172.21.15.159:0/2534127941 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:59.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.508+0000 7f4ae7000640 1 -- 172.21.15.159:0/2534127941 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563faf7fb400 msgr2=0x563faf6fb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.508+0000 7f4ae7000640 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563faf7fb400 0x563faf6fb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.508+0000 7f4ae6600640 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563faf7fbc00 0x563faf6fbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.508+0000 7f4ae7000640 1 -- 172.21.15.159:0/2534127941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563faf7fbc00 msgr2=0x563faf6fbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.508+0000 7f4ae7000640 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563faf7fbc00 0x563faf6fbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.508+0000 7f4ae7000640 1 -- 172.21.15.159:0/2534127941 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fae966780 con 0x563faf7fb800 2024-04-04T06:15:59.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.508+0000 7f4ae7000640 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563faf7fb800 0x563faf6fb600 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x563faf813350 tx=0x563faf6b24e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:59.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.508+0000 7f4adf800640 1 -- 172.21.15.159:0/2534127941 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563fae958820 con 0x563faf7fb800 2024-04-04T06:15:59.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.508+0000 7f4adf800640 1 -- 172.21.15.159:0/2534127941 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563fae95efc0 con 0x563faf7fb800 2024-04-04T06:15:59.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.508+0000 7f4adf800640 1 -- 172.21.15.159:0/2534127941 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563fae958680 con 0x563faf7fb800 2024-04-04T06:15:59.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.509+0000 7f4ae9639280 1 -- 172.21.15.159:0/2534127941 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563fae9674a0 con 0x563faf7fb800 2024-04-04T06:15:59.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.509+0000 7f4ae9639280 1 -- 172.21.15.159:0/2534127941 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fae966960 con 0x563faf7fb800 2024-04-04T06:15:59.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.509+0000 7f4ae9639280 4 main: JournalTool: resolving pool 14 2024-04-04T06:15:59.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.510+0000 7f4adf800640 1 -- 172.21.15.159:0/2534127941 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x563faf87ac00 con 0x563faf7fb800 2024-04-04T06:15:59.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.511+0000 7f4adf800640 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x563faf873c00 0x563faf6fc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.511+0000 7f4adf800640 1 -- 172.21.15.159:0/2534127941 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x563faf874c00 con 0x563faf7fb800 2024-04-04T06:15:59.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.511+0000 7f4ae9639280 4 main: JournalTool: creating IoCtx.. 2024-04-04T06:15:59.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.511+0000 7f4ae9639280 4 main: Executing for rank 0 2024-04-04T06:15:59.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.511+0000 7f4ae9639280 4 Filtering by inode '1099511627777' 2024-04-04T06:15:59.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.511+0000 7f4ae9639280 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x563faec36000 0x563faf6fc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.511+0000 7f4ae9639280 1 -- 172.21.15.159:0/2534127941 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) -- 0x563faec36400 con 0x563faec36000 2024-04-04T06:15:59.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.512+0000 7f4ae7a00640 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x563faf873c00 0x563faf6fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.512+0000 7f4ae6600640 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x563faec36000 0x563faf6fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.513+0000 7f4ae7a00640 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x563faf873c00 0x563faf6fc100 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x563fae962f00 tx=0x563faf817d40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:59.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.513+0000 7f4ae6600640 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x563faec36000 0x563faf6fc680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:59.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.514+0000 7f4ae6600640 1 -- 172.21.15.159:0/2534127941 <== osd.7 v2:172.21.15.178:6810/1066817938 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x563faea2e000 con 0x563faec36000 2024-04-04T06:15:59.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.514+0000 7f4ae9639280 4 Pointer 400.00000000 is readable 2024-04-04T06:15:59.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.514+0000 7f4ae9639280 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-04T06:15:59.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.514+0000 7f4ae9639280 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x563faec36800 0x563faf6fcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.514+0000 7f4ae9639280 1 -- 172.21.15.159:0/2534127941 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) -- 0x563faec36c00 con 0x563faec36800 2024-04-04T06:15:59.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.515+0000 7f4ae7a00640 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x563faec36800 0x563faf6fcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.516+0000 7f4ae7a00640 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x563faec36800 0x563faf6fcc00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:59.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.522+0000 7f4ae7a00640 1 -- 172.21.15.159:0/2534127941 <== osd.4 v2:172.21.15.178:6812/4035991429 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv32 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x563faea2e500 con 0x563faec36800 2024-04-04T06:15:59.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.522+0000 7f4ae9639280 10 Header 0x400000 0x407d95 0x408111 2024-04-04T06:15:59.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.522+0000 7f4ae9639280 10 Starting journal scan from offset 0x407d95 2024-04-04T06:15:59.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.523+0000 7f4ae9639280 1 -- 172.21.15.159:0/2534127941 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 32149~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) -- 0x563faec37000 con 0x563faec36800 2024-04-04T06:15:59.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.524+0000 7f4ae7a00640 1 -- 172.21.15.159:0/2534127941 <== osd.4 v2:172.21.15.178:6812/4035991429 2 ==== osd_op_reply(3 200.00000001 [read 32149~892 out=892b] v0'0 uv29 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x563faea2e500 con 0x563faec36800 2024-04-04T06:15:59.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.524+0000 7f4ae9639280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-04T06:15:59.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.524+0000 7f4ae9639280 10 Parsing data, 0x37c bytes available 2024-04-04T06:15:59.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.524+0000 7f4ae9639280 10 Attempting decode at 0x407d95 2024-04-04T06:15:59.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.524+0000 7f4ae9639280 10 Consumed 0x37c bytes 2024-04-04T06:15:59.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.524+0000 7f4ae9639280 10 Valid entry at 0x407d95 2024-04-04T06:15:59.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.524+0000 7f4ae9639280 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x563faec36400 0x563faf6fd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.524+0000 7f4ae9639280 1 -- 172.21.15.159:0/2534127941 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) -- 0x563faec37400 con 0x563faec36400 2024-04-04T06:15:59.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.524+0000 7f4ae7000640 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x563faec36400 0x563faf6fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.525+0000 7f4ae7000640 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x563faec36400 0x563faf6fd180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:59.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.527+0000 7f4ae7000640 1 -- 172.21.15.159:0/2534127941 <== osd.1 v2:172.21.15.159:6816/3765338297 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x563faea2e780 con 0x563faec36400 2024-04-04T06:15:59.522 INFO:teuthology.orchestra.run.smithi159.stdout:Events by type: 2024-04-04T06:15:59.522 INFO:teuthology.orchestra.run.smithi159.stdout:Errors: 0 2024-04-04T06:15:59.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.527+0000 7f4ae9639280 4 Reached end of journal objects 2024-04-04T06:15:59.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.527+0000 7f4ae9639280 4 Scanned objects, 0 missing, 1 valid 2024-04-04T06:15:59.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.527+0000 7f4ae9639280 4 Events scanned, 0 gaps 2024-04-04T06:15:59.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.527+0000 7f4ae9639280 4 Found 1 valid events 2024-04-04T06:15:59.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.527+0000 7f4ae9639280 4 Selected 0 events events for processing 2024-04-04T06:15:59.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.527+0000 7f4ae9639280 4 0 events 2024-04-04T06:15:59.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.527+0000 7f4ae9639280 1 -- 172.21.15.159:0/2534127941 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x563faec36400 msgr2=0x563faf6fd180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.527+0000 7f4ae9639280 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x563faec36400 0x563faf6fd180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.527+0000 7f4ae9639280 1 -- 172.21.15.159:0/2534127941 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x563faec36800 msgr2=0x563faf6fcc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.527+0000 7f4ae9639280 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x563faec36800 0x563faf6fcc00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.527+0000 7f4ae9639280 1 -- 172.21.15.159:0/2534127941 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x563faec36000 msgr2=0x563faf6fc680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.527+0000 7f4ae9639280 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x563faec36000 0x563faf6fc680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.528+0000 7f4ae6600640 1 -- 172.21.15.159:0/2534127941 reap_dead start 2024-04-04T06:15:59.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.528+0000 7f4ae9639280 1 -- 172.21.15.159:0/2534127941 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x563faf873c00 msgr2=0x563faf6fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.528+0000 7f4ae9639280 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x563faf873c00 0x563faf6fc100 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x563fae962f00 tx=0x563faf817d40 comp rx=0 tx=0).stop 2024-04-04T06:15:59.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.528+0000 7f4ae9639280 1 -- 172.21.15.159:0/2534127941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563faf7fb800 msgr2=0x563faf6fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.528+0000 7f4ae9639280 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563faf7fb800 0x563faf6fb600 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x563faf813350 tx=0x563faf6b24e0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.528+0000 7f4ae9639280 1 -- 172.21.15.159:0/2534127941 shutdown_connections 2024-04-04T06:15:59.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.528+0000 7f4ae9639280 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x563faf873c00 0x563faf6fc100 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.528+0000 7f4ae9639280 1 --2- 172.21.15.159:0/2534127941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563faf7fb800 0x563faf6fb600 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.528+0000 7f4ae9639280 1 -- 172.21.15.159:0/2534127941 >> 172.21.15.159:0/2534127941 conn(0x563faf7fb000 msgr2=0x563faea1b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:59.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.528+0000 7f4ae9639280 1 -- 172.21.15.159:0/2534127941 shutdown_connections 2024-04-04T06:15:59.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.528+0000 7f4ae9639280 1 -- 172.21.15.159:0/2534127941 wait complete. 2024-04-04T06:15:59.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.528+0000 7f4ae9639280 1 -- 172.21.15.159:0/3425401056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fae932c00 msgr2=0x563faf6fa000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.528+0000 7f4ae9639280 1 --2- 172.21.15.159:0/3425401056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fae932c00 0x563faf6fa000 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x563faf74f3e0 tx=0x563faf6c44e0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.528+0000 7f4ae9639280 1 -- 172.21.15.159:0/3425401056 shutdown_connections 2024-04-04T06:15:59.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.528+0000 7f4ae9639280 1 --2- 172.21.15.159:0/3425401056 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563fae933c00 0x563faf6fab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.528+0000 7f4ae9639280 1 --2- 172.21.15.159:0/3425401056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fae933800 0x563faf6fa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.528+0000 7f4ae9639280 1 --2- 172.21.15.159:0/3425401056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fae932c00 0x563faf6fa000 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.528+0000 7f4ae9639280 1 -- 172.21.15.159:0/3425401056 >> 172.21.15.159:0/3425401056 conn(0x563fae933000 msgr2=0x563faea1a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:59.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.529+0000 7f4ae9639280 1 -- 172.21.15.159:0/3425401056 shutdown_connections 2024-04-04T06:15:59.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.529+0000 7f4ae9639280 1 -- 172.21.15.159:0/3425401056 wait complete. 2024-04-04T06:15:59.528 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.122812, result: Events by type: Errors: 0 2024-04-04T06:15:59.528 DEBUG:teuthology.orchestra.run.smithi159:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-04-04T06:15:59.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.616+0000 7f1d5d40d280 1 -- start start 2024-04-04T06:15:59.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.616+0000 7f1d5d40d280 1 Processor -- start 2024-04-04T06:15:59.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.616+0000 7f1d5d40d280 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576981a0400 0x55769819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.616+0000 7f1d5d40d280 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576981a0800 0x55769819d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.616+0000 7f1d5d40d280 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5576981a1c00 0x55769819c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.617+0000 7f1d5d40d280 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5576981ce000 con 0x5576981a0800 2024-04-04T06:15:59.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.617+0000 7f1d5d40d280 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5576981ce180 con 0x5576981a0400 2024-04-04T06:15:59.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.617+0000 7f1d5d40d280 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5576981ce300 con 0x5576981a1c00 2024-04-04T06:15:59.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.617+0000 7f1d59e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576981a0800 0x55769819d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.617+0000 7f1d59e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576981a0800 0x55769819d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52598/0 (socket says 172.21.15.159:52598) 2024-04-04T06:15:59.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.617+0000 7f1d59e00640 1 -- 172.21.15.159:0/4005072344 learned_addr learned my addr 172.21.15.159:0/4005072344 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:59.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.617+0000 7f1d5a800640 1 --2- 172.21.15.159:0/4005072344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576981a0400 0x55769819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.617+0000 7f1d5b200640 1 --2- 172.21.15.159:0/4005072344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5576981a1c00 0x55769819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.617+0000 7f1d59e00640 1 -- 172.21.15.159:0/4005072344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5576981a1c00 msgr2=0x55769819c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:59.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.617+0000 7f1d59e00640 1 --2- 172.21.15.159:0/4005072344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5576981a1c00 0x55769819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.617+0000 7f1d59e00640 1 -- 172.21.15.159:0/4005072344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576981a0400 msgr2=0x55769819cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:59.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.617+0000 7f1d59e00640 1 --2- 172.21.15.159:0/4005072344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576981a0400 0x55769819cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.617+0000 7f1d59e00640 1 -- 172.21.15.159:0/4005072344 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557697408780 con 0x5576981a0800 2024-04-04T06:15:59.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.617+0000 7f1d5a800640 1 --2- 172.21.15.159:0/4005072344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576981a0400 0x55769819cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:59.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.618+0000 7f1d59e00640 1 --2- 172.21.15.159:0/4005072344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576981a0800 0x55769819d080 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x5576981f3410 tx=0x557698168120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45234c910c8d2d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:59.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.618+0000 7f1d57600640 1 -- 172.21.15.159:0/4005072344 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5576973fa9c0 con 0x5576981a0800 2024-04-04T06:15:59.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.618+0000 7f1d57600640 1 -- 172.21.15.159:0/4005072344 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x557697400e00 con 0x5576981a0800 2024-04-04T06:15:59.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.618+0000 7f1d57600640 1 -- 172.21.15.159:0/4005072344 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5576973fab60 con 0x5576981a0800 2024-04-04T06:15:59.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.618+0000 7f1d5d40d280 1 -- 172.21.15.159:0/4005072344 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557697408000 con 0x5576981a0800 2024-04-04T06:15:59.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.618+0000 7f1d5d40d280 1 -- 172.21.15.159:0/4005072344 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({fsmap=0}) -- 0x557697409a40 con 0x5576981a0800 2024-04-04T06:15:59.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.618+0000 7f1d5d40d280 4 waiting for MDS map... 2024-04-04T06:15:59.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.618+0000 7f1d57600640 1 -- 172.21.15.159:0/4005072344 <== mon.0 v2:172.21.15.159:3300/0 4 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x5576981d4400 con 0x5576981a0800 2024-04-04T06:15:59.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.618+0000 7f1d57600640 1 -- 172.21.15.159:0/4005072344 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 85) ==== 2885+0+0 (secure 0 0 0) 0x55769742f180 con 0x5576981a0800 2024-04-04T06:15:59.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.618+0000 7f1d5d40d280 4 Got MDS map 85 2024-04-04T06:15:59.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.618+0000 7f1d5d40d280 10 main: JournalTool::main 2024-04-04T06:15:59.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.618+0000 7f1d5d40d280 4 main: JournalTool: connecting to RADOS... 2024-04-04T06:15:59.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.619+0000 7f1d5d40d280 1 Processor -- start 2024-04-04T06:15:59.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.619+0000 7f1d5d40d280 1 -- start start 2024-04-04T06:15:59.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.620+0000 7f1d5d40d280 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576981a1800 0x55769819c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.620+0000 7f1d5d40d280 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557698297000 0x55769819d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.620+0000 7f1d5d40d280 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557698297400 0x55769819db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.620+0000 7f1d5d40d280 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5576981ce480 con 0x557698297000 2024-04-04T06:15:59.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.620+0000 7f1d5d40d280 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5576981ce600 con 0x5576981a1800 2024-04-04T06:15:59.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.620+0000 7f1d5d40d280 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5576981ce780 con 0x557698297400 2024-04-04T06:15:59.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.620+0000 7f1d5a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557698297000 0x55769819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.620+0000 7f1d5a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557698297000 0x55769819d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52604/0 (socket says 172.21.15.159:52604) 2024-04-04T06:15:59.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.620+0000 7f1d5a800640 1 -- 172.21.15.159:0/1745585952 learned_addr learned my addr 172.21.15.159:0/1745585952 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:59.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.621+0000 7f1d5a800640 1 -- 172.21.15.159:0/1745585952 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557698297400 msgr2=0x55769819db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:59.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.621+0000 7f1d5a800640 1 --2- 172.21.15.159:0/1745585952 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557698297400 0x55769819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.621+0000 7f1d5a800640 1 -- 172.21.15.159:0/1745585952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576981a1800 msgr2=0x55769819c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:59.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.621+0000 7f1d5a800640 1 --2- 172.21.15.159:0/1745585952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576981a1800 0x55769819c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.621+0000 7f1d5a800640 1 -- 172.21.15.159:0/1745585952 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557697409860 con 0x557698297000 2024-04-04T06:15:59.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.621+0000 7f1d5a800640 1 --2- 172.21.15.159:0/1745585952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557698297000 0x55769819d600 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x5576982a0900 tx=0x5576982a4cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2cc507d7c46be4 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:59.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.621+0000 7f1d54e00640 1 -- 172.21.15.159:0/1745585952 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5576973fa9c0 con 0x557698297000 2024-04-04T06:15:59.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.621+0000 7f1d54e00640 1 -- 172.21.15.159:0/1745585952 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x557697400700 con 0x557698297000 2024-04-04T06:15:59.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.621+0000 7f1d54e00640 1 -- 172.21.15.159:0/1745585952 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5576973fab60 con 0x557698297000 2024-04-04T06:15:59.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.622+0000 7f1d5d40d280 1 -- 172.21.15.159:0/1745585952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557698297000 msgr2=0x55769819d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.622+0000 7f1d5d40d280 1 --2- 172.21.15.159:0/1745585952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557698297000 0x55769819d600 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x5576982a0900 tx=0x5576982a4cc0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.622+0000 7f1d5d40d280 1 -- 172.21.15.159:0/1745585952 shutdown_connections 2024-04-04T06:15:59.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.622+0000 7f1d5d40d280 1 --2- 172.21.15.159:0/1745585952 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557698297400 0x55769819db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.622+0000 7f1d5d40d280 1 --2- 172.21.15.159:0/1745585952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557698297000 0x55769819d600 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.622+0000 7f1d5d40d280 1 --2- 172.21.15.159:0/1745585952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576981a1800 0x55769819c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.622+0000 7f1d5d40d280 1 -- 172.21.15.159:0/1745585952 >> 172.21.15.159:0/1745585952 conn(0x5576981a1400 msgr2=0x557698153800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:59.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.622+0000 7f1d5d40d280 1 -- 172.21.15.159:0/1745585952 shutdown_connections 2024-04-04T06:15:59.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.622+0000 7f1d5d40d280 1 -- 172.21.15.159:0/1745585952 wait complete. 2024-04-04T06:15:59.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.622+0000 7f1d5d40d280 1 Processor -- start 2024-04-04T06:15:59.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.623+0000 7f1d5d40d280 1 -- start start 2024-04-04T06:15:59.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.623+0000 7f1d5d40d280 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576981a1800 0x55769819c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.623+0000 7f1d5d40d280 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557698297000 0x55769819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.623+0000 7f1d5d40d280 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557698297400 0x55769819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.623+0000 7f1d5d40d280 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5576981cfe00 con 0x557698297000 2024-04-04T06:15:59.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.623+0000 7f1d5d40d280 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5576981cfc80 con 0x5576981a1800 2024-04-04T06:15:59.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.623+0000 7f1d5d40d280 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5576981cfb00 con 0x557698297400 2024-04-04T06:15:59.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.624+0000 7f1d5a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557698297000 0x55769819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.624+0000 7f1d5a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557698297000 0x55769819d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52606/0 (socket says 172.21.15.159:52606) 2024-04-04T06:15:59.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.624+0000 7f1d5a800640 1 -- 172.21.15.159:0/266353656 learned_addr learned my addr 172.21.15.159:0/266353656 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:59.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.624+0000 7f1d5b200640 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576981a1800 0x55769819c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.624+0000 7f1d59e00640 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557698297400 0x55769819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.624+0000 7f1d5a800640 1 -- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557698297400 msgr2=0x55769819db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.624+0000 7f1d5a800640 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557698297400 0x55769819db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.624+0000 7f1d5a800640 1 -- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576981a1800 msgr2=0x55769819c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.624+0000 7f1d5a800640 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576981a1800 0x55769819c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.624+0000 7f1d5a800640 1 -- 172.21.15.159:0/266353656 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576974094a0 con 0x557698297000 2024-04-04T06:15:59.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.624+0000 7f1d5a800640 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557698297000 0x55769819d600 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x5576982a15c0 tx=0x5576982a4c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:59.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.625+0000 7f1d53000640 1 -- 172.21.15.159:0/266353656 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5576973fa820 con 0x557698297000 2024-04-04T06:15:59.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.625+0000 7f1d53000640 1 -- 172.21.15.159:0/266353656 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5576974001c0 con 0x557698297000 2024-04-04T06:15:59.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.625+0000 7f1d5d40d280 1 -- 172.21.15.159:0/266353656 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557697408960 con 0x557698297000 2024-04-04T06:15:59.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.625+0000 7f1d53000640 1 -- 172.21.15.159:0/266353656 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5576973fa9c0 con 0x557698297000 2024-04-04T06:15:59.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.625+0000 7f1d5d40d280 1 -- 172.21.15.159:0/266353656 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557697408780 con 0x557698297000 2024-04-04T06:15:59.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.625+0000 7f1d5d40d280 4 main: JournalTool: resolving pool 14 2024-04-04T06:15:59.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.626+0000 7f1d53000640 1 -- 172.21.15.159:0/266353656 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x557698300c00 con 0x557698297000 2024-04-04T06:15:59.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.627+0000 7f1d53000640 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x557698306c00 0x55769819e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.627+0000 7f1d53000640 1 -- 172.21.15.159:0/266353656 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x5576982fea00 con 0x557698297000 2024-04-04T06:15:59.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.627+0000 7f1d5d40d280 4 main: JournalTool: creating IoCtx.. 2024-04-04T06:15:59.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.627+0000 7f1d5d40d280 4 main: Executing for rank 0 2024-04-04T06:15:59.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.627+0000 7f1d5d40d280 4 Filtering by inode '1099511627778' 2024-04-04T06:15:59.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.627+0000 7f1d5d40d280 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x557698297800 0x55769819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.627+0000 7f1d5d40d280 1 -- 172.21.15.159:0/266353656 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) -- 0x557698297c00 con 0x557698297800 2024-04-04T06:15:59.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.628+0000 7f1d5b200640 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x557698306c00 0x55769819e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.628+0000 7f1d59e00640 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x557698297800 0x55769819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.629+0000 7f1d59e00640 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x557698297800 0x55769819e680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:59.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.629+0000 7f1d5b200640 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x557698306c00 0x55769819e100 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x557698156840 tx=0x557698168180 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:59.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.630+0000 7f1d59e00640 1 -- 172.21.15.159:0/266353656 <== osd.7 v2:172.21.15.178:6810/1066817938 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x5576974b0000 con 0x557698297800 2024-04-04T06:15:59.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.630+0000 7f1d5d40d280 4 Pointer 400.00000000 is readable 2024-04-04T06:15:59.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.630+0000 7f1d5d40d280 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-04T06:15:59.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.630+0000 7f1d5d40d280 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x557698307000 0x55769819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.630+0000 7f1d5d40d280 1 -- 172.21.15.159:0/266353656 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) -- 0x557698307400 con 0x557698307000 2024-04-04T06:15:59.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.630+0000 7f1d5b200640 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x557698307000 0x55769819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.631+0000 7f1d5b200640 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x557698307000 0x55769819ec00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:59.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.632+0000 7f1d5b200640 1 -- 172.21.15.159:0/266353656 <== osd.4 v2:172.21.15.178:6812/4035991429 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv32 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x5576974b0500 con 0x557698307000 2024-04-04T06:15:59.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.632+0000 7f1d5d40d280 10 Header 0x400000 0x407d95 0x408111 2024-04-04T06:15:59.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.632+0000 7f1d5d40d280 10 Starting journal scan from offset 0x407d95 2024-04-04T06:15:59.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.632+0000 7f1d5d40d280 1 -- 172.21.15.159:0/266353656 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 32149~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) -- 0x557698307800 con 0x557698307000 2024-04-04T06:15:59.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.633+0000 7f1d5b200640 1 -- 172.21.15.159:0/266353656 <== osd.4 v2:172.21.15.178:6812/4035991429 2 ==== osd_op_reply(3 200.00000001 [read 32149~892 out=892b] v0'0 uv29 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x5576974b0500 con 0x557698307000 2024-04-04T06:15:59.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.633+0000 7f1d5d40d280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-04T06:15:59.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.633+0000 7f1d5d40d280 10 Parsing data, 0x37c bytes available 2024-04-04T06:15:59.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.633+0000 7f1d5d40d280 10 Attempting decode at 0x407d95 2024-04-04T06:15:59.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.633+0000 7f1d5d40d280 10 Consumed 0x37c bytes 2024-04-04T06:15:59.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.633+0000 7f1d5d40d280 10 Valid entry at 0x407d95 2024-04-04T06:15:59.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.633+0000 7f1d5d40d280 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x557698307c00 0x55769819f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.633+0000 7f1d5d40d280 1 -- 172.21.15.159:0/266353656 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) -- 0x557697754000 con 0x557698307c00 2024-04-04T06:15:59.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.633+0000 7f1d5a800640 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x557698307c00 0x55769819f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5a800640 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x557698307c00 0x55769819f180 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:59.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5a800640 1 -- 172.21.15.159:0/266353656 <== osd.1 v2:172.21.15.159:6816/3765338297 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5576974b0780 con 0x557698307c00 2024-04-04T06:15:59.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 4 Reached end of journal objects 2024-04-04T06:15:59.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 4 Scanned objects, 0 missing, 1 valid 2024-04-04T06:15:59.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 4 Events scanned, 0 gaps 2024-04-04T06:15:59.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 4 Found 1 valid events 2024-04-04T06:15:59.629 INFO:teuthology.orchestra.run.smithi159.stdout:Events by type: 2024-04-04T06:15:59.629 INFO:teuthology.orchestra.run.smithi159.stdout:Errors: 0 2024-04-04T06:15:59.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 4 Selected 0 events events for processing 2024-04-04T06:15:59.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 4 0 events 2024-04-04T06:15:59.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 1 -- 172.21.15.159:0/266353656 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x557698307c00 msgr2=0x55769819f180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x557698307c00 0x55769819f180 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 1 -- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x557698307000 msgr2=0x55769819ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x557698307000 0x55769819ec00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 1 -- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x557698297800 msgr2=0x55769819e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x557698297800 0x55769819e680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5a800640 1 -- 172.21.15.159:0/266353656 reap_dead start 2024-04-04T06:15:59.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 1 -- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x557698306c00 msgr2=0x55769819e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x557698306c00 0x55769819e100 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x557698156840 tx=0x557698168180 comp rx=0 tx=0).stop 2024-04-04T06:15:59.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 1 -- 172.21.15.159:0/266353656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557698297000 msgr2=0x55769819d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557698297000 0x55769819d600 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x5576982a15c0 tx=0x5576982a4c00 comp rx=0 tx=0).stop 2024-04-04T06:15:59.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 1 -- 172.21.15.159:0/266353656 shutdown_connections 2024-04-04T06:15:59.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557698297000 0x55769819d600 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 1 --2- 172.21.15.159:0/266353656 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x557698306c00 0x55769819e100 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 1 -- 172.21.15.159:0/266353656 >> 172.21.15.159:0/266353656 conn(0x5576981a1400 msgr2=0x557698153800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:59.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 1 -- 172.21.15.159:0/266353656 shutdown_connections 2024-04-04T06:15:59.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.634+0000 7f1d5d40d280 1 -- 172.21.15.159:0/266353656 wait complete. 2024-04-04T06:15:59.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.635+0000 7f1d5d40d280 1 -- 172.21.15.159:0/4005072344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576981a0800 msgr2=0x55769819d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.635+0000 7f1d5d40d280 1 --2- 172.21.15.159:0/4005072344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576981a0800 0x55769819d080 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x5576981f3410 tx=0x557698168120 comp rx=0 tx=0).stop 2024-04-04T06:15:59.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.635+0000 7f1d5d40d280 1 -- 172.21.15.159:0/4005072344 shutdown_connections 2024-04-04T06:15:59.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.635+0000 7f1d5d40d280 1 --2- 172.21.15.159:0/4005072344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5576981a1c00 0x55769819c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.635+0000 7f1d5d40d280 1 --2- 172.21.15.159:0/4005072344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576981a0800 0x55769819d080 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.635+0000 7f1d5d40d280 1 --2- 172.21.15.159:0/4005072344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5576981a0400 0x55769819cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.635+0000 7f1d5d40d280 1 -- 172.21.15.159:0/4005072344 >> 172.21.15.159:0/4005072344 conn(0x5576973d5000 msgr2=0x557698152000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:59.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.635+0000 7f1d5d40d280 1 -- 172.21.15.159:0/4005072344 shutdown_connections 2024-04-04T06:15:59.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.635+0000 7f1d5d40d280 1 -- 172.21.15.159:0/4005072344 wait complete. 2024-04-04T06:15:59.635 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.106663, result: Events by type: Errors: 0 2024-04-04T06:15:59.635 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-04-04T06:15:59.720 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.726+0000 7f5602a41f40 1 Processor -- start 2024-04-04T06:15:59.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.726+0000 7f5602a41f40 1 -- start start 2024-04-04T06:15:59.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.727+0000 7f5602a41f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55e4002850 0x55ea70e962f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.727+0000 7f5602a41f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea70e83a80 0x55ea70dec900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.727+0000 7f5602a41f40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea70e8f780 0x55ea70dece40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.727+0000 7f5602a41f40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ea70d652c0 con 0x55ea70e8f780 2024-04-04T06:15:59.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.727+0000 7f5602a41f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ea70d69310 con 0x55ea70e83a80 2024-04-04T06:15:59.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.727+0000 7f5602a41f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ea70e0b980 con 0x7f55e4002850 2024-04-04T06:15:59.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.727+0000 7f55ffe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea70e8f780 0x55ea70dece40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.727+0000 7f55ffe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea70e8f780 0x55ea70dece40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52618/0 (socket says 172.21.15.159:52618) 2024-04-04T06:15:59.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.727+0000 7f55ffe00640 1 -- 172.21.15.159:0/2373898315 learned_addr learned my addr 172.21.15.159:0/2373898315 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:59.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.727+0000 7f55ffe00640 1 -- 172.21.15.159:0/2373898315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55e4002850 msgr2=0x55ea70e962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:59.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.727+0000 7f55ffe00640 1 --2- 172.21.15.159:0/2373898315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55e4002850 0x55ea70e962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.727+0000 7f55ffe00640 1 -- 172.21.15.159:0/2373898315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea70e83a80 msgr2=0x55ea70dec900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:15:59.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.727+0000 7f55ffe00640 1 --2- 172.21.15.159:0/2373898315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea70e83a80 0x55ea70dec900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.727+0000 7f55ffe00640 1 -- 172.21.15.159:0/2373898315 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea70dbef20 con 0x55ea70e8f780 2024-04-04T06:15:59.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.727+0000 7f55ffe00640 1 --2- 172.21.15.159:0/2373898315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea70e8f780 0x55ea70dece40 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x55ea70e089d0 tx=0x7f55f800dba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d373b9f357c476a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:59.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.728+0000 7f55fd600640 1 -- 172.21.15.159:0/2373898315 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f800b830 con 0x55ea70e8f780 2024-04-04T06:15:59.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.728+0000 7f55fd600640 1 -- 172.21.15.159:0/2373898315 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f800b9d0 con 0x55ea70e8f780 2024-04-04T06:15:59.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.728+0000 7f55fd600640 1 -- 172.21.15.159:0/2373898315 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f800bda0 con 0x55ea70e8f780 2024-04-04T06:15:59.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.728+0000 7f5602a41f40 1 -- 172.21.15.159:0/2373898315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea70e8f780 msgr2=0x55ea70dece40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.728+0000 7f5602a41f40 1 --2- 172.21.15.159:0/2373898315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea70e8f780 0x55ea70dece40 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x55ea70e089d0 tx=0x7f55f800dba0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.728+0000 7f5602a41f40 1 -- 172.21.15.159:0/2373898315 shutdown_connections 2024-04-04T06:15:59.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.728+0000 7f5602a41f40 1 --2- 172.21.15.159:0/2373898315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea70e8f780 0x55ea70dece40 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.728+0000 7f5602a41f40 1 --2- 172.21.15.159:0/2373898315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea70e83a80 0x55ea70dec900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.728+0000 7f5602a41f40 1 --2- 172.21.15.159:0/2373898315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55e4002850 0x55ea70e962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.728+0000 7f5602a41f40 1 -- 172.21.15.159:0/2373898315 >> 172.21.15.159:0/2373898315 conn(0x55ea70e0c630 msgr2=0x55ea70debe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:59.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.728+0000 7f5602a41f40 1 -- 172.21.15.159:0/2373898315 shutdown_connections 2024-04-04T06:15:59.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.728+0000 7f5602a41f40 1 -- 172.21.15.159:0/2373898315 wait complete. 2024-04-04T06:15:59.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.729+0000 7f5602a41f40 1 Processor -- start 2024-04-04T06:15:59.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.729+0000 7f5602a41f40 1 -- start start 2024-04-04T06:15:59.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.730+0000 7f5602a41f40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea70e83a80 0x55ea70e82c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.730+0000 7f5602a41f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ea70e8f780 0x55ea70e8aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.730+0000 7f5602a41f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55e4002850 0x55ea70e8af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.730+0000 7f5602a41f40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ea70e838b0 con 0x55ea70e83a80 2024-04-04T06:15:59.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.730+0000 7f5602a41f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ea70dedb60 con 0x7f55e4002850 2024-04-04T06:15:59.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.730+0000 7f5602a41f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ea70e8c7b0 con 0x55ea70e8f780 2024-04-04T06:15:59.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.730+0000 7f55ff400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea70e83a80 0x55ea70e82c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.730+0000 7f55ff400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea70e83a80 0x55ea70e82c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52630/0 (socket says 172.21.15.159:52630) 2024-04-04T06:15:59.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.730+0000 7f55ff400640 1 -- 172.21.15.159:0/1878155333 learned_addr learned my addr 172.21.15.159:0/1878155333 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:15:59.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.731+0000 7f55ff400640 1 -- 172.21.15.159:0/1878155333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ea70e8f780 msgr2=0x55ea70e8aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.731+0000 7f55ff400640 1 --2- 172.21.15.159:0/1878155333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ea70e8f780 0x55ea70e8aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.731+0000 7f55ff400640 1 -- 172.21.15.159:0/1878155333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55e4002850 msgr2=0x55ea70e8af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.731+0000 7f55ffe00640 1 --2- 172.21.15.159:0/1878155333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55e4002850 0x55ea70e8af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.731+0000 7f55ff400640 1 --2- 172.21.15.159:0/1878155333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55e4002850 0x55ea70e8af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.731+0000 7f55ff400640 1 -- 172.21.15.159:0/1878155333 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea70e98de0 con 0x55ea70e83a80 2024-04-04T06:15:59.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.731+0000 7f55ffe00640 1 --2- 172.21.15.159:0/1878155333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55e4002850 0x55ea70e8af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:15:59.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.731+0000 7f55ff400640 1 --2- 172.21.15.159:0/1878155333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea70e83a80 0x55ea70e82c90 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f55f0003210 tx=0x7f55f000b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:59.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.731+0000 7f55f7e00640 1 -- 172.21.15.159:0/1878155333 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0015020 con 0x55ea70e83a80 2024-04-04T06:15:59.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.731+0000 7f55f7e00640 1 -- 172.21.15.159:0/1878155333 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f0006220 con 0x55ea70e83a80 2024-04-04T06:15:59.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.731+0000 7f55f7e00640 1 -- 172.21.15.159:0/1878155333 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0005660 con 0x55ea70e83a80 2024-04-04T06:15:59.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.731+0000 7f5602a41f40 1 -- 172.21.15.159:0/1878155333 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ea70decb50 con 0x55ea70e83a80 2024-04-04T06:15:59.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.732+0000 7f5602a41f40 1 -- 172.21.15.159:0/1878155333 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea70ded110 con 0x55ea70e83a80 2024-04-04T06:15:59.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.734+0000 7f55f7e00640 1 -- 172.21.15.159:0/1878155333 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55f0005800 con 0x55ea70e83a80 2024-04-04T06:15:59.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.734+0000 7f55f7e00640 1 --2- 172.21.15.159:0/1878155333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f55e006b2d0 0x7f55e006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.735+0000 7f55f7e00640 1 -- 172.21.15.159:0/1878155333 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f55f008a560 con 0x55ea70e83a80 2024-04-04T06:15:59.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.735+0000 7f5602a41f40 1 --2- 172.21.15.159:0/1878155333 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x55ea70dbdde0 0x55ea70e85a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.735+0000 7f5602a41f40 1 -- 172.21.15.159:0/1878155333 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55ea70e85f90 con 0x55ea70dbdde0 2024-04-04T06:15:59.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.735+0000 7f55ffe00640 1 --2- 172.21.15.159:0/1878155333 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x55ea70dbdde0 0x55ea70e85a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.735+0000 7f55fea00640 1 --2- 172.21.15.159:0/1878155333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f55e006b2d0 0x7f55e006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.736+0000 7f55ffe00640 1 --2- 172.21.15.159:0/1878155333 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x55ea70dbdde0 0x55ea70e85a50 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:59.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.736+0000 7f55fea00640 1 --2- 172.21.15.159:0/1878155333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f55e006b2d0 0x7f55e006d790 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009810 tx=0x7f55ec003020 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:15:59.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.740+0000 7f55ffe00640 1 -- 172.21.15.159:0/1878155333 <== osd.2 v2:172.21.15.159:6808/2619731047 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v102'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f55f800a070 con 0x55ea70dbdde0 2024-04-04T06:15:59.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.740+0000 7f5602a41f40 1 -- 172.21.15.159:0/1878155333 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x55ea70dbdde0 msgr2=0x55ea70e85a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.740+0000 7f5602a41f40 1 --2- 172.21.15.159:0/1878155333 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x55ea70dbdde0 0x55ea70e85a50 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:15:59.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.740+0000 7f5602a41f40 1 -- 172.21.15.159:0/1878155333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f55e006b2d0 msgr2=0x7f55e006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.740+0000 7f5602a41f40 1 --2- 172.21.15.159:0/1878155333 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f55e006b2d0 0x7f55e006d790 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009810 tx=0x7f55ec003020 comp rx=0 tx=0).stop 2024-04-04T06:15:59.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.740+0000 7f5602a41f40 1 -- 172.21.15.159:0/1878155333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea70e83a80 msgr2=0x55ea70e82c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:15:59.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.740+0000 7f5602a41f40 1 --2- 172.21.15.159:0/1878155333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea70e83a80 0x55ea70e82c90 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f55f0003210 tx=0x7f55f000b570 comp rx=0 tx=0).stop 2024-04-04T06:15:59.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.740+0000 7f55ffe00640 1 -- 172.21.15.159:0/1878155333 reap_dead start 2024-04-04T06:15:59.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.740+0000 7f5602a41f40 1 -- 172.21.15.159:0/1878155333 shutdown_connections 2024-04-04T06:15:59.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.740+0000 7f5602a41f40 1 -- 172.21.15.159:0/1878155333 >> 172.21.15.159:0/1878155333 conn(0x55ea70e0c630 msgr2=0x55ea70df0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:15:59.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.740+0000 7f5602a41f40 1 -- 172.21.15.159:0/1878155333 shutdown_connections 2024-04-04T06:15:59.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:15:59.740+0000 7f5602a41f40 1 -- 172.21.15.159:0/1878155333 wait complete. 2024-04-04T06:15:59.740 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-04T06:15:59.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.002+0000 7f7645200640 1 Processor -- start 2024-04-04T06:15:59.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.002+0000 7f7645200640 1 -- start start 2024-04-04T06:15:59.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.003+0000 7f7645200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f764013f190 0x7f76400cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.003+0000 7f7645200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f76400cc880 0x7f76400cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.003+0000 7f7645200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f764006d620 0x7f764006da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:15:59.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.003+0000 7f7645200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f76400cd200 con 0x7f764013f190 2024-04-04T06:15:59.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.004+0000 7f7645200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f76400ce890 con 0x7f764006d620 2024-04-04T06:15:59.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.004+0000 7f7645200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f76400ceb50 con 0x7f76400cc880 2024-04-04T06:15:59.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.004+0000 7f763f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f764013f190 0x7f76400cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:15:59.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.004+0000 7f763f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f764013f190 0x7f76400cf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52632/0 (socket says 172.21.15.159:52632) 2024-04-04T06:15:59.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.004+0000 7f763f400640 1 -- 172.21.15.159:0/112349612 learned_addr learned my addr 172.21.15.159:0/112349612 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:00.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.004+0000 7f763fe00640 1 --2- 172.21.15.159:0/112349612 >> v2:172.21.15.178:3300/0 conn(0x7f764006d620 0x7f764006da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:00.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.004+0000 7f7636a00640 1 --2- 172.21.15.159:0/112349612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f76400cc880 0x7f76400cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:00.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.004+0000 7f763f400640 1 -- 172.21.15.159:0/112349612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f76400cc880 msgr2=0x7f76400cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:00.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.004+0000 7f763f400640 1 --2- 172.21.15.159:0/112349612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f76400cc880 0x7f76400cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.004+0000 7f763f400640 1 -- 172.21.15.159:0/112349612 >> v2:172.21.15.178:3300/0 conn(0x7f764006d620 msgr2=0x7f764006da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:00.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.004+0000 7f763f400640 1 --2- 172.21.15.159:0/112349612 >> v2:172.21.15.178:3300/0 conn(0x7f764006d620 0x7f764006da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.004+0000 7f763f400640 1 -- 172.21.15.159:0/112349612 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76400cee30 con 0x7f764013f190 2024-04-04T06:16:00.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.004+0000 7f7636a00640 1 --2- 172.21.15.159:0/112349612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f76400cc880 0x7f76400cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:00.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.005+0000 7f763fe00640 1 --2- 172.21.15.159:0/112349612 >> v2:172.21.15.178:3300/0 conn(0x7f764006d620 0x7f764006da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:00.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.006+0000 7f763f400640 1 --2- 172.21.15.159:0/112349612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764013f190 0x7f76400cf630 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f76400b2c40 tx=0x7f763002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527a9bd81531efa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:00.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.006+0000 7f763ea00640 1 -- 172.21.15.159:0/112349612 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630035020 con 0x7f764013f190 2024-04-04T06:16:00.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.006+0000 7f763ea00640 1 -- 172.21.15.159:0/112349612 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f763002d040 con 0x7f764013f190 2024-04-04T06:16:00.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.006+0000 7f763ea00640 1 -- 172.21.15.159:0/112349612 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630031a80 con 0x7f764013f190 2024-04-04T06:16:00.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.007+0000 7f7645200640 1 -- 172.21.15.159:0/112349612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764013f190 msgr2=0x7f76400cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:00.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.007+0000 7f7645200640 1 --2- 172.21.15.159:0/112349612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764013f190 0x7f76400cf630 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f76400b2c40 tx=0x7f763002b900 comp rx=0 tx=0).stop 2024-04-04T06:16:00.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.007+0000 7f7645200640 1 -- 172.21.15.159:0/112349612 shutdown_connections 2024-04-04T06:16:00.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.007+0000 7f7645200640 1 --2- 172.21.15.159:0/112349612 >> v2:172.21.15.178:3300/0 conn(0x7f764006d620 0x7f764006da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.007+0000 7f7645200640 1 --2- 172.21.15.159:0/112349612 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f76400cc880 0x7f76400cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.007+0000 7f7645200640 1 --2- 172.21.15.159:0/112349612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764013f190 0x7f76400cf630 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.007+0000 7f7645200640 1 -- 172.21.15.159:0/112349612 >> 172.21.15.159:0/112349612 conn(0x7f76400b1ba0 msgr2=0x7f76400b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:00.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.007+0000 7f7645200640 1 -- 172.21.15.159:0/112349612 shutdown_connections 2024-04-04T06:16:00.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.007+0000 7f7645200640 1 -- 172.21.15.159:0/112349612 wait complete. 2024-04-04T06:16:00.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.009+0000 7f7645200640 1 Processor -- start 2024-04-04T06:16:00.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.009+0000 7f7645200640 1 -- start start 2024-04-04T06:16:00.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.010+0000 7f7645200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f764006d620 0x7f76400b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:00.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.010+0000 7f7645200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76400cc880 0x7f764013cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:00.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.010+0000 7f7645200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764013f190 0x7f76401f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:00.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.010+0000 7f7645200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f764013f570 con 0x7f764013f190 2024-04-04T06:16:00.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.010+0000 7f7645200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7640160c70 con 0x7f76400cc880 2024-04-04T06:16:00.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.010+0000 7f7645200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f76400cfb70 con 0x7f764006d620 2024-04-04T06:16:00.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.010+0000 7f763fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764013f190 0x7f76401f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:00.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.010+0000 7f763fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764013f190 0x7f76401f3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52642/0 (socket says 172.21.15.159:52642) 2024-04-04T06:16:00.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.010+0000 7f763fe00640 1 -- 172.21.15.159:0/2186442553 learned_addr learned my addr 172.21.15.159:0/2186442553 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:00.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.010+0000 7f7636a00640 1 --2- 172.21.15.159:0/2186442553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76400cc880 0x7f764013cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:00.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.010+0000 7f763fe00640 1 -- 172.21.15.159:0/2186442553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f764006d620 msgr2=0x7f76400b15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:00.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.010+0000 7f763fe00640 1 --2- 172.21.15.159:0/2186442553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f764006d620 0x7f76400b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.010+0000 7f763fe00640 1 -- 172.21.15.159:0/2186442553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76400cc880 msgr2=0x7f764013cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:00.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.010+0000 7f763fe00640 1 --2- 172.21.15.159:0/2186442553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76400cc880 0x7f764013cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.011+0000 7f763fe00640 1 -- 172.21.15.159:0/2186442553 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764013d120 con 0x7f764013f190 2024-04-04T06:16:00.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.011+0000 7f763fe00640 1 --2- 172.21.15.159:0/2186442553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764013f190 0x7f76401f3f10 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f763800cd50 tx=0x7f763800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:00.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.011+0000 7f7637e00640 1 -- 172.21.15.159:0/2186442553 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763800f680 con 0x7f764013f190 2024-04-04T06:16:00.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.011+0000 7f7637e00640 1 -- 172.21.15.159:0/2186442553 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f763800f840 con 0x7f764013f190 2024-04-04T06:16:00.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.011+0000 7f7645200640 1 -- 172.21.15.159:0/2186442553 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76400af6f0 con 0x7f764013f190 2024-04-04T06:16:00.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.011+0000 7f7637e00640 1 -- 172.21.15.159:0/2186442553 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763800fc30 con 0x7f764013f190 2024-04-04T06:16:00.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.011+0000 7f7645200640 1 -- 172.21.15.159:0/2186442553 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76401f4ab0 con 0x7f764013f190 2024-04-04T06:16:00.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.012+0000 7f7645200640 1 -- 172.21.15.159:0/2186442553 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76400afba0 con 0x7f764013f190 2024-04-04T06:16:00.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.013+0000 7f7637e00640 1 -- 172.21.15.159:0/2186442553 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76380213f0 con 0x7f764013f190 2024-04-04T06:16:00.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.014+0000 7f7637e00640 1 --2- 172.21.15.159:0/2186442553 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f761c06c8f0 0x7f761c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:00.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.014+0000 7f7637e00640 1 -- 172.21.15.159:0/2186442553 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f763808c9d0 con 0x7f764013f190 2024-04-04T06:16:00.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.015+0000 7f763f400640 1 --2- 172.21.15.159:0/2186442553 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f761c06c8f0 0x7f761c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:00.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.016+0000 7f763f400640 1 --2- 172.21.15.159:0/2186442553 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f761c06c8f0 0x7f761c06edb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7630036000 tx=0x7f7630030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:00.013 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.019+0000 7f7637e00640 1 -- 172.21.15.159:0/2186442553 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f764013d120 con 0x7f764013f190 2024-04-04T06:16:00.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.261+0000 7f7645200640 1 -- 172.21.15.159:0/2186442553 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f7640073610 con 0x7f764013f190 2024-04-04T06:16:00.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.486+0000 7f7637e00640 1 -- 172.21.15.159:0/2186442553 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v86) ==== 162+0+0 (secure 0 0 0) 0x7f7640073610 con 0x7f764013f190 2024-04-04T06:16:00.481 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-04T06:16:00.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.489+0000 7f7645200640 1 -- 172.21.15.159:0/2186442553 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f761c06c8f0 msgr2=0x7f761c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:00.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.489+0000 7f7645200640 1 --2- 172.21.15.159:0/2186442553 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f761c06c8f0 0x7f761c06edb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7630036000 tx=0x7f7630030000 comp rx=0 tx=0).stop 2024-04-04T06:16:00.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.490+0000 7f7645200640 1 -- 172.21.15.159:0/2186442553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764013f190 msgr2=0x7f76401f3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:00.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.490+0000 7f7645200640 1 --2- 172.21.15.159:0/2186442553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764013f190 0x7f76401f3f10 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f763800cd50 tx=0x7f763800a650 comp rx=0 tx=0).stop 2024-04-04T06:16:00.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.490+0000 7f7645200640 1 -- 172.21.15.159:0/2186442553 shutdown_connections 2024-04-04T06:16:00.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.490+0000 7f7645200640 1 --2- 172.21.15.159:0/2186442553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764013f190 0x7f76401f3f10 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.490+0000 7f7645200640 1 --2- 172.21.15.159:0/2186442553 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f761c06c8f0 0x7f761c06edb0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.490+0000 7f7645200640 1 --2- 172.21.15.159:0/2186442553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76400cc880 0x7f764013cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.490+0000 7f7645200640 1 --2- 172.21.15.159:0/2186442553 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f764006d620 0x7f76400b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.490+0000 7f7645200640 1 -- 172.21.15.159:0/2186442553 >> 172.21.15.159:0/2186442553 conn(0x7f76400b1ba0 msgr2=0x7f764006da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:00.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.490+0000 7f7645200640 1 -- 172.21.15.159:0/2186442553 shutdown_connections 2024-04-04T06:16:00.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.490+0000 7f7645200640 1 -- 172.21.15.159:0/2186442553 wait complete. 2024-04-04T06:16:00.508 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:16:00.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.697+0000 7fee95200640 1 Processor -- start 2024-04-04T06:16:00.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.697+0000 7fee95200640 1 -- start start 2024-04-04T06:16:00.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.699+0000 7fee95200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fee9015c390 0x7fee90073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:00.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.699+0000 7fee95200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee900af830 0x7fee90160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:00.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.699+0000 7fee95200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee9015fe80 con 0x7fee90160b80 2024-04-04T06:16:00.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.699+0000 7fee95200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fee9015c770 con 0x7fee9015c390 2024-04-04T06:16:00.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.699+0000 7fee95200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fee900ad120 con 0x7fee900af830 2024-04-04T06:16:00.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.699+0000 7fee8e800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fee90160b80 0x7fee900d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52680/0 (socket says 172.21.15.159:52680) 2024-04-04T06:16:00.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.699+0000 7fee8e800640 1 -- 172.21.15.159:0/1102047499 learned_addr learned my addr 172.21.15.159:0/1102047499 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:16:00.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.699+0000 7fee8de00640 1 --2- 172.21.15.159:0/1102047499 >> v2:172.21.15.178:3300/0 conn(0x7fee9015c390 0x7fee90073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:00.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.699+0000 7fee8d400640 1 --2- 172.21.15.159:0/1102047499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee900af830 0x7fee90160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:00.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.699+0000 7fee8ca00640 1 -- 172.21.15.159:0/1102047499 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2495694408 0 0) 0x7fee9015fe80 con 0x7fee90160b80 2024-04-04T06:16:00.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.700+0000 7fee8ca00640 1 -- 172.21.15.159:0/1102047499 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee74003520 con 0x7fee90160b80 2024-04-04T06:16:00.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.700+0000 7fee8ca00640 1 -- 172.21.15.159:0/1102047499 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 498727072 0 0) 0x7fee74003520 con 0x7fee90160b80 2024-04-04T06:16:00.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.700+0000 7fee8ca00640 1 -- 172.21.15.159:0/1102047499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee900af830 msgr2=0x7fee90160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:00.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.700+0000 7fee8ca00640 1 --2- 172.21.15.159:0/1102047499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee900af830 0x7fee90160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.700+0000 7fee8ca00640 1 -- 172.21.15.159:0/1102047499 >> v2:172.21.15.178:3300/0 conn(0x7fee9015c390 msgr2=0x7fee90073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:00.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.700+0000 7fee8ca00640 1 --2- 172.21.15.159:0/1102047499 >> v2:172.21.15.178:3300/0 conn(0x7fee9015c390 0x7fee90073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.700+0000 7fee8ca00640 1 -- 172.21.15.159:0/1102047499 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee900ad400 con 0x7fee90160b80 2024-04-04T06:16:00.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.700+0000 7fee8ca00640 1 -- 172.21.15.159:0/1102047499 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fee880019f0 con 0x7fee90160b80 2024-04-04T06:16:00.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.700+0000 7fee8ca00640 1 -- 172.21.15.159:0/1102047499 >> v1:172.21.15.159:6789/0 conn(0x7fee90160b80 legacy=0x7fee900d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:00.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.700+0000 7fee8ca00640 1 --2- 172.21.15.159:0/1102047499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee740039b0 0x7fee74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:00.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.700+0000 7fee8ca00640 1 --2- 172.21.15.159:0/1102047499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee74004800 0x7fee74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:00.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.700+0000 7fee8ca00640 1 --2- 172.21.15.159:0/1102047499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee900af830 0x7fee740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:00.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.700+0000 7fee8ca00640 1 -- 172.21.15.159:0/1102047499 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fee900ad120 con 0x7fee740039b0 2024-04-04T06:16:00.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.700+0000 7fee8ca00640 1 -- 172.21.15.159:0/1102047499 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fee7400b9c0 con 0x7fee74004800 2024-04-04T06:16:00.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.700+0000 7fee8ca00640 1 -- 172.21.15.159:0/1102047499 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fee7400bb40 con 0x7fee900af830 2024-04-04T06:16:00.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.700+0000 7fee8de00640 1 --2- 172.21.15.159:0/1102047499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee740039b0 0x7fee74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:00.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.701+0000 7fee8d400640 1 --2- 172.21.15.159:0/1102047499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee74004800 0x7fee74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:00.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.701+0000 7fee8de00640 1 -- 172.21.15.159:0/1102047499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee900af830 msgr2=0x7fee740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:00.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.701+0000 7fee8de00640 1 --2- 172.21.15.159:0/1102047499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee900af830 0x7fee740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.701+0000 7fee8de00640 1 -- 172.21.15.159:0/1102047499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee74004800 msgr2=0x7fee74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:00.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.701+0000 7fee8de00640 1 --2- 172.21.15.159:0/1102047499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee74004800 0x7fee74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.701+0000 7fee8de00640 1 -- 172.21.15.159:0/1102047499 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee74003520 con 0x7fee740039b0 2024-04-04T06:16:00.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.701+0000 7fee8d400640 1 --2- 172.21.15.159:0/1102047499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee74004800 0x7fee74004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:00.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.702+0000 7fee8de00640 1 --2- 172.21.15.159:0/1102047499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee740039b0 0x7fee74003dd0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fee80009ac0 tx=0x7fee74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fbcbffd42e6735 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:00.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.702+0000 7fee8ca00640 1 -- 172.21.15.159:0/1102047499 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80039070 con 0x7fee740039b0 2024-04-04T06:16:00.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.702+0000 7fee8ca00640 1 -- 172.21.15.159:0/1102047499 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee8002c710 con 0x7fee740039b0 2024-04-04T06:16:00.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.702+0000 7fee8ca00640 1 -- 172.21.15.159:0/1102047499 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80009690 con 0x7fee740039b0 2024-04-04T06:16:00.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.703+0000 7fee95200640 1 -- 172.21.15.159:0/1102047499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee740039b0 msgr2=0x7fee74003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:00.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.703+0000 7fee95200640 1 --2- 172.21.15.159:0/1102047499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee740039b0 0x7fee74003dd0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fee80009ac0 tx=0x7fee74005830 comp rx=0 tx=0).stop 2024-04-04T06:16:00.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.703+0000 7fee8e800640 1 -- 172.21.15.159:0/1102047499 reap_dead start 2024-04-04T06:16:00.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.703+0000 7fee95200640 1 -- 172.21.15.159:0/1102047499 shutdown_connections 2024-04-04T06:16:00.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.703+0000 7fee95200640 1 -- 172.21.15.159:0/1102047499 >> 172.21.15.159:0/1102047499 conn(0x7fee9006b990 msgr2=0x7fee9006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:00.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.703+0000 7fee95200640 1 -- 172.21.15.159:0/1102047499 shutdown_connections 2024-04-04T06:16:00.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.703+0000 7fee95200640 1 -- 172.21.15.159:0/1102047499 wait complete. 2024-04-04T06:16:00.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.705+0000 7fee95200640 1 Processor -- start 2024-04-04T06:16:00.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.705+0000 7fee95200640 1 -- start start 2024-04-04T06:16:00.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.706+0000 7fee95200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee740039b0 0x7fee901f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:00.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.706+0000 7fee95200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee90153810 0x7fee9015ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:00.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.706+0000 7fee95200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee901468e0 0x7fee9015f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:00.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.706+0000 7fee95200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fee900cf980 con 0x7fee90153810 2024-04-04T06:16:00.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.706+0000 7fee95200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fee90160120 con 0x7fee740039b0 2024-04-04T06:16:00.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.706+0000 7fee95200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fee900d0d60 con 0x7fee901468e0 2024-04-04T06:16:00.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.706+0000 7fee8d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee90153810 0x7fee9015ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:00.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.706+0000 7fee8d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee90153810 0x7fee9015ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52654/0 (socket says 172.21.15.159:52654) 2024-04-04T06:16:00.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.706+0000 7fee8d400640 1 -- 172.21.15.159:0/3818438424 learned_addr learned my addr 172.21.15.159:0/3818438424 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:00.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.706+0000 7fee8d400640 1 -- 172.21.15.159:0/3818438424 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee901468e0 msgr2=0x7fee9015f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:00.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.706+0000 7fee8d400640 1 --2- 172.21.15.159:0/3818438424 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee901468e0 0x7fee9015f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.706+0000 7fee8de00640 1 --2- 172.21.15.159:0/3818438424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee740039b0 0x7fee901f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:00.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.706+0000 7fee8d400640 1 -- 172.21.15.159:0/3818438424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee740039b0 msgr2=0x7fee901f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:00.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.706+0000 7fee8d400640 1 --2- 172.21.15.159:0/3818438424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee740039b0 0x7fee901f60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.706+0000 7fee8d400640 1 -- 172.21.15.159:0/3818438424 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee901f6600 con 0x7fee90153810 2024-04-04T06:16:00.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.707+0000 7fee8d400640 1 --2- 172.21.15.159:0/3818438424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee90153810 0x7fee9015ec70 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0108e0 tx=0x7fee7c010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:00.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.707+0000 7fee86000640 1 -- 172.21.15.159:0/3818438424 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c00f040 con 0x7fee90153810 2024-04-04T06:16:00.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.707+0000 7fee86000640 1 -- 172.21.15.159:0/3818438424 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee7c00ebc0 con 0x7fee90153810 2024-04-04T06:16:00.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.707+0000 7fee86000640 1 -- 172.21.15.159:0/3818438424 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c01c400 con 0x7fee90153810 2024-04-04T06:16:00.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.707+0000 7fee95200640 1 -- 172.21.15.159:0/3818438424 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee901e6b90 con 0x7fee90153810 2024-04-04T06:16:00.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.707+0000 7fee95200640 1 -- 172.21.15.159:0/3818438424 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee9015cff0 con 0x7fee90153810 2024-04-04T06:16:00.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.708+0000 7fee95200640 1 -- 172.21.15.159:0/3818438424 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee900685c0 con 0x7fee90153810 2024-04-04T06:16:00.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.709+0000 7fee86000640 1 -- 172.21.15.159:0/3818438424 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee7c01c5a0 con 0x7fee90153810 2024-04-04T06:16:00.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.710+0000 7fee86000640 1 --2- 172.21.15.159:0/3818438424 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fee7006c8f0 0x7fee7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:00.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.710+0000 7fee86000640 1 -- 172.21.15.159:0/3818438424 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7fee7c08c3a0 con 0x7fee90153810 2024-04-04T06:16:00.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.711+0000 7fee8de00640 1 --2- 172.21.15.159:0/3818438424 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fee7006c8f0 0x7fee7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:00.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.712+0000 7fee8de00640 1 --2- 172.21.15.159:0/3818438424 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fee7006c8f0 0x7fee7006edb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fee80036000 tx=0x7fee80030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:00.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.715+0000 7fee86000640 1 -- 172.21.15.159:0/3818438424 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fee7c018030 con 0x7fee90153810 2024-04-04T06:16:00.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.958+0000 7fee95200640 1 -- 172.21.15.159:0/3818438424 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee9006f1b0 con 0x7fee90153810 2024-04-04T06:16:00.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.959+0000 7fee86000640 1 -- 172.21.15.159:0/3818438424 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5184 (secure 0 0 0) 0x7fee7c05a080 con 0x7fee90153810 2024-04-04T06:16:00.954 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:00.954 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":87,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5546,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/40219476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":40219476},{"type":"v1","addr":"172.21.15.178:6837","nonce":40219476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/4060483382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4060483382},{"type":"v1","addr":"172.21.15.178:6836","nonce":4060483382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8187,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1945665739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1945665739},{"type":"v1","addr":"172.21.15.159:6833","nonce":1945665739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:15:07.704835+0000","modified":"2024-04-04T06:16:00.492732+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8535":{"gid":8535,"name":"a","rank":0,"incarnation":87,"state":"up:replay","state_seq":1,"addr":"172.21.15.159:6835/1003751501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1003751501},{"type":"v1","addr":"172.21.15.159:6835","nonce":1003751501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-04T06:16:00.955 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 87 2024-04-04T06:16:00.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.962+0000 7fee95200640 1 -- 172.21.15.159:0/3818438424 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fee7006c8f0 msgr2=0x7fee7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:00.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.962+0000 7fee95200640 1 --2- 172.21.15.159:0/3818438424 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fee7006c8f0 0x7fee7006edb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fee80036000 tx=0x7fee80030000 comp rx=0 tx=0).stop 2024-04-04T06:16:00.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.962+0000 7fee95200640 1 -- 172.21.15.159:0/3818438424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee90153810 msgr2=0x7fee9015ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:00.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.962+0000 7fee95200640 1 --2- 172.21.15.159:0/3818438424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee90153810 0x7fee9015ec70 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0108e0 tx=0x7fee7c010da0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.962+0000 7fee95200640 1 -- 172.21.15.159:0/3818438424 shutdown_connections 2024-04-04T06:16:00.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.962+0000 7fee95200640 1 --2- 172.21.15.159:0/3818438424 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fee901468e0 0x7fee9015f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.962+0000 7fee95200640 1 --2- 172.21.15.159:0/3818438424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee90153810 0x7fee9015ec70 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.962+0000 7fee95200640 1 --2- 172.21.15.159:0/3818438424 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fee7006c8f0 0x7fee7006edb0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.962+0000 7fee95200640 1 --2- 172.21.15.159:0/3818438424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee740039b0 0x7fee901f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:00.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.962+0000 7fee95200640 1 -- 172.21.15.159:0/3818438424 >> 172.21.15.159:0/3818438424 conn(0x7fee9006b990 msgr2=0x7fee900d0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:00.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.963+0000 7fee95200640 1 -- 172.21.15.159:0/3818438424 shutdown_connections 2024-04-04T06:16:00.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:00.963+0000 7fee95200640 1 -- 172.21.15.159:0/3818438424 wait complete. 2024-04-04T06:16:01.379 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:15:07.704835+0000', 'modified': '2024-04-04T06:16:00.492732+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 102, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8535}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8535': {'gid': 8535, 'name': 'a', 'rank': 0, 'incarnation': 87, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.159:6835/1003751501', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 1003751501}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 1003751501}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-04T06:16:01.379 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8535:up:replay 2024-04-04T06:16:02.380 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:16:02.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.605+0000 7fbc5da00640 1 Processor -- start 2024-04-04T06:16:02.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.605+0000 7fbc5da00640 1 -- start start 2024-04-04T06:16:02.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.606+0000 7fbc5da00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbc5815c390 0x7fbc58073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:02.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.606+0000 7fbc5da00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc580af830 0x7fbc58160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:02.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.606+0000 7fbc5da00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc5815fe80 con 0x7fbc58160b80 2024-04-04T06:16:02.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.606+0000 7fbc5da00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fbc5815c770 con 0x7fbc5815c390 2024-04-04T06:16:02.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.606+0000 7fbc5da00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbc580ad120 con 0x7fbc580af830 2024-04-04T06:16:02.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.607+0000 7fbc57400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fbc58160b80 0x7fbc580d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52690/0 (socket says 172.21.15.159:52690) 2024-04-04T06:16:02.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.607+0000 7fbc57400640 1 -- 172.21.15.159:0/3056169959 learned_addr learned my addr 172.21.15.159:0/3056169959 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:16:02.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.607+0000 7fbc56a00640 1 --2- 172.21.15.159:0/3056169959 >> v2:172.21.15.178:3300/0 conn(0x7fbc5815c390 0x7fbc58073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:02.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.607+0000 7fbc56000640 1 --2- 172.21.15.159:0/3056169959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc580af830 0x7fbc58160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:02.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.607+0000 7fbc55600640 1 -- 172.21.15.159:0/3056169959 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2979272408 0 0) 0x7fbc5815fe80 con 0x7fbc58160b80 2024-04-04T06:16:02.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.607+0000 7fbc55600640 1 -- 172.21.15.159:0/3056169959 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc3c003520 con 0x7fbc58160b80 2024-04-04T06:16:02.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.608+0000 7fbc55600640 1 -- 172.21.15.159:0/3056169959 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1375502556 0 0) 0x7fbc3c003520 con 0x7fbc58160b80 2024-04-04T06:16:02.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.608+0000 7fbc55600640 1 -- 172.21.15.159:0/3056169959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc580af830 msgr2=0x7fbc58160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:02.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.608+0000 7fbc55600640 1 --2- 172.21.15.159:0/3056169959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc580af830 0x7fbc58160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:02.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.608+0000 7fbc55600640 1 -- 172.21.15.159:0/3056169959 >> v2:172.21.15.178:3300/0 conn(0x7fbc5815c390 msgr2=0x7fbc58073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.608+0000 7fbc55600640 1 --2- 172.21.15.159:0/3056169959 >> v2:172.21.15.178:3300/0 conn(0x7fbc5815c390 0x7fbc58073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.608+0000 7fbc55600640 1 -- 172.21.15.159:0/3056169959 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc580ad400 con 0x7fbc58160b80 2024-04-04T06:16:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.608+0000 7fbc55600640 1 -- 172.21.15.159:0/3056169959 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fbc50001670 con 0x7fbc58160b80 2024-04-04T06:16:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.608+0000 7fbc55600640 1 -- 172.21.15.159:0/3056169959 >> v1:172.21.15.159:6789/0 conn(0x7fbc58160b80 legacy=0x7fbc580d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.608+0000 7fbc55600640 1 --2- 172.21.15.159:0/3056169959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc3c0039b0 0x7fbc3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.608+0000 7fbc55600640 1 --2- 172.21.15.159:0/3056169959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc3c004800 0x7fbc3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.608+0000 7fbc55600640 1 --2- 172.21.15.159:0/3056169959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc580af830 0x7fbc3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.608+0000 7fbc55600640 1 -- 172.21.15.159:0/3056169959 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbc580ad120 con 0x7fbc3c004800 2024-04-04T06:16:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.608+0000 7fbc55600640 1 -- 172.21.15.159:0/3056169959 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c00b9c0 con 0x7fbc3c0039b0 2024-04-04T06:16:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.608+0000 7fbc55600640 1 -- 172.21.15.159:0/3056169959 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbc3c00bb70 con 0x7fbc580af830 2024-04-04T06:16:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.608+0000 7fbc56000640 1 --2- 172.21.15.159:0/3056169959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc3c004800 0x7fbc3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.608+0000 7fbc56000640 1 -- 172.21.15.159:0/3056169959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc580af830 msgr2=0x7fbc3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.608+0000 7fbc56000640 1 --2- 172.21.15.159:0/3056169959 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc580af830 0x7fbc3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.608+0000 7fbc56a00640 1 --2- 172.21.15.159:0/3056169959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc3c0039b0 0x7fbc3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.608+0000 7fbc56000640 1 -- 172.21.15.159:0/3056169959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc3c0039b0 msgr2=0x7fbc3c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:02.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.608+0000 7fbc56000640 1 --2- 172.21.15.159:0/3056169959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc3c0039b0 0x7fbc3c003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:02.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.608+0000 7fbc56000640 1 -- 172.21.15.159:0/3056169959 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3c003520 con 0x7fbc3c004800 2024-04-04T06:16:02.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.609+0000 7fbc56000640 1 --2- 172.21.15.159:0/3056169959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc3c004800 0x7fbc3c004c80 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fbc48009fc0 tx=0x7fbc3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c766857f99f63261 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:02.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.610+0000 7fbc55600640 1 -- 172.21.15.159:0/3056169959 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc48039070 con 0x7fbc3c004800 2024-04-04T06:16:02.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.610+0000 7fbc55600640 1 -- 172.21.15.159:0/3056169959 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc4802cc60 con 0x7fbc3c004800 2024-04-04T06:16:02.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.610+0000 7fbc55600640 1 -- 172.21.15.159:0/3056169959 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc48009c10 con 0x7fbc3c004800 2024-04-04T06:16:02.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.610+0000 7fbc5da00640 1 -- 172.21.15.159:0/3056169959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc3c004800 msgr2=0x7fbc3c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:02.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.610+0000 7fbc5da00640 1 --2- 172.21.15.159:0/3056169959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc3c004800 0x7fbc3c004c80 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fbc48009fc0 tx=0x7fbc3c005830 comp rx=0 tx=0).stop 2024-04-04T06:16:02.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.610+0000 7fbc57400640 1 -- 172.21.15.159:0/3056169959 reap_dead start 2024-04-04T06:16:02.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.610+0000 7fbc5da00640 1 -- 172.21.15.159:0/3056169959 shutdown_connections 2024-04-04T06:16:02.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.610+0000 7fbc5da00640 1 -- 172.21.15.159:0/3056169959 >> 172.21.15.159:0/3056169959 conn(0x7fbc5806b990 msgr2=0x7fbc5806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:02.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.611+0000 7fbc5da00640 1 -- 172.21.15.159:0/3056169959 shutdown_connections 2024-04-04T06:16:02.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.611+0000 7fbc5da00640 1 -- 172.21.15.159:0/3056169959 wait complete. 2024-04-04T06:16:02.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.612+0000 7fbc5da00640 1 Processor -- start 2024-04-04T06:16:02.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.612+0000 7fbc5da00640 1 -- start start 2024-04-04T06:16:02.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.613+0000 7fbc5da00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc3c004800 0x7fbc5814f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:02.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.613+0000 7fbc5da00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc581e8280 0x7fbc581e8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:02.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.613+0000 7fbc5da00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc5815dbf0 0x7fbc5815e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:02.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.613+0000 7fbc5da00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbc580cf980 con 0x7fbc5815dbf0 2024-04-04T06:16:02.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.613+0000 7fbc5da00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc58160120 con 0x7fbc581e8280 2024-04-04T06:16:02.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.613+0000 7fbc5da00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbc580d0d60 con 0x7fbc3c004800 2024-04-04T06:16:02.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.614+0000 7fbc57400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc5815dbf0 0x7fbc5815e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:02.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.614+0000 7fbc57400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc5815dbf0 0x7fbc5815e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52680/0 (socket says 172.21.15.159:52680) 2024-04-04T06:16:02.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.614+0000 7fbc57400640 1 -- 172.21.15.159:0/89691467 learned_addr learned my addr 172.21.15.159:0/89691467 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:02.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.614+0000 7fbc56000640 1 --2- 172.21.15.159:0/89691467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc581e8280 0x7fbc581e8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:02.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.614+0000 7fbc56a00640 1 --2- 172.21.15.159:0/89691467 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc3c004800 0x7fbc5814f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:02.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.614+0000 7fbc57400640 1 -- 172.21.15.159:0/89691467 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc3c004800 msgr2=0x7fbc5814f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:02.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.614+0000 7fbc57400640 1 --2- 172.21.15.159:0/89691467 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc3c004800 0x7fbc5814f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:02.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.614+0000 7fbc57400640 1 -- 172.21.15.159:0/89691467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc581e8280 msgr2=0x7fbc581e8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:02.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.614+0000 7fbc57400640 1 --2- 172.21.15.159:0/89691467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc581e8280 0x7fbc581e8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:02.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.614+0000 7fbc57400640 1 -- 172.21.15.159:0/89691467 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5814fe90 con 0x7fbc5815dbf0 2024-04-04T06:16:02.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.614+0000 7fbc56000640 1 --2- 172.21.15.159:0/89691467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc581e8280 0x7fbc581e8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:02.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.614+0000 7fbc56a00640 1 --2- 172.21.15.159:0/89691467 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc3c004800 0x7fbc5814f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:02.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.614+0000 7fbc57400640 1 --2- 172.21.15.159:0/89691467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc5815dbf0 0x7fbc5815e070 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fbc58073530 tx=0x7fbc5000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:02.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.615+0000 7fbc4ea00640 1 -- 172.21.15.159:0/89691467 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50004bf0 con 0x7fbc5815dbf0 2024-04-04T06:16:02.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.615+0000 7fbc4ea00640 1 -- 172.21.15.159:0/89691467 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc50004d90 con 0x7fbc5815dbf0 2024-04-04T06:16:02.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.615+0000 7fbc4ea00640 1 -- 172.21.15.159:0/89691467 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50007270 con 0x7fbc5815dbf0 2024-04-04T06:16:02.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.615+0000 7fbc5da00640 1 -- 172.21.15.159:0/89691467 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc581e71e0 con 0x7fbc5815dbf0 2024-04-04T06:16:02.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.615+0000 7fbc5da00640 1 -- 172.21.15.159:0/89691467 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc581e77a0 con 0x7fbc5815dbf0 2024-04-04T06:16:02.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.616+0000 7fbc5da00640 1 -- 172.21.15.159:0/89691467 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc580685c0 con 0x7fbc5815dbf0 2024-04-04T06:16:02.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.617+0000 7fbc4ea00640 1 -- 172.21.15.159:0/89691467 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc50007410 con 0x7fbc5815dbf0 2024-04-04T06:16:02.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.618+0000 7fbc4ea00640 1 --2- 172.21.15.159:0/89691467 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc3406c8f0 0x7fbc3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:02.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.618+0000 7fbc4ea00640 1 -- 172.21.15.159:0/89691467 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7fbc5008afd0 con 0x7fbc5815dbf0 2024-04-04T06:16:02.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.618+0000 7fbc56a00640 1 --2- 172.21.15.159:0/89691467 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc3406c8f0 0x7fbc3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:02.614 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.620+0000 7fbc56a00640 1 --2- 172.21.15.159:0/89691467 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc3406c8f0 0x7fbc3406edb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbc40006fd0 tx=0x7fbc40008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:02.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.622+0000 7fbc4ea00640 1 -- 172.21.15.159:0/89691467 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbc5001a020 con 0x7fbc5815dbf0 2024-04-04T06:16:02.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.861+0000 7fbc5da00640 1 -- 172.21.15.159:0/89691467 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc580b07c0 con 0x7fbc5815dbf0 2024-04-04T06:16:02.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.862+0000 7fbc4ea00640 1 -- 172.21.15.159:0/89691467 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5184 (secure 0 0 0) 0x7fbc5814fe90 con 0x7fbc5815dbf0 2024-04-04T06:16:02.856 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:02.856 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":89,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5546,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/40219476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":40219476},{"type":"v1","addr":"172.21.15.178:6837","nonce":40219476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/4060483382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4060483382},{"type":"v1","addr":"172.21.15.178:6836","nonce":4060483382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8187,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1945665739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1945665739},{"type":"v1","addr":"172.21.15.159:6833","nonce":1945665739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:15:07.704835+0000","modified":"2024-04-04T06:16:01.513798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8535":{"gid":8535,"name":"a","rank":0,"incarnation":87,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.159:6835/1003751501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1003751501},{"type":"v1","addr":"172.21.15.159:6835","nonce":1003751501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-04T06:16:02.857 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 89 2024-04-04T06:16:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.864+0000 7fbc5da00640 1 -- 172.21.15.159:0/89691467 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc3406c8f0 msgr2=0x7fbc3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.864+0000 7fbc5da00640 1 --2- 172.21.15.159:0/89691467 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc3406c8f0 0x7fbc3406edb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbc40006fd0 tx=0x7fbc40008040 comp rx=0 tx=0).stop 2024-04-04T06:16:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.864+0000 7fbc5da00640 1 -- 172.21.15.159:0/89691467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc5815dbf0 msgr2=0x7fbc5815e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.864+0000 7fbc5da00640 1 --2- 172.21.15.159:0/89691467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc5815dbf0 0x7fbc5815e070 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fbc58073530 tx=0x7fbc5000c890 comp rx=0 tx=0).stop 2024-04-04T06:16:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.864+0000 7fbc5da00640 1 -- 172.21.15.159:0/89691467 shutdown_connections 2024-04-04T06:16:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.864+0000 7fbc5da00640 1 --2- 172.21.15.159:0/89691467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc5815dbf0 0x7fbc5815e070 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.864+0000 7fbc5da00640 1 --2- 172.21.15.159:0/89691467 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbc3406c8f0 0x7fbc3406edb0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.864+0000 7fbc5da00640 1 --2- 172.21.15.159:0/89691467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc581e8280 0x7fbc581e8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:02.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.864+0000 7fbc5da00640 1 --2- 172.21.15.159:0/89691467 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc3c004800 0x7fbc5814f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:02.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.864+0000 7fbc5da00640 1 -- 172.21.15.159:0/89691467 >> 172.21.15.159:0/89691467 conn(0x7fbc5806b990 msgr2=0x7fbc5806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:02.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.865+0000 7fbc5da00640 1 -- 172.21.15.159:0/89691467 shutdown_connections 2024-04-04T06:16:02.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:02.865+0000 7fbc5da00640 1 -- 172.21.15.159:0/89691467 wait complete. 2024-04-04T06:16:03.281 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:15:07.704835+0000', 'modified': '2024-04-04T06:16:01.513798+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 102, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8535}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8535': {'gid': 8535, 'name': 'a', 'rank': 0, 'incarnation': 87, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.159:6835/1003751501', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 1003751501}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 1003751501}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-04T06:16:03.281 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8535:up:rejoin 2024-04-04T06:16:04.282 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:16:04.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.499+0000 7f4661000640 1 Processor -- start 2024-04-04T06:16:04.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.499+0000 7f4661000640 1 -- start start 2024-04-04T06:16:04.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.501+0000 7f4661000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f465c15c650 0x7f465c0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:04.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.501+0000 7f4661000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f465c06b570 0x7f465c0d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:04.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.501+0000 7f4661000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f465c0d0ca0 0x7f465c0ae020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:04.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.501+0000 7f4661000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f465c0d02d0 con 0x7f465c0d0ca0 2024-04-04T06:16:04.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.501+0000 7f4661000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f465c0ce090 con 0x7f465c06b570 2024-04-04T06:16:04.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.501+0000 7f4661000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f465c0ce360 con 0x7f465c15c650 2024-04-04T06:16:04.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.501+0000 7f465a800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f465c0d0ca0 0x7f465c0ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:04.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.501+0000 7f465a800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f465c0d0ca0 0x7f465c0ae020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36004/0 (socket says 172.21.15.159:36004) 2024-04-04T06:16:04.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.501+0000 7f465a800640 1 -- 172.21.15.159:0/1166585790 learned_addr learned my addr 172.21.15.159:0/1166585790 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:04.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.501+0000 7f4659e00640 1 --2- 172.21.15.159:0/1166585790 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f465c15c650 0x7f465c0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:04.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.501+0000 7f4651400640 1 --2- 172.21.15.159:0/1166585790 >> v2:172.21.15.178:3300/0 conn(0x7f465c06b570 0x7f465c0d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:04.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.502+0000 7f465a800640 1 -- 172.21.15.159:0/1166585790 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f465c15c650 msgr2=0x7f465c0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:04.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.502+0000 7f465a800640 1 --2- 172.21.15.159:0/1166585790 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f465c15c650 0x7f465c0cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:04.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.502+0000 7f465a800640 1 -- 172.21.15.159:0/1166585790 >> v2:172.21.15.178:3300/0 conn(0x7f465c06b570 msgr2=0x7f465c0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:04.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.502+0000 7f465a800640 1 --2- 172.21.15.159:0/1166585790 >> v2:172.21.15.178:3300/0 conn(0x7f465c06b570 0x7f465c0d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:04.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.502+0000 7f465a800640 1 -- 172.21.15.159:0/1166585790 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465c0ce640 con 0x7f465c0d0ca0 2024-04-04T06:16:04.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.502+0000 7f4651400640 1 --2- 172.21.15.159:0/1166585790 >> v2:172.21.15.178:3300/0 conn(0x7f465c06b570 0x7f465c0d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:16:04.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.503+0000 7f465a800640 1 --2- 172.21.15.159:0/1166585790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f465c0d0ca0 0x7f465c0ae020 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f465c06d660 tx=0x7f465402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d677e217575f9ff server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:04.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.503+0000 7f4659400640 1 -- 172.21.15.159:0/1166585790 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465402cb00 con 0x7f465c0d0ca0 2024-04-04T06:16:04.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.503+0000 7f4659400640 1 -- 172.21.15.159:0/1166585790 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f465402cca0 con 0x7f465c0d0ca0 2024-04-04T06:16:04.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.503+0000 7f4659400640 1 -- 172.21.15.159:0/1166585790 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654034610 con 0x7f465c0d0ca0 2024-04-04T06:16:04.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.504+0000 7f4661000640 1 -- 172.21.15.159:0/1166585790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f465c0d0ca0 msgr2=0x7f465c0ae020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:04.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.504+0000 7f4661000640 1 --2- 172.21.15.159:0/1166585790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f465c0d0ca0 0x7f465c0ae020 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f465c06d660 tx=0x7f465402bf60 comp rx=0 tx=0).stop 2024-04-04T06:16:04.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.504+0000 7f4661000640 1 -- 172.21.15.159:0/1166585790 shutdown_connections 2024-04-04T06:16:04.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.504+0000 7f4661000640 1 --2- 172.21.15.159:0/1166585790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f465c0d0ca0 0x7f465c0ae020 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:04.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.504+0000 7f4661000640 1 --2- 172.21.15.159:0/1166585790 >> v2:172.21.15.178:3300/0 conn(0x7f465c06b570 0x7f465c0d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:04.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.504+0000 7f4661000640 1 --2- 172.21.15.159:0/1166585790 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f465c15c650 0x7f465c0cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:04.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.504+0000 7f4661000640 1 -- 172.21.15.159:0/1166585790 >> 172.21.15.159:0/1166585790 conn(0x7f465c0b2d40 msgr2=0x7f465c0b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:04.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.504+0000 7f4661000640 1 -- 172.21.15.159:0/1166585790 shutdown_connections 2024-04-04T06:16:04.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.504+0000 7f4661000640 1 -- 172.21.15.159:0/1166585790 wait complete. 2024-04-04T06:16:04.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.507+0000 7f4661000640 1 Processor -- start 2024-04-04T06:16:04.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.507+0000 7f4661000640 1 -- start start 2024-04-04T06:16:04.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.508+0000 7f4661000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f465c06b570 0x7f465c15ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:04.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.508+0000 7f4661000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f465c0d0ca0 0x7f465c1f3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:04.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.508+0000 7f4661000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f465c15c650 0x7f465c15d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:04.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.508+0000 7f4661000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f465c0ae560 con 0x7f465c06b570 2024-04-04T06:16:04.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.508+0000 7f4661000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f465c0cff60 con 0x7f465c0d0ca0 2024-04-04T06:16:04.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.508+0000 7f4661000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f465c0ae6e0 con 0x7f465c15c650 2024-04-04T06:16:04.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.508+0000 7f4659e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f465c06b570 0x7f465c15ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:04.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.508+0000 7f4659e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f465c06b570 0x7f465c15ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36020/0 (socket says 172.21.15.159:36020) 2024-04-04T06:16:04.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.508+0000 7f4659e00640 1 -- 172.21.15.159:0/2168584283 learned_addr learned my addr 172.21.15.159:0/2168584283 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:04.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.509+0000 7f4651400640 1 --2- 172.21.15.159:0/2168584283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f465c0d0ca0 0x7f465c1f3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:04.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.509+0000 7f465a800640 1 --2- 172.21.15.159:0/2168584283 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f465c15c650 0x7f465c15d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:04.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.509+0000 7f4659e00640 1 -- 172.21.15.159:0/2168584283 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f465c15c650 msgr2=0x7f465c15d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:04.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.509+0000 7f4659e00640 1 --2- 172.21.15.159:0/2168584283 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f465c15c650 0x7f465c15d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:04.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.509+0000 7f4659e00640 1 -- 172.21.15.159:0/2168584283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f465c0d0ca0 msgr2=0x7f465c1f3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:04.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.509+0000 7f4659e00640 1 --2- 172.21.15.159:0/2168584283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f465c0d0ca0 0x7f465c1f3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:04.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.509+0000 7f4659e00640 1 -- 172.21.15.159:0/2168584283 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465c1f4910 con 0x7f465c06b570 2024-04-04T06:16:04.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.509+0000 7f4651400640 1 --2- 172.21.15.159:0/2168584283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f465c0d0ca0 0x7f465c1f3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:04.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.509+0000 7f4659e00640 1 --2- 172.21.15.159:0/2168584283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f465c06b570 0x7f465c15ad80 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f464400e870 tx=0x7f464400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:04.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.509+0000 7f4652a00640 1 -- 172.21.15.159:0/2168584283 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644017070 con 0x7f465c06b570 2024-04-04T06:16:04.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.509+0000 7f4652a00640 1 -- 172.21.15.159:0/2168584283 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f464400c940 con 0x7f465c06b570 2024-04-04T06:16:04.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.509+0000 7f4652a00640 1 -- 172.21.15.159:0/2168584283 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464400cd30 con 0x7f465c06b570 2024-04-04T06:16:04.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.509+0000 7f4661000640 1 -- 172.21.15.159:0/2168584283 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f465c1f4bf0 con 0x7f465c06b570 2024-04-04T06:16:04.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.510+0000 7f4661000640 1 -- 172.21.15.159:0/2168584283 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f465c0c6040 con 0x7f465c06b570 2024-04-04T06:16:04.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.511+0000 7f4661000640 1 -- 172.21.15.159:0/2168584283 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f465c068cd0 con 0x7f465c06b570 2024-04-04T06:16:04.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.512+0000 7f4652a00640 1 -- 172.21.15.159:0/2168584283 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f464401a070 con 0x7f465c06b570 2024-04-04T06:16:04.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.513+0000 7f4652a00640 1 --2- 172.21.15.159:0/2168584283 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f463806c8f0 0x7f463806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:04.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.513+0000 7f4652a00640 1 -- 172.21.15.159:0/2168584283 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f464408be10 con 0x7f465c06b570 2024-04-04T06:16:04.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.514+0000 7f4651400640 1 --2- 172.21.15.159:0/2168584283 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f463806c8f0 0x7f463806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:04.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.515+0000 7f4651400640 1 --2- 172.21.15.159:0/2168584283 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f463806c8f0 0x7f463806edb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f464c006fd0 tx=0x7f464c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:04.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.518+0000 7f4652a00640 1 -- 172.21.15.159:0/2168584283 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f464408f030 con 0x7f465c06b570 2024-04-04T06:16:04.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.822+0000 7f4661000640 1 -- 172.21.15.159:0/2168584283 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f465c0b6d30 con 0x7f465c06b570 2024-04-04T06:16:04.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.823+0000 7f4652a00640 1 -- 172.21.15.159:0/2168584283 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5191 (secure 0 0 0) 0x7f464405a1d0 con 0x7f465c06b570 2024-04-04T06:16:04.817 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:04.817 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5546,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/40219476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":40219476},{"type":"v1","addr":"172.21.15.178:6837","nonce":40219476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/4060483382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4060483382},{"type":"v1","addr":"172.21.15.178:6836","nonce":4060483382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8187,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1945665739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1945665739},{"type":"v1","addr":"172.21.15.159:6833","nonce":1945665739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:15:07.704835+0000","modified":"2024-04-04T06:16:03.519116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8535":{"gid":8535,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6835/1003751501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1003751501},{"type":"v1","addr":"172.21.15.159:6835","nonce":1003751501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8535,"qdb_cluster":[8535]},"id":7}]} 2024-04-04T06:16:04.818 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 90 2024-04-04T06:16:04.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.825+0000 7f4661000640 1 -- 172.21.15.159:0/2168584283 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f463806c8f0 msgr2=0x7f463806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:04.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.825+0000 7f4661000640 1 --2- 172.21.15.159:0/2168584283 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f463806c8f0 0x7f463806edb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f464c006fd0 tx=0x7f464c008040 comp rx=0 tx=0).stop 2024-04-04T06:16:04.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.825+0000 7f4661000640 1 -- 172.21.15.159:0/2168584283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f465c06b570 msgr2=0x7f465c15ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:04.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.825+0000 7f4661000640 1 --2- 172.21.15.159:0/2168584283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f465c06b570 0x7f465c15ad80 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f464400e870 tx=0x7f464400ed30 comp rx=0 tx=0).stop 2024-04-04T06:16:04.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.825+0000 7f4661000640 1 -- 172.21.15.159:0/2168584283 shutdown_connections 2024-04-04T06:16:04.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.825+0000 7f4661000640 1 --2- 172.21.15.159:0/2168584283 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f465c15c650 0x7f465c15d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:04.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.825+0000 7f4661000640 1 --2- 172.21.15.159:0/2168584283 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f463806c8f0 0x7f463806edb0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:04.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.825+0000 7f4661000640 1 --2- 172.21.15.159:0/2168584283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f465c0d0ca0 0x7f465c1f3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:04.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.825+0000 7f4661000640 1 --2- 172.21.15.159:0/2168584283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f465c06b570 0x7f465c15ad80 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:04.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.825+0000 7f4661000640 1 -- 172.21.15.159:0/2168584283 >> 172.21.15.159:0/2168584283 conn(0x7f465c0b2d40 msgr2=0x7f465c0b5e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:04.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.825+0000 7f4661000640 1 -- 172.21.15.159:0/2168584283 shutdown_connections 2024-04-04T06:16:04.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:04.825+0000 7f4661000640 1 -- 172.21.15.159:0/2168584283 wait complete. 2024-04-04T06:16:05.243 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:15:07.704835+0000', 'modified': '2024-04-04T06:16:03.519116+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 102, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8535}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8535': {'gid': 8535, 'name': 'a', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.159:6835/1003751501', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 1003751501}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 1003751501}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8535, 'qdb_cluster': [8535]} 2024-04-04T06:16:05.243 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-04T06:16:05.243 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-04T06:16:05.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.462+0000 7f3a77000640 1 Processor -- start 2024-04-04T06:16:05.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.462+0000 7f3a77000640 1 -- start start 2024-04-04T06:16:05.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.464+0000 7f3a77000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a7006b570 0x7f3a7006f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.464+0000 7f3a77000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a700b4710 con 0x7f3a70141900 2024-04-04T06:16:05.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.464+0000 7f3a77000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a7015fde0 con 0x7f3a7006b950 2024-04-04T06:16:05.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.464+0000 7f3a77000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3a701609f0 con 0x7f3a7006b570 2024-04-04T06:16:05.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.464+0000 7f3a76600640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f3a70141900 0x7f3a70141d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:34124/0 (socket says 172.21.15.159:34124) 2024-04-04T06:16:05.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.464+0000 7f3a76600640 1 -- 172.21.15.159:0/1201734917 learned_addr learned my addr 172.21.15.159:0/1201734917 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:16:05.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.465+0000 7f3a6fe00640 1 -- 172.21.15.159:0/1201734917 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 562609514 0 0) 0x7f3a700b4710 con 0x7f3a70141900 2024-04-04T06:16:05.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.465+0000 7f3a6fe00640 1 -- 172.21.15.159:0/1201734917 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a58003520 con 0x7f3a70141900 2024-04-04T06:16:05.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.465+0000 7f3a75c00640 1 --2- 172.21.15.159:0/1201734917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a7006b570 0x7f3a7006f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.465+0000 7f3a6fe00640 1 -- 172.21.15.159:0/1201734917 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 719926618 0 0) 0x7f3a58003520 con 0x7f3a70141900 2024-04-04T06:16:05.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.465+0000 7f3a6fe00640 1 -- 172.21.15.159:0/1201734917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a7006b570 msgr2=0x7f3a7006f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.465+0000 7f3a6fe00640 1 --2- 172.21.15.159:0/1201734917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a7006b570 0x7f3a7006f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.465+0000 7f3a6fe00640 1 -- 172.21.15.159:0/1201734917 >> v1:172.21.15.178:6789/0 conn(0x7f3a7006b950 legacy=0x7f3a700b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.465+0000 7f3a6fe00640 1 -- 172.21.15.159:0/1201734917 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a701600c0 con 0x7f3a70141900 2024-04-04T06:16:05.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.465+0000 7f3a6fe00640 1 -- 172.21.15.159:0/1201734917 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f3a680019f0 con 0x7f3a70141900 2024-04-04T06:16:05.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.465+0000 7f3a6fe00640 1 -- 172.21.15.159:0/1201734917 >> v1:172.21.15.159:6789/0 conn(0x7f3a70141900 legacy=0x7f3a70141d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.465+0000 7f3a6fe00640 1 --2- 172.21.15.159:0/1201734917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a580039e0 0x7f3a58003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.465+0000 7f3a6fe00640 1 --2- 172.21.15.159:0/1201734917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a7006b570 0x7f3a58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.466+0000 7f3a6fe00640 1 --2- 172.21.15.159:0/1201734917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a58004d40 0x7f3a580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.466+0000 7f3a6fe00640 1 -- 172.21.15.159:0/1201734917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701609f0 con 0x7f3a580039e0 2024-04-04T06:16:05.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.466+0000 7f3a6fe00640 1 -- 172.21.15.159:0/1201734917 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5800bb50 con 0x7f3a58004d40 2024-04-04T06:16:05.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.466+0000 7f3a6fe00640 1 -- 172.21.15.159:0/1201734917 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3a5800bd20 con 0x7f3a7006b570 2024-04-04T06:16:05.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.466+0000 7f3a75c00640 1 --2- 172.21.15.159:0/1201734917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a580039e0 0x7f3a58003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.466+0000 7f3a76600640 1 --2- 172.21.15.159:0/1201734917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a58004d40 0x7f3a580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.466+0000 7f3a75c00640 1 -- 172.21.15.159:0/1201734917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a7006b570 msgr2=0x7f3a58004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.466+0000 7f3a75c00640 1 --2- 172.21.15.159:0/1201734917 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a7006b570 0x7f3a58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.466+0000 7f3a75c00640 1 -- 172.21.15.159:0/1201734917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a58004d40 msgr2=0x7f3a580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.466+0000 7f3a75c00640 1 --2- 172.21.15.159:0/1201734917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a58004d40 0x7f3a580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.466+0000 7f3a75c00640 1 -- 172.21.15.159:0/1201734917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a58003520 con 0x7f3a580039e0 2024-04-04T06:16:05.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.466+0000 7f3a76600640 1 --2- 172.21.15.159:0/1201734917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a58004d40 0x7f3a580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:05.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.467+0000 7f3a75c00640 1 --2- 172.21.15.159:0/1201734917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a580039e0 0x7f3a58003e00 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400ba60 tx=0x7f3a58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e29cfdb1d75d5bb server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:05.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.468+0000 7f3a6fe00640 1 -- 172.21.15.159:0/1201734917 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64036020 con 0x7f3a580039e0 2024-04-04T06:16:05.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.468+0000 7f3a6fe00640 1 -- 172.21.15.159:0/1201734917 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a64032680 con 0x7f3a580039e0 2024-04-04T06:16:05.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.468+0000 7f3a6fe00640 1 -- 172.21.15.159:0/1201734917 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6400b600 con 0x7f3a580039e0 2024-04-04T06:16:05.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.468+0000 7f3a77000640 1 -- 172.21.15.159:0/1201734917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a580039e0 msgr2=0x7f3a58003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.468+0000 7f3a77000640 1 --2- 172.21.15.159:0/1201734917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a580039e0 0x7f3a58003e00 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400ba60 tx=0x7f3a58005830 comp rx=0 tx=0).stop 2024-04-04T06:16:05.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.468+0000 7f3a76600640 1 -- 172.21.15.159:0/1201734917 reap_dead start 2024-04-04T06:16:05.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.468+0000 7f3a77000640 1 -- 172.21.15.159:0/1201734917 shutdown_connections 2024-04-04T06:16:05.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.468+0000 7f3a77000640 1 -- 172.21.15.159:0/1201734917 >> 172.21.15.159:0/1201734917 conn(0x7f3a700b2c90 msgr2=0x7f3a700b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:05.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.468+0000 7f3a77000640 1 -- 172.21.15.159:0/1201734917 shutdown_connections 2024-04-04T06:16:05.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.468+0000 7f3a77000640 1 -- 172.21.15.159:0/1201734917 wait complete. 2024-04-04T06:16:05.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.470+0000 7f3a77000640 1 Processor -- start 2024-04-04T06:16:05.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.470+0000 7f3a77000640 1 -- start start 2024-04-04T06:16:05.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.471+0000 7f3a77000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a580039e0 0x7f3a70152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.471+0000 7f3a77000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a7013a690 0x7f3a7013f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.471+0000 7f3a77000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a700cfe30 0x7f3a7013acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.471+0000 7f3a77000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7006bdb0 con 0x7f3a7013a690 2024-04-04T06:16:05.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.471+0000 7f3a77000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3a70160680 con 0x7f3a580039e0 2024-04-04T06:16:05.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.471+0000 7f3a77000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3a70070cb0 con 0x7f3a700cfe30 2024-04-04T06:16:05.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.471+0000 7f3a75200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a7013a690 0x7f3a7013f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.472+0000 7f3a75200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a7013a690 0x7f3a7013f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36038/0 (socket says 172.21.15.159:36038) 2024-04-04T06:16:05.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.472+0000 7f3a75200640 1 -- 172.21.15.159:0/1233368585 learned_addr learned my addr 172.21.15.159:0/1233368585 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:05.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.472+0000 7f3a75200640 1 -- 172.21.15.159:0/1233368585 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a700cfe30 msgr2=0x7f3a7013acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.472+0000 7f3a75200640 1 --2- 172.21.15.159:0/1233368585 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a700cfe30 0x7f3a7013acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.472+0000 7f3a75200640 1 -- 172.21.15.159:0/1233368585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a580039e0 msgr2=0x7f3a70152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.472+0000 7f3a75200640 1 --2- 172.21.15.159:0/1233368585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a580039e0 0x7f3a70152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.472+0000 7f3a75200640 1 -- 172.21.15.159:0/1233368585 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7013cf00 con 0x7f3a7013a690 2024-04-04T06:16:05.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.472+0000 7f3a75200640 1 --2- 172.21.15.159:0/1233368585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a7013a690 0x7f3a7013f710 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00c430 tx=0x7f3a5c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:05.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.473+0000 7f3a6d600640 1 -- 172.21.15.159:0/1233368585 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c017070 con 0x7f3a7013a690 2024-04-04T06:16:05.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.473+0000 7f3a6d600640 1 -- 172.21.15.159:0/1233368585 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a5c0052f0 con 0x7f3a7013a690 2024-04-04T06:16:05.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.473+0000 7f3a77000640 1 -- 172.21.15.159:0/1233368585 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7013d1e0 con 0x7f3a7013a690 2024-04-04T06:16:05.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.473+0000 7f3a6d600640 1 -- 172.21.15.159:0/1233368585 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c003830 con 0x7f3a7013a690 2024-04-04T06:16:05.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.473+0000 7f3a77000640 1 -- 172.21.15.159:0/1233368585 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a7013fc50 con 0x7f3a7013a690 2024-04-04T06:16:05.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.475+0000 7f3a6d600640 1 -- 172.21.15.159:0/1233368585 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a5c01a070 con 0x7f3a7013a690 2024-04-04T06:16:05.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.476+0000 7f3a6d600640 1 --2- 172.21.15.159:0/1233368585 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3a5006c8f0 0x7f3a5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.476+0000 7f3a6d600640 1 -- 172.21.15.159:0/1233368585 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f3a5c08af80 con 0x7f3a7013a690 2024-04-04T06:16:05.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.477+0000 7f3a75c00640 1 --2- 172.21.15.159:0/1233368585 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3a5006c8f0 0x7f3a5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.478+0000 7f3a75c00640 1 --2- 172.21.15.159:0/1233368585 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3a5006c8f0 0x7f3a5006edb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400ba30 tx=0x7f3a64032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:05.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.478+0000 7f3a7b21d740 1 Processor -- start 2024-04-04T06:16:05.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.479+0000 7f3a7b21d740 1 -- start start 2024-04-04T06:16:05.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.480+0000 7f3a7b21d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f00c876850 0x55f00c88a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.480+0000 7f3a7b21d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c876c30 0x55f00c879090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.480+0000 7f3a7b21d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f00c8795d0 0x55f00c879a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.480+0000 7f3a7b21d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f00c810ab0 con 0x55f00c876c30 2024-04-04T06:16:05.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.480+0000 7f3a7b21d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f00c7b2fa0 con 0x55f00c876850 2024-04-04T06:16:05.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.480+0000 7f3a7b21d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55f00c75b560 con 0x55f00c8795d0 2024-04-04T06:16:05.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.481+0000 7f3a75200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c876c30 0x55f00c879090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.481+0000 7f3a75200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c876c30 0x55f00c879090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36042/0 (socket says 172.21.15.159:36042) 2024-04-04T06:16:05.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.481+0000 7f3a75200640 1 -- 172.21.15.159:0/658182098 learned_addr learned my addr 172.21.15.159:0/658182098 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:05.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.481+0000 7f3a75200640 1 -- 172.21.15.159:0/658182098 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f00c8795d0 msgr2=0x55f00c879a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.481+0000 7f3a75200640 1 --2- 172.21.15.159:0/658182098 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f00c8795d0 0x55f00c879a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.481+0000 7f3a75200640 1 -- 172.21.15.159:0/658182098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f00c876850 msgr2=0x55f00c88a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.481+0000 7f3a75200640 1 --2- 172.21.15.159:0/658182098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f00c876850 0x55f00c88a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.481+0000 7f3a75200640 1 -- 172.21.15.159:0/658182098 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f00c4c72d0 con 0x55f00c876c30 2024-04-04T06:16:05.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.481+0000 7f3a75200640 1 --2- 172.21.15.159:0/658182098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c876c30 0x55f00c879090 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0893b0 tx=0x7f3a5c0586b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae64028a163819f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:05.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.481+0000 7f3a62000640 1 -- 172.21.15.159:0/658182098 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c056f40 con 0x55f00c876c30 2024-04-04T06:16:05.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.481+0000 7f3a62000640 1 -- 172.21.15.159:0/658182098 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a5c05f040 con 0x55f00c876c30 2024-04-04T06:16:05.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.481+0000 7f3a62000640 1 -- 172.21.15.159:0/658182098 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c058090 con 0x55f00c876c30 2024-04-04T06:16:05.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.482+0000 7f3a7b21d740 1 -- 172.21.15.159:0/658182098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c876c30 msgr2=0x55f00c879090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.482+0000 7f3a7b21d740 1 --2- 172.21.15.159:0/658182098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c876c30 0x55f00c879090 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0893b0 tx=0x7f3a5c0586b0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.482+0000 7f3a7b21d740 1 -- 172.21.15.159:0/658182098 shutdown_connections 2024-04-04T06:16:05.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.482+0000 7f3a7b21d740 1 --2- 172.21.15.159:0/658182098 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f00c8795d0 0x55f00c879a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.482+0000 7f3a7b21d740 1 --2- 172.21.15.159:0/658182098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c876c30 0x55f00c879090 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.482+0000 7f3a7b21d740 1 --2- 172.21.15.159:0/658182098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f00c876850 0x55f00c88a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.482+0000 7f3a7b21d740 1 -- 172.21.15.159:0/658182098 >> 172.21.15.159:0/658182098 conn(0x55f00c84b510 msgr2=0x55f00c894090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:05.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.482+0000 7f3a7b21d740 1 -- 172.21.15.159:0/658182098 shutdown_connections 2024-04-04T06:16:05.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.483+0000 7f3a7b21d740 1 -- 172.21.15.159:0/658182098 wait complete. 2024-04-04T06:16:05.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.483+0000 7f3a7b21d740 1 -- start start 2024-04-04T06:16:05.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.483+0000 7f3a7b21d740 1 Processor -- start 2024-04-04T06:16:05.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.484+0000 7f3a7b21d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f00c876c30 0x55f00c8af840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.484+0000 7f3a7b21d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f00c8795d0 0x55f00c86f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.484+0000 7f3a7b21d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c8ac8f0 0x55f00c8acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.481 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.484+0000 7f3a7b21d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f00c76e920 con 0x55f00c8ac8f0 2024-04-04T06:16:05.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.484+0000 7f3a7b21d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f00c7aef50 con 0x55f00c876c30 2024-04-04T06:16:05.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.484+0000 7f3a7b21d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55f00c84ef60 con 0x55f00c8795d0 2024-04-04T06:16:05.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.484+0000 7f3a7b21d740 10 client.0 _ll_register_callbacks cb 0x55f00c76e310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:16:05.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.485+0000 7f3a76600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c8ac8f0 0x55f00c8acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.485+0000 7f3a76600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c8ac8f0 0x55f00c8acd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36050/0 (socket says 172.21.15.159:36050) 2024-04-04T06:16:05.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.485+0000 7f3a76600640 1 -- 172.21.15.159:0/1002204018 learned_addr learned my addr 172.21.15.159:0/1002204018 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:05.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.485+0000 7f3a75200640 1 --2- 172.21.15.159:0/1002204018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f00c8795d0 0x55f00c86f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.485+0000 7f3a76600640 1 -- 172.21.15.159:0/1002204018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f00c8795d0 msgr2=0x55f00c86f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.485+0000 7f3a76600640 1 --2- 172.21.15.159:0/1002204018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f00c8795d0 0x55f00c86f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.485+0000 7f3a75c00640 1 --2- 172.21.15.159:0/1002204018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f00c876c30 0x55f00c8af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.485+0000 7f3a76600640 1 -- 172.21.15.159:0/1002204018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f00c876c30 msgr2=0x55f00c8af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.485+0000 7f3a76600640 1 --2- 172.21.15.159:0/1002204018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f00c876c30 0x55f00c8af840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.482 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.485+0000 7f3a76600640 1 -- 172.21.15.159:0/1002204018 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f00c5916b0 con 0x55f00c8ac8f0 2024-04-04T06:16:05.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.486+0000 7f3a76600640 1 --2- 172.21.15.159:0/1002204018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c8ac8f0 0x55f00c8acd70 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f3a70141ce0 tx=0x7f3a6800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57924c092d3437ac server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:05.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.486+0000 7f3a56a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:16:05.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.486+0000 7f3a56a00640 1 -- 172.21.15.159:0/1002204018 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6800b600 con 0x55f00c8ac8f0 2024-04-04T06:16:05.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.486+0000 7f3a56a00640 1 -- 172.21.15.159:0/1002204018 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f3a68013ae0 con 0x55f00c8ac8f0 2024-04-04T06:16:05.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.486+0000 7f3a7b21d740 1 -- 172.21.15.159:0/1002204018 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a5c05f040 con 0x55f00c8ac8f0 2024-04-04T06:16:05.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.486+0000 7f3a56a00640 1 -- 172.21.15.159:0/1002204018 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f3a68013ce0 con 0x55f00c8ac8f0 2024-04-04T06:16:05.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.486+0000 7f3a7b21d740 10 client.8604 fetch_fsmap learned FSMap version 90 2024-04-04T06:16:05.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.486+0000 7f3a7b21d740 1 -- 172.21.15.159:0/1002204018 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55f00c87ba90 con 0x55f00c8ac8f0 2024-04-04T06:16:05.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.487+0000 7f3a56a00640 1 -- 172.21.15.159:0/1002204018 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a5c05f040 con 0x55f00c8ac8f0 2024-04-04T06:16:05.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.487+0000 7f3a56a00640 1 -- 172.21.15.159:0/1002204018 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 90) ==== 3494+0+0 (secure 0 0 0) 0x7f3a6800b7a0 con 0x55f00c8ac8f0 2024-04-04T06:16:05.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.487+0000 7f3a7b21d740 10 client.8604 fetch_fsmap finished waiting for FSMap version 90 2024-04-04T06:16:05.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.487+0000 7f3a7b21d740 10 client.8604 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:16:05.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.487+0000 7f3a7b21d740 20 client.8604 populate_metadata read hostname 'smithi159' 2024-04-04T06:16:05.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.487+0000 7f3a7b21d740 1 --2- 172.21.15.159:0/1002204018 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55f00c8b85d0 0x55f00c8ba9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.487+0000 7f3a7b21d740 4 client.8604 mds_command: new command op to 8535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:16:05.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.487+0000 7f3a7b21d740 1 -- 172.21.15.159:0/1002204018 --> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f00c7c04e0 con 0x55f00c8b85d0 2024-04-04T06:16:05.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.487+0000 7f3a75c00640 1 --2- 172.21.15.159:0/1002204018 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55f00c8b85d0 0x55f00c8ba9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.488+0000 7f3a75c00640 1 --2- 172.21.15.159:0/1002204018 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55f00c8b85d0 0x55f00c8ba9c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9065943fb8708c2 server_cookie=16788b37ae163ee4 in_seq=0 out_seq=0 2024-04-04T06:16:05.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.488+0000 7f3a56a00640 10 client.8604 ms_handle_connect on v2:172.21.15.159:6834/1003751501 2024-04-04T06:16:05.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.488+0000 7f3a56a00640 1 -- 172.21.15.159:0/1002204018 <== mds.0 v2:172.21.15.159:6834/1003751501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f00c7c04e0 con 0x55f00c8b85d0 2024-04-04T06:16:05.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.488+0000 7f3a56a00640 10 client.8604 handle_command_reply: tid=0 2024-04-04T06:16:05.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.489+0000 7f3a7b21d740 1 client.8604 shutdown 2024-04-04T06:16:05.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.489+0000 7f3a7b21d740 1 -- 172.21.15.159:0/1002204018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c8ac8f0 msgr2=0x55f00c8acd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.489+0000 7f3a7b21d740 1 --2- 172.21.15.159:0/1002204018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c8ac8f0 0x55f00c8acd70 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f3a70141ce0 tx=0x7f3a6800dcc0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.489+0000 7f3a7b21d740 1 -- 172.21.15.159:0/1002204018 shutdown_connections 2024-04-04T06:16:05.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.489+0000 7f3a7b21d740 1 --2- 172.21.15.159:0/1002204018 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55f00c8b85d0 0x55f00c8ba9c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.489+0000 7f3a7b21d740 1 --2- 172.21.15.159:0/1002204018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c8ac8f0 0x55f00c8acd70 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.489+0000 7f3a7b21d740 1 --2- 172.21.15.159:0/1002204018 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f00c8795d0 0x55f00c86f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.489+0000 7f3a7b21d740 1 --2- 172.21.15.159:0/1002204018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f00c876c30 0x55f00c8af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.489+0000 7f3a56a00640 0 client.8604 ms_handle_reset on v2:172.21.15.159:6834/1003751501 2024-04-04T06:16:05.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.489+0000 7f3a7b21d740 1 -- 172.21.15.159:0/1002204018 >> 172.21.15.159:0/1002204018 conn(0x55f00c876850 msgr2=0x55f00c848a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:05.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.489+0000 7f3a7b21d740 1 -- 172.21.15.159:0/1002204018 shutdown_connections 2024-04-04T06:16:05.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.489+0000 7f3a7b21d740 1 -- 172.21.15.159:0/1002204018 wait complete. 2024-04-04T06:16:05.486 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.489+0000 7f3a7b21d740 20 client.8604 trim_cache size 0 max 16384 2024-04-04T06:16:05.500 INFO:tasks.ceph.mds.a.smithi159.stderr:2024-04-04T06:16:05.505+0000 7f1ea3200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:16:05.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.509+0000 7f3a7b21d740 1 Processor -- start 2024-04-04T06:16:05.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.509+0000 7f3a7b21d740 1 -- start start 2024-04-04T06:16:05.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.510+0000 7f3a7b21d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f00c876c30 0x55f00c86ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.510+0000 7f3a7b21d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f00c8795d0 0x55f00c86f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.510+0000 7f3a7b21d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c8ac8f0 0x55f00c870a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.510+0000 7f3a7b21d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701582b0 con 0x55f00c8ac8f0 2024-04-04T06:16:05.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.510+0000 7f3a7b21d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3a70140820 con 0x55f00c8795d0 2024-04-04T06:16:05.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.510+0000 7f3a7b21d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3a70140e70 con 0x55f00c876c30 2024-04-04T06:16:05.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.510+0000 7f3a76600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c8ac8f0 0x55f00c870a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.510+0000 7f3a76600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c8ac8f0 0x55f00c870a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36052/0 (socket says 172.21.15.159:36052) 2024-04-04T06:16:05.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.510+0000 7f3a76600640 1 -- 172.21.15.159:0/3038337685 learned_addr learned my addr 172.21.15.159:0/3038337685 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:05.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.511+0000 7f3a75c00640 1 --2- 172.21.15.159:0/3038337685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f00c876c30 0x55f00c86ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.511+0000 7f3a75200640 1 --2- 172.21.15.159:0/3038337685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f00c8795d0 0x55f00c86f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.511+0000 7f3a76600640 1 -- 172.21.15.159:0/3038337685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f00c876c30 msgr2=0x55f00c86ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.511+0000 7f3a76600640 1 --2- 172.21.15.159:0/3038337685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f00c876c30 0x55f00c86ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.511+0000 7f3a76600640 1 -- 172.21.15.159:0/3038337685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f00c8795d0 msgr2=0x55f00c86f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.511+0000 7f3a76600640 1 --2- 172.21.15.159:0/3038337685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f00c8795d0 0x55f00c86f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.511+0000 7f3a76600640 1 -- 172.21.15.159:0/3038337685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f00c87c190 con 0x55f00c8ac8f0 2024-04-04T06:16:05.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.511+0000 7f3a75200640 1 --2- 172.21.15.159:0/3038337685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f00c8795d0 0x55f00c86f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:05.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.511+0000 7f3a76600640 1 --2- 172.21.15.159:0/3038337685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c8ac8f0 0x55f00c870a80 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800eff0 tx=0x7f3a68007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7ee5252bf1d2d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:05.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.511+0000 7f3a61600640 1 -- 172.21.15.159:0/3038337685 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68023040 con 0x55f00c8ac8f0 2024-04-04T06:16:05.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.511+0000 7f3a61600640 1 -- 172.21.15.159:0/3038337685 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a6800b800 con 0x55f00c8ac8f0 2024-04-04T06:16:05.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.511+0000 7f3a61600640 1 -- 172.21.15.159:0/3038337685 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68013650 con 0x55f00c8ac8f0 2024-04-04T06:16:05.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.512+0000 7f3a7b21d740 1 -- 172.21.15.159:0/3038337685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c8ac8f0 msgr2=0x55f00c870a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.512+0000 7f3a7b21d740 1 --2- 172.21.15.159:0/3038337685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c8ac8f0 0x55f00c870a80 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800eff0 tx=0x7f3a68007000 comp rx=0 tx=0).stop 2024-04-04T06:16:05.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.512+0000 7f3a7b21d740 1 -- 172.21.15.159:0/3038337685 shutdown_connections 2024-04-04T06:16:05.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.512+0000 7f3a7b21d740 1 --2- 172.21.15.159:0/3038337685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c8ac8f0 0x55f00c870a80 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.512+0000 7f3a7b21d740 1 --2- 172.21.15.159:0/3038337685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f00c8795d0 0x55f00c86f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.512+0000 7f3a7b21d740 1 --2- 172.21.15.159:0/3038337685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f00c876c30 0x55f00c86ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.512+0000 7f3a7b21d740 1 -- 172.21.15.159:0/3038337685 >> 172.21.15.159:0/3038337685 conn(0x55f00c876850 msgr2=0x55f00c87d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:05.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.512+0000 7f3a7b21d740 1 -- 172.21.15.159:0/3038337685 shutdown_connections 2024-04-04T06:16:05.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.512+0000 7f3a7b21d740 1 -- 172.21.15.159:0/3038337685 wait complete. 2024-04-04T06:16:05.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.513+0000 7f3a7b21d740 1 -- start start 2024-04-04T06:16:05.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.513+0000 7f3a7b21d740 1 Processor -- start 2024-04-04T06:16:05.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.514+0000 7f3a7b21d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c876c30 0x55f00c8c02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.514+0000 7f3a7b21d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f00c8795d0 0x55f00c875be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.514+0000 7f3a7b21d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f00c8ac8f0 0x55f00c8adf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.514+0000 7f3a7b21d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f00c87ab60 con 0x55f00c876c30 2024-04-04T06:16:05.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.514+0000 7f3a7b21d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f00c8ad7e0 con 0x55f00c8ac8f0 2024-04-04T06:16:05.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.514+0000 7f3a7b21d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55f00c878e50 con 0x55f00c8795d0 2024-04-04T06:16:05.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.514+0000 7f3a7b21d740 10 client.0 _ll_register_callbacks cb 0x7f3a6800d4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:16:05.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.514+0000 7f3a75c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c876c30 0x55f00c8c02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.514+0000 7f3a75c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c876c30 0x55f00c8c02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36068/0 (socket says 172.21.15.159:36068) 2024-04-04T06:16:05.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.514+0000 7f3a75c00640 1 -- 172.21.15.159:0/1885714937 learned_addr learned my addr 172.21.15.159:0/1885714937 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:05.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.515+0000 7f3a76600640 1 --2- 172.21.15.159:0/1885714937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f00c8ac8f0 0x55f00c8adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.515+0000 7f3a75200640 1 --2- 172.21.15.159:0/1885714937 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f00c8795d0 0x55f00c875be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.515+0000 7f3a75c00640 1 -- 172.21.15.159:0/1885714937 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f00c8795d0 msgr2=0x55f00c875be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.515+0000 7f3a75c00640 1 --2- 172.21.15.159:0/1885714937 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f00c8795d0 0x55f00c875be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.515+0000 7f3a75c00640 1 -- 172.21.15.159:0/1885714937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f00c8ac8f0 msgr2=0x55f00c8adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.515+0000 7f3a75c00640 1 --2- 172.21.15.159:0/1885714937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f00c8ac8f0 0x55f00c8adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.515+0000 7f3a75c00640 1 -- 172.21.15.159:0/1885714937 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f00c8c0800 con 0x55f00c876c30 2024-04-04T06:16:05.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.515+0000 7f3a75c00640 1 --2- 172.21.15.159:0/1885714937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c876c30 0x55f00c8c02c0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f3a6402bfd0 tx=0x7f3a6402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f838edc6e2256146 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:05.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.515+0000 7f3a56a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:16:05.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.515+0000 7f3a56a00640 1 -- 172.21.15.159:0/1885714937 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64042070 con 0x55f00c876c30 2024-04-04T06:16:05.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.516+0000 7f3a7b21d740 1 -- 172.21.15.159:0/1885714937 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a6800b800 con 0x55f00c876c30 2024-04-04T06:16:05.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.516+0000 7f3a56a00640 1 -- 172.21.15.159:0/1885714937 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f3a64004220 con 0x55f00c876c30 2024-04-04T06:16:05.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.516+0000 7f3a56a00640 1 -- 172.21.15.159:0/1885714937 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f3a64036040 con 0x55f00c876c30 2024-04-04T06:16:05.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.516+0000 7f3a7b21d740 10 client.8610 fetch_fsmap learned FSMap version 90 2024-04-04T06:16:05.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.516+0000 7f3a7b21d740 1 -- 172.21.15.159:0/1885714937 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55f00c8beb80 con 0x55f00c876c30 2024-04-04T06:16:05.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.516+0000 7f3a56a00640 1 -- 172.21.15.159:0/1885714937 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a6800b800 con 0x55f00c876c30 2024-04-04T06:16:05.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.516+0000 7f3a56a00640 1 -- 172.21.15.159:0/1885714937 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 90) ==== 3494+0+0 (secure 0 0 0) 0x7f3a64062080 con 0x55f00c876c30 2024-04-04T06:16:05.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.516+0000 7f3a7b21d740 10 client.8610 fetch_fsmap finished waiting for FSMap version 90 2024-04-04T06:16:05.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.516+0000 7f3a7b21d740 10 client.8610 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:16:05.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.516+0000 7f3a7b21d740 20 client.8610 populate_metadata read hostname 'smithi159' 2024-04-04T06:16:05.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.516+0000 7f3a7b21d740 1 --2- 172.21.15.159:0/1885714937 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55f00c9164d0 0x55f00c9168b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.516+0000 7f3a7b21d740 4 client.8610 mds_command: new command op to 8535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-04T06:16:05.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.516+0000 7f3a7b21d740 1 -- 172.21.15.159:0/1885714937 --> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] -- command(tid 0: {"prefix": "status"}) -- 0x55f00c7ba830 con 0x55f00c9164d0 2024-04-04T06:16:05.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.517+0000 7f3a75200640 1 --2- 172.21.15.159:0/1885714937 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55f00c9164d0 0x55f00c9168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.517+0000 7f3a75200640 1 --2- 172.21.15.159:0/1885714937 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55f00c9164d0 0x55f00c9168b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbc4d2bc3ebed1e8 server_cookie=90161ff978c449f6 in_seq=0 out_seq=0 2024-04-04T06:16:05.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.517+0000 7f3a56a00640 10 client.8610 ms_handle_connect on v2:172.21.15.159:6834/1003751501 2024-04-04T06:16:05.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.517+0000 7f3a56a00640 1 -- 172.21.15.159:0/1885714937 <== mds.0 v2:172.21.15.159:6834/1003751501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55f00c7ba830 con 0x55f00c9164d0 2024-04-04T06:16:05.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.517+0000 7f3a56a00640 10 client.8610 handle_command_reply: tid=0 2024-04-04T06:16:05.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.517+0000 7f3a7b21d740 1 client.8610 shutdown 2024-04-04T06:16:05.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.518+0000 7f3a7b21d740 1 -- 172.21.15.159:0/1885714937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c876c30 msgr2=0x55f00c8c02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.518+0000 7f3a7b21d740 1 --2- 172.21.15.159:0/1885714937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c876c30 0x55f00c8c02c0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f3a6402bfd0 tx=0x7f3a6402e000 comp rx=0 tx=0).stop 2024-04-04T06:16:05.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.518+0000 7f3a7b21d740 1 -- 172.21.15.159:0/1885714937 shutdown_connections 2024-04-04T06:16:05.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.518+0000 7f3a7b21d740 1 --2- 172.21.15.159:0/1885714937 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55f00c9164d0 0x55f00c9168b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.518+0000 7f3a7b21d740 1 --2- 172.21.15.159:0/1885714937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f00c8ac8f0 0x55f00c8adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.518+0000 7f3a7b21d740 1 --2- 172.21.15.159:0/1885714937 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55f00c8795d0 0x55f00c875be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.518+0000 7f3a56a00640 0 client.8610 ms_handle_reset on v2:172.21.15.159:6834/1003751501 2024-04-04T06:16:05.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.518+0000 7f3a7b21d740 1 --2- 172.21.15.159:0/1885714937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f00c876c30 0x55f00c8c02c0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.518+0000 7f3a7b21d740 1 -- 172.21.15.159:0/1885714937 >> 172.21.15.159:0/1885714937 conn(0x55f00c876850 msgr2=0x55f00c870d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:05.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.518+0000 7f3a7b21d740 1 -- 172.21.15.159:0/1885714937 shutdown_connections 2024-04-04T06:16:05.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.518+0000 7f3a7b21d740 1 -- 172.21.15.159:0/1885714937 wait complete. 2024-04-04T06:16:05.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.518+0000 7f3a7b21d740 20 client.8610 trim_cache size 0 max 16384 2024-04-04T06:16:05.516 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:16:05.517 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_fsid": "30311045-f714-48fa-832b-4cb2ba8bf0bf", 2024-04-04T06:16:05.517 INFO:teuthology.orchestra.run.smithi159.stdout: "whoami": 0, 2024-04-04T06:16:05.517 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 8535, 2024-04-04T06:16:05.517 INFO:teuthology.orchestra.run.smithi159.stdout: "want_state": "up:active", 2024-04-04T06:16:05.517 INFO:teuthology.orchestra.run.smithi159.stdout: "state": "up:active", 2024-04-04T06:16:05.517 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs", 2024-04-04T06:16:05.517 INFO:teuthology.orchestra.run.smithi159.stdout: "rank_uptime": 5.0148296170000002, 2024-04-04T06:16:05.517 INFO:teuthology.orchestra.run.smithi159.stdout: "mdsmap_epoch": 90, 2024-04-04T06:16:05.517 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch": 102, 2024-04-04T06:16:05.517 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch_barrier": 102, 2024-04-04T06:16:05.517 INFO:teuthology.orchestra.run.smithi159.stdout: "uptime": 6.0297837330000004 2024-04-04T06:16:05.517 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:16:05.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.519+0000 7f3a77000640 1 -- 172.21.15.159:0/1233368585 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3a5006c8f0 msgr2=0x7f3a5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.519+0000 7f3a77000640 1 --2- 172.21.15.159:0/1233368585 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3a5006c8f0 0x7f3a5006edb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400ba30 tx=0x7f3a64032000 comp rx=0 tx=0).stop 2024-04-04T06:16:05.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.519+0000 7f3a77000640 1 -- 172.21.15.159:0/1233368585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a7013a690 msgr2=0x7f3a7013f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.519+0000 7f3a77000640 1 --2- 172.21.15.159:0/1233368585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a7013a690 0x7f3a7013f710 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00c430 tx=0x7f3a5c00c8f0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.519+0000 7f3a77000640 1 -- 172.21.15.159:0/1233368585 shutdown_connections 2024-04-04T06:16:05.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.519+0000 7f3a77000640 1 --2- 172.21.15.159:0/1233368585 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a700cfe30 0x7f3a7013acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.519+0000 7f3a77000640 1 --2- 172.21.15.159:0/1233368585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a7013a690 0x7f3a7013f710 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.519+0000 7f3a77000640 1 --2- 172.21.15.159:0/1233368585 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3a5006c8f0 0x7f3a5006edb0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.519+0000 7f3a77000640 1 --2- 172.21.15.159:0/1233368585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a580039e0 0x7f3a70152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.519+0000 7f3a77000640 1 -- 172.21.15.159:0/1233368585 >> 172.21.15.159:0/1233368585 conn(0x7f3a700b2c90 msgr2=0x7f3a7006f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:05.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.519+0000 7f3a77000640 1 -- 172.21.15.159:0/1233368585 shutdown_connections 2024-04-04T06:16:05.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.519+0000 7f3a77000640 1 -- 172.21.15.159:0/1233368585 wait complete. 2024-04-04T06:16:05.538 DEBUG:teuthology.orchestra.run.smithi159:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-04T06:16:05.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:16:05.545 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-04T06:16:05.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.738+0000 7fdea2600640 1 Processor -- start 2024-04-04T06:16:05.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.738+0000 7fdea2600640 1 -- start start 2024-04-04T06:16:05.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.739+0000 7fdea2600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fde9c0d3d50 0x7fde9c0d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.740+0000 7fdea2600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde9c0d23b0 con 0x7fde9c0b6310 2024-04-04T06:16:05.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.740+0000 7fdea2600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde9c0d2690 con 0x7fde9c15e420 2024-04-04T06:16:05.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.740+0000 7fdea2600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fde9c0d2fc0 con 0x7fde9c0d3d50 2024-04-04T06:16:05.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.740+0000 7fdea1c00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fde9c0b6310 0x7fde9c0b7460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:34128/0 (socket says 172.21.15.159:34128) 2024-04-04T06:16:05.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.740+0000 7fdea1c00640 1 -- 172.21.15.159:0/2220193325 learned_addr learned my addr 172.21.15.159:0/2220193325 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:16:05.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.740+0000 7fde93e00640 1 --2- 172.21.15.159:0/2220193325 >> v2:172.21.15.178:3300/0 conn(0x7fde9c0d3d50 0x7fde9c0d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.740+0000 7fde9be00640 1 -- 172.21.15.159:0/2220193325 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 242270857 0 0) 0x7fde9c0d23b0 con 0x7fde9c0b6310 2024-04-04T06:16:05.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.741+0000 7fde9be00640 1 -- 172.21.15.159:0/2220193325 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde80003520 con 0x7fde9c0b6310 2024-04-04T06:16:05.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.741+0000 7fde9be00640 1 -- 172.21.15.159:0/2220193325 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2786005155 0 0) 0x7fde80003520 con 0x7fde9c0b6310 2024-04-04T06:16:05.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.741+0000 7fde9be00640 1 -- 172.21.15.159:0/2220193325 >> v2:172.21.15.178:3300/0 conn(0x7fde9c0d3d50 msgr2=0x7fde9c0d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.741+0000 7fde9be00640 1 --2- 172.21.15.159:0/2220193325 >> v2:172.21.15.178:3300/0 conn(0x7fde9c0d3d50 0x7fde9c0d3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.741+0000 7fde9be00640 1 -- 172.21.15.159:0/2220193325 >> v1:172.21.15.178:6789/0 conn(0x7fde9c15e420 legacy=0x7fde9c0b9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.741+0000 7fde9be00640 1 -- 172.21.15.159:0/2220193325 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c0d2970 con 0x7fde9c0b6310 2024-04-04T06:16:05.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.741+0000 7fde9be00640 1 -- 172.21.15.159:0/2220193325 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fde940019f0 con 0x7fde9c0b6310 2024-04-04T06:16:05.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.741+0000 7fde9be00640 1 -- 172.21.15.159:0/2220193325 >> v1:172.21.15.159:6789/0 conn(0x7fde9c0b6310 legacy=0x7fde9c0b7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.741+0000 7fde9be00640 1 --2- 172.21.15.159:0/2220193325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde800039b0 0x7fde80003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.741+0000 7fde9be00640 1 --2- 172.21.15.159:0/2220193325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde80004800 0x7fde80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.741+0000 7fde9be00640 1 --2- 172.21.15.159:0/2220193325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde800051c0 0x7fde8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.741+0000 7fde9be00640 1 -- 172.21.15.159:0/2220193325 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c0d2fc0 con 0x7fde800039b0 2024-04-04T06:16:05.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.741+0000 7fde9be00640 1 -- 172.21.15.159:0/2220193325 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fde8000c090 con 0x7fde800051c0 2024-04-04T06:16:05.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.741+0000 7fde9be00640 1 -- 172.21.15.159:0/2220193325 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fde8000c290 con 0x7fde80004800 2024-04-04T06:16:05.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.741+0000 7fdea1200640 1 --2- 172.21.15.159:0/2220193325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde800039b0 0x7fde80003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.742+0000 7fdea1c00640 1 --2- 172.21.15.159:0/2220193325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde800051c0 0x7fde8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.742+0000 7fdea1200640 1 -- 172.21.15.159:0/2220193325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde80004800 msgr2=0x7fde80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.742+0000 7fde93e00640 1 --2- 172.21.15.159:0/2220193325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde80004800 0x7fde80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.742+0000 7fdea1200640 1 --2- 172.21.15.159:0/2220193325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde80004800 0x7fde80004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.742+0000 7fdea1200640 1 -- 172.21.15.159:0/2220193325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde800051c0 msgr2=0x7fde8000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.742+0000 7fdea1200640 1 --2- 172.21.15.159:0/2220193325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde800051c0 0x7fde8000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.742+0000 7fdea1200640 1 -- 172.21.15.159:0/2220193325 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde80003520 con 0x7fde800039b0 2024-04-04T06:16:05.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.742+0000 7fdea1c00640 1 --2- 172.21.15.159:0/2220193325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde800051c0 0x7fde8000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:05.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.742+0000 7fdea1c00640 1 -- 172.21.15.159:0/2220193325 reap_dead start 2024-04-04T06:16:05.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.743+0000 7fdea1200640 1 --2- 172.21.15.159:0/2220193325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde800039b0 0x7fde80003df0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fde8c001750 tx=0x7fde80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4818b2e07aade6f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:05.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.743+0000 7fde9be00640 1 -- 172.21.15.159:0/2220193325 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c001370 con 0x7fde800039b0 2024-04-04T06:16:05.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.743+0000 7fde9be00640 1 -- 172.21.15.159:0/2220193325 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde8c0052d0 con 0x7fde800039b0 2024-04-04T06:16:05.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.743+0000 7fde9be00640 1 -- 172.21.15.159:0/2220193325 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c003830 con 0x7fde800039b0 2024-04-04T06:16:05.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.744+0000 7fdea2600640 1 -- 172.21.15.159:0/2220193325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde800039b0 msgr2=0x7fde80003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.744+0000 7fdea2600640 1 --2- 172.21.15.159:0/2220193325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde800039b0 0x7fde80003df0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fde8c001750 tx=0x7fde80005830 comp rx=0 tx=0).stop 2024-04-04T06:16:05.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.744+0000 7fdea2600640 1 -- 172.21.15.159:0/2220193325 shutdown_connections 2024-04-04T06:16:05.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.744+0000 7fdea2600640 1 --2- 172.21.15.159:0/2220193325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde800039b0 0x7fde80003df0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.744+0000 7fdea2600640 1 -- 172.21.15.159:0/2220193325 >> 172.21.15.159:0/2220193325 conn(0x7fde9c0676a0 msgr2=0x7fde9c067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:05.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.744+0000 7fdea2600640 1 -- 172.21.15.159:0/2220193325 shutdown_connections 2024-04-04T06:16:05.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.744+0000 7fdea2600640 1 -- 172.21.15.159:0/2220193325 wait complete. 2024-04-04T06:16:05.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.746+0000 7fdea2600640 1 Processor -- start 2024-04-04T06:16:05.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.747+0000 7fdea2600640 1 -- start start 2024-04-04T06:16:05.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.748+0000 7fdea2600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde800039b0 0x7fde9c1522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.748+0000 7fdea2600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde9c1f4600 0x7fde9c0cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.748+0000 7fdea2600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde9c0cc350 0x7fde9c0b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.748+0000 7fdea2600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c0b6740 con 0x7fde9c0cc350 2024-04-04T06:16:05.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.748+0000 7fdea2600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c15e800 con 0x7fde9c1f4600 2024-04-04T06:16:05.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.748+0000 7fdea2600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fde9c0b68c0 con 0x7fde800039b0 2024-04-04T06:16:05.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.748+0000 7fdea1c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde9c0cc350 0x7fde9c0b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.749+0000 7fdea1c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde9c0cc350 0x7fde9c0b4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36084/0 (socket says 172.21.15.159:36084) 2024-04-04T06:16:05.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.749+0000 7fdea1c00640 1 -- 172.21.15.159:0/296900761 learned_addr learned my addr 172.21.15.159:0/296900761 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:05.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.749+0000 7fde93e00640 1 --2- 172.21.15.159:0/296900761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde9c1f4600 0x7fde9c0cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.749+0000 7fdea1200640 1 --2- 172.21.15.159:0/296900761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde800039b0 0x7fde9c1522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.749+0000 7fdea1c00640 1 -- 172.21.15.159:0/296900761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde800039b0 msgr2=0x7fde9c1522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.749+0000 7fdea1c00640 1 --2- 172.21.15.159:0/296900761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde800039b0 0x7fde9c1522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.749+0000 7fdea1c00640 1 -- 172.21.15.159:0/296900761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde9c1f4600 msgr2=0x7fde9c0cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.749+0000 7fdea1c00640 1 --2- 172.21.15.159:0/296900761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde9c1f4600 0x7fde9c0cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.749+0000 7fdea1c00640 1 -- 172.21.15.159:0/296900761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c152b80 con 0x7fde9c0cc350 2024-04-04T06:16:05.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.749+0000 7fdea1c00640 1 --2- 172.21.15.159:0/296900761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde9c0cc350 0x7fde9c0b4400 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fde9c06d5f0 tx=0x7fde9400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:05.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.750+0000 7fde99600640 1 -- 172.21.15.159:0/296900761 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94004d60 con 0x7fde9c0cc350 2024-04-04T06:16:05.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.750+0000 7fde99600640 1 -- 172.21.15.159:0/296900761 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde9400bb20 con 0x7fde9c0cc350 2024-04-04T06:16:05.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.750+0000 7fde99600640 1 -- 172.21.15.159:0/296900761 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94007110 con 0x7fde9c0cc350 2024-04-04T06:16:05.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.750+0000 7fdea2600640 1 -- 172.21.15.159:0/296900761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9c0c8fc0 con 0x7fde9c0cc350 2024-04-04T06:16:05.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.750+0000 7fdea2600640 1 -- 172.21.15.159:0/296900761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde9c0c9580 con 0x7fde9c0cc350 2024-04-04T06:16:05.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.754+0000 7fde99600640 1 -- 172.21.15.159:0/296900761 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde9401b020 con 0x7fde9c0cc350 2024-04-04T06:16:05.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.754+0000 7fdea6997740 1 Processor -- start 2024-04-04T06:16:05.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.754+0000 7fdea6997740 1 -- start start 2024-04-04T06:16:05.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.755+0000 7fde99600640 1 --2- 172.21.15.159:0/296900761 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde7806c920 0x7fde7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.755+0000 7fdea6997740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557e74c42180 0x557e74c42560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.755+0000 7fdea6997740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c47fa0 0x557e74c461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.755+0000 7fde99600640 1 -- 172.21.15.159:0/296900761 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7fde9408bd80 con 0x7fde9c0cc350 2024-04-04T06:16:05.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.755+0000 7fdea6997740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e74c8f890 0x557e74c8fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.755+0000 7fdea6997740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557e74bac280 con 0x557e74c47fa0 2024-04-04T06:16:05.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.755+0000 7fdea6997740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557e74b546f0 con 0x557e74c8f890 2024-04-04T06:16:05.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.755+0000 7fdea6997740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x557e74ba8230 con 0x557e74c42180 2024-04-04T06:16:05.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.756+0000 7fde93e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c47fa0 0x557e74c461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.756+0000 7fde93e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c47fa0 0x557e74c461d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36086/0 (socket says 172.21.15.159:36086) 2024-04-04T06:16:05.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.756+0000 7fde93e00640 1 -- 172.21.15.159:0/2864180362 learned_addr learned my addr 172.21.15.159:0/2864180362 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:05.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.756+0000 7fde93e00640 1 --2- 172.21.15.159:0/296900761 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde7806c920 0x7fde7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.756+0000 7fdea1200640 1 --2- 172.21.15.159:0/2864180362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557e74c42180 0x557e74c42560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.756+0000 7fdea1c00640 1 --2- 172.21.15.159:0/2864180362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e74c8f890 0x557e74c8fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.756+0000 7fde93e00640 1 -- 172.21.15.159:0/2864180362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557e74c42180 msgr2=0x557e74c42560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.756+0000 7fde93e00640 1 --2- 172.21.15.159:0/2864180362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557e74c42180 0x557e74c42560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.756+0000 7fde93e00640 1 -- 172.21.15.159:0/2864180362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e74c8f890 msgr2=0x557e74c8fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.756+0000 7fde93e00640 1 --2- 172.21.15.159:0/2864180362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e74c8f890 0x557e74c8fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.756+0000 7fde93e00640 1 -- 172.21.15.159:0/2864180362 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e749887e0 con 0x557e74c47fa0 2024-04-04T06:16:05.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.757+0000 7fde93e00640 1 --2- 172.21.15.159:0/2864180362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c47fa0 0x557e74c461d0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fde84019480 tx=0x7fde84009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ddceb194e5480f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:05.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.757+0000 7fde91600640 1 -- 172.21.15.159:0/2864180362 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8400f040 con 0x557e74c47fa0 2024-04-04T06:16:05.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.757+0000 7fde91600640 1 -- 172.21.15.159:0/2864180362 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde84016c00 con 0x557e74c47fa0 2024-04-04T06:16:05.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.757+0000 7fde91600640 1 -- 172.21.15.159:0/2864180362 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde840044c0 con 0x557e74c47fa0 2024-04-04T06:16:05.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.758+0000 7fdea6997740 1 -- 172.21.15.159:0/2864180362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c47fa0 msgr2=0x557e74c461d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.758+0000 7fdea6997740 1 --2- 172.21.15.159:0/2864180362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c47fa0 0x557e74c461d0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fde84019480 tx=0x7fde84009f90 comp rx=0 tx=0).stop 2024-04-04T06:16:05.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.758+0000 7fde93e00640 1 --2- 172.21.15.159:0/296900761 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde7806c920 0x7fde7806ede0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fde84023000 tx=0x7fde8401f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:05.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.758+0000 7fdea6997740 1 -- 172.21.15.159:0/2864180362 shutdown_connections 2024-04-04T06:16:05.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.758+0000 7fdea6997740 1 --2- 172.21.15.159:0/2864180362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e74c8f890 0x557e74c8fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.758+0000 7fdea6997740 1 --2- 172.21.15.159:0/2864180362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c47fa0 0x557e74c461d0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.758+0000 7fdea6997740 1 --2- 172.21.15.159:0/2864180362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557e74c42180 0x557e74c42560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.758+0000 7fdea6997740 1 -- 172.21.15.159:0/2864180362 >> 172.21.15.159:0/2864180362 conn(0x557e74c44200 msgr2=0x557e74c45ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:05.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.758+0000 7fdea6997740 1 -- 172.21.15.159:0/2864180362 shutdown_connections 2024-04-04T06:16:05.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.758+0000 7fdea6997740 1 -- 172.21.15.159:0/2864180362 wait complete. 2024-04-04T06:16:05.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.759+0000 7fdea6997740 1 -- start start 2024-04-04T06:16:05.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.760+0000 7fdea6997740 1 Processor -- start 2024-04-04T06:16:05.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.761+0000 7fdea6997740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c47fa0 0x557e74c75f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.761+0000 7fdea6997740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e74c8f890 0x557e74c73250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.761+0000 7fdea6997740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557e74c88be0 0x557e74c73790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.761+0000 7fdea6997740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557e74c6e8f0 con 0x557e74c47fa0 2024-04-04T06:16:05.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.761+0000 7fdea6997740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557e74b67aa0 con 0x557e74c8f890 2024-04-04T06:16:05.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.761+0000 7fdea6997740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x557e74c6eb40 con 0x557e74c88be0 2024-04-04T06:16:05.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.761+0000 7fdea6997740 10 client.0 _ll_register_callbacks cb 0x557e74b67490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:16:05.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.762+0000 7fdea1200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c47fa0 0x557e74c75f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.762+0000 7fdea1200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c47fa0 0x557e74c75f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36092/0 (socket says 172.21.15.159:36092) 2024-04-04T06:16:05.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.762+0000 7fdea1200640 1 -- 172.21.15.159:0/2119310688 learned_addr learned my addr 172.21.15.159:0/2119310688 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:05.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.762+0000 7fdea1c00640 1 --2- 172.21.15.159:0/2119310688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557e74c88be0 0x557e74c73790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.757 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.762+0000 7fde93e00640 1 --2- 172.21.15.159:0/2119310688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e74c8f890 0x557e74c73250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.762+0000 7fdea1200640 1 -- 172.21.15.159:0/2119310688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557e74c88be0 msgr2=0x557e74c73790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.762+0000 7fdea1200640 1 --2- 172.21.15.159:0/2119310688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557e74c88be0 0x557e74c73790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.762+0000 7fdea1200640 1 -- 172.21.15.159:0/2119310688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e74c8f890 msgr2=0x557e74c73250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.762+0000 7fdea1200640 1 --2- 172.21.15.159:0/2119310688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e74c8f890 0x557e74c73250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.762+0000 7fdea1200640 1 -- 172.21.15.159:0/2119310688 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e749e8f70 con 0x557e74c47fa0 2024-04-04T06:16:05.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.762+0000 7fdea1c00640 1 --2- 172.21.15.159:0/2119310688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557e74c88be0 0x557e74c73790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:16:05.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.763+0000 7fdea1200640 1 --2- 172.21.15.159:0/2119310688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c47fa0 0x557e74c75f60 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fde8c001870 tx=0x7fde8c001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b8e575a6df21f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:05.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.763+0000 7fde8a000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:16:05.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.763+0000 7fde8a000640 1 -- 172.21.15.159:0/2119310688 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c007250 con 0x557e74c47fa0 2024-04-04T06:16:05.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.763+0000 7fde8a000640 1 -- 172.21.15.159:0/2119310688 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7fde8c0073f0 con 0x557e74c47fa0 2024-04-04T06:16:05.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.763+0000 7fdea6997740 1 -- 172.21.15.159:0/2119310688 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde84016c00 con 0x557e74c47fa0 2024-04-04T06:16:05.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.763+0000 7fde8a000640 1 -- 172.21.15.159:0/2119310688 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fde8c0075f0 con 0x557e74c47fa0 2024-04-04T06:16:05.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.763+0000 7fdea6997740 10 client.8622 fetch_fsmap learned FSMap version 90 2024-04-04T06:16:05.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.763+0000 7fdea6997740 1 -- 172.21.15.159:0/2119310688 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x557e74880170 con 0x557e74c47fa0 2024-04-04T06:16:05.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.764+0000 7fde8a000640 1 -- 172.21.15.159:0/2119310688 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde84016c00 con 0x557e74c47fa0 2024-04-04T06:16:05.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.764+0000 7fde8a000640 1 -- 172.21.15.159:0/2119310688 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 90) ==== 3494+0+0 (secure 0 0 0) 0x7fde8c0067d0 con 0x557e74c47fa0 2024-04-04T06:16:05.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.764+0000 7fdea6997740 10 client.8622 fetch_fsmap finished waiting for FSMap version 90 2024-04-04T06:16:05.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.764+0000 7fdea6997740 10 client.8622 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-04T06:16:05.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.764+0000 7fdea6997740 20 client.8622 populate_metadata read hostname 'smithi159' 2024-04-04T06:16:05.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.764+0000 7fdea6997740 1 --2- 172.21.15.159:0/2119310688 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x557e74c74dc0 0x557e74ca0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.764+0000 7fdea6997740 4 client.8622 mds_command: new command op to 8535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:16:05.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.764+0000 7fdea6997740 1 -- 172.21.15.159:0/2119310688 --> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e74bb97c0 con 0x557e74c74dc0 2024-04-04T06:16:05.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.764+0000 7fde93e00640 1 --2- 172.21.15.159:0/2119310688 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x557e74c74dc0 0x557e74ca0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.765+0000 7fde8a000640 10 client.8622 ms_handle_connect on v2:172.21.15.159:6834/1003751501 2024-04-04T06:16:05.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.765+0000 7fde93e00640 1 --2- 172.21.15.159:0/2119310688 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x557e74c74dc0 0x557e74ca0e60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23b4b9ed4d62ad6e server_cookie=2af973a6124a0f5d in_seq=0 out_seq=0 2024-04-04T06:16:05.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.766+0000 7fde8a000640 1 -- 172.21.15.159:0/2119310688 <== mds.0 v2:172.21.15.159:6834/1003751501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e74bb97c0 con 0x557e74c74dc0 2024-04-04T06:16:05.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.766+0000 7fde8a000640 10 client.8622 handle_command_reply: tid=0 2024-04-04T06:16:05.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.766+0000 7fdea6997740 1 client.8622 shutdown 2024-04-04T06:16:05.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.766+0000 7fdea6997740 1 -- 172.21.15.159:0/2119310688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c47fa0 msgr2=0x557e74c75f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.766+0000 7fdea6997740 1 --2- 172.21.15.159:0/2119310688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c47fa0 0x557e74c75f60 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fde8c001870 tx=0x7fde8c001190 comp rx=0 tx=0).stop 2024-04-04T06:16:05.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.766+0000 7fdea6997740 1 -- 172.21.15.159:0/2119310688 shutdown_connections 2024-04-04T06:16:05.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.766+0000 7fdea6997740 1 --2- 172.21.15.159:0/2119310688 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x557e74c74dc0 0x557e74ca0e60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.766+0000 7fdea6997740 1 --2- 172.21.15.159:0/2119310688 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557e74c88be0 0x557e74c73790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.766+0000 7fdea6997740 1 --2- 172.21.15.159:0/2119310688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e74c8f890 0x557e74c73250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.766+0000 7fdea6997740 1 --2- 172.21.15.159:0/2119310688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c47fa0 0x557e74c75f60 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.766+0000 7fdea6997740 1 -- 172.21.15.159:0/2119310688 >> 172.21.15.159:0/2119310688 conn(0x557e74c42180 msgr2=0x557e74c41720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:05.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.766+0000 7fde8a000640 0 client.8622 ms_handle_reset on v2:172.21.15.159:6834/1003751501 2024-04-04T06:16:05.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.767+0000 7fdea6997740 1 -- 172.21.15.159:0/2119310688 shutdown_connections 2024-04-04T06:16:05.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.767+0000 7fdea6997740 1 -- 172.21.15.159:0/2119310688 wait complete. 2024-04-04T06:16:05.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.767+0000 7fdea6997740 20 client.8622 trim_cache size 0 max 16384 2024-04-04T06:16:05.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.771+0000 7f8496e00640 1 Processor -- start 2024-04-04T06:16:05.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.771+0000 7f8496e00640 1 -- start start 2024-04-04T06:16:05.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.772+0000 7f8496e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8490145ce0 0x7f84900acff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.772+0000 7f8496e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f84900ad530 0x7f84900aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.773+0000 7f8496e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f849006e4e0 con 0x7f84900af130 2024-04-04T06:16:05.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.773+0000 7f8496e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f849006e7c0 con 0x7f84900ad530 2024-04-04T06:16:05.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.773+0000 7f8496e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f849006eaa0 con 0x7f8490145ce0 2024-04-04T06:16:05.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.773+0000 7f8487400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f84900ad530 0x7f84900aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.773+0000 7f8487400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f84900ad530 0x7f84900aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36104/0 (socket says 172.21.15.159:36104) 2024-04-04T06:16:05.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.773+0000 7f8487400640 1 -- 172.21.15.159:0/564975740 learned_addr learned my addr 172.21.15.159:0/564975740 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:05.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.773+0000 7f8487400640 1 -- 172.21.15.159:0/564975740 >> v2:172.21.15.178:3300/0 conn(0x7f8490145ce0 msgr2=0x7f84900acff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.773+0000 7f848fe00640 1 --2- 172.21.15.159:0/564975740 >> v2:172.21.15.178:3300/0 conn(0x7f8490145ce0 0x7f84900acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.773+0000 7f8487400640 1 --2- 172.21.15.159:0/564975740 >> v2:172.21.15.178:3300/0 conn(0x7f8490145ce0 0x7f84900acff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.773+0000 7f8487400640 1 -- 172.21.15.159:0/564975740 >> v1:172.21.15.178:6789/0 conn(0x7f84900af130 legacy=0x7f849015fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.773+0000 7f8487400640 1 -- 172.21.15.159:0/564975740 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849006ed80 con 0x7f84900ad530 2024-04-04T06:16:05.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.774+0000 7f8487400640 1 --2- 172.21.15.159:0/564975740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84900ad530 0x7f84900aebf0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f849006f430 tx=0x7f847802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62365efc365415d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:05.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.775+0000 7f848f400640 1 -- 172.21.15.159:0/564975740 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478035020 con 0x7f84900ad530 2024-04-04T06:16:05.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.775+0000 7f848f400640 1 -- 172.21.15.159:0/564975740 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8478031870 con 0x7f84900ad530 2024-04-04T06:16:05.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.775+0000 7f848f400640 1 -- 172.21.15.159:0/564975740 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478031c40 con 0x7f84900ad530 2024-04-04T06:16:05.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.775+0000 7f8496e00640 1 -- 172.21.15.159:0/564975740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84900ad530 msgr2=0x7f84900aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.775+0000 7f8496e00640 1 --2- 172.21.15.159:0/564975740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84900ad530 0x7f84900aebf0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f849006f430 tx=0x7f847802b910 comp rx=0 tx=0).stop 2024-04-04T06:16:05.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.775+0000 7f8496e00640 1 -- 172.21.15.159:0/564975740 shutdown_connections 2024-04-04T06:16:05.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.775+0000 7f8496e00640 1 --2- 172.21.15.159:0/564975740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84900ad530 0x7f84900aebf0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.775+0000 7f8496e00640 1 --2- 172.21.15.159:0/564975740 >> v2:172.21.15.178:3300/0 conn(0x7f8490145ce0 0x7f84900acff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.775+0000 7f8496e00640 1 -- 172.21.15.159:0/564975740 >> 172.21.15.159:0/564975740 conn(0x7f849006b570 msgr2=0x7f849006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:05.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.775+0000 7f8496e00640 1 -- 172.21.15.159:0/564975740 shutdown_connections 2024-04-04T06:16:05.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.775+0000 7f8496e00640 1 -- 172.21.15.159:0/564975740 wait complete. 2024-04-04T06:16:05.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.777+0000 7f8496e00640 1 Processor -- start 2024-04-04T06:16:05.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.777+0000 7f8496e00640 1 -- start start 2024-04-04T06:16:05.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.778+0000 7f8496e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84900ad530 0x7f849009cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.778+0000 7f8496e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84900af130 0x7f84900b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.778+0000 7f8496e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8490145ce0 0x7f849015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.778+0000 7f8496e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84900af580 con 0x7f84900ad530 2024-04-04T06:16:05.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.778+0000 7f8496e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f849006f060 con 0x7f8490145ce0 2024-04-04T06:16:05.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.778+0000 7f8496e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f84901460c0 con 0x7f84900af130 2024-04-04T06:16:05.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.778+0000 7f848fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84900ad530 0x7f849009cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.779+0000 7f848fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84900ad530 0x7f849009cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36118/0 (socket says 172.21.15.159:36118) 2024-04-04T06:16:05.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.779+0000 7f848fe00640 1 -- 172.21.15.159:0/3170980393 learned_addr learned my addr 172.21.15.159:0/3170980393 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:05.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.779+0000 7f8487400640 1 --2- 172.21.15.159:0/3170980393 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84900af130 0x7f84900b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.779+0000 7f848fe00640 1 -- 172.21.15.159:0/3170980393 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84900af130 msgr2=0x7f84900b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.779+0000 7f848fe00640 1 --2- 172.21.15.159:0/3170980393 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84900af130 0x7f84900b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.779+0000 7f8494e00640 1 --2- 172.21.15.159:0/3170980393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8490145ce0 0x7f849015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.779+0000 7f848fe00640 1 -- 172.21.15.159:0/3170980393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8490145ce0 msgr2=0x7f849015b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.779+0000 7f848fe00640 1 --2- 172.21.15.159:0/3170980393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8490145ce0 0x7f849015b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.779+0000 7f848fe00640 1 -- 172.21.15.159:0/3170980393 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849015d250 con 0x7f84900ad530 2024-04-04T06:16:05.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.779+0000 7f8487400640 1 --2- 172.21.15.159:0/3170980393 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84900af130 0x7f84900b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:05.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.779+0000 7f8494e00640 1 --2- 172.21.15.159:0/3170980393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8490145ce0 0x7f849015b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:05.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.779+0000 7f848fe00640 1 --2- 172.21.15.159:0/3170980393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84900ad530 0x7f849009cb30 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f848000ccd0 tx=0x7f848000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:05.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.780+0000 7f848cc00640 1 -- 172.21.15.159:0/3170980393 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480003990 con 0x7f84900ad530 2024-04-04T06:16:05.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.780+0000 7f848cc00640 1 -- 172.21.15.159:0/3170980393 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8480003b30 con 0x7f84900ad530 2024-04-04T06:16:05.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.780+0000 7f848cc00640 1 -- 172.21.15.159:0/3170980393 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480004020 con 0x7f84900ad530 2024-04-04T06:16:05.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.780+0000 7f8496e00640 1 -- 172.21.15.159:0/3170980393 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f849015d530 con 0x7f84900ad530 2024-04-04T06:16:05.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.780+0000 7f8496e00640 1 -- 172.21.15.159:0/3170980393 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f849015daf0 con 0x7f84900ad530 2024-04-04T06:16:05.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.780+0000 7f8496e00640 1 -- 172.21.15.159:0/3170980393 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f84900685c0 con 0x7f84900ad530 2024-04-04T06:16:05.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.783+0000 7f848cc00640 1 -- 172.21.15.159:0/3170980393 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84800041c0 con 0x7f84900ad530 2024-04-04T06:16:05.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.784+0000 7f848cc00640 1 --2- 172.21.15.159:0/3170980393 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f846c06c8f0 0x7f846c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.784+0000 7f848cc00640 1 -- 172.21.15.159:0/3170980393 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f848008c660 con 0x7f84900ad530 2024-04-04T06:16:05.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.784+0000 7f848cc00640 1 -- 172.21.15.159:0/3170980393 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+516 (secure 0 0 0) 0x7f8480014030 con 0x7f84900ad530 2024-04-04T06:16:05.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.784+0000 7f8496e00640 1 -- 172.21.15.159:0/3170980393 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f849013f750 con 0x7f84900ad530 2024-04-04T06:16:05.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.785+0000 7f8487400640 1 --2- 172.21.15.159:0/3170980393 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f846c06c8f0 0x7f846c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.786+0000 7f8487400640 1 --2- 172.21.15.159:0/3170980393 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f846c06c8f0 0x7f846c06edb0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8478036000 tx=0x7f8478008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:05.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.802+0000 7fdea6997740 1 Processor -- start 2024-04-04T06:16:05.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.802+0000 7fdea6997740 1 -- start start 2024-04-04T06:16:05.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.803+0000 7fdea6997740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e74c47fa0 0x557e74c715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.803+0000 7fdea6997740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557e74c88be0 0x557e74c45990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.803+0000 7fdea6997740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c8f890 0x557e74c73050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.803+0000 7fdea6997740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c2010d0 con 0x557e74c8f890 2024-04-04T06:16:05.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.803+0000 7fdea6997740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c200140 con 0x557e74c47fa0 2024-04-04T06:16:05.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.803+0000 7fdea6997740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fde9c0cab60 con 0x557e74c88be0 2024-04-04T06:16:05.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.804+0000 7fdea1c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c8f890 0x557e74c73050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.804+0000 7fdea1c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c8f890 0x557e74c73050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36126/0 (socket says 172.21.15.159:36126) 2024-04-04T06:16:05.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.804+0000 7fdea1c00640 1 -- 172.21.15.159:0/1014331969 learned_addr learned my addr 172.21.15.159:0/1014331969 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:05.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.804+0000 7fde93e00640 1 --2- 172.21.15.159:0/1014331969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557e74c88be0 0x557e74c45990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.804+0000 7fdea1c00640 1 -- 172.21.15.159:0/1014331969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557e74c88be0 msgr2=0x557e74c45990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.804+0000 7fdea1c00640 1 --2- 172.21.15.159:0/1014331969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557e74c88be0 0x557e74c45990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.804+0000 7fdea1c00640 1 -- 172.21.15.159:0/1014331969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e74c47fa0 msgr2=0x557e74c715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.804+0000 7fdea1200640 1 --2- 172.21.15.159:0/1014331969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e74c47fa0 0x557e74c715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.804+0000 7fdea1c00640 1 --2- 172.21.15.159:0/1014331969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e74c47fa0 0x557e74c715d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.804+0000 7fdea1c00640 1 -- 172.21.15.159:0/1014331969 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e74c428d0 con 0x557e74c8f890 2024-04-04T06:16:05.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.804+0000 7fdea1c00640 1 --2- 172.21.15.159:0/1014331969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c8f890 0x557e74c73050 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fde94059a50 tx=0x7fde94016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4851061863508 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:05.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.805+0000 7fde90c00640 1 -- 172.21.15.159:0/1014331969 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde940019d0 con 0x557e74c8f890 2024-04-04T06:16:05.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.805+0000 7fde90c00640 1 -- 172.21.15.159:0/1014331969 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde94057b60 con 0x557e74c8f890 2024-04-04T06:16:05.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.805+0000 7fde90c00640 1 -- 172.21.15.159:0/1014331969 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9401bdc0 con 0x557e74c8f890 2024-04-04T06:16:05.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.805+0000 7fdea6997740 1 -- 172.21.15.159:0/1014331969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c8f890 msgr2=0x557e74c73050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.805+0000 7fdea6997740 1 --2- 172.21.15.159:0/1014331969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c8f890 0x557e74c73050 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fde94059a50 tx=0x7fde94016040 comp rx=0 tx=0).stop 2024-04-04T06:16:05.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.805+0000 7fdea6997740 1 -- 172.21.15.159:0/1014331969 shutdown_connections 2024-04-04T06:16:05.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.805+0000 7fdea6997740 1 --2- 172.21.15.159:0/1014331969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c8f890 0x557e74c73050 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.805+0000 7fdea6997740 1 --2- 172.21.15.159:0/1014331969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557e74c88be0 0x557e74c45990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.805+0000 7fdea6997740 1 --2- 172.21.15.159:0/1014331969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e74c47fa0 0x557e74c715d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.805+0000 7fdea6997740 1 -- 172.21.15.159:0/1014331969 >> 172.21.15.159:0/1014331969 conn(0x557e74c42180 msgr2=0x557e74c6f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:05.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.806+0000 7fdea6997740 1 -- 172.21.15.159:0/1014331969 shutdown_connections 2024-04-04T06:16:05.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.806+0000 7fdea6997740 1 -- 172.21.15.159:0/1014331969 wait complete. 2024-04-04T06:16:05.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.806+0000 7fdea6997740 1 -- start start 2024-04-04T06:16:05.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.806+0000 7fdea6997740 1 Processor -- start 2024-04-04T06:16:05.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.807+0000 7fdea6997740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557e74c47fa0 0x557e74cb02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.807+0000 7fdea6997740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c88be0 0x557e74ced470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.807+0000 7fdea6997740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e74c8f890 0x557e74ca6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.807+0000 7fdea6997740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557e74c9ccf0 con 0x557e74c88be0 2024-04-04T06:16:05.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.807+0000 7fdea6997740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557e74c725b0 con 0x557e74c8f890 2024-04-04T06:16:05.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.807+0000 7fdea6997740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x557e74cb08a0 con 0x557e74c47fa0 2024-04-04T06:16:05.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.807+0000 7fdea6997740 10 client.0 _ll_register_callbacks cb 0x7fde800036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:16:05.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.808+0000 7fde93e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c88be0 0x557e74ced470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.808+0000 7fde93e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c88be0 0x557e74ced470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36138/0 (socket says 172.21.15.159:36138) 2024-04-04T06:16:05.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.808+0000 7fde93e00640 1 -- 172.21.15.159:0/2109154271 learned_addr learned my addr 172.21.15.159:0/2109154271 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:05.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.808+0000 7fdea1c00640 1 --2- 172.21.15.159:0/2109154271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e74c8f890 0x557e74ca6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.808+0000 7fdea1200640 1 --2- 172.21.15.159:0/2109154271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557e74c47fa0 0x557e74cb02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.808+0000 7fde93e00640 1 -- 172.21.15.159:0/2109154271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557e74c47fa0 msgr2=0x557e74cb02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.808+0000 7fde93e00640 1 --2- 172.21.15.159:0/2109154271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557e74c47fa0 0x557e74cb02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.808+0000 7fde93e00640 1 -- 172.21.15.159:0/2109154271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e74c8f890 msgr2=0x557e74ca6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:05.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.808+0000 7fde93e00640 1 --2- 172.21.15.159:0/2109154271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e74c8f890 0x557e74ca6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.808+0000 7fde93e00640 1 -- 172.21.15.159:0/2109154271 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e74c70b70 con 0x557e74c88be0 2024-04-04T06:16:05.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.808+0000 7fde93e00640 1 --2- 172.21.15.159:0/2109154271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c88be0 0x557e74ced470 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fde84009230 tx=0x7fde84008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fd292b0740984a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:05.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.808+0000 7fde8a000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:16:05.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.809+0000 7fde8a000640 1 -- 172.21.15.159:0/2109154271 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84019250 con 0x557e74c88be0 2024-04-04T06:16:05.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.809+0000 7fdea6997740 1 -- 172.21.15.159:0/2109154271 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde94057b60 con 0x557e74c88be0 2024-04-04T06:16:05.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.809+0000 7fde8a000640 1 -- 172.21.15.159:0/2109154271 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7fde84016a00 con 0x557e74c88be0 2024-04-04T06:16:05.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.809+0000 7fde8a000640 1 -- 172.21.15.159:0/2109154271 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fde84049050 con 0x557e74c88be0 2024-04-04T06:16:05.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.809+0000 7fdea6997740 10 client.8634 fetch_fsmap learned FSMap version 90 2024-04-04T06:16:05.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.809+0000 7fdea6997740 1 -- 172.21.15.159:0/2109154271 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x557e74ca72e0 con 0x557e74c88be0 2024-04-04T06:16:05.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.809+0000 7fde8a000640 1 -- 172.21.15.159:0/2109154271 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde94057b60 con 0x557e74c88be0 2024-04-04T06:16:05.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.809+0000 7fde8a000640 1 -- 172.21.15.159:0/2109154271 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 90) ==== 3494+0+0 (secure 0 0 0) 0x7fde8404c020 con 0x557e74c88be0 2024-04-04T06:16:05.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.809+0000 7fdea6997740 10 client.8634 fetch_fsmap finished waiting for FSMap version 90 2024-04-04T06:16:05.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.809+0000 7fdea6997740 10 client.8634 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-04T06:16:05.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.809+0000 7fdea6997740 20 client.8634 populate_metadata read hostname 'smithi159' 2024-04-04T06:16:05.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.809+0000 7fdea6997740 1 --2- 172.21.15.159:0/2109154271 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x557e74c45e30 0x557e74c9bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:05.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.809+0000 7fdea6997740 4 client.8634 mds_command: new command op to 8535 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-04T06:16:05.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.809+0000 7fdea6997740 1 -- 172.21.15.159:0/2109154271 --> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x557e74bb3b10 con 0x557e74c45e30 2024-04-04T06:16:05.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.810+0000 7fdea1200640 1 --2- 172.21.15.159:0/2109154271 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x557e74c45e30 0x557e74c9bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:05.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.810+0000 7fdea1200640 1 --2- 172.21.15.159:0/2109154271 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x557e74c45e30 0x557e74c9bf80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb47215ee8a705ef server_cookie=b400fdf28a3f456d in_seq=0 out_seq=0 2024-04-04T06:16:05.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.810+0000 7fde8a000640 10 client.8634 ms_handle_connect on v2:172.21.15.159:6834/1003751501 2024-04-04T06:16:05.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.811+0000 7fde8a000640 1 -- 172.21.15.159:0/2109154271 <== mds.0 v2:172.21.15.159:6834/1003751501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557e74bb3b10 con 0x557e74c45e30 2024-04-04T06:16:05.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.811+0000 7fde8a000640 10 client.8634 handle_command_reply: tid=0 2024-04-04T06:16:05.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.811+0000 7fdea6997740 1 client.8634 shutdown 2024-04-04T06:16:05.810 INFO:tasks.ceph.mds.a.smithi159.stderr:2024-04-04T06:16:05.811+0000 7f1e9e200640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-04T06:16:05.810 INFO:tasks.ceph.mds.a.smithi159.stderr:2024-04-04T06:16:05.811+0000 7f1e9e200640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-04T06:16:05.810 INFO:tasks.ceph.mds.a.smithi159.stderr:2024-04-04T06:16:05.811+0000 7f1e9e200640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-04T06:16:05.810 INFO:tasks.ceph.mds.a.smithi159.stderr:2024-04-04T06:16:05.811+0000 7f1e9e200640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-04T06:16:05.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.811+0000 7fdea6997740 1 -- 172.21.15.159:0/2109154271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c88be0 msgr2=0x557e74ced470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.811+0000 7fdea6997740 1 --2- 172.21.15.159:0/2109154271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c88be0 0x557e74ced470 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fde84009230 tx=0x7fde84008040 comp rx=0 tx=0).stop 2024-04-04T06:16:05.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.812+0000 7fdea6997740 1 -- 172.21.15.159:0/2109154271 shutdown_connections 2024-04-04T06:16:05.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.812+0000 7fdea6997740 1 --2- 172.21.15.159:0/2109154271 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x557e74c45e30 0x557e74c9bf80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.812+0000 7fdea6997740 1 --2- 172.21.15.159:0/2109154271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557e74c8f890 0x557e74ca6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.812+0000 7fdea6997740 1 --2- 172.21.15.159:0/2109154271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e74c88be0 0x557e74ced470 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.812+0000 7fdea6997740 1 --2- 172.21.15.159:0/2109154271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x557e74c47fa0 0x557e74cb02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.812+0000 7fdea6997740 1 -- 172.21.15.159:0/2109154271 >> 172.21.15.159:0/2109154271 conn(0x557e74c42180 msgr2=0x557e74c73050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:05.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.812+0000 7fde8a000640 0 client.8634 ms_handle_reset on v2:172.21.15.159:6834/1003751501 2024-04-04T06:16:05.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.812+0000 7fdea6997740 1 -- 172.21.15.159:0/2109154271 shutdown_connections 2024-04-04T06:16:05.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.812+0000 7fdea6997740 1 -- 172.21.15.159:0/2109154271 wait complete. 2024-04-04T06:16:05.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.812+0000 7fdea6997740 20 client.8634 trim_cache size 0 max 16384 2024-04-04T06:16:05.813 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:16:05.813 INFO:teuthology.orchestra.run.smithi159.stdout: "return_code": 0, 2024-04-04T06:16:05.813 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_tag": "19125a53-96df-4c06-9e73-876052fc799f", 2024-04-04T06:16:05.813 INFO:teuthology.orchestra.run.smithi159.stdout: "mode": "asynchronous" 2024-04-04T06:16:05.813 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:16:05.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.814+0000 7fdea2600640 1 -- 172.21.15.159:0/296900761 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde7806c920 msgr2=0x7fde7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.814+0000 7fdea2600640 1 --2- 172.21.15.159:0/296900761 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde7806c920 0x7fde7806ede0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fde84023000 tx=0x7fde8401f000 comp rx=0 tx=0).stop 2024-04-04T06:16:05.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.814+0000 7fdea2600640 1 -- 172.21.15.159:0/296900761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde9c0cc350 msgr2=0x7fde9c0b4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:05.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.814+0000 7fdea2600640 1 --2- 172.21.15.159:0/296900761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde9c0cc350 0x7fde9c0b4400 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fde9c06d5f0 tx=0x7fde9400b4a0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.814+0000 7fdea2600640 1 -- 172.21.15.159:0/296900761 shutdown_connections 2024-04-04T06:16:05.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.814+0000 7fdea2600640 1 --2- 172.21.15.159:0/296900761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde9c0cc350 0x7fde9c0b4400 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.814+0000 7fdea2600640 1 --2- 172.21.15.159:0/296900761 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fde7806c920 0x7fde7806ede0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.814+0000 7fdea2600640 1 --2- 172.21.15.159:0/296900761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde9c1f4600 0x7fde9c0cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.814+0000 7fdea2600640 1 --2- 172.21.15.159:0/296900761 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde800039b0 0x7fde9c1522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:05.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.814+0000 7fdea2600640 1 -- 172.21.15.159:0/296900761 >> 172.21.15.159:0/296900761 conn(0x7fde9c0676a0 msgr2=0x7fde9c0b7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:05.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.814+0000 7fdea2600640 1 -- 172.21.15.159:0/296900761 shutdown_connections 2024-04-04T06:16:05.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:05.814+0000 7fdea2600640 1 -- 172.21.15.159:0/296900761 wait complete. 2024-04-04T06:16:05.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:16:05.826 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-04T06:16:06.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.026+0000 7f0462400640 1 Processor -- start 2024-04-04T06:16:06.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.027+0000 7f0462400640 1 -- start start 2024-04-04T06:16:06.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.028+0000 7f0462400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f045c0cc880 0x7f045c0cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:06.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.028+0000 7f0462400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f045c06d620 0x7f045c06da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:06.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.028+0000 7f0462400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f045c0ce890 con 0x7f045c13f190 2024-04-04T06:16:06.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.028+0000 7f0462400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f045c0cd230 con 0x7f045c0cc880 2024-04-04T06:16:06.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.028+0000 7f0462400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f045c0ceb50 con 0x7f045c06d620 2024-04-04T06:16:06.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.029+0000 7f045be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f045c0cc880 0x7f045c0cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:06.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.029+0000 7f045be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f045c0cc880 0x7f045c0cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36146/0 (socket says 172.21.15.159:36146) 2024-04-04T06:16:06.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.029+0000 7f045be00640 1 -- 172.21.15.159:0/3844890274 learned_addr learned my addr 172.21.15.159:0/3844890274 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:06.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.029+0000 7f045be00640 1 -- 172.21.15.159:0/3844890274 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f045c06d620 msgr2=0x7f045c06da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:06.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.029+0000 7f045be00640 1 --2- 172.21.15.159:0/3844890274 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f045c06d620 0x7f045c06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.029+0000 7f045be00640 1 -- 172.21.15.159:0/3844890274 >> v1:172.21.15.178:6789/0 conn(0x7f045c13f190 legacy=0x7f045c0cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:06.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.029+0000 7f045be00640 1 -- 172.21.15.159:0/3844890274 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045c0cee30 con 0x7f045c0cc880 2024-04-04T06:16:06.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.030+0000 7f045be00640 1 --2- 172.21.15.159:0/3844890274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f045c0cc880 0x7f045c0cccf0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f0448009e50 tx=0x7f044802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0219ddbfd97b33 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:06.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.031+0000 7f045b400640 1 -- 172.21.15.159:0/3844890274 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044802cb00 con 0x7f045c0cc880 2024-04-04T06:16:06.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.031+0000 7f045b400640 1 -- 172.21.15.159:0/3844890274 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f044802cca0 con 0x7f045c0cc880 2024-04-04T06:16:06.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.031+0000 7f045b400640 1 -- 172.21.15.159:0/3844890274 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448034610 con 0x7f045c0cc880 2024-04-04T06:16:06.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.031+0000 7f0462400640 1 -- 172.21.15.159:0/3844890274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f045c0cc880 msgr2=0x7f045c0cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:06.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.031+0000 7f0462400640 1 --2- 172.21.15.159:0/3844890274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f045c0cc880 0x7f045c0cccf0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f0448009e50 tx=0x7f044802bf60 comp rx=0 tx=0).stop 2024-04-04T06:16:06.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.032+0000 7f0462400640 1 -- 172.21.15.159:0/3844890274 shutdown_connections 2024-04-04T06:16:06.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.032+0000 7f0462400640 1 --2- 172.21.15.159:0/3844890274 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f045c06d620 0x7f045c06da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.032+0000 7f0462400640 1 --2- 172.21.15.159:0/3844890274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f045c0cc880 0x7f045c0cccf0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.032+0000 7f0462400640 1 -- 172.21.15.159:0/3844890274 >> 172.21.15.159:0/3844890274 conn(0x7f045c0b1ba0 msgr2=0x7f045c0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:06.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.032+0000 7f0462400640 1 -- 172.21.15.159:0/3844890274 shutdown_connections 2024-04-04T06:16:06.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.032+0000 7f0462400640 1 -- 172.21.15.159:0/3844890274 wait complete. 2024-04-04T06:16:06.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.033+0000 7f0462400640 1 Processor -- start 2024-04-04T06:16:06.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.033+0000 7f0462400640 1 -- start start 2024-04-04T06:16:06.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.034+0000 7f0462400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f045c06d620 0x7f045c155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:06.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.034+0000 7f0462400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f045c0cc880 0x7f045c137970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:06.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.034+0000 7f0462400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f045c13f190 0x7f045c1e92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:06.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.034+0000 7f0462400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f045c13f570 con 0x7f045c06d620 2024-04-04T06:16:06.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.034+0000 7f0462400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f045c160c70 con 0x7f045c13f190 2024-04-04T06:16:06.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.034+0000 7f0462400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f045c06df90 con 0x7f045c0cc880 2024-04-04T06:16:06.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.035+0000 7f0461000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f045c06d620 0x7f045c155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:06.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.035+0000 7f0461000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f045c06d620 0x7f045c155cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36158/0 (socket says 172.21.15.159:36158) 2024-04-04T06:16:06.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.035+0000 7f0461000640 1 -- 172.21.15.159:0/575307223 learned_addr learned my addr 172.21.15.159:0/575307223 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:06.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.035+0000 7f0461000640 1 -- 172.21.15.159:0/575307223 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f045c0cc880 msgr2=0x7f045c137970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:06.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.035+0000 7f0461000640 1 --2- 172.21.15.159:0/575307223 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f045c0cc880 0x7f045c137970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.035+0000 7f0461000640 1 -- 172.21.15.159:0/575307223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f045c13f190 msgr2=0x7f045c1e92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:06.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.035+0000 7f0461000640 1 --2- 172.21.15.159:0/575307223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f045c13f190 0x7f045c1e92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.035+0000 7f0461000640 1 -- 172.21.15.159:0/575307223 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045c1e6af0 con 0x7f045c06d620 2024-04-04T06:16:06.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.035+0000 7f0461000640 1 --2- 172.21.15.159:0/575307223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f045c06d620 0x7f045c155cc0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f044c00c500 tx=0x7f044c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:06.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.036+0000 7f0458c00640 1 -- 172.21.15.159:0/575307223 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c0052d0 con 0x7f045c06d620 2024-04-04T06:16:06.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.036+0000 7f0458c00640 1 -- 172.21.15.159:0/575307223 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f044c005470 con 0x7f045c06d620 2024-04-04T06:16:06.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.036+0000 7f0462400640 1 -- 172.21.15.159:0/575307223 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f045c0b43e0 con 0x7f045c06d620 2024-04-04T06:16:06.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.036+0000 7f0458c00640 1 -- 172.21.15.159:0/575307223 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c012850 con 0x7f045c06d620 2024-04-04T06:16:06.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.036+0000 7f0462400640 1 -- 172.21.15.159:0/575307223 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f045c1e7070 con 0x7f045c06d620 2024-04-04T06:16:06.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.038+0000 7f0458c00640 1 -- 172.21.15.159:0/575307223 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f044c0129f0 con 0x7f045c06d620 2024-04-04T06:16:06.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.039+0000 7f0458c00640 1 --2- 172.21.15.159:0/575307223 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f043c06c8f0 0x7f043c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:06.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.039+0000 7f0458c00640 1 -- 172.21.15.159:0/575307223 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f044c08b830 con 0x7f045c06d620 2024-04-04T06:16:06.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.039+0000 7f0466813740 1 Processor -- start 2024-04-04T06:16:06.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.040+0000 7f0466813740 1 -- start start 2024-04-04T06:16:06.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.040+0000 7f045be00640 1 --2- 172.21.15.159:0/575307223 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f043c06c8f0 0x7f043c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:06.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.040+0000 7f0466813740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eef6428740 0x55eef63fb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:06.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.040+0000 7f0466813740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eef6428b20 0x55eef642b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:06.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.040+0000 7f0466813740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef63fbc20 0x55eef642ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:06.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.040+0000 7f0466813740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55eef63666d0 con 0x55eef63fbc20 2024-04-04T06:16:06.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.040+0000 7f0466813740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eef630e770 con 0x55eef6428740 2024-04-04T06:16:06.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.040+0000 7f0466813740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55eef6362680 con 0x55eef6428b20 2024-04-04T06:16:06.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.041+0000 7f0461a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef63fbc20 0x55eef642ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:06.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.041+0000 7f0461a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef63fbc20 0x55eef642ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36170/0 (socket says 172.21.15.159:36170) 2024-04-04T06:16:06.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.041+0000 7f0461a00640 1 -- 172.21.15.159:0/3170773364 learned_addr learned my addr 172.21.15.159:0/3170773364 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:06.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.041+0000 7f045be00640 1 --2- 172.21.15.159:0/575307223 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f043c06c8f0 0x7f043c06edb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0448035000 tx=0x7f044802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:06.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.041+0000 7f0461000640 1 --2- 172.21.15.159:0/3170773364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eef6428740 0x55eef63fb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:06.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.041+0000 7f045be00640 1 --2- 172.21.15.159:0/3170773364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eef6428b20 0x55eef642b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:06.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.041+0000 7f0461a00640 1 -- 172.21.15.159:0/3170773364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eef6428b20 msgr2=0x55eef642b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:06.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.041+0000 7f0461a00640 1 --2- 172.21.15.159:0/3170773364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eef6428b20 0x55eef642b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.041+0000 7f0461a00640 1 -- 172.21.15.159:0/3170773364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eef6428740 msgr2=0x55eef63fb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:06.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.041+0000 7f0461a00640 1 --2- 172.21.15.159:0/3170773364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eef6428740 0x55eef63fb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.041+0000 7f0461a00640 1 -- 172.21.15.159:0/3170773364 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eef61447d0 con 0x55eef63fbc20 2024-04-04T06:16:06.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.042+0000 7f0461a00640 1 --2- 172.21.15.159:0/3170773364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef63fbc20 0x55eef642ba10 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f0454009700 tx=0x7f0454009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb61987bbb4bae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:06.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.042+0000 7f0451600640 1 -- 172.21.15.159:0/3170773364 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454015020 con 0x55eef63fbc20 2024-04-04T06:16:06.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.042+0000 7f0451600640 1 -- 172.21.15.159:0/3170773364 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f045400b860 con 0x55eef63fbc20 2024-04-04T06:16:06.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.042+0000 7f0451600640 1 -- 172.21.15.159:0/3170773364 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045400bc30 con 0x55eef63fbc20 2024-04-04T06:16:06.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.042+0000 7f0466813740 1 -- 172.21.15.159:0/3170773364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef63fbc20 msgr2=0x55eef642ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:06.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.042+0000 7f0466813740 1 --2- 172.21.15.159:0/3170773364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef63fbc20 0x55eef642ba10 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f0454009700 tx=0x7f0454009bc0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.043+0000 7f0466813740 1 -- 172.21.15.159:0/3170773364 shutdown_connections 2024-04-04T06:16:06.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.043+0000 7f0466813740 1 --2- 172.21.15.159:0/3170773364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef63fbc20 0x55eef642ba10 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.043+0000 7f0466813740 1 --2- 172.21.15.159:0/3170773364 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eef6428b20 0x55eef642b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.043+0000 7f0466813740 1 --2- 172.21.15.159:0/3170773364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eef6428740 0x55eef63fb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.043+0000 7f0466813740 1 -- 172.21.15.159:0/3170773364 >> 172.21.15.159:0/3170773364 conn(0x55eef63fd950 msgr2=0x55eef64464d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:06.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.043+0000 7f0466813740 1 -- 172.21.15.159:0/3170773364 shutdown_connections 2024-04-04T06:16:06.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.043+0000 7f0466813740 1 -- 172.21.15.159:0/3170773364 wait complete. 2024-04-04T06:16:06.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.043+0000 7f0466813740 1 -- start start 2024-04-04T06:16:06.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.044+0000 7f0466813740 1 Processor -- start 2024-04-04T06:16:06.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.045+0000 7f0466813740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef6428740 0x55eef642ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:06.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.045+0000 7f0466813740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eef6428b20 0x55eef642dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:06.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.045+0000 7f0466813740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eef64299d0 0x55eef64265c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:06.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.045+0000 7f0466813740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55eef63ffff0 con 0x55eef6428740 2024-04-04T06:16:06.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.045+0000 7f0466813740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eef6321aa0 con 0x55eef6428b20 2024-04-04T06:16:06.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.045+0000 7f0466813740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55eef6402830 con 0x55eef64299d0 2024-04-04T06:16:06.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.045+0000 7f0466813740 10 client.0 _ll_register_callbacks cb 0x55eef6321490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:16:06.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.045+0000 7f0461000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef6428740 0x55eef642ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:06.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.045+0000 7f0461000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef6428740 0x55eef642ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36180/0 (socket says 172.21.15.159:36180) 2024-04-04T06:16:06.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.045+0000 7f0461000640 1 -- 172.21.15.159:0/3820410864 learned_addr learned my addr 172.21.15.159:0/3820410864 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:06.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.045+0000 7f045be00640 1 --2- 172.21.15.159:0/3820410864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eef6428b20 0x55eef642dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:06.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.045+0000 7f0461a00640 1 --2- 172.21.15.159:0/3820410864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eef64299d0 0x55eef64265c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:06.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.045+0000 7f0461000640 1 -- 172.21.15.159:0/3820410864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eef64299d0 msgr2=0x55eef64265c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:06.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.045+0000 7f0461000640 1 --2- 172.21.15.159:0/3820410864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eef64299d0 0x55eef64265c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.045+0000 7f0461000640 1 -- 172.21.15.159:0/3820410864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eef6428b20 msgr2=0x55eef642dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:06.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.045+0000 7f0461000640 1 --2- 172.21.15.159:0/3820410864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eef6428b20 0x55eef642dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.045+0000 7f0461000640 1 -- 172.21.15.159:0/3820410864 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eef607a3f0 con 0x55eef6428740 2024-04-04T06:16:06.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.046+0000 7f0461000640 1 --2- 172.21.15.159:0/3820410864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef6428740 0x55eef642ae80 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f044c000f60 tx=0x7f044c05d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcdafefe67cd6419 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:06.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.046+0000 7f0446000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:16:06.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.046+0000 7f0446000640 1 -- 172.21.15.159:0/3820410864 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c055ae0 con 0x55eef6428740 2024-04-04T06:16:06.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.046+0000 7f0446000640 1 -- 172.21.15.159:0/3820410864 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f044c061bc0 con 0x55eef6428740 2024-04-04T06:16:06.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.046+0000 7f0466813740 1 -- 172.21.15.159:0/3820410864 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f045400b860 con 0x55eef6428740 2024-04-04T06:16:06.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.046+0000 7f0446000640 1 -- 172.21.15.159:0/3820410864 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f044c055c80 con 0x55eef6428740 2024-04-04T06:16:06.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.046+0000 7f0466813740 10 client.8646 fetch_fsmap learned FSMap version 90 2024-04-04T06:16:06.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.046+0000 7f0466813740 1 -- 172.21.15.159:0/3820410864 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55eef603a0f0 con 0x55eef6428740 2024-04-04T06:16:06.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.046+0000 7f0446000640 1 -- 172.21.15.159:0/3820410864 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f045400b860 con 0x55eef6428740 2024-04-04T06:16:06.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.046+0000 7f0446000640 1 -- 172.21.15.159:0/3820410864 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 90) ==== 3494+0+0 (secure 0 0 0) 0x7f044c0771a0 con 0x55eef6428740 2024-04-04T06:16:06.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.046+0000 7f0466813740 10 client.8646 fetch_fsmap finished waiting for FSMap version 90 2024-04-04T06:16:06.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.046+0000 7f0466813740 10 client.8646 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-04T06:16:06.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.047+0000 7f0466813740 20 client.8646 populate_metadata read hostname 'smithi159' 2024-04-04T06:16:06.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.047+0000 7f0466813740 1 --2- 172.21.15.159:0/3820410864 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55eef641cc90 0x55eef642c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:06.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.047+0000 7f0466813740 4 client.8646 mds_command: new command op to 8535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:16:06.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.047+0000 7f0466813740 1 -- 172.21.15.159:0/3820410864 --> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eef6373c10 con 0x55eef641cc90 2024-04-04T06:16:06.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.047+0000 7f045be00640 1 --2- 172.21.15.159:0/3820410864 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55eef641cc90 0x55eef642c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:06.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.047+0000 7f045be00640 1 --2- 172.21.15.159:0/3820410864 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55eef641cc90 0x55eef642c670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df52e834facdb3e2 server_cookie=54211e0743f1e190 in_seq=0 out_seq=0 2024-04-04T06:16:06.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.047+0000 7f0446000640 10 client.8646 ms_handle_connect on v2:172.21.15.159:6834/1003751501 2024-04-04T06:16:06.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.048+0000 7f0446000640 1 -- 172.21.15.159:0/3820410864 <== mds.0 v2:172.21.15.159:6834/1003751501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eef6373c10 con 0x55eef641cc90 2024-04-04T06:16:06.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.048+0000 7f0446000640 10 client.8646 handle_command_reply: tid=0 2024-04-04T06:16:06.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.048+0000 7f0466813740 1 client.8646 shutdown 2024-04-04T06:16:06.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.048+0000 7f0466813740 1 -- 172.21.15.159:0/3820410864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef6428740 msgr2=0x55eef642ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:06.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.048+0000 7f0466813740 1 --2- 172.21.15.159:0/3820410864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef6428740 0x55eef642ae80 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f044c000f60 tx=0x7f044c05d820 comp rx=0 tx=0).stop 2024-04-04T06:16:06.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.048+0000 7f0466813740 1 -- 172.21.15.159:0/3820410864 shutdown_connections 2024-04-04T06:16:06.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.048+0000 7f0466813740 1 --2- 172.21.15.159:0/3820410864 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55eef641cc90 0x55eef642c670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.048+0000 7f0466813740 1 --2- 172.21.15.159:0/3820410864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eef64299d0 0x55eef64265c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.048+0000 7f0466813740 1 --2- 172.21.15.159:0/3820410864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eef6428b20 0x55eef642dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.048+0000 7f0466813740 1 --2- 172.21.15.159:0/3820410864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef6428740 0x55eef642ae80 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.048+0000 7f0466813740 1 -- 172.21.15.159:0/3820410864 >> 172.21.15.159:0/3820410864 conn(0x55eef63fbc20 msgr2=0x55eef63ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:06.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.048+0000 7f0446000640 0 client.8646 ms_handle_reset on v2:172.21.15.159:6834/1003751501 2024-04-04T06:16:06.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.049+0000 7f0466813740 1 -- 172.21.15.159:0/3820410864 shutdown_connections 2024-04-04T06:16:06.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.049+0000 7f0466813740 1 -- 172.21.15.159:0/3820410864 wait complete. 2024-04-04T06:16:06.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.049+0000 7f0466813740 20 client.8646 trim_cache size 0 max 16384 2024-04-04T06:16:06.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.067+0000 7f0466813740 1 Processor -- start 2024-04-04T06:16:06.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.067+0000 7f0466813740 1 -- start start 2024-04-04T06:16:06.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.068+0000 7f0466813740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eef6428740 0x55eef64c3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:06.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.068+0000 7f0466813740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eef6428b20 0x55eef6449a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:06.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.068+0000 7f0466813740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef64299d0 0x55eef6449f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:06.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.068+0000 7f0466813740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f045c200be0 con 0x55eef64299d0 2024-04-04T06:16:06.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.068+0000 7f0466813740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1ffda0 con 0x55eef6428740 2024-04-04T06:16:06.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.068+0000 7f0466813740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f045c1e7e20 con 0x55eef6428b20 2024-04-04T06:16:06.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.069+0000 7f0461a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef64299d0 0x55eef6449f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:06.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.069+0000 7f0461a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef64299d0 0x55eef6449f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36194/0 (socket says 172.21.15.159:36194) 2024-04-04T06:16:06.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.069+0000 7f0461a00640 1 -- 172.21.15.159:0/3217932793 learned_addr learned my addr 172.21.15.159:0/3217932793 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:06.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.069+0000 7f0461000640 1 --2- 172.21.15.159:0/3217932793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eef6428740 0x55eef64c3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:06.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.069+0000 7f045be00640 1 --2- 172.21.15.159:0/3217932793 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eef6428b20 0x55eef6449a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:06.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.069+0000 7f0461a00640 1 -- 172.21.15.159:0/3217932793 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eef6428b20 msgr2=0x55eef6449a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:06.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.069+0000 7f0461a00640 1 --2- 172.21.15.159:0/3217932793 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eef6428b20 0x55eef6449a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.069+0000 7f0461a00640 1 -- 172.21.15.159:0/3217932793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eef6428740 msgr2=0x55eef64c3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:06.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.069+0000 7f0461a00640 1 --2- 172.21.15.159:0/3217932793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eef6428740 0x55eef64c3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.069+0000 7f0461a00640 1 -- 172.21.15.159:0/3217932793 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eef6426cf0 con 0x55eef64299d0 2024-04-04T06:16:06.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.069+0000 7f045be00640 1 --2- 172.21.15.159:0/3217932793 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eef6428b20 0x55eef6449a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:06.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.070+0000 7f0461a00640 1 --2- 172.21.15.159:0/3217932793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef64299d0 0x55eef6449f80 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f04540090b0 tx=0x7f0454010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e283b2e55ea09fad server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:06.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.070+0000 7f0450c00640 1 -- 172.21.15.159:0/3217932793 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454015030 con 0x55eef64299d0 2024-04-04T06:16:06.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.070+0000 7f0450c00640 1 -- 172.21.15.159:0/3217932793 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f045400b940 con 0x55eef64299d0 2024-04-04T06:16:06.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.070+0000 7f0450c00640 1 -- 172.21.15.159:0/3217932793 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045400bd30 con 0x55eef64299d0 2024-04-04T06:16:06.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.070+0000 7f0466813740 1 -- 172.21.15.159:0/3217932793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef64299d0 msgr2=0x55eef6449f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:06.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.070+0000 7f0466813740 1 --2- 172.21.15.159:0/3217932793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef64299d0 0x55eef6449f80 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f04540090b0 tx=0x7f0454010000 comp rx=0 tx=0).stop 2024-04-04T06:16:06.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.070+0000 7f0466813740 1 -- 172.21.15.159:0/3217932793 shutdown_connections 2024-04-04T06:16:06.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.070+0000 7f0466813740 1 --2- 172.21.15.159:0/3217932793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef64299d0 0x55eef6449f80 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.071+0000 7f0466813740 1 --2- 172.21.15.159:0/3217932793 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eef6428b20 0x55eef6449a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.071+0000 7f0466813740 1 --2- 172.21.15.159:0/3217932793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eef6428740 0x55eef64c3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.071+0000 7f0466813740 1 -- 172.21.15.159:0/3217932793 >> 172.21.15.159:0/3217932793 conn(0x55eef63fbc20 msgr2=0x55eef642d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:06.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.071+0000 7f0466813740 1 -- 172.21.15.159:0/3217932793 shutdown_connections 2024-04-04T06:16:06.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.071+0000 7f0466813740 1 -- 172.21.15.159:0/3217932793 wait complete. 2024-04-04T06:16:06.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.071+0000 7f0466813740 1 -- start start 2024-04-04T06:16:06.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.071+0000 7f0466813740 1 Processor -- start 2024-04-04T06:16:06.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.072+0000 7f0466813740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef6428740 0x55eef64c29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:06.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.072+0000 7f0466813740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eef6428b20 0x55eef64b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:06.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.072+0000 7f0466813740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eef64299d0 0x55eef6446900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:06.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.072+0000 7f0466813740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55eef63f6f30 con 0x55eef6428740 2024-04-04T06:16:06.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.072+0000 7f0466813740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1e8470 con 0x55eef64299d0 2024-04-04T06:16:06.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.072+0000 7f0466813740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55eef642b8b0 con 0x55eef6428b20 2024-04-04T06:16:06.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.073+0000 7f0466813740 10 client.0 _ll_register_callbacks cb 0x7f044c059c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:16:06.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.073+0000 7f0461000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef6428740 0x55eef64c29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:06.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.073+0000 7f0461000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef6428740 0x55eef64c29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:36204/0 (socket says 172.21.15.159:36204) 2024-04-04T06:16:06.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.073+0000 7f0461000640 1 -- 172.21.15.159:0/3890994681 learned_addr learned my addr 172.21.15.159:0/3890994681 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:06.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.073+0000 7f045be00640 1 --2- 172.21.15.159:0/3890994681 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eef6428b20 0x55eef64b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:06.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.073+0000 7f0461000640 1 -- 172.21.15.159:0/3890994681 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eef6428b20 msgr2=0x55eef64b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:06.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.073+0000 7f0461000640 1 --2- 172.21.15.159:0/3890994681 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eef6428b20 0x55eef64b23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.073+0000 7f0461000640 1 -- 172.21.15.159:0/3890994681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eef64299d0 msgr2=0x55eef6446900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:06.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.073+0000 7f0461000640 1 --2- 172.21.15.159:0/3890994681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eef64299d0 0x55eef6446900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.073+0000 7f0461000640 1 -- 172.21.15.159:0/3890994681 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eef646af00 con 0x55eef6428740 2024-04-04T06:16:06.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.073+0000 7f0461000640 1 --2- 172.21.15.159:0/3890994681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef6428740 0x55eef64c29b0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f044c000fb0 tx=0x7f044c05c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b0f140869d28cd server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:06.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.073+0000 7f0446000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:16:06.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.074+0000 7f0446000640 1 -- 172.21.15.159:0/3890994681 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c087070 con 0x55eef6428740 2024-04-04T06:16:06.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.074+0000 7f0446000640 1 -- 172.21.15.159:0/3890994681 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f044c0588c0 con 0x55eef6428740 2024-04-04T06:16:06.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.074+0000 7f0466813740 1 -- 172.21.15.159:0/3890994681 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f045400b940 con 0x55eef6428740 2024-04-04T06:16:06.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.074+0000 7f0446000640 1 -- 172.21.15.159:0/3890994681 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f044c055ae0 con 0x55eef6428740 2024-04-04T06:16:06.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.074+0000 7f0466813740 10 client.8652 fetch_fsmap learned FSMap version 90 2024-04-04T06:16:06.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.074+0000 7f0466813740 1 -- 172.21.15.159:0/3890994681 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55eef646a360 con 0x55eef6428740 2024-04-04T06:16:06.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.074+0000 7f0446000640 1 -- 172.21.15.159:0/3890994681 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f045400b940 con 0x55eef6428740 2024-04-04T06:16:06.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.074+0000 7f0446000640 1 -- 172.21.15.159:0/3890994681 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 90) ==== 3494+0+0 (secure 0 0 0) 0x7f044c0551c0 con 0x55eef6428740 2024-04-04T06:16:06.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.074+0000 7f0466813740 10 client.8652 fetch_fsmap finished waiting for FSMap version 90 2024-04-04T06:16:06.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.074+0000 7f0466813740 10 client.8652 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-04T06:16:06.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.074+0000 7f0466813740 20 client.8652 populate_metadata read hostname 'smithi159' 2024-04-04T06:16:06.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.074+0000 7f0466813740 1 --2- 172.21.15.159:0/3890994681 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55eef6460e30 0x55eef64699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:06.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.074+0000 7f0466813740 4 client.8652 mds_command: new command op to 8535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-04T06:16:06.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.074+0000 7f0466813740 1 -- 172.21.15.159:0/3890994681 --> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eef636df60 con 0x55eef6460e30 2024-04-04T06:16:06.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.075+0000 7f045be00640 1 --2- 172.21.15.159:0/3890994681 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55eef6460e30 0x55eef64699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:06.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.075+0000 7f045be00640 1 --2- 172.21.15.159:0/3890994681 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55eef6460e30 0x55eef64699a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab82756b55ceb0b1 server_cookie=d89ce78267d621e7 in_seq=0 out_seq=0 2024-04-04T06:16:06.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.075+0000 7f0446000640 10 client.8652 ms_handle_connect on v2:172.21.15.159:6834/1003751501 2024-04-04T06:16:06.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.075+0000 7f0446000640 1 -- 172.21.15.159:0/3890994681 <== mds.0 v2:172.21.15.159:6834/1003751501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55eef636df60 con 0x55eef6460e30 2024-04-04T06:16:06.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.075+0000 7f0446000640 10 client.8652 handle_command_reply: tid=0 2024-04-04T06:16:06.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.075+0000 7f0466813740 1 client.8652 shutdown 2024-04-04T06:16:06.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.075+0000 7f0466813740 1 -- 172.21.15.159:0/3890994681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef6428740 msgr2=0x55eef64c29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:06.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.075+0000 7f0466813740 1 --2- 172.21.15.159:0/3890994681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef6428740 0x55eef64c29b0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f044c000fb0 tx=0x7f044c05c070 comp rx=0 tx=0).stop 2024-04-04T06:16:06.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.076+0000 7f0466813740 1 -- 172.21.15.159:0/3890994681 shutdown_connections 2024-04-04T06:16:06.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.076+0000 7f0466813740 1 --2- 172.21.15.159:0/3890994681 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55eef6460e30 0x55eef64699a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.076+0000 7f0466813740 1 --2- 172.21.15.159:0/3890994681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eef64299d0 0x55eef6446900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.076+0000 7f0466813740 1 --2- 172.21.15.159:0/3890994681 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eef6428b20 0x55eef64b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.076+0000 7f0466813740 1 --2- 172.21.15.159:0/3890994681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eef6428740 0x55eef64c29b0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.076+0000 7f0466813740 1 -- 172.21.15.159:0/3890994681 >> 172.21.15.159:0/3890994681 conn(0x55eef63fbc20 msgr2=0x55eef642d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:06.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.076+0000 7f0446000640 0 client.8652 ms_handle_reset on v2:172.21.15.159:6834/1003751501 2024-04-04T06:16:06.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.076+0000 7f0466813740 1 -- 172.21.15.159:0/3890994681 shutdown_connections 2024-04-04T06:16:06.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.076+0000 7f0466813740 1 -- 172.21.15.159:0/3890994681 wait complete. 2024-04-04T06:16:06.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.076+0000 7f0466813740 20 client.8652 trim_cache size 0 max 16384 2024-04-04T06:16:06.074 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:16:06.075 INFO:teuthology.orchestra.run.smithi159.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-04T06:16:06.075 INFO:teuthology.orchestra.run.smithi159.stdout: "scrubs": { 2024-04-04T06:16:06.075 INFO:teuthology.orchestra.run.smithi159.stdout: "19125a53-96df-4c06-9e73-876052fc799f": { 2024-04-04T06:16:06.075 INFO:teuthology.orchestra.run.smithi159.stdout: "path": "/", 2024-04-04T06:16:06.075 INFO:teuthology.orchestra.run.smithi159.stdout: "tag": "19125a53-96df-4c06-9e73-876052fc799f", 2024-04-04T06:16:06.075 INFO:teuthology.orchestra.run.smithi159.stdout: "options": "recursive,repair" 2024-04-04T06:16:06.075 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:06.075 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:06.075 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:16:06.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.076+0000 7f0462400640 1 -- 172.21.15.159:0/575307223 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f043c06c8f0 msgr2=0x7f043c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:06.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.076+0000 7f0462400640 1 --2- 172.21.15.159:0/575307223 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f043c06c8f0 0x7f043c06edb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0448035000 tx=0x7f044802e000 comp rx=0 tx=0).stop 2024-04-04T06:16:06.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.076+0000 7f0462400640 1 -- 172.21.15.159:0/575307223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f045c06d620 msgr2=0x7f045c155cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:06.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.076+0000 7f0462400640 1 --2- 172.21.15.159:0/575307223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f045c06d620 0x7f045c155cc0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f044c00c500 tx=0x7f044c00c9c0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.077+0000 7f0462400640 1 -- 172.21.15.159:0/575307223 shutdown_connections 2024-04-04T06:16:06.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.077+0000 7f0462400640 1 --2- 172.21.15.159:0/575307223 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f043c06c8f0 0x7f043c06edb0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.077+0000 7f0462400640 1 --2- 172.21.15.159:0/575307223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f045c13f190 0x7f045c1e92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.077+0000 7f0462400640 1 --2- 172.21.15.159:0/575307223 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f045c0cc880 0x7f045c137970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.077+0000 7f0462400640 1 --2- 172.21.15.159:0/575307223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f045c06d620 0x7f045c155cc0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:06.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.077+0000 7f0462400640 1 -- 172.21.15.159:0/575307223 >> 172.21.15.159:0/575307223 conn(0x7f045c0b1ba0 msgr2=0x7f045c15eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:06.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.077+0000 7f0462400640 1 -- 172.21.15.159:0/575307223 shutdown_connections 2024-04-04T06:16:06.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.077+0000 7f0462400640 1 -- 172.21.15.159:0/575307223 wait complete. 2024-04-04T06:16:06.087 INFO:tasks.cephfs.filesystem:scrub status for tag:19125a53-96df-4c06-9e73-876052fc799f - {'path': '/', 'tag': '19125a53-96df-4c06-9e73-876052fc799f', 'options': 'recursive,repair'} 2024-04-04T06:16:06.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:06.537+0000 7f848cc00640 1 -- 172.21.15.159:0/3170980393 <== mon.0 v2:172.21.15.159:3300/0 7 ==== log(8 entries from seq 1 at 2024-04-04T06:16:05.812023+0000) ==== 1407+0+0 (secure 0 0 0) 0x7f84800583c0 con 0x7f84900ad530 2024-04-04T06:16:06.533 INFO:teuthology.orchestra.run.smithi159.stdout: cluster: 2024-04-04T06:16:06.533 INFO:teuthology.orchestra.run.smithi159.stdout: id: 30311045-f714-48fa-832b-4cb2ba8bf0bf 2024-04-04T06:16:06.533 INFO:teuthology.orchestra.run.smithi159.stdout: health: HEALTH_OK 2024-04-04T06:16:06.533 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:06.533 INFO:teuthology.orchestra.run.smithi159.stdout: services: 2024-04-04T06:16:06.533 INFO:teuthology.orchestra.run.smithi159.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-04-04T06:16:06.533 INFO:teuthology.orchestra.run.smithi159.stdout: mgr: x(active, since 10m), standbys: y 2024-04-04T06:16:06.533 INFO:teuthology.orchestra.run.smithi159.stdout: mds: 1/1 daemons up, 3 standby 2024-04-04T06:16:06.533 INFO:teuthology.orchestra.run.smithi159.stdout: osd: 8 osds: 8 up (since 10m), 8 in (since 10m) 2024-04-04T06:16:06.533 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:06.533 INFO:teuthology.orchestra.run.smithi159.stdout: data: 2024-04-04T06:16:06.533 INFO:teuthology.orchestra.run.smithi159.stdout: volumes: 1/1 healthy 2024-04-04T06:16:06.533 INFO:teuthology.orchestra.run.smithi159.stdout: pools: 3 pools, 97 pgs 2024-04-04T06:16:06.534 INFO:teuthology.orchestra.run.smithi159.stdout: objects: 30 objects, 19 MiB 2024-04-04T06:16:06.534 INFO:teuthology.orchestra.run.smithi159.stdout: usage: 307 MiB used, 720 GiB / 720 GiB avail 2024-04-04T06:16:06.534 INFO:teuthology.orchestra.run.smithi159.stdout: pgs: 97 active+clean 2024-04-04T06:16:06.534 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:06.534 INFO:teuthology.orchestra.run.smithi159.stdout: io: 2024-04-04T06:16:06.534 INFO:teuthology.orchestra.run.smithi159.stdout: client: 1.8 KiB/s rd, 102 B/s wr, 1 op/s rd, 0 op/s wr 2024-04-04T06:16:06.534 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:06.534 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:06.534 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:16:05.812023+0000 mds.a [INF] scrub queued for path: / 2024-04-04T06:16:06.534 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:16:05.812026+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-04T06:16:06.534 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:16:05.812030+0000 mds.a [INF] scrub summary: active paths [/] 2024-04-04T06:16:06.534 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:16:05.812877+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-04-04T06:16:06.534 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:16:05.812884+0000 mds.a [ERR] inode table repaired for inode: 0x10000000001 2024-04-04T06:16:06.534 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:16:05.812977+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-04-04T06:16:06.535 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:16:05.812983+0000 mds.a [ERR] inode table repaired for inode: 0x10000000002 2024-04-04T06:16:06.535 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:16:05.813980+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-04T06:16:10.544 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:16:09.492055+0000 mds.a [INF] scrub summary: idle 2024-04-04T06:16:10.545 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:10.549+0000 7f848cc00640 1 -- 172.21.15.159:0/3170980393 <== mon.0 v2:172.21.15.159:3300/0 8 ==== log(1 entries from seq 9 at 2024-04-04T06:16:09.492055+0000) ==== 189+0+0 (secure 0 0 0) 0x7f8480058620 con 0x7f84900ad530 2024-04-04T06:16:13.513 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:16:13.523 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:16:36.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:16:36.089 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-04T06:16:36.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.321+0000 7fbf14a00640 1 Processor -- start 2024-04-04T06:16:36.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.321+0000 7fbf14a00640 1 -- start start 2024-04-04T06:16:36.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.323+0000 7fbf14a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fbf100d37b0 0x7fbf100b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.323+0000 7fbf14a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf100d2710 con 0x7fbf100b67f0 2024-04-04T06:16:36.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.323+0000 7fbf14a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf100d29f0 con 0x7fbf1015e4c0 2024-04-04T06:16:36.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.323+0000 7fbf14a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fbf1015e8a0 con 0x7fbf100d37b0 2024-04-04T06:16:36.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.323+0000 7fbf07e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fbf100d37b0 0x7fbf100b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.323+0000 7fbf0fe00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fbf100b67f0 0x7fbf100b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37654/0 (socket says 172.21.15.159:37654) 2024-04-04T06:16:36.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.323+0000 7fbf0fe00640 1 -- 172.21.15.159:0/1335613888 learned_addr learned my addr 172.21.15.159:0/1335613888 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:16:36.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.323+0000 7fbf0ea00640 1 -- 172.21.15.159:0/1335613888 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3743786620 0 0) 0x7fbf100d2710 con 0x7fbf100b67f0 2024-04-04T06:16:36.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.324+0000 7fbf0ea00640 1 -- 172.21.15.159:0/1335613888 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbef4003520 con 0x7fbf100b67f0 2024-04-04T06:16:36.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.324+0000 7fbf07e00640 1 -- 172.21.15.159:0/1335613888 >> v1:172.21.15.178:6789/0 conn(0x7fbf1015e4c0 legacy=0x7fbf100b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:36.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.324+0000 7fbf07e00640 1 -- 172.21.15.159:0/1335613888 >> v1:172.21.15.159:6789/0 conn(0x7fbf100b67f0 legacy=0x7fbf100b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:36.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.324+0000 7fbf07e00640 1 -- 172.21.15.159:0/1335613888 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf100d2cd0 con 0x7fbf100d37b0 2024-04-04T06:16:36.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.325+0000 7fbf07e00640 1 --2- 172.21.15.159:0/1335613888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf100d37b0 0x7fbf100b62b0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fbef80097a0 tx=0x7fbef802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddffc68e0e8dc4e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:36.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.325+0000 7fbf0ea00640 1 -- 172.21.15.159:0/1335613888 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8034070 con 0x7fbf100d37b0 2024-04-04T06:16:36.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.325+0000 7fbf0ea00640 1 -- 172.21.15.159:0/1335613888 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef8030610 con 0x7fbf100d37b0 2024-04-04T06:16:36.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.325+0000 7fbf0ea00640 1 -- 172.21.15.159:0/1335613888 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef80309c0 con 0x7fbf100d37b0 2024-04-04T06:16:36.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.326+0000 7fbf14a00640 1 -- 172.21.15.159:0/1335613888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf100d37b0 msgr2=0x7fbf100b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:36.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.326+0000 7fbf14a00640 1 --2- 172.21.15.159:0/1335613888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf100d37b0 0x7fbf100b62b0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fbef80097a0 tx=0x7fbef802b960 comp rx=0 tx=0).stop 2024-04-04T06:16:36.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.326+0000 7fbf14a00640 1 -- 172.21.15.159:0/1335613888 shutdown_connections 2024-04-04T06:16:36.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.326+0000 7fbf14a00640 1 --2- 172.21.15.159:0/1335613888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf100d37b0 0x7fbf100b62b0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.326+0000 7fbf14a00640 1 -- 172.21.15.159:0/1335613888 >> 172.21.15.159:0/1335613888 conn(0x7fbf100676a0 msgr2=0x7fbf10067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:36.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.326+0000 7fbf14a00640 1 -- 172.21.15.159:0/1335613888 shutdown_connections 2024-04-04T06:16:36.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.326+0000 7fbf14a00640 1 -- 172.21.15.159:0/1335613888 wait complete. 2024-04-04T06:16:36.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.328+0000 7fbf14a00640 1 Processor -- start 2024-04-04T06:16:36.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.328+0000 7fbf14a00640 1 -- start start 2024-04-04T06:16:36.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.329+0000 7fbf14a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf100b67f0 0x7fbf100ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.329+0000 7fbf14a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf100d37b0 0x7fbf1014b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.329+0000 7fbf14a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf1015e4c0 0x7fbf10160390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.329+0000 7fbf14a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbf100d3320 con 0x7fbf100b67f0 2024-04-04T06:16:36.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.329+0000 7fbf14a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbf100d4040 con 0x7fbf100d37b0 2024-04-04T06:16:36.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.329+0000 7fbf14a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbf100b6c20 con 0x7fbf1015e4c0 2024-04-04T06:16:36.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.329+0000 7fbf0f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf100b67f0 0x7fbf100ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.329+0000 7fbf0f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf100b67f0 0x7fbf100ca040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51234/0 (socket says 172.21.15.159:51234) 2024-04-04T06:16:36.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.329+0000 7fbf0f400640 1 -- 172.21.15.159:0/1940434142 learned_addr learned my addr 172.21.15.159:0/1940434142 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:36.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.329+0000 7fbf07e00640 1 --2- 172.21.15.159:0/1940434142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf100d37b0 0x7fbf1014b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.330+0000 7fbf0f400640 1 -- 172.21.15.159:0/1940434142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf1015e4c0 msgr2=0x7fbf10160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:36.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.330+0000 7fbf0fe00640 1 --2- 172.21.15.159:0/1940434142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf1015e4c0 0x7fbf10160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.330+0000 7fbf0f400640 1 --2- 172.21.15.159:0/1940434142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf1015e4c0 0x7fbf10160390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.330+0000 7fbf0f400640 1 -- 172.21.15.159:0/1940434142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf100d37b0 msgr2=0x7fbf1014b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:36.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.330+0000 7fbf0f400640 1 --2- 172.21.15.159:0/1940434142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf100d37b0 0x7fbf1014b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.330+0000 7fbf0f400640 1 -- 172.21.15.159:0/1940434142 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf100cac30 con 0x7fbf100b67f0 2024-04-04T06:16:36.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.330+0000 7fbf07e00640 1 --2- 172.21.15.159:0/1940434142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf100d37b0 0x7fbf1014b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:36.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.330+0000 7fbf0f400640 1 --2- 172.21.15.159:0/1940434142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf100b67f0 0x7fbf100ca040 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000c400 tx=0x7fbf0000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:36.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.330+0000 7fbf07400640 1 -- 172.21.15.159:0/1940434142 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00017070 con 0x7fbf100b67f0 2024-04-04T06:16:36.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.330+0000 7fbf07400640 1 -- 172.21.15.159:0/1940434142 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf000052d0 con 0x7fbf100b67f0 2024-04-04T06:16:36.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.330+0000 7fbf07400640 1 -- 172.21.15.159:0/1940434142 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00003830 con 0x7fbf100b67f0 2024-04-04T06:16:36.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.330+0000 7fbf14a00640 1 -- 172.21.15.159:0/1940434142 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf100c8890 con 0x7fbf100b67f0 2024-04-04T06:16:36.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.330+0000 7fbf14a00640 1 -- 172.21.15.159:0/1940434142 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf100c8e50 con 0x7fbf100b67f0 2024-04-04T06:16:36.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.334+0000 7fbf07400640 1 -- 172.21.15.159:0/1940434142 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf0001a070 con 0x7fbf100b67f0 2024-04-04T06:16:36.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.334+0000 7fbf18c6d740 1 Processor -- start 2024-04-04T06:16:36.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.334+0000 7fbf07400640 1 --2- 172.21.15.159:0/1940434142 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbeec06c8f0 0x7fbeec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.334+0000 7fbf18c6d740 1 -- start start 2024-04-04T06:16:36.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.335+0000 7fbf07400640 1 -- 172.21.15.159:0/1940434142 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7fbf0008afb0 con 0x7fbf100b67f0 2024-04-04T06:16:36.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.335+0000 7fbf0fe00640 1 --2- 172.21.15.159:0/1940434142 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbeec06c8f0 0x7fbeec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.336+0000 7fbf18c6d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a67e740 0x55559a6516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.336+0000 7fbf18c6d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55559a67eb20 0x55559a6814d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.336+0000 7fbf18c6d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55559a651c20 0x55559a681a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.336+0000 7fbf18c6d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55559a5bc6d0 con 0x55559a67e740 2024-04-04T06:16:36.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.336+0000 7fbf18c6d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55559a564770 con 0x55559a651c20 2024-04-04T06:16:36.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.336+0000 7fbf18c6d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55559a5b8680 con 0x55559a67eb20 2024-04-04T06:16:36.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.336+0000 7fbf0f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a67e740 0x55559a6516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.336+0000 7fbf0f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a67e740 0x55559a6516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51250/0 (socket says 172.21.15.159:51250) 2024-04-04T06:16:36.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.336+0000 7fbf0f400640 1 -- 172.21.15.159:0/195935597 learned_addr learned my addr 172.21.15.159:0/195935597 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:36.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.336+0000 7fbf0fe00640 1 --2- 172.21.15.159:0/1940434142 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbeec06c8f0 0x7fbeec06edb0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbf08001560 tx=0x7fbf08009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:36.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.337+0000 7fbf0fe00640 1 --2- 172.21.15.159:0/195935597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55559a651c20 0x55559a681a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.337+0000 7fbf0f400640 1 -- 172.21.15.159:0/195935597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55559a67eb20 msgr2=0x55559a6814d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:36.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.337+0000 7fbf0f400640 1 --2- 172.21.15.159:0/195935597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55559a67eb20 0x55559a6814d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.337+0000 7fbf0f400640 1 -- 172.21.15.159:0/195935597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55559a651c20 msgr2=0x55559a681a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:36.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.337+0000 7fbf0f400640 1 --2- 172.21.15.159:0/195935597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55559a651c20 0x55559a681a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.337+0000 7fbf0f400640 1 -- 172.21.15.159:0/195935597 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55559a39a7d0 con 0x55559a67e740 2024-04-04T06:16:36.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.337+0000 7fbf0f400640 1 --2- 172.21.15.159:0/195935597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a67e740 0x55559a6516e0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fbf00061290 tx=0x7fbf00058ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e290249ac0695eb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:36.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.337+0000 7fbeff400640 1 -- 172.21.15.159:0/195935597 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00071170 con 0x55559a67e740 2024-04-04T06:16:36.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.337+0000 7fbeff400640 1 -- 172.21.15.159:0/195935597 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf00056ee0 con 0x55559a67e740 2024-04-04T06:16:36.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.337+0000 7fbeff400640 1 -- 172.21.15.159:0/195935597 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00058030 con 0x55559a67e740 2024-04-04T06:16:36.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.338+0000 7fbf18c6d740 1 -- 172.21.15.159:0/195935597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a67e740 msgr2=0x55559a6516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:36.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.338+0000 7fbf18c6d740 1 --2- 172.21.15.159:0/195935597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a67e740 0x55559a6516e0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fbf00061290 tx=0x7fbf00058ca0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.338+0000 7fbf18c6d740 1 -- 172.21.15.159:0/195935597 shutdown_connections 2024-04-04T06:16:36.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.338+0000 7fbf18c6d740 1 --2- 172.21.15.159:0/195935597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55559a651c20 0x55559a681a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.338+0000 7fbf18c6d740 1 --2- 172.21.15.159:0/195935597 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55559a67eb20 0x55559a6814d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.338+0000 7fbf18c6d740 1 --2- 172.21.15.159:0/195935597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a67e740 0x55559a6516e0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.338+0000 7fbf18c6d740 1 -- 172.21.15.159:0/195935597 >> 172.21.15.159:0/195935597 conn(0x55559a653950 msgr2=0x55559a69c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:36.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.338+0000 7fbf18c6d740 1 -- 172.21.15.159:0/195935597 shutdown_connections 2024-04-04T06:16:36.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.338+0000 7fbf18c6d740 1 -- 172.21.15.159:0/195935597 wait complete. 2024-04-04T06:16:36.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.339+0000 7fbf18c6d740 1 -- start start 2024-04-04T06:16:36.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.339+0000 7fbf18c6d740 1 Processor -- start 2024-04-04T06:16:36.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.340+0000 7fbf18c6d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55559a67e740 0x55559a6ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.340+0000 7fbf18c6d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55559a67eb20 0x55559a683660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.340+0000 7fbf18c6d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a683ba0 0x55559a688740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.340+0000 7fbf18c6d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55559a655ff0 con 0x55559a683ba0 2024-04-04T06:16:36.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.340+0000 7fbf18c6d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55559a577aa0 con 0x55559a67e740 2024-04-04T06:16:36.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.340+0000 7fbf18c6d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55559a658830 con 0x55559a67eb20 2024-04-04T06:16:36.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.340+0000 7fbf18c6d740 10 client.0 _ll_register_callbacks cb 0x55559a577490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:16:36.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.341+0000 7fbf0fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a683ba0 0x55559a688740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.341+0000 7fbf0fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a683ba0 0x55559a688740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51252/0 (socket says 172.21.15.159:51252) 2024-04-04T06:16:36.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.341+0000 7fbf0fe00640 1 -- 172.21.15.159:0/263355753 learned_addr learned my addr 172.21.15.159:0/263355753 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:36.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.341+0000 7fbf07e00640 1 --2- 172.21.15.159:0/263355753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55559a67eb20 0x55559a683660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.341+0000 7fbf0f400640 1 --2- 172.21.15.159:0/263355753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55559a67e740 0x55559a6ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.341+0000 7fbf0fe00640 1 -- 172.21.15.159:0/263355753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55559a67eb20 msgr2=0x55559a683660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:36.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.341+0000 7fbf0fe00640 1 --2- 172.21.15.159:0/263355753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55559a67eb20 0x55559a683660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.341+0000 7fbf0fe00640 1 -- 172.21.15.159:0/263355753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55559a67e740 msgr2=0x55559a6ac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:36.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.341+0000 7fbf0fe00640 1 --2- 172.21.15.159:0/263355753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55559a67e740 0x55559a6ac5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.341+0000 7fbf0fe00640 1 -- 172.21.15.159:0/263355753 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55559a2d03f0 con 0x55559a683ba0 2024-04-04T06:16:36.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.341+0000 7fbf0fe00640 1 --2- 172.21.15.159:0/263355753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a683ba0 0x55559a688740 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fbf08013bc0 tx=0x7fbf08001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3157991696a979ea server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:36.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.341+0000 7fbef3e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:16:36.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.342+0000 7fbef3e00640 1 -- 172.21.15.159:0/263355753 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08012e60 con 0x55559a683ba0 2024-04-04T06:16:36.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.342+0000 7fbef3e00640 1 -- 172.21.15.159:0/263355753 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7fbf08020ae0 con 0x55559a683ba0 2024-04-04T06:16:36.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.342+0000 7fbf18c6d740 1 -- 172.21.15.159:0/263355753 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf00056ee0 con 0x55559a683ba0 2024-04-04T06:16:36.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.342+0000 7fbef3e00640 1 -- 172.21.15.159:0/263355753 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fbf08020ce0 con 0x55559a683ba0 2024-04-04T06:16:36.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.342+0000 7fbf18c6d740 10 client.8667 fetch_fsmap learned FSMap version 90 2024-04-04T06:16:36.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.342+0000 7fbf18c6d740 1 -- 172.21.15.159:0/263355753 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55559a2900f0 con 0x55559a683ba0 2024-04-04T06:16:36.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.342+0000 7fbef3e00640 1 -- 172.21.15.159:0/263355753 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf00056ee0 con 0x55559a683ba0 2024-04-04T06:16:36.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.342+0000 7fbef3e00640 1 -- 172.21.15.159:0/263355753 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 90) ==== 3494+0+0 (secure 0 0 0) 0x7fbf08020420 con 0x55559a683ba0 2024-04-04T06:16:36.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.342+0000 7fbf18c6d740 10 client.8667 fetch_fsmap finished waiting for FSMap version 90 2024-04-04T06:16:36.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.342+0000 7fbf18c6d740 10 client.8667 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-04T06:16:36.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.342+0000 7fbf18c6d740 20 client.8667 populate_metadata read hostname 'smithi159' 2024-04-04T06:16:36.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.342+0000 7fbf18c6d740 1 --2- 172.21.15.159:0/263355753 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55559a6b06d0 0x55559a6b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.342+0000 7fbf18c6d740 4 client.8667 mds_command: new command op to 8535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:16:36.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.342+0000 7fbf18c6d740 1 -- 172.21.15.159:0/263355753 --> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55559a5c9c10 con 0x55559a6b06d0 2024-04-04T06:16:36.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.343+0000 7fbf0f400640 1 --2- 172.21.15.159:0/263355753 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55559a6b06d0 0x55559a6b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.343+0000 7fbf0f400640 1 --2- 172.21.15.159:0/263355753 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55559a6b06d0 0x55559a6b0b00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe5bc317fb5f83fb server_cookie=7e184e53e31d69de in_seq=0 out_seq=0 2024-04-04T06:16:36.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.343+0000 7fbef3e00640 10 client.8667 ms_handle_connect on v2:172.21.15.159:6834/1003751501 2024-04-04T06:16:36.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.344+0000 7fbef3e00640 1 -- 172.21.15.159:0/263355753 <== mds.0 v2:172.21.15.159:6834/1003751501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55559a5c9c10 con 0x55559a6b06d0 2024-04-04T06:16:36.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.344+0000 7fbef3e00640 10 client.8667 handle_command_reply: tid=0 2024-04-04T06:16:36.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.344+0000 7fbf18c6d740 1 client.8667 shutdown 2024-04-04T06:16:36.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.344+0000 7fbf18c6d740 1 -- 172.21.15.159:0/263355753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a683ba0 msgr2=0x55559a688740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:36.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.344+0000 7fbf18c6d740 1 --2- 172.21.15.159:0/263355753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a683ba0 0x55559a688740 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fbf08013bc0 tx=0x7fbf08001250 comp rx=0 tx=0).stop 2024-04-04T06:16:36.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.344+0000 7fbf18c6d740 1 -- 172.21.15.159:0/263355753 shutdown_connections 2024-04-04T06:16:36.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.344+0000 7fbf18c6d740 1 --2- 172.21.15.159:0/263355753 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55559a6b06d0 0x55559a6b0b00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.344+0000 7fbf18c6d740 1 --2- 172.21.15.159:0/263355753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a683ba0 0x55559a688740 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.344+0000 7fbf18c6d740 1 --2- 172.21.15.159:0/263355753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55559a67eb20 0x55559a683660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.344+0000 7fbf18c6d740 1 --2- 172.21.15.159:0/263355753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55559a67e740 0x55559a6ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.344+0000 7fbef3e00640 0 client.8667 ms_handle_reset on v2:172.21.15.159:6834/1003751501 2024-04-04T06:16:36.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.344+0000 7fbf18c6d740 1 -- 172.21.15.159:0/263355753 >> 172.21.15.159:0/263355753 conn(0x55559a651c20 msgr2=0x55559a6556c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:36.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.344+0000 7fbf18c6d740 1 -- 172.21.15.159:0/263355753 shutdown_connections 2024-04-04T06:16:36.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.344+0000 7fbf18c6d740 1 -- 172.21.15.159:0/263355753 wait complete. 2024-04-04T06:16:36.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.345+0000 7fbf18c6d740 20 client.8667 trim_cache size 0 max 16384 2024-04-04T06:16:36.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.363+0000 7fbf18c6d740 1 Processor -- start 2024-04-04T06:16:36.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.364+0000 7fbf18c6d740 1 -- start start 2024-04-04T06:16:36.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.364+0000 7fbf18c6d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55559a67e740 0x55559a655ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.364+0000 7fbf18c6d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55559a67eb20 0x55559a6811e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.364+0000 7fbf18c6d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a683ba0 0x55559a681720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.364+0000 7fbf18c6d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbf10200910 con 0x55559a683ba0 2024-04-04T06:16:36.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.364+0000 7fbf18c6d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbf101ffaa0 con 0x55559a67e740 2024-04-04T06:16:36.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.364+0000 7fbf18c6d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbf1015f620 con 0x55559a67eb20 2024-04-04T06:16:36.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.365+0000 7fbf0fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a683ba0 0x55559a681720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.365+0000 7fbf0fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a683ba0 0x55559a681720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51258/0 (socket says 172.21.15.159:51258) 2024-04-04T06:16:36.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.365+0000 7fbf0fe00640 1 -- 172.21.15.159:0/3859098324 learned_addr learned my addr 172.21.15.159:0/3859098324 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:36.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.365+0000 7fbf0f400640 1 --2- 172.21.15.159:0/3859098324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55559a67e740 0x55559a655ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.365+0000 7fbf07e00640 1 --2- 172.21.15.159:0/3859098324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55559a67eb20 0x55559a6811e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.365+0000 7fbf0fe00640 1 -- 172.21.15.159:0/3859098324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55559a67eb20 msgr2=0x55559a6811e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:36.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.365+0000 7fbf0fe00640 1 --2- 172.21.15.159:0/3859098324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55559a67eb20 0x55559a6811e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.365+0000 7fbf0fe00640 1 -- 172.21.15.159:0/3859098324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55559a67e740 msgr2=0x55559a655ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:36.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.365+0000 7fbf0fe00640 1 --2- 172.21.15.159:0/3859098324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55559a67e740 0x55559a655ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.365+0000 7fbf0fe00640 1 -- 172.21.15.159:0/3859098324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55559a688ab0 con 0x55559a683ba0 2024-04-04T06:16:36.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.365+0000 7fbf0f400640 1 --2- 172.21.15.159:0/3859098324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55559a67e740 0x55559a655ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:36.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.365+0000 7fbf07e00640 1 --2- 172.21.15.159:0/3859098324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55559a67eb20 0x55559a6811e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:36.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.366+0000 7fbf0fe00640 1 --2- 172.21.15.159:0/3859098324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a683ba0 0x55559a681720 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fbf08009010 tx=0x7fbf0801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58286b47aa963bd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:36.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.366+0000 7fbefea00640 1 -- 172.21.15.159:0/3859098324 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0802f040 con 0x55559a683ba0 2024-04-04T06:16:36.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.366+0000 7fbefea00640 1 -- 172.21.15.159:0/3859098324 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf08012c60 con 0x55559a683ba0 2024-04-04T06:16:36.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.366+0000 7fbefea00640 1 -- 172.21.15.159:0/3859098324 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08020e80 con 0x55559a683ba0 2024-04-04T06:16:36.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.367+0000 7fbf18c6d740 1 -- 172.21.15.159:0/3859098324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a683ba0 msgr2=0x55559a681720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:36.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.367+0000 7fbf18c6d740 1 --2- 172.21.15.159:0/3859098324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a683ba0 0x55559a681720 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fbf08009010 tx=0x7fbf0801d000 comp rx=0 tx=0).stop 2024-04-04T06:16:36.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.367+0000 7fbf18c6d740 1 -- 172.21.15.159:0/3859098324 shutdown_connections 2024-04-04T06:16:36.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.367+0000 7fbf18c6d740 1 --2- 172.21.15.159:0/3859098324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a683ba0 0x55559a681720 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.367+0000 7fbf18c6d740 1 --2- 172.21.15.159:0/3859098324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55559a67eb20 0x55559a6811e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.367+0000 7fbf18c6d740 1 --2- 172.21.15.159:0/3859098324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55559a67e740 0x55559a655ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.367+0000 7fbf18c6d740 1 -- 172.21.15.159:0/3859098324 >> 172.21.15.159:0/3859098324 conn(0x55559a651c20 msgr2=0x55559a6ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:36.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.367+0000 7fbf18c6d740 1 -- 172.21.15.159:0/3859098324 shutdown_connections 2024-04-04T06:16:36.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.367+0000 7fbf18c6d740 1 -- 172.21.15.159:0/3859098324 wait complete. 2024-04-04T06:16:36.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.368+0000 7fbf18c6d740 1 -- start start 2024-04-04T06:16:36.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.368+0000 7fbf18c6d740 1 Processor -- start 2024-04-04T06:16:36.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.369+0000 7fbf18c6d740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55559a67e740 0x55559a6afc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.369+0000 7fbf18c6d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a67eb20 0x55559a681660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.369+0000 7fbf18c6d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55559a683ba0 0x55559a67ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.369+0000 7fbf18c6d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55559a6b0cc0 con 0x55559a67eb20 2024-04-04T06:16:36.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.369+0000 7fbf18c6d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbf100cc6c0 con 0x55559a683ba0 2024-04-04T06:16:36.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.369+0000 7fbf18c6d740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55559a657e50 con 0x55559a67e740 2024-04-04T06:16:36.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.369+0000 7fbf18c6d740 10 client.0 _ll_register_callbacks cb 0x7fbf080137e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:16:36.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.369+0000 7fbf07e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a67eb20 0x55559a681660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.369+0000 7fbf07e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a67eb20 0x55559a681660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51264/0 (socket says 172.21.15.159:51264) 2024-04-04T06:16:36.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.369+0000 7fbf07e00640 1 -- 172.21.15.159:0/3406350357 learned_addr learned my addr 172.21.15.159:0/3406350357 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:36.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.369+0000 7fbf0f400640 1 --2- 172.21.15.159:0/3406350357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55559a67e740 0x55559a6afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.370+0000 7fbf0fe00640 1 --2- 172.21.15.159:0/3406350357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55559a683ba0 0x55559a67ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.370+0000 7fbf07e00640 1 -- 172.21.15.159:0/3406350357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55559a67e740 msgr2=0x55559a6afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:36.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.370+0000 7fbf07e00640 1 --2- 172.21.15.159:0/3406350357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55559a67e740 0x55559a6afc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.370+0000 7fbf07e00640 1 -- 172.21.15.159:0/3406350357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55559a683ba0 msgr2=0x55559a67ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:36.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.370+0000 7fbf07e00640 1 --2- 172.21.15.159:0/3406350357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55559a683ba0 0x55559a67ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.370+0000 7fbf07e00640 1 -- 172.21.15.159:0/3406350357 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55559a717c00 con 0x55559a67eb20 2024-04-04T06:16:36.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.370+0000 7fbf07e00640 1 --2- 172.21.15.159:0/3406350357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a67eb20 0x55559a681660 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fbef80098c0 tx=0x7fbef80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4768cc30c4c9275d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:36.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.370+0000 7fbef3e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:16:36.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.370+0000 7fbef3e00640 1 -- 172.21.15.159:0/3406350357 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8034040 con 0x55559a67eb20 2024-04-04T06:16:36.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.370+0000 7fbef3e00640 1 -- 172.21.15.159:0/3406350357 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7fbef8003cd0 con 0x55559a67eb20 2024-04-04T06:16:36.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.371+0000 7fbf18c6d740 1 -- 172.21.15.159:0/3406350357 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf08012c60 con 0x55559a67eb20 2024-04-04T06:16:36.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.371+0000 7fbef3e00640 1 -- 172.21.15.159:0/3406350357 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fbef8050050 con 0x55559a67eb20 2024-04-04T06:16:36.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.371+0000 7fbf18c6d740 10 client.8673 fetch_fsmap learned FSMap version 90 2024-04-04T06:16:36.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.371+0000 7fbf18c6d740 1 -- 172.21.15.159:0/3406350357 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55559a717890 con 0x55559a67eb20 2024-04-04T06:16:36.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.371+0000 7fbef3e00640 1 -- 172.21.15.159:0/3406350357 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf08012c60 con 0x55559a67eb20 2024-04-04T06:16:36.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.371+0000 7fbef3e00640 1 -- 172.21.15.159:0/3406350357 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 90) ==== 3494+0+0 (secure 0 0 0) 0x7fbef8053020 con 0x55559a67eb20 2024-04-04T06:16:36.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.371+0000 7fbf18c6d740 10 client.8673 fetch_fsmap finished waiting for FSMap version 90 2024-04-04T06:16:36.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.371+0000 7fbf18c6d740 10 client.8673 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-04-04T06:16:36.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.371+0000 7fbf18c6d740 20 client.8673 populate_metadata read hostname 'smithi159' 2024-04-04T06:16:36.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.371+0000 7fbf18c6d740 1 --2- 172.21.15.159:0/3406350357 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55559a67d360 0x55559a69edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.371+0000 7fbf18c6d740 4 client.8673 mds_command: new command op to 8535 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-04T06:16:36.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.371+0000 7fbf18c6d740 1 -- 172.21.15.159:0/3406350357 --> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55559a5c3f60 con 0x55559a67d360 2024-04-04T06:16:36.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.372+0000 7fbf0f400640 1 --2- 172.21.15.159:0/3406350357 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55559a67d360 0x55559a69edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.372+0000 7fbf0f400640 1 --2- 172.21.15.159:0/3406350357 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55559a67d360 0x55559a69edf0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d2887020ace6252 server_cookie=c6593113f9e8c7ba in_seq=0 out_seq=0 2024-04-04T06:16:36.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.372+0000 7fbef3e00640 10 client.8673 ms_handle_connect on v2:172.21.15.159:6834/1003751501 2024-04-04T06:16:36.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.372+0000 7fbef3e00640 1 -- 172.21.15.159:0/3406350357 <== mds.0 v2:172.21.15.159:6834/1003751501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55559a5c3f60 con 0x55559a67d360 2024-04-04T06:16:36.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.372+0000 7fbef3e00640 10 client.8673 handle_command_reply: tid=0 2024-04-04T06:16:36.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.372+0000 7fbf18c6d740 1 client.8673 shutdown 2024-04-04T06:16:36.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.373+0000 7fbf18c6d740 1 -- 172.21.15.159:0/3406350357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a67eb20 msgr2=0x55559a681660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:36.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.373+0000 7fbf18c6d740 1 --2- 172.21.15.159:0/3406350357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a67eb20 0x55559a681660 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fbef80098c0 tx=0x7fbef80046c0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.373+0000 7fbf18c6d740 1 -- 172.21.15.159:0/3406350357 shutdown_connections 2024-04-04T06:16:36.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.373+0000 7fbf18c6d740 1 --2- 172.21.15.159:0/3406350357 >> [v2:172.21.15.159:6834/1003751501,v1:172.21.15.159:6835/1003751501] conn(0x55559a67d360 0x55559a69edf0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.373+0000 7fbf18c6d740 1 --2- 172.21.15.159:0/3406350357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55559a683ba0 0x55559a67ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.373+0000 7fbf18c6d740 1 --2- 172.21.15.159:0/3406350357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55559a67eb20 0x55559a681660 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.373+0000 7fbf18c6d740 1 --2- 172.21.15.159:0/3406350357 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55559a67e740 0x55559a6afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.373+0000 7fbef3e00640 0 client.8673 ms_handle_reset on v2:172.21.15.159:6834/1003751501 2024-04-04T06:16:36.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.373+0000 7fbf18c6d740 1 -- 172.21.15.159:0/3406350357 >> 172.21.15.159:0/3406350357 conn(0x55559a651c20 msgr2=0x55559a680f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:36.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.373+0000 7fbf18c6d740 1 -- 172.21.15.159:0/3406350357 shutdown_connections 2024-04-04T06:16:36.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.373+0000 7fbf18c6d740 1 -- 172.21.15.159:0/3406350357 wait complete. 2024-04-04T06:16:36.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.373+0000 7fbf18c6d740 20 client.8673 trim_cache size 0 max 16384 2024-04-04T06:16:36.372 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:16:36.372 INFO:teuthology.orchestra.run.smithi159.stdout: "status": "no active scrubs running", 2024-04-04T06:16:36.372 INFO:teuthology.orchestra.run.smithi159.stdout: "scrubs": {} 2024-04-04T06:16:36.372 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:16:36.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.374+0000 7fbf14a00640 1 -- 172.21.15.159:0/1940434142 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbeec06c8f0 msgr2=0x7fbeec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:36.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.374+0000 7fbf14a00640 1 --2- 172.21.15.159:0/1940434142 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbeec06c8f0 0x7fbeec06edb0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbf08001560 tx=0x7fbf08009040 comp rx=0 tx=0).stop 2024-04-04T06:16:36.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.374+0000 7fbf14a00640 1 -- 172.21.15.159:0/1940434142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf100b67f0 msgr2=0x7fbf100ca040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:36.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.374+0000 7fbf14a00640 1 --2- 172.21.15.159:0/1940434142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf100b67f0 0x7fbf100ca040 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000c400 tx=0x7fbf0000c8c0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.374+0000 7fbf14a00640 1 -- 172.21.15.159:0/1940434142 shutdown_connections 2024-04-04T06:16:36.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.374+0000 7fbf14a00640 1 --2- 172.21.15.159:0/1940434142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbf1015e4c0 0x7fbf10160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.374+0000 7fbf14a00640 1 --2- 172.21.15.159:0/1940434142 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbeec06c8f0 0x7fbeec06edb0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.374+0000 7fbf14a00640 1 --2- 172.21.15.159:0/1940434142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf100d37b0 0x7fbf1014b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.374+0000 7fbf14a00640 1 --2- 172.21.15.159:0/1940434142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf100b67f0 0x7fbf100ca040 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.374+0000 7fbf14a00640 1 -- 172.21.15.159:0/1940434142 >> 172.21.15.159:0/1940434142 conn(0x7fbf100676a0 msgr2=0x7fbf100b93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:36.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.374+0000 7fbf14a00640 1 -- 172.21.15.159:0/1940434142 shutdown_connections 2024-04-04T06:16:36.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.374+0000 7fbf14a00640 1 -- 172.21.15.159:0/1940434142 wait complete. 2024-04-04T06:16:36.387 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-04T06:16:36.488 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-04T06:16:36.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.698+0000 7f6545800640 1 Processor -- start 2024-04-04T06:16:36.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.699+0000 7f6545800640 1 -- start start 2024-04-04T06:16:36.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.701+0000 7f6545800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65400cf700 0x7f6540168fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.701+0000 7f6545800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f654015e880 con 0x7f654015e4a0 2024-04-04T06:16:36.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.701+0000 7f6545800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65400d28d0 con 0x7f65400d2310 2024-04-04T06:16:36.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.701+0000 7f6545800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f65400d34e0 con 0x7f65400cf700 2024-04-04T06:16:36.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.701+0000 7f653fe00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f654015e4a0 0x7f65400b9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37662/0 (socket says 172.21.15.159:37662) 2024-04-04T06:16:36.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.701+0000 7f653fe00640 1 -- 172.21.15.159:0/2962870326 learned_addr learned my addr 172.21.15.159:0/2962870326 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:16:36.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.702+0000 7f6544e00640 1 --2- 172.21.15.159:0/2962870326 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65400cf700 0x7f6540168fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.702+0000 7f653ea00640 1 -- 172.21.15.159:0/2962870326 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2961464701 0 0) 0x7f654015e880 con 0x7f654015e4a0 2024-04-04T06:16:36.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.702+0000 7f653ea00640 1 -- 172.21.15.159:0/2962870326 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6524003520 con 0x7f654015e4a0 2024-04-04T06:16:36.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.702+0000 7f653ea00640 1 -- 172.21.15.159:0/2962870326 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2424343144 0 0) 0x7f6524003520 con 0x7f654015e4a0 2024-04-04T06:16:36.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.702+0000 7f653ea00640 1 -- 172.21.15.159:0/2962870326 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65400cf700 msgr2=0x7f6540168fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:36.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.702+0000 7f653ea00640 1 --2- 172.21.15.159:0/2962870326 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65400cf700 0x7f6540168fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.702+0000 7f653ea00640 1 -- 172.21.15.159:0/2962870326 >> v1:172.21.15.178:6789/0 conn(0x7f65400d2310 legacy=0x7f65400ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:36.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.702+0000 7f653ea00640 1 -- 172.21.15.159:0/2962870326 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65400d2bb0 con 0x7f654015e4a0 2024-04-04T06:16:36.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.702+0000 7f653ea00640 1 -- 172.21.15.159:0/2962870326 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f6530001670 con 0x7f654015e4a0 2024-04-04T06:16:36.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.702+0000 7f653ea00640 1 -- 172.21.15.159:0/2962870326 >> v1:172.21.15.159:6789/0 conn(0x7f654015e4a0 legacy=0x7f65400b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:36.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.702+0000 7f653ea00640 1 --2- 172.21.15.159:0/2962870326 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65400cf700 0x7f65240039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.703+0000 7f653ea00640 1 --2- 172.21.15.159:0/2962870326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6524003f60 0x7f6524004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.703+0000 7f653ea00640 1 --2- 172.21.15.159:0/2962870326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6524004d40 0x7f65240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.703+0000 7f653ea00640 1 -- 172.21.15.159:0/2962870326 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f65400d34e0 con 0x7f6524003f60 2024-04-04T06:16:36.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.703+0000 7f653ea00640 1 -- 172.21.15.159:0/2962870326 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f652400bb50 con 0x7f6524004d40 2024-04-04T06:16:36.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.703+0000 7f653ea00640 1 -- 172.21.15.159:0/2962870326 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f652400bd50 con 0x7f65400cf700 2024-04-04T06:16:36.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.703+0000 7f653f400640 1 --2- 172.21.15.159:0/2962870326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6524003f60 0x7f6524004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.703+0000 7f6544e00640 1 --2- 172.21.15.159:0/2962870326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6524004d40 0x7f65240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.703+0000 7f653fe00640 1 --2- 172.21.15.159:0/2962870326 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65400cf700 0x7f65240039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.703+0000 7f653f400640 1 -- 172.21.15.159:0/2962870326 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65400cf700 msgr2=0x7f65240039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:36.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.703+0000 7f653f400640 1 --2- 172.21.15.159:0/2962870326 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65400cf700 0x7f65240039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.703+0000 7f653f400640 1 -- 172.21.15.159:0/2962870326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6524004d40 msgr2=0x7f65240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:36.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.703+0000 7f653f400640 1 --2- 172.21.15.159:0/2962870326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6524004d40 0x7f65240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.703+0000 7f653f400640 1 -- 172.21.15.159:0/2962870326 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6524003520 con 0x7f6524003f60 2024-04-04T06:16:36.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.704+0000 7f653f400640 1 --2- 172.21.15.159:0/2962870326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6524003f60 0x7f6524004800 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f652c001750 tx=0x7f6524005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc5595c6dc8e8a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:36.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.705+0000 7f653ea00640 1 -- 172.21.15.159:0/2962870326 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c001370 con 0x7f6524003f60 2024-04-04T06:16:36.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.705+0000 7f653ea00640 1 -- 172.21.15.159:0/2962870326 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f652c0052d0 con 0x7f6524003f60 2024-04-04T06:16:36.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.705+0000 7f653ea00640 1 -- 172.21.15.159:0/2962870326 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c003830 con 0x7f6524003f60 2024-04-04T06:16:36.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.705+0000 7f6545800640 1 -- 172.21.15.159:0/2962870326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6524003f60 msgr2=0x7f6524004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:36.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.705+0000 7f6545800640 1 --2- 172.21.15.159:0/2962870326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6524003f60 0x7f6524004800 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f652c001750 tx=0x7f6524005830 comp rx=0 tx=0).stop 2024-04-04T06:16:36.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.705+0000 7f6544e00640 1 -- 172.21.15.159:0/2962870326 reap_dead start 2024-04-04T06:16:36.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.706+0000 7f6545800640 1 -- 172.21.15.159:0/2962870326 shutdown_connections 2024-04-04T06:16:36.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.706+0000 7f6545800640 1 -- 172.21.15.159:0/2962870326 >> 172.21.15.159:0/2962870326 conn(0x7f65400676d0 msgr2=0x7f6540067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:36.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.706+0000 7f6545800640 1 -- 172.21.15.159:0/2962870326 shutdown_connections 2024-04-04T06:16:36.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.706+0000 7f6545800640 1 -- 172.21.15.159:0/2962870326 wait complete. 2024-04-04T06:16:36.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.707+0000 7f6545800640 1 Processor -- start 2024-04-04T06:16:36.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.708+0000 7f6545800640 1 -- start start 2024-04-04T06:16:36.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.708+0000 7f6545800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6524003f60 0x7f6540160d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.708+0000 7f6545800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65401551e0 0x7f65400ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.709+0000 7f6545800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6540155660 0x7f65401f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.709+0000 7f6545800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f65400d3970 con 0x7f65401551e0 2024-04-04T06:16:36.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.709+0000 7f6545800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f65400d3170 con 0x7f6540155660 2024-04-04T06:16:36.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.709+0000 7f6545800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f65400b83f0 con 0x7f6524003f60 2024-04-04T06:16:36.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.709+0000 7f653f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65401551e0 0x7f65400ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.709+0000 7f653f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65401551e0 0x7f65400ca020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51282/0 (socket says 172.21.15.159:51282) 2024-04-04T06:16:36.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.709+0000 7f653f400640 1 -- 172.21.15.159:0/2072865693 learned_addr learned my addr 172.21.15.159:0/2072865693 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:36.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.709+0000 7f6544e00640 1 --2- 172.21.15.159:0/2072865693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6540155660 0x7f65401f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.709+0000 7f653fe00640 1 --2- 172.21.15.159:0/2072865693 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6524003f60 0x7f6540160d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.709+0000 7f653f400640 1 -- 172.21.15.159:0/2072865693 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6524003f60 msgr2=0x7f6540160d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:36.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.709+0000 7f653f400640 1 --2- 172.21.15.159:0/2072865693 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6524003f60 0x7f6540160d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.709+0000 7f653f400640 1 -- 172.21.15.159:0/2072865693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6540155660 msgr2=0x7f65401f73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:36.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.709+0000 7f653f400640 1 --2- 172.21.15.159:0/2072865693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6540155660 0x7f65401f73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:36.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.709+0000 7f653f400640 1 -- 172.21.15.159:0/2072865693 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65401580a0 con 0x7f65401551e0 2024-04-04T06:16:36.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.709+0000 7f653fe00640 1 --2- 172.21.15.159:0/2072865693 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6524003f60 0x7f6540160d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:36.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.710+0000 7f653f400640 1 --2- 172.21.15.159:0/2072865693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65401551e0 0x7f65400ca020 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f652c001870 tx=0x7f652c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:36.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.710+0000 7f6537e00640 1 -- 172.21.15.159:0/2072865693 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c041070 con 0x7f65401551e0 2024-04-04T06:16:36.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.710+0000 7f6537e00640 1 -- 172.21.15.159:0/2072865693 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f652c005480 con 0x7f65401551e0 2024-04-04T06:16:36.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.710+0000 7f6537e00640 1 -- 172.21.15.159:0/2072865693 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c001370 con 0x7f65401551e0 2024-04-04T06:16:36.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.710+0000 7f6545800640 1 -- 172.21.15.159:0/2072865693 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6540158380 con 0x7f65401551e0 2024-04-04T06:16:36.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.710+0000 7f6545800640 1 -- 172.21.15.159:0/2072865693 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6540160810 con 0x7f65401551e0 2024-04-04T06:16:36.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.712+0000 7f6545800640 1 -- 172.21.15.159:0/2072865693 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6540075940 con 0x7f65401551e0 2024-04-04T06:16:36.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.713+0000 7f6537e00640 1 -- 172.21.15.159:0/2072865693 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f652c044070 con 0x7f65401551e0 2024-04-04T06:16:36.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.713+0000 7f6537e00640 1 --2- 172.21.15.159:0/2072865693 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f652006c8f0 0x7f652006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:36.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.714+0000 7f6537e00640 1 -- 172.21.15.159:0/2072865693 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f652c0ad690 con 0x7f65401551e0 2024-04-04T06:16:36.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.714+0000 7f653fe00640 1 --2- 172.21.15.159:0/2072865693 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f652006c8f0 0x7f652006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:36.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.715+0000 7f653fe00640 1 --2- 172.21.15.159:0/2072865693 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f652006c8f0 0x7f652006edb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6530001200 tx=0x7f6530009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:36.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:36.718+0000 7f6537e00640 1 -- 172.21.15.159:0/2072865693 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f652c07ba60 con 0x7f65401551e0 2024-04-04T06:16:37.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.032+0000 7f6545800640 1 -- 172.21.15.159:0/2072865693 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f654015d9a0 con 0x7f65401551e0 2024-04-04T06:16:37.028 INFO:tasks.ceph.mon.a.smithi159.stderr:2024-04-04T06:16:37.033+0000 7f1995c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-04T06:16:37.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.037+0000 7f6537e00640 1 -- 172.21.15.159:0/2072865693 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) ==== 161+0+0 (secure 0 0 0) 0x7f652c07b360 con 0x7f65401551e0 2024-04-04T06:16:37.031 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-04T06:16:37.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.039+0000 7f6545800640 1 -- 172.21.15.159:0/2072865693 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f652006c8f0 msgr2=0x7f652006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:37.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.039+0000 7f6545800640 1 --2- 172.21.15.159:0/2072865693 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f652006c8f0 0x7f652006edb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6530001200 tx=0x7f6530009040 comp rx=0 tx=0).stop 2024-04-04T06:16:37.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.039+0000 7f6545800640 1 -- 172.21.15.159:0/2072865693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65401551e0 msgr2=0x7f65400ca020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:37.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.039+0000 7f6545800640 1 --2- 172.21.15.159:0/2072865693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65401551e0 0x7f65400ca020 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f652c001870 tx=0x7f652c001120 comp rx=0 tx=0).stop 2024-04-04T06:16:37.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.040+0000 7f6545800640 1 -- 172.21.15.159:0/2072865693 shutdown_connections 2024-04-04T06:16:37.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.040+0000 7f6545800640 1 --2- 172.21.15.159:0/2072865693 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f652006c8f0 0x7f652006edb0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.040+0000 7f6545800640 1 --2- 172.21.15.159:0/2072865693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6540155660 0x7f65401f73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.040+0000 7f6545800640 1 --2- 172.21.15.159:0/2072865693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65401551e0 0x7f65400ca020 secure :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f652c001870 tx=0x7f652c001120 comp rx=0 tx=0).stop 2024-04-04T06:16:37.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.040+0000 7f6545800640 1 --2- 172.21.15.159:0/2072865693 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6524003f60 0x7f6540160d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.040+0000 7f6545800640 1 -- 172.21.15.159:0/2072865693 >> 172.21.15.159:0/2072865693 conn(0x7f65400676d0 msgr2=0x7f654015ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:37.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.041+0000 7f6545800640 1 -- 172.21.15.159:0/2072865693 shutdown_connections 2024-04-04T06:16:37.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.041+0000 7f6545800640 1 -- 172.21.15.159:0/2072865693 wait complete. 2024-04-04T06:16:37.047 INFO:tasks.ceph.mds.a.smithi159.stderr: -1313> 2024-04-04T06:16:05.505+0000 7f1ea3200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:16:37.048 INFO:tasks.ceph.mds.a.smithi159.stderr: -1143> 2024-04-04T06:16:05.811+0000 7f1e9e200640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-04T06:16:37.048 INFO:tasks.ceph.mds.a.smithi159.stderr: -1140> 2024-04-04T06:16:05.811+0000 7f1e9e200640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-04T06:16:37.048 INFO:tasks.ceph.mds.a.smithi159.stderr: -1123> 2024-04-04T06:16:05.811+0000 7f1e9e200640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-04T06:16:37.048 INFO:tasks.ceph.mds.a.smithi159.stderr: -1120> 2024-04-04T06:16:05.811+0000 7f1e9e200640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-04T06:16:37.055 DEBUG:teuthology.orchestra.run.smithi159:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-04-04T06:16:37.095 INFO:tasks.ceph.mds.a.smithi159.stdout:starting mds.a at 2024-04-04T06:16:37.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.140+0000 7fd5f3e71280 1 -- start start 2024-04-04T06:16:37.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.140+0000 7fd5f3e71280 1 Processor -- start 2024-04-04T06:16:37.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.141+0000 7fd5f3e71280 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602e3166c00 0x5602e3ed2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:37.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.141+0000 7fd5f3e71280 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602e3167800 0x5602e3ed2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:37.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.141+0000 7fd5f3e71280 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5602e3167c00 0x5602e3ed2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:37.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.141+0000 7fd5f3e71280 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5602e3f44180 con 0x5602e3166c00 2024-04-04T06:16:37.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.141+0000 7fd5f3e71280 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5602e31c8900 con 0x5602e3167800 2024-04-04T06:16:37.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.141+0000 7fd5f3e71280 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5602e31c8480 con 0x5602e3167c00 2024-04-04T06:16:37.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.141+0000 7fd5f2000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602e3166c00 0x5602e3ed2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:37.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.141+0000 7fd5f2000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602e3166c00 0x5602e3ed2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51314/0 (socket says 172.21.15.159:51314) 2024-04-04T06:16:37.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.141+0000 7fd5f2000640 1 -- 172.21.15.159:0/3239193683 learned_addr learned my addr 172.21.15.159:0/3239193683 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:37.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.142+0000 7fd5f1600640 1 --2- 172.21.15.159:0/3239193683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602e3167800 0x5602e3ed2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:37.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.142+0000 7fd5f2a00640 1 --2- 172.21.15.159:0/3239193683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5602e3167c00 0x5602e3ed2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:37.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.142+0000 7fd5f2000640 1 -- 172.21.15.159:0/3239193683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5602e3167c00 msgr2=0x5602e3ed2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:37.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.142+0000 7fd5f2000640 1 --2- 172.21.15.159:0/3239193683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5602e3167c00 0x5602e3ed2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.142+0000 7fd5f2000640 1 -- 172.21.15.159:0/3239193683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602e3167800 msgr2=0x5602e3ed2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:37.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.142+0000 7fd5f2000640 1 --2- 172.21.15.159:0/3239193683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602e3167800 0x5602e3ed2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.142+0000 7fd5f2000640 1 -- 172.21.15.159:0/3239193683 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602e319a780 con 0x5602e3166c00 2024-04-04T06:16:37.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.142+0000 7fd5f2000640 1 --2- 172.21.15.159:0/3239193683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602e3166c00 0x5602e3ed2000 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x5602e3f2ed50 tx=0x5602e3efb380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a525fb242c85dc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:37.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.142+0000 7fd5eee00640 1 -- 172.21.15.159:0/3239193683 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5602e318c9c0 con 0x5602e3166c00 2024-04-04T06:16:37.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.142+0000 7fd5eee00640 1 -- 172.21.15.159:0/3239193683 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5602e3192700 con 0x5602e3166c00 2024-04-04T06:16:37.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.142+0000 7fd5eee00640 1 -- 172.21.15.159:0/3239193683 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5602e318c680 con 0x5602e3166c00 2024-04-04T06:16:37.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.142+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3239193683 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602e319b4a0 con 0x5602e3166c00 2024-04-04T06:16:37.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.142+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3239193683 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5602e319a000 con 0x5602e3166c00 2024-04-04T06:16:37.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.142+0000 7fd5f3e71280 4 waiting for MDS map... 2024-04-04T06:16:37.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.142+0000 7fd5eee00640 1 -- 172.21.15.159:0/3239193683 <== mon.0 v2:172.21.15.159:3300/0 4 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x5602e402a200 con 0x5602e3166c00 2024-04-04T06:16:37.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.143+0000 7fd5eee00640 1 -- 172.21.15.159:0/3239193683 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 91) ==== 2885+0+0 (secure 0 0 0) 0x5602e31c1180 con 0x5602e3166c00 2024-04-04T06:16:37.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.143+0000 7fd5f3e71280 4 Got MDS map 91 2024-04-04T06:16:37.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.143+0000 7fd5f3e71280 10 main: main 2024-04-04T06:16:37.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.143+0000 7fd5f3e71280 4 main: connecting to RADOS... 2024-04-04T06:16:37.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.144+0000 7fd5f3e71280 1 Processor -- start 2024-04-04T06:16:37.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.144+0000 7fd5f3e71280 1 -- start start 2024-04-04T06:16:37.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.144+0000 7fd5f3e71280 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5602e408e000 0x5602e3ed3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:37.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.144+0000 7fd5f3e71280 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602e408e400 0x5602e3ed3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:37.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.145+0000 7fd5f3e71280 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602e408e800 0x5602e3ed3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:37.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.145+0000 7fd5f3e71280 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5602e31c8180 con 0x5602e408e400 2024-04-04T06:16:37.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.145+0000 7fd5f3e71280 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5602e3f44d80 con 0x5602e408e800 2024-04-04T06:16:37.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.145+0000 7fd5f3e71280 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5602e3f44f00 con 0x5602e408e000 2024-04-04T06:16:37.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.145+0000 7fd5f2000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602e408e400 0x5602e3ed3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:37.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.145+0000 7fd5f2000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602e408e400 0x5602e3ed3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51328/0 (socket says 172.21.15.159:51328) 2024-04-04T06:16:37.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.145+0000 7fd5f2000640 1 -- 172.21.15.159:0/3114900873 learned_addr learned my addr 172.21.15.159:0/3114900873 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:37.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.145+0000 7fd5f1600640 1 --2- 172.21.15.159:0/3114900873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602e408e800 0x5602e3ed3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:37.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.145+0000 7fd5f2a00640 1 --2- 172.21.15.159:0/3114900873 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5602e408e000 0x5602e3ed3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:37.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.145+0000 7fd5f2000640 1 -- 172.21.15.159:0/3114900873 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5602e408e000 msgr2=0x5602e3ed3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:37.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.145+0000 7fd5f2000640 1 --2- 172.21.15.159:0/3114900873 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5602e408e000 0x5602e3ed3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.145+0000 7fd5f2000640 1 -- 172.21.15.159:0/3114900873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602e408e800 msgr2=0x5602e3ed3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:37.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.145+0000 7fd5f2000640 1 --2- 172.21.15.159:0/3114900873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602e408e800 0x5602e3ed3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.145+0000 7fd5f2000640 1 -- 172.21.15.159:0/3114900873 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602e319a3c0 con 0x5602e408e400 2024-04-04T06:16:37.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.145+0000 7fd5f1600640 1 --2- 172.21.15.159:0/3114900873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602e408e800 0x5602e3ed3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:37.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.145+0000 7fd5f2000640 1 --2- 172.21.15.159:0/3114900873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602e408e400 0x5602e3ed3600 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x5602e3f2f170 tx=0x5602e3efb440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdce9f9ab96995c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:37.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.146+0000 7fd5ec600640 1 -- 172.21.15.159:0/3114900873 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5602e318cb60 con 0x5602e408e400 2024-04-04T06:16:37.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.146+0000 7fd5ec600640 1 -- 172.21.15.159:0/3114900873 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5602e3192fc0 con 0x5602e408e400 2024-04-04T06:16:37.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.146+0000 7fd5ec600640 1 -- 172.21.15.159:0/3114900873 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5602e318c820 con 0x5602e408e400 2024-04-04T06:16:37.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.146+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3114900873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602e408e400 msgr2=0x5602e3ed3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:37.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.146+0000 7fd5f3e71280 1 --2- 172.21.15.159:0/3114900873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602e408e400 0x5602e3ed3600 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x5602e3f2f170 tx=0x5602e3efb440 comp rx=0 tx=0).stop 2024-04-04T06:16:37.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.146+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3114900873 shutdown_connections 2024-04-04T06:16:37.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.146+0000 7fd5f3e71280 1 --2- 172.21.15.159:0/3114900873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602e408e800 0x5602e3ed3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.146+0000 7fd5f3e71280 1 --2- 172.21.15.159:0/3114900873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602e408e400 0x5602e3ed3600 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.146+0000 7fd5f3e71280 1 --2- 172.21.15.159:0/3114900873 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5602e408e000 0x5602e3ed3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.146+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3114900873 >> 172.21.15.159:0/3114900873 conn(0x5602e3f4fc00 msgr2=0x5602e325b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:37.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.146+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3114900873 shutdown_connections 2024-04-04T06:16:37.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.146+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3114900873 wait complete. 2024-04-04T06:16:37.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.147+0000 7fd5f3e71280 1 Processor -- start 2024-04-04T06:16:37.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.147+0000 7fd5f3e71280 1 -- start start 2024-04-04T06:16:37.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.148+0000 7fd5f3e71280 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5602e408e000 0x5602e3ed3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:37.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.148+0000 7fd5f3e71280 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602e408e400 0x5602e3ed3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:37.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.148+0000 7fd5f3e71280 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602e408e800 0x5602e3ed3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:37.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.148+0000 7fd5f3e71280 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5602e3f45200 con 0x5602e408e800 2024-04-04T06:16:37.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.148+0000 7fd5f3e71280 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5602e3f45080 con 0x5602e408e400 2024-04-04T06:16:37.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.148+0000 7fd5f3e71280 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5602e3f45380 con 0x5602e408e000 2024-04-04T06:16:37.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.148+0000 7fd5f1600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602e408e800 0x5602e3ed3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:37.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.148+0000 7fd5f1600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602e408e800 0x5602e3ed3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51336/0 (socket says 172.21.15.159:51336) 2024-04-04T06:16:37.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.148+0000 7fd5f1600640 1 -- 172.21.15.159:0/3086141733 learned_addr learned my addr 172.21.15.159:0/3086141733 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:37.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.149+0000 7fd5f2a00640 1 --2- 172.21.15.159:0/3086141733 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5602e408e000 0x5602e3ed3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:37.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.149+0000 7fd5f2000640 1 --2- 172.21.15.159:0/3086141733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602e408e400 0x5602e3ed3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:37.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.149+0000 7fd5f1600640 1 -- 172.21.15.159:0/3086141733 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5602e408e000 msgr2=0x5602e3ed3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:37.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.149+0000 7fd5f1600640 1 --2- 172.21.15.159:0/3086141733 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5602e408e000 0x5602e3ed3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.149+0000 7fd5f1600640 1 -- 172.21.15.159:0/3086141733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602e408e400 msgr2=0x5602e3ed3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:37.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.149+0000 7fd5f1600640 1 --2- 172.21.15.159:0/3086141733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602e408e400 0x5602e3ed3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.149+0000 7fd5f1600640 1 -- 172.21.15.159:0/3086141733 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602e319a960 con 0x5602e408e800 2024-04-04T06:16:37.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.149+0000 7fd5f2a00640 1 --2- 172.21.15.159:0/3086141733 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5602e408e000 0x5602e3ed3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:37.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.149+0000 7fd5f1600640 1 --2- 172.21.15.159:0/3086141733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602e408e800 0x5602e3ed3b80 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x5602e3f33260 tx=0x5602e403ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:37.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.149+0000 7fd5ea800640 1 -- 172.21.15.159:0/3086141733 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5602e318cb60 con 0x5602e408e800 2024-04-04T06:16:37.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.149+0000 7fd5ea800640 1 -- 172.21.15.159:0/3086141733 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5602e3192a80 con 0x5602e408e800 2024-04-04T06:16:37.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.149+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3086141733 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5602e319b680 con 0x5602e408e800 2024-04-04T06:16:37.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.149+0000 7fd5ea800640 1 -- 172.21.15.159:0/3086141733 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5602e318c820 con 0x5602e408e800 2024-04-04T06:16:37.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.149+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3086141733 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602e319b860 con 0x5602e408e800 2024-04-04T06:16:37.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.149+0000 7fd5f3e71280 4 main: resolving pool 14 2024-04-04T06:16:37.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.151+0000 7fd5ea800640 1 -- 172.21.15.159:0/3086141733 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x5602e3f60c00 con 0x5602e408e800 2024-04-04T06:16:37.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.152+0000 7fd5ea800640 1 --2- 172.21.15.159:0/3086141733 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x5602e3f68c00 0x5602e3ed4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:37.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.152+0000 7fd5ea800640 1 -- 172.21.15.159:0/3086141733 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x5602e4095000 con 0x5602e408e800 2024-04-04T06:16:37.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.152+0000 7fd5f3e71280 4 main: creating IoCtx.. 2024-04-04T06:16:37.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.152+0000 7fd5f3e71280 1 --2- 172.21.15.159:0/3086141733 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x5602e408ec00 0x5602e3ed4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:37.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.152+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3086141733 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103) -- 0x5602e408f000 con 0x5602e408ec00 2024-04-04T06:16:37.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.152+0000 7fd5f2000640 1 --2- 172.21.15.159:0/3086141733 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x5602e408ec00 0x5602e3ed4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:37.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.152+0000 7fd5f2a00640 1 --2- 172.21.15.159:0/3086141733 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x5602e3f68c00 0x5602e3ed4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:37.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.153+0000 7fd5f2000640 1 --2- 172.21.15.159:0/3086141733 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x5602e408ec00 0x5602e3ed4680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:37.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.153+0000 7fd5f2a00640 1 --2- 172.21.15.159:0/3086141733 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x5602e3f68c00 0x5602e3ed4100 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x5602e3f30090 tx=0x5602e403f140 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:37.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.155+0000 7fd5f2000640 1 -- 172.21.15.159:0/3086141733 <== osd.2 v2:172.21.15.159:6808/2619731047 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x5602e3ee8000 con 0x5602e408ec00 2024-04-04T06:16:37.151 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:16:37.151 INFO:teuthology.orchestra.run.smithi159.stdout: "0": { 2024-04-04T06:16:37.151 INFO:teuthology.orchestra.run.smithi159.stdout: "data": { 2024-04-04T06:16:37.152 INFO:teuthology.orchestra.run.smithi159.stdout: "version": 5, 2024-04-04T06:16:37.152 INFO:teuthology.orchestra.run.smithi159.stdout: "inotable": { 2024-04-04T06:16:37.152 INFO:teuthology.orchestra.run.smithi159.stdout: "projected_free": [ 2024-04-04T06:16:37.152 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:37.152 INFO:teuthology.orchestra.run.smithi159.stdout: "start": 1099511627779, 2024-04-04T06:16:37.152 INFO:teuthology.orchestra.run.smithi159.stdout: "len": 1099511627773 2024-04-04T06:16:37.152 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:37.152 INFO:teuthology.orchestra.run.smithi159.stdout: ], 2024-04-04T06:16:37.152 INFO:teuthology.orchestra.run.smithi159.stdout: "free": [ 2024-04-04T06:16:37.152 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:37.152 INFO:teuthology.orchestra.run.smithi159.stdout: "start": 1099511627779, 2024-04-04T06:16:37.152 INFO:teuthology.orchestra.run.smithi159.stdout: "len": 1099511627773 2024-04-04T06:16:37.153 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:37.153 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-04-04T06:16:37.153 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:37.153 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:16:37.153 INFO:teuthology.orchestra.run.smithi159.stdout: "result": 0 2024-04-04T06:16:37.153 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:37.153 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:16:37.153 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:37.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.156+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3086141733 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x5602e408ec00 msgr2=0x5602e3ed4680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:37.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.156+0000 7fd5f3e71280 1 --2- 172.21.15.159:0/3086141733 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x5602e408ec00 0x5602e3ed4680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.156+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3086141733 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x5602e3f68c00 msgr2=0x5602e3ed4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:37.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.156+0000 7fd5f3e71280 1 --2- 172.21.15.159:0/3086141733 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x5602e3f68c00 0x5602e3ed4100 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x5602e3f30090 tx=0x5602e403f140 comp rx=0 tx=0).stop 2024-04-04T06:16:37.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.156+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3086141733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602e408e800 msgr2=0x5602e3ed3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:37.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.156+0000 7fd5f3e71280 1 --2- 172.21.15.159:0/3086141733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602e408e800 0x5602e3ed3b80 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x5602e3f33260 tx=0x5602e403ecc0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.156+0000 7fd5f1600640 1 -- 172.21.15.159:0/3086141733 reap_dead start 2024-04-04T06:16:37.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.156+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3086141733 shutdown_connections 2024-04-04T06:16:37.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.156+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3086141733 >> 172.21.15.159:0/3086141733 conn(0x5602e3f4fc00 msgr2=0x5602e325b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:37.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.156+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3086141733 shutdown_connections 2024-04-04T06:16:37.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.156+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3086141733 wait complete. 2024-04-04T06:16:37.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.156+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3239193683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602e3166c00 msgr2=0x5602e3ed2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:37.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.156+0000 7fd5f3e71280 1 --2- 172.21.15.159:0/3239193683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602e3166c00 0x5602e3ed2000 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x5602e3f2ed50 tx=0x5602e3efb380 comp rx=0 tx=0).stop 2024-04-04T06:16:37.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.157+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3239193683 shutdown_connections 2024-04-04T06:16:37.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.157+0000 7fd5f3e71280 1 --2- 172.21.15.159:0/3239193683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5602e3167c00 0x5602e3ed2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.157+0000 7fd5f3e71280 1 --2- 172.21.15.159:0/3239193683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5602e3167800 0x5602e3ed2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.157+0000 7fd5f3e71280 1 --2- 172.21.15.159:0/3239193683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602e3166c00 0x5602e3ed2000 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.157+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3239193683 >> 172.21.15.159:0/3239193683 conn(0x5602e3167000 msgr2=0x5602e325a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:37.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.157+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3239193683 shutdown_connections 2024-04-04T06:16:37.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.157+0000 7fd5f3e71280 1 -- 172.21.15.159:0/3239193683 wait complete. 2024-04-04T06:16:37.157 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.101682, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-04-04T06:16:37.157 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-04T06:16:37.157 DEBUG:teuthology.orchestra.run.smithi159:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:16:37.303 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:16:37.416 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-04T06:16:37.416 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:16:37.417 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:16:37.417 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:16:37.475 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:16:37.475 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:16:37.475 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:16:37.475 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:16:37.532 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:16:37.532 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:16:37.532 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:16:37.532 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:16:37.591 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:16:37.591 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:16:37.591 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:16:37.591 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:16:37.647 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:16:37.647 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:16:37.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.871+0000 7fd20bc00640 1 Processor -- start 2024-04-04T06:16:37.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.871+0000 7fd20bc00640 1 -- start start 2024-04-04T06:16:37.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.872+0000 7fd20bc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd20415c390 0x7fd204073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:37.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.872+0000 7fd20bc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd2040af830 0x7fd204160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:37.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.872+0000 7fd20bc00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd204160b80 0x7fd2040d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:37.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.872+0000 7fd20bc00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fd20415c770 con 0x7fd2040af830 2024-04-04T06:16:37.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.872+0000 7fd20bc00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd20415fe80 con 0x7fd204160b80 2024-04-04T06:16:37.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.872+0000 7fd20bc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd2040ad120 con 0x7fd20415c390 2024-04-04T06:16:37.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.873+0000 7fd203e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd2040af830 0x7fd204160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:37.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.873+0000 7fd203e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd2040af830 0x7fd204160640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51352/0 (socket says 172.21.15.159:51352) 2024-04-04T06:16:37.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.873+0000 7fd203e00640 1 -- 172.21.15.159:0/1640367920 learned_addr learned my addr 172.21.15.159:0/1640367920 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:37.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.873+0000 7fd209200640 1 --2- 172.21.15.159:0/1640367920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd20415c390 0x7fd204073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:37.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.873+0000 7fd209c00640 1 --2- 172.21.15.159:0/1640367920 >> v2:172.21.15.178:3300/0 conn(0x7fd204160b80 0x7fd2040d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:37.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.873+0000 7fd203e00640 1 -- 172.21.15.159:0/1640367920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd20415c390 msgr2=0x7fd204073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:37.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.873+0000 7fd203e00640 1 --2- 172.21.15.159:0/1640367920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd20415c390 0x7fd204073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.873+0000 7fd203e00640 1 -- 172.21.15.159:0/1640367920 >> v2:172.21.15.178:3300/0 conn(0x7fd204160b80 msgr2=0x7fd2040d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:37.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.873+0000 7fd203e00640 1 --2- 172.21.15.159:0/1640367920 >> v2:172.21.15.178:3300/0 conn(0x7fd204160b80 0x7fd2040d0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.873+0000 7fd203e00640 1 -- 172.21.15.159:0/1640367920 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2040ad400 con 0x7fd2040af830 2024-04-04T06:16:37.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.873+0000 7fd209c00640 1 --2- 172.21.15.159:0/1640367920 >> v2:172.21.15.178:3300/0 conn(0x7fd204160b80 0x7fd2040d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:37.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.874+0000 7fd203e00640 1 --2- 172.21.15.159:0/1640367920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2040af830 0x7fd204160640 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fd20406eaa0 tx=0x7fd1f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a1356df4f21934 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:37.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.875+0000 7fd203400640 1 -- 172.21.15.159:0/1640367920 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0035020 con 0x7fd2040af830 2024-04-04T06:16:37.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.875+0000 7fd203400640 1 -- 172.21.15.159:0/1640367920 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1f0031870 con 0x7fd2040af830 2024-04-04T06:16:37.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.875+0000 7fd203400640 1 -- 172.21.15.159:0/1640367920 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0031c40 con 0x7fd2040af830 2024-04-04T06:16:37.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.875+0000 7fd20bc00640 1 -- 172.21.15.159:0/1640367920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2040af830 msgr2=0x7fd204160640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:37.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.875+0000 7fd20bc00640 1 --2- 172.21.15.159:0/1640367920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2040af830 0x7fd204160640 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fd20406eaa0 tx=0x7fd1f002b910 comp rx=0 tx=0).stop 2024-04-04T06:16:37.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.876+0000 7fd20bc00640 1 -- 172.21.15.159:0/1640367920 shutdown_connections 2024-04-04T06:16:37.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.876+0000 7fd20bc00640 1 --2- 172.21.15.159:0/1640367920 >> v2:172.21.15.178:3300/0 conn(0x7fd204160b80 0x7fd2040d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.876+0000 7fd20bc00640 1 --2- 172.21.15.159:0/1640367920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2040af830 0x7fd204160640 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.876+0000 7fd20bc00640 1 --2- 172.21.15.159:0/1640367920 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd20415c390 0x7fd204073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.876+0000 7fd20bc00640 1 -- 172.21.15.159:0/1640367920 >> 172.21.15.159:0/1640367920 conn(0x7fd20406b990 msgr2=0x7fd20406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:37.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.876+0000 7fd20bc00640 1 -- 172.21.15.159:0/1640367920 shutdown_connections 2024-04-04T06:16:37.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.876+0000 7fd20bc00640 1 -- 172.21.15.159:0/1640367920 wait complete. 2024-04-04T06:16:37.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.877+0000 7fd20bc00640 1 Processor -- start 2024-04-04T06:16:37.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.878+0000 7fd20bc00640 1 -- start start 2024-04-04T06:16:37.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.878+0000 7fd20bc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2040af830 0x7fd2041f4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:37.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.878+0000 7fd20bc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd20415c390 0x7fd20409a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:37.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.878+0000 7fd20bc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd204160b80 0x7fd20409a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:37.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.879+0000 7fd20bc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd2040cf980 con 0x7fd204160b80 2024-04-04T06:16:37.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.879+0000 7fd20bc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd204160120 con 0x7fd2040af830 2024-04-04T06:16:37.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.879+0000 7fd20bc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd2040d0b90 con 0x7fd20415c390 2024-04-04T06:16:37.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.879+0000 7fd209c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd204160b80 0x7fd20409a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:37.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.879+0000 7fd209c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd204160b80 0x7fd20409a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51362/0 (socket says 172.21.15.159:51362) 2024-04-04T06:16:37.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.879+0000 7fd209c00640 1 -- 172.21.15.159:0/3051057236 learned_addr learned my addr 172.21.15.159:0/3051057236 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:37.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.879+0000 7fd203e00640 1 --2- 172.21.15.159:0/3051057236 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd20415c390 0x7fd20409a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:37.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.879+0000 7fd209200640 1 --2- 172.21.15.159:0/3051057236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2040af830 0x7fd2041f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:37.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.879+0000 7fd209c00640 1 -- 172.21.15.159:0/3051057236 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd20415c390 msgr2=0x7fd20409a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:37.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.879+0000 7fd209c00640 1 --2- 172.21.15.159:0/3051057236 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd20415c390 0x7fd20409a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.879+0000 7fd209c00640 1 -- 172.21.15.159:0/3051057236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2040af830 msgr2=0x7fd2041f4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:37.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.879+0000 7fd209c00640 1 --2- 172.21.15.159:0/3051057236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2040af830 0x7fd2041f4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:37.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.879+0000 7fd209c00640 1 -- 172.21.15.159:0/3051057236 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20415e8b0 con 0x7fd204160b80 2024-04-04T06:16:37.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.880+0000 7fd209c00640 1 --2- 172.21.15.159:0/3051057236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd204160b80 0x7fd20409a7e0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00cd80 tx=0x7fd1fc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:37.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.880+0000 7fd200c00640 1 -- 172.21.15.159:0/3051057236 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc003b90 con 0x7fd204160b80 2024-04-04T06:16:37.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.880+0000 7fd200c00640 1 -- 172.21.15.159:0/3051057236 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1fc003d50 con 0x7fd204160b80 2024-04-04T06:16:37.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.880+0000 7fd200c00640 1 -- 172.21.15.159:0/3051057236 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc004260 con 0x7fd204160b80 2024-04-04T06:16:37.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.880+0000 7fd20bc00640 1 -- 172.21.15.159:0/3051057236 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd20415eb90 con 0x7fd204160b80 2024-04-04T06:16:37.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.880+0000 7fd20bc00640 1 -- 172.21.15.159:0/3051057236 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd20415f150 con 0x7fd204160b80 2024-04-04T06:16:37.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.881+0000 7fd20bc00640 1 -- 172.21.15.159:0/3051057236 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2040685c0 con 0x7fd204160b80 2024-04-04T06:16:37.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.882+0000 7fd200c00640 1 -- 172.21.15.159:0/3051057236 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd1fc004400 con 0x7fd204160b80 2024-04-04T06:16:37.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.883+0000 7fd200c00640 1 --2- 172.21.15.159:0/3051057236 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd1e406c8f0 0x7fd1e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:37.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.883+0000 7fd200c00640 1 -- 172.21.15.159:0/3051057236 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7fd1fc08c670 con 0x7fd204160b80 2024-04-04T06:16:37.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.884+0000 7fd209200640 1 --2- 172.21.15.159:0/3051057236 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd1e406c8f0 0x7fd1e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:37.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.885+0000 7fd209200640 1 --2- 172.21.15.159:0/3051057236 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd1e406c8f0 0x7fd1e406edb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800a970 tx=0x7fd1f8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:37.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:37.888+0000 7fd200c00640 1 -- 172.21.15.159:0/3051057236 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd1fc014030 con 0x7fd204160b80 2024-04-04T06:16:38.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.129+0000 7fd20bc00640 1 -- 172.21.15.159:0/3051057236 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd20406f1b0 con 0x7fd204160b80 2024-04-04T06:16:38.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.130+0000 7fd200c00640 1 -- 172.21.15.159:0/3051057236 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5176 (secure 0 0 0) 0x7fd1fc05a990 con 0x7fd204160b80 2024-04-04T06:16:38.125 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:38.125 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":92,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5546,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/40219476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":40219476},{"type":"v1","addr":"172.21.15.178:6837","nonce":40219476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/4060483382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4060483382},{"type":"v1","addr":"172.21.15.178:6836","nonce":4060483382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8187,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1945665739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1945665739},{"type":"v1","addr":"172.21.15.159:6833","nonce":1945665739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8688,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3776862819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3776862819},{"type":"v1","addr":"172.21.15.159:6835","nonce":3776862819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":91,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:15:07.704835+0000","modified":"2024-04-04T06:16:37.034446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-04T06:16:38.126 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 92 2024-04-04T06:16:38.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.133+0000 7fd20bc00640 1 -- 172.21.15.159:0/3051057236 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd1e406c8f0 msgr2=0x7fd1e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:38.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.133+0000 7fd20bc00640 1 --2- 172.21.15.159:0/3051057236 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd1e406c8f0 0x7fd1e406edb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800a970 tx=0x7fd1f8008040 comp rx=0 tx=0).stop 2024-04-04T06:16:38.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.133+0000 7fd20bc00640 1 -- 172.21.15.159:0/3051057236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd204160b80 msgr2=0x7fd20409a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:38.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.133+0000 7fd20bc00640 1 --2- 172.21.15.159:0/3051057236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd204160b80 0x7fd20409a7e0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00cd80 tx=0x7fd1fc00a650 comp rx=0 tx=0).stop 2024-04-04T06:16:38.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.133+0000 7fd20bc00640 1 -- 172.21.15.159:0/3051057236 shutdown_connections 2024-04-04T06:16:38.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.133+0000 7fd20bc00640 1 --2- 172.21.15.159:0/3051057236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd204160b80 0x7fd20409a7e0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.133+0000 7fd20bc00640 1 --2- 172.21.15.159:0/3051057236 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd20415c390 0x7fd20409a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.133+0000 7fd20bc00640 1 --2- 172.21.15.159:0/3051057236 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd1e406c8f0 0x7fd1e406edb0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.133+0000 7fd20bc00640 1 --2- 172.21.15.159:0/3051057236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2040af830 0x7fd2041f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.133+0000 7fd20bc00640 1 -- 172.21.15.159:0/3051057236 >> 172.21.15.159:0/3051057236 conn(0x7fd20406b990 msgr2=0x7fd20406bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:38.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.133+0000 7fd20bc00640 1 -- 172.21.15.159:0/3051057236 shutdown_connections 2024-04-04T06:16:38.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.133+0000 7fd20bc00640 1 -- 172.21.15.159:0/3051057236 wait complete. 2024-04-04T06:16:38.148 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:16:38.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.361+0000 7f5460000640 1 Processor -- start 2024-04-04T06:16:38.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.361+0000 7f5460000640 1 -- start start 2024-04-04T06:16:38.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.362+0000 7f5460000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f54580d1610 0x7f54580c7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:38.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.362+0000 7f5460000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f545815eae0 con 0x7f545815e700 2024-04-04T06:16:38.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.362+0000 7f5460000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54580cfeb0 con 0x7f54580b4530 2024-04-04T06:16:38.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.363+0000 7f5460000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f54580d0190 con 0x7f54580d1610 2024-04-04T06:16:38.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.363+0000 7f545d600640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f545815e700 0x7f54580b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37688/0 (socket says 172.21.15.159:37688) 2024-04-04T06:16:38.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.363+0000 7f545d600640 1 -- 172.21.15.159:0/1493069969 learned_addr learned my addr 172.21.15.159:0/1493069969 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:16:38.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.363+0000 7f545e000640 1 --2- 172.21.15.159:0/1493069969 >> v2:172.21.15.178:3300/0 conn(0x7f54580d1610 0x7f54580c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:38.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.363+0000 7f5457e00640 1 -- 172.21.15.159:0/1493069969 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2783515340 0 0) 0x7f545815eae0 con 0x7f545815e700 2024-04-04T06:16:38.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.363+0000 7f5457e00640 1 -- 172.21.15.159:0/1493069969 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5440003520 con 0x7f545815e700 2024-04-04T06:16:38.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.364+0000 7f5457e00640 1 -- 172.21.15.159:0/1493069969 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4021622179 0 0) 0x7f5440003520 con 0x7f545815e700 2024-04-04T06:16:38.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.364+0000 7f5457e00640 1 -- 172.21.15.159:0/1493069969 >> v2:172.21.15.178:3300/0 conn(0x7f54580d1610 msgr2=0x7f54580c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:38.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.364+0000 7f5457e00640 1 --2- 172.21.15.159:0/1493069969 >> v2:172.21.15.178:3300/0 conn(0x7f54580d1610 0x7f54580c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.364+0000 7f5457e00640 1 -- 172.21.15.159:0/1493069969 >> v1:172.21.15.178:6789/0 conn(0x7f54580b4530 legacy=0x7f54580c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:38.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.364+0000 7f5457e00640 1 -- 172.21.15.159:0/1493069969 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54580d0470 con 0x7f545815e700 2024-04-04T06:16:38.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.364+0000 7f5457e00640 1 -- 172.21.15.159:0/1493069969 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f544c001670 con 0x7f545815e700 2024-04-04T06:16:38.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.364+0000 7f5457e00640 1 -- 172.21.15.159:0/1493069969 >> v1:172.21.15.159:6789/0 conn(0x7f545815e700 legacy=0x7f54580b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:38.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.364+0000 7f5457e00640 1 --2- 172.21.15.159:0/1493069969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54400039b0 0x7f5440003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:38.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.364+0000 7f5457e00640 1 --2- 172.21.15.159:0/1493069969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5440004800 0x7f5440004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:38.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.364+0000 7f5457e00640 1 --2- 172.21.15.159:0/1493069969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54400052e0 0x7f544000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:38.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.364+0000 7f5457e00640 1 -- 172.21.15.159:0/1493069969 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f54580d0190 con 0x7f5440004800 2024-04-04T06:16:38.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.364+0000 7f5457e00640 1 -- 172.21.15.159:0/1493069969 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f544000c190 con 0x7f54400039b0 2024-04-04T06:16:38.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.364+0000 7f5457e00640 1 -- 172.21.15.159:0/1493069969 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f544000c390 con 0x7f54400052e0 2024-04-04T06:16:38.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.364+0000 7f545cc00640 1 --2- 172.21.15.159:0/1493069969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5440004800 0x7f5440004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:38.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.364+0000 7f545cc00640 1 -- 172.21.15.159:0/1493069969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54400052e0 msgr2=0x7f544000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:38.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.364+0000 7f545cc00640 1 --2- 172.21.15.159:0/1493069969 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54400052e0 0x7f544000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.364+0000 7f545cc00640 1 -- 172.21.15.159:0/1493069969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54400039b0 msgr2=0x7f5440003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:38.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.364+0000 7f545cc00640 1 --2- 172.21.15.159:0/1493069969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54400039b0 0x7f5440003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.364+0000 7f545cc00640 1 -- 172.21.15.159:0/1493069969 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5440003520 con 0x7f5440004800 2024-04-04T06:16:38.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.364+0000 7f545e000640 1 -- 172.21.15.159:0/1493069969 reap_dead start 2024-04-04T06:16:38.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.365+0000 7f545cc00640 1 --2- 172.21.15.159:0/1493069969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5440004800 0x7f5440004c80 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f5448003d80 tx=0x7f5440005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe63bfaa93f3a57 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:38.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.366+0000 7f5457e00640 1 -- 172.21.15.159:0/1493069969 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448037070 con 0x7f5440004800 2024-04-04T06:16:38.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.366+0000 7f5457e00640 1 -- 172.21.15.159:0/1493069969 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5448032600 con 0x7f5440004800 2024-04-04T06:16:38.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.366+0000 7f5457e00640 1 -- 172.21.15.159:0/1493069969 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448003980 con 0x7f5440004800 2024-04-04T06:16:38.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.366+0000 7f5460000640 1 -- 172.21.15.159:0/1493069969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5440004800 msgr2=0x7f5440004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:38.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.366+0000 7f5460000640 1 --2- 172.21.15.159:0/1493069969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5440004800 0x7f5440004c80 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f5448003d80 tx=0x7f5440005830 comp rx=0 tx=0).stop 2024-04-04T06:16:38.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.366+0000 7f5460000640 1 -- 172.21.15.159:0/1493069969 shutdown_connections 2024-04-04T06:16:38.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.366+0000 7f5460000640 1 --2- 172.21.15.159:0/1493069969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5440004800 0x7f5440004c80 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.366+0000 7f5460000640 1 -- 172.21.15.159:0/1493069969 >> 172.21.15.159:0/1493069969 conn(0x7f54580d2240 msgr2=0x7f54580d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:38.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.366+0000 7f5460000640 1 -- 172.21.15.159:0/1493069969 shutdown_connections 2024-04-04T06:16:38.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.366+0000 7f5460000640 1 -- 172.21.15.159:0/1493069969 wait complete. 2024-04-04T06:16:38.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.368+0000 7f5460000640 1 Processor -- start 2024-04-04T06:16:38.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.368+0000 7f5460000640 1 -- start start 2024-04-04T06:16:38.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.369+0000 7f5460000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5440004800 0x7f5458151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:38.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.369+0000 7f5460000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5458155de0 0x7f5458158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:38.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.369+0000 7f5460000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54581587b0 0x7f545815fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:38.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.369+0000 7f5460000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f545806d1b0 con 0x7f5458155de0 2024-04-04T06:16:38.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.369+0000 7f5460000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f54580d0a30 con 0x7f5440004800 2024-04-04T06:16:38.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.369+0000 7f5460000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f545806d330 con 0x7f54581587b0 2024-04-04T06:16:38.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.369+0000 7f545cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5458155de0 0x7f5458158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:38.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.369+0000 7f545cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5458155de0 0x7f5458158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51384/0 (socket says 172.21.15.159:51384) 2024-04-04T06:16:38.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.369+0000 7f545cc00640 1 -- 172.21.15.159:0/1399930508 learned_addr learned my addr 172.21.15.159:0/1399930508 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:38.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.370+0000 7f545cc00640 1 -- 172.21.15.159:0/1399930508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54581587b0 msgr2=0x7f545815fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:38.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.370+0000 7f545cc00640 1 --2- 172.21.15.159:0/1399930508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54581587b0 0x7f545815fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.370+0000 7f545cc00640 1 -- 172.21.15.159:0/1399930508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5440004800 msgr2=0x7f5458151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:38.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.370+0000 7f545cc00640 1 --2- 172.21.15.159:0/1399930508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5440004800 0x7f5458151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.370+0000 7f545cc00640 1 -- 172.21.15.159:0/1399930508 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5458156df0 con 0x7f5458155de0 2024-04-04T06:16:38.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.370+0000 7f545cc00640 1 --2- 172.21.15.159:0/1399930508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5458155de0 0x7f5458158270 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f5448003ea0 tx=0x7f54480036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:38.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.370+0000 7f5455600640 1 -- 172.21.15.159:0/1399930508 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448042030 con 0x7f5458155de0 2024-04-04T06:16:38.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.370+0000 7f5455600640 1 -- 172.21.15.159:0/1399930508 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5448032400 con 0x7f5458155de0 2024-04-04T06:16:38.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.370+0000 7f5460000640 1 -- 172.21.15.159:0/1399930508 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5458156b10 con 0x7f5458155de0 2024-04-04T06:16:38.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.370+0000 7f5455600640 1 -- 172.21.15.159:0/1399930508 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448037040 con 0x7f5458155de0 2024-04-04T06:16:38.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.370+0000 7f5460000640 1 -- 172.21.15.159:0/1399930508 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5458155850 con 0x7f5458155de0 2024-04-04T06:16:38.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.372+0000 7f5460000640 1 -- 172.21.15.159:0/1399930508 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54580685c0 con 0x7f5458155de0 2024-04-04T06:16:38.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.372+0000 7f5455600640 1 -- 172.21.15.159:0/1399930508 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5448045070 con 0x7f5458155de0 2024-04-04T06:16:38.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.374+0000 7f5455600640 1 --2- 172.21.15.159:0/1399930508 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f543c06c8f0 0x7f543c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:38.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.374+0000 7f5455600640 1 -- 172.21.15.159:0/1399930508 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7f54480ad680 con 0x7f5458155de0 2024-04-04T06:16:38.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.374+0000 7f545d600640 1 --2- 172.21.15.159:0/1399930508 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f543c06c8f0 0x7f543c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:38.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.375+0000 7f545d600640 1 --2- 172.21.15.159:0/1399930508 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f543c06c8f0 0x7f543c06edb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f544c000f60 tx=0x7f544c0015c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:38.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.378+0000 7f5455600640 1 -- 172.21.15.159:0/1399930508 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f544807b9f0 con 0x7f5458155de0 2024-04-04T06:16:38.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.581+0000 7f5460000640 1 -- 172.21.15.159:0/1399930508 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54580ce600 con 0x7f5458155de0 2024-04-04T06:16:38.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.583+0000 7f5455600640 1 -- 172.21.15.159:0/1399930508 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16886 (secure 0 0 0) 0x7f544807b2f0 con 0x7f5458155de0 2024-04-04T06:16:38.578 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:38.578 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":103,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:16:37.034092+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:15:05.537357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-04T06:15:06.689304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6835/1003751501":"2024-04-05T06:16:37.034044+0000","172.21.15.159:6835/2253383212":"2024-04-05T06:15:59.385487+0000","172.21.15.159:6834/1003751501":"2024-04-05T06:16:37.034044+0000","172.21.15.159:6834/2253383212":"2024-04-05T06:15:59.385487+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:16:38.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.586+0000 7f5460000640 1 -- 172.21.15.159:0/1399930508 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f543c06c8f0 msgr2=0x7f543c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:38.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.586+0000 7f5460000640 1 --2- 172.21.15.159:0/1399930508 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f543c06c8f0 0x7f543c06edb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f544c000f60 tx=0x7f544c0015c0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.586+0000 7f5460000640 1 -- 172.21.15.159:0/1399930508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5458155de0 msgr2=0x7f5458158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:38.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.586+0000 7f5460000640 1 --2- 172.21.15.159:0/1399930508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5458155de0 0x7f5458158270 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f5448003ea0 tx=0x7f54480036f0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.586+0000 7f5460000640 1 -- 172.21.15.159:0/1399930508 shutdown_connections 2024-04-04T06:16:38.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.586+0000 7f5460000640 1 --2- 172.21.15.159:0/1399930508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54581587b0 0x7f545815fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.586+0000 7f5460000640 1 --2- 172.21.15.159:0/1399930508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5458155de0 0x7f5458158270 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.586+0000 7f5460000640 1 --2- 172.21.15.159:0/1399930508 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f543c06c8f0 0x7f543c06edb0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.586+0000 7f5460000640 1 --2- 172.21.15.159:0/1399930508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5440004800 0x7f5458151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.586+0000 7f5460000640 1 -- 172.21.15.159:0/1399930508 >> 172.21.15.159:0/1399930508 conn(0x7f54580d2240 msgr2=0x7f54580c7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:38.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.586+0000 7f5460000640 1 -- 172.21.15.159:0/1399930508 shutdown_connections 2024-04-04T06:16:38.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.587+0000 7f5460000640 1 -- 172.21.15.159:0/1399930508 wait complete. 2024-04-04T06:16:38.599 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:15:05.537357+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:15:06.689304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:16:38.599 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:16:38.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.816+0000 7fceb7800640 1 Processor -- start 2024-04-04T06:16:38.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.816+0000 7fceb7800640 1 -- start start 2024-04-04T06:16:38.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.818+0000 7fceb7800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fceb006c580 0x7fceb006ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:38.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.818+0000 7fceb7800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fceb006ea40 0x7fceb006eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:38.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.818+0000 7fceb7800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceb0067900 con 0x7fceb01590c0 2024-04-04T06:16:38.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.818+0000 7fceb7800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fceb006e5b0 con 0x7fceb006c580 2024-04-04T06:16:38.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.818+0000 7fceb7800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fceb00b47d0 con 0x7fceb006ea40 2024-04-04T06:16:38.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.818+0000 7fceb4e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fceb01590c0 0x7fceb015f8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37700/0 (socket says 172.21.15.159:37700) 2024-04-04T06:16:38.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.818+0000 7fceb4e00640 1 -- 172.21.15.159:0/3851910340 learned_addr learned my addr 172.21.15.159:0/3851910340 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:16:38.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.819+0000 7fceb5800640 1 --2- 172.21.15.159:0/3851910340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fceb006ea40 0x7fceb006eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:38.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.819+0000 7fceafe00640 1 --2- 172.21.15.159:0/3851910340 >> v2:172.21.15.178:3300/0 conn(0x7fceb006c580 0x7fceb006ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:38.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.819+0000 7fceaf400640 1 -- 172.21.15.159:0/3851910340 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2111346871 0 0) 0x7fceb0067900 con 0x7fceb01590c0 2024-04-04T06:16:38.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.819+0000 7fceaf400640 1 -- 172.21.15.159:0/3851910340 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce90003520 con 0x7fceb01590c0 2024-04-04T06:16:38.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.819+0000 7fceaf400640 1 -- 172.21.15.159:0/3851910340 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3439707479 0 0) 0x7fce90003520 con 0x7fceb01590c0 2024-04-04T06:16:38.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.819+0000 7fceaf400640 1 -- 172.21.15.159:0/3851910340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fceb006ea40 msgr2=0x7fceb006eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:38.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.819+0000 7fceaf400640 1 --2- 172.21.15.159:0/3851910340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fceb006ea40 0x7fceb006eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.819+0000 7fceaf400640 1 -- 172.21.15.159:0/3851910340 >> v2:172.21.15.178:3300/0 conn(0x7fceb006c580 msgr2=0x7fceb006ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:38.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.819+0000 7fceaf400640 1 --2- 172.21.15.159:0/3851910340 >> v2:172.21.15.178:3300/0 conn(0x7fceb006c580 0x7fceb006ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.819+0000 7fceaf400640 1 -- 172.21.15.159:0/3851910340 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb00b4ab0 con 0x7fceb01590c0 2024-04-04T06:16:38.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.819+0000 7fceaf400640 1 -- 172.21.15.159:0/3851910340 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fcea0001670 con 0x7fceb01590c0 2024-04-04T06:16:38.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.819+0000 7fceaf400640 1 -- 172.21.15.159:0/3851910340 >> v1:172.21.15.159:6789/0 conn(0x7fceb01590c0 legacy=0x7fceb015f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:38.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.819+0000 7fceaf400640 1 --2- 172.21.15.159:0/3851910340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce900039b0 0x7fce90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:38.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.820+0000 7fceaf400640 1 --2- 172.21.15.159:0/3851910340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce90004800 0x7fce90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:38.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.820+0000 7fceaf400640 1 --2- 172.21.15.159:0/3851910340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fceb006ea40 0x7fce900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:38.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.820+0000 7fceaf400640 1 -- 172.21.15.159:0/3851910340 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fceb006e5b0 con 0x7fce900039b0 2024-04-04T06:16:38.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.820+0000 7fceaf400640 1 -- 172.21.15.159:0/3851910340 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fceb00b47d0 con 0x7fce90004800 2024-04-04T06:16:38.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.820+0000 7fceaf400640 1 -- 172.21.15.159:0/3851910340 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fce9000b9c0 con 0x7fceb006ea40 2024-04-04T06:16:38.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.820+0000 7fceb4e00640 1 --2- 172.21.15.159:0/3851910340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce900039b0 0x7fce90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:38.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.820+0000 7fceb4e00640 1 -- 172.21.15.159:0/3851910340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fceb006ea40 msgr2=0x7fce900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:38.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.820+0000 7fceb4e00640 1 --2- 172.21.15.159:0/3851910340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fceb006ea40 0x7fce900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.820+0000 7fceb4e00640 1 -- 172.21.15.159:0/3851910340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce90004800 msgr2=0x7fce90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:38.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.820+0000 7fceb4e00640 1 --2- 172.21.15.159:0/3851910340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce90004800 0x7fce90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.820+0000 7fceb4e00640 1 -- 172.21.15.159:0/3851910340 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce90003520 con 0x7fce900039b0 2024-04-04T06:16:38.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.821+0000 7fceb4e00640 1 --2- 172.21.15.159:0/3851910340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce900039b0 0x7fce90003dd0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fcea000c210 tx=0x7fce90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba09fcd4e35cee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:38.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.822+0000 7fceaf400640 1 -- 172.21.15.159:0/3851910340 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0007d50 con 0x7fce900039b0 2024-04-04T06:16:38.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.822+0000 7fceaf400640 1 -- 172.21.15.159:0/3851910340 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea0030070 con 0x7fce900039b0 2024-04-04T06:16:38.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.822+0000 7fceaf400640 1 -- 172.21.15.159:0/3851910340 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea002cd80 con 0x7fce900039b0 2024-04-04T06:16:38.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.822+0000 7fceb7800640 1 -- 172.21.15.159:0/3851910340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce900039b0 msgr2=0x7fce90003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:38.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.822+0000 7fceb7800640 1 --2- 172.21.15.159:0/3851910340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce900039b0 0x7fce90003dd0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fcea000c210 tx=0x7fce90005830 comp rx=0 tx=0).stop 2024-04-04T06:16:38.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.822+0000 7fceb5800640 1 -- 172.21.15.159:0/3851910340 reap_dead start 2024-04-04T06:16:38.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.822+0000 7fceb7800640 1 -- 172.21.15.159:0/3851910340 shutdown_connections 2024-04-04T06:16:38.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.822+0000 7fceb7800640 1 -- 172.21.15.159:0/3851910340 >> 172.21.15.159:0/3851910340 conn(0x7fceb0071e60 msgr2=0x7fceb0072270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:38.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.822+0000 7fceb7800640 1 -- 172.21.15.159:0/3851910340 shutdown_connections 2024-04-04T06:16:38.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.822+0000 7fceb7800640 1 -- 172.21.15.159:0/3851910340 wait complete. 2024-04-04T06:16:38.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.824+0000 7fceb7800640 1 Processor -- start 2024-04-04T06:16:38.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.824+0000 7fceb7800640 1 -- start start 2024-04-04T06:16:38.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.825+0000 7fceb7800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce900039b0 0x7fceb014e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:38.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.825+0000 7fceb7800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fceb0152ad0 0x7fceb0150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:38.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.825+0000 7fceb7800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fceb014ea30 0x7fceb014eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:38.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.825+0000 7fceb7800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fceb00b4380 con 0x7fce900039b0 2024-04-04T06:16:38.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.825+0000 7fceb7800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fceb006e240 con 0x7fceb014ea30 2024-04-04T06:16:38.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.825+0000 7fceb7800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fceb00b4500 con 0x7fceb0152ad0 2024-04-04T06:16:38.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.825+0000 7fceb4e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce900039b0 0x7fceb014e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:38.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.825+0000 7fceb4e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce900039b0 0x7fceb014e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51392/0 (socket says 172.21.15.159:51392) 2024-04-04T06:16:38.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.825+0000 7fceb4e00640 1 -- 172.21.15.159:0/4019657326 learned_addr learned my addr 172.21.15.159:0/4019657326 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:38.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.826+0000 7fceb5800640 1 --2- 172.21.15.159:0/4019657326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fceb014ea30 0x7fceb014eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:38.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.826+0000 7fceafe00640 1 --2- 172.21.15.159:0/4019657326 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fceb0152ad0 0x7fceb0150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:38.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.826+0000 7fceb4e00640 1 -- 172.21.15.159:0/4019657326 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fceb0152ad0 msgr2=0x7fceb0150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:38.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.826+0000 7fceb4e00640 1 --2- 172.21.15.159:0/4019657326 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fceb0152ad0 0x7fceb0150d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.826+0000 7fceb4e00640 1 -- 172.21.15.159:0/4019657326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fceb014ea30 msgr2=0x7fceb014eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:38.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.826+0000 7fceb4e00640 1 --2- 172.21.15.159:0/4019657326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fceb014ea30 0x7fceb014eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:38.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.826+0000 7fceb4e00640 1 -- 172.21.15.159:0/4019657326 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb0152ff0 con 0x7fce900039b0 2024-04-04T06:16:38.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.826+0000 7fceb5800640 1 --2- 172.21.15.159:0/4019657326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fceb014ea30 0x7fceb014eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:38.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.826+0000 7fceb4e00640 1 --2- 172.21.15.159:0/4019657326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce900039b0 0x7fceb014e170 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fcea000c330 tx=0x7fcea0001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:38.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.826+0000 7fceacc00640 1 -- 172.21.15.159:0/4019657326 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0001c90 con 0x7fce900039b0 2024-04-04T06:16:38.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.827+0000 7fceacc00640 1 -- 172.21.15.159:0/4019657326 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea0004bf0 con 0x7fce900039b0 2024-04-04T06:16:38.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.827+0000 7fceb7800640 1 -- 172.21.15.159:0/4019657326 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb01532d0 con 0x7fce900039b0 2024-04-04T06:16:38.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.827+0000 7fceacc00640 1 -- 172.21.15.159:0/4019657326 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0007070 con 0x7fce900039b0 2024-04-04T06:16:38.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.827+0000 7fceb7800640 1 -- 172.21.15.159:0/4019657326 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb0153890 con 0x7fce900039b0 2024-04-04T06:16:38.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.828+0000 7fceb7800640 1 -- 172.21.15.159:0/4019657326 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb00685c0 con 0x7fce900039b0 2024-04-04T06:16:38.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.829+0000 7fceacc00640 1 -- 172.21.15.159:0/4019657326 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcea0047020 con 0x7fce900039b0 2024-04-04T06:16:38.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.830+0000 7fceacc00640 1 --2- 172.21.15.159:0/4019657326 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce8c06c920 0x7fce8c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:38.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.830+0000 7fceacc00640 1 -- 172.21.15.159:0/4019657326 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7fcea00acef0 con 0x7fce900039b0 2024-04-04T06:16:38.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.831+0000 7fceafe00640 1 --2- 172.21.15.159:0/4019657326 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce8c06c920 0x7fce8c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:38.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.832+0000 7fceafe00640 1 --2- 172.21.15.159:0/4019657326 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce8c06c920 0x7fce8c06ede0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fceb00b5360 tx=0x7fce9800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:38.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:38.835+0000 7fceacc00640 1 -- 172.21.15.159:0/4019657326 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcea007b1d0 con 0x7fce900039b0 2024-04-04T06:16:39.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.035+0000 7fceb7800640 1 -- 172.21.15.159:0/4019657326 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fceb0067590 con 0x7fce900039b0 2024-04-04T06:16:39.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.036+0000 7fceacc00640 1 -- 172.21.15.159:0/4019657326 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) ==== 93+0+10 (secure 0 0 0) 0x7fcea007aad0 con 0x7fce900039b0 2024-04-04T06:16:39.031 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:16:39.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.038+0000 7fceb7800640 1 -- 172.21.15.159:0/4019657326 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce8c06c920 msgr2=0x7fce8c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:39.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.038+0000 7fceb7800640 1 --2- 172.21.15.159:0/4019657326 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce8c06c920 0x7fce8c06ede0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fceb00b5360 tx=0x7fce9800b040 comp rx=0 tx=0).stop 2024-04-04T06:16:39.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.038+0000 7fceb7800640 1 -- 172.21.15.159:0/4019657326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce900039b0 msgr2=0x7fceb014e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:39.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.038+0000 7fceb7800640 1 --2- 172.21.15.159:0/4019657326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce900039b0 0x7fceb014e170 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fcea000c330 tx=0x7fcea0001a40 comp rx=0 tx=0).stop 2024-04-04T06:16:39.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.039+0000 7fceb7800640 1 -- 172.21.15.159:0/4019657326 shutdown_connections 2024-04-04T06:16:39.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.039+0000 7fceb7800640 1 --2- 172.21.15.159:0/4019657326 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce8c06c920 0x7fce8c06ede0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.039+0000 7fceb7800640 1 --2- 172.21.15.159:0/4019657326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fceb014ea30 0x7fceb014eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.039+0000 7fceb7800640 1 --2- 172.21.15.159:0/4019657326 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fceb0152ad0 0x7fceb0150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.039+0000 7fceb7800640 1 --2- 172.21.15.159:0/4019657326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce900039b0 0x7fceb014e170 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.039+0000 7fceb7800640 1 -- 172.21.15.159:0/4019657326 >> 172.21.15.159:0/4019657326 conn(0x7fceb0071e60 msgr2=0x7fceb0162240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:39.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.039+0000 7fceb7800640 1 -- 172.21.15.159:0/4019657326 shutdown_connections 2024-04-04T06:16:39.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.039+0000 7fceb7800640 1 -- 172.21.15.159:0/4019657326 wait complete. 2024-04-04T06:16:39.054 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:16:39.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.277+0000 7fc476200640 1 Processor -- start 2024-04-04T06:16:39.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.278+0000 7fc476200640 1 -- start start 2024-04-04T06:16:39.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.279+0000 7fc476200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc470153580 0x7fc4700d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:39.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.279+0000 7fc476200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc4700af4b0 0x7fc4701600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:39.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.279+0000 7fc476200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4700afab0 con 0x7fc4700d07c0 2024-04-04T06:16:39.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.279+0000 7fc476200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fc4700d0c30 con 0x7fc4700af4b0 2024-04-04T06:16:39.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.279+0000 7fc476200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc47015cc80 con 0x7fc470153580 2024-04-04T06:16:39.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.279+0000 7fc466a00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fc4700d07c0 0x7fc4700aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37714/0 (socket says 172.21.15.159:37714) 2024-04-04T06:16:39.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.279+0000 7fc466a00640 1 -- 172.21.15.159:0/1445360258 learned_addr learned my addr 172.21.15.159:0/1445360258 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:16:39.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.279+0000 7fc46fe00640 1 --2- 172.21.15.159:0/1445360258 >> v2:172.21.15.159:3300/0 conn(0x7fc4700af4b0 0x7fc4701600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:39.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.279+0000 7fc46ea00640 1 -- 172.21.15.159:0/1445360258 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1093668987 0 0) 0x7fc4700afab0 con 0x7fc4700d07c0 2024-04-04T06:16:39.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.279+0000 7fc46f400640 1 --2- 172.21.15.159:0/1445360258 >> v2:172.21.15.178:3300/0 conn(0x7fc470153580 0x7fc4700d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:39.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.280+0000 7fc46ea00640 1 -- 172.21.15.159:0/1445360258 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc454003520 con 0x7fc4700d07c0 2024-04-04T06:16:39.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.280+0000 7fc46fe00640 1 -- 172.21.15.159:0/1445360258 >> v2:172.21.15.178:3300/0 conn(0x7fc470153580 msgr2=0x7fc4700d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:39.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.280+0000 7fc46fe00640 1 --2- 172.21.15.159:0/1445360258 >> v2:172.21.15.178:3300/0 conn(0x7fc470153580 0x7fc4700d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.280+0000 7fc46fe00640 1 -- 172.21.15.159:0/1445360258 >> v1:172.21.15.159:6789/0 conn(0x7fc4700d07c0 legacy=0x7fc4700aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:39.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.280+0000 7fc46fe00640 1 -- 172.21.15.159:0/1445360258 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47015da00 con 0x7fc4700af4b0 2024-04-04T06:16:39.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.281+0000 7fc46fe00640 1 --2- 172.21.15.159:0/1445360258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4700af4b0 0x7fc4701600d0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fc4700b5270 tx=0x7fc46802be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f5ab7d45c835b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:39.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.281+0000 7fc46ea00640 1 -- 172.21.15.159:0/1445360258 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc468039070 con 0x7fc4700af4b0 2024-04-04T06:16:39.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.281+0000 7fc46ea00640 1 -- 172.21.15.159:0/1445360258 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc46802cbe0 con 0x7fc4700af4b0 2024-04-04T06:16:39.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.281+0000 7fc46ea00640 1 -- 172.21.15.159:0/1445360258 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc468034440 con 0x7fc4700af4b0 2024-04-04T06:16:39.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.282+0000 7fc476200640 1 -- 172.21.15.159:0/1445360258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4700af4b0 msgr2=0x7fc4701600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:39.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.282+0000 7fc476200640 1 --2- 172.21.15.159:0/1445360258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4700af4b0 0x7fc4701600d0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fc4700b5270 tx=0x7fc46802be30 comp rx=0 tx=0).stop 2024-04-04T06:16:39.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.282+0000 7fc476200640 1 -- 172.21.15.159:0/1445360258 shutdown_connections 2024-04-04T06:16:39.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.282+0000 7fc476200640 1 --2- 172.21.15.159:0/1445360258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4700af4b0 0x7fc4701600d0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.282+0000 7fc476200640 1 --2- 172.21.15.159:0/1445360258 >> v2:172.21.15.178:3300/0 conn(0x7fc470153580 0x7fc4700d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.282+0000 7fc476200640 1 -- 172.21.15.159:0/1445360258 >> 172.21.15.159:0/1445360258 conn(0x7fc47006b750 msgr2=0x7fc47006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:39.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.282+0000 7fc476200640 1 -- 172.21.15.159:0/1445360258 shutdown_connections 2024-04-04T06:16:39.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.282+0000 7fc476200640 1 -- 172.21.15.159:0/1445360258 wait complete. 2024-04-04T06:16:39.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.284+0000 7fc476200640 1 Processor -- start 2024-04-04T06:16:39.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.284+0000 7fc476200640 1 -- start start 2024-04-04T06:16:39.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.285+0000 7fc476200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4700af4b0 0x7fc47013fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:39.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.285+0000 7fc476200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4700d07c0 0x7fc4700cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:39.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.285+0000 7fc476200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc470153580 0x7fc47013e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:39.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.285+0000 7fc476200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc47015d4e0 con 0x7fc4700af4b0 2024-04-04T06:16:39.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.285+0000 7fc476200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc47015d200 con 0x7fc470153580 2024-04-04T06:16:39.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.285+0000 7fc476200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc470153960 con 0x7fc4700d07c0 2024-04-04T06:16:39.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.285+0000 7fc46f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4700af4b0 0x7fc47013fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:39.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.285+0000 7fc46f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4700af4b0 0x7fc47013fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51398/0 (socket says 172.21.15.159:51398) 2024-04-04T06:16:39.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.285+0000 7fc46f400640 1 -- 172.21.15.159:0/1837197691 learned_addr learned my addr 172.21.15.159:0/1837197691 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:39.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.286+0000 7fc46f400640 1 -- 172.21.15.159:0/1837197691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4700d07c0 msgr2=0x7fc4700cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:39.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.286+0000 7fc46f400640 1 --2- 172.21.15.159:0/1837197691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4700d07c0 0x7fc4700cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.286+0000 7fc46f400640 1 -- 172.21.15.159:0/1837197691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc470153580 msgr2=0x7fc47013e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:39.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.286+0000 7fc46f400640 1 --2- 172.21.15.159:0/1837197691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc470153580 0x7fc47013e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.286+0000 7fc46f400640 1 -- 172.21.15.159:0/1837197691 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47013ebc0 con 0x7fc4700af4b0 2024-04-04T06:16:39.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.286+0000 7fc46f400640 1 --2- 172.21.15.159:0/1837197691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4700af4b0 0x7fc47013fc60 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fc46000d830 tx=0x7fc46000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:39.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.286+0000 7fc467e00640 1 -- 172.21.15.159:0/1837197691 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46000b6c0 con 0x7fc4700af4b0 2024-04-04T06:16:39.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.286+0000 7fc467e00640 1 -- 172.21.15.159:0/1837197691 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc460019070 con 0x7fc4700af4b0 2024-04-04T06:16:39.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.286+0000 7fc467e00640 1 -- 172.21.15.159:0/1837197691 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46000ba80 con 0x7fc4700af4b0 2024-04-04T06:16:39.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.286+0000 7fc476200640 1 -- 172.21.15.159:0/1837197691 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47013eea0 con 0x7fc4700af4b0 2024-04-04T06:16:39.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.286+0000 7fc476200640 1 -- 172.21.15.159:0/1837197691 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc47013f460 con 0x7fc4700af4b0 2024-04-04T06:16:39.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.287+0000 7fc476200640 1 -- 172.21.15.159:0/1837197691 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4700685c0 con 0x7fc4700af4b0 2024-04-04T06:16:39.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.288+0000 7fc467e00640 1 -- 172.21.15.159:0/1837197691 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc46001e460 con 0x7fc4700af4b0 2024-04-04T06:16:39.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.290+0000 7fc467e00640 1 --2- 172.21.15.159:0/1837197691 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc44c06c8f0 0x7fc44c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:39.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.290+0000 7fc467e00640 1 -- 172.21.15.159:0/1837197691 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7fc46008c380 con 0x7fc4700af4b0 2024-04-04T06:16:39.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.290+0000 7fc466a00640 1 --2- 172.21.15.159:0/1837197691 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc44c06c8f0 0x7fc44c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:39.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.292+0000 7fc466a00640 1 --2- 172.21.15.159:0/1837197691 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc44c06c8f0 0x7fc44c06edb0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc458000f60 tx=0x7fc4580078f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:39.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.294+0000 7fc467e00640 1 -- 172.21.15.159:0/1837197691 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc46005a660 con 0x7fc4700af4b0 2024-04-04T06:16:39.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.494+0000 7fc476200640 1 -- 172.21.15.159:0/1837197691 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc47006bbd0 con 0x7fc4700af4b0 2024-04-04T06:16:39.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.495+0000 7fc467e00640 1 -- 172.21.15.159:0/1837197691 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) ==== 104+0+11 (secure 0 0 0) 0x7fc460059f60 con 0x7fc4700af4b0 2024-04-04T06:16:39.489 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:16:39.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.497+0000 7fc476200640 1 -- 172.21.15.159:0/1837197691 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc44c06c8f0 msgr2=0x7fc44c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:39.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.498+0000 7fc476200640 1 --2- 172.21.15.159:0/1837197691 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc44c06c8f0 0x7fc44c06edb0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc458000f60 tx=0x7fc4580078f0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.498+0000 7fc476200640 1 -- 172.21.15.159:0/1837197691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4700af4b0 msgr2=0x7fc47013fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:39.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.498+0000 7fc476200640 1 --2- 172.21.15.159:0/1837197691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4700af4b0 0x7fc47013fc60 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fc46000d830 tx=0x7fc46000dcf0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.498+0000 7fc476200640 1 -- 172.21.15.159:0/1837197691 shutdown_connections 2024-04-04T06:16:39.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.498+0000 7fc476200640 1 --2- 172.21.15.159:0/1837197691 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc44c06c8f0 0x7fc44c06edb0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.498+0000 7fc476200640 1 --2- 172.21.15.159:0/1837197691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc470153580 0x7fc47013e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.498+0000 7fc476200640 1 --2- 172.21.15.159:0/1837197691 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4700d07c0 0x7fc4700cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.498+0000 7fc476200640 1 --2- 172.21.15.159:0/1837197691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4700af4b0 0x7fc47013fc60 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.498+0000 7fc476200640 1 -- 172.21.15.159:0/1837197691 >> 172.21.15.159:0/1837197691 conn(0x7fc47006b750 msgr2=0x7fc47015e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:39.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.498+0000 7fc476200640 1 -- 172.21.15.159:0/1837197691 shutdown_connections 2024-04-04T06:16:39.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.498+0000 7fc476200640 1 -- 172.21.15.159:0/1837197691 wait complete. 2024-04-04T06:16:39.512 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:16:39.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.737+0000 7f7fbe600640 1 Processor -- start 2024-04-04T06:16:39.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.738+0000 7f7fbe600640 1 -- start start 2024-04-04T06:16:39.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.739+0000 7f7fbe600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7fb80d0280 0x7f7fb80d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:39.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.739+0000 7f7fbe600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fb8141c10 0x7f7fb8071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:39.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.739+0000 7f7fbe600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fb815ec80 con 0x7f7fb8071eb0 2024-04-04T06:16:39.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.739+0000 7f7fbe600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f7fb80d0ba0 con 0x7f7fb80d0280 2024-04-04T06:16:39.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.739+0000 7f7fbe600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7fb815ef20 con 0x7f7fb8141c10 2024-04-04T06:16:39.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.739+0000 7f7fb7400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7fb80d0280 0x7f7fb80d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:39.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.739+0000 7f7fb7400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7fb80d0280 0x7f7fb80d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51402/0 (socket says 172.21.15.159:51402) 2024-04-04T06:16:39.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.739+0000 7f7fb7400640 1 -- 172.21.15.159:0/2960636698 learned_addr learned my addr 172.21.15.159:0/2960636698 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:39.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.740+0000 7f7fb7400640 1 -- 172.21.15.159:0/2960636698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fb8141c10 msgr2=0x7f7fb8071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:39.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.740+0000 7f7fb7400640 1 --2- 172.21.15.159:0/2960636698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fb8141c10 0x7f7fb8071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.740+0000 7f7fb7400640 1 -- 172.21.15.159:0/2960636698 >> v1:172.21.15.178:6789/0 conn(0x7f7fb8071eb0 legacy=0x7f7fb806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:39.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.740+0000 7f7fb7400640 1 -- 172.21.15.159:0/2960636698 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb815f200 con 0x7f7fb80d0280 2024-04-04T06:16:39.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.741+0000 7f7fb7400640 1 --2- 172.21.15.159:0/2960636698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fb80d0280 0x7f7fb80d0660 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8009e50 tx=0x7f7fa802bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62e081b3778391b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:39.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.741+0000 7f7fb6000640 1 -- 172.21.15.159:0/2960636698 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa802caa0 con 0x7f7fb80d0280 2024-04-04T06:16:39.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.741+0000 7f7fb6000640 1 -- 172.21.15.159:0/2960636698 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fa802cc40 con 0x7f7fb80d0280 2024-04-04T06:16:39.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.741+0000 7f7fb6000640 1 -- 172.21.15.159:0/2960636698 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa80343f0 con 0x7f7fb80d0280 2024-04-04T06:16:39.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.742+0000 7f7fbe600640 1 -- 172.21.15.159:0/2960636698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fb80d0280 msgr2=0x7f7fb80d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:39.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.742+0000 7f7fbe600640 1 --2- 172.21.15.159:0/2960636698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fb80d0280 0x7f7fb80d0660 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8009e50 tx=0x7f7fa802bf00 comp rx=0 tx=0).stop 2024-04-04T06:16:39.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.742+0000 7f7fbe600640 1 -- 172.21.15.159:0/2960636698 shutdown_connections 2024-04-04T06:16:39.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.742+0000 7f7fbe600640 1 --2- 172.21.15.159:0/2960636698 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fb8141c10 0x7f7fb8071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.742+0000 7f7fbe600640 1 --2- 172.21.15.159:0/2960636698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fb80d0280 0x7f7fb80d0660 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.742+0000 7f7fbe600640 1 -- 172.21.15.159:0/2960636698 >> 172.21.15.159:0/2960636698 conn(0x7f7fb806b750 msgr2=0x7f7fb806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:39.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.742+0000 7f7fbe600640 1 -- 172.21.15.159:0/2960636698 shutdown_connections 2024-04-04T06:16:39.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.742+0000 7f7fbe600640 1 -- 172.21.15.159:0/2960636698 wait complete. 2024-04-04T06:16:39.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.744+0000 7f7fbe600640 1 Processor -- start 2024-04-04T06:16:39.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.744+0000 7f7fbe600640 1 -- start start 2024-04-04T06:16:39.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.745+0000 7f7fbe600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fb8071eb0 0x7f7fb8145950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:39.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.745+0000 7f7fbe600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fb80d0280 0x7f7fb8156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:39.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.745+0000 7f7fbe600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fb8141c10 0x7f7fb81f7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:39.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.745+0000 7f7fbe600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb8158e90 con 0x7f7fb8141c10 2024-04-04T06:16:39.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.745+0000 7f7fbe600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb815f7c0 con 0x7f7fb80d0280 2024-04-04T06:16:39.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.745+0000 7f7fbe600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7fb81f7b90 con 0x7f7fb8071eb0 2024-04-04T06:16:39.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.745+0000 7f7fb7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fb8141c10 0x7f7fb81f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:39.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.745+0000 7f7fb7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fb8141c10 0x7f7fb81f7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51412/0 (socket says 172.21.15.159:51412) 2024-04-04T06:16:39.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.745+0000 7f7fb7e00640 1 -- 172.21.15.159:0/1777930324 learned_addr learned my addr 172.21.15.159:0/1777930324 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:39.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.745+0000 7f7fb6a00640 1 --2- 172.21.15.159:0/1777930324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fb80d0280 0x7f7fb8156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:39.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.745+0000 7f7fb7400640 1 --2- 172.21.15.159:0/1777930324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fb8071eb0 0x7f7fb8145950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:39.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.745+0000 7f7fb7e00640 1 -- 172.21.15.159:0/1777930324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fb8071eb0 msgr2=0x7f7fb8145950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:39.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.746+0000 7f7fb7e00640 1 --2- 172.21.15.159:0/1777930324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fb8071eb0 0x7f7fb8145950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.746+0000 7f7fb7e00640 1 -- 172.21.15.159:0/1777930324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fb80d0280 msgr2=0x7f7fb8156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:39.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.746+0000 7f7fb7e00640 1 --2- 172.21.15.159:0/1777930324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fb80d0280 0x7f7fb8156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.746+0000 7f7fb7e00640 1 -- 172.21.15.159:0/1777930324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb81448b0 con 0x7f7fb8141c10 2024-04-04T06:16:39.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.746+0000 7f7fb7400640 1 --2- 172.21.15.159:0/1777930324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fb8071eb0 0x7f7fb8145950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:39.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.746+0000 7f7fb7e00640 1 --2- 172.21.15.159:0/1777930324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fb8141c10 0x7f7fb81f7650 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000c390 tx=0x7f7fb000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:39.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.746+0000 7f7faf400640 1 -- 172.21.15.159:0/1777930324 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb0017070 con 0x7f7fb8141c10 2024-04-04T06:16:39.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.746+0000 7f7faf400640 1 -- 172.21.15.159:0/1777930324 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fb00035f0 con 0x7f7fb8141c10 2024-04-04T06:16:39.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.746+0000 7f7fbe600640 1 -- 172.21.15.159:0/1777930324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fb8144b90 con 0x7f7fb8141c10 2024-04-04T06:16:39.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.746+0000 7f7faf400640 1 -- 172.21.15.159:0/1777930324 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb00039e0 con 0x7f7fb8141c10 2024-04-04T06:16:39.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.746+0000 7f7fbe600640 1 -- 172.21.15.159:0/1777930324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fb8145150 con 0x7f7fb8141c10 2024-04-04T06:16:39.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.747+0000 7f7fbe600640 1 -- 172.21.15.159:0/1777930324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fb8158620 con 0x7f7fb8141c10 2024-04-04T06:16:39.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.748+0000 7f7faf400640 1 -- 172.21.15.159:0/1777930324 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7fb001a070 con 0x7f7fb8141c10 2024-04-04T06:16:39.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.749+0000 7f7faf400640 1 --2- 172.21.15.159:0/1777930324 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7f9806c920 0x7f7f9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:39.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.749+0000 7f7faf400640 1 -- 172.21.15.159:0/1777930324 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7f7fb008afe0 con 0x7f7fb8141c10 2024-04-04T06:16:39.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.750+0000 7f7fb7400640 1 --2- 172.21.15.159:0/1777930324 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7f9806c920 0x7f7f9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:39.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.751+0000 7f7fb7400640 1 --2- 172.21.15.159:0/1777930324 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7f9806c920 0x7f7f9806ede0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8035000 tx=0x7f7fa8030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:39.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.754+0000 7f7faf400640 1 -- 172.21.15.159:0/1777930324 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7fb8158620 con 0x7f7fb8141c10 2024-04-04T06:16:39.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.955+0000 7f7fbe600640 1 -- 172.21.15.159:0/1777930324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7fb80685c0 con 0x7f7fb8141c10 2024-04-04T06:16:39.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.956+0000 7f7faf400640 1 -- 172.21.15.159:0/1777930324 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) ==== 100+0+11 (secure 0 0 0) 0x7f7fb005f030 con 0x7f7fb8141c10 2024-04-04T06:16:39.950 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:16:39.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.958+0000 7f7fbe600640 1 -- 172.21.15.159:0/1777930324 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7f9806c920 msgr2=0x7f7f9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:39.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.958+0000 7f7fbe600640 1 --2- 172.21.15.159:0/1777930324 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7f9806c920 0x7f7f9806ede0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8035000 tx=0x7f7fa8030000 comp rx=0 tx=0).stop 2024-04-04T06:16:39.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.958+0000 7f7fbe600640 1 -- 172.21.15.159:0/1777930324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fb8141c10 msgr2=0x7f7fb81f7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:39.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.958+0000 7f7fbe600640 1 --2- 172.21.15.159:0/1777930324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fb8141c10 0x7f7fb81f7650 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000c390 tx=0x7f7fb000c850 comp rx=0 tx=0).stop 2024-04-04T06:16:39.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.958+0000 7f7fbe600640 1 -- 172.21.15.159:0/1777930324 shutdown_connections 2024-04-04T06:16:39.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.958+0000 7f7fbe600640 1 --2- 172.21.15.159:0/1777930324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fb8141c10 0x7f7fb81f7650 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.958+0000 7f7fbe600640 1 --2- 172.21.15.159:0/1777930324 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7f9806c920 0x7f7f9806ede0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.958+0000 7f7fbe600640 1 --2- 172.21.15.159:0/1777930324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fb80d0280 0x7f7fb8156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.958+0000 7f7fbe600640 1 --2- 172.21.15.159:0/1777930324 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7fb8071eb0 0x7f7fb8145950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:39.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.958+0000 7f7fbe600640 1 -- 172.21.15.159:0/1777930324 >> 172.21.15.159:0/1777930324 conn(0x7f7fb806b750 msgr2=0x7f7fb8157af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:39.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.958+0000 7f7fbe600640 1 -- 172.21.15.159:0/1777930324 shutdown_connections 2024-04-04T06:16:39.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:39.958+0000 7f7fbe600640 1 -- 172.21.15.159:0/1777930324 wait complete. 2024-04-04T06:16:39.972 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:16:40.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.182+0000 7fe53ba00640 1 Processor -- start 2024-04-04T06:16:40.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.183+0000 7fe53ba00640 1 -- start start 2024-04-04T06:16:40.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.184+0000 7fe53ba00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe53415d120 0x7fe534162e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:40.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.184+0000 7fe53ba00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe5340b3500 0x7fe5340b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:40.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.184+0000 7fe53ba00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5341633d0 con 0x7fe5340af630 2024-04-04T06:16:40.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.184+0000 7fe53ba00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe5340723c0 con 0x7fe5340b3500 2024-04-04T06:16:40.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.184+0000 7fe53ba00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe534163650 con 0x7fe53415d120 2024-04-04T06:16:40.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.185+0000 7fe52be00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fe5340af630 0x7fe534166b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37730/0 (socket says 172.21.15.159:37730) 2024-04-04T06:16:40.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.185+0000 7fe52be00640 1 -- 172.21.15.159:0/2511005913 learned_addr learned my addr 172.21.15.159:0/2511005913 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:16:40.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.185+0000 7fe539a00640 1 --2- 172.21.15.159:0/2511005913 >> v2:172.21.15.178:3300/0 conn(0x7fe5340b3500 0x7fe5340b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:40.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.185+0000 7fe539000640 1 --2- 172.21.15.159:0/2511005913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe53415d120 0x7fe534162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:40.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.185+0000 7fe533e00640 1 -- 172.21.15.159:0/2511005913 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1417942835 0 0) 0x7fe51c001110 con 0x7fe5340af630 2024-04-04T06:16:40.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.185+0000 7fe533e00640 1 -- 172.21.15.159:0/2511005913 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe518003520 con 0x7fe5340af630 2024-04-04T06:16:40.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.185+0000 7fe533e00640 1 -- 172.21.15.159:0/2511005913 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1004494758 0 0) 0x7fe518003520 con 0x7fe5340af630 2024-04-04T06:16:40.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.185+0000 7fe533e00640 1 -- 172.21.15.159:0/2511005913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe53415d120 msgr2=0x7fe534162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:40.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.185+0000 7fe533e00640 1 --2- 172.21.15.159:0/2511005913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe53415d120 0x7fe534162e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:40.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.185+0000 7fe533e00640 1 -- 172.21.15.159:0/2511005913 >> v2:172.21.15.178:3300/0 conn(0x7fe5340b3500 msgr2=0x7fe5340b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:40.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.185+0000 7fe533e00640 1 --2- 172.21.15.159:0/2511005913 >> v2:172.21.15.178:3300/0 conn(0x7fe5340b3500 0x7fe5340b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:40.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.185+0000 7fe533e00640 1 -- 172.21.15.159:0/2511005913 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe534163930 con 0x7fe5340af630 2024-04-04T06:16:40.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.185+0000 7fe533e00640 1 -- 172.21.15.159:0/2511005913 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fe51c001840 con 0x7fe5340af630 2024-04-04T06:16:40.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.186+0000 7fe533e00640 1 -- 172.21.15.159:0/2511005913 >> v1:172.21.15.159:6789/0 conn(0x7fe5340af630 legacy=0x7fe534166b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:40.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.186+0000 7fe533e00640 1 --2- 172.21.15.159:0/2511005913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe53415d120 0x7fe5180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:40.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.186+0000 7fe533e00640 1 --2- 172.21.15.159:0/2511005913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe518003f30 0x7fe518004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:40.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.186+0000 7fe533e00640 1 --2- 172.21.15.159:0/2511005913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe518004d40 0x7fe5180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:40.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.186+0000 7fe533e00640 1 -- 172.21.15.159:0/2511005913 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe5340723c0 con 0x7fe518004d40 2024-04-04T06:16:40.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.186+0000 7fe533e00640 1 -- 172.21.15.159:0/2511005913 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe534163650 con 0x7fe518003f30 2024-04-04T06:16:40.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.186+0000 7fe533e00640 1 -- 172.21.15.159:0/2511005913 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe51800b9c0 con 0x7fe53415d120 2024-04-04T06:16:40.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.186+0000 7fe539a00640 1 --2- 172.21.15.159:0/2511005913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe518004d40 0x7fe5180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:40.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.186+0000 7fe52be00640 1 --2- 172.21.15.159:0/2511005913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe518003f30 0x7fe518004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:40.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.186+0000 7fe539000640 1 --2- 172.21.15.159:0/2511005913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe53415d120 0x7fe5180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:40.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.186+0000 7fe539a00640 1 -- 172.21.15.159:0/2511005913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe53415d120 msgr2=0x7fe5180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:40.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.186+0000 7fe539a00640 1 --2- 172.21.15.159:0/2511005913 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe53415d120 0x7fe5180039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:40.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.186+0000 7fe539a00640 1 -- 172.21.15.159:0/2511005913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe518003f30 msgr2=0x7fe518004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:40.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.186+0000 7fe539a00640 1 --2- 172.21.15.159:0/2511005913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe518003f30 0x7fe518004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:40.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.186+0000 7fe539a00640 1 -- 172.21.15.159:0/2511005913 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe518003520 con 0x7fe518004d40 2024-04-04T06:16:40.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.188+0000 7fe539a00640 1 --2- 172.21.15.159:0/2511005913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe518004d40 0x7fe5180051c0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009fc0 tx=0x7fe518005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb69008587aad1f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:40.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.188+0000 7fe533e00640 1 -- 172.21.15.159:0/2511005913 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c039070 con 0x7fe518004d40 2024-04-04T06:16:40.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.188+0000 7fe533e00640 1 -- 172.21.15.159:0/2511005913 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe52c02cc40 con 0x7fe518004d40 2024-04-04T06:16:40.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.188+0000 7fe533e00640 1 -- 172.21.15.159:0/2511005913 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c009c10 con 0x7fe518004d40 2024-04-04T06:16:40.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.188+0000 7fe53ba00640 1 -- 172.21.15.159:0/2511005913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe518004d40 msgr2=0x7fe5180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:40.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.188+0000 7fe53ba00640 1 --2- 172.21.15.159:0/2511005913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe518004d40 0x7fe5180051c0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009fc0 tx=0x7fe518005830 comp rx=0 tx=0).stop 2024-04-04T06:16:40.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.188+0000 7fe539a00640 1 -- 172.21.15.159:0/2511005913 reap_dead start 2024-04-04T06:16:40.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.189+0000 7fe53ba00640 1 -- 172.21.15.159:0/2511005913 shutdown_connections 2024-04-04T06:16:40.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.189+0000 7fe53ba00640 1 -- 172.21.15.159:0/2511005913 >> 172.21.15.159:0/2511005913 conn(0x7fe53406b990 msgr2=0x7fe53406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:40.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.189+0000 7fe53ba00640 1 -- 172.21.15.159:0/2511005913 shutdown_connections 2024-04-04T06:16:40.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.189+0000 7fe53ba00640 1 -- 172.21.15.159:0/2511005913 wait complete. 2024-04-04T06:16:40.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.190+0000 7fe53ba00640 1 Processor -- start 2024-04-04T06:16:40.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.191+0000 7fe53ba00640 1 -- start start 2024-04-04T06:16:40.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.191+0000 7fe53ba00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe518004d40 0x7fe5341fbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:40.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.191+0000 7fe53ba00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5340ce230 0x7fe5340ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:40.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.191+0000 7fe53ba00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5340b0790 0x7fe5340b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:40.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.191+0000 7fe53ba00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe534167250 con 0x7fe518004d40 2024-04-04T06:16:40.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.191+0000 7fe53ba00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe534166620 con 0x7fe5340b0790 2024-04-04T06:16:40.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.191+0000 7fe53ba00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe5341673d0 con 0x7fe5340ce230 2024-04-04T06:16:40.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.192+0000 7fe539000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe518004d40 0x7fe5341fbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:40.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.192+0000 7fe539000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe518004d40 0x7fe5341fbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51438/0 (socket says 172.21.15.159:51438) 2024-04-04T06:16:40.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.192+0000 7fe539000640 1 -- 172.21.15.159:0/1319285180 learned_addr learned my addr 172.21.15.159:0/1319285180 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:40.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.192+0000 7fe539000640 1 -- 172.21.15.159:0/1319285180 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5340ce230 msgr2=0x7fe5340ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:40.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.192+0000 7fe539000640 1 --2- 172.21.15.159:0/1319285180 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5340ce230 0x7fe5340ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:40.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.192+0000 7fe539a00640 1 --2- 172.21.15.159:0/1319285180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5340b0790 0x7fe5340b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:40.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.192+0000 7fe539000640 1 -- 172.21.15.159:0/1319285180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5340b0790 msgr2=0x7fe5340b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:40.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.192+0000 7fe539000640 1 --2- 172.21.15.159:0/1319285180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5340b0790 0x7fe5340b0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:40.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.192+0000 7fe539000640 1 -- 172.21.15.159:0/1319285180 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5341f8f50 con 0x7fe518004d40 2024-04-04T06:16:40.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.192+0000 7fe539000640 1 --2- 172.21.15.159:0/1319285180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe518004d40 0x7fe5341fbd70 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fe524011630 tx=0x7fe524011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:40.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.193+0000 7fe531600640 1 -- 172.21.15.159:0/1319285180 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52400f920 con 0x7fe518004d40 2024-04-04T06:16:40.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.193+0000 7fe531600640 1 -- 172.21.15.159:0/1319285180 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe52400fac0 con 0x7fe518004d40 2024-04-04T06:16:40.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.193+0000 7fe531600640 1 -- 172.21.15.159:0/1319285180 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52400fe70 con 0x7fe518004d40 2024-04-04T06:16:40.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.193+0000 7fe53ba00640 1 -- 172.21.15.159:0/1319285180 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5341f9230 con 0x7fe518004d40 2024-04-04T06:16:40.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.193+0000 7fe53ba00640 1 -- 172.21.15.159:0/1319285180 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5341f97f0 con 0x7fe518004d40 2024-04-04T06:16:40.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.194+0000 7fe53ba00640 1 -- 172.21.15.159:0/1319285180 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5340685c0 con 0x7fe518004d40 2024-04-04T06:16:40.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.195+0000 7fe531600640 1 -- 172.21.15.159:0/1319285180 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe524015460 con 0x7fe518004d40 2024-04-04T06:16:40.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.196+0000 7fe531600640 1 --2- 172.21.15.159:0/1319285180 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe51006c8f0 0x7fe51006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:40.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.196+0000 7fe531600640 1 -- 172.21.15.159:0/1319285180 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7fe52408c610 con 0x7fe518004d40 2024-04-04T06:16:40.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.196+0000 7fe52be00640 1 --2- 172.21.15.159:0/1319285180 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe51006c8f0 0x7fe51006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:40.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.197+0000 7fe52be00640 1 --2- 172.21.15.159:0/1319285180 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe51006c8f0 0x7fe51006edb0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe51c000f60 tx=0x7fe51c000ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:40.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.200+0000 7fe531600640 1 -- 172.21.15.159:0/1319285180 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe524019030 con 0x7fe518004d40 2024-04-04T06:16:40.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.437+0000 7fe53ba00640 1 -- 172.21.15.159:0/1319285180 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe53406bf90 con 0x7fe518004d40 2024-04-04T06:16:40.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.438+0000 7fe531600640 1 -- 172.21.15.159:0/1319285180 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5176 (secure 0 0 0) 0x7fe52405a8f0 con 0x7fe518004d40 2024-04-04T06:16:40.433 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:40.433 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":92,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5546,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/40219476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":40219476},{"type":"v1","addr":"172.21.15.178:6837","nonce":40219476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/4060483382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4060483382},{"type":"v1","addr":"172.21.15.178:6836","nonce":4060483382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8187,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1945665739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1945665739},{"type":"v1","addr":"172.21.15.159:6833","nonce":1945665739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8688,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3776862819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3776862819},{"type":"v1","addr":"172.21.15.159:6835","nonce":3776862819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":91,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:15:07.704835+0000","modified":"2024-04-04T06:16:37.034446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-04T06:16:40.434 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 92 2024-04-04T06:16:40.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.441+0000 7fe53ba00640 1 -- 172.21.15.159:0/1319285180 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe51006c8f0 msgr2=0x7fe51006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:40.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.441+0000 7fe53ba00640 1 --2- 172.21.15.159:0/1319285180 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe51006c8f0 0x7fe51006edb0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe51c000f60 tx=0x7fe51c000ed0 comp rx=0 tx=0).stop 2024-04-04T06:16:40.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.441+0000 7fe53ba00640 1 -- 172.21.15.159:0/1319285180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe518004d40 msgr2=0x7fe5341fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:40.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.441+0000 7fe53ba00640 1 --2- 172.21.15.159:0/1319285180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe518004d40 0x7fe5341fbd70 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fe524011630 tx=0x7fe524011af0 comp rx=0 tx=0).stop 2024-04-04T06:16:40.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.441+0000 7fe53ba00640 1 -- 172.21.15.159:0/1319285180 shutdown_connections 2024-04-04T06:16:40.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.441+0000 7fe53ba00640 1 --2- 172.21.15.159:0/1319285180 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe51006c8f0 0x7fe51006edb0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:40.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.441+0000 7fe53ba00640 1 --2- 172.21.15.159:0/1319285180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5340b0790 0x7fe5340b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:40.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.441+0000 7fe53ba00640 1 --2- 172.21.15.159:0/1319285180 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe5340ce230 0x7fe5340ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:40.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.441+0000 7fe53ba00640 1 --2- 172.21.15.159:0/1319285180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe518004d40 0x7fe5341fbd70 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:40.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.441+0000 7fe53ba00640 1 -- 172.21.15.159:0/1319285180 >> 172.21.15.159:0/1319285180 conn(0x7fe53406b990 msgr2=0x7fe534162960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:40.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.441+0000 7fe53ba00640 1 -- 172.21.15.159:0/1319285180 shutdown_connections 2024-04-04T06:16:40.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.441+0000 7fe53ba00640 1 -- 172.21.15.159:0/1319285180 wait complete. 2024-04-04T06:16:40.454 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:16:40.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.666+0000 7f4d45a00640 1 Processor -- start 2024-04-04T06:16:40.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.667+0000 7f4d45a00640 1 -- start start 2024-04-04T06:16:40.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.668+0000 7f4d45a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4d4015c390 0x7f4d4006b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:40.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.668+0000 7f4d45a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4d400cdcc0 0x7f4d400cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:40.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.668+0000 7f4d45a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d400ce1c0 con 0x7f4d4006bab0 2024-04-04T06:16:40.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.668+0000 7f4d45a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f4d4006bf10 con 0x7f4d4015c390 2024-04-04T06:16:40.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.668+0000 7f4d45a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4d400cea60 con 0x7f4d400cdcc0 2024-04-04T06:16:40.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.669+0000 7f4d3fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4d4015c390 0x7f4d4006b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:40.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.669+0000 7f4d3fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4d4015c390 0x7f4d4006b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51452/0 (socket says 172.21.15.159:51452) 2024-04-04T06:16:40.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.669+0000 7f4d3fe00640 1 -- 172.21.15.159:0/3048446376 learned_addr learned my addr 172.21.15.159:0/3048446376 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:40.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.669+0000 7f4d45000640 1 --2- 172.21.15.159:0/3048446376 >> v2:172.21.15.178:3300/0 conn(0x7f4d400cdcc0 0x7f4d400cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:40.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.669+0000 7f4d3fe00640 1 -- 172.21.15.159:0/3048446376 >> v2:172.21.15.178:3300/0 conn(0x7f4d400cdcc0 msgr2=0x7f4d400cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:40.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.669+0000 7f4d3fe00640 1 --2- 172.21.15.159:0/3048446376 >> v2:172.21.15.178:3300/0 conn(0x7f4d400cdcc0 0x7f4d400cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:40.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.669+0000 7f4d3fe00640 1 -- 172.21.15.159:0/3048446376 >> v1:172.21.15.178:6789/0 conn(0x7f4d4006bab0 legacy=0x7f4d400cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:40.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.669+0000 7f4d3fe00640 1 -- 172.21.15.159:0/3048446376 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d400ce4a0 con 0x7f4d4015c390 2024-04-04T06:16:40.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.669+0000 7f4d45000640 1 --2- 172.21.15.159:0/3048446376 >> v2:172.21.15.178:3300/0 conn(0x7f4d400cdcc0 0x7f4d400cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:40.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.670+0000 7f4d3fe00640 1 --2- 172.21.15.159:0/3048446376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d4015c390 0x7f4d4006b570 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f4d40072660 tx=0x7f4d3002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417d04b8678ecef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:40.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.670+0000 7f4d3ea00640 1 -- 172.21.15.159:0/3048446376 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30035020 con 0x7f4d4015c390 2024-04-04T06:16:40.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.671+0000 7f4d3ea00640 1 -- 172.21.15.159:0/3048446376 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d30031870 con 0x7f4d4015c390 2024-04-04T06:16:40.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.671+0000 7f4d3ea00640 1 -- 172.21.15.159:0/3048446376 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30031c40 con 0x7f4d4015c390 2024-04-04T06:16:40.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.671+0000 7f4d45a00640 1 -- 172.21.15.159:0/3048446376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d4015c390 msgr2=0x7f4d4006b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:40.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.671+0000 7f4d45a00640 1 --2- 172.21.15.159:0/3048446376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d4015c390 0x7f4d4006b570 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f4d40072660 tx=0x7f4d3002b910 comp rx=0 tx=0).stop 2024-04-04T06:16:40.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.671+0000 7f4d45a00640 1 -- 172.21.15.159:0/3048446376 shutdown_connections 2024-04-04T06:16:40.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.671+0000 7f4d45a00640 1 --2- 172.21.15.159:0/3048446376 >> v2:172.21.15.178:3300/0 conn(0x7f4d400cdcc0 0x7f4d400cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:40.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.671+0000 7f4d45a00640 1 --2- 172.21.15.159:0/3048446376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d4015c390 0x7f4d4006b570 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:40.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.671+0000 7f4d45a00640 1 -- 172.21.15.159:0/3048446376 >> 172.21.15.159:0/3048446376 conn(0x7f4d4009aa20 msgr2=0x7f4d400b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:40.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.671+0000 7f4d45a00640 1 -- 172.21.15.159:0/3048446376 shutdown_connections 2024-04-04T06:16:40.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.671+0000 7f4d45a00640 1 -- 172.21.15.159:0/3048446376 wait complete. 2024-04-04T06:16:40.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.673+0000 7f4d45a00640 1 Processor -- start 2024-04-04T06:16:40.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.673+0000 7f4d45a00640 1 -- start start 2024-04-04T06:16:40.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.674+0000 7f4d45a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d4006bab0 0x7f4d400b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:40.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.674+0000 7f4d45a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d400cdcc0 0x7f4d401f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:40.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.674+0000 7f4d45a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d4015c390 0x7f4d401f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:40.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.674+0000 7f4d45a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4d400cf940 con 0x7f4d400cdcc0 2024-04-04T06:16:40.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.674+0000 7f4d45a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4d400cef70 con 0x7f4d4015c390 2024-04-04T06:16:40.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.674+0000 7f4d45a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4d400cfac0 con 0x7f4d4006bab0 2024-04-04T06:16:40.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.674+0000 7f4d3f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d400cdcc0 0x7f4d401f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:40.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.674+0000 7f4d3f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d400cdcc0 0x7f4d401f3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51458/0 (socket says 172.21.15.159:51458) 2024-04-04T06:16:40.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.674+0000 7f4d3f400640 1 -- 172.21.15.159:0/1332622624 learned_addr learned my addr 172.21.15.159:0/1332622624 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:40.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.674+0000 7f4d3fe00640 1 --2- 172.21.15.159:0/1332622624 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d4006bab0 0x7f4d400b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:40.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.674+0000 7f4d3f400640 1 -- 172.21.15.159:0/1332622624 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d4006bab0 msgr2=0x7f4d400b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:40.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.674+0000 7f4d45000640 1 --2- 172.21.15.159:0/1332622624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d4015c390 0x7f4d401f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:40.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.674+0000 7f4d3f400640 1 --2- 172.21.15.159:0/1332622624 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d4006bab0 0x7f4d400b6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:40.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.674+0000 7f4d3f400640 1 -- 172.21.15.159:0/1332622624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d4015c390 msgr2=0x7f4d401f3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:40.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.674+0000 7f4d3f400640 1 --2- 172.21.15.159:0/1332622624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d4015c390 0x7f4d401f3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:40.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.675+0000 7f4d3f400640 1 -- 172.21.15.159:0/1332622624 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4006e8f0 con 0x7f4d400cdcc0 2024-04-04T06:16:40.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.675+0000 7f4d3f400640 1 --2- 172.21.15.159:0/1332622624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d400cdcc0 0x7f4d401f3ab0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800c7b0 tx=0x7f4d2800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:40.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.675+0000 7f4d37e00640 1 -- 172.21.15.159:0/1332622624 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28016020 con 0x7f4d400cdcc0 2024-04-04T06:16:40.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.675+0000 7f4d37e00640 1 -- 172.21.15.159:0/1332622624 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d2800aa50 con 0x7f4d400cdcc0 2024-04-04T06:16:40.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.675+0000 7f4d45a00640 1 -- 172.21.15.159:0/1332622624 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d401f45f0 con 0x7f4d400cdcc0 2024-04-04T06:16:40.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.675+0000 7f4d37e00640 1 -- 172.21.15.159:0/1332622624 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2800ae40 con 0x7f4d400cdcc0 2024-04-04T06:16:40.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.675+0000 7f4d45a00640 1 -- 172.21.15.159:0/1332622624 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d400ca2a0 con 0x7f4d400cdcc0 2024-04-04T06:16:40.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.676+0000 7f4d45a00640 1 -- 172.21.15.159:0/1332622624 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d40075940 con 0x7f4d400cdcc0 2024-04-04T06:16:40.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.677+0000 7f4d37e00640 1 -- 172.21.15.159:0/1332622624 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d28019020 con 0x7f4d400cdcc0 2024-04-04T06:16:40.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.678+0000 7f4d37e00640 1 --2- 172.21.15.159:0/1332622624 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4d1c06c8f0 0x7f4d1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:40.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.678+0000 7f4d37e00640 1 -- 172.21.15.159:0/1332622624 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7f4d2808bb90 con 0x7f4d400cdcc0 2024-04-04T06:16:40.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.679+0000 7f4d3fe00640 1 --2- 172.21.15.159:0/1332622624 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4d1c06c8f0 0x7f4d1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:40.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.680+0000 7f4d3fe00640 1 --2- 172.21.15.159:0/1332622624 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4d1c06c8f0 0x7f4d1c06edb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4d30036000 tx=0x7f4d30008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:40.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.683+0000 7f4d37e00640 1 -- 172.21.15.159:0/1332622624 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4d28020030 con 0x7f4d400cdcc0 2024-04-04T06:16:40.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.880+0000 7f4d45a00640 1 -- 172.21.15.159:0/1332622624 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d400d1ae0 con 0x7f4d400cdcc0 2024-04-04T06:16:40.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.883+0000 7f4d37e00640 1 -- 172.21.15.159:0/1332622624 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16886 (secure 0 0 0) 0x7f4d28059fa0 con 0x7f4d400cdcc0 2024-04-04T06:16:40.878 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:40.879 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":103,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:16:37.034092+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:15:05.537357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-04T06:15:06.689304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6835/1003751501":"2024-04-05T06:16:37.034044+0000","172.21.15.159:6835/2253383212":"2024-04-05T06:15:59.385487+0000","172.21.15.159:6834/1003751501":"2024-04-05T06:16:37.034044+0000","172.21.15.159:6834/2253383212":"2024-04-05T06:15:59.385487+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:16:40.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.886+0000 7f4d45a00640 1 -- 172.21.15.159:0/1332622624 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4d1c06c8f0 msgr2=0x7f4d1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:40.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.886+0000 7f4d45a00640 1 --2- 172.21.15.159:0/1332622624 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4d1c06c8f0 0x7f4d1c06edb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4d30036000 tx=0x7f4d30008030 comp rx=0 tx=0).stop 2024-04-04T06:16:40.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.886+0000 7f4d45a00640 1 -- 172.21.15.159:0/1332622624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d400cdcc0 msgr2=0x7f4d401f3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:40.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.887+0000 7f4d45a00640 1 --2- 172.21.15.159:0/1332622624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d400cdcc0 0x7f4d401f3ab0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800c7b0 tx=0x7f4d2800cc70 comp rx=0 tx=0).stop 2024-04-04T06:16:40.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.887+0000 7f4d45a00640 1 -- 172.21.15.159:0/1332622624 shutdown_connections 2024-04-04T06:16:40.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.887+0000 7f4d45a00640 1 --2- 172.21.15.159:0/1332622624 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4d1c06c8f0 0x7f4d1c06edb0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:40.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.887+0000 7f4d45a00640 1 --2- 172.21.15.159:0/1332622624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d4015c390 0x7f4d401f3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:40.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.887+0000 7f4d45a00640 1 --2- 172.21.15.159:0/1332622624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d400cdcc0 0x7f4d401f3ab0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:40.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.887+0000 7f4d45a00640 1 --2- 172.21.15.159:0/1332622624 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d4006bab0 0x7f4d400b6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:40.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.887+0000 7f4d45a00640 1 -- 172.21.15.159:0/1332622624 >> 172.21.15.159:0/1332622624 conn(0x7f4d4009aa20 msgr2=0x7f4d400b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:40.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.887+0000 7f4d45a00640 1 -- 172.21.15.159:0/1332622624 shutdown_connections 2024-04-04T06:16:40.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:40.887+0000 7f4d45a00640 1 -- 172.21.15.159:0/1332622624 wait complete. 2024-04-04T06:16:40.900 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-04T06:16:40.901 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:16:41.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.112+0000 7fea63c00640 1 Processor -- start 2024-04-04T06:16:41.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.112+0000 7fea63c00640 1 -- start start 2024-04-04T06:16:41.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.114+0000 7fea63c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea5c15c390 0x7fea5c073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:41.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.114+0000 7fea63c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fea5c0d0650 0x7fea5c0d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:41.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.114+0000 7fea63c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea5c15fe80 con 0x7fea5c0af830 2024-04-04T06:16:41.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.114+0000 7fea63c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fea5c0d1000 con 0x7fea5c0d0650 2024-04-04T06:16:41.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.114+0000 7fea63c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea5c15c770 con 0x7fea5c15c390 2024-04-04T06:16:41.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.114+0000 7fea61c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fea5c0d0650 0x7fea5c0d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:41.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.114+0000 7fea61200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea5c15c390 0x7fea5c073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:41.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.115+0000 7fea61c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fea5c0d0650 0x7fea5c0d0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:52942/0 (socket says 172.21.15.159:52942) 2024-04-04T06:16:41.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.115+0000 7fea61c00640 1 -- 172.21.15.159:0/879394615 learned_addr learned my addr 172.21.15.159:0/879394615 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:41.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.115+0000 7fea61200640 1 -- 172.21.15.159:0/879394615 >> v2:172.21.15.178:3300/0 conn(0x7fea5c0d0650 msgr2=0x7fea5c0d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:41.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.115+0000 7fea61200640 1 --2- 172.21.15.159:0/879394615 >> v2:172.21.15.178:3300/0 conn(0x7fea5c0d0650 0x7fea5c0d0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:41.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.115+0000 7fea61200640 1 -- 172.21.15.159:0/879394615 >> v1:172.21.15.178:6789/0 conn(0x7fea5c0af830 legacy=0x7fea5c160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:41.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.115+0000 7fea61200640 1 -- 172.21.15.159:0/879394615 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c0ad120 con 0x7fea5c15c390 2024-04-04T06:16:41.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.115+0000 7fea61c00640 1 --2- 172.21.15.159:0/879394615 >> v2:172.21.15.178:3300/0 conn(0x7fea5c0d0650 0x7fea5c0d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:16:41.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.117+0000 7fea61200640 1 --2- 172.21.15.159:0/879394615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea5c15c390 0x7fea5c073790 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0097a0 tx=0x7fea4c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93b8912725a34d57 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:41.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.117+0000 7fea5be00640 1 -- 172.21.15.159:0/879394615 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c035020 con 0x7fea5c15c390 2024-04-04T06:16:41.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.118+0000 7fea5be00640 1 -- 172.21.15.159:0/879394615 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea4c031830 con 0x7fea5c15c390 2024-04-04T06:16:41.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.118+0000 7fea5be00640 1 -- 172.21.15.159:0/879394615 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c031c00 con 0x7fea5c15c390 2024-04-04T06:16:41.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.118+0000 7fea63c00640 1 -- 172.21.15.159:0/879394615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea5c15c390 msgr2=0x7fea5c073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:41.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.118+0000 7fea63c00640 1 --2- 172.21.15.159:0/879394615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea5c15c390 0x7fea5c073790 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0097a0 tx=0x7fea4c02b8d0 comp rx=0 tx=0).stop 2024-04-04T06:16:41.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.118+0000 7fea63c00640 1 -- 172.21.15.159:0/879394615 shutdown_connections 2024-04-04T06:16:41.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.118+0000 7fea63c00640 1 --2- 172.21.15.159:0/879394615 >> v2:172.21.15.178:3300/0 conn(0x7fea5c0d0650 0x7fea5c0d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:41.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.118+0000 7fea63c00640 1 --2- 172.21.15.159:0/879394615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea5c15c390 0x7fea5c073790 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:41.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.118+0000 7fea63c00640 1 -- 172.21.15.159:0/879394615 >> 172.21.15.159:0/879394615 conn(0x7fea5c06b990 msgr2=0x7fea5c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:41.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.118+0000 7fea63c00640 1 -- 172.21.15.159:0/879394615 shutdown_connections 2024-04-04T06:16:41.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.118+0000 7fea63c00640 1 -- 172.21.15.159:0/879394615 wait complete. 2024-04-04T06:16:41.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.120+0000 7fea63c00640 1 Processor -- start 2024-04-04T06:16:41.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.121+0000 7fea63c00640 1 -- start start 2024-04-04T06:16:41.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.122+0000 7fea63c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea5c0af830 0x7fea5c09c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:41.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.122+0000 7fea63c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c0d0650 0x7fea5c1f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:41.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.122+0000 7fea63c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea5c15c390 0x7fea5c09a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:41.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.122+0000 7fea63c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c160120 con 0x7fea5c0af830 2024-04-04T06:16:41.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.122+0000 7fea63c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c0ad6e0 con 0x7fea5c0d0650 2024-04-04T06:16:41.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.122+0000 7fea63c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea5c160d50 con 0x7fea5c15c390 2024-04-04T06:16:41.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.122+0000 7fea61200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea5c0af830 0x7fea5c09c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:41.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.122+0000 7fea61200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea5c0af830 0x7fea5c09c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51468/0 (socket says 172.21.15.159:51468) 2024-04-04T06:16:41.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.122+0000 7fea61200640 1 -- 172.21.15.159:0/2297102179 learned_addr learned my addr 172.21.15.159:0/2297102179 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:41.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.122+0000 7fea53e00640 1 --2- 172.21.15.159:0/2297102179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c0d0650 0x7fea5c1f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:41.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.123+0000 7fea61c00640 1 --2- 172.21.15.159:0/2297102179 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea5c15c390 0x7fea5c09a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:41.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.123+0000 7fea61200640 1 -- 172.21.15.159:0/2297102179 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea5c15c390 msgr2=0x7fea5c09a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:41.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.123+0000 7fea61200640 1 --2- 172.21.15.159:0/2297102179 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea5c15c390 0x7fea5c09a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:41.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.123+0000 7fea61200640 1 -- 172.21.15.159:0/2297102179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c0d0650 msgr2=0x7fea5c1f2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:41.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.123+0000 7fea61200640 1 --2- 172.21.15.159:0/2297102179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c0d0650 0x7fea5c1f2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:41.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.123+0000 7fea61200640 1 -- 172.21.15.159:0/2297102179 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c09c810 con 0x7fea5c0af830 2024-04-04T06:16:41.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.123+0000 7fea53e00640 1 --2- 172.21.15.159:0/2297102179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c0d0650 0x7fea5c1f2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:41.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.123+0000 7fea61200640 1 --2- 172.21.15.159:0/2297102179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea5c0af830 0x7fea5c09c2d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fea4c02bee0 tx=0x7fea4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:41.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.123+0000 7fea59600640 1 -- 172.21.15.159:0/2297102179 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c035040 con 0x7fea5c0af830 2024-04-04T06:16:41.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.123+0000 7fea59600640 1 -- 172.21.15.159:0/2297102179 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea4c03b920 con 0x7fea5c0af830 2024-04-04T06:16:41.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.123+0000 7fea63c00640 1 -- 172.21.15.159:0/2297102179 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5c148540 con 0x7fea5c0af830 2024-04-04T06:16:41.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.123+0000 7fea59600640 1 -- 172.21.15.159:0/2297102179 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c03bd10 con 0x7fea5c0af830 2024-04-04T06:16:41.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.123+0000 7fea63c00640 1 -- 172.21.15.159:0/2297102179 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea5c159b10 con 0x7fea5c0af830 2024-04-04T06:16:41.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.125+0000 7fea63c00640 1 -- 172.21.15.159:0/2297102179 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea5c0685c0 con 0x7fea5c0af830 2024-04-04T06:16:41.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.125+0000 7fea59600640 1 -- 172.21.15.159:0/2297102179 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea4c038020 con 0x7fea5c0af830 2024-04-04T06:16:41.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.126+0000 7fea59600640 1 --2- 172.21.15.159:0/2297102179 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea3806c920 0x7fea3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:41.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.126+0000 7fea59600640 1 -- 172.21.15.159:0/2297102179 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7fea4c0adcb0 con 0x7fea5c0af830 2024-04-04T06:16:41.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.127+0000 7fea53e00640 1 --2- 172.21.15.159:0/2297102179 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea3806c920 0x7fea3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:41.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.128+0000 7fea53e00640 1 --2- 172.21.15.159:0/2297102179 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea3806c920 0x7fea3806ede0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fea440015e0 tx=0x7fea44009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:41.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.131+0000 7fea59600640 1 -- 172.21.15.159:0/2297102179 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fea4c07c040 con 0x7fea5c0af830 2024-04-04T06:16:41.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.352+0000 7fea63c00640 1 -- 172.21.15.159:0/2297102179 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea5c0cf690 con 0x7fea5c0af830 2024-04-04T06:16:41.348 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.353+0000 7fea59600640 1 -- 172.21.15.159:0/2297102179 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5176 (secure 0 0 0) 0x7fea4c031800 con 0x7fea5c0af830 2024-04-04T06:16:41.348 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:41.348 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":92,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5546,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/40219476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":40219476},{"type":"v1","addr":"172.21.15.178:6837","nonce":40219476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/4060483382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4060483382},{"type":"v1","addr":"172.21.15.178:6836","nonce":4060483382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8187,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1945665739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1945665739},{"type":"v1","addr":"172.21.15.159:6833","nonce":1945665739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8688,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3776862819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3776862819},{"type":"v1","addr":"172.21.15.159:6835","nonce":3776862819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":91,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:15:07.704835+0000","modified":"2024-04-04T06:16:37.034446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-04T06:16:41.349 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 92 2024-04-04T06:16:41.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.356+0000 7fea63c00640 1 -- 172.21.15.159:0/2297102179 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea3806c920 msgr2=0x7fea3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:41.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.356+0000 7fea63c00640 1 --2- 172.21.15.159:0/2297102179 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea3806c920 0x7fea3806ede0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fea440015e0 tx=0x7fea44009040 comp rx=0 tx=0).stop 2024-04-04T06:16:41.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.356+0000 7fea63c00640 1 -- 172.21.15.159:0/2297102179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea5c0af830 msgr2=0x7fea5c09c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:41.352 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.356+0000 7fea63c00640 1 --2- 172.21.15.159:0/2297102179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea5c0af830 0x7fea5c09c2d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fea4c02bee0 tx=0x7fea4c0046c0 comp rx=0 tx=0).stop 2024-04-04T06:16:41.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.356+0000 7fea63c00640 1 -- 172.21.15.159:0/2297102179 shutdown_connections 2024-04-04T06:16:41.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.356+0000 7fea63c00640 1 --2- 172.21.15.159:0/2297102179 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea5c15c390 0x7fea5c09a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:41.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.356+0000 7fea63c00640 1 --2- 172.21.15.159:0/2297102179 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fea3806c920 0x7fea3806ede0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:41.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.356+0000 7fea63c00640 1 --2- 172.21.15.159:0/2297102179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c0d0650 0x7fea5c1f2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:41.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.356+0000 7fea63c00640 1 --2- 172.21.15.159:0/2297102179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea5c0af830 0x7fea5c09c2d0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:41.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.356+0000 7fea63c00640 1 -- 172.21.15.159:0/2297102179 >> 172.21.15.159:0/2297102179 conn(0x7fea5c06b990 msgr2=0x7fea5c0b1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:41.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.356+0000 7fea63c00640 1 -- 172.21.15.159:0/2297102179 shutdown_connections 2024-04-04T06:16:41.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.357+0000 7fea63c00640 1 -- 172.21.15.159:0/2297102179 wait complete. 2024-04-04T06:16:41.369 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:16:41.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.582+0000 7f3798000640 1 Processor -- start 2024-04-04T06:16:41.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.582+0000 7f3798000640 1 -- start start 2024-04-04T06:16:41.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.583+0000 7f3798000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37900af830 0x7f3790073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:41.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.583+0000 7f3798000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f37900d0650 0x7f37900d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:41.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.583+0000 7f3798000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f379015fe80 con 0x7f379015c390 2024-04-04T06:16:41.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.583+0000 7f3798000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f379015c770 con 0x7f37900d0650 2024-04-04T06:16:41.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.583+0000 7f3798000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f37900ad120 con 0x7f37900af830 2024-04-04T06:16:41.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.584+0000 7f3795600640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f379015c390 0x7f3790160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37734/0 (socket says 172.21.15.159:37734) 2024-04-04T06:16:41.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.584+0000 7f3795600640 1 -- 172.21.15.159:0/693634113 learned_addr learned my addr 172.21.15.159:0/693634113 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:16:41.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.584+0000 7f378fe00640 1 -- 172.21.15.159:0/693634113 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1526193608 0 0) 0x7f379015fe80 con 0x7f379015c390 2024-04-04T06:16:41.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.584+0000 7f3794c00640 1 --2- 172.21.15.159:0/693634113 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37900af830 0x7f3790073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:41.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.584+0000 7f378fe00640 1 -- 172.21.15.159:0/693634113 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3778003520 con 0x7f379015c390 2024-04-04T06:16:41.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.584+0000 7f3796000640 1 --2- 172.21.15.159:0/693634113 >> v2:172.21.15.178:3300/0 conn(0x7f37900d0650 0x7f37900d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:41.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.584+0000 7f378fe00640 1 -- 172.21.15.159:0/693634113 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1493369109 0 0) 0x7f3778003520 con 0x7f379015c390 2024-04-04T06:16:41.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.584+0000 7f378fe00640 1 -- 172.21.15.159:0/693634113 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37900af830 msgr2=0x7f3790073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:41.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.584+0000 7f378fe00640 1 --2- 172.21.15.159:0/693634113 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37900af830 0x7f3790073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:41.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.584+0000 7f378fe00640 1 -- 172.21.15.159:0/693634113 >> v2:172.21.15.178:3300/0 conn(0x7f37900d0650 msgr2=0x7f37900d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:41.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.584+0000 7f378fe00640 1 --2- 172.21.15.159:0/693634113 >> v2:172.21.15.178:3300/0 conn(0x7f37900d0650 0x7f37900d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:41.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.584+0000 7f378fe00640 1 -- 172.21.15.159:0/693634113 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37900ad400 con 0x7f379015c390 2024-04-04T06:16:41.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.584+0000 7f378fe00640 1 -- 172.21.15.159:0/693634113 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f3784001670 con 0x7f379015c390 2024-04-04T06:16:41.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.584+0000 7f3796000640 1 --2- 172.21.15.159:0/693634113 >> v2:172.21.15.178:3300/0 conn(0x7f37900d0650 0x7f37900d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:16:41.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.585+0000 7f378fe00640 1 -- 172.21.15.159:0/693634113 >> v1:172.21.15.159:6789/0 conn(0x7f379015c390 legacy=0x7f3790160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:41.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.585+0000 7f378fe00640 1 --2- 172.21.15.159:0/693634113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37780039b0 0x7f3778003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:41.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.585+0000 7f378fe00640 1 --2- 172.21.15.159:0/693634113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3778004800 0x7f3778004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:41.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.585+0000 7f378fe00640 1 --2- 172.21.15.159:0/693634113 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37900af830 0x7f37780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:41.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.585+0000 7f378fe00640 1 -- 172.21.15.159:0/693634113 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f37900ad120 con 0x7f3778004800 2024-04-04T06:16:41.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.585+0000 7f378fe00640 1 -- 172.21.15.159:0/693634113 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f377800b9c0 con 0x7f37780039b0 2024-04-04T06:16:41.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.585+0000 7f378fe00640 1 -- 172.21.15.159:0/693634113 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f377800bb40 con 0x7f37900af830 2024-04-04T06:16:41.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.585+0000 7f3794c00640 1 --2- 172.21.15.159:0/693634113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3778004800 0x7f3778004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:41.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.585+0000 7f3795600640 1 --2- 172.21.15.159:0/693634113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37780039b0 0x7f3778003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:41.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.585+0000 7f3794c00640 1 -- 172.21.15.159:0/693634113 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37900af830 msgr2=0x7f37780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:41.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.585+0000 7f3796000640 1 --2- 172.21.15.159:0/693634113 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37900af830 0x7f37780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:41.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.585+0000 7f3794c00640 1 --2- 172.21.15.159:0/693634113 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f37900af830 0x7f37780051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:41.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.585+0000 7f3794c00640 1 -- 172.21.15.159:0/693634113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37780039b0 msgr2=0x7f3778003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:41.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.585+0000 7f3794c00640 1 --2- 172.21.15.159:0/693634113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37780039b0 0x7f3778003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:41.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.585+0000 7f3794c00640 1 -- 172.21.15.159:0/693634113 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3778003520 con 0x7f3778004800 2024-04-04T06:16:41.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.585+0000 7f3795600640 1 --2- 172.21.15.159:0/693634113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37780039b0 0x7f3778003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:41.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.586+0000 7f3794c00640 1 --2- 172.21.15.159:0/693634113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3778004800 0x7f3778004c80 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f377c009ac0 tx=0x7f3778005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c969507c9ca45893 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:41.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.587+0000 7f378fe00640 1 -- 172.21.15.159:0/693634113 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c039070 con 0x7f3778004800 2024-04-04T06:16:41.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.587+0000 7f378fe00640 1 -- 172.21.15.159:0/693634113 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f377c02c6b0 con 0x7f3778004800 2024-04-04T06:16:41.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.587+0000 7f378fe00640 1 -- 172.21.15.159:0/693634113 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c009690 con 0x7f3778004800 2024-04-04T06:16:41.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.587+0000 7f3798000640 1 -- 172.21.15.159:0/693634113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3778004800 msgr2=0x7f3778004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:41.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.587+0000 7f3798000640 1 --2- 172.21.15.159:0/693634113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3778004800 0x7f3778004c80 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f377c009ac0 tx=0x7f3778005830 comp rx=0 tx=0).stop 2024-04-04T06:16:41.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.587+0000 7f3796000640 1 -- 172.21.15.159:0/693634113 reap_dead start 2024-04-04T06:16:41.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.587+0000 7f3798000640 1 -- 172.21.15.159:0/693634113 shutdown_connections 2024-04-04T06:16:41.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.587+0000 7f3798000640 1 -- 172.21.15.159:0/693634113 >> 172.21.15.159:0/693634113 conn(0x7f379006b990 msgr2=0x7f379006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:41.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.587+0000 7f3798000640 1 -- 172.21.15.159:0/693634113 shutdown_connections 2024-04-04T06:16:41.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.587+0000 7f3798000640 1 -- 172.21.15.159:0/693634113 wait complete. 2024-04-04T06:16:41.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.589+0000 7f3798000640 1 Processor -- start 2024-04-04T06:16:41.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.589+0000 7f3798000640 1 -- start start 2024-04-04T06:16:41.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.590+0000 7f3798000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3778004800 0x7f3790151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:41.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.590+0000 7f3798000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37901e7c40 0x7f3790152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:41.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.590+0000 7f3798000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37901e65f0 0x7f37900b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:41.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.590+0000 7f3798000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3790160d50 con 0x7f37901e65f0 2024-04-04T06:16:41.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.590+0000 7f3798000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3790160120 con 0x7f37901e7c40 2024-04-04T06:16:41.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.590+0000 7f3798000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f37900cf980 con 0x7f3778004800 2024-04-04T06:16:41.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.591+0000 7f3796000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37901e65f0 0x7f37900b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:41.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.591+0000 7f3796000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37901e65f0 0x7f37900b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51480/0 (socket says 172.21.15.159:51480) 2024-04-04T06:16:41.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.591+0000 7f3796000640 1 -- 172.21.15.159:0/1342355908 learned_addr learned my addr 172.21.15.159:0/1342355908 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:41.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.591+0000 7f3794c00640 1 --2- 172.21.15.159:0/1342355908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37901e7c40 0x7f3790152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:41.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.591+0000 7f3795600640 1 --2- 172.21.15.159:0/1342355908 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3778004800 0x7f3790151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:41.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.591+0000 7f3796000640 1 -- 172.21.15.159:0/1342355908 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3778004800 msgr2=0x7f3790151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:41.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.591+0000 7f3796000640 1 --2- 172.21.15.159:0/1342355908 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3778004800 0x7f3790151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:41.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.591+0000 7f3796000640 1 -- 172.21.15.159:0/1342355908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37901e7c40 msgr2=0x7f3790152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:41.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.591+0000 7f3796000640 1 --2- 172.21.15.159:0/1342355908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37901e7c40 0x7f3790152bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:41.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.591+0000 7f3796000640 1 -- 172.21.15.159:0/1342355908 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379013e440 con 0x7f37901e65f0 2024-04-04T06:16:41.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.591+0000 7f3796000640 1 --2- 172.21.15.159:0/1342355908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37901e65f0 0x7f37900b0480 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f379006eee0 tx=0x7f3788004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:41.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.592+0000 7f378d600640 1 -- 172.21.15.159:0/1342355908 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788019070 con 0x7f37901e65f0 2024-04-04T06:16:41.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.592+0000 7f378d600640 1 -- 172.21.15.159:0/1342355908 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3788007c70 con 0x7f37901e65f0 2024-04-04T06:16:41.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.592+0000 7f378d600640 1 -- 172.21.15.159:0/1342355908 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788014640 con 0x7f37901e65f0 2024-04-04T06:16:41.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.592+0000 7f3798000640 1 -- 172.21.15.159:0/1342355908 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379013e720 con 0x7f37901e65f0 2024-04-04T06:16:41.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.592+0000 7f3798000640 1 -- 172.21.15.159:0/1342355908 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f379013ece0 con 0x7f37901e65f0 2024-04-04T06:16:41.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.593+0000 7f3798000640 1 -- 172.21.15.159:0/1342355908 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37900685c0 con 0x7f37901e65f0 2024-04-04T06:16:41.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.594+0000 7f378d600640 1 -- 172.21.15.159:0/1342355908 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f378801c070 con 0x7f37901e65f0 2024-04-04T06:16:41.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.595+0000 7f378d600640 1 --2- 172.21.15.159:0/1342355908 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f377006c8f0 0x7f377006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:41.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.595+0000 7f378d600640 1 -- 172.21.15.159:0/1342355908 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7f378808b7a0 con 0x7f37901e65f0 2024-04-04T06:16:41.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.595+0000 7f3795600640 1 --2- 172.21.15.159:0/1342355908 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f377006c8f0 0x7f377006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:41.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.596+0000 7f3795600640 1 --2- 172.21.15.159:0/1342355908 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f377006c8f0 0x7f377006edb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3784000f60 tx=0x7f3784009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:41.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.599+0000 7f378d600640 1 -- 172.21.15.159:0/1342355908 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3788007e20 con 0x7f37901e65f0 2024-04-04T06:16:41.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.843+0000 7f3798000640 1 -- 172.21.15.159:0/1342355908 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f379006f1b0 con 0x7f37901e65f0 2024-04-04T06:16:41.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.844+0000 7f378d600640 1 -- 172.21.15.159:0/1342355908 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5176 (secure 0 0 0) 0x7f3788059a80 con 0x7f37901e65f0 2024-04-04T06:16:41.839 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:41.839 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":92,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5546,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/40219476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":40219476},{"type":"v1","addr":"172.21.15.178:6837","nonce":40219476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/4060483382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4060483382},{"type":"v1","addr":"172.21.15.178:6836","nonce":4060483382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8187,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1945665739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1945665739},{"type":"v1","addr":"172.21.15.159:6833","nonce":1945665739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8688,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3776862819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3776862819},{"type":"v1","addr":"172.21.15.159:6835","nonce":3776862819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":91,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:15:07.704835+0000","modified":"2024-04-04T06:16:37.034446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-04T06:16:41.839 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 92 2024-04-04T06:16:41.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.846+0000 7f3798000640 1 -- 172.21.15.159:0/1342355908 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f377006c8f0 msgr2=0x7f377006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:41.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.846+0000 7f3798000640 1 --2- 172.21.15.159:0/1342355908 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f377006c8f0 0x7f377006edb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3784000f60 tx=0x7f3784009040 comp rx=0 tx=0).stop 2024-04-04T06:16:41.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.846+0000 7f3798000640 1 -- 172.21.15.159:0/1342355908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37901e65f0 msgr2=0x7f37900b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:41.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.846+0000 7f3798000640 1 --2- 172.21.15.159:0/1342355908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37901e65f0 0x7f37900b0480 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f379006eee0 tx=0x7f3788004770 comp rx=0 tx=0).stop 2024-04-04T06:16:41.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.847+0000 7f3798000640 1 -- 172.21.15.159:0/1342355908 shutdown_connections 2024-04-04T06:16:41.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.847+0000 7f3798000640 1 --2- 172.21.15.159:0/1342355908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37901e65f0 0x7f37900b0480 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:41.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.847+0000 7f3798000640 1 --2- 172.21.15.159:0/1342355908 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f377006c8f0 0x7f377006edb0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:41.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.847+0000 7f3798000640 1 --2- 172.21.15.159:0/1342355908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37901e7c40 0x7f3790152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:41.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.847+0000 7f3798000640 1 --2- 172.21.15.159:0/1342355908 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3778004800 0x7f3790151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:41.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.847+0000 7f3798000640 1 -- 172.21.15.159:0/1342355908 >> 172.21.15.159:0/1342355908 conn(0x7f379006b990 msgr2=0x7f37900aece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:41.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.847+0000 7f3798000640 1 -- 172.21.15.159:0/1342355908 shutdown_connections 2024-04-04T06:16:41.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:41.847+0000 7f3798000640 1 -- 172.21.15.159:0/1342355908 wait complete. 2024-04-04T06:16:41.861 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:16:42.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.067+0000 7f610ea00640 1 Processor -- start 2024-04-04T06:16:42.062 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.067+0000 7f610ea00640 1 -- start start 2024-04-04T06:16:42.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.068+0000 7f610ea00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f610815b310 0x7f61080727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:42.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.068+0000 7f610ea00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f61080d0780 0x7f61080d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:42.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.068+0000 7f610ea00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f610815b6f0 con 0x7f61080b4f90 2024-04-04T06:16:42.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.068+0000 7f610ea00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f6108072e80 con 0x7f61080d0780 2024-04-04T06:16:42.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.068+0000 7f610ea00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6108073160 con 0x7f610815b310 2024-04-04T06:16:42.063 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.069+0000 7f60ff400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f61080b4f90 0x7f610815f7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37742/0 (socket says 172.21.15.159:37742) 2024-04-04T06:16:42.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.069+0000 7f60ff400640 1 -- 172.21.15.159:0/221526128 learned_addr learned my addr 172.21.15.159:0/221526128 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:16:42.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.069+0000 7f6107e00640 1 --2- 172.21.15.159:0/221526128 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f610815b310 0x7f61080727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:42.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.069+0000 7f610ca00640 1 --2- 172.21.15.159:0/221526128 >> v2:172.21.15.178:3300/0 conn(0x7f61080d0780 0x7f61080d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:42.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.069+0000 7f6107400640 1 -- 172.21.15.159:0/221526128 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3654154004 0 0) 0x7f610815b6f0 con 0x7f61080b4f90 2024-04-04T06:16:42.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.069+0000 7f6107400640 1 -- 172.21.15.159:0/221526128 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60ec003520 con 0x7f61080b4f90 2024-04-04T06:16:42.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f6107400640 1 -- 172.21.15.159:0/221526128 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2442433608 0 0) 0x7f60ec003520 con 0x7f61080b4f90 2024-04-04T06:16:42.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f6107400640 1 -- 172.21.15.159:0/221526128 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f610815b310 msgr2=0x7f61080727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:42.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f6107400640 1 --2- 172.21.15.159:0/221526128 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f610815b310 0x7f61080727f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:42.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f6107400640 1 -- 172.21.15.159:0/221526128 >> v2:172.21.15.178:3300/0 conn(0x7f61080d0780 msgr2=0x7f61080d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:42.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f6107400640 1 --2- 172.21.15.159:0/221526128 >> v2:172.21.15.178:3300/0 conn(0x7f61080d0780 0x7f61080d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:42.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f6107400640 1 -- 172.21.15.159:0/221526128 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610815efe0 con 0x7f61080b4f90 2024-04-04T06:16:42.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f6107400640 1 -- 172.21.15.159:0/221526128 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f60f0001670 con 0x7f61080b4f90 2024-04-04T06:16:42.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f6107400640 1 -- 172.21.15.159:0/221526128 >> v1:172.21.15.159:6789/0 conn(0x7f61080b4f90 legacy=0x7f610815f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:42.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f6107400640 1 --2- 172.21.15.159:0/221526128 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60ec0039b0 0x7f60ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:42.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f6107400640 1 --2- 172.21.15.159:0/221526128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60ec004800 0x7f60ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:42.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f6107400640 1 --2- 172.21.15.159:0/221526128 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60ec0051c0 0x7f60ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:42.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f6107400640 1 -- 172.21.15.159:0/221526128 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6108072e80 con 0x7f60ec0039b0 2024-04-04T06:16:42.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f6107400640 1 -- 172.21.15.159:0/221526128 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6108073160 con 0x7f60ec004800 2024-04-04T06:16:42.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f6107400640 1 -- 172.21.15.159:0/221526128 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f60ec00bfe0 con 0x7f60ec0051c0 2024-04-04T06:16:42.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f6107e00640 1 --2- 172.21.15.159:0/221526128 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f610815b310 0x7f61080727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:16:42.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f6107e00640 1 --2- 172.21.15.159:0/221526128 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60ec0039b0 0x7f60ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:42.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f60ff400640 1 --2- 172.21.15.159:0/221526128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60ec004800 0x7f60ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:42.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f6107e00640 1 -- 172.21.15.159:0/221526128 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60ec0051c0 msgr2=0x7f60ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:42.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f6107e00640 1 --2- 172.21.15.159:0/221526128 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60ec0051c0 0x7f60ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:42.068 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f6107e00640 1 -- 172.21.15.159:0/221526128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60ec004800 msgr2=0x7f60ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:42.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f6107e00640 1 --2- 172.21.15.159:0/221526128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60ec004800 0x7f60ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:42.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.070+0000 7f6107e00640 1 -- 172.21.15.159:0/221526128 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ec003520 con 0x7f60ec0039b0 2024-04-04T06:16:42.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.072+0000 7f6107e00640 1 --2- 172.21.15.159:0/221526128 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60ec0039b0 0x7f60ec003dd0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f61080d06d0 tx=0x7f60ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bebe428bff0fb150 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:42.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.072+0000 7f6107400640 1 -- 172.21.15.159:0/221526128 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f80097d0 con 0x7f60ec0039b0 2024-04-04T06:16:42.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.072+0000 7f6107400640 1 -- 172.21.15.159:0/221526128 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60f8004030 con 0x7f60ec0039b0 2024-04-04T06:16:42.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.072+0000 7f6107400640 1 -- 172.21.15.159:0/221526128 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8004400 con 0x7f60ec0039b0 2024-04-04T06:16:42.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.072+0000 7f610ea00640 1 -- 172.21.15.159:0/221526128 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60ec0039b0 msgr2=0x7f60ec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:42.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.072+0000 7f610ea00640 1 --2- 172.21.15.159:0/221526128 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60ec0039b0 0x7f60ec003dd0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f61080d06d0 tx=0x7f60ec005830 comp rx=0 tx=0).stop 2024-04-04T06:16:42.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.072+0000 7f610ca00640 1 -- 172.21.15.159:0/221526128 reap_dead start 2024-04-04T06:16:42.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.072+0000 7f610ea00640 1 -- 172.21.15.159:0/221526128 shutdown_connections 2024-04-04T06:16:42.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.072+0000 7f610ea00640 1 -- 172.21.15.159:0/221526128 >> 172.21.15.159:0/221526128 conn(0x7f610806b990 msgr2=0x7f610806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:42.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.073+0000 7f610ea00640 1 -- 172.21.15.159:0/221526128 shutdown_connections 2024-04-04T06:16:42.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.073+0000 7f610ea00640 1 -- 172.21.15.159:0/221526128 wait complete. 2024-04-04T06:16:42.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.074+0000 7f610ea00640 1 Processor -- start 2024-04-04T06:16:42.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.074+0000 7f610ea00640 1 -- start start 2024-04-04T06:16:42.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.075+0000 7f610ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60ec0039b0 0x7f61081465a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:42.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.075+0000 7f610ea00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6108156460 0x7f61081568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:42.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.075+0000 7f610ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61080cc8c0 0x7f61080ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:42.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.075+0000 7f610ea00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6108073440 con 0x7f61080cc8c0 2024-04-04T06:16:42.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.075+0000 7f610ea00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f61080b53f0 con 0x7f60ec0039b0 2024-04-04T06:16:42.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.075+0000 7f610ea00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f61080727f0 con 0x7f6108156460 2024-04-04T06:16:42.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.076+0000 7f610ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61080cc8c0 0x7f61080ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:42.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.076+0000 7f610ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61080cc8c0 0x7f61080ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51498/0 (socket says 172.21.15.159:51498) 2024-04-04T06:16:42.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.076+0000 7f610ca00640 1 -- 172.21.15.159:0/3801976498 learned_addr learned my addr 172.21.15.159:0/3801976498 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:42.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.076+0000 7f610ca00640 1 -- 172.21.15.159:0/3801976498 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6108156460 msgr2=0x7f61081568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:42.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.076+0000 7f610ca00640 1 --2- 172.21.15.159:0/3801976498 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6108156460 0x7f61081568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:42.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.076+0000 7f610ca00640 1 -- 172.21.15.159:0/3801976498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60ec0039b0 msgr2=0x7f61081465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:42.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.076+0000 7f610ca00640 1 --2- 172.21.15.159:0/3801976498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60ec0039b0 0x7f61081465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:42.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.076+0000 7f610ca00640 1 -- 172.21.15.159:0/3801976498 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6108146ae0 con 0x7f61080cc8c0 2024-04-04T06:16:42.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.076+0000 7f610ca00640 1 --2- 172.21.15.159:0/3801976498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61080cc8c0 0x7f61080ccd40 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f610000ee50 tx=0x7f610000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:42.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.077+0000 7f6104c00640 1 -- 172.21.15.159:0/3801976498 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610000bbf0 con 0x7f61080cc8c0 2024-04-04T06:16:42.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.077+0000 7f6104c00640 1 -- 172.21.15.159:0/3801976498 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f610001a070 con 0x7f61080cc8c0 2024-04-04T06:16:42.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.077+0000 7f6104c00640 1 -- 172.21.15.159:0/3801976498 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6100014400 con 0x7f61080cc8c0 2024-04-04T06:16:42.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.077+0000 7f610ea00640 1 -- 172.21.15.159:0/3801976498 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f610815bca0 con 0x7f61080cc8c0 2024-04-04T06:16:42.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.077+0000 7f610ea00640 1 -- 172.21.15.159:0/3801976498 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f610815c260 con 0x7f61080cc8c0 2024-04-04T06:16:42.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.078+0000 7f610ea00640 1 -- 172.21.15.159:0/3801976498 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6108070100 con 0x7f61080cc8c0 2024-04-04T06:16:42.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.079+0000 7f6104c00640 1 -- 172.21.15.159:0/3801976498 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6100013070 con 0x7f61080cc8c0 2024-04-04T06:16:42.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.080+0000 7f6104c00640 1 --2- 172.21.15.159:0/3801976498 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f60e406c920 0x7f60e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:42.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.080+0000 7f6104c00640 1 -- 172.21.15.159:0/3801976498 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7f610008bcc0 con 0x7f61080cc8c0 2024-04-04T06:16:42.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.080+0000 7f6107e00640 1 --2- 172.21.15.159:0/3801976498 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f60e406c920 0x7f60e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:42.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.081+0000 7f6107e00640 1 --2- 172.21.15.159:0/3801976498 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f60e406c920 0x7f60e406ede0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f60f8035000 tx=0x7f60f802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:42.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.084+0000 7f6104c00640 1 -- 172.21.15.159:0/3801976498 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6108070100 con 0x7f61080cc8c0 2024-04-04T06:16:42.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.278+0000 7f610ea00640 1 -- 172.21.15.159:0/3801976498 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61080685c0 con 0x7f61080cc8c0 2024-04-04T06:16:42.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.281+0000 7f6104c00640 1 -- 172.21.15.159:0/3801976498 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16886 (secure 0 0 0) 0x7f6100092080 con 0x7f61080cc8c0 2024-04-04T06:16:42.276 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:42.277 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":103,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:16:37.034092+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:15:05.537357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-04T06:15:06.689304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6835/1003751501":"2024-04-05T06:16:37.034044+0000","172.21.15.159:6835/2253383212":"2024-04-05T06:15:59.385487+0000","172.21.15.159:6834/1003751501":"2024-04-05T06:16:37.034044+0000","172.21.15.159:6834/2253383212":"2024-04-05T06:15:59.385487+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:16:42.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.284+0000 7f610ea00640 1 -- 172.21.15.159:0/3801976498 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f60e406c920 msgr2=0x7f60e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:42.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.284+0000 7f610ea00640 1 --2- 172.21.15.159:0/3801976498 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f60e406c920 0x7f60e406ede0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f60f8035000 tx=0x7f60f802e000 comp rx=0 tx=0).stop 2024-04-04T06:16:42.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.284+0000 7f610ea00640 1 -- 172.21.15.159:0/3801976498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61080cc8c0 msgr2=0x7f61080ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:42.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.284+0000 7f610ea00640 1 --2- 172.21.15.159:0/3801976498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61080cc8c0 0x7f61080ccd40 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f610000ee50 tx=0x7f610000b480 comp rx=0 tx=0).stop 2024-04-04T06:16:42.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.284+0000 7f610ea00640 1 -- 172.21.15.159:0/3801976498 shutdown_connections 2024-04-04T06:16:42.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.284+0000 7f610ea00640 1 --2- 172.21.15.159:0/3801976498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61080cc8c0 0x7f61080ccd40 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:42.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.284+0000 7f610ea00640 1 --2- 172.21.15.159:0/3801976498 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6108156460 0x7f61081568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:42.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.284+0000 7f610ea00640 1 --2- 172.21.15.159:0/3801976498 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f60e406c920 0x7f60e406ede0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:42.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.284+0000 7f610ea00640 1 --2- 172.21.15.159:0/3801976498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60ec0039b0 0x7f61081465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:42.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.284+0000 7f610ea00640 1 -- 172.21.15.159:0/3801976498 >> 172.21.15.159:0/3801976498 conn(0x7f610806b990 msgr2=0x7f61080b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:42.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.284+0000 7f610ea00640 1 -- 172.21.15.159:0/3801976498 shutdown_connections 2024-04-04T06:16:42.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.284+0000 7f610ea00640 1 -- 172.21.15.159:0/3801976498 wait complete. 2024-04-04T06:16:42.296 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-04T06:16:42.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.501+0000 7f49c7c00640 1 Processor -- start 2024-04-04T06:16:42.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.502+0000 7f49c7c00640 1 -- start start 2024-04-04T06:16:42.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.503+0000 7f49c7c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49c015e860 0x7f49c006d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:42.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.503+0000 7f49c7c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49c006dfa0 con 0x7f49c00d4ae0 2024-04-04T06:16:42.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.503+0000 7f49c7c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49c006e240 con 0x7f49c006da30 2024-04-04T06:16:42.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.503+0000 7f49c7c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f49c00cf220 con 0x7f49c015e860 2024-04-04T06:16:42.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.503+0000 7f49c5c00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f49c00d4ae0 0x7f49c00cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:37748/0 (socket says 172.21.15.159:37748) 2024-04-04T06:16:42.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.503+0000 7f49c5c00640 1 -- 172.21.15.159:0/1369008332 learned_addr learned my addr 172.21.15.159:0/1369008332 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:16:42.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.504+0000 7f49c5200640 1 --2- 172.21.15.159:0/1369008332 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49c015e860 0x7f49c006d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:42.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.504+0000 7f49bf400640 1 -- 172.21.15.159:0/1369008332 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 155583704 0 0) 0x7f49c006dfa0 con 0x7f49c00d4ae0 2024-04-04T06:16:42.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.504+0000 7f49bf400640 1 -- 172.21.15.159:0/1369008332 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49a4003520 con 0x7f49c00d4ae0 2024-04-04T06:16:42.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.504+0000 7f49bf400640 1 -- 172.21.15.159:0/1369008332 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4112205456 0 0) 0x7f49a4003520 con 0x7f49c00d4ae0 2024-04-04T06:16:42.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.504+0000 7f49bf400640 1 -- 172.21.15.159:0/1369008332 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49c015e860 msgr2=0x7f49c006d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:42.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.504+0000 7f49bf400640 1 --2- 172.21.15.159:0/1369008332 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49c015e860 0x7f49c006d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:42.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.504+0000 7f49bf400640 1 -- 172.21.15.159:0/1369008332 >> v1:172.21.15.178:6789/0 conn(0x7f49c006da30 legacy=0x7f49c00d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:42.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.504+0000 7f49bf400640 1 -- 172.21.15.159:0/1369008332 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c006e520 con 0x7f49c00d4ae0 2024-04-04T06:16:42.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.504+0000 7f49bf400640 1 -- 172.21.15.159:0/1369008332 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f49b8001a10 con 0x7f49c00d4ae0 2024-04-04T06:16:42.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.504+0000 7f49bf400640 1 -- 172.21.15.159:0/1369008332 >> v1:172.21.15.159:6789/0 conn(0x7f49c00d4ae0 legacy=0x7f49c00cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:42.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.504+0000 7f49bf400640 1 --2- 172.21.15.159:0/1369008332 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49c015e860 0x7f49a40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:42.502 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.505+0000 7f49bf400640 1 --2- 172.21.15.159:0/1369008332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49a4003f60 0x7f49a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:42.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.505+0000 7f49bf400640 1 --2- 172.21.15.159:0/1369008332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49a4004d40 0x7f49a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:42.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.505+0000 7f49bf400640 1 -- 172.21.15.159:0/1369008332 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49c00cf220 con 0x7f49a4003f60 2024-04-04T06:16:42.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.505+0000 7f49bf400640 1 -- 172.21.15.159:0/1369008332 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f49a400baa0 con 0x7f49a4004d40 2024-04-04T06:16:42.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.505+0000 7f49bf400640 1 -- 172.21.15.159:0/1369008332 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f49a400bca0 con 0x7f49c015e860 2024-04-04T06:16:42.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.505+0000 7f49bfe00640 1 --2- 172.21.15.159:0/1369008332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49a4003f60 0x7f49a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:42.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.505+0000 7f49c5200640 1 --2- 172.21.15.159:0/1369008332 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49c015e860 0x7f49a40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:42.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.505+0000 7f49c5c00640 1 --2- 172.21.15.159:0/1369008332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49a4004d40 0x7f49a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:42.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.505+0000 7f49bfe00640 1 -- 172.21.15.159:0/1369008332 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49c015e860 msgr2=0x7f49a40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:42.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.505+0000 7f49bfe00640 1 --2- 172.21.15.159:0/1369008332 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49c015e860 0x7f49a40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:42.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.505+0000 7f49bfe00640 1 -- 172.21.15.159:0/1369008332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49a4004d40 msgr2=0x7f49a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:42.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.505+0000 7f49bfe00640 1 --2- 172.21.15.159:0/1369008332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49a4004d40 0x7f49a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:42.503 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.505+0000 7f49bfe00640 1 -- 172.21.15.159:0/1369008332 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a4003520 con 0x7f49a4003f60 2024-04-04T06:16:42.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.507+0000 7f49bfe00640 1 --2- 172.21.15.159:0/1369008332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49a4003f60 0x7f49a4004800 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f49a8001750 tx=0x7f49a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f6cf787d860a31 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:42.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.507+0000 7f49bf400640 1 -- 172.21.15.159:0/1369008332 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a8001370 con 0x7f49a4003f60 2024-04-04T06:16:42.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.507+0000 7f49bf400640 1 -- 172.21.15.159:0/1369008332 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49a80052d0 con 0x7f49a4003f60 2024-04-04T06:16:42.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.507+0000 7f49bf400640 1 -- 172.21.15.159:0/1369008332 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a8003830 con 0x7f49a4003f60 2024-04-04T06:16:42.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.507+0000 7f49c7c00640 1 -- 172.21.15.159:0/1369008332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49a4003f60 msgr2=0x7f49a4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:42.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.507+0000 7f49c7c00640 1 --2- 172.21.15.159:0/1369008332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49a4003f60 0x7f49a4004800 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f49a8001750 tx=0x7f49a4005830 comp rx=0 tx=0).stop 2024-04-04T06:16:42.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.507+0000 7f49c5c00640 1 -- 172.21.15.159:0/1369008332 reap_dead start 2024-04-04T06:16:42.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.508+0000 7f49c7c00640 1 -- 172.21.15.159:0/1369008332 shutdown_connections 2024-04-04T06:16:42.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.508+0000 7f49c7c00640 1 -- 172.21.15.159:0/1369008332 >> 172.21.15.159:0/1369008332 conn(0x7f49c009d7d0 msgr2=0x7f49c008bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:42.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.508+0000 7f49c7c00640 1 -- 172.21.15.159:0/1369008332 shutdown_connections 2024-04-04T06:16:42.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.508+0000 7f49c7c00640 1 -- 172.21.15.159:0/1369008332 wait complete. 2024-04-04T06:16:42.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.509+0000 7f49c7c00640 1 Processor -- start 2024-04-04T06:16:42.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.509+0000 7f49c7c00640 1 -- start start 2024-04-04T06:16:42.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.510+0000 7f49c7c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49a4003f60 0x7f49c00cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:42.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.510+0000 7f49c7c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49c015bd20 0x7f49c00b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:42.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.510+0000 7f49c7c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49c01f4330 0x7f49c0152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:42.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.510+0000 7f49c7c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49c00cef30 con 0x7f49c01f4330 2024-04-04T06:16:42.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.510+0000 7f49c7c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f49c00cedb0 con 0x7f49a4003f60 2024-04-04T06:16:42.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.510+0000 7f49c7c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f49c006d4f0 con 0x7f49c015bd20 2024-04-04T06:16:42.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.511+0000 7f49c5c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49c01f4330 0x7f49c0152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:42.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.511+0000 7f49c5c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49c01f4330 0x7f49c0152630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:51510/0 (socket says 172.21.15.159:51510) 2024-04-04T06:16:42.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.511+0000 7f49c5c00640 1 -- 172.21.15.159:0/451122986 learned_addr learned my addr 172.21.15.159:0/451122986 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:42.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.511+0000 7f49bfe00640 1 --2- 172.21.15.159:0/451122986 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49c015bd20 0x7f49c00b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:42.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.511+0000 7f49c5200640 1 --2- 172.21.15.159:0/451122986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49a4003f60 0x7f49c00cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:42.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.511+0000 7f49c5c00640 1 -- 172.21.15.159:0/451122986 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49c015bd20 msgr2=0x7f49c00b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:42.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.511+0000 7f49c5c00640 1 --2- 172.21.15.159:0/451122986 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49c015bd20 0x7f49c00b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:42.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.511+0000 7f49c5c00640 1 -- 172.21.15.159:0/451122986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49a4003f60 msgr2=0x7f49c00cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:42.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.511+0000 7f49c5c00640 1 --2- 172.21.15.159:0/451122986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49a4003f60 0x7f49c00cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:42.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.511+0000 7f49c5c00640 1 -- 172.21.15.159:0/451122986 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c00cba50 con 0x7f49c01f4330 2024-04-04T06:16:42.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.511+0000 7f49c5c00640 1 --2- 172.21.15.159:0/451122986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49c01f4330 0x7f49c0152630 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f49b8000f80 tx=0x7f49b800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:42.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.512+0000 7f49bcc00640 1 -- 172.21.15.159:0/451122986 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b800ee20 con 0x7f49c01f4330 2024-04-04T06:16:42.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.512+0000 7f49bcc00640 1 -- 172.21.15.159:0/451122986 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49b8018070 con 0x7f49c01f4330 2024-04-04T06:16:42.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.512+0000 7f49bcc00640 1 -- 172.21.15.159:0/451122986 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b8013660 con 0x7f49c01f4330 2024-04-04T06:16:42.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.512+0000 7f49c7c00640 1 -- 172.21.15.159:0/451122986 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c00cbd00 con 0x7f49c01f4330 2024-04-04T06:16:42.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.512+0000 7f49c7c00640 1 -- 172.21.15.159:0/451122986 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c00cc2c0 con 0x7f49c01f4330 2024-04-04T06:16:42.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.513+0000 7f49c7c00640 1 -- 172.21.15.159:0/451122986 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c00685c0 con 0x7f49c01f4330 2024-04-04T06:16:42.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.514+0000 7f49bcc00640 1 -- 172.21.15.159:0/451122986 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49b8013800 con 0x7f49c01f4330 2024-04-04T06:16:42.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.515+0000 7f49bcc00640 1 --2- 172.21.15.159:0/451122986 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f499c06c8f0 0x7f499c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:42.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.515+0000 7f49bcc00640 1 -- 172.21.15.159:0/451122986 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7f49b808bf30 con 0x7f49c01f4330 2024-04-04T06:16:42.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.515+0000 7f49c5200640 1 --2- 172.21.15.159:0/451122986 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f499c06c8f0 0x7f499c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:42.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.517+0000 7f49c5200640 1 --2- 172.21.15.159:0/451122986 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f499c06c8f0 0x7f499c06edb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009950 tx=0x7f49b0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:42.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.519+0000 7f49bcc00640 1 -- 172.21.15.159:0/451122986 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f49b805a2b0 con 0x7f49c01f4330 2024-04-04T06:16:42.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:42.754+0000 7f49c7c00640 1 -- 172.21.15.159:0/451122986 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f49c00d5110 con 0x7f49c01f4330 2024-04-04T06:16:43.652 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:16:44.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:43.684+0000 7f49bcc00640 1 -- 172.21.15.159:0/451122986 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v93) ==== 161+0+0 (secure 0 0 0) 0x7f49b8059bb0 con 0x7f49c01f4330 2024-04-04T06:16:44.195 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-04T06:16:44.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:43.689+0000 7f49c7c00640 1 -- 172.21.15.159:0/451122986 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f499c06c8f0 msgr2=0x7f499c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:44.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:43.689+0000 7f49c7c00640 1 --2- 172.21.15.159:0/451122986 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f499c06c8f0 0x7f499c06edb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009950 tx=0x7f49b0008040 comp rx=0 tx=0).stop 2024-04-04T06:16:44.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:43.689+0000 7f49c7c00640 1 -- 172.21.15.159:0/451122986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49c01f4330 msgr2=0x7f49c0152630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:44.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:43.689+0000 7f49c7c00640 1 --2- 172.21.15.159:0/451122986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49c01f4330 0x7f49c0152630 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f49b8000f80 tx=0x7f49b800e530 comp rx=0 tx=0).stop 2024-04-04T06:16:44.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:43.689+0000 7f49c7c00640 1 -- 172.21.15.159:0/451122986 shutdown_connections 2024-04-04T06:16:44.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:43.689+0000 7f49c7c00640 1 --2- 172.21.15.159:0/451122986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49c01f4330 0x7f49c0152630 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:44.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:43.689+0000 7f49c7c00640 1 --2- 172.21.15.159:0/451122986 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49c015bd20 0x7f49c00b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:44.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:43.689+0000 7f49c7c00640 1 --2- 172.21.15.159:0/451122986 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f499c06c8f0 0x7f499c06edb0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:44.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:43.689+0000 7f49c7c00640 1 --2- 172.21.15.159:0/451122986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49a4003f60 0x7f49c00cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:44.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:43.689+0000 7f49c7c00640 1 -- 172.21.15.159:0/451122986 >> 172.21.15.159:0/451122986 conn(0x7f49c009d7d0 msgr2=0x7f49c00b51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:44.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:43.690+0000 7f49c7c00640 1 -- 172.21.15.159:0/451122986 shutdown_connections 2024-04-04T06:16:44.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:43.690+0000 7f49c7c00640 1 -- 172.21.15.159:0/451122986 wait complete. 2024-04-04T06:16:44.197 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-04T06:16:44.245 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:16:44.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.440+0000 7f6a50200640 1 Processor -- start 2024-04-04T06:16:44.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.440+0000 7f6a50200640 1 -- start start 2024-04-04T06:16:44.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.441+0000 7f6a50200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a480af4b0 0x7f6a48167460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:44.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.442+0000 7f6a50200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a480afab0 con 0x7f6a480d0990 2024-04-04T06:16:44.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.442+0000 7f6a50200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a4815cc80 con 0x7f6a48153580 2024-04-04T06:16:44.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.442+0000 7f6a50200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6a4815da00 con 0x7f6a480af4b0 2024-04-04T06:16:44.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.442+0000 7f6a4ce00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f6a480d0990 0x7f6a480aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40284/0 (socket says 172.21.15.159:40284) 2024-04-04T06:16:44.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.442+0000 7f6a4ce00640 1 -- 172.21.15.159:0/2708082232 learned_addr learned my addr 172.21.15.159:0/2708082232 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:16:44.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.442+0000 7f6a47e00640 1 -- 172.21.15.159:0/2708082232 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 977409378 0 0) 0x7f6a480afab0 con 0x7f6a480d0990 2024-04-04T06:16:44.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.442+0000 7f6a4e200640 1 --2- 172.21.15.159:0/2708082232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a480af4b0 0x7f6a48167460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:44.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.442+0000 7f6a47e00640 1 -- 172.21.15.159:0/2708082232 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a30003520 con 0x7f6a480d0990 2024-04-04T06:16:44.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.442+0000 7f6a47e00640 1 -- 172.21.15.159:0/2708082232 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2577636253 0 0) 0x7f6a30003520 con 0x7f6a480d0990 2024-04-04T06:16:44.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.443+0000 7f6a47e00640 1 -- 172.21.15.159:0/2708082232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a480af4b0 msgr2=0x7f6a48167460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:44.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.443+0000 7f6a47e00640 1 --2- 172.21.15.159:0/2708082232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a480af4b0 0x7f6a48167460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:44.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.443+0000 7f6a47e00640 1 -- 172.21.15.159:0/2708082232 >> v1:172.21.15.178:6789/0 conn(0x7f6a48153580 legacy=0x7f6a480d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:44.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.443+0000 7f6a47e00640 1 -- 172.21.15.159:0/2708082232 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4815cf20 con 0x7f6a480d0990 2024-04-04T06:16:44.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.443+0000 7f6a47e00640 1 -- 172.21.15.159:0/2708082232 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f6a34001670 con 0x7f6a480d0990 2024-04-04T06:16:44.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.443+0000 7f6a4e200640 1 --2- 172.21.15.159:0/2708082232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a480af4b0 0x7f6a48167460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:16:44.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.443+0000 7f6a47e00640 1 -- 172.21.15.159:0/2708082232 >> v1:172.21.15.159:6789/0 conn(0x7f6a480d0990 legacy=0x7f6a480aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:44.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.443+0000 7f6a47e00640 1 --2- 172.21.15.159:0/2708082232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a300039e0 0x7f6a30003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:44.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.443+0000 7f6a47e00640 1 --2- 172.21.15.159:0/2708082232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a30004800 0x7f6a30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:44.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.443+0000 7f6a47e00640 1 --2- 172.21.15.159:0/2708082232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a480af4b0 0x7f6a300052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:44.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.443+0000 7f6a47e00640 1 -- 172.21.15.159:0/2708082232 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4815da00 con 0x7f6a30004800 2024-04-04T06:16:44.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.443+0000 7f6a47e00640 1 -- 172.21.15.159:0/2708082232 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3000bba0 con 0x7f6a300039e0 2024-04-04T06:16:44.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.443+0000 7f6a47e00640 1 -- 172.21.15.159:0/2708082232 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6a3000bda0 con 0x7f6a480af4b0 2024-04-04T06:16:44.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.443+0000 7f6a4ce00640 1 --2- 172.21.15.159:0/2708082232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a30004800 0x7f6a30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:44.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.443+0000 7f6a4d800640 1 --2- 172.21.15.159:0/2708082232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a300039e0 0x7f6a30003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:44.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.443+0000 7f6a4e200640 1 --2- 172.21.15.159:0/2708082232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a480af4b0 0x7f6a300052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:44.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.443+0000 7f6a4ce00640 1 -- 172.21.15.159:0/2708082232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a480af4b0 msgr2=0x7f6a300052b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:44.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.443+0000 7f6a4ce00640 1 --2- 172.21.15.159:0/2708082232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a480af4b0 0x7f6a300052b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:44.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.443+0000 7f6a4ce00640 1 -- 172.21.15.159:0/2708082232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a300039e0 msgr2=0x7f6a30003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:44.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.443+0000 7f6a4ce00640 1 --2- 172.21.15.159:0/2708082232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a300039e0 0x7f6a30003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:44.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.443+0000 7f6a4ce00640 1 -- 172.21.15.159:0/2708082232 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a30003520 con 0x7f6a30004800 2024-04-04T06:16:44.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.444+0000 7f6a4ce00640 1 --2- 172.21.15.159:0/2708082232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a30004800 0x7f6a30004c80 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f6a4806f820 tx=0x7f6a30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8a568328441795 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:44.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.445+0000 7f6a47e00640 1 -- 172.21.15.159:0/2708082232 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34007c20 con 0x7f6a30004800 2024-04-04T06:16:44.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.445+0000 7f6a47e00640 1 -- 172.21.15.159:0/2708082232 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a34030070 con 0x7f6a30004800 2024-04-04T06:16:44.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.445+0000 7f6a47e00640 1 -- 172.21.15.159:0/2708082232 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34005d30 con 0x7f6a30004800 2024-04-04T06:16:44.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.445+0000 7f6a50200640 1 -- 172.21.15.159:0/2708082232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a30004800 msgr2=0x7f6a30004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:44.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.445+0000 7f6a50200640 1 --2- 172.21.15.159:0/2708082232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a30004800 0x7f6a30004c80 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f6a4806f820 tx=0x7f6a30005830 comp rx=0 tx=0).stop 2024-04-04T06:16:44.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.445+0000 7f6a4e200640 1 -- 172.21.15.159:0/2708082232 reap_dead start 2024-04-04T06:16:44.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.445+0000 7f6a50200640 1 -- 172.21.15.159:0/2708082232 shutdown_connections 2024-04-04T06:16:44.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.445+0000 7f6a50200640 1 -- 172.21.15.159:0/2708082232 >> 172.21.15.159:0/2708082232 conn(0x7f6a4806b750 msgr2=0x7f6a4806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:44.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.445+0000 7f6a50200640 1 -- 172.21.15.159:0/2708082232 shutdown_connections 2024-04-04T06:16:44.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.445+0000 7f6a50200640 1 -- 172.21.15.159:0/2708082232 wait complete. 2024-04-04T06:16:44.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.447+0000 7f6a50200640 1 Processor -- start 2024-04-04T06:16:44.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.447+0000 7f6a50200640 1 -- start start 2024-04-04T06:16:44.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.448+0000 7f6a50200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a30004800 0x7f6a48072570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:44.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.448+0000 7f6a50200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a480cc220 0x7f6a4815b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:44.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.448+0000 7f6a50200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a48154c60 0x7f6a48155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:44.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.448+0000 7f6a50200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6a48153960 con 0x7f6a30004800 2024-04-04T06:16:44.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.448+0000 7f6a50200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4815d4e0 con 0x7f6a480cc220 2024-04-04T06:16:44.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.448+0000 7f6a50200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6a48165450 con 0x7f6a48154c60 2024-04-04T06:16:44.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.448+0000 7f6a4d800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a30004800 0x7f6a48072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:44.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.448+0000 7f6a4d800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a30004800 0x7f6a48072570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52312/0 (socket says 172.21.15.159:52312) 2024-04-04T06:16:44.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.448+0000 7f6a4d800640 1 -- 172.21.15.159:0/1900418883 learned_addr learned my addr 172.21.15.159:0/1900418883 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:44.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.449+0000 7f6a4d800640 1 -- 172.21.15.159:0/1900418883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a48154c60 msgr2=0x7f6a48155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:44.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.449+0000 7f6a4d800640 1 --2- 172.21.15.159:0/1900418883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a48154c60 0x7f6a48155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:44.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.449+0000 7f6a4d800640 1 -- 172.21.15.159:0/1900418883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a480cc220 msgr2=0x7f6a4815b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:44.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.449+0000 7f6a4d800640 1 --2- 172.21.15.159:0/1900418883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a480cc220 0x7f6a4815b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:44.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.449+0000 7f6a4d800640 1 -- 172.21.15.159:0/1900418883 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a480717b0 con 0x7f6a30004800 2024-04-04T06:16:44.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.449+0000 7f6a4d800640 1 --2- 172.21.15.159:0/1900418883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a30004800 0x7f6a48072570 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00e440 tx=0x7f6a3c00e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:44.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.449+0000 7f6a45600640 1 -- 172.21.15.159:0/1900418883 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c0052e0 con 0x7f6a30004800 2024-04-04T06:16:44.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.449+0000 7f6a45600640 1 -- 172.21.15.159:0/1900418883 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a3c005480 con 0x7f6a30004800 2024-04-04T06:16:44.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.449+0000 7f6a45600640 1 -- 172.21.15.159:0/1900418883 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c007020 con 0x7f6a30004800 2024-04-04T06:16:44.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.449+0000 7f6a50200640 1 -- 172.21.15.159:0/1900418883 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a48071a90 con 0x7f6a30004800 2024-04-04T06:16:44.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.450+0000 7f6a50200640 1 -- 172.21.15.159:0/1900418883 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a4815bd60 con 0x7f6a30004800 2024-04-04T06:16:44.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.450+0000 7f6a50200640 1 -- 172.21.15.159:0/1900418883 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a480685c0 con 0x7f6a30004800 2024-04-04T06:16:44.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.452+0000 7f6a45600640 1 -- 172.21.15.159:0/1900418883 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a3c01a020 con 0x7f6a30004800 2024-04-04T06:16:44.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.452+0000 7f6a45600640 1 --2- 172.21.15.159:0/1900418883 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6a2806c8f0 0x7f6a2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:44.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.452+0000 7f6a45600640 1 -- 172.21.15.159:0/1900418883 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7f6a3c08b270 con 0x7f6a30004800 2024-04-04T06:16:44.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.453+0000 7f6a4ce00640 1 --2- 172.21.15.159:0/1900418883 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6a2806c8f0 0x7f6a2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:44.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.454+0000 7f6a4ce00640 1 --2- 172.21.15.159:0/1900418883 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6a2806c8f0 0x7f6a2806edb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6a34034000 tx=0x7f6a34039040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:44.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.459+0000 7f6a45600640 1 -- 172.21.15.159:0/1900418883 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6a3c01a310 con 0x7f6a30004800 2024-04-04T06:16:44.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:44.696+0000 7f6a50200640 1 -- 172.21.15.159:0/1900418883 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6a480b40f0 con 0x7f6a30004800 2024-04-04T06:16:45.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.699+0000 7f6a45600640 1 -- 172.21.15.159:0/1900418883 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v94) ==== 104+0+0 (secure 0 0 0) 0x7f6a3c059550 con 0x7f6a30004800 2024-04-04T06:16:45.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.702+0000 7f6a50200640 1 -- 172.21.15.159:0/1900418883 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6a2806c8f0 msgr2=0x7f6a2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:45.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.702+0000 7f6a50200640 1 --2- 172.21.15.159:0/1900418883 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6a2806c8f0 0x7f6a2806edb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6a34034000 tx=0x7f6a34039040 comp rx=0 tx=0).stop 2024-04-04T06:16:45.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.702+0000 7f6a50200640 1 -- 172.21.15.159:0/1900418883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a30004800 msgr2=0x7f6a48072570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:45.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.702+0000 7f6a50200640 1 --2- 172.21.15.159:0/1900418883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a30004800 0x7f6a48072570 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00e440 tx=0x7f6a3c00e900 comp rx=0 tx=0).stop 2024-04-04T06:16:45.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.702+0000 7f6a50200640 1 -- 172.21.15.159:0/1900418883 shutdown_connections 2024-04-04T06:16:45.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.702+0000 7f6a50200640 1 --2- 172.21.15.159:0/1900418883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6a48154c60 0x7f6a48155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:45.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.702+0000 7f6a50200640 1 --2- 172.21.15.159:0/1900418883 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6a2806c8f0 0x7f6a2806edb0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:45.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.702+0000 7f6a50200640 1 --2- 172.21.15.159:0/1900418883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a480cc220 0x7f6a4815b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:45.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.702+0000 7f6a50200640 1 --2- 172.21.15.159:0/1900418883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a30004800 0x7f6a48072570 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:45.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.702+0000 7f6a50200640 1 -- 172.21.15.159:0/1900418883 >> 172.21.15.159:0/1900418883 conn(0x7f6a4806b750 msgr2=0x7f6a480d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:45.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.702+0000 7f6a50200640 1 -- 172.21.15.159:0/1900418883 shutdown_connections 2024-04-04T06:16:45.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.703+0000 7f6a50200640 1 -- 172.21.15.159:0/1900418883 wait complete. 2024-04-04T06:16:45.717 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-04T06:16:45.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.937+0000 7f1659400640 1 Processor -- start 2024-04-04T06:16:45.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.938+0000 7f1659400640 1 -- start start 2024-04-04T06:16:45.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.939+0000 7f1659400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f16540acf10 0x7f165409dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:45.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.939+0000 7f1659400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f16540ad2f0 0x7f16541453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:45.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.939+0000 7f1659400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f165415cb90 con 0x7f165415ff70 2024-04-04T06:16:45.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.939+0000 7f1659400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f1654145f60 con 0x7f16540ad2f0 2024-04-04T06:16:45.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.939+0000 7f1659400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f165415c950 con 0x7f16540acf10 2024-04-04T06:16:45.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.939+0000 7f1651e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f16540ad2f0 0x7f16541453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:45.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.939+0000 7f1651e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f16540ad2f0 0x7f16541453f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52324/0 (socket says 172.21.15.159:52324) 2024-04-04T06:16:45.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.939+0000 7f1651e00640 1 -- 172.21.15.159:0/3249344873 learned_addr learned my addr 172.21.15.159:0/3249344873 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:45.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.940+0000 7f1651e00640 1 -- 172.21.15.159:0/3249344873 >> v2:172.21.15.178:3300/0 conn(0x7f16540acf10 msgr2=0x7f165409dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:45.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.940+0000 7f1651e00640 1 --2- 172.21.15.159:0/3249344873 >> v2:172.21.15.178:3300/0 conn(0x7f16540acf10 0x7f165409dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:45.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.940+0000 7f1651e00640 1 -- 172.21.15.159:0/3249344873 >> v1:172.21.15.159:6789/0 conn(0x7f165415ff70 legacy=0x7f1654160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:45.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.940+0000 7f1651e00640 1 -- 172.21.15.159:0/3249344873 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165415ce70 con 0x7f16540ad2f0 2024-04-04T06:16:45.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.941+0000 7f1651e00640 1 --2- 172.21.15.159:0/3249344873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16540ad2f0 0x7f16541453f0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f165406c580 tx=0x7f164402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a828047a95deb496 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:45.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.941+0000 7f1651400640 1 -- 172.21.15.159:0/3249344873 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164402ca80 con 0x7f16540ad2f0 2024-04-04T06:16:45.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.941+0000 7f1651400640 1 -- 172.21.15.159:0/3249344873 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f164402cc20 con 0x7f16540ad2f0 2024-04-04T06:16:45.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.941+0000 7f1651400640 1 -- 172.21.15.159:0/3249344873 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16440343f0 con 0x7f16540ad2f0 2024-04-04T06:16:45.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.942+0000 7f1659400640 1 -- 172.21.15.159:0/3249344873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16540ad2f0 msgr2=0x7f16541453f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:45.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.942+0000 7f1659400640 1 --2- 172.21.15.159:0/3249344873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16540ad2f0 0x7f16541453f0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f165406c580 tx=0x7f164402bee0 comp rx=0 tx=0).stop 2024-04-04T06:16:45.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.942+0000 7f1659400640 1 -- 172.21.15.159:0/3249344873 shutdown_connections 2024-04-04T06:16:45.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.942+0000 7f1659400640 1 --2- 172.21.15.159:0/3249344873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16540ad2f0 0x7f16541453f0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:45.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.942+0000 7f1659400640 1 --2- 172.21.15.159:0/3249344873 >> v2:172.21.15.178:3300/0 conn(0x7f16540acf10 0x7f165409dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:45.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.942+0000 7f1659400640 1 -- 172.21.15.159:0/3249344873 >> 172.21.15.159:0/3249344873 conn(0x7f165406bba0 msgr2=0x7f16540b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:45.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.942+0000 7f1659400640 1 -- 172.21.15.159:0/3249344873 shutdown_connections 2024-04-04T06:16:45.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.942+0000 7f1659400640 1 -- 172.21.15.159:0/3249344873 wait complete. 2024-04-04T06:16:45.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.944+0000 7f1659400640 1 Processor -- start 2024-04-04T06:16:45.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.944+0000 7f1659400640 1 -- start start 2024-04-04T06:16:45.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.945+0000 7f1659400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16540acf10 0x7f165406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:45.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.945+0000 7f1659400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16540ad2f0 0x7f165413f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:45.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.945+0000 7f1659400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f165415ff70 0x7f16540b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:45.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.945+0000 7f1659400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f165415d430 con 0x7f16540acf10 2024-04-04T06:16:45.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.945+0000 7f1659400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f165415d7a0 con 0x7f16540ad2f0 2024-04-04T06:16:45.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.945+0000 7f1659400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f16540ad7a0 con 0x7f165415ff70 2024-04-04T06:16:45.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.945+0000 7f1652800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16540acf10 0x7f165406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:45.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.945+0000 7f1652800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16540acf10 0x7f165406d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52336/0 (socket says 172.21.15.159:52336) 2024-04-04T06:16:45.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.945+0000 7f1652800640 1 -- 172.21.15.159:0/2951256251 learned_addr learned my addr 172.21.15.159:0/2951256251 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:45.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.945+0000 7f1651e00640 1 --2- 172.21.15.159:0/2951256251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16540ad2f0 0x7f165413f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:45.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.945+0000 7f1652800640 1 -- 172.21.15.159:0/2951256251 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f165415ff70 msgr2=0x7f16540b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:45.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.945+0000 7f1652800640 1 --2- 172.21.15.159:0/2951256251 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f165415ff70 0x7f16540b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:45.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.945+0000 7f1652800640 1 -- 172.21.15.159:0/2951256251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16540ad2f0 msgr2=0x7f165413f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:45.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.945+0000 7f1652800640 1 --2- 172.21.15.159:0/2951256251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16540ad2f0 0x7f165413f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:45.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.945+0000 7f1652800640 1 -- 172.21.15.159:0/2951256251 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1654142310 con 0x7f16540acf10 2024-04-04T06:16:45.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.946+0000 7f1652800640 1 --2- 172.21.15.159:0/2951256251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16540acf10 0x7f165406d5d0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f163c00b4d0 tx=0x7f163c00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:45.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.946+0000 7f164aa00640 1 -- 172.21.15.159:0/2951256251 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c014070 con 0x7f16540acf10 2024-04-04T06:16:45.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.946+0000 7f164aa00640 1 -- 172.21.15.159:0/2951256251 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f163c010870 con 0x7f16540acf10 2024-04-04T06:16:45.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.946+0000 7f1659400640 1 -- 172.21.15.159:0/2951256251 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16541425f0 con 0x7f16540acf10 2024-04-04T06:16:45.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.946+0000 7f164aa00640 1 -- 172.21.15.159:0/2951256251 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c010c60 con 0x7f16540acf10 2024-04-04T06:16:45.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.946+0000 7f1659400640 1 -- 172.21.15.159:0/2951256251 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16541430d0 con 0x7f16540acf10 2024-04-04T06:16:45.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.947+0000 7f1659400640 1 -- 172.21.15.159:0/2951256251 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16540685c0 con 0x7f16540acf10 2024-04-04T06:16:45.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.948+0000 7f164aa00640 1 -- 172.21.15.159:0/2951256251 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f163c017070 con 0x7f16540acf10 2024-04-04T06:16:45.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.949+0000 7f164aa00640 1 --2- 172.21.15.159:0/2951256251 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f163006c8f0 0x7f163006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:45.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.949+0000 7f164aa00640 1 -- 172.21.15.159:0/2951256251 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7f163c08b970 con 0x7f16540acf10 2024-04-04T06:16:45.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.950+0000 7f1651e00640 1 --2- 172.21.15.159:0/2951256251 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f163006c8f0 0x7f163006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:45.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.951+0000 7f1651e00640 1 --2- 172.21.15.159:0/2951256251 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f163006c8f0 0x7f163006edb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1644035000 tx=0x7f1644030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:45.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:45.953+0000 7f164aa00640 1 -- 172.21.15.159:0/2951256251 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f163c059d80 con 0x7f16540acf10 2024-04-04T06:16:46.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.156+0000 7f1659400640 1 -- 172.21.15.159:0/2951256251 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f16540b3030 con 0x7f16540acf10 2024-04-04T06:16:46.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.702+0000 7f164aa00640 1 -- 172.21.15.159:0/2951256251 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v104) ==== 181+0+0 (secure 0 0 0) 0x7f163c059680 con 0x7f16540acf10 2024-04-04T06:16:46.697 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_metadata' removed 2024-04-04T06:16:46.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.708+0000 7f1659400640 1 -- 172.21.15.159:0/2951256251 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f163006c8f0 msgr2=0x7f163006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:46.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.708+0000 7f1659400640 1 --2- 172.21.15.159:0/2951256251 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f163006c8f0 0x7f163006edb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1644035000 tx=0x7f1644030000 comp rx=0 tx=0).stop 2024-04-04T06:16:46.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.708+0000 7f1659400640 1 -- 172.21.15.159:0/2951256251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16540acf10 msgr2=0x7f165406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:46.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.708+0000 7f1659400640 1 --2- 172.21.15.159:0/2951256251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16540acf10 0x7f165406d5d0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f163c00b4d0 tx=0x7f163c00b990 comp rx=0 tx=0).stop 2024-04-04T06:16:46.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.708+0000 7f1659400640 1 -- 172.21.15.159:0/2951256251 shutdown_connections 2024-04-04T06:16:46.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.708+0000 7f1659400640 1 --2- 172.21.15.159:0/2951256251 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f165415ff70 0x7f16540b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:46.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.708+0000 7f1659400640 1 --2- 172.21.15.159:0/2951256251 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f163006c8f0 0x7f163006edb0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:46.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.708+0000 7f1659400640 1 --2- 172.21.15.159:0/2951256251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16540ad2f0 0x7f165413f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:46.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.708+0000 7f1659400640 1 --2- 172.21.15.159:0/2951256251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16540acf10 0x7f165406d5d0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:46.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.708+0000 7f1659400640 1 -- 172.21.15.159:0/2951256251 >> 172.21.15.159:0/2951256251 conn(0x7f165406bba0 msgr2=0x7f1654160380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:46.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.711+0000 7f1659400640 1 -- 172.21.15.159:0/2951256251 shutdown_connections 2024-04-04T06:16:46.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.711+0000 7f1659400640 1 -- 172.21.15.159:0/2951256251 wait complete. 2024-04-04T06:16:46.733 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-04T06:16:46.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.927+0000 7f4977c00640 1 Processor -- start 2024-04-04T06:16:46.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.927+0000 7f4977c00640 1 -- start start 2024-04-04T06:16:46.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.928+0000 7f4977c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49700b3940 0x7f49700adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:46.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.928+0000 7f4977c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f49700ae4f0 0x7f49700ae930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:46.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.928+0000 7f4977c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f497006ea10 con 0x7f4970072720 2024-04-04T06:16:46.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.928+0000 7f4977c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f497006ecf0 con 0x7f49700ae4f0 2024-04-04T06:16:46.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.928+0000 7f4977c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f497006efd0 con 0x7f49700b3940 2024-04-04T06:16:46.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.929+0000 7f4975c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f49700ae4f0 0x7f49700ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:46.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.929+0000 7f4975c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f49700ae4f0 0x7f49700ae930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52350/0 (socket says 172.21.15.159:52350) 2024-04-04T06:16:46.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.929+0000 7f4975c00640 1 -- 172.21.15.159:0/4249679417 learned_addr learned my addr 172.21.15.159:0/4249679417 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:46.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.929+0000 7f4967e00640 1 --2- 172.21.15.159:0/4249679417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49700b3940 0x7f49700adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:46.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.929+0000 7f4975c00640 1 -- 172.21.15.159:0/4249679417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49700b3940 msgr2=0x7f49700adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:46.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.929+0000 7f4975c00640 1 --2- 172.21.15.159:0/4249679417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49700b3940 0x7f49700adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:46.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.929+0000 7f4975c00640 1 -- 172.21.15.159:0/4249679417 >> v1:172.21.15.178:6789/0 conn(0x7f4970072720 legacy=0x7f4970072b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:46.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.929+0000 7f4975c00640 1 -- 172.21.15.159:0/4249679417 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497006f2b0 con 0x7f49700ae4f0 2024-04-04T06:16:46.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.931+0000 7f4975c00640 1 --2- 172.21.15.159:0/4249679417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49700ae4f0 0x7f49700ae930 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f496800d150 tx=0x7f496802e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3aa51b0d70c99c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:46.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.931+0000 7f496fe00640 1 -- 172.21.15.159:0/4249679417 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496800b0d0 con 0x7f49700ae4f0 2024-04-04T06:16:46.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.931+0000 7f496fe00640 1 -- 172.21.15.159:0/4249679417 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f496800b270 con 0x7f49700ae4f0 2024-04-04T06:16:46.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.931+0000 7f496fe00640 1 -- 172.21.15.159:0/4249679417 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4968009600 con 0x7f49700ae4f0 2024-04-04T06:16:46.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.932+0000 7f4977c00640 1 -- 172.21.15.159:0/4249679417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49700ae4f0 msgr2=0x7f49700ae930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:46.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.932+0000 7f4977c00640 1 --2- 172.21.15.159:0/4249679417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49700ae4f0 0x7f49700ae930 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f496800d150 tx=0x7f496802e160 comp rx=0 tx=0).stop 2024-04-04T06:16:46.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.932+0000 7f4977c00640 1 -- 172.21.15.159:0/4249679417 shutdown_connections 2024-04-04T06:16:46.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.932+0000 7f4977c00640 1 --2- 172.21.15.159:0/4249679417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49700ae4f0 0x7f49700ae930 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:46.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.932+0000 7f4977c00640 1 --2- 172.21.15.159:0/4249679417 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49700b3940 0x7f49700adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:46.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.932+0000 7f4977c00640 1 -- 172.21.15.159:0/4249679417 >> 172.21.15.159:0/4249679417 conn(0x7f497006bba0 msgr2=0x7f49700b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:46.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.932+0000 7f4977c00640 1 -- 172.21.15.159:0/4249679417 shutdown_connections 2024-04-04T06:16:46.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.932+0000 7f4977c00640 1 -- 172.21.15.159:0/4249679417 wait complete. 2024-04-04T06:16:46.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.934+0000 7f4977c00640 1 Processor -- start 2024-04-04T06:16:46.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.934+0000 7f4977c00640 1 -- start start 2024-04-04T06:16:46.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.935+0000 7f4977c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4970072720 0x7f497009cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:46.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.935+0000 7f4977c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49700ae4f0 0x7f49701f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:46.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.935+0000 7f4977c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49700b3940 0x7f497014eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:46.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.935+0000 7f4977c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f497006e690 con 0x7f4970072720 2024-04-04T06:16:46.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.935+0000 7f4977c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f49700b3d80 con 0x7f49700b3940 2024-04-04T06:16:46.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.935+0000 7f4977c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f497015eab0 con 0x7f49700ae4f0 2024-04-04T06:16:46.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.935+0000 7f4975200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4970072720 0x7f497009cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:46.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.935+0000 7f4975200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4970072720 0x7f497009cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52366/0 (socket says 172.21.15.159:52366) 2024-04-04T06:16:46.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.935+0000 7f4975200640 1 -- 172.21.15.159:0/135384649 learned_addr learned my addr 172.21.15.159:0/135384649 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:46.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.935+0000 7f4967e00640 1 --2- 172.21.15.159:0/135384649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49700ae4f0 0x7f49701f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:46.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.935+0000 7f4975200640 1 -- 172.21.15.159:0/135384649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49700ae4f0 msgr2=0x7f49701f49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:46.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.935+0000 7f4975200640 1 --2- 172.21.15.159:0/135384649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49700ae4f0 0x7f49701f49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:46.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.935+0000 7f4975200640 1 -- 172.21.15.159:0/135384649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49700b3940 msgr2=0x7f497014eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:46.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.935+0000 7f4975200640 1 --2- 172.21.15.159:0/135384649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49700b3940 0x7f497014eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:46.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.935+0000 7f4975200640 1 -- 172.21.15.159:0/135384649 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497009d410 con 0x7f4970072720 2024-04-04T06:16:46.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.935+0000 7f4967e00640 1 --2- 172.21.15.159:0/135384649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49700ae4f0 0x7f49701f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:46.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.936+0000 7f4975200640 1 --2- 172.21.15.159:0/135384649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4970072720 0x7f497009cd00 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f496000c3c0 tx=0x7f496000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:46.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.936+0000 7f496d600640 1 -- 172.21.15.159:0/135384649 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49600052d0 con 0x7f4970072720 2024-04-04T06:16:46.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.936+0000 7f496d600640 1 -- 172.21.15.159:0/135384649 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4960005490 con 0x7f4970072720 2024-04-04T06:16:46.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.936+0000 7f496d600640 1 -- 172.21.15.159:0/135384649 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960007260 con 0x7f4970072720 2024-04-04T06:16:46.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.936+0000 7f4977c00640 1 -- 172.21.15.159:0/135384649 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f497009d6f0 con 0x7f4970072720 2024-04-04T06:16:46.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.936+0000 7f4977c00640 1 -- 172.21.15.159:0/135384649 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f497014f0b0 con 0x7f4970072720 2024-04-04T06:16:46.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.937+0000 7f4977c00640 1 -- 172.21.15.159:0/135384649 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49700685c0 con 0x7f4970072720 2024-04-04T06:16:46.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.938+0000 7f496d600640 1 -- 172.21.15.159:0/135384649 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4960007400 con 0x7f4970072720 2024-04-04T06:16:46.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.939+0000 7f496d600640 1 --2- 172.21.15.159:0/135384649 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f494c06c8f0 0x7f494c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:46.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.939+0000 7f496d600640 1 -- 172.21.15.159:0/135384649 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5443+0+0 (secure 0 0 0) 0x7f496008aec0 con 0x7f4970072720 2024-04-04T06:16:46.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.940+0000 7f4967e00640 1 --2- 172.21.15.159:0/135384649 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f494c06c8f0 0x7f494c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:46.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.941+0000 7f4967e00640 1 --2- 172.21.15.159:0/135384649 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f494c06c8f0 0x7f494c06edb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4970154dc0 tx=0x7f4958008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:46.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:46.943+0000 7f496d600640 1 -- 172.21.15.159:0/135384649 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4960059420 con 0x7f4970072720 2024-04-04T06:16:47.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.144+0000 7f4977c00640 1 -- 172.21.15.159:0/135384649 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f49700cc4d0 con 0x7f4970072720 2024-04-04T06:16:47.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.713+0000 7f496d600640 1 -- 172.21.15.159:0/135384649 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v105) ==== 169+0+0 (secure 0 0 0) 0x7f4960058d20 con 0x7f4970072720 2024-04-04T06:16:47.708 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_data' removed 2024-04-04T06:16:47.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.716+0000 7f4977c00640 1 -- 172.21.15.159:0/135384649 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f494c06c8f0 msgr2=0x7f494c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:47.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.716+0000 7f4977c00640 1 --2- 172.21.15.159:0/135384649 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f494c06c8f0 0x7f494c06edb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4970154dc0 tx=0x7f4958008040 comp rx=0 tx=0).stop 2024-04-04T06:16:47.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.716+0000 7f4977c00640 1 -- 172.21.15.159:0/135384649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4970072720 msgr2=0x7f497009cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:47.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.716+0000 7f4977c00640 1 --2- 172.21.15.159:0/135384649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4970072720 0x7f497009cd00 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f496000c3c0 tx=0x7f496000c880 comp rx=0 tx=0).stop 2024-04-04T06:16:47.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.716+0000 7f4977c00640 1 -- 172.21.15.159:0/135384649 shutdown_connections 2024-04-04T06:16:47.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.716+0000 7f4977c00640 1 --2- 172.21.15.159:0/135384649 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f494c06c8f0 0x7f494c06edb0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:47.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.716+0000 7f4977c00640 1 --2- 172.21.15.159:0/135384649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49700b3940 0x7f497014eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:47.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.716+0000 7f4977c00640 1 --2- 172.21.15.159:0/135384649 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49700ae4f0 0x7f49701f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:47.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.716+0000 7f4977c00640 1 --2- 172.21.15.159:0/135384649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4970072720 0x7f497009cd00 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:47.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.716+0000 7f4977c00640 1 -- 172.21.15.159:0/135384649 >> 172.21.15.159:0/135384649 conn(0x7f497006bba0 msgr2=0x7f49700b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:47.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.716+0000 7f4977c00640 1 -- 172.21.15.159:0/135384649 shutdown_connections 2024-04-04T06:16:47.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.716+0000 7f4977c00640 1 -- 172.21.15.159:0/135384649 wait complete. 2024-04-04T06:16:47.740 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-04T06:16:47.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.955+0000 7f2124a00640 1 Processor -- start 2024-04-04T06:16:47.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.955+0000 7f2124a00640 1 -- start start 2024-04-04T06:16:47.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.956+0000 7f2124a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2120155550 0x7f2120155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:47.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.956+0000 7f2124a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f21200d0990 0x7f21200aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:47.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.956+0000 7f2124a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21200b40b0 con 0x7f212015f800 2024-04-04T06:16:47.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.957+0000 7f2124a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f21200b4390 con 0x7f21200d0990 2024-04-04T06:16:47.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.957+0000 7f2124a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f21200b4670 con 0x7f2120155550 2024-04-04T06:16:47.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.957+0000 7f211e800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f21200d0990 0x7f21200aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:47.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.957+0000 7f211de00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2120155550 0x7f2120155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:47.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.957+0000 7f211e800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f21200d0990 0x7f21200aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:58830/0 (socket says 172.21.15.159:58830) 2024-04-04T06:16:47.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.957+0000 7f211e800640 1 -- 172.21.15.159:0/2948190521 learned_addr learned my addr 172.21.15.159:0/2948190521 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:47.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.958+0000 7f211e800640 1 -- 172.21.15.159:0/2948190521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2120155550 msgr2=0x7f2120155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:47.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.958+0000 7f211e800640 1 --2- 172.21.15.159:0/2948190521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2120155550 0x7f2120155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:47.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.958+0000 7f211e800640 1 -- 172.21.15.159:0/2948190521 >> v1:172.21.15.178:6789/0 conn(0x7f212015f800 legacy=0x7f21200d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:47.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.958+0000 7f211e800640 1 -- 172.21.15.159:0/2948190521 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21200b4950 con 0x7f21200d0990 2024-04-04T06:16:47.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.959+0000 7f211e800640 1 --2- 172.21.15.159:0/2948190521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21200d0990 0x7f21200aede0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2118009e50 tx=0x7f211802bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=321efbdfd7d92576 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:47.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.960+0000 7f211d400640 1 -- 172.21.15.159:0/2948190521 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211802ca80 con 0x7f21200d0990 2024-04-04T06:16:47.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.960+0000 7f211d400640 1 -- 172.21.15.159:0/2948190521 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f211802cc20 con 0x7f21200d0990 2024-04-04T06:16:47.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.960+0000 7f211d400640 1 -- 172.21.15.159:0/2948190521 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21180343f0 con 0x7f21200d0990 2024-04-04T06:16:47.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.961+0000 7f2124a00640 1 -- 172.21.15.159:0/2948190521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21200d0990 msgr2=0x7f21200aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:47.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.961+0000 7f2124a00640 1 --2- 172.21.15.159:0/2948190521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21200d0990 0x7f21200aede0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2118009e50 tx=0x7f211802bee0 comp rx=0 tx=0).stop 2024-04-04T06:16:47.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.961+0000 7f2124a00640 1 -- 172.21.15.159:0/2948190521 shutdown_connections 2024-04-04T06:16:47.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.961+0000 7f2124a00640 1 --2- 172.21.15.159:0/2948190521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21200d0990 0x7f21200aede0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:47.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.961+0000 7f2124a00640 1 --2- 172.21.15.159:0/2948190521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2120155550 0x7f2120155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:47.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.961+0000 7f2124a00640 1 -- 172.21.15.159:0/2948190521 >> 172.21.15.159:0/2948190521 conn(0x7f212006b750 msgr2=0x7f212006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:47.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.961+0000 7f2124a00640 1 -- 172.21.15.159:0/2948190521 shutdown_connections 2024-04-04T06:16:47.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.961+0000 7f2124a00640 1 -- 172.21.15.159:0/2948190521 wait complete. 2024-04-04T06:16:47.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.962+0000 7f2124a00640 1 Processor -- start 2024-04-04T06:16:47.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.963+0000 7f2124a00640 1 -- start start 2024-04-04T06:16:47.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.963+0000 7f2124a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21200d0990 0x7f21200982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:47.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.964+0000 7f2124a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2120155550 0x7f2120099fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:47.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.964+0000 7f2124a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f212015f800 0x7f2120149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:47.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.964+0000 7f2124a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f212015ba20 con 0x7f2120155550 2024-04-04T06:16:47.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.964+0000 7f2124a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f212015fc80 con 0x7f212015f800 2024-04-04T06:16:47.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.964+0000 7f2124a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f212015bba0 con 0x7f21200d0990 2024-04-04T06:16:47.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.964+0000 7f2115400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2120155550 0x7f2120099fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:47.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.964+0000 7f2115400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2120155550 0x7f2120099fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52382/0 (socket says 172.21.15.159:52382) 2024-04-04T06:16:47.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.964+0000 7f2115400640 1 -- 172.21.15.159:0/2054056632 learned_addr learned my addr 172.21.15.159:0/2054056632 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:47.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.964+0000 7f211de00640 1 --2- 172.21.15.159:0/2054056632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21200d0990 0x7f21200982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:47.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.964+0000 7f211e800640 1 --2- 172.21.15.159:0/2054056632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f212015f800 0x7f2120149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:47.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.964+0000 7f2115400640 1 -- 172.21.15.159:0/2054056632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21200d0990 msgr2=0x7f21200982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:47.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.964+0000 7f2115400640 1 --2- 172.21.15.159:0/2054056632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21200d0990 0x7f21200982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:47.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.964+0000 7f2115400640 1 -- 172.21.15.159:0/2054056632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f212015f800 msgr2=0x7f2120149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:47.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.964+0000 7f2115400640 1 --2- 172.21.15.159:0/2054056632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f212015f800 0x7f2120149150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:47.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.964+0000 7f2115400640 1 -- 172.21.15.159:0/2054056632 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21200afb80 con 0x7f2120155550 2024-04-04T06:16:47.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.965+0000 7f2115400640 1 --2- 172.21.15.159:0/2054056632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2120155550 0x7f2120099fe0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f210800c460 tx=0x7f210800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:47.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.965+0000 7f2116a00640 1 -- 172.21.15.159:0/2054056632 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21080052d0 con 0x7f2120155550 2024-04-04T06:16:47.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.965+0000 7f2116a00640 1 -- 172.21.15.159:0/2054056632 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2108012630 con 0x7f2120155550 2024-04-04T06:16:47.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.965+0000 7f2124a00640 1 -- 172.21.15.159:0/2054056632 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21200afe60 con 0x7f2120155550 2024-04-04T06:16:47.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.965+0000 7f2116a00640 1 -- 172.21.15.159:0/2054056632 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21080054b0 con 0x7f2120155550 2024-04-04T06:16:47.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.965+0000 7f2124a00640 1 -- 172.21.15.159:0/2054056632 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21200b0420 con 0x7f2120155550 2024-04-04T06:16:47.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.966+0000 7f2124a00640 1 -- 172.21.15.159:0/2054056632 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2120149f10 con 0x7f2120155550 2024-04-04T06:16:47.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.967+0000 7f2116a00640 1 -- 172.21.15.159:0/2054056632 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2108012a20 con 0x7f2120155550 2024-04-04T06:16:47.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.968+0000 7f2116a00640 1 --2- 172.21.15.159:0/2054056632 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f20fc06c8f0 0x7f20fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:47.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.968+0000 7f2116a00640 1 -- 172.21.15.159:0/2054056632 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5029+0+0 (secure 0 0 0) 0x7f210808b470 con 0x7f2120155550 2024-04-04T06:16:47.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.969+0000 7f211de00640 1 --2- 172.21.15.159:0/2054056632 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f20fc06c8f0 0x7f20fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:47.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.970+0000 7f211de00640 1 --2- 172.21.15.159:0/2054056632 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f20fc06c8f0 0x7f20fc06edb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2110009ad0 tx=0x7f2110008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:47.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:47.973+0000 7f2116a00640 1 -- 172.21.15.159:0/2054056632 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2108059b40 con 0x7f2120155550 2024-04-04T06:16:48.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.175+0000 7f2124a00640 1 -- 172.21.15.159:0/2054056632 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f21200685c0 con 0x7f2120155550 2024-04-04T06:16:48.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.181+0000 7f2116a00640 1 -- 172.21.15.159:0/2054056632 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f2108059440 con 0x7f2120155550 2024-04-04T06:16:48.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.183+0000 7f2124a00640 1 -- 172.21.15.159:0/2054056632 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f20fc06c8f0 msgr2=0x7f20fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:48.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.183+0000 7f2124a00640 1 --2- 172.21.15.159:0/2054056632 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f20fc06c8f0 0x7f20fc06edb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2110009ad0 tx=0x7f2110008040 comp rx=0 tx=0).stop 2024-04-04T06:16:48.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.183+0000 7f2124a00640 1 -- 172.21.15.159:0/2054056632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2120155550 msgr2=0x7f2120099fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:48.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.183+0000 7f2124a00640 1 --2- 172.21.15.159:0/2054056632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2120155550 0x7f2120099fe0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f210800c460 tx=0x7f210800c920 comp rx=0 tx=0).stop 2024-04-04T06:16:48.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.183+0000 7f2124a00640 1 -- 172.21.15.159:0/2054056632 shutdown_connections 2024-04-04T06:16:48.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.183+0000 7f2124a00640 1 --2- 172.21.15.159:0/2054056632 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f20fc06c8f0 0x7f20fc06edb0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:48.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.183+0000 7f2124a00640 1 --2- 172.21.15.159:0/2054056632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f212015f800 0x7f2120149150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:48.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.183+0000 7f2124a00640 1 --2- 172.21.15.159:0/2054056632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2120155550 0x7f2120099fe0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:48.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.183+0000 7f2124a00640 1 --2- 172.21.15.159:0/2054056632 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21200d0990 0x7f21200982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:48.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.183+0000 7f2124a00640 1 -- 172.21.15.159:0/2054056632 >> 172.21.15.159:0/2054056632 conn(0x7f212006b750 msgr2=0x7f212015cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:48.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.184+0000 7f2124a00640 1 -- 172.21.15.159:0/2054056632 shutdown_connections 2024-04-04T06:16:48.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.184+0000 7f2124a00640 1 -- 172.21.15.159:0/2054056632 wait complete. 2024-04-04T06:16:48.196 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-04T06:16:48.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.405+0000 7f0fb7400640 1 Processor -- start 2024-04-04T06:16:48.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.405+0000 7f0fb7400640 1 -- start start 2024-04-04T06:16:48.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.406+0000 7f0fb7400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0fb00d0990 0x7f0fb00aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:48.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.406+0000 7f0fb7400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0fb00af2e0 0x7f0fb01600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:48.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.406+0000 7f0fb7400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fb00afab0 con 0x7f0fb0153580 2024-04-04T06:16:48.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.406+0000 7f0fb7400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f0fb015cc80 con 0x7f0fb00d0990 2024-04-04T06:16:48.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.406+0000 7f0fb7400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0fb015da00 con 0x7f0fb00af2e0 2024-04-04T06:16:48.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.406+0000 7f0fafe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0fb00d0990 0x7f0fb00aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:48.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.407+0000 7f0fb4a00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f0fb0153580 0x7f0fb00d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40312/0 (socket says 172.21.15.159:40312) 2024-04-04T06:16:48.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.407+0000 7f0fb4a00640 1 -- 172.21.15.159:0/1309017950 learned_addr learned my addr 172.21.15.159:0/1309017950 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:16:48.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.407+0000 7f0fafe00640 1 -- 172.21.15.159:0/1309017950 >> v2:172.21.15.178:3300/0 conn(0x7f0fb00af2e0 msgr2=0x7f0fb01600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:48.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.407+0000 7f0fafe00640 1 --2- 172.21.15.159:0/1309017950 >> v2:172.21.15.178:3300/0 conn(0x7f0fb00af2e0 0x7f0fb01600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:48.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.407+0000 7f0fafe00640 1 -- 172.21.15.159:0/1309017950 >> v1:172.21.15.159:6789/0 conn(0x7f0fb0153580 legacy=0x7f0fb00d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:48.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.407+0000 7f0fafe00640 1 -- 172.21.15.159:0/1309017950 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb015cf20 con 0x7f0fb00d0990 2024-04-04T06:16:48.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.408+0000 7f0fafe00640 1 --2- 172.21.15.159:0/1309017950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fb00d0990 0x7f0fb00aeda0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f0fb006efa0 tx=0x7f0f9c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34313f51c9a3fc8a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:48.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.408+0000 7f0faf400640 1 -- 172.21.15.159:0/1309017950 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c0315a0 con 0x7f0fb00d0990 2024-04-04T06:16:48.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.408+0000 7f0faf400640 1 -- 172.21.15.159:0/1309017950 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f9c037070 con 0x7f0fb00d0990 2024-04-04T06:16:48.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.408+0000 7f0faf400640 1 -- 172.21.15.159:0/1309017950 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c031960 con 0x7f0fb00d0990 2024-04-04T06:16:48.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.409+0000 7f0fb7400640 1 -- 172.21.15.159:0/1309017950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fb00d0990 msgr2=0x7f0fb00aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:48.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.409+0000 7f0fb7400640 1 --2- 172.21.15.159:0/1309017950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fb00d0990 0x7f0fb00aeda0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f0fb006efa0 tx=0x7f0f9c02b840 comp rx=0 tx=0).stop 2024-04-04T06:16:48.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.409+0000 7f0fb7400640 1 -- 172.21.15.159:0/1309017950 shutdown_connections 2024-04-04T06:16:48.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.409+0000 7f0fb7400640 1 --2- 172.21.15.159:0/1309017950 >> v2:172.21.15.178:3300/0 conn(0x7f0fb00af2e0 0x7f0fb01600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:48.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.409+0000 7f0fb7400640 1 --2- 172.21.15.159:0/1309017950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fb00d0990 0x7f0fb00aeda0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:48.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.409+0000 7f0fb7400640 1 -- 172.21.15.159:0/1309017950 >> 172.21.15.159:0/1309017950 conn(0x7f0fb006b750 msgr2=0x7f0fb006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:48.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.409+0000 7f0fb7400640 1 -- 172.21.15.159:0/1309017950 shutdown_connections 2024-04-04T06:16:48.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.409+0000 7f0fb7400640 1 -- 172.21.15.159:0/1309017950 wait complete. 2024-04-04T06:16:48.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.411+0000 7f0fb7400640 1 Processor -- start 2024-04-04T06:16:48.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.411+0000 7f0fb7400640 1 -- start start 2024-04-04T06:16:48.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.412+0000 7f0fb7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fb00af2e0 0x7f0fb014f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:48.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.412+0000 7f0fb7400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fb00d0990 0x7f0fb014e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:48.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.412+0000 7f0fb7400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fb0153580 0x7f0fb00cee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:48.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.412+0000 7f0fb7400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb0153960 con 0x7f0fb00af2e0 2024-04-04T06:16:48.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.412+0000 7f0fb7400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb015d4e0 con 0x7f0fb00d0990 2024-04-04T06:16:48.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.412+0000 7f0fb7400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0fb00af6c0 con 0x7f0fb0153580 2024-04-04T06:16:48.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.412+0000 7f0fb4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fb00af2e0 0x7f0fb014f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:48.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.412+0000 7f0fb4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fb00af2e0 0x7f0fb014f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52402/0 (socket says 172.21.15.159:52402) 2024-04-04T06:16:48.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.412+0000 7f0fb4a00640 1 -- 172.21.15.159:0/1244670017 learned_addr learned my addr 172.21.15.159:0/1244670017 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:48.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.413+0000 7f0fafe00640 1 --2- 172.21.15.159:0/1244670017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fb00d0990 0x7f0fb014e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:48.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.413+0000 7f0fb4a00640 1 -- 172.21.15.159:0/1244670017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fb0153580 msgr2=0x7f0fb00cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:48.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.413+0000 7f0fb4a00640 1 --2- 172.21.15.159:0/1244670017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fb0153580 0x7f0fb00cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:48.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.413+0000 7f0fb4a00640 1 -- 172.21.15.159:0/1244670017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fb00d0990 msgr2=0x7f0fb014e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:48.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.413+0000 7f0fb4a00640 1 --2- 172.21.15.159:0/1244670017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fb00d0990 0x7f0fb014e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:48.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.413+0000 7f0fb4a00640 1 -- 172.21.15.159:0/1244670017 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb0150a10 con 0x7f0fb00af2e0 2024-04-04T06:16:48.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.413+0000 7f0fb4a00640 1 --2- 172.21.15.159:0/1244670017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fb00af2e0 0x7f0fb014f8b0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400c400 tx=0x7f0fa400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:48.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.413+0000 7f0facc00640 1 -- 172.21.15.159:0/1244670017 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4017070 con 0x7f0fb00af2e0 2024-04-04T06:16:48.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.413+0000 7f0facc00640 1 -- 172.21.15.159:0/1244670017 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa4005ce0 con 0x7f0fb00af2e0 2024-04-04T06:16:48.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.413+0000 7f0facc00640 1 -- 172.21.15.159:0/1244670017 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa40072f0 con 0x7f0fb00af2e0 2024-04-04T06:16:48.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.413+0000 7f0fb7400640 1 -- 172.21.15.159:0/1244670017 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb014fe90 con 0x7f0fb00af2e0 2024-04-04T06:16:48.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.414+0000 7f0fb7400640 1 -- 172.21.15.159:0/1244670017 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb0150450 con 0x7f0fb00af2e0 2024-04-04T06:16:48.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.415+0000 7f0fb7400640 1 -- 172.21.15.159:0/1244670017 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fb00685c0 con 0x7f0fb00af2e0 2024-04-04T06:16:48.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.416+0000 7f0facc00640 1 -- 172.21.15.159:0/1244670017 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0fa401a070 con 0x7f0fb00af2e0 2024-04-04T06:16:48.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.417+0000 7f0facc00640 1 --2- 172.21.15.159:0/1244670017 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f9006c8f0 0x7f0f9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:48.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.417+0000 7f0facc00640 1 -- 172.21.15.159:0/1244670017 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5029+0+0 (secure 0 0 0) 0x7f0fa408aca0 con 0x7f0fb00af2e0 2024-04-04T06:16:48.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.418+0000 7f0fafe00640 1 --2- 172.21.15.159:0/1244670017 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f9006c8f0 0x7f0f9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:48.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.419+0000 7f0fafe00640 1 --2- 172.21.15.159:0/1244670017 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f9006c8f0 0x7f0f9006edb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c036000 tx=0x7f0f9c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:48.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.422+0000 7f0facc00640 1 -- 172.21.15.159:0/1244670017 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0fa4059370 con 0x7f0fb00af2e0 2024-04-04T06:16:48.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.699+0000 7f0fb7400640 1 -- 172.21.15.159:0/1244670017 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f0fb006eca0 con 0x7f0fb00af2e0 2024-04-04T06:16:48.710 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.716+0000 7f0facc00640 1 -- 172.21.15.159:0/1244670017 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v519) ==== 146+0+0 (secure 0 0 0) 0x7f0fa405f070 con 0x7f0fb00af2e0 2024-04-04T06:16:48.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.718+0000 7f0fb7400640 1 -- 172.21.15.159:0/1244670017 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f9006c8f0 msgr2=0x7f0f9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:48.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.718+0000 7f0fb7400640 1 --2- 172.21.15.159:0/1244670017 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f9006c8f0 0x7f0f9006edb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c036000 tx=0x7f0f9c008030 comp rx=0 tx=0).stop 2024-04-04T06:16:48.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.718+0000 7f0fb7400640 1 -- 172.21.15.159:0/1244670017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fb00af2e0 msgr2=0x7f0fb014f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:48.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.718+0000 7f0fb7400640 1 --2- 172.21.15.159:0/1244670017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fb00af2e0 0x7f0fb014f8b0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400c400 tx=0x7f0fa400c8c0 comp rx=0 tx=0).stop 2024-04-04T06:16:48.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.718+0000 7f0fb7400640 1 -- 172.21.15.159:0/1244670017 shutdown_connections 2024-04-04T06:16:48.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.718+0000 7f0fb7400640 1 --2- 172.21.15.159:0/1244670017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0fb0153580 0x7f0fb00cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:48.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.718+0000 7f0fb7400640 1 --2- 172.21.15.159:0/1244670017 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f9006c8f0 0x7f0f9006edb0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:48.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.718+0000 7f0fb7400640 1 --2- 172.21.15.159:0/1244670017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fb00d0990 0x7f0fb014e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:48.715 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.718+0000 7f0fb7400640 1 --2- 172.21.15.159:0/1244670017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fb00af2e0 0x7f0fb014f8b0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:48.715 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.718+0000 7f0fb7400640 1 -- 172.21.15.159:0/1244670017 >> 172.21.15.159:0/1244670017 conn(0x7f0fb006b750 msgr2=0x7f0fb00d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:48.715 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.718+0000 7f0fb7400640 1 -- 172.21.15.159:0/1244670017 shutdown_connections 2024-04-04T06:16:48.715 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.718+0000 7f0fb7400640 1 -- 172.21.15.159:0/1244670017 wait complete. 2024-04-04T06:16:48.733 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-04T06:16:48.734 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-04T06:16:48.734 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:16:48.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.939+0000 7f608cc00640 1 Processor -- start 2024-04-04T06:16:48.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.939+0000 7f608cc00640 1 -- start start 2024-04-04T06:16:48.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.941+0000 7f608cc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f608815b310 0x7f60880727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:48.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.941+0000 7f608cc00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f60880b4f90 0x7f608815f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:48.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.941+0000 7f608cc00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f608815b6f0 con 0x7f60880d0780 2024-04-04T06:16:48.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.941+0000 7f608cc00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f6088072e80 con 0x7f608815b310 2024-04-04T06:16:48.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.941+0000 7f608cc00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f6088073160 con 0x7f60880b4f90 2024-04-04T06:16:48.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.941+0000 7f6085e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f608815b310 0x7f60880727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:48.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.941+0000 7f6086800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f60880d0780 0x7f60880adc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40316/0 (socket says 172.21.15.159:40316) 2024-04-04T06:16:48.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.941+0000 7f6086800640 1 -- 172.21.15.159:0/3366938166 learned_addr learned my addr 172.21.15.159:0/3366938166 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:16:48.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.941+0000 7f6085e00640 1 -- 172.21.15.159:0/3366938166 >> v2:172.21.15.178:3300/0 conn(0x7f60880b4f90 msgr2=0x7f608815f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:48.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.941+0000 7f6085400640 1 --2- 172.21.15.159:0/3366938166 >> v2:172.21.15.178:3300/0 conn(0x7f60880b4f90 0x7f608815f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:48.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.942+0000 7f6085e00640 1 --2- 172.21.15.159:0/3366938166 >> v2:172.21.15.178:3300/0 conn(0x7f60880b4f90 0x7f608815f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:48.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.942+0000 7f6085e00640 1 -- 172.21.15.159:0/3366938166 >> v1:172.21.15.159:6789/0 conn(0x7f60880d0780 legacy=0x7f60880adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:48.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.942+0000 7f6085e00640 1 -- 172.21.15.159:0/3366938166 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f608815efe0 con 0x7f608815b310 2024-04-04T06:16:48.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.942+0000 7f6084a00640 1 -- 172.21.15.159:0/3366938166 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 907248149 0 0) 0x7f608815b6f0 con 0x7f60880d0780 2024-04-04T06:16:48.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.943+0000 7f6085e00640 1 --2- 172.21.15.159:0/3366938166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f608815b310 0x7f60880727f0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f6088071200 tx=0x7f607802b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddd91abc397e27a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:48.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.943+0000 7f6084a00640 1 -- 172.21.15.159:0/3366938166 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078038070 con 0x7f608815b310 2024-04-04T06:16:48.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.943+0000 7f6084a00640 1 -- 172.21.15.159:0/3366938166 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6078031710 con 0x7f608815b310 2024-04-04T06:16:48.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.943+0000 7f6084a00640 1 -- 172.21.15.159:0/3366938166 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078031ae0 con 0x7f608815b310 2024-04-04T06:16:48.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.944+0000 7f608cc00640 1 -- 172.21.15.159:0/3366938166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f608815b310 msgr2=0x7f60880727f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:48.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.944+0000 7f608cc00640 1 --2- 172.21.15.159:0/3366938166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f608815b310 0x7f60880727f0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f6088071200 tx=0x7f607802b7b0 comp rx=0 tx=0).stop 2024-04-04T06:16:48.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.944+0000 7f608cc00640 1 -- 172.21.15.159:0/3366938166 shutdown_connections 2024-04-04T06:16:48.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.944+0000 7f608cc00640 1 --2- 172.21.15.159:0/3366938166 >> v2:172.21.15.178:3300/0 conn(0x7f60880b4f90 0x7f608815f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:48.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.944+0000 7f608cc00640 1 --2- 172.21.15.159:0/3366938166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f608815b310 0x7f60880727f0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:48.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.944+0000 7f608cc00640 1 -- 172.21.15.159:0/3366938166 >> 172.21.15.159:0/3366938166 conn(0x7f608806b990 msgr2=0x7f608806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:48.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.944+0000 7f608cc00640 1 -- 172.21.15.159:0/3366938166 shutdown_connections 2024-04-04T06:16:48.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.944+0000 7f608cc00640 1 -- 172.21.15.159:0/3366938166 wait complete. 2024-04-04T06:16:48.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.946+0000 7f608cc00640 1 Processor -- start 2024-04-04T06:16:48.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.946+0000 7f608cc00640 1 -- start start 2024-04-04T06:16:48.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.947+0000 7f608cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60880b4f90 0x7f60880cf930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:48.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.947+0000 7f608cc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60880d0780 0x7f6088147e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:48.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.947+0000 7f608cc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f608815b310 0x7f608815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:48.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.947+0000 7f608cc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6088073440 con 0x7f60880b4f90 2024-04-04T06:16:48.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.947+0000 7f608cc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f60880b53f0 con 0x7f60880d0780 2024-04-04T06:16:48.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.947+0000 7f608cc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f608815fd20 con 0x7f608815b310 2024-04-04T06:16:48.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.947+0000 7f6085e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60880b4f90 0x7f60880cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:48.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.947+0000 7f6085e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60880b4f90 0x7f60880cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52420/0 (socket says 172.21.15.159:52420) 2024-04-04T06:16:48.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.947+0000 7f6085e00640 1 -- 172.21.15.159:0/1473907566 learned_addr learned my addr 172.21.15.159:0/1473907566 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:48.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.948+0000 7f6085e00640 1 -- 172.21.15.159:0/1473907566 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f608815b310 msgr2=0x7f608815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:48.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.948+0000 7f6085e00640 1 --2- 172.21.15.159:0/1473907566 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f608815b310 0x7f608815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:48.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.948+0000 7f6085e00640 1 -- 172.21.15.159:0/1473907566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60880d0780 msgr2=0x7f6088147e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:48.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.948+0000 7f6085e00640 1 --2- 172.21.15.159:0/1473907566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60880d0780 0x7f6088147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:48.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.948+0000 7f6085e00640 1 -- 172.21.15.159:0/1473907566 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f608815b940 con 0x7f60880b4f90 2024-04-04T06:16:48.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.948+0000 7f6085e00640 1 --2- 172.21.15.159:0/1473907566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60880b4f90 0x7f60880cf930 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f6078036040 tx=0x7f60780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:48.948 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.948+0000 7f607e000640 1 -- 172.21.15.159:0/1473907566 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078038040 con 0x7f60880b4f90 2024-04-04T06:16:48.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.948+0000 7f607e000640 1 -- 172.21.15.159:0/1473907566 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6078033a40 con 0x7f60880b4f90 2024-04-04T06:16:48.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.948+0000 7f607e000640 1 -- 172.21.15.159:0/1473907566 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078033e10 con 0x7f60880b4f90 2024-04-04T06:16:48.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.949+0000 7f608cc00640 1 -- 172.21.15.159:0/1473907566 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f608815bc20 con 0x7f60880b4f90 2024-04-04T06:16:48.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.949+0000 7f608cc00640 1 -- 172.21.15.159:0/1473907566 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f608814abf0 con 0x7f60880b4f90 2024-04-04T06:16:48.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.950+0000 7f608cc00640 1 -- 172.21.15.159:0/1473907566 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60880685c0 con 0x7f60880b4f90 2024-04-04T06:16:48.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.951+0000 7f607e000640 1 -- 172.21.15.159:0/1473907566 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f607803b070 con 0x7f60880b4f90 2024-04-04T06:16:48.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.951+0000 7f607e000640 1 --2- 172.21.15.159:0/1473907566 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f606806c8f0 0x7f606806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:48.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.952+0000 7f607e000640 1 -- 172.21.15.159:0/1473907566 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5029+0+0 (secure 0 0 0) 0x7f60780ad630 con 0x7f60880b4f90 2024-04-04T06:16:48.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.952+0000 7f6085400640 1 --2- 172.21.15.159:0/1473907566 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f606806c8f0 0x7f606806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:48.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.953+0000 7f6085400640 1 --2- 172.21.15.159:0/1473907566 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f606806c8f0 0x7f606806edb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f608809e3b0 tx=0x7f6074008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:48.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:48.956+0000 7f607e000640 1 -- 172.21.15.159:0/1473907566 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f6078042020 con 0x7f60880b4f90 2024-04-04T06:16:49.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.158+0000 7f608cc00640 1 -- 172.21.15.159:0/1473907566 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f608815ec20 con 0x7f60880b4f90 2024-04-04T06:16:49.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.159+0000 7f607e000640 1 -- 172.21.15.159:0/1473907566 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+13168 (secure 0 0 0) 0x7f607807bc60 con 0x7f60880b4f90 2024-04-04T06:16:49.155 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:49.155 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":105,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:16:47.703825+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6835/1003751501":"2024-04-05T06:16:37.034044+0000","172.21.15.159:6835/2253383212":"2024-04-05T06:15:59.385487+0000","172.21.15.159:6834/1003751501":"2024-04-05T06:16:37.034044+0000","172.21.15.159:6834/2253383212":"2024-04-05T06:15:59.385487+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:16:49.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.162+0000 7f608cc00640 1 -- 172.21.15.159:0/1473907566 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f606806c8f0 msgr2=0x7f606806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:49.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.162+0000 7f608cc00640 1 --2- 172.21.15.159:0/1473907566 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f606806c8f0 0x7f606806edb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f608809e3b0 tx=0x7f6074008040 comp rx=0 tx=0).stop 2024-04-04T06:16:49.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.162+0000 7f608cc00640 1 -- 172.21.15.159:0/1473907566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60880b4f90 msgr2=0x7f60880cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:49.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.162+0000 7f608cc00640 1 --2- 172.21.15.159:0/1473907566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60880b4f90 0x7f60880cf930 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f6078036040 tx=0x7f60780046c0 comp rx=0 tx=0).stop 2024-04-04T06:16:49.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.162+0000 7f608cc00640 1 -- 172.21.15.159:0/1473907566 shutdown_connections 2024-04-04T06:16:49.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.162+0000 7f608cc00640 1 --2- 172.21.15.159:0/1473907566 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f608815b310 0x7f608815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:49.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.162+0000 7f608cc00640 1 --2- 172.21.15.159:0/1473907566 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f606806c8f0 0x7f606806edb0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:49.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.162+0000 7f608cc00640 1 --2- 172.21.15.159:0/1473907566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60880d0780 0x7f6088147e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:49.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.162+0000 7f608cc00640 1 --2- 172.21.15.159:0/1473907566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60880b4f90 0x7f60880cf930 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:49.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.162+0000 7f608cc00640 1 -- 172.21.15.159:0/1473907566 >> 172.21.15.159:0/1473907566 conn(0x7f608806b990 msgr2=0x7f6088156900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:49.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.163+0000 7f608cc00640 1 -- 172.21.15.159:0/1473907566 shutdown_connections 2024-04-04T06:16:49.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.163+0000 7f608cc00640 1 -- 172.21.15.159:0/1473907566 wait complete. 2024-04-04T06:16:49.176 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:16:49.177 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:16:49.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.374+0000 7f9aa3200640 1 Processor -- start 2024-04-04T06:16:49.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.374+0000 7f9aa3200640 1 -- start start 2024-04-04T06:16:49.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.376+0000 7f9aa3200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9aa415e4c0 0x7f9aa40b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:49.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.376+0000 7f9aa3200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aa40d2710 con 0x7f9aa40d37b0 2024-04-04T06:16:49.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.376+0000 7f9aa3200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aa40d29f0 con 0x7f9aa40b69c0 2024-04-04T06:16:49.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.376+0000 7f9aa3200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f9aa415e8a0 con 0x7f9aa415e4c0 2024-04-04T06:16:49.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.376+0000 7f9aa1400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f9aa40d37b0 0x7f9aa40b62b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40330/0 (socket says 172.21.15.159:40330) 2024-04-04T06:16:49.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.376+0000 7f9aa1400640 1 -- 172.21.15.159:0/1218603332 learned_addr learned my addr 172.21.15.159:0/1218603332 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:16:49.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.376+0000 7f9aa1e00640 1 --2- 172.21.15.159:0/1218603332 >> v2:172.21.15.159:3300/0 conn(0x7f9aa415e4c0 0x7f9aa40b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:49.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.377+0000 7f9aa0a00640 1 -- 172.21.15.159:0/1218603332 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1482915705 0 0) 0x7f9aa40d2710 con 0x7f9aa40d37b0 2024-04-04T06:16:49.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.377+0000 7f9aa0a00640 1 -- 172.21.15.159:0/1218603332 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a88003520 con 0x7f9aa40d37b0 2024-04-04T06:16:49.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.377+0000 7f9aa1e00640 1 -- 172.21.15.159:0/1218603332 >> v1:172.21.15.178:6789/0 conn(0x7f9aa40b69c0 legacy=0x7f9aa40b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:49.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.377+0000 7f9aa1e00640 1 -- 172.21.15.159:0/1218603332 >> v1:172.21.15.159:6789/0 conn(0x7f9aa40d37b0 legacy=0x7f9aa40b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:49.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.377+0000 7f9aa1e00640 1 -- 172.21.15.159:0/1218603332 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa40d2cd0 con 0x7f9aa415e4c0 2024-04-04T06:16:49.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.379+0000 7f9aa1e00640 1 --2- 172.21.15.159:0/1218603332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa415e4c0 0x7f9aa40b9bf0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f9a940097a0 tx=0x7f9a9402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8cac4008d1eac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:49.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.379+0000 7f9aa0a00640 1 -- 172.21.15.159:0/1218603332 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a94034070 con 0x7f9aa415e4c0 2024-04-04T06:16:49.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.379+0000 7f9aa0a00640 1 -- 172.21.15.159:0/1218603332 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a94030610 con 0x7f9aa415e4c0 2024-04-04T06:16:49.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.379+0000 7f9aa0a00640 1 -- 172.21.15.159:0/1218603332 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a940309e0 con 0x7f9aa415e4c0 2024-04-04T06:16:49.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.379+0000 7f9aa3200640 1 -- 172.21.15.159:0/1218603332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa415e4c0 msgr2=0x7f9aa40b9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:49.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.379+0000 7f9aa3200640 1 --2- 172.21.15.159:0/1218603332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa415e4c0 0x7f9aa40b9bf0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f9a940097a0 tx=0x7f9a9402b960 comp rx=0 tx=0).stop 2024-04-04T06:16:49.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.380+0000 7f9aa3200640 1 -- 172.21.15.159:0/1218603332 shutdown_connections 2024-04-04T06:16:49.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.380+0000 7f9aa3200640 1 --2- 172.21.15.159:0/1218603332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa415e4c0 0x7f9aa40b9bf0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:49.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.380+0000 7f9aa3200640 1 -- 172.21.15.159:0/1218603332 >> 172.21.15.159:0/1218603332 conn(0x7f9aa40676a0 msgr2=0x7f9aa4067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:49.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.380+0000 7f9aa3200640 1 -- 172.21.15.159:0/1218603332 shutdown_connections 2024-04-04T06:16:49.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.380+0000 7f9aa3200640 1 -- 172.21.15.159:0/1218603332 wait complete. 2024-04-04T06:16:49.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.381+0000 7f9aa3200640 1 Processor -- start 2024-04-04T06:16:49.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.381+0000 7f9aa3200640 1 -- start start 2024-04-04T06:16:49.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.382+0000 7f9aa3200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9aa40b69c0 0x7f9aa4151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:49.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.382+0000 7f9aa3200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aa40d37b0 0x7f9aa40ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:49.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.382+0000 7f9aa3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa415e4c0 0x7f9aa40b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:49.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.382+0000 7f9aa3200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa40d3320 con 0x7f9aa415e4c0 2024-04-04T06:16:49.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.382+0000 7f9aa3200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa40d4040 con 0x7f9aa40b69c0 2024-04-04T06:16:49.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.382+0000 7f9aa3200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa40ba130 con 0x7f9aa40d37b0 2024-04-04T06:16:49.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.383+0000 7f9aa2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa415e4c0 0x7f9aa40b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:49.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.383+0000 7f9aa2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa415e4c0 0x7f9aa40b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52436/0 (socket says 172.21.15.159:52436) 2024-04-04T06:16:49.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.383+0000 7f9aa2800640 1 -- 172.21.15.159:0/3753259874 learned_addr learned my addr 172.21.15.159:0/3753259874 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:49.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.383+0000 7f9aa2800640 1 -- 172.21.15.159:0/3753259874 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aa40d37b0 msgr2=0x7f9aa40ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:49.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.383+0000 7f9aa2800640 1 --2- 172.21.15.159:0/3753259874 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aa40d37b0 0x7f9aa40ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:49.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.383+0000 7f9aa1e00640 1 --2- 172.21.15.159:0/3753259874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9aa40b69c0 0x7f9aa4151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:49.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.383+0000 7f9aa2800640 1 -- 172.21.15.159:0/3753259874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9aa40b69c0 msgr2=0x7f9aa4151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:49.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.383+0000 7f9aa2800640 1 --2- 172.21.15.159:0/3753259874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9aa40b69c0 0x7f9aa4151ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:49.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.383+0000 7f9aa2800640 1 -- 172.21.15.159:0/3753259874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa41610f0 con 0x7f9aa415e4c0 2024-04-04T06:16:49.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.383+0000 7f9aa2800640 1 --2- 172.21.15.159:0/3753259874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa415e4c0 0x7f9aa40b3ff0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c00c680 tx=0x7f9a9c00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:49.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.384+0000 7f9a9a000640 1 -- 172.21.15.159:0/3753259874 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c003b70 con 0x7f9aa415e4c0 2024-04-04T06:16:49.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.384+0000 7f9a9a000640 1 -- 172.21.15.159:0/3753259874 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a9c003d10 con 0x7f9aa415e4c0 2024-04-04T06:16:49.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.384+0000 7f9a9a000640 1 -- 172.21.15.159:0/3753259874 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c005310 con 0x7f9aa415e4c0 2024-04-04T06:16:49.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.384+0000 7f9aa3200640 1 -- 172.21.15.159:0/3753259874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa41613d0 con 0x7f9aa415e4c0 2024-04-04T06:16:49.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.384+0000 7f9aa3200640 1 -- 172.21.15.159:0/3753259874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa4161990 con 0x7f9aa415e4c0 2024-04-04T06:16:49.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.385+0000 7f9aa3200640 1 -- 172.21.15.159:0/3753259874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aa4075940 con 0x7f9aa415e4c0 2024-04-04T06:16:49.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.386+0000 7f9a9a000640 1 -- 172.21.15.159:0/3753259874 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a9c0054b0 con 0x7f9aa415e4c0 2024-04-04T06:16:49.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.386+0000 7f9a9a000640 1 --2- 172.21.15.159:0/3753259874 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9a8406c8f0 0x7f9a8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:49.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.387+0000 7f9a9a000640 1 -- 172.21.15.159:0/3753259874 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5029+0+0 (secure 0 0 0) 0x7f9a9c08b780 con 0x7f9aa415e4c0 2024-04-04T06:16:49.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.387+0000 7f9aa1e00640 1 --2- 172.21.15.159:0/3753259874 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9a8406c8f0 0x7f9a8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:49.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.388+0000 7f9aa1e00640 1 --2- 172.21.15.159:0/3753259874 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9a8406c8f0 0x7f9a8406edb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9a94036000 tx=0x7f9a94008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:49.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.391+0000 7f9a9a000640 1 -- 172.21.15.159:0/3753259874 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9a9c059db0 con 0x7f9aa415e4c0 2024-04-04T06:16:49.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.575+0000 7f9aa3200640 1 -- 172.21.15.159:0/3753259874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9aa415a7f0 con 0x7f9aa415e4c0 2024-04-04T06:16:49.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.576+0000 7f9a9a000640 1 -- 172.21.15.159:0/3753259874 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7f9a9c0596b0 con 0x7f9aa415e4c0 2024-04-04T06:16:49.571 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:16:49.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.578+0000 7f9aa3200640 1 -- 172.21.15.159:0/3753259874 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9a8406c8f0 msgr2=0x7f9a8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:49.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.578+0000 7f9aa3200640 1 --2- 172.21.15.159:0/3753259874 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9a8406c8f0 0x7f9a8406edb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9a94036000 tx=0x7f9a94008030 comp rx=0 tx=0).stop 2024-04-04T06:16:49.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.578+0000 7f9aa3200640 1 -- 172.21.15.159:0/3753259874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa415e4c0 msgr2=0x7f9aa40b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:49.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.578+0000 7f9aa3200640 1 --2- 172.21.15.159:0/3753259874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa415e4c0 0x7f9aa40b3ff0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c00c680 tx=0x7f9a9c00cb40 comp rx=0 tx=0).stop 2024-04-04T06:16:49.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.579+0000 7f9aa3200640 1 -- 172.21.15.159:0/3753259874 shutdown_connections 2024-04-04T06:16:49.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.579+0000 7f9aa3200640 1 --2- 172.21.15.159:0/3753259874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9aa415e4c0 0x7f9aa40b3ff0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:49.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.579+0000 7f9aa3200640 1 --2- 172.21.15.159:0/3753259874 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9aa40d37b0 0x7f9aa40ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:49.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.579+0000 7f9aa3200640 1 --2- 172.21.15.159:0/3753259874 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9a8406c8f0 0x7f9a8406edb0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:49.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.579+0000 7f9aa3200640 1 --2- 172.21.15.159:0/3753259874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9aa40b69c0 0x7f9aa4151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:49.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.579+0000 7f9aa3200640 1 -- 172.21.15.159:0/3753259874 >> 172.21.15.159:0/3753259874 conn(0x7f9aa40676a0 msgr2=0x7f9aa415b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:49.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.579+0000 7f9aa3200640 1 -- 172.21.15.159:0/3753259874 shutdown_connections 2024-04-04T06:16:49.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.579+0000 7f9aa3200640 1 -- 172.21.15.159:0/3753259874 wait complete. 2024-04-04T06:16:49.591 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-04T06:16:49.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.799+0000 7f85d3c00640 1 Processor -- start 2024-04-04T06:16:49.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.799+0000 7f85d3c00640 1 -- start start 2024-04-04T06:16:49.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.800+0000 7f85d3c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f85cc15e860 0x7f85cc06d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:49.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.800+0000 7f85d3c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f85cc06da30 0x7f85cc0d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:49.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.800+0000 7f85d3c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85cc06dfa0 con 0x7f85cc0d4910 2024-04-04T06:16:49.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.800+0000 7f85d3c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f85cc0cf220 con 0x7f85cc06da30 2024-04-04T06:16:49.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.800+0000 7f85d3c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f85cc0d4cf0 con 0x7f85cc15e860 2024-04-04T06:16:49.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.800+0000 7f85cbe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f85cc06da30 0x7f85cc0d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:49.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.801+0000 7f85cbe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f85cc06da30 0x7f85cc0d43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52450/0 (socket says 172.21.15.159:52450) 2024-04-04T06:16:49.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.801+0000 7f85cbe00640 1 -- 172.21.15.159:0/1686879196 learned_addr learned my addr 172.21.15.159:0/1686879196 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:49.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.801+0000 7f85d1200640 1 --2- 172.21.15.159:0/1686879196 >> v2:172.21.15.178:3300/0 conn(0x7f85cc15e860 0x7f85cc06d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:49.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.801+0000 7f85cbe00640 1 -- 172.21.15.159:0/1686879196 >> v2:172.21.15.178:3300/0 conn(0x7f85cc15e860 msgr2=0x7f85cc06d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:49.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.801+0000 7f85cbe00640 1 --2- 172.21.15.159:0/1686879196 >> v2:172.21.15.178:3300/0 conn(0x7f85cc15e860 0x7f85cc06d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:49.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.801+0000 7f85cbe00640 1 -- 172.21.15.159:0/1686879196 >> v1:172.21.15.178:6789/0 conn(0x7f85cc0d4910 legacy=0x7f85cc0d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:49.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.801+0000 7f85cbe00640 1 -- 172.21.15.159:0/1686879196 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85cc06e240 con 0x7f85cc06da30 2024-04-04T06:16:49.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.802+0000 7f85cbe00640 1 --2- 172.21.15.159:0/1686879196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85cc06da30 0x7f85cc0d43d0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f85cc162160 tx=0x7f85b802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd6c9939d4076e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:49.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.802+0000 7f85cb400640 1 -- 172.21.15.159:0/1686879196 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8035020 con 0x7f85cc06da30 2024-04-04T06:16:49.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.802+0000 7f85cb400640 1 -- 172.21.15.159:0/1686879196 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85b802d040 con 0x7f85cc06da30 2024-04-04T06:16:49.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.802+0000 7f85cb400640 1 -- 172.21.15.159:0/1686879196 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8031a80 con 0x7f85cc06da30 2024-04-04T06:16:49.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.803+0000 7f85d3c00640 1 -- 172.21.15.159:0/1686879196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85cc06da30 msgr2=0x7f85cc0d43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:49.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.803+0000 7f85d3c00640 1 --2- 172.21.15.159:0/1686879196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85cc06da30 0x7f85cc0d43d0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f85cc162160 tx=0x7f85b802b900 comp rx=0 tx=0).stop 2024-04-04T06:16:49.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.803+0000 7f85d3c00640 1 -- 172.21.15.159:0/1686879196 shutdown_connections 2024-04-04T06:16:49.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.803+0000 7f85d3c00640 1 --2- 172.21.15.159:0/1686879196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85cc06da30 0x7f85cc0d43d0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:49.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.803+0000 7f85d3c00640 1 --2- 172.21.15.159:0/1686879196 >> v2:172.21.15.178:3300/0 conn(0x7f85cc15e860 0x7f85cc06d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:49.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.803+0000 7f85d3c00640 1 -- 172.21.15.159:0/1686879196 >> 172.21.15.159:0/1686879196 conn(0x7f85cc09d7d0 msgr2=0x7f85cc08bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:49.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.803+0000 7f85d3c00640 1 -- 172.21.15.159:0/1686879196 shutdown_connections 2024-04-04T06:16:49.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.803+0000 7f85d3c00640 1 -- 172.21.15.159:0/1686879196 wait complete. 2024-04-04T06:16:49.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.805+0000 7f85d3c00640 1 Processor -- start 2024-04-04T06:16:49.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.805+0000 7f85d3c00640 1 -- start start 2024-04-04T06:16:49.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.806+0000 7f85d3c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85cc06da30 0x7f85cc0b7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:49.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.806+0000 7f85d3c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85cc0d4910 0x7f85cc1f4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:49.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.806+0000 7f85d3c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85cc15e860 0x7f85cc1f4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:49.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.806+0000 7f85d3c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc0cedb0 con 0x7f85cc15e860 2024-04-04T06:16:49.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.806+0000 7f85d3c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc0d2c50 con 0x7f85cc06da30 2024-04-04T06:16:49.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.806+0000 7f85d3c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f85cc0cef30 con 0x7f85cc0d4910 2024-04-04T06:16:49.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.806+0000 7f85d1c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85cc15e860 0x7f85cc1f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:49.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.806+0000 7f85d1c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85cc15e860 0x7f85cc1f4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52452/0 (socket says 172.21.15.159:52452) 2024-04-04T06:16:49.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.806+0000 7f85d1c00640 1 -- 172.21.15.159:0/4051159109 learned_addr learned my addr 172.21.15.159:0/4051159109 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:49.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.806+0000 7f85d1200640 1 --2- 172.21.15.159:0/4051159109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85cc06da30 0x7f85cc0b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:49.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.807+0000 7f85cbe00640 1 --2- 172.21.15.159:0/4051159109 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85cc0d4910 0x7f85cc1f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:49.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.807+0000 7f85d1c00640 1 -- 172.21.15.159:0/4051159109 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85cc0d4910 msgr2=0x7f85cc1f4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:49.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.807+0000 7f85d1c00640 1 --2- 172.21.15.159:0/4051159109 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85cc0d4910 0x7f85cc1f4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:49.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.807+0000 7f85d1c00640 1 -- 172.21.15.159:0/4051159109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85cc06da30 msgr2=0x7f85cc0b7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:49.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.807+0000 7f85d1c00640 1 --2- 172.21.15.159:0/4051159109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85cc06da30 0x7f85cc0b7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:49.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.807+0000 7f85d1c00640 1 -- 172.21.15.159:0/4051159109 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85cc0b7940 con 0x7f85cc15e860 2024-04-04T06:16:49.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.807+0000 7f85d1200640 1 --2- 172.21.15.159:0/4051159109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85cc06da30 0x7f85cc0b7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:49.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.807+0000 7f85d1c00640 1 --2- 172.21.15.159:0/4051159109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85cc15e860 0x7f85cc1f4780 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f85c400c7b0 tx=0x7f85c400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:49.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.807+0000 7f85c8c00640 1 -- 172.21.15.159:0/4051159109 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c4017070 con 0x7f85cc15e860 2024-04-04T06:16:49.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.807+0000 7f85c8c00640 1 -- 172.21.15.159:0/4051159109 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c4003da0 con 0x7f85cc15e860 2024-04-04T06:16:49.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.807+0000 7f85c8c00640 1 -- 172.21.15.159:0/4051159109 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c40052b0 con 0x7f85cc15e860 2024-04-04T06:16:49.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.807+0000 7f85d3c00640 1 -- 172.21.15.159:0/4051159109 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85cc0b7c20 con 0x7f85cc15e860 2024-04-04T06:16:49.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.807+0000 7f85d3c00640 1 -- 172.21.15.159:0/4051159109 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85cc0b81e0 con 0x7f85cc15e860 2024-04-04T06:16:49.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.809+0000 7f85d3c00640 1 -- 172.21.15.159:0/4051159109 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85cc0b8420 con 0x7f85cc15e860 2024-04-04T06:16:49.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.809+0000 7f85c8c00640 1 -- 172.21.15.159:0/4051159109 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f85c4005450 con 0x7f85cc15e860 2024-04-04T06:16:49.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.810+0000 7f85c8c00640 1 --2- 172.21.15.159:0/4051159109 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f85ac06c8f0 0x7f85ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:49.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.811+0000 7f85c8c00640 1 -- 172.21.15.159:0/4051159109 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5029+0+0 (secure 0 0 0) 0x7f85c408af40 con 0x7f85cc15e860 2024-04-04T06:16:49.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.811+0000 7f85d1200640 1 --2- 172.21.15.159:0/4051159109 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f85ac06c8f0 0x7f85ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:49.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.812+0000 7f85d1200640 1 --2- 172.21.15.159:0/4051159109 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f85ac06c8f0 0x7f85ac06edb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00b820 tx=0x7f85bc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:49.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:49.815+0000 7f85c8c00640 1 -- 172.21.15.159:0/4051159109 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f85cc0b8420 con 0x7f85cc15e860 2024-04-04T06:16:50.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.069+0000 7f85d3c00640 1 -- 172.21.15.159:0/4051159109 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f85cc0d5590 con 0x7f85cc15e860 2024-04-04T06:16:50.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.728+0000 7f85c8c00640 1 -- 172.21.15.159:0/4051159109 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v521) ==== 147+0+0 (secure 0 0 0) 0x7f85cc0d5590 con 0x7f85cc15e860 2024-04-04T06:16:50.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.732+0000 7f85d3c00640 1 -- 172.21.15.159:0/4051159109 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f85ac06c8f0 msgr2=0x7f85ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:50.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.732+0000 7f85d3c00640 1 --2- 172.21.15.159:0/4051159109 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f85ac06c8f0 0x7f85ac06edb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00b820 tx=0x7f85bc008040 comp rx=0 tx=0).stop 2024-04-04T06:16:50.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.732+0000 7f85d3c00640 1 -- 172.21.15.159:0/4051159109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85cc15e860 msgr2=0x7f85cc1f4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:50.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.732+0000 7f85d3c00640 1 --2- 172.21.15.159:0/4051159109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85cc15e860 0x7f85cc1f4780 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f85c400c7b0 tx=0x7f85c400cc70 comp rx=0 tx=0).stop 2024-04-04T06:16:50.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.732+0000 7f85d3c00640 1 -- 172.21.15.159:0/4051159109 shutdown_connections 2024-04-04T06:16:50.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.732+0000 7f85d3c00640 1 --2- 172.21.15.159:0/4051159109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85cc15e860 0x7f85cc1f4780 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:50.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.732+0000 7f85d3c00640 1 --2- 172.21.15.159:0/4051159109 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85cc0d4910 0x7f85cc1f4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:50.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.732+0000 7f85d3c00640 1 --2- 172.21.15.159:0/4051159109 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f85ac06c8f0 0x7f85ac06edb0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:50.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.732+0000 7f85d3c00640 1 --2- 172.21.15.159:0/4051159109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85cc06da30 0x7f85cc0b7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:50.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.732+0000 7f85d3c00640 1 -- 172.21.15.159:0/4051159109 >> 172.21.15.159:0/4051159109 conn(0x7f85cc09d7d0 msgr2=0x7f85cc0cffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:50.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.732+0000 7f85d3c00640 1 -- 172.21.15.159:0/4051159109 shutdown_connections 2024-04-04T06:16:50.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.732+0000 7f85d3c00640 1 -- 172.21.15.159:0/4051159109 wait complete. 2024-04-04T06:16:50.750 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-04T06:16:50.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.968+0000 7f81c9400640 1 Processor -- start 2024-04-04T06:16:50.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.968+0000 7f81c9400640 1 -- start start 2024-04-04T06:16:50.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.969+0000 7f81c9400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f81c415e860 0x7f81c406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:50.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.969+0000 7f81c9400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f81c406da30 0x7f81c40d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:50.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.969+0000 7f81c9400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81c406dfa0 con 0x7f81c40d4910 2024-04-04T06:16:50.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.969+0000 7f81c9400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f81c40cf220 con 0x7f81c406da30 2024-04-04T06:16:50.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.969+0000 7f81c9400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f81c40d4cf0 con 0x7f81c415e860 2024-04-04T06:16:50.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.970+0000 7f81c1e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f81c406da30 0x7f81c40d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:50.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.970+0000 7f81c1e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f81c406da30 0x7f81c40d43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52466/0 (socket says 172.21.15.159:52466) 2024-04-04T06:16:50.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.970+0000 7f81c1e00640 1 -- 172.21.15.159:0/1643027798 learned_addr learned my addr 172.21.15.159:0/1643027798 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:50.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.970+0000 7f81c1e00640 1 -- 172.21.15.159:0/1643027798 >> v2:172.21.15.178:3300/0 conn(0x7f81c415e860 msgr2=0x7f81c406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:50.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.970+0000 7f81c1e00640 1 --2- 172.21.15.159:0/1643027798 >> v2:172.21.15.178:3300/0 conn(0x7f81c415e860 0x7f81c406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:50.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.970+0000 7f81c1e00640 1 -- 172.21.15.159:0/1643027798 >> v1:172.21.15.178:6789/0 conn(0x7f81c40d4910 legacy=0x7f81c40d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:50.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.970+0000 7f81c1e00640 1 -- 172.21.15.159:0/1643027798 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c406e240 con 0x7f81c406da30 2024-04-04T06:16:50.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.971+0000 7f81c1e00640 1 --2- 172.21.15.159:0/1643027798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81c406da30 0x7f81c40d43d0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f81c4162160 tx=0x7f81ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3e85367e402219 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:50.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.971+0000 7f81c1400640 1 -- 172.21.15.159:0/1643027798 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac035020 con 0x7f81c406da30 2024-04-04T06:16:50.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.972+0000 7f81c1400640 1 -- 172.21.15.159:0/1643027798 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81ac031870 con 0x7f81c406da30 2024-04-04T06:16:50.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.972+0000 7f81c1400640 1 -- 172.21.15.159:0/1643027798 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac031c40 con 0x7f81c406da30 2024-04-04T06:16:50.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.972+0000 7f81c9400640 1 -- 172.21.15.159:0/1643027798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81c406da30 msgr2=0x7f81c40d43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:50.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.972+0000 7f81c9400640 1 --2- 172.21.15.159:0/1643027798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81c406da30 0x7f81c40d43d0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f81c4162160 tx=0x7f81ac02b910 comp rx=0 tx=0).stop 2024-04-04T06:16:50.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.972+0000 7f81c9400640 1 -- 172.21.15.159:0/1643027798 shutdown_connections 2024-04-04T06:16:50.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.972+0000 7f81c9400640 1 --2- 172.21.15.159:0/1643027798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81c406da30 0x7f81c40d43d0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:50.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.972+0000 7f81c9400640 1 --2- 172.21.15.159:0/1643027798 >> v2:172.21.15.178:3300/0 conn(0x7f81c415e860 0x7f81c406d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:50.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.972+0000 7f81c9400640 1 -- 172.21.15.159:0/1643027798 >> 172.21.15.159:0/1643027798 conn(0x7f81c409d7d0 msgr2=0x7f81c408bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:50.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.972+0000 7f81c9400640 1 -- 172.21.15.159:0/1643027798 shutdown_connections 2024-04-04T06:16:50.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.972+0000 7f81c9400640 1 -- 172.21.15.159:0/1643027798 wait complete. 2024-04-04T06:16:50.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.974+0000 7f81c9400640 1 Processor -- start 2024-04-04T06:16:50.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.975+0000 7f81c9400640 1 -- start start 2024-04-04T06:16:50.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.976+0000 7f81c9400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81c406da30 0x7f81c415d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:50.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.976+0000 7f81c9400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81c40d4910 0x7f81c415fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:50.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.976+0000 7f81c9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81c415e860 0x7f81c40cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:50.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.976+0000 7f81c9400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f81c40cedb0 con 0x7f81c415e860 2024-04-04T06:16:50.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.976+0000 7f81c9400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f81c40d2c50 con 0x7f81c406da30 2024-04-04T06:16:50.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.976+0000 7f81c9400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f81c40cef30 con 0x7f81c40d4910 2024-04-04T06:16:50.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.976+0000 7f81c3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81c415e860 0x7f81c40cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:50.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.976+0000 7f81c3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81c415e860 0x7f81c40cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52482/0 (socket says 172.21.15.159:52482) 2024-04-04T06:16:50.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.976+0000 7f81c3200640 1 -- 172.21.15.159:0/2993482084 learned_addr learned my addr 172.21.15.159:0/2993482084 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:50.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.976+0000 7f81c2800640 1 --2- 172.21.15.159:0/2993482084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81c406da30 0x7f81c415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:50.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.976+0000 7f81c1e00640 1 --2- 172.21.15.159:0/2993482084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81c40d4910 0x7f81c415fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:50.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.977+0000 7f81c3200640 1 -- 172.21.15.159:0/2993482084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81c40d4910 msgr2=0x7f81c415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:50.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.977+0000 7f81c3200640 1 --2- 172.21.15.159:0/2993482084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81c40d4910 0x7f81c415fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:50.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.977+0000 7f81c3200640 1 -- 172.21.15.159:0/2993482084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81c406da30 msgr2=0x7f81c415d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:50.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.977+0000 7f81c3200640 1 --2- 172.21.15.159:0/2993482084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81c406da30 0x7f81c415d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:50.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.977+0000 7f81c3200640 1 -- 172.21.15.159:0/2993482084 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c4160f50 con 0x7f81c415e860 2024-04-04T06:16:50.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.977+0000 7f81c1e00640 1 --2- 172.21.15.159:0/2993482084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81c40d4910 0x7f81c415fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:50.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.977+0000 7f81c3200640 1 --2- 172.21.15.159:0/2993482084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81c415e860 0x7f81c40cd520 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00c780 tx=0x7f81bc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:50.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.977+0000 7f81baa00640 1 -- 172.21.15.159:0/2993482084 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc017070 con 0x7f81c415e860 2024-04-04T06:16:50.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.977+0000 7f81baa00640 1 -- 172.21.15.159:0/2993482084 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81bc003da0 con 0x7f81c415e860 2024-04-04T06:16:50.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.977+0000 7f81baa00640 1 -- 172.21.15.159:0/2993482084 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc0052b0 con 0x7f81c415e860 2024-04-04T06:16:50.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.977+0000 7f81c9400640 1 -- 172.21.15.159:0/2993482084 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c40c80a0 con 0x7f81c415e860 2024-04-04T06:16:50.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.977+0000 7f81c9400640 1 -- 172.21.15.159:0/2993482084 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c4160990 con 0x7f81c415e860 2024-04-04T06:16:50.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.978+0000 7f81c9400640 1 -- 172.21.15.159:0/2993482084 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c40c8270 con 0x7f81c415e860 2024-04-04T06:16:50.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.979+0000 7f81baa00640 1 -- 172.21.15.159:0/2993482084 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f81bc01a070 con 0x7f81c415e860 2024-04-04T06:16:50.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.980+0000 7f81baa00640 1 --2- 172.21.15.159:0/2993482084 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f81a006c920 0x7f81a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:50.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.980+0000 7f81baa00640 1 -- 172.21.15.159:0/2993482084 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5029+0+0 (secure 0 0 0) 0x7f81bc08aef0 con 0x7f81c415e860 2024-04-04T06:16:50.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.981+0000 7f81c2800640 1 --2- 172.21.15.159:0/2993482084 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f81a006c920 0x7f81a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:50.976 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.982+0000 7f81c2800640 1 --2- 172.21.15.159:0/2993482084 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f81a006c920 0x7f81a006ede0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f81c40b6bd0 tx=0x7f81b4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:50.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:50.986+0000 7f81baa00640 1 -- 172.21.15.159:0/2993482084 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f81bc0595b0 con 0x7f81c415e860 2024-04-04T06:16:51.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.187+0000 7f81c9400640 1 -- 172.21.15.159:0/2993482084 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f81c40ce750 con 0x7f81c415e860 2024-04-04T06:16:51.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.194+0000 7f81baa00640 1 -- 172.21.15.159:0/2993482084 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f81bc058eb0 con 0x7f81c415e860 2024-04-04T06:16:51.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.197+0000 7f81c9400640 1 -- 172.21.15.159:0/2993482084 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f81a006c920 msgr2=0x7f81a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:51.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.197+0000 7f81c9400640 1 --2- 172.21.15.159:0/2993482084 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f81a006c920 0x7f81a006ede0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f81c40b6bd0 tx=0x7f81b4008040 comp rx=0 tx=0).stop 2024-04-04T06:16:51.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.197+0000 7f81c9400640 1 -- 172.21.15.159:0/2993482084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81c415e860 msgr2=0x7f81c40cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:51.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.197+0000 7f81c9400640 1 --2- 172.21.15.159:0/2993482084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81c415e860 0x7f81c40cd520 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00c780 tx=0x7f81bc00cc40 comp rx=0 tx=0).stop 2024-04-04T06:16:51.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.197+0000 7f81c9400640 1 -- 172.21.15.159:0/2993482084 shutdown_connections 2024-04-04T06:16:51.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.197+0000 7f81c9400640 1 --2- 172.21.15.159:0/2993482084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81c415e860 0x7f81c40cd520 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:51.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.197+0000 7f81c9400640 1 --2- 172.21.15.159:0/2993482084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81c40d4910 0x7f81c415fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:51.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.197+0000 7f81c9400640 1 --2- 172.21.15.159:0/2993482084 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f81a006c920 0x7f81a006ede0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:51.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.198+0000 7f81c9400640 1 --2- 172.21.15.159:0/2993482084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81c406da30 0x7f81c415d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:51.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.198+0000 7f81c9400640 1 -- 172.21.15.159:0/2993482084 >> 172.21.15.159:0/2993482084 conn(0x7f81c409d7d0 msgr2=0x7f81c40b4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:51.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.198+0000 7f81c9400640 1 -- 172.21.15.159:0/2993482084 shutdown_connections 2024-04-04T06:16:51.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.198+0000 7f81c9400640 1 -- 172.21.15.159:0/2993482084 wait complete. 2024-04-04T06:16:51.209 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-04T06:16:51.210 DEBUG:teuthology.orchestra.run.smithi159:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:16:51.315 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:16:51.447 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-04T06:16:51.447 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:16:51.447 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:16:51.503 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:16:51.503 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:16:51.503 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:16:51.561 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:16:51.561 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:16:51.561 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:16:51.618 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:16:51.618 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:16:51.618 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:16:51.674 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:16:51.675 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-04T06:16:51.675 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:16:51.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.900+0000 7fe965600640 1 Processor -- start 2024-04-04T06:16:51.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.900+0000 7fe965600640 1 -- start start 2024-04-04T06:16:51.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.901+0000 7fe965600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe96015c390 0x7fe960073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:51.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.901+0000 7fe965600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe9600d0650 0x7fe9600d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:51.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.901+0000 7fe965600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe96015fe80 con 0x7fe9600af830 2024-04-04T06:16:51.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.901+0000 7fe965600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe96015c770 con 0x7fe96015c390 2024-04-04T06:16:51.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.901+0000 7fe965600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe9600d0fe0 con 0x7fe9600d0650 2024-04-04T06:16:51.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.902+0000 7fe95ea00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe96015c390 0x7fe960073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:51.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.902+0000 7fe95ea00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe96015c390 0x7fe960073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52500/0 (socket says 172.21.15.159:52500) 2024-04-04T06:16:51.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.902+0000 7fe95ea00640 1 -- 172.21.15.159:0/715002811 learned_addr learned my addr 172.21.15.159:0/715002811 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:51.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.902+0000 7fe95f400640 1 --2- 172.21.15.159:0/715002811 >> v2:172.21.15.178:3300/0 conn(0x7fe9600d0650 0x7fe9600d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:51.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.902+0000 7fe95ea00640 1 -- 172.21.15.159:0/715002811 >> v2:172.21.15.178:3300/0 conn(0x7fe9600d0650 msgr2=0x7fe9600d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:51.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.902+0000 7fe95ea00640 1 --2- 172.21.15.159:0/715002811 >> v2:172.21.15.178:3300/0 conn(0x7fe9600d0650 0x7fe9600d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:51.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.902+0000 7fe95ea00640 1 -- 172.21.15.159:0/715002811 >> v1:172.21.15.178:6789/0 conn(0x7fe9600af830 legacy=0x7fe960160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:51.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.902+0000 7fe95ea00640 1 -- 172.21.15.159:0/715002811 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9600ad120 con 0x7fe96015c390 2024-04-04T06:16:51.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.904+0000 7fe95ea00640 1 --2- 172.21.15.159:0/715002811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe96015c390 0x7fe960073790 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fe96009e420 tx=0x7fe95002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d2f2c03f341f44 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:51.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.904+0000 7fe95d600640 1 -- 172.21.15.159:0/715002811 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950035020 con 0x7fe96015c390 2024-04-04T06:16:51.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.904+0000 7fe95d600640 1 -- 172.21.15.159:0/715002811 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe950031870 con 0x7fe96015c390 2024-04-04T06:16:51.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.904+0000 7fe95d600640 1 -- 172.21.15.159:0/715002811 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950031c40 con 0x7fe96015c390 2024-04-04T06:16:51.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.905+0000 7fe965600640 1 -- 172.21.15.159:0/715002811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe96015c390 msgr2=0x7fe960073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:51.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.905+0000 7fe965600640 1 --2- 172.21.15.159:0/715002811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe96015c390 0x7fe960073790 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fe96009e420 tx=0x7fe95002b910 comp rx=0 tx=0).stop 2024-04-04T06:16:51.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.905+0000 7fe965600640 1 -- 172.21.15.159:0/715002811 shutdown_connections 2024-04-04T06:16:51.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.905+0000 7fe965600640 1 --2- 172.21.15.159:0/715002811 >> v2:172.21.15.178:3300/0 conn(0x7fe9600d0650 0x7fe9600d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:51.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.905+0000 7fe965600640 1 --2- 172.21.15.159:0/715002811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe96015c390 0x7fe960073790 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:51.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.905+0000 7fe965600640 1 -- 172.21.15.159:0/715002811 >> 172.21.15.159:0/715002811 conn(0x7fe96006b990 msgr2=0x7fe96006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:51.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.905+0000 7fe965600640 1 -- 172.21.15.159:0/715002811 shutdown_connections 2024-04-04T06:16:51.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.905+0000 7fe965600640 1 -- 172.21.15.159:0/715002811 wait complete. 2024-04-04T06:16:51.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.907+0000 7fe965600640 1 Processor -- start 2024-04-04T06:16:51.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.907+0000 7fe965600640 1 -- start start 2024-04-04T06:16:51.908 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.908+0000 7fe965600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9600af830 0x7fe96015f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:51.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.908+0000 7fe965600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9600d0650 0x7fe960151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:51.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.908+0000 7fe965600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe96015c390 0x7fe960150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:51.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.908+0000 7fe965600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe960160120 con 0x7fe9600d0650 2024-04-04T06:16:51.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.908+0000 7fe965600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe9600ad6e0 con 0x7fe96015c390 2024-04-04T06:16:51.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.908+0000 7fe965600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe960160d50 con 0x7fe9600af830 2024-04-04T06:16:51.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.908+0000 7fe95e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9600d0650 0x7fe960151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:51.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.908+0000 7fe95e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9600d0650 0x7fe960151af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52508/0 (socket says 172.21.15.159:52508) 2024-04-04T06:16:51.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.908+0000 7fe95e000640 1 -- 172.21.15.159:0/1760842588 learned_addr learned my addr 172.21.15.159:0/1760842588 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:51.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.909+0000 7fe95f400640 1 --2- 172.21.15.159:0/1760842588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe96015c390 0x7fe960150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:51.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.909+0000 7fe95e000640 1 -- 172.21.15.159:0/1760842588 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9600af830 msgr2=0x7fe96015f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:51.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.909+0000 7fe95ea00640 1 --2- 172.21.15.159:0/1760842588 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9600af830 0x7fe96015f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:51.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.909+0000 7fe95e000640 1 --2- 172.21.15.159:0/1760842588 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9600af830 0x7fe96015f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:51.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.909+0000 7fe95e000640 1 -- 172.21.15.159:0/1760842588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe96015c390 msgr2=0x7fe960150720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:51.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.909+0000 7fe95e000640 1 --2- 172.21.15.159:0/1760842588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe96015c390 0x7fe960150720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:51.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.909+0000 7fe95e000640 1 -- 172.21.15.159:0/1760842588 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96015cb30 con 0x7fe9600d0650 2024-04-04T06:16:51.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.909+0000 7fe95f400640 1 --2- 172.21.15.159:0/1760842588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe96015c390 0x7fe960150720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:51.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.909+0000 7fe95e000640 1 --2- 172.21.15.159:0/1760842588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9600d0650 0x7fe960151af0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00b420 tx=0x7fe94c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:51.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.909+0000 7fe956a00640 1 -- 172.21.15.159:0/1760842588 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c011830 con 0x7fe9600d0650 2024-04-04T06:16:51.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.909+0000 7fe956a00640 1 -- 172.21.15.159:0/1760842588 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe94c016070 con 0x7fe9600d0650 2024-04-04T06:16:51.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.909+0000 7fe956a00640 1 -- 172.21.15.159:0/1760842588 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c011c10 con 0x7fe9600d0650 2024-04-04T06:16:51.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.909+0000 7fe965600640 1 -- 172.21.15.159:0/1760842588 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe96015ce10 con 0x7fe9600d0650 2024-04-04T06:16:51.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.910+0000 7fe965600640 1 -- 172.21.15.159:0/1760842588 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe960152030 con 0x7fe9600d0650 2024-04-04T06:16:51.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.911+0000 7fe965600640 1 -- 172.21.15.159:0/1760842588 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9600685c0 con 0x7fe9600d0650 2024-04-04T06:16:51.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.912+0000 7fe956a00640 1 -- 172.21.15.159:0/1760842588 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe94c01b460 con 0x7fe9600d0650 2024-04-04T06:16:51.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.912+0000 7fe956a00640 1 --2- 172.21.15.159:0/1760842588 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe94006c920 0x7fe94006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:51.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.913+0000 7fe956a00640 1 -- 172.21.15.159:0/1760842588 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5029+0+0 (secure 0 0 0) 0x7fe94c08b450 con 0x7fe9600d0650 2024-04-04T06:16:51.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.913+0000 7fe95ea00640 1 --2- 172.21.15.159:0/1760842588 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe94006c920 0x7fe94006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:51.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.914+0000 7fe95ea00640 1 --2- 172.21.15.159:0/1760842588 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe94006c920 0x7fe94006ede0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe950036000 tx=0x7fe950008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:51.912 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:51.918+0000 7fe956a00640 1 -- 172.21.15.159:0/1760842588 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe94c020020 con 0x7fe9600d0650 2024-04-04T06:16:52.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.158+0000 7fe965600640 1 -- 172.21.15.159:0/1760842588 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe96006f1b0 con 0x7fe9600d0650 2024-04-04T06:16:52.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.159+0000 7fe956a00640 1 -- 172.21.15.159:0/1760842588 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+3875 (secure 0 0 0) 0x7fe94c059a80 con 0x7fe9600d0650 2024-04-04T06:16:52.154 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:52.154 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":94,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5546,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/40219476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":40219476},{"type":"v1","addr":"172.21.15.178:6837","nonce":40219476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/4060483382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4060483382},{"type":"v1","addr":"172.21.15.178:6836","nonce":4060483382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8187,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1945665739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1945665739},{"type":"v1","addr":"172.21.15.159:6833","nonce":1945665739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8688,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3776862819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3776862819},{"type":"v1","addr":"172.21.15.159:6835","nonce":3776862819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[]} 2024-04-04T06:16:52.154 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 94 2024-04-04T06:16:52.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.162+0000 7fe965600640 1 -- 172.21.15.159:0/1760842588 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe94006c920 msgr2=0x7fe94006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:52.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.162+0000 7fe965600640 1 --2- 172.21.15.159:0/1760842588 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe94006c920 0x7fe94006ede0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe950036000 tx=0x7fe950008030 comp rx=0 tx=0).stop 2024-04-04T06:16:52.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.162+0000 7fe965600640 1 -- 172.21.15.159:0/1760842588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9600d0650 msgr2=0x7fe960151af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:52.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.162+0000 7fe965600640 1 --2- 172.21.15.159:0/1760842588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9600d0650 0x7fe960151af0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00b420 tx=0x7fe94c00b8e0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.162+0000 7fe965600640 1 -- 172.21.15.159:0/1760842588 shutdown_connections 2024-04-04T06:16:52.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.162+0000 7fe965600640 1 --2- 172.21.15.159:0/1760842588 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe94006c920 0x7fe94006ede0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.162+0000 7fe965600640 1 --2- 172.21.15.159:0/1760842588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe96015c390 0x7fe960150720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.162+0000 7fe965600640 1 --2- 172.21.15.159:0/1760842588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9600d0650 0x7fe960151af0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.162+0000 7fe965600640 1 --2- 172.21.15.159:0/1760842588 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9600af830 0x7fe96015f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.162+0000 7fe965600640 1 -- 172.21.15.159:0/1760842588 >> 172.21.15.159:0/1760842588 conn(0x7fe96006b990 msgr2=0x7fe9600b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:52.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.162+0000 7fe965600640 1 -- 172.21.15.159:0/1760842588 shutdown_connections 2024-04-04T06:16:52.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.162+0000 7fe965600640 1 -- 172.21.15.159:0/1760842588 wait complete. 2024-04-04T06:16:52.177 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-04T06:16:52.177 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-04T06:16:52.177 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-04T06:16:52.177 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:16:52.178 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-04T06:16:52.178 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-04T06:16:52.178 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-04T06:16:52.178 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:16:52.178 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-04T06:16:52.178 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-04T06:16:52.178 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-04T06:16:52.179 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:16:52.179 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-04T06:16:52.179 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-04T06:16:52.179 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-04T06:16:52.179 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:16:52.200 INFO:tasks.ceph.mds.a:Stopped 2024-04-04T06:16:52.200 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-04T06:16:52.201 INFO:tasks.ceph.mds.c:Stopped 2024-04-04T06:16:52.201 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-04T06:16:52.203 INFO:tasks.ceph.mds.d:Stopped 2024-04-04T06:16:52.203 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-04T06:16:52.204 INFO:tasks.ceph.mds.b:Stopped 2024-04-04T06:16:52.204 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-04T06:16:52.248 INFO:tasks.ceph.mds.a:Started 2024-04-04T06:16:52.248 DEBUG:teuthology.parallel:result is None 2024-04-04T06:16:52.250 INFO:tasks.ceph.mds.d:Started 2024-04-04T06:16:52.250 DEBUG:teuthology.parallel:result is None 2024-04-04T06:16:52.254 INFO:tasks.ceph.mds.c:Started 2024-04-04T06:16:52.254 DEBUG:teuthology.parallel:result is None 2024-04-04T06:16:52.256 INFO:tasks.ceph.mds.b:Started 2024-04-04T06:16:52.256 DEBUG:teuthology.parallel:result is None 2024-04-04T06:16:52.256 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-04T06:16:52.403 INFO:tasks.ceph.mds.a.smithi159.stdout:starting mds.a at 2024-04-04T06:16:52.403 INFO:tasks.ceph.mds.c.smithi159.stdout:starting mds.c at 2024-04-04T06:16:52.412 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2024-04-04T06:16:52.412 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2024-04-04T06:16:52.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.446+0000 7f3ffe400640 1 Processor -- start 2024-04-04T06:16:52.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.446+0000 7f3ffe400640 1 -- start start 2024-04-04T06:16:52.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.447+0000 7f3ffe400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3ff815e530 0x7f3ff80cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:52.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.447+0000 7f3ffe400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3ff80cf330 0x7f3ff80d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:52.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.447+0000 7f3ffe400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ff8162350 con 0x7f3ff806e1b0 2024-04-04T06:16:52.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.447+0000 7f3ffe400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f3ff8162ed0 con 0x7f3ff80cf330 2024-04-04T06:16:52.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.447+0000 7f3ffe400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3ff8162630 con 0x7f3ff815e530 2024-04-04T06:16:52.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.447+0000 7f3ffda00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3ff80cf330 0x7f3ff80d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:52.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.448+0000 7f3ffda00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3ff80cf330 0x7f3ff80d1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52532/0 (socket says 172.21.15.159:52532) 2024-04-04T06:16:52.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.448+0000 7f3ffda00640 1 -- 172.21.15.159:0/3807092331 learned_addr learned my addr 172.21.15.159:0/3807092331 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:52.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.448+0000 7f3ff7e00640 1 --1- 172.21.15.159:0/3807092331 >> v1:172.21.15.159:6789/0 conn(0x7f3ff806e1b0 0x7f3ff80cec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40358/0 (socket says 172.21.15.159:40358) 2024-04-04T06:16:52.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.448+0000 7f3ffd000640 1 --2- 172.21.15.159:0/3807092331 >> v2:172.21.15.178:3300/0 conn(0x7f3ff815e530 0x7f3ff80cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:52.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.448+0000 7f3ffda00640 1 -- 172.21.15.159:0/3807092331 >> v2:172.21.15.178:3300/0 conn(0x7f3ff815e530 msgr2=0x7f3ff80cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:52.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.448+0000 7f3ffda00640 1 --2- 172.21.15.159:0/3807092331 >> v2:172.21.15.178:3300/0 conn(0x7f3ff815e530 0x7f3ff80cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.448+0000 7f3ffda00640 1 -- 172.21.15.159:0/3807092331 >> v1:172.21.15.159:6789/0 conn(0x7f3ff806e1b0 legacy=0x7f3ff80cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:52.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.448+0000 7f3ffda00640 1 -- 172.21.15.159:0/3807092331 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff8162910 con 0x7f3ff80cf330 2024-04-04T06:16:52.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.448+0000 7f3ffd000640 1 --2- 172.21.15.159:0/3807092331 >> v2:172.21.15.178:3300/0 conn(0x7f3ff815e530 0x7f3ff80cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:16:52.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.449+0000 7f3ffda00640 1 --2- 172.21.15.159:0/3807092331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff80cf330 0x7f3ff80d1a30 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80680e0 tx=0x7f3ff002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae7a933b28e9d69 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:52.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.449+0000 7f3ff7400640 1 -- 172.21.15.159:0/3807092331 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff002ca80 con 0x7f3ff80cf330 2024-04-04T06:16:52.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.449+0000 7f3ff7400640 1 -- 172.21.15.159:0/3807092331 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ff002cc20 con 0x7f3ff80cf330 2024-04-04T06:16:52.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.449+0000 7f3ff7400640 1 -- 172.21.15.159:0/3807092331 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff00343f0 con 0x7f3ff80cf330 2024-04-04T06:16:52.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.450+0000 7f3ffe400640 1 -- 172.21.15.159:0/3807092331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff80cf330 msgr2=0x7f3ff80d1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:52.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.450+0000 7f3ffe400640 1 --2- 172.21.15.159:0/3807092331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff80cf330 0x7f3ff80d1a30 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80680e0 tx=0x7f3ff002bee0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.450+0000 7f3ffe400640 1 -- 172.21.15.159:0/3807092331 shutdown_connections 2024-04-04T06:16:52.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.450+0000 7f3ffe400640 1 --2- 172.21.15.159:0/3807092331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff80cf330 0x7f3ff80d1a30 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.450+0000 7f3ffe400640 1 --2- 172.21.15.159:0/3807092331 >> v2:172.21.15.178:3300/0 conn(0x7f3ff815e530 0x7f3ff80cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.450+0000 7f3ffe400640 1 -- 172.21.15.159:0/3807092331 >> 172.21.15.159:0/3807092331 conn(0x7f3ff8067280 msgr2=0x7f3ff8067660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:52.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.450+0000 7f3ffe400640 1 -- 172.21.15.159:0/3807092331 shutdown_connections 2024-04-04T06:16:52.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.450+0000 7f3ffe400640 1 -- 172.21.15.159:0/3807092331 wait complete. 2024-04-04T06:16:52.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.452+0000 7f3ffe400640 1 Processor -- start 2024-04-04T06:16:52.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.452+0000 7f3ffe400640 1 -- start start 2024-04-04T06:16:52.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.453+0000 7f3ffe400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff806e1b0 0x7f3ff8159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:52.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.453+0000 7f3ffe400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ff80cf330 0x7f3ff81f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:52.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.453+0000 7f3ffe400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ff815e530 0x7f3ff815f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:52.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.453+0000 7f3ffe400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff80cf780 con 0x7f3ff806e1b0 2024-04-04T06:16:52.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.453+0000 7f3ffe400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff8161fe0 con 0x7f3ff80cf330 2024-04-04T06:16:52.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.453+0000 7f3ffe400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff80cf900 con 0x7f3ff815e530 2024-04-04T06:16:52.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.453+0000 7f3ffd000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff806e1b0 0x7f3ff8159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:52.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.453+0000 7f3ffd000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff806e1b0 0x7f3ff8159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52546/0 (socket says 172.21.15.159:52546) 2024-04-04T06:16:52.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.453+0000 7f3ffd000640 1 -- 172.21.15.159:0/1931316095 learned_addr learned my addr 172.21.15.159:0/1931316095 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:52.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.454+0000 7f3ffda00640 1 --2- 172.21.15.159:0/1931316095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ff815e530 0x7f3ff815f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:52.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.454+0000 7f3ff7e00640 1 --2- 172.21.15.159:0/1931316095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ff80cf330 0x7f3ff81f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:52.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.454+0000 7f3ffd000640 1 -- 172.21.15.159:0/1931316095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ff815e530 msgr2=0x7f3ff815f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:52.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.454+0000 7f3ffd000640 1 --2- 172.21.15.159:0/1931316095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ff815e530 0x7f3ff815f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.454+0000 7f3ffd000640 1 -- 172.21.15.159:0/1931316095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ff80cf330 msgr2=0x7f3ff81f3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:52.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.454+0000 7f3ffd000640 1 --2- 172.21.15.159:0/1931316095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ff80cf330 0x7f3ff81f3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.454+0000 7f3ffd000640 1 -- 172.21.15.159:0/1931316095 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff8159b70 con 0x7f3ff806e1b0 2024-04-04T06:16:52.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.454+0000 7f3ffd000640 1 --2- 172.21.15.159:0/1931316095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff806e1b0 0x7f3ff8159630 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00ebe0 tx=0x7f3fec00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:52.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.454+0000 7f3ff4c00640 1 -- 172.21.15.159:0/1931316095 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec017070 con 0x7f3ff806e1b0 2024-04-04T06:16:52.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.454+0000 7f3ff4c00640 1 -- 172.21.15.159:0/1931316095 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fec00cc30 con 0x7f3ff806e1b0 2024-04-04T06:16:52.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.454+0000 7f3ff4c00640 1 -- 172.21.15.159:0/1931316095 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec013470 con 0x7f3ff806e1b0 2024-04-04T06:16:52.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.454+0000 7f3ffe400640 1 -- 172.21.15.159:0/1931316095 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff81f47f0 con 0x7f3ff806e1b0 2024-04-04T06:16:52.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.455+0000 7f3ffe400640 1 -- 172.21.15.159:0/1931316095 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff81f4db0 con 0x7f3ff806e1b0 2024-04-04T06:16:52.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.456+0000 7f3ffe400640 1 -- 172.21.15.159:0/1931316095 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ff8075940 con 0x7f3ff806e1b0 2024-04-04T06:16:52.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.456+0000 7f3ff4c00640 1 -- 172.21.15.159:0/1931316095 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3fec01a070 con 0x7f3ff806e1b0 2024-04-04T06:16:52.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.457+0000 7f3ff4c00640 1 --2- 172.21.15.159:0/1931316095 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3fd806c8f0 0x7f3fd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:52.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.457+0000 7f3ff4c00640 1 -- 172.21.15.159:0/1931316095 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5029+0+0 (secure 0 0 0) 0x7f3fec08b7c0 con 0x7f3ff806e1b0 2024-04-04T06:16:52.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.458+0000 7f3ff7e00640 1 --2- 172.21.15.159:0/1931316095 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3fd806c8f0 0x7f3fd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:52.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.459+0000 7f3ff7e00640 1 --2- 172.21.15.159:0/1931316095 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3fd806c8f0 0x7f3fd806edb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8159eb0 tx=0x7f3fe4007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:52.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.462+0000 7f3ff4c00640 1 -- 172.21.15.159:0/1931316095 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3ff8159b70 con 0x7f3ff806e1b0 2024-04-04T06:16:52.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.670+0000 7f3ffe400640 1 -- 172.21.15.159:0/1931316095 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3ff8070260 con 0x7f3ff806e1b0 2024-04-04T06:16:52.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.757+0000 7f3ff4c00640 1 -- 172.21.15.159:0/1931316095 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v107) ==== 97+0+0 (secure 0 0 0) 0x7f3fec059830 con 0x7f3ff806e1b0 2024-04-04T06:16:52.753 INFO:teuthology.orchestra.run.smithi159.stderr: removed all blocklist entries 2024-04-04T06:16:52.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.760+0000 7f3ffe400640 1 -- 172.21.15.159:0/1931316095 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3fd806c8f0 msgr2=0x7f3fd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:52.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.760+0000 7f3ffe400640 1 --2- 172.21.15.159:0/1931316095 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3fd806c8f0 0x7f3fd806edb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8159eb0 tx=0x7f3fe4007e80 comp rx=0 tx=0).stop 2024-04-04T06:16:52.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.760+0000 7f3ffe400640 1 -- 172.21.15.159:0/1931316095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff806e1b0 msgr2=0x7f3ff8159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:52.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.760+0000 7f3ffe400640 1 --2- 172.21.15.159:0/1931316095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff806e1b0 0x7f3ff8159630 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00ebe0 tx=0x7f3fec00c520 comp rx=0 tx=0).stop 2024-04-04T06:16:52.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.760+0000 7f3ffe400640 1 -- 172.21.15.159:0/1931316095 shutdown_connections 2024-04-04T06:16:52.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.760+0000 7f3ffe400640 1 --2- 172.21.15.159:0/1931316095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ff815e530 0x7f3ff815f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.760+0000 7f3ffe400640 1 --2- 172.21.15.159:0/1931316095 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3fd806c8f0 0x7f3fd806edb0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.760+0000 7f3ffe400640 1 --2- 172.21.15.159:0/1931316095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ff80cf330 0x7f3ff81f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.760+0000 7f3ffe400640 1 --2- 172.21.15.159:0/1931316095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff806e1b0 0x7f3ff8159630 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.760+0000 7f3ffe400640 1 -- 172.21.15.159:0/1931316095 >> 172.21.15.159:0/1931316095 conn(0x7f3ff8067280 msgr2=0x7f3ff806d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:52.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.760+0000 7f3ffe400640 1 -- 172.21.15.159:0/1931316095 shutdown_connections 2024-04-04T06:16:52.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.760+0000 7f3ffe400640 1 -- 172.21.15.159:0/1931316095 wait complete. 2024-04-04T06:16:52.775 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-04T06:16:52.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.974+0000 7fb676400640 1 Processor -- start 2024-04-04T06:16:52.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.975+0000 7fb676400640 1 -- start start 2024-04-04T06:16:52.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.976+0000 7fb676400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6700b4650 0x7fb67006c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:52.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.976+0000 7fb676400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb6700b4a30 0x7fb6700b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:52.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.976+0000 7fb676400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb670158580 0x7fb6701589d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:52.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.976+0000 7fb676400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fb670141b30 con 0x7fb670158580 2024-04-04T06:16:52.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.976+0000 7fb676400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb67015ede0 con 0x7fb6700b4a30 2024-04-04T06:16:52.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.976+0000 7fb676400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb670141d80 con 0x7fb6700b4650 2024-04-04T06:16:52.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.976+0000 7fb66fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb670158580 0x7fb6701589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:52.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.976+0000 7fb66fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb670158580 0x7fb6701589d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52562/0 (socket says 172.21.15.159:52562) 2024-04-04T06:16:52.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.976+0000 7fb66fe00640 1 -- 172.21.15.159:0/1399621380 learned_addr learned my addr 172.21.15.159:0/1399621380 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:52.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.977+0000 7fb66fe00640 1 -- 172.21.15.159:0/1399621380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6700b4650 msgr2=0x7fb67006c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:52.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.977+0000 7fb66fe00640 1 --2- 172.21.15.159:0/1399621380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6700b4650 0x7fb67006c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.977+0000 7fb66fe00640 1 -- 172.21.15.159:0/1399621380 >> v2:172.21.15.178:3300/0 conn(0x7fb6700b4a30 msgr2=0x7fb6700b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:52.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.977+0000 7fb66fe00640 1 --2- 172.21.15.159:0/1399621380 >> v2:172.21.15.178:3300/0 conn(0x7fb6700b4a30 0x7fb6700b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.972 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.977+0000 7fb66fe00640 1 -- 172.21.15.159:0/1399621380 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb670142060 con 0x7fb670158580 2024-04-04T06:16:52.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.978+0000 7fb66fe00640 1 --2- 172.21.15.159:0/1399621380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb670158580 0x7fb6701589d0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fb6700d32e0 tx=0x7fb66802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d69dd8aed325ec server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:52.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.978+0000 7fb66e000640 1 -- 172.21.15.159:0/1399621380 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb668035020 con 0x7fb670158580 2024-04-04T06:16:52.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.978+0000 7fb66e000640 1 -- 172.21.15.159:0/1399621380 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb668031870 con 0x7fb670158580 2024-04-04T06:16:52.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.978+0000 7fb66e000640 1 -- 172.21.15.159:0/1399621380 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb668031c40 con 0x7fb670158580 2024-04-04T06:16:52.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.979+0000 7fb676400640 1 -- 172.21.15.159:0/1399621380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb670158580 msgr2=0x7fb6701589d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:52.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.979+0000 7fb676400640 1 --2- 172.21.15.159:0/1399621380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb670158580 0x7fb6701589d0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fb6700d32e0 tx=0x7fb66802b910 comp rx=0 tx=0).stop 2024-04-04T06:16:52.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.979+0000 7fb676400640 1 -- 172.21.15.159:0/1399621380 shutdown_connections 2024-04-04T06:16:52.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.979+0000 7fb676400640 1 --2- 172.21.15.159:0/1399621380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb670158580 0x7fb6701589d0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.979+0000 7fb676400640 1 --2- 172.21.15.159:0/1399621380 >> v2:172.21.15.178:3300/0 conn(0x7fb6700b4a30 0x7fb6700b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.979+0000 7fb676400640 1 --2- 172.21.15.159:0/1399621380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6700b4650 0x7fb67006c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.979+0000 7fb676400640 1 -- 172.21.15.159:0/1399621380 >> 172.21.15.159:0/1399621380 conn(0x7fb670067670 msgr2=0x7fb670067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:52.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.979+0000 7fb676400640 1 -- 172.21.15.159:0/1399621380 shutdown_connections 2024-04-04T06:16:52.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.979+0000 7fb676400640 1 -- 172.21.15.159:0/1399621380 wait complete. 2024-04-04T06:16:52.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.980+0000 7fb676400640 1 Processor -- start 2024-04-04T06:16:52.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.980+0000 7fb676400640 1 -- start start 2024-04-04T06:16:52.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.981+0000 7fb676400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6700b4650 0x7fb6701440a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:52.977 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.981+0000 7fb676400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6700b4a30 0x7fb67015a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:52.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.981+0000 7fb676400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb670158580 0x7fb67015ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:52.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.981+0000 7fb676400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb670142900 con 0x7fb6700b4a30 2024-04-04T06:16:52.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.981+0000 7fb676400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb670142620 con 0x7fb670158580 2024-04-04T06:16:52.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.981+0000 7fb676400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb670158f10 con 0x7fb6700b4650 2024-04-04T06:16:52.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.982+0000 7fb66ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6700b4a30 0x7fb67015a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:52.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.982+0000 7fb66ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6700b4a30 0x7fb67015a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52578/0 (socket says 172.21.15.159:52578) 2024-04-04T06:16:52.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.982+0000 7fb66ea00640 1 -- 172.21.15.159:0/970398904 learned_addr learned my addr 172.21.15.159:0/970398904 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:52.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.982+0000 7fb66f400640 1 --2- 172.21.15.159:0/970398904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6700b4650 0x7fb6701440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:52.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.982+0000 7fb66fe00640 1 --2- 172.21.15.159:0/970398904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb670158580 0x7fb67015ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:52.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.982+0000 7fb66ea00640 1 -- 172.21.15.159:0/970398904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6700b4650 msgr2=0x7fb6701440a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:52.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.982+0000 7fb66ea00640 1 --2- 172.21.15.159:0/970398904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6700b4650 0x7fb6701440a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.982+0000 7fb66ea00640 1 -- 172.21.15.159:0/970398904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb670158580 msgr2=0x7fb67015ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:52.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.982+0000 7fb66ea00640 1 --2- 172.21.15.159:0/970398904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb670158580 0x7fb67015ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:52.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.982+0000 7fb66ea00640 1 -- 172.21.15.159:0/970398904 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6700cd2c0 con 0x7fb6700b4a30 2024-04-04T06:16:52.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.982+0000 7fb66ea00640 1 --2- 172.21.15.159:0/970398904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6700b4a30 0x7fb67015a8c0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00b410 tx=0x7fb65c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:52.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.983+0000 7fb667400640 1 -- 172.21.15.159:0/970398904 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c015020 con 0x7fb6700b4a30 2024-04-04T06:16:52.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.983+0000 7fb667400640 1 -- 172.21.15.159:0/970398904 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb65c011850 con 0x7fb6700b4a30 2024-04-04T06:16:52.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.983+0000 7fb676400640 1 -- 172.21.15.159:0/970398904 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6700cd5a0 con 0x7fb6700b4a30 2024-04-04T06:16:52.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.983+0000 7fb667400640 1 -- 172.21.15.159:0/970398904 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c011c40 con 0x7fb6700b4a30 2024-04-04T06:16:52.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.983+0000 7fb676400640 1 -- 172.21.15.159:0/970398904 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6700cdb60 con 0x7fb6700b4a30 2024-04-04T06:16:52.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.984+0000 7fb676400640 1 -- 172.21.15.159:0/970398904 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6700b73d0 con 0x7fb6700b4a30 2024-04-04T06:16:52.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.985+0000 7fb667400640 1 -- 172.21.15.159:0/970398904 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb65c018020 con 0x7fb6700b4a30 2024-04-04T06:16:52.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.986+0000 7fb667400640 1 --2- 172.21.15.159:0/970398904 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb65006c8f0 0x7fb65006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:52.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.986+0000 7fb667400640 1 -- 172.21.15.159:0/970398904 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4857+0+0 (secure 0 0 0) 0x7fb65c08b7b0 con 0x7fb6700b4a30 2024-04-04T06:16:52.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.986+0000 7fb66f400640 1 --2- 172.21.15.159:0/970398904 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb65006c8f0 0x7fb65006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:52.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.987+0000 7fb66f400640 1 --2- 172.21.15.159:0/970398904 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb65006c8f0 0x7fb65006edb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb660009f20 tx=0x7fb6600094f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:52.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:52.991+0000 7fb667400640 1 -- 172.21.15.159:0/970398904 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb6700b73d0 con 0x7fb6700b4a30 2024-04-04T06:16:53.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.261+0000 7fb676400640 1 -- 172.21.15.159:0/970398904 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb6700685c0 con 0x7fb6700b4a30 2024-04-04T06:16:53.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.263+0000 7fb667400640 1 -- 172.21.15.159:0/970398904 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7fb65c01f020 con 0x7fb6700b4a30 2024-04-04T06:16:53.258 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:53.258 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:16:53.258 INFO:teuthology.orchestra.run.smithi159.stdout: "auth_dump": [ 2024-04-04T06:16:53.258 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:53.258 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.a", 2024-04-04T06:16:53.258 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5mvoZ2FxAAsVEQrkiICIH6T7ZzaqcWbA==", 2024-04-04T06:16:53.258 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:16:53.258 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:16:53.258 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:16:53.258 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:16:53.259 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:16:53.259 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:53.259 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:16:53.259 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:53.259 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.b", 2024-04-04T06:16:53.259 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5m+rccORAA6I/lQ+22QfWYL9+aoOFJ0w==", 2024-04-04T06:16:53.259 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:16:53.259 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:16:53.259 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:16:53.259 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:16:53.259 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:16:53.259 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:53.259 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:16:53.260 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:53.260 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.c", 2024-04-04T06:16:53.260 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5mjmQcKBAA8BjpQce6G7EycXOkyVECpQ==", 2024-04-04T06:16:53.260 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:16:53.260 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:16:53.260 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:16:53.260 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:16:53.260 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:16:53.260 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:53.260 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:16:53.260 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:53.260 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mds.d", 2024-04-04T06:16:53.260 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5m04hoDRAAKdhRI7nosZp7d4y7dVvzvg==", 2024-04-04T06:16:53.260 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:16:53.260 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:16:53.261 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:16:53.261 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:16:53.261 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:16:53.261 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:53.261 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:16:53.261 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:53.261 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.0", 2024-04-04T06:16:53.261 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQABQw5mCe7fNRAA6ZKxWvnMLf4jhAlpjudFRA==", 2024-04-04T06:16:53.261 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:16:53.261 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:16:53.261 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:16:53.261 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:16:53.261 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:53.261 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:16:53.261 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:53.262 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.1", 2024-04-04T06:16:53.262 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAFQw5mvIqTExAAU8Pm5alxFtHEL4qM9MavPw==", 2024-04-04T06:16:53.262 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:16:53.262 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:16:53.262 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:16:53.262 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:16:53.263 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:53.263 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:16:53.263 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:53.263 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.2", 2024-04-04T06:16:53.263 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAIQw5mAHblHRAAqEDe5dIGTDpdy2OsQyzJXA==", 2024-04-04T06:16:53.263 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:16:53.263 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:16:53.263 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:16:53.263 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:16:53.263 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:53.263 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:16:53.263 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:53.263 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.3", 2024-04-04T06:16:53.263 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQALQw5mwK7rJhAADQBAqE0Eej9rn91Wxhq/KA==", 2024-04-04T06:16:53.264 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:16:53.264 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:16:53.264 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:16:53.264 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:16:53.264 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:53.264 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:16:53.264 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:53.264 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.4", 2024-04-04T06:16:53.264 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAQQw5m/6I8GRAA2P7jI+AXYld2J11nDfhAUQ==", 2024-04-04T06:16:53.264 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:16:53.264 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:16:53.264 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:16:53.264 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:16:53.264 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:53.264 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:16:53.265 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:53.265 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.5", 2024-04-04T06:16:53.265 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQATQw5mv5HoMBAAn2mp2ys5KjS0UygOcX4vvg==", 2024-04-04T06:16:53.265 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:16:53.265 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:16:53.265 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:16:53.265 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:16:53.265 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:53.265 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:16:53.265 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:53.265 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.6", 2024-04-04T06:16:53.265 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAWQw5mx12gORAAT2LhrfEB5BJSl0ypgospzg==", 2024-04-04T06:16:53.265 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:16:53.265 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:16:53.265 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:16:53.266 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:16:53.266 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:53.266 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:16:53.266 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:53.266 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "osd.7", 2024-04-04T06:16:53.266 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQAaQw5m6HEOBxAAVegLH3bjVM7r1A6tK5eqHQ==", 2024-04-04T06:16:53.266 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:16:53.266 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow profile osd", 2024-04-04T06:16:53.266 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile osd", 2024-04-04T06:16:53.266 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:16:53.266 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:53.266 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:16:53.266 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:53.266 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.0", 2024-04-04T06:16:53.267 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mqNKJGxAA4jP9cadhRIyhQ3lssttWIw==", 2024-04-04T06:16:53.267 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:16:53.267 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:16:53.267 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow r", 2024-04-04T06:16:53.267 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-04T06:16:53.267 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:53.267 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:16:53.267 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:53.267 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.1", 2024-04-04T06:16:53.267 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mRkdOMBAA1ae1roIwrlB49DFQUoFSTg==", 2024-04-04T06:16:53.267 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:16:53.267 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:16:53.267 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:16:53.267 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:16:53.267 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:16:53.268 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:53.268 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:16:53.268 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:53.268 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.2", 2024-04-04T06:16:53.268 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mtW2/JRAAucxSSN9q0EQtI2a/P2yozQ==", 2024-04-04T06:16:53.268 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:16:53.268 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:16:53.268 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:16:53.268 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:16:53.268 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:16:53.269 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:53.269 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:16:53.269 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:53.269 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.3", 2024-04-04T06:16:53.269 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD9Qg5mgqD0OhAAov+3aDGyGEoqxep8+SBpSA==", 2024-04-04T06:16:53.269 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:16:53.269 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow", 2024-04-04T06:16:53.269 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow r", 2024-04-04T06:16:53.269 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow rw", 2024-04-04T06:16:53.269 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow rwx" 2024-04-04T06:16:53.270 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:53.270 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:16:53.270 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:53.270 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "client.admin", 2024-04-04T06:16:53.270 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD7Qg5m0O/iCBAAsWTzlEA0P60XYuxmJiAb4A==", 2024-04-04T06:16:53.270 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:16:53.270 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow *", 2024-04-04T06:16:53.270 INFO:teuthology.orchestra.run.smithi159.stdout: "mgr": "allow *", 2024-04-04T06:16:53.270 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow *", 2024-04-04T06:16:53.270 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:16:53.270 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:53.270 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:16:53.270 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:53.270 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mgr.x", 2024-04-04T06:16:53.271 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD8Qg5mhqbBCxAASBknMIj4uu7OiB3jn7WP8Q==", 2024-04-04T06:16:53.271 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:16:53.271 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow *", 2024-04-04T06:16:53.271 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile mgr", 2024-04-04T06:16:53.271 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:16:53.271 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:53.271 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-04-04T06:16:53.271 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:53.271 INFO:teuthology.orchestra.run.smithi159.stdout: "entity": "mgr.y", 2024-04-04T06:16:53.271 INFO:teuthology.orchestra.run.smithi159.stdout: "key": "AQD7Qg5miXT0ORAAL6y306CPAfEGQKAJ0no4uw==", 2024-04-04T06:16:53.271 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": { 2024-04-04T06:16:53.271 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": "allow *", 2024-04-04T06:16:53.271 INFO:teuthology.orchestra.run.smithi159.stdout: "mon": "allow profile mgr", 2024-04-04T06:16:53.271 INFO:teuthology.orchestra.run.smithi159.stdout: "osd": "allow *" 2024-04-04T06:16:53.271 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:53.272 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:53.272 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-04-04T06:16:53.272 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:16:53.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.265+0000 7fb676400640 1 -- 172.21.15.159:0/970398904 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb65006c8f0 msgr2=0x7fb65006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.265+0000 7fb676400640 1 --2- 172.21.15.159:0/970398904 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb65006c8f0 0x7fb65006edb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb660009f20 tx=0x7fb6600094f0 comp rx=0 tx=0).stop 2024-04-04T06:16:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.265+0000 7fb676400640 1 -- 172.21.15.159:0/970398904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6700b4a30 msgr2=0x7fb67015a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.265+0000 7fb676400640 1 --2- 172.21.15.159:0/970398904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6700b4a30 0x7fb67015a8c0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00b410 tx=0x7fb65c00b8d0 comp rx=0 tx=0).stop 2024-04-04T06:16:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.265+0000 7fb676400640 1 -- 172.21.15.159:0/970398904 shutdown_connections 2024-04-04T06:16:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.265+0000 7fb676400640 1 --2- 172.21.15.159:0/970398904 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb65006c8f0 0x7fb65006edb0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.265+0000 7fb676400640 1 --2- 172.21.15.159:0/970398904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb670158580 0x7fb67015ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.265+0000 7fb676400640 1 --2- 172.21.15.159:0/970398904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6700b4a30 0x7fb67015a8c0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.265+0000 7fb676400640 1 --2- 172.21.15.159:0/970398904 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb6700b4650 0x7fb6701440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.265+0000 7fb676400640 1 -- 172.21.15.159:0/970398904 >> 172.21.15.159:0/970398904 conn(0x7fb670067670 msgr2=0x7fb670161070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.265+0000 7fb676400640 1 -- 172.21.15.159:0/970398904 shutdown_connections 2024-04-04T06:16:53.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.265+0000 7fb676400640 1 -- 172.21.15.159:0/970398904 wait complete. 2024-04-04T06:16:53.278 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:16:53.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.492+0000 7fce9ec00640 1 Processor -- start 2024-04-04T06:16:53.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.492+0000 7fce9ec00640 1 -- start start 2024-04-04T06:16:53.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.494+0000 7fce9ec00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce980727f0 0x7fce980b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:53.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.494+0000 7fce9ec00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fce980d0780 0x7fce980d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:53.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.494+0000 7fce9ec00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce9815b6f0 con 0x7fce9815b310 2024-04-04T06:16:53.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.494+0000 7fce9ec00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fce98072c50 con 0x7fce980d0780 2024-04-04T06:16:53.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.494+0000 7fce9ec00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fce98072e80 con 0x7fce980727f0 2024-04-04T06:16:53.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.494+0000 7fce97e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fce9815b310 0x7fce9815f7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40360/0 (socket says 172.21.15.159:40360) 2024-04-04T06:16:53.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.494+0000 7fce97e00640 1 -- 172.21.15.159:0/1966620546 learned_addr learned my addr 172.21.15.159:0/1966620546 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:16:53.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.495+0000 7fce8f400640 1 --2- 172.21.15.159:0/1966620546 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce980727f0 0x7fce980b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:53.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.495+0000 7fce9cc00640 1 --2- 172.21.15.159:0/1966620546 >> v2:172.21.15.178:3300/0 conn(0x7fce980d0780 0x7fce980d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:53.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.495+0000 7fce97400640 1 -- 172.21.15.159:0/1966620546 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1195701086 0 0) 0x7fce9815b6f0 con 0x7fce9815b310 2024-04-04T06:16:53.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.495+0000 7fce97400640 1 -- 172.21.15.159:0/1966620546 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce7c003520 con 0x7fce9815b310 2024-04-04T06:16:53.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.495+0000 7fce97400640 1 -- 172.21.15.159:0/1966620546 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3765463434 0 0) 0x7fce7c003520 con 0x7fce9815b310 2024-04-04T06:16:53.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.495+0000 7fce97400640 1 -- 172.21.15.159:0/1966620546 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce980727f0 msgr2=0x7fce980b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:53.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.495+0000 7fce97400640 1 --2- 172.21.15.159:0/1966620546 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce980727f0 0x7fce980b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:53.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.495+0000 7fce97400640 1 -- 172.21.15.159:0/1966620546 >> v2:172.21.15.178:3300/0 conn(0x7fce980d0780 msgr2=0x7fce980d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:53.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.495+0000 7fce97400640 1 --2- 172.21.15.159:0/1966620546 >> v2:172.21.15.178:3300/0 conn(0x7fce980d0780 0x7fce980d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:53.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.495+0000 7fce97400640 1 -- 172.21.15.159:0/1966620546 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce98073160 con 0x7fce9815b310 2024-04-04T06:16:53.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.495+0000 7fce97400640 1 -- 172.21.15.159:0/1966620546 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fce88001670 con 0x7fce9815b310 2024-04-04T06:16:53.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.495+0000 7fce97400640 1 -- 172.21.15.159:0/1966620546 >> v1:172.21.15.159:6789/0 conn(0x7fce9815b310 legacy=0x7fce9815f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:53.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.495+0000 7fce97400640 1 --2- 172.21.15.159:0/1966620546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce7c0039b0 0x7fce7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:53.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.495+0000 7fce97400640 1 --2- 172.21.15.159:0/1966620546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce7c004800 0x7fce7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:53.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.495+0000 7fce97400640 1 --2- 172.21.15.159:0/1966620546 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce980727f0 0x7fce7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:53.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.495+0000 7fce97400640 1 -- 172.21.15.159:0/1966620546 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fce98072e80 con 0x7fce7c0039b0 2024-04-04T06:16:53.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.495+0000 7fce97400640 1 -- 172.21.15.159:0/1966620546 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c00b9c0 con 0x7fce7c004800 2024-04-04T06:16:53.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.495+0000 7fce97400640 1 -- 172.21.15.159:0/1966620546 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fce7c00bbc0 con 0x7fce980727f0 2024-04-04T06:16:53.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.495+0000 7fce97e00640 1 --2- 172.21.15.159:0/1966620546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce7c0039b0 0x7fce7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:53.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.496+0000 7fce97e00640 1 -- 172.21.15.159:0/1966620546 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce980727f0 msgr2=0x7fce7c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:53.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.496+0000 7fce97e00640 1 --2- 172.21.15.159:0/1966620546 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce980727f0 0x7fce7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:53.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.496+0000 7fce97e00640 1 -- 172.21.15.159:0/1966620546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce7c004800 msgr2=0x7fce7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:53.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.496+0000 7fce97e00640 1 --2- 172.21.15.159:0/1966620546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce7c004800 0x7fce7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:53.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.496+0000 7fce97e00640 1 -- 172.21.15.159:0/1966620546 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7c003520 con 0x7fce7c0039b0 2024-04-04T06:16:53.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.497+0000 7fce97e00640 1 --2- 172.21.15.159:0/1966620546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce7c0039b0 0x7fce7c003dd0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fce8800c240 tx=0x7fce7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db31972e3cb5f045 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:53.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.497+0000 7fce97400640 1 -- 172.21.15.159:0/1966620546 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce88007d30 con 0x7fce7c0039b0 2024-04-04T06:16:53.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.497+0000 7fce97400640 1 -- 172.21.15.159:0/1966620546 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce88030070 con 0x7fce7c0039b0 2024-04-04T06:16:53.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.497+0000 7fce97400640 1 -- 172.21.15.159:0/1966620546 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8802cd60 con 0x7fce7c0039b0 2024-04-04T06:16:53.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.498+0000 7fce9ec00640 1 -- 172.21.15.159:0/1966620546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce7c0039b0 msgr2=0x7fce7c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:53.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.498+0000 7fce9ec00640 1 --2- 172.21.15.159:0/1966620546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce7c0039b0 0x7fce7c003dd0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fce8800c240 tx=0x7fce7c005830 comp rx=0 tx=0).stop 2024-04-04T06:16:53.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.498+0000 7fce9cc00640 1 -- 172.21.15.159:0/1966620546 reap_dead start 2024-04-04T06:16:53.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.498+0000 7fce9ec00640 1 -- 172.21.15.159:0/1966620546 shutdown_connections 2024-04-04T06:16:53.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.498+0000 7fce9ec00640 1 -- 172.21.15.159:0/1966620546 >> 172.21.15.159:0/1966620546 conn(0x7fce9806b990 msgr2=0x7fce9806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:53.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.498+0000 7fce9ec00640 1 -- 172.21.15.159:0/1966620546 shutdown_connections 2024-04-04T06:16:53.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.498+0000 7fce9ec00640 1 -- 172.21.15.159:0/1966620546 wait complete. 2024-04-04T06:16:53.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.499+0000 7fce9ec00640 1 Processor -- start 2024-04-04T06:16:53.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.500+0000 7fce9ec00640 1 -- start start 2024-04-04T06:16:53.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.500+0000 7fce9ec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce7c0039b0 0x7fce9815c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:53.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.500+0000 7fce9ec00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce98154550 0x7fce9814d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:53.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.500+0000 7fce9ec00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce98150200 0x7fce98150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:53.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.500+0000 7fce9ec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fce98073440 con 0x7fce7c0039b0 2024-04-04T06:16:53.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.500+0000 7fce9ec00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fce9815f2c0 con 0x7fce98154550 2024-04-04T06:16:53.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.500+0000 7fce9ec00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fce980b4f90 con 0x7fce98150200 2024-04-04T06:16:53.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.501+0000 7fce97e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce7c0039b0 0x7fce9815c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:53.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.501+0000 7fce97e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce7c0039b0 0x7fce9815c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:52586/0 (socket says 172.21.15.159:52586) 2024-04-04T06:16:53.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.501+0000 7fce97e00640 1 -- 172.21.15.159:0/3784017013 learned_addr learned my addr 172.21.15.159:0/3784017013 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:53.497 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.501+0000 7fce97e00640 1 -- 172.21.15.159:0/3784017013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce98150200 msgr2=0x7fce98150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:53.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.501+0000 7fce97e00640 1 --2- 172.21.15.159:0/3784017013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce98150200 0x7fce98150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:53.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.501+0000 7fce97e00640 1 -- 172.21.15.159:0/3784017013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce98154550 msgr2=0x7fce9814d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:53.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.501+0000 7fce8f400640 1 --2- 172.21.15.159:0/3784017013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce98154550 0x7fce9814d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:53.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.501+0000 7fce97e00640 1 --2- 172.21.15.159:0/3784017013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce98154550 0x7fce9814d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:53.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.501+0000 7fce97e00640 1 -- 172.21.15.159:0/3784017013 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9815ce50 con 0x7fce7c0039b0 2024-04-04T06:16:53.498 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.501+0000 7fce97e00640 1 --2- 172.21.15.159:0/3784017013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce7c0039b0 0x7fce9815c560 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fce8802c570 tx=0x7fce98073330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:53.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.502+0000 7fce94c00640 1 -- 172.21.15.159:0/3784017013 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce88046070 con 0x7fce7c0039b0 2024-04-04T06:16:53.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.502+0000 7fce94c00640 1 -- 172.21.15.159:0/3784017013 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce88004bf0 con 0x7fce7c0039b0 2024-04-04T06:16:53.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.502+0000 7fce9ec00640 1 -- 172.21.15.159:0/3784017013 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce98154ae0 con 0x7fce7c0039b0 2024-04-04T06:16:53.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.502+0000 7fce94c00640 1 -- 172.21.15.159:0/3784017013 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce88007d30 con 0x7fce7c0039b0 2024-04-04T06:16:53.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.502+0000 7fce9ec00640 1 -- 172.21.15.159:0/3784017013 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce9815d3d0 con 0x7fce7c0039b0 2024-04-04T06:16:53.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.503+0000 7fce9ec00640 1 -- 172.21.15.159:0/3784017013 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce980685c0 con 0x7fce7c0039b0 2024-04-04T06:16:53.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.504+0000 7fce94c00640 1 -- 172.21.15.159:0/3784017013 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce88007070 con 0x7fce7c0039b0 2024-04-04T06:16:53.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.505+0000 7fce94c00640 1 --2- 172.21.15.159:0/3784017013 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce7406c8f0 0x7fce7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:53.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.505+0000 7fce94c00640 1 -- 172.21.15.159:0/3784017013 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4857+0+0 (secure 0 0 0) 0x7fce880ac920 con 0x7fce7c0039b0 2024-04-04T06:16:53.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.506+0000 7fce8f400640 1 --2- 172.21.15.159:0/3784017013 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce7406c8f0 0x7fce7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:53.501 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.507+0000 7fce8f400640 1 --2- 172.21.15.159:0/3784017013 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce7406c8f0 0x7fce7406edb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fce980ae940 tx=0x7fce80008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:53.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.509+0000 7fce94c00640 1 -- 172.21.15.159:0/3784017013 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fce8807aac0 con 0x7fce7c0039b0 2024-04-04T06:16:53.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.703+0000 7fce9ec00640 1 -- 172.21.15.159:0/3784017013 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce9815ec20 con 0x7fce7c0039b0 2024-04-04T06:16:53.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.704+0000 7fce94c00640 1 -- 172.21.15.159:0/3784017013 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+12905 (secure 0 0 0) 0x7fce8807edc0 con 0x7fce7c0039b0 2024-04-04T06:16:53.699 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:53.699 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":109,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:16:52.770035+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:16:53.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.706+0000 7fce9ec00640 1 -- 172.21.15.159:0/3784017013 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce7406c8f0 msgr2=0x7fce7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:53.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.706+0000 7fce9ec00640 1 --2- 172.21.15.159:0/3784017013 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce7406c8f0 0x7fce7406edb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fce980ae940 tx=0x7fce80008040 comp rx=0 tx=0).stop 2024-04-04T06:16:53.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.706+0000 7fce9ec00640 1 -- 172.21.15.159:0/3784017013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce7c0039b0 msgr2=0x7fce9815c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:53.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.706+0000 7fce9ec00640 1 --2- 172.21.15.159:0/3784017013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce7c0039b0 0x7fce9815c560 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fce8802c570 tx=0x7fce98073330 comp rx=0 tx=0).stop 2024-04-04T06:16:53.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.706+0000 7fce9ec00640 1 -- 172.21.15.159:0/3784017013 shutdown_connections 2024-04-04T06:16:53.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.706+0000 7fce9ec00640 1 --2- 172.21.15.159:0/3784017013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fce98150200 0x7fce98150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:53.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.706+0000 7fce9ec00640 1 --2- 172.21.15.159:0/3784017013 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fce7406c8f0 0x7fce7406edb0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:53.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.706+0000 7fce9ec00640 1 --2- 172.21.15.159:0/3784017013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce98154550 0x7fce9814d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:53.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.706+0000 7fce9ec00640 1 --2- 172.21.15.159:0/3784017013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce7c0039b0 0x7fce9815c560 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:53.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.706+0000 7fce9ec00640 1 -- 172.21.15.159:0/3784017013 >> 172.21.15.159:0/3784017013 conn(0x7fce9806b990 msgr2=0x7fce980b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:53.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.707+0000 7fce9ec00640 1 -- 172.21.15.159:0/3784017013 shutdown_connections 2024-04-04T06:16:53.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.707+0000 7fce9ec00640 1 -- 172.21.15.159:0/3784017013 wait complete. 2024-04-04T06:16:53.720 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:16:53.721 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:16:53.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.926+0000 7fe763e00640 1 Processor -- start 2024-04-04T06:16:53.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.926+0000 7fe763e00640 1 -- start start 2024-04-04T06:16:53.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.928+0000 7fe763e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe7640b3ec0 0x7fe76406d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:53.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.928+0000 7fe763e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe76406db60 0x7fe7640add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:53.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.928+0000 7fe763e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe76415c330 con 0x7fe76415bf50 2024-04-04T06:16:53.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.928+0000 7fe763e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe76406dff0 con 0x7fe7640b3ec0 2024-04-04T06:16:53.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.928+0000 7fe763e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe76406ee40 con 0x7fe76406db60 2024-04-04T06:16:53.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.928+0000 7fe75be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe7640b3ec0 0x7fe76406d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:53.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.929+0000 7fe75be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe7640b3ec0 0x7fe76406d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43892/0 (socket says 172.21.15.159:43892) 2024-04-04T06:16:53.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.929+0000 7fe75be00640 1 -- 172.21.15.159:0/1103646118 learned_addr learned my addr 172.21.15.159:0/1103646118 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:53.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.929+0000 7fe763400640 1 --2- 172.21.15.159:0/1103646118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe76406db60 0x7fe7640add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:53.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.929+0000 7fe75be00640 1 -- 172.21.15.159:0/1103646118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe76406db60 msgr2=0x7fe7640add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:53.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.929+0000 7fe75be00640 1 --2- 172.21.15.159:0/1103646118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe76406db60 0x7fe7640add30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:53.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.929+0000 7fe75be00640 1 -- 172.21.15.159:0/1103646118 >> v1:172.21.15.178:6789/0 conn(0x7fe76415bf50 legacy=0x7fe7640b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:53.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.929+0000 7fe75be00640 1 -- 172.21.15.159:0/1103646118 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76406e230 con 0x7fe7640b3ec0 2024-04-04T06:16:53.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.930+0000 7fe75be00640 1 --2- 172.21.15.159:0/1103646118 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7640b3ec0 0x7fe76406d620 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009870 tx=0x7fe74c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fd1caa48d74e29 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:53.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.930+0000 7fe762000640 1 -- 172.21.15.159:0/1103646118 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c034070 con 0x7fe7640b3ec0 2024-04-04T06:16:53.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.930+0000 7fe762000640 1 -- 172.21.15.159:0/1103646118 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe74c030840 con 0x7fe7640b3ec0 2024-04-04T06:16:53.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.930+0000 7fe762000640 1 -- 172.21.15.159:0/1103646118 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c030c10 con 0x7fe7640b3ec0 2024-04-04T06:16:53.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.931+0000 7fe763e00640 1 -- 172.21.15.159:0/1103646118 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7640b3ec0 msgr2=0x7fe76406d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:53.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.931+0000 7fe763e00640 1 --2- 172.21.15.159:0/1103646118 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7640b3ec0 0x7fe76406d620 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009870 tx=0x7fe74c02b980 comp rx=0 tx=0).stop 2024-04-04T06:16:53.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.931+0000 7fe763e00640 1 -- 172.21.15.159:0/1103646118 shutdown_connections 2024-04-04T06:16:53.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.931+0000 7fe763e00640 1 --2- 172.21.15.159:0/1103646118 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe76406db60 0x7fe7640add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:53.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.931+0000 7fe763e00640 1 --2- 172.21.15.159:0/1103646118 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7640b3ec0 0x7fe76406d620 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:53.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.931+0000 7fe763e00640 1 -- 172.21.15.159:0/1103646118 >> 172.21.15.159:0/1103646118 conn(0x7fe7640b1ba0 msgr2=0x7fe7640b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:53.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.931+0000 7fe763e00640 1 -- 172.21.15.159:0/1103646118 shutdown_connections 2024-04-04T06:16:53.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.931+0000 7fe763e00640 1 -- 172.21.15.159:0/1103646118 wait complete. 2024-04-04T06:16:53.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.933+0000 7fe763e00640 1 Processor -- start 2024-04-04T06:16:53.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.933+0000 7fe763e00640 1 -- start start 2024-04-04T06:16:53.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.934+0000 7fe763e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe76406db60 0x7fe7641f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:53.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.934+0000 7fe763e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7640b3ec0 0x7fe7640c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:53.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.934+0000 7fe763e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe76415bf50 0x7fe76414cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:53.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.934+0000 7fe763e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe76406e7f0 con 0x7fe76406db60 2024-04-04T06:16:53.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.934+0000 7fe763e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe76415ed30 con 0x7fe76415bf50 2024-04-04T06:16:53.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.934+0000 7fe763e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe76406ead0 con 0x7fe7640b3ec0 2024-04-04T06:16:53.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.934+0000 7fe762a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe76406db60 0x7fe7641f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:53.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.934+0000 7fe762a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe76406db60 0x7fe7641f5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43896/0 (socket says 172.21.15.159:43896) 2024-04-04T06:16:53.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.934+0000 7fe762a00640 1 -- 172.21.15.159:0/2248535066 learned_addr learned my addr 172.21.15.159:0/2248535066 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:53.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.934+0000 7fe75be00640 1 --2- 172.21.15.159:0/2248535066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7640b3ec0 0x7fe7640c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:53.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.934+0000 7fe762a00640 1 -- 172.21.15.159:0/2248535066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7640b3ec0 msgr2=0x7fe7640c4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:53.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.934+0000 7fe762a00640 1 --2- 172.21.15.159:0/2248535066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7640b3ec0 0x7fe7640c4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:53.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.934+0000 7fe762a00640 1 -- 172.21.15.159:0/2248535066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe76415bf50 msgr2=0x7fe76414cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:53.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.934+0000 7fe762a00640 1 --2- 172.21.15.159:0/2248535066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe76415bf50 0x7fe76414cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:53.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.934+0000 7fe762a00640 1 -- 172.21.15.159:0/2248535066 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7640c5350 con 0x7fe76406db60 2024-04-04T06:16:53.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.935+0000 7fe762a00640 1 --2- 172.21.15.159:0/2248535066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe76406db60 0x7fe7641f5250 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fe75400c430 tx=0x7fe75400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:53.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.935+0000 7fe75b400640 1 -- 172.21.15.159:0/2248535066 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754011790 con 0x7fe76406db60 2024-04-04T06:16:53.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.935+0000 7fe75b400640 1 -- 172.21.15.159:0/2248535066 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe754017070 con 0x7fe76406db60 2024-04-04T06:16:53.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.935+0000 7fe763e00640 1 -- 172.21.15.159:0/2248535066 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe76414d2b0 con 0x7fe76406db60 2024-04-04T06:16:53.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.935+0000 7fe75b400640 1 -- 172.21.15.159:0/2248535066 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754011bb0 con 0x7fe76406db60 2024-04-04T06:16:53.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.935+0000 7fe763e00640 1 -- 172.21.15.159:0/2248535066 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7640c58b0 con 0x7fe76406db60 2024-04-04T06:16:53.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.936+0000 7fe763e00640 1 -- 172.21.15.159:0/2248535066 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe764075940 con 0x7fe76406db60 2024-04-04T06:16:53.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.937+0000 7fe75b400640 1 -- 172.21.15.159:0/2248535066 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe75401c460 con 0x7fe76406db60 2024-04-04T06:16:53.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.938+0000 7fe75b400640 1 --2- 172.21.15.159:0/2248535066 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe74006c920 0x7fe74006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:53.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.938+0000 7fe75b400640 1 -- 172.21.15.159:0/2248535066 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4857+0+0 (secure 0 0 0) 0x7fe75408b720 con 0x7fe76406db60 2024-04-04T06:16:53.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.939+0000 7fe75be00640 1 --2- 172.21.15.159:0/2248535066 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe74006c920 0x7fe74006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:53.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.940+0000 7fe75be00640 1 --2- 172.21.15.159:0/2248535066 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe74006c920 0x7fe74006ede0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe74c036000 tx=0x7fe74c03b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:53.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:53.942+0000 7fe75b400640 1 -- 172.21.15.159:0/2248535066 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe76414d2b0 con 0x7fe76406db60 2024-04-04T06:16:54.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.140+0000 7fe763e00640 1 -- 172.21.15.159:0/2248535066 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe76415a240 con 0x7fe76406db60 2024-04-04T06:16:54.136 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.141+0000 7fe75b400640 1 -- 172.21.15.159:0/2248535066 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7fe7640c5350 con 0x7fe76406db60 2024-04-04T06:16:54.136 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:16:54.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.143+0000 7fe763e00640 1 -- 172.21.15.159:0/2248535066 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe74006c920 msgr2=0x7fe74006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:54.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.143+0000 7fe763e00640 1 --2- 172.21.15.159:0/2248535066 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe74006c920 0x7fe74006ede0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe74c036000 tx=0x7fe74c03b040 comp rx=0 tx=0).stop 2024-04-04T06:16:54.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.143+0000 7fe763e00640 1 -- 172.21.15.159:0/2248535066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe76406db60 msgr2=0x7fe7641f5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:54.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.143+0000 7fe763e00640 1 --2- 172.21.15.159:0/2248535066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe76406db60 0x7fe7641f5250 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fe75400c430 tx=0x7fe75400c8f0 comp rx=0 tx=0).stop 2024-04-04T06:16:54.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.144+0000 7fe763e00640 1 -- 172.21.15.159:0/2248535066 shutdown_connections 2024-04-04T06:16:54.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.144+0000 7fe763e00640 1 --2- 172.21.15.159:0/2248535066 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe74006c920 0x7fe74006ede0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:54.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.144+0000 7fe763e00640 1 --2- 172.21.15.159:0/2248535066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe76415bf50 0x7fe76414cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:54.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.144+0000 7fe763e00640 1 --2- 172.21.15.159:0/2248535066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe7640b3ec0 0x7fe7640c4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:54.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.144+0000 7fe763e00640 1 --2- 172.21.15.159:0/2248535066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe76406db60 0x7fe7641f5250 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:54.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.144+0000 7fe763e00640 1 -- 172.21.15.159:0/2248535066 >> 172.21.15.159:0/2248535066 conn(0x7fe7640b1ba0 msgr2=0x7fe764160200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:54.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.144+0000 7fe763e00640 1 -- 172.21.15.159:0/2248535066 shutdown_connections 2024-04-04T06:16:54.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.144+0000 7fe763e00640 1 -- 172.21.15.159:0/2248535066 wait complete. 2024-04-04T06:16:54.156 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-04T06:16:54.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.370+0000 7efd47400640 1 Processor -- start 2024-04-04T06:16:54.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.370+0000 7efd47400640 1 -- start start 2024-04-04T06:16:54.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.372+0000 7efd47400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efd40155550 0x7efd40155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:54.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.372+0000 7efd47400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7efd400d0990 0x7efd400aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:54.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.372+0000 7efd47400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd400b40b0 con 0x7efd4015f800 2024-04-04T06:16:54.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.372+0000 7efd47400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7efd400b4390 con 0x7efd400d0990 2024-04-04T06:16:54.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.372+0000 7efd47400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7efd400b4670 con 0x7efd40155550 2024-04-04T06:16:54.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.372+0000 7efd45400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7efd400d0990 0x7efd400aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:54.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.372+0000 7efd45400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7efd400d0990 0x7efd400aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43906/0 (socket says 172.21.15.159:43906) 2024-04-04T06:16:54.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.372+0000 7efd3fe00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7efd4015f800 0x7efd400d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45212/0 (socket says 172.21.15.159:45212) 2024-04-04T06:16:54.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.372+0000 7efd45400640 1 -- 172.21.15.159:0/2709158660 learned_addr learned my addr 172.21.15.159:0/2709158660 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:54.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.372+0000 7efd44a00640 1 --2- 172.21.15.159:0/2709158660 >> v2:172.21.15.178:3300/0 conn(0x7efd40155550 0x7efd40155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:54.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.373+0000 7efd45400640 1 -- 172.21.15.159:0/2709158660 >> v2:172.21.15.178:3300/0 conn(0x7efd40155550 msgr2=0x7efd40155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:54.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.373+0000 7efd45400640 1 --2- 172.21.15.159:0/2709158660 >> v2:172.21.15.178:3300/0 conn(0x7efd40155550 0x7efd40155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:54.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.373+0000 7efd45400640 1 -- 172.21.15.159:0/2709158660 >> v1:172.21.15.159:6789/0 conn(0x7efd4015f800 legacy=0x7efd400d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:54.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.373+0000 7efd45400640 1 -- 172.21.15.159:0/2709158660 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd400b4950 con 0x7efd400d0990 2024-04-04T06:16:54.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.373+0000 7efd3f400640 1 -- 172.21.15.159:0/2709158660 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4253494728 0 0) 0x7efd400b40b0 con 0x7efd4015f800 2024-04-04T06:16:54.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.374+0000 7efd45400640 1 --2- 172.21.15.159:0/2709158660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd400d0990 0x7efd400aede0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7efd40073ac0 tx=0x7efd3802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beaed9ca9800a677 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:54.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.374+0000 7efd3f400640 1 -- 172.21.15.159:0/2709158660 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3802ca80 con 0x7efd400d0990 2024-04-04T06:16:54.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.374+0000 7efd3f400640 1 -- 172.21.15.159:0/2709158660 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd3802cc20 con 0x7efd400d0990 2024-04-04T06:16:54.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.374+0000 7efd3f400640 1 -- 172.21.15.159:0/2709158660 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd380343f0 con 0x7efd400d0990 2024-04-04T06:16:54.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.375+0000 7efd47400640 1 -- 172.21.15.159:0/2709158660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd400d0990 msgr2=0x7efd400aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:54.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.375+0000 7efd47400640 1 --2- 172.21.15.159:0/2709158660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd400d0990 0x7efd400aede0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7efd40073ac0 tx=0x7efd3802bee0 comp rx=0 tx=0).stop 2024-04-04T06:16:54.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.375+0000 7efd47400640 1 -- 172.21.15.159:0/2709158660 shutdown_connections 2024-04-04T06:16:54.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.375+0000 7efd47400640 1 --2- 172.21.15.159:0/2709158660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd400d0990 0x7efd400aede0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:54.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.375+0000 7efd47400640 1 --2- 172.21.15.159:0/2709158660 >> v2:172.21.15.178:3300/0 conn(0x7efd40155550 0x7efd40155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:54.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.375+0000 7efd47400640 1 -- 172.21.15.159:0/2709158660 >> 172.21.15.159:0/2709158660 conn(0x7efd4006b750 msgr2=0x7efd4006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:54.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.375+0000 7efd47400640 1 -- 172.21.15.159:0/2709158660 shutdown_connections 2024-04-04T06:16:54.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.375+0000 7efd47400640 1 -- 172.21.15.159:0/2709158660 wait complete. 2024-04-04T06:16:54.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.376+0000 7efd47400640 1 Processor -- start 2024-04-04T06:16:54.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.377+0000 7efd47400640 1 -- start start 2024-04-04T06:16:54.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.377+0000 7efd47400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd400d0990 0x7efd401f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:54.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.377+0000 7efd47400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd40155550 0x7efd400ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:54.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.377+0000 7efd47400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd4015f800 0x7efd400ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:54.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.378+0000 7efd47400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efd4015ba20 con 0x7efd400d0990 2024-04-04T06:16:54.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.378+0000 7efd47400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efd4015fc80 con 0x7efd40155550 2024-04-04T06:16:54.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.378+0000 7efd47400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efd4015bba0 con 0x7efd4015f800 2024-04-04T06:16:54.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.378+0000 7efd44a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd400d0990 0x7efd401f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:54.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.378+0000 7efd44a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd400d0990 0x7efd401f5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43922/0 (socket says 172.21.15.159:43922) 2024-04-04T06:16:54.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.378+0000 7efd44a00640 1 -- 172.21.15.159:0/3008952142 learned_addr learned my addr 172.21.15.159:0/3008952142 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:54.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.378+0000 7efd44a00640 1 -- 172.21.15.159:0/3008952142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd4015f800 msgr2=0x7efd400ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:54.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.378+0000 7efd44a00640 1 --2- 172.21.15.159:0/3008952142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd4015f800 0x7efd400ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:54.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.378+0000 7efd44a00640 1 -- 172.21.15.159:0/3008952142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd40155550 msgr2=0x7efd400ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:54.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.378+0000 7efd3fe00640 1 --2- 172.21.15.159:0/3008952142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd40155550 0x7efd400ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:54.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.378+0000 7efd44a00640 1 --2- 172.21.15.159:0/3008952142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd40155550 0x7efd400ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:54.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.378+0000 7efd44a00640 1 -- 172.21.15.159:0/3008952142 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd400af320 con 0x7efd400d0990 2024-04-04T06:16:54.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.378+0000 7efd3fe00640 1 --2- 172.21.15.159:0/3008952142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd40155550 0x7efd400ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:54.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.379+0000 7efd44a00640 1 --2- 172.21.15.159:0/3008952142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd400d0990 0x7efd401f5030 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7efd3400d800 tx=0x7efd3400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:54.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.379+0000 7efd3cc00640 1 -- 172.21.15.159:0/3008952142 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3400b6c0 con 0x7efd400d0990 2024-04-04T06:16:54.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.379+0000 7efd3cc00640 1 -- 172.21.15.159:0/3008952142 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd34019070 con 0x7efd400d0990 2024-04-04T06:16:54.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.379+0000 7efd47400640 1 -- 172.21.15.159:0/3008952142 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd400af600 con 0x7efd400d0990 2024-04-04T06:16:54.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.379+0000 7efd3cc00640 1 -- 172.21.15.159:0/3008952142 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3400ba80 con 0x7efd400d0990 2024-04-04T06:16:54.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.379+0000 7efd47400640 1 -- 172.21.15.159:0/3008952142 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd400afbc0 con 0x7efd400d0990 2024-04-04T06:16:54.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.380+0000 7efd47400640 1 -- 172.21.15.159:0/3008952142 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd400685c0 con 0x7efd400d0990 2024-04-04T06:16:54.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.381+0000 7efd3cc00640 1 -- 172.21.15.159:0/3008952142 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd3400bc20 con 0x7efd400d0990 2024-04-04T06:16:54.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.382+0000 7efd3cc00640 1 --2- 172.21.15.159:0/3008952142 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efd2006c920 0x7efd2006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:54.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.382+0000 7efd3cc00640 1 -- 172.21.15.159:0/3008952142 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4857+0+0 (secure 0 0 0) 0x7efd3408bce0 con 0x7efd400d0990 2024-04-04T06:16:54.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.383+0000 7efd3fe00640 1 --2- 172.21.15.159:0/3008952142 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efd2006c920 0x7efd2006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:54.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.384+0000 7efd3fe00640 1 --2- 172.21.15.159:0/3008952142 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efd2006c920 0x7efd2006ede0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7efd2c007fd0 tx=0x7efd2c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:54.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.387+0000 7efd3cc00640 1 -- 172.21.15.159:0/3008952142 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7efd3401c360 con 0x7efd400d0990 2024-04-04T06:16:54.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.571+0000 7efd47400640 1 -- 172.21.15.159:0/3008952142 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7efd400b3ad0 con 0x7efd400d0990 2024-04-04T06:16:54.567 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.572+0000 7efd3cc00640 1 -- 172.21.15.159:0/3008952142 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v109) ==== 84+0+65 (secure 0 0 0) 0x7efd3405a3c0 con 0x7efd400d0990 2024-04-04T06:16:54.568 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:54.568 INFO:teuthology.orchestra.run.smithi159.stdout:[ 2024-04-04T06:16:54.568 INFO:teuthology.orchestra.run.smithi159.stdout: { 2024-04-04T06:16:54.568 INFO:teuthology.orchestra.run.smithi159.stdout: "poolnum": 1, 2024-04-04T06:16:54.568 INFO:teuthology.orchestra.run.smithi159.stdout: "poolname": ".mgr" 2024-04-04T06:16:54.568 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-04-04T06:16:54.568 INFO:teuthology.orchestra.run.smithi159.stdout:] 2024-04-04T06:16:54.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.575+0000 7efd47400640 1 -- 172.21.15.159:0/3008952142 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efd2006c920 msgr2=0x7efd2006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:54.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.575+0000 7efd47400640 1 --2- 172.21.15.159:0/3008952142 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efd2006c920 0x7efd2006ede0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7efd2c007fd0 tx=0x7efd2c009040 comp rx=0 tx=0).stop 2024-04-04T06:16:54.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.575+0000 7efd47400640 1 -- 172.21.15.159:0/3008952142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd400d0990 msgr2=0x7efd401f5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:54.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.575+0000 7efd47400640 1 --2- 172.21.15.159:0/3008952142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd400d0990 0x7efd401f5030 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7efd3400d800 tx=0x7efd3400dcc0 comp rx=0 tx=0).stop 2024-04-04T06:16:54.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.575+0000 7efd47400640 1 -- 172.21.15.159:0/3008952142 shutdown_connections 2024-04-04T06:16:54.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.575+0000 7efd47400640 1 --2- 172.21.15.159:0/3008952142 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd4015f800 0x7efd400ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:54.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.575+0000 7efd47400640 1 --2- 172.21.15.159:0/3008952142 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efd2006c920 0x7efd2006ede0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:54.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.575+0000 7efd47400640 1 --2- 172.21.15.159:0/3008952142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd40155550 0x7efd400ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:54.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.575+0000 7efd47400640 1 --2- 172.21.15.159:0/3008952142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd400d0990 0x7efd401f5030 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:54.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.575+0000 7efd47400640 1 -- 172.21.15.159:0/3008952142 >> 172.21.15.159:0/3008952142 conn(0x7efd4006b750 msgr2=0x7efd4006d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:54.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.575+0000 7efd47400640 1 -- 172.21.15.159:0/3008952142 shutdown_connections 2024-04-04T06:16:54.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.575+0000 7efd47400640 1 -- 172.21.15.159:0/3008952142 wait complete. 2024-04-04T06:16:54.587 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-04T06:16:54.588 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-04T06:16:54.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.802+0000 7ff497800640 1 Processor -- start 2024-04-04T06:16:54.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.802+0000 7ff497800640 1 -- start start 2024-04-04T06:16:54.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.804+0000 7ff497800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff490072720 0x7ff4900b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:54.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.804+0000 7ff497800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff490072b00 0x7ff49006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:54.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.804+0000 7ff497800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff49006ea10 con 0x7ff49006db10 2024-04-04T06:16:54.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.804+0000 7ff497800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7ff4900b3e80 con 0x7ff490072b00 2024-04-04T06:16:54.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.804+0000 7ff497800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff49006ecf0 con 0x7ff490072720 2024-04-04T06:16:54.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.804+0000 7ff48fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff490072b00 0x7ff49006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:54.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.804+0000 7ff48fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff490072b00 0x7ff49006d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43936/0 (socket says 172.21.15.159:43936) 2024-04-04T06:16:54.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.804+0000 7ff48fe00640 1 -- 172.21.15.159:0/1334171971 learned_addr learned my addr 172.21.15.159:0/1334171971 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:54.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.804+0000 7ff494e00640 1 --2- 172.21.15.159:0/1334171971 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff490072720 0x7ff4900b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:54.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.805+0000 7ff48fe00640 1 -- 172.21.15.159:0/1334171971 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff490072720 msgr2=0x7ff4900b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:54.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.805+0000 7ff48fe00640 1 --2- 172.21.15.159:0/1334171971 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff490072720 0x7ff4900b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:54.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.805+0000 7ff48fe00640 1 -- 172.21.15.159:0/1334171971 >> v1:172.21.15.178:6789/0 conn(0x7ff49006db10 legacy=0x7ff4900adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:54.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.805+0000 7ff48fe00640 1 -- 172.21.15.159:0/1334171971 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49006efd0 con 0x7ff490072b00 2024-04-04T06:16:54.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.806+0000 7ff48fe00640 1 --2- 172.21.15.159:0/1334171971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff490072b00 0x7ff49006d5d0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009870 tx=0x7ff47c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b847949fe6565d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:54.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.806+0000 7ff48f400640 1 -- 172.21.15.159:0/1334171971 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c034070 con 0x7ff490072b00 2024-04-04T06:16:54.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.807+0000 7ff48f400640 1 -- 172.21.15.159:0/1334171971 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff47c030840 con 0x7ff490072b00 2024-04-04T06:16:54.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.807+0000 7ff48f400640 1 -- 172.21.15.159:0/1334171971 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c030c10 con 0x7ff490072b00 2024-04-04T06:16:54.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.807+0000 7ff497800640 1 -- 172.21.15.159:0/1334171971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff490072b00 msgr2=0x7ff49006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:54.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.807+0000 7ff497800640 1 --2- 172.21.15.159:0/1334171971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff490072b00 0x7ff49006d5d0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009870 tx=0x7ff47c02b980 comp rx=0 tx=0).stop 2024-04-04T06:16:54.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.807+0000 7ff497800640 1 -- 172.21.15.159:0/1334171971 shutdown_connections 2024-04-04T06:16:54.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.807+0000 7ff497800640 1 --2- 172.21.15.159:0/1334171971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff490072b00 0x7ff49006d5d0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:54.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.807+0000 7ff497800640 1 --2- 172.21.15.159:0/1334171971 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff490072720 0x7ff4900b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:54.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.807+0000 7ff497800640 1 -- 172.21.15.159:0/1334171971 >> 172.21.15.159:0/1334171971 conn(0x7ff49006bba0 msgr2=0x7ff4900b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:54.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.808+0000 7ff497800640 1 -- 172.21.15.159:0/1334171971 shutdown_connections 2024-04-04T06:16:54.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.808+0000 7ff497800640 1 -- 172.21.15.159:0/1334171971 wait complete. 2024-04-04T06:16:54.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.809+0000 7ff497800640 1 Processor -- start 2024-04-04T06:16:54.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.809+0000 7ff497800640 1 -- start start 2024-04-04T06:16:54.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.810+0000 7ff497800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff49006db10 0x7ff4900ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:54.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.810+0000 7ff497800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff490072720 0x7ff49014dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:54.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.810+0000 7ff497800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff490072b00 0x7ff49014e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:54.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.810+0000 7ff497800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff49006e690 con 0x7ff490072720 2024-04-04T06:16:54.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.810+0000 7ff497800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff49006f590 con 0x7ff490072b00 2024-04-04T06:16:54.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.810+0000 7ff497800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff49015eab0 con 0x7ff49006db10 2024-04-04T06:16:54.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.810+0000 7ff48fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff490072720 0x7ff49014dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:54.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.811+0000 7ff48fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff490072720 0x7ff49014dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43950/0 (socket says 172.21.15.159:43950) 2024-04-04T06:16:54.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.811+0000 7ff48fe00640 1 -- 172.21.15.159:0/3172704361 learned_addr learned my addr 172.21.15.159:0/3172704361 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:54.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.811+0000 7ff494e00640 1 --2- 172.21.15.159:0/3172704361 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff49006db10 0x7ff4900ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:54.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.811+0000 7ff495800640 1 --2- 172.21.15.159:0/3172704361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff490072b00 0x7ff49014e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:54.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.811+0000 7ff48fe00640 1 -- 172.21.15.159:0/3172704361 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff49006db10 msgr2=0x7ff4900ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:54.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.811+0000 7ff48fe00640 1 --2- 172.21.15.159:0/3172704361 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff49006db10 0x7ff4900ce530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:54.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.811+0000 7ff48fe00640 1 -- 172.21.15.159:0/3172704361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff490072b00 msgr2=0x7ff49014e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:54.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.811+0000 7ff48fe00640 1 --2- 172.21.15.159:0/3172704361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff490072b00 0x7ff49014e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:54.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.811+0000 7ff48fe00640 1 -- 172.21.15.159:0/3172704361 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4900cea70 con 0x7ff490072720 2024-04-04T06:16:54.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.811+0000 7ff48fe00640 1 --2- 172.21.15.159:0/3172704361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff490072720 0x7ff49014dbe0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0090b0 tx=0x7ff47c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:54.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.811+0000 7ff48cc00640 1 -- 172.21.15.159:0/3172704361 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c034040 con 0x7ff490072720 2024-04-04T06:16:54.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.812+0000 7ff48cc00640 1 -- 172.21.15.159:0/3172704361 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff47c030640 con 0x7ff490072720 2024-04-04T06:16:54.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.812+0000 7ff48cc00640 1 -- 172.21.15.159:0/3172704361 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c0423f0 con 0x7ff490072720 2024-04-04T06:16:54.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.812+0000 7ff497800640 1 -- 172.21.15.159:0/3172704361 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49014eb60 con 0x7ff490072720 2024-04-04T06:16:54.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.812+0000 7ff497800640 1 -- 172.21.15.159:0/3172704361 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff490157250 con 0x7ff490072720 2024-04-04T06:16:54.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.813+0000 7ff497800640 1 -- 172.21.15.159:0/3172704361 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff49006df60 con 0x7ff490072720 2024-04-04T06:16:54.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.814+0000 7ff48cc00640 1 -- 172.21.15.159:0/3172704361 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff47c037070 con 0x7ff490072720 2024-04-04T06:16:54.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.814+0000 7ff48cc00640 1 --2- 172.21.15.159:0/3172704361 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff47006c8f0 0x7ff47006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:54.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.815+0000 7ff48cc00640 1 -- 172.21.15.159:0/3172704361 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4857+0+0 (secure 0 0 0) 0x7ff47c0adc10 con 0x7ff490072720 2024-04-04T06:16:54.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.815+0000 7ff494e00640 1 --2- 172.21.15.159:0/3172704361 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff47006c8f0 0x7ff47006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:54.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.816+0000 7ff494e00640 1 --2- 172.21.15.159:0/3172704361 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff47006c8f0 0x7ff47006edb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff484006fd0 tx=0x7ff484008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:54.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:54.819+0000 7ff48cc00640 1 -- 172.21.15.159:0/3172704361 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff49006df60 con 0x7ff490072720 2024-04-04T06:16:55.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:55.019+0000 7ff497800640 1 -- 172.21.15.159:0/3172704361 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff4900685c0 con 0x7ff490072720 2024-04-04T06:16:55.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:55.778+0000 7ff48cc00640 1 -- 172.21.15.159:0/3172704361 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v110) ==== 138+0+0 (secure 0 0 0) 0x7ff47c07c410 con 0x7ff490072720 2024-04-04T06:16:55.773 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_metadata' created 2024-04-04T06:16:55.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:55.782+0000 7ff497800640 1 -- 172.21.15.159:0/3172704361 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff47006c8f0 msgr2=0x7ff47006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:55.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:55.782+0000 7ff497800640 1 --2- 172.21.15.159:0/3172704361 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff47006c8f0 0x7ff47006edb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff484006fd0 tx=0x7ff484008040 comp rx=0 tx=0).stop 2024-04-04T06:16:55.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:55.782+0000 7ff497800640 1 -- 172.21.15.159:0/3172704361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff490072720 msgr2=0x7ff49014dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:55.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:55.782+0000 7ff497800640 1 --2- 172.21.15.159:0/3172704361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff490072720 0x7ff49014dbe0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0090b0 tx=0x7ff47c0046c0 comp rx=0 tx=0).stop 2024-04-04T06:16:55.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:55.782+0000 7ff497800640 1 -- 172.21.15.159:0/3172704361 shutdown_connections 2024-04-04T06:16:55.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:55.782+0000 7ff497800640 1 --2- 172.21.15.159:0/3172704361 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff47006c8f0 0x7ff47006edb0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:55.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:55.782+0000 7ff497800640 1 --2- 172.21.15.159:0/3172704361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff490072b00 0x7ff49014e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:55.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:55.782+0000 7ff497800640 1 --2- 172.21.15.159:0/3172704361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff490072720 0x7ff49014dbe0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:55.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:55.782+0000 7ff497800640 1 --2- 172.21.15.159:0/3172704361 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff49006db10 0x7ff4900ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:55.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:55.782+0000 7ff497800640 1 -- 172.21.15.159:0/3172704361 >> 172.21.15.159:0/3172704361 conn(0x7ff49006bba0 msgr2=0x7ff49015d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:55.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:55.782+0000 7ff497800640 1 -- 172.21.15.159:0/3172704361 shutdown_connections 2024-04-04T06:16:55.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:55.783+0000 7ff497800640 1 -- 172.21.15.159:0/3172704361 wait complete. 2024-04-04T06:16:55.802 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-04T06:16:55.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.004+0000 7fb01c400640 1 Processor -- start 2024-04-04T06:16:55.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.004+0000 7fb01c400640 1 -- start start 2024-04-04T06:16:56.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.005+0000 7fb01c400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb014158ea0 0x7fb0140b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:56.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.005+0000 7fb01c400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0140b3f90 0x7fb01406c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:56.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.005+0000 7fb01c400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb01406cac0 0x7fb01415ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:56.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.005+0000 7fb01c400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fb01415baa0 con 0x7fb01406cac0 2024-04-04T06:16:56.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.005+0000 7fb01c400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb01415bd80 con 0x7fb014158ea0 2024-04-04T06:16:56.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.005+0000 7fb01c400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb01415c060 con 0x7fb0140b3f90 2024-04-04T06:16:56.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.006+0000 7fb01a400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb01406cac0 0x7fb01415ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:56.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.006+0000 7fb01a400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb01406cac0 0x7fb01415ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43962/0 (socket says 172.21.15.159:43962) 2024-04-04T06:16:56.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.006+0000 7fb01a400640 1 -- 172.21.15.159:0/3108947757 learned_addr learned my addr 172.21.15.159:0/3108947757 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:56.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.006+0000 7fb019a00640 1 --2- 172.21.15.159:0/3108947757 >> v2:172.21.15.178:3300/0 conn(0x7fb014158ea0 0x7fb0140b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:56.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.006+0000 7fb019000640 1 --2- 172.21.15.159:0/3108947757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0140b3f90 0x7fb01406c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:56.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.006+0000 7fb01a400640 1 -- 172.21.15.159:0/3108947757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0140b3f90 msgr2=0x7fb01406c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:56.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.006+0000 7fb01a400640 1 --2- 172.21.15.159:0/3108947757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0140b3f90 0x7fb01406c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:56.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.006+0000 7fb01a400640 1 -- 172.21.15.159:0/3108947757 >> v2:172.21.15.178:3300/0 conn(0x7fb014158ea0 msgr2=0x7fb0140b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:56.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.006+0000 7fb01a400640 1 --2- 172.21.15.159:0/3108947757 >> v2:172.21.15.178:3300/0 conn(0x7fb014158ea0 0x7fb0140b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:56.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.007+0000 7fb01a400640 1 -- 172.21.15.159:0/3108947757 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01415c340 con 0x7fb01406cac0 2024-04-04T06:16:56.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.007+0000 7fb019000640 1 --2- 172.21.15.159:0/3108947757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0140b3f90 0x7fb01406c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:16:56.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.008+0000 7fb01a400640 1 --2- 172.21.15.159:0/3108947757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb01406cac0 0x7fb01415ef80 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fb01415bf90 tx=0x7fb00402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aedfd37d95ef1b58 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:56.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.008+0000 7fb013e00640 1 -- 172.21.15.159:0/3108947757 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb004035020 con 0x7fb01406cac0 2024-04-04T06:16:56.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.008+0000 7fb013e00640 1 -- 172.21.15.159:0/3108947757 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb004031640 con 0x7fb01406cac0 2024-04-04T06:16:56.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.008+0000 7fb013e00640 1 -- 172.21.15.159:0/3108947757 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00402d040 con 0x7fb01406cac0 2024-04-04T06:16:56.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.008+0000 7fb01c400640 1 -- 172.21.15.159:0/3108947757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb01406cac0 msgr2=0x7fb01415ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:56.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.008+0000 7fb01c400640 1 --2- 172.21.15.159:0/3108947757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb01406cac0 0x7fb01415ef80 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fb01415bf90 tx=0x7fb00402b8f0 comp rx=0 tx=0).stop 2024-04-04T06:16:56.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.009+0000 7fb01c400640 1 -- 172.21.15.159:0/3108947757 shutdown_connections 2024-04-04T06:16:56.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.009+0000 7fb01c400640 1 --2- 172.21.15.159:0/3108947757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb01406cac0 0x7fb01415ef80 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:56.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.009+0000 7fb01c400640 1 --2- 172.21.15.159:0/3108947757 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0140b3f90 0x7fb01406c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:56.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.009+0000 7fb01c400640 1 --2- 172.21.15.159:0/3108947757 >> v2:172.21.15.178:3300/0 conn(0x7fb014158ea0 0x7fb0140b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:56.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.009+0000 7fb01c400640 1 -- 172.21.15.159:0/3108947757 >> 172.21.15.159:0/3108947757 conn(0x7fb014071e20 msgr2=0x7fb014072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:56.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.009+0000 7fb01c400640 1 -- 172.21.15.159:0/3108947757 shutdown_connections 2024-04-04T06:16:56.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.009+0000 7fb01c400640 1 -- 172.21.15.159:0/3108947757 wait complete. 2024-04-04T06:16:56.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.010+0000 7fb01c400640 1 Processor -- start 2024-04-04T06:16:56.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.010+0000 7fb01c400640 1 -- start start 2024-04-04T06:16:56.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.011+0000 7fb01c400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb01406cac0 0x7fb01414dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:56.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.011+0000 7fb01c400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0140b3f90 0x7fb0141644a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:56.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.011+0000 7fb01c400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb014158ea0 0x7fb0141649e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:56.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.011+0000 7fb01c400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb0140b0320 con 0x7fb0140b3f90 2024-04-04T06:16:56.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.011+0000 7fb01c400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb014159280 con 0x7fb01406cac0 2024-04-04T06:16:56.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.011+0000 7fb01c400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb0140b04a0 con 0x7fb014158ea0 2024-04-04T06:16:56.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.012+0000 7fb019000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0140b3f90 0x7fb0141644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:56.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.012+0000 7fb019000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0140b3f90 0x7fb0141644a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43968/0 (socket says 172.21.15.159:43968) 2024-04-04T06:16:56.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.012+0000 7fb019000640 1 -- 172.21.15.159:0/2275167984 learned_addr learned my addr 172.21.15.159:0/2275167984 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:56.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.012+0000 7fb019000640 1 -- 172.21.15.159:0/2275167984 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb014158ea0 msgr2=0x7fb0141649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:56.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.012+0000 7fb019000640 1 --2- 172.21.15.159:0/2275167984 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb014158ea0 0x7fb0141649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:56.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.012+0000 7fb019000640 1 -- 172.21.15.159:0/2275167984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb01406cac0 msgr2=0x7fb01414dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:56.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.012+0000 7fb019000640 1 --2- 172.21.15.159:0/2275167984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb01406cac0 0x7fb01414dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:56.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.012+0000 7fb019000640 1 -- 172.21.15.159:0/2275167984 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb014166210 con 0x7fb0140b3f90 2024-04-04T06:16:56.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.012+0000 7fb019000640 1 --2- 172.21.15.159:0/2275167984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0140b3f90 0x7fb0141644a0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fb00000e840 tx=0x7fb00000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:56.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.013+0000 7fb011600640 1 -- 172.21.15.159:0/2275167984 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00000f040 con 0x7fb0140b3f90 2024-04-04T06:16:56.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.013+0000 7fb011600640 1 -- 172.21.15.159:0/2275167984 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb000004030 con 0x7fb0140b3f90 2024-04-04T06:16:56.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.013+0000 7fb01c400640 1 -- 172.21.15.159:0/2275167984 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0141664f0 con 0x7fb0140b3f90 2024-04-04T06:16:56.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.013+0000 7fb011600640 1 -- 172.21.15.159:0/2275167984 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000004400 con 0x7fb0140b3f90 2024-04-04T06:16:56.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.013+0000 7fb01c400640 1 -- 172.21.15.159:0/2275167984 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb01414e220 con 0x7fb0140b3f90 2024-04-04T06:16:56.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.014+0000 7fb01c400640 1 -- 172.21.15.159:0/2275167984 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb014072fb0 con 0x7fb0140b3f90 2024-04-04T06:16:56.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.015+0000 7fb011600640 1 -- 172.21.15.159:0/2275167984 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb00000c6d0 con 0x7fb0140b3f90 2024-04-04T06:16:56.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.016+0000 7fb011600640 1 --2- 172.21.15.159:0/2275167984 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faff406c8f0 0x7faff406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:56.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.016+0000 7fb011600640 1 -- 172.21.15.159:0/2275167984 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5227+0+0 (secure 0 0 0) 0x7fb00008b680 con 0x7fb0140b3f90 2024-04-04T06:16:56.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.016+0000 7fb019a00640 1 --2- 172.21.15.159:0/2275167984 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faff406c8f0 0x7faff406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:56.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.018+0000 7fb019a00640 1 --2- 172.21.15.159:0/2275167984 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faff406c8f0 0x7faff406edb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009fa0 tx=0x7fb00c00b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:56.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.021+0000 7fb011600640 1 -- 172.21.15.159:0/2275167984 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb014072fb0 con 0x7fb0140b3f90 2024-04-04T06:16:56.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.225+0000 7fb01c400640 1 -- 172.21.15.159:0/2275167984 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb014068cd0 con 0x7fb0140b3f90 2024-04-04T06:16:56.780 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.785+0000 7fb011600640 1 -- 172.21.15.159:0/2275167984 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v111) ==== 170+0+0 (secure 0 0 0) 0x7fb000017020 con 0x7fb0140b3f90 2024-04-04T06:16:56.780 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_data' created 2024-04-04T06:16:56.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.789+0000 7fb01c400640 1 -- 172.21.15.159:0/2275167984 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faff406c8f0 msgr2=0x7faff406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:56.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.789+0000 7fb01c400640 1 --2- 172.21.15.159:0/2275167984 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faff406c8f0 0x7faff406edb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009fa0 tx=0x7fb00c00b040 comp rx=0 tx=0).stop 2024-04-04T06:16:56.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.789+0000 7fb01c400640 1 -- 172.21.15.159:0/2275167984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0140b3f90 msgr2=0x7fb0141644a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:56.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.789+0000 7fb01c400640 1 --2- 172.21.15.159:0/2275167984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0140b3f90 0x7fb0141644a0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fb00000e840 tx=0x7fb00000ed00 comp rx=0 tx=0).stop 2024-04-04T06:16:56.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.790+0000 7fb01c400640 1 -- 172.21.15.159:0/2275167984 shutdown_connections 2024-04-04T06:16:56.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.790+0000 7fb01c400640 1 --2- 172.21.15.159:0/2275167984 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb014158ea0 0x7fb0141649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:56.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.790+0000 7fb01c400640 1 --2- 172.21.15.159:0/2275167984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0140b3f90 0x7fb0141644a0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:56.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.790+0000 7fb01c400640 1 --2- 172.21.15.159:0/2275167984 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7faff406c8f0 0x7faff406edb0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:56.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.790+0000 7fb01c400640 1 --2- 172.21.15.159:0/2275167984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb01406cac0 0x7fb01414dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:56.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.790+0000 7fb01c400640 1 -- 172.21.15.159:0/2275167984 >> 172.21.15.159:0/2275167984 conn(0x7fb014071e20 msgr2=0x7fb0140cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:56.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.790+0000 7fb01c400640 1 -- 172.21.15.159:0/2275167984 shutdown_connections 2024-04-04T06:16:56.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:56.790+0000 7fb01c400640 1 -- 172.21.15.159:0/2275167984 wait complete. 2024-04-04T06:16:56.824 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-04T06:16:57.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.017+0000 7fe45dc00640 1 Processor -- start 2024-04-04T06:16:57.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.017+0000 7fe45dc00640 1 -- start start 2024-04-04T06:16:57.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.019+0000 7fe45dc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe458141c10 0x7fe458071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:57.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.019+0000 7fe45dc00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe458071eb0 0x7fe458162160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:57.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.019+0000 7fe45dc00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe45815ec80 con 0x7fe4580d0280 2024-04-04T06:16:57.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.019+0000 7fe45dc00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fe45815ef20 con 0x7fe458141c10 2024-04-04T06:16:57.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.019+0000 7fe45dc00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fe45815f200 con 0x7fe458071eb0 2024-04-04T06:16:57.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.019+0000 7fe456000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe458141c10 0x7fe458071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:57.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.019+0000 7fe456000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fe458141c10 0x7fe458071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43974/0 (socket says 172.21.15.159:43974) 2024-04-04T06:16:57.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.019+0000 7fe456000640 1 -- 172.21.15.159:0/3044205927 learned_addr learned my addr 172.21.15.159:0/3044205927 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:57.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.019+0000 7fe457400640 1 --2- 172.21.15.159:0/3044205927 >> v2:172.21.15.178:3300/0 conn(0x7fe458071eb0 0x7fe458162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:57.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.020+0000 7fe456000640 1 -- 172.21.15.159:0/3044205927 >> v2:172.21.15.178:3300/0 conn(0x7fe458071eb0 msgr2=0x7fe458162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:57.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.020+0000 7fe456000640 1 --2- 172.21.15.159:0/3044205927 >> v2:172.21.15.178:3300/0 conn(0x7fe458071eb0 0x7fe458162160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:57.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.020+0000 7fe456000640 1 -- 172.21.15.159:0/3044205927 >> v1:172.21.15.178:6789/0 conn(0x7fe4580d0280 legacy=0x7fe4580d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:57.015 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.020+0000 7fe456000640 1 -- 172.21.15.159:0/3044205927 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45815f4e0 con 0x7fe458141c10 2024-04-04T06:16:57.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.021+0000 7fe456000640 1 --2- 172.21.15.159:0/3044205927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe458141c10 0x7fe458071970 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fe4580cfac0 tx=0x7fe44002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7770f4205394f5a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:57.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.021+0000 7fe455600640 1 -- 172.21.15.159:0/3044205927 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440035020 con 0x7fe458141c10 2024-04-04T06:16:57.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.021+0000 7fe455600640 1 -- 172.21.15.159:0/3044205927 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe440031870 con 0x7fe458141c10 2024-04-04T06:16:57.016 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.021+0000 7fe455600640 1 -- 172.21.15.159:0/3044205927 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440031c40 con 0x7fe458141c10 2024-04-04T06:16:57.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.022+0000 7fe45dc00640 1 -- 172.21.15.159:0/3044205927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe458141c10 msgr2=0x7fe458071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:57.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.022+0000 7fe45dc00640 1 --2- 172.21.15.159:0/3044205927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe458141c10 0x7fe458071970 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fe4580cfac0 tx=0x7fe44002b910 comp rx=0 tx=0).stop 2024-04-04T06:16:57.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.022+0000 7fe45dc00640 1 -- 172.21.15.159:0/3044205927 shutdown_connections 2024-04-04T06:16:57.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.022+0000 7fe45dc00640 1 --2- 172.21.15.159:0/3044205927 >> v2:172.21.15.178:3300/0 conn(0x7fe458071eb0 0x7fe458162160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:57.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.022+0000 7fe45dc00640 1 --2- 172.21.15.159:0/3044205927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe458141c10 0x7fe458071970 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:57.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.022+0000 7fe45dc00640 1 -- 172.21.15.159:0/3044205927 >> 172.21.15.159:0/3044205927 conn(0x7fe45806b750 msgr2=0x7fe45806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:57.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.022+0000 7fe45dc00640 1 -- 172.21.15.159:0/3044205927 shutdown_connections 2024-04-04T06:16:57.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.022+0000 7fe45dc00640 1 -- 172.21.15.159:0/3044205927 wait complete. 2024-04-04T06:16:57.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.024+0000 7fe45dc00640 1 Processor -- start 2024-04-04T06:16:57.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.024+0000 7fe45dc00640 1 -- start start 2024-04-04T06:16:57.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.025+0000 7fe45dc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe458071eb0 0x7fe45809d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:57.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.025+0000 7fe45dc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4580d0280 0x7fe4581f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:57.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.025+0000 7fe45dc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe458141c10 0x7fe4581e6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:57.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.025+0000 7fe45dc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe458158e90 con 0x7fe458141c10 2024-04-04T06:16:57.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.025+0000 7fe45dc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe45806f340 con 0x7fe458071eb0 2024-04-04T06:16:57.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.025+0000 7fe45dc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe4581f4820 con 0x7fe4580d0280 2024-04-04T06:16:57.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.025+0000 7fe457400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe458141c10 0x7fe4581e6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:57.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.025+0000 7fe457400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe458141c10 0x7fe4581e6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:43984/0 (socket says 172.21.15.159:43984) 2024-04-04T06:16:57.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.025+0000 7fe457400640 1 -- 172.21.15.159:0/1547026376 learned_addr learned my addr 172.21.15.159:0/1547026376 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:57.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.025+0000 7fe456000640 1 --2- 172.21.15.159:0/1547026376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4580d0280 0x7fe4581f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:57.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.025+0000 7fe456a00640 1 --2- 172.21.15.159:0/1547026376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe458071eb0 0x7fe45809d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:57.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.025+0000 7fe457400640 1 -- 172.21.15.159:0/1547026376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4580d0280 msgr2=0x7fe4581f42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:57.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.025+0000 7fe457400640 1 --2- 172.21.15.159:0/1547026376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4580d0280 0x7fe4581f42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:57.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.026+0000 7fe457400640 1 -- 172.21.15.159:0/1547026376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe458071eb0 msgr2=0x7fe45809d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:57.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.026+0000 7fe457400640 1 --2- 172.21.15.159:0/1547026376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe458071eb0 0x7fe45809d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:57.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.026+0000 7fe457400640 1 -- 172.21.15.159:0/1547026376 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45809c3a0 con 0x7fe458141c10 2024-04-04T06:16:57.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.026+0000 7fe456a00640 1 --2- 172.21.15.159:0/1547026376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe458071eb0 0x7fe45809d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:57.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.026+0000 7fe457400640 1 --2- 172.21.15.159:0/1547026376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe458141c10 0x7fe4581e6d70 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fe450007e40 tx=0x7fe45000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:57.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.026+0000 7fe44ea00640 1 -- 172.21.15.159:0/1547026376 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe450014620 con 0x7fe458141c10 2024-04-04T06:16:57.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.026+0000 7fe44ea00640 1 -- 172.21.15.159:0/1547026376 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe450019070 con 0x7fe458141c10 2024-04-04T06:16:57.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.026+0000 7fe44ea00640 1 -- 172.21.15.159:0/1547026376 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe450014a20 con 0x7fe458141c10 2024-04-04T06:16:57.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.026+0000 7fe45dc00640 1 -- 172.21.15.159:0/1547026376 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe45809c680 con 0x7fe458141c10 2024-04-04T06:16:57.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.026+0000 7fe45dc00640 1 -- 172.21.15.159:0/1547026376 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe45809cc40 con 0x7fe458141c10 2024-04-04T06:16:57.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.027+0000 7fe45dc00640 1 -- 172.21.15.159:0/1547026376 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4580685c0 con 0x7fe458141c10 2024-04-04T06:16:57.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.029+0000 7fe44ea00640 1 -- 172.21.15.159:0/1547026376 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe450014bc0 con 0x7fe458141c10 2024-04-04T06:16:57.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.030+0000 7fe44ea00640 1 --2- 172.21.15.159:0/1547026376 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe43406c8f0 0x7fe43406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:57.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.030+0000 7fe44ea00640 1 -- 172.21.15.159:0/1547026376 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5609+0+0 (secure 0 0 0) 0x7fe45008d2f0 con 0x7fe458141c10 2024-04-04T06:16:57.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.030+0000 7fe456a00640 1 --2- 172.21.15.159:0/1547026376 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe43406c8f0 0x7fe43406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:57.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.031+0000 7fe456a00640 1 --2- 172.21.15.159:0/1547026376 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe43406c8f0 0x7fe43406edb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe448003eb0 tx=0x7fe448009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:57.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.034+0000 7fe44ea00640 1 -- 172.21.15.159:0/1547026376 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fe45005b7f0 con 0x7fe458141c10 2024-04-04T06:16:57.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.274+0000 7fe45dc00640 1 -- 172.21.15.159:0/1547026376 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe4580cfdb0 con 0x7fe458141c10 2024-04-04T06:16:57.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.841+0000 7fe44ea00640 1 -- 172.21.15.159:0/1547026376 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v97) ==== 174+0+0 (secure 0 0 0) 0x7fe45005b0f0 con 0x7fe458141c10 2024-04-04T06:16:57.844 INFO:teuthology.orchestra.run.smithi159.stderr:new fs with metadata pool 16 and data pool 17 2024-04-04T06:16:57.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.854+0000 7fe45dc00640 1 -- 172.21.15.159:0/1547026376 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe43406c8f0 msgr2=0x7fe43406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:57.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.854+0000 7fe45dc00640 1 --2- 172.21.15.159:0/1547026376 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe43406c8f0 0x7fe43406edb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe448003eb0 tx=0x7fe448009040 comp rx=0 tx=0).stop 2024-04-04T06:16:57.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.854+0000 7fe45dc00640 1 -- 172.21.15.159:0/1547026376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe458141c10 msgr2=0x7fe4581e6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:57.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.854+0000 7fe45dc00640 1 --2- 172.21.15.159:0/1547026376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe458141c10 0x7fe4581e6d70 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fe450007e40 tx=0x7fe45000e4b0 comp rx=0 tx=0).stop 2024-04-04T06:16:57.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.855+0000 7fe45dc00640 1 -- 172.21.15.159:0/1547026376 shutdown_connections 2024-04-04T06:16:57.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.855+0000 7fe45dc00640 1 --2- 172.21.15.159:0/1547026376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe458141c10 0x7fe4581e6d70 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:57.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.855+0000 7fe45dc00640 1 --2- 172.21.15.159:0/1547026376 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe4580d0280 0x7fe4581f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:57.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.855+0000 7fe45dc00640 1 --2- 172.21.15.159:0/1547026376 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe43406c8f0 0x7fe43406edb0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:57.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.855+0000 7fe45dc00640 1 --2- 172.21.15.159:0/1547026376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe458071eb0 0x7fe45809d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:57.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.855+0000 7fe45dc00640 1 -- 172.21.15.159:0/1547026376 >> 172.21.15.159:0/1547026376 conn(0x7fe45806b750 msgr2=0x7fe4580afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:57.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.856+0000 7fe45dc00640 1 -- 172.21.15.159:0/1547026376 shutdown_connections 2024-04-04T06:16:57.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:57.856+0000 7fe45dc00640 1 -- 172.21.15.159:0/1547026376 wait complete. 2024-04-04T06:16:57.885 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:16:58.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.069+0000 7efe82a00640 1 Processor -- start 2024-04-04T06:16:58.064 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.069+0000 7efe82a00640 1 -- start start 2024-04-04T06:16:58.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.070+0000 7efe82a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7efe7c15c390 0x7efe7c073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:58.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.070+0000 7efe82a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe7c15fe80 con 0x7efe7c0d0650 2024-04-04T06:16:58.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.070+0000 7efe82a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe7c0ad120 con 0x7efe7c0af830 2024-04-04T06:16:58.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.070+0000 7efe82a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7efe7c15c770 con 0x7efe7c15c390 2024-04-04T06:16:58.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.071+0000 7efe7be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7efe7c15c390 0x7efe7c073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:58.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.071+0000 7efe7be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7efe7c15c390 0x7efe7c073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44000/0 (socket says 172.21.15.159:44000) 2024-04-04T06:16:58.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.071+0000 7efe7be00640 1 -- 172.21.15.159:0/2147953644 learned_addr learned my addr 172.21.15.159:0/2147953644 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:58.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.071+0000 7efe7be00640 1 -- 172.21.15.159:0/2147953644 >> v1:172.21.15.178:6789/0 conn(0x7efe7c0af830 legacy=0x7efe7c160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:58.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.071+0000 7efe7be00640 1 -- 172.21.15.159:0/2147953644 >> v1:172.21.15.159:6789/0 conn(0x7efe7c0d0650 legacy=0x7efe7c071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:58.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.071+0000 7efe7be00640 1 -- 172.21.15.159:0/2147953644 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7c0ad400 con 0x7efe7c15c390 2024-04-04T06:16:58.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.072+0000 7efe7be00640 1 --2- 172.21.15.159:0/2147953644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe7c15c390 0x7efe7c073790 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009870 tx=0x7efe6c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d920e9e89b51e5d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:58.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.073+0000 7efe7aa00640 1 -- 172.21.15.159:0/2147953644 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c034070 con 0x7efe7c15c390 2024-04-04T06:16:58.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.073+0000 7efe7aa00640 1 -- 172.21.15.159:0/2147953644 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe6c030860 con 0x7efe7c15c390 2024-04-04T06:16:58.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.073+0000 7efe7aa00640 1 -- 172.21.15.159:0/2147953644 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c030c30 con 0x7efe7c15c390 2024-04-04T06:16:58.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.073+0000 7efe82a00640 1 -- 172.21.15.159:0/2147953644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe7c15c390 msgr2=0x7efe7c073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:58.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.073+0000 7efe82a00640 1 --2- 172.21.15.159:0/2147953644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe7c15c390 0x7efe7c073790 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009870 tx=0x7efe6c02b9a0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.073+0000 7efe82a00640 1 -- 172.21.15.159:0/2147953644 shutdown_connections 2024-04-04T06:16:58.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.073+0000 7efe82a00640 1 --2- 172.21.15.159:0/2147953644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe7c15c390 0x7efe7c073790 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.073+0000 7efe82a00640 1 -- 172.21.15.159:0/2147953644 >> 172.21.15.159:0/2147953644 conn(0x7efe7c06b990 msgr2=0x7efe7c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:58.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.074+0000 7efe82a00640 1 -- 172.21.15.159:0/2147953644 shutdown_connections 2024-04-04T06:16:58.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.074+0000 7efe82a00640 1 -- 172.21.15.159:0/2147953644 wait complete. 2024-04-04T06:16:58.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.075+0000 7efe82a00640 1 Processor -- start 2024-04-04T06:16:58.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.075+0000 7efe82a00640 1 -- start start 2024-04-04T06:16:58.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.076+0000 7efe82a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe7c0af830 0x7efe7c150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:58.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.076+0000 7efe82a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe7c0d0650 0x7efe7c15d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:58.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.076+0000 7efe82a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe7c15c390 0x7efe7c1e9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:58.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.076+0000 7efe82a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c160d50 con 0x7efe7c0d0650 2024-04-04T06:16:58.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.076+0000 7efe82a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c160120 con 0x7efe7c0af830 2024-04-04T06:16:58.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.076+0000 7efe82a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efe7c0cf980 con 0x7efe7c15c390 2024-04-04T06:16:58.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.077+0000 7efe7b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe7c0d0650 0x7efe7c15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:58.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.077+0000 7efe7b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe7c0d0650 0x7efe7c15d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44002/0 (socket says 172.21.15.159:44002) 2024-04-04T06:16:58.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.077+0000 7efe7b400640 1 -- 172.21.15.159:0/3700984188 learned_addr learned my addr 172.21.15.159:0/3700984188 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:58.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.077+0000 7efe7be00640 1 --2- 172.21.15.159:0/3700984188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe7c0af830 0x7efe7c150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:58.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.077+0000 7efe7b400640 1 -- 172.21.15.159:0/3700984188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe7c15c390 msgr2=0x7efe7c1e9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:58.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.077+0000 7efe7b400640 1 --2- 172.21.15.159:0/3700984188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe7c15c390 0x7efe7c1e9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.077+0000 7efe7b400640 1 -- 172.21.15.159:0/3700984188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe7c0af830 msgr2=0x7efe7c150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:58.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.077+0000 7efe7b400640 1 --2- 172.21.15.159:0/3700984188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe7c0af830 0x7efe7c150860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.077+0000 7efe7b400640 1 -- 172.21.15.159:0/3700984188 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7c15dc40 con 0x7efe7c0d0650 2024-04-04T06:16:58.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.077+0000 7efe7b400640 1 --2- 172.21.15.159:0/3700984188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe7c0d0650 0x7efe7c15d280 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7efe6800c460 tx=0x7efe6800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:58.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.078+0000 7efe73e00640 1 -- 172.21.15.159:0/3700984188 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68016020 con 0x7efe7c0d0650 2024-04-04T06:16:58.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.078+0000 7efe73e00640 1 -- 172.21.15.159:0/3700984188 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe68012870 con 0x7efe7c0d0650 2024-04-04T06:16:58.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.078+0000 7efe73e00640 1 -- 172.21.15.159:0/3700984188 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68012c60 con 0x7efe7c0d0650 2024-04-04T06:16:58.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.078+0000 7efe82a00640 1 -- 172.21.15.159:0/3700984188 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe7c156130 con 0x7efe7c0d0650 2024-04-04T06:16:58.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.078+0000 7efe82a00640 1 -- 172.21.15.159:0/3700984188 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe7c1513d0 con 0x7efe7c0d0650 2024-04-04T06:16:58.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.079+0000 7efe82a00640 1 -- 172.21.15.159:0/3700984188 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe7c0685c0 con 0x7efe7c0d0650 2024-04-04T06:16:58.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.080+0000 7efe73e00640 1 -- 172.21.15.159:0/3700984188 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe68019020 con 0x7efe7c0d0650 2024-04-04T06:16:58.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.081+0000 7efe73e00640 1 --2- 172.21.15.159:0/3700984188 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efe5c06c8f0 0x7efe5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:58.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.081+0000 7efe73e00640 1 -- 172.21.15.159:0/3700984188 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7efe6808b950 con 0x7efe7c0d0650 2024-04-04T06:16:58.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.082+0000 7efe7be00640 1 --2- 172.21.15.159:0/3700984188 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efe5c06c8f0 0x7efe5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:58.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.083+0000 7efe7be00640 1 --2- 172.21.15.159:0/3700984188 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efe5c06c8f0 0x7efe5c06edb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7efe6c036000 tx=0x7efe6c03b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:58.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.086+0000 7efe73e00640 1 -- 172.21.15.159:0/3700984188 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7efe68020030 con 0x7efe7c0d0650 2024-04-04T06:16:58.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.286+0000 7efe82a00640 1 -- 172.21.15.159:0/3700984188 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe7c06f1b0 con 0x7efe7c0d0650 2024-04-04T06:16:58.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.289+0000 7efe73e00640 1 -- 172.21.15.159:0/3700984188 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+16652 (secure 0 0 0) 0x7efe68019310 con 0x7efe7c0d0650 2024-04-04T06:16:58.287 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:58.287 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":112,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:16:57.276871+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:16:55.036975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-04T06:16:56.239022+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:16:58.287 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.291+0000 7efe82a00640 1 -- 172.21.15.159:0/3700984188 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efe5c06c8f0 msgr2=0x7efe5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:58.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.291+0000 7efe82a00640 1 --2- 172.21.15.159:0/3700984188 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efe5c06c8f0 0x7efe5c06edb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7efe6c036000 tx=0x7efe6c03b040 comp rx=0 tx=0).stop 2024-04-04T06:16:58.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.291+0000 7efe82a00640 1 -- 172.21.15.159:0/3700984188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe7c0d0650 msgr2=0x7efe7c15d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:58.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.291+0000 7efe82a00640 1 --2- 172.21.15.159:0/3700984188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe7c0d0650 0x7efe7c15d280 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7efe6800c460 tx=0x7efe6800c920 comp rx=0 tx=0).stop 2024-04-04T06:16:58.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.292+0000 7efe82a00640 1 -- 172.21.15.159:0/3700984188 shutdown_connections 2024-04-04T06:16:58.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.292+0000 7efe82a00640 1 --2- 172.21.15.159:0/3700984188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe7c15c390 0x7efe7c1e9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.292+0000 7efe82a00640 1 --2- 172.21.15.159:0/3700984188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe7c0d0650 0x7efe7c15d280 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.292+0000 7efe82a00640 1 --2- 172.21.15.159:0/3700984188 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efe5c06c8f0 0x7efe5c06edb0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.292+0000 7efe82a00640 1 --2- 172.21.15.159:0/3700984188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe7c0af830 0x7efe7c150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.292+0000 7efe82a00640 1 -- 172.21.15.159:0/3700984188 >> 172.21.15.159:0/3700984188 conn(0x7efe7c06b990 msgr2=0x7efe7c0b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:58.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.292+0000 7efe82a00640 1 -- 172.21.15.159:0/3700984188 shutdown_connections 2024-04-04T06:16:58.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.292+0000 7efe82a00640 1 -- 172.21.15.159:0/3700984188 wait complete. 2024-04-04T06:16:58.304 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:16:58.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.511+0000 7f421d600640 1 Processor -- start 2024-04-04T06:16:58.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.512+0000 7f421d600640 1 -- start start 2024-04-04T06:16:58.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.513+0000 7f421d600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f421815b310 0x7f42180727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:58.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.513+0000 7f421d600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f421815b6f0 con 0x7f42180b4f90 2024-04-04T06:16:58.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.513+0000 7f421d600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4218072e80 con 0x7f42180d0780 2024-04-04T06:16:58.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.513+0000 7f421d600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f4218073160 con 0x7f421815b310 2024-04-04T06:16:58.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.513+0000 7f4216a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f421815b310 0x7f42180727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:58.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.513+0000 7f4216a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f421815b310 0x7f42180727f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44014/0 (socket says 172.21.15.159:44014) 2024-04-04T06:16:58.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.513+0000 7f4216a00640 1 -- 172.21.15.159:0/4090772215 learned_addr learned my addr 172.21.15.159:0/4090772215 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:58.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.513+0000 7f4216000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f42180b4f90 0x7f421815f7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45224/0 (socket says 172.21.15.159:45224) 2024-04-04T06:16:58.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.514+0000 7f4216a00640 1 -- 172.21.15.159:0/4090772215 >> v1:172.21.15.178:6789/0 conn(0x7f42180d0780 legacy=0x7f42180adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:58.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.514+0000 7f4216a00640 1 -- 172.21.15.159:0/4090772215 >> v1:172.21.15.159:6789/0 conn(0x7f42180b4f90 legacy=0x7f421815f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:58.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.514+0000 7f4215600640 1 -- 172.21.15.159:0/4090772215 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 313304063 0 0) 0x7f421815b6f0 con 0x7f42180b4f90 2024-04-04T06:16:58.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.514+0000 7f4216a00640 1 -- 172.21.15.159:0/4090772215 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421815efe0 con 0x7f421815b310 2024-04-04T06:16:58.509 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.515+0000 7f4216a00640 1 --2- 172.21.15.159:0/4090772215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f421815b310 0x7f42180727f0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f4208009870 tx=0x7f420802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da5556eac82df26 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:58.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.515+0000 7f4215600640 1 -- 172.21.15.159:0/4090772215 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420802e070 con 0x7f421815b310 2024-04-04T06:16:58.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.515+0000 7f4215600640 1 -- 172.21.15.159:0/4090772215 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4208037070 con 0x7f421815b310 2024-04-04T06:16:58.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.515+0000 7f4215600640 1 -- 172.21.15.159:0/4090772215 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4208031a00 con 0x7f421815b310 2024-04-04T06:16:58.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.516+0000 7f421d600640 1 -- 172.21.15.159:0/4090772215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f421815b310 msgr2=0x7f42180727f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:58.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.516+0000 7f421d600640 1 --2- 172.21.15.159:0/4090772215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f421815b310 0x7f42180727f0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f4208009870 tx=0x7f420802b880 comp rx=0 tx=0).stop 2024-04-04T06:16:58.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.516+0000 7f421d600640 1 -- 172.21.15.159:0/4090772215 shutdown_connections 2024-04-04T06:16:58.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.516+0000 7f421d600640 1 --2- 172.21.15.159:0/4090772215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f421815b310 0x7f42180727f0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.516+0000 7f421d600640 1 -- 172.21.15.159:0/4090772215 >> 172.21.15.159:0/4090772215 conn(0x7f421806b990 msgr2=0x7f421806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:58.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.516+0000 7f421d600640 1 -- 172.21.15.159:0/4090772215 shutdown_connections 2024-04-04T06:16:58.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.516+0000 7f421d600640 1 -- 172.21.15.159:0/4090772215 wait complete. 2024-04-04T06:16:58.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.518+0000 7f421d600640 1 Processor -- start 2024-04-04T06:16:58.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.518+0000 7f421d600640 1 -- start start 2024-04-04T06:16:58.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.519+0000 7f421d600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42180b4f90 0x7f4218147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:58.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.519+0000 7f421d600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42180d0780 0x7f421815caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:58.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.519+0000 7f421d600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f421815b310 0x7f42181e4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:58.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.519+0000 7f421d600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4218073440 con 0x7f421815b310 2024-04-04T06:16:58.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.519+0000 7f421d600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f42180b53f0 con 0x7f42180d0780 2024-04-04T06:16:58.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.519+0000 7f421d600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f42180ae390 con 0x7f42180b4f90 2024-04-04T06:16:58.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.519+0000 7f4217400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f421815b310 0x7f42181e4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:58.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.519+0000 7f4217400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f421815b310 0x7f42181e4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44028/0 (socket says 172.21.15.159:44028) 2024-04-04T06:16:58.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.519+0000 7f4217400640 1 -- 172.21.15.159:0/1882273269 learned_addr learned my addr 172.21.15.159:0/1882273269 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:58.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.520+0000 7f4216000640 1 --2- 172.21.15.159:0/1882273269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42180d0780 0x7f421815caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:58.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.520+0000 7f4217400640 1 -- 172.21.15.159:0/1882273269 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42180b4f90 msgr2=0x7f4218147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:58.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.520+0000 7f4217400640 1 --2- 172.21.15.159:0/1882273269 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42180b4f90 0x7f4218147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.520+0000 7f4217400640 1 -- 172.21.15.159:0/1882273269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42180d0780 msgr2=0x7f421815caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:58.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.520+0000 7f4217400640 1 --2- 172.21.15.159:0/1882273269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42180d0780 0x7f421815caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.520+0000 7f4217400640 1 -- 172.21.15.159:0/1882273269 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4218151bd0 con 0x7f421815b310 2024-04-04T06:16:58.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.520+0000 7f4217400640 1 --2- 172.21.15.159:0/1882273269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f421815b310 0x7f42181e4560 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f421000c440 tx=0x7f421000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:58.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.520+0000 7f420ea00640 1 -- 172.21.15.159:0/1882273269 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210011790 con 0x7f421815b310 2024-04-04T06:16:58.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.520+0000 7f420ea00640 1 -- 172.21.15.159:0/1882273269 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4210017070 con 0x7f421815b310 2024-04-04T06:16:58.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.520+0000 7f420ea00640 1 -- 172.21.15.159:0/1882273269 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210011b70 con 0x7f421815b310 2024-04-04T06:16:58.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.521+0000 7f421d600640 1 -- 172.21.15.159:0/1882273269 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4218151eb0 con 0x7f421815b310 2024-04-04T06:16:58.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.521+0000 7f421d600640 1 -- 172.21.15.159:0/1882273269 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f421815cfe0 con 0x7f421815b310 2024-04-04T06:16:58.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.522+0000 7f421d600640 1 -- 172.21.15.159:0/1882273269 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42180685c0 con 0x7f421815b310 2024-04-04T06:16:58.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.523+0000 7f420ea00640 1 -- 172.21.15.159:0/1882273269 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4210011d10 con 0x7f421815b310 2024-04-04T06:16:58.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.524+0000 7f420ea00640 1 --2- 172.21.15.159:0/1882273269 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f41f406c8f0 0x7f41f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:58.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.524+0000 7f420ea00640 1 -- 172.21.15.159:0/1882273269 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f421008b940 con 0x7f421815b310 2024-04-04T06:16:58.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.524+0000 7f4216a00640 1 --2- 172.21.15.159:0/1882273269 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f41f406c8f0 0x7f41f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:58.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.525+0000 7f4216a00640 1 --2- 172.21.15.159:0/1882273269 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f41f406c8f0 0x7f41f406edb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4208036000 tx=0x7f4208030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:58.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.528+0000 7f420ea00640 1 -- 172.21.15.159:0/1882273269 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4210059d60 con 0x7f421815b310 2024-04-04T06:16:58.721 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.727+0000 7f421d600640 1 -- 172.21.15.159:0/1882273269 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f421815ec20 con 0x7f421815b310 2024-04-04T06:16:58.724 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.730+0000 7f420ea00640 1 -- 172.21.15.159:0/1882273269 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+16652 (secure 0 0 0) 0x7f4210059660 con 0x7f421815b310 2024-04-04T06:16:58.725 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:16:58.725 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":112,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:16:57.276871+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:16:55.036975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-04T06:16:56.239022+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:16:58.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.732+0000 7f421d600640 1 -- 172.21.15.159:0/1882273269 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f41f406c8f0 msgr2=0x7f41f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:58.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.732+0000 7f421d600640 1 --2- 172.21.15.159:0/1882273269 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f41f406c8f0 0x7f41f406edb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4208036000 tx=0x7f4208030000 comp rx=0 tx=0).stop 2024-04-04T06:16:58.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.732+0000 7f421d600640 1 -- 172.21.15.159:0/1882273269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f421815b310 msgr2=0x7f42181e4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:58.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.732+0000 7f421d600640 1 --2- 172.21.15.159:0/1882273269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f421815b310 0x7f42181e4560 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f421000c440 tx=0x7f421000c900 comp rx=0 tx=0).stop 2024-04-04T06:16:58.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.733+0000 7f421d600640 1 -- 172.21.15.159:0/1882273269 shutdown_connections 2024-04-04T06:16:58.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.733+0000 7f421d600640 1 --2- 172.21.15.159:0/1882273269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f421815b310 0x7f42181e4560 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.733+0000 7f421d600640 1 --2- 172.21.15.159:0/1882273269 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f41f406c8f0 0x7f41f406edb0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.733+0000 7f421d600640 1 --2- 172.21.15.159:0/1882273269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42180d0780 0x7f421815caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.733+0000 7f421d600640 1 --2- 172.21.15.159:0/1882273269 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f42180b4f90 0x7f4218147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.733+0000 7f421d600640 1 -- 172.21.15.159:0/1882273269 >> 172.21.15.159:0/1882273269 conn(0x7f421806b990 msgr2=0x7f4218154be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:58.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.733+0000 7f421d600640 1 -- 172.21.15.159:0/1882273269 shutdown_connections 2024-04-04T06:16:58.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.733+0000 7f421d600640 1 -- 172.21.15.159:0/1882273269 wait complete. 2024-04-04T06:16:58.746 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-04T06:16:58.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.958+0000 7fb17d600640 1 Processor -- start 2024-04-04T06:16:58.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.959+0000 7fb17d600640 1 -- start start 2024-04-04T06:16:58.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.961+0000 7fb17d600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb1780d1bf0 0x7fb17815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:58.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.961+0000 7fb17d600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb178070170 0x7fb17806b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:58.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.961+0000 7fb17d600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1780705f0 0x7fb17806bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:58.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.961+0000 7fb17d600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fb1780d1fd0 con 0x7fb178070170 2024-04-04T06:16:58.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.961+0000 7fb17d600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb17815eb80 con 0x7fb1780d1bf0 2024-04-04T06:16:58.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.961+0000 7fb17d600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb17815edc0 con 0x7fb1780705f0 2024-04-04T06:16:58.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.962+0000 7fb16e000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb178070170 0x7fb17806b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:58.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.962+0000 7fb16e000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb178070170 0x7fb17806b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44038/0 (socket says 172.21.15.159:44038) 2024-04-04T06:16:58.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.962+0000 7fb16e000640 1 -- 172.21.15.159:0/2850190727 learned_addr learned my addr 172.21.15.159:0/2850190727 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:58.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.962+0000 7fb176a00640 1 --2- 172.21.15.159:0/2850190727 >> v2:172.21.15.178:3300/0 conn(0x7fb1780d1bf0 0x7fb17815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:58.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.962+0000 7fb16e000640 1 -- 172.21.15.159:0/2850190727 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1780705f0 msgr2=0x7fb17806bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:58.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.962+0000 7fb177400640 1 --2- 172.21.15.159:0/2850190727 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1780705f0 0x7fb17806bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:58.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.962+0000 7fb16e000640 1 --2- 172.21.15.159:0/2850190727 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1780705f0 0x7fb17806bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.962+0000 7fb16e000640 1 -- 172.21.15.159:0/2850190727 >> v2:172.21.15.178:3300/0 conn(0x7fb1780d1bf0 msgr2=0x7fb17815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:16:58.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.962+0000 7fb16e000640 1 --2- 172.21.15.159:0/2850190727 >> v2:172.21.15.178:3300/0 conn(0x7fb1780d1bf0 0x7fb17815f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.962+0000 7fb16e000640 1 -- 172.21.15.159:0/2850190727 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1780b5e60 con 0x7fb178070170 2024-04-04T06:16:58.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.962+0000 7fb176a00640 1 --2- 172.21.15.159:0/2850190727 >> v2:172.21.15.178:3300/0 conn(0x7fb1780d1bf0 0x7fb17815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:58.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.963+0000 7fb16e000640 1 --2- 172.21.15.159:0/2850190727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb178070170 0x7fb17806b570 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fb1780b3d50 tx=0x7fb16002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb92b9f0ed9373f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:58.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.963+0000 7fb176000640 1 -- 172.21.15.159:0/2850190727 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160035020 con 0x7fb178070170 2024-04-04T06:16:58.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.963+0000 7fb176000640 1 -- 172.21.15.159:0/2850190727 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb16002d040 con 0x7fb178070170 2024-04-04T06:16:58.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.964+0000 7fb176000640 1 -- 172.21.15.159:0/2850190727 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160031a80 con 0x7fb178070170 2024-04-04T06:16:58.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.964+0000 7fb17d600640 1 -- 172.21.15.159:0/2850190727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb178070170 msgr2=0x7fb17806b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:58.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.964+0000 7fb17d600640 1 --2- 172.21.15.159:0/2850190727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb178070170 0x7fb17806b570 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fb1780b3d50 tx=0x7fb16002b900 comp rx=0 tx=0).stop 2024-04-04T06:16:58.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.964+0000 7fb17d600640 1 -- 172.21.15.159:0/2850190727 shutdown_connections 2024-04-04T06:16:58.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.964+0000 7fb17d600640 1 --2- 172.21.15.159:0/2850190727 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1780705f0 0x7fb17806bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.964+0000 7fb17d600640 1 --2- 172.21.15.159:0/2850190727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb178070170 0x7fb17806b570 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.964+0000 7fb17d600640 1 --2- 172.21.15.159:0/2850190727 >> v2:172.21.15.178:3300/0 conn(0x7fb1780d1bf0 0x7fb17815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.964+0000 7fb17d600640 1 -- 172.21.15.159:0/2850190727 >> 172.21.15.159:0/2850190727 conn(0x7fb1780b6560 msgr2=0x7fb1780b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:58.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.964+0000 7fb17d600640 1 -- 172.21.15.159:0/2850190727 shutdown_connections 2024-04-04T06:16:58.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.964+0000 7fb17d600640 1 -- 172.21.15.159:0/2850190727 wait complete. 2024-04-04T06:16:58.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.967+0000 7fb17d600640 1 Processor -- start 2024-04-04T06:16:58.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.967+0000 7fb17d600640 1 -- start start 2024-04-04T06:16:58.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.968+0000 7fb17d600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb178070170 0x7fb178149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:58.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.968+0000 7fb17d600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1780705f0 0x7fb17809d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:58.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.968+0000 7fb17d600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1780d1bf0 0x7fb178160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:58.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.968+0000 7fb17d600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb17815fae0 con 0x7fb178070170 2024-04-04T06:16:58.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.969+0000 7fb17d600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb17815f080 con 0x7fb1780d1bf0 2024-04-04T06:16:58.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.969+0000 7fb17d600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb1780b5b10 con 0x7fb1780705f0 2024-04-04T06:16:58.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.969+0000 7fb176a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb178070170 0x7fb178149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:58.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.969+0000 7fb176a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb178070170 0x7fb178149f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44050/0 (socket says 172.21.15.159:44050) 2024-04-04T06:16:58.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.969+0000 7fb176a00640 1 -- 172.21.15.159:0/2130736561 learned_addr learned my addr 172.21.15.159:0/2130736561 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:16:58.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.969+0000 7fb16e000640 1 --2- 172.21.15.159:0/2130736561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1780705f0 0x7fb17809d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:58.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.969+0000 7fb177400640 1 --2- 172.21.15.159:0/2130736561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1780d1bf0 0x7fb178160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:58.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.969+0000 7fb176a00640 1 -- 172.21.15.159:0/2130736561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1780705f0 msgr2=0x7fb17809d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:58.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.969+0000 7fb176a00640 1 --2- 172.21.15.159:0/2130736561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1780705f0 0x7fb17809d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.969+0000 7fb176a00640 1 -- 172.21.15.159:0/2130736561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1780d1bf0 msgr2=0x7fb178160ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:58.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.969+0000 7fb176a00640 1 --2- 172.21.15.159:0/2130736561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1780d1bf0 0x7fb178160ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:58.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.969+0000 7fb176a00640 1 -- 172.21.15.159:0/2130736561 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb178159790 con 0x7fb178070170 2024-04-04T06:16:58.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.969+0000 7fb16e000640 1 --2- 172.21.15.159:0/2130736561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1780705f0 0x7fb17809d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:16:58.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.969+0000 7fb176a00640 1 --2- 172.21.15.159:0/2130736561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb178070170 0x7fb178149f60 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fb16800c7f0 tx=0x7fb16800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:58.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.970+0000 7fb16f400640 1 -- 172.21.15.159:0/2130736561 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb168015070 con 0x7fb178070170 2024-04-04T06:16:58.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.970+0000 7fb16f400640 1 -- 172.21.15.159:0/2130736561 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb168007a30 con 0x7fb178070170 2024-04-04T06:16:58.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.970+0000 7fb16f400640 1 -- 172.21.15.159:0/2130736561 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb168007e00 con 0x7fb178070170 2024-04-04T06:16:58.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.970+0000 7fb17d600640 1 -- 172.21.15.159:0/2130736561 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb178159a70 con 0x7fb178070170 2024-04-04T06:16:58.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.970+0000 7fb17d600640 1 -- 172.21.15.159:0/2130736561 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb17815a030 con 0x7fb178070170 2024-04-04T06:16:58.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.971+0000 7fb17d600640 1 -- 172.21.15.159:0/2130736561 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1780685c0 con 0x7fb178070170 2024-04-04T06:16:58.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.972+0000 7fb16f400640 1 -- 172.21.15.159:0/2130736561 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb168018070 con 0x7fb178070170 2024-04-04T06:16:58.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.973+0000 7fb16f400640 1 --2- 172.21.15.159:0/2130736561 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb15406c8f0 0x7fb15406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:16:58.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.973+0000 7fb16f400640 1 -- 172.21.15.159:0/2130736561 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7fb16808bb70 con 0x7fb178070170 2024-04-04T06:16:58.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.973+0000 7fb16e000640 1 --2- 172.21.15.159:0/2130736561 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb15406c8f0 0x7fb15406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:16:58.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.975+0000 7fb16e000640 1 --2- 172.21.15.159:0/2130736561 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb15406c8f0 0x7fb15406edb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb160036000 tx=0x7fb160008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:16:58.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:58.977+0000 7fb16f400640 1 -- 172.21.15.159:0/2130736561 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb168059f90 con 0x7fb178070170 2024-04-04T06:16:59.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:59.217+0000 7fb17d600640 1 -- 172.21.15.159:0/2130736561 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb1780b0290 con 0x7fb178070170 2024-04-04T06:16:59.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:59.860+0000 7fb16f400640 1 -- 172.21.15.159:0/2130736561 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v100) ==== 118+0+0 (secure 0 0 0) 0x7fb168059890 con 0x7fb178070170 2024-04-04T06:16:59.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:59.864+0000 7fb17d600640 1 -- 172.21.15.159:0/2130736561 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb15406c8f0 msgr2=0x7fb15406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:59.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:59.864+0000 7fb17d600640 1 --2- 172.21.15.159:0/2130736561 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb15406c8f0 0x7fb15406edb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb160036000 tx=0x7fb160008030 comp rx=0 tx=0).stop 2024-04-04T06:16:59.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:59.864+0000 7fb17d600640 1 -- 172.21.15.159:0/2130736561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb178070170 msgr2=0x7fb178149f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:16:59.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:59.864+0000 7fb17d600640 1 --2- 172.21.15.159:0/2130736561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb178070170 0x7fb178149f60 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fb16800c7f0 tx=0x7fb16800cbf0 comp rx=0 tx=0).stop 2024-04-04T06:16:59.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:59.864+0000 7fb17d600640 1 -- 172.21.15.159:0/2130736561 shutdown_connections 2024-04-04T06:16:59.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:59.864+0000 7fb17d600640 1 --2- 172.21.15.159:0/2130736561 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb15406c8f0 0x7fb15406edb0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:59.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:59.864+0000 7fb17d600640 1 --2- 172.21.15.159:0/2130736561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1780d1bf0 0x7fb178160ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:59.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:59.864+0000 7fb17d600640 1 --2- 172.21.15.159:0/2130736561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1780705f0 0x7fb17809d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:59.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:59.864+0000 7fb17d600640 1 --2- 172.21.15.159:0/2130736561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb178070170 0x7fb178149f60 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:16:59.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:59.864+0000 7fb17d600640 1 -- 172.21.15.159:0/2130736561 >> 172.21.15.159:0/2130736561 conn(0x7fb1780b6560 msgr2=0x7fb17815fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:16:59.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:59.865+0000 7fb17d600640 1 -- 172.21.15.159:0/2130736561 shutdown_connections 2024-04-04T06:16:59.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:16:59.865+0000 7fb17d600640 1 -- 172.21.15.159:0/2130736561 wait complete. 2024-04-04T06:16:59.881 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-04T06:16:59.882 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-04T06:17:00.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.100+0000 7f3e8fc00640 1 Processor -- start 2024-04-04T06:17:00.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.100+0000 7f3e8fc00640 1 -- start start 2024-04-04T06:17:00.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.101+0000 7f3e8fc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3e88153580 0x7f3e880d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:00.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.101+0000 7f3e8fc00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e880afab0 con 0x7f3e880af4b0 2024-04-04T06:17:00.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.101+0000 7f3e8fc00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e8815cc80 con 0x7f3e880d07c0 2024-04-04T06:17:00.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.101+0000 7f3e8fc00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f3e880d0c30 con 0x7f3e88153580 2024-04-04T06:17:00.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.102+0000 7f3e8d200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3e88153580 0x7f3e880d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:00.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.102+0000 7f3e8dc00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f3e880af4b0 0x7f3e881600d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45230/0 (socket says 172.21.15.159:45230) 2024-04-04T06:17:00.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.102+0000 7f3e8dc00640 1 -- 172.21.15.159:0/4255770723 learned_addr learned my addr 172.21.15.159:0/4255770723 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:00.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.102+0000 7f3e87400640 1 -- 172.21.15.159:0/4255770723 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1532621398 0 0) 0x7f3e880afab0 con 0x7f3e880af4b0 2024-04-04T06:17:00.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.102+0000 7f3e87400640 1 -- 172.21.15.159:0/4255770723 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e68003520 con 0x7f3e880af4b0 2024-04-04T06:17:00.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.102+0000 7f3e8d200640 1 -- 172.21.15.159:0/4255770723 >> v1:172.21.15.178:6789/0 conn(0x7f3e880d07c0 legacy=0x7f3e880aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:00.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.102+0000 7f3e8d200640 1 -- 172.21.15.159:0/4255770723 >> v1:172.21.15.159:6789/0 conn(0x7f3e880af4b0 legacy=0x7f3e881600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:00.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.102+0000 7f3e8d200640 1 -- 172.21.15.159:0/4255770723 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e8815da00 con 0x7f3e88153580 2024-04-04T06:17:00.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.104+0000 7f3e8d200640 1 --2- 172.21.15.159:0/4255770723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e88153580 0x7f3e880d0280 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f3e780097a0 tx=0x7f3e7802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b47a77a0d9f8bd server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:00.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.105+0000 7f3e87400640 1 -- 172.21.15.159:0/4255770723 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78035020 con 0x7f3e88153580 2024-04-04T06:17:00.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.105+0000 7f3e87400640 1 -- 172.21.15.159:0/4255770723 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e78031830 con 0x7f3e88153580 2024-04-04T06:17:00.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.105+0000 7f3e87400640 1 -- 172.21.15.159:0/4255770723 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78031c00 con 0x7f3e88153580 2024-04-04T06:17:00.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.105+0000 7f3e8fc00640 1 -- 172.21.15.159:0/4255770723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e88153580 msgr2=0x7f3e880d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:00.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.105+0000 7f3e8fc00640 1 --2- 172.21.15.159:0/4255770723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e88153580 0x7f3e880d0280 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f3e780097a0 tx=0x7f3e7802b8d0 comp rx=0 tx=0).stop 2024-04-04T06:17:00.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.105+0000 7f3e8fc00640 1 -- 172.21.15.159:0/4255770723 shutdown_connections 2024-04-04T06:17:00.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.105+0000 7f3e8fc00640 1 --2- 172.21.15.159:0/4255770723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e88153580 0x7f3e880d0280 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:00.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.105+0000 7f3e8fc00640 1 -- 172.21.15.159:0/4255770723 >> 172.21.15.159:0/4255770723 conn(0x7f3e8806b750 msgr2=0x7f3e8806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:00.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.106+0000 7f3e8fc00640 1 -- 172.21.15.159:0/4255770723 shutdown_connections 2024-04-04T06:17:00.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.106+0000 7f3e8fc00640 1 -- 172.21.15.159:0/4255770723 wait complete. 2024-04-04T06:17:00.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.108+0000 7f3e8fc00640 1 Processor -- start 2024-04-04T06:17:00.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.108+0000 7f3e8fc00640 1 -- start start 2024-04-04T06:17:00.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.109+0000 7f3e8fc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e880af4b0 0x7f3e881491f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:00.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.109+0000 7f3e8fc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e880d07c0 0x7f3e88071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:00.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.110+0000 7f3e8fc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e88153580 0x7f3e88092080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:00.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.110+0000 7f3e8fc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8815d4e0 con 0x7f3e880d07c0 2024-04-04T06:17:00.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.110+0000 7f3e8fc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8815d200 con 0x7f3e880af4b0 2024-04-04T06:17:00.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.110+0000 7f3e8fc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3e88153960 con 0x7f3e88153580 2024-04-04T06:17:00.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.110+0000 7f3e87e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e880d07c0 0x7f3e88071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:00.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.110+0000 7f3e87e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e880d07c0 0x7f3e88071aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44068/0 (socket says 172.21.15.159:44068) 2024-04-04T06:17:00.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.110+0000 7f3e87e00640 1 -- 172.21.15.159:0/683293257 learned_addr learned my addr 172.21.15.159:0/683293257 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:00.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.110+0000 7f3e87e00640 1 -- 172.21.15.159:0/683293257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e88153580 msgr2=0x7f3e88092080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:00.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.110+0000 7f3e8d200640 1 --2- 172.21.15.159:0/683293257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e880af4b0 0x7f3e881491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:00.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.110+0000 7f3e87e00640 1 --2- 172.21.15.159:0/683293257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e88153580 0x7f3e88092080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:00.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.110+0000 7f3e87e00640 1 -- 172.21.15.159:0/683293257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e880af4b0 msgr2=0x7f3e881491f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:00.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.110+0000 7f3e87e00640 1 --2- 172.21.15.159:0/683293257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e880af4b0 0x7f3e881491f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:00.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.110+0000 7f3e87e00640 1 -- 172.21.15.159:0/683293257 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e88072480 con 0x7f3e880d07c0 2024-04-04T06:17:00.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.111+0000 7f3e87e00640 1 --2- 172.21.15.159:0/683293257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e880d07c0 0x7f3e88071aa0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000c730 tx=0x7f3e7000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:00.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.111+0000 7f3e84c00640 1 -- 172.21.15.159:0/683293257 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e70016020 con 0x7f3e880d07c0 2024-04-04T06:17:00.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.111+0000 7f3e84c00640 1 -- 172.21.15.159:0/683293257 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e70006d50 con 0x7f3e880d07c0 2024-04-04T06:17:00.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.111+0000 7f3e84c00640 1 -- 172.21.15.159:0/683293257 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e70003bb0 con 0x7f3e880d07c0 2024-04-04T06:17:00.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.111+0000 7f3e8fc00640 1 -- 172.21.15.159:0/683293257 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e88072760 con 0x7f3e880d07c0 2024-04-04T06:17:00.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.111+0000 7f3e8fc00640 1 -- 172.21.15.159:0/683293257 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e88072d20 con 0x7f3e880d07c0 2024-04-04T06:17:00.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.112+0000 7f3e8fc00640 1 -- 172.21.15.159:0/683293257 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e880685c0 con 0x7f3e880d07c0 2024-04-04T06:17:00.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.114+0000 7f3e84c00640 1 -- 172.21.15.159:0/683293257 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e7000a850 con 0x7f3e880d07c0 2024-04-04T06:17:00.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.114+0000 7f3e84c00640 1 --2- 172.21.15.159:0/683293257 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3e6406c8f0 0x7f3e6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:00.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.114+0000 7f3e84c00640 1 -- 172.21.15.159:0/683293257 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f3e7008b800 con 0x7f3e880d07c0 2024-04-04T06:17:00.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.115+0000 7f3e8d200640 1 --2- 172.21.15.159:0/683293257 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3e6406c8f0 0x7f3e6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:00.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.116+0000 7f3e8d200640 1 --2- 172.21.15.159:0/683293257 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3e6406c8f0 0x7f3e6406edb0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3e78036000 tx=0x7f3e78008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:00.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.119+0000 7f3e84c00640 1 -- 172.21.15.159:0/683293257 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3e70020020 con 0x7f3e880d07c0 2024-04-04T06:17:00.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.358+0000 7f3e8fc00640 1 -- 172.21.15.159:0/683293257 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3e880af1b0 con 0x7f3e880d07c0 2024-04-04T06:17:00.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.863+0000 7f3e84c00640 1 -- 172.21.15.159:0/683293257 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v101) ==== 122+0+0 (secure 0 0 0) 0x7f3e70019310 con 0x7f3e880d07c0 2024-04-04T06:17:00.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.868+0000 7f3e8fc00640 1 -- 172.21.15.159:0/683293257 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3e6406c8f0 msgr2=0x7f3e6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:00.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.868+0000 7f3e8fc00640 1 --2- 172.21.15.159:0/683293257 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3e6406c8f0 0x7f3e6406edb0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3e78036000 tx=0x7f3e78008030 comp rx=0 tx=0).stop 2024-04-04T06:17:00.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.868+0000 7f3e8fc00640 1 -- 172.21.15.159:0/683293257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e880d07c0 msgr2=0x7f3e88071aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:00.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.868+0000 7f3e8fc00640 1 --2- 172.21.15.159:0/683293257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e880d07c0 0x7f3e88071aa0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000c730 tx=0x7f3e7000cbf0 comp rx=0 tx=0).stop 2024-04-04T06:17:00.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.868+0000 7f3e8fc00640 1 -- 172.21.15.159:0/683293257 shutdown_connections 2024-04-04T06:17:00.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.868+0000 7f3e8fc00640 1 --2- 172.21.15.159:0/683293257 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3e88153580 0x7f3e88092080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:00.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.868+0000 7f3e8fc00640 1 --2- 172.21.15.159:0/683293257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e880d07c0 0x7f3e88071aa0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:00.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.868+0000 7f3e8fc00640 1 --2- 172.21.15.159:0/683293257 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3e6406c8f0 0x7f3e6406edb0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:00.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.868+0000 7f3e8fc00640 1 --2- 172.21.15.159:0/683293257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e880af4b0 0x7f3e881491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:00.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.868+0000 7f3e8fc00640 1 -- 172.21.15.159:0/683293257 >> 172.21.15.159:0/683293257 conn(0x7f3e8806b750 msgr2=0x7f3e880b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:00.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.868+0000 7f3e8fc00640 1 -- 172.21.15.159:0/683293257 shutdown_connections 2024-04-04T06:17:00.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:00.868+0000 7f3e8fc00640 1 -- 172.21.15.159:0/683293257 wait complete. 2024-04-04T06:17:00.885 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-04T06:17:00.885 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:17:01.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.092+0000 7fc324a00640 1 Processor -- start 2024-04-04T06:17:01.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.093+0000 7fc324a00640 1 -- start start 2024-04-04T06:17:01.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.094+0000 7fc324a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc320161b00 0x7fc320162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:01.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.094+0000 7fc324a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc3200d0d60 0x7fc3200b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:01.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.094+0000 7fc324a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc320071b60 con 0x7fc3200ad120 2024-04-04T06:17:01.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.094+0000 7fc324a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fc320161ee0 con 0x7fc320161b00 2024-04-04T06:17:01.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.094+0000 7fc324a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc3200b3450 con 0x7fc3200d0d60 2024-04-04T06:17:01.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.094+0000 7fc31de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc320161b00 0x7fc320162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:01.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.095+0000 7fc31de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc320161b00 0x7fc320162ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44084/0 (socket says 172.21.15.159:44084) 2024-04-04T06:17:01.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.095+0000 7fc31de00640 1 -- 172.21.15.159:0/1674750022 learned_addr learned my addr 172.21.15.159:0/1674750022 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:01.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.095+0000 7fc31e800640 1 --2- 172.21.15.159:0/1674750022 >> v2:172.21.15.178:3300/0 conn(0x7fc3200d0d60 0x7fc3200b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:01.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.095+0000 7fc31de00640 1 -- 172.21.15.159:0/1674750022 >> v2:172.21.15.178:3300/0 conn(0x7fc3200d0d60 msgr2=0x7fc3200b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:01.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.095+0000 7fc31de00640 1 --2- 172.21.15.159:0/1674750022 >> v2:172.21.15.178:3300/0 conn(0x7fc3200d0d60 0x7fc3200b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:01.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.095+0000 7fc31de00640 1 -- 172.21.15.159:0/1674750022 >> v1:172.21.15.178:6789/0 conn(0x7fc3200ad120 legacy=0x7fc3200d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:01.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.095+0000 7fc31de00640 1 -- 172.21.15.159:0/1674750022 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc320071e40 con 0x7fc320161b00 2024-04-04T06:17:01.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.097+0000 7fc31de00640 1 --2- 172.21.15.159:0/1674750022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc320161b00 0x7fc320162ab0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fc3200b63b0 tx=0x7fc31002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce9fc2597abbae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:01.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.097+0000 7fc31d400640 1 -- 172.21.15.159:0/1674750022 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31002ca50 con 0x7fc320161b00 2024-04-04T06:17:01.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.097+0000 7fc31d400640 1 -- 172.21.15.159:0/1674750022 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc31002cbf0 con 0x7fc320161b00 2024-04-04T06:17:01.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.097+0000 7fc31d400640 1 -- 172.21.15.159:0/1674750022 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3100343f0 con 0x7fc320161b00 2024-04-04T06:17:01.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.098+0000 7fc324a00640 1 -- 172.21.15.159:0/1674750022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc320161b00 msgr2=0x7fc320162ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:01.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.098+0000 7fc324a00640 1 --2- 172.21.15.159:0/1674750022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc320161b00 0x7fc320162ab0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fc3200b63b0 tx=0x7fc31002beb0 comp rx=0 tx=0).stop 2024-04-04T06:17:01.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.098+0000 7fc324a00640 1 -- 172.21.15.159:0/1674750022 shutdown_connections 2024-04-04T06:17:01.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.098+0000 7fc324a00640 1 --2- 172.21.15.159:0/1674750022 >> v2:172.21.15.178:3300/0 conn(0x7fc3200d0d60 0x7fc3200b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:01.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.098+0000 7fc324a00640 1 --2- 172.21.15.159:0/1674750022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc320161b00 0x7fc320162ab0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:01.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.098+0000 7fc324a00640 1 -- 172.21.15.159:0/1674750022 >> 172.21.15.159:0/1674750022 conn(0x7fc32006b990 msgr2=0x7fc32006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:01.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.098+0000 7fc324a00640 1 -- 172.21.15.159:0/1674750022 shutdown_connections 2024-04-04T06:17:01.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.098+0000 7fc324a00640 1 -- 172.21.15.159:0/1674750022 wait complete. 2024-04-04T06:17:01.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.100+0000 7fc324a00640 1 Processor -- start 2024-04-04T06:17:01.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.100+0000 7fc324a00640 1 -- start start 2024-04-04T06:17:01.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.101+0000 7fc324a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3200ad120 0x7fc32014d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:01.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.101+0000 7fc324a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3200d0d60 0x7fc32014bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:01.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.101+0000 7fc324a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc320161b00 0x7fc32014c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:01.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.101+0000 7fc324a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc320162ff0 con 0x7fc3200ad120 2024-04-04T06:17:01.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.101+0000 7fc324a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc3200b3730 con 0x7fc3200d0d60 2024-04-04T06:17:01.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.101+0000 7fc324a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc3200717f0 con 0x7fc320161b00 2024-04-04T06:17:01.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.101+0000 7fc31de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3200ad120 0x7fc32014d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:01.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.101+0000 7fc31de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3200ad120 0x7fc32014d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44090/0 (socket says 172.21.15.159:44090) 2024-04-04T06:17:01.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.101+0000 7fc31de00640 1 -- 172.21.15.159:0/1508892927 learned_addr learned my addr 172.21.15.159:0/1508892927 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:01.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.101+0000 7fc31e800640 1 --2- 172.21.15.159:0/1508892927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc320161b00 0x7fc32014c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:01.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.101+0000 7fc315400640 1 --2- 172.21.15.159:0/1508892927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3200d0d60 0x7fc32014bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:01.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.102+0000 7fc31de00640 1 -- 172.21.15.159:0/1508892927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc320161b00 msgr2=0x7fc32014c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:01.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.102+0000 7fc31de00640 1 --2- 172.21.15.159:0/1508892927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc320161b00 0x7fc32014c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:01.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.102+0000 7fc31de00640 1 -- 172.21.15.159:0/1508892927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3200d0d60 msgr2=0x7fc32014bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:01.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.102+0000 7fc31de00640 1 --2- 172.21.15.159:0/1508892927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3200d0d60 0x7fc32014bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:01.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.102+0000 7fc31de00640 1 -- 172.21.15.159:0/1508892927 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc320149f90 con 0x7fc3200ad120 2024-04-04T06:17:01.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.102+0000 7fc315400640 1 --2- 172.21.15.159:0/1508892927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3200d0d60 0x7fc32014bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:01.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.102+0000 7fc31de00640 1 --2- 172.21.15.159:0/1508892927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3200ad120 0x7fc32014d370 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fc310009f10 tx=0x7fc310004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:01.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.102+0000 7fc316a00640 1 -- 172.21.15.159:0/1508892927 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310043020 con 0x7fc3200ad120 2024-04-04T06:17:01.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.103+0000 7fc316a00640 1 -- 172.21.15.159:0/1508892927 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc31003e450 con 0x7fc3200ad120 2024-04-04T06:17:01.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.103+0000 7fc316a00640 1 -- 172.21.15.159:0/1508892927 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310009510 con 0x7fc3200ad120 2024-04-04T06:17:01.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.103+0000 7fc324a00640 1 -- 172.21.15.159:0/1508892927 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32014a270 con 0x7fc3200ad120 2024-04-04T06:17:01.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.103+0000 7fc324a00640 1 -- 172.21.15.159:0/1508892927 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc32014a830 con 0x7fc3200ad120 2024-04-04T06:17:01.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.104+0000 7fc324a00640 1 -- 172.21.15.159:0/1508892927 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3200685c0 con 0x7fc3200ad120 2024-04-04T06:17:01.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.106+0000 7fc316a00640 1 -- 172.21.15.159:0/1508892927 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc310046020 con 0x7fc3200ad120 2024-04-04T06:17:01.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.107+0000 7fc316a00640 1 --2- 172.21.15.159:0/1508892927 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc2fc06c8f0 0x7fc2fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:01.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.107+0000 7fc316a00640 1 -- 172.21.15.159:0/1508892927 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7fc3100ae420 con 0x7fc3200ad120 2024-04-04T06:17:01.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.107+0000 7fc315400640 1 --2- 172.21.15.159:0/1508892927 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc2fc06c8f0 0x7fc2fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:01.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.108+0000 7fc315400640 1 --2- 172.21.15.159:0/1508892927 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc2fc06c8f0 0x7fc2fc06edb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc3201f8070 tx=0x7fc308008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:01.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.111+0000 7fc316a00640 1 -- 172.21.15.159:0/1508892927 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc31004d030 con 0x7fc3200ad120 2024-04-04T06:17:01.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.378+0000 7fc324a00640 1 -- 172.21.15.159:0/1508892927 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc32006f3a0 con 0x7fc3200ad120 2024-04-04T06:17:01.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.380+0000 7fc316a00640 1 -- 172.21.15.159:0/1508892927 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5197 (secure 0 0 0) 0x7fc31007c7b0 con 0x7fc3200ad120 2024-04-04T06:17:01.375 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:01.375 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":101,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5732,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1498324773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1498324773},{"type":"v1","addr":"172.21.15.178:6837","nonce":1498324773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2455031662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":2455031662},{"type":"v1","addr":"172.21.15.159:6835","nonce":2455031662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6343,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1823228151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1823228151},{"type":"v1","addr":"172.21.15.178:6836","nonce":1823228151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:16:57.277344+0000","modified":"2024-04-04T06:17:00.359830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8853},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8853":{"gid":8853,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6834/1017894858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1017894858},{"type":"v1","addr":"172.21.15.159:6834","nonce":1017894858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8853,"qdb_cluster":[8853]},"id":8}]} 2024-04-04T06:17:01.375 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 101 2024-04-04T06:17:01.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.382+0000 7fc324a00640 1 -- 172.21.15.159:0/1508892927 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc2fc06c8f0 msgr2=0x7fc2fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:01.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.382+0000 7fc324a00640 1 --2- 172.21.15.159:0/1508892927 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc2fc06c8f0 0x7fc2fc06edb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc3201f8070 tx=0x7fc308008040 comp rx=0 tx=0).stop 2024-04-04T06:17:01.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.382+0000 7fc324a00640 1 -- 172.21.15.159:0/1508892927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3200ad120 msgr2=0x7fc32014d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:01.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.382+0000 7fc324a00640 1 --2- 172.21.15.159:0/1508892927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3200ad120 0x7fc32014d370 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fc310009f10 tx=0x7fc310004a40 comp rx=0 tx=0).stop 2024-04-04T06:17:01.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.383+0000 7fc324a00640 1 -- 172.21.15.159:0/1508892927 shutdown_connections 2024-04-04T06:17:01.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.383+0000 7fc324a00640 1 --2- 172.21.15.159:0/1508892927 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc320161b00 0x7fc32014c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:01.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.383+0000 7fc324a00640 1 --2- 172.21.15.159:0/1508892927 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc2fc06c8f0 0x7fc2fc06edb0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:01.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.383+0000 7fc324a00640 1 --2- 172.21.15.159:0/1508892927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3200d0d60 0x7fc32014bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:01.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.383+0000 7fc324a00640 1 --2- 172.21.15.159:0/1508892927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3200ad120 0x7fc32014d370 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:01.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.383+0000 7fc324a00640 1 -- 172.21.15.159:0/1508892927 >> 172.21.15.159:0/1508892927 conn(0x7fc32006b990 msgr2=0x7fc3200af1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:01.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.383+0000 7fc324a00640 1 -- 172.21.15.159:0/1508892927 shutdown_connections 2024-04-04T06:17:01.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.383+0000 7fc324a00640 1 -- 172.21.15.159:0/1508892927 wait complete. 2024-04-04T06:17:01.395 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:17:01.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.598+0000 7febbce00640 1 Processor -- start 2024-04-04T06:17:01.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.598+0000 7febbce00640 1 -- start start 2024-04-04T06:17:01.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.600+0000 7febbce00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7febb80d0e90 0x7febb80b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:01.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.600+0000 7febbce00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febb815b6f0 con 0x7febb815b310 2024-04-04T06:17:01.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.600+0000 7febbce00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febb8072e80 con 0x7febb80727f0 2024-04-04T06:17:01.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.600+0000 7febbce00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7febb8072c50 con 0x7febb80d0e90 2024-04-04T06:17:01.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.600+0000 7febb5e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7febb815b310 0x7febb815f7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45232/0 (socket says 172.21.15.159:45232) 2024-04-04T06:17:01.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.600+0000 7febb5e00640 1 -- 172.21.15.159:0/3868893101 learned_addr learned my addr 172.21.15.159:0/3868893101 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:01.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.600+0000 7febb6800640 1 --2- 172.21.15.159:0/3868893101 >> v2:172.21.15.178:3300/0 conn(0x7febb80d0e90 0x7febb80b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:01.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.601+0000 7febb5400640 1 -- 172.21.15.159:0/3868893101 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1440486234 0 0) 0x7febb815b6f0 con 0x7febb815b310 2024-04-04T06:17:01.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.601+0000 7febb5400640 1 -- 172.21.15.159:0/3868893101 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb9c003520 con 0x7febb815b310 2024-04-04T06:17:01.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.601+0000 7febb5400640 1 -- 172.21.15.159:0/3868893101 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3678676574 0 0) 0x7feb9c003520 con 0x7febb815b310 2024-04-04T06:17:01.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.601+0000 7febb5400640 1 -- 172.21.15.159:0/3868893101 >> v2:172.21.15.178:3300/0 conn(0x7febb80d0e90 msgr2=0x7febb80b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:01.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.601+0000 7febb5400640 1 --2- 172.21.15.159:0/3868893101 >> v2:172.21.15.178:3300/0 conn(0x7febb80d0e90 0x7febb80b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:01.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.601+0000 7febb5400640 1 -- 172.21.15.159:0/3868893101 >> v1:172.21.15.178:6789/0 conn(0x7febb80727f0 legacy=0x7febb80d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:01.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.601+0000 7febb5400640 1 -- 172.21.15.159:0/3868893101 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb8073160 con 0x7febb815b310 2024-04-04T06:17:01.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.601+0000 7febb5400640 1 -- 172.21.15.159:0/3868893101 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7feba8001670 con 0x7febb815b310 2024-04-04T06:17:01.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.601+0000 7febb5400640 1 -- 172.21.15.159:0/3868893101 >> v1:172.21.15.159:6789/0 conn(0x7febb815b310 legacy=0x7febb815f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:01.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.601+0000 7febb5400640 1 --2- 172.21.15.159:0/3868893101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb9c003a30 0x7feb9c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:01.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.601+0000 7febb5400640 1 --2- 172.21.15.159:0/3868893101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb9c004800 0x7feb9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:01.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.601+0000 7febb5400640 1 --2- 172.21.15.159:0/3868893101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb9c0052e0 0x7feb9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:01.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.601+0000 7febb5400640 1 -- 172.21.15.159:0/3868893101 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c00c200 con 0x7feb9c0052e0 2024-04-04T06:17:01.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.601+0000 7febb5400640 1 -- 172.21.15.159:0/3868893101 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c00c3b0 con 0x7feb9c004800 2024-04-04T06:17:01.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.601+0000 7febb5400640 1 -- 172.21.15.159:0/3868893101 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feb9c00c5b0 con 0x7feb9c003a30 2024-04-04T06:17:01.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.602+0000 7febb6800640 1 --2- 172.21.15.159:0/3868893101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb9c0052e0 0x7feb9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:01.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.602+0000 7febb5e00640 1 --2- 172.21.15.159:0/3868893101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb9c003a30 0x7feb9c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:01.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.602+0000 7febafe00640 1 --2- 172.21.15.159:0/3868893101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb9c004800 0x7feb9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:01.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.602+0000 7febb6800640 1 -- 172.21.15.159:0/3868893101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb9c003a30 msgr2=0x7feb9c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:01.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.602+0000 7febb6800640 1 --2- 172.21.15.159:0/3868893101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb9c003a30 0x7feb9c003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:01.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.602+0000 7febb6800640 1 -- 172.21.15.159:0/3868893101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb9c004800 msgr2=0x7feb9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:01.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.602+0000 7febb6800640 1 --2- 172.21.15.159:0/3868893101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb9c004800 0x7feb9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:01.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.602+0000 7febb6800640 1 -- 172.21.15.159:0/3868893101 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9c003520 con 0x7feb9c0052e0 2024-04-04T06:17:01.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.602+0000 7febafe00640 1 --2- 172.21.15.159:0/3868893101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb9c004800 0x7feb9c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:01.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.603+0000 7febb6800640 1 -- 172.21.15.159:0/3868893101 reap_dead start 2024-04-04T06:17:01.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.603+0000 7febb6800640 1 --2- 172.21.15.159:0/3868893101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb9c0052e0 0x7feb9c00b9c0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7febb80ad9a0 tx=0x7feb9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c4a30c824a513e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:01.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.603+0000 7febb5400640 1 -- 172.21.15.159:0/3868893101 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0009c10 con 0x7feb9c0052e0 2024-04-04T06:17:01.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.603+0000 7febb5400640 1 -- 172.21.15.159:0/3868893101 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febb0003b90 con 0x7feb9c0052e0 2024-04-04T06:17:01.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.603+0000 7febb5400640 1 -- 172.21.15.159:0/3868893101 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb002c990 con 0x7feb9c0052e0 2024-04-04T06:17:01.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.604+0000 7febbce00640 1 -- 172.21.15.159:0/3868893101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb9c0052e0 msgr2=0x7feb9c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:01.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.604+0000 7febbce00640 1 --2- 172.21.15.159:0/3868893101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb9c0052e0 0x7feb9c00b9c0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7febb80ad9a0 tx=0x7feb9c005830 comp rx=0 tx=0).stop 2024-04-04T06:17:01.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.604+0000 7febbce00640 1 -- 172.21.15.159:0/3868893101 shutdown_connections 2024-04-04T06:17:01.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.604+0000 7febbce00640 1 --2- 172.21.15.159:0/3868893101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb9c0052e0 0x7feb9c00b9c0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:01.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.604+0000 7febbce00640 1 -- 172.21.15.159:0/3868893101 >> 172.21.15.159:0/3868893101 conn(0x7febb806b990 msgr2=0x7febb806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:01.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.604+0000 7febbce00640 1 -- 172.21.15.159:0/3868893101 shutdown_connections 2024-04-04T06:17:01.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.604+0000 7febbce00640 1 -- 172.21.15.159:0/3868893101 wait complete. 2024-04-04T06:17:01.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.606+0000 7febbce00640 1 Processor -- start 2024-04-04T06:17:01.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.606+0000 7febbce00640 1 -- start start 2024-04-04T06:17:01.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.607+0000 7febbce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb9c0052e0 0x7febb81532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:01.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.607+0000 7febbce00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febb8145dd0 0x7febb8153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:01.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.607+0000 7febbce00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febb80cc8c0 0x7febb80ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:01.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.607+0000 7febbce00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7febb8073440 con 0x7feb9c0052e0 2024-04-04T06:17:01.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.607+0000 7febbce00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7febb815f2c0 con 0x7febb8145dd0 2024-04-04T06:17:01.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.607+0000 7febbce00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7febb80b4f90 con 0x7febb80cc8c0 2024-04-04T06:17:01.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.607+0000 7febb5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb9c0052e0 0x7febb81532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:01.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.607+0000 7febb5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb9c0052e0 0x7febb81532d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44114/0 (socket says 172.21.15.159:44114) 2024-04-04T06:17:01.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.607+0000 7febb5e00640 1 -- 172.21.15.159:0/207929990 learned_addr learned my addr 172.21.15.159:0/207929990 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:01.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.608+0000 7febb5e00640 1 -- 172.21.15.159:0/207929990 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febb80cc8c0 msgr2=0x7febb80ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:01.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.608+0000 7febb5e00640 1 --2- 172.21.15.159:0/207929990 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febb80cc8c0 0x7febb80ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:01.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.608+0000 7febb5e00640 1 -- 172.21.15.159:0/207929990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febb8145dd0 msgr2=0x7febb8153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:01.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.608+0000 7febb5e00640 1 --2- 172.21.15.159:0/207929990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febb8145dd0 0x7febb8153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:01.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.608+0000 7febb5e00640 1 -- 172.21.15.159:0/207929990 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb8146350 con 0x7feb9c0052e0 2024-04-04T06:17:01.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.608+0000 7febb5e00640 1 --2- 172.21.15.159:0/207929990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb9c0052e0 0x7febb81532d0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7feba800d7a0 tx=0x7feba800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:01.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.608+0000 7febae000640 1 -- 172.21.15.159:0/207929990 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8019070 con 0x7feb9c0052e0 2024-04-04T06:17:01.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.608+0000 7febae000640 1 -- 172.21.15.159:0/207929990 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feba80072d0 con 0x7feb9c0052e0 2024-04-04T06:17:01.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.608+0000 7febae000640 1 -- 172.21.15.159:0/207929990 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba80047e0 con 0x7feb9c0052e0 2024-04-04T06:17:01.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.608+0000 7febbce00640 1 -- 172.21.15.159:0/207929990 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb8146630 con 0x7feb9c0052e0 2024-04-04T06:17:01.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.609+0000 7febbce00640 1 -- 172.21.15.159:0/207929990 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb8146bf0 con 0x7feb9c0052e0 2024-04-04T06:17:01.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.610+0000 7febbce00640 1 -- 172.21.15.159:0/207929990 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febb80685c0 con 0x7feb9c0052e0 2024-04-04T06:17:01.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.612+0000 7febae000640 1 -- 172.21.15.159:0/207929990 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feba8004980 con 0x7feb9c0052e0 2024-04-04T06:17:01.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.613+0000 7febae000640 1 --2- 172.21.15.159:0/207929990 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feb9406c8c0 0x7feb9406ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:01.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.613+0000 7febae000640 1 -- 172.21.15.159:0/207929990 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7feba808b3b0 con 0x7feb9c0052e0 2024-04-04T06:17:01.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.613+0000 7febafe00640 1 --2- 172.21.15.159:0/207929990 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feb9406c8c0 0x7feb9406ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:01.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.614+0000 7febafe00640 1 --2- 172.21.15.159:0/207929990 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feb9406c8c0 0x7feb9406ed80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7febb80afb50 tx=0x7feba0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:01.612 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.617+0000 7febae000640 1 -- 172.21.15.159:0/207929990 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7feba8004c70 con 0x7feb9c0052e0 2024-04-04T06:17:01.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.839+0000 7febbce00640 1 -- 172.21.15.159:0/207929990 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7febb8146dc0 con 0x7feb9c0052e0 2024-04-04T06:17:01.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.841+0000 7febae000640 1 -- 172.21.15.159:0/207929990 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16639 (secure 0 0 0) 0x7feba8059740 con 0x7feb9c0052e0 2024-04-04T06:17:01.836 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:01.837 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":113,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:16:58.838677+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:16:55.036975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-04T06:16:56.239022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:17:01.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.844+0000 7febbce00640 1 -- 172.21.15.159:0/207929990 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feb9406c8c0 msgr2=0x7feb9406ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:01.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.844+0000 7febbce00640 1 --2- 172.21.15.159:0/207929990 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feb9406c8c0 0x7feb9406ed80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7febb80afb50 tx=0x7feba0009040 comp rx=0 tx=0).stop 2024-04-04T06:17:01.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.844+0000 7febbce00640 1 -- 172.21.15.159:0/207929990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb9c0052e0 msgr2=0x7febb81532d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:01.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.844+0000 7febbce00640 1 --2- 172.21.15.159:0/207929990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb9c0052e0 0x7febb81532d0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7feba800d7a0 tx=0x7feba800dc60 comp rx=0 tx=0).stop 2024-04-04T06:17:01.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.844+0000 7febbce00640 1 -- 172.21.15.159:0/207929990 shutdown_connections 2024-04-04T06:17:01.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.844+0000 7febbce00640 1 --2- 172.21.15.159:0/207929990 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7febb80cc8c0 0x7febb80ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:01.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.844+0000 7febbce00640 1 --2- 172.21.15.159:0/207929990 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feb9406c8c0 0x7feb9406ed80 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:01.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.844+0000 7febbce00640 1 --2- 172.21.15.159:0/207929990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febb8145dd0 0x7febb8153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:01.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.844+0000 7febbce00640 1 --2- 172.21.15.159:0/207929990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb9c0052e0 0x7febb81532d0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:01.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.844+0000 7febbce00640 1 -- 172.21.15.159:0/207929990 >> 172.21.15.159:0/207929990 conn(0x7febb806b990 msgr2=0x7febb806bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:01.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.844+0000 7febbce00640 1 -- 172.21.15.159:0/207929990 shutdown_connections 2024-04-04T06:17:01.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:01.845+0000 7febbce00640 1 -- 172.21.15.159:0/207929990 wait complete. 2024-04-04T06:17:01.860 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-04T06:17:01.860 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:17:02.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.070+0000 7fd44f000640 1 Processor -- start 2024-04-04T06:17:02.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.070+0000 7fd44f000640 1 -- start start 2024-04-04T06:17:02.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.073+0000 7fd44f000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd4480d0d60 0x7fd448073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:02.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.073+0000 7fd44f000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd44815fe80 con 0x7fd4480af830 2024-04-04T06:17:02.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.073+0000 7fd44f000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4480ad120 con 0x7fd44815c390 2024-04-04T06:17:02.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.073+0000 7fd44f000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fd44815c770 con 0x7fd4480d0d60 2024-04-04T06:17:02.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.073+0000 7fd44d000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd4480d0d60 0x7fd448073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:02.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.073+0000 7fd447400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fd4480af830 0x7fd4480d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45246/0 (socket says 172.21.15.159:45246) 2024-04-04T06:17:02.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.073+0000 7fd447400640 1 -- 172.21.15.159:0/1949226707 learned_addr learned my addr 172.21.15.159:0/1949226707 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:02.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.074+0000 7fd44d000640 1 -- 172.21.15.159:0/1949226707 >> v1:172.21.15.178:6789/0 conn(0x7fd44815c390 legacy=0x7fd448160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:02.069 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.074+0000 7fd44d000640 1 -- 172.21.15.159:0/1949226707 >> v1:172.21.15.159:6789/0 conn(0x7fd4480af830 legacy=0x7fd4480d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:02.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.074+0000 7fd44d000640 1 -- 172.21.15.159:0/1949226707 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4480ad400 con 0x7fd4480d0d60 2024-04-04T06:17:02.070 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.074+0000 7fd446a00640 1 -- 172.21.15.159:0/1949226707 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3324942972 0 0) 0x7fd44815fe80 con 0x7fd4480af830 2024-04-04T06:17:02.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.075+0000 7fd44d000640 1 --2- 172.21.15.159:0/1949226707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4480d0d60 0x7fd448073790 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fd440009d90 tx=0x7fd44002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e20a8452b9946c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:02.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.075+0000 7fd446a00640 1 -- 172.21.15.159:0/1949226707 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440039070 con 0x7fd4480d0d60 2024-04-04T06:17:02.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.075+0000 7fd446a00640 1 -- 172.21.15.159:0/1949226707 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd44002cc60 con 0x7fd4480d0d60 2024-04-04T06:17:02.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.075+0000 7fd446a00640 1 -- 172.21.15.159:0/1949226707 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440034660 con 0x7fd4480d0d60 2024-04-04T06:17:02.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.076+0000 7fd44f000640 1 -- 172.21.15.159:0/1949226707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4480d0d60 msgr2=0x7fd448073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:02.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.076+0000 7fd44f000640 1 --2- 172.21.15.159:0/1949226707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4480d0d60 0x7fd448073790 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fd440009d90 tx=0x7fd44002beb0 comp rx=0 tx=0).stop 2024-04-04T06:17:02.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.076+0000 7fd44f000640 1 -- 172.21.15.159:0/1949226707 shutdown_connections 2024-04-04T06:17:02.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.076+0000 7fd44f000640 1 --2- 172.21.15.159:0/1949226707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4480d0d60 0x7fd448073790 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:02.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.076+0000 7fd44f000640 1 -- 172.21.15.159:0/1949226707 >> 172.21.15.159:0/1949226707 conn(0x7fd44806b990 msgr2=0x7fd44806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:02.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.076+0000 7fd44f000640 1 -- 172.21.15.159:0/1949226707 shutdown_connections 2024-04-04T06:17:02.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.076+0000 7fd44f000640 1 -- 172.21.15.159:0/1949226707 wait complete. 2024-04-04T06:17:02.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.078+0000 7fd44f000640 1 Processor -- start 2024-04-04T06:17:02.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.078+0000 7fd44f000640 1 -- start start 2024-04-04T06:17:02.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.079+0000 7fd44f000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4480af830 0x7fd4480b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:02.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.079+0000 7fd44f000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4480d0d60 0x7fd44814e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:02.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.079+0000 7fd44f000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd44815c390 0x7fd4480b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:02.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.079+0000 7fd44f000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd448160d50 con 0x7fd4480af830 2024-04-04T06:17:02.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.079+0000 7fd44f000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd448160120 con 0x7fd4480d0d60 2024-04-04T06:17:02.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.079+0000 7fd44f000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd4480cf980 con 0x7fd44815c390 2024-04-04T06:17:02.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.079+0000 7fd447e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4480af830 0x7fd4480b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:02.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.079+0000 7fd447e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4480af830 0x7fd4480b1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44122/0 (socket says 172.21.15.159:44122) 2024-04-04T06:17:02.075 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.079+0000 7fd447e00640 1 -- 172.21.15.159:0/2996650482 learned_addr learned my addr 172.21.15.159:0/2996650482 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:02.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.080+0000 7fd447400640 1 --2- 172.21.15.159:0/2996650482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4480d0d60 0x7fd44814e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:02.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.080+0000 7fd44d000640 1 --2- 172.21.15.159:0/2996650482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd44815c390 0x7fd4480b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:02.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.080+0000 7fd447e00640 1 -- 172.21.15.159:0/2996650482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd44815c390 msgr2=0x7fd4480b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:02.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.080+0000 7fd447e00640 1 --2- 172.21.15.159:0/2996650482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd44815c390 0x7fd4480b44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:02.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.080+0000 7fd447e00640 1 -- 172.21.15.159:0/2996650482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4480d0d60 msgr2=0x7fd44814e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:02.076 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.080+0000 7fd447e00640 1 --2- 172.21.15.159:0/2996650482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4480d0d60 0x7fd44814e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:02.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.080+0000 7fd447e00640 1 -- 172.21.15.159:0/2996650482 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4480b0b10 con 0x7fd4480af830 2024-04-04T06:17:02.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.080+0000 7fd447400640 1 --2- 172.21.15.159:0/2996650482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4480d0d60 0x7fd44814e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:02.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.080+0000 7fd447e00640 1 --2- 172.21.15.159:0/2996650482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4480af830 0x7fd4480b1bb0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fd43800b3f0 tx=0x7fd43800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:02.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.081+0000 7fd43fe00640 1 -- 172.21.15.159:0/2996650482 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438015020 con 0x7fd4480af830 2024-04-04T06:17:02.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.081+0000 7fd43fe00640 1 -- 172.21.15.159:0/2996650482 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd438011830 con 0x7fd4480af830 2024-04-04T06:17:02.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.081+0000 7fd43fe00640 1 -- 172.21.15.159:0/2996650482 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438011c20 con 0x7fd4480af830 2024-04-04T06:17:02.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.081+0000 7fd44f000640 1 -- 172.21.15.159:0/2996650482 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4480b0df0 con 0x7fd4480af830 2024-04-04T06:17:02.077 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.081+0000 7fd44f000640 1 -- 172.21.15.159:0/2996650482 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4480b13b0 con 0x7fd4480af830 2024-04-04T06:17:02.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.082+0000 7fd44f000640 1 -- 172.21.15.159:0/2996650482 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4480685c0 con 0x7fd4480af830 2024-04-04T06:17:02.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.083+0000 7fd43fe00640 1 -- 172.21.15.159:0/2996650482 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd438018020 con 0x7fd4480af830 2024-04-04T06:17:02.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.084+0000 7fd43fe00640 1 --2- 172.21.15.159:0/2996650482 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd42406c920 0x7fd42406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:02.078 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.084+0000 7fd43fe00640 1 -- 172.21.15.159:0/2996650482 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7fd43808b730 con 0x7fd4480af830 2024-04-04T06:17:02.079 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.084+0000 7fd447400640 1 --2- 172.21.15.159:0/2996650482 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd42406c920 0x7fd42406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:02.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.085+0000 7fd447400640 1 --2- 172.21.15.159:0/2996650482 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd42406c920 0x7fd42406ede0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd430007fd0 tx=0x7fd430009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:02.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.093+0000 7fd43fe00640 1 -- 172.21.15.159:0/2996650482 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd43801f030 con 0x7fd4480af830 2024-04-04T06:17:02.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.291+0000 7fd44f000640 1 -- 172.21.15.159:0/2996650482 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd44806c130 con 0x7fd42406c920 2024-04-04T06:17:02.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.309+0000 7fd43fe00640 1 -- 172.21.15.159:0/2996650482 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248429 (secure 0 0 0) 0x7fd44806c130 con 0x7fd42406c920 2024-04-04T06:17:02.307 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:02.309 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:17:02.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.315+0000 7fd44f000640 1 -- 172.21.15.159:0/2996650482 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd42406c920 msgr2=0x7fd42406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:02.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.315+0000 7fd44f000640 1 --2- 172.21.15.159:0/2996650482 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd42406c920 0x7fd42406ede0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd430007fd0 tx=0x7fd430009040 comp rx=0 tx=0).stop 2024-04-04T06:17:02.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.315+0000 7fd44f000640 1 -- 172.21.15.159:0/2996650482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4480af830 msgr2=0x7fd4480b1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:02.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.315+0000 7fd44f000640 1 --2- 172.21.15.159:0/2996650482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4480af830 0x7fd4480b1bb0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fd43800b3f0 tx=0x7fd43800b8b0 comp rx=0 tx=0).stop 2024-04-04T06:17:02.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.315+0000 7fd44f000640 1 -- 172.21.15.159:0/2996650482 shutdown_connections 2024-04-04T06:17:02.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.315+0000 7fd44f000640 1 --2- 172.21.15.159:0/2996650482 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd44815c390 0x7fd4480b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:02.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.315+0000 7fd44f000640 1 --2- 172.21.15.159:0/2996650482 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd42406c920 0x7fd42406ede0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:02.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.315+0000 7fd44f000640 1 --2- 172.21.15.159:0/2996650482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4480d0d60 0x7fd44814e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:02.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.315+0000 7fd44f000640 1 --2- 172.21.15.159:0/2996650482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4480af830 0x7fd4480b1bb0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:02.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.315+0000 7fd44f000640 1 -- 172.21.15.159:0/2996650482 >> 172.21.15.159:0/2996650482 conn(0x7fd44806b990 msgr2=0x7fd448159980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:02.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.315+0000 7fd44f000640 1 -- 172.21.15.159:0/2996650482 shutdown_connections 2024-04-04T06:17:02.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.315+0000 7fd44f000640 1 -- 172.21.15.159:0/2996650482 wait complete. 2024-04-04T06:17:02.338 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-04-04T06:17:00.962376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122,"ondisk_log_size":122,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":277956,"kb_used_data":11864,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754696764,"statfs":{"total":773094113280,"available":772809486336,"internally_reserved":0,"allocated":12148736,"data_stored":5921784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.180982"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.795578+0000","last_change":"2024-04-04T06:16:56.795578+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.795578+0000","last_undegraded":"2024-04-04T06:16:56.795578+0000","last_fullsized":"2024-04-04T06:16:56.795578+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856681+0000","last_change":"2024-04-04T06:16:57.850062+0000","last_active":"2024-04-04T06:16:58.856681+0000","last_peered":"2024-04-04T06:16:58.856681+0000","last_clean":"2024-04-04T06:16:58.856681+0000","last_became_active":"2024-04-04T06:16:57.849768+0000","last_became_peered":"2024-04-04T06:16:57.849768+0000","last_unstale":"2024-04-04T06:16:58.856681+0000","last_undegraded":"2024-04-04T06:16:58.856681+0000","last_fullsized":"2024-04-04T06:16:58.856681+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:20:36.274557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.815390+0000","last_change":"2024-04-04T06:16:56.815390+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.815390+0000","last_undegraded":"2024-04-04T06:16:56.815390+0000","last_fullsized":"2024-04-04T06:16:56.815390+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.795247+0000","last_change":"2024-04-04T06:16:56.795247+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.795247+0000","last_undegraded":"2024-04-04T06:16:56.795247+0000","last_fullsized":"2024-04-04T06:16:56.795247+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.801944+0000","last_change":"2024-04-04T06:16:56.801944+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.801944+0000","last_undegraded":"2024-04-04T06:16:56.801944+0000","last_fullsized":"2024-04-04T06:16:56.801944+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.809513+0000","last_change":"2024-04-04T06:16:56.809513+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.809513+0000","last_undegraded":"2024-04-04T06:16:56.809513+0000","last_fullsized":"2024-04-04T06:16:56.809513+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855381+0000","last_change":"2024-04-04T06:16:57.853261+0000","last_active":"2024-04-04T06:16:58.855381+0000","last_peered":"2024-04-04T06:16:58.855381+0000","last_clean":"2024-04-04T06:16:58.855381+0000","last_became_active":"2024-04-04T06:16:57.852925+0000","last_became_peered":"2024-04-04T06:16:57.852925+0000","last_unstale":"2024-04-04T06:16:58.855381+0000","last_undegraded":"2024-04-04T06:16:58.855381+0000","last_fullsized":"2024-04-04T06:16:58.855381+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:43:19.761558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.807559+0000","last_change":"2024-04-04T06:16:56.807559+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.807559+0000","last_undegraded":"2024-04-04T06:16:56.807559+0000","last_fullsized":"2024-04-04T06:16:56.807559+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.794893+0000","last_change":"2024-04-04T06:16:56.794893+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.794893+0000","last_undegraded":"2024-04-04T06:16:56.794893+0000","last_fullsized":"2024-04-04T06:16:56.794893+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.794052+0000","last_change":"2024-04-04T06:16:56.794052+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.794052+0000","last_undegraded":"2024-04-04T06:16:56.794052+0000","last_fullsized":"2024-04-04T06:16:56.794052+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.857193+0000","last_change":"2024-04-04T06:16:57.852970+0000","last_active":"2024-04-04T06:16:58.857193+0000","last_peered":"2024-04-04T06:16:58.857193+0000","last_clean":"2024-04-04T06:16:58.857193+0000","last_became_active":"2024-04-04T06:16:57.852398+0000","last_became_peered":"2024-04-04T06:16:57.852398+0000","last_unstale":"2024-04-04T06:16:58.857193+0000","last_undegraded":"2024-04-04T06:16:58.857193+0000","last_fullsized":"2024-04-04T06:16:58.857193+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:25:13.863001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.794715+0000","last_change":"2024-04-04T06:16:56.794715+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.794715+0000","last_undegraded":"2024-04-04T06:16:56.794715+0000","last_fullsized":"2024-04-04T06:16:56.794715+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.815700+0000","last_change":"2024-04-04T06:16:56.815700+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.815700+0000","last_undegraded":"2024-04-04T06:16:56.815700+0000","last_fullsized":"2024-04-04T06:16:56.815700+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.809448+0000","last_change":"2024-04-04T06:16:56.809448+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.809448+0000","last_undegraded":"2024-04-04T06:16:56.809448+0000","last_fullsized":"2024-04-04T06:16:56.809448+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.809974+0000","last_change":"2024-04-04T06:16:56.809974+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.809974+0000","last_undegraded":"2024-04-04T06:16:56.809974+0000","last_fullsized":"2024-04-04T06:16:56.809974+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.797477+0000","last_change":"2024-04-04T06:16:56.797477+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.797477+0000","last_undegraded":"2024-04-04T06:16:56.797477+0000","last_fullsized":"2024-04-04T06:16:56.797477+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856787+0000","last_change":"2024-04-04T06:16:57.861016+0000","last_active":"2024-04-04T06:16:58.856787+0000","last_peered":"2024-04-04T06:16:58.856787+0000","last_clean":"2024-04-04T06:16:58.856787+0000","last_became_active":"2024-04-04T06:16:57.860213+0000","last_became_peered":"2024-04-04T06:16:57.860213+0000","last_unstale":"2024-04-04T06:16:58.856787+0000","last_undegraded":"2024-04-04T06:16:58.856787+0000","last_fullsized":"2024-04-04T06:16:58.856787+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:41:22.250390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.796165+0000","last_change":"2024-04-04T06:16:56.796165+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.796165+0000","last_undegraded":"2024-04-04T06:16:56.796165+0000","last_fullsized":"2024-04-04T06:16:56.796165+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.814584+0000","last_change":"2024-04-04T06:16:56.814584+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.814584+0000","last_undegraded":"2024-04-04T06:16:56.814584+0000","last_fullsized":"2024-04-04T06:16:56.814584+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856739+0000","last_change":"2024-04-04T06:16:57.858770+0000","last_active":"2024-04-04T06:16:58.856739+0000","last_peered":"2024-04-04T06:16:58.856739+0000","last_clean":"2024-04-04T06:16:58.856739+0000","last_became_active":"2024-04-04T06:16:57.856816+0000","last_became_peered":"2024-04-04T06:16:57.856816+0000","last_unstale":"2024-04-04T06:16:58.856739+0000","last_undegraded":"2024-04-04T06:16:58.856739+0000","last_fullsized":"2024-04-04T06:16:58.856739+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:42:54.207457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.857204+0000","last_change":"2024-04-04T06:16:57.840896+0000","last_active":"2024-04-04T06:16:58.857204+0000","last_peered":"2024-04-04T06:16:58.857204+0000","last_clean":"2024-04-04T06:16:58.857204+0000","last_became_active":"2024-04-04T06:16:57.840145+0000","last_became_peered":"2024-04-04T06:16:57.840145+0000","last_unstale":"2024-04-04T06:16:58.857204+0000","last_undegraded":"2024-04-04T06:16:58.857204+0000","last_fullsized":"2024-04-04T06:16:58.857204+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:56:07.453527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.817019+0000","last_change":"2024-04-04T06:16:56.817019+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.817019+0000","last_undegraded":"2024-04-04T06:16:56.817019+0000","last_fullsized":"2024-04-04T06:16:56.817019+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855555+0000","last_change":"2024-04-04T06:16:57.855453+0000","last_active":"2024-04-04T06:16:58.855555+0000","last_peered":"2024-04-04T06:16:58.855555+0000","last_clean":"2024-04-04T06:16:58.855555+0000","last_became_active":"2024-04-04T06:16:57.855172+0000","last_became_peered":"2024-04-04T06:16:57.855172+0000","last_unstale":"2024-04-04T06:16:58.855555+0000","last_undegraded":"2024-04-04T06:16:58.855555+0000","last_fullsized":"2024-04-04T06:16:58.855555+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:17:03.994940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.816316+0000","last_change":"2024-04-04T06:16:56.816316+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.816316+0000","last_undegraded":"2024-04-04T06:16:56.816316+0000","last_fullsized":"2024-04-04T06:16:56.816316+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.857033+0000","last_change":"2024-04-04T06:16:57.859703+0000","last_active":"2024-04-04T06:16:58.857033+0000","last_peered":"2024-04-04T06:16:58.857033+0000","last_clean":"2024-04-04T06:16:58.857033+0000","last_became_active":"2024-04-04T06:16:57.859398+0000","last_became_peered":"2024-04-04T06:16:57.859398+0000","last_unstale":"2024-04-04T06:16:58.857033+0000","last_undegraded":"2024-04-04T06:16:58.857033+0000","last_fullsized":"2024-04-04T06:16:58.857033+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:29:25.780543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.793316+0000","last_change":"2024-04-04T06:16:56.793316+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.793316+0000","last_undegraded":"2024-04-04T06:16:56.793316+0000","last_fullsized":"2024-04-04T06:16:56.793316+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.810088+0000","last_change":"2024-04-04T06:16:56.810088+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.810088+0000","last_undegraded":"2024-04-04T06:16:56.810088+0000","last_fullsized":"2024-04-04T06:16:56.810088+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.815732+0000","last_change":"2024-04-04T06:16:56.815732+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.815732+0000","last_undegraded":"2024-04-04T06:16:56.815732+0000","last_fullsized":"2024-04-04T06:16:56.815732+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856807+0000","last_change":"2024-04-04T06:16:57.859118+0000","last_active":"2024-04-04T06:16:58.856807+0000","last_peered":"2024-04-04T06:16:58.856807+0000","last_clean":"2024-04-04T06:16:58.856807+0000","last_became_active":"2024-04-04T06:16:57.858242+0000","last_became_peered":"2024-04-04T06:16:57.858242+0000","last_unstale":"2024-04-04T06:16:58.856807+0000","last_undegraded":"2024-04-04T06:16:58.856807+0000","last_fullsized":"2024-04-04T06:16:58.856807+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:16:58.119204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855126+0000","last_change":"2024-04-04T06:16:57.838809+0000","last_active":"2024-04-04T06:16:58.855126+0000","last_peered":"2024-04-04T06:16:58.855126+0000","last_clean":"2024-04-04T06:16:58.855126+0000","last_became_active":"2024-04-04T06:16:57.837497+0000","last_became_peered":"2024-04-04T06:16:57.837497+0000","last_unstale":"2024-04-04T06:16:58.855126+0000","last_undegraded":"2024-04-04T06:16:58.855126+0000","last_fullsized":"2024-04-04T06:16:58.855126+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:05:00.308975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.796582+0000","last_change":"2024-04-04T06:16:56.796582+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.796582+0000","last_undegraded":"2024-04-04T06:16:56.796582+0000","last_fullsized":"2024-04-04T06:16:56.796582+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.808526+0000","last_change":"2024-04-04T06:16:56.808526+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.808526+0000","last_undegraded":"2024-04-04T06:16:56.808526+0000","last_fullsized":"2024-04-04T06:16:56.808526+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.808552+0000","last_change":"2024-04-04T06:16:56.808552+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.808552+0000","last_undegraded":"2024-04-04T06:16:56.808552+0000","last_fullsized":"2024-04-04T06:16:56.808552+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"112'4","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855930+0000","last_change":"2024-04-04T06:16:56.826026+0000","last_active":"2024-04-04T06:16:58.855930+0000","last_peered":"2024-04-04T06:16:58.855930+0000","last_clean":"2024-04-04T06:16:58.855930+0000","last_became_active":"2024-04-04T06:16:56.825749+0000","last_became_peered":"2024-04-04T06:16:56.825749+0000","last_unstale":"2024-04-04T06:16:58.855930+0000","last_undegraded":"2024-04-04T06:16:58.855930+0000","last_fullsized":"2024-04-04T06:16:58.855930+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:44:30.609701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.796084+0000","last_change":"2024-04-04T06:16:56.796084+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.796084+0000","last_undegraded":"2024-04-04T06:16:56.796084+0000","last_fullsized":"2024-04-04T06:16:56.796084+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.829285+0000","last_change":"2024-04-04T06:16:56.828520+0000","last_active":"2024-04-04T06:16:56.829285+0000","last_peered":"2024-04-04T06:16:56.829285+0000","last_clean":"2024-04-04T06:16:56.829285+0000","last_became_active":"2024-04-04T06:16:56.828125+0000","last_became_peered":"2024-04-04T06:16:56.828125+0000","last_unstale":"2024-04-04T06:16:56.829285+0000","last_undegraded":"2024-04-04T06:16:56.829285+0000","last_fullsized":"2024-04-04T06:16:56.829285+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:04:31.414046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.807591+0000","last_change":"2024-04-04T06:16:56.807591+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.807591+0000","last_undegraded":"2024-04-04T06:16:56.807591+0000","last_fullsized":"2024-04-04T06:16:56.807591+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.823269+0000","last_change":"2024-04-04T06:16:56.817318+0000","last_active":"2024-04-04T06:16:56.823269+0000","last_peered":"2024-04-04T06:16:56.823269+0000","last_clean":"2024-04-04T06:16:56.823269+0000","last_became_active":"2024-04-04T06:16:56.816995+0000","last_became_peered":"2024-04-04T06:16:56.816995+0000","last_unstale":"2024-04-04T06:16:56.823269+0000","last_undegraded":"2024-04-04T06:16:56.823269+0000","last_fullsized":"2024-04-04T06:16:56.823269+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:02:04.420894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.813683+0000","last_change":"2024-04-04T06:16:56.813683+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.813683+0000","last_undegraded":"2024-04-04T06:16:56.813683+0000","last_fullsized":"2024-04-04T06:16:56.813683+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.823058+0000","last_change":"2024-04-04T06:16:56.813821+0000","last_active":"2024-04-04T06:16:56.823058+0000","last_peered":"2024-04-04T06:16:56.823058+0000","last_clean":"2024-04-04T06:16:56.823058+0000","last_became_active":"2024-04-04T06:16:56.813555+0000","last_became_peered":"2024-04-04T06:16:56.813555+0000","last_unstale":"2024-04-04T06:16:56.823058+0000","last_undegraded":"2024-04-04T06:16:56.823058+0000","last_fullsized":"2024-04-04T06:16:56.823058+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:52:53.198860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.795444+0000","last_change":"2024-04-04T06:16:56.795444+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.795444+0000","last_undegraded":"2024-04-04T06:16:56.795444+0000","last_fullsized":"2024-04-04T06:16:56.795444+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854845+0000","last_change":"2024-04-04T06:16:56.813703+0000","last_active":"2024-04-04T06:16:58.854845+0000","last_peered":"2024-04-04T06:16:58.854845+0000","last_clean":"2024-04-04T06:16:58.854845+0000","last_became_active":"2024-04-04T06:16:56.813224+0000","last_became_peered":"2024-04-04T06:16:56.813224+0000","last_unstale":"2024-04-04T06:16:58.854845+0000","last_undegraded":"2024-04-04T06:16:58.854845+0000","last_fullsized":"2024-04-04T06:16:58.854845+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:28:44.071981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856364+0000","last_change":"2024-04-04T06:16:57.851938+0000","last_active":"2024-04-04T06:16:58.856364+0000","last_peered":"2024-04-04T06:16:58.856364+0000","last_clean":"2024-04-04T06:16:58.856364+0000","last_became_active":"2024-04-04T06:16:57.850346+0000","last_became_peered":"2024-04-04T06:16:57.850346+0000","last_unstale":"2024-04-04T06:16:58.856364+0000","last_undegraded":"2024-04-04T06:16:58.856364+0000","last_fullsized":"2024-04-04T06:16:58.856364+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:42:36.526974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.824886+0000","last_change":"2024-04-04T06:16:56.822599+0000","last_active":"2024-04-04T06:16:56.824886+0000","last_peered":"2024-04-04T06:16:56.824886+0000","last_clean":"2024-04-04T06:16:56.824886+0000","last_became_active":"2024-04-04T06:16:56.822130+0000","last_became_peered":"2024-04-04T06:16:56.822130+0000","last_unstale":"2024-04-04T06:16:56.824886+0000","last_undegraded":"2024-04-04T06:16:56.824886+0000","last_fullsized":"2024-04-04T06:16:56.824886+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:15:58.869588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.796987+0000","last_change":"2024-04-04T06:16:56.796987+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.796987+0000","last_undegraded":"2024-04-04T06:16:56.796987+0000","last_fullsized":"2024-04-04T06:16:56.796987+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855249+0000","last_change":"2024-04-04T06:16:56.822615+0000","last_active":"2024-04-04T06:16:58.855249+0000","last_peered":"2024-04-04T06:16:58.855249+0000","last_clean":"2024-04-04T06:16:58.855249+0000","last_became_active":"2024-04-04T06:16:56.819761+0000","last_became_peered":"2024-04-04T06:16:56.819761+0000","last_unstale":"2024-04-04T06:16:58.855249+0000","last_undegraded":"2024-04-04T06:16:58.855249+0000","last_fullsized":"2024-04-04T06:16:58.855249+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:10:06.219773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854643+0000","last_change":"2024-04-04T06:16:57.853254+0000","last_active":"2024-04-04T06:16:58.854643+0000","last_peered":"2024-04-04T06:16:58.854643+0000","last_clean":"2024-04-04T06:16:58.854643+0000","last_became_active":"2024-04-04T06:16:57.852500+0000","last_became_peered":"2024-04-04T06:16:57.852500+0000","last_unstale":"2024-04-04T06:16:58.854643+0000","last_undegraded":"2024-04-04T06:16:58.854643+0000","last_fullsized":"2024-04-04T06:16:58.854643+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:47.042156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.822886+0000","last_change":"2024-04-04T06:16:56.816173+0000","last_active":"2024-04-04T06:16:56.822886+0000","last_peered":"2024-04-04T06:16:56.822886+0000","last_clean":"2024-04-04T06:16:56.822886+0000","last_became_active":"2024-04-04T06:16:56.815796+0000","last_became_peered":"2024-04-04T06:16:56.815796+0000","last_unstale":"2024-04-04T06:16:56.822886+0000","last_undegraded":"2024-04-04T06:16:56.822886+0000","last_fullsized":"2024-04-04T06:16:56.822886+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:46:18.307967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856214+0000","last_change":"2024-04-04T06:16:57.863523+0000","last_active":"2024-04-04T06:16:58.856214+0000","last_peered":"2024-04-04T06:16:58.856214+0000","last_clean":"2024-04-04T06:16:58.856214+0000","last_became_active":"2024-04-04T06:16:57.861142+0000","last_became_peered":"2024-04-04T06:16:57.861142+0000","last_unstale":"2024-04-04T06:16:58.856214+0000","last_undegraded":"2024-04-04T06:16:58.856214+0000","last_fullsized":"2024-04-04T06:16:58.856214+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:50:12.226068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.823105+0000","last_change":"2024-04-04T06:16:56.817070+0000","last_active":"2024-04-04T06:16:56.823105+0000","last_peered":"2024-04-04T06:16:56.823105+0000","last_clean":"2024-04-04T06:16:56.823105+0000","last_became_active":"2024-04-04T06:16:56.816742+0000","last_became_peered":"2024-04-04T06:16:56.816742+0000","last_unstale":"2024-04-04T06:16:56.823105+0000","last_undegraded":"2024-04-04T06:16:56.823105+0000","last_fullsized":"2024-04-04T06:16:56.823105+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:09:45.329512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.794919+0000","last_change":"2024-04-04T06:16:56.794919+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.794919+0000","last_undegraded":"2024-04-04T06:16:56.794919+0000","last_fullsized":"2024-04-04T06:16:56.794919+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.824720+0000","last_change":"2024-04-04T06:16:56.822568+0000","last_active":"2024-04-04T06:16:56.824720+0000","last_peered":"2024-04-04T06:16:56.824720+0000","last_clean":"2024-04-04T06:16:56.824720+0000","last_became_active":"2024-04-04T06:16:56.822237+0000","last_became_peered":"2024-04-04T06:16:56.822237+0000","last_unstale":"2024-04-04T06:16:56.824720+0000","last_undegraded":"2024-04-04T06:16:56.824720+0000","last_fullsized":"2024-04-04T06:16:56.824720+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:50:55.464717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.815062+0000","last_change":"2024-04-04T06:16:56.815062+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.815062+0000","last_undegraded":"2024-04-04T06:16:56.815062+0000","last_fullsized":"2024-04-04T06:16:56.815062+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.814979+0000","last_change":"2024-04-04T06:16:56.811481+0000","last_active":"2024-04-04T06:16:56.814979+0000","last_peered":"2024-04-04T06:16:56.814979+0000","last_clean":"2024-04-04T06:16:56.814979+0000","last_became_active":"2024-04-04T06:16:56.811190+0000","last_became_peered":"2024-04-04T06:16:56.811190+0000","last_unstale":"2024-04-04T06:16:56.814979+0000","last_undegraded":"2024-04-04T06:16:56.814979+0000","last_fullsized":"2024-04-04T06:16:56.814979+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:41:19.705887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855312+0000","last_change":"2024-04-04T06:16:57.855846+0000","last_active":"2024-04-04T06:16:58.855312+0000","last_peered":"2024-04-04T06:16:58.855312+0000","last_clean":"2024-04-04T06:16:58.855312+0000","last_became_active":"2024-04-04T06:16:57.855487+0000","last_became_peered":"2024-04-04T06:16:57.855487+0000","last_unstale":"2024-04-04T06:16:58.855312+0000","last_undegraded":"2024-04-04T06:16:58.855312+0000","last_fullsized":"2024-04-04T06:16:58.855312+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:25:12.522619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.822910+0000","last_change":"2024-04-04T06:16:56.816249+0000","last_active":"2024-04-04T06:16:56.822910+0000","last_peered":"2024-04-04T06:16:56.822910+0000","last_clean":"2024-04-04T06:16:56.822910+0000","last_became_active":"2024-04-04T06:16:56.815980+0000","last_became_peered":"2024-04-04T06:16:56.815980+0000","last_unstale":"2024-04-04T06:16:56.822910+0000","last_undegraded":"2024-04-04T06:16:56.822910+0000","last_fullsized":"2024-04-04T06:16:56.822910+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:38:39.241002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.811065+0000","last_change":"2024-04-04T06:16:56.811065+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.811065+0000","last_undegraded":"2024-04-04T06:16:56.811065+0000","last_fullsized":"2024-04-04T06:16:56.811065+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.826345+0000","last_change":"2024-04-04T06:16:56.825811+0000","last_active":"2024-04-04T06:16:56.826345+0000","last_peered":"2024-04-04T06:16:56.826345+0000","last_clean":"2024-04-04T06:16:56.826345+0000","last_became_active":"2024-04-04T06:16:56.825491+0000","last_became_peered":"2024-04-04T06:16:56.825491+0000","last_unstale":"2024-04-04T06:16:56.826345+0000","last_undegraded":"2024-04-04T06:16:56.826345+0000","last_fullsized":"2024-04-04T06:16:56.826345+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:28.483864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.812435+0000","last_change":"2024-04-04T06:16:56.812435+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.812435+0000","last_undegraded":"2024-04-04T06:16:56.812435+0000","last_fullsized":"2024-04-04T06:16:56.812435+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.814395+0000","last_change":"2024-04-04T06:16:56.808070+0000","last_active":"2024-04-04T06:16:56.814395+0000","last_peered":"2024-04-04T06:16:56.814395+0000","last_clean":"2024-04-04T06:16:56.814395+0000","last_became_active":"2024-04-04T06:16:56.807074+0000","last_became_peered":"2024-04-04T06:16:56.807074+0000","last_unstale":"2024-04-04T06:16:56.814395+0000","last_undegraded":"2024-04-04T06:16:56.814395+0000","last_fullsized":"2024-04-04T06:16:56.814395+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:12:24.143284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.809129+0000","last_change":"2024-04-04T06:16:56.809129+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.809129+0000","last_undegraded":"2024-04-04T06:16:56.809129+0000","last_fullsized":"2024-04-04T06:16:56.809129+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.823004+0000","last_change":"2024-04-04T06:16:56.814462+0000","last_active":"2024-04-04T06:16:56.823004+0000","last_peered":"2024-04-04T06:16:56.823004+0000","last_clean":"2024-04-04T06:16:56.823004+0000","last_became_active":"2024-04-04T06:16:56.813042+0000","last_became_peered":"2024-04-04T06:16:56.813042+0000","last_unstale":"2024-04-04T06:16:56.823004+0000","last_undegraded":"2024-04-04T06:16:56.823004+0000","last_fullsized":"2024-04-04T06:16:56.823004+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:23:44.093177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.802313+0000","last_change":"2024-04-04T06:16:56.802313+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.802313+0000","last_undegraded":"2024-04-04T06:16:56.802313+0000","last_fullsized":"2024-04-04T06:16:56.802313+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.818315+0000","last_change":"2024-04-04T06:16:56.812543+0000","last_active":"2024-04-04T06:16:56.818315+0000","last_peered":"2024-04-04T06:16:56.818315+0000","last_clean":"2024-04-04T06:16:56.818315+0000","last_became_active":"2024-04-04T06:16:56.812240+0000","last_became_peered":"2024-04-04T06:16:56.812240+0000","last_unstale":"2024-04-04T06:16:56.818315+0000","last_undegraded":"2024-04-04T06:16:56.818315+0000","last_fullsized":"2024-04-04T06:16:56.818315+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:49:28.135341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.808083+0000","last_change":"2024-04-04T06:16:56.808083+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.808083+0000","last_undegraded":"2024-04-04T06:16:56.808083+0000","last_fullsized":"2024-04-04T06:16:56.808083+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.824636+0000","last_change":"2024-04-04T06:16:56.822207+0000","last_active":"2024-04-04T06:16:56.824636+0000","last_peered":"2024-04-04T06:16:56.824636+0000","last_clean":"2024-04-04T06:16:56.824636+0000","last_became_active":"2024-04-04T06:16:56.821906+0000","last_became_peered":"2024-04-04T06:16:56.821906+0000","last_unstale":"2024-04-04T06:16:56.824636+0000","last_undegraded":"2024-04-04T06:16:56.824636+0000","last_fullsized":"2024-04-04T06:16:56.824636+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:28:49.663613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.814086+0000","last_change":"2024-04-04T06:16:56.814086+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.814086+0000","last_undegraded":"2024-04-04T06:16:56.814086+0000","last_fullsized":"2024-04-04T06:16:56.814086+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.825760+0000","last_change":"2024-04-04T06:16:56.818843+0000","last_active":"2024-04-04T06:16:56.825760+0000","last_peered":"2024-04-04T06:16:56.825760+0000","last_clean":"2024-04-04T06:16:56.825760+0000","last_became_active":"2024-04-04T06:16:56.818532+0000","last_became_peered":"2024-04-04T06:16:56.818532+0000","last_unstale":"2024-04-04T06:16:56.825760+0000","last_undegraded":"2024-04-04T06:16:56.825760+0000","last_fullsized":"2024-04-04T06:16:56.825760+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:28:13.269672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.811863+0000","last_change":"2024-04-04T06:16:56.811863+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.811863+0000","last_undegraded":"2024-04-04T06:16:56.811863+0000","last_fullsized":"2024-04-04T06:16:56.811863+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854548+0000","last_change":"2024-04-04T06:16:56.817026+0000","last_active":"2024-04-04T06:16:58.854548+0000","last_peered":"2024-04-04T06:16:58.854548+0000","last_clean":"2024-04-04T06:16:58.854548+0000","last_became_active":"2024-04-04T06:16:56.816488+0000","last_became_peered":"2024-04-04T06:16:56.816488+0000","last_unstale":"2024-04-04T06:16:58.854548+0000","last_undegraded":"2024-04-04T06:16:58.854548+0000","last_fullsized":"2024-04-04T06:16:58.854548+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:11:49.042689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.808636+0000","last_change":"2024-04-04T06:16:56.808636+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.808636+0000","last_undegraded":"2024-04-04T06:16:56.808636+0000","last_fullsized":"2024-04-04T06:16:56.808636+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.824529+0000","last_change":"2024-04-04T06:16:56.821203+0000","last_active":"2024-04-04T06:16:56.824529+0000","last_peered":"2024-04-04T06:16:56.824529+0000","last_clean":"2024-04-04T06:16:56.824529+0000","last_became_active":"2024-04-04T06:16:56.820739+0000","last_became_peered":"2024-04-04T06:16:56.820739+0000","last_unstale":"2024-04-04T06:16:56.824529+0000","last_undegraded":"2024-04-04T06:16:56.824529+0000","last_fullsized":"2024-04-04T06:16:56.824529+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:06:13.571199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.795568+0000","last_change":"2024-04-04T06:16:56.795568+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.795568+0000","last_undegraded":"2024-04-04T06:16:56.795568+0000","last_fullsized":"2024-04-04T06:16:56.795568+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.822827+0000","last_change":"2024-04-04T06:16:56.816260+0000","last_active":"2024-04-04T06:16:56.822827+0000","last_peered":"2024-04-04T06:16:56.822827+0000","last_clean":"2024-04-04T06:16:56.822827+0000","last_became_active":"2024-04-04T06:16:56.815988+0000","last_became_peered":"2024-04-04T06:16:56.815988+0000","last_unstale":"2024-04-04T06:16:56.822827+0000","last_undegraded":"2024-04-04T06:16:56.822827+0000","last_fullsized":"2024-04-04T06:16:56.822827+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:11:45.216267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.806526+0000","last_change":"2024-04-04T06:16:56.806526+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.806526+0000","last_undegraded":"2024-04-04T06:16:56.806526+0000","last_fullsized":"2024-04-04T06:16:56.806526+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.824945+0000","last_change":"2024-04-04T06:16:56.823585+0000","last_active":"2024-04-04T06:16:56.824945+0000","last_peered":"2024-04-04T06:16:56.824945+0000","last_clean":"2024-04-04T06:16:56.824945+0000","last_became_active":"2024-04-04T06:16:56.823294+0000","last_became_peered":"2024-04-04T06:16:56.823294+0000","last_unstale":"2024-04-04T06:16:56.824945+0000","last_undegraded":"2024-04-04T06:16:56.824945+0000","last_fullsized":"2024-04-04T06:16:56.824945+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:28:49.663613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.805594+0000","last_change":"2024-04-04T06:16:56.805594+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.805594+0000","last_undegraded":"2024-04-04T06:16:56.805594+0000","last_fullsized":"2024-04-04T06:16:56.805594+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.810421+0000","last_change":"2024-04-04T06:16:56.806783+0000","last_active":"2024-04-04T06:16:56.810421+0000","last_peered":"2024-04-04T06:16:56.810421+0000","last_clean":"2024-04-04T06:16:56.810421+0000","last_became_active":"2024-04-04T06:16:56.806293+0000","last_became_peered":"2024-04-04T06:16:56.806293+0000","last_unstale":"2024-04-04T06:16:56.810421+0000","last_undegraded":"2024-04-04T06:16:56.810421+0000","last_fullsized":"2024-04-04T06:16:56.810421+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:37:39.827821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.794903+0000","last_change":"2024-04-04T06:16:56.794903+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.794903+0000","last_undegraded":"2024-04-04T06:16:56.794903+0000","last_fullsized":"2024-04-04T06:16:56.794903+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.822668+0000","last_change":"2024-04-04T06:16:56.817822+0000","last_active":"2024-04-04T06:16:56.822668+0000","last_peered":"2024-04-04T06:16:56.822668+0000","last_clean":"2024-04-04T06:16:56.822668+0000","last_became_active":"2024-04-04T06:16:56.816539+0000","last_became_peered":"2024-04-04T06:16:56.816539+0000","last_unstale":"2024-04-04T06:16:56.822668+0000","last_undegraded":"2024-04-04T06:16:56.822668+0000","last_fullsized":"2024-04-04T06:16:56.822668+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:40:02.467791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.794244+0000","last_change":"2024-04-04T06:16:56.794244+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.794244+0000","last_undegraded":"2024-04-04T06:16:56.794244+0000","last_fullsized":"2024-04-04T06:16:56.794244+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.825725+0000","last_change":"2024-04-04T06:16:56.819463+0000","last_active":"2024-04-04T06:16:56.825725+0000","last_peered":"2024-04-04T06:16:56.825725+0000","last_clean":"2024-04-04T06:16:56.825725+0000","last_became_active":"2024-04-04T06:16:56.816034+0000","last_became_peered":"2024-04-04T06:16:56.816034+0000","last_unstale":"2024-04-04T06:16:56.825725+0000","last_undegraded":"2024-04-04T06:16:56.825725+0000","last_fullsized":"2024-04-04T06:16:56.825725+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:13:18.335553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.793602+0000","last_change":"2024-04-04T06:16:56.793602+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.793602+0000","last_undegraded":"2024-04-04T06:16:56.793602+0000","last_fullsized":"2024-04-04T06:16:56.793602+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.805734+0000","last_change":"2024-04-04T06:16:56.803667+0000","last_active":"2024-04-04T06:16:56.805734+0000","last_peered":"2024-04-04T06:16:56.805734+0000","last_clean":"2024-04-04T06:16:56.805734+0000","last_became_active":"2024-04-04T06:16:56.803353+0000","last_became_peered":"2024-04-04T06:16:56.803353+0000","last_unstale":"2024-04-04T06:16:56.805734+0000","last_undegraded":"2024-04-04T06:16:56.805734+0000","last_fullsized":"2024-04-04T06:16:56.805734+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:12:11.893578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.795146+0000","last_change":"2024-04-04T06:16:56.795146+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.795146+0000","last_undegraded":"2024-04-04T06:16:56.795146+0000","last_fullsized":"2024-04-04T06:16:56.795146+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.823935+0000","last_change":"2024-04-04T06:16:56.820108+0000","last_active":"2024-04-04T06:16:56.823935+0000","last_peered":"2024-04-04T06:16:56.823935+0000","last_clean":"2024-04-04T06:16:56.823935+0000","last_became_active":"2024-04-04T06:16:56.819768+0000","last_became_peered":"2024-04-04T06:16:56.819768+0000","last_unstale":"2024-04-04T06:16:56.823935+0000","last_undegraded":"2024-04-04T06:16:56.823935+0000","last_fullsized":"2024-04-04T06:16:56.823935+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:44:04.731918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855977+0000","last_change":"2024-04-04T06:16:57.844825+0000","last_active":"2024-04-04T06:16:58.855977+0000","last_peered":"2024-04-04T06:16:58.855977+0000","last_clean":"2024-04-04T06:16:58.855977+0000","last_became_active":"2024-04-04T06:16:57.843101+0000","last_became_peered":"2024-04-04T06:16:57.843101+0000","last_unstale":"2024-04-04T06:16:58.855977+0000","last_undegraded":"2024-04-04T06:16:58.855977+0000","last_fullsized":"2024-04-04T06:16:58.855977+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:41:12.600534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854882+0000","last_change":"2024-04-04T06:16:56.823376+0000","last_active":"2024-04-04T06:16:58.854882+0000","last_peered":"2024-04-04T06:16:58.854882+0000","last_clean":"2024-04-04T06:16:58.854882+0000","last_became_active":"2024-04-04T06:16:56.822325+0000","last_became_peered":"2024-04-04T06:16:56.822325+0000","last_unstale":"2024-04-04T06:16:58.854882+0000","last_undegraded":"2024-04-04T06:16:58.854882+0000","last_fullsized":"2024-04-04T06:16:58.854882+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:17:31.397079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856868+0000","last_change":"2024-04-04T06:16:57.861595+0000","last_active":"2024-04-04T06:16:58.856868+0000","last_peered":"2024-04-04T06:16:58.856868+0000","last_clean":"2024-04-04T06:16:58.856868+0000","last_became_active":"2024-04-04T06:16:57.860975+0000","last_became_peered":"2024-04-04T06:16:57.860975+0000","last_unstale":"2024-04-04T06:16:58.856868+0000","last_undegraded":"2024-04-04T06:16:58.856868+0000","last_fullsized":"2024-04-04T06:16:58.856868+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:29:44.470791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855757+0000","last_change":"2024-04-04T06:16:56.822556+0000","last_active":"2024-04-04T06:16:58.855757+0000","last_peered":"2024-04-04T06:16:58.855757+0000","last_clean":"2024-04-04T06:16:58.855757+0000","last_became_active":"2024-04-04T06:16:56.821979+0000","last_became_peered":"2024-04-04T06:16:56.821979+0000","last_unstale":"2024-04-04T06:16:58.855757+0000","last_undegraded":"2024-04-04T06:16:58.855757+0000","last_fullsized":"2024-04-04T06:16:58.855757+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:59:42.200129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.805318+0000","last_change":"2024-04-04T06:16:56.805318+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.805318+0000","last_undegraded":"2024-04-04T06:16:56.805318+0000","last_fullsized":"2024-04-04T06:16:56.805318+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.824612+0000","last_change":"2024-04-04T06:16:56.818500+0000","last_active":"2024-04-04T06:16:56.824612+0000","last_peered":"2024-04-04T06:16:56.824612+0000","last_clean":"2024-04-04T06:16:56.824612+0000","last_became_active":"2024-04-04T06:16:56.816829+0000","last_became_peered":"2024-04-04T06:16:56.816829+0000","last_unstale":"2024-04-04T06:16:56.824612+0000","last_undegraded":"2024-04-04T06:16:56.824612+0000","last_fullsized":"2024-04-04T06:16:56.824612+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:06:13.571199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.793318+0000","last_change":"2024-04-04T06:16:56.793318+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.793318+0000","last_undegraded":"2024-04-04T06:16:56.793318+0000","last_fullsized":"2024-04-04T06:16:56.793318+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.810449+0000","last_change":"2024-04-04T06:16:56.807000+0000","last_active":"2024-04-04T06:16:56.810449+0000","last_peered":"2024-04-04T06:16:56.810449+0000","last_clean":"2024-04-04T06:16:56.810449+0000","last_became_active":"2024-04-04T06:16:56.806687+0000","last_became_peered":"2024-04-04T06:16:56.806687+0000","last_unstale":"2024-04-04T06:16:56.810449+0000","last_undegraded":"2024-04-04T06:16:56.810449+0000","last_fullsized":"2024-04-04T06:16:56.810449+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:45:42.675350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.806584+0000","last_change":"2024-04-04T06:16:56.806584+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.806584+0000","last_undegraded":"2024-04-04T06:16:56.806584+0000","last_fullsized":"2024-04-04T06:16:56.806584+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854824+0000","last_change":"2024-04-04T06:16:56.819104+0000","last_active":"2024-04-04T06:16:58.854824+0000","last_peered":"2024-04-04T06:16:58.854824+0000","last_clean":"2024-04-04T06:16:58.854824+0000","last_became_active":"2024-04-04T06:16:56.818492+0000","last_became_peered":"2024-04-04T06:16:56.818492+0000","last_unstale":"2024-04-04T06:16:58.854824+0000","last_undegraded":"2024-04-04T06:16:58.854824+0000","last_fullsized":"2024-04-04T06:16:58.854824+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:14:15.299577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":334,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.819399+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:16:56.819399+0000","last_peered":"2024-04-04T06:16:56.819399+0000","last_clean":"2024-04-04T06:16:56.819399+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:16:56.819399+0000","last_undegraded":"2024-04-04T06:16:56.819399+0000","last_fullsized":"2024-04-04T06:16:56.819399+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509577,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37248,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334592,"statfs":{"total":96636764160,"available":96598622208,"internally_reserved":0,"allocated":1953792,"data_stored":1178086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509579,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36668,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335172,"statfs":{"total":96636764160,"available":96599216128,"internally_reserved":0,"allocated":1359872,"data_stored":587718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509580,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1359872,"data_stored":587718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509579,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1359872,"data_stored":587718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509578,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1359872,"data_stored":587718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509580,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1953792,"data_stored":1178086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36712,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335128,"statfs":{"total":96636764160,"available":96599171072,"internally_reserved":0,"allocated":1404928,"data_stored":607672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542282,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36712,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335128,"statfs":{"total":96636764160,"available":96599171072,"internally_reserved":0,"allocated":1396736,"data_stored":607068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:17:02.343 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:17:02.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.591+0000 7fb36f200640 1 Processor -- start 2024-04-04T06:17:02.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.591+0000 7fb36f200640 1 -- start start 2024-04-04T06:17:02.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.593+0000 7fb36f200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb36815c390 0x7fb368073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:02.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.593+0000 7fb36f200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb3680d0650 0x7fb3680d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:02.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.593+0000 7fb36f200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb36815fe80 con 0x7fb3680af830 2024-04-04T06:17:02.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.593+0000 7fb36f200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fb36815c770 con 0x7fb3680d0650 2024-04-04T06:17:02.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.593+0000 7fb36f200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb3680d0fe0 con 0x7fb36815c390 2024-04-04T06:17:02.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.593+0000 7fb36d200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb3680d0650 0x7fb3680d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:02.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.593+0000 7fb36d200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb3680d0650 0x7fb3680d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44138/0 (socket says 172.21.15.159:44138) 2024-04-04T06:17:02.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.593+0000 7fb36d200640 1 -- 172.21.15.159:0/3775949522 learned_addr learned my addr 172.21.15.159:0/3775949522 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:02.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.593+0000 7fb367e00640 1 --2- 172.21.15.159:0/3775949522 >> v2:172.21.15.178:3300/0 conn(0x7fb36815c390 0x7fb368073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:02.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.594+0000 7fb36d200640 1 -- 172.21.15.159:0/3775949522 >> v2:172.21.15.178:3300/0 conn(0x7fb36815c390 msgr2=0x7fb368073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:02.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.594+0000 7fb36d200640 1 --2- 172.21.15.159:0/3775949522 >> v2:172.21.15.178:3300/0 conn(0x7fb36815c390 0x7fb368073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:02.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.594+0000 7fb36d200640 1 -- 172.21.15.159:0/3775949522 >> v1:172.21.15.178:6789/0 conn(0x7fb3680af830 legacy=0x7fb368160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:02.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.594+0000 7fb36d200640 1 -- 172.21.15.159:0/3775949522 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3680ad120 con 0x7fb3680d0650 2024-04-04T06:17:02.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.595+0000 7fb36d200640 1 --2- 172.21.15.159:0/3775949522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3680d0650 0x7fb3680d0aa0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fb3680b63e0 tx=0x7fb36002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56840c2b9467266a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:02.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.595+0000 7fb367400640 1 -- 172.21.15.159:0/3775949522 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36002ca80 con 0x7fb3680d0650 2024-04-04T06:17:02.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.595+0000 7fb367400640 1 -- 172.21.15.159:0/3775949522 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36002cc20 con 0x7fb3680d0650 2024-04-04T06:17:02.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.595+0000 7fb367400640 1 -- 172.21.15.159:0/3775949522 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3600343f0 con 0x7fb3680d0650 2024-04-04T06:17:02.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.596+0000 7fb36f200640 1 -- 172.21.15.159:0/3775949522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3680d0650 msgr2=0x7fb3680d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:02.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.596+0000 7fb36f200640 1 --2- 172.21.15.159:0/3775949522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3680d0650 0x7fb3680d0aa0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fb3680b63e0 tx=0x7fb36002bee0 comp rx=0 tx=0).stop 2024-04-04T06:17:02.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.596+0000 7fb36f200640 1 -- 172.21.15.159:0/3775949522 shutdown_connections 2024-04-04T06:17:02.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.596+0000 7fb36f200640 1 --2- 172.21.15.159:0/3775949522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3680d0650 0x7fb3680d0aa0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:02.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.596+0000 7fb36f200640 1 --2- 172.21.15.159:0/3775949522 >> v2:172.21.15.178:3300/0 conn(0x7fb36815c390 0x7fb368073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:02.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.596+0000 7fb36f200640 1 -- 172.21.15.159:0/3775949522 >> 172.21.15.159:0/3775949522 conn(0x7fb36806b990 msgr2=0x7fb36806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:02.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.596+0000 7fb36f200640 1 -- 172.21.15.159:0/3775949522 shutdown_connections 2024-04-04T06:17:02.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.596+0000 7fb36f200640 1 -- 172.21.15.159:0/3775949522 wait complete. 2024-04-04T06:17:02.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.598+0000 7fb36f200640 1 Processor -- start 2024-04-04T06:17:02.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.599+0000 7fb36f200640 1 -- start start 2024-04-04T06:17:02.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.600+0000 7fb36f200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3680af830 0x7fb3681f5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:02.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.600+0000 7fb36f200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3680d0650 0x7fb36815f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:02.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.600+0000 7fb36f200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb36815c390 0x7fb3681f80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:02.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.600+0000 7fb36f200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb368160120 con 0x7fb36815c390 2024-04-04T06:17:02.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.600+0000 7fb36f200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb3680ad6e0 con 0x7fb3680af830 2024-04-04T06:17:02.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.600+0000 7fb36f200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb368160d50 con 0x7fb3680d0650 2024-04-04T06:17:02.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.600+0000 7fb36d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb36815c390 0x7fb3681f80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:02.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.600+0000 7fb36d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb36815c390 0x7fb3681f80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44142/0 (socket says 172.21.15.159:44142) 2024-04-04T06:17:02.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.600+0000 7fb36d200640 1 -- 172.21.15.159:0/2449356319 learned_addr learned my addr 172.21.15.159:0/2449356319 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:02.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.601+0000 7fb367e00640 1 --2- 172.21.15.159:0/2449356319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3680af830 0x7fb3681f5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:02.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.601+0000 7fb35f400640 1 --2- 172.21.15.159:0/2449356319 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3680d0650 0x7fb36815f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:02.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.601+0000 7fb36d200640 1 -- 172.21.15.159:0/2449356319 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3680d0650 msgr2=0x7fb36815f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:02.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.601+0000 7fb36d200640 1 --2- 172.21.15.159:0/2449356319 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3680d0650 0x7fb36815f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:02.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.601+0000 7fb36d200640 1 -- 172.21.15.159:0/2449356319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3680af830 msgr2=0x7fb3681f5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:02.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.601+0000 7fb36d200640 1 --2- 172.21.15.159:0/2449356319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3680af830 0x7fb3681f5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:02.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.601+0000 7fb36d200640 1 -- 172.21.15.159:0/2449356319 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3681f4ac0 con 0x7fb36815c390 2024-04-04T06:17:02.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.601+0000 7fb36d200640 1 --2- 172.21.15.159:0/2449356319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb36815c390 0x7fb3681f80b0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fb360036040 tx=0x7fb360004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:02.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.602+0000 7fb364c00640 1 -- 172.21.15.159:0/2449356319 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360043020 con 0x7fb36815c390 2024-04-04T06:17:02.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.602+0000 7fb364c00640 1 -- 172.21.15.159:0/2449356319 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36003e450 con 0x7fb36815c390 2024-04-04T06:17:02.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.602+0000 7fb36f200640 1 -- 172.21.15.159:0/2449356319 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3681f8b70 con 0x7fb36815c390 2024-04-04T06:17:02.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.602+0000 7fb364c00640 1 -- 172.21.15.159:0/2449356319 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360009510 con 0x7fb36815c390 2024-04-04T06:17:02.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.602+0000 7fb36f200640 1 -- 172.21.15.159:0/2449356319 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3681f4d40 con 0x7fb36815c390 2024-04-04T06:17:02.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.603+0000 7fb36f200640 1 -- 172.21.15.159:0/2449356319 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3680685c0 con 0x7fb36815c390 2024-04-04T06:17:02.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.604+0000 7fb364c00640 1 -- 172.21.15.159:0/2449356319 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb360046020 con 0x7fb36815c390 2024-04-04T06:17:02.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.605+0000 7fb364c00640 1 --2- 172.21.15.159:0/2449356319 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb34406c8f0 0x7fb34406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:02.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.605+0000 7fb364c00640 1 -- 172.21.15.159:0/2449356319 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7fb3600ae470 con 0x7fb36815c390 2024-04-04T06:17:02.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.605+0000 7fb367e00640 1 --2- 172.21.15.159:0/2449356319 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb34406c8f0 0x7fb34406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:02.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.606+0000 7fb367e00640 1 --2- 172.21.15.159:0/2449356319 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb34406c8f0 0x7fb34406edb0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb350006fd0 tx=0x7fb350008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:02.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.609+0000 7fb364c00640 1 -- 172.21.15.159:0/2449356319 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb36004d030 con 0x7fb36815c390 2024-04-04T06:17:02.751 INFO:tasks.ceph.mds.a.smithi159.stderr:2024-04-04T06:17:02.756+0000 7f4abd600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:17:02.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.839+0000 7fb36f200640 1 -- 172.21.15.159:0/2449356319 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb36806d8b0 con 0x7fb34406c8f0 2024-04-04T06:17:02.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.857+0000 7fb364c00640 1 -- 172.21.15.159:0/2449356319 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248429 (secure 0 0 0) 0x7fb36806d8b0 con 0x7fb34406c8f0 2024-04-04T06:17:02.855 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:02.856 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:17:02.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.862+0000 7fb36f200640 1 -- 172.21.15.159:0/2449356319 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb34406c8f0 msgr2=0x7fb34406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:02.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.862+0000 7fb36f200640 1 --2- 172.21.15.159:0/2449356319 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb34406c8f0 0x7fb34406edb0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb350006fd0 tx=0x7fb350008040 comp rx=0 tx=0).stop 2024-04-04T06:17:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.862+0000 7fb36f200640 1 -- 172.21.15.159:0/2449356319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb36815c390 msgr2=0x7fb3681f80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.862+0000 7fb36f200640 1 --2- 172.21.15.159:0/2449356319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb36815c390 0x7fb3681f80b0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fb360036040 tx=0x7fb360004a40 comp rx=0 tx=0).stop 2024-04-04T06:17:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.862+0000 7fb36f200640 1 -- 172.21.15.159:0/2449356319 shutdown_connections 2024-04-04T06:17:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.862+0000 7fb36f200640 1 --2- 172.21.15.159:0/2449356319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb36815c390 0x7fb3681f80b0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.862+0000 7fb36f200640 1 --2- 172.21.15.159:0/2449356319 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb3680d0650 0x7fb36815f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.862+0000 7fb36f200640 1 --2- 172.21.15.159:0/2449356319 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb34406c8f0 0x7fb34406edb0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.862+0000 7fb36f200640 1 --2- 172.21.15.159:0/2449356319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3680af830 0x7fb3681f5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.862+0000 7fb36f200640 1 -- 172.21.15.159:0/2449356319 >> 172.21.15.159:0/2449356319 conn(0x7fb36806b990 msgr2=0x7fb368160640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.862+0000 7fb36f200640 1 -- 172.21.15.159:0/2449356319 shutdown_connections 2024-04-04T06:17:02.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:02.862+0000 7fb36f200640 1 -- 172.21.15.159:0/2449356319 wait complete. 2024-04-04T06:17:02.878 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-04-04T06:17:00.962376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122,"ondisk_log_size":122,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":277956,"kb_used_data":11864,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754696764,"statfs":{"total":773094113280,"available":772809486336,"internally_reserved":0,"allocated":12148736,"data_stored":5921784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.180982"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.795578+0000","last_change":"2024-04-04T06:16:56.795578+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.795578+0000","last_undegraded":"2024-04-04T06:16:56.795578+0000","last_fullsized":"2024-04-04T06:16:56.795578+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856681+0000","last_change":"2024-04-04T06:16:57.850062+0000","last_active":"2024-04-04T06:16:58.856681+0000","last_peered":"2024-04-04T06:16:58.856681+0000","last_clean":"2024-04-04T06:16:58.856681+0000","last_became_active":"2024-04-04T06:16:57.849768+0000","last_became_peered":"2024-04-04T06:16:57.849768+0000","last_unstale":"2024-04-04T06:16:58.856681+0000","last_undegraded":"2024-04-04T06:16:58.856681+0000","last_fullsized":"2024-04-04T06:16:58.856681+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:20:36.274557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.815390+0000","last_change":"2024-04-04T06:16:56.815390+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.815390+0000","last_undegraded":"2024-04-04T06:16:56.815390+0000","last_fullsized":"2024-04-04T06:16:56.815390+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.795247+0000","last_change":"2024-04-04T06:16:56.795247+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.795247+0000","last_undegraded":"2024-04-04T06:16:56.795247+0000","last_fullsized":"2024-04-04T06:16:56.795247+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.801944+0000","last_change":"2024-04-04T06:16:56.801944+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.801944+0000","last_undegraded":"2024-04-04T06:16:56.801944+0000","last_fullsized":"2024-04-04T06:16:56.801944+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.809513+0000","last_change":"2024-04-04T06:16:56.809513+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.809513+0000","last_undegraded":"2024-04-04T06:16:56.809513+0000","last_fullsized":"2024-04-04T06:16:56.809513+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855381+0000","last_change":"2024-04-04T06:16:57.853261+0000","last_active":"2024-04-04T06:16:58.855381+0000","last_peered":"2024-04-04T06:16:58.855381+0000","last_clean":"2024-04-04T06:16:58.855381+0000","last_became_active":"2024-04-04T06:16:57.852925+0000","last_became_peered":"2024-04-04T06:16:57.852925+0000","last_unstale":"2024-04-04T06:16:58.855381+0000","last_undegraded":"2024-04-04T06:16:58.855381+0000","last_fullsized":"2024-04-04T06:16:58.855381+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:43:19.761558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.807559+0000","last_change":"2024-04-04T06:16:56.807559+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.807559+0000","last_undegraded":"2024-04-04T06:16:56.807559+0000","last_fullsized":"2024-04-04T06:16:56.807559+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.794893+0000","last_change":"2024-04-04T06:16:56.794893+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.794893+0000","last_undegraded":"2024-04-04T06:16:56.794893+0000","last_fullsized":"2024-04-04T06:16:56.794893+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.794052+0000","last_change":"2024-04-04T06:16:56.794052+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.794052+0000","last_undegraded":"2024-04-04T06:16:56.794052+0000","last_fullsized":"2024-04-04T06:16:56.794052+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.857193+0000","last_change":"2024-04-04T06:16:57.852970+0000","last_active":"2024-04-04T06:16:58.857193+0000","last_peered":"2024-04-04T06:16:58.857193+0000","last_clean":"2024-04-04T06:16:58.857193+0000","last_became_active":"2024-04-04T06:16:57.852398+0000","last_became_peered":"2024-04-04T06:16:57.852398+0000","last_unstale":"2024-04-04T06:16:58.857193+0000","last_undegraded":"2024-04-04T06:16:58.857193+0000","last_fullsized":"2024-04-04T06:16:58.857193+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:25:13.863001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.794715+0000","last_change":"2024-04-04T06:16:56.794715+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.794715+0000","last_undegraded":"2024-04-04T06:16:56.794715+0000","last_fullsized":"2024-04-04T06:16:56.794715+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.815700+0000","last_change":"2024-04-04T06:16:56.815700+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.815700+0000","last_undegraded":"2024-04-04T06:16:56.815700+0000","last_fullsized":"2024-04-04T06:16:56.815700+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.809448+0000","last_change":"2024-04-04T06:16:56.809448+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.809448+0000","last_undegraded":"2024-04-04T06:16:56.809448+0000","last_fullsized":"2024-04-04T06:16:56.809448+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.809974+0000","last_change":"2024-04-04T06:16:56.809974+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.809974+0000","last_undegraded":"2024-04-04T06:16:56.809974+0000","last_fullsized":"2024-04-04T06:16:56.809974+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.797477+0000","last_change":"2024-04-04T06:16:56.797477+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.797477+0000","last_undegraded":"2024-04-04T06:16:56.797477+0000","last_fullsized":"2024-04-04T06:16:56.797477+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856787+0000","last_change":"2024-04-04T06:16:57.861016+0000","last_active":"2024-04-04T06:16:58.856787+0000","last_peered":"2024-04-04T06:16:58.856787+0000","last_clean":"2024-04-04T06:16:58.856787+0000","last_became_active":"2024-04-04T06:16:57.860213+0000","last_became_peered":"2024-04-04T06:16:57.860213+0000","last_unstale":"2024-04-04T06:16:58.856787+0000","last_undegraded":"2024-04-04T06:16:58.856787+0000","last_fullsized":"2024-04-04T06:16:58.856787+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:41:22.250390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.796165+0000","last_change":"2024-04-04T06:16:56.796165+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.796165+0000","last_undegraded":"2024-04-04T06:16:56.796165+0000","last_fullsized":"2024-04-04T06:16:56.796165+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.814584+0000","last_change":"2024-04-04T06:16:56.814584+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.814584+0000","last_undegraded":"2024-04-04T06:16:56.814584+0000","last_fullsized":"2024-04-04T06:16:56.814584+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856739+0000","last_change":"2024-04-04T06:16:57.858770+0000","last_active":"2024-04-04T06:16:58.856739+0000","last_peered":"2024-04-04T06:16:58.856739+0000","last_clean":"2024-04-04T06:16:58.856739+0000","last_became_active":"2024-04-04T06:16:57.856816+0000","last_became_peered":"2024-04-04T06:16:57.856816+0000","last_unstale":"2024-04-04T06:16:58.856739+0000","last_undegraded":"2024-04-04T06:16:58.856739+0000","last_fullsized":"2024-04-04T06:16:58.856739+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:42:54.207457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.857204+0000","last_change":"2024-04-04T06:16:57.840896+0000","last_active":"2024-04-04T06:16:58.857204+0000","last_peered":"2024-04-04T06:16:58.857204+0000","last_clean":"2024-04-04T06:16:58.857204+0000","last_became_active":"2024-04-04T06:16:57.840145+0000","last_became_peered":"2024-04-04T06:16:57.840145+0000","last_unstale":"2024-04-04T06:16:58.857204+0000","last_undegraded":"2024-04-04T06:16:58.857204+0000","last_fullsized":"2024-04-04T06:16:58.857204+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:56:07.453527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.817019+0000","last_change":"2024-04-04T06:16:56.817019+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.817019+0000","last_undegraded":"2024-04-04T06:16:56.817019+0000","last_fullsized":"2024-04-04T06:16:56.817019+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855555+0000","last_change":"2024-04-04T06:16:57.855453+0000","last_active":"2024-04-04T06:16:58.855555+0000","last_peered":"2024-04-04T06:16:58.855555+0000","last_clean":"2024-04-04T06:16:58.855555+0000","last_became_active":"2024-04-04T06:16:57.855172+0000","last_became_peered":"2024-04-04T06:16:57.855172+0000","last_unstale":"2024-04-04T06:16:58.855555+0000","last_undegraded":"2024-04-04T06:16:58.855555+0000","last_fullsized":"2024-04-04T06:16:58.855555+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:17:03.994940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.816316+0000","last_change":"2024-04-04T06:16:56.816316+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.816316+0000","last_undegraded":"2024-04-04T06:16:56.816316+0000","last_fullsized":"2024-04-04T06:16:56.816316+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.857033+0000","last_change":"2024-04-04T06:16:57.859703+0000","last_active":"2024-04-04T06:16:58.857033+0000","last_peered":"2024-04-04T06:16:58.857033+0000","last_clean":"2024-04-04T06:16:58.857033+0000","last_became_active":"2024-04-04T06:16:57.859398+0000","last_became_peered":"2024-04-04T06:16:57.859398+0000","last_unstale":"2024-04-04T06:16:58.857033+0000","last_undegraded":"2024-04-04T06:16:58.857033+0000","last_fullsized":"2024-04-04T06:16:58.857033+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:29:25.780543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.793316+0000","last_change":"2024-04-04T06:16:56.793316+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.793316+0000","last_undegraded":"2024-04-04T06:16:56.793316+0000","last_fullsized":"2024-04-04T06:16:56.793316+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.810088+0000","last_change":"2024-04-04T06:16:56.810088+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.810088+0000","last_undegraded":"2024-04-04T06:16:56.810088+0000","last_fullsized":"2024-04-04T06:16:56.810088+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.815732+0000","last_change":"2024-04-04T06:16:56.815732+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.815732+0000","last_undegraded":"2024-04-04T06:16:56.815732+0000","last_fullsized":"2024-04-04T06:16:56.815732+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856807+0000","last_change":"2024-04-04T06:16:57.859118+0000","last_active":"2024-04-04T06:16:58.856807+0000","last_peered":"2024-04-04T06:16:58.856807+0000","last_clean":"2024-04-04T06:16:58.856807+0000","last_became_active":"2024-04-04T06:16:57.858242+0000","last_became_peered":"2024-04-04T06:16:57.858242+0000","last_unstale":"2024-04-04T06:16:58.856807+0000","last_undegraded":"2024-04-04T06:16:58.856807+0000","last_fullsized":"2024-04-04T06:16:58.856807+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:16:58.119204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855126+0000","last_change":"2024-04-04T06:16:57.838809+0000","last_active":"2024-04-04T06:16:58.855126+0000","last_peered":"2024-04-04T06:16:58.855126+0000","last_clean":"2024-04-04T06:16:58.855126+0000","last_became_active":"2024-04-04T06:16:57.837497+0000","last_became_peered":"2024-04-04T06:16:57.837497+0000","last_unstale":"2024-04-04T06:16:58.855126+0000","last_undegraded":"2024-04-04T06:16:58.855126+0000","last_fullsized":"2024-04-04T06:16:58.855126+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:05:00.308975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.796582+0000","last_change":"2024-04-04T06:16:56.796582+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.796582+0000","last_undegraded":"2024-04-04T06:16:56.796582+0000","last_fullsized":"2024-04-04T06:16:56.796582+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.808526+0000","last_change":"2024-04-04T06:16:56.808526+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.808526+0000","last_undegraded":"2024-04-04T06:16:56.808526+0000","last_fullsized":"2024-04-04T06:16:56.808526+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.808552+0000","last_change":"2024-04-04T06:16:56.808552+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.808552+0000","last_undegraded":"2024-04-04T06:16:56.808552+0000","last_fullsized":"2024-04-04T06:16:56.808552+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"112'4","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855930+0000","last_change":"2024-04-04T06:16:56.826026+0000","last_active":"2024-04-04T06:16:58.855930+0000","last_peered":"2024-04-04T06:16:58.855930+0000","last_clean":"2024-04-04T06:16:58.855930+0000","last_became_active":"2024-04-04T06:16:56.825749+0000","last_became_peered":"2024-04-04T06:16:56.825749+0000","last_unstale":"2024-04-04T06:16:58.855930+0000","last_undegraded":"2024-04-04T06:16:58.855930+0000","last_fullsized":"2024-04-04T06:16:58.855930+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:44:30.609701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.796084+0000","last_change":"2024-04-04T06:16:56.796084+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.796084+0000","last_undegraded":"2024-04-04T06:16:56.796084+0000","last_fullsized":"2024-04-04T06:16:56.796084+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.829285+0000","last_change":"2024-04-04T06:16:56.828520+0000","last_active":"2024-04-04T06:16:56.829285+0000","last_peered":"2024-04-04T06:16:56.829285+0000","last_clean":"2024-04-04T06:16:56.829285+0000","last_became_active":"2024-04-04T06:16:56.828125+0000","last_became_peered":"2024-04-04T06:16:56.828125+0000","last_unstale":"2024-04-04T06:16:56.829285+0000","last_undegraded":"2024-04-04T06:16:56.829285+0000","last_fullsized":"2024-04-04T06:16:56.829285+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:04:31.414046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.807591+0000","last_change":"2024-04-04T06:16:56.807591+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.807591+0000","last_undegraded":"2024-04-04T06:16:56.807591+0000","last_fullsized":"2024-04-04T06:16:56.807591+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.823269+0000","last_change":"2024-04-04T06:16:56.817318+0000","last_active":"2024-04-04T06:16:56.823269+0000","last_peered":"2024-04-04T06:16:56.823269+0000","last_clean":"2024-04-04T06:16:56.823269+0000","last_became_active":"2024-04-04T06:16:56.816995+0000","last_became_peered":"2024-04-04T06:16:56.816995+0000","last_unstale":"2024-04-04T06:16:56.823269+0000","last_undegraded":"2024-04-04T06:16:56.823269+0000","last_fullsized":"2024-04-04T06:16:56.823269+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:02:04.420894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.813683+0000","last_change":"2024-04-04T06:16:56.813683+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.813683+0000","last_undegraded":"2024-04-04T06:16:56.813683+0000","last_fullsized":"2024-04-04T06:16:56.813683+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.823058+0000","last_change":"2024-04-04T06:16:56.813821+0000","last_active":"2024-04-04T06:16:56.823058+0000","last_peered":"2024-04-04T06:16:56.823058+0000","last_clean":"2024-04-04T06:16:56.823058+0000","last_became_active":"2024-04-04T06:16:56.813555+0000","last_became_peered":"2024-04-04T06:16:56.813555+0000","last_unstale":"2024-04-04T06:16:56.823058+0000","last_undegraded":"2024-04-04T06:16:56.823058+0000","last_fullsized":"2024-04-04T06:16:56.823058+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:52:53.198860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.795444+0000","last_change":"2024-04-04T06:16:56.795444+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.795444+0000","last_undegraded":"2024-04-04T06:16:56.795444+0000","last_fullsized":"2024-04-04T06:16:56.795444+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854845+0000","last_change":"2024-04-04T06:16:56.813703+0000","last_active":"2024-04-04T06:16:58.854845+0000","last_peered":"2024-04-04T06:16:58.854845+0000","last_clean":"2024-04-04T06:16:58.854845+0000","last_became_active":"2024-04-04T06:16:56.813224+0000","last_became_peered":"2024-04-04T06:16:56.813224+0000","last_unstale":"2024-04-04T06:16:58.854845+0000","last_undegraded":"2024-04-04T06:16:58.854845+0000","last_fullsized":"2024-04-04T06:16:58.854845+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:28:44.071981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856364+0000","last_change":"2024-04-04T06:16:57.851938+0000","last_active":"2024-04-04T06:16:58.856364+0000","last_peered":"2024-04-04T06:16:58.856364+0000","last_clean":"2024-04-04T06:16:58.856364+0000","last_became_active":"2024-04-04T06:16:57.850346+0000","last_became_peered":"2024-04-04T06:16:57.850346+0000","last_unstale":"2024-04-04T06:16:58.856364+0000","last_undegraded":"2024-04-04T06:16:58.856364+0000","last_fullsized":"2024-04-04T06:16:58.856364+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:42:36.526974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.824886+0000","last_change":"2024-04-04T06:16:56.822599+0000","last_active":"2024-04-04T06:16:56.824886+0000","last_peered":"2024-04-04T06:16:56.824886+0000","last_clean":"2024-04-04T06:16:56.824886+0000","last_became_active":"2024-04-04T06:16:56.822130+0000","last_became_peered":"2024-04-04T06:16:56.822130+0000","last_unstale":"2024-04-04T06:16:56.824886+0000","last_undegraded":"2024-04-04T06:16:56.824886+0000","last_fullsized":"2024-04-04T06:16:56.824886+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:15:58.869588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.796987+0000","last_change":"2024-04-04T06:16:56.796987+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.796987+0000","last_undegraded":"2024-04-04T06:16:56.796987+0000","last_fullsized":"2024-04-04T06:16:56.796987+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855249+0000","last_change":"2024-04-04T06:16:56.822615+0000","last_active":"2024-04-04T06:16:58.855249+0000","last_peered":"2024-04-04T06:16:58.855249+0000","last_clean":"2024-04-04T06:16:58.855249+0000","last_became_active":"2024-04-04T06:16:56.819761+0000","last_became_peered":"2024-04-04T06:16:56.819761+0000","last_unstale":"2024-04-04T06:16:58.855249+0000","last_undegraded":"2024-04-04T06:16:58.855249+0000","last_fullsized":"2024-04-04T06:16:58.855249+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:10:06.219773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854643+0000","last_change":"2024-04-04T06:16:57.853254+0000","last_active":"2024-04-04T06:16:58.854643+0000","last_peered":"2024-04-04T06:16:58.854643+0000","last_clean":"2024-04-04T06:16:58.854643+0000","last_became_active":"2024-04-04T06:16:57.852500+0000","last_became_peered":"2024-04-04T06:16:57.852500+0000","last_unstale":"2024-04-04T06:16:58.854643+0000","last_undegraded":"2024-04-04T06:16:58.854643+0000","last_fullsized":"2024-04-04T06:16:58.854643+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:47.042156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.822886+0000","last_change":"2024-04-04T06:16:56.816173+0000","last_active":"2024-04-04T06:16:56.822886+0000","last_peered":"2024-04-04T06:16:56.822886+0000","last_clean":"2024-04-04T06:16:56.822886+0000","last_became_active":"2024-04-04T06:16:56.815796+0000","last_became_peered":"2024-04-04T06:16:56.815796+0000","last_unstale":"2024-04-04T06:16:56.822886+0000","last_undegraded":"2024-04-04T06:16:56.822886+0000","last_fullsized":"2024-04-04T06:16:56.822886+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:46:18.307967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856214+0000","last_change":"2024-04-04T06:16:57.863523+0000","last_active":"2024-04-04T06:16:58.856214+0000","last_peered":"2024-04-04T06:16:58.856214+0000","last_clean":"2024-04-04T06:16:58.856214+0000","last_became_active":"2024-04-04T06:16:57.861142+0000","last_became_peered":"2024-04-04T06:16:57.861142+0000","last_unstale":"2024-04-04T06:16:58.856214+0000","last_undegraded":"2024-04-04T06:16:58.856214+0000","last_fullsized":"2024-04-04T06:16:58.856214+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:50:12.226068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.823105+0000","last_change":"2024-04-04T06:16:56.817070+0000","last_active":"2024-04-04T06:16:56.823105+0000","last_peered":"2024-04-04T06:16:56.823105+0000","last_clean":"2024-04-04T06:16:56.823105+0000","last_became_active":"2024-04-04T06:16:56.816742+0000","last_became_peered":"2024-04-04T06:16:56.816742+0000","last_unstale":"2024-04-04T06:16:56.823105+0000","last_undegraded":"2024-04-04T06:16:56.823105+0000","last_fullsized":"2024-04-04T06:16:56.823105+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:09:45.329512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.794919+0000","last_change":"2024-04-04T06:16:56.794919+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.794919+0000","last_undegraded":"2024-04-04T06:16:56.794919+0000","last_fullsized":"2024-04-04T06:16:56.794919+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.824720+0000","last_change":"2024-04-04T06:16:56.822568+0000","last_active":"2024-04-04T06:16:56.824720+0000","last_peered":"2024-04-04T06:16:56.824720+0000","last_clean":"2024-04-04T06:16:56.824720+0000","last_became_active":"2024-04-04T06:16:56.822237+0000","last_became_peered":"2024-04-04T06:16:56.822237+0000","last_unstale":"2024-04-04T06:16:56.824720+0000","last_undegraded":"2024-04-04T06:16:56.824720+0000","last_fullsized":"2024-04-04T06:16:56.824720+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:50:55.464717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.815062+0000","last_change":"2024-04-04T06:16:56.815062+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.815062+0000","last_undegraded":"2024-04-04T06:16:56.815062+0000","last_fullsized":"2024-04-04T06:16:56.815062+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.814979+0000","last_change":"2024-04-04T06:16:56.811481+0000","last_active":"2024-04-04T06:16:56.814979+0000","last_peered":"2024-04-04T06:16:56.814979+0000","last_clean":"2024-04-04T06:16:56.814979+0000","last_became_active":"2024-04-04T06:16:56.811190+0000","last_became_peered":"2024-04-04T06:16:56.811190+0000","last_unstale":"2024-04-04T06:16:56.814979+0000","last_undegraded":"2024-04-04T06:16:56.814979+0000","last_fullsized":"2024-04-04T06:16:56.814979+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:41:19.705887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855312+0000","last_change":"2024-04-04T06:16:57.855846+0000","last_active":"2024-04-04T06:16:58.855312+0000","last_peered":"2024-04-04T06:16:58.855312+0000","last_clean":"2024-04-04T06:16:58.855312+0000","last_became_active":"2024-04-04T06:16:57.855487+0000","last_became_peered":"2024-04-04T06:16:57.855487+0000","last_unstale":"2024-04-04T06:16:58.855312+0000","last_undegraded":"2024-04-04T06:16:58.855312+0000","last_fullsized":"2024-04-04T06:16:58.855312+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:25:12.522619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.822910+0000","last_change":"2024-04-04T06:16:56.816249+0000","last_active":"2024-04-04T06:16:56.822910+0000","last_peered":"2024-04-04T06:16:56.822910+0000","last_clean":"2024-04-04T06:16:56.822910+0000","last_became_active":"2024-04-04T06:16:56.815980+0000","last_became_peered":"2024-04-04T06:16:56.815980+0000","last_unstale":"2024-04-04T06:16:56.822910+0000","last_undegraded":"2024-04-04T06:16:56.822910+0000","last_fullsized":"2024-04-04T06:16:56.822910+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:38:39.241002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.811065+0000","last_change":"2024-04-04T06:16:56.811065+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.811065+0000","last_undegraded":"2024-04-04T06:16:56.811065+0000","last_fullsized":"2024-04-04T06:16:56.811065+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.826345+0000","last_change":"2024-04-04T06:16:56.825811+0000","last_active":"2024-04-04T06:16:56.826345+0000","last_peered":"2024-04-04T06:16:56.826345+0000","last_clean":"2024-04-04T06:16:56.826345+0000","last_became_active":"2024-04-04T06:16:56.825491+0000","last_became_peered":"2024-04-04T06:16:56.825491+0000","last_unstale":"2024-04-04T06:16:56.826345+0000","last_undegraded":"2024-04-04T06:16:56.826345+0000","last_fullsized":"2024-04-04T06:16:56.826345+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:28.483864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.812435+0000","last_change":"2024-04-04T06:16:56.812435+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.812435+0000","last_undegraded":"2024-04-04T06:16:56.812435+0000","last_fullsized":"2024-04-04T06:16:56.812435+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.814395+0000","last_change":"2024-04-04T06:16:56.808070+0000","last_active":"2024-04-04T06:16:56.814395+0000","last_peered":"2024-04-04T06:16:56.814395+0000","last_clean":"2024-04-04T06:16:56.814395+0000","last_became_active":"2024-04-04T06:16:56.807074+0000","last_became_peered":"2024-04-04T06:16:56.807074+0000","last_unstale":"2024-04-04T06:16:56.814395+0000","last_undegraded":"2024-04-04T06:16:56.814395+0000","last_fullsized":"2024-04-04T06:16:56.814395+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:12:24.143284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.809129+0000","last_change":"2024-04-04T06:16:56.809129+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.809129+0000","last_undegraded":"2024-04-04T06:16:56.809129+0000","last_fullsized":"2024-04-04T06:16:56.809129+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.823004+0000","last_change":"2024-04-04T06:16:56.814462+0000","last_active":"2024-04-04T06:16:56.823004+0000","last_peered":"2024-04-04T06:16:56.823004+0000","last_clean":"2024-04-04T06:16:56.823004+0000","last_became_active":"2024-04-04T06:16:56.813042+0000","last_became_peered":"2024-04-04T06:16:56.813042+0000","last_unstale":"2024-04-04T06:16:56.823004+0000","last_undegraded":"2024-04-04T06:16:56.823004+0000","last_fullsized":"2024-04-04T06:16:56.823004+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:23:44.093177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.802313+0000","last_change":"2024-04-04T06:16:56.802313+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.802313+0000","last_undegraded":"2024-04-04T06:16:56.802313+0000","last_fullsized":"2024-04-04T06:16:56.802313+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.818315+0000","last_change":"2024-04-04T06:16:56.812543+0000","last_active":"2024-04-04T06:16:56.818315+0000","last_peered":"2024-04-04T06:16:56.818315+0000","last_clean":"2024-04-04T06:16:56.818315+0000","last_became_active":"2024-04-04T06:16:56.812240+0000","last_became_peered":"2024-04-04T06:16:56.812240+0000","last_unstale":"2024-04-04T06:16:56.818315+0000","last_undegraded":"2024-04-04T06:16:56.818315+0000","last_fullsized":"2024-04-04T06:16:56.818315+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:49:28.135341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.808083+0000","last_change":"2024-04-04T06:16:56.808083+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.808083+0000","last_undegraded":"2024-04-04T06:16:56.808083+0000","last_fullsized":"2024-04-04T06:16:56.808083+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.824636+0000","last_change":"2024-04-04T06:16:56.822207+0000","last_active":"2024-04-04T06:16:56.824636+0000","last_peered":"2024-04-04T06:16:56.824636+0000","last_clean":"2024-04-04T06:16:56.824636+0000","last_became_active":"2024-04-04T06:16:56.821906+0000","last_became_peered":"2024-04-04T06:16:56.821906+0000","last_unstale":"2024-04-04T06:16:56.824636+0000","last_undegraded":"2024-04-04T06:16:56.824636+0000","last_fullsized":"2024-04-04T06:16:56.824636+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:28:49.663613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.814086+0000","last_change":"2024-04-04T06:16:56.814086+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.814086+0000","last_undegraded":"2024-04-04T06:16:56.814086+0000","last_fullsized":"2024-04-04T06:16:56.814086+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.825760+0000","last_change":"2024-04-04T06:16:56.818843+0000","last_active":"2024-04-04T06:16:56.825760+0000","last_peered":"2024-04-04T06:16:56.825760+0000","last_clean":"2024-04-04T06:16:56.825760+0000","last_became_active":"2024-04-04T06:16:56.818532+0000","last_became_peered":"2024-04-04T06:16:56.818532+0000","last_unstale":"2024-04-04T06:16:56.825760+0000","last_undegraded":"2024-04-04T06:16:56.825760+0000","last_fullsized":"2024-04-04T06:16:56.825760+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:28:13.269672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.811863+0000","last_change":"2024-04-04T06:16:56.811863+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.811863+0000","last_undegraded":"2024-04-04T06:16:56.811863+0000","last_fullsized":"2024-04-04T06:16:56.811863+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854548+0000","last_change":"2024-04-04T06:16:56.817026+0000","last_active":"2024-04-04T06:16:58.854548+0000","last_peered":"2024-04-04T06:16:58.854548+0000","last_clean":"2024-04-04T06:16:58.854548+0000","last_became_active":"2024-04-04T06:16:56.816488+0000","last_became_peered":"2024-04-04T06:16:56.816488+0000","last_unstale":"2024-04-04T06:16:58.854548+0000","last_undegraded":"2024-04-04T06:16:58.854548+0000","last_fullsized":"2024-04-04T06:16:58.854548+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:11:49.042689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.808636+0000","last_change":"2024-04-04T06:16:56.808636+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.808636+0000","last_undegraded":"2024-04-04T06:16:56.808636+0000","last_fullsized":"2024-04-04T06:16:56.808636+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.824529+0000","last_change":"2024-04-04T06:16:56.821203+0000","last_active":"2024-04-04T06:16:56.824529+0000","last_peered":"2024-04-04T06:16:56.824529+0000","last_clean":"2024-04-04T06:16:56.824529+0000","last_became_active":"2024-04-04T06:16:56.820739+0000","last_became_peered":"2024-04-04T06:16:56.820739+0000","last_unstale":"2024-04-04T06:16:56.824529+0000","last_undegraded":"2024-04-04T06:16:56.824529+0000","last_fullsized":"2024-04-04T06:16:56.824529+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:06:13.571199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.795568+0000","last_change":"2024-04-04T06:16:56.795568+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.795568+0000","last_undegraded":"2024-04-04T06:16:56.795568+0000","last_fullsized":"2024-04-04T06:16:56.795568+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.822827+0000","last_change":"2024-04-04T06:16:56.816260+0000","last_active":"2024-04-04T06:16:56.822827+0000","last_peered":"2024-04-04T06:16:56.822827+0000","last_clean":"2024-04-04T06:16:56.822827+0000","last_became_active":"2024-04-04T06:16:56.815988+0000","last_became_peered":"2024-04-04T06:16:56.815988+0000","last_unstale":"2024-04-04T06:16:56.822827+0000","last_undegraded":"2024-04-04T06:16:56.822827+0000","last_fullsized":"2024-04-04T06:16:56.822827+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:11:45.216267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.806526+0000","last_change":"2024-04-04T06:16:56.806526+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.806526+0000","last_undegraded":"2024-04-04T06:16:56.806526+0000","last_fullsized":"2024-04-04T06:16:56.806526+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.824945+0000","last_change":"2024-04-04T06:16:56.823585+0000","last_active":"2024-04-04T06:16:56.824945+0000","last_peered":"2024-04-04T06:16:56.824945+0000","last_clean":"2024-04-04T06:16:56.824945+0000","last_became_active":"2024-04-04T06:16:56.823294+0000","last_became_peered":"2024-04-04T06:16:56.823294+0000","last_unstale":"2024-04-04T06:16:56.824945+0000","last_undegraded":"2024-04-04T06:16:56.824945+0000","last_fullsized":"2024-04-04T06:16:56.824945+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:28:49.663613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.805594+0000","last_change":"2024-04-04T06:16:56.805594+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.805594+0000","last_undegraded":"2024-04-04T06:16:56.805594+0000","last_fullsized":"2024-04-04T06:16:56.805594+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.810421+0000","last_change":"2024-04-04T06:16:56.806783+0000","last_active":"2024-04-04T06:16:56.810421+0000","last_peered":"2024-04-04T06:16:56.810421+0000","last_clean":"2024-04-04T06:16:56.810421+0000","last_became_active":"2024-04-04T06:16:56.806293+0000","last_became_peered":"2024-04-04T06:16:56.806293+0000","last_unstale":"2024-04-04T06:16:56.810421+0000","last_undegraded":"2024-04-04T06:16:56.810421+0000","last_fullsized":"2024-04-04T06:16:56.810421+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:37:39.827821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.794903+0000","last_change":"2024-04-04T06:16:56.794903+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.794903+0000","last_undegraded":"2024-04-04T06:16:56.794903+0000","last_fullsized":"2024-04-04T06:16:56.794903+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.822668+0000","last_change":"2024-04-04T06:16:56.817822+0000","last_active":"2024-04-04T06:16:56.822668+0000","last_peered":"2024-04-04T06:16:56.822668+0000","last_clean":"2024-04-04T06:16:56.822668+0000","last_became_active":"2024-04-04T06:16:56.816539+0000","last_became_peered":"2024-04-04T06:16:56.816539+0000","last_unstale":"2024-04-04T06:16:56.822668+0000","last_undegraded":"2024-04-04T06:16:56.822668+0000","last_fullsized":"2024-04-04T06:16:56.822668+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:40:02.467791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.794244+0000","last_change":"2024-04-04T06:16:56.794244+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.794244+0000","last_undegraded":"2024-04-04T06:16:56.794244+0000","last_fullsized":"2024-04-04T06:16:56.794244+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.825725+0000","last_change":"2024-04-04T06:16:56.819463+0000","last_active":"2024-04-04T06:16:56.825725+0000","last_peered":"2024-04-04T06:16:56.825725+0000","last_clean":"2024-04-04T06:16:56.825725+0000","last_became_active":"2024-04-04T06:16:56.816034+0000","last_became_peered":"2024-04-04T06:16:56.816034+0000","last_unstale":"2024-04-04T06:16:56.825725+0000","last_undegraded":"2024-04-04T06:16:56.825725+0000","last_fullsized":"2024-04-04T06:16:56.825725+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:13:18.335553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.793602+0000","last_change":"2024-04-04T06:16:56.793602+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.793602+0000","last_undegraded":"2024-04-04T06:16:56.793602+0000","last_fullsized":"2024-04-04T06:16:56.793602+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.805734+0000","last_change":"2024-04-04T06:16:56.803667+0000","last_active":"2024-04-04T06:16:56.805734+0000","last_peered":"2024-04-04T06:16:56.805734+0000","last_clean":"2024-04-04T06:16:56.805734+0000","last_became_active":"2024-04-04T06:16:56.803353+0000","last_became_peered":"2024-04-04T06:16:56.803353+0000","last_unstale":"2024-04-04T06:16:56.805734+0000","last_undegraded":"2024-04-04T06:16:56.805734+0000","last_fullsized":"2024-04-04T06:16:56.805734+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:12:11.893578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.795146+0000","last_change":"2024-04-04T06:16:56.795146+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.795146+0000","last_undegraded":"2024-04-04T06:16:56.795146+0000","last_fullsized":"2024-04-04T06:16:56.795146+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.823935+0000","last_change":"2024-04-04T06:16:56.820108+0000","last_active":"2024-04-04T06:16:56.823935+0000","last_peered":"2024-04-04T06:16:56.823935+0000","last_clean":"2024-04-04T06:16:56.823935+0000","last_became_active":"2024-04-04T06:16:56.819768+0000","last_became_peered":"2024-04-04T06:16:56.819768+0000","last_unstale":"2024-04-04T06:16:56.823935+0000","last_undegraded":"2024-04-04T06:16:56.823935+0000","last_fullsized":"2024-04-04T06:16:56.823935+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:44:04.731918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855977+0000","last_change":"2024-04-04T06:16:57.844825+0000","last_active":"2024-04-04T06:16:58.855977+0000","last_peered":"2024-04-04T06:16:58.855977+0000","last_clean":"2024-04-04T06:16:58.855977+0000","last_became_active":"2024-04-04T06:16:57.843101+0000","last_became_peered":"2024-04-04T06:16:57.843101+0000","last_unstale":"2024-04-04T06:16:58.855977+0000","last_undegraded":"2024-04-04T06:16:58.855977+0000","last_fullsized":"2024-04-04T06:16:58.855977+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:41:12.600534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854882+0000","last_change":"2024-04-04T06:16:56.823376+0000","last_active":"2024-04-04T06:16:58.854882+0000","last_peered":"2024-04-04T06:16:58.854882+0000","last_clean":"2024-04-04T06:16:58.854882+0000","last_became_active":"2024-04-04T06:16:56.822325+0000","last_became_peered":"2024-04-04T06:16:56.822325+0000","last_unstale":"2024-04-04T06:16:58.854882+0000","last_undegraded":"2024-04-04T06:16:58.854882+0000","last_fullsized":"2024-04-04T06:16:58.854882+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:17:31.397079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856868+0000","last_change":"2024-04-04T06:16:57.861595+0000","last_active":"2024-04-04T06:16:58.856868+0000","last_peered":"2024-04-04T06:16:58.856868+0000","last_clean":"2024-04-04T06:16:58.856868+0000","last_became_active":"2024-04-04T06:16:57.860975+0000","last_became_peered":"2024-04-04T06:16:57.860975+0000","last_unstale":"2024-04-04T06:16:58.856868+0000","last_undegraded":"2024-04-04T06:16:58.856868+0000","last_fullsized":"2024-04-04T06:16:58.856868+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:29:44.470791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855757+0000","last_change":"2024-04-04T06:16:56.822556+0000","last_active":"2024-04-04T06:16:58.855757+0000","last_peered":"2024-04-04T06:16:58.855757+0000","last_clean":"2024-04-04T06:16:58.855757+0000","last_became_active":"2024-04-04T06:16:56.821979+0000","last_became_peered":"2024-04-04T06:16:56.821979+0000","last_unstale":"2024-04-04T06:16:58.855757+0000","last_undegraded":"2024-04-04T06:16:58.855757+0000","last_fullsized":"2024-04-04T06:16:58.855757+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:59:42.200129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.805318+0000","last_change":"2024-04-04T06:16:56.805318+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.805318+0000","last_undegraded":"2024-04-04T06:16:56.805318+0000","last_fullsized":"2024-04-04T06:16:56.805318+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.824612+0000","last_change":"2024-04-04T06:16:56.818500+0000","last_active":"2024-04-04T06:16:56.824612+0000","last_peered":"2024-04-04T06:16:56.824612+0000","last_clean":"2024-04-04T06:16:56.824612+0000","last_became_active":"2024-04-04T06:16:56.816829+0000","last_became_peered":"2024-04-04T06:16:56.816829+0000","last_unstale":"2024-04-04T06:16:56.824612+0000","last_undegraded":"2024-04-04T06:16:56.824612+0000","last_fullsized":"2024-04-04T06:16:56.824612+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:06:13.571199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.793318+0000","last_change":"2024-04-04T06:16:56.793318+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.793318+0000","last_undegraded":"2024-04-04T06:16:56.793318+0000","last_fullsized":"2024-04-04T06:16:56.793318+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.810449+0000","last_change":"2024-04-04T06:16:56.807000+0000","last_active":"2024-04-04T06:16:56.810449+0000","last_peered":"2024-04-04T06:16:56.810449+0000","last_clean":"2024-04-04T06:16:56.810449+0000","last_became_active":"2024-04-04T06:16:56.806687+0000","last_became_peered":"2024-04-04T06:16:56.806687+0000","last_unstale":"2024-04-04T06:16:56.810449+0000","last_undegraded":"2024-04-04T06:16:56.810449+0000","last_fullsized":"2024-04-04T06:16:56.810449+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:45:42.675350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-04-04T06:16:56.806584+0000","last_change":"2024-04-04T06:16:56.806584+0000","last_active":"2024-04-04T06:16:56.778822+0000","last_peered":"2024-04-04T06:16:56.778822+0000","last_clean":"2024-04-04T06:16:56.778822+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-04T06:16:56.806584+0000","last_undegraded":"2024-04-04T06:16:56.806584+0000","last_fullsized":"2024-04-04T06:16:56.806584+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854824+0000","last_change":"2024-04-04T06:16:56.819104+0000","last_active":"2024-04-04T06:16:58.854824+0000","last_peered":"2024-04-04T06:16:58.854824+0000","last_clean":"2024-04-04T06:16:58.854824+0000","last_became_active":"2024-04-04T06:16:56.818492+0000","last_became_peered":"2024-04-04T06:16:56.818492+0000","last_unstale":"2024-04-04T06:16:58.854824+0000","last_undegraded":"2024-04-04T06:16:58.854824+0000","last_fullsized":"2024-04-04T06:16:58.854824+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:14:15.299577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":334,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-04T06:16:56.819399+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:16:56.819399+0000","last_peered":"2024-04-04T06:16:56.819399+0000","last_clean":"2024-04-04T06:16:56.819399+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:16:56.819399+0000","last_undegraded":"2024-04-04T06:16:56.819399+0000","last_fullsized":"2024-04-04T06:16:56.819399+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509577,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37248,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334592,"statfs":{"total":96636764160,"available":96598622208,"internally_reserved":0,"allocated":1953792,"data_stored":1178086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509579,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36668,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335172,"statfs":{"total":96636764160,"available":96599216128,"internally_reserved":0,"allocated":1359872,"data_stored":587718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509580,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1359872,"data_stored":587718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509579,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1359872,"data_stored":587718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509578,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1359872,"data_stored":587718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509580,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1953792,"data_stored":1178086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36712,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335128,"statfs":{"total":96636764160,"available":96599171072,"internally_reserved":0,"allocated":1404928,"data_stored":607672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542282,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36712,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335128,"statfs":{"total":96636764160,"available":96599171072,"internally_reserved":0,"allocated":1396736,"data_stored":607068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:17:02.882 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:17:03.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.128+0000 7f0a1c000640 1 Processor -- start 2024-04-04T06:17:03.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.128+0000 7f0a1c000640 1 -- start start 2024-04-04T06:17:03.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.130+0000 7f0a1c000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0a1415eab0 0x7f0a1406c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:03.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.130+0000 7f0a1c000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a140b5990 0x7f0a1406cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:03.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.130+0000 7f0a1c000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a140b53e0 con 0x7f0a14072a00 2024-04-04T06:17:03.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.130+0000 7f0a1c000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0a140b56c0 con 0x7f0a1415eab0 2024-04-04T06:17:03.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.130+0000 7f0a1c000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0a140adc80 con 0x7f0a140b5990 2024-04-04T06:17:03.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.130+0000 7f0a1a000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f0a14072a00 0x7f0a140b3990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:45254/0 (socket says 172.21.15.159:45254) 2024-04-04T06:17:03.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.130+0000 7f0a1a000640 1 -- 172.21.15.159:0/1716573320 learned_addr learned my addr 172.21.15.159:0/1716573320 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:03.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.130+0000 7f0a19600640 1 --2- 172.21.15.159:0/1716573320 >> v2:172.21.15.178:3300/0 conn(0x7f0a1415eab0 0x7f0a1406c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:03.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.131+0000 7f0a18c00640 1 --2- 172.21.15.159:0/1716573320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a140b5990 0x7f0a1406cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:03.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.131+0000 7f0a13e00640 1 -- 172.21.15.159:0/1716573320 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2057086646 0 0) 0x7f0a140b53e0 con 0x7f0a14072a00 2024-04-04T06:17:03.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.131+0000 7f0a13e00640 1 -- 172.21.15.159:0/1716573320 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09fc003520 con 0x7f0a14072a00 2024-04-04T06:17:03.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.131+0000 7f0a13e00640 1 -- 172.21.15.159:0/1716573320 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 493483334 0 0) 0x7f09fc003520 con 0x7f0a14072a00 2024-04-04T06:17:03.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.131+0000 7f0a13e00640 1 -- 172.21.15.159:0/1716573320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a140b5990 msgr2=0x7f0a1406cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:03.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.131+0000 7f0a13e00640 1 --2- 172.21.15.159:0/1716573320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a140b5990 0x7f0a1406cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:03.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.131+0000 7f0a13e00640 1 -- 172.21.15.159:0/1716573320 >> v2:172.21.15.178:3300/0 conn(0x7f0a1415eab0 msgr2=0x7f0a1406c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:03.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.131+0000 7f0a13e00640 1 --2- 172.21.15.159:0/1716573320 >> v2:172.21.15.178:3300/0 conn(0x7f0a1415eab0 0x7f0a1406c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:03.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.131+0000 7f0a13e00640 1 -- 172.21.15.159:0/1716573320 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a140af490 con 0x7f0a14072a00 2024-04-04T06:17:03.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.131+0000 7f0a13e00640 1 -- 172.21.15.159:0/1716573320 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f0a0c0019f0 con 0x7f0a14072a00 2024-04-04T06:17:03.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.131+0000 7f0a13e00640 1 -- 172.21.15.159:0/1716573320 >> v1:172.21.15.159:6789/0 conn(0x7f0a14072a00 legacy=0x7f0a140b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:03.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.131+0000 7f0a13e00640 1 --2- 172.21.15.159:0/1716573320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a140b5990 0x7f09fc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:03.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.131+0000 7f0a13e00640 1 --2- 172.21.15.159:0/1716573320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09fc003f30 0x7f09fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:03.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.131+0000 7f0a13e00640 1 --2- 172.21.15.159:0/1716573320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09fc004d40 0x7f09fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:03.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.131+0000 7f0a13e00640 1 -- 172.21.15.159:0/1716573320 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0a140b56c0 con 0x7f09fc003f30 2024-04-04T06:17:03.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.131+0000 7f0a13e00640 1 -- 172.21.15.159:0/1716573320 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0a140adc80 con 0x7f09fc004d40 2024-04-04T06:17:03.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.131+0000 7f0a13e00640 1 -- 172.21.15.159:0/1716573320 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f09fc00b9c0 con 0x7f0a140b5990 2024-04-04T06:17:03.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.132+0000 7f0a18c00640 1 --2- 172.21.15.159:0/1716573320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09fc003f30 0x7f09fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:03.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.132+0000 7f0a1a000640 1 --2- 172.21.15.159:0/1716573320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09fc004d40 0x7f09fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:03.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.132+0000 7f0a19600640 1 --2- 172.21.15.159:0/1716573320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a140b5990 0x7f09fc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:03.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.132+0000 7f0a18c00640 1 -- 172.21.15.159:0/1716573320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a140b5990 msgr2=0x7f09fc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:03.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.132+0000 7f0a18c00640 1 --2- 172.21.15.159:0/1716573320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a140b5990 0x7f09fc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:03.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.132+0000 7f0a18c00640 1 -- 172.21.15.159:0/1716573320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09fc004d40 msgr2=0x7f09fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:03.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.132+0000 7f0a18c00640 1 --2- 172.21.15.159:0/1716573320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09fc004d40 0x7f09fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:03.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.132+0000 7f0a18c00640 1 -- 172.21.15.159:0/1716573320 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09fc003520 con 0x7f09fc003f30 2024-04-04T06:17:03.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.132+0000 7f0a19600640 1 --2- 172.21.15.159:0/1716573320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a140b5990 0x7f09fc0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:03.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.133+0000 7f0a18c00640 1 --2- 172.21.15.159:0/1716573320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09fc003f30 0x7f09fc004800 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009a90 tx=0x7f09fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3928075e06018 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:03.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.133+0000 7f0a13e00640 1 -- 172.21.15.159:0/1716573320 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a00039070 con 0x7f09fc003f30 2024-04-04T06:17:03.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.133+0000 7f0a13e00640 1 -- 172.21.15.159:0/1716573320 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a0002c670 con 0x7f09fc003f30 2024-04-04T06:17:03.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.133+0000 7f0a13e00640 1 -- 172.21.15.159:0/1716573320 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a00009690 con 0x7f09fc003f30 2024-04-04T06:17:03.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.134+0000 7f0a1c000640 1 -- 172.21.15.159:0/1716573320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09fc003f30 msgr2=0x7f09fc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:03.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.134+0000 7f0a1c000640 1 --2- 172.21.15.159:0/1716573320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09fc003f30 0x7f09fc004800 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009a90 tx=0x7f09fc005830 comp rx=0 tx=0).stop 2024-04-04T06:17:03.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.134+0000 7f0a1a000640 1 -- 172.21.15.159:0/1716573320 reap_dead start 2024-04-04T06:17:03.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.134+0000 7f0a1c000640 1 -- 172.21.15.159:0/1716573320 shutdown_connections 2024-04-04T06:17:03.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.134+0000 7f0a1c000640 1 -- 172.21.15.159:0/1716573320 >> 172.21.15.159:0/1716573320 conn(0x7f0a14071df0 msgr2=0x7f0a140721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:03.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.134+0000 7f0a1c000640 1 -- 172.21.15.159:0/1716573320 shutdown_connections 2024-04-04T06:17:03.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.134+0000 7f0a1c000640 1 -- 172.21.15.159:0/1716573320 wait complete. 2024-04-04T06:17:03.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.136+0000 7f0a1c000640 1 Processor -- start 2024-04-04T06:17:03.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.136+0000 7f0a1c000640 1 -- start start 2024-04-04T06:17:03.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.137+0000 7f0a1c000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09fc003f30 0x7f0a14147e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:03.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.137+0000 7f0a1c000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a141472c0 0x7f0a1415fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:03.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.137+0000 7f0a1c000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a141601c0 0x7f0a140ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:03.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.137+0000 7f0a1c000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1406cc40 con 0x7f0a141601c0 2024-04-04T06:17:03.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.137+0000 7f0a1c000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1406cac0 con 0x7f09fc003f30 2024-04-04T06:17:03.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.137+0000 7f0a1c000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0a1406cdc0 con 0x7f0a141472c0 2024-04-04T06:17:03.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.137+0000 7f0a1a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a141601c0 0x7f0a140ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:03.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.137+0000 7f0a1a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a141601c0 0x7f0a140ae6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44170/0 (socket says 172.21.15.159:44170) 2024-04-04T06:17:03.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.137+0000 7f0a1a000640 1 -- 172.21.15.159:0/1614845997 learned_addr learned my addr 172.21.15.159:0/1614845997 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:03.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.137+0000 7f0a19600640 1 --2- 172.21.15.159:0/1614845997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09fc003f30 0x7f0a14147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:03.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.138+0000 7f0a18c00640 1 --2- 172.21.15.159:0/1614845997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a141472c0 0x7f0a1415fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:03.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.138+0000 7f0a1a000640 1 -- 172.21.15.159:0/1614845997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a141472c0 msgr2=0x7f0a1415fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:03.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.138+0000 7f0a1a000640 1 --2- 172.21.15.159:0/1614845997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a141472c0 0x7f0a1415fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:03.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.138+0000 7f0a1a000640 1 -- 172.21.15.159:0/1614845997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09fc003f30 msgr2=0x7f0a14147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:03.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.138+0000 7f0a1a000640 1 --2- 172.21.15.159:0/1614845997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09fc003f30 0x7f0a14147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:03.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.138+0000 7f0a1a000640 1 -- 172.21.15.159:0/1614845997 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a14160740 con 0x7f0a141601c0 2024-04-04T06:17:03.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.138+0000 7f0a1a000640 1 --2- 172.21.15.159:0/1614845997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a141601c0 0x7f0a140ae6a0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f0a14071170 tx=0x7f0a0c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:03.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.138+0000 7f0a11600640 1 -- 172.21.15.159:0/1614845997 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c00ec10 con 0x7f0a141601c0 2024-04-04T06:17:03.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.138+0000 7f0a11600640 1 -- 172.21.15.159:0/1614845997 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a0c018070 con 0x7f0a141601c0 2024-04-04T06:17:03.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.138+0000 7f0a11600640 1 -- 172.21.15.159:0/1614845997 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c013420 con 0x7f0a141601c0 2024-04-04T06:17:03.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.138+0000 7f0a1c000640 1 -- 172.21.15.159:0/1614845997 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a140ce770 con 0x7f0a141601c0 2024-04-04T06:17:03.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.138+0000 7f0a1c000640 1 -- 172.21.15.159:0/1614845997 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a140ced30 con 0x7f0a141601c0 2024-04-04T06:17:03.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.140+0000 7f0a1c000640 1 -- 172.21.15.159:0/1614845997 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a140685c0 con 0x7f0a141601c0 2024-04-04T06:17:03.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.141+0000 7f0a11600640 1 -- 172.21.15.159:0/1614845997 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a0c0135e0 con 0x7f0a141601c0 2024-04-04T06:17:03.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.142+0000 7f0a11600640 1 --2- 172.21.15.159:0/1614845997 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f09f406c920 0x7f09f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:03.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.142+0000 7f0a11600640 1 -- 172.21.15.159:0/1614845997 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f0a0c08bfd0 con 0x7f0a141601c0 2024-04-04T06:17:03.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.143+0000 7f0a19600640 1 --2- 172.21.15.159:0/1614845997 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f09f406c920 0x7f09f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:03.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.144+0000 7f0a19600640 1 --2- 172.21.15.159:0/1614845997 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f09f406c920 0x7f09f406ede0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009b20 tx=0x7f0a08008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:03.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.146+0000 7f0a11600640 1 -- 172.21.15.159:0/1614845997 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f0a0c05a360 con 0x7f0a141601c0 2024-04-04T06:17:03.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.344+0000 7f0a1c000640 1 -- 172.21.15.159:0/1614845997 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0a140b4aa0 con 0x7f09f406c920 2024-04-04T06:17:03.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.362+0000 7f0a11600640 1 -- 172.21.15.159:0/1614845997 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252483 (secure 0 0 0) 0x7f0a140b4aa0 con 0x7f09f406c920 2024-04-04T06:17:03.361 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:03.362 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:17:03.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.368+0000 7f0a1c000640 1 -- 172.21.15.159:0/1614845997 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f09f406c920 msgr2=0x7f09f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:03.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.368+0000 7f0a1c000640 1 --2- 172.21.15.159:0/1614845997 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f09f406c920 0x7f09f406ede0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009b20 tx=0x7f0a08008040 comp rx=0 tx=0).stop 2024-04-04T06:17:03.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.369+0000 7f0a1c000640 1 -- 172.21.15.159:0/1614845997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a141601c0 msgr2=0x7f0a140ae6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:03.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.369+0000 7f0a1c000640 1 --2- 172.21.15.159:0/1614845997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a141601c0 0x7f0a140ae6a0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f0a14071170 tx=0x7f0a0c00e530 comp rx=0 tx=0).stop 2024-04-04T06:17:03.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.369+0000 7f0a1c000640 1 -- 172.21.15.159:0/1614845997 shutdown_connections 2024-04-04T06:17:03.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.369+0000 7f0a1c000640 1 --2- 172.21.15.159:0/1614845997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a141601c0 0x7f0a140ae6a0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:03.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.369+0000 7f0a1c000640 1 --2- 172.21.15.159:0/1614845997 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a141472c0 0x7f0a1415fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:03.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.369+0000 7f0a1c000640 1 --2- 172.21.15.159:0/1614845997 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f09f406c920 0x7f09f406ede0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:03.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.369+0000 7f0a1c000640 1 --2- 172.21.15.159:0/1614845997 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09fc003f30 0x7f0a14147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:03.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.369+0000 7f0a1c000640 1 -- 172.21.15.159:0/1614845997 >> 172.21.15.159:0/1614845997 conn(0x7f0a14071df0 msgr2=0x7f0a140b0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:03.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.369+0000 7f0a1c000640 1 -- 172.21.15.159:0/1614845997 shutdown_connections 2024-04-04T06:17:03.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.369+0000 7f0a1c000640 1 -- 172.21.15.159:0/1614845997 wait complete. 2024-04-04T06:17:03.383 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-04-04T06:17:02.963103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278240,"kb_used_data":12136,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754696480,"statfs":{"total":773094113280,"available":772809195520,"internally_reserved":0,"allocated":12427264,"data_stored":6042100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.174190"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378643+0000","last_change":"2024-04-04T06:16:57.861743+0000","last_active":"2024-04-04T06:16:59.378643+0000","last_peered":"2024-04-04T06:16:59.378643+0000","last_clean":"2024-04-04T06:16:59.378643+0000","last_became_active":"2024-04-04T06:16:57.860873+0000","last_became_peered":"2024-04-04T06:16:57.860873+0000","last_unstale":"2024-04-04T06:16:59.378643+0000","last_undegraded":"2024-04-04T06:16:59.378643+0000","last_fullsized":"2024-04-04T06:16:59.378643+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:18:01.077209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856681+0000","last_change":"2024-04-04T06:16:57.850062+0000","last_active":"2024-04-04T06:16:58.856681+0000","last_peered":"2024-04-04T06:16:58.856681+0000","last_clean":"2024-04-04T06:16:58.856681+0000","last_became_active":"2024-04-04T06:16:57.849768+0000","last_became_peered":"2024-04-04T06:16:57.849768+0000","last_unstale":"2024-04-04T06:16:58.856681+0000","last_undegraded":"2024-04-04T06:16:58.856681+0000","last_fullsized":"2024-04-04T06:16:58.856681+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:20:36.274557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378633+0000","last_change":"2024-04-04T06:16:57.857033+0000","last_active":"2024-04-04T06:16:59.378633+0000","last_peered":"2024-04-04T06:16:59.378633+0000","last_clean":"2024-04-04T06:16:59.378633+0000","last_became_active":"2024-04-04T06:16:57.856617+0000","last_became_peered":"2024-04-04T06:16:57.856617+0000","last_unstale":"2024-04-04T06:16:59.378633+0000","last_undegraded":"2024-04-04T06:16:59.378633+0000","last_fullsized":"2024-04-04T06:16:59.378633+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:51:22.883892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.379030+0000","last_change":"2024-04-04T06:16:57.858984+0000","last_active":"2024-04-04T06:16:59.379030+0000","last_peered":"2024-04-04T06:16:59.379030+0000","last_clean":"2024-04-04T06:16:59.379030+0000","last_became_active":"2024-04-04T06:16:57.858427+0000","last_became_peered":"2024-04-04T06:16:57.858427+0000","last_unstale":"2024-04-04T06:16:59.379030+0000","last_undegraded":"2024-04-04T06:16:59.379030+0000","last_fullsized":"2024-04-04T06:16:59.379030+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:08:45.785102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222986+0000","last_change":"2024-04-04T06:16:57.852939+0000","last_active":"2024-04-04T06:16:59.222986+0000","last_peered":"2024-04-04T06:16:59.222986+0000","last_clean":"2024-04-04T06:16:59.222986+0000","last_became_active":"2024-04-04T06:16:57.852656+0000","last_became_peered":"2024-04-04T06:16:57.852656+0000","last_unstale":"2024-04-04T06:16:59.222986+0000","last_undegraded":"2024-04-04T06:16:59.222986+0000","last_fullsized":"2024-04-04T06:16:59.222986+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:04:27.627269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.376770+0000","last_change":"2024-04-04T06:16:57.858617+0000","last_active":"2024-04-04T06:16:59.376770+0000","last_peered":"2024-04-04T06:16:59.376770+0000","last_clean":"2024-04-04T06:16:59.376770+0000","last_became_active":"2024-04-04T06:16:57.858300+0000","last_became_peered":"2024-04-04T06:16:57.858300+0000","last_unstale":"2024-04-04T06:16:59.376770+0000","last_undegraded":"2024-04-04T06:16:59.376770+0000","last_fullsized":"2024-04-04T06:16:59.376770+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:36:49.667730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855381+0000","last_change":"2024-04-04T06:16:57.853261+0000","last_active":"2024-04-04T06:16:58.855381+0000","last_peered":"2024-04-04T06:16:58.855381+0000","last_clean":"2024-04-04T06:16:58.855381+0000","last_became_active":"2024-04-04T06:16:57.852925+0000","last_became_peered":"2024-04-04T06:16:57.852925+0000","last_unstale":"2024-04-04T06:16:58.855381+0000","last_undegraded":"2024-04-04T06:16:58.855381+0000","last_fullsized":"2024-04-04T06:16:58.855381+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:43:19.761558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377985+0000","last_change":"2024-04-04T06:16:57.854647+0000","last_active":"2024-04-04T06:16:59.377985+0000","last_peered":"2024-04-04T06:16:59.377985+0000","last_clean":"2024-04-04T06:16:59.377985+0000","last_became_active":"2024-04-04T06:16:57.854376+0000","last_became_peered":"2024-04-04T06:16:57.854376+0000","last_unstale":"2024-04-04T06:16:59.377985+0000","last_undegraded":"2024-04-04T06:16:59.377985+0000","last_fullsized":"2024-04-04T06:16:59.377985+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:18:08.210854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378876+0000","last_change":"2024-04-04T06:16:57.859251+0000","last_active":"2024-04-04T06:16:59.378876+0000","last_peered":"2024-04-04T06:16:59.378876+0000","last_clean":"2024-04-04T06:16:59.378876+0000","last_became_active":"2024-04-04T06:16:57.857756+0000","last_became_peered":"2024-04-04T06:16:57.857756+0000","last_unstale":"2024-04-04T06:16:59.378876+0000","last_undegraded":"2024-04-04T06:16:59.378876+0000","last_fullsized":"2024-04-04T06:16:59.378876+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:39:44.553729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378627+0000","last_change":"2024-04-04T06:16:57.865147+0000","last_active":"2024-04-04T06:16:59.378627+0000","last_peered":"2024-04-04T06:16:59.378627+0000","last_clean":"2024-04-04T06:16:59.378627+0000","last_became_active":"2024-04-04T06:16:57.863954+0000","last_became_peered":"2024-04-04T06:16:57.863954+0000","last_unstale":"2024-04-04T06:16:59.378627+0000","last_undegraded":"2024-04-04T06:16:59.378627+0000","last_fullsized":"2024-04-04T06:16:59.378627+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:45:25.455248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.857193+0000","last_change":"2024-04-04T06:16:57.852970+0000","last_active":"2024-04-04T06:16:58.857193+0000","last_peered":"2024-04-04T06:16:58.857193+0000","last_clean":"2024-04-04T06:16:58.857193+0000","last_became_active":"2024-04-04T06:16:57.852398+0000","last_became_peered":"2024-04-04T06:16:57.852398+0000","last_unstale":"2024-04-04T06:16:58.857193+0000","last_undegraded":"2024-04-04T06:16:58.857193+0000","last_fullsized":"2024-04-04T06:16:58.857193+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:25:13.863001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.379292+0000","last_change":"2024-04-04T06:16:57.868403+0000","last_active":"2024-04-04T06:16:59.379292+0000","last_peered":"2024-04-04T06:16:59.379292+0000","last_clean":"2024-04-04T06:16:59.379292+0000","last_became_active":"2024-04-04T06:16:57.867661+0000","last_became_peered":"2024-04-04T06:16:57.867661+0000","last_unstale":"2024-04-04T06:16:59.379292+0000","last_undegraded":"2024-04-04T06:16:59.379292+0000","last_fullsized":"2024-04-04T06:16:59.379292+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:49:12.167101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378304+0000","last_change":"2024-04-04T06:16:57.864603+0000","last_active":"2024-04-04T06:16:59.378304+0000","last_peered":"2024-04-04T06:16:59.378304+0000","last_clean":"2024-04-04T06:16:59.378304+0000","last_became_active":"2024-04-04T06:16:57.864334+0000","last_became_peered":"2024-04-04T06:16:57.864334+0000","last_unstale":"2024-04-04T06:16:59.378304+0000","last_undegraded":"2024-04-04T06:16:59.378304+0000","last_fullsized":"2024-04-04T06:16:59.378304+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:54:31.781352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377932+0000","last_change":"2024-04-04T06:16:57.860035+0000","last_active":"2024-04-04T06:16:59.377932+0000","last_peered":"2024-04-04T06:16:59.377932+0000","last_clean":"2024-04-04T06:16:59.377932+0000","last_became_active":"2024-04-04T06:16:57.858072+0000","last_became_peered":"2024-04-04T06:16:57.858072+0000","last_unstale":"2024-04-04T06:16:59.377932+0000","last_undegraded":"2024-04-04T06:16:59.377932+0000","last_fullsized":"2024-04-04T06:16:59.377932+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:52:19.546676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.376787+0000","last_change":"2024-04-04T06:16:57.853280+0000","last_active":"2024-04-04T06:16:59.376787+0000","last_peered":"2024-04-04T06:16:59.376787+0000","last_clean":"2024-04-04T06:16:59.376787+0000","last_became_active":"2024-04-04T06:16:57.852744+0000","last_became_peered":"2024-04-04T06:16:57.852744+0000","last_unstale":"2024-04-04T06:16:59.376787+0000","last_undegraded":"2024-04-04T06:16:59.376787+0000","last_fullsized":"2024-04-04T06:16:59.376787+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:04:27.627269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.376384+0000","last_change":"2024-04-04T06:16:57.866193+0000","last_active":"2024-04-04T06:16:59.376384+0000","last_peered":"2024-04-04T06:16:59.376384+0000","last_clean":"2024-04-04T06:16:59.376384+0000","last_became_active":"2024-04-04T06:16:57.865888+0000","last_became_peered":"2024-04-04T06:16:57.865888+0000","last_unstale":"2024-04-04T06:16:59.376384+0000","last_undegraded":"2024-04-04T06:16:59.376384+0000","last_fullsized":"2024-04-04T06:16:59.376384+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:46:20.508997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856787+0000","last_change":"2024-04-04T06:16:57.861016+0000","last_active":"2024-04-04T06:16:58.856787+0000","last_peered":"2024-04-04T06:16:58.856787+0000","last_clean":"2024-04-04T06:16:58.856787+0000","last_became_active":"2024-04-04T06:16:57.860213+0000","last_became_peered":"2024-04-04T06:16:57.860213+0000","last_unstale":"2024-04-04T06:16:58.856787+0000","last_undegraded":"2024-04-04T06:16:58.856787+0000","last_fullsized":"2024-04-04T06:16:58.856787+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:41:22.250390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.866106+0000","last_change":"2024-04-04T06:16:57.852666+0000","last_active":"2024-04-04T06:16:58.866106+0000","last_peered":"2024-04-04T06:16:58.866106+0000","last_clean":"2024-04-04T06:16:58.866106+0000","last_became_active":"2024-04-04T06:16:57.851465+0000","last_became_peered":"2024-04-04T06:16:57.851465+0000","last_unstale":"2024-04-04T06:16:58.866106+0000","last_undegraded":"2024-04-04T06:16:58.866106+0000","last_fullsized":"2024-04-04T06:16:58.866106+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:54:31.781352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378739+0000","last_change":"2024-04-04T06:16:57.873661+0000","last_active":"2024-04-04T06:16:59.378739+0000","last_peered":"2024-04-04T06:16:59.378739+0000","last_clean":"2024-04-04T06:16:59.378739+0000","last_became_active":"2024-04-04T06:16:57.873342+0000","last_became_peered":"2024-04-04T06:16:57.873342+0000","last_unstale":"2024-04-04T06:16:59.378739+0000","last_undegraded":"2024-04-04T06:16:59.378739+0000","last_fullsized":"2024-04-04T06:16:59.378739+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:05:27.280058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856739+0000","last_change":"2024-04-04T06:16:57.858770+0000","last_active":"2024-04-04T06:16:58.856739+0000","last_peered":"2024-04-04T06:16:58.856739+0000","last_clean":"2024-04-04T06:16:58.856739+0000","last_became_active":"2024-04-04T06:16:57.856816+0000","last_became_peered":"2024-04-04T06:16:57.856816+0000","last_unstale":"2024-04-04T06:16:58.856739+0000","last_undegraded":"2024-04-04T06:16:58.856739+0000","last_fullsized":"2024-04-04T06:16:58.856739+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:42:54.207457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.857204+0000","last_change":"2024-04-04T06:16:57.840896+0000","last_active":"2024-04-04T06:16:58.857204+0000","last_peered":"2024-04-04T06:16:58.857204+0000","last_clean":"2024-04-04T06:16:58.857204+0000","last_became_active":"2024-04-04T06:16:57.840145+0000","last_became_peered":"2024-04-04T06:16:57.840145+0000","last_unstale":"2024-04-04T06:16:58.857204+0000","last_undegraded":"2024-04-04T06:16:58.857204+0000","last_fullsized":"2024-04-04T06:16:58.857204+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:56:07.453527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850326+0000","last_change":"2024-04-04T06:16:57.862396+0000","last_active":"2024-04-04T06:16:58.850326+0000","last_peered":"2024-04-04T06:16:58.850326+0000","last_clean":"2024-04-04T06:16:58.850326+0000","last_became_active":"2024-04-04T06:16:57.862112+0000","last_became_peered":"2024-04-04T06:16:57.862112+0000","last_unstale":"2024-04-04T06:16:58.850326+0000","last_undegraded":"2024-04-04T06:16:58.850326+0000","last_fullsized":"2024-04-04T06:16:58.850326+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:20:36.274557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855555+0000","last_change":"2024-04-04T06:16:57.855453+0000","last_active":"2024-04-04T06:16:58.855555+0000","last_peered":"2024-04-04T06:16:58.855555+0000","last_clean":"2024-04-04T06:16:58.855555+0000","last_became_active":"2024-04-04T06:16:57.855172+0000","last_became_peered":"2024-04-04T06:16:57.855172+0000","last_unstale":"2024-04-04T06:16:58.855555+0000","last_undegraded":"2024-04-04T06:16:58.855555+0000","last_fullsized":"2024-04-04T06:16:58.855555+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:17:03.994940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850188+0000","last_change":"2024-04-04T06:16:57.854852+0000","last_active":"2024-04-04T06:16:58.850188+0000","last_peered":"2024-04-04T06:16:58.850188+0000","last_clean":"2024-04-04T06:16:58.850188+0000","last_became_active":"2024-04-04T06:16:57.854610+0000","last_became_peered":"2024-04-04T06:16:57.854610+0000","last_unstale":"2024-04-04T06:16:58.850188+0000","last_undegraded":"2024-04-04T06:16:58.850188+0000","last_fullsized":"2024-04-04T06:16:58.850188+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:17:03.994940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.857033+0000","last_change":"2024-04-04T06:16:57.859703+0000","last_active":"2024-04-04T06:16:58.857033+0000","last_peered":"2024-04-04T06:16:58.857033+0000","last_clean":"2024-04-04T06:16:58.857033+0000","last_became_active":"2024-04-04T06:16:57.859398+0000","last_became_peered":"2024-04-04T06:16:57.859398+0000","last_unstale":"2024-04-04T06:16:58.857033+0000","last_undegraded":"2024-04-04T06:16:58.857033+0000","last_fullsized":"2024-04-04T06:16:58.857033+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:29:25.780543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378891+0000","last_change":"2024-04-04T06:16:57.861831+0000","last_active":"2024-04-04T06:16:59.378891+0000","last_peered":"2024-04-04T06:16:59.378891+0000","last_clean":"2024-04-04T06:16:59.378891+0000","last_became_active":"2024-04-04T06:16:57.861541+0000","last_became_peered":"2024-04-04T06:16:57.861541+0000","last_unstale":"2024-04-04T06:16:59.378891+0000","last_undegraded":"2024-04-04T06:16:59.378891+0000","last_fullsized":"2024-04-04T06:16:59.378891+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:45:32.671346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378148+0000","last_change":"2024-04-04T06:16:57.871771+0000","last_active":"2024-04-04T06:16:59.378148+0000","last_peered":"2024-04-04T06:16:59.378148+0000","last_clean":"2024-04-04T06:16:59.378148+0000","last_became_active":"2024-04-04T06:16:57.871480+0000","last_became_peered":"2024-04-04T06:16:57.871480+0000","last_unstale":"2024-04-04T06:16:59.378148+0000","last_undegraded":"2024-04-04T06:16:59.378148+0000","last_fullsized":"2024-04-04T06:16:59.378148+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:46:04.591130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850668+0000","last_change":"2024-04-04T06:16:57.867129+0000","last_active":"2024-04-04T06:16:58.850668+0000","last_peered":"2024-04-04T06:16:58.850668+0000","last_clean":"2024-04-04T06:16:58.850668+0000","last_became_active":"2024-04-04T06:16:57.866770+0000","last_became_peered":"2024-04-04T06:16:57.866770+0000","last_unstale":"2024-04-04T06:16:58.850668+0000","last_undegraded":"2024-04-04T06:16:58.850668+0000","last_fullsized":"2024-04-04T06:16:58.850668+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:29:44.470791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856807+0000","last_change":"2024-04-04T06:16:57.859118+0000","last_active":"2024-04-04T06:16:58.856807+0000","last_peered":"2024-04-04T06:16:58.856807+0000","last_clean":"2024-04-04T06:16:58.856807+0000","last_became_active":"2024-04-04T06:16:57.858242+0000","last_became_peered":"2024-04-04T06:16:57.858242+0000","last_unstale":"2024-04-04T06:16:58.856807+0000","last_undegraded":"2024-04-04T06:16:58.856807+0000","last_fullsized":"2024-04-04T06:16:58.856807+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:16:58.119204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855126+0000","last_change":"2024-04-04T06:16:57.838809+0000","last_active":"2024-04-04T06:16:58.855126+0000","last_peered":"2024-04-04T06:16:58.855126+0000","last_clean":"2024-04-04T06:16:58.855126+0000","last_became_active":"2024-04-04T06:16:57.837497+0000","last_became_peered":"2024-04-04T06:16:57.837497+0000","last_unstale":"2024-04-04T06:16:58.855126+0000","last_undegraded":"2024-04-04T06:16:58.855126+0000","last_fullsized":"2024-04-04T06:16:58.855126+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:05:00.308975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377216+0000","last_change":"2024-04-04T06:16:57.863907+0000","last_active":"2024-04-04T06:16:59.377216+0000","last_peered":"2024-04-04T06:16:59.377216+0000","last_clean":"2024-04-04T06:16:59.377216+0000","last_became_active":"2024-04-04T06:16:57.863538+0000","last_became_peered":"2024-04-04T06:16:57.863538+0000","last_unstale":"2024-04-04T06:16:59.377216+0000","last_undegraded":"2024-04-04T06:16:59.377216+0000","last_fullsized":"2024-04-04T06:16:59.377216+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:53:37.135784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.376938+0000","last_change":"2024-04-04T06:16:57.853449+0000","last_active":"2024-04-04T06:16:59.376938+0000","last_peered":"2024-04-04T06:16:59.376938+0000","last_clean":"2024-04-04T06:16:59.376938+0000","last_became_active":"2024-04-04T06:16:57.853171+0000","last_became_peered":"2024-04-04T06:16:57.853171+0000","last_unstale":"2024-04-04T06:16:59.376938+0000","last_undegraded":"2024-04-04T06:16:59.376938+0000","last_fullsized":"2024-04-04T06:16:59.376938+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:27:39.563053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378077+0000","last_change":"2024-04-04T06:16:57.861937+0000","last_active":"2024-04-04T06:16:59.378077+0000","last_peered":"2024-04-04T06:16:59.378077+0000","last_clean":"2024-04-04T06:16:59.378077+0000","last_became_active":"2024-04-04T06:16:57.861668+0000","last_became_peered":"2024-04-04T06:16:57.861668+0000","last_unstale":"2024-04-04T06:16:59.378077+0000","last_undegraded":"2024-04-04T06:16:59.378077+0000","last_fullsized":"2024-04-04T06:16:59.378077+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:38:00.487062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"112'4","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855930+0000","last_change":"2024-04-04T06:16:56.826026+0000","last_active":"2024-04-04T06:16:58.855930+0000","last_peered":"2024-04-04T06:16:58.855930+0000","last_clean":"2024-04-04T06:16:58.855930+0000","last_became_active":"2024-04-04T06:16:56.825749+0000","last_became_peered":"2024-04-04T06:16:56.825749+0000","last_unstale":"2024-04-04T06:16:58.855930+0000","last_undegraded":"2024-04-04T06:16:58.855930+0000","last_fullsized":"2024-04-04T06:16:58.855930+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:44:30.609701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.866566+0000","last_change":"2024-04-04T06:16:57.859524+0000","last_active":"2024-04-04T06:16:58.866566+0000","last_peered":"2024-04-04T06:16:58.866566+0000","last_clean":"2024-04-04T06:16:58.866566+0000","last_became_active":"2024-04-04T06:16:57.859203+0000","last_became_peered":"2024-04-04T06:16:57.859203+0000","last_unstale":"2024-04-04T06:16:58.866566+0000","last_undegraded":"2024-04-04T06:16:58.866566+0000","last_fullsized":"2024-04-04T06:16:58.866566+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:09:22.693053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378327+0000","last_change":"2024-04-04T06:16:56.828520+0000","last_active":"2024-04-04T06:16:59.378327+0000","last_peered":"2024-04-04T06:16:59.378327+0000","last_clean":"2024-04-04T06:16:59.378327+0000","last_became_active":"2024-04-04T06:16:56.828125+0000","last_became_peered":"2024-04-04T06:16:56.828125+0000","last_unstale":"2024-04-04T06:16:59.378327+0000","last_undegraded":"2024-04-04T06:16:59.378327+0000","last_fullsized":"2024-04-04T06:16:59.378327+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:04:22.860294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377506+0000","last_change":"2024-04-04T06:16:57.859001+0000","last_active":"2024-04-04T06:16:59.377506+0000","last_peered":"2024-04-04T06:16:59.377506+0000","last_clean":"2024-04-04T06:16:59.377506+0000","last_became_active":"2024-04-04T06:16:57.858712+0000","last_became_peered":"2024-04-04T06:16:57.858712+0000","last_unstale":"2024-04-04T06:16:59.377506+0000","last_undegraded":"2024-04-04T06:16:59.377506+0000","last_fullsized":"2024-04-04T06:16:59.377506+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:09:33.583383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222894+0000","last_change":"2024-04-04T06:16:56.817318+0000","last_active":"2024-04-04T06:16:59.222894+0000","last_peered":"2024-04-04T06:16:59.222894+0000","last_clean":"2024-04-04T06:16:59.222894+0000","last_became_active":"2024-04-04T06:16:56.816995+0000","last_became_peered":"2024-04-04T06:16:56.816995+0000","last_unstale":"2024-04-04T06:16:59.222894+0000","last_undegraded":"2024-04-04T06:16:59.222894+0000","last_fullsized":"2024-04-04T06:16:59.222894+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:02:07.681535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378954+0000","last_change":"2024-04-04T06:16:57.869420+0000","last_active":"2024-04-04T06:16:59.378954+0000","last_peered":"2024-04-04T06:16:59.378954+0000","last_clean":"2024-04-04T06:16:59.378954+0000","last_became_active":"2024-04-04T06:16:57.869124+0000","last_became_peered":"2024-04-04T06:16:57.869124+0000","last_unstale":"2024-04-04T06:16:59.378954+0000","last_undegraded":"2024-04-04T06:16:59.378954+0000","last_fullsized":"2024-04-04T06:16:59.378954+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:10:39.218040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222549+0000","last_change":"2024-04-04T06:16:56.813821+0000","last_active":"2024-04-04T06:16:59.222549+0000","last_peered":"2024-04-04T06:16:59.222549+0000","last_clean":"2024-04-04T06:16:59.222549+0000","last_became_active":"2024-04-04T06:16:56.813555+0000","last_became_peered":"2024-04-04T06:16:56.813555+0000","last_unstale":"2024-04-04T06:16:59.222549+0000","last_undegraded":"2024-04-04T06:16:59.222549+0000","last_fullsized":"2024-04-04T06:16:59.222549+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:22:21.703895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377925+0000","last_change":"2024-04-04T06:16:57.858500+0000","last_active":"2024-04-04T06:16:59.377925+0000","last_peered":"2024-04-04T06:16:59.377925+0000","last_clean":"2024-04-04T06:16:59.377925+0000","last_became_active":"2024-04-04T06:16:57.856744+0000","last_became_peered":"2024-04-04T06:16:57.856744+0000","last_unstale":"2024-04-04T06:16:59.377925+0000","last_undegraded":"2024-04-04T06:16:59.377925+0000","last_fullsized":"2024-04-04T06:16:59.377925+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:44:29.963390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854845+0000","last_change":"2024-04-04T06:16:56.813703+0000","last_active":"2024-04-04T06:16:58.854845+0000","last_peered":"2024-04-04T06:16:58.854845+0000","last_clean":"2024-04-04T06:16:58.854845+0000","last_became_active":"2024-04-04T06:16:56.813224+0000","last_became_peered":"2024-04-04T06:16:56.813224+0000","last_unstale":"2024-04-04T06:16:58.854845+0000","last_undegraded":"2024-04-04T06:16:58.854845+0000","last_fullsized":"2024-04-04T06:16:58.854845+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:28:44.071981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856364+0000","last_change":"2024-04-04T06:16:57.851938+0000","last_active":"2024-04-04T06:16:58.856364+0000","last_peered":"2024-04-04T06:16:58.856364+0000","last_clean":"2024-04-04T06:16:58.856364+0000","last_became_active":"2024-04-04T06:16:57.850346+0000","last_became_peered":"2024-04-04T06:16:57.850346+0000","last_unstale":"2024-04-04T06:16:58.856364+0000","last_undegraded":"2024-04-04T06:16:58.856364+0000","last_fullsized":"2024-04-04T06:16:58.856364+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:42:36.526974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850561+0000","last_change":"2024-04-04T06:16:56.822599+0000","last_active":"2024-04-04T06:16:58.850561+0000","last_peered":"2024-04-04T06:16:58.850561+0000","last_clean":"2024-04-04T06:16:58.850561+0000","last_became_active":"2024-04-04T06:16:56.822130+0000","last_became_peered":"2024-04-04T06:16:56.822130+0000","last_unstale":"2024-04-04T06:16:58.850561+0000","last_undegraded":"2024-04-04T06:16:58.850561+0000","last_fullsized":"2024-04-04T06:16:58.850561+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:59:42.200129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.376345+0000","last_change":"2024-04-04T06:16:57.849755+0000","last_active":"2024-04-04T06:16:59.376345+0000","last_peered":"2024-04-04T06:16:59.376345+0000","last_clean":"2024-04-04T06:16:59.376345+0000","last_became_active":"2024-04-04T06:16:57.849341+0000","last_became_peered":"2024-04-04T06:16:57.849341+0000","last_unstale":"2024-04-04T06:16:59.376345+0000","last_undegraded":"2024-04-04T06:16:59.376345+0000","last_fullsized":"2024-04-04T06:16:59.376345+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:52:09.021010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855249+0000","last_change":"2024-04-04T06:16:56.822615+0000","last_active":"2024-04-04T06:16:58.855249+0000","last_peered":"2024-04-04T06:16:58.855249+0000","last_clean":"2024-04-04T06:16:58.855249+0000","last_became_active":"2024-04-04T06:16:56.819761+0000","last_became_peered":"2024-04-04T06:16:56.819761+0000","last_unstale":"2024-04-04T06:16:58.855249+0000","last_undegraded":"2024-04-04T06:16:58.855249+0000","last_fullsized":"2024-04-04T06:16:58.855249+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:10:06.219773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854643+0000","last_change":"2024-04-04T06:16:57.853254+0000","last_active":"2024-04-04T06:16:58.854643+0000","last_peered":"2024-04-04T06:16:58.854643+0000","last_clean":"2024-04-04T06:16:58.854643+0000","last_became_active":"2024-04-04T06:16:57.852500+0000","last_became_peered":"2024-04-04T06:16:57.852500+0000","last_unstale":"2024-04-04T06:16:58.854643+0000","last_undegraded":"2024-04-04T06:16:58.854643+0000","last_fullsized":"2024-04-04T06:16:58.854643+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:47.042156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.376485+0000","last_change":"2024-04-04T06:16:56.816173+0000","last_active":"2024-04-04T06:16:59.376485+0000","last_peered":"2024-04-04T06:16:59.376485+0000","last_clean":"2024-04-04T06:16:59.376485+0000","last_became_active":"2024-04-04T06:16:56.815796+0000","last_became_peered":"2024-04-04T06:16:56.815796+0000","last_unstale":"2024-04-04T06:16:59.376485+0000","last_undegraded":"2024-04-04T06:16:59.376485+0000","last_fullsized":"2024-04-04T06:16:59.376485+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:12:54.232107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856214+0000","last_change":"2024-04-04T06:16:57.863523+0000","last_active":"2024-04-04T06:16:58.856214+0000","last_peered":"2024-04-04T06:16:58.856214+0000","last_clean":"2024-04-04T06:16:58.856214+0000","last_became_active":"2024-04-04T06:16:57.861142+0000","last_became_peered":"2024-04-04T06:16:57.861142+0000","last_unstale":"2024-04-04T06:16:58.856214+0000","last_undegraded":"2024-04-04T06:16:58.856214+0000","last_fullsized":"2024-04-04T06:16:58.856214+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:50:12.226068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222684+0000","last_change":"2024-04-04T06:16:56.817070+0000","last_active":"2024-04-04T06:16:59.222684+0000","last_peered":"2024-04-04T06:16:59.222684+0000","last_clean":"2024-04-04T06:16:59.222684+0000","last_became_active":"2024-04-04T06:16:56.816742+0000","last_became_peered":"2024-04-04T06:16:56.816742+0000","last_unstale":"2024-04-04T06:16:59.222684+0000","last_undegraded":"2024-04-04T06:16:59.222684+0000","last_fullsized":"2024-04-04T06:16:59.222684+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:10:49.284692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.866544+0000","last_change":"2024-04-04T06:16:57.865580+0000","last_active":"2024-04-04T06:16:58.866544+0000","last_peered":"2024-04-04T06:16:58.866544+0000","last_clean":"2024-04-04T06:16:58.866544+0000","last_became_active":"2024-04-04T06:16:57.863731+0000","last_became_peered":"2024-04-04T06:16:57.863731+0000","last_unstale":"2024-04-04T06:16:58.866544+0000","last_undegraded":"2024-04-04T06:16:58.866544+0000","last_fullsized":"2024-04-04T06:16:58.866544+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:05:27.280058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850433+0000","last_change":"2024-04-04T06:16:56.822568+0000","last_active":"2024-04-04T06:16:58.850433+0000","last_peered":"2024-04-04T06:16:58.850433+0000","last_clean":"2024-04-04T06:16:58.850433+0000","last_became_active":"2024-04-04T06:16:56.822237+0000","last_became_peered":"2024-04-04T06:16:56.822237+0000","last_unstale":"2024-04-04T06:16:58.850433+0000","last_undegraded":"2024-04-04T06:16:58.850433+0000","last_fullsized":"2024-04-04T06:16:58.850433+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:13:56.182899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850676+0000","last_change":"2024-04-04T06:16:57.851850+0000","last_active":"2024-04-04T06:16:58.850676+0000","last_peered":"2024-04-04T06:16:58.850676+0000","last_clean":"2024-04-04T06:16:58.850676+0000","last_became_active":"2024-04-04T06:16:57.851529+0000","last_became_peered":"2024-04-04T06:16:57.851529+0000","last_unstale":"2024-04-04T06:16:58.850676+0000","last_undegraded":"2024-04-04T06:16:58.850676+0000","last_fullsized":"2024-04-04T06:16:58.850676+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:16:58.119204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"112'3","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378418+0000","last_change":"2024-04-04T06:16:56.811481+0000","last_active":"2024-04-04T06:16:59.378418+0000","last_peered":"2024-04-04T06:16:59.378418+0000","last_clean":"2024-04-04T06:16:59.378418+0000","last_became_active":"2024-04-04T06:16:56.811190+0000","last_became_peered":"2024-04-04T06:16:56.811190+0000","last_unstale":"2024-04-04T06:16:59.378418+0000","last_undegraded":"2024-04-04T06:16:59.378418+0000","last_fullsized":"2024-04-04T06:16:59.378418+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:41:11.586175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855312+0000","last_change":"2024-04-04T06:16:57.855846+0000","last_active":"2024-04-04T06:16:58.855312+0000","last_peered":"2024-04-04T06:16:58.855312+0000","last_clean":"2024-04-04T06:16:58.855312+0000","last_became_active":"2024-04-04T06:16:57.855487+0000","last_became_peered":"2024-04-04T06:16:57.855487+0000","last_unstale":"2024-04-04T06:16:58.855312+0000","last_undegraded":"2024-04-04T06:16:58.855312+0000","last_fullsized":"2024-04-04T06:16:58.855312+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:25:12.522619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"112'6","reported_seq":20,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.379530+0000","last_change":"2024-04-04T06:16:56.816249+0000","last_active":"2024-04-04T06:16:59.379530+0000","last_peered":"2024-04-04T06:16:59.379530+0000","last_clean":"2024-04-04T06:16:59.379530+0000","last_became_active":"2024-04-04T06:16:56.815980+0000","last_became_peered":"2024-04-04T06:16:56.815980+0000","last_unstale":"2024-04-04T06:16:59.379530+0000","last_undegraded":"2024-04-04T06:16:59.379530+0000","last_fullsized":"2024-04-04T06:16:59.379530+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:20:35.260197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850475+0000","last_change":"2024-04-04T06:16:57.852377+0000","last_active":"2024-04-04T06:16:58.850475+0000","last_peered":"2024-04-04T06:16:58.850475+0000","last_clean":"2024-04-04T06:16:58.850475+0000","last_became_active":"2024-04-04T06:16:57.850593+0000","last_became_peered":"2024-04-04T06:16:57.850593+0000","last_unstale":"2024-04-04T06:16:58.850475+0000","last_undegraded":"2024-04-04T06:16:58.850475+0000","last_fullsized":"2024-04-04T06:16:58.850475+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:41:22.250390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377300+0000","last_change":"2024-04-04T06:16:56.825811+0000","last_active":"2024-04-04T06:16:59.377300+0000","last_peered":"2024-04-04T06:16:59.377300+0000","last_clean":"2024-04-04T06:16:59.377300+0000","last_became_active":"2024-04-04T06:16:56.825491+0000","last_became_peered":"2024-04-04T06:16:56.825491+0000","last_unstale":"2024-04-04T06:16:59.377300+0000","last_undegraded":"2024-04-04T06:16:59.377300+0000","last_fullsized":"2024-04-04T06:16:59.377300+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:22:21.703895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850892+0000","last_change":"2024-04-04T06:16:57.854997+0000","last_active":"2024-04-04T06:16:58.850892+0000","last_peered":"2024-04-04T06:16:58.850892+0000","last_clean":"2024-04-04T06:16:58.850892+0000","last_became_active":"2024-04-04T06:16:57.854710+0000","last_became_peered":"2024-04-04T06:16:57.854710+0000","last_unstale":"2024-04-04T06:16:58.850892+0000","last_undegraded":"2024-04-04T06:16:58.850892+0000","last_fullsized":"2024-04-04T06:16:58.850892+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:29:25.780543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.865913+0000","last_change":"2024-04-04T06:16:56.808070+0000","last_active":"2024-04-04T06:16:58.865913+0000","last_peered":"2024-04-04T06:16:58.865913+0000","last_clean":"2024-04-04T06:16:58.865913+0000","last_became_active":"2024-04-04T06:16:56.807074+0000","last_became_peered":"2024-04-04T06:16:56.807074+0000","last_unstale":"2024-04-04T06:16:58.865913+0000","last_undegraded":"2024-04-04T06:16:58.865913+0000","last_fullsized":"2024-04-04T06:16:58.865913+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:05:44.678207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850292+0000","last_change":"2024-04-04T06:16:57.840280+0000","last_active":"2024-04-04T06:16:58.850292+0000","last_peered":"2024-04-04T06:16:58.850292+0000","last_clean":"2024-04-04T06:16:58.850292+0000","last_became_active":"2024-04-04T06:16:57.839601+0000","last_became_peered":"2024-04-04T06:16:57.839601+0000","last_unstale":"2024-04-04T06:16:58.850292+0000","last_undegraded":"2024-04-04T06:16:58.850292+0000","last_fullsized":"2024-04-04T06:16:58.850292+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:41:12.600534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"112'8","reported_seq":22,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222426+0000","last_change":"2024-04-04T06:16:56.814462+0000","last_active":"2024-04-04T06:16:59.222426+0000","last_peered":"2024-04-04T06:16:59.222426+0000","last_clean":"2024-04-04T06:16:59.222426+0000","last_became_active":"2024-04-04T06:16:56.813042+0000","last_became_peered":"2024-04-04T06:16:56.813042+0000","last_unstale":"2024-04-04T06:16:59.222426+0000","last_undegraded":"2024-04-04T06:16:59.222426+0000","last_fullsized":"2024-04-04T06:16:59.222426+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:13:39.485055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222788+0000","last_change":"2024-04-04T06:16:57.852435+0000","last_active":"2024-04-04T06:16:59.222788+0000","last_peered":"2024-04-04T06:16:59.222788+0000","last_clean":"2024-04-04T06:16:59.222788+0000","last_became_active":"2024-04-04T06:16:57.851997+0000","last_became_peered":"2024-04-04T06:16:57.851997+0000","last_unstale":"2024-04-04T06:16:59.222788+0000","last_undegraded":"2024-04-04T06:16:59.222788+0000","last_fullsized":"2024-04-04T06:16:59.222788+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:36:49.667730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.379260+0000","last_change":"2024-04-04T06:16:56.812543+0000","last_active":"2024-04-04T06:16:59.379260+0000","last_peered":"2024-04-04T06:16:59.379260+0000","last_clean":"2024-04-04T06:16:59.379260+0000","last_became_active":"2024-04-04T06:16:56.812240+0000","last_became_peered":"2024-04-04T06:16:56.812240+0000","last_unstale":"2024-04-04T06:16:59.379260+0000","last_undegraded":"2024-04-04T06:16:59.379260+0000","last_fullsized":"2024-04-04T06:16:59.379260+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:25:12.848641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377607+0000","last_change":"2024-04-04T06:16:57.863716+0000","last_active":"2024-04-04T06:16:59.377607+0000","last_peered":"2024-04-04T06:16:59.377607+0000","last_clean":"2024-04-04T06:16:59.377607+0000","last_became_active":"2024-04-04T06:16:57.862780+0000","last_became_peered":"2024-04-04T06:16:57.862780+0000","last_unstale":"2024-04-04T06:16:59.377607+0000","last_undegraded":"2024-04-04T06:16:59.377607+0000","last_fullsized":"2024-04-04T06:16:59.377607+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:56:59.538463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222812+0000","last_change":"2024-04-04T06:16:56.822207+0000","last_active":"2024-04-04T06:16:59.222812+0000","last_peered":"2024-04-04T06:16:59.222812+0000","last_clean":"2024-04-04T06:16:59.222812+0000","last_became_active":"2024-04-04T06:16:56.821906+0000","last_became_peered":"2024-04-04T06:16:56.821906+0000","last_unstale":"2024-04-04T06:16:59.222812+0000","last_undegraded":"2024-04-04T06:16:59.222812+0000","last_fullsized":"2024-04-04T06:16:59.222812+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:52:52.166280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378788+0000","last_change":"2024-04-04T06:16:57.865766+0000","last_active":"2024-04-04T06:16:59.378788+0000","last_peered":"2024-04-04T06:16:59.378788+0000","last_clean":"2024-04-04T06:16:59.378788+0000","last_became_active":"2024-04-04T06:16:57.865457+0000","last_became_peered":"2024-04-04T06:16:57.865457+0000","last_unstale":"2024-04-04T06:16:59.378788+0000","last_undegraded":"2024-04-04T06:16:59.378788+0000","last_fullsized":"2024-04-04T06:16:59.378788+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:09:22.693053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.866035+0000","last_change":"2024-04-04T06:16:56.818843+0000","last_active":"2024-04-04T06:16:58.866035+0000","last_peered":"2024-04-04T06:16:58.866035+0000","last_clean":"2024-04-04T06:16:58.866035+0000","last_became_active":"2024-04-04T06:16:56.818532+0000","last_became_peered":"2024-04-04T06:16:56.818532+0000","last_unstale":"2024-04-04T06:16:58.866035+0000","last_undegraded":"2024-04-04T06:16:58.866035+0000","last_fullsized":"2024-04-04T06:16:58.866035+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:31:12.755633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.379063+0000","last_change":"2024-04-04T06:16:57.859384+0000","last_active":"2024-04-04T06:16:59.379063+0000","last_peered":"2024-04-04T06:16:59.379063+0000","last_clean":"2024-04-04T06:16:59.379063+0000","last_became_active":"2024-04-04T06:16:57.859026+0000","last_became_peered":"2024-04-04T06:16:57.859026+0000","last_unstale":"2024-04-04T06:16:59.379063+0000","last_undegraded":"2024-04-04T06:16:59.379063+0000","last_fullsized":"2024-04-04T06:16:59.379063+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:03:57.236044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854548+0000","last_change":"2024-04-04T06:16:56.817026+0000","last_active":"2024-04-04T06:16:58.854548+0000","last_peered":"2024-04-04T06:16:58.854548+0000","last_clean":"2024-04-04T06:16:58.854548+0000","last_became_active":"2024-04-04T06:16:56.816488+0000","last_became_peered":"2024-04-04T06:16:56.816488+0000","last_unstale":"2024-04-04T06:16:58.854548+0000","last_undegraded":"2024-04-04T06:16:58.854548+0000","last_fullsized":"2024-04-04T06:16:58.854548+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:11:49.042689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377911+0000","last_change":"2024-04-04T06:16:57.859829+0000","last_active":"2024-04-04T06:16:59.377911+0000","last_peered":"2024-04-04T06:16:59.377911+0000","last_clean":"2024-04-04T06:16:59.377911+0000","last_became_active":"2024-04-04T06:16:57.859535+0000","last_became_peered":"2024-04-04T06:16:57.859535+0000","last_unstale":"2024-04-04T06:16:59.377911+0000","last_undegraded":"2024-04-04T06:16:59.377911+0000","last_fullsized":"2024-04-04T06:16:59.377911+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:27:35.132513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377508+0000","last_change":"2024-04-04T06:16:56.821203+0000","last_active":"2024-04-04T06:16:59.377508+0000","last_peered":"2024-04-04T06:16:59.377508+0000","last_clean":"2024-04-04T06:16:59.377508+0000","last_became_active":"2024-04-04T06:16:56.820739+0000","last_became_peered":"2024-04-04T06:16:56.820739+0000","last_unstale":"2024-04-04T06:16:59.377508+0000","last_undegraded":"2024-04-04T06:16:59.377508+0000","last_fullsized":"2024-04-04T06:16:59.377508+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:13:39.485055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378104+0000","last_change":"2024-04-04T06:16:57.862428+0000","last_active":"2024-04-04T06:16:59.378104+0000","last_peered":"2024-04-04T06:16:59.378104+0000","last_clean":"2024-04-04T06:16:59.378104+0000","last_became_active":"2024-04-04T06:16:57.862160+0000","last_became_peered":"2024-04-04T06:16:57.862160+0000","last_unstale":"2024-04-04T06:16:59.378104+0000","last_undegraded":"2024-04-04T06:16:59.378104+0000","last_fullsized":"2024-04-04T06:16:59.378104+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:15:07.419885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222426+0000","last_change":"2024-04-04T06:16:56.816260+0000","last_active":"2024-04-04T06:16:59.222426+0000","last_peered":"2024-04-04T06:16:59.222426+0000","last_clean":"2024-04-04T06:16:59.222426+0000","last_became_active":"2024-04-04T06:16:56.815988+0000","last_became_peered":"2024-04-04T06:16:56.815988+0000","last_unstale":"2024-04-04T06:16:59.222426+0000","last_undegraded":"2024-04-04T06:16:59.222426+0000","last_fullsized":"2024-04-04T06:16:59.222426+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:28.483864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377989+0000","last_change":"2024-04-04T06:16:57.862761+0000","last_active":"2024-04-04T06:16:59.377989+0000","last_peered":"2024-04-04T06:16:59.377989+0000","last_clean":"2024-04-04T06:16:59.377989+0000","last_became_active":"2024-04-04T06:16:57.862519+0000","last_became_peered":"2024-04-04T06:16:57.862519+0000","last_unstale":"2024-04-04T06:16:59.377989+0000","last_undegraded":"2024-04-04T06:16:59.377989+0000","last_fullsized":"2024-04-04T06:16:59.377989+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:16:04.429520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377028+0000","last_change":"2024-04-04T06:16:56.823585+0000","last_active":"2024-04-04T06:16:59.377028+0000","last_peered":"2024-04-04T06:16:59.377028+0000","last_clean":"2024-04-04T06:16:59.377028+0000","last_became_active":"2024-04-04T06:16:56.823294+0000","last_became_peered":"2024-04-04T06:16:56.823294+0000","last_unstale":"2024-04-04T06:16:59.377028+0000","last_undegraded":"2024-04-04T06:16:59.377028+0000","last_fullsized":"2024-04-04T06:16:59.377028+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:02:07.681535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850448+0000","last_change":"2024-04-04T06:16:57.868618+0000","last_active":"2024-04-04T06:16:58.850448+0000","last_peered":"2024-04-04T06:16:58.850448+0000","last_clean":"2024-04-04T06:16:58.850448+0000","last_became_active":"2024-04-04T06:16:57.866435+0000","last_became_peered":"2024-04-04T06:16:57.866435+0000","last_unstale":"2024-04-04T06:16:58.850448+0000","last_undegraded":"2024-04-04T06:16:58.850448+0000","last_fullsized":"2024-04-04T06:16:58.850448+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:42:54.207457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.376665+0000","last_change":"2024-04-04T06:16:56.806783+0000","last_active":"2024-04-04T06:16:59.376665+0000","last_peered":"2024-04-04T06:16:59.376665+0000","last_clean":"2024-04-04T06:16:59.376665+0000","last_became_active":"2024-04-04T06:16:56.806293+0000","last_became_peered":"2024-04-04T06:16:56.806293+0000","last_unstale":"2024-04-04T06:16:59.376665+0000","last_undegraded":"2024-04-04T06:16:59.376665+0000","last_fullsized":"2024-04-04T06:16:59.376665+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:56:06.439167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.866304+0000","last_change":"2024-04-04T06:16:57.867317+0000","last_active":"2024-04-04T06:16:58.866304+0000","last_peered":"2024-04-04T06:16:58.866304+0000","last_clean":"2024-04-04T06:16:58.866304+0000","last_became_active":"2024-04-04T06:16:57.866053+0000","last_became_peered":"2024-04-04T06:16:57.866053+0000","last_unstale":"2024-04-04T06:16:58.866304+0000","last_undegraded":"2024-04-04T06:16:58.866304+0000","last_fullsized":"2024-04-04T06:16:58.866304+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:51:22.883892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850667+0000","last_change":"2024-04-04T06:16:56.817822+0000","last_active":"2024-04-04T06:16:58.850667+0000","last_peered":"2024-04-04T06:16:58.850667+0000","last_clean":"2024-04-04T06:16:58.850667+0000","last_became_active":"2024-04-04T06:16:56.816539+0000","last_became_peered":"2024-04-04T06:16:56.816539+0000","last_unstale":"2024-04-04T06:16:58.850667+0000","last_undegraded":"2024-04-04T06:16:58.850667+0000","last_fullsized":"2024-04-04T06:16:58.850667+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:10:06.219773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.379113+0000","last_change":"2024-04-04T06:16:57.859002+0000","last_active":"2024-04-04T06:16:59.379113+0000","last_peered":"2024-04-04T06:16:59.379113+0000","last_clean":"2024-04-04T06:16:59.379113+0000","last_became_active":"2024-04-04T06:16:57.857269+0000","last_became_peered":"2024-04-04T06:16:57.857269+0000","last_unstale":"2024-04-04T06:16:59.379113+0000","last_undegraded":"2024-04-04T06:16:59.379113+0000","last_fullsized":"2024-04-04T06:16:59.379113+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:05:24.301901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"113'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.866526+0000","last_change":"2024-04-04T06:16:56.819463+0000","last_active":"2024-04-04T06:16:58.866526+0000","last_peered":"2024-04-04T06:16:58.866526+0000","last_clean":"2024-04-04T06:16:58.866526+0000","last_became_active":"2024-04-04T06:16:56.816034+0000","last_became_peered":"2024-04-04T06:16:56.816034+0000","last_unstale":"2024-04-04T06:16:58.866526+0000","last_undegraded":"2024-04-04T06:16:58.866526+0000","last_fullsized":"2024-04-04T06:16:58.866526+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:14:56.497336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.379427+0000","last_change":"2024-04-04T06:16:57.856907+0000","last_active":"2024-04-04T06:16:59.379427+0000","last_peered":"2024-04-04T06:16:59.379427+0000","last_clean":"2024-04-04T06:16:59.379427+0000","last_became_active":"2024-04-04T06:16:57.856616+0000","last_became_peered":"2024-04-04T06:16:57.856616+0000","last_unstale":"2024-04-04T06:16:59.379427+0000","last_undegraded":"2024-04-04T06:16:59.379427+0000","last_fullsized":"2024-04-04T06:16:59.379427+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:34:43.213737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378572+0000","last_change":"2024-04-04T06:16:56.803667+0000","last_active":"2024-04-04T06:16:59.378572+0000","last_peered":"2024-04-04T06:16:59.378572+0000","last_clean":"2024-04-04T06:16:59.378572+0000","last_became_active":"2024-04-04T06:16:56.803353+0000","last_became_peered":"2024-04-04T06:16:56.803353+0000","last_unstale":"2024-04-04T06:16:59.378572+0000","last_undegraded":"2024-04-04T06:16:59.378572+0000","last_fullsized":"2024-04-04T06:16:59.378572+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:42:35.512614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.866190+0000","last_change":"2024-04-04T06:16:57.870240+0000","last_active":"2024-04-04T06:16:58.866190+0000","last_peered":"2024-04-04T06:16:58.866190+0000","last_clean":"2024-04-04T06:16:58.866190+0000","last_became_active":"2024-04-04T06:16:57.869874+0000","last_became_peered":"2024-04-04T06:16:57.869874+0000","last_unstale":"2024-04-04T06:16:58.866190+0000","last_undegraded":"2024-04-04T06:16:58.866190+0000","last_fullsized":"2024-04-04T06:16:58.866190+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:01:10.335447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850959+0000","last_change":"2024-04-04T06:16:56.820108+0000","last_active":"2024-04-04T06:16:58.850959+0000","last_peered":"2024-04-04T06:16:58.850959+0000","last_clean":"2024-04-04T06:16:58.850959+0000","last_became_active":"2024-04-04T06:16:56.819768+0000","last_became_peered":"2024-04-04T06:16:56.819768+0000","last_unstale":"2024-04-04T06:16:58.850959+0000","last_undegraded":"2024-04-04T06:16:58.850959+0000","last_fullsized":"2024-04-04T06:16:58.850959+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:19:35.089037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855977+0000","last_change":"2024-04-04T06:16:57.844825+0000","last_active":"2024-04-04T06:16:58.855977+0000","last_peered":"2024-04-04T06:16:58.855977+0000","last_clean":"2024-04-04T06:16:58.855977+0000","last_became_active":"2024-04-04T06:16:57.843101+0000","last_became_peered":"2024-04-04T06:16:57.843101+0000","last_unstale":"2024-04-04T06:16:58.855977+0000","last_undegraded":"2024-04-04T06:16:58.855977+0000","last_fullsized":"2024-04-04T06:16:58.855977+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:41:12.600534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854882+0000","last_change":"2024-04-04T06:16:56.823376+0000","last_active":"2024-04-04T06:16:58.854882+0000","last_peered":"2024-04-04T06:16:58.854882+0000","last_clean":"2024-04-04T06:16:58.854882+0000","last_became_active":"2024-04-04T06:16:56.822325+0000","last_became_peered":"2024-04-04T06:16:56.822325+0000","last_unstale":"2024-04-04T06:16:58.854882+0000","last_undegraded":"2024-04-04T06:16:58.854882+0000","last_fullsized":"2024-04-04T06:16:58.854882+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:17:31.397079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856868+0000","last_change":"2024-04-04T06:16:57.861595+0000","last_active":"2024-04-04T06:16:58.856868+0000","last_peered":"2024-04-04T06:16:58.856868+0000","last_clean":"2024-04-04T06:16:58.856868+0000","last_became_active":"2024-04-04T06:16:57.860975+0000","last_became_peered":"2024-04-04T06:16:57.860975+0000","last_unstale":"2024-04-04T06:16:58.856868+0000","last_undegraded":"2024-04-04T06:16:58.856868+0000","last_fullsized":"2024-04-04T06:16:58.856868+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:29:44.470791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855757+0000","last_change":"2024-04-04T06:16:56.822556+0000","last_active":"2024-04-04T06:16:58.855757+0000","last_peered":"2024-04-04T06:16:58.855757+0000","last_clean":"2024-04-04T06:16:58.855757+0000","last_became_active":"2024-04-04T06:16:56.821979+0000","last_became_peered":"2024-04-04T06:16:56.821979+0000","last_unstale":"2024-04-04T06:16:58.855757+0000","last_undegraded":"2024-04-04T06:16:58.855757+0000","last_fullsized":"2024-04-04T06:16:58.855757+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:59:42.200129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378485+0000","last_change":"2024-04-04T06:16:57.861874+0000","last_active":"2024-04-04T06:16:59.378485+0000","last_peered":"2024-04-04T06:16:59.378485+0000","last_clean":"2024-04-04T06:16:59.378485+0000","last_became_active":"2024-04-04T06:16:57.861605+0000","last_became_peered":"2024-04-04T06:16:57.861605+0000","last_unstale":"2024-04-04T06:16:59.378485+0000","last_undegraded":"2024-04-04T06:16:59.378485+0000","last_fullsized":"2024-04-04T06:16:59.378485+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:01:10.335447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222503+0000","last_change":"2024-04-04T06:16:56.818500+0000","last_active":"2024-04-04T06:16:59.222503+0000","last_peered":"2024-04-04T06:16:59.222503+0000","last_clean":"2024-04-04T06:16:59.222503+0000","last_became_active":"2024-04-04T06:16:56.816829+0000","last_became_peered":"2024-04-04T06:16:56.816829+0000","last_unstale":"2024-04-04T06:16:59.222503+0000","last_undegraded":"2024-04-04T06:16:59.222503+0000","last_fullsized":"2024-04-04T06:16:59.222503+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:37:13.781466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377554+0000","last_change":"2024-04-04T06:16:57.862871+0000","last_active":"2024-04-04T06:16:59.377554+0000","last_peered":"2024-04-04T06:16:59.377554+0000","last_clean":"2024-04-04T06:16:59.377554+0000","last_became_active":"2024-04-04T06:16:57.857674+0000","last_became_peered":"2024-04-04T06:16:57.857674+0000","last_unstale":"2024-04-04T06:16:59.377554+0000","last_undegraded":"2024-04-04T06:16:59.377554+0000","last_fullsized":"2024-04-04T06:16:59.377554+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:12:53.084370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377182+0000","last_change":"2024-04-04T06:16:56.807000+0000","last_active":"2024-04-04T06:16:59.377182+0000","last_peered":"2024-04-04T06:16:59.377182+0000","last_clean":"2024-04-04T06:16:59.377182+0000","last_became_active":"2024-04-04T06:16:56.806687+0000","last_became_peered":"2024-04-04T06:16:56.806687+0000","last_unstale":"2024-04-04T06:16:59.377182+0000","last_undegraded":"2024-04-04T06:16:59.377182+0000","last_fullsized":"2024-04-04T06:16:59.377182+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:02:40.213409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850291+0000","last_change":"2024-04-04T06:16:57.856787+0000","last_active":"2024-04-04T06:16:58.850291+0000","last_peered":"2024-04-04T06:16:58.850291+0000","last_clean":"2024-04-04T06:16:58.850291+0000","last_became_active":"2024-04-04T06:16:57.856455+0000","last_became_peered":"2024-04-04T06:16:57.856455+0000","last_unstale":"2024-04-04T06:16:58.850291+0000","last_undegraded":"2024-04-04T06:16:58.850291+0000","last_fullsized":"2024-04-04T06:16:58.850291+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:42:36.526974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854824+0000","last_change":"2024-04-04T06:16:56.819104+0000","last_active":"2024-04-04T06:16:58.854824+0000","last_peered":"2024-04-04T06:16:58.854824+0000","last_clean":"2024-04-04T06:16:58.854824+0000","last_became_active":"2024-04-04T06:16:56.818492+0000","last_became_peered":"2024-04-04T06:16:56.818492+0000","last_unstale":"2024-04-04T06:16:58.854824+0000","last_undegraded":"2024-04-04T06:16:58.854824+0000","last_fullsized":"2024-04-04T06:16:58.854824+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:14:15.299577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":338,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378631+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:16:59.378631+0000","last_peered":"2024-04-04T06:16:59.378631+0000","last_clean":"2024-04-04T06:16:59.378631+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:16:59.378631+0000","last_undegraded":"2024-04-04T06:16:59.378631+0000","last_fullsized":"2024-04-04T06:16:59.378631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509578,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37296,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334544,"statfs":{"total":96636764160,"available":96598573056,"internally_reserved":0,"allocated":1998848,"data_stored":1198052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509580,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36712,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335128,"statfs":{"total":96636764160,"available":96599171072,"internally_reserved":0,"allocated":1400832,"data_stored":607138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509581,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1413120,"data_stored":608750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509580,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1396736,"data_stored":607068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509579,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1404928,"data_stored":607192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509581,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2011136,"data_stored":1199160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36712,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335128,"statfs":{"total":96636764160,"available":96599171072,"internally_reserved":0,"allocated":1404928,"data_stored":607672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542282,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36712,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335128,"statfs":{"total":96636764160,"available":96599171072,"internally_reserved":0,"allocated":1396736,"data_stored":607068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:17:03.426 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-04T06:17:03.630 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.635+0000 7fc3eac00640 1 Processor -- start 2024-04-04T06:17:03.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.636+0000 7fc3eac00640 1 -- start start 2024-04-04T06:17:03.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.637+0000 7fc3eac00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3e4153580 0x7fc3e40d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:03.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.637+0000 7fc3eac00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc3e40af4b0 0x7fc3e41600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:03.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.637+0000 7fc3eac00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3e40afab0 con 0x7fc3e40d07c0 2024-04-04T06:17:03.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.637+0000 7fc3eac00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc3e40d0c00 con 0x7fc3e40af4b0 2024-04-04T06:17:03.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.637+0000 7fc3eac00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e415cc80 con 0x7fc3e4153580 2024-04-04T06:17:03.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.638+0000 7fc3e3e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3e4153580 0x7fc3e40d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:03.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.638+0000 7fc3e8c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc3e40af4b0 0x7fc3e41600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:03.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.638+0000 7fc3e3e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3e4153580 0x7fc3e40d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.159:54978/0 (socket says 172.21.15.159:54978) 2024-04-04T06:17:03.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.638+0000 7fc3e3e00640 1 -- 172.21.15.159:0/2908883581 learned_addr learned my addr 172.21.15.159:0/2908883581 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:03.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.638+0000 7fc3e3e00640 1 -- 172.21.15.159:0/2908883581 >> v2:172.21.15.178:3300/0 conn(0x7fc3e40af4b0 msgr2=0x7fc3e41600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:03.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.638+0000 7fc3e3e00640 1 --2- 172.21.15.159:0/2908883581 >> v2:172.21.15.178:3300/0 conn(0x7fc3e40af4b0 0x7fc3e41600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:03.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.638+0000 7fc3e3e00640 1 -- 172.21.15.159:0/2908883581 >> v1:172.21.15.178:6789/0 conn(0x7fc3e40d07c0 legacy=0x7fc3e40aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:03.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.638+0000 7fc3e3e00640 1 -- 172.21.15.159:0/2908883581 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e415da00 con 0x7fc3e4153580 2024-04-04T06:17:03.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.640+0000 7fc3e3e00640 1 --2- 172.21.15.159:0/2908883581 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3e4153580 0x7fc3e40d0280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009e50 tx=0x7fc3d402bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95a117ec2fd154e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:03.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.640+0000 7fc3e3400640 1 -- 172.21.15.159:0/2908883581 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d402ca70 con 0x7fc3e4153580 2024-04-04T06:17:03.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.640+0000 7fc3e3400640 1 -- 172.21.15.159:0/2908883581 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d402cc10 con 0x7fc3e4153580 2024-04-04T06:17:03.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.640+0000 7fc3e3400640 1 -- 172.21.15.159:0/2908883581 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d40343f0 con 0x7fc3e4153580 2024-04-04T06:17:03.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.641+0000 7fc3eac00640 1 -- 172.21.15.159:0/2908883581 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3e4153580 msgr2=0x7fc3e40d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:03.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.641+0000 7fc3eac00640 1 --2- 172.21.15.159:0/2908883581 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3e4153580 0x7fc3e40d0280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009e50 tx=0x7fc3d402bed0 comp rx=0 tx=0).stop 2024-04-04T06:17:03.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.641+0000 7fc3eac00640 1 -- 172.21.15.159:0/2908883581 shutdown_connections 2024-04-04T06:17:03.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.641+0000 7fc3eac00640 1 --2- 172.21.15.159:0/2908883581 >> v2:172.21.15.178:3300/0 conn(0x7fc3e40af4b0 0x7fc3e41600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:03.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.641+0000 7fc3eac00640 1 --2- 172.21.15.159:0/2908883581 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3e4153580 0x7fc3e40d0280 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:03.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.641+0000 7fc3eac00640 1 -- 172.21.15.159:0/2908883581 >> 172.21.15.159:0/2908883581 conn(0x7fc3e406b750 msgr2=0x7fc3e406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:03.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.641+0000 7fc3eac00640 1 -- 172.21.15.159:0/2908883581 shutdown_connections 2024-04-04T06:17:03.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.641+0000 7fc3eac00640 1 -- 172.21.15.159:0/2908883581 wait complete. 2024-04-04T06:17:03.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.643+0000 7fc3eac00640 1 Processor -- start 2024-04-04T06:17:03.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.643+0000 7fc3eac00640 1 -- start start 2024-04-04T06:17:03.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.644+0000 7fc3eac00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3e40af4b0 0x7fc3e41515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:03.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.644+0000 7fc3eac00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3e40d07c0 0x7fc3e4151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:03.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.644+0000 7fc3eac00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3e4153580 0x7fc3e4155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:03.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.644+0000 7fc3eac00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e415d4e0 con 0x7fc3e4153580 2024-04-04T06:17:03.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.644+0000 7fc3eac00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e415d200 con 0x7fc3e40d07c0 2024-04-04T06:17:03.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.644+0000 7fc3eac00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e4153960 con 0x7fc3e40af4b0 2024-04-04T06:17:03.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.644+0000 7fc3e8c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3e4153580 0x7fc3e4155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:03.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.644+0000 7fc3e8c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3e4153580 0x7fc3e4155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44174/0 (socket says 172.21.15.159:44174) 2024-04-04T06:17:03.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.644+0000 7fc3e8c00640 1 -- 172.21.15.159:0/343833146 learned_addr learned my addr 172.21.15.159:0/343833146 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:03.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.644+0000 7fc3e3e00640 1 --2- 172.21.15.159:0/343833146 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3e40af4b0 0x7fc3e41515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:03.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.644+0000 7fc3db400640 1 --2- 172.21.15.159:0/343833146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3e40d07c0 0x7fc3e4151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:03.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.644+0000 7fc3e8c00640 1 -- 172.21.15.159:0/343833146 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3e40af4b0 msgr2=0x7fc3e41515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:03.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.644+0000 7fc3e8c00640 1 --2- 172.21.15.159:0/343833146 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3e40af4b0 0x7fc3e41515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:03.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.644+0000 7fc3e8c00640 1 -- 172.21.15.159:0/343833146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3e40d07c0 msgr2=0x7fc3e4151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:03.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.644+0000 7fc3e8c00640 1 --2- 172.21.15.159:0/343833146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3e40d07c0 0x7fc3e4151ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:03.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.644+0000 7fc3e8c00640 1 -- 172.21.15.159:0/343833146 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e41593a0 con 0x7fc3e4153580 2024-04-04T06:17:03.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.644+0000 7fc3db400640 1 --2- 172.21.15.159:0/343833146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3e40d07c0 0x7fc3e4151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:03.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.645+0000 7fc3e8c00640 1 --2- 172.21.15.159:0/343833146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3e4153580 0x7fc3e4155d10 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00e7f0 tx=0x7fc3dc00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:03.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.645+0000 7fc3e0c00640 1 -- 172.21.15.159:0/343833146 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc017070 con 0x7fc3e4153580 2024-04-04T06:17:03.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.645+0000 7fc3e0c00640 1 -- 172.21.15.159:0/343833146 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3dc00ee40 con 0x7fc3e4153580 2024-04-04T06:17:03.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.645+0000 7fc3e0c00640 1 -- 172.21.15.159:0/343833146 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc00cb80 con 0x7fc3e4153580 2024-04-04T06:17:03.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.645+0000 7fc3eac00640 1 -- 172.21.15.159:0/343833146 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e4159680 con 0x7fc3e4153580 2024-04-04T06:17:03.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.645+0000 7fc3eac00640 1 -- 172.21.15.159:0/343833146 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e4159c40 con 0x7fc3e4153580 2024-04-04T06:17:03.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.646+0000 7fc3eac00640 1 -- 172.21.15.159:0/343833146 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e40685c0 con 0x7fc3e4153580 2024-04-04T06:17:03.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.647+0000 7fc3e0c00640 1 -- 172.21.15.159:0/343833146 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3dc01a070 con 0x7fc3e4153580 2024-04-04T06:17:03.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.648+0000 7fc3e0c00640 1 --2- 172.21.15.159:0/343833146 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc3c006c920 0x7fc3c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:03.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.648+0000 7fc3e0c00640 1 -- 172.21.15.159:0/343833146 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7fc3dc08bd50 con 0x7fc3e4153580 2024-04-04T06:17:03.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.649+0000 7fc3e3e00640 1 --2- 172.21.15.159:0/343833146 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc3c006c920 0x7fc3c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:03.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.650+0000 7fc3e3e00640 1 --2- 172.21.15.159:0/343833146 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc3c006c920 0x7fc3c006ede0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4035000 tx=0x7fc3d4030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:03.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.654+0000 7fc3e0c00640 1 -- 172.21.15.159:0/343833146 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc3dc05a170 con 0x7fc3e4153580 2024-04-04T06:17:03.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.929+0000 7fc3eac00640 1 -- 172.21.15.159:0/343833146 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fc3e406eca0 con 0x7fc3e4153580 2024-04-04T06:17:03.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.930+0000 7fc3e0c00640 1 -- 172.21.15.159:0/343833146 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1177 (secure 0 0 0) 0x7fc3dc059a70 con 0x7fc3e4153580 2024-04-04T06:17:03.925 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:03.925 INFO:teuthology.orchestra.run.smithi159.stdout:{"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":696,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1712210741,"num_in_osds":8,"osd_in_since":1712210732,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":284917760,"bytes_avail":772809195520,"bytes_total":773094113280,"write_bytes_sec":2321,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":101,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8853}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":28,"modified":"2024-04-04T06:16:02.941422+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-04T06:17:03.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.933+0000 7fc3eac00640 1 -- 172.21.15.159:0/343833146 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc3c006c920 msgr2=0x7fc3c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:03.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.933+0000 7fc3eac00640 1 --2- 172.21.15.159:0/343833146 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc3c006c920 0x7fc3c006ede0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4035000 tx=0x7fc3d4030000 comp rx=0 tx=0).stop 2024-04-04T06:17:03.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.933+0000 7fc3eac00640 1 -- 172.21.15.159:0/343833146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3e4153580 msgr2=0x7fc3e4155d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:03.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.933+0000 7fc3eac00640 1 --2- 172.21.15.159:0/343833146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3e4153580 0x7fc3e4155d10 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00e7f0 tx=0x7fc3dc00ebf0 comp rx=0 tx=0).stop 2024-04-04T06:17:03.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.933+0000 7fc3eac00640 1 -- 172.21.15.159:0/343833146 shutdown_connections 2024-04-04T06:17:03.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.933+0000 7fc3eac00640 1 --2- 172.21.15.159:0/343833146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3e4153580 0x7fc3e4155d10 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:03.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.933+0000 7fc3eac00640 1 --2- 172.21.15.159:0/343833146 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc3c006c920 0x7fc3c006ede0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:03.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.933+0000 7fc3eac00640 1 --2- 172.21.15.159:0/343833146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3e40d07c0 0x7fc3e4151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:03.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.933+0000 7fc3eac00640 1 --2- 172.21.15.159:0/343833146 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc3e40af4b0 0x7fc3e41515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:03.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.933+0000 7fc3eac00640 1 -- 172.21.15.159:0/343833146 >> 172.21.15.159:0/343833146 conn(0x7fc3e406b750 msgr2=0x7fc3e415b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:03.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.933+0000 7fc3eac00640 1 -- 172.21.15.159:0/343833146 shutdown_connections 2024-04-04T06:17:03.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:03.933+0000 7fc3eac00640 1 -- 172.21.15.159:0/343833146 wait complete. 2024-04-04T06:17:03.947 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-04T06:17:03.947 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:17:04.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.158+0000 7f4ff9000640 1 Processor -- start 2024-04-04T06:17:04.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.158+0000 7f4ff9000640 1 -- start start 2024-04-04T06:17:04.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.159+0000 7f4ff9000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4ff415e750 0x7f4ff40cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:04.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.159+0000 7f4ff9000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4ff40cd270 0x7f4ff406b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:04.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.159+0000 7f4ff9000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ff40cd830 con 0x7f4ff406bab0 2024-04-04T06:17:04.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.159+0000 7f4ff9000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f4ff406bf20 con 0x7f4ff40cd270 2024-04-04T06:17:04.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.159+0000 7f4ff9000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4ff40cdb10 con 0x7f4ff415e750 2024-04-04T06:17:04.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.159+0000 7f4fe9400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4ff40cd270 0x7f4ff406b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:04.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.160+0000 7f4fe9400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4ff40cd270 0x7f4ff406b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59192/0 (socket says 172.21.15.159:59192) 2024-04-04T06:17:04.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.160+0000 7f4fe9400640 1 -- 172.21.15.159:0/1754307245 learned_addr learned my addr 172.21.15.159:0/1754307245 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:04.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.160+0000 7f4ff1e00640 1 --2- 172.21.15.159:0/1754307245 >> v2:172.21.15.178:3300/0 conn(0x7f4ff415e750 0x7f4ff40cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:04.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.160+0000 7f4fe9400640 1 -- 172.21.15.159:0/1754307245 >> v2:172.21.15.178:3300/0 conn(0x7f4ff415e750 msgr2=0x7f4ff40cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:04.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.160+0000 7f4fe9400640 1 --2- 172.21.15.159:0/1754307245 >> v2:172.21.15.178:3300/0 conn(0x7f4ff415e750 0x7f4ff40cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:04.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.160+0000 7f4fe9400640 1 -- 172.21.15.159:0/1754307245 >> v1:172.21.15.178:6789/0 conn(0x7f4ff406bab0 legacy=0x7f4ff41622d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:04.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.160+0000 7f4fe9400640 1 -- 172.21.15.159:0/1754307245 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff40cddf0 con 0x7f4ff40cd270 2024-04-04T06:17:04.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.160+0000 7f4ff1e00640 1 --2- 172.21.15.159:0/1754307245 >> v2:172.21.15.178:3300/0 conn(0x7f4ff415e750 0x7f4ff40cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:04.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.162+0000 7f4fe9400640 1 --2- 172.21.15.159:0/1754307245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ff40cd270 0x7f4ff406b570 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40d10a0 tx=0x7f4fdc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a0a11cc0e43258 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:04.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.162+0000 7f4ff1400640 1 -- 172.21.15.159:0/1754307245 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc035020 con 0x7f4ff40cd270 2024-04-04T06:17:04.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.162+0000 7f4ff1400640 1 -- 172.21.15.159:0/1754307245 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fdc031870 con 0x7f4ff40cd270 2024-04-04T06:17:04.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.162+0000 7f4ff1400640 1 -- 172.21.15.159:0/1754307245 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc031c40 con 0x7f4ff40cd270 2024-04-04T06:17:04.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.163+0000 7f4ff9000640 1 -- 172.21.15.159:0/1754307245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ff40cd270 msgr2=0x7f4ff406b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:04.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.163+0000 7f4ff9000640 1 --2- 172.21.15.159:0/1754307245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ff40cd270 0x7f4ff406b570 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40d10a0 tx=0x7f4fdc02b910 comp rx=0 tx=0).stop 2024-04-04T06:17:04.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.163+0000 7f4ff9000640 1 -- 172.21.15.159:0/1754307245 shutdown_connections 2024-04-04T06:17:04.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.163+0000 7f4ff9000640 1 --2- 172.21.15.159:0/1754307245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ff40cd270 0x7f4ff406b570 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:04.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.163+0000 7f4ff9000640 1 --2- 172.21.15.159:0/1754307245 >> v2:172.21.15.178:3300/0 conn(0x7f4ff415e750 0x7f4ff40cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:04.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.163+0000 7f4ff9000640 1 -- 172.21.15.159:0/1754307245 >> 172.21.15.159:0/1754307245 conn(0x7f4ff40b2cb0 msgr2=0x7f4ff40b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:04.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.163+0000 7f4ff9000640 1 -- 172.21.15.159:0/1754307245 shutdown_connections 2024-04-04T06:17:04.159 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.163+0000 7f4ff9000640 1 -- 172.21.15.159:0/1754307245 wait complete. 2024-04-04T06:17:04.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.165+0000 7f4ff9000640 1 Processor -- start 2024-04-04T06:17:04.160 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.165+0000 7f4ff9000640 1 -- start start 2024-04-04T06:17:04.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.166+0000 7f4ff9000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ff406bab0 0x7f4ff4156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:04.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.166+0000 7f4ff9000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ff40cd270 0x7f4ff40b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:04.162 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.166+0000 7f4ff9000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ff415e750 0x7f4ff40b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:04.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.166+0000 7f4ff9000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff415eb30 con 0x7f4ff40cd270 2024-04-04T06:17:04.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.166+0000 7f4ff9000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff40ce3b0 con 0x7f4ff415e750 2024-04-04T06:17:04.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.166+0000 7f4ff9000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4ff40ce8d0 con 0x7f4ff406bab0 2024-04-04T06:17:04.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.166+0000 7f4fe9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ff40cd270 0x7f4ff40b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:04.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.166+0000 7f4fe9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ff40cd270 0x7f4ff40b4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59194/0 (socket says 172.21.15.159:59194) 2024-04-04T06:17:04.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.166+0000 7f4fe9400640 1 -- 172.21.15.159:0/1916633032 learned_addr learned my addr 172.21.15.159:0/1916633032 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:04.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.166+0000 7f4ff1e00640 1 --2- 172.21.15.159:0/1916633032 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ff406bab0 0x7f4ff4156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:04.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.166+0000 7f4fe9400640 1 -- 172.21.15.159:0/1916633032 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ff406bab0 msgr2=0x7f4ff4156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:04.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.166+0000 7f4ff2800640 1 --2- 172.21.15.159:0/1916633032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ff415e750 0x7f4ff40b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:04.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.166+0000 7f4fe9400640 1 --2- 172.21.15.159:0/1916633032 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ff406bab0 0x7f4ff4156b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:04.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.166+0000 7f4fe9400640 1 -- 172.21.15.159:0/1916633032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ff415e750 msgr2=0x7f4ff40b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:04.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.166+0000 7f4fe9400640 1 --2- 172.21.15.159:0/1916633032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ff415e750 0x7f4ff40b45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:04.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.167+0000 7f4fe9400640 1 -- 172.21.15.159:0/1916633032 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff40b4df0 con 0x7f4ff40cd270 2024-04-04T06:17:04.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.167+0000 7f4ff2800640 1 --2- 172.21.15.159:0/1916633032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ff415e750 0x7f4ff40b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:04.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.167+0000 7f4fe9400640 1 --2- 172.21.15.159:0/1916633032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ff40cd270 0x7f4ff40b4080 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc02b950 tx=0x7f4fdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:04.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.167+0000 7f4feaa00640 1 -- 172.21.15.159:0/1916633032 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc040070 con 0x7f4ff40cd270 2024-04-04T06:17:04.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.167+0000 7f4feaa00640 1 -- 172.21.15.159:0/1916633032 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fdc03a420 con 0x7f4ff40cd270 2024-04-04T06:17:04.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.167+0000 7f4feaa00640 1 -- 172.21.15.159:0/1916633032 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc035040 con 0x7f4ff40cd270 2024-04-04T06:17:04.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.167+0000 7f4ff9000640 1 -- 172.21.15.159:0/1916633032 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff40b50d0 con 0x7f4ff40cd270 2024-04-04T06:17:04.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.167+0000 7f4ff9000640 1 -- 172.21.15.159:0/1916633032 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff40b5690 con 0x7f4ff40cd270 2024-04-04T06:17:04.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.168+0000 7f4ff9000640 1 -- 172.21.15.159:0/1916633032 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ff40685c0 con 0x7f4ff40cd270 2024-04-04T06:17:04.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.170+0000 7f4feaa00640 1 -- 172.21.15.159:0/1916633032 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4fdc038020 con 0x7f4ff40cd270 2024-04-04T06:17:04.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.171+0000 7f4feaa00640 1 --2- 172.21.15.159:0/1916633032 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4fd006c8f0 0x7f4fd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:04.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.171+0000 7f4feaa00640 1 -- 172.21.15.159:0/1916633032 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f4fdc0adb40 con 0x7f4ff40cd270 2024-04-04T06:17:04.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.171+0000 7f4ff1e00640 1 --2- 172.21.15.159:0/1916633032 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4fd006c8f0 0x7f4fd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:04.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.172+0000 7f4ff1e00640 1 --2- 172.21.15.159:0/1916633032 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4fd006c8f0 0x7f4fd006edb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4ff41523c0 tx=0x7f4fe4008450 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:04.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.179+0000 7f4feaa00640 1 -- 172.21.15.159:0/1916633032 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4fdc07c000 con 0x7f4ff40cd270 2024-04-04T06:17:04.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.376+0000 7f4ff9000640 1 -- 172.21.15.159:0/1916633032 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ff415ad40 con 0x7f4fd006c8f0 2024-04-04T06:17:04.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.394+0000 7f4feaa00640 1 -- 172.21.15.159:0/1916633032 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252483 (secure 0 0 0) 0x7f4ff415ad40 con 0x7f4fd006c8f0 2024-04-04T06:17:04.392 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:04.393 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:17:04.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.399+0000 7f4ff9000640 1 -- 172.21.15.159:0/1916633032 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4fd006c8f0 msgr2=0x7f4fd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:04.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.399+0000 7f4ff9000640 1 --2- 172.21.15.159:0/1916633032 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4fd006c8f0 0x7f4fd006edb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4ff41523c0 tx=0x7f4fe4008450 comp rx=0 tx=0).stop 2024-04-04T06:17:04.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.399+0000 7f4ff9000640 1 -- 172.21.15.159:0/1916633032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ff40cd270 msgr2=0x7f4ff40b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:04.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.399+0000 7f4ff9000640 1 --2- 172.21.15.159:0/1916633032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ff40cd270 0x7f4ff40b4080 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc02b950 tx=0x7f4fdc0046c0 comp rx=0 tx=0).stop 2024-04-04T06:17:04.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.399+0000 7f4ff9000640 1 -- 172.21.15.159:0/1916633032 shutdown_connections 2024-04-04T06:17:04.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.399+0000 7f4ff9000640 1 --2- 172.21.15.159:0/1916633032 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4fd006c8f0 0x7f4fd006edb0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:04.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.399+0000 7f4ff9000640 1 --2- 172.21.15.159:0/1916633032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ff415e750 0x7f4ff40b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:04.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.399+0000 7f4ff9000640 1 --2- 172.21.15.159:0/1916633032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ff40cd270 0x7f4ff40b4080 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:04.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.399+0000 7f4ff9000640 1 --2- 172.21.15.159:0/1916633032 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4ff406bab0 0x7f4ff4156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:04.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.399+0000 7f4ff9000640 1 -- 172.21.15.159:0/1916633032 >> 172.21.15.159:0/1916633032 conn(0x7f4ff40b2cb0 msgr2=0x7f4ff415b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:04.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.399+0000 7f4ff9000640 1 -- 172.21.15.159:0/1916633032 shutdown_connections 2024-04-04T06:17:04.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:04.399+0000 7f4ff9000640 1 -- 172.21.15.159:0/1916633032 wait complete. 2024-04-04T06:17:04.412 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-04-04T06:17:02.963103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278240,"kb_used_data":12136,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754696480,"statfs":{"total":773094113280,"available":772809195520,"internally_reserved":0,"allocated":12427264,"data_stored":6042100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.174190"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378643+0000","last_change":"2024-04-04T06:16:57.861743+0000","last_active":"2024-04-04T06:16:59.378643+0000","last_peered":"2024-04-04T06:16:59.378643+0000","last_clean":"2024-04-04T06:16:59.378643+0000","last_became_active":"2024-04-04T06:16:57.860873+0000","last_became_peered":"2024-04-04T06:16:57.860873+0000","last_unstale":"2024-04-04T06:16:59.378643+0000","last_undegraded":"2024-04-04T06:16:59.378643+0000","last_fullsized":"2024-04-04T06:16:59.378643+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:18:01.077209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856681+0000","last_change":"2024-04-04T06:16:57.850062+0000","last_active":"2024-04-04T06:16:58.856681+0000","last_peered":"2024-04-04T06:16:58.856681+0000","last_clean":"2024-04-04T06:16:58.856681+0000","last_became_active":"2024-04-04T06:16:57.849768+0000","last_became_peered":"2024-04-04T06:16:57.849768+0000","last_unstale":"2024-04-04T06:16:58.856681+0000","last_undegraded":"2024-04-04T06:16:58.856681+0000","last_fullsized":"2024-04-04T06:16:58.856681+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:20:36.274557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378633+0000","last_change":"2024-04-04T06:16:57.857033+0000","last_active":"2024-04-04T06:16:59.378633+0000","last_peered":"2024-04-04T06:16:59.378633+0000","last_clean":"2024-04-04T06:16:59.378633+0000","last_became_active":"2024-04-04T06:16:57.856617+0000","last_became_peered":"2024-04-04T06:16:57.856617+0000","last_unstale":"2024-04-04T06:16:59.378633+0000","last_undegraded":"2024-04-04T06:16:59.378633+0000","last_fullsized":"2024-04-04T06:16:59.378633+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:51:22.883892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.379030+0000","last_change":"2024-04-04T06:16:57.858984+0000","last_active":"2024-04-04T06:16:59.379030+0000","last_peered":"2024-04-04T06:16:59.379030+0000","last_clean":"2024-04-04T06:16:59.379030+0000","last_became_active":"2024-04-04T06:16:57.858427+0000","last_became_peered":"2024-04-04T06:16:57.858427+0000","last_unstale":"2024-04-04T06:16:59.379030+0000","last_undegraded":"2024-04-04T06:16:59.379030+0000","last_fullsized":"2024-04-04T06:16:59.379030+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:08:45.785102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222986+0000","last_change":"2024-04-04T06:16:57.852939+0000","last_active":"2024-04-04T06:16:59.222986+0000","last_peered":"2024-04-04T06:16:59.222986+0000","last_clean":"2024-04-04T06:16:59.222986+0000","last_became_active":"2024-04-04T06:16:57.852656+0000","last_became_peered":"2024-04-04T06:16:57.852656+0000","last_unstale":"2024-04-04T06:16:59.222986+0000","last_undegraded":"2024-04-04T06:16:59.222986+0000","last_fullsized":"2024-04-04T06:16:59.222986+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:04:27.627269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.376770+0000","last_change":"2024-04-04T06:16:57.858617+0000","last_active":"2024-04-04T06:16:59.376770+0000","last_peered":"2024-04-04T06:16:59.376770+0000","last_clean":"2024-04-04T06:16:59.376770+0000","last_became_active":"2024-04-04T06:16:57.858300+0000","last_became_peered":"2024-04-04T06:16:57.858300+0000","last_unstale":"2024-04-04T06:16:59.376770+0000","last_undegraded":"2024-04-04T06:16:59.376770+0000","last_fullsized":"2024-04-04T06:16:59.376770+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:36:49.667730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855381+0000","last_change":"2024-04-04T06:16:57.853261+0000","last_active":"2024-04-04T06:16:58.855381+0000","last_peered":"2024-04-04T06:16:58.855381+0000","last_clean":"2024-04-04T06:16:58.855381+0000","last_became_active":"2024-04-04T06:16:57.852925+0000","last_became_peered":"2024-04-04T06:16:57.852925+0000","last_unstale":"2024-04-04T06:16:58.855381+0000","last_undegraded":"2024-04-04T06:16:58.855381+0000","last_fullsized":"2024-04-04T06:16:58.855381+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:43:19.761558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377985+0000","last_change":"2024-04-04T06:16:57.854647+0000","last_active":"2024-04-04T06:16:59.377985+0000","last_peered":"2024-04-04T06:16:59.377985+0000","last_clean":"2024-04-04T06:16:59.377985+0000","last_became_active":"2024-04-04T06:16:57.854376+0000","last_became_peered":"2024-04-04T06:16:57.854376+0000","last_unstale":"2024-04-04T06:16:59.377985+0000","last_undegraded":"2024-04-04T06:16:59.377985+0000","last_fullsized":"2024-04-04T06:16:59.377985+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:18:08.210854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378876+0000","last_change":"2024-04-04T06:16:57.859251+0000","last_active":"2024-04-04T06:16:59.378876+0000","last_peered":"2024-04-04T06:16:59.378876+0000","last_clean":"2024-04-04T06:16:59.378876+0000","last_became_active":"2024-04-04T06:16:57.857756+0000","last_became_peered":"2024-04-04T06:16:57.857756+0000","last_unstale":"2024-04-04T06:16:59.378876+0000","last_undegraded":"2024-04-04T06:16:59.378876+0000","last_fullsized":"2024-04-04T06:16:59.378876+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:39:44.553729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378627+0000","last_change":"2024-04-04T06:16:57.865147+0000","last_active":"2024-04-04T06:16:59.378627+0000","last_peered":"2024-04-04T06:16:59.378627+0000","last_clean":"2024-04-04T06:16:59.378627+0000","last_became_active":"2024-04-04T06:16:57.863954+0000","last_became_peered":"2024-04-04T06:16:57.863954+0000","last_unstale":"2024-04-04T06:16:59.378627+0000","last_undegraded":"2024-04-04T06:16:59.378627+0000","last_fullsized":"2024-04-04T06:16:59.378627+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:45:25.455248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.857193+0000","last_change":"2024-04-04T06:16:57.852970+0000","last_active":"2024-04-04T06:16:58.857193+0000","last_peered":"2024-04-04T06:16:58.857193+0000","last_clean":"2024-04-04T06:16:58.857193+0000","last_became_active":"2024-04-04T06:16:57.852398+0000","last_became_peered":"2024-04-04T06:16:57.852398+0000","last_unstale":"2024-04-04T06:16:58.857193+0000","last_undegraded":"2024-04-04T06:16:58.857193+0000","last_fullsized":"2024-04-04T06:16:58.857193+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:25:13.863001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.379292+0000","last_change":"2024-04-04T06:16:57.868403+0000","last_active":"2024-04-04T06:16:59.379292+0000","last_peered":"2024-04-04T06:16:59.379292+0000","last_clean":"2024-04-04T06:16:59.379292+0000","last_became_active":"2024-04-04T06:16:57.867661+0000","last_became_peered":"2024-04-04T06:16:57.867661+0000","last_unstale":"2024-04-04T06:16:59.379292+0000","last_undegraded":"2024-04-04T06:16:59.379292+0000","last_fullsized":"2024-04-04T06:16:59.379292+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:49:12.167101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378304+0000","last_change":"2024-04-04T06:16:57.864603+0000","last_active":"2024-04-04T06:16:59.378304+0000","last_peered":"2024-04-04T06:16:59.378304+0000","last_clean":"2024-04-04T06:16:59.378304+0000","last_became_active":"2024-04-04T06:16:57.864334+0000","last_became_peered":"2024-04-04T06:16:57.864334+0000","last_unstale":"2024-04-04T06:16:59.378304+0000","last_undegraded":"2024-04-04T06:16:59.378304+0000","last_fullsized":"2024-04-04T06:16:59.378304+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:54:31.781352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377932+0000","last_change":"2024-04-04T06:16:57.860035+0000","last_active":"2024-04-04T06:16:59.377932+0000","last_peered":"2024-04-04T06:16:59.377932+0000","last_clean":"2024-04-04T06:16:59.377932+0000","last_became_active":"2024-04-04T06:16:57.858072+0000","last_became_peered":"2024-04-04T06:16:57.858072+0000","last_unstale":"2024-04-04T06:16:59.377932+0000","last_undegraded":"2024-04-04T06:16:59.377932+0000","last_fullsized":"2024-04-04T06:16:59.377932+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:52:19.546676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.376787+0000","last_change":"2024-04-04T06:16:57.853280+0000","last_active":"2024-04-04T06:16:59.376787+0000","last_peered":"2024-04-04T06:16:59.376787+0000","last_clean":"2024-04-04T06:16:59.376787+0000","last_became_active":"2024-04-04T06:16:57.852744+0000","last_became_peered":"2024-04-04T06:16:57.852744+0000","last_unstale":"2024-04-04T06:16:59.376787+0000","last_undegraded":"2024-04-04T06:16:59.376787+0000","last_fullsized":"2024-04-04T06:16:59.376787+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:04:27.627269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.376384+0000","last_change":"2024-04-04T06:16:57.866193+0000","last_active":"2024-04-04T06:16:59.376384+0000","last_peered":"2024-04-04T06:16:59.376384+0000","last_clean":"2024-04-04T06:16:59.376384+0000","last_became_active":"2024-04-04T06:16:57.865888+0000","last_became_peered":"2024-04-04T06:16:57.865888+0000","last_unstale":"2024-04-04T06:16:59.376384+0000","last_undegraded":"2024-04-04T06:16:59.376384+0000","last_fullsized":"2024-04-04T06:16:59.376384+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:46:20.508997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856787+0000","last_change":"2024-04-04T06:16:57.861016+0000","last_active":"2024-04-04T06:16:58.856787+0000","last_peered":"2024-04-04T06:16:58.856787+0000","last_clean":"2024-04-04T06:16:58.856787+0000","last_became_active":"2024-04-04T06:16:57.860213+0000","last_became_peered":"2024-04-04T06:16:57.860213+0000","last_unstale":"2024-04-04T06:16:58.856787+0000","last_undegraded":"2024-04-04T06:16:58.856787+0000","last_fullsized":"2024-04-04T06:16:58.856787+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:41:22.250390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.866106+0000","last_change":"2024-04-04T06:16:57.852666+0000","last_active":"2024-04-04T06:16:58.866106+0000","last_peered":"2024-04-04T06:16:58.866106+0000","last_clean":"2024-04-04T06:16:58.866106+0000","last_became_active":"2024-04-04T06:16:57.851465+0000","last_became_peered":"2024-04-04T06:16:57.851465+0000","last_unstale":"2024-04-04T06:16:58.866106+0000","last_undegraded":"2024-04-04T06:16:58.866106+0000","last_fullsized":"2024-04-04T06:16:58.866106+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:54:31.781352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378739+0000","last_change":"2024-04-04T06:16:57.873661+0000","last_active":"2024-04-04T06:16:59.378739+0000","last_peered":"2024-04-04T06:16:59.378739+0000","last_clean":"2024-04-04T06:16:59.378739+0000","last_became_active":"2024-04-04T06:16:57.873342+0000","last_became_peered":"2024-04-04T06:16:57.873342+0000","last_unstale":"2024-04-04T06:16:59.378739+0000","last_undegraded":"2024-04-04T06:16:59.378739+0000","last_fullsized":"2024-04-04T06:16:59.378739+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:05:27.280058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856739+0000","last_change":"2024-04-04T06:16:57.858770+0000","last_active":"2024-04-04T06:16:58.856739+0000","last_peered":"2024-04-04T06:16:58.856739+0000","last_clean":"2024-04-04T06:16:58.856739+0000","last_became_active":"2024-04-04T06:16:57.856816+0000","last_became_peered":"2024-04-04T06:16:57.856816+0000","last_unstale":"2024-04-04T06:16:58.856739+0000","last_undegraded":"2024-04-04T06:16:58.856739+0000","last_fullsized":"2024-04-04T06:16:58.856739+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:42:54.207457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.857204+0000","last_change":"2024-04-04T06:16:57.840896+0000","last_active":"2024-04-04T06:16:58.857204+0000","last_peered":"2024-04-04T06:16:58.857204+0000","last_clean":"2024-04-04T06:16:58.857204+0000","last_became_active":"2024-04-04T06:16:57.840145+0000","last_became_peered":"2024-04-04T06:16:57.840145+0000","last_unstale":"2024-04-04T06:16:58.857204+0000","last_undegraded":"2024-04-04T06:16:58.857204+0000","last_fullsized":"2024-04-04T06:16:58.857204+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:56:07.453527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850326+0000","last_change":"2024-04-04T06:16:57.862396+0000","last_active":"2024-04-04T06:16:58.850326+0000","last_peered":"2024-04-04T06:16:58.850326+0000","last_clean":"2024-04-04T06:16:58.850326+0000","last_became_active":"2024-04-04T06:16:57.862112+0000","last_became_peered":"2024-04-04T06:16:57.862112+0000","last_unstale":"2024-04-04T06:16:58.850326+0000","last_undegraded":"2024-04-04T06:16:58.850326+0000","last_fullsized":"2024-04-04T06:16:58.850326+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:20:36.274557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855555+0000","last_change":"2024-04-04T06:16:57.855453+0000","last_active":"2024-04-04T06:16:58.855555+0000","last_peered":"2024-04-04T06:16:58.855555+0000","last_clean":"2024-04-04T06:16:58.855555+0000","last_became_active":"2024-04-04T06:16:57.855172+0000","last_became_peered":"2024-04-04T06:16:57.855172+0000","last_unstale":"2024-04-04T06:16:58.855555+0000","last_undegraded":"2024-04-04T06:16:58.855555+0000","last_fullsized":"2024-04-04T06:16:58.855555+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:17:03.994940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850188+0000","last_change":"2024-04-04T06:16:57.854852+0000","last_active":"2024-04-04T06:16:58.850188+0000","last_peered":"2024-04-04T06:16:58.850188+0000","last_clean":"2024-04-04T06:16:58.850188+0000","last_became_active":"2024-04-04T06:16:57.854610+0000","last_became_peered":"2024-04-04T06:16:57.854610+0000","last_unstale":"2024-04-04T06:16:58.850188+0000","last_undegraded":"2024-04-04T06:16:58.850188+0000","last_fullsized":"2024-04-04T06:16:58.850188+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:17:03.994940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.857033+0000","last_change":"2024-04-04T06:16:57.859703+0000","last_active":"2024-04-04T06:16:58.857033+0000","last_peered":"2024-04-04T06:16:58.857033+0000","last_clean":"2024-04-04T06:16:58.857033+0000","last_became_active":"2024-04-04T06:16:57.859398+0000","last_became_peered":"2024-04-04T06:16:57.859398+0000","last_unstale":"2024-04-04T06:16:58.857033+0000","last_undegraded":"2024-04-04T06:16:58.857033+0000","last_fullsized":"2024-04-04T06:16:58.857033+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:29:25.780543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378891+0000","last_change":"2024-04-04T06:16:57.861831+0000","last_active":"2024-04-04T06:16:59.378891+0000","last_peered":"2024-04-04T06:16:59.378891+0000","last_clean":"2024-04-04T06:16:59.378891+0000","last_became_active":"2024-04-04T06:16:57.861541+0000","last_became_peered":"2024-04-04T06:16:57.861541+0000","last_unstale":"2024-04-04T06:16:59.378891+0000","last_undegraded":"2024-04-04T06:16:59.378891+0000","last_fullsized":"2024-04-04T06:16:59.378891+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:45:32.671346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378148+0000","last_change":"2024-04-04T06:16:57.871771+0000","last_active":"2024-04-04T06:16:59.378148+0000","last_peered":"2024-04-04T06:16:59.378148+0000","last_clean":"2024-04-04T06:16:59.378148+0000","last_became_active":"2024-04-04T06:16:57.871480+0000","last_became_peered":"2024-04-04T06:16:57.871480+0000","last_unstale":"2024-04-04T06:16:59.378148+0000","last_undegraded":"2024-04-04T06:16:59.378148+0000","last_fullsized":"2024-04-04T06:16:59.378148+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:46:04.591130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850668+0000","last_change":"2024-04-04T06:16:57.867129+0000","last_active":"2024-04-04T06:16:58.850668+0000","last_peered":"2024-04-04T06:16:58.850668+0000","last_clean":"2024-04-04T06:16:58.850668+0000","last_became_active":"2024-04-04T06:16:57.866770+0000","last_became_peered":"2024-04-04T06:16:57.866770+0000","last_unstale":"2024-04-04T06:16:58.850668+0000","last_undegraded":"2024-04-04T06:16:58.850668+0000","last_fullsized":"2024-04-04T06:16:58.850668+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:29:44.470791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856807+0000","last_change":"2024-04-04T06:16:57.859118+0000","last_active":"2024-04-04T06:16:58.856807+0000","last_peered":"2024-04-04T06:16:58.856807+0000","last_clean":"2024-04-04T06:16:58.856807+0000","last_became_active":"2024-04-04T06:16:57.858242+0000","last_became_peered":"2024-04-04T06:16:57.858242+0000","last_unstale":"2024-04-04T06:16:58.856807+0000","last_undegraded":"2024-04-04T06:16:58.856807+0000","last_fullsized":"2024-04-04T06:16:58.856807+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:16:58.119204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855126+0000","last_change":"2024-04-04T06:16:57.838809+0000","last_active":"2024-04-04T06:16:58.855126+0000","last_peered":"2024-04-04T06:16:58.855126+0000","last_clean":"2024-04-04T06:16:58.855126+0000","last_became_active":"2024-04-04T06:16:57.837497+0000","last_became_peered":"2024-04-04T06:16:57.837497+0000","last_unstale":"2024-04-04T06:16:58.855126+0000","last_undegraded":"2024-04-04T06:16:58.855126+0000","last_fullsized":"2024-04-04T06:16:58.855126+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:05:00.308975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377216+0000","last_change":"2024-04-04T06:16:57.863907+0000","last_active":"2024-04-04T06:16:59.377216+0000","last_peered":"2024-04-04T06:16:59.377216+0000","last_clean":"2024-04-04T06:16:59.377216+0000","last_became_active":"2024-04-04T06:16:57.863538+0000","last_became_peered":"2024-04-04T06:16:57.863538+0000","last_unstale":"2024-04-04T06:16:59.377216+0000","last_undegraded":"2024-04-04T06:16:59.377216+0000","last_fullsized":"2024-04-04T06:16:59.377216+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:53:37.135784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.376938+0000","last_change":"2024-04-04T06:16:57.853449+0000","last_active":"2024-04-04T06:16:59.376938+0000","last_peered":"2024-04-04T06:16:59.376938+0000","last_clean":"2024-04-04T06:16:59.376938+0000","last_became_active":"2024-04-04T06:16:57.853171+0000","last_became_peered":"2024-04-04T06:16:57.853171+0000","last_unstale":"2024-04-04T06:16:59.376938+0000","last_undegraded":"2024-04-04T06:16:59.376938+0000","last_fullsized":"2024-04-04T06:16:59.376938+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:27:39.563053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378077+0000","last_change":"2024-04-04T06:16:57.861937+0000","last_active":"2024-04-04T06:16:59.378077+0000","last_peered":"2024-04-04T06:16:59.378077+0000","last_clean":"2024-04-04T06:16:59.378077+0000","last_became_active":"2024-04-04T06:16:57.861668+0000","last_became_peered":"2024-04-04T06:16:57.861668+0000","last_unstale":"2024-04-04T06:16:59.378077+0000","last_undegraded":"2024-04-04T06:16:59.378077+0000","last_fullsized":"2024-04-04T06:16:59.378077+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:38:00.487062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"112'4","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855930+0000","last_change":"2024-04-04T06:16:56.826026+0000","last_active":"2024-04-04T06:16:58.855930+0000","last_peered":"2024-04-04T06:16:58.855930+0000","last_clean":"2024-04-04T06:16:58.855930+0000","last_became_active":"2024-04-04T06:16:56.825749+0000","last_became_peered":"2024-04-04T06:16:56.825749+0000","last_unstale":"2024-04-04T06:16:58.855930+0000","last_undegraded":"2024-04-04T06:16:58.855930+0000","last_fullsized":"2024-04-04T06:16:58.855930+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:44:30.609701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.866566+0000","last_change":"2024-04-04T06:16:57.859524+0000","last_active":"2024-04-04T06:16:58.866566+0000","last_peered":"2024-04-04T06:16:58.866566+0000","last_clean":"2024-04-04T06:16:58.866566+0000","last_became_active":"2024-04-04T06:16:57.859203+0000","last_became_peered":"2024-04-04T06:16:57.859203+0000","last_unstale":"2024-04-04T06:16:58.866566+0000","last_undegraded":"2024-04-04T06:16:58.866566+0000","last_fullsized":"2024-04-04T06:16:58.866566+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:09:22.693053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378327+0000","last_change":"2024-04-04T06:16:56.828520+0000","last_active":"2024-04-04T06:16:59.378327+0000","last_peered":"2024-04-04T06:16:59.378327+0000","last_clean":"2024-04-04T06:16:59.378327+0000","last_became_active":"2024-04-04T06:16:56.828125+0000","last_became_peered":"2024-04-04T06:16:56.828125+0000","last_unstale":"2024-04-04T06:16:59.378327+0000","last_undegraded":"2024-04-04T06:16:59.378327+0000","last_fullsized":"2024-04-04T06:16:59.378327+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:04:22.860294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377506+0000","last_change":"2024-04-04T06:16:57.859001+0000","last_active":"2024-04-04T06:16:59.377506+0000","last_peered":"2024-04-04T06:16:59.377506+0000","last_clean":"2024-04-04T06:16:59.377506+0000","last_became_active":"2024-04-04T06:16:57.858712+0000","last_became_peered":"2024-04-04T06:16:57.858712+0000","last_unstale":"2024-04-04T06:16:59.377506+0000","last_undegraded":"2024-04-04T06:16:59.377506+0000","last_fullsized":"2024-04-04T06:16:59.377506+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:09:33.583383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222894+0000","last_change":"2024-04-04T06:16:56.817318+0000","last_active":"2024-04-04T06:16:59.222894+0000","last_peered":"2024-04-04T06:16:59.222894+0000","last_clean":"2024-04-04T06:16:59.222894+0000","last_became_active":"2024-04-04T06:16:56.816995+0000","last_became_peered":"2024-04-04T06:16:56.816995+0000","last_unstale":"2024-04-04T06:16:59.222894+0000","last_undegraded":"2024-04-04T06:16:59.222894+0000","last_fullsized":"2024-04-04T06:16:59.222894+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:02:07.681535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378954+0000","last_change":"2024-04-04T06:16:57.869420+0000","last_active":"2024-04-04T06:16:59.378954+0000","last_peered":"2024-04-04T06:16:59.378954+0000","last_clean":"2024-04-04T06:16:59.378954+0000","last_became_active":"2024-04-04T06:16:57.869124+0000","last_became_peered":"2024-04-04T06:16:57.869124+0000","last_unstale":"2024-04-04T06:16:59.378954+0000","last_undegraded":"2024-04-04T06:16:59.378954+0000","last_fullsized":"2024-04-04T06:16:59.378954+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:10:39.218040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222549+0000","last_change":"2024-04-04T06:16:56.813821+0000","last_active":"2024-04-04T06:16:59.222549+0000","last_peered":"2024-04-04T06:16:59.222549+0000","last_clean":"2024-04-04T06:16:59.222549+0000","last_became_active":"2024-04-04T06:16:56.813555+0000","last_became_peered":"2024-04-04T06:16:56.813555+0000","last_unstale":"2024-04-04T06:16:59.222549+0000","last_undegraded":"2024-04-04T06:16:59.222549+0000","last_fullsized":"2024-04-04T06:16:59.222549+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:22:21.703895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377925+0000","last_change":"2024-04-04T06:16:57.858500+0000","last_active":"2024-04-04T06:16:59.377925+0000","last_peered":"2024-04-04T06:16:59.377925+0000","last_clean":"2024-04-04T06:16:59.377925+0000","last_became_active":"2024-04-04T06:16:57.856744+0000","last_became_peered":"2024-04-04T06:16:57.856744+0000","last_unstale":"2024-04-04T06:16:59.377925+0000","last_undegraded":"2024-04-04T06:16:59.377925+0000","last_fullsized":"2024-04-04T06:16:59.377925+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:44:29.963390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854845+0000","last_change":"2024-04-04T06:16:56.813703+0000","last_active":"2024-04-04T06:16:58.854845+0000","last_peered":"2024-04-04T06:16:58.854845+0000","last_clean":"2024-04-04T06:16:58.854845+0000","last_became_active":"2024-04-04T06:16:56.813224+0000","last_became_peered":"2024-04-04T06:16:56.813224+0000","last_unstale":"2024-04-04T06:16:58.854845+0000","last_undegraded":"2024-04-04T06:16:58.854845+0000","last_fullsized":"2024-04-04T06:16:58.854845+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:28:44.071981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856364+0000","last_change":"2024-04-04T06:16:57.851938+0000","last_active":"2024-04-04T06:16:58.856364+0000","last_peered":"2024-04-04T06:16:58.856364+0000","last_clean":"2024-04-04T06:16:58.856364+0000","last_became_active":"2024-04-04T06:16:57.850346+0000","last_became_peered":"2024-04-04T06:16:57.850346+0000","last_unstale":"2024-04-04T06:16:58.856364+0000","last_undegraded":"2024-04-04T06:16:58.856364+0000","last_fullsized":"2024-04-04T06:16:58.856364+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:42:36.526974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850561+0000","last_change":"2024-04-04T06:16:56.822599+0000","last_active":"2024-04-04T06:16:58.850561+0000","last_peered":"2024-04-04T06:16:58.850561+0000","last_clean":"2024-04-04T06:16:58.850561+0000","last_became_active":"2024-04-04T06:16:56.822130+0000","last_became_peered":"2024-04-04T06:16:56.822130+0000","last_unstale":"2024-04-04T06:16:58.850561+0000","last_undegraded":"2024-04-04T06:16:58.850561+0000","last_fullsized":"2024-04-04T06:16:58.850561+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:59:42.200129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.376345+0000","last_change":"2024-04-04T06:16:57.849755+0000","last_active":"2024-04-04T06:16:59.376345+0000","last_peered":"2024-04-04T06:16:59.376345+0000","last_clean":"2024-04-04T06:16:59.376345+0000","last_became_active":"2024-04-04T06:16:57.849341+0000","last_became_peered":"2024-04-04T06:16:57.849341+0000","last_unstale":"2024-04-04T06:16:59.376345+0000","last_undegraded":"2024-04-04T06:16:59.376345+0000","last_fullsized":"2024-04-04T06:16:59.376345+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:52:09.021010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855249+0000","last_change":"2024-04-04T06:16:56.822615+0000","last_active":"2024-04-04T06:16:58.855249+0000","last_peered":"2024-04-04T06:16:58.855249+0000","last_clean":"2024-04-04T06:16:58.855249+0000","last_became_active":"2024-04-04T06:16:56.819761+0000","last_became_peered":"2024-04-04T06:16:56.819761+0000","last_unstale":"2024-04-04T06:16:58.855249+0000","last_undegraded":"2024-04-04T06:16:58.855249+0000","last_fullsized":"2024-04-04T06:16:58.855249+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:10:06.219773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854643+0000","last_change":"2024-04-04T06:16:57.853254+0000","last_active":"2024-04-04T06:16:58.854643+0000","last_peered":"2024-04-04T06:16:58.854643+0000","last_clean":"2024-04-04T06:16:58.854643+0000","last_became_active":"2024-04-04T06:16:57.852500+0000","last_became_peered":"2024-04-04T06:16:57.852500+0000","last_unstale":"2024-04-04T06:16:58.854643+0000","last_undegraded":"2024-04-04T06:16:58.854643+0000","last_fullsized":"2024-04-04T06:16:58.854643+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:47.042156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.376485+0000","last_change":"2024-04-04T06:16:56.816173+0000","last_active":"2024-04-04T06:16:59.376485+0000","last_peered":"2024-04-04T06:16:59.376485+0000","last_clean":"2024-04-04T06:16:59.376485+0000","last_became_active":"2024-04-04T06:16:56.815796+0000","last_became_peered":"2024-04-04T06:16:56.815796+0000","last_unstale":"2024-04-04T06:16:59.376485+0000","last_undegraded":"2024-04-04T06:16:59.376485+0000","last_fullsized":"2024-04-04T06:16:59.376485+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:12:54.232107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856214+0000","last_change":"2024-04-04T06:16:57.863523+0000","last_active":"2024-04-04T06:16:58.856214+0000","last_peered":"2024-04-04T06:16:58.856214+0000","last_clean":"2024-04-04T06:16:58.856214+0000","last_became_active":"2024-04-04T06:16:57.861142+0000","last_became_peered":"2024-04-04T06:16:57.861142+0000","last_unstale":"2024-04-04T06:16:58.856214+0000","last_undegraded":"2024-04-04T06:16:58.856214+0000","last_fullsized":"2024-04-04T06:16:58.856214+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:50:12.226068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222684+0000","last_change":"2024-04-04T06:16:56.817070+0000","last_active":"2024-04-04T06:16:59.222684+0000","last_peered":"2024-04-04T06:16:59.222684+0000","last_clean":"2024-04-04T06:16:59.222684+0000","last_became_active":"2024-04-04T06:16:56.816742+0000","last_became_peered":"2024-04-04T06:16:56.816742+0000","last_unstale":"2024-04-04T06:16:59.222684+0000","last_undegraded":"2024-04-04T06:16:59.222684+0000","last_fullsized":"2024-04-04T06:16:59.222684+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:10:49.284692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.866544+0000","last_change":"2024-04-04T06:16:57.865580+0000","last_active":"2024-04-04T06:16:58.866544+0000","last_peered":"2024-04-04T06:16:58.866544+0000","last_clean":"2024-04-04T06:16:58.866544+0000","last_became_active":"2024-04-04T06:16:57.863731+0000","last_became_peered":"2024-04-04T06:16:57.863731+0000","last_unstale":"2024-04-04T06:16:58.866544+0000","last_undegraded":"2024-04-04T06:16:58.866544+0000","last_fullsized":"2024-04-04T06:16:58.866544+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:05:27.280058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850433+0000","last_change":"2024-04-04T06:16:56.822568+0000","last_active":"2024-04-04T06:16:58.850433+0000","last_peered":"2024-04-04T06:16:58.850433+0000","last_clean":"2024-04-04T06:16:58.850433+0000","last_became_active":"2024-04-04T06:16:56.822237+0000","last_became_peered":"2024-04-04T06:16:56.822237+0000","last_unstale":"2024-04-04T06:16:58.850433+0000","last_undegraded":"2024-04-04T06:16:58.850433+0000","last_fullsized":"2024-04-04T06:16:58.850433+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:13:56.182899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850676+0000","last_change":"2024-04-04T06:16:57.851850+0000","last_active":"2024-04-04T06:16:58.850676+0000","last_peered":"2024-04-04T06:16:58.850676+0000","last_clean":"2024-04-04T06:16:58.850676+0000","last_became_active":"2024-04-04T06:16:57.851529+0000","last_became_peered":"2024-04-04T06:16:57.851529+0000","last_unstale":"2024-04-04T06:16:58.850676+0000","last_undegraded":"2024-04-04T06:16:58.850676+0000","last_fullsized":"2024-04-04T06:16:58.850676+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:16:58.119204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"112'3","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378418+0000","last_change":"2024-04-04T06:16:56.811481+0000","last_active":"2024-04-04T06:16:59.378418+0000","last_peered":"2024-04-04T06:16:59.378418+0000","last_clean":"2024-04-04T06:16:59.378418+0000","last_became_active":"2024-04-04T06:16:56.811190+0000","last_became_peered":"2024-04-04T06:16:56.811190+0000","last_unstale":"2024-04-04T06:16:59.378418+0000","last_undegraded":"2024-04-04T06:16:59.378418+0000","last_fullsized":"2024-04-04T06:16:59.378418+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:41:11.586175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855312+0000","last_change":"2024-04-04T06:16:57.855846+0000","last_active":"2024-04-04T06:16:58.855312+0000","last_peered":"2024-04-04T06:16:58.855312+0000","last_clean":"2024-04-04T06:16:58.855312+0000","last_became_active":"2024-04-04T06:16:57.855487+0000","last_became_peered":"2024-04-04T06:16:57.855487+0000","last_unstale":"2024-04-04T06:16:58.855312+0000","last_undegraded":"2024-04-04T06:16:58.855312+0000","last_fullsized":"2024-04-04T06:16:58.855312+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:25:12.522619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"112'6","reported_seq":20,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.379530+0000","last_change":"2024-04-04T06:16:56.816249+0000","last_active":"2024-04-04T06:16:59.379530+0000","last_peered":"2024-04-04T06:16:59.379530+0000","last_clean":"2024-04-04T06:16:59.379530+0000","last_became_active":"2024-04-04T06:16:56.815980+0000","last_became_peered":"2024-04-04T06:16:56.815980+0000","last_unstale":"2024-04-04T06:16:59.379530+0000","last_undegraded":"2024-04-04T06:16:59.379530+0000","last_fullsized":"2024-04-04T06:16:59.379530+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:20:35.260197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850475+0000","last_change":"2024-04-04T06:16:57.852377+0000","last_active":"2024-04-04T06:16:58.850475+0000","last_peered":"2024-04-04T06:16:58.850475+0000","last_clean":"2024-04-04T06:16:58.850475+0000","last_became_active":"2024-04-04T06:16:57.850593+0000","last_became_peered":"2024-04-04T06:16:57.850593+0000","last_unstale":"2024-04-04T06:16:58.850475+0000","last_undegraded":"2024-04-04T06:16:58.850475+0000","last_fullsized":"2024-04-04T06:16:58.850475+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:41:22.250390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377300+0000","last_change":"2024-04-04T06:16:56.825811+0000","last_active":"2024-04-04T06:16:59.377300+0000","last_peered":"2024-04-04T06:16:59.377300+0000","last_clean":"2024-04-04T06:16:59.377300+0000","last_became_active":"2024-04-04T06:16:56.825491+0000","last_became_peered":"2024-04-04T06:16:56.825491+0000","last_unstale":"2024-04-04T06:16:59.377300+0000","last_undegraded":"2024-04-04T06:16:59.377300+0000","last_fullsized":"2024-04-04T06:16:59.377300+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:22:21.703895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850892+0000","last_change":"2024-04-04T06:16:57.854997+0000","last_active":"2024-04-04T06:16:58.850892+0000","last_peered":"2024-04-04T06:16:58.850892+0000","last_clean":"2024-04-04T06:16:58.850892+0000","last_became_active":"2024-04-04T06:16:57.854710+0000","last_became_peered":"2024-04-04T06:16:57.854710+0000","last_unstale":"2024-04-04T06:16:58.850892+0000","last_undegraded":"2024-04-04T06:16:58.850892+0000","last_fullsized":"2024-04-04T06:16:58.850892+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:29:25.780543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.865913+0000","last_change":"2024-04-04T06:16:56.808070+0000","last_active":"2024-04-04T06:16:58.865913+0000","last_peered":"2024-04-04T06:16:58.865913+0000","last_clean":"2024-04-04T06:16:58.865913+0000","last_became_active":"2024-04-04T06:16:56.807074+0000","last_became_peered":"2024-04-04T06:16:56.807074+0000","last_unstale":"2024-04-04T06:16:58.865913+0000","last_undegraded":"2024-04-04T06:16:58.865913+0000","last_fullsized":"2024-04-04T06:16:58.865913+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:05:44.678207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850292+0000","last_change":"2024-04-04T06:16:57.840280+0000","last_active":"2024-04-04T06:16:58.850292+0000","last_peered":"2024-04-04T06:16:58.850292+0000","last_clean":"2024-04-04T06:16:58.850292+0000","last_became_active":"2024-04-04T06:16:57.839601+0000","last_became_peered":"2024-04-04T06:16:57.839601+0000","last_unstale":"2024-04-04T06:16:58.850292+0000","last_undegraded":"2024-04-04T06:16:58.850292+0000","last_fullsized":"2024-04-04T06:16:58.850292+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:41:12.600534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"112'8","reported_seq":22,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222426+0000","last_change":"2024-04-04T06:16:56.814462+0000","last_active":"2024-04-04T06:16:59.222426+0000","last_peered":"2024-04-04T06:16:59.222426+0000","last_clean":"2024-04-04T06:16:59.222426+0000","last_became_active":"2024-04-04T06:16:56.813042+0000","last_became_peered":"2024-04-04T06:16:56.813042+0000","last_unstale":"2024-04-04T06:16:59.222426+0000","last_undegraded":"2024-04-04T06:16:59.222426+0000","last_fullsized":"2024-04-04T06:16:59.222426+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:13:39.485055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222788+0000","last_change":"2024-04-04T06:16:57.852435+0000","last_active":"2024-04-04T06:16:59.222788+0000","last_peered":"2024-04-04T06:16:59.222788+0000","last_clean":"2024-04-04T06:16:59.222788+0000","last_became_active":"2024-04-04T06:16:57.851997+0000","last_became_peered":"2024-04-04T06:16:57.851997+0000","last_unstale":"2024-04-04T06:16:59.222788+0000","last_undegraded":"2024-04-04T06:16:59.222788+0000","last_fullsized":"2024-04-04T06:16:59.222788+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:36:49.667730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.379260+0000","last_change":"2024-04-04T06:16:56.812543+0000","last_active":"2024-04-04T06:16:59.379260+0000","last_peered":"2024-04-04T06:16:59.379260+0000","last_clean":"2024-04-04T06:16:59.379260+0000","last_became_active":"2024-04-04T06:16:56.812240+0000","last_became_peered":"2024-04-04T06:16:56.812240+0000","last_unstale":"2024-04-04T06:16:59.379260+0000","last_undegraded":"2024-04-04T06:16:59.379260+0000","last_fullsized":"2024-04-04T06:16:59.379260+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:25:12.848641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377607+0000","last_change":"2024-04-04T06:16:57.863716+0000","last_active":"2024-04-04T06:16:59.377607+0000","last_peered":"2024-04-04T06:16:59.377607+0000","last_clean":"2024-04-04T06:16:59.377607+0000","last_became_active":"2024-04-04T06:16:57.862780+0000","last_became_peered":"2024-04-04T06:16:57.862780+0000","last_unstale":"2024-04-04T06:16:59.377607+0000","last_undegraded":"2024-04-04T06:16:59.377607+0000","last_fullsized":"2024-04-04T06:16:59.377607+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:56:59.538463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222812+0000","last_change":"2024-04-04T06:16:56.822207+0000","last_active":"2024-04-04T06:16:59.222812+0000","last_peered":"2024-04-04T06:16:59.222812+0000","last_clean":"2024-04-04T06:16:59.222812+0000","last_became_active":"2024-04-04T06:16:56.821906+0000","last_became_peered":"2024-04-04T06:16:56.821906+0000","last_unstale":"2024-04-04T06:16:59.222812+0000","last_undegraded":"2024-04-04T06:16:59.222812+0000","last_fullsized":"2024-04-04T06:16:59.222812+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:52:52.166280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378788+0000","last_change":"2024-04-04T06:16:57.865766+0000","last_active":"2024-04-04T06:16:59.378788+0000","last_peered":"2024-04-04T06:16:59.378788+0000","last_clean":"2024-04-04T06:16:59.378788+0000","last_became_active":"2024-04-04T06:16:57.865457+0000","last_became_peered":"2024-04-04T06:16:57.865457+0000","last_unstale":"2024-04-04T06:16:59.378788+0000","last_undegraded":"2024-04-04T06:16:59.378788+0000","last_fullsized":"2024-04-04T06:16:59.378788+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:09:22.693053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.866035+0000","last_change":"2024-04-04T06:16:56.818843+0000","last_active":"2024-04-04T06:16:58.866035+0000","last_peered":"2024-04-04T06:16:58.866035+0000","last_clean":"2024-04-04T06:16:58.866035+0000","last_became_active":"2024-04-04T06:16:56.818532+0000","last_became_peered":"2024-04-04T06:16:56.818532+0000","last_unstale":"2024-04-04T06:16:58.866035+0000","last_undegraded":"2024-04-04T06:16:58.866035+0000","last_fullsized":"2024-04-04T06:16:58.866035+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:31:12.755633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.379063+0000","last_change":"2024-04-04T06:16:57.859384+0000","last_active":"2024-04-04T06:16:59.379063+0000","last_peered":"2024-04-04T06:16:59.379063+0000","last_clean":"2024-04-04T06:16:59.379063+0000","last_became_active":"2024-04-04T06:16:57.859026+0000","last_became_peered":"2024-04-04T06:16:57.859026+0000","last_unstale":"2024-04-04T06:16:59.379063+0000","last_undegraded":"2024-04-04T06:16:59.379063+0000","last_fullsized":"2024-04-04T06:16:59.379063+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:03:57.236044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854548+0000","last_change":"2024-04-04T06:16:56.817026+0000","last_active":"2024-04-04T06:16:58.854548+0000","last_peered":"2024-04-04T06:16:58.854548+0000","last_clean":"2024-04-04T06:16:58.854548+0000","last_became_active":"2024-04-04T06:16:56.816488+0000","last_became_peered":"2024-04-04T06:16:56.816488+0000","last_unstale":"2024-04-04T06:16:58.854548+0000","last_undegraded":"2024-04-04T06:16:58.854548+0000","last_fullsized":"2024-04-04T06:16:58.854548+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:11:49.042689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377911+0000","last_change":"2024-04-04T06:16:57.859829+0000","last_active":"2024-04-04T06:16:59.377911+0000","last_peered":"2024-04-04T06:16:59.377911+0000","last_clean":"2024-04-04T06:16:59.377911+0000","last_became_active":"2024-04-04T06:16:57.859535+0000","last_became_peered":"2024-04-04T06:16:57.859535+0000","last_unstale":"2024-04-04T06:16:59.377911+0000","last_undegraded":"2024-04-04T06:16:59.377911+0000","last_fullsized":"2024-04-04T06:16:59.377911+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:27:35.132513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377508+0000","last_change":"2024-04-04T06:16:56.821203+0000","last_active":"2024-04-04T06:16:59.377508+0000","last_peered":"2024-04-04T06:16:59.377508+0000","last_clean":"2024-04-04T06:16:59.377508+0000","last_became_active":"2024-04-04T06:16:56.820739+0000","last_became_peered":"2024-04-04T06:16:56.820739+0000","last_unstale":"2024-04-04T06:16:59.377508+0000","last_undegraded":"2024-04-04T06:16:59.377508+0000","last_fullsized":"2024-04-04T06:16:59.377508+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:13:39.485055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378104+0000","last_change":"2024-04-04T06:16:57.862428+0000","last_active":"2024-04-04T06:16:59.378104+0000","last_peered":"2024-04-04T06:16:59.378104+0000","last_clean":"2024-04-04T06:16:59.378104+0000","last_became_active":"2024-04-04T06:16:57.862160+0000","last_became_peered":"2024-04-04T06:16:57.862160+0000","last_unstale":"2024-04-04T06:16:59.378104+0000","last_undegraded":"2024-04-04T06:16:59.378104+0000","last_fullsized":"2024-04-04T06:16:59.378104+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:15:07.419885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222426+0000","last_change":"2024-04-04T06:16:56.816260+0000","last_active":"2024-04-04T06:16:59.222426+0000","last_peered":"2024-04-04T06:16:59.222426+0000","last_clean":"2024-04-04T06:16:59.222426+0000","last_became_active":"2024-04-04T06:16:56.815988+0000","last_became_peered":"2024-04-04T06:16:56.815988+0000","last_unstale":"2024-04-04T06:16:59.222426+0000","last_undegraded":"2024-04-04T06:16:59.222426+0000","last_fullsized":"2024-04-04T06:16:59.222426+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:28.483864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377989+0000","last_change":"2024-04-04T06:16:57.862761+0000","last_active":"2024-04-04T06:16:59.377989+0000","last_peered":"2024-04-04T06:16:59.377989+0000","last_clean":"2024-04-04T06:16:59.377989+0000","last_became_active":"2024-04-04T06:16:57.862519+0000","last_became_peered":"2024-04-04T06:16:57.862519+0000","last_unstale":"2024-04-04T06:16:59.377989+0000","last_undegraded":"2024-04-04T06:16:59.377989+0000","last_fullsized":"2024-04-04T06:16:59.377989+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:16:04.429520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377028+0000","last_change":"2024-04-04T06:16:56.823585+0000","last_active":"2024-04-04T06:16:59.377028+0000","last_peered":"2024-04-04T06:16:59.377028+0000","last_clean":"2024-04-04T06:16:59.377028+0000","last_became_active":"2024-04-04T06:16:56.823294+0000","last_became_peered":"2024-04-04T06:16:56.823294+0000","last_unstale":"2024-04-04T06:16:59.377028+0000","last_undegraded":"2024-04-04T06:16:59.377028+0000","last_fullsized":"2024-04-04T06:16:59.377028+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:02:07.681535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850448+0000","last_change":"2024-04-04T06:16:57.868618+0000","last_active":"2024-04-04T06:16:58.850448+0000","last_peered":"2024-04-04T06:16:58.850448+0000","last_clean":"2024-04-04T06:16:58.850448+0000","last_became_active":"2024-04-04T06:16:57.866435+0000","last_became_peered":"2024-04-04T06:16:57.866435+0000","last_unstale":"2024-04-04T06:16:58.850448+0000","last_undegraded":"2024-04-04T06:16:58.850448+0000","last_fullsized":"2024-04-04T06:16:58.850448+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:42:54.207457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.376665+0000","last_change":"2024-04-04T06:16:56.806783+0000","last_active":"2024-04-04T06:16:59.376665+0000","last_peered":"2024-04-04T06:16:59.376665+0000","last_clean":"2024-04-04T06:16:59.376665+0000","last_became_active":"2024-04-04T06:16:56.806293+0000","last_became_peered":"2024-04-04T06:16:56.806293+0000","last_unstale":"2024-04-04T06:16:59.376665+0000","last_undegraded":"2024-04-04T06:16:59.376665+0000","last_fullsized":"2024-04-04T06:16:59.376665+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:56:06.439167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.866304+0000","last_change":"2024-04-04T06:16:57.867317+0000","last_active":"2024-04-04T06:16:58.866304+0000","last_peered":"2024-04-04T06:16:58.866304+0000","last_clean":"2024-04-04T06:16:58.866304+0000","last_became_active":"2024-04-04T06:16:57.866053+0000","last_became_peered":"2024-04-04T06:16:57.866053+0000","last_unstale":"2024-04-04T06:16:58.866304+0000","last_undegraded":"2024-04-04T06:16:58.866304+0000","last_fullsized":"2024-04-04T06:16:58.866304+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:51:22.883892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850667+0000","last_change":"2024-04-04T06:16:56.817822+0000","last_active":"2024-04-04T06:16:58.850667+0000","last_peered":"2024-04-04T06:16:58.850667+0000","last_clean":"2024-04-04T06:16:58.850667+0000","last_became_active":"2024-04-04T06:16:56.816539+0000","last_became_peered":"2024-04-04T06:16:56.816539+0000","last_unstale":"2024-04-04T06:16:58.850667+0000","last_undegraded":"2024-04-04T06:16:58.850667+0000","last_fullsized":"2024-04-04T06:16:58.850667+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:10:06.219773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.379113+0000","last_change":"2024-04-04T06:16:57.859002+0000","last_active":"2024-04-04T06:16:59.379113+0000","last_peered":"2024-04-04T06:16:59.379113+0000","last_clean":"2024-04-04T06:16:59.379113+0000","last_became_active":"2024-04-04T06:16:57.857269+0000","last_became_peered":"2024-04-04T06:16:57.857269+0000","last_unstale":"2024-04-04T06:16:59.379113+0000","last_undegraded":"2024-04-04T06:16:59.379113+0000","last_fullsized":"2024-04-04T06:16:59.379113+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:05:24.301901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"113'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.866526+0000","last_change":"2024-04-04T06:16:56.819463+0000","last_active":"2024-04-04T06:16:58.866526+0000","last_peered":"2024-04-04T06:16:58.866526+0000","last_clean":"2024-04-04T06:16:58.866526+0000","last_became_active":"2024-04-04T06:16:56.816034+0000","last_became_peered":"2024-04-04T06:16:56.816034+0000","last_unstale":"2024-04-04T06:16:58.866526+0000","last_undegraded":"2024-04-04T06:16:58.866526+0000","last_fullsized":"2024-04-04T06:16:58.866526+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:14:56.497336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.379427+0000","last_change":"2024-04-04T06:16:57.856907+0000","last_active":"2024-04-04T06:16:59.379427+0000","last_peered":"2024-04-04T06:16:59.379427+0000","last_clean":"2024-04-04T06:16:59.379427+0000","last_became_active":"2024-04-04T06:16:57.856616+0000","last_became_peered":"2024-04-04T06:16:57.856616+0000","last_unstale":"2024-04-04T06:16:59.379427+0000","last_undegraded":"2024-04-04T06:16:59.379427+0000","last_fullsized":"2024-04-04T06:16:59.379427+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:34:43.213737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378572+0000","last_change":"2024-04-04T06:16:56.803667+0000","last_active":"2024-04-04T06:16:59.378572+0000","last_peered":"2024-04-04T06:16:59.378572+0000","last_clean":"2024-04-04T06:16:59.378572+0000","last_became_active":"2024-04-04T06:16:56.803353+0000","last_became_peered":"2024-04-04T06:16:56.803353+0000","last_unstale":"2024-04-04T06:16:59.378572+0000","last_undegraded":"2024-04-04T06:16:59.378572+0000","last_fullsized":"2024-04-04T06:16:59.378572+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:42:35.512614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.866190+0000","last_change":"2024-04-04T06:16:57.870240+0000","last_active":"2024-04-04T06:16:58.866190+0000","last_peered":"2024-04-04T06:16:58.866190+0000","last_clean":"2024-04-04T06:16:58.866190+0000","last_became_active":"2024-04-04T06:16:57.869874+0000","last_became_peered":"2024-04-04T06:16:57.869874+0000","last_unstale":"2024-04-04T06:16:58.866190+0000","last_undegraded":"2024-04-04T06:16:58.866190+0000","last_fullsized":"2024-04-04T06:16:58.866190+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:01:10.335447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850959+0000","last_change":"2024-04-04T06:16:56.820108+0000","last_active":"2024-04-04T06:16:58.850959+0000","last_peered":"2024-04-04T06:16:58.850959+0000","last_clean":"2024-04-04T06:16:58.850959+0000","last_became_active":"2024-04-04T06:16:56.819768+0000","last_became_peered":"2024-04-04T06:16:56.819768+0000","last_unstale":"2024-04-04T06:16:58.850959+0000","last_undegraded":"2024-04-04T06:16:58.850959+0000","last_fullsized":"2024-04-04T06:16:58.850959+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:19:35.089037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855977+0000","last_change":"2024-04-04T06:16:57.844825+0000","last_active":"2024-04-04T06:16:58.855977+0000","last_peered":"2024-04-04T06:16:58.855977+0000","last_clean":"2024-04-04T06:16:58.855977+0000","last_became_active":"2024-04-04T06:16:57.843101+0000","last_became_peered":"2024-04-04T06:16:57.843101+0000","last_unstale":"2024-04-04T06:16:58.855977+0000","last_undegraded":"2024-04-04T06:16:58.855977+0000","last_fullsized":"2024-04-04T06:16:58.855977+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:41:12.600534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854882+0000","last_change":"2024-04-04T06:16:56.823376+0000","last_active":"2024-04-04T06:16:58.854882+0000","last_peered":"2024-04-04T06:16:58.854882+0000","last_clean":"2024-04-04T06:16:58.854882+0000","last_became_active":"2024-04-04T06:16:56.822325+0000","last_became_peered":"2024-04-04T06:16:56.822325+0000","last_unstale":"2024-04-04T06:16:58.854882+0000","last_undegraded":"2024-04-04T06:16:58.854882+0000","last_fullsized":"2024-04-04T06:16:58.854882+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:17:31.397079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856868+0000","last_change":"2024-04-04T06:16:57.861595+0000","last_active":"2024-04-04T06:16:58.856868+0000","last_peered":"2024-04-04T06:16:58.856868+0000","last_clean":"2024-04-04T06:16:58.856868+0000","last_became_active":"2024-04-04T06:16:57.860975+0000","last_became_peered":"2024-04-04T06:16:57.860975+0000","last_unstale":"2024-04-04T06:16:58.856868+0000","last_undegraded":"2024-04-04T06:16:58.856868+0000","last_fullsized":"2024-04-04T06:16:58.856868+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:29:44.470791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855757+0000","last_change":"2024-04-04T06:16:56.822556+0000","last_active":"2024-04-04T06:16:58.855757+0000","last_peered":"2024-04-04T06:16:58.855757+0000","last_clean":"2024-04-04T06:16:58.855757+0000","last_became_active":"2024-04-04T06:16:56.821979+0000","last_became_peered":"2024-04-04T06:16:56.821979+0000","last_unstale":"2024-04-04T06:16:58.855757+0000","last_undegraded":"2024-04-04T06:16:58.855757+0000","last_fullsized":"2024-04-04T06:16:58.855757+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:59:42.200129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378485+0000","last_change":"2024-04-04T06:16:57.861874+0000","last_active":"2024-04-04T06:16:59.378485+0000","last_peered":"2024-04-04T06:16:59.378485+0000","last_clean":"2024-04-04T06:16:59.378485+0000","last_became_active":"2024-04-04T06:16:57.861605+0000","last_became_peered":"2024-04-04T06:16:57.861605+0000","last_unstale":"2024-04-04T06:16:59.378485+0000","last_undegraded":"2024-04-04T06:16:59.378485+0000","last_fullsized":"2024-04-04T06:16:59.378485+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:01:10.335447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222503+0000","last_change":"2024-04-04T06:16:56.818500+0000","last_active":"2024-04-04T06:16:59.222503+0000","last_peered":"2024-04-04T06:16:59.222503+0000","last_clean":"2024-04-04T06:16:59.222503+0000","last_became_active":"2024-04-04T06:16:56.816829+0000","last_became_peered":"2024-04-04T06:16:56.816829+0000","last_unstale":"2024-04-04T06:16:59.222503+0000","last_undegraded":"2024-04-04T06:16:59.222503+0000","last_fullsized":"2024-04-04T06:16:59.222503+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:37:13.781466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377554+0000","last_change":"2024-04-04T06:16:57.862871+0000","last_active":"2024-04-04T06:16:59.377554+0000","last_peered":"2024-04-04T06:16:59.377554+0000","last_clean":"2024-04-04T06:16:59.377554+0000","last_became_active":"2024-04-04T06:16:57.857674+0000","last_became_peered":"2024-04-04T06:16:57.857674+0000","last_unstale":"2024-04-04T06:16:59.377554+0000","last_undegraded":"2024-04-04T06:16:59.377554+0000","last_fullsized":"2024-04-04T06:16:59.377554+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:12:53.084370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377182+0000","last_change":"2024-04-04T06:16:56.807000+0000","last_active":"2024-04-04T06:16:59.377182+0000","last_peered":"2024-04-04T06:16:59.377182+0000","last_clean":"2024-04-04T06:16:59.377182+0000","last_became_active":"2024-04-04T06:16:56.806687+0000","last_became_peered":"2024-04-04T06:16:56.806687+0000","last_unstale":"2024-04-04T06:16:59.377182+0000","last_undegraded":"2024-04-04T06:16:59.377182+0000","last_fullsized":"2024-04-04T06:16:59.377182+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:02:40.213409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850291+0000","last_change":"2024-04-04T06:16:57.856787+0000","last_active":"2024-04-04T06:16:58.850291+0000","last_peered":"2024-04-04T06:16:58.850291+0000","last_clean":"2024-04-04T06:16:58.850291+0000","last_became_active":"2024-04-04T06:16:57.856455+0000","last_became_peered":"2024-04-04T06:16:57.856455+0000","last_unstale":"2024-04-04T06:16:58.850291+0000","last_undegraded":"2024-04-04T06:16:58.850291+0000","last_fullsized":"2024-04-04T06:16:58.850291+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:42:36.526974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854824+0000","last_change":"2024-04-04T06:16:56.819104+0000","last_active":"2024-04-04T06:16:58.854824+0000","last_peered":"2024-04-04T06:16:58.854824+0000","last_clean":"2024-04-04T06:16:58.854824+0000","last_became_active":"2024-04-04T06:16:56.818492+0000","last_became_peered":"2024-04-04T06:16:56.818492+0000","last_unstale":"2024-04-04T06:16:58.854824+0000","last_undegraded":"2024-04-04T06:16:58.854824+0000","last_fullsized":"2024-04-04T06:16:58.854824+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:14:15.299577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":338,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378631+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:16:59.378631+0000","last_peered":"2024-04-04T06:16:59.378631+0000","last_clean":"2024-04-04T06:16:59.378631+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:16:59.378631+0000","last_undegraded":"2024-04-04T06:16:59.378631+0000","last_fullsized":"2024-04-04T06:16:59.378631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509578,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37296,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334544,"statfs":{"total":96636764160,"available":96598573056,"internally_reserved":0,"allocated":1998848,"data_stored":1198052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509580,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36712,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335128,"statfs":{"total":96636764160,"available":96599171072,"internally_reserved":0,"allocated":1400832,"data_stored":607138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509581,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1413120,"data_stored":608750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509580,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1396736,"data_stored":607068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509579,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1404928,"data_stored":607192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509581,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2011136,"data_stored":1199160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36712,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335128,"statfs":{"total":96636764160,"available":96599171072,"internally_reserved":0,"allocated":1404928,"data_stored":607672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542282,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36712,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335128,"statfs":{"total":96636764160,"available":96599171072,"internally_reserved":0,"allocated":1396736,"data_stored":607068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:17:07.417 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:17:07.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.638+0000 7f9497a00640 1 Processor -- start 2024-04-04T06:17:07.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.638+0000 7f9497a00640 1 -- start start 2024-04-04T06:17:07.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.639+0000 7f9497a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f949015c390 0x7f9490073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:07.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.639+0000 7f9497a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f94900af830 0x7f9490160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:07.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.639+0000 7f9497a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f949015fe80 con 0x7f9490160b80 2024-04-04T06:17:07.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.639+0000 7f9497a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f949015c770 con 0x7f94900af830 2024-04-04T06:17:07.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.639+0000 7f9497a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f94900ad120 con 0x7f949015c390 2024-04-04T06:17:07.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.640+0000 7f948fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f94900af830 0x7f9490160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:07.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.640+0000 7f948fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f94900af830 0x7f9490160640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59206/0 (socket says 172.21.15.159:59206) 2024-04-04T06:17:07.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.640+0000 7f948fe00640 1 -- 172.21.15.159:0/3646239809 learned_addr learned my addr 172.21.15.159:0/3646239809 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:07.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.640+0000 7f9495000640 1 --2- 172.21.15.159:0/3646239809 >> v2:172.21.15.178:3300/0 conn(0x7f949015c390 0x7f9490073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:07.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.640+0000 7f948fe00640 1 -- 172.21.15.159:0/3646239809 >> v2:172.21.15.178:3300/0 conn(0x7f949015c390 msgr2=0x7f9490073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:07.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.640+0000 7f948fe00640 1 --2- 172.21.15.159:0/3646239809 >> v2:172.21.15.178:3300/0 conn(0x7f949015c390 0x7f9490073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:07.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.640+0000 7f948fe00640 1 -- 172.21.15.159:0/3646239809 >> v1:172.21.15.178:6789/0 conn(0x7f9490160b80 legacy=0x7f94900d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:07.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.640+0000 7f948fe00640 1 -- 172.21.15.159:0/3646239809 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94900ad400 con 0x7f94900af830 2024-04-04T06:17:07.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.642+0000 7f948fe00640 1 --2- 172.21.15.159:0/3646239809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94900af830 0x7f9490160640 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f94900b63e0 tx=0x7f947c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d49a1b46546ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:07.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.643+0000 7f948f400640 1 -- 172.21.15.159:0/3646239809 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c035020 con 0x7f94900af830 2024-04-04T06:17:07.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.643+0000 7f948f400640 1 -- 172.21.15.159:0/3646239809 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f947c031870 con 0x7f94900af830 2024-04-04T06:17:07.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.643+0000 7f948f400640 1 -- 172.21.15.159:0/3646239809 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c031c40 con 0x7f94900af830 2024-04-04T06:17:07.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.643+0000 7f9497a00640 1 -- 172.21.15.159:0/3646239809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94900af830 msgr2=0x7f9490160640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:07.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.643+0000 7f9497a00640 1 --2- 172.21.15.159:0/3646239809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94900af830 0x7f9490160640 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f94900b63e0 tx=0x7f947c02b910 comp rx=0 tx=0).stop 2024-04-04T06:17:07.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.643+0000 7f9497a00640 1 -- 172.21.15.159:0/3646239809 shutdown_connections 2024-04-04T06:17:07.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.643+0000 7f9497a00640 1 --2- 172.21.15.159:0/3646239809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94900af830 0x7f9490160640 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:07.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.643+0000 7f9497a00640 1 --2- 172.21.15.159:0/3646239809 >> v2:172.21.15.178:3300/0 conn(0x7f949015c390 0x7f9490073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:07.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.643+0000 7f9497a00640 1 -- 172.21.15.159:0/3646239809 >> 172.21.15.159:0/3646239809 conn(0x7f949006b990 msgr2=0x7f949006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:07.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.644+0000 7f9497a00640 1 -- 172.21.15.159:0/3646239809 shutdown_connections 2024-04-04T06:17:07.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.644+0000 7f9497a00640 1 -- 172.21.15.159:0/3646239809 wait complete. 2024-04-04T06:17:07.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.645+0000 7f9497a00640 1 Processor -- start 2024-04-04T06:17:07.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.645+0000 7f9497a00640 1 -- start start 2024-04-04T06:17:07.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.646+0000 7f9497a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94900af830 0x7f9490145f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:07.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.646+0000 7f9497a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f949015c390 0x7f94900afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:07.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.646+0000 7f9497a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9490160b80 0x7f949015c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:07.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.646+0000 7f9497a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f94900cf980 con 0x7f949015c390 2024-04-04T06:17:07.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.646+0000 7f9497a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9490160120 con 0x7f94900af830 2024-04-04T06:17:07.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.646+0000 7f9497a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f94900d0d60 con 0x7f9490160b80 2024-04-04T06:17:07.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.647+0000 7f948fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f949015c390 0x7f94900afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:07.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.647+0000 7f948fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f949015c390 0x7f94900afff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59208/0 (socket says 172.21.15.159:59208) 2024-04-04T06:17:07.641 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.647+0000 7f948fe00640 1 -- 172.21.15.159:0/2889915094 learned_addr learned my addr 172.21.15.159:0/2889915094 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:07.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.647+0000 7f9495000640 1 --2- 172.21.15.159:0/2889915094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94900af830 0x7f9490145f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:07.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.647+0000 7f9495a00640 1 --2- 172.21.15.159:0/2889915094 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9490160b80 0x7f949015c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:07.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.647+0000 7f948fe00640 1 -- 172.21.15.159:0/2889915094 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9490160b80 msgr2=0x7f949015c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:07.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.647+0000 7f948fe00640 1 --2- 172.21.15.159:0/2889915094 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9490160b80 0x7f949015c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:07.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.647+0000 7f948fe00640 1 -- 172.21.15.159:0/2889915094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94900af830 msgr2=0x7f9490145f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:07.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.647+0000 7f948fe00640 1 --2- 172.21.15.159:0/2889915094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94900af830 0x7f9490145f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:07.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.647+0000 7f948fe00640 1 -- 172.21.15.159:0/2889915094 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94900b0950 con 0x7f949015c390 2024-04-04T06:17:07.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.647+0000 7f9495a00640 1 --2- 172.21.15.159:0/2889915094 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9490160b80 0x7f949015c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:07.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.647+0000 7f948fe00640 1 --2- 172.21.15.159:0/2889915094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f949015c390 0x7f94900afff0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f947c009960 tx=0x7f947c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:07.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.648+0000 7f948cc00640 1 -- 172.21.15.159:0/2889915094 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c035040 con 0x7f949015c390 2024-04-04T06:17:07.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.648+0000 7f948cc00640 1 -- 172.21.15.159:0/2889915094 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f947c030d00 con 0x7f949015c390 2024-04-04T06:17:07.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.648+0000 7f948cc00640 1 -- 172.21.15.159:0/2889915094 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c0456c0 con 0x7f949015c390 2024-04-04T06:17:07.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.648+0000 7f9497a00640 1 -- 172.21.15.159:0/2889915094 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94900b0bf0 con 0x7f949015c390 2024-04-04T06:17:07.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.648+0000 7f9497a00640 1 -- 172.21.15.159:0/2889915094 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94900b0530 con 0x7f949015c390 2024-04-04T06:17:07.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.649+0000 7f9497a00640 1 -- 172.21.15.159:0/2889915094 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94900685c0 con 0x7f949015c390 2024-04-04T06:17:07.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.650+0000 7f948cc00640 1 -- 172.21.15.159:0/2889915094 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f947c038020 con 0x7f949015c390 2024-04-04T06:17:07.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.651+0000 7f948cc00640 1 --2- 172.21.15.159:0/2889915094 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f947006c8f0 0x7f947006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:07.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.651+0000 7f948cc00640 1 -- 172.21.15.159:0/2889915094 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f947c0addd0 con 0x7f949015c390 2024-04-04T06:17:07.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.651+0000 7f9495000640 1 --2- 172.21.15.159:0/2889915094 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f947006c8f0 0x7f947006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:07.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.652+0000 7f9495000640 1 --2- 172.21.15.159:0/2889915094 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f947006c8f0 0x7f947006edb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9484006fd0 tx=0x7f9484008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:07.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.655+0000 7f948cc00640 1 -- 172.21.15.159:0/2889915094 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f947c07c1f0 con 0x7f949015c390 2024-04-04T06:17:07.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.852+0000 7f9497a00640 1 -- 172.21.15.159:0/2889915094 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f949015a8a0 con 0x7f947006c8f0 2024-04-04T06:17:07.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.872+0000 7f948cc00640 1 -- 172.21.15.159:0/2889915094 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252371 (secure 0 0 0) 0x7f949015a8a0 con 0x7f947006c8f0 2024-04-04T06:17:07.870 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:07.872 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:17:07.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.879+0000 7f9497a00640 1 -- 172.21.15.159:0/2889915094 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f947006c8f0 msgr2=0x7f947006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:07.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.879+0000 7f9497a00640 1 --2- 172.21.15.159:0/2889915094 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f947006c8f0 0x7f947006edb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9484006fd0 tx=0x7f9484008040 comp rx=0 tx=0).stop 2024-04-04T06:17:07.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.879+0000 7f9497a00640 1 -- 172.21.15.159:0/2889915094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f949015c390 msgr2=0x7f94900afff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:07.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.879+0000 7f9497a00640 1 --2- 172.21.15.159:0/2889915094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f949015c390 0x7f94900afff0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f947c009960 tx=0x7f947c0046c0 comp rx=0 tx=0).stop 2024-04-04T06:17:07.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.879+0000 7f9497a00640 1 -- 172.21.15.159:0/2889915094 shutdown_connections 2024-04-04T06:17:07.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.879+0000 7f9497a00640 1 --2- 172.21.15.159:0/2889915094 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9490160b80 0x7f949015c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:07.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.879+0000 7f9497a00640 1 --2- 172.21.15.159:0/2889915094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f949015c390 0x7f94900afff0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:07.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.879+0000 7f9497a00640 1 --2- 172.21.15.159:0/2889915094 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f947006c8f0 0x7f947006edb0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:07.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.879+0000 7f9497a00640 1 --2- 172.21.15.159:0/2889915094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94900af830 0x7f9490145f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:07.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.879+0000 7f9497a00640 1 -- 172.21.15.159:0/2889915094 >> 172.21.15.159:0/2889915094 conn(0x7f949006b990 msgr2=0x7f949015ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:07.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.879+0000 7f9497a00640 1 -- 172.21.15.159:0/2889915094 shutdown_connections 2024-04-04T06:17:07.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:07.879+0000 7f9497a00640 1 -- 172.21.15.159:0/2889915094 wait complete. 2024-04-04T06:17:07.898 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2024-04-04T06:17:06.964719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278240,"kb_used_data":12136,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754696480,"statfs":{"total":773094113280,"available":772809195520,"internally_reserved":0,"allocated":12427264,"data_stored":6042100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.143349"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378643+0000","last_change":"2024-04-04T06:16:57.861743+0000","last_active":"2024-04-04T06:16:59.378643+0000","last_peered":"2024-04-04T06:16:59.378643+0000","last_clean":"2024-04-04T06:16:59.378643+0000","last_became_active":"2024-04-04T06:16:57.860873+0000","last_became_peered":"2024-04-04T06:16:57.860873+0000","last_unstale":"2024-04-04T06:16:59.378643+0000","last_undegraded":"2024-04-04T06:16:59.378643+0000","last_fullsized":"2024-04-04T06:16:59.378643+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:18:01.077209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856681+0000","last_change":"2024-04-04T06:16:57.850062+0000","last_active":"2024-04-04T06:16:58.856681+0000","last_peered":"2024-04-04T06:16:58.856681+0000","last_clean":"2024-04-04T06:16:58.856681+0000","last_became_active":"2024-04-04T06:16:57.849768+0000","last_became_peered":"2024-04-04T06:16:57.849768+0000","last_unstale":"2024-04-04T06:16:58.856681+0000","last_undegraded":"2024-04-04T06:16:58.856681+0000","last_fullsized":"2024-04-04T06:16:58.856681+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:20:36.274557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378633+0000","last_change":"2024-04-04T06:16:57.857033+0000","last_active":"2024-04-04T06:16:59.378633+0000","last_peered":"2024-04-04T06:16:59.378633+0000","last_clean":"2024-04-04T06:16:59.378633+0000","last_became_active":"2024-04-04T06:16:57.856617+0000","last_became_peered":"2024-04-04T06:16:57.856617+0000","last_unstale":"2024-04-04T06:16:59.378633+0000","last_undegraded":"2024-04-04T06:16:59.378633+0000","last_fullsized":"2024-04-04T06:16:59.378633+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:51:22.883892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.379030+0000","last_change":"2024-04-04T06:16:57.858984+0000","last_active":"2024-04-04T06:16:59.379030+0000","last_peered":"2024-04-04T06:16:59.379030+0000","last_clean":"2024-04-04T06:16:59.379030+0000","last_became_active":"2024-04-04T06:16:57.858427+0000","last_became_peered":"2024-04-04T06:16:57.858427+0000","last_unstale":"2024-04-04T06:16:59.379030+0000","last_undegraded":"2024-04-04T06:16:59.379030+0000","last_fullsized":"2024-04-04T06:16:59.379030+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:08:45.785102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222986+0000","last_change":"2024-04-04T06:16:57.852939+0000","last_active":"2024-04-04T06:16:59.222986+0000","last_peered":"2024-04-04T06:16:59.222986+0000","last_clean":"2024-04-04T06:16:59.222986+0000","last_became_active":"2024-04-04T06:16:57.852656+0000","last_became_peered":"2024-04-04T06:16:57.852656+0000","last_unstale":"2024-04-04T06:16:59.222986+0000","last_undegraded":"2024-04-04T06:16:59.222986+0000","last_fullsized":"2024-04-04T06:16:59.222986+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:04:27.627269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.376770+0000","last_change":"2024-04-04T06:16:57.858617+0000","last_active":"2024-04-04T06:16:59.376770+0000","last_peered":"2024-04-04T06:16:59.376770+0000","last_clean":"2024-04-04T06:16:59.376770+0000","last_became_active":"2024-04-04T06:16:57.858300+0000","last_became_peered":"2024-04-04T06:16:57.858300+0000","last_unstale":"2024-04-04T06:16:59.376770+0000","last_undegraded":"2024-04-04T06:16:59.376770+0000","last_fullsized":"2024-04-04T06:16:59.376770+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:36:49.667730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855381+0000","last_change":"2024-04-04T06:16:57.853261+0000","last_active":"2024-04-04T06:16:58.855381+0000","last_peered":"2024-04-04T06:16:58.855381+0000","last_clean":"2024-04-04T06:16:58.855381+0000","last_became_active":"2024-04-04T06:16:57.852925+0000","last_became_peered":"2024-04-04T06:16:57.852925+0000","last_unstale":"2024-04-04T06:16:58.855381+0000","last_undegraded":"2024-04-04T06:16:58.855381+0000","last_fullsized":"2024-04-04T06:16:58.855381+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:43:19.761558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377985+0000","last_change":"2024-04-04T06:16:57.854647+0000","last_active":"2024-04-04T06:16:59.377985+0000","last_peered":"2024-04-04T06:16:59.377985+0000","last_clean":"2024-04-04T06:16:59.377985+0000","last_became_active":"2024-04-04T06:16:57.854376+0000","last_became_peered":"2024-04-04T06:16:57.854376+0000","last_unstale":"2024-04-04T06:16:59.377985+0000","last_undegraded":"2024-04-04T06:16:59.377985+0000","last_fullsized":"2024-04-04T06:16:59.377985+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:18:08.210854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378876+0000","last_change":"2024-04-04T06:16:57.859251+0000","last_active":"2024-04-04T06:16:59.378876+0000","last_peered":"2024-04-04T06:16:59.378876+0000","last_clean":"2024-04-04T06:16:59.378876+0000","last_became_active":"2024-04-04T06:16:57.857756+0000","last_became_peered":"2024-04-04T06:16:57.857756+0000","last_unstale":"2024-04-04T06:16:59.378876+0000","last_undegraded":"2024-04-04T06:16:59.378876+0000","last_fullsized":"2024-04-04T06:16:59.378876+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:39:44.553729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378627+0000","last_change":"2024-04-04T06:16:57.865147+0000","last_active":"2024-04-04T06:16:59.378627+0000","last_peered":"2024-04-04T06:16:59.378627+0000","last_clean":"2024-04-04T06:16:59.378627+0000","last_became_active":"2024-04-04T06:16:57.863954+0000","last_became_peered":"2024-04-04T06:16:57.863954+0000","last_unstale":"2024-04-04T06:16:59.378627+0000","last_undegraded":"2024-04-04T06:16:59.378627+0000","last_fullsized":"2024-04-04T06:16:59.378627+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:45:25.455248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.857193+0000","last_change":"2024-04-04T06:16:57.852970+0000","last_active":"2024-04-04T06:16:58.857193+0000","last_peered":"2024-04-04T06:16:58.857193+0000","last_clean":"2024-04-04T06:16:58.857193+0000","last_became_active":"2024-04-04T06:16:57.852398+0000","last_became_peered":"2024-04-04T06:16:57.852398+0000","last_unstale":"2024-04-04T06:16:58.857193+0000","last_undegraded":"2024-04-04T06:16:58.857193+0000","last_fullsized":"2024-04-04T06:16:58.857193+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:25:13.863001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.379292+0000","last_change":"2024-04-04T06:16:57.868403+0000","last_active":"2024-04-04T06:16:59.379292+0000","last_peered":"2024-04-04T06:16:59.379292+0000","last_clean":"2024-04-04T06:16:59.379292+0000","last_became_active":"2024-04-04T06:16:57.867661+0000","last_became_peered":"2024-04-04T06:16:57.867661+0000","last_unstale":"2024-04-04T06:16:59.379292+0000","last_undegraded":"2024-04-04T06:16:59.379292+0000","last_fullsized":"2024-04-04T06:16:59.379292+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:49:12.167101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378304+0000","last_change":"2024-04-04T06:16:57.864603+0000","last_active":"2024-04-04T06:16:59.378304+0000","last_peered":"2024-04-04T06:16:59.378304+0000","last_clean":"2024-04-04T06:16:59.378304+0000","last_became_active":"2024-04-04T06:16:57.864334+0000","last_became_peered":"2024-04-04T06:16:57.864334+0000","last_unstale":"2024-04-04T06:16:59.378304+0000","last_undegraded":"2024-04-04T06:16:59.378304+0000","last_fullsized":"2024-04-04T06:16:59.378304+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:54:31.781352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377932+0000","last_change":"2024-04-04T06:16:57.860035+0000","last_active":"2024-04-04T06:16:59.377932+0000","last_peered":"2024-04-04T06:16:59.377932+0000","last_clean":"2024-04-04T06:16:59.377932+0000","last_became_active":"2024-04-04T06:16:57.858072+0000","last_became_peered":"2024-04-04T06:16:57.858072+0000","last_unstale":"2024-04-04T06:16:59.377932+0000","last_undegraded":"2024-04-04T06:16:59.377932+0000","last_fullsized":"2024-04-04T06:16:59.377932+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:52:19.546676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.376787+0000","last_change":"2024-04-04T06:16:57.853280+0000","last_active":"2024-04-04T06:16:59.376787+0000","last_peered":"2024-04-04T06:16:59.376787+0000","last_clean":"2024-04-04T06:16:59.376787+0000","last_became_active":"2024-04-04T06:16:57.852744+0000","last_became_peered":"2024-04-04T06:16:57.852744+0000","last_unstale":"2024-04-04T06:16:59.376787+0000","last_undegraded":"2024-04-04T06:16:59.376787+0000","last_fullsized":"2024-04-04T06:16:59.376787+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:04:27.627269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.376384+0000","last_change":"2024-04-04T06:16:57.866193+0000","last_active":"2024-04-04T06:16:59.376384+0000","last_peered":"2024-04-04T06:16:59.376384+0000","last_clean":"2024-04-04T06:16:59.376384+0000","last_became_active":"2024-04-04T06:16:57.865888+0000","last_became_peered":"2024-04-04T06:16:57.865888+0000","last_unstale":"2024-04-04T06:16:59.376384+0000","last_undegraded":"2024-04-04T06:16:59.376384+0000","last_fullsized":"2024-04-04T06:16:59.376384+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:46:20.508997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856787+0000","last_change":"2024-04-04T06:16:57.861016+0000","last_active":"2024-04-04T06:16:58.856787+0000","last_peered":"2024-04-04T06:16:58.856787+0000","last_clean":"2024-04-04T06:16:58.856787+0000","last_became_active":"2024-04-04T06:16:57.860213+0000","last_became_peered":"2024-04-04T06:16:57.860213+0000","last_unstale":"2024-04-04T06:16:58.856787+0000","last_undegraded":"2024-04-04T06:16:58.856787+0000","last_fullsized":"2024-04-04T06:16:58.856787+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:41:22.250390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.866106+0000","last_change":"2024-04-04T06:16:57.852666+0000","last_active":"2024-04-04T06:16:58.866106+0000","last_peered":"2024-04-04T06:16:58.866106+0000","last_clean":"2024-04-04T06:16:58.866106+0000","last_became_active":"2024-04-04T06:16:57.851465+0000","last_became_peered":"2024-04-04T06:16:57.851465+0000","last_unstale":"2024-04-04T06:16:58.866106+0000","last_undegraded":"2024-04-04T06:16:58.866106+0000","last_fullsized":"2024-04-04T06:16:58.866106+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:54:31.781352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378739+0000","last_change":"2024-04-04T06:16:57.873661+0000","last_active":"2024-04-04T06:16:59.378739+0000","last_peered":"2024-04-04T06:16:59.378739+0000","last_clean":"2024-04-04T06:16:59.378739+0000","last_became_active":"2024-04-04T06:16:57.873342+0000","last_became_peered":"2024-04-04T06:16:57.873342+0000","last_unstale":"2024-04-04T06:16:59.378739+0000","last_undegraded":"2024-04-04T06:16:59.378739+0000","last_fullsized":"2024-04-04T06:16:59.378739+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:05:27.280058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856739+0000","last_change":"2024-04-04T06:16:57.858770+0000","last_active":"2024-04-04T06:16:58.856739+0000","last_peered":"2024-04-04T06:16:58.856739+0000","last_clean":"2024-04-04T06:16:58.856739+0000","last_became_active":"2024-04-04T06:16:57.856816+0000","last_became_peered":"2024-04-04T06:16:57.856816+0000","last_unstale":"2024-04-04T06:16:58.856739+0000","last_undegraded":"2024-04-04T06:16:58.856739+0000","last_fullsized":"2024-04-04T06:16:58.856739+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:42:54.207457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.857204+0000","last_change":"2024-04-04T06:16:57.840896+0000","last_active":"2024-04-04T06:16:58.857204+0000","last_peered":"2024-04-04T06:16:58.857204+0000","last_clean":"2024-04-04T06:16:58.857204+0000","last_became_active":"2024-04-04T06:16:57.840145+0000","last_became_peered":"2024-04-04T06:16:57.840145+0000","last_unstale":"2024-04-04T06:16:58.857204+0000","last_undegraded":"2024-04-04T06:16:58.857204+0000","last_fullsized":"2024-04-04T06:16:58.857204+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:56:07.453527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850326+0000","last_change":"2024-04-04T06:16:57.862396+0000","last_active":"2024-04-04T06:16:58.850326+0000","last_peered":"2024-04-04T06:16:58.850326+0000","last_clean":"2024-04-04T06:16:58.850326+0000","last_became_active":"2024-04-04T06:16:57.862112+0000","last_became_peered":"2024-04-04T06:16:57.862112+0000","last_unstale":"2024-04-04T06:16:58.850326+0000","last_undegraded":"2024-04-04T06:16:58.850326+0000","last_fullsized":"2024-04-04T06:16:58.850326+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:20:36.274557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855555+0000","last_change":"2024-04-04T06:16:57.855453+0000","last_active":"2024-04-04T06:16:58.855555+0000","last_peered":"2024-04-04T06:16:58.855555+0000","last_clean":"2024-04-04T06:16:58.855555+0000","last_became_active":"2024-04-04T06:16:57.855172+0000","last_became_peered":"2024-04-04T06:16:57.855172+0000","last_unstale":"2024-04-04T06:16:58.855555+0000","last_undegraded":"2024-04-04T06:16:58.855555+0000","last_fullsized":"2024-04-04T06:16:58.855555+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:17:03.994940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850188+0000","last_change":"2024-04-04T06:16:57.854852+0000","last_active":"2024-04-04T06:16:58.850188+0000","last_peered":"2024-04-04T06:16:58.850188+0000","last_clean":"2024-04-04T06:16:58.850188+0000","last_became_active":"2024-04-04T06:16:57.854610+0000","last_became_peered":"2024-04-04T06:16:57.854610+0000","last_unstale":"2024-04-04T06:16:58.850188+0000","last_undegraded":"2024-04-04T06:16:58.850188+0000","last_fullsized":"2024-04-04T06:16:58.850188+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:17:03.994940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.857033+0000","last_change":"2024-04-04T06:16:57.859703+0000","last_active":"2024-04-04T06:16:58.857033+0000","last_peered":"2024-04-04T06:16:58.857033+0000","last_clean":"2024-04-04T06:16:58.857033+0000","last_became_active":"2024-04-04T06:16:57.859398+0000","last_became_peered":"2024-04-04T06:16:57.859398+0000","last_unstale":"2024-04-04T06:16:58.857033+0000","last_undegraded":"2024-04-04T06:16:58.857033+0000","last_fullsized":"2024-04-04T06:16:58.857033+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:29:25.780543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378891+0000","last_change":"2024-04-04T06:16:57.861831+0000","last_active":"2024-04-04T06:16:59.378891+0000","last_peered":"2024-04-04T06:16:59.378891+0000","last_clean":"2024-04-04T06:16:59.378891+0000","last_became_active":"2024-04-04T06:16:57.861541+0000","last_became_peered":"2024-04-04T06:16:57.861541+0000","last_unstale":"2024-04-04T06:16:59.378891+0000","last_undegraded":"2024-04-04T06:16:59.378891+0000","last_fullsized":"2024-04-04T06:16:59.378891+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:45:32.671346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378148+0000","last_change":"2024-04-04T06:16:57.871771+0000","last_active":"2024-04-04T06:16:59.378148+0000","last_peered":"2024-04-04T06:16:59.378148+0000","last_clean":"2024-04-04T06:16:59.378148+0000","last_became_active":"2024-04-04T06:16:57.871480+0000","last_became_peered":"2024-04-04T06:16:57.871480+0000","last_unstale":"2024-04-04T06:16:59.378148+0000","last_undegraded":"2024-04-04T06:16:59.378148+0000","last_fullsized":"2024-04-04T06:16:59.378148+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:46:04.591130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850668+0000","last_change":"2024-04-04T06:16:57.867129+0000","last_active":"2024-04-04T06:16:58.850668+0000","last_peered":"2024-04-04T06:16:58.850668+0000","last_clean":"2024-04-04T06:16:58.850668+0000","last_became_active":"2024-04-04T06:16:57.866770+0000","last_became_peered":"2024-04-04T06:16:57.866770+0000","last_unstale":"2024-04-04T06:16:58.850668+0000","last_undegraded":"2024-04-04T06:16:58.850668+0000","last_fullsized":"2024-04-04T06:16:58.850668+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:29:44.470791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856807+0000","last_change":"2024-04-04T06:16:57.859118+0000","last_active":"2024-04-04T06:16:58.856807+0000","last_peered":"2024-04-04T06:16:58.856807+0000","last_clean":"2024-04-04T06:16:58.856807+0000","last_became_active":"2024-04-04T06:16:57.858242+0000","last_became_peered":"2024-04-04T06:16:57.858242+0000","last_unstale":"2024-04-04T06:16:58.856807+0000","last_undegraded":"2024-04-04T06:16:58.856807+0000","last_fullsized":"2024-04-04T06:16:58.856807+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:16:58.119204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855126+0000","last_change":"2024-04-04T06:16:57.838809+0000","last_active":"2024-04-04T06:16:58.855126+0000","last_peered":"2024-04-04T06:16:58.855126+0000","last_clean":"2024-04-04T06:16:58.855126+0000","last_became_active":"2024-04-04T06:16:57.837497+0000","last_became_peered":"2024-04-04T06:16:57.837497+0000","last_unstale":"2024-04-04T06:16:58.855126+0000","last_undegraded":"2024-04-04T06:16:58.855126+0000","last_fullsized":"2024-04-04T06:16:58.855126+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:05:00.308975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377216+0000","last_change":"2024-04-04T06:16:57.863907+0000","last_active":"2024-04-04T06:16:59.377216+0000","last_peered":"2024-04-04T06:16:59.377216+0000","last_clean":"2024-04-04T06:16:59.377216+0000","last_became_active":"2024-04-04T06:16:57.863538+0000","last_became_peered":"2024-04-04T06:16:57.863538+0000","last_unstale":"2024-04-04T06:16:59.377216+0000","last_undegraded":"2024-04-04T06:16:59.377216+0000","last_fullsized":"2024-04-04T06:16:59.377216+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:53:37.135784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.376938+0000","last_change":"2024-04-04T06:16:57.853449+0000","last_active":"2024-04-04T06:16:59.376938+0000","last_peered":"2024-04-04T06:16:59.376938+0000","last_clean":"2024-04-04T06:16:59.376938+0000","last_became_active":"2024-04-04T06:16:57.853171+0000","last_became_peered":"2024-04-04T06:16:57.853171+0000","last_unstale":"2024-04-04T06:16:59.376938+0000","last_undegraded":"2024-04-04T06:16:59.376938+0000","last_fullsized":"2024-04-04T06:16:59.376938+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:27:39.563053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378077+0000","last_change":"2024-04-04T06:16:57.861937+0000","last_active":"2024-04-04T06:16:59.378077+0000","last_peered":"2024-04-04T06:16:59.378077+0000","last_clean":"2024-04-04T06:16:59.378077+0000","last_became_active":"2024-04-04T06:16:57.861668+0000","last_became_peered":"2024-04-04T06:16:57.861668+0000","last_unstale":"2024-04-04T06:16:59.378077+0000","last_undegraded":"2024-04-04T06:16:59.378077+0000","last_fullsized":"2024-04-04T06:16:59.378077+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:38:00.487062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"112'4","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855930+0000","last_change":"2024-04-04T06:16:56.826026+0000","last_active":"2024-04-04T06:16:58.855930+0000","last_peered":"2024-04-04T06:16:58.855930+0000","last_clean":"2024-04-04T06:16:58.855930+0000","last_became_active":"2024-04-04T06:16:56.825749+0000","last_became_peered":"2024-04-04T06:16:56.825749+0000","last_unstale":"2024-04-04T06:16:58.855930+0000","last_undegraded":"2024-04-04T06:16:58.855930+0000","last_fullsized":"2024-04-04T06:16:58.855930+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:44:30.609701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.866566+0000","last_change":"2024-04-04T06:16:57.859524+0000","last_active":"2024-04-04T06:16:58.866566+0000","last_peered":"2024-04-04T06:16:58.866566+0000","last_clean":"2024-04-04T06:16:58.866566+0000","last_became_active":"2024-04-04T06:16:57.859203+0000","last_became_peered":"2024-04-04T06:16:57.859203+0000","last_unstale":"2024-04-04T06:16:58.866566+0000","last_undegraded":"2024-04-04T06:16:58.866566+0000","last_fullsized":"2024-04-04T06:16:58.866566+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:09:22.693053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378327+0000","last_change":"2024-04-04T06:16:56.828520+0000","last_active":"2024-04-04T06:16:59.378327+0000","last_peered":"2024-04-04T06:16:59.378327+0000","last_clean":"2024-04-04T06:16:59.378327+0000","last_became_active":"2024-04-04T06:16:56.828125+0000","last_became_peered":"2024-04-04T06:16:56.828125+0000","last_unstale":"2024-04-04T06:16:59.378327+0000","last_undegraded":"2024-04-04T06:16:59.378327+0000","last_fullsized":"2024-04-04T06:16:59.378327+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:04:22.860294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377506+0000","last_change":"2024-04-04T06:16:57.859001+0000","last_active":"2024-04-04T06:16:59.377506+0000","last_peered":"2024-04-04T06:16:59.377506+0000","last_clean":"2024-04-04T06:16:59.377506+0000","last_became_active":"2024-04-04T06:16:57.858712+0000","last_became_peered":"2024-04-04T06:16:57.858712+0000","last_unstale":"2024-04-04T06:16:59.377506+0000","last_undegraded":"2024-04-04T06:16:59.377506+0000","last_fullsized":"2024-04-04T06:16:59.377506+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:09:33.583383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222894+0000","last_change":"2024-04-04T06:16:56.817318+0000","last_active":"2024-04-04T06:16:59.222894+0000","last_peered":"2024-04-04T06:16:59.222894+0000","last_clean":"2024-04-04T06:16:59.222894+0000","last_became_active":"2024-04-04T06:16:56.816995+0000","last_became_peered":"2024-04-04T06:16:56.816995+0000","last_unstale":"2024-04-04T06:16:59.222894+0000","last_undegraded":"2024-04-04T06:16:59.222894+0000","last_fullsized":"2024-04-04T06:16:59.222894+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:02:07.681535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378954+0000","last_change":"2024-04-04T06:16:57.869420+0000","last_active":"2024-04-04T06:16:59.378954+0000","last_peered":"2024-04-04T06:16:59.378954+0000","last_clean":"2024-04-04T06:16:59.378954+0000","last_became_active":"2024-04-04T06:16:57.869124+0000","last_became_peered":"2024-04-04T06:16:57.869124+0000","last_unstale":"2024-04-04T06:16:59.378954+0000","last_undegraded":"2024-04-04T06:16:59.378954+0000","last_fullsized":"2024-04-04T06:16:59.378954+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:10:39.218040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222549+0000","last_change":"2024-04-04T06:16:56.813821+0000","last_active":"2024-04-04T06:16:59.222549+0000","last_peered":"2024-04-04T06:16:59.222549+0000","last_clean":"2024-04-04T06:16:59.222549+0000","last_became_active":"2024-04-04T06:16:56.813555+0000","last_became_peered":"2024-04-04T06:16:56.813555+0000","last_unstale":"2024-04-04T06:16:59.222549+0000","last_undegraded":"2024-04-04T06:16:59.222549+0000","last_fullsized":"2024-04-04T06:16:59.222549+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:22:21.703895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377925+0000","last_change":"2024-04-04T06:16:57.858500+0000","last_active":"2024-04-04T06:16:59.377925+0000","last_peered":"2024-04-04T06:16:59.377925+0000","last_clean":"2024-04-04T06:16:59.377925+0000","last_became_active":"2024-04-04T06:16:57.856744+0000","last_became_peered":"2024-04-04T06:16:57.856744+0000","last_unstale":"2024-04-04T06:16:59.377925+0000","last_undegraded":"2024-04-04T06:16:59.377925+0000","last_fullsized":"2024-04-04T06:16:59.377925+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:44:29.963390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854845+0000","last_change":"2024-04-04T06:16:56.813703+0000","last_active":"2024-04-04T06:16:58.854845+0000","last_peered":"2024-04-04T06:16:58.854845+0000","last_clean":"2024-04-04T06:16:58.854845+0000","last_became_active":"2024-04-04T06:16:56.813224+0000","last_became_peered":"2024-04-04T06:16:56.813224+0000","last_unstale":"2024-04-04T06:16:58.854845+0000","last_undegraded":"2024-04-04T06:16:58.854845+0000","last_fullsized":"2024-04-04T06:16:58.854845+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:28:44.071981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856364+0000","last_change":"2024-04-04T06:16:57.851938+0000","last_active":"2024-04-04T06:16:58.856364+0000","last_peered":"2024-04-04T06:16:58.856364+0000","last_clean":"2024-04-04T06:16:58.856364+0000","last_became_active":"2024-04-04T06:16:57.850346+0000","last_became_peered":"2024-04-04T06:16:57.850346+0000","last_unstale":"2024-04-04T06:16:58.856364+0000","last_undegraded":"2024-04-04T06:16:58.856364+0000","last_fullsized":"2024-04-04T06:16:58.856364+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:42:36.526974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850561+0000","last_change":"2024-04-04T06:16:56.822599+0000","last_active":"2024-04-04T06:16:58.850561+0000","last_peered":"2024-04-04T06:16:58.850561+0000","last_clean":"2024-04-04T06:16:58.850561+0000","last_became_active":"2024-04-04T06:16:56.822130+0000","last_became_peered":"2024-04-04T06:16:56.822130+0000","last_unstale":"2024-04-04T06:16:58.850561+0000","last_undegraded":"2024-04-04T06:16:58.850561+0000","last_fullsized":"2024-04-04T06:16:58.850561+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:59:42.200129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.376345+0000","last_change":"2024-04-04T06:16:57.849755+0000","last_active":"2024-04-04T06:16:59.376345+0000","last_peered":"2024-04-04T06:16:59.376345+0000","last_clean":"2024-04-04T06:16:59.376345+0000","last_became_active":"2024-04-04T06:16:57.849341+0000","last_became_peered":"2024-04-04T06:16:57.849341+0000","last_unstale":"2024-04-04T06:16:59.376345+0000","last_undegraded":"2024-04-04T06:16:59.376345+0000","last_fullsized":"2024-04-04T06:16:59.376345+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:52:09.021010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855249+0000","last_change":"2024-04-04T06:16:56.822615+0000","last_active":"2024-04-04T06:16:58.855249+0000","last_peered":"2024-04-04T06:16:58.855249+0000","last_clean":"2024-04-04T06:16:58.855249+0000","last_became_active":"2024-04-04T06:16:56.819761+0000","last_became_peered":"2024-04-04T06:16:56.819761+0000","last_unstale":"2024-04-04T06:16:58.855249+0000","last_undegraded":"2024-04-04T06:16:58.855249+0000","last_fullsized":"2024-04-04T06:16:58.855249+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:10:06.219773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854643+0000","last_change":"2024-04-04T06:16:57.853254+0000","last_active":"2024-04-04T06:16:58.854643+0000","last_peered":"2024-04-04T06:16:58.854643+0000","last_clean":"2024-04-04T06:16:58.854643+0000","last_became_active":"2024-04-04T06:16:57.852500+0000","last_became_peered":"2024-04-04T06:16:57.852500+0000","last_unstale":"2024-04-04T06:16:58.854643+0000","last_undegraded":"2024-04-04T06:16:58.854643+0000","last_fullsized":"2024-04-04T06:16:58.854643+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:03:47.042156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.376485+0000","last_change":"2024-04-04T06:16:56.816173+0000","last_active":"2024-04-04T06:16:59.376485+0000","last_peered":"2024-04-04T06:16:59.376485+0000","last_clean":"2024-04-04T06:16:59.376485+0000","last_became_active":"2024-04-04T06:16:56.815796+0000","last_became_peered":"2024-04-04T06:16:56.815796+0000","last_unstale":"2024-04-04T06:16:59.376485+0000","last_undegraded":"2024-04-04T06:16:59.376485+0000","last_fullsized":"2024-04-04T06:16:59.376485+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:12:54.232107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856214+0000","last_change":"2024-04-04T06:16:57.863523+0000","last_active":"2024-04-04T06:16:58.856214+0000","last_peered":"2024-04-04T06:16:58.856214+0000","last_clean":"2024-04-04T06:16:58.856214+0000","last_became_active":"2024-04-04T06:16:57.861142+0000","last_became_peered":"2024-04-04T06:16:57.861142+0000","last_unstale":"2024-04-04T06:16:58.856214+0000","last_undegraded":"2024-04-04T06:16:58.856214+0000","last_fullsized":"2024-04-04T06:16:58.856214+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:50:12.226068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222684+0000","last_change":"2024-04-04T06:16:56.817070+0000","last_active":"2024-04-04T06:16:59.222684+0000","last_peered":"2024-04-04T06:16:59.222684+0000","last_clean":"2024-04-04T06:16:59.222684+0000","last_became_active":"2024-04-04T06:16:56.816742+0000","last_became_peered":"2024-04-04T06:16:56.816742+0000","last_unstale":"2024-04-04T06:16:59.222684+0000","last_undegraded":"2024-04-04T06:16:59.222684+0000","last_fullsized":"2024-04-04T06:16:59.222684+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:10:49.284692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.866544+0000","last_change":"2024-04-04T06:16:57.865580+0000","last_active":"2024-04-04T06:16:58.866544+0000","last_peered":"2024-04-04T06:16:58.866544+0000","last_clean":"2024-04-04T06:16:58.866544+0000","last_became_active":"2024-04-04T06:16:57.863731+0000","last_became_peered":"2024-04-04T06:16:57.863731+0000","last_unstale":"2024-04-04T06:16:58.866544+0000","last_undegraded":"2024-04-04T06:16:58.866544+0000","last_fullsized":"2024-04-04T06:16:58.866544+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:05:27.280058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850433+0000","last_change":"2024-04-04T06:16:56.822568+0000","last_active":"2024-04-04T06:16:58.850433+0000","last_peered":"2024-04-04T06:16:58.850433+0000","last_clean":"2024-04-04T06:16:58.850433+0000","last_became_active":"2024-04-04T06:16:56.822237+0000","last_became_peered":"2024-04-04T06:16:56.822237+0000","last_unstale":"2024-04-04T06:16:58.850433+0000","last_undegraded":"2024-04-04T06:16:58.850433+0000","last_fullsized":"2024-04-04T06:16:58.850433+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:13:56.182899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850676+0000","last_change":"2024-04-04T06:16:57.851850+0000","last_active":"2024-04-04T06:16:58.850676+0000","last_peered":"2024-04-04T06:16:58.850676+0000","last_clean":"2024-04-04T06:16:58.850676+0000","last_became_active":"2024-04-04T06:16:57.851529+0000","last_became_peered":"2024-04-04T06:16:57.851529+0000","last_unstale":"2024-04-04T06:16:58.850676+0000","last_undegraded":"2024-04-04T06:16:58.850676+0000","last_fullsized":"2024-04-04T06:16:58.850676+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:16:58.119204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"112'3","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378418+0000","last_change":"2024-04-04T06:16:56.811481+0000","last_active":"2024-04-04T06:16:59.378418+0000","last_peered":"2024-04-04T06:16:59.378418+0000","last_clean":"2024-04-04T06:16:59.378418+0000","last_became_active":"2024-04-04T06:16:56.811190+0000","last_became_peered":"2024-04-04T06:16:56.811190+0000","last_unstale":"2024-04-04T06:16:59.378418+0000","last_undegraded":"2024-04-04T06:16:59.378418+0000","last_fullsized":"2024-04-04T06:16:59.378418+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:41:11.586175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855312+0000","last_change":"2024-04-04T06:16:57.855846+0000","last_active":"2024-04-04T06:16:58.855312+0000","last_peered":"2024-04-04T06:16:58.855312+0000","last_clean":"2024-04-04T06:16:58.855312+0000","last_became_active":"2024-04-04T06:16:57.855487+0000","last_became_peered":"2024-04-04T06:16:57.855487+0000","last_unstale":"2024-04-04T06:16:58.855312+0000","last_undegraded":"2024-04-04T06:16:58.855312+0000","last_fullsized":"2024-04-04T06:16:58.855312+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:25:12.522619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"112'6","reported_seq":20,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.379530+0000","last_change":"2024-04-04T06:16:56.816249+0000","last_active":"2024-04-04T06:16:59.379530+0000","last_peered":"2024-04-04T06:16:59.379530+0000","last_clean":"2024-04-04T06:16:59.379530+0000","last_became_active":"2024-04-04T06:16:56.815980+0000","last_became_peered":"2024-04-04T06:16:56.815980+0000","last_unstale":"2024-04-04T06:16:59.379530+0000","last_undegraded":"2024-04-04T06:16:59.379530+0000","last_fullsized":"2024-04-04T06:16:59.379530+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:20:35.260197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850475+0000","last_change":"2024-04-04T06:16:57.852377+0000","last_active":"2024-04-04T06:16:58.850475+0000","last_peered":"2024-04-04T06:16:58.850475+0000","last_clean":"2024-04-04T06:16:58.850475+0000","last_became_active":"2024-04-04T06:16:57.850593+0000","last_became_peered":"2024-04-04T06:16:57.850593+0000","last_unstale":"2024-04-04T06:16:58.850475+0000","last_undegraded":"2024-04-04T06:16:58.850475+0000","last_fullsized":"2024-04-04T06:16:58.850475+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:41:22.250390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377300+0000","last_change":"2024-04-04T06:16:56.825811+0000","last_active":"2024-04-04T06:16:59.377300+0000","last_peered":"2024-04-04T06:16:59.377300+0000","last_clean":"2024-04-04T06:16:59.377300+0000","last_became_active":"2024-04-04T06:16:56.825491+0000","last_became_peered":"2024-04-04T06:16:56.825491+0000","last_unstale":"2024-04-04T06:16:59.377300+0000","last_undegraded":"2024-04-04T06:16:59.377300+0000","last_fullsized":"2024-04-04T06:16:59.377300+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:22:21.703895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850892+0000","last_change":"2024-04-04T06:16:57.854997+0000","last_active":"2024-04-04T06:16:58.850892+0000","last_peered":"2024-04-04T06:16:58.850892+0000","last_clean":"2024-04-04T06:16:58.850892+0000","last_became_active":"2024-04-04T06:16:57.854710+0000","last_became_peered":"2024-04-04T06:16:57.854710+0000","last_unstale":"2024-04-04T06:16:58.850892+0000","last_undegraded":"2024-04-04T06:16:58.850892+0000","last_fullsized":"2024-04-04T06:16:58.850892+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:29:25.780543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.865913+0000","last_change":"2024-04-04T06:16:56.808070+0000","last_active":"2024-04-04T06:16:58.865913+0000","last_peered":"2024-04-04T06:16:58.865913+0000","last_clean":"2024-04-04T06:16:58.865913+0000","last_became_active":"2024-04-04T06:16:56.807074+0000","last_became_peered":"2024-04-04T06:16:56.807074+0000","last_unstale":"2024-04-04T06:16:58.865913+0000","last_undegraded":"2024-04-04T06:16:58.865913+0000","last_fullsized":"2024-04-04T06:16:58.865913+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:05:44.678207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850292+0000","last_change":"2024-04-04T06:16:57.840280+0000","last_active":"2024-04-04T06:16:58.850292+0000","last_peered":"2024-04-04T06:16:58.850292+0000","last_clean":"2024-04-04T06:16:58.850292+0000","last_became_active":"2024-04-04T06:16:57.839601+0000","last_became_peered":"2024-04-04T06:16:57.839601+0000","last_unstale":"2024-04-04T06:16:58.850292+0000","last_undegraded":"2024-04-04T06:16:58.850292+0000","last_fullsized":"2024-04-04T06:16:58.850292+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:41:12.600534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"112'8","reported_seq":22,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222426+0000","last_change":"2024-04-04T06:16:56.814462+0000","last_active":"2024-04-04T06:16:59.222426+0000","last_peered":"2024-04-04T06:16:59.222426+0000","last_clean":"2024-04-04T06:16:59.222426+0000","last_became_active":"2024-04-04T06:16:56.813042+0000","last_became_peered":"2024-04-04T06:16:56.813042+0000","last_unstale":"2024-04-04T06:16:59.222426+0000","last_undegraded":"2024-04-04T06:16:59.222426+0000","last_fullsized":"2024-04-04T06:16:59.222426+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:13:39.485055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222788+0000","last_change":"2024-04-04T06:16:57.852435+0000","last_active":"2024-04-04T06:16:59.222788+0000","last_peered":"2024-04-04T06:16:59.222788+0000","last_clean":"2024-04-04T06:16:59.222788+0000","last_became_active":"2024-04-04T06:16:57.851997+0000","last_became_peered":"2024-04-04T06:16:57.851997+0000","last_unstale":"2024-04-04T06:16:59.222788+0000","last_undegraded":"2024-04-04T06:16:59.222788+0000","last_fullsized":"2024-04-04T06:16:59.222788+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:36:49.667730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.379260+0000","last_change":"2024-04-04T06:16:56.812543+0000","last_active":"2024-04-04T06:16:59.379260+0000","last_peered":"2024-04-04T06:16:59.379260+0000","last_clean":"2024-04-04T06:16:59.379260+0000","last_became_active":"2024-04-04T06:16:56.812240+0000","last_became_peered":"2024-04-04T06:16:56.812240+0000","last_unstale":"2024-04-04T06:16:59.379260+0000","last_undegraded":"2024-04-04T06:16:59.379260+0000","last_fullsized":"2024-04-04T06:16:59.379260+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:25:12.848641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377607+0000","last_change":"2024-04-04T06:16:57.863716+0000","last_active":"2024-04-04T06:16:59.377607+0000","last_peered":"2024-04-04T06:16:59.377607+0000","last_clean":"2024-04-04T06:16:59.377607+0000","last_became_active":"2024-04-04T06:16:57.862780+0000","last_became_peered":"2024-04-04T06:16:57.862780+0000","last_unstale":"2024-04-04T06:16:59.377607+0000","last_undegraded":"2024-04-04T06:16:59.377607+0000","last_fullsized":"2024-04-04T06:16:59.377607+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:56:59.538463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222812+0000","last_change":"2024-04-04T06:16:56.822207+0000","last_active":"2024-04-04T06:16:59.222812+0000","last_peered":"2024-04-04T06:16:59.222812+0000","last_clean":"2024-04-04T06:16:59.222812+0000","last_became_active":"2024-04-04T06:16:56.821906+0000","last_became_peered":"2024-04-04T06:16:56.821906+0000","last_unstale":"2024-04-04T06:16:59.222812+0000","last_undegraded":"2024-04-04T06:16:59.222812+0000","last_fullsized":"2024-04-04T06:16:59.222812+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:52:52.166280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378788+0000","last_change":"2024-04-04T06:16:57.865766+0000","last_active":"2024-04-04T06:16:59.378788+0000","last_peered":"2024-04-04T06:16:59.378788+0000","last_clean":"2024-04-04T06:16:59.378788+0000","last_became_active":"2024-04-04T06:16:57.865457+0000","last_became_peered":"2024-04-04T06:16:57.865457+0000","last_unstale":"2024-04-04T06:16:59.378788+0000","last_undegraded":"2024-04-04T06:16:59.378788+0000","last_fullsized":"2024-04-04T06:16:59.378788+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:09:22.693053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.866035+0000","last_change":"2024-04-04T06:16:56.818843+0000","last_active":"2024-04-04T06:16:58.866035+0000","last_peered":"2024-04-04T06:16:58.866035+0000","last_clean":"2024-04-04T06:16:58.866035+0000","last_became_active":"2024-04-04T06:16:56.818532+0000","last_became_peered":"2024-04-04T06:16:56.818532+0000","last_unstale":"2024-04-04T06:16:58.866035+0000","last_undegraded":"2024-04-04T06:16:58.866035+0000","last_fullsized":"2024-04-04T06:16:58.866035+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:31:12.755633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.379063+0000","last_change":"2024-04-04T06:16:57.859384+0000","last_active":"2024-04-04T06:16:59.379063+0000","last_peered":"2024-04-04T06:16:59.379063+0000","last_clean":"2024-04-04T06:16:59.379063+0000","last_became_active":"2024-04-04T06:16:57.859026+0000","last_became_peered":"2024-04-04T06:16:57.859026+0000","last_unstale":"2024-04-04T06:16:59.379063+0000","last_undegraded":"2024-04-04T06:16:59.379063+0000","last_fullsized":"2024-04-04T06:16:59.379063+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:03:57.236044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854548+0000","last_change":"2024-04-04T06:16:56.817026+0000","last_active":"2024-04-04T06:16:58.854548+0000","last_peered":"2024-04-04T06:16:58.854548+0000","last_clean":"2024-04-04T06:16:58.854548+0000","last_became_active":"2024-04-04T06:16:56.816488+0000","last_became_peered":"2024-04-04T06:16:56.816488+0000","last_unstale":"2024-04-04T06:16:58.854548+0000","last_undegraded":"2024-04-04T06:16:58.854548+0000","last_fullsized":"2024-04-04T06:16:58.854548+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:11:49.042689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377911+0000","last_change":"2024-04-04T06:16:57.859829+0000","last_active":"2024-04-04T06:16:59.377911+0000","last_peered":"2024-04-04T06:16:59.377911+0000","last_clean":"2024-04-04T06:16:59.377911+0000","last_became_active":"2024-04-04T06:16:57.859535+0000","last_became_peered":"2024-04-04T06:16:57.859535+0000","last_unstale":"2024-04-04T06:16:59.377911+0000","last_undegraded":"2024-04-04T06:16:59.377911+0000","last_fullsized":"2024-04-04T06:16:59.377911+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:27:35.132513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377508+0000","last_change":"2024-04-04T06:16:56.821203+0000","last_active":"2024-04-04T06:16:59.377508+0000","last_peered":"2024-04-04T06:16:59.377508+0000","last_clean":"2024-04-04T06:16:59.377508+0000","last_became_active":"2024-04-04T06:16:56.820739+0000","last_became_peered":"2024-04-04T06:16:56.820739+0000","last_unstale":"2024-04-04T06:16:59.377508+0000","last_undegraded":"2024-04-04T06:16:59.377508+0000","last_fullsized":"2024-04-04T06:16:59.377508+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:13:39.485055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378104+0000","last_change":"2024-04-04T06:16:57.862428+0000","last_active":"2024-04-04T06:16:59.378104+0000","last_peered":"2024-04-04T06:16:59.378104+0000","last_clean":"2024-04-04T06:16:59.378104+0000","last_became_active":"2024-04-04T06:16:57.862160+0000","last_became_peered":"2024-04-04T06:16:57.862160+0000","last_unstale":"2024-04-04T06:16:59.378104+0000","last_undegraded":"2024-04-04T06:16:59.378104+0000","last_fullsized":"2024-04-04T06:16:59.378104+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:15:07.419885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222426+0000","last_change":"2024-04-04T06:16:56.816260+0000","last_active":"2024-04-04T06:16:59.222426+0000","last_peered":"2024-04-04T06:16:59.222426+0000","last_clean":"2024-04-04T06:16:59.222426+0000","last_became_active":"2024-04-04T06:16:56.815988+0000","last_became_peered":"2024-04-04T06:16:56.815988+0000","last_unstale":"2024-04-04T06:16:59.222426+0000","last_undegraded":"2024-04-04T06:16:59.222426+0000","last_fullsized":"2024-04-04T06:16:59.222426+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:56:28.483864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377989+0000","last_change":"2024-04-04T06:16:57.862761+0000","last_active":"2024-04-04T06:16:59.377989+0000","last_peered":"2024-04-04T06:16:59.377989+0000","last_clean":"2024-04-04T06:16:59.377989+0000","last_became_active":"2024-04-04T06:16:57.862519+0000","last_became_peered":"2024-04-04T06:16:57.862519+0000","last_unstale":"2024-04-04T06:16:59.377989+0000","last_undegraded":"2024-04-04T06:16:59.377989+0000","last_fullsized":"2024-04-04T06:16:59.377989+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:16:04.429520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377028+0000","last_change":"2024-04-04T06:16:56.823585+0000","last_active":"2024-04-04T06:16:59.377028+0000","last_peered":"2024-04-04T06:16:59.377028+0000","last_clean":"2024-04-04T06:16:59.377028+0000","last_became_active":"2024-04-04T06:16:56.823294+0000","last_became_peered":"2024-04-04T06:16:56.823294+0000","last_unstale":"2024-04-04T06:16:59.377028+0000","last_undegraded":"2024-04-04T06:16:59.377028+0000","last_fullsized":"2024-04-04T06:16:59.377028+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T09:02:07.681535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850448+0000","last_change":"2024-04-04T06:16:57.868618+0000","last_active":"2024-04-04T06:16:58.850448+0000","last_peered":"2024-04-04T06:16:58.850448+0000","last_clean":"2024-04-04T06:16:58.850448+0000","last_became_active":"2024-04-04T06:16:57.866435+0000","last_became_peered":"2024-04-04T06:16:57.866435+0000","last_unstale":"2024-04-04T06:16:58.850448+0000","last_undegraded":"2024-04-04T06:16:58.850448+0000","last_fullsized":"2024-04-04T06:16:58.850448+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:42:54.207457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.376665+0000","last_change":"2024-04-04T06:16:56.806783+0000","last_active":"2024-04-04T06:16:59.376665+0000","last_peered":"2024-04-04T06:16:59.376665+0000","last_clean":"2024-04-04T06:16:59.376665+0000","last_became_active":"2024-04-04T06:16:56.806293+0000","last_became_peered":"2024-04-04T06:16:56.806293+0000","last_unstale":"2024-04-04T06:16:59.376665+0000","last_undegraded":"2024-04-04T06:16:59.376665+0000","last_fullsized":"2024-04-04T06:16:59.376665+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:56:06.439167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.866304+0000","last_change":"2024-04-04T06:16:57.867317+0000","last_active":"2024-04-04T06:16:58.866304+0000","last_peered":"2024-04-04T06:16:58.866304+0000","last_clean":"2024-04-04T06:16:58.866304+0000","last_became_active":"2024-04-04T06:16:57.866053+0000","last_became_peered":"2024-04-04T06:16:57.866053+0000","last_unstale":"2024-04-04T06:16:58.866304+0000","last_undegraded":"2024-04-04T06:16:58.866304+0000","last_fullsized":"2024-04-04T06:16:58.866304+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:51:22.883892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850667+0000","last_change":"2024-04-04T06:16:56.817822+0000","last_active":"2024-04-04T06:16:58.850667+0000","last_peered":"2024-04-04T06:16:58.850667+0000","last_clean":"2024-04-04T06:16:58.850667+0000","last_became_active":"2024-04-04T06:16:56.816539+0000","last_became_peered":"2024-04-04T06:16:56.816539+0000","last_unstale":"2024-04-04T06:16:58.850667+0000","last_undegraded":"2024-04-04T06:16:58.850667+0000","last_fullsized":"2024-04-04T06:16:58.850667+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:10:06.219773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.379113+0000","last_change":"2024-04-04T06:16:57.859002+0000","last_active":"2024-04-04T06:16:59.379113+0000","last_peered":"2024-04-04T06:16:59.379113+0000","last_clean":"2024-04-04T06:16:59.379113+0000","last_became_active":"2024-04-04T06:16:57.857269+0000","last_became_peered":"2024-04-04T06:16:57.857269+0000","last_unstale":"2024-04-04T06:16:59.379113+0000","last_undegraded":"2024-04-04T06:16:59.379113+0000","last_fullsized":"2024-04-04T06:16:59.379113+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:05:24.301901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"113'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.866526+0000","last_change":"2024-04-04T06:16:56.819463+0000","last_active":"2024-04-04T06:16:58.866526+0000","last_peered":"2024-04-04T06:16:58.866526+0000","last_clean":"2024-04-04T06:16:58.866526+0000","last_became_active":"2024-04-04T06:16:56.816034+0000","last_became_peered":"2024-04-04T06:16:56.816034+0000","last_unstale":"2024-04-04T06:16:58.866526+0000","last_undegraded":"2024-04-04T06:16:58.866526+0000","last_fullsized":"2024-04-04T06:16:58.866526+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T18:14:56.497336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.379427+0000","last_change":"2024-04-04T06:16:57.856907+0000","last_active":"2024-04-04T06:16:59.379427+0000","last_peered":"2024-04-04T06:16:59.379427+0000","last_clean":"2024-04-04T06:16:59.379427+0000","last_became_active":"2024-04-04T06:16:57.856616+0000","last_became_peered":"2024-04-04T06:16:57.856616+0000","last_unstale":"2024-04-04T06:16:59.379427+0000","last_undegraded":"2024-04-04T06:16:59.379427+0000","last_fullsized":"2024-04-04T06:16:59.379427+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T16:34:43.213737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378572+0000","last_change":"2024-04-04T06:16:56.803667+0000","last_active":"2024-04-04T06:16:59.378572+0000","last_peered":"2024-04-04T06:16:59.378572+0000","last_clean":"2024-04-04T06:16:59.378572+0000","last_became_active":"2024-04-04T06:16:56.803353+0000","last_became_peered":"2024-04-04T06:16:56.803353+0000","last_unstale":"2024-04-04T06:16:59.378572+0000","last_undegraded":"2024-04-04T06:16:59.378572+0000","last_fullsized":"2024-04-04T06:16:59.378572+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:42:35.512614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.866190+0000","last_change":"2024-04-04T06:16:57.870240+0000","last_active":"2024-04-04T06:16:58.866190+0000","last_peered":"2024-04-04T06:16:58.866190+0000","last_clean":"2024-04-04T06:16:58.866190+0000","last_became_active":"2024-04-04T06:16:57.869874+0000","last_became_peered":"2024-04-04T06:16:57.869874+0000","last_unstale":"2024-04-04T06:16:58.866190+0000","last_undegraded":"2024-04-04T06:16:58.866190+0000","last_fullsized":"2024-04-04T06:16:58.866190+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:01:10.335447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850959+0000","last_change":"2024-04-04T06:16:56.820108+0000","last_active":"2024-04-04T06:16:58.850959+0000","last_peered":"2024-04-04T06:16:58.850959+0000","last_clean":"2024-04-04T06:16:58.850959+0000","last_became_active":"2024-04-04T06:16:56.819768+0000","last_became_peered":"2024-04-04T06:16:56.819768+0000","last_unstale":"2024-04-04T06:16:58.850959+0000","last_undegraded":"2024-04-04T06:16:58.850959+0000","last_fullsized":"2024-04-04T06:16:58.850959+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T17:19:35.089037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855977+0000","last_change":"2024-04-04T06:16:57.844825+0000","last_active":"2024-04-04T06:16:58.855977+0000","last_peered":"2024-04-04T06:16:58.855977+0000","last_clean":"2024-04-04T06:16:58.855977+0000","last_became_active":"2024-04-04T06:16:57.843101+0000","last_became_peered":"2024-04-04T06:16:57.843101+0000","last_unstale":"2024-04-04T06:16:58.855977+0000","last_undegraded":"2024-04-04T06:16:58.855977+0000","last_fullsized":"2024-04-04T06:16:58.855977+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T08:41:12.600534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854882+0000","last_change":"2024-04-04T06:16:56.823376+0000","last_active":"2024-04-04T06:16:58.854882+0000","last_peered":"2024-04-04T06:16:58.854882+0000","last_clean":"2024-04-04T06:16:58.854882+0000","last_became_active":"2024-04-04T06:16:56.822325+0000","last_became_peered":"2024-04-04T06:16:56.822325+0000","last_unstale":"2024-04-04T06:16:58.854882+0000","last_undegraded":"2024-04-04T06:16:58.854882+0000","last_fullsized":"2024-04-04T06:16:58.854882+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T06:17:31.397079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.856868+0000","last_change":"2024-04-04T06:16:57.861595+0000","last_active":"2024-04-04T06:16:58.856868+0000","last_peered":"2024-04-04T06:16:58.856868+0000","last_clean":"2024-04-04T06:16:58.856868+0000","last_became_active":"2024-04-04T06:16:57.860975+0000","last_became_peered":"2024-04-04T06:16:57.860975+0000","last_unstale":"2024-04-04T06:16:58.856868+0000","last_undegraded":"2024-04-04T06:16:58.856868+0000","last_fullsized":"2024-04-04T06:16:58.856868+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:29:44.470791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.855757+0000","last_change":"2024-04-04T06:16:56.822556+0000","last_active":"2024-04-04T06:16:58.855757+0000","last_peered":"2024-04-04T06:16:58.855757+0000","last_clean":"2024-04-04T06:16:58.855757+0000","last_became_active":"2024-04-04T06:16:56.821979+0000","last_became_peered":"2024-04-04T06:16:56.821979+0000","last_unstale":"2024-04-04T06:16:58.855757+0000","last_undegraded":"2024-04-04T06:16:58.855757+0000","last_fullsized":"2024-04-04T06:16:58.855757+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T13:59:42.200129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378485+0000","last_change":"2024-04-04T06:16:57.861874+0000","last_active":"2024-04-04T06:16:59.378485+0000","last_peered":"2024-04-04T06:16:59.378485+0000","last_clean":"2024-04-04T06:16:59.378485+0000","last_became_active":"2024-04-04T06:16:57.861605+0000","last_became_peered":"2024-04-04T06:16:57.861605+0000","last_unstale":"2024-04-04T06:16:59.378485+0000","last_undegraded":"2024-04-04T06:16:59.378485+0000","last_fullsized":"2024-04-04T06:16:59.378485+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:01:10.335447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.222503+0000","last_change":"2024-04-04T06:16:56.818500+0000","last_active":"2024-04-04T06:16:59.222503+0000","last_peered":"2024-04-04T06:16:59.222503+0000","last_clean":"2024-04-04T06:16:59.222503+0000","last_became_active":"2024-04-04T06:16:56.816829+0000","last_became_peered":"2024-04-04T06:16:56.816829+0000","last_unstale":"2024-04-04T06:16:59.222503+0000","last_undegraded":"2024-04-04T06:16:59.222503+0000","last_fullsized":"2024-04-04T06:16:59.222503+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T10:37:13.781466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377554+0000","last_change":"2024-04-04T06:16:57.862871+0000","last_active":"2024-04-04T06:16:59.377554+0000","last_peered":"2024-04-04T06:16:59.377554+0000","last_clean":"2024-04-04T06:16:59.377554+0000","last_became_active":"2024-04-04T06:16:57.857674+0000","last_became_peered":"2024-04-04T06:16:57.857674+0000","last_unstale":"2024-04-04T06:16:59.377554+0000","last_undegraded":"2024-04-04T06:16:59.377554+0000","last_fullsized":"2024-04-04T06:16:59.377554+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T11:12:53.084370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.377182+0000","last_change":"2024-04-04T06:16:56.807000+0000","last_active":"2024-04-04T06:16:59.377182+0000","last_peered":"2024-04-04T06:16:59.377182+0000","last_clean":"2024-04-04T06:16:59.377182+0000","last_became_active":"2024-04-04T06:16:56.806687+0000","last_became_peered":"2024-04-04T06:16:56.806687+0000","last_unstale":"2024-04-04T06:16:59.377182+0000","last_undegraded":"2024-04-04T06:16:59.377182+0000","last_fullsized":"2024-04-04T06:16:59.377182+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:02:40.213409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.850291+0000","last_change":"2024-04-04T06:16:57.856787+0000","last_active":"2024-04-04T06:16:58.850291+0000","last_peered":"2024-04-04T06:16:58.850291+0000","last_clean":"2024-04-04T06:16:58.850291+0000","last_became_active":"2024-04-04T06:16:57.856455+0000","last_became_peered":"2024-04-04T06:16:57.856455+0000","last_unstale":"2024-04-04T06:16:58.850291+0000","last_undegraded":"2024-04-04T06:16:58.850291+0000","last_fullsized":"2024-04-04T06:16:58.850291+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:56.778822+0000","last_clean_scrub_stamp":"2024-04-04T06:16:56.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T15:42:36.526974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:58.854824+0000","last_change":"2024-04-04T06:16:56.819104+0000","last_active":"2024-04-04T06:16:58.854824+0000","last_peered":"2024-04-04T06:16:58.854824+0000","last_clean":"2024-04-04T06:16:58.854824+0000","last_became_active":"2024-04-04T06:16:56.818492+0000","last_became_peered":"2024-04-04T06:16:56.818492+0000","last_unstale":"2024-04-04T06:16:58.854824+0000","last_undegraded":"2024-04-04T06:16:58.854824+0000","last_fullsized":"2024-04-04T06:16:58.854824+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:16:55.764462+0000","last_clean_scrub_stamp":"2024-04-04T06:16:55.764462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T14:14:15.299577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":338,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-04T06:16:59.378631+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:16:59.378631+0000","last_peered":"2024-04-04T06:16:59.378631+0000","last_clean":"2024-04-04T06:16:59.378631+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:16:59.378631+0000","last_undegraded":"2024-04-04T06:16:59.378631+0000","last_fullsized":"2024-04-04T06:16:59.378631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509579,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37296,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334544,"statfs":{"total":96636764160,"available":96598573056,"internally_reserved":0,"allocated":1998848,"data_stored":1198052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509581,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36712,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335128,"statfs":{"total":96636764160,"available":96599171072,"internally_reserved":0,"allocated":1400832,"data_stored":607138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509582,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1413120,"data_stored":608750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509581,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":1396736,"data_stored":607068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509580,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1404928,"data_stored":607192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509582,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2011136,"data_stored":1199160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542285,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36712,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335128,"statfs":{"total":96636764160,"available":96599171072,"internally_reserved":0,"allocated":1404928,"data_stored":607672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542283,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36712,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335128,"statfs":{"total":96636764160,"available":96599171072,"internally_reserved":0,"allocated":1396736,"data_stored":607068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:17:07.902 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-04T06:17:07.903 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-04T06:17:08.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.169+0000 7fa13a400640 1 Processor -- start 2024-04-04T06:17:08.163 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.170+0000 7fa13a400640 1 -- start start 2024-04-04T06:17:08.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.171+0000 7fa13a400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1340b4db0 0x7fa1340cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:08.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.171+0000 7fa13a400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1340b5ce0 con 0x7fa134161f60 2024-04-04T06:17:08.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.171+0000 7fa13a400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1340d27f0 con 0x7fa1340d1580 2024-04-04T06:17:08.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.171+0000 7fa13a400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa1340d2a90 con 0x7fa1340b4db0 2024-04-04T06:17:08.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.171+0000 7fa133400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fa134161f60 0x7fa1340d0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41430/0 (socket says 172.21.15.159:41430) 2024-04-04T06:17:08.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.171+0000 7fa133400640 1 -- 172.21.15.159:0/4093120301 learned_addr learned my addr 172.21.15.159:0/4093120301 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:08.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.172+0000 7fa133e00640 1 --2- 172.21.15.159:0/4093120301 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1340b4db0 0x7fa1340cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:08.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.172+0000 7fa132a00640 1 -- 172.21.15.159:0/4093120301 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1924823644 0 0) 0x7fa124001110 con 0x7fa134161f60 2024-04-04T06:17:08.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.172+0000 7fa132a00640 1 -- 172.21.15.159:0/4093120301 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa118003520 con 0x7fa134161f60 2024-04-04T06:17:08.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.172+0000 7fa132a00640 1 -- 172.21.15.159:0/4093120301 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2494579632 0 0) 0x7fa118003520 con 0x7fa134161f60 2024-04-04T06:17:08.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.172+0000 7fa132a00640 1 -- 172.21.15.159:0/4093120301 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1340b4db0 msgr2=0x7fa1340cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:08.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.172+0000 7fa132a00640 1 --2- 172.21.15.159:0/4093120301 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1340b4db0 0x7fa1340cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:08.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.172+0000 7fa132a00640 1 -- 172.21.15.159:0/4093120301 >> v1:172.21.15.178:6789/0 conn(0x7fa1340d1580 legacy=0x7fa1340cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:08.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.172+0000 7fa132a00640 1 -- 172.21.15.159:0/4093120301 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa134162340 con 0x7fa134161f60 2024-04-04T06:17:08.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.172+0000 7fa132a00640 1 -- 172.21.15.159:0/4093120301 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fa124001840 con 0x7fa134161f60 2024-04-04T06:17:08.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.172+0000 7fa132a00640 1 -- 172.21.15.159:0/4093120301 >> v1:172.21.15.159:6789/0 conn(0x7fa134161f60 legacy=0x7fa1340d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:08.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.173+0000 7fa132a00640 1 --2- 172.21.15.159:0/4093120301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1180039e0 0x7fa118003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:08.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.173+0000 7fa132a00640 1 --2- 172.21.15.159:0/4093120301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa118004800 0x7fa118004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:08.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.173+0000 7fa132a00640 1 --2- 172.21.15.159:0/4093120301 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1340b4db0 0x7fa1180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:08.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.173+0000 7fa132a00640 1 -- 172.21.15.159:0/4093120301 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa1340d2a90 con 0x7fa1180039e0 2024-04-04T06:17:08.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.173+0000 7fa132a00640 1 -- 172.21.15.159:0/4093120301 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa11800ba70 con 0x7fa118004800 2024-04-04T06:17:08.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.173+0000 7fa132a00640 1 -- 172.21.15.159:0/4093120301 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa11800bc70 con 0x7fa1340b4db0 2024-04-04T06:17:08.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.173+0000 7fa133400640 1 --2- 172.21.15.159:0/4093120301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1180039e0 0x7fa118003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:08.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.173+0000 7fa12be00640 1 --2- 172.21.15.159:0/4093120301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa118004800 0x7fa118004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:08.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.173+0000 7fa133400640 1 -- 172.21.15.159:0/4093120301 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1340b4db0 msgr2=0x7fa1180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:08.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.173+0000 7fa133400640 1 --2- 172.21.15.159:0/4093120301 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1340b4db0 0x7fa1180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:08.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.173+0000 7fa133400640 1 -- 172.21.15.159:0/4093120301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa118004800 msgr2=0x7fa118004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:08.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.173+0000 7fa133400640 1 --2- 172.21.15.159:0/4093120301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa118004800 0x7fa118004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:08.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.173+0000 7fa133400640 1 -- 172.21.15.159:0/4093120301 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa118003520 con 0x7fa1180039e0 2024-04-04T06:17:08.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.175+0000 7fa133400640 1 --2- 172.21.15.159:0/4093120301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1180039e0 0x7fa118003e00 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fa124001050 tx=0x7fa118005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831f4ba36ac29828 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:08.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.175+0000 7fa132a00640 1 -- 172.21.15.159:0/4093120301 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12400c0e0 con 0x7fa1180039e0 2024-04-04T06:17:08.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.175+0000 7fa132a00640 1 -- 172.21.15.159:0/4093120301 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa124030070 con 0x7fa1180039e0 2024-04-04T06:17:08.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.175+0000 7fa132a00640 1 -- 172.21.15.159:0/4093120301 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1240058d0 con 0x7fa1180039e0 2024-04-04T06:17:08.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.176+0000 7fa13a400640 1 -- 172.21.15.159:0/4093120301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1180039e0 msgr2=0x7fa118003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:08.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.176+0000 7fa13a400640 1 --2- 172.21.15.159:0/4093120301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1180039e0 0x7fa118003e00 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fa124001050 tx=0x7fa118005830 comp rx=0 tx=0).stop 2024-04-04T06:17:08.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.176+0000 7fa133e00640 1 -- 172.21.15.159:0/4093120301 reap_dead start 2024-04-04T06:17:08.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.176+0000 7fa13a400640 1 -- 172.21.15.159:0/4093120301 shutdown_connections 2024-04-04T06:17:08.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.176+0000 7fa13a400640 1 -- 172.21.15.159:0/4093120301 >> 172.21.15.159:0/4093120301 conn(0x7fa13408bfb0 msgr2=0x7fa1340d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:08.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.176+0000 7fa13a400640 1 -- 172.21.15.159:0/4093120301 shutdown_connections 2024-04-04T06:17:08.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.176+0000 7fa13a400640 1 -- 172.21.15.159:0/4093120301 wait complete. 2024-04-04T06:17:08.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.178+0000 7fa13a400640 1 Processor -- start 2024-04-04T06:17:08.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.178+0000 7fa13a400640 1 -- start start 2024-04-04T06:17:08.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.179+0000 7fa13a400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1180039e0 0x7fa1340cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:08.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.179+0000 7fa13a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1340b7a90 0x7fa1340a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:08.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.179+0000 7fa13a400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1340b7f10 0x7fa1340a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:08.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.179+0000 7fa13a400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa1340d3310 con 0x7fa1340b7a90 2024-04-04T06:17:08.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.179+0000 7fa13a400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa1340d3030 con 0x7fa1340b7f10 2024-04-04T06:17:08.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.179+0000 7fa13a400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa1340b3170 con 0x7fa1180039e0 2024-04-04T06:17:08.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.179+0000 7fa12be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1340b7a90 0x7fa1340a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:08.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.179+0000 7fa12be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1340b7a90 0x7fa1340a13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59226/0 (socket says 172.21.15.159:59226) 2024-04-04T06:17:08.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.179+0000 7fa12be00640 1 -- 172.21.15.159:0/3764265080 learned_addr learned my addr 172.21.15.159:0/3764265080 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:08.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.180+0000 7fa133e00640 1 --2- 172.21.15.159:0/3764265080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1340b7f10 0x7fa1340a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:08.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.180+0000 7fa133400640 1 --2- 172.21.15.159:0/3764265080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1180039e0 0x7fa1340cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:08.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.180+0000 7fa12be00640 1 -- 172.21.15.159:0/3764265080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1180039e0 msgr2=0x7fa1340cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:08.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.180+0000 7fa12be00640 1 --2- 172.21.15.159:0/3764265080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1180039e0 0x7fa1340cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:08.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.180+0000 7fa12be00640 1 -- 172.21.15.159:0/3764265080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1340b7f10 msgr2=0x7fa1340a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:08.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.180+0000 7fa12be00640 1 --2- 172.21.15.159:0/3764265080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1340b7f10 0x7fa1340a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:08.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.180+0000 7fa12be00640 1 -- 172.21.15.159:0/3764265080 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13415e840 con 0x7fa1340b7a90 2024-04-04T06:17:08.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.180+0000 7fa133400640 1 --2- 172.21.15.159:0/3764265080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1180039e0 0x7fa1340cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:08.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.180+0000 7fa133e00640 1 --2- 172.21.15.159:0/3764265080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1340b7f10 0x7fa1340a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:08.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.180+0000 7fa12be00640 1 --2- 172.21.15.159:0/3764265080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1340b7a90 0x7fa1340a13c0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00e500 tx=0x7fa11c00e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:08.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.180+0000 7fa12b400640 1 -- 172.21.15.159:0/3764265080 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c0052e0 con 0x7fa1340b7a90 2024-04-04T06:17:08.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.181+0000 7fa12b400640 1 -- 172.21.15.159:0/3764265080 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa11c018070 con 0x7fa1340b7a90 2024-04-04T06:17:08.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.181+0000 7fa13a400640 1 -- 172.21.15.159:0/3764265080 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1340c9ca0 con 0x7fa1340b7a90 2024-04-04T06:17:08.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.181+0000 7fa12b400640 1 -- 172.21.15.159:0/3764265080 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c003810 con 0x7fa1340b7a90 2024-04-04T06:17:08.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.181+0000 7fa13a400640 1 -- 172.21.15.159:0/3764265080 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1340ca260 con 0x7fa1340b7a90 2024-04-04T06:17:08.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.182+0000 7fa13a400640 1 -- 172.21.15.159:0/3764265080 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1340ba330 con 0x7fa1340b7a90 2024-04-04T06:17:08.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.183+0000 7fa12b400640 1 -- 172.21.15.159:0/3764265080 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa11c0039b0 con 0x7fa1340b7a90 2024-04-04T06:17:08.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.184+0000 7fa12b400640 1 --2- 172.21.15.159:0/3764265080 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa11006c8f0 0x7fa11006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:08.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.184+0000 7fa12b400640 1 -- 172.21.15.159:0/3764265080 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7fa11c08b920 con 0x7fa1340b7a90 2024-04-04T06:17:08.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.184+0000 7fa133400640 1 --2- 172.21.15.159:0/3764265080 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa11006c8f0 0x7fa11006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:08.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.185+0000 7fa133400640 1 --2- 172.21.15.159:0/3764265080 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa11006c8f0 0x7fa11006edb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa1180057e0 tx=0x7fa124007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:08.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.189+0000 7fa12b400640 1 -- 172.21.15.159:0/3764265080 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa1340ba330 con 0x7fa1340b7a90 2024-04-04T06:17:08.485 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.492+0000 7fa13a400640 1 -- 172.21.15.159:0/3764265080 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa1340685c0 con 0x7fa1340b7a90 2024-04-04T06:17:08.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.495+0000 7fa12b400640 1 -- 172.21.15.159:0/3764265080 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7fa11c059cc0 con 0x7fa1340b7a90 2024-04-04T06:17:08.489 INFO:teuthology.orchestra.run.smithi159.stdout:[client.0] 2024-04-04T06:17:08.489 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQD9Qg5mqNKJGxAA4jP9cadhRIyhQ3lssttWIw== 2024-04-04T06:17:08.489 INFO:teuthology.orchestra.run.smithi159.stdout: caps mds = "allow" 2024-04-04T06:17:08.489 INFO:teuthology.orchestra.run.smithi159.stdout: caps mon = "allow r" 2024-04-04T06:17:08.490 INFO:teuthology.orchestra.run.smithi159.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-04T06:17:08.490 INFO:teuthology.orchestra.run.smithi159.stderr:updated caps for client.0 2024-04-04T06:17:08.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.498+0000 7fa13a400640 1 -- 172.21.15.159:0/3764265080 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa11006c8f0 msgr2=0x7fa11006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:08.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.498+0000 7fa13a400640 1 --2- 172.21.15.159:0/3764265080 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa11006c8f0 0x7fa11006edb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa1180057e0 tx=0x7fa124007000 comp rx=0 tx=0).stop 2024-04-04T06:17:08.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.498+0000 7fa13a400640 1 -- 172.21.15.159:0/3764265080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1340b7a90 msgr2=0x7fa1340a13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:08.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.498+0000 7fa13a400640 1 --2- 172.21.15.159:0/3764265080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1340b7a90 0x7fa1340a13c0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00e500 tx=0x7fa11c00e9c0 comp rx=0 tx=0).stop 2024-04-04T06:17:08.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.498+0000 7fa13a400640 1 -- 172.21.15.159:0/3764265080 shutdown_connections 2024-04-04T06:17:08.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.498+0000 7fa13a400640 1 --2- 172.21.15.159:0/3764265080 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa11006c8f0 0x7fa11006edb0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:08.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.498+0000 7fa13a400640 1 --2- 172.21.15.159:0/3764265080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1340b7f10 0x7fa1340a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:08.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.498+0000 7fa13a400640 1 --2- 172.21.15.159:0/3764265080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1340b7a90 0x7fa1340a13c0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:08.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.498+0000 7fa13a400640 1 --2- 172.21.15.159:0/3764265080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1180039e0 0x7fa1340cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:08.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.498+0000 7fa13a400640 1 -- 172.21.15.159:0/3764265080 >> 172.21.15.159:0/3764265080 conn(0x7fa13408bfb0 msgr2=0x7fa13406cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:08.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.498+0000 7fa13a400640 1 -- 172.21.15.159:0/3764265080 shutdown_connections 2024-04-04T06:17:08.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.498+0000 7fa13a400640 1 -- 172.21.15.159:0/3764265080 wait complete. 2024-04-04T06:17:08.512 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:17:08.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.729+0000 7f36a3200640 1 Processor -- start 2024-04-04T06:17:08.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.729+0000 7f36a3200640 1 -- start start 2024-04-04T06:17:08.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.730+0000 7f36a3200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f369c0cf700 0x7f369c168fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:08.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.730+0000 7f36a3200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f369c15e880 con 0x7f369c15e4a0 2024-04-04T06:17:08.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.730+0000 7f36a3200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f369c0d28d0 con 0x7f369c0d2310 2024-04-04T06:17:08.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.730+0000 7f36a3200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f369c0d34e0 con 0x7f369c0cf700 2024-04-04T06:17:08.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.730+0000 7f36a1e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f369c15e4a0 0x7f369c0b9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41438/0 (socket says 172.21.15.159:41438) 2024-04-04T06:17:08.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.731+0000 7f36a1e00640 1 -- 172.21.15.159:0/3161303028 learned_addr learned my addr 172.21.15.159:0/3161303028 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:08.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.731+0000 7f36a0a00640 1 -- 172.21.15.159:0/3161303028 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1666770203 0 0) 0x7f369c15e880 con 0x7f369c15e4a0 2024-04-04T06:17:08.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.731+0000 7f36a2800640 1 --2- 172.21.15.159:0/3161303028 >> v2:172.21.15.178:3300/0 conn(0x7f369c0cf700 0x7f369c168fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:08.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.731+0000 7f36a0a00640 1 -- 172.21.15.159:0/3161303028 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3684003520 con 0x7f369c15e4a0 2024-04-04T06:17:08.727 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.731+0000 7f36a0a00640 1 -- 172.21.15.159:0/3161303028 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1875057702 0 0) 0x7f3684003520 con 0x7f369c15e4a0 2024-04-04T06:17:08.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.731+0000 7f36a0a00640 1 -- 172.21.15.159:0/3161303028 >> v2:172.21.15.178:3300/0 conn(0x7f369c0cf700 msgr2=0x7f369c168fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:08.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.731+0000 7f36a0a00640 1 --2- 172.21.15.159:0/3161303028 >> v2:172.21.15.178:3300/0 conn(0x7f369c0cf700 0x7f369c168fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:08.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.731+0000 7f36a0a00640 1 -- 172.21.15.159:0/3161303028 >> v1:172.21.15.178:6789/0 conn(0x7f369c0d2310 legacy=0x7f369c0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:08.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.731+0000 7f36a0a00640 1 -- 172.21.15.159:0/3161303028 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369c0d2bb0 con 0x7f369c15e4a0 2024-04-04T06:17:08.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.731+0000 7f36a0a00640 1 -- 172.21.15.159:0/3161303028 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f3690001690 con 0x7f369c15e4a0 2024-04-04T06:17:08.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.731+0000 7f36a0a00640 1 -- 172.21.15.159:0/3161303028 >> v1:172.21.15.159:6789/0 conn(0x7f369c15e4a0 legacy=0x7f369c0b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:08.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.731+0000 7f36a0a00640 1 --2- 172.21.15.159:0/3161303028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36840039b0 0x7f3684003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:08.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.732+0000 7f36a0a00640 1 --2- 172.21.15.159:0/3161303028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3684004800 0x7f3684004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:08.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.732+0000 7f36a0a00640 1 --2- 172.21.15.159:0/3161303028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36840051c0 0x7f368400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:08.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.732+0000 7f36a0a00640 1 -- 172.21.15.159:0/3161303028 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f369c0d34e0 con 0x7f3684004800 2024-04-04T06:17:08.728 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.732+0000 7f36a0a00640 1 -- 172.21.15.159:0/3161303028 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f368400c0f0 con 0x7f36840051c0 2024-04-04T06:17:08.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.732+0000 7f36a0a00640 1 -- 172.21.15.159:0/3161303028 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f368400c2f0 con 0x7f36840039b0 2024-04-04T06:17:08.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.732+0000 7f36a1400640 1 --2- 172.21.15.159:0/3161303028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3684004800 0x7f3684004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:08.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.732+0000 7f36a2800640 1 --2- 172.21.15.159:0/3161303028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36840051c0 0x7f368400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:08.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.732+0000 7f36a1e00640 1 --2- 172.21.15.159:0/3161303028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36840039b0 0x7f3684003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:08.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.732+0000 7f36a1400640 1 -- 172.21.15.159:0/3161303028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36840039b0 msgr2=0x7f3684003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:08.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.732+0000 7f36a1400640 1 --2- 172.21.15.159:0/3161303028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36840039b0 0x7f3684003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:08.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.732+0000 7f36a1400640 1 -- 172.21.15.159:0/3161303028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36840051c0 msgr2=0x7f368400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:08.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.732+0000 7f36a1400640 1 --2- 172.21.15.159:0/3161303028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36840051c0 0x7f368400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:08.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.732+0000 7f36a1400640 1 -- 172.21.15.159:0/3161303028 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3684003520 con 0x7f3684004800 2024-04-04T06:17:08.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.732+0000 7f36a2800640 1 -- 172.21.15.159:0/3161303028 reap_dead start 2024-04-04T06:17:08.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.734+0000 7f36a1400640 1 --2- 172.21.15.159:0/3161303028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3684004800 0x7f3684004c80 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f368c003db0 tx=0x7f3684005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3096efbc484f7721 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:08.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.734+0000 7f36a0a00640 1 -- 172.21.15.159:0/3161303028 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c037070 con 0x7f3684004800 2024-04-04T06:17:08.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.734+0000 7f36a0a00640 1 -- 172.21.15.159:0/3161303028 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f368c02be50 con 0x7f3684004800 2024-04-04T06:17:08.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.734+0000 7f36a0a00640 1 -- 172.21.15.159:0/3161303028 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c003980 con 0x7f3684004800 2024-04-04T06:17:08.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.735+0000 7f36a3200640 1 -- 172.21.15.159:0/3161303028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3684004800 msgr2=0x7f3684004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:08.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.735+0000 7f36a3200640 1 --2- 172.21.15.159:0/3161303028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3684004800 0x7f3684004c80 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f368c003db0 tx=0x7f3684005830 comp rx=0 tx=0).stop 2024-04-04T06:17:08.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.735+0000 7f36a3200640 1 -- 172.21.15.159:0/3161303028 shutdown_connections 2024-04-04T06:17:08.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.735+0000 7f36a3200640 1 --2- 172.21.15.159:0/3161303028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3684004800 0x7f3684004c80 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:08.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.735+0000 7f36a3200640 1 -- 172.21.15.159:0/3161303028 >> 172.21.15.159:0/3161303028 conn(0x7f369c0676d0 msgr2=0x7f369c067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:08.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.735+0000 7f36a3200640 1 -- 172.21.15.159:0/3161303028 shutdown_connections 2024-04-04T06:17:08.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.735+0000 7f36a3200640 1 -- 172.21.15.159:0/3161303028 wait complete. 2024-04-04T06:17:08.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.737+0000 7f36a3200640 1 Processor -- start 2024-04-04T06:17:08.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.737+0000 7f36a3200640 1 -- start start 2024-04-04T06:17:08.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.738+0000 7f36a3200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3684004800 0x7f369c1f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:08.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.738+0000 7f36a3200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f369c1583b0 0x7f369c14fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:08.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.738+0000 7f36a3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f369c150680 0x7f369c1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:08.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.738+0000 7f36a3200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f369c0d3970 con 0x7f369c150680 2024-04-04T06:17:08.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.738+0000 7f36a3200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f369c0d3170 con 0x7f3684004800 2024-04-04T06:17:08.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.738+0000 7f36a3200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f369c0ceff0 con 0x7f369c1583b0 2024-04-04T06:17:08.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.739+0000 7f36a2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f369c150680 0x7f369c1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:08.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.739+0000 7f36a2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f369c150680 0x7f369c1599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59232/0 (socket says 172.21.15.159:59232) 2024-04-04T06:17:08.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.739+0000 7f36a2800640 1 -- 172.21.15.159:0/965507874 learned_addr learned my addr 172.21.15.159:0/965507874 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:08.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.739+0000 7f36a1e00640 1 --2- 172.21.15.159:0/965507874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3684004800 0x7f369c1f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:08.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.739+0000 7f36a1400640 1 --2- 172.21.15.159:0/965507874 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f369c1583b0 0x7f369c14fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:08.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.739+0000 7f36a2800640 1 -- 172.21.15.159:0/965507874 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f369c1583b0 msgr2=0x7f369c14fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:08.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.739+0000 7f36a2800640 1 --2- 172.21.15.159:0/965507874 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f369c1583b0 0x7f369c14fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:08.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.739+0000 7f36a2800640 1 -- 172.21.15.159:0/965507874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3684004800 msgr2=0x7f369c1f7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:08.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.739+0000 7f36a2800640 1 --2- 172.21.15.159:0/965507874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3684004800 0x7f369c1f7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:08.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.739+0000 7f36a2800640 1 -- 172.21.15.159:0/965507874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369c1f82d0 con 0x7f369c150680 2024-04-04T06:17:08.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.740+0000 7f36a2800640 1 --2- 172.21.15.159:0/965507874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f369c150680 0x7f369c1599b0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f369c0b6e40 tx=0x7f36980115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:08.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.740+0000 7f3696000640 1 -- 172.21.15.159:0/965507874 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698011e10 con 0x7f369c150680 2024-04-04T06:17:08.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.740+0000 7f3696000640 1 -- 172.21.15.159:0/965507874 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f369801b070 con 0x7f369c150680 2024-04-04T06:17:08.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.740+0000 7f3696000640 1 -- 172.21.15.159:0/965507874 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698003990 con 0x7f369c150680 2024-04-04T06:17:08.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.740+0000 7f36a3200640 1 -- 172.21.15.159:0/965507874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f369c1601f0 con 0x7f369c150680 2024-04-04T06:17:08.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.740+0000 7f36a3200640 1 -- 172.21.15.159:0/965507874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f369c158c30 con 0x7f369c150680 2024-04-04T06:17:08.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.742+0000 7f36a3200640 1 -- 172.21.15.159:0/965507874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f369c1603c0 con 0x7f369c150680 2024-04-04T06:17:08.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.742+0000 7f3696000640 1 -- 172.21.15.159:0/965507874 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3698005d10 con 0x7f369c150680 2024-04-04T06:17:08.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.743+0000 7f3696000640 1 --2- 172.21.15.159:0/965507874 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f368006c920 0x7f368006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:08.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.744+0000 7f3696000640 1 -- 172.21.15.159:0/965507874 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f369808c100 con 0x7f369c150680 2024-04-04T06:17:08.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.744+0000 7f36a1e00640 1 --2- 172.21.15.159:0/965507874 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f368006c920 0x7f368006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:08.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.745+0000 7f36a1e00640 1 --2- 172.21.15.159:0/965507874 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f368006c920 0x7f368006ede0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3690001eb0 tx=0x7f3690009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:08.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.747+0000 7f3696000640 1 -- 172.21.15.159:0/965507874 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f369c1f82d0 con 0x7f369c150680 2024-04-04T06:17:08.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.988+0000 7f36a3200640 1 -- 172.21.15.159:0/965507874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f369c075940 con 0x7f369c150680 2024-04-04T06:17:08.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.989+0000 7f3696000640 1 -- 172.21.15.159:0/965507874 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5197 (secure 0 0 0) 0x7f3698014350 con 0x7f369c150680 2024-04-04T06:17:08.983 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:08.983 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":101,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5732,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1498324773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1498324773},{"type":"v1","addr":"172.21.15.178:6837","nonce":1498324773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2455031662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":2455031662},{"type":"v1","addr":"172.21.15.159:6835","nonce":2455031662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6343,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1823228151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1823228151},{"type":"v1","addr":"172.21.15.178:6836","nonce":1823228151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:16:57.277344+0000","modified":"2024-04-04T06:17:00.359830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8853},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8853":{"gid":8853,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6834/1017894858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1017894858},{"type":"v1","addr":"172.21.15.159:6834","nonce":1017894858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8853,"qdb_cluster":[8853]},"id":8}]} 2024-04-04T06:17:08.984 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 101 2024-04-04T06:17:08.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.992+0000 7f36a3200640 1 -- 172.21.15.159:0/965507874 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f368006c920 msgr2=0x7f368006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:08.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.992+0000 7f36a3200640 1 --2- 172.21.15.159:0/965507874 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f368006c920 0x7f368006ede0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3690001eb0 tx=0x7f3690009040 comp rx=0 tx=0).stop 2024-04-04T06:17:08.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.992+0000 7f36a3200640 1 -- 172.21.15.159:0/965507874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f369c150680 msgr2=0x7f369c1599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:08.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.992+0000 7f36a3200640 1 --2- 172.21.15.159:0/965507874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f369c150680 0x7f369c1599b0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f369c0b6e40 tx=0x7f36980115a0 comp rx=0 tx=0).stop 2024-04-04T06:17:08.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.992+0000 7f36a3200640 1 -- 172.21.15.159:0/965507874 shutdown_connections 2024-04-04T06:17:08.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.992+0000 7f36a3200640 1 --2- 172.21.15.159:0/965507874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f369c150680 0x7f369c1599b0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:08.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.992+0000 7f36a3200640 1 --2- 172.21.15.159:0/965507874 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f369c1583b0 0x7f369c14fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:08.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.992+0000 7f36a3200640 1 --2- 172.21.15.159:0/965507874 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f368006c920 0x7f368006ede0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:08.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.992+0000 7f36a3200640 1 --2- 172.21.15.159:0/965507874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3684004800 0x7f369c1f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:08.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.992+0000 7f36a3200640 1 -- 172.21.15.159:0/965507874 >> 172.21.15.159:0/965507874 conn(0x7f369c0676d0 msgr2=0x7f369c15c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:08.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.992+0000 7f36a3200640 1 -- 172.21.15.159:0/965507874 shutdown_connections 2024-04-04T06:17:08.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:08.993+0000 7f36a3200640 1 -- 172.21.15.159:0/965507874 wait complete. 2024-04-04T06:17:09.409 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 101, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:16:57.277344+0000', 'modified': '2024-04-04T06:17:00.359830+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8853}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8853': {'gid': 8853, 'name': 'a', 'rank': 0, 'incarnation': 97, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.159:6834/1017894858', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 1017894858}, {'type': 'v1', 'addr': '172.21.15.159:6834', 'nonce': 1017894858}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8853, 'qdb_cluster': [8853]} 2024-04-04T06:17:09.409 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-04T06:17:09.409 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-04T06:17:09.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.645+0000 7f41dcc00640 1 Processor -- start 2024-04-04T06:17:09.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.646+0000 7f41dcc00640 1 -- start start 2024-04-04T06:17:09.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.647+0000 7f41dcc00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f41d815e4a0 0x7f41d80b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.647+0000 7f41dcc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f41d80cf700 0x7f41d80b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.647+0000 7f41dcc00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41d815e880 con 0x7f41d80d2310 2024-04-04T06:17:09.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.647+0000 7f41dcc00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f41d80d34e0 con 0x7f41d80cf700 2024-04-04T06:17:09.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.647+0000 7f41dcc00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f41d80d28d0 con 0x7f41d815e4a0 2024-04-04T06:17:09.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.647+0000 7f41d7e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f41d80cf700 0x7f41d80b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.648+0000 7f41d7e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f41d80cf700 0x7f41d80b73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59242/0 (socket says 172.21.15.159:59242) 2024-04-04T06:17:09.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.648+0000 7f41d7e00640 1 -- 172.21.15.159:0/2526340575 learned_addr learned my addr 172.21.15.159:0/2526340575 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:09.642 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.648+0000 7f41d7400640 1 --2- 172.21.15.159:0/2526340575 >> v2:172.21.15.178:3300/0 conn(0x7f41d815e4a0 0x7f41d80b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.648+0000 7f41d7e00640 1 -- 172.21.15.159:0/2526340575 >> v2:172.21.15.178:3300/0 conn(0x7f41d815e4a0 msgr2=0x7f41d80b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:09.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.648+0000 7f41d7e00640 1 --2- 172.21.15.159:0/2526340575 >> v2:172.21.15.178:3300/0 conn(0x7f41d815e4a0 0x7f41d80b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.648+0000 7f41d7e00640 1 -- 172.21.15.159:0/2526340575 >> v1:172.21.15.159:6789/0 conn(0x7f41d80d2310 legacy=0x7f41d80ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:09.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.648+0000 7f41d7e00640 1 -- 172.21.15.159:0/2526340575 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d80d2bb0 con 0x7f41d80cf700 2024-04-04T06:17:09.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.648+0000 7f41d7400640 1 --2- 172.21.15.159:0/2526340575 >> v2:172.21.15.178:3300/0 conn(0x7f41d815e4a0 0x7f41d80b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:09.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.649+0000 7f41d7e00640 1 --2- 172.21.15.159:0/2526340575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41d80cf700 0x7f41d80b73e0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f41d80cfb20 tx=0x7f41d002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0054bf09766c6c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:09.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.649+0000 7f41d6000640 1 -- 172.21.15.159:0/2526340575 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d002ca80 con 0x7f41d80cf700 2024-04-04T06:17:09.644 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.649+0000 7f41d6000640 1 -- 172.21.15.159:0/2526340575 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d002cc20 con 0x7f41d80cf700 2024-04-04T06:17:09.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.649+0000 7f41d6000640 1 -- 172.21.15.159:0/2526340575 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d00343f0 con 0x7f41d80cf700 2024-04-04T06:17:09.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.650+0000 7f41dcc00640 1 -- 172.21.15.159:0/2526340575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41d80cf700 msgr2=0x7f41d80b73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:09.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.650+0000 7f41dcc00640 1 --2- 172.21.15.159:0/2526340575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41d80cf700 0x7f41d80b73e0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f41d80cfb20 tx=0x7f41d002bee0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.650+0000 7f41dcc00640 1 -- 172.21.15.159:0/2526340575 shutdown_connections 2024-04-04T06:17:09.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.650+0000 7f41dcc00640 1 --2- 172.21.15.159:0/2526340575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41d80cf700 0x7f41d80b73e0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.650+0000 7f41dcc00640 1 --2- 172.21.15.159:0/2526340575 >> v2:172.21.15.178:3300/0 conn(0x7f41d815e4a0 0x7f41d80b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.650+0000 7f41dcc00640 1 -- 172.21.15.159:0/2526340575 >> 172.21.15.159:0/2526340575 conn(0x7f41d80676d0 msgr2=0x7f41d8067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:09.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.650+0000 7f41dcc00640 1 -- 172.21.15.159:0/2526340575 shutdown_connections 2024-04-04T06:17:09.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.650+0000 7f41dcc00640 1 -- 172.21.15.159:0/2526340575 wait complete. 2024-04-04T06:17:09.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.652+0000 7f41dcc00640 1 Processor -- start 2024-04-04T06:17:09.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.652+0000 7f41dcc00640 1 -- start start 2024-04-04T06:17:09.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.653+0000 7f41dcc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f41d80cf700 0x7f41d81535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.653+0000 7f41dcc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41d80d2310 0x7f41d8151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.653+0000 7f41dcc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41d815e4a0 0x7f41d8152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.653+0000 7f41dcc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f41d80d3970 con 0x7f41d80d2310 2024-04-04T06:17:09.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.653+0000 7f41dcc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f41d80d3170 con 0x7f41d815e4a0 2024-04-04T06:17:09.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.653+0000 7f41dcc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f41d80ba1a0 con 0x7f41d80cf700 2024-04-04T06:17:09.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.653+0000 7f41d6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41d80d2310 0x7f41d8151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.653+0000 7f41d6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41d80d2310 0x7f41d8151ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59252/0 (socket says 172.21.15.159:59252) 2024-04-04T06:17:09.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.653+0000 7f41d6a00640 1 -- 172.21.15.159:0/2806158529 learned_addr learned my addr 172.21.15.159:0/2806158529 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:09.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.654+0000 7f41d7400640 1 --2- 172.21.15.159:0/2806158529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f41d80cf700 0x7f41d81535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.654+0000 7f41d7e00640 1 --2- 172.21.15.159:0/2806158529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41d815e4a0 0x7f41d8152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.654+0000 7f41d6a00640 1 -- 172.21.15.159:0/2806158529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f41d80cf700 msgr2=0x7f41d81535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:09.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.654+0000 7f41d6a00640 1 --2- 172.21.15.159:0/2806158529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f41d80cf700 0x7f41d81535c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.654+0000 7f41d6a00640 1 -- 172.21.15.159:0/2806158529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41d815e4a0 msgr2=0x7f41d8152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:09.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.654+0000 7f41d6a00640 1 --2- 172.21.15.159:0/2806158529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41d815e4a0 0x7f41d8152410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.654+0000 7f41d6a00640 1 -- 172.21.15.159:0/2806158529 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d81f40c0 con 0x7f41d80d2310 2024-04-04T06:17:09.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.654+0000 7f41d6a00640 1 --2- 172.21.15.159:0/2806158529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41d80d2310 0x7f41d8151ed0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f41c400c460 tx=0x7f41c400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:09.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.654+0000 7f41cf400640 1 -- 172.21.15.159:0/2806158529 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c40052d0 con 0x7f41d80d2310 2024-04-04T06:17:09.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.654+0000 7f41cf400640 1 -- 172.21.15.159:0/2806158529 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c4012630 con 0x7f41d80d2310 2024-04-04T06:17:09.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.654+0000 7f41cf400640 1 -- 172.21.15.159:0/2806158529 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c40054b0 con 0x7f41d80d2310 2024-04-04T06:17:09.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.654+0000 7f41dcc00640 1 -- 172.21.15.159:0/2806158529 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d8152a40 con 0x7f41d80d2310 2024-04-04T06:17:09.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.654+0000 7f41dcc00640 1 -- 172.21.15.159:0/2806158529 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d8153000 con 0x7f41d80d2310 2024-04-04T06:17:09.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.657+0000 7f41cf400640 1 -- 172.21.15.159:0/2806158529 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41c4012a20 con 0x7f41d80d2310 2024-04-04T06:17:09.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.658+0000 7f41cf400640 1 --2- 172.21.15.159:0/2806158529 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f41b806c8f0 0x7f41b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.658+0000 7f41cf400640 1 -- 172.21.15.159:0/2806158529 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f41c408b7b0 con 0x7f41d80d2310 2024-04-04T06:17:09.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.659+0000 7f41e0fad740 1 Processor -- start 2024-04-04T06:17:09.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.659+0000 7f41d7400640 1 --2- 172.21.15.159:0/2806158529 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f41b806c8f0 0x7f41b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.659+0000 7f41e0fad740 1 -- start start 2024-04-04T06:17:09.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.660+0000 7f41e0fad740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1decc2300 0x55c1dec952a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.660+0000 7f41e0fad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc26e0 0x55c1decc5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.660+0000 7f41e0fad740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c1dec957e0 0x55c1decc55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.660+0000 7f41e0fad740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c1dec5cab0 con 0x55c1decc26e0 2024-04-04T06:17:09.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.660+0000 7f41e0fad740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c1debfefa0 con 0x55c1decc2300 2024-04-04T06:17:09.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.660+0000 7f41e0fad740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c1deba7560 con 0x55c1dec957e0 2024-04-04T06:17:09.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.660+0000 7f41d6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc26e0 0x55c1decc5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.660+0000 7f41d6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc26e0 0x55c1decc5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59266/0 (socket says 172.21.15.159:59266) 2024-04-04T06:17:09.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.660+0000 7f41d6a00640 1 -- 172.21.15.159:0/4016121577 learned_addr learned my addr 172.21.15.159:0/4016121577 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:09.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.660+0000 7f41d7400640 1 --2- 172.21.15.159:0/2806158529 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f41b806c8f0 0x7f41b806edb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f41c8008850 tx=0x7f41c8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:09.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.660+0000 7f41d7e00640 1 --2- 172.21.15.159:0/4016121577 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c1dec957e0 0x55c1decc55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.660+0000 7f41d7400640 1 --2- 172.21.15.159:0/4016121577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1decc2300 0x55c1dec952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.660+0000 7f41d6a00640 1 -- 172.21.15.159:0/4016121577 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c1dec957e0 msgr2=0x55c1decc55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:09.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.660+0000 7f41d6a00640 1 --2- 172.21.15.159:0/4016121577 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c1dec957e0 0x55c1decc55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.660+0000 7f41d6a00640 1 -- 172.21.15.159:0/4016121577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1decc2300 msgr2=0x55c1dec952a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:09.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.660+0000 7f41d6a00640 1 --2- 172.21.15.159:0/4016121577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1decc2300 0x55c1dec952a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.660+0000 7f41d6a00640 1 -- 172.21.15.159:0/4016121577 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1de9132d0 con 0x55c1decc26e0 2024-04-04T06:17:09.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.661+0000 7f41d6a00640 1 --2- 172.21.15.159:0/4016121577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc26e0 0x55c1decc5090 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f41c4000ff0 tx=0x7f41c4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d540a1afb771c8a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:09.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.661+0000 7f41c3400640 1 -- 172.21.15.159:0/4016121577 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4062020 con 0x55c1decc26e0 2024-04-04T06:17:09.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.661+0000 7f41c3400640 1 -- 172.21.15.159:0/4016121577 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c406de50 con 0x55c1decc26e0 2024-04-04T06:17:09.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.661+0000 7f41c3400640 1 -- 172.21.15.159:0/4016121577 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4012d10 con 0x55c1decc26e0 2024-04-04T06:17:09.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.661+0000 7f41e0fad740 1 -- 172.21.15.159:0/4016121577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc26e0 msgr2=0x55c1decc5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:09.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.661+0000 7f41e0fad740 1 --2- 172.21.15.159:0/4016121577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc26e0 0x55c1decc5090 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f41c4000ff0 tx=0x7f41c4012000 comp rx=0 tx=0).stop 2024-04-04T06:17:09.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.661+0000 7f41e0fad740 1 -- 172.21.15.159:0/4016121577 shutdown_connections 2024-04-04T06:17:09.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.661+0000 7f41e0fad740 1 --2- 172.21.15.159:0/4016121577 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c1dec957e0 0x55c1decc55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.657 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.661+0000 7f41e0fad740 1 --2- 172.21.15.159:0/4016121577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc26e0 0x55c1decc5090 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.661+0000 7f41e0fad740 1 --2- 172.21.15.159:0/4016121577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1decc2300 0x55c1dec952a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.661+0000 7f41e0fad740 1 -- 172.21.15.159:0/4016121577 >> 172.21.15.159:0/4016121577 conn(0x55c1dec97510 msgr2=0x55c1dece0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:09.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.662+0000 7f41e0fad740 1 -- 172.21.15.159:0/4016121577 shutdown_connections 2024-04-04T06:17:09.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.662+0000 7f41e0fad740 1 -- 172.21.15.159:0/4016121577 wait complete. 2024-04-04T06:17:09.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.662+0000 7f41e0fad740 1 -- start start 2024-04-04T06:17:09.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.662+0000 7f41e0fad740 1 Processor -- start 2024-04-04T06:17:09.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.663+0000 7f41e0fad740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c1decc2300 0x55c1decdb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.663+0000 7f41e0fad740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1decc26e0 0x55c1decc7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.663+0000 7f41e0fad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc7760 0x55c1decf0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.663+0000 7f41e0fad740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c1debba920 con 0x55c1decc7760 2024-04-04T06:17:09.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.663+0000 7f41e0fad740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c1debfaf50 con 0x55c1decc26e0 2024-04-04T06:17:09.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.663+0000 7f41e0fad740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c1dec99ab0 con 0x55c1decc2300 2024-04-04T06:17:09.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.663+0000 7f41e0fad740 10 client.0 _ll_register_callbacks cb 0x55c1debba310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:17:09.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.664+0000 7f41d7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc7760 0x55c1decf0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.664+0000 7f41d7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc7760 0x55c1decf0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59270/0 (socket says 172.21.15.159:59270) 2024-04-04T06:17:09.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.664+0000 7f41d7e00640 1 -- 172.21.15.159:0/158537544 learned_addr learned my addr 172.21.15.159:0/158537544 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:09.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.664+0000 7f41d7400640 1 --2- 172.21.15.159:0/158537544 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c1decc2300 0x55c1decdb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.664+0000 7f41d6a00640 1 --2- 172.21.15.159:0/158537544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1decc26e0 0x55c1decc7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.664+0000 7f41d7e00640 1 -- 172.21.15.159:0/158537544 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c1decc2300 msgr2=0x55c1decdb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:09.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.664+0000 7f41d7e00640 1 --2- 172.21.15.159:0/158537544 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c1decc2300 0x55c1decdb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.664+0000 7f41d7e00640 1 -- 172.21.15.159:0/158537544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1decc26e0 msgr2=0x55c1decc7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:09.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.664+0000 7f41d7e00640 1 --2- 172.21.15.159:0/158537544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1decc26e0 0x55c1decc7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.664+0000 7f41d7e00640 1 -- 172.21.15.159:0/158537544 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1de9dd6b0 con 0x55c1decc7760 2024-04-04T06:17:09.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.665+0000 7f41d7e00640 1 --2- 172.21.15.159:0/158537544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc7760 0x55c1decf0380 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f41d0036040 tx=0x7f41d0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331721bc1af3ac0c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:09.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.665+0000 7f41b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:17:09.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.665+0000 7f41b7e00640 1 -- 172.21.15.159:0/158537544 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0044070 con 0x55c1decc7760 2024-04-04T06:17:09.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.665+0000 7f41b7e00640 1 -- 172.21.15.159:0/158537544 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f41d0034b30 con 0x55c1decc7760 2024-04-04T06:17:09.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.665+0000 7f41e0fad740 1 -- 172.21.15.159:0/158537544 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41c406de50 con 0x55c1decc7760 2024-04-04T06:17:09.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.665+0000 7f41b7e00640 1 -- 172.21.15.159:0/158537544 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7f41d004f050 con 0x55c1decc7760 2024-04-04T06:17:09.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.665+0000 7f41e0fad740 10 client.9024 fetch_fsmap learned FSMap version 101 2024-04-04T06:17:09.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.665+0000 7f41e0fad740 1 -- 172.21.15.159:0/158537544 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) -- 0x55c1decf2ec0 con 0x55c1decc7760 2024-04-04T06:17:09.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.666+0000 7f41b7e00640 1 -- 172.21.15.159:0/158537544 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c406de50 con 0x55c1decc7760 2024-04-04T06:17:09.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.666+0000 7f41b7e00640 1 -- 172.21.15.159:0/158537544 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 101) ==== 3494+0+0 (secure 0 0 0) 0x7f41d0052020 con 0x55c1decc7760 2024-04-04T06:17:09.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.666+0000 7f41e0fad740 10 client.9024 fetch_fsmap finished waiting for FSMap version 101 2024-04-04T06:17:09.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.666+0000 7f41e0fad740 10 client.9024 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:17:09.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.666+0000 7f41e0fad740 20 client.9024 populate_metadata read hostname 'smithi159' 2024-04-04T06:17:09.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.666+0000 7f41e0fad740 1 --2- 172.21.15.159:0/158537544 >> [v2:172.21.15.159:6832/1017894858,v1:172.21.15.159:6834/1017894858] conn(0x55c1decf9ed0 0x55c1decfe130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.666+0000 7f41e0fad740 4 client.9024 mds_command: new command op to 8853 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:17:09.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.666+0000 7f41e0fad740 1 -- 172.21.15.159:0/158537544 --> [v2:172.21.15.159:6832/1017894858,v1:172.21.15.159:6834/1017894858] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1dec0c4e0 con 0x55c1decf9ed0 2024-04-04T06:17:09.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.666+0000 7f41d7400640 1 --2- 172.21.15.159:0/158537544 >> [v2:172.21.15.159:6832/1017894858,v1:172.21.15.159:6834/1017894858] conn(0x55c1decf9ed0 0x55c1decfe130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.667+0000 7f41d7400640 1 --2- 172.21.15.159:0/158537544 >> [v2:172.21.15.159:6832/1017894858,v1:172.21.15.159:6834/1017894858] conn(0x55c1decf9ed0 0x55c1decfe130 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8d528b961787316 server_cookie=9af8a8e30a4f651f in_seq=0 out_seq=0 2024-04-04T06:17:09.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.667+0000 7f41b7e00640 10 client.9024 ms_handle_connect on v2:172.21.15.159:6832/1017894858 2024-04-04T06:17:09.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.667+0000 7f41b7e00640 1 -- 172.21.15.159:0/158537544 <== mds.0 v2:172.21.15.159:6832/1017894858 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c1dec0c4e0 con 0x55c1decf9ed0 2024-04-04T06:17:09.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.668+0000 7f41b7e00640 10 client.9024 handle_command_reply: tid=0 2024-04-04T06:17:09.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.668+0000 7f41e0fad740 1 client.9024 shutdown 2024-04-04T06:17:09.666 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.668+0000 7f41e0fad740 1 -- 172.21.15.159:0/158537544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc7760 msgr2=0x55c1decf0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:09.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.668+0000 7f41e0fad740 1 --2- 172.21.15.159:0/158537544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc7760 0x55c1decf0380 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f41d0036040 tx=0x7f41d0004ab0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.668+0000 7f41e0fad740 1 -- 172.21.15.159:0/158537544 shutdown_connections 2024-04-04T06:17:09.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.668+0000 7f41e0fad740 1 --2- 172.21.15.159:0/158537544 >> [v2:172.21.15.159:6832/1017894858,v1:172.21.15.159:6834/1017894858] conn(0x55c1decf9ed0 0x55c1decfe130 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.668+0000 7f41e0fad740 1 --2- 172.21.15.159:0/158537544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc7760 0x55c1decf0380 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.668+0000 7f41e0fad740 1 --2- 172.21.15.159:0/158537544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1decc26e0 0x55c1decc7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.668+0000 7f41b7e00640 0 client.9024 ms_handle_reset on v2:172.21.15.159:6832/1017894858 2024-04-04T06:17:09.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.668+0000 7f41e0fad740 1 --2- 172.21.15.159:0/158537544 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c1decc2300 0x55c1decdb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.668+0000 7f41e0fad740 1 -- 172.21.15.159:0/158537544 >> 172.21.15.159:0/158537544 conn(0x55c1dec957e0 msgr2=0x55c1decd9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:09.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.668+0000 7f41e0fad740 1 -- 172.21.15.159:0/158537544 shutdown_connections 2024-04-04T06:17:09.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.668+0000 7f41e0fad740 1 -- 172.21.15.159:0/158537544 wait complete. 2024-04-04T06:17:09.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.668+0000 7f41e0fad740 20 client.9024 trim_cache size 0 max 16384 2024-04-04T06:17:09.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.687+0000 7f41e0fad740 1 Processor -- start 2024-04-04T06:17:09.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.687+0000 7f41e0fad740 1 -- start start 2024-04-04T06:17:09.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.688+0000 7f41e0fad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc2300 0x55c1dece12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.688+0000 7f41e0fad740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c1decc26e0 0x55c1decc0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.688+0000 7f41e0fad740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1decc7760 0x55c1decc4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.688+0000 7f41e0fad740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f41d81ff8c0 con 0x55c1decc2300 2024-04-04T06:17:09.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.688+0000 7f41e0fad740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f41d81f6120 con 0x55c1decc7760 2024-04-04T06:17:09.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.688+0000 7f41e0fad740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f41d81f6770 con 0x55c1decc26e0 2024-04-04T06:17:09.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.688+0000 7f41d7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc2300 0x55c1dece12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.688+0000 7f41d7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc2300 0x55c1dece12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59280/0 (socket says 172.21.15.159:59280) 2024-04-04T06:17:09.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.688+0000 7f41d7400640 1 -- 172.21.15.159:0/2558211523 learned_addr learned my addr 172.21.15.159:0/2558211523 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:09.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.689+0000 7f41d7400640 1 -- 172.21.15.159:0/2558211523 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c1decc26e0 msgr2=0x55c1decc0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:09.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.689+0000 7f41d7400640 1 --2- 172.21.15.159:0/2558211523 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c1decc26e0 0x55c1decc0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.689+0000 7f41d7400640 1 -- 172.21.15.159:0/2558211523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1decc7760 msgr2=0x55c1decc4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:09.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.689+0000 7f41d7400640 1 --2- 172.21.15.159:0/2558211523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1decc7760 0x55c1decc4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.689+0000 7f41d7400640 1 -- 172.21.15.159:0/2558211523 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1dec01440 con 0x55c1decc2300 2024-04-04T06:17:09.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.689+0000 7f41d7400640 1 --2- 172.21.15.159:0/2558211523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc2300 0x55c1dece12e0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f41c800d590 tx=0x7f41c8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6af941127888a24 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:09.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.689+0000 7f41c2a00640 1 -- 172.21.15.159:0/2558211523 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c802c070 con 0x55c1decc2300 2024-04-04T06:17:09.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.689+0000 7f41c2a00640 1 -- 172.21.15.159:0/2558211523 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c800ddc0 con 0x55c1decc2300 2024-04-04T06:17:09.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.689+0000 7f41c2a00640 1 -- 172.21.15.159:0/2558211523 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8012a40 con 0x55c1decc2300 2024-04-04T06:17:09.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.690+0000 7f41e0fad740 1 -- 172.21.15.159:0/2558211523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc2300 msgr2=0x55c1dece12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:09.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.690+0000 7f41e0fad740 1 --2- 172.21.15.159:0/2558211523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc2300 0x55c1dece12e0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f41c800d590 tx=0x7f41c8013000 comp rx=0 tx=0).stop 2024-04-04T06:17:09.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.690+0000 7f41e0fad740 1 -- 172.21.15.159:0/2558211523 shutdown_connections 2024-04-04T06:17:09.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.690+0000 7f41e0fad740 1 --2- 172.21.15.159:0/2558211523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1decc7760 0x55c1decc4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.690+0000 7f41e0fad740 1 --2- 172.21.15.159:0/2558211523 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c1decc26e0 0x55c1decc0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.690+0000 7f41e0fad740 1 --2- 172.21.15.159:0/2558211523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc2300 0x55c1dece12e0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.690+0000 7f41e0fad740 1 -- 172.21.15.159:0/2558211523 >> 172.21.15.159:0/2558211523 conn(0x55c1dec957e0 msgr2=0x55c1dece2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:09.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.690+0000 7f41e0fad740 1 -- 172.21.15.159:0/2558211523 shutdown_connections 2024-04-04T06:17:09.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.690+0000 7f41e0fad740 1 -- 172.21.15.159:0/2558211523 wait complete. 2024-04-04T06:17:09.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.691+0000 7f41e0fad740 1 -- start start 2024-04-04T06:17:09.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.691+0000 7f41e0fad740 1 Processor -- start 2024-04-04T06:17:09.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.692+0000 7f41e0fad740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1decc2300 0x55c1decda6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.692+0000 7f41e0fad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc26e0 0x55c1decdac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.692+0000 7f41e0fad740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c1decc7760 0x55c1ded52ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.692+0000 7f41e0fad740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c1decf9d50 con 0x55c1decc26e0 2024-04-04T06:17:09.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.692+0000 7f41e0fad740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c1decf99a0 con 0x55c1decc2300 2024-04-04T06:17:09.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.692+0000 7f41e0fad740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55c1decfa2e0 con 0x55c1decc7760 2024-04-04T06:17:09.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.692+0000 7f41e0fad740 10 client.0 _ll_register_callbacks cb 0x7f41d002ccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:17:09.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.692+0000 7f41d6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc26e0 0x55c1decdac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.692+0000 7f41d6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc26e0 0x55c1decdac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59286/0 (socket says 172.21.15.159:59286) 2024-04-04T06:17:09.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.692+0000 7f41d6a00640 1 -- 172.21.15.159:0/3844477627 learned_addr learned my addr 172.21.15.159:0/3844477627 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:09.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.692+0000 7f41d6a00640 1 -- 172.21.15.159:0/3844477627 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c1decc7760 msgr2=0x55c1ded52ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:09.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.692+0000 7f41d7400640 1 --2- 172.21.15.159:0/3844477627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1decc2300 0x55c1decda6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.691 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.692+0000 7f41d7e00640 1 --2- 172.21.15.159:0/3844477627 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c1decc7760 0x55c1ded52ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.692+0000 7f41d6a00640 1 --2- 172.21.15.159:0/3844477627 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c1decc7760 0x55c1ded52ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.692+0000 7f41d6a00640 1 -- 172.21.15.159:0/3844477627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1decc2300 msgr2=0x55c1decda6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:09.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.692+0000 7f41d6a00640 1 --2- 172.21.15.159:0/3844477627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1decc2300 0x55c1decda6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.692+0000 7f41d6a00640 1 -- 172.21.15.159:0/3844477627 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1dec9b860 con 0x55c1decc26e0 2024-04-04T06:17:09.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.693+0000 7f41d6a00640 1 --2- 172.21.15.159:0/3844477627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc26e0 0x55c1decdac00 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f41c40558a0 tx=0x7f41c4089b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5444c1ed8bea56 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:09.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.693+0000 7f41b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:17:09.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.693+0000 7f41b7e00640 1 -- 172.21.15.159:0/3844477627 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c406ee60 con 0x55c1decc26e0 2024-04-04T06:17:09.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.693+0000 7f41b7e00640 1 -- 172.21.15.159:0/3844477627 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f41c4055ab0 con 0x55c1decc26e0 2024-04-04T06:17:09.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.693+0000 7f41e0fad740 1 -- 172.21.15.159:0/3844477627 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41c800ddc0 con 0x55c1decc26e0 2024-04-04T06:17:09.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.693+0000 7f41b7e00640 1 -- 172.21.15.159:0/3844477627 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7f41c4062020 con 0x55c1decc26e0 2024-04-04T06:17:09.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.693+0000 7f41e0fad740 10 client.9030 fetch_fsmap learned FSMap version 101 2024-04-04T06:17:09.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.693+0000 7f41e0fad740 1 -- 172.21.15.159:0/3844477627 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) -- 0x55c1ded53d30 con 0x55c1decc26e0 2024-04-04T06:17:09.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.693+0000 7f41b7e00640 1 -- 172.21.15.159:0/3844477627 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c800ddc0 con 0x55c1decc26e0 2024-04-04T06:17:09.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.694+0000 7f41b7e00640 1 -- 172.21.15.159:0/3844477627 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 101) ==== 3494+0+0 (secure 0 0 0) 0x7f41c4058890 con 0x55c1decc26e0 2024-04-04T06:17:09.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.694+0000 7f41e0fad740 10 client.9030 fetch_fsmap finished waiting for FSMap version 101 2024-04-04T06:17:09.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.694+0000 7f41e0fad740 10 client.9030 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:17:09.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.694+0000 7f41e0fad740 20 client.9030 populate_metadata read hostname 'smithi159' 2024-04-04T06:17:09.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.694+0000 7f41e0fad740 1 --2- 172.21.15.159:0/3844477627 >> [v2:172.21.15.159:6832/1017894858,v1:172.21.15.159:6834/1017894858] conn(0x55c1ded53560 0x55c1decc5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.694+0000 7f41e0fad740 4 client.9030 mds_command: new command op to 8853 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-04T06:17:09.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.694+0000 7f41e0fad740 1 -- 172.21.15.159:0/3844477627 --> [v2:172.21.15.159:6832/1017894858,v1:172.21.15.159:6834/1017894858] -- command(tid 0: {"prefix": "status"}) -- 0x55c1dec06830 con 0x55c1ded53560 2024-04-04T06:17:09.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.694+0000 7f41d7400640 1 --2- 172.21.15.159:0/3844477627 >> [v2:172.21.15.159:6832/1017894858,v1:172.21.15.159:6834/1017894858] conn(0x55c1ded53560 0x55c1decc5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.694+0000 7f41b7e00640 10 client.9030 ms_handle_connect on v2:172.21.15.159:6832/1017894858 2024-04-04T06:17:09.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.694+0000 7f41d7400640 1 --2- 172.21.15.159:0/3844477627 >> [v2:172.21.15.159:6832/1017894858,v1:172.21.15.159:6834/1017894858] conn(0x55c1ded53560 0x55c1decc5e70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7764d3111b52c8a server_cookie=d02068d03f042219 in_seq=0 out_seq=0 2024-04-04T06:17:09.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.695+0000 7f41b7e00640 1 -- 172.21.15.159:0/3844477627 <== mds.0 v2:172.21.15.159:6832/1017894858 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55c1dec06830 con 0x55c1ded53560 2024-04-04T06:17:09.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.695+0000 7f41b7e00640 10 client.9030 handle_command_reply: tid=0 2024-04-04T06:17:09.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.695+0000 7f41e0fad740 1 client.9030 shutdown 2024-04-04T06:17:09.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.695+0000 7f41e0fad740 1 -- 172.21.15.159:0/3844477627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc26e0 msgr2=0x55c1decdac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:09.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.695+0000 7f41e0fad740 1 --2- 172.21.15.159:0/3844477627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc26e0 0x55c1decdac00 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f41c40558a0 tx=0x7f41c4089b10 comp rx=0 tx=0).stop 2024-04-04T06:17:09.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.695+0000 7f41e0fad740 1 -- 172.21.15.159:0/3844477627 shutdown_connections 2024-04-04T06:17:09.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.695+0000 7f41e0fad740 1 --2- 172.21.15.159:0/3844477627 >> [v2:172.21.15.159:6832/1017894858,v1:172.21.15.159:6834/1017894858] conn(0x55c1ded53560 0x55c1decc5e70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.695+0000 7f41e0fad740 1 --2- 172.21.15.159:0/3844477627 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55c1decc7760 0x55c1ded52ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.695+0000 7f41e0fad740 1 --2- 172.21.15.159:0/3844477627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1decc26e0 0x55c1decdac00 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.695+0000 7f41e0fad740 1 --2- 172.21.15.159:0/3844477627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c1decc2300 0x55c1decda6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.695+0000 7f41e0fad740 1 -- 172.21.15.159:0/3844477627 >> 172.21.15.159:0/3844477627 conn(0x55c1dec957e0 msgr2=0x55c1dece2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:09.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.695+0000 7f41b7e00640 0 client.9030 ms_handle_reset on v2:172.21.15.159:6832/1017894858 2024-04-04T06:17:09.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.695+0000 7f41e0fad740 1 -- 172.21.15.159:0/3844477627 shutdown_connections 2024-04-04T06:17:09.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.695+0000 7f41e0fad740 1 -- 172.21.15.159:0/3844477627 wait complete. 2024-04-04T06:17:09.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.695+0000 7f41e0fad740 20 client.9030 trim_cache size 0 max 16384 2024-04-04T06:17:09.695 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:17:09.695 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_fsid": "30311045-f714-48fa-832b-4cb2ba8bf0bf", 2024-04-04T06:17:09.696 INFO:teuthology.orchestra.run.smithi159.stdout: "whoami": 0, 2024-04-04T06:17:09.696 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 8853, 2024-04-04T06:17:09.696 INFO:teuthology.orchestra.run.smithi159.stdout: "want_state": "up:active", 2024-04-04T06:17:09.696 INFO:teuthology.orchestra.run.smithi159.stdout: "state": "up:active", 2024-04-04T06:17:09.696 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs", 2024-04-04T06:17:09.696 INFO:teuthology.orchestra.run.smithi159.stdout: "rank_uptime": 11.850932911999999, 2024-04-04T06:17:09.696 INFO:teuthology.orchestra.run.smithi159.stdout: "mdsmap_epoch": 101, 2024-04-04T06:17:09.696 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch": 113, 2024-04-04T06:17:09.696 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch_barrier": 113, 2024-04-04T06:17:09.696 INFO:teuthology.orchestra.run.smithi159.stdout: "uptime": 17.281803785000001 2024-04-04T06:17:09.696 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:17:09.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.696+0000 7f41dcc00640 1 -- 172.21.15.159:0/2806158529 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f41b806c8f0 msgr2=0x7f41b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:09.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.696+0000 7f41dcc00640 1 --2- 172.21.15.159:0/2806158529 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f41b806c8f0 0x7f41b806edb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f41c8008850 tx=0x7f41c8008040 comp rx=0 tx=0).stop 2024-04-04T06:17:09.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.696+0000 7f41dcc00640 1 -- 172.21.15.159:0/2806158529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41d80d2310 msgr2=0x7f41d8151ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:09.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.696+0000 7f41dcc00640 1 --2- 172.21.15.159:0/2806158529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41d80d2310 0x7f41d8151ed0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f41c400c460 tx=0x7f41c400c920 comp rx=0 tx=0).stop 2024-04-04T06:17:09.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.696+0000 7f41dcc00640 1 -- 172.21.15.159:0/2806158529 shutdown_connections 2024-04-04T06:17:09.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.696+0000 7f41dcc00640 1 --2- 172.21.15.159:0/2806158529 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f41b806c8f0 0x7f41b806edb0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.696+0000 7f41dcc00640 1 --2- 172.21.15.159:0/2806158529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41d815e4a0 0x7f41d8152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.696+0000 7f41dcc00640 1 --2- 172.21.15.159:0/2806158529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41d80d2310 0x7f41d8151ed0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.696+0000 7f41dcc00640 1 --2- 172.21.15.159:0/2806158529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f41d80cf700 0x7f41d81535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.696+0000 7f41dcc00640 1 -- 172.21.15.159:0/2806158529 >> 172.21.15.159:0/2806158529 conn(0x7f41d80676d0 msgr2=0x7f41d80d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:09.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.696+0000 7f41dcc00640 1 -- 172.21.15.159:0/2806158529 shutdown_connections 2024-04-04T06:17:09.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.696+0000 7f41dcc00640 1 -- 172.21.15.159:0/2806158529 wait complete. 2024-04-04T06:17:09.706 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-04T06:17:09.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.900+0000 7f7bfde00640 1 Processor -- start 2024-04-04T06:17:09.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.900+0000 7f7bfde00640 1 -- start start 2024-04-04T06:17:09.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.902+0000 7f7bfde00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7bf8161f60 0x7f7bf80d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.902+0000 7f7bfde00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bf80b3170 0x7f7bf80d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.902+0000 7f7bfde00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bf80b5ce0 con 0x7f7bf80b3da0 2024-04-04T06:17:09.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.902+0000 7f7bfde00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f7bf80d27f0 con 0x7f7bf8161f60 2024-04-04T06:17:09.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.902+0000 7f7bfde00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7bf80d2a90 con 0x7f7bf80b3170 2024-04-04T06:17:09.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.902+0000 7f7bf6a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7bf8161f60 0x7f7bf80d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.902+0000 7f7bf6a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7bf8161f60 0x7f7bf80d0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59298/0 (socket says 172.21.15.159:59298) 2024-04-04T06:17:09.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.902+0000 7f7bf6a00640 1 -- 172.21.15.159:0/2767381795 learned_addr learned my addr 172.21.15.159:0/2767381795 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:09.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.902+0000 7f7bf7400640 1 --1- 172.21.15.159:0/2767381795 >> v1:172.21.15.159:6789/0 conn(0x7f7bf80b3da0 0x7f7bf80b4220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41462/0 (socket says 172.21.15.159:41462) 2024-04-04T06:17:09.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.903+0000 7f7befe00640 1 --2- 172.21.15.159:0/2767381795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bf80b3170 0x7f7bf80d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.903+0000 7f7bf6a00640 1 -- 172.21.15.159:0/2767381795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bf80b3170 msgr2=0x7f7bf80d13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:09.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.903+0000 7f7bf6a00640 1 --2- 172.21.15.159:0/2767381795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bf80b3170 0x7f7bf80d13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.903+0000 7f7bf6a00640 1 -- 172.21.15.159:0/2767381795 >> v1:172.21.15.159:6789/0 conn(0x7f7bf80b3da0 legacy=0x7f7bf80b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:09.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.903+0000 7f7bf6a00640 1 -- 172.21.15.159:0/2767381795 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf8162340 con 0x7f7bf8161f60 2024-04-04T06:17:09.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.903+0000 7f7bf6000640 1 -- 172.21.15.159:0/2767381795 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 542770587 0 0) 0x7f7bf0001490 con 0x7f7bf80b3da0 2024-04-04T06:17:09.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.904+0000 7f7bf6a00640 1 --2- 172.21.15.159:0/2767381795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bf8161f60 0x7f7bf80d0e70 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009870 tx=0x7f7be802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d528546540dbc0fa server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:09.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.904+0000 7f7bf6000640 1 -- 172.21.15.159:0/2767381795 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8034070 con 0x7f7bf8161f60 2024-04-04T06:17:09.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.904+0000 7f7bf6000640 1 -- 172.21.15.159:0/2767381795 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7be8030840 con 0x7f7bf8161f60 2024-04-04T06:17:09.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.904+0000 7f7bf6000640 1 -- 172.21.15.159:0/2767381795 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8030c10 con 0x7f7bf8161f60 2024-04-04T06:17:09.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.905+0000 7f7bfde00640 1 -- 172.21.15.159:0/2767381795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bf8161f60 msgr2=0x7f7bf80d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:09.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.905+0000 7f7bfde00640 1 --2- 172.21.15.159:0/2767381795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bf8161f60 0x7f7bf80d0e70 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009870 tx=0x7f7be802b980 comp rx=0 tx=0).stop 2024-04-04T06:17:09.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.905+0000 7f7bfde00640 1 -- 172.21.15.159:0/2767381795 shutdown_connections 2024-04-04T06:17:09.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.905+0000 7f7bfde00640 1 --2- 172.21.15.159:0/2767381795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bf80b3170 0x7f7bf80d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.905+0000 7f7bfde00640 1 --2- 172.21.15.159:0/2767381795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bf8161f60 0x7f7bf80d0e70 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.905+0000 7f7bfde00640 1 -- 172.21.15.159:0/2767381795 >> 172.21.15.159:0/2767381795 conn(0x7f7bf808bfb0 msgr2=0x7f7bf80d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:09.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.905+0000 7f7bfde00640 1 -- 172.21.15.159:0/2767381795 shutdown_connections 2024-04-04T06:17:09.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.905+0000 7f7bfde00640 1 -- 172.21.15.159:0/2767381795 wait complete. 2024-04-04T06:17:09.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.906+0000 7f7bfde00640 1 Processor -- start 2024-04-04T06:17:09.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.907+0000 7f7bfde00640 1 -- start start 2024-04-04T06:17:09.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.907+0000 7f7bfde00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bf80b3170 0x7f7bf81f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.907+0000 7f7bfde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bf80b3da0 0x7f7bf80cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.907+0000 7f7bfde00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bf8161f60 0x7f7bf815dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.907+0000 7f7bfde00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf80d3310 con 0x7f7bf80b3da0 2024-04-04T06:17:09.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.907+0000 7f7bfde00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf80d3030 con 0x7f7bf80b3170 2024-04-04T06:17:09.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.907+0000 7f7bfde00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7bf80d00e0 con 0x7f7bf8161f60 2024-04-04T06:17:09.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.908+0000 7f7befe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bf80b3da0 0x7f7bf80cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.908+0000 7f7befe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bf80b3da0 0x7f7bf80cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59300/0 (socket says 172.21.15.159:59300) 2024-04-04T06:17:09.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.908+0000 7f7befe00640 1 -- 172.21.15.159:0/3298504498 learned_addr learned my addr 172.21.15.159:0/3298504498 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:09.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.908+0000 7f7befe00640 1 -- 172.21.15.159:0/3298504498 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bf8161f60 msgr2=0x7f7bf815dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:09.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.908+0000 7f7befe00640 1 --2- 172.21.15.159:0/3298504498 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bf8161f60 0x7f7bf815dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.908+0000 7f7befe00640 1 -- 172.21.15.159:0/3298504498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bf80b3170 msgr2=0x7f7bf81f4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:09.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.908+0000 7f7befe00640 1 --2- 172.21.15.159:0/3298504498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bf80b3170 0x7f7bf81f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:09.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.908+0000 7f7befe00640 1 -- 172.21.15.159:0/3298504498 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf81f7a30 con 0x7f7bf80b3da0 2024-04-04T06:17:09.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.908+0000 7f7befe00640 1 --2- 172.21.15.159:0/3298504498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bf80b3da0 0x7f7bf80cddc0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f7be000d830 tx=0x7f7be000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:09.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.909+0000 7f7bef400640 1 -- 172.21.15.159:0/3298504498 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be000b660 con 0x7f7bf80b3da0 2024-04-04T06:17:09.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.909+0000 7f7bef400640 1 -- 172.21.15.159:0/3298504498 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7be0019070 con 0x7f7bf80b3da0 2024-04-04T06:17:09.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.909+0000 7f7bfde00640 1 -- 172.21.15.159:0/3298504498 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf81f7d10 con 0x7f7bf80b3da0 2024-04-04T06:17:09.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.909+0000 7f7bef400640 1 -- 172.21.15.159:0/3298504498 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be000ba20 con 0x7f7bf80b3da0 2024-04-04T06:17:09.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.909+0000 7f7bfde00640 1 -- 172.21.15.159:0/3298504498 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf81f82d0 con 0x7f7bf80b3da0 2024-04-04T06:17:09.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.910+0000 7f7bfde00640 1 -- 172.21.15.159:0/3298504498 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bf80685c0 con 0x7f7bf80b3da0 2024-04-04T06:17:09.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.911+0000 7f7bef400640 1 -- 172.21.15.159:0/3298504498 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7be001e460 con 0x7f7bf80b3da0 2024-04-04T06:17:09.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.912+0000 7f7bef400640 1 --2- 172.21.15.159:0/3298504498 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7bd406c920 0x7f7bd406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:09.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.912+0000 7f7bef400640 1 -- 172.21.15.159:0/3298504498 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f7be008c1a0 con 0x7f7bf80b3da0 2024-04-04T06:17:09.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.913+0000 7f7bf6a00640 1 --2- 172.21.15.159:0/3298504498 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7bd406c920 0x7f7bd406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:09.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.914+0000 7f7bf6a00640 1 --2- 172.21.15.159:0/3298504498 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7bd406c920 0x7f7bd406ede0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7be8036000 tx=0x7f7be8030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:09.910 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:09.917+0000 7f7bef400640 1 -- 172.21.15.159:0/3298504498 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7be005a5c0 con 0x7f7bf80b3da0 2024-04-04T06:17:10.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:10.145+0000 7f7bfde00640 1 -- 172.21.15.159:0/3298504498 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7bf80b2520 con 0x7f7bf80b3da0 2024-04-04T06:17:10.139 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:10.145+0000 7f7bef400640 1 -- 172.21.15.159:0/3298504498 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v101) ==== 53+0+73 (secure 0 0 0) 0x7f7be0059ec0 con 0x7f7bf80b3da0 2024-04-04T06:17:10.139 INFO:teuthology.orchestra.run.smithi159.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-04T06:17:10.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:10.148+0000 7f7bfde00640 1 -- 172.21.15.159:0/3298504498 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7bd406c920 msgr2=0x7f7bd406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:10.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:10.148+0000 7f7bfde00640 1 --2- 172.21.15.159:0/3298504498 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7bd406c920 0x7f7bd406ede0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7be8036000 tx=0x7f7be8030000 comp rx=0 tx=0).stop 2024-04-04T06:17:10.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:10.148+0000 7f7bfde00640 1 -- 172.21.15.159:0/3298504498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bf80b3da0 msgr2=0x7f7bf80cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:10.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:10.148+0000 7f7bfde00640 1 --2- 172.21.15.159:0/3298504498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bf80b3da0 0x7f7bf80cddc0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f7be000d830 tx=0x7f7be000dcf0 comp rx=0 tx=0).stop 2024-04-04T06:17:10.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:10.148+0000 7f7bfde00640 1 -- 172.21.15.159:0/3298504498 shutdown_connections 2024-04-04T06:17:10.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:10.148+0000 7f7bfde00640 1 --2- 172.21.15.159:0/3298504498 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7bf8161f60 0x7f7bf815dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:10.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:10.148+0000 7f7bfde00640 1 --2- 172.21.15.159:0/3298504498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bf80b3da0 0x7f7bf80cddc0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:10.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:10.148+0000 7f7bfde00640 1 --2- 172.21.15.159:0/3298504498 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7bd406c920 0x7f7bd406ede0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:10.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:10.148+0000 7f7bfde00640 1 --2- 172.21.15.159:0/3298504498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bf80b3170 0x7f7bf81f4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:10.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:10.148+0000 7f7bfde00640 1 -- 172.21.15.159:0/3298504498 >> 172.21.15.159:0/3298504498 conn(0x7f7bf808bfb0 msgr2=0x7f7bf80b1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:10.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:10.148+0000 7f7bfde00640 1 -- 172.21.15.159:0/3298504498 shutdown_connections 2024-04-04T06:17:10.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:10.148+0000 7f7bfde00640 1 -- 172.21.15.159:0/3298504498 wait complete. 2024-04-04T06:17:10.160 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-04T06:17:10.160 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-04T06:17:10.160 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi159.front.sepia.ceph.com 2024-04-04T06:17:10.160 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-04T06:17:10.160 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-04T06:17:10.160 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-04T06:17:10.160 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-04T06:17:10.160 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-04T06:17:10.160 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:17:10.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:10.161 DEBUG:teuthology.orchestra.run.smithi159:> ip addr 2024-04-04T06:17:10.208 INFO:teuthology.orchestra.run.smithi159.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:17:10.208 INFO:teuthology.orchestra.run.smithi159.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:17:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:17:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 ::1/128 scope host 2024-04-04T06:17:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:10.209 INFO:teuthology.orchestra.run.smithi159.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:17:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c6 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:17:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f0 2024-04-04T06:17:10.209 INFO:teuthology.orchestra.run.smithi159.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:17:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c7 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:17:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f1 2024-04-04T06:17:10.209 INFO:teuthology.orchestra.run.smithi159.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:17:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4e brd ff:ff:ff:ff:ff:ff 2024-04-04T06:17:10.210 INFO:teuthology.orchestra.run.smithi159.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-04T06:17:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:17:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: inet 172.21.15.159/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-04T06:17:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft 41903sec preferred_lft 41903sec 2024-04-04T06:17:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::ec4:7aff:fe88:724f/64 scope link 2024-04-04T06:17:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:10.210 INFO:teuthology.orchestra.run.smithi159.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-04T06:17:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:17:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-04T06:17:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::989e:e4ff:fea5:7568/64 scope link 2024-04-04T06:17:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:10.210 INFO:teuthology.orchestra.run.smithi159.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:17:10.211 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether a2:e3:e0:cb:0a:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-04T06:17:10.211 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::a0e3:e0ff:fecb:a73/64 scope link 2024-04-04T06:17:10.211 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:10.211 INFO:teuthology.orchestra.run.smithi159.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:17:10.211 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-04T06:17:10.211 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::c0a:8aff:fe54:b58f/64 scope link 2024-04-04T06:17:10.211 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:10.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:10.213 DEBUG:teuthology.orchestra.run.smithi159:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-04T06:17:10.290 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:17:10.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:10.294 DEBUG:teuthology.orchestra.run.smithi159:> route 2024-04-04T06:17:10.337 INFO:teuthology.orchestra.run.smithi159.stdout:Kernel IP routing table 2024-04-04T06:17:10.337 INFO:teuthology.orchestra.run.smithi159.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-04T06:17:10.337 INFO:teuthology.orchestra.run.smithi159.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-04T06:17:10.337 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-04T06:17:10.337 INFO:teuthology.orchestra.run.smithi159.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-04T06:17:10.338 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:17:10.338 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:17:10.338 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-04T06:17:10.338 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-04T06:17:10.338 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-04T06:17:10.338 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:17:10.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:17:10.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:10.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:10.604 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-04-04T06:17:10.652 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-04T06:17:10.652 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-04T06:17:10.652 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:17:10.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:10.653 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set brx.0 up 2024-04-04T06:17:10.730 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:17:10.730 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-04T06:17:10.777 INFO:teuthology.orchestra.run.smithi159.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:17:10.777 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:17:10.777 DEBUG:teuthology.orchestra.run.smithi159:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:17:10.822 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-04T06:17:10.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:10.823 DEBUG:teuthology.orchestra.run.smithi159:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:17:10.998 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:17:10.999 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:17:10.999 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:17:11.017 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:17:11.018 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:17:11.018 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:17:11.018 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:17:11.018 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:17:11.042 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:17:11.043 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:17:11.043 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:17:11.043 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:17:11.043 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:17:11.043 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-04T06:17:11.043 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-04T06:17:11.043 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30311045-f714-48fa-832b-4cb2ba8bf0bf" will pass to kernel 2024-04-04T06:17:11.044 INFO:teuthology.orchestra.run.smithi159.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-04T06:17:11.044 INFO:teuthology.orchestra.run.smithi159.stdout: You just mounted a file system that supports labels which does not 2024-04-04T06:17:11.045 INFO:teuthology.orchestra.run.smithi159.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-04T06:17:11.045 INFO:teuthology.orchestra.run.smithi159.stdout: applications will generate AVC messages and not be allowed access to 2024-04-04T06:17:11.045 INFO:teuthology.orchestra.run.smithi159.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-04T06:17:11.048 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-04T06:17:11.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:11.049 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:17:11.146 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:17:11.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:17:11.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:11.272 INFO:teuthology.orchestra.run.smithi159.stdout:client6412 2024-04-04T06:17:11.273 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:17:11.273 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:17:11.317 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:11.317 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:11.317 DEBUG:teuthology.orchestra.run.smithi159:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-04-04T06:17:11.317 DEBUG:teuthology.orchestra.run.smithi159:> parent = os.path.join(mount_point, "parent") 2024-04-04T06:17:11.317 DEBUG:teuthology.orchestra.run.smithi159:> os.mkdir(parent) 2024-04-04T06:17:11.317 DEBUG:teuthology.orchestra.run.smithi159:> flushed = os.path.join(parent, "flushed") 2024-04-04T06:17:11.317 DEBUG:teuthology.orchestra.run.smithi159:> os.mkdir(flushed) 2024-04-04T06:17:11.317 DEBUG:teuthology.orchestra.run.smithi159:> for f in ["alpha", "bravo", "charlie"]: 2024-04-04T06:17:11.317 DEBUG:teuthology.orchestra.run.smithi159:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-04-04T06:17:11.317 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:11.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:11.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:11.567 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-04T06:17:11.667 INFO:teuthology.orchestra.run.smithi159.stdout:./ 2024-04-04T06:17:11.667 INFO:teuthology.orchestra.run.smithi159.stdout:./parent 2024-04-04T06:17:11.667 INFO:teuthology.orchestra.run.smithi159.stdout:./parent/flushed 2024-04-04T06:17:11.667 INFO:teuthology.orchestra.run.smithi159.stdout:./parent/flushed/charlie 2024-04-04T06:17:11.667 INFO:teuthology.orchestra.run.smithi159.stdout:./parent/flushed/alpha 2024-04-04T06:17:11.667 INFO:teuthology.orchestra.run.smithi159.stdout:./parent/flushed/bravo 2024-04-04T06:17:11.668 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:11.668 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:11.668 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:11.668 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:11.668 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-04T06:17:11.668 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:11.785 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:17:11.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:11.802 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:11.803 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:11.803 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:11.803 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:11.803 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-04T06:17:11.803 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:11.968 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627776 2024-04-04T06:17:11.971 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:11.981 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:11.981 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:11.981 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:11.981 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:11.981 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-04T06:17:11.981 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:12.163 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627777 2024-04-04T06:17:12.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:12.177 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:12.177 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:12.177 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:12.178 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:12.178 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-04T06:17:12.178 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:12.349 INFO:teuthology.orchestra.run.smithi159.stdout:1099511628280 2024-04-04T06:17:12.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:12.364 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:12.364 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:12.364 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:12.364 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:12.365 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-04T06:17:12.365 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:12.538 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627778 2024-04-04T06:17:12.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:12.551 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:12.551 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:12.551 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:12.551 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:12.551 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-04T06:17:12.551 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:12.752 INFO:teuthology.orchestra.run.smithi159.stdout:1099511628279 2024-04-04T06:17:12.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:12.766 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:17:12.766 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:17:12.856 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:17:12.856 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:17:12.946 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:17:13.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.198+0000 7f994bc00640 1 Processor -- start 2024-04-04T06:17:13.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.198+0000 7f994bc00640 1 -- start start 2024-04-04T06:17:13.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.201+0000 7f994bc00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f99440cfb20 0x7f99440cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:13.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.201+0000 7f994bc00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f994415ebc0 con 0x7f994415e7e0 2024-04-04T06:17:13.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.201+0000 7f994bc00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f994406d1c0 con 0x7f99440d1950 2024-04-04T06:17:13.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.201+0000 7f994bc00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f99440d1db0 con 0x7f99440cfb20 2024-04-04T06:17:13.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.201+0000 7f9949200640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f994415e7e0 0x7f994406ca00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41476/0 (socket says 172.21.15.159:41476) 2024-04-04T06:17:13.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.201+0000 7f9949200640 1 -- 172.21.15.159:0/1972836314 learned_addr learned my addr 172.21.15.159:0/1972836314 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:13.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.202+0000 7f9943400640 1 -- 172.21.15.159:0/1972836314 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3212021398 0 0) 0x7f994415ebc0 con 0x7f994415e7e0 2024-04-04T06:17:13.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.202+0000 7f9943400640 1 -- 172.21.15.159:0/1972836314 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9928003520 con 0x7f994415e7e0 2024-04-04T06:17:13.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.202+0000 7f9949c00640 1 --2- 172.21.15.159:0/1972836314 >> v2:172.21.15.178:3300/0 conn(0x7f99440cfb20 0x7f99440cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:13.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.202+0000 7f9943400640 1 -- 172.21.15.159:0/1972836314 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1814004972 0 0) 0x7f9928003520 con 0x7f994415e7e0 2024-04-04T06:17:13.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.202+0000 7f9943400640 1 -- 172.21.15.159:0/1972836314 >> v2:172.21.15.178:3300/0 conn(0x7f99440cfb20 msgr2=0x7f99440cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:13.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.202+0000 7f9943400640 1 --2- 172.21.15.159:0/1972836314 >> v2:172.21.15.178:3300/0 conn(0x7f99440cfb20 0x7f99440cff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:13.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.202+0000 7f9943400640 1 -- 172.21.15.159:0/1972836314 >> v1:172.21.15.178:6789/0 conn(0x7f99440d1950 legacy=0x7f99440d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:13.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.202+0000 7f9943400640 1 -- 172.21.15.159:0/1972836314 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994406d4a0 con 0x7f994415e7e0 2024-04-04T06:17:13.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.202+0000 7f9943400640 1 -- 172.21.15.159:0/1972836314 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f9934001670 con 0x7f994415e7e0 2024-04-04T06:17:13.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.202+0000 7f9943400640 1 -- 172.21.15.159:0/1972836314 >> v1:172.21.15.159:6789/0 conn(0x7f994415e7e0 legacy=0x7f994406ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:13.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.202+0000 7f9943400640 1 --2- 172.21.15.159:0/1972836314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99280039b0 0x7f9928003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:13.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.202+0000 7f9943400640 1 --2- 172.21.15.159:0/1972836314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9928004800 0x7f9928004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:13.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.202+0000 7f9943400640 1 --2- 172.21.15.159:0/1972836314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99280052e0 0x7f992800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:13.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.202+0000 7f9943400640 1 -- 172.21.15.159:0/1972836314 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f99440d1db0 con 0x7f99280052e0 2024-04-04T06:17:13.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.202+0000 7f9943400640 1 -- 172.21.15.159:0/1972836314 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f992800c1b0 con 0x7f99280039b0 2024-04-04T06:17:13.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.202+0000 7f9943400640 1 -- 172.21.15.159:0/1972836314 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f992800c3b0 con 0x7f9928004800 2024-04-04T06:17:13.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.203+0000 7f9949c00640 1 --2- 172.21.15.159:0/1972836314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99280052e0 0x7f992800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:13.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.203+0000 7f9949200640 1 --2- 172.21.15.159:0/1972836314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99280039b0 0x7f9928003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:13.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.203+0000 7f9949c00640 1 -- 172.21.15.159:0/1972836314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9928004800 msgr2=0x7f9928004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:13.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.203+0000 7f9949c00640 1 --2- 172.21.15.159:0/1972836314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9928004800 0x7f9928004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:13.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.203+0000 7f9949c00640 1 -- 172.21.15.159:0/1972836314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99280039b0 msgr2=0x7f9928003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:13.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.203+0000 7f9949c00640 1 --2- 172.21.15.159:0/1972836314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99280039b0 0x7f9928003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:13.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.203+0000 7f9949c00640 1 -- 172.21.15.159:0/1972836314 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9928003520 con 0x7f99280052e0 2024-04-04T06:17:13.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.204+0000 7f9949c00640 1 -- 172.21.15.159:0/1972836314 reap_dead start 2024-04-04T06:17:13.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.204+0000 7f9949c00640 1 --2- 172.21.15.159:0/1972836314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99280052e0 0x7f992800b9c0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f993c007330 tx=0x7f9928005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2f22ff50894947 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:13.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.204+0000 7f9943400640 1 -- 172.21.15.159:0/1972836314 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c009be0 con 0x7f99280052e0 2024-04-04T06:17:13.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.205+0000 7f9943400640 1 -- 172.21.15.159:0/1972836314 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f993c02ccd0 con 0x7f99280052e0 2024-04-04T06:17:13.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.205+0000 7f9943400640 1 -- 172.21.15.159:0/1972836314 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c003bd0 con 0x7f99280052e0 2024-04-04T06:17:13.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.205+0000 7f994bc00640 1 -- 172.21.15.159:0/1972836314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99280052e0 msgr2=0x7f992800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:13.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.205+0000 7f994bc00640 1 --2- 172.21.15.159:0/1972836314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99280052e0 0x7f992800b9c0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f993c007330 tx=0x7f9928005830 comp rx=0 tx=0).stop 2024-04-04T06:17:13.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.205+0000 7f994bc00640 1 -- 172.21.15.159:0/1972836314 shutdown_connections 2024-04-04T06:17:13.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.205+0000 7f994bc00640 1 --2- 172.21.15.159:0/1972836314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99280052e0 0x7f992800b9c0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:13.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.205+0000 7f994bc00640 1 -- 172.21.15.159:0/1972836314 >> 172.21.15.159:0/1972836314 conn(0x7f994409d7c0 msgr2=0x7f994406bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:13.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.205+0000 7f994bc00640 1 -- 172.21.15.159:0/1972836314 shutdown_connections 2024-04-04T06:17:13.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.205+0000 7f994bc00640 1 -- 172.21.15.159:0/1972836314 wait complete. 2024-04-04T06:17:13.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.207+0000 7f994bc00640 1 Processor -- start 2024-04-04T06:17:13.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.207+0000 7f994bc00640 1 -- start start 2024-04-04T06:17:13.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.208+0000 7f994bc00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99280052e0 0x7f99441529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:13.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.208+0000 7f994bc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99440cab20 0x7f99440c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:13.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.208+0000 7f994bc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99440cd490 0x7f99440cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:13.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.208+0000 7f994bc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f99440cefd0 con 0x7f99440cd490 2024-04-04T06:17:13.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.208+0000 7f994bc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f99440cf600 con 0x7f99440cab20 2024-04-04T06:17:13.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.208+0000 7f994bc00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f99440cff90 con 0x7f99280052e0 2024-04-04T06:17:13.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.209+0000 7f9949c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99440cd490 0x7f99440cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:13.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.209+0000 7f9949c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99440cd490 0x7f99440cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:59330/0 (socket says 172.21.15.159:59330) 2024-04-04T06:17:13.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.209+0000 7f9949c00640 1 -- 172.21.15.159:0/2457579881 learned_addr learned my addr 172.21.15.159:0/2457579881 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:13.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.209+0000 7f9943e00640 1 --2- 172.21.15.159:0/2457579881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99440cab20 0x7f99440c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:13.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.209+0000 7f9949200640 1 --2- 172.21.15.159:0/2457579881 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99280052e0 0x7f99441529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:13.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.209+0000 7f9949c00640 1 -- 172.21.15.159:0/2457579881 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99280052e0 msgr2=0x7f99441529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:13.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.209+0000 7f9949c00640 1 --2- 172.21.15.159:0/2457579881 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99280052e0 0x7f99441529b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:13.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.209+0000 7f9949c00640 1 -- 172.21.15.159:0/2457579881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99440cab20 msgr2=0x7f99440c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:13.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.209+0000 7f9949c00640 1 --2- 172.21.15.159:0/2457579881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99440cab20 0x7f99440c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:13.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.209+0000 7f9949c00640 1 -- 172.21.15.159:0/2457579881 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99440cc6d0 con 0x7f99440cd490 2024-04-04T06:17:13.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.209+0000 7f9949c00640 1 --2- 172.21.15.159:0/2457579881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99440cd490 0x7f99440cd910 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f993c02c740 tx=0x7f993c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:13.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.210+0000 7f9940c00640 1 -- 172.21.15.159:0/2457579881 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c042070 con 0x7f99440cd490 2024-04-04T06:17:13.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.210+0000 7f9940c00640 1 -- 172.21.15.159:0/2457579881 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f993c003e10 con 0x7f99440cd490 2024-04-04T06:17:13.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.210+0000 7f9940c00640 1 -- 172.21.15.159:0/2457579881 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c039070 con 0x7f99440cd490 2024-04-04T06:17:13.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.210+0000 7f994bc00640 1 -- 172.21.15.159:0/2457579881 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99440cc9b0 con 0x7f99440cd490 2024-04-04T06:17:13.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.210+0000 7f994bc00640 1 -- 172.21.15.159:0/2457579881 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9944152ef0 con 0x7f99440cd490 2024-04-04T06:17:13.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.211+0000 7f994bc00640 1 -- 172.21.15.159:0/2457579881 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99440685c0 con 0x7f99440cd490 2024-04-04T06:17:13.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.213+0000 7f9940c00640 1 -- 172.21.15.159:0/2457579881 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f993c045070 con 0x7f99440cd490 2024-04-04T06:17:13.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.214+0000 7f9940c00640 1 --2- 172.21.15.159:0/2457579881 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f992406c920 0x7f992406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:13.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.214+0000 7f9940c00640 1 -- 172.21.15.159:0/2457579881 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f993c0acf60 con 0x7f99440cd490 2024-04-04T06:17:13.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.215+0000 7f9949200640 1 --2- 172.21.15.159:0/2457579881 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f992406c920 0x7f992406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:13.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.216+0000 7f9949200640 1 --2- 172.21.15.159:0/2457579881 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f992406c920 0x7f992406ede0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f994408e700 tx=0x7f9934009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:13.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.219+0000 7f9940c00640 1 -- 172.21.15.159:0/2457579881 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f993c07b420 con 0x7f99440cd490 2024-04-04T06:17:13.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.421+0000 7f994bc00640 1 -- 172.21.15.159:0/2457579881 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99440d5300 con 0x7f99440cd490 2024-04-04T06:17:13.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.422+0000 7f9940c00640 1 -- 172.21.15.159:0/2457579881 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16639 (secure 0 0 0) 0x7f993c07ad20 con 0x7f99440cd490 2024-04-04T06:17:13.418 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:13.418 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":113,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:16:58.838677+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:16:55.036975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-04T06:16:56.239022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:17:13.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.425+0000 7f994bc00640 1 -- 172.21.15.159:0/2457579881 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f992406c920 msgr2=0x7f992406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:13.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.425+0000 7f994bc00640 1 --2- 172.21.15.159:0/2457579881 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f992406c920 0x7f992406ede0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f994408e700 tx=0x7f9934009040 comp rx=0 tx=0).stop 2024-04-04T06:17:13.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.425+0000 7f994bc00640 1 -- 172.21.15.159:0/2457579881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99440cd490 msgr2=0x7f99440cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:13.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.425+0000 7f994bc00640 1 --2- 172.21.15.159:0/2457579881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99440cd490 0x7f99440cd910 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f993c02c740 tx=0x7f993c004ab0 comp rx=0 tx=0).stop 2024-04-04T06:17:13.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.425+0000 7f994bc00640 1 -- 172.21.15.159:0/2457579881 shutdown_connections 2024-04-04T06:17:13.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.425+0000 7f994bc00640 1 --2- 172.21.15.159:0/2457579881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99440cd490 0x7f99440cd910 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:13.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.425+0000 7f994bc00640 1 --2- 172.21.15.159:0/2457579881 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f992406c920 0x7f992406ede0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:13.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.425+0000 7f994bc00640 1 --2- 172.21.15.159:0/2457579881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99440cab20 0x7f99440c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:13.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.425+0000 7f994bc00640 1 --2- 172.21.15.159:0/2457579881 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99280052e0 0x7f99441529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:13.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.425+0000 7f994bc00640 1 -- 172.21.15.159:0/2457579881 >> 172.21.15.159:0/2457579881 conn(0x7f994409d7c0 msgr2=0x7f99440b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:13.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.425+0000 7f994bc00640 1 -- 172.21.15.159:0/2457579881 shutdown_connections 2024-04-04T06:17:13.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.426+0000 7f994bc00640 1 -- 172.21.15.159:0/2457579881 wait complete. 2024-04-04T06:17:13.441 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:16:55.036975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:16:56.239022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:17:13.442 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:17:13.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.688+0000 7f12f3200640 1 Processor -- start 2024-04-04T06:17:13.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.688+0000 7f12f3200640 1 -- start start 2024-04-04T06:17:13.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.689+0000 7f12f3200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12f416b0e0 0x7f12f408b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:13.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.689+0000 7f12f3200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f12f40713e0 0x7f12f4071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:13.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.689+0000 7f12f3200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12f416b4c0 con 0x7f12f4071da0 2024-04-04T06:17:13.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.689+0000 7f12f3200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f12f4072230 con 0x7f12f40713e0 2024-04-04T06:17:13.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.689+0000 7f12f3200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f12f406d620 con 0x7f12f416b0e0 2024-04-04T06:17:13.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.690+0000 7f12f1e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12f416b0e0 0x7f12f408b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:13.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.690+0000 7f12f1400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f12f40713e0 0x7f12f4071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:13.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.690+0000 7f12f1400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f12f40713e0 0x7f12f4071860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:54236/0 (socket says 172.21.15.159:54236) 2024-04-04T06:17:13.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.690+0000 7f12f1400640 1 -- 172.21.15.159:0/3778790599 learned_addr learned my addr 172.21.15.159:0/3778790599 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:13.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.691+0000 7f12f0a00640 1 -- 172.21.15.159:0/3778790599 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3656572185 0 0) 0x7f12f416b4c0 con 0x7f12f4071da0 2024-04-04T06:17:13.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.878+0000 7f12f0a00640 1 -- 172.21.15.159:0/3778790599 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12d8003500 con 0x7f12f4071da0 2024-04-04T06:17:13.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.879+0000 7f12f1400640 1 -- 172.21.15.159:0/3778790599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12f416b0e0 msgr2=0x7f12f408b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:13.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.879+0000 7f12f1400640 1 --2- 172.21.15.159:0/3778790599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12f416b0e0 0x7f12f408b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:13.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.879+0000 7f12f1400640 1 -- 172.21.15.159:0/3778790599 >> v1:172.21.15.178:6789/0 conn(0x7f12f4071da0 legacy=0x7f12f406dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:13.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.879+0000 7f12f0a00640 1 -- 172.21.15.159:0/3778790599 <== mon.1 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2257669878 0 0) 0x7f12d8003500 con 0x7f12f4071da0 2024-04-04T06:17:13.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.879+0000 7f12f1400640 1 -- 172.21.15.159:0/3778790599 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f40b4280 con 0x7f12f40713e0 2024-04-04T06:17:13.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.881+0000 7f12f1400640 1 --2- 172.21.15.159:0/3778790599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12f40713e0 0x7f12f4071860 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f12e000a910 tx=0x7f12e000a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25f2da5e3df1e365 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:13.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.882+0000 7f12f0a00640 1 -- 172.21.15.159:0/3778790599 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e0032070 con 0x7f12f40713e0 2024-04-04T06:17:13.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.882+0000 7f12f0a00640 1 -- 172.21.15.159:0/3778790599 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12e002c780 con 0x7f12f40713e0 2024-04-04T06:17:13.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.882+0000 7f12f0a00640 1 -- 172.21.15.159:0/3778790599 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec0019f0 con 0x7f12f40713e0 2024-04-04T06:17:13.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.883+0000 7f12f3200640 1 -- 172.21.15.159:0/3778790599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12f40713e0 msgr2=0x7f12f4071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:13.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.883+0000 7f12f3200640 1 --2- 172.21.15.159:0/3778790599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12f40713e0 0x7f12f4071860 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f12e000a910 tx=0x7f12e000a250 comp rx=0 tx=0).stop 2024-04-04T06:17:13.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.884+0000 7f12f3200640 1 -- 172.21.15.159:0/3778790599 shutdown_connections 2024-04-04T06:17:13.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.884+0000 7f12f3200640 1 --2- 172.21.15.159:0/3778790599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12f40713e0 0x7f12f4071860 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:13.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.884+0000 7f12f3200640 1 --2- 172.21.15.159:0/3778790599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12f416b0e0 0x7f12f408b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:13.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.884+0000 7f12f3200640 1 -- 172.21.15.159:0/3778790599 >> 172.21.15.159:0/3778790599 conn(0x7f12f40b1ba0 msgr2=0x7f12f40b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:13.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.884+0000 7f12f3200640 1 -- 172.21.15.159:0/3778790599 shutdown_connections 2024-04-04T06:17:13.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.884+0000 7f12f3200640 1 -- 172.21.15.159:0/3778790599 wait complete. 2024-04-04T06:17:13.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.888+0000 7f12f3200640 1 Processor -- start 2024-04-04T06:17:13.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.888+0000 7f12f3200640 1 -- start start 2024-04-04T06:17:13.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.890+0000 7f12f3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f40713e0 0x7f12f40d0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:13.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.890+0000 7f12f3200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12f4071da0 0x7f12f40ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:13.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.890+0000 7f12f3200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12f416b0e0 0x7f12f40cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:13.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.890+0000 7f12f3200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f12f40b4840 con 0x7f12f40713e0 2024-04-04T06:17:13.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.890+0000 7f12f3200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f12f40b53c0 con 0x7f12f416b0e0 2024-04-04T06:17:13.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.890+0000 7f12f3200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f12f408b7f0 con 0x7f12f4071da0 2024-04-04T06:17:13.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.891+0000 7f12f1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f40713e0 0x7f12f40d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:13.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.891+0000 7f12f1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f40713e0 0x7f12f40d0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37258/0 (socket says 172.21.15.159:37258) 2024-04-04T06:17:13.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.891+0000 7f12f1e00640 1 -- 172.21.15.159:0/3861479767 learned_addr learned my addr 172.21.15.159:0/3861479767 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:13.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.891+0000 7f12f1400640 1 --2- 172.21.15.159:0/3861479767 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12f4071da0 0x7f12f40ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:13.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.891+0000 7f12f2800640 1 --2- 172.21.15.159:0/3861479767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12f416b0e0 0x7f12f40cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:13.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.891+0000 7f12f1e00640 1 -- 172.21.15.159:0/3861479767 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12f4071da0 msgr2=0x7f12f40ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:13.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.891+0000 7f12f1e00640 1 --2- 172.21.15.159:0/3861479767 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12f4071da0 0x7f12f40ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:13.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.891+0000 7f12f1e00640 1 -- 172.21.15.159:0/3861479767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12f416b0e0 msgr2=0x7f12f40cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:13.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.891+0000 7f12f1e00640 1 --2- 172.21.15.159:0/3861479767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12f416b0e0 0x7f12f40cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:13.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.891+0000 7f12f1e00640 1 -- 172.21.15.159:0/3861479767 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f4156400 con 0x7f12f40713e0 2024-04-04T06:17:13.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.892+0000 7f12f1e00640 1 --2- 172.21.15.159:0/3861479767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f40713e0 0x7f12f40d0970 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f12e4012ab0 tx=0x7f12e4012f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:13.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.892+0000 7f12ea000640 1 -- 172.21.15.159:0/3861479767 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e4010a80 con 0x7f12f40713e0 2024-04-04T06:17:13.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.892+0000 7f12ea000640 1 -- 172.21.15.159:0/3861479767 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12e401d070 con 0x7f12f40713e0 2024-04-04T06:17:13.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.892+0000 7f12ea000640 1 -- 172.21.15.159:0/3861479767 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e4010e60 con 0x7f12f40713e0 2024-04-04T06:17:13.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.892+0000 7f12f3200640 1 -- 172.21.15.159:0/3861479767 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f41566e0 con 0x7f12f40713e0 2024-04-04T06:17:13.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.893+0000 7f12f3200640 1 -- 172.21.15.159:0/3861479767 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f4156ca0 con 0x7f12f40713e0 2024-04-04T06:17:13.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.894+0000 7f12f3200640 1 -- 172.21.15.159:0/3861479767 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12f40723b0 con 0x7f12f40713e0 2024-04-04T06:17:13.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.895+0000 7f12ea000640 1 -- 172.21.15.159:0/3861479767 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f12e4016050 con 0x7f12f40713e0 2024-04-04T06:17:13.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.896+0000 7f12ea000640 1 --2- 172.21.15.159:0/3861479767 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f12d406c920 0x7f12d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:13.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.896+0000 7f12ea000640 1 -- 172.21.15.159:0/3861479767 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f12e408f880 con 0x7f12f40713e0 2024-04-04T06:17:13.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.896+0000 7f12f1400640 1 --2- 172.21.15.159:0/3861479767 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f12d406c920 0x7f12d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:13.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.898+0000 7f12f1400640 1 --2- 172.21.15.159:0/3861479767 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f12d406c920 0x7f12d406ede0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f12e0037000 tx=0x7f12e0032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:13.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:13.900+0000 7f12ea000640 1 -- 172.21.15.159:0/3861479767 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f12f40723b0 con 0x7f12f40713e0 2024-04-04T06:17:14.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.098+0000 7f12f3200640 1 -- 172.21.15.159:0/3861479767 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f12f4075940 con 0x7f12f40713e0 2024-04-04T06:17:14.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.100+0000 7f12ea000640 1 -- 172.21.15.159:0/3861479767 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7f12e4026020 con 0x7f12f40713e0 2024-04-04T06:17:14.094 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:17:14.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.102+0000 7f12f3200640 1 -- 172.21.15.159:0/3861479767 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f12d406c920 msgr2=0x7f12d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:14.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.102+0000 7f12f3200640 1 --2- 172.21.15.159:0/3861479767 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f12d406c920 0x7f12d406ede0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f12e0037000 tx=0x7f12e0032000 comp rx=0 tx=0).stop 2024-04-04T06:17:14.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.102+0000 7f12f3200640 1 -- 172.21.15.159:0/3861479767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f40713e0 msgr2=0x7f12f40d0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:14.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.102+0000 7f12f3200640 1 --2- 172.21.15.159:0/3861479767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f40713e0 0x7f12f40d0970 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f12e4012ab0 tx=0x7f12e4012f70 comp rx=0 tx=0).stop 2024-04-04T06:17:14.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.102+0000 7f12f3200640 1 -- 172.21.15.159:0/3861479767 shutdown_connections 2024-04-04T06:17:14.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.102+0000 7f12f3200640 1 --2- 172.21.15.159:0/3861479767 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f12d406c920 0x7f12d406ede0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:14.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.102+0000 7f12f3200640 1 --2- 172.21.15.159:0/3861479767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12f416b0e0 0x7f12f40cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:14.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.102+0000 7f12f3200640 1 --2- 172.21.15.159:0/3861479767 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12f4071da0 0x7f12f40ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:14.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.102+0000 7f12f3200640 1 --2- 172.21.15.159:0/3861479767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f40713e0 0x7f12f40d0970 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:14.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.102+0000 7f12f3200640 1 -- 172.21.15.159:0/3861479767 >> 172.21.15.159:0/3861479767 conn(0x7f12f40b1ba0 msgr2=0x7f12f416b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:14.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.103+0000 7f12f3200640 1 -- 172.21.15.159:0/3861479767 shutdown_connections 2024-04-04T06:17:14.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.103+0000 7f12f3200640 1 -- 172.21.15.159:0/3861479767 wait complete. 2024-04-04T06:17:14.118 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:17:14.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.332+0000 7fcd47e00640 1 Processor -- start 2024-04-04T06:17:14.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.332+0000 7fcd47e00640 1 -- start start 2024-04-04T06:17:14.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.333+0000 7fcd47e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcd480d2bc0 0x7fcd480b5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:14.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.334+0000 7fcd47e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd4806dde0 con 0x7fcd4806fab0 2024-04-04T06:17:14.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.334+0000 7fcd47e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd480b5b90 con 0x7fcd4815f6f0 2024-04-04T06:17:14.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.334+0000 7fcd47e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fcd480b5e70 con 0x7fcd480d2bc0 2024-04-04T06:17:14.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.334+0000 7fcd47400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fcd4806fab0 0x7fcd4815fc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40982/0 (socket says 172.21.15.159:40982) 2024-04-04T06:17:14.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.334+0000 7fcd47400640 1 -- 172.21.15.159:0/785162327 learned_addr learned my addr 172.21.15.159:0/785162327 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:14.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.334+0000 7fcd46000640 1 -- 172.21.15.159:0/785162327 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2961730985 0 0) 0x7fcd4806dde0 con 0x7fcd4806fab0 2024-04-04T06:17:14.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.334+0000 7fcd46000640 1 -- 172.21.15.159:0/785162327 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd2c003520 con 0x7fcd4806fab0 2024-04-04T06:17:14.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.334+0000 7fcd46a00640 1 --2- 172.21.15.159:0/785162327 >> v2:172.21.15.178:3300/0 conn(0x7fcd480d2bc0 0x7fcd480b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:14.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.335+0000 7fcd46000640 1 -- 172.21.15.159:0/785162327 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 65705862 0 0) 0x7fcd2c003520 con 0x7fcd4806fab0 2024-04-04T06:17:14.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.335+0000 7fcd46000640 1 -- 172.21.15.159:0/785162327 >> v2:172.21.15.178:3300/0 conn(0x7fcd480d2bc0 msgr2=0x7fcd480b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:14.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.335+0000 7fcd46000640 1 --2- 172.21.15.159:0/785162327 >> v2:172.21.15.178:3300/0 conn(0x7fcd480d2bc0 0x7fcd480b5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:14.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.335+0000 7fcd46000640 1 -- 172.21.15.159:0/785162327 >> v1:172.21.15.178:6789/0 conn(0x7fcd4815f6f0 legacy=0x7fcd4806d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:14.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.335+0000 7fcd46000640 1 -- 172.21.15.159:0/785162327 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4806cbf0 con 0x7fcd4806fab0 2024-04-04T06:17:14.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.335+0000 7fcd46000640 1 -- 172.21.15.159:0/785162327 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fcd400019f0 con 0x7fcd4806fab0 2024-04-04T06:17:14.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.335+0000 7fcd46000640 1 -- 172.21.15.159:0/785162327 >> v1:172.21.15.159:6789/0 conn(0x7fcd4806fab0 legacy=0x7fcd4815fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:14.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.335+0000 7fcd46000640 1 --2- 172.21.15.159:0/785162327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd2c0039b0 0x7fcd2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:14.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.335+0000 7fcd46000640 1 --2- 172.21.15.159:0/785162327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd2c004800 0x7fcd2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:14.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.335+0000 7fcd46000640 1 --2- 172.21.15.159:0/785162327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd2c0052e0 0x7fcd2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:14.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.335+0000 7fcd46000640 1 -- 172.21.15.159:0/785162327 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcd480b5e70 con 0x7fcd2c0052e0 2024-04-04T06:17:14.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.335+0000 7fcd46000640 1 -- 172.21.15.159:0/785162327 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c00c1b0 con 0x7fcd2c0039b0 2024-04-04T06:17:14.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.335+0000 7fcd46000640 1 -- 172.21.15.159:0/785162327 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcd2c00c3b0 con 0x7fcd2c004800 2024-04-04T06:17:14.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.335+0000 7fcd47400640 1 --2- 172.21.15.159:0/785162327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd2c0052e0 0x7fcd2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:14.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.336+0000 7fcd46a00640 1 --2- 172.21.15.159:0/785162327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd2c0039b0 0x7fcd2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:14.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.336+0000 7fcd3fe00640 1 --2- 172.21.15.159:0/785162327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd2c004800 0x7fcd2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:14.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.336+0000 7fcd47400640 1 -- 172.21.15.159:0/785162327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd2c004800 msgr2=0x7fcd2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:14.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.336+0000 7fcd47400640 1 --2- 172.21.15.159:0/785162327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd2c004800 0x7fcd2c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:14.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.336+0000 7fcd47400640 1 -- 172.21.15.159:0/785162327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd2c0039b0 msgr2=0x7fcd2c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:14.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.336+0000 7fcd47400640 1 --2- 172.21.15.159:0/785162327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd2c0039b0 0x7fcd2c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:14.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.336+0000 7fcd47400640 1 -- 172.21.15.159:0/785162327 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2c003520 con 0x7fcd2c0052e0 2024-04-04T06:17:14.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.337+0000 7fcd47400640 1 -- 172.21.15.159:0/785162327 reap_dead start 2024-04-04T06:17:14.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.337+0000 7fcd47400640 1 --2- 172.21.15.159:0/785162327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd2c0052e0 0x7fcd2c00b9c0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000a830 tx=0x7fcd2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99029fd6128f1e5a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:14.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.337+0000 7fcd46000640 1 -- 172.21.15.159:0/785162327 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4000a480 con 0x7fcd2c0052e0 2024-04-04T06:17:14.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.337+0000 7fcd46000640 1 -- 172.21.15.159:0/785162327 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd40034630 con 0x7fcd2c0052e0 2024-04-04T06:17:14.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.337+0000 7fcd46000640 1 -- 172.21.15.159:0/785162327 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40034a00 con 0x7fcd2c0052e0 2024-04-04T06:17:14.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.338+0000 7fcd47e00640 1 -- 172.21.15.159:0/785162327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd2c0052e0 msgr2=0x7fcd2c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:14.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.338+0000 7fcd47e00640 1 --2- 172.21.15.159:0/785162327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd2c0052e0 0x7fcd2c00b9c0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000a830 tx=0x7fcd2c005830 comp rx=0 tx=0).stop 2024-04-04T06:17:14.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.338+0000 7fcd47e00640 1 -- 172.21.15.159:0/785162327 shutdown_connections 2024-04-04T06:17:14.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.338+0000 7fcd47e00640 1 --2- 172.21.15.159:0/785162327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd2c0052e0 0x7fcd2c00b9c0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:14.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.338+0000 7fcd47e00640 1 -- 172.21.15.159:0/785162327 >> 172.21.15.159:0/785162327 conn(0x7fcd48067460 msgr2=0x7fcd48067840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:14.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.338+0000 7fcd47e00640 1 -- 172.21.15.159:0/785162327 shutdown_connections 2024-04-04T06:17:14.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.338+0000 7fcd47e00640 1 -- 172.21.15.159:0/785162327 wait complete. 2024-04-04T06:17:14.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.340+0000 7fcd47e00640 1 Processor -- start 2024-04-04T06:17:14.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.340+0000 7fcd47e00640 1 -- start start 2024-04-04T06:17:14.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.341+0000 7fcd47e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd2c0052e0 0x7fcd48150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:14.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.341+0000 7fcd47e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd481f4680 0x7fcd48155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:14.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.341+0000 7fcd47e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd481560e0 0x7fcd48161690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:14.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.341+0000 7fcd47e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4806ff40 con 0x7fcd2c0052e0 2024-04-04T06:17:14.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.341+0000 7fcd47e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4806d1b0 con 0x7fcd481560e0 2024-04-04T06:17:14.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.341+0000 7fcd47e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fcd480700c0 con 0x7fcd481f4680 2024-04-04T06:17:14.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.341+0000 7fcd46a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd2c0052e0 0x7fcd48150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:14.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.341+0000 7fcd46a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd2c0052e0 0x7fcd48150790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37262/0 (socket says 172.21.15.159:37262) 2024-04-04T06:17:14.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.341+0000 7fcd46a00640 1 -- 172.21.15.159:0/895841963 learned_addr learned my addr 172.21.15.159:0/895841963 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:14.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.341+0000 7fcd3fe00640 1 --2- 172.21.15.159:0/895841963 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd481f4680 0x7fcd48155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:14.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.341+0000 7fcd47400640 1 --2- 172.21.15.159:0/895841963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd481560e0 0x7fcd48161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:14.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.342+0000 7fcd46a00640 1 -- 172.21.15.159:0/895841963 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd481f4680 msgr2=0x7fcd48155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:14.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.342+0000 7fcd46a00640 1 --2- 172.21.15.159:0/895841963 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd481f4680 0x7fcd48155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:14.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.342+0000 7fcd46a00640 1 -- 172.21.15.159:0/895841963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd481560e0 msgr2=0x7fcd48161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:14.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.342+0000 7fcd46a00640 1 --2- 172.21.15.159:0/895841963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd481560e0 0x7fcd48161690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:14.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.342+0000 7fcd46a00640 1 -- 172.21.15.159:0/895841963 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd481623c0 con 0x7fcd2c0052e0 2024-04-04T06:17:14.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.342+0000 7fcd3fe00640 1 --2- 172.21.15.159:0/895841963 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd481f4680 0x7fcd48155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:14.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.342+0000 7fcd47400640 1 --2- 172.21.15.159:0/895841963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd481560e0 0x7fcd48161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:14.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.342+0000 7fcd46a00640 1 --2- 172.21.15.159:0/895841963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd2c0052e0 0x7fcd48150790 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800f5e0 tx=0x7fcd3800faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:14.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.342+0000 7fcd3ea00640 1 -- 172.21.15.159:0/895841963 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd38013810 con 0x7fcd2c0052e0 2024-04-04T06:17:14.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.342+0000 7fcd3ea00640 1 -- 172.21.15.159:0/895841963 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd3801a070 con 0x7fcd2c0052e0 2024-04-04T06:17:14.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.342+0000 7fcd3ea00640 1 -- 172.21.15.159:0/895841963 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd38013bb0 con 0x7fcd2c0052e0 2024-04-04T06:17:14.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.342+0000 7fcd47e00640 1 -- 172.21.15.159:0/895841963 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd48151670 con 0x7fcd2c0052e0 2024-04-04T06:17:14.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.342+0000 7fcd47e00640 1 -- 172.21.15.159:0/895841963 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd48151c30 con 0x7fcd2c0052e0 2024-04-04T06:17:14.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.344+0000 7fcd47e00640 1 -- 172.21.15.159:0/895841963 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd48075940 con 0x7fcd2c0052e0 2024-04-04T06:17:14.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.345+0000 7fcd3ea00640 1 -- 172.21.15.159:0/895841963 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd38012070 con 0x7fcd2c0052e0 2024-04-04T06:17:14.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.345+0000 7fcd3ea00640 1 --2- 172.21.15.159:0/895841963 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcd2406c920 0x7fcd2406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:14.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.346+0000 7fcd3ea00640 1 -- 172.21.15.159:0/895841963 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7fcd3808bb20 con 0x7fcd2c0052e0 2024-04-04T06:17:14.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.346+0000 7fcd3fe00640 1 --2- 172.21.15.159:0/895841963 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcd2406c920 0x7fcd2406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:14.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.347+0000 7fcd3fe00640 1 --2- 172.21.15.159:0/895841963 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcd2406c920 0x7fcd2406ede0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcd30000c00 tx=0x7fcd30009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:14.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.354+0000 7fcd3ea00640 1 -- 172.21.15.159:0/895841963 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fcd3801c310 con 0x7fcd2c0052e0 2024-04-04T06:17:14.381 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:17:14.387 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:17:14.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.599+0000 7fcd47e00640 1 -- 172.21.15.159:0/895841963 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcd48071e10 con 0x7fcd2c0052e0 2024-04-04T06:17:14.595 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.600+0000 7fcd3ea00640 1 -- 172.21.15.159:0/895841963 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) ==== 104+0+11 (secure 0 0 0) 0x7fcd38059f30 con 0x7fcd2c0052e0 2024-04-04T06:17:14.595 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:17:14.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.604+0000 7fcd47e00640 1 -- 172.21.15.159:0/895841963 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcd2406c920 msgr2=0x7fcd2406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:14.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.604+0000 7fcd47e00640 1 --2- 172.21.15.159:0/895841963 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcd2406c920 0x7fcd2406ede0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcd30000c00 tx=0x7fcd30009040 comp rx=0 tx=0).stop 2024-04-04T06:17:14.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.604+0000 7fcd47e00640 1 -- 172.21.15.159:0/895841963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd2c0052e0 msgr2=0x7fcd48150790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:14.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.604+0000 7fcd47e00640 1 --2- 172.21.15.159:0/895841963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd2c0052e0 0x7fcd48150790 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800f5e0 tx=0x7fcd3800faa0 comp rx=0 tx=0).stop 2024-04-04T06:17:14.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.604+0000 7fcd47e00640 1 -- 172.21.15.159:0/895841963 shutdown_connections 2024-04-04T06:17:14.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.604+0000 7fcd47e00640 1 --2- 172.21.15.159:0/895841963 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fcd2406c920 0x7fcd2406ede0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:14.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.604+0000 7fcd47e00640 1 --2- 172.21.15.159:0/895841963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd481560e0 0x7fcd48161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:14.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.604+0000 7fcd47e00640 1 --2- 172.21.15.159:0/895841963 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcd481f4680 0x7fcd48155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:14.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.604+0000 7fcd47e00640 1 --2- 172.21.15.159:0/895841963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd2c0052e0 0x7fcd48150790 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:14.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.604+0000 7fcd47e00640 1 -- 172.21.15.159:0/895841963 >> 172.21.15.159:0/895841963 conn(0x7fcd48067460 msgr2=0x7fcd4815c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:14.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.604+0000 7fcd47e00640 1 -- 172.21.15.159:0/895841963 shutdown_connections 2024-04-04T06:17:14.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.604+0000 7fcd47e00640 1 -- 172.21.15.159:0/895841963 wait complete. 2024-04-04T06:17:14.618 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:17:14.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.828+0000 7fbfd6600640 1 Processor -- start 2024-04-04T06:17:14.945 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.828+0000 7fbfd6600640 1 -- start start 2024-04-04T06:17:14.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.829+0000 7fbfd6600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbfd00d06c0 0x7fbfd0169260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:14.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.830+0000 7fbfd6600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfd006dfa0 con 0x7fbfd00d43d0 2024-04-04T06:17:14.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.830+0000 7fbfd6600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfd006e240 con 0x7fbfd015e860 2024-04-04T06:17:14.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.830+0000 7fbfd6600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fbfd00cf220 con 0x7fbfd00d06c0 2024-04-04T06:17:14.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.830+0000 7fbfcea00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fbfd00d43d0 0x7fbfd00d47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:40992/0 (socket says 172.21.15.159:40992) 2024-04-04T06:17:14.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.830+0000 7fbfcea00640 1 -- 172.21.15.159:0/3008772316 learned_addr learned my addr 172.21.15.159:0/3008772316 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:14.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.830+0000 7fbfcfe00640 1 --2- 172.21.15.159:0/3008772316 >> v2:172.21.15.178:3300/0 conn(0x7fbfd00d06c0 0x7fbfd0169260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:14.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.830+0000 7fbfce000640 1 -- 172.21.15.159:0/3008772316 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1137199567 0 0) 0x7fbfd006dfa0 con 0x7fbfd00d43d0 2024-04-04T06:17:14.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.830+0000 7fbfce000640 1 -- 172.21.15.159:0/3008772316 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfb4003520 con 0x7fbfd00d43d0 2024-04-04T06:17:14.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.831+0000 7fbfce000640 1 -- 172.21.15.159:0/3008772316 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2606468753 0 0) 0x7fbfb4003520 con 0x7fbfd00d43d0 2024-04-04T06:17:14.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.831+0000 7fbfce000640 1 -- 172.21.15.159:0/3008772316 >> v2:172.21.15.178:3300/0 conn(0x7fbfd00d06c0 msgr2=0x7fbfd0169260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:14.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.831+0000 7fbfce000640 1 --2- 172.21.15.159:0/3008772316 >> v2:172.21.15.178:3300/0 conn(0x7fbfd00d06c0 0x7fbfd0169260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:14.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.831+0000 7fbfce000640 1 -- 172.21.15.159:0/3008772316 >> v1:172.21.15.178:6789/0 conn(0x7fbfd015e860 legacy=0x7fbfd006d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:14.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.831+0000 7fbfce000640 1 -- 172.21.15.159:0/3008772316 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd006e520 con 0x7fbfd00d43d0 2024-04-04T06:17:14.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.831+0000 7fbfce000640 1 -- 172.21.15.159:0/3008772316 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fbfbc001690 con 0x7fbfd00d43d0 2024-04-04T06:17:14.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.831+0000 7fbfce000640 1 -- 172.21.15.159:0/3008772316 >> v1:172.21.15.159:6789/0 conn(0x7fbfd00d43d0 legacy=0x7fbfd00d47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:14.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.831+0000 7fbfce000640 1 --2- 172.21.15.159:0/3008772316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb40039b0 0x7fbfb4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:14.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.831+0000 7fbfce000640 1 --2- 172.21.15.159:0/3008772316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfb4004800 0x7fbfb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:14.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.831+0000 7fbfce000640 1 --2- 172.21.15.159:0/3008772316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfb40051c0 0x7fbfb400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:14.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.831+0000 7fbfce000640 1 -- 172.21.15.159:0/3008772316 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd00cf220 con 0x7fbfb4004800 2024-04-04T06:17:14.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.831+0000 7fbfce000640 1 -- 172.21.15.159:0/3008772316 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb400c110 con 0x7fbfb40039b0 2024-04-04T06:17:14.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.831+0000 7fbfce000640 1 -- 172.21.15.159:0/3008772316 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbfb400c310 con 0x7fbfb40051c0 2024-04-04T06:17:14.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.831+0000 7fbfcea00640 1 --2- 172.21.15.159:0/3008772316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfb4004800 0x7fbfb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:14.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.831+0000 7fbfcf400640 1 --2- 172.21.15.159:0/3008772316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb40039b0 0x7fbfb4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:14.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.832+0000 7fbfcea00640 1 -- 172.21.15.159:0/3008772316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfb40051c0 msgr2=0x7fbfb400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:14.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.832+0000 7fbfcea00640 1 --2- 172.21.15.159:0/3008772316 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfb40051c0 0x7fbfb400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:14.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.832+0000 7fbfcea00640 1 -- 172.21.15.159:0/3008772316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb40039b0 msgr2=0x7fbfb4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:14.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.832+0000 7fbfcea00640 1 --2- 172.21.15.159:0/3008772316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb40039b0 0x7fbfb4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:14.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.832+0000 7fbfcea00640 1 -- 172.21.15.159:0/3008772316 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb4003520 con 0x7fbfb4004800 2024-04-04T06:17:14.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.832+0000 7fbfcf400640 1 --2- 172.21.15.159:0/3008772316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb40039b0 0x7fbfb4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:14.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.832+0000 7fbfcfe00640 1 -- 172.21.15.159:0/3008772316 reap_dead start 2024-04-04T06:17:14.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.833+0000 7fbfcea00640 1 --2- 172.21.15.159:0/3008772316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfb4004800 0x7fbfb4004c80 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00c210 tx=0x7fbfb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2c40cc87f6c59c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:14.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.833+0000 7fbfce000640 1 -- 172.21.15.159:0/3008772316 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc007d10 con 0x7fbfb4004800 2024-04-04T06:17:14.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.833+0000 7fbfce000640 1 -- 172.21.15.159:0/3008772316 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfbc030070 con 0x7fbfb4004800 2024-04-04T06:17:14.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.833+0000 7fbfce000640 1 -- 172.21.15.159:0/3008772316 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc005d70 con 0x7fbfb4004800 2024-04-04T06:17:14.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.833+0000 7fbfd6600640 1 -- 172.21.15.159:0/3008772316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfb4004800 msgr2=0x7fbfb4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:14.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.833+0000 7fbfd6600640 1 --2- 172.21.15.159:0/3008772316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfb4004800 0x7fbfb4004c80 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00c210 tx=0x7fbfb4005830 comp rx=0 tx=0).stop 2024-04-04T06:17:14.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.834+0000 7fbfd6600640 1 -- 172.21.15.159:0/3008772316 shutdown_connections 2024-04-04T06:17:14.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.834+0000 7fbfd6600640 1 --2- 172.21.15.159:0/3008772316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfb4004800 0x7fbfb4004c80 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:14.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.834+0000 7fbfd6600640 1 -- 172.21.15.159:0/3008772316 >> 172.21.15.159:0/3008772316 conn(0x7fbfd009d7d0 msgr2=0x7fbfd008bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:14.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.834+0000 7fbfd6600640 1 -- 172.21.15.159:0/3008772316 shutdown_connections 2024-04-04T06:17:14.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.834+0000 7fbfd6600640 1 -- 172.21.15.159:0/3008772316 wait complete. 2024-04-04T06:17:14.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.835+0000 7fbfd6600640 1 Processor -- start 2024-04-04T06:17:14.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.835+0000 7fbfd6600640 1 -- start start 2024-04-04T06:17:14.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.836+0000 7fbfd6600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfb4004800 0x7fbfd00cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:14.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.836+0000 7fbfd6600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfd00b8520 0x7fbfd00cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:14.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.836+0000 7fbfd6600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfd0154650 0x7fbfd0154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:14.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.836+0000 7fbfd6600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd00cef30 con 0x7fbfb4004800 2024-04-04T06:17:14.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.836+0000 7fbfd6600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd00cedb0 con 0x7fbfd0154650 2024-04-04T06:17:14.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.836+0000 7fbfd6600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd00d47e0 con 0x7fbfd00b8520 2024-04-04T06:17:14.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.836+0000 7fbfcf400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfb4004800 0x7fbfd00cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:14.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.837+0000 7fbfcf400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfb4004800 0x7fbfd00cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37284/0 (socket says 172.21.15.159:37284) 2024-04-04T06:17:14.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.837+0000 7fbfcf400640 1 -- 172.21.15.159:0/3621995551 learned_addr learned my addr 172.21.15.159:0/3621995551 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:14.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.837+0000 7fbfcea00640 1 --2- 172.21.15.159:0/3621995551 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfd00b8520 0x7fbfd00cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:14.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.837+0000 7fbfcf400640 1 -- 172.21.15.159:0/3621995551 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfd00b8520 msgr2=0x7fbfd00cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:14.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.837+0000 7fbfcf400640 1 --2- 172.21.15.159:0/3621995551 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfd00b8520 0x7fbfd00cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:14.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.837+0000 7fbfcf400640 1 -- 172.21.15.159:0/3621995551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfd0154650 msgr2=0x7fbfd0154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:14.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.837+0000 7fbfcf400640 1 --2- 172.21.15.159:0/3621995551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfd0154650 0x7fbfd0154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:14.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.837+0000 7fbfcf400640 1 -- 172.21.15.159:0/3621995551 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd00b8980 con 0x7fbfb4004800 2024-04-04T06:17:14.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.837+0000 7fbfcf400640 1 --2- 172.21.15.159:0/3621995551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfb4004800 0x7fbfd00cbf20 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000d480 tx=0x7fbfc000d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:14.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.837+0000 7fbfc7400640 1 -- 172.21.15.159:0/3621995551 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0005480 con 0x7fbfb4004800 2024-04-04T06:17:14.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.838+0000 7fbfc7400640 1 -- 172.21.15.159:0/3621995551 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc0017070 con 0x7fbfb4004800 2024-04-04T06:17:14.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.838+0000 7fbfc7400640 1 -- 172.21.15.159:0/3621995551 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0003830 con 0x7fbfb4004800 2024-04-04T06:17:14.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.838+0000 7fbfd6600640 1 -- 172.21.15.159:0/3621995551 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd0154ff0 con 0x7fbfb4004800 2024-04-04T06:17:14.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.838+0000 7fbfd6600640 1 -- 172.21.15.159:0/3621995551 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd00cc900 con 0x7fbfb4004800 2024-04-04T06:17:14.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.839+0000 7fbfd6600640 1 -- 172.21.15.159:0/3621995551 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd00685c0 con 0x7fbfb4004800 2024-04-04T06:17:14.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.840+0000 7fbfc7400640 1 -- 172.21.15.159:0/3621995551 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbfc00039d0 con 0x7fbfb4004800 2024-04-04T06:17:14.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.841+0000 7fbfc7400640 1 --2- 172.21.15.159:0/3621995551 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbfb006c8f0 0x7fbfb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:14.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.841+0000 7fbfc7400640 1 -- 172.21.15.159:0/3621995551 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7fbfc008b900 con 0x7fbfb4004800 2024-04-04T06:17:14.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.841+0000 7fbfcea00640 1 --2- 172.21.15.159:0/3621995551 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbfb006c8f0 0x7fbfb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:14.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.842+0000 7fbfcea00640 1 --2- 172.21.15.159:0/3621995551 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbfb006c8f0 0x7fbfb006edb0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc034000 tx=0x7fbfbc039040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:14.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:14.845+0000 7fbfc7400640 1 -- 172.21.15.159:0/3621995551 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbfc0059e80 con 0x7fbfb4004800 2024-04-04T06:17:15.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.043+0000 7fbfd6600640 1 -- 172.21.15.159:0/3621995551 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbfd006cbd0 con 0x7fbfb4004800 2024-04-04T06:17:15.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.044+0000 7fbfc7400640 1 -- 172.21.15.159:0/3621995551 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) ==== 100+0+11 (secure 0 0 0) 0x7fbfc0059780 con 0x7fbfb4004800 2024-04-04T06:17:15.039 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:17:15.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.046+0000 7fbfd6600640 1 -- 172.21.15.159:0/3621995551 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbfb006c8f0 msgr2=0x7fbfb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:15.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.046+0000 7fbfd6600640 1 --2- 172.21.15.159:0/3621995551 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbfb006c8f0 0x7fbfb006edb0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc034000 tx=0x7fbfbc039040 comp rx=0 tx=0).stop 2024-04-04T06:17:15.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.046+0000 7fbfd6600640 1 -- 172.21.15.159:0/3621995551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfb4004800 msgr2=0x7fbfd00cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:15.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.046+0000 7fbfd6600640 1 --2- 172.21.15.159:0/3621995551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfb4004800 0x7fbfd00cbf20 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000d480 tx=0x7fbfc000d940 comp rx=0 tx=0).stop 2024-04-04T06:17:15.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.046+0000 7fbfd6600640 1 -- 172.21.15.159:0/3621995551 shutdown_connections 2024-04-04T06:17:15.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.046+0000 7fbfd6600640 1 --2- 172.21.15.159:0/3621995551 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbfb006c8f0 0x7fbfb006edb0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.046+0000 7fbfd6600640 1 --2- 172.21.15.159:0/3621995551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfd0154650 0x7fbfd0154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.046+0000 7fbfd6600640 1 --2- 172.21.15.159:0/3621995551 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfd00b8520 0x7fbfd00cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.047+0000 7fbfd6600640 1 --2- 172.21.15.159:0/3621995551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfb4004800 0x7fbfd00cbf20 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.047+0000 7fbfd6600640 1 -- 172.21.15.159:0/3621995551 >> 172.21.15.159:0/3621995551 conn(0x7fbfd009d7d0 msgr2=0x7fbfd00b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:15.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.047+0000 7fbfd6600640 1 -- 172.21.15.159:0/3621995551 shutdown_connections 2024-04-04T06:17:15.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.047+0000 7fbfd6600640 1 -- 172.21.15.159:0/3621995551 wait complete. 2024-04-04T06:17:15.060 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-04T06:17:15.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.258+0000 7f0757000640 1 Processor -- start 2024-04-04T06:17:15.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.258+0000 7f0757000640 1 -- start start 2024-04-04T06:17:15.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.260+0000 7f0757000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0750153580 0x7f07500d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.260+0000 7f0757000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07500af4b0 0x7f07501600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.260+0000 7f0757000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07500afab0 con 0x7f07500d07c0 2024-04-04T06:17:15.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.260+0000 7f0757000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f07500d0c30 con 0x7f0750153580 2024-04-04T06:17:15.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.260+0000 7f0757000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f075015cc80 con 0x7f07500af4b0 2024-04-04T06:17:15.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.260+0000 7f074fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0750153580 0x7f07500d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:15.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.260+0000 7f074fe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f0750153580 0x7f07500d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37298/0 (socket says 172.21.15.159:37298) 2024-04-04T06:17:15.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.260+0000 7f074fe00640 1 -- 172.21.15.159:0/3217340830 learned_addr learned my addr 172.21.15.159:0/3217340830 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:15.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.261+0000 7f0755000640 1 --2- 172.21.15.159:0/3217340830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07500af4b0 0x7f07501600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:15.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.261+0000 7f074fe00640 1 -- 172.21.15.159:0/3217340830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07500af4b0 msgr2=0x7f07501600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:15.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.261+0000 7f074fe00640 1 --2- 172.21.15.159:0/3217340830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07500af4b0 0x7f07501600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.261+0000 7f074fe00640 1 -- 172.21.15.159:0/3217340830 >> v1:172.21.15.178:6789/0 conn(0x7f07500d07c0 legacy=0x7f07500aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:15.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.261+0000 7f074fe00640 1 -- 172.21.15.159:0/3217340830 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075015da00 con 0x7f0750153580 2024-04-04T06:17:15.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.262+0000 7f074fe00640 1 --2- 172.21.15.159:0/3217340830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0750153580 0x7f07500d0280 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f0740009e50 tx=0x7f074002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8780eaa0235608c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:15.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.262+0000 7f074ea00640 1 -- 172.21.15.159:0/3217340830 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074002ca80 con 0x7f0750153580 2024-04-04T06:17:15.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.262+0000 7f074ea00640 1 -- 172.21.15.159:0/3217340830 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f074002cc20 con 0x7f0750153580 2024-04-04T06:17:15.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.262+0000 7f074ea00640 1 -- 172.21.15.159:0/3217340830 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07400343f0 con 0x7f0750153580 2024-04-04T06:17:15.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.263+0000 7f0757000640 1 -- 172.21.15.159:0/3217340830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0750153580 msgr2=0x7f07500d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:15.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.263+0000 7f0757000640 1 --2- 172.21.15.159:0/3217340830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0750153580 0x7f07500d0280 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f0740009e50 tx=0x7f074002bee0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.263+0000 7f0757000640 1 -- 172.21.15.159:0/3217340830 shutdown_connections 2024-04-04T06:17:15.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.263+0000 7f0757000640 1 --2- 172.21.15.159:0/3217340830 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07500af4b0 0x7f07501600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.263+0000 7f0757000640 1 --2- 172.21.15.159:0/3217340830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0750153580 0x7f07500d0280 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.263+0000 7f0757000640 1 -- 172.21.15.159:0/3217340830 >> 172.21.15.159:0/3217340830 conn(0x7f075006b750 msgr2=0x7f075006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:15.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.263+0000 7f0757000640 1 -- 172.21.15.159:0/3217340830 shutdown_connections 2024-04-04T06:17:15.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.263+0000 7f0757000640 1 -- 172.21.15.159:0/3217340830 wait complete. 2024-04-04T06:17:15.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.265+0000 7f0757000640 1 Processor -- start 2024-04-04T06:17:15.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.265+0000 7f0757000640 1 -- start start 2024-04-04T06:17:15.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.266+0000 7f0757000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07500af4b0 0x7f0750151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.266+0000 7f0757000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07500d07c0 0x7f07500ce610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.266+0000 7f0757000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0750153580 0x7f07500afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.266+0000 7f0757000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f075015d4e0 con 0x7f0750153580 2024-04-04T06:17:15.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.266+0000 7f0757000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f075015d200 con 0x7f07500d07c0 2024-04-04T06:17:15.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.266+0000 7f0757000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0750153960 con 0x7f07500af4b0 2024-04-04T06:17:15.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.266+0000 7f0755000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0750153580 0x7f07500afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:15.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.266+0000 7f0755000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0750153580 0x7f07500afca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37312/0 (socket says 172.21.15.159:37312) 2024-04-04T06:17:15.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.266+0000 7f0755000640 1 -- 172.21.15.159:0/2998148653 learned_addr learned my addr 172.21.15.159:0/2998148653 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:15.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.267+0000 7f0755000640 1 -- 172.21.15.159:0/2998148653 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07500af4b0 msgr2=0x7f0750151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:15.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.267+0000 7f0755000640 1 --2- 172.21.15.159:0/2998148653 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07500af4b0 0x7f0750151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.267+0000 7f0755000640 1 -- 172.21.15.159:0/2998148653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07500d07c0 msgr2=0x7f07500ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:15.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.267+0000 7f0755000640 1 --2- 172.21.15.159:0/2998148653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07500d07c0 0x7f07500ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.267+0000 7f0755000640 1 -- 172.21.15.159:0/2998148653 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0750158120 con 0x7f0750153580 2024-04-04T06:17:15.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.267+0000 7f0755000640 1 --2- 172.21.15.159:0/2998148653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0750153580 0x7f07500afca0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f074800d600 tx=0x7f074800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:15.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.267+0000 7f0747e00640 1 -- 172.21.15.159:0/2998148653 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07480125f0 con 0x7f0750153580 2024-04-04T06:17:15.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.267+0000 7f0747e00640 1 -- 172.21.15.159:0/2998148653 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0748019070 con 0x7f0750153580 2024-04-04T06:17:15.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.267+0000 7f0747e00640 1 -- 172.21.15.159:0/2998148653 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07480129f0 con 0x7f0750153580 2024-04-04T06:17:15.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.267+0000 7f0757000640 1 -- 172.21.15.159:0/2998148653 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0750158400 con 0x7f0750153580 2024-04-04T06:17:15.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.267+0000 7f0757000640 1 -- 172.21.15.159:0/2998148653 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07501589c0 con 0x7f0750153580 2024-04-04T06:17:15.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.269+0000 7f0757000640 1 -- 172.21.15.159:0/2998148653 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0750158df0 con 0x7f0750153580 2024-04-04T06:17:15.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.270+0000 7f0747e00640 1 -- 172.21.15.159:0/2998148653 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0748012b90 con 0x7f0750153580 2024-04-04T06:17:15.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.271+0000 7f0747e00640 1 --2- 172.21.15.159:0/2998148653 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f073006c8f0 0x7f073006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.271+0000 7f0747e00640 1 -- 172.21.15.159:0/2998148653 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f074808bec0 con 0x7f0750153580 2024-04-04T06:17:15.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.271+0000 7f074fe00640 1 --2- 172.21.15.159:0/2998148653 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f073006c8f0 0x7f073006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:15.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.273+0000 7f074fe00640 1 --2- 172.21.15.159:0/2998148653 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f073006c8f0 0x7f073006edb0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0740035000 tx=0x7f074002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:15.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.277+0000 7f0747e00640 1 -- 172.21.15.159:0/2998148653 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f074805a300 con 0x7f0750153580 2024-04-04T06:17:15.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.490+0000 7f0757000640 1 -- 172.21.15.159:0/2998148653 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f07500b5080 con 0x7f0750153580 2024-04-04T06:17:15.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.491+0000 7f0747e00640 1 -- 172.21.15.159:0/2998148653 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v113) ==== 80+0+0 (secure 0 0 0) 0x7f0748077e20 con 0x7f0750153580 2024-04-04T06:17:15.487 INFO:teuthology.orchestra.run.smithi159.stderr:listed 0 entries 2024-04-04T06:17:15.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.493+0000 7f0757000640 1 -- 172.21.15.159:0/2998148653 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f073006c8f0 msgr2=0x7f073006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:15.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.493+0000 7f0757000640 1 --2- 172.21.15.159:0/2998148653 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f073006c8f0 0x7f073006edb0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0740035000 tx=0x7f074002e000 comp rx=0 tx=0).stop 2024-04-04T06:17:15.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.493+0000 7f0757000640 1 -- 172.21.15.159:0/2998148653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0750153580 msgr2=0x7f07500afca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:15.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.493+0000 7f0757000640 1 --2- 172.21.15.159:0/2998148653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0750153580 0x7f07500afca0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f074800d600 tx=0x7f074800dac0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.494+0000 7f0757000640 1 -- 172.21.15.159:0/2998148653 shutdown_connections 2024-04-04T06:17:15.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.494+0000 7f0757000640 1 --2- 172.21.15.159:0/2998148653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0750153580 0x7f07500afca0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.494+0000 7f0757000640 1 --2- 172.21.15.159:0/2998148653 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f073006c8f0 0x7f073006edb0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.494+0000 7f0757000640 1 --2- 172.21.15.159:0/2998148653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07500d07c0 0x7f07500ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.494+0000 7f0757000640 1 --2- 172.21.15.159:0/2998148653 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07500af4b0 0x7f0750151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.494+0000 7f0757000640 1 -- 172.21.15.159:0/2998148653 >> 172.21.15.159:0/2998148653 conn(0x7f075006b750 msgr2=0x7f07500b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:15.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.494+0000 7f0757000640 1 -- 172.21.15.159:0/2998148653 shutdown_connections 2024-04-04T06:17:15.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.494+0000 7f0757000640 1 -- 172.21.15.159:0/2998148653 wait complete. 2024-04-04T06:17:15.507 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-04T06:17:15.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:15.507 DEBUG:teuthology.orchestra.run.smithi159:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-04T06:17:15.626 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:17:15.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:15.626 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-04T06:17:15.683 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-04T06:17:15.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.859+0000 7f817b800640 1 Processor -- start 2024-04-04T06:17:15.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.859+0000 7f817b800640 1 -- start start 2024-04-04T06:17:15.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.860+0000 7f817b800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f81740d0990 0x7f81740aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.860+0000 7f817b800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81740afab0 con 0x7f81740af2e0 2024-04-04T06:17:15.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.860+0000 7f817b800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f817415cc80 con 0x7f8174153580 2024-04-04T06:17:15.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.860+0000 7f817b800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f817415da00 con 0x7f81740d0990 2024-04-04T06:17:15.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.861+0000 7f8179800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f81740af2e0 0x7f81741600d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41004/0 (socket says 172.21.15.159:41004) 2024-04-04T06:17:15.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.861+0000 7f8179800640 1 -- 172.21.15.159:0/3162175728 learned_addr learned my addr 172.21.15.159:0/3162175728 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:15.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.861+0000 7f8173e00640 1 --2- 172.21.15.159:0/3162175728 >> v2:172.21.15.178:3300/0 conn(0x7f81740d0990 0x7f81740aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:15.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.861+0000 7f8173400640 1 -- 172.21.15.159:0/3162175728 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 941515278 0 0) 0x7f81740afab0 con 0x7f81740af2e0 2024-04-04T06:17:15.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.861+0000 7f8173400640 1 -- 172.21.15.159:0/3162175728 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f815c003520 con 0x7f81740af2e0 2024-04-04T06:17:15.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.861+0000 7f8173400640 1 -- 172.21.15.159:0/3162175728 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3534785339 0 0) 0x7f815c003520 con 0x7f81740af2e0 2024-04-04T06:17:15.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.861+0000 7f8173400640 1 -- 172.21.15.159:0/3162175728 >> v2:172.21.15.178:3300/0 conn(0x7f81740d0990 msgr2=0x7f81740aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:15.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.861+0000 7f8173400640 1 --2- 172.21.15.159:0/3162175728 >> v2:172.21.15.178:3300/0 conn(0x7f81740d0990 0x7f81740aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.861+0000 7f8173400640 1 -- 172.21.15.159:0/3162175728 >> v1:172.21.15.178:6789/0 conn(0x7f8174153580 legacy=0x7f81740d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:15.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.861+0000 7f8173400640 1 -- 172.21.15.159:0/3162175728 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817415cf20 con 0x7f81740af2e0 2024-04-04T06:17:15.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.861+0000 7f8173400640 1 -- 172.21.15.159:0/3162175728 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f816c0019f0 con 0x7f81740af2e0 2024-04-04T06:17:15.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.862+0000 7f8173400640 1 -- 172.21.15.159:0/3162175728 >> v1:172.21.15.159:6789/0 conn(0x7f81740af2e0 legacy=0x7f81741600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:15.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.862+0000 7f8173400640 1 --2- 172.21.15.159:0/3162175728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f815c0039b0 0x7f815c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.862+0000 7f8173400640 1 --2- 172.21.15.159:0/3162175728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f815c004800 0x7f815c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.862+0000 7f8173400640 1 --2- 172.21.15.159:0/3162175728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f815c0051c0 0x7f815c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.862+0000 7f8173400640 1 -- 172.21.15.159:0/3162175728 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f817415da00 con 0x7f815c004800 2024-04-04T06:17:15.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.862+0000 7f8173400640 1 -- 172.21.15.159:0/3162175728 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f815c00c160 con 0x7f815c0039b0 2024-04-04T06:17:15.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.862+0000 7f8173400640 1 -- 172.21.15.159:0/3162175728 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f815c00c360 con 0x7f815c0051c0 2024-04-04T06:17:15.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.862+0000 7f8173e00640 1 --2- 172.21.15.159:0/3162175728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f815c004800 0x7f815c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:15.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.862+0000 7f8173e00640 1 -- 172.21.15.159:0/3162175728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f815c0051c0 msgr2=0x7f815c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:15.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.862+0000 7f8173e00640 1 --2- 172.21.15.159:0/3162175728 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f815c0051c0 0x7f815c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.862+0000 7f8173e00640 1 -- 172.21.15.159:0/3162175728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f815c0039b0 msgr2=0x7f815c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:15.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.862+0000 7f8173e00640 1 --2- 172.21.15.159:0/3162175728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f815c0039b0 0x7f815c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.862+0000 7f8173e00640 1 -- 172.21.15.159:0/3162175728 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815c003520 con 0x7f815c004800 2024-04-04T06:17:15.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.862+0000 7f8179800640 1 -- 172.21.15.159:0/3162175728 reap_dead start 2024-04-04T06:17:15.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.863+0000 7f8173e00640 1 --2- 172.21.15.159:0/3162175728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f815c004800 0x7f815c004c80 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f817406f820 tx=0x7f815c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be73c4c64b2f1e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:15.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.864+0000 7f8173400640 1 -- 172.21.15.159:0/3162175728 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160039070 con 0x7f815c004800 2024-04-04T06:17:15.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.864+0000 7f8173400640 1 -- 172.21.15.159:0/3162175728 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f816002c660 con 0x7f815c004800 2024-04-04T06:17:15.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.864+0000 7f8173400640 1 -- 172.21.15.159:0/3162175728 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160009690 con 0x7f815c004800 2024-04-04T06:17:15.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.864+0000 7f817b800640 1 -- 172.21.15.159:0/3162175728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f815c004800 msgr2=0x7f815c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:15.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.864+0000 7f817b800640 1 --2- 172.21.15.159:0/3162175728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f815c004800 0x7f815c004c80 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f817406f820 tx=0x7f815c005830 comp rx=0 tx=0).stop 2024-04-04T06:17:15.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.865+0000 7f817b800640 1 -- 172.21.15.159:0/3162175728 shutdown_connections 2024-04-04T06:17:15.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.865+0000 7f817b800640 1 --2- 172.21.15.159:0/3162175728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f815c004800 0x7f815c004c80 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.865+0000 7f817b800640 1 -- 172.21.15.159:0/3162175728 >> 172.21.15.159:0/3162175728 conn(0x7f817406b750 msgr2=0x7f817406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:15.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.865+0000 7f817b800640 1 -- 172.21.15.159:0/3162175728 shutdown_connections 2024-04-04T06:17:15.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.865+0000 7f817b800640 1 -- 172.21.15.159:0/3162175728 wait complete. 2024-04-04T06:17:15.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.867+0000 7f817b800640 1 Processor -- start 2024-04-04T06:17:15.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.868+0000 7f817b800640 1 -- start start 2024-04-04T06:17:15.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.869+0000 7f817b800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f815c004800 0x7f8174152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.869+0000 7f817b800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81740ce4d0 0x7f8174072990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.869+0000 7f817b800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81740ce950 0x7f817409b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.869+0000 7f817b800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8174153960 con 0x7f81740ce4d0 2024-04-04T06:17:15.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.869+0000 7f817b800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f817415d4e0 con 0x7f81740ce950 2024-04-04T06:17:15.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.869+0000 7f817b800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f81740af6c0 con 0x7f815c004800 2024-04-04T06:17:15.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.869+0000 7f8173e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81740ce4d0 0x7f8174072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:15.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.869+0000 7f8173e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81740ce4d0 0x7f8174072990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37338/0 (socket says 172.21.15.159:37338) 2024-04-04T06:17:15.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.869+0000 7f8173e00640 1 -- 172.21.15.159:0/2620087525 learned_addr learned my addr 172.21.15.159:0/2620087525 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:15.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.870+0000 7f8173e00640 1 -- 172.21.15.159:0/2620087525 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f815c004800 msgr2=0x7f8174152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:15.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.870+0000 7f8173e00640 1 --2- 172.21.15.159:0/2620087525 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f815c004800 0x7f8174152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.870+0000 7f8173e00640 1 -- 172.21.15.159:0/2620087525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81740ce950 msgr2=0x7f817409b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:15.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.870+0000 7f8173e00640 1 --2- 172.21.15.159:0/2620087525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81740ce950 0x7f817409b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.870+0000 7f8173e00640 1 -- 172.21.15.159:0/2620087525 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81741f8820 con 0x7f81740ce4d0 2024-04-04T06:17:15.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.870+0000 7f8173e00640 1 --2- 172.21.15.159:0/2620087525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81740ce4d0 0x7f8174072990 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f8160037040 tx=0x7f8160004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:15.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.870+0000 7f8170c00640 1 -- 172.21.15.159:0/2620087525 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160042020 con 0x7f81740ce4d0 2024-04-04T06:17:15.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.870+0000 7f8170c00640 1 -- 172.21.15.159:0/2620087525 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f816003a040 con 0x7f81740ce4d0 2024-04-04T06:17:15.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.870+0000 7f8170c00640 1 -- 172.21.15.159:0/2620087525 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160039070 con 0x7f81740ce4d0 2024-04-04T06:17:15.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.870+0000 7f817b800640 1 -- 172.21.15.159:0/2620087525 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8174071610 con 0x7f81740ce4d0 2024-04-04T06:17:15.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.871+0000 7f817b800640 1 -- 172.21.15.159:0/2620087525 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8174071bd0 con 0x7f81740ce4d0 2024-04-04T06:17:15.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.873+0000 7f8170c00640 1 -- 172.21.15.159:0/2620087525 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8160045020 con 0x7f81740ce4d0 2024-04-04T06:17:15.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.874+0000 7f8170c00640 1 --2- 172.21.15.159:0/2620087525 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f815406c920 0x7f815406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.868 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.874+0000 7f8170c00640 1 -- 172.21.15.159:0/2620087525 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f81600acf90 con 0x7f81740ce4d0 2024-04-04T06:17:15.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.874+0000 7f8178e00640 1 --2- 172.21.15.159:0/2620087525 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f815406c920 0x7f815406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:15.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.875+0000 7f8178e00640 1 --2- 172.21.15.159:0/2620087525 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f815406c920 0x7f815406ede0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f817415f1b0 tx=0x7f81680079b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:15.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.876+0000 7f817e5dd740 1 Processor -- start 2024-04-04T06:17:15.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.876+0000 7f817e5dd740 1 -- start start 2024-04-04T06:17:15.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.877+0000 7f817e5dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c0125520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.877+0000 7f817e5dd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1c0152960 0x55b1c0155310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.877+0000 7f817e5dd740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b1c0125a60 0x55b1c0155850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.877+0000 7f817e5dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b1c00906d0 con 0x55b1c0152580 2024-04-04T06:17:15.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.877+0000 7f817e5dd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b1c0038770 con 0x55b1c0152960 2024-04-04T06:17:15.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.877+0000 7f817e5dd740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55b1c008c680 con 0x55b1c0125a60 2024-04-04T06:17:15.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.878+0000 7f8178e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c0125520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:15.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.878+0000 7f8178e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c0125520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37346/0 (socket says 172.21.15.159:37346) 2024-04-04T06:17:15.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.878+0000 7f8178e00640 1 -- 172.21.15.159:0/3904635225 learned_addr learned my addr 172.21.15.159:0/3904635225 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:15.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.878+0000 7f8178e00640 1 -- 172.21.15.159:0/3904635225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b1c0125a60 msgr2=0x55b1c0155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:15.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.878+0000 7f8178e00640 1 --2- 172.21.15.159:0/3904635225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b1c0125a60 0x55b1c0155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.878+0000 7f8178e00640 1 -- 172.21.15.159:0/3904635225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1c0152960 msgr2=0x55b1c0155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:15.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.878+0000 7f8178e00640 1 --2- 172.21.15.159:0/3904635225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1c0152960 0x55b1c0155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.878+0000 7f8178e00640 1 -- 172.21.15.159:0/3904635225 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1bfe6e7d0 con 0x55b1c0152580 2024-04-04T06:17:15.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.878+0000 7f8178e00640 1 --2- 172.21.15.159:0/3904635225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c0125520 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f81680168d0 tx=0x7f816800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc1ecccc23dc592 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:15.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.879+0000 7f8165600640 1 -- 172.21.15.159:0/3904635225 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81680162e0 con 0x55b1c0152580 2024-04-04T06:17:15.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.879+0000 7f8165600640 1 -- 172.21.15.159:0/3904635225 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8168005110 con 0x55b1c0152580 2024-04-04T06:17:15.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.879+0000 7f8165600640 1 -- 172.21.15.159:0/3904635225 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81680138b0 con 0x55b1c0152580 2024-04-04T06:17:15.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.879+0000 7f817e5dd740 1 -- 172.21.15.159:0/3904635225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 msgr2=0x55b1c0125520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:15.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.879+0000 7f817e5dd740 1 --2- 172.21.15.159:0/3904635225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c0125520 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f81680168d0 tx=0x7f816800f000 comp rx=0 tx=0).stop 2024-04-04T06:17:15.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.879+0000 7f817e5dd740 1 -- 172.21.15.159:0/3904635225 shutdown_connections 2024-04-04T06:17:15.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.879+0000 7f817e5dd740 1 --2- 172.21.15.159:0/3904635225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b1c0125a60 0x55b1c0155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.879+0000 7f817e5dd740 1 --2- 172.21.15.159:0/3904635225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1c0152960 0x55b1c0155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.879+0000 7f817e5dd740 1 --2- 172.21.15.159:0/3904635225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c0125520 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.879+0000 7f817e5dd740 1 -- 172.21.15.159:0/3904635225 >> 172.21.15.159:0/3904635225 conn(0x55b1c0127790 msgr2=0x55b1c0170310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:15.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.879+0000 7f817e5dd740 1 -- 172.21.15.159:0/3904635225 shutdown_connections 2024-04-04T06:17:15.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.879+0000 7f817e5dd740 1 -- 172.21.15.159:0/3904635225 wait complete. 2024-04-04T06:17:15.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.880+0000 7f817e5dd740 1 -- start start 2024-04-04T06:17:15.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.880+0000 7f817e5dd740 1 Processor -- start 2024-04-04T06:17:15.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.881+0000 7f817e5dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c01807b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.881+0000 7f817e5dd740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b1c0152960 0x55b1c016f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.881+0000 7f817e5dd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1c0181b80 0x55b1c016fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.881+0000 7f817e5dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b1c0129e30 con 0x55b1c0152580 2024-04-04T06:17:15.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.881+0000 7f817e5dd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b1c004baa0 con 0x55b1c0181b80 2024-04-04T06:17:15.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.881+0000 7f817e5dd740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55b1c012c670 con 0x55b1c0152960 2024-04-04T06:17:15.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.881+0000 7f817e5dd740 10 client.0 _ll_register_callbacks cb 0x55b1c004b490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:17:15.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.881+0000 7f8178e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c01807b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:15.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.881+0000 7f8178e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c01807b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37348/0 (socket says 172.21.15.159:37348) 2024-04-04T06:17:15.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.882+0000 7f8178e00640 1 -- 172.21.15.159:0/2518863455 learned_addr learned my addr 172.21.15.159:0/2518863455 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:15.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.882+0000 7f8178e00640 1 -- 172.21.15.159:0/2518863455 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b1c0152960 msgr2=0x55b1c016f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:15.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.882+0000 7f8173e00640 1 --2- 172.21.15.159:0/2518863455 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b1c0152960 0x55b1c016f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:15.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.882+0000 7f8178e00640 1 --2- 172.21.15.159:0/2518863455 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b1c0152960 0x55b1c016f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.882+0000 7f8179800640 1 --2- 172.21.15.159:0/2518863455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1c0181b80 0x55b1c016fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:15.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.882+0000 7f8178e00640 1 -- 172.21.15.159:0/2518863455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1c0181b80 msgr2=0x55b1c016fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:15.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.882+0000 7f8178e00640 1 --2- 172.21.15.159:0/2518863455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1c0181b80 0x55b1c016fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.882+0000 7f8178e00640 1 -- 172.21.15.159:0/2518863455 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1bfda43f0 con 0x55b1c0152580 2024-04-04T06:17:15.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.882+0000 7f8178e00640 1 --2- 172.21.15.159:0/2518863455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c01807b0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f8168013780 tx=0x7f8168013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10285301ce2256ce server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:15.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.882+0000 7f815a000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:17:15.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.882+0000 7f815a000640 1 -- 172.21.15.159:0/2518863455 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168026070 con 0x55b1c0152580 2024-04-04T06:17:15.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.882+0000 7f815a000640 1 -- 172.21.15.159:0/2518863455 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f8168013cc0 con 0x55b1c0152580 2024-04-04T06:17:15.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.883+0000 7f817e5dd740 1 -- 172.21.15.159:0/2518863455 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8168005110 con 0x55b1c0152580 2024-04-04T06:17:15.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.883+0000 7f815a000640 1 -- 172.21.15.159:0/2518863455 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7f8168007050 con 0x55b1c0152580 2024-04-04T06:17:15.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.883+0000 7f817e5dd740 10 client.9078 fetch_fsmap learned FSMap version 101 2024-04-04T06:17:15.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.883+0000 7f817e5dd740 1 -- 172.21.15.159:0/2518863455 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) -- 0x55b1bfd640f0 con 0x55b1c0152580 2024-04-04T06:17:15.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.883+0000 7f815a000640 1 -- 172.21.15.159:0/2518863455 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8168005110 con 0x55b1c0152580 2024-04-04T06:17:15.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.883+0000 7f815a000640 1 -- 172.21.15.159:0/2518863455 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 101) ==== 3494+0+0 (secure 0 0 0) 0x7f81680071f0 con 0x55b1c0152580 2024-04-04T06:17:15.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.883+0000 7f817e5dd740 10 client.9078 fetch_fsmap finished waiting for FSMap version 101 2024-04-04T06:17:15.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.883+0000 7f817e5dd740 10 client.9078 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-04T06:17:15.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.883+0000 7f817e5dd740 20 client.9078 populate_metadata read hostname 'smithi159' 2024-04-04T06:17:15.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.883+0000 7f817e5dd740 1 --2- 172.21.15.159:0/2518863455 >> [v2:172.21.15.159:6832/1017894858,v1:172.21.15.159:6834/1017894858] conn(0x55b1c0180cf0 0x55b1c0158f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.883+0000 7f817e5dd740 4 client.9078 mds_command: new command op to 8853 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:17:15.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.883+0000 7f817e5dd740 1 -- 172.21.15.159:0/2518863455 --> [v2:172.21.15.159:6832/1017894858,v1:172.21.15.159:6834/1017894858] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1c009dc10 con 0x55b1c0180cf0 2024-04-04T06:17:15.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.884+0000 7f8173e00640 1 --2- 172.21.15.159:0/2518863455 >> [v2:172.21.15.159:6832/1017894858,v1:172.21.15.159:6834/1017894858] conn(0x55b1c0180cf0 0x55b1c0158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:15.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.884+0000 7f8173e00640 1 --2- 172.21.15.159:0/2518863455 >> [v2:172.21.15.159:6832/1017894858,v1:172.21.15.159:6834/1017894858] conn(0x55b1c0180cf0 0x55b1c0158f40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11eac1e04f12406d server_cookie=5f6dee8d56dac5e3 in_seq=0 out_seq=0 2024-04-04T06:17:15.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.884+0000 7f815a000640 10 client.9078 ms_handle_connect on v2:172.21.15.159:6832/1017894858 2024-04-04T06:17:15.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.885+0000 7f815a000640 1 -- 172.21.15.159:0/2518863455 <== mds.0 v2:172.21.15.159:6832/1017894858 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b1c009dc10 con 0x55b1c0180cf0 2024-04-04T06:17:15.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.885+0000 7f815a000640 10 client.9078 handle_command_reply: tid=0 2024-04-04T06:17:15.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.885+0000 7f817e5dd740 1 client.9078 shutdown 2024-04-04T06:17:15.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.885+0000 7f817e5dd740 1 -- 172.21.15.159:0/2518863455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 msgr2=0x55b1c01807b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:15.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.885+0000 7f817e5dd740 1 --2- 172.21.15.159:0/2518863455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c01807b0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f8168013780 tx=0x7f8168013000 comp rx=0 tx=0).stop 2024-04-04T06:17:15.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.885+0000 7f817e5dd740 1 -- 172.21.15.159:0/2518863455 shutdown_connections 2024-04-04T06:17:15.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.885+0000 7f817e5dd740 1 --2- 172.21.15.159:0/2518863455 >> [v2:172.21.15.159:6832/1017894858,v1:172.21.15.159:6834/1017894858] conn(0x55b1c0180cf0 0x55b1c0158f40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.885+0000 7f817e5dd740 1 --2- 172.21.15.159:0/2518863455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1c0181b80 0x55b1c016fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.885+0000 7f817e5dd740 1 --2- 172.21.15.159:0/2518863455 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b1c0152960 0x55b1c016f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.885+0000 7f817e5dd740 1 --2- 172.21.15.159:0/2518863455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c01807b0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.885+0000 7f815a000640 0 client.9078 ms_handle_reset on v2:172.21.15.159:6832/1017894858 2024-04-04T06:17:15.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.885+0000 7f817e5dd740 1 -- 172.21.15.159:0/2518863455 >> 172.21.15.159:0/2518863455 conn(0x55b1c0125a60 msgr2=0x55b1c0129530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:15.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.885+0000 7f817e5dd740 1 -- 172.21.15.159:0/2518863455 shutdown_connections 2024-04-04T06:17:15.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.885+0000 7f817e5dd740 1 -- 172.21.15.159:0/2518863455 wait complete. 2024-04-04T06:17:15.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.886+0000 7f817e5dd740 20 client.9078 trim_cache size 0 max 16384 2024-04-04T06:17:15.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.901+0000 7f817e5dd740 1 Processor -- start 2024-04-04T06:17:15.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.901+0000 7f817e5dd740 1 -- start start 2024-04-04T06:17:15.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.902+0000 7f817e5dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c01249d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.902+0000 7f817e5dd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1c0152960 0x55b1c016f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.902+0000 7f817e5dd740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b1c0181b80 0x55b1c016f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.902+0000 7f817e5dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f81742013e0 con 0x55b1c0152580 2024-04-04T06:17:15.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.902+0000 7f817e5dd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8174200440 con 0x55b1c0152960 2024-04-04T06:17:15.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.902+0000 7f817e5dd740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8174158950 con 0x55b1c0181b80 2024-04-04T06:17:15.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.903+0000 7f8178e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c01249d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:15.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.903+0000 7f8178e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c01249d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37356/0 (socket says 172.21.15.159:37356) 2024-04-04T06:17:15.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.903+0000 7f8178e00640 1 -- 172.21.15.159:0/1785633306 learned_addr learned my addr 172.21.15.159:0/1785633306 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:15.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.903+0000 7f8179800640 1 --2- 172.21.15.159:0/1785633306 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b1c0181b80 0x55b1c016f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:15.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.903+0000 7f8173e00640 1 --2- 172.21.15.159:0/1785633306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1c0152960 0x55b1c016f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:15.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.903+0000 7f8178e00640 1 -- 172.21.15.159:0/1785633306 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b1c0181b80 msgr2=0x55b1c016f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:15.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.903+0000 7f8178e00640 1 --2- 172.21.15.159:0/1785633306 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b1c0181b80 0x55b1c016f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.899 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.903+0000 7f8178e00640 1 -- 172.21.15.159:0/1785633306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1c0152960 msgr2=0x55b1c016f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:15.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.903+0000 7f8178e00640 1 --2- 172.21.15.159:0/1785633306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1c0152960 0x55b1c016f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.903+0000 7f8178e00640 1 -- 172.21.15.159:0/1785633306 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1c0154ff0 con 0x55b1c0152580 2024-04-04T06:17:15.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.903+0000 7f8173e00640 1 --2- 172.21.15.159:0/1785633306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1c0152960 0x55b1c016f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:15.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.903+0000 7f8178e00640 1 --2- 172.21.15.159:0/1785633306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c01249d0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f81680166d0 tx=0x7f8168026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c9aa99169a5ec8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:15.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.904+0000 7f8164c00640 1 -- 172.21.15.159:0/1785633306 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81680071f0 con 0x55b1c0152580 2024-04-04T06:17:15.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.904+0000 7f8164c00640 1 -- 172.21.15.159:0/1785633306 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81680050f0 con 0x55b1c0152580 2024-04-04T06:17:15.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.904+0000 7f8164c00640 1 -- 172.21.15.159:0/1785633306 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168013d00 con 0x55b1c0152580 2024-04-04T06:17:15.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.904+0000 7f817e5dd740 1 -- 172.21.15.159:0/1785633306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 msgr2=0x55b1c01249d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:15.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.904+0000 7f817e5dd740 1 --2- 172.21.15.159:0/1785633306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c01249d0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f81680166d0 tx=0x7f8168026000 comp rx=0 tx=0).stop 2024-04-04T06:17:15.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.905+0000 7f817e5dd740 1 -- 172.21.15.159:0/1785633306 shutdown_connections 2024-04-04T06:17:15.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.905+0000 7f817e5dd740 1 --2- 172.21.15.159:0/1785633306 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b1c0181b80 0x55b1c016f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.905+0000 7f817e5dd740 1 --2- 172.21.15.159:0/1785633306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1c0152960 0x55b1c016f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.905+0000 7f817e5dd740 1 --2- 172.21.15.159:0/1785633306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c01249d0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.905+0000 7f817e5dd740 1 -- 172.21.15.159:0/1785633306 >> 172.21.15.159:0/1785633306 conn(0x55b1c0125a60 msgr2=0x55b1c01805c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:15.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.905+0000 7f817e5dd740 1 -- 172.21.15.159:0/1785633306 shutdown_connections 2024-04-04T06:17:15.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.905+0000 7f817e5dd740 1 -- 172.21.15.159:0/1785633306 wait complete. 2024-04-04T06:17:15.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.905+0000 7f817e5dd740 1 -- start start 2024-04-04T06:17:15.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.905+0000 7f817e5dd740 1 Processor -- start 2024-04-04T06:17:15.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.906+0000 7f817e5dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c01734e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.906+0000 7f817e5dd740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b1c0152960 0x55b1c0173a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.906+0000 7f817e5dd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1c0181b80 0x55b1c016c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.906+0000 7f817e5dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f81741ffdf0 con 0x55b1c0152580 2024-04-04T06:17:15.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.906+0000 7f817e5dd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b1c016d7b0 con 0x55b1c0181b80 2024-04-04T06:17:15.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.906+0000 7f817e5dd740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55b1c01250e0 con 0x55b1c0152960 2024-04-04T06:17:15.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.906+0000 7f817e5dd740 10 client.0 _ll_register_callbacks cb 0x7f8168005430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:17:15.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.907+0000 7f8178e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c01734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:15.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.907+0000 7f8178e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c01734e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37358/0 (socket says 172.21.15.159:37358) 2024-04-04T06:17:15.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.907+0000 7f8178e00640 1 -- 172.21.15.159:0/1587866479 learned_addr learned my addr 172.21.15.159:0/1587866479 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:15.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.907+0000 7f8173e00640 1 --2- 172.21.15.159:0/1587866479 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b1c0152960 0x55b1c0173a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:15.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.907+0000 7f8178e00640 1 -- 172.21.15.159:0/1587866479 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b1c0152960 msgr2=0x55b1c0173a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:15.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.907+0000 7f8178e00640 1 --2- 172.21.15.159:0/1587866479 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b1c0152960 0x55b1c0173a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.907+0000 7f8178e00640 1 -- 172.21.15.159:0/1587866479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1c0181b80 msgr2=0x55b1c016c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:15.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.907+0000 7f8178e00640 1 --2- 172.21.15.159:0/1587866479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1c0181b80 0x55b1c016c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.907+0000 7f8178e00640 1 -- 172.21.15.159:0/1587866479 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1c01eeae0 con 0x55b1c0152580 2024-04-04T06:17:15.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.907+0000 7f8178e00640 1 --2- 172.21.15.159:0/1587866479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c01734e0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f8168014010 tx=0x7f8168012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3729f4886a7f04f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:15.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.907+0000 7f815a000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:17:15.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.908+0000 7f815a000640 1 -- 172.21.15.159:0/1587866479 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168026070 con 0x55b1c0152580 2024-04-04T06:17:15.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.908+0000 7f815a000640 1 -- 172.21.15.159:0/1587866479 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f81680067d0 con 0x55b1c0152580 2024-04-04T06:17:15.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.908+0000 7f817e5dd740 1 -- 172.21.15.159:0/1587866479 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81680050f0 con 0x55b1c0152580 2024-04-04T06:17:15.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.908+0000 7f815a000640 1 -- 172.21.15.159:0/1587866479 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7f816804d050 con 0x55b1c0152580 2024-04-04T06:17:15.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.908+0000 7f817e5dd740 10 client.9084 fetch_fsmap learned FSMap version 101 2024-04-04T06:17:15.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.908+0000 7f817e5dd740 1 -- 172.21.15.159:0/1587866479 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) -- 0x55b1c01ee340 con 0x55b1c0152580 2024-04-04T06:17:15.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.908+0000 7f815a000640 1 -- 172.21.15.159:0/1587866479 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81680050f0 con 0x55b1c0152580 2024-04-04T06:17:15.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.908+0000 7f815a000640 1 -- 172.21.15.159:0/1587866479 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 101) ==== 3494+0+0 (secure 0 0 0) 0x7f816803c030 con 0x55b1c0152580 2024-04-04T06:17:15.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.908+0000 7f817e5dd740 10 client.9084 fetch_fsmap finished waiting for FSMap version 101 2024-04-04T06:17:15.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.908+0000 7f817e5dd740 10 client.9084 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-04T06:17:15.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.908+0000 7f817e5dd740 20 client.9084 populate_metadata read hostname 'smithi159' 2024-04-04T06:17:15.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.908+0000 7f817e5dd740 1 --2- 172.21.15.159:0/1587866479 >> [v2:172.21.15.159:6832/1017894858,v1:172.21.15.159:6834/1017894858] conn(0x55b1c0167cc0 0x55b1c016c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:15.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.908+0000 7f817e5dd740 4 client.9084 mds_command: new command op to 8853 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-04T06:17:15.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.908+0000 7f817e5dd740 1 -- 172.21.15.159:0/1587866479 --> [v2:172.21.15.159:6832/1017894858,v1:172.21.15.159:6834/1017894858] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b1c0097f60 con 0x55b1c0167cc0 2024-04-04T06:17:15.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.909+0000 7f8173e00640 1 --2- 172.21.15.159:0/1587866479 >> [v2:172.21.15.159:6832/1017894858,v1:172.21.15.159:6834/1017894858] conn(0x55b1c0167cc0 0x55b1c016c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:15.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.909+0000 7f8173e00640 1 --2- 172.21.15.159:0/1587866479 >> [v2:172.21.15.159:6832/1017894858,v1:172.21.15.159:6834/1017894858] conn(0x55b1c0167cc0 0x55b1c016c570 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efa24aed5144f918 server_cookie=e12784914e463e00 in_seq=0 out_seq=0 2024-04-04T06:17:15.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.909+0000 7f815a000640 10 client.9084 ms_handle_connect on v2:172.21.15.159:6832/1017894858 2024-04-04T06:17:15.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.922+0000 7f815a000640 1 -- 172.21.15.159:0/1587866479 <== mds.0 v2:172.21.15.159:6832/1017894858 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b1c0097f60 con 0x55b1c0167cc0 2024-04-04T06:17:15.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.922+0000 7f815a000640 10 client.9084 handle_command_reply: tid=0 2024-04-04T06:17:15.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.922+0000 7f817e5dd740 1 client.9084 shutdown 2024-04-04T06:17:15.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.922+0000 7f817e5dd740 1 -- 172.21.15.159:0/1587866479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 msgr2=0x55b1c01734e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:15.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.922+0000 7f817e5dd740 1 --2- 172.21.15.159:0/1587866479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c01734e0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f8168014010 tx=0x7f8168012000 comp rx=0 tx=0).stop 2024-04-04T06:17:15.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.922+0000 7f817e5dd740 1 -- 172.21.15.159:0/1587866479 shutdown_connections 2024-04-04T06:17:15.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.922+0000 7f817e5dd740 1 --2- 172.21.15.159:0/1587866479 >> [v2:172.21.15.159:6832/1017894858,v1:172.21.15.159:6834/1017894858] conn(0x55b1c0167cc0 0x55b1c016c570 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.922+0000 7f817e5dd740 1 --2- 172.21.15.159:0/1587866479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1c0181b80 0x55b1c016c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.922+0000 7f815a000640 0 client.9084 ms_handle_reset on v2:172.21.15.159:6832/1017894858 2024-04-04T06:17:15.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.922+0000 7f817e5dd740 1 --2- 172.21.15.159:0/1587866479 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55b1c0152960 0x55b1c0173a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.922+0000 7f817e5dd740 1 --2- 172.21.15.159:0/1587866479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1c0152580 0x55b1c01734e0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.922+0000 7f817e5dd740 1 -- 172.21.15.159:0/1587866479 >> 172.21.15.159:0/1587866479 conn(0x55b1c0125a60 msgr2=0x55b1c01249d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:15.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.922+0000 7f817e5dd740 1 -- 172.21.15.159:0/1587866479 shutdown_connections 2024-04-04T06:17:15.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.922+0000 7f817e5dd740 1 -- 172.21.15.159:0/1587866479 wait complete. 2024-04-04T06:17:15.919 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.923+0000 7f817e5dd740 20 client.9084 trim_cache size 0 max 16384 2024-04-04T06:17:15.919 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:17:15.919 INFO:teuthology.orchestra.run.smithi159.stdout: "message": "", 2024-04-04T06:17:15.919 INFO:teuthology.orchestra.run.smithi159.stdout: "return_code": 0 2024-04-04T06:17:15.919 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:17:15.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.923+0000 7f817b800640 1 -- 172.21.15.159:0/2620087525 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f815406c920 msgr2=0x7f815406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:15.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.923+0000 7f817b800640 1 --2- 172.21.15.159:0/2620087525 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f815406c920 0x7f815406ede0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f817415f1b0 tx=0x7f81680079b0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.923+0000 7f817b800640 1 -- 172.21.15.159:0/2620087525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81740ce4d0 msgr2=0x7f8174072990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:15.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.923+0000 7f817b800640 1 --2- 172.21.15.159:0/2620087525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81740ce4d0 0x7f8174072990 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f8160037040 tx=0x7f8160004730 comp rx=0 tx=0).stop 2024-04-04T06:17:15.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.923+0000 7f817b800640 1 -- 172.21.15.159:0/2620087525 shutdown_connections 2024-04-04T06:17:15.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.923+0000 7f817b800640 1 --2- 172.21.15.159:0/2620087525 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f815406c920 0x7f815406ede0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.920 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.923+0000 7f817b800640 1 --2- 172.21.15.159:0/2620087525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81740ce950 0x7f817409b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.923+0000 7f817b800640 1 --2- 172.21.15.159:0/2620087525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81740ce4d0 0x7f8174072990 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.923+0000 7f817b800640 1 --2- 172.21.15.159:0/2620087525 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f815c004800 0x7f8174152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:15.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.923+0000 7f817b800640 1 -- 172.21.15.159:0/2620087525 >> 172.21.15.159:0/2620087525 conn(0x7f817406b750 msgr2=0x7f817406d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:15.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.924+0000 7f817b800640 1 -- 172.21.15.159:0/2620087525 shutdown_connections 2024-04-04T06:17:15.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:15.924+0000 7f817b800640 1 -- 172.21.15.159:0/2620087525 wait complete. 2024-04-04T06:17:15.934 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-04T06:17:16.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.115+0000 7f7c2e600640 1 Processor -- start 2024-04-04T06:17:16.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.115+0000 7f7c2e600640 1 -- start start 2024-04-04T06:17:16.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.117+0000 7f7c2e600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7c28153580 0x7f7c280d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:16.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.117+0000 7f7c2e600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7c280d07c0 0x7f7c280aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:16.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.117+0000 7f7c2e600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c280afab0 con 0x7f7c280af2e0 2024-04-04T06:17:16.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.117+0000 7f7c2e600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f7c280d0c30 con 0x7f7c280d07c0 2024-04-04T06:17:16.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.117+0000 7f7c2e600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f7c2815cc80 con 0x7f7c28153580 2024-04-04T06:17:16.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.117+0000 7f7c1ea00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7c280d07c0 0x7f7c280aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:16.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.117+0000 7f7c1ea00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7c280d07c0 0x7f7c280aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37362/0 (socket says 172.21.15.159:37362) 2024-04-04T06:17:16.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.117+0000 7f7c1ea00640 1 -- 172.21.15.159:0/2085917813 learned_addr learned my addr 172.21.15.159:0/2085917813 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:16.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.117+0000 7f7c27400640 1 --2- 172.21.15.159:0/2085917813 >> v2:172.21.15.178:3300/0 conn(0x7f7c28153580 0x7f7c280d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:16.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.117+0000 7f7c1ea00640 1 -- 172.21.15.159:0/2085917813 >> v2:172.21.15.178:3300/0 conn(0x7f7c28153580 msgr2=0x7f7c280d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:16.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.117+0000 7f7c1ea00640 1 --2- 172.21.15.159:0/2085917813 >> v2:172.21.15.178:3300/0 conn(0x7f7c28153580 0x7f7c280d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:16.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.117+0000 7f7c1ea00640 1 -- 172.21.15.159:0/2085917813 >> v1:172.21.15.178:6789/0 conn(0x7f7c280af2e0 legacy=0x7f7c2806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:16.113 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.118+0000 7f7c1ea00640 1 -- 172.21.15.159:0/2085917813 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2815da00 con 0x7f7c280d07c0 2024-04-04T06:17:16.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.119+0000 7f7c1ea00640 1 --2- 172.21.15.159:0/2085917813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c280d07c0 0x7f7c280aeda0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f7c280b5270 tx=0x7f7c1002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d2a2e182403896 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:16.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.120+0000 7f7c26a00640 1 -- 172.21.15.159:0/2085917813 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1002e070 con 0x7f7c280d07c0 2024-04-04T06:17:16.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.120+0000 7f7c26a00640 1 -- 172.21.15.159:0/2085917813 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c10037070 con 0x7f7c280d07c0 2024-04-04T06:17:16.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.120+0000 7f7c26a00640 1 -- 172.21.15.159:0/2085917813 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10031a00 con 0x7f7c280d07c0 2024-04-04T06:17:16.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.120+0000 7f7c2e600640 1 -- 172.21.15.159:0/2085917813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c280d07c0 msgr2=0x7f7c280aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:16.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.120+0000 7f7c2e600640 1 --2- 172.21.15.159:0/2085917813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c280d07c0 0x7f7c280aeda0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f7c280b5270 tx=0x7f7c1002b880 comp rx=0 tx=0).stop 2024-04-04T06:17:16.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.120+0000 7f7c2e600640 1 -- 172.21.15.159:0/2085917813 shutdown_connections 2024-04-04T06:17:16.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.120+0000 7f7c2e600640 1 --2- 172.21.15.159:0/2085917813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c280d07c0 0x7f7c280aeda0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:16.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.120+0000 7f7c2e600640 1 --2- 172.21.15.159:0/2085917813 >> v2:172.21.15.178:3300/0 conn(0x7f7c28153580 0x7f7c280d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:16.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.120+0000 7f7c2e600640 1 -- 172.21.15.159:0/2085917813 >> 172.21.15.159:0/2085917813 conn(0x7f7c2806b750 msgr2=0x7f7c2806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:16.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.121+0000 7f7c2e600640 1 -- 172.21.15.159:0/2085917813 shutdown_connections 2024-04-04T06:17:16.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.121+0000 7f7c2e600640 1 -- 172.21.15.159:0/2085917813 wait complete. 2024-04-04T06:17:16.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.122+0000 7f7c2e600640 1 Processor -- start 2024-04-04T06:17:16.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.122+0000 7f7c2e600640 1 -- start start 2024-04-04T06:17:16.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.123+0000 7f7c2e600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c280af2e0 0x7f7c2814d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:16.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.123+0000 7f7c2e600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c280d07c0 0x7f7c280cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:16.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.123+0000 7f7c2e600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c28153580 0x7f7c2815a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:16.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.123+0000 7f7c2e600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2815d4e0 con 0x7f7c28153580 2024-04-04T06:17:16.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.123+0000 7f7c2e600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2815d200 con 0x7f7c280d07c0 2024-04-04T06:17:16.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.123+0000 7f7c2e600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7c28153960 con 0x7f7c280af2e0 2024-04-04T06:17:16.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.123+0000 7f7c27e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c28153580 0x7f7c2815a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:16.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.124+0000 7f7c27e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c28153580 0x7f7c2815a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37368/0 (socket says 172.21.15.159:37368) 2024-04-04T06:17:16.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.124+0000 7f7c27e00640 1 -- 172.21.15.159:0/1927169238 learned_addr learned my addr 172.21.15.159:0/1927169238 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:16.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.124+0000 7f7c1ea00640 1 --2- 172.21.15.159:0/1927169238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c280d07c0 0x7f7c280cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:16.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.124+0000 7f7c27400640 1 --2- 172.21.15.159:0/1927169238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c280af2e0 0x7f7c2814d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:16.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.124+0000 7f7c27e00640 1 -- 172.21.15.159:0/1927169238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c280af2e0 msgr2=0x7f7c2814d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:16.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.124+0000 7f7c27e00640 1 --2- 172.21.15.159:0/1927169238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c280af2e0 0x7f7c2814d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:16.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.124+0000 7f7c27e00640 1 -- 172.21.15.159:0/1927169238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c280d07c0 msgr2=0x7f7c280cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:16.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.124+0000 7f7c27e00640 1 --2- 172.21.15.159:0/1927169238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c280d07c0 0x7f7c280cf2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:16.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.124+0000 7f7c27e00640 1 -- 172.21.15.159:0/1927169238 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c280c96b0 con 0x7f7c28153580 2024-04-04T06:17:16.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.124+0000 7f7c27e00640 1 --2- 172.21.15.159:0/1927169238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c28153580 0x7f7c2815a790 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000b9a0 tx=0x7f7c2000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:16.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.124+0000 7f7c1fe00640 1 -- 172.21.15.159:0/1927169238 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20019070 con 0x7f7c28153580 2024-04-04T06:17:16.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.125+0000 7f7c1fe00640 1 -- 172.21.15.159:0/1927169238 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c2000cba0 con 0x7f7c28153580 2024-04-04T06:17:16.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.125+0000 7f7c1fe00640 1 -- 172.21.15.159:0/1927169238 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20014440 con 0x7f7c28153580 2024-04-04T06:17:16.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.125+0000 7f7c2e600640 1 -- 172.21.15.159:0/1927169238 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c28073600 con 0x7f7c28153580 2024-04-04T06:17:16.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.125+0000 7f7c2e600640 1 -- 172.21.15.159:0/1927169238 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c28154fd0 con 0x7f7c28153580 2024-04-04T06:17:16.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.126+0000 7f7c2e600640 1 -- 172.21.15.159:0/1927169238 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c28073380 con 0x7f7c28153580 2024-04-04T06:17:16.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.127+0000 7f7c1fe00640 1 -- 172.21.15.159:0/1927169238 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c2001c070 con 0x7f7c28153580 2024-04-04T06:17:16.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.128+0000 7f7c1fe00640 1 --2- 172.21.15.159:0/1927169238 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7c0406c8f0 0x7f7c0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:16.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.128+0000 7f7c1fe00640 1 -- 172.21.15.159:0/1927169238 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f7c2008b730 con 0x7f7c28153580 2024-04-04T06:17:16.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.128+0000 7f7c27400640 1 --2- 172.21.15.159:0/1927169238 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7c0406c8f0 0x7f7c0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:16.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.129+0000 7f7c27400640 1 --2- 172.21.15.159:0/1927169238 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7c0406c8f0 0x7f7c0406edb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7c18006fd0 tx=0x7f7c18008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:16.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.132+0000 7f7c1fe00640 1 -- 172.21.15.159:0/1927169238 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7c28073380 con 0x7f7c28153580 2024-04-04T06:17:16.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.366+0000 7f7c2e600640 1 -- 172.21.15.159:0/1927169238 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7c2806eca0 con 0x7f7c28153580 2024-04-04T06:17:16.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.367+0000 7f7c1fe00640 1 -- 172.21.15.159:0/1927169238 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v101) ==== 53+0+73 (secure 0 0 0) 0x7f7c280c96b0 con 0x7f7c28153580 2024-04-04T06:17:16.361 INFO:teuthology.orchestra.run.smithi159.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-04T06:17:16.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.369+0000 7f7c2e600640 1 -- 172.21.15.159:0/1927169238 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7c0406c8f0 msgr2=0x7f7c0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:16.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.369+0000 7f7c2e600640 1 --2- 172.21.15.159:0/1927169238 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7c0406c8f0 0x7f7c0406edb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7c18006fd0 tx=0x7f7c18008040 comp rx=0 tx=0).stop 2024-04-04T06:17:16.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.369+0000 7f7c2e600640 1 -- 172.21.15.159:0/1927169238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c28153580 msgr2=0x7f7c2815a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:16.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.369+0000 7f7c2e600640 1 --2- 172.21.15.159:0/1927169238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c28153580 0x7f7c2815a790 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000b9a0 tx=0x7f7c2000be60 comp rx=0 tx=0).stop 2024-04-04T06:17:16.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.369+0000 7f7c2e600640 1 -- 172.21.15.159:0/1927169238 shutdown_connections 2024-04-04T06:17:16.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.369+0000 7f7c2e600640 1 --2- 172.21.15.159:0/1927169238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c28153580 0x7f7c2815a790 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:16.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.369+0000 7f7c2e600640 1 --2- 172.21.15.159:0/1927169238 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7c0406c8f0 0x7f7c0406edb0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:16.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.369+0000 7f7c2e600640 1 --2- 172.21.15.159:0/1927169238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c280d07c0 0x7f7c280cf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:16.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.369+0000 7f7c2e600640 1 --2- 172.21.15.159:0/1927169238 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7c280af2e0 0x7f7c2814d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:16.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.369+0000 7f7c2e600640 1 -- 172.21.15.159:0/1927169238 >> 172.21.15.159:0/1927169238 conn(0x7f7c2806b750 msgr2=0x7f7c28159b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:16.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.369+0000 7f7c2e600640 1 -- 172.21.15.159:0/1927169238 shutdown_connections 2024-04-04T06:17:16.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16.369+0000 7f7c2e600640 1 -- 172.21.15.159:0/1927169238 wait complete. 2024-04-04T06:17:16.382 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-04T06:17:16.382 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-04T06:17:16.382 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi159.front.sepia.ceph.com 2024-04-04T06:17:16.382 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-04T06:17:16.383 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-04T06:17:16.383 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-04T06:17:16.383 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-04T06:17:16.383 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-04T06:17:16.383 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:17:16.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:16.383 DEBUG:teuthology.orchestra.run.smithi159:> ip addr 2024-04-04T06:17:16.436 INFO:teuthology.orchestra.run.smithi159.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:17:16.436 INFO:teuthology.orchestra.run.smithi159.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:17:16.436 INFO:teuthology.orchestra.run.smithi159.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:17:16.437 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:16.437 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 ::1/128 scope host 2024-04-04T06:17:16.437 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:16.437 INFO:teuthology.orchestra.run.smithi159.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:17:16.437 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c6 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:17:16.437 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f0 2024-04-04T06:17:16.437 INFO:teuthology.orchestra.run.smithi159.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:17:16.437 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c7 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:17:16.437 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f1 2024-04-04T06:17:16.437 INFO:teuthology.orchestra.run.smithi159.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:17:16.437 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4e brd ff:ff:ff:ff:ff:ff 2024-04-04T06:17:16.437 INFO:teuthology.orchestra.run.smithi159.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-04T06:17:16.437 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:17:16.438 INFO:teuthology.orchestra.run.smithi159.stdout: inet 172.21.15.159/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-04T06:17:16.438 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft 41896sec preferred_lft 41896sec 2024-04-04T06:17:16.438 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::ec4:7aff:fe88:724f/64 scope link 2024-04-04T06:17:16.438 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:16.438 INFO:teuthology.orchestra.run.smithi159.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-04T06:17:16.438 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:17:16.438 INFO:teuthology.orchestra.run.smithi159.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-04T06:17:16.438 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:16.438 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::989e:e4ff:fea5:7568/64 scope link 2024-04-04T06:17:16.438 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:16.439 INFO:teuthology.orchestra.run.smithi159.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:17:16.439 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether a2:e3:e0:cb:0a:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-04T06:17:16.439 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::a0e3:e0ff:fecb:a73/64 scope link 2024-04-04T06:17:16.439 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:16.439 INFO:teuthology.orchestra.run.smithi159.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:17:16.439 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-04T06:17:16.439 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::c0a:8aff:fe54:b58f/64 scope link 2024-04-04T06:17:16.439 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:16.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:16.440 DEBUG:teuthology.orchestra.run.smithi159:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-04T06:17:16.517 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:17:16.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:16.521 DEBUG:teuthology.orchestra.run.smithi159:> route 2024-04-04T06:17:16.565 INFO:teuthology.orchestra.run.smithi159.stdout:Kernel IP routing table 2024-04-04T06:17:16.565 INFO:teuthology.orchestra.run.smithi159.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-04T06:17:16.565 INFO:teuthology.orchestra.run.smithi159.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-04T06:17:16.565 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-04T06:17:16.565 INFO:teuthology.orchestra.run.smithi159.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-04T06:17:16.566 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:17:16.566 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:17:16.566 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-04T06:17:16.566 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-04T06:17:16.566 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-04T06:17:16.566 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:17:16.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:17:16.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:16.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:16.839 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-04-04T06:17:16.883 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-04T06:17:16.883 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-04T06:17:16.883 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:17:16.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:16.883 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set brx.0 up 2024-04-04T06:17:17.014 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:17:17.014 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-04T06:17:17.066 INFO:teuthology.orchestra.run.smithi159.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:17:17.066 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:17:17.066 DEBUG:teuthology.orchestra.run.smithi159:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:17:17.114 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-04T06:17:17.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:17.114 DEBUG:teuthology.orchestra.run.smithi159:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:17:17.251 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:17:17.251 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:17:17.251 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:17:17.271 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:17:17.271 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:17:17.271 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:17:17.271 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:17:17.271 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:17:17.296 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:17:17.296 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:17:17.296 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:17:17.296 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:17:17.296 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:17:17.296 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-04T06:17:17.297 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-04T06:17:17.297 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30311045-f714-48fa-832b-4cb2ba8bf0bf" will pass to kernel 2024-04-04T06:17:17.300 INFO:teuthology.orchestra.run.smithi159.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-04T06:17:17.300 INFO:teuthology.orchestra.run.smithi159.stdout: You just mounted a file system that supports labels which does not 2024-04-04T06:17:17.300 INFO:teuthology.orchestra.run.smithi159.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-04T06:17:17.300 INFO:teuthology.orchestra.run.smithi159.stdout: applications will generate AVC messages and not be allowed access to 2024-04-04T06:17:17.301 INFO:teuthology.orchestra.run.smithi159.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-04T06:17:17.304 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-04T06:17:17.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:17.305 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:17:17.388 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:17:17.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:17:17.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:17.505 INFO:teuthology.orchestra.run.smithi159.stdout:client6454 2024-04-04T06:17:17.505 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:17:17.505 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:17:17.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:17.553 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-04-04T06:17:17.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:17.653 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-04-04T06:17:17.758 INFO:teuthology.orchestra.run.smithi159.stderr:8+0 records in 2024-04-04T06:17:17.758 INFO:teuthology.orchestra.run.smithi159.stderr:8+0 records out 2024-04-04T06:17:17.758 INFO:teuthology.orchestra.run.smithi159.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0573116 s, 146 MB/s 2024-04-04T06:17:17.760 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:17.760 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:17.760 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:17.760 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:17.760 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-04T06:17:17.760 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:17.893 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627779 2024-04-04T06:17:17.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:17.906 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:17.907 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:17.907 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:17.907 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:17.907 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-04T06:17:17.907 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:18.089 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627780 2024-04-04T06:17:18.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:18.101 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:17:18.101 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:17:18.200 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:17:18.200 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:17:18.253 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:17:18.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.467+0000 7fa972a00640 1 Processor -- start 2024-04-04T06:17:18.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.467+0000 7fa972a00640 1 -- start start 2024-04-04T06:17:18.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.470+0000 7fa972a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa96c15b310 0x7fa96c0727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:18.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.470+0000 7fa972a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa96c0d0780 0x7fa96c0d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:18.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.470+0000 7fa972a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa96c15b6f0 con 0x7fa96c0b4f90 2024-04-04T06:17:18.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.470+0000 7fa972a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fa96c072e80 con 0x7fa96c15b310 2024-04-04T06:17:18.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.470+0000 7fa972a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa96c073160 con 0x7fa96c0d0780 2024-04-04T06:17:18.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.470+0000 7fa96be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa96c15b310 0x7fa96c0727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:18.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.470+0000 7fa96be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa96c15b310 0x7fa96c0727f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37374/0 (socket says 172.21.15.159:37374) 2024-04-04T06:17:18.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.470+0000 7fa96be00640 1 -- 172.21.15.159:0/50828670 learned_addr learned my addr 172.21.15.159:0/50828670 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:18.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.471+0000 7fa970a00640 1 --2- 172.21.15.159:0/50828670 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa96c0d0780 0x7fa96c0d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:18.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.471+0000 7fa96be00640 1 -- 172.21.15.159:0/50828670 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa96c0d0780 msgr2=0x7fa96c0d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:18.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.471+0000 7fa96be00640 1 --2- 172.21.15.159:0/50828670 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa96c0d0780 0x7fa96c0d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:18.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.471+0000 7fa96be00640 1 -- 172.21.15.159:0/50828670 >> v1:172.21.15.178:6789/0 conn(0x7fa96c0b4f90 legacy=0x7fa96c15f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:18.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.471+0000 7fa96be00640 1 -- 172.21.15.159:0/50828670 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96c15efe0 con 0x7fa96c15b310 2024-04-04T06:17:18.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.472+0000 7fa96be00640 1 --2- 172.21.15.159:0/50828670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa96c15b310 0x7fa96c0727f0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009e50 tx=0x7fa95c02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706c6ee37c1833d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:18.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.472+0000 7fa96aa00640 1 -- 172.21.15.159:0/50828670 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c02c950 con 0x7fa96c15b310 2024-04-04T06:17:18.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.472+0000 7fa96aa00640 1 -- 172.21.15.159:0/50828670 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa95c02caf0 con 0x7fa96c15b310 2024-04-04T06:17:18.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.472+0000 7fa96aa00640 1 -- 172.21.15.159:0/50828670 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c0343f0 con 0x7fa96c15b310 2024-04-04T06:17:18.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.473+0000 7fa972a00640 1 -- 172.21.15.159:0/50828670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa96c15b310 msgr2=0x7fa96c0727f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:18.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.473+0000 7fa972a00640 1 --2- 172.21.15.159:0/50828670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa96c15b310 0x7fa96c0727f0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009e50 tx=0x7fa95c02bdb0 comp rx=0 tx=0).stop 2024-04-04T06:17:18.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.473+0000 7fa972a00640 1 -- 172.21.15.159:0/50828670 shutdown_connections 2024-04-04T06:17:18.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.473+0000 7fa972a00640 1 --2- 172.21.15.159:0/50828670 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa96c0d0780 0x7fa96c0d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:18.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.473+0000 7fa972a00640 1 --2- 172.21.15.159:0/50828670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa96c15b310 0x7fa96c0727f0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:18.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.473+0000 7fa972a00640 1 -- 172.21.15.159:0/50828670 >> 172.21.15.159:0/50828670 conn(0x7fa96c06b990 msgr2=0x7fa96c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:18.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.473+0000 7fa972a00640 1 -- 172.21.15.159:0/50828670 shutdown_connections 2024-04-04T06:17:18.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.473+0000 7fa972a00640 1 -- 172.21.15.159:0/50828670 wait complete. 2024-04-04T06:17:18.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.475+0000 7fa972a00640 1 Processor -- start 2024-04-04T06:17:18.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.475+0000 7fa972a00640 1 -- start start 2024-04-04T06:17:18.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.477+0000 7fa972a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa96c0b4f90 0x7fa96c14c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:18.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.477+0000 7fa972a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa96c0d0780 0x7fa96c1e8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:18.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.477+0000 7fa972a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa96c15b310 0x7fa96c1e8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:18.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.477+0000 7fa972a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c073440 con 0x7fa96c15b310 2024-04-04T06:17:18.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.477+0000 7fa972a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c0b53f0 con 0x7fa96c0b4f90 2024-04-04T06:17:18.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.477+0000 7fa972a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa96c0adc80 con 0x7fa96c0d0780 2024-04-04T06:17:18.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.477+0000 7fa970a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa96c15b310 0x7fa96c1e8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:18.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.477+0000 7fa970a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa96c15b310 0x7fa96c1e8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37378/0 (socket says 172.21.15.159:37378) 2024-04-04T06:17:18.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.477+0000 7fa970a00640 1 -- 172.21.15.159:0/3900881932 learned_addr learned my addr 172.21.15.159:0/3900881932 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:18.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.478+0000 7fa96be00640 1 --2- 172.21.15.159:0/3900881932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa96c0b4f90 0x7fa96c14c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:18.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.478+0000 7fa970a00640 1 -- 172.21.15.159:0/3900881932 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa96c0d0780 msgr2=0x7fa96c1e8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:18.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.478+0000 7fa970a00640 1 --2- 172.21.15.159:0/3900881932 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa96c0d0780 0x7fa96c1e8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:18.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.478+0000 7fa970a00640 1 -- 172.21.15.159:0/3900881932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa96c0b4f90 msgr2=0x7fa96c14c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:18.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.478+0000 7fa970a00640 1 --2- 172.21.15.159:0/3900881932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa96c0b4f90 0x7fa96c14c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:18.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.478+0000 7fa970a00640 1 -- 172.21.15.159:0/3900881932 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96c15ca00 con 0x7fa96c15b310 2024-04-04T06:17:18.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.478+0000 7fa96be00640 1 --2- 172.21.15.159:0/3900881932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa96c0b4f90 0x7fa96c14c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:18.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.478+0000 7fa970a00640 1 --2- 172.21.15.159:0/3900881932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa96c15b310 0x7fa96c1e8db0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fa96400c820 tx=0x7fa96400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:18.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.478+0000 7fa963e00640 1 -- 172.21.15.159:0/3900881932 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964015070 con 0x7fa96c15b310 2024-04-04T06:17:18.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.478+0000 7fa963e00640 1 -- 172.21.15.159:0/3900881932 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa964004020 con 0x7fa96c15b310 2024-04-04T06:17:18.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.478+0000 7fa963e00640 1 -- 172.21.15.159:0/3900881932 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964004410 con 0x7fa96c15b310 2024-04-04T06:17:18.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.478+0000 7fa972a00640 1 -- 172.21.15.159:0/3900881932 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96c15cce0 con 0x7fa96c15b310 2024-04-04T06:17:18.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.478+0000 7fa972a00640 1 -- 172.21.15.159:0/3900881932 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa96c15d2a0 con 0x7fa96c15b310 2024-04-04T06:17:18.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.480+0000 7fa972a00640 1 -- 172.21.15.159:0/3900881932 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa96c0685c0 con 0x7fa96c15b310 2024-04-04T06:17:18.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.481+0000 7fa963e00640 1 -- 172.21.15.159:0/3900881932 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa964007830 con 0x7fa96c15b310 2024-04-04T06:17:18.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.482+0000 7fa963e00640 1 --2- 172.21.15.159:0/3900881932 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa94c06c8f0 0x7fa94c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:18.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.483+0000 7fa963e00640 1 -- 172.21.15.159:0/3900881932 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7fa96408ba70 con 0x7fa96c15b310 2024-04-04T06:17:18.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.483+0000 7fa96be00640 1 --2- 172.21.15.159:0/3900881932 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa94c06c8f0 0x7fa94c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:18.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.484+0000 7fa96be00640 1 --2- 172.21.15.159:0/3900881932 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa94c06c8f0 0x7fa94c06edb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa95c035000 tx=0x7fa95c030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:18.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.489+0000 7fa963e00640 1 -- 172.21.15.159:0/3900881932 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa964059e00 con 0x7fa96c15b310 2024-04-04T06:17:18.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.691+0000 7fa972a00640 1 -- 172.21.15.159:0/3900881932 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa96c0d0010 con 0x7fa96c15b310 2024-04-04T06:17:18.690 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.695+0000 7fa963e00640 1 -- 172.21.15.159:0/3900881932 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16639 (secure 0 0 0) 0x7fa964059700 con 0x7fa96c15b310 2024-04-04T06:17:18.690 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:18.690 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":113,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:16:58.838677+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:16:55.036975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-04T06:16:56.239022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:17:18.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.697+0000 7fa972a00640 1 -- 172.21.15.159:0/3900881932 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa94c06c8f0 msgr2=0x7fa94c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:18.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.697+0000 7fa972a00640 1 --2- 172.21.15.159:0/3900881932 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa94c06c8f0 0x7fa94c06edb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa95c035000 tx=0x7fa95c030000 comp rx=0 tx=0).stop 2024-04-04T06:17:18.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.697+0000 7fa972a00640 1 -- 172.21.15.159:0/3900881932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa96c15b310 msgr2=0x7fa96c1e8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:18.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.697+0000 7fa972a00640 1 --2- 172.21.15.159:0/3900881932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa96c15b310 0x7fa96c1e8db0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fa96400c820 tx=0x7fa96400cce0 comp rx=0 tx=0).stop 2024-04-04T06:17:18.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.698+0000 7fa972a00640 1 -- 172.21.15.159:0/3900881932 shutdown_connections 2024-04-04T06:17:18.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.698+0000 7fa972a00640 1 --2- 172.21.15.159:0/3900881932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa96c15b310 0x7fa96c1e8db0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:18.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.698+0000 7fa972a00640 1 --2- 172.21.15.159:0/3900881932 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa96c0d0780 0x7fa96c1e8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:18.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.698+0000 7fa972a00640 1 --2- 172.21.15.159:0/3900881932 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa94c06c8f0 0x7fa94c06edb0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:18.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.698+0000 7fa972a00640 1 --2- 172.21.15.159:0/3900881932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa96c0b4f90 0x7fa96c14c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:18.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.698+0000 7fa972a00640 1 -- 172.21.15.159:0/3900881932 >> 172.21.15.159:0/3900881932 conn(0x7fa96c06b990 msgr2=0x7fa96c156fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:18.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.698+0000 7fa972a00640 1 -- 172.21.15.159:0/3900881932 shutdown_connections 2024-04-04T06:17:18.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.698+0000 7fa972a00640 1 -- 172.21.15.159:0/3900881932 wait complete. 2024-04-04T06:17:18.711 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:16:55.036975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:16:56.239022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:17:18.711 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:17:18.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.926+0000 7fa7b1c00640 1 Processor -- start 2024-04-04T06:17:18.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.926+0000 7fa7b1c00640 1 -- start start 2024-04-04T06:17:18.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.928+0000 7fa7b1c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa7ac0b0670 0x7fa7ac15f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:18.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.928+0000 7fa7b1c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7ac0b5b10 con 0x7fa7ac070120 2024-04-04T06:17:18.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.928+0000 7fa7b1c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7ac15fdf0 con 0x7fa7ac06bc80 2024-04-04T06:17:18.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.928+0000 7fa7b1c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa7ac0b63e0 con 0x7fa7ac0b0670 2024-04-04T06:17:18.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.928+0000 7fa7aa000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fa7ac070120 0x7fa7ac06b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41016/0 (socket says 172.21.15.159:41016) 2024-04-04T06:17:18.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.928+0000 7fa7aa000640 1 -- 172.21.15.159:0/2501755154 learned_addr learned my addr 172.21.15.159:0/2501755154 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:18.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.929+0000 7fa7aaa00640 1 --2- 172.21.15.159:0/2501755154 >> v2:172.21.15.178:3300/0 conn(0x7fa7ac0b0670 0x7fa7ac15f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:18.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.929+0000 7fa7a9600640 1 -- 172.21.15.159:0/2501755154 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1005747298 0 0) 0x7fa798001110 con 0x7fa7ac070120 2024-04-04T06:17:18.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.929+0000 7fa7a9600640 1 -- 172.21.15.159:0/2501755154 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa790003520 con 0x7fa7ac070120 2024-04-04T06:17:18.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.929+0000 7fa7a9600640 1 -- 172.21.15.159:0/2501755154 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3362255007 0 0) 0x7fa790003520 con 0x7fa7ac070120 2024-04-04T06:17:18.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.929+0000 7fa7a9600640 1 -- 172.21.15.159:0/2501755154 >> v2:172.21.15.178:3300/0 conn(0x7fa7ac0b0670 msgr2=0x7fa7ac15f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:18.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.929+0000 7fa7a9600640 1 --2- 172.21.15.159:0/2501755154 >> v2:172.21.15.178:3300/0 conn(0x7fa7ac0b0670 0x7fa7ac15f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:18.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.929+0000 7fa7a9600640 1 -- 172.21.15.159:0/2501755154 >> v1:172.21.15.178:6789/0 conn(0x7fa7ac06bc80 legacy=0x7fa7ac168450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:18.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.929+0000 7fa7a9600640 1 -- 172.21.15.159:0/2501755154 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ac0705e0 con 0x7fa7ac070120 2024-04-04T06:17:18.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.929+0000 7fa7a9600640 1 -- 172.21.15.159:0/2501755154 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fa798001840 con 0x7fa7ac070120 2024-04-04T06:17:18.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.929+0000 7fa7a9600640 1 -- 172.21.15.159:0/2501755154 >> v1:172.21.15.159:6789/0 conn(0x7fa7ac070120 legacy=0x7fa7ac06b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:18.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.929+0000 7fa7a9600640 1 --2- 172.21.15.159:0/2501755154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa790003a00 0x7fa790003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:18.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.929+0000 7fa7a9600640 1 --2- 172.21.15.159:0/2501755154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa790004800 0x7fa790004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:18.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.929+0000 7fa7a9600640 1 --2- 172.21.15.159:0/2501755154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7900051c0 0x7fa79000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:18.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.930+0000 7fa7a9600640 1 -- 172.21.15.159:0/2501755154 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac0b63e0 con 0x7fa790004800 2024-04-04T06:17:18.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.930+0000 7fa7a9600640 1 -- 172.21.15.159:0/2501755154 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa79000c0c0 con 0x7fa790003a00 2024-04-04T06:17:18.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.930+0000 7fa7a9600640 1 -- 172.21.15.159:0/2501755154 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa79000c2c0 con 0x7fa7900051c0 2024-04-04T06:17:18.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.930+0000 7fa7aa000640 1 --2- 172.21.15.159:0/2501755154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa790004800 0x7fa790004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:18.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.930+0000 7fa7ab400640 1 --2- 172.21.15.159:0/2501755154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7900051c0 0x7fa79000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:18.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.930+0000 7fa7aaa00640 1 --2- 172.21.15.159:0/2501755154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa790003a00 0x7fa790003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:18.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.930+0000 7fa7aa000640 1 -- 172.21.15.159:0/2501755154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7900051c0 msgr2=0x7fa79000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:18.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.930+0000 7fa7aa000640 1 --2- 172.21.15.159:0/2501755154 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7900051c0 0x7fa79000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:18.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.930+0000 7fa7aa000640 1 -- 172.21.15.159:0/2501755154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa790003a00 msgr2=0x7fa790003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:18.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.930+0000 7fa7aa000640 1 --2- 172.21.15.159:0/2501755154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa790003a00 0x7fa790003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:18.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.930+0000 7fa7ab400640 1 -- 172.21.15.159:0/2501755154 reap_dead start 2024-04-04T06:17:18.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.930+0000 7fa7aa000640 1 -- 172.21.15.159:0/2501755154 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa790003520 con 0x7fa790004800 2024-04-04T06:17:18.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.930+0000 7fa7aaa00640 1 --2- 172.21.15.159:0/2501755154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa790003a00 0x7fa790003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:18.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.931+0000 7fa7aa000640 1 --2- 172.21.15.159:0/2501755154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa790004800 0x7fa790004c80 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fa79800c320 tx=0x7fa790005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c4023f19d23b7b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:18.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.932+0000 7fa7a9600640 1 -- 172.21.15.159:0/2501755154 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa798007e10 con 0x7fa790004800 2024-04-04T06:17:18.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.932+0000 7fa7a9600640 1 -- 172.21.15.159:0/2501755154 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa798030070 con 0x7fa790004800 2024-04-04T06:17:18.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.932+0000 7fa7a9600640 1 -- 172.21.15.159:0/2501755154 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7980058d0 con 0x7fa790004800 2024-04-04T06:17:18.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.932+0000 7fa7b1c00640 1 -- 172.21.15.159:0/2501755154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa790004800 msgr2=0x7fa790004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:18.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.932+0000 7fa7b1c00640 1 --2- 172.21.15.159:0/2501755154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa790004800 0x7fa790004c80 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fa79800c320 tx=0x7fa790005830 comp rx=0 tx=0).stop 2024-04-04T06:17:18.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.933+0000 7fa7b1c00640 1 -- 172.21.15.159:0/2501755154 shutdown_connections 2024-04-04T06:17:18.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.933+0000 7fa7b1c00640 1 --2- 172.21.15.159:0/2501755154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa790004800 0x7fa790004c80 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:18.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.933+0000 7fa7b1c00640 1 -- 172.21.15.159:0/2501755154 >> 172.21.15.159:0/2501755154 conn(0x7fa7ac0b65a0 msgr2=0x7fa7ac0b69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:18.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.933+0000 7fa7b1c00640 1 -- 172.21.15.159:0/2501755154 shutdown_connections 2024-04-04T06:17:18.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.933+0000 7fa7b1c00640 1 -- 172.21.15.159:0/2501755154 wait complete. 2024-04-04T06:17:18.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.936+0000 7fa7b1c00640 1 Processor -- start 2024-04-04T06:17:18.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.936+0000 7fa7b1c00640 1 -- start start 2024-04-04T06:17:18.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.937+0000 7fa7b1c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa790004800 0x7fa7ac142580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:18.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.937+0000 7fa7b1c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7ac0b1610 0x7fa7ac0b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:18.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.937+0000 7fa7b1c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7ac0ce9a0 0x7fa7ac0b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:18.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.937+0000 7fa7b1c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac0b6070 con 0x7fa790004800 2024-04-04T06:17:18.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.937+0000 7fa7b1c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac06c3f0 con 0x7fa7ac0ce9a0 2024-04-04T06:17:18.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.937+0000 7fa7b1c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa7ac0b0a50 con 0x7fa7ac0b1610 2024-04-04T06:17:18.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.938+0000 7fa7aaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa790004800 0x7fa7ac142580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:18.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.938+0000 7fa7aaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa790004800 0x7fa7ac142580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37402/0 (socket says 172.21.15.159:37402) 2024-04-04T06:17:18.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.938+0000 7fa7aaa00640 1 -- 172.21.15.159:0/1985217643 learned_addr learned my addr 172.21.15.159:0/1985217643 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:18.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.938+0000 7fa7aaa00640 1 -- 172.21.15.159:0/1985217643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7ac0b1610 msgr2=0x7fa7ac0b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:18.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.938+0000 7fa7aaa00640 1 --2- 172.21.15.159:0/1985217643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7ac0b1610 0x7fa7ac0b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:18.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.938+0000 7fa7aaa00640 1 -- 172.21.15.159:0/1985217643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7ac0ce9a0 msgr2=0x7fa7ac0b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:18.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.938+0000 7fa7aaa00640 1 --2- 172.21.15.159:0/1985217643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7ac0ce9a0 0x7fa7ac0b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:18.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.938+0000 7fa7aaa00640 1 -- 172.21.15.159:0/1985217643 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ac0b1c70 con 0x7fa790004800 2024-04-04T06:17:18.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.938+0000 7fa7aaa00640 1 --2- 172.21.15.159:0/1985217643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa790004800 0x7fa7ac142580 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0107f0 tx=0x7fa79c010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:18.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.939+0000 7fa7a2a00640 1 -- 172.21.15.159:0/1985217643 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c00f040 con 0x7fa790004800 2024-04-04T06:17:18.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.939+0000 7fa7a2a00640 1 -- 172.21.15.159:0/1985217643 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa79c00eb60 con 0x7fa790004800 2024-04-04T06:17:18.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.939+0000 7fa7a2a00640 1 -- 172.21.15.159:0/1985217643 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c01c400 con 0x7fa790004800 2024-04-04T06:17:18.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.939+0000 7fa7b1c00640 1 -- 172.21.15.159:0/1985217643 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7ac0cdbe0 con 0x7fa790004800 2024-04-04T06:17:18.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.939+0000 7fa7b1c00640 1 -- 172.21.15.159:0/1985217643 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7ac0ce1a0 con 0x7fa790004800 2024-04-04T06:17:18.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.940+0000 7fa7b1c00640 1 -- 172.21.15.159:0/1985217643 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7ac068cd0 con 0x7fa790004800 2024-04-04T06:17:18.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.941+0000 7fa7a2a00640 1 -- 172.21.15.159:0/1985217643 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa79c01c5a0 con 0x7fa790004800 2024-04-04T06:17:18.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.942+0000 7fa7a2a00640 1 --2- 172.21.15.159:0/1985217643 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa78c06c8f0 0x7fa78c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:18.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.942+0000 7fa7a2a00640 1 -- 172.21.15.159:0/1985217643 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7fa79c08c120 con 0x7fa790004800 2024-04-04T06:17:18.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.943+0000 7fa7aa000640 1 --2- 172.21.15.159:0/1985217643 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa78c06c8f0 0x7fa78c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:18.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.944+0000 7fa7aa000640 1 --2- 172.21.15.159:0/1985217643 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa78c06c8f0 0x7fa78c06edb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa798034000 tx=0x7fa798007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:18.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:18.947+0000 7fa7a2a00640 1 -- 172.21.15.159:0/1985217643 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa79c018030 con 0x7fa790004800 2024-04-04T06:17:19.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.147+0000 7fa7b1c00640 1 -- 172.21.15.159:0/1985217643 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa7ac0af730 con 0x7fa790004800 2024-04-04T06:17:19.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.148+0000 7fa7a2a00640 1 -- 172.21.15.159:0/1985217643 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7fa79c05a550 con 0x7fa790004800 2024-04-04T06:17:19.143 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:17:19.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.150+0000 7fa7b1c00640 1 -- 172.21.15.159:0/1985217643 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa78c06c8f0 msgr2=0x7fa78c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:19.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.151+0000 7fa7b1c00640 1 --2- 172.21.15.159:0/1985217643 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa78c06c8f0 0x7fa78c06edb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa798034000 tx=0x7fa798007000 comp rx=0 tx=0).stop 2024-04-04T06:17:19.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.151+0000 7fa7b1c00640 1 -- 172.21.15.159:0/1985217643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa790004800 msgr2=0x7fa7ac142580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:19.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.151+0000 7fa7b1c00640 1 --2- 172.21.15.159:0/1985217643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa790004800 0x7fa7ac142580 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0107f0 tx=0x7fa79c010cb0 comp rx=0 tx=0).stop 2024-04-04T06:17:19.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.151+0000 7fa7b1c00640 1 -- 172.21.15.159:0/1985217643 shutdown_connections 2024-04-04T06:17:19.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.151+0000 7fa7b1c00640 1 --2- 172.21.15.159:0/1985217643 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa78c06c8f0 0x7fa78c06edb0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:19.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.151+0000 7fa7b1c00640 1 --2- 172.21.15.159:0/1985217643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7ac0ce9a0 0x7fa7ac0b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:19.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.151+0000 7fa7b1c00640 1 --2- 172.21.15.159:0/1985217643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa7ac0b1610 0x7fa7ac0b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:19.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.151+0000 7fa7b1c00640 1 --2- 172.21.15.159:0/1985217643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa790004800 0x7fa7ac142580 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:19.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.151+0000 7fa7b1c00640 1 -- 172.21.15.159:0/1985217643 >> 172.21.15.159:0/1985217643 conn(0x7fa7ac0b65a0 msgr2=0x7fa7ac0ae400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:19.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.151+0000 7fa7b1c00640 1 -- 172.21.15.159:0/1985217643 shutdown_connections 2024-04-04T06:17:19.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.151+0000 7fa7b1c00640 1 -- 172.21.15.159:0/1985217643 wait complete. 2024-04-04T06:17:19.167 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:17:19.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.372+0000 7fb50b400640 1 Processor -- start 2024-04-04T06:17:19.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.372+0000 7fb50b400640 1 -- start start 2024-04-04T06:17:19.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.374+0000 7fb50b400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5040d1560 0x7fb50406c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:19.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.374+0000 7fb50b400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb504071060 con 0x7fb50406cbc0 2024-04-04T06:17:19.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.374+0000 7fb50b400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb504071340 con 0x7fb50415e590 2024-04-04T06:17:19.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.374+0000 7fb50b400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb5040d19a0 con 0x7fb5040d1560 2024-04-04T06:17:19.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.374+0000 7fb509400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fb50406cbc0 0x7fb5040cfd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41026/0 (socket says 172.21.15.159:41026) 2024-04-04T06:17:19.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.374+0000 7fb509400640 1 -- 172.21.15.159:0/3508121156 learned_addr learned my addr 172.21.15.159:0/3508121156 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:19.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.375+0000 7fb503400640 1 -- 172.21.15.159:0/3508121156 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 82148978 0 0) 0x7fb504071060 con 0x7fb50406cbc0 2024-04-04T06:17:19.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.375+0000 7fb503e00640 1 --2- 172.21.15.159:0/3508121156 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5040d1560 0x7fb50406c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:19.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.375+0000 7fb503400640 1 -- 172.21.15.159:0/3508121156 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4ec003520 con 0x7fb50406cbc0 2024-04-04T06:17:19.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.375+0000 7fb503400640 1 -- 172.21.15.159:0/3508121156 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3122833961 0 0) 0x7fb4ec003520 con 0x7fb50406cbc0 2024-04-04T06:17:19.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.375+0000 7fb503400640 1 -- 172.21.15.159:0/3508121156 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5040d1560 msgr2=0x7fb50406c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:19.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.375+0000 7fb503400640 1 --2- 172.21.15.159:0/3508121156 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5040d1560 0x7fb50406c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:19.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.375+0000 7fb503400640 1 -- 172.21.15.159:0/3508121156 >> v1:172.21.15.178:6789/0 conn(0x7fb50415e590 legacy=0x7fb5040b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:19.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.375+0000 7fb503400640 1 -- 172.21.15.159:0/3508121156 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb504071620 con 0x7fb50406cbc0 2024-04-04T06:17:19.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.375+0000 7fb503400640 1 -- 172.21.15.159:0/3508121156 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fb4fc0019f0 con 0x7fb50406cbc0 2024-04-04T06:17:19.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.375+0000 7fb503400640 1 -- 172.21.15.159:0/3508121156 >> v1:172.21.15.159:6789/0 conn(0x7fb50406cbc0 legacy=0x7fb5040cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:19.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.375+0000 7fb503400640 1 --2- 172.21.15.159:0/3508121156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4ec0039e0 0x7fb4ec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:19.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.375+0000 7fb503400640 1 --2- 172.21.15.159:0/3508121156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4ec004800 0x7fb4ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:19.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.375+0000 7fb503400640 1 --2- 172.21.15.159:0/3508121156 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5040d1560 0x7fb4ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:19.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.375+0000 7fb503400640 1 -- 172.21.15.159:0/3508121156 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb5040d19a0 con 0x7fb4ec004800 2024-04-04T06:17:19.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.375+0000 7fb503400640 1 -- 172.21.15.159:0/3508121156 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec00bad0 con 0x7fb4ec0039e0 2024-04-04T06:17:19.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.375+0000 7fb503400640 1 -- 172.21.15.159:0/3508121156 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb4ec00bcd0 con 0x7fb5040d1560 2024-04-04T06:17:19.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.375+0000 7fb503e00640 1 --2- 172.21.15.159:0/3508121156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4ec004800 0x7fb4ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:19.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.376+0000 7fb508a00640 1 --2- 172.21.15.159:0/3508121156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4ec0039e0 0x7fb4ec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:19.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.376+0000 7fb503e00640 1 -- 172.21.15.159:0/3508121156 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5040d1560 msgr2=0x7fb4ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:19.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.376+0000 7fb503e00640 1 --2- 172.21.15.159:0/3508121156 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5040d1560 0x7fb4ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:19.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.376+0000 7fb503e00640 1 -- 172.21.15.159:0/3508121156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4ec0039e0 msgr2=0x7fb4ec003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:19.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.376+0000 7fb503e00640 1 --2- 172.21.15.159:0/3508121156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4ec0039e0 0x7fb4ec003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:19.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.376+0000 7fb503e00640 1 -- 172.21.15.159:0/3508121156 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4ec003520 con 0x7fb4ec004800 2024-04-04T06:17:19.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.376+0000 7fb508a00640 1 --2- 172.21.15.159:0/3508121156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4ec0039e0 0x7fb4ec003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:19.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.378+0000 7fb503e00640 1 --2- 172.21.15.159:0/3508121156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4ec004800 0x7fb4ec004c80 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009a90 tx=0x7fb4ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdad39087845ac4d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:19.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.378+0000 7fb503400640 1 -- 172.21.15.159:0/3508121156 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0039070 con 0x7fb4ec004800 2024-04-04T06:17:19.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.378+0000 7fb503400640 1 -- 172.21.15.159:0/3508121156 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4f002c6b0 con 0x7fb4ec004800 2024-04-04T06:17:19.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.378+0000 7fb503400640 1 -- 172.21.15.159:0/3508121156 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0009690 con 0x7fb4ec004800 2024-04-04T06:17:19.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.378+0000 7fb50b400640 1 -- 172.21.15.159:0/3508121156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4ec004800 msgr2=0x7fb4ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:19.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.378+0000 7fb50b400640 1 --2- 172.21.15.159:0/3508121156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4ec004800 0x7fb4ec004c80 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009a90 tx=0x7fb4ec005830 comp rx=0 tx=0).stop 2024-04-04T06:17:19.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.378+0000 7fb509400640 1 -- 172.21.15.159:0/3508121156 reap_dead start 2024-04-04T06:17:19.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.378+0000 7fb50b400640 1 -- 172.21.15.159:0/3508121156 shutdown_connections 2024-04-04T06:17:19.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.378+0000 7fb50b400640 1 -- 172.21.15.159:0/3508121156 >> 172.21.15.159:0/3508121156 conn(0x7fb5040d2240 msgr2=0x7fb5040d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:19.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.379+0000 7fb50b400640 1 -- 172.21.15.159:0/3508121156 shutdown_connections 2024-04-04T06:17:19.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.379+0000 7fb50b400640 1 -- 172.21.15.159:0/3508121156 wait complete. 2024-04-04T06:17:19.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.380+0000 7fb50b400640 1 Processor -- start 2024-04-04T06:17:19.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.380+0000 7fb50b400640 1 -- start start 2024-04-04T06:17:19.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.381+0000 7fb50b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4ec004800 0x7fb5040b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:19.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.381+0000 7fb50b400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb50414f700 0x7fb50414efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:19.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.381+0000 7fb50b400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5041570e0 0x7fb504157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:19.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.381+0000 7fb50b400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb504071be0 con 0x7fb4ec004800 2024-04-04T06:17:19.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.381+0000 7fb50b400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb50406cff0 con 0x7fb5041570e0 2024-04-04T06:17:19.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.381+0000 7fb50b400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb50415e970 con 0x7fb50414f700 2024-04-04T06:17:19.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.381+0000 7fb508a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4ec004800 0x7fb5040b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:19.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.382+0000 7fb508a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4ec004800 0x7fb5040b4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37422/0 (socket says 172.21.15.159:37422) 2024-04-04T06:17:19.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.382+0000 7fb508a00640 1 -- 172.21.15.159:0/95590063 learned_addr learned my addr 172.21.15.159:0/95590063 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:19.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.382+0000 7fb503e00640 1 --2- 172.21.15.159:0/95590063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb50414f700 0x7fb50414efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:19.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.382+0000 7fb509400640 1 --2- 172.21.15.159:0/95590063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5041570e0 0x7fb504157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:19.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.382+0000 7fb508a00640 1 -- 172.21.15.159:0/95590063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb50414f700 msgr2=0x7fb50414efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:19.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.382+0000 7fb508a00640 1 --2- 172.21.15.159:0/95590063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb50414f700 0x7fb50414efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:19.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.382+0000 7fb508a00640 1 -- 172.21.15.159:0/95590063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5041570e0 msgr2=0x7fb504157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:19.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.382+0000 7fb508a00640 1 --2- 172.21.15.159:0/95590063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5041570e0 0x7fb504157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:19.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.382+0000 7fb508a00640 1 -- 172.21.15.159:0/95590063 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50414f500 con 0x7fb4ec004800 2024-04-04T06:17:19.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.382+0000 7fb509400640 1 --2- 172.21.15.159:0/95590063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5041570e0 0x7fb504157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:19.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.382+0000 7fb503e00640 1 --2- 172.21.15.159:0/95590063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb50414f700 0x7fb50414efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:19.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.382+0000 7fb508a00640 1 --2- 172.21.15.159:0/95590063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4ec004800 0x7fb5040b4f80 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800d7d0 tx=0x7fb4f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:19.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.382+0000 7fb500c00640 1 -- 172.21.15.159:0/95590063 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8005070 con 0x7fb4ec004800 2024-04-04T06:17:19.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.383+0000 7fb500c00640 1 -- 172.21.15.159:0/95590063 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4f8005230 con 0x7fb4ec004800 2024-04-04T06:17:19.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.383+0000 7fb500c00640 1 -- 172.21.15.159:0/95590063 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8007030 con 0x7fb4ec004800 2024-04-04T06:17:19.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.383+0000 7fb50b400640 1 -- 172.21.15.159:0/95590063 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50415a650 con 0x7fb4ec004800 2024-04-04T06:17:19.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.383+0000 7fb50b400640 1 -- 172.21.15.159:0/95590063 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5040b5840 con 0x7fb4ec004800 2024-04-04T06:17:19.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.384+0000 7fb50b400640 1 -- 172.21.15.159:0/95590063 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5040685c0 con 0x7fb4ec004800 2024-04-04T06:17:19.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.385+0000 7fb500c00640 1 -- 172.21.15.159:0/95590063 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4f801a020 con 0x7fb4ec004800 2024-04-04T06:17:19.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.386+0000 7fb500c00640 1 --2- 172.21.15.159:0/95590063 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb4e406c8f0 0x7fb4e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:19.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.386+0000 7fb500c00640 1 -- 172.21.15.159:0/95590063 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7fb4f808b370 con 0x7fb4ec004800 2024-04-04T06:17:19.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.387+0000 7fb503e00640 1 --2- 172.21.15.159:0/95590063 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb4e406c8f0 0x7fb4e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:19.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.388+0000 7fb503e00640 1 --2- 172.21.15.159:0/95590063 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb4e406c8f0 0x7fb4e406edb0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0036000 tx=0x7fb4f0030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:19.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.391+0000 7fb500c00640 1 -- 172.21.15.159:0/95590063 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb50414f500 con 0x7fb4ec004800 2024-04-04T06:17:19.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.594+0000 7fb50b400640 1 -- 172.21.15.159:0/95590063 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb504161fa0 con 0x7fb4ec004800 2024-04-04T06:17:19.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.595+0000 7fb500c00640 1 -- 172.21.15.159:0/95590063 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) ==== 104+0+11 (secure 0 0 0) 0x7fb4f8059790 con 0x7fb4ec004800 2024-04-04T06:17:19.590 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:17:19.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.597+0000 7fb50b400640 1 -- 172.21.15.159:0/95590063 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb4e406c8f0 msgr2=0x7fb4e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:19.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.597+0000 7fb50b400640 1 --2- 172.21.15.159:0/95590063 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb4e406c8f0 0x7fb4e406edb0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0036000 tx=0x7fb4f0030000 comp rx=0 tx=0).stop 2024-04-04T06:17:19.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.597+0000 7fb50b400640 1 -- 172.21.15.159:0/95590063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4ec004800 msgr2=0x7fb5040b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:19.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.597+0000 7fb50b400640 1 --2- 172.21.15.159:0/95590063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4ec004800 0x7fb5040b4f80 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800d7d0 tx=0x7fb4f800dc90 comp rx=0 tx=0).stop 2024-04-04T06:17:19.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.598+0000 7fb50b400640 1 -- 172.21.15.159:0/95590063 shutdown_connections 2024-04-04T06:17:19.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.598+0000 7fb50b400640 1 --2- 172.21.15.159:0/95590063 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb4e406c8f0 0x7fb4e406edb0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:19.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.598+0000 7fb50b400640 1 --2- 172.21.15.159:0/95590063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5041570e0 0x7fb504157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:19.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.598+0000 7fb50b400640 1 --2- 172.21.15.159:0/95590063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb50414f700 0x7fb50414efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:19.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.598+0000 7fb50b400640 1 --2- 172.21.15.159:0/95590063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4ec004800 0x7fb5040b4f80 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:19.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.598+0000 7fb50b400640 1 -- 172.21.15.159:0/95590063 >> 172.21.15.159:0/95590063 conn(0x7fb5040d2240 msgr2=0x7fb5040d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:19.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.598+0000 7fb50b400640 1 -- 172.21.15.159:0/95590063 shutdown_connections 2024-04-04T06:17:19.594 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.598+0000 7fb50b400640 1 -- 172.21.15.159:0/95590063 wait complete. 2024-04-04T06:17:19.612 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:17:19.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.833+0000 7fa80ee00640 1 Processor -- start 2024-04-04T06:17:19.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.833+0000 7fa80ee00640 1 -- start start 2024-04-04T06:17:19.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.835+0000 7fa80ee00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa80815be20 0x7fa80806c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:19.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.835+0000 7fa80ee00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa80806cac0 0x7fa80806e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:19.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.835+0000 7fa80ee00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa808070690 con 0x7fa808144450 2024-04-04T06:17:19.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.835+0000 7fa80ee00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fa808144830 con 0x7fa80815be20 2024-04-04T06:17:19.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.835+0000 7fa80ee00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa80815e670 con 0x7fa80806cac0 2024-04-04T06:17:19.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.835+0000 7fa7ffe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa80815be20 0x7fa80806c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:19.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.835+0000 7fa7ffe00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa80815be20 0x7fa80806c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37428/0 (socket says 172.21.15.159:37428) 2024-04-04T06:17:19.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.835+0000 7fa7ffe00640 1 -- 172.21.15.159:0/944499987 learned_addr learned my addr 172.21.15.159:0/944499987 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:19.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.836+0000 7fa7ffe00640 1 -- 172.21.15.159:0/944499987 >> v2:172.21.15.178:3300/0 conn(0x7fa80806cac0 msgr2=0x7fa80806e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:19.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.836+0000 7fa7ffe00640 1 --2- 172.21.15.159:0/944499987 >> v2:172.21.15.178:3300/0 conn(0x7fa80806cac0 0x7fa80806e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:19.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.836+0000 7fa7ffe00640 1 -- 172.21.15.159:0/944499987 >> v1:172.21.15.178:6789/0 conn(0x7fa808144450 legacy=0x7fa80806ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:19.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.836+0000 7fa7ffe00640 1 -- 172.21.15.159:0/944499987 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80815e8c0 con 0x7fa80815be20 2024-04-04T06:17:19.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.837+0000 7fa7ffe00640 1 --2- 172.21.15.159:0/944499987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa80815be20 0x7fa80806c580 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fa80806ff00 tx=0x7fa7f002e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130c30443e39567e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:19.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.837+0000 7fa807400640 1 -- 172.21.15.159:0/944499987 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f003b070 con 0x7fa80815be20 2024-04-04T06:17:19.832 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.837+0000 7fa807400640 1 -- 172.21.15.159:0/944499987 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f002ee00 con 0x7fa80815be20 2024-04-04T06:17:19.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.837+0000 7fa807400640 1 -- 172.21.15.159:0/944499987 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0036660 con 0x7fa80815be20 2024-04-04T06:17:19.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.838+0000 7fa80ee00640 1 -- 172.21.15.159:0/944499987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa80815be20 msgr2=0x7fa80806c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:19.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.838+0000 7fa80ee00640 1 --2- 172.21.15.159:0/944499987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa80815be20 0x7fa80806c580 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fa80806ff00 tx=0x7fa7f002e050 comp rx=0 tx=0).stop 2024-04-04T06:17:19.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.838+0000 7fa80ee00640 1 -- 172.21.15.159:0/944499987 shutdown_connections 2024-04-04T06:17:19.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.838+0000 7fa80ee00640 1 --2- 172.21.15.159:0/944499987 >> v2:172.21.15.178:3300/0 conn(0x7fa80806cac0 0x7fa80806e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:19.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.838+0000 7fa80ee00640 1 --2- 172.21.15.159:0/944499987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa80815be20 0x7fa80806c580 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:19.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.838+0000 7fa80ee00640 1 -- 172.21.15.159:0/944499987 >> 172.21.15.159:0/944499987 conn(0x7fa808071e20 msgr2=0x7fa808072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:19.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.838+0000 7fa80ee00640 1 -- 172.21.15.159:0/944499987 shutdown_connections 2024-04-04T06:17:19.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.838+0000 7fa80ee00640 1 -- 172.21.15.159:0/944499987 wait complete. 2024-04-04T06:17:19.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.840+0000 7fa80ee00640 1 Processor -- start 2024-04-04T06:17:19.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.841+0000 7fa80ee00640 1 -- start start 2024-04-04T06:17:19.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.842+0000 7fa80ee00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa80806cac0 0x7fa808159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:19.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.842+0000 7fa80ee00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa808144450 0x7fa808162c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:19.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.842+0000 7fa80ee00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa80815be20 0x7fa80813ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:19.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.842+0000 7fa80ee00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa80815ee80 con 0x7fa80815be20 2024-04-04T06:17:19.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.842+0000 7fa80ee00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa80815f160 con 0x7fa808144450 2024-04-04T06:17:19.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.842+0000 7fa80ee00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa808067280 con 0x7fa80806cac0 2024-04-04T06:17:19.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.843+0000 7fa80ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa80815be20 0x7fa80813ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:19.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.843+0000 7fa80ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa80815be20 0x7fa80813ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37444/0 (socket says 172.21.15.159:37444) 2024-04-04T06:17:19.838 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.843+0000 7fa80ce00640 1 -- 172.21.15.159:0/3176405462 learned_addr learned my addr 172.21.15.159:0/3176405462 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:19.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.843+0000 7fa80ce00640 1 -- 172.21.15.159:0/3176405462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa80806cac0 msgr2=0x7fa808159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:19.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.843+0000 7fa807e00640 1 --2- 172.21.15.159:0/3176405462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa80806cac0 0x7fa808159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:19.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.843+0000 7fa7ffe00640 1 --2- 172.21.15.159:0/3176405462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa808144450 0x7fa808162c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:19.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.843+0000 7fa80ce00640 1 --2- 172.21.15.159:0/3176405462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa80806cac0 0x7fa808159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:19.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.843+0000 7fa80ce00640 1 -- 172.21.15.159:0/3176405462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa808144450 msgr2=0x7fa808162c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:19.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.843+0000 7fa80ce00640 1 --2- 172.21.15.159:0/3176405462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa808144450 0x7fa808162c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:19.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.843+0000 7fa80ce00640 1 -- 172.21.15.159:0/3176405462 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8081635c0 con 0x7fa80815be20 2024-04-04T06:17:19.839 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.843+0000 7fa80ce00640 1 --2- 172.21.15.159:0/3176405462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa80815be20 0x7fa80813ff90 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fa80000b970 tx=0x7fa80000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:19.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.844+0000 7fa804c00640 1 -- 172.21.15.159:0/3176405462 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800019070 con 0x7fa80815be20 2024-04-04T06:17:19.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.844+0000 7fa804c00640 1 -- 172.21.15.159:0/3176405462 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa80000cb60 con 0x7fa80815be20 2024-04-04T06:17:19.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.844+0000 7fa804c00640 1 -- 172.21.15.159:0/3176405462 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800014440 con 0x7fa80815be20 2024-04-04T06:17:19.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.844+0000 7fa80ee00640 1 -- 172.21.15.159:0/3176405462 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa808159250 con 0x7fa80815be20 2024-04-04T06:17:19.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.844+0000 7fa80ee00640 1 -- 172.21.15.159:0/3176405462 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa80815a270 con 0x7fa80815be20 2024-04-04T06:17:19.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.845+0000 7fa80ee00640 1 -- 172.21.15.159:0/3176405462 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa808068cd0 con 0x7fa80815be20 2024-04-04T06:17:19.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.846+0000 7fa804c00640 1 -- 172.21.15.159:0/3176405462 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa80001c070 con 0x7fa80815be20 2024-04-04T06:17:19.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.847+0000 7fa804c00640 1 --2- 172.21.15.159:0/3176405462 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa7e406c8f0 0x7fa7e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:19.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.847+0000 7fa804c00640 1 -- 172.21.15.159:0/3176405462 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7fa80008b8d0 con 0x7fa80815be20 2024-04-04T06:17:19.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.847+0000 7fa807e00640 1 --2- 172.21.15.159:0/3176405462 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa7e406c8f0 0x7fa7e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:19.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.849+0000 7fa807e00640 1 --2- 172.21.15.159:0/3176405462 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa7e406c8f0 0x7fa7e406edb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80015d0 tx=0x7fa7f8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:19.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:19.852+0000 7fa804c00640 1 -- 172.21.15.159:0/3176405462 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa800059cf0 con 0x7fa80815be20 2024-04-04T06:17:20.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.184+0000 7fa80ee00640 1 -- 172.21.15.159:0/3176405462 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa8080ccfd0 con 0x7fa80815be20 2024-04-04T06:17:20.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.185+0000 7fa804c00640 1 -- 172.21.15.159:0/3176405462 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) ==== 100+0+11 (secure 0 0 0) 0x7fa8000595f0 con 0x7fa80815be20 2024-04-04T06:17:20.180 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:17:20.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.187+0000 7fa80ee00640 1 -- 172.21.15.159:0/3176405462 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa7e406c8f0 msgr2=0x7fa7e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:20.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.187+0000 7fa80ee00640 1 --2- 172.21.15.159:0/3176405462 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa7e406c8f0 0x7fa7e406edb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80015d0 tx=0x7fa7f8009040 comp rx=0 tx=0).stop 2024-04-04T06:17:20.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.187+0000 7fa80ee00640 1 -- 172.21.15.159:0/3176405462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa80815be20 msgr2=0x7fa80813ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:20.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.187+0000 7fa80ee00640 1 --2- 172.21.15.159:0/3176405462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa80815be20 0x7fa80813ff90 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fa80000b970 tx=0x7fa80000be30 comp rx=0 tx=0).stop 2024-04-04T06:17:20.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.188+0000 7fa80ee00640 1 -- 172.21.15.159:0/3176405462 shutdown_connections 2024-04-04T06:17:20.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.188+0000 7fa80ee00640 1 --2- 172.21.15.159:0/3176405462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa80815be20 0x7fa80813ff90 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:20.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.188+0000 7fa80ee00640 1 --2- 172.21.15.159:0/3176405462 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa7e406c8f0 0x7fa7e406edb0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:20.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.188+0000 7fa80ee00640 1 --2- 172.21.15.159:0/3176405462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa808144450 0x7fa808162c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:20.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.188+0000 7fa80ee00640 1 --2- 172.21.15.159:0/3176405462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa80806cac0 0x7fa808159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:20.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.188+0000 7fa80ee00640 1 -- 172.21.15.159:0/3176405462 >> 172.21.15.159:0/3176405462 conn(0x7fa808071e20 msgr2=0x7fa8080b60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:20.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.188+0000 7fa80ee00640 1 -- 172.21.15.159:0/3176405462 shutdown_connections 2024-04-04T06:17:20.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.188+0000 7fa80ee00640 1 -- 172.21.15.159:0/3176405462 wait complete. 2024-04-04T06:17:20.199 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-04T06:17:20.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.395+0000 7f3aeb600640 1 Processor -- start 2024-04-04T06:17:20.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.395+0000 7f3aeb600640 1 -- start start 2024-04-04T06:17:20.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.396+0000 7f3aeb600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3ae415e6d0 0x7f3ae40d21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:20.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.396+0000 7f3aeb600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3ae40d0cb0 0x7f3ae40d4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:20.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.397+0000 7f3aeb600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ae40ceef0 con 0x7f3ae40d4590 2024-04-04T06:17:20.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.397+0000 7f3aeb600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f3ae415eab0 con 0x7f3ae415e6d0 2024-04-04T06:17:20.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.397+0000 7f3aeb600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3ae40d1220 con 0x7f3ae40d0cb0 2024-04-04T06:17:20.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.397+0000 7f3ae8c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3ae415e6d0 0x7f3ae40d21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:20.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.397+0000 7f3ae9600640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f3ae40d4590 0x7f3ae41621c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41030/0 (socket says 172.21.15.159:41030) 2024-04-04T06:17:20.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.397+0000 7f3ae9600640 1 -- 172.21.15.159:0/3345061618 learned_addr learned my addr 172.21.15.159:0/3345061618 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:20.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.397+0000 7f3adbe00640 1 --2- 172.21.15.159:0/3345061618 >> v2:172.21.15.178:3300/0 conn(0x7f3ae40d0cb0 0x7f3ae40d4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:20.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.397+0000 7f3ae8c00640 1 -- 172.21.15.159:0/3345061618 >> v2:172.21.15.178:3300/0 conn(0x7f3ae40d0cb0 msgr2=0x7f3ae40d4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:20.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.397+0000 7f3ae8c00640 1 --2- 172.21.15.159:0/3345061618 >> v2:172.21.15.178:3300/0 conn(0x7f3ae40d0cb0 0x7f3ae40d4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:20.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.397+0000 7f3ae8c00640 1 -- 172.21.15.159:0/3345061618 >> v1:172.21.15.159:6789/0 conn(0x7f3ae40d4590 legacy=0x7f3ae41621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:20.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.398+0000 7f3ae8c00640 1 -- 172.21.15.159:0/3345061618 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae40d1500 con 0x7f3ae415e6d0 2024-04-04T06:17:20.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.397+0000 7f3ae3e00640 1 -- 172.21.15.159:0/3345061618 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2783148736 0 0) 0x7f3ae40ceef0 con 0x7f3ae40d4590 2024-04-04T06:17:20.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.399+0000 7f3ae8c00640 1 --2- 172.21.15.159:0/3345061618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ae415e6d0 0x7f3ae40d21e0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f3ae409de60 tx=0x7f3ad402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4540b7d822c9fa12 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:20.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.399+0000 7f3ae3e00640 1 -- 172.21.15.159:0/3345061618 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4035020 con 0x7f3ae415e6d0 2024-04-04T06:17:20.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.399+0000 7f3ae3e00640 1 -- 172.21.15.159:0/3345061618 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ad4031830 con 0x7f3ae415e6d0 2024-04-04T06:17:20.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.399+0000 7f3ae3e00640 1 -- 172.21.15.159:0/3345061618 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4031c00 con 0x7f3ae415e6d0 2024-04-04T06:17:20.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.399+0000 7f3aeb600640 1 -- 172.21.15.159:0/3345061618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ae415e6d0 msgr2=0x7f3ae40d21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:20.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.399+0000 7f3aeb600640 1 --2- 172.21.15.159:0/3345061618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ae415e6d0 0x7f3ae40d21e0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f3ae409de60 tx=0x7f3ad402b8d0 comp rx=0 tx=0).stop 2024-04-04T06:17:20.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.400+0000 7f3aeb600640 1 -- 172.21.15.159:0/3345061618 shutdown_connections 2024-04-04T06:17:20.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.400+0000 7f3aeb600640 1 --2- 172.21.15.159:0/3345061618 >> v2:172.21.15.178:3300/0 conn(0x7f3ae40d0cb0 0x7f3ae40d4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:20.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.400+0000 7f3aeb600640 1 --2- 172.21.15.159:0/3345061618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ae415e6d0 0x7f3ae40d21e0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:20.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.400+0000 7f3aeb600640 1 -- 172.21.15.159:0/3345061618 >> 172.21.15.159:0/3345061618 conn(0x7f3ae409d7b0 msgr2=0x7f3ae408bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:20.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.400+0000 7f3aeb600640 1 -- 172.21.15.159:0/3345061618 shutdown_connections 2024-04-04T06:17:20.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.400+0000 7f3aeb600640 1 -- 172.21.15.159:0/3345061618 wait complete. 2024-04-04T06:17:20.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.401+0000 7f3aeb600640 1 Processor -- start 2024-04-04T06:17:20.396 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.401+0000 7f3aeb600640 1 -- start start 2024-04-04T06:17:20.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.402+0000 7f3aeb600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ae40d0cb0 0x7f3ae41f3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:20.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.402+0000 7f3aeb600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ae40d4590 0x7f3ae40b96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:20.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.402+0000 7f3aeb600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ae415e6d0 0x7f3ae40b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:20.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.402+0000 7f3aeb600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae40d4a20 con 0x7f3ae40d4590 2024-04-04T06:17:20.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.402+0000 7f3aeb600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae40d1ac0 con 0x7f3ae415e6d0 2024-04-04T06:17:20.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.402+0000 7f3aeb600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae40d2720 con 0x7f3ae40d0cb0 2024-04-04T06:17:20.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.403+0000 7f3adbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ae40d4590 0x7f3ae40b96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:20.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.403+0000 7f3adbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ae40d4590 0x7f3ae40b96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37462/0 (socket says 172.21.15.159:37462) 2024-04-04T06:17:20.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.403+0000 7f3adbe00640 1 -- 172.21.15.159:0/2833541569 learned_addr learned my addr 172.21.15.159:0/2833541569 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:20.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.403+0000 7f3ae9600640 1 --2- 172.21.15.159:0/2833541569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ae415e6d0 0x7f3ae40b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:20.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.403+0000 7f3ae8c00640 1 --2- 172.21.15.159:0/2833541569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ae40d0cb0 0x7f3ae41f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:20.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.403+0000 7f3adbe00640 1 -- 172.21.15.159:0/2833541569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ae40d0cb0 msgr2=0x7f3ae41f3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:20.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.403+0000 7f3adbe00640 1 --2- 172.21.15.159:0/2833541569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ae40d0cb0 0x7f3ae41f3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:20.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.403+0000 7f3adbe00640 1 -- 172.21.15.159:0/2833541569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ae415e6d0 msgr2=0x7f3ae40b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:20.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.403+0000 7f3adbe00640 1 --2- 172.21.15.159:0/2833541569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ae415e6d0 0x7f3ae40b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:20.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.403+0000 7f3adbe00640 1 -- 172.21.15.159:0/2833541569 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae40c8220 con 0x7f3ae40d4590 2024-04-04T06:17:20.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.403+0000 7f3ae8c00640 1 --2- 172.21.15.159:0/2833541569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ae40d0cb0 0x7f3ae41f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:20.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.404+0000 7f3adbe00640 1 --2- 172.21.15.159:0/2833541569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ae40d4590 0x7f3ae40b96b0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00d7d0 tx=0x7f3acc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:20.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.404+0000 7f3ae1600640 1 -- 172.21.15.159:0/2833541569 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc018020 con 0x7f3ae40d4590 2024-04-04T06:17:20.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.404+0000 7f3ae1600640 1 -- 172.21.15.159:0/2833541569 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3acc0096c0 con 0x7f3ae40d4590 2024-04-04T06:17:20.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.404+0000 7f3aeb600640 1 -- 172.21.15.159:0/2833541569 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae415bfd0 con 0x7f3ae40d4590 2024-04-04T06:17:20.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.404+0000 7f3ae1600640 1 -- 172.21.15.159:0/2833541569 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc009ab0 con 0x7f3ae40d4590 2024-04-04T06:17:20.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.404+0000 7f3aeb600640 1 -- 172.21.15.159:0/2833541569 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae40c84e0 con 0x7f3ae40d4590 2024-04-04T06:17:20.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.405+0000 7f3aeb600640 1 -- 172.21.15.159:0/2833541569 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae40685c0 con 0x7f3ae40d4590 2024-04-04T06:17:20.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.407+0000 7f3ae1600640 1 -- 172.21.15.159:0/2833541569 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3acc01b020 con 0x7f3ae40d4590 2024-04-04T06:17:20.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.408+0000 7f3ae1600640 1 --2- 172.21.15.159:0/2833541569 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3ac006c8f0 0x7f3ac006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:20.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.409+0000 7f3ae1600640 1 -- 172.21.15.159:0/2833541569 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f3acc08c0d0 con 0x7f3ae40d4590 2024-04-04T06:17:20.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.409+0000 7f3ae8c00640 1 --2- 172.21.15.159:0/2833541569 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3ac006c8f0 0x7f3ac006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:20.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.410+0000 7f3ae8c00640 1 --2- 172.21.15.159:0/2833541569 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3ac006c8f0 0x7f3ac006edb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4036000 tx=0x7f3ad4008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:20.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.413+0000 7f3ae1600640 1 -- 172.21.15.159:0/2833541569 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3acc022020 con 0x7f3ae40d4590 2024-04-04T06:17:20.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.628+0000 7f3aeb600640 1 -- 172.21.15.159:0/2833541569 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3ae406c670 con 0x7f3ae40d4590 2024-04-04T06:17:20.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.628+0000 7f3ae1600640 1 -- 172.21.15.159:0/2833541569 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v113) ==== 80+0+0 (secure 0 0 0) 0x7f3acc011340 con 0x7f3ae40d4590 2024-04-04T06:17:20.624 INFO:teuthology.orchestra.run.smithi159.stderr:listed 0 entries 2024-04-04T06:17:20.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.631+0000 7f3aeb600640 1 -- 172.21.15.159:0/2833541569 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3ac006c8f0 msgr2=0x7f3ac006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:20.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.631+0000 7f3aeb600640 1 --2- 172.21.15.159:0/2833541569 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3ac006c8f0 0x7f3ac006edb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4036000 tx=0x7f3ad4008030 comp rx=0 tx=0).stop 2024-04-04T06:17:20.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.631+0000 7f3aeb600640 1 -- 172.21.15.159:0/2833541569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ae40d4590 msgr2=0x7f3ae40b96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:20.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.631+0000 7f3aeb600640 1 --2- 172.21.15.159:0/2833541569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ae40d4590 0x7f3ae40b96b0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00d7d0 tx=0x7f3acc00dc90 comp rx=0 tx=0).stop 2024-04-04T06:17:20.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.632+0000 7f3aeb600640 1 -- 172.21.15.159:0/2833541569 shutdown_connections 2024-04-04T06:17:20.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.632+0000 7f3aeb600640 1 --2- 172.21.15.159:0/2833541569 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3ac006c8f0 0x7f3ac006edb0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:20.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.632+0000 7f3aeb600640 1 --2- 172.21.15.159:0/2833541569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ae415e6d0 0x7f3ae40b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:20.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.632+0000 7f3aeb600640 1 --2- 172.21.15.159:0/2833541569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ae40d4590 0x7f3ae40b96b0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:20.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.632+0000 7f3aeb600640 1 --2- 172.21.15.159:0/2833541569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ae40d0cb0 0x7f3ae41f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:20.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.632+0000 7f3aeb600640 1 -- 172.21.15.159:0/2833541569 >> 172.21.15.159:0/2833541569 conn(0x7f3ae409d7b0 msgr2=0x7f3ae40d2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:20.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.632+0000 7f3aeb600640 1 -- 172.21.15.159:0/2833541569 shutdown_connections 2024-04-04T06:17:20.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:20.632+0000 7f3aeb600640 1 -- 172.21.15.159:0/2833541569 wait complete. 2024-04-04T06:17:20.646 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-04T06:17:20.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:20.646 DEBUG:teuthology.orchestra.run.smithi159:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-04T06:17:20.866 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:17:20.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:20.866 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-04T06:17:20.915 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:17:20.915 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:17:20.995 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:17:20.995 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:17:20.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:20.995 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-04T06:17:21.079 INFO:teuthology.orchestra.run.smithi159.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-04T06:17:21.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-04T06:17:21.080 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-04T06:17:21.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.330+0000 7efe6d600640 1 Processor -- start 2024-04-04T06:17:21.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.330+0000 7efe6d600640 1 -- start start 2024-04-04T06:17:21.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.332+0000 7efe6d600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe680d02a0 0x7efe68165c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:21.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.332+0000 7efe6d600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe68071460 con 0x7efe680cf1b0 2024-04-04T06:17:21.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.332+0000 7efe6d600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe680721b0 con 0x7efe681442c0 2024-04-04T06:17:21.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.332+0000 7efe6d600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efe68071d20 con 0x7efe680d02a0 2024-04-04T06:17:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.333+0000 7efe67400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7efe680cf1b0 0x7efe680cfb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41042/0 (socket says 172.21.15.159:41042) 2024-04-04T06:17:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.333+0000 7efe67400640 1 -- 172.21.15.159:0/256892040 learned_addr learned my addr 172.21.15.159:0/256892040 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.333+0000 7efe6cc00640 1 --2- 172.21.15.159:0/256892040 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe680d02a0 0x7efe68165c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.333+0000 7efe66a00640 1 -- 172.21.15.159:0/256892040 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3309014748 0 0) 0x7efe68071460 con 0x7efe680cf1b0 2024-04-04T06:17:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.333+0000 7efe66a00640 1 -- 172.21.15.159:0/256892040 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe4c003520 con 0x7efe680cf1b0 2024-04-04T06:17:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.333+0000 7efe66a00640 1 -- 172.21.15.159:0/256892040 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2801476327 0 0) 0x7efe4c003520 con 0x7efe680cf1b0 2024-04-04T06:17:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.333+0000 7efe66a00640 1 -- 172.21.15.159:0/256892040 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe680d02a0 msgr2=0x7efe68165c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.333+0000 7efe66a00640 1 --2- 172.21.15.159:0/256892040 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe680d02a0 0x7efe68165c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.333+0000 7efe66a00640 1 -- 172.21.15.159:0/256892040 >> v1:172.21.15.178:6789/0 conn(0x7efe681442c0 legacy=0x7efe680ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.333+0000 7efe66a00640 1 -- 172.21.15.159:0/256892040 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe680716d0 con 0x7efe680cf1b0 2024-04-04T06:17:21.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.333+0000 7efe66a00640 1 -- 172.21.15.159:0/256892040 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7efe54001670 con 0x7efe680cf1b0 2024-04-04T06:17:21.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.334+0000 7efe66a00640 1 -- 172.21.15.159:0/256892040 >> v1:172.21.15.159:6789/0 conn(0x7efe680cf1b0 legacy=0x7efe680cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:21.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.334+0000 7efe66a00640 1 --2- 172.21.15.159:0/256892040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe4c0039e0 0x7efe4c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:21.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.334+0000 7efe66a00640 1 --2- 172.21.15.159:0/256892040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe4c004800 0x7efe4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:21.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.334+0000 7efe66a00640 1 --2- 172.21.15.159:0/256892040 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe680d02a0 0x7efe4c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:21.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.334+0000 7efe66a00640 1 -- 172.21.15.159:0/256892040 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efe68071d20 con 0x7efe4c004800 2024-04-04T06:17:21.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.334+0000 7efe66a00640 1 -- 172.21.15.159:0/256892040 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c00bbb0 con 0x7efe4c0039e0 2024-04-04T06:17:21.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.334+0000 7efe66a00640 1 -- 172.21.15.159:0/256892040 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efe4c00bdb0 con 0x7efe680d02a0 2024-04-04T06:17:21.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.334+0000 7efe67400640 1 --2- 172.21.15.159:0/256892040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe4c004800 0x7efe4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:21.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.334+0000 7efe6cc00640 1 --2- 172.21.15.159:0/256892040 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe680d02a0 0x7efe4c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:21.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.334+0000 7efe67e00640 1 --2- 172.21.15.159:0/256892040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe4c0039e0 0x7efe4c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:21.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.334+0000 7efe67400640 1 -- 172.21.15.159:0/256892040 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe680d02a0 msgr2=0x7efe4c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:21.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.334+0000 7efe67400640 1 --2- 172.21.15.159:0/256892040 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe680d02a0 0x7efe4c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:21.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.335+0000 7efe67400640 1 -- 172.21.15.159:0/256892040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe4c0039e0 msgr2=0x7efe4c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:21.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.335+0000 7efe67400640 1 --2- 172.21.15.159:0/256892040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe4c0039e0 0x7efe4c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:21.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.335+0000 7efe67400640 1 -- 172.21.15.159:0/256892040 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4c003520 con 0x7efe4c004800 2024-04-04T06:17:21.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.335+0000 7efe67e00640 1 --2- 172.21.15.159:0/256892040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe4c0039e0 0x7efe4c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:21.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.336+0000 7efe67400640 1 --2- 172.21.15.159:0/256892040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe4c004800 0x7efe4c004c80 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7efe5400c240 tx=0x7efe4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d5ac734f0663d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:21.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.336+0000 7efe66a00640 1 -- 172.21.15.159:0/256892040 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54007d10 con 0x7efe4c004800 2024-04-04T06:17:21.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.336+0000 7efe66a00640 1 -- 172.21.15.159:0/256892040 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe54030070 con 0x7efe4c004800 2024-04-04T06:17:21.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.336+0000 7efe66a00640 1 -- 172.21.15.159:0/256892040 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5402cd40 con 0x7efe4c004800 2024-04-04T06:17:21.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.336+0000 7efe6d600640 1 -- 172.21.15.159:0/256892040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe4c004800 msgr2=0x7efe4c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:21.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.336+0000 7efe6d600640 1 --2- 172.21.15.159:0/256892040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe4c004800 0x7efe4c004c80 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7efe5400c240 tx=0x7efe4c005830 comp rx=0 tx=0).stop 2024-04-04T06:17:21.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.336+0000 7efe6cc00640 1 -- 172.21.15.159:0/256892040 reap_dead start 2024-04-04T06:17:21.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.337+0000 7efe6d600640 1 -- 172.21.15.159:0/256892040 shutdown_connections 2024-04-04T06:17:21.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.337+0000 7efe6d600640 1 -- 172.21.15.159:0/256892040 >> 172.21.15.159:0/256892040 conn(0x7efe680ccad0 msgr2=0x7efe680cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:21.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.337+0000 7efe6d600640 1 -- 172.21.15.159:0/256892040 shutdown_connections 2024-04-04T06:17:21.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.337+0000 7efe6d600640 1 -- 172.21.15.159:0/256892040 wait complete. 2024-04-04T06:17:21.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.339+0000 7efe6d600640 1 Processor -- start 2024-04-04T06:17:21.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.339+0000 7efe6d600640 1 -- start start 2024-04-04T06:17:21.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.341+0000 7efe6d600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe4c004800 0x7efe681f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:21.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.341+0000 7efe6d600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe681e3700 0x7efe6815e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:21.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.341+0000 7efe6d600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe681f69c0 0x7efe680b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:21.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.341+0000 7efe6d600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efe681446a0 con 0x7efe4c004800 2024-04-04T06:17:21.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.341+0000 7efe6d600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe680d0770 con 0x7efe681f69c0 2024-04-04T06:17:21.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.341+0000 7efe6d600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7efe681e4810 con 0x7efe681e3700 2024-04-04T06:17:21.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.341+0000 7efe67e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe4c004800 0x7efe681f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:21.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.341+0000 7efe67e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe4c004800 0x7efe681f3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37476/0 (socket says 172.21.15.159:37476) 2024-04-04T06:17:21.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.341+0000 7efe67e00640 1 -- 172.21.15.159:0/305648602 learned_addr learned my addr 172.21.15.159:0/305648602 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:21.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.341+0000 7efe67400640 1 --2- 172.21.15.159:0/305648602 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe681e3700 0x7efe6815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:21.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.342+0000 7efe67e00640 1 -- 172.21.15.159:0/305648602 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe681e3700 msgr2=0x7efe6815e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:21.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.342+0000 7efe67e00640 1 --2- 172.21.15.159:0/305648602 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe681e3700 0x7efe6815e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:21.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.342+0000 7efe67e00640 1 -- 172.21.15.159:0/305648602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe681f69c0 msgr2=0x7efe680b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:21.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.342+0000 7efe67e00640 1 --2- 172.21.15.159:0/305648602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe681f69c0 0x7efe680b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:21.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.342+0000 7efe67e00640 1 -- 172.21.15.159:0/305648602 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe681f4230 con 0x7efe4c004800 2024-04-04T06:17:21.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.342+0000 7efe67e00640 1 --2- 172.21.15.159:0/305648602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe4c004800 0x7efe681f3cf0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7efe5800db10 tx=0x7efe580020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:21.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.342+0000 7efe5fe00640 1 -- 172.21.15.159:0/305648602 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe58018070 con 0x7efe4c004800 2024-04-04T06:17:21.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.342+0000 7efe5fe00640 1 -- 172.21.15.159:0/305648602 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe58003da0 con 0x7efe4c004800 2024-04-04T06:17:21.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.342+0000 7efe5fe00640 1 -- 172.21.15.159:0/305648602 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe58006d70 con 0x7efe4c004800 2024-04-04T06:17:21.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.342+0000 7efe6d600640 1 -- 172.21.15.159:0/305648602 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe681e3c00 con 0x7efe4c004800 2024-04-04T06:17:21.339 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.342+0000 7efe6d600640 1 -- 172.21.15.159:0/305648602 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe681e41c0 con 0x7efe4c004800 2024-04-04T06:17:21.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.344+0000 7efe6d600640 1 -- 172.21.15.159:0/305648602 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe68075940 con 0x7efe4c004800 2024-04-04T06:17:21.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.344+0000 7efe5fe00640 1 -- 172.21.15.159:0/305648602 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe5801b070 con 0x7efe4c004800 2024-04-04T06:17:21.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.345+0000 7efe5fe00640 1 --2- 172.21.15.159:0/305648602 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efe4806c8f0 0x7efe4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:21.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.345+0000 7efe5fe00640 1 -- 172.21.15.159:0/305648602 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7efe5808b3f0 con 0x7efe4c004800 2024-04-04T06:17:21.341 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.346+0000 7efe67400640 1 --2- 172.21.15.159:0/305648602 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efe4806c8f0 0x7efe4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:21.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.347+0000 7efe67400640 1 --2- 172.21.15.159:0/305648602 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efe4806c8f0 0x7efe4806edb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7efe54034000 tx=0x7efe54009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:21.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.350+0000 7efe5fe00640 1 -- 172.21.15.159:0/305648602 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7efe58059820 con 0x7efe4c004800 2024-04-04T06:17:21.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.594+0000 7efe6d600640 1 -- 172.21.15.159:0/305648602 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efe680b6040 con 0x7efe4c004800 2024-04-04T06:17:21.590 INFO:tasks.ceph.mon.a.smithi159.stderr:2024-04-04T06:17:21.596+0000 7f1995c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-04T06:17:21.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.598+0000 7efe5fe00640 1 -- 172.21.15.159:0/305648602 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v102) ==== 161+0+0 (secure 0 0 0) 0x7efe58059120 con 0x7efe4c004800 2024-04-04T06:17:21.593 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-04T06:17:21.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.602+0000 7efe6d600640 1 -- 172.21.15.159:0/305648602 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efe4806c8f0 msgr2=0x7efe4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:21.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.602+0000 7efe6d600640 1 --2- 172.21.15.159:0/305648602 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efe4806c8f0 0x7efe4806edb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7efe54034000 tx=0x7efe54009030 comp rx=0 tx=0).stop 2024-04-04T06:17:21.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.602+0000 7efe6d600640 1 -- 172.21.15.159:0/305648602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe4c004800 msgr2=0x7efe681f3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:21.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.602+0000 7efe6d600640 1 --2- 172.21.15.159:0/305648602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe4c004800 0x7efe681f3cf0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7efe5800db10 tx=0x7efe580020c0 comp rx=0 tx=0).stop 2024-04-04T06:17:21.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.603+0000 7efe6d600640 1 -- 172.21.15.159:0/305648602 shutdown_connections 2024-04-04T06:17:21.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.603+0000 7efe6d600640 1 --2- 172.21.15.159:0/305648602 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7efe4806c8f0 0x7efe4806edb0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:21.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.603+0000 7efe6d600640 1 --2- 172.21.15.159:0/305648602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe681f69c0 0x7efe680b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:21.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.603+0000 7efe6d600640 1 --2- 172.21.15.159:0/305648602 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe681e3700 0x7efe6815e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:21.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.603+0000 7efe6d600640 1 --2- 172.21.15.159:0/305648602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe4c004800 0x7efe681f3cf0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:21.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.603+0000 7efe6d600640 1 -- 172.21.15.159:0/305648602 >> 172.21.15.159:0/305648602 conn(0x7efe680ccad0 msgr2=0x7efe680b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:21.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.603+0000 7efe6d600640 1 -- 172.21.15.159:0/305648602 shutdown_connections 2024-04-04T06:17:21.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:21.603+0000 7efe6d600640 1 -- 172.21.15.159:0/305648602 wait complete. 2024-04-04T06:17:21.616 INFO:tasks.ceph.mds.a.smithi159.stderr: -8759> 2024-04-04T06:17:02.756+0000 7f4abd600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:17:21.621 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-04T06:17:21.627 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-04T06:17:21.694 INFO:tasks.ceph.mds.a.smithi159.stdout:starting mds.a at 2024-04-04T06:17:21.759 INFO:teuthology.orchestra.run.smithi159.stdout:[global] 2024-04-04T06:17:21.759 INFO:teuthology.orchestra.run.smithi159.stdout: chdir = "" 2024-04-04T06:17:21.759 INFO:teuthology.orchestra.run.smithi159.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-04T06:17:21.759 INFO:teuthology.orchestra.run.smithi159.stdout: auth supported = cephx 2024-04-04T06:17:21.759 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.759 INFO:teuthology.orchestra.run.smithi159.stdout: filestore xattr use omap = true 2024-04-04T06:17:21.759 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.759 INFO:teuthology.orchestra.run.smithi159.stdout: mon clock drift allowed = 1.000 2024-04-04T06:17:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: osd crush chooseleaf type = 0 2024-04-04T06:17:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: auth debug = true 2024-04-04T06:17:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: ms die on old message = true 2024-04-04T06:17:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: ms die on bug = true 2024-04-04T06:17:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-04T06:17:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: mon pg warn max object skew = 0 2024-04-04T06:17:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: # disable pg_autoscaler by default for new pools 2024-04-04T06:17:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-04T06:17:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.760 INFO:teuthology.orchestra.run.smithi159.stdout: osd pool default size = 2 2024-04-04T06:17:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd allow primary affinity = true 2024-04-04T06:17:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd allow pg remap = true 2024-04-04T06:17:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on legacy crush tunables = false 2024-04-04T06:17:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on crush straw calc version zero = false 2024-04-04T06:17:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on no sortbitwise = false 2024-04-04T06:17:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on osd down out interval zero = false 2024-04-04T06:17:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on too few osds = false 2024-04-04T06:17:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-04T06:17:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-04T06:17:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: mon_allow_pool_size_one = true 2024-04-04T06:17:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-04T06:17:21.761 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: osd default data pool replay window = 5 2024-04-04T06:17:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: mon allow pool delete = true 2024-04-04T06:17:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: mon cluster log file level = debug 2024-04-04T06:17:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: debug asserts on shutdown = true 2024-04-04T06:17:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: mon health detail to clog = false 2024-04-04T06:17:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: mon host = "172.21.15.159,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-04-04T06:17:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: ms die on skipped message = False 2024-04-04T06:17:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: fsid = 30311045-f714-48fa-832b-4cb2ba8bf0bf 2024-04-04T06:17:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.762 INFO:teuthology.orchestra.run.smithi159.stdout:[osd] 2024-04-04T06:17:21.762 INFO:teuthology.orchestra.run.smithi159.stdout: osd journal size = 100 2024-04-04T06:17:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: osd scrub load threshold = 5.0 2024-04-04T06:17:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: osd scrub max interval = 600 2024-04-04T06:17:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: osd mclock profile = high_recovery_ops 2024-04-04T06:17:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: osd recover clone overlap = true 2024-04-04T06:17:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: osd recovery max chunk = 1048576 2024-04-04T06:17:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug shutdown = true 2024-04-04T06:17:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug op order = true 2024-04-04T06:17:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug verify stray on activate = true 2024-04-04T06:17:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug trim objects = true 2024-04-04T06:17:21.763 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: osd open classes on start = true 2024-04-04T06:17:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug pg log writeout = true 2024-04-04T06:17:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: osd deep scrub update digest min age = 30 2024-04-04T06:17:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: osd map max advance = 10 2024-04-04T06:17:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: journal zero on create = true 2024-04-04T06:17:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.765 INFO:teuthology.orchestra.run.smithi159.stdout: filestore ondisk finisher threads = 3 2024-04-04T06:17:21.765 INFO:teuthology.orchestra.run.smithi159.stdout: filestore apply finisher threads = 3 2024-04-04T06:17:21.765 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.765 INFO:teuthology.orchestra.run.smithi159.stdout: bdev debug aio = true 2024-04-04T06:17:21.765 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug misdirected ops = true 2024-04-04T06:17:21.765 INFO:teuthology.orchestra.run.smithi159.stdout: bdev async discard = True 2024-04-04T06:17:21.765 INFO:teuthology.orchestra.run.smithi159.stdout: bdev enable discard = True 2024-04-04T06:17:21.765 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore allocator = bitmap 2024-04-04T06:17:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore block size = 96636764160 2024-04-04T06:17:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore fsck on mount = True 2024-04-04T06:17:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: debug bluefs = 1/20 2024-04-04T06:17:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: debug bluestore = 1/20 2024-04-04T06:17:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:17:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: debug osd = 20 2024-04-04T06:17:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: debug rocksdb = 4/10 2024-04-04T06:17:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd backfillfull_ratio = 0.85 2024-04-04T06:17:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd full ratio = 0.9 2024-04-04T06:17:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd nearfull ratio = 0.8 2024-04-04T06:17:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: osd failsafe full ratio = 0.95 2024-04-04T06:17:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: osd objectstore = bluestore 2024-04-04T06:17:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: osd op complaint time = 180 2024-04-04T06:17:21.766 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.767 INFO:teuthology.orchestra.run.smithi159.stdout:[mgr] 2024-04-04T06:17:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:17:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: debug mgr = 20 2024-04-04T06:17:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: debug mon = 20 2024-04-04T06:17:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: debug auth = 20 2024-04-04T06:17:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min pgs per osd = 4 2024-04-04T06:17:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min bytes per osd = 10 2024-04-04T06:17:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: mgr/telemetry/nag = false 2024-04-04T06:17:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: debug client = 20 2024-04-04T06:17:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.767 INFO:teuthology.orchestra.run.smithi159.stdout:[mon] 2024-04-04T06:17:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:17:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: debug mon = 20 2024-04-04T06:17:21.767 INFO:teuthology.orchestra.run.smithi159.stdout: debug paxos = 20 2024-04-04T06:17:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: debug auth = 20 2024-04-04T06:17:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: mon data avail warn = 5 2024-04-04T06:17:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: mon mgr mkfs grace = 240 2024-04-04T06:17:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min pgs per osd = 4 2024-04-04T06:17:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd reporter subtree level = osd 2024-04-04T06:17:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd prime pg temp = true 2024-04-04T06:17:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min bytes per osd = 10 2024-04-04T06:17:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-04T06:17:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: auth mon ticket ttl = 660 # 11m 2024-04-04T06:17:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: auth service ticket ttl = 240 # 4m 2024-04-04T06:17:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: # don't complain about insecure global_id in the test suite 2024-04-04T06:17:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-04T06:17:21.768 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-04T06:17:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: # 1m isn't quite enough 2024-04-04T06:17:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: mon_down_mkfs_grace = 2m 2024-04-04T06:17:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_filestore_osds = false 2024-04-04T06:17:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: mon down mkfs grace = 300 2024-04-04T06:17:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: mon op complaint time = 120 2024-04-04T06:17:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:21.769 INFO:teuthology.orchestra.run.smithi159.stdout:[client] 2024-04-04T06:17:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: rgw cache enabled = true 2024-04-04T06:17:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: rgw enable ops log = true 2024-04-04T06:17:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: rgw enable usage log = true 2024-04-04T06:17:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-04T06:17:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-04T06:17:21.769 INFO:teuthology.orchestra.run.smithi159.stdout: client mount timeout = 600 2024-04-04T06:17:21.770 INFO:teuthology.orchestra.run.smithi159.stdout: debug client = 20 2024-04-04T06:17:21.770 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:17:21.770 INFO:teuthology.orchestra.run.smithi159.stdout: pid file = "" 2024-04-04T06:17:21.770 INFO:teuthology.orchestra.run.smithi159.stdout: rados mon op timeout = 900 2024-04-04T06:17:21.770 INFO:teuthology.orchestra.run.smithi159.stdout: rados osd op timeout = 900 2024-04-04T06:17:21.770 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.a] 2024-04-04T06:17:21.770 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.b] 2024-04-04T06:17:21.770 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.c] 2024-04-04T06:17:21.770 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.a] 2024-04-04T06:17:21.770 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.c] 2024-04-04T06:17:21.770 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.b] 2024-04-04T06:17:21.770 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.d] 2024-04-04T06:17:21.770 INFO:teuthology.orchestra.run.smithi159.stdout:[mds] 2024-04-04T06:17:21.770 INFO:teuthology.orchestra.run.smithi159.stdout: debug mds = 20 2024-04-04T06:17:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: debug mds balancer = 20 2024-04-04T06:17:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:17:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: mds debug frag = True 2024-04-04T06:17:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: mds debug scatterstat = True 2024-04-04T06:17:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: mds op complaint time = 180 2024-04-04T06:17:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: mds verify scatter = False 2024-04-04T06:17:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: osd op complaint time = 180 2024-04-04T06:17:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: rados mon op timeout = 900 2024-04-04T06:17:21.771 INFO:teuthology.orchestra.run.smithi159.stdout: rados osd op timeout = 900 2024-04-04T06:17:21.793 INFO:teuthology.orchestra.run.smithi178.stdout:[global] 2024-04-04T06:17:21.793 INFO:teuthology.orchestra.run.smithi178.stdout: chdir = "" 2024-04-04T06:17:21.794 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-04T06:17:21.794 INFO:teuthology.orchestra.run.smithi178.stdout: auth supported = cephx 2024-04-04T06:17:21.794 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.794 INFO:teuthology.orchestra.run.smithi178.stdout: filestore xattr use omap = true 2024-04-04T06:17:21.794 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.794 INFO:teuthology.orchestra.run.smithi178.stdout: mon clock drift allowed = 1.000 2024-04-04T06:17:21.794 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.794 INFO:teuthology.orchestra.run.smithi178.stdout: osd crush chooseleaf type = 0 2024-04-04T06:17:21.794 INFO:teuthology.orchestra.run.smithi178.stdout: auth debug = true 2024-04-04T06:17:21.794 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.794 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on old message = true 2024-04-04T06:17:21.794 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on bug = true 2024-04-04T06:17:21.794 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.795 INFO:teuthology.orchestra.run.smithi178.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-04T06:17:21.795 INFO:teuthology.orchestra.run.smithi178.stdout: mon pg warn max object skew = 0 2024-04-04T06:17:21.795 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.795 INFO:teuthology.orchestra.run.smithi178.stdout: # disable pg_autoscaler by default for new pools 2024-04-04T06:17:21.795 INFO:teuthology.orchestra.run.smithi178.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-04T06:17:21.795 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.795 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default size = 2 2024-04-04T06:17:21.795 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.795 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow primary affinity = true 2024-04-04T06:17:21.795 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow pg remap = true 2024-04-04T06:17:21.795 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on legacy crush tunables = false 2024-04-04T06:17:21.795 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on crush straw calc version zero = false 2024-04-04T06:17:21.796 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on no sortbitwise = false 2024-04-04T06:17:21.796 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on osd down out interval zero = false 2024-04-04T06:17:21.796 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on too few osds = false 2024-04-04T06:17:21.796 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-04T06:17:21.796 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-04T06:17:21.797 INFO:teuthology.orchestra.run.smithi178.stdout: mon_allow_pool_size_one = true 2024-04-04T06:17:21.797 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.797 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-04T06:17:21.797 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.797 INFO:teuthology.orchestra.run.smithi178.stdout: osd default data pool replay window = 5 2024-04-04T06:17:21.797 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.797 INFO:teuthology.orchestra.run.smithi178.stdout: mon allow pool delete = true 2024-04-04T06:17:21.797 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.797 INFO:teuthology.orchestra.run.smithi178.stdout: mon cluster log file level = debug 2024-04-04T06:17:21.797 INFO:teuthology.orchestra.run.smithi178.stdout: debug asserts on shutdown = true 2024-04-04T06:17:21.797 INFO:teuthology.orchestra.run.smithi178.stdout: mon health detail to clog = false 2024-04-04T06:17:21.797 INFO:teuthology.orchestra.run.smithi178.stdout: mon host = "172.21.15.159,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-04-04T06:17:21.797 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on skipped message = False 2024-04-04T06:17:21.797 INFO:teuthology.orchestra.run.smithi178.stdout: fsid = 30311045-f714-48fa-832b-4cb2ba8bf0bf 2024-04-04T06:17:21.798 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.798 INFO:teuthology.orchestra.run.smithi178.stdout:[osd] 2024-04-04T06:17:21.798 INFO:teuthology.orchestra.run.smithi178.stdout: osd journal size = 100 2024-04-04T06:17:21.798 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.798 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub load threshold = 5.0 2024-04-04T06:17:21.798 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub max interval = 600 2024-04-04T06:17:21.798 INFO:teuthology.orchestra.run.smithi178.stdout: osd mclock profile = high_recovery_ops 2024-04-04T06:17:21.798 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.798 INFO:teuthology.orchestra.run.smithi178.stdout: osd recover clone overlap = true 2024-04-04T06:17:21.798 INFO:teuthology.orchestra.run.smithi178.stdout: osd recovery max chunk = 1048576 2024-04-04T06:17:21.798 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.798 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug shutdown = true 2024-04-04T06:17:21.798 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug op order = true 2024-04-04T06:17:21.798 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify stray on activate = true 2024-04-04T06:17:21.799 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug trim objects = true 2024-04-04T06:17:21.799 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.799 INFO:teuthology.orchestra.run.smithi178.stdout: osd open classes on start = true 2024-04-04T06:17:21.799 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug pg log writeout = true 2024-04-04T06:17:21.799 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.799 INFO:teuthology.orchestra.run.smithi178.stdout: osd deep scrub update digest min age = 30 2024-04-04T06:17:21.799 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.799 INFO:teuthology.orchestra.run.smithi178.stdout: osd map max advance = 10 2024-04-04T06:17:21.799 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.799 INFO:teuthology.orchestra.run.smithi178.stdout: journal zero on create = true 2024-04-04T06:17:21.799 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.799 INFO:teuthology.orchestra.run.smithi178.stdout: filestore ondisk finisher threads = 3 2024-04-04T06:17:21.799 INFO:teuthology.orchestra.run.smithi178.stdout: filestore apply finisher threads = 3 2024-04-04T06:17:21.799 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.800 INFO:teuthology.orchestra.run.smithi178.stdout: bdev debug aio = true 2024-04-04T06:17:21.800 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug misdirected ops = true 2024-04-04T06:17:21.800 INFO:teuthology.orchestra.run.smithi178.stdout: bdev async discard = True 2024-04-04T06:17:21.800 INFO:teuthology.orchestra.run.smithi178.stdout: bdev enable discard = True 2024-04-04T06:17:21.800 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore allocator = bitmap 2024-04-04T06:17:21.800 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore block size = 96636764160 2024-04-04T06:17:21.800 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore fsck on mount = True 2024-04-04T06:17:21.800 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluefs = 1/20 2024-04-04T06:17:21.800 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluestore = 1/20 2024-04-04T06:17:21.800 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:17:21.800 INFO:teuthology.orchestra.run.smithi178.stdout: debug osd = 20 2024-04-04T06:17:21.800 INFO:teuthology.orchestra.run.smithi178.stdout: debug rocksdb = 4/10 2024-04-04T06:17:21.800 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd backfillfull_ratio = 0.85 2024-04-04T06:17:21.800 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd full ratio = 0.9 2024-04-04T06:17:21.800 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd nearfull ratio = 0.8 2024-04-04T06:17:21.801 INFO:teuthology.orchestra.run.smithi178.stdout: osd failsafe full ratio = 0.95 2024-04-04T06:17:21.801 INFO:teuthology.orchestra.run.smithi178.stdout: osd objectstore = bluestore 2024-04-04T06:17:21.801 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-04-04T06:17:21.801 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.801 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr] 2024-04-04T06:17:21.801 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:17:21.801 INFO:teuthology.orchestra.run.smithi178.stdout: debug mgr = 20 2024-04-04T06:17:21.801 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-04T06:17:21.801 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-04T06:17:21.801 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-04T06:17:21.801 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-04T06:17:21.801 INFO:teuthology.orchestra.run.smithi178.stdout: mgr/telemetry/nag = false 2024-04-04T06:17:21.801 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-04-04T06:17:21.801 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.801 INFO:teuthology.orchestra.run.smithi178.stdout:[mon] 2024-04-04T06:17:21.802 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:17:21.802 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-04T06:17:21.802 INFO:teuthology.orchestra.run.smithi178.stdout: debug paxos = 20 2024-04-04T06:17:21.802 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-04T06:17:21.802 INFO:teuthology.orchestra.run.smithi178.stdout: mon data avail warn = 5 2024-04-04T06:17:21.802 INFO:teuthology.orchestra.run.smithi178.stdout: mon mgr mkfs grace = 240 2024-04-04T06:17:21.802 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-04T06:17:21.802 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd reporter subtree level = osd 2024-04-04T06:17:21.802 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd prime pg temp = true 2024-04-04T06:17:21.802 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-04T06:17:21.802 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.802 INFO:teuthology.orchestra.run.smithi178.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-04T06:17:21.802 INFO:teuthology.orchestra.run.smithi178.stdout: auth mon ticket ttl = 660 # 11m 2024-04-04T06:17:21.802 INFO:teuthology.orchestra.run.smithi178.stdout: auth service ticket ttl = 240 # 4m 2024-04-04T06:17:21.803 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.803 INFO:teuthology.orchestra.run.smithi178.stdout: # don't complain about insecure global_id in the test suite 2024-04-04T06:17:21.803 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-04T06:17:21.803 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-04T06:17:21.803 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.803 INFO:teuthology.orchestra.run.smithi178.stdout: # 1m isn't quite enough 2024-04-04T06:17:21.803 INFO:teuthology.orchestra.run.smithi178.stdout: mon_down_mkfs_grace = 2m 2024-04-04T06:17:21.803 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.803 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_filestore_osds = false 2024-04-04T06:17:21.803 INFO:teuthology.orchestra.run.smithi178.stdout: mon down mkfs grace = 300 2024-04-04T06:17:21.803 INFO:teuthology.orchestra.run.smithi178.stdout: mon op complaint time = 120 2024-04-04T06:17:21.803 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:21.803 INFO:teuthology.orchestra.run.smithi178.stdout:[client] 2024-04-04T06:17:21.803 INFO:teuthology.orchestra.run.smithi178.stdout: rgw cache enabled = true 2024-04-04T06:17:21.804 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable ops log = true 2024-04-04T06:17:21.804 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable usage log = true 2024-04-04T06:17:21.804 INFO:teuthology.orchestra.run.smithi178.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-04T06:17:21.804 INFO:teuthology.orchestra.run.smithi178.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-04T06:17:21.804 INFO:teuthology.orchestra.run.smithi178.stdout: client mount timeout = 600 2024-04-04T06:17:21.804 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-04-04T06:17:21.804 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:17:21.804 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = "" 2024-04-04T06:17:21.804 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-04-04T06:17:21.804 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-04-04T06:17:21.804 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.a] 2024-04-04T06:17:21.804 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.b] 2024-04-04T06:17:21.804 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.c] 2024-04-04T06:17:21.804 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.a] 2024-04-04T06:17:21.805 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.c] 2024-04-04T06:17:21.805 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.b] 2024-04-04T06:17:21.805 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.d] 2024-04-04T06:17:21.805 INFO:teuthology.orchestra.run.smithi178.stdout:[mds] 2024-04-04T06:17:21.805 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds = 20 2024-04-04T06:17:21.805 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds balancer = 20 2024-04-04T06:17:21.805 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:17:21.805 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug frag = True 2024-04-04T06:17:21.805 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug scatterstat = True 2024-04-04T06:17:21.805 INFO:teuthology.orchestra.run.smithi178.stdout: mds op complaint time = 180 2024-04-04T06:17:21.805 INFO:teuthology.orchestra.run.smithi178.stdout: mds verify scatter = False 2024-04-04T06:17:21.805 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-04-04T06:17:21.805 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-04-04T06:17:21.805 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-04-04T06:17:21.840 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-04T06:17:21.848 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-04T06:17:22.006 INFO:teuthology.orchestra.run.smithi159.stdout:[global] 2024-04-04T06:17:22.007 INFO:teuthology.orchestra.run.smithi159.stdout: chdir = "" 2024-04-04T06:17:22.007 INFO:teuthology.orchestra.run.smithi159.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-04T06:17:22.007 INFO:teuthology.orchestra.run.smithi159.stdout: auth supported = cephx 2024-04-04T06:17:22.007 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.007 INFO:teuthology.orchestra.run.smithi159.stdout: filestore xattr use omap = true 2024-04-04T06:17:22.007 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.007 INFO:teuthology.orchestra.run.smithi159.stdout: mon clock drift allowed = 1.000 2024-04-04T06:17:22.007 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.007 INFO:teuthology.orchestra.run.smithi159.stdout: osd crush chooseleaf type = 0 2024-04-04T06:17:22.007 INFO:teuthology.orchestra.run.smithi159.stdout: auth debug = true 2024-04-04T06:17:22.007 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.007 INFO:teuthology.orchestra.run.smithi159.stdout: ms die on old message = true 2024-04-04T06:17:22.007 INFO:teuthology.orchestra.run.smithi159.stdout: ms die on bug = true 2024-04-04T06:17:22.007 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.008 INFO:teuthology.orchestra.run.smithi159.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-04T06:17:22.008 INFO:teuthology.orchestra.run.smithi159.stdout: mon pg warn max object skew = 0 2024-04-04T06:17:22.008 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.008 INFO:teuthology.orchestra.run.smithi159.stdout: # disable pg_autoscaler by default for new pools 2024-04-04T06:17:22.008 INFO:teuthology.orchestra.run.smithi159.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-04T06:17:22.008 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.008 INFO:teuthology.orchestra.run.smithi159.stdout: osd pool default size = 2 2024-04-04T06:17:22.008 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.008 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd allow primary affinity = true 2024-04-04T06:17:22.008 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd allow pg remap = true 2024-04-04T06:17:22.008 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on legacy crush tunables = false 2024-04-04T06:17:22.008 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on crush straw calc version zero = false 2024-04-04T06:17:22.008 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on no sortbitwise = false 2024-04-04T06:17:22.008 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on osd down out interval zero = false 2024-04-04T06:17:22.008 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on too few osds = false 2024-04-04T06:17:22.009 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-04T06:17:22.009 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-04T06:17:22.009 INFO:teuthology.orchestra.run.smithi159.stdout: mon_allow_pool_size_one = true 2024-04-04T06:17:22.009 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.009 INFO:teuthology.orchestra.run.smithi159.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-04T06:17:22.009 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.009 INFO:teuthology.orchestra.run.smithi159.stdout: osd default data pool replay window = 5 2024-04-04T06:17:22.009 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.009 INFO:teuthology.orchestra.run.smithi159.stdout: mon allow pool delete = true 2024-04-04T06:17:22.009 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.009 INFO:teuthology.orchestra.run.smithi159.stdout: mon cluster log file level = debug 2024-04-04T06:17:22.009 INFO:teuthology.orchestra.run.smithi159.stdout: debug asserts on shutdown = true 2024-04-04T06:17:22.009 INFO:teuthology.orchestra.run.smithi159.stdout: mon health detail to clog = false 2024-04-04T06:17:22.009 INFO:teuthology.orchestra.run.smithi159.stdout: mon host = "172.21.15.159,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-04-04T06:17:22.010 INFO:teuthology.orchestra.run.smithi159.stdout: ms die on skipped message = False 2024-04-04T06:17:22.010 INFO:teuthology.orchestra.run.smithi159.stdout: fsid = 30311045-f714-48fa-832b-4cb2ba8bf0bf 2024-04-04T06:17:22.010 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.010 INFO:teuthology.orchestra.run.smithi159.stdout:[osd] 2024-04-04T06:17:22.010 INFO:teuthology.orchestra.run.smithi159.stdout: osd journal size = 100 2024-04-04T06:17:22.010 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.010 INFO:teuthology.orchestra.run.smithi159.stdout: osd scrub load threshold = 5.0 2024-04-04T06:17:22.010 INFO:teuthology.orchestra.run.smithi159.stdout: osd scrub max interval = 600 2024-04-04T06:17:22.010 INFO:teuthology.orchestra.run.smithi159.stdout: osd mclock profile = high_recovery_ops 2024-04-04T06:17:22.010 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.010 INFO:teuthology.orchestra.run.smithi159.stdout: osd recover clone overlap = true 2024-04-04T06:17:22.010 INFO:teuthology.orchestra.run.smithi159.stdout: osd recovery max chunk = 1048576 2024-04-04T06:17:22.010 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.010 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug shutdown = true 2024-04-04T06:17:22.010 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug op order = true 2024-04-04T06:17:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug verify stray on activate = true 2024-04-04T06:17:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug trim objects = true 2024-04-04T06:17:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: osd open classes on start = true 2024-04-04T06:17:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug pg log writeout = true 2024-04-04T06:17:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: osd deep scrub update digest min age = 30 2024-04-04T06:17:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: osd map max advance = 10 2024-04-04T06:17:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: journal zero on create = true 2024-04-04T06:17:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: filestore ondisk finisher threads = 3 2024-04-04T06:17:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: filestore apply finisher threads = 3 2024-04-04T06:17:22.011 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: bdev debug aio = true 2024-04-04T06:17:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug misdirected ops = true 2024-04-04T06:17:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: bdev async discard = True 2024-04-04T06:17:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: bdev enable discard = True 2024-04-04T06:17:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore allocator = bitmap 2024-04-04T06:17:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore block size = 96636764160 2024-04-04T06:17:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore fsck on mount = True 2024-04-04T06:17:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: debug bluefs = 1/20 2024-04-04T06:17:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: debug bluestore = 1/20 2024-04-04T06:17:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:17:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: debug osd = 20 2024-04-04T06:17:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: debug rocksdb = 4/10 2024-04-04T06:17:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd backfillfull_ratio = 0.85 2024-04-04T06:17:22.012 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd full ratio = 0.9 2024-04-04T06:17:22.013 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd nearfull ratio = 0.8 2024-04-04T06:17:22.013 INFO:teuthology.orchestra.run.smithi159.stdout: osd failsafe full ratio = 0.95 2024-04-04T06:17:22.013 INFO:teuthology.orchestra.run.smithi159.stdout: osd objectstore = bluestore 2024-04-04T06:17:22.013 INFO:teuthology.orchestra.run.smithi159.stdout: osd op complaint time = 180 2024-04-04T06:17:22.013 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.013 INFO:teuthology.orchestra.run.smithi159.stdout:[mgr] 2024-04-04T06:17:22.013 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:17:22.013 INFO:teuthology.orchestra.run.smithi159.stdout: debug mgr = 20 2024-04-04T06:17:22.013 INFO:teuthology.orchestra.run.smithi159.stdout: debug mon = 20 2024-04-04T06:17:22.013 INFO:teuthology.orchestra.run.smithi159.stdout: debug auth = 20 2024-04-04T06:17:22.013 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min pgs per osd = 4 2024-04-04T06:17:22.013 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min bytes per osd = 10 2024-04-04T06:17:22.013 INFO:teuthology.orchestra.run.smithi159.stdout: mgr/telemetry/nag = false 2024-04-04T06:17:22.013 INFO:teuthology.orchestra.run.smithi159.stdout: debug client = 20 2024-04-04T06:17:22.013 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.014 INFO:teuthology.orchestra.run.smithi159.stdout:[mon] 2024-04-04T06:17:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:17:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: debug mon = 20 2024-04-04T06:17:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: debug paxos = 20 2024-04-04T06:17:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: debug auth = 20 2024-04-04T06:17:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: mon data avail warn = 5 2024-04-04T06:17:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: mon mgr mkfs grace = 240 2024-04-04T06:17:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min pgs per osd = 4 2024-04-04T06:17:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd reporter subtree level = osd 2024-04-04T06:17:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd prime pg temp = true 2024-04-04T06:17:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min bytes per osd = 10 2024-04-04T06:17:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-04T06:17:22.014 INFO:teuthology.orchestra.run.smithi159.stdout: auth mon ticket ttl = 660 # 11m 2024-04-04T06:17:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: auth service ticket ttl = 240 # 4m 2024-04-04T06:17:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: # don't complain about insecure global_id in the test suite 2024-04-04T06:17:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-04T06:17:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-04T06:17:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: # 1m isn't quite enough 2024-04-04T06:17:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: mon_down_mkfs_grace = 2m 2024-04-04T06:17:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_filestore_osds = false 2024-04-04T06:17:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: mon down mkfs grace = 300 2024-04-04T06:17:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: mon op complaint time = 120 2024-04-04T06:17:22.015 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:22.015 INFO:teuthology.orchestra.run.smithi159.stdout:[client] 2024-04-04T06:17:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: rgw cache enabled = true 2024-04-04T06:17:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: rgw enable ops log = true 2024-04-04T06:17:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: rgw enable usage log = true 2024-04-04T06:17:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-04T06:17:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-04T06:17:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: client mount timeout = 600 2024-04-04T06:17:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: debug client = 20 2024-04-04T06:17:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:17:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: pid file = "" 2024-04-04T06:17:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: rados mon op timeout = 900 2024-04-04T06:17:22.016 INFO:teuthology.orchestra.run.smithi159.stdout: rados osd op timeout = 900 2024-04-04T06:17:22.016 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.a] 2024-04-04T06:17:22.016 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.b] 2024-04-04T06:17:22.016 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.c] 2024-04-04T06:17:22.016 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.a] 2024-04-04T06:17:22.017 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.c] 2024-04-04T06:17:22.017 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.b] 2024-04-04T06:17:22.017 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.d] 2024-04-04T06:17:22.017 INFO:teuthology.orchestra.run.smithi159.stdout:[mds] 2024-04-04T06:17:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: debug mds = 20 2024-04-04T06:17:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: debug mds balancer = 20 2024-04-04T06:17:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:17:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: mds debug frag = True 2024-04-04T06:17:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: mds debug scatterstat = False 2024-04-04T06:17:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: mds op complaint time = 180 2024-04-04T06:17:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: mds verify scatter = False 2024-04-04T06:17:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: osd op complaint time = 180 2024-04-04T06:17:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: rados mon op timeout = 900 2024-04-04T06:17:22.017 INFO:teuthology.orchestra.run.smithi159.stdout: rados osd op timeout = 900 2024-04-04T06:17:22.018 INFO:teuthology.orchestra.run.smithi178.stdout:[global] 2024-04-04T06:17:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: chdir = "" 2024-04-04T06:17:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-04T06:17:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: auth supported = cephx 2024-04-04T06:17:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: filestore xattr use omap = true 2024-04-04T06:17:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: mon clock drift allowed = 1.000 2024-04-04T06:17:22.018 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.019 INFO:teuthology.orchestra.run.smithi178.stdout: osd crush chooseleaf type = 0 2024-04-04T06:17:22.019 INFO:teuthology.orchestra.run.smithi178.stdout: auth debug = true 2024-04-04T06:17:22.019 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.019 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on old message = true 2024-04-04T06:17:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on bug = true 2024-04-04T06:17:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-04T06:17:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: mon pg warn max object skew = 0 2024-04-04T06:17:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: # disable pg_autoscaler by default for new pools 2024-04-04T06:17:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-04T06:17:22.022 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default size = 2 2024-04-04T06:17:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow primary affinity = true 2024-04-04T06:17:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow pg remap = true 2024-04-04T06:17:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on legacy crush tunables = false 2024-04-04T06:17:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on crush straw calc version zero = false 2024-04-04T06:17:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on no sortbitwise = false 2024-04-04T06:17:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on osd down out interval zero = false 2024-04-04T06:17:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on too few osds = false 2024-04-04T06:17:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-04T06:17:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-04T06:17:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: mon_allow_pool_size_one = true 2024-04-04T06:17:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-04T06:17:22.023 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: osd default data pool replay window = 5 2024-04-04T06:17:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: mon allow pool delete = true 2024-04-04T06:17:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: mon cluster log file level = debug 2024-04-04T06:17:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: debug asserts on shutdown = true 2024-04-04T06:17:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: mon health detail to clog = false 2024-04-04T06:17:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: mon host = "172.21.15.159,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-04-04T06:17:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on skipped message = False 2024-04-04T06:17:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: fsid = 30311045-f714-48fa-832b-4cb2ba8bf0bf 2024-04-04T06:17:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.024 INFO:teuthology.orchestra.run.smithi178.stdout:[osd] 2024-04-04T06:17:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: osd journal size = 100 2024-04-04T06:17:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.024 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub load threshold = 5.0 2024-04-04T06:17:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub max interval = 600 2024-04-04T06:17:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: osd mclock profile = high_recovery_ops 2024-04-04T06:17:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: osd recover clone overlap = true 2024-04-04T06:17:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: osd recovery max chunk = 1048576 2024-04-04T06:17:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug shutdown = true 2024-04-04T06:17:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug op order = true 2024-04-04T06:17:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify stray on activate = true 2024-04-04T06:17:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug trim objects = true 2024-04-04T06:17:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: osd open classes on start = true 2024-04-04T06:17:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug pg log writeout = true 2024-04-04T06:17:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.025 INFO:teuthology.orchestra.run.smithi178.stdout: osd deep scrub update digest min age = 30 2024-04-04T06:17:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: osd map max advance = 10 2024-04-04T06:17:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: journal zero on create = true 2024-04-04T06:17:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: filestore ondisk finisher threads = 3 2024-04-04T06:17:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: filestore apply finisher threads = 3 2024-04-04T06:17:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: bdev debug aio = true 2024-04-04T06:17:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug misdirected ops = true 2024-04-04T06:17:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: bdev async discard = True 2024-04-04T06:17:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: bdev enable discard = True 2024-04-04T06:17:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore allocator = bitmap 2024-04-04T06:17:22.026 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore block size = 96636764160 2024-04-04T06:17:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore fsck on mount = True 2024-04-04T06:17:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluefs = 1/20 2024-04-04T06:17:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluestore = 1/20 2024-04-04T06:17:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:17:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: debug osd = 20 2024-04-04T06:17:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: debug rocksdb = 4/10 2024-04-04T06:17:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd backfillfull_ratio = 0.85 2024-04-04T06:17:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd full ratio = 0.9 2024-04-04T06:17:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd nearfull ratio = 0.8 2024-04-04T06:17:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: osd failsafe full ratio = 0.95 2024-04-04T06:17:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: osd objectstore = bluestore 2024-04-04T06:17:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-04-04T06:17:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.027 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr] 2024-04-04T06:17:22.027 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:17:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: debug mgr = 20 2024-04-04T06:17:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-04T06:17:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-04T06:17:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-04T06:17:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-04T06:17:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: mgr/telemetry/nag = false 2024-04-04T06:17:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-04-04T06:17:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.028 INFO:teuthology.orchestra.run.smithi178.stdout:[mon] 2024-04-04T06:17:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:17:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-04T06:17:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: debug paxos = 20 2024-04-04T06:17:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-04T06:17:22.028 INFO:teuthology.orchestra.run.smithi178.stdout: mon data avail warn = 5 2024-04-04T06:17:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: mon mgr mkfs grace = 240 2024-04-04T06:17:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-04T06:17:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd reporter subtree level = osd 2024-04-04T06:17:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd prime pg temp = true 2024-04-04T06:17:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-04T06:17:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-04T06:17:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: auth mon ticket ttl = 660 # 11m 2024-04-04T06:17:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: auth service ticket ttl = 240 # 4m 2024-04-04T06:17:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: # don't complain about insecure global_id in the test suite 2024-04-04T06:17:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-04T06:17:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-04T06:17:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: # 1m isn't quite enough 2024-04-04T06:17:22.029 INFO:teuthology.orchestra.run.smithi178.stdout: mon_down_mkfs_grace = 2m 2024-04-04T06:17:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_filestore_osds = false 2024-04-04T06:17:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: mon down mkfs grace = 300 2024-04-04T06:17:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: mon op complaint time = 120 2024-04-04T06:17:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:22.030 INFO:teuthology.orchestra.run.smithi178.stdout:[client] 2024-04-04T06:17:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: rgw cache enabled = true 2024-04-04T06:17:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable ops log = true 2024-04-04T06:17:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable usage log = true 2024-04-04T06:17:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-04T06:17:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-04T06:17:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: client mount timeout = 600 2024-04-04T06:17:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-04-04T06:17:22.030 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:17:22.031 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = "" 2024-04-04T06:17:22.031 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-04-04T06:17:22.031 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-04-04T06:17:22.031 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.a] 2024-04-04T06:17:22.031 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.b] 2024-04-04T06:17:22.031 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.c] 2024-04-04T06:17:22.031 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.a] 2024-04-04T06:17:22.031 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.c] 2024-04-04T06:17:22.031 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.b] 2024-04-04T06:17:22.031 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.d] 2024-04-04T06:17:22.031 INFO:teuthology.orchestra.run.smithi178.stdout:[mds] 2024-04-04T06:17:22.031 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds = 20 2024-04-04T06:17:22.031 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds balancer = 20 2024-04-04T06:17:22.031 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:17:22.031 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug frag = True 2024-04-04T06:17:22.032 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug scatterstat = False 2024-04-04T06:17:22.032 INFO:teuthology.orchestra.run.smithi178.stdout: mds op complaint time = 180 2024-04-04T06:17:22.032 INFO:teuthology.orchestra.run.smithi178.stdout: mds verify scatter = False 2024-04-04T06:17:22.032 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-04-04T06:17:22.032 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-04-04T06:17:22.032 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-04-04T06:17:22.061 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-04-04T06:17:22.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.176+0000 7fdbeea48f40 1 Processor -- start 2024-04-04T06:17:22.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.176+0000 7fdbeea48f40 1 -- start start 2024-04-04T06:17:22.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.177+0000 7fdbeea48f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558968d87510 0x558968d89ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:22.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.177+0000 7fdbeea48f40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558968e17590 0x558968e18160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:22.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.177+0000 7fdbeea48f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558968e14f70 0x558968e12080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:22.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.177+0000 7fdbeea48f40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558968d400e0 con 0x558968e17590 2024-04-04T06:17:22.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.177+0000 7fdbeea48f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558968e1b560 con 0x558968e14f70 2024-04-04T06:17:22.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.177+0000 7fdbeea48f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x558968e1a820 con 0x558968d87510 2024-04-04T06:17:22.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.177+0000 7fdbe2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558968e17590 0x558968e18160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:22.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.177+0000 7fdbe2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558968e17590 0x558968e18160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37516/0 (socket says 172.21.15.159:37516) 2024-04-04T06:17:22.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.177+0000 7fdbe2a00640 1 -- 172.21.15.159:0/3518400256 learned_addr learned my addr 172.21.15.159:0/3518400256 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:22.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.178+0000 7fdbe2a00640 1 -- 172.21.15.159:0/3518400256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558968d87510 msgr2=0x558968d89ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:22.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.178+0000 7fdbe2a00640 1 --2- 172.21.15.159:0/3518400256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558968d87510 0x558968d89ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:22.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.178+0000 7fdbe2a00640 1 -- 172.21.15.159:0/3518400256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558968e14f70 msgr2=0x558968e12080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:22.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.178+0000 7fdbe2a00640 1 --2- 172.21.15.159:0/3518400256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558968e14f70 0x558968e12080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:22.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.178+0000 7fdbe2a00640 1 -- 172.21.15.159:0/3518400256 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558968c844b0 con 0x558968e17590 2024-04-04T06:17:22.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.178+0000 7fdbe2a00640 1 --2- 172.21.15.159:0/3518400256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558968e17590 0x558968e18160 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400a770 tx=0x7fdbd400a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0f126f92cb6548 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:22.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.178+0000 7fdbea000640 1 -- 172.21.15.159:0/3518400256 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4019070 con 0x558968e17590 2024-04-04T06:17:22.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.178+0000 7fdbea000640 1 -- 172.21.15.159:0/3518400256 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbd4007600 con 0x558968e17590 2024-04-04T06:17:22.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.178+0000 7fdbea000640 1 -- 172.21.15.159:0/3518400256 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd40079d0 con 0x558968e17590 2024-04-04T06:17:22.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.179+0000 7fdbeea48f40 1 -- 172.21.15.159:0/3518400256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558968e17590 msgr2=0x558968e18160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:22.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.179+0000 7fdbeea48f40 1 --2- 172.21.15.159:0/3518400256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558968e17590 0x558968e18160 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400a770 tx=0x7fdbd400a860 comp rx=0 tx=0).stop 2024-04-04T06:17:22.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.179+0000 7fdbeea48f40 1 -- 172.21.15.159:0/3518400256 shutdown_connections 2024-04-04T06:17:22.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.179+0000 7fdbeea48f40 1 --2- 172.21.15.159:0/3518400256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558968e14f70 0x558968e12080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:22.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.179+0000 7fdbeea48f40 1 --2- 172.21.15.159:0/3518400256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558968e17590 0x558968e18160 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:22.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.179+0000 7fdbeea48f40 1 --2- 172.21.15.159:0/3518400256 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558968d87510 0x558968d89ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:22.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.179+0000 7fdbeea48f40 1 -- 172.21.15.159:0/3518400256 >> 172.21.15.159:0/3518400256 conn(0x558968d408a0 msgr2=0x558968d41510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:22.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.179+0000 7fdbeea48f40 1 -- 172.21.15.159:0/3518400256 shutdown_connections 2024-04-04T06:17:22.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.179+0000 7fdbeea48f40 1 -- 172.21.15.159:0/3518400256 wait complete. 2024-04-04T06:17:22.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.180+0000 7fdbeea48f40 1 Processor -- start 2024-04-04T06:17:22.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.180+0000 7fdbeea48f40 1 -- start start 2024-04-04T06:17:22.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.181+0000 7fdbeea48f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558968d87510 0x558968d820e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:22.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.181+0000 7fdbeea48f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558968e17590 0x558968e15b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:22.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.181+0000 7fdbeea48f40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558968e16050 0x558968d6bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:22.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.181+0000 7fdbeea48f40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558968e19c10 con 0x558968e16050 2024-04-04T06:17:22.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.181+0000 7fdbeea48f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558968e220c0 con 0x558968e17590 2024-04-04T06:17:22.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.181+0000 7fdbeea48f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x558968d88240 con 0x558968d87510 2024-04-04T06:17:22.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.181+0000 7fdbebe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558968e16050 0x558968d6bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:22.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.181+0000 7fdbebe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558968e16050 0x558968d6bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37530/0 (socket says 172.21.15.159:37530) 2024-04-04T06:17:22.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.181+0000 7fdbebe00640 1 -- 172.21.15.159:0/4082553798 learned_addr learned my addr 172.21.15.159:0/4082553798 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:22.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.182+0000 7fdbe2a00640 1 --2- 172.21.15.159:0/4082553798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558968e17590 0x558968e15b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:22.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.182+0000 7fdbebe00640 1 -- 172.21.15.159:0/4082553798 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558968d87510 msgr2=0x558968d820e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:22.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.182+0000 7fdbeb400640 1 --2- 172.21.15.159:0/4082553798 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558968d87510 0x558968d820e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:22.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.182+0000 7fdbebe00640 1 --2- 172.21.15.159:0/4082553798 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x558968d87510 0x558968d820e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:22.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.182+0000 7fdbebe00640 1 -- 172.21.15.159:0/4082553798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558968e17590 msgr2=0x558968e15b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:22.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.182+0000 7fdbebe00640 1 --2- 172.21.15.159:0/4082553798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558968e17590 0x558968e15b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:22.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.182+0000 7fdbebe00640 1 -- 172.21.15.159:0/4082553798 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558968d6ba60 con 0x558968e16050 2024-04-04T06:17:22.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.182+0000 7fdbebe00640 1 --2- 172.21.15.159:0/4082553798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558968e16050 0x558968d6bf40 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4055040 tx=0x7fdbe40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:22.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.182+0000 7fdbe8c00640 1 -- 172.21.15.159:0/4082553798 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4061070 con 0x558968e16050 2024-04-04T06:17:22.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.182+0000 7fdbe8c00640 1 -- 172.21.15.159:0/4082553798 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbe405c870 con 0x558968e16050 2024-04-04T06:17:22.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.182+0000 7fdbe8c00640 1 -- 172.21.15.159:0/4082553798 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4057040 con 0x558968e16050 2024-04-04T06:17:22.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.182+0000 7fdbeea48f40 1 -- 172.21.15.159:0/4082553798 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558968e141d0 con 0x558968e16050 2024-04-04T06:17:22.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.183+0000 7fdbeea48f40 1 -- 172.21.15.159:0/4082553798 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558968e18710 con 0x558968e16050 2024-04-04T06:17:22.181 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.184+0000 7fdbe8c00640 1 -- 172.21.15.159:0/4082553798 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdbe405a070 con 0x558968e16050 2024-04-04T06:17:22.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.185+0000 7fdbe8c00640 1 --2- 172.21.15.159:0/4082553798 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdbc806b2d0 0x7fdbc806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:22.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.185+0000 7fdbe8c00640 1 -- 172.21.15.159:0/4082553798 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5795+0+0 (secure 0 0 0) 0x7fdbe40cbe40 con 0x558968e16050 2024-04-04T06:17:22.182 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.186+0000 7fdbeea48f40 1 --2- 172.21.15.159:0/4082553798 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x558968d6ce40 0x558968e0ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:22.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.186+0000 7fdbeea48f40 1 -- 172.21.15.159:0/4082553798 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x558968e0bfd0 con 0x558968d6ce40 2024-04-04T06:17:22.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.186+0000 7fdbe2a00640 1 --2- 172.21.15.159:0/4082553798 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x558968d6ce40 0x558968e0ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:22.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.186+0000 7fdbeb400640 1 --2- 172.21.15.159:0/4082553798 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdbc806b2d0 0x7fdbc806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:22.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.186+0000 7fdbe2a00640 1 --2- 172.21.15.159:0/4082553798 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x558968d6ce40 0x558968e0ba90 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:22.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.187+0000 7fdbeb400640 1 --2- 172.21.15.159:0/4082553798 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdbc806b2d0 0x7fdbc806d790 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc005700 tx=0x7fdbdc007040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:22.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.190+0000 7fdbe2a00640 1 -- 172.21.15.159:0/4082553798 <== osd.3 v2:172.21.15.159:6820/2070760205 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v114'5 uv5 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdbd401c040 con 0x558968d6ce40 2024-04-04T06:17:22.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.191+0000 7fdbeea48f40 1 -- 172.21.15.159:0/4082553798 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x558968d6ce40 msgr2=0x558968e0ba90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:22.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.191+0000 7fdbeea48f40 1 --2- 172.21.15.159:0/4082553798 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x558968d6ce40 0x558968e0ba90 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:22.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.191+0000 7fdbeea48f40 1 -- 172.21.15.159:0/4082553798 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdbc806b2d0 msgr2=0x7fdbc806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:22.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.191+0000 7fdbeea48f40 1 --2- 172.21.15.159:0/4082553798 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdbc806b2d0 0x7fdbc806d790 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc005700 tx=0x7fdbdc007040 comp rx=0 tx=0).stop 2024-04-04T06:17:22.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.191+0000 7fdbeea48f40 1 -- 172.21.15.159:0/4082553798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558968e16050 msgr2=0x558968d6bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:22.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.191+0000 7fdbeea48f40 1 --2- 172.21.15.159:0/4082553798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558968e16050 0x558968d6bf40 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4055040 tx=0x7fdbe40057c0 comp rx=0 tx=0).stop 2024-04-04T06:17:22.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.191+0000 7fdbebe00640 1 -- 172.21.15.159:0/4082553798 reap_dead start 2024-04-04T06:17:22.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.191+0000 7fdbeea48f40 1 -- 172.21.15.159:0/4082553798 shutdown_connections 2024-04-04T06:17:22.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.191+0000 7fdbeea48f40 1 -- 172.21.15.159:0/4082553798 >> 172.21.15.159:0/4082553798 conn(0x558968d408a0 msgr2=0x558968d89ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:22.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.191+0000 7fdbeea48f40 1 -- 172.21.15.159:0/4082553798 shutdown_connections 2024-04-04T06:17:22.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.191+0000 7fdbeea48f40 1 -- 172.21.15.159:0/4082553798 wait complete. 2024-04-04T06:17:22.190 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-04T06:17:22.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.391+0000 7f7dcf200640 1 Processor -- start 2024-04-04T06:17:22.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.392+0000 7f7dcf200640 1 -- start start 2024-04-04T06:17:22.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.393+0000 7f7dcf200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7dd00b69c0 0x7f7dd0168f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:22.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.393+0000 7f7dcf200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dd00d2710 con 0x7f7dd015e4c0 2024-04-04T06:17:22.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.393+0000 7f7dcf200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dd00d29f0 con 0x7f7dd00d37b0 2024-04-04T06:17:22.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.393+0000 7f7dcf200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd015e8a0 con 0x7f7dd00b69c0 2024-04-04T06:17:22.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.393+0000 7f7dcde00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f7dd015e4c0 0x7f7dd00b9bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:41070/0 (socket says 172.21.15.159:41070) 2024-04-04T06:17:22.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.393+0000 7f7dcde00640 1 -- 172.21.15.159:0/2652557499 learned_addr learned my addr 172.21.15.159:0/2652557499 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:22.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.394+0000 7f7dce800640 1 --2- 172.21.15.159:0/2652557499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7dd00b69c0 0x7f7dd0168f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:22.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.394+0000 7f7dcd400640 1 -- 172.21.15.159:0/2652557499 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 663261911 0 0) 0x7f7dd00d2710 con 0x7f7dd015e4c0 2024-04-04T06:17:22.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.394+0000 7f7dcd400640 1 -- 172.21.15.159:0/2652557499 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7db4003520 con 0x7f7dd015e4c0 2024-04-04T06:17:22.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.394+0000 7f7dcd400640 1 -- 172.21.15.159:0/2652557499 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1031391663 0 0) 0x7f7db4003520 con 0x7f7dd015e4c0 2024-04-04T06:17:22.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.394+0000 7f7dcd400640 1 -- 172.21.15.159:0/2652557499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7dd00b69c0 msgr2=0x7f7dd0168f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:22.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.394+0000 7f7dcd400640 1 --2- 172.21.15.159:0/2652557499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7dd00b69c0 0x7f7dd0168f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:22.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.394+0000 7f7dcd400640 1 -- 172.21.15.159:0/2652557499 >> v1:172.21.15.178:6789/0 conn(0x7f7dd00d37b0 legacy=0x7f7dd00b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:22.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.394+0000 7f7dcd400640 1 -- 172.21.15.159:0/2652557499 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd00d2cd0 con 0x7f7dd015e4c0 2024-04-04T06:17:22.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.394+0000 7f7dcd400640 1 -- 172.21.15.159:0/2652557499 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f7dc0001670 con 0x7f7dd015e4c0 2024-04-04T06:17:22.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.394+0000 7f7dcd400640 1 -- 172.21.15.159:0/2652557499 >> v1:172.21.15.159:6789/0 conn(0x7f7dd015e4c0 legacy=0x7f7dd00b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:22.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.394+0000 7f7dcd400640 1 --2- 172.21.15.159:0/2652557499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7db40039e0 0x7f7db4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:22.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.394+0000 7f7dcd400640 1 --2- 172.21.15.159:0/2652557499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7dd00b69c0 0x7f7db4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:22.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.394+0000 7f7dcd400640 1 --2- 172.21.15.159:0/2652557499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7db4004d40 0x7f7db40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:22.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.394+0000 7f7dcd400640 1 -- 172.21.15.159:0/2652557499 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7db400ba70 con 0x7f7db4004d40 2024-04-04T06:17:22.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.394+0000 7f7dcd400640 1 -- 172.21.15.159:0/2652557499 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7db400bc20 con 0x7f7db40039e0 2024-04-04T06:17:22.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.394+0000 7f7dcd400640 1 -- 172.21.15.159:0/2652557499 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7db400be20 con 0x7f7dd00b69c0 2024-04-04T06:17:22.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.395+0000 7f7dce800640 1 --2- 172.21.15.159:0/2652557499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7db4004d40 0x7f7db40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:22.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.395+0000 7f7dcde00640 1 --2- 172.21.15.159:0/2652557499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7db40039e0 0x7f7db4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:22.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.395+0000 7f7dc5400640 1 --2- 172.21.15.159:0/2652557499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7dd00b69c0 0x7f7db4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:22.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.395+0000 7f7dce800640 1 -- 172.21.15.159:0/2652557499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7dd00b69c0 msgr2=0x7f7db4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:22.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.395+0000 7f7dce800640 1 --2- 172.21.15.159:0/2652557499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7dd00b69c0 0x7f7db4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:22.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.395+0000 7f7dce800640 1 -- 172.21.15.159:0/2652557499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7db40039e0 msgr2=0x7f7db4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:22.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.395+0000 7f7dce800640 1 --2- 172.21.15.159:0/2652557499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7db40039e0 0x7f7db4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:22.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.395+0000 7f7dce800640 1 -- 172.21.15.159:0/2652557499 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db4003520 con 0x7f7db4004d40 2024-04-04T06:17:22.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.396+0000 7f7dce800640 1 --2- 172.21.15.159:0/2652557499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7db4004d40 0x7f7db40051c0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00d1b30 tx=0x7f7db4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cbfba6cec0af7a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:22.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.397+0000 7f7dcd400640 1 -- 172.21.15.159:0/2652557499 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8039070 con 0x7f7db4004d40 2024-04-04T06:17:22.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.397+0000 7f7dcd400640 1 -- 172.21.15.159:0/2652557499 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dc802cb90 con 0x7f7db4004d40 2024-04-04T06:17:22.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.397+0000 7f7dcd400640 1 -- 172.21.15.159:0/2652557499 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8009c10 con 0x7f7db4004d40 2024-04-04T06:17:22.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.397+0000 7f7dcf200640 1 -- 172.21.15.159:0/2652557499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7db4004d40 msgr2=0x7f7db40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:22.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.397+0000 7f7dcf200640 1 --2- 172.21.15.159:0/2652557499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7db4004d40 0x7f7db40051c0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00d1b30 tx=0x7f7db4005830 comp rx=0 tx=0).stop 2024-04-04T06:17:22.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.397+0000 7f7dce800640 1 -- 172.21.15.159:0/2652557499 reap_dead start 2024-04-04T06:17:22.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.397+0000 7f7dcf200640 1 -- 172.21.15.159:0/2652557499 shutdown_connections 2024-04-04T06:17:22.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.397+0000 7f7dcf200640 1 -- 172.21.15.159:0/2652557499 >> 172.21.15.159:0/2652557499 conn(0x7f7dd00676a0 msgr2=0x7f7dd0067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:22.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.397+0000 7f7dcf200640 1 -- 172.21.15.159:0/2652557499 shutdown_connections 2024-04-04T06:17:22.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.397+0000 7f7dcf200640 1 -- 172.21.15.159:0/2652557499 wait complete. 2024-04-04T06:17:22.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.399+0000 7f7dcf200640 1 Processor -- start 2024-04-04T06:17:22.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.399+0000 7f7dcf200640 1 -- start start 2024-04-04T06:17:22.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.400+0000 7f7dcf200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7db4004d40 0x7f7dd01562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:22.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.400+0000 7f7dcf200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dd00cd8d0 0x7f7dd0157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:22.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.400+0000 7f7dcf200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7dd0157540 0x7f7dd00c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:22.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.400+0000 7f7dcf200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd00d3320 con 0x7f7dd00cd8d0 2024-04-04T06:17:22.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.400+0000 7f7dcf200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd00d4040 con 0x7f7dd0157540 2024-04-04T06:17:22.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.400+0000 7f7dcf200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd00d2350 con 0x7f7db4004d40 2024-04-04T06:17:22.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.400+0000 7f7dc5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dd00cd8d0 0x7f7dd0157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:22.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.400+0000 7f7dc5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dd00cd8d0 0x7f7dd0157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37542/0 (socket says 172.21.15.159:37542) 2024-04-04T06:17:22.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.400+0000 7f7dc5400640 1 -- 172.21.15.159:0/2398080627 learned_addr learned my addr 172.21.15.159:0/2398080627 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:22.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.401+0000 7f7dc5400640 1 -- 172.21.15.159:0/2398080627 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7db4004d40 msgr2=0x7f7dd01562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:22.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.401+0000 7f7dc5400640 1 --2- 172.21.15.159:0/2398080627 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7db4004d40 0x7f7dd01562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:22.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.401+0000 7f7dc5400640 1 -- 172.21.15.159:0/2398080627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7dd0157540 msgr2=0x7f7dd00c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:22.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.401+0000 7f7dc5400640 1 --2- 172.21.15.159:0/2398080627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7dd0157540 0x7f7dd00c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:22.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.401+0000 7f7dc5400640 1 -- 172.21.15.159:0/2398080627 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd0155230 con 0x7f7dd00cd8d0 2024-04-04T06:17:22.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.401+0000 7f7dc5400640 1 --2- 172.21.15.159:0/2398080627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dd00cd8d0 0x7f7dd0157000 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f7db800e580 tx=0x7f7db800ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:22.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.401+0000 7f7dc6a00640 1 -- 172.21.15.159:0/2398080627 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db80052e0 con 0x7f7dd00cd8d0 2024-04-04T06:17:22.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.401+0000 7f7dc6a00640 1 -- 172.21.15.159:0/2398080627 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db8018070 con 0x7f7dd00cd8d0 2024-04-04T06:17:22.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.401+0000 7f7dcf200640 1 -- 172.21.15.159:0/2398080627 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd0155510 con 0x7f7dd00cd8d0 2024-04-04T06:17:22.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.401+0000 7f7dc6a00640 1 -- 172.21.15.159:0/2398080627 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8003810 con 0x7f7dd00cd8d0 2024-04-04T06:17:22.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.402+0000 7f7dcf200640 1 -- 172.21.15.159:0/2398080627 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd0155ad0 con 0x7f7dd00cd8d0 2024-04-04T06:17:22.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.403+0000 7f7dcf200640 1 -- 172.21.15.159:0/2398080627 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dd0075940 con 0x7f7dd00cd8d0 2024-04-04T06:17:22.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.403+0000 7f7dc6a00640 1 -- 172.21.15.159:0/2398080627 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7db80039b0 con 0x7f7dd00cd8d0 2024-04-04T06:17:22.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.404+0000 7f7dc6a00640 1 --2- 172.21.15.159:0/2398080627 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7dac06c8f0 0x7f7dac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:22.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.404+0000 7f7dc6a00640 1 -- 172.21.15.159:0/2398080627 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5795+0+0 (secure 0 0 0) 0x7f7db808b8a0 con 0x7f7dd00cd8d0 2024-04-04T06:17:22.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.405+0000 7f7dcde00640 1 --2- 172.21.15.159:0/2398080627 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7dac06c8f0 0x7f7dac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:22.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.406+0000 7f7dcde00640 1 --2- 172.21.15.159:0/2398080627 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7dac06c8f0 0x7f7dac06edb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0001550 tx=0x7f7dc0007c30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:22.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.409+0000 7f7dc6a00640 1 -- 172.21.15.159:0/2398080627 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f7db8059be0 con 0x7f7dd00cd8d0 2024-04-04T06:17:22.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.653+0000 7f7dcf200640 1 -- 172.21.15.159:0/2398080627 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f7dd00b50e0 con 0x7f7dd00cd8d0 2024-04-04T06:17:22.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.806+0000 7f7dc6a00640 1 -- 172.21.15.159:0/2398080627 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v104) ==== 162+0+0 (secure 0 0 0) 0x7f7db80594e0 con 0x7f7dd00cd8d0 2024-04-04T06:17:22.802 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-04T06:17:22.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.811+0000 7f7dcf200640 1 -- 172.21.15.159:0/2398080627 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7dac06c8f0 msgr2=0x7f7dac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:22.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.811+0000 7f7dcf200640 1 --2- 172.21.15.159:0/2398080627 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7dac06c8f0 0x7f7dac06edb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0001550 tx=0x7f7dc0007c30 comp rx=0 tx=0).stop 2024-04-04T06:17:22.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.811+0000 7f7dcf200640 1 -- 172.21.15.159:0/2398080627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dd00cd8d0 msgr2=0x7f7dd0157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:22.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.811+0000 7f7dcf200640 1 --2- 172.21.15.159:0/2398080627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dd00cd8d0 0x7f7dd0157000 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f7db800e580 tx=0x7f7db800ea40 comp rx=0 tx=0).stop 2024-04-04T06:17:22.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.811+0000 7f7dcf200640 1 -- 172.21.15.159:0/2398080627 shutdown_connections 2024-04-04T06:17:22.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.811+0000 7f7dcf200640 1 --2- 172.21.15.159:0/2398080627 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f7dac06c8f0 0x7f7dac06edb0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:22.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.811+0000 7f7dcf200640 1 --2- 172.21.15.159:0/2398080627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7dd0157540 0x7f7dd00c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:22.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.811+0000 7f7dcf200640 1 --2- 172.21.15.159:0/2398080627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dd00cd8d0 0x7f7dd0157000 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:22.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.811+0000 7f7dcf200640 1 --2- 172.21.15.159:0/2398080627 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7db4004d40 0x7f7dd01562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:22.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.811+0000 7f7dcf200640 1 -- 172.21.15.159:0/2398080627 >> 172.21.15.159:0/2398080627 conn(0x7f7dd00676a0 msgr2=0x7f7dd015c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:22.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.811+0000 7f7dcf200640 1 -- 172.21.15.159:0/2398080627 shutdown_connections 2024-04-04T06:17:22.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:22.812+0000 7f7dcf200640 1 -- 172.21.15.159:0/2398080627 wait complete. 2024-04-04T06:17:22.828 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:17:23.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.022+0000 7f9f47400640 1 Processor -- start 2024-04-04T06:17:23.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.022+0000 7f9f47400640 1 -- start start 2024-04-04T06:17:23.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.023+0000 7f9f47400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9f4006cc90 0x7f9f400ae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:23.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.023+0000 7f9f47400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9f400b5990 0x7f9f400ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:23.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.023+0000 7f9f47400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f400b53e0 con 0x7f9f4015eab0 2024-04-04T06:17:23.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.023+0000 7f9f47400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f9f400b56c0 con 0x7f9f400b5990 2024-04-04T06:17:23.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.023+0000 7f9f47400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f9f400adc80 con 0x7f9f4006cc90 2024-04-04T06:17:23.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.024+0000 7f9f45400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9f400b5990 0x7f9f400ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:23.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.024+0000 7f9f45400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9f400b5990 0x7f9f400ae9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37546/0 (socket says 172.21.15.159:37546) 2024-04-04T06:17:23.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.024+0000 7f9f45400640 1 -- 172.21.15.159:0/3566574357 learned_addr learned my addr 172.21.15.159:0/3566574357 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:23.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.024+0000 7f9f37e00640 1 --2- 172.21.15.159:0/3566574357 >> v2:172.21.15.178:3300/0 conn(0x7f9f4006cc90 0x7f9f400ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:23.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.024+0000 7f9f45400640 1 -- 172.21.15.159:0/3566574357 >> v2:172.21.15.178:3300/0 conn(0x7f9f4006cc90 msgr2=0x7f9f400ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:23.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.024+0000 7f9f45400640 1 --2- 172.21.15.159:0/3566574357 >> v2:172.21.15.178:3300/0 conn(0x7f9f4006cc90 0x7f9f400ae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:23.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.024+0000 7f9f45400640 1 -- 172.21.15.159:0/3566574357 >> v1:172.21.15.178:6789/0 conn(0x7f9f4015eab0 legacy=0x7f9f4006c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:23.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.024+0000 7f9f45400640 1 -- 172.21.15.159:0/3566574357 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f400b49a0 con 0x7f9f400b5990 2024-04-04T06:17:23.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.025+0000 7f9f45400640 1 --2- 172.21.15.159:0/3566574357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f400b5990 0x7f9f400ae9c0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f9f40071d80 tx=0x7f9f3802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae1aea2019d760 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:23.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.026+0000 7f9f3fe00640 1 -- 172.21.15.159:0/3566574357 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3802ca80 con 0x7f9f400b5990 2024-04-04T06:17:23.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.026+0000 7f9f3fe00640 1 -- 172.21.15.159:0/3566574357 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f3802cc20 con 0x7f9f400b5990 2024-04-04T06:17:23.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.026+0000 7f9f3fe00640 1 -- 172.21.15.159:0/3566574357 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f380343f0 con 0x7f9f400b5990 2024-04-04T06:17:23.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.026+0000 7f9f47400640 1 -- 172.21.15.159:0/3566574357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f400b5990 msgr2=0x7f9f400ae9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:23.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.026+0000 7f9f47400640 1 --2- 172.21.15.159:0/3566574357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f400b5990 0x7f9f400ae9c0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f9f40071d80 tx=0x7f9f3802bee0 comp rx=0 tx=0).stop 2024-04-04T06:17:23.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.026+0000 7f9f47400640 1 -- 172.21.15.159:0/3566574357 shutdown_connections 2024-04-04T06:17:23.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.026+0000 7f9f47400640 1 --2- 172.21.15.159:0/3566574357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f400b5990 0x7f9f400ae9c0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:23.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.026+0000 7f9f47400640 1 --2- 172.21.15.159:0/3566574357 >> v2:172.21.15.178:3300/0 conn(0x7f9f4006cc90 0x7f9f400ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:23.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.026+0000 7f9f47400640 1 -- 172.21.15.159:0/3566574357 >> 172.21.15.159:0/3566574357 conn(0x7f9f40071df0 msgr2=0x7f9f400721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:23.023 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.026+0000 7f9f47400640 1 -- 172.21.15.159:0/3566574357 shutdown_connections 2024-04-04T06:17:23.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.026+0000 7f9f47400640 1 -- 172.21.15.159:0/3566574357 wait complete. 2024-04-04T06:17:23.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.028+0000 7f9f47400640 1 Processor -- start 2024-04-04T06:17:23.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.028+0000 7f9f47400640 1 -- start start 2024-04-04T06:17:23.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.029+0000 7f9f47400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f4006cc90 0x7f9f4014be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:23.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.029+0000 7f9f47400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f400b5990 0x7f9f4015a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:23.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.029+0000 7f9f47400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f4015eab0 0x7f9f400b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:23.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.029+0000 7f9f47400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9f400aef00 con 0x7f9f4006cc90 2024-04-04T06:17:23.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.029+0000 7f9f47400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9f40072c50 con 0x7f9f400b5990 2024-04-04T06:17:23.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.029+0000 7f9f47400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9f400af080 con 0x7f9f4015eab0 2024-04-04T06:17:23.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.029+0000 7f9f44a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f4006cc90 0x7f9f4014be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:23.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.029+0000 7f9f44a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f4006cc90 0x7f9f4014be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37548/0 (socket says 172.21.15.159:37548) 2024-04-04T06:17:23.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.030+0000 7f9f44a00640 1 -- 172.21.15.159:0/1856051237 learned_addr learned my addr 172.21.15.159:0/1856051237 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:23.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.030+0000 7f9f45400640 1 --2- 172.21.15.159:0/1856051237 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f4015eab0 0x7f9f400b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:23.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.030+0000 7f9f37e00640 1 --2- 172.21.15.159:0/1856051237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f400b5990 0x7f9f4015a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:23.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.030+0000 7f9f44a00640 1 -- 172.21.15.159:0/1856051237 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f4015eab0 msgr2=0x7f9f400b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:23.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.030+0000 7f9f44a00640 1 --2- 172.21.15.159:0/1856051237 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f4015eab0 0x7f9f400b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:23.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.030+0000 7f9f44a00640 1 -- 172.21.15.159:0/1856051237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f400b5990 msgr2=0x7f9f4015a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:23.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.030+0000 7f9f44a00640 1 --2- 172.21.15.159:0/1856051237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f400b5990 0x7f9f4015a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:23.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.030+0000 7f9f44a00640 1 -- 172.21.15.159:0/1856051237 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f40160e40 con 0x7f9f4006cc90 2024-04-04T06:17:23.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.030+0000 7f9f37e00640 1 --2- 172.21.15.159:0/1856051237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f400b5990 0x7f9f4015a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:23.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.030+0000 7f9f44a00640 1 --2- 172.21.15.159:0/1856051237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f4006cc90 0x7f9f4014be80 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000b410 tx=0x7f9f3000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:23.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.031+0000 7f9f3d600640 1 -- 172.21.15.159:0/1856051237 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30015020 con 0x7f9f4006cc90 2024-04-04T06:17:23.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.031+0000 7f9f3d600640 1 -- 172.21.15.159:0/1856051237 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f30011830 con 0x7f9f4006cc90 2024-04-04T06:17:23.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.031+0000 7f9f3d600640 1 -- 172.21.15.159:0/1856051237 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30011c20 con 0x7f9f4006cc90 2024-04-04T06:17:23.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.031+0000 7f9f47400640 1 -- 172.21.15.159:0/1856051237 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f40157ff0 con 0x7f9f4006cc90 2024-04-04T06:17:23.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.031+0000 7f9f47400640 1 -- 172.21.15.159:0/1856051237 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f400b41c0 con 0x7f9f4006cc90 2024-04-04T06:17:23.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.032+0000 7f9f47400640 1 -- 172.21.15.159:0/1856051237 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f4006f950 con 0x7f9f4006cc90 2024-04-04T06:17:23.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.033+0000 7f9f3d600640 1 -- 172.21.15.159:0/1856051237 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f30018020 con 0x7f9f4006cc90 2024-04-04T06:17:23.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.034+0000 7f9f3d600640 1 --2- 172.21.15.159:0/1856051237 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9f1c06c8f0 0x7f9f1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:23.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.034+0000 7f9f3d600640 1 -- 172.21.15.159:0/1856051237 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5795+0+0 (secure 0 0 0) 0x7f9f3008bb90 con 0x7f9f4006cc90 2024-04-04T06:17:23.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.035+0000 7f9f37e00640 1 --2- 172.21.15.159:0/1856051237 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9f1c06c8f0 0x7f9f1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:23.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.036+0000 7f9f37e00640 1 --2- 172.21.15.159:0/1856051237 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9f1c06c8f0 0x7f9f1c06edb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9f40159cc0 tx=0x7f9f28008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:23.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.039+0000 7f9f3d600640 1 -- 172.21.15.159:0/1856051237 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9f4006f950 con 0x7f9f4006cc90 2024-04-04T06:17:23.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.279+0000 7f9f47400640 1 -- 172.21.15.159:0/1856051237 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f400685c0 con 0x7f9f4006cc90 2024-04-04T06:17:23.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.281+0000 7f9f3d600640 1 -- 172.21.15.159:0/1856051237 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5193 (secure 0 0 0) 0x7f9f3001f020 con 0x7f9f4006cc90 2024-04-04T06:17:23.276 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:23.276 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":104,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5732,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1498324773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1498324773},{"type":"v1","addr":"172.21.15.178:6837","nonce":1498324773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2455031662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":2455031662},{"type":"v1","addr":"172.21.15.159:6835","nonce":2455031662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6343,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1823228151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1823228151},{"type":"v1","addr":"172.21.15.178:6836","nonce":1823228151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:16:57.277344+0000","modified":"2024-04-04T06:17:22.803736+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9135":{"gid":9135,"name":"a","rank":0,"incarnation":104,"state":"up:replay","state_seq":1,"addr":"172.21.15.159:6834/2439261212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2439261212},{"type":"v1","addr":"172.21.15.159:6834","nonce":2439261212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-04T06:17:23.277 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 104 2024-04-04T06:17:23.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.283+0000 7f9f47400640 1 -- 172.21.15.159:0/1856051237 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9f1c06c8f0 msgr2=0x7f9f1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:23.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.283+0000 7f9f47400640 1 --2- 172.21.15.159:0/1856051237 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9f1c06c8f0 0x7f9f1c06edb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9f40159cc0 tx=0x7f9f28008040 comp rx=0 tx=0).stop 2024-04-04T06:17:23.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.283+0000 7f9f47400640 1 -- 172.21.15.159:0/1856051237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f4006cc90 msgr2=0x7f9f4014be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:23.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.283+0000 7f9f47400640 1 --2- 172.21.15.159:0/1856051237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f4006cc90 0x7f9f4014be80 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000b410 tx=0x7f9f3000b8d0 comp rx=0 tx=0).stop 2024-04-04T06:17:23.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.283+0000 7f9f47400640 1 -- 172.21.15.159:0/1856051237 shutdown_connections 2024-04-04T06:17:23.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.283+0000 7f9f47400640 1 --2- 172.21.15.159:0/1856051237 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f4015eab0 0x7f9f400b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:23.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.283+0000 7f9f47400640 1 --2- 172.21.15.159:0/1856051237 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9f1c06c8f0 0x7f9f1c06edb0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:23.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.283+0000 7f9f47400640 1 --2- 172.21.15.159:0/1856051237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f400b5990 0x7f9f4015a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:23.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.284+0000 7f9f47400640 1 --2- 172.21.15.159:0/1856051237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f4006cc90 0x7f9f4014be80 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:23.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.284+0000 7f9f47400640 1 -- 172.21.15.159:0/1856051237 >> 172.21.15.159:0/1856051237 conn(0x7f9f40071df0 msgr2=0x7f9f4015dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:23.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.284+0000 7f9f47400640 1 -- 172.21.15.159:0/1856051237 shutdown_connections 2024-04-04T06:17:23.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:23.284+0000 7f9f47400640 1 -- 172.21.15.159:0/1856051237 wait complete. 2024-04-04T06:17:23.699 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:16:57.277344+0000', 'modified': '2024-04-04T06:17:22.803736+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 114, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9135}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9135': {'gid': 9135, 'name': 'a', 'rank': 0, 'incarnation': 104, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.159:6834/2439261212', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 2439261212}, {'type': 'v1', 'addr': '172.21.15.159:6834', 'nonce': 2439261212}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-04T06:17:23.699 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9135:up:replay 2024-04-04T06:17:24.700 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:17:24.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.926+0000 7f27a7e00640 1 Processor -- start 2024-04-04T06:17:24.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.926+0000 7f27a7e00640 1 -- start start 2024-04-04T06:17:24.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.927+0000 7f27a7e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f27a015b310 0x7f27a00727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:24.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.927+0000 7f27a7e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f27a00d0780 0x7f27a00d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:24.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.927+0000 7f27a7e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27a015b6f0 con 0x7f27a00b4f90 2024-04-04T06:17:24.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.927+0000 7f27a7e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f27a0072e80 con 0x7f27a00d0780 2024-04-04T06:17:24.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.927+0000 7f27a7e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f27a0073160 con 0x7f27a015b310 2024-04-04T06:17:24.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.928+0000 7f27a5e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f27a00d0780 0x7f27a00d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:24.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.928+0000 7f27a5e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f27a00d0780 0x7f27a00d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48286/0 (socket says 172.21.15.159:48286) 2024-04-04T06:17:24.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.928+0000 7f27a5e00640 1 -- 172.21.15.159:0/1827666635 learned_addr learned my addr 172.21.15.159:0/1827666635 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:24.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.928+0000 7f27a5400640 1 --2- 172.21.15.159:0/1827666635 >> v2:172.21.15.178:3300/0 conn(0x7f27a015b310 0x7f27a00727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:24.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.928+0000 7f27a5e00640 1 -- 172.21.15.159:0/1827666635 >> v2:172.21.15.178:3300/0 conn(0x7f27a015b310 msgr2=0x7f27a00727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:24.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.928+0000 7f27a5e00640 1 --2- 172.21.15.159:0/1827666635 >> v2:172.21.15.178:3300/0 conn(0x7f27a015b310 0x7f27a00727f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:24.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.928+0000 7f27a5e00640 1 -- 172.21.15.159:0/1827666635 >> v1:172.21.15.178:6789/0 conn(0x7f27a00b4f90 legacy=0x7f27a015f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:24.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.928+0000 7f27a5e00640 1 -- 172.21.15.159:0/1827666635 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a015efe0 con 0x7f27a00d0780 2024-04-04T06:17:24.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.929+0000 7f27a5e00640 1 --2- 172.21.15.159:0/1827666635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27a00d0780 0x7f27a00d0c10 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f27a006fa40 tx=0x7f279802bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad038af4532406d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:24.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.929+0000 7f279fe00640 1 -- 172.21.15.159:0/1827666635 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279802c960 con 0x7f27a00d0780 2024-04-04T06:17:24.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.929+0000 7f279fe00640 1 -- 172.21.15.159:0/1827666635 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f279802cb00 con 0x7f27a00d0780 2024-04-04T06:17:24.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.929+0000 7f279fe00640 1 -- 172.21.15.159:0/1827666635 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27980343f0 con 0x7f27a00d0780 2024-04-04T06:17:24.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.930+0000 7f27a7e00640 1 -- 172.21.15.159:0/1827666635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27a00d0780 msgr2=0x7f27a00d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:24.925 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.930+0000 7f27a7e00640 1 --2- 172.21.15.159:0/1827666635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27a00d0780 0x7f27a00d0c10 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f27a006fa40 tx=0x7f279802bdc0 comp rx=0 tx=0).stop 2024-04-04T06:17:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.930+0000 7f27a7e00640 1 -- 172.21.15.159:0/1827666635 shutdown_connections 2024-04-04T06:17:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.930+0000 7f27a7e00640 1 --2- 172.21.15.159:0/1827666635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27a00d0780 0x7f27a00d0c10 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.930+0000 7f27a7e00640 1 --2- 172.21.15.159:0/1827666635 >> v2:172.21.15.178:3300/0 conn(0x7f27a015b310 0x7f27a00727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.930+0000 7f27a7e00640 1 -- 172.21.15.159:0/1827666635 >> 172.21.15.159:0/1827666635 conn(0x7f27a006b990 msgr2=0x7f27a006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.930+0000 7f27a7e00640 1 -- 172.21.15.159:0/1827666635 shutdown_connections 2024-04-04T06:17:24.926 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.930+0000 7f27a7e00640 1 -- 172.21.15.159:0/1827666635 wait complete. 2024-04-04T06:17:24.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.932+0000 7f27a7e00640 1 Processor -- start 2024-04-04T06:17:24.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.932+0000 7f27a7e00640 1 -- start start 2024-04-04T06:17:24.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.933+0000 7f27a7e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27a00b4f90 0x7f27a0155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:24.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.933+0000 7f27a7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27a00d0780 0x7f27a0156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:24.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.933+0000 7f27a7e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27a015b310 0x7f27a0156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:24.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.933+0000 7f27a7e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f27a0073440 con 0x7f27a00d0780 2024-04-04T06:17:24.929 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.933+0000 7f27a7e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f27a00b53f0 con 0x7f27a015b310 2024-04-04T06:17:24.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.933+0000 7f27a7e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f27a00adc80 con 0x7f27a00b4f90 2024-04-04T06:17:24.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.934+0000 7f27a4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27a00d0780 0x7f27a0156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:24.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.934+0000 7f27a4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27a00d0780 0x7f27a0156050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48288/0 (socket says 172.21.15.159:48288) 2024-04-04T06:17:24.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.934+0000 7f27a4a00640 1 -- 172.21.15.159:0/55432321 learned_addr learned my addr 172.21.15.159:0/55432321 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:24.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.934+0000 7f27a5e00640 1 --2- 172.21.15.159:0/55432321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27a015b310 0x7f27a0156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:24.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.934+0000 7f27a4a00640 1 -- 172.21.15.159:0/55432321 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27a00b4f90 msgr2=0x7f27a0155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:24.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.934+0000 7f27a4a00640 1 --2- 172.21.15.159:0/55432321 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27a00b4f90 0x7f27a0155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:24.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.934+0000 7f27a4a00640 1 -- 172.21.15.159:0/55432321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27a015b310 msgr2=0x7f27a0156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:24.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.934+0000 7f27a4a00640 1 --2- 172.21.15.159:0/55432321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27a015b310 0x7f27a0156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:24.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.934+0000 7f27a4a00640 1 -- 172.21.15.159:0/55432321 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a01f8260 con 0x7f27a00d0780 2024-04-04T06:17:24.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.935+0000 7f27a4a00640 1 --2- 172.21.15.159:0/55432321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27a00d0780 0x7f27a0156050 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f278c00c430 tx=0x7f278c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:24.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.935+0000 7f279d600640 1 -- 172.21.15.159:0/55432321 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c011790 con 0x7f27a00d0780 2024-04-04T06:17:24.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.935+0000 7f279d600640 1 -- 172.21.15.159:0/55432321 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f278c017070 con 0x7f27a00d0780 2024-04-04T06:17:24.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.935+0000 7f279d600640 1 -- 172.21.15.159:0/55432321 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c011bb0 con 0x7f27a00d0780 2024-04-04T06:17:24.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.935+0000 7f27a7e00640 1 -- 172.21.15.159:0/55432321 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a01f8500 con 0x7f27a00d0780 2024-04-04T06:17:24.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.935+0000 7f27a7e00640 1 -- 172.21.15.159:0/55432321 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a01f8ac0 con 0x7f27a00d0780 2024-04-04T06:17:24.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.936+0000 7f27a7e00640 1 -- 172.21.15.159:0/55432321 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27a00685c0 con 0x7f27a00d0780 2024-04-04T06:17:24.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.937+0000 7f279d600640 1 -- 172.21.15.159:0/55432321 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f278c01c460 con 0x7f27a00d0780 2024-04-04T06:17:24.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.938+0000 7f279d600640 1 --2- 172.21.15.159:0/55432321 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f278006c8f0 0x7f278006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:24.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.938+0000 7f279d600640 1 -- 172.21.15.159:0/55432321 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5795+0+0 (secure 0 0 0) 0x7f278c08bae0 con 0x7f27a00d0780 2024-04-04T06:17:24.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.938+0000 7f27a5400640 1 --2- 172.21.15.159:0/55432321 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f278006c8f0 0x7f278006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:24.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.940+0000 7f27a5400640 1 --2- 172.21.15.159:0/55432321 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f278006c8f0 0x7f278006edb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2794006fd0 tx=0x7f2794008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:24.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:24.943+0000 7f279d600640 1 -- 172.21.15.159:0/55432321 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f278c059ea0 con 0x7f27a00d0780 2024-04-04T06:17:25.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:25.183+0000 7f27a7e00640 1 -- 172.21.15.159:0/55432321 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27a015ec20 con 0x7f27a00d0780 2024-04-04T06:17:25.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:25.185+0000 7f279d600640 1 -- 172.21.15.159:0/55432321 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5193 (secure 0 0 0) 0x7f278c0597a0 con 0x7f27a00d0780 2024-04-04T06:17:25.184 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:25.184 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":106,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5732,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1498324773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1498324773},{"type":"v1","addr":"172.21.15.178:6837","nonce":1498324773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2455031662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":2455031662},{"type":"v1","addr":"172.21.15.159:6835","nonce":2455031662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6343,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1823228151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1823228151},{"type":"v1","addr":"172.21.15.178:6836","nonce":1823228151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:16:57.277344+0000","modified":"2024-04-04T06:17:23.820760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9135":{"gid":9135,"name":"a","rank":0,"incarnation":104,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.159:6834/2439261212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2439261212},{"type":"v1","addr":"172.21.15.159:6834","nonce":2439261212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-04T06:17:25.185 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 106 2024-04-04T06:17:25.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:25.187+0000 7f27a7e00640 1 -- 172.21.15.159:0/55432321 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f278006c8f0 msgr2=0x7f278006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:25.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:25.187+0000 7f27a7e00640 1 --2- 172.21.15.159:0/55432321 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f278006c8f0 0x7f278006edb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2794006fd0 tx=0x7f2794008040 comp rx=0 tx=0).stop 2024-04-04T06:17:25.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:25.187+0000 7f27a7e00640 1 -- 172.21.15.159:0/55432321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27a00d0780 msgr2=0x7f27a0156050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:25.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:25.187+0000 7f27a7e00640 1 --2- 172.21.15.159:0/55432321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27a00d0780 0x7f27a0156050 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f278c00c430 tx=0x7f278c00c8f0 comp rx=0 tx=0).stop 2024-04-04T06:17:25.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:25.187+0000 7f27a7e00640 1 -- 172.21.15.159:0/55432321 shutdown_connections 2024-04-04T06:17:25.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:25.187+0000 7f27a7e00640 1 --2- 172.21.15.159:0/55432321 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f278006c8f0 0x7f278006edb0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:25.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:25.187+0000 7f27a7e00640 1 --2- 172.21.15.159:0/55432321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27a015b310 0x7f27a0156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:25.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:25.188+0000 7f27a7e00640 1 --2- 172.21.15.159:0/55432321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27a00d0780 0x7f27a0156050 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:25.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:25.188+0000 7f27a7e00640 1 --2- 172.21.15.159:0/55432321 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f27a00b4f90 0x7f27a0155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:25.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:25.188+0000 7f27a7e00640 1 -- 172.21.15.159:0/55432321 >> 172.21.15.159:0/55432321 conn(0x7f27a006b990 msgr2=0x7f27a00b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:25.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:25.188+0000 7f27a7e00640 1 -- 172.21.15.159:0/55432321 shutdown_connections 2024-04-04T06:17:25.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:25.188+0000 7f27a7e00640 1 -- 172.21.15.159:0/55432321 wait complete. 2024-04-04T06:17:25.609 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 106, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:16:57.277344+0000', 'modified': '2024-04-04T06:17:23.820760+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 114, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9135}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9135': {'gid': 9135, 'name': 'a', 'rank': 0, 'incarnation': 104, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.159:6834/2439261212', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 2439261212}, {'type': 'v1', 'addr': '172.21.15.159:6834', 'nonce': 2439261212}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-04T06:17:25.610 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9135:up:rejoin 2024-04-04T06:17:26.610 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:17:26.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.830+0000 7f929b200640 1 Processor -- start 2024-04-04T06:17:26.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.830+0000 7f929b200640 1 -- start start 2024-04-04T06:17:26.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.833+0000 7f929b200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f929415e7e0 0x7f92940d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:26.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.833+0000 7f929b200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f929415ebc0 con 0x7f92940cfb20 2024-04-04T06:17:26.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.833+0000 7f929b200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f929406d1c0 con 0x7f929406ca00 2024-04-04T06:17:26.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.833+0000 7f929b200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f929406d4a0 con 0x7f929415e7e0 2024-04-04T06:17:26.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.833+0000 7f9299200640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f92940cfb20 0x7f92940d43e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:50966/0 (socket says 172.21.15.159:50966) 2024-04-04T06:17:26.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.833+0000 7f9299200640 1 -- 172.21.15.159:0/1415927767 learned_addr learned my addr 172.21.15.159:0/1415927767 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:26.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.834+0000 7f9293e00640 1 --2- 172.21.15.159:0/1415927767 >> v2:172.21.15.178:3300/0 conn(0x7f929415e7e0 0x7f92940d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:26.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.835+0000 7f9293e00640 1 -- 172.21.15.159:0/1415927767 >> v1:172.21.15.178:6789/0 conn(0x7f929406ca00 legacy=0x7f92940d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:26.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.835+0000 7f9293e00640 1 -- 172.21.15.159:0/1415927767 >> v1:172.21.15.159:6789/0 conn(0x7f92940cfb20 legacy=0x7f92940d43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:26.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.835+0000 7f9293e00640 1 -- 172.21.15.159:0/1415927767 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92940cf320 con 0x7f929415e7e0 2024-04-04T06:17:26.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.836+0000 7f9293e00640 1 --2- 172.21.15.159:0/1415927767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f929415e7e0 0x7f92940d1950 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f92840097a0 tx=0x7f928402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb42f980e70cf628 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:26.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.838+0000 7f9292a00640 1 -- 172.21.15.159:0/1415927767 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9284035020 con 0x7f929415e7e0 2024-04-04T06:17:26.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.838+0000 7f9292a00640 1 -- 172.21.15.159:0/1415927767 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9284031830 con 0x7f929415e7e0 2024-04-04T06:17:26.833 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.838+0000 7f9292a00640 1 -- 172.21.15.159:0/1415927767 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9284031c00 con 0x7f929415e7e0 2024-04-04T06:17:26.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.839+0000 7f929b200640 1 -- 172.21.15.159:0/1415927767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f929415e7e0 msgr2=0x7f92940d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:26.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.839+0000 7f929b200640 1 --2- 172.21.15.159:0/1415927767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f929415e7e0 0x7f92940d1950 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f92840097a0 tx=0x7f928402b8d0 comp rx=0 tx=0).stop 2024-04-04T06:17:26.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.839+0000 7f929b200640 1 -- 172.21.15.159:0/1415927767 shutdown_connections 2024-04-04T06:17:26.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.839+0000 7f929b200640 1 --2- 172.21.15.159:0/1415927767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f929415e7e0 0x7f92940d1950 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:26.834 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.839+0000 7f929b200640 1 -- 172.21.15.159:0/1415927767 >> 172.21.15.159:0/1415927767 conn(0x7f929409d7c0 msgr2=0x7f929406bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:26.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.839+0000 7f929b200640 1 -- 172.21.15.159:0/1415927767 shutdown_connections 2024-04-04T06:17:26.835 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.839+0000 7f929b200640 1 -- 172.21.15.159:0/1415927767 wait complete. 2024-04-04T06:17:26.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.841+0000 7f929b200640 1 Processor -- start 2024-04-04T06:17:26.836 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.841+0000 7f929b200640 1 -- start start 2024-04-04T06:17:26.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.843+0000 7f929b200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f929406ca00 0x7f9294152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:26.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.843+0000 7f929b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92940cfb20 0x7f92940c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:26.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.843+0000 7f929b200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f929415e7e0 0x7f929415ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:26.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.843+0000 7f929b200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f929406ce60 con 0x7f92940cfb20 2024-04-04T06:17:26.840 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.843+0000 7f929b200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f92940cefd0 con 0x7f929415e7e0 2024-04-04T06:17:26.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.843+0000 7f929b200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f92940d2f30 con 0x7f929406ca00 2024-04-04T06:17:26.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.843+0000 7f9293400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92940cfb20 0x7f92940c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:26.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.843+0000 7f9293400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92940cfb20 0x7f92940c9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48298/0 (socket says 172.21.15.159:48298) 2024-04-04T06:17:26.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.843+0000 7f9293400640 1 -- 172.21.15.159:0/547284283 learned_addr learned my addr 172.21.15.159:0/547284283 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:26.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.843+0000 7f9299200640 1 --2- 172.21.15.159:0/547284283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f929415e7e0 0x7f929415ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:26.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.843+0000 7f9293e00640 1 --2- 172.21.15.159:0/547284283 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f929406ca00 0x7f9294152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:26.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.844+0000 7f9293400640 1 -- 172.21.15.159:0/547284283 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f929406ca00 msgr2=0x7f9294152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:26.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.844+0000 7f9293400640 1 --2- 172.21.15.159:0/547284283 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f929406ca00 0x7f9294152460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:26.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.844+0000 7f9293400640 1 -- 172.21.15.159:0/547284283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f929415e7e0 msgr2=0x7f929415ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:26.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.844+0000 7f9293400640 1 --2- 172.21.15.159:0/547284283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f929415e7e0 0x7f929415ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:26.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.844+0000 7f9293400640 1 -- 172.21.15.159:0/547284283 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92940ca340 con 0x7f92940cfb20 2024-04-04T06:17:26.841 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.844+0000 7f9293400640 1 --2- 172.21.15.159:0/547284283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92940cfb20 0x7f92940c9e00 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f928000c400 tx=0x7f928000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:26.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.844+0000 7f928be00640 1 -- 172.21.15.159:0/547284283 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9280017070 con 0x7f92940cfb20 2024-04-04T06:17:26.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.844+0000 7f928be00640 1 -- 172.21.15.159:0/547284283 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92800052d0 con 0x7f92940cfb20 2024-04-04T06:17:26.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.844+0000 7f928be00640 1 -- 172.21.15.159:0/547284283 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9280003830 con 0x7f92940cfb20 2024-04-04T06:17:26.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.844+0000 7f929b200640 1 -- 172.21.15.159:0/547284283 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f929415f010 con 0x7f92940cfb20 2024-04-04T06:17:26.842 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.845+0000 7f929b200640 1 -- 172.21.15.159:0/547284283 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f929415f5d0 con 0x7f92940cfb20 2024-04-04T06:17:26.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.846+0000 7f929b200640 1 -- 172.21.15.159:0/547284283 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92940685c0 con 0x7f92940cfb20 2024-04-04T06:17:26.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.847+0000 7f928be00640 1 -- 172.21.15.159:0/547284283 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f928001a070 con 0x7f92940cfb20 2024-04-04T06:17:26.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.848+0000 7f928be00640 1 --2- 172.21.15.159:0/547284283 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f927406c8f0 0x7f927406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:26.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.848+0000 7f928be00640 1 -- 172.21.15.159:0/547284283 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5795+0+0 (secure 0 0 0) 0x7f928008af40 con 0x7f92940cfb20 2024-04-04T06:17:26.843 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.848+0000 7f9293e00640 1 --2- 172.21.15.159:0/547284283 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f927406c8f0 0x7f927406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:26.844 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.850+0000 7f9293e00640 1 --2- 172.21.15.159:0/547284283 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f927406c8f0 0x7f927406edb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9284036000 tx=0x7f9284008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:26.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:26.858+0000 7f928be00640 1 -- 172.21.15.159:0/547284283 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f92940ca340 con 0x7f92940cfb20 2024-04-04T06:17:27.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.103+0000 7f929b200640 1 -- 172.21.15.159:0/547284283 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92940ca570 con 0x7f92940cfb20 2024-04-04T06:17:27.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.104+0000 7f928be00640 1 -- 172.21.15.159:0/547284283 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+5200 (secure 0 0 0) 0x7f9280059300 con 0x7f92940cfb20 2024-04-04T06:17:27.099 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:27.100 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":107,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5732,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1498324773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1498324773},{"type":"v1","addr":"172.21.15.178:6837","nonce":1498324773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2455031662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":2455031662},{"type":"v1","addr":"172.21.15.159:6835","nonce":2455031662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6343,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1823228151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1823228151},{"type":"v1","addr":"172.21.15.178:6836","nonce":1823228151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:16:57.277344+0000","modified":"2024-04-04T06:17:25.827822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9135":{"gid":9135,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6834/2439261212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2439261212},{"type":"v1","addr":"172.21.15.159:6834","nonce":2439261212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9135,"qdb_cluster":[9135]},"id":8}]} 2024-04-04T06:17:27.100 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 107 2024-04-04T06:17:27.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.107+0000 7f929b200640 1 -- 172.21.15.159:0/547284283 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f927406c8f0 msgr2=0x7f927406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:27.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.107+0000 7f929b200640 1 --2- 172.21.15.159:0/547284283 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f927406c8f0 0x7f927406edb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9284036000 tx=0x7f9284008030 comp rx=0 tx=0).stop 2024-04-04T06:17:27.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.107+0000 7f929b200640 1 -- 172.21.15.159:0/547284283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92940cfb20 msgr2=0x7f92940c9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:27.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.107+0000 7f929b200640 1 --2- 172.21.15.159:0/547284283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92940cfb20 0x7f92940c9e00 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f928000c400 tx=0x7f928000c8c0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.107+0000 7f929b200640 1 -- 172.21.15.159:0/547284283 shutdown_connections 2024-04-04T06:17:27.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.107+0000 7f929b200640 1 --2- 172.21.15.159:0/547284283 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f927406c8f0 0x7f927406edb0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.107+0000 7f929b200640 1 --2- 172.21.15.159:0/547284283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f929415e7e0 0x7f929415ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.107+0000 7f929b200640 1 --2- 172.21.15.159:0/547284283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92940cfb20 0x7f92940c9e00 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.107+0000 7f929b200640 1 --2- 172.21.15.159:0/547284283 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f929406ca00 0x7f9294152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.107+0000 7f929b200640 1 -- 172.21.15.159:0/547284283 >> 172.21.15.159:0/547284283 conn(0x7f929409d7c0 msgr2=0x7f92940b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:27.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.107+0000 7f929b200640 1 -- 172.21.15.159:0/547284283 shutdown_connections 2024-04-04T06:17:27.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.107+0000 7f929b200640 1 -- 172.21.15.159:0/547284283 wait complete. 2024-04-04T06:17:27.525 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 107, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:16:57.277344+0000', 'modified': '2024-04-04T06:17:25.827822+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 114, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9135}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9135': {'gid': 9135, 'name': 'a', 'rank': 0, 'incarnation': 104, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.159:6834/2439261212', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 2439261212}, {'type': 'v1', 'addr': '172.21.15.159:6834', 'nonce': 2439261212}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9135, 'qdb_cluster': [9135]} 2024-04-04T06:17:27.525 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-04T06:17:27.526 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-04T06:17:27.610 INFO:tasks.ceph.mds.a.smithi159.stderr:2024-04-04T06:17:27.615+0000 7fc6eee00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:17:27.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.743+0000 7f06eda00640 1 Processor -- start 2024-04-04T06:17:27.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.744+0000 7f06eda00640 1 -- start start 2024-04-04T06:17:27.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.745+0000 7f06eda00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06e8155550 0x7f06e8155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:27.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.745+0000 7f06eda00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f06e815f800 0x7f06e80d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:27.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.745+0000 7f06eda00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06e80b40b0 con 0x7f06e80d07c0 2024-04-04T06:17:27.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.745+0000 7f06eda00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f06e80d0c30 con 0x7f06e815f800 2024-04-04T06:17:27.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.745+0000 7f06eda00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f06e80b4390 con 0x7f06e8155550 2024-04-04T06:17:27.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.745+0000 7f06de000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f06e815f800 0x7f06e80d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:27.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.746+0000 7f06de000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f06e815f800 0x7f06e80d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48306/0 (socket says 172.21.15.159:48306) 2024-04-04T06:17:27.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.746+0000 7f06de000640 1 -- 172.21.15.159:0/3209632521 learned_addr learned my addr 172.21.15.159:0/3209632521 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:27.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.746+0000 7f06e6a00640 1 --2- 172.21.15.159:0/3209632521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06e8155550 0x7f06e8155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:27.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.746+0000 7f06de000640 1 -- 172.21.15.159:0/3209632521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06e8155550 msgr2=0x7f06e8155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:27.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.746+0000 7f06de000640 1 --2- 172.21.15.159:0/3209632521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06e8155550 0x7f06e8155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.746+0000 7f06de000640 1 -- 172.21.15.159:0/3209632521 >> v1:172.21.15.178:6789/0 conn(0x7f06e80d07c0 legacy=0x7f06e80aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:27.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.746+0000 7f06de000640 1 -- 172.21.15.159:0/3209632521 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e80b4670 con 0x7f06e815f800 2024-04-04T06:17:27.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.747+0000 7f06de000640 1 --2- 172.21.15.159:0/3209632521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06e815f800 0x7f06e80d0280 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009870 tx=0x7f06d802db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8746e45c619312a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:27.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.747+0000 7f06e6000640 1 -- 172.21.15.159:0/3209632521 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d803b070 con 0x7f06e815f800 2024-04-04T06:17:27.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.748+0000 7f06e6000640 1 -- 172.21.15.159:0/3209632521 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06d802e940 con 0x7f06e815f800 2024-04-04T06:17:27.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.748+0000 7f06e6000640 1 -- 172.21.15.159:0/3209632521 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d802ed10 con 0x7f06e815f800 2024-04-04T06:17:27.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.748+0000 7f06eda00640 1 -- 172.21.15.159:0/3209632521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06e815f800 msgr2=0x7f06e80d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:27.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.748+0000 7f06eda00640 1 --2- 172.21.15.159:0/3209632521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06e815f800 0x7f06e80d0280 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009870 tx=0x7f06d802db90 comp rx=0 tx=0).stop 2024-04-04T06:17:27.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.748+0000 7f06eda00640 1 -- 172.21.15.159:0/3209632521 shutdown_connections 2024-04-04T06:17:27.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.748+0000 7f06eda00640 1 --2- 172.21.15.159:0/3209632521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06e815f800 0x7f06e80d0280 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.748+0000 7f06eda00640 1 --2- 172.21.15.159:0/3209632521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06e8155550 0x7f06e8155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.748+0000 7f06eda00640 1 -- 172.21.15.159:0/3209632521 >> 172.21.15.159:0/3209632521 conn(0x7f06e806b750 msgr2=0x7f06e806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:27.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.748+0000 7f06eda00640 1 -- 172.21.15.159:0/3209632521 shutdown_connections 2024-04-04T06:17:27.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.748+0000 7f06eda00640 1 -- 172.21.15.159:0/3209632521 wait complete. 2024-04-04T06:17:27.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.750+0000 7f06eda00640 1 Processor -- start 2024-04-04T06:17:27.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.750+0000 7f06eda00640 1 -- start start 2024-04-04T06:17:27.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.751+0000 7f06eda00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06e80d07c0 0x7f06e806e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:27.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.751+0000 7f06eda00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06e8155550 0x7f06e809ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:27.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.751+0000 7f06eda00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06e815f800 0x7f06e81f03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:27.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.751+0000 7f06eda00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f06e815fc80 con 0x7f06e80d07c0 2024-04-04T06:17:27.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.751+0000 7f06eda00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f06e80b4c30 con 0x7f06e815f800 2024-04-04T06:17:27.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.751+0000 7f06eda00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f06e815ba20 con 0x7f06e8155550 2024-04-04T06:17:27.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.752+0000 7f06e6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06e80d07c0 0x7f06e806e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:27.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.752+0000 7f06e6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06e80d07c0 0x7f06e806e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48312/0 (socket says 172.21.15.159:48312) 2024-04-04T06:17:27.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.752+0000 7f06e6a00640 1 -- 172.21.15.159:0/194557484 learned_addr learned my addr 172.21.15.159:0/194557484 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:27.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.752+0000 7f06e6a00640 1 -- 172.21.15.159:0/194557484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06e8155550 msgr2=0x7f06e809ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:27.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.752+0000 7f06e6a00640 1 --2- 172.21.15.159:0/194557484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06e8155550 0x7f06e809ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.752+0000 7f06e7400640 1 --2- 172.21.15.159:0/194557484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06e815f800 0x7f06e81f03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:27.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.752+0000 7f06e6a00640 1 -- 172.21.15.159:0/194557484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06e815f800 msgr2=0x7f06e81f03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:27.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.752+0000 7f06e6a00640 1 --2- 172.21.15.159:0/194557484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06e815f800 0x7f06e81f03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.752+0000 7f06e6a00640 1 -- 172.21.15.159:0/194557484 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e809cd90 con 0x7f06e80d07c0 2024-04-04T06:17:27.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.752+0000 7f06e7400640 1 --2- 172.21.15.159:0/194557484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06e815f800 0x7f06e81f03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:27.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.752+0000 7f06e6a00640 1 --2- 172.21.15.159:0/194557484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06e80d07c0 0x7f06e806e510 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f06d000d5f0 tx=0x7f06d000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:27.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.753+0000 7f06df400640 1 -- 172.21.15.159:0/194557484 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d00125f0 con 0x7f06e80d07c0 2024-04-04T06:17:27.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.753+0000 7f06df400640 1 -- 172.21.15.159:0/194557484 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06d0019070 con 0x7f06e80d07c0 2024-04-04T06:17:27.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.753+0000 7f06df400640 1 -- 172.21.15.159:0/194557484 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d00129d0 con 0x7f06e80d07c0 2024-04-04T06:17:27.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.753+0000 7f06eda00640 1 -- 172.21.15.159:0/194557484 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e809a2c0 con 0x7f06e80d07c0 2024-04-04T06:17:27.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.753+0000 7f06eda00640 1 -- 172.21.15.159:0/194557484 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e809a880 con 0x7f06e80d07c0 2024-04-04T06:17:27.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.755+0000 7f06df400640 1 -- 172.21.15.159:0/194557484 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f06d0012b70 con 0x7f06e80d07c0 2024-04-04T06:17:27.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.756+0000 7f06df400640 1 --2- 172.21.15.159:0/194557484 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f06c406c8f0 0x7f06c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:27.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.756+0000 7f06df400640 1 -- 172.21.15.159:0/194557484 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5795+0+0 (secure 0 0 0) 0x7f06d008bfc0 con 0x7f06e80d07c0 2024-04-04T06:17:27.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.757+0000 7f06f076e740 1 Processor -- start 2024-04-04T06:17:27.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.757+0000 7f06de000640 1 --2- 172.21.15.159:0/194557484 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f06c406c8f0 0x7f06c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:27.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.757+0000 7f06f076e740 1 -- start start 2024-04-04T06:17:27.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.758+0000 7f06f076e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef8f3e1cb0 0x55ef8f3df870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:27.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.758+0000 7f06de000640 1 --2- 172.21.15.159:0/194557484 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f06c406c8f0 0x7f06c406edb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f06d8038000 tx=0x7f06d8030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:27.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.758+0000 7f06f076e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef8f3c14f0 0x55ef8f3991e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:27.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.758+0000 7f06f076e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f399720 0x55ef8f3c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:27.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.758+0000 7f06f076e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ef8f35bab0 con 0x55ef8f399720 2024-04-04T06:17:27.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.758+0000 7f06f076e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ef8f2fdfa0 con 0x55ef8f3e1cb0 2024-04-04T06:17:27.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.758+0000 7f06f076e740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ef8f2a6560 con 0x55ef8f3c14f0 2024-04-04T06:17:27.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.758+0000 7f06e7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f399720 0x55ef8f3c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:27.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.758+0000 7f06e7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f399720 0x55ef8f3c3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48320/0 (socket says 172.21.15.159:48320) 2024-04-04T06:17:27.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.758+0000 7f06e7400640 1 -- 172.21.15.159:0/3792262448 learned_addr learned my addr 172.21.15.159:0/3792262448 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:27.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.758+0000 7f06e6a00640 1 --2- 172.21.15.159:0/3792262448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef8f3e1cb0 0x55ef8f3df870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:27.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.758+0000 7f06de000640 1 --2- 172.21.15.159:0/3792262448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef8f3c14f0 0x55ef8f3991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:27.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.758+0000 7f06e7400640 1 -- 172.21.15.159:0/3792262448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef8f3c14f0 msgr2=0x55ef8f3991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:27.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.759+0000 7f06e7400640 1 --2- 172.21.15.159:0/3792262448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef8f3c14f0 0x55ef8f3991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.759+0000 7f06e7400640 1 -- 172.21.15.159:0/3792262448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef8f3e1cb0 msgr2=0x55ef8f3df870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:27.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.759+0000 7f06e7400640 1 --2- 172.21.15.159:0/3792262448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef8f3e1cb0 0x55ef8f3df870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.759+0000 7f06e7400640 1 -- 172.21.15.159:0/3792262448 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef8f0122d0 con 0x55ef8f399720 2024-04-04T06:17:27.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.759+0000 7f06e7400640 1 --2- 172.21.15.159:0/3792262448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f399720 0x55ef8f3c3ec0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f06e0001dd0 tx=0x7f06e000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ecb914e9233cf14 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:27.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.759+0000 7f06d7400640 1 -- 172.21.15.159:0/3792262448 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e0018070 con 0x55ef8f399720 2024-04-04T06:17:27.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.759+0000 7f06d7400640 1 -- 172.21.15.159:0/3792262448 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06e000de30 con 0x55ef8f399720 2024-04-04T06:17:27.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.759+0000 7f06d7400640 1 -- 172.21.15.159:0/3792262448 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e0003b90 con 0x55ef8f399720 2024-04-04T06:17:27.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.760+0000 7f06f076e740 1 -- 172.21.15.159:0/3792262448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f399720 msgr2=0x55ef8f3c3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:27.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.760+0000 7f06f076e740 1 --2- 172.21.15.159:0/3792262448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f399720 0x55ef8f3c3ec0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f06e0001dd0 tx=0x7f06e000d520 comp rx=0 tx=0).stop 2024-04-04T06:17:27.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.760+0000 7f06f076e740 1 -- 172.21.15.159:0/3792262448 shutdown_connections 2024-04-04T06:17:27.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.760+0000 7f06f076e740 1 --2- 172.21.15.159:0/3792262448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f399720 0x55ef8f3c3ec0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.760+0000 7f06f076e740 1 --2- 172.21.15.159:0/3792262448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef8f3c14f0 0x55ef8f3991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.760+0000 7f06f076e740 1 --2- 172.21.15.159:0/3792262448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef8f3e1cb0 0x55ef8f3df870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.760+0000 7f06f076e740 1 -- 172.21.15.159:0/3792262448 >> 172.21.15.159:0/3792262448 conn(0x55ef8f396340 msgr2=0x55ef8f3deec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:27.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.760+0000 7f06f076e740 1 -- 172.21.15.159:0/3792262448 shutdown_connections 2024-04-04T06:17:27.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.760+0000 7f06f076e740 1 -- 172.21.15.159:0/3792262448 wait complete. 2024-04-04T06:17:27.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.761+0000 7f06f076e740 1 -- start start 2024-04-04T06:17:27.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.761+0000 7f06f076e740 1 Processor -- start 2024-04-04T06:17:27.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.762+0000 7f06f076e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef8f3c14f0 0x55ef8f3de1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:27.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.762+0000 7f06f076e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f3e1cb0 0x55ef8f3d6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:27.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.762+0000 7f06f076e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef8f3da3b0 0x55ef8f3dbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:27.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.762+0000 7f06f076e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ef8f2b9920 con 0x55ef8f3e1cb0 2024-04-04T06:17:27.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.762+0000 7f06f076e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ef8f2f9f50 con 0x55ef8f3da3b0 2024-04-04T06:17:27.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.762+0000 7f06f076e740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ef8f398640 con 0x55ef8f3c14f0 2024-04-04T06:17:27.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.762+0000 7f06f076e740 10 client.0 _ll_register_callbacks cb 0x55ef8f2b9310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:17:27.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.762+0000 7f06de000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f3e1cb0 0x55ef8f3d6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:27.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.762+0000 7f06de000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f3e1cb0 0x55ef8f3d6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48336/0 (socket says 172.21.15.159:48336) 2024-04-04T06:17:27.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.762+0000 7f06de000640 1 -- 172.21.15.159:0/2868036367 learned_addr learned my addr 172.21.15.159:0/2868036367 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:27.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.763+0000 7f06e7400640 1 --2- 172.21.15.159:0/2868036367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef8f3da3b0 0x55ef8f3dbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:27.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.763+0000 7f06de000640 1 -- 172.21.15.159:0/2868036367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef8f3c14f0 msgr2=0x55ef8f3de1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:27.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.763+0000 7f06e6a00640 1 --2- 172.21.15.159:0/2868036367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef8f3c14f0 0x55ef8f3de1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:27.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.763+0000 7f06de000640 1 --2- 172.21.15.159:0/2868036367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef8f3c14f0 0x55ef8f3de1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.763+0000 7f06de000640 1 -- 172.21.15.159:0/2868036367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef8f3da3b0 msgr2=0x55ef8f3dbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:27.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.763+0000 7f06de000640 1 --2- 172.21.15.159:0/2868036367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef8f3da3b0 0x55ef8f3dbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.763+0000 7f06de000640 1 -- 172.21.15.159:0/2868036367 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef8f0dc6b0 con 0x55ef8f3e1cb0 2024-04-04T06:17:27.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.763+0000 7f06de000640 1 --2- 172.21.15.159:0/2868036367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f3e1cb0 0x55ef8f3d6470 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f06d8039040 tx=0x7f06d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd0dc30950fda19 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:27.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.763+0000 7f06cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:17:27.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.763+0000 7f06cbe00640 1 -- 172.21.15.159:0/2868036367 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d803b070 con 0x55ef8f3e1cb0 2024-04-04T06:17:27.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.763+0000 7f06cbe00640 1 -- 172.21.15.159:0/2868036367 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 5795+0+0 (secure 0 0 0) 0x7f06d804db00 con 0x55ef8f3e1cb0 2024-04-04T06:17:27.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.763+0000 7f06f076e740 1 -- 172.21.15.159:0/2868036367 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06e000de30 con 0x55ef8f3e1cb0 2024-04-04T06:17:27.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.764+0000 7f06cbe00640 1 -- 172.21.15.159:0/2868036367 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f06d804dd00 con 0x55ef8f3e1cb0 2024-04-04T06:17:27.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.764+0000 7f06f076e740 10 client.9177 fetch_fsmap learned FSMap version 107 2024-04-04T06:17:27.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.764+0000 7f06f076e740 1 -- 172.21.15.159:0/2868036367 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x55ef8f3d69b0 con 0x55ef8f3e1cb0 2024-04-04T06:17:27.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.764+0000 7f06cbe00640 1 -- 172.21.15.159:0/2868036367 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06e000de30 con 0x55ef8f3e1cb0 2024-04-04T06:17:27.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.764+0000 7f06cbe00640 1 -- 172.21.15.159:0/2868036367 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 107) ==== 3494+0+0 (secure 0 0 0) 0x7f06d804d020 con 0x55ef8f3e1cb0 2024-04-04T06:17:27.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.764+0000 7f06f076e740 10 client.9177 fetch_fsmap finished waiting for FSMap version 107 2024-04-04T06:17:27.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.764+0000 7f06f076e740 10 client.9177 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:17:27.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.764+0000 7f06f076e740 20 client.9177 populate_metadata read hostname 'smithi159' 2024-04-04T06:17:27.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.764+0000 7f06f076e740 1 --2- 172.21.15.159:0/2868036367 >> [v2:172.21.15.159:6832/2439261212,v1:172.21.15.159:6834/2439261212] conn(0x55ef8f3da7b0 0x55ef8f3d7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:27.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.764+0000 7f06f076e740 4 client.9177 mds_command: new command op to 9135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:17:27.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.764+0000 7f06f076e740 1 -- 172.21.15.159:0/2868036367 --> [v2:172.21.15.159:6832/2439261212,v1:172.21.15.159:6834/2439261212] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef8f30b4e0 con 0x55ef8f3da7b0 2024-04-04T06:17:27.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.764+0000 7f06e6a00640 1 --2- 172.21.15.159:0/2868036367 >> [v2:172.21.15.159:6832/2439261212,v1:172.21.15.159:6834/2439261212] conn(0x55ef8f3da7b0 0x55ef8f3d7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:27.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.765+0000 7f06e6a00640 1 --2- 172.21.15.159:0/2868036367 >> [v2:172.21.15.159:6832/2439261212,v1:172.21.15.159:6834/2439261212] conn(0x55ef8f3da7b0 0x55ef8f3d7780 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b41786e21b38c4 server_cookie=9091a8a4539670 in_seq=0 out_seq=0 2024-04-04T06:17:27.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.765+0000 7f06cbe00640 10 client.9177 ms_handle_connect on v2:172.21.15.159:6832/2439261212 2024-04-04T06:17:27.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.766+0000 7f06cbe00640 1 -- 172.21.15.159:0/2868036367 <== mds.0 v2:172.21.15.159:6832/2439261212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef8f30b4e0 con 0x55ef8f3da7b0 2024-04-04T06:17:27.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.766+0000 7f06cbe00640 10 client.9177 handle_command_reply: tid=0 2024-04-04T06:17:27.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.766+0000 7f06f076e740 1 client.9177 shutdown 2024-04-04T06:17:27.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.766+0000 7f06f076e740 1 -- 172.21.15.159:0/2868036367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f3e1cb0 msgr2=0x55ef8f3d6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:27.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.766+0000 7f06f076e740 1 --2- 172.21.15.159:0/2868036367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f3e1cb0 0x55ef8f3d6470 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f06d8039040 tx=0x7f06d8004730 comp rx=0 tx=0).stop 2024-04-04T06:17:27.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.766+0000 7f06f076e740 1 -- 172.21.15.159:0/2868036367 shutdown_connections 2024-04-04T06:17:27.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.766+0000 7f06f076e740 1 --2- 172.21.15.159:0/2868036367 >> [v2:172.21.15.159:6832/2439261212,v1:172.21.15.159:6834/2439261212] conn(0x55ef8f3da7b0 0x55ef8f3d7780 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.766+0000 7f06f076e740 1 --2- 172.21.15.159:0/2868036367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef8f3da3b0 0x55ef8f3dbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.766+0000 7f06f076e740 1 --2- 172.21.15.159:0/2868036367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f3e1cb0 0x55ef8f3d6470 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.766+0000 7f06f076e740 1 --2- 172.21.15.159:0/2868036367 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef8f3c14f0 0x55ef8f3de1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.766+0000 7f06f076e740 1 -- 172.21.15.159:0/2868036367 >> 172.21.15.159:0/2868036367 conn(0x55ef8f399720 msgr2=0x55ef8f3dede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:27.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.766+0000 7f06cbe00640 0 client.9177 ms_handle_reset on v2:172.21.15.159:6832/2439261212 2024-04-04T06:17:27.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.766+0000 7f06f076e740 1 -- 172.21.15.159:0/2868036367 shutdown_connections 2024-04-04T06:17:27.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.766+0000 7f06f076e740 1 -- 172.21.15.159:0/2868036367 wait complete. 2024-04-04T06:17:27.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.766+0000 7f06f076e740 20 client.9177 trim_cache size 0 max 16384 2024-04-04T06:17:27.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.786+0000 7f06f076e740 1 Processor -- start 2024-04-04T06:17:27.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.786+0000 7f06f076e740 1 -- start start 2024-04-04T06:17:27.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.787+0000 7f06f076e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef8f3c14f0 0x55ef8f406d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:27.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.787+0000 7f06f076e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef8f3da3b0 0x55ef8f3d60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:27.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.787+0000 7f06f076e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f3e1cb0 0x55ef8f3f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:27.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.787+0000 7f06f076e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f06e81f4680 con 0x55ef8f3e1cb0 2024-04-04T06:17:27.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.787+0000 7f06f076e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f06e81f39e0 con 0x55ef8f3da3b0 2024-04-04T06:17:27.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.787+0000 7f06f076e740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f06e81f4030 con 0x55ef8f3c14f0 2024-04-04T06:17:27.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.787+0000 7f06e7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f3e1cb0 0x55ef8f3f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:27.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.787+0000 7f06e7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f3e1cb0 0x55ef8f3f5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48344/0 (socket says 172.21.15.159:48344) 2024-04-04T06:17:27.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.787+0000 7f06e7400640 1 -- 172.21.15.159:0/1380610619 learned_addr learned my addr 172.21.15.159:0/1380610619 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:27.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.788+0000 7f06de000640 1 --2- 172.21.15.159:0/1380610619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef8f3da3b0 0x55ef8f3d60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:27.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.788+0000 7f06e6a00640 1 --2- 172.21.15.159:0/1380610619 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef8f3c14f0 0x55ef8f406d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:27.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.788+0000 7f06e7400640 1 -- 172.21.15.159:0/1380610619 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef8f3c14f0 msgr2=0x55ef8f406d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:27.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.788+0000 7f06e7400640 1 --2- 172.21.15.159:0/1380610619 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef8f3c14f0 0x55ef8f406d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.788+0000 7f06e7400640 1 -- 172.21.15.159:0/1380610619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef8f3da3b0 msgr2=0x55ef8f3d60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:27.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.788+0000 7f06e7400640 1 --2- 172.21.15.159:0/1380610619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef8f3da3b0 0x55ef8f3d60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.788+0000 7f06e7400640 1 -- 172.21.15.159:0/1380610619 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef8f39a6c0 con 0x55ef8f3e1cb0 2024-04-04T06:17:27.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.788+0000 7f06e6a00640 1 --2- 172.21.15.159:0/1380610619 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef8f3c14f0 0x55ef8f406d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:27.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.788+0000 7f06e7400640 1 --2- 172.21.15.159:0/1380610619 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f3e1cb0 0x55ef8f3f5d90 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f06e0002230 tx=0x7f06e000b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c962670c91308e2c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:27.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.788+0000 7f06d6a00640 1 -- 172.21.15.159:0/1380610619 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e0018070 con 0x55ef8f3e1cb0 2024-04-04T06:17:27.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.788+0000 7f06d6a00640 1 -- 172.21.15.159:0/1380610619 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06e0019040 con 0x55ef8f3e1cb0 2024-04-04T06:17:27.785 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.788+0000 7f06d6a00640 1 -- 172.21.15.159:0/1380610619 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e000de50 con 0x55ef8f3e1cb0 2024-04-04T06:17:27.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.789+0000 7f06f076e740 1 -- 172.21.15.159:0/1380610619 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f3e1cb0 msgr2=0x55ef8f3f5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:27.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.789+0000 7f06f076e740 1 --2- 172.21.15.159:0/1380610619 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f3e1cb0 0x55ef8f3f5d90 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f06e0002230 tx=0x7f06e000b000 comp rx=0 tx=0).stop 2024-04-04T06:17:27.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.789+0000 7f06f076e740 1 -- 172.21.15.159:0/1380610619 shutdown_connections 2024-04-04T06:17:27.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.789+0000 7f06f076e740 1 --2- 172.21.15.159:0/1380610619 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f3e1cb0 0x55ef8f3f5d90 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.789+0000 7f06f076e740 1 --2- 172.21.15.159:0/1380610619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef8f3da3b0 0x55ef8f3d60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.789+0000 7f06f076e740 1 --2- 172.21.15.159:0/1380610619 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef8f3c14f0 0x55ef8f406d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.789+0000 7f06f076e740 1 -- 172.21.15.159:0/1380610619 >> 172.21.15.159:0/1380610619 conn(0x55ef8f399720 msgr2=0x55ef8f3e11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:27.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.789+0000 7f06f076e740 1 -- 172.21.15.159:0/1380610619 shutdown_connections 2024-04-04T06:17:27.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.790+0000 7f06f076e740 1 -- 172.21.15.159:0/1380610619 wait complete. 2024-04-04T06:17:27.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.790+0000 7f06f076e740 1 -- start start 2024-04-04T06:17:27.786 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.790+0000 7f06f076e740 1 Processor -- start 2024-04-04T06:17:27.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.791+0000 7f06f076e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f3c14f0 0x55ef8f457750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:27.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.791+0000 7f06f076e740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef8f3da3b0 0x55ef8f3981c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:27.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.791+0000 7f06f076e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef8f3e1cb0 0x55ef8f398700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:27.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.791+0000 7f06f076e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ef8f3dc2c0 con 0x55ef8f3c14f0 2024-04-04T06:17:27.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.791+0000 7f06f076e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ef8f3d6c60 con 0x55ef8f3e1cb0 2024-04-04T06:17:27.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.791+0000 7f06f076e740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ef8f3bcf50 con 0x55ef8f3da3b0 2024-04-04T06:17:27.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.791+0000 7f06f076e740 10 client.0 _ll_register_callbacks cb 0x7f06d802ec10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:17:27.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.792+0000 7f06e6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f3c14f0 0x55ef8f457750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:27.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.792+0000 7f06e6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f3c14f0 0x55ef8f457750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48358/0 (socket says 172.21.15.159:48358) 2024-04-04T06:17:27.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.792+0000 7f06e6a00640 1 -- 172.21.15.159:0/1803400818 learned_addr learned my addr 172.21.15.159:0/1803400818 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:27.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.792+0000 7f06e6a00640 1 -- 172.21.15.159:0/1803400818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef8f3da3b0 msgr2=0x55ef8f3981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:27.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.792+0000 7f06e6a00640 1 --2- 172.21.15.159:0/1803400818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef8f3da3b0 0x55ef8f3981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.792+0000 7f06e6a00640 1 -- 172.21.15.159:0/1803400818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef8f3e1cb0 msgr2=0x55ef8f398700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:27.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.792+0000 7f06e6a00640 1 --2- 172.21.15.159:0/1803400818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef8f3e1cb0 0x55ef8f398700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.792+0000 7f06e6a00640 1 -- 172.21.15.159:0/1803400818 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef8f3c83b0 con 0x55ef8f3c14f0 2024-04-04T06:17:27.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.792+0000 7f06e6a00640 1 --2- 172.21.15.159:0/1803400818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f3c14f0 0x55ef8f457750 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f06d00560f0 tx=0x7f06d000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569a8fd72049f894 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:27.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.792+0000 7f06cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:17:27.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.792+0000 7f06cbe00640 1 -- 172.21.15.159:0/1803400818 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0072220 con 0x55ef8f3c14f0 2024-04-04T06:17:27.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.793+0000 7f06cbe00640 1 -- 172.21.15.159:0/1803400818 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 5795+0+0 (secure 0 0 0) 0x7f06d0060040 con 0x55ef8f3c14f0 2024-04-04T06:17:27.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.793+0000 7f06f076e740 1 -- 172.21.15.159:0/1803400818 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06e0019040 con 0x55ef8f3c14f0 2024-04-04T06:17:27.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.793+0000 7f06cbe00640 1 -- 172.21.15.159:0/1803400818 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f06d0057f90 con 0x55ef8f3c14f0 2024-04-04T06:17:27.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.793+0000 7f06f076e740 10 client.9183 fetch_fsmap learned FSMap version 107 2024-04-04T06:17:27.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.793+0000 7f06f076e740 1 -- 172.21.15.159:0/1803400818 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x55ef8f45e130 con 0x55ef8f3c14f0 2024-04-04T06:17:27.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.793+0000 7f06cbe00640 1 -- 172.21.15.159:0/1803400818 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06e0019040 con 0x55ef8f3c14f0 2024-04-04T06:17:27.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.793+0000 7f06cbe00640 1 -- 172.21.15.159:0/1803400818 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 107) ==== 3494+0+0 (secure 0 0 0) 0x7f06d0013070 con 0x55ef8f3c14f0 2024-04-04T06:17:27.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.793+0000 7f06f076e740 10 client.9183 fetch_fsmap finished waiting for FSMap version 107 2024-04-04T06:17:27.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.793+0000 7f06f076e740 10 client.9183 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:17:27.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.793+0000 7f06f076e740 20 client.9183 populate_metadata read hostname 'smithi159' 2024-04-04T06:17:27.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.793+0000 7f06f076e740 1 --2- 172.21.15.159:0/1803400818 >> [v2:172.21.15.159:6832/2439261212,v1:172.21.15.159:6834/2439261212] conn(0x55ef8f3e0a40 0x55ef8f3bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:27.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.793+0000 7f06f076e740 4 client.9183 mds_command: new command op to 9135 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-04T06:17:27.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.793+0000 7f06f076e740 1 -- 172.21.15.159:0/1803400818 --> [v2:172.21.15.159:6832/2439261212,v1:172.21.15.159:6834/2439261212] -- command(tid 0: {"prefix": "status"}) -- 0x55ef8f305830 con 0x55ef8f3e0a40 2024-04-04T06:17:27.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.793+0000 7f06de000640 1 --2- 172.21.15.159:0/1803400818 >> [v2:172.21.15.159:6832/2439261212,v1:172.21.15.159:6834/2439261212] conn(0x55ef8f3e0a40 0x55ef8f3bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:27.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.794+0000 7f06de000640 1 --2- 172.21.15.159:0/1803400818 >> [v2:172.21.15.159:6832/2439261212,v1:172.21.15.159:6834/2439261212] conn(0x55ef8f3e0a40 0x55ef8f3bfdb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5ffa7ab0eb65404 server_cookie=9ca540ee58013caa in_seq=0 out_seq=0 2024-04-04T06:17:27.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.794+0000 7f06cbe00640 10 client.9183 ms_handle_connect on v2:172.21.15.159:6832/2439261212 2024-04-04T06:17:27.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.794+0000 7f06cbe00640 1 -- 172.21.15.159:0/1803400818 <== mds.0 v2:172.21.15.159:6832/2439261212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55ef8f305830 con 0x55ef8f3e0a40 2024-04-04T06:17:27.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.794+0000 7f06cbe00640 10 client.9183 handle_command_reply: tid=0 2024-04-04T06:17:27.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.794+0000 7f06f076e740 1 client.9183 shutdown 2024-04-04T06:17:27.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.794+0000 7f06f076e740 1 -- 172.21.15.159:0/1803400818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f3c14f0 msgr2=0x55ef8f457750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:27.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.794+0000 7f06f076e740 1 --2- 172.21.15.159:0/1803400818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f3c14f0 0x55ef8f457750 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f06d00560f0 tx=0x7f06d000b260 comp rx=0 tx=0).stop 2024-04-04T06:17:27.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.794+0000 7f06f076e740 1 -- 172.21.15.159:0/1803400818 shutdown_connections 2024-04-04T06:17:27.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.794+0000 7f06f076e740 1 --2- 172.21.15.159:0/1803400818 >> [v2:172.21.15.159:6832/2439261212,v1:172.21.15.159:6834/2439261212] conn(0x55ef8f3e0a40 0x55ef8f3bfdb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.794+0000 7f06f076e740 1 --2- 172.21.15.159:0/1803400818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ef8f3e1cb0 0x55ef8f398700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.794+0000 7f06f076e740 1 --2- 172.21.15.159:0/1803400818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ef8f3da3b0 0x55ef8f3981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.794+0000 7f06f076e740 1 --2- 172.21.15.159:0/1803400818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ef8f3c14f0 0x55ef8f457750 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.794+0000 7f06f076e740 1 -- 172.21.15.159:0/1803400818 >> 172.21.15.159:0/1803400818 conn(0x55ef8f399720 msgr2=0x55ef8f3f5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:27.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.795+0000 7f06cbe00640 0 client.9183 ms_handle_reset on v2:172.21.15.159:6832/2439261212 2024-04-04T06:17:27.793 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.795+0000 7f06f076e740 1 -- 172.21.15.159:0/1803400818 shutdown_connections 2024-04-04T06:17:27.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.795+0000 7f06f076e740 1 -- 172.21.15.159:0/1803400818 wait complete. 2024-04-04T06:17:27.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.795+0000 7f06f076e740 20 client.9183 trim_cache size 0 max 16384 2024-04-04T06:17:27.794 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:17:27.794 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_fsid": "30311045-f714-48fa-832b-4cb2ba8bf0bf", 2024-04-04T06:17:27.794 INFO:teuthology.orchestra.run.smithi159.stdout: "whoami": 0, 2024-04-04T06:17:27.794 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 9135, 2024-04-04T06:17:27.794 INFO:teuthology.orchestra.run.smithi159.stdout: "want_state": "up:active", 2024-04-04T06:17:27.794 INFO:teuthology.orchestra.run.smithi159.stdout: "state": "up:active", 2024-04-04T06:17:27.794 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs", 2024-04-04T06:17:27.795 INFO:teuthology.orchestra.run.smithi159.stdout: "rank_uptime": 4.9867551350000001, 2024-04-04T06:17:27.795 INFO:teuthology.orchestra.run.smithi159.stdout: "mdsmap_epoch": 107, 2024-04-04T06:17:27.795 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch": 114, 2024-04-04T06:17:27.795 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch_barrier": 114, 2024-04-04T06:17:27.795 INFO:teuthology.orchestra.run.smithi159.stdout: "uptime": 6.0916451450000002 2024-04-04T06:17:27.795 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:17:27.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.795+0000 7f06eda00640 1 -- 172.21.15.159:0/194557484 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f06c406c8f0 msgr2=0x7f06c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:27.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.795+0000 7f06eda00640 1 --2- 172.21.15.159:0/194557484 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f06c406c8f0 0x7f06c406edb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f06d8038000 tx=0x7f06d8030000 comp rx=0 tx=0).stop 2024-04-04T06:17:27.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.795+0000 7f06eda00640 1 -- 172.21.15.159:0/194557484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06e80d07c0 msgr2=0x7f06e806e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:27.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.795+0000 7f06eda00640 1 --2- 172.21.15.159:0/194557484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06e80d07c0 0x7f06e806e510 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f06d000d5f0 tx=0x7f06d000dab0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.796+0000 7f06eda00640 1 -- 172.21.15.159:0/194557484 shutdown_connections 2024-04-04T06:17:27.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.796+0000 7f06eda00640 1 --2- 172.21.15.159:0/194557484 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f06c406c8f0 0x7f06c406edb0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.796+0000 7f06eda00640 1 --2- 172.21.15.159:0/194557484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06e815f800 0x7f06e81f03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.796+0000 7f06eda00640 1 --2- 172.21.15.159:0/194557484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f06e8155550 0x7f06e809ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.796+0000 7f06eda00640 1 --2- 172.21.15.159:0/194557484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06e80d07c0 0x7f06e806e510 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:27.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.796+0000 7f06eda00640 1 -- 172.21.15.159:0/194557484 >> 172.21.15.159:0/194557484 conn(0x7f06e806b750 msgr2=0x7f06e815caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:27.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.796+0000 7f06eda00640 1 -- 172.21.15.159:0/194557484 shutdown_connections 2024-04-04T06:17:27.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:27.796+0000 7f06eda00640 1 -- 172.21.15.159:0/194557484 wait complete. 2024-04-04T06:17:27.808 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-04T06:17:28.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.036+0000 7fa422000640 1 Processor -- start 2024-04-04T06:17:28.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.036+0000 7fa422000640 1 -- start start 2024-04-04T06:17:28.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.038+0000 7fa422000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa41c0ccc40 0x7fa41c06c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:28.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.038+0000 7fa422000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa41c0b4710 con 0x7fa41c0afd80 2024-04-04T06:17:28.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.038+0000 7fa422000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa41c0b4980 con 0x7fa41c06cac0 2024-04-04T06:17:28.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.038+0000 7fa422000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa41c0b5740 con 0x7fa41c0ccc40 2024-04-04T06:17:28.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.038+0000 7fa41b400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fa41c0afd80 0x7fa41c15d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:50972/0 (socket says 172.21.15.159:50972) 2024-04-04T06:17:28.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.038+0000 7fa41b400640 1 -- 172.21.15.159:0/2584833845 learned_addr learned my addr 172.21.15.159:0/2584833845 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:28.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.039+0000 7fa413e00640 1 --2- 172.21.15.159:0/2584833845 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa41c0ccc40 0x7fa41c06c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:28.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.039+0000 7fa41aa00640 1 -- 172.21.15.159:0/2584833845 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 519355869 0 0) 0x7fa41c0b4710 con 0x7fa41c0afd80 2024-04-04T06:17:28.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.039+0000 7fa41aa00640 1 -- 172.21.15.159:0/2584833845 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa400003520 con 0x7fa41c0afd80 2024-04-04T06:17:28.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.039+0000 7fa41aa00640 1 -- 172.21.15.159:0/2584833845 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 730228426 0 0) 0x7fa400003520 con 0x7fa41c0afd80 2024-04-04T06:17:28.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.039+0000 7fa41aa00640 1 -- 172.21.15.159:0/2584833845 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa41c0ccc40 msgr2=0x7fa41c06c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:28.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.039+0000 7fa41aa00640 1 --2- 172.21.15.159:0/2584833845 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa41c0ccc40 0x7fa41c06c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:28.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.039+0000 7fa41aa00640 1 -- 172.21.15.159:0/2584833845 >> v1:172.21.15.178:6789/0 conn(0x7fa41c06cac0 legacy=0x7fa41c0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:28.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.039+0000 7fa41aa00640 1 -- 172.21.15.159:0/2584833845 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41c0b4c60 con 0x7fa41c0afd80 2024-04-04T06:17:28.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.039+0000 7fa41aa00640 1 -- 172.21.15.159:0/2584833845 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fa40c001690 con 0x7fa41c0afd80 2024-04-04T06:17:28.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.039+0000 7fa41aa00640 1 -- 172.21.15.159:0/2584833845 >> v1:172.21.15.159:6789/0 conn(0x7fa41c0afd80 legacy=0x7fa41c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:28.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.039+0000 7fa41aa00640 1 --2- 172.21.15.159:0/2584833845 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa41c0ccc40 0x7fa4000039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:28.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.039+0000 7fa41aa00640 1 --2- 172.21.15.159:0/2584833845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa400003f60 0x7fa400004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:28.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.039+0000 7fa41aa00640 1 --2- 172.21.15.159:0/2584833845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa400004d40 0x7fa4000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:28.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.039+0000 7fa41aa00640 1 -- 172.21.15.159:0/2584833845 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c0b5740 con 0x7fa400003f60 2024-04-04T06:17:28.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.039+0000 7fa41aa00640 1 -- 172.21.15.159:0/2584833845 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa40000baa0 con 0x7fa400004d40 2024-04-04T06:17:28.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.039+0000 7fa41aa00640 1 -- 172.21.15.159:0/2584833845 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa40000bca0 con 0x7fa41c0ccc40 2024-04-04T06:17:28.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.040+0000 7fa413e00640 1 --2- 172.21.15.159:0/2584833845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa400003f60 0x7fa400004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:28.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.040+0000 7fa41be00640 1 --2- 172.21.15.159:0/2584833845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa400004d40 0x7fa4000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:28.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.040+0000 7fa41b400640 1 --2- 172.21.15.159:0/2584833845 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa41c0ccc40 0x7fa4000039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:28.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.040+0000 7fa413e00640 1 -- 172.21.15.159:0/2584833845 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa41c0ccc40 msgr2=0x7fa4000039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:28.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.040+0000 7fa413e00640 1 --2- 172.21.15.159:0/2584833845 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa41c0ccc40 0x7fa4000039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:28.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.040+0000 7fa413e00640 1 -- 172.21.15.159:0/2584833845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa400004d40 msgr2=0x7fa4000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:28.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.040+0000 7fa413e00640 1 --2- 172.21.15.159:0/2584833845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa400004d40 0x7fa4000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:28.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.040+0000 7fa413e00640 1 -- 172.21.15.159:0/2584833845 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa400003520 con 0x7fa400003f60 2024-04-04T06:17:28.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.041+0000 7fa413e00640 1 --2- 172.21.15.159:0/2584833845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa400003f60 0x7fa400004800 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fa404009ac0 tx=0x7fa400005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7563fe2799f76e93 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:28.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.041+0000 7fa41aa00640 1 -- 172.21.15.159:0/2584833845 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404039070 con 0x7fa400003f60 2024-04-04T06:17:28.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.042+0000 7fa41aa00640 1 -- 172.21.15.159:0/2584833845 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40402c6b0 con 0x7fa400003f60 2024-04-04T06:17:28.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.042+0000 7fa41aa00640 1 -- 172.21.15.159:0/2584833845 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404009690 con 0x7fa400003f60 2024-04-04T06:17:28.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.042+0000 7fa422000640 1 -- 172.21.15.159:0/2584833845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa400003f60 msgr2=0x7fa400004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:28.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.042+0000 7fa422000640 1 --2- 172.21.15.159:0/2584833845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa400003f60 0x7fa400004800 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fa404009ac0 tx=0x7fa400005830 comp rx=0 tx=0).stop 2024-04-04T06:17:28.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.042+0000 7fa41be00640 1 -- 172.21.15.159:0/2584833845 reap_dead start 2024-04-04T06:17:28.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.042+0000 7fa422000640 1 -- 172.21.15.159:0/2584833845 shutdown_connections 2024-04-04T06:17:28.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.042+0000 7fa422000640 1 -- 172.21.15.159:0/2584833845 >> 172.21.15.159:0/2584833845 conn(0x7fa41c071e20 msgr2=0x7fa41c072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:28.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.042+0000 7fa422000640 1 -- 172.21.15.159:0/2584833845 shutdown_connections 2024-04-04T06:17:28.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.042+0000 7fa422000640 1 -- 172.21.15.159:0/2584833845 wait complete. 2024-04-04T06:17:28.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.044+0000 7fa422000640 1 Processor -- start 2024-04-04T06:17:28.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.044+0000 7fa422000640 1 -- start start 2024-04-04T06:17:28.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.045+0000 7fa422000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa400003f60 0x7fa41c137db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:28.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.045+0000 7fa422000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa41c138440 0x7fa41c1388a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:28.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.045+0000 7fa422000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa41c14d540 0x7fa41c14d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:28.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.045+0000 7fa422000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c0b5220 con 0x7fa400003f60 2024-04-04T06:17:28.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.045+0000 7fa422000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c06cf30 con 0x7fa41c138440 2024-04-04T06:17:28.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.045+0000 7fa422000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa41c0b0160 con 0x7fa41c14d540 2024-04-04T06:17:28.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.045+0000 7fa41b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa400003f60 0x7fa41c137db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:28.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.045+0000 7fa41b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa400003f60 0x7fa41c137db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48384/0 (socket says 172.21.15.159:48384) 2024-04-04T06:17:28.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.045+0000 7fa41b400640 1 -- 172.21.15.159:0/2061998841 learned_addr learned my addr 172.21.15.159:0/2061998841 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:28.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.046+0000 7fa41b400640 1 -- 172.21.15.159:0/2061998841 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa41c14d540 msgr2=0x7fa41c14d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:28.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.046+0000 7fa413e00640 1 --2- 172.21.15.159:0/2061998841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa41c138440 0x7fa41c1388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:28.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.046+0000 7fa41b400640 1 --2- 172.21.15.159:0/2061998841 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa41c14d540 0x7fa41c14d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:28.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.046+0000 7fa41b400640 1 -- 172.21.15.159:0/2061998841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa41c138440 msgr2=0x7fa41c1388a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:28.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.046+0000 7fa41b400640 1 --2- 172.21.15.159:0/2061998841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa41c138440 0x7fa41c1388a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:28.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.046+0000 7fa41b400640 1 -- 172.21.15.159:0/2061998841 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41c14e0e0 con 0x7fa400003f60 2024-04-04T06:17:28.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.046+0000 7fa41b400640 1 --2- 172.21.15.159:0/2061998841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa400003f60 0x7fa41c137db0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00e740 tx=0x7fa40c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:28.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.046+0000 7fa413400640 1 -- 172.21.15.159:0/2061998841 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c00ee50 con 0x7fa400003f60 2024-04-04T06:17:28.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.046+0000 7fa413400640 1 -- 172.21.15.159:0/2061998841 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40c018070 con 0x7fa400003f60 2024-04-04T06:17:28.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.046+0000 7fa422000640 1 -- 172.21.15.159:0/2061998841 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41c14e3a0 con 0x7fa400003f60 2024-04-04T06:17:28.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.046+0000 7fa413400640 1 -- 172.21.15.159:0/2061998841 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40c00cb10 con 0x7fa400003f60 2024-04-04T06:17:28.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.047+0000 7fa422000640 1 -- 172.21.15.159:0/2061998841 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa41c14e960 con 0x7fa400003f60 2024-04-04T06:17:28.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.048+0000 7fa422000640 1 -- 172.21.15.159:0/2061998841 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa41c160d20 con 0x7fa400003f60 2024-04-04T06:17:28.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.049+0000 7fa413400640 1 -- 172.21.15.159:0/2061998841 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa40c00ccb0 con 0x7fa400003f60 2024-04-04T06:17:28.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.050+0000 7fa413400640 1 --2- 172.21.15.159:0/2061998841 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa3f806c920 0x7fa3f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:28.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.050+0000 7fa413400640 1 -- 172.21.15.159:0/2061998841 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5795+0+0 (secure 0 0 0) 0x7fa40c08b6f0 con 0x7fa400003f60 2024-04-04T06:17:28.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.051+0000 7fa413e00640 1 --2- 172.21.15.159:0/2061998841 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa3f806c920 0x7fa3f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:28.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.052+0000 7fa413e00640 1 --2- 172.21.15.159:0/2061998841 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa3f806c920 0x7fa3f806ede0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa404036000 tx=0x7fa404030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:28.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.055+0000 7fa413400640 1 -- 172.21.15.159:0/2061998841 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa40c059ac0 con 0x7fa400003f60 2024-04-04T06:17:28.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.293+0000 7fa422000640 1 -- 172.21.15.159:0/2061998841 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa41c156550 con 0x7fa400003f60 2024-04-04T06:17:28.289 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.294+0000 7fa413400640 1 -- 172.21.15.159:0/2061998841 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v107) ==== 53+0+73 (secure 0 0 0) 0x7fa40c0593c0 con 0x7fa400003f60 2024-04-04T06:17:28.289 INFO:teuthology.orchestra.run.smithi159.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-04T06:17:28.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.297+0000 7fa422000640 1 -- 172.21.15.159:0/2061998841 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa3f806c920 msgr2=0x7fa3f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:28.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.297+0000 7fa422000640 1 --2- 172.21.15.159:0/2061998841 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa3f806c920 0x7fa3f806ede0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa404036000 tx=0x7fa404030000 comp rx=0 tx=0).stop 2024-04-04T06:17:28.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.297+0000 7fa422000640 1 -- 172.21.15.159:0/2061998841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa400003f60 msgr2=0x7fa41c137db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:28.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.297+0000 7fa422000640 1 --2- 172.21.15.159:0/2061998841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa400003f60 0x7fa41c137db0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00e740 tx=0x7fa40c00ec00 comp rx=0 tx=0).stop 2024-04-04T06:17:28.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.297+0000 7fa422000640 1 -- 172.21.15.159:0/2061998841 shutdown_connections 2024-04-04T06:17:28.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.297+0000 7fa422000640 1 --2- 172.21.15.159:0/2061998841 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa41c14d540 0x7fa41c14d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:28.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.297+0000 7fa422000640 1 --2- 172.21.15.159:0/2061998841 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa3f806c920 0x7fa3f806ede0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:28.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.297+0000 7fa422000640 1 --2- 172.21.15.159:0/2061998841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa41c138440 0x7fa41c1388a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:28.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.297+0000 7fa422000640 1 --2- 172.21.15.159:0/2061998841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa400003f60 0x7fa41c137db0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:28.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.297+0000 7fa422000640 1 -- 172.21.15.159:0/2061998841 >> 172.21.15.159:0/2061998841 conn(0x7fa41c071e20 msgr2=0x7fa41c15bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:28.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.297+0000 7fa422000640 1 -- 172.21.15.159:0/2061998841 shutdown_connections 2024-04-04T06:17:28.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28.297+0000 7fa422000640 1 -- 172.21.15.159:0/2061998841 wait complete. 2024-04-04T06:17:28.311 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-04T06:17:28.311 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-04T06:17:28.311 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi159.front.sepia.ceph.com 2024-04-04T06:17:28.312 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-04T06:17:28.312 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-04T06:17:28.312 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-04T06:17:28.312 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-04T06:17:28.312 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-04T06:17:28.312 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:17:28.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:28.312 DEBUG:teuthology.orchestra.run.smithi159:> ip addr 2024-04-04T06:17:28.358 INFO:teuthology.orchestra.run.smithi159.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:17:28.358 INFO:teuthology.orchestra.run.smithi159.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:17:28.358 INFO:teuthology.orchestra.run.smithi159.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:17:28.359 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:28.359 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 ::1/128 scope host 2024-04-04T06:17:28.359 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:28.359 INFO:teuthology.orchestra.run.smithi159.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:17:28.359 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c6 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:17:28.359 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f0 2024-04-04T06:17:28.359 INFO:teuthology.orchestra.run.smithi159.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:17:28.359 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c7 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:17:28.359 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f1 2024-04-04T06:17:28.359 INFO:teuthology.orchestra.run.smithi159.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:17:28.359 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4e brd ff:ff:ff:ff:ff:ff 2024-04-04T06:17:28.359 INFO:teuthology.orchestra.run.smithi159.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-04T06:17:28.360 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:17:28.360 INFO:teuthology.orchestra.run.smithi159.stdout: inet 172.21.15.159/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-04T06:17:28.360 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft 41885sec preferred_lft 41885sec 2024-04-04T06:17:28.360 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::ec4:7aff:fe88:724f/64 scope link 2024-04-04T06:17:28.360 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:28.360 INFO:teuthology.orchestra.run.smithi159.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-04T06:17:28.360 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:17:28.360 INFO:teuthology.orchestra.run.smithi159.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-04T06:17:28.360 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:28.360 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::989e:e4ff:fea5:7568/64 scope link 2024-04-04T06:17:28.360 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:28.360 INFO:teuthology.orchestra.run.smithi159.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:17:28.360 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether a2:e3:e0:cb:0a:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-04T06:17:28.360 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::a0e3:e0ff:fecb:a73/64 scope link 2024-04-04T06:17:28.361 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:28.361 INFO:teuthology.orchestra.run.smithi159.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:17:28.361 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-04T06:17:28.361 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::c0a:8aff:fe54:b58f/64 scope link 2024-04-04T06:17:28.361 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:28.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:28.361 DEBUG:teuthology.orchestra.run.smithi159:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-04T06:17:28.490 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:17:28.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:28.493 DEBUG:teuthology.orchestra.run.smithi159:> route 2024-04-04T06:17:28.540 INFO:teuthology.orchestra.run.smithi159.stdout:Kernel IP routing table 2024-04-04T06:17:28.540 INFO:teuthology.orchestra.run.smithi159.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-04T06:17:28.541 INFO:teuthology.orchestra.run.smithi159.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-04T06:17:28.541 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-04T06:17:28.541 INFO:teuthology.orchestra.run.smithi159.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-04T06:17:28.541 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:17:28.541 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:17:28.541 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-04T06:17:28.541 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-04T06:17:28.541 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-04T06:17:28.541 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:17:28.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:17:28.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:28.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:28.832 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-04-04T06:17:28.882 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-04T06:17:28.882 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-04T06:17:28.883 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:17:28.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:28.884 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set brx.0 up 2024-04-04T06:17:28.962 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:17:28.962 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-04T06:17:29.009 INFO:teuthology.orchestra.run.smithi159.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:17:29.009 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:17:29.010 DEBUG:teuthology.orchestra.run.smithi159:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:17:29.102 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-04T06:17:29.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:29.103 DEBUG:teuthology.orchestra.run.smithi159:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:17:29.239 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:17:29.239 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:17:29.239 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:17:29.259 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:17:29.259 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:17:29.259 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:17:29.259 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:17:29.259 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:17:29.284 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:17:29.284 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:17:29.285 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:17:29.285 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:17:29.285 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:17:29.285 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-04T06:17:29.285 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-04T06:17:29.285 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30311045-f714-48fa-832b-4cb2ba8bf0bf" will pass to kernel 2024-04-04T06:17:29.288 INFO:teuthology.orchestra.run.smithi159.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-04T06:17:29.288 INFO:teuthology.orchestra.run.smithi159.stdout: You just mounted a file system that supports labels which does not 2024-04-04T06:17:29.288 INFO:teuthology.orchestra.run.smithi159.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-04T06:17:29.288 INFO:teuthology.orchestra.run.smithi159.stdout: applications will generate AVC messages and not be allowed access to 2024-04-04T06:17:29.288 INFO:teuthology.orchestra.run.smithi159.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-04T06:17:29.292 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-04T06:17:29.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:29.292 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:17:29.371 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:17:29.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:17:29.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:29.480 INFO:teuthology.orchestra.run.smithi159.stdout:client9192 2024-04-04T06:17:29.481 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:17:29.481 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:17:29.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:29.530 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-04T06:17:29.629 INFO:teuthology.orchestra.run.smithi159.stdout:./ 2024-04-04T06:17:29.629 INFO:teuthology.orchestra.run.smithi159.stdout:./parent 2024-04-04T06:17:29.629 INFO:teuthology.orchestra.run.smithi159.stdout:./parent/flushed 2024-04-04T06:17:29.629 INFO:teuthology.orchestra.run.smithi159.stdout:./parent/flushed/charlie 2024-04-04T06:17:29.629 INFO:teuthology.orchestra.run.smithi159.stdout:./parent/flushed/alpha 2024-04-04T06:17:29.629 INFO:teuthology.orchestra.run.smithi159.stdout:./parent/unflushed 2024-04-04T06:17:29.629 INFO:teuthology.orchestra.run.smithi159.stdout:./parent/unflushed/jfile 2024-04-04T06:17:29.630 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:29.631 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:29.631 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:29.631 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:29.631 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-04T06:17:29.631 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:29.755 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:17:29.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:29.769 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:29.769 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:29.769 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:29.769 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:29.769 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-04T06:17:29.769 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:29.941 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627776 2024-04-04T06:17:29.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:29.953 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:29.953 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:29.953 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:29.953 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:29.953 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-04T06:17:29.953 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:30.118 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627777 2024-04-04T06:17:30.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:30.131 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:30.132 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:30.132 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:30.132 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:30.132 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-04T06:17:30.132 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:30.301 INFO:teuthology.orchestra.run.smithi159.stdout:1099511628280 2024-04-04T06:17:30.305 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:30.314 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:30.314 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:30.314 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:30.314 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:30.314 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-04T06:17:30.315 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:30.488 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627778 2024-04-04T06:17:30.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:30.503 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:30.503 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:30.503 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:30.503 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:30.504 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-04T06:17:30.504 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:30.677 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627779 2024-04-04T06:17:30.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:30.691 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:30.691 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:30.691 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:30.691 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:30.692 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-04T06:17:30.692 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:30.868 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627780 2024-04-04T06:17:30.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:30.882 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:17:30.882 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:17:30.975 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:17:30.975 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:17:31.018 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:17:31.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.278+0000 7fc16b200640 1 Processor -- start 2024-04-04T06:17:31.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.278+0000 7fc16b200640 1 -- start start 2024-04-04T06:17:31.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.280+0000 7fc16b200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc16c0d2310 0x7fc16c0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:31.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.280+0000 7fc16b200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc16c0cf530 0x7fc16c0b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:31.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.280+0000 7fc16b200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc16c15e880 con 0x7fc16c15e4a0 2024-04-04T06:17:31.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.280+0000 7fc16b200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fc16c0cf9a0 con 0x7fc16c0d2310 2024-04-04T06:17:31.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.280+0000 7fc16b200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fc16c0d34e0 con 0x7fc16c0cf530 2024-04-04T06:17:31.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.280+0000 7fc161400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc16c0d2310 0x7fc16c0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:31.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.280+0000 7fc169e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fc16c15e4a0 0x7fc16c0b9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:50980/0 (socket says 172.21.15.159:50980) 2024-04-04T06:17:31.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.280+0000 7fc169e00640 1 -- 172.21.15.159:0/2722959946 learned_addr learned my addr 172.21.15.159:0/2722959946 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:31.276 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.281+0000 7fc16a800640 1 --2- 172.21.15.159:0/2722959946 >> v2:172.21.15.178:3300/0 conn(0x7fc16c0cf530 0x7fc16c0b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:31.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.281+0000 7fc161400640 1 -- 172.21.15.159:0/2722959946 >> v2:172.21.15.178:3300/0 conn(0x7fc16c0cf530 msgr2=0x7fc16c0b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:31.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.281+0000 7fc161400640 1 --2- 172.21.15.159:0/2722959946 >> v2:172.21.15.178:3300/0 conn(0x7fc16c0cf530 0x7fc16c0b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:31.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.281+0000 7fc161400640 1 -- 172.21.15.159:0/2722959946 >> v1:172.21.15.159:6789/0 conn(0x7fc16c15e4a0 legacy=0x7fc16c0b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:31.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.281+0000 7fc161400640 1 -- 172.21.15.159:0/2722959946 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16c0d28d0 con 0x7fc16c0d2310 2024-04-04T06:17:31.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.281+0000 7fc169400640 1 -- 172.21.15.159:0/2722959946 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 659737832 0 0) 0x7fc16c15e880 con 0x7fc16c15e4a0 2024-04-04T06:17:31.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.282+0000 7fc161400640 1 --2- 172.21.15.159:0/2722959946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc16c0d2310 0x7fc16c0ceff0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0d3660 tx=0x7fc15402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d31c01ce5e96b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:31.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.282+0000 7fc169400640 1 -- 172.21.15.159:0/2722959946 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1540315a0 con 0x7fc16c0d2310 2024-04-04T06:17:31.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.282+0000 7fc169400640 1 -- 172.21.15.159:0/2722959946 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc154037070 con 0x7fc16c0d2310 2024-04-04T06:17:31.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.282+0000 7fc169400640 1 -- 172.21.15.159:0/2722959946 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154031960 con 0x7fc16c0d2310 2024-04-04T06:17:31.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.283+0000 7fc16b200640 1 -- 172.21.15.159:0/2722959946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc16c0d2310 msgr2=0x7fc16c0ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:31.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.283+0000 7fc16b200640 1 --2- 172.21.15.159:0/2722959946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc16c0d2310 0x7fc16c0ceff0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0d3660 tx=0x7fc15402b840 comp rx=0 tx=0).stop 2024-04-04T06:17:31.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.283+0000 7fc16b200640 1 -- 172.21.15.159:0/2722959946 shutdown_connections 2024-04-04T06:17:31.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.283+0000 7fc16b200640 1 --2- 172.21.15.159:0/2722959946 >> v2:172.21.15.178:3300/0 conn(0x7fc16c0cf530 0x7fc16c0b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:31.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.283+0000 7fc16b200640 1 --2- 172.21.15.159:0/2722959946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc16c0d2310 0x7fc16c0ceff0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:31.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.283+0000 7fc16b200640 1 -- 172.21.15.159:0/2722959946 >> 172.21.15.159:0/2722959946 conn(0x7fc16c0676d0 msgr2=0x7fc16c067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:31.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.283+0000 7fc16b200640 1 -- 172.21.15.159:0/2722959946 shutdown_connections 2024-04-04T06:17:31.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.283+0000 7fc16b200640 1 -- 172.21.15.159:0/2722959946 wait complete. 2024-04-04T06:17:31.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.285+0000 7fc16b200640 1 Processor -- start 2024-04-04T06:17:31.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.285+0000 7fc16b200640 1 -- start start 2024-04-04T06:17:31.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.286+0000 7fc16b200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc16c0cf530 0x7fc16c0c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:31.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.286+0000 7fc16b200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc16c0d2310 0x7fc16c0d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:31.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.286+0000 7fc16b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc16c15e4a0 0x7fc16c0d4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:31.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.286+0000 7fc16b200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c0d3170 con 0x7fc16c15e4a0 2024-04-04T06:17:31.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.286+0000 7fc16b200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c0d2e90 con 0x7fc16c0cf530 2024-04-04T06:17:31.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.286+0000 7fc16b200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc16c0d3970 con 0x7fc16c0d2310 2024-04-04T06:17:31.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.286+0000 7fc16a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc16c15e4a0 0x7fc16c0d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:31.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.286+0000 7fc16a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc16c15e4a0 0x7fc16c0d4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48404/0 (socket says 172.21.15.159:48404) 2024-04-04T06:17:31.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.286+0000 7fc16a800640 1 -- 172.21.15.159:0/1595635699 learned_addr learned my addr 172.21.15.159:0/1595635699 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:31.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.286+0000 7fc169e00640 1 --2- 172.21.15.159:0/1595635699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc16c0cf530 0x7fc16c0c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:31.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.286+0000 7fc16a800640 1 -- 172.21.15.159:0/1595635699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc16c0d2310 msgr2=0x7fc16c0d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:31.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.287+0000 7fc161400640 1 --2- 172.21.15.159:0/1595635699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc16c0d2310 0x7fc16c0d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:31.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.287+0000 7fc16a800640 1 --2- 172.21.15.159:0/1595635699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc16c0d2310 0x7fc16c0d3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:31.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.287+0000 7fc16a800640 1 -- 172.21.15.159:0/1595635699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc16c0cf530 msgr2=0x7fc16c0c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:31.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.287+0000 7fc16a800640 1 --2- 172.21.15.159:0/1595635699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc16c0cf530 0x7fc16c0c70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:31.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.287+0000 7fc16a800640 1 -- 172.21.15.159:0/1595635699 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16c0c9c70 con 0x7fc16c15e4a0 2024-04-04T06:17:31.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.287+0000 7fc169e00640 1 --2- 172.21.15.159:0/1595635699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc16c0cf530 0x7fc16c0c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:31.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.287+0000 7fc161400640 1 --2- 172.21.15.159:0/1595635699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc16c0d2310 0x7fc16c0d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:31.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.287+0000 7fc16a800640 1 --2- 172.21.15.159:0/1595635699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc16c15e4a0 0x7fc16c0d4150 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fc16400d800 tx=0x7fc16400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:31.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.287+0000 7fc162a00640 1 -- 172.21.15.159:0/1595635699 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16400b6c0 con 0x7fc16c15e4a0 2024-04-04T06:17:31.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.287+0000 7fc162a00640 1 -- 172.21.15.159:0/1595635699 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc164019070 con 0x7fc16c15e4a0 2024-04-04T06:17:31.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.287+0000 7fc162a00640 1 -- 172.21.15.159:0/1595635699 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16400bac0 con 0x7fc16c15e4a0 2024-04-04T06:17:31.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.287+0000 7fc16b200640 1 -- 172.21.15.159:0/1595635699 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc16c0c9f50 con 0x7fc16c15e4a0 2024-04-04T06:17:31.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.287+0000 7fc16b200640 1 -- 172.21.15.159:0/1595635699 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc16c0ca510 con 0x7fc16c15e4a0 2024-04-04T06:17:31.284 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.289+0000 7fc16b200640 1 -- 172.21.15.159:0/1595635699 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc16c075940 con 0x7fc16c15e4a0 2024-04-04T06:17:31.285 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.290+0000 7fc162a00640 1 -- 172.21.15.159:0/1595635699 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc16400bc60 con 0x7fc16c15e4a0 2024-04-04T06:17:31.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.291+0000 7fc162a00640 1 --2- 172.21.15.159:0/1595635699 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc14806c8f0 0x7fc14806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:31.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.291+0000 7fc162a00640 1 -- 172.21.15.159:0/1595635699 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5795+0+0 (secure 0 0 0) 0x7fc16408bff0 con 0x7fc16c15e4a0 2024-04-04T06:17:31.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.292+0000 7fc169e00640 1 --2- 172.21.15.159:0/1595635699 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc14806c8f0 0x7fc14806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:31.288 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.293+0000 7fc169e00640 1 --2- 172.21.15.159:0/1595635699 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc14806c8f0 0x7fc14806edb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc15c007fd0 tx=0x7fc15c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:31.293 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.298+0000 7fc162a00640 1 -- 172.21.15.159:0/1595635699 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc16405a3f0 con 0x7fc16c15e4a0 2024-04-04T06:17:31.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.499+0000 7fc16b200640 1 -- 172.21.15.159:0/1595635699 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc16c0d4c70 con 0x7fc16c15e4a0 2024-04-04T06:17:31.496 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.501+0000 7fc162a00640 1 -- 172.21.15.159:0/1595635699 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+16770 (secure 0 0 0) 0x7fc164059cf0 con 0x7fc16c15e4a0 2024-04-04T06:17:31.497 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:31.497 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":114,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:17:21.596799+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:16:55.036975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-04T06:16:56.239022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6834/1017894858":"2024-04-05T06:17:21.596764+0000","172.21.15.159:6832/1017894858":"2024-04-05T06:17:21.596764+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:17:31.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.504+0000 7fc16b200640 1 -- 172.21.15.159:0/1595635699 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc14806c8f0 msgr2=0x7fc14806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:31.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.504+0000 7fc16b200640 1 --2- 172.21.15.159:0/1595635699 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc14806c8f0 0x7fc14806edb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc15c007fd0 tx=0x7fc15c009040 comp rx=0 tx=0).stop 2024-04-04T06:17:31.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.504+0000 7fc16b200640 1 -- 172.21.15.159:0/1595635699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc16c15e4a0 msgr2=0x7fc16c0d4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:31.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.504+0000 7fc16b200640 1 --2- 172.21.15.159:0/1595635699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc16c15e4a0 0x7fc16c0d4150 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fc16400d800 tx=0x7fc16400dcc0 comp rx=0 tx=0).stop 2024-04-04T06:17:31.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.504+0000 7fc16b200640 1 -- 172.21.15.159:0/1595635699 shutdown_connections 2024-04-04T06:17:31.499 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.504+0000 7fc16b200640 1 --2- 172.21.15.159:0/1595635699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc16c15e4a0 0x7fc16c0d4150 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:31.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.504+0000 7fc16b200640 1 --2- 172.21.15.159:0/1595635699 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc16c0d2310 0x7fc16c0d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:31.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.504+0000 7fc16b200640 1 --2- 172.21.15.159:0/1595635699 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc14806c8f0 0x7fc14806edb0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:31.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.504+0000 7fc16b200640 1 --2- 172.21.15.159:0/1595635699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc16c0cf530 0x7fc16c0c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:31.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.504+0000 7fc16b200640 1 -- 172.21.15.159:0/1595635699 >> 172.21.15.159:0/1595635699 conn(0x7fc16c0676d0 msgr2=0x7fc16c15c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:31.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.504+0000 7fc16b200640 1 -- 172.21.15.159:0/1595635699 shutdown_connections 2024-04-04T06:17:31.500 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.504+0000 7fc16b200640 1 -- 172.21.15.159:0/1595635699 wait complete. 2024-04-04T06:17:31.521 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:16:55.036975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:16:56.239022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:17:31.521 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:17:31.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.736+0000 7ff495800640 1 Processor -- start 2024-04-04T06:17:31.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.736+0000 7ff495800640 1 -- start start 2024-04-04T06:17:31.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.737+0000 7ff495800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff49015ca70 0x7ff4900cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:31.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.737+0000 7ff495800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4900c7b40 con 0x7ff4900d0370 2024-04-04T06:17:31.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.737+0000 7ff495800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4900c7dd0 con 0x7ff49015d630 2024-04-04T06:17:31.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.737+0000 7ff495800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7ff4900c80b0 con 0x7ff49015ca70 2024-04-04T06:17:31.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.738+0000 7ff48f400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7ff4900d0370 0x7ff4900ce8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:50986/0 (socket says 172.21.15.159:50986) 2024-04-04T06:17:31.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.738+0000 7ff48f400640 1 -- 172.21.15.159:0/2758884268 learned_addr learned my addr 172.21.15.159:0/2758884268 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:31.733 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.738+0000 7ff48e000640 1 --2- 172.21.15.159:0/2758884268 >> v2:172.21.15.159:3300/0 conn(0x7ff49015ca70 0x7ff4900cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:31.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.738+0000 7ff48d600640 1 -- 172.21.15.159:0/2758884268 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3791389892 0 0) 0x7ff4900c7b40 con 0x7ff4900d0370 2024-04-04T06:17:31.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.738+0000 7ff48d600640 1 -- 172.21.15.159:0/2758884268 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff474003520 con 0x7ff4900d0370 2024-04-04T06:17:31.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.738+0000 7ff48e000640 1 -- 172.21.15.159:0/2758884268 >> v1:172.21.15.178:6789/0 conn(0x7ff49015d630 legacy=0x7ff4900c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:31.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.738+0000 7ff48e000640 1 -- 172.21.15.159:0/2758884268 >> v1:172.21.15.159:6789/0 conn(0x7ff4900d0370 legacy=0x7ff4900ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:31.734 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.738+0000 7ff48e000640 1 -- 172.21.15.159:0/2758884268 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4900c8390 con 0x7ff49015ca70 2024-04-04T06:17:31.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.740+0000 7ff48e000640 1 --2- 172.21.15.159:0/2758884268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff49015ca70 0x7ff4900cfe30 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0097a0 tx=0x7ff47c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8d76901a9304d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:31.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.740+0000 7ff48d600640 1 -- 172.21.15.159:0/2758884268 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c035020 con 0x7ff49015ca70 2024-04-04T06:17:31.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.740+0000 7ff48d600640 1 -- 172.21.15.159:0/2758884268 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff47c031830 con 0x7ff49015ca70 2024-04-04T06:17:31.735 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.740+0000 7ff48d600640 1 -- 172.21.15.159:0/2758884268 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c031be0 con 0x7ff49015ca70 2024-04-04T06:17:31.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.741+0000 7ff495800640 1 -- 172.21.15.159:0/2758884268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff49015ca70 msgr2=0x7ff4900cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:31.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.741+0000 7ff495800640 1 --2- 172.21.15.159:0/2758884268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff49015ca70 0x7ff4900cfe30 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0097a0 tx=0x7ff47c02b8d0 comp rx=0 tx=0).stop 2024-04-04T06:17:31.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.741+0000 7ff495800640 1 -- 172.21.15.159:0/2758884268 shutdown_connections 2024-04-04T06:17:31.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.741+0000 7ff495800640 1 --2- 172.21.15.159:0/2758884268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff49015ca70 0x7ff4900cfe30 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:31.736 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.741+0000 7ff495800640 1 -- 172.21.15.159:0/2758884268 >> 172.21.15.159:0/2758884268 conn(0x7ff4900b3e90 msgr2=0x7ff4900b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:31.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.741+0000 7ff495800640 1 -- 172.21.15.159:0/2758884268 shutdown_connections 2024-04-04T06:17:31.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.741+0000 7ff495800640 1 -- 172.21.15.159:0/2758884268 wait complete. 2024-04-04T06:17:31.738 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.744+0000 7ff495800640 1 Processor -- start 2024-04-04T06:17:31.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.744+0000 7ff495800640 1 -- start start 2024-04-04T06:17:31.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.746+0000 7ff495800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4900d0370 0x7ff4901f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:31.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.746+0000 7ff495800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff49015ca70 0x7ff4901531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:31.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.746+0000 7ff495800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff49015d630 0x7ff4900b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:31.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.746+0000 7ff495800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff49015f0c0 con 0x7ff49015ca70 2024-04-04T06:17:31.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.746+0000 7ff495800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff49015ef40 con 0x7ff49015d630 2024-04-04T06:17:31.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.746+0000 7ff495800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff49015f240 con 0x7ff4900d0370 2024-04-04T06:17:31.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.746+0000 7ff48e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff49015ca70 0x7ff4901531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:31.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.746+0000 7ff48e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff49015ca70 0x7ff4901531d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48414/0 (socket says 172.21.15.159:48414) 2024-04-04T06:17:31.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.746+0000 7ff48e000640 1 -- 172.21.15.159:0/2585469343 learned_addr learned my addr 172.21.15.159:0/2585469343 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:31.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.746+0000 7ff48ea00640 1 --2- 172.21.15.159:0/2585469343 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4900d0370 0x7ff4901f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:31.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.747+0000 7ff48f400640 1 --2- 172.21.15.159:0/2585469343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff49015d630 0x7ff4900b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:31.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.747+0000 7ff48e000640 1 -- 172.21.15.159:0/2585469343 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4900d0370 msgr2=0x7ff4901f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:31.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.747+0000 7ff48e000640 1 --2- 172.21.15.159:0/2585469343 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4900d0370 0x7ff4901f43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:31.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.747+0000 7ff48e000640 1 -- 172.21.15.159:0/2585469343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff49015d630 msgr2=0x7ff4900b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:31.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.747+0000 7ff48e000640 1 --2- 172.21.15.159:0/2585469343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff49015d630 0x7ff4900b2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:31.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.747+0000 7ff48e000640 1 -- 172.21.15.159:0/2585469343 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff490153710 con 0x7ff49015ca70 2024-04-04T06:17:31.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.747+0000 7ff48f400640 1 --2- 172.21.15.159:0/2585469343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff49015d630 0x7ff4900b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:31.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.747+0000 7ff48e000640 1 --2- 172.21.15.159:0/2585469343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff49015ca70 0x7ff4901531d0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0097a0 tx=0x7ff47c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:31.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.747+0000 7ff486a00640 1 -- 172.21.15.159:0/2585469343 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c035070 con 0x7ff49015ca70 2024-04-04T06:17:31.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.747+0000 7ff486a00640 1 -- 172.21.15.159:0/2585469343 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff47c004230 con 0x7ff49015ca70 2024-04-04T06:17:31.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.747+0000 7ff495800640 1 -- 172.21.15.159:0/2585469343 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4900b6840 con 0x7ff49015ca70 2024-04-04T06:17:31.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.747+0000 7ff486a00640 1 -- 172.21.15.159:0/2585469343 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c03a630 con 0x7ff49015ca70 2024-04-04T06:17:31.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.748+0000 7ff495800640 1 -- 172.21.15.159:0/2585469343 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff490153fe0 con 0x7ff49015ca70 2024-04-04T06:17:31.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.749+0000 7ff495800640 1 -- 172.21.15.159:0/2585469343 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff490153af0 con 0x7ff49015ca70 2024-04-04T06:17:31.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.750+0000 7ff486a00640 1 -- 172.21.15.159:0/2585469343 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff47c038020 con 0x7ff49015ca70 2024-04-04T06:17:31.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.751+0000 7ff486a00640 1 --2- 172.21.15.159:0/2585469343 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff47006c920 0x7ff47006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:31.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.752+0000 7ff486a00640 1 -- 172.21.15.159:0/2585469343 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5795+0+0 (secure 0 0 0) 0x7ff47c0ada20 con 0x7ff49015ca70 2024-04-04T06:17:31.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.752+0000 7ff48ea00640 1 --2- 172.21.15.159:0/2585469343 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff47006c920 0x7ff47006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:31.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.753+0000 7ff48ea00640 1 --2- 172.21.15.159:0/2585469343 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff47006c920 0x7ff47006ede0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff480007fd0 tx=0x7ff480009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:31.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.754+0000 7ff486a00640 1 -- 172.21.15.159:0/2585469343 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff47c040070 con 0x7ff49015ca70 2024-04-04T06:17:31.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.980+0000 7ff495800640 1 -- 172.21.15.159:0/2585469343 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff490068cd0 con 0x7ff49015ca70 2024-04-04T06:17:31.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.980+0000 7ff486a00640 1 -- 172.21.15.159:0/2585469343 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) ==== 93+0+10 (secure 0 0 0) 0x7ff47c07be60 con 0x7ff49015ca70 2024-04-04T06:17:31.976 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:17:31.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.983+0000 7ff495800640 1 -- 172.21.15.159:0/2585469343 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff47006c920 msgr2=0x7ff47006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:31.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.983+0000 7ff495800640 1 --2- 172.21.15.159:0/2585469343 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff47006c920 0x7ff47006ede0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff480007fd0 tx=0x7ff480009040 comp rx=0 tx=0).stop 2024-04-04T06:17:31.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.983+0000 7ff495800640 1 -- 172.21.15.159:0/2585469343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff49015ca70 msgr2=0x7ff4901531d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:31.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.983+0000 7ff495800640 1 --2- 172.21.15.159:0/2585469343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff49015ca70 0x7ff4901531d0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0097a0 tx=0x7ff47c0046c0 comp rx=0 tx=0).stop 2024-04-04T06:17:31.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.983+0000 7ff495800640 1 -- 172.21.15.159:0/2585469343 shutdown_connections 2024-04-04T06:17:31.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.983+0000 7ff495800640 1 --2- 172.21.15.159:0/2585469343 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff47006c920 0x7ff47006ede0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:31.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.983+0000 7ff495800640 1 --2- 172.21.15.159:0/2585469343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff49015d630 0x7ff4900b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:31.978 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.983+0000 7ff495800640 1 --2- 172.21.15.159:0/2585469343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff49015ca70 0x7ff4901531d0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:31.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.983+0000 7ff495800640 1 --2- 172.21.15.159:0/2585469343 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4900d0370 0x7ff4901f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:31.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.983+0000 7ff495800640 1 -- 172.21.15.159:0/2585469343 >> 172.21.15.159:0/2585469343 conn(0x7ff4900b3e90 msgr2=0x7ff49014f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:31.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.983+0000 7ff495800640 1 -- 172.21.15.159:0/2585469343 shutdown_connections 2024-04-04T06:17:31.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:31.983+0000 7ff495800640 1 -- 172.21.15.159:0/2585469343 wait complete. 2024-04-04T06:17:31.999 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:17:32.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.205+0000 7fbd97000640 1 Processor -- start 2024-04-04T06:17:32.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.205+0000 7fbd97000640 1 -- start start 2024-04-04T06:17:32.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.207+0000 7fbd97000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd900d3e40 0x7fbd900b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:32.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.207+0000 7fbd97000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbd900cef20 0x7fbd900cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:32.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.207+0000 7fbd97000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd900d2da0 con 0x7fbd9015e460 2024-04-04T06:17:32.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.207+0000 7fbd97000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fbd900b8320 con 0x7fbd900cef20 2024-04-04T06:17:32.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.207+0000 7fbd97000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbd900d3080 con 0x7fbd900d3e40 2024-04-04T06:17:32.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.207+0000 7fbd95c00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fbd9015e460 0x7fbd900d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:50990/0 (socket says 172.21.15.159:50990) 2024-04-04T06:17:32.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.207+0000 7fbd95c00640 1 -- 172.21.15.159:0/913463736 learned_addr learned my addr 172.21.15.159:0/913463736 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:32.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.207+0000 7fbd95200640 1 --2- 172.21.15.159:0/913463736 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd900d3e40 0x7fbd900b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:32.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.207+0000 7fbd96600640 1 --2- 172.21.15.159:0/913463736 >> v2:172.21.15.178:3300/0 conn(0x7fbd900cef20 0x7fbd900cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:32.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.207+0000 7fbd8fe00640 1 -- 172.21.15.159:0/913463736 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 229716449 0 0) 0x7fbd900d2da0 con 0x7fbd9015e460 2024-04-04T06:17:32.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.208+0000 7fbd8fe00640 1 -- 172.21.15.159:0/913463736 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd78003520 con 0x7fbd9015e460 2024-04-04T06:17:32.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.208+0000 7fbd8fe00640 1 -- 172.21.15.159:0/913463736 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2968206235 0 0) 0x7fbd78003520 con 0x7fbd9015e460 2024-04-04T06:17:32.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.208+0000 7fbd8fe00640 1 -- 172.21.15.159:0/913463736 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd900d3e40 msgr2=0x7fbd900b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:32.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.208+0000 7fbd8fe00640 1 --2- 172.21.15.159:0/913463736 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd900d3e40 0x7fbd900b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:32.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.208+0000 7fbd8fe00640 1 -- 172.21.15.159:0/913463736 >> v2:172.21.15.178:3300/0 conn(0x7fbd900cef20 msgr2=0x7fbd900cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:32.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.208+0000 7fbd8fe00640 1 --2- 172.21.15.159:0/913463736 >> v2:172.21.15.178:3300/0 conn(0x7fbd900cef20 0x7fbd900cf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:32.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.208+0000 7fbd8fe00640 1 -- 172.21.15.159:0/913463736 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd900d3360 con 0x7fbd9015e460 2024-04-04T06:17:32.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.208+0000 7fbd8fe00640 1 -- 172.21.15.159:0/913463736 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fbd84001670 con 0x7fbd9015e460 2024-04-04T06:17:32.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.208+0000 7fbd8fe00640 1 -- 172.21.15.159:0/913463736 >> v1:172.21.15.159:6789/0 conn(0x7fbd9015e460 legacy=0x7fbd900d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:32.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.208+0000 7fbd8fe00640 1 --2- 172.21.15.159:0/913463736 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd900d3e40 0x7fbd780039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:32.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.208+0000 7fbd8fe00640 1 --2- 172.21.15.159:0/913463736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd78003f30 0x7fbd78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:32.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.208+0000 7fbd8fe00640 1 --2- 172.21.15.159:0/913463736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd78004d40 0x7fbd780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:32.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.208+0000 7fbd8fe00640 1 -- 172.21.15.159:0/913463736 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbd900b8320 con 0x7fbd78003f30 2024-04-04T06:17:32.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.208+0000 7fbd8fe00640 1 -- 172.21.15.159:0/913463736 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbd900d3080 con 0x7fbd78004d40 2024-04-04T06:17:32.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.208+0000 7fbd8fe00640 1 -- 172.21.15.159:0/913463736 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbd7800ba10 con 0x7fbd900d3e40 2024-04-04T06:17:32.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.208+0000 7fbd95200640 1 --2- 172.21.15.159:0/913463736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd78003f30 0x7fbd78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:32.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.209+0000 7fbd95c00640 1 --2- 172.21.15.159:0/913463736 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd900d3e40 0x7fbd780039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:32.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.209+0000 7fbd96600640 1 --2- 172.21.15.159:0/913463736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd78004d40 0x7fbd780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:32.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.209+0000 7fbd95200640 1 -- 172.21.15.159:0/913463736 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd900d3e40 msgr2=0x7fbd780039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:32.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.209+0000 7fbd95200640 1 --2- 172.21.15.159:0/913463736 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd900d3e40 0x7fbd780039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:32.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.209+0000 7fbd95200640 1 -- 172.21.15.159:0/913463736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd78004d40 msgr2=0x7fbd780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:32.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.209+0000 7fbd95200640 1 --2- 172.21.15.159:0/913463736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd78004d40 0x7fbd780051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:32.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.209+0000 7fbd95200640 1 -- 172.21.15.159:0/913463736 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd78003520 con 0x7fbd78003f30 2024-04-04T06:17:32.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.210+0000 7fbd95200640 1 --2- 172.21.15.159:0/913463736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd78003f30 0x7fbd78004800 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009a80 tx=0x7fbd78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8cfced622ed0ccd server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:32.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.210+0000 7fbd8fe00640 1 -- 172.21.15.159:0/913463736 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c009680 con 0x7fbd78003f30 2024-04-04T06:17:32.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.210+0000 7fbd8fe00640 1 -- 172.21.15.159:0/913463736 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd7c005990 con 0x7fbd78003f30 2024-04-04T06:17:32.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.210+0000 7fbd8fe00640 1 -- 172.21.15.159:0/913463736 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c005d80 con 0x7fbd78003f30 2024-04-04T06:17:32.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.211+0000 7fbd97000640 1 -- 172.21.15.159:0/913463736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd78003f30 msgr2=0x7fbd78004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:32.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.211+0000 7fbd97000640 1 --2- 172.21.15.159:0/913463736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd78003f30 0x7fbd78004800 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009a80 tx=0x7fbd78005830 comp rx=0 tx=0).stop 2024-04-04T06:17:32.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.211+0000 7fbd96600640 1 -- 172.21.15.159:0/913463736 reap_dead start 2024-04-04T06:17:32.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.211+0000 7fbd97000640 1 -- 172.21.15.159:0/913463736 shutdown_connections 2024-04-04T06:17:32.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.211+0000 7fbd97000640 1 -- 172.21.15.159:0/913463736 >> 172.21.15.159:0/913463736 conn(0x7fbd900678b0 msgr2=0x7fbd900d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:32.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.211+0000 7fbd97000640 1 -- 172.21.15.159:0/913463736 shutdown_connections 2024-04-04T06:17:32.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.211+0000 7fbd97000640 1 -- 172.21.15.159:0/913463736 wait complete. 2024-04-04T06:17:32.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.213+0000 7fbd97000640 1 Processor -- start 2024-04-04T06:17:32.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.213+0000 7fbd97000640 1 -- start start 2024-04-04T06:17:32.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.214+0000 7fbd97000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd78003f30 0x7fbd90156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:32.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.214+0000 7fbd97000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd901f7530 0x7fbd90156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:32.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.214+0000 7fbd97000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd90159f20 0x7fbd9015a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:32.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.214+0000 7fbd97000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbd900b9e20 con 0x7fbd901f7530 2024-04-04T06:17:32.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.214+0000 7fbd97000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbd900d3920 con 0x7fbd90159f20 2024-04-04T06:17:32.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.214+0000 7fbd97000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbd900b9fa0 con 0x7fbd78003f30 2024-04-04T06:17:32.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.214+0000 7fbd95200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd901f7530 0x7fbd90156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:32.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.214+0000 7fbd95200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd901f7530 0x7fbd90156ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48438/0 (socket says 172.21.15.159:48438) 2024-04-04T06:17:32.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.214+0000 7fbd95200640 1 -- 172.21.15.159:0/1669124567 learned_addr learned my addr 172.21.15.159:0/1669124567 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:32.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.214+0000 7fbd95c00640 1 --2- 172.21.15.159:0/1669124567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd78003f30 0x7fbd90156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:32.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.214+0000 7fbd96600640 1 --2- 172.21.15.159:0/1669124567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd90159f20 0x7fbd9015a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:32.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.215+0000 7fbd95200640 1 -- 172.21.15.159:0/1669124567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd78003f30 msgr2=0x7fbd90156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:32.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.215+0000 7fbd95200640 1 --2- 172.21.15.159:0/1669124567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd78003f30 0x7fbd90156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:32.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.215+0000 7fbd95200640 1 -- 172.21.15.159:0/1669124567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd90159f20 msgr2=0x7fbd9015a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:32.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.215+0000 7fbd95200640 1 --2- 172.21.15.159:0/1669124567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd90159f20 0x7fbd9015a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:32.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.215+0000 7fbd95200640 1 -- 172.21.15.159:0/1669124567 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd901f7af0 con 0x7fbd901f7530 2024-04-04T06:17:32.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.215+0000 7fbd95c00640 1 --2- 172.21.15.159:0/1669124567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd78003f30 0x7fbd90156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:32.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.215+0000 7fbd95200640 1 --2- 172.21.15.159:0/1669124567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd901f7530 0x7fbd90156ff0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c036040 tx=0x7fbd7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:32.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.215+0000 7fbd8d600640 1 -- 172.21.15.159:0/1669124567 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c043070 con 0x7fbd901f7530 2024-04-04T06:17:32.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.215+0000 7fbd8d600640 1 -- 172.21.15.159:0/1669124567 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd7c02cca0 con 0x7fbd901f7530 2024-04-04T06:17:32.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.215+0000 7fbd97000640 1 -- 172.21.15.159:0/1669124567 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd901f7dd0 con 0x7fbd901f7530 2024-04-04T06:17:32.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.215+0000 7fbd8d600640 1 -- 172.21.15.159:0/1669124567 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c004240 con 0x7fbd901f7530 2024-04-04T06:17:32.212 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.215+0000 7fbd97000640 1 -- 172.21.15.159:0/1669124567 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd901f8390 con 0x7fbd901f7530 2024-04-04T06:17:32.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.217+0000 7fbd97000640 1 -- 172.21.15.159:0/1669124567 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd90075940 con 0x7fbd901f7530 2024-04-04T06:17:32.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.217+0000 7fbd8d600640 1 -- 172.21.15.159:0/1669124567 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd7c03d040 con 0x7fbd901f7530 2024-04-04T06:17:32.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.218+0000 7fbd8d600640 1 --2- 172.21.15.159:0/1669124567 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd7006c8f0 0x7fbd7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:32.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.218+0000 7fbd8d600640 1 -- 172.21.15.159:0/1669124567 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5795+0+0 (secure 0 0 0) 0x7fbd7c0ada60 con 0x7fbd901f7530 2024-04-04T06:17:32.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.219+0000 7fbd95c00640 1 --2- 172.21.15.159:0/1669124567 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd7006c8f0 0x7fbd7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:32.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.220+0000 7fbd95c00640 1 --2- 172.21.15.159:0/1669124567 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd7006c8f0 0x7fbd7006edb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbd84001eb0 tx=0x7fbd84001dc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:32.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.223+0000 7fbd8d600640 1 -- 172.21.15.159:0/1669124567 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbd7c07bfe0 con 0x7fbd901f7530 2024-04-04T06:17:32.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.425+0000 7fbd97000640 1 -- 172.21.15.159:0/1669124567 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbd900d0d40 con 0x7fbd901f7530 2024-04-04T06:17:32.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.426+0000 7fbd8d600640 1 -- 172.21.15.159:0/1669124567 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) ==== 104+0+11 (secure 0 0 0) 0x7fbd7c080e20 con 0x7fbd901f7530 2024-04-04T06:17:32.422 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:17:32.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.429+0000 7fbd97000640 1 -- 172.21.15.159:0/1669124567 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd7006c8f0 msgr2=0x7fbd7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:32.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.429+0000 7fbd97000640 1 --2- 172.21.15.159:0/1669124567 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd7006c8f0 0x7fbd7006edb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbd84001eb0 tx=0x7fbd84001dc0 comp rx=0 tx=0).stop 2024-04-04T06:17:32.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.429+0000 7fbd97000640 1 -- 172.21.15.159:0/1669124567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd901f7530 msgr2=0x7fbd90156ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:32.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.429+0000 7fbd97000640 1 --2- 172.21.15.159:0/1669124567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd901f7530 0x7fbd90156ff0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c036040 tx=0x7fbd7c004730 comp rx=0 tx=0).stop 2024-04-04T06:17:32.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.429+0000 7fbd97000640 1 -- 172.21.15.159:0/1669124567 shutdown_connections 2024-04-04T06:17:32.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.429+0000 7fbd97000640 1 --2- 172.21.15.159:0/1669124567 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbd7006c8f0 0x7fbd7006edb0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:32.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.429+0000 7fbd97000640 1 --2- 172.21.15.159:0/1669124567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd90159f20 0x7fbd9015a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:32.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.429+0000 7fbd97000640 1 --2- 172.21.15.159:0/1669124567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd901f7530 0x7fbd90156ff0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:32.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.429+0000 7fbd97000640 1 --2- 172.21.15.159:0/1669124567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbd78003f30 0x7fbd90156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:32.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.429+0000 7fbd97000640 1 -- 172.21.15.159:0/1669124567 >> 172.21.15.159:0/1669124567 conn(0x7fbd900678b0 msgr2=0x7fbd900d4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:32.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.429+0000 7fbd97000640 1 -- 172.21.15.159:0/1669124567 shutdown_connections 2024-04-04T06:17:32.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.429+0000 7fbd97000640 1 -- 172.21.15.159:0/1669124567 wait complete. 2024-04-04T06:17:32.442 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:17:32.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.663+0000 7f83d7200640 1 Processor -- start 2024-04-04T06:17:32.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.663+0000 7f83d7200640 1 -- start start 2024-04-04T06:17:32.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.664+0000 7f83d7200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d815a590 0x7f83d80cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:32.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.665+0000 7f83d7200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f83d806f8f0 0x7f83d80b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:32.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.665+0000 7f83d7200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d80cdcf0 con 0x7f83d815a970 2024-04-04T06:17:32.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.665+0000 7f83d7200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f83d806b570 con 0x7f83d806f8f0 2024-04-04T06:17:32.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.665+0000 7f83d7200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f83d806f280 con 0x7f83d815a590 2024-04-04T06:17:32.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.665+0000 7f83d5400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f83d806f8f0 0x7f83d80b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:32.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.665+0000 7f83d5400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f83d806f8f0 0x7f83d80b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48448/0 (socket says 172.21.15.159:48448) 2024-04-04T06:17:32.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.665+0000 7f83d5400640 1 -- 172.21.15.159:0/4212164942 learned_addr learned my addr 172.21.15.159:0/4212164942 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:32.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.665+0000 7f83d5e00640 1 --2- 172.21.15.159:0/4212164942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d815a590 0x7f83d80cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:32.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.665+0000 7f83d5400640 1 -- 172.21.15.159:0/4212164942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d815a590 msgr2=0x7f83d80cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:32.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.665+0000 7f83d5400640 1 --2- 172.21.15.159:0/4212164942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d815a590 0x7f83d80cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:32.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.665+0000 7f83d5400640 1 -- 172.21.15.159:0/4212164942 >> v1:172.21.15.159:6789/0 conn(0x7f83d815a970 legacy=0x7f83d806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:32.661 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.666+0000 7f83d5400640 1 -- 172.21.15.159:0/4212164942 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d806b7c0 con 0x7f83d806f8f0 2024-04-04T06:17:32.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.667+0000 7f83d5400640 1 --2- 172.21.15.159:0/4212164942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d806f8f0 0x7f83d80b4220 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009870 tx=0x7f83c402ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d60ed12d7846d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:32.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.667+0000 7f83d4a00640 1 -- 172.21.15.159:0/4212164942 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c402e850 con 0x7f83d806f8f0 2024-04-04T06:17:32.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.667+0000 7f83d4a00640 1 -- 172.21.15.159:0/4212164942 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c402e9f0 con 0x7f83d806f8f0 2024-04-04T06:17:32.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.667+0000 7f83d4a00640 1 -- 172.21.15.159:0/4212164942 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c402edc0 con 0x7f83d806f8f0 2024-04-04T06:17:32.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.668+0000 7f83d7200640 1 -- 172.21.15.159:0/4212164942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d806f8f0 msgr2=0x7f83d80b4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:32.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.668+0000 7f83d7200640 1 --2- 172.21.15.159:0/4212164942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d806f8f0 0x7f83d80b4220 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009870 tx=0x7f83c402ca90 comp rx=0 tx=0).stop 2024-04-04T06:17:32.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.668+0000 7f83d7200640 1 -- 172.21.15.159:0/4212164942 shutdown_connections 2024-04-04T06:17:32.663 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.668+0000 7f83d7200640 1 --2- 172.21.15.159:0/4212164942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d806f8f0 0x7f83d80b4220 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:32.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.668+0000 7f83d7200640 1 --2- 172.21.15.159:0/4212164942 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d815a590 0x7f83d80cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:32.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.668+0000 7f83d7200640 1 -- 172.21.15.159:0/4212164942 >> 172.21.15.159:0/4212164942 conn(0x7f83d80d0770 msgr2=0x7f83d80d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:32.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.668+0000 7f83d7200640 1 -- 172.21.15.159:0/4212164942 shutdown_connections 2024-04-04T06:17:32.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.668+0000 7f83d7200640 1 -- 172.21.15.159:0/4212164942 wait complete. 2024-04-04T06:17:32.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.670+0000 7f83d7200640 1 Processor -- start 2024-04-04T06:17:32.665 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.670+0000 7f83d7200640 1 -- start start 2024-04-04T06:17:32.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.672+0000 7f83d7200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d806f8f0 0x7f83d814fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:32.668 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.672+0000 7f83d7200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d815a590 0x7f83d815c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:32.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.672+0000 7f83d7200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d815a970 0x7f83d815cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:32.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.672+0000 7f83d7200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f83d806ee30 con 0x7f83d815a970 2024-04-04T06:17:32.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.672+0000 7f83d7200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f83d806bd80 con 0x7f83d806f8f0 2024-04-04T06:17:32.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.672+0000 7f83d7200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f83d806efb0 con 0x7f83d815a590 2024-04-04T06:17:32.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.672+0000 7f83d6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d815a970 0x7f83d815cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:32.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.672+0000 7f83d6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d815a970 0x7f83d815cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48458/0 (socket says 172.21.15.159:48458) 2024-04-04T06:17:32.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.672+0000 7f83d6800640 1 -- 172.21.15.159:0/3875034633 learned_addr learned my addr 172.21.15.159:0/3875034633 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:32.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.672+0000 7f83d5400640 1 --2- 172.21.15.159:0/3875034633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d815a590 0x7f83d815c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:32.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.672+0000 7f83d5e00640 1 --2- 172.21.15.159:0/3875034633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d806f8f0 0x7f83d814fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:32.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.673+0000 7f83d6800640 1 -- 172.21.15.159:0/3875034633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d815a590 msgr2=0x7f83d815c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:32.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.673+0000 7f83d6800640 1 --2- 172.21.15.159:0/3875034633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d815a590 0x7f83d815c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:32.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.673+0000 7f83d6800640 1 -- 172.21.15.159:0/3875034633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d806f8f0 msgr2=0x7f83d814fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:32.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.673+0000 7f83d6800640 1 --2- 172.21.15.159:0/3875034633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d806f8f0 0x7f83d814fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:32.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.673+0000 7f83d6800640 1 -- 172.21.15.159:0/3875034633 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d8154a00 con 0x7f83d815a970 2024-04-04T06:17:32.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.673+0000 7f83d5e00640 1 --2- 172.21.15.159:0/3875034633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d806f8f0 0x7f83d814fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:32.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.673+0000 7f83d6800640 1 --2- 172.21.15.159:0/3875034633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d815a970 0x7f83d815cb60 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f83d000c6e0 tx=0x7f83d000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:32.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.673+0000 7f83ce000640 1 -- 172.21.15.159:0/3875034633 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d0003b70 con 0x7f83d815a970 2024-04-04T06:17:32.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.673+0000 7f83ce000640 1 -- 172.21.15.159:0/3875034633 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83d0003d30 con 0x7f83d815a970 2024-04-04T06:17:32.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.673+0000 7f83d7200640 1 -- 172.21.15.159:0/3875034633 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d8154ce0 con 0x7f83d815a970 2024-04-04T06:17:32.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.673+0000 7f83ce000640 1 -- 172.21.15.159:0/3875034633 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d0005310 con 0x7f83d815a970 2024-04-04T06:17:32.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.673+0000 7f83d7200640 1 -- 172.21.15.159:0/3875034633 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d8150170 con 0x7f83d815a970 2024-04-04T06:17:32.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.674+0000 7f83d7200640 1 -- 172.21.15.159:0/3875034633 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d8075940 con 0x7f83d815a970 2024-04-04T06:17:32.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.675+0000 7f83ce000640 1 -- 172.21.15.159:0/3875034633 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83d00054b0 con 0x7f83d815a970 2024-04-04T06:17:32.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.676+0000 7f83ce000640 1 --2- 172.21.15.159:0/3875034633 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f83b806c8f0 0x7f83b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:32.671 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.676+0000 7f83ce000640 1 -- 172.21.15.159:0/3875034633 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5795+0+0 (secure 0 0 0) 0x7f83d008ba30 con 0x7f83d815a970 2024-04-04T06:17:32.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.677+0000 7f83d5e00640 1 --2- 172.21.15.159:0/3875034633 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f83b806c8f0 0x7f83b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:32.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.678+0000 7f83d5e00640 1 --2- 172.21.15.159:0/3875034633 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f83b806c8f0 0x7f83b806edb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f83c8006fd0 tx=0x7f83c8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:32.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.681+0000 7f83ce000640 1 -- 172.21.15.159:0/3875034633 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f83d0059df0 con 0x7f83d815a970 2024-04-04T06:17:32.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.881+0000 7f83d7200640 1 -- 172.21.15.159:0/3875034633 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f83d814f9f0 con 0x7f83d815a970 2024-04-04T06:17:32.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.882+0000 7f83ce000640 1 -- 172.21.15.159:0/3875034633 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) ==== 100+0+11 (secure 0 0 0) 0x7f83d00596f0 con 0x7f83d815a970 2024-04-04T06:17:32.877 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:17:32.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.884+0000 7f83d7200640 1 -- 172.21.15.159:0/3875034633 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f83b806c8f0 msgr2=0x7f83b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:32.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.884+0000 7f83d7200640 1 --2- 172.21.15.159:0/3875034633 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f83b806c8f0 0x7f83b806edb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f83c8006fd0 tx=0x7f83c8008040 comp rx=0 tx=0).stop 2024-04-04T06:17:32.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.884+0000 7f83d7200640 1 -- 172.21.15.159:0/3875034633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d815a970 msgr2=0x7f83d815cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:32.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.884+0000 7f83d7200640 1 --2- 172.21.15.159:0/3875034633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d815a970 0x7f83d815cb60 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f83d000c6e0 tx=0x7f83d000cba0 comp rx=0 tx=0).stop 2024-04-04T06:17:32.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.885+0000 7f83d7200640 1 -- 172.21.15.159:0/3875034633 shutdown_connections 2024-04-04T06:17:32.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.885+0000 7f83d7200640 1 --2- 172.21.15.159:0/3875034633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d815a970 0x7f83d815cb60 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:32.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.885+0000 7f83d7200640 1 --2- 172.21.15.159:0/3875034633 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f83d815a590 0x7f83d815c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:32.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.885+0000 7f83d7200640 1 --2- 172.21.15.159:0/3875034633 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f83b806c8f0 0x7f83b806edb0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:32.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.885+0000 7f83d7200640 1 --2- 172.21.15.159:0/3875034633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83d806f8f0 0x7f83d814fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:32.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.885+0000 7f83d7200640 1 -- 172.21.15.159:0/3875034633 >> 172.21.15.159:0/3875034633 conn(0x7f83d80d0770 msgr2=0x7f83d80cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:32.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.885+0000 7f83d7200640 1 -- 172.21.15.159:0/3875034633 shutdown_connections 2024-04-04T06:17:32.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:32.885+0000 7f83d7200640 1 -- 172.21.15.159:0/3875034633 wait complete. 2024-04-04T06:17:32.900 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-04T06:17:33.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.109+0000 7f6392c00640 1 Processor -- start 2024-04-04T06:17:33.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.109+0000 7f6392c00640 1 -- start start 2024-04-04T06:17:33.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.111+0000 7f6392c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f638c153580 0x7f638c0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:33.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.111+0000 7f6392c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f638c0af4b0 0x7f638c1600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:33.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.111+0000 7f6392c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f638c0afab0 con 0x7f638c0d07c0 2024-04-04T06:17:33.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.111+0000 7f6392c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f638c0d0c30 con 0x7f638c153580 2024-04-04T06:17:33.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.111+0000 7f6392c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f638c15cc80 con 0x7f638c0af4b0 2024-04-04T06:17:33.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.111+0000 7f638be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f638c153580 0x7f638c0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:33.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.111+0000 7f638be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f638c153580 0x7f638c0d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48472/0 (socket says 172.21.15.159:48472) 2024-04-04T06:17:33.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.111+0000 7f638be00640 1 -- 172.21.15.159:0/2125802605 learned_addr learned my addr 172.21.15.159:0/2125802605 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:33.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.112+0000 7f6390c00640 1 --2- 172.21.15.159:0/2125802605 >> v2:172.21.15.178:3300/0 conn(0x7f638c0af4b0 0x7f638c1600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:33.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.112+0000 7f638be00640 1 -- 172.21.15.159:0/2125802605 >> v2:172.21.15.178:3300/0 conn(0x7f638c0af4b0 msgr2=0x7f638c1600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:33.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.112+0000 7f638be00640 1 --2- 172.21.15.159:0/2125802605 >> v2:172.21.15.178:3300/0 conn(0x7f638c0af4b0 0x7f638c1600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:33.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.112+0000 7f638be00640 1 -- 172.21.15.159:0/2125802605 >> v1:172.21.15.178:6789/0 conn(0x7f638c0d07c0 legacy=0x7f638c0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:33.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.112+0000 7f638be00640 1 -- 172.21.15.159:0/2125802605 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638c15da00 con 0x7f638c153580 2024-04-04T06:17:33.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.113+0000 7f638be00640 1 --2- 172.21.15.159:0/2125802605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c153580 0x7f638c0d0280 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f638c15a6b0 tx=0x7f637c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be04be6f5add614 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:33.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.113+0000 7f638b400640 1 -- 172.21.15.159:0/2125802605 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c02e070 con 0x7f638c153580 2024-04-04T06:17:33.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.113+0000 7f638b400640 1 -- 172.21.15.159:0/2125802605 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f637c037070 con 0x7f638c153580 2024-04-04T06:17:33.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.113+0000 7f638b400640 1 -- 172.21.15.159:0/2125802605 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c031a00 con 0x7f638c153580 2024-04-04T06:17:33.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.114+0000 7f6392c00640 1 -- 172.21.15.159:0/2125802605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c153580 msgr2=0x7f638c0d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:33.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.114+0000 7f6392c00640 1 --2- 172.21.15.159:0/2125802605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c153580 0x7f638c0d0280 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f638c15a6b0 tx=0x7f637c02b880 comp rx=0 tx=0).stop 2024-04-04T06:17:33.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.114+0000 7f6392c00640 1 -- 172.21.15.159:0/2125802605 shutdown_connections 2024-04-04T06:17:33.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.114+0000 7f6392c00640 1 --2- 172.21.15.159:0/2125802605 >> v2:172.21.15.178:3300/0 conn(0x7f638c0af4b0 0x7f638c1600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:33.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.114+0000 7f6392c00640 1 --2- 172.21.15.159:0/2125802605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c153580 0x7f638c0d0280 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:33.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.114+0000 7f6392c00640 1 -- 172.21.15.159:0/2125802605 >> 172.21.15.159:0/2125802605 conn(0x7f638c06b750 msgr2=0x7f638c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:33.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.114+0000 7f6392c00640 1 -- 172.21.15.159:0/2125802605 shutdown_connections 2024-04-04T06:17:33.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.114+0000 7f6392c00640 1 -- 172.21.15.159:0/2125802605 wait complete. 2024-04-04T06:17:33.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.116+0000 7f6392c00640 1 Processor -- start 2024-04-04T06:17:33.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.116+0000 7f6392c00640 1 -- start start 2024-04-04T06:17:33.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.117+0000 7f6392c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f638c0af4b0 0x7f638c1ef570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:33.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.117+0000 7f6392c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f638c0d07c0 0x7f638c157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:33.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.117+0000 7f6392c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c153580 0x7f638c154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:33.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.117+0000 7f6392c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f638c15d4e0 con 0x7f638c153580 2024-04-04T06:17:33.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.117+0000 7f6392c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f638c15d200 con 0x7f638c0d07c0 2024-04-04T06:17:33.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.117+0000 7f6392c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f638c153960 con 0x7f638c0af4b0 2024-04-04T06:17:33.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.117+0000 7f6390c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c153580 0x7f638c154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:33.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.117+0000 7f6390c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c153580 0x7f638c154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48478/0 (socket says 172.21.15.159:48478) 2024-04-04T06:17:33.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.117+0000 7f6390c00640 1 -- 172.21.15.159:0/2066754181 learned_addr learned my addr 172.21.15.159:0/2066754181 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:33.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.118+0000 7f638be00640 1 --2- 172.21.15.159:0/2066754181 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f638c0af4b0 0x7f638c1ef570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:33.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.118+0000 7f6383400640 1 --2- 172.21.15.159:0/2066754181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f638c0d07c0 0x7f638c157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:33.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.118+0000 7f6390c00640 1 -- 172.21.15.159:0/2066754181 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f638c0af4b0 msgr2=0x7f638c1ef570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:33.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.118+0000 7f6390c00640 1 --2- 172.21.15.159:0/2066754181 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f638c0af4b0 0x7f638c1ef570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:33.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.118+0000 7f6390c00640 1 -- 172.21.15.159:0/2066754181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f638c0d07c0 msgr2=0x7f638c157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:33.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.118+0000 7f6390c00640 1 --2- 172.21.15.159:0/2066754181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f638c0d07c0 0x7f638c157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:33.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.118+0000 7f6390c00640 1 -- 172.21.15.159:0/2066754181 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638c153da0 con 0x7f638c153580 2024-04-04T06:17:33.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.118+0000 7f638be00640 1 --2- 172.21.15.159:0/2066754181 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f638c0af4b0 0x7f638c1ef570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:33.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.118+0000 7f6383400640 1 --2- 172.21.15.159:0/2066754181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f638c0d07c0 0x7f638c157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:33.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.118+0000 7f6390c00640 1 --2- 172.21.15.159:0/2066754181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c153580 0x7f638c154e40 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f6384007e90 tx=0x7f638400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:33.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.118+0000 7f6388c00640 1 -- 172.21.15.159:0/2066754181 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384014420 con 0x7f638c153580 2024-04-04T06:17:33.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.119+0000 7f6388c00640 1 -- 172.21.15.159:0/2066754181 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6384019070 con 0x7f638c153580 2024-04-04T06:17:33.116 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.119+0000 7f6392c00640 1 -- 172.21.15.159:0/2066754181 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f638c154080 con 0x7f638c153580 2024-04-04T06:17:33.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.119+0000 7f6388c00640 1 -- 172.21.15.159:0/2066754181 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384014800 con 0x7f638c153580 2024-04-04T06:17:33.117 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.119+0000 7f6392c00640 1 -- 172.21.15.159:0/2066754181 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f638c154640 con 0x7f638c153580 2024-04-04T06:17:33.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.120+0000 7f6392c00640 1 -- 172.21.15.159:0/2066754181 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f638c0685c0 con 0x7f638c153580 2024-04-04T06:17:33.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.121+0000 7f6388c00640 1 -- 172.21.15.159:0/2066754181 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f63840149a0 con 0x7f638c153580 2024-04-04T06:17:33.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.122+0000 7f6388c00640 1 --2- 172.21.15.159:0/2066754181 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f636806c920 0x7f636806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:33.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.122+0000 7f6388c00640 1 -- 172.21.15.159:0/2066754181 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5795+0+0 (secure 0 0 0) 0x7f638408c390 con 0x7f638c153580 2024-04-04T06:17:33.118 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.122+0000 7f638be00640 1 --2- 172.21.15.159:0/2066754181 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f636806c920 0x7f636806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:33.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.123+0000 7f638be00640 1 --2- 172.21.15.159:0/2066754181 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f636806c920 0x7f636806ede0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f637c036000 tx=0x7f637c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:33.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.131+0000 7f6388c00640 1 -- 172.21.15.159:0/2066754181 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f638405a760 con 0x7f638c153580 2024-04-04T06:17:33.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.345+0000 7f6392c00640 1 -- 172.21.15.159:0/2066754181 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f638c06eca0 con 0x7f638c153580 2024-04-04T06:17:33.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.346+0000 7f6388c00640 1 -- 172.21.15.159:0/2066754181 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v114) ==== 80+0+124 (secure 0 0 0) 0x7f638405a060 con 0x7f638c153580 2024-04-04T06:17:33.341 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6834/1017894858 2024-04-05T06:17:21.596764+0000 2024-04-04T06:17:33.341 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6832/1017894858 2024-04-05T06:17:21.596764+0000 2024-04-04T06:17:33.341 INFO:teuthology.orchestra.run.smithi159.stderr:listed 2 entries 2024-04-04T06:17:33.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.349+0000 7f6392c00640 1 -- 172.21.15.159:0/2066754181 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f636806c920 msgr2=0x7f636806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:33.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.349+0000 7f6392c00640 1 --2- 172.21.15.159:0/2066754181 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f636806c920 0x7f636806ede0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f637c036000 tx=0x7f637c008030 comp rx=0 tx=0).stop 2024-04-04T06:17:33.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.349+0000 7f6392c00640 1 -- 172.21.15.159:0/2066754181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c153580 msgr2=0x7f638c154e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:33.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.349+0000 7f6392c00640 1 --2- 172.21.15.159:0/2066754181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c153580 0x7f638c154e40 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f6384007e90 tx=0x7f638400e4b0 comp rx=0 tx=0).stop 2024-04-04T06:17:33.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.349+0000 7f6392c00640 1 -- 172.21.15.159:0/2066754181 shutdown_connections 2024-04-04T06:17:33.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.349+0000 7f6392c00640 1 --2- 172.21.15.159:0/2066754181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c153580 0x7f638c154e40 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:33.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.349+0000 7f6392c00640 1 --2- 172.21.15.159:0/2066754181 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f636806c920 0x7f636806ede0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:33.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.349+0000 7f6392c00640 1 --2- 172.21.15.159:0/2066754181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f638c0d07c0 0x7f638c157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:33.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.349+0000 7f6392c00640 1 --2- 172.21.15.159:0/2066754181 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f638c0af4b0 0x7f638c1ef570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:33.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.349+0000 7f6392c00640 1 -- 172.21.15.159:0/2066754181 >> 172.21.15.159:0/2066754181 conn(0x7f638c06b750 msgr2=0x7f638c0afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:33.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.349+0000 7f6392c00640 1 -- 172.21.15.159:0/2066754181 shutdown_connections 2024-04-04T06:17:33.345 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.349+0000 7f6392c00640 1 -- 172.21.15.159:0/2066754181 wait complete. 2024-04-04T06:17:33.361 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-04T06:17:33.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:33.361 DEBUG:teuthology.orchestra.run.smithi159:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-04T06:17:33.448 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:17:33.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:33.448 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-04T06:17:33.495 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:17:33.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.745+0000 7fbb94c00640 1 Processor -- start 2024-04-04T06:17:33.740 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.745+0000 7fbb94c00640 1 -- start start 2024-04-04T06:17:33.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.746+0000 7fbb94c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fbb900d0e90 0x7fbb900b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:33.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.747+0000 7fbb94c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb9015b6f0 con 0x7fbb9015b310 2024-04-04T06:17:33.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.747+0000 7fbb94c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb90072e80 con 0x7fbb900727f0 2024-04-04T06:17:33.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.747+0000 7fbb94c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fbb90072c50 con 0x7fbb900d0e90 2024-04-04T06:17:33.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.747+0000 7fbb8de00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fbb9015b310 0x7fbb9015f7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:51020/0 (socket says 172.21.15.159:51020) 2024-04-04T06:17:33.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.747+0000 7fbb8de00640 1 -- 172.21.15.159:0/1535863471 learned_addr learned my addr 172.21.15.159:0/1535863471 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:33.742 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.747+0000 7fbb8e800640 1 --2- 172.21.15.159:0/1535863471 >> v2:172.21.15.159:3300/0 conn(0x7fbb900d0e90 0x7fbb900b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:33.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.747+0000 7fbb8e800640 1 -- 172.21.15.159:0/1535863471 >> v1:172.21.15.178:6789/0 conn(0x7fbb900727f0 legacy=0x7fbb900d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:33.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.747+0000 7fbb8ca00640 1 -- 172.21.15.159:0/1535863471 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 813101432 0 0) 0x7fbb9015b6f0 con 0x7fbb9015b310 2024-04-04T06:17:33.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.747+0000 7fbb8e800640 1 -- 172.21.15.159:0/1535863471 >> v1:172.21.15.159:6789/0 conn(0x7fbb9015b310 legacy=0x7fbb9015f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:33.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.747+0000 7fbb8e800640 1 -- 172.21.15.159:0/1535863471 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb90073160 con 0x7fbb900d0e90 2024-04-04T06:17:33.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.749+0000 7fbb8e800640 1 --2- 172.21.15.159:0/1535863471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb900d0e90 0x7fbb900b4f90 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009dc0 tx=0x7fbb8802bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc73cdccfbccd6e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:33.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.749+0000 7fbb8ca00640 1 -- 172.21.15.159:0/1535863471 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88039070 con 0x7fbb900d0e90 2024-04-04T06:17:33.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.749+0000 7fbb8ca00640 1 -- 172.21.15.159:0/1535863471 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb8802cb20 con 0x7fbb900d0e90 2024-04-04T06:17:33.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.749+0000 7fbb8ca00640 1 -- 172.21.15.159:0/1535863471 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88034440 con 0x7fbb900d0e90 2024-04-04T06:17:33.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.750+0000 7fbb94c00640 1 -- 172.21.15.159:0/1535863471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb900d0e90 msgr2=0x7fbb900b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:33.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.750+0000 7fbb94c00640 1 --2- 172.21.15.159:0/1535863471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb900d0e90 0x7fbb900b4f90 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009dc0 tx=0x7fbb8802bd70 comp rx=0 tx=0).stop 2024-04-04T06:17:33.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.750+0000 7fbb94c00640 1 -- 172.21.15.159:0/1535863471 shutdown_connections 2024-04-04T06:17:33.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.750+0000 7fbb94c00640 1 --2- 172.21.15.159:0/1535863471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb900d0e90 0x7fbb900b4f90 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:33.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.750+0000 7fbb94c00640 1 -- 172.21.15.159:0/1535863471 >> 172.21.15.159:0/1535863471 conn(0x7fbb9006b990 msgr2=0x7fbb9006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:33.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.750+0000 7fbb94c00640 1 -- 172.21.15.159:0/1535863471 shutdown_connections 2024-04-04T06:17:33.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.750+0000 7fbb94c00640 1 -- 172.21.15.159:0/1535863471 wait complete. 2024-04-04T06:17:33.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.752+0000 7fbb94c00640 1 Processor -- start 2024-04-04T06:17:33.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.753+0000 7fbb94c00640 1 -- start start 2024-04-04T06:17:33.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.754+0000 7fbb94c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb900727f0 0x7fbb90150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:33.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.754+0000 7fbb94c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb900d0e90 0x7fbb9015df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:33.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.754+0000 7fbb94c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb9015b310 0x7fbb90157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:33.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.754+0000 7fbb94c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbb90073440 con 0x7fbb900727f0 2024-04-04T06:17:33.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.754+0000 7fbb94c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9015f2c0 con 0x7fbb900d0e90 2024-04-04T06:17:33.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.754+0000 7fbb94c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fbb900adc80 con 0x7fbb9015b310 2024-04-04T06:17:33.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.755+0000 7fbb8de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb900727f0 0x7fbb90150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:33.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.755+0000 7fbb8de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb900727f0 0x7fbb90150030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48488/0 (socket says 172.21.15.159:48488) 2024-04-04T06:17:33.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.755+0000 7fbb8de00640 1 -- 172.21.15.159:0/2211194955 learned_addr learned my addr 172.21.15.159:0/2211194955 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:33.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.755+0000 7fbb8d400640 1 --2- 172.21.15.159:0/2211194955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb900d0e90 0x7fbb9015df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:33.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.755+0000 7fbb8e800640 1 --2- 172.21.15.159:0/2211194955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb9015b310 0x7fbb90157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:33.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.755+0000 7fbb8de00640 1 -- 172.21.15.159:0/2211194955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb9015b310 msgr2=0x7fbb90157810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:33.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.755+0000 7fbb8de00640 1 --2- 172.21.15.159:0/2211194955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb9015b310 0x7fbb90157810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:33.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.755+0000 7fbb8de00640 1 -- 172.21.15.159:0/2211194955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb900d0e90 msgr2=0x7fbb9015df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:33.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.755+0000 7fbb8de00640 1 --2- 172.21.15.159:0/2211194955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb900d0e90 0x7fbb9015df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:33.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.755+0000 7fbb8de00640 1 -- 172.21.15.159:0/2211194955 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb90156770 con 0x7fbb900727f0 2024-04-04T06:17:33.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.755+0000 7fbb8e800640 1 --2- 172.21.15.159:0/2211194955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb9015b310 0x7fbb90157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:33.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.755+0000 7fbb8de00640 1 --2- 172.21.15.159:0/2211194955 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb900727f0 0x7fbb90150030 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000c430 tx=0x7fbb8000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:33.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.756+0000 7fbb86000640 1 -- 172.21.15.159:0/2211194955 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb800074e0 con 0x7fbb900727f0 2024-04-04T06:17:33.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.756+0000 7fbb86000640 1 -- 172.21.15.159:0/2211194955 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb800047e0 con 0x7fbb900727f0 2024-04-04T06:17:33.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.756+0000 7fbb86000640 1 -- 172.21.15.159:0/2211194955 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb80004bd0 con 0x7fbb900727f0 2024-04-04T06:17:33.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.756+0000 7fbb94c00640 1 -- 172.21.15.159:0/2211194955 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb90156a50 con 0x7fbb900727f0 2024-04-04T06:17:33.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.756+0000 7fbb94c00640 1 -- 172.21.15.159:0/2211194955 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb90157010 con 0x7fbb900727f0 2024-04-04T06:17:33.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.757+0000 7fbb94c00640 1 -- 172.21.15.159:0/2211194955 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb900685c0 con 0x7fbb900727f0 2024-04-04T06:17:33.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.758+0000 7fbb86000640 1 -- 172.21.15.159:0/2211194955 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb80011070 con 0x7fbb900727f0 2024-04-04T06:17:33.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.760+0000 7fbb86000640 1 --2- 172.21.15.159:0/2211194955 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbb7006c8f0 0x7fbb7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:33.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.760+0000 7fbb86000640 1 -- 172.21.15.159:0/2211194955 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5795+0+0 (secure 0 0 0) 0x7fbb8008b8a0 con 0x7fbb900727f0 2024-04-04T06:17:33.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.760+0000 7fbb8d400640 1 --2- 172.21.15.159:0/2211194955 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbb7006c8f0 0x7fbb7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:33.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.761+0000 7fbb8d400640 1 --2- 172.21.15.159:0/2211194955 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbb7006c8f0 0x7fbb7006edb0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fbb9006f8b0 tx=0x7fbb7c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:33.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:33.763+0000 7fbb86000640 1 -- 172.21.15.159:0/2211194955 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fbb80059c60 con 0x7fbb900727f0 2024-04-04T06:17:33.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.004+0000 7fbb94c00640 1 -- 172.21.15.159:0/2211194955 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb90155140 con 0x7fbb900727f0 2024-04-04T06:17:34.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.005+0000 7fbb86000640 1 -- 172.21.15.159:0/2211194955 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+5200 (secure 0 0 0) 0x7fbb80059560 con 0x7fbb900727f0 2024-04-04T06:17:34.001 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:34.001 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":107,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5732,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1498324773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1498324773},{"type":"v1","addr":"172.21.15.178:6837","nonce":1498324773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2455031662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":2455031662},{"type":"v1","addr":"172.21.15.159:6835","nonce":2455031662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6343,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1823228151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1823228151},{"type":"v1","addr":"172.21.15.178:6836","nonce":1823228151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:16:57.277344+0000","modified":"2024-04-04T06:17:25.827822+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9135":{"gid":9135,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6834/2439261212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2439261212},{"type":"v1","addr":"172.21.15.159:6834","nonce":2439261212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9135,"qdb_cluster":[9135]},"id":8}]} 2024-04-04T06:17:34.001 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 107 2024-04-04T06:17:34.003 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.008+0000 7fbb94c00640 1 -- 172.21.15.159:0/2211194955 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbb7006c8f0 msgr2=0x7fbb7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.008+0000 7fbb94c00640 1 --2- 172.21.15.159:0/2211194955 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbb7006c8f0 0x7fbb7006edb0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fbb9006f8b0 tx=0x7fbb7c009040 comp rx=0 tx=0).stop 2024-04-04T06:17:34.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.008+0000 7fbb94c00640 1 -- 172.21.15.159:0/2211194955 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb900727f0 msgr2=0x7fbb90150030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.008+0000 7fbb94c00640 1 --2- 172.21.15.159:0/2211194955 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb900727f0 0x7fbb90150030 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000c430 tx=0x7fbb8000c8f0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.008+0000 7fbb94c00640 1 -- 172.21.15.159:0/2211194955 shutdown_connections 2024-04-04T06:17:34.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.008+0000 7fbb94c00640 1 --2- 172.21.15.159:0/2211194955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbb9015b310 0x7fbb90157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.008+0000 7fbb94c00640 1 --2- 172.21.15.159:0/2211194955 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fbb7006c8f0 0x7fbb7006edb0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.008+0000 7fbb94c00640 1 --2- 172.21.15.159:0/2211194955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb900d0e90 0x7fbb9015df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.004 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.008+0000 7fbb94c00640 1 --2- 172.21.15.159:0/2211194955 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb900727f0 0x7fbb90150030 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.008+0000 7fbb94c00640 1 -- 172.21.15.159:0/2211194955 >> 172.21.15.159:0/2211194955 conn(0x7fbb9006b990 msgr2=0x7fbb90152ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:34.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.008+0000 7fbb94c00640 1 -- 172.21.15.159:0/2211194955 shutdown_connections 2024-04-04T06:17:34.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.008+0000 7fbb94c00640 1 -- 172.21.15.159:0/2211194955 wait complete. 2024-04-04T06:17:34.022 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parent mytag123 2024-04-04T06:17:34.247 INFO:teuthology.orchestra.run.smithi159.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-04-04T06:17:34.248 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-04-04T06:17:34.248 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-04T06:17:34.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.352+0000 7ffba6ad9f40 1 Processor -- start 2024-04-04T06:17:34.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.352+0000 7ffba6ad9f40 1 -- start start 2024-04-04T06:17:34.349 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.354+0000 7ffba6ad9f40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ead48294c0 0x55ead4811be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.354+0000 7ffba6ad9f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ead482ab60 0x55ead4812120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.354+0000 7ffba6ad9f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ead48b6ad0 0x55ead4810850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.354+0000 7ffba6ad9f40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ead4829cb0 con 0x55ead48294c0 2024-04-04T06:17:34.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.354+0000 7ffba6ad9f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ead47882c0 con 0x55ead48b6ad0 2024-04-04T06:17:34.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.354+0000 7ffba6ad9f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ead4829f50 con 0x55ead482ab60 2024-04-04T06:17:34.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.354+0000 7ffba3c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ead48294c0 0x55ead4811be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.354+0000 7ffba3c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ead48294c0 0x55ead4811be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41668/0 (socket says 172.21.15.159:41668) 2024-04-04T06:17:34.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.354+0000 7ffba3c00640 1 -- 172.21.15.159:0/3227640875 learned_addr learned my addr 172.21.15.159:0/3227640875 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:34.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.354+0000 7ffb9be00640 1 --2- 172.21.15.159:0/3227640875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ead482ab60 0x55ead4812120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.355+0000 7ffba3c00640 1 -- 172.21.15.159:0/3227640875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ead482ab60 msgr2=0x55ead4812120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:34.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.355+0000 7ffba3c00640 1 --2- 172.21.15.159:0/3227640875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ead482ab60 0x55ead4812120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.355+0000 7ffba3c00640 1 -- 172.21.15.159:0/3227640875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ead48b6ad0 msgr2=0x55ead4810850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:34.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.355+0000 7ffba4600640 1 --2- 172.21.15.159:0/3227640875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ead48b6ad0 0x55ead4810850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.355+0000 7ffba3c00640 1 --2- 172.21.15.159:0/3227640875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ead48b6ad0 0x55ead4810850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.355+0000 7ffba3c00640 1 -- 172.21.15.159:0/3227640875 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ead48b33e0 con 0x55ead48294c0 2024-04-04T06:17:34.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.355+0000 7ffb9be00640 1 --2- 172.21.15.159:0/3227640875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ead482ab60 0x55ead4812120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:34.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.355+0000 7ffba3c00640 1 --2- 172.21.15.159:0/3227640875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ead48294c0 0x55ead4811be0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7ffb9400a770 tx=0x7ffb9400a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc2adf4454ad4b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:34.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.355+0000 7ffba2800640 1 -- 172.21.15.159:0/3227640875 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94019070 con 0x55ead48294c0 2024-04-04T06:17:34.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.355+0000 7ffba2800640 1 -- 172.21.15.159:0/3227640875 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb94007600 con 0x55ead48294c0 2024-04-04T06:17:34.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.355+0000 7ffba2800640 1 -- 172.21.15.159:0/3227640875 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb940079d0 con 0x55ead48294c0 2024-04-04T06:17:34.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.356+0000 7ffba6ad9f40 1 -- 172.21.15.159:0/3227640875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ead48294c0 msgr2=0x55ead4811be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.356+0000 7ffba6ad9f40 1 --2- 172.21.15.159:0/3227640875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ead48294c0 0x55ead4811be0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7ffb9400a770 tx=0x7ffb9400a860 comp rx=0 tx=0).stop 2024-04-04T06:17:34.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.356+0000 7ffba6ad9f40 1 -- 172.21.15.159:0/3227640875 shutdown_connections 2024-04-04T06:17:34.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.356+0000 7ffba6ad9f40 1 --2- 172.21.15.159:0/3227640875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ead48b6ad0 0x55ead4810850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.356+0000 7ffba6ad9f40 1 --2- 172.21.15.159:0/3227640875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ead482ab60 0x55ead4812120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.356+0000 7ffba6ad9f40 1 --2- 172.21.15.159:0/3227640875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ead48294c0 0x55ead4811be0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.356+0000 7ffba6ad9f40 1 -- 172.21.15.159:0/3227640875 >> 172.21.15.159:0/3227640875 conn(0x55ead480ccf0 msgr2=0x55ead48c0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:34.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.356+0000 7ffba6ad9f40 1 -- 172.21.15.159:0/3227640875 shutdown_connections 2024-04-04T06:17:34.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.356+0000 7ffba6ad9f40 1 -- 172.21.15.159:0/3227640875 wait complete. 2024-04-04T06:17:34.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.357+0000 7ffba6ad9f40 1 Processor -- start 2024-04-04T06:17:34.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.357+0000 7ffba6ad9f40 1 -- start start 2024-04-04T06:17:34.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.358+0000 7ffba6ad9f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ead48294c0 0x55ead48b8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.358+0000 7ffba6ad9f40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ead482ab60 0x55ead47e15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.358+0000 7ffba6ad9f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ead48b6ad0 0x55ead48a8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.358+0000 7ffba6ad9f40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ead480d450 con 0x55ead482ab60 2024-04-04T06:17:34.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.358+0000 7ffba6ad9f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ead480d170 con 0x55ead48b6ad0 2024-04-04T06:17:34.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.358+0000 7ffba6ad9f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55ead480d730 con 0x55ead48294c0 2024-04-04T06:17:34.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.358+0000 7ffb9be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ead482ab60 0x55ead47e15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.358+0000 7ffb9be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ead482ab60 0x55ead47e15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41684/0 (socket says 172.21.15.159:41684) 2024-04-04T06:17:34.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.358+0000 7ffb9be00640 1 -- 172.21.15.159:0/1152970970 learned_addr learned my addr 172.21.15.159:0/1152970970 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:34.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.359+0000 7ffba3c00640 1 --2- 172.21.15.159:0/1152970970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ead48294c0 0x55ead48b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.359+0000 7ffba4600640 1 --2- 172.21.15.159:0/1152970970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ead48b6ad0 0x55ead48a8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.359+0000 7ffb9be00640 1 -- 172.21.15.159:0/1152970970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ead48294c0 msgr2=0x55ead48b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.359+0000 7ffb9be00640 1 --2- 172.21.15.159:0/1152970970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55ead48294c0 0x55ead48b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.359+0000 7ffb9be00640 1 -- 172.21.15.159:0/1152970970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ead48b6ad0 msgr2=0x55ead48a8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.359+0000 7ffb9be00640 1 --2- 172.21.15.159:0/1152970970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ead48b6ad0 0x55ead48a8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.359+0000 7ffb9be00640 1 -- 172.21.15.159:0/1152970970 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ead47e1af0 con 0x55ead482ab60 2024-04-04T06:17:34.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.359+0000 7ffba4600640 1 --2- 172.21.15.159:0/1152970970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ead48b6ad0 0x55ead48a8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:34.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.359+0000 7ffb9be00640 1 --2- 172.21.15.159:0/1152970970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ead482ab60 0x55ead47e15b0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c000f60 tx=0x7ffb8c00a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:34.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.359+0000 7ffba1400640 1 -- 172.21.15.159:0/1152970970 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c008040 con 0x55ead482ab60 2024-04-04T06:17:34.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.359+0000 7ffba1400640 1 -- 172.21.15.159:0/1152970970 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb8c014020 con 0x55ead482ab60 2024-04-04T06:17:34.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.359+0000 7ffba1400640 1 -- 172.21.15.159:0/1152970970 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c00f400 con 0x55ead482ab60 2024-04-04T06:17:34.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.359+0000 7ffba6ad9f40 1 -- 172.21.15.159:0/1152970970 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ead47e1dc0 con 0x55ead482ab60 2024-04-04T06:17:34.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.360+0000 7ffba6ad9f40 1 -- 172.21.15.159:0/1152970970 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ead47e2380 con 0x55ead482ab60 2024-04-04T06:17:34.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-04T06:17:34.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.362+0000 7ffba1400640 1 -- 172.21.15.159:0/1152970970 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb8c00f5a0 con 0x55ead482ab60 2024-04-04T06:17:34.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.362+0000 7ffba1400640 1 --2- 172.21.15.159:0/1152970970 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffb8006b2d0 0x7ffb8006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.362+0000 7ffba1400640 1 -- 172.21.15.159:0/1152970970 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5795+0+0 (secure 0 0 0) 0x7ffb8c08a930 con 0x55ead482ab60 2024-04-04T06:17:34.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.363+0000 7ffba6ad9f40 1 --2- 172.21.15.159:0/1152970970 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x55ead48a8850 0x55ead4951750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.363+0000 7ffba6ad9f40 1 -- 172.21.15.159:0/1152970970 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:1 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55ead4951c90 con 0x55ead48a8850 2024-04-04T06:17:34.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.363+0000 7ffba4600640 1 --2- 172.21.15.159:0/1152970970 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x55ead48a8850 0x55ead4951750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.363+0000 7ffba3c00640 1 --2- 172.21.15.159:0/1152970970 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffb8006b2d0 0x7ffb8006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.364+0000 7ffba4600640 1 --2- 172.21.15.159:0/1152970970 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x55ead48a8850 0x55ead4951750 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:34.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.364+0000 7ffba3c00640 1 --2- 172.21.15.159:0/1152970970 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffb8006b2d0 0x7ffb8006d790 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ffb94000c00 tx=0x7ffb9400c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:34.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.364+0000 7ffba4600640 1 -- 172.21.15.159:0/1152970970 <== osd.0 v2:172.21.15.159:6800/455382518 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7ffb9c052930 con 0x55ead48a8850 2024-04-04T06:17:34.525 INFO:teuthology.orchestra.run.smithi159.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-04-04T06:17:34.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.364+0000 7ffba6ad9f40 1 -- 172.21.15.159:0/1152970970 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x55ead48a8850 msgr2=0x55ead4951750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.364+0000 7ffba6ad9f40 1 --2- 172.21.15.159:0/1152970970 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x55ead48a8850 0x55ead4951750 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.364+0000 7ffba6ad9f40 1 -- 172.21.15.159:0/1152970970 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffb8006b2d0 msgr2=0x7ffb8006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.364+0000 7ffba6ad9f40 1 --2- 172.21.15.159:0/1152970970 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffb8006b2d0 0x7ffb8006d790 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ffb94000c00 tx=0x7ffb9400c000 comp rx=0 tx=0).stop 2024-04-04T06:17:34.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.364+0000 7ffba6ad9f40 1 -- 172.21.15.159:0/1152970970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ead482ab60 msgr2=0x55ead47e15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.364+0000 7ffba6ad9f40 1 --2- 172.21.15.159:0/1152970970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ead482ab60 0x55ead47e15b0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c000f60 tx=0x7ffb8c00a5d0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.364+0000 7ffba4600640 1 -- 172.21.15.159:0/1152970970 reap_dead start 2024-04-04T06:17:34.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.365+0000 7ffba6ad9f40 1 -- 172.21.15.159:0/1152970970 shutdown_connections 2024-04-04T06:17:34.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.365+0000 7ffba6ad9f40 1 -- 172.21.15.159:0/1152970970 >> 172.21.15.159:0/1152970970 conn(0x55ead480ccf0 msgr2=0x55ead4811be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:34.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.365+0000 7ffba6ad9f40 1 -- 172.21.15.159:0/1152970970 shutdown_connections 2024-04-04T06:17:34.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.365+0000 7ffba6ad9f40 1 -- 172.21.15.159:0/1152970970 wait complete. 2024-04-04T06:17:34.527 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-04T06:17:34.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.641+0000 7f157b037f40 1 Processor -- start 2024-04-04T06:17:34.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.641+0000 7f157b037f40 1 -- start start 2024-04-04T06:17:34.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.642+0000 7f157b037f40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdb863240 0x55bbdb7c0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.642+0000 7f157b037f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbdb869270 0x55bbdb853d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.642+0000 7f157b037f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbdb869db0 0x55bbdb854290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.642+0000 7f157b037f40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bbdb7de980 con 0x55bbdb863240 2024-04-04T06:17:34.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.642+0000 7f157b037f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bbdb7382c0 con 0x55bbdb869db0 2024-04-04T06:17:34.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.642+0000 7f157b037f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bbdb7dc160 con 0x55bbdb869270 2024-04-04T06:17:34.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.642+0000 7f1577a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdb863240 0x55bbdb7c0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.642+0000 7f1577a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdb863240 0x55bbdb7c0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41696/0 (socket says 172.21.15.159:41696) 2024-04-04T06:17:34.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.642+0000 7f1577a00640 1 -- 172.21.15.159:0/2608922121 learned_addr learned my addr 172.21.15.159:0/2608922121 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:34.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.642+0000 7f1577a00640 1 -- 172.21.15.159:0/2608922121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbdb869270 msgr2=0x55bbdb853d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:34.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.642+0000 7f1577a00640 1 --2- 172.21.15.159:0/2608922121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbdb869270 0x55bbdb853d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.642+0000 7f1577a00640 1 -- 172.21.15.159:0/2608922121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbdb869db0 msgr2=0x55bbdb854290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:34.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.642+0000 7f1577a00640 1 --2- 172.21.15.159:0/2608922121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbdb869db0 0x55bbdb854290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.642+0000 7f1577a00640 1 -- 172.21.15.159:0/2608922121 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbdb7bfdd0 con 0x55bbdb863240 2024-04-04T06:17:34.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.643+0000 7f1577a00640 1 --2- 172.21.15.159:0/2608922121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdb863240 0x55bbdb7c0560 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f156800a4f0 tx=0x7f156804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9603cff1544ff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:34.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.643+0000 7f1576600640 1 -- 172.21.15.159:0/2608922121 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1568061070 con 0x55bbdb863240 2024-04-04T06:17:34.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.643+0000 7f1576600640 1 -- 172.21.15.159:0/2608922121 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f156805c660 con 0x55bbdb863240 2024-04-04T06:17:34.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.643+0000 7f1576600640 1 -- 172.21.15.159:0/2608922121 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1568057040 con 0x55bbdb863240 2024-04-04T06:17:34.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.644+0000 7f157b037f40 1 -- 172.21.15.159:0/2608922121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdb863240 msgr2=0x55bbdb7c0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.644+0000 7f157b037f40 1 --2- 172.21.15.159:0/2608922121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdb863240 0x55bbdb7c0560 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f156800a4f0 tx=0x7f156804c000 comp rx=0 tx=0).stop 2024-04-04T06:17:34.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.644+0000 7f157b037f40 1 -- 172.21.15.159:0/2608922121 shutdown_connections 2024-04-04T06:17:34.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.644+0000 7f157b037f40 1 --2- 172.21.15.159:0/2608922121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbdb869db0 0x55bbdb854290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.644+0000 7f157b037f40 1 --2- 172.21.15.159:0/2608922121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbdb869270 0x55bbdb853d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.644+0000 7f157b037f40 1 --2- 172.21.15.159:0/2608922121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdb863240 0x55bbdb7c0560 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.644+0000 7f157b037f40 1 -- 172.21.15.159:0/2608922121 >> 172.21.15.159:0/2608922121 conn(0x55bbdb7df630 msgr2=0x55bbdb7c3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:34.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.644+0000 7f157b037f40 1 -- 172.21.15.159:0/2608922121 shutdown_connections 2024-04-04T06:17:34.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.644+0000 7f157b037f40 1 -- 172.21.15.159:0/2608922121 wait complete. 2024-04-04T06:17:34.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.645+0000 7f157b037f40 1 Processor -- start 2024-04-04T06:17:34.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.645+0000 7f157b037f40 1 -- start start 2024-04-04T06:17:34.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.646+0000 7f157b037f40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbdb863240 0x55bbdb854980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.646+0000 7f157b037f40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbdb869270 0x55bbdb85d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.646+0000 7f157b037f40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdb869db0 0x55bbdb85dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.646+0000 7f157b037f40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bbdb7c2aa0 con 0x55bbdb869db0 2024-04-04T06:17:34.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.646+0000 7f157b037f40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bbdb7c2d60 con 0x55bbdb863240 2024-04-04T06:17:34.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.646+0000 7f157b037f40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55bbdb7c0040 con 0x55bbdb869270 2024-04-04T06:17:34.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.646+0000 7f1578400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdb869db0 0x55bbdb85dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.646+0000 7f1578400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdb869db0 0x55bbdb85dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41700/0 (socket says 172.21.15.159:41700) 2024-04-04T06:17:34.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.646+0000 7f1578400640 1 -- 172.21.15.159:0/1091557183 learned_addr learned my addr 172.21.15.159:0/1091557183 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:34.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.646+0000 7f1578400640 1 -- 172.21.15.159:0/1091557183 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbdb869270 msgr2=0x55bbdb85d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.646+0000 7f156fe00640 1 --2- 172.21.15.159:0/1091557183 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbdb869270 0x55bbdb85d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.646+0000 7f1577a00640 1 --2- 172.21.15.159:0/1091557183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbdb863240 0x55bbdb854980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.646+0000 7f1578400640 1 --2- 172.21.15.159:0/1091557183 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbdb869270 0x55bbdb85d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.647+0000 7f1578400640 1 -- 172.21.15.159:0/1091557183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbdb863240 msgr2=0x55bbdb854980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.647+0000 7f1578400640 1 --2- 172.21.15.159:0/1091557183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbdb863240 0x55bbdb854980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.647+0000 7f1578400640 1 -- 172.21.15.159:0/1091557183 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbdb790760 con 0x55bbdb869db0 2024-04-04T06:17:34.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.647+0000 7f156fe00640 1 --2- 172.21.15.159:0/1091557183 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55bbdb869270 0x55bbdb85d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:34.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.647+0000 7f1577a00640 1 --2- 172.21.15.159:0/1091557183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bbdb863240 0x55bbdb854980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:34.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.647+0000 7f1578400640 1 --2- 172.21.15.159:0/1091557183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdb869db0 0x55bbdb85dec0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f1570005fc0 tx=0x7f157000a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:34.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.647+0000 7f1575200640 1 -- 172.21.15.159:0/1091557183 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570013430 con 0x55bbdb869db0 2024-04-04T06:17:34.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.647+0000 7f1575200640 1 -- 172.21.15.159:0/1091557183 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1570018070 con 0x55bbdb869db0 2024-04-04T06:17:34.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.647+0000 7f1575200640 1 -- 172.21.15.159:0/1091557183 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570013810 con 0x55bbdb869db0 2024-04-04T06:17:34.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.647+0000 7f157b037f40 1 -- 172.21.15.159:0/1091557183 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bbdb790a40 con 0x55bbdb869db0 2024-04-04T06:17:34.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.647+0000 7f157b037f40 1 -- 172.21.15.159:0/1091557183 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbdb8622f0 con 0x55bbdb869db0 2024-04-04T06:17:34.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.649+0000 7f1575200640 1 -- 172.21.15.159:0/1091557183 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f15700139b0 con 0x55bbdb869db0 2024-04-04T06:17:34.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.650+0000 7f1575200640 1 --2- 172.21.15.159:0/1091557183 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f155406b300 0x7f155406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.650+0000 7f1575200640 1 -- 172.21.15.159:0/1091557183 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5795+0+0 (secure 0 0 0) 0x7f157008a9b0 con 0x55bbdb869db0 2024-04-04T06:17:34.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.651+0000 7f157b037f40 1 --2- 172.21.15.159:0/1091557183 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55bbdb85e400 0x55bbdb8568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.651+0000 7f157b037f40 1 -- 172.21.15.159:0/1091557183 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:1 17.12 17:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x55bbdb856e20 con 0x55bbdb85e400 2024-04-04T06:17:34.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.651+0000 7f1577a00640 1 --2- 172.21.15.159:0/1091557183 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f155406b300 0x7f155406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.651+0000 7f156fe00640 1 --2- 172.21.15.159:0/1091557183 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55bbdb85e400 0x55bbdb8568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.652+0000 7f156fe00640 1 --2- 172.21.15.159:0/1091557183 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55bbdb85e400 0x55bbdb8568e0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:34.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.652+0000 7f1577a00640 1 --2- 172.21.15.159:0/1091557183 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f155406b300 0x7f155406d7c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1568055040 tx=0x7f156804f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:34.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.652+0000 7f156fe00640 1 -- 172.21.15.159:0/1091557183 <== osd.3 v2:172.21.15.159:6820/2070760205 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f156000b6d0 con 0x55bbdb85e400 2024-04-04T06:17:34.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.652+0000 7f157b037f40 1 -- 172.21.15.159:0/1091557183 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55bbdb85e400 msgr2=0x55bbdb8568e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.652+0000 7f157b037f40 1 --2- 172.21.15.159:0/1091557183 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x55bbdb85e400 0x55bbdb8568e0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.652+0000 7f157b037f40 1 -- 172.21.15.159:0/1091557183 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f155406b300 msgr2=0x7f155406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.652+0000 7f157b037f40 1 --2- 172.21.15.159:0/1091557183 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f155406b300 0x7f155406d7c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1568055040 tx=0x7f156804f000 comp rx=0 tx=0).stop 2024-04-04T06:17:34.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.652+0000 7f157b037f40 1 -- 172.21.15.159:0/1091557183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdb869db0 msgr2=0x55bbdb85dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.652+0000 7f157b037f40 1 --2- 172.21.15.159:0/1091557183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbdb869db0 0x55bbdb85dec0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f1570005fc0 tx=0x7f157000a570 comp rx=0 tx=0).stop 2024-04-04T06:17:34.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.652+0000 7f1578400640 1 -- 172.21.15.159:0/1091557183 reap_dead start 2024-04-04T06:17:34.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.652+0000 7f157b037f40 1 -- 172.21.15.159:0/1091557183 shutdown_connections 2024-04-04T06:17:34.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.652+0000 7f157b037f40 1 -- 172.21.15.159:0/1091557183 >> 172.21.15.159:0/1091557183 conn(0x55bbdb7df630 msgr2=0x55bbdb85e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:34.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.653+0000 7f157b037f40 1 -- 172.21.15.159:0/1091557183 shutdown_connections 2024-04-04T06:17:34.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.653+0000 7f157b037f40 1 -- 172.21.15.159:0/1091557183 wait complete. 2024-04-04T06:17:34.657 INFO:teuthology.orchestra.run.smithi159.stdout:mytag123 2024-04-04T06:17:34.657 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-04T06:17:34.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.754+0000 7fb1f420ff40 1 Processor -- start 2024-04-04T06:17:34.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.754+0000 7fb1f420ff40 1 -- start start 2024-04-04T06:17:34.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.755+0000 7fb1f420ff40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1d80022a0 0x562c430557c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.755+0000 7fb1f420ff40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c43053950 0x562c43053dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.755+0000 7fb1f420ff40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c43054310 0x562c431c1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.766 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.755+0000 7fb1f420ff40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562c430a0160 con 0x7fb1d80022a0 2024-04-04T06:17:34.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.755+0000 7fb1f420ff40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562c42ffc2c0 con 0x562c43053950 2024-04-04T06:17:34.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.755+0000 7fb1f420ff40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562c430568d0 con 0x562c43054310 2024-04-04T06:17:34.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.756+0000 7fb1f0800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1d80022a0 0x562c430557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.756+0000 7fb1f0800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1d80022a0 0x562c430557c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41706/0 (socket says 172.21.15.159:41706) 2024-04-04T06:17:34.767 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.756+0000 7fb1f0800640 1 -- 172.21.15.159:0/1298199948 learned_addr learned my addr 172.21.15.159:0/1298199948 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:34.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.756+0000 7fb1efe00640 1 --2- 172.21.15.159:0/1298199948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c43053950 0x562c43053dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.756+0000 7fb1f0800640 1 -- 172.21.15.159:0/1298199948 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c43054310 msgr2=0x562c431c1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:34.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.756+0000 7fb1f0800640 1 --2- 172.21.15.159:0/1298199948 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c43054310 0x562c431c1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.756+0000 7fb1f0800640 1 -- 172.21.15.159:0/1298199948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c43053950 msgr2=0x562c43053dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:34.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.756+0000 7fb1f0800640 1 --2- 172.21.15.159:0/1298199948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c43053950 0x562c43053dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.768 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.756+0000 7fb1f0800640 1 -- 172.21.15.159:0/1298199948 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c43054790 con 0x7fb1d80022a0 2024-04-04T06:17:34.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.756+0000 7fb1f0800640 1 --2- 172.21.15.159:0/1298199948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1d80022a0 0x562c430557c0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fb1e401d080 tx=0x7fb1e4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8286cfaab73e9837 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:34.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.757+0000 7fb1eea00640 1 -- 172.21.15.159:0/1298199948 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4057040 con 0x7fb1d80022a0 2024-04-04T06:17:34.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.757+0000 7fb1eea00640 1 -- 172.21.15.159:0/1298199948 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1e4062070 con 0x7fb1d80022a0 2024-04-04T06:17:34.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.757+0000 7fb1eea00640 1 -- 172.21.15.159:0/1298199948 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4009410 con 0x7fb1d80022a0 2024-04-04T06:17:34.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.757+0000 7fb1f420ff40 1 -- 172.21.15.159:0/1298199948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1d80022a0 msgr2=0x562c430557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.757+0000 7fb1f420ff40 1 --2- 172.21.15.159:0/1298199948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1d80022a0 0x562c430557c0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fb1e401d080 tx=0x7fb1e4002fb0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.769 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.757+0000 7fb1f420ff40 1 -- 172.21.15.159:0/1298199948 shutdown_connections 2024-04-04T06:17:34.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.757+0000 7fb1f420ff40 1 --2- 172.21.15.159:0/1298199948 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562c43054310 0x562c431c1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.757+0000 7fb1f420ff40 1 --2- 172.21.15.159:0/1298199948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c43053950 0x562c43053dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.757+0000 7fb1f420ff40 1 --2- 172.21.15.159:0/1298199948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1d80022a0 0x562c430557c0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.757+0000 7fb1f420ff40 1 -- 172.21.15.159:0/1298199948 >> 172.21.15.159:0/1298199948 conn(0x562c430a3630 msgr2=0x562c430847a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:34.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.757+0000 7fb1f420ff40 1 -- 172.21.15.159:0/1298199948 shutdown_connections 2024-04-04T06:17:34.770 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.757+0000 7fb1f420ff40 1 -- 172.21.15.159:0/1298199948 wait complete. 2024-04-04T06:17:34.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.759+0000 7fb1f420ff40 1 Processor -- start 2024-04-04T06:17:34.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.759+0000 7fb1f420ff40 1 -- start start 2024-04-04T06:17:34.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.760+0000 7fb1f420ff40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c43053950 0x562c43132ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.771 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.760+0000 7fb1f420ff40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c43054310 0x562c43117cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.760+0000 7fb1f420ff40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1d80022a0 0x562c43118210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.760+0000 7fb1f420ff40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562c430878f0 con 0x562c43054310 2024-04-04T06:17:34.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.760+0000 7fb1f420ff40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562c43086bb0 con 0x562c43053950 2024-04-04T06:17:34.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.760+0000 7fb1f420ff40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562c4312cb30 con 0x7fb1d80022a0 2024-04-04T06:17:34.772 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.760+0000 7fb1efe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c43054310 0x562c43117cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.761+0000 7fb1efe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c43054310 0x562c43117cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41716/0 (socket says 172.21.15.159:41716) 2024-04-04T06:17:34.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.761+0000 7fb1efe00640 1 -- 172.21.15.159:0/3587278744 learned_addr learned my addr 172.21.15.159:0/3587278744 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:34.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.761+0000 7fb1efe00640 1 -- 172.21.15.159:0/3587278744 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1d80022a0 msgr2=0x562c43118210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.761+0000 7fb1efe00640 1 --2- 172.21.15.159:0/3587278744 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1d80022a0 0x562c43118210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.761+0000 7fb1efe00640 1 -- 172.21.15.159:0/3587278744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c43053950 msgr2=0x562c43132ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.761+0000 7fb1f0800640 1 --2- 172.21.15.159:0/3587278744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c43053950 0x562c43132ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.761+0000 7fb1efe00640 1 --2- 172.21.15.159:0/3587278744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c43053950 0x562c43132ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.761+0000 7fb1efe00640 1 -- 172.21.15.159:0/3587278744 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c431329d0 con 0x562c43054310 2024-04-04T06:17:34.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.761+0000 7fb1efe00640 1 --2- 172.21.15.159:0/3587278744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c43054310 0x562c43117cd0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009ed0 tx=0x7fb1dc0073f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:34.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.761+0000 7fb1ed600640 1 -- 172.21.15.159:0/3587278744 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc01a070 con 0x562c43054310 2024-04-04T06:17:34.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.762+0000 7fb1ed600640 1 -- 172.21.15.159:0/3587278744 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1dc015600 con 0x562c43054310 2024-04-04T06:17:34.774 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.762+0000 7fb1f420ff40 1 -- 172.21.15.159:0/3587278744 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562c43084520 con 0x562c43054310 2024-04-04T06:17:34.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.762+0000 7fb1ed600640 1 -- 172.21.15.159:0/3587278744 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc0159d0 con 0x562c43054310 2024-04-04T06:17:34.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.762+0000 7fb1f420ff40 1 -- 172.21.15.159:0/3587278744 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c43084ae0 con 0x562c43054310 2024-04-04T06:17:34.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.764+0000 7fb1ed600640 1 -- 172.21.15.159:0/3587278744 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb1dc01d070 con 0x562c43054310 2024-04-04T06:17:34.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.764+0000 7fb1ed600640 1 --2- 172.21.15.159:0/3587278744 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb1d006b2d0 0x7fb1d006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.765+0000 7fb1ed600640 1 -- 172.21.15.159:0/3587278744 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5795+0+0 (secure 0 0 0) 0x7fb1dc089e00 con 0x562c43054310 2024-04-04T06:17:34.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.765+0000 7fb1f420ff40 1 --2- 172.21.15.159:0/3587278744 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x562c43126410 0x562c431c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.775 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.765+0000 7fb1f420ff40 1 -- 172.21.15.159:0/3587278744 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:1 17.19 17:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) -- 0x562c431c6260 con 0x562c43126410 2024-04-04T06:17:34.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.765+0000 7fb1f0800640 1 --2- 172.21.15.159:0/3587278744 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb1d006b2d0 0x7fb1d006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.766+0000 7fb1f1200640 1 --2- 172.21.15.159:0/3587278744 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x562c43126410 0x562c431c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.766+0000 7fb1f0800640 1 --2- 172.21.15.159:0/3587278744 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb1d006b2d0 0x7fb1d006d790 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009760 tx=0x7fb1e404c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:34.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.767+0000 7fb1f1200640 1 --2- 172.21.15.159:0/3587278744 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x562c43126410 0x562c431c5d20 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:34.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.767+0000 7fb1f1200640 1 -- 172.21.15.159:0/3587278744 <== osd.5 v2:172.21.15.178:6802/63746765 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7fb1e8003450 con 0x562c43126410 2024-04-04T06:17:34.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.767+0000 7fb1f420ff40 1 -- 172.21.15.159:0/3587278744 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x562c43126410 msgr2=0x562c431c5d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.767+0000 7fb1f420ff40 1 --2- 172.21.15.159:0/3587278744 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x562c43126410 0x562c431c5d20 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.768+0000 7fb1f420ff40 1 -- 172.21.15.159:0/3587278744 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb1d006b2d0 msgr2=0x7fb1d006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.768+0000 7fb1f420ff40 1 --2- 172.21.15.159:0/3587278744 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb1d006b2d0 0x7fb1d006d790 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009760 tx=0x7fb1e404c000 comp rx=0 tx=0).stop 2024-04-04T06:17:34.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.768+0000 7fb1f420ff40 1 -- 172.21.15.159:0/3587278744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c43054310 msgr2=0x562c43117cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.768+0000 7fb1f420ff40 1 --2- 172.21.15.159:0/3587278744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c43054310 0x562c43117cd0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009ed0 tx=0x7fb1dc0073f0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.768+0000 7fb1f1200640 1 -- 172.21.15.159:0/3587278744 reap_dead start 2024-04-04T06:17:34.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.768+0000 7fb1f420ff40 1 -- 172.21.15.159:0/3587278744 shutdown_connections 2024-04-04T06:17:34.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.768+0000 7fb1f420ff40 1 -- 172.21.15.159:0/3587278744 >> 172.21.15.159:0/3587278744 conn(0x562c430a3630 msgr2=0x562c430557c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:34.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.768+0000 7fb1f420ff40 1 -- 172.21.15.159:0/3587278744 shutdown_connections 2024-04-04T06:17:34.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.768+0000 7fb1f420ff40 1 -- 172.21.15.159:0/3587278744 wait complete. 2024-04-04T06:17:34.779 INFO:teuthology.orchestra.run.smithi159.stdout:mytag123 2024-04-04T06:17:34.780 DEBUG:teuthology.orchestra.run.smithi159:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-04-04T06:17:34.877 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.880+0000 7ff85444cf40 1 Processor -- start 2024-04-04T06:17:34.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.880+0000 7ff85444cf40 1 -- start start 2024-04-04T06:17:34.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.881+0000 7ff85444cf40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591933be240 0x5591932ea920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.881+0000 7ff85444cf40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591933c4270 0x5591932eb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.881+0000 7ff85444cf40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5591933c4bb0 0x55919331ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.881+0000 7ff85444cf40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559193337160 con 0x5591933c4270 2024-04-04T06:17:34.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.881+0000 7ff85444cf40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5591932932c0 con 0x5591933be240 2024-04-04T06:17:34.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.881+0000 7ff85444cf40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5591932ed8d0 con 0x5591933c4bb0 2024-04-04T06:17:34.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.882+0000 7ff84be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591933c4270 0x5591932eb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.882+0000 7ff84be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591933c4270 0x5591932eb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41742/0 (socket says 172.21.15.159:41742) 2024-04-04T06:17:34.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.882+0000 7ff84be00640 1 -- 172.21.15.159:0/823591134 learned_addr learned my addr 172.21.15.159:0/823591134 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:34.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.882+0000 7ff84be00640 1 -- 172.21.15.159:0/823591134 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5591933c4bb0 msgr2=0x55919331ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:34.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.882+0000 7ff84be00640 1 --2- 172.21.15.159:0/823591134 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5591933c4bb0 0x55919331ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.882+0000 7ff84be00640 1 -- 172.21.15.159:0/823591134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591933be240 msgr2=0x5591932ea920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:34.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.882+0000 7ff84be00640 1 --2- 172.21.15.159:0/823591134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591933be240 0x5591932ea920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.882+0000 7ff84be00640 1 -- 172.21.15.159:0/823591134 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591933b9aa0 con 0x5591933c4270 2024-04-04T06:17:34.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.883+0000 7ff84be00640 1 --2- 172.21.15.159:0/823591134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591933c4270 0x5591932eb8e0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7ff83c009790 tx=0x7ff83c009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed18610902b76cf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:34.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.883+0000 7ff84aa00640 1 -- 172.21.15.159:0/823591134 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c017070 con 0x5591933c4270 2024-04-04T06:17:34.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.883+0000 7ff84aa00640 1 -- 172.21.15.159:0/823591134 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff83c0090d0 con 0x5591933c4270 2024-04-04T06:17:34.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.883+0000 7ff84aa00640 1 -- 172.21.15.159:0/823591134 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c006b60 con 0x5591933c4270 2024-04-04T06:17:34.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.883+0000 7ff85444cf40 1 -- 172.21.15.159:0/823591134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591933c4270 msgr2=0x5591932eb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.883+0000 7ff85444cf40 1 --2- 172.21.15.159:0/823591134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591933c4270 0x5591932eb8e0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7ff83c009790 tx=0x7ff83c009c50 comp rx=0 tx=0).stop 2024-04-04T06:17:34.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.884+0000 7ff85444cf40 1 -- 172.21.15.159:0/823591134 shutdown_connections 2024-04-04T06:17:34.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.884+0000 7ff85444cf40 1 --2- 172.21.15.159:0/823591134 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5591933c4bb0 0x55919331ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.884+0000 7ff85444cf40 1 --2- 172.21.15.159:0/823591134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591933c4270 0x5591932eb8e0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.884+0000 7ff85444cf40 1 --2- 172.21.15.159:0/823591134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591933be240 0x5591932ea920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.884+0000 7ff85444cf40 1 -- 172.21.15.159:0/823591134 >> 172.21.15.159:0/823591134 conn(0x55919333a630 msgr2=0x55919331f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:34.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.884+0000 7ff85444cf40 1 -- 172.21.15.159:0/823591134 shutdown_connections 2024-04-04T06:17:34.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.884+0000 7ff85444cf40 1 -- 172.21.15.159:0/823591134 wait complete. 2024-04-04T06:17:34.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.885+0000 7ff85444cf40 1 Processor -- start 2024-04-04T06:17:34.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.885+0000 7ff85444cf40 1 -- start start 2024-04-04T06:17:34.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.886+0000 7ff85444cf40 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5591933be240 0x5591933b8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.886+0000 7ff85444cf40 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591933c4270 0x5591933be620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.886+0000 7ff85444cf40 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591933c4bb0 0x5591933b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.886+0000 7ff85444cf40 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5591933ba0b0 con 0x5591933c4bb0 2024-04-04T06:17:34.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.886+0000 7ff85444cf40 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5591933c6530 con 0x5591933c4270 2024-04-04T06:17:34.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.886+0000 7ff85444cf40 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x5591933ba420 con 0x5591933be240 2024-04-04T06:17:34.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.887+0000 7ff851800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591933c4bb0 0x5591933b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.887+0000 7ff851800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591933c4bb0 0x5591933b8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41758/0 (socket says 172.21.15.159:41758) 2024-04-04T06:17:34.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.887+0000 7ff851800640 1 -- 172.21.15.159:0/1399798875 learned_addr learned my addr 172.21.15.159:0/1399798875 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:34.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.887+0000 7ff851800640 1 -- 172.21.15.159:0/1399798875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5591933be240 msgr2=0x5591933b8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.887+0000 7ff851800640 1 --2- 172.21.15.159:0/1399798875 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5591933be240 0x5591933b8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.887+0000 7ff84be00640 1 --2- 172.21.15.159:0/1399798875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591933c4270 0x5591933be620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.887+0000 7ff851800640 1 -- 172.21.15.159:0/1399798875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591933c4270 msgr2=0x5591933be620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.887+0000 7ff851800640 1 --2- 172.21.15.159:0/1399798875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591933c4270 0x5591933be620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.887+0000 7ff851800640 1 -- 172.21.15.159:0/1399798875 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591932f0ee0 con 0x5591933c4bb0 2024-04-04T06:17:34.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.888+0000 7ff851800640 1 --2- 172.21.15.159:0/1399798875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591933c4bb0 0x5591933b8c30 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7ff84c0020e0 tx=0x7ff84c00b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:34.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.888+0000 7ff849600640 1 -- 172.21.15.159:0/1399798875 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c016070 con 0x5591933c4bb0 2024-04-04T06:17:34.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.888+0000 7ff849600640 1 -- 172.21.15.159:0/1399798875 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff84c006310 con 0x5591933c4bb0 2024-04-04T06:17:34.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.888+0000 7ff849600640 1 -- 172.21.15.159:0/1399798875 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c004700 con 0x5591933c4bb0 2024-04-04T06:17:34.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.888+0000 7ff85444cf40 1 -- 172.21.15.159:0/1399798875 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5591932f11c0 con 0x5591933c4bb0 2024-04-04T06:17:34.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.888+0000 7ff85444cf40 1 -- 172.21.15.159:0/1399798875 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591933b8580 con 0x5591933c4bb0 2024-04-04T06:17:34.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.890+0000 7ff849600640 1 -- 172.21.15.159:0/1399798875 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff84c019070 con 0x5591933c4bb0 2024-04-04T06:17:34.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.891+0000 7ff849600640 1 --2- 172.21.15.159:0/1399798875 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff83006b2d0 0x7ff83006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.892+0000 7ff849600640 1 -- 172.21.15.159:0/1399798875 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7ff84c089a40 con 0x5591933c4bb0 2024-04-04T06:17:34.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.892+0000 7ff85444cf40 1 --2- 172.21.15.159:0/1399798875 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x559193460e80 0x559193461260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:34.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.892+0000 7ff85444cf40 1 -- 172.21.15.159:0/1399798875 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:1 17.5 17:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x5591934617a0 con 0x559193460e80 2024-04-04T06:17:34.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.893+0000 7ff850e00640 1 --2- 172.21.15.159:0/1399798875 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff83006b2d0 0x7ff83006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.893+0000 7ff84be00640 1 --2- 172.21.15.159:0/1399798875 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x559193460e80 0x559193461260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:34.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.894+0000 7ff84be00640 1 --2- 172.21.15.159:0/1399798875 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x559193460e80 0x559193461260 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:34.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.895+0000 7ff850e00640 1 --2- 172.21.15.159:0/1399798875 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff83006b2d0 0x7ff83006d790 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff844054000 tx=0x7ff84404c070 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:34.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.902+0000 7ff84be00640 1 -- 172.21.15.159:0/1399798875 <== osd.6 v2:172.21.15.178:6826/198949157 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7ff83c01a040 con 0x559193460e80 2024-04-04T06:17:34.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.902+0000 7ff85444cf40 1 -- 172.21.15.159:0/1399798875 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x559193460e80 msgr2=0x559193461260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.902+0000 7ff85444cf40 1 --2- 172.21.15.159:0/1399798875 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x559193460e80 0x559193461260 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.902+0000 7ff85444cf40 1 -- 172.21.15.159:0/1399798875 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff83006b2d0 msgr2=0x7ff83006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.902+0000 7ff85444cf40 1 --2- 172.21.15.159:0/1399798875 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff83006b2d0 0x7ff83006d790 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff844054000 tx=0x7ff84404c070 comp rx=0 tx=0).stop 2024-04-04T06:17:34.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.903+0000 7ff85444cf40 1 -- 172.21.15.159:0/1399798875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591933c4bb0 msgr2=0x5591933b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:34.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.903+0000 7ff85444cf40 1 --2- 172.21.15.159:0/1399798875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591933c4bb0 0x5591933b8c30 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7ff84c0020e0 tx=0x7ff84c00b4b0 comp rx=0 tx=0).stop 2024-04-04T06:17:34.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.903+0000 7ff851800640 1 -- 172.21.15.159:0/1399798875 reap_dead start 2024-04-04T06:17:34.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.903+0000 7ff85444cf40 1 -- 172.21.15.159:0/1399798875 shutdown_connections 2024-04-04T06:17:34.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.903+0000 7ff85444cf40 1 -- 172.21.15.159:0/1399798875 >> 172.21.15.159:0/1399798875 conn(0x55919333a630 msgr2=0x55919331eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:34.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.903+0000 7ff85444cf40 1 -- 172.21.15.159:0/1399798875 shutdown_connections 2024-04-04T06:17:34.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:34.903+0000 7ff85444cf40 1 -- 172.21.15.159:0/1399798875 wait complete. 2024-04-04T06:17:34.910 INFO:teuthology.orchestra.run.smithi159.stdout:mytag123 2024-04-04T06:17:34.911 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-04T06:17:35.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.135+0000 7f6d14e00640 1 Processor -- start 2024-04-04T06:17:35.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.135+0000 7f6d14e00640 1 -- start start 2024-04-04T06:17:35.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.136+0000 7f6d14e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6d1015e460 0x7f6d100b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.142 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.136+0000 7f6d14e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d100d3e40 0x7f6d100d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.136+0000 7f6d14e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d100b8320 con 0x7f6d100cef20 2024-04-04T06:17:35.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.136+0000 7f6d14e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f6d100d2da0 con 0x7f6d1015e460 2024-04-04T06:17:35.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.136+0000 7f6d14e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6d100d3080 con 0x7f6d100d3e40 2024-04-04T06:17:35.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.137+0000 7f6d0fe00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f6d100cef20 0x7f6d1006c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:50018/0 (socket says 172.21.15.159:50018) 2024-04-04T06:17:35.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.137+0000 7f6d0fe00640 1 -- 172.21.15.159:0/1775557090 learned_addr learned my addr 172.21.15.159:0/1775557090 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:35.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.137+0000 7f6d06a00640 1 --2- 172.21.15.159:0/1775557090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d100d3e40 0x7f6d100d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.137+0000 7f6d0f400640 1 --2- 172.21.15.159:0/1775557090 >> v2:172.21.15.178:3300/0 conn(0x7f6d1015e460 0x7f6d100b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.137+0000 7f6d0ea00640 1 -- 172.21.15.159:0/1775557090 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 146527633 0 0) 0x7f6d100b8320 con 0x7f6d100cef20 2024-04-04T06:17:35.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.137+0000 7f6d0ea00640 1 -- 172.21.15.159:0/1775557090 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6cf4003520 con 0x7f6d100cef20 2024-04-04T06:17:35.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.138+0000 7f6d0ea00640 1 -- 172.21.15.159:0/1775557090 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4058920648 0 0) 0x7f6cf4003520 con 0x7f6d100cef20 2024-04-04T06:17:35.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.138+0000 7f6d0ea00640 1 -- 172.21.15.159:0/1775557090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d100d3e40 msgr2=0x7f6d100d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.138+0000 7f6d0ea00640 1 --2- 172.21.15.159:0/1775557090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d100d3e40 0x7f6d100d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.138+0000 7f6d0ea00640 1 -- 172.21.15.159:0/1775557090 >> v2:172.21.15.178:3300/0 conn(0x7f6d1015e460 msgr2=0x7f6d100b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.143 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.138+0000 7f6d0ea00640 1 --2- 172.21.15.159:0/1775557090 >> v2:172.21.15.178:3300/0 conn(0x7f6d1015e460 0x7f6d100b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.138+0000 7f6d0ea00640 1 -- 172.21.15.159:0/1775557090 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d100d3360 con 0x7f6d100cef20 2024-04-04T06:17:35.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.138+0000 7f6d0ea00640 1 -- 172.21.15.159:0/1775557090 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f6d080019f0 con 0x7f6d100cef20 2024-04-04T06:17:35.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.138+0000 7f6d0ea00640 1 -- 172.21.15.159:0/1775557090 >> v1:172.21.15.159:6789/0 conn(0x7f6d100cef20 legacy=0x7f6d1006c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.138+0000 7f6d0ea00640 1 --2- 172.21.15.159:0/1775557090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d100d3e40 0x7f6cf40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.138+0000 7f6d0ea00640 1 --2- 172.21.15.159:0/1775557090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf4003f80 0x7f6cf4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.138+0000 7f6d0ea00640 1 --2- 172.21.15.159:0/1775557090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cf4004d40 0x7f6cf40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.138+0000 7f6d0ea00640 1 -- 172.21.15.159:0/1775557090 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6d100d2da0 con 0x7f6cf4003f80 2024-04-04T06:17:35.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.138+0000 7f6d0ea00640 1 -- 172.21.15.159:0/1775557090 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6d100d3080 con 0x7f6cf4004d40 2024-04-04T06:17:35.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.138+0000 7f6d0ea00640 1 -- 172.21.15.159:0/1775557090 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf400b9c0 con 0x7f6d100d3e40 2024-04-04T06:17:35.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.138+0000 7f6d06a00640 1 --2- 172.21.15.159:0/1775557090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf4003f80 0x7f6cf4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.138+0000 7f6d0f400640 1 --2- 172.21.15.159:0/1775557090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d100d3e40 0x7f6cf40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.138+0000 7f6d0fe00640 1 --2- 172.21.15.159:0/1775557090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cf4004d40 0x7f6cf40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.139+0000 7f6d06a00640 1 -- 172.21.15.159:0/1775557090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d100d3e40 msgr2=0x7f6cf40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.144 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.139+0000 7f6d06a00640 1 --2- 172.21.15.159:0/1775557090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d100d3e40 0x7f6cf40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.139+0000 7f6d06a00640 1 -- 172.21.15.159:0/1775557090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cf4004d40 msgr2=0x7f6cf40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.139+0000 7f6d06a00640 1 --2- 172.21.15.159:0/1775557090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cf4004d40 0x7f6cf40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.139+0000 7f6d06a00640 1 -- 172.21.15.159:0/1775557090 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf4003520 con 0x7f6cf4003f80 2024-04-04T06:17:35.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.139+0000 7f6d0fe00640 1 --2- 172.21.15.159:0/1775557090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cf4004d40 0x7f6cf40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:35.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.139+0000 7f6d0f400640 1 --2- 172.21.15.159:0/1775557090 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d100d3e40 0x7f6cf40039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:35.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.140+0000 7f6d06a00640 1 --2- 172.21.15.159:0/1775557090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf4003f80 0x7f6cf4004800 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8009a80 tx=0x7f6cf4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07cb931d6d5e745 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.140+0000 7f6d0ea00640 1 -- 172.21.15.159:0/1775557090 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8009680 con 0x7f6cf4003f80 2024-04-04T06:17:35.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.140+0000 7f6d0ea00640 1 -- 172.21.15.159:0/1775557090 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cf8005990 con 0x7f6cf4003f80 2024-04-04T06:17:35.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.140+0000 7f6d0ea00640 1 -- 172.21.15.159:0/1775557090 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8005d80 con 0x7f6cf4003f80 2024-04-04T06:17:35.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.140+0000 7f6d14e00640 1 -- 172.21.15.159:0/1775557090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf4003f80 msgr2=0x7f6cf4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.140+0000 7f6d14e00640 1 --2- 172.21.15.159:0/1775557090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf4003f80 0x7f6cf4004800 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8009a80 tx=0x7f6cf4005830 comp rx=0 tx=0).stop 2024-04-04T06:17:35.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.140+0000 7f6d0fe00640 1 -- 172.21.15.159:0/1775557090 reap_dead start 2024-04-04T06:17:35.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.141+0000 7f6d14e00640 1 -- 172.21.15.159:0/1775557090 shutdown_connections 2024-04-04T06:17:35.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.141+0000 7f6d14e00640 1 -- 172.21.15.159:0/1775557090 >> 172.21.15.159:0/1775557090 conn(0x7f6d100678b0 msgr2=0x7f6d100d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:35.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.141+0000 7f6d14e00640 1 -- 172.21.15.159:0/1775557090 shutdown_connections 2024-04-04T06:17:35.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.141+0000 7f6d14e00640 1 -- 172.21.15.159:0/1775557090 wait complete. 2024-04-04T06:17:35.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.142+0000 7f6d14e00640 1 Processor -- start 2024-04-04T06:17:35.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.143+0000 7f6d14e00640 1 -- start start 2024-04-04T06:17:35.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.143+0000 7f6d14e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf4003f80 0x7f6d1015a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.144+0000 7f6d14e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d1015f1c0 0x7f6d1015f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.144+0000 7f6d14e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d101f3c70 0x7f6d101f40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.144+0000 7f6d14e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6d100cf370 con 0x7f6cf4003f80 2024-04-04T06:17:35.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.144+0000 7f6d14e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6d100d3920 con 0x7f6d1015f1c0 2024-04-04T06:17:35.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.144+0000 7f6d14e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6d100cf4f0 con 0x7f6d101f3c70 2024-04-04T06:17:35.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.144+0000 7f6d0f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf4003f80 0x7f6d1015a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.144+0000 7f6d0f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf4003f80 0x7f6d1015a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41770/0 (socket says 172.21.15.159:41770) 2024-04-04T06:17:35.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.144+0000 7f6d0f400640 1 -- 172.21.15.159:0/1714356910 learned_addr learned my addr 172.21.15.159:0/1714356910 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:35.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.144+0000 7f6d0f400640 1 -- 172.21.15.159:0/1714356910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d101f3c70 msgr2=0x7f6d101f40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.144+0000 7f6d0f400640 1 --2- 172.21.15.159:0/1714356910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d101f3c70 0x7f6d101f40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.144+0000 7f6d0f400640 1 -- 172.21.15.159:0/1714356910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d1015f1c0 msgr2=0x7f6d1015f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.144+0000 7f6d06a00640 1 --2- 172.21.15.159:0/1714356910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d1015f1c0 0x7f6d1015f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.144+0000 7f6d0f400640 1 --2- 172.21.15.159:0/1714356910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d1015f1c0 0x7f6d1015f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.144+0000 7f6d0f400640 1 -- 172.21.15.159:0/1714356910 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d10161710 con 0x7f6cf4003f80 2024-04-04T06:17:35.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.145+0000 7f6d0f400640 1 --2- 172.21.15.159:0/1714356910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf4003f80 0x7f6d1015a9c0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f6d00010a00 tx=0x7f6d00010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.145+0000 7f6d07e00640 1 -- 172.21.15.159:0/1714356910 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0000f040 con 0x7f6cf4003f80 2024-04-04T06:17:35.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.145+0000 7f6d07e00640 1 -- 172.21.15.159:0/1714356910 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d0000ec20 con 0x7f6cf4003f80 2024-04-04T06:17:35.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.145+0000 7f6d14e00640 1 -- 172.21.15.159:0/1714356910 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d101f4630 con 0x7f6cf4003f80 2024-04-04T06:17:35.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.145+0000 7f6d07e00640 1 -- 172.21.15.159:0/1714356910 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0001c400 con 0x7f6cf4003f80 2024-04-04T06:17:35.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.145+0000 7f6d14e00640 1 -- 172.21.15.159:0/1714356910 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d101f4ed0 con 0x7f6cf4003f80 2024-04-04T06:17:35.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.148+0000 7f6d07e00640 1 -- 172.21.15.159:0/1714356910 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d0001c5a0 con 0x7f6cf4003f80 2024-04-04T06:17:35.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.149+0000 7f6d07e00640 1 --2- 172.21.15.159:0/1714356910 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6cec06c920 0x7f6cec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.149+0000 7f6d07e00640 1 -- 172.21.15.159:0/1714356910 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f6d0008c340 con 0x7f6cf4003f80 2024-04-04T06:17:35.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.149+0000 7f6d06a00640 1 --2- 172.21.15.159:0/1714356910 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6cec06c920 0x7f6cec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.150+0000 7f6d191b3740 1 Processor -- start 2024-04-04T06:17:35.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.150+0000 7f6d191b3740 1 -- start start 2024-04-04T06:17:35.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.151+0000 7f6d191b3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a32eb23740 0x55a32eaf66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.151+0000 7f6d191b3740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a32eb23b20 0x55a32eb264d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.164 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.151+0000 7f6d191b3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eaf6c20 0x55a32eb26a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.151+0000 7f6d191b3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a32ea616d0 con 0x55a32eaf6c20 2024-04-04T06:17:35.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.151+0000 7f6d191b3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a32ea09770 con 0x55a32eb23740 2024-04-04T06:17:35.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.151+0000 7f6d191b3740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a32ea5d680 con 0x55a32eb23b20 2024-04-04T06:17:35.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.151+0000 7f6d06a00640 1 --2- 172.21.15.159:0/1714356910 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6cec06c920 0x7f6cec06ede0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8035000 tx=0x7f6cf8030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.151+0000 7f6d0fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eaf6c20 0x55a32eb26a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.151+0000 7f6d0fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eaf6c20 0x55a32eb26a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41778/0 (socket says 172.21.15.159:41778) 2024-04-04T06:17:35.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.151+0000 7f6d0fe00640 1 -- 172.21.15.159:0/1449082325 learned_addr learned my addr 172.21.15.159:0/1449082325 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:35.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.151+0000 7f6d0f400640 1 --2- 172.21.15.159:0/1449082325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a32eb23740 0x55a32eaf66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.151+0000 7f6d06a00640 1 --2- 172.21.15.159:0/1449082325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a32eb23b20 0x55a32eb264d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.151+0000 7f6d0fe00640 1 -- 172.21.15.159:0/1449082325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a32eb23b20 msgr2=0x55a32eb264d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.151+0000 7f6d0fe00640 1 --2- 172.21.15.159:0/1449082325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a32eb23b20 0x55a32eb264d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.151+0000 7f6d0fe00640 1 -- 172.21.15.159:0/1449082325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a32eb23740 msgr2=0x55a32eaf66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.151+0000 7f6d0fe00640 1 --2- 172.21.15.159:0/1449082325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a32eb23740 0x55a32eaf66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.151+0000 7f6d0fe00640 1 -- 172.21.15.159:0/1449082325 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a32e83f7d0 con 0x55a32eaf6c20 2024-04-04T06:17:35.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.152+0000 7f6d0fe00640 1 --2- 172.21.15.159:0/1449082325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eaf6c20 0x55a32eb26a10 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f6d100d0350 tx=0x7f6d0800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad009f02d38f3e84 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.152+0000 7f6cffe00640 1 -- 172.21.15.159:0/1449082325 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0800db50 con 0x55a32eaf6c20 2024-04-04T06:17:35.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.152+0000 7f6cffe00640 1 -- 172.21.15.159:0/1449082325 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d0800dcf0 con 0x55a32eaf6c20 2024-04-04T06:17:35.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.152+0000 7f6cffe00640 1 -- 172.21.15.159:0/1449082325 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0800dea0 con 0x55a32eaf6c20 2024-04-04T06:17:35.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.153+0000 7f6d191b3740 1 -- 172.21.15.159:0/1449082325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eaf6c20 msgr2=0x55a32eb26a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.153+0000 7f6d191b3740 1 --2- 172.21.15.159:0/1449082325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eaf6c20 0x55a32eb26a10 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f6d100d0350 tx=0x7f6d0800d520 comp rx=0 tx=0).stop 2024-04-04T06:17:35.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.153+0000 7f6d191b3740 1 -- 172.21.15.159:0/1449082325 shutdown_connections 2024-04-04T06:17:35.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.153+0000 7f6d191b3740 1 --2- 172.21.15.159:0/1449082325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eaf6c20 0x55a32eb26a10 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.153+0000 7f6d191b3740 1 --2- 172.21.15.159:0/1449082325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a32eb23b20 0x55a32eb264d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.153+0000 7f6d191b3740 1 --2- 172.21.15.159:0/1449082325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a32eb23740 0x55a32eaf66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.153+0000 7f6d191b3740 1 -- 172.21.15.159:0/1449082325 >> 172.21.15.159:0/1449082325 conn(0x55a32eaf8950 msgr2=0x55a32eb414d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:35.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.153+0000 7f6d191b3740 1 -- 172.21.15.159:0/1449082325 shutdown_connections 2024-04-04T06:17:35.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.153+0000 7f6d191b3740 1 -- 172.21.15.159:0/1449082325 wait complete. 2024-04-04T06:17:35.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.154+0000 7f6d191b3740 1 -- start start 2024-04-04T06:17:35.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.154+0000 7f6d191b3740 1 Processor -- start 2024-04-04T06:17:35.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.155+0000 7f6d191b3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a32eb23740 0x55a32eb26580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.155+0000 7f6d191b3740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a32eb23b20 0x55a32eb52560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.155+0000 7f6d191b3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eb24f20 0x55a32eb1bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.155+0000 7f6d191b3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a32eafaff0 con 0x55a32eb24f20 2024-04-04T06:17:35.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.155+0000 7f6d191b3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a32ea1caa0 con 0x55a32eb23740 2024-04-04T06:17:35.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.155+0000 7f6d191b3740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a32eafd830 con 0x55a32eb23b20 2024-04-04T06:17:35.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.155+0000 7f6d191b3740 10 client.0 _ll_register_callbacks cb 0x55a32ea1c490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:17:35.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.155+0000 7f6d0fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eb24f20 0x55a32eb1bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.155+0000 7f6d0fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eb24f20 0x55a32eb1bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41786/0 (socket says 172.21.15.159:41786) 2024-04-04T06:17:35.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.155+0000 7f6d0fe00640 1 -- 172.21.15.159:0/1149163432 learned_addr learned my addr 172.21.15.159:0/1149163432 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:35.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.155+0000 7f6d0f400640 1 --2- 172.21.15.159:0/1149163432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a32eb23740 0x55a32eb26580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.155+0000 7f6d06a00640 1 --2- 172.21.15.159:0/1149163432 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a32eb23b20 0x55a32eb52560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.155+0000 7f6d0fe00640 1 -- 172.21.15.159:0/1149163432 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a32eb23b20 msgr2=0x55a32eb52560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.155+0000 7f6d0fe00640 1 --2- 172.21.15.159:0/1149163432 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a32eb23b20 0x55a32eb52560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.155+0000 7f6d0fe00640 1 -- 172.21.15.159:0/1149163432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a32eb23740 msgr2=0x55a32eb26580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.155+0000 7f6d0fe00640 1 --2- 172.21.15.159:0/1149163432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a32eb23740 0x55a32eb26580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.155+0000 7f6d0fe00640 1 -- 172.21.15.159:0/1149163432 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a32e7753f0 con 0x55a32eb24f20 2024-04-04T06:17:35.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.155+0000 7f6d0f400640 1 --2- 172.21.15.159:0/1149163432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a32eb23740 0x55a32eb26580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:35.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.156+0000 7f6d0fe00640 1 --2- 172.21.15.159:0/1149163432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eb24f20 0x55a32eb1bb20 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800d560 tx=0x7f6d080012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedc3389fa09e891 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.156+0000 7f6cfcc00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:17:35.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.156+0000 7f6cfcc00640 1 -- 172.21.15.159:0/1149163432 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0800dea0 con 0x55a32eb24f20 2024-04-04T06:17:35.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.156+0000 7f6cfcc00640 1 -- 172.21.15.159:0/1149163432 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f6d08014630 con 0x55a32eb24f20 2024-04-04T06:17:35.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.156+0000 7f6d191b3740 1 -- 172.21.15.159:0/1149163432 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d0800dcf0 con 0x55a32eb24f20 2024-04-04T06:17:35.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.156+0000 7f6cfcc00640 1 -- 172.21.15.159:0/1149163432 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f6d0800db50 con 0x55a32eb24f20 2024-04-04T06:17:35.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.156+0000 7f6d191b3740 10 client.9294 fetch_fsmap learned FSMap version 107 2024-04-04T06:17:35.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.156+0000 7f6d191b3740 1 -- 172.21.15.159:0/1149163432 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x55a32e7350f0 con 0x55a32eb24f20 2024-04-04T06:17:35.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.157+0000 7f6cfcc00640 1 -- 172.21.15.159:0/1149163432 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d0800dcf0 con 0x55a32eb24f20 2024-04-04T06:17:35.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.157+0000 7f6cfcc00640 1 -- 172.21.15.159:0/1149163432 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 107) ==== 3494+0+0 (secure 0 0 0) 0x7f6d08014830 con 0x55a32eb24f20 2024-04-04T06:17:35.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.157+0000 7f6d191b3740 10 client.9294 fetch_fsmap finished waiting for FSMap version 107 2024-04-04T06:17:35.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.157+0000 7f6d191b3740 10 client.9294 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-04T06:17:35.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.157+0000 7f6d191b3740 20 client.9294 populate_metadata read hostname 'smithi159' 2024-04-04T06:17:35.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.157+0000 7f6d191b3740 1 --2- 172.21.15.159:0/1149163432 >> [v2:172.21.15.159:6832/2439261212,v1:172.21.15.159:6834/2439261212] conn(0x55a32eb52d40 0x55a32eb53170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.157+0000 7f6d191b3740 4 client.9294 mds_command: new command op to 9135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:17:35.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.157+0000 7f6d191b3740 1 -- 172.21.15.159:0/1149163432 --> [v2:172.21.15.159:6832/2439261212,v1:172.21.15.159:6834/2439261212] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a32ea6ec10 con 0x55a32eb52d40 2024-04-04T06:17:35.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.157+0000 7f6d0f400640 1 --2- 172.21.15.159:0/1149163432 >> [v2:172.21.15.159:6832/2439261212,v1:172.21.15.159:6834/2439261212] conn(0x55a32eb52d40 0x55a32eb53170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.158+0000 7f6d0f400640 1 --2- 172.21.15.159:0/1149163432 >> [v2:172.21.15.159:6832/2439261212,v1:172.21.15.159:6834/2439261212] conn(0x55a32eb52d40 0x55a32eb53170 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=572707882a2466f4 server_cookie=f3115d9ba5ef6ce9 in_seq=0 out_seq=0 2024-04-04T06:17:35.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.158+0000 7f6cfcc00640 10 client.9294 ms_handle_connect on v2:172.21.15.159:6832/2439261212 2024-04-04T06:17:35.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.159+0000 7f6cfcc00640 1 -- 172.21.15.159:0/1149163432 <== mds.0 v2:172.21.15.159:6832/2439261212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a32ea6ec10 con 0x55a32eb52d40 2024-04-04T06:17:35.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.159+0000 7f6cfcc00640 10 client.9294 handle_command_reply: tid=0 2024-04-04T06:17:35.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.159+0000 7f6d191b3740 1 client.9294 shutdown 2024-04-04T06:17:35.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.159+0000 7f6d191b3740 1 -- 172.21.15.159:0/1149163432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eb24f20 msgr2=0x55a32eb1bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.159+0000 7f6d191b3740 1 --2- 172.21.15.159:0/1149163432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eb24f20 0x55a32eb1bb20 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800d560 tx=0x7f6d080012d0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.159+0000 7f6d191b3740 1 -- 172.21.15.159:0/1149163432 shutdown_connections 2024-04-04T06:17:35.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.159+0000 7f6d191b3740 1 --2- 172.21.15.159:0/1149163432 >> [v2:172.21.15.159:6832/2439261212,v1:172.21.15.159:6834/2439261212] conn(0x55a32eb52d40 0x55a32eb53170 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.159+0000 7f6d191b3740 1 --2- 172.21.15.159:0/1149163432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eb24f20 0x55a32eb1bb20 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.159+0000 7f6d191b3740 1 --2- 172.21.15.159:0/1149163432 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a32eb23b20 0x55a32eb52560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.159+0000 7f6d191b3740 1 --2- 172.21.15.159:0/1149163432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a32eb23740 0x55a32eb26580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.159+0000 7f6cfcc00640 0 client.9294 ms_handle_reset on v2:172.21.15.159:6832/2439261212 2024-04-04T06:17:35.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.159+0000 7f6d191b3740 1 -- 172.21.15.159:0/1149163432 >> 172.21.15.159:0/1149163432 conn(0x55a32eaf6c20 msgr2=0x55a32eb44950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:35.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.159+0000 7f6d191b3740 1 -- 172.21.15.159:0/1149163432 shutdown_connections 2024-04-04T06:17:35.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.159+0000 7f6d191b3740 1 -- 172.21.15.159:0/1149163432 wait complete. 2024-04-04T06:17:35.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.159+0000 7f6d191b3740 20 client.9294 trim_cache size 0 max 16384 2024-04-04T06:17:35.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.177+0000 7f6d191b3740 1 Processor -- start 2024-04-04T06:17:35.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.177+0000 7f6d191b3740 1 -- start start 2024-04-04T06:17:35.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.178+0000 7f6d191b3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eb23740 0x55a32eb524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.178+0000 7f6d191b3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a32eb23b20 0x55a32eaf1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.178+0000 7f6d191b3740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a32eb24f20 0x55a32eb44f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.178+0000 7f6d191b3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6d10201100 con 0x55a32eb23740 2024-04-04T06:17:35.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.178+0000 7f6d191b3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6d10200100 con 0x55a32eb23b20 2024-04-04T06:17:35.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.178+0000 7f6d191b3740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6d101ff5b0 con 0x55a32eb24f20 2024-04-04T06:17:35.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.178+0000 7f6d0f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eb23740 0x55a32eb524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.178+0000 7f6d0f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eb23740 0x55a32eb524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41800/0 (socket says 172.21.15.159:41800) 2024-04-04T06:17:35.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.178+0000 7f6d0f400640 1 -- 172.21.15.159:0/1053689176 learned_addr learned my addr 172.21.15.159:0/1053689176 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:35.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.178+0000 7f6d0f400640 1 -- 172.21.15.159:0/1053689176 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a32eb24f20 msgr2=0x55a32eb44f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.178+0000 7f6d0f400640 1 --2- 172.21.15.159:0/1053689176 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a32eb24f20 0x55a32eb44f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.178+0000 7f6d0f400640 1 -- 172.21.15.159:0/1053689176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a32eb23b20 msgr2=0x55a32eaf1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.178+0000 7f6d06a00640 1 --2- 172.21.15.159:0/1053689176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a32eb23b20 0x55a32eaf1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.179+0000 7f6d0f400640 1 --2- 172.21.15.159:0/1053689176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a32eb23b20 0x55a32eaf1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.179+0000 7f6d0f400640 1 -- 172.21.15.159:0/1053689176 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a32eb28d20 con 0x55a32eb23740 2024-04-04T06:17:35.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.179+0000 7f6d0f400640 1 --2- 172.21.15.159:0/1053689176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eb23740 0x55a32eb524a0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f6d0005a190 tx=0x7f6d0008c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793e09a1cade7370 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.179+0000 7f6cff400640 1 -- 172.21.15.159:0/1053689176 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0008a850 con 0x55a32eb23740 2024-04-04T06:17:35.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.179+0000 7f6cff400640 1 -- 172.21.15.159:0/1053689176 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d000582d0 con 0x55a32eb23740 2024-04-04T06:17:35.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.179+0000 7f6cff400640 1 -- 172.21.15.159:0/1053689176 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00056640 con 0x55a32eb23740 2024-04-04T06:17:35.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.180+0000 7f6d191b3740 1 -- 172.21.15.159:0/1053689176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eb23740 msgr2=0x55a32eb524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.180+0000 7f6d191b3740 1 --2- 172.21.15.159:0/1053689176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eb23740 0x55a32eb524a0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f6d0005a190 tx=0x7f6d0008c040 comp rx=0 tx=0).stop 2024-04-04T06:17:35.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.180+0000 7f6d191b3740 1 -- 172.21.15.159:0/1053689176 shutdown_connections 2024-04-04T06:17:35.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.180+0000 7f6d191b3740 1 --2- 172.21.15.159:0/1053689176 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a32eb24f20 0x55a32eb44f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.180+0000 7f6d191b3740 1 --2- 172.21.15.159:0/1053689176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a32eb23b20 0x55a32eaf1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.180+0000 7f6d191b3740 1 --2- 172.21.15.159:0/1053689176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eb23740 0x55a32eb524a0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.180+0000 7f6d191b3740 1 -- 172.21.15.159:0/1053689176 >> 172.21.15.159:0/1053689176 conn(0x55a32eaf6c20 msgr2=0x55a32eb29d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:35.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.180+0000 7f6d191b3740 1 -- 172.21.15.159:0/1053689176 shutdown_connections 2024-04-04T06:17:35.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.180+0000 7f6d191b3740 1 -- 172.21.15.159:0/1053689176 wait complete. 2024-04-04T06:17:35.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.181+0000 7f6d191b3740 1 -- start start 2024-04-04T06:17:35.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.181+0000 7f6d191b3740 1 Processor -- start 2024-04-04T06:17:35.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.183+0000 7f6d191b3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eb23740 0x55a32eb1ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.183+0000 7f6d191b3740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a32eb23b20 0x55a32eb4fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.183+0000 7f6d191b3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a32eb24f20 0x55a32eb187c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.183+0000 7f6d191b3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a32eb449d0 con 0x55a32eb23740 2024-04-04T06:17:35.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.183+0000 7f6d191b3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6d101ffb00 con 0x55a32eb24f20 2024-04-04T06:17:35.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.183+0000 7f6d191b3740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55a32eb1b700 con 0x55a32eb23b20 2024-04-04T06:17:35.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.183+0000 7f6d191b3740 10 client.0 _ll_register_callbacks cb 0x7f6d08007430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:17:35.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.183+0000 7f6d0f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eb23740 0x55a32eb1ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.183+0000 7f6d0f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eb23740 0x55a32eb1ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41816/0 (socket says 172.21.15.159:41816) 2024-04-04T06:17:35.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.183+0000 7f6d0f400640 1 -- 172.21.15.159:0/124732529 learned_addr learned my addr 172.21.15.159:0/124732529 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:35.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.184+0000 7f6d06a00640 1 --2- 172.21.15.159:0/124732529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a32eb23b20 0x55a32eb4fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.184+0000 7f6d0f400640 1 -- 172.21.15.159:0/124732529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a32eb23b20 msgr2=0x55a32eb4fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.184+0000 7f6d0f400640 1 --2- 172.21.15.159:0/124732529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a32eb23b20 0x55a32eb4fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.184+0000 7f6d0fe00640 1 --2- 172.21.15.159:0/124732529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a32eb24f20 0x55a32eb187c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.184+0000 7f6d0f400640 1 -- 172.21.15.159:0/124732529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a32eb24f20 msgr2=0x55a32eb187c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.184+0000 7f6d0f400640 1 --2- 172.21.15.159:0/124732529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a32eb24f20 0x55a32eb187c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.184+0000 7f6d0f400640 1 -- 172.21.15.159:0/124732529 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a32eaf6130 con 0x55a32eb23740 2024-04-04T06:17:35.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.184+0000 7f6d0f400640 1 --2- 172.21.15.159:0/124732529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eb23740 0x55a32eb1ee10 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f6d00059cd0 tx=0x7f6d00070040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f597aa4f9a09690 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.184+0000 7f6cfcc00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:17:35.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.184+0000 7f6cfcc00640 1 -- 172.21.15.159:0/124732529 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00056640 con 0x55a32eb23740 2024-04-04T06:17:35.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.184+0000 7f6cfcc00640 1 -- 172.21.15.159:0/124732529 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f6d0005a230 con 0x55a32eb23740 2024-04-04T06:17:35.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.184+0000 7f6d191b3740 1 -- 172.21.15.159:0/124732529 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d000582d0 con 0x55a32eb23740 2024-04-04T06:17:35.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.185+0000 7f6cfcc00640 1 -- 172.21.15.159:0/124732529 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f6d000567e0 con 0x55a32eb23740 2024-04-04T06:17:35.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.185+0000 7f6d191b3740 10 client.9300 fetch_fsmap learned FSMap version 107 2024-04-04T06:17:35.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.185+0000 7f6d191b3740 1 -- 172.21.15.159:0/124732529 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x55a32eb50010 con 0x55a32eb23740 2024-04-04T06:17:35.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.185+0000 7f6cfcc00640 1 -- 172.21.15.159:0/124732529 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d000582d0 con 0x55a32eb23740 2024-04-04T06:17:35.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.185+0000 7f6cfcc00640 1 -- 172.21.15.159:0/124732529 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 107) ==== 3494+0+0 (secure 0 0 0) 0x7f6d00062720 con 0x55a32eb23740 2024-04-04T06:17:35.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.185+0000 7f6d191b3740 10 client.9300 fetch_fsmap finished waiting for FSMap version 107 2024-04-04T06:17:35.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.185+0000 7f6d191b3740 10 client.9300 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-04T06:17:35.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.185+0000 7f6d191b3740 20 client.9300 populate_metadata read hostname 'smithi159' 2024-04-04T06:17:35.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.185+0000 7f6d191b3740 1 --2- 172.21.15.159:0/124732529 >> [v2:172.21.15.159:6832/2439261212,v1:172.21.15.159:6834/2439261212] conn(0x55a32ebbb9a0 0x55a32eb437c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.185+0000 7f6d191b3740 4 client.9300 mds_command: new command op to 9135 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-04T06:17:35.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.185+0000 7f6d191b3740 1 -- 172.21.15.159:0/124732529 --> [v2:172.21.15.159:6832/2439261212,v1:172.21.15.159:6834/2439261212] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a32eb29b40 con 0x55a32ebbb9a0 2024-04-04T06:17:35.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.185+0000 7f6d06a00640 1 --2- 172.21.15.159:0/124732529 >> [v2:172.21.15.159:6832/2439261212,v1:172.21.15.159:6834/2439261212] conn(0x55a32ebbb9a0 0x55a32eb437c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.186+0000 7f6cfcc00640 10 client.9300 ms_handle_connect on v2:172.21.15.159:6832/2439261212 2024-04-04T06:17:35.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.186+0000 7f6d06a00640 1 --2- 172.21.15.159:0/124732529 >> [v2:172.21.15.159:6832/2439261212,v1:172.21.15.159:6834/2439261212] conn(0x55a32ebbb9a0 0x55a32eb437c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=528d6c03bb28a06f server_cookie=4a10459dbc550a98 in_seq=0 out_seq=0 2024-04-04T06:17:35.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.199+0000 7f6cfcc00640 1 -- 172.21.15.159:0/124732529 <== mds.0 v2:172.21.15.159:6832/2439261212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a32eb29b40 con 0x55a32ebbb9a0 2024-04-04T06:17:35.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.199+0000 7f6cfcc00640 10 client.9300 handle_command_reply: tid=0 2024-04-04T06:17:35.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.199+0000 7f6d191b3740 1 client.9300 shutdown 2024-04-04T06:17:35.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.199+0000 7f6d191b3740 1 -- 172.21.15.159:0/124732529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eb23740 msgr2=0x55a32eb1ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.199+0000 7f6d191b3740 1 --2- 172.21.15.159:0/124732529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eb23740 0x55a32eb1ee10 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f6d00059cd0 tx=0x7f6d00070040 comp rx=0 tx=0).stop 2024-04-04T06:17:35.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.199+0000 7f6d191b3740 1 -- 172.21.15.159:0/124732529 shutdown_connections 2024-04-04T06:17:35.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.199+0000 7f6d191b3740 1 --2- 172.21.15.159:0/124732529 >> [v2:172.21.15.159:6832/2439261212,v1:172.21.15.159:6834/2439261212] conn(0x55a32ebbb9a0 0x55a32eb437c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.199+0000 7f6d191b3740 1 --2- 172.21.15.159:0/124732529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a32eb24f20 0x55a32eb187c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.199+0000 7f6d191b3740 1 --2- 172.21.15.159:0/124732529 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55a32eb23b20 0x55a32eb4fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.199+0000 7f6cfcc00640 0 client.9300 ms_handle_reset on v2:172.21.15.159:6832/2439261212 2024-04-04T06:17:35.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.199+0000 7f6d191b3740 1 --2- 172.21.15.159:0/124732529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32eb23740 0x55a32eb1ee10 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.199+0000 7f6d191b3740 1 -- 172.21.15.159:0/124732529 >> 172.21.15.159:0/124732529 conn(0x55a32eaf6c20 msgr2=0x55a32eafa240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:35.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.200+0000 7f6d191b3740 1 -- 172.21.15.159:0/124732529 shutdown_connections 2024-04-04T06:17:35.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.200+0000 7f6d191b3740 1 -- 172.21.15.159:0/124732529 wait complete. 2024-04-04T06:17:35.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.200+0000 7f6d191b3740 20 client.9300 trim_cache size 0 max 16384 2024-04-04T06:17:35.198 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:17:35.198 INFO:teuthology.orchestra.run.smithi159.stdout: "message": "", 2024-04-04T06:17:35.198 INFO:teuthology.orchestra.run.smithi159.stdout: "return_code": 0 2024-04-04T06:17:35.198 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:17:35.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.201+0000 7f6d14e00640 1 -- 172.21.15.159:0/1714356910 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6cec06c920 msgr2=0x7f6cec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.201+0000 7f6d14e00640 1 --2- 172.21.15.159:0/1714356910 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6cec06c920 0x7f6cec06ede0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8035000 tx=0x7f6cf8030000 comp rx=0 tx=0).stop 2024-04-04T06:17:35.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.201+0000 7f6d14e00640 1 -- 172.21.15.159:0/1714356910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf4003f80 msgr2=0x7f6d1015a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.201+0000 7f6d14e00640 1 --2- 172.21.15.159:0/1714356910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf4003f80 0x7f6d1015a9c0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f6d00010a00 tx=0x7f6d00010ec0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.201+0000 7f6d14e00640 1 -- 172.21.15.159:0/1714356910 shutdown_connections 2024-04-04T06:17:35.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.201+0000 7f6d14e00640 1 --2- 172.21.15.159:0/1714356910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6d101f3c70 0x7f6d101f40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.201+0000 7f6d14e00640 1 --2- 172.21.15.159:0/1714356910 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6cec06c920 0x7f6cec06ede0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.201+0000 7f6d14e00640 1 --2- 172.21.15.159:0/1714356910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d1015f1c0 0x7f6d1015f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.201+0000 7f6d14e00640 1 --2- 172.21.15.159:0/1714356910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6cf4003f80 0x7f6d1015a9c0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.201+0000 7f6d14e00640 1 -- 172.21.15.159:0/1714356910 >> 172.21.15.159:0/1714356910 conn(0x7f6d100678b0 msgr2=0x7f6d100b5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:35.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.201+0000 7f6d14e00640 1 -- 172.21.15.159:0/1714356910 shutdown_connections 2024-04-04T06:17:35.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.201+0000 7f6d14e00640 1 -- 172.21.15.159:0/1714356910 wait complete. 2024-04-04T06:17:35.212 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-04T06:17:35.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.407+0000 7fa4f9a00640 1 Processor -- start 2024-04-04T06:17:35.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.407+0000 7fa4f9a00640 1 -- start start 2024-04-04T06:17:35.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.408+0000 7fa4f9a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa4f415fc50 0x7fa4f40cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.404 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.408+0000 7fa4f9a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4f40b3150 con 0x7fa4f40cc490 2024-04-04T06:17:35.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.408+0000 7fa4f9a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4f41600e0 con 0x7fa4f40cc0b0 2024-04-04T06:17:35.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.408+0000 7fa4f9a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fa4f415c610 con 0x7fa4f415fc50 2024-04-04T06:17:35.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.409+0000 7fa4f3400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa4f415fc50 0x7fa4f40cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.409+0000 7fa4f9000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fa4f40cc490 0x7fa4f415ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:50024/0 (socket says 172.21.15.159:50024) 2024-04-04T06:17:35.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.409+0000 7fa4f9000640 1 -- 172.21.15.159:0/2176126158 learned_addr learned my addr 172.21.15.159:0/2176126158 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:35.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.409+0000 7fa4f3400640 1 -- 172.21.15.159:0/2176126158 >> v1:172.21.15.178:6789/0 conn(0x7fa4f40cc0b0 legacy=0x7fa4f406dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.409+0000 7fa4f3400640 1 -- 172.21.15.159:0/2176126158 >> v1:172.21.15.159:6789/0 conn(0x7fa4f40cc490 legacy=0x7fa4f415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.409+0000 7fa4f3400640 1 -- 172.21.15.159:0/2176126158 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f41603c0 con 0x7fa4f415fc50 2024-04-04T06:17:35.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.411+0000 7fa4f3400640 1 --2- 172.21.15.159:0/2176126158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4f415fc50 0x7fa4f40cfdc0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00097a0 tx=0x7fa4e002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7418ede256e12 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.411+0000 7fa4f2a00640 1 -- 172.21.15.159:0/2176126158 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0035020 con 0x7fa4f415fc50 2024-04-04T06:17:35.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.411+0000 7fa4f2a00640 1 -- 172.21.15.159:0/2176126158 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e0031830 con 0x7fa4f415fc50 2024-04-04T06:17:35.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.411+0000 7fa4f2a00640 1 -- 172.21.15.159:0/2176126158 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0031c00 con 0x7fa4f415fc50 2024-04-04T06:17:35.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.412+0000 7fa4f9a00640 1 -- 172.21.15.159:0/2176126158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4f415fc50 msgr2=0x7fa4f40cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.412+0000 7fa4f9a00640 1 --2- 172.21.15.159:0/2176126158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4f415fc50 0x7fa4f40cfdc0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00097a0 tx=0x7fa4e002b8d0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.412+0000 7fa4f9a00640 1 -- 172.21.15.159:0/2176126158 shutdown_connections 2024-04-04T06:17:35.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.412+0000 7fa4f9a00640 1 --2- 172.21.15.159:0/2176126158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4f415fc50 0x7fa4f40cfdc0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.412+0000 7fa4f9a00640 1 -- 172.21.15.159:0/2176126158 >> 172.21.15.159:0/2176126158 conn(0x7fa4f40ccad0 msgr2=0x7fa4f40cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:35.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.412+0000 7fa4f9a00640 1 -- 172.21.15.159:0/2176126158 shutdown_connections 2024-04-04T06:17:35.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.412+0000 7fa4f9a00640 1 -- 172.21.15.159:0/2176126158 wait complete. 2024-04-04T06:17:35.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.414+0000 7fa4f9a00640 1 Processor -- start 2024-04-04T06:17:35.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.414+0000 7fa4f9a00640 1 -- start start 2024-04-04T06:17:35.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.415+0000 7fa4f9a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4f40cc0b0 0x7fa4f4132500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.415+0000 7fa4f9a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4f40cc490 0x7fa4f41547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.415+0000 7fa4f9a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4f41f46d0 0x7fa4f40ce380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.415+0000 7fa4f9a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f40d0300 con 0x7fa4f40cc0b0 2024-04-04T06:17:35.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.415+0000 7fa4f9a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f4160980 con 0x7fa4f41f46d0 2024-04-04T06:17:35.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.415+0000 7fa4f9a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f40d0480 con 0x7fa4f40cc490 2024-04-04T06:17:35.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.415+0000 7fa4f3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4f40cc0b0 0x7fa4f4132500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.415+0000 7fa4f3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4f40cc0b0 0x7fa4f4132500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41828/0 (socket says 172.21.15.159:41828) 2024-04-04T06:17:35.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.415+0000 7fa4f3e00640 1 -- 172.21.15.159:0/2617777963 learned_addr learned my addr 172.21.15.159:0/2617777963 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:35.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.415+0000 7fa4f3400640 1 --2- 172.21.15.159:0/2617777963 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4f40cc490 0x7fa4f41547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.415+0000 7fa4f9000640 1 --2- 172.21.15.159:0/2617777963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4f41f46d0 0x7fa4f40ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.416+0000 7fa4f3e00640 1 -- 172.21.15.159:0/2617777963 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4f40cc490 msgr2=0x7fa4f41547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.416+0000 7fa4f3e00640 1 --2- 172.21.15.159:0/2617777963 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4f40cc490 0x7fa4f41547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.416+0000 7fa4f3e00640 1 -- 172.21.15.159:0/2617777963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4f41f46d0 msgr2=0x7fa4f40ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.416+0000 7fa4f3e00640 1 --2- 172.21.15.159:0/2617777963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4f41f46d0 0x7fa4f40ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.416+0000 7fa4f3e00640 1 -- 172.21.15.159:0/2617777963 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f4155250 con 0x7fa4f40cc0b0 2024-04-04T06:17:35.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.416+0000 7fa4f3e00640 1 --2- 172.21.15.159:0/2617777963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4f40cc0b0 0x7fa4f4132500 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400c460 tx=0x7fa4e400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.416+0000 7fa4ebe00640 1 -- 172.21.15.159:0/2617777963 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4016020 con 0x7fa4f40cc0b0 2024-04-04T06:17:35.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.416+0000 7fa4ebe00640 1 -- 172.21.15.159:0/2617777963 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e4012870 con 0x7fa4f40cc0b0 2024-04-04T06:17:35.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.416+0000 7fa4ebe00640 1 -- 172.21.15.159:0/2617777963 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4012c60 con 0x7fa4f40cc0b0 2024-04-04T06:17:35.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.416+0000 7fa4f9a00640 1 -- 172.21.15.159:0/2617777963 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f4155530 con 0x7fa4f40cc0b0 2024-04-04T06:17:35.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.416+0000 7fa4f9a00640 1 -- 172.21.15.159:0/2617777963 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f40afbb0 con 0x7fa4f40cc0b0 2024-04-04T06:17:35.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.417+0000 7fa4f9a00640 1 -- 172.21.15.159:0/2617777963 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f4075940 con 0x7fa4f40cc0b0 2024-04-04T06:17:35.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.419+0000 7fa4ebe00640 1 -- 172.21.15.159:0/2617777963 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4e4019020 con 0x7fa4f40cc0b0 2024-04-04T06:17:35.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.420+0000 7fa4ebe00640 1 --2- 172.21.15.159:0/2617777963 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa4d406c8f0 0x7fa4d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.420+0000 7fa4ebe00640 1 -- 172.21.15.159:0/2617777963 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7fa4e408b920 con 0x7fa4f40cc0b0 2024-04-04T06:17:35.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.421+0000 7fa4f3400640 1 --2- 172.21.15.159:0/2617777963 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa4d406c8f0 0x7fa4d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.422+0000 7fa4f3400640 1 --2- 172.21.15.159:0/2617777963 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa4d406c8f0 0x7fa4d406edb0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0036000 tx=0x7fa4e003c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.424+0000 7fa4ebe00640 1 -- 172.21.15.159:0/2617777963 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa4e4020030 con 0x7fa4f40cc0b0 2024-04-04T06:17:35.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.665+0000 7fa4f9a00640 1 -- 172.21.15.159:0/2617777963 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa4f40b1a80 con 0x7fa4f40cc0b0 2024-04-04T06:17:35.661 INFO:tasks.ceph.mon.a.smithi159.stderr:2024-04-04T06:17:35.666+0000 7f1995c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-04T06:17:35.664 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.669+0000 7fa4ebe00640 1 -- 172.21.15.159:0/2617777963 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v108) ==== 161+0+0 (secure 0 0 0) 0x7fa4e4059cc0 con 0x7fa4f40cc0b0 2024-04-04T06:17:35.664 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-04T06:17:35.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.673+0000 7fa4f9a00640 1 -- 172.21.15.159:0/2617777963 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa4d406c8f0 msgr2=0x7fa4d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.673+0000 7fa4f9a00640 1 --2- 172.21.15.159:0/2617777963 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa4d406c8f0 0x7fa4d406edb0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0036000 tx=0x7fa4e003c040 comp rx=0 tx=0).stop 2024-04-04T06:17:35.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.673+0000 7fa4f9a00640 1 -- 172.21.15.159:0/2617777963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4f40cc0b0 msgr2=0x7fa4f4132500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.673+0000 7fa4f9a00640 1 --2- 172.21.15.159:0/2617777963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4f40cc0b0 0x7fa4f4132500 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400c460 tx=0x7fa4e400c920 comp rx=0 tx=0).stop 2024-04-04T06:17:35.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.674+0000 7fa4f9a00640 1 -- 172.21.15.159:0/2617777963 shutdown_connections 2024-04-04T06:17:35.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.674+0000 7fa4f9a00640 1 --2- 172.21.15.159:0/2617777963 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa4d406c8f0 0x7fa4d406edb0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.674+0000 7fa4f9a00640 1 --2- 172.21.15.159:0/2617777963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4f41f46d0 0x7fa4f40ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.674+0000 7fa4f9a00640 1 --2- 172.21.15.159:0/2617777963 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa4f40cc490 0x7fa4f41547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.674+0000 7fa4f9a00640 1 --2- 172.21.15.159:0/2617777963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4f40cc0b0 0x7fa4f4132500 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.674+0000 7fa4f9a00640 1 -- 172.21.15.159:0/2617777963 >> 172.21.15.159:0/2617777963 conn(0x7fa4f40ccad0 msgr2=0x7fa4f4071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:35.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.674+0000 7fa4f9a00640 1 -- 172.21.15.159:0/2617777963 shutdown_connections 2024-04-04T06:17:35.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.674+0000 7fa4f9a00640 1 -- 172.21.15.159:0/2617777963 wait complete. 2024-04-04T06:17:35.682 INFO:tasks.ceph.mds.a.smithi159.stderr: -1952> 2024-04-04T06:17:27.615+0000 7fc6eee00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:17:35.691 DEBUG:teuthology.orchestra.run.smithi159:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-04-04T06:17:35.743 INFO:tasks.ceph.mds.a.smithi159.stdout:starting mds.a at 2024-04-04T06:17:35.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.783+0000 7fadf0760280 1 -- start start 2024-04-04T06:17:35.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.783+0000 7fadf0760280 1 Processor -- start 2024-04-04T06:17:35.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.784+0000 7fadf0760280 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ba4838400 0x562ba4831600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.784+0000 7fadf0760280 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ba3ad5c00 0x562ba4830b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.784+0000 7fadf0760280 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba4838800 0x562ba4831b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.784+0000 7fadf0760280 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562ba48ae480 con 0x562ba4838800 2024-04-04T06:17:35.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.784+0000 7fadf0760280 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562ba48ae600 con 0x562ba4838400 2024-04-04T06:17:35.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.784+0000 7fadf0760280 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562ba48ae780 con 0x562ba3ad5c00 2024-04-04T06:17:35.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.784+0000 7fadef200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba4838800 0x562ba4831b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.784+0000 7fadef200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba4838800 0x562ba4831b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41870/0 (socket says 172.21.15.159:41870) 2024-04-04T06:17:35.781 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.784+0000 7fadef200640 1 -- 172.21.15.159:0/1668434844 learned_addr learned my addr 172.21.15.159:0/1668434844 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:35.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.785+0000 7fadef200640 1 -- 172.21.15.159:0/1668434844 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ba3ad5c00 msgr2=0x562ba4830b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.785+0000 7fadef200640 1 --2- 172.21.15.159:0/1668434844 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ba3ad5c00 0x562ba4830b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.785+0000 7fadef200640 1 -- 172.21.15.159:0/1668434844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ba4838400 msgr2=0x562ba4831600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.785+0000 7fadef200640 1 --2- 172.21.15.159:0/1668434844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ba4838400 0x562ba4831600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.785+0000 7fadef200640 1 -- 172.21.15.159:0/1668434844 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ba3b08780 con 0x562ba4838800 2024-04-04T06:17:35.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.785+0000 7fadef200640 1 --2- 172.21.15.159:0/1668434844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba4838800 0x562ba4831b80 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x562ba49033e0 tx=0x562ba4868c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16e7ab4b2c0a2e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.785+0000 7fadeb600640 1 -- 172.21.15.159:0/1668434844 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562ba3afad00 con 0x562ba4838800 2024-04-04T06:17:35.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.785+0000 7fadeb600640 1 -- 172.21.15.159:0/1668434844 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x562ba3b00fc0 con 0x562ba4838800 2024-04-04T06:17:35.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.785+0000 7fadeb600640 1 -- 172.21.15.159:0/1668434844 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562ba3afa680 con 0x562ba4838800 2024-04-04T06:17:35.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.785+0000 7fadf0760280 1 -- 172.21.15.159:0/1668434844 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ba3b08000 con 0x562ba4838800 2024-04-04T06:17:35.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.785+0000 7fadf0760280 1 -- 172.21.15.159:0/1668434844 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({fsmap=0}) -- 0x562ba3b09680 con 0x562ba4838800 2024-04-04T06:17:35.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.785+0000 7fadf0760280 4 waiting for MDS map... 2024-04-04T06:17:35.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.785+0000 7fadeb600640 1 -- 172.21.15.159:0/1668434844 <== mon.0 v2:172.21.15.159:3300/0 4 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x562ba49aa000 con 0x562ba4838800 2024-04-04T06:17:35.782 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.786+0000 7fadeb600640 1 -- 172.21.15.159:0/1668434844 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 108) ==== 2885+0+0 (secure 0 0 0) 0x562ba3b2f180 con 0x562ba4838800 2024-04-04T06:17:35.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.786+0000 7fadf0760280 4 Got MDS map 108 2024-04-04T06:17:35.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.786+0000 7fadf0760280 10 main: JournalTool::main 2024-04-04T06:17:35.783 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.786+0000 7fadf0760280 4 main: JournalTool: connecting to RADOS... 2024-04-04T06:17:35.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.786+0000 7fadf0760280 1 Processor -- start 2024-04-04T06:17:35.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.787+0000 7fadf0760280 1 -- start start 2024-04-04T06:17:35.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.787+0000 7fadf0760280 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ba49a8c00 0x562ba4831080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.787+0000 7fadf0760280 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ba49a9000 0x562ba4830580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.787+0000 7fadf0760280 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba49a9400 0x562ba4832100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.787+0000 7fadf0760280 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562ba3b36180 con 0x562ba49a9400 2024-04-04T06:17:35.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.787+0000 7fadf0760280 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562ba48aef00 con 0x562ba49a8c00 2024-04-04T06:17:35.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.787+0000 7fadf0760280 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562ba48af080 con 0x562ba49a9000 2024-04-04T06:17:35.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.788+0000 7fadede00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba49a9400 0x562ba4832100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.788+0000 7fadede00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba49a9400 0x562ba4832100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41884/0 (socket says 172.21.15.159:41884) 2024-04-04T06:17:35.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.788+0000 7fadede00640 1 -- 172.21.15.159:0/2327901036 learned_addr learned my addr 172.21.15.159:0/2327901036 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:35.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.788+0000 7fadede00640 1 --2- 172.21.15.159:0/2327901036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ba49a8c00 0x562ba4831080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.788+0000 7fadee800640 1 --2- 172.21.15.159:0/2327901036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ba49a9000 0x562ba4830580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.788+0000 7fadede00640 1 -- 172.21.15.159:0/2327901036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ba49a9000 msgr2=0x562ba4830580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.788+0000 7fadede00640 1 --2- 172.21.15.159:0/2327901036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ba49a9000 0x562ba4830580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.788+0000 7fadede00640 1 -- 172.21.15.159:0/2327901036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ba49a8c00 msgr2=0x562ba4831080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.788+0000 7fadede00640 1 --2- 172.21.15.159:0/2327901036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ba49a8c00 0x562ba4831080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.788+0000 7fadede00640 1 -- 172.21.15.159:0/2327901036 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ba3b09860 con 0x562ba49a9400 2024-04-04T06:17:35.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.788+0000 7fadede00640 1 --2- 172.21.15.159:0/2327901036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba49a9400 0x562ba4832100 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x562ba4902de0 tx=0x562ba49c4d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42cccf7c1d991d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.789+0000 7fade8e00640 1 -- 172.21.15.159:0/2327901036 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562ba3afad00 con 0x562ba49a9400 2024-04-04T06:17:35.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.789+0000 7fade8e00640 1 -- 172.21.15.159:0/2327901036 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x562ba3b00700 con 0x562ba49a9400 2024-04-04T06:17:35.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.789+0000 7fade8e00640 1 -- 172.21.15.159:0/2327901036 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562ba3afa680 con 0x562ba49a9400 2024-04-04T06:17:35.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.789+0000 7fadf0760280 1 -- 172.21.15.159:0/2327901036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba49a9400 msgr2=0x562ba4832100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.789+0000 7fadf0760280 1 --2- 172.21.15.159:0/2327901036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba49a9400 0x562ba4832100 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x562ba4902de0 tx=0x562ba49c4d20 comp rx=0 tx=0).stop 2024-04-04T06:17:35.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.789+0000 7fadf0760280 1 -- 172.21.15.159:0/2327901036 shutdown_connections 2024-04-04T06:17:35.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.789+0000 7fadf0760280 1 --2- 172.21.15.159:0/2327901036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba49a9400 0x562ba4832100 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.789+0000 7fadf0760280 1 --2- 172.21.15.159:0/2327901036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ba49a9000 0x562ba4830580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.789+0000 7fadf0760280 1 --2- 172.21.15.159:0/2327901036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ba49a8c00 0x562ba4831080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.789+0000 7fadf0760280 1 -- 172.21.15.159:0/2327901036 >> 172.21.15.159:0/2327901036 conn(0x562ba4838000 msgr2=0x562ba3be1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:35.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.789+0000 7fadf0760280 1 -- 172.21.15.159:0/2327901036 shutdown_connections 2024-04-04T06:17:35.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.789+0000 7fadf0760280 1 -- 172.21.15.159:0/2327901036 wait complete. 2024-04-04T06:17:35.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.790+0000 7fadf0760280 1 Processor -- start 2024-04-04T06:17:35.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.790+0000 7fadf0760280 1 -- start start 2024-04-04T06:17:35.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.791+0000 7fadf0760280 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ba49a8c00 0x562ba4833180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.791+0000 7fadf0760280 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba49a9000 0x562ba4832c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.791+0000 7fadf0760280 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ba49a9400 0x562ba4832680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.791+0000 7fadf0760280 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562ba48af380 con 0x562ba49a9000 2024-04-04T06:17:35.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.791+0000 7fadf0760280 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562ba48af200 con 0x562ba49a8c00 2024-04-04T06:17:35.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.791+0000 7fadf0760280 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562ba48af500 con 0x562ba49a9400 2024-04-04T06:17:35.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.791+0000 7fadee800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba49a9000 0x562ba4832c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.791+0000 7fadee800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba49a9000 0x562ba4832c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41894/0 (socket says 172.21.15.159:41894) 2024-04-04T06:17:35.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.791+0000 7fadee800640 1 -- 172.21.15.159:0/3841587172 learned_addr learned my addr 172.21.15.159:0/3841587172 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:35.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.791+0000 7fadee800640 1 -- 172.21.15.159:0/3841587172 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ba49a9400 msgr2=0x562ba4832680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.791+0000 7fadee800640 1 --2- 172.21.15.159:0/3841587172 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ba49a9400 0x562ba4832680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.791+0000 7fadee800640 1 -- 172.21.15.159:0/3841587172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ba49a8c00 msgr2=0x562ba4833180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.791+0000 7fadee800640 1 --2- 172.21.15.159:0/3841587172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ba49a8c00 0x562ba4833180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.791+0000 7fadee800640 1 -- 172.21.15.159:0/3841587172 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ba3b094a0 con 0x562ba49a9000 2024-04-04T06:17:35.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.792+0000 7fadee800640 1 --2- 172.21.15.159:0/3841587172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba49a9000 0x562ba4832c00 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x562ba49c3410 tx=0x562ba49c4f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.792+0000 7fade7000640 1 -- 172.21.15.159:0/3841587172 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562ba3afad00 con 0x562ba49a9000 2024-04-04T06:17:35.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.792+0000 7fade7000640 1 -- 172.21.15.159:0/3841587172 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x562ba3b00e00 con 0x562ba49a9000 2024-04-04T06:17:35.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.792+0000 7fade7000640 1 -- 172.21.15.159:0/3841587172 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562ba3afa680 con 0x562ba49a9000 2024-04-04T06:17:35.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.792+0000 7fadf0760280 1 -- 172.21.15.159:0/3841587172 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562ba3b08960 con 0x562ba49a9000 2024-04-04T06:17:35.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.792+0000 7fadf0760280 1 -- 172.21.15.159:0/3841587172 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ba3b083c0 con 0x562ba49a9000 2024-04-04T06:17:35.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.792+0000 7fadf0760280 4 main: JournalTool: resolving pool 16 2024-04-04T06:17:35.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.794+0000 7fade7000640 1 -- 172.21.15.159:0/3841587172 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x562ba49d4c00 con 0x562ba49a9000 2024-04-04T06:17:35.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.794+0000 7fade7000640 1 --2- 172.21.15.159:0/3841587172 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x562ba4a1ac00 0x562ba4833700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.794+0000 7fade7000640 1 -- 172.21.15.159:0/3841587172 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x562ba4a1c800 con 0x562ba49a9000 2024-04-04T06:17:35.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.794+0000 7fadf0760280 4 main: JournalTool: creating IoCtx.. 2024-04-04T06:17:35.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.794+0000 7fadf0760280 4 main: Executing for rank 0 2024-04-04T06:17:35.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.795+0000 7fadede00640 1 --2- 172.21.15.159:0/3841587172 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x562ba4a1ac00 0x562ba4833700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.795+0000 7fadf0760280 1 -- start start 2024-04-04T06:17:35.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.795+0000 7fadf0760280 1 Processor -- start 2024-04-04T06:17:35.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.796+0000 7fadede00640 1 --2- 172.21.15.159:0/3841587172 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x562ba4a1ac00 0x562ba4833700 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x562ba4902de0 tx=0x562ba49c4c00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.796+0000 7fadf0760280 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ba49a9c00 0x562ba4832100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.796+0000 7fadf0760280 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba4a1b000 0x562ba4830580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.796+0000 7fadf0760280 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ba4a1b400 0x562ba4831080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.796+0000 7fadf0760280 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562ba48af080 con 0x562ba4a1b000 2024-04-04T06:17:35.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.796+0000 7fadf0760280 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562ba48aef00 con 0x562ba4a1b400 2024-04-04T06:17:35.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.796+0000 7fadf0760280 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562ba48af800 con 0x562ba49a9c00 2024-04-04T06:17:35.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.797+0000 7fadee800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba4a1b000 0x562ba4830580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.797+0000 7fadee800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba4a1b000 0x562ba4830580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41908/0 (socket says 172.21.15.159:41908) 2024-04-04T06:17:35.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.797+0000 7fadee800640 1 -- 172.21.15.159:0/3232578575 learned_addr learned my addr 172.21.15.159:0/3232578575 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:35.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.797+0000 7fadede00640 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ba4a1b400 0x562ba4831080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.797+0000 7fadef200640 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ba49a9c00 0x562ba4832100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.797+0000 7fadee800640 1 -- 172.21.15.159:0/3232578575 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ba49a9c00 msgr2=0x562ba4832100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.797+0000 7fadee800640 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ba49a9c00 0x562ba4832100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.797+0000 7fadee800640 1 -- 172.21.15.159:0/3232578575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ba4a1b400 msgr2=0x562ba4831080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.797+0000 7fadee800640 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ba4a1b400 0x562ba4831080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.797+0000 7fadee800640 1 -- 172.21.15.159:0/3232578575 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ba3b09860 con 0x562ba4a1b000 2024-04-04T06:17:35.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.797+0000 7fadee800640 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba4a1b000 0x562ba4830580 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x562ba4a57470 tx=0x562ba49c5860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51ae92f9f1e83c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.797+0000 7fade3400640 1 -- 172.21.15.159:0/3232578575 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562ba3dd96c0 con 0x562ba4a1b000 2024-04-04T06:17:35.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.797+0000 7fade3400640 1 -- 172.21.15.159:0/3232578575 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x562ba3b00c40 con 0x562ba4a1b000 2024-04-04T06:17:35.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.797+0000 7fade3400640 1 -- 172.21.15.159:0/3232578575 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562ba3dd9860 con 0x562ba4a1b000 2024-04-04T06:17:35.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.797+0000 7fadf0760280 1 -- 172.21.15.159:0/3232578575 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ba3b094a0 con 0x562ba4a1b000 2024-04-04T06:17:35.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.798+0000 7fadf0760280 1 -- 172.21.15.159:0/3232578575 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({fsmap=0}) -- 0x562ba3b08960 con 0x562ba4a1b000 2024-04-04T06:17:35.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.798+0000 7fadf0760280 4 waiting for MDS map... 2024-04-04T06:17:35.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.798+0000 7fade3400640 1 -- 172.21.15.159:0/3232578575 <== mon.0 v2:172.21.15.159:3300/0 4 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x562ba3eaa200 con 0x562ba4a1b000 2024-04-04T06:17:35.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.798+0000 7fade3400640 1 -- 172.21.15.159:0/3232578575 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 108) ==== 2885+0+0 (secure 0 0 0) 0x562ba3b2f500 con 0x562ba4a1b000 2024-04-04T06:17:35.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.798+0000 7fadf0760280 4 Got MDS map 108 2024-04-04T06:17:35.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.798+0000 7fadf0760280 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x562ba4a1b800 0x562ba3ec2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.798+0000 7fadf0760280 1 -- 172.21.15.159:0/3232578575 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:1 16.f 16:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x562ba4a1bc00 con 0x562ba4a1b800 2024-04-04T06:17:35.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.798+0000 7fadef200640 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x562ba4a1b800 0x562ba3ec2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.799+0000 7fadef200640 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x562ba4a1b800 0x562ba3ec2000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=bbab564d6cb36898 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.805+0000 7fadef200640 1 -- 172.21.15.159:0/3232578575 <== osd.0 v2:172.21.15.159:6800/455382518 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v116'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x562ba3bd0000 con 0x562ba4a1b800 2024-04-04T06:17:35.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.805+0000 7fadf0760280 1 -- 172.21.15.159:0/3232578575 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:2 16.14 16:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x562ba3ebc400 con 0x562ba4a1b800 2024-04-04T06:17:35.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.806+0000 7fadef200640 1 -- 172.21.15.159:0/3232578575 <== osd.0 v2:172.21.15.159:6800/455382518 2 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v116'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x562ba3bd0000 con 0x562ba4a1b800 2024-04-04T06:17:35.804 INFO:teuthology.orchestra.run.smithi159.stdout:writing EResetJournal entry 2024-04-04T06:17:35.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.806+0000 7fadf0760280 1 -- 172.21.15.159:0/3232578575 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:3 16.14 16:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x562ba3ebc800 con 0x562ba4a1b800 2024-04-04T06:17:35.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.807+0000 7fadf0760280 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x562ba3ebcc00 0x562ba3ec2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.807+0000 7fadf0760280 1 -- 172.21.15.159:0/3232578575 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:4 16.1b 16:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x562ba3ebd000 con 0x562ba3ebcc00 2024-04-04T06:17:35.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.807+0000 7fadf0760280 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x562ba3ebd400 0x562ba3ec2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.807+0000 7fadf0760280 1 -- 172.21.15.159:0/3232578575 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:5 16.b 16:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x562ba3ebd800 con 0x562ba3ebd400 2024-04-04T06:17:35.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.807+0000 7fadf0760280 1 -- 172.21.15.159:0/3232578575 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:6 16.1f 16:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x562ba3ebdc00 con 0x562ba3ebcc00 2024-04-04T06:17:35.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.807+0000 7fadf0760280 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x562ba3ed0000 0x562ba3ec3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.807+0000 7fadf0760280 1 -- 172.21.15.159:0/3232578575 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:7 16.1a 16:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x562ba3ed0400 con 0x562ba3ed0000 2024-04-04T06:17:35.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.807+0000 7fadf0760280 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x562ba3ed0800 0x562ba3ec3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.807+0000 7fadf0760280 1 -- 172.21.15.159:0/3232578575 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:8 16.1e 16:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x562ba3ed0c00 con 0x562ba3ed0800 2024-04-04T06:17:35.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.807+0000 7fadef200640 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x562ba3ebd400 0x562ba3ec2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.807+0000 7fadede00640 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x562ba3ebcc00 0x562ba3ec2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.808 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.808+0000 7fadee800640 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x562ba3ed0000 0x562ba3ec3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.808+0000 7fadef200640 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x562ba3ebd400 0x562ba3ec2b00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=444b3a333c883864 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.808+0000 7fadef200640 1 -- 172.21.15.159:0/3232578575 <== osd.0 v2:172.21.15.159:6800/455382518 3 ==== osd_op_reply(3 200.00000001 [delete] v116'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x562ba3bd0000 con 0x562ba4a1b800 2024-04-04T06:17:35.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.809+0000 7fadee800640 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x562ba3ed0000 0x562ba3ec3080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=6e19cf7e88a56f3f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.809+0000 7fadede00640 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x562ba3ed0800 0x562ba3ec3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.809+0000 7fadede00640 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x562ba3ebcc00 0x562ba3ec2580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=fce0cebd22ca90ce server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.810+0000 7fadede00640 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x562ba3ed0800 0x562ba3ec3600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=d1f2c13a69103241 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.811+0000 7fadef200640 1 -- 172.21.15.159:0/3232578575 <== osd.1 v2:172.21.15.159:6816/3765338297 1 ==== osd_op_reply(5 200.00000003 [delete] v116'6 uv5 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x562ba3bd0000 con 0x562ba3ebd400 2024-04-04T06:17:35.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.812+0000 7fadee800640 1 -- 172.21.15.159:0/3232578575 <== osd.3 v2:172.21.15.159:6820/2070760205 1 ==== osd_op_reply(7 200.00000005 [delete] v116'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x562ba3bd0500 con 0x562ba3ed0000 2024-04-04T06:17:35.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.813+0000 7fadede00640 1 -- 172.21.15.159:0/3232578575 <== osd.5 v2:172.21.15.178:6802/63746765 1 ==== osd_op_reply(4 200.00000002 [delete] v116'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x562ba3bd0780 con 0x562ba3ebcc00 2024-04-04T06:17:35.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.813+0000 7fade1600640 1 -- 172.21.15.159:0/3232578575 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:9 16.14 16:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x562ba4a1bc00 con 0x562ba4a1b800 2024-04-04T06:17:35.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.814+0000 7fadede00640 1 -- 172.21.15.159:0/3232578575 <== osd.5 v2:172.21.15.178:6802/63746765 2 ==== osd_op_reply(6 200.00000004 [delete] v116'9 uv8 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x562ba3bd0780 con 0x562ba3ebcc00 2024-04-04T06:17:35.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.814+0000 7fadef200640 1 -- 172.21.15.159:0/3232578575 <== osd.0 v2:172.21.15.159:6800/455382518 4 ==== osd_op_reply(9 200.00000001 [write 0~48] v116'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x562ba3bd0000 con 0x562ba4a1b800 2024-04-04T06:17:35.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.815+0000 7fadede00640 1 -- 172.21.15.159:0/3232578575 <== osd.7 v2:172.21.15.178:6810/1066817938 1 ==== osd_op_reply(8 200.00000006 [delete] v116'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x562ba3bd0780 con 0x562ba3ed0800 2024-04-04T06:17:35.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.815+0000 7fadf0760280 4 Successfully wrote new journal pointer and header for rank 8:0 2024-04-04T06:17:35.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.815+0000 7fadf0760280 1 -- 172.21.15.159:0/3232578575 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x562ba4a1b800 msgr2=0x562ba3ec2000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.815+0000 7fadf0760280 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x562ba4a1b800 0x562ba3ec2000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.815+0000 7fadf0760280 1 -- 172.21.15.159:0/3232578575 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x562ba3ebd400 msgr2=0x562ba3ec2b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.815+0000 7fadf0760280 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x562ba3ebd400 0x562ba3ec2b00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.815+0000 7fadf0760280 1 -- 172.21.15.159:0/3232578575 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x562ba3ed0000 msgr2=0x562ba3ec3080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.815+0000 7fadf0760280 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x562ba3ed0000 0x562ba3ec3080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.815+0000 7fadf0760280 1 -- 172.21.15.159:0/3232578575 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x562ba3ebcc00 msgr2=0x562ba3ec2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.815+0000 7fadf0760280 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x562ba3ebcc00 0x562ba3ec2580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.815+0000 7fadf0760280 1 -- 172.21.15.159:0/3232578575 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x562ba3ed0800 msgr2=0x562ba3ec3600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.815+0000 7fadf0760280 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x562ba3ed0800 0x562ba3ec3600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.815+0000 7fadede00640 1 -- 172.21.15.159:0/3232578575 reap_dead start 2024-04-04T06:17:35.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.815+0000 7fadede00640 1 -- 172.21.15.159:0/3232578575 reap_dead start 2024-04-04T06:17:35.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.815+0000 7fadf0760280 1 -- 172.21.15.159:0/3232578575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba4a1b000 msgr2=0x562ba4830580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.815+0000 7fadf0760280 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba4a1b000 0x562ba4830580 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x562ba4a57470 tx=0x562ba49c5860 comp rx=0 tx=0).stop 2024-04-04T06:17:35.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.815+0000 7fadf0760280 1 -- 172.21.15.159:0/3232578575 shutdown_connections 2024-04-04T06:17:35.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.815+0000 7fadf0760280 1 --2- 172.21.15.159:0/3232578575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba4a1b000 0x562ba4830580 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.815+0000 7fadf0760280 1 -- 172.21.15.159:0/3232578575 >> 172.21.15.159:0/3232578575 conn(0x562ba49a9800 msgr2=0x562ba3e8e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:35.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.815+0000 7fadf0760280 1 -- 172.21.15.159:0/3232578575 shutdown_connections 2024-04-04T06:17:35.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.815+0000 7fadf0760280 1 -- 172.21.15.159:0/3232578575 wait complete. 2024-04-04T06:17:35.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.816+0000 7fadf0760280 1 -- 172.21.15.159:0/3841587172 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x562ba4a1ac00 msgr2=0x562ba4833700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.816+0000 7fadf0760280 1 --2- 172.21.15.159:0/3841587172 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x562ba4a1ac00 0x562ba4833700 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x562ba4902de0 tx=0x562ba49c4c00 comp rx=0 tx=0).stop 2024-04-04T06:17:35.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.816+0000 7fadf0760280 1 -- 172.21.15.159:0/3841587172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba49a9000 msgr2=0x562ba4832c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.816+0000 7fadf0760280 1 --2- 172.21.15.159:0/3841587172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba49a9000 0x562ba4832c00 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x562ba49c3410 tx=0x562ba49c4f00 comp rx=0 tx=0).stop 2024-04-04T06:17:35.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.816+0000 7fadf0760280 1 -- 172.21.15.159:0/3841587172 shutdown_connections 2024-04-04T06:17:35.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.816+0000 7fadf0760280 1 --2- 172.21.15.159:0/3841587172 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ba49a9400 0x562ba4832680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.816+0000 7fadf0760280 1 --2- 172.21.15.159:0/3841587172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba49a9000 0x562ba4832c00 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.816+0000 7fadf0760280 1 --2- 172.21.15.159:0/3841587172 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x562ba4a1ac00 0x562ba4833700 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.816+0000 7fadf0760280 1 --2- 172.21.15.159:0/3841587172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ba49a8c00 0x562ba4833180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.816+0000 7fadf0760280 1 -- 172.21.15.159:0/3841587172 >> 172.21.15.159:0/3841587172 conn(0x562ba4838000 msgr2=0x562ba3be1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:35.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.816+0000 7fadf0760280 1 -- 172.21.15.159:0/3841587172 shutdown_connections 2024-04-04T06:17:35.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.816+0000 7fadf0760280 1 -- 172.21.15.159:0/3841587172 wait complete. 2024-04-04T06:17:35.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.816+0000 7fadf0760280 1 -- 172.21.15.159:0/1668434844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba4838800 msgr2=0x562ba4831b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.816+0000 7fadf0760280 1 --2- 172.21.15.159:0/1668434844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba4838800 0x562ba4831b80 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x562ba49033e0 tx=0x562ba4868c00 comp rx=0 tx=0).stop 2024-04-04T06:17:35.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.816+0000 7fadf0760280 1 -- 172.21.15.159:0/1668434844 shutdown_connections 2024-04-04T06:17:35.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.816+0000 7fadf0760280 1 --2- 172.21.15.159:0/1668434844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ba4838800 0x562ba4831b80 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.816+0000 7fadf0760280 1 --2- 172.21.15.159:0/1668434844 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562ba3ad5c00 0x562ba4830b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.816+0000 7fadf0760280 1 --2- 172.21.15.159:0/1668434844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562ba4838400 0x562ba4831600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.816+0000 7fadf0760280 1 -- 172.21.15.159:0/1668434844 >> 172.21.15.159:0/1668434844 conn(0x562ba3ad5000 msgr2=0x562ba3bdf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:35.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.817+0000 7fadf0760280 1 -- 172.21.15.159:0/1668434844 shutdown_connections 2024-04-04T06:17:35.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.817+0000 7fadf0760280 1 -- 172.21.15.159:0/1668434844 wait complete. 2024-04-04T06:17:35.821 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.130210, result: writing EResetJournal entry 2024-04-04T06:17:35.821 DEBUG:teuthology.orchestra.run.smithi159:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-04T06:17:35.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.961+0000 7f58e2631740 1 -- start start 2024-04-04T06:17:35.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.961+0000 7f58e2631740 1 Processor -- start 2024-04-04T06:17:35.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.962+0000 7f58e2631740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562776630000 0x562776629080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.962+0000 7f58e2631740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562775857c00 0x562776628b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.962+0000 7f58e2631740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562776630800 0x562776629b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.962+0000 7f58e2631740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562776632480 con 0x562775857c00 2024-04-04T06:17:35.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.962+0000 7f58e2631740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562776632600 con 0x562776630000 2024-04-04T06:17:35.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.962+0000 7f58e2631740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562776632780 con 0x562776630800 2024-04-04T06:17:35.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.962+0000 7f58dee00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562775857c00 0x562776628b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.962+0000 7f58dee00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562775857c00 0x562776628b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41924/0 (socket says 172.21.15.159:41924) 2024-04-04T06:17:35.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.962+0000 7f58dee00640 1 -- 172.21.15.159:0/1384582689 learned_addr learned my addr 172.21.15.159:0/1384582689 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:35.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.963+0000 7f58e0200640 1 --2- 172.21.15.159:0/1384582689 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562776630800 0x562776629b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.963+0000 7f58df800640 1 --2- 172.21.15.159:0/1384582689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562776630000 0x562776629080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.963+0000 7f58dee00640 1 -- 172.21.15.159:0/1384582689 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562776630800 msgr2=0x562776629b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.963+0000 7f58dee00640 1 --2- 172.21.15.159:0/1384582689 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562776630800 0x562776629b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.963+0000 7f58dee00640 1 -- 172.21.15.159:0/1384582689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562776630000 msgr2=0x562776629080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.963+0000 7f58dee00640 1 --2- 172.21.15.159:0/1384582689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562776630000 0x562776629080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.963+0000 7f58dee00640 1 -- 172.21.15.159:0/1384582689 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562775883680 con 0x562775857c00 2024-04-04T06:17:35.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.963+0000 7f58dee00640 1 --2- 172.21.15.159:0/1384582689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562775857c00 0x562776628b00 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x5627766815f0 tx=0x5627765eecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d5a20c03adb2e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.963+0000 7f58dc600640 1 -- 172.21.15.159:0/1384582689 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56277587cb60 con 0x562775857c00 2024-04-04T06:17:35.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.963+0000 7f58dc600640 1 -- 172.21.15.159:0/1384582689 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5627758a01c0 con 0x562775857c00 2024-04-04T06:17:35.960 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.963+0000 7f58dc600640 1 -- 172.21.15.159:0/1384582689 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56277587cd00 con 0x562775857c00 2024-04-04T06:17:35.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.963+0000 7f58e2631740 1 -- 172.21.15.159:0/1384582689 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562775882b40 con 0x562775857c00 2024-04-04T06:17:35.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.963+0000 7f58e2631740 1 -- 172.21.15.159:0/1384582689 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5627758823c0 con 0x562775857c00 2024-04-04T06:17:35.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.963+0000 7f58e2631740 4 waiting for MDS map... 2024-04-04T06:17:35.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.964+0000 7f58dc600640 1 -- 172.21.15.159:0/1384582689 <== mon.0 v2:172.21.15.159:3300/0 4 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x56277666a400 con 0x562775857c00 2024-04-04T06:17:35.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.964+0000 7f58dc600640 1 -- 172.21.15.159:0/1384582689 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 108) ==== 2885+0+0 (secure 0 0 0) 0x5627758af180 con 0x562775857c00 2024-04-04T06:17:35.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.964+0000 7f58e2631740 4 Got MDS map 108 2024-04-04T06:17:35.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.964+0000 7f58e2631740 4 datascan.main: Using metadata pool output 2024-04-04T06:17:35.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.964+0000 7f58e2631740 4 datascan.main: connecting to RADOS... 2024-04-04T06:17:35.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.965+0000 7f58e2631740 1 Processor -- start 2024-04-04T06:17:35.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.965+0000 7f58e2631740 1 -- start start 2024-04-04T06:17:35.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.966+0000 7f58e2631740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56277673f400 0x562776629600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.966+0000 7f58e2631740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56277673f800 0x562776628580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.966+0000 7f58e2631740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56277673fc00 0x56277662a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.966+0000 7f58e2631740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5627758b6180 con 0x56277673f800 2024-04-04T06:17:35.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.966+0000 7f58e2631740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562776632f00 con 0x56277673fc00 2024-04-04T06:17:35.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.966+0000 7f58e2631740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562776633080 con 0x56277673f400 2024-04-04T06:17:35.966 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.966+0000 7f58df800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56277673f800 0x562776628580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.966+0000 7f58df800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56277673f800 0x562776628580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41932/0 (socket says 172.21.15.159:41932) 2024-04-04T06:17:35.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.966+0000 7f58df800640 1 -- 172.21.15.159:0/2778583052 learned_addr learned my addr 172.21.15.159:0/2778583052 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:35.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.966+0000 7f58dee00640 1 --2- 172.21.15.159:0/2778583052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56277673fc00 0x56277662a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.966+0000 7f58e0200640 1 --2- 172.21.15.159:0/2778583052 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56277673f400 0x562776629600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.966+0000 7f58df800640 1 -- 172.21.15.159:0/2778583052 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56277673f400 msgr2=0x562776629600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.966+0000 7f58df800640 1 --2- 172.21.15.159:0/2778583052 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56277673f400 0x562776629600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.966+0000 7f58df800640 1 -- 172.21.15.159:0/2778583052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56277673fc00 msgr2=0x56277662a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:35.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.966+0000 7f58df800640 1 --2- 172.21.15.159:0/2778583052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56277673fc00 0x56277662a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.966+0000 7f58df800640 1 -- 172.21.15.159:0/2778583052 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627758834a0 con 0x56277673f800 2024-04-04T06:17:35.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.967+0000 7f58df800640 1 --2- 172.21.15.159:0/2778583052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56277673f800 0x562776628580 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x5627767429f0 tx=0x562776748c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d456f637e40e75 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.967+0000 7f58d9e00640 1 -- 172.21.15.159:0/2778583052 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56277587cb60 con 0x56277673f800 2024-04-04T06:17:35.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.967+0000 7f58d9e00640 1 -- 172.21.15.159:0/2778583052 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5627758a0c40 con 0x56277673f800 2024-04-04T06:17:35.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.967+0000 7f58d9e00640 1 -- 172.21.15.159:0/2778583052 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56277587cd00 con 0x56277673f800 2024-04-04T06:17:35.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.967+0000 7f58e2631740 1 -- 172.21.15.159:0/2778583052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56277673f800 msgr2=0x562776628580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.967+0000 7f58e2631740 1 --2- 172.21.15.159:0/2778583052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56277673f800 0x562776628580 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x5627767429f0 tx=0x562776748c00 comp rx=0 tx=0).stop 2024-04-04T06:17:35.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.967+0000 7f58e2631740 1 -- 172.21.15.159:0/2778583052 shutdown_connections 2024-04-04T06:17:35.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.967+0000 7f58e2631740 1 --2- 172.21.15.159:0/2778583052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56277673fc00 0x56277662a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.967+0000 7f58e2631740 1 --2- 172.21.15.159:0/2778583052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56277673f800 0x562776628580 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.967+0000 7f58e2631740 1 --2- 172.21.15.159:0/2778583052 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56277673f400 0x562776629600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.967+0000 7f58e2631740 1 -- 172.21.15.159:0/2778583052 >> 172.21.15.159:0/2778583052 conn(0x562776630400 msgr2=0x562775965800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:35.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.968+0000 7f58e2631740 1 -- 172.21.15.159:0/2778583052 shutdown_connections 2024-04-04T06:17:35.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.968+0000 7f58e2631740 1 -- 172.21.15.159:0/2778583052 wait complete. 2024-04-04T06:17:35.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.968+0000 7f58e2631740 1 Processor -- start 2024-04-04T06:17:35.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.968+0000 7f58e2631740 1 -- start start 2024-04-04T06:17:35.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.969+0000 7f58e2631740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56277673f400 0x56277662b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.968 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.969+0000 7f58e2631740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56277673f800 0x56277662ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.969+0000 7f58e2631740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56277673fc00 0x56277662a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.969+0000 7f58e2631740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562776633380 con 0x56277673fc00 2024-04-04T06:17:35.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.969+0000 7f58e2631740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562776633200 con 0x56277673f400 2024-04-04T06:17:35.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.969+0000 7f58e2631740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x562776633500 con 0x56277673f800 2024-04-04T06:17:35.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.969+0000 7f58dee00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56277673fc00 0x56277662a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.970+0000 7f58dee00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56277673fc00 0x56277662a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41946/0 (socket says 172.21.15.159:41946) 2024-04-04T06:17:35.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.970+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 learned_addr learned my addr 172.21.15.159:0/162290522 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:35.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.970+0000 7f58df800640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56277673f800 0x56277662ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.970+0000 7f58e0200640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56277673f400 0x56277662b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.970+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56277673f800 msgr2=0x56277662ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.970+0000 7f58dee00640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x56277673f800 0x56277662ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.970+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56277673f400 msgr2=0x56277662b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:35.969 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.970+0000 7f58dee00640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56277673f400 0x56277662b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:35.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.970+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562775883680 con 0x56277673fc00 2024-04-04T06:17:35.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.970+0000 7f58dee00640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56277673fc00 0x56277662a680 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x562776681ad0 tx=0x5627759192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.970+0000 7f58d8000640 1 -- 172.21.15.159:0/162290522 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56277587c680 con 0x56277673fc00 2024-04-04T06:17:35.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.970+0000 7f58d8000640 1 -- 172.21.15.159:0/162290522 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5627758a0e00 con 0x56277673fc00 2024-04-04T06:17:35.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.970+0000 7f58d8000640 1 -- 172.21.15.159:0/162290522 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56277587c820 con 0x56277673fc00 2024-04-04T06:17:35.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.970+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5627758832c0 con 0x56277673fc00 2024-04-04T06:17:35.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.971+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562775883a40 con 0x56277673fc00 2024-04-04T06:17:35.970 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.971+0000 7f58e2631740 4 datascan.init: resolving metadata pool 16 2024-04-04T06:17:35.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.972+0000 7f58d8000640 1 -- 172.21.15.159:0/162290522 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x5627767a0c00 con 0x56277673fc00 2024-04-04T06:17:35.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.973+0000 7f58d8000640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x562776798c00 0x56277662b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.973+0000 7f58d8000640 1 -- 172.21.15.159:0/162290522 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x56277679ae00 con 0x56277673fc00 2024-04-04T06:17:35.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.973+0000 7f58e2631740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-04T06:17:35.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.973+0000 7f58e2631740 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-04-04T06:17:35.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.973+0000 7f58e2631740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-04T06:17:35.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.973+0000 7f58e2631740 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x562776799000 0x56277662a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.973+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562776799400 con 0x562776799000 2024-04-04T06:17:35.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.973+0000 7f58e0200640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x562776798c00 0x56277662b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.973 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.974+0000 7f58df800640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x562776799000 0x56277662a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.975+0000 7f58df800640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x562776799000 0x56277662a100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.975+0000 7f58e0200640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x562776798c00 0x56277662b700 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x5627765c8210 tx=0x562776749020 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.975+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.7 v2:172.21.15.178:6810/1066817938 1 ==== osd_op_reply(1 [pgnls start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562776799000 2024-04-04T06:17:35.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.976+0000 7f58d9400640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x562776798800 0x562775be0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.976+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:2 17.20 17:04000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562776798000 con 0x562776798800 2024-04-04T06:17:35.974 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.976+0000 7f58e0200640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x562776798800 0x562775be0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.976+0000 7f58e0200640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x562776798800 0x562775be0000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.977+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.1 v2:172.21.15.159:6816/3765338297 1 ==== osd_op_reply(2 [pgnls start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6500 con 0x562776798800 2024-04-04T06:17:35.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.977+0000 7f58d9e00640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x562776799400 0x562775be0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.977+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:3 17.10 17:08000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd6000 con 0x562776799400 2024-04-04T06:17:35.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.977+0000 7f58df800640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x562776799400 0x562775be0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.977+0000 7f58df800640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x562776799400 0x562775be0580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.978+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.3 v2:172.21.15.159:6820/2070760205 1 ==== osd_op_reply(3 [pgnls start_epoch 116 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562776799400 2024-04-04T06:17:35.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.978+0000 7f58d9400640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x562776798400 0x562775be0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.978+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:4 17.30 17:0c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd6800 con 0x562776798400 2024-04-04T06:17:35.975 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.978+0000 7f58dee00640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x562776798400 0x562775be0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.979+0000 7f58dee00640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x562776798400 0x562775be0b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.979+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.0 v2:172.21.15.159:6800/455382518 1 ==== osd_op_reply(4 [pgnls start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6780 con 0x562776798400 2024-04-04T06:17:35.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.979+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:5 17.8 17:10000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd6400 con 0x562776799000 2024-04-04T06:17:35.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.980+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.7 v2:172.21.15.178:6810/1066817938 2 ==== osd_op_reply(5 [pgnls start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562776799000 2024-04-04T06:17:35.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.980+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:6 17.28 17:14000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd6c00 con 0x562776799400 2024-04-04T06:17:35.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.980+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.3 v2:172.21.15.159:6820/2070760205 2 ==== osd_op_reply(6 [pgnls start_epoch 116 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562776799400 2024-04-04T06:17:35.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.980+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:7 17.18 17:18000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd7400 con 0x562776798400 2024-04-04T06:17:35.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.980+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.0 v2:172.21.15.159:6800/455382518 2 ==== osd_op_reply(7 [pgnls start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6780 con 0x562776798400 2024-04-04T06:17:35.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.980+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:8 17.38 17:1c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd7000 con 0x562776799000 2024-04-04T06:17:35.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.981+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.7 v2:172.21.15.178:6810/1066817938 3 ==== osd_op_reply(8 [pgnls start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562776799000 2024-04-04T06:17:35.979 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.981+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:9 17.4 17:20000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd7800 con 0x562776798400 2024-04-04T06:17:35.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.982+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.0 v2:172.21.15.159:6800/455382518 3 ==== osd_op_reply(9 [pgnls start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6780 con 0x562776798400 2024-04-04T06:17:35.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.982+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:10 17.24 17:24000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bf2000 con 0x562776799400 2024-04-04T06:17:35.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.982+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.3 v2:172.21.15.159:6820/2070760205 3 ==== osd_op_reply(10 [pgnls start_epoch 116 out=49b] v113'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562776799400 2024-04-04T06:17:35.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.982+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:11 17.14 17:28000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd7c00 con 0x562776799400 2024-04-04T06:17:35.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.982+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.3 v2:172.21.15.159:6820/2070760205 4 ==== osd_op_reply(11 [pgnls start_epoch 116 out=49b] v113'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562776799400 2024-04-04T06:17:35.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.982+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:12 17.34 17:2c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bf2400 con 0x562776798400 2024-04-04T06:17:35.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.983+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.0 v2:172.21.15.159:6800/455382518 4 ==== osd_op_reply(12 [pgnls start_epoch 116 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6780 con 0x562776798400 2024-04-04T06:17:35.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.983+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:13 17.c 17:30000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd6000 con 0x562776799000 2024-04-04T06:17:35.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.984+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.7 v2:172.21.15.178:6810/1066817938 4 ==== osd_op_reply(13 [pgnls start_epoch 116 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562776799000 2024-04-04T06:17:35.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.984+0000 7f58d9400640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x562775bf2800 0x562775be1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.980 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.984+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:14 17.2c 17:34000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bf2c00 con 0x562775bf2800 2024-04-04T06:17:35.981 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.984+0000 7f58e0200640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x562775bf2800 0x562775be1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.985+0000 7f58e0200640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x562775bf2800 0x562775be1080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.986+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.4 v2:172.21.15.178:6812/4035991429 1 ==== osd_op_reply(14 [pgnls start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6500 con 0x562775bf2800 2024-04-04T06:17:35.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.986+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:15 17.1c 17:38000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd6400 con 0x562776798800 2024-04-04T06:17:35.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.986+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.1 v2:172.21.15.159:6816/3765338297 2 ==== osd_op_reply(15 [pgnls start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6500 con 0x562776798800 2024-04-04T06:17:35.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.986+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:16 17.3c 17:3c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bf2c00 con 0x562776798400 2024-04-04T06:17:35.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.987+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.0 v2:172.21.15.159:6800/455382518 5 ==== osd_op_reply(16 [pgnls start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6780 con 0x562776798400 2024-04-04T06:17:35.982 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.987+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:17 17.2 17:40000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd6c00 con 0x562775bf2800 2024-04-04T06:17:35.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.988+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.4 v2:172.21.15.178:6812/4035991429 2 ==== osd_op_reply(17 [pgnls start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6500 con 0x562775bf2800 2024-04-04T06:17:35.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.988+0000 7f58d9400640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x562775bf2000 0x562775be1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.983 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.988+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:18 17.22 17:44000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bf3000 con 0x562775bf2000 2024-04-04T06:17:35.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.989+0000 7f58df800640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x562775bf2000 0x562775be1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.985 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.990+0000 7f58df800640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x562775bf2000 0x562775be1600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.990+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.5 v2:172.21.15.178:6802/63746765 1 ==== osd_op_reply(18 [pgnls start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562775bf2000 2024-04-04T06:17:35.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.991+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:19 17.12 17:48000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd7000 con 0x562776799400 2024-04-04T06:17:35.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.991+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.3 v2:172.21.15.159:6820/2070760205 5 ==== osd_op_reply(19 [pgnls start_epoch 116 out=81b] v114'22 uv22 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5627765d6000 con 0x562776799400 2024-04-04T06:17:35.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.991+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:20 17.12 17:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x562776799800 con 0x562776799400 2024-04-04T06:17:35.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.992+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.3 v2:172.21.15.159:6820/2070760205 6 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5627765d6000 con 0x562776799400 2024-04-04T06:17:35.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.992+0000 7f58e2631740 10 datascan.operator(): handling object 1099511627778.0 2024-04-04T06:17:35.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.992+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:21 17.12 17:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x562776799c00 con 0x562776799400 2024-04-04T06:17:35.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.995+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.3 v2:172.21.15.159:6820/2070760205 7 ==== osd_op_reply(21 10000000002.00000000 [call] v116'23 uv23 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5627765d6000 con 0x562776799400 2024-04-04T06:17:35.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.995+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:22 17.32 17:4c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bf3000 con 0x562776799000 2024-04-04T06:17:35.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.996+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.7 v2:172.21.15.178:6810/1066817938 5 ==== osd_op_reply(22 [pgnls start_epoch 116 out=49b] v113'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562776799000 2024-04-04T06:17:35.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.996+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:23 17.a 17:50000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bf3800 con 0x562776799000 2024-04-04T06:17:35.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.997+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.7 v2:172.21.15.178:6810/1066817938 6 ==== osd_op_reply(23 [pgnls start_epoch 116 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562776799000 2024-04-04T06:17:35.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.997+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:24 17.2a 17:54000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd6400 con 0x562776799400 2024-04-04T06:17:35.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.997+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.3 v2:172.21.15.159:6820/2070760205 8 ==== osd_op_reply(24 [pgnls start_epoch 116 out=49b] v113'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562776799400 2024-04-04T06:17:35.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.997+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:25 17.1a 17:58000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bf3c00 con 0x562776798400 2024-04-04T06:17:35.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.998+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.0 v2:172.21.15.159:6800/455382518 6 ==== osd_op_reply(25 [pgnls start_epoch 116 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6780 con 0x562776798400 2024-04-04T06:17:35.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.998+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:26 17.3a 17:5c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562776798000 con 0x562776798800 2024-04-04T06:17:35.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.998+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.1 v2:172.21.15.159:6816/3765338297 3 ==== osd_op_reply(26 [pgnls start_epoch 116 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6500 con 0x562776798800 2024-04-04T06:17:35.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.998+0000 7f58d9e00640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x562775bf2400 0x562775be2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.998+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:27 17.6 17:60000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bf2c00 con 0x562775bf2400 2024-04-04T06:17:35.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:35.999+0000 7f58dee00640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x562775bf2400 0x562775be2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.000+0000 7f58dee00640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x562775bf2400 0x562775be2680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.001+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.6 v2:172.21.15.178:6826/198949157 1 ==== osd_op_reply(27 [pgnls start_epoch 116 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6780 con 0x562775bf2400 2024-04-04T06:17:35.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.001+0000 7f58d9400640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x562775c14000 0x562775be1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:35.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.001+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:28 17.26 17:64000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775c14400 con 0x562775c14000 2024-04-04T06:17:35.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.001+0000 7f58e0200640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x562775c14000 0x562775be1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:35.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.002+0000 7f58e0200640 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x562775c14000 0x562775be1b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:35.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.002+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.2 v2:172.21.15.159:6808/2619731047 1 ==== osd_op_reply(28 [pgnls start_epoch 116 out=49b] v113'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6500 con 0x562775c14000 2024-04-04T06:17:35.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.002+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:29 17.16 17:68000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bf3c00 con 0x562775bf2000 2024-04-04T06:17:36.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.003+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.5 v2:172.21.15.178:6802/63746765 2 ==== osd_op_reply(29 [pgnls start_epoch 116 out=49b] v113'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562775bf2000 2024-04-04T06:17:36.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.003+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:30 17.36 17:6c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775c14800 con 0x562776798400 2024-04-04T06:17:36.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.003+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.0 v2:172.21.15.159:6800/455382518 7 ==== osd_op_reply(30 [pgnls start_epoch 116 out=81b] v113'33 uv17 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5627765d6780 con 0x562776798400 2024-04-04T06:17:36.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.004+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:31 17.36 17:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x562775bf3400 con 0x562776798400 2024-04-04T06:17:36.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.004+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.0 v2:172.21.15.159:6800/455382518 8 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5627765d6780 con 0x562776798400 2024-04-04T06:17:36.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.004+0000 7f58e2631740 10 datascan.operator(): handling object 1099511628279.0 2024-04-04T06:17:36.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.004+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:32 17.36 17:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x562776799800 con 0x562776798400 2024-04-04T06:17:36.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.006+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.0 v2:172.21.15.159:6800/455382518 9 ==== osd_op_reply(32 100000001f7.00000000 [call] v116'34 uv34 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5627765d6780 con 0x562776798400 2024-04-04T06:17:36.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.006+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:33 17.e 17:70000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562776799c00 con 0x562775c14000 2024-04-04T06:17:36.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.006+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.2 v2:172.21.15.159:6808/2619731047 2 ==== osd_op_reply(33 [pgnls start_epoch 116 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6500 con 0x562775c14000 2024-04-04T06:17:36.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.006+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:34 17.2e 17:74000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775c14c00 con 0x562775bf2400 2024-04-04T06:17:36.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.007+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.6 v2:172.21.15.178:6826/198949157 2 ==== osd_op_reply(34 [pgnls start_epoch 116 out=49b] v113'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6780 con 0x562775bf2400 2024-04-04T06:17:36.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.007+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:35 17.1e 17:78000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd6800 con 0x562776798800 2024-04-04T06:17:36.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.007+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.1 v2:172.21.15.159:6816/3765338297 4 ==== osd_op_reply(35 [pgnls start_epoch 116 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6500 con 0x562776798800 2024-04-04T06:17:36.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.007+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:36 17.3e 17:7c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775c15000 con 0x562775bf2800 2024-04-04T06:17:36.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.008+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.4 v2:172.21.15.178:6812/4035991429 3 ==== osd_op_reply(36 [pgnls start_epoch 116 out=49b] v113'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6500 con 0x562775bf2800 2024-04-04T06:17:36.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.008+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:37 17.1 17:80000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd7400 con 0x562776798400 2024-04-04T06:17:36.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.009+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.0 v2:172.21.15.159:6800/455382518 10 ==== osd_op_reply(37 [pgnls start_epoch 116 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6780 con 0x562776798400 2024-04-04T06:17:36.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.009+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:38 17.21 17:84000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775c15400 con 0x562776798400 2024-04-04T06:17:36.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.009+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.0 v2:172.21.15.159:6800/455382518 11 ==== osd_op_reply(38 [pgnls start_epoch 116 out=49b] v113'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6780 con 0x562776798400 2024-04-04T06:17:36.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.010+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:39 17.11 17:88000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd7800 con 0x562776799400 2024-04-04T06:17:36.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.010+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.3 v2:172.21.15.159:6820/2070760205 9 ==== osd_op_reply(39 [pgnls start_epoch 116 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562776799400 2024-04-04T06:17:36.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.010+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:40 17.31 17:8c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562776799800 con 0x562776798800 2024-04-04T06:17:36.006 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.011+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.1 v2:172.21.15.159:6816/3765338297 5 ==== osd_op_reply(40 [pgnls start_epoch 116 out=81b] v113'15 uv8 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5627765d6500 con 0x562776798800 2024-04-04T06:17:36.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.011+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:41 17.31 17:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x562775bd7c00 con 0x562776798800 2024-04-04T06:17:36.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.011+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.1 v2:172.21.15.159:6816/3765338297 6 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5627765d6500 con 0x562776798800 2024-04-04T06:17:36.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.011+0000 7f58e2631740 10 datascan.operator(): handling object 1099511627780.1 2024-04-04T06:17:36.007 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.011+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:42 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x562775bd6000 con 0x562775bf2400 2024-04-04T06:17:36.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.015+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.6 v2:172.21.15.178:6826/198949157 3 ==== osd_op_reply(42 10000000004.00000000 [call] v116'14 uv14 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5627765d6780 con 0x562775bf2400 2024-04-04T06:17:36.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.015+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:43 17.9 17:90000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd7000 con 0x562776799400 2024-04-04T06:17:36.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.015+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.3 v2:172.21.15.159:6820/2070760205 10 ==== osd_op_reply(43 [pgnls start_epoch 116 out=49b] v113'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562776799400 2024-04-04T06:17:36.010 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.015+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:44 17.29 17:94000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562776798000 con 0x562775bf2800 2024-04-04T06:17:36.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.016+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.4 v2:172.21.15.178:6812/4035991429 4 ==== osd_op_reply(44 [pgnls start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6500 con 0x562775bf2800 2024-04-04T06:17:36.011 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.016+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:45 17.19 17:98000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775c15000 con 0x562775bf2000 2024-04-04T06:17:36.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.017+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.5 v2:172.21.15.178:6802/63746765 3 ==== osd_op_reply(45 [pgnls start_epoch 116 out=81b] v114'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5627765d6000 con 0x562775bf2000 2024-04-04T06:17:36.012 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.018+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:46 17.19 17:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x562775c15000 con 0x562775bf2000 2024-04-04T06:17:36.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.019+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.5 v2:172.21.15.178:6802/63746765 4 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5627765d6000 con 0x562775bf2000 2024-04-04T06:17:36.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.019+0000 7f58e2631740 10 datascan.operator(): handling object 1099511628280.0 2024-04-04T06:17:36.014 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.019+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:47 17.19 17:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x562775c15400 con 0x562775bf2000 2024-04-04T06:17:36.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.023+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.5 v2:172.21.15.178:6802/63746765 5 ==== osd_op_reply(47 100000001f8.00000000 [call] v116'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5627765d6000 con 0x562775bf2000 2024-04-04T06:17:36.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.023+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:48 17.39 17:9c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd6400 con 0x562775bf2000 2024-04-04T06:17:36.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.024+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.5 v2:172.21.15.178:6802/63746765 6 ==== osd_op_reply(48 [pgnls start_epoch 116 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562775bf2000 2024-04-04T06:17:36.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.024+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:49 17.5 17:a0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775c14800 con 0x562775bf2400 2024-04-04T06:17:36.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.025+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.6 v2:172.21.15.178:6826/198949157 4 ==== osd_op_reply(49 [pgnls start_epoch 116 out=81b] v116'14 uv14 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5627765d6780 con 0x562775bf2400 2024-04-04T06:17:36.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.025+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:50 17.5 17:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x562775bd7800 con 0x562775bf2400 2024-04-04T06:17:36.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.026+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.6 v2:172.21.15.178:6826/198949157 5 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5627765d6780 con 0x562775bf2400 2024-04-04T06:17:36.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.026+0000 7f58e2631740 10 datascan.operator(): handling object 1099511627780.0 2024-04-04T06:17:36.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.026+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:51 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x562775bd7000 con 0x562775bf2400 2024-04-04T06:17:36.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.029+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.6 v2:172.21.15.178:6826/198949157 6 ==== osd_op_reply(51 10000000004.00000000 [call] v116'15 uv15 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5627765d6780 con 0x562775bf2400 2024-04-04T06:17:36.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.029+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:52 17.25 17:a4000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bf3000 con 0x562776798800 2024-04-04T06:17:36.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.030+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.1 v2:172.21.15.159:6816/3765338297 7 ==== osd_op_reply(52 [pgnls start_epoch 116 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6500 con 0x562776798800 2024-04-04T06:17:36.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.030+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:53 17.15 17:a8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775c14c00 con 0x562775bf2000 2024-04-04T06:17:36.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.031+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.5 v2:172.21.15.178:6802/63746765 7 ==== osd_op_reply(53 [pgnls start_epoch 116 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562775bf2000 2024-04-04T06:17:36.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.031+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:54 17.35 17:ac000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562776799c00 con 0x562775bf2800 2024-04-04T06:17:36.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.033+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.4 v2:172.21.15.178:6812/4035991429 5 ==== osd_op_reply(54 [pgnls start_epoch 116 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6500 con 0x562775bf2800 2024-04-04T06:17:36.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.033+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:55 17.d 17:b0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562776799800 con 0x562776799000 2024-04-04T06:17:36.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.034+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.7 v2:172.21.15.178:6810/1066817938 7 ==== osd_op_reply(55 [pgnls start_epoch 116 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562776799000 2024-04-04T06:17:36.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.034+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:56 17.2d 17:b4000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd7400 con 0x562776799000 2024-04-04T06:17:36.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.035+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.7 v2:172.21.15.178:6810/1066817938 8 ==== osd_op_reply(56 [pgnls start_epoch 116 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562776799000 2024-04-04T06:17:36.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.035+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:57 17.1d 17:b8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562776798000 con 0x562776799000 2024-04-04T06:17:36.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.036+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.7 v2:172.21.15.178:6810/1066817938 9 ==== osd_op_reply(57 [pgnls start_epoch 116 out=49b] v113'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562776799000 2024-04-04T06:17:36.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.036+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:58 17.3d 17:bc000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd6c00 con 0x562776799000 2024-04-04T06:17:36.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.037+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.7 v2:172.21.15.178:6810/1066817938 10 ==== osd_op_reply(58 [pgnls start_epoch 116 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562776799000 2024-04-04T06:17:36.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.037+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:59 17.3 17:c0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd6800 con 0x562775bf2800 2024-04-04T06:17:36.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.038+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.4 v2:172.21.15.178:6812/4035991429 6 ==== osd_op_reply(59 [pgnls start_epoch 116 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6500 con 0x562775bf2800 2024-04-04T06:17:36.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.038+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:60 17.23 17:c4000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bf2c00 con 0x562775bf2800 2024-04-04T06:17:36.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.040+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.4 v2:172.21.15.178:6812/4035991429 7 ==== osd_op_reply(60 [pgnls start_epoch 116 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6500 con 0x562775bf2800 2024-04-04T06:17:36.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.040+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:61 17.13 17:c8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd7c00 con 0x562776799400 2024-04-04T06:17:36.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.041+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.3 v2:172.21.15.159:6820/2070760205 11 ==== osd_op_reply(61 [pgnls start_epoch 116 out=49b] v113'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562776799400 2024-04-04T06:17:36.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.041+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:62 17.33 17:cc000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bf3400 con 0x562776799000 2024-04-04T06:17:36.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.042+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.7 v2:172.21.15.178:6810/1066817938 11 ==== osd_op_reply(62 [pgnls start_epoch 116 out=49b] v113'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562776799000 2024-04-04T06:17:36.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.042+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:63 17.b 17:d0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775c15000 con 0x562775bf2000 2024-04-04T06:17:36.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.043+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 <== osd.5 v2:172.21.15.178:6802/63746765 8 ==== osd_op_reply(63 [pgnls start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6000 con 0x562775bf2000 2024-04-04T06:17:36.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.043+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:64 17.2b 17:d4000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775c14400 con 0x562775bf2800 2024-04-04T06:17:36.039 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.044+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.4 v2:172.21.15.178:6812/4035991429 8 ==== osd_op_reply(64 [pgnls start_epoch 116 out=49b] v113'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6500 con 0x562775bf2800 2024-04-04T06:17:36.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.045+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:65 17.1b 17:d8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775c15400 con 0x562775bf2800 2024-04-04T06:17:36.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.046+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.4 v2:172.21.15.178:6812/4035991429 9 ==== osd_op_reply(65 [pgnls start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6500 con 0x562775bf2800 2024-04-04T06:17:36.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.046+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:66 17.3b 17:dc000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775c15800 con 0x562775c14000 2024-04-04T06:17:36.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.046+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.2 v2:172.21.15.159:6808/2619731047 3 ==== osd_op_reply(66 [pgnls start_epoch 116 out=49b] v113'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6500 con 0x562775c14000 2024-04-04T06:17:36.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.047+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:67 17.7 17:e0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775c14c00 con 0x562776798400 2024-04-04T06:17:36.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.047+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.0 v2:172.21.15.159:6800/455382518 12 ==== osd_op_reply(67 [pgnls start_epoch 116 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6780 con 0x562776798400 2024-04-04T06:17:36.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.048+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:68 17.27 17:e4000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775c15c00 con 0x562775bf2800 2024-04-04T06:17:36.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.049+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.4 v2:172.21.15.178:6812/4035991429 10 ==== osd_op_reply(68 [pgnls start_epoch 116 out=49b] v113'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6500 con 0x562775bf2800 2024-04-04T06:17:36.044 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.049+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:69 17.17 17:e8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd6400 con 0x562775bf2400 2024-04-04T06:17:36.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.050+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.6 v2:172.21.15.178:6826/198949157 7 ==== osd_op_reply(69 [pgnls start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6780 con 0x562775bf2400 2024-04-04T06:17:36.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.050+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:70 17.37 17:ec000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775c14400 con 0x562776798400 2024-04-04T06:17:36.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.051+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.0 v2:172.21.15.159:6800/455382518 13 ==== osd_op_reply(70 [pgnls start_epoch 116 out=49b] v113'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6780 con 0x562776798400 2024-04-04T06:17:36.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.051+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:71 17.f 17:f0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd7400 con 0x562776798800 2024-04-04T06:17:36.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.052+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.1 v2:172.21.15.159:6816/3765338297 8 ==== osd_op_reply(71 [pgnls start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6500 con 0x562776798800 2024-04-04T06:17:36.047 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.052+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:72 17.2f 17:f4000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775c15400 con 0x562775bf2800 2024-04-04T06:17:36.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.053+0000 7f58e0200640 1 -- 172.21.15.159:0/162290522 <== osd.4 v2:172.21.15.178:6812/4035991429 11 ==== osd_op_reply(72 [pgnls start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6500 con 0x562775bf2800 2024-04-04T06:17:36.048 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.053+0000 7f58d9e00640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:73 17.1f 17:f8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bd6c00 con 0x562775bf2400 2024-04-04T06:17:36.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.054+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.6 v2:172.21.15.178:6826/198949157 8 ==== osd_op_reply(73 [pgnls start_epoch 116 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6780 con 0x562775bf2400 2024-04-04T06:17:36.049 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.055+0000 7f58d9400640 1 -- 172.21.15.159:0/162290522 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:74 17.3f 17:fc000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562775bf3000 con 0x562776798400 2024-04-04T06:17:36.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.055+0000 7f58dee00640 1 -- 172.21.15.159:0/162290522 <== osd.0 v2:172.21.15.159:6800/455382518 14 ==== osd_op_reply(74 [pgnls start_epoch 116 out=49b] v113'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5627765d6780 con 0x562776798400 2024-04-04T06:17:36.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.055+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x562776798400 msgr2=0x562775be0b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.050 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.055+0000 7f58e2631740 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x562776798400 0x562775be0b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.051 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.055+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x562776798800 msgr2=0x562775be0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.055+0000 7f58e2631740 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x562776798800 0x562775be0000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x562775c14000 msgr2=0x562775be1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58e2631740 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x562775c14000 0x562775be1b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x562776799400 msgr2=0x562775be0580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.052 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58e2631740 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x562776799400 0x562775be0580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 reap_dead start 2024-04-04T06:17:36.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x562775bf2800 msgr2=0x562775be1080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58e2631740 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x562775bf2800 0x562775be1080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x562775bf2000 msgr2=0x562775be1600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58e2631740 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x562775bf2000 0x562775be1600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x562775bf2400 msgr2=0x562775be2680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58e2631740 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x562775bf2400 0x562775be2680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x562776799000 msgr2=0x56277662a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58e2631740 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x562776799000 0x56277662a100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58df800640 1 -- 172.21.15.159:0/162290522 reap_dead start 2024-04-04T06:17:36.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x562776798c00 msgr2=0x56277662b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58e2631740 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x562776798c00 0x56277662b700 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x5627765c8210 tx=0x562776749020 comp rx=0 tx=0).stop 2024-04-04T06:17:36.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56277673fc00 msgr2=0x56277662a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58e2631740 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56277673fc00 0x56277662a680 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x562776681ad0 tx=0x5627759192c0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 shutdown_connections 2024-04-04T06:17:36.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58e2631740 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56277673fc00 0x56277662a680 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58e2631740 1 --2- 172.21.15.159:0/162290522 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x562776798c00 0x56277662b700 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.056+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 >> 172.21.15.159:0/162290522 conn(0x562776630400 msgr2=0x562775965800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:36.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.057+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 shutdown_connections 2024-04-04T06:17:36.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.057+0000 7f58e2631740 1 -- 172.21.15.159:0/162290522 wait complete. 2024-04-04T06:17:36.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.057+0000 7f58e2631740 1 -- 172.21.15.159:0/1384582689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562775857c00 msgr2=0x562776628b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.057+0000 7f58e2631740 1 --2- 172.21.15.159:0/1384582689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562775857c00 0x562776628b00 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x5627766815f0 tx=0x5627765eecc0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.058+0000 7f58e2631740 1 -- 172.21.15.159:0/1384582689 shutdown_connections 2024-04-04T06:17:36.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.058+0000 7f58e2631740 1 --2- 172.21.15.159:0/1384582689 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x562776630800 0x562776629b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.058+0000 7f58e2631740 1 --2- 172.21.15.159:0/1384582689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562775857c00 0x562776628b00 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.058+0000 7f58e2631740 1 --2- 172.21.15.159:0/1384582689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562776630000 0x562776629080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.058+0000 7f58e2631740 1 -- 172.21.15.159:0/1384582689 >> 172.21.15.159:0/1384582689 conn(0x562775857000 msgr2=0x562775965000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:36.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.058+0000 7f58e2631740 1 -- 172.21.15.159:0/1384582689 shutdown_connections 2024-04-04T06:17:36.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.058+0000 7f58e2631740 1 -- 172.21.15.159:0/1384582689 wait complete. 2024-04-04T06:17:36.063 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.241657, result: 2024-04-04T06:17:36.063 DEBUG:teuthology.orchestra.run.smithi159:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-04-04T06:17:36.165 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.170+0000 7f946b82a740 1 -- start start 2024-04-04T06:17:36.166 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.170+0000 7f946b82a740 1 Processor -- start 2024-04-04T06:17:36.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.171+0000 7f946b82a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e1f1b4400 0x561e1f1ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.171+0000 7f946b82a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e1f1b4800 0x561e1f1acb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.171+0000 7f946b82a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561e1f1b5c00 0x561e1f1adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.171+0000 7f946b82a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561e1f22a480 con 0x561e1f1b4800 2024-04-04T06:17:36.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.171+0000 7f946b82a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561e1f22a600 con 0x561e1f1b4400 2024-04-04T06:17:36.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.171+0000 7f946b82a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x561e1f22a780 con 0x561e1f1b5c00 2024-04-04T06:17:36.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.172+0000 7f9468000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e1f1b4800 0x561e1f1acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.172+0000 7f9468000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e1f1b4800 0x561e1f1acb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41962/0 (socket says 172.21.15.159:41962) 2024-04-04T06:17:36.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.172+0000 7f9468000640 1 -- 172.21.15.159:0/1100290808 learned_addr learned my addr 172.21.15.159:0/1100290808 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:36.167 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.172+0000 7f9468000640 1 -- 172.21.15.159:0/1100290808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561e1f1b5c00 msgr2=0x561e1f1adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:36.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.172+0000 7f9469400640 1 --2- 172.21.15.159:0/1100290808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561e1f1b5c00 0x561e1f1adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.172+0000 7f9468a00640 1 --2- 172.21.15.159:0/1100290808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e1f1b4400 0x561e1f1ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.172+0000 7f9468000640 1 --2- 172.21.15.159:0/1100290808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561e1f1b5c00 0x561e1f1adb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.172+0000 7f9468000640 1 -- 172.21.15.159:0/1100290808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e1f1b4400 msgr2=0x561e1f1ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:36.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.172+0000 7f9468000640 1 --2- 172.21.15.159:0/1100290808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e1f1b4400 0x561e1f1ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.168 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.172+0000 7f9468000640 1 -- 172.21.15.159:0/1100290808 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e1e479680 con 0x561e1f1b4800 2024-04-04T06:17:36.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.172+0000 7f9468000640 1 --2- 172.21.15.159:0/1100290808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e1f1b4800 0x561e1f1acb00 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x561e1f27b980 tx=0x561e1f1e7020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7acb9155aa5f840 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.173+0000 7f9465800640 1 -- 172.21.15.159:0/1100290808 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561e1e472d00 con 0x561e1f1b4800 2024-04-04T06:17:36.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.173+0000 7f9465800640 1 -- 172.21.15.159:0/1100290808 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561e1e496e00 con 0x561e1f1b4800 2024-04-04T06:17:36.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.173+0000 7f9465800640 1 -- 172.21.15.159:0/1100290808 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561e1e472680 con 0x561e1f1b4800 2024-04-04T06:17:36.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.173+0000 7f946b82a740 1 -- 172.21.15.159:0/1100290808 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e1e4792c0 con 0x561e1f1b4800 2024-04-04T06:17:36.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.173+0000 7f946b82a740 1 -- 172.21.15.159:0/1100290808 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({fsmap=0}) -- 0x561e1e479a40 con 0x561e1f1b4800 2024-04-04T06:17:36.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.173+0000 7f946b82a740 4 waiting for MDS map... 2024-04-04T06:17:36.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.173+0000 7f9465800640 1 -- 172.21.15.159:0/1100290808 <== mon.0 v2:172.21.15.159:3300/0 4 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x561e1f25e000 con 0x561e1f1b4800 2024-04-04T06:17:36.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.173+0000 7f9465800640 1 -- 172.21.15.159:0/1100290808 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 108) ==== 2885+0+0 (secure 0 0 0) 0x561e1e4a5180 con 0x561e1f1b4800 2024-04-04T06:17:36.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.173+0000 7f946b82a740 4 Got MDS map 108 2024-04-04T06:17:36.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.173+0000 7f946b82a740 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-04-04T06:17:36.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.173+0000 7f946b82a740 4 datascan.main: Using metadata pool output 2024-04-04T06:17:36.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.173+0000 7f946b82a740 4 datascan.main: connecting to RADOS... 2024-04-04T06:17:36.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.174+0000 7f946b82a740 1 Processor -- start 2024-04-04T06:17:36.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.174+0000 7f946b82a740 1 -- start start 2024-04-04T06:17:36.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.175+0000 7f946b82a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e1f1b5800 0x561e1f1ad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.175+0000 7f946b82a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e1f31b000 0x561e1f1ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.175+0000 7f946b82a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561e1f31b400 0x561e1f1ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.175+0000 7f946b82a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561e1e4ac180 con 0x561e1f31b000 2024-04-04T06:17:36.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.175+0000 7f946b82a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561e1f22af00 con 0x561e1f1b5800 2024-04-04T06:17:36.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.175+0000 7f946b82a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x561e1f22b080 con 0x561e1f31b400 2024-04-04T06:17:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.175+0000 7f9468a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e1f31b000 0x561e1f1ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.175+0000 7f9468a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e1f31b000 0x561e1f1ac580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41964/0 (socket says 172.21.15.159:41964) 2024-04-04T06:17:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.175+0000 7f9468a00640 1 -- 172.21.15.159:0/472999342 learned_addr learned my addr 172.21.15.159:0/472999342 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.176+0000 7f9468a00640 1 -- 172.21.15.159:0/472999342 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561e1f31b400 msgr2=0x561e1f1ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.176+0000 7f9468a00640 1 --2- 172.21.15.159:0/472999342 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561e1f31b400 0x561e1f1ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.176+0000 7f9468a00640 1 -- 172.21.15.159:0/472999342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e1f1b5800 msgr2=0x561e1f1ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.176+0000 7f9468a00640 1 --2- 172.21.15.159:0/472999342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e1f1b5800 0x561e1f1ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.176+0000 7f9468a00640 1 -- 172.21.15.159:0/472999342 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e1e4783c0 con 0x561e1f31b000 2024-04-04T06:17:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.176+0000 7f9468a00640 1 --2- 172.21.15.159:0/472999342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e1f31b000 0x561e1f1ac580 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x561e1f330b70 tx=0x561e1f336c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7991a604af80ad server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.176+0000 7f9463000640 1 -- 172.21.15.159:0/472999342 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561e1e472d00 con 0x561e1f31b000 2024-04-04T06:17:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.176+0000 7f9463000640 1 -- 172.21.15.159:0/472999342 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561e1e496c40 con 0x561e1f31b000 2024-04-04T06:17:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.176+0000 7f9463000640 1 -- 172.21.15.159:0/472999342 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561e1e472680 con 0x561e1f31b000 2024-04-04T06:17:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.177+0000 7f946b82a740 1 -- 172.21.15.159:0/472999342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e1f31b000 msgr2=0x561e1f1ac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.177+0000 7f946b82a740 1 --2- 172.21.15.159:0/472999342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e1f31b000 0x561e1f1ac580 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x561e1f330b70 tx=0x561e1f336c00 comp rx=0 tx=0).stop 2024-04-04T06:17:36.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.177+0000 7f946b82a740 1 -- 172.21.15.159:0/472999342 shutdown_connections 2024-04-04T06:17:36.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.177+0000 7f946b82a740 1 --2- 172.21.15.159:0/472999342 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561e1f31b400 0x561e1f1ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.177+0000 7f946b82a740 1 --2- 172.21.15.159:0/472999342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e1f31b000 0x561e1f1ac580 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.177+0000 7f946b82a740 1 --2- 172.21.15.159:0/472999342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e1f1b5800 0x561e1f1ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.177+0000 7f946b82a740 1 -- 172.21.15.159:0/472999342 >> 172.21.15.159:0/472999342 conn(0x561e1f1b5400 msgr2=0x561e1e539800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:36.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.177+0000 7f946b82a740 1 -- 172.21.15.159:0/472999342 shutdown_connections 2024-04-04T06:17:36.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.177+0000 7f946b82a740 1 -- 172.21.15.159:0/472999342 wait complete. 2024-04-04T06:17:36.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.178+0000 7f946b82a740 1 Processor -- start 2024-04-04T06:17:36.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.178+0000 7f946b82a740 1 -- start start 2024-04-04T06:17:36.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.179+0000 7f946b82a740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561e1f1b5800 0x561e1f1af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.179+0000 7f946b82a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e1f31b000 0x561e1f1aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.179+0000 7f946b82a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e1f31b400 0x561e1f1ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.179+0000 7f946b82a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561e1f22b380 con 0x561e1f31b400 2024-04-04T06:17:36.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.179+0000 7f946b82a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561e1f22b200 con 0x561e1f31b000 2024-04-04T06:17:36.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.179+0000 7f946b82a740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x561e1f22b500 con 0x561e1f1b5800 2024-04-04T06:17:36.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.179+0000 7f9468000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e1f31b400 0x561e1f1ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.179+0000 7f9468000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e1f31b400 0x561e1f1ae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41974/0 (socket says 172.21.15.159:41974) 2024-04-04T06:17:36.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.179+0000 7f9468000640 1 -- 172.21.15.159:0/2978730605 learned_addr learned my addr 172.21.15.159:0/2978730605 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:36.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.179+0000 7f9468a00640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e1f31b000 0x561e1f1aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.179+0000 7f9469400640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561e1f1b5800 0x561e1f1af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.179+0000 7f9468000640 1 -- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561e1f1b5800 msgr2=0x561e1f1af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.179+0000 7f9468000640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561e1f1b5800 0x561e1f1af180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.179+0000 7f9468000640 1 -- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e1f31b000 msgr2=0x561e1f1aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.179+0000 7f9468000640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e1f31b000 0x561e1f1aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.179+0000 7f9468000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e1e479680 con 0x561e1f31b400 2024-04-04T06:17:36.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.179+0000 7f9468a00640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e1f31b000 0x561e1f1aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:36.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.180+0000 7f9468000640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e1f31b400 0x561e1f1ae680 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x561e1f331860 tx=0x561e1f1e6d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.180+0000 7f9461200640 1 -- 172.21.15.159:0/2978730605 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561e1e472b60 con 0x561e1f31b400 2024-04-04T06:17:36.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.180+0000 7f9461200640 1 -- 172.21.15.159:0/2978730605 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x561e1e496a80 con 0x561e1f31b400 2024-04-04T06:17:36.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.180+0000 7f9461200640 1 -- 172.21.15.159:0/2978730605 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561e1e472d00 con 0x561e1f31b400 2024-04-04T06:17:36.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.180+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561e1e4792c0 con 0x561e1f31b400 2024-04-04T06:17:36.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.180+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e1e478b40 con 0x561e1f31b400 2024-04-04T06:17:36.180 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.180+0000 7f946b82a740 4 datascan.init: resolving metadata pool 16 2024-04-04T06:17:36.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.182+0000 7f9461200640 1 -- 172.21.15.159:0/2978730605 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x561e1f34ac00 con 0x561e1f31b400 2024-04-04T06:17:36.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.182+0000 7f9461200640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x561e1f38fc00 0x561e1f1af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.182+0000 7f9461200640 1 -- 172.21.15.159:0/2978730605 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x561e1f395000 con 0x561e1f31b400 2024-04-04T06:17:36.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.183+0000 7f946b82a740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-04T06:17:36.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.183+0000 7f946b82a740 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-04-04T06:17:36.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.183+0000 7f946b82a740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-04T06:17:36.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.183+0000 7f946b82a740 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x561e1f31b800 0x561e1f1ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.183+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:1 16.14 16:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x561e1f31bc00 con 0x561e1f31b800 2024-04-04T06:17:36.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.183+0000 7f9469400640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x561e1f38fc00 0x561e1f1af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.183+0000 7f9468a00640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x561e1f31b800 0x561e1f1ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.183+0000 7f9468a00640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x561e1f31b800 0x561e1f1ae100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.184+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.0 v2:172.21.15.159:6800/455382518 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv35 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f31b800 2024-04-04T06:17:36.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.184+0000 7f946b82a740 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x561e1f31bc00 0x561e1f1ac580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.184+0000 7f9469400640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x561e1f38fc00 0x561e1f1af700 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x561e1f1cce40 tx=0x561e1f1e7080 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.184+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:2 16.1f 16:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x561e1f38e800 con 0x561e1f31bc00 2024-04-04T06:17:36.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.184+0000 7f9469400640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x561e1f31bc00 0x561e1f1ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.185+0000 7f9469400640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x561e1f31bc00 0x561e1f1ac580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.186+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.5 v2:172.21.15.178:6802/63746765 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1f31bc00 2024-04-04T06:17:36.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.186+0000 7f946b82a740 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x561e1f38e000 0x561e1f1ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.186+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:3 17.0 17:00000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1f38e400 con 0x561e1f38e000 2024-04-04T06:17:36.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.187+0000 7f9468a00640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x561e1f38e000 0x561e1f1ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.188+0000 7f9468a00640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x561e1f38e000 0x561e1f1ad080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.185 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.189+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.7 v2:172.21.15.178:6810/1066817938 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f38e000 2024-04-04T06:17:36.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.189+0000 7f9462600640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x561e1e7e6000 0x561e1e7ec000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.186 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.189+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:4 17.20 17:04000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7e6400 con 0x561e1e7e6000 2024-04-04T06:17:36.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.189+0000 7f9468000640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x561e1e7e6000 0x561e1e7ec000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.190+0000 7f9468000640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x561e1e7e6000 0x561e1e7ec000 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.190+0000 7f9468000640 1 -- 172.21.15.159:0/2978730605 <== osd.1 v2:172.21.15.159:6816/3765338297 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce780 con 0x561e1e7e6000 2024-04-04T06:17:36.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.190+0000 7f9463000640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x561e1e7e6c00 0x561e1e7ec580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.190+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:5 17.10 17:08000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7e7000 con 0x561e1e7e6c00 2024-04-04T06:17:36.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.190+0000 7f9469400640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x561e1e7e6c00 0x561e1e7ec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.191+0000 7f9469400640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x561e1e7e6c00 0x561e1e7ec580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.191+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.3 v2:172.21.15.159:6820/2070760205 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 116 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1e7e6c00 2024-04-04T06:17:36.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.191+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:6 17.30 17:0c000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7e6800 con 0x561e1f31b800 2024-04-04T06:17:36.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.192+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.0 v2:172.21.15.159:6800/455382518 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f31b800 2024-04-04T06:17:36.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.192+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:7 17.8 17:10000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7e7400 con 0x561e1f38e000 2024-04-04T06:17:36.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.193+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.7 v2:172.21.15.178:6810/1066817938 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f38e000 2024-04-04T06:17:36.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.193+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:8 17.28 17:14000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7e7800 con 0x561e1e7e6c00 2024-04-04T06:17:36.189 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.193+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.3 v2:172.21.15.159:6820/2070760205 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 116 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1e7e6c00 2024-04-04T06:17:36.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.193+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:9 17.18 17:18000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f0000 con 0x561e1f31b800 2024-04-04T06:17:36.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.194+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.0 v2:172.21.15.159:6800/455382518 3 ==== osd_op_reply(9 [pgnls-filter start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f31b800 2024-04-04T06:17:36.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.194+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:10 17.38 17:1c000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7e7c00 con 0x561e1f38e000 2024-04-04T06:17:36.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.195+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.7 v2:172.21.15.178:6810/1066817938 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f38e000 2024-04-04T06:17:36.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.195+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:11 17.4 17:20000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f0400 con 0x561e1f31b800 2024-04-04T06:17:36.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.195+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.0 v2:172.21.15.159:6800/455382518 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f31b800 2024-04-04T06:17:36.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.195+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:12 17.24 17:24000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7e6800 con 0x561e1e7e6c00 2024-04-04T06:17:36.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.196+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.3 v2:172.21.15.159:6820/2070760205 3 ==== osd_op_reply(12 [pgnls-filter start_epoch 116 out=49b] v113'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1e7e6c00 2024-04-04T06:17:36.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.196+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:13 17.14 17:28000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f0800 con 0x561e1e7e6c00 2024-04-04T06:17:36.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.196+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.3 v2:172.21.15.159:6820/2070760205 4 ==== osd_op_reply(13 [pgnls-filter start_epoch 116 out=49b] v113'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1e7e6c00 2024-04-04T06:17:36.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.196+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:14 17.34 17:2c000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7e7400 con 0x561e1f31b800 2024-04-04T06:17:36.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.196+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.0 v2:172.21.15.159:6800/455382518 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 116 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f31b800 2024-04-04T06:17:36.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.196+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:15 17.c 17:30000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f0000 con 0x561e1f38e000 2024-04-04T06:17:36.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.197+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.7 v2:172.21.15.178:6810/1066817938 4 ==== osd_op_reply(15 [pgnls-filter start_epoch 116 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f38e000 2024-04-04T06:17:36.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.197+0000 7f9462600640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x561e1e7e7c00 0x561e1e7ecb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.197+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:16 17.2c 17:34000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1f38e400 con 0x561e1e7e7c00 2024-04-04T06:17:36.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.198+0000 7f9468a00640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x561e1e7e7c00 0x561e1e7ecb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.199+0000 7f9468a00640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x561e1e7e7c00 0x561e1e7ecb00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.199+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.4 v2:172.21.15.178:6812/4035991429 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1e7e7c00 2024-04-04T06:17:36.194 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.200+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:17 17.1c 17:38000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f0c00 con 0x561e1e7e6000 2024-04-04T06:17:36.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.200+0000 7f9468000640 1 -- 172.21.15.159:0/2978730605 <== osd.1 v2:172.21.15.159:6816/3765338297 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce780 con 0x561e1e7e6000 2024-04-04T06:17:36.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.200+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:18 17.3c 17:3c000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f1800 con 0x561e1f31b800 2024-04-04T06:17:36.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.200+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.0 v2:172.21.15.159:6800/455382518 6 ==== osd_op_reply(18 [pgnls-filter start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f31b800 2024-04-04T06:17:36.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.201+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:19 17.2 17:40000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f1000 con 0x561e1e7e7c00 2024-04-04T06:17:36.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.201+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.4 v2:172.21.15.178:6812/4035991429 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1e7e7c00 2024-04-04T06:17:36.196 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.201+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:20 17.22 17:44000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f1c00 con 0x561e1f31bc00 2024-04-04T06:17:36.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.202+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.5 v2:172.21.15.178:6802/63746765 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1f31bc00 2024-04-04T06:17:36.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.202+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:21 17.12 17:48000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f1400 con 0x561e1e7e6c00 2024-04-04T06:17:36.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.203+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.3 v2:172.21.15.159:6820/2070760205 5 ==== osd_op_reply(21 [pgnls-filter start_epoch 116 out=81b] v116'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1e7e6c00 2024-04-04T06:17:36.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.203+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:22 17.0 17:00000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f0800 con 0x561e1f38e000 2024-04-04T06:17:36.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.204+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.7 v2:172.21.15.178:6810/1066817938 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f38e000 2024-04-04T06:17:36.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.204+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:23 17.20 17:04000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f1c00 con 0x561e1e7e6000 2024-04-04T06:17:36.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.205+0000 7f9468000640 1 -- 172.21.15.159:0/2978730605 <== osd.1 v2:172.21.15.159:6816/3765338297 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce780 con 0x561e1e7e6000 2024-04-04T06:17:36.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.205+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:24 17.10 17:08000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fa000 con 0x561e1e7e6c00 2024-04-04T06:17:36.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.205+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.3 v2:172.21.15.159:6820/2070760205 6 ==== osd_op_reply(24 [pgnls-filter start_epoch 116 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1e7e6c00 2024-04-04T06:17:36.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.205+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:25 17.30 17:0c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f0400 con 0x561e1f31b800 2024-04-04T06:17:36.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.206+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.0 v2:172.21.15.159:6800/455382518 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f31b800 2024-04-04T06:17:36.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.206+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:26 17.8 17:10000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fa400 con 0x561e1f38e000 2024-04-04T06:17:36.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.207+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.7 v2:172.21.15.178:6810/1066817938 6 ==== osd_op_reply(26 [pgnls-filter start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f38e000 2024-04-04T06:17:36.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.207+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:27 17.28 17:14000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f0000 con 0x561e1e7e6c00 2024-04-04T06:17:36.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.207+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.3 v2:172.21.15.159:6820/2070760205 7 ==== osd_op_reply(27 [pgnls-filter start_epoch 116 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1e7e6c00 2024-04-04T06:17:36.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.207+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:28 17.18 17:18000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fa800 con 0x561e1f31b800 2024-04-04T06:17:36.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.208+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.0 v2:172.21.15.159:6800/455382518 8 ==== osd_op_reply(28 [pgnls-filter start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f31b800 2024-04-04T06:17:36.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.208+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:29 17.38 17:1c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f1800 con 0x561e1f38e000 2024-04-04T06:17:36.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.209+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.7 v2:172.21.15.178:6810/1066817938 7 ==== osd_op_reply(29 [pgnls-filter start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f38e000 2024-04-04T06:17:36.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.209+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:30 17.4 17:20000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fac00 con 0x561e1f31b800 2024-04-04T06:17:36.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.209+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.0 v2:172.21.15.159:6800/455382518 9 ==== osd_op_reply(30 [pgnls-filter start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f31b800 2024-04-04T06:17:36.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.209+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:31 17.24 17:24000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f1000 con 0x561e1e7e6c00 2024-04-04T06:17:36.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.210+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.3 v2:172.21.15.159:6820/2070760205 8 ==== osd_op_reply(31 [pgnls-filter start_epoch 116 out=49b] v113'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1e7e6c00 2024-04-04T06:17:36.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.210+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:32 17.14 17:28000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fb000 con 0x561e1e7e6c00 2024-04-04T06:17:36.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.210+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.3 v2:172.21.15.159:6820/2070760205 9 ==== osd_op_reply(32 [pgnls-filter start_epoch 116 out=49b] v113'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1e7e6c00 2024-04-04T06:17:36.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.210+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:33 17.34 17:2c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fb800 con 0x561e1f31b800 2024-04-04T06:17:36.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.210+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.0 v2:172.21.15.159:6800/455382518 10 ==== osd_op_reply(33 [pgnls-filter start_epoch 116 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f31b800 2024-04-04T06:17:36.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.211+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:34 17.c 17:30000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fb400 con 0x561e1f38e000 2024-04-04T06:17:36.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.212+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.7 v2:172.21.15.178:6810/1066817938 8 ==== osd_op_reply(34 [pgnls-filter start_epoch 116 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f38e000 2024-04-04T06:17:36.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.212+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:35 17.2c 17:34000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fbc00 con 0x561e1e7e7c00 2024-04-04T06:17:36.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.213+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.4 v2:172.21.15.178:6812/4035991429 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1e7e7c00 2024-04-04T06:17:36.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.213+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:36 17.1c 17:38000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fa400 con 0x561e1e7e6000 2024-04-04T06:17:36.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.213+0000 7f9468000640 1 -- 172.21.15.159:0/2978730605 <== osd.1 v2:172.21.15.159:6816/3765338297 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce780 con 0x561e1e7e6000 2024-04-04T06:17:36.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.213+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:37 17.3c 17:3c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fa400 con 0x561e1f31b800 2024-04-04T06:17:36.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.214+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.0 v2:172.21.15.159:6800/455382518 11 ==== osd_op_reply(37 [pgnls-filter start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f31b800 2024-04-04T06:17:36.209 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.214+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:38 17.2 17:40000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fa800 con 0x561e1e7e7c00 2024-04-04T06:17:36.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.215+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.4 v2:172.21.15.178:6812/4035991429 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1e7e7c00 2024-04-04T06:17:36.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.215+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:39 17.22 17:44000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f0c00 con 0x561e1f31bc00 2024-04-04T06:17:36.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.216+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.5 v2:172.21.15.178:6802/63746765 3 ==== osd_op_reply(39 [pgnls-filter start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1f31bc00 2024-04-04T06:17:36.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.216+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:40 17.12 17:48000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fac00 con 0x561e1e7e6c00 2024-04-04T06:17:36.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.216+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.3 v2:172.21.15.159:6820/2070760205 10 ==== osd_op_reply(40 [pgnls-filter start_epoch 116 out=49b] v116'23 uv23 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1e7e6c00 2024-04-04T06:17:36.211 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.217+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:41 17.32 17:4c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f1c00 con 0x561e1f38e000 2024-04-04T06:17:36.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.218+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.7 v2:172.21.15.178:6810/1066817938 9 ==== osd_op_reply(41 [pgnls-filter start_epoch 116 out=49b] v113'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f38e000 2024-04-04T06:17:36.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.218+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:42 17.a 17:50000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fb800 con 0x561e1f38e000 2024-04-04T06:17:36.213 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.219+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.7 v2:172.21.15.178:6810/1066817938 10 ==== osd_op_reply(42 [pgnls-filter start_epoch 116 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f38e000 2024-04-04T06:17:36.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.219+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:43 17.2a 17:54000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7e6400 con 0x561e1e7e6c00 2024-04-04T06:17:36.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.219+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.3 v2:172.21.15.159:6820/2070760205 11 ==== osd_op_reply(43 [pgnls-filter start_epoch 116 out=49b] v113'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1e7e6c00 2024-04-04T06:17:36.214 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.219+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:44 17.1a 17:58000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f0800 con 0x561e1f31b800 2024-04-04T06:17:36.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.220+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.0 v2:172.21.15.159:6800/455382518 12 ==== osd_op_reply(44 [pgnls-filter start_epoch 116 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f31b800 2024-04-04T06:17:36.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.220+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:45 17.3a 17:5c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1f38f800 con 0x561e1e7e6000 2024-04-04T06:17:36.215 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.220+0000 7f9468000640 1 -- 172.21.15.159:0/2978730605 <== osd.1 v2:172.21.15.159:6816/3765338297 5 ==== osd_op_reply(45 [pgnls-filter start_epoch 116 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce780 con 0x561e1e7e6000 2024-04-04T06:17:36.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.220+0000 7f9462600640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x561e1e7f0400 0x561e1e7ed080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.221+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:46 17.6 17:60000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f1800 con 0x561e1e7f0400 2024-04-04T06:17:36.216 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.221+0000 7f9468000640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x561e1e7f0400 0x561e1e7ed080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.217 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.222+0000 7f9468000640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x561e1e7f0400 0x561e1e7ed080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.223+0000 7f9468000640 1 -- 172.21.15.159:0/2978730605 <== osd.6 v2:172.21.15.178:6826/198949157 1 ==== osd_op_reply(46 [pgnls-filter start_epoch 116 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce780 con 0x561e1e7f0400 2024-04-04T06:17:36.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.223+0000 7f9463000640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x561e1f38f400 0x561e1e7ed600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.218 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.223+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:47 17.26 17:64000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fa000 con 0x561e1f38f400 2024-04-04T06:17:36.219 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.224+0000 7f9469400640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x561e1f38f400 0x561e1e7ed600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.225+0000 7f9469400640 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x561e1f38f400 0x561e1e7ed600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.225+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.2 v2:172.21.15.159:6808/2619731047 1 ==== osd_op_reply(47 [pgnls-filter start_epoch 116 out=49b] v113'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1f38f400 2024-04-04T06:17:36.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.225+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:48 17.16 17:68000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e80c000 con 0x561e1f31bc00 2024-04-04T06:17:36.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.226+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.5 v2:172.21.15.178:6802/63746765 4 ==== osd_op_reply(48 [pgnls-filter start_epoch 116 out=49b] v113'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1f31bc00 2024-04-04T06:17:36.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.226+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:49 17.36 17:6c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fb000 con 0x561e1f31b800 2024-04-04T06:17:36.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.227+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.0 v2:172.21.15.159:6800/455382518 13 ==== osd_op_reply(49 [pgnls-filter start_epoch 116 out=81b] v116'34 uv34 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f31b800 2024-04-04T06:17:36.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.227+0000 7f946b82a740 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-04-04T06:17:36.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.227+0000 7f946b82a740 10 datascan.operator(): handling object 100000001f7.0 2024-04-04T06:17:36.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.227+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:50 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x561e1f38e400 con 0x561e1f31b800 2024-04-04T06:17:36.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.228+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.0 v2:172.21.15.159:6800/455382518 14 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) ==== 416+0+194 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f31b800 2024-04-04T06:17:36.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.228+0000 7f946b82a740 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-04-04T06:17:36.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.228+0000 7f946b82a740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-04-04T06:17:36.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.228+0000 7f946b82a740 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-04-04T06:17:36.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.228+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:51 16.7 16:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7e7000 con 0x561e1e7e6c00 2024-04-04T06:17:36.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.229+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.3 v2:172.21.15.159:6820/2070760205 12 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1e7e6c00 2024-04-04T06:17:36.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.229+0000 7f946b82a740 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-04-04T06:17:36.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.229+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:52 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7e7800 con 0x561e1f38f400 2024-04-04T06:17:36.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.229+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.2 v2:172.21.15.159:6808/2619731047 2 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=585b] v0'0 uv12 ondisk = 0) ==== 164+0+585 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1f38f400 2024-04-04T06:17:36.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.229+0000 7f946b82a740 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-04-04T06:17:36.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.230+0000 7f946b82a740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-04-04T06:17:36.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.230+0000 7f946b82a740 20 datascan.get_frag_of: frag is * 2024-04-04T06:17:36.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.230+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:53 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7e6800 con 0x561e1e7e6c00 2024-04-04T06:17:36.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.230+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.3 v2:172.21.15.159:6820/2070760205 13 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1e7e6c00 2024-04-04T06:17:36.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.230+0000 7f946b82a740 10 datascan.find_or_create_dirfrag: read_version = 8 2024-04-04T06:17:36.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.230+0000 7f946b82a740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-04-04T06:17:36.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.230+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:54 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7e7400 con 0x561e1e7e6c00 2024-04-04T06:17:36.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.231+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.3 v2:172.21.15.159:6820/2070760205 14 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1e7e6c00 2024-04-04T06:17:36.226 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.231+0000 7f946b82a740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-04T06:17:36.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.231+0000 7f946b82a740 20 datascan.read_dentry: bravo_head not found in result 2024-04-04T06:17:36.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.231+0000 7f946b82a740 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-04-04T06:17:36.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.231+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:55 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fb400 con 0x561e1e7e6c00 2024-04-04T06:17:36.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.233+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.3 v2:172.21.15.159:6820/2070760205 15 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v116'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1e7e6c00 2024-04-04T06:17:36.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.233+0000 7f946b82a740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-04-04T06:17:36.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.233+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:56 17.e 17:70000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fbc00 con 0x561e1f38f400 2024-04-04T06:17:36.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.234+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.2 v2:172.21.15.159:6808/2619731047 3 ==== osd_op_reply(56 [pgnls-filter start_epoch 116 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1f38f400 2024-04-04T06:17:36.228 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.234+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:57 17.2e 17:74000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fac00 con 0x561e1e7f0400 2024-04-04T06:17:36.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.235+0000 7f9468000640 1 -- 172.21.15.159:0/2978730605 <== osd.6 v2:172.21.15.178:6826/198949157 2 ==== osd_op_reply(57 [pgnls-filter start_epoch 116 out=49b] v113'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce780 con 0x561e1e7f0400 2024-04-04T06:17:36.230 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.235+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:58 17.1e 17:78000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e80c400 con 0x561e1e7e6000 2024-04-04T06:17:36.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.235+0000 7f9468000640 1 -- 172.21.15.159:0/2978730605 <== osd.1 v2:172.21.15.159:6816/3765338297 6 ==== osd_op_reply(58 [pgnls-filter start_epoch 116 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce780 con 0x561e1e7e6000 2024-04-04T06:17:36.231 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.235+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:59 17.3e 17:7c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f1400 con 0x561e1e7e7c00 2024-04-04T06:17:36.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.236+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.4 v2:172.21.15.178:6812/4035991429 5 ==== osd_op_reply(59 [pgnls-filter start_epoch 116 out=49b] v113'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1e7e7c00 2024-04-04T06:17:36.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.237+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:60 17.1 17:80000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e80c800 con 0x561e1f31b800 2024-04-04T06:17:36.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.237+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.0 v2:172.21.15.159:6800/455382518 15 ==== osd_op_reply(60 [pgnls-filter start_epoch 116 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f31b800 2024-04-04T06:17:36.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.237+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:61 17.21 17:84000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f0000 con 0x561e1f31b800 2024-04-04T06:17:36.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.238+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.0 v2:172.21.15.159:6800/455382518 16 ==== osd_op_reply(61 [pgnls-filter start_epoch 116 out=49b] v113'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f31b800 2024-04-04T06:17:36.233 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.238+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:62 17.11 17:88000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e80cc00 con 0x561e1e7e6c00 2024-04-04T06:17:36.234 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.238+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.3 v2:172.21.15.159:6820/2070760205 16 ==== osd_op_reply(62 [pgnls-filter start_epoch 116 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1e7e6c00 2024-04-04T06:17:36.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.239+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:63 17.31 17:8c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f1000 con 0x561e1e7e6000 2024-04-04T06:17:36.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.239+0000 7f9468000640 1 -- 172.21.15.159:0/2978730605 <== osd.1 v2:172.21.15.159:6816/3765338297 7 ==== osd_op_reply(63 [pgnls-filter start_epoch 116 out=49b] v113'15 uv8 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce780 con 0x561e1e7e6000 2024-04-04T06:17:36.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.239+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:64 17.9 17:90000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e80d000 con 0x561e1e7e6c00 2024-04-04T06:17:36.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.240+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.3 v2:172.21.15.159:6820/2070760205 17 ==== osd_op_reply(64 [pgnls-filter start_epoch 116 out=49b] v113'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1e7e6c00 2024-04-04T06:17:36.235 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.240+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:65 17.29 17:94000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f0c00 con 0x561e1e7e7c00 2024-04-04T06:17:36.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.241+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.4 v2:172.21.15.178:6812/4035991429 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1e7e7c00 2024-04-04T06:17:36.236 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.241+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:66 17.19 17:98000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e80d400 con 0x561e1f31bc00 2024-04-04T06:17:36.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.242+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.5 v2:172.21.15.178:6802/63746765 5 ==== osd_op_reply(66 [pgnls-filter start_epoch 116 out=49b] v116'24 uv24 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1f31bc00 2024-04-04T06:17:36.238 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.243+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:67 17.39 17:9c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fa000 con 0x561e1f31bc00 2024-04-04T06:17:36.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.244+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.5 v2:172.21.15.178:6802/63746765 6 ==== osd_op_reply(67 [pgnls-filter start_epoch 116 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1f31bc00 2024-04-04T06:17:36.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.244+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e80d800 con 0x561e1e7f0400 2024-04-04T06:17:36.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.245+0000 7f9468000640 1 -- 172.21.15.159:0/2978730605 <== osd.6 v2:172.21.15.178:6826/198949157 3 ==== osd_op_reply(68 [pgnls-filter start_epoch 116 out=49b] v116'15 uv15 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce780 con 0x561e1e7f0400 2024-04-04T06:17:36.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.245+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:69 17.25 17:a4000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fb400 con 0x561e1e7e6000 2024-04-04T06:17:36.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.246+0000 7f9468000640 1 -- 172.21.15.159:0/2978730605 <== osd.1 v2:172.21.15.159:6816/3765338297 8 ==== osd_op_reply(69 [pgnls-filter start_epoch 116 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce780 con 0x561e1e7e6000 2024-04-04T06:17:36.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.246+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:70 17.15 17:a8000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e80dc00 con 0x561e1f31bc00 2024-04-04T06:17:36.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.247+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.5 v2:172.21.15.178:6802/63746765 7 ==== osd_op_reply(70 [pgnls-filter start_epoch 116 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1f31bc00 2024-04-04T06:17:36.242 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.247+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:71 17.35 17:ac000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7e6400 con 0x561e1e7e7c00 2024-04-04T06:17:36.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.248+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.4 v2:172.21.15.178:6812/4035991429 7 ==== osd_op_reply(71 [pgnls-filter start_epoch 116 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1e7e7c00 2024-04-04T06:17:36.243 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.248+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:72 17.d 17:b0000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fac00 con 0x561e1f38e000 2024-04-04T06:17:36.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.249+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.7 v2:172.21.15.178:6810/1066817938 11 ==== osd_op_reply(72 [pgnls-filter start_epoch 116 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f38e000 2024-04-04T06:17:36.244 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.249+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:73 17.2d 17:b4000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7e7000 con 0x561e1f38e000 2024-04-04T06:17:36.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.251+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.7 v2:172.21.15.178:6810/1066817938 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 116 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f38e000 2024-04-04T06:17:36.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.251+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:74 17.1d 17:b8000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fb400 con 0x561e1f38e000 2024-04-04T06:17:36.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.252+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.7 v2:172.21.15.178:6810/1066817938 13 ==== osd_op_reply(74 [pgnls-filter start_epoch 116 out=49b] v113'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f38e000 2024-04-04T06:17:36.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.252+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:75 17.3d 17:bc000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7e7800 con 0x561e1f38e000 2024-04-04T06:17:36.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.253+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.7 v2:172.21.15.178:6810/1066817938 14 ==== osd_op_reply(75 [pgnls-filter start_epoch 116 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f38e000 2024-04-04T06:17:36.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.253+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:76 17.3 17:c0000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e80c400 con 0x561e1e7e7c00 2024-04-04T06:17:36.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.254+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.4 v2:172.21.15.178:6812/4035991429 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 116 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1e7e7c00 2024-04-04T06:17:36.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.254+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:77 17.23 17:c4000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7e6800 con 0x561e1e7e7c00 2024-04-04T06:17:36.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.255+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.4 v2:172.21.15.178:6812/4035991429 9 ==== osd_op_reply(77 [pgnls-filter start_epoch 116 out=49b] v113'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1e7e7c00 2024-04-04T06:17:36.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.256+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:78 17.13 17:c8000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e80d800 con 0x561e1e7e6c00 2024-04-04T06:17:36.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.257+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.3 v2:172.21.15.159:6820/2070760205 18 ==== osd_op_reply(78 [pgnls-filter start_epoch 116 out=49b] v113'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1e7e6c00 2024-04-04T06:17:36.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.257+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:79 17.33 17:cc000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7e7400 con 0x561e1f38e000 2024-04-04T06:17:36.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.258+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.7 v2:172.21.15.178:6810/1066817938 15 ==== osd_op_reply(79 [pgnls-filter start_epoch 116 out=49b] v113'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f38e000 2024-04-04T06:17:36.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.258+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:80 17.b 17:d0000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f1800 con 0x561e1f31bc00 2024-04-04T06:17:36.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.259+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.5 v2:172.21.15.178:6802/63746765 8 ==== osd_op_reply(80 [pgnls-filter start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1f31bc00 2024-04-04T06:17:36.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.259+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:81 17.2b 17:d4000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e80c000 con 0x561e1e7e7c00 2024-04-04T06:17:36.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.260+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.4 v2:172.21.15.178:6812/4035991429 10 ==== osd_op_reply(81 [pgnls-filter start_epoch 116 out=49b] v113'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1e7e7c00 2024-04-04T06:17:36.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.260+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:82 17.1b 17:d8000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f1000 con 0x561e1e7e7c00 2024-04-04T06:17:36.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.261+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.4 v2:172.21.15.178:6812/4035991429 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1e7e7c00 2024-04-04T06:17:36.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.261+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:83 17.3b 17:dc000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7e6400 con 0x561e1f38f400 2024-04-04T06:17:36.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.262+0000 7f9469400640 1 -- 172.21.15.159:0/2978730605 <== osd.2 v2:172.21.15.159:6808/2619731047 4 ==== osd_op_reply(83 [pgnls-filter start_epoch 116 out=49b] v113'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce500 con 0x561e1f38f400 2024-04-04T06:17:36.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.262+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:84 17.7 17:e0000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1f38f800 con 0x561e1f31b800 2024-04-04T06:17:36.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.263+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.0 v2:172.21.15.159:6800/455382518 17 ==== osd_op_reply(84 [pgnls-filter start_epoch 116 out=49b] v113'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f31b800 2024-04-04T06:17:36.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.263+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:85 17.27 17:e4000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7e7000 con 0x561e1e7e7c00 2024-04-04T06:17:36.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.264+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.4 v2:172.21.15.178:6812/4035991429 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 116 out=49b] v113'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1e7e7c00 2024-04-04T06:17:36.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.264+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:86 17.17 17:e8000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7e6400 con 0x561e1e7f0400 2024-04-04T06:17:36.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.265+0000 7f9468000640 1 -- 172.21.15.159:0/2978730605 <== osd.6 v2:172.21.15.178:6826/198949157 4 ==== osd_op_reply(86 [pgnls-filter start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce780 con 0x561e1e7f0400 2024-04-04T06:17:36.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.265+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:87 17.37 17:ec000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7fac00 con 0x561e1f31b800 2024-04-04T06:17:36.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.266+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.0 v2:172.21.15.159:6800/455382518 18 ==== osd_op_reply(87 [pgnls-filter start_epoch 116 out=49b] v113'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f31b800 2024-04-04T06:17:36.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.266+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:88 17.f 17:f0000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f1800 con 0x561e1e7e6000 2024-04-04T06:17:36.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.267+0000 7f9468000640 1 -- 172.21.15.159:0/2978730605 <== osd.1 v2:172.21.15.159:6816/3765338297 9 ==== osd_op_reply(88 [pgnls-filter start_epoch 116 out=49b] v113'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce780 con 0x561e1e7e6000 2024-04-04T06:17:36.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.267+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:89 17.2f 17:f4000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f1400 con 0x561e1e7e7c00 2024-04-04T06:17:36.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.268+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.4 v2:172.21.15.178:6812/4035991429 13 ==== osd_op_reply(89 [pgnls-filter start_epoch 116 out=49b] v113'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1e7e7c00 2024-04-04T06:17:36.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.269+0000 7f9462600640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:90 17.1f 17:f8000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e80cc00 con 0x561e1e7f0400 2024-04-04T06:17:36.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.269+0000 7f9468000640 1 -- 172.21.15.159:0/2978730605 <== osd.6 v2:172.21.15.178:6826/198949157 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 116 out=49b] v113'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce780 con 0x561e1e7f0400 2024-04-04T06:17:36.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.270+0000 7f9463000640 1 -- 172.21.15.159:0/2978730605 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:91 17.3f 17:fc000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x561e1e7f0000 con 0x561e1f31b800 2024-04-04T06:17:36.265 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.270+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 <== osd.0 v2:172.21.15.159:6800/455382518 19 ==== osd_op_reply(91 [pgnls-filter start_epoch 116 out=49b] v113'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x561e1f1ce000 con 0x561e1f31b800 2024-04-04T06:17:36.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.271+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x561e1f31b800 msgr2=0x561e1f1ae100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.271+0000 7f946b82a740 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x561e1f31b800 0x561e1f1ae100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.271+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x561e1e7e6000 msgr2=0x561e1e7ec000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.271+0000 7f946b82a740 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x561e1e7e6000 0x561e1e7ec000 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.271+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x561e1f38f400 msgr2=0x561e1e7ed600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.271+0000 7f946b82a740 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x561e1f38f400 0x561e1e7ed600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.271+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x561e1e7e6c00 msgr2=0x561e1e7ec580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.271+0000 7f946b82a740 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x561e1e7e6c00 0x561e1e7ec580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.271+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 reap_dead start 2024-04-04T06:17:36.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.271+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x561e1e7e7c00 msgr2=0x561e1e7ecb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.271+0000 7f946b82a740 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x561e1e7e7c00 0x561e1e7ecb00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.271+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x561e1f31bc00 msgr2=0x561e1f1ac580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.271+0000 7f946b82a740 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x561e1f31bc00 0x561e1f1ac580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.271+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x561e1e7f0400 msgr2=0x561e1e7ed080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.271+0000 7f946b82a740 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x561e1e7f0400 0x561e1e7ed080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.271+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x561e1f38e000 msgr2=0x561e1f1ad080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.271+0000 7f946b82a740 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x561e1f38e000 0x561e1f1ad080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.272+0000 7f9468a00640 1 -- 172.21.15.159:0/2978730605 reap_dead start 2024-04-04T06:17:36.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.272+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x561e1f38fc00 msgr2=0x561e1f1af700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.272+0000 7f946b82a740 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x561e1f38fc00 0x561e1f1af700 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x561e1f1cce40 tx=0x561e1f1e7080 comp rx=0 tx=0).stop 2024-04-04T06:17:36.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.272+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e1f31b400 msgr2=0x561e1f1ae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.272+0000 7f946b82a740 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e1f31b400 0x561e1f1ae680 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x561e1f331860 tx=0x561e1f1e6d20 comp rx=0 tx=0).stop 2024-04-04T06:17:36.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.272+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 shutdown_connections 2024-04-04T06:17:36.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.272+0000 7f946b82a740 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e1f31b400 0x561e1f1ae680 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.272+0000 7f946b82a740 1 --2- 172.21.15.159:0/2978730605 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x561e1f38fc00 0x561e1f1af700 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.272+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 >> 172.21.15.159:0/2978730605 conn(0x561e1f1b5400 msgr2=0x561e1e539800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:36.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.273+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 shutdown_connections 2024-04-04T06:17:36.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.273+0000 7f946b82a740 1 -- 172.21.15.159:0/2978730605 wait complete. 2024-04-04T06:17:36.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.274+0000 7f946b82a740 1 -- 172.21.15.159:0/1100290808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e1f1b4800 msgr2=0x561e1f1acb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.274+0000 7f946b82a740 1 --2- 172.21.15.159:0/1100290808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e1f1b4800 0x561e1f1acb00 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x561e1f27b980 tx=0x561e1f1e7020 comp rx=0 tx=0).stop 2024-04-04T06:17:36.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.274+0000 7f946b82a740 1 -- 172.21.15.159:0/1100290808 shutdown_connections 2024-04-04T06:17:36.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.274+0000 7f946b82a740 1 --2- 172.21.15.159:0/1100290808 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x561e1f1b5c00 0x561e1f1adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.274+0000 7f946b82a740 1 --2- 172.21.15.159:0/1100290808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e1f1b4800 0x561e1f1acb00 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.274+0000 7f946b82a740 1 --2- 172.21.15.159:0/1100290808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e1f1b4400 0x561e1f1ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.274+0000 7f946b82a740 1 -- 172.21.15.159:0/1100290808 >> 172.21.15.159:0/1100290808 conn(0x561e1e44d000 msgr2=0x561e1e537800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:36.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.274+0000 7f946b82a740 1 -- 172.21.15.159:0/1100290808 shutdown_connections 2024-04-04T06:17:36.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.274+0000 7f946b82a740 1 -- 172.21.15.159:0/1100290808 wait complete. 2024-04-04T06:17:36.279 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.216210, result: 2024-04-04T06:17:36.280 DEBUG:teuthology.orchestra.run.smithi159:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-04T06:17:36.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.390+0000 7fa1b7853740 1 -- start start 2024-04-04T06:17:36.386 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.390+0000 7fa1b7853740 1 Processor -- start 2024-04-04T06:17:36.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.391+0000 7fa1b7853740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563b5ddbcc00 0x563b5eb1c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.391+0000 7fa1b7853740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b5ddbd800 0x563b5eb1c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.391+0000 7fa1b7853740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b5ddbdc00 0x563b5eb1cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.391+0000 7fa1b7853740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563b5eb24180 con 0x563b5ddbdc00 2024-04-04T06:17:36.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.391+0000 7fa1b7853740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563b5eb24300 con 0x563b5ddbd800 2024-04-04T06:17:36.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.391+0000 7fa1b7853740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x563b5eb24480 con 0x563b5ddbcc00 2024-04-04T06:17:36.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.391+0000 7fa1b5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b5ddbdc00 0x563b5eb1cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.391+0000 7fa1b5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b5ddbdc00 0x563b5eb1cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41984/0 (socket says 172.21.15.159:41984) 2024-04-04T06:17:36.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.391+0000 7fa1b5e00640 1 -- 172.21.15.159:0/3130279389 learned_addr learned my addr 172.21.15.159:0/3130279389 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:36.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.392+0000 7fa1b4a00640 1 --2- 172.21.15.159:0/3130279389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b5ddbd800 0x563b5eb1c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.392+0000 7fa1b5400640 1 --2- 172.21.15.159:0/3130279389 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563b5ddbcc00 0x563b5eb1c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.392+0000 7fa1b5e00640 1 -- 172.21.15.159:0/3130279389 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563b5ddbcc00 msgr2=0x563b5eb1c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:36.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.392+0000 7fa1b5e00640 1 --2- 172.21.15.159:0/3130279389 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563b5ddbcc00 0x563b5eb1c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.392+0000 7fa1b5e00640 1 -- 172.21.15.159:0/3130279389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b5ddbd800 msgr2=0x563b5eb1c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:36.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.392+0000 7fa1b5e00640 1 --2- 172.21.15.159:0/3130279389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b5ddbd800 0x563b5eb1c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.392+0000 7fa1b5e00640 1 -- 172.21.15.159:0/3130279389 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b5dde9680 con 0x563b5ddbdc00 2024-04-04T06:17:36.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.392+0000 7fa1b5e00640 1 --2- 172.21.15.159:0/3130279389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b5ddbdc00 0x563b5eb1cb00 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x563b5ebe1050 tx=0x563b5eb56e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b5bb333d53db9e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.392+0000 7fa1b2200640 1 -- 172.21.15.159:0/3130279389 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563b5dde29c0 con 0x563b5ddbdc00 2024-04-04T06:17:36.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.393+0000 7fa1b2200640 1 -- 172.21.15.159:0/3130279389 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563b5de061c0 con 0x563b5ddbdc00 2024-04-04T06:17:36.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.393+0000 7fa1b2200640 1 -- 172.21.15.159:0/3130279389 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563b5dde2680 con 0x563b5ddbdc00 2024-04-04T06:17:36.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.393+0000 7fa1b7853740 1 -- 172.21.15.159:0/3130279389 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b5dde9860 con 0x563b5ddbdc00 2024-04-04T06:17:36.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.393+0000 7fa1b7853740 1 -- 172.21.15.159:0/3130279389 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({fsmap=0}) -- 0x563b5dde92c0 con 0x563b5ddbdc00 2024-04-04T06:17:36.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.393+0000 7fa1b7853740 4 waiting for MDS map... 2024-04-04T06:17:36.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.393+0000 7fa1b2200640 1 -- 172.21.15.159:0/3130279389 <== mon.0 v2:172.21.15.159:3300/0 4 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x563b5ec8c200 con 0x563b5ddbdc00 2024-04-04T06:17:36.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.393+0000 7fa1b2200640 1 -- 172.21.15.159:0/3130279389 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 108) ==== 2885+0+0 (secure 0 0 0) 0x563b5de15180 con 0x563b5ddbdc00 2024-04-04T06:17:36.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.393+0000 7fa1b7853740 4 Got MDS map 108 2024-04-04T06:17:36.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.393+0000 7fa1b7853740 4 datascan.main: Using metadata pool output 2024-04-04T06:17:36.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.393+0000 7fa1b7853740 4 datascan.main: connecting to RADOS... 2024-04-04T06:17:36.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.394+0000 7fa1b7853740 1 Processor -- start 2024-04-04T06:17:36.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.394+0000 7fa1b7853740 1 -- start start 2024-04-04T06:17:36.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.395+0000 7fa1b7853740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b5ecf0000 0x563b5eb1d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.395+0000 7fa1b7853740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563b5ecf0400 0x563b5eb1d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.395+0000 7fa1b7853740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b5ecf0800 0x563b5eb1db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.395+0000 7fa1b7853740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563b5eb24600 con 0x563b5ecf0800 2024-04-04T06:17:36.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.395+0000 7fa1b7853740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563b5eb24780 con 0x563b5ecf0000 2024-04-04T06:17:36.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.395+0000 7fa1b7853740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x563b5eb24900 con 0x563b5ecf0400 2024-04-04T06:17:36.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.395+0000 7fa1b4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b5ecf0800 0x563b5eb1db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.395+0000 7fa1b4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b5ecf0800 0x563b5eb1db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41990/0 (socket says 172.21.15.159:41990) 2024-04-04T06:17:36.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.395+0000 7fa1b4a00640 1 -- 172.21.15.159:0/2596693390 learned_addr learned my addr 172.21.15.159:0/2596693390 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:36.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.396+0000 7fa1b4a00640 1 --2- 172.21.15.159:0/2596693390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b5ecf0000 0x563b5eb1d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.396+0000 7fa1b4a00640 1 -- 172.21.15.159:0/2596693390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563b5ecf0400 msgr2=0x563b5eb1d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:36.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.396+0000 7fa1b5400640 1 --2- 172.21.15.159:0/2596693390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563b5ecf0400 0x563b5eb1d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.396+0000 7fa1b4a00640 1 --2- 172.21.15.159:0/2596693390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563b5ecf0400 0x563b5eb1d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.396+0000 7fa1b4a00640 1 -- 172.21.15.159:0/2596693390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b5ecf0000 msgr2=0x563b5eb1d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:36.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.396+0000 7fa1b4a00640 1 --2- 172.21.15.159:0/2596693390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b5ecf0000 0x563b5eb1d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.396+0000 7fa1b4a00640 1 -- 172.21.15.159:0/2596693390 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b5dde94a0 con 0x563b5ecf0800 2024-04-04T06:17:36.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.396+0000 7fa1b4a00640 1 --2- 172.21.15.159:0/2596693390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b5ecf0800 0x563b5eb1db80 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x563b5eba6810 tx=0x563b5eca0cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40c605540e20d08 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.396+0000 7fa1afa00640 1 -- 172.21.15.159:0/2596693390 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563b5dde29c0 con 0x563b5ecf0800 2024-04-04T06:17:36.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.396+0000 7fa1afa00640 1 -- 172.21.15.159:0/2596693390 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563b5de06e00 con 0x563b5ecf0800 2024-04-04T06:17:36.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.396+0000 7fa1afa00640 1 -- 172.21.15.159:0/2596693390 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563b5dde2680 con 0x563b5ecf0800 2024-04-04T06:17:36.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.397+0000 7fa1b7853740 1 -- 172.21.15.159:0/2596693390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b5ecf0800 msgr2=0x563b5eb1db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.397+0000 7fa1b7853740 1 --2- 172.21.15.159:0/2596693390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b5ecf0800 0x563b5eb1db80 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x563b5eba6810 tx=0x563b5eca0cc0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.397+0000 7fa1b7853740 1 -- 172.21.15.159:0/2596693390 shutdown_connections 2024-04-04T06:17:36.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.397+0000 7fa1b7853740 1 --2- 172.21.15.159:0/2596693390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b5ecf0800 0x563b5eb1db80 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.397+0000 7fa1b7853740 1 --2- 172.21.15.159:0/2596693390 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563b5ecf0400 0x563b5eb1d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.397+0000 7fa1b7853740 1 --2- 172.21.15.159:0/2596693390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b5ecf0000 0x563b5eb1d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.397+0000 7fa1b7853740 1 -- 172.21.15.159:0/2596693390 >> 172.21.15.159:0/2596693390 conn(0x563b5ebafc00 msgr2=0x563b5dea7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:36.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.397+0000 7fa1b7853740 1 -- 172.21.15.159:0/2596693390 shutdown_connections 2024-04-04T06:17:36.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.397+0000 7fa1b7853740 1 -- 172.21.15.159:0/2596693390 wait complete. 2024-04-04T06:17:36.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.398+0000 7fa1b7853740 1 Processor -- start 2024-04-04T06:17:36.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.398+0000 7fa1b7853740 1 -- start start 2024-04-04T06:17:36.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.399+0000 7fa1b7853740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563b5ecf0000 0x563b5eb1d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.399+0000 7fa1b7853740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b5ecf0400 0x563b5eb1d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.399+0000 7fa1b7853740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b5ecf0800 0x563b5eb1db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.399+0000 7fa1b7853740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563b5eb25c80 con 0x563b5ecf0800 2024-04-04T06:17:36.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.399+0000 7fa1b7853740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563b5eb25b00 con 0x563b5ecf0400 2024-04-04T06:17:36.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.399+0000 7fa1b7853740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x563b5eb25980 con 0x563b5ecf0000 2024-04-04T06:17:36.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.399+0000 7fa1b4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b5ecf0800 0x563b5eb1db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.399+0000 7fa1b4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b5ecf0800 0x563b5eb1db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:41996/0 (socket says 172.21.15.159:41996) 2024-04-04T06:17:36.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.399+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 learned_addr learned my addr 172.21.15.159:0/126768414 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:36.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.400+0000 7fa1b5400640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b5ecf0400 0x563b5eb1d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.400+0000 7fa1b4a00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563b5ecf0000 0x563b5eb1d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.400+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563b5ecf0000 msgr2=0x563b5eb1d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.400+0000 7fa1b4a00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563b5ecf0000 0x563b5eb1d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.400+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b5ecf0400 msgr2=0x563b5eb1d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.400+0000 7fa1b4a00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b5ecf0400 0x563b5eb1d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.400+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b5dde90e0 con 0x563b5ecf0800 2024-04-04T06:17:36.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.400+0000 7fa1b4a00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b5ecf0800 0x563b5eb1db80 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x563b5eba7ad0 tx=0x563b5eca0c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.400+0000 7fa1adc00640 1 -- 172.21.15.159:0/126768414 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563b5dde24e0 con 0x563b5ecf0800 2024-04-04T06:17:36.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.400+0000 7fa1adc00640 1 -- 172.21.15.159:0/126768414 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563b5de06c40 con 0x563b5ecf0800 2024-04-04T06:17:36.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.400+0000 7fa1adc00640 1 -- 172.21.15.159:0/126768414 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563b5dde29c0 con 0x563b5ecf0800 2024-04-04T06:17:36.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.400+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563b5dde9a40 con 0x563b5ecf0800 2024-04-04T06:17:36.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.401+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b5dde83c0 con 0x563b5ecf0800 2024-04-04T06:17:36.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.401+0000 7fa1b7853740 4 datascan.init: resolving metadata pool 16 2024-04-04T06:17:36.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.402+0000 7fa1adc00640 1 -- 172.21.15.159:0/126768414 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x563b5ecb8c00 con 0x563b5ecf0800 2024-04-04T06:17:36.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.403+0000 7fa1adc00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x563b5e138000 0x563b5eb1e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.403+0000 7fa1adc00640 1 -- 172.21.15.159:0/126768414 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x563b5ed04a00 con 0x563b5ecf0800 2024-04-04T06:17:36.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.403+0000 7fa1b7853740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-04T06:17:36.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.403+0000 7fa1b7853740 4 datascan.main: resolving metadata pool 16 2024-04-04T06:17:36.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.403+0000 7fa1b7853740 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x563b5ecf0c00 0x563b5eb1e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.403+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:1 16.0 16:00000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ecf1000 con 0x563b5ecf0c00 2024-04-04T06:17:36.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.403+0000 7fa1b4a00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x563b5ecf0c00 0x563b5eb1e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.403+0000 7fa1b5400640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x563b5e138000 0x563b5eb1e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.404+0000 7fa1b4a00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x563b5ecf0c00 0x563b5eb1e680 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.404+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.0 v2:172.21.15.159:6800/455382518 1 ==== osd_op_reply(1 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5ecf0c00 2024-04-04T06:17:36.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.404+0000 7fa1b4a00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x563b5ecf1000 0x563b5e146000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.404+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:2 16.10 16:08000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5e139000 con 0x563b5ecf1000 2024-04-04T06:17:36.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.405+0000 7fa1b5400640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x563b5e138000 0x563b5eb1e100 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x563b5eb3cd20 tx=0x563b5eca1140 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.405+0000 7fa1b5e00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x563b5ecf1000 0x563b5e146000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.405+0000 7fa1b5e00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x563b5ecf1000 0x563b5e146000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.406+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 1 ==== osd_op_reply(2 [pgnls start_epoch 116 out=177b] v115'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.406+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:3 16.8 16:10000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5e139000 con 0x563b5ecf0c00 2024-04-04T06:17:36.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.406+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.0 v2:172.21.15.159:6800/455382518 2 ==== osd_op_reply(3 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5ecf0c00 2024-04-04T06:17:36.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.406+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:4 16.18 16:18000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5e139000 con 0x563b5ecf0c00 2024-04-04T06:17:36.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.406+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.0 v2:172.21.15.159:6800/455382518 3 ==== osd_op_reply(4 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5ecf0c00 2024-04-04T06:17:36.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.406+0000 7fa1b4a00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x563b5e139000 0x563b5e146580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.406+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:5 16.4 16:20000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5e139400 con 0x563b5e139000 2024-04-04T06:17:36.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.407+0000 7fa1b5400640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x563b5e139000 0x563b5e146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.407+0000 7fa1b5400640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x563b5e139000 0x563b5e146580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.407+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.3 v2:172.21.15.159:6820/2070760205 1 ==== osd_op_reply(5 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139000 2024-04-04T06:17:36.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.407+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:6 16.14 16:28000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5e139400 con 0x563b5ecf0c00 2024-04-04T06:17:36.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.408+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.0 v2:172.21.15.159:6800/455382518 4 ==== osd_op_reply(6 [pgnls start_epoch 116 out=149b] v116'42 uv42 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5ecf0c00 2024-04-04T06:17:36.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.408+0000 7fa1b4a00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x563b5e139400 0x563b5e146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.408+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:7 16.c 16:30000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5e139800 con 0x563b5e139400 2024-04-04T06:17:36.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.408+0000 7fa1b4a00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x563b5e139400 0x563b5e146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.409+0000 7fa1b4a00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x563b5e139400 0x563b5e146b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.410+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.5 v2:172.21.15.178:6802/63746765 1 ==== osd_op_reply(7 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5e139400 2024-04-04T06:17:36.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.410+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:8 16.1c 16:38000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5e139800 con 0x563b5ecf1000 2024-04-04T06:17:36.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.411+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 2 ==== osd_op_reply(8 [pgnls start_epoch 116 out=73b] v115'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.411+0000 7fa1b5e00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x563b5e139800 0x563b5e147080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.411+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:9 16.2 16:40000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5e139c00 con 0x563b5e139800 2024-04-04T06:17:36.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.411+0000 7fa1b5e00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x563b5e139800 0x563b5e147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.412+0000 7fa1b5e00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x563b5e139800 0x563b5e147080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.413+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.4 v2:172.21.15.178:6812/4035991429 1 ==== osd_op_reply(9 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5e139800 2024-04-04T06:17:36.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.413+0000 7fa1b5e00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x563b5e139c00 0x563b5e147600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.413+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:10 16.12 16:48000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc0000 con 0x563b5e139c00 2024-04-04T06:17:36.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.413+0000 7fa1b5400640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x563b5e139c00 0x563b5e147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.414+0000 7fa1b5400640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x563b5e139c00 0x563b5e147600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.415+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.6 v2:172.21.15.178:6826/198949157 1 ==== osd_op_reply(10 [pgnls start_epoch 116 out=81b] v115'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139c00 2024-04-04T06:17:36.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.415+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:11 16.a 16:50000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc0000 con 0x563b5ecf1000 2024-04-04T06:17:36.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.416+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 3 ==== osd_op_reply(11 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.416+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:12 16.1a 16:58000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc0000 con 0x563b5e139000 2024-04-04T06:17:36.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.416+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.3 v2:172.21.15.159:6820/2070760205 2 ==== osd_op_reply(12 [pgnls start_epoch 116 out=49b] v116'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139000 2024-04-04T06:17:36.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.416+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:13 16.6 16:60000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc0000 con 0x563b5e139c00 2024-04-04T06:17:36.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.417+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.6 v2:172.21.15.178:6826/198949157 2 ==== osd_op_reply(13 [pgnls start_epoch 116 out=77b] v115'10 uv10 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139c00 2024-04-04T06:17:36.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.417+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:14 16.16 16:68000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc0000 con 0x563b5e139000 2024-04-04T06:17:36.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.418+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.3 v2:172.21.15.159:6820/2070760205 3 ==== osd_op_reply(14 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139000 2024-04-04T06:17:36.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.418+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:15 16.e 16:70000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc0000 con 0x563b5ecf1000 2024-04-04T06:17:36.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.418+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 4 ==== osd_op_reply(15 [pgnls start_epoch 116 out=73b] v115'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.418+0000 7fa1b5e00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x563b5ebc0000 0x563b5e147b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.418+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:16 16.1e 16:78000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc0400 con 0x563b5ebc0000 2024-04-04T06:17:36.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.419+0000 7fa1b4a00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x563b5ebc0000 0x563b5e147b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.420+0000 7fa1b4a00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x563b5ebc0000 0x563b5e147b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.420+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.7 v2:172.21.15.178:6810/1066817938 1 ==== osd_op_reply(16 [pgnls start_epoch 116 out=49b] v116'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5ebc0000 2024-04-04T06:17:36.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.420+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:17 16.1 16:80000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc0400 con 0x563b5ecf1000 2024-04-04T06:17:36.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.421+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 5 ==== osd_op_reply(17 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.421+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:18 16.11 16:88000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc0400 con 0x563b5e139400 2024-04-04T06:17:36.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.422+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.5 v2:172.21.15.178:6802/63746765 2 ==== osd_op_reply(18 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5e139400 2024-04-04T06:17:36.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.422+0000 7fa1b4a00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x563b5ebc0400 0x563b5e148100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:36.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.422+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:19 16.9 16:90000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc1400 con 0x563b5ebc0400 2024-04-04T06:17:36.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.422+0000 7fa1b5e00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x563b5ebc0400 0x563b5e148100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:36.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.422+0000 7fa1b5e00640 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x563b5ebc0400 0x563b5e148100 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:36.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.423+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.1 v2:172.21.15.159:6816/3765338297 1 ==== osd_op_reply(19 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ebc0400 2024-04-04T06:17:36.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.423+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:20 16.19 16:98000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc1400 con 0x563b5e139800 2024-04-04T06:17:36.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.424+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.4 v2:172.21.15.178:6812/4035991429 2 ==== osd_op_reply(20 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5e139800 2024-04-04T06:17:36.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.424+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:21 16.5 16:a0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc1400 con 0x563b5ecf0c00 2024-04-04T06:17:36.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.424+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.0 v2:172.21.15.159:6800/455382518 5 ==== osd_op_reply(21 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5ecf0c00 2024-04-04T06:17:36.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.424+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:22 16.15 16:a8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc1400 con 0x563b5ecf0c00 2024-04-04T06:17:36.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.425+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.0 v2:172.21.15.159:6800/455382518 6 ==== osd_op_reply(22 [pgnls start_epoch 116 out=97b] v115'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5ecf0c00 2024-04-04T06:17:36.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.425+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:23 16.d 16:b0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc1400 con 0x563b5e139c00 2024-04-04T06:17:36.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.425+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.6 v2:172.21.15.178:6826/198949157 3 ==== osd_op_reply(23 [pgnls start_epoch 116 out=74b] v112'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139c00 2024-04-04T06:17:36.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.425+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:24 16.1d 16:b8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc1400 con 0x563b5ecf1000 2024-04-04T06:17:36.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.426+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 6 ==== osd_op_reply(24 [pgnls start_epoch 116 out=73b] v115'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.426+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:25 16.3 16:c0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc1400 con 0x563b5e139400 2024-04-04T06:17:36.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.427+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.5 v2:172.21.15.178:6802/63746765 3 ==== osd_op_reply(25 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5e139400 2024-04-04T06:17:36.422 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.427+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:26 16.13 16:c8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc1400 con 0x563b5ebc0400 2024-04-04T06:17:36.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.428+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.1 v2:172.21.15.159:6816/3765338297 2 ==== osd_op_reply(26 [pgnls start_epoch 116 out=98b] v115'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ebc0400 2024-04-04T06:17:36.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.428+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:27 16.b 16:d0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc1400 con 0x563b5ebc0400 2024-04-04T06:17:36.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.428+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.1 v2:172.21.15.159:6816/3765338297 3 ==== osd_op_reply(27 [pgnls start_epoch 116 out=76b] v116'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ebc0400 2024-04-04T06:17:36.423 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.428+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:28 16.1b 16:d8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc1400 con 0x563b5e139400 2024-04-04T06:17:36.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.429+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.5 v2:172.21.15.178:6802/63746765 4 ==== osd_op_reply(28 [pgnls start_epoch 116 out=49b] v116'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5e139400 2024-04-04T06:17:36.424 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.429+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:29 16.7 16:e0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc1400 con 0x563b5e139000 2024-04-04T06:17:36.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.430+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.3 v2:172.21.15.159:6820/2070760205 4 ==== osd_op_reply(29 [pgnls start_epoch 116 out=105b] v116'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139000 2024-04-04T06:17:36.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.430+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:30 16.17 16:e8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc1400 con 0x563b5ecf1000 2024-04-04T06:17:36.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.430+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 7 ==== osd_op_reply(30 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.425 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.430+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:31 16.f 16:f0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc1400 con 0x563b5ecf0c00 2024-04-04T06:17:36.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.431+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.0 v2:172.21.15.159:6800/455382518 7 ==== osd_op_reply(31 [pgnls start_epoch 116 out=73b] v116'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5ecf0c00 2024-04-04T06:17:36.426 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.431+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:32 16.1f 16:f8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc1400 con 0x563b5e139400 2024-04-04T06:17:36.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.432+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.5 v2:172.21.15.178:6802/63746765 5 ==== osd_op_reply(32 [pgnls start_epoch 116 out=101b] v116'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5e139400 2024-04-04T06:17:36.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.432+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-04T06:17:36.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.432+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:33 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc6000 con 0x563b5ecf1000 2024-04-04T06:17:36.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.433+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.433+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-04T06:17:36.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.433+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:34 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc6400 con 0x563b5ecf1000 2024-04-04T06:17:36.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.433+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.433+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-04T06:17:36.428 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.433+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:35 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc6800 con 0x563b5ecf1000 2024-04-04T06:17:36.429 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.434+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) ==== 164+0+1169 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.434+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-04T06:17:36.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.434+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:36 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc6c00 con 0x563b5ecf1000 2024-04-04T06:17:36.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.434+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.434+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-04T06:17:36.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.434+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:37 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7000 con 0x563b5ecf1000 2024-04-04T06:17:36.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.435+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.435+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-04T06:17:36.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.435+0000 7fa1b7853740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-04T06:17:36.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.435+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-04T06:17:36.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.435+0000 7fa1b7853740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-04T06:17:36.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.435+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-04T06:17:36.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.435+0000 7fa1b7853740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-04T06:17:36.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.435+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-04T06:17:36.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.435+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:38 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5ecf0c00 2024-04-04T06:17:36.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.435+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.0 v2:172.21.15.159:6800/455382518 8 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv37 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5ecf0c00 2024-04-04T06:17:36.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.435+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-04T06:17:36.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.435+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:39 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc6000 con 0x563b5ecf1000 2024-04-04T06:17:36.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.436+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.436+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-04-04T06:17:36.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.436+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:40 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc6400 con 0x563b5e139c00 2024-04-04T06:17:36.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.437+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.6 v2:172.21.15.178:6826/198949157 4 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=592b] v0'0 uv3 ondisk = 0) ==== 164+0+592 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139c00 2024-04-04T06:17:36.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.437+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-04T06:17:36.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.437+0000 7fa1b7853740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-04T06:17:36.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.437+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-04T06:17:36.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.437+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:41 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc6800 con 0x563b5ecf1000 2024-04-04T06:17:36.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.438+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 14 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.438+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-04T06:17:36.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.438+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:42 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc6c00 con 0x563b5ecf0c00 2024-04-04T06:17:36.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.438+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.0 v2:172.21.15.159:6800/455382518 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5ecf0c00 2024-04-04T06:17:36.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.438+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-04T06:17:36.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.438+0000 7fa1b7853740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-04T06:17:36.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.438+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-04T06:17:36.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.438+0000 7fa1b7853740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-04T06:17:36.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.438+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-04T06:17:36.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.438+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:43 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc1000 con 0x563b5ecf1000 2024-04-04T06:17:36.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.439+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.439+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-04T06:17:36.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.439+0000 7fa1b7853740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-04T06:17:36.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.439+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-04T06:17:36.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.439+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:44 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc0c00 con 0x563b5ebc0400 2024-04-04T06:17:36.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.440+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.1 v2:172.21.15.159:6816/3765338297 4 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ebc0400 2024-04-04T06:17:36.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.440+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-04T06:17:36.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.440+0000 7fa1b7853740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-04T06:17:36.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.440+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-04T06:17:36.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.440+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:45 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc0800 con 0x563b5e139000 2024-04-04T06:17:36.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.440+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.3 v2:172.21.15.159:6820/2070760205 5 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139000 2024-04-04T06:17:36.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.440+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-04-04T06:17:36.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.440+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:46 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7800 con 0x563b5e139000 2024-04-04T06:17:36.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.441+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.3 v2:172.21.15.159:6820/2070760205 6 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1683b] v0'0 uv9 ondisk = 0) ==== 164+0+1683 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139000 2024-04-04T06:17:36.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.441+0000 7fa1b7853740 20 datascan.scan_links: injected ino detected 2024-04-04T06:17:36.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.441+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-04T06:17:36.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.441+0000 7fa1b7853740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-04T06:17:36.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.441+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-04T06:17:36.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.441+0000 7fa1b7853740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-04T06:17:36.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.441+0000 7fa1b7853740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-04T06:17:36.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.441+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:47 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7c00 con 0x563b5e139400 2024-04-04T06:17:36.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.442+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.5 v2:172.21.15.178:6802/63746765 6 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5e139400 2024-04-04T06:17:36.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.442+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:48 16.0 16:00000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5ecf0c00 2024-04-04T06:17:36.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.443+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.0 v2:172.21.15.159:6800/455382518 10 ==== osd_op_reply(48 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5ecf0c00 2024-04-04T06:17:36.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.443+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:49 16.10 16:08000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5ecf1000 2024-04-04T06:17:36.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.444+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 16 ==== osd_op_reply(49 [pgnls start_epoch 116 out=177b] v115'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.444+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:50 16.8 16:10000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5ecf0c00 2024-04-04T06:17:36.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.444+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.0 v2:172.21.15.159:6800/455382518 11 ==== osd_op_reply(50 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5ecf0c00 2024-04-04T06:17:36.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.444+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:51 16.18 16:18000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5ecf0c00 2024-04-04T06:17:36.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.445+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.0 v2:172.21.15.159:6800/455382518 12 ==== osd_op_reply(51 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5ecf0c00 2024-04-04T06:17:36.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.445+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:52 16.4 16:20000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5e139000 2024-04-04T06:17:36.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.445+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.3 v2:172.21.15.159:6820/2070760205 7 ==== osd_op_reply(52 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139000 2024-04-04T06:17:36.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.445+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:53 16.14 16:28000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5ecf0c00 2024-04-04T06:17:36.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.446+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.0 v2:172.21.15.159:6800/455382518 13 ==== osd_op_reply(53 [pgnls start_epoch 116 out=149b] v116'42 uv42 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5ecf0c00 2024-04-04T06:17:36.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.446+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:54 16.c 16:30000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5e139400 2024-04-04T06:17:36.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.447+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.5 v2:172.21.15.178:6802/63746765 7 ==== osd_op_reply(54 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5e139400 2024-04-04T06:17:36.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.447+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:55 16.1c 16:38000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5ecf1000 2024-04-04T06:17:36.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.448+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 17 ==== osd_op_reply(55 [pgnls start_epoch 116 out=73b] v115'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.448+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:56 16.2 16:40000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5e139800 2024-04-04T06:17:36.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.449+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.4 v2:172.21.15.178:6812/4035991429 3 ==== osd_op_reply(56 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5e139800 2024-04-04T06:17:36.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.449+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:57 16.12 16:48000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5e139c00 2024-04-04T06:17:36.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.450+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.6 v2:172.21.15.178:6826/198949157 5 ==== osd_op_reply(57 [pgnls start_epoch 116 out=81b] v115'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139c00 2024-04-04T06:17:36.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.450+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:58 16.a 16:50000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5ecf1000 2024-04-04T06:17:36.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.451+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 18 ==== osd_op_reply(58 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.451+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:59 16.1a 16:58000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5e139000 2024-04-04T06:17:36.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.451+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.3 v2:172.21.15.159:6820/2070760205 8 ==== osd_op_reply(59 [pgnls start_epoch 116 out=49b] v116'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139000 2024-04-04T06:17:36.446 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.451+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:60 16.6 16:60000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5e139c00 2024-04-04T06:17:36.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.452+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.6 v2:172.21.15.178:6826/198949157 6 ==== osd_op_reply(60 [pgnls start_epoch 116 out=77b] v115'10 uv10 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139c00 2024-04-04T06:17:36.447 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.452+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:61 16.16 16:68000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5e139000 2024-04-04T06:17:36.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.453+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.3 v2:172.21.15.159:6820/2070760205 9 ==== osd_op_reply(61 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139000 2024-04-04T06:17:36.448 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.453+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:62 16.e 16:70000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5ecf1000 2024-04-04T06:17:36.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.453+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 19 ==== osd_op_reply(62 [pgnls start_epoch 116 out=73b] v115'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.453+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- osd_op(unknown.0.0:63 16.1e 16:78000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5ebc0000 2024-04-04T06:17:36.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.454+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.7 v2:172.21.15.178:6810/1066817938 2 ==== osd_op_reply(63 [pgnls start_epoch 116 out=49b] v116'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5ebc0000 2024-04-04T06:17:36.449 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.454+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:64 16.1 16:80000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5ecf1000 2024-04-04T06:17:36.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.455+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 20 ==== osd_op_reply(64 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.455+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:65 16.11 16:88000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5e139400 2024-04-04T06:17:36.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.456+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.5 v2:172.21.15.178:6802/63746765 8 ==== osd_op_reply(65 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5e139400 2024-04-04T06:17:36.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.456+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:66 16.9 16:90000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5ebc0400 2024-04-04T06:17:36.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.457+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.1 v2:172.21.15.159:6816/3765338297 5 ==== osd_op_reply(66 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ebc0400 2024-04-04T06:17:36.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.457+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- osd_op(unknown.0.0:67 16.19 16:98000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5e139800 2024-04-04T06:17:36.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.458+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.4 v2:172.21.15.178:6812/4035991429 4 ==== osd_op_reply(67 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5e139800 2024-04-04T06:17:36.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.458+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:68 16.5 16:a0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5ecf0c00 2024-04-04T06:17:36.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.458+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.0 v2:172.21.15.159:6800/455382518 14 ==== osd_op_reply(68 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5ecf0c00 2024-04-04T06:17:36.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.458+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:69 16.15 16:a8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5ecf0c00 2024-04-04T06:17:36.453 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.459+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.0 v2:172.21.15.159:6800/455382518 15 ==== osd_op_reply(69 [pgnls start_epoch 116 out=97b] v115'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5ecf0c00 2024-04-04T06:17:36.454 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.459+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:70 16.d 16:b0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5e139c00 2024-04-04T06:17:36.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.460+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.6 v2:172.21.15.178:6826/198949157 7 ==== osd_op_reply(70 [pgnls start_epoch 116 out=74b] v112'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139c00 2024-04-04T06:17:36.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.460+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:71 16.1d 16:b8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5ecf1000 2024-04-04T06:17:36.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.461+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 21 ==== osd_op_reply(71 [pgnls start_epoch 116 out=73b] v115'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.461+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:72 16.3 16:c0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5e139400 2024-04-04T06:17:36.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.462+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.5 v2:172.21.15.178:6802/63746765 9 ==== osd_op_reply(72 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5e139400 2024-04-04T06:17:36.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.462+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:73 16.13 16:c8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5ebc0400 2024-04-04T06:17:36.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.462+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.1 v2:172.21.15.159:6816/3765338297 6 ==== osd_op_reply(73 [pgnls start_epoch 116 out=98b] v115'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ebc0400 2024-04-04T06:17:36.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.462+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:74 16.b 16:d0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5ebc0400 2024-04-04T06:17:36.457 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.463+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.1 v2:172.21.15.159:6816/3765338297 7 ==== osd_op_reply(74 [pgnls start_epoch 116 out=76b] v116'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ebc0400 2024-04-04T06:17:36.458 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.463+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:75 16.1b 16:d8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5e139400 2024-04-04T06:17:36.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.464+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.5 v2:172.21.15.178:6802/63746765 10 ==== osd_op_reply(75 [pgnls start_epoch 116 out=49b] v116'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5e139400 2024-04-04T06:17:36.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.464+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:76 16.7 16:e0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5e139000 2024-04-04T06:17:36.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.465+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.3 v2:172.21.15.159:6820/2070760205 10 ==== osd_op_reply(76 [pgnls start_epoch 116 out=105b] v116'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139000 2024-04-04T06:17:36.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.465+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:77 16.17 16:e8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5ecf1000 2024-04-04T06:17:36.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.466+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 22 ==== osd_op_reply(77 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.466+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:78 16.f 16:f0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5ecf0c00 2024-04-04T06:17:36.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.466+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.0 v2:172.21.15.159:6800/455382518 16 ==== osd_op_reply(78 [pgnls start_epoch 116 out=73b] v116'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5ecf0c00 2024-04-04T06:17:36.461 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.466+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:79 16.1f 16:f8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7400 con 0x563b5e139400 2024-04-04T06:17:36.462 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.468+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.5 v2:172.21.15.178:6802/63746765 11 ==== osd_op_reply(79 [pgnls start_epoch 116 out=101b] v116'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5e139400 2024-04-04T06:17:36.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.468+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-04T06:17:36.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.468+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:80 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc6c00 con 0x563b5ecf1000 2024-04-04T06:17:36.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.468+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 23 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.468+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-04T06:17:36.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.468+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:81 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7c00 con 0x563b5ecf1000 2024-04-04T06:17:36.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.469+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 24 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.469+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-04T06:17:36.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.469+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:82 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc6400 con 0x563b5ecf1000 2024-04-04T06:17:36.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.470+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 25 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) ==== 164+0+1169 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.470+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-04T06:17:36.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.470+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:83 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc1c00 con 0x563b5ecf1000 2024-04-04T06:17:36.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.471+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 26 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.465 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.471+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-04T06:17:36.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.471+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:84 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc1800 con 0x563b5ecf1000 2024-04-04T06:17:36.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.471+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 27 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.471+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-04T06:17:36.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.471+0000 7fa1b7853740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-04T06:17:36.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.471+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-04T06:17:36.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.471+0000 7fa1b7853740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-04T06:17:36.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.471+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-04T06:17:36.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.471+0000 7fa1b7853740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-04T06:17:36.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.471+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-04T06:17:36.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.471+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:85 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc1400 con 0x563b5ecf0c00 2024-04-04T06:17:36.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.472+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.0 v2:172.21.15.159:6800/455382518 17 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv37 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5ecf0c00 2024-04-04T06:17:36.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.472+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-04T06:17:36.467 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.472+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:86 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc0800 con 0x563b5ecf1000 2024-04-04T06:17:36.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.473+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 28 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.473+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-04-04T06:17:36.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.473+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:87 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5e138c00 con 0x563b5e139c00 2024-04-04T06:17:36.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.474+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.6 v2:172.21.15.178:6826/198949157 8 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=592b] v0'0 uv3 ondisk = 0) ==== 164+0+592 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139c00 2024-04-04T06:17:36.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.474+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-04T06:17:36.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.474+0000 7fa1b7853740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-04T06:17:36.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.474+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-04T06:17:36.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.474+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:88 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7000 con 0x563b5ecf1000 2024-04-04T06:17:36.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.475+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 29 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.475+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-04T06:17:36.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.475+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- osd_op(unknown.0.0:89 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc6000 con 0x563b5ecf0c00 2024-04-04T06:17:36.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.476+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.0 v2:172.21.15.159:6800/455382518 18 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5ecf0c00 2024-04-04T06:17:36.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.476+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-04T06:17:36.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.476+0000 7fa1b7853740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-04T06:17:36.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.476+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-04T06:17:36.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.476+0000 7fa1b7853740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-04T06:17:36.472 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.476+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-04T06:17:36.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.476+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- osd_op(unknown.0.0:90 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc6800 con 0x563b5ecf1000 2024-04-04T06:17:36.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.477+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.2 v2:172.21.15.159:6808/2619731047 30 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ecf1000 2024-04-04T06:17:36.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.477+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-04T06:17:36.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.477+0000 7fa1b7853740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-04T06:17:36.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.477+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-04T06:17:36.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.477+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:91 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc6c00 con 0x563b5ebc0400 2024-04-04T06:17:36.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.478+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.1 v2:172.21.15.159:6816/3765338297 8 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ebc0400 2024-04-04T06:17:36.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.479+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-04T06:17:36.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.479+0000 7fa1b7853740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-04T06:17:36.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.479+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-04T06:17:36.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.479+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:92 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7c00 con 0x563b5e139000 2024-04-04T06:17:36.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.479+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.3 v2:172.21.15.159:6820/2070760205 11 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139000 2024-04-04T06:17:36.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.480+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-04-04T06:17:36.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.480+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:93 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc6400 con 0x563b5e139000 2024-04-04T06:17:36.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.480+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.3 v2:172.21.15.159:6820/2070760205 12 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1683b] v0'0 uv9 ondisk = 0) ==== 164+0+1683 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139000 2024-04-04T06:17:36.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.480+0000 7fa1b7853740 20 datascan.scan_links: injected ino detected 2024-04-04T06:17:36.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.480+0000 7fa1b7853740 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-04-04T06:17:36.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.480+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-04T06:17:36.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.480+0000 7fa1b7853740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-04T06:17:36.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.480+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-04T06:17:36.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.480+0000 7fa1b7853740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-04T06:17:36.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.480+0000 7fa1b7853740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-04T06:17:36.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.480+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- osd_op(unknown.0.0:94 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7800 con 0x563b5e139400 2024-04-04T06:17:36.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.482+0000 7fa1b4a00640 1 -- 172.21.15.159:0/126768414 <== osd.5 v2:172.21.15.178:6802/63746765 12 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x563b5eb3e000 con 0x563b5e139400 2024-04-04T06:17:36.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.482+0000 7fa1b7853740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-04-04T06:17:36.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.482+0000 7fa1b7853740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-04T06:17:36.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.482+0000 7fa1b7853740 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-04-04T06:17:36.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.482+0000 7fa1b7853740 10 datascan.scan_links: processing 1 injected_inos 2024-04-04T06:17:36.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.482+0000 7fa1b7853740 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-04-04T06:17:36.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.482+0000 7fa1b7853740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-04-04T06:17:36.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.482+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:95 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc1000 con 0x563b5e139000 2024-04-04T06:17:36.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.483+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.3 v2:172.21.15.159:6820/2070760205 13 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139000 2024-04-04T06:17:36.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.483+0000 7fa1b7853740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-04T06:17:36.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.483+0000 7fa1b7853740 20 datascan.scan_links: first is now 2 2024-04-04T06:17:36.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.483+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- osd_op(unknown.0.0:96 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc0c00 con 0x563b5e139000 2024-04-04T06:17:36.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.486+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.3 v2:172.21.15.159:6820/2070760205 14 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v116'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139000 2024-04-04T06:17:36.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.486+0000 7fa1b7853740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-04-04T06:17:36.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.486+0000 7fa1b7853740 10 datascan.scan_links: updating inotable 2024-04-04T06:17:36.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.486+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:97 16.13 16:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc1000 con 0x563b5ebc0400 2024-04-04T06:17:36.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.487+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.1 v2:172.21.15.159:6816/3765338297 9 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ebc0400 2024-04-04T06:17:36.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.487+0000 7fa1b7853740 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-04-04T06:17:36.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.487+0000 7fa1b7853740 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-04-04T06:17:36.484 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.488+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- osd_op(unknown.0.0:98 16.13 16:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc0c00 con 0x563b5ebc0400 2024-04-04T06:17:36.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.492+0000 7fa1b5e00640 1 -- 172.21.15.159:0/126768414 <== osd.1 v2:172.21.15.159:6816/3765338297 10 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v116'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x563b5eb3e500 con 0x563b5ebc0400 2024-04-04T06:17:36.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.492+0000 7fa1b7853740 10 datascan.scan_links: updating snaptable 2024-04-04T06:17:36.487 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.492+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- osd_op(unknown.0.0:99 16.d 16:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x563b5ebc7c00 con 0x563b5e139c00 2024-04-04T06:17:36.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.493+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 <== osd.6 v2:172.21.15.178:6826/198949157 9 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x563b5eb3e780 con 0x563b5e139c00 2024-04-04T06:17:36.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x563b5ecf0c00 msgr2=0x563b5eb1e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b7853740 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x563b5ecf0c00 0x563b5eb1e680 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x563b5ebc0400 msgr2=0x563b5e148100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b7853740 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x563b5ebc0400 0x563b5e148100 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x563b5ecf1000 msgr2=0x563b5e146000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b7853740 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x563b5ecf1000 0x563b5e146000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x563b5e139000 msgr2=0x563b5e146580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b7853740 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x563b5e139000 0x563b5e146580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x563b5e139800 msgr2=0x563b5e147080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b7853740 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x563b5e139800 0x563b5e147080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x563b5e139400 msgr2=0x563b5e146b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 reap_dead start 2024-04-04T06:17:36.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b7853740 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x563b5e139400 0x563b5e146b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 reap_dead start 2024-04-04T06:17:36.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x563b5e139c00 msgr2=0x563b5e147600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b7853740 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x563b5e139c00 0x563b5e147600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x563b5ebc0000 msgr2=0x563b5e147b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b7853740 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x563b5ebc0000 0x563b5e147b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b5400640 1 -- 172.21.15.159:0/126768414 reap_dead start 2024-04-04T06:17:36.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x563b5e138000 msgr2=0x563b5eb1e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.492 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b7853740 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x563b5e138000 0x563b5eb1e100 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x563b5eb3cd20 tx=0x563b5eca1140 comp rx=0 tx=0).stop 2024-04-04T06:17:36.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b5ecf0800 msgr2=0x563b5eb1db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.494+0000 7fa1b7853740 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b5ecf0800 0x563b5eb1db80 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x563b5eba7ad0 tx=0x563b5eca0c00 comp rx=0 tx=0).stop 2024-04-04T06:17:36.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.495+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 shutdown_connections 2024-04-04T06:17:36.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.495+0000 7fa1b7853740 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b5ecf0800 0x563b5eb1db80 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.495+0000 7fa1b7853740 1 --2- 172.21.15.159:0/126768414 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x563b5e138000 0x563b5eb1e100 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.495+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 >> 172.21.15.159:0/126768414 conn(0x563b5ebafc00 msgr2=0x563b5dea7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:36.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.495+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 shutdown_connections 2024-04-04T06:17:36.494 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.495+0000 7fa1b7853740 1 -- 172.21.15.159:0/126768414 wait complete. 2024-04-04T06:17:36.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.496+0000 7fa1b7853740 1 -- 172.21.15.159:0/3130279389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b5ddbdc00 msgr2=0x563b5eb1cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:36.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.496+0000 7fa1b7853740 1 --2- 172.21.15.159:0/3130279389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b5ddbdc00 0x563b5eb1cb00 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x563b5ebe1050 tx=0x563b5eb56e40 comp rx=0 tx=0).stop 2024-04-04T06:17:36.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.496+0000 7fa1b7853740 1 -- 172.21.15.159:0/3130279389 shutdown_connections 2024-04-04T06:17:36.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.496+0000 7fa1b7853740 1 --2- 172.21.15.159:0/3130279389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b5ddbdc00 0x563b5eb1cb00 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.496+0000 7fa1b7853740 1 --2- 172.21.15.159:0/3130279389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b5ddbd800 0x563b5eb1c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.496+0000 7fa1b7853740 1 --2- 172.21.15.159:0/3130279389 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x563b5ddbcc00 0x563b5eb1c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:36.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.497+0000 7fa1b7853740 1 -- 172.21.15.159:0/3130279389 >> 172.21.15.159:0/3130279389 conn(0x563b5ddbd000 msgr2=0x563b5dea7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:36.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.497+0000 7fa1b7853740 1 -- 172.21.15.159:0/3130279389 shutdown_connections 2024-04-04T06:17:36.495 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:36.497+0000 7fa1b7853740 1 -- 172.21.15.159:0/3130279389 wait complete. 2024-04-04T06:17:36.503 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.223021, result: 2024-04-04T06:17:36.504 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-04T06:17:36.512 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-04T06:17:36.679 INFO:teuthology.orchestra.run.smithi159.stdout:[global] 2024-04-04T06:17:36.680 INFO:teuthology.orchestra.run.smithi159.stdout: chdir = "" 2024-04-04T06:17:36.680 INFO:teuthology.orchestra.run.smithi159.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-04T06:17:36.680 INFO:teuthology.orchestra.run.smithi159.stdout: auth supported = cephx 2024-04-04T06:17:36.680 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.680 INFO:teuthology.orchestra.run.smithi159.stdout: filestore xattr use omap = true 2024-04-04T06:17:36.680 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.680 INFO:teuthology.orchestra.run.smithi159.stdout: mon clock drift allowed = 1.000 2024-04-04T06:17:36.680 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.680 INFO:teuthology.orchestra.run.smithi159.stdout: osd crush chooseleaf type = 0 2024-04-04T06:17:36.680 INFO:teuthology.orchestra.run.smithi159.stdout: auth debug = true 2024-04-04T06:17:36.680 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.680 INFO:teuthology.orchestra.run.smithi159.stdout: ms die on old message = true 2024-04-04T06:17:36.681 INFO:teuthology.orchestra.run.smithi159.stdout: ms die on bug = true 2024-04-04T06:17:36.681 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.681 INFO:teuthology.orchestra.run.smithi159.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-04T06:17:36.681 INFO:teuthology.orchestra.run.smithi159.stdout: mon pg warn max object skew = 0 2024-04-04T06:17:36.681 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.681 INFO:teuthology.orchestra.run.smithi159.stdout: # disable pg_autoscaler by default for new pools 2024-04-04T06:17:36.681 INFO:teuthology.orchestra.run.smithi159.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-04T06:17:36.681 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.681 INFO:teuthology.orchestra.run.smithi159.stdout: osd pool default size = 2 2024-04-04T06:17:36.681 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.681 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd allow primary affinity = true 2024-04-04T06:17:36.681 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd allow pg remap = true 2024-04-04T06:17:36.681 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on legacy crush tunables = false 2024-04-04T06:17:36.681 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on crush straw calc version zero = false 2024-04-04T06:17:36.681 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on no sortbitwise = false 2024-04-04T06:17:36.682 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on osd down out interval zero = false 2024-04-04T06:17:36.682 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on too few osds = false 2024-04-04T06:17:36.682 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-04T06:17:36.682 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-04T06:17:36.682 INFO:teuthology.orchestra.run.smithi159.stdout: mon_allow_pool_size_one = true 2024-04-04T06:17:36.682 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.682 INFO:teuthology.orchestra.run.smithi159.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-04T06:17:36.682 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.682 INFO:teuthology.orchestra.run.smithi159.stdout: osd default data pool replay window = 5 2024-04-04T06:17:36.682 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.682 INFO:teuthology.orchestra.run.smithi159.stdout: mon allow pool delete = true 2024-04-04T06:17:36.682 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.682 INFO:teuthology.orchestra.run.smithi159.stdout: mon cluster log file level = debug 2024-04-04T06:17:36.682 INFO:teuthology.orchestra.run.smithi159.stdout: debug asserts on shutdown = true 2024-04-04T06:17:36.682 INFO:teuthology.orchestra.run.smithi159.stdout: mon health detail to clog = false 2024-04-04T06:17:36.683 INFO:teuthology.orchestra.run.smithi159.stdout: mon host = "172.21.15.159,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-04-04T06:17:36.683 INFO:teuthology.orchestra.run.smithi159.stdout: ms die on skipped message = False 2024-04-04T06:17:36.683 INFO:teuthology.orchestra.run.smithi159.stdout: fsid = 30311045-f714-48fa-832b-4cb2ba8bf0bf 2024-04-04T06:17:36.683 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.683 INFO:teuthology.orchestra.run.smithi159.stdout:[osd] 2024-04-04T06:17:36.683 INFO:teuthology.orchestra.run.smithi159.stdout: osd journal size = 100 2024-04-04T06:17:36.683 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.683 INFO:teuthology.orchestra.run.smithi159.stdout: osd scrub load threshold = 5.0 2024-04-04T06:17:36.683 INFO:teuthology.orchestra.run.smithi159.stdout: osd scrub max interval = 600 2024-04-04T06:17:36.683 INFO:teuthology.orchestra.run.smithi159.stdout: osd mclock profile = high_recovery_ops 2024-04-04T06:17:36.683 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.683 INFO:teuthology.orchestra.run.smithi159.stdout: osd recover clone overlap = true 2024-04-04T06:17:36.683 INFO:teuthology.orchestra.run.smithi159.stdout: osd recovery max chunk = 1048576 2024-04-04T06:17:36.683 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.683 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug shutdown = true 2024-04-04T06:17:36.684 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug op order = true 2024-04-04T06:17:36.684 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug verify stray on activate = true 2024-04-04T06:17:36.684 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug trim objects = true 2024-04-04T06:17:36.684 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.684 INFO:teuthology.orchestra.run.smithi159.stdout: osd open classes on start = true 2024-04-04T06:17:36.684 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug pg log writeout = true 2024-04-04T06:17:36.684 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.684 INFO:teuthology.orchestra.run.smithi159.stdout: osd deep scrub update digest min age = 30 2024-04-04T06:17:36.684 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.684 INFO:teuthology.orchestra.run.smithi159.stdout: osd map max advance = 10 2024-04-04T06:17:36.684 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.684 INFO:teuthology.orchestra.run.smithi159.stdout: journal zero on create = true 2024-04-04T06:17:36.684 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.684 INFO:teuthology.orchestra.run.smithi159.stdout: filestore ondisk finisher threads = 3 2024-04-04T06:17:36.684 INFO:teuthology.orchestra.run.smithi159.stdout: filestore apply finisher threads = 3 2024-04-04T06:17:36.684 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.685 INFO:teuthology.orchestra.run.smithi159.stdout: bdev debug aio = true 2024-04-04T06:17:36.685 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug misdirected ops = true 2024-04-04T06:17:36.685 INFO:teuthology.orchestra.run.smithi159.stdout: bdev async discard = True 2024-04-04T06:17:36.685 INFO:teuthology.orchestra.run.smithi159.stdout: bdev enable discard = True 2024-04-04T06:17:36.685 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore allocator = bitmap 2024-04-04T06:17:36.685 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore block size = 96636764160 2024-04-04T06:17:36.685 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore fsck on mount = True 2024-04-04T06:17:36.685 INFO:teuthology.orchestra.run.smithi159.stdout: debug bluefs = 1/20 2024-04-04T06:17:36.685 INFO:teuthology.orchestra.run.smithi159.stdout: debug bluestore = 1/20 2024-04-04T06:17:36.685 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:17:36.685 INFO:teuthology.orchestra.run.smithi159.stdout: debug osd = 20 2024-04-04T06:17:36.685 INFO:teuthology.orchestra.run.smithi159.stdout: debug rocksdb = 4/10 2024-04-04T06:17:36.685 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd backfillfull_ratio = 0.85 2024-04-04T06:17:36.685 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd full ratio = 0.9 2024-04-04T06:17:36.685 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd nearfull ratio = 0.8 2024-04-04T06:17:36.686 INFO:teuthology.orchestra.run.smithi159.stdout: osd failsafe full ratio = 0.95 2024-04-04T06:17:36.686 INFO:teuthology.orchestra.run.smithi159.stdout: osd objectstore = bluestore 2024-04-04T06:17:36.686 INFO:teuthology.orchestra.run.smithi159.stdout: osd op complaint time = 180 2024-04-04T06:17:36.686 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.686 INFO:teuthology.orchestra.run.smithi159.stdout:[mgr] 2024-04-04T06:17:36.686 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:17:36.686 INFO:teuthology.orchestra.run.smithi159.stdout: debug mgr = 20 2024-04-04T06:17:36.686 INFO:teuthology.orchestra.run.smithi159.stdout: debug mon = 20 2024-04-04T06:17:36.686 INFO:teuthology.orchestra.run.smithi159.stdout: debug auth = 20 2024-04-04T06:17:36.686 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min pgs per osd = 4 2024-04-04T06:17:36.686 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min bytes per osd = 10 2024-04-04T06:17:36.687 INFO:teuthology.orchestra.run.smithi159.stdout: mgr/telemetry/nag = false 2024-04-04T06:17:36.687 INFO:teuthology.orchestra.run.smithi159.stdout: debug client = 20 2024-04-04T06:17:36.687 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.687 INFO:teuthology.orchestra.run.smithi159.stdout:[mon] 2024-04-04T06:17:36.687 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:17:36.687 INFO:teuthology.orchestra.run.smithi159.stdout: debug mon = 20 2024-04-04T06:17:36.687 INFO:teuthology.orchestra.run.smithi159.stdout: debug paxos = 20 2024-04-04T06:17:36.687 INFO:teuthology.orchestra.run.smithi159.stdout: debug auth = 20 2024-04-04T06:17:36.687 INFO:teuthology.orchestra.run.smithi159.stdout: mon data avail warn = 5 2024-04-04T06:17:36.687 INFO:teuthology.orchestra.run.smithi159.stdout: mon mgr mkfs grace = 240 2024-04-04T06:17:36.687 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min pgs per osd = 4 2024-04-04T06:17:36.687 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd reporter subtree level = osd 2024-04-04T06:17:36.688 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd prime pg temp = true 2024-04-04T06:17:36.688 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min bytes per osd = 10 2024-04-04T06:17:36.688 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.688 INFO:teuthology.orchestra.run.smithi159.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-04T06:17:36.688 INFO:teuthology.orchestra.run.smithi159.stdout: auth mon ticket ttl = 660 # 11m 2024-04-04T06:17:36.688 INFO:teuthology.orchestra.run.smithi159.stdout: auth service ticket ttl = 240 # 4m 2024-04-04T06:17:36.688 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.688 INFO:teuthology.orchestra.run.smithi159.stdout: # don't complain about insecure global_id in the test suite 2024-04-04T06:17:36.688 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-04T06:17:36.688 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-04T06:17:36.688 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.688 INFO:teuthology.orchestra.run.smithi159.stdout: # 1m isn't quite enough 2024-04-04T06:17:36.689 INFO:teuthology.orchestra.run.smithi159.stdout: mon_down_mkfs_grace = 2m 2024-04-04T06:17:36.689 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.689 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_filestore_osds = false 2024-04-04T06:17:36.689 INFO:teuthology.orchestra.run.smithi159.stdout: mon down mkfs grace = 300 2024-04-04T06:17:36.689 INFO:teuthology.orchestra.run.smithi159.stdout: mon op complaint time = 120 2024-04-04T06:17:36.689 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.689 INFO:teuthology.orchestra.run.smithi159.stdout:[client] 2024-04-04T06:17:36.689 INFO:teuthology.orchestra.run.smithi159.stdout: rgw cache enabled = true 2024-04-04T06:17:36.689 INFO:teuthology.orchestra.run.smithi159.stdout: rgw enable ops log = true 2024-04-04T06:17:36.689 INFO:teuthology.orchestra.run.smithi159.stdout: rgw enable usage log = true 2024-04-04T06:17:36.689 INFO:teuthology.orchestra.run.smithi159.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-04T06:17:36.689 INFO:teuthology.orchestra.run.smithi159.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-04T06:17:36.689 INFO:teuthology.orchestra.run.smithi159.stdout: client mount timeout = 600 2024-04-04T06:17:36.689 INFO:teuthology.orchestra.run.smithi159.stdout: debug client = 20 2024-04-04T06:17:36.690 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:17:36.690 INFO:teuthology.orchestra.run.smithi159.stdout: pid file = "" 2024-04-04T06:17:36.690 INFO:teuthology.orchestra.run.smithi159.stdout: rados mon op timeout = 900 2024-04-04T06:17:36.690 INFO:teuthology.orchestra.run.smithi159.stdout: rados osd op timeout = 900 2024-04-04T06:17:36.690 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.a] 2024-04-04T06:17:36.690 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.b] 2024-04-04T06:17:36.690 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.c] 2024-04-04T06:17:36.690 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.a] 2024-04-04T06:17:36.690 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.c] 2024-04-04T06:17:36.694 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.b] 2024-04-04T06:17:36.695 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.d] 2024-04-04T06:17:36.695 INFO:teuthology.orchestra.run.smithi159.stdout:[mds] 2024-04-04T06:17:36.695 INFO:teuthology.orchestra.run.smithi159.stdout: debug mds = 20 2024-04-04T06:17:36.695 INFO:teuthology.orchestra.run.smithi159.stdout: debug mds balancer = 20 2024-04-04T06:17:36.695 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:17:36.695 INFO:teuthology.orchestra.run.smithi159.stdout: mds debug frag = True 2024-04-04T06:17:36.695 INFO:teuthology.orchestra.run.smithi159.stdout: mds debug scatterstat = False 2024-04-04T06:17:36.695 INFO:teuthology.orchestra.run.smithi159.stdout: mds op complaint time = 180 2024-04-04T06:17:36.695 INFO:teuthology.orchestra.run.smithi159.stdout: mds verify scatter = True 2024-04-04T06:17:36.695 INFO:teuthology.orchestra.run.smithi159.stdout: osd op complaint time = 180 2024-04-04T06:17:36.695 INFO:teuthology.orchestra.run.smithi159.stdout: rados mon op timeout = 900 2024-04-04T06:17:36.696 INFO:teuthology.orchestra.run.smithi159.stdout: rados osd op timeout = 900 2024-04-04T06:17:36.696 INFO:teuthology.orchestra.run.smithi178.stdout:[global] 2024-04-04T06:17:36.696 INFO:teuthology.orchestra.run.smithi178.stdout: chdir = "" 2024-04-04T06:17:36.696 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-04T06:17:36.696 INFO:teuthology.orchestra.run.smithi178.stdout: auth supported = cephx 2024-04-04T06:17:36.696 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.696 INFO:teuthology.orchestra.run.smithi178.stdout: filestore xattr use omap = true 2024-04-04T06:17:36.697 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.697 INFO:teuthology.orchestra.run.smithi178.stdout: mon clock drift allowed = 1.000 2024-04-04T06:17:36.697 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.697 INFO:teuthology.orchestra.run.smithi178.stdout: osd crush chooseleaf type = 0 2024-04-04T06:17:36.698 INFO:teuthology.orchestra.run.smithi178.stdout: auth debug = true 2024-04-04T06:17:36.698 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.698 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on old message = true 2024-04-04T06:17:36.698 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on bug = true 2024-04-04T06:17:36.698 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.699 INFO:teuthology.orchestra.run.smithi178.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-04T06:17:36.699 INFO:teuthology.orchestra.run.smithi178.stdout: mon pg warn max object skew = 0 2024-04-04T06:17:36.699 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.699 INFO:teuthology.orchestra.run.smithi178.stdout: # disable pg_autoscaler by default for new pools 2024-04-04T06:17:36.699 INFO:teuthology.orchestra.run.smithi178.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-04T06:17:36.699 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.699 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default size = 2 2024-04-04T06:17:36.699 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.699 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow primary affinity = true 2024-04-04T06:17:36.699 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow pg remap = true 2024-04-04T06:17:36.699 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on legacy crush tunables = false 2024-04-04T06:17:36.699 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on crush straw calc version zero = false 2024-04-04T06:17:36.699 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on no sortbitwise = false 2024-04-04T06:17:36.700 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on osd down out interval zero = false 2024-04-04T06:17:36.700 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on too few osds = false 2024-04-04T06:17:36.700 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-04T06:17:36.700 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-04T06:17:36.700 INFO:teuthology.orchestra.run.smithi178.stdout: mon_allow_pool_size_one = true 2024-04-04T06:17:36.700 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.700 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-04T06:17:36.700 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.700 INFO:teuthology.orchestra.run.smithi178.stdout: osd default data pool replay window = 5 2024-04-04T06:17:36.700 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.700 INFO:teuthology.orchestra.run.smithi178.stdout: mon allow pool delete = true 2024-04-04T06:17:36.701 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.701 INFO:teuthology.orchestra.run.smithi178.stdout: mon cluster log file level = debug 2024-04-04T06:17:36.701 INFO:teuthology.orchestra.run.smithi178.stdout: debug asserts on shutdown = true 2024-04-04T06:17:36.702 INFO:teuthology.orchestra.run.smithi178.stdout: mon health detail to clog = false 2024-04-04T06:17:36.702 INFO:teuthology.orchestra.run.smithi178.stdout: mon host = "172.21.15.159,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-04-04T06:17:36.702 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on skipped message = False 2024-04-04T06:17:36.702 INFO:teuthology.orchestra.run.smithi178.stdout: fsid = 30311045-f714-48fa-832b-4cb2ba8bf0bf 2024-04-04T06:17:36.702 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.702 INFO:teuthology.orchestra.run.smithi178.stdout:[osd] 2024-04-04T06:17:36.702 INFO:teuthology.orchestra.run.smithi178.stdout: osd journal size = 100 2024-04-04T06:17:36.702 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.702 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub load threshold = 5.0 2024-04-04T06:17:36.703 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub max interval = 600 2024-04-04T06:17:36.703 INFO:teuthology.orchestra.run.smithi178.stdout: osd mclock profile = high_recovery_ops 2024-04-04T06:17:36.703 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.703 INFO:teuthology.orchestra.run.smithi178.stdout: osd recover clone overlap = true 2024-04-04T06:17:36.703 INFO:teuthology.orchestra.run.smithi178.stdout: osd recovery max chunk = 1048576 2024-04-04T06:17:36.703 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.703 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug shutdown = true 2024-04-04T06:17:36.703 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug op order = true 2024-04-04T06:17:36.703 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify stray on activate = true 2024-04-04T06:17:36.703 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug trim objects = true 2024-04-04T06:17:36.703 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.704 INFO:teuthology.orchestra.run.smithi178.stdout: osd open classes on start = true 2024-04-04T06:17:36.704 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug pg log writeout = true 2024-04-04T06:17:36.704 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.704 INFO:teuthology.orchestra.run.smithi178.stdout: osd deep scrub update digest min age = 30 2024-04-04T06:17:36.704 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.704 INFO:teuthology.orchestra.run.smithi178.stdout: osd map max advance = 10 2024-04-04T06:17:36.704 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.704 INFO:teuthology.orchestra.run.smithi178.stdout: journal zero on create = true 2024-04-04T06:17:36.705 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.705 INFO:teuthology.orchestra.run.smithi178.stdout: filestore ondisk finisher threads = 3 2024-04-04T06:17:36.705 INFO:teuthology.orchestra.run.smithi178.stdout: filestore apply finisher threads = 3 2024-04-04T06:17:36.705 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.705 INFO:teuthology.orchestra.run.smithi178.stdout: bdev debug aio = true 2024-04-04T06:17:36.705 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug misdirected ops = true 2024-04-04T06:17:36.705 INFO:teuthology.orchestra.run.smithi178.stdout: bdev async discard = True 2024-04-04T06:17:36.706 INFO:teuthology.orchestra.run.smithi178.stdout: bdev enable discard = True 2024-04-04T06:17:36.706 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore allocator = bitmap 2024-04-04T06:17:36.706 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore block size = 96636764160 2024-04-04T06:17:36.706 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore fsck on mount = True 2024-04-04T06:17:36.706 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluefs = 1/20 2024-04-04T06:17:36.707 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluestore = 1/20 2024-04-04T06:17:36.707 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:17:36.707 INFO:teuthology.orchestra.run.smithi178.stdout: debug osd = 20 2024-04-04T06:17:36.707 INFO:teuthology.orchestra.run.smithi178.stdout: debug rocksdb = 4/10 2024-04-04T06:17:36.707 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd backfillfull_ratio = 0.85 2024-04-04T06:17:36.707 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd full ratio = 0.9 2024-04-04T06:17:36.707 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd nearfull ratio = 0.8 2024-04-04T06:17:36.707 INFO:teuthology.orchestra.run.smithi178.stdout: osd failsafe full ratio = 0.95 2024-04-04T06:17:36.707 INFO:teuthology.orchestra.run.smithi178.stdout: osd objectstore = bluestore 2024-04-04T06:17:36.708 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-04-04T06:17:36.708 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.708 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr] 2024-04-04T06:17:36.708 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:17:36.708 INFO:teuthology.orchestra.run.smithi178.stdout: debug mgr = 20 2024-04-04T06:17:36.708 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-04T06:17:36.708 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-04T06:17:36.708 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-04T06:17:36.708 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-04T06:17:36.708 INFO:teuthology.orchestra.run.smithi178.stdout: mgr/telemetry/nag = false 2024-04-04T06:17:36.708 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-04-04T06:17:36.708 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.708 INFO:teuthology.orchestra.run.smithi178.stdout:[mon] 2024-04-04T06:17:36.708 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:17:36.709 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-04T06:17:36.709 INFO:teuthology.orchestra.run.smithi178.stdout: debug paxos = 20 2024-04-04T06:17:36.709 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-04T06:17:36.709 INFO:teuthology.orchestra.run.smithi178.stdout: mon data avail warn = 5 2024-04-04T06:17:36.709 INFO:teuthology.orchestra.run.smithi178.stdout: mon mgr mkfs grace = 240 2024-04-04T06:17:36.709 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-04T06:17:36.709 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd reporter subtree level = osd 2024-04-04T06:17:36.709 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd prime pg temp = true 2024-04-04T06:17:36.709 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-04T06:17:36.709 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.709 INFO:teuthology.orchestra.run.smithi178.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-04T06:17:36.709 INFO:teuthology.orchestra.run.smithi178.stdout: auth mon ticket ttl = 660 # 11m 2024-04-04T06:17:36.709 INFO:teuthology.orchestra.run.smithi178.stdout: auth service ticket ttl = 240 # 4m 2024-04-04T06:17:36.709 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.710 INFO:teuthology.orchestra.run.smithi178.stdout: # don't complain about insecure global_id in the test suite 2024-04-04T06:17:36.710 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-04T06:17:36.710 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-04T06:17:36.710 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.710 INFO:teuthology.orchestra.run.smithi178.stdout: # 1m isn't quite enough 2024-04-04T06:17:36.710 INFO:teuthology.orchestra.run.smithi178.stdout: mon_down_mkfs_grace = 2m 2024-04-04T06:17:36.710 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.710 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_filestore_osds = false 2024-04-04T06:17:36.710 INFO:teuthology.orchestra.run.smithi178.stdout: mon down mkfs grace = 300 2024-04-04T06:17:36.711 INFO:teuthology.orchestra.run.smithi178.stdout: mon op complaint time = 120 2024-04-04T06:17:36.711 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.711 INFO:teuthology.orchestra.run.smithi178.stdout:[client] 2024-04-04T06:17:36.711 INFO:teuthology.orchestra.run.smithi178.stdout: rgw cache enabled = true 2024-04-04T06:17:36.711 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable ops log = true 2024-04-04T06:17:36.711 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable usage log = true 2024-04-04T06:17:36.712 INFO:teuthology.orchestra.run.smithi178.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-04T06:17:36.712 INFO:teuthology.orchestra.run.smithi178.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-04T06:17:36.712 INFO:teuthology.orchestra.run.smithi178.stdout: client mount timeout = 600 2024-04-04T06:17:36.712 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-04-04T06:17:36.712 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:17:36.712 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = "" 2024-04-04T06:17:36.712 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-04-04T06:17:36.712 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-04-04T06:17:36.712 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.a] 2024-04-04T06:17:36.712 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.b] 2024-04-04T06:17:36.712 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.c] 2024-04-04T06:17:36.712 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.a] 2024-04-04T06:17:36.713 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.c] 2024-04-04T06:17:36.713 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.b] 2024-04-04T06:17:36.713 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.d] 2024-04-04T06:17:36.713 INFO:teuthology.orchestra.run.smithi178.stdout:[mds] 2024-04-04T06:17:36.713 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds = 20 2024-04-04T06:17:36.713 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds balancer = 20 2024-04-04T06:17:36.713 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:17:36.713 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug frag = True 2024-04-04T06:17:36.713 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug scatterstat = False 2024-04-04T06:17:36.713 INFO:teuthology.orchestra.run.smithi178.stdout: mds op complaint time = 180 2024-04-04T06:17:36.713 INFO:teuthology.orchestra.run.smithi178.stdout: mds verify scatter = True 2024-04-04T06:17:36.713 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-04-04T06:17:36.713 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-04-04T06:17:36.713 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-04-04T06:17:36.734 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-04T06:17:36.740 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-04T06:17:36.902 INFO:teuthology.orchestra.run.smithi159.stdout:[global] 2024-04-04T06:17:36.902 INFO:teuthology.orchestra.run.smithi159.stdout: chdir = "" 2024-04-04T06:17:36.902 INFO:teuthology.orchestra.run.smithi159.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-04T06:17:36.902 INFO:teuthology.orchestra.run.smithi159.stdout: auth supported = cephx 2024-04-04T06:17:36.902 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.902 INFO:teuthology.orchestra.run.smithi159.stdout: filestore xattr use omap = true 2024-04-04T06:17:36.902 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.902 INFO:teuthology.orchestra.run.smithi159.stdout: mon clock drift allowed = 1.000 2024-04-04T06:17:36.902 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.903 INFO:teuthology.orchestra.run.smithi159.stdout: osd crush chooseleaf type = 0 2024-04-04T06:17:36.903 INFO:teuthology.orchestra.run.smithi159.stdout: auth debug = true 2024-04-04T06:17:36.903 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.903 INFO:teuthology.orchestra.run.smithi159.stdout: ms die on old message = true 2024-04-04T06:17:36.903 INFO:teuthology.orchestra.run.smithi159.stdout: ms die on bug = true 2024-04-04T06:17:36.903 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.903 INFO:teuthology.orchestra.run.smithi159.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-04T06:17:36.903 INFO:teuthology.orchestra.run.smithi159.stdout: mon pg warn max object skew = 0 2024-04-04T06:17:36.903 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.903 INFO:teuthology.orchestra.run.smithi159.stdout: # disable pg_autoscaler by default for new pools 2024-04-04T06:17:36.904 INFO:teuthology.orchestra.run.smithi159.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-04T06:17:36.904 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.904 INFO:teuthology.orchestra.run.smithi159.stdout: osd pool default size = 2 2024-04-04T06:17:36.904 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.904 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd allow primary affinity = true 2024-04-04T06:17:36.904 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd allow pg remap = true 2024-04-04T06:17:36.904 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on legacy crush tunables = false 2024-04-04T06:17:36.904 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on crush straw calc version zero = false 2024-04-04T06:17:36.904 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on no sortbitwise = false 2024-04-04T06:17:36.904 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on osd down out interval zero = false 2024-04-04T06:17:36.904 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on too few osds = false 2024-04-04T06:17:36.904 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-04T06:17:36.904 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-04T06:17:36.904 INFO:teuthology.orchestra.run.smithi159.stdout: mon_allow_pool_size_one = true 2024-04-04T06:17:36.904 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.905 INFO:teuthology.orchestra.run.smithi159.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-04T06:17:36.905 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.905 INFO:teuthology.orchestra.run.smithi159.stdout: osd default data pool replay window = 5 2024-04-04T06:17:36.905 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.905 INFO:teuthology.orchestra.run.smithi159.stdout: mon allow pool delete = true 2024-04-04T06:17:36.905 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.905 INFO:teuthology.orchestra.run.smithi159.stdout: mon cluster log file level = debug 2024-04-04T06:17:36.905 INFO:teuthology.orchestra.run.smithi159.stdout: debug asserts on shutdown = true 2024-04-04T06:17:36.905 INFO:teuthology.orchestra.run.smithi159.stdout: mon health detail to clog = false 2024-04-04T06:17:36.905 INFO:teuthology.orchestra.run.smithi159.stdout: mon host = "172.21.15.159,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-04-04T06:17:36.905 INFO:teuthology.orchestra.run.smithi159.stdout: ms die on skipped message = False 2024-04-04T06:17:36.905 INFO:teuthology.orchestra.run.smithi159.stdout: fsid = 30311045-f714-48fa-832b-4cb2ba8bf0bf 2024-04-04T06:17:36.905 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.905 INFO:teuthology.orchestra.run.smithi159.stdout:[osd] 2024-04-04T06:17:36.906 INFO:teuthology.orchestra.run.smithi159.stdout: osd journal size = 100 2024-04-04T06:17:36.906 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.906 INFO:teuthology.orchestra.run.smithi159.stdout: osd scrub load threshold = 5.0 2024-04-04T06:17:36.906 INFO:teuthology.orchestra.run.smithi159.stdout: osd scrub max interval = 600 2024-04-04T06:17:36.906 INFO:teuthology.orchestra.run.smithi159.stdout: osd mclock profile = high_recovery_ops 2024-04-04T06:17:36.906 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.906 INFO:teuthology.orchestra.run.smithi159.stdout: osd recover clone overlap = true 2024-04-04T06:17:36.906 INFO:teuthology.orchestra.run.smithi159.stdout: osd recovery max chunk = 1048576 2024-04-04T06:17:36.906 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.906 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug shutdown = true 2024-04-04T06:17:36.906 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug op order = true 2024-04-04T06:17:36.906 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug verify stray on activate = true 2024-04-04T06:17:36.906 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug trim objects = true 2024-04-04T06:17:36.906 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.907 INFO:teuthology.orchestra.run.smithi159.stdout: osd open classes on start = true 2024-04-04T06:17:36.907 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug pg log writeout = true 2024-04-04T06:17:36.907 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.907 INFO:teuthology.orchestra.run.smithi159.stdout: osd deep scrub update digest min age = 30 2024-04-04T06:17:36.907 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.907 INFO:teuthology.orchestra.run.smithi159.stdout: osd map max advance = 10 2024-04-04T06:17:36.907 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.907 INFO:teuthology.orchestra.run.smithi159.stdout: journal zero on create = true 2024-04-04T06:17:36.907 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.907 INFO:teuthology.orchestra.run.smithi159.stdout: filestore ondisk finisher threads = 3 2024-04-04T06:17:36.907 INFO:teuthology.orchestra.run.smithi159.stdout: filestore apply finisher threads = 3 2024-04-04T06:17:36.907 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.907 INFO:teuthology.orchestra.run.smithi159.stdout: bdev debug aio = true 2024-04-04T06:17:36.908 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug misdirected ops = true 2024-04-04T06:17:36.908 INFO:teuthology.orchestra.run.smithi159.stdout: bdev async discard = True 2024-04-04T06:17:36.908 INFO:teuthology.orchestra.run.smithi159.stdout: bdev enable discard = True 2024-04-04T06:17:36.908 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore allocator = bitmap 2024-04-04T06:17:36.908 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore block size = 96636764160 2024-04-04T06:17:36.908 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore fsck on mount = True 2024-04-04T06:17:36.908 INFO:teuthology.orchestra.run.smithi159.stdout: debug bluefs = 1/20 2024-04-04T06:17:36.909 INFO:teuthology.orchestra.run.smithi159.stdout: debug bluestore = 1/20 2024-04-04T06:17:36.909 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:17:36.909 INFO:teuthology.orchestra.run.smithi159.stdout: debug osd = 20 2024-04-04T06:17:36.909 INFO:teuthology.orchestra.run.smithi159.stdout: debug rocksdb = 4/10 2024-04-04T06:17:36.909 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd backfillfull_ratio = 0.85 2024-04-04T06:17:36.909 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd full ratio = 0.9 2024-04-04T06:17:36.910 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd nearfull ratio = 0.8 2024-04-04T06:17:36.910 INFO:teuthology.orchestra.run.smithi159.stdout: osd failsafe full ratio = 0.95 2024-04-04T06:17:36.910 INFO:teuthology.orchestra.run.smithi159.stdout: osd objectstore = bluestore 2024-04-04T06:17:36.910 INFO:teuthology.orchestra.run.smithi159.stdout: osd op complaint time = 180 2024-04-04T06:17:36.910 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.910 INFO:teuthology.orchestra.run.smithi159.stdout:[mgr] 2024-04-04T06:17:36.910 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:17:36.910 INFO:teuthology.orchestra.run.smithi159.stdout: debug mgr = 20 2024-04-04T06:17:36.910 INFO:teuthology.orchestra.run.smithi159.stdout: debug mon = 20 2024-04-04T06:17:36.910 INFO:teuthology.orchestra.run.smithi159.stdout: debug auth = 20 2024-04-04T06:17:36.910 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min pgs per osd = 4 2024-04-04T06:17:36.910 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min bytes per osd = 10 2024-04-04T06:17:36.911 INFO:teuthology.orchestra.run.smithi159.stdout: mgr/telemetry/nag = false 2024-04-04T06:17:36.911 INFO:teuthology.orchestra.run.smithi159.stdout: debug client = 20 2024-04-04T06:17:36.911 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.911 INFO:teuthology.orchestra.run.smithi159.stdout:[mon] 2024-04-04T06:17:36.911 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:17:36.911 INFO:teuthology.orchestra.run.smithi159.stdout: debug mon = 20 2024-04-04T06:17:36.911 INFO:teuthology.orchestra.run.smithi159.stdout: debug paxos = 20 2024-04-04T06:17:36.911 INFO:teuthology.orchestra.run.smithi159.stdout: debug auth = 20 2024-04-04T06:17:36.911 INFO:teuthology.orchestra.run.smithi159.stdout: mon data avail warn = 5 2024-04-04T06:17:36.911 INFO:teuthology.orchestra.run.smithi159.stdout: mon mgr mkfs grace = 240 2024-04-04T06:17:36.911 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min pgs per osd = 4 2024-04-04T06:17:36.911 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd reporter subtree level = osd 2024-04-04T06:17:36.911 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd prime pg temp = true 2024-04-04T06:17:36.911 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min bytes per osd = 10 2024-04-04T06:17:36.912 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.912 INFO:teuthology.orchestra.run.smithi159.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-04T06:17:36.912 INFO:teuthology.orchestra.run.smithi159.stdout: auth mon ticket ttl = 660 # 11m 2024-04-04T06:17:36.912 INFO:teuthology.orchestra.run.smithi159.stdout: auth service ticket ttl = 240 # 4m 2024-04-04T06:17:36.912 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.912 INFO:teuthology.orchestra.run.smithi159.stdout: # don't complain about insecure global_id in the test suite 2024-04-04T06:17:36.912 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-04T06:17:36.912 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-04T06:17:36.912 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.912 INFO:teuthology.orchestra.run.smithi159.stdout: # 1m isn't quite enough 2024-04-04T06:17:36.912 INFO:teuthology.orchestra.run.smithi159.stdout: mon_down_mkfs_grace = 2m 2024-04-04T06:17:36.912 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.912 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_filestore_osds = false 2024-04-04T06:17:36.912 INFO:teuthology.orchestra.run.smithi159.stdout: mon down mkfs grace = 300 2024-04-04T06:17:36.912 INFO:teuthology.orchestra.run.smithi159.stdout: mon op complaint time = 120 2024-04-04T06:17:36.913 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:36.913 INFO:teuthology.orchestra.run.smithi159.stdout:[client] 2024-04-04T06:17:36.913 INFO:teuthology.orchestra.run.smithi159.stdout: rgw cache enabled = true 2024-04-04T06:17:36.913 INFO:teuthology.orchestra.run.smithi159.stdout: rgw enable ops log = true 2024-04-04T06:17:36.913 INFO:teuthology.orchestra.run.smithi159.stdout: rgw enable usage log = true 2024-04-04T06:17:36.913 INFO:teuthology.orchestra.run.smithi159.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-04T06:17:36.913 INFO:teuthology.orchestra.run.smithi159.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-04T06:17:36.913 INFO:teuthology.orchestra.run.smithi159.stdout: client mount timeout = 600 2024-04-04T06:17:36.913 INFO:teuthology.orchestra.run.smithi159.stdout: debug client = 20 2024-04-04T06:17:36.913 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:17:36.913 INFO:teuthology.orchestra.run.smithi159.stdout: pid file = "" 2024-04-04T06:17:36.913 INFO:teuthology.orchestra.run.smithi159.stdout: rados mon op timeout = 900 2024-04-04T06:17:36.913 INFO:teuthology.orchestra.run.smithi159.stdout: rados osd op timeout = 900 2024-04-04T06:17:36.913 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.a] 2024-04-04T06:17:36.913 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.b] 2024-04-04T06:17:36.914 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.c] 2024-04-04T06:17:36.914 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.a] 2024-04-04T06:17:36.914 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.c] 2024-04-04T06:17:36.914 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.b] 2024-04-04T06:17:36.914 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.d] 2024-04-04T06:17:36.914 INFO:teuthology.orchestra.run.smithi159.stdout:[mds] 2024-04-04T06:17:36.914 INFO:teuthology.orchestra.run.smithi159.stdout: debug mds = 20 2024-04-04T06:17:36.914 INFO:teuthology.orchestra.run.smithi159.stdout: debug mds balancer = 20 2024-04-04T06:17:36.914 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2024-04-04T06:17:36.914 INFO:teuthology.orchestra.run.smithi159.stdout: mds debug frag = True 2024-04-04T06:17:36.914 INFO:teuthology.orchestra.run.smithi159.stdout: mds debug scatterstat = True 2024-04-04T06:17:36.914 INFO:teuthology.orchestra.run.smithi159.stdout: mds op complaint time = 180 2024-04-04T06:17:36.914 INFO:teuthology.orchestra.run.smithi159.stdout: mds verify scatter = True 2024-04-04T06:17:36.914 INFO:teuthology.orchestra.run.smithi159.stdout: osd op complaint time = 180 2024-04-04T06:17:36.914 INFO:teuthology.orchestra.run.smithi159.stdout: rados mon op timeout = 900 2024-04-04T06:17:36.915 INFO:teuthology.orchestra.run.smithi159.stdout: rados osd op timeout = 900 2024-04-04T06:17:36.915 INFO:teuthology.orchestra.run.smithi178.stdout:[global] 2024-04-04T06:17:36.915 INFO:teuthology.orchestra.run.smithi178.stdout: chdir = "" 2024-04-04T06:17:36.915 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-04T06:17:36.915 INFO:teuthology.orchestra.run.smithi178.stdout: auth supported = cephx 2024-04-04T06:17:36.915 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.916 INFO:teuthology.orchestra.run.smithi178.stdout: filestore xattr use omap = true 2024-04-04T06:17:36.916 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.916 INFO:teuthology.orchestra.run.smithi178.stdout: mon clock drift allowed = 1.000 2024-04-04T06:17:36.916 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.916 INFO:teuthology.orchestra.run.smithi178.stdout: osd crush chooseleaf type = 0 2024-04-04T06:17:36.916 INFO:teuthology.orchestra.run.smithi178.stdout: auth debug = true 2024-04-04T06:17:36.916 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.916 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on old message = true 2024-04-04T06:17:36.916 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on bug = true 2024-04-04T06:17:36.916 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.916 INFO:teuthology.orchestra.run.smithi178.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-04T06:17:36.916 INFO:teuthology.orchestra.run.smithi178.stdout: mon pg warn max object skew = 0 2024-04-04T06:17:36.916 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.916 INFO:teuthology.orchestra.run.smithi178.stdout: # disable pg_autoscaler by default for new pools 2024-04-04T06:17:36.917 INFO:teuthology.orchestra.run.smithi178.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-04T06:17:36.917 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.917 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default size = 2 2024-04-04T06:17:36.917 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.917 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow primary affinity = true 2024-04-04T06:17:36.917 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow pg remap = true 2024-04-04T06:17:36.917 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on legacy crush tunables = false 2024-04-04T06:17:36.917 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on crush straw calc version zero = false 2024-04-04T06:17:36.917 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on no sortbitwise = false 2024-04-04T06:17:36.917 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on osd down out interval zero = false 2024-04-04T06:17:36.917 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on too few osds = false 2024-04-04T06:17:36.917 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-04T06:17:36.917 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-04T06:17:36.917 INFO:teuthology.orchestra.run.smithi178.stdout: mon_allow_pool_size_one = true 2024-04-04T06:17:36.918 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.918 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-04T06:17:36.918 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.918 INFO:teuthology.orchestra.run.smithi178.stdout: osd default data pool replay window = 5 2024-04-04T06:17:36.918 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.918 INFO:teuthology.orchestra.run.smithi178.stdout: mon allow pool delete = true 2024-04-04T06:17:36.918 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.918 INFO:teuthology.orchestra.run.smithi178.stdout: mon cluster log file level = debug 2024-04-04T06:17:36.918 INFO:teuthology.orchestra.run.smithi178.stdout: debug asserts on shutdown = true 2024-04-04T06:17:36.918 INFO:teuthology.orchestra.run.smithi178.stdout: mon health detail to clog = false 2024-04-04T06:17:36.918 INFO:teuthology.orchestra.run.smithi178.stdout: mon host = "172.21.15.159,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2024-04-04T06:17:36.918 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on skipped message = False 2024-04-04T06:17:36.919 INFO:teuthology.orchestra.run.smithi178.stdout: fsid = 30311045-f714-48fa-832b-4cb2ba8bf0bf 2024-04-04T06:17:36.919 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.919 INFO:teuthology.orchestra.run.smithi178.stdout:[osd] 2024-04-04T06:17:36.919 INFO:teuthology.orchestra.run.smithi178.stdout: osd journal size = 100 2024-04-04T06:17:36.919 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.919 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub load threshold = 5.0 2024-04-04T06:17:36.919 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub max interval = 600 2024-04-04T06:17:36.919 INFO:teuthology.orchestra.run.smithi178.stdout: osd mclock profile = high_recovery_ops 2024-04-04T06:17:36.920 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.920 INFO:teuthology.orchestra.run.smithi178.stdout: osd recover clone overlap = true 2024-04-04T06:17:36.920 INFO:teuthology.orchestra.run.smithi178.stdout: osd recovery max chunk = 1048576 2024-04-04T06:17:36.920 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.920 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug shutdown = true 2024-04-04T06:17:36.920 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug op order = true 2024-04-04T06:17:36.920 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify stray on activate = true 2024-04-04T06:17:36.920 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug trim objects = true 2024-04-04T06:17:36.920 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.920 INFO:teuthology.orchestra.run.smithi178.stdout: osd open classes on start = true 2024-04-04T06:17:36.920 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug pg log writeout = true 2024-04-04T06:17:36.920 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.920 INFO:teuthology.orchestra.run.smithi178.stdout: osd deep scrub update digest min age = 30 2024-04-04T06:17:36.920 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.920 INFO:teuthology.orchestra.run.smithi178.stdout: osd map max advance = 10 2024-04-04T06:17:36.921 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.921 INFO:teuthology.orchestra.run.smithi178.stdout: journal zero on create = true 2024-04-04T06:17:36.921 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.921 INFO:teuthology.orchestra.run.smithi178.stdout: filestore ondisk finisher threads = 3 2024-04-04T06:17:36.921 INFO:teuthology.orchestra.run.smithi178.stdout: filestore apply finisher threads = 3 2024-04-04T06:17:36.921 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.921 INFO:teuthology.orchestra.run.smithi178.stdout: bdev debug aio = true 2024-04-04T06:17:36.921 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug misdirected ops = true 2024-04-04T06:17:36.921 INFO:teuthology.orchestra.run.smithi178.stdout: bdev async discard = True 2024-04-04T06:17:36.921 INFO:teuthology.orchestra.run.smithi178.stdout: bdev enable discard = True 2024-04-04T06:17:36.921 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore allocator = bitmap 2024-04-04T06:17:36.921 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore block size = 96636764160 2024-04-04T06:17:36.921 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore fsck on mount = True 2024-04-04T06:17:36.922 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluefs = 1/20 2024-04-04T06:17:36.922 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluestore = 1/20 2024-04-04T06:17:36.922 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:17:36.922 INFO:teuthology.orchestra.run.smithi178.stdout: debug osd = 20 2024-04-04T06:17:36.922 INFO:teuthology.orchestra.run.smithi178.stdout: debug rocksdb = 4/10 2024-04-04T06:17:36.922 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd backfillfull_ratio = 0.85 2024-04-04T06:17:36.922 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd full ratio = 0.9 2024-04-04T06:17:36.922 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd nearfull ratio = 0.8 2024-04-04T06:17:36.922 INFO:teuthology.orchestra.run.smithi178.stdout: osd failsafe full ratio = 0.95 2024-04-04T06:17:36.922 INFO:teuthology.orchestra.run.smithi178.stdout: osd objectstore = bluestore 2024-04-04T06:17:36.922 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-04-04T06:17:36.922 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.922 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr] 2024-04-04T06:17:36.922 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:17:36.923 INFO:teuthology.orchestra.run.smithi178.stdout: debug mgr = 20 2024-04-04T06:17:36.923 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-04T06:17:36.923 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-04T06:17:36.923 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-04T06:17:36.923 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-04T06:17:36.923 INFO:teuthology.orchestra.run.smithi178.stdout: mgr/telemetry/nag = false 2024-04-04T06:17:36.923 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-04-04T06:17:36.923 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.923 INFO:teuthology.orchestra.run.smithi178.stdout:[mon] 2024-04-04T06:17:36.923 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:17:36.923 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-04T06:17:36.923 INFO:teuthology.orchestra.run.smithi178.stdout: debug paxos = 20 2024-04-04T06:17:36.923 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-04T06:17:36.923 INFO:teuthology.orchestra.run.smithi178.stdout: mon data avail warn = 5 2024-04-04T06:17:36.924 INFO:teuthology.orchestra.run.smithi178.stdout: mon mgr mkfs grace = 240 2024-04-04T06:17:36.924 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-04T06:17:36.924 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd reporter subtree level = osd 2024-04-04T06:17:36.924 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd prime pg temp = true 2024-04-04T06:17:36.924 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-04T06:17:36.924 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.924 INFO:teuthology.orchestra.run.smithi178.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-04T06:17:36.924 INFO:teuthology.orchestra.run.smithi178.stdout: auth mon ticket ttl = 660 # 11m 2024-04-04T06:17:36.924 INFO:teuthology.orchestra.run.smithi178.stdout: auth service ticket ttl = 240 # 4m 2024-04-04T06:17:36.924 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.924 INFO:teuthology.orchestra.run.smithi178.stdout: # don't complain about insecure global_id in the test suite 2024-04-04T06:17:36.924 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-04T06:17:36.924 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-04T06:17:36.925 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.925 INFO:teuthology.orchestra.run.smithi178.stdout: # 1m isn't quite enough 2024-04-04T06:17:36.925 INFO:teuthology.orchestra.run.smithi178.stdout: mon_down_mkfs_grace = 2m 2024-04-04T06:17:36.925 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.925 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_filestore_osds = false 2024-04-04T06:17:36.925 INFO:teuthology.orchestra.run.smithi178.stdout: mon down mkfs grace = 300 2024-04-04T06:17:36.925 INFO:teuthology.orchestra.run.smithi178.stdout: mon op complaint time = 120 2024-04-04T06:17:36.925 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-04T06:17:36.925 INFO:teuthology.orchestra.run.smithi178.stdout:[client] 2024-04-04T06:17:36.925 INFO:teuthology.orchestra.run.smithi178.stdout: rgw cache enabled = true 2024-04-04T06:17:36.925 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable ops log = true 2024-04-04T06:17:36.925 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable usage log = true 2024-04-04T06:17:36.925 INFO:teuthology.orchestra.run.smithi178.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-04T06:17:36.925 INFO:teuthology.orchestra.run.smithi178.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-04T06:17:36.926 INFO:teuthology.orchestra.run.smithi178.stdout: client mount timeout = 600 2024-04-04T06:17:36.926 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2024-04-04T06:17:36.926 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:17:36.926 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = "" 2024-04-04T06:17:36.926 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-04-04T06:17:36.926 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-04-04T06:17:36.926 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.a] 2024-04-04T06:17:36.926 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.b] 2024-04-04T06:17:36.926 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.c] 2024-04-04T06:17:36.926 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.a] 2024-04-04T06:17:36.926 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.c] 2024-04-04T06:17:36.926 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.b] 2024-04-04T06:17:36.926 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.d] 2024-04-04T06:17:36.927 INFO:teuthology.orchestra.run.smithi178.stdout:[mds] 2024-04-04T06:17:36.927 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds = 20 2024-04-04T06:17:36.927 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds balancer = 20 2024-04-04T06:17:36.927 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-04T06:17:36.927 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug frag = True 2024-04-04T06:17:36.927 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug scatterstat = True 2024-04-04T06:17:36.927 INFO:teuthology.orchestra.run.smithi178.stdout: mds op complaint time = 180 2024-04-04T06:17:36.927 INFO:teuthology.orchestra.run.smithi178.stdout: mds verify scatter = True 2024-04-04T06:17:36.927 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2024-04-04T06:17:36.927 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2024-04-04T06:17:36.927 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2024-04-04T06:17:36.955 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-04T06:17:37.171 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.176+0000 7f0488800640 1 Processor -- start 2024-04-04T06:17:37.172 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.177+0000 7f0488800640 1 -- start start 2024-04-04T06:17:37.173 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.178+0000 7f0488800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0480162920 0x7f04801670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:37.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.178+0000 7f0488800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f04800d0990 0x7f0480071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:37.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.178+0000 7f0488800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0480071d60 con 0x7f0480163ac0 2024-04-04T06:17:37.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.178+0000 7f0488800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f0480163f20 con 0x7f04800d0990 2024-04-04T06:17:37.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.178+0000 7f0488800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f04801665d0 con 0x7f0480162920 2024-04-04T06:17:37.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.178+0000 7f0485400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f0480163ac0 0x7f04800d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:50040/0 (socket says 172.21.15.159:50040) 2024-04-04T06:17:37.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.178+0000 7f0485400640 1 -- 172.21.15.159:0/198385206 learned_addr learned my addr 172.21.15.159:0/198385206 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:37.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.178+0000 7f0486800640 1 --2- 172.21.15.159:0/198385206 >> v2:172.21.15.159:3300/0 conn(0x7f04800d0990 0x7f0480071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:37.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.179+0000 7f0485e00640 1 --2- 172.21.15.159:0/198385206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0480162920 0x7f04801670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:37.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.179+0000 7f0486800640 1 -- 172.21.15.159:0/198385206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0480162920 msgr2=0x7f04801670b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:37.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.179+0000 7f0486800640 1 --2- 172.21.15.159:0/198385206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0480162920 0x7f04801670b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:37.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.179+0000 7f0486800640 1 -- 172.21.15.159:0/198385206 >> v1:172.21.15.159:6789/0 conn(0x7f0480163ac0 legacy=0x7f04800d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:37.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.179+0000 7f0486800640 1 -- 172.21.15.159:0/198385206 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04801668b0 con 0x7f04800d0990 2024-04-04T06:17:37.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.180+0000 7f0486800640 1 --2- 172.21.15.159:0/198385206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04800d0990 0x7f0480071610 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f047c009d90 tx=0x7f047c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d05c5fe1812403 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:37.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.180+0000 7f0484a00640 1 -- 172.21.15.159:0/198385206 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c039070 con 0x7f04800d0990 2024-04-04T06:17:37.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.180+0000 7f0484a00640 1 -- 172.21.15.159:0/198385206 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f047c02cbd0 con 0x7f04800d0990 2024-04-04T06:17:37.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.180+0000 7f0484a00640 1 -- 172.21.15.159:0/198385206 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c034440 con 0x7f04800d0990 2024-04-04T06:17:37.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.181+0000 7f0488800640 1 -- 172.21.15.159:0/198385206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04800d0990 msgr2=0x7f0480071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:37.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.181+0000 7f0488800640 1 --2- 172.21.15.159:0/198385206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04800d0990 0x7f0480071610 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f047c009d90 tx=0x7f047c02be20 comp rx=0 tx=0).stop 2024-04-04T06:17:37.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.181+0000 7f0488800640 1 -- 172.21.15.159:0/198385206 shutdown_connections 2024-04-04T06:17:37.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.181+0000 7f0488800640 1 --2- 172.21.15.159:0/198385206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04800d0990 0x7f0480071610 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:37.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.181+0000 7f0488800640 1 --2- 172.21.15.159:0/198385206 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0480162920 0x7f04801670b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:37.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.181+0000 7f0488800640 1 -- 172.21.15.159:0/198385206 >> 172.21.15.159:0/198385206 conn(0x7f048006b750 msgr2=0x7f048006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:37.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.181+0000 7f0488800640 1 -- 172.21.15.159:0/198385206 shutdown_connections 2024-04-04T06:17:37.177 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.181+0000 7f0488800640 1 -- 172.21.15.159:0/198385206 wait complete. 2024-04-04T06:17:37.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.183+0000 7f0488800640 1 Processor -- start 2024-04-04T06:17:37.178 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.183+0000 7f0488800640 1 -- start start 2024-04-04T06:17:37.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.184+0000 7f0488800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04800d0990 0x7f0480152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:37.183 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.184+0000 7f0488800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0480162920 0x7f04800cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:37.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.184+0000 7f0488800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0480163ac0 0x7f048006d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:37.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.184+0000 7f0488800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0480072040 con 0x7f04800d0990 2024-04-04T06:17:37.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.184+0000 7f0488800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0480162d00 con 0x7f0480163ac0 2024-04-04T06:17:37.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.184+0000 7f0488800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f04800721c0 con 0x7f0480162920 2024-04-04T06:17:37.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.184+0000 7f0485e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04800d0990 0x7f0480152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:37.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.184+0000 7f0485e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04800d0990 0x7f0480152590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42012/0 (socket says 172.21.15.159:42012) 2024-04-04T06:17:37.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.184+0000 7f0485e00640 1 -- 172.21.15.159:0/1975434841 learned_addr learned my addr 172.21.15.159:0/1975434841 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:37.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.185+0000 7f0486800640 1 --2- 172.21.15.159:0/1975434841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0480163ac0 0x7f048006d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:37.184 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.185+0000 7f0485e00640 1 -- 172.21.15.159:0/1975434841 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0480162920 msgr2=0x7f04800cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:37.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.185+0000 7f0485e00640 1 --2- 172.21.15.159:0/1975434841 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0480162920 0x7f04800cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:37.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.185+0000 7f0485e00640 1 -- 172.21.15.159:0/1975434841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0480163ac0 msgr2=0x7f048006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:37.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.185+0000 7f0485e00640 1 --2- 172.21.15.159:0/1975434841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0480163ac0 0x7f048006d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:37.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.185+0000 7f0485e00640 1 -- 172.21.15.159:0/1975434841 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04800ce080 con 0x7f04800d0990 2024-04-04T06:17:37.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.185+0000 7f0485e00640 1 --2- 172.21.15.159:0/1975434841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04800d0990 0x7f0480152590 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f047400d830 tx=0x7f047400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:37.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.185+0000 7f047a000640 1 -- 172.21.15.159:0/1975434841 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047400b660 con 0x7f04800d0990 2024-04-04T06:17:37.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.185+0000 7f047a000640 1 -- 172.21.15.159:0/1975434841 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0474019070 con 0x7f04800d0990 2024-04-04T06:17:37.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.185+0000 7f047a000640 1 -- 172.21.15.159:0/1975434841 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047400ba20 con 0x7f04800d0990 2024-04-04T06:17:37.187 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.185+0000 7f0488800640 1 -- 172.21.15.159:0/1975434841 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04800b3cf0 con 0x7f04800d0990 2024-04-04T06:17:37.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.186+0000 7f0488800640 1 -- 172.21.15.159:0/1975434841 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04800b42b0 con 0x7f04800d0990 2024-04-04T06:17:37.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.187+0000 7f0488800640 1 -- 172.21.15.159:0/1975434841 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04800685c0 con 0x7f04800d0990 2024-04-04T06:17:37.188 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.188+0000 7f047a000640 1 -- 172.21.15.159:0/1975434841 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f047401e460 con 0x7f04800d0990 2024-04-04T06:17:37.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.189+0000 7f047a000640 1 --2- 172.21.15.159:0/1975434841 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f046406c8f0 0x7f046406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:37.190 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.189+0000 7f047a000640 1 -- 172.21.15.159:0/1975434841 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f047408c2d0 con 0x7f04800d0990 2024-04-04T06:17:37.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.190+0000 7f0485400640 1 --2- 172.21.15.159:0/1975434841 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f046406c8f0 0x7f046406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:37.191 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.191+0000 7f0485400640 1 --2- 172.21.15.159:0/1975434841 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f046406c8f0 0x7f046406edb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0470000f60 tx=0x7f0470007a80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:37.192 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.194+0000 7f047a000640 1 -- 172.21.15.159:0/1975434841 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f04800ce080 con 0x7f04800d0990 2024-04-04T06:17:37.431 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.436+0000 7f0488800640 1 -- 172.21.15.159:0/1975434841 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f048006ef60 con 0x7f04800d0990 2024-04-04T06:17:37.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.675+0000 7f047a000640 1 -- 172.21.15.159:0/1975434841 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v110) ==== 162+0+0 (secure 0 0 0) 0x7f047405a600 con 0x7f04800d0990 2024-04-04T06:17:37.671 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-04T06:17:37.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.679+0000 7f0488800640 1 -- 172.21.15.159:0/1975434841 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f046406c8f0 msgr2=0x7f046406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:37.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.679+0000 7f0488800640 1 --2- 172.21.15.159:0/1975434841 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f046406c8f0 0x7f046406edb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0470000f60 tx=0x7f0470007a80 comp rx=0 tx=0).stop 2024-04-04T06:17:37.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.679+0000 7f0488800640 1 -- 172.21.15.159:0/1975434841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04800d0990 msgr2=0x7f0480152590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:37.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.679+0000 7f0488800640 1 --2- 172.21.15.159:0/1975434841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04800d0990 0x7f0480152590 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f047400d830 tx=0x7f047400dcf0 comp rx=0 tx=0).stop 2024-04-04T06:17:37.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.679+0000 7f0488800640 1 -- 172.21.15.159:0/1975434841 shutdown_connections 2024-04-04T06:17:37.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.679+0000 7f0488800640 1 --2- 172.21.15.159:0/1975434841 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f046406c8f0 0x7f046406edb0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:37.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.679+0000 7f0488800640 1 --2- 172.21.15.159:0/1975434841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0480163ac0 0x7f048006d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:37.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.679+0000 7f0488800640 1 --2- 172.21.15.159:0/1975434841 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0480162920 0x7f04800cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:37.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.679+0000 7f0488800640 1 --2- 172.21.15.159:0/1975434841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04800d0990 0x7f0480152590 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:37.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.679+0000 7f0488800640 1 -- 172.21.15.159:0/1975434841 >> 172.21.15.159:0/1975434841 conn(0x7f048006b750 msgr2=0x7f048006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:37.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.679+0000 7f0488800640 1 -- 172.21.15.159:0/1975434841 shutdown_connections 2024-04-04T06:17:37.675 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.679+0000 7f0488800640 1 -- 172.21.15.159:0/1975434841 wait complete. 2024-04-04T06:17:37.697 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:17:37.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.894+0000 7f2420a00640 1 Processor -- start 2024-04-04T06:17:37.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.894+0000 7f2420a00640 1 -- start start 2024-04-04T06:17:37.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.895+0000 7f2420a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f241c0cc0b0 0x7f241c06dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:37.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.895+0000 7f2420a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f241c06e4f0 con 0x7f241c15fc50 2024-04-04T06:17:37.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.895+0000 7f2420a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f241c0b3150 con 0x7f241c0d04d0 2024-04-04T06:17:37.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.895+0000 7f2420a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f241c15c610 con 0x7f241c0cc0b0 2024-04-04T06:17:37.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.896+0000 7f241a800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f241c0cc0b0 0x7f241c06dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:37.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.896+0000 7f2413e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f241c15fc50 0x7f241c0cfdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:50042/0 (socket says 172.21.15.159:50042) 2024-04-04T06:17:37.891 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.896+0000 7f2413e00640 1 -- 172.21.15.159:0/4008793630 learned_addr learned my addr 172.21.15.159:0/4008793630 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:37.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.896+0000 7f2419e00640 1 -- 172.21.15.159:0/4008793630 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 653582250 0 0) 0x7f241c06e4f0 con 0x7f241c15fc50 2024-04-04T06:17:37.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.896+0000 7f2419e00640 1 -- 172.21.15.159:0/4008793630 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2400003520 con 0x7f241c15fc50 2024-04-04T06:17:37.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.896+0000 7f241a800640 1 -- 172.21.15.159:0/4008793630 >> v1:172.21.15.178:6789/0 conn(0x7f241c0d04d0 legacy=0x7f241c15ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:37.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.896+0000 7f241a800640 1 -- 172.21.15.159:0/4008793630 >> v1:172.21.15.159:6789/0 conn(0x7f241c15fc50 legacy=0x7f241c0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:37.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.896+0000 7f241a800640 1 -- 172.21.15.159:0/4008793630 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c1600e0 con 0x7f241c0cc0b0 2024-04-04T06:17:37.892 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.898+0000 7f241a800640 1 --2- 172.21.15.159:0/4008793630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f241c0cc0b0 0x7f241c06dfb0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f240c0097a0 tx=0x7f240c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b8e357cfc81552 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:37.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.898+0000 7f2419e00640 1 -- 172.21.15.159:0/4008793630 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c0315a0 con 0x7f241c0cc0b0 2024-04-04T06:17:37.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.898+0000 7f2419e00640 1 -- 172.21.15.159:0/4008793630 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f240c037070 con 0x7f241c0cc0b0 2024-04-04T06:17:37.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.898+0000 7f2419e00640 1 -- 172.21.15.159:0/4008793630 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c031940 con 0x7f241c0cc0b0 2024-04-04T06:17:37.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.899+0000 7f2420a00640 1 -- 172.21.15.159:0/4008793630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f241c0cc0b0 msgr2=0x7f241c06dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:37.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.899+0000 7f2420a00640 1 --2- 172.21.15.159:0/4008793630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f241c0cc0b0 0x7f241c06dfb0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f240c0097a0 tx=0x7f240c02b840 comp rx=0 tx=0).stop 2024-04-04T06:17:37.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.899+0000 7f2420a00640 1 -- 172.21.15.159:0/4008793630 shutdown_connections 2024-04-04T06:17:37.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.899+0000 7f2420a00640 1 --2- 172.21.15.159:0/4008793630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f241c0cc0b0 0x7f241c06dfb0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:37.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.899+0000 7f2420a00640 1 -- 172.21.15.159:0/4008793630 >> 172.21.15.159:0/4008793630 conn(0x7f241c0ccad0 msgr2=0x7f241c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:37.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.899+0000 7f2420a00640 1 -- 172.21.15.159:0/4008793630 shutdown_connections 2024-04-04T06:17:37.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.899+0000 7f2420a00640 1 -- 172.21.15.159:0/4008793630 wait complete. 2024-04-04T06:17:37.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.901+0000 7f2420a00640 1 Processor -- start 2024-04-04T06:17:37.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.901+0000 7f2420a00640 1 -- start start 2024-04-04T06:17:37.902 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.902+0000 7f2420a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f241c0cc0b0 0x7f241c137f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:37.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.902+0000 7f2420a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f241c0d04d0 0x7f241c1f7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:37.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.902+0000 7f2420a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f241c13e540 0x7f241c0b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:37.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.902+0000 7f2420a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f241c160980 con 0x7f241c13e540 2024-04-04T06:17:37.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.902+0000 7f2420a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1606a0 con 0x7f241c0d04d0 2024-04-04T06:17:37.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.902+0000 7f2420a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f241c0cc490 con 0x7f241c0cc0b0 2024-04-04T06:17:37.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.902+0000 7f241b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f241c13e540 0x7f241c0b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:37.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.902+0000 7f241b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f241c13e540 0x7f241c0b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42034/0 (socket says 172.21.15.159:42034) 2024-04-04T06:17:37.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.902+0000 7f241b200640 1 -- 172.21.15.159:0/2978829403 learned_addr learned my addr 172.21.15.159:0/2978829403 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:37.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.902+0000 7f241a800640 1 --2- 172.21.15.159:0/2978829403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f241c0cc0b0 0x7f241c137f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:37.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.903+0000 7f241b200640 1 -- 172.21.15.159:0/2978829403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f241c0cc0b0 msgr2=0x7f241c137f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:37.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.903+0000 7f241b200640 1 --2- 172.21.15.159:0/2978829403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f241c0cc0b0 0x7f241c137f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:37.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.903+0000 7f2413e00640 1 --2- 172.21.15.159:0/2978829403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f241c0d04d0 0x7f241c1f7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:37.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.903+0000 7f241b200640 1 -- 172.21.15.159:0/2978829403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f241c0d04d0 msgr2=0x7f241c1f7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:37.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.903+0000 7f241b200640 1 --2- 172.21.15.159:0/2978829403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f241c0d04d0 0x7f241c1f7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:37.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.903+0000 7f241b200640 1 -- 172.21.15.159:0/2978829403 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c13ac00 con 0x7f241c13e540 2024-04-04T06:17:37.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.903+0000 7f241b200640 1 --2- 172.21.15.159:0/2978829403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f241c13e540 0x7f241c0b1d30 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f241400c780 tx=0x7f241400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:37.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.903+0000 7f2412a00640 1 -- 172.21.15.159:0/2978829403 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414004b60 con 0x7f241c13e540 2024-04-04T06:17:37.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.903+0000 7f2412a00640 1 -- 172.21.15.159:0/2978829403 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2414004d00 con 0x7f241c13e540 2024-04-04T06:17:37.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.903+0000 7f2412a00640 1 -- 172.21.15.159:0/2978829403 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241400aa90 con 0x7f241c13e540 2024-04-04T06:17:37.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.903+0000 7f2420a00640 1 -- 172.21.15.159:0/2978829403 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241c13aee0 con 0x7f241c13e540 2024-04-04T06:17:37.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.903+0000 7f2420a00640 1 -- 172.21.15.159:0/2978829403 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241c0b1530 con 0x7f241c13e540 2024-04-04T06:17:37.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.905+0000 7f2420a00640 1 -- 172.21.15.159:0/2978829403 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f241c075940 con 0x7f241c13e540 2024-04-04T06:17:37.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.905+0000 7f2412a00640 1 -- 172.21.15.159:0/2978829403 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f241400ac30 con 0x7f241c13e540 2024-04-04T06:17:37.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.906+0000 7f2412a00640 1 --2- 172.21.15.159:0/2978829403 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f23f806c8f0 0x7f23f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:37.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.907+0000 7f2412a00640 1 -- 172.21.15.159:0/2978829403 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f241408bf40 con 0x7f241c13e540 2024-04-04T06:17:37.904 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.907+0000 7f241a800640 1 --2- 172.21.15.159:0/2978829403 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f23f806c8f0 0x7f23f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:37.905 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.908+0000 7f241a800640 1 --2- 172.21.15.159:0/2978829403 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f23f806c8f0 0x7f23f806edb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f240c036000 tx=0x7f240c03d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:37.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:37.911+0000 7f2412a00640 1 -- 172.21.15.159:0/2978829403 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2414020070 con 0x7f241c13e540 2024-04-04T06:17:38.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:38.151+0000 7f2420a00640 1 -- 172.21.15.159:0/2978829403 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f241c06d6e0 con 0x7f241c13e540 2024-04-04T06:17:38.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:38.152+0000 7f2412a00640 1 -- 172.21.15.159:0/2978829403 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5190 (secure 0 0 0) 0x7f241405a270 con 0x7f241c13e540 2024-04-04T06:17:38.147 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:38.147 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5732,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1498324773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1498324773},{"type":"v1","addr":"172.21.15.178:6837","nonce":1498324773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2455031662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":2455031662},{"type":"v1","addr":"172.21.15.159:6835","nonce":2455031662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6343,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1823228151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1823228151},{"type":"v1","addr":"172.21.15.178:6836","nonce":1823228151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:16:57.277344+0000","modified":"2024-04-04T06:17:37.681567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9312":{"gid":9312,"name":"a","rank":0,"incarnation":111,"state":"up:replay","state_seq":1,"addr":"172.21.15.159:6834/643543929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":643543929},{"type":"v1","addr":"172.21.15.159:6834","nonce":643543929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-04T06:17:38.147 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 111 2024-04-04T06:17:38.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:38.154+0000 7f2420a00640 1 -- 172.21.15.159:0/2978829403 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f23f806c8f0 msgr2=0x7f23f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:38.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:38.154+0000 7f2420a00640 1 --2- 172.21.15.159:0/2978829403 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f23f806c8f0 0x7f23f806edb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f240c036000 tx=0x7f240c03d040 comp rx=0 tx=0).stop 2024-04-04T06:17:38.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:38.154+0000 7f2420a00640 1 -- 172.21.15.159:0/2978829403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f241c13e540 msgr2=0x7f241c0b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:38.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:38.154+0000 7f2420a00640 1 --2- 172.21.15.159:0/2978829403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f241c13e540 0x7f241c0b1d30 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f241400c780 tx=0x7f241400cc40 comp rx=0 tx=0).stop 2024-04-04T06:17:38.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:38.155+0000 7f2420a00640 1 -- 172.21.15.159:0/2978829403 shutdown_connections 2024-04-04T06:17:38.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:38.155+0000 7f2420a00640 1 --2- 172.21.15.159:0/2978829403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f241c13e540 0x7f241c0b1d30 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:38.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:38.155+0000 7f2420a00640 1 --2- 172.21.15.159:0/2978829403 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f23f806c8f0 0x7f23f806edb0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:38.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:38.155+0000 7f2420a00640 1 --2- 172.21.15.159:0/2978829403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f241c0d04d0 0x7f241c1f7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:38.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:38.155+0000 7f2420a00640 1 --2- 172.21.15.159:0/2978829403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f241c0cc0b0 0x7f241c137f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:38.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:38.155+0000 7f2420a00640 1 -- 172.21.15.159:0/2978829403 >> 172.21.15.159:0/2978829403 conn(0x7f241c0ccad0 msgr2=0x7f241c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:38.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:38.155+0000 7f2420a00640 1 -- 172.21.15.159:0/2978829403 shutdown_connections 2024-04-04T06:17:38.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:38.155+0000 7f2420a00640 1 -- 172.21.15.159:0/2978829403 wait complete. 2024-04-04T06:17:38.571 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 111, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:16:57.277344+0000', 'modified': '2024-04-04T06:17:37.681567+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 116, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9312}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9312': {'gid': 9312, 'name': 'a', 'rank': 0, 'incarnation': 111, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.159:6834/643543929', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 643543929}, {'type': 'v1', 'addr': '172.21.15.159:6834', 'nonce': 643543929}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-04T06:17:38.571 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9312:up:replay 2024-04-04T06:17:39.571 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:17:39.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.794+0000 7f1e84c00640 1 Processor -- start 2024-04-04T06:17:39.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.794+0000 7f1e84c00640 1 -- start start 2024-04-04T06:17:39.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.795+0000 7f1e84c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f1e8006f8e0 0x7f1e8015bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:39.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.795+0000 7f1e84c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1e800d0780 0x7f1e800d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:39.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.795+0000 7f1e84c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e8015aaf0 0x7f1e8015af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:39.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.795+0000 7f1e84c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f1e800d1120 con 0x7f1e8006f8e0 2024-04-04T06:17:39.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.796+0000 7f1e84c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f1e800b5320 con 0x7f1e800d0780 2024-04-04T06:17:39.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.796+0000 7f1e84c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1e8015b4c0 con 0x7f1e8015aaf0 2024-04-04T06:17:39.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.796+0000 7f1e7de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f1e8006f8e0 0x7f1e8015bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:39.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.796+0000 7f1e7de00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f1e8006f8e0 0x7f1e8015bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42048/0 (socket says 172.21.15.159:42048) 2024-04-04T06:17:39.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.796+0000 7f1e7de00640 1 -- 172.21.15.159:0/352623403 learned_addr learned my addr 172.21.15.159:0/352623403 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:39.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.796+0000 7f1e7e800640 1 --2- 172.21.15.159:0/352623403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e8015aaf0 0x7f1e8015af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:39.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.796+0000 7f1e7d400640 1 --2- 172.21.15.159:0/352623403 >> v2:172.21.15.178:3300/0 conn(0x7f1e800d0780 0x7f1e800d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:39.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.796+0000 7f1e7de00640 1 -- 172.21.15.159:0/352623403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e8015aaf0 msgr2=0x7f1e8015af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:39.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.796+0000 7f1e7de00640 1 --2- 172.21.15.159:0/352623403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e8015aaf0 0x7f1e8015af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:39.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.796+0000 7f1e7de00640 1 -- 172.21.15.159:0/352623403 >> v2:172.21.15.178:3300/0 conn(0x7f1e800d0780 msgr2=0x7f1e800d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:39.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.796+0000 7f1e7de00640 1 --2- 172.21.15.159:0/352623403 >> v2:172.21.15.178:3300/0 conn(0x7f1e800d0780 0x7f1e800d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:39.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.796+0000 7f1e7de00640 1 -- 172.21.15.159:0/352623403 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8015b720 con 0x7f1e8006f8e0 2024-04-04T06:17:39.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.796+0000 7f1e7d400640 1 --2- 172.21.15.159:0/352623403 >> v2:172.21.15.178:3300/0 conn(0x7f1e800d0780 0x7f1e800d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:39.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.798+0000 7f1e7de00640 1 --2- 172.21.15.159:0/352623403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e8006f8e0 0x7f1e8015bcb0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f1e800b2e50 tx=0x7f1e7002e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387bdbc2b5430c9b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:39.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.798+0000 7f1e7ca00640 1 -- 172.21.15.159:0/352623403 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7003b070 con 0x7f1e8006f8e0 2024-04-04T06:17:39.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.798+0000 7f1e7ca00640 1 -- 172.21.15.159:0/352623403 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e70036440 con 0x7f1e8006f8e0 2024-04-04T06:17:39.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.798+0000 7f1e7ca00640 1 -- 172.21.15.159:0/352623403 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70036810 con 0x7f1e8006f8e0 2024-04-04T06:17:39.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.798+0000 7f1e84c00640 1 -- 172.21.15.159:0/352623403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e8006f8e0 msgr2=0x7f1e8015bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:39.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.798+0000 7f1e84c00640 1 --2- 172.21.15.159:0/352623403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e8006f8e0 0x7f1e8015bcb0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f1e800b2e50 tx=0x7f1e7002e0c0 comp rx=0 tx=0).stop 2024-04-04T06:17:39.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.799+0000 7f1e84c00640 1 -- 172.21.15.159:0/352623403 shutdown_connections 2024-04-04T06:17:39.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.799+0000 7f1e84c00640 1 --2- 172.21.15.159:0/352623403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e8015aaf0 0x7f1e8015af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:39.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.799+0000 7f1e84c00640 1 --2- 172.21.15.159:0/352623403 >> v2:172.21.15.178:3300/0 conn(0x7f1e800d0780 0x7f1e800d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:39.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.799+0000 7f1e84c00640 1 --2- 172.21.15.159:0/352623403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e8006f8e0 0x7f1e8015bcb0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:39.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.799+0000 7f1e84c00640 1 -- 172.21.15.159:0/352623403 >> 172.21.15.159:0/352623403 conn(0x7f1e8006b990 msgr2=0x7f1e8006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:39.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.799+0000 7f1e84c00640 1 -- 172.21.15.159:0/352623403 shutdown_connections 2024-04-04T06:17:39.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.799+0000 7f1e84c00640 1 -- 172.21.15.159:0/352623403 wait complete. 2024-04-04T06:17:39.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.800+0000 7f1e84c00640 1 Processor -- start 2024-04-04T06:17:39.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.800+0000 7f1e84c00640 1 -- start start 2024-04-04T06:17:39.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.801+0000 7f1e84c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e8006f8e0 0x7f1e801f09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:39.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.801+0000 7f1e84c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e800d0780 0x7f1e8015d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:39.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.801+0000 7f1e84c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e8015aaf0 0x7f1e800998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:39.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.801+0000 7f1e84c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1e800b34c0 con 0x7f1e800d0780 2024-04-04T06:17:39.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.801+0000 7f1e84c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1e800b31e0 con 0x7f1e8006f8e0 2024-04-04T06:17:39.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.801+0000 7f1e84c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f1e800b37a0 con 0x7f1e8015aaf0 2024-04-04T06:17:39.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.802+0000 7f1e7d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e800d0780 0x7f1e8015d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:39.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.802+0000 7f1e7d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e800d0780 0x7f1e8015d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42064/0 (socket says 172.21.15.159:42064) 2024-04-04T06:17:39.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.802+0000 7f1e7d400640 1 -- 172.21.15.159:0/556837015 learned_addr learned my addr 172.21.15.159:0/556837015 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:39.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.802+0000 7f1e7e800640 1 --2- 172.21.15.159:0/556837015 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e8015aaf0 0x7f1e800998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:39.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.802+0000 7f1e7de00640 1 --2- 172.21.15.159:0/556837015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e8006f8e0 0x7f1e801f09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:39.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.802+0000 7f1e7d400640 1 -- 172.21.15.159:0/556837015 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e8015aaf0 msgr2=0x7f1e800998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:39.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.802+0000 7f1e7d400640 1 --2- 172.21.15.159:0/556837015 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e8015aaf0 0x7f1e800998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:39.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.802+0000 7f1e7d400640 1 -- 172.21.15.159:0/556837015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e8006f8e0 msgr2=0x7f1e801f09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:39.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.802+0000 7f1e7d400640 1 --2- 172.21.15.159:0/556837015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e8006f8e0 0x7f1e801f09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:39.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.802+0000 7f1e7d400640 1 -- 172.21.15.159:0/556837015 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e801f0ee0 con 0x7f1e800d0780 2024-04-04T06:17:39.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.803+0000 7f1e7d400640 1 --2- 172.21.15.159:0/556837015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e800d0780 0x7f1e8015d130 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00c7f0 tx=0x7f1e6c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:39.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.803+0000 7f1e76000640 1 -- 172.21.15.159:0/556837015 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c015070 con 0x7f1e800d0780 2024-04-04T06:17:39.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.803+0000 7f1e76000640 1 -- 172.21.15.159:0/556837015 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e6c007820 con 0x7f1e800d0780 2024-04-04T06:17:39.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.803+0000 7f1e84c00640 1 -- 172.21.15.159:0/556837015 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8015c090 con 0x7f1e800d0780 2024-04-04T06:17:39.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.803+0000 7f1e76000640 1 -- 172.21.15.159:0/556837015 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c007c10 con 0x7f1e800d0780 2024-04-04T06:17:39.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.803+0000 7f1e84c00640 1 -- 172.21.15.159:0/556837015 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e8015c650 con 0x7f1e800d0780 2024-04-04T06:17:39.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.804+0000 7f1e84c00640 1 -- 172.21.15.159:0/556837015 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e800685c0 con 0x7f1e800d0780 2024-04-04T06:17:39.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.805+0000 7f1e76000640 1 -- 172.21.15.159:0/556837015 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e6c018070 con 0x7f1e800d0780 2024-04-04T06:17:39.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.806+0000 7f1e76000640 1 --2- 172.21.15.159:0/556837015 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1e6006c920 0x7f1e6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:39.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.806+0000 7f1e76000640 1 -- 172.21.15.159:0/556837015 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f1e6c08bce0 con 0x7f1e800d0780 2024-04-04T06:17:39.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.807+0000 7f1e7de00640 1 --2- 172.21.15.159:0/556837015 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1e6006c920 0x7f1e6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:39.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.808+0000 7f1e7de00640 1 --2- 172.21.15.159:0/556837015 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1e6006c920 0x7f1e6006ede0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1e70038000 tx=0x7f1e70032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:39.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:39.811+0000 7f1e76000640 1 -- 172.21.15.159:0/556837015 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f1e801f0ee0 con 0x7f1e800d0780 2024-04-04T06:17:40.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:40.046+0000 7f1e84c00640 1 -- 172.21.15.159:0/556837015 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e8006bfc0 con 0x7f1e800d0780 2024-04-04T06:17:40.043 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:40.047+0000 7f1e76000640 1 -- 172.21.15.159:0/556837015 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5190 (secure 0 0 0) 0x7f1e6c059f90 con 0x7f1e800d0780 2024-04-04T06:17:40.043 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:40.043 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":113,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5732,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1498324773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1498324773},{"type":"v1","addr":"172.21.15.178:6837","nonce":1498324773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2455031662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":2455031662},{"type":"v1","addr":"172.21.15.159:6835","nonce":2455031662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6343,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1823228151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1823228151},{"type":"v1","addr":"172.21.15.178:6836","nonce":1823228151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:16:57.277344+0000","modified":"2024-04-04T06:17:38.703359+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9312":{"gid":9312,"name":"a","rank":0,"incarnation":111,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.159:6834/643543929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":643543929},{"type":"v1","addr":"172.21.15.159:6834","nonce":643543929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-04T06:17:40.043 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 113 2024-04-04T06:17:40.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:40.050+0000 7f1e84c00640 1 -- 172.21.15.159:0/556837015 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1e6006c920 msgr2=0x7f1e6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:40.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:40.050+0000 7f1e84c00640 1 --2- 172.21.15.159:0/556837015 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1e6006c920 0x7f1e6006ede0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1e70038000 tx=0x7f1e70032000 comp rx=0 tx=0).stop 2024-04-04T06:17:40.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:40.050+0000 7f1e84c00640 1 -- 172.21.15.159:0/556837015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e800d0780 msgr2=0x7f1e8015d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:40.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:40.050+0000 7f1e84c00640 1 --2- 172.21.15.159:0/556837015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e800d0780 0x7f1e8015d130 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00c7f0 tx=0x7f1e6c00ccb0 comp rx=0 tx=0).stop 2024-04-04T06:17:40.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:40.050+0000 7f1e84c00640 1 -- 172.21.15.159:0/556837015 shutdown_connections 2024-04-04T06:17:40.045 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:40.050+0000 7f1e84c00640 1 --2- 172.21.15.159:0/556837015 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e8015aaf0 0x7f1e800998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:40.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:40.050+0000 7f1e84c00640 1 --2- 172.21.15.159:0/556837015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e800d0780 0x7f1e8015d130 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:40.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:40.050+0000 7f1e84c00640 1 --2- 172.21.15.159:0/556837015 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f1e6006c920 0x7f1e6006ede0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:40.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:40.050+0000 7f1e84c00640 1 --2- 172.21.15.159:0/556837015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e8006f8e0 0x7f1e801f09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:40.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:40.050+0000 7f1e84c00640 1 -- 172.21.15.159:0/556837015 >> 172.21.15.159:0/556837015 conn(0x7f1e8006b990 msgr2=0x7f1e800af620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:40.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:40.050+0000 7f1e84c00640 1 -- 172.21.15.159:0/556837015 shutdown_connections 2024-04-04T06:17:40.046 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:40.050+0000 7f1e84c00640 1 -- 172.21.15.159:0/556837015 wait complete. 2024-04-04T06:17:40.467 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 113, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:16:57.277344+0000', 'modified': '2024-04-04T06:17:38.703359+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 116, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9312}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9312': {'gid': 9312, 'name': 'a', 'rank': 0, 'incarnation': 111, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.159:6834/643543929', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 643543929}, {'type': 'v1', 'addr': '172.21.15.159:6834', 'nonce': 643543929}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-04T06:17:40.467 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9312:up:rejoin 2024-04-04T06:17:41.468 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:17:41.679 INFO:tasks.ceph.mds.a.smithi159.stderr:2024-04-04T06:17:41.684+0000 7f272e600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:17:41.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.687+0000 7f57b9400640 1 Processor -- start 2024-04-04T06:17:41.682 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.687+0000 7f57b9400640 1 -- start start 2024-04-04T06:17:41.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.689+0000 7f57b9400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f57b415e750 0x7f57b40cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:41.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.689+0000 7f57b9400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57b40cd270 0x7f57b406b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:41.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.689+0000 7f57b9400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57b40cd830 con 0x7f57b406bab0 2024-04-04T06:17:41.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.689+0000 7f57b9400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f57b40cdb10 con 0x7f57b415e750 2024-04-04T06:17:41.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.689+0000 7f57b9400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f57b40cddf0 con 0x7f57b40cd270 2024-04-04T06:17:41.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.689+0000 7f57b2800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f57b415e750 0x7f57b40cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:41.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.689+0000 7f57b2800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f57b415e750 0x7f57b40cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42072/0 (socket says 172.21.15.159:42072) 2024-04-04T06:17:41.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.689+0000 7f57b2800640 1 -- 172.21.15.159:0/3965718795 learned_addr learned my addr 172.21.15.159:0/3965718795 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:41.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.689+0000 7f57b3200640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f57b406bab0 0x7f57b41622d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:50058/0 (socket says 172.21.15.159:50058) 2024-04-04T06:17:41.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.690+0000 7f57a9e00640 1 --2- 172.21.15.159:0/3965718795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57b40cd270 0x7f57b406b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:41.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.690+0000 7f57b2800640 1 -- 172.21.15.159:0/3965718795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57b40cd270 msgr2=0x7f57b406b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:41.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.690+0000 7f57b2800640 1 --2- 172.21.15.159:0/3965718795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57b40cd270 0x7f57b406b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:41.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.690+0000 7f57b2800640 1 -- 172.21.15.159:0/3965718795 >> v1:172.21.15.159:6789/0 conn(0x7f57b406bab0 legacy=0x7f57b41622d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:41.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.690+0000 7f57b2800640 1 -- 172.21.15.159:0/3965718795 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b40ce0d0 con 0x7f57b415e750 2024-04-04T06:17:41.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.691+0000 7f57b2800640 1 --2- 172.21.15.159:0/3965718795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57b415e750 0x7f57b40cfa20 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009e50 tx=0x7f57a402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e412b2a17f7592 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:41.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.691+0000 7f57b1e00640 1 -- 172.21.15.159:0/3965718795 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a402cb00 con 0x7f57b415e750 2024-04-04T06:17:41.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.691+0000 7f57b1e00640 1 -- 172.21.15.159:0/3965718795 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a402cca0 con 0x7f57b415e750 2024-04-04T06:17:41.686 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.691+0000 7f57b1e00640 1 -- 172.21.15.159:0/3965718795 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4034610 con 0x7f57b415e750 2024-04-04T06:17:41.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.692+0000 7f57b9400640 1 -- 172.21.15.159:0/3965718795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57b415e750 msgr2=0x7f57b40cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:41.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.692+0000 7f57b9400640 1 --2- 172.21.15.159:0/3965718795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57b415e750 0x7f57b40cfa20 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009e50 tx=0x7f57a402bf60 comp rx=0 tx=0).stop 2024-04-04T06:17:41.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.692+0000 7f57b9400640 1 -- 172.21.15.159:0/3965718795 shutdown_connections 2024-04-04T06:17:41.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.692+0000 7f57b9400640 1 --2- 172.21.15.159:0/3965718795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57b40cd270 0x7f57b406b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:41.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.692+0000 7f57b9400640 1 --2- 172.21.15.159:0/3965718795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57b415e750 0x7f57b40cfa20 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:41.687 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.692+0000 7f57b9400640 1 -- 172.21.15.159:0/3965718795 >> 172.21.15.159:0/3965718795 conn(0x7f57b40b2cb0 msgr2=0x7f57b40b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:41.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.692+0000 7f57b9400640 1 -- 172.21.15.159:0/3965718795 shutdown_connections 2024-04-04T06:17:41.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.692+0000 7f57b9400640 1 -- 172.21.15.159:0/3965718795 wait complete. 2024-04-04T06:17:41.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.694+0000 7f57b9400640 1 Processor -- start 2024-04-04T06:17:41.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.694+0000 7f57b9400640 1 -- start start 2024-04-04T06:17:41.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.695+0000 7f57b9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57b406bab0 0x7f57b4148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:41.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.695+0000 7f57b9400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57b40cd270 0x7f57b41e8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:41.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.695+0000 7f57b9400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57b415e750 0x7f57b415ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:41.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.695+0000 7f57b9400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f57b40ce8d0 con 0x7f57b406bab0 2024-04-04T06:17:41.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.695+0000 7f57b9400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f57b415eb30 con 0x7f57b40cd270 2024-04-04T06:17:41.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.695+0000 7f57b9400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f57b40cea50 con 0x7f57b415e750 2024-04-04T06:17:41.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.695+0000 7f57b2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57b406bab0 0x7f57b4148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:41.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.695+0000 7f57b2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57b406bab0 0x7f57b4148aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42074/0 (socket says 172.21.15.159:42074) 2024-04-04T06:17:41.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.695+0000 7f57b2800640 1 -- 172.21.15.159:0/3310639552 learned_addr learned my addr 172.21.15.159:0/3310639552 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:41.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.696+0000 7f57a9e00640 1 --2- 172.21.15.159:0/3310639552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57b40cd270 0x7f57b41e8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:41.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.696+0000 7f57b3200640 1 --2- 172.21.15.159:0/3310639552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57b415e750 0x7f57b415ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:41.692 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.696+0000 7f57b2800640 1 -- 172.21.15.159:0/3310639552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57b415e750 msgr2=0x7f57b415ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:41.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.696+0000 7f57b2800640 1 --2- 172.21.15.159:0/3310639552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57b415e750 0x7f57b415ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:41.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.696+0000 7f57b2800640 1 -- 172.21.15.159:0/3310639552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57b40cd270 msgr2=0x7f57b41e8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:41.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.696+0000 7f57b2800640 1 --2- 172.21.15.159:0/3310639552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57b40cd270 0x7f57b41e8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:41.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.696+0000 7f57b2800640 1 -- 172.21.15.159:0/3310639552 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b41f80d0 con 0x7f57b406bab0 2024-04-04T06:17:41.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.696+0000 7f57b2800640 1 --2- 172.21.15.159:0/3310639552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57b406bab0 0x7f57b4148aa0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009f70 tx=0x7f57a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:41.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.696+0000 7f57ab400640 1 -- 172.21.15.159:0/3310639552 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4043020 con 0x7f57b406bab0 2024-04-04T06:17:41.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.696+0000 7f57ab400640 1 -- 172.21.15.159:0/3310639552 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a402ce50 con 0x7f57b406bab0 2024-04-04T06:17:41.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.696+0000 7f57ab400640 1 -- 172.21.15.159:0/3310639552 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4009510 con 0x7f57b406bab0 2024-04-04T06:17:41.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.696+0000 7f57b9400640 1 -- 172.21.15.159:0/3310639552 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b41f83b0 con 0x7f57b406bab0 2024-04-04T06:17:41.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.697+0000 7f57b9400640 1 -- 172.21.15.159:0/3310639552 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b41f8970 con 0x7f57b406bab0 2024-04-04T06:17:41.693 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.699+0000 7f57ab400640 1 -- 172.21.15.159:0/3310639552 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57a4046020 con 0x7f57b406bab0 2024-04-04T06:17:41.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.698+0000 7f57b9400640 1 -- 172.21.15.159:0/3310639552 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57b40685c0 con 0x7f57b406bab0 2024-04-04T06:17:41.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.699+0000 7f57ab400640 1 --2- 172.21.15.159:0/3310639552 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f579006c8f0 0x7f579006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:41.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.700+0000 7f57ab400640 1 -- 172.21.15.159:0/3310639552 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f57a40ae2e0 con 0x7f57b406bab0 2024-04-04T06:17:41.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.700+0000 7f57a9e00640 1 --2- 172.21.15.159:0/3310639552 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f579006c8f0 0x7f579006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:41.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.702+0000 7f57a9e00640 1 --2- 172.21.15.159:0/3310639552 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f579006c8f0 0x7f579006edb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f57b40cd0f0 tx=0x7f579c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:41.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.704+0000 7f57ab400640 1 -- 172.21.15.159:0/3310639552 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f57a404d030 con 0x7f57b406bab0 2024-04-04T06:17:41.949 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.954+0000 7f57b9400640 1 -- 172.21.15.159:0/3310639552 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57b406c0a0 con 0x7f57b406bab0 2024-04-04T06:17:41.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.955+0000 7f57ab400640 1 -- 172.21.15.159:0/3310639552 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5197 (secure 0 0 0) 0x7f57a407c620 con 0x7f57b406bab0 2024-04-04T06:17:41.950 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:41.950 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":114,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5732,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1498324773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1498324773},{"type":"v1","addr":"172.21.15.178:6837","nonce":1498324773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2455031662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":2455031662},{"type":"v1","addr":"172.21.15.159:6835","nonce":2455031662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6343,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1823228151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1823228151},{"type":"v1","addr":"172.21.15.178:6836","nonce":1823228151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:16:57.277344+0000","modified":"2024-04-04T06:17:40.711526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9312":{"gid":9312,"name":"a","rank":0,"incarnation":111,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6834/643543929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":643543929},{"type":"v1","addr":"172.21.15.159:6834","nonce":643543929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9312,"qdb_cluster":[9312]},"id":8}]} 2024-04-04T06:17:41.950 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 114 2024-04-04T06:17:41.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.957+0000 7f57b9400640 1 -- 172.21.15.159:0/3310639552 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f579006c8f0 msgr2=0x7f579006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:41.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.957+0000 7f57b9400640 1 --2- 172.21.15.159:0/3310639552 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f579006c8f0 0x7f579006edb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f57b40cd0f0 tx=0x7f579c008040 comp rx=0 tx=0).stop 2024-04-04T06:17:41.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.957+0000 7f57b9400640 1 -- 172.21.15.159:0/3310639552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57b406bab0 msgr2=0x7f57b4148aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:41.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.958+0000 7f57b9400640 1 --2- 172.21.15.159:0/3310639552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57b406bab0 0x7f57b4148aa0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009f70 tx=0x7f57a4004a40 comp rx=0 tx=0).stop 2024-04-04T06:17:41.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.958+0000 7f57b9400640 1 -- 172.21.15.159:0/3310639552 shutdown_connections 2024-04-04T06:17:41.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.958+0000 7f57b9400640 1 --2- 172.21.15.159:0/3310639552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57b415e750 0x7f57b415ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:41.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.958+0000 7f57b9400640 1 --2- 172.21.15.159:0/3310639552 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f579006c8f0 0x7f579006edb0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:41.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.958+0000 7f57b9400640 1 --2- 172.21.15.159:0/3310639552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57b40cd270 0x7f57b41e8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:41.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.958+0000 7f57b9400640 1 --2- 172.21.15.159:0/3310639552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57b406bab0 0x7f57b4148aa0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:41.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.958+0000 7f57b9400640 1 -- 172.21.15.159:0/3310639552 >> 172.21.15.159:0/3310639552 conn(0x7f57b40b2cb0 msgr2=0x7f57b4074580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:41.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.958+0000 7f57b9400640 1 -- 172.21.15.159:0/3310639552 shutdown_connections 2024-04-04T06:17:41.954 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:41.958+0000 7f57b9400640 1 -- 172.21.15.159:0/3310639552 wait complete. 2024-04-04T06:17:42.374 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 114, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-04T06:16:57.277344+0000', 'modified': '2024-04-04T06:17:40.711526+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 116, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9312}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9312': {'gid': 9312, 'name': 'a', 'rank': 0, 'incarnation': 111, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.159:6834/643543929', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 643543929}, {'type': 'v1', 'addr': '172.21.15.159:6834', 'nonce': 643543929}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9312, 'qdb_cluster': [9312]} 2024-04-04T06:17:42.374 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-04T06:17:42.374 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-04T06:17:42.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.584+0000 7f76df400640 1 Processor -- start 2024-04-04T06:17:42.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.584+0000 7f76df400640 1 -- start start 2024-04-04T06:17:42.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.586+0000 7f76df400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f76d8155550 0x7f76d8155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.586+0000 7f76df400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f76d815f800 0x7f76d80d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.586+0000 7f76df400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76d80b40b0 con 0x7f76d80d07c0 2024-04-04T06:17:42.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.586+0000 7f76df400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f76d80d0bf0 con 0x7f76d8155550 2024-04-04T06:17:42.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.586+0000 7f76df400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f76d80b4390 con 0x7f76d815f800 2024-04-04T06:17:42.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.586+0000 7f76dca00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f76d8155550 0x7f76d8155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.586+0000 7f76dca00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f76d8155550 0x7f76d8155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42078/0 (socket says 172.21.15.159:42078) 2024-04-04T06:17:42.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.586+0000 7f76dd400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f76d80d07c0 0x7f76d80aede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:50066/0 (socket says 172.21.15.159:50066) 2024-04-04T06:17:42.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.586+0000 7f76dca00640 1 -- 172.21.15.159:0/3669295199 learned_addr learned my addr 172.21.15.159:0/3669295199 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:42.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.586+0000 7f76d7e00640 1 --2- 172.21.15.159:0/3669295199 >> v2:172.21.15.178:3300/0 conn(0x7f76d815f800 0x7f76d80d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.587+0000 7f76dca00640 1 -- 172.21.15.159:0/3669295199 >> v2:172.21.15.178:3300/0 conn(0x7f76d815f800 msgr2=0x7f76d80d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:42.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.587+0000 7f76dca00640 1 --2- 172.21.15.159:0/3669295199 >> v2:172.21.15.178:3300/0 conn(0x7f76d815f800 0x7f76d80d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.587+0000 7f76dca00640 1 -- 172.21.15.159:0/3669295199 >> v1:172.21.15.159:6789/0 conn(0x7f76d80d07c0 legacy=0x7f76d80aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:42.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.587+0000 7f76dca00640 1 -- 172.21.15.159:0/3669295199 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d80b4670 con 0x7f76d8155550 2024-04-04T06:17:42.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.587+0000 7f76d7400640 1 -- 172.21.15.159:0/3669295199 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3603829440 0 0) 0x7f76d80b40b0 con 0x7f76d80d07c0 2024-04-04T06:17:42.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.588+0000 7f76dca00640 1 --2- 172.21.15.159:0/3669295199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76d8155550 0x7f76d8155930 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f76d80d0dd0 tx=0x7f76cc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3232d542864ebd server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:42.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.589+0000 7f76d7400640 1 -- 172.21.15.159:0/3669295199 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc035020 con 0x7f76d8155550 2024-04-04T06:17:42.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.589+0000 7f76d7400640 1 -- 172.21.15.159:0/3669295199 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76cc031870 con 0x7f76d8155550 2024-04-04T06:17:42.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.589+0000 7f76d7400640 1 -- 172.21.15.159:0/3669295199 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc031c40 con 0x7f76d8155550 2024-04-04T06:17:42.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.589+0000 7f76df400640 1 -- 172.21.15.159:0/3669295199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76d8155550 msgr2=0x7f76d8155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:42.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.589+0000 7f76df400640 1 --2- 172.21.15.159:0/3669295199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76d8155550 0x7f76d8155930 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f76d80d0dd0 tx=0x7f76cc02b910 comp rx=0 tx=0).stop 2024-04-04T06:17:42.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.589+0000 7f76df400640 1 -- 172.21.15.159:0/3669295199 shutdown_connections 2024-04-04T06:17:42.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.589+0000 7f76df400640 1 --2- 172.21.15.159:0/3669295199 >> v2:172.21.15.178:3300/0 conn(0x7f76d815f800 0x7f76d80d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.589+0000 7f76df400640 1 --2- 172.21.15.159:0/3669295199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76d8155550 0x7f76d8155930 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.589+0000 7f76df400640 1 -- 172.21.15.159:0/3669295199 >> 172.21.15.159:0/3669295199 conn(0x7f76d806b750 msgr2=0x7f76d806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:42.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.590+0000 7f76df400640 1 -- 172.21.15.159:0/3669295199 shutdown_connections 2024-04-04T06:17:42.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.590+0000 7f76df400640 1 -- 172.21.15.159:0/3669295199 wait complete. 2024-04-04T06:17:42.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.591+0000 7f76df400640 1 Processor -- start 2024-04-04T06:17:42.586 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.591+0000 7f76df400640 1 -- start start 2024-04-04T06:17:42.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.592+0000 7f76df400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f76d80d07c0 0x7f76d8099970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.592+0000 7f76df400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d8155550 0x7f76d80cf120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.592+0000 7f76df400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76d815f800 0x7f76d80ae330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.592+0000 7f76df400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f76d815fc80 con 0x7f76d815f800 2024-04-04T06:17:42.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.592+0000 7f76df400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f76d80b4c30 con 0x7f76d8155550 2024-04-04T06:17:42.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.592+0000 7f76df400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f76d815ba20 con 0x7f76d80d07c0 2024-04-04T06:17:42.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.593+0000 7f76dd400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76d815f800 0x7f76d80ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.593+0000 7f76dd400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76d815f800 0x7f76d80ae330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42088/0 (socket says 172.21.15.159:42088) 2024-04-04T06:17:42.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.593+0000 7f76dd400640 1 -- 172.21.15.159:0/3907773 learned_addr learned my addr 172.21.15.159:0/3907773 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:42.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.593+0000 7f76d7e00640 1 --2- 172.21.15.159:0/3907773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d8155550 0x7f76d80cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.593+0000 7f76dca00640 1 --2- 172.21.15.159:0/3907773 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f76d80d07c0 0x7f76d8099970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.593+0000 7f76dd400640 1 -- 172.21.15.159:0/3907773 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f76d80d07c0 msgr2=0x7f76d8099970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:42.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.593+0000 7f76dd400640 1 --2- 172.21.15.159:0/3907773 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f76d80d07c0 0x7f76d8099970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.593+0000 7f76dd400640 1 -- 172.21.15.159:0/3907773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d8155550 msgr2=0x7f76d80cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:42.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.593+0000 7f76dd400640 1 --2- 172.21.15.159:0/3907773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d8155550 0x7f76d80cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.593+0000 7f76dd400640 1 -- 172.21.15.159:0/3907773 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d8099eb0 con 0x7f76d815f800 2024-04-04T06:17:42.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.593+0000 7f76d7e00640 1 --2- 172.21.15.159:0/3907773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d8155550 0x7f76d80cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:42.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.593+0000 7f76dca00640 1 --2- 172.21.15.159:0/3907773 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f76d80d07c0 0x7f76d8099970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:42.589 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.593+0000 7f76dd400640 1 --2- 172.21.15.159:0/3907773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76d815f800 0x7f76d80ae330 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f76d000c700 tx=0x7f76d000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:42.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.594+0000 7f76d4c00640 1 -- 172.21.15.159:0/3907773 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d0018070 con 0x7f76d815f800 2024-04-04T06:17:42.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.594+0000 7f76d4c00640 1 -- 172.21.15.159:0/3907773 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76d000ce10 con 0x7f76d815f800 2024-04-04T06:17:42.590 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.594+0000 7f76df400640 1 -- 172.21.15.159:0/3907773 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d80cdda0 con 0x7f76d815f800 2024-04-04T06:17:42.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.594+0000 7f76d4c00640 1 -- 172.21.15.159:0/3907773 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d000a860 con 0x7f76d815f800 2024-04-04T06:17:42.591 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.594+0000 7f76df400640 1 -- 172.21.15.159:0/3907773 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d80ce640 con 0x7f76d815f800 2024-04-04T06:17:42.592 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.597+0000 7f76d4c00640 1 -- 172.21.15.159:0/3907773 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76d001b070 con 0x7f76d815f800 2024-04-04T06:17:42.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.598+0000 7f76e21b6740 1 Processor -- start 2024-04-04T06:17:42.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.598+0000 7f76e21b6740 1 -- start start 2024-04-04T06:17:42.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.598+0000 7f76d4c00640 1 --2- 172.21.15.159:0/3907773 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f76b806c8f0 0x7f76b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.593 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.598+0000 7f76d4c00640 1 -- 172.21.15.159:0/3907773 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f76d008b1b0 con 0x7f76d815f800 2024-04-04T06:17:42.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.599+0000 7f76e21b6740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eaad41f130 0x55eaad3f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.599+0000 7f76e21b6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f510 0x55eaad421ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.596 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.599+0000 7f76e21b6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3f2610 0x55eaad422400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.599+0000 7f76e21b6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55eaad3b9ab0 con 0x55eaad41f510 2024-04-04T06:17:42.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.599+0000 7f76e21b6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eaad35bfa0 con 0x55eaad3f2610 2024-04-04T06:17:42.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.599+0000 7f76e21b6740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55eaad304560 con 0x55eaad41f130 2024-04-04T06:17:42.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.599+0000 7f76d7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f510 0x55eaad421ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.599+0000 7f76d7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f510 0x55eaad421ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42092/0 (socket says 172.21.15.159:42092) 2024-04-04T06:17:42.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.599+0000 7f76d7e00640 1 -- 172.21.15.159:0/2299898279 learned_addr learned my addr 172.21.15.159:0/2299898279 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:42.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.599+0000 7f76d7e00640 1 --2- 172.21.15.159:0/3907773 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f76b806c8f0 0x7f76b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.599+0000 7f76dd400640 1 --2- 172.21.15.159:0/2299898279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3f2610 0x55eaad422400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.599+0000 7f76dca00640 1 --2- 172.21.15.159:0/2299898279 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eaad41f130 0x55eaad3f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.600+0000 7f76d7e00640 1 -- 172.21.15.159:0/2299898279 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eaad41f130 msgr2=0x55eaad3f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:42.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.600+0000 7f76d7e00640 1 --2- 172.21.15.159:0/2299898279 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eaad41f130 0x55eaad3f20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.600+0000 7f76d7e00640 1 -- 172.21.15.159:0/2299898279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3f2610 msgr2=0x55eaad422400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:42.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.600+0000 7f76d7e00640 1 --2- 172.21.15.159:0/2299898279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3f2610 0x55eaad422400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.600+0000 7f76d7e00640 1 -- 172.21.15.159:0/2299898279 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaad0702d0 con 0x55eaad41f510 2024-04-04T06:17:42.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.600+0000 7f76d7e00640 1 --2- 172.21.15.159:0/2299898279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f510 0x55eaad421ec0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f76c4015480 tx=0x7f76c4015940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574beaeb9529cab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:42.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.600+0000 7f76c9200640 1 -- 172.21.15.159:0/2299898279 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c400f040 con 0x55eaad41f510 2024-04-04T06:17:42.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.600+0000 7f76c9200640 1 -- 172.21.15.159:0/2299898279 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c4018850 con 0x55eaad41f510 2024-04-04T06:17:42.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.600+0000 7f76c9200640 1 -- 172.21.15.159:0/2299898279 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c4018c00 con 0x55eaad41f510 2024-04-04T06:17:42.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.601+0000 7f76e21b6740 1 -- 172.21.15.159:0/2299898279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f510 msgr2=0x55eaad421ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:42.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.601+0000 7f76e21b6740 1 --2- 172.21.15.159:0/2299898279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f510 0x55eaad421ec0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f76c4015480 tx=0x7f76c4015940 comp rx=0 tx=0).stop 2024-04-04T06:17:42.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.601+0000 7f76d7e00640 1 --2- 172.21.15.159:0/3907773 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f76b806c8f0 0x7f76b806edb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f76c401f000 tx=0x7f76c401d000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:42.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.601+0000 7f76e21b6740 1 -- 172.21.15.159:0/2299898279 shutdown_connections 2024-04-04T06:17:42.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.601+0000 7f76e21b6740 1 --2- 172.21.15.159:0/2299898279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad3f2610 0x55eaad422400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.601+0000 7f76e21b6740 1 --2- 172.21.15.159:0/2299898279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f510 0x55eaad421ec0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.601+0000 7f76e21b6740 1 --2- 172.21.15.159:0/2299898279 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eaad41f130 0x55eaad3f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.601+0000 7f76e21b6740 1 -- 172.21.15.159:0/2299898279 >> 172.21.15.159:0/2299898279 conn(0x55eaad3f4340 msgr2=0x55eaad43cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:42.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.601+0000 7f76e21b6740 1 -- 172.21.15.159:0/2299898279 shutdown_connections 2024-04-04T06:17:42.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.601+0000 7f76e21b6740 1 -- 172.21.15.159:0/2299898279 wait complete. 2024-04-04T06:17:42.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.602+0000 7f76e21b6740 1 -- start start 2024-04-04T06:17:42.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.602+0000 7f76e21b6740 1 Processor -- start 2024-04-04T06:17:42.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.603+0000 7f76e21b6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad41f130 0x55eaad424c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.603+0000 7f76e21b6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f510 0x55eaad425730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.603+0000 7f76e21b6740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eaad425c70 0x55eaad40cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.603+0000 7f76e21b6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55eaad317920 con 0x55eaad41f510 2024-04-04T06:17:42.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.603+0000 7f76e21b6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eaad357f50 con 0x55eaad41f130 2024-04-04T06:17:42.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.603+0000 7f76e21b6740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55eaad3f68e0 con 0x55eaad425c70 2024-04-04T06:17:42.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.603+0000 7f76e21b6740 10 client.0 _ll_register_callbacks cb 0x55eaad317310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:17:42.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.604+0000 7f76d7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f510 0x55eaad425730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.604+0000 7f76d7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f510 0x55eaad425730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42108/0 (socket says 172.21.15.159:42108) 2024-04-04T06:17:42.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.604+0000 7f76d7e00640 1 -- 172.21.15.159:0/3405834958 learned_addr learned my addr 172.21.15.159:0/3405834958 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:42.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.604+0000 7f76dd400640 1 --2- 172.21.15.159:0/3405834958 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eaad425c70 0x55eaad40cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.604+0000 7f76d7e00640 1 -- 172.21.15.159:0/3405834958 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eaad425c70 msgr2=0x55eaad40cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:42.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.604+0000 7f76dca00640 1 --2- 172.21.15.159:0/3405834958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad41f130 0x55eaad424c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.604+0000 7f76d7e00640 1 --2- 172.21.15.159:0/3405834958 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eaad425c70 0x55eaad40cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.604+0000 7f76d7e00640 1 -- 172.21.15.159:0/3405834958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad41f130 msgr2=0x55eaad424c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:42.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.604+0000 7f76d7e00640 1 --2- 172.21.15.159:0/3405834958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad41f130 0x55eaad424c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.604+0000 7f76d7e00640 1 -- 172.21.15.159:0/3405834958 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaad13a6b0 con 0x55eaad41f510 2024-04-04T06:17:42.603 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.604+0000 7f76d7e00640 1 --2- 172.21.15.159:0/3405834958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f510 0x55eaad425730 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f76c4018ed0 tx=0x7f76c4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680c10b1c0095ac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:42.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.604+0000 7f76bd600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:17:42.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.605+0000 7f76bd600640 1 -- 172.21.15.159:0/3405834958 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c400b1c0 con 0x55eaad41f510 2024-04-04T06:17:42.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.605+0000 7f76bd600640 1 -- 172.21.15.159:0/3405834958 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f76c4017d40 con 0x55eaad41f510 2024-04-04T06:17:42.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.605+0000 7f76e21b6740 1 -- 172.21.15.159:0/3405834958 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76c4018850 con 0x55eaad41f510 2024-04-04T06:17:42.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.605+0000 7f76bd600640 1 -- 172.21.15.159:0/3405834958 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7f76c4005990 con 0x55eaad41f510 2024-04-04T06:17:42.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.605+0000 7f76e21b6740 10 client.9411 fetch_fsmap learned FSMap version 114 2024-04-04T06:17:42.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.605+0000 7f76e21b6740 1 -- 172.21.15.159:0/3405834958 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x55eaad424490 con 0x55eaad41f510 2024-04-04T06:17:42.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.605+0000 7f76bd600640 1 -- 172.21.15.159:0/3405834958 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c4018850 con 0x55eaad41f510 2024-04-04T06:17:42.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.605+0000 7f76bd600640 1 -- 172.21.15.159:0/3405834958 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 114) ==== 3494+0+0 (secure 0 0 0) 0x7f76c403d020 con 0x55eaad41f510 2024-04-04T06:17:42.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.605+0000 7f76e21b6740 10 client.9411 fetch_fsmap finished waiting for FSMap version 114 2024-04-04T06:17:42.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.605+0000 7f76e21b6740 10 client.9411 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:17:42.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.605+0000 7f76e21b6740 20 client.9411 populate_metadata read hostname 'smithi159' 2024-04-04T06:17:42.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.605+0000 7f76e21b6740 1 --2- 172.21.15.159:0/3405834958 >> [v2:172.21.15.159:6832/643543929,v1:172.21.15.159:6834/643543929] conn(0x55eaad458b10 0x55eaad45af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.605+0000 7f76e21b6740 4 client.9411 mds_command: new command op to 9312 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-04T06:17:42.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.605+0000 7f76e21b6740 1 -- 172.21.15.159:0/3405834958 --> [v2:172.21.15.159:6832/643543929,v1:172.21.15.159:6834/643543929] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eaad3694e0 con 0x55eaad458b10 2024-04-04T06:17:42.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.606+0000 7f76dca00640 1 --2- 172.21.15.159:0/3405834958 >> [v2:172.21.15.159:6832/643543929,v1:172.21.15.159:6834/643543929] conn(0x55eaad458b10 0x55eaad45af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.606+0000 7f76dca00640 1 --2- 172.21.15.159:0/3405834958 >> [v2:172.21.15.159:6832/643543929,v1:172.21.15.159:6834/643543929] conn(0x55eaad458b10 0x55eaad45af00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d97a1ca459f7e49e server_cookie=327f66aeca5ff2af in_seq=0 out_seq=0 2024-04-04T06:17:42.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.606+0000 7f76bd600640 10 client.9411 ms_handle_connect on v2:172.21.15.159:6832/643543929 2024-04-04T06:17:42.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.608+0000 7f76bd600640 1 -- 172.21.15.159:0/3405834958 <== mds.0 v2:172.21.15.159:6832/643543929 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eaad3694e0 con 0x55eaad458b10 2024-04-04T06:17:42.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.608+0000 7f76bd600640 10 client.9411 handle_command_reply: tid=0 2024-04-04T06:17:42.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.608+0000 7f76e21b6740 1 client.9411 shutdown 2024-04-04T06:17:42.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.608+0000 7f76e21b6740 1 -- 172.21.15.159:0/3405834958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f510 msgr2=0x55eaad425730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:42.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.608+0000 7f76e21b6740 1 --2- 172.21.15.159:0/3405834958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f510 0x55eaad425730 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f76c4018ed0 tx=0x7f76c4008040 comp rx=0 tx=0).stop 2024-04-04T06:17:42.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.608+0000 7f76e21b6740 1 -- 172.21.15.159:0/3405834958 shutdown_connections 2024-04-04T06:17:42.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.608+0000 7f76e21b6740 1 --2- 172.21.15.159:0/3405834958 >> [v2:172.21.15.159:6832/643543929,v1:172.21.15.159:6834/643543929] conn(0x55eaad458b10 0x55eaad45af00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.608+0000 7f76e21b6740 1 --2- 172.21.15.159:0/3405834958 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eaad425c70 0x55eaad40cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.608+0000 7f76e21b6740 1 --2- 172.21.15.159:0/3405834958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f510 0x55eaad425730 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.608+0000 7f76e21b6740 1 --2- 172.21.15.159:0/3405834958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad41f130 0x55eaad424c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.608+0000 7f76e21b6740 1 -- 172.21.15.159:0/3405834958 >> 172.21.15.159:0/3405834958 conn(0x55eaad3f2610 msgr2=0x55eaad3f0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:42.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.608+0000 7f76bd600640 0 client.9411 ms_handle_reset on v2:172.21.15.159:6832/643543929 2024-04-04T06:17:42.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.608+0000 7f76e21b6740 1 -- 172.21.15.159:0/3405834958 shutdown_connections 2024-04-04T06:17:42.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.608+0000 7f76e21b6740 1 -- 172.21.15.159:0/3405834958 wait complete. 2024-04-04T06:17:42.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.608+0000 7f76e21b6740 20 client.9411 trim_cache size 0 max 16384 2024-04-04T06:17:42.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.628+0000 7f76e21b6740 1 Processor -- start 2024-04-04T06:17:42.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.628+0000 7f76e21b6740 1 -- start start 2024-04-04T06:17:42.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.629+0000 7f76e21b6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f130 0x55eaad433f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.629+0000 7f76e21b6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad41f510 0x55eaad422eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.629+0000 7f76e21b6740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eaad425c70 0x55eaad4233f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.629+0000 7f76e21b6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f76d81ffc30 con 0x55eaad41f130 2024-04-04T06:17:42.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.629+0000 7f76e21b6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f76d8167e70 con 0x55eaad41f510 2024-04-04T06:17:42.625 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.629+0000 7f76e21b6740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f76d81684c0 con 0x55eaad425c70 2024-04-04T06:17:42.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.629+0000 7f76dca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f130 0x55eaad433f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.629+0000 7f76dca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f130 0x55eaad433f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42110/0 (socket says 172.21.15.159:42110) 2024-04-04T06:17:42.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.629+0000 7f76dca00640 1 -- 172.21.15.159:0/3207181368 learned_addr learned my addr 172.21.15.159:0/3207181368 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:42.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.630+0000 7f76dca00640 1 -- 172.21.15.159:0/3207181368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eaad425c70 msgr2=0x55eaad4233f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:42.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.630+0000 7f76dca00640 1 --2- 172.21.15.159:0/3207181368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eaad425c70 0x55eaad4233f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.630+0000 7f76dca00640 1 -- 172.21.15.159:0/3207181368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad41f510 msgr2=0x55eaad422eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:42.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.630+0000 7f76dca00640 1 --2- 172.21.15.159:0/3207181368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad41f510 0x55eaad422eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.630+0000 7f76dca00640 1 -- 172.21.15.159:0/3207181368 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaad422930 con 0x55eaad41f130 2024-04-04T06:17:42.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.630+0000 7f76dca00640 1 --2- 172.21.15.159:0/3207181368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f130 0x55eaad433f10 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f76cc0090b0 tx=0x7f76cc008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c966171bf559612b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:42.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.630+0000 7f76bfe00640 1 -- 172.21.15.159:0/3207181368 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc035040 con 0x55eaad41f130 2024-04-04T06:17:42.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.630+0000 7f76bfe00640 1 -- 172.21.15.159:0/3207181368 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76cc04f070 con 0x55eaad41f130 2024-04-04T06:17:42.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.630+0000 7f76bfe00640 1 -- 172.21.15.159:0/3207181368 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc031970 con 0x55eaad41f130 2024-04-04T06:17:42.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.631+0000 7f76e21b6740 1 -- 172.21.15.159:0/3207181368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f130 msgr2=0x55eaad433f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:42.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.631+0000 7f76e21b6740 1 --2- 172.21.15.159:0/3207181368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f130 0x55eaad433f10 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f76cc0090b0 tx=0x7f76cc008030 comp rx=0 tx=0).stop 2024-04-04T06:17:42.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.631+0000 7f76e21b6740 1 -- 172.21.15.159:0/3207181368 shutdown_connections 2024-04-04T06:17:42.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.631+0000 7f76e21b6740 1 --2- 172.21.15.159:0/3207181368 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eaad425c70 0x55eaad4233f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.631+0000 7f76e21b6740 1 --2- 172.21.15.159:0/3207181368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad41f510 0x55eaad422eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.631+0000 7f76e21b6740 1 --2- 172.21.15.159:0/3207181368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f130 0x55eaad433f10 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.631+0000 7f76e21b6740 1 -- 172.21.15.159:0/3207181368 >> 172.21.15.159:0/3207181368 conn(0x55eaad3f2610 msgr2=0x55eaad434e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:42.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.631+0000 7f76e21b6740 1 -- 172.21.15.159:0/3207181368 shutdown_connections 2024-04-04T06:17:42.628 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.631+0000 7f76e21b6740 1 -- 172.21.15.159:0/3207181368 wait complete. 2024-04-04T06:17:42.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.632+0000 7f76e21b6740 1 -- start start 2024-04-04T06:17:42.629 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.632+0000 7f76e21b6740 1 Processor -- start 2024-04-04T06:17:42.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.634+0000 7f76e21b6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad41f130 0x55eaad40b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.634+0000 7f76e21b6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f510 0x55eaad4b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.634+0000 7f76e21b6740 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eaad425c70 0x55eaad4198e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.634+0000 7f76e21b6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55eaad40c7a0 con 0x55eaad41f510 2024-04-04T06:17:42.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.634+0000 7f76e21b6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55eaad40d700 con 0x55eaad41f130 2024-04-04T06:17:42.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.634+0000 7f76e21b6740 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x55eaad421340 con 0x55eaad425c70 2024-04-04T06:17:42.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.634+0000 7f76e21b6740 10 client.0 _ll_register_callbacks cb 0x7f76c4018d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-04T06:17:42.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.634+0000 7f76d7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f510 0x55eaad4b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.634+0000 7f76d7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f510 0x55eaad4b72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42112/0 (socket says 172.21.15.159:42112) 2024-04-04T06:17:42.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.634+0000 7f76d7e00640 1 -- 172.21.15.159:0/1371665375 learned_addr learned my addr 172.21.15.159:0/1371665375 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:42.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.635+0000 7f76dca00640 1 --2- 172.21.15.159:0/1371665375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad41f130 0x55eaad40b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.635+0000 7f76dd400640 1 --2- 172.21.15.159:0/1371665375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eaad425c70 0x55eaad4198e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.635+0000 7f76d7e00640 1 -- 172.21.15.159:0/1371665375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eaad425c70 msgr2=0x55eaad4198e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:42.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.635+0000 7f76d7e00640 1 --2- 172.21.15.159:0/1371665375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eaad425c70 0x55eaad4198e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.635+0000 7f76d7e00640 1 -- 172.21.15.159:0/1371665375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad41f130 msgr2=0x55eaad40b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:42.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.635+0000 7f76d7e00640 1 --2- 172.21.15.159:0/1371665375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad41f130 0x55eaad40b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.635+0000 7f76d7e00640 1 -- 172.21.15.159:0/1371665375 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaad422e90 con 0x55eaad41f510 2024-04-04T06:17:42.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.635+0000 7f76dca00640 1 --2- 172.21.15.159:0/1371665375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad41f130 0x55eaad40b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:17:42.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.635+0000 7f76d7e00640 1 --2- 172.21.15.159:0/1371665375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f510 0x55eaad4b72c0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f76c40153d0 tx=0x7f76c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18352f85b6b1a938 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:42.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.635+0000 7f76bd600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-04-04T06:17:42.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.636+0000 7f76bd600640 1 -- 172.21.15.159:0/1371665375 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c400b1c0 con 0x55eaad41f510 2024-04-04T06:17:42.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.636+0000 7f76e21b6740 1 -- 172.21.15.159:0/1371665375 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76cc04f070 con 0x55eaad41f510 2024-04-04T06:17:42.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.636+0000 7f76bd600640 1 -- 172.21.15.159:0/1371665375 <== mon.0 v2:172.21.15.159:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f76c4017d40 con 0x55eaad41f510 2024-04-04T06:17:42.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.636+0000 7f76bd600640 1 -- 172.21.15.159:0/1371665375 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7f76c4018850 con 0x55eaad41f510 2024-04-04T06:17:42.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.636+0000 7f76e21b6740 10 client.9417 fetch_fsmap learned FSMap version 114 2024-04-04T06:17:42.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.636+0000 7f76e21b6740 1 -- 172.21.15.159:0/1371665375 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x55eaad4351a0 con 0x55eaad41f510 2024-04-04T06:17:42.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.636+0000 7f76bd600640 1 -- 172.21.15.159:0/1371665375 <== mon.0 v2:172.21.15.159:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76cc04f070 con 0x55eaad41f510 2024-04-04T06:17:42.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.636+0000 7f76bd600640 1 -- 172.21.15.159:0/1371665375 <== mon.0 v2:172.21.15.159:3300/0 5 ==== fsmap(e 114) ==== 3494+0+0 (secure 0 0 0) 0x7f76c403d070 con 0x55eaad41f510 2024-04-04T06:17:42.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.636+0000 7f76e21b6740 10 client.9417 fetch_fsmap finished waiting for FSMap version 114 2024-04-04T06:17:42.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.636+0000 7f76e21b6740 10 client.9417 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-04T06:17:42.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.636+0000 7f76e21b6740 20 client.9417 populate_metadata read hostname 'smithi159' 2024-04-04T06:17:42.633 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.636+0000 7f76e21b6740 1 --2- 172.21.15.159:0/1371665375 >> [v2:172.21.15.159:6832/643543929,v1:172.21.15.159:6834/643543929] conn(0x55eaad41a030 0x55eaad45ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.636+0000 7f76e21b6740 4 client.9417 mds_command: new command op to 9312 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-04T06:17:42.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.636+0000 7f76e21b6740 1 -- 172.21.15.159:0/1371665375 --> [v2:172.21.15.159:6832/643543929,v1:172.21.15.159:6834/643543929] -- command(tid 0: {"prefix": "status"}) -- 0x55eaad363830 con 0x55eaad41a030 2024-04-04T06:17:42.634 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.637+0000 7f76dca00640 1 --2- 172.21.15.159:0/1371665375 >> [v2:172.21.15.159:6832/643543929,v1:172.21.15.159:6834/643543929] conn(0x55eaad41a030 0x55eaad45ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.637+0000 7f76dca00640 1 --2- 172.21.15.159:0/1371665375 >> [v2:172.21.15.159:6832/643543929,v1:172.21.15.159:6834/643543929] conn(0x55eaad41a030 0x55eaad45ef20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cfdcbdcc0af6446 server_cookie=34f44e8852cf4693 in_seq=0 out_seq=0 2024-04-04T06:17:42.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.637+0000 7f76bd600640 10 client.9417 ms_handle_connect on v2:172.21.15.159:6832/643543929 2024-04-04T06:17:42.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.637+0000 7f76bd600640 1 -- 172.21.15.159:0/1371665375 <== mds.0 v2:172.21.15.159:6832/643543929 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55eaad363830 con 0x55eaad41a030 2024-04-04T06:17:42.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.637+0000 7f76bd600640 10 client.9417 handle_command_reply: tid=0 2024-04-04T06:17:42.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.637+0000 7f76e21b6740 1 client.9417 shutdown 2024-04-04T06:17:42.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.638+0000 7f76e21b6740 1 -- 172.21.15.159:0/1371665375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f510 msgr2=0x55eaad4b72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:42.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.638+0000 7f76e21b6740 1 --2- 172.21.15.159:0/1371665375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f510 0x55eaad4b72c0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f76c40153d0 tx=0x7f76c4004730 comp rx=0 tx=0).stop 2024-04-04T06:17:42.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.638+0000 7f76e21b6740 1 -- 172.21.15.159:0/1371665375 shutdown_connections 2024-04-04T06:17:42.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.638+0000 7f76e21b6740 1 --2- 172.21.15.159:0/1371665375 >> [v2:172.21.15.159:6832/643543929,v1:172.21.15.159:6834/643543929] conn(0x55eaad41a030 0x55eaad45ef20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.638+0000 7f76e21b6740 1 --2- 172.21.15.159:0/1371665375 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55eaad425c70 0x55eaad4198e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.638+0000 7f76e21b6740 1 --2- 172.21.15.159:0/1371665375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eaad41f510 0x55eaad4b72c0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.638+0000 7f76e21b6740 1 --2- 172.21.15.159:0/1371665375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55eaad41f130 0x55eaad40b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.638+0000 7f76bd600640 0 client.9417 ms_handle_reset on v2:172.21.15.159:6832/643543929 2024-04-04T06:17:42.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.638+0000 7f76e21b6740 1 -- 172.21.15.159:0/1371665375 >> 172.21.15.159:0/1371665375 conn(0x55eaad3f2610 msgr2=0x55eaad433f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:42.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.638+0000 7f76e21b6740 1 -- 172.21.15.159:0/1371665375 shutdown_connections 2024-04-04T06:17:42.636 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.638+0000 7f76e21b6740 1 -- 172.21.15.159:0/1371665375 wait complete. 2024-04-04T06:17:42.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.638+0000 7f76e21b6740 20 client.9417 trim_cache size 0 max 16384 2024-04-04T06:17:42.637 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:17:42.637 INFO:teuthology.orchestra.run.smithi159.stdout: "cluster_fsid": "30311045-f714-48fa-832b-4cb2ba8bf0bf", 2024-04-04T06:17:42.638 INFO:teuthology.orchestra.run.smithi159.stdout: "whoami": 0, 2024-04-04T06:17:42.638 INFO:teuthology.orchestra.run.smithi159.stdout: "id": 9312, 2024-04-04T06:17:42.638 INFO:teuthology.orchestra.run.smithi159.stdout: "want_state": "up:active", 2024-04-04T06:17:42.638 INFO:teuthology.orchestra.run.smithi159.stdout: "state": "up:active", 2024-04-04T06:17:42.638 INFO:teuthology.orchestra.run.smithi159.stdout: "fs_name": "cephfs", 2024-04-04T06:17:42.638 INFO:teuthology.orchestra.run.smithi159.stdout: "rank_uptime": 4.9520608060000004, 2024-04-04T06:17:42.638 INFO:teuthology.orchestra.run.smithi159.stdout: "mdsmap_epoch": 114, 2024-04-04T06:17:42.638 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch": 116, 2024-04-04T06:17:42.638 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_epoch_barrier": 116, 2024-04-04T06:17:42.638 INFO:teuthology.orchestra.run.smithi159.stdout: "uptime": 6.8853565110000003 2024-04-04T06:17:42.638 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:17:42.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.639+0000 7f76df400640 1 -- 172.21.15.159:0/3907773 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f76b806c8f0 msgr2=0x7f76b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:42.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.639+0000 7f76df400640 1 --2- 172.21.15.159:0/3907773 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f76b806c8f0 0x7f76b806edb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f76c401f000 tx=0x7f76c401d000 comp rx=0 tx=0).stop 2024-04-04T06:17:42.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.639+0000 7f76df400640 1 -- 172.21.15.159:0/3907773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76d815f800 msgr2=0x7f76d80ae330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:42.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.639+0000 7f76df400640 1 --2- 172.21.15.159:0/3907773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76d815f800 0x7f76d80ae330 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f76d000c700 tx=0x7f76d000cbc0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.639+0000 7f76df400640 1 -- 172.21.15.159:0/3907773 shutdown_connections 2024-04-04T06:17:42.639 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.639+0000 7f76df400640 1 --2- 172.21.15.159:0/3907773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76d815f800 0x7f76d80ae330 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.639+0000 7f76df400640 1 --2- 172.21.15.159:0/3907773 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f76b806c8f0 0x7f76b806edb0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.639+0000 7f76df400640 1 --2- 172.21.15.159:0/3907773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76d8155550 0x7f76d80cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.639+0000 7f76df400640 1 --2- 172.21.15.159:0/3907773 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f76d80d07c0 0x7f76d8099970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.639+0000 7f76df400640 1 -- 172.21.15.159:0/3907773 >> 172.21.15.159:0/3907773 conn(0x7f76d806b750 msgr2=0x7f76d815ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:42.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.639+0000 7f76df400640 1 -- 172.21.15.159:0/3907773 shutdown_connections 2024-04-04T06:17:42.640 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.639+0000 7f76df400640 1 -- 172.21.15.159:0/3907773 wait complete. 2024-04-04T06:17:42.651 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-04T06:17:42.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.850+0000 7f3dfda00640 1 Processor -- start 2024-04-04T06:17:42.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.850+0000 7f3dfda00640 1 -- start start 2024-04-04T06:17:42.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.851+0000 7f3dfda00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3df80cf050 0x7f3df815f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.846 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.851+0000 7f3dfda00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3df80d2110 0x7f3df80d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.851+0000 7f3dfda00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3df80cf6c0 con 0x7f3df80d0e70 2024-04-04T06:17:42.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.851+0000 7f3dfda00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3df815fc20 con 0x7f3df80cf050 2024-04-04T06:17:42.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.851+0000 7f3dfda00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3df80cf9a0 con 0x7f3df80d2110 2024-04-04T06:17:42.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.852+0000 7f3df6a00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f3df80d0e70 0x7f3df80d1250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:50078/0 (socket says 172.21.15.159:50078) 2024-04-04T06:17:42.847 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.852+0000 7f3df6a00640 1 -- 172.21.15.159:0/2147837923 learned_addr learned my addr 172.21.15.159:0/2147837923 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:42.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.852+0000 7f3df6000640 1 -- 172.21.15.159:0/2147837923 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1822492100 0 0) 0x7f3df80cf6c0 con 0x7f3df80d0e70 2024-04-04T06:17:42.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.852+0000 7f3df7400640 1 --2- 172.21.15.159:0/2147837923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3df80d2110 0x7f3df80d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.852+0000 7f3defe00640 1 --2- 172.21.15.159:0/2147837923 >> v2:172.21.15.178:3300/0 conn(0x7f3df80cf050 0x7f3df815f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.852+0000 7f3df6000640 1 -- 172.21.15.159:0/2147837923 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ddc003520 con 0x7f3df80d0e70 2024-04-04T06:17:42.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.853+0000 7f3df6000640 1 -- 172.21.15.159:0/2147837923 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2923714783 0 0) 0x7f3ddc003520 con 0x7f3df80d0e70 2024-04-04T06:17:42.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.853+0000 7f3df6000640 1 -- 172.21.15.159:0/2147837923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3df80d2110 msgr2=0x7f3df80d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:42.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.853+0000 7f3df6000640 1 --2- 172.21.15.159:0/2147837923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3df80d2110 0x7f3df80d25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.853+0000 7f3df6000640 1 -- 172.21.15.159:0/2147837923 >> v2:172.21.15.178:3300/0 conn(0x7f3df80cf050 msgr2=0x7f3df815f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:42.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.853+0000 7f3df6000640 1 --2- 172.21.15.159:0/2147837923 >> v2:172.21.15.178:3300/0 conn(0x7f3df80cf050 0x7f3df815f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.853+0000 7f3df6000640 1 -- 172.21.15.159:0/2147837923 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df80cfc80 con 0x7f3df80d0e70 2024-04-04T06:17:42.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.853+0000 7f3df6000640 1 -- 172.21.15.159:0/2147837923 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f3de8001670 con 0x7f3df80d0e70 2024-04-04T06:17:42.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.853+0000 7f3df6000640 1 -- 172.21.15.159:0/2147837923 >> v1:172.21.15.159:6789/0 conn(0x7f3df80d0e70 legacy=0x7f3df80d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:42.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.853+0000 7f3df6000640 1 --2- 172.21.15.159:0/2147837923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ddc0039b0 0x7f3ddc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.853+0000 7f3df6000640 1 --2- 172.21.15.159:0/2147837923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3df80d2110 0x7f3ddc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.853+0000 7f3df6000640 1 --2- 172.21.15.159:0/2147837923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ddc004d40 0x7f3ddc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.853+0000 7f3df6000640 1 -- 172.21.15.159:0/2147837923 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3df815fc20 con 0x7f3ddc004d40 2024-04-04T06:17:42.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.853+0000 7f3df6000640 1 -- 172.21.15.159:0/2147837923 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3df80cf9a0 con 0x7f3ddc0039b0 2024-04-04T06:17:42.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.853+0000 7f3df6000640 1 -- 172.21.15.159:0/2147837923 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3ddc00b9c0 con 0x7f3df80d2110 2024-04-04T06:17:42.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.853+0000 7f3df7400640 1 --2- 172.21.15.159:0/2147837923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ddc004d40 0x7f3ddc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.854+0000 7f3df6a00640 1 --2- 172.21.15.159:0/2147837923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ddc0039b0 0x7f3ddc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.854+0000 7f3defe00640 1 --2- 172.21.15.159:0/2147837923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3df80d2110 0x7f3ddc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.854+0000 7f3df7400640 1 -- 172.21.15.159:0/2147837923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3df80d2110 msgr2=0x7f3ddc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:42.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.854+0000 7f3df7400640 1 --2- 172.21.15.159:0/2147837923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3df80d2110 0x7f3ddc004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.854+0000 7f3df7400640 1 -- 172.21.15.159:0/2147837923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ddc0039b0 msgr2=0x7f3ddc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:42.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.854+0000 7f3df7400640 1 --2- 172.21.15.159:0/2147837923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ddc0039b0 0x7f3ddc003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.854+0000 7f3df7400640 1 -- 172.21.15.159:0/2147837923 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ddc003520 con 0x7f3ddc004d40 2024-04-04T06:17:42.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.855+0000 7f3df7400640 1 --2- 172.21.15.159:0/2147837923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ddc004d40 0x7f3ddc0051c0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f3df000e170 tx=0x7f3ddc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7840a097d5182dc server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:42.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.855+0000 7f3df6000640 1 -- 172.21.15.159:0/2147837923 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0039070 con 0x7f3ddc004d40 2024-04-04T06:17:42.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.855+0000 7f3df6000640 1 -- 172.21.15.159:0/2147837923 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df002cbf0 con 0x7f3ddc004d40 2024-04-04T06:17:42.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.855+0000 7f3df6000640 1 -- 172.21.15.159:0/2147837923 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0009c10 con 0x7f3ddc004d40 2024-04-04T06:17:42.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.856+0000 7f3dfda00640 1 -- 172.21.15.159:0/2147837923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ddc004d40 msgr2=0x7f3ddc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:42.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.856+0000 7f3dfda00640 1 --2- 172.21.15.159:0/2147837923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ddc004d40 0x7f3ddc0051c0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f3df000e170 tx=0x7f3ddc005830 comp rx=0 tx=0).stop 2024-04-04T06:17:42.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.856+0000 7f3df7400640 1 -- 172.21.15.159:0/2147837923 reap_dead start 2024-04-04T06:17:42.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.856+0000 7f3dfda00640 1 -- 172.21.15.159:0/2147837923 shutdown_connections 2024-04-04T06:17:42.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.856+0000 7f3dfda00640 1 -- 172.21.15.159:0/2147837923 >> 172.21.15.159:0/2147837923 conn(0x7f3df808bfb0 msgr2=0x7f3df80d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:42.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.856+0000 7f3dfda00640 1 -- 172.21.15.159:0/2147837923 shutdown_connections 2024-04-04T06:17:42.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.856+0000 7f3dfda00640 1 -- 172.21.15.159:0/2147837923 wait complete. 2024-04-04T06:17:42.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.858+0000 7f3dfda00640 1 Processor -- start 2024-04-04T06:17:42.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.858+0000 7f3dfda00640 1 -- start start 2024-04-04T06:17:42.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.859+0000 7f3dfda00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ddc004d40 0x7f3df8155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.859+0000 7f3dfda00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3df80cdfa0 0x7f3df80cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.859+0000 7f3dfda00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3df815d5d0 0x7f3df815da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.859+0000 7f3dfda00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3df815e910 con 0x7f3ddc004d40 2024-04-04T06:17:42.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.859+0000 7f3dfda00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3df80d0240 con 0x7f3df80cdfa0 2024-04-04T06:17:42.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.859+0000 7f3dfda00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3df815ea90 con 0x7f3df815d5d0 2024-04-04T06:17:42.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.859+0000 7f3df6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ddc004d40 0x7f3df8155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.859+0000 7f3df6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ddc004d40 0x7f3df8155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42124/0 (socket says 172.21.15.159:42124) 2024-04-04T06:17:42.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.859+0000 7f3df6a00640 1 -- 172.21.15.159:0/2690048533 learned_addr learned my addr 172.21.15.159:0/2690048533 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:42.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.859+0000 7f3defe00640 1 --2- 172.21.15.159:0/2690048533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3df80cdfa0 0x7f3df80cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.859+0000 7f3df6a00640 1 -- 172.21.15.159:0/2690048533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3df815d5d0 msgr2=0x7f3df815da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:42.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.859+0000 7f3df6a00640 1 --2- 172.21.15.159:0/2690048533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3df815d5d0 0x7f3df815da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.859+0000 7f3df6a00640 1 -- 172.21.15.159:0/2690048533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3df80cdfa0 msgr2=0x7f3df80cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:42.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.859+0000 7f3df6a00640 1 --2- 172.21.15.159:0/2690048533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3df80cdfa0 0x7f3df80cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:42.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.859+0000 7f3df6a00640 1 -- 172.21.15.159:0/2690048533 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df80ccf00 con 0x7f3ddc004d40 2024-04-04T06:17:42.855 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.860+0000 7f3df6a00640 1 --2- 172.21.15.159:0/2690048533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ddc004d40 0x7f3df8155ff0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f3de800d890 tx=0x7f3de800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:42.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.860+0000 7f3deea00640 1 -- 172.21.15.159:0/2690048533 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de8019070 con 0x7f3ddc004d40 2024-04-04T06:17:42.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.860+0000 7f3deea00640 1 -- 172.21.15.159:0/2690048533 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3de80074e0 con 0x7f3ddc004d40 2024-04-04T06:17:42.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.860+0000 7f3deea00640 1 -- 172.21.15.159:0/2690048533 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de8004a00 con 0x7f3ddc004d40 2024-04-04T06:17:42.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.860+0000 7f3dfda00640 1 -- 172.21.15.159:0/2690048533 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df80cd1e0 con 0x7f3ddc004d40 2024-04-04T06:17:42.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.860+0000 7f3dfda00640 1 -- 172.21.15.159:0/2690048533 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df80cd7a0 con 0x7f3ddc004d40 2024-04-04T06:17:42.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.861+0000 7f3dfda00640 1 -- 172.21.15.159:0/2690048533 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3df8068cd0 con 0x7f3ddc004d40 2024-04-04T06:17:42.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.862+0000 7f3deea00640 1 -- 172.21.15.159:0/2690048533 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3de8004ba0 con 0x7f3ddc004d40 2024-04-04T06:17:42.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.863+0000 7f3deea00640 1 --2- 172.21.15.159:0/2690048533 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3dd406c920 0x7f3dd406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:42.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.864+0000 7f3deea00640 1 -- 172.21.15.159:0/2690048533 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f3de808b5c0 con 0x7f3ddc004d40 2024-04-04T06:17:42.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.864+0000 7f3defe00640 1 --2- 172.21.15.159:0/2690048533 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3dd406c920 0x7f3dd406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:42.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.865+0000 7f3defe00640 1 --2- 172.21.15.159:0/2690048533 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3dd406c920 0x7f3dd406ede0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3df806d990 tx=0x7f3de0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:42.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:42.869+0000 7f3deea00640 1 -- 172.21.15.159:0/2690048533 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3de8059990 con 0x7f3ddc004d40 2024-04-04T06:17:43.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:43.104+0000 7f3dfda00640 1 -- 172.21.15.159:0/2690048533 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3df80d1b80 con 0x7f3ddc004d40 2024-04-04T06:17:43.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:43.105+0000 7f3deea00640 1 -- 172.21.15.159:0/2690048533 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v114) ==== 53+0+73 (secure 0 0 0) 0x7f3de8089e20 con 0x7f3ddc004d40 2024-04-04T06:17:43.100 INFO:teuthology.orchestra.run.smithi159.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-04T06:17:43.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:43.108+0000 7f3dfda00640 1 -- 172.21.15.159:0/2690048533 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3dd406c920 msgr2=0x7f3dd406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:43.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:43.108+0000 7f3dfda00640 1 --2- 172.21.15.159:0/2690048533 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3dd406c920 0x7f3dd406ede0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3df806d990 tx=0x7f3de0008040 comp rx=0 tx=0).stop 2024-04-04T06:17:43.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:43.108+0000 7f3dfda00640 1 -- 172.21.15.159:0/2690048533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ddc004d40 msgr2=0x7f3df8155ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:43.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:43.108+0000 7f3dfda00640 1 --2- 172.21.15.159:0/2690048533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ddc004d40 0x7f3df8155ff0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f3de800d890 tx=0x7f3de800dd50 comp rx=0 tx=0).stop 2024-04-04T06:17:43.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:43.108+0000 7f3dfda00640 1 -- 172.21.15.159:0/2690048533 shutdown_connections 2024-04-04T06:17:43.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:43.108+0000 7f3dfda00640 1 --2- 172.21.15.159:0/2690048533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3df815d5d0 0x7f3df815da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:43.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:43.108+0000 7f3dfda00640 1 --2- 172.21.15.159:0/2690048533 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3dd406c920 0x7f3dd406ede0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:43.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:43.108+0000 7f3dfda00640 1 --2- 172.21.15.159:0/2690048533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3df80cdfa0 0x7f3df80cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:43.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:43.108+0000 7f3dfda00640 1 --2- 172.21.15.159:0/2690048533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ddc004d40 0x7f3df8155ff0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:43.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:43.108+0000 7f3dfda00640 1 -- 172.21.15.159:0/2690048533 >> 172.21.15.159:0/2690048533 conn(0x7f3df808bfb0 msgr2=0x7f3df806de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:43.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:43.108+0000 7f3dfda00640 1 -- 172.21.15.159:0/2690048533 shutdown_connections 2024-04-04T06:17:43.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:43.108+0000 7f3dfda00640 1 -- 172.21.15.159:0/2690048533 wait complete. 2024-04-04T06:17:43.120 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-04T06:17:43.120 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-04T06:17:43.120 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi159.front.sepia.ceph.com 2024-04-04T06:17:43.121 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-04T06:17:43.121 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-04T06:17:43.121 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-04T06:17:43.121 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-04T06:17:43.121 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-04T06:17:43.121 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:17:43.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:43.121 DEBUG:teuthology.orchestra.run.smithi159:> ip addr 2024-04-04T06:17:43.167 INFO:teuthology.orchestra.run.smithi159.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-04T06:17:43.168 INFO:teuthology.orchestra.run.smithi159.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-04T06:17:43.168 INFO:teuthology.orchestra.run.smithi159.stdout: inet 127.0.0.1/8 scope host lo 2024-04-04T06:17:43.168 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:43.168 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 ::1/128 scope host 2024-04-04T06:17:43.168 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:43.168 INFO:teuthology.orchestra.run.smithi159.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:17:43.168 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c6 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:17:43.168 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f0 2024-04-04T06:17:43.168 INFO:teuthology.orchestra.run.smithi159.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:17:43.168 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:d9:d8:c7 brd ff:ff:ff:ff:ff:ff 2024-04-04T06:17:43.168 INFO:teuthology.orchestra.run.smithi159.stdout: altname enp7s0f1 2024-04-04T06:17:43.169 INFO:teuthology.orchestra.run.smithi159.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-04T06:17:43.169 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4e brd ff:ff:ff:ff:ff:ff 2024-04-04T06:17:43.169 INFO:teuthology.orchestra.run.smithi159.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-04T06:17:43.169 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0c:c4:7a:88:72:4f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:17:43.169 INFO:teuthology.orchestra.run.smithi159.stdout: inet 172.21.15.159/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-04T06:17:43.169 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft 41870sec preferred_lft 41870sec 2024-04-04T06:17:43.169 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::ec4:7aff:fe88:724f/64 scope link 2024-04-04T06:17:43.169 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:43.169 INFO:teuthology.orchestra.run.smithi159.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-04T06:17:43.169 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff 2024-04-04T06:17:43.169 INFO:teuthology.orchestra.run.smithi159.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-04T06:17:43.169 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:43.169 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::989e:e4ff:fea5:7568/64 scope link 2024-04-04T06:17:43.169 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:43.169 INFO:teuthology.orchestra.run.smithi159.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:17:43.170 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether a2:e3:e0:cb:0a:73 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-04T06:17:43.170 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::a0e3:e0ff:fecb:a73/64 scope link 2024-04-04T06:17:43.170 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:43.170 INFO:teuthology.orchestra.run.smithi159.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-04T06:17:43.170 INFO:teuthology.orchestra.run.smithi159.stdout: link/ether 0e:0a:8a:54:b5:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-04T06:17:43.170 INFO:teuthology.orchestra.run.smithi159.stdout: inet6 fe80::c0a:8aff:fe54:b58f/64 scope link 2024-04-04T06:17:43.170 INFO:teuthology.orchestra.run.smithi159.stdout: valid_lft forever preferred_lft forever 2024-04-04T06:17:43.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:43.170 DEBUG:teuthology.orchestra.run.smithi159:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-04T06:17:43.286 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:17:43.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:43.287 DEBUG:teuthology.orchestra.run.smithi159:> route 2024-04-04T06:17:43.328 INFO:teuthology.orchestra.run.smithi159.stdout:Kernel IP routing table 2024-04-04T06:17:43.328 INFO:teuthology.orchestra.run.smithi159.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-04T06:17:43.328 INFO:teuthology.orchestra.run.smithi159.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-04T06:17:43.328 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-04T06:17:43.328 INFO:teuthology.orchestra.run.smithi159.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-04T06:17:43.328 DEBUG:teuthology.orchestra.run.smithi159:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-04T06:17:43.328 DEBUG:teuthology.orchestra.run.smithi159:> set -e 2024-04-04T06:17:43.329 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-04T06:17:43.329 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-04T06:17:43.329 DEBUG:teuthology.orchestra.run.smithi159:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-04T06:17:43.329 DEBUG:teuthology.orchestra.run.smithi159:> ') 2024-04-04T06:17:43.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-04T06:17:43.597 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:43.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:43.607 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-04-04T06:17:43.654 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-04T06:17:43.654 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-04T06:17:43.655 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:17:43.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:43.655 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link set brx.0 up 2024-04-04T06:17:43.793 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:17:43.793 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-04T06:17:43.844 INFO:teuthology.orchestra.run.smithi159.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-04T06:17:43.844 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-04T06:17:43.844 DEBUG:teuthology.orchestra.run.smithi159:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:17:43.890 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-04T06:17:43.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:43.890 DEBUG:teuthology.orchestra.run.smithi159:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:17:44.063 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:17:44.063 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:17:44.063 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:17:44.083 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:17:44.084 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:17:44.084 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:17:44.084 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:17:44.084 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:17:44.106 INFO:teuthology.orchestra.run.smithi159.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-04T06:17:44.106 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-04T06:17:44.106 INFO:teuthology.orchestra.run.smithi159.stdout:invalid new device string format 2024-04-04T06:17:44.106 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:3300,172.21.15.178:3300,172.21.15.178:3301" 2024-04-04T06:17:44.106 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-04T06:17:44.106 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: resolved to: "172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790" 2024-04-04T06:17:44.106 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.159:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2024-04-04T06:17:44.107 INFO:teuthology.orchestra.run.smithi159.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=30311045-f714-48fa-832b-4cb2ba8bf0bf" will pass to kernel 2024-04-04T06:17:44.108 INFO:teuthology.orchestra.run.smithi159.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-04T06:17:44.108 INFO:teuthology.orchestra.run.smithi159.stdout: You just mounted a file system that supports labels which does not 2024-04-04T06:17:44.108 INFO:teuthology.orchestra.run.smithi159.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-04T06:17:44.108 INFO:teuthology.orchestra.run.smithi159.stdout: applications will generate AVC messages and not be allowed access to 2024-04-04T06:17:44.108 INFO:teuthology.orchestra.run.smithi159.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-04T06:17:44.111 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-04T06:17:44.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:44.111 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-04T06:17:44.187 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-04T06:17:44.286 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-04T06:17:44.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:44.299 INFO:teuthology.orchestra.run.smithi159.stdout:client9426 2024-04-04T06:17:44.299 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:17:44.299 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:17:44.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:44.341 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-04T06:17:44.437 INFO:teuthology.orchestra.run.smithi159.stdout:./ 2024-04-04T06:17:44.438 INFO:teuthology.orchestra.run.smithi159.stdout:./parent 2024-04-04T06:17:44.438 INFO:teuthology.orchestra.run.smithi159.stdout:./parent/flushed 2024-04-04T06:17:44.438 INFO:teuthology.orchestra.run.smithi159.stdout:./parent/flushed/charlie 2024-04-04T06:17:44.438 INFO:teuthology.orchestra.run.smithi159.stdout:./parent/flushed/alpha 2024-04-04T06:17:44.438 INFO:teuthology.orchestra.run.smithi159.stdout:./parent/flushed/bravo 2024-04-04T06:17:44.438 INFO:teuthology.orchestra.run.smithi159.stdout:./parent/unflushed 2024-04-04T06:17:44.438 INFO:teuthology.orchestra.run.smithi159.stdout:./parent/unflushed/jfile 2024-04-04T06:17:44.439 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:44.439 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:44.439 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:44.439 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:44.439 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-04T06:17:44.439 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:44.527 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:17:44.531 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:17:44.557 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-04-04T06:17:44.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:44.574 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:44.574 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:44.574 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:44.574 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:44.574 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-04T06:17:44.574 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:44.736 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627776 2024-04-04T06:17:44.741 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:44.754 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:44.754 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:44.754 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:44.754 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:44.754 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-04T06:17:44.754 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:44.919 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627777 2024-04-04T06:17:44.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:44.932 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:44.932 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:44.933 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:44.933 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:44.933 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-04T06:17:44.933 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:45.109 INFO:teuthology.orchestra.run.smithi159.stdout:1099511628280 2024-04-04T06:17:45.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:45.123 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:45.123 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:45.123 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:45.123 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:45.123 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-04T06:17:45.124 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:45.289 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627778 2024-04-04T06:17:45.292 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:45.301 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:45.301 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:45.301 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:45.301 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:45.301 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-04T06:17:45.301 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:45.504 INFO:teuthology.orchestra.run.smithi159.stdout:1099511628279 2024-04-04T06:17:45.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:45.522 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:45.522 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:45.522 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:45.522 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:45.523 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-04T06:17:45.523 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:45.704 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627779 2024-04-04T06:17:45.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:45.717 DEBUG:teuthology.orchestra.run.smithi159:> stdin-killer -- python3 -c ' 2024-04-04T06:17:45.717 DEBUG:teuthology.orchestra.run.smithi159:> import os 2024-04-04T06:17:45.717 DEBUG:teuthology.orchestra.run.smithi159:> import stat 2024-04-04T06:17:45.717 DEBUG:teuthology.orchestra.run.smithi159:> 2024-04-04T06:17:45.717 DEBUG:teuthology.orchestra.run.smithi159:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-04T06:17:45.717 DEBUG:teuthology.orchestra.run.smithi159:> ' 2024-04-04T06:17:45.885 INFO:teuthology.orchestra.run.smithi159.stdout:1099511627780 2024-04-04T06:17:45.888 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-04T06:17:45.897 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-04T06:17:45.897 DEBUG:teuthology.orchestra.run.smithi159:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:17:46.049 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-04T06:17:46.156 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-04T06:17:46.156 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:17:46.156 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:17:46.156 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:17:46.209 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:17:46.209 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:17:46.254 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:17:46.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.517+0000 7feb73c00640 1 Processor -- start 2024-04-04T06:17:46.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.517+0000 7feb73c00640 1 -- start start 2024-04-04T06:17:46.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.518+0000 7feb73c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7feb6c15b310 0x7feb6c0727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:46.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.518+0000 7feb73c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb6c0d0780 0x7feb6c0d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:46.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.518+0000 7feb73c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6c15b6f0 con 0x7feb6c0b4f90 2024-04-04T06:17:46.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.519+0000 7feb73c00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7feb6c072e80 con 0x7feb6c15b310 2024-04-04T06:17:46.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.519+0000 7feb73c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feb6c073160 con 0x7feb6c0d0780 2024-04-04T06:17:46.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.519+0000 7feb71200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7feb6c15b310 0x7feb6c0727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:46.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.519+0000 7feb6be00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7feb6c0b4f90 0x7feb6c15f7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:42898/0 (socket says 172.21.15.159:42898) 2024-04-04T06:17:46.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.519+0000 7feb6be00640 1 -- 172.21.15.159:0/4016961484 learned_addr learned my addr 172.21.15.159:0/4016961484 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:46.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.519+0000 7feb71200640 1 -- 172.21.15.159:0/4016961484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb6c0d0780 msgr2=0x7feb6c0d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:46.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.519+0000 7feb71200640 1 --2- 172.21.15.159:0/4016961484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb6c0d0780 0x7feb6c0d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:46.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.519+0000 7feb71200640 1 -- 172.21.15.159:0/4016961484 >> v1:172.21.15.159:6789/0 conn(0x7feb6c0b4f90 legacy=0x7feb6c15f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:46.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.519+0000 7feb71200640 1 -- 172.21.15.159:0/4016961484 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c15efe0 con 0x7feb6c15b310 2024-04-04T06:17:46.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.519+0000 7feb6b400640 1 -- 172.21.15.159:0/4016961484 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 390796595 0 0) 0x7feb6c15b6f0 con 0x7feb6c0b4f90 2024-04-04T06:17:46.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.522+0000 7feb71200640 1 --2- 172.21.15.159:0/4016961484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb6c15b310 0x7feb6c0727f0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7feb5c009d90 tx=0x7feb5c02bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96aa47f2b05e5a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:46.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.523+0000 7feb6b400640 1 -- 172.21.15.159:0/4016961484 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c039070 con 0x7feb6c15b310 2024-04-04T06:17:46.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.523+0000 7feb6b400640 1 -- 172.21.15.159:0/4016961484 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb5c02cb30 con 0x7feb6c15b310 2024-04-04T06:17:46.518 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.523+0000 7feb6b400640 1 -- 172.21.15.159:0/4016961484 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c034440 con 0x7feb6c15b310 2024-04-04T06:17:46.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.523+0000 7feb73c00640 1 -- 172.21.15.159:0/4016961484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb6c15b310 msgr2=0x7feb6c0727f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:46.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.523+0000 7feb73c00640 1 --2- 172.21.15.159:0/4016961484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb6c15b310 0x7feb6c0727f0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7feb5c009d90 tx=0x7feb5c02bd80 comp rx=0 tx=0).stop 2024-04-04T06:17:46.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.524+0000 7feb73c00640 1 -- 172.21.15.159:0/4016961484 shutdown_connections 2024-04-04T06:17:46.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.524+0000 7feb73c00640 1 --2- 172.21.15.159:0/4016961484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb6c0d0780 0x7feb6c0d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:46.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.524+0000 7feb73c00640 1 --2- 172.21.15.159:0/4016961484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb6c15b310 0x7feb6c0727f0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:46.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.524+0000 7feb73c00640 1 -- 172.21.15.159:0/4016961484 >> 172.21.15.159:0/4016961484 conn(0x7feb6c06b990 msgr2=0x7feb6c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:46.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.524+0000 7feb73c00640 1 -- 172.21.15.159:0/4016961484 shutdown_connections 2024-04-04T06:17:46.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.524+0000 7feb73c00640 1 -- 172.21.15.159:0/4016961484 wait complete. 2024-04-04T06:17:46.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.525+0000 7feb73c00640 1 Processor -- start 2024-04-04T06:17:46.520 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.526+0000 7feb73c00640 1 -- start start 2024-04-04T06:17:46.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.526+0000 7feb73c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb6c0b4f90 0x7feb6c151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:46.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.526+0000 7feb73c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb6c0d0780 0x7feb6c154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:46.522 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.527+0000 7feb73c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb6c1550a0 0x7feb6c06e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:46.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.527+0000 7feb73c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c073440 con 0x7feb6c0b4f90 2024-04-04T06:17:46.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.527+0000 7feb73c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c0b53f0 con 0x7feb6c0d0780 2024-04-04T06:17:46.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.527+0000 7feb73c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feb6c0adc80 con 0x7feb6c1550a0 2024-04-04T06:17:46.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.527+0000 7feb71200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb6c0b4f90 0x7feb6c151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:46.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.527+0000 7feb71200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb6c0b4f90 0x7feb6c151730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45830/0 (socket says 172.21.15.159:45830) 2024-04-04T06:17:46.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.527+0000 7feb71200640 1 -- 172.21.15.159:0/2305584745 learned_addr learned my addr 172.21.15.159:0/2305584745 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:46.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.527+0000 7feb71200640 1 -- 172.21.15.159:0/2305584745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb6c1550a0 msgr2=0x7feb6c06e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:46.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.527+0000 7feb71200640 1 --2- 172.21.15.159:0/2305584745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb6c1550a0 0x7feb6c06e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:46.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.527+0000 7feb71200640 1 -- 172.21.15.159:0/2305584745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb6c0d0780 msgr2=0x7feb6c154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:46.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.527+0000 7feb71200640 1 --2- 172.21.15.159:0/2305584745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb6c0d0780 0x7feb6c154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:46.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.527+0000 7feb71200640 1 -- 172.21.15.159:0/2305584745 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c153460 con 0x7feb6c0b4f90 2024-04-04T06:17:46.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.528+0000 7feb71200640 1 --2- 172.21.15.159:0/2305584745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb6c0b4f90 0x7feb6c151730 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7feb5c037040 tx=0x7feb5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:46.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.528+0000 7feb68c00640 1 -- 172.21.15.159:0/2305584745 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c044070 con 0x7feb6c0b4f90 2024-04-04T06:17:46.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.528+0000 7feb68c00640 1 -- 172.21.15.159:0/2305584745 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb5c003b90 con 0x7feb6c0b4f90 2024-04-04T06:17:46.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.528+0000 7feb68c00640 1 -- 172.21.15.159:0/2305584745 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c039040 con 0x7feb6c0b4f90 2024-04-04T06:17:46.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.528+0000 7feb73c00640 1 -- 172.21.15.159:0/2305584745 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6c153740 con 0x7feb6c0b4f90 2024-04-04T06:17:46.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.528+0000 7feb73c00640 1 -- 172.21.15.159:0/2305584745 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6c153d00 con 0x7feb6c0b4f90 2024-04-04T06:17:46.524 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.529+0000 7feb73c00640 1 -- 172.21.15.159:0/2305584745 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6c0685c0 con 0x7feb6c0b4f90 2024-04-04T06:17:46.525 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.530+0000 7feb68c00640 1 -- 172.21.15.159:0/2305584745 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb5c003f80 con 0x7feb6c0b4f90 2024-04-04T06:17:46.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.531+0000 7feb68c00640 1 --2- 172.21.15.159:0/2305584745 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feb4806c920 0x7feb4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:46.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.531+0000 7feb68c00640 1 -- 172.21.15.159:0/2305584745 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7feb5c0ae960 con 0x7feb6c0b4f90 2024-04-04T06:17:46.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.532+0000 7feb6be00640 1 --2- 172.21.15.159:0/2305584745 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feb4806c920 0x7feb4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:46.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.533+0000 7feb6be00640 1 --2- 172.21.15.159:0/2305584745 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feb4806c920 0x7feb4806ede0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7feb54007fd0 tx=0x7feb54007920 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:46.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.541+0000 7feb68c00640 1 -- 172.21.15.159:0/2305584745 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7feb5c07cc90 con 0x7feb6c0b4f90 2024-04-04T06:17:46.737 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.742+0000 7feb73c00640 1 -- 172.21.15.159:0/2305584745 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb6c15ec20 con 0x7feb6c0b4f90 2024-04-04T06:17:46.739 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.744+0000 7feb68c00640 1 -- 172.21.15.159:0/2305584745 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+16995 (secure 0 0 0) 0x7feb5c07c590 con 0x7feb6c0b4f90 2024-04-04T06:17:46.740 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:46.740 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":116,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:17:35.667542+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:16:55.036975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-04T06:16:56.239022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.b","mappings":[{"from":0,"to":5}]},{"pgid":"17.26","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6834/2439261212":"2024-04-05T06:17:35.667507+0000","172.21.15.159:6832/2439261212":"2024-04-05T06:17:35.667507+0000","172.21.15.159:6834/1017894858":"2024-04-05T06:17:21.596764+0000","172.21.15.159:6832/1017894858":"2024-04-05T06:17:21.596764+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:17:46.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.747+0000 7feb73c00640 1 -- 172.21.15.159:0/2305584745 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feb4806c920 msgr2=0x7feb4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:46.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.747+0000 7feb73c00640 1 --2- 172.21.15.159:0/2305584745 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feb4806c920 0x7feb4806ede0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7feb54007fd0 tx=0x7feb54007920 comp rx=0 tx=0).stop 2024-04-04T06:17:46.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.747+0000 7feb73c00640 1 -- 172.21.15.159:0/2305584745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb6c0b4f90 msgr2=0x7feb6c151730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:46.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.747+0000 7feb73c00640 1 --2- 172.21.15.159:0/2305584745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb6c0b4f90 0x7feb6c151730 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7feb5c037040 tx=0x7feb5c004a40 comp rx=0 tx=0).stop 2024-04-04T06:17:46.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.747+0000 7feb73c00640 1 -- 172.21.15.159:0/2305584745 shutdown_connections 2024-04-04T06:17:46.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.747+0000 7feb73c00640 1 --2- 172.21.15.159:0/2305584745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feb6c1550a0 0x7feb6c06e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:46.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.747+0000 7feb73c00640 1 --2- 172.21.15.159:0/2305584745 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feb4806c920 0x7feb4806ede0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:46.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.747+0000 7feb73c00640 1 --2- 172.21.15.159:0/2305584745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb6c0d0780 0x7feb6c154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:46.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.747+0000 7feb73c00640 1 --2- 172.21.15.159:0/2305584745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb6c0b4f90 0x7feb6c151730 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:46.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.747+0000 7feb73c00640 1 -- 172.21.15.159:0/2305584745 >> 172.21.15.159:0/2305584745 conn(0x7feb6c06b990 msgr2=0x7feb6c0ae010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:46.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.748+0000 7feb73c00640 1 -- 172.21.15.159:0/2305584745 shutdown_connections 2024-04-04T06:17:46.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.748+0000 7feb73c00640 1 -- 172.21.15.159:0/2305584745 wait complete. 2024-04-04T06:17:46.763 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:16:55.036975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:16:56.239022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:17:46.764 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:17:46.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.989+0000 7f4838600640 1 Processor -- start 2024-04-04T06:17:46.984 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.989+0000 7f4838600640 1 -- start start 2024-04-04T06:17:46.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.991+0000 7f4838600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f483015d630 0x7f48300c7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:46.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.991+0000 7f4838600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f48300d0540 0x7f48300ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:46.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.991+0000 7f4838600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48300c7dd0 con 0x7f483015ca70 2024-04-04T06:17:46.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.991+0000 7f4838600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f48300c80b0 con 0x7f48300d0540 2024-04-04T06:17:46.986 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.991+0000 7f4838600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f48300c8390 con 0x7f483015d630 2024-04-04T06:17:46.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.992+0000 7f4835c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f483015d630 0x7f48300c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:46.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.992+0000 7f4835200640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f483015ca70 0x7f48300cfe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.159:39520/0 (socket says 172.21.15.159:39520) 2024-04-04T06:17:46.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.992+0000 7f4835200640 1 -- 172.21.15.159:0/3182915697 learned_addr learned my addr 172.21.15.159:0/3182915697 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:46.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.992+0000 7f4836600640 1 --2- 172.21.15.159:0/3182915697 >> v2:172.21.15.178:3300/0 conn(0x7f48300d0540 0x7f48300ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:46.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.992+0000 7f482fe00640 1 -- 172.21.15.159:0/3182915697 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2324428842 0 0) 0x7f48300c7dd0 con 0x7f483015ca70 2024-04-04T06:17:46.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.993+0000 7f482fe00640 1 -- 172.21.15.159:0/3182915697 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4818003520 con 0x7f483015ca70 2024-04-04T06:17:46.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.993+0000 7f4836600640 1 -- 172.21.15.159:0/3182915697 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f483015d630 msgr2=0x7f48300c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:46.988 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.993+0000 7f4836600640 1 --2- 172.21.15.159:0/3182915697 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f483015d630 0x7f48300c7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:46.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.993+0000 7f4836600640 1 -- 172.21.15.159:0/3182915697 >> v1:172.21.15.178:6789/0 conn(0x7f483015ca70 legacy=0x7f48300cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:46.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.993+0000 7f4836600640 1 -- 172.21.15.159:0/3182915697 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48300c8670 con 0x7f48300d0540 2024-04-04T06:17:46.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.993+0000 7f4835c00640 1 --2- 172.21.15.159:0/3182915697 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f483015d630 0x7f48300c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-04T06:17:46.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.994+0000 7f4836600640 1 --2- 172.21.15.159:0/3182915697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48300d0540 0x7f48300ce8d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4828009d90 tx=0x7f482802beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=102edb7e27dd3eac server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:46.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.994+0000 7f482fe00640 1 -- 172.21.15.159:0/3182915697 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828039070 con 0x7f48300d0540 2024-04-04T06:17:46.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.995+0000 7f482fe00640 1 -- 172.21.15.159:0/3182915697 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f482802cc60 con 0x7f48300d0540 2024-04-04T06:17:46.990 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.995+0000 7f482fe00640 1 -- 172.21.15.159:0/3182915697 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828034640 con 0x7f48300d0540 2024-04-04T06:17:46.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.995+0000 7f4838600640 1 -- 172.21.15.159:0/3182915697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48300d0540 msgr2=0x7f48300ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:46.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.995+0000 7f4838600640 1 --2- 172.21.15.159:0/3182915697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48300d0540 0x7f48300ce8d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4828009d90 tx=0x7f482802beb0 comp rx=0 tx=0).stop 2024-04-04T06:17:46.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.995+0000 7f4838600640 1 -- 172.21.15.159:0/3182915697 shutdown_connections 2024-04-04T06:17:46.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.995+0000 7f4838600640 1 --2- 172.21.15.159:0/3182915697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48300d0540 0x7f48300ce8d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:46.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.995+0000 7f4838600640 1 --2- 172.21.15.159:0/3182915697 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f483015d630 0x7f48300c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:46.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.995+0000 7f4838600640 1 -- 172.21.15.159:0/3182915697 >> 172.21.15.159:0/3182915697 conn(0x7f48300b3e90 msgr2=0x7f48300b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:46.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.995+0000 7f4838600640 1 -- 172.21.15.159:0/3182915697 shutdown_connections 2024-04-04T06:17:46.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.995+0000 7f4838600640 1 -- 172.21.15.159:0/3182915697 wait complete. 2024-04-04T06:17:46.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.997+0000 7f4838600640 1 Processor -- start 2024-04-04T06:17:46.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.997+0000 7f4838600640 1 -- start start 2024-04-04T06:17:46.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.998+0000 7f4838600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48300d0540 0x7f4830152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:46.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.998+0000 7f4838600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f483015ca70 0x7f483006c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:46.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.998+0000 7f4838600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f483015d630 0x7f48300ab870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:46.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.998+0000 7f4838600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f48300cef90 con 0x7f483015d630 2024-04-04T06:17:46.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.998+0000 7f4838600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f48300cee10 con 0x7f483015ca70 2024-04-04T06:17:46.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.998+0000 7f4838600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f48300cf110 con 0x7f48300d0540 2024-04-04T06:17:46.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.999+0000 7f4836600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f483015d630 0x7f48300ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:46.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.999+0000 7f4836600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f483015d630 0x7f48300ab870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45844/0 (socket says 172.21.15.159:45844) 2024-04-04T06:17:46.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.999+0000 7f4836600640 1 -- 172.21.15.159:0/3664618960 learned_addr learned my addr 172.21.15.159:0/3664618960 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:46.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.999+0000 7f4835200640 1 --2- 172.21.15.159:0/3664618960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f483015ca70 0x7f483006c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:46.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.999+0000 7f4835c00640 1 --2- 172.21.15.159:0/3664618960 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48300d0540 0x7f4830152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:46.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.999+0000 7f4836600640 1 -- 172.21.15.159:0/3664618960 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48300d0540 msgr2=0x7f4830152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:46.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.999+0000 7f4836600640 1 --2- 172.21.15.159:0/3664618960 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48300d0540 0x7f4830152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:46.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.999+0000 7f4836600640 1 -- 172.21.15.159:0/3664618960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f483015ca70 msgr2=0x7f483006c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:46.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.999+0000 7f4836600640 1 --2- 172.21.15.159:0/3664618960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f483015ca70 0x7f483006c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:46.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.999+0000 7f4836600640 1 -- 172.21.15.159:0/3664618960 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483006d620 con 0x7f483015d630 2024-04-04T06:17:46.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:46.999+0000 7f4836600640 1 --2- 172.21.15.159:0/3664618960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f483015d630 0x7f48300ab870 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f4828009d90 tx=0x7f4828004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:46.996 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.000+0000 7f482d600640 1 -- 172.21.15.159:0/3664618960 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828039050 con 0x7f483015d630 2024-04-04T06:17:46.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.000+0000 7f482d600640 1 -- 172.21.15.159:0/3664618960 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4828045070 con 0x7f483015d630 2024-04-04T06:17:46.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.000+0000 7f482d600640 1 -- 172.21.15.159:0/3664618960 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828009450 con 0x7f483015d630 2024-04-04T06:17:46.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.000+0000 7f4838600640 1 -- 172.21.15.159:0/3664618960 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f483006d060 con 0x7f483015d630 2024-04-04T06:17:46.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.000+0000 7f4838600640 1 -- 172.21.15.159:0/3664618960 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48300a9520 con 0x7f483015d630 2024-04-04T06:17:46.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.001+0000 7f4838600640 1 -- 172.21.15.159:0/3664618960 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4830075a70 con 0x7f483015d630 2024-04-04T06:17:46.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.002+0000 7f482d600640 1 -- 172.21.15.159:0/3664618960 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f482803e830 con 0x7f483015d630 2024-04-04T06:17:46.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.003+0000 7f482d600640 1 --2- 172.21.15.159:0/3664618960 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f481006c8f0 0x7f481006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:46.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.003+0000 7f482d600640 1 -- 172.21.15.159:0/3664618960 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f48280ae7e0 con 0x7f483015d630 2024-04-04T06:17:46.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.004+0000 7f4835c00640 1 --2- 172.21.15.159:0/3664618960 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f481006c8f0 0x7f481006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:47.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.005+0000 7f4835c00640 1 --2- 172.21.15.159:0/3664618960 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f481006c8f0 0x7f481006edb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f4824009a30 tx=0x7f4824008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:47.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.007+0000 7f482d600640 1 -- 172.21.15.159:0/3664618960 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4830075a70 con 0x7f483015d630 2024-04-04T06:17:47.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.207+0000 7f4838600640 1 -- 172.21.15.159:0/3664618960 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4830068cd0 con 0x7f483015d630 2024-04-04T06:17:47.202 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.208+0000 7f482d600640 1 -- 172.21.15.159:0/3664618960 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) ==== 93+0+10 (secure 0 0 0) 0x7f4828032e20 con 0x7f483015d630 2024-04-04T06:17:47.203 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:17:47.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.210+0000 7f4838600640 1 -- 172.21.15.159:0/3664618960 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f481006c8f0 msgr2=0x7f481006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:47.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.210+0000 7f4838600640 1 --2- 172.21.15.159:0/3664618960 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f481006c8f0 0x7f481006edb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f4824009a30 tx=0x7f4824008040 comp rx=0 tx=0).stop 2024-04-04T06:17:47.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.210+0000 7f4838600640 1 -- 172.21.15.159:0/3664618960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f483015d630 msgr2=0x7f48300ab870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:47.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.210+0000 7f4838600640 1 --2- 172.21.15.159:0/3664618960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f483015d630 0x7f48300ab870 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f4828009d90 tx=0x7f4828004a40 comp rx=0 tx=0).stop 2024-04-04T06:17:47.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.210+0000 7f4838600640 1 -- 172.21.15.159:0/3664618960 shutdown_connections 2024-04-04T06:17:47.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.210+0000 7f4838600640 1 --2- 172.21.15.159:0/3664618960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f483015d630 0x7f48300ab870 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:47.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.210+0000 7f4838600640 1 --2- 172.21.15.159:0/3664618960 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f481006c8f0 0x7f481006edb0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:47.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.210+0000 7f4838600640 1 --2- 172.21.15.159:0/3664618960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f483015ca70 0x7f483006c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:47.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.210+0000 7f4838600640 1 --2- 172.21.15.159:0/3664618960 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48300d0540 0x7f4830152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:47.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.210+0000 7f4838600640 1 -- 172.21.15.159:0/3664618960 >> 172.21.15.159:0/3664618960 conn(0x7f48300b3e90 msgr2=0x7f483015ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:47.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.210+0000 7f4838600640 1 -- 172.21.15.159:0/3664618960 shutdown_connections 2024-04-04T06:17:47.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.210+0000 7f4838600640 1 -- 172.21.15.159:0/3664618960 wait complete. 2024-04-04T06:17:47.225 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:17:47.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.434+0000 7facefa00640 1 Processor -- start 2024-04-04T06:17:47.430 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.435+0000 7facefa00640 1 -- start start 2024-04-04T06:17:47.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.437+0000 7facefa00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7face815e6d0 0x7face80d21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:47.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.437+0000 7facefa00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7face80d0cb0 0x7face80d4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:47.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.437+0000 7facefa00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7face80ceef0 con 0x7face80d4590 2024-04-04T06:17:47.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.437+0000 7facefa00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7face815eab0 con 0x7face80d0cb0 2024-04-04T06:17:47.432 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.437+0000 7facefa00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7face80d1220 con 0x7face815e6d0 2024-04-04T06:17:47.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.437+0000 7face7e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7face80d0cb0 0x7face80d4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:47.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.437+0000 7face7e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7face80d0cb0 0x7face80d4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45858/0 (socket says 172.21.15.159:45858) 2024-04-04T06:17:47.433 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.437+0000 7face7e00640 1 -- 172.21.15.159:0/2144025921 learned_addr learned my addr 172.21.15.159:0/2144025921 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:47.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.438+0000 7face7e00640 1 -- 172.21.15.159:0/2144025921 >> v2:172.21.15.178:3300/0 conn(0x7face815e6d0 msgr2=0x7face80d21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:47.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.438+0000 7faced000640 1 --2- 172.21.15.159:0/2144025921 >> v2:172.21.15.178:3300/0 conn(0x7face815e6d0 0x7face80d21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:47.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.438+0000 7face7e00640 1 --2- 172.21.15.159:0/2144025921 >> v2:172.21.15.178:3300/0 conn(0x7face815e6d0 0x7face80d21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:47.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.438+0000 7face7e00640 1 -- 172.21.15.159:0/2144025921 >> v1:172.21.15.178:6789/0 conn(0x7face80d4590 legacy=0x7face81621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:47.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.438+0000 7face7e00640 1 -- 172.21.15.159:0/2144025921 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face80d1500 con 0x7face80d0cb0 2024-04-04T06:17:47.434 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.439+0000 7face7e00640 1 --2- 172.21.15.159:0/2144025921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7face80d0cb0 0x7face80d4050 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7face809de60 tx=0x7facd402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb382419b8d895a server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:47.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.439+0000 7face7400640 1 -- 172.21.15.159:0/2144025921 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd402d040 con 0x7face80d0cb0 2024-04-04T06:17:47.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.439+0000 7face7400640 1 -- 172.21.15.159:0/2144025921 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd4004230 con 0x7face80d0cb0 2024-04-04T06:17:47.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.439+0000 7face7400640 1 -- 172.21.15.159:0/2144025921 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4003610 con 0x7face80d0cb0 2024-04-04T06:17:47.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.440+0000 7facefa00640 1 -- 172.21.15.159:0/2144025921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7face80d0cb0 msgr2=0x7face80d4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:47.435 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.440+0000 7facefa00640 1 --2- 172.21.15.159:0/2144025921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7face80d0cb0 0x7face80d4050 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7face809de60 tx=0x7facd402b8f0 comp rx=0 tx=0).stop 2024-04-04T06:17:47.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.440+0000 7facefa00640 1 -- 172.21.15.159:0/2144025921 shutdown_connections 2024-04-04T06:17:47.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.440+0000 7facefa00640 1 --2- 172.21.15.159:0/2144025921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7face80d0cb0 0x7face80d4050 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:47.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.440+0000 7facefa00640 1 --2- 172.21.15.159:0/2144025921 >> v2:172.21.15.178:3300/0 conn(0x7face815e6d0 0x7face80d21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:47.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.440+0000 7facefa00640 1 -- 172.21.15.159:0/2144025921 >> 172.21.15.159:0/2144025921 conn(0x7face809d7b0 msgr2=0x7face808bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:47.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.440+0000 7facefa00640 1 -- 172.21.15.159:0/2144025921 shutdown_connections 2024-04-04T06:17:47.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.440+0000 7facefa00640 1 -- 172.21.15.159:0/2144025921 wait complete. 2024-04-04T06:17:47.436 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.442+0000 7facefa00640 1 Processor -- start 2024-04-04T06:17:47.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.442+0000 7facefa00640 1 -- start start 2024-04-04T06:17:47.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.443+0000 7facefa00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7face80d0cb0 0x7face80c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:47.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.443+0000 7facefa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7face80d4590 0x7face8153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:47.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.443+0000 7facefa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7face815e6d0 0x7face80ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:47.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.443+0000 7facefa00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7face80d4a20 con 0x7face80d4590 2024-04-04T06:17:47.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.443+0000 7facefa00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7face80d1ac0 con 0x7face815e6d0 2024-04-04T06:17:47.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.443+0000 7facefa00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7face80d2720 con 0x7face80d0cb0 2024-04-04T06:17:47.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.443+0000 7face7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7face80d4590 0x7face8153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:47.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.443+0000 7face7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7face80d4590 0x7face8153c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45874/0 (socket says 172.21.15.159:45874) 2024-04-04T06:17:47.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.443+0000 7face7e00640 1 -- 172.21.15.159:0/3216131693 learned_addr learned my addr 172.21.15.159:0/3216131693 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:47.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.443+0000 7faced000640 1 --2- 172.21.15.159:0/3216131693 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7face80d0cb0 0x7face80c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:47.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.443+0000 7face7e00640 1 -- 172.21.15.159:0/3216131693 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7face80d0cb0 msgr2=0x7face80c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:47.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.443+0000 7faceda00640 1 --2- 172.21.15.159:0/3216131693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7face815e6d0 0x7face80ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:47.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.443+0000 7face7e00640 1 --2- 172.21.15.159:0/3216131693 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7face80d0cb0 0x7face80c9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:47.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.443+0000 7face7e00640 1 -- 172.21.15.159:0/3216131693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7face815e6d0 msgr2=0x7face80ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:47.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.443+0000 7face7e00640 1 --2- 172.21.15.159:0/3216131693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7face815e6d0 0x7face80ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:47.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.443+0000 7face7e00640 1 -- 172.21.15.159:0/3216131693 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face80cade0 con 0x7face80d4590 2024-04-04T06:17:47.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.443+0000 7faced000640 1 --2- 172.21.15.159:0/3216131693 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7face80d0cb0 0x7face80c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:47.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.443+0000 7faceda00640 1 --2- 172.21.15.159:0/3216131693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7face815e6d0 0x7face80ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:47.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.444+0000 7face7e00640 1 --2- 172.21.15.159:0/3216131693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7face80d4590 0x7face8153c00 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7facd402b930 tx=0x7facd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:47.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.444+0000 7face4c00640 1 -- 172.21.15.159:0/3216131693 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4003e50 con 0x7face80d4590 2024-04-04T06:17:47.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.444+0000 7face4c00640 1 -- 172.21.15.159:0/3216131693 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd4006040 con 0x7face80d4590 2024-04-04T06:17:47.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.444+0000 7face4c00640 1 -- 172.21.15.159:0/3216131693 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4006450 con 0x7face80d4590 2024-04-04T06:17:47.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.444+0000 7facefa00640 1 -- 172.21.15.159:0/3216131693 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face80c8da0 con 0x7face80d4590 2024-04-04T06:17:47.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.444+0000 7facefa00640 1 -- 172.21.15.159:0/3216131693 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face80c9040 con 0x7face80d4590 2024-04-04T06:17:47.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.446+0000 7facefa00640 1 -- 172.21.15.159:0/3216131693 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face80685c0 con 0x7face80d4590 2024-04-04T06:17:47.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.447+0000 7face4c00640 1 -- 172.21.15.159:0/3216131693 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7facd4042020 con 0x7face80d4590 2024-04-04T06:17:47.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.448+0000 7face4c00640 1 --2- 172.21.15.159:0/3216131693 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7facc806ca90 0x7facc806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:47.442 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.448+0000 7face4c00640 1 -- 172.21.15.159:0/3216131693 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7facd40ada80 con 0x7face80d4590 2024-04-04T06:17:47.443 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.448+0000 7faced000640 1 --2- 172.21.15.159:0/3216131693 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7facc806ca90 0x7facc806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:47.444 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.450+0000 7faced000640 1 --2- 172.21.15.159:0/3216131693 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7facc806ca90 0x7facc806ef50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7face80cd750 tx=0x7facdc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:47.451 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.457+0000 7face4c00640 1 -- 172.21.15.159:0/3216131693 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7facd40afe20 con 0x7face80d4590 2024-04-04T06:17:47.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.655+0000 7facefa00640 1 -- 172.21.15.159:0/3216131693 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7face80b2600 con 0x7face80d4590 2024-04-04T06:17:47.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.656+0000 7face4c00640 1 -- 172.21.15.159:0/3216131693 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) ==== 104+0+11 (secure 0 0 0) 0x7facd407bd20 con 0x7face80d4590 2024-04-04T06:17:47.651 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:17:47.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.659+0000 7facefa00640 1 -- 172.21.15.159:0/3216131693 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7facc806ca90 msgr2=0x7facc806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:47.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.659+0000 7facefa00640 1 --2- 172.21.15.159:0/3216131693 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7facc806ca90 0x7facc806ef50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7face80cd750 tx=0x7facdc008040 comp rx=0 tx=0).stop 2024-04-04T06:17:47.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.659+0000 7facefa00640 1 -- 172.21.15.159:0/3216131693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7face80d4590 msgr2=0x7face8153c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:47.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.659+0000 7facefa00640 1 --2- 172.21.15.159:0/3216131693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7face80d4590 0x7face8153c00 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7facd402b930 tx=0x7facd40046c0 comp rx=0 tx=0).stop 2024-04-04T06:17:47.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.659+0000 7facefa00640 1 -- 172.21.15.159:0/3216131693 shutdown_connections 2024-04-04T06:17:47.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.659+0000 7facefa00640 1 --2- 172.21.15.159:0/3216131693 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7facc806ca90 0x7facc806ef50 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:47.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.659+0000 7facefa00640 1 --2- 172.21.15.159:0/3216131693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7face815e6d0 0x7face80ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:47.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.659+0000 7facefa00640 1 --2- 172.21.15.159:0/3216131693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7face80d4590 0x7face8153c00 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:47.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.659+0000 7facefa00640 1 --2- 172.21.15.159:0/3216131693 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7face80d0cb0 0x7face80c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:47.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.659+0000 7facefa00640 1 -- 172.21.15.159:0/3216131693 >> 172.21.15.159:0/3216131693 conn(0x7face809d7b0 msgr2=0x7face80b2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:47.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.659+0000 7facefa00640 1 -- 172.21.15.159:0/3216131693 shutdown_connections 2024-04-04T06:17:47.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.659+0000 7facefa00640 1 -- 172.21.15.159:0/3216131693 wait complete. 2024-04-04T06:17:47.672 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:17:47.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.875+0000 7f08d0200640 1 Processor -- start 2024-04-04T06:17:47.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.875+0000 7f08d0200640 1 -- start start 2024-04-04T06:17:47.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.877+0000 7f08d0200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f08c80cf050 0x7f08c815f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:47.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.877+0000 7f08d0200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08c80d2110 0x7f08c80d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:47.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.877+0000 7f08d0200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08c80cf6c0 con 0x7f08c80d0e70 2024-04-04T06:17:47.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.877+0000 7f08d0200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f08c815fc20 con 0x7f08c80cf050 2024-04-04T06:17:47.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.877+0000 7f08d0200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f08c80cf9a0 con 0x7f08c80d2110 2024-04-04T06:17:47.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.877+0000 7f08cce00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f08c80cf050 0x7f08c815f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:47.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.877+0000 7f08cce00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f08c80cf050 0x7f08c815f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45884/0 (socket says 172.21.15.159:45884) 2024-04-04T06:17:47.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.877+0000 7f08cce00640 1 -- 172.21.15.159:0/844786144 learned_addr learned my addr 172.21.15.159:0/844786144 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:47.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.878+0000 7f08ce200640 1 --2- 172.21.15.159:0/844786144 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08c80d2110 0x7f08c80d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:47.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.878+0000 7f08cce00640 1 -- 172.21.15.159:0/844786144 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08c80d2110 msgr2=0x7f08c80d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:47.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.878+0000 7f08cce00640 1 --2- 172.21.15.159:0/844786144 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08c80d2110 0x7f08c80d25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:47.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.878+0000 7f08cce00640 1 -- 172.21.15.159:0/844786144 >> v1:172.21.15.178:6789/0 conn(0x7f08c80d0e70 legacy=0x7f08c80d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:47.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.878+0000 7f08cce00640 1 -- 172.21.15.159:0/844786144 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c80cfc80 con 0x7f08c80cf050 2024-04-04T06:17:47.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.879+0000 7f08cce00640 1 --2- 172.21.15.159:0/844786144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c80cf050 0x7f08c815f6e0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009870 tx=0x7f08b802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dcd52b82eaeb30e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:47.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.879+0000 7f08c7e00640 1 -- 172.21.15.159:0/844786144 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8035020 con 0x7f08c80cf050 2024-04-04T06:17:47.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.879+0000 7f08c7e00640 1 -- 172.21.15.159:0/844786144 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08b8031640 con 0x7f08c80cf050 2024-04-04T06:17:47.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.879+0000 7f08c7e00640 1 -- 172.21.15.159:0/844786144 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b802d040 con 0x7f08c80cf050 2024-04-04T06:17:47.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.880+0000 7f08d0200640 1 -- 172.21.15.159:0/844786144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c80cf050 msgr2=0x7f08c815f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:47.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.880+0000 7f08d0200640 1 --2- 172.21.15.159:0/844786144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c80cf050 0x7f08c815f6e0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009870 tx=0x7f08b802b8f0 comp rx=0 tx=0).stop 2024-04-04T06:17:47.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.880+0000 7f08d0200640 1 -- 172.21.15.159:0/844786144 shutdown_connections 2024-04-04T06:17:47.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.880+0000 7f08d0200640 1 --2- 172.21.15.159:0/844786144 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08c80d2110 0x7f08c80d25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:47.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.880+0000 7f08d0200640 1 --2- 172.21.15.159:0/844786144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c80cf050 0x7f08c815f6e0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:47.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.880+0000 7f08d0200640 1 -- 172.21.15.159:0/844786144 >> 172.21.15.159:0/844786144 conn(0x7f08c808bfb0 msgr2=0x7f08c80d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:47.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.880+0000 7f08d0200640 1 -- 172.21.15.159:0/844786144 shutdown_connections 2024-04-04T06:17:47.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.880+0000 7f08d0200640 1 -- 172.21.15.159:0/844786144 wait complete. 2024-04-04T06:17:47.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.883+0000 7f08d0200640 1 Processor -- start 2024-04-04T06:17:47.878 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.883+0000 7f08d0200640 1 -- start start 2024-04-04T06:17:47.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.885+0000 7f08d0200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08c80cf050 0x7f08c8160010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:47.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.885+0000 7f08d0200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08c80d0e70 0x7f08c80cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:47.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.885+0000 7f08d0200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c80d2110 0x7f08c81544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:47.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.885+0000 7f08d0200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f08c815e910 con 0x7f08c80d2110 2024-04-04T06:17:47.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.885+0000 7f08d0200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f08c80d0240 con 0x7f08c80d0e70 2024-04-04T06:17:47.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.885+0000 7f08d0200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f08c815ea90 con 0x7f08c80cf050 2024-04-04T06:17:47.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.885+0000 7f08ce200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c80d2110 0x7f08c81544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:47.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.885+0000 7f08ce200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c80d2110 0x7f08c81544f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45890/0 (socket says 172.21.15.159:45890) 2024-04-04T06:17:47.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.885+0000 7f08ce200640 1 -- 172.21.15.159:0/2313751504 learned_addr learned my addr 172.21.15.159:0/2313751504 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:47.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.886+0000 7f08cce00640 1 --2- 172.21.15.159:0/2313751504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08c80d0e70 0x7f08c80cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:47.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.886+0000 7f08cd800640 1 --2- 172.21.15.159:0/2313751504 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08c80cf050 0x7f08c8160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:47.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.886+0000 7f08ce200640 1 -- 172.21.15.159:0/2313751504 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08c80cf050 msgr2=0x7f08c8160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:47.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.886+0000 7f08ce200640 1 --2- 172.21.15.159:0/2313751504 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08c80cf050 0x7f08c8160010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:47.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.886+0000 7f08ce200640 1 -- 172.21.15.159:0/2313751504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08c80d0e70 msgr2=0x7f08c80cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:47.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.886+0000 7f08ce200640 1 --2- 172.21.15.159:0/2313751504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08c80d0e70 0x7f08c80cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:47.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.886+0000 7f08ce200640 1 -- 172.21.15.159:0/2313751504 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c80cbf80 con 0x7f08c80d2110 2024-04-04T06:17:47.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.886+0000 7f08cce00640 1 --2- 172.21.15.159:0/2313751504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08c80d0e70 0x7f08c80cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:47.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.886+0000 7f08ce200640 1 --2- 172.21.15.159:0/2313751504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c80d2110 0x7f08c81544f0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f08c0007e40 tx=0x7f08c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:47.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.886+0000 7f08c5600640 1 -- 172.21.15.159:0/2313751504 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c0014620 con 0x7f08c80d2110 2024-04-04T06:17:47.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.886+0000 7f08c5600640 1 -- 172.21.15.159:0/2313751504 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c0019070 con 0x7f08c80d2110 2024-04-04T06:17:47.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.886+0000 7f08c5600640 1 -- 172.21.15.159:0/2313751504 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c0014a20 con 0x7f08c80d2110 2024-04-04T06:17:47.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.886+0000 7f08d0200640 1 -- 172.21.15.159:0/2313751504 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c80cc260 con 0x7f08c80d2110 2024-04-04T06:17:47.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.886+0000 7f08d0200640 1 -- 172.21.15.159:0/2313751504 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c80cc820 con 0x7f08c80d2110 2024-04-04T06:17:47.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.887+0000 7f08d0200640 1 -- 172.21.15.159:0/2313751504 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08c8068cd0 con 0x7f08c80d2110 2024-04-04T06:17:47.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.889+0000 7f08c5600640 1 -- 172.21.15.159:0/2313751504 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08c0014bc0 con 0x7f08c80d2110 2024-04-04T06:17:47.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.890+0000 7f08c5600640 1 --2- 172.21.15.159:0/2313751504 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f08ac06c8f0 0x7f08ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:47.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.890+0000 7f08c5600640 1 -- 172.21.15.159:0/2313751504 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f08c008c550 con 0x7f08c80d2110 2024-04-04T06:17:47.885 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.891+0000 7f08cd800640 1 --2- 172.21.15.159:0/2313751504 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f08ac06c8f0 0x7f08ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:47.887 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.892+0000 7f08cd800640 1 --2- 172.21.15.159:0/2313751504 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f08ac06c8f0 0x7f08ac06edb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f08bc003d90 tx=0x7f08bc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:47.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:47.899+0000 7f08c5600640 1 -- 172.21.15.159:0/2313751504 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f08c005a890 con 0x7f08c80d2110 2024-04-04T06:17:48.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.099+0000 7f08d0200640 1 -- 172.21.15.159:0/2313751504 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f08c80d1b80 con 0x7f08c80d2110 2024-04-04T06:17:48.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.100+0000 7f08c5600640 1 -- 172.21.15.159:0/2313751504 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) ==== 100+0+11 (secure 0 0 0) 0x7f08c005a190 con 0x7f08c80d2110 2024-04-04T06:17:48.095 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:17:48.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.103+0000 7f08d0200640 1 -- 172.21.15.159:0/2313751504 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f08ac06c8f0 msgr2=0x7f08ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:48.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.103+0000 7f08d0200640 1 --2- 172.21.15.159:0/2313751504 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f08ac06c8f0 0x7f08ac06edb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f08bc003d90 tx=0x7f08bc009040 comp rx=0 tx=0).stop 2024-04-04T06:17:48.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.103+0000 7f08d0200640 1 -- 172.21.15.159:0/2313751504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c80d2110 msgr2=0x7f08c81544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:48.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.103+0000 7f08d0200640 1 --2- 172.21.15.159:0/2313751504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c80d2110 0x7f08c81544f0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f08c0007e40 tx=0x7f08c000e4b0 comp rx=0 tx=0).stop 2024-04-04T06:17:48.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.103+0000 7f08d0200640 1 -- 172.21.15.159:0/2313751504 shutdown_connections 2024-04-04T06:17:48.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.103+0000 7f08d0200640 1 --2- 172.21.15.159:0/2313751504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c80d2110 0x7f08c81544f0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:48.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.103+0000 7f08d0200640 1 --2- 172.21.15.159:0/2313751504 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f08ac06c8f0 0x7f08ac06edb0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:48.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.103+0000 7f08d0200640 1 --2- 172.21.15.159:0/2313751504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08c80d0e70 0x7f08c80cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:48.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.103+0000 7f08d0200640 1 --2- 172.21.15.159:0/2313751504 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f08c80cf050 0x7f08c8160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:48.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.103+0000 7f08d0200640 1 -- 172.21.15.159:0/2313751504 >> 172.21.15.159:0/2313751504 conn(0x7f08c808bfb0 msgr2=0x7f08c8074580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:48.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.103+0000 7f08d0200640 1 -- 172.21.15.159:0/2313751504 shutdown_connections 2024-04-04T06:17:48.099 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.103+0000 7f08d0200640 1 -- 172.21.15.159:0/2313751504 wait complete. 2024-04-04T06:17:48.116 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-04T06:17:48.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.317+0000 7ffa2b600640 1 Processor -- start 2024-04-04T06:17:48.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.318+0000 7ffa2b600640 1 -- start start 2024-04-04T06:17:48.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.319+0000 7ffa2b600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ffa24155550 0x7ffa24155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:48.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.319+0000 7ffa2b600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ffa2415f800 0x7ffa240d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:48.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.319+0000 7ffa2b600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa240d07c0 0x7ffa240aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:48.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.319+0000 7ffa2b600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7ffa240d0c10 con 0x7ffa2415f800 2024-04-04T06:17:48.315 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.319+0000 7ffa2b600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ffa240b40b0 con 0x7ffa24155550 2024-04-04T06:17:48.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.319+0000 7ffa2b600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ffa240b4390 con 0x7ffa240d07c0 2024-04-04T06:17:48.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.320+0000 7ffa1be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ffa2415f800 0x7ffa240d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:48.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.320+0000 7ffa1be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ffa2415f800 0x7ffa240d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45898/0 (socket says 172.21.15.159:45898) 2024-04-04T06:17:48.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.320+0000 7ffa1be00640 1 -- 172.21.15.159:0/3257153862 learned_addr learned my addr 172.21.15.159:0/3257153862 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:48.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.320+0000 7ffa29600640 1 --2- 172.21.15.159:0/3257153862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa240d07c0 0x7ffa240aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:48.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.320+0000 7ffa28c00640 1 --2- 172.21.15.159:0/3257153862 >> v2:172.21.15.178:3300/0 conn(0x7ffa24155550 0x7ffa24155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:48.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.320+0000 7ffa1be00640 1 -- 172.21.15.159:0/3257153862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa240d07c0 msgr2=0x7ffa240aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:48.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.320+0000 7ffa1be00640 1 --2- 172.21.15.159:0/3257153862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa240d07c0 0x7ffa240aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:48.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.320+0000 7ffa1be00640 1 -- 172.21.15.159:0/3257153862 >> v2:172.21.15.178:3300/0 conn(0x7ffa24155550 msgr2=0x7ffa24155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:48.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.320+0000 7ffa1be00640 1 --2- 172.21.15.159:0/3257153862 >> v2:172.21.15.178:3300/0 conn(0x7ffa24155550 0x7ffa24155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:48.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.320+0000 7ffa1be00640 1 -- 172.21.15.159:0/3257153862 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa240b4670 con 0x7ffa2415f800 2024-04-04T06:17:48.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.320+0000 7ffa29600640 1 --2- 172.21.15.159:0/3257153862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa240d07c0 0x7ffa240aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:17:48.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.322+0000 7ffa1be00640 1 --2- 172.21.15.159:0/3257153862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa2415f800 0x7ffa240d0280 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7ffa24073ac0 tx=0x7ffa0c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6014a1d25500172 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:48.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.322+0000 7ffa23e00640 1 -- 172.21.15.159:0/3257153862 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c035020 con 0x7ffa2415f800 2024-04-04T06:17:48.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.322+0000 7ffa23e00640 1 -- 172.21.15.159:0/3257153862 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa0c031870 con 0x7ffa2415f800 2024-04-04T06:17:48.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.322+0000 7ffa23e00640 1 -- 172.21.15.159:0/3257153862 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c031c40 con 0x7ffa2415f800 2024-04-04T06:17:48.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.323+0000 7ffa2b600640 1 -- 172.21.15.159:0/3257153862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa2415f800 msgr2=0x7ffa240d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:48.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.323+0000 7ffa2b600640 1 --2- 172.21.15.159:0/3257153862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa2415f800 0x7ffa240d0280 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7ffa24073ac0 tx=0x7ffa0c02b910 comp rx=0 tx=0).stop 2024-04-04T06:17:48.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.323+0000 7ffa2b600640 1 -- 172.21.15.159:0/3257153862 shutdown_connections 2024-04-04T06:17:48.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.323+0000 7ffa2b600640 1 --2- 172.21.15.159:0/3257153862 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa240d07c0 0x7ffa240aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:48.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.323+0000 7ffa2b600640 1 --2- 172.21.15.159:0/3257153862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa2415f800 0x7ffa240d0280 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:48.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.323+0000 7ffa2b600640 1 --2- 172.21.15.159:0/3257153862 >> v2:172.21.15.178:3300/0 conn(0x7ffa24155550 0x7ffa24155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:48.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.323+0000 7ffa2b600640 1 -- 172.21.15.159:0/3257153862 >> 172.21.15.159:0/3257153862 conn(0x7ffa2406b750 msgr2=0x7ffa2406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:48.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.323+0000 7ffa2b600640 1 -- 172.21.15.159:0/3257153862 shutdown_connections 2024-04-04T06:17:48.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.323+0000 7ffa2b600640 1 -- 172.21.15.159:0/3257153862 wait complete. 2024-04-04T06:17:48.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.325+0000 7ffa2b600640 1 Processor -- start 2024-04-04T06:17:48.320 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.325+0000 7ffa2b600640 1 -- start start 2024-04-04T06:17:48.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.327+0000 7ffa2b600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa240d07c0 0x7ffa240998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:48.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.327+0000 7ffa2b600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa24155550 0x7ffa241f4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:48.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.327+0000 7ffa2b600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa2415f800 0x7ffa241f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:48.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.327+0000 7ffa2b600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2415fc80 con 0x7ffa240d07c0 2024-04-04T06:17:48.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.327+0000 7ffa2b600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffa240b4c30 con 0x7ffa24155550 2024-04-04T06:17:48.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.327+0000 7ffa2b600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ffa2415ba20 con 0x7ffa2415f800 2024-04-04T06:17:48.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.327+0000 7ffa28c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa240d07c0 0x7ffa240998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:48.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.327+0000 7ffa28c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa240d07c0 0x7ffa240998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45910/0 (socket says 172.21.15.159:45910) 2024-04-04T06:17:48.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.327+0000 7ffa28c00640 1 -- 172.21.15.159:0/1757894446 learned_addr learned my addr 172.21.15.159:0/1757894446 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:48.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.328+0000 7ffa1be00640 1 --2- 172.21.15.159:0/1757894446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa24155550 0x7ffa241f4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:48.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.328+0000 7ffa28c00640 1 -- 172.21.15.159:0/1757894446 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa2415f800 msgr2=0x7ffa241f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:48.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.328+0000 7ffa28c00640 1 --2- 172.21.15.159:0/1757894446 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa2415f800 0x7ffa241f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:48.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.328+0000 7ffa28c00640 1 -- 172.21.15.159:0/1757894446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa24155550 msgr2=0x7ffa241f4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:48.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.328+0000 7ffa28c00640 1 --2- 172.21.15.159:0/1757894446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa24155550 0x7ffa241f4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:48.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.328+0000 7ffa28c00640 1 -- 172.21.15.159:0/1757894446 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa240cf150 con 0x7ffa240d07c0 2024-04-04T06:17:48.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.328+0000 7ffa28c00640 1 --2- 172.21.15.159:0/1757894446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa240d07c0 0x7ffa240998e0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400d800 tx=0x7ffa1400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:48.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.328+0000 7ffa21600640 1 -- 172.21.15.159:0/1757894446 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1400b6c0 con 0x7ffa240d07c0 2024-04-04T06:17:48.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.328+0000 7ffa21600640 1 -- 172.21.15.159:0/1757894446 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa14019070 con 0x7ffa240d07c0 2024-04-04T06:17:48.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.328+0000 7ffa2b600640 1 -- 172.21.15.159:0/1757894446 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa240cf430 con 0x7ffa240d07c0 2024-04-04T06:17:48.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.328+0000 7ffa21600640 1 -- 172.21.15.159:0/1757894446 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1400ba80 con 0x7ffa240d07c0 2024-04-04T06:17:48.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.328+0000 7ffa2b600640 1 -- 172.21.15.159:0/1757894446 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa2415b130 con 0x7ffa240d07c0 2024-04-04T06:17:48.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.330+0000 7ffa2b600640 1 -- 172.21.15.159:0/1757894446 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa240685c0 con 0x7ffa240d07c0 2024-04-04T06:17:48.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.331+0000 7ffa21600640 1 -- 172.21.15.159:0/1757894446 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa1400bc20 con 0x7ffa240d07c0 2024-04-04T06:17:48.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.332+0000 7ffa21600640 1 --2- 172.21.15.159:0/1757894446 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffa0006c8f0 0x7ffa0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:48.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.332+0000 7ffa21600640 1 -- 172.21.15.159:0/1757894446 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7ffa1408c0e0 con 0x7ffa240d07c0 2024-04-04T06:17:48.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.332+0000 7ffa1be00640 1 --2- 172.21.15.159:0/1757894446 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffa0006c8f0 0x7ffa0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:48.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.333+0000 7ffa1be00640 1 --2- 172.21.15.159:0/1757894446 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffa0006c8f0 0x7ffa0006edb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c036000 tx=0x7ffa0c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:48.335 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.341+0000 7ffa21600640 1 -- 172.21.15.159:0/1757894446 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ffa1405a380 con 0x7ffa240d07c0 2024-04-04T06:17:48.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.554+0000 7ffa2b600640 1 -- 172.21.15.159:0/1757894446 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ffa240cef30 con 0x7ffa240d07c0 2024-04-04T06:17:48.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.555+0000 7ffa21600640 1 -- 172.21.15.159:0/1757894446 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v116) ==== 80+0+248 (secure 0 0 0) 0x7ffa14059c80 con 0x7ffa240d07c0 2024-04-04T06:17:48.550 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6834/2439261212 2024-04-05T06:17:35.667507+0000 2024-04-04T06:17:48.550 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6832/2439261212 2024-04-05T06:17:35.667507+0000 2024-04-04T06:17:48.550 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6834/1017894858 2024-04-05T06:17:21.596764+0000 2024-04-04T06:17:48.550 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159:6832/1017894858 2024-04-05T06:17:21.596764+0000 2024-04-04T06:17:48.551 INFO:teuthology.orchestra.run.smithi159.stderr:listed 4 entries 2024-04-04T06:17:48.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.557+0000 7ffa2b600640 1 -- 172.21.15.159:0/1757894446 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffa0006c8f0 msgr2=0x7ffa0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:48.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.557+0000 7ffa2b600640 1 --2- 172.21.15.159:0/1757894446 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffa0006c8f0 0x7ffa0006edb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c036000 tx=0x7ffa0c008030 comp rx=0 tx=0).stop 2024-04-04T06:17:48.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.557+0000 7ffa2b600640 1 -- 172.21.15.159:0/1757894446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa240d07c0 msgr2=0x7ffa240998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:48.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.557+0000 7ffa2b600640 1 --2- 172.21.15.159:0/1757894446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa240d07c0 0x7ffa240998e0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400d800 tx=0x7ffa1400dcc0 comp rx=0 tx=0).stop 2024-04-04T06:17:48.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.557+0000 7ffa2b600640 1 -- 172.21.15.159:0/1757894446 shutdown_connections 2024-04-04T06:17:48.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.557+0000 7ffa2b600640 1 --2- 172.21.15.159:0/1757894446 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa2415f800 0x7ffa241f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:48.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.557+0000 7ffa2b600640 1 --2- 172.21.15.159:0/1757894446 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffa0006c8f0 0x7ffa0006edb0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:48.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.557+0000 7ffa2b600640 1 --2- 172.21.15.159:0/1757894446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa24155550 0x7ffa241f4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:48.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.557+0000 7ffa2b600640 1 --2- 172.21.15.159:0/1757894446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa240d07c0 0x7ffa240998e0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:48.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.557+0000 7ffa2b600640 1 -- 172.21.15.159:0/1757894446 >> 172.21.15.159:0/1757894446 conn(0x7ffa2406b750 msgr2=0x7ffa24071610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:48.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.557+0000 7ffa2b600640 1 -- 172.21.15.159:0/1757894446 shutdown_connections 2024-04-04T06:17:48.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:48.557+0000 7ffa2b600640 1 -- 172.21.15.159:0/1757894446 wait complete. 2024-04-04T06:17:48.569 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-04T06:17:48.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:48.569 DEBUG:teuthology.orchestra.run.smithi159:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-04T06:17:48.667 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:17:48.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:48.667 DEBUG:teuthology.orchestra.run.smithi159:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-04T06:17:48.712 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:17:48.713 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:17:48.713 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:17:48.771 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:17:48.772 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:17:48.772 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:17:48.772 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:17:48.830 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:17:48.830 INFO:tasks.cephfs.mount:Terminating background process 2024-04-04T06:17:48.830 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:17:48.830 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:17:48.887 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:17:48.887 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:17:49.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.096+0000 7ff5df400640 1 Processor -- start 2024-04-04T06:17:49.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.096+0000 7ff5df400640 1 -- start start 2024-04-04T06:17:49.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.098+0000 7ff5df400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff5d815c390 0x7ff5d8073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:49.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.098+0000 7ff5df400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5d80d0650 0x7ff5d80d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:49.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.098+0000 7ff5df400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5d815fe80 con 0x7ff5d80af830 2024-04-04T06:17:49.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.098+0000 7ff5df400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7ff5d815c770 con 0x7ff5d815c390 2024-04-04T06:17:49.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.098+0000 7ff5df400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff5d80ad120 con 0x7ff5d80d0650 2024-04-04T06:17:49.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.098+0000 7ff5dca00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff5d815c390 0x7ff5d8073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:49.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.098+0000 7ff5d7e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7ff5d80af830 0x7ff5d8160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:42912/0 (socket says 172.21.15.159:42912) 2024-04-04T06:17:49.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.098+0000 7ff5d7e00640 1 -- 172.21.15.159:0/3195472093 learned_addr learned my addr 172.21.15.159:0/3195472093 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:49.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.099+0000 7ff5dd400640 1 --2- 172.21.15.159:0/3195472093 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5d80d0650 0x7ff5d80d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:49.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.099+0000 7ff5dca00640 1 -- 172.21.15.159:0/3195472093 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5d80d0650 msgr2=0x7ff5d80d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:49.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.099+0000 7ff5dca00640 1 --2- 172.21.15.159:0/3195472093 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5d80d0650 0x7ff5d80d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:49.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.099+0000 7ff5dca00640 1 -- 172.21.15.159:0/3195472093 >> v1:172.21.15.159:6789/0 conn(0x7ff5d80af830 legacy=0x7ff5d8160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:49.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.099+0000 7ff5dca00640 1 -- 172.21.15.159:0/3195472093 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d80ad400 con 0x7ff5d815c390 2024-04-04T06:17:49.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.099+0000 7ff5d7400640 1 -- 172.21.15.159:0/3195472093 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4178273806 0 0) 0x7ff5d815fe80 con 0x7ff5d80af830 2024-04-04T06:17:49.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.100+0000 7ff5dca00640 1 --2- 172.21.15.159:0/3195472093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5d815c390 0x7ff5d8073790 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc0097a0 tx=0x7ff5cc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506f48a3b25802dd server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:49.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.100+0000 7ff5d7400640 1 -- 172.21.15.159:0/3195472093 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc035020 con 0x7ff5d815c390 2024-04-04T06:17:49.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.100+0000 7ff5d7400640 1 -- 172.21.15.159:0/3195472093 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5cc031830 con 0x7ff5d815c390 2024-04-04T06:17:49.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.100+0000 7ff5d7400640 1 -- 172.21.15.159:0/3195472093 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc031c00 con 0x7ff5d815c390 2024-04-04T06:17:49.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.101+0000 7ff5df400640 1 -- 172.21.15.159:0/3195472093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5d815c390 msgr2=0x7ff5d8073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:49.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.101+0000 7ff5df400640 1 --2- 172.21.15.159:0/3195472093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5d815c390 0x7ff5d8073790 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc0097a0 tx=0x7ff5cc02b8d0 comp rx=0 tx=0).stop 2024-04-04T06:17:49.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.101+0000 7ff5df400640 1 -- 172.21.15.159:0/3195472093 shutdown_connections 2024-04-04T06:17:49.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.101+0000 7ff5df400640 1 --2- 172.21.15.159:0/3195472093 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5d80d0650 0x7ff5d80d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:49.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.101+0000 7ff5df400640 1 --2- 172.21.15.159:0/3195472093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5d815c390 0x7ff5d8073790 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:49.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.101+0000 7ff5df400640 1 -- 172.21.15.159:0/3195472093 >> 172.21.15.159:0/3195472093 conn(0x7ff5d806b990 msgr2=0x7ff5d806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:49.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.101+0000 7ff5df400640 1 -- 172.21.15.159:0/3195472093 shutdown_connections 2024-04-04T06:17:49.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.101+0000 7ff5df400640 1 -- 172.21.15.159:0/3195472093 wait complete. 2024-04-04T06:17:49.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.103+0000 7ff5df400640 1 Processor -- start 2024-04-04T06:17:49.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.103+0000 7ff5df400640 1 -- start start 2024-04-04T06:17:49.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.104+0000 7ff5df400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5d80af830 0x7ff5d814d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:49.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.104+0000 7ff5df400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5d80d0650 0x7ff5d8158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:49.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.104+0000 7ff5df400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5d815c390 0x7ff5d8156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:49.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.104+0000 7ff5df400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d8160d50 con 0x7ff5d815c390 2024-04-04T06:17:49.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.104+0000 7ff5df400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d8160120 con 0x7ff5d80d0650 2024-04-04T06:17:49.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.104+0000 7ff5df400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff5d80cf980 con 0x7ff5d80af830 2024-04-04T06:17:49.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.104+0000 7ff5dd400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5d815c390 0x7ff5d8156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:49.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.104+0000 7ff5dd400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5d815c390 0x7ff5d8156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45930/0 (socket says 172.21.15.159:45930) 2024-04-04T06:17:49.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.104+0000 7ff5dd400640 1 -- 172.21.15.159:0/2425982685 learned_addr learned my addr 172.21.15.159:0/2425982685 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:49.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.105+0000 7ff5d7e00640 1 --2- 172.21.15.159:0/2425982685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5d80d0650 0x7ff5d8158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:49.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.105+0000 7ff5dca00640 1 --2- 172.21.15.159:0/2425982685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5d80af830 0x7ff5d814d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:49.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.105+0000 7ff5dd400640 1 -- 172.21.15.159:0/2425982685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5d80af830 msgr2=0x7ff5d814d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:49.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.105+0000 7ff5dd400640 1 --2- 172.21.15.159:0/2425982685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5d80af830 0x7ff5d814d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:49.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.105+0000 7ff5dd400640 1 -- 172.21.15.159:0/2425982685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5d80d0650 msgr2=0x7ff5d8158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:49.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.105+0000 7ff5dd400640 1 --2- 172.21.15.159:0/2425982685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5d80d0650 0x7ff5d8158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:49.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.105+0000 7ff5dd400640 1 -- 172.21.15.159:0/2425982685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d81594c0 con 0x7ff5d815c390 2024-04-04T06:17:49.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.105+0000 7ff5d7e00640 1 --2- 172.21.15.159:0/2425982685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5d80d0650 0x7ff5d8158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:49.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.105+0000 7ff5dca00640 1 --2- 172.21.15.159:0/2425982685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5d80af830 0x7ff5d814d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:49.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.105+0000 7ff5dd400640 1 --2- 172.21.15.159:0/2425982685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5d815c390 0x7ff5d8156ea0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0007e70 tx=0x7ff5d000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:49.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.105+0000 7ff5d4c00640 1 -- 172.21.15.159:0/2425982685 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0014420 con 0x7ff5d815c390 2024-04-04T06:17:49.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.105+0000 7ff5d4c00640 1 -- 172.21.15.159:0/2425982685 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d0019070 con 0x7ff5d815c390 2024-04-04T06:17:49.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.105+0000 7ff5d4c00640 1 -- 172.21.15.159:0/2425982685 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0014820 con 0x7ff5d815c390 2024-04-04T06:17:49.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.105+0000 7ff5df400640 1 -- 172.21.15.159:0/2425982685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d81597a0 con 0x7ff5d815c390 2024-04-04T06:17:49.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.106+0000 7ff5df400640 1 -- 172.21.15.159:0/2425982685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d8159d60 con 0x7ff5d815c390 2024-04-04T06:17:49.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.107+0000 7ff5df400640 1 -- 172.21.15.159:0/2425982685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5d80685c0 con 0x7ff5d815c390 2024-04-04T06:17:49.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.108+0000 7ff5d4c00640 1 -- 172.21.15.159:0/2425982685 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5d00149c0 con 0x7ff5d815c390 2024-04-04T06:17:49.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.109+0000 7ff5d4c00640 1 --2- 172.21.15.159:0/2425982685 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff5b806c8f0 0x7ff5b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:49.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.109+0000 7ff5d4c00640 1 -- 172.21.15.159:0/2425982685 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7ff5d008c5b0 con 0x7ff5d815c390 2024-04-04T06:17:49.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.109+0000 7ff5dca00640 1 --2- 172.21.15.159:0/2425982685 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff5b806c8f0 0x7ff5b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:49.105 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.110+0000 7ff5dca00640 1 --2- 172.21.15.159:0/2425982685 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff5b806c8f0 0x7ff5b806edb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc036000 tx=0x7ff5cc008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:49.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.113+0000 7ff5d4c00640 1 -- 172.21.15.159:0/2425982685 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff5d005a850 con 0x7ff5d815c390 2024-04-04T06:17:49.346 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.351+0000 7ff5df400640 1 -- 172.21.15.159:0/2425982685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5d806f1b0 con 0x7ff5d815c390 2024-04-04T06:17:49.347 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.352+0000 7ff5d4c00640 1 -- 172.21.15.159:0/2425982685 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5197 (secure 0 0 0) 0x7ff5d005a150 con 0x7ff5d815c390 2024-04-04T06:17:49.347 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:49.347 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":114,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5732,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1498324773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1498324773},{"type":"v1","addr":"172.21.15.178:6837","nonce":1498324773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2455031662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":2455031662},{"type":"v1","addr":"172.21.15.159:6835","nonce":2455031662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6343,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1823228151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1823228151},{"type":"v1","addr":"172.21.15.178:6836","nonce":1823228151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:16:57.277344+0000","modified":"2024-04-04T06:17:40.711526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9312":{"gid":9312,"name":"a","rank":0,"incarnation":111,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6834/643543929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":643543929},{"type":"v1","addr":"172.21.15.159:6834","nonce":643543929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9312,"qdb_cluster":[9312]},"id":8}]} 2024-04-04T06:17:49.348 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 114 2024-04-04T06:17:49.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.354+0000 7ff5df400640 1 -- 172.21.15.159:0/2425982685 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff5b806c8f0 msgr2=0x7ff5b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:49.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.354+0000 7ff5df400640 1 --2- 172.21.15.159:0/2425982685 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff5b806c8f0 0x7ff5b806edb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc036000 tx=0x7ff5cc008030 comp rx=0 tx=0).stop 2024-04-04T06:17:49.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.355+0000 7ff5df400640 1 -- 172.21.15.159:0/2425982685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5d815c390 msgr2=0x7ff5d8156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:49.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.355+0000 7ff5df400640 1 --2- 172.21.15.159:0/2425982685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5d815c390 0x7ff5d8156ea0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0007e70 tx=0x7ff5d000e4b0 comp rx=0 tx=0).stop 2024-04-04T06:17:49.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.355+0000 7ff5df400640 1 -- 172.21.15.159:0/2425982685 shutdown_connections 2024-04-04T06:17:49.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.355+0000 7ff5df400640 1 --2- 172.21.15.159:0/2425982685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5d815c390 0x7ff5d8156ea0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:49.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.355+0000 7ff5df400640 1 --2- 172.21.15.159:0/2425982685 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff5b806c8f0 0x7ff5b806edb0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:49.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.355+0000 7ff5df400640 1 --2- 172.21.15.159:0/2425982685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5d80d0650 0x7ff5d8158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:49.350 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.355+0000 7ff5df400640 1 --2- 172.21.15.159:0/2425982685 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5d80af830 0x7ff5d814d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:49.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.355+0000 7ff5df400640 1 -- 172.21.15.159:0/2425982685 >> 172.21.15.159:0/2425982685 conn(0x7ff5d806b990 msgr2=0x7ff5d80afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:49.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.355+0000 7ff5df400640 1 -- 172.21.15.159:0/2425982685 shutdown_connections 2024-04-04T06:17:49.351 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.355+0000 7ff5df400640 1 -- 172.21.15.159:0/2425982685 wait complete. 2024-04-04T06:17:49.368 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:17:49.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.572+0000 7fadb7600640 1 Processor -- start 2024-04-04T06:17:49.568 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.573+0000 7fadb7600640 1 -- start start 2024-04-04T06:17:49.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.574+0000 7fadb7600640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fadb00d2bf0 0x7fadb00cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:49.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.574+0000 7fadb7600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb00d3140 con 0x7fadb015fa10 2024-04-04T06:17:49.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.574+0000 7fadb7600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb00d3420 con 0x7fadb00d0c30 2024-04-04T06:17:49.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.574+0000 7fadb7600640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fadb00cf2b0 con 0x7fadb00d2bf0 2024-04-04T06:17:49.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.574+0000 7fadb4c00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fadb015fa10 0x7fadb00b55a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:42920/0 (socket says 172.21.15.159:42920) 2024-04-04T06:17:49.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.574+0000 7fadb4c00640 1 -- 172.21.15.159:0/1627682792 learned_addr learned my addr 172.21.15.159:0/1627682792 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:49.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.575+0000 7fada7e00640 1 --2- 172.21.15.159:0/1627682792 >> v2:172.21.15.178:3300/0 conn(0x7fadb00d2bf0 0x7fadb00cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:49.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.575+0000 7fadafe00640 1 -- 172.21.15.159:0/1627682792 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2400543952 0 0) 0x7fadb00d3140 con 0x7fadb015fa10 2024-04-04T06:17:49.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.575+0000 7fadafe00640 1 -- 172.21.15.159:0/1627682792 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad94003520 con 0x7fadb015fa10 2024-04-04T06:17:49.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.575+0000 7fadafe00640 1 -- 172.21.15.159:0/1627682792 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4047967977 0 0) 0x7fad94003520 con 0x7fadb015fa10 2024-04-04T06:17:49.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.575+0000 7fadafe00640 1 -- 172.21.15.159:0/1627682792 >> v2:172.21.15.178:3300/0 conn(0x7fadb00d2bf0 msgr2=0x7fadb00cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:49.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.575+0000 7fadafe00640 1 --2- 172.21.15.159:0/1627682792 >> v2:172.21.15.178:3300/0 conn(0x7fadb00d2bf0 0x7fadb00cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:49.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.575+0000 7fadafe00640 1 -- 172.21.15.159:0/1627682792 >> v1:172.21.15.178:6789/0 conn(0x7fadb00d0c30 legacy=0x7fadb01603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:49.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.575+0000 7fadafe00640 1 -- 172.21.15.159:0/1627682792 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb00cf590 con 0x7fadb015fa10 2024-04-04T06:17:49.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.575+0000 7fadafe00640 1 -- 172.21.15.159:0/1627682792 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fada00019f0 con 0x7fadb015fa10 2024-04-04T06:17:49.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.575+0000 7fadafe00640 1 -- 172.21.15.159:0/1627682792 >> v1:172.21.15.159:6789/0 conn(0x7fadb015fa10 legacy=0x7fadb00b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:49.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.575+0000 7fadafe00640 1 --2- 172.21.15.159:0/1627682792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad940039b0 0x7fad94003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:49.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.576+0000 7fadafe00640 1 --2- 172.21.15.159:0/1627682792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad94004800 0x7fad94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:49.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.576+0000 7fadafe00640 1 --2- 172.21.15.159:0/1627682792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad940051c0 0x7fad9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:49.572 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.576+0000 7fadafe00640 1 -- 172.21.15.159:0/1627682792 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fadb00cf2b0 con 0x7fad94004800 2024-04-04T06:17:49.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.576+0000 7fadafe00640 1 -- 172.21.15.159:0/1627682792 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fad9400c070 con 0x7fad940039b0 2024-04-04T06:17:49.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.576+0000 7fadafe00640 1 -- 172.21.15.159:0/1627682792 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fad9400c270 con 0x7fad940051c0 2024-04-04T06:17:49.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.576+0000 7fada7e00640 1 --2- 172.21.15.159:0/1627682792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad94004800 0x7fad94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:49.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.576+0000 7fadb4c00640 1 --2- 172.21.15.159:0/1627682792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad940039b0 0x7fad94003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:49.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.576+0000 7fadb5600640 1 --2- 172.21.15.159:0/1627682792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad940051c0 0x7fad9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:49.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.576+0000 7fada7e00640 1 -- 172.21.15.159:0/1627682792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad940051c0 msgr2=0x7fad9400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:49.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.576+0000 7fada7e00640 1 --2- 172.21.15.159:0/1627682792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad940051c0 0x7fad9400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:49.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.576+0000 7fada7e00640 1 -- 172.21.15.159:0/1627682792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad940039b0 msgr2=0x7fad94003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:49.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.576+0000 7fada7e00640 1 --2- 172.21.15.159:0/1627682792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad940039b0 0x7fad94003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:49.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.576+0000 7fada7e00640 1 -- 172.21.15.159:0/1627682792 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad94003520 con 0x7fad94004800 2024-04-04T06:17:49.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.576+0000 7fadb5600640 1 -- 172.21.15.159:0/1627682792 reap_dead start 2024-04-04T06:17:49.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.577+0000 7fada7e00640 1 --2- 172.21.15.159:0/1627682792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad94004800 0x7fad94004c80 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fad98009ab0 tx=0x7fad94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e4be5a5987ba14 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:49.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.578+0000 7fadafe00640 1 -- 172.21.15.159:0/1627682792 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98009680 con 0x7fad94004800 2024-04-04T06:17:49.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.578+0000 7fadafe00640 1 -- 172.21.15.159:0/1627682792 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad98005990 con 0x7fad94004800 2024-04-04T06:17:49.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.578+0000 7fadafe00640 1 -- 172.21.15.159:0/1627682792 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98005d80 con 0x7fad94004800 2024-04-04T06:17:49.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.578+0000 7fadb7600640 1 -- 172.21.15.159:0/1627682792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad94004800 msgr2=0x7fad94004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:49.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.578+0000 7fadb7600640 1 --2- 172.21.15.159:0/1627682792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad94004800 0x7fad94004c80 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fad98009ab0 tx=0x7fad94005830 comp rx=0 tx=0).stop 2024-04-04T06:17:49.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.579+0000 7fadb7600640 1 -- 172.21.15.159:0/1627682792 shutdown_connections 2024-04-04T06:17:49.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.579+0000 7fadb7600640 1 --2- 172.21.15.159:0/1627682792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad94004800 0x7fad94004c80 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:49.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.579+0000 7fadb7600640 1 -- 172.21.15.159:0/1627682792 >> 172.21.15.159:0/1627682792 conn(0x7fadb008bfb0 msgr2=0x7fadb00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:49.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.579+0000 7fadb7600640 1 -- 172.21.15.159:0/1627682792 shutdown_connections 2024-04-04T06:17:49.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.579+0000 7fadb7600640 1 -- 172.21.15.159:0/1627682792 wait complete. 2024-04-04T06:17:49.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.581+0000 7fadb7600640 1 Processor -- start 2024-04-04T06:17:49.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.582+0000 7fadb7600640 1 -- start start 2024-04-04T06:17:49.578 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.583+0000 7fadb7600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad94004800 0x7fadb01ee550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:49.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.583+0000 7fadb7600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadb0158430 0x7fadb01588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:49.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.583+0000 7fadb7600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fadb015d850 0x7fadb015dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:49.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.583+0000 7fadb7600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fadb015ff70 con 0x7fadb015d850 2024-04-04T06:17:49.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.583+0000 7fadb7600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fadb015fdf0 con 0x7fadb0158430 2024-04-04T06:17:49.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.583+0000 7fadb7600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fadb00d10a0 con 0x7fad94004800 2024-04-04T06:17:49.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.583+0000 7fadb5600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fadb015d850 0x7fadb015dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:49.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.583+0000 7fadb5600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fadb015d850 0x7fadb015dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45958/0 (socket says 172.21.15.159:45958) 2024-04-04T06:17:49.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.583+0000 7fadb5600640 1 -- 172.21.15.159:0/4132656252 learned_addr learned my addr 172.21.15.159:0/4132656252 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:49.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.583+0000 7fada7e00640 1 --2- 172.21.15.159:0/4132656252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadb0158430 0x7fadb01588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:49.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.583+0000 7fadb4c00640 1 --2- 172.21.15.159:0/4132656252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad94004800 0x7fadb01ee550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:49.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.583+0000 7fadb5600640 1 -- 172.21.15.159:0/4132656252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad94004800 msgr2=0x7fadb01ee550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:49.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.583+0000 7fadb5600640 1 --2- 172.21.15.159:0/4132656252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad94004800 0x7fadb01ee550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:49.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.583+0000 7fadb5600640 1 -- 172.21.15.159:0/4132656252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadb0158430 msgr2=0x7fadb01588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:49.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.583+0000 7fadb5600640 1 --2- 172.21.15.159:0/4132656252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadb0158430 0x7fadb01588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:49.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.584+0000 7fadb5600640 1 -- 172.21.15.159:0/4132656252 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb015e640 con 0x7fadb015d850 2024-04-04T06:17:49.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.584+0000 7fadb5600640 1 --2- 172.21.15.159:0/4132656252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fadb015d850 0x7fadb015dcd0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fadb00d4390 tx=0x7fada800d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:49.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.584+0000 7fadad600640 1 -- 172.21.15.159:0/4132656252 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8018070 con 0x7fadb015d850 2024-04-04T06:17:49.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.584+0000 7fadad600640 1 -- 172.21.15.159:0/4132656252 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada80052d0 con 0x7fadb015d850 2024-04-04T06:17:49.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.584+0000 7fadad600640 1 -- 172.21.15.159:0/4132656252 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8003810 con 0x7fadb015d850 2024-04-04T06:17:49.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.584+0000 7fadb7600640 1 -- 172.21.15.159:0/4132656252 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb0158fa0 con 0x7fadb015d850 2024-04-04T06:17:49.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.584+0000 7fadb7600640 1 -- 172.21.15.159:0/4132656252 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb0159560 con 0x7fadb015d850 2024-04-04T06:17:49.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.586+0000 7fadb7600640 1 -- 172.21.15.159:0/4132656252 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb0068cd0 con 0x7fadb015d850 2024-04-04T06:17:49.582 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.587+0000 7fadad600640 1 -- 172.21.15.159:0/4132656252 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fada80039b0 con 0x7fadb015d850 2024-04-04T06:17:49.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.588+0000 7fadad600640 1 --2- 172.21.15.159:0/4132656252 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fad8c06c920 0x7fad8c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:49.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.589+0000 7fadad600640 1 -- 172.21.15.159:0/4132656252 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7fada808b0e0 con 0x7fadb015d850 2024-04-04T06:17:49.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.589+0000 7fadb4c00640 1 --2- 172.21.15.159:0/4132656252 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fad8c06c920 0x7fad8c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:49.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.590+0000 7fadb4c00640 1 --2- 172.21.15.159:0/4132656252 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fad8c06c920 0x7fad8c06ede0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fada0002050 tx=0x7fada0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:49.588 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.593+0000 7fadad600640 1 -- 172.21.15.159:0/4132656252 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fada8059420 con 0x7fadb015d850 2024-04-04T06:17:49.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.796+0000 7fadb7600640 1 -- 172.21.15.159:0/4132656252 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fadb00ba3d0 con 0x7fadb015d850 2024-04-04T06:17:49.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.797+0000 7fadad600640 1 -- 172.21.15.159:0/4132656252 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+16995 (secure 0 0 0) 0x7fada8058d20 con 0x7fadb015d850 2024-04-04T06:17:49.793 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:49.793 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":116,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:17:35.667542+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:16:55.036975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-04T06:16:56.239022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.b","mappings":[{"from":0,"to":5}]},{"pgid":"17.26","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6834/2439261212":"2024-04-05T06:17:35.667507+0000","172.21.15.159:6832/2439261212":"2024-04-05T06:17:35.667507+0000","172.21.15.159:6834/1017894858":"2024-04-05T06:17:21.596764+0000","172.21.15.159:6832/1017894858":"2024-04-05T06:17:21.596764+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:17:49.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.800+0000 7fadb7600640 1 -- 172.21.15.159:0/4132656252 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fad8c06c920 msgr2=0x7fad8c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:49.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.800+0000 7fadb7600640 1 --2- 172.21.15.159:0/4132656252 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fad8c06c920 0x7fad8c06ede0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fada0002050 tx=0x7fada0009040 comp rx=0 tx=0).stop 2024-04-04T06:17:49.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.800+0000 7fadb7600640 1 -- 172.21.15.159:0/4132656252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fadb015d850 msgr2=0x7fadb015dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:49.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.800+0000 7fadb7600640 1 --2- 172.21.15.159:0/4132656252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fadb015d850 0x7fadb015dcd0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fadb00d4390 tx=0x7fada800d890 comp rx=0 tx=0).stop 2024-04-04T06:17:49.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.800+0000 7fadb7600640 1 -- 172.21.15.159:0/4132656252 shutdown_connections 2024-04-04T06:17:49.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.800+0000 7fadb7600640 1 --2- 172.21.15.159:0/4132656252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fadb015d850 0x7fadb015dcd0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:49.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.800+0000 7fadb7600640 1 --2- 172.21.15.159:0/4132656252 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fad8c06c920 0x7fad8c06ede0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:49.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.800+0000 7fadb7600640 1 --2- 172.21.15.159:0/4132656252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadb0158430 0x7fadb01588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:49.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.800+0000 7fadb7600640 1 --2- 172.21.15.159:0/4132656252 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad94004800 0x7fadb01ee550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:49.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.800+0000 7fadb7600640 1 -- 172.21.15.159:0/4132656252 >> 172.21.15.159:0/4132656252 conn(0x7fadb008bfb0 msgr2=0x7fadb006c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:49.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.800+0000 7fadb7600640 1 -- 172.21.15.159:0/4132656252 shutdown_connections 2024-04-04T06:17:49.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:49.800+0000 7fadb7600640 1 -- 172.21.15.159:0/4132656252 wait complete. 2024-04-04T06:17:49.815 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-04T06:05:40.750131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-04T06:16:55.036975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-04T06:16:56.239022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-04T06:17:49.815 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-04T06:17:50.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.025+0000 7fd317a00640 1 Processor -- start 2024-04-04T06:17:50.020 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.025+0000 7fd317a00640 1 -- start start 2024-04-04T06:17:50.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.027+0000 7fd317a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd310160a30 0x7fd31006b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:50.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.027+0000 7fd317a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd310145d60 0x7fd31006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:50.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.027+0000 7fd317a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3100b5ed0 con 0x7fd3101457e0 2024-04-04T06:17:50.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.027+0000 7fd317a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fd310145bc0 con 0x7fd310160a30 2024-04-04T06:17:50.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.027+0000 7fd317a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd3100afca0 con 0x7fd310145d60 2024-04-04T06:17:50.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.027+0000 7fd307e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd310160a30 0x7fd31006b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:50.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.027+0000 7fd307e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fd310160a30 0x7fd31006b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45972/0 (socket says 172.21.15.159:45972) 2024-04-04T06:17:50.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.027+0000 7fd307e00640 1 -- 172.21.15.159:0/4078808880 learned_addr learned my addr 172.21.15.159:0/4078808880 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:50.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.027+0000 7fd315a00640 1 --2- 172.21.15.159:0/4078808880 >> v2:172.21.15.178:3300/0 conn(0x7fd310145d60 0x7fd31006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:50.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.027+0000 7fd307e00640 1 -- 172.21.15.159:0/4078808880 >> v2:172.21.15.178:3300/0 conn(0x7fd310145d60 msgr2=0x7fd31006bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:50.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.027+0000 7fd307e00640 1 --2- 172.21.15.159:0/4078808880 >> v2:172.21.15.178:3300/0 conn(0x7fd310145d60 0x7fd31006bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.028+0000 7fd307e00640 1 -- 172.21.15.159:0/4078808880 >> v1:172.21.15.159:6789/0 conn(0x7fd3101457e0 legacy=0x7fd3100b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:50.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.028+0000 7fd307e00640 1 -- 172.21.15.159:0/4078808880 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3100b61b0 con 0x7fd310160a30 2024-04-04T06:17:50.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.028+0000 7fd30fe00640 1 -- 172.21.15.159:0/4078808880 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1980408928 0 0) 0x7fd3100b5ed0 con 0x7fd3101457e0 2024-04-04T06:17:50.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.029+0000 7fd307e00640 1 --2- 172.21.15.159:0/4078808880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd310160a30 0x7fd31006b570 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fd310068120 tx=0x7fd2f802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846170e5a0f2662c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:50.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.029+0000 7fd30fe00640 1 -- 172.21.15.159:0/4078808880 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8035020 con 0x7fd310160a30 2024-04-04T06:17:50.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.029+0000 7fd30fe00640 1 -- 172.21.15.159:0/4078808880 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2f8031870 con 0x7fd310160a30 2024-04-04T06:17:50.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.029+0000 7fd30fe00640 1 -- 172.21.15.159:0/4078808880 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8031c40 con 0x7fd310160a30 2024-04-04T06:17:50.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.029+0000 7fd317a00640 1 -- 172.21.15.159:0/4078808880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd310160a30 msgr2=0x7fd31006b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:50.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.029+0000 7fd317a00640 1 --2- 172.21.15.159:0/4078808880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd310160a30 0x7fd31006b570 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fd310068120 tx=0x7fd2f802b910 comp rx=0 tx=0).stop 2024-04-04T06:17:50.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.030+0000 7fd317a00640 1 -- 172.21.15.159:0/4078808880 shutdown_connections 2024-04-04T06:17:50.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.030+0000 7fd317a00640 1 --2- 172.21.15.159:0/4078808880 >> v2:172.21.15.178:3300/0 conn(0x7fd310145d60 0x7fd31006bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.030+0000 7fd317a00640 1 --2- 172.21.15.159:0/4078808880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd310160a30 0x7fd31006b570 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.030+0000 7fd317a00640 1 -- 172.21.15.159:0/4078808880 >> 172.21.15.159:0/4078808880 conn(0x7fd3100b65a0 msgr2=0x7fd3100b69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:50.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.030+0000 7fd317a00640 1 -- 172.21.15.159:0/4078808880 shutdown_connections 2024-04-04T06:17:50.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.030+0000 7fd317a00640 1 -- 172.21.15.159:0/4078808880 wait complete. 2024-04-04T06:17:50.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.031+0000 7fd317a00640 1 Processor -- start 2024-04-04T06:17:50.027 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.032+0000 7fd317a00640 1 -- start start 2024-04-04T06:17:50.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.033+0000 7fd317a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3101457e0 0x7fd3101f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:50.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.033+0000 7fd317a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd310145d60 0x7fd3101f7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:50.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.033+0000 7fd317a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd310160a30 0x7fd310132540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:50.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.033+0000 7fd317a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd3100af9f0 con 0x7fd310145d60 2024-04-04T06:17:50.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.033+0000 7fd317a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3100af870 con 0x7fd310160a30 2024-04-04T06:17:50.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.033+0000 7fd317a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd3100ae1e0 con 0x7fd3101457e0 2024-04-04T06:17:50.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.033+0000 7fd307e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd310145d60 0x7fd3101f7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:50.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.033+0000 7fd307e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd310145d60 0x7fd3101f7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45974/0 (socket says 172.21.15.159:45974) 2024-04-04T06:17:50.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.033+0000 7fd307e00640 1 -- 172.21.15.159:0/3018391863 learned_addr learned my addr 172.21.15.159:0/3018391863 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:50.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.033+0000 7fd315000640 1 --2- 172.21.15.159:0/3018391863 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3101457e0 0x7fd3101f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:50.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.033+0000 7fd315a00640 1 --2- 172.21.15.159:0/3018391863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd310160a30 0x7fd310132540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:50.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.033+0000 7fd307e00640 1 -- 172.21.15.159:0/3018391863 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3101457e0 msgr2=0x7fd3101f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:50.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.033+0000 7fd307e00640 1 --2- 172.21.15.159:0/3018391863 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3101457e0 0x7fd3101f4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.033+0000 7fd307e00640 1 -- 172.21.15.159:0/3018391863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd310160a30 msgr2=0x7fd310132540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:50.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.033+0000 7fd307e00640 1 --2- 172.21.15.159:0/3018391863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd310160a30 0x7fd310132540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.033+0000 7fd307e00640 1 -- 172.21.15.159:0/3018391863 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd310133000 con 0x7fd310145d60 2024-04-04T06:17:50.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.034+0000 7fd307e00640 1 --2- 172.21.15.159:0/3018391863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd310145d60 0x7fd3101f7720 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8000c00 tx=0x7fd2f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:50.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.034+0000 7fd30d600640 1 -- 172.21.15.159:0/3018391863 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8040070 con 0x7fd310145d60 2024-04-04T06:17:50.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.034+0000 7fd30d600640 1 -- 172.21.15.159:0/3018391863 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2f8030d00 con 0x7fd310145d60 2024-04-04T06:17:50.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.034+0000 7fd30d600640 1 -- 172.21.15.159:0/3018391863 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8035040 con 0x7fd310145d60 2024-04-04T06:17:50.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.034+0000 7fd317a00640 1 -- 172.21.15.159:0/3018391863 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3101332e0 con 0x7fd310145d60 2024-04-04T06:17:50.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.034+0000 7fd317a00640 1 -- 172.21.15.159:0/3018391863 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3101338a0 con 0x7fd310145d60 2024-04-04T06:17:50.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.036+0000 7fd306000640 1 -- 172.21.15.159:0/3018391863 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd310068cd0 con 0x7fd310145d60 2024-04-04T06:17:50.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.037+0000 7fd30d600640 1 -- 172.21.15.159:0/3018391863 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2f8038020 con 0x7fd310145d60 2024-04-04T06:17:50.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.038+0000 7fd30d600640 1 --2- 172.21.15.159:0/3018391863 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd2ec06c8f0 0x7fd2ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:50.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.038+0000 7fd30d600640 1 -- 172.21.15.159:0/3018391863 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7fd2f80adb00 con 0x7fd310145d60 2024-04-04T06:17:50.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.039+0000 7fd315000640 1 --2- 172.21.15.159:0/3018391863 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd2ec06c8f0 0x7fd2ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:50.034 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.040+0000 7fd315000640 1 --2- 172.21.15.159:0/3018391863 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd2ec06c8f0 0x7fd2ec06edb0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd300000f60 tx=0x7fd300009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:50.037 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.042+0000 7fd30d600640 1 -- 172.21.15.159:0/3018391863 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fd2f807be30 con 0x7fd310145d60 2024-04-04T06:17:50.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.242+0000 7fd306000640 1 -- 172.21.15.159:0/3018391863 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd31006bdd0 con 0x7fd310145d60 2024-04-04T06:17:50.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.242+0000 7fd30d600640 1 -- 172.21.15.159:0/3018391863 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) ==== 93+0+10 (secure 0 0 0) 0x7fd2f80ade20 con 0x7fd310145d60 2024-04-04T06:17:50.238 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 1 2024-04-04T06:17:50.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.245+0000 7fd306000640 1 -- 172.21.15.159:0/3018391863 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd2ec06c8f0 msgr2=0x7fd2ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:50.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.245+0000 7fd306000640 1 --2- 172.21.15.159:0/3018391863 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd2ec06c8f0 0x7fd2ec06edb0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd300000f60 tx=0x7fd300009040 comp rx=0 tx=0).stop 2024-04-04T06:17:50.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.245+0000 7fd306000640 1 -- 172.21.15.159:0/3018391863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd310145d60 msgr2=0x7fd3101f7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:50.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.245+0000 7fd306000640 1 --2- 172.21.15.159:0/3018391863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd310145d60 0x7fd3101f7720 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8000c00 tx=0x7fd2f80046c0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.245+0000 7fd306000640 1 -- 172.21.15.159:0/3018391863 shutdown_connections 2024-04-04T06:17:50.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.245+0000 7fd306000640 1 --2- 172.21.15.159:0/3018391863 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd2ec06c8f0 0x7fd2ec06edb0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.245+0000 7fd306000640 1 --2- 172.21.15.159:0/3018391863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd310160a30 0x7fd310132540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.245+0000 7fd306000640 1 --2- 172.21.15.159:0/3018391863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd310145d60 0x7fd3101f7720 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.245+0000 7fd306000640 1 --2- 172.21.15.159:0/3018391863 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3101457e0 0x7fd3101f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.245+0000 7fd306000640 1 -- 172.21.15.159:0/3018391863 >> 172.21.15.159:0/3018391863 conn(0x7fd3100b65a0 msgr2=0x7fd3100b0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:50.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.245+0000 7fd306000640 1 -- 172.21.15.159:0/3018391863 shutdown_connections 2024-04-04T06:17:50.241 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.245+0000 7fd306000640 1 -- 172.21.15.159:0/3018391863 wait complete. 2024-04-04T06:17:50.258 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-04T06:17:50.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.471+0000 7fdae4e00640 1 Processor -- start 2024-04-04T06:17:50.466 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.471+0000 7fdae4e00640 1 -- start start 2024-04-04T06:17:50.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.472+0000 7fdae4e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdae0161f60 0x7fdae00d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:50.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.472+0000 7fdae4e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fdae00d13b0 0x7fdae00d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:50.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.472+0000 7fdae4e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdae00b5ce0 con 0x7fdae00b3170 2024-04-04T06:17:50.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.472+0000 7fdae4e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fdae00d0620 con 0x7fdae00d13b0 2024-04-04T06:17:50.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.472+0000 7fdae4e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdae00d27f0 con 0x7fdae0161f60 2024-04-04T06:17:50.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.473+0000 7fdade800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fdae00d13b0 0x7fdae00d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:50.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.473+0000 7fdade800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fdae00d13b0 0x7fdae00d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45978/0 (socket says 172.21.15.159:45978) 2024-04-04T06:17:50.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.473+0000 7fdade800640 1 -- 172.21.15.159:0/2378370396 learned_addr learned my addr 172.21.15.159:0/2378370396 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:50.468 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.473+0000 7fdadde00640 1 --2- 172.21.15.159:0/2378370396 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdae0161f60 0x7fdae00d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:50.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.474+0000 7fdade800640 1 -- 172.21.15.159:0/2378370396 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdae0161f60 msgr2=0x7fdae00d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:50.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.474+0000 7fdade800640 1 --2- 172.21.15.159:0/2378370396 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdae0161f60 0x7fdae00d0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.474+0000 7fdade800640 1 -- 172.21.15.159:0/2378370396 >> v1:172.21.15.178:6789/0 conn(0x7fdae00b3170 legacy=0x7fdae00b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:50.469 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.474+0000 7fdade800640 1 -- 172.21.15.159:0/2378370396 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae00d2a90 con 0x7fdae00d13b0 2024-04-04T06:17:50.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.475+0000 7fdade800640 1 --2- 172.21.15.159:0/2378370396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdae00d13b0 0x7fdae00d00e0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009e50 tx=0x7fdad802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347b43ce3f54fe53 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:50.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.475+0000 7fdadca00640 1 -- 172.21.15.159:0/2378370396 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad802cb00 con 0x7fdae00d13b0 2024-04-04T06:17:50.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.475+0000 7fdadca00640 1 -- 172.21.15.159:0/2378370396 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdad802cca0 con 0x7fdae00d13b0 2024-04-04T06:17:50.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.475+0000 7fdadca00640 1 -- 172.21.15.159:0/2378370396 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad8034610 con 0x7fdae00d13b0 2024-04-04T06:17:50.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.476+0000 7fdae4e00640 1 -- 172.21.15.159:0/2378370396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdae00d13b0 msgr2=0x7fdae00d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:50.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.476+0000 7fdae4e00640 1 --2- 172.21.15.159:0/2378370396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdae00d13b0 0x7fdae00d00e0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009e50 tx=0x7fdad802bf60 comp rx=0 tx=0).stop 2024-04-04T06:17:50.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.476+0000 7fdae4e00640 1 -- 172.21.15.159:0/2378370396 shutdown_connections 2024-04-04T06:17:50.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.476+0000 7fdae4e00640 1 --2- 172.21.15.159:0/2378370396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdae00d13b0 0x7fdae00d00e0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.476+0000 7fdae4e00640 1 --2- 172.21.15.159:0/2378370396 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdae0161f60 0x7fdae00d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.476+0000 7fdae4e00640 1 -- 172.21.15.159:0/2378370396 >> 172.21.15.159:0/2378370396 conn(0x7fdae008bfb0 msgr2=0x7fdae00d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:50.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.476+0000 7fdae4e00640 1 -- 172.21.15.159:0/2378370396 shutdown_connections 2024-04-04T06:17:50.471 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.476+0000 7fdae4e00640 1 -- 172.21.15.159:0/2378370396 wait complete. 2024-04-04T06:17:50.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.478+0000 7fdae4e00640 1 Processor -- start 2024-04-04T06:17:50.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.478+0000 7fdae4e00640 1 -- start start 2024-04-04T06:17:50.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.479+0000 7fdae4e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdae00b3170 0x7fdae00b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:50.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.479+0000 7fdae4e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdae00d13b0 0x7fdae00c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:50.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.479+0000 7fdae4e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdae0161f60 0x7fdae00c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:50.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.479+0000 7fdae4e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdae00d3030 con 0x7fdae00b3170 2024-04-04T06:17:50.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.479+0000 7fdae4e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdae00d2d50 con 0x7fdae0161f60 2024-04-04T06:17:50.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.479+0000 7fdae4e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fdae00d3310 con 0x7fdae00d13b0 2024-04-04T06:17:50.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.479+0000 7fdadde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdae00b3170 0x7fdae00b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:50.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.479+0000 7fdadde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdae00b3170 0x7fdae00b2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45992/0 (socket says 172.21.15.159:45992) 2024-04-04T06:17:50.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.479+0000 7fdadde00640 1 -- 172.21.15.159:0/2269998076 learned_addr learned my addr 172.21.15.159:0/2269998076 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:50.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.480+0000 7fdade800640 1 --2- 172.21.15.159:0/2269998076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdae0161f60 0x7fdae00c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:50.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.480+0000 7fdadd400640 1 --2- 172.21.15.159:0/2269998076 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdae00d13b0 0x7fdae00c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:50.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.480+0000 7fdadde00640 1 -- 172.21.15.159:0/2269998076 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdae00d13b0 msgr2=0x7fdae00c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:50.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.480+0000 7fdadde00640 1 --2- 172.21.15.159:0/2269998076 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdae00d13b0 0x7fdae00c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.476 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.480+0000 7fdadde00640 1 -- 172.21.15.159:0/2269998076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdae0161f60 msgr2=0x7fdae00c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:50.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.480+0000 7fdadde00640 1 --2- 172.21.15.159:0/2269998076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdae0161f60 0x7fdae00c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.480+0000 7fdadde00640 1 -- 172.21.15.159:0/2269998076 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae00c8190 con 0x7fdae00b3170 2024-04-04T06:17:50.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.480+0000 7fdade800640 1 --2- 172.21.15.159:0/2269998076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdae0161f60 0x7fdae00c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:50.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.480+0000 7fdadd400640 1 --2- 172.21.15.159:0/2269998076 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdae00d13b0 0x7fdae00c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:50.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.480+0000 7fdadde00640 1 --2- 172.21.15.159:0/2269998076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdae00b3170 0x7fdae00b2310 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fdad000e9b0 tx=0x7fdad000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:50.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.481+0000 7fdad6000640 1 -- 172.21.15.159:0/2269998076 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0017070 con 0x7fdae00b3170 2024-04-04T06:17:50.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.481+0000 7fdad6000640 1 -- 172.21.15.159:0/2269998076 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdad000c9a0 con 0x7fdae00b3170 2024-04-04T06:17:50.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.481+0000 7fdad6000640 1 -- 172.21.15.159:0/2269998076 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad000cd90 con 0x7fdae00b3170 2024-04-04T06:17:50.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.481+0000 7fdae4e00640 1 -- 172.21.15.159:0/2269998076 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae00c8470 con 0x7fdae00b3170 2024-04-04T06:17:50.477 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.481+0000 7fdae4e00640 1 -- 172.21.15.159:0/2269998076 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae00c8a30 con 0x7fdae00b3170 2024-04-04T06:17:50.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.482+0000 7fdae4e00640 1 -- 172.21.15.159:0/2269998076 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdae00685c0 con 0x7fdae00b3170 2024-04-04T06:17:50.478 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.483+0000 7fdad6000640 1 -- 172.21.15.159:0/2269998076 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdad001a070 con 0x7fdae00b3170 2024-04-04T06:17:50.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.484+0000 7fdad6000640 1 --2- 172.21.15.159:0/2269998076 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdac006c8f0 0x7fdac006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:50.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.484+0000 7fdad6000640 1 -- 172.21.15.159:0/2269998076 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7fdad008c010 con 0x7fdae00b3170 2024-04-04T06:17:50.479 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.484+0000 7fdadd400640 1 --2- 172.21.15.159:0/2269998076 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdac006c8f0 0x7fdac006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:50.480 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.486+0000 7fdadd400640 1 --2- 172.21.15.159:0/2269998076 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdac006c8f0 0x7fdac006edb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fdae00b3140 tx=0x7fdacc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:50.483 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.489+0000 7fdad6000640 1 -- 172.21.15.159:0/2269998076 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fdad005a350 con 0x7fdae00b3170 2024-04-04T06:17:50.684 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.688+0000 7fdae4e00640 1 -- 172.21.15.159:0/2269998076 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdae00cf1a0 con 0x7fdae00b3170 2024-04-04T06:17:50.685 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.689+0000 7fdad6000640 1 -- 172.21.15.159:0/2269998076 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) ==== 104+0+11 (secure 0 0 0) 0x7fdad0059c50 con 0x7fdae00b3170 2024-04-04T06:17:50.685 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 32 2024-04-04T06:17:50.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.693+0000 7fdae4e00640 1 -- 172.21.15.159:0/2269998076 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdac006c8f0 msgr2=0x7fdac006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:50.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.693+0000 7fdae4e00640 1 --2- 172.21.15.159:0/2269998076 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdac006c8f0 0x7fdac006edb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fdae00b3140 tx=0x7fdacc009040 comp rx=0 tx=0).stop 2024-04-04T06:17:50.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.693+0000 7fdae4e00640 1 -- 172.21.15.159:0/2269998076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdae00b3170 msgr2=0x7fdae00b2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:50.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.693+0000 7fdae4e00640 1 --2- 172.21.15.159:0/2269998076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdae00b3170 0x7fdae00b2310 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fdad000e9b0 tx=0x7fdad000ee70 comp rx=0 tx=0).stop 2024-04-04T06:17:50.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.693+0000 7fdae4e00640 1 -- 172.21.15.159:0/2269998076 shutdown_connections 2024-04-04T06:17:50.688 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.693+0000 7fdae4e00640 1 --2- 172.21.15.159:0/2269998076 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fdac006c8f0 0x7fdac006edb0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.693+0000 7fdae4e00640 1 --2- 172.21.15.159:0/2269998076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdae0161f60 0x7fdae00c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.693+0000 7fdae4e00640 1 --2- 172.21.15.159:0/2269998076 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdae00d13b0 0x7fdae00c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.693+0000 7fdae4e00640 1 --2- 172.21.15.159:0/2269998076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdae00b3170 0x7fdae00b2310 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.693+0000 7fdae4e00640 1 -- 172.21.15.159:0/2269998076 >> 172.21.15.159:0/2269998076 conn(0x7fdae008bfb0 msgr2=0x7fdae0160560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:50.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.693+0000 7fdae4e00640 1 -- 172.21.15.159:0/2269998076 shutdown_connections 2024-04-04T06:17:50.689 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.693+0000 7fdae4e00640 1 -- 172.21.15.159:0/2269998076 wait complete. 2024-04-04T06:17:50.715 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-04T06:17:50.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.929+0000 7f4f09800640 1 Processor -- start 2024-04-04T06:17:50.924 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.929+0000 7f4f09800640 1 -- start start 2024-04-04T06:17:50.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.931+0000 7f4f09800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f041590c0 0x7f4f0415f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:50.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.931+0000 7f4f09800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4f0406c580 0x7f4f0406ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:50.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.931+0000 7f4f09800640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4f04162440 0x7f4f041628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:50.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.931+0000 7f4f09800640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f4f0406e5b0 con 0x7f4f04162440 2024-04-04T06:17:50.927 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.931+0000 7f4f09800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f4f0415fe10 con 0x7f4f0406c580 2024-04-04T06:17:50.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.931+0000 7f4f09800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4f04067900 con 0x7f4f041590c0 2024-04-04T06:17:50.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.931+0000 7f4f03400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4f04162440 0x7f4f041628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:50.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.931+0000 7f4f03400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4f04162440 0x7f4f041628a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:46008/0 (socket says 172.21.15.159:46008) 2024-04-04T06:17:50.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.931+0000 7f4f03400640 1 -- 172.21.15.159:0/2837546236 learned_addr learned my addr 172.21.15.159:0/2837546236 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:50.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.932+0000 7f4f02a00640 1 --2- 172.21.15.159:0/2837546236 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f041590c0 0x7f4f0415f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:50.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.932+0000 7f4f02000640 1 --2- 172.21.15.159:0/2837546236 >> v2:172.21.15.178:3300/0 conn(0x7f4f0406c580 0x7f4f0406ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:50.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.932+0000 7f4f03400640 1 -- 172.21.15.159:0/2837546236 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f041590c0 msgr2=0x7f4f0415f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:50.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.932+0000 7f4f03400640 1 --2- 172.21.15.159:0/2837546236 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f041590c0 0x7f4f0415f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.932+0000 7f4f03400640 1 -- 172.21.15.159:0/2837546236 >> v2:172.21.15.178:3300/0 conn(0x7f4f0406c580 msgr2=0x7f4f0406ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:50.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.932+0000 7f4f03400640 1 --2- 172.21.15.159:0/2837546236 >> v2:172.21.15.178:3300/0 conn(0x7f4f0406c580 0x7f4f0406ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.928 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.932+0000 7f4f03400640 1 -- 172.21.15.159:0/2837546236 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f040b47d0 con 0x7f4f04162440 2024-04-04T06:17:50.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.934+0000 7f4f03400640 1 --2- 172.21.15.159:0/2837546236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f04162440 0x7f4f041628a0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f4f04070a10 tx=0x7f4efc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210eab7d0fff2e57 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:50.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.934+0000 7f4f01600640 1 -- 172.21.15.159:0/2837546236 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc02ca70 con 0x7f4f04162440 2024-04-04T06:17:50.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.934+0000 7f4f01600640 1 -- 172.21.15.159:0/2837546236 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4efc02cc10 con 0x7f4f04162440 2024-04-04T06:17:50.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.934+0000 7f4f01600640 1 -- 172.21.15.159:0/2837546236 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc0343f0 con 0x7f4f04162440 2024-04-04T06:17:50.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.935+0000 7f4f09800640 1 -- 172.21.15.159:0/2837546236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f04162440 msgr2=0x7f4f041628a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:50.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.935+0000 7f4f09800640 1 --2- 172.21.15.159:0/2837546236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f04162440 0x7f4f041628a0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f4f04070a10 tx=0x7f4efc02bed0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.935+0000 7f4f09800640 1 -- 172.21.15.159:0/2837546236 shutdown_connections 2024-04-04T06:17:50.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.935+0000 7f4f09800640 1 --2- 172.21.15.159:0/2837546236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f04162440 0x7f4f041628a0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.935+0000 7f4f09800640 1 --2- 172.21.15.159:0/2837546236 >> v2:172.21.15.178:3300/0 conn(0x7f4f0406c580 0x7f4f0406ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.935+0000 7f4f09800640 1 --2- 172.21.15.159:0/2837546236 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f041590c0 0x7f4f0415f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.935+0000 7f4f09800640 1 -- 172.21.15.159:0/2837546236 >> 172.21.15.159:0/2837546236 conn(0x7f4f04071e60 msgr2=0x7f4f04072270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:50.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.935+0000 7f4f09800640 1 -- 172.21.15.159:0/2837546236 shutdown_connections 2024-04-04T06:17:50.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.935+0000 7f4f09800640 1 -- 172.21.15.159:0/2837546236 wait complete. 2024-04-04T06:17:50.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.937+0000 7f4f09800640 1 Processor -- start 2024-04-04T06:17:50.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.937+0000 7f4f09800640 1 -- start start 2024-04-04T06:17:50.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.938+0000 7f4f09800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f0406c580 0x7f4f041f8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:50.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.938+0000 7f4f09800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f041590c0 0x7f4f0414b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:50.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.938+0000 7f4f09800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f04162440 0x7f4f04157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:50.935 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.938+0000 7f4f09800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0406e240 con 0x7f4f0406c580 2024-04-04T06:17:50.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.938+0000 7f4f09800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0406df60 con 0x7f4f041590c0 2024-04-04T06:17:50.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.938+0000 7f4f09800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4f040b4380 con 0x7f4f04162440 2024-04-04T06:17:50.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.939+0000 7f4f02a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f0406c580 0x7f4f041f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:50.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.939+0000 7f4f02a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f0406c580 0x7f4f041f8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:46012/0 (socket says 172.21.15.159:46012) 2024-04-04T06:17:50.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.939+0000 7f4f02a00640 1 -- 172.21.15.159:0/826508797 learned_addr learned my addr 172.21.15.159:0/826508797 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:50.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.939+0000 7f4f02000640 1 --2- 172.21.15.159:0/826508797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f041590c0 0x7f4f0414b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:50.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.939+0000 7f4f03400640 1 --2- 172.21.15.159:0/826508797 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f04162440 0x7f4f04157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:50.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.939+0000 7f4f02a00640 1 -- 172.21.15.159:0/826508797 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f04162440 msgr2=0x7f4f04157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:50.936 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.939+0000 7f4f02a00640 1 --2- 172.21.15.159:0/826508797 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f04162440 0x7f4f04157020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.939+0000 7f4f02a00640 1 -- 172.21.15.159:0/826508797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f041590c0 msgr2=0x7f4f0414b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:50.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.939+0000 7f4f02a00640 1 --2- 172.21.15.159:0/826508797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f041590c0 0x7f4f0414b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:50.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.939+0000 7f4f02a00640 1 -- 172.21.15.159:0/826508797 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f04155d60 con 0x7f4f0406c580 2024-04-04T06:17:50.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.939+0000 7f4f02000640 1 --2- 172.21.15.159:0/826508797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f041590c0 0x7f4f0414b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:50.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.939+0000 7f4f02a00640 1 --2- 172.21.15.159:0/826508797 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f0406c580 0x7f4f041f8640 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400d800 tx=0x7f4ef400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:50.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.940+0000 7f4efaa00640 1 -- 172.21.15.159:0/826508797 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef400b6c0 con 0x7f4f0406c580 2024-04-04T06:17:50.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.940+0000 7f4efaa00640 1 -- 172.21.15.159:0/826508797 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef4019070 con 0x7f4f0406c580 2024-04-04T06:17:50.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.940+0000 7f4efaa00640 1 -- 172.21.15.159:0/826508797 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef400ba80 con 0x7f4f0406c580 2024-04-04T06:17:50.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.940+0000 7f4f09800640 1 -- 172.21.15.159:0/826508797 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f040cd3f0 con 0x7f4f0406c580 2024-04-04T06:17:50.938 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.940+0000 7f4f09800640 1 -- 172.21.15.159:0/826508797 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f040cd9b0 con 0x7f4f0406c580 2024-04-04T06:17:50.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.941+0000 7f4f09800640 1 -- 172.21.15.159:0/826508797 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f040685c0 con 0x7f4f0406c580 2024-04-04T06:17:50.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.942+0000 7f4efaa00640 1 -- 172.21.15.159:0/826508797 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ef400bc20 con 0x7f4f0406c580 2024-04-04T06:17:50.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.943+0000 7f4efaa00640 1 --2- 172.21.15.159:0/826508797 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ee406c8f0 0x7f4ee406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:50.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.943+0000 7f4efaa00640 1 -- 172.21.15.159:0/826508797 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f4ef408c130 con 0x7f4f0406c580 2024-04-04T06:17:50.939 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.944+0000 7f4f02000640 1 --2- 172.21.15.159:0/826508797 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ee406c8f0 0x7f4ee406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:50.940 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.945+0000 7f4f02000640 1 --2- 172.21.15.159:0/826508797 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ee406c8f0 0x7f4ee406edb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4f04155490 tx=0x7f4ef0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:50.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:50.947+0000 7f4efaa00640 1 -- 172.21.15.159:0/826508797 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4f04155d60 con 0x7f4f0406c580 2024-04-04T06:17:51.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.154+0000 7f4f09800640 1 -- 172.21.15.159:0/826508797 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4f04067590 con 0x7f4f0406c580 2024-04-04T06:17:51.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.155+0000 7f4efaa00640 1 -- 172.21.15.159:0/826508797 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) ==== 100+0+11 (secure 0 0 0) 0x7f4ef401c360 con 0x7f4f0406c580 2024-04-04T06:17:51.150 INFO:teuthology.orchestra.run.smithi159.stdout:pg_num: 64 2024-04-04T06:17:51.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.158+0000 7f4f09800640 1 -- 172.21.15.159:0/826508797 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ee406c8f0 msgr2=0x7f4ee406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:51.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.158+0000 7f4f09800640 1 --2- 172.21.15.159:0/826508797 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ee406c8f0 0x7f4ee406edb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4f04155490 tx=0x7f4ef0008040 comp rx=0 tx=0).stop 2024-04-04T06:17:51.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.158+0000 7f4f09800640 1 -- 172.21.15.159:0/826508797 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f0406c580 msgr2=0x7f4f041f8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:51.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.158+0000 7f4f09800640 1 --2- 172.21.15.159:0/826508797 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f0406c580 0x7f4f041f8640 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400d800 tx=0x7f4ef400dcc0 comp rx=0 tx=0).stop 2024-04-04T06:17:51.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.158+0000 7f4f09800640 1 -- 172.21.15.159:0/826508797 shutdown_connections 2024-04-04T06:17:51.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.158+0000 7f4f09800640 1 --2- 172.21.15.159:0/826508797 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f04162440 0x7f4f04157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:51.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.158+0000 7f4f09800640 1 --2- 172.21.15.159:0/826508797 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4ee406c8f0 0x7f4ee406edb0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:51.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.158+0000 7f4f09800640 1 --2- 172.21.15.159:0/826508797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f041590c0 0x7f4f0414b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:51.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.158+0000 7f4f09800640 1 --2- 172.21.15.159:0/826508797 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f0406c580 0x7f4f041f8640 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:51.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.158+0000 7f4f09800640 1 -- 172.21.15.159:0/826508797 >> 172.21.15.159:0/826508797 conn(0x7f4f04071e60 msgr2=0x7f4f040b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:51.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.158+0000 7f4f09800640 1 -- 172.21.15.159:0/826508797 shutdown_connections 2024-04-04T06:17:51.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.158+0000 7f4f09800640 1 -- 172.21.15.159:0/826508797 wait complete. 2024-04-04T06:17:51.170 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:17:51.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.377+0000 7fc75e000640 1 Processor -- start 2024-04-04T06:17:51.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.377+0000 7fc75e000640 1 -- start start 2024-04-04T06:17:51.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.378+0000 7fc75e000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc75815e4a0 0x7fc7580b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:51.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.378+0000 7fc75e000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7580cf700 0x7fc7580b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:51.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.378+0000 7fc75e000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc75815e880 con 0x7fc7580d2310 2024-04-04T06:17:51.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.378+0000 7fc75e000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fc7580d34e0 con 0x7fc75815e4a0 2024-04-04T06:17:51.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.378+0000 7fc75e000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc7580d28d0 con 0x7fc7580cf700 2024-04-04T06:17:51.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.378+0000 7fc75cc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc75815e4a0 0x7fc7580b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:51.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.378+0000 7fc75cc00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc75815e4a0 0x7fc7580b9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:46026/0 (socket says 172.21.15.159:46026) 2024-04-04T06:17:51.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.378+0000 7fc75cc00640 1 -- 172.21.15.159:0/2775455293 learned_addr learned my addr 172.21.15.159:0/2775455293 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:51.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.379+0000 7fc75d600640 1 --2- 172.21.15.159:0/2775455293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7580cf700 0x7fc7580b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:51.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.379+0000 7fc75cc00640 1 -- 172.21.15.159:0/2775455293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7580cf700 msgr2=0x7fc7580b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:51.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.379+0000 7fc75cc00640 1 --2- 172.21.15.159:0/2775455293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7580cf700 0x7fc7580b73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:51.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.379+0000 7fc75cc00640 1 -- 172.21.15.159:0/2775455293 >> v1:172.21.15.178:6789/0 conn(0x7fc7580d2310 legacy=0x7fc7580ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:51.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.379+0000 7fc75cc00640 1 -- 172.21.15.159:0/2775455293 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7580d2bb0 con 0x7fc75815e4a0 2024-04-04T06:17:51.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.380+0000 7fc75cc00640 1 --2- 172.21.15.159:0/2775455293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc75815e4a0 0x7fc7580b9c60 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fc7580d3660 tx=0x7fc74802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fbce65d767e2a63 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:51.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.380+0000 7fc757e00640 1 -- 172.21.15.159:0/2775455293 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7480317c0 con 0x7fc75815e4a0 2024-04-04T06:17:51.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.380+0000 7fc757e00640 1 -- 172.21.15.159:0/2775455293 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc748037070 con 0x7fc75815e4a0 2024-04-04T06:17:51.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.380+0000 7fc757e00640 1 -- 172.21.15.159:0/2775455293 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748031b80 con 0x7fc75815e4a0 2024-04-04T06:17:51.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.381+0000 7fc75e000640 1 -- 172.21.15.159:0/2775455293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc75815e4a0 msgr2=0x7fc7580b9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:51.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.381+0000 7fc75e000640 1 --2- 172.21.15.159:0/2775455293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc75815e4a0 0x7fc7580b9c60 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fc7580d3660 tx=0x7fc74802b860 comp rx=0 tx=0).stop 2024-04-04T06:17:51.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.381+0000 7fc75e000640 1 -- 172.21.15.159:0/2775455293 shutdown_connections 2024-04-04T06:17:51.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.381+0000 7fc75e000640 1 --2- 172.21.15.159:0/2775455293 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7580cf700 0x7fc7580b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:51.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.381+0000 7fc75e000640 1 --2- 172.21.15.159:0/2775455293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc75815e4a0 0x7fc7580b9c60 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:51.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.381+0000 7fc75e000640 1 -- 172.21.15.159:0/2775455293 >> 172.21.15.159:0/2775455293 conn(0x7fc7580676d0 msgr2=0x7fc758067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:51.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.381+0000 7fc75e000640 1 -- 172.21.15.159:0/2775455293 shutdown_connections 2024-04-04T06:17:51.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.381+0000 7fc75e000640 1 -- 172.21.15.159:0/2775455293 wait complete. 2024-04-04T06:17:51.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.383+0000 7fc75e000640 1 Processor -- start 2024-04-04T06:17:51.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.383+0000 7fc75e000640 1 -- start start 2024-04-04T06:17:51.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.384+0000 7fc75e000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7580cf700 0x7fc758160170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:51.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.384+0000 7fc75e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7580d2310 0x7fc758152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:51.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.384+0000 7fc75e000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc75815e4a0 0x7fc7580ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:51.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.384+0000 7fc75e000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc7580d3970 con 0x7fc7580d2310 2024-04-04T06:17:51.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.384+0000 7fc75e000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc7580d3170 con 0x7fc75815e4a0 2024-04-04T06:17:51.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.384+0000 7fc75e000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc7580ba1a0 con 0x7fc7580cf700 2024-04-04T06:17:51.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.384+0000 7fc74fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7580d2310 0x7fc758152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:51.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.384+0000 7fc74fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7580d2310 0x7fc758152af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:46028/0 (socket says 172.21.15.159:46028) 2024-04-04T06:17:51.380 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.384+0000 7fc74fe00640 1 -- 172.21.15.159:0/3422051286 learned_addr learned my addr 172.21.15.159:0/3422051286 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:51.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.385+0000 7fc75d600640 1 --2- 172.21.15.159:0/3422051286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc75815e4a0 0x7fc7580ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:51.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.385+0000 7fc75cc00640 1 --2- 172.21.15.159:0/3422051286 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7580cf700 0x7fc758160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:51.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.385+0000 7fc74fe00640 1 -- 172.21.15.159:0/3422051286 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7580cf700 msgr2=0x7fc758160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:51.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.385+0000 7fc74fe00640 1 --2- 172.21.15.159:0/3422051286 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7580cf700 0x7fc758160170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:51.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.385+0000 7fc74fe00640 1 -- 172.21.15.159:0/3422051286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc75815e4a0 msgr2=0x7fc7580ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:51.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.385+0000 7fc74fe00640 1 --2- 172.21.15.159:0/3422051286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc75815e4a0 0x7fc7580ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:51.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.385+0000 7fc74fe00640 1 -- 172.21.15.159:0/3422051286 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc758153030 con 0x7fc7580d2310 2024-04-04T06:17:51.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.385+0000 7fc75d600640 1 --2- 172.21.15.159:0/3422051286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc75815e4a0 0x7fc7580ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:51.381 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.385+0000 7fc74fe00640 1 --2- 172.21.15.159:0/3422051286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7580d2310 0x7fc758152af0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fc74000c430 tx=0x7fc74000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:51.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.385+0000 7fc755600640 1 -- 172.21.15.159:0/3422051286 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc740011790 con 0x7fc7580d2310 2024-04-04T06:17:51.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.385+0000 7fc755600640 1 -- 172.21.15.159:0/3422051286 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc740017070 con 0x7fc7580d2310 2024-04-04T06:17:51.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.385+0000 7fc755600640 1 -- 172.21.15.159:0/3422051286 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc740011bb0 con 0x7fc7580d2310 2024-04-04T06:17:51.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.385+0000 7fc75e000640 1 -- 172.21.15.159:0/3422051286 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc758160830 con 0x7fc7580d2310 2024-04-04T06:17:51.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.386+0000 7fc75e000640 1 -- 172.21.15.159:0/3422051286 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7580c9020 con 0x7fc7580d2310 2024-04-04T06:17:51.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.387+0000 7fc75e000640 1 -- 172.21.15.159:0/3422051286 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7580c8e10 con 0x7fc7580d2310 2024-04-04T06:17:51.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.388+0000 7fc755600640 1 -- 172.21.15.159:0/3422051286 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc74001c460 con 0x7fc7580d2310 2024-04-04T06:17:51.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.389+0000 7fc755600640 1 --2- 172.21.15.159:0/3422051286 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc73406c8f0 0x7fc73406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:51.383 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.389+0000 7fc755600640 1 -- 172.21.15.159:0/3422051286 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7fc74008bb40 con 0x7fc7580d2310 2024-04-04T06:17:51.384 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.389+0000 7fc75cc00640 1 --2- 172.21.15.159:0/3422051286 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc73406c8f0 0x7fc73406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:51.385 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.390+0000 7fc75cc00640 1 --2- 172.21.15.159:0/3422051286 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc73406c8f0 0x7fc73406edb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc748036000 tx=0x7fc748008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:51.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.393+0000 7fc755600640 1 -- 172.21.15.159:0/3422051286 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc740059e80 con 0x7fc7580d2310 2024-04-04T06:17:51.626 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.631+0000 7fc75e000640 1 -- 172.21.15.159:0/3422051286 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc758075940 con 0x7fc7580d2310 2024-04-04T06:17:51.627 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.632+0000 7fc755600640 1 -- 172.21.15.159:0/3422051286 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5197 (secure 0 0 0) 0x7fc740059780 con 0x7fc7580d2310 2024-04-04T06:17:51.627 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:51.627 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":114,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5732,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1498324773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1498324773},{"type":"v1","addr":"172.21.15.178:6837","nonce":1498324773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2455031662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":2455031662},{"type":"v1","addr":"172.21.15.159:6835","nonce":2455031662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6343,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1823228151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1823228151},{"type":"v1","addr":"172.21.15.178:6836","nonce":1823228151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:16:57.277344+0000","modified":"2024-04-04T06:17:40.711526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9312":{"gid":9312,"name":"a","rank":0,"incarnation":111,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6834/643543929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":643543929},{"type":"v1","addr":"172.21.15.159:6834","nonce":643543929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9312,"qdb_cluster":[9312]},"id":8}]} 2024-04-04T06:17:51.628 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 114 2024-04-04T06:17:51.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.635+0000 7fc75e000640 1 -- 172.21.15.159:0/3422051286 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc73406c8f0 msgr2=0x7fc73406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:51.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.635+0000 7fc75e000640 1 --2- 172.21.15.159:0/3422051286 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc73406c8f0 0x7fc73406edb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc748036000 tx=0x7fc748008030 comp rx=0 tx=0).stop 2024-04-04T06:17:51.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.635+0000 7fc75e000640 1 -- 172.21.15.159:0/3422051286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7580d2310 msgr2=0x7fc758152af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:51.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.635+0000 7fc75e000640 1 --2- 172.21.15.159:0/3422051286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7580d2310 0x7fc758152af0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fc74000c430 tx=0x7fc74000c8f0 comp rx=0 tx=0).stop 2024-04-04T06:17:51.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.636+0000 7fc75e000640 1 -- 172.21.15.159:0/3422051286 shutdown_connections 2024-04-04T06:17:51.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.636+0000 7fc75e000640 1 --2- 172.21.15.159:0/3422051286 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc73406c8f0 0x7fc73406edb0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:51.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.636+0000 7fc75e000640 1 --2- 172.21.15.159:0/3422051286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc75815e4a0 0x7fc7580ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:51.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.636+0000 7fc75e000640 1 --2- 172.21.15.159:0/3422051286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7580d2310 0x7fc758152af0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:51.631 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.636+0000 7fc75e000640 1 --2- 172.21.15.159:0/3422051286 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc7580cf700 0x7fc758160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.636+0000 7fc75e000640 1 -- 172.21.15.159:0/3422051286 >> 172.21.15.159:0/3422051286 conn(0x7fc7580676d0 msgr2=0x7fc7580b93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.636+0000 7fc75e000640 1 -- 172.21.15.159:0/3422051286 shutdown_connections 2024-04-04T06:17:51.632 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.636+0000 7fc75e000640 1 -- 172.21.15.159:0/3422051286 wait complete. 2024-04-04T06:17:51.650 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:17:51.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.863+0000 7ff7e0c00640 1 Processor -- start 2024-04-04T06:17:51.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.863+0000 7ff7e0c00640 1 -- start start 2024-04-04T06:17:51.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.865+0000 7ff7e0c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff7dc0cf700 0x7ff7dc168fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:51.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.865+0000 7ff7e0c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7dc15e880 con 0x7ff7dc15e4a0 2024-04-04T06:17:51.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.865+0000 7ff7e0c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7dc0d28d0 con 0x7ff7dc0d2310 2024-04-04T06:17:51.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.865+0000 7ff7e0c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ff7dc0d34e0 con 0x7ff7dc0cf700 2024-04-04T06:17:51.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.866+0000 7ff7db400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7ff7dc15e4a0 0x7ff7dc0b9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:42938/0 (socket says 172.21.15.159:42938) 2024-04-04T06:17:51.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.866+0000 7ff7db400640 1 -- 172.21.15.159:0/502383952 learned_addr learned my addr 172.21.15.159:0/502383952 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:51.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.866+0000 7ff7dbe00640 1 --2- 172.21.15.159:0/502383952 >> v2:172.21.15.178:3300/0 conn(0x7ff7dc0cf700 0x7ff7dc168fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:51.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.866+0000 7ff7da000640 1 -- 172.21.15.159:0/502383952 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1921819671 0 0) 0x7ff7dc15e880 con 0x7ff7dc15e4a0 2024-04-04T06:17:51.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.866+0000 7ff7da000640 1 -- 172.21.15.159:0/502383952 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7c0003520 con 0x7ff7dc15e4a0 2024-04-04T06:17:51.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.866+0000 7ff7da000640 1 -- 172.21.15.159:0/502383952 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1861495216 0 0) 0x7ff7c0003520 con 0x7ff7dc15e4a0 2024-04-04T06:17:51.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.866+0000 7ff7da000640 1 -- 172.21.15.159:0/502383952 >> v2:172.21.15.178:3300/0 conn(0x7ff7dc0cf700 msgr2=0x7ff7dc168fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:51.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.866+0000 7ff7da000640 1 --2- 172.21.15.159:0/502383952 >> v2:172.21.15.178:3300/0 conn(0x7ff7dc0cf700 0x7ff7dc168fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:51.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.866+0000 7ff7da000640 1 -- 172.21.15.159:0/502383952 >> v1:172.21.15.178:6789/0 conn(0x7ff7dc0d2310 legacy=0x7ff7dc0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:51.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.866+0000 7ff7da000640 1 -- 172.21.15.159:0/502383952 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7dc0d2bb0 con 0x7ff7dc15e4a0 2024-04-04T06:17:51.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.866+0000 7ff7da000640 1 -- 172.21.15.159:0/502383952 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7ff7cc001690 con 0x7ff7dc15e4a0 2024-04-04T06:17:51.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.866+0000 7ff7da000640 1 -- 172.21.15.159:0/502383952 >> v1:172.21.15.159:6789/0 conn(0x7ff7dc15e4a0 legacy=0x7ff7dc0b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:51.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.867+0000 7ff7da000640 1 --2- 172.21.15.159:0/502383952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7c0003a00 0x7ff7c0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:51.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.867+0000 7ff7da000640 1 --2- 172.21.15.159:0/502383952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7c0004800 0x7ff7c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:51.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.867+0000 7ff7da000640 1 --2- 172.21.15.159:0/502383952 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7c0005210 0x7ff7c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:51.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.867+0000 7ff7da000640 1 -- 172.21.15.159:0/502383952 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc0d34e0 con 0x7ff7c0003a00 2024-04-04T06:17:51.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.867+0000 7ff7da000640 1 -- 172.21.15.159:0/502383952 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c000c160 con 0x7ff7c0004800 2024-04-04T06:17:51.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.867+0000 7ff7da000640 1 -- 172.21.15.159:0/502383952 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff7c000c360 con 0x7ff7c0005210 2024-04-04T06:17:51.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.867+0000 7ff7db400640 1 --2- 172.21.15.159:0/502383952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7c0003a00 0x7ff7c0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:51.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.867+0000 7ff7daa00640 1 --2- 172.21.15.159:0/502383952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7c0004800 0x7ff7c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:51.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.867+0000 7ff7db400640 1 -- 172.21.15.159:0/502383952 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7c0005210 msgr2=0x7ff7c000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:51.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.867+0000 7ff7dbe00640 1 --2- 172.21.15.159:0/502383952 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7c0005210 0x7ff7c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:51.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.867+0000 7ff7db400640 1 --2- 172.21.15.159:0/502383952 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7c0005210 0x7ff7c000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:51.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.867+0000 7ff7db400640 1 -- 172.21.15.159:0/502383952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7c0004800 msgr2=0x7ff7c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:51.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.867+0000 7ff7db400640 1 --2- 172.21.15.159:0/502383952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7c0004800 0x7ff7c0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:51.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.867+0000 7ff7db400640 1 -- 172.21.15.159:0/502383952 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c0003520 con 0x7ff7c0003a00 2024-04-04T06:17:51.865 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.867+0000 7ff7dbe00640 1 -- 172.21.15.159:0/502383952 reap_dead start 2024-04-04T06:17:51.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.868+0000 7ff7db400640 1 --2- 172.21.15.159:0/502383952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7c0003a00 0x7ff7c0003e20 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc15ea50 tx=0x7ff7c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7bbcb72b5a4bbe server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:51.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.869+0000 7ff7da000640 1 -- 172.21.15.159:0/502383952 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc007c40 con 0x7ff7c0003a00 2024-04-04T06:17:51.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.869+0000 7ff7da000640 1 -- 172.21.15.159:0/502383952 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7cc030070 con 0x7ff7c0003a00 2024-04-04T06:17:51.866 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.869+0000 7ff7da000640 1 -- 172.21.15.159:0/502383952 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc005d50 con 0x7ff7c0003a00 2024-04-04T06:17:51.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.869+0000 7ff7e0c00640 1 -- 172.21.15.159:0/502383952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7c0003a00 msgr2=0x7ff7c0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:51.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.869+0000 7ff7e0c00640 1 --2- 172.21.15.159:0/502383952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7c0003a00 0x7ff7c0003e20 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc15ea50 tx=0x7ff7c0005830 comp rx=0 tx=0).stop 2024-04-04T06:17:51.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.870+0000 7ff7e0c00640 1 -- 172.21.15.159:0/502383952 shutdown_connections 2024-04-04T06:17:51.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.870+0000 7ff7e0c00640 1 --2- 172.21.15.159:0/502383952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7c0003a00 0x7ff7c0003e20 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:51.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.870+0000 7ff7e0c00640 1 -- 172.21.15.159:0/502383952 >> 172.21.15.159:0/502383952 conn(0x7ff7dc0676d0 msgr2=0x7ff7dc067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:51.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.870+0000 7ff7e0c00640 1 -- 172.21.15.159:0/502383952 shutdown_connections 2024-04-04T06:17:51.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.870+0000 7ff7e0c00640 1 -- 172.21.15.159:0/502383952 wait complete. 2024-04-04T06:17:51.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.871+0000 7ff7e0c00640 1 Processor -- start 2024-04-04T06:17:51.867 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.872+0000 7ff7e0c00640 1 -- start start 2024-04-04T06:17:51.869 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.872+0000 7ff7e0c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7c0003a00 0x7ff7dc0b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:51.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.872+0000 7ff7e0c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7dc159950 0x7ff7dc160650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:51.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.872+0000 7ff7e0c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7dc158d90 0x7ff7dc159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:51.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.872+0000 7ff7e0c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc0d3970 con 0x7ff7c0003a00 2024-04-04T06:17:51.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.872+0000 7ff7e0c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc0d3170 con 0x7ff7dc158d90 2024-04-04T06:17:51.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.872+0000 7ff7e0c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff7dc0ceff0 con 0x7ff7dc159950 2024-04-04T06:17:51.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.873+0000 7ff7db400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7c0003a00 0x7ff7dc0b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:51.870 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.873+0000 7ff7db400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7c0003a00 0x7ff7dc0b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:46046/0 (socket says 172.21.15.159:46046) 2024-04-04T06:17:51.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.873+0000 7ff7db400640 1 -- 172.21.15.159:0/201229795 learned_addr learned my addr 172.21.15.159:0/201229795 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:51.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.873+0000 7ff7daa00640 1 --2- 172.21.15.159:0/201229795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7dc159950 0x7ff7dc160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:51.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.873+0000 7ff7dbe00640 1 --2- 172.21.15.159:0/201229795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7dc158d90 0x7ff7dc159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:51.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.873+0000 7ff7db400640 1 -- 172.21.15.159:0/201229795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7dc159950 msgr2=0x7ff7dc160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:51.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.873+0000 7ff7db400640 1 --2- 172.21.15.159:0/201229795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7dc159950 0x7ff7dc160650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:51.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.873+0000 7ff7db400640 1 -- 172.21.15.159:0/201229795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7dc158d90 msgr2=0x7ff7dc159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:51.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.873+0000 7ff7db400640 1 --2- 172.21.15.159:0/201229795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7dc158d90 0x7ff7dc159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:51.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.873+0000 7ff7db400640 1 -- 172.21.15.159:0/201229795 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7dc0aad50 con 0x7ff7c0003a00 2024-04-04T06:17:51.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.873+0000 7ff7dbe00640 1 --2- 172.21.15.159:0/201229795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7dc158d90 0x7ff7dc159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:51.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.874+0000 7ff7db400640 1 --2- 172.21.15.159:0/201229795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7c0003a00 0x7ff7dc0b3570 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc000c00 tx=0x7ff7dc0d2aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:51.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.874+0000 7ff7d3400640 1 -- 172.21.15.159:0/201229795 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc045070 con 0x7ff7c0003a00 2024-04-04T06:17:51.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.874+0000 7ff7d3400640 1 -- 172.21.15.159:0/201229795 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7cc000ed0 con 0x7ff7c0003a00 2024-04-04T06:17:51.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.874+0000 7ff7d3400640 1 -- 172.21.15.159:0/201229795 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc001b80 con 0x7ff7c0003a00 2024-04-04T06:17:51.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.874+0000 7ff7e0c00640 1 -- 172.21.15.159:0/201229795 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7dc0ab030 con 0x7ff7c0003a00 2024-04-04T06:17:51.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.874+0000 7ff7e0c00640 1 -- 172.21.15.159:0/201229795 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7dc0ccdb0 con 0x7ff7c0003a00 2024-04-04T06:17:51.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.876+0000 7ff7e0c00640 1 -- 172.21.15.159:0/201229795 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7dc075940 con 0x7ff7c0003a00 2024-04-04T06:17:51.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.876+0000 7ff7d3400640 1 -- 172.21.15.159:0/201229795 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7cc048070 con 0x7ff7c0003a00 2024-04-04T06:17:51.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.877+0000 7ff7d3400640 1 --2- 172.21.15.159:0/201229795 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff7bc06c8f0 0x7ff7bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:51.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.877+0000 7ff7d3400640 1 -- 172.21.15.159:0/201229795 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7ff7cc0acad0 con 0x7ff7c0003a00 2024-04-04T06:17:51.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.878+0000 7ff7daa00640 1 --2- 172.21.15.159:0/201229795 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff7bc06c8f0 0x7ff7bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:51.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.879+0000 7ff7daa00640 1 --2- 172.21.15.159:0/201229795 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff7bc06c8f0 0x7ff7bc06edb0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8007fd0 tx=0x7ff7c8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:51.881 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:51.886+0000 7ff7d3400640 1 -- 172.21.15.159:0/201229795 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff7cc07a850 con 0x7ff7c0003a00 2024-04-04T06:17:52.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.090+0000 7ff7e0c00640 1 -- 172.21.15.159:0/201229795 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7dc075940 con 0x7ff7c0003a00 2024-04-04T06:17:52.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.093+0000 7ff7d3400640 1 -- 172.21.15.159:0/201229795 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+16995 (secure 0 0 0) 0x7ff7cc07eb50 con 0x7ff7c0003a00 2024-04-04T06:17:52.088 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:52.089 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":116,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:17:35.667542+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:16:55.036975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-04T06:16:56.239022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.b","mappings":[{"from":0,"to":5}]},{"pgid":"17.26","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6834/2439261212":"2024-04-05T06:17:35.667507+0000","172.21.15.159:6832/2439261212":"2024-04-05T06:17:35.667507+0000","172.21.15.159:6834/1017894858":"2024-04-05T06:17:21.596764+0000","172.21.15.159:6832/1017894858":"2024-04-05T06:17:21.596764+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:17:52.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.095+0000 7ff7e0c00640 1 -- 172.21.15.159:0/201229795 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff7bc06c8f0 msgr2=0x7ff7bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:52.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.096+0000 7ff7e0c00640 1 --2- 172.21.15.159:0/201229795 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff7bc06c8f0 0x7ff7bc06edb0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8007fd0 tx=0x7ff7c8009040 comp rx=0 tx=0).stop 2024-04-04T06:17:52.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.096+0000 7ff7e0c00640 1 -- 172.21.15.159:0/201229795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7c0003a00 msgr2=0x7ff7dc0b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:52.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.096+0000 7ff7e0c00640 1 --2- 172.21.15.159:0/201229795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7c0003a00 0x7ff7dc0b3570 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc000c00 tx=0x7ff7dc0d2aa0 comp rx=0 tx=0).stop 2024-04-04T06:17:52.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.096+0000 7ff7e0c00640 1 -- 172.21.15.159:0/201229795 shutdown_connections 2024-04-04T06:17:52.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.096+0000 7ff7e0c00640 1 --2- 172.21.15.159:0/201229795 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff7bc06c8f0 0x7ff7bc06edb0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:52.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.096+0000 7ff7e0c00640 1 --2- 172.21.15.159:0/201229795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7dc158d90 0x7ff7dc159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:52.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.096+0000 7ff7e0c00640 1 --2- 172.21.15.159:0/201229795 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7dc159950 0x7ff7dc160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:52.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.096+0000 7ff7e0c00640 1 --2- 172.21.15.159:0/201229795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7c0003a00 0x7ff7dc0b3570 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:52.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.096+0000 7ff7e0c00640 1 -- 172.21.15.159:0/201229795 >> 172.21.15.159:0/201229795 conn(0x7ff7dc0676d0 msgr2=0x7ff7dc15b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:52.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.096+0000 7ff7e0c00640 1 -- 172.21.15.159:0/201229795 shutdown_connections 2024-04-04T06:17:52.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.096+0000 7ff7e0c00640 1 -- 172.21.15.159:0/201229795 wait complete. 2024-04-04T06:17:52.109 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-04T06:17:52.110 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:17:52.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.316+0000 7f13c2800640 1 Processor -- start 2024-04-04T06:17:52.311 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.316+0000 7f13c2800640 1 -- start start 2024-04-04T06:17:52.312 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.318+0000 7f13c2800640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f13bc0d2310 0x7f13bc0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:52.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.318+0000 7f13c2800640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13bc15e880 con 0x7f13bc0cf530 2024-04-04T06:17:52.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.318+0000 7f13c2800640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13bc0d28d0 con 0x7f13bc15e4a0 2024-04-04T06:17:52.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.318+0000 7f13c2800640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f13bc0cf9a0 con 0x7f13bc0d2310 2024-04-04T06:17:52.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.318+0000 7f13c1e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f13bc0cf530 0x7f13bc0b73e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:42942/0 (socket says 172.21.15.159:42942) 2024-04-04T06:17:52.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.318+0000 7f13c1e00640 1 -- 172.21.15.159:0/197001933 learned_addr learned my addr 172.21.15.159:0/197001933 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:52.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.318+0000 7f13c0a00640 1 --2- 172.21.15.159:0/197001933 >> v2:172.21.15.178:3300/0 conn(0x7f13bc0d2310 0x7f13bc0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:52.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.319+0000 7f13bbe00640 1 -- 172.21.15.159:0/197001933 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2113107992 0 0) 0x7f13bc15e880 con 0x7f13bc0cf530 2024-04-04T06:17:52.314 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.319+0000 7f13bbe00640 1 -- 172.21.15.159:0/197001933 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13a4003520 con 0x7f13bc0cf530 2024-04-04T06:17:52.316 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.319+0000 7f13bbe00640 1 -- 172.21.15.159:0/197001933 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2761556847 0 0) 0x7f13a4003520 con 0x7f13bc0cf530 2024-04-04T06:17:52.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.319+0000 7f13bbe00640 1 -- 172.21.15.159:0/197001933 >> v2:172.21.15.178:3300/0 conn(0x7f13bc0d2310 msgr2=0x7f13bc0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:52.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.319+0000 7f13bbe00640 1 --2- 172.21.15.159:0/197001933 >> v2:172.21.15.178:3300/0 conn(0x7f13bc0d2310 0x7f13bc0ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:52.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.319+0000 7f13bbe00640 1 -- 172.21.15.159:0/197001933 >> v1:172.21.15.178:6789/0 conn(0x7f13bc15e4a0 legacy=0x7f13bc0b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:52.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.319+0000 7f13bbe00640 1 -- 172.21.15.159:0/197001933 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13bc0d2bb0 con 0x7f13bc0cf530 2024-04-04T06:17:52.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.319+0000 7f13bbe00640 1 -- 172.21.15.159:0/197001933 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f13b40019f0 con 0x7f13bc0cf530 2024-04-04T06:17:52.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.319+0000 7f13bbe00640 1 -- 172.21.15.159:0/197001933 >> v1:172.21.15.159:6789/0 conn(0x7f13bc0cf530 legacy=0x7f13bc0b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:52.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.319+0000 7f13bbe00640 1 --2- 172.21.15.159:0/197001933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13a40039b0 0x7f13a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:52.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.319+0000 7f13bbe00640 1 --2- 172.21.15.159:0/197001933 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a4004800 0x7f13a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:52.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.319+0000 7f13bbe00640 1 --2- 172.21.15.159:0/197001933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13a40051c0 0x7f13a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:52.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.319+0000 7f13bbe00640 1 -- 172.21.15.159:0/197001933 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc0cf9a0 con 0x7f13a40051c0 2024-04-04T06:17:52.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.319+0000 7f13bbe00640 1 -- 172.21.15.159:0/197001933 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f13a400c160 con 0x7f13a40039b0 2024-04-04T06:17:52.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.319+0000 7f13bbe00640 1 -- 172.21.15.159:0/197001933 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f13a400c360 con 0x7f13a4004800 2024-04-04T06:17:52.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.320+0000 7f13c1e00640 1 --2- 172.21.15.159:0/197001933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13a40051c0 0x7f13a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:52.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.320+0000 7f13c0a00640 1 --2- 172.21.15.159:0/197001933 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a4004800 0x7f13a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:52.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.320+0000 7f13c1400640 1 --2- 172.21.15.159:0/197001933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13a40039b0 0x7f13a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:52.323 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.320+0000 7f13c1e00640 1 -- 172.21.15.159:0/197001933 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a4004800 msgr2=0x7f13a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:52.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.320+0000 7f13c1e00640 1 --2- 172.21.15.159:0/197001933 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a4004800 0x7f13a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:52.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.320+0000 7f13c1e00640 1 -- 172.21.15.159:0/197001933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13a40039b0 msgr2=0x7f13a4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:52.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.320+0000 7f13c1e00640 1 --2- 172.21.15.159:0/197001933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13a40039b0 0x7f13a4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:52.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.320+0000 7f13c1e00640 1 -- 172.21.15.159:0/197001933 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a4003520 con 0x7f13a40051c0 2024-04-04T06:17:52.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.322+0000 7f13c1e00640 1 -- 172.21.15.159:0/197001933 reap_dead start 2024-04-04T06:17:52.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.322+0000 7f13c1e00640 1 --2- 172.21.15.159:0/197001933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13a40051c0 0x7f13a400b9c0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f13b400a840 tx=0x7f13a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aecc4c7d92351544 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:52.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.323+0000 7f13bbe00640 1 -- 172.21.15.159:0/197001933 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b400a460 con 0x7f13a40051c0 2024-04-04T06:17:52.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.323+0000 7f13bbe00640 1 -- 172.21.15.159:0/197001933 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b4034650 con 0x7f13a40051c0 2024-04-04T06:17:52.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.323+0000 7f13bbe00640 1 -- 172.21.15.159:0/197001933 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4034a40 con 0x7f13a40051c0 2024-04-04T06:17:52.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.323+0000 7f13c2800640 1 -- 172.21.15.159:0/197001933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13a40051c0 msgr2=0x7f13a400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:52.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.323+0000 7f13c2800640 1 --2- 172.21.15.159:0/197001933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13a40051c0 0x7f13a400b9c0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f13b400a840 tx=0x7f13a4005830 comp rx=0 tx=0).stop 2024-04-04T06:17:52.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.323+0000 7f13c2800640 1 -- 172.21.15.159:0/197001933 shutdown_connections 2024-04-04T06:17:52.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.323+0000 7f13c2800640 1 --2- 172.21.15.159:0/197001933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13a40051c0 0x7f13a400b9c0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:52.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.323+0000 7f13c2800640 1 -- 172.21.15.159:0/197001933 >> 172.21.15.159:0/197001933 conn(0x7f13bc0676d0 msgr2=0x7f13bc067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:52.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.324+0000 7f13c2800640 1 -- 172.21.15.159:0/197001933 shutdown_connections 2024-04-04T06:17:52.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.324+0000 7f13c2800640 1 -- 172.21.15.159:0/197001933 wait complete. 2024-04-04T06:17:52.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.325+0000 7f13c2800640 1 Processor -- start 2024-04-04T06:17:52.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.325+0000 7f13c2800640 1 -- start start 2024-04-04T06:17:52.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.326+0000 7f13c2800640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a40051c0 0x7f13bc0cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:52.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.326+0000 7f13c2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13bc0aac20 0x7f13bc0b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:52.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.326+0000 7f13c2800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13bc0ac430 0x7f13bc0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:52.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.326+0000 7f13c2800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc0d3170 con 0x7f13bc0aac20 2024-04-04T06:17:52.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.326+0000 7f13c2800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc0d34e0 con 0x7f13bc0ac430 2024-04-04T06:17:52.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.326+0000 7f13c2800640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f13bc0d3970 con 0x7f13a40051c0 2024-04-04T06:17:52.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.327+0000 7f13c0a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13bc0aac20 0x7f13bc0b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:52.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.327+0000 7f13c0a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13bc0aac20 0x7f13bc0b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:46058/0 (socket says 172.21.15.159:46058) 2024-04-04T06:17:52.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.327+0000 7f13c0a00640 1 -- 172.21.15.159:0/2699518912 learned_addr learned my addr 172.21.15.159:0/2699518912 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.327+0000 7f13c1400640 1 --2- 172.21.15.159:0/2699518912 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a40051c0 0x7f13bc0cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.327+0000 7f13c0a00640 1 -- 172.21.15.159:0/2699518912 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a40051c0 msgr2=0x7f13bc0cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.327+0000 7f13c0a00640 1 --2- 172.21.15.159:0/2699518912 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a40051c0 0x7f13bc0cdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.327+0000 7f13c0a00640 1 -- 172.21.15.159:0/2699518912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13bc0ac430 msgr2=0x7f13bc0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.327+0000 7f13c0a00640 1 --2- 172.21.15.159:0/2699518912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13bc0ac430 0x7f13bc0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.327+0000 7f13c0a00640 1 -- 172.21.15.159:0/2699518912 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13bc0ab390 con 0x7f13bc0aac20 2024-04-04T06:17:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.327+0000 7f13c0a00640 1 --2- 172.21.15.159:0/2699518912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13bc0aac20 0x7f13bc0b35d0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f13a8011540 tx=0x7f13a8011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.328+0000 7f13b9600640 1 -- 172.21.15.159:0/2699518912 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8015930 con 0x7f13bc0aac20 2024-04-04T06:17:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.328+0000 7f13b9600640 1 -- 172.21.15.159:0/2699518912 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13a8015ad0 con 0x7f13bc0aac20 2024-04-04T06:17:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.328+0000 7f13b9600640 1 -- 172.21.15.159:0/2699518912 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a800f460 con 0x7f13bc0aac20 2024-04-04T06:17:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.328+0000 7f13c2800640 1 -- 172.21.15.159:0/2699518912 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13bc0ab670 con 0x7f13bc0aac20 2024-04-04T06:17:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.328+0000 7f13c2800640 1 -- 172.21.15.159:0/2699518912 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13bc0abc30 con 0x7f13bc0aac20 2024-04-04T06:17:52.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.329+0000 7f13c2800640 1 -- 172.21.15.159:0/2699518912 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13bc0ceff0 con 0x7f13bc0aac20 2024-04-04T06:17:52.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.330+0000 7f13b9600640 1 -- 172.21.15.159:0/2699518912 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13a800f600 con 0x7f13bc0aac20 2024-04-04T06:17:52.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.331+0000 7f13b9600640 1 --2- 172.21.15.159:0/2699518912 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f139c06c920 0x7f139c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:52.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.331+0000 7f13b9600640 1 -- 172.21.15.159:0/2699518912 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f13a808c4e0 con 0x7f13bc0aac20 2024-04-04T06:17:52.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.331+0000 7f13c1400640 1 --2- 172.21.15.159:0/2699518912 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f139c06c920 0x7f139c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:52.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.333+0000 7f13c1400640 1 --2- 172.21.15.159:0/2699518912 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f139c06c920 0x7f139c06ede0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f13b0007fd0 tx=0x7f13b0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:52.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.336+0000 7f13b9600640 1 -- 172.21.15.159:0/2699518912 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f13bc0ceff0 con 0x7f13bc0aac20 2024-04-04T06:17:52.569 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.574+0000 7f13c2800640 1 -- 172.21.15.159:0/2699518912 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13bc075940 con 0x7f13bc0aac20 2024-04-04T06:17:52.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.575+0000 7f13b9600640 1 -- 172.21.15.159:0/2699518912 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5197 (secure 0 0 0) 0x7f13a8019030 con 0x7f13bc0aac20 2024-04-04T06:17:52.570 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:52.570 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":114,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5732,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1498324773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1498324773},{"type":"v1","addr":"172.21.15.178:6837","nonce":1498324773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2455031662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":2455031662},{"type":"v1","addr":"172.21.15.159:6835","nonce":2455031662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6343,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1823228151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1823228151},{"type":"v1","addr":"172.21.15.178:6836","nonce":1823228151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:16:57.277344+0000","modified":"2024-04-04T06:17:40.711526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9312":{"gid":9312,"name":"a","rank":0,"incarnation":111,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6834/643543929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":643543929},{"type":"v1","addr":"172.21.15.159:6834","nonce":643543929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9312,"qdb_cluster":[9312]},"id":8}]} 2024-04-04T06:17:52.571 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 114 2024-04-04T06:17:52.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.578+0000 7f13c2800640 1 -- 172.21.15.159:0/2699518912 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f139c06c920 msgr2=0x7f139c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:52.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.578+0000 7f13c2800640 1 --2- 172.21.15.159:0/2699518912 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f139c06c920 0x7f139c06ede0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f13b0007fd0 tx=0x7f13b0009040 comp rx=0 tx=0).stop 2024-04-04T06:17:52.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.578+0000 7f13c2800640 1 -- 172.21.15.159:0/2699518912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13bc0aac20 msgr2=0x7f13bc0b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:52.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.578+0000 7f13c2800640 1 --2- 172.21.15.159:0/2699518912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13bc0aac20 0x7f13bc0b35d0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f13a8011540 tx=0x7f13a8011a00 comp rx=0 tx=0).stop 2024-04-04T06:17:52.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.578+0000 7f13c2800640 1 -- 172.21.15.159:0/2699518912 shutdown_connections 2024-04-04T06:17:52.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.578+0000 7f13c2800640 1 --2- 172.21.15.159:0/2699518912 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f139c06c920 0x7f139c06ede0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:52.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.578+0000 7f13c2800640 1 --2- 172.21.15.159:0/2699518912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13bc0ac430 0x7f13bc0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:52.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.578+0000 7f13c2800640 1 --2- 172.21.15.159:0/2699518912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13bc0aac20 0x7f13bc0b35d0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:52.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.578+0000 7f13c2800640 1 --2- 172.21.15.159:0/2699518912 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a40051c0 0x7f13bc0cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:52.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.578+0000 7f13c2800640 1 -- 172.21.15.159:0/2699518912 >> 172.21.15.159:0/2699518912 conn(0x7f13bc0676d0 msgr2=0x7f13bc15c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:52.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.578+0000 7f13c2800640 1 -- 172.21.15.159:0/2699518912 shutdown_connections 2024-04-04T06:17:52.574 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.578+0000 7f13c2800640 1 -- 172.21.15.159:0/2699518912 wait complete. 2024-04-04T06:17:52.592 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:17:52.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.798+0000 7f8e21000640 1 Processor -- start 2024-04-04T06:17:52.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.799+0000 7f8e21000640 1 -- start start 2024-04-04T06:17:52.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.800+0000 7f8e21000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f8e1c0cf670 0x7f8e1c0d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:52.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.800+0000 7f8e21000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e1c0afcc0 0x7f8e1c15c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:52.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.800+0000 7f8e21000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e1c0cfdf0 con 0x7f8e1c15c310 2024-04-04T06:17:52.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.800+0000 7f8e21000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f8e1c072f00 con 0x7f8e1c0cf670 2024-04-04T06:17:52.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.800+0000 7f8e21000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8e1c0731e0 con 0x7f8e1c0afcc0 2024-04-04T06:17:52.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.800+0000 7f8e1b400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f8e1c15c310 0x7f8e1c06ead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:42952/0 (socket says 172.21.15.159:42952) 2024-04-04T06:17:52.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.800+0000 7f8e1b400640 1 -- 172.21.15.159:0/812233130 learned_addr learned my addr 172.21.15.159:0/812233130 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:52.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.800+0000 7f8e12a00640 1 --2- 172.21.15.159:0/812233130 >> v2:172.21.15.159:3300/0 conn(0x7f8e1c0cf670 0x7f8e1c0d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:52.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.801+0000 7f8e1be00640 1 --2- 172.21.15.159:0/812233130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e1c0afcc0 0x7f8e1c15c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:52.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.801+0000 7f8e12a00640 1 -- 172.21.15.159:0/812233130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e1c0afcc0 msgr2=0x7f8e1c15c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:52.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.801+0000 7f8e12a00640 1 --2- 172.21.15.159:0/812233130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e1c0afcc0 0x7f8e1c15c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:52.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.801+0000 7f8e12a00640 1 -- 172.21.15.159:0/812233130 >> v1:172.21.15.159:6789/0 conn(0x7f8e1c15c310 legacy=0x7f8e1c06ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:52.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.801+0000 7f8e12a00640 1 -- 172.21.15.159:0/812233130 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1c0b04d0 con 0x7f8e1c0cf670 2024-04-04T06:17:52.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.801+0000 7f8e1aa00640 1 -- 172.21.15.159:0/812233130 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3749482257 0 0) 0x7f8e1c0cfdf0 con 0x7f8e1c15c310 2024-04-04T06:17:52.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.802+0000 7f8e12a00640 1 --2- 172.21.15.159:0/812233130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e1c0cf670 0x7f8e1c0d03c0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f8e040097a0 tx=0x7f8e0402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fcafaef4f16e30f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:52.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.802+0000 7f8e1aa00640 1 -- 172.21.15.159:0/812233130 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04035020 con 0x7f8e1c0cf670 2024-04-04T06:17:52.797 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.802+0000 7f8e1aa00640 1 -- 172.21.15.159:0/812233130 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e04031830 con 0x7f8e1c0cf670 2024-04-04T06:17:52.799 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.802+0000 7f8e1aa00640 1 -- 172.21.15.159:0/812233130 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04031c00 con 0x7f8e1c0cf670 2024-04-04T06:17:52.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.803+0000 7f8e21000640 1 -- 172.21.15.159:0/812233130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e1c0cf670 msgr2=0x7f8e1c0d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:52.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.803+0000 7f8e21000640 1 --2- 172.21.15.159:0/812233130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e1c0cf670 0x7f8e1c0d03c0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f8e040097a0 tx=0x7f8e0402b8d0 comp rx=0 tx=0).stop 2024-04-04T06:17:52.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.803+0000 7f8e21000640 1 -- 172.21.15.159:0/812233130 shutdown_connections 2024-04-04T06:17:52.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.803+0000 7f8e21000640 1 --2- 172.21.15.159:0/812233130 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e1c0afcc0 0x7f8e1c15c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:52.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.803+0000 7f8e21000640 1 --2- 172.21.15.159:0/812233130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e1c0cf670 0x7f8e1c0d03c0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:52.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.803+0000 7f8e21000640 1 -- 172.21.15.159:0/812233130 >> 172.21.15.159:0/812233130 conn(0x7f8e1c071e70 msgr2=0x7f8e1c072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:52.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.803+0000 7f8e21000640 1 -- 172.21.15.159:0/812233130 shutdown_connections 2024-04-04T06:17:52.800 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.803+0000 7f8e21000640 1 -- 172.21.15.159:0/812233130 wait complete. 2024-04-04T06:17:52.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.805+0000 7f8e21000640 1 Processor -- start 2024-04-04T06:17:52.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.805+0000 7f8e21000640 1 -- start start 2024-04-04T06:17:52.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.806+0000 7f8e21000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e1c0afcc0 0x7f8e1c1e4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:52.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.806+0000 7f8e21000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e1c0cf670 0x7f8e1c1e6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:52.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.806+0000 7f8e21000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e1c15c310 0x7f8e1c1463c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:52.802 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.806+0000 7f8e21000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c15cc30 con 0x7f8e1c0afcc0 2024-04-04T06:17:52.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.806+0000 7f8e21000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c06f1e0 con 0x7f8e1c15c310 2024-04-04T06:17:52.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.806+0000 7f8e21000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8e1c15cdb0 con 0x7f8e1c0cf670 2024-04-04T06:17:52.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.806+0000 7f8e1b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e1c0afcc0 0x7f8e1c1e4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:52.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.806+0000 7f8e1b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e1c0afcc0 0x7f8e1c1e4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:46090/0 (socket says 172.21.15.159:46090) 2024-04-04T06:17:52.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.806+0000 7f8e1b400640 1 -- 172.21.15.159:0/671423223 learned_addr learned my addr 172.21.15.159:0/671423223 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:52.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.807+0000 7f8e1b400640 1 -- 172.21.15.159:0/671423223 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e1c0cf670 msgr2=0x7f8e1c1e6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:52.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.807+0000 7f8e1b400640 1 --2- 172.21.15.159:0/671423223 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e1c0cf670 0x7f8e1c1e6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:52.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.807+0000 7f8e1b400640 1 -- 172.21.15.159:0/671423223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e1c15c310 msgr2=0x7f8e1c1463c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:52.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.807+0000 7f8e1b400640 1 --2- 172.21.15.159:0/671423223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e1c15c310 0x7f8e1c1463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:52.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.807+0000 7f8e1b400640 1 -- 172.21.15.159:0/671423223 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1c1e4ff0 con 0x7f8e1c0afcc0 2024-04-04T06:17:52.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.807+0000 7f8e1b400640 1 --2- 172.21.15.159:0/671423223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e1c0afcc0 0x7f8e1c1e4960 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00c460 tx=0x7f8e0c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:52.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.807+0000 7f8e13e00640 1 -- 172.21.15.159:0/671423223 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c018070 con 0x7f8e1c0afcc0 2024-04-04T06:17:52.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.807+0000 7f8e13e00640 1 -- 172.21.15.159:0/671423223 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e0c007500 con 0x7f8e1c0afcc0 2024-04-04T06:17:52.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.807+0000 7f8e13e00640 1 -- 172.21.15.159:0/671423223 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c004a20 con 0x7f8e1c0afcc0 2024-04-04T06:17:52.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.807+0000 7f8e21000640 1 -- 172.21.15.159:0/671423223 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e1c1e52d0 con 0x7f8e1c0afcc0 2024-04-04T06:17:52.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.807+0000 7f8e21000640 1 -- 172.21.15.159:0/671423223 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e1c1e5890 con 0x7f8e1c0afcc0 2024-04-04T06:17:52.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.809+0000 7f8e21000640 1 -- 172.21.15.159:0/671423223 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e1c075940 con 0x7f8e1c0afcc0 2024-04-04T06:17:52.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.810+0000 7f8e13e00640 1 -- 172.21.15.159:0/671423223 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e0c004bc0 con 0x7f8e1c0afcc0 2024-04-04T06:17:52.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.811+0000 7f8e13e00640 1 --2- 172.21.15.159:0/671423223 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8df806c920 0x7f8df806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:52.805 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.811+0000 7f8e13e00640 1 -- 172.21.15.159:0/671423223 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f8e0c08ae90 con 0x7f8e1c0afcc0 2024-04-04T06:17:52.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.811+0000 7f8e12a00640 1 --2- 172.21.15.159:0/671423223 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8df806c920 0x7f8df806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:52.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.812+0000 7f8e12a00640 1 --2- 172.21.15.159:0/671423223 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8df806c920 0x7f8df806ede0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8e04036000 tx=0x7f8e04030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:52.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:52.816+0000 7f8e13e00640 1 -- 172.21.15.159:0/671423223 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f8e0c059260 con 0x7f8e1c0afcc0 2024-04-04T06:17:53.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.140+0000 7f8e21000640 1 -- 172.21.15.159:0/671423223 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e1c0674a0 con 0x7f8e1c0afcc0 2024-04-04T06:17:53.138 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.141+0000 7f8e13e00640 1 -- 172.21.15.159:0/671423223 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5197 (secure 0 0 0) 0x7f8e0c058b60 con 0x7f8e1c0afcc0 2024-04-04T06:17:53.139 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:53.139 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":114,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5732,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1498324773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1498324773},{"type":"v1","addr":"172.21.15.178:6837","nonce":1498324773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2455031662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":2455031662},{"type":"v1","addr":"172.21.15.159:6835","nonce":2455031662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6343,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1823228151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1823228151},{"type":"v1","addr":"172.21.15.178:6836","nonce":1823228151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-04T06:16:57.277344+0000","modified":"2024-04-04T06:17:40.711526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9312":{"gid":9312,"name":"a","rank":0,"incarnation":111,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6834/643543929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":643543929},{"type":"v1","addr":"172.21.15.159:6834","nonce":643543929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9312,"qdb_cluster":[9312]},"id":8}]} 2024-04-04T06:17:53.140 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 114 2024-04-04T06:17:53.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.144+0000 7f8e21000640 1 -- 172.21.15.159:0/671423223 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8df806c920 msgr2=0x7f8df806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:53.140 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.144+0000 7f8e21000640 1 --2- 172.21.15.159:0/671423223 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8df806c920 0x7f8df806ede0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8e04036000 tx=0x7f8e04030000 comp rx=0 tx=0).stop 2024-04-04T06:17:53.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.144+0000 7f8e21000640 1 -- 172.21.15.159:0/671423223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e1c0afcc0 msgr2=0x7f8e1c1e4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:53.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.144+0000 7f8e21000640 1 --2- 172.21.15.159:0/671423223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e1c0afcc0 0x7f8e1c1e4960 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00c460 tx=0x7f8e0c00c920 comp rx=0 tx=0).stop 2024-04-04T06:17:53.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.144+0000 7f8e21000640 1 -- 172.21.15.159:0/671423223 shutdown_connections 2024-04-04T06:17:53.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.144+0000 7f8e21000640 1 --2- 172.21.15.159:0/671423223 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f8df806c920 0x7f8df806ede0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:53.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.144+0000 7f8e21000640 1 --2- 172.21.15.159:0/671423223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e1c15c310 0x7f8e1c1463c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:53.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.144+0000 7f8e21000640 1 --2- 172.21.15.159:0/671423223 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8e1c0cf670 0x7f8e1c1e6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:53.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.144+0000 7f8e21000640 1 --2- 172.21.15.159:0/671423223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e1c0afcc0 0x7f8e1c1e4960 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:53.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.144+0000 7f8e21000640 1 -- 172.21.15.159:0/671423223 >> 172.21.15.159:0/671423223 conn(0x7f8e1c071e70 msgr2=0x7f8e1c06c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:53.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.145+0000 7f8e21000640 1 -- 172.21.15.159:0/671423223 shutdown_connections 2024-04-04T06:17:53.141 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.145+0000 7f8e21000640 1 -- 172.21.15.159:0/671423223 wait complete. 2024-04-04T06:17:53.157 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:17:53.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.357+0000 7f3a9a400640 1 Processor -- start 2024-04-04T06:17:53.353 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.357+0000 7f3a9a400640 1 -- start start 2024-04-04T06:17:53.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.358+0000 7f3a9a400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3a9415e700 0x7f3a940b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:53.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.358+0000 7f3a9a400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a940d1cf0 0x7f3a940b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:53.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.359+0000 7f3a9a400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a9415eae0 con 0x7f3a940d1610 2024-04-04T06:17:53.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.359+0000 7f3a9a400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f3a940cfeb0 con 0x7f3a9415e700 2024-04-04T06:17:53.354 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.359+0000 7f3a9a400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3a940d0190 con 0x7f3a940d1cf0 2024-04-04T06:17:53.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.359+0000 7f3a93400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3a9415e700 0x7f3a940b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:53.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.359+0000 7f3a8be00640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f3a940d1610 0x7f3a94070fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.159:39604/0 (socket says 172.21.15.159:39604) 2024-04-04T06:17:53.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.359+0000 7f3a8be00640 1 -- 172.21.15.159:0/2642841484 learned_addr learned my addr 172.21.15.159:0/2642841484 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:53.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.359+0000 7f3a93e00640 1 --2- 172.21.15.159:0/2642841484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a940d1cf0 0x7f3a940b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:53.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.360+0000 7f3a93400640 1 -- 172.21.15.159:0/2642841484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a940d1cf0 msgr2=0x7f3a940b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:53.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.360+0000 7f3a93400640 1 --2- 172.21.15.159:0/2642841484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a940d1cf0 0x7f3a940b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:53.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.360+0000 7f3a93400640 1 -- 172.21.15.159:0/2642841484 >> v1:172.21.15.178:6789/0 conn(0x7f3a940d1610 legacy=0x7f3a94070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:53.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.360+0000 7f3a93400640 1 -- 172.21.15.159:0/2642841484 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a940d0470 con 0x7f3a9415e700 2024-04-04T06:17:53.356 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.361+0000 7f3a93400640 1 --2- 172.21.15.159:0/2642841484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a9415e700 0x7f3a940b3e20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3a840097a0 tx=0x7f3a8402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ece1d21807d437e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:53.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.362+0000 7f3a92a00640 1 -- 172.21.15.159:0/2642841484 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a840315a0 con 0x7f3a9415e700 2024-04-04T06:17:53.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.362+0000 7f3a92a00640 1 -- 172.21.15.159:0/2642841484 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a84037070 con 0x7f3a9415e700 2024-04-04T06:17:53.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.362+0000 7f3a92a00640 1 -- 172.21.15.159:0/2642841484 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84031960 con 0x7f3a9415e700 2024-04-04T06:17:53.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.362+0000 7f3a9a400640 1 -- 172.21.15.159:0/2642841484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a9415e700 msgr2=0x7f3a940b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:53.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.362+0000 7f3a9a400640 1 --2- 172.21.15.159:0/2642841484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a9415e700 0x7f3a940b3e20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3a840097a0 tx=0x7f3a8402b840 comp rx=0 tx=0).stop 2024-04-04T06:17:53.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.362+0000 7f3a9a400640 1 -- 172.21.15.159:0/2642841484 shutdown_connections 2024-04-04T06:17:53.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.362+0000 7f3a9a400640 1 --2- 172.21.15.159:0/2642841484 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a940d1cf0 0x7f3a940b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:53.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.362+0000 7f3a9a400640 1 --2- 172.21.15.159:0/2642841484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a9415e700 0x7f3a940b3e20 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:53.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.362+0000 7f3a9a400640 1 -- 172.21.15.159:0/2642841484 >> 172.21.15.159:0/2642841484 conn(0x7f3a940d2240 msgr2=0x7f3a940d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:53.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.362+0000 7f3a9a400640 1 -- 172.21.15.159:0/2642841484 shutdown_connections 2024-04-04T06:17:53.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.362+0000 7f3a9a400640 1 -- 172.21.15.159:0/2642841484 wait complete. 2024-04-04T06:17:53.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.364+0000 7f3a9a400640 1 Processor -- start 2024-04-04T06:17:53.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.364+0000 7f3a9a400640 1 -- start start 2024-04-04T06:17:53.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.365+0000 7f3a9a400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a940d1610 0x7f3a940b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:53.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.365+0000 7f3a9a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a940d1cf0 0x7f3a94157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:53.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.365+0000 7f3a9a400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a9415e700 0x7f3a941565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:53.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.365+0000 7f3a9a400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9406d1b0 con 0x7f3a940d1cf0 2024-04-04T06:17:53.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.365+0000 7f3a9a400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3a940d0a30 con 0x7f3a9415e700 2024-04-04T06:17:53.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.365+0000 7f3a9a400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3a9406d330 con 0x7f3a940d1610 2024-04-04T06:17:53.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.365+0000 7f3a8be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a940d1cf0 0x7f3a94157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:53.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.365+0000 7f3a8be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a940d1cf0 0x7f3a94157b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:46106/0 (socket says 172.21.15.159:46106) 2024-04-04T06:17:53.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.365+0000 7f3a8be00640 1 -- 172.21.15.159:0/1612021294 learned_addr learned my addr 172.21.15.159:0/1612021294 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:53.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.366+0000 7f3a93e00640 1 --2- 172.21.15.159:0/1612021294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a9415e700 0x7f3a941565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:53.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.366+0000 7f3a93400640 1 --2- 172.21.15.159:0/1612021294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a940d1610 0x7f3a940b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:53.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.366+0000 7f3a8be00640 1 -- 172.21.15.159:0/1612021294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a940d1610 msgr2=0x7f3a940b7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:53.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.366+0000 7f3a8be00640 1 --2- 172.21.15.159:0/1612021294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a940d1610 0x7f3a940b7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:53.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.366+0000 7f3a8be00640 1 -- 172.21.15.159:0/1612021294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a9415e700 msgr2=0x7f3a941565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:53.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.366+0000 7f3a8be00640 1 --2- 172.21.15.159:0/1612021294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a9415e700 0x7f3a941565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:53.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.366+0000 7f3a8be00640 1 -- 172.21.15.159:0/1612021294 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a941f7ec0 con 0x7f3a940d1cf0 2024-04-04T06:17:53.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.366+0000 7f3a8be00640 1 --2- 172.21.15.159:0/1612021294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a940d1cf0 0x7f3a94157b50 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00c440 tx=0x7f3a7c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:53.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.366+0000 7f3a8b400640 1 -- 172.21.15.159:0/1612021294 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c017070 con 0x7f3a940d1cf0 2024-04-04T06:17:53.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.366+0000 7f3a8b400640 1 -- 172.21.15.159:0/1612021294 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a7c005ce0 con 0x7f3a940d1cf0 2024-04-04T06:17:53.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.367+0000 7f3a9a400640 1 -- 172.21.15.159:0/1612021294 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a941f81a0 con 0x7f3a940d1cf0 2024-04-04T06:17:53.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.367+0000 7f3a8b400640 1 -- 172.21.15.159:0/1612021294 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c0072f0 con 0x7f3a940d1cf0 2024-04-04T06:17:53.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.367+0000 7f3a9a400640 1 -- 172.21.15.159:0/1612021294 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a94153720 con 0x7f3a940d1cf0 2024-04-04T06:17:53.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.368+0000 7f3a9a400640 1 -- 172.21.15.159:0/1612021294 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a940685c0 con 0x7f3a940d1cf0 2024-04-04T06:17:53.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.369+0000 7f3a8b400640 1 -- 172.21.15.159:0/1612021294 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a7c01a070 con 0x7f3a940d1cf0 2024-04-04T06:17:53.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.370+0000 7f3a8b400640 1 --2- 172.21.15.159:0/1612021294 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3a7006c8f0 0x7f3a7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:53.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.370+0000 7f3a8b400640 1 -- 172.21.15.159:0/1612021294 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f3a7c08af10 con 0x7f3a940d1cf0 2024-04-04T06:17:53.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.370+0000 7f3a93400640 1 --2- 172.21.15.159:0/1612021294 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3a7006c8f0 0x7f3a7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:53.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.371+0000 7f3a93400640 1 --2- 172.21.15.159:0/1612021294 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3a7006c8f0 0x7f3a7006edb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3a84036000 tx=0x7f3a8403d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:53.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.374+0000 7f3a8b400640 1 -- 172.21.15.159:0/1612021294 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f3a7c0592e0 con 0x7f3a940d1cf0 2024-04-04T06:17:53.570 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.575+0000 7f3a9a400640 1 -- 172.21.15.159:0/1612021294 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a940b8470 con 0x7f3a940d1cf0 2024-04-04T06:17:53.573 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.578+0000 7f3a8b400640 1 -- 172.21.15.159:0/1612021294 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+16995 (secure 0 0 0) 0x7f3a7c058be0 con 0x7f3a940d1cf0 2024-04-04T06:17:53.574 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:53.574 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":116,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:17:35.667542+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-04T06:16:55.036975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-04T06:16:56.239022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.b","mappings":[{"from":0,"to":5}]},{"pgid":"17.26","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6834/2439261212":"2024-04-05T06:17:35.667507+0000","172.21.15.159:6832/2439261212":"2024-04-05T06:17:35.667507+0000","172.21.15.159:6834/1017894858":"2024-04-05T06:17:21.596764+0000","172.21.15.159:6832/1017894858":"2024-04-05T06:17:21.596764+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:17:53.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.580+0000 7f3a9a400640 1 -- 172.21.15.159:0/1612021294 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3a7006c8f0 msgr2=0x7f3a7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:53.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.580+0000 7f3a9a400640 1 --2- 172.21.15.159:0/1612021294 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3a7006c8f0 0x7f3a7006edb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3a84036000 tx=0x7f3a8403d040 comp rx=0 tx=0).stop 2024-04-04T06:17:53.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.580+0000 7f3a9a400640 1 -- 172.21.15.159:0/1612021294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a940d1cf0 msgr2=0x7f3a94157b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:53.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.580+0000 7f3a9a400640 1 --2- 172.21.15.159:0/1612021294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a940d1cf0 0x7f3a94157b50 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00c440 tx=0x7f3a7c00c900 comp rx=0 tx=0).stop 2024-04-04T06:17:53.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.581+0000 7f3a9a400640 1 -- 172.21.15.159:0/1612021294 shutdown_connections 2024-04-04T06:17:53.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.581+0000 7f3a9a400640 1 --2- 172.21.15.159:0/1612021294 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f3a7006c8f0 0x7f3a7006edb0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:53.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.581+0000 7f3a9a400640 1 --2- 172.21.15.159:0/1612021294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a9415e700 0x7f3a941565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:53.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.581+0000 7f3a9a400640 1 --2- 172.21.15.159:0/1612021294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a940d1cf0 0x7f3a94157b50 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:53.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.581+0000 7f3a9a400640 1 --2- 172.21.15.159:0/1612021294 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a940d1610 0x7f3a940b7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:53.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.581+0000 7f3a9a400640 1 -- 172.21.15.159:0/1612021294 >> 172.21.15.159:0/1612021294 conn(0x7f3a940d2240 msgr2=0x7f3a94160fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:53.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.581+0000 7f3a9a400640 1 -- 172.21.15.159:0/1612021294 shutdown_connections 2024-04-04T06:17:53.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.581+0000 7f3a9a400640 1 -- 172.21.15.159:0/1612021294 wait complete. 2024-04-04T06:17:53.597 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-04T06:17:53.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.820+0000 7f847a200640 1 Processor -- start 2024-04-04T06:17:53.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.820+0000 7f847a200640 1 -- start start 2024-04-04T06:17:53.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.822+0000 7f847a200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f84741457e0 0x7f84740b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:53.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.822+0000 7f847a200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84740b4580 0x7f847406b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:53.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.822+0000 7f847a200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84740b5ed0 con 0x7f8474160a30 2024-04-04T06:17:53.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.822+0000 7f847a200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f8474145bc0 con 0x7f84741457e0 2024-04-04T06:17:53.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.822+0000 7f847a200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f84740afca0 con 0x7f84740b4580 2024-04-04T06:17:53.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.822+0000 7f8473400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f84741457e0 0x7f84740b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:53.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.822+0000 7f8473e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f8474160a30 0x7f84740b0130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:42962/0 (socket says 172.21.15.159:42962) 2024-04-04T06:17:53.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.822+0000 7f8473e00640 1 -- 172.21.15.159:0/2830470339 learned_addr learned my addr 172.21.15.159:0/2830470339 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:53.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.822+0000 7f8472a00640 1 --2- 172.21.15.159:0/2830470339 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84740b4580 0x7f847406b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:53.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.823+0000 7f8473400640 1 -- 172.21.15.159:0/2830470339 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84740b4580 msgr2=0x7f847406b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:53.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.823+0000 7f8473400640 1 --2- 172.21.15.159:0/2830470339 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84740b4580 0x7f847406b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:53.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.823+0000 7f8473400640 1 -- 172.21.15.159:0/2830470339 >> v1:172.21.15.159:6789/0 conn(0x7f8474160a30 legacy=0x7f84740b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:53.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.823+0000 7f8473400640 1 -- 172.21.15.159:0/2830470339 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84740b61b0 con 0x7f84741457e0 2024-04-04T06:17:53.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.823+0000 7f8472000640 1 -- 172.21.15.159:0/2830470339 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3874658755 0 0) 0x7f84740b5ed0 con 0x7f8474160a30 2024-04-04T06:17:53.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.824+0000 7f8473400640 1 --2- 172.21.15.159:0/2830470339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84741457e0 0x7f84740b4040 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f84640097a0 tx=0x7f846402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2359c9832a7049 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:53.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.824+0000 7f8472000640 1 -- 172.21.15.159:0/2830470339 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464031780 con 0x7f84741457e0 2024-04-04T06:17:53.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.824+0000 7f8472000640 1 -- 172.21.15.159:0/2830470339 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8464037070 con 0x7f84741457e0 2024-04-04T06:17:53.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.824+0000 7f8472000640 1 -- 172.21.15.159:0/2830470339 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464031b40 con 0x7f84741457e0 2024-04-04T06:17:53.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.825+0000 7f847a200640 1 -- 172.21.15.159:0/2830470339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84741457e0 msgr2=0x7f84740b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:53.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.825+0000 7f847a200640 1 --2- 172.21.15.159:0/2830470339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84741457e0 0x7f84740b4040 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f84640097a0 tx=0x7f846402b820 comp rx=0 tx=0).stop 2024-04-04T06:17:53.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.825+0000 7f847a200640 1 -- 172.21.15.159:0/2830470339 shutdown_connections 2024-04-04T06:17:53.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.825+0000 7f847a200640 1 --2- 172.21.15.159:0/2830470339 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84740b4580 0x7f847406b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:53.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.825+0000 7f847a200640 1 --2- 172.21.15.159:0/2830470339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84741457e0 0x7f84740b4040 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:53.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.825+0000 7f847a200640 1 -- 172.21.15.159:0/2830470339 >> 172.21.15.159:0/2830470339 conn(0x7f84740b65a0 msgr2=0x7f84740b69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:53.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.825+0000 7f847a200640 1 -- 172.21.15.159:0/2830470339 shutdown_connections 2024-04-04T06:17:53.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.825+0000 7f847a200640 1 -- 172.21.15.159:0/2830470339 wait complete. 2024-04-04T06:17:53.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.826+0000 7f847a200640 1 Processor -- start 2024-04-04T06:17:53.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.827+0000 7f847a200640 1 -- start start 2024-04-04T06:17:53.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.827+0000 7f847a200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84740b4580 0x7f847409cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:53.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.827+0000 7f847a200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84741457e0 0x7f8474134db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:53.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.827+0000 7f847a200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8474160a30 0x7f847409e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:53.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.828+0000 7f847a200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84740ae360 con 0x7f84741457e0 2024-04-04T06:17:53.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.828+0000 7f847a200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f84740ae1e0 con 0x7f84740b4580 2024-04-04T06:17:53.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.828+0000 7f847a200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f84740b0840 con 0x7f8474160a30 2024-04-04T06:17:53.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.828+0000 7f8472a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84741457e0 0x7f8474134db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:53.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.828+0000 7f8472a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84741457e0 0x7f8474134db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:46132/0 (socket says 172.21.15.159:46132) 2024-04-04T06:17:53.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.828+0000 7f8472a00640 1 -- 172.21.15.159:0/4096382399 learned_addr learned my addr 172.21.15.159:0/4096382399 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:53.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.828+0000 7f8473400640 1 --2- 172.21.15.159:0/4096382399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84740b4580 0x7f847409cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:53.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.828+0000 7f8472a00640 1 -- 172.21.15.159:0/4096382399 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8474160a30 msgr2=0x7f847409e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:53.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.828+0000 7f8472a00640 1 --2- 172.21.15.159:0/4096382399 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8474160a30 0x7f847409e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:53.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.828+0000 7f8472a00640 1 -- 172.21.15.159:0/4096382399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84740b4580 msgr2=0x7f847409cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:53.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.828+0000 7f8472a00640 1 --2- 172.21.15.159:0/4096382399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84740b4580 0x7f847409cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:53.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.828+0000 7f8472a00640 1 -- 172.21.15.159:0/4096382399 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84741352f0 con 0x7f84741457e0 2024-04-04T06:17:53.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.829+0000 7f8472a00640 1 --2- 172.21.15.159:0/4096382399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84741457e0 0x7f8474134db0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f846000d830 tx=0x7f846000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:53.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.829+0000 7f846b400640 1 -- 172.21.15.159:0/4096382399 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846000b6c0 con 0x7f84741457e0 2024-04-04T06:17:53.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.829+0000 7f846b400640 1 -- 172.21.15.159:0/4096382399 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8460019070 con 0x7f84741457e0 2024-04-04T06:17:53.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.829+0000 7f847a200640 1 -- 172.21.15.159:0/4096382399 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f847415f4c0 con 0x7f84741457e0 2024-04-04T06:17:53.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.829+0000 7f846b400640 1 -- 172.21.15.159:0/4096382399 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846000baa0 con 0x7f84741457e0 2024-04-04T06:17:53.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.829+0000 7f847a200640 1 -- 172.21.15.159:0/4096382399 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f847415fa80 con 0x7f84741457e0 2024-04-04T06:17:53.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.830+0000 7f847a200640 1 -- 172.21.15.159:0/4096382399 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8474068cd0 con 0x7f84741457e0 2024-04-04T06:17:53.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.831+0000 7f846b400640 1 -- 172.21.15.159:0/4096382399 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f846001e460 con 0x7f84741457e0 2024-04-04T06:17:53.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.832+0000 7f846b400640 1 --2- 172.21.15.159:0/4096382399 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f845406c8f0 0x7f845406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:53.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.832+0000 7f846b400640 1 -- 172.21.15.159:0/4096382399 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f846008c360 con 0x7f84741457e0 2024-04-04T06:17:53.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.833+0000 7f8473400640 1 --2- 172.21.15.159:0/4096382399 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f845406c8f0 0x7f845406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:53.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.834+0000 7f8473400640 1 --2- 172.21.15.159:0/4096382399 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f845406c8f0 0x7f845406edb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8464036000 tx=0x7f8464008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:53.831 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:53.836+0000 7f846b400640 1 -- 172.21.15.159:0/4096382399 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f84741352f0 con 0x7f84741457e0 2024-04-04T06:17:54.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.116+0000 7f847a200640 1 -- 172.21.15.159:0/4096382399 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8474160590 con 0x7f84741457e0 2024-04-04T06:17:54.111 INFO:tasks.ceph.mon.a.smithi159.stderr:2024-04-04T06:17:54.117+0000 7f1995c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-04T06:17:54.115 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.120+0000 7f846b400640 1 -- 172.21.15.159:0/4096382399 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v115) ==== 161+0+0 (secure 0 0 0) 0x7f846005a690 con 0x7f84741457e0 2024-04-04T06:17:54.115 INFO:teuthology.orchestra.run.smithi159.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-04T06:17:54.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.124+0000 7f847a200640 1 -- 172.21.15.159:0/4096382399 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f845406c8f0 msgr2=0x7f845406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:54.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.124+0000 7f847a200640 1 --2- 172.21.15.159:0/4096382399 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f845406c8f0 0x7f845406edb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8464036000 tx=0x7f8464008030 comp rx=0 tx=0).stop 2024-04-04T06:17:54.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.124+0000 7f847a200640 1 -- 172.21.15.159:0/4096382399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84741457e0 msgr2=0x7f8474134db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:54.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.124+0000 7f847a200640 1 --2- 172.21.15.159:0/4096382399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84741457e0 0x7f8474134db0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f846000d830 tx=0x7f846000dcf0 comp rx=0 tx=0).stop 2024-04-04T06:17:54.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.125+0000 7f847a200640 1 -- 172.21.15.159:0/4096382399 shutdown_connections 2024-04-04T06:17:54.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.125+0000 7f847a200640 1 --2- 172.21.15.159:0/4096382399 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8474160a30 0x7f847409e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:54.120 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.125+0000 7f847a200640 1 --2- 172.21.15.159:0/4096382399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84741457e0 0x7f8474134db0 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:54.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.125+0000 7f847a200640 1 --2- 172.21.15.159:0/4096382399 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f845406c8f0 0x7f845406edb0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:54.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.125+0000 7f847a200640 1 --2- 172.21.15.159:0/4096382399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84740b4580 0x7f847409cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:54.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.125+0000 7f847a200640 1 -- 172.21.15.159:0/4096382399 >> 172.21.15.159:0/4096382399 conn(0x7f84740b65a0 msgr2=0x7f847406fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:54.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.125+0000 7f847a200640 1 -- 172.21.15.159:0/4096382399 shutdown_connections 2024-04-04T06:17:54.121 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.125+0000 7f847a200640 1 -- 172.21.15.159:0/4096382399 wait complete. 2024-04-04T06:17:54.136 INFO:tasks.ceph.mds.a.smithi159.stderr: -1498> 2024-04-04T06:17:41.684+0000 7f272e600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-04T06:17:54.137 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-04T06:17:54.195 INFO:tasks.ceph.mds.a.smithi159.stdout:starting mds.a at 2024-04-04T06:17:54.326 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.331+0000 7f05e0c00640 1 Processor -- start 2024-04-04T06:17:54.327 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.331+0000 7f05e0c00640 1 -- start start 2024-04-04T06:17:54.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.333+0000 7f05e0c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05dc0d0280 0x7f05dc0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:54.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.333+0000 7f05e0c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f05dc141c10 0x7f05dc071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:54.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.333+0000 7f05e0c00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05dc15ec80 con 0x7f05dc071eb0 2024-04-04T06:17:54.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.333+0000 7f05e0c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f05dc0d0ba0 con 0x7f05dc141c10 2024-04-04T06:17:54.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.333+0000 7f05e0c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f05dc15ef20 con 0x7f05dc0d0280 2024-04-04T06:17:54.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.333+0000 7f05da800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f05dc071eb0 0x7f05dc06d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52190/0 (socket says 172.21.15.159:52190) 2024-04-04T06:17:54.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.333+0000 7f05da800640 1 -- 172.21.15.159:0/2513558087 learned_addr learned my addr 172.21.15.159:0/2513558087 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:54.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.333+0000 7f05d1400640 1 --2- 172.21.15.159:0/2513558087 >> v2:172.21.15.178:3300/0 conn(0x7f05dc141c10 0x7f05dc071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:54.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.333+0000 7f05d9e00640 1 --2- 172.21.15.159:0/2513558087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05dc0d0280 0x7f05dc0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:54.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.334+0000 7f05d9400640 1 -- 172.21.15.159:0/2513558087 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4017620218 0 0) 0x7f05dc15ec80 con 0x7f05dc071eb0 2024-04-04T06:17:54.330 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.334+0000 7f05d9400640 1 -- 172.21.15.159:0/2513558087 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05c0003520 con 0x7f05dc071eb0 2024-04-04T06:17:54.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.334+0000 7f05d9400640 1 -- 172.21.15.159:0/2513558087 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1992965912 0 0) 0x7f05c0003520 con 0x7f05dc071eb0 2024-04-04T06:17:54.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.334+0000 7f05d9400640 1 -- 172.21.15.159:0/2513558087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05dc0d0280 msgr2=0x7f05dc0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:54.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.334+0000 7f05d9400640 1 --2- 172.21.15.159:0/2513558087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05dc0d0280 0x7f05dc0d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:54.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.334+0000 7f05d9400640 1 -- 172.21.15.159:0/2513558087 >> v2:172.21.15.178:3300/0 conn(0x7f05dc141c10 msgr2=0x7f05dc071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:54.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.334+0000 7f05d9400640 1 --2- 172.21.15.159:0/2513558087 >> v2:172.21.15.178:3300/0 conn(0x7f05dc141c10 0x7f05dc071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:54.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.334+0000 7f05d9400640 1 -- 172.21.15.159:0/2513558087 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05dc15f200 con 0x7f05dc071eb0 2024-04-04T06:17:54.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.334+0000 7f05d9400640 1 -- 172.21.15.159:0/2513558087 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f05d4001670 con 0x7f05dc071eb0 2024-04-04T06:17:54.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.334+0000 7f05d9400640 1 -- 172.21.15.159:0/2513558087 >> v1:172.21.15.159:6789/0 conn(0x7f05dc071eb0 legacy=0x7f05dc06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:54.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.334+0000 7f05d9400640 1 --2- 172.21.15.159:0/2513558087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05c00039b0 0x7f05c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:54.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.334+0000 7f05d9400640 1 --2- 172.21.15.159:0/2513558087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05dc0d0280 0x7f05c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:54.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.334+0000 7f05d9400640 1 --2- 172.21.15.159:0/2513558087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05c0004d40 0x7f05c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:54.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.334+0000 7f05d9400640 1 -- 172.21.15.159:0/2513558087 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc0d0ba0 con 0x7f05c0004d40 2024-04-04T06:17:54.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.334+0000 7f05d9400640 1 -- 172.21.15.159:0/2513558087 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc15ef20 con 0x7f05c00039b0 2024-04-04T06:17:54.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.334+0000 7f05d9400640 1 -- 172.21.15.159:0/2513558087 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f05c000b9c0 con 0x7f05dc0d0280 2024-04-04T06:17:54.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.335+0000 7f05da800640 1 --2- 172.21.15.159:0/2513558087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05c0004d40 0x7f05c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:54.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.335+0000 7f05d1400640 1 --2- 172.21.15.159:0/2513558087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05dc0d0280 0x7f05c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:54.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.335+0000 7f05d9e00640 1 --2- 172.21.15.159:0/2513558087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05c00039b0 0x7f05c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:54.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.335+0000 7f05da800640 1 -- 172.21.15.159:0/2513558087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05dc0d0280 msgr2=0x7f05c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:54.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.335+0000 7f05da800640 1 --2- 172.21.15.159:0/2513558087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05dc0d0280 0x7f05c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:54.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.335+0000 7f05da800640 1 -- 172.21.15.159:0/2513558087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05c00039b0 msgr2=0x7f05c0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:54.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.335+0000 7f05da800640 1 --2- 172.21.15.159:0/2513558087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05c00039b0 0x7f05c0003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:54.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.335+0000 7f05da800640 1 -- 172.21.15.159:0/2513558087 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c0003520 con 0x7f05c0004d40 2024-04-04T06:17:54.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.335+0000 7f05d1400640 1 --2- 172.21.15.159:0/2513558087 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05dc0d0280 0x7f05c0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:54.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.336+0000 7f05da800640 1 --2- 172.21.15.159:0/2513558087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05c0004d40 0x7f05c00051c0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f05d4007fd0 tx=0x7f05c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c16567aecd9608 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:54.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.336+0000 7f05d9400640 1 -- 172.21.15.159:0/2513558087 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4007d50 con 0x7f05c0004d40 2024-04-04T06:17:54.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.336+0000 7f05d9400640 1 -- 172.21.15.159:0/2513558087 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05d4030070 con 0x7f05c0004d40 2024-04-04T06:17:54.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.336+0000 7f05d9400640 1 -- 172.21.15.159:0/2513558087 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d402cd40 con 0x7f05c0004d40 2024-04-04T06:17:54.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.337+0000 7f05e0c00640 1 -- 172.21.15.159:0/2513558087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05c0004d40 msgr2=0x7f05c00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:54.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.337+0000 7f05e0c00640 1 --2- 172.21.15.159:0/2513558087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05c0004d40 0x7f05c00051c0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f05d4007fd0 tx=0x7f05c0005830 comp rx=0 tx=0).stop 2024-04-04T06:17:54.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.337+0000 7f05da800640 1 -- 172.21.15.159:0/2513558087 reap_dead start 2024-04-04T06:17:54.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.337+0000 7f05e0c00640 1 -- 172.21.15.159:0/2513558087 shutdown_connections 2024-04-04T06:17:54.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.337+0000 7f05e0c00640 1 -- 172.21.15.159:0/2513558087 >> 172.21.15.159:0/2513558087 conn(0x7f05dc06b750 msgr2=0x7f05dc06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:54.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.337+0000 7f05e0c00640 1 -- 172.21.15.159:0/2513558087 shutdown_connections 2024-04-04T06:17:54.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.337+0000 7f05e0c00640 1 -- 172.21.15.159:0/2513558087 wait complete. 2024-04-04T06:17:54.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.339+0000 7f05e0c00640 1 Processor -- start 2024-04-04T06:17:54.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.339+0000 7f05e0c00640 1 -- start start 2024-04-04T06:17:54.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.340+0000 7f05e0c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05c0004d40 0x7f05dc09a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:54.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.340+0000 7f05e0c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05dc1559e0 0x7f05dc155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:54.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.340+0000 7f05e0c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05dc0af090 0x7f05dc0af510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.340+0000 7f05e0c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc158e90 con 0x7f05dc1559e0 2024-04-04T06:17:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.340+0000 7f05e0c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc15f7c0 con 0x7f05c0004d40 2024-04-04T06:17:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.340+0000 7f05e0c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f05dc1563a0 con 0x7f05dc0af090 2024-04-04T06:17:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.340+0000 7f05d1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05dc1559e0 0x7f05dc155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.340+0000 7f05d1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05dc1559e0 0x7f05dc155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:54960/0 (socket says 172.21.15.159:54960) 2024-04-04T06:17:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.340+0000 7f05d1400640 1 -- 172.21.15.159:0/4019891434 learned_addr learned my addr 172.21.15.159:0/4019891434 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.341+0000 7f05d1400640 1 -- 172.21.15.159:0/4019891434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05dc0af090 msgr2=0x7f05dc0af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.341+0000 7f05d1400640 1 --2- 172.21.15.159:0/4019891434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05dc0af090 0x7f05dc0af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.341+0000 7f05d9e00640 1 --2- 172.21.15.159:0/4019891434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05c0004d40 0x7f05dc09a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.341+0000 7f05d1400640 1 -- 172.21.15.159:0/4019891434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05c0004d40 msgr2=0x7f05dc09a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.341+0000 7f05d1400640 1 --2- 172.21.15.159:0/4019891434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05c0004d40 0x7f05dc09a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.341+0000 7f05d1400640 1 -- 172.21.15.159:0/4019891434 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05dc0adde0 con 0x7f05dc1559e0 2024-04-04T06:17:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.341+0000 7f05d1400640 1 --2- 172.21.15.159:0/4019891434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05dc1559e0 0x7f05dc155e60 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f05c4010880 tx=0x7f05c4010d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.341+0000 7f05d2a00640 1 -- 172.21.15.159:0/4019891434 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c400f040 con 0x7f05dc1559e0 2024-04-04T06:17:54.337 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.341+0000 7f05d2a00640 1 -- 172.21.15.159:0/4019891434 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c400eb60 con 0x7f05dc1559e0 2024-04-04T06:17:54.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.341+0000 7f05d2a00640 1 -- 172.21.15.159:0/4019891434 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c401c400 con 0x7f05dc1559e0 2024-04-04T06:17:54.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.341+0000 7f05e0c00640 1 -- 172.21.15.159:0/4019891434 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05dc0ae0c0 con 0x7f05dc1559e0 2024-04-04T06:17:54.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.341+0000 7f05e0c00640 1 -- 172.21.15.159:0/4019891434 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05dc0ae680 con 0x7f05dc1559e0 2024-04-04T06:17:54.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.343+0000 7f05e0c00640 1 -- 172.21.15.159:0/4019891434 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05dc0685c0 con 0x7f05dc1559e0 2024-04-04T06:17:54.338 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.344+0000 7f05d2a00640 1 -- 172.21.15.159:0/4019891434 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05c401c5a0 con 0x7f05dc1559e0 2024-04-04T06:17:54.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.345+0000 7f05d2a00640 1 --2- 172.21.15.159:0/4019891434 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f05b806c8f0 0x7f05b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:54.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.345+0000 7f05d2a00640 1 -- 172.21.15.159:0/4019891434 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6025+0+0 (secure 0 0 0) 0x7f05c408c420 con 0x7f05dc1559e0 2024-04-04T06:17:54.340 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.346+0000 7f05d9e00640 1 --2- 172.21.15.159:0/4019891434 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f05b806c8f0 0x7f05b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:54.342 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.347+0000 7f05d9e00640 1 --2- 172.21.15.159:0/4019891434 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f05b806c8f0 0x7f05b806edb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f05cc003a50 tx=0x7f05cc00b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:54.344 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.350+0000 7f05d2a00640 1 -- 172.21.15.159:0/4019891434 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f05c4018030 con 0x7f05dc1559e0 2024-04-04T06:17:54.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:54.581+0000 7f05e0c00640 1 -- 172.21.15.159:0/4019891434 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f05dc0cfdb0 con 0x7f05dc1559e0 2024-04-04T06:17:55.119 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.124+0000 7f05d2a00640 1 -- 172.21.15.159:0/4019891434 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v116) ==== 104+0+0 (secure 0 0 0) 0x7f05c405a020 con 0x7f05dc1559e0 2024-04-04T06:17:55.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.127+0000 7f05e0c00640 1 -- 172.21.15.159:0/4019891434 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f05b806c8f0 msgr2=0x7f05b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:55.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.127+0000 7f05e0c00640 1 --2- 172.21.15.159:0/4019891434 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f05b806c8f0 0x7f05b806edb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f05cc003a50 tx=0x7f05cc00b040 comp rx=0 tx=0).stop 2024-04-04T06:17:55.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.128+0000 7f05e0c00640 1 -- 172.21.15.159:0/4019891434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05dc1559e0 msgr2=0x7f05dc155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:55.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.128+0000 7f05e0c00640 1 --2- 172.21.15.159:0/4019891434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05dc1559e0 0x7f05dc155e60 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f05c4010880 tx=0x7f05c4010d40 comp rx=0 tx=0).stop 2024-04-04T06:17:55.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.128+0000 7f05e0c00640 1 -- 172.21.15.159:0/4019891434 shutdown_connections 2024-04-04T06:17:55.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.128+0000 7f05e0c00640 1 --2- 172.21.15.159:0/4019891434 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f05dc0af090 0x7f05dc0af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:55.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.128+0000 7f05e0c00640 1 --2- 172.21.15.159:0/4019891434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05dc1559e0 0x7f05dc155e60 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:55.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.128+0000 7f05e0c00640 1 --2- 172.21.15.159:0/4019891434 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f05b806c8f0 0x7f05b806edb0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:55.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.128+0000 7f05e0c00640 1 --2- 172.21.15.159:0/4019891434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05c0004d40 0x7f05dc09a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:55.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.128+0000 7f05e0c00640 1 -- 172.21.15.159:0/4019891434 >> 172.21.15.159:0/4019891434 conn(0x7f05dc06b750 msgr2=0x7f05dc0b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:55.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.128+0000 7f05e0c00640 1 -- 172.21.15.159:0/4019891434 shutdown_connections 2024-04-04T06:17:55.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.128+0000 7f05e0c00640 1 -- 172.21.15.159:0/4019891434 wait complete. 2024-04-04T06:17:55.147 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-04T06:17:55.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.360+0000 7ff2bd400640 1 Processor -- start 2024-04-04T06:17:55.355 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.360+0000 7ff2bd400640 1 -- start start 2024-04-04T06:17:55.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.361+0000 7ff2bd400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff2b8072720 0x7ff2b8167300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:55.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.361+0000 7ff2bd400640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2b815cb90 con 0x7ff2b809dcf0 2024-04-04T06:17:55.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.361+0000 7ff2bd400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2b815ce70 con 0x7ff2b80acf10 2024-04-04T06:17:55.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.361+0000 7ff2bd400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7ff2b8145f60 con 0x7ff2b8072720 2024-04-04T06:17:55.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.362+0000 7ff2b7200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff2b8072720 0x7ff2b8167300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:55.357 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.362+0000 7ff2b5e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7ff2b809dcf0 0x7ff2b81453f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52198/0 (socket says 172.21.15.159:52198) 2024-04-04T06:17:55.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.362+0000 7ff2b7200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7ff2b8072720 0x7ff2b8167300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:54976/0 (socket says 172.21.15.159:54976) 2024-04-04T06:17:55.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.362+0000 7ff2b5e00640 1 -- 172.21.15.159:0/2692976645 learned_addr learned my addr 172.21.15.159:0/2692976645 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:55.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.362+0000 7ff2b7200640 1 -- 172.21.15.159:0/2692976645 >> v1:172.21.15.178:6789/0 conn(0x7ff2b80acf10 legacy=0x7ff2b80ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:55.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.362+0000 7ff2b7200640 1 -- 172.21.15.159:0/2692976645 >> v1:172.21.15.159:6789/0 conn(0x7ff2b809dcf0 legacy=0x7ff2b81453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:55.358 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.362+0000 7ff2b7200640 1 -- 172.21.15.159:0/2692976645 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b815d150 con 0x7ff2b8072720 2024-04-04T06:17:55.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.363+0000 7ff2b7200640 1 --2- 172.21.15.159:0/2692976645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2b8072720 0x7ff2b8167300 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009d90 tx=0x7ff2b002bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8387c51eff202a6c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:55.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.363+0000 7ff2b5400640 1 -- 172.21.15.159:0/2692976645 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0039070 con 0x7ff2b8072720 2024-04-04T06:17:55.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.363+0000 7ff2b5400640 1 -- 172.21.15.159:0/2692976645 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b002cb40 con 0x7ff2b8072720 2024-04-04T06:17:55.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.363+0000 7ff2b5400640 1 -- 172.21.15.159:0/2692976645 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0034440 con 0x7ff2b8072720 2024-04-04T06:17:55.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.364+0000 7ff2bd400640 1 -- 172.21.15.159:0/2692976645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2b8072720 msgr2=0x7ff2b8167300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:55.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.364+0000 7ff2bd400640 1 --2- 172.21.15.159:0/2692976645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2b8072720 0x7ff2b8167300 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009d90 tx=0x7ff2b002bd90 comp rx=0 tx=0).stop 2024-04-04T06:17:55.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.364+0000 7ff2bd400640 1 -- 172.21.15.159:0/2692976645 shutdown_connections 2024-04-04T06:17:55.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.364+0000 7ff2bd400640 1 --2- 172.21.15.159:0/2692976645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2b8072720 0x7ff2b8167300 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:55.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.364+0000 7ff2bd400640 1 -- 172.21.15.159:0/2692976645 >> 172.21.15.159:0/2692976645 conn(0x7ff2b806bba0 msgr2=0x7ff2b80b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:55.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.364+0000 7ff2bd400640 1 -- 172.21.15.159:0/2692976645 shutdown_connections 2024-04-04T06:17:55.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.364+0000 7ff2bd400640 1 -- 172.21.15.159:0/2692976645 wait complete. 2024-04-04T06:17:55.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.366+0000 7ff2bd400640 1 Processor -- start 2024-04-04T06:17:55.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.366+0000 7ff2bd400640 1 -- start start 2024-04-04T06:17:55.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.367+0000 7ff2bd400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b8072720 0x7ff2b815b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:55.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.367+0000 7ff2bd400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2b809dcf0 0x7ff2b815be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:55.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.367+0000 7ff2bd400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2b80acf10 0x7ff2b806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:55.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.367+0000 7ff2bd400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b815d7a0 con 0x7ff2b809dcf0 2024-04-04T06:17:55.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.367+0000 7ff2bd400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b815c950 con 0x7ff2b8072720 2024-04-04T06:17:55.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.367+0000 7ff2bd400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b8167840 con 0x7ff2b80acf10 2024-04-04T06:17:55.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.367+0000 7ff2b5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2b809dcf0 0x7ff2b815be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:55.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.367+0000 7ff2b5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2b809dcf0 0x7ff2b815be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:54986/0 (socket says 172.21.15.159:54986) 2024-04-04T06:17:55.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.367+0000 7ff2b5e00640 1 -- 172.21.15.159:0/1809837561 learned_addr learned my addr 172.21.15.159:0/1809837561 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:55.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.367+0000 7ff2b6800640 1 --2- 172.21.15.159:0/1809837561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b8072720 0x7ff2b815b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:55.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.367+0000 7ff2b5e00640 1 -- 172.21.15.159:0/1809837561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2b80acf10 msgr2=0x7ff2b806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:55.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.367+0000 7ff2b5e00640 1 --2- 172.21.15.159:0/1809837561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2b80acf10 0x7ff2b806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:55.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.367+0000 7ff2b5e00640 1 -- 172.21.15.159:0/1809837561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b8072720 msgr2=0x7ff2b815b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:55.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.367+0000 7ff2b5e00640 1 --2- 172.21.15.159:0/1809837561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b8072720 0x7ff2b815b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:55.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.367+0000 7ff2b5e00640 1 -- 172.21.15.159:0/1809837561 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b806e680 con 0x7ff2b809dcf0 2024-04-04T06:17:55.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.368+0000 7ff2b6800640 1 --2- 172.21.15.159:0/1809837561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b8072720 0x7ff2b815b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:55.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.368+0000 7ff2b5e00640 1 --2- 172.21.15.159:0/1809837561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2b809dcf0 0x7ff2b815be30 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400c400 tx=0x7ff2a400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:55.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.368+0000 7ff2aea00640 1 -- 172.21.15.159:0/1809837561 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4017070 con 0x7ff2b809dcf0 2024-04-04T06:17:55.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.368+0000 7ff2aea00640 1 -- 172.21.15.159:0/1809837561 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2a4005ce0 con 0x7ff2b809dcf0 2024-04-04T06:17:55.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.368+0000 7ff2aea00640 1 -- 172.21.15.159:0/1809837561 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40072f0 con 0x7ff2b809dcf0 2024-04-04T06:17:55.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.368+0000 7ff2bd400640 1 -- 172.21.15.159:0/1809837561 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b806e8f0 con 0x7ff2b809dcf0 2024-04-04T06:17:55.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.368+0000 7ff2bd400640 1 -- 172.21.15.159:0/1809837561 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b806eeb0 con 0x7ff2b809dcf0 2024-04-04T06:17:55.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.369+0000 7ff2bd400640 1 -- 172.21.15.159:0/1809837561 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b80685c0 con 0x7ff2b809dcf0 2024-04-04T06:17:55.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.370+0000 7ff2aea00640 1 -- 172.21.15.159:0/1809837561 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff2a401a070 con 0x7ff2b809dcf0 2024-04-04T06:17:55.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.371+0000 7ff2aea00640 1 --2- 172.21.15.159:0/1809837561 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff29806c8f0 0x7ff29806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:55.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.371+0000 7ff2aea00640 1 -- 172.21.15.159:0/1809837561 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6025+0+0 (secure 0 0 0) 0x7ff2a408b0a0 con 0x7ff2b809dcf0 2024-04-04T06:17:55.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.372+0000 7ff2b6800640 1 --2- 172.21.15.159:0/1809837561 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff29806c8f0 0x7ff29806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:55.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.373+0000 7ff2b6800640 1 --2- 172.21.15.159:0/1809837561 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff29806c8f0 0x7ff29806edb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800a830 tx=0x7ff2a8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:55.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.375+0000 7ff2aea00640 1 -- 172.21.15.159:0/1809837561 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ff2a4059390 con 0x7ff2b809dcf0 2024-04-04T06:17:55.571 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:55.575+0000 7ff2bd400640 1 -- 172.21.15.159:0/1809837561 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff2b8139f40 con 0x7ff2b809dcf0 2024-04-04T06:17:56.123 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.127+0000 7ff2aea00640 1 -- 172.21.15.159:0/1809837561 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v118) ==== 181+0+0 (secure 0 0 0) 0x7ff2a4058c90 con 0x7ff2b809dcf0 2024-04-04T06:17:56.123 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_metadata' removed 2024-04-04T06:17:56.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.131+0000 7ff2bd400640 1 -- 172.21.15.159:0/1809837561 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff29806c8f0 msgr2=0x7ff29806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:56.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.131+0000 7ff2bd400640 1 --2- 172.21.15.159:0/1809837561 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff29806c8f0 0x7ff29806edb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800a830 tx=0x7ff2a8008040 comp rx=0 tx=0).stop 2024-04-04T06:17:56.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.131+0000 7ff2bd400640 1 -- 172.21.15.159:0/1809837561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2b809dcf0 msgr2=0x7ff2b815be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:56.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.131+0000 7ff2bd400640 1 --2- 172.21.15.159:0/1809837561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2b809dcf0 0x7ff2b815be30 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400c400 tx=0x7ff2a400c8c0 comp rx=0 tx=0).stop 2024-04-04T06:17:56.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.133+0000 7ff2bd400640 1 -- 172.21.15.159:0/1809837561 shutdown_connections 2024-04-04T06:17:56.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.133+0000 7ff2bd400640 1 --2- 172.21.15.159:0/1809837561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff2b80acf10 0x7ff2b806d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:56.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.133+0000 7ff2bd400640 1 --2- 172.21.15.159:0/1809837561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2b809dcf0 0x7ff2b815be30 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:56.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.133+0000 7ff2bd400640 1 --2- 172.21.15.159:0/1809837561 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ff29806c8f0 0x7ff29806edb0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:56.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.133+0000 7ff2bd400640 1 --2- 172.21.15.159:0/1809837561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b8072720 0x7ff2b815b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:56.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.133+0000 7ff2bd400640 1 -- 172.21.15.159:0/1809837561 >> 172.21.15.159:0/1809837561 conn(0x7ff2b806bba0 msgr2=0x7ff2b809c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:56.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.134+0000 7ff2bd400640 1 -- 172.21.15.159:0/1809837561 shutdown_connections 2024-04-04T06:17:56.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.134+0000 7ff2bd400640 1 -- 172.21.15.159:0/1809837561 wait complete. 2024-04-04T06:17:56.152 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-04T06:17:56.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.369+0000 7fea1be00640 1 Processor -- start 2024-04-04T06:17:56.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.369+0000 7fea1be00640 1 -- start start 2024-04-04T06:17:56.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.370+0000 7fea1be00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea140d0c90 0x7fea140d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:56.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.370+0000 7fea1be00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fea141610d0 0x7fea14161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:56.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.370+0000 7fea1be00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea140d21c0 con 0x7fea1406e310 2024-04-04T06:17:56.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.370+0000 7fea1be00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fea140d1630 con 0x7fea141610d0 2024-04-04T06:17:56.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.370+0000 7fea1be00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea140d53c0 con 0x7fea140d0c90 2024-04-04T06:17:56.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.371+0000 7fea19400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fea1406e310 0x7fea140cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52206/0 (socket says 172.21.15.159:52206) 2024-04-04T06:17:56.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.371+0000 7fea19400640 1 -- 172.21.15.159:0/365325163 learned_addr learned my addr 172.21.15.159:0/365325163 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:56.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.371+0000 7fea18a00640 1 --2- 172.21.15.159:0/365325163 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea140d0c90 0x7fea140d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:56.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.371+0000 7fea19e00640 1 --2- 172.21.15.159:0/365325163 >> v2:172.21.15.178:3300/0 conn(0x7fea141610d0 0x7fea14161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:56.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.371+0000 7fea13e00640 1 -- 172.21.15.159:0/365325163 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4072791099 0 0) 0x7fea140d21c0 con 0x7fea1406e310 2024-04-04T06:17:56.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.371+0000 7fea13e00640 1 -- 172.21.15.159:0/365325163 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9f8003520 con 0x7fea1406e310 2024-04-04T06:17:56.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.372+0000 7fea13e00640 1 -- 172.21.15.159:0/365325163 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 326644591 0 0) 0x7fe9f8003520 con 0x7fea1406e310 2024-04-04T06:17:56.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.372+0000 7fea13e00640 1 -- 172.21.15.159:0/365325163 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea140d0c90 msgr2=0x7fea140d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:56.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.372+0000 7fea13e00640 1 --2- 172.21.15.159:0/365325163 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea140d0c90 0x7fea140d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:56.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.372+0000 7fea13e00640 1 -- 172.21.15.159:0/365325163 >> v2:172.21.15.178:3300/0 conn(0x7fea141610d0 msgr2=0x7fea14161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.372+0000 7fea13e00640 1 --2- 172.21.15.159:0/365325163 >> v2:172.21.15.178:3300/0 conn(0x7fea141610d0 0x7fea14161560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.372+0000 7fea13e00640 1 -- 172.21.15.159:0/365325163 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea140d24a0 con 0x7fea1406e310 2024-04-04T06:17:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.372+0000 7fea13e00640 1 -- 172.21.15.159:0/365325163 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fea04001670 con 0x7fea1406e310 2024-04-04T06:17:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.372+0000 7fea13e00640 1 -- 172.21.15.159:0/365325163 >> v1:172.21.15.159:6789/0 conn(0x7fea1406e310 legacy=0x7fea140cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.372+0000 7fea13e00640 1 --2- 172.21.15.159:0/365325163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9f80039e0 0x7fe9f8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.372+0000 7fea13e00640 1 --2- 172.21.15.159:0/365325163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9f8004800 0x7fe9f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.372+0000 7fea13e00640 1 --2- 172.21.15.159:0/365325163 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea140d0c90 0x7fe9f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.372+0000 7fea13e00640 1 -- 172.21.15.159:0/365325163 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fea140d53c0 con 0x7fe9f80039e0 2024-04-04T06:17:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.372+0000 7fea13e00640 1 -- 172.21.15.159:0/365325163 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f800b9c0 con 0x7fe9f8004800 2024-04-04T06:17:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.372+0000 7fea13e00640 1 -- 172.21.15.159:0/365325163 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fe9f800bb40 con 0x7fea140d0c90 2024-04-04T06:17:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.372+0000 7fea19400640 1 --2- 172.21.15.159:0/365325163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9f80039e0 0x7fe9f8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.372+0000 7fea19e00640 1 --2- 172.21.15.159:0/365325163 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea140d0c90 0x7fe9f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.373+0000 7fea18a00640 1 --2- 172.21.15.159:0/365325163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9f8004800 0x7fe9f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:56.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.373+0000 7fea19400640 1 -- 172.21.15.159:0/365325163 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea140d0c90 msgr2=0x7fe9f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:56.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.373+0000 7fea19400640 1 --2- 172.21.15.159:0/365325163 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea140d0c90 0x7fe9f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:56.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.373+0000 7fea19400640 1 -- 172.21.15.159:0/365325163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9f8004800 msgr2=0x7fe9f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:56.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.373+0000 7fea19400640 1 --2- 172.21.15.159:0/365325163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9f8004800 0x7fe9f8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:56.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.373+0000 7fea19400640 1 -- 172.21.15.159:0/365325163 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f8003520 con 0x7fe9f80039e0 2024-04-04T06:17:56.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.373+0000 7fea19e00640 1 --2- 172.21.15.159:0/365325163 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea140d0c90 0x7fe9f80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:56.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.374+0000 7fea19400640 1 --2- 172.21.15.159:0/365325163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9f80039e0 0x7fe9f8003e00 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fea0400c210 tx=0x7fe9f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60de099f9f52692 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:56.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.374+0000 7fea13e00640 1 -- 172.21.15.159:0/365325163 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea04007d30 con 0x7fe9f80039e0 2024-04-04T06:17:56.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.374+0000 7fea13e00640 1 -- 172.21.15.159:0/365325163 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea04030070 con 0x7fe9f80039e0 2024-04-04T06:17:56.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.374+0000 7fea13e00640 1 -- 172.21.15.159:0/365325163 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0402cd60 con 0x7fe9f80039e0 2024-04-04T06:17:56.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.375+0000 7fea1be00640 1 -- 172.21.15.159:0/365325163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9f80039e0 msgr2=0x7fe9f8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:56.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.375+0000 7fea1be00640 1 --2- 172.21.15.159:0/365325163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9f80039e0 0x7fe9f8003e00 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fea0400c210 tx=0x7fe9f8005830 comp rx=0 tx=0).stop 2024-04-04T06:17:56.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.375+0000 7fea19e00640 1 -- 172.21.15.159:0/365325163 reap_dead start 2024-04-04T06:17:56.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.375+0000 7fea1be00640 1 -- 172.21.15.159:0/365325163 shutdown_connections 2024-04-04T06:17:56.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.375+0000 7fea1be00640 1 -- 172.21.15.159:0/365325163 >> 172.21.15.159:0/365325163 conn(0x7fea1408bfb0 msgr2=0x7fea140d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:56.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.375+0000 7fea1be00640 1 -- 172.21.15.159:0/365325163 shutdown_connections 2024-04-04T06:17:56.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.375+0000 7fea1be00640 1 -- 172.21.15.159:0/365325163 wait complete. 2024-04-04T06:17:56.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.377+0000 7fea1be00640 1 Processor -- start 2024-04-04T06:17:56.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.377+0000 7fea1be00640 1 -- start start 2024-04-04T06:17:56.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.378+0000 7fea1be00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9f80039e0 0x7fea1415d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:56.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.378+0000 7fea1be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea1415c1a0 0x7fea1415d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:56.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.378+0000 7fea1be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea1415df20 0x7fea140cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:56.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.378+0000 7fea1be00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fea140d2d40 con 0x7fea1415df20 2024-04-04T06:17:56.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.378+0000 7fea1be00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea140d2a60 con 0x7fea1415c1a0 2024-04-04T06:17:56.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.378+0000 7fea1be00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fea140d3260 con 0x7fe9f80039e0 2024-04-04T06:17:56.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.378+0000 7fea19e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea1415df20 0x7fea140cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:56.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.378+0000 7fea19e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea1415df20 0x7fea140cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:54992/0 (socket says 172.21.15.159:54992) 2024-04-04T06:17:56.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.378+0000 7fea19e00640 1 -- 172.21.15.159:0/1490611643 learned_addr learned my addr 172.21.15.159:0/1490611643 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:56.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.378+0000 7fea18a00640 1 --2- 172.21.15.159:0/1490611643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea1415c1a0 0x7fea1415d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:56.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.378+0000 7fea19400640 1 --2- 172.21.15.159:0/1490611643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9f80039e0 0x7fea1415d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:56.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.378+0000 7fea19e00640 1 -- 172.21.15.159:0/1490611643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9f80039e0 msgr2=0x7fea1415d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:56.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.378+0000 7fea19e00640 1 --2- 172.21.15.159:0/1490611643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9f80039e0 0x7fea1415d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:56.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.378+0000 7fea19e00640 1 -- 172.21.15.159:0/1490611643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea1415c1a0 msgr2=0x7fea1415d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:56.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.378+0000 7fea19e00640 1 --2- 172.21.15.159:0/1490611643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea1415c1a0 0x7fea1415d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:56.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.378+0000 7fea19e00640 1 -- 172.21.15.159:0/1490611643 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea140cb5c0 con 0x7fea1415df20 2024-04-04T06:17:56.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.379+0000 7fea19e00640 1 --2- 172.21.15.159:0/1490611643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea1415df20 0x7fea140cc660 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fea1408d1e0 tx=0x7fea0c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:56.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.379+0000 7fea11600640 1 -- 172.21.15.159:0/1490611643 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c016420 con 0x7fea1415df20 2024-04-04T06:17:56.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.379+0000 7fea11600640 1 -- 172.21.15.159:0/1490611643 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea0c01b070 con 0x7fea1415df20 2024-04-04T06:17:56.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.379+0000 7fea11600640 1 -- 172.21.15.159:0/1490611643 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c0167e0 con 0x7fea1415df20 2024-04-04T06:17:56.375 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.379+0000 7fea1be00640 1 -- 172.21.15.159:0/1490611643 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea140cb8a0 con 0x7fea1415df20 2024-04-04T06:17:56.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.379+0000 7fea1be00640 1 -- 172.21.15.159:0/1490611643 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea140cbe60 con 0x7fea1415df20 2024-04-04T06:17:56.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.380+0000 7fea1be00640 1 -- 172.21.15.159:0/1490611643 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea140685c0 con 0x7fea1415df20 2024-04-04T06:17:56.376 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.381+0000 7fea11600640 1 -- 172.21.15.159:0/1490611643 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea0c005d10 con 0x7fea1415df20 2024-04-04T06:17:56.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.384+0000 7fea11600640 1 --2- 172.21.15.159:0/1490611643 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe9f406c920 0x7fe9f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:56.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.384+0000 7fea11600640 1 -- 172.21.15.159:0/1490611643 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5587+0+0 (secure 0 0 0) 0x7fea0c08c530 con 0x7fea1415df20 2024-04-04T06:17:56.378 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.384+0000 7fea19400640 1 --2- 172.21.15.159:0/1490611643 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe9f406c920 0x7fe9f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:56.379 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.384+0000 7fea19400640 1 --2- 172.21.15.159:0/1490611643 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe9f406c920 0x7fe9f406ede0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fea04034000 tx=0x7fea04009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:56.382 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.388+0000 7fea11600640 1 -- 172.21.15.159:0/1490611643 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fea0c025020 con 0x7fea1415df20 2024-04-04T06:17:56.587 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:56.592+0000 7fea1be00640 1 -- 172.21.15.159:0/1490611643 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fea140d1bb0 con 0x7fea1415df20 2024-04-04T06:17:57.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.131+0000 7fea11600640 1 -- 172.21.15.159:0/1490611643 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v119) ==== 169+0+0 (secure 0 0 0) 0x7fea0c05a9c0 con 0x7fea1415df20 2024-04-04T06:17:57.126 INFO:teuthology.orchestra.run.smithi159.stderr:pool 'cephfs_data' removed 2024-04-04T06:17:57.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.135+0000 7fea1be00640 1 -- 172.21.15.159:0/1490611643 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe9f406c920 msgr2=0x7fe9f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:57.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.135+0000 7fea1be00640 1 --2- 172.21.15.159:0/1490611643 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe9f406c920 0x7fe9f406ede0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fea04034000 tx=0x7fea04009030 comp rx=0 tx=0).stop 2024-04-04T06:17:57.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.135+0000 7fea1be00640 1 -- 172.21.15.159:0/1490611643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea1415df20 msgr2=0x7fea140cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:57.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.135+0000 7fea1be00640 1 --2- 172.21.15.159:0/1490611643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea1415df20 0x7fea140cc660 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fea1408d1e0 tx=0x7fea0c00f520 comp rx=0 tx=0).stop 2024-04-04T06:17:57.131 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.136+0000 7fea1be00640 1 -- 172.21.15.159:0/1490611643 shutdown_connections 2024-04-04T06:17:57.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.136+0000 7fea1be00640 1 --2- 172.21.15.159:0/1490611643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea1415df20 0x7fea140cc660 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:57.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.136+0000 7fea1be00640 1 --2- 172.21.15.159:0/1490611643 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fe9f406c920 0x7fe9f406ede0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:57.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.136+0000 7fea1be00640 1 --2- 172.21.15.159:0/1490611643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea1415c1a0 0x7fea1415d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:57.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.136+0000 7fea1be00640 1 --2- 172.21.15.159:0/1490611643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9f80039e0 0x7fea1415d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:57.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.136+0000 7fea1be00640 1 -- 172.21.15.159:0/1490611643 >> 172.21.15.159:0/1490611643 conn(0x7fea1408bfb0 msgr2=0x7fea14160930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:57.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.137+0000 7fea1be00640 1 -- 172.21.15.159:0/1490611643 shutdown_connections 2024-04-04T06:17:57.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.137+0000 7fea1be00640 1 -- 172.21.15.159:0/1490611643 wait complete. 2024-04-04T06:17:57.157 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-04T06:17:57.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.365+0000 7f39d9a00640 1 Processor -- start 2024-04-04T06:17:57.360 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.365+0000 7f39d9a00640 1 -- start start 2024-04-04T06:17:57.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.367+0000 7f39d9a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f39d4155550 0x7f39d4155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:57.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.367+0000 7f39d9a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f39d415f800 0x7f39d40d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:57.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.367+0000 7f39d9a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39d40b40b0 con 0x7f39d40d07c0 2024-04-04T06:17:57.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.367+0000 7f39d9a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f39d40d0bf0 con 0x7f39d4155550 2024-04-04T06:17:57.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.367+0000 7f39d9a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f39d40b4390 con 0x7f39d415f800 2024-04-04T06:17:57.361 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.367+0000 7f39d2a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f39d4155550 0x7f39d4155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:57.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.367+0000 7f39d3400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f39d40d07c0 0x7f39d40aede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52216/0 (socket says 172.21.15.159:52216) 2024-04-04T06:17:57.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.367+0000 7f39d3400640 1 -- 172.21.15.159:0/935143298 learned_addr learned my addr 172.21.15.159:0/935143298 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:57.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.367+0000 7f39d2000640 1 --2- 172.21.15.159:0/935143298 >> v2:172.21.15.178:3300/0 conn(0x7f39d415f800 0x7f39d40d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:57.362 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.367+0000 7f39d2a00640 1 -- 172.21.15.159:0/935143298 >> v2:172.21.15.178:3300/0 conn(0x7f39d415f800 msgr2=0x7f39d40d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:57.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.367+0000 7f39d2a00640 1 --2- 172.21.15.159:0/935143298 >> v2:172.21.15.178:3300/0 conn(0x7f39d415f800 0x7f39d40d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:57.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.367+0000 7f39d2a00640 1 -- 172.21.15.159:0/935143298 >> v1:172.21.15.159:6789/0 conn(0x7f39d40d07c0 legacy=0x7f39d40aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:57.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.367+0000 7f39d2a00640 1 -- 172.21.15.159:0/935143298 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d40b4670 con 0x7f39d4155550 2024-04-04T06:17:57.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.367+0000 7f39d1600640 1 -- 172.21.15.159:0/935143298 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3849907242 0 0) 0x7f39d40b40b0 con 0x7f39d40d07c0 2024-04-04T06:17:57.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.368+0000 7f39d2a00640 1 --2- 172.21.15.159:0/935143298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39d4155550 0x7f39d4155930 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f39d40d0dd0 tx=0x7f39c402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f87912dfc7be9f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:57.363 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.369+0000 7f39d1600640 1 -- 172.21.15.159:0/935143298 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c40315a0 con 0x7f39d4155550 2024-04-04T06:17:57.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.369+0000 7f39d1600640 1 -- 172.21.15.159:0/935143298 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c4037070 con 0x7f39d4155550 2024-04-04T06:17:57.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.369+0000 7f39d1600640 1 -- 172.21.15.159:0/935143298 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4031960 con 0x7f39d4155550 2024-04-04T06:17:57.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.369+0000 7f39d9a00640 1 -- 172.21.15.159:0/935143298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39d4155550 msgr2=0x7f39d4155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:57.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.369+0000 7f39d9a00640 1 --2- 172.21.15.159:0/935143298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39d4155550 0x7f39d4155930 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f39d40d0dd0 tx=0x7f39c402b840 comp rx=0 tx=0).stop 2024-04-04T06:17:57.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.369+0000 7f39d9a00640 1 -- 172.21.15.159:0/935143298 shutdown_connections 2024-04-04T06:17:57.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.369+0000 7f39d9a00640 1 --2- 172.21.15.159:0/935143298 >> v2:172.21.15.178:3300/0 conn(0x7f39d415f800 0x7f39d40d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:57.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.369+0000 7f39d9a00640 1 --2- 172.21.15.159:0/935143298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39d4155550 0x7f39d4155930 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:57.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.369+0000 7f39d9a00640 1 -- 172.21.15.159:0/935143298 >> 172.21.15.159:0/935143298 conn(0x7f39d406b750 msgr2=0x7f39d406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:57.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.370+0000 7f39d9a00640 1 -- 172.21.15.159:0/935143298 shutdown_connections 2024-04-04T06:17:57.365 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.370+0000 7f39d9a00640 1 -- 172.21.15.159:0/935143298 wait complete. 2024-04-04T06:17:57.366 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.371+0000 7f39d9a00640 1 Processor -- start 2024-04-04T06:17:57.367 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.371+0000 7f39d9a00640 1 -- start start 2024-04-04T06:17:57.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.372+0000 7f39d9a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39d40d07c0 0x7f39d40cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:57.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.372+0000 7f39d9a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39d4155550 0x7f39d41f50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:57.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.372+0000 7f39d9a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39d415f800 0x7f39d40ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:57.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.372+0000 7f39d9a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f39d415fc80 con 0x7f39d40d07c0 2024-04-04T06:17:57.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.372+0000 7f39d9a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f39d40b4c30 con 0x7f39d4155550 2024-04-04T06:17:57.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.372+0000 7f39d9a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f39d415ba20 con 0x7f39d415f800 2024-04-04T06:17:57.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.373+0000 7f39d2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39d40d07c0 0x7f39d40cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:57.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.373+0000 7f39d2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39d40d07c0 0x7f39d40cd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55010/0 (socket says 172.21.15.159:55010) 2024-04-04T06:17:57.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.373+0000 7f39d2a00640 1 -- 172.21.15.159:0/2589412936 learned_addr learned my addr 172.21.15.159:0/2589412936 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:57.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.373+0000 7f39d2000640 1 --2- 172.21.15.159:0/2589412936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39d4155550 0x7f39d41f50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:57.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.373+0000 7f39d3400640 1 --2- 172.21.15.159:0/2589412936 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39d415f800 0x7f39d40ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:57.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.373+0000 7f39d2a00640 1 -- 172.21.15.159:0/2589412936 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39d415f800 msgr2=0x7f39d40ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:57.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.373+0000 7f39d2a00640 1 --2- 172.21.15.159:0/2589412936 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39d415f800 0x7f39d40ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:57.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.373+0000 7f39d2a00640 1 -- 172.21.15.159:0/2589412936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39d4155550 msgr2=0x7f39d41f50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:57.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.373+0000 7f39d2a00640 1 --2- 172.21.15.159:0/2589412936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39d4155550 0x7f39d41f50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:57.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.373+0000 7f39d2a00640 1 -- 172.21.15.159:0/2589412936 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d40cdbe0 con 0x7f39d40d07c0 2024-04-04T06:17:57.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.373+0000 7f39d2a00640 1 --2- 172.21.15.159:0/2589412936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39d40d07c0 0x7f39d40cd6a0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f39c4034040 tx=0x7f39c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:57.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.374+0000 7f39caa00640 1 -- 172.21.15.159:0/2589412936 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4004230 con 0x7f39d40d07c0 2024-04-04T06:17:57.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.374+0000 7f39caa00640 1 -- 172.21.15.159:0/2589412936 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c40043d0 con 0x7f39d40d07c0 2024-04-04T06:17:57.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.374+0000 7f39caa00640 1 -- 172.21.15.159:0/2589412936 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4003850 con 0x7f39d40d07c0 2024-04-04T06:17:57.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.374+0000 7f39d9a00640 1 -- 172.21.15.159:0/2589412936 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d40af050 con 0x7f39d40d07c0 2024-04-04T06:17:57.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.374+0000 7f39d9a00640 1 -- 172.21.15.159:0/2589412936 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d40af610 con 0x7f39d40d07c0 2024-04-04T06:17:57.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.375+0000 7f39d9a00640 1 -- 172.21.15.159:0/2589412936 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d40685c0 con 0x7f39d40d07c0 2024-04-04T06:17:57.371 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.376+0000 7f39caa00640 1 -- 172.21.15.159:0/2589412936 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f39c40039f0 con 0x7f39d40d07c0 2024-04-04T06:17:57.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.377+0000 7f39caa00640 1 --2- 172.21.15.159:0/2589412936 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f39b406c8f0 0x7f39b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:57.372 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.377+0000 7f39caa00640 1 -- 172.21.15.159:0/2589412936 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f39c402f030 con 0x7f39d40d07c0 2024-04-04T06:17:57.373 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.378+0000 7f39d2000640 1 --2- 172.21.15.159:0/2589412936 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f39b406c8f0 0x7f39b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:57.374 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.379+0000 7f39d2000640 1 --2- 172.21.15.159:0/2589412936 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f39b406c8f0 0x7f39b406edb0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f39d40b1590 tx=0x7f39c0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:57.377 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.382+0000 7f39caa00640 1 -- 172.21.15.159:0/2589412936 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f39c4041070 con 0x7f39d40d07c0 2024-04-04T06:17:57.575 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.581+0000 7f39d9a00640 1 -- 172.21.15.159:0/2589412936 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f39d40b3ad0 con 0x7f39d40d07c0 2024-04-04T06:17:57.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.587+0000 7f39caa00640 1 -- 172.21.15.159:0/2589412936 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f39c4044340 con 0x7f39d40d07c0 2024-04-04T06:17:57.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.589+0000 7f39d9a00640 1 -- 172.21.15.159:0/2589412936 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f39b406c8f0 msgr2=0x7f39b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:57.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.589+0000 7f39d9a00640 1 --2- 172.21.15.159:0/2589412936 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f39b406c8f0 0x7f39b406edb0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f39d40b1590 tx=0x7f39c0008040 comp rx=0 tx=0).stop 2024-04-04T06:17:57.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.589+0000 7f39d9a00640 1 -- 172.21.15.159:0/2589412936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39d40d07c0 msgr2=0x7f39d40cd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:57.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.589+0000 7f39d9a00640 1 --2- 172.21.15.159:0/2589412936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39d40d07c0 0x7f39d40cd6a0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f39c4034040 tx=0x7f39c40046c0 comp rx=0 tx=0).stop 2024-04-04T06:17:57.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.590+0000 7f39d9a00640 1 -- 172.21.15.159:0/2589412936 shutdown_connections 2024-04-04T06:17:57.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.590+0000 7f39d9a00640 1 --2- 172.21.15.159:0/2589412936 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39d415f800 0x7f39d40ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:57.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.590+0000 7f39d9a00640 1 --2- 172.21.15.159:0/2589412936 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f39b406c8f0 0x7f39b406edb0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:57.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.590+0000 7f39d9a00640 1 --2- 172.21.15.159:0/2589412936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39d4155550 0x7f39d41f50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:57.584 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.590+0000 7f39d9a00640 1 --2- 172.21.15.159:0/2589412936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39d40d07c0 0x7f39d40cd6a0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:57.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.590+0000 7f39d9a00640 1 -- 172.21.15.159:0/2589412936 >> 172.21.15.159:0/2589412936 conn(0x7f39d406b750 msgr2=0x7f39d4071b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:57.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.590+0000 7f39d9a00640 1 -- 172.21.15.159:0/2589412936 shutdown_connections 2024-04-04T06:17:57.585 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.590+0000 7f39d9a00640 1 -- 172.21.15.159:0/2589412936 wait complete. 2024-04-04T06:17:57.602 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-04T06:17:57.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.815+0000 7fa1f2600640 1 Processor -- start 2024-04-04T06:17:57.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.815+0000 7fa1f2600640 1 -- start start 2024-04-04T06:17:57.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.817+0000 7fa1f2600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fa1ec144ad0 0x7fa1ec0b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:57.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.817+0000 7fa1f2600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1ec15fba0 con 0x7fa1ec15f710 2024-04-04T06:17:57.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.817+0000 7fa1f2600640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1ec144eb0 con 0x7fa1ec072f50 2024-04-04T06:17:57.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.817+0000 7fa1f2600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fa1ec0cdad0 con 0x7fa1ec144ad0 2024-04-04T06:17:57.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.817+0000 7fa1e3e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fa1ec15f710 0x7fa1ec072840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52226/0 (socket says 172.21.15.159:52226) 2024-04-04T06:17:57.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.817+0000 7fa1e3e00640 1 -- 172.21.15.159:0/1044034932 learned_addr learned my addr 172.21.15.159:0/1044034932 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:17:57.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.817+0000 7fa1f1200640 1 --2- 172.21.15.159:0/1044034932 >> v2:172.21.15.159:3300/0 conn(0x7fa1ec144ad0 0x7fa1ec0b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:57.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.818+0000 7fa1f1200640 1 -- 172.21.15.159:0/1044034932 >> v1:172.21.15.178:6789/0 conn(0x7fa1ec072f50 legacy=0x7fa1ec0cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:57.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.818+0000 7fa1f1200640 1 -- 172.21.15.159:0/1044034932 >> v1:172.21.15.159:6789/0 conn(0x7fa1ec15f710 legacy=0x7fa1ec072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:57.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.818+0000 7fa1f1200640 1 -- 172.21.15.159:0/1044034932 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ec0b36f0 con 0x7fa1ec144ad0 2024-04-04T06:17:57.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.819+0000 7fa1f1200640 1 --2- 172.21.15.159:0/1044034932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1ec144ad0 0x7fa1ec0b2ff0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009d90 tx=0x7fa1dc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649d40098d3d1bf server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:57.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.819+0000 7fa1ebe00640 1 -- 172.21.15.159:0/1044034932 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc039070 con 0x7fa1ec144ad0 2024-04-04T06:17:57.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.820+0000 7fa1ebe00640 1 -- 172.21.15.159:0/1044034932 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1dc02cc60 con 0x7fa1ec144ad0 2024-04-04T06:17:57.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.820+0000 7fa1ebe00640 1 -- 172.21.15.159:0/1044034932 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc034660 con 0x7fa1ec144ad0 2024-04-04T06:17:57.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.820+0000 7fa1f2600640 1 -- 172.21.15.159:0/1044034932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1ec144ad0 msgr2=0x7fa1ec0b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:57.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.820+0000 7fa1f2600640 1 --2- 172.21.15.159:0/1044034932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1ec144ad0 0x7fa1ec0b2ff0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009d90 tx=0x7fa1dc02beb0 comp rx=0 tx=0).stop 2024-04-04T06:17:57.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.820+0000 7fa1f2600640 1 -- 172.21.15.159:0/1044034932 shutdown_connections 2024-04-04T06:17:57.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.820+0000 7fa1f2600640 1 --2- 172.21.15.159:0/1044034932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1ec144ad0 0x7fa1ec0b2ff0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:57.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.820+0000 7fa1f2600640 1 -- 172.21.15.159:0/1044034932 >> 172.21.15.159:0/1044034932 conn(0x7fa1ec06b570 msgr2=0x7fa1ec06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:57.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.820+0000 7fa1f2600640 1 -- 172.21.15.159:0/1044034932 shutdown_connections 2024-04-04T06:17:57.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.820+0000 7fa1f2600640 1 -- 172.21.15.159:0/1044034932 wait complete. 2024-04-04T06:17:57.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.822+0000 7fa1f2600640 1 Processor -- start 2024-04-04T06:17:57.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.822+0000 7fa1f2600640 1 -- start start 2024-04-04T06:17:57.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.823+0000 7fa1f2600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1ec072f50 0x7fa1ec0b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:57.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.823+0000 7fa1f2600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1ec144ad0 0x7fa1ec15a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:57.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.823+0000 7fa1f2600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1ec15f710 0x7fa1ec15ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:57.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.823+0000 7fa1f2600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec0b3f90 con 0x7fa1ec144ad0 2024-04-04T06:17:57.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.823+0000 7fa1f2600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec0b3cb0 con 0x7fa1ec072f50 2024-04-04T06:17:57.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.823+0000 7fa1f2600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa1ec0ce670 con 0x7fa1ec15f710 2024-04-04T06:17:57.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.823+0000 7fa1e3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1ec144ad0 0x7fa1ec15a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:57.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.823+0000 7fa1e3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1ec144ad0 0x7fa1ec15a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55030/0 (socket says 172.21.15.159:55030) 2024-04-04T06:17:57.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.823+0000 7fa1e3e00640 1 -- 172.21.15.159:0/53044188 learned_addr learned my addr 172.21.15.159:0/53044188 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:57.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.824+0000 7fa1e3e00640 1 -- 172.21.15.159:0/53044188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1ec15f710 msgr2=0x7fa1ec15ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:57.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.824+0000 7fa1e3e00640 1 --2- 172.21.15.159:0/53044188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1ec15f710 0x7fa1ec15ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:57.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.824+0000 7fa1e3e00640 1 -- 172.21.15.159:0/53044188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1ec072f50 msgr2=0x7fa1ec0b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:57.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.824+0000 7fa1e3e00640 1 --2- 172.21.15.159:0/53044188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1ec072f50 0x7fa1ec0b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:57.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.824+0000 7fa1e3e00640 1 -- 172.21.15.159:0/53044188 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ec15c010 con 0x7fa1ec144ad0 2024-04-04T06:17:57.819 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.824+0000 7fa1e3e00640 1 --2- 172.21.15.159:0/53044188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1ec144ad0 0x7fa1ec15a620 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400c3c0 tx=0x7fa1d400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:57.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.824+0000 7fa1e9600640 1 -- 172.21.15.159:0/53044188 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4017070 con 0x7fa1ec144ad0 2024-04-04T06:17:57.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.824+0000 7fa1e9600640 1 -- 172.21.15.159:0/53044188 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d4005ce0 con 0x7fa1ec144ad0 2024-04-04T06:17:57.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.824+0000 7fa1e9600640 1 -- 172.21.15.159:0/53044188 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d40072f0 con 0x7fa1ec144ad0 2024-04-04T06:17:57.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.824+0000 7fa1f2600640 1 -- 172.21.15.159:0/53044188 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ec15c2f0 con 0x7fa1ec144ad0 2024-04-04T06:17:57.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.825+0000 7fa1f2600640 1 -- 172.21.15.159:0/53044188 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ec15c8b0 con 0x7fa1ec144ad0 2024-04-04T06:17:57.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.826+0000 7fa1f2600640 1 -- 172.21.15.159:0/53044188 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1ec075940 con 0x7fa1ec144ad0 2024-04-04T06:17:57.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.827+0000 7fa1e9600640 1 -- 172.21.15.159:0/53044188 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1d401a070 con 0x7fa1ec144ad0 2024-04-04T06:17:57.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.827+0000 7fa1e9600640 1 --2- 172.21.15.159:0/53044188 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa1c806c8f0 0x7fa1c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:57.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.828+0000 7fa1e9600640 1 -- 172.21.15.159:0/53044188 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7fa1d408ad40 con 0x7fa1ec144ad0 2024-04-04T06:17:57.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.828+0000 7fa1f1200640 1 --2- 172.21.15.159:0/53044188 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa1c806c8f0 0x7fa1c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:57.823 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.829+0000 7fa1f1200640 1 --2- 172.21.15.159:0/53044188 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa1c806c8f0 0x7fa1c806edb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc036000 tx=0x7fa1dc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:57.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:57.833+0000 7fa1e9600640 1 -- 172.21.15.159:0/53044188 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa1d4059320 con 0x7fa1ec144ad0 2024-04-04T06:17:58.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:58.108+0000 7fa1f2600640 1 -- 172.21.15.159:0/53044188 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7fa1ec0d0330 con 0x7fa1ec144ad0 2024-04-04T06:17:58.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:58.135+0000 7fa1e9600640 1 -- 172.21.15.159:0/53044188 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v581) ==== 144+0+0 (secure 0 0 0) 0x7fa1d4058c20 con 0x7fa1ec144ad0 2024-04-04T06:17:58.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:58.138+0000 7fa1f2600640 1 -- 172.21.15.159:0/53044188 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa1c806c8f0 msgr2=0x7fa1c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:58.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:58.138+0000 7fa1f2600640 1 --2- 172.21.15.159:0/53044188 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa1c806c8f0 0x7fa1c806edb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc036000 tx=0x7fa1dc02e000 comp rx=0 tx=0).stop 2024-04-04T06:17:58.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:58.138+0000 7fa1f2600640 1 -- 172.21.15.159:0/53044188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1ec144ad0 msgr2=0x7fa1ec15a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:58.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:58.138+0000 7fa1f2600640 1 --2- 172.21.15.159:0/53044188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1ec144ad0 0x7fa1ec15a620 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400c3c0 tx=0x7fa1d400c880 comp rx=0 tx=0).stop 2024-04-04T06:17:58.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:58.138+0000 7fa1f2600640 1 -- 172.21.15.159:0/53044188 shutdown_connections 2024-04-04T06:17:58.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:58.138+0000 7fa1f2600640 1 --2- 172.21.15.159:0/53044188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa1ec15f710 0x7fa1ec15ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:58.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:58.138+0000 7fa1f2600640 1 --2- 172.21.15.159:0/53044188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1ec144ad0 0x7fa1ec15a620 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:58.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:58.138+0000 7fa1f2600640 1 --2- 172.21.15.159:0/53044188 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa1c806c8f0 0x7fa1c806edb0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:58.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:58.138+0000 7fa1f2600640 1 --2- 172.21.15.159:0/53044188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1ec072f50 0x7fa1ec0b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:58.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:58.138+0000 7fa1f2600640 1 -- 172.21.15.159:0/53044188 >> 172.21.15.159:0/53044188 conn(0x7fa1ec06b570 msgr2=0x7fa1ec15eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:58.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:58.138+0000 7fa1f2600640 1 -- 172.21.15.159:0/53044188 shutdown_connections 2024-04-04T06:17:58.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:58.138+0000 7fa1f2600640 1 -- 172.21.15.159:0/53044188 wait complete. 2024-04-04T06:17:58.151 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-04T06:17:58.152 INFO:tasks.cephfs_test_runner: 2024-04-04T06:17:58.152 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-04T06:17:58.152 INFO:tasks.cephfs_test_runner:Ran 7 tests in 680.043s 2024-04-04T06:17:58.152 INFO:tasks.cephfs_test_runner: 2024-04-04T06:17:58.152 INFO:tasks.cephfs_test_runner:OK 2024-04-04T06:17:58.152 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-04T06:17:58.162 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-04T06:17:58.185 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-04T06:17:58.185 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:17:58.185 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:17:58.241 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:17:58.241 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-04-04T06:17:58.241 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:17:58.292 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-04T06:17:58.292 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:17:58.292 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:17:58.349 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:17:58.349 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-04T06:17:58.349 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-04T06:17:58.400 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053408,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249164k,nr_inodes=812291,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498332k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-04T06:17:58.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:58.400 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-04-04T06:17:58.457 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-04T06:17:58.457 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-04T06:17:58.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:58.458 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-04T06:17:58.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:58.552 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-04T06:17:58.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:58.640 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link delete ceph-brx 2024-04-04T06:17:58.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:58.774 DEBUG:teuthology.orchestra.run.smithi159:> ip netns list 2024-04-04T06:17:58.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:58.821 DEBUG:teuthology.orchestra.run.smithi159:> sudo ip link delete ceph-brx 2024-04-04T06:17:58.959 INFO:teuthology.orchestra.run.smithi159.stderr:Cannot find device "ceph-brx" 2024-04-04T06:17:58.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-04T06:17:58.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:58.963 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2024-04-04T06:17:59.021 INFO:teuthology.orchestra.run.smithi178.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-04T06:17:59.021 INFO:teuthology.orchestra.run.smithi178.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-04T06:17:59.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:59.022 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-04T06:17:59.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:59.117 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-04T06:17:59.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:59.203 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link delete ceph-brx 2024-04-04T06:17:59.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:59.328 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2024-04-04T06:17:59.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-04T06:17:59.372 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link delete ceph-brx 2024-04-04T06:17:59.508 INFO:teuthology.orchestra.run.smithi178.stderr:Cannot find device "ceph-brx" 2024-04-04T06:17:59.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-04T06:17:59.512 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-04T06:17:59.522 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-04T06:17:59.522 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:17:59.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.748+0000 7f4c36000640 1 Processor -- start 2024-04-04T06:17:59.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.749+0000 7f4c36000640 1 -- start start 2024-04-04T06:17:59.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.750+0000 7f4c36000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4c3015c390 0x7f4c30073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:59.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.750+0000 7f4c36000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c300af830 0x7f4c30160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:59.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.750+0000 7f4c36000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c3015fe80 con 0x7f4c30160b80 2024-04-04T06:17:59.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.750+0000 7f4c36000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f4c3015c770 con 0x7f4c3015c390 2024-04-04T06:17:59.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.750+0000 7f4c36000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4c300ad120 con 0x7f4c300af830 2024-04-04T06:17:59.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.750+0000 7f4c2f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4c3015c390 0x7f4c30073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:59.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.750+0000 7f4c2f400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f4c3015c390 0x7f4c30073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55040/0 (socket says 172.21.15.159:55040) 2024-04-04T06:17:59.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.750+0000 7f4c2f400640 1 -- 172.21.15.159:0/2107346340 learned_addr learned my addr 172.21.15.159:0/2107346340 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:59.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.751+0000 7f4c26a00640 1 --2- 172.21.15.159:0/2107346340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c300af830 0x7f4c30160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:59.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.751+0000 7f4c2f400640 1 -- 172.21.15.159:0/2107346340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c300af830 msgr2=0x7f4c30160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:59.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.751+0000 7f4c2f400640 1 --2- 172.21.15.159:0/2107346340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c300af830 0x7f4c30160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:59.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.751+0000 7f4c2f400640 1 -- 172.21.15.159:0/2107346340 >> v1:172.21.15.178:6789/0 conn(0x7f4c30160b80 legacy=0x7f4c300d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:17:59.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.751+0000 7f4c2f400640 1 -- 172.21.15.159:0/2107346340 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c300ad400 con 0x7f4c3015c390 2024-04-04T06:17:59.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.751+0000 7f4c26a00640 1 --2- 172.21.15.159:0/2107346340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c300af830 0x7f4c30160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:17:59.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.752+0000 7f4c2f400640 1 --2- 172.21.15.159:0/2107346340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c3015c390 0x7f4c30073790 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009870 tx=0x7f4c2002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35dc14162fe60c8d server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:59.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.752+0000 7f4c2ea00640 1 -- 172.21.15.159:0/2107346340 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20035020 con 0x7f4c3015c390 2024-04-04T06:17:59.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.752+0000 7f4c2ea00640 1 -- 172.21.15.159:0/2107346340 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c20031640 con 0x7f4c3015c390 2024-04-04T06:17:59.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.752+0000 7f4c2ea00640 1 -- 172.21.15.159:0/2107346340 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2002d040 con 0x7f4c3015c390 2024-04-04T06:17:59.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.753+0000 7f4c36000640 1 -- 172.21.15.159:0/2107346340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c3015c390 msgr2=0x7f4c30073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:59.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.753+0000 7f4c36000640 1 --2- 172.21.15.159:0/2107346340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c3015c390 0x7f4c30073790 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009870 tx=0x7f4c2002b8f0 comp rx=0 tx=0).stop 2024-04-04T06:17:59.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.753+0000 7f4c36000640 1 -- 172.21.15.159:0/2107346340 shutdown_connections 2024-04-04T06:17:59.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.753+0000 7f4c36000640 1 --2- 172.21.15.159:0/2107346340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c300af830 0x7f4c30160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:59.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.753+0000 7f4c36000640 1 --2- 172.21.15.159:0/2107346340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c3015c390 0x7f4c30073790 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:59.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.753+0000 7f4c36000640 1 -- 172.21.15.159:0/2107346340 >> 172.21.15.159:0/2107346340 conn(0x7f4c3006b990 msgr2=0x7f4c3006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:59.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.753+0000 7f4c36000640 1 -- 172.21.15.159:0/2107346340 shutdown_connections 2024-04-04T06:17:59.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.753+0000 7f4c36000640 1 -- 172.21.15.159:0/2107346340 wait complete. 2024-04-04T06:17:59.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.755+0000 7f4c36000640 1 Processor -- start 2024-04-04T06:17:59.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.755+0000 7f4c36000640 1 -- start start 2024-04-04T06:17:59.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.756+0000 7f4c36000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c300af830 0x7f4c3015ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:59.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.756+0000 7f4c36000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c3015c390 0x7f4c301e5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:59.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.756+0000 7f4c36000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c30160b80 0x7f4c301e6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:59.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.756+0000 7f4c36000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4c300cf980 con 0x7f4c300af830 2024-04-04T06:17:59.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.756+0000 7f4c36000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4c30160120 con 0x7f4c30160b80 2024-04-04T06:17:59.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.756+0000 7f4c36000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f4c300d0d60 con 0x7f4c3015c390 2024-04-04T06:17:59.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.756+0000 7f4c2f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c300af830 0x7f4c3015ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:59.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.756+0000 7f4c2f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c300af830 0x7f4c3015ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55054/0 (socket says 172.21.15.159:55054) 2024-04-04T06:17:59.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.756+0000 7f4c2f400640 1 -- 172.21.15.159:0/1933949676 learned_addr learned my addr 172.21.15.159:0/1933949676 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:17:59.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.757+0000 7f4c2f400640 1 -- 172.21.15.159:0/1933949676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c3015c390 msgr2=0x7f4c301e5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:59.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.757+0000 7f4c2f400640 1 --2- 172.21.15.159:0/1933949676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c3015c390 0x7f4c301e5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:59.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.757+0000 7f4c2fe00640 1 --2- 172.21.15.159:0/1933949676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c30160b80 0x7f4c301e6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:59.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.757+0000 7f4c2f400640 1 -- 172.21.15.159:0/1933949676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c30160b80 msgr2=0x7f4c301e6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:59.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.757+0000 7f4c2f400640 1 --2- 172.21.15.159:0/1933949676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c30160b80 0x7f4c301e6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:59.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.757+0000 7f4c2f400640 1 -- 172.21.15.159:0/1933949676 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c301e84f0 con 0x7f4c300af830 2024-04-04T06:17:59.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.757+0000 7f4c2f400640 1 --2- 172.21.15.159:0/1933949676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c300af830 0x7f4c3015ee10 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f4c2002b930 tx=0x7f4c200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:59.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.757+0000 7f4c27e00640 1 -- 172.21.15.159:0/1933949676 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20040070 con 0x7f4c300af830 2024-04-04T06:17:59.752 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.757+0000 7f4c27e00640 1 -- 172.21.15.159:0/1933949676 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c20039c10 con 0x7f4c300af830 2024-04-04T06:17:59.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.757+0000 7f4c27e00640 1 -- 172.21.15.159:0/1933949676 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20035040 con 0x7f4c300af830 2024-04-04T06:17:59.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.757+0000 7f4c36000640 1 -- 172.21.15.159:0/1933949676 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c3015f350 con 0x7f4c300af830 2024-04-04T06:17:59.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.758+0000 7f4c36000640 1 -- 172.21.15.159:0/1933949676 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c3015db20 con 0x7f4c300af830 2024-04-04T06:17:59.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.759+0000 7f4c36000640 1 -- 172.21.15.159:0/1933949676 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c30157fd0 con 0x7f4c300af830 2024-04-04T06:17:59.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.759+0000 7f4c27e00640 1 -- 172.21.15.159:0/1933949676 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c20038020 con 0x7f4c300af830 2024-04-04T06:17:59.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.760+0000 7f4c27e00640 1 --2- 172.21.15.159:0/1933949676 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4c0c06c8f0 0x7f4c0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:17:59.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.760+0000 7f4c27e00640 1 -- 172.21.15.159:0/1933949676 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f4c200ad480 con 0x7f4c300af830 2024-04-04T06:17:59.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.761+0000 7f4c26a00640 1 --2- 172.21.15.159:0/1933949676 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4c0c06c8f0 0x7f4c0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:17:59.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.762+0000 7f4c26a00640 1 --2- 172.21.15.159:0/1933949676 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4c0c06c8f0 0x7f4c0c06edb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4c18006fd0 tx=0x7f4c18008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:17:59.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.765+0000 7f4c27e00640 1 -- 172.21.15.159:0/1933949676 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f4c30157fd0 con 0x7f4c300af830 2024-04-04T06:17:59.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.962+0000 7f4c36000640 1 -- 172.21.15.159:0/1933949676 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4c300685c0 con 0x7f4c0c06c8f0 2024-04-04T06:17:59.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.965+0000 7f4c27e00640 1 -- 172.21.15.159:0/1933949676 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13523 (secure 0 0 0) 0x7f4c300685c0 con 0x7f4c0c06c8f0 2024-04-04T06:17:59.959 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:17:59.959 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:17:59.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.967+0000 7f4c36000640 1 -- 172.21.15.159:0/1933949676 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4c0c06c8f0 msgr2=0x7f4c0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:59.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.967+0000 7f4c36000640 1 --2- 172.21.15.159:0/1933949676 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4c0c06c8f0 0x7f4c0c06edb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4c18006fd0 tx=0x7f4c18008040 comp rx=0 tx=0).stop 2024-04-04T06:17:59.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.967+0000 7f4c36000640 1 -- 172.21.15.159:0/1933949676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c300af830 msgr2=0x7f4c3015ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:17:59.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.967+0000 7f4c36000640 1 --2- 172.21.15.159:0/1933949676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c300af830 0x7f4c3015ee10 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f4c2002b930 tx=0x7f4c200046c0 comp rx=0 tx=0).stop 2024-04-04T06:17:59.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.968+0000 7f4c36000640 1 -- 172.21.15.159:0/1933949676 shutdown_connections 2024-04-04T06:17:59.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.968+0000 7f4c36000640 1 --2- 172.21.15.159:0/1933949676 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f4c0c06c8f0 0x7f4c0c06edb0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:59.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.968+0000 7f4c36000640 1 --2- 172.21.15.159:0/1933949676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c30160b80 0x7f4c301e6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:59.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.968+0000 7f4c36000640 1 --2- 172.21.15.159:0/1933949676 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4c3015c390 0x7f4c301e5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:59.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.968+0000 7f4c36000640 1 --2- 172.21.15.159:0/1933949676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c300af830 0x7f4c3015ee10 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:17:59.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.968+0000 7f4c36000640 1 -- 172.21.15.159:0/1933949676 >> 172.21.15.159:0/1933949676 conn(0x7f4c3006b990 msgr2=0x7f4c300afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:17:59.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.968+0000 7f4c36000640 1 -- 172.21.15.159:0/1933949676 shutdown_connections 2024-04-04T06:17:59.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:17:59.968+0000 7f4c36000640 1 -- 172.21.15.159:0/1933949676 wait complete. 2024-04-04T06:17:59.981 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2024-04-04T06:17:58.985500+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":295016,"kb_used_data":28936,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754679704,"statfs":{"total":773094113280,"available":772792016896,"internally_reserved":0,"allocated":29630464,"data_stored":23025841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392250,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-178,"num_write_kb":-8306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002414"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":349,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-04T06:17:56.887104+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:17:56.887104+0000","last_peered":"2024-04-04T06:17:56.887104+0000","last_clean":"2024-04-04T06:17:56.887104+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:17:56.887104+0000","last_undegraded":"2024-04-04T06:17:56.887104+0000","last_fullsized":"2024-04-04T06:17:56.887104+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37340,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334500,"statfs":{"total":96636764160,"available":96598528000,"internally_reserved":0,"allocated":2048000,"data_stored":1222796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509591,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40856,"kb_used_data":5516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330984,"statfs":{"total":96636764160,"available":96594927616,"internally_reserved":0,"allocated":5648384,"data_stored":4826193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509592,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36708,"kb_used_data":5528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335132,"statfs":{"total":96636764160,"available":96599175168,"internally_reserved":0,"allocated":5660672,"data_stored":4829018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509591,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1449984,"data_stored":631817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509590,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":5524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":5656576,"data_stored":4832349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509592,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37292,"kb_used_data":6112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":6258688,"data_stored":5419428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542295,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36768,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335072,"statfs":{"total":96636764160,"available":96599113728,"internally_reserved":0,"allocated":1462272,"data_stored":632428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542293,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36752,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335088,"statfs":{"total":96636764160,"available":96599130112,"internally_reserved":0,"allocated":1445888,"data_stored":631812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:17:59.982 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:18:00.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.199+0000 7f47dbe00640 1 Processor -- start 2024-04-04T06:18:00.193 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.199+0000 7f47dbe00640 1 -- start start 2024-04-04T06:18:00.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.200+0000 7f47dbe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47d415c390 0x7f47d4073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:00.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.200+0000 7f47dbe00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f47d40af830 0x7f47d4160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:00.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.200+0000 7f47dbe00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47d415fe80 con 0x7f47d4160b80 2024-04-04T06:18:00.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.200+0000 7f47dbe00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f47d415c770 con 0x7f47d40af830 2024-04-04T06:18:00.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.200+0000 7f47dbe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f47d40ad120 con 0x7f47d415c390 2024-04-04T06:18:00.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.201+0000 7f47d9e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f47d4160b80 0x7f47d40d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52238/0 (socket says 172.21.15.159:52238) 2024-04-04T06:18:00.195 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.201+0000 7f47d9e00640 1 -- 172.21.15.159:0/3474530344 learned_addr learned my addr 172.21.15.159:0/3474530344 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:18:00.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.201+0000 7f47d8a00640 1 --2- 172.21.15.159:0/3474530344 >> v2:172.21.15.178:3300/0 conn(0x7f47d40af830 0x7f47d4160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:00.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.201+0000 7f47d9400640 1 --2- 172.21.15.159:0/3474530344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47d415c390 0x7f47d4073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:00.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.201+0000 7f47d3e00640 1 -- 172.21.15.159:0/3474530344 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 997703072 0 0) 0x7f47d415fe80 con 0x7f47d4160b80 2024-04-04T06:18:00.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.201+0000 7f47d3e00640 1 -- 172.21.15.159:0/3474530344 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47bc003520 con 0x7f47d4160b80 2024-04-04T06:18:00.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.202+0000 7f47d3e00640 1 -- 172.21.15.159:0/3474530344 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2165122189 0 0) 0x7f47bc003520 con 0x7f47d4160b80 2024-04-04T06:18:00.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.202+0000 7f47d3e00640 1 -- 172.21.15.159:0/3474530344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47d415c390 msgr2=0x7f47d4073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:00.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.202+0000 7f47d3e00640 1 --2- 172.21.15.159:0/3474530344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47d415c390 0x7f47d4073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:00.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.202+0000 7f47d3e00640 1 -- 172.21.15.159:0/3474530344 >> v2:172.21.15.178:3300/0 conn(0x7f47d40af830 msgr2=0x7f47d4160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:00.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.202+0000 7f47d3e00640 1 --2- 172.21.15.159:0/3474530344 >> v2:172.21.15.178:3300/0 conn(0x7f47d40af830 0x7f47d4160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:00.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.202+0000 7f47d3e00640 1 -- 172.21.15.159:0/3474530344 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d40ad400 con 0x7f47d4160b80 2024-04-04T06:18:00.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.202+0000 7f47d3e00640 1 -- 172.21.15.159:0/3474530344 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f47cc0019f0 con 0x7f47d4160b80 2024-04-04T06:18:00.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.202+0000 7f47d3e00640 1 -- 172.21.15.159:0/3474530344 >> v1:172.21.15.159:6789/0 conn(0x7f47d4160b80 legacy=0x7f47d40d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:00.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.202+0000 7f47d3e00640 1 --2- 172.21.15.159:0/3474530344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47bc0039b0 0x7f47bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:00.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.202+0000 7f47d3e00640 1 --2- 172.21.15.159:0/3474530344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47bc004800 0x7f47bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:00.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.202+0000 7f47d3e00640 1 --2- 172.21.15.159:0/3474530344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47d415c390 0x7f47bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:00.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.202+0000 7f47d3e00640 1 -- 172.21.15.159:0/3474530344 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f47d40ad120 con 0x7f47bc004800 2024-04-04T06:18:00.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.202+0000 7f47d3e00640 1 -- 172.21.15.159:0/3474530344 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc00b9c0 con 0x7f47bc0039b0 2024-04-04T06:18:00.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.202+0000 7f47d3e00640 1 -- 172.21.15.159:0/3474530344 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f47bc00bb40 con 0x7f47d415c390 2024-04-04T06:18:00.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.202+0000 7f47d8a00640 1 --2- 172.21.15.159:0/3474530344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47bc004800 0x7f47bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:00.198 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.203+0000 7f47d9400640 1 --2- 172.21.15.159:0/3474530344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47bc0039b0 0x7f47bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:00.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.203+0000 7f47d9e00640 1 --2- 172.21.15.159:0/3474530344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47d415c390 0x7f47bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:00.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.203+0000 7f47d8a00640 1 -- 172.21.15.159:0/3474530344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47d415c390 msgr2=0x7f47bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:00.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.203+0000 7f47d8a00640 1 --2- 172.21.15.159:0/3474530344 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47d415c390 0x7f47bc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:00.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.203+0000 7f47d8a00640 1 -- 172.21.15.159:0/3474530344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47bc0039b0 msgr2=0x7f47bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:00.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.203+0000 7f47d8a00640 1 --2- 172.21.15.159:0/3474530344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47bc0039b0 0x7f47bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:00.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.203+0000 7f47d8a00640 1 -- 172.21.15.159:0/3474530344 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47bc003520 con 0x7f47bc004800 2024-04-04T06:18:00.199 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.203+0000 7f47d9400640 1 --2- 172.21.15.159:0/3474530344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47bc0039b0 0x7f47bc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:18:00.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.204+0000 7f47d8a00640 1 --2- 172.21.15.159:0/3474530344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47bc004800 0x7f47bc004c80 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009ac0 tx=0x7f47bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e620906f32c310 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:00.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.204+0000 7f47d3e00640 1 -- 172.21.15.159:0/3474530344 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0039070 con 0x7f47bc004800 2024-04-04T06:18:00.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.204+0000 7f47d3e00640 1 -- 172.21.15.159:0/3474530344 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c002c6d0 con 0x7f47bc004800 2024-04-04T06:18:00.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.204+0000 7f47d3e00640 1 -- 172.21.15.159:0/3474530344 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0009690 con 0x7f47bc004800 2024-04-04T06:18:00.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.205+0000 7f47dbe00640 1 -- 172.21.15.159:0/3474530344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47bc004800 msgr2=0x7f47bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:00.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.205+0000 7f47dbe00640 1 --2- 172.21.15.159:0/3474530344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47bc004800 0x7f47bc004c80 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009ac0 tx=0x7f47bc005830 comp rx=0 tx=0).stop 2024-04-04T06:18:00.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.205+0000 7f47d9e00640 1 -- 172.21.15.159:0/3474530344 reap_dead start 2024-04-04T06:18:00.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.205+0000 7f47dbe00640 1 -- 172.21.15.159:0/3474530344 shutdown_connections 2024-04-04T06:18:00.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.205+0000 7f47dbe00640 1 -- 172.21.15.159:0/3474530344 >> 172.21.15.159:0/3474530344 conn(0x7f47d406b990 msgr2=0x7f47d406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:00.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.205+0000 7f47dbe00640 1 -- 172.21.15.159:0/3474530344 shutdown_connections 2024-04-04T06:18:00.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.205+0000 7f47dbe00640 1 -- 172.21.15.159:0/3474530344 wait complete. 2024-04-04T06:18:00.201 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.207+0000 7f47dbe00640 1 Processor -- start 2024-04-04T06:18:00.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.207+0000 7f47dbe00640 1 -- start start 2024-04-04T06:18:00.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.208+0000 7f47dbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47bc004800 0x7f47d41562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:00.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.208+0000 7f47dbe00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47d40b1b10 0x7f47d40b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:00.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.208+0000 7f47dbe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47d41f89c0 0x7f47d41f8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:00.203 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.208+0000 7f47dbe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f47d40cf980 con 0x7f47bc004800 2024-04-04T06:18:00.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.208+0000 7f47dbe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f47d4160120 con 0x7f47d41f89c0 2024-04-04T06:18:00.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.208+0000 7f47dbe00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f47d40d0d60 con 0x7f47d40b1b10 2024-04-04T06:18:00.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.208+0000 7f47d9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47bc004800 0x7f47d41562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:00.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.208+0000 7f47d9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47bc004800 0x7f47d41562e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55082/0 (socket says 172.21.15.159:55082) 2024-04-04T06:18:00.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.208+0000 7f47d9400640 1 -- 172.21.15.159:0/3742808651 learned_addr learned my addr 172.21.15.159:0/3742808651 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:00.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.208+0000 7f47d8a00640 1 --2- 172.21.15.159:0/3742808651 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47d40b1b10 0x7f47d40b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:00.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.208+0000 7f47d9400640 1 -- 172.21.15.159:0/3742808651 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47d40b1b10 msgr2=0x7f47d40b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:00.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.208+0000 7f47d9400640 1 --2- 172.21.15.159:0/3742808651 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47d40b1b10 0x7f47d40b1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:00.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.208+0000 7f47d9400640 1 -- 172.21.15.159:0/3742808651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47d41f89c0 msgr2=0x7f47d41f8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:00.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.208+0000 7f47d9400640 1 --2- 172.21.15.159:0/3742808651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47d41f89c0 0x7f47d41f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:00.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.208+0000 7f47d9400640 1 -- 172.21.15.159:0/3742808651 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d415d6b0 con 0x7f47bc004800 2024-04-04T06:18:00.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.209+0000 7f47d9400640 1 --2- 172.21.15.159:0/3742808651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47bc004800 0x7f47d41562e0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f47c80108e0 tx=0x7f47c8010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:00.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.209+0000 7f47d1600640 1 -- 172.21.15.159:0/3742808651 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c800f040 con 0x7f47bc004800 2024-04-04T06:18:00.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.209+0000 7f47d1600640 1 -- 172.21.15.159:0/3742808651 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c800ec20 con 0x7f47bc004800 2024-04-04T06:18:00.204 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.209+0000 7f47dbe00640 1 -- 172.21.15.159:0/3742808651 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d415c8f0 con 0x7f47bc004800 2024-04-04T06:18:00.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.209+0000 7f47d1600640 1 -- 172.21.15.159:0/3742808651 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c801c400 con 0x7f47bc004800 2024-04-04T06:18:00.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.209+0000 7f47dbe00640 1 -- 172.21.15.159:0/3742808651 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d415ceb0 con 0x7f47bc004800 2024-04-04T06:18:00.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.210+0000 7f47dbe00640 1 -- 172.21.15.159:0/3742808651 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d40685c0 con 0x7f47bc004800 2024-04-04T06:18:00.205 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.211+0000 7f47d1600640 1 -- 172.21.15.159:0/3742808651 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47c801c5a0 con 0x7f47bc004800 2024-04-04T06:18:00.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.212+0000 7f47d1600640 1 --2- 172.21.15.159:0/3742808651 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f47b406c920 0x7f47b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:00.206 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.213+0000 7f47d1600640 1 -- 172.21.15.159:0/3742808651 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f47c808c070 con 0x7f47bc004800 2024-04-04T06:18:00.207 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.213+0000 7f47d8a00640 1 --2- 172.21.15.159:0/3742808651 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f47b406c920 0x7f47b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:00.208 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.214+0000 7f47d8a00640 1 --2- 172.21.15.159:0/3742808651 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f47b406c920 0x7f47b406ede0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f47c0036000 tx=0x7f47c0030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:00.210 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.216+0000 7f47d1600640 1 -- 172.21.15.159:0/3742808651 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f47c8018030 con 0x7f47bc004800 2024-04-04T06:18:00.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.413+0000 7f47dbe00640 1 -- 172.21.15.159:0/3742808651 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f47d406f1b0 con 0x7f47b406c920 2024-04-04T06:18:00.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.415+0000 7f47d1600640 1 -- 172.21.15.159:0/3742808651 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13523 (secure 0 0 0) 0x7f47d406f1b0 con 0x7f47b406c920 2024-04-04T06:18:00.410 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:18:00.410 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:18:00.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.418+0000 7f47dbe00640 1 -- 172.21.15.159:0/3742808651 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f47b406c920 msgr2=0x7f47b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:00.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.418+0000 7f47dbe00640 1 --2- 172.21.15.159:0/3742808651 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f47b406c920 0x7f47b406ede0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f47c0036000 tx=0x7f47c0030000 comp rx=0 tx=0).stop 2024-04-04T06:18:00.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.418+0000 7f47dbe00640 1 -- 172.21.15.159:0/3742808651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47bc004800 msgr2=0x7f47d41562e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:00.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.418+0000 7f47dbe00640 1 --2- 172.21.15.159:0/3742808651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47bc004800 0x7f47d41562e0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f47c80108e0 tx=0x7f47c8010da0 comp rx=0 tx=0).stop 2024-04-04T06:18:00.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.418+0000 7f47dbe00640 1 -- 172.21.15.159:0/3742808651 shutdown_connections 2024-04-04T06:18:00.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.418+0000 7f47dbe00640 1 --2- 172.21.15.159:0/3742808651 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f47b406c920 0x7f47b406ede0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:00.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.418+0000 7f47dbe00640 1 --2- 172.21.15.159:0/3742808651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47d41f89c0 0x7f47d41f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:00.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.418+0000 7f47dbe00640 1 --2- 172.21.15.159:0/3742808651 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47d40b1b10 0x7f47d40b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:00.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.418+0000 7f47dbe00640 1 --2- 172.21.15.159:0/3742808651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47bc004800 0x7f47d41562e0 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:00.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.418+0000 7f47dbe00640 1 -- 172.21.15.159:0/3742808651 >> 172.21.15.159:0/3742808651 conn(0x7f47d406b990 msgr2=0x7f47d40aece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:00.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.419+0000 7f47dbe00640 1 -- 172.21.15.159:0/3742808651 shutdown_connections 2024-04-04T06:18:00.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.419+0000 7f47dbe00640 1 -- 172.21.15.159:0/3742808651 wait complete. 2024-04-04T06:18:00.433 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2024-04-04T06:17:58.985500+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":295016,"kb_used_data":28936,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754679704,"statfs":{"total":773094113280,"available":772792016896,"internally_reserved":0,"allocated":29630464,"data_stored":23025841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392250,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-178,"num_write_kb":-8306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002414"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":349,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-04T06:17:56.887104+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:17:56.887104+0000","last_peered":"2024-04-04T06:17:56.887104+0000","last_clean":"2024-04-04T06:17:56.887104+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:17:56.887104+0000","last_undegraded":"2024-04-04T06:17:56.887104+0000","last_fullsized":"2024-04-04T06:17:56.887104+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37340,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334500,"statfs":{"total":96636764160,"available":96598528000,"internally_reserved":0,"allocated":2048000,"data_stored":1222796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509591,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40856,"kb_used_data":5516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330984,"statfs":{"total":96636764160,"available":96594927616,"internally_reserved":0,"allocated":5648384,"data_stored":4826193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509592,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36708,"kb_used_data":5528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335132,"statfs":{"total":96636764160,"available":96599175168,"internally_reserved":0,"allocated":5660672,"data_stored":4829018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509591,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1449984,"data_stored":631817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509590,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":5524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":5656576,"data_stored":4832349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509592,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37292,"kb_used_data":6112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":6258688,"data_stored":5419428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542295,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36768,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335072,"statfs":{"total":96636764160,"available":96599113728,"internally_reserved":0,"allocated":1462272,"data_stored":632428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542293,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36752,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335088,"statfs":{"total":96636764160,"available":96599130112,"internally_reserved":0,"allocated":1445888,"data_stored":631812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:18:00.433 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-04T06:18:00.433 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:18:00.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.649+0000 7f99c7a00640 1 Processor -- start 2024-04-04T06:18:00.643 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.649+0000 7f99c7a00640 1 -- start start 2024-04-04T06:18:00.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.651+0000 7f99c7a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f99c00d0e90 0x7f99c00b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:00.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.651+0000 7f99c7a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99c015b6f0 con 0x7f99c015b310 2024-04-04T06:18:00.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.651+0000 7f99c7a00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99c0072e80 con 0x7f99c00727f0 2024-04-04T06:18:00.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.651+0000 7f99c7a00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f99c0072c50 con 0x7f99c00d0e90 2024-04-04T06:18:00.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.651+0000 7f99c5a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f99c00d0e90 0x7f99c00b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:00.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.651+0000 7f99c5000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f99c015b310 0x7f99c015f7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52250/0 (socket says 172.21.15.159:52250) 2024-04-04T06:18:00.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.651+0000 7f99c5a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f99c00d0e90 0x7f99c00b4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55098/0 (socket says 172.21.15.159:55098) 2024-04-04T06:18:00.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.651+0000 7f99c5000640 1 -- 172.21.15.159:0/812358142 learned_addr learned my addr 172.21.15.159:0/812358142 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:18:00.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.651+0000 7f99c5a00640 1 -- 172.21.15.159:0/812358142 >> v1:172.21.15.178:6789/0 conn(0x7f99c00727f0 legacy=0x7f99c00d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:00.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.652+0000 7f99c5a00640 1 -- 172.21.15.159:0/812358142 >> v1:172.21.15.159:6789/0 conn(0x7f99c015b310 legacy=0x7f99c015f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:00.646 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.652+0000 7f99c5a00640 1 -- 172.21.15.159:0/812358142 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c0073160 con 0x7f99c00d0e90 2024-04-04T06:18:00.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.653+0000 7f99c5a00640 1 --2- 172.21.15.159:0/812358142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99c00d0e90 0x7f99c00b4f90 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009dc0 tx=0x7f99b802df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4960511f7f97a39 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:00.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.653+0000 7f99bfe00640 1 -- 172.21.15.159:0/812358142 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b803b070 con 0x7f99c00d0e90 2024-04-04T06:18:00.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.653+0000 7f99bfe00640 1 -- 172.21.15.159:0/812358142 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b802ed30 con 0x7f99c00d0e90 2024-04-04T06:18:00.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.653+0000 7f99bfe00640 1 -- 172.21.15.159:0/812358142 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8036660 con 0x7f99c00d0e90 2024-04-04T06:18:00.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.654+0000 7f99c7a00640 1 -- 172.21.15.159:0/812358142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99c00d0e90 msgr2=0x7f99c00b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:00.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.654+0000 7f99c7a00640 1 --2- 172.21.15.159:0/812358142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99c00d0e90 0x7f99c00b4f90 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009dc0 tx=0x7f99b802df80 comp rx=0 tx=0).stop 2024-04-04T06:18:00.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.654+0000 7f99c7a00640 1 -- 172.21.15.159:0/812358142 shutdown_connections 2024-04-04T06:18:00.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.654+0000 7f99c7a00640 1 --2- 172.21.15.159:0/812358142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99c00d0e90 0x7f99c00b4f90 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:00.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.654+0000 7f99c7a00640 1 -- 172.21.15.159:0/812358142 >> 172.21.15.159:0/812358142 conn(0x7f99c006b990 msgr2=0x7f99c006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:00.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.654+0000 7f99c7a00640 1 -- 172.21.15.159:0/812358142 shutdown_connections 2024-04-04T06:18:00.649 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.654+0000 7f99c7a00640 1 -- 172.21.15.159:0/812358142 wait complete. 2024-04-04T06:18:00.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.656+0000 7f99c7a00640 1 Processor -- start 2024-04-04T06:18:00.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.656+0000 7f99c7a00640 1 -- start start 2024-04-04T06:18:00.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.657+0000 7f99c7a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99c00727f0 0x7f99c0152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:00.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.657+0000 7f99c7a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99c00d0e90 0x7f99c01e4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:00.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.657+0000 7f99c7a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99c015b310 0x7f99c01471a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:00.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.657+0000 7f99c7a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f99c0073440 con 0x7f99c015b310 2024-04-04T06:18:00.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.657+0000 7f99c7a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f99c015f2c0 con 0x7f99c00727f0 2024-04-04T06:18:00.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.657+0000 7f99c7a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f99c00adc80 con 0x7f99c00d0e90 2024-04-04T06:18:00.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.658+0000 7f99c5a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99c015b310 0x7f99c01471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:00.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.658+0000 7f99c5a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99c015b310 0x7f99c01471a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55108/0 (socket says 172.21.15.159:55108) 2024-04-04T06:18:00.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.658+0000 7f99c5a00640 1 -- 172.21.15.159:0/193352176 learned_addr learned my addr 172.21.15.159:0/193352176 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:00.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.658+0000 7f99c5000640 1 --2- 172.21.15.159:0/193352176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99c00727f0 0x7f99c0152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:00.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.658+0000 7f99b7e00640 1 --2- 172.21.15.159:0/193352176 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99c00d0e90 0x7f99c01e4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:00.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.658+0000 7f99c5a00640 1 -- 172.21.15.159:0/193352176 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99c00d0e90 msgr2=0x7f99c01e4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:00.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.658+0000 7f99c5a00640 1 --2- 172.21.15.159:0/193352176 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99c00d0e90 0x7f99c01e4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:00.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.658+0000 7f99c5a00640 1 -- 172.21.15.159:0/193352176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99c00727f0 msgr2=0x7f99c0152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:00.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.658+0000 7f99c5a00640 1 --2- 172.21.15.159:0/193352176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99c00727f0 0x7f99c0152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:00.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.658+0000 7f99c5a00640 1 -- 172.21.15.159:0/193352176 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c0157230 con 0x7f99c015b310 2024-04-04T06:18:00.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.659+0000 7f99c5a00640 1 --2- 172.21.15.159:0/193352176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99c015b310 0x7f99c01471a0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f99b8039040 tx=0x7f99b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:00.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.659+0000 7f99bd600640 1 -- 172.21.15.159:0/193352176 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b803b040 con 0x7f99c015b310 2024-04-04T06:18:00.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.659+0000 7f99bd600640 1 -- 172.21.15.159:0/193352176 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b800e080 con 0x7f99c015b310 2024-04-04T06:18:00.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.659+0000 7f99c7a00640 1 -- 172.21.15.159:0/193352176 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c0157510 con 0x7f99c015b310 2024-04-04T06:18:00.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.659+0000 7f99bd600640 1 -- 172.21.15.159:0/193352176 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b803f660 con 0x7f99c015b310 2024-04-04T06:18:00.654 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.659+0000 7f99c7a00640 1 -- 172.21.15.159:0/193352176 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c0157ad0 con 0x7f99c015b310 2024-04-04T06:18:00.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.660+0000 7f99c7a00640 1 -- 172.21.15.159:0/193352176 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c00685c0 con 0x7f99c015b310 2024-04-04T06:18:00.655 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.661+0000 7f99bd600640 1 -- 172.21.15.159:0/193352176 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f99b8046020 con 0x7f99c015b310 2024-04-04T06:18:00.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.662+0000 7f99bd600640 1 --2- 172.21.15.159:0/193352176 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f999c06c8f0 0x7f999c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:00.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.662+0000 7f99bd600640 1 -- 172.21.15.159:0/193352176 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f99b80ad0c0 con 0x7f99c015b310 2024-04-04T06:18:00.656 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.663+0000 7f99c5000640 1 --2- 172.21.15.159:0/193352176 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f999c06c8f0 0x7f999c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:00.658 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.664+0000 7f99c5000640 1 --2- 172.21.15.159:0/193352176 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f999c06c8f0 0x7f999c06edb0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f99b0000f60 tx=0x7f99b0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:00.660 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.667+0000 7f99bd600640 1 -- 172.21.15.159:0/193352176 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f99b804d020 con 0x7f99c015b310 2024-04-04T06:18:00.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.863+0000 7f99c7a00640 1 -- 172.21.15.159:0/193352176 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99c015ec20 con 0x7f999c06c8f0 2024-04-04T06:18:00.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.865+0000 7f99bd600640 1 -- 172.21.15.159:0/193352176 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13523 (secure 0 0 0) 0x7f99c015ec20 con 0x7f999c06c8f0 2024-04-04T06:18:00.860 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:18:00.860 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:18:00.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.868+0000 7f99c7a00640 1 -- 172.21.15.159:0/193352176 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f999c06c8f0 msgr2=0x7f999c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:00.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.868+0000 7f99c7a00640 1 --2- 172.21.15.159:0/193352176 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f999c06c8f0 0x7f999c06edb0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f99b0000f60 tx=0x7f99b0009040 comp rx=0 tx=0).stop 2024-04-04T06:18:00.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.868+0000 7f99c7a00640 1 -- 172.21.15.159:0/193352176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99c015b310 msgr2=0x7f99c01471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:00.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.868+0000 7f99c7a00640 1 --2- 172.21.15.159:0/193352176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99c015b310 0x7f99c01471a0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f99b8039040 tx=0x7f99b8004a40 comp rx=0 tx=0).stop 2024-04-04T06:18:00.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.869+0000 7f99c7a00640 1 -- 172.21.15.159:0/193352176 shutdown_connections 2024-04-04T06:18:00.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.869+0000 7f99c7a00640 1 --2- 172.21.15.159:0/193352176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99c015b310 0x7f99c01471a0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:00.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.869+0000 7f99c7a00640 1 --2- 172.21.15.159:0/193352176 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f99c00d0e90 0x7f99c01e4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:00.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.869+0000 7f99c7a00640 1 --2- 172.21.15.159:0/193352176 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f999c06c8f0 0x7f999c06edb0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:00.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.869+0000 7f99c7a00640 1 --2- 172.21.15.159:0/193352176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99c00727f0 0x7f99c0152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:00.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.869+0000 7f99c7a00640 1 -- 172.21.15.159:0/193352176 >> 172.21.15.159:0/193352176 conn(0x7f99c006b990 msgr2=0x7f99c00b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:00.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.869+0000 7f99c7a00640 1 -- 172.21.15.159:0/193352176 shutdown_connections 2024-04-04T06:18:00.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:00.869+0000 7f99c7a00640 1 -- 172.21.15.159:0/193352176 wait complete. 2024-04-04T06:18:00.882 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2024-04-04T06:17:58.985500+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":295016,"kb_used_data":28936,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754679704,"statfs":{"total":773094113280,"available":772792016896,"internally_reserved":0,"allocated":29630464,"data_stored":23025841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392250,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-178,"num_write_kb":-8306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002414"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":349,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-04T06:17:56.887104+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:17:56.887104+0000","last_peered":"2024-04-04T06:17:56.887104+0000","last_clean":"2024-04-04T06:17:56.887104+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:17:56.887104+0000","last_undegraded":"2024-04-04T06:17:56.887104+0000","last_fullsized":"2024-04-04T06:17:56.887104+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37340,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334500,"statfs":{"total":96636764160,"available":96598528000,"internally_reserved":0,"allocated":2048000,"data_stored":1222796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509591,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40856,"kb_used_data":5516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330984,"statfs":{"total":96636764160,"available":96594927616,"internally_reserved":0,"allocated":5648384,"data_stored":4826193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509592,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36708,"kb_used_data":5528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335132,"statfs":{"total":96636764160,"available":96599175168,"internally_reserved":0,"allocated":5660672,"data_stored":4829018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509591,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1449984,"data_stored":631817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509590,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":5524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":5656576,"data_stored":4832349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509592,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37292,"kb_used_data":6112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":6258688,"data_stored":5419428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542295,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36768,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335072,"statfs":{"total":96636764160,"available":96599113728,"internally_reserved":0,"allocated":1462272,"data_stored":632428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542293,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36752,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335088,"statfs":{"total":96636764160,"available":96599130112,"internally_reserved":0,"allocated":1445888,"data_stored":631812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:18:00.882 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-04T06:18:01.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.089+0000 7feafd200640 1 Processor -- start 2024-04-04T06:18:01.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.090+0000 7feafd200640 1 -- start start 2024-04-04T06:18:01.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.091+0000 7feafd200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7feaf815c390 0x7feaf8073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:01.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.091+0000 7feafd200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf80af830 0x7feaf8160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:01.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.091+0000 7feafd200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaf815fe80 con 0x7feaf8160b80 2024-04-04T06:18:01.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.091+0000 7feafd200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7feaf815c770 con 0x7feaf815c390 2024-04-04T06:18:01.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.091+0000 7feafd200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feaf80ad120 con 0x7feaf80af830 2024-04-04T06:18:01.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.091+0000 7feaf6800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7feaf8160b80 0x7feaf80d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52252/0 (socket says 172.21.15.159:52252) 2024-04-04T06:18:01.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.091+0000 7feaf6800640 1 -- 172.21.15.159:0/3512760772 learned_addr learned my addr 172.21.15.159:0/3512760772 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:18:01.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.092+0000 7feaf5400640 1 --2- 172.21.15.159:0/3512760772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf80af830 0x7feaf8160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:01.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.092+0000 7feaf5e00640 1 --2- 172.21.15.159:0/3512760772 >> v2:172.21.15.178:3300/0 conn(0x7feaf815c390 0x7feaf8073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:01.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.092+0000 7feaf4a00640 1 -- 172.21.15.159:0/3512760772 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2105316273 0 0) 0x7feaf815fe80 con 0x7feaf8160b80 2024-04-04T06:18:01.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.092+0000 7feaf4a00640 1 -- 172.21.15.159:0/3512760772 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fead8003520 con 0x7feaf8160b80 2024-04-04T06:18:01.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.092+0000 7feaf4a00640 1 -- 172.21.15.159:0/3512760772 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 457477854 0 0) 0x7fead8003520 con 0x7feaf8160b80 2024-04-04T06:18:01.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.092+0000 7feaf4a00640 1 -- 172.21.15.159:0/3512760772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf80af830 msgr2=0x7feaf8160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:01.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.092+0000 7feaf4a00640 1 --2- 172.21.15.159:0/3512760772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf80af830 0x7feaf8160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:01.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.092+0000 7feaf4a00640 1 -- 172.21.15.159:0/3512760772 >> v2:172.21.15.178:3300/0 conn(0x7feaf815c390 msgr2=0x7feaf8073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:01.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.092+0000 7feaf4a00640 1 --2- 172.21.15.159:0/3512760772 >> v2:172.21.15.178:3300/0 conn(0x7feaf815c390 0x7feaf8073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:01.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.092+0000 7feaf4a00640 1 -- 172.21.15.159:0/3512760772 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf80ad400 con 0x7feaf8160b80 2024-04-04T06:18:01.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.092+0000 7feaf4a00640 1 -- 172.21.15.159:0/3512760772 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7feaf0001670 con 0x7feaf8160b80 2024-04-04T06:18:01.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.092+0000 7feaf4a00640 1 -- 172.21.15.159:0/3512760772 >> v1:172.21.15.159:6789/0 conn(0x7feaf8160b80 legacy=0x7feaf80d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:01.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.092+0000 7feaf4a00640 1 --2- 172.21.15.159:0/3512760772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf80af830 0x7fead80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:01.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.092+0000 7feaf4a00640 1 --2- 172.21.15.159:0/3512760772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fead8003f30 0x7fead8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:01.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.092+0000 7feaf4a00640 1 --2- 172.21.15.159:0/3512760772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fead8004d40 0x7fead80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:01.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.092+0000 7feaf4a00640 1 -- 172.21.15.159:0/3512760772 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feaf80ad120 con 0x7fead8004d40 2024-04-04T06:18:01.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.092+0000 7feaf4a00640 1 -- 172.21.15.159:0/3512760772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fead800b9c0 con 0x7fead8003f30 2024-04-04T06:18:01.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.092+0000 7feaf4a00640 1 -- 172.21.15.159:0/3512760772 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fead800bb90 con 0x7feaf80af830 2024-04-04T06:18:01.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.092+0000 7feaf5e00640 1 --2- 172.21.15.159:0/3512760772 >> v2:172.21.15.178:3300/0 conn(0x7feaf815c390 0x7feaf8073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-04T06:18:01.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.092+0000 7feaf6800640 1 --2- 172.21.15.159:0/3512760772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fead8004d40 0x7fead80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:01.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.093+0000 7feaf5400640 1 --2- 172.21.15.159:0/3512760772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fead8003f30 0x7fead8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:01.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.093+0000 7feaf6800640 1 -- 172.21.15.159:0/3512760772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf80af830 msgr2=0x7fead80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:01.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.093+0000 7feaf6800640 1 --2- 172.21.15.159:0/3512760772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf80af830 0x7fead80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:01.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.093+0000 7feaf6800640 1 -- 172.21.15.159:0/3512760772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fead8003f30 msgr2=0x7fead8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:01.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.093+0000 7feaf6800640 1 --2- 172.21.15.159:0/3512760772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fead8003f30 0x7fead8004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:01.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.093+0000 7feaf6800640 1 -- 172.21.15.159:0/3512760772 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead8003520 con 0x7fead8004d40 2024-04-04T06:18:01.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.094+0000 7feaf6800640 1 --2- 172.21.15.159:0/3512760772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fead8004d40 0x7fead80051c0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7feaf0007fd0 tx=0x7fead8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb13fcd92e2de4fe server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:01.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.094+0000 7feaf4a00640 1 -- 172.21.15.159:0/3512760772 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0007d50 con 0x7fead8004d40 2024-04-04T06:18:01.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.094+0000 7feaf4a00640 1 -- 172.21.15.159:0/3512760772 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf0030070 con 0x7fead8004d40 2024-04-04T06:18:01.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.094+0000 7feaf4a00640 1 -- 172.21.15.159:0/3512760772 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf002cd80 con 0x7fead8004d40 2024-04-04T06:18:01.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.095+0000 7feafd200640 1 -- 172.21.15.159:0/3512760772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fead8004d40 msgr2=0x7fead80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:01.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.095+0000 7feafd200640 1 --2- 172.21.15.159:0/3512760772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fead8004d40 0x7fead80051c0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7feaf0007fd0 tx=0x7fead8005830 comp rx=0 tx=0).stop 2024-04-04T06:18:01.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.095+0000 7feaf6800640 1 -- 172.21.15.159:0/3512760772 reap_dead start 2024-04-04T06:18:01.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.095+0000 7feafd200640 1 -- 172.21.15.159:0/3512760772 shutdown_connections 2024-04-04T06:18:01.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.095+0000 7feafd200640 1 -- 172.21.15.159:0/3512760772 >> 172.21.15.159:0/3512760772 conn(0x7feaf806b990 msgr2=0x7feaf806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:01.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.095+0000 7feafd200640 1 -- 172.21.15.159:0/3512760772 shutdown_connections 2024-04-04T06:18:01.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.095+0000 7feafd200640 1 -- 172.21.15.159:0/3512760772 wait complete. 2024-04-04T06:18:01.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.097+0000 7feafd200640 1 Processor -- start 2024-04-04T06:18:01.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.097+0000 7feafd200640 1 -- start start 2024-04-04T06:18:01.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.098+0000 7feafd200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fead8004d40 0x7feaf815efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:01.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.098+0000 7feafd200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf8155090 0x7feaf8151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:01.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.098+0000 7feafd200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feaf81f7e20 0x7feaf81f82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:01.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.098+0000 7feafd200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feaf80cf980 con 0x7feaf81f7e20 2024-04-04T06:18:01.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.098+0000 7feafd200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feaf8160120 con 0x7fead8004d40 2024-04-04T06:18:01.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.098+0000 7feafd200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feaf80d0d60 con 0x7feaf8155090 2024-04-04T06:18:01.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.098+0000 7feaf6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feaf81f7e20 0x7feaf81f82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:01.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.098+0000 7feaf6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feaf81f7e20 0x7feaf81f82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55120/0 (socket says 172.21.15.159:55120) 2024-04-04T06:18:01.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.098+0000 7feaf6800640 1 -- 172.21.15.159:0/1833671420 learned_addr learned my addr 172.21.15.159:0/1833671420 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:01.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.098+0000 7feaf5e00640 1 --2- 172.21.15.159:0/1833671420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fead8004d40 0x7feaf815efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:01.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.098+0000 7feaf5400640 1 --2- 172.21.15.159:0/1833671420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf8155090 0x7feaf8151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:01.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.098+0000 7feaf6800640 1 -- 172.21.15.159:0/1833671420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf8155090 msgr2=0x7feaf8151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:01.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.098+0000 7feaf6800640 1 --2- 172.21.15.159:0/1833671420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf8155090 0x7feaf8151f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:01.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.098+0000 7feaf6800640 1 -- 172.21.15.159:0/1833671420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fead8004d40 msgr2=0x7feaf815efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:01.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.098+0000 7feaf6800640 1 --2- 172.21.15.159:0/1833671420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fead8004d40 0x7feaf815efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:01.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.098+0000 7feaf6800640 1 -- 172.21.15.159:0/1833671420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf80b3b50 con 0x7feaf81f7e20 2024-04-04T06:18:01.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.099+0000 7feaf6800640 1 --2- 172.21.15.159:0/1833671420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feaf81f7e20 0x7feaf81f82b0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7feaf002dd70 tx=0x7feaf80ad5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:01.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.099+0000 7feaee000640 1 -- 172.21.15.159:0/1833671420 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0007d50 con 0x7feaf81f7e20 2024-04-04T06:18:01.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.099+0000 7feaee000640 1 -- 172.21.15.159:0/1833671420 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf0004bf0 con 0x7feaf81f7e20 2024-04-04T06:18:01.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.099+0000 7feafd200640 1 -- 172.21.15.159:0/1833671420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf80b3e30 con 0x7feaf81f7e20 2024-04-04T06:18:01.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.099+0000 7feaee000640 1 -- 172.21.15.159:0/1833671420 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0001c80 con 0x7feaf81f7e20 2024-04-04T06:18:01.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.099+0000 7feafd200640 1 -- 172.21.15.159:0/1833671420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf80b43f0 con 0x7feaf81f7e20 2024-04-04T06:18:01.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.100+0000 7feafd200640 1 -- 172.21.15.159:0/1833671420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf8073100 con 0x7feaf81f7e20 2024-04-04T06:18:01.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.101+0000 7feaee000640 1 -- 172.21.15.159:0/1833671420 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feaf0007070 con 0x7feaf81f7e20 2024-04-04T06:18:01.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.102+0000 7feaee000640 1 --2- 172.21.15.159:0/1833671420 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fead406c920 0x7fead406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:01.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.102+0000 7feaee000640 1 -- 172.21.15.159:0/1833671420 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7feaf00ac4b0 con 0x7feaf81f7e20 2024-04-04T06:18:01.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.103+0000 7feaf5e00640 1 --2- 172.21.15.159:0/1833671420 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fead406c920 0x7fead406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:01.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.104+0000 7feaf5e00640 1 --2- 172.21.15.159:0/1833671420 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fead406c920 0x7fead406ede0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7feae8006fd0 tx=0x7feae8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:01.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.107+0000 7feaee000640 1 -- 172.21.15.159:0/1833671420 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7feaf007aa00 con 0x7feaf81f7e20 2024-04-04T06:18:01.302 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.308+0000 7feafd200640 1 -- 172.21.15.159:0/1833671420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feaf80685c0 con 0x7feaf81f7e20 2024-04-04T06:18:01.303 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.310+0000 7feaee000640 1 -- 172.21.15.159:0/1833671420 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+13306 (secure 0 0 0) 0x7feaf007a300 con 0x7feaf81f7e20 2024-04-04T06:18:01.304 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:18:01.304 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":119,"fsid":"30311045-f714-48fa-832b-4cb2ba8bf0bf","created":"2024-04-04T06:05:27.666873+0000","modified":"2024-04-04T06:17:57.128471+0000","last_up_change":"2024-04-04T06:05:41.062861+0000","last_in_change":"2024-04-04T06:05:32.259925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-04T06:05:40.750131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6fd6dfe9-0d71-40c2-a535-7bb5be04f685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6801","nonce":455382518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6803","nonce":455382518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6807","nonce":455382518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":455382518},{"type":"v1","addr":"172.21.15.159:6805","nonce":455382518}]},"public_addr":"172.21.15.159:6801/455382518","cluster_addr":"172.21.15.159:6803/455382518","heartbeat_back_addr":"172.21.15.159:6807/455382518","heartbeat_front_addr":"172.21.15.159:6805/455382518","state":["exists","up"]},{"osd":1,"uuid":"3c4ff6d4-2734-4993-a993-cb75cce59323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6817","nonce":3765338297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6819","nonce":3765338297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6830","nonce":3765338297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3765338297},{"type":"v1","addr":"172.21.15.159:6825","nonce":3765338297}]},"public_addr":"172.21.15.159:6817/3765338297","cluster_addr":"172.21.15.159:6819/3765338297","heartbeat_back_addr":"172.21.15.159:6830/3765338297","heartbeat_front_addr":"172.21.15.159:6825/3765338297","state":["exists","up"]},{"osd":2,"uuid":"a00bdbab-b841-4e3e-88ea-c1bf744b3753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6809","nonce":2619731047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6811","nonce":2619731047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6815","nonce":2619731047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2619731047},{"type":"v1","addr":"172.21.15.159:6813","nonce":2619731047}]},"public_addr":"172.21.15.159:6809/2619731047","cluster_addr":"172.21.15.159:6811/2619731047","heartbeat_back_addr":"172.21.15.159:6815/2619731047","heartbeat_front_addr":"172.21.15.159:6813/2619731047","state":["exists","up"]},{"osd":3,"uuid":"34e5c351-621f-4a57-9c6a-24582faca139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6821","nonce":2070760205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6823","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6824","nonce":2070760205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6829","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6831","nonce":2070760205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2070760205},{"type":"v1","addr":"172.21.15.159:6827","nonce":2070760205}]},"public_addr":"172.21.15.159:6821/2070760205","cluster_addr":"172.21.15.159:6824/2070760205","heartbeat_back_addr":"172.21.15.159:6831/2070760205","heartbeat_front_addr":"172.21.15.159:6827/2070760205","state":["exists","up"]},{"osd":4,"uuid":"2e340e67-8903-40e4-87aa-8e03b538ad9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6814","nonce":4035991429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6818","nonce":4035991429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6825","nonce":4035991429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":4035991429},{"type":"v1","addr":"172.21.15.178:6822","nonce":4035991429}]},"public_addr":"172.21.15.178:6814/4035991429","cluster_addr":"172.21.15.178:6818/4035991429","heartbeat_back_addr":"172.21.15.178:6825/4035991429","heartbeat_front_addr":"172.21.15.178:6822/4035991429","state":["exists","up"]},{"osd":5,"uuid":"9d0aead7-b517-4d97-b37e-747b7fc1c03d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6803","nonce":63746765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6805","nonce":63746765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6809","nonce":63746765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":63746765},{"type":"v1","addr":"172.21.15.178:6807","nonce":63746765}]},"public_addr":"172.21.15.178:6803/63746765","cluster_addr":"172.21.15.178:6805/63746765","heartbeat_back_addr":"172.21.15.178:6809/63746765","heartbeat_front_addr":"172.21.15.178:6807/63746765","state":["exists","up"]},{"osd":6,"uuid":"472edb50-4e29-40ab-bee7-91b081541e1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6827","nonce":198949157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6829","nonce":198949157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6833","nonce":198949157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":198949157},{"type":"v1","addr":"172.21.15.178:6831","nonce":198949157}]},"public_addr":"172.21.15.178:6827/198949157","cluster_addr":"172.21.15.178:6829/198949157","heartbeat_back_addr":"172.21.15.178:6833/198949157","heartbeat_front_addr":"172.21.15.178:6831/198949157","state":["exists","up"]},{"osd":7,"uuid":"3ea70d65-8690-4b91-a011-b3897a96e780","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6811","nonce":1066817938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6815","nonce":1066817938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6823","nonce":1066817938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6817","nonce":1066817938},{"type":"v1","addr":"172.21.15.178:6819","nonce":1066817938}]},"public_addr":"172.21.15.178:6811/1066817938","cluster_addr":"172.21.15.178:6815/1066817938","heartbeat_back_addr":"172.21.15.178:6823/1066817938","heartbeat_front_addr":"172.21.15.178:6819/1066817938","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.536047+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.633565+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.524649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.330821+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.573229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.478399+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.815195+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-04T06:05:37.832826+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.159:6834/643543929":"2024-04-05T06:17:54.117877+0000","172.21.15.159:6832/643543929":"2024-04-05T06:17:54.117877+0000","172.21.15.159:6834/2439261212":"2024-04-05T06:17:35.667507+0000","172.21.15.159:6832/2439261212":"2024-04-05T06:17:35.667507+0000","172.21.15.159:6834/1017894858":"2024-04-05T06:17:21.596764+0000","172.21.15.159:6832/1017894858":"2024-04-05T06:17:21.596764+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-04T06:18:01.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.312+0000 7feafd200640 1 -- 172.21.15.159:0/1833671420 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fead406c920 msgr2=0x7fead406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:01.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.312+0000 7feafd200640 1 --2- 172.21.15.159:0/1833671420 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fead406c920 0x7fead406ede0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7feae8006fd0 tx=0x7feae8008040 comp rx=0 tx=0).stop 2024-04-04T06:18:01.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.312+0000 7feafd200640 1 -- 172.21.15.159:0/1833671420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feaf81f7e20 msgr2=0x7feaf81f82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:01.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.312+0000 7feafd200640 1 --2- 172.21.15.159:0/1833671420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feaf81f7e20 0x7feaf81f82b0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7feaf002dd70 tx=0x7feaf80ad5d0 comp rx=0 tx=0).stop 2024-04-04T06:18:01.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.312+0000 7feafd200640 1 -- 172.21.15.159:0/1833671420 shutdown_connections 2024-04-04T06:18:01.307 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.312+0000 7feafd200640 1 --2- 172.21.15.159:0/1833671420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feaf81f7e20 0x7feaf81f82b0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:01.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.312+0000 7feafd200640 1 --2- 172.21.15.159:0/1833671420 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feaf8155090 0x7feaf8151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:01.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.312+0000 7feafd200640 1 --2- 172.21.15.159:0/1833671420 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fead406c920 0x7fead406ede0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:01.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.312+0000 7feafd200640 1 --2- 172.21.15.159:0/1833671420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fead8004d40 0x7feaf815efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:01.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.312+0000 7feafd200640 1 -- 172.21.15.159:0/1833671420 >> 172.21.15.159:0/1833671420 conn(0x7feaf806b990 msgr2=0x7feaf80b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:01.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.313+0000 7feafd200640 1 -- 172.21.15.159:0/1833671420 shutdown_connections 2024-04-04T06:18:01.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:01.313+0000 7feafd200640 1 -- 172.21.15.159:0/1833671420 wait complete. 2024-04-04T06:18:02.327 INFO:tasks.ceph:Scrubbing osd.0 2024-04-04T06:18:02.327 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-04T06:18:02.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.557+0000 7f0f47200640 1 Processor -- start 2024-04-04T06:18:02.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.557+0000 7f0f47200640 1 -- start start 2024-04-04T06:18:02.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.559+0000 7f0f47200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0f4815e420 0x7f0f480b9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:02.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.559+0000 7f0f47200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f480b6310 0x7f0f480b6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:02.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.559+0000 7f0f47200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f480d23b0 con 0x7f0f480d3d50 2024-04-04T06:18:02.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.559+0000 7f0f47200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f0f480b6cc0 con 0x7f0f4815e420 2024-04-04T06:18:02.553 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.559+0000 7f0f47200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0f480d2fc0 con 0x7f0f480b6310 2024-04-04T06:18:02.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.559+0000 7f0f45400640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f0f480d3d50 0x7f0f480d3450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52254/0 (socket says 172.21.15.159:52254) 2024-04-04T06:18:02.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.559+0000 7f0f45400640 1 -- 172.21.15.159:0/129919638 learned_addr learned my addr 172.21.15.159:0/129919638 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:18:02.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.560+0000 7f0f45e00640 1 --2- 172.21.15.159:0/129919638 >> v2:172.21.15.178:3300/0 conn(0x7f0f4815e420 0x7f0f480b9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:02.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.560+0000 7f0f46800640 1 --2- 172.21.15.159:0/129919638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f480b6310 0x7f0f480b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:02.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.560+0000 7f0f44a00640 1 -- 172.21.15.159:0/129919638 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2976035078 0 0) 0x7f0f480d23b0 con 0x7f0f480d3d50 2024-04-04T06:18:02.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.560+0000 7f0f44a00640 1 -- 172.21.15.159:0/129919638 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f2c003520 con 0x7f0f480d3d50 2024-04-04T06:18:02.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.560+0000 7f0f44a00640 1 -- 172.21.15.159:0/129919638 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2093734534 0 0) 0x7f0f2c003520 con 0x7f0f480d3d50 2024-04-04T06:18:02.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.560+0000 7f0f44a00640 1 -- 172.21.15.159:0/129919638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f480b6310 msgr2=0x7f0f480b6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:02.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.560+0000 7f0f44a00640 1 --2- 172.21.15.159:0/129919638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f480b6310 0x7f0f480b6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:02.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.560+0000 7f0f44a00640 1 -- 172.21.15.159:0/129919638 >> v2:172.21.15.178:3300/0 conn(0x7f0f4815e420 msgr2=0x7f0f480b9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:02.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.560+0000 7f0f44a00640 1 --2- 172.21.15.159:0/129919638 >> v2:172.21.15.178:3300/0 conn(0x7f0f4815e420 0x7f0f480b9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:02.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.560+0000 7f0f44a00640 1 -- 172.21.15.159:0/129919638 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f480d2690 con 0x7f0f480d3d50 2024-04-04T06:18:02.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.560+0000 7f0f44a00640 1 -- 172.21.15.159:0/129919638 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f0f30001670 con 0x7f0f480d3d50 2024-04-04T06:18:02.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.560+0000 7f0f44a00640 1 -- 172.21.15.159:0/129919638 >> v1:172.21.15.159:6789/0 conn(0x7f0f480d3d50 legacy=0x7f0f480d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:02.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.560+0000 7f0f44a00640 1 --2- 172.21.15.159:0/129919638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f2c0039b0 0x7f0f2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:02.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.560+0000 7f0f44a00640 1 --2- 172.21.15.159:0/129919638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f2c004800 0x7f0f2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:02.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.560+0000 7f0f44a00640 1 --2- 172.21.15.159:0/129919638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f480b6310 0x7f0f2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:02.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.560+0000 7f0f44a00640 1 -- 172.21.15.159:0/129919638 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0f480b6cc0 con 0x7f0f2c0039b0 2024-04-04T06:18:02.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.560+0000 7f0f44a00640 1 -- 172.21.15.159:0/129919638 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0f480d2fc0 con 0x7f0f2c004800 2024-04-04T06:18:02.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.560+0000 7f0f44a00640 1 -- 172.21.15.159:0/129919638 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0f2c00b9c0 con 0x7f0f480b6310 2024-04-04T06:18:02.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.561+0000 7f0f45e00640 1 --2- 172.21.15.159:0/129919638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f2c0039b0 0x7f0f2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:02.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.561+0000 7f0f45400640 1 --2- 172.21.15.159:0/129919638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f2c004800 0x7f0f2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:02.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.561+0000 7f0f45e00640 1 -- 172.21.15.159:0/129919638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f480b6310 msgr2=0x7f0f2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:02.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.561+0000 7f0f45e00640 1 --2- 172.21.15.159:0/129919638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f480b6310 0x7f0f2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:02.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.561+0000 7f0f45e00640 1 -- 172.21.15.159:0/129919638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f2c004800 msgr2=0x7f0f2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:02.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.561+0000 7f0f45e00640 1 --2- 172.21.15.159:0/129919638 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f2c004800 0x7f0f2c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:02.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.561+0000 7f0f45e00640 1 -- 172.21.15.159:0/129919638 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2c003520 con 0x7f0f2c0039b0 2024-04-04T06:18:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.562+0000 7f0f45e00640 1 --2- 172.21.15.159:0/129919638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f2c0039b0 0x7f0f2c003dd0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009fc0 tx=0x7f0f2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e51261aa9f6f0af server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.562+0000 7f0f44a00640 1 -- 172.21.15.159:0/129919638 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38039070 con 0x7f0f2c0039b0 2024-04-04T06:18:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.562+0000 7f0f44a00640 1 -- 172.21.15.159:0/129919638 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f3802cc60 con 0x7f0f2c0039b0 2024-04-04T06:18:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.562+0000 7f0f44a00640 1 -- 172.21.15.159:0/129919638 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38009c10 con 0x7f0f2c0039b0 2024-04-04T06:18:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.563+0000 7f0f47200640 1 -- 172.21.15.159:0/129919638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f2c0039b0 msgr2=0x7f0f2c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.563+0000 7f0f47200640 1 --2- 172.21.15.159:0/129919638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f2c0039b0 0x7f0f2c003dd0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009fc0 tx=0x7f0f2c005830 comp rx=0 tx=0).stop 2024-04-04T06:18:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.563+0000 7f0f46800640 1 -- 172.21.15.159:0/129919638 reap_dead start 2024-04-04T06:18:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.563+0000 7f0f47200640 1 -- 172.21.15.159:0/129919638 shutdown_connections 2024-04-04T06:18:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.563+0000 7f0f47200640 1 -- 172.21.15.159:0/129919638 >> 172.21.15.159:0/129919638 conn(0x7f0f480676a0 msgr2=0x7f0f48067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:02.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.563+0000 7f0f47200640 1 -- 172.21.15.159:0/129919638 shutdown_connections 2024-04-04T06:18:02.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.563+0000 7f0f47200640 1 -- 172.21.15.159:0/129919638 wait complete. 2024-04-04T06:18:02.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.565+0000 7f0f47200640 1 Processor -- start 2024-04-04T06:18:02.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.565+0000 7f0f47200640 1 -- start start 2024-04-04T06:18:02.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.566+0000 7f0f47200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f2c0039b0 0x7f0f481f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:02.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.566+0000 7f0f47200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f48157b20 0x7f0f481f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:02.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.566+0000 7f0f47200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f4815fec0 0x7f0f48160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:02.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.566+0000 7f0f47200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0f480ba190 con 0x7f0f2c0039b0 2024-04-04T06:18:02.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.566+0000 7f0f47200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0f480d2c50 con 0x7f0f48157b20 2024-04-04T06:18:02.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.566+0000 7f0f47200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0f4815e800 con 0x7f0f4815fec0 2024-04-04T06:18:02.561 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.566+0000 7f0f45e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f2c0039b0 0x7f0f481f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:02.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.567+0000 7f0f45e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f2c0039b0 0x7f0f481f3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55136/0 (socket says 172.21.15.159:55136) 2024-04-04T06:18:02.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.567+0000 7f0f45e00640 1 -- 172.21.15.159:0/52715088 learned_addr learned my addr 172.21.15.159:0/52715088 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:02.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.567+0000 7f0f45e00640 1 -- 172.21.15.159:0/52715088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f4815fec0 msgr2=0x7f0f48160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:02.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.567+0000 7f0f45e00640 1 --2- 172.21.15.159:0/52715088 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f4815fec0 0x7f0f48160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:02.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.567+0000 7f0f45e00640 1 -- 172.21.15.159:0/52715088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f48157b20 msgr2=0x7f0f481f4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:02.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.567+0000 7f0f45e00640 1 --2- 172.21.15.159:0/52715088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f48157b20 0x7f0f481f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:02.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.567+0000 7f0f45e00640 1 -- 172.21.15.159:0/52715088 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f481f4730 con 0x7f0f2c0039b0 2024-04-04T06:18:02.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.567+0000 7f0f45e00640 1 --2- 172.21.15.159:0/52715088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f2c0039b0 0x7f0f481f3c00 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f0f38037040 tx=0x7f0f38009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:02.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.567+0000 7f0f3e000640 1 -- 172.21.15.159:0/52715088 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38039050 con 0x7f0f2c0039b0 2024-04-04T06:18:02.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.568+0000 7f0f3e000640 1 -- 172.21.15.159:0/52715088 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f38045070 con 0x7f0f2c0039b0 2024-04-04T06:18:02.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.568+0000 7f0f3e000640 1 -- 172.21.15.159:0/52715088 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38040690 con 0x7f0f2c0039b0 2024-04-04T06:18:02.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.568+0000 7f0f47200640 1 -- 172.21.15.159:0/52715088 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f481f4a10 con 0x7f0f2c0039b0 2024-04-04T06:18:02.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.568+0000 7f0f47200640 1 -- 172.21.15.159:0/52715088 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f480c92f0 con 0x7f0f2c0039b0 2024-04-04T06:18:02.563 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.569+0000 7f0f47200640 1 -- 172.21.15.159:0/52715088 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0f480c8e00 con 0x7f0f2c0039b0 2024-04-04T06:18:02.564 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.570+0000 7f0f3e000640 1 -- 172.21.15.159:0/52715088 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f38005d10 con 0x7f0f2c0039b0 2024-04-04T06:18:02.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.571+0000 7f0f3e000640 1 --2- 172.21.15.159:0/52715088 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f2406c920 0x7f0f2406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:02.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.571+0000 7f0f3e000640 1 -- 172.21.15.159:0/52715088 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f0f380ae080 con 0x7f0f2c0039b0 2024-04-04T06:18:02.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.571+0000 7f0f3e000640 1 --2- 172.21.15.159:0/52715088 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x7f0f24072920 0x7f0f24074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:02.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.571+0000 7f0f3e000640 1 -- 172.21.15.159:0/52715088 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0f3800df90 con 0x7f0f24072920 2024-04-04T06:18:02.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.571+0000 7f0f3e000640 1 -- 172.21.15.159:0/52715088 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7f0f380b5050 con 0x7f0f2c0039b0 2024-04-04T06:18:02.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.571+0000 7f0f46800640 1 --2- 172.21.15.159:0/52715088 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x7f0f24072920 0x7f0f24074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:02.565 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.571+0000 7f0f45400640 1 --2- 172.21.15.159:0/52715088 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f2406c920 0x7f0f2406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:02.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.572+0000 7f0f46800640 1 --2- 172.21.15.159:0/52715088 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x7f0f24072920 0x7f0f24074d40 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:02.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.572+0000 7f0f45400640 1 --2- 172.21.15.159:0/52715088 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f2406c920 0x7f0f2406ede0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0f30001920 tx=0x7f0f30009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:02.566 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.573+0000 7f0f3e000640 1 -- 172.21.15.159:0/52715088 <== osd.0 v2:172.21.15.159:6800/455382518 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0f3800df90 con 0x7f0f24072920 2024-04-04T06:18:02.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.587+0000 7f0f47200640 1 -- 172.21.15.159:0/52715088 --> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0f48075940 con 0x7f0f24072920 2024-04-04T06:18:02.581 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.588+0000 7f0f3e000640 1 -- 172.21.15.159:0/52715088 <== osd.0 v2:172.21.15.159:6800/455382518 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0f48075940 con 0x7f0f24072920 2024-04-04T06:18:02.582 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:18:02.582 INFO:teuthology.orchestra.run.smithi159.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-04T06:18:02.582 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:18:02.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.588+0000 7f0f47200640 1 -- 172.21.15.159:0/52715088 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x7f0f24072920 msgr2=0x7f0f24074d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:02.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.588+0000 7f0f47200640 1 --2- 172.21.15.159:0/52715088 >> [v2:172.21.15.159:6800/455382518,v1:172.21.15.159:6801/455382518] conn(0x7f0f24072920 0x7f0f24074d40 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:02.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.588+0000 7f0f47200640 1 -- 172.21.15.159:0/52715088 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f2406c920 msgr2=0x7f0f2406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:02.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.588+0000 7f0f47200640 1 --2- 172.21.15.159:0/52715088 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0f2406c920 0x7f0f2406ede0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0f30001920 tx=0x7f0f30009040 comp rx=0 tx=0).stop 2024-04-04T06:18:02.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.588+0000 7f0f47200640 1 -- 172.21.15.159:0/52715088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f2c0039b0 msgr2=0x7f0f481f3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:02.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.588+0000 7f0f47200640 1 --2- 172.21.15.159:0/52715088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f2c0039b0 0x7f0f481f3c00 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f0f38037040 tx=0x7f0f38009750 comp rx=0 tx=0).stop 2024-04-04T06:18:02.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.588+0000 7f0f46800640 1 -- 172.21.15.159:0/52715088 reap_dead start 2024-04-04T06:18:02.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.588+0000 7f0f47200640 1 -- 172.21.15.159:0/52715088 shutdown_connections 2024-04-04T06:18:02.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.588+0000 7f0f47200640 1 -- 172.21.15.159:0/52715088 >> 172.21.15.159:0/52715088 conn(0x7f0f480676a0 msgr2=0x7f0f480b5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:02.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.589+0000 7f0f47200640 1 -- 172.21.15.159:0/52715088 shutdown_connections 2024-04-04T06:18:02.583 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.589+0000 7f0f47200640 1 -- 172.21.15.159:0/52715088 wait complete. 2024-04-04T06:18:02.600 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-04T06:18:02.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.816+0000 7fa94ec00640 1 Processor -- start 2024-04-04T06:18:02.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.816+0000 7fa94ec00640 1 -- start start 2024-04-04T06:18:02.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.818+0000 7fa94ec00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa9480cf1b0 0x7fa9480cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:02.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.818+0000 7fa94ec00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9480d00d0 0x7fa9480b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:02.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.818+0000 7fa94ec00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa948071460 con 0x7fa9481442c0 2024-04-04T06:18:02.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.818+0000 7fa94ec00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fa948071d20 con 0x7fa9480cf1b0 2024-04-04T06:18:02.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.818+0000 7fa94ec00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa9480721b0 con 0x7fa9480d00d0 2024-04-04T06:18:02.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.818+0000 7fa94d800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fa9481442c0 0x7fa9480ceaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52266/0 (socket says 172.21.15.159:52266) 2024-04-04T06:18:02.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.818+0000 7fa94d800640 1 -- 172.21.15.159:0/1687864084 learned_addr learned my addr 172.21.15.159:0/1687864084 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:18:02.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.818+0000 7fa94e200640 1 --2- 172.21.15.159:0/1687864084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9480d00d0 0x7fa9480b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:02.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.818+0000 7fa94ce00640 1 --2- 172.21.15.159:0/1687864084 >> v2:172.21.15.178:3300/0 conn(0x7fa9480cf1b0 0x7fa9480cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:02.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.818+0000 7fa947e00640 1 -- 172.21.15.159:0/1687864084 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1526355146 0 0) 0x7fa948071460 con 0x7fa9481442c0 2024-04-04T06:18:02.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.819+0000 7fa947e00640 1 -- 172.21.15.159:0/1687864084 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa930003520 con 0x7fa9481442c0 2024-04-04T06:18:02.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.819+0000 7fa94ce00640 1 -- 172.21.15.159:0/1687864084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9480d00d0 msgr2=0x7fa9480b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:02.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.819+0000 7fa94ce00640 1 --2- 172.21.15.159:0/1687864084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9480d00d0 0x7fa9480b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:02.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.819+0000 7fa94ce00640 1 -- 172.21.15.159:0/1687864084 >> v1:172.21.15.159:6789/0 conn(0x7fa9481442c0 legacy=0x7fa9480ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:02.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.819+0000 7fa94ce00640 1 -- 172.21.15.159:0/1687864084 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9480716d0 con 0x7fa9480cf1b0 2024-04-04T06:18:02.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.821+0000 7fa94ce00640 1 --2- 172.21.15.159:0/1687864084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9480cf1b0 0x7fa9480cfb90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa9380097a0 tx=0x7fa93802b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d13d1b76c1030b server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:02.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.821+0000 7fa947e00640 1 -- 172.21.15.159:0/1687864084 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938034070 con 0x7fa9480cf1b0 2024-04-04T06:18:02.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.821+0000 7fa947e00640 1 -- 172.21.15.159:0/1687864084 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa938030800 con 0x7fa9480cf1b0 2024-04-04T06:18:02.815 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.821+0000 7fa947e00640 1 -- 172.21.15.159:0/1687864084 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938030bb0 con 0x7fa9480cf1b0 2024-04-04T06:18:02.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.822+0000 7fa94ec00640 1 -- 172.21.15.159:0/1687864084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9480cf1b0 msgr2=0x7fa9480cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:02.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.822+0000 7fa94ec00640 1 --2- 172.21.15.159:0/1687864084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9480cf1b0 0x7fa9480cfb90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa9380097a0 tx=0x7fa93802b940 comp rx=0 tx=0).stop 2024-04-04T06:18:02.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.822+0000 7fa94ec00640 1 -- 172.21.15.159:0/1687864084 shutdown_connections 2024-04-04T06:18:02.816 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.822+0000 7fa94ec00640 1 --2- 172.21.15.159:0/1687864084 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9480d00d0 0x7fa9480b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:02.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.822+0000 7fa94ec00640 1 --2- 172.21.15.159:0/1687864084 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9480cf1b0 0x7fa9480cfb90 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:02.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.822+0000 7fa94ec00640 1 -- 172.21.15.159:0/1687864084 >> 172.21.15.159:0/1687864084 conn(0x7fa9480ccad0 msgr2=0x7fa9480cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:02.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.822+0000 7fa94ec00640 1 -- 172.21.15.159:0/1687864084 shutdown_connections 2024-04-04T06:18:02.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.822+0000 7fa94ec00640 1 -- 172.21.15.159:0/1687864084 wait complete. 2024-04-04T06:18:02.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.824+0000 7fa94ec00640 1 Processor -- start 2024-04-04T06:18:02.817 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.824+0000 7fa94ec00640 1 -- start start 2024-04-04T06:18:02.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.825+0000 7fa94ec00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9480cf1b0 0x7fa9481406c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:02.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.825+0000 7fa94ec00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9480d00d0 0x7fa94815d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:02.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.825+0000 7fa94ec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9481442c0 0x7fa94815d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:02.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.825+0000 7fa94ec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481446a0 con 0x7fa9481442c0 2024-04-04T06:18:02.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.825+0000 7fa94ec00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa9480d0770 con 0x7fa9480d00d0 2024-04-04T06:18:02.820 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.825+0000 7fa94ec00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fa9480c6b10 con 0x7fa9480cf1b0 2024-04-04T06:18:02.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.825+0000 7fa94e200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9481442c0 0x7fa94815d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:02.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.825+0000 7fa94e200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9481442c0 0x7fa94815d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55138/0 (socket says 172.21.15.159:55138) 2024-04-04T06:18:02.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.825+0000 7fa94e200640 1 -- 172.21.15.159:0/931514234 learned_addr learned my addr 172.21.15.159:0/931514234 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:02.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.825+0000 7fa94d800640 1 --2- 172.21.15.159:0/931514234 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9480cf1b0 0x7fa9481406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:02.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.825+0000 7fa94ce00640 1 --2- 172.21.15.159:0/931514234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9480d00d0 0x7fa94815d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:02.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.825+0000 7fa94e200640 1 -- 172.21.15.159:0/931514234 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9480cf1b0 msgr2=0x7fa9481406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:02.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.825+0000 7fa94e200640 1 --2- 172.21.15.159:0/931514234 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9480cf1b0 0x7fa9481406c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:02.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.825+0000 7fa94e200640 1 -- 172.21.15.159:0/931514234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9480d00d0 msgr2=0x7fa94815d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:02.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.825+0000 7fa94e200640 1 --2- 172.21.15.159:0/931514234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9480d00d0 0x7fa94815d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:02.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.825+0000 7fa94e200640 1 -- 172.21.15.159:0/931514234 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9481f4cc0 con 0x7fa9481442c0 2024-04-04T06:18:02.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.825+0000 7fa94ce00640 1 --2- 172.21.15.159:0/931514234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9480d00d0 0x7fa94815d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:18:02.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.826+0000 7fa94e200640 1 --2- 172.21.15.159:0/931514234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9481442c0 0x7fa94815d650 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fa94000efc0 tx=0x7fa94000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:02.821 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.826+0000 7fa945600640 1 -- 172.21.15.159:0/931514234 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940018020 con 0x7fa9481442c0 2024-04-04T06:18:02.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.826+0000 7fa945600640 1 -- 172.21.15.159:0/931514234 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa940014480 con 0x7fa9481442c0 2024-04-04T06:18:02.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.826+0000 7fa94ec00640 1 -- 172.21.15.159:0/931514234 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9480c4fb0 con 0x7fa9481442c0 2024-04-04T06:18:02.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.826+0000 7fa945600640 1 -- 172.21.15.159:0/931514234 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940014850 con 0x7fa9481442c0 2024-04-04T06:18:02.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.826+0000 7fa94ec00640 1 -- 172.21.15.159:0/931514234 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9480c5570 con 0x7fa9481442c0 2024-04-04T06:18:02.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.827+0000 7fa94ec00640 1 -- 172.21.15.159:0/931514234 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa948075940 con 0x7fa9481442c0 2024-04-04T06:18:02.822 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.828+0000 7fa945600640 1 -- 172.21.15.159:0/931514234 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa940005d10 con 0x7fa9481442c0 2024-04-04T06:18:02.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.830+0000 7fa945600640 1 --2- 172.21.15.159:0/931514234 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa92c06c8f0 0x7fa92c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:02.824 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.831+0000 7fa945600640 1 -- 172.21.15.159:0/931514234 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7fa94008c300 con 0x7fa9481442c0 2024-04-04T06:18:02.825 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.831+0000 7fa94d800640 1 --2- 172.21.15.159:0/931514234 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa92c06c8f0 0x7fa92c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:02.826 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.832+0000 7fa94d800640 1 --2- 172.21.15.159:0/931514234 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa92c06c8f0 0x7fa92c06edb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa93c005880 tx=0x7fa93c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:02.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:02.835+0000 7fa945600640 1 -- 172.21.15.159:0/931514234 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fa940022070 con 0x7fa9481442c0 2024-04-04T06:18:03.031 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.036+0000 7fa94ec00640 1 -- 172.21.15.159:0/931514234 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fa9480cfb80 con 0x7fa92c06c8f0 2024-04-04T06:18:03.032 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.038+0000 7fa945600640 1 -- 172.21.15.159:0/931514234 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa9480cfb80 con 0x7fa92c06c8f0 2024-04-04T06:18:03.032 INFO:teuthology.orchestra.run.smithi159.stderr:instructed osd(s) 0 to deep-scrub 2024-04-04T06:18:03.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.040+0000 7fa94ec00640 1 -- 172.21.15.159:0/931514234 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa92c06c8f0 msgr2=0x7fa92c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:03.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.040+0000 7fa94ec00640 1 --2- 172.21.15.159:0/931514234 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa92c06c8f0 0x7fa92c06edb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa93c005880 tx=0x7fa93c009040 comp rx=0 tx=0).stop 2024-04-04T06:18:03.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.040+0000 7fa94ec00640 1 -- 172.21.15.159:0/931514234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9481442c0 msgr2=0x7fa94815d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:03.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.040+0000 7fa94ec00640 1 --2- 172.21.15.159:0/931514234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9481442c0 0x7fa94815d650 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fa94000efc0 tx=0x7fa94000c490 comp rx=0 tx=0).stop 2024-04-04T06:18:03.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.040+0000 7fa94ec00640 1 -- 172.21.15.159:0/931514234 shutdown_connections 2024-04-04T06:18:03.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.040+0000 7fa94ec00640 1 --2- 172.21.15.159:0/931514234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9481442c0 0x7fa94815d650 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.040+0000 7fa94ec00640 1 --2- 172.21.15.159:0/931514234 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fa92c06c8f0 0x7fa92c06edb0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.040+0000 7fa94ec00640 1 --2- 172.21.15.159:0/931514234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9480d00d0 0x7fa94815d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.040+0000 7fa94ec00640 1 --2- 172.21.15.159:0/931514234 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9480cf1b0 0x7fa9481406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.035 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.040+0000 7fa94ec00640 1 -- 172.21.15.159:0/931514234 >> 172.21.15.159:0/931514234 conn(0x7fa9480ccad0 msgr2=0x7fa9480b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:03.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.041+0000 7fa94ec00640 1 -- 172.21.15.159:0/931514234 shutdown_connections 2024-04-04T06:18:03.036 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.041+0000 7fa94ec00640 1 -- 172.21.15.159:0/931514234 wait complete. 2024-04-04T06:18:03.053 INFO:tasks.ceph:Scrubbing osd.1 2024-04-04T06:18:03.053 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-04T06:18:03.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.274+0000 7f36b1400640 1 Processor -- start 2024-04-04T06:18:03.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.274+0000 7f36b1400640 1 -- start start 2024-04-04T06:18:03.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.276+0000 7f36b1400640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f36ac0acf10 0x7f36ac09dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:03.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.276+0000 7f36b1400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f36ac0ad2f0 0x7f36ac1453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:03.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.276+0000 7f36b1400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36ac15cb90 con 0x7f36ac15ff70 2024-04-04T06:18:03.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.276+0000 7f36b1400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f36ac145f60 con 0x7f36ac0ad2f0 2024-04-04T06:18:03.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.276+0000 7f36b1400640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f36ac15c950 con 0x7f36ac0acf10 2024-04-04T06:18:03.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.276+0000 7f36a9e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f36ac0ad2f0 0x7f36ac1453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:03.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.276+0000 7f36a9e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f36ac0ad2f0 0x7f36ac1453f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55146/0 (socket says 172.21.15.159:55146) 2024-04-04T06:18:03.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.276+0000 7f36a9e00640 1 -- 172.21.15.159:0/3193434683 learned_addr learned my addr 172.21.15.159:0/3193434683 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:03.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.277+0000 7f36aa800640 1 --2- 172.21.15.159:0/3193434683 >> v2:172.21.15.178:3300/0 conn(0x7f36ac0acf10 0x7f36ac09dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:03.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.277+0000 7f36a9e00640 1 -- 172.21.15.159:0/3193434683 >> v2:172.21.15.178:3300/0 conn(0x7f36ac0acf10 msgr2=0x7f36ac09dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:03.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.277+0000 7f36a9e00640 1 --2- 172.21.15.159:0/3193434683 >> v2:172.21.15.178:3300/0 conn(0x7f36ac0acf10 0x7f36ac09dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.277+0000 7f36a9e00640 1 -- 172.21.15.159:0/3193434683 >> v1:172.21.15.178:6789/0 conn(0x7f36ac15ff70 legacy=0x7f36ac160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:03.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.277+0000 7f36a9e00640 1 -- 172.21.15.159:0/3193434683 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ac15ce70 con 0x7f36ac0ad2f0 2024-04-04T06:18:03.272 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.277+0000 7f36aa800640 1 --2- 172.21.15.159:0/3193434683 >> v2:172.21.15.178:3300/0 conn(0x7f36ac0acf10 0x7f36ac09dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:18:03.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.278+0000 7f36a9e00640 1 --2- 172.21.15.159:0/3193434683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36ac0ad2f0 0x7f36ac1453f0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f36ac06c580 tx=0x7f369802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7173feb1271280c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:03.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.278+0000 7f36a9400640 1 -- 172.21.15.159:0/3193434683 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369802e070 con 0x7f36ac0ad2f0 2024-04-04T06:18:03.273 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.279+0000 7f36a9400640 1 -- 172.21.15.159:0/3193434683 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3698037070 con 0x7f36ac0ad2f0 2024-04-04T06:18:03.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.279+0000 7f36a9400640 1 -- 172.21.15.159:0/3193434683 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698031a00 con 0x7f36ac0ad2f0 2024-04-04T06:18:03.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.279+0000 7f36b1400640 1 -- 172.21.15.159:0/3193434683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36ac0ad2f0 msgr2=0x7f36ac1453f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:03.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.279+0000 7f36b1400640 1 --2- 172.21.15.159:0/3193434683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36ac0ad2f0 0x7f36ac1453f0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f36ac06c580 tx=0x7f369802b880 comp rx=0 tx=0).stop 2024-04-04T06:18:03.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.279+0000 7f36b1400640 1 -- 172.21.15.159:0/3193434683 shutdown_connections 2024-04-04T06:18:03.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.279+0000 7f36b1400640 1 --2- 172.21.15.159:0/3193434683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36ac0ad2f0 0x7f36ac1453f0 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.279+0000 7f36b1400640 1 --2- 172.21.15.159:0/3193434683 >> v2:172.21.15.178:3300/0 conn(0x7f36ac0acf10 0x7f36ac09dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.279+0000 7f36b1400640 1 -- 172.21.15.159:0/3193434683 >> 172.21.15.159:0/3193434683 conn(0x7f36ac06bba0 msgr2=0x7f36ac0b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:03.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.279+0000 7f36b1400640 1 -- 172.21.15.159:0/3193434683 shutdown_connections 2024-04-04T06:18:03.274 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.279+0000 7f36b1400640 1 -- 172.21.15.159:0/3193434683 wait complete. 2024-04-04T06:18:03.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.281+0000 7f36b1400640 1 Processor -- start 2024-04-04T06:18:03.275 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.281+0000 7f36b1400640 1 -- start start 2024-04-04T06:18:03.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.282+0000 7f36b1400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36ac0acf10 0x7f36ac13be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:03.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.282+0000 7f36b1400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36ac0ad2f0 0x7f36ac1f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:03.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.282+0000 7f36b1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36ac15ff70 0x7f36ac1f6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:03.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.282+0000 7f36b1400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac15d430 con 0x7f36ac15ff70 2024-04-04T06:18:03.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.282+0000 7f36b1400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac15d7a0 con 0x7f36ac0acf10 2024-04-04T06:18:03.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.282+0000 7f36b1400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f36ac0ad7a0 con 0x7f36ac0ad2f0 2024-04-04T06:18:03.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.282+0000 7f36ab200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36ac15ff70 0x7f36ac1f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:03.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.282+0000 7f36ab200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36ac15ff70 0x7f36ac1f6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55148/0 (socket says 172.21.15.159:55148) 2024-04-04T06:18:03.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.282+0000 7f36ab200640 1 -- 172.21.15.159:0/478736308 learned_addr learned my addr 172.21.15.159:0/478736308 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:03.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.283+0000 7f36aa800640 1 --2- 172.21.15.159:0/478736308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36ac0acf10 0x7f36ac13be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:03.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.283+0000 7f36a9e00640 1 --2- 172.21.15.159:0/478736308 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36ac0ad2f0 0x7f36ac1f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:03.277 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.283+0000 7f36ab200640 1 -- 172.21.15.159:0/478736308 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36ac0ad2f0 msgr2=0x7f36ac1f5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:03.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.283+0000 7f36ab200640 1 --2- 172.21.15.159:0/478736308 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36ac0ad2f0 0x7f36ac1f5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.283+0000 7f36ab200640 1 -- 172.21.15.159:0/478736308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36ac0acf10 msgr2=0x7f36ac13be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:03.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.283+0000 7f36ab200640 1 --2- 172.21.15.159:0/478736308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36ac0acf10 0x7f36ac13be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.278 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.283+0000 7f36ab200640 1 -- 172.21.15.159:0/478736308 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ac13c8a0 con 0x7f36ac15ff70 2024-04-04T06:18:03.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.283+0000 7f36ab200640 1 --2- 172.21.15.159:0/478736308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36ac15ff70 0x7f36ac1f6410 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f36a400b9c0 tx=0x7f36a400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:03.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.283+0000 7f36a2a00640 1 -- 172.21.15.159:0/478736308 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4019070 con 0x7f36ac15ff70 2024-04-04T06:18:03.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.284+0000 7f36a2a00640 1 -- 172.21.15.159:0/478736308 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a400cba0 con 0x7f36ac15ff70 2024-04-04T06:18:03.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.284+0000 7f36a2a00640 1 -- 172.21.15.159:0/478736308 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4014440 con 0x7f36ac15ff70 2024-04-04T06:18:03.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.284+0000 7f36b1400640 1 -- 172.21.15.159:0/478736308 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ac13cb80 con 0x7f36ac15ff70 2024-04-04T06:18:03.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.284+0000 7f36b1400640 1 -- 172.21.15.159:0/478736308 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ac1415d0 con 0x7f36ac15ff70 2024-04-04T06:18:03.279 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.284+0000 7f36b1400640 1 -- 172.21.15.159:0/478736308 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f36ac13cdc0 con 0x7f36ac15ff70 2024-04-04T06:18:03.280 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.286+0000 7f36a2a00640 1 -- 172.21.15.159:0/478736308 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36a401c070 con 0x7f36ac15ff70 2024-04-04T06:18:03.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.287+0000 7f36a2a00640 1 --2- 172.21.15.159:0/478736308 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f368c06c8f0 0x7f368c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:03.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.287+0000 7f36a2a00640 1 -- 172.21.15.159:0/478736308 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f36a408b6f0 con 0x7f36ac15ff70 2024-04-04T06:18:03.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.287+0000 7f36a2a00640 1 --2- 172.21.15.159:0/478736308 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x7f368c0728f0 0x7f368c074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:03.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.287+0000 7f36a2a00640 1 -- 172.21.15.159:0/478736308 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f368c0753c0 con 0x7f368c0728f0 2024-04-04T06:18:03.281 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.287+0000 7f36a2a00640 1 -- 172.21.15.159:0/478736308 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7f36a4092050 con 0x7f36ac15ff70 2024-04-04T06:18:03.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.287+0000 7f36a9e00640 1 --2- 172.21.15.159:0/478736308 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x7f368c0728f0 0x7f368c074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:03.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.287+0000 7f36aa800640 1 --2- 172.21.15.159:0/478736308 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f368c06c8f0 0x7f368c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:03.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.288+0000 7f36a9e00640 1 --2- 172.21.15.159:0/478736308 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x7f368c0728f0 0x7f368c074d10 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:03.282 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.288+0000 7f36aa800640 1 --2- 172.21.15.159:0/478736308 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f368c06c8f0 0x7f368c06edb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f369c00a990 tx=0x7f369c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:03.283 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.289+0000 7f36a2a00640 1 -- 172.21.15.159:0/478736308 <== osd.1 v2:172.21.15.159:6816/3765338297 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f368c0753c0 con 0x7f368c0728f0 2024-04-04T06:18:03.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.303+0000 7f36b1400640 1 -- 172.21.15.159:0/478736308 --> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f36ac0685c0 con 0x7f368c0728f0 2024-04-04T06:18:03.297 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.304+0000 7f36a2a00640 1 -- 172.21.15.159:0/478736308 <== osd.1 v2:172.21.15.159:6816/3765338297 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f36ac0685c0 con 0x7f368c0728f0 2024-04-04T06:18:03.298 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:18:03.298 INFO:teuthology.orchestra.run.smithi159.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-04T06:18:03.298 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:18:03.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.304+0000 7f36b1400640 1 -- 172.21.15.159:0/478736308 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x7f368c0728f0 msgr2=0x7f368c074d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:03.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.304+0000 7f36b1400640 1 --2- 172.21.15.159:0/478736308 >> [v2:172.21.15.159:6816/3765338297,v1:172.21.15.159:6817/3765338297] conn(0x7f368c0728f0 0x7f368c074d10 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.304+0000 7f36b1400640 1 -- 172.21.15.159:0/478736308 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f368c06c8f0 msgr2=0x7f368c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:03.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.304+0000 7f36b1400640 1 --2- 172.21.15.159:0/478736308 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f368c06c8f0 0x7f368c06edb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f369c00a990 tx=0x7f369c008040 comp rx=0 tx=0).stop 2024-04-04T06:18:03.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.304+0000 7f36b1400640 1 -- 172.21.15.159:0/478736308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36ac15ff70 msgr2=0x7f36ac1f6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:03.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.304+0000 7f36b1400640 1 --2- 172.21.15.159:0/478736308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36ac15ff70 0x7f36ac1f6410 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f36a400b9c0 tx=0x7f36a400be80 comp rx=0 tx=0).stop 2024-04-04T06:18:03.299 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.304+0000 7f36ab200640 1 -- 172.21.15.159:0/478736308 reap_dead start 2024-04-04T06:18:03.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.304+0000 7f36b1400640 1 -- 172.21.15.159:0/478736308 shutdown_connections 2024-04-04T06:18:03.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.305+0000 7f36b1400640 1 -- 172.21.15.159:0/478736308 >> 172.21.15.159:0/478736308 conn(0x7f36ac06bba0 msgr2=0x7f36ac15eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:03.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.305+0000 7f36b1400640 1 -- 172.21.15.159:0/478736308 shutdown_connections 2024-04-04T06:18:03.300 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.305+0000 7f36b1400640 1 -- 172.21.15.159:0/478736308 wait complete. 2024-04-04T06:18:03.316 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-04T06:18:03.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.533+0000 7f84c3e00640 1 Processor -- start 2024-04-04T06:18:03.528 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.533+0000 7f84c3e00640 1 -- start start 2024-04-04T06:18:03.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.535+0000 7f84c3e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84c415e4a0 0x7f84c40b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:03.529 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.535+0000 7f84c3e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f84c40d2310 0x7f84c40ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:03.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.535+0000 7f84c3e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84c415e880 con 0x7f84c40cf530 2024-04-04T06:18:03.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.535+0000 7f84c3e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f84c40d34e0 con 0x7f84c40d2310 2024-04-04T06:18:03.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.535+0000 7f84c3e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f84c40d28d0 con 0x7f84c415e4a0 2024-04-04T06:18:03.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.535+0000 7f84c2000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f84c40d2310 0x7f84c40ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:03.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.535+0000 7f84c2000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f84c40d2310 0x7f84c40ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55158/0 (socket says 172.21.15.159:55158) 2024-04-04T06:18:03.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.535+0000 7f84c2000640 1 -- 172.21.15.159:0/3342476440 learned_addr learned my addr 172.21.15.159:0/3342476440 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:03.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.535+0000 7f84c3400640 1 --1- 172.21.15.159:0/3342476440 >> v1:172.21.15.159:6789/0 conn(0x7f84c40cf530 0x7f84c40d3c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52272/0 (socket says 172.21.15.159:52272) 2024-04-04T06:18:03.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.535+0000 7f84c2a00640 1 --2- 172.21.15.159:0/3342476440 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84c415e4a0 0x7f84c40b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:03.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.535+0000 7f84c2000640 1 -- 172.21.15.159:0/3342476440 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84c415e4a0 msgr2=0x7f84c40b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:03.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.535+0000 7f84c2000640 1 --2- 172.21.15.159:0/3342476440 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84c415e4a0 0x7f84c40b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.535+0000 7f84c2000640 1 -- 172.21.15.159:0/3342476440 >> v1:172.21.15.159:6789/0 conn(0x7f84c40cf530 legacy=0x7f84c40d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:03.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.536+0000 7f84c2000640 1 -- 172.21.15.159:0/3342476440 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c40d2bb0 con 0x7f84c40d2310 2024-04-04T06:18:03.530 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.536+0000 7f84c1600640 1 -- 172.21.15.159:0/3342476440 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3501773450 0 0) 0x7f84c415e880 con 0x7f84c40cf530 2024-04-04T06:18:03.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.537+0000 7f84c2000640 1 --2- 172.21.15.159:0/3342476440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c40d2310 0x7f84c40ceff0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009870 tx=0x7f84ac02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5975fe423d38e916 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:03.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.537+0000 7f84c1600640 1 -- 172.21.15.159:0/3342476440 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac034070 con 0x7f84c40d2310 2024-04-04T06:18:03.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.537+0000 7f84c1600640 1 -- 172.21.15.159:0/3342476440 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84ac030840 con 0x7f84c40d2310 2024-04-04T06:18:03.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.537+0000 7f84c1600640 1 -- 172.21.15.159:0/3342476440 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac030c10 con 0x7f84c40d2310 2024-04-04T06:18:03.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.537+0000 7f84c3e00640 1 -- 172.21.15.159:0/3342476440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c40d2310 msgr2=0x7f84c40ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:03.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.537+0000 7f84c3e00640 1 --2- 172.21.15.159:0/3342476440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c40d2310 0x7f84c40ceff0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009870 tx=0x7f84ac02b980 comp rx=0 tx=0).stop 2024-04-04T06:18:03.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.538+0000 7f84c3e00640 1 -- 172.21.15.159:0/3342476440 shutdown_connections 2024-04-04T06:18:03.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.538+0000 7f84c3e00640 1 --2- 172.21.15.159:0/3342476440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c40d2310 0x7f84c40ceff0 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.538+0000 7f84c3e00640 1 --2- 172.21.15.159:0/3342476440 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84c415e4a0 0x7f84c40b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.538+0000 7f84c3e00640 1 -- 172.21.15.159:0/3342476440 >> 172.21.15.159:0/3342476440 conn(0x7f84c40676d0 msgr2=0x7f84c4067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:03.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.538+0000 7f84c3e00640 1 -- 172.21.15.159:0/3342476440 shutdown_connections 2024-04-04T06:18:03.532 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.538+0000 7f84c3e00640 1 -- 172.21.15.159:0/3342476440 wait complete. 2024-04-04T06:18:03.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.539+0000 7f84c3e00640 1 Processor -- start 2024-04-04T06:18:03.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.540+0000 7f84c3e00640 1 -- start start 2024-04-04T06:18:03.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.540+0000 7f84c3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c40cf530 0x7f84c40c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:03.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.541+0000 7f84c3e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84c40d2310 0x7f84c40b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:03.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.541+0000 7f84c3e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84c415e4a0 0x7f84c40b9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:03.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.541+0000 7f84c3e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84c40d3970 con 0x7f84c40cf530 2024-04-04T06:18:03.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.541+0000 7f84c3e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f84c40d3170 con 0x7f84c415e4a0 2024-04-04T06:18:03.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.541+0000 7f84c3e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f84c40ba1a0 con 0x7f84c40d2310 2024-04-04T06:18:03.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.541+0000 7f84c2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c40cf530 0x7f84c40c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:03.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.541+0000 7f84c2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c40cf530 0x7f84c40c9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55174/0 (socket says 172.21.15.159:55174) 2024-04-04T06:18:03.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.541+0000 7f84c2a00640 1 -- 172.21.15.159:0/1559910262 learned_addr learned my addr 172.21.15.159:0/1559910262 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:03.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.541+0000 7f84c2a00640 1 -- 172.21.15.159:0/1559910262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84c40d2310 msgr2=0x7f84c40b8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:03.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.541+0000 7f84c2000640 1 --2- 172.21.15.159:0/1559910262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84c40d2310 0x7f84c40b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:03.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.541+0000 7f84c2a00640 1 --2- 172.21.15.159:0/1559910262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84c40d2310 0x7f84c40b8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.541+0000 7f84c2a00640 1 -- 172.21.15.159:0/1559910262 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84c415e4a0 msgr2=0x7f84c40b9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:03.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.541+0000 7f84c3400640 1 --2- 172.21.15.159:0/1559910262 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84c415e4a0 0x7f84c40b9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:03.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.541+0000 7f84c2a00640 1 --2- 172.21.15.159:0/1559910262 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84c415e4a0 0x7f84c40b9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.541+0000 7f84c2a00640 1 -- 172.21.15.159:0/1559910262 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c4150a80 con 0x7f84c40cf530 2024-04-04T06:18:03.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.542+0000 7f84c2a00640 1 --2- 172.21.15.159:0/1559910262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c40cf530 0x7f84c40c9910 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f84b4007e40 tx=0x7f84b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:03.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.542+0000 7f84baa00640 1 -- 172.21.15.159:0/1559910262 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b4014620 con 0x7f84c40cf530 2024-04-04T06:18:03.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.542+0000 7f84baa00640 1 -- 172.21.15.159:0/1559910262 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84b4019070 con 0x7f84c40cf530 2024-04-04T06:18:03.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.542+0000 7f84baa00640 1 -- 172.21.15.159:0/1559910262 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b4014a00 con 0x7f84c40cf530 2024-04-04T06:18:03.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.542+0000 7f84c3e00640 1 -- 172.21.15.159:0/1559910262 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c40c7f60 con 0x7f84c40cf530 2024-04-04T06:18:03.537 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.542+0000 7f84c3e00640 1 -- 172.21.15.159:0/1559910262 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c40c8520 con 0x7f84c40cf530 2024-04-04T06:18:03.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.543+0000 7f84c3e00640 1 -- 172.21.15.159:0/1559910262 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84c415d190 con 0x7f84c40cf530 2024-04-04T06:18:03.538 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.544+0000 7f84baa00640 1 -- 172.21.15.159:0/1559910262 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84b4014ba0 con 0x7f84c40cf530 2024-04-04T06:18:03.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.545+0000 7f84baa00640 1 --2- 172.21.15.159:0/1559910262 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f84a006c8f0 0x7f84a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:03.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.545+0000 7f84baa00640 1 -- 172.21.15.159:0/1559910262 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f84b408c110 con 0x7f84c40cf530 2024-04-04T06:18:03.539 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.546+0000 7f84c2000640 1 --2- 172.21.15.159:0/1559910262 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f84a006c8f0 0x7f84a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:03.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.547+0000 7f84c2000640 1 --2- 172.21.15.159:0/1559910262 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f84a006c8f0 0x7f84a006edb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f84ac036000 tx=0x7f84ac03b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:03.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.549+0000 7f84baa00640 1 -- 172.21.15.159:0/1559910262 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f84c415d190 con 0x7f84c40cf530 2024-04-04T06:18:03.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.752+0000 7f84c3e00640 1 -- 172.21.15.159:0/1559910262 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f84c4075940 con 0x7f84a006c8f0 2024-04-04T06:18:03.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.753+0000 7f84baa00640 1 -- 172.21.15.159:0/1559910262 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f84c4075940 con 0x7f84a006c8f0 2024-04-04T06:18:03.747 INFO:teuthology.orchestra.run.smithi159.stderr:instructed osd(s) 1 to deep-scrub 2024-04-04T06:18:03.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.755+0000 7f84c3e00640 1 -- 172.21.15.159:0/1559910262 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f84a006c8f0 msgr2=0x7f84a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:03.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.755+0000 7f84c3e00640 1 --2- 172.21.15.159:0/1559910262 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f84a006c8f0 0x7f84a006edb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f84ac036000 tx=0x7f84ac03b040 comp rx=0 tx=0).stop 2024-04-04T06:18:03.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.755+0000 7f84c3e00640 1 -- 172.21.15.159:0/1559910262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c40cf530 msgr2=0x7f84c40c9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:03.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.755+0000 7f84c3e00640 1 --2- 172.21.15.159:0/1559910262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c40cf530 0x7f84c40c9910 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f84b4007e40 tx=0x7f84b400e4b0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.756+0000 7f84c3e00640 1 -- 172.21.15.159:0/1559910262 shutdown_connections 2024-04-04T06:18:03.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.756+0000 7f84c3e00640 1 --2- 172.21.15.159:0/1559910262 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f84a006c8f0 0x7f84a006edb0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.756+0000 7f84c3e00640 1 --2- 172.21.15.159:0/1559910262 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84c415e4a0 0x7f84c40b9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.756+0000 7f84c3e00640 1 --2- 172.21.15.159:0/1559910262 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84c40d2310 0x7f84c40b8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.756+0000 7f84c3e00640 1 --2- 172.21.15.159:0/1559910262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c40cf530 0x7f84c40c9910 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.756+0000 7f84c3e00640 1 -- 172.21.15.159:0/1559910262 >> 172.21.15.159:0/1559910262 conn(0x7f84c40676d0 msgr2=0x7f84c4067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:03.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.756+0000 7f84c3e00640 1 -- 172.21.15.159:0/1559910262 shutdown_connections 2024-04-04T06:18:03.751 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.756+0000 7f84c3e00640 1 -- 172.21.15.159:0/1559910262 wait complete. 2024-04-04T06:18:03.769 INFO:tasks.ceph:Scrubbing osd.2 2024-04-04T06:18:03.769 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-04T06:18:03.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.993+0000 7fd6a8a00640 1 Processor -- start 2024-04-04T06:18:03.987 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.993+0000 7fd6a8a00640 1 -- start start 2024-04-04T06:18:03.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.994+0000 7fd6a8a00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd6a40acf10 0x7fd6a409dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:03.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.994+0000 7fd6a8a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6a4071710 0x7fd6a4071b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:03.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.994+0000 7fd6a8a00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6a415cb90 con 0x7fd6a40ad2f0 2024-04-04T06:18:03.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.994+0000 7fd6a8a00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fd6a4145f60 con 0x7fd6a40acf10 2024-04-04T06:18:03.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.994+0000 7fd6a8a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a415c950 con 0x7fd6a4071710 2024-04-04T06:18:03.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.995+0000 7fd69be00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fd6a40ad2f0 0x7fd6a41453f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52016/0 (socket says 172.21.15.159:52016) 2024-04-04T06:18:03.989 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.995+0000 7fd69be00640 1 -- 172.21.15.159:0/2289141037 learned_addr learned my addr 172.21.15.159:0/2289141037 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:18:03.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.995+0000 7fd6a2800640 1 --2- 172.21.15.159:0/2289141037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6a4071710 0x7fd6a4071b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:03.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.995+0000 7fd6a1e00640 1 --2- 172.21.15.159:0/2289141037 >> v2:172.21.15.178:3300/0 conn(0x7fd6a40acf10 0x7fd6a409dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:03.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.995+0000 7fd6a1400640 1 -- 172.21.15.159:0/2289141037 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3525121923 0 0) 0x7fd6a415cb90 con 0x7fd6a40ad2f0 2024-04-04T06:18:03.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.995+0000 7fd6a1400640 1 -- 172.21.15.159:0/2289141037 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd688003520 con 0x7fd6a40ad2f0 2024-04-04T06:18:03.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.996+0000 7fd6a1400640 1 -- 172.21.15.159:0/2289141037 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3658163179 0 0) 0x7fd688003520 con 0x7fd6a40ad2f0 2024-04-04T06:18:03.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.996+0000 7fd6a1400640 1 -- 172.21.15.159:0/2289141037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6a4071710 msgr2=0x7fd6a4071b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:03.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.996+0000 7fd6a1400640 1 --2- 172.21.15.159:0/2289141037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6a4071710 0x7fd6a4071b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.996+0000 7fd6a1400640 1 -- 172.21.15.159:0/2289141037 >> v2:172.21.15.178:3300/0 conn(0x7fd6a40acf10 msgr2=0x7fd6a409dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:03.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.996+0000 7fd6a1400640 1 --2- 172.21.15.159:0/2289141037 >> v2:172.21.15.178:3300/0 conn(0x7fd6a40acf10 0x7fd6a409dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.996+0000 7fd6a1400640 1 -- 172.21.15.159:0/2289141037 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a415ce70 con 0x7fd6a40ad2f0 2024-04-04T06:18:03.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.996+0000 7fd6a1400640 1 -- 172.21.15.159:0/2289141037 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7fd68c001670 con 0x7fd6a40ad2f0 2024-04-04T06:18:03.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.996+0000 7fd6a1400640 1 -- 172.21.15.159:0/2289141037 >> v1:172.21.15.159:6789/0 conn(0x7fd6a40ad2f0 legacy=0x7fd6a41453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:03.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.996+0000 7fd6a1400640 1 --2- 172.21.15.159:0/2289141037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6880039b0 0x7fd688003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:03.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.996+0000 7fd6a1400640 1 --2- 172.21.15.159:0/2289141037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd688004800 0x7fd688004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:03.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.996+0000 7fd6a1400640 1 --2- 172.21.15.159:0/2289141037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6a4071710 0x7fd6880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:03.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.996+0000 7fd6a1400640 1 -- 172.21.15.159:0/2289141037 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a4145f60 con 0x7fd688004800 2024-04-04T06:18:03.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.996+0000 7fd6a1400640 1 -- 172.21.15.159:0/2289141037 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a415c950 con 0x7fd6880039b0 2024-04-04T06:18:03.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.996+0000 7fd6a1400640 1 -- 172.21.15.159:0/2289141037 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd68800b9c0 con 0x7fd6a4071710 2024-04-04T06:18:03.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.996+0000 7fd69be00640 1 --2- 172.21.15.159:0/2289141037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd688004800 0x7fd688004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:03.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.997+0000 7fd69be00640 1 -- 172.21.15.159:0/2289141037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6a4071710 msgr2=0x7fd6880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:03.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.997+0000 7fd69be00640 1 --2- 172.21.15.159:0/2289141037 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd6a4071710 0x7fd6880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.997+0000 7fd69be00640 1 -- 172.21.15.159:0/2289141037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6880039b0 msgr2=0x7fd688003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:03.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.997+0000 7fd69be00640 1 --2- 172.21.15.159:0/2289141037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6880039b0 0x7fd688003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.997+0000 7fd69be00640 1 -- 172.21.15.159:0/2289141037 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd688003520 con 0x7fd688004800 2024-04-04T06:18:03.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.998+0000 7fd69be00640 1 --2- 172.21.15.159:0/2289141037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd688004800 0x7fd688004c80 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00c210 tx=0x7fd688005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f809b0dea876b33 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:03.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.998+0000 7fd6a1400640 1 -- 172.21.15.159:0/2289141037 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c007d70 con 0x7fd688004800 2024-04-04T06:18:03.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.998+0000 7fd6a1400640 1 -- 172.21.15.159:0/2289141037 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd68c030070 con 0x7fd688004800 2024-04-04T06:18:03.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.998+0000 7fd6a1400640 1 -- 172.21.15.159:0/2289141037 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c02cd60 con 0x7fd688004800 2024-04-04T06:18:03.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.998+0000 7fd6a8a00640 1 -- 172.21.15.159:0/2289141037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd688004800 msgr2=0x7fd688004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:03.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.999+0000 7fd6a8a00640 1 --2- 172.21.15.159:0/2289141037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd688004800 0x7fd688004c80 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00c210 tx=0x7fd688005830 comp rx=0 tx=0).stop 2024-04-04T06:18:03.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.999+0000 7fd6a2800640 1 -- 172.21.15.159:0/2289141037 reap_dead start 2024-04-04T06:18:03.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.999+0000 7fd6a8a00640 1 -- 172.21.15.159:0/2289141037 shutdown_connections 2024-04-04T06:18:03.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.999+0000 7fd6a8a00640 1 -- 172.21.15.159:0/2289141037 >> 172.21.15.159:0/2289141037 conn(0x7fd6a406bba0 msgr2=0x7fd6a40b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:03.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.999+0000 7fd6a8a00640 1 -- 172.21.15.159:0/2289141037 shutdown_connections 2024-04-04T06:18:03.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:03.999+0000 7fd6a8a00640 1 -- 172.21.15.159:0/2289141037 wait complete. 2024-04-04T06:18:03.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.000+0000 7fd6a8a00640 1 Processor -- start 2024-04-04T06:18:03.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.001+0000 7fd6a8a00640 1 -- start start 2024-04-04T06:18:03.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.002+0000 7fd6a8a00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd688004800 0x7fd6a413b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:03.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.002+0000 7fd6a8a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6a413c870 0x7fd6a413ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:03.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.002+0000 7fd6a8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6a40b0b50 0x7fd6a40b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:03.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.002+0000 7fd6a8a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a415d430 con 0x7fd6a40b0b50 2024-04-04T06:18:03.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.002+0000 7fd6a8a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a415d7a0 con 0x7fd6a413c870 2024-04-04T06:18:03.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.002+0000 7fd6a8a00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a40ad7a0 con 0x7fd688004800 2024-04-04T06:18:03.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.002+0000 7fd6a2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6a40b0b50 0x7fd6a40b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:03.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.002+0000 7fd6a2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6a40b0b50 0x7fd6a40b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35424/0 (socket says 172.21.15.159:35424) 2024-04-04T06:18:03.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.002+0000 7fd6a2800640 1 -- 172.21.15.159:0/2209771426 learned_addr learned my addr 172.21.15.159:0/2209771426 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:03.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.002+0000 7fd69be00640 1 --2- 172.21.15.159:0/2209771426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6a413c870 0x7fd6a413ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:03.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.002+0000 7fd6a1e00640 1 --2- 172.21.15.159:0/2209771426 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd688004800 0x7fd6a413b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:03.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.002+0000 7fd6a2800640 1 -- 172.21.15.159:0/2209771426 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd688004800 msgr2=0x7fd6a413b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:03.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.002+0000 7fd6a2800640 1 --2- 172.21.15.159:0/2209771426 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd688004800 0x7fd6a413b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.002+0000 7fd6a2800640 1 -- 172.21.15.159:0/2209771426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6a413c870 msgr2=0x7fd6a413ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:03.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.002+0000 7fd6a2800640 1 --2- 172.21.15.159:0/2209771426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6a413c870 0x7fd6a413ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:03.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.002+0000 7fd6a2800640 1 -- 172.21.15.159:0/2209771426 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a41f45c0 con 0x7fd6a40b0b50 2024-04-04T06:18:03.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.002+0000 7fd69be00640 1 --2- 172.21.15.159:0/2209771426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd6a413c870 0x7fd6a413ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:18:03.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.003+0000 7fd6a2800640 1 --2- 172.21.15.159:0/2209771426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6a40b0b50 0x7fd6a40b0fd0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40cfaf0 tx=0x7fd69c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:03.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.003+0000 7fd69a000640 1 -- 172.21.15.159:0/2209771426 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c00ba80 con 0x7fd6a40b0b50 2024-04-04T06:18:03.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.003+0000 7fd69a000640 1 -- 172.21.15.159:0/2209771426 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd69c01a070 con 0x7fd6a40b0b50 2024-04-04T06:18:03.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.003+0000 7fd69a000640 1 -- 172.21.15.159:0/2209771426 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c00be40 con 0x7fd6a40b0b50 2024-04-04T06:18:03.998 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.003+0000 7fd6a8a00640 1 -- 172.21.15.159:0/2209771426 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a40b1880 con 0x7fd6a40b0b50 2024-04-04T06:18:03.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.003+0000 7fd6a8a00640 1 -- 172.21.15.159:0/2209771426 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a415edd0 con 0x7fd6a40b0b50 2024-04-04T06:18:03.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.004+0000 7fd6a8a00640 1 -- 172.21.15.159:0/2209771426 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd6a40685c0 con 0x7fd6a40b0b50 2024-04-04T06:18:03.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.005+0000 7fd69a000640 1 -- 172.21.15.159:0/2209771426 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd69c005d10 con 0x7fd6a40b0b50 2024-04-04T06:18:04.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.006+0000 7fd69a000640 1 --2- 172.21.15.159:0/2209771426 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd68006c920 0x7fd68006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.006+0000 7fd69a000640 1 -- 172.21.15.159:0/2209771426 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7fd69c08c350 con 0x7fd6a40b0b50 2024-04-04T06:18:04.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.006+0000 7fd69a000640 1 --2- 172.21.15.159:0/2209771426 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x7fd680072920 0x7fd680074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.006+0000 7fd69a000640 1 -- 172.21.15.159:0/2209771426 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd6800753f0 con 0x7fd680072920 2024-04-04T06:18:04.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.007+0000 7fd69a000640 1 -- 172.21.15.159:0/2209771426 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7fd69c08c700 con 0x7fd6a40b0b50 2024-04-04T06:18:04.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.007+0000 7fd69be00640 1 --2- 172.21.15.159:0/2209771426 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x7fd680072920 0x7fd680074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:04.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.007+0000 7fd6a1e00640 1 --2- 172.21.15.159:0/2209771426 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd68006c920 0x7fd68006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:04.001 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.007+0000 7fd69be00640 1 --2- 172.21.15.159:0/2209771426 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x7fd680072920 0x7fd680074d40 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:04.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.008+0000 7fd6a1e00640 1 --2- 172.21.15.159:0/2209771426 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd68006c920 0x7fd68006ede0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd694004770 tx=0x7fd694008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:04.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.008+0000 7fd69a000640 1 -- 172.21.15.159:0/2209771426 <== osd.2 v2:172.21.15.159:6808/2619731047 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd6800753f0 con 0x7fd680072920 2024-04-04T06:18:04.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.023+0000 7fd6a8a00640 1 -- 172.21.15.159:0/2209771426 --> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd6a40b2900 con 0x7fd680072920 2024-04-04T06:18:04.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.023+0000 7fd69a000640 1 -- 172.21.15.159:0/2209771426 <== osd.2 v2:172.21.15.159:6808/2619731047 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd6a40b2900 con 0x7fd680072920 2024-04-04T06:18:04.017 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:18:04.017 INFO:teuthology.orchestra.run.smithi159.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-04T06:18:04.018 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:18:04.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.023+0000 7fd6a8a00640 1 -- 172.21.15.159:0/2209771426 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x7fd680072920 msgr2=0x7fd680074d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:04.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.023+0000 7fd6a8a00640 1 --2- 172.21.15.159:0/2209771426 >> [v2:172.21.15.159:6808/2619731047,v1:172.21.15.159:6809/2619731047] conn(0x7fd680072920 0x7fd680074d40 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.024+0000 7fd6a8a00640 1 -- 172.21.15.159:0/2209771426 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd68006c920 msgr2=0x7fd68006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:04.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.024+0000 7fd6a8a00640 1 --2- 172.21.15.159:0/2209771426 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fd68006c920 0x7fd68006ede0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd694004770 tx=0x7fd694008040 comp rx=0 tx=0).stop 2024-04-04T06:18:04.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.024+0000 7fd6a8a00640 1 -- 172.21.15.159:0/2209771426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6a40b0b50 msgr2=0x7fd6a40b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:04.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.024+0000 7fd6a8a00640 1 --2- 172.21.15.159:0/2209771426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6a40b0b50 0x7fd6a40b0fd0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40cfaf0 tx=0x7fd69c00b480 comp rx=0 tx=0).stop 2024-04-04T06:18:04.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.024+0000 7fd6a2800640 1 -- 172.21.15.159:0/2209771426 reap_dead start 2024-04-04T06:18:04.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.024+0000 7fd6a8a00640 1 -- 172.21.15.159:0/2209771426 shutdown_connections 2024-04-04T06:18:04.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.024+0000 7fd6a8a00640 1 -- 172.21.15.159:0/2209771426 >> 172.21.15.159:0/2209771426 conn(0x7fd6a406bba0 msgr2=0x7fd6a4071af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:04.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.024+0000 7fd6a8a00640 1 -- 172.21.15.159:0/2209771426 shutdown_connections 2024-04-04T06:18:04.019 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.024+0000 7fd6a8a00640 1 -- 172.21.15.159:0/2209771426 wait complete. 2024-04-04T06:18:04.036 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-04T06:18:04.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.251+0000 7f8681e00640 1 Processor -- start 2024-04-04T06:18:04.246 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.251+0000 7f8681e00640 1 -- start start 2024-04-04T06:18:04.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.253+0000 7f8681e00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f867c0d1bf0 0x7f867c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.253+0000 7f8681e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f867c070170 0x7f867c06b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.247 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.253+0000 7f8681e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f867c15edc0 con 0x7f867c0705f0 2024-04-04T06:18:04.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.253+0000 7f8681e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f867c0d1fd0 con 0x7f867c070170 2024-04-04T06:18:04.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.253+0000 7f8681e00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f867c15eb80 con 0x7f867c0d1bf0 2024-04-04T06:18:04.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.253+0000 7f8672000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f867c070170 0x7f867c06b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:04.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.253+0000 7f8672000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f867c070170 0x7f867c06b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35428/0 (socket says 172.21.15.159:35428) 2024-04-04T06:18:04.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.253+0000 7f8672000640 1 -- 172.21.15.159:0/2555705933 learned_addr learned my addr 172.21.15.159:0/2555705933 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:04.248 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.254+0000 7f867aa00640 1 --2- 172.21.15.159:0/2555705933 >> v2:172.21.15.178:3300/0 conn(0x7f867c0d1bf0 0x7f867c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:04.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.254+0000 7f8672000640 1 -- 172.21.15.159:0/2555705933 >> v2:172.21.15.178:3300/0 conn(0x7f867c0d1bf0 msgr2=0x7f867c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:04.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.254+0000 7f8672000640 1 --2- 172.21.15.159:0/2555705933 >> v2:172.21.15.178:3300/0 conn(0x7f867c0d1bf0 0x7f867c15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.254+0000 7f8672000640 1 -- 172.21.15.159:0/2555705933 >> v1:172.21.15.159:6789/0 conn(0x7f867c0705f0 legacy=0x7f867c06e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:04.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.254+0000 7f8672000640 1 -- 172.21.15.159:0/2555705933 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867c0b5e60 con 0x7f867c070170 2024-04-04T06:18:04.249 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.254+0000 7f867aa00640 1 --2- 172.21.15.159:0/2555705933 >> v2:172.21.15.178:3300/0 conn(0x7f867c0d1bf0 0x7f867c15f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:18:04.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.255+0000 7f8672000640 1 --2- 172.21.15.159:0/2555705933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f867c070170 0x7f867c06b570 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f867c0b3d50 tx=0x7f866402b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650976990c6e431c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:04.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.255+0000 7f867a000640 1 -- 172.21.15.159:0/2555705933 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664034070 con 0x7f867c070170 2024-04-04T06:18:04.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.255+0000 7f867a000640 1 -- 172.21.15.159:0/2555705933 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8664030870 con 0x7f867c070170 2024-04-04T06:18:04.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.255+0000 7f867a000640 1 -- 172.21.15.159:0/2555705933 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664030c40 con 0x7f867c070170 2024-04-04T06:18:04.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.256+0000 7f8681e00640 1 -- 172.21.15.159:0/2555705933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f867c070170 msgr2=0x7f867c06b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:04.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.256+0000 7f8681e00640 1 --2- 172.21.15.159:0/2555705933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f867c070170 0x7f867c06b570 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f867c0b3d50 tx=0x7f866402b9b0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.256+0000 7f8681e00640 1 -- 172.21.15.159:0/2555705933 shutdown_connections 2024-04-04T06:18:04.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.256+0000 7f8681e00640 1 --2- 172.21.15.159:0/2555705933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f867c070170 0x7f867c06b570 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.256+0000 7f8681e00640 1 --2- 172.21.15.159:0/2555705933 >> v2:172.21.15.178:3300/0 conn(0x7f867c0d1bf0 0x7f867c15f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.256+0000 7f8681e00640 1 -- 172.21.15.159:0/2555705933 >> 172.21.15.159:0/2555705933 conn(0x7f867c0b6560 msgr2=0x7f867c0b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:04.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.256+0000 7f8681e00640 1 -- 172.21.15.159:0/2555705933 shutdown_connections 2024-04-04T06:18:04.251 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.256+0000 7f8681e00640 1 -- 172.21.15.159:0/2555705933 wait complete. 2024-04-04T06:18:04.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.258+0000 7f8681e00640 1 Processor -- start 2024-04-04T06:18:04.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.258+0000 7f8681e00640 1 -- start start 2024-04-04T06:18:04.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.259+0000 7f8681e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f867c070170 0x7f867c09b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.259+0000 7f8681e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f867c0705f0 0x7f867c09db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.259+0000 7f8681e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f867c0d1bf0 0x7f867c0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.259+0000 7f8681e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f867c15fae0 con 0x7f867c070170 2024-04-04T06:18:04.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.259+0000 7f8681e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f867c15f080 con 0x7f867c0d1bf0 2024-04-04T06:18:04.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.259+0000 7f8681e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f867c0b5b10 con 0x7f867c0705f0 2024-04-04T06:18:04.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.259+0000 7f867aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f867c070170 0x7f867c09b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:04.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.259+0000 7f867aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f867c070170 0x7f867c09b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35444/0 (socket says 172.21.15.159:35444) 2024-04-04T06:18:04.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.259+0000 7f867aa00640 1 -- 172.21.15.159:0/2924332630 learned_addr learned my addr 172.21.15.159:0/2924332630 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:04.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.260+0000 7f867aa00640 1 -- 172.21.15.159:0/2924332630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f867c0705f0 msgr2=0x7f867c09db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:04.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.260+0000 7f8672000640 1 --2- 172.21.15.159:0/2924332630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f867c0705f0 0x7f867c09db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:04.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.260+0000 7f867b400640 1 --2- 172.21.15.159:0/2924332630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f867c0d1bf0 0x7f867c0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:04.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.260+0000 7f867aa00640 1 --2- 172.21.15.159:0/2924332630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f867c0705f0 0x7f867c09db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.260+0000 7f867aa00640 1 -- 172.21.15.159:0/2924332630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f867c0d1bf0 msgr2=0x7f867c0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:04.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.260+0000 7f867aa00640 1 --2- 172.21.15.159:0/2924332630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f867c0d1bf0 0x7f867c0b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.260+0000 7f867aa00640 1 -- 172.21.15.159:0/2924332630 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867c09e0a0 con 0x7f867c070170 2024-04-04T06:18:04.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.260+0000 7f867aa00640 1 --2- 172.21.15.159:0/2924332630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f867c070170 0x7f867c09b2c0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f866c00c7f0 tx=0x7f866c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:04.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.260+0000 7f8673e00640 1 -- 172.21.15.159:0/2924332630 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c015070 con 0x7f867c070170 2024-04-04T06:18:04.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.260+0000 7f8673e00640 1 -- 172.21.15.159:0/2924332630 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f866c007820 con 0x7f867c070170 2024-04-04T06:18:04.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.260+0000 7f8673e00640 1 -- 172.21.15.159:0/2924332630 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c007bd0 con 0x7f867c070170 2024-04-04T06:18:04.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.260+0000 7f8681e00640 1 -- 172.21.15.159:0/2924332630 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f867c09e360 con 0x7f867c070170 2024-04-04T06:18:04.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.261+0000 7f8681e00640 1 -- 172.21.15.159:0/2924332630 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f867c09e920 con 0x7f867c070170 2024-04-04T06:18:04.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.262+0000 7f8681e00640 1 -- 172.21.15.159:0/2924332630 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f867c0685c0 con 0x7f867c070170 2024-04-04T06:18:04.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.263+0000 7f8673e00640 1 -- 172.21.15.159:0/2924332630 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f866c018070 con 0x7f867c070170 2024-04-04T06:18:04.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.263+0000 7f8673e00640 1 --2- 172.21.15.159:0/2924332630 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f865806c8f0 0x7f865806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.264+0000 7f8673e00640 1 -- 172.21.15.159:0/2924332630 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f866c08b980 con 0x7f867c070170 2024-04-04T06:18:04.258 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.264+0000 7f8672000640 1 --2- 172.21.15.159:0/2924332630 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f865806c8f0 0x7f865806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:04.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.265+0000 7f8672000640 1 --2- 172.21.15.159:0/2924332630 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f865806c8f0 0x7f865806edb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f8664036000 tx=0x7f866403b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:04.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.268+0000 7f8673e00640 1 -- 172.21.15.159:0/2924332630 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f866c05a090 con 0x7f867c070170 2024-04-04T06:18:04.459 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.465+0000 7f8681e00640 1 -- 172.21.15.159:0/2924332630 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f867c06f8f0 con 0x7f865806c8f0 2024-04-04T06:18:04.460 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.466+0000 7f8673e00640 1 -- 172.21.15.159:0/2924332630 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f867c06f8f0 con 0x7f865806c8f0 2024-04-04T06:18:04.460 INFO:teuthology.orchestra.run.smithi159.stderr:instructed osd(s) 2 to deep-scrub 2024-04-04T06:18:04.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.469+0000 7f8681e00640 1 -- 172.21.15.159:0/2924332630 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f865806c8f0 msgr2=0x7f865806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:04.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.469+0000 7f8681e00640 1 --2- 172.21.15.159:0/2924332630 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f865806c8f0 0x7f865806edb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f8664036000 tx=0x7f866403b040 comp rx=0 tx=0).stop 2024-04-04T06:18:04.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.469+0000 7f8681e00640 1 -- 172.21.15.159:0/2924332630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f867c070170 msgr2=0x7f867c09b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:04.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.469+0000 7f8681e00640 1 --2- 172.21.15.159:0/2924332630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f867c070170 0x7f867c09b2c0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f866c00c7f0 tx=0x7f866c00ccb0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.469+0000 7f8681e00640 1 -- 172.21.15.159:0/2924332630 shutdown_connections 2024-04-04T06:18:04.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.469+0000 7f8681e00640 1 --2- 172.21.15.159:0/2924332630 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f865806c8f0 0x7f865806edb0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.463 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.469+0000 7f8681e00640 1 --2- 172.21.15.159:0/2924332630 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f867c0d1bf0 0x7f867c0b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.469+0000 7f8681e00640 1 --2- 172.21.15.159:0/2924332630 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f867c0705f0 0x7f867c09db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.469+0000 7f8681e00640 1 --2- 172.21.15.159:0/2924332630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f867c070170 0x7f867c09b2c0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.469+0000 7f8681e00640 1 -- 172.21.15.159:0/2924332630 >> 172.21.15.159:0/2924332630 conn(0x7f867c0b6560 msgr2=0x7f867c160ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:04.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.469+0000 7f8681e00640 1 -- 172.21.15.159:0/2924332630 shutdown_connections 2024-04-04T06:18:04.464 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.469+0000 7f8681e00640 1 -- 172.21.15.159:0/2924332630 wait complete. 2024-04-04T06:18:04.481 INFO:tasks.ceph:Scrubbing osd.3 2024-04-04T06:18:04.481 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-04T06:18:04.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.700+0000 7f0b96000640 1 Processor -- start 2024-04-04T06:18:04.694 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.700+0000 7f0b96000640 1 -- start start 2024-04-04T06:18:04.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.701+0000 7f0b96000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b9006de60 0x7f0b900b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.701+0000 7f0b96000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b900b4380 con 0x7f0b900ccf90 2024-04-04T06:18:04.695 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.701+0000 7f0b96000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b900b4660 con 0x7f0b900b2ea0 2024-04-04T06:18:04.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.701+0000 7f0b96000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0b900b4f90 con 0x7f0b9006de60 2024-04-04T06:18:04.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.702+0000 7f0b95600640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f0b900ccf90 0x7f0b900cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52022/0 (socket says 172.21.15.159:52022) 2024-04-04T06:18:04.696 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.702+0000 7f0b95600640 1 -- 172.21.15.159:0/1918832157 learned_addr learned my addr 172.21.15.159:0/1918832157 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:18:04.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.702+0000 7f0b94c00640 1 --2- 172.21.15.159:0/1918832157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b9006de60 0x7f0b900b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:04.697 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.702+0000 7f0b8fe00640 1 -- 172.21.15.159:0/1918832157 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1604395311 0 0) 0x7f0b900b4380 con 0x7f0b900ccf90 2024-04-04T06:18:04.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.702+0000 7f0b8fe00640 1 -- 172.21.15.159:0/1918832157 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b74003520 con 0x7f0b900ccf90 2024-04-04T06:18:04.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.702+0000 7f0b8fe00640 1 -- 172.21.15.159:0/1918832157 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 728944744 0 0) 0x7f0b74003520 con 0x7f0b900ccf90 2024-04-04T06:18:04.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.702+0000 7f0b8fe00640 1 -- 172.21.15.159:0/1918832157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b9006de60 msgr2=0x7f0b900b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:04.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.702+0000 7f0b8fe00640 1 --2- 172.21.15.159:0/1918832157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b9006de60 0x7f0b900b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.702+0000 7f0b8fe00640 1 -- 172.21.15.159:0/1918832157 >> v1:172.21.15.178:6789/0 conn(0x7f0b900b2ea0 legacy=0x7f0b9006e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:04.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.702+0000 7f0b8fe00640 1 -- 172.21.15.159:0/1918832157 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b900b4940 con 0x7f0b900ccf90 2024-04-04T06:18:04.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.702+0000 7f0b8fe00640 1 -- 172.21.15.159:0/1918832157 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7f0b880019f0 con 0x7f0b900ccf90 2024-04-04T06:18:04.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.702+0000 7f0b8fe00640 1 -- 172.21.15.159:0/1918832157 >> v1:172.21.15.159:6789/0 conn(0x7f0b900ccf90 legacy=0x7f0b900cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:04.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.703+0000 7f0b8fe00640 1 --2- 172.21.15.159:0/1918832157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b740039e0 0x7f0b74003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.703+0000 7f0b8fe00640 1 --2- 172.21.15.159:0/1918832157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b74004800 0x7f0b74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.703+0000 7f0b8fe00640 1 --2- 172.21.15.159:0/1918832157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b9006de60 0x7f0b740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.703+0000 7f0b8fe00640 1 -- 172.21.15.159:0/1918832157 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0b900b4f90 con 0x7f0b740039e0 2024-04-04T06:18:04.698 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.703+0000 7f0b8fe00640 1 -- 172.21.15.159:0/1918832157 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7400ba70 con 0x7f0b74004800 2024-04-04T06:18:04.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.703+0000 7f0b8fe00640 1 -- 172.21.15.159:0/1918832157 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7400bc70 con 0x7f0b9006de60 2024-04-04T06:18:04.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.703+0000 7f0b94c00640 1 --2- 172.21.15.159:0/1918832157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b740039e0 0x7f0b74003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:04.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.703+0000 7f0b94c00640 1 -- 172.21.15.159:0/1918832157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b9006de60 msgr2=0x7f0b740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:04.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.703+0000 7f0b94c00640 1 --2- 172.21.15.159:0/1918832157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b9006de60 0x7f0b740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.703+0000 7f0b94c00640 1 -- 172.21.15.159:0/1918832157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b74004800 msgr2=0x7f0b74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:04.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.703+0000 7f0b94c00640 1 --2- 172.21.15.159:0/1918832157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b74004800 0x7f0b74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.699 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.703+0000 7f0b94c00640 1 -- 172.21.15.159:0/1918832157 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b74003520 con 0x7f0b740039e0 2024-04-04T06:18:04.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.704+0000 7f0b94c00640 1 --2- 172.21.15.159:0/1918832157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b740039e0 0x7f0b74003e00 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009a90 tx=0x7f0b74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd7f491b8958292 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:04.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.704+0000 7f0b8fe00640 1 -- 172.21.15.159:0/1918832157 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80039070 con 0x7f0b740039e0 2024-04-04T06:18:04.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.705+0000 7f0b8fe00640 1 -- 172.21.15.159:0/1918832157 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b8002c6b0 con 0x7f0b740039e0 2024-04-04T06:18:04.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.705+0000 7f0b8fe00640 1 -- 172.21.15.159:0/1918832157 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80009690 con 0x7f0b740039e0 2024-04-04T06:18:04.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.705+0000 7f0b96000640 1 -- 172.21.15.159:0/1918832157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b740039e0 msgr2=0x7f0b74003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:04.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.705+0000 7f0b96000640 1 --2- 172.21.15.159:0/1918832157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b740039e0 0x7f0b74003e00 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009a90 tx=0x7f0b74005830 comp rx=0 tx=0).stop 2024-04-04T06:18:04.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.705+0000 7f0b95600640 1 -- 172.21.15.159:0/1918832157 reap_dead start 2024-04-04T06:18:04.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.705+0000 7f0b96000640 1 -- 172.21.15.159:0/1918832157 shutdown_connections 2024-04-04T06:18:04.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.705+0000 7f0b96000640 1 -- 172.21.15.159:0/1918832157 >> 172.21.15.159:0/1918832157 conn(0x7f0b900b1d80 msgr2=0x7f0b900b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:04.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.705+0000 7f0b96000640 1 -- 172.21.15.159:0/1918832157 shutdown_connections 2024-04-04T06:18:04.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.705+0000 7f0b96000640 1 -- 172.21.15.159:0/1918832157 wait complete. 2024-04-04T06:18:04.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.707+0000 7f0b96000640 1 Processor -- start 2024-04-04T06:18:04.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.707+0000 7f0b96000640 1 -- start start 2024-04-04T06:18:04.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.708+0000 7f0b96000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b740039e0 0x7f0b90149e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.708+0000 7f0b96000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b9014c520 0x7f0b90137580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.703 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.708+0000 7f0b96000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b900adc70 0x7f0b900ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.708+0000 7f0b96000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0b90073440 con 0x7f0b900adc70 2024-04-04T06:18:04.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.708+0000 7f0b96000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9015aac0 con 0x7f0b9014c520 2024-04-04T06:18:04.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.708+0000 7f0b96000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f0b900b3d90 con 0x7f0b740039e0 2024-04-04T06:18:04.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.708+0000 7f0b95600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b900adc70 0x7f0b900ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:04.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.708+0000 7f0b95600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b900adc70 0x7f0b900ae0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35472/0 (socket says 172.21.15.159:35472) 2024-04-04T06:18:04.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.708+0000 7f0b95600640 1 -- 172.21.15.159:0/1443371780 learned_addr learned my addr 172.21.15.159:0/1443371780 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:04.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.709+0000 7f0b94c00640 1 --2- 172.21.15.159:0/1443371780 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b740039e0 0x7f0b90149e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:04.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.709+0000 7f0b95600640 1 -- 172.21.15.159:0/1443371780 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b740039e0 msgr2=0x7f0b90149e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:04.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.709+0000 7f0b95600640 1 --2- 172.21.15.159:0/1443371780 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b740039e0 0x7f0b90149e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.709+0000 7f0b95600640 1 -- 172.21.15.159:0/1443371780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b9014c520 msgr2=0x7f0b90137580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:04.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.709+0000 7f0b95600640 1 --2- 172.21.15.159:0/1443371780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b9014c520 0x7f0b90137580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.709+0000 7f0b95600640 1 -- 172.21.15.159:0/1443371780 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b90148390 con 0x7f0b900adc70 2024-04-04T06:18:04.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.709+0000 7f0b95600640 1 --2- 172.21.15.159:0/1443371780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b900adc70 0x7f0b900ae0f0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f0b900b5e10 tx=0x7f0b8800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:04.704 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.709+0000 7f0b8d600640 1 -- 172.21.15.159:0/1443371780 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8800a5b0 con 0x7f0b900adc70 2024-04-04T06:18:04.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.709+0000 7f0b8d600640 1 -- 172.21.15.159:0/1443371780 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b8800a790 con 0x7f0b900adc70 2024-04-04T06:18:04.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.709+0000 7f0b8d600640 1 -- 172.21.15.159:0/1443371780 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8800ab80 con 0x7f0b900adc70 2024-04-04T06:18:04.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.709+0000 7f0b96000640 1 -- 172.21.15.159:0/1443371780 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b90148670 con 0x7f0b900adc70 2024-04-04T06:18:04.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.709+0000 7f0b96000640 1 -- 172.21.15.159:0/1443371780 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b90148c30 con 0x7f0b900adc70 2024-04-04T06:18:04.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.710+0000 7f0b96000640 1 -- 172.21.15.159:0/1443371780 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b90149010 con 0x7f0b900adc70 2024-04-04T06:18:04.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.711+0000 7f0b8d600640 1 -- 172.21.15.159:0/1443371780 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b88011070 con 0x7f0b900adc70 2024-04-04T06:18:04.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.712+0000 7f0b8d600640 1 --2- 172.21.15.159:0/1443371780 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0b6c06c920 0x7f0b6c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.712+0000 7f0b8d600640 1 -- 172.21.15.159:0/1443371780 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f0b8808b050 con 0x7f0b900adc70 2024-04-04T06:18:04.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.713+0000 7f0b8d600640 1 --2- 172.21.15.159:0/1443371780 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x7f0b6c072920 0x7f0b6c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.713+0000 7f0b8d600640 1 -- 172.21.15.159:0/1443371780 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b6c0753f0 con 0x7f0b6c072920 2024-04-04T06:18:04.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.713+0000 7f0b8d600640 1 -- 172.21.15.159:0/1443371780 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7f0b88092050 con 0x7f0b900adc70 2024-04-04T06:18:04.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.713+0000 7f0b87e00640 1 --2- 172.21.15.159:0/1443371780 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x7f0b6c072920 0x7f0b6c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:04.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.713+0000 7f0b94c00640 1 --2- 172.21.15.159:0/1443371780 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0b6c06c920 0x7f0b6c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:04.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.713+0000 7f0b87e00640 1 --2- 172.21.15.159:0/1443371780 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x7f0b6c072920 0x7f0b6c074d40 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:04.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.714+0000 7f0b94c00640 1 --2- 172.21.15.159:0/1443371780 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0b6c06c920 0x7f0b6c06ede0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0b80036000 tx=0x7f0b80030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:04.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.715+0000 7f0b8d600640 1 -- 172.21.15.159:0/1443371780 <== osd.3 v2:172.21.15.159:6820/2070760205 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0b6c0753f0 con 0x7f0b6c072920 2024-04-04T06:18:04.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.729+0000 7f0b96000640 1 -- 172.21.15.159:0/1443371780 --> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0b90075940 con 0x7f0b6c072920 2024-04-04T06:18:04.723 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.730+0000 7f0b8d600640 1 -- 172.21.15.159:0/1443371780 <== osd.3 v2:172.21.15.159:6820/2070760205 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0b90075940 con 0x7f0b6c072920 2024-04-04T06:18:04.724 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:18:04.724 INFO:teuthology.orchestra.run.smithi159.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-04T06:18:04.724 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:18:04.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.730+0000 7f0b96000640 1 -- 172.21.15.159:0/1443371780 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x7f0b6c072920 msgr2=0x7f0b6c074d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:04.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.730+0000 7f0b96000640 1 --2- 172.21.15.159:0/1443371780 >> [v2:172.21.15.159:6820/2070760205,v1:172.21.15.159:6821/2070760205] conn(0x7f0b6c072920 0x7f0b6c074d40 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.730+0000 7f0b96000640 1 -- 172.21.15.159:0/1443371780 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0b6c06c920 msgr2=0x7f0b6c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:04.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.730+0000 7f0b96000640 1 --2- 172.21.15.159:0/1443371780 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f0b6c06c920 0x7f0b6c06ede0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0b80036000 tx=0x7f0b80030000 comp rx=0 tx=0).stop 2024-04-04T06:18:04.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.730+0000 7f0b96000640 1 -- 172.21.15.159:0/1443371780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b900adc70 msgr2=0x7f0b900ae0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:04.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.730+0000 7f0b96000640 1 --2- 172.21.15.159:0/1443371780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b900adc70 0x7f0b900ae0f0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f0b900b5e10 tx=0x7f0b8800cf00 comp rx=0 tx=0).stop 2024-04-04T06:18:04.725 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.730+0000 7f0b95600640 1 -- 172.21.15.159:0/1443371780 reap_dead start 2024-04-04T06:18:04.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.731+0000 7f0b96000640 1 -- 172.21.15.159:0/1443371780 shutdown_connections 2024-04-04T06:18:04.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.731+0000 7f0b96000640 1 -- 172.21.15.159:0/1443371780 >> 172.21.15.159:0/1443371780 conn(0x7f0b900b1d80 msgr2=0x7f0b900afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:04.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.731+0000 7f0b96000640 1 -- 172.21.15.159:0/1443371780 shutdown_connections 2024-04-04T06:18:04.726 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.731+0000 7f0b96000640 1 -- 172.21.15.159:0/1443371780 wait complete. 2024-04-04T06:18:04.742 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-04T06:18:04.950 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.956+0000 7ffa93200640 1 Processor -- start 2024-04-04T06:18:04.951 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.956+0000 7ffa93200640 1 -- start start 2024-04-04T06:18:04.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.958+0000 7ffa93200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa8c141c10 0x7ffa8c071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.958+0000 7ffa93200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ffa8c071eb0 0x7ffa8c162160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.958+0000 7ffa93200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa8c15ec80 con 0x7ffa8c0d0280 2024-04-04T06:18:04.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.958+0000 7ffa93200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7ffa8c15ef20 con 0x7ffa8c071eb0 2024-04-04T06:18:04.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.958+0000 7ffa93200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ffa8c15f200 con 0x7ffa8c141c10 2024-04-04T06:18:04.952 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.958+0000 7ffa8be00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7ffa8c0d0280 0x7ffa8c0d0660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52038/0 (socket says 172.21.15.159:52038) 2024-04-04T06:18:04.953 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.958+0000 7ffa8be00640 1 -- 172.21.15.159:0/2095625492 learned_addr learned my addr 172.21.15.159:0/2095625492 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:18:04.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.959+0000 7ffa8b400640 1 --2- 172.21.15.159:0/2095625492 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa8c141c10 0x7ffa8c071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:04.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.959+0000 7ffa91200640 1 --2- 172.21.15.159:0/2095625492 >> v2:172.21.15.178:3300/0 conn(0x7ffa8c071eb0 0x7ffa8c162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:04.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.959+0000 7ffa8aa00640 1 -- 172.21.15.159:0/2095625492 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2769070604 0 0) 0x7ffa8c15ec80 con 0x7ffa8c0d0280 2024-04-04T06:18:04.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.959+0000 7ffa8aa00640 1 -- 172.21.15.159:0/2095625492 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa70003520 con 0x7ffa8c0d0280 2024-04-04T06:18:04.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.959+0000 7ffa8aa00640 1 -- 172.21.15.159:0/2095625492 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 484311659 0 0) 0x7ffa70003520 con 0x7ffa8c0d0280 2024-04-04T06:18:04.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.959+0000 7ffa8aa00640 1 -- 172.21.15.159:0/2095625492 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa8c141c10 msgr2=0x7ffa8c071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:04.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.959+0000 7ffa8aa00640 1 --2- 172.21.15.159:0/2095625492 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa8c141c10 0x7ffa8c071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.959+0000 7ffa8aa00640 1 -- 172.21.15.159:0/2095625492 >> v2:172.21.15.178:3300/0 conn(0x7ffa8c071eb0 msgr2=0x7ffa8c162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:04.955 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.959+0000 7ffa8aa00640 1 --2- 172.21.15.159:0/2095625492 >> v2:172.21.15.178:3300/0 conn(0x7ffa8c071eb0 0x7ffa8c162160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.959+0000 7ffa8aa00640 1 -- 172.21.15.159:0/2095625492 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8c15f4e0 con 0x7ffa8c0d0280 2024-04-04T06:18:04.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.959+0000 7ffa8aa00640 1 -- 172.21.15.159:0/2095625492 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7ffa840019f0 con 0x7ffa8c0d0280 2024-04-04T06:18:04.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.959+0000 7ffa8aa00640 1 -- 172.21.15.159:0/2095625492 >> v1:172.21.15.159:6789/0 conn(0x7ffa8c0d0280 legacy=0x7ffa8c0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:04.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.959+0000 7ffa8aa00640 1 --2- 172.21.15.159:0/2095625492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa700039b0 0x7ffa70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.959+0000 7ffa8aa00640 1 --2- 172.21.15.159:0/2095625492 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa8c141c10 0x7ffa70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.959+0000 7ffa8aa00640 1 --2- 172.21.15.159:0/2095625492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa70004d40 0x7ffa700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.959+0000 7ffa8aa00640 1 -- 172.21.15.159:0/2095625492 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c15ef20 con 0x7ffa700039b0 2024-04-04T06:18:04.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.959+0000 7ffa8aa00640 1 -- 172.21.15.159:0/2095625492 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c15f200 con 0x7ffa70004d40 2024-04-04T06:18:04.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.959+0000 7ffa8aa00640 1 -- 172.21.15.159:0/2095625492 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ffa7000b9c0 con 0x7ffa8c141c10 2024-04-04T06:18:04.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.960+0000 7ffa8be00640 1 --2- 172.21.15.159:0/2095625492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa700039b0 0x7ffa70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:04.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.960+0000 7ffa8b400640 1 --2- 172.21.15.159:0/2095625492 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa8c141c10 0x7ffa70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:04.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.960+0000 7ffa91200640 1 --2- 172.21.15.159:0/2095625492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa70004d40 0x7ffa700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:04.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.960+0000 7ffa8be00640 1 -- 172.21.15.159:0/2095625492 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa8c141c10 msgr2=0x7ffa70004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:04.956 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.960+0000 7ffa8be00640 1 --2- 172.21.15.159:0/2095625492 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa8c141c10 0x7ffa70004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.960+0000 7ffa8be00640 1 -- 172.21.15.159:0/2095625492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa70004d40 msgr2=0x7ffa700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:04.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.960+0000 7ffa8be00640 1 --2- 172.21.15.159:0/2095625492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa70004d40 0x7ffa700051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.960+0000 7ffa8be00640 1 -- 172.21.15.159:0/2095625492 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa70003520 con 0x7ffa700039b0 2024-04-04T06:18:04.957 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.961+0000 7ffa8be00640 1 --2- 172.21.15.159:0/2095625492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa700039b0 0x7ffa70003dd0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400a920 tx=0x7ffa70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62112979217fd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:04.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.961+0000 7ffa8aa00640 1 -- 172.21.15.159:0/2095625492 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8400a4c0 con 0x7ffa700039b0 2024-04-04T06:18:04.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.961+0000 7ffa8aa00640 1 -- 172.21.15.159:0/2095625492 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa84034650 con 0x7ffa700039b0 2024-04-04T06:18:04.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.961+0000 7ffa8aa00640 1 -- 172.21.15.159:0/2095625492 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa84034a40 con 0x7ffa700039b0 2024-04-04T06:18:04.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.962+0000 7ffa93200640 1 -- 172.21.15.159:0/2095625492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa700039b0 msgr2=0x7ffa70003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:04.958 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.962+0000 7ffa93200640 1 --2- 172.21.15.159:0/2095625492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa700039b0 0x7ffa70003dd0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400a920 tx=0x7ffa70005830 comp rx=0 tx=0).stop 2024-04-04T06:18:04.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.962+0000 7ffa91200640 1 -- 172.21.15.159:0/2095625492 reap_dead start 2024-04-04T06:18:04.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.962+0000 7ffa93200640 1 -- 172.21.15.159:0/2095625492 shutdown_connections 2024-04-04T06:18:04.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.962+0000 7ffa93200640 1 -- 172.21.15.159:0/2095625492 >> 172.21.15.159:0/2095625492 conn(0x7ffa8c06b750 msgr2=0x7ffa8c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:04.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.962+0000 7ffa93200640 1 -- 172.21.15.159:0/2095625492 shutdown_connections 2024-04-04T06:18:04.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.962+0000 7ffa93200640 1 -- 172.21.15.159:0/2095625492 wait complete. 2024-04-04T06:18:04.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.964+0000 7ffa93200640 1 Processor -- start 2024-04-04T06:18:04.959 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.964+0000 7ffa93200640 1 -- start start 2024-04-04T06:18:04.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.965+0000 7ffa93200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa700039b0 0x7ffa8c1578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.965+0000 7ffa93200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa8c1464f0 0x7ffa8c1471c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.965+0000 7ffa93200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa8c0ae6d0 0x7ffa8c0aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.965+0000 7ffa93200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c158e90 con 0x7ffa8c0ae6d0 2024-04-04T06:18:04.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.965+0000 7ffa93200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c06f340 con 0x7ffa700039b0 2024-04-04T06:18:04.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.965+0000 7ffa93200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7ffa8c1f8b60 con 0x7ffa8c1464f0 2024-04-04T06:18:04.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.965+0000 7ffa91200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa8c0ae6d0 0x7ffa8c0aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:04.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.965+0000 7ffa91200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa8c0ae6d0 0x7ffa8c0aeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35490/0 (socket says 172.21.15.159:35490) 2024-04-04T06:18:04.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.965+0000 7ffa91200640 1 -- 172.21.15.159:0/3230482248 learned_addr learned my addr 172.21.15.159:0/3230482248 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:04.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.966+0000 7ffa91200640 1 -- 172.21.15.159:0/3230482248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa8c1464f0 msgr2=0x7ffa8c1471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:04.961 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.966+0000 7ffa91200640 1 --2- 172.21.15.159:0/3230482248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa8c1464f0 0x7ffa8c1471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.966+0000 7ffa91200640 1 -- 172.21.15.159:0/3230482248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa700039b0 msgr2=0x7ffa8c1578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:04.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.966+0000 7ffa91200640 1 --2- 172.21.15.159:0/3230482248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa700039b0 0x7ffa8c1578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:04.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.966+0000 7ffa91200640 1 -- 172.21.15.159:0/3230482248 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8c1f7f70 con 0x7ffa8c0ae6d0 2024-04-04T06:18:04.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.966+0000 7ffa91200640 1 --2- 172.21.15.159:0/3230482248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa8c0ae6d0 0x7ffa8c0aeb50 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c010790 tx=0x7ffa7c010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:04.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.966+0000 7ffa83e00640 1 -- 172.21.15.159:0/3230482248 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c00f040 con 0x7ffa8c0ae6d0 2024-04-04T06:18:04.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.966+0000 7ffa83e00640 1 -- 172.21.15.159:0/3230482248 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa7c01c070 con 0x7ffa8c0ae6d0 2024-04-04T06:18:04.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.966+0000 7ffa93200640 1 -- 172.21.15.159:0/3230482248 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8c1f8230 con 0x7ffa8c0ae6d0 2024-04-04T06:18:04.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.966+0000 7ffa83e00640 1 -- 172.21.15.159:0/3230482248 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c00eb20 con 0x7ffa8c0ae6d0 2024-04-04T06:18:04.962 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.966+0000 7ffa93200640 1 -- 172.21.15.159:0/3230482248 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa8c1f87f0 con 0x7ffa8c0ae6d0 2024-04-04T06:18:04.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.967+0000 7ffa93200640 1 -- 172.21.15.159:0/3230482248 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa8c0685c0 con 0x7ffa8c0ae6d0 2024-04-04T06:18:04.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.968+0000 7ffa83e00640 1 -- 172.21.15.159:0/3230482248 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa7c00ecc0 con 0x7ffa8c0ae6d0 2024-04-04T06:18:04.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.969+0000 7ffa83e00640 1 --2- 172.21.15.159:0/3230482248 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffa6806c920 0x7ffa6806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:04.963 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.969+0000 7ffa83e00640 1 -- 172.21.15.159:0/3230482248 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7ffa7c08b150 con 0x7ffa8c0ae6d0 2024-04-04T06:18:04.964 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.970+0000 7ffa8be00640 1 --2- 172.21.15.159:0/3230482248 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffa6806c920 0x7ffa6806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:04.965 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.971+0000 7ffa8be00640 1 --2- 172.21.15.159:0/3230482248 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffa6806c920 0x7ffa6806ede0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ffa84034000 tx=0x7ffa84007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:04.967 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:04.973+0000 7ffa83e00640 1 -- 172.21.15.159:0/3230482248 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7ffa7c025020 con 0x7ffa8c0ae6d0 2024-04-04T06:18:05.169 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.175+0000 7ffa93200640 1 -- 172.21.15.159:0/3230482248 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7ffa8c06f930 con 0x7ffa6806c920 2024-04-04T06:18:05.170 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.176+0000 7ffa83e00640 1 -- 172.21.15.159:0/3230482248 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ffa8c06f930 con 0x7ffa6806c920 2024-04-04T06:18:05.170 INFO:teuthology.orchestra.run.smithi159.stderr:instructed osd(s) 3 to deep-scrub 2024-04-04T06:18:05.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.179+0000 7ffa93200640 1 -- 172.21.15.159:0/3230482248 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffa6806c920 msgr2=0x7ffa6806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:05.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.179+0000 7ffa93200640 1 --2- 172.21.15.159:0/3230482248 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffa6806c920 0x7ffa6806ede0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ffa84034000 tx=0x7ffa84007000 comp rx=0 tx=0).stop 2024-04-04T06:18:05.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.179+0000 7ffa93200640 1 -- 172.21.15.159:0/3230482248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa8c0ae6d0 msgr2=0x7ffa8c0aeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:05.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.179+0000 7ffa93200640 1 --2- 172.21.15.159:0/3230482248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa8c0ae6d0 0x7ffa8c0aeb50 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c010790 tx=0x7ffa7c010c50 comp rx=0 tx=0).stop 2024-04-04T06:18:05.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.179+0000 7ffa93200640 1 -- 172.21.15.159:0/3230482248 shutdown_connections 2024-04-04T06:18:05.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.179+0000 7ffa93200640 1 --2- 172.21.15.159:0/3230482248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa8c0ae6d0 0x7ffa8c0aeb50 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:05.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.179+0000 7ffa93200640 1 --2- 172.21.15.159:0/3230482248 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa8c1464f0 0x7ffa8c1471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:05.174 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.179+0000 7ffa93200640 1 --2- 172.21.15.159:0/3230482248 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7ffa6806c920 0x7ffa6806ede0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:05.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.179+0000 7ffa93200640 1 --2- 172.21.15.159:0/3230482248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa700039b0 0x7ffa8c1578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:05.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.179+0000 7ffa93200640 1 -- 172.21.15.159:0/3230482248 >> 172.21.15.159:0/3230482248 conn(0x7ffa8c06b750 msgr2=0x7ffa8c161440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:05.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.179+0000 7ffa93200640 1 -- 172.21.15.159:0/3230482248 shutdown_connections 2024-04-04T06:18:05.175 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.179+0000 7ffa93200640 1 -- 172.21.15.159:0/3230482248 wait complete. 2024-04-04T06:18:05.193 INFO:tasks.ceph:Scrubbing osd.4 2024-04-04T06:18:05.193 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-04T06:18:05.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.413+0000 7f5155000640 1 Processor -- start 2024-04-04T06:18:05.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.413+0000 7f5155000640 1 -- start start 2024-04-04T06:18:05.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.414+0000 7f5155000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5150072720 0x7f51500b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:05.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.414+0000 7f5155000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5150072b00 0x7f515006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:05.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.414+0000 7f5155000640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f515006ea10 con 0x7f515006db10 2024-04-04T06:18:05.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.414+0000 7f5155000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f51500b3e80 con 0x7f5150072b00 2024-04-04T06:18:05.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.414+0000 7f5155000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f515006ecf0 con 0x7f5150072720 2024-04-04T06:18:05.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.415+0000 7f5145400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5150072b00 0x7f515006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:05.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.415+0000 7f5145400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5150072b00 0x7f515006d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35500/0 (socket says 172.21.15.159:35500) 2024-04-04T06:18:05.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.415+0000 7f514e800640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f515006db10 0x7f51500adfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52042/0 (socket says 172.21.15.159:52042) 2024-04-04T06:18:05.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.415+0000 7f5145400640 1 -- 172.21.15.159:0/3753243712 learned_addr learned my addr 172.21.15.159:0/3753243712 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:05.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.415+0000 7f514de00640 1 --2- 172.21.15.159:0/3753243712 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5150072720 0x7f51500b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:05.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.415+0000 7f5145400640 1 -- 172.21.15.159:0/3753243712 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5150072720 msgr2=0x7f51500b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:05.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.415+0000 7f5145400640 1 --2- 172.21.15.159:0/3753243712 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5150072720 0x7f51500b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:05.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.415+0000 7f5145400640 1 -- 172.21.15.159:0/3753243712 >> v1:172.21.15.159:6789/0 conn(0x7f515006db10 legacy=0x7f51500adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:05.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.415+0000 7f5145400640 1 -- 172.21.15.159:0/3753243712 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515006efd0 con 0x7f5150072b00 2024-04-04T06:18:05.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.415+0000 7f514d400640 1 -- 172.21.15.159:0/3753243712 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1481246926 0 0) 0x7f515006ea10 con 0x7f515006db10 2024-04-04T06:18:05.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.416+0000 7f5145400640 1 --2- 172.21.15.159:0/3753243712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5150072b00 0x7f515006d5d0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f5138009870 tx=0x7f513802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb807fbed4ba760 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:05.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.417+0000 7f514d400640 1 -- 172.21.15.159:0/3753243712 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138034070 con 0x7f5150072b00 2024-04-04T06:18:05.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.417+0000 7f514d400640 1 -- 172.21.15.159:0/3753243712 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5138030840 con 0x7f5150072b00 2024-04-04T06:18:05.411 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.417+0000 7f514d400640 1 -- 172.21.15.159:0/3753243712 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138030c10 con 0x7f5150072b00 2024-04-04T06:18:05.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.417+0000 7f5155000640 1 -- 172.21.15.159:0/3753243712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5150072b00 msgr2=0x7f515006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:05.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.417+0000 7f5155000640 1 --2- 172.21.15.159:0/3753243712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5150072b00 0x7f515006d5d0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f5138009870 tx=0x7f513802b980 comp rx=0 tx=0).stop 2024-04-04T06:18:05.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.417+0000 7f5155000640 1 -- 172.21.15.159:0/3753243712 shutdown_connections 2024-04-04T06:18:05.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.417+0000 7f5155000640 1 --2- 172.21.15.159:0/3753243712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5150072b00 0x7f515006d5d0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:05.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.417+0000 7f5155000640 1 --2- 172.21.15.159:0/3753243712 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5150072720 0x7f51500b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:05.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.417+0000 7f5155000640 1 -- 172.21.15.159:0/3753243712 >> 172.21.15.159:0/3753243712 conn(0x7f515006bba0 msgr2=0x7f51500b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:05.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.417+0000 7f5155000640 1 -- 172.21.15.159:0/3753243712 shutdown_connections 2024-04-04T06:18:05.412 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.417+0000 7f5155000640 1 -- 172.21.15.159:0/3753243712 wait complete. 2024-04-04T06:18:05.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.419+0000 7f5155000640 1 Processor -- start 2024-04-04T06:18:05.413 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.419+0000 7f5155000640 1 -- start start 2024-04-04T06:18:05.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.420+0000 7f5155000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f515006db10 0x7f515014ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:05.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.420+0000 7f5155000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150072720 0x7f515014e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:05.414 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.420+0000 7f5155000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5150072b00 0x7f51500c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:05.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.420+0000 7f5155000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f515006e690 con 0x7f5150072b00 2024-04-04T06:18:05.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.420+0000 7f5155000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f515006f590 con 0x7f5150072720 2024-04-04T06:18:05.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.420+0000 7f5155000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f515015eab0 con 0x7f515006db10 2024-04-04T06:18:05.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.420+0000 7f514e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5150072b00 0x7f51500c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:05.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.420+0000 7f514e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5150072b00 0x7f51500c9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35510/0 (socket says 172.21.15.159:35510) 2024-04-04T06:18:05.415 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.420+0000 7f514e800640 1 -- 172.21.15.159:0/1419832410 learned_addr learned my addr 172.21.15.159:0/1419832410 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:05.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.421+0000 7f514e800640 1 -- 172.21.15.159:0/1419832410 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f515006db10 msgr2=0x7f515014ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:05.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.421+0000 7f514e800640 1 --2- 172.21.15.159:0/1419832410 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f515006db10 0x7f515014ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:05.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.421+0000 7f514e800640 1 -- 172.21.15.159:0/1419832410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150072720 msgr2=0x7f515014e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:05.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.421+0000 7f514e800640 1 --2- 172.21.15.159:0/1419832410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5150072720 0x7f515014e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:05.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.421+0000 7f514e800640 1 -- 172.21.15.159:0/1419832410 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515014ecc0 con 0x7f5150072b00 2024-04-04T06:18:05.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.421+0000 7f514e800640 1 --2- 172.21.15.159:0/1419832410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5150072b00 0x7f51500c9de0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f514800c730 tx=0x7f514800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:05.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.421+0000 7f5146a00640 1 -- 172.21.15.159:0/1419832410 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5148018070 con 0x7f5150072b00 2024-04-04T06:18:05.416 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.421+0000 7f5146a00640 1 -- 172.21.15.159:0/1419832410 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f514800ce40 con 0x7f5150072b00 2024-04-04T06:18:05.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.421+0000 7f5146a00640 1 -- 172.21.15.159:0/1419832410 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514800a860 con 0x7f5150072b00 2024-04-04T06:18:05.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.421+0000 7f5155000640 1 -- 172.21.15.159:0/1419832410 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515014ef60 con 0x7f5150072b00 2024-04-04T06:18:05.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.422+0000 7f5155000640 1 -- 172.21.15.159:0/1419832410 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f515014f520 con 0x7f5150072b00 2024-04-04T06:18:05.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.422+0000 7f5155000640 1 -- 172.21.15.159:0/1419832410 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f51500685c0 con 0x7f5150072b00 2024-04-04T06:18:05.417 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.424+0000 7f5146a00640 1 -- 172.21.15.159:0/1419832410 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f514801b070 con 0x7f5150072b00 2024-04-04T06:18:05.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.424+0000 7f5146a00640 1 --2- 172.21.15.159:0/1419832410 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f512c06c8f0 0x7f512c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:05.418 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.425+0000 7f5146a00640 1 -- 172.21.15.159:0/1419832410 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f514808afd0 con 0x7f5150072b00 2024-04-04T06:18:05.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.425+0000 7f5146a00640 1 --2- 172.21.15.159:0/1419832410 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x7f512c0728f0 0x7f512c074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:05.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.425+0000 7f5146a00640 1 -- 172.21.15.159:0/1419832410 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f512c0753c0 con 0x7f512c0728f0 2024-04-04T06:18:05.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.425+0000 7f5146a00640 1 -- 172.21.15.159:0/1419832410 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7f514808d5f0 con 0x7f5150072b00 2024-04-04T06:18:05.419 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.425+0000 7f514de00640 1 --2- 172.21.15.159:0/1419832410 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f512c06c8f0 0x7f512c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:05.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.426+0000 7f5145400640 1 --2- 172.21.15.159:0/1419832410 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x7f512c0728f0 0x7f512c074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:05.420 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.427+0000 7f514de00640 1 --2- 172.21.15.159:0/1419832410 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f512c06c8f0 0x7f512c06edb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5140006fd0 tx=0x7f5140008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:05.421 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.427+0000 7f5145400640 1 --2- 172.21.15.159:0/1419832410 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x7f512c0728f0 0x7f512c074d10 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:05.427 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.429+0000 7f5146a00640 1 -- 172.21.15.159:0/1419832410 <== osd.4 v2:172.21.15.178:6812/4035991429 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f512c0753c0 con 0x7f512c0728f0 2024-04-04T06:18:05.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.444+0000 7f5155000640 1 -- 172.21.15.159:0/1419832410 --> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f51500cc4d0 con 0x7f512c0728f0 2024-04-04T06:18:05.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.445+0000 7f5146a00640 1 -- 172.21.15.159:0/1419832410 <== osd.4 v2:172.21.15.178:6812/4035991429 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f51500cc4d0 con 0x7f512c0728f0 2024-04-04T06:18:05.439 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:18:05.439 INFO:teuthology.orchestra.run.smithi159.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-04T06:18:05.439 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:18:05.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.445+0000 7f5155000640 1 -- 172.21.15.159:0/1419832410 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x7f512c0728f0 msgr2=0x7f512c074d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:05.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.445+0000 7f5155000640 1 --2- 172.21.15.159:0/1419832410 >> [v2:172.21.15.178:6812/4035991429,v1:172.21.15.178:6814/4035991429] conn(0x7f512c0728f0 0x7f512c074d10 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:05.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.446+0000 7f5155000640 1 -- 172.21.15.159:0/1419832410 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f512c06c8f0 msgr2=0x7f512c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:05.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.446+0000 7f5155000640 1 --2- 172.21.15.159:0/1419832410 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f512c06c8f0 0x7f512c06edb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5140006fd0 tx=0x7f5140008040 comp rx=0 tx=0).stop 2024-04-04T06:18:05.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.446+0000 7f5155000640 1 -- 172.21.15.159:0/1419832410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5150072b00 msgr2=0x7f51500c9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:05.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.446+0000 7f5155000640 1 --2- 172.21.15.159:0/1419832410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5150072b00 0x7f51500c9de0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f514800c730 tx=0x7f514800cbf0 comp rx=0 tx=0).stop 2024-04-04T06:18:05.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.446+0000 7f514e800640 1 -- 172.21.15.159:0/1419832410 reap_dead start 2024-04-04T06:18:05.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.446+0000 7f5155000640 1 -- 172.21.15.159:0/1419832410 shutdown_connections 2024-04-04T06:18:05.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.446+0000 7f5155000640 1 -- 172.21.15.159:0/1419832410 >> 172.21.15.159:0/1419832410 conn(0x7f515006bba0 msgr2=0x7f5150071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:05.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.446+0000 7f5155000640 1 -- 172.21.15.159:0/1419832410 shutdown_connections 2024-04-04T06:18:05.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.446+0000 7f5155000640 1 -- 172.21.15.159:0/1419832410 wait complete. 2024-04-04T06:18:05.458 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-04T06:18:05.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.672+0000 7f51ff200640 1 Processor -- start 2024-04-04T06:18:05.667 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.672+0000 7f51ff200640 1 -- start start 2024-04-04T06:18:05.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.674+0000 7f51ff200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5200147a40 0x7f52000cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:05.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.674+0000 7f51ff200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5200161f30 con 0x7f52000d01c0 2024-04-04T06:18:05.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.674+0000 7f51ff200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52000ce7e0 con 0x7f52000ce380 2024-04-04T06:18:05.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.674+0000 7f51ff200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f5200162b00 con 0x7f5200147a40 2024-04-04T06:18:05.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.674+0000 7f51fde00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5200147a40 0x7f52000cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:05.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.674+0000 7f51fde00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5200147a40 0x7f52000cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35526/0 (socket says 172.21.15.159:35526) 2024-04-04T06:18:05.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.674+0000 7f51fde00640 1 -- 172.21.15.159:0/3821737925 learned_addr learned my addr 172.21.15.159:0/3821737925 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:05.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.674+0000 7f51fd400640 1 --1- 172.21.15.159:0/3821737925 >> v1:172.21.15.159:6789/0 conn(0x7f52000d01c0 0x7f52000cdc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52048/0 (socket says 172.21.15.159:52048) 2024-04-04T06:18:05.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.675+0000 7f51fde00640 1 -- 172.21.15.159:0/3821737925 >> v1:172.21.15.178:6789/0 conn(0x7f52000ce380 legacy=0x7f520006da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:05.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.675+0000 7f51fde00640 1 -- 172.21.15.159:0/3821737925 >> v1:172.21.15.159:6789/0 conn(0x7f52000d01c0 legacy=0x7f52000cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:05.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.675+0000 7f51fde00640 1 -- 172.21.15.159:0/3821737925 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52001621d0 con 0x7f5200147a40 2024-04-04T06:18:05.670 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.675+0000 7f51fca00640 1 -- 172.21.15.159:0/3821737925 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 466888731 0 0) 0x7f5200161f30 con 0x7f52000d01c0 2024-04-04T06:18:05.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.676+0000 7f51fde00640 1 --2- 172.21.15.159:0/3821737925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5200147a40 0x7f52000cfc80 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009870 tx=0x7f51f002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44cf5e43012ed58c server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:05.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.676+0000 7f51fca00640 1 -- 172.21.15.159:0/3821737925 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0034070 con 0x7f5200147a40 2024-04-04T06:18:05.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.676+0000 7f51fca00640 1 -- 172.21.15.159:0/3821737925 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f0030860 con 0x7f5200147a40 2024-04-04T06:18:05.672 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.676+0000 7f51fca00640 1 -- 172.21.15.159:0/3821737925 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0030c30 con 0x7f5200147a40 2024-04-04T06:18:05.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.677+0000 7f51ff200640 1 -- 172.21.15.159:0/3821737925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5200147a40 msgr2=0x7f52000cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:05.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.677+0000 7f51ff200640 1 --2- 172.21.15.159:0/3821737925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5200147a40 0x7f52000cfc80 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009870 tx=0x7f51f002b9a0 comp rx=0 tx=0).stop 2024-04-04T06:18:05.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.677+0000 7f51ff200640 1 -- 172.21.15.159:0/3821737925 shutdown_connections 2024-04-04T06:18:05.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.677+0000 7f51ff200640 1 --2- 172.21.15.159:0/3821737925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5200147a40 0x7f52000cfc80 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:05.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.677+0000 7f51ff200640 1 -- 172.21.15.159:0/3821737925 >> 172.21.15.159:0/3821737925 conn(0x7f52000ccad0 msgr2=0x7f52000cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:05.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.677+0000 7f51ff200640 1 -- 172.21.15.159:0/3821737925 shutdown_connections 2024-04-04T06:18:05.673 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.677+0000 7f51ff200640 1 -- 172.21.15.159:0/3821737925 wait complete. 2024-04-04T06:18:05.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.679+0000 7f51ff200640 1 Processor -- start 2024-04-04T06:18:05.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.679+0000 7f51ff200640 1 -- start start 2024-04-04T06:18:05.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.680+0000 7f51ff200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52000ce380 0x7f5200146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:05.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.680+0000 7f51ff200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52000d01c0 0x7f520013f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:05.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.680+0000 7f51ff200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5200147a40 0x7f520013a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:05.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.680+0000 7f51ff200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f52000d0620 con 0x7f52000d01c0 2024-04-04T06:18:05.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.680+0000 7f51ff200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5200162790 con 0x7f52000ce380 2024-04-04T06:18:05.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.680+0000 7f51ff200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f5200147e20 con 0x7f5200147a40 2024-04-04T06:18:05.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.680+0000 7f51fd400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52000d01c0 0x7f520013f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:05.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.680+0000 7f51fd400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52000d01c0 0x7f520013f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35528/0 (socket says 172.21.15.159:35528) 2024-04-04T06:18:05.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.680+0000 7f51fd400640 1 -- 172.21.15.159:0/1927982406 learned_addr learned my addr 172.21.15.159:0/1927982406 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:05.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.680+0000 7f51fde00640 1 --2- 172.21.15.159:0/1927982406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52000ce380 0x7f5200146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:05.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.681+0000 7f51fe800640 1 --2- 172.21.15.159:0/1927982406 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5200147a40 0x7f520013a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:05.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.681+0000 7f51fd400640 1 -- 172.21.15.159:0/1927982406 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5200147a40 msgr2=0x7f520013a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:05.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.681+0000 7f51fd400640 1 --2- 172.21.15.159:0/1927982406 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5200147a40 0x7f520013a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:05.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.681+0000 7f51fd400640 1 -- 172.21.15.159:0/1927982406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52000ce380 msgr2=0x7f5200146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:05.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.681+0000 7f51fd400640 1 --2- 172.21.15.159:0/1927982406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52000ce380 0x7f5200146bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:05.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.681+0000 7f51fd400640 1 -- 172.21.15.159:0/1927982406 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520013d530 con 0x7f52000d01c0 2024-04-04T06:18:05.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.681+0000 7f51fd400640 1 --2- 172.21.15.159:0/1927982406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52000d01c0 0x7f520013f530 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f51e800c820 tx=0x7f51e800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:05.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.681+0000 7f51f6000640 1 -- 172.21.15.159:0/1927982406 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8017070 con 0x7f52000d01c0 2024-04-04T06:18:05.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.681+0000 7f51f6000640 1 -- 172.21.15.159:0/1927982406 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e800a640 con 0x7f52000d01c0 2024-04-04T06:18:05.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.681+0000 7f51f6000640 1 -- 172.21.15.159:0/1927982406 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e800aa30 con 0x7f52000d01c0 2024-04-04T06:18:05.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.681+0000 7f51ff200640 1 -- 172.21.15.159:0/1927982406 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520013d810 con 0x7f52000d01c0 2024-04-04T06:18:05.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.681+0000 7f51ff200640 1 -- 172.21.15.159:0/1927982406 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52001395d0 con 0x7f52000d01c0 2024-04-04T06:18:05.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.682+0000 7f51ff200640 1 -- 172.21.15.159:0/1927982406 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52001436e0 con 0x7f52000d01c0 2024-04-04T06:18:05.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.683+0000 7f51f6000640 1 -- 172.21.15.159:0/1927982406 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51e801a070 con 0x7f52000d01c0 2024-04-04T06:18:05.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.684+0000 7f51f6000640 1 --2- 172.21.15.159:0/1927982406 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f51dc06c8f0 0x7f51dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:05.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.684+0000 7f51f6000640 1 -- 172.21.15.159:0/1927982406 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f51e808b0d0 con 0x7f52000d01c0 2024-04-04T06:18:05.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.685+0000 7f51fde00640 1 --2- 172.21.15.159:0/1927982406 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f51dc06c8f0 0x7f51dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:05.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.686+0000 7f51fde00640 1 --2- 172.21.15.159:0/1927982406 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f51dc06c8f0 0x7f51dc06edb0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f51f0036000 tx=0x7f51f0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:05.683 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.689+0000 7f51f6000640 1 -- 172.21.15.159:0/1927982406 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f52001436e0 con 0x7f52000d01c0 2024-04-04T06:18:05.884 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.890+0000 7f51ff200640 1 -- 172.21.15.159:0/1927982406 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f5200075940 con 0x7f51dc06c8f0 2024-04-04T06:18:05.886 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.891+0000 7f51f6000640 1 -- 172.21.15.159:0/1927982406 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5200075940 con 0x7f51dc06c8f0 2024-04-04T06:18:05.886 INFO:teuthology.orchestra.run.smithi159.stderr:instructed osd(s) 4 to deep-scrub 2024-04-04T06:18:05.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.894+0000 7f51ff200640 1 -- 172.21.15.159:0/1927982406 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f51dc06c8f0 msgr2=0x7f51dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:05.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.894+0000 7f51ff200640 1 --2- 172.21.15.159:0/1927982406 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f51dc06c8f0 0x7f51dc06edb0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f51f0036000 tx=0x7f51f0008030 comp rx=0 tx=0).stop 2024-04-04T06:18:05.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.894+0000 7f51ff200640 1 -- 172.21.15.159:0/1927982406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52000d01c0 msgr2=0x7f520013f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:05.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.894+0000 7f51ff200640 1 --2- 172.21.15.159:0/1927982406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52000d01c0 0x7f520013f530 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f51e800c820 tx=0x7f51e800cce0 comp rx=0 tx=0).stop 2024-04-04T06:18:05.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.894+0000 7f51ff200640 1 -- 172.21.15.159:0/1927982406 shutdown_connections 2024-04-04T06:18:05.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.894+0000 7f51ff200640 1 --2- 172.21.15.159:0/1927982406 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5200147a40 0x7f520013a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:05.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.894+0000 7f51ff200640 1 --2- 172.21.15.159:0/1927982406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52000d01c0 0x7f520013f530 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:05.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.894+0000 7f51ff200640 1 --2- 172.21.15.159:0/1927982406 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f51dc06c8f0 0x7f51dc06edb0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:05.889 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.894+0000 7f51ff200640 1 --2- 172.21.15.159:0/1927982406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52000ce380 0x7f5200146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:05.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.894+0000 7f51ff200640 1 -- 172.21.15.159:0/1927982406 >> 172.21.15.159:0/1927982406 conn(0x7f52000ccad0 msgr2=0x7f520006dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:05.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.894+0000 7f51ff200640 1 -- 172.21.15.159:0/1927982406 shutdown_connections 2024-04-04T06:18:05.890 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:05.894+0000 7f51ff200640 1 -- 172.21.15.159:0/1927982406 wait complete. 2024-04-04T06:18:05.908 INFO:tasks.ceph:Scrubbing osd.5 2024-04-04T06:18:05.909 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-04T06:18:06.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.128+0000 7f5559000640 1 Processor -- start 2024-04-04T06:18:06.122 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.128+0000 7f5559000640 1 -- start start 2024-04-04T06:18:06.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.129+0000 7f5559000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5554072720 0x7f55540b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:06.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.129+0000 7f5559000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5554072b00 0x7f555406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:06.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.129+0000 7f5559000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f555406ea10 con 0x7f555406db10 2024-04-04T06:18:06.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.129+0000 7f5559000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f55540b3e80 con 0x7f5554072b00 2024-04-04T06:18:06.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.129+0000 7f5559000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f555406ecf0 con 0x7f5554072720 2024-04-04T06:18:06.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.130+0000 7f5549400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5554072b00 0x7f555406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:06.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.130+0000 7f5549400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f5554072b00 0x7f555406d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35542/0 (socket says 172.21.15.159:35542) 2024-04-04T06:18:06.124 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.130+0000 7f5549400640 1 -- 172.21.15.159:0/2269942753 learned_addr learned my addr 172.21.15.159:0/2269942753 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:06.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.130+0000 7f5551e00640 1 --2- 172.21.15.159:0/2269942753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5554072720 0x7f55540b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:06.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.130+0000 7f5549400640 1 -- 172.21.15.159:0/2269942753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5554072720 msgr2=0x7f55540b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:06.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.130+0000 7f5549400640 1 --2- 172.21.15.159:0/2269942753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5554072720 0x7f55540b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.130+0000 7f5549400640 1 -- 172.21.15.159:0/2269942753 >> v1:172.21.15.178:6789/0 conn(0x7f555406db10 legacy=0x7f55540adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:06.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.130+0000 7f5549400640 1 -- 172.21.15.159:0/2269942753 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555406efd0 con 0x7f5554072b00 2024-04-04T06:18:06.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.130+0000 7f5551e00640 1 --2- 172.21.15.159:0/2269942753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5554072720 0x7f55540b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:18:06.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.131+0000 7f5549400640 1 --2- 172.21.15.159:0/2269942753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5554072b00 0x7f555406d5d0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f553c009870 tx=0x7f553c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4ca894e66fbb13 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:06.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.132+0000 7f5551400640 1 -- 172.21.15.159:0/2269942753 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c035020 con 0x7f5554072b00 2024-04-04T06:18:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.132+0000 7f5551400640 1 -- 172.21.15.159:0/2269942753 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f553c031640 con 0x7f5554072b00 2024-04-04T06:18:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.132+0000 7f5551400640 1 -- 172.21.15.159:0/2269942753 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c02d040 con 0x7f5554072b00 2024-04-04T06:18:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.132+0000 7f5559000640 1 -- 172.21.15.159:0/2269942753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5554072b00 msgr2=0x7f555406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.132+0000 7f5559000640 1 --2- 172.21.15.159:0/2269942753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5554072b00 0x7f555406d5d0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f553c009870 tx=0x7f553c02b8f0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.132+0000 7f5559000640 1 -- 172.21.15.159:0/2269942753 shutdown_connections 2024-04-04T06:18:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.132+0000 7f5559000640 1 --2- 172.21.15.159:0/2269942753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5554072b00 0x7f555406d5d0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.132+0000 7f5559000640 1 --2- 172.21.15.159:0/2269942753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5554072720 0x7f55540b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.132+0000 7f5559000640 1 -- 172.21.15.159:0/2269942753 >> 172.21.15.159:0/2269942753 conn(0x7f555406bba0 msgr2=0x7f55540b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.132+0000 7f5559000640 1 -- 172.21.15.159:0/2269942753 shutdown_connections 2024-04-04T06:18:06.127 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.132+0000 7f5559000640 1 -- 172.21.15.159:0/2269942753 wait complete. 2024-04-04T06:18:06.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.134+0000 7f5559000640 1 Processor -- start 2024-04-04T06:18:06.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.134+0000 7f5559000640 1 -- start start 2024-04-04T06:18:06.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.135+0000 7f5559000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f555406db10 0x7f55540cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:06.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.135+0000 7f5559000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5554072720 0x7f55540cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:06.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.135+0000 7f5559000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5554072b00 0x7f55540b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:06.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.135+0000 7f5559000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f555406e690 con 0x7f5554072720 2024-04-04T06:18:06.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.135+0000 7f5559000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f555406f590 con 0x7f555406db10 2024-04-04T06:18:06.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.135+0000 7f5559000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f555415eab0 con 0x7f5554072b00 2024-04-04T06:18:06.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.135+0000 7f5549400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5554072720 0x7f55540cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:06.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.135+0000 7f5549400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5554072720 0x7f55540cca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35556/0 (socket says 172.21.15.159:35556) 2024-04-04T06:18:06.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.135+0000 7f5549400640 1 -- 172.21.15.159:0/3674803645 learned_addr learned my addr 172.21.15.159:0/3674803645 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:06.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.136+0000 7f5551e00640 1 --2- 172.21.15.159:0/3674803645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f555406db10 0x7f55540cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:06.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.136+0000 7f5552800640 1 --2- 172.21.15.159:0/3674803645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5554072b00 0x7f55540b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:06.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.136+0000 7f5549400640 1 -- 172.21.15.159:0/3674803645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5554072b00 msgr2=0x7f55540b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:06.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.136+0000 7f5549400640 1 --2- 172.21.15.159:0/3674803645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5554072b00 0x7f55540b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.136+0000 7f5549400640 1 -- 172.21.15.159:0/3674803645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f555406db10 msgr2=0x7f55540cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:06.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.136+0000 7f5549400640 1 --2- 172.21.15.159:0/3674803645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f555406db10 0x7f55540cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.136+0000 7f5549400640 1 -- 172.21.15.159:0/3674803645 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55540cd330 con 0x7f5554072720 2024-04-04T06:18:06.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.136+0000 7f5552800640 1 --2- 172.21.15.159:0/3674803645 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5554072b00 0x7f55540b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:18:06.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.136+0000 7f5551e00640 1 --2- 172.21.15.159:0/3674803645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f555406db10 0x7f55540cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:18:06.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.136+0000 7f5549400640 1 --2- 172.21.15.159:0/3674803645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5554072720 0x7f55540cca30 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f553c02b930 tx=0x7f553c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:06.132 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.136+0000 7f554aa00640 1 -- 172.21.15.159:0/3674803645 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c040070 con 0x7f5554072720 2024-04-04T06:18:06.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.137+0000 7f554aa00640 1 -- 172.21.15.159:0/3674803645 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f553c039c10 con 0x7f5554072720 2024-04-04T06:18:06.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.137+0000 7f554aa00640 1 -- 172.21.15.159:0/3674803645 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c035040 con 0x7f5554072720 2024-04-04T06:18:06.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.137+0000 7f5559000640 1 -- 172.21.15.159:0/3674803645 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55540cd610 con 0x7f5554072720 2024-04-04T06:18:06.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.137+0000 7f5559000640 1 -- 172.21.15.159:0/3674803645 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55540cdbd0 con 0x7f5554072720 2024-04-04T06:18:06.133 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.138+0000 7f5559000640 1 -- 172.21.15.159:0/3674803645 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f55540685c0 con 0x7f5554072720 2024-04-04T06:18:06.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.139+0000 7f554aa00640 1 -- 172.21.15.159:0/3674803645 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f553c04b060 con 0x7f5554072720 2024-04-04T06:18:06.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.140+0000 7f554aa00640 1 --2- 172.21.15.159:0/3674803645 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f553006c8f0 0x7f553006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:06.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.140+0000 7f554aa00640 1 -- 172.21.15.159:0/3674803645 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f553c0ad530 con 0x7f5554072720 2024-04-04T06:18:06.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.140+0000 7f554aa00640 1 --2- 172.21.15.159:0/3674803645 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x7f55300728f0 0x7f5530074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:06.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.140+0000 7f554aa00640 1 -- 172.21.15.159:0/3674803645 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f55300753c0 con 0x7f55300728f0 2024-04-04T06:18:06.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.140+0000 7f554aa00640 1 -- 172.21.15.159:0/3674803645 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7f553c0b4050 con 0x7f5554072720 2024-04-04T06:18:06.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.140+0000 7f5551e00640 1 --2- 172.21.15.159:0/3674803645 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f553006c8f0 0x7f553006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:06.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.141+0000 7f5552800640 1 --2- 172.21.15.159:0/3674803645 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x7f55300728f0 0x7f5530074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:06.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.142+0000 7f5552800640 1 --2- 172.21.15.159:0/3674803645 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x7f55300728f0 0x7f5530074d10 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:06.135 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.142+0000 7f5551e00640 1 --2- 172.21.15.159:0/3674803645 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f553006c8f0 0x7f553006edb0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5544006fd0 tx=0x7f5544008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:06.137 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.143+0000 7f554aa00640 1 -- 172.21.15.159:0/3674803645 <== osd.5 v2:172.21.15.178:6802/63746765 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f55300753c0 con 0x7f55300728f0 2024-04-04T06:18:06.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.157+0000 7f5559000640 1 -- 172.21.15.159:0/3674803645 --> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f55540cc4d0 con 0x7f55300728f0 2024-04-04T06:18:06.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.158+0000 7f554aa00640 1 -- 172.21.15.159:0/3674803645 <== osd.5 v2:172.21.15.178:6802/63746765 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f55540cc4d0 con 0x7f55300728f0 2024-04-04T06:18:06.153 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:18:06.153 INFO:teuthology.orchestra.run.smithi159.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-04T06:18:06.154 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:18:06.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.159+0000 7f5559000640 1 -- 172.21.15.159:0/3674803645 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x7f55300728f0 msgr2=0x7f5530074d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:06.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.159+0000 7f5559000640 1 --2- 172.21.15.159:0/3674803645 >> [v2:172.21.15.178:6802/63746765,v1:172.21.15.178:6803/63746765] conn(0x7f55300728f0 0x7f5530074d10 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.159+0000 7f5559000640 1 -- 172.21.15.159:0/3674803645 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f553006c8f0 msgr2=0x7f553006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:06.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.159+0000 7f5559000640 1 --2- 172.21.15.159:0/3674803645 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f553006c8f0 0x7f553006edb0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5544006fd0 tx=0x7f5544008040 comp rx=0 tx=0).stop 2024-04-04T06:18:06.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.159+0000 7f5559000640 1 -- 172.21.15.159:0/3674803645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5554072720 msgr2=0x7f55540cca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:06.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.159+0000 7f5559000640 1 --2- 172.21.15.159:0/3674803645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5554072720 0x7f55540cca30 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f553c02b930 tx=0x7f553c0046c0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.159+0000 7f5552800640 1 -- 172.21.15.159:0/3674803645 reap_dead start 2024-04-04T06:18:06.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.159+0000 7f5559000640 1 -- 172.21.15.159:0/3674803645 shutdown_connections 2024-04-04T06:18:06.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.159+0000 7f5559000640 1 -- 172.21.15.159:0/3674803645 >> 172.21.15.159:0/3674803645 conn(0x7f555406bba0 msgr2=0x7f55540b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:06.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.159+0000 7f5559000640 1 -- 172.21.15.159:0/3674803645 shutdown_connections 2024-04-04T06:18:06.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.159+0000 7f5559000640 1 -- 172.21.15.159:0/3674803645 wait complete. 2024-04-04T06:18:06.174 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-04T06:18:06.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.392+0000 7f2e63c00640 1 Processor -- start 2024-04-04T06:18:06.387 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.392+0000 7f2e63c00640 1 -- start start 2024-04-04T06:18:06.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.393+0000 7f2e63c00640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2e5c155550 0x7f2e5c155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:06.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.393+0000 7f2e63c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e5c0d0990 0x7f2e5c0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:06.388 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.393+0000 7f2e63c00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e5c0b40b0 con 0x7f2e5c15f800 2024-04-04T06:18:06.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.393+0000 7f2e63c00640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f2e5c0b4390 con 0x7f2e5c155550 2024-04-04T06:18:06.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.393+0000 7f2e63c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2e5c0b4670 con 0x7f2e5c0d0990 2024-04-04T06:18:06.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.394+0000 7f2e61c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e5c0d0990 0x7f2e5c0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:06.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.394+0000 7f2e61200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2e5c155550 0x7f2e5c155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:06.389 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.394+0000 7f2e61200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2e5c155550 0x7f2e5c155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.159:45028/0 (socket says 172.21.15.159:45028) 2024-04-04T06:18:06.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.394+0000 7f2e61200640 1 -- 172.21.15.159:0/582048818 learned_addr learned my addr 172.21.15.159:0/582048818 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:06.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.394+0000 7f2e53e00640 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f2e5c15f800 0x7f2e5c0d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.159:47608/0 (socket says 172.21.15.159:47608) 2024-04-04T06:18:06.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.395+0000 7f2e61200640 1 -- 172.21.15.159:0/582048818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e5c0d0990 msgr2=0x7f2e5c0aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:06.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.395+0000 7f2e61200640 1 --2- 172.21.15.159:0/582048818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e5c0d0990 0x7f2e5c0aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.390 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.395+0000 7f2e61200640 1 -- 172.21.15.159:0/582048818 >> v1:172.21.15.178:6789/0 conn(0x7f2e5c15f800 legacy=0x7f2e5c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:06.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.395+0000 7f2e61200640 1 -- 172.21.15.159:0/582048818 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5c0b4950 con 0x7f2e5c155550 2024-04-04T06:18:06.391 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.395+0000 7f2e5be00640 1 -- 172.21.15.159:0/582048818 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 150017250 0 0) 0x7f2e5c0b40b0 con 0x7f2e5c15f800 2024-04-04T06:18:06.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.397+0000 7f2e61200640 1 --2- 172.21.15.159:0/582048818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e5c155550 0x7f2e5c155930 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009870 tx=0x7f2e4c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f6de95bd430f114 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:06.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.397+0000 7f2e5be00640 1 -- 172.21.15.159:0/582048818 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c034070 con 0x7f2e5c155550 2024-04-04T06:18:06.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.397+0000 7f2e5be00640 1 -- 172.21.15.159:0/582048818 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e4c030840 con 0x7f2e5c155550 2024-04-04T06:18:06.392 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.397+0000 7f2e5be00640 1 -- 172.21.15.159:0/582048818 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c030c10 con 0x7f2e5c155550 2024-04-04T06:18:06.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.398+0000 7f2e63c00640 1 -- 172.21.15.159:0/582048818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e5c155550 msgr2=0x7f2e5c155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:06.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.398+0000 7f2e63c00640 1 --2- 172.21.15.159:0/582048818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e5c155550 0x7f2e5c155930 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009870 tx=0x7f2e4c02b980 comp rx=0 tx=0).stop 2024-04-04T06:18:06.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.398+0000 7f2e63c00640 1 -- 172.21.15.159:0/582048818 shutdown_connections 2024-04-04T06:18:06.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.398+0000 7f2e63c00640 1 --2- 172.21.15.159:0/582048818 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e5c0d0990 0x7f2e5c0aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.393 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.398+0000 7f2e63c00640 1 --2- 172.21.15.159:0/582048818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e5c155550 0x7f2e5c155930 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.398+0000 7f2e63c00640 1 -- 172.21.15.159:0/582048818 >> 172.21.15.159:0/582048818 conn(0x7f2e5c06b750 msgr2=0x7f2e5c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:06.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.398+0000 7f2e63c00640 1 -- 172.21.15.159:0/582048818 shutdown_connections 2024-04-04T06:18:06.394 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.398+0000 7f2e63c00640 1 -- 172.21.15.159:0/582048818 wait complete. 2024-04-04T06:18:06.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.400+0000 7f2e63c00640 1 Processor -- start 2024-04-04T06:18:06.395 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.401+0000 7f2e63c00640 1 -- start start 2024-04-04T06:18:06.397 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.402+0000 7f2e63c00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e5c0d0990 0x7f2e5c06f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:06.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.402+0000 7f2e63c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e5c155550 0x7f2e5c0cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:06.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.402+0000 7f2e63c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e5c15f800 0x7f2e5c0aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:06.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.402+0000 7f2e63c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c15ba20 con 0x7f2e5c15f800 2024-04-04T06:18:06.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.402+0000 7f2e63c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c15fc80 con 0x7f2e5c155550 2024-04-04T06:18:06.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.402+0000 7f2e63c00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f2e5c15bba0 con 0x7f2e5c0d0990 2024-04-04T06:18:06.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.402+0000 7f2e61c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e5c15f800 0x7f2e5c0aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:06.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.402+0000 7f2e61c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e5c15f800 0x7f2e5c0aed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35570/0 (socket says 172.21.15.159:35570) 2024-04-04T06:18:06.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.402+0000 7f2e61c00640 1 -- 172.21.15.159:0/1793705197 learned_addr learned my addr 172.21.15.159:0/1793705197 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:06.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.402+0000 7f2e53e00640 1 --2- 172.21.15.159:0/1793705197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e5c155550 0x7f2e5c0cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:06.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.402+0000 7f2e61200640 1 --2- 172.21.15.159:0/1793705197 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e5c0d0990 0x7f2e5c06f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:06.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.403+0000 7f2e61c00640 1 -- 172.21.15.159:0/1793705197 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e5c0d0990 msgr2=0x7f2e5c06f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:06.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.403+0000 7f2e61c00640 1 --2- 172.21.15.159:0/1793705197 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e5c0d0990 0x7f2e5c06f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.398 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.403+0000 7f2e61c00640 1 -- 172.21.15.159:0/1793705197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e5c155550 msgr2=0x7f2e5c0cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:06.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.403+0000 7f2e61c00640 1 --2- 172.21.15.159:0/1793705197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e5c155550 0x7f2e5c0cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.399 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.403+0000 7f2e61c00640 1 -- 172.21.15.159:0/1793705197 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5c0b0e90 con 0x7f2e5c15f800 2024-04-04T06:18:06.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.403+0000 7f2e61c00640 1 --2- 172.21.15.159:0/1793705197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e5c15f800 0x7f2e5c0aed30 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400efc0 tx=0x7f2e5400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:06.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.403+0000 7f2e59600640 1 -- 172.21.15.159:0/1793705197 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54018020 con 0x7f2e5c15f800 2024-04-04T06:18:06.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.403+0000 7f2e59600640 1 -- 172.21.15.159:0/1793705197 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e54014480 con 0x7f2e5c15f800 2024-04-04T06:18:06.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.403+0000 7f2e63c00640 1 -- 172.21.15.159:0/1793705197 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e5c14de60 con 0x7f2e5c15f800 2024-04-04T06:18:06.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.403+0000 7f2e59600640 1 -- 172.21.15.159:0/1793705197 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54009e60 con 0x7f2e5c15f800 2024-04-04T06:18:06.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.404+0000 7f2e63c00640 1 -- 172.21.15.159:0/1793705197 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e5c0ae810 con 0x7f2e5c15f800 2024-04-04T06:18:06.400 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.404+0000 7f2e63c00640 1 -- 172.21.15.159:0/1793705197 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e5c0685c0 con 0x7f2e5c15f800 2024-04-04T06:18:06.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.405+0000 7f2e59600640 1 -- 172.21.15.159:0/1793705197 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e54005d10 con 0x7f2e5c15f800 2024-04-04T06:18:06.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.406+0000 7f2e59600640 1 --2- 172.21.15.159:0/1793705197 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2e3806c8f0 0x7f2e3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:06.401 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.406+0000 7f2e59600640 1 -- 172.21.15.159:0/1793705197 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f2e5408c5e0 con 0x7f2e5c15f800 2024-04-04T06:18:06.402 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.407+0000 7f2e61200640 1 --2- 172.21.15.159:0/1793705197 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2e3806c8f0 0x7f2e3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:06.403 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.408+0000 7f2e61200640 1 --2- 172.21.15.159:0/1793705197 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2e3806c8f0 0x7f2e3806edb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c036000 tx=0x7f2e4c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:06.406 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.411+0000 7f2e59600640 1 -- 172.21.15.159:0/1793705197 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f2e54022030 con 0x7f2e5c15f800 2024-04-04T06:18:06.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.610+0000 7f2e63c00640 1 -- 172.21.15.159:0/1793705197 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f2e5c144d70 con 0x7f2e3806c8f0 2024-04-04T06:18:06.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.611+0000 7f2e59600640 1 -- 172.21.15.159:0/1793705197 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2e5c144d70 con 0x7f2e3806c8f0 2024-04-04T06:18:06.606 INFO:teuthology.orchestra.run.smithi159.stderr:instructed osd(s) 5 to deep-scrub 2024-04-04T06:18:06.609 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.614+0000 7f2e63c00640 1 -- 172.21.15.159:0/1793705197 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2e3806c8f0 msgr2=0x7f2e3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:06.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.614+0000 7f2e63c00640 1 --2- 172.21.15.159:0/1793705197 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2e3806c8f0 0x7f2e3806edb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c036000 tx=0x7f2e4c008030 comp rx=0 tx=0).stop 2024-04-04T06:18:06.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.614+0000 7f2e63c00640 1 -- 172.21.15.159:0/1793705197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e5c15f800 msgr2=0x7f2e5c0aed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:06.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.614+0000 7f2e63c00640 1 --2- 172.21.15.159:0/1793705197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e5c15f800 0x7f2e5c0aed30 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400efc0 tx=0x7f2e5400c490 comp rx=0 tx=0).stop 2024-04-04T06:18:06.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.614+0000 7f2e63c00640 1 -- 172.21.15.159:0/1793705197 shutdown_connections 2024-04-04T06:18:06.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.614+0000 7f2e63c00640 1 --2- 172.21.15.159:0/1793705197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e5c15f800 0x7f2e5c0aed30 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.614+0000 7f2e63c00640 1 --2- 172.21.15.159:0/1793705197 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f2e3806c8f0 0x7f2e3806edb0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.614+0000 7f2e63c00640 1 --2- 172.21.15.159:0/1793705197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e5c155550 0x7f2e5c0cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.614+0000 7f2e63c00640 1 --2- 172.21.15.159:0/1793705197 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2e5c0d0990 0x7f2e5c06f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.614+0000 7f2e63c00640 1 -- 172.21.15.159:0/1793705197 >> 172.21.15.159:0/1793705197 conn(0x7f2e5c06b750 msgr2=0x7f2e5c15d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:06.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.614+0000 7f2e63c00640 1 -- 172.21.15.159:0/1793705197 shutdown_connections 2024-04-04T06:18:06.610 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.614+0000 7f2e63c00640 1 -- 172.21.15.159:0/1793705197 wait complete. 2024-04-04T06:18:06.627 INFO:tasks.ceph:Scrubbing osd.6 2024-04-04T06:18:06.627 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-04T06:18:06.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.853+0000 7f314b600640 1 Processor -- start 2024-04-04T06:18:06.848 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.853+0000 7f314b600640 1 -- start start 2024-04-04T06:18:06.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.855+0000 7f314b600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3144072720 0x7f31440b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:06.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.855+0000 7f314b600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31440aefc0 0x7f314406dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:06.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.855+0000 7f314b600640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f314406ea10 con 0x7f3144072b00 2024-04-04T06:18:06.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.855+0000 7f314b600640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f31440b3e80 con 0x7f3144072720 2024-04-04T06:18:06.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.855+0000 7f314b600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f314406ecf0 con 0x7f31440aefc0 2024-04-04T06:18:06.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.855+0000 7f3148c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3144072720 0x7f31440b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:06.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.855+0000 7f3148c00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f3144072720 0x7f31440b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35578/0 (socket says 172.21.15.159:35578) 2024-04-04T06:18:06.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.855+0000 7f3148c00640 1 -- 172.21.15.159:0/991951596 learned_addr learned my addr 172.21.15.159:0/991951596 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:06.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.856+0000 7f3149600640 1 --2- 172.21.15.159:0/991951596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31440aefc0 0x7f314406dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:06.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.856+0000 7f3148c00640 1 -- 172.21.15.159:0/991951596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31440aefc0 msgr2=0x7f314406dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:06.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.856+0000 7f3148c00640 1 --2- 172.21.15.159:0/991951596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31440aefc0 0x7f314406dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.856+0000 7f3148c00640 1 -- 172.21.15.159:0/991951596 >> v1:172.21.15.159:6789/0 conn(0x7f3144072b00 legacy=0x7f314406d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:06.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.856+0000 7f3148c00640 1 -- 172.21.15.159:0/991951596 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314406efd0 con 0x7f3144072720 2024-04-04T06:18:06.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.857+0000 7f3148c00640 1 --2- 172.21.15.159:0/991951596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3144072720 0x7f31440b3940 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f3138009870 tx=0x7f313802db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c4a5732b8473f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:06.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.857+0000 7f3143400640 1 -- 172.21.15.159:0/991951596 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313803b070 con 0x7f3144072720 2024-04-04T06:18:06.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.857+0000 7f3143400640 1 -- 172.21.15.159:0/991951596 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f313802e940 con 0x7f3144072720 2024-04-04T06:18:06.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.857+0000 7f3143400640 1 -- 172.21.15.159:0/991951596 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313802ed10 con 0x7f3144072720 2024-04-04T06:18:06.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.858+0000 7f314b600640 1 -- 172.21.15.159:0/991951596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3144072720 msgr2=0x7f31440b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:06.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.858+0000 7f314b600640 1 --2- 172.21.15.159:0/991951596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3144072720 0x7f31440b3940 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f3138009870 tx=0x7f313802db90 comp rx=0 tx=0).stop 2024-04-04T06:18:06.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.858+0000 7f314b600640 1 -- 172.21.15.159:0/991951596 shutdown_connections 2024-04-04T06:18:06.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.858+0000 7f314b600640 1 --2- 172.21.15.159:0/991951596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f31440aefc0 0x7f314406dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.858+0000 7f314b600640 1 --2- 172.21.15.159:0/991951596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3144072720 0x7f31440b3940 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.858+0000 7f314b600640 1 -- 172.21.15.159:0/991951596 >> 172.21.15.159:0/991951596 conn(0x7f314406bba0 msgr2=0x7f31440b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:06.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.858+0000 7f314b600640 1 -- 172.21.15.159:0/991951596 shutdown_connections 2024-04-04T06:18:06.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.858+0000 7f314b600640 1 -- 172.21.15.159:0/991951596 wait complete. 2024-04-04T06:18:06.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.860+0000 7f314b600640 1 Processor -- start 2024-04-04T06:18:06.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.860+0000 7f314b600640 1 -- start start 2024-04-04T06:18:06.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.861+0000 7f314b600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3144072720 0x7f314409d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:06.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.861+0000 7f314b600640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3144072b00 0x7f314409b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:06.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.861+0000 7f314b600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31440aefc0 0x7f314414e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:06.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.861+0000 7f314b600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f314406e690 con 0x7f3144072720 2024-04-04T06:18:06.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.861+0000 7f314b600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f314406f590 con 0x7f31440aefc0 2024-04-04T06:18:06.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.861+0000 7f314b600640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f3144072f70 con 0x7f3144072b00 2024-04-04T06:18:06.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.861+0000 7f3148c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3144072720 0x7f314409d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:06.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.861+0000 7f3148c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3144072720 0x7f314409d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35588/0 (socket says 172.21.15.159:35588) 2024-04-04T06:18:06.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.861+0000 7f3148c00640 1 -- 172.21.15.159:0/2165136351 learned_addr learned my addr 172.21.15.159:0/2165136351 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:06.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.861+0000 7f3143e00640 1 --2- 172.21.15.159:0/2165136351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3144072b00 0x7f314409b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:06.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.861+0000 7f3148c00640 1 -- 172.21.15.159:0/2165136351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3144072b00 msgr2=0x7f314409b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:06.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.861+0000 7f3148c00640 1 --2- 172.21.15.159:0/2165136351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3144072b00 0x7f314409b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.861+0000 7f3148c00640 1 -- 172.21.15.159:0/2165136351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31440aefc0 msgr2=0x7f314414e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:06.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.861+0000 7f3148c00640 1 --2- 172.21.15.159:0/2165136351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31440aefc0 0x7f314414e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.861+0000 7f3148c00640 1 -- 172.21.15.159:0/2165136351 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31440cd9e0 con 0x7f3144072720 2024-04-04T06:18:06.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.862+0000 7f3148c00640 1 --2- 172.21.15.159:0/2165136351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3144072720 0x7f314409d0a0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f3138000c00 tx=0x7f31380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:06.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.862+0000 7f3140c00640 1 -- 172.21.15.159:0/2165136351 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313803b040 con 0x7f3144072720 2024-04-04T06:18:06.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.862+0000 7f3140c00640 1 -- 172.21.15.159:0/2165136351 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3138046070 con 0x7f3144072720 2024-04-04T06:18:06.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.862+0000 7f3140c00640 1 -- 172.21.15.159:0/2165136351 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3138041440 con 0x7f3144072720 2024-04-04T06:18:06.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.862+0000 7f314b600640 1 -- 172.21.15.159:0/2165136351 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31441f0610 con 0x7f3144072720 2024-04-04T06:18:06.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.862+0000 7f314b600640 1 -- 172.21.15.159:0/2165136351 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31441f0bd0 con 0x7f3144072720 2024-04-04T06:18:06.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.863+0000 7f314b600640 1 -- 172.21.15.159:0/2165136351 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f31440685c0 con 0x7f3144072720 2024-04-04T06:18:06.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.864+0000 7f3140c00640 1 -- 172.21.15.159:0/2165136351 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f313802e800 con 0x7f3144072720 2024-04-04T06:18:06.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.865+0000 7f3140c00640 1 --2- 172.21.15.159:0/2165136351 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f312406c8f0 0x7f312406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:06.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.865+0000 7f3140c00640 1 -- 172.21.15.159:0/2165136351 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f31380afab0 con 0x7f3144072720 2024-04-04T06:18:06.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.865+0000 7f3140c00640 1 --2- 172.21.15.159:0/2165136351 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x7f31240728f0 0x7f3124074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:06.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.865+0000 7f3140c00640 1 -- 172.21.15.159:0/2165136351 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3138030ea0 con 0x7f31240728f0 2024-04-04T06:18:06.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.865+0000 7f3140c00640 1 -- 172.21.15.159:0/2165136351 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7f31380b42c0 con 0x7f3144072720 2024-04-04T06:18:06.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.866+0000 7f3143e00640 1 --2- 172.21.15.159:0/2165136351 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f312406c8f0 0x7f312406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:06.861 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.866+0000 7f3149600640 1 --2- 172.21.15.159:0/2165136351 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x7f31240728f0 0x7f3124074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:06.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.867+0000 7f3143e00640 1 --2- 172.21.15.159:0/2165136351 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f312406c8f0 0x7f312406edb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f31440b3820 tx=0x7f3130009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:06.862 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.867+0000 7f3149600640 1 --2- 172.21.15.159:0/2165136351 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x7f31240728f0 0x7f3124074d10 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:06.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.870+0000 7f3140c00640 1 -- 172.21.15.159:0/2165136351 <== osd.6 v2:172.21.15.178:6826/198949157 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3138030ea0 con 0x7f31240728f0 2024-04-04T06:18:06.879 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.884+0000 7f314b600640 1 -- 172.21.15.159:0/2165136351 --> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f31440cc4d0 con 0x7f31240728f0 2024-04-04T06:18:06.880 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.885+0000 7f3140c00640 1 -- 172.21.15.159:0/2165136351 <== osd.6 v2:172.21.15.178:6826/198949157 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f31440cc4d0 con 0x7f31240728f0 2024-04-04T06:18:06.880 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:18:06.881 INFO:teuthology.orchestra.run.smithi159.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-04T06:18:06.881 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:18:06.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.886+0000 7f314b600640 1 -- 172.21.15.159:0/2165136351 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x7f31240728f0 msgr2=0x7f3124074d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:06.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.886+0000 7f314b600640 1 --2- 172.21.15.159:0/2165136351 >> [v2:172.21.15.178:6826/198949157,v1:172.21.15.178:6827/198949157] conn(0x7f31240728f0 0x7f3124074d10 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.886+0000 7f314b600640 1 -- 172.21.15.159:0/2165136351 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f312406c8f0 msgr2=0x7f312406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:06.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.886+0000 7f314b600640 1 --2- 172.21.15.159:0/2165136351 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f312406c8f0 0x7f312406edb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f31440b3820 tx=0x7f3130009040 comp rx=0 tx=0).stop 2024-04-04T06:18:06.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.886+0000 7f314b600640 1 -- 172.21.15.159:0/2165136351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3144072720 msgr2=0x7f314409d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:06.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.886+0000 7f314b600640 1 --2- 172.21.15.159:0/2165136351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3144072720 0x7f314409d0a0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f3138000c00 tx=0x7f31380046c0 comp rx=0 tx=0).stop 2024-04-04T06:18:06.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.886+0000 7f3149600640 1 -- 172.21.15.159:0/2165136351 reap_dead start 2024-04-04T06:18:06.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.886+0000 7f314b600640 1 -- 172.21.15.159:0/2165136351 shutdown_connections 2024-04-04T06:18:06.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.886+0000 7f314b600640 1 -- 172.21.15.159:0/2165136351 >> 172.21.15.159:0/2165136351 conn(0x7f314406bba0 msgr2=0x7f3144071610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:06.882 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.886+0000 7f314b600640 1 -- 172.21.15.159:0/2165136351 shutdown_connections 2024-04-04T06:18:06.883 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:06.887+0000 7f314b600640 1 -- 172.21.15.159:0/2165136351 wait complete. 2024-04-04T06:18:06.897 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-04T06:18:07.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.102+0000 7f778c400640 1 Processor -- start 2024-04-04T06:18:07.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.102+0000 7f778c400640 1 -- start start 2024-04-04T06:18:07.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.104+0000 7f778c400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7784072100 0x7f778406f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:07.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.104+0000 7f778c400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7784162d90 0x7f778406efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:07.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.104+0000 7f778c400640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7784163270 con 0x7f778415e960 2024-04-04T06:18:07.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.104+0000 7f778c400640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f778406e220 con 0x7f7784162d90 2024-04-04T06:18:07.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.104+0000 7f778c400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f778406e500 con 0x7f7784072100 2024-04-04T06:18:07.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.104+0000 7f778a400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7784162d90 0x7f778406efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:07.100 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.104+0000 7f778a400640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f7784162d90 0x7f778406efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35590/0 (socket says 172.21.15.159:35590) 2024-04-04T06:18:07.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.104+0000 7f778a400640 1 -- 172.21.15.159:0/1664668245 learned_addr learned my addr 172.21.15.159:0/1664668245 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:07.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.105+0000 7f7789000640 1 --2- 172.21.15.159:0/1664668245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7784072100 0x7f778406f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:07.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.105+0000 7f778a400640 1 -- 172.21.15.159:0/1664668245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7784072100 msgr2=0x7f778406f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:07.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.105+0000 7f778a400640 1 --2- 172.21.15.159:0/1664668245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7784072100 0x7f778406f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.105+0000 7f778a400640 1 -- 172.21.15.159:0/1664668245 >> v1:172.21.15.178:6789/0 conn(0x7f778415e960 legacy=0x7f77840719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:07.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.105+0000 7f778a400640 1 -- 172.21.15.159:0/1664668245 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778406e7e0 con 0x7f7784162d90 2024-04-04T06:18:07.101 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.105+0000 7f7789000640 1 --2- 172.21.15.159:0/1664668245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7784072100 0x7f778406f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:18:07.102 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.106+0000 7f778a400640 1 --2- 172.21.15.159:0/1664668245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7784162d90 0x7f778406efe0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f777c009870 tx=0x7f777c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbdead9a8158c4e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:07.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.106+0000 7f7783e00640 1 -- 172.21.15.159:0/1664668245 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c035020 con 0x7f7784162d90 2024-04-04T06:18:07.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.106+0000 7f7783e00640 1 -- 172.21.15.159:0/1664668245 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f777c031640 con 0x7f7784162d90 2024-04-04T06:18:07.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.106+0000 7f7783e00640 1 -- 172.21.15.159:0/1664668245 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c02d040 con 0x7f7784162d90 2024-04-04T06:18:07.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.107+0000 7f778c400640 1 -- 172.21.15.159:0/1664668245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7784162d90 msgr2=0x7f778406efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:07.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.107+0000 7f778c400640 1 --2- 172.21.15.159:0/1664668245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7784162d90 0x7f778406efe0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f777c009870 tx=0x7f777c02b8f0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.107+0000 7f778c400640 1 -- 172.21.15.159:0/1664668245 shutdown_connections 2024-04-04T06:18:07.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.107+0000 7f778c400640 1 --2- 172.21.15.159:0/1664668245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7784162d90 0x7f778406efe0 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.107+0000 7f778c400640 1 --2- 172.21.15.159:0/1664668245 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7784072100 0x7f778406f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.107+0000 7f778c400640 1 -- 172.21.15.159:0/1664668245 >> 172.21.15.159:0/1664668245 conn(0x7f778406bba0 msgr2=0x7f77840b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:07.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.107+0000 7f778c400640 1 -- 172.21.15.159:0/1664668245 shutdown_connections 2024-04-04T06:18:07.103 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.107+0000 7f778c400640 1 -- 172.21.15.159:0/1664668245 wait complete. 2024-04-04T06:18:07.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.109+0000 7f778c400640 1 Processor -- start 2024-04-04T06:18:07.104 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.109+0000 7f778c400640 1 -- start start 2024-04-04T06:18:07.106 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.110+0000 7f778c400640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7784072100 0x7f778415bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:07.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.110+0000 7f778c400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f778415e960 0x7f77841f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:07.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.110+0000 7f778c400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7784162d90 0x7f778414df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:07.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.110+0000 7f778c400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f77840b3450 con 0x7f7784162d90 2024-04-04T06:18:07.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.110+0000 7f778c400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f77840701f0 con 0x7f778415e960 2024-04-04T06:18:07.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.110+0000 7f778c400640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f77840b35d0 con 0x7f7784072100 2024-04-04T06:18:07.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.110+0000 7f778a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7784162d90 0x7f778414df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:07.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.110+0000 7f778a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7784162d90 0x7f778414df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35606/0 (socket says 172.21.15.159:35606) 2024-04-04T06:18:07.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.110+0000 7f778a400640 1 -- 172.21.15.159:0/2886753926 learned_addr learned my addr 172.21.15.159:0/2886753926 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:07.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.111+0000 7f778a400640 1 -- 172.21.15.159:0/2886753926 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7784072100 msgr2=0x7f778415bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:07.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.111+0000 7f7789a00640 1 --2- 172.21.15.159:0/2886753926 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7784072100 0x7f778415bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:07.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.111+0000 7f7789000640 1 --2- 172.21.15.159:0/2886753926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f778415e960 0x7f77841f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:07.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.111+0000 7f778a400640 1 --2- 172.21.15.159:0/2886753926 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7784072100 0x7f778415bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.107 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.111+0000 7f778a400640 1 -- 172.21.15.159:0/2886753926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f778415e960 msgr2=0x7f77841f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:07.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.111+0000 7f778a400640 1 --2- 172.21.15.159:0/2886753926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f778415e960 0x7f77841f4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.111+0000 7f778a400640 1 -- 172.21.15.159:0/2886753926 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778415c5c0 con 0x7f7784162d90 2024-04-04T06:18:07.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.111+0000 7f7789a00640 1 --2- 172.21.15.159:0/2886753926 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7784072100 0x7f778415bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:18:07.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.111+0000 7f778a400640 1 --2- 172.21.15.159:0/2886753926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7784162d90 0x7f778414df80 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f777c009870 tx=0x7f777c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:07.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.111+0000 7f7781600640 1 -- 172.21.15.159:0/2886753926 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c040070 con 0x7f7784162d90 2024-04-04T06:18:07.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.111+0000 7f7781600640 1 -- 172.21.15.159:0/2886753926 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f777c039c10 con 0x7f7784162d90 2024-04-04T06:18:07.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.111+0000 7f7781600640 1 -- 172.21.15.159:0/2886753926 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c035040 con 0x7f7784162d90 2024-04-04T06:18:07.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.111+0000 7f778c400640 1 -- 172.21.15.159:0/2886753926 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f778415c8a0 con 0x7f7784162d90 2024-04-04T06:18:07.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.112+0000 7f778c400640 1 -- 172.21.15.159:0/2886753926 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f778415ce60 con 0x7f7784162d90 2024-04-04T06:18:07.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.113+0000 7f778c400640 1 -- 172.21.15.159:0/2886753926 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77840685c0 con 0x7f7784162d90 2024-04-04T06:18:07.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.114+0000 7f7781600640 1 -- 172.21.15.159:0/2886753926 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f777c04b060 con 0x7f7784162d90 2024-04-04T06:18:07.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.114+0000 7f7781600640 1 --2- 172.21.15.159:0/2886753926 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f776406c8f0 0x7f776406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:07.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.115+0000 7f7781600640 1 -- 172.21.15.159:0/2886753926 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f777c0ad390 con 0x7f7784162d90 2024-04-04T06:18:07.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.115+0000 7f7789a00640 1 --2- 172.21.15.159:0/2886753926 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f776406c8f0 0x7f776406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:07.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.116+0000 7f7789a00640 1 --2- 172.21.15.159:0/2886753926 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f776406c8f0 0x7f776406edb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7778004520 tx=0x7f7778009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:07.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.119+0000 7f7781600640 1 -- 172.21.15.159:0/2886753926 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f777c07ba00 con 0x7f7784162d90 2024-04-04T06:18:07.317 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.322+0000 7f778c400640 1 -- 172.21.15.159:0/2886753926 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f77840b31e0 con 0x7f776406c8f0 2024-04-04T06:18:07.318 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.323+0000 7f7781600640 1 -- 172.21.15.159:0/2886753926 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f77840b31e0 con 0x7f776406c8f0 2024-04-04T06:18:07.318 INFO:teuthology.orchestra.run.smithi159.stderr:instructed osd(s) 6 to deep-scrub 2024-04-04T06:18:07.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.325+0000 7f778c400640 1 -- 172.21.15.159:0/2886753926 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f776406c8f0 msgr2=0x7f776406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:07.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.325+0000 7f778c400640 1 --2- 172.21.15.159:0/2886753926 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f776406c8f0 0x7f776406edb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7778004520 tx=0x7f7778009040 comp rx=0 tx=0).stop 2024-04-04T06:18:07.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.325+0000 7f778c400640 1 -- 172.21.15.159:0/2886753926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7784162d90 msgr2=0x7f778414df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:07.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.325+0000 7f778c400640 1 --2- 172.21.15.159:0/2886753926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7784162d90 0x7f778414df80 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f777c009870 tx=0x7f777c0046c0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.326+0000 7f778c400640 1 -- 172.21.15.159:0/2886753926 shutdown_connections 2024-04-04T06:18:07.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.326+0000 7f778c400640 1 --2- 172.21.15.159:0/2886753926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7784162d90 0x7f778414df80 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.326+0000 7f778c400640 1 --2- 172.21.15.159:0/2886753926 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f776406c8f0 0x7f776406edb0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.326+0000 7f778c400640 1 --2- 172.21.15.159:0/2886753926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f778415e960 0x7f77841f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.326+0000 7f778c400640 1 --2- 172.21.15.159:0/2886753926 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7784072100 0x7f778415bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.326+0000 7f778c400640 1 -- 172.21.15.159:0/2886753926 >> 172.21.15.159:0/2886753926 conn(0x7f778406bba0 msgr2=0x7f77840b1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:07.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.326+0000 7f778c400640 1 -- 172.21.15.159:0/2886753926 shutdown_connections 2024-04-04T06:18:07.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.326+0000 7f778c400640 1 -- 172.21.15.159:0/2886753926 wait complete. 2024-04-04T06:18:07.339 INFO:tasks.ceph:Scrubbing osd.7 2024-04-04T06:18:07.339 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-04T06:18:07.546 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.551+0000 7f6e47e00640 1 Processor -- start 2024-04-04T06:18:07.547 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.551+0000 7f6e47e00640 1 -- start start 2024-04-04T06:18:07.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.552+0000 7f6e47e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f6e48144980 0x7f6e4815ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:07.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.552+0000 7f6e47e00640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e48071540 con 0x7f6e4815f150 2024-04-04T06:18:07.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.552+0000 7f6e47e00640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e48144d60 con 0x7f6e480b35b0 2024-04-04T06:18:07.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.552+0000 7f6e47e00640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f6e480b3b60 con 0x7f6e48144980 2024-04-04T06:18:07.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.553+0000 7f6e46a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f6e48144980 0x7f6e4815ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:07.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.553+0000 7f6e46000640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f6e4815f150 0x7f6e480b2ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:52062/0 (socket says 172.21.15.159:52062) 2024-04-04T06:18:07.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.553+0000 7f6e46000640 1 -- 172.21.15.159:0/4170307749 learned_addr learned my addr 172.21.15.159:0/4170307749 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:18:07.548 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.553+0000 7f6e46a00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f6e48144980 0x7f6e4815ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35616/0 (socket says 172.21.15.159:35616) 2024-04-04T06:18:07.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.553+0000 7f6e46a00640 1 -- 172.21.15.159:0/4170307749 >> v1:172.21.15.178:6789/0 conn(0x7f6e480b35b0 legacy=0x7f6e48071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:07.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.553+0000 7f6e46a00640 1 -- 172.21.15.159:0/4170307749 >> v1:172.21.15.159:6789/0 conn(0x7f6e4815f150 legacy=0x7f6e480b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:07.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.553+0000 7f6e46a00640 1 -- 172.21.15.159:0/4170307749 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e480b3e40 con 0x7f6e48144980 2024-04-04T06:18:07.549 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.553+0000 7f6e45600640 1 -- 172.21.15.159:0/4170307749 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1745996212 0 0) 0x7f6e48071540 con 0x7f6e4815f150 2024-04-04T06:18:07.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.554+0000 7f6e46a00640 1 --2- 172.21.15.159:0/4170307749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e48144980 0x7f6e4815ec10 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800ba40 tx=0x7f6e3802e010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf094550376362f server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:07.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.555+0000 7f6e45600640 1 -- 172.21.15.159:0/4170307749 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3803b070 con 0x7f6e48144980 2024-04-04T06:18:07.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.555+0000 7f6e45600640 1 -- 172.21.15.159:0/4170307749 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e3802edc0 con 0x7f6e48144980 2024-04-04T06:18:07.550 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.555+0000 7f6e45600640 1 -- 172.21.15.159:0/4170307749 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38036660 con 0x7f6e48144980 2024-04-04T06:18:07.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.555+0000 7f6e47e00640 1 -- 172.21.15.159:0/4170307749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e48144980 msgr2=0x7f6e4815ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:07.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.555+0000 7f6e47e00640 1 --2- 172.21.15.159:0/4170307749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e48144980 0x7f6e4815ec10 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800ba40 tx=0x7f6e3802e010 comp rx=0 tx=0).stop 2024-04-04T06:18:07.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.555+0000 7f6e47e00640 1 -- 172.21.15.159:0/4170307749 shutdown_connections 2024-04-04T06:18:07.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.555+0000 7f6e47e00640 1 --2- 172.21.15.159:0/4170307749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e48144980 0x7f6e4815ec10 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.555+0000 7f6e47e00640 1 -- 172.21.15.159:0/4170307749 >> 172.21.15.159:0/4170307749 conn(0x7f6e480b1d80 msgr2=0x7f6e480b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:07.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.556+0000 7f6e47e00640 1 -- 172.21.15.159:0/4170307749 shutdown_connections 2024-04-04T06:18:07.551 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.556+0000 7f6e47e00640 1 -- 172.21.15.159:0/4170307749 wait complete. 2024-04-04T06:18:07.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.557+0000 7f6e47e00640 1 Processor -- start 2024-04-04T06:18:07.552 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.557+0000 7f6e47e00640 1 -- start start 2024-04-04T06:18:07.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.558+0000 7f6e47e00640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e480b35b0 0x7f6e48136730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:07.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.558+0000 7f6e47e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e48144980 0x7f6e481e7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:07.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.558+0000 7f6e47e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e4815f150 0x7f6e481e5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:07.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.558+0000 7f6e47e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6e480722d0 con 0x7f6e48144980 2024-04-04T06:18:07.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.558+0000 7f6e47e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6e48072150 con 0x7f6e4815f150 2024-04-04T06:18:07.554 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.558+0000 7f6e47e00640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f6e48072450 con 0x7f6e480b35b0 2024-04-04T06:18:07.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.558+0000 7f6e46000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e48144980 0x7f6e481e7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:07.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.559+0000 7f6e46000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e48144980 0x7f6e481e7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35632/0 (socket says 172.21.15.159:35632) 2024-04-04T06:18:07.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.559+0000 7f6e46000640 1 -- 172.21.15.159:0/3917822170 learned_addr learned my addr 172.21.15.159:0/3917822170 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:07.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.559+0000 7f6e46a00640 1 --2- 172.21.15.159:0/3917822170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e480b35b0 0x7f6e48136730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:07.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.559+0000 7f6e47400640 1 --2- 172.21.15.159:0/3917822170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e4815f150 0x7f6e481e5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:07.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.559+0000 7f6e46000640 1 -- 172.21.15.159:0/3917822170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e480b35b0 msgr2=0x7f6e48136730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:07.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.559+0000 7f6e46000640 1 --2- 172.21.15.159:0/3917822170 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6e480b35b0 0x7f6e48136730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.559+0000 7f6e46000640 1 -- 172.21.15.159:0/3917822170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e4815f150 msgr2=0x7f6e481e5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:07.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.559+0000 7f6e46000640 1 --2- 172.21.15.159:0/3917822170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e4815f150 0x7f6e481e5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.559+0000 7f6e46000640 1 -- 172.21.15.159:0/3917822170 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e48136c70 con 0x7f6e48144980 2024-04-04T06:18:07.555 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.559+0000 7f6e46000640 1 --2- 172.21.15.159:0/3917822170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e48144980 0x7f6e481e7310 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f6e3400c430 tx=0x7f6e3400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:07.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.560+0000 7f6e3ea00640 1 -- 172.21.15.159:0/3917822170 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e340074e0 con 0x7f6e48144980 2024-04-04T06:18:07.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.560+0000 7f6e3ea00640 1 -- 172.21.15.159:0/3917822170 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e340047e0 con 0x7f6e48144980 2024-04-04T06:18:07.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.560+0000 7f6e3ea00640 1 -- 172.21.15.159:0/3917822170 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e34004bd0 con 0x7f6e48144980 2024-04-04T06:18:07.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.560+0000 7f6e47e00640 1 -- 172.21.15.159:0/3917822170 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e481e6270 con 0x7f6e48144980 2024-04-04T06:18:07.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.560+0000 7f6e47e00640 1 -- 172.21.15.159:0/3917822170 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e481e6830 con 0x7f6e48144980 2024-04-04T06:18:07.556 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.561+0000 7f6e47e00640 1 -- 172.21.15.159:0/3917822170 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6e48075940 con 0x7f6e48144980 2024-04-04T06:18:07.557 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.562+0000 7f6e3ea00640 1 -- 172.21.15.159:0/3917822170 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e34011070 con 0x7f6e48144980 2024-04-04T06:18:07.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.563+0000 7f6e3ea00640 1 --2- 172.21.15.159:0/3917822170 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6e2806c8f0 0x7f6e2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:07.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.563+0000 7f6e3ea00640 1 -- 172.21.15.159:0/3917822170 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f6e3408b5c0 con 0x7f6e48144980 2024-04-04T06:18:07.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.563+0000 7f6e3ea00640 1 --2- 172.21.15.159:0/3917822170 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x7f6e280728f0 0x7f6e28074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:07.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.563+0000 7f6e3ea00640 1 -- 172.21.15.159:0/3917822170 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6e3401ee50 con 0x7f6e280728f0 2024-04-04T06:18:07.558 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.563+0000 7f6e3ea00640 1 -- 172.21.15.159:0/3917822170 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7f6e34092050 con 0x7f6e48144980 2024-04-04T06:18:07.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.563+0000 7f6e46a00640 1 --2- 172.21.15.159:0/3917822170 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6e2806c8f0 0x7f6e2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:07.559 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.564+0000 7f6e47400640 1 --2- 172.21.15.159:0/3917822170 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x7f6e280728f0 0x7f6e28074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:07.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.565+0000 7f6e46a00640 1 --2- 172.21.15.159:0/3917822170 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6e2806c8f0 0x7f6e2806edb0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6e38038000 tx=0x7f6e38008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:07.560 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.565+0000 7f6e47400640 1 --2- 172.21.15.159:0/3917822170 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x7f6e280728f0 0x7f6e28074d10 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:07.562 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.567+0000 7f6e3ea00640 1 -- 172.21.15.159:0/3917822170 <== osd.7 v2:172.21.15.178:6810/1066817938 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6e3401ee50 con 0x7f6e280728f0 2024-04-04T06:18:07.576 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.581+0000 7f6e47e00640 1 -- 172.21.15.159:0/3917822170 --> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6e48073140 con 0x7f6e280728f0 2024-04-04T06:18:07.577 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.582+0000 7f6e3ea00640 1 -- 172.21.15.159:0/3917822170 <== osd.7 v2:172.21.15.178:6810/1066817938 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6e48073140 con 0x7f6e280728f0 2024-04-04T06:18:07.578 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-04-04T06:18:07.578 INFO:teuthology.orchestra.run.smithi159.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-04T06:18:07.578 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-04-04T06:18:07.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.583+0000 7f6e47e00640 1 -- 172.21.15.159:0/3917822170 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x7f6e280728f0 msgr2=0x7f6e28074d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:07.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.583+0000 7f6e47e00640 1 --2- 172.21.15.159:0/3917822170 >> [v2:172.21.15.178:6810/1066817938,v1:172.21.15.178:6811/1066817938] conn(0x7f6e280728f0 0x7f6e28074d10 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.583+0000 7f6e47e00640 1 -- 172.21.15.159:0/3917822170 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6e2806c8f0 msgr2=0x7f6e2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:07.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.583+0000 7f6e47e00640 1 --2- 172.21.15.159:0/3917822170 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f6e2806c8f0 0x7f6e2806edb0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6e38038000 tx=0x7f6e38008040 comp rx=0 tx=0).stop 2024-04-04T06:18:07.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.583+0000 7f6e47e00640 1 -- 172.21.15.159:0/3917822170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e48144980 msgr2=0x7f6e481e7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:07.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.583+0000 7f6e47e00640 1 --2- 172.21.15.159:0/3917822170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e48144980 0x7f6e481e7310 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f6e3400c430 tx=0x7f6e3400c8f0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.583+0000 7f6e47400640 1 -- 172.21.15.159:0/3917822170 reap_dead start 2024-04-04T06:18:07.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.583+0000 7f6e47e00640 1 -- 172.21.15.159:0/3917822170 shutdown_connections 2024-04-04T06:18:07.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.583+0000 7f6e47e00640 1 -- 172.21.15.159:0/3917822170 >> 172.21.15.159:0/3917822170 conn(0x7f6e480b1d80 msgr2=0x7f6e480b2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:07.579 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.583+0000 7f6e47e00640 1 -- 172.21.15.159:0/3917822170 shutdown_connections 2024-04-04T06:18:07.580 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.583+0000 7f6e47e00640 1 -- 172.21.15.159:0/3917822170 wait complete. 2024-04-04T06:18:07.596 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-04T06:18:07.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.805+0000 7fb270000640 1 Processor -- start 2024-04-04T06:18:07.801 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.806+0000 7fb270000640 1 -- start start 2024-04-04T06:18:07.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.807+0000 7fb270000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb268153580 0x7fb2680d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:07.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.807+0000 7fb270000640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb2680d07c0 0x7fb2680aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:07.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.807+0000 7fb270000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2680af2e0 0x7fb26806d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:07.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.807+0000 7fb270000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fb2680d0c30 con 0x7fb268153580 2024-04-04T06:18:07.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.807+0000 7fb270000640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7fb2680afab0 con 0x7fb2680d07c0 2024-04-04T06:18:07.803 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.807+0000 7fb270000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb26815cc80 con 0x7fb2680af2e0 2024-04-04T06:18:07.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.808+0000 7fb26d600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb268153580 0x7fb2680d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:07.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.808+0000 7fb26d600640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fb268153580 0x7fb2680d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35644/0 (socket says 172.21.15.159:35644) 2024-04-04T06:18:07.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.808+0000 7fb26d600640 1 -- 172.21.15.159:0/2574631796 learned_addr learned my addr 172.21.15.159:0/2574631796 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:07.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.808+0000 7fb26e000640 1 --2- 172.21.15.159:0/2574631796 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2680af2e0 0x7fb26806d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:07.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.808+0000 7fb26cc00640 1 --2- 172.21.15.159:0/2574631796 >> v2:172.21.15.178:3300/0 conn(0x7fb2680d07c0 0x7fb2680aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:07.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.808+0000 7fb26d600640 1 -- 172.21.15.159:0/2574631796 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2680af2e0 msgr2=0x7fb26806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:07.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.808+0000 7fb26d600640 1 --2- 172.21.15.159:0/2574631796 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2680af2e0 0x7fb26806d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.808+0000 7fb26d600640 1 -- 172.21.15.159:0/2574631796 >> v2:172.21.15.178:3300/0 conn(0x7fb2680d07c0 msgr2=0x7fb2680aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:07.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.808+0000 7fb26d600640 1 --2- 172.21.15.159:0/2574631796 >> v2:172.21.15.178:3300/0 conn(0x7fb2680d07c0 0x7fb2680aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.804 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.808+0000 7fb26d600640 1 -- 172.21.15.159:0/2574631796 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26815da00 con 0x7fb268153580 2024-04-04T06:18:07.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.809+0000 7fb26d600640 1 --2- 172.21.15.159:0/2574631796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb268153580 0x7fb2680d0280 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fb2680cff70 tx=0x7fb25c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157cb96c88f0768e server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:07.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.810+0000 7fb267e00640 1 -- 172.21.15.159:0/2574631796 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c02e070 con 0x7fb268153580 2024-04-04T06:18:07.806 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.810+0000 7fb267e00640 1 -- 172.21.15.159:0/2574631796 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb25c037070 con 0x7fb268153580 2024-04-04T06:18:07.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.810+0000 7fb267e00640 1 -- 172.21.15.159:0/2574631796 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c031a00 con 0x7fb268153580 2024-04-04T06:18:07.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.810+0000 7fb270000640 1 -- 172.21.15.159:0/2574631796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb268153580 msgr2=0x7fb2680d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:07.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.810+0000 7fb270000640 1 --2- 172.21.15.159:0/2574631796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb268153580 0x7fb2680d0280 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fb2680cff70 tx=0x7fb25c02b880 comp rx=0 tx=0).stop 2024-04-04T06:18:07.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.810+0000 7fb270000640 1 -- 172.21.15.159:0/2574631796 shutdown_connections 2024-04-04T06:18:07.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.810+0000 7fb270000640 1 --2- 172.21.15.159:0/2574631796 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2680af2e0 0x7fb26806d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.810+0000 7fb270000640 1 --2- 172.21.15.159:0/2574631796 >> v2:172.21.15.178:3300/0 conn(0x7fb2680d07c0 0x7fb2680aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.810+0000 7fb270000640 1 --2- 172.21.15.159:0/2574631796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb268153580 0x7fb2680d0280 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.810+0000 7fb270000640 1 -- 172.21.15.159:0/2574631796 >> 172.21.15.159:0/2574631796 conn(0x7fb26806b750 msgr2=0x7fb26806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:07.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.810+0000 7fb270000640 1 -- 172.21.15.159:0/2574631796 shutdown_connections 2024-04-04T06:18:07.807 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.810+0000 7fb270000640 1 -- 172.21.15.159:0/2574631796 wait complete. 2024-04-04T06:18:07.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.812+0000 7fb270000640 1 Processor -- start 2024-04-04T06:18:07.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.812+0000 7fb270000640 1 -- start start 2024-04-04T06:18:07.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.813+0000 7fb270000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2680af2e0 0x7fb268148010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:07.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.813+0000 7fb270000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2680d07c0 0x7fb2680cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:07.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.813+0000 7fb270000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb268153580 0x7fb2680cee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:07.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.813+0000 7fb270000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb26815d4e0 con 0x7fb268153580 2024-04-04T06:18:07.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.813+0000 7fb270000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb26815d200 con 0x7fb2680af2e0 2024-04-04T06:18:07.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.813+0000 7fb270000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fb268153960 con 0x7fb2680d07c0 2024-04-04T06:18:07.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.813+0000 7fb26e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb268153580 0x7fb2680cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:07.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.813+0000 7fb26e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb268153580 0x7fb2680cee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35656/0 (socket says 172.21.15.159:35656) 2024-04-04T06:18:07.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.813+0000 7fb26e000640 1 -- 172.21.15.159:0/989817756 learned_addr learned my addr 172.21.15.159:0/989817756 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:07.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.814+0000 7fb26e000640 1 -- 172.21.15.159:0/989817756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2680d07c0 msgr2=0x7fb2680cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:07.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.814+0000 7fb26cc00640 1 --2- 172.21.15.159:0/989817756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2680d07c0 0x7fb2680cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:07.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.814+0000 7fb26d600640 1 --2- 172.21.15.159:0/989817756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2680af2e0 0x7fb268148010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:07.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.814+0000 7fb26e000640 1 --2- 172.21.15.159:0/989817756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2680d07c0 0x7fb2680cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.814+0000 7fb26e000640 1 -- 172.21.15.159:0/989817756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2680af2e0 msgr2=0x7fb268148010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:07.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.814+0000 7fb26e000640 1 --2- 172.21.15.159:0/989817756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2680af2e0 0x7fb268148010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:07.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.814+0000 7fb26e000640 1 -- 172.21.15.159:0/989817756 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb268155a20 con 0x7fb268153580 2024-04-04T06:18:07.810 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.814+0000 7fb26e000640 1 --2- 172.21.15.159:0/989817756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb268153580 0x7fb2680cee30 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fb260007d50 tx=0x7fb26000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:07.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.814+0000 7fb265600640 1 -- 172.21.15.159:0/989817756 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb260014420 con 0x7fb268153580 2024-04-04T06:18:07.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.815+0000 7fb265600640 1 -- 172.21.15.159:0/989817756 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb260019070 con 0x7fb268153580 2024-04-04T06:18:07.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.815+0000 7fb265600640 1 -- 172.21.15.159:0/989817756 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb260014820 con 0x7fb268153580 2024-04-04T06:18:07.811 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.815+0000 7fb270000640 1 -- 172.21.15.159:0/989817756 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb268155d00 con 0x7fb268153580 2024-04-04T06:18:07.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.815+0000 7fb270000640 1 -- 172.21.15.159:0/989817756 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2681562c0 con 0x7fb268153580 2024-04-04T06:18:07.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.816+0000 7fb270000640 1 -- 172.21.15.159:0/989817756 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2680685c0 con 0x7fb268153580 2024-04-04T06:18:07.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.817+0000 7fb265600640 1 -- 172.21.15.159:0/989817756 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2600149c0 con 0x7fb268153580 2024-04-04T06:18:07.812 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.818+0000 7fb265600640 1 --2- 172.21.15.159:0/989817756 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb24806c920 0x7fb24806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:07.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.818+0000 7fb265600640 1 -- 172.21.15.159:0/989817756 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7fb26008c0e0 con 0x7fb268153580 2024-04-04T06:18:07.813 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.818+0000 7fb26d600640 1 --2- 172.21.15.159:0/989817756 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb24806c920 0x7fb24806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:07.814 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.819+0000 7fb26d600640 1 --2- 172.21.15.159:0/989817756 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb24806c920 0x7fb24806ede0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb25c036000 tx=0x7fb25c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:07.818 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:07.823+0000 7fb265600640 1 -- 172.21.15.159:0/989817756 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fb26005a750 con 0x7fb268153580 2024-04-04T06:18:08.017 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.022+0000 7fb270000640 1 -- 172.21.15.159:0/989817756 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fb26815a240 con 0x7fb24806c920 2024-04-04T06:18:08.018 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.023+0000 7fb265600640 1 -- 172.21.15.159:0/989817756 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb25c037040 con 0x7fb24806c920 2024-04-04T06:18:08.018 INFO:teuthology.orchestra.run.smithi159.stderr:instructed osd(s) 7 to deep-scrub 2024-04-04T06:18:08.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.025+0000 7fb270000640 1 -- 172.21.15.159:0/989817756 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb24806c920 msgr2=0x7fb24806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:08.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.025+0000 7fb270000640 1 --2- 172.21.15.159:0/989817756 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb24806c920 0x7fb24806ede0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb25c036000 tx=0x7fb25c008030 comp rx=0 tx=0).stop 2024-04-04T06:18:08.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.025+0000 7fb270000640 1 -- 172.21.15.159:0/989817756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb268153580 msgr2=0x7fb2680cee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:08.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.025+0000 7fb270000640 1 --2- 172.21.15.159:0/989817756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb268153580 0x7fb2680cee30 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fb260007d50 tx=0x7fb26000e4b0 comp rx=0 tx=0).stop 2024-04-04T06:18:08.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.025+0000 7fb270000640 1 -- 172.21.15.159:0/989817756 shutdown_connections 2024-04-04T06:18:08.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.025+0000 7fb270000640 1 --2- 172.21.15.159:0/989817756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb268153580 0x7fb2680cee30 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:08.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.025+0000 7fb270000640 1 --2- 172.21.15.159:0/989817756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb2680d07c0 0x7fb2680cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:08.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.025+0000 7fb270000640 1 --2- 172.21.15.159:0/989817756 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fb24806c920 0x7fb24806ede0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:08.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.025+0000 7fb270000640 1 --2- 172.21.15.159:0/989817756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2680af2e0 0x7fb268148010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:08.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.025+0000 7fb270000640 1 -- 172.21.15.159:0/989817756 >> 172.21.15.159:0/989817756 conn(0x7fb26806b750 msgr2=0x7fb2680b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:08.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.026+0000 7fb270000640 1 -- 172.21.15.159:0/989817756 shutdown_connections 2024-04-04T06:18:08.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.026+0000 7fb270000640 1 -- 172.21.15.159:0/989817756 wait complete. 2024-04-04T06:18:08.040 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:18:08.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.254+0000 7fc38e000640 1 Processor -- start 2024-04-04T06:18:08.250 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.255+0000 7fc38e000640 1 -- start start 2024-04-04T06:18:08.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.256+0000 7fc38e000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc38806cc20 0x7fc38806e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:08.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.256+0000 7fc38e000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc3881606b0 0x7fc38806b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:08.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.256+0000 7fc38e000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3880b0450 con 0x7fc3880cf950 2024-04-04T06:18:08.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.256+0000 7fc38e000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7fc3880cf4c0 con 0x7fc3881606b0 2024-04-04T06:18:08.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.256+0000 7fc38e000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc3880ceb90 con 0x7fc38806cc20 2024-04-04T06:18:08.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.257+0000 7fc387e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc3881606b0 0x7fc38806b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:08.252 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.257+0000 7fc387e00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7fc3881606b0 0x7fc38806b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35668/0 (socket says 172.21.15.159:35668) 2024-04-04T06:18:08.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.257+0000 7fc387e00640 1 -- 172.21.15.159:0/2104563463 learned_addr learned my addr 172.21.15.159:0/2104563463 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:08.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.257+0000 7fc387400640 1 --2- 172.21.15.159:0/2104563463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc38806cc20 0x7fc38806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:08.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.257+0000 7fc387e00640 1 -- 172.21.15.159:0/2104563463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc38806cc20 msgr2=0x7fc38806e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:08.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.257+0000 7fc387e00640 1 --2- 172.21.15.159:0/2104563463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc38806cc20 0x7fc38806e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:08.253 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.257+0000 7fc387e00640 1 -- 172.21.15.159:0/2104563463 >> v1:172.21.15.178:6789/0 conn(0x7fc3880cf950 legacy=0x7fc38815ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:08.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.257+0000 7fc387e00640 1 -- 172.21.15.159:0/2104563463 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3880cee70 con 0x7fc3881606b0 2024-04-04T06:18:08.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.259+0000 7fc387e00640 1 --2- 172.21.15.159:0/2104563463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3881606b0 0x7fc38806b570 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fc3880cf640 tx=0x7fc38002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb82cad2c19bb996 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:08.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.259+0000 7fc386000640 1 -- 172.21.15.159:0/2104563463 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38002caf0 con 0x7fc3881606b0 2024-04-04T06:18:08.254 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.259+0000 7fc386000640 1 -- 172.21.15.159:0/2104563463 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc38002cc90 con 0x7fc3881606b0 2024-04-04T06:18:08.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.259+0000 7fc386000640 1 -- 172.21.15.159:0/2104563463 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380034610 con 0x7fc3881606b0 2024-04-04T06:18:08.255 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.260+0000 7fc38e000640 1 -- 172.21.15.159:0/2104563463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3881606b0 msgr2=0x7fc38806b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:08.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.260+0000 7fc38e000640 1 --2- 172.21.15.159:0/2104563463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3881606b0 0x7fc38806b570 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fc3880cf640 tx=0x7fc38002bf50 comp rx=0 tx=0).stop 2024-04-04T06:18:08.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.260+0000 7fc38e000640 1 -- 172.21.15.159:0/2104563463 shutdown_connections 2024-04-04T06:18:08.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.260+0000 7fc38e000640 1 --2- 172.21.15.159:0/2104563463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3881606b0 0x7fc38806b570 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:08.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.260+0000 7fc38e000640 1 --2- 172.21.15.159:0/2104563463 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc38806cc20 0x7fc38806e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:08.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.260+0000 7fc38e000640 1 -- 172.21.15.159:0/2104563463 >> 172.21.15.159:0/2104563463 conn(0x7fc3880ce400 msgr2=0x7fc388071e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:08.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.260+0000 7fc38e000640 1 -- 172.21.15.159:0/2104563463 shutdown_connections 2024-04-04T06:18:08.256 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.260+0000 7fc38e000640 1 -- 172.21.15.159:0/2104563463 wait complete. 2024-04-04T06:18:08.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.262+0000 7fc38e000640 1 Processor -- start 2024-04-04T06:18:08.257 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.262+0000 7fc38e000640 1 -- start start 2024-04-04T06:18:08.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.263+0000 7fc38e000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc38806cc20 0x7fc388143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:08.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.263+0000 7fc38e000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3880cf950 0x7fc388158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:08.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.263+0000 7fc38e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3881606b0 0x7fc3881596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:08.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.263+0000 7fc38e000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc3880b0000 con 0x7fc3881606b0 2024-04-04T06:18:08.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.263+0000 7fc38e000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc388160d80 con 0x7fc3880cf950 2024-04-04T06:18:08.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.263+0000 7fc38e000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fc3880b0180 con 0x7fc38806cc20 2024-04-04T06:18:08.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.263+0000 7fc387e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3881606b0 0x7fc3881596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:08.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.263+0000 7fc387e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3881606b0 0x7fc3881596a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35674/0 (socket says 172.21.15.159:35674) 2024-04-04T06:18:08.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.263+0000 7fc387e00640 1 -- 172.21.15.159:0/4260514512 learned_addr learned my addr 172.21.15.159:0/4260514512 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:08.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.264+0000 7fc387e00640 1 -- 172.21.15.159:0/4260514512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc38806cc20 msgr2=0x7fc388143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:08.259 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.264+0000 7fc387e00640 1 --2- 172.21.15.159:0/4260514512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc38806cc20 0x7fc388143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:08.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.264+0000 7fc387e00640 1 -- 172.21.15.159:0/4260514512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3880cf950 msgr2=0x7fc388158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:08.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.264+0000 7fc387e00640 1 --2- 172.21.15.159:0/4260514512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3880cf950 0x7fc388158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:08.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.264+0000 7fc387e00640 1 -- 172.21.15.159:0/4260514512 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc388144330 con 0x7fc3881606b0 2024-04-04T06:18:08.260 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.264+0000 7fc387e00640 1 --2- 172.21.15.159:0/4260514512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3881606b0 0x7fc3881596a0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fc380009f40 tx=0x7fc380004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:08.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.264+0000 7fc37f400640 1 -- 172.21.15.159:0/4260514512 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380043020 con 0x7fc3881606b0 2024-04-04T06:18:08.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.264+0000 7fc37f400640 1 -- 172.21.15.159:0/4260514512 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc38003e450 con 0x7fc3881606b0 2024-04-04T06:18:08.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.264+0000 7fc38e000640 1 -- 172.21.15.159:0/4260514512 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc388144610 con 0x7fc3881606b0 2024-04-04T06:18:08.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.264+0000 7fc37f400640 1 -- 172.21.15.159:0/4260514512 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380009510 con 0x7fc3881606b0 2024-04-04T06:18:08.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.264+0000 7fc38e000640 1 -- 172.21.15.159:0/4260514512 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc388144bd0 con 0x7fc3881606b0 2024-04-04T06:18:08.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.266+0000 7fc38e000640 1 -- 172.21.15.159:0/4260514512 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc388068cd0 con 0x7fc3881606b0 2024-04-04T06:18:08.261 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.266+0000 7fc37f400640 1 -- 172.21.15.159:0/4260514512 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc380046020 con 0x7fc3881606b0 2024-04-04T06:18:08.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.267+0000 7fc37f400640 1 --2- 172.21.15.159:0/4260514512 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc36806c8f0 0x7fc36806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:08.262 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.267+0000 7fc37f400640 1 -- 172.21.15.159:0/4260514512 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7fc3800ae150 con 0x7fc3881606b0 2024-04-04T06:18:08.263 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.268+0000 7fc387400640 1 --2- 172.21.15.159:0/4260514512 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc36806c8f0 0x7fc36806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:08.264 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.269+0000 7fc387400640 1 --2- 172.21.15.159:0/4260514512 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc36806c8f0 0x7fc36806edb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc3780098f0 tx=0x7fc378008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:08.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.272+0000 7fc37f400640 1 -- 172.21.15.159:0/4260514512 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fc38004d030 con 0x7fc3881606b0 2024-04-04T06:18:08.450 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.455+0000 7fc38e000640 1 -- 172.21.15.159:0/4260514512 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc38815e0b0 con 0x7fc36806c8f0 2024-04-04T06:18:08.452 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.457+0000 7fc37f400640 1 -- 172.21.15.159:0/4260514512 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13391 (secure 0 0 0) 0x7fc38815e0b0 con 0x7fc36806c8f0 2024-04-04T06:18:08.453 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:18:08.453 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:18:08.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.459+0000 7fc38e000640 1 -- 172.21.15.159:0/4260514512 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc36806c8f0 msgr2=0x7fc36806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:08.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.459+0000 7fc38e000640 1 --2- 172.21.15.159:0/4260514512 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc36806c8f0 0x7fc36806edb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc3780098f0 tx=0x7fc378008040 comp rx=0 tx=0).stop 2024-04-04T06:18:08.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.459+0000 7fc38e000640 1 -- 172.21.15.159:0/4260514512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3881606b0 msgr2=0x7fc3881596a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:08.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.459+0000 7fc38e000640 1 --2- 172.21.15.159:0/4260514512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3881606b0 0x7fc3881596a0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fc380009f40 tx=0x7fc380004a40 comp rx=0 tx=0).stop 2024-04-04T06:18:08.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.460+0000 7fc38e000640 1 -- 172.21.15.159:0/4260514512 shutdown_connections 2024-04-04T06:18:08.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.460+0000 7fc38e000640 1 --2- 172.21.15.159:0/4260514512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3881606b0 0x7fc3881596a0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:08.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.460+0000 7fc38e000640 1 --2- 172.21.15.159:0/4260514512 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7fc36806c8f0 0x7fc36806edb0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:08.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.460+0000 7fc38e000640 1 --2- 172.21.15.159:0/4260514512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3880cf950 0x7fc388158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:08.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.460+0000 7fc38e000640 1 --2- 172.21.15.159:0/4260514512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc38806cc20 0x7fc388143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:08.455 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.460+0000 7fc38e000640 1 -- 172.21.15.159:0/4260514512 >> 172.21.15.159:0/4260514512 conn(0x7fc3880ce400 msgr2=0x7fc38815f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:08.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.460+0000 7fc38e000640 1 -- 172.21.15.159:0/4260514512 shutdown_connections 2024-04-04T06:18:08.456 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:08.460+0000 7fc38e000640 1 -- 172.21.15.159:0/4260514512 wait complete. 2024-04-04T06:18:08.472 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-04-04T06:18:06.987676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":278808,"kb_used_data":12712,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754695912,"statfs":{"total":773094113280,"available":772808613888,"internally_reserved":0,"allocated":13017088,"data_stored":6331264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8388625,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":-32,"num_read_kb":-13,"num_write":-29,"num_write_kb":-8195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002979"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":351,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-04T06:17:57.385769+0000","last_change":"2024-04-04T06:05:42.425591+0000","last_active":"2024-04-04T06:17:57.385769+0000","last_peered":"2024-04-04T06:17:57.385769+0000","last_clean":"2024-04-04T06:17:57.385769+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:17:57.385769+0000","last_undegraded":"2024-04-04T06:17:57.385769+0000","last_fullsized":"2024-04-04T06:17:57.385769+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-04T06:05:41.062861+0000","last_clean_scrub_stamp":"2024-04-04T06:05:41.062861+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T12:07:20.181907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509591,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37364,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334476,"statfs":{"total":96636764160,"available":96598503424,"internally_reserved":0,"allocated":2072576,"data_stored":1234184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509593,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36784,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335056,"statfs":{"total":96636764160,"available":96599097344,"internally_reserved":0,"allocated":1478656,"data_stored":643816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509594,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1478656,"data_stored":643816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509593,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32620,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1478656,"data_stored":643816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509592,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1478656,"data_stored":643816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509594,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33208,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338632,"statfs":{"total":96636764160,"available":96602759168,"internally_reserved":0,"allocated":2072576,"data_stored":1234184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542297,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36788,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":1478656,"data_stored":643816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542295,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36788,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":1478656,"data_stored":643816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:18:08.476 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-04T06:05:41.062861+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=4, tm_hour=6, tm_min=5, tm_sec=41, tm_wday=3, tm_yday=95, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=4, tm_hour=6, tm_min=18, tm_sec=1, tm_wday=3, tm_yday=95, tm_isdst=0) 2024-04-04T06:18:08.476 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-04T06:18:14.726 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:18:14.734 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:18:28.477 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-04T06:18:28.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.704+0000 7fef0e200640 1 Processor -- start 2024-04-04T06:18:28.700 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.705+0000 7fef0e200640 1 -- start start 2024-04-04T06:18:28.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.706+0000 7fef0e200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef080d0d60 0x7fef08073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:28.701 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.706+0000 7fef0e200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef0815fe80 con 0x7fef080af830 2024-04-04T06:18:28.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.706+0000 7fef0e200640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef080ad120 con 0x7fef0815c390 2024-04-04T06:18:28.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.706+0000 7fef0e200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fef0815c770 con 0x7fef080d0d60 2024-04-04T06:18:28.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.706+0000 7feefea00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7fef080af830 0x7fef080d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:53934/0 (socket says 172.21.15.159:53934) 2024-04-04T06:18:28.702 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.706+0000 7feefea00640 1 -- 172.21.15.159:0/4282409674 learned_addr learned my addr 172.21.15.159:0/4282409674 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:18:28.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.707+0000 7fef07e00640 1 --2- 172.21.15.159:0/4282409674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef080d0d60 0x7fef08073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:28.705 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.707+0000 7fef06a00640 1 -- 172.21.15.159:0/4282409674 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2919085102 0 0) 0x7fef0815fe80 con 0x7fef080af830 2024-04-04T06:18:28.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.707+0000 7fef06a00640 1 -- 172.21.15.159:0/4282409674 --> v1:172.21.15.159:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feeec003520 con 0x7fef080af830 2024-04-04T06:18:28.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.707+0000 7fef06a00640 1 -- 172.21.15.159:0/4282409674 <== mon.0 v1:172.21.15.159:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2107313826 0 0) 0x7feeec003520 con 0x7fef080af830 2024-04-04T06:18:28.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.707+0000 7fef06a00640 1 -- 172.21.15.159:0/4282409674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef080d0d60 msgr2=0x7fef08073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:28.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.707+0000 7fef06a00640 1 --2- 172.21.15.159:0/4282409674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef080d0d60 0x7fef08073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:28.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.707+0000 7fef06a00640 1 -- 172.21.15.159:0/4282409674 >> v1:172.21.15.178:6789/0 conn(0x7fef0815c390 legacy=0x7fef08160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:28.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.707+0000 7fef06a00640 1 -- 172.21.15.159:0/4282409674 --> v1:172.21.15.159:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef080ad400 con 0x7fef080af830 2024-04-04T06:18:28.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.707+0000 7fef06a00640 1 -- 172.21.15.159:0/4282409674 <== mon.0 v1:172.21.15.159:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 472288951 0 0) 0x7feef0001670 con 0x7fef080af830 2024-04-04T06:18:28.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.707+0000 7fef06a00640 1 -- 172.21.15.159:0/4282409674 >> v1:172.21.15.159:6789/0 conn(0x7fef080af830 legacy=0x7fef080d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:28.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.707+0000 7fef06a00640 1 --2- 172.21.15.159:0/4282409674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeec0039e0 0x7feeec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:28.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.708+0000 7fef06a00640 1 --2- 172.21.15.159:0/4282409674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef080d0d60 0x7feeec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:28.706 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.708+0000 7fef06a00640 1 --2- 172.21.15.159:0/4282409674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feeec004d40 0x7feeec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:28.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.708+0000 7fef06a00640 1 -- 172.21.15.159:0/4282409674 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feeec00ba70 con 0x7feeec004d40 2024-04-04T06:18:28.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.708+0000 7fef06a00640 1 -- 172.21.15.159:0/4282409674 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feeec00bc20 con 0x7feeec0039e0 2024-04-04T06:18:28.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.708+0000 7fef06a00640 1 -- 172.21.15.159:0/4282409674 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7feeec00be20 con 0x7fef080d0d60 2024-04-04T06:18:28.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.708+0000 7fef07e00640 1 --2- 172.21.15.159:0/4282409674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feeec004d40 0x7feeec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:28.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.708+0000 7feefea00640 1 --2- 172.21.15.159:0/4282409674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef080d0d60 0x7feeec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:28.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.708+0000 7fef07e00640 1 -- 172.21.15.159:0/4282409674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef080d0d60 msgr2=0x7feeec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:28.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.708+0000 7fef07e00640 1 --2- 172.21.15.159:0/4282409674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef080d0d60 0x7feeec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:28.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.708+0000 7fef07400640 1 --2- 172.21.15.159:0/4282409674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeec0039e0 0x7feeec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:28.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.708+0000 7fef07e00640 1 -- 172.21.15.159:0/4282409674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeec0039e0 msgr2=0x7feeec003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:28.707 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.708+0000 7fef07e00640 1 --2- 172.21.15.159:0/4282409674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeec0039e0 0x7feeec003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:28.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.708+0000 7fef07e00640 1 -- 172.21.15.159:0/4282409674 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeec003520 con 0x7feeec004d40 2024-04-04T06:18:28.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.708+0000 7fef07400640 1 --2- 172.21.15.159:0/4282409674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feeec0039e0 0x7feeec003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:18:28.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.708+0000 7feefea00640 1 --2- 172.21.15.159:0/4282409674 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef080d0d60 0x7feeec004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:18:28.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.709+0000 7fef07e00640 1 --2- 172.21.15.159:0/4282409674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feeec004d40 0x7feeec0051c0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fef080b5470 tx=0x7feeec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8c9bd34062b2ef server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:28.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.710+0000 7fef06a00640 1 -- 172.21.15.159:0/4282409674 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef00039070 con 0x7feeec004d40 2024-04-04T06:18:28.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.710+0000 7fef06a00640 1 -- 172.21.15.159:0/4282409674 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef0002cb60 con 0x7feeec004d40 2024-04-04T06:18:28.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.710+0000 7fef06a00640 1 -- 172.21.15.159:0/4282409674 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef00009c10 con 0x7feeec004d40 2024-04-04T06:18:28.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.710+0000 7fef0e200640 1 -- 172.21.15.159:0/4282409674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feeec004d40 msgr2=0x7feeec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:28.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.710+0000 7fef0e200640 1 --2- 172.21.15.159:0/4282409674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feeec004d40 0x7feeec0051c0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fef080b5470 tx=0x7feeec005830 comp rx=0 tx=0).stop 2024-04-04T06:18:28.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.710+0000 7fef07e00640 1 -- 172.21.15.159:0/4282409674 reap_dead start 2024-04-04T06:18:28.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.710+0000 7fef0e200640 1 -- 172.21.15.159:0/4282409674 shutdown_connections 2024-04-04T06:18:28.708 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.710+0000 7fef0e200640 1 -- 172.21.15.159:0/4282409674 >> 172.21.15.159:0/4282409674 conn(0x7fef0806b990 msgr2=0x7fef0806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:28.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.710+0000 7fef0e200640 1 -- 172.21.15.159:0/4282409674 shutdown_connections 2024-04-04T06:18:28.709 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.710+0000 7fef0e200640 1 -- 172.21.15.159:0/4282409674 wait complete. 2024-04-04T06:18:28.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.712+0000 7fef0e200640 1 Processor -- start 2024-04-04T06:18:28.711 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.712+0000 7fef0e200640 1 -- start start 2024-04-04T06:18:28.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.713+0000 7fef0e200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeec004d40 0x7fef0814d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:28.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.713+0000 7fef0e200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef080ce7d0 0x7fef081f80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:28.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.713+0000 7fef0e200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef080cec50 0x7fef081f85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:28.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.713+0000 7fef0e200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fef08160d50 con 0x7fef080cec50 2024-04-04T06:18:28.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.713+0000 7fef0e200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fef08160120 con 0x7fef080ce7d0 2024-04-04T06:18:28.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.713+0000 7fef0e200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7fef080cf980 con 0x7feeec004d40 2024-04-04T06:18:28.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.714+0000 7fef07e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef080cec50 0x7fef081f85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:28.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.714+0000 7fef07e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef080cec50 0x7fef081f85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38058/0 (socket says 172.21.15.159:38058) 2024-04-04T06:18:28.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.714+0000 7fef07e00640 1 -- 172.21.15.159:0/476720540 learned_addr learned my addr 172.21.15.159:0/476720540 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:28.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.714+0000 7feefea00640 1 --2- 172.21.15.159:0/476720540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef080ce7d0 0x7fef081f80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:28.712 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.714+0000 7fef07400640 1 --2- 172.21.15.159:0/476720540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeec004d40 0x7fef0814d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:28.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.714+0000 7fef07e00640 1 -- 172.21.15.159:0/476720540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeec004d40 msgr2=0x7fef0814d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:28.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.714+0000 7fef07e00640 1 --2- 172.21.15.159:0/476720540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeec004d40 0x7fef0814d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:28.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.714+0000 7fef07e00640 1 -- 172.21.15.159:0/476720540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef080ce7d0 msgr2=0x7fef081f80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:28.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.714+0000 7fef07e00640 1 --2- 172.21.15.159:0/476720540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef080ce7d0 0x7fef081f80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:28.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.714+0000 7fef07e00640 1 -- 172.21.15.159:0/476720540 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef081594b0 con 0x7fef080cec50 2024-04-04T06:18:28.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.714+0000 7feefea00640 1 --2- 172.21.15.159:0/476720540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef080ce7d0 0x7fef081f80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:18:28.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.714+0000 7fef07e00640 1 --2- 172.21.15.159:0/476720540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef080cec50 0x7fef081f85f0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fef0002c5d0 tx=0x7fef00009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:28.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.715+0000 7feeffe00640 1 -- 172.21.15.159:0/476720540 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef00042020 con 0x7fef080cec50 2024-04-04T06:18:28.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.715+0000 7feeffe00640 1 -- 172.21.15.159:0/476720540 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef0003a040 con 0x7fef080cec50 2024-04-04T06:18:28.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.715+0000 7feeffe00640 1 -- 172.21.15.159:0/476720540 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef00039070 con 0x7fef080cec50 2024-04-04T06:18:28.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.715+0000 7fef0e200640 1 -- 172.21.15.159:0/476720540 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0809a610 con 0x7fef080cec50 2024-04-04T06:18:28.713 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.715+0000 7fef0e200640 1 -- 172.21.15.159:0/476720540 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef080b3240 con 0x7fef080cec50 2024-04-04T06:18:28.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.716+0000 7fef0e200640 1 -- 172.21.15.159:0/476720540 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef080759e0 con 0x7fef080cec50 2024-04-04T06:18:28.714 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.717+0000 7feeffe00640 1 -- 172.21.15.159:0/476720540 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef00045020 con 0x7fef080cec50 2024-04-04T06:18:28.715 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.718+0000 7feeffe00640 1 --2- 172.21.15.159:0/476720540 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feee406c920 0x7feee406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:28.715 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.718+0000 7feeffe00640 1 -- 172.21.15.159:0/476720540 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7fef000ad7d0 con 0x7fef080cec50 2024-04-04T06:18:28.715 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.719+0000 7fef07400640 1 --2- 172.21.15.159:0/476720540 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feee406c920 0x7feee406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:28.715 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.720+0000 7fef07400640 1 --2- 172.21.15.159:0/476720540 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feee406c920 0x7feee406ede0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fef081f91e0 tx=0x7feef8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:28.718 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.723+0000 7feeffe00640 1 -- 172.21.15.159:0/476720540 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7fef080759e0 con 0x7fef080cec50 2024-04-04T06:18:28.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.920+0000 7fef0e200640 1 -- 172.21.15.159:0/476720540 --> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fef080685c0 con 0x7feee406c920 2024-04-04T06:18:28.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.923+0000 7feeffe00640 1 -- 172.21.15.159:0/476720540 <== mgr.4102 v2:172.21.15.178:6800/3272374747 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13379 (secure 0 0 0) 0x7fef080685c0 con 0x7feee406c920 2024-04-04T06:18:28.918 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:18:28.918 INFO:teuthology.orchestra.run.smithi159.stderr:dumped all 2024-04-04T06:18:28.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.925+0000 7fef0e200640 1 -- 172.21.15.159:0/476720540 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feee406c920 msgr2=0x7feee406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:28.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.925+0000 7fef0e200640 1 --2- 172.21.15.159:0/476720540 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feee406c920 0x7feee406ede0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fef081f91e0 tx=0x7feef8009040 comp rx=0 tx=0).stop 2024-04-04T06:18:28.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.925+0000 7fef0e200640 1 -- 172.21.15.159:0/476720540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef080cec50 msgr2=0x7fef081f85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:28.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.925+0000 7fef0e200640 1 --2- 172.21.15.159:0/476720540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef080cec50 0x7fef081f85f0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fef0002c5d0 tx=0x7fef00009750 comp rx=0 tx=0).stop 2024-04-04T06:18:28.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.926+0000 7fef0e200640 1 -- 172.21.15.159:0/476720540 shutdown_connections 2024-04-04T06:18:28.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.926+0000 7fef0e200640 1 --2- 172.21.15.159:0/476720540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef080cec50 0x7fef081f85f0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:28.921 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.926+0000 7fef0e200640 1 --2- 172.21.15.159:0/476720540 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7feee406c920 0x7feee406ede0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:28.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.926+0000 7fef0e200640 1 --2- 172.21.15.159:0/476720540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef080ce7d0 0x7fef081f80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:28.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.926+0000 7fef0e200640 1 --2- 172.21.15.159:0/476720540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7feeec004d40 0x7fef0814d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:28.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.926+0000 7fef0e200640 1 -- 172.21.15.159:0/476720540 >> 172.21.15.159:0/476720540 conn(0x7fef0806b990 msgr2=0x7fef080b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:28.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.926+0000 7fef0e200640 1 -- 172.21.15.159:0/476720540 shutdown_connections 2024-04-04T06:18:28.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:28.926+0000 7fef0e200640 1 -- 172.21.15.159:0/476720540 wait complete. 2024-04-04T06:18:28.940 INFO:teuthology.orchestra.run.smithi159.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2024-04-04T06:18:26.993580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":278796,"kb_used_data":12712,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754695924,"statfs":{"total":773094113280,"available":772808626176,"internally_reserved":0,"allocated":13017088,"data_stored":6331264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003654"},"pg_stats":[{"pgid":"1.0","version":"119'119","reported_seq":362,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-04T06:18:08.153742+0000","last_change":"2024-04-04T06:18:08.153818+0000","last_active":"2024-04-04T06:18:08.153742+0000","last_peered":"2024-04-04T06:18:08.153742+0000","last_clean":"2024-04-04T06:18:08.153742+0000","last_became_active":"2024-04-04T06:05:42.423305+0000","last_became_peered":"2024-04-04T06:05:42.423305+0000","last_unstale":"2024-04-04T06:18:08.153742+0000","last_undegraded":"2024-04-04T06:18:08.153742+0000","last_fullsized":"2024-04-04T06:18:08.153742+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'119","last_scrub_stamp":"2024-04-04T06:18:08.153527+0000","last_deep_scrub":"119'119","last_deep_scrub_stamp":"2024-04-04T06:18:08.153527+0000","last_clean_scrub_stamp":"2024-04-04T06:18:08.153527+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-05T07:04:08.564909+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509595,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37360,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334480,"statfs":{"total":96636764160,"available":96598507520,"internally_reserved":0,"allocated":2072576,"data_stored":1234184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509597,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36784,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335056,"statfs":{"total":96636764160,"available":96599097344,"internally_reserved":0,"allocated":1478656,"data_stored":643816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509598,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1478656,"data_stored":643816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509597,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32620,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1478656,"data_stored":643816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1478656,"data_stored":643816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509598,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33200,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338640,"statfs":{"total":96636764160,"available":96602767360,"internally_reserved":0,"allocated":2072576,"data_stored":1234184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542301,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36788,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":1478656,"data_stored":643816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542299,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36788,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":1478656,"data_stored":643816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-04T06:18:28.941 DEBUG:teuthology.orchestra.run.smithi159:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-04T06:18:29.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.150+0000 7f9c6d200640 1 Processor -- start 2024-04-04T06:18:29.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.150+0000 7f9c6d200640 1 -- start start 2024-04-04T06:18:29.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.151+0000 7f9c6d200640 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9c680d0990 0x7f9c680aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:29.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.151+0000 7f9c6d200640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f9c680af2e0 0x7f9c681600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:29.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.151+0000 7f9c6d200640 1 -- --> v1:172.21.15.159:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c680afab0 con 0x7f9c68153580 2024-04-04T06:18:29.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.151+0000 7f9c6d200640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f9c6815cc80 con 0x7f9c680af2e0 2024-04-04T06:18:29.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.151+0000 7f9c6d200640 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 -- 0x7f9c6815da00 con 0x7f9c680d0990 2024-04-04T06:18:29.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.151+0000 7f9c65e00640 1 --1- >> v1:172.21.15.159:6789/0 conn(0x7f9c68153580 0x7f9c680d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.159:6789/0 says I am v1:172.21.15.159:53950/0 (socket says 172.21.15.159:53950) 2024-04-04T06:18:29.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.151+0000 7f9c65e00640 1 -- 172.21.15.159:0/684958330 learned_addr learned my addr 172.21.15.159:0/684958330 (peer_addr_for_me v1:172.21.15.159:0/0) 2024-04-04T06:18:29.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.152+0000 7f9c66800640 1 --2- 172.21.15.159:0/684958330 >> v2:172.21.15.159:3300/0 conn(0x7f9c680af2e0 0x7f9c681600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:29.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.152+0000 7f9c5d400640 1 --2- 172.21.15.159:0/684958330 >> v2:172.21.15.178:3300/0 conn(0x7f9c680d0990 0x7f9c680aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:29.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.152+0000 7f9c65400640 1 -- 172.21.15.159:0/684958330 <== mon.0 v1:172.21.15.159:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2143760022 0 0) 0x7f9c680afab0 con 0x7f9c68153580 2024-04-04T06:18:29.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.152+0000 7f9c66800640 1 -- 172.21.15.159:0/684958330 >> v2:172.21.15.178:3300/0 conn(0x7f9c680d0990 msgr2=0x7f9c680aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:29.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.152+0000 7f9c66800640 1 --2- 172.21.15.159:0/684958330 >> v2:172.21.15.178:3300/0 conn(0x7f9c680d0990 0x7f9c680aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:29.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.152+0000 7f9c66800640 1 -- 172.21.15.159:0/684958330 >> v1:172.21.15.159:6789/0 conn(0x7f9c68153580 legacy=0x7f9c680d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:29.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.152+0000 7f9c66800640 1 -- 172.21.15.159:0/684958330 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6815cf20 con 0x7f9c680af2e0 2024-04-04T06:18:29.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.152+0000 7f9c5d400640 1 --2- 172.21.15.159:0/684958330 >> v2:172.21.15.178:3300/0 conn(0x7f9c680d0990 0x7f9c680aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:18:29.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.153+0000 7f9c66800640 1 --2- 172.21.15.159:0/684958330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c680af2e0 0x7f9c681600d0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f9c6815a6b0 tx=0x7f9c6002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b2eaf871f6d962 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:29.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.153+0000 7f9c65400640 1 -- 172.21.15.159:0/684958330 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60039070 con 0x7f9c680af2e0 2024-04-04T06:18:29.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.153+0000 7f9c65400640 1 -- 172.21.15.159:0/684958330 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c6002cbf0 con 0x7f9c680af2e0 2024-04-04T06:18:29.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.153+0000 7f9c65400640 1 -- 172.21.15.159:0/684958330 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60034440 con 0x7f9c680af2e0 2024-04-04T06:18:29.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.154+0000 7f9c6d200640 1 -- 172.21.15.159:0/684958330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c680af2e0 msgr2=0x7f9c681600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:29.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.154+0000 7f9c6d200640 1 --2- 172.21.15.159:0/684958330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c680af2e0 0x7f9c681600d0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f9c6815a6b0 tx=0x7f9c6002be40 comp rx=0 tx=0).stop 2024-04-04T06:18:29.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.154+0000 7f9c6d200640 1 -- 172.21.15.159:0/684958330 shutdown_connections 2024-04-04T06:18:29.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.154+0000 7f9c6d200640 1 --2- 172.21.15.159:0/684958330 >> v2:172.21.15.178:3300/0 conn(0x7f9c680d0990 0x7f9c680aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:29.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.154+0000 7f9c6d200640 1 --2- 172.21.15.159:0/684958330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c680af2e0 0x7f9c681600d0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:29.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.154+0000 7f9c6d200640 1 -- 172.21.15.159:0/684958330 >> 172.21.15.159:0/684958330 conn(0x7f9c6806b750 msgr2=0x7f9c6806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:29.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.154+0000 7f9c6d200640 1 -- 172.21.15.159:0/684958330 shutdown_connections 2024-04-04T06:18:29.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.154+0000 7f9c6d200640 1 -- 172.21.15.159:0/684958330 wait complete. 2024-04-04T06:18:29.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.156+0000 7f9c6d200640 1 Processor -- start 2024-04-04T06:18:29.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.156+0000 7f9c6d200640 1 -- start start 2024-04-04T06:18:29.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.157+0000 7f9c6d200640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c680af2e0 0x7f9c681525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:29.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.157+0000 7f9c6d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c680d0990 0x7f9c680cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:29.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.157+0000 7f9c6d200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c68153580 0x7f9c680cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:29.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.157+0000 7f9c6d200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9c68153960 con 0x7f9c680d0990 2024-04-04T06:18:29.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.157+0000 7f9c6d200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6815d4e0 con 0x7f9c68153580 2024-04-04T06:18:29.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.157+0000 7f9c6d200640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f9c680af6c0 con 0x7f9c680af2e0 2024-04-04T06:18:29.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.157+0000 7f9c5d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c680d0990 0x7f9c680cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:29.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.157+0000 7f9c5d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c680d0990 0x7f9c680cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38072/0 (socket says 172.21.15.159:38072) 2024-04-04T06:18:29.153 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.157+0000 7f9c5d400640 1 -- 172.21.15.159:0/1046216013 learned_addr learned my addr 172.21.15.159:0/1046216013 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:29.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.158+0000 7f9c65e00640 1 --2- 172.21.15.159:0/1046216013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c680af2e0 0x7f9c681525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:29.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.158+0000 7f9c66800640 1 --2- 172.21.15.159:0/1046216013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c68153580 0x7f9c680cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:29.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.158+0000 7f9c5d400640 1 -- 172.21.15.159:0/1046216013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c680af2e0 msgr2=0x7f9c681525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:29.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.158+0000 7f9c5d400640 1 --2- 172.21.15.159:0/1046216013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c680af2e0 0x7f9c681525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:29.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.158+0000 7f9c5d400640 1 -- 172.21.15.159:0/1046216013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c68153580 msgr2=0x7f9c680cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:29.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.158+0000 7f9c5d400640 1 --2- 172.21.15.159:0/1046216013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c68153580 0x7f9c680cda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:29.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.158+0000 7f9c5d400640 1 -- 172.21.15.159:0/1046216013 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6809a190 con 0x7f9c680d0990 2024-04-04T06:18:29.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.158+0000 7f9c65e00640 1 --2- 172.21.15.159:0/1046216013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c680af2e0 0x7f9c681525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:18:29.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.158+0000 7f9c5d400640 1 --2- 172.21.15.159:0/1046216013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c680d0990 0x7f9c680cd4f0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f9c5000c7f0 tx=0x7f9c5000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:29.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.158+0000 7f9c5ea00640 1 -- 172.21.15.159:0/1046216013 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50015070 con 0x7f9c680d0990 2024-04-04T06:18:29.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.158+0000 7f9c5ea00640 1 -- 172.21.15.159:0/1046216013 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c50007820 con 0x7f9c680d0990 2024-04-04T06:18:29.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.158+0000 7f9c5ea00640 1 -- 172.21.15.159:0/1046216013 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50007bf0 con 0x7f9c680d0990 2024-04-04T06:18:29.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.159+0000 7f9c6d200640 1 -- 172.21.15.159:0/1046216013 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c6809a470 con 0x7f9c680d0990 2024-04-04T06:18:29.155 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.159+0000 7f9c6d200640 1 -- 172.21.15.159:0/1046216013 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c6809aa30 con 0x7f9c680d0990 2024-04-04T06:18:29.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.160+0000 7f9c6d200640 1 -- 172.21.15.159:0/1046216013 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c680685c0 con 0x7f9c680d0990 2024-04-04T06:18:29.156 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.161+0000 7f9c5ea00640 1 -- 172.21.15.159:0/1046216013 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c50018070 con 0x7f9c680d0990 2024-04-04T06:18:29.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.162+0000 7f9c5ea00640 1 --2- 172.21.15.159:0/1046216013 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9c4406c920 0x7f9c4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:29.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.162+0000 7f9c5ea00640 1 -- 172.21.15.159:0/1046216013 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f9c5008ba70 con 0x7f9c680d0990 2024-04-04T06:18:29.157 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.162+0000 7f9c65e00640 1 --2- 172.21.15.159:0/1046216013 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9c4406c920 0x7f9c4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:29.158 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.163+0000 7f9c65e00640 1 --2- 172.21.15.159:0/1046216013 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9c4406c920 0x7f9c4406ede0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9c58000f60 tx=0x7f9c58009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:29.161 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.166+0000 7f9c5ea00640 1 -- 172.21.15.159:0/1046216013 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f9c50059fc0 con 0x7f9c680d0990 2024-04-04T06:18:29.359 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.364+0000 7f9c6d200640 1 -- 172.21.15.159:0/1046216013 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f9c680d0590 con 0x7f9c680d0990 2024-04-04T06:18:29.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.369+0000 7f9c5ea00640 1 -- 172.21.15.159:0/1046216013 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) ==== 123+0+0 (secure 0 0 0) 0x7f9c500598c0 con 0x7f9c680d0990 2024-04-04T06:18:29.364 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.370+0000 7f9c5ea00640 1 -- 172.21.15.159:0/1046216013 <== mon.0 v2:172.21.15.159:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9c50091080 con 0x7f9c680d0990 2024-04-04T06:18:29.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.372+0000 7f9c6d200640 1 -- 172.21.15.159:0/1046216013 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9c4406c920 msgr2=0x7f9c4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:29.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.372+0000 7f9c6d200640 1 --2- 172.21.15.159:0/1046216013 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9c4406c920 0x7f9c4406ede0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9c58000f60 tx=0x7f9c58009040 comp rx=0 tx=0).stop 2024-04-04T06:18:29.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.372+0000 7f9c6d200640 1 -- 172.21.15.159:0/1046216013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c680d0990 msgr2=0x7f9c680cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:29.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.372+0000 7f9c6d200640 1 --2- 172.21.15.159:0/1046216013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c680d0990 0x7f9c680cd4f0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f9c5000c7f0 tx=0x7f9c5000ccb0 comp rx=0 tx=0).stop 2024-04-04T06:18:29.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.372+0000 7f9c6d200640 1 -- 172.21.15.159:0/1046216013 shutdown_connections 2024-04-04T06:18:29.368 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.372+0000 7f9c6d200640 1 --2- 172.21.15.159:0/1046216013 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f9c4406c920 0x7f9c4406ede0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:29.369 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.372+0000 7f9c6d200640 1 --2- 172.21.15.159:0/1046216013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c68153580 0x7f9c680cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:29.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.372+0000 7f9c6d200640 1 --2- 172.21.15.159:0/1046216013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c680d0990 0x7f9c680cd4f0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:29.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.372+0000 7f9c6d200640 1 --2- 172.21.15.159:0/1046216013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9c680af2e0 0x7f9c681525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:29.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.372+0000 7f9c6d200640 1 -- 172.21.15.159:0/1046216013 >> 172.21.15.159:0/1046216013 conn(0x7f9c6806b750 msgr2=0x7f9c68071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:29.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.373+0000 7f9c6d200640 1 -- 172.21.15.159:0/1046216013 shutdown_connections 2024-04-04T06:18:29.370 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.373+0000 7f9c6d200640 1 -- 172.21.15.159:0/1046216013 wait complete. 2024-04-04T06:18:29.385 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-04T06:18:29.386 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-04T06:18:29.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.600+0000 7f885a000640 1 Processor -- start 2024-04-04T06:18:29.599 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.600+0000 7f885a000640 1 -- start start 2024-04-04T06:18:29.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.601+0000 7f885a000640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f88540cf1b0 0x7f88540cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:29.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.601+0000 7f885a000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88540d00d0 0x7f88540b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:29.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.601+0000 7f885a000640 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8854071460 con 0x7f88541442c0 2024-04-04T06:18:29.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.601+0000 7f885a000640 1 -- --> v2:172.21.15.159:3300/0 -- mon_getmap magic: 0 -- 0x7f8854071d20 con 0x7f88540cf1b0 2024-04-04T06:18:29.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.601+0000 7f885a000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f88540721b0 con 0x7f88540d00d0 2024-04-04T06:18:29.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.602+0000 7f884be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f88540cf1b0 0x7f88540cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:29.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.602+0000 7f884be00640 1 --2- >> v2:172.21.15.159:3300/0 conn(0x7f88540cf1b0 0x7f88540cfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38074/0 (socket says 172.21.15.159:38074) 2024-04-04T06:18:29.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.602+0000 7f884be00640 1 -- 172.21.15.159:0/1022767930 learned_addr learned my addr 172.21.15.159:0/1022767930 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:29.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.602+0000 7f8859600640 1 --2- 172.21.15.159:0/1022767930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88540d00d0 0x7f88540b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:29.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.602+0000 7f884be00640 1 -- 172.21.15.159:0/1022767930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88540d00d0 msgr2=0x7f88540b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:29.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.602+0000 7f884be00640 1 --2- 172.21.15.159:0/1022767930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88540d00d0 0x7f88540b3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:29.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.602+0000 7f884be00640 1 -- 172.21.15.159:0/1022767930 >> v1:172.21.15.178:6789/0 conn(0x7f88541442c0 legacy=0x7f88540ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-04T06:18:29.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.602+0000 7f884be00640 1 -- 172.21.15.159:0/1022767930 --> v2:172.21.15.159:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88540716d0 con 0x7f88540cf1b0 2024-04-04T06:18:29.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.603+0000 7f884be00640 1 --2- 172.21.15.159:0/1022767930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88540cf1b0 0x7f88540cfb90 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f883c009870 tx=0x7f883c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ad299f8a5bf5b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:29.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.603+0000 7f8853e00640 1 -- 172.21.15.159:0/1022767930 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c034070 con 0x7f88540cf1b0 2024-04-04T06:18:29.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.604+0000 7f8853e00640 1 -- 172.21.15.159:0/1022767930 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f883c030840 con 0x7f88540cf1b0 2024-04-04T06:18:29.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.604+0000 7f8853e00640 1 -- 172.21.15.159:0/1022767930 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c030c30 con 0x7f88540cf1b0 2024-04-04T06:18:29.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.604+0000 7f885a000640 1 -- 172.21.15.159:0/1022767930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88540cf1b0 msgr2=0x7f88540cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:29.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.604+0000 7f885a000640 1 --2- 172.21.15.159:0/1022767930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88540cf1b0 0x7f88540cfb90 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f883c009870 tx=0x7f883c02b980 comp rx=0 tx=0).stop 2024-04-04T06:18:29.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.604+0000 7f885a000640 1 -- 172.21.15.159:0/1022767930 shutdown_connections 2024-04-04T06:18:29.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.604+0000 7f885a000640 1 --2- 172.21.15.159:0/1022767930 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88540d00d0 0x7f88540b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:29.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.604+0000 7f885a000640 1 --2- 172.21.15.159:0/1022767930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88540cf1b0 0x7f88540cfb90 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:29.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.604+0000 7f885a000640 1 -- 172.21.15.159:0/1022767930 >> 172.21.15.159:0/1022767930 conn(0x7f88540ccad0 msgr2=0x7f88540cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:29.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.604+0000 7f885a000640 1 -- 172.21.15.159:0/1022767930 shutdown_connections 2024-04-04T06:18:29.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.604+0000 7f885a000640 1 -- 172.21.15.159:0/1022767930 wait complete. 2024-04-04T06:18:29.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.606+0000 7f885a000640 1 Processor -- start 2024-04-04T06:18:29.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.606+0000 7f885a000640 1 -- start start 2024-04-04T06:18:29.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.607+0000 7f885a000640 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88540cf1b0 0x7f885415d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:29.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.607+0000 7f885a000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88540d00d0 0x7f885415d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:29.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.607+0000 7f885a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88541442c0 0x7f8854159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:29.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.607+0000 7f885a000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f88541446a0 con 0x7f88541442c0 2024-04-04T06:18:29.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.607+0000 7f885a000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f88540d0770 con 0x7f88540d00d0 2024-04-04T06:18:29.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.607+0000 7f885a000640 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 -- 0x7f8854157990 con 0x7f88540cf1b0 2024-04-04T06:18:29.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.607+0000 7f8859600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88541442c0 0x7f8854159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:29.604 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.607+0000 7f8859600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88541442c0 0x7f8854159960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:38080/0 (socket says 172.21.15.159:38080) 2024-04-04T06:18:29.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.607+0000 7f8859600640 1 -- 172.21.15.159:0/2124204791 learned_addr learned my addr 172.21.15.159:0/2124204791 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-04-04T06:18:29.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.608+0000 7f8858c00640 1 --2- 172.21.15.159:0/2124204791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88540cf1b0 0x7f885415d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:29.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.608+0000 7f884be00640 1 --2- 172.21.15.159:0/2124204791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88540d00d0 0x7f885415d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:29.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.608+0000 7f8859600640 1 -- 172.21.15.159:0/2124204791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88540cf1b0 msgr2=0x7f885415d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:29.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.608+0000 7f8859600640 1 --2- 172.21.15.159:0/2124204791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88540cf1b0 0x7f885415d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:29.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.608+0000 7f8859600640 1 -- 172.21.15.159:0/2124204791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88540d00d0 msgr2=0x7f885415d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:29.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.608+0000 7f8859600640 1 --2- 172.21.15.159:0/2124204791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88540d00d0 0x7f885415d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:29.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.608+0000 7f8859600640 1 -- 172.21.15.159:0/2124204791 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885413bbc0 con 0x7f88541442c0 2024-04-04T06:18:29.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.608+0000 7f8858c00640 1 --2- 172.21.15.159:0/2124204791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88540cf1b0 0x7f885415d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:18:29.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.608+0000 7f884be00640 1 --2- 172.21.15.159:0/2124204791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88540d00d0 0x7f885415d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-04T06:18:29.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.608+0000 7f8859600640 1 --2- 172.21.15.159:0/2124204791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88541442c0 0x7f8854159960 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f884c00cd70 tx=0x7f884c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:29.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.608+0000 7f8851600640 1 -- 172.21.15.159:0/2124204791 <== mon.0 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c003b90 con 0x7f88541442c0 2024-04-04T06:18:29.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.608+0000 7f8851600640 1 -- 172.21.15.159:0/2124204791 <== mon.0 v2:172.21.15.159:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f884c003d30 con 0x7f88541442c0 2024-04-04T06:18:29.605 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.608+0000 7f885a000640 1 -- 172.21.15.159:0/2124204791 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885415dcd0 con 0x7f88541442c0 2024-04-04T06:18:29.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.608+0000 7f8851600640 1 -- 172.21.15.159:0/2124204791 <== mon.0 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c004260 con 0x7f88541442c0 2024-04-04T06:18:29.606 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.609+0000 7f885a000640 1 -- 172.21.15.159:0/2124204791 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8854157340 con 0x7f88541442c0 2024-04-04T06:18:29.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.610+0000 7f885a000640 1 -- 172.21.15.159:0/2124204791 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8854075940 con 0x7f88541442c0 2024-04-04T06:18:29.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.610+0000 7f8851600640 1 -- 172.21.15.159:0/2124204791 <== mon.0 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f884c004400 con 0x7f88541442c0 2024-04-04T06:18:29.607 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.611+0000 7f8851600640 1 --2- 172.21.15.159:0/2124204791 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f883006c8f0 0x7f883006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-04T06:18:29.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.611+0000 7f8851600640 1 -- 172.21.15.159:0/2124204791 <== mon.0 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f884c08c530 con 0x7f88541442c0 2024-04-04T06:18:29.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.612+0000 7f8858c00640 1 --2- 172.21.15.159:0/2124204791 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f883006c8f0 0x7f883006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-04T06:18:29.608 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.613+0000 7f8858c00640 1 --2- 172.21.15.159:0/2124204791 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f883006c8f0 0x7f883006edb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8844003e50 tx=0x7f8844009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-04T06:18:29.611 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.616+0000 7f8851600640 1 -- 172.21.15.159:0/2124204791 <== mon.0 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187049 (secure 0 0 0) 0x7f884c05aa80 con 0x7f88541442c0 2024-04-04T06:18:29.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.857+0000 7f885a000640 1 -- 172.21.15.159:0/2124204791 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88540b0890 con 0x7f88541442c0 2024-04-04T06:18:29.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.858+0000 7f8851600640 1 -- 172.21.15.159:0/2124204791 <== mon.0 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+3883 (secure 0 0 0) 0x7f884c05a380 con 0x7f88541442c0 2024-04-04T06:18:29.854 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-04-04T06:18:29.854 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":116,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5732,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/1498324773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6835","nonce":1498324773},{"type":"v1","addr":"172.21.15.178:6837","nonce":1498324773}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/2455031662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6833","nonce":2455031662},{"type":"v1","addr":"172.21.15.159:6835","nonce":2455031662}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":6343,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6836/1823228151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1823228151},{"type":"v1","addr":"172.21.15.178:6836","nonce":1823228151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":9537,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6834/1023874803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1023874803},{"type":"v1","addr":"172.21.15.159:6834","nonce":1023874803}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[]} 2024-04-04T06:18:29.854 INFO:teuthology.orchestra.run.smithi159.stderr:dumped fsmap epoch 116 2024-04-04T06:18:29.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.861+0000 7f885a000640 1 -- 172.21.15.159:0/2124204791 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f883006c8f0 msgr2=0x7f883006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:29.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.861+0000 7f885a000640 1 --2- 172.21.15.159:0/2124204791 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f883006c8f0 0x7f883006edb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8844003e50 tx=0x7f8844009040 comp rx=0 tx=0).stop 2024-04-04T06:18:29.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.861+0000 7f885a000640 1 -- 172.21.15.159:0/2124204791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88541442c0 msgr2=0x7f8854159960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-04T06:18:29.856 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.861+0000 7f885a000640 1 --2- 172.21.15.159:0/2124204791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88541442c0 0x7f8854159960 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f884c00cd70 tx=0x7f884c00a660 comp rx=0 tx=0).stop 2024-04-04T06:18:29.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.861+0000 7f885a000640 1 -- 172.21.15.159:0/2124204791 shutdown_connections 2024-04-04T06:18:29.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.861+0000 7f885a000640 1 --2- 172.21.15.159:0/2124204791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88541442c0 0x7f8854159960 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:29.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.861+0000 7f885a000640 1 --2- 172.21.15.159:0/2124204791 >> [v2:172.21.15.178:6800/3272374747,v1:172.21.15.178:6801/3272374747] conn(0x7f883006c8f0 0x7f883006edb0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:29.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.861+0000 7f885a000640 1 --2- 172.21.15.159:0/2124204791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88540d00d0 0x7f885415d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:29.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.861+0000 7f885a000640 1 --2- 172.21.15.159:0/2124204791 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f88540cf1b0 0x7f885415d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-04T06:18:29.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.861+0000 7f885a000640 1 -- 172.21.15.159:0/2124204791 >> 172.21.15.159:0/2124204791 conn(0x7f88540ccad0 msgr2=0x7f88540b23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-04T06:18:29.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.861+0000 7f885a000640 1 -- 172.21.15.159:0/2124204791 shutdown_connections 2024-04-04T06:18:29.857 INFO:teuthology.orchestra.run.smithi159.stderr:2024-04-04T06:18:29.861+0000 7f885a000640 1 -- 172.21.15.159:0/2124204791 wait complete. 2024-04-04T06:18:29.876 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-04T06:18:29.876 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-04T06:18:29.876 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-04T06:18:29.877 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:18:29.893 INFO:tasks.ceph.mds.a:Stopped 2024-04-04T06:18:29.893 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-04T06:18:29.893 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:18:29.957 INFO:tasks.ceph.mds.c:Stopped 2024-04-04T06:18:29.958 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-04T06:18:29.958 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:18:29.978 INFO:tasks.ceph.mds.b:Stopped 2024-04-04T06:18:29.978 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-04T06:18:29.978 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:18:30.049 INFO:tasks.ceph.mds.d:Stopped 2024-04-04T06:18:30.049 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-04T06:18:30.049 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-04T06:18:30.049 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:18:30.138 INFO:tasks.ceph.osd.0:Stopped 2024-04-04T06:18:30.138 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-04T06:18:30.138 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:18:30.217 INFO:tasks.ceph.osd.1:Stopped 2024-04-04T06:18:30.217 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-04T06:18:30.217 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:18:30.281 INFO:tasks.ceph.osd.2:Stopped 2024-04-04T06:18:30.282 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-04T06:18:30.282 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:18:30.364 INFO:tasks.ceph.osd.3:Stopped 2024-04-04T06:18:30.365 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-04T06:18:30.365 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:18:30.430 INFO:tasks.ceph.osd.4:Stopped 2024-04-04T06:18:30.430 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-04T06:18:30.431 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:18:30.491 INFO:tasks.ceph.osd.5:Stopped 2024-04-04T06:18:30.492 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-04T06:18:30.492 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:18:30.542 INFO:tasks.ceph.osd.6:Stopped 2024-04-04T06:18:30.542 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-04T06:18:30.542 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:18:30.627 INFO:tasks.ceph.osd.7:Stopped 2024-04-04T06:18:30.627 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-04T06:18:30.627 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-04T06:18:30.627 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:18:35.400 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-04-04T06:18:36.729 INFO:tasks.ceph.mgr.y:Stopped 2024-04-04T06:18:36.730 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-04T06:18:36.730 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:18:36.813 INFO:tasks.ceph.mgr.x:Stopped 2024-04-04T06:18:36.813 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-04T06:18:36.813 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-04T06:18:36.814 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:18:36.850 INFO:tasks.ceph.mon.a:Stopped 2024-04-04T06:18:36.851 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-04T06:18:36.851 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:18:36.893 INFO:tasks.ceph.mon.b:Stopped 2024-04-04T06:18:36.894 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-04T06:18:36.894 INFO:teuthology.orchestra.run:waiting for 300 2024-04-04T06:18:36.980 INFO:tasks.ceph.mon.c:Stopped 2024-04-04T06:18:36.980 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-04T06:18:36.980 DEBUG:teuthology.orchestra.run.smithi159:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-04T06:18:37.091 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:11:05.580204+0000 mds.a (mds.0) 1 : cluster [WRN] Corrupt dentry 'file_to_be_damaged' in dir frag 0x10000000000: Invalid tag char 'd' pos 0: Malformed input [buffer:3](/subdir) 2024-04-04T06:18:37.093 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-04-04T06:18:37.093 DEBUG:teuthology.orchestra.run.smithi159:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-04T06:18:37.191 DEBUG:teuthology.orchestra.run.smithi159:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-04T06:18:37.278 INFO:teuthology.orchestra.run.smithi159.stdout:2024-04-04T06:12:57.115301+0000 mds.a (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-04T06:18:37.279 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:18:37.279 DEBUG:teuthology.orchestra.run.smithi159:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-04T06:18:37.523 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:18:37.524 DEBUG:teuthology.orchestra.run.smithi159:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-04T06:18:37.678 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:18:37.678 DEBUG:teuthology.orchestra.run.smithi159:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-04T06:18:37.831 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:18:37.832 DEBUG:teuthology.orchestra.run.smithi159:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-04T06:18:38.026 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-04T06:18:38.026 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-04T06:18:38.319 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-04T06:18:38.319 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-04T06:18:38.430 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-04T06:18:38.430 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-04T06:18:38.596 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-04T06:18:38.596 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-04T06:18:38.757 INFO:tasks.ceph:Archiving mon data... 2024-04-04T06:18:38.758 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2024-04-04_05:15:36-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi/7639210/data/mon.a.tgz 2024-04-04T06:18:38.758 DEBUG:teuthology.orchestra.run.smithi159:> mktemp 2024-04-04T06:18:38.815 INFO:teuthology.orchestra.run.smithi159.stdout:/tmp/tmp.4wlIwlp2Lq 2024-04-04T06:18:38.816 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.4wlIwlp2Lq 2024-04-04T06:18:39.011 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 0666 /tmp/tmp.4wlIwlp2Lq 2024-04-04T06:18:39.163 DEBUG:teuthology.orchestra.remote:smithi159:/tmp/tmp.4wlIwlp2Lq is 446KB 2024-04-04T06:18:39.275 DEBUG:teuthology.orchestra.run.smithi159:> rm -fr /tmp/tmp.4wlIwlp2Lq 2024-04-04T06:18:39.332 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2024-04-04_05:15:36-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi/7639210/data/mon.b.tgz 2024-04-04T06:18:39.332 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-04-04T06:18:39.389 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.rRGQ80BVjO 2024-04-04T06:18:39.389 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.rRGQ80BVjO 2024-04-04T06:18:39.599 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.rRGQ80BVjO 2024-04-04T06:18:39.746 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.rRGQ80BVjO is 463KB 2024-04-04T06:18:39.852 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.rRGQ80BVjO 2024-04-04T06:18:39.909 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2024-04-04_05:15:36-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi/7639210/data/mon.c.tgz 2024-04-04T06:18:39.909 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-04-04T06:18:39.958 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.5A1mnq3Mp2 2024-04-04T06:18:39.959 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.5A1mnq3Mp2 2024-04-04T06:18:40.156 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.5A1mnq3Mp2 2024-04-04T06:18:40.306 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.5A1mnq3Mp2 is 463KB 2024-04-04T06:18:40.401 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-04-04T06:18:40.416 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.5A1mnq3Mp2 2024-04-04T06:18:40.472 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-04T06:18:40.473 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-04T06:18:40.480 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-04T06:18:40.629 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-04T06:18:40.684 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-04T06:18:40.684 INFO:tasks.ceph:Archiving crash dumps... 2024-04-04T06:18:40.686 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-04-04_05:15:36-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi/7639210/remote/smithi159/crash 2024-04-04T06:18:40.687 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-04T06:18:40.784 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-04-04_05:15:36-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi/7639210/remote/smithi178/crash 2024-04-04T06:18:40.785 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-04T06:18:40.880 INFO:tasks.ceph:Shutting down logrotate 2024-04-04T06:18:40.881 DEBUG:teuthology.orchestra.run.smithi159:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:18:40.888 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-04T06:18:41.045 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-04T06:18:41.145 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-04T06:18:41.244 INFO:tasks.ceph:Compressing logs... 2024-04-04T06:18:41.245 DEBUG:teuthology.orchestra.run.smithi159:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-04T06:18:41.252 DEBUG:teuthology.orchestra.run.smithi178:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-04T06:18:41.343 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37102.log 2024-04-04T06:18:41.345 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60589.log 2024-04-04T06:18:41.345 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.37102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50115.log 2024-04-04T06:18:41.346 INFO:teuthology.orchestra.run.smithi159.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37102.log.gz 2024-04-04T06:18:41.346 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.60589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73635.log 2024-04-04T06:18:41.346 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60589.log.gz 2024-04-04T06:18:41.347 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-04T06:18:41.347 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41350.log 2024-04-04T06:18:41.348 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.50115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73125.log 2024-04-04T06:18:41.349 INFO:teuthology.orchestra.run.smithi159.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50115.log.gz 2024-04-04T06:18:41.349 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.73635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66175.log 2024-04-04T06:18:41.349 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73635.log.gz 2024-04-04T06:18:41.349 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.73125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62750.log 2024-04-04T06:18:41.349 INFO:teuthology.orchestra.run.smithi159.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73125.log.gz 2024-04-04T06:18:41.354 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37075.log 2024-04-04T06:18:41.354 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.1.41350.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41350.log.gz 2024-04-04T06:18:41.354 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36906.log 2024-04-04T06:18:41.354 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37075.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-04T06:18:41.354 INFO:teuthology.orchestra.run.smithi178.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37075.log.gz 2024-04-04T06:18:41.355 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36906.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36906.log.gz 2024-04-04T06:18:41.355 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-04T06:18:41.355 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-04T06:18:41.355 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36866.log 2024-04-04T06:18:41.356 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.66175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69627.log 2024-04-04T06:18:41.356 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66175.log.gz 2024-04-04T06:18:41.357 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61310.log 2024-04-04T06:18:41.357 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62750.log.gz 2024-04-04T06:18:41.357 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.69627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79158.log 2024-04-04T06:18:41.357 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69627.log.gz 2024-04-04T06:18:41.357 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48183.log 2024-04-04T06:18:41.357 INFO:teuthology.orchestra.run.smithi159.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61310.log.gz 2024-04-04T06:18:41.357 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.79158.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-04T06:18:41.357 INFO:teuthology.orchestra.run.smithi159.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.79158.log.gz 2024-04-04T06:18:41.357 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54253.log 2024-04-04T06:18:41.357 INFO:teuthology.orchestra.run.smithi159.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48183.log.gz 2024-04-04T06:18:41.357 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79349.log 2024-04-04T06:18:41.357 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66919.log 2024-04-04T06:18:41.357 INFO:teuthology.orchestra.run.smithi159.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54253.log.gz 2024-04-04T06:18:41.359 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36946.log 2024-04-04T06:18:41.359 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36866.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-04T06:18:41.359 INFO:teuthology.orchestra.run.smithi178.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36866.log.gz 2024-04-04T06:18:41.359 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36946.log: 93.8%gzip -- replaced with /var/log/ceph/ceph.audit.log.gz -5 2024-04-04T06:18:41.359 INFO:teuthology.orchestra.run.smithi178.stderr: --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-04T06:18:41.359 INFO:teuthology.orchestra.run.smithi178.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36946.log.gz 2024-04-04T06:18:41.359 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41369.log 2024-04-04T06:18:41.362 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.79349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62665.log 2024-04-04T06:18:41.362 INFO:teuthology.orchestra.run.smithi159.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79349.log.gz 2024-04-04T06:18:41.362 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.66919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43810.log 2024-04-04T06:18:41.362 INFO:teuthology.orchestra.run.smithi159.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66919.log.gz 2024-04-04T06:18:41.362 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56284.log 2024-04-04T06:18:41.362 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62665.log.gz 2024-04-04T06:18:41.362 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54974.log 2024-04-04T06:18:41.362 INFO:teuthology.orchestra.run.smithi159.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43810.log.gz 2024-04-04T06:18:41.362 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.56284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70994.log 2024-04-04T06:18:41.364 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37117.log 2024-04-04T06:18:41.364 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.1.41369.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41369.log.gz 2024-04-04T06:18:41.364 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-04T06:18:41.364 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41768.log 2024-04-04T06:18:41.364 INFO:teuthology.orchestra.run.smithi178.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37117.log.gz 2024-04-04T06:18:41.365 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54974.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56284.log.gz 2024-04-04T06:18:41.365 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50809.log 2024-04-04T06:18:41.365 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54974.log.gz 2024-04-04T06:18:41.365 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.70994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80105.log 2024-04-04T06:18:41.365 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70994.log.gz 2024-04-04T06:18:41.365 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.50809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69525.log 2024-04-04T06:18:41.367 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-04T06:18:41.367 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.3.41768.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41768.log.gz 2024-04-04T06:18:41.367 INFO:teuthology.orchestra.run.smithi159.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50809.log.gz 2024-04-04T06:18:41.367 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.80105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69711.log 2024-04-04T06:18:41.367 INFO:teuthology.orchestra.run.smithi159.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80105.log.gz 2024-04-04T06:18:41.368 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.69525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42428.log 2024-04-04T06:18:41.368 INFO:teuthology.orchestra.run.smithi159.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.69525.log.gz 2024-04-04T06:18:41.368 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.69711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50893.log 2024-04-04T06:18:41.369 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37026.log 2024-04-04T06:18:41.370 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42428.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.69711.log.gz 2024-04-04T06:18:41.370 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57060.log 2024-04-04T06:18:41.370 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42428.log.gz 2024-04-04T06:18:41.370 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.50893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72812.log 2024-04-04T06:18:41.370 INFO:teuthology.orchestra.run.smithi159.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50893.log.gz 2024-04-04T06:18:41.371 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71198.log 2024-04-04T06:18:41.372 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57060.log.gz 2024-04-04T06:18:41.372 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.72812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64504.log 2024-04-04T06:18:41.373 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41764.log 2024-04-04T06:18:41.373 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37026.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-04T06:18:41.373 INFO:teuthology.orchestra.run.smithi178.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37026.log.gz 2024-04-04T06:18:41.373 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36986.log 2024-04-04T06:18:41.374 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72812.log.gz 2024-04-04T06:18:41.374 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.71198.log: gzip -5 88.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.71198.log.gz /var/log/ceph/ceph-client.0.67125.log 2024-04-04T06:18:41.374 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-04-04T06:18:41.374 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47259.log 2024-04-04T06:18:41.374 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64504.log.gz 2024-04-04T06:18:41.375 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.67125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71120.log 2024-04-04T06:18:41.375 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67125.log.gz 2024-04-04T06:18:41.377 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76858.log 2024-04-04T06:18:41.377 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47259.log.gz 2024-04-04T06:18:41.378 INFO:teuthology.orchestra.run.smithi178.stderr: 92.5%/var/log/ceph/ceph-client.admin.36986.log: /var/log/ceph/ceph-client.3.41764.log: /var/log/ceph/ceph-mon.b.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.3.41764.log.gz 2024-04-04T06:18:41.378 INFO:teuthology.orchestra.run.smithi178.stderr: -5 --verbose -- -- replaced with /var/log/ceph/ceph.log.gz /var/log/ceph/ceph-client.admin.36826.log 2024-04-04T06:18:41.378 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-04-04T06:18:41.378 INFO:teuthology.orchestra.run.smithi178.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36986.log.gz 2024-04-04T06:18:41.379 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.71120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48141.log 2024-04-04T06:18:41.379 INFO:teuthology.orchestra.run.smithi159.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71120.log.gz 2024-04-04T06:18:41.379 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.76858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58292.log 2024-04-04T06:18:41.380 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76858.log.gz 2024-04-04T06:18:41.380 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51289.log 2024-04-04T06:18:41.380 INFO:teuthology.orchestra.run.smithi159.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48141.log.gz 2024-04-04T06:18:41.381 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46170.log 2024-04-04T06:18:41.381 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58292.log.gz 2024-04-04T06:18:41.382 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-04T06:18:41.382 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.36826.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36826.log.gz 2024-04-04T06:18:41.384 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.51289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43154.log 2024-04-04T06:18:41.384 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51289.log.gz 2024-04-04T06:18:41.384 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66566.log 2024-04-04T06:18:41.384 INFO:teuthology.orchestra.run.smithi159.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46170.log.gz 2024-04-04T06:18:41.384 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63696.log 2024-04-04T06:18:41.384 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43154.log.gz 2024-04-04T06:18:41.385 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.66566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48015.log 2024-04-04T06:18:41.385 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72086.log 2024-04-04T06:18:41.386 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63696.log.gz 2024-04-04T06:18:41.386 INFO:teuthology.orchestra.run.smithi159.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.66566.log.gz 2024-04-04T06:18:41.387 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-04T06:18:41.387 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62049.log 2024-04-04T06:18:41.387 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48015.log.gz 2024-04-04T06:18:41.388 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.72086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68098.log 2024-04-04T06:18:41.388 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62049.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.72086.log.gz 2024-04-04T06:18:41.389 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44234.log 2024-04-04T06:18:41.389 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62049.log.gz 2024-04-04T06:18:41.390 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.68098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56650.log 2024-04-04T06:18:41.390 INFO:teuthology.orchestra.run.smithi159.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68098.log.gz 2024-04-04T06:18:41.391 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79769.log 2024-04-04T06:18:41.391 INFO:teuthology.orchestra.run.smithi159.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44234.log.gz 2024-04-04T06:18:41.392 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.56650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64793.log 2024-04-04T06:18:41.393 INFO:teuthology.orchestra.run.smithi159.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56650.log.gz 2024-04-04T06:18:41.393 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.79769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65165.log 2024-04-04T06:18:41.393 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79769.log.gz 2024-04-04T06:18:41.394 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48554.log 2024-04-04T06:18:41.394 INFO:teuthology.orchestra.run.smithi159.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.64793.log.gz 2024-04-04T06:18:41.395 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.65165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60673.log 2024-04-04T06:18:41.395 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65165.log.gz 2024-04-04T06:18:41.396 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51714.log 2024-04-04T06:18:41.396 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48554.log.gz 2024-04-04T06:18:41.397 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.60673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60715.log 2024-04-04T06:18:41.397 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60673.log.gz 2024-04-04T06:18:41.398 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.51714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50851.log 2024-04-04T06:18:41.398 INFO:teuthology.orchestra.run.smithi159.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51714.log.gz 2024-04-04T06:18:41.399 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.60715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66301.log 2024-04-04T06:18:41.399 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60715.log.gz 2024-04-04T06:18:41.399 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mon.c.log: 92.1% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-04T06:18:41.400 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.50851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68495.log 2024-04-04T06:18:41.401 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50851.log.gz 2024-04-04T06:18:41.401 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.66301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59870.log 2024-04-04T06:18:41.401 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66301.log.gz 2024-04-04T06:18:41.402 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.68495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55325.log 2024-04-04T06:18:41.402 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68495.log.gz 2024-04-04T06:18:41.403 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.59870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79685.log 2024-04-04T06:18:41.403 INFO:teuthology.orchestra.run.smithi159.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.59870.log.gz 2024-04-04T06:18:41.404 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.55325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79475.log 2024-04-04T06:18:41.404 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55325.log.gz 2024-04-04T06:18:41.405 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.79685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44024.log 2024-04-04T06:18:41.405 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79685.log.gz 2024-04-04T06:18:41.406 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.79475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71036.log 2024-04-04T06:18:41.406 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79475.log.gz 2024-04-04T06:18:41.407 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53576.log 2024-04-04T06:18:41.407 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44024.log.gz 2024-04-04T06:18:41.408 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.71036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60859.log 2024-04-04T06:18:41.408 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71036.log.gz 2024-04-04T06:18:41.409 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52021.log 2024-04-04T06:18:41.409 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53576.log.gz 2024-04-04T06:18:41.410 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.60859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54211.log 2024-04-04T06:18:41.411 INFO:teuthology.orchestra.run.smithi159.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.60859.log.gz 2024-04-04T06:18:41.411 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77644.log 2024-04-04T06:18:41.411 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52021.log.gz 2024-04-04T06:18:41.412 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57929.log 2024-04-04T06:18:41.412 INFO:teuthology.orchestra.run.smithi159.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54211.log.gz 2024-04-04T06:18:41.413 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.77644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58540.log 2024-04-04T06:18:41.413 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77644.log.gz 2024-04-04T06:18:41.414 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79727.log 2024-04-04T06:18:41.414 INFO:teuthology.orchestra.run.smithi159.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57929.log.gz 2024-04-04T06:18:41.415 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61732.log 2024-04-04T06:18:41.415 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58540.log.gz 2024-04-04T06:18:41.416 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.79727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74198.log 2024-04-04T06:18:41.416 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79727.log.gz 2024-04-04T06:18:41.417 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.61732.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61732.log.gz 2024-04-04T06:18:41.417 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63822.log 2024-04-04T06:18:41.418 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.74198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64462.log 2024-04-04T06:18:41.418 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74198.log.gz 2024-04-04T06:18:41.419 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44462.log 2024-04-04T06:18:41.419 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63822.log.gz 2024-04-04T06:18:41.420 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56542.log 2024-04-04T06:18:41.420 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64462.log.gz 2024-04-04T06:18:41.421 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44462.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44462.log.gz 2024-04-04T06:18:41.421 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61488.log 2024-04-04T06:18:41.422 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.56542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57658.log 2024-04-04T06:18:41.422 INFO:teuthology.orchestra.run.smithi159.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.56542.log.gz 2024-04-04T06:18:41.423 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52151.log 2024-04-04T06:18:41.423 INFO:teuthology.orchestra.run.smithi159.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61488.log.gz 2024-04-04T06:18:41.424 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65819.log 2024-04-04T06:18:41.424 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57658.log.gz 2024-04-04T06:18:41.425 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47433.log 2024-04-04T06:18:41.425 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52151.log.gz 2024-04-04T06:18:41.426 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.65819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63343.log 2024-04-04T06:18:41.427 INFO:teuthology.orchestra.run.smithi159.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.65819.log.gz 2024-04-04T06:18:41.427 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66483.log 2024-04-04T06:18:41.427 INFO:teuthology.orchestra.run.smithi159.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47433.log.gz 2024-04-04T06:18:41.428 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62437.log 2024-04-04T06:18:41.428 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63343.log.gz 2024-04-04T06:18:41.429 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.66483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48723.log 2024-04-04T06:18:41.429 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66483.log.gz 2024-04-04T06:18:41.430 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73862.log 2024-04-04T06:18:41.430 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62437.log.gz 2024-04-04T06:18:41.431 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48723.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48723.log.gz 2024-04-04T06:18:41.431 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56584.log 2024-04-04T06:18:41.432 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.73862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78887.log 2024-04-04T06:18:41.433 INFO:teuthology.orchestra.run.smithi159.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73862.log.gz 2024-04-04T06:18:41.433 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.56584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57760.log 2024-04-04T06:18:41.434 INFO:teuthology.orchestra.run.smithi159.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56584.log.gz 2024-04-04T06:18:41.434 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.78887.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78887.log.gz 2024-04-04T06:18:41.435 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44630.log 2024-04-04T06:18:41.436 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45750.log 2024-04-04T06:18:41.436 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57760.log.gz 2024-04-04T06:18:41.437 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43826.log 2024-04-04T06:18:41.437 INFO:teuthology.orchestra.run.smithi159.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44630.log.gz 2024-04-04T06:18:41.438 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.45750.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45750.log.gz 2024-04-04T06:18:41.438 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46590.log 2024-04-04T06:18:41.439 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50421.log 2024-04-04T06:18:41.439 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43826.log.gz 2024-04-04T06:18:41.440 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49755.log 2024-04-04T06:18:41.440 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46590.log.gz 2024-04-04T06:18:41.441 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.50421.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.50421.log.gz 2024-04-04T06:18:41.441 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58679.log 2024-04-04T06:18:41.442 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.49755.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49755.log.gz 2024-04-04T06:18:41.442 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44504.log 2024-04-04T06:18:41.444 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58679.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58679.log.gz 2024-04-04T06:18:41.444 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43146.log 2024-04-04T06:18:41.445 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44504.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44504.log.gz 2024-04-04T06:18:41.445 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46380.log 2024-04-04T06:18:41.446 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53003.log 2024-04-04T06:18:41.446 INFO:teuthology.orchestra.run.smithi159.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43146.log.gz 2024-04-04T06:18:41.448 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44192.log 2024-04-04T06:18:41.448 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46380.log.gz 2024-04-04T06:18:41.448 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66723.log 2024-04-04T06:18:41.448 INFO:teuthology.orchestra.run.smithi159.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53003.log.gz 2024-04-04T06:18:41.449 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54505.log 2024-04-04T06:18:41.449 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44192.log.gz 2024-04-04T06:18:41.450 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.66723.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66723.log.gz 2024-04-04T06:18:41.450 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73209.log 2024-04-04T06:18:41.451 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54505.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.54505.log.gz 2024-04-04T06:18:41.451 INFO:teuthology.orchestra.run.smithi159.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54930.log 2024-04-04T06:18:41.452 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.73209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57322.log 2024-04-04T06:18:41.452 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73209.log.gz 2024-04-04T06:18:41.453 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54930.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54930.log.gz 2024-04-04T06:18:41.453 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67357.log 2024-04-04T06:18:41.454 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56815.log 2024-04-04T06:18:41.454 INFO:teuthology.orchestra.run.smithi159.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57322.log.gz 2024-04-04T06:18:41.455 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.67357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62185.log 2024-04-04T06:18:41.455 INFO:teuthology.orchestra.run.smithi159.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.67357.log.gz 2024-04-04T06:18:41.456 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.56815.log: gzip 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56815.log.gz -5 2024-04-04T06:18:41.456 INFO:teuthology.orchestra.run.smithi159.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47847.log 2024-04-04T06:18:41.457 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62007.log 2024-04-04T06:18:41.458 INFO:teuthology.orchestra.run.smithi159.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62185.log.gz 2024-04-04T06:18:41.458 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64262.log 2024-04-04T06:18:41.458 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47847.log.gz 2024-04-04T06:18:41.459 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62007.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62007.log.gz 2024-04-04T06:18:41.459 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43487.log 2024-04-04T06:18:41.460 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.64262.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64262.log.gz 2024-04-04T06:18:41.460 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51205.log 2024-04-04T06:18:41.461 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76252.log 2024-04-04T06:18:41.461 INFO:teuthology.orchestra.run.smithi159.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43487.log.gz 2024-04-04T06:18:41.462 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.51205.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51205.log.gz 2024-04-04T06:18:41.463 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67668.log 2024-04-04T06:18:41.463 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.76252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75737.log 2024-04-04T06:18:41.463 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.76252.log.gz 2024-04-04T06:18:41.464 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.67668.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67668.log.gz 2024-04-04T06:18:41.465 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50384.log 2024-04-04T06:18:41.466 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.75737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77212.log 2024-04-04T06:18:41.466 INFO:teuthology.orchestra.run.smithi159.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75737.log.gz 2024-04-04T06:18:41.466 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.50384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80345.log 2024-04-04T06:18:41.467 INFO:teuthology.orchestra.run.smithi159.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50384.log.gz 2024-04-04T06:18:41.467 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.77212.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77212.log.gz 2024-04-04T06:18:41.468 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67888.log 2024-04-04T06:18:41.469 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.80345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66506.log 2024-04-04T06:18:41.469 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80345.log.gz 2024-04-04T06:18:41.470 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.67888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46086.log 2024-04-04T06:18:41.470 INFO:teuthology.orchestra.run.smithi159.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67888.log.gz 2024-04-04T06:18:41.471 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.66506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78331.log 2024-04-04T06:18:41.472 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46086.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.66506.log.gz 2024-04-04T06:18:41.472 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75056.log 2024-04-04T06:18:41.472 INFO:teuthology.orchestra.run.smithi159.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46086.log.gz 2024-04-04T06:18:41.473 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.78331.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78331.log.gz 2024-04-04T06:18:41.473 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65057.log 2024-04-04T06:18:41.474 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.75056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47805.log 2024-04-04T06:18:41.474 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75056.log.gz 2024-04-04T06:18:41.475 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.65057.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65057.log.gz 2024-04-04T06:18:41.475 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57616.log 2024-04-04T06:18:41.476 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60547.log 2024-04-04T06:18:41.476 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47805.log.gz 2024-04-04T06:18:41.477 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76186.log 2024-04-04T06:18:41.477 INFO:teuthology.orchestra.run.smithi159.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57616.log.gz 2024-04-04T06:18:41.478 INFO:teuthology.orchestra.run.smithi178.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-04T06:18:41.479 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.60547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72644.log 2024-04-04T06:18:41.479 INFO:teuthology.orchestra.run.smithi159.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60547.log.gz 2024-04-04T06:18:41.479 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.76186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51103.log 2024-04-04T06:18:41.482 INFO:teuthology.orchestra.run.smithi159.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.76186.log.gz 2024-04-04T06:18:41.482 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.72644.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72644.log.gz 2024-04-04T06:18:41.483 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59357.log 2024-04-04T06:18:41.483 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.51103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42512.log 2024-04-04T06:18:41.483 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51103.log.gz 2024-04-04T06:18:41.483 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.59357.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59357.log.gz 2024-04-04T06:18:41.483 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42764.log 2024-04-04T06:18:41.484 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80063.log 2024-04-04T06:18:41.484 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42512.log.gz 2024-04-04T06:18:41.484 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43139.log 2024-04-04T06:18:41.484 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42764.log.gz 2024-04-04T06:18:41.485 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.80063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50361.log 2024-04-04T06:18:41.485 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80063.log.gz 2024-04-04T06:18:41.486 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50725.log 2024-04-04T06:18:41.486 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43139.log.gz 2024-04-04T06:18:41.487 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.50361.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50361.log.gz 2024-04-04T06:18:41.488 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68962.log 2024-04-04T06:18:41.488 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.50725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43828.log 2024-04-04T06:18:41.489 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50725.log.gz 2024-04-04T06:18:41.490 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.68962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63385.log 2024-04-04T06:18:41.490 INFO:teuthology.orchestra.run.smithi159.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68962.log.gz 2024-04-04T06:18:41.491 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68308.log 2024-04-04T06:18:41.491 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43828.log.gz 2024-04-04T06:18:41.492 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63385.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63385.log.gz 2024-04-04T06:18:41.492 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75134.log 2024-04-04T06:18:41.493 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.68308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54337.log 2024-04-04T06:18:41.493 INFO:teuthology.orchestra.run.smithi159.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68308.log.gz 2024-04-04T06:18:41.494 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.75134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54761.log 2024-04-04T06:18:41.494 INFO:teuthology.orchestra.run.smithi159.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.75134.log.gz 2024-04-04T06:18:41.495 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55073.log 2024-04-04T06:18:41.495 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54337.log.gz 2024-04-04T06:18:41.496 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54761.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54761.log.gz 2024-04-04T06:18:41.496 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67626.log 2024-04-04T06:18:41.497 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.55073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55577.log 2024-04-04T06:18:41.497 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55073.log.gz 2024-04-04T06:18:41.498 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.67626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64915.log 2024-04-04T06:18:41.498 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67626.log.gz 2024-04-04T06:18:41.499 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.55577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58013.log 2024-04-04T06:18:41.499 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55577.log.gz 2024-04-04T06:18:41.500 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46464.log 88.2% 2024-04-04T06:18:41.500 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64915.log.gz 2024-04-04T06:18:41.501 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71311.log 2024-04-04T06:18:41.501 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58013.log.gz 2024-04-04T06:18:41.502 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77235.log 2024-04-04T06:18:41.503 INFO:teuthology.orchestra.run.smithi159.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46464.log.gz 2024-04-04T06:18:41.503 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.71311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48597.log 2024-04-04T06:18:41.504 INFO:teuthology.orchestra.run.smithi159.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.71311.log.gz 2024-04-04T06:18:41.504 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.77235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74072.log 2024-04-04T06:18:41.505 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48597.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.77235.log.gz 2024-04-04T06:18:41.506 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77053.log 2024-04-04T06:18:41.506 INFO:teuthology.orchestra.run.smithi159.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48597.log.gz 2024-04-04T06:18:41.507 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.74072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42554.log 2024-04-04T06:18:41.507 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74072.log.gz 2024-04-04T06:18:41.508 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.77053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62834.log 2024-04-04T06:18:41.508 INFO:teuthology.orchestra.run.smithi159.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.77053.log.gz 2024-04-04T06:18:41.509 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70691.log 2024-04-04T06:18:41.509 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42554.log.gz 2024-04-04T06:18:41.510 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76144.log 2024-04-04T06:18:41.510 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62834.log.gz 2024-04-04T06:18:41.511 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.70691.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70691.log.gz 2024-04-04T06:18:41.511 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78677.log 2024-04-04T06:18:41.512 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.76144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52277.log 2024-04-04T06:18:41.512 INFO:teuthology.orchestra.run.smithi159.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76144.log.gz 2024-04-04T06:18:41.513 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.78677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46887.log 2024-04-04T06:18:41.513 INFO:teuthology.orchestra.run.smithi159.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.78677.log.gz 2024-04-04T06:18:41.514 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52277.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52277.log.gz 2024-04-04T06:18:41.514 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65885.log 2024-04-04T06:18:41.515 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58893.log 2024-04-04T06:18:41.515 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46887.log.gz 2024-04-04T06:18:41.516 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.65885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69311.log 2024-04-04T06:18:41.517 INFO:teuthology.orchestra.run.smithi159.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.65885.log.gz 2024-04-04T06:18:41.517 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67710.log 89.0% 2024-04-04T06:18:41.517 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58893.log.gz 2024-04-04T06:18:41.518 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.69311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54888.log 2024-04-04T06:18:41.518 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69311.log.gz 2024-04-04T06:18:41.519 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.67710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44378.log 2024-04-04T06:18:41.519 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67710.log.gz 2024-04-04T06:18:41.520 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54888.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54888.log.gz 2024-04-04T06:18:41.520 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61052.log 2024-04-04T06:18:41.522 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76440.log 2024-04-04T06:18:41.522 INFO:teuthology.orchestra.run.smithi159.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44378.log.gz 2024-04-04T06:18:41.523 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63864.log 2024-04-04T06:18:41.523 INFO:teuthology.orchestra.run.smithi159.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61052.log.gz 2024-04-04T06:18:41.523 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.76440.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.76440.log.gz 2024-04-04T06:18:41.524 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54846.log 2024-04-04T06:18:41.525 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75653.log 2024-04-04T06:18:41.525 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63864.log.gz 2024-04-04T06:18:41.526 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56109.log 2024-04-04T06:18:41.526 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54846.log.gz 2024-04-04T06:18:41.527 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.75653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48933.log 2024-04-04T06:18:41.527 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75653.log.gz 2024-04-04T06:18:41.528 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.56109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53002.log 2024-04-04T06:18:41.528 INFO:teuthology.orchestra.run.smithi159.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.56109.log.gz 2024-04-04T06:18:41.529 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48933.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48933.log.gz 2024-04-04T06:18:41.529 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48099.log 2024-04-04T06:18:41.530 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62395.log 2024-04-04T06:18:41.531 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48099.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53002.log.gz 2024-04-04T06:18:41.531 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose 88.8% -- -- replaced with /var/log/ceph/ceph-client.admin.48099.log.gz 2024-04-04T06:18:41.531 INFO:teuthology.orchestra.run.smithi159.stderr: /var/log/ceph/ceph-client.admin.76098.log 2024-04-04T06:18:41.532 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64074.log 2024-04-04T06:18:41.532 INFO:teuthology.orchestra.run.smithi159.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62395.log.gz 2024-04-04T06:18:41.533 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.76098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42130.log 2024-04-04T06:18:41.534 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76098.log.gz 2024-04-04T06:18:41.534 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64074.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64074.log.gz 2024-04-04T06:18:41.535 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55661.log 2024-04-04T06:18:41.536 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53534.log 2024-04-04T06:18:41.536 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42130.log.gz 2024-04-04T06:18:41.537 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.55661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52403.log 2024-04-04T06:18:41.537 INFO:teuthology.orchestra.run.smithi159.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.55661.log.gz 2024-04-04T06:18:41.538 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48512.log 2024-04-04T06:18:41.538 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53534.log.gz 2024-04-04T06:18:41.539 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55535.log 2024-04-04T06:18:41.539 INFO:teuthology.orchestra.run.smithi159.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52403.log.gz 2024-04-04T06:18:41.540 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61813.log 2024-04-04T06:18:41.540 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48512.log.gz 2024-04-04T06:18:41.541 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.55535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43814.log 2024-04-04T06:18:41.541 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55535.log.gz 2024-04-04T06:18:41.542 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53492.log 2024-04-04T06:18:41.542 INFO:teuthology.orchestra.run.smithi159.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.61813.log.gz 2024-04-04T06:18:41.543 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43814.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43814.log.gz 2024-04-04T06:18:41.543 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57186.log 2024-04-04T06:18:41.544 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70154.log 2024-04-04T06:18:41.545 INFO:teuthology.orchestra.run.smithi159.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53492.log.gz 2024-04-04T06:18:41.545 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53174.log 2024-04-04T06:18:41.546 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57186.log.gz 2024-04-04T06:18:41.547 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.70154.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.70154.log.gz 2024-04-04T06:18:41.547 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43156.log 2024-04-04T06:18:41.548 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64588.log 2024-04-04T06:18:41.548 INFO:teuthology.orchestra.run.smithi178.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-04T06:18:41.549 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43156.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53174.log.gz 2024-04-04T06:18:41.549 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59169.log 2024-04-04T06:18:41.549 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43156.log.gz 2024-04-04T06:18:41.550 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64588.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64588.log.gz 2024-04-04T06:18:41.550 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46674.log 2024-04-04T06:18:41.551 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.59169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77752.log 2024-04-04T06:18:41.551 INFO:teuthology.orchestra.run.smithi159.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59169.log.gz 2024-04-04T06:18:41.552 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77556.log 2024-04-04T06:18:41.552 INFO:teuthology.orchestra.run.smithi159.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46674.log.gz 2024-04-04T06:18:41.553 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.77752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72602.log 2024-04-04T06:18:41.553 INFO:teuthology.orchestra.run.smithi159.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77752.log.gz 2024-04-04T06:18:41.554 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.77556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67531.log 2024-04-04T06:18:41.555 INFO:teuthology.orchestra.run.smithi159.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77556.log.gz 2024-04-04T06:18:41.556 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.72602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58763.log 2024-04-04T06:18:41.556 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72602.log.gz 2024-04-04T06:18:41.557 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.67531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53754.log 2024-04-04T06:18:41.557 INFO:teuthology.orchestra.run.smithi159.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67531.log.gz 2024-04-04T06:18:41.558 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73946.log 2024-04-04T06:18:41.558 INFO:teuthology.orchestra.run.smithi159.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58763.log.gz 2024-04-04T06:18:41.559 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63427.log 2024-04-04T06:18:41.559 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53754.log.gz 2024-04-04T06:18:41.560 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.73946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71078.log 2024-04-04T06:18:41.560 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73946.log.gz 2024-04-04T06:18:41.561 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52875.log 2024-04-04T06:18:41.561 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63427.log.gz 2024-04-04T06:18:41.562 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.71078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50495.log 2024-04-04T06:18:41.562 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71078.log.gz 2024-04-04T06:18:41.563 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68453.log 2024-04-04T06:18:41.563 INFO:teuthology.orchestra.run.smithi159.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.52875.log.gz 2024-04-04T06:18:41.565 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.50495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50683.log 2024-04-04T06:18:41.565 INFO:teuthology.orchestra.run.smithi159.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.50495.log.gz 2024-04-04T06:18:41.565 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.68453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52319.log 2024-04-04T06:18:41.566 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68453.log.gz 2024-04-04T06:18:41.567 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.50683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59361.log 2024-04-04T06:18:41.567 INFO:teuthology.orchestra.run.smithi159.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50683.log.gz 2024-04-04T06:18:41.568 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62918.log 2024-04-04T06:18:41.568 INFO:teuthology.orchestra.run.smithi159.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52319.log.gz 2024-04-04T06:18:41.569 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.59361.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59361.log.gz 2024-04-04T06:18:41.569 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50767.log 2024-04-04T06:18:41.570 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76056.log 2024-04-04T06:18:41.570 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62918.log.gz 2024-04-04T06:18:41.571 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.50767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42974.log 2024-04-04T06:18:41.571 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50767.log.gz 2024-04-04T06:18:41.572 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.76056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52699.log 2024-04-04T06:18:41.572 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76056.log.gz 2024-04-04T06:18:41.573 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73816.log 2024-04-04T06:18:41.574 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42974.log.gz 2024-04-04T06:18:41.574 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.52699.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52699.log.gz 2024-04-04T06:18:41.574 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62876.log 2024-04-04T06:18:41.576 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.73816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44108.log 2024-04-04T06:18:41.576 INFO:teuthology.orchestra.run.smithi159.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.73816.log.gz 2024-04-04T06:18:41.577 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60382.log 2024-04-04T06:18:41.577 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62876.log.gz 2024-04-04T06:18:41.578 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54295.log 2024-04-04T06:18:41.578 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44108.log.gz 2024-04-04T06:18:41.579 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.60382.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.60382.log.gz 2024-04-04T06:18:41.579 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58055.log 2024-04-04T06:18:41.580 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61118.log 2024-04-04T06:18:41.580 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54295.log.gz 2024-04-04T06:18:41.581 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58055.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.58055.log.gz 2024-04-04T06:18:41.581 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78845.log 2024-04-04T06:18:41.582 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49063.log 2024-04-04T06:18:41.583 INFO:teuthology.orchestra.run.smithi159.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61118.log.gz 2024-04-04T06:18:41.583 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.78845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59847.log 2024-04-04T06:18:41.584 INFO:teuthology.orchestra.run.smithi159.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78845.log.gz 2024-04-04T06:18:41.584 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.49063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58396.log 2024-04-04T06:18:41.584 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49063.log.gz 2024-04-04T06:18:41.586 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.59847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47133.log 2024-04-04T06:18:41.586 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59847.log.gz 2024-04-04T06:18:41.587 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78761.log 2024-04-04T06:18:41.587 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58396.log.gz 2024-04-04T06:18:41.588 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47133.log: gzip -5 --verbose -- 89.0% /var/log/ceph/ceph-client.admin.70196.log 2024-04-04T06:18:41.588 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47133.log.gz 2024-04-04T06:18:41.589 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.78761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64266.log 2024-04-04T06:18:41.589 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78761.log.gz 2024-04-04T06:18:41.590 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.70196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71999.log 2024-04-04T06:18:41.590 INFO:teuthology.orchestra.run.smithi159.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70196.log.gz 2024-04-04T06:18:41.591 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.64266.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64266.log.gz 2024-04-04T06:18:41.591 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-04T06:18:41.592 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.71999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68266.log 2024-04-04T06:18:41.592 INFO:teuthology.orchestra.run.smithi159.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71999.log.gz 2024-04-04T06:18:41.593 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42848.log 2024-04-04T06:18:41.595 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.68266.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68266.log.gz 2024-04-04T06:18:41.595 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50977.log 2024-04-04T06:18:41.595 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55157.log 2024-04-04T06:18:41.596 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42848.log.gz 2024-04-04T06:18:41.596 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.50977.log: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-04T06:18:41.597 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50977.log.gz 2024-04-04T06:18:41.597 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73774.log 2024-04-04T06:18:41.598 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.55157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53796.log 2024-04-04T06:18:41.598 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55157.log.gz 2024-04-04T06:18:41.599 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.73774.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.73774.log.gz -5 2024-04-04T06:18:41.599 INFO:teuthology.orchestra.run.smithi159.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49021.log 2024-04-04T06:18:41.600 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62708.log 2024-04-04T06:18:41.600 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53796.log.gz 2024-04-04T06:18:41.602 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.49021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45918.log 2024-04-04T06:18:41.602 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49021.log.gz 2024-04-04T06:18:41.603 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59785.log 2024-04-04T06:18:41.603 INFO:teuthology.orchestra.run.smithi159.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62708.log.gz 2024-04-04T06:18:41.604 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.45918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64848.log 2024-04-04T06:18:41.604 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45918.log.gz 2024-04-04T06:18:41.605 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.59785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53712.log 2024-04-04T06:18:41.605 INFO:teuthology.orchestra.run.smithi159.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59785.log.gz 2024-04-04T06:18:41.606 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60386.log 2024-04-04T06:18:41.606 INFO:teuthology.orchestra.run.smithi159.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.64848.log.gz 2024-04-04T06:18:41.607 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60086.log 2024-04-04T06:18:41.607 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53712.log.gz 2024-04-04T06:18:41.608 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.60386.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.0.60386.log.gz 2024-04-04T06:18:41.608 INFO:teuthology.orchestra.run.smithi159.stderr: /var/log/ceph/ceph-client.admin.58584.log 2024-04-04T06:18:41.609 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.60086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54421.log 2024-04-04T06:18:41.609 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60086.log.gz 2024-04-04T06:18:41.610 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60631.log 2024-04-04T06:18:41.611 INFO:teuthology.orchestra.run.smithi159.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58584.log.gz 2024-04-04T06:18:41.612 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55367.log 2024-04-04T06:18:41.612 INFO:teuthology.orchestra.run.smithi159.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54421.log.gz 2024-04-04T06:18:41.613 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.60631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52193.log 2024-04-04T06:18:41.613 INFO:teuthology.orchestra.run.smithi159.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60631.log.gz 2024-04-04T06:18:41.614 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.55367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77598.log 2024-04-04T06:18:41.614 INFO:teuthology.orchestra.run.smithi159.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55367.log.gz 2024-04-04T06:18:41.615 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78289.log 2024-04-04T06:18:41.615 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52193.log.gz 2024-04-04T06:18:41.616 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.77598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50458.log 2024-04-04T06:18:41.616 INFO:teuthology.orchestra.run.smithi159.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77598.log.gz 2024-04-04T06:18:41.617 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.78289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43475.log 2024-04-04T06:18:41.617 INFO:teuthology.orchestra.run.smithi159.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78289.log.gz 2024-04-04T06:18:41.618 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.50458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42087.log 2024-04-04T06:18:41.618 INFO:teuthology.orchestra.run.smithi159.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.50458.log.gz 2024-04-04T06:18:41.619 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47973.log 2024-04-04T06:18:41.619 INFO:teuthology.orchestra.run.smithi159.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43475.log.gz 2024-04-04T06:18:41.620 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46548.log 2024-04-04T06:18:41.621 INFO:teuthology.orchestra.run.smithi159.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42087.log.gz 2024-04-04T06:18:41.622 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71632.log 2024-04-04T06:18:41.622 INFO:teuthology.orchestra.run.smithi159.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47973.log.gz 2024-04-04T06:18:41.623 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68537.log 2024-04-04T06:18:41.623 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46548.log.gz 2024-04-04T06:18:41.624 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.71632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58935.log 2024-04-04T06:18:41.624 INFO:teuthology.orchestra.run.smithi159.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.71632.log.gz 2024-04-04T06:18:41.625 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.68537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74912.log 2024-04-04T06:18:41.625 INFO:teuthology.orchestra.run.smithi159.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68537.log.gz 2024-04-04T06:18:41.626 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46296.log 2024-04-04T06:18:41.626 INFO:teuthology.orchestra.run.smithi159.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58935.log.gz 2024-04-04T06:18:41.627 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.74912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64008.log 2024-04-04T06:18:41.627 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74912.log.gz 2024-04-04T06:18:41.628 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log 2024-04-04T06:18:41.628 INFO:teuthology.orchestra.run.smithi159.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46296.log.gz 2024-04-04T06:18:41.629 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42638.log 2024-04-04T06:18:41.630 INFO:teuthology.orchestra.run.smithi159.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.64008.log.gz 2024-04-04T06:18:41.631 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63097.log 2024-04-04T06:18:41.631 INFO:teuthology.orchestra.run.smithi159.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62792.log.gz 2024-04-04T06:18:41.631 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42638.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42638.log.gz 2024-04-04T06:18:41.632 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50935.log 2024-04-04T06:18:41.633 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63097.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63097.log.gz 2024-04-04T06:18:41.633 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59019.log 2024-04-04T06:18:41.634 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.50935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78551.log 2024-04-04T06:18:41.635 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.59019.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50935.log.gz 2024-04-04T06:18:41.635 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59019.log.gz 2024-04-04T06:18:41.635 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49501.log 2024-04-04T06:18:41.636 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.78551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44546.log 2024-04-04T06:18:41.636 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78551.log.gz 2024-04-04T06:18:41.637 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.49501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68800.log 2024-04-04T06:18:41.637 INFO:teuthology.orchestra.run.smithi159.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.49501.log.gz 2024-04-04T06:18:41.638 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42386.log 2024-04-04T06:18:41.639 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44546.log.gz 2024-04-04T06:18:41.639 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.68800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46212.log 2024-04-04T06:18:41.639 INFO:teuthology.orchestra.run.smithi159.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68800.log.gz 2024-04-04T06:18:41.640 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42386.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42386.log.gz 2024-04-04T06:18:41.640 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72170.log 2024-04-04T06:18:41.642 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72686.log 2024-04-04T06:18:41.642 INFO:teuthology.orchestra.run.smithi159.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46212.log.gz 2024-04-04T06:18:41.643 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.72170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63002.log 2024-04-04T06:18:41.644 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.72686.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.72170.log.gz 2024-04-04T06:18:41.644 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44066.log 2024-04-04T06:18:41.644 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72686.log.gz 2024-04-04T06:18:41.645 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71826.log 2024-04-04T06:18:41.645 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63002.log.gz 2024-04-04T06:18:41.646 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44066.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44066.log.gz 2024-04-04T06:18:41.646 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52445.log 2024-04-04T06:18:41.647 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.71826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57532.log 2024-04-04T06:18:41.647 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71826.log.gz 2024-04-04T06:18:41.648 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60793.log 2024-04-04T06:18:41.648 INFO:teuthology.orchestra.run.smithi159.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.52445.log.gz 2024-04-04T06:18:41.649 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59665.log 2024-04-04T06:18:41.649 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57532.log.gz 2024-04-04T06:18:41.650 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.60793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70443.log 2024-04-04T06:18:41.650 INFO:teuthology.orchestra.run.smithi159.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.60793.log.gz 2024-04-04T06:18:41.651 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.59665.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59665.log.gz 2024-04-04T06:18:41.651 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51970.log 2024-04-04T06:18:41.652 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.70443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65603.log 2024-04-04T06:18:41.652 INFO:teuthology.orchestra.run.smithi159.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.70443.log.gz 2024-04-04T06:18:41.653 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.51970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53898.log 2024-04-04T06:18:41.653 INFO:teuthology.orchestra.run.smithi159.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51970.log.gz 2024-04-04T06:18:41.654 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.65603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69669.log 2024-04-04T06:18:41.655 INFO:teuthology.orchestra.run.smithi159.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.65603.log.gz 2024-04-04T06:18:41.655 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53898.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53898.log.gz 2024-04-04T06:18:41.655 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62091.log 2024-04-04T06:18:41.656 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.69669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46632.log 2024-04-04T06:18:41.657 INFO:teuthology.orchestra.run.smithi159.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69669.log.gz 2024-04-04T06:18:41.657 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62091.log: gzip -5 88.7% --verbose -- replaced with /var/log/ceph/ceph-client.admin.62091.log.gz -- 2024-04-04T06:18:41.658 INFO:teuthology.orchestra.run.smithi159.stderr: /var/log/ceph/ceph-client.admin.75569.log 2024-04-04T06:18:41.658 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49105.log 2024-04-04T06:18:41.659 INFO:teuthology.orchestra.run.smithi159.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46632.log.gz 2024-04-04T06:18:41.659 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.75569.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75569.log.gz 2024-04-04T06:18:41.660 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72770.log 2024-04-04T06:18:41.661 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.49105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64770.log 2024-04-04T06:18:41.661 INFO:teuthology.orchestra.run.smithi159.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49105.log.gz 2024-04-04T06:18:41.662 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.72770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57102.log 2024-04-04T06:18:41.662 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72770.log.gz 2024-04-04T06:18:41.663 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77090.log 2024-04-04T06:18:41.663 INFO:teuthology.orchestra.run.smithi159.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64770.log.gz 2024-04-04T06:18:41.664 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54108.log 2024-04-04T06:18:41.664 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57102.log.gz 2024-04-04T06:18:41.665 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.77090.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77090.log.gz 2024-04-04T06:18:41.665 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53450.log 2024-04-04T06:18:41.666 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54108.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54108.log.gz 2024-04-04T06:18:41.666 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63906.log 2024-04-04T06:18:41.667 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60194.log 2024-04-04T06:18:41.667 INFO:teuthology.orchestra.run.smithi159.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53450.log.gz 2024-04-04T06:18:41.668 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59623.log 2024-04-04T06:18:41.668 INFO:teuthology.orchestra.run.smithi159.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63906.log.gz 2024-04-04T06:18:41.669 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.60194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43150.log 2024-04-04T06:18:41.669 INFO:teuthology.orchestra.run.smithi159.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60194.log.gz 2024-04-04T06:18:41.670 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.59623.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59623.log.gz 2024-04-04T06:18:41.670 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70499.log 2024-04-04T06:18:41.671 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36847.log 2024-04-04T06:18:41.671 INFO:teuthology.orchestra.run.smithi159.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43150.log.gz 2024-04-04T06:18:41.672 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.70499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51882.log 2024-04-04T06:18:41.672 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70499.log.gz 2024-04-04T06:18:41.673 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.36847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66421.log 2024-04-04T06:18:41.673 INFO:teuthology.orchestra.run.smithi159.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.36847.log.gz 2024-04-04T06:18:41.674 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.51882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66259.log 2024-04-04T06:18:41.674 INFO:teuthology.orchestra.run.smithi159.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51882.log.gz 2024-04-04T06:18:41.675 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.66421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47175.log 2024-04-04T06:18:41.675 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66421.log.gz 2024-04-04T06:18:41.676 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.66259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72914.log 2024-04-04T06:18:41.676 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66259.log.gz 2024-04-04T06:18:41.677 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47175.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47175.log.gz -5 2024-04-04T06:18:41.677 INFO:teuthology.orchestra.run.smithi159.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50199.log 2024-04-04T06:18:41.678 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.72914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73593.log 2024-04-04T06:18:41.678 INFO:teuthology.orchestra.run.smithi159.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72914.log.gz 2024-04-04T06:18:41.679 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.50199.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50199.log.gz 2024-04-04T06:18:41.679 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69567.log 2024-04-04T06:18:41.680 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.73593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42890.log 2024-04-04T06:18:41.680 INFO:teuthology.orchestra.run.smithi159.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73593.log.gz 2024-04-04T06:18:41.681 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.69567.log: gzip 89.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.69567.log.gz --verbose 2024-04-04T06:18:41.681 INFO:teuthology.orchestra.run.smithi159.stderr: -- /var/log/ceph/ceph-client.admin.76936.log 2024-04-04T06:18:41.682 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49249.log 2024-04-04T06:18:41.682 INFO:teuthology.orchestra.run.smithi159.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42890.log.gz 2024-04-04T06:18:41.683 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.76936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37215.log 2024-04-04T06:18:41.683 INFO:teuthology.orchestra.run.smithi159.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76936.log.gz 2024-04-04T06:18:41.686 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.49249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69174.log 2024-04-04T06:18:41.686 INFO:teuthology.orchestra.run.smithi159.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49249.log.gz 2024-04-04T06:18:41.686 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.37215.log: gzip 88.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.37215.log.gz --verbose 2024-04-04T06:18:41.687 INFO:teuthology.orchestra.run.smithi159.stderr: -- /var/log/ceph/ceph-client.admin.74282.log 2024-04-04T06:18:41.687 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.69174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65353.log 2024-04-04T06:18:41.687 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69174.log.gz 2024-04-04T06:18:41.687 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.74282.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74282.log.gz 2024-04-04T06:18:41.688 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77385.log 2024-04-04T06:18:41.689 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.65353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68663.log 2024-04-04T06:18:41.689 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65353.log.gz 2024-04-04T06:18:41.690 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.77385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59103.log 2024-04-04T06:18:41.690 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.68663.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.77385.log.gz 2024-04-04T06:18:41.691 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61418.log 2024-04-04T06:18:41.691 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68663.log.gz 2024-04-04T06:18:41.692 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.59103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76816.log 2024-04-04T06:18:41.693 INFO:teuthology.orchestra.run.smithi159.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.59103.log.gz 2024-04-04T06:18:41.693 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47931.log 2024-04-04T06:18:41.693 INFO:teuthology.orchestra.run.smithi159.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.61418.log.gz 2024-04-04T06:18:41.694 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.76816.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76816.log.gz 2024-04-04T06:18:41.694 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-04T06:18:41.695 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43155.log 2024-04-04T06:18:41.695 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47931.log.gz 2024-04-04T06:18:41.696 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47721.log 2024-04-04T06:18:41.697 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43155.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43155.log.gz 2024-04-04T06:18:41.697 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65518.log 2024-04-04T06:18:41.698 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67129.log 2024-04-04T06:18:41.699 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47721.log.gz 2024-04-04T06:18:41.699 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.65518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49417.log 2024-04-04T06:18:41.700 INFO:teuthology.orchestra.run.smithi159.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.65518.log.gz 2024-04-04T06:18:41.700 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.67129.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67129.log.gz 2024-04-04T06:18:41.701 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74954.log 2024-04-04T06:18:41.702 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.49417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44588.log 2024-04-04T06:18:41.702 INFO:teuthology.orchestra.run.smithi159.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49417.log.gz 2024-04-04T06:18:41.703 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.74954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66035.log 2024-04-04T06:18:41.703 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74954.log.gz 2024-04-04T06:18:41.704 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68182.log 2024-04-04T06:18:41.704 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44588.log.gz 2024-04-04T06:18:41.705 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.66035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59061.log 2024-04-04T06:18:41.705 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66035.log.gz 2024-04-04T06:18:41.706 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.68182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65969.log 2024-04-04T06:18:41.706 INFO:teuthology.orchestra.run.smithi159.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68182.log.gz 2024-04-04T06:18:41.707 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.59061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59581.log 2024-04-04T06:18:41.707 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59061.log.gz 2024-04-04T06:18:41.708 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.65969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70238.log 2024-04-04T06:18:41.708 INFO:teuthology.orchestra.run.smithi159.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.65969.log.gz 2024-04-04T06:18:41.709 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.59581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65669.log 88.2% 2024-04-04T06:18:41.709 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59581.log.gz 2024-04-04T06:18:41.710 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.70238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42680.log 2024-04-04T06:18:41.710 INFO:teuthology.orchestra.run.smithi159.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70238.log.gz 2024-04-04T06:18:41.711 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.65669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44420.log 2024-04-04T06:18:41.711 INFO:teuthology.orchestra.run.smithi159.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.65669.log.gz 2024-04-04T06:18:41.712 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42680.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42680.log.gz 2024-04-04T06:18:41.712 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56857.log 2024-04-04T06:18:41.713 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73679.log 2024-04-04T06:18:41.713 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44420.log.gz 2024-04-04T06:18:41.714 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.56857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51061.log 2024-04-04T06:18:41.714 INFO:teuthology.orchestra.run.smithi159.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.56857.log.gz 2024-04-04T06:18:41.715 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.73679.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73679.log.gz 2024-04-04T06:18:41.715 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78929.log 2024-04-04T06:18:41.716 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.51061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77016.log 2024-04-04T06:18:41.716 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51061.log.gz 2024-04-04T06:18:41.717 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.78929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79853.log 2024-04-04T06:18:41.717 INFO:teuthology.orchestra.run.smithi159.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78929.log.gz 2024-04-04T06:18:41.718 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.77016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56476.log 2024-04-04T06:18:41.718 INFO:teuthology.orchestra.run.smithi159.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.77016.log.gz 2024-04-04T06:18:41.719 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.79853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62605.log 2024-04-04T06:18:41.719 INFO:teuthology.orchestra.run.smithi159.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.79853.log.gz 2024-04-04T06:18:41.720 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.56476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42045.log 2024-04-04T06:18:41.721 INFO:teuthology.orchestra.run.smithi159.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56476.log.gz 2024-04-04T06:18:41.721 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41868.log 2024-04-04T06:18:41.721 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62605.log.gz 2024-04-04T06:18:41.722 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72560.log 2024-04-04T06:18:41.722 INFO:teuthology.orchestra.run.smithi159.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42045.log.gz 2024-04-04T06:18:41.723 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.41868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62960.log 88.9% 2024-04-04T06:18:41.723 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41868.log.gz 2024-04-04T06:18:41.724 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.72560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58480.log 2024-04-04T06:18:41.724 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72560.log.gz 2024-04-04T06:18:41.725 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76774.log 2024-04-04T06:18:41.725 INFO:teuthology.orchestra.run.smithi159.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62960.log.gz 2024-04-04T06:18:41.726 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70952.log 2024-04-04T06:18:41.726 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58480.log.gz 2024-04-04T06:18:41.727 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.76774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44810.log 2024-04-04T06:18:41.727 INFO:teuthology.orchestra.run.smithi159.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76774.log.gz 2024-04-04T06:18:41.728 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.70952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78247.log 2024-04-04T06:18:41.728 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70952.log.gz 2024-04-04T06:18:41.729 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79643.log 2024-04-04T06:18:41.729 INFO:teuthology.orchestra.run.smithi159.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44810.log.gz 2024-04-04T06:18:41.730 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.78247.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78247.log.gz 2024-04-04T06:18:41.731 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57700.log 2024-04-04T06:18:41.732 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.79643.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79643.log.gz 2024-04-04T06:18:41.732 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62353.log 2024-04-04T06:18:41.733 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45613.log 2024-04-04T06:18:41.733 INFO:teuthology.orchestra.run.smithi159.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57700.log.gz 2024-04-04T06:18:41.734 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61965.log 2024-04-04T06:18:41.734 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62353.log.gz 2024-04-04T06:18:41.735 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.2.45613.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45613.log.gz 2024-04-04T06:18:41.735 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68056.log 2024-04-04T06:18:41.736 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42470.log 2024-04-04T06:18:41.736 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61965.log.gz 2024-04-04T06:18:41.737 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.68056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73904.log 2024-04-04T06:18:41.737 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68056.log.gz 2024-04-04T06:18:41.738 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78635.log 2024-04-04T06:18:41.738 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42470.log.gz 2024-04-04T06:18:41.739 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.73904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66811.log 2024-04-04T06:18:41.739 INFO:teuthology.orchestra.run.smithi159.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73904.log.gz 2024-04-04T06:18:41.740 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.78635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66608.log 2024-04-04T06:18:41.740 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78635.log.gz 2024-04-04T06:18:41.741 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.66811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77343.log 2024-04-04T06:18:41.741 INFO:teuthology.orchestra.run.smithi159.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.66811.log.gz 2024-04-04T06:18:41.742 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.66608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51019.log 2024-04-04T06:18:41.743 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.77343.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.66608.log.gz 2024-04-04T06:18:41.743 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73385.log 2024-04-04T06:18:41.744 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.51019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49375.log 2024-04-04T06:18:41.745 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.73385.log: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.77343.log.gz 2024-04-04T06:18:41.745 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51019.log.gz 2024-04-04T06:18:41.745 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65452.log 2024-04-04T06:18:41.745 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73385.log.gz 2024-04-04T06:18:41.746 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.49375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51798.log 2024-04-04T06:18:41.746 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49375.log.gz 2024-04-04T06:18:41.748 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.65452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79895.log 2024-04-04T06:18:41.748 INFO:teuthology.orchestra.run.smithi159.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.65452.log.gz 2024-04-04T06:18:41.749 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.51798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43808.log 2024-04-04T06:18:41.749 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51798.log.gz 2024-04-04T06:18:41.750 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.79895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55727.log 2024-04-04T06:18:41.750 INFO:teuthology.orchestra.run.smithi159.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79895.log.gz 2024-04-04T06:18:41.751 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51632.log 2024-04-04T06:18:41.751 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43808.log.gz 2024-04-04T06:18:41.752 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.55727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55283.log 2024-04-04T06:18:41.752 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55727.log.gz 2024-04-04T06:18:41.753 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.51632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68880.log 2024-04-04T06:18:41.753 INFO:teuthology.orchestra.run.smithi159.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51632.log.gz 2024-04-04T06:18:41.754 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.55283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67423.log 2024-04-04T06:18:41.754 INFO:teuthology.orchestra.run.smithi159.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55283.log.gz 2024-04-04T06:18:41.755 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.68880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69046.log 2024-04-04T06:18:41.755 INFO:teuthology.orchestra.run.smithi159.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68880.log.gz 2024-04-04T06:18:41.756 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.67423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78509.log 2024-04-04T06:18:41.756 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67423.log.gz 2024-04-04T06:18:41.757 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.69046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71549.log 2024-04-04T06:18:41.757 INFO:teuthology.orchestra.run.smithi159.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69046.log.gz 2024-04-04T06:18:41.758 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.78509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79937.log 2024-04-04T06:18:41.758 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78509.log.gz 2024-04-04T06:18:41.759 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.71549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67930.log 2024-04-04T06:18:41.759 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71549.log.gz 2024-04-04T06:18:41.760 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.79937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54803.log 2024-04-04T06:18:41.760 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79937.log.gz 2024-04-04T06:18:41.761 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.67930.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67930.log.gz 2024-04-04T06:18:41.761 INFO:teuthology.orchestra.run.smithi159.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.69969.log 2024-04-04T06:18:41.762 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71572.log 2024-04-04T06:18:41.762 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54803.log.gz 2024-04-04T06:18:41.763 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.69969.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69969.log.gz 2024-04-04T06:18:41.763 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64546.log 2024-04-04T06:18:41.764 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.71572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51331.log 2024-04-04T06:18:41.764 INFO:teuthology.orchestra.run.smithi159.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.71572.log.gz 2024-04-04T06:18:41.765 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74620.log 2024-04-04T06:18:41.765 INFO:teuthology.orchestra.run.smithi159.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64546.log.gz 2024-04-04T06:18:41.766 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.51331.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51331.log.gz 2024-04-04T06:18:41.766 INFO:teuthology.orchestra.run.smithi159.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46929.log 2024-04-04T06:18:41.767 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.74620.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.74620.log.gz 2024-04-04T06:18:41.767 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55619.log 2024-04-04T06:18:41.768 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46929.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46929.log.gz 2024-04-04T06:18:41.768 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71784.log 2024-04-04T06:18:41.769 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.55619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57845.log 2024-04-04T06:18:41.769 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55619.log.gz 2024-04-04T06:18:41.770 INFO:teuthology.orchestra.run.smithi178.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-04T06:18:41.771 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.71784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50319.log 2024-04-04T06:18:41.771 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71784.log.gz 2024-04-04T06:18:41.772 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66077.log 2024-04-04T06:18:41.772 INFO:teuthology.orchestra.run.smithi159.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57845.log.gz 2024-04-04T06:18:41.773 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.50319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59944.log 2024-04-04T06:18:41.773 INFO:teuthology.orchestra.run.smithi159.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50319.log.gz 2024-04-04T06:18:41.773 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.66077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48765.log 2024-04-04T06:18:41.774 INFO:teuthology.orchestra.run.smithi159.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.66077.log.gz 2024-04-04T06:18:41.775 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.59944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69399.log 2024-04-04T06:18:41.775 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59944.log.gz 2024-04-04T06:18:41.776 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47217.log 2024-04-04T06:18:41.776 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48765.log.gz 2024-04-04T06:18:41.777 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.69399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56434.log 2024-04-04T06:18:41.777 INFO:teuthology.orchestra.run.smithi159.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69399.log.gz 2024-04-04T06:18:41.778 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47217.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47217.log.gz 2024-04-04T06:18:41.778 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52703.log 2024-04-04T06:18:41.779 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.56434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57364.log 2024-04-04T06:18:41.779 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56434.log.gz 2024-04-04T06:18:41.780 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.52703.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52703.log.gz 2024-04-04T06:18:41.780 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80147.log 2024-04-04T06:18:41.781 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46044.log 2024-04-04T06:18:41.781 INFO:teuthology.orchestra.run.smithi159.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57364.log.gz 2024-04-04T06:18:41.782 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.80147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58851.log 2024-04-04T06:18:41.782 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80147.log.gz 2024-04-04T06:18:41.784 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54066.log 2024-04-04T06:18:41.784 INFO:teuthology.orchestra.run.smithi159.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46044.log.gz 2024-04-04T06:18:41.785 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58851.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58851.log.gz 2024-04-04T06:18:41.785 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76444.log 2024-04-04T06:18:41.786 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50532.log 2024-04-04T06:18:41.786 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54066.log.gz 2024-04-04T06:18:41.786 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.76444.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.76444.log.gz 2024-04-04T06:18:41.787 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72391.log 2024-04-04T06:18:41.788 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.50532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52928.log 2024-04-04T06:18:41.788 INFO:teuthology.orchestra.run.smithi159.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.50532.log.gz 2024-04-04T06:18:41.789 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.72391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73041.log 2024-04-04T06:18:41.789 INFO:teuthology.orchestra.run.smithi159.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.72391.log.gz 2024-04-04T06:18:41.790 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67273.log 2024-04-04T06:18:41.790 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52928.log.gz 2024-04-04T06:18:41.791 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.73041.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73041.log.gz 2024-04-04T06:18:41.791 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59539.log 2024-04-04T06:18:41.792 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.67273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68411.log 2024-04-04T06:18:41.793 INFO:teuthology.orchestra.run.smithi159.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.67273.log.gz 2024-04-04T06:18:41.793 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.59539.log: gzip -5 --verbose 89.4% -- -- replaced with /var/log/ceph/ceph-client.admin.59539.log.gz /var/log/ceph/ceph-client.admin.42806.log 2024-04-04T06:18:41.793 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-04-04T06:18:41.794 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.68411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52511.log 2024-04-04T06:18:41.794 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68411.log.gz 2024-04-04T06:18:41.795 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42806.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42806.log.gz 2024-04-04T06:18:41.795 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71730.log 2024-04-04T06:18:41.796 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43490.log 2024-04-04T06:18:41.797 INFO:teuthology.orchestra.run.smithi159.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52511.log.gz 2024-04-04T06:18:41.797 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.71730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47763.log 2024-04-04T06:18:41.797 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71730.log.gz 2024-04-04T06:18:41.798 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43490.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-04T06:18:41.798 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43490.log.gz 2024-04-04T06:18:41.800 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56749.log 2024-04-04T06:18:41.800 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47763.log.gz 2024-04-04T06:18:41.801 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56064.log 2024-04-04T06:18:41.802 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.56749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56368.log 2024-04-04T06:18:41.802 INFO:teuthology.orchestra.run.smithi159.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56749.log.gz 2024-04-04T06:18:41.803 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.56064.log: gzip 88.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.56064.log.gz --verbose 2024-04-04T06:18:41.803 INFO:teuthology.orchestra.run.smithi159.stderr: -- /var/log/ceph/ceph-client.admin.46422.log 2024-04-04T06:18:41.804 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.56368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63177.log 2024-04-04T06:18:41.805 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46422.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56368.log.gz 2024-04-04T06:18:41.806 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67972.log 2024-04-04T06:18:41.806 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46422.log.gz 2024-04-04T06:18:41.807 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63177.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63177.log.gz 2024-04-04T06:18:41.807 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79811.log 2024-04-04T06:18:41.808 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.67972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44672.log 2024-04-04T06:18:41.809 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67972.log.gz 2024-04-04T06:18:41.809 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.79811.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79811.log.gz -5 2024-04-04T06:18:41.810 INFO:teuthology.orchestra.run.smithi159.stderr: --verbose -- /var/log/ceph/ceph-client.admin.80303.log 2024-04-04T06:18:41.810 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63471.log 2024-04-04T06:18:41.812 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.80303.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44672.log.gz 2024-04-04T06:18:41.812 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46002.log 2024-04-04T06:18:41.812 INFO:teuthology.orchestra.run.smithi159.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80303.log.gz 2024-04-04T06:18:41.813 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52832.log 2024-04-04T06:18:41.813 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63471.log.gz 2024-04-04T06:18:41.814 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51247.log 2024-04-04T06:18:41.814 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46002.log.gz 2024-04-04T06:18:41.815 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52832.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52832.log.gz -5 2024-04-04T06:18:41.815 INFO:teuthology.orchestra.run.smithi159.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47517.log 2024-04-04T06:18:41.816 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.51247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74114.log 2024-04-04T06:18:41.817 INFO:teuthology.orchestra.run.smithi159.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51247.log.gz 2024-04-04T06:18:41.818 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51926.log 2024-04-04T06:18:41.818 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47517.log.gz 2024-04-04T06:18:41.819 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.74114.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74114.log.gz 2024-04-04T06:18:41.819 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66217.log 2024-04-04T06:18:41.820 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61923.log/var/log/ceph/ceph-client.admin.51926.log: 2024-04-04T06:18:41.821 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.66217.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51926.log.gz 2024-04-04T06:18:41.821 INFO:teuthology.orchestra.run.smithi159.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66217.log.gz 2024-04-04T06:18:41.821 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76732.log 2024-04-04T06:18:41.822 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54379.log 2024-04-04T06:18:41.823 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61923.log.gz 2024-04-04T06:18:41.823 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.76732.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76732.log.gz 2024-04-04T06:18:41.824 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48977.log 2024-04-04T06:18:41.824 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43827.log 2024-04-04T06:18:41.825 INFO:teuthology.orchestra.run.smithi159.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54379.log.gz 2024-04-04T06:18:41.826 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48977.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48977.log.gz 2024-04-04T06:18:41.826 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60040.log 2024-04-04T06:18:41.827 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71674.log 2024-04-04T06:18:41.827 INFO:teuthology.orchestra.run.smithi159.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43827.log.gz 2024-04-04T06:18:41.828 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.60040.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60040.log.gz 2024-04-04T06:18:41.828 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49333.log 2024-04-04T06:18:41.829 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.71674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49147.log 2024-04-04T06:18:41.830 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.49333.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.71674.log.gz 2024-04-04T06:18:41.830 INFO:teuthology.orchestra.run.smithi159.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49333.log.gz 2024-04-04T06:18:41.830 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54024.log 2024-04-04T06:18:41.831 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.49147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41787.log 2024-04-04T06:18:41.832 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49147.log.gz 2024-04-04T06:18:41.832 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54024.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54024.log.gz 2024-04-04T06:18:41.833 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76690.log 2024-04-04T06:18:41.833 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.41787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79116.log 2024-04-04T06:18:41.834 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41787.log.gz 2024-04-04T06:18:41.835 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.76690.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76690.log.gz 2024-04-04T06:18:41.835 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72518.log 2024-04-04T06:18:41.836 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.79116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74624.log 2024-04-04T06:18:41.836 INFO:teuthology.orchestra.run.smithi159.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79116.log.gz 2024-04-04T06:18:41.837 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.72518.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72518.log.gz 2024-04-04T06:18:41.837 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57144.log 2024-04-04T06:18:41.837 INFO:teuthology.orchestra.run.smithi178.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-04T06:18:41.838 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.74624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75946.log 2024-04-04T06:18:41.838 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.74624.log.gz 2024-04-04T06:18:41.839 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57144.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57144.log.gz 2024-04-04T06:18:41.839 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68579.log 2024-04-04T06:18:41.840 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.75946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64969.log 2024-04-04T06:18:41.841 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.68579.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.75946.log.gz 2024-04-04T06:18:41.842 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68579.log.gz 2024-04-04T06:18:41.842 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72956.log 2024-04-04T06:18:41.843 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63650.log 2024-04-04T06:18:41.843 INFO:teuthology.orchestra.run.smithi159.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64969.log.gz 2024-04-04T06:18:41.844 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.72956.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72956.log.gz 2024-04-04T06:18:41.844 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72259.log 2024-04-04T06:18:41.845 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69441.log 2024-04-04T06:18:41.845 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63650.log.gz 2024-04-04T06:18:41.846 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.72259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56162.log 2024-04-04T06:18:41.846 INFO:teuthology.orchestra.run.smithi159.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.72259.log.gz 2024-04-04T06:18:41.847 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.69441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75200.log 2024-04-04T06:18:41.847 INFO:teuthology.orchestra.run.smithi159.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.69441.log.gz 2024-04-04T06:18:41.849 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.56162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51552.log 2024-04-04T06:18:41.849 INFO:teuthology.orchestra.run.smithi159.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.56162.log.gz 2024-04-04T06:18:41.850 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.75200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61736.log 2024-04-04T06:18:41.850 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75200.log.gz 2024-04-04T06:18:41.851 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.51552.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51552.log.gz 2024-04-04T06:18:41.851 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48351.log 2024-04-04T06:18:41.852 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.61736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74366.log 2024-04-04T06:18:41.852 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61736.log.gz 2024-04-04T06:18:41.853 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48351.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48351.log.gz 2024-04-04T06:18:41.853 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57490.log 2024-04-04T06:18:41.854 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.74366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66853.log 2024-04-04T06:18:41.856 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57490.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.74366.log.gz 2024-04-04T06:18:41.856 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57490.log.gz 2024-04-04T06:18:41.856 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79074.log 2024-04-04T06:18:41.857 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.66853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80263.log 2024-04-04T06:18:41.857 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.79074.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.66853.log.gz 2024-04-04T06:18:41.858 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79074.log.gz 2024-04-04T06:18:41.858 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63258.log 2024-04-04T06:18:41.859 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57448.log 2024-04-04T06:18:41.859 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.80263.log: /var/log/ceph/ceph-client.admin.63258.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80263.log.gz 2024-04-04T06:18:41.860 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- 88.6% /var/log/ceph/ceph.tmp-client.admin.30819.log 2024-04-04T06:18:41.860 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63258.log.gz 2024-04-04T06:18:41.861 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53670.log 2024-04-04T06:18:41.861 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57448.log.gz 2024-04-04T06:18:41.862 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph.tmp-client.admin.30819.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30819.log.gz 2024-04-04T06:18:41.862 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75611.log 2024-04-04T06:18:41.863 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48807.log 2024-04-04T06:18:41.863 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53670.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53670.log.gz 2024-04-04T06:18:41.864 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.75611.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75611.log.gz 2024-04-04T06:18:41.864 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47637.log 2024-04-04T06:18:41.865 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57803.log 2024-04-04T06:18:41.866 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48807.log.gz 2024-04-04T06:18:41.866 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47637.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47637.log.gz 2024-04-04T06:18:41.866 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72476.log 2024-04-04T06:18:41.867 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63566.log 2024-04-04T06:18:41.868 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57803.log.gz 2024-04-04T06:18:41.869 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.72476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53982.log 2024-04-04T06:18:41.869 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72476.log.gz 2024-04-04T06:18:41.870 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58438.log 2024-04-04T06:18:41.870 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63566.log.gz 2024-04-04T06:18:41.871 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53982.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53982.log.gz 2024-04-04T06:18:41.871 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80021.log 2024-04-04T06:18:41.872 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49567.log 2024-04-04T06:18:41.873 INFO:teuthology.orchestra.run.smithi159.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.58438.log.gz 2024-04-04T06:18:41.873 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.80021.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80021.log.gz 2024-04-04T06:18:41.874 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58354.log 2024-04-04T06:18:41.874 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.49567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53838.log 2024-04-04T06:18:41.875 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49567.log.gz 2024-04-04T06:18:41.876 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58354.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58354.log.gz 2024-04-04T06:18:41.876 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52235.log 2024-04-04T06:18:41.876 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44150.log 2024-04-04T06:18:41.877 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53838.log.gz 2024-04-04T06:18:41.878 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52235.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52235.log.gz 2024-04-04T06:18:41.878 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79032.log 2024-04-04T06:18:41.879 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74240.log 2024-04-04T06:18:41.879 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44150.log.gz 2024-04-04T06:18:41.880 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.79032.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79032.log.gz 2024-04-04T06:18:41.880 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71399.log 2024-04-04T06:18:41.881 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.74240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44294.log 2024-04-04T06:18:41.881 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74240.log.gz 2024-04-04T06:18:41.882 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.71399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43153.log 2024-04-04T06:18:41.882 INFO:teuthology.orchestra.run.smithi159.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.71399.log.gz 2024-04-04T06:18:41.883 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76979.log 2024-04-04T06:18:41.883 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44294.log.gz 2024-04-04T06:18:41.884 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43153.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43153.log.gz 2024-04-04T06:18:41.885 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54600.log 2024-04-04T06:18:41.886 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.76979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58192.log 2024-04-04T06:18:41.887 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54600.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.76979.log.gz 2024-04-04T06:18:41.887 INFO:teuthology.orchestra.run.smithi159.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54600.log.gz 2024-04-04T06:18:41.887 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63948.log 2024-04-04T06:18:41.887 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73988.log 2024-04-04T06:18:41.888 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58192.log.gz 2024-04-04T06:18:41.889 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63948.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63948.log.gz 2024-04-04T06:18:41.889 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42172.log 2024-04-04T06:18:41.890 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.73988.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-04T06:18:41.890 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73988.log.gz 2024-04-04T06:18:41.891 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53240.log 2024-04-04T06:18:41.891 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42172.log.gz 2024-04-04T06:18:41.894 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79433.log 2024-04-04T06:18:41.894 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79517.log 2024-04-04T06:18:41.894 INFO:teuthology.orchestra.run.smithi159.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53240.log.gz 2024-04-04T06:18:41.895 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.79433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66343.log 2024-04-04T06:18:41.895 INFO:teuthology.orchestra.run.smithi159.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79433.log.gz 2024-04-04T06:18:41.896 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.79517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57971.log 2024-04-04T06:18:41.896 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79517.log.gz 2024-04-04T06:18:41.898 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.66343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63608.log 2024-04-04T06:18:41.898 INFO:teuthology.orchestra.run.smithi159.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66343.log.gz 2024-04-04T06:18:41.898 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62563.log 2024-04-04T06:18:41.899 INFO:teuthology.orchestra.run.smithi159.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57971.log.gz 2024-04-04T06:18:41.900 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69777.log 2024-04-04T06:18:41.900 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63608.log.gz 2024-04-04T06:18:41.901 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42596.log 2024-04-04T06:18:41.901 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62563.log.gz 2024-04-04T06:18:41.902 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.69777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51457.log 2024-04-04T06:18:41.903 INFO:teuthology.orchestra.run.smithi159.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-04T06:18:41.903 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69777.log.gz 2024-04-04T06:18:41.904 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50641.log 2024-04-04T06:18:41.904 INFO:teuthology.orchestra.run.smithi159.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42596.log.gz 2024-04-04T06:18:41.905 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.51457.log: gzip -5 88.7% --verbose -- replaced with /var/log/ceph/ceph-client.admin.51457.log.gz -- 2024-04-04T06:18:41.905 INFO:teuthology.orchestra.run.smithi159.stderr: /var/log/ceph/ceph-client.admin.43479.log 2024-04-04T06:18:41.906 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.50641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72434.log 2024-04-04T06:18:41.906 INFO:teuthology.orchestra.run.smithi159.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50641.log.gz 2024-04-04T06:18:41.907 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46254.log 2024-04-04T06:18:41.907 INFO:teuthology.orchestra.run.smithi159.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43479.log.gz 2024-04-04T06:18:41.908 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.72434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79979.log 2024-04-04T06:18:41.909 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46254.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72434.log.gz 2024-04-04T06:18:41.909 INFO:teuthology.orchestra.run.smithi159.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46254.log.gz 2024-04-04T06:18:41.909 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47475.log 2024-04-04T06:18:41.911 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.79979.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79979.log.gz 2024-04-04T06:18:41.911 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48681.log 2024-04-04T06:18:41.911 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63780.log 2024-04-04T06:18:41.912 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48681.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47475.log.gz 2024-04-04T06:18:41.913 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48681.log.gz 2024-04-04T06:18:41.913 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55409.log 2024-04-04T06:18:41.914 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48849.log 2024-04-04T06:18:41.914 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63780.log.gz 2024-04-04T06:18:41.915 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.55409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78989.log 2024-04-04T06:18:41.915 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55409.log.gz 2024-04-04T06:18:41.917 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48849.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48849.log.gz 2024-04-04T06:18:41.917 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71914.log 2024-04-04T06:18:41.918 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.78989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60944.log 2024-04-04T06:18:41.918 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.71914.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78989.log.gz 2024-04-04T06:18:41.919 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55451.log 2024-04-04T06:18:41.919 INFO:teuthology.orchestra.run.smithi159.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.71914.log.gz 2024-04-04T06:18:41.920 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.60944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73466.log 2024-04-04T06:18:41.920 INFO:teuthology.orchestra.run.smithi159.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.60944.log.gz 2024-04-04T06:18:41.921 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.55451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74432.log 2024-04-04T06:18:41.922 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55451.log.gz 2024-04-04T06:18:41.922 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.73466.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73466.log.gz 2024-04-04T06:18:41.922 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61010.log 2024-04-04T06:18:41.923 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62311.log/var/log/ceph/ceph-client.admin.74432.log: 2024-04-04T06:18:41.924 INFO:teuthology.orchestra.run.smithi159.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74432.log.gz 2024-04-04T06:18:41.925 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61010.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61010.log.gz 2024-04-04T06:18:41.925 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50241.log 2024-04-04T06:18:41.925 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55241.log 2024-04-04T06:18:41.926 INFO:teuthology.orchestra.run.smithi159.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62311.log.gz 2024-04-04T06:18:41.927 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.50241.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.50241.log.gz 2024-04-04T06:18:41.927 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58977.log 2024-04-04T06:18:41.928 INFO:teuthology.orchestra.run.smithi159.stderr:gzip/var/log/ceph/ceph-client.admin.55241.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46845.log 2024-04-04T06:18:41.928 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58977.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55241.log.gz 2024-04-04T06:18:41.929 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58977.log.gz 2024-04-04T06:18:41.929 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78719.log 2024-04-04T06:18:41.930 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65011.log 2024-04-04T06:18:41.930 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46845.log.gz 2024-04-04T06:18:41.931 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.78719.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78719.log.gz 2024-04-04T06:18:41.931 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43484.log 2024-04-04T06:18:41.932 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.65011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66669.log 2024-04-04T06:18:41.932 INFO:teuthology.orchestra.run.smithi159.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65011.log.gz 2024-04-04T06:18:41.933 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43484.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43484.log.gz 2024-04-04T06:18:41.933 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45834.log 2024-04-04T06:18:41.934 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.66669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48891.log 2024-04-04T06:18:41.935 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.45834.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66669.log.gz 2024-04-04T06:18:41.935 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45834.log.gz 2024-04-04T06:18:41.935 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56923.log 2024-04-04T06:18:41.936 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77686.log 2024-04-04T06:18:41.937 INFO:teuthology.orchestra.run.smithi159.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48891.log.gz 2024-04-04T06:18:41.938 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.56923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42256.log 2024-04-04T06:18:41.938 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56923.log.gz 2024-04-04T06:18:41.939 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.77686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72872.log 2024-04-04T06:18:41.939 INFO:teuthology.orchestra.run.smithi159.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.77686.log.gz 2024-04-04T06:18:41.940 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70112.log 2024-04-04T06:18:41.940 INFO:teuthology.orchestra.run.smithi159.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42256.log.gz 2024-04-04T06:18:41.941 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.72872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72999.log 2024-04-04T06:18:41.941 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72872.log.gz 2024-04-04T06:18:41.942 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.70112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53090.log 2024-04-04T06:18:41.943 INFO:teuthology.orchestra.run.smithi159.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70112.log.gz 2024-04-04T06:18:41.944 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.72999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44336.log 2024-04-04T06:18:41.944 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72999.log.gz 2024-04-04T06:18:41.945 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77146.log 2024-04-04T06:18:41.946 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.44336.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.53090.log.gz 2024-04-04T06:18:41.946 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73723.log 2024-04-04T06:18:41.946 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44336.log.gz 2024-04-04T06:18:41.947 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.77146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69965.log 2024-04-04T06:18:41.947 INFO:teuthology.orchestra.run.smithi159.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.77146.log.gz 2024-04-04T06:18:41.948 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.73723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78593.log 2024-04-04T06:18:41.948 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73723.log.gz 2024-04-04T06:18:41.949 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.69965.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69965.log.gz 2024-04-04T06:18:41.949 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67752.log 2024-04-04T06:18:41.951 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.78593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69004.log 2024-04-04T06:18:41.951 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78593.log.gz 2024-04-04T06:18:41.952 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.67752.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67752.log.gz 2024-04-04T06:18:41.952 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57406.log 2024-04-04T06:18:41.953 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.69004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45876.log 2024-04-04T06:18:41.954 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57406.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69004.log.gz 2024-04-04T06:18:41.954 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65099.log 2024-04-04T06:18:41.954 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57406.log.gz 2024-04-04T06:18:41.955 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.45876.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45876.log.gz 2024-04-04T06:18:41.955 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64630.log 2024-04-04T06:18:41.956 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.65099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56218.log 2024-04-04T06:18:41.957 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64630.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.65099.log.gz 2024-04-04T06:18:41.958 INFO:teuthology.orchestra.run.smithi159.stderr: 89.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.64630.log.gz 2024-04-04T06:18:41.958 INFO:teuthology.orchestra.run.smithi159.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.66765.log 2024-04-04T06:18:41.959 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.56218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48431.log 2024-04-04T06:18:41.959 INFO:teuthology.orchestra.run.smithi159.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56218.log.gz 2024-04-04T06:18:41.960 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.66765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59998.log 2024-04-04T06:18:41.960 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66765.log.gz 2024-04-04T06:18:41.961 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52105.log 2024-04-04T06:18:41.961 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48431.log.gz 2024-04-04T06:18:41.962 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.59998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55915.log 2024-04-04T06:18:41.962 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59998.log.gz 2024-04-04T06:18:41.963 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52105.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52105.log.gz 2024-04-04T06:18:41.963 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69218.log 2024-04-04T06:18:41.965 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.55915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75849.log 2024-04-04T06:18:41.965 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55915.log.gz 2024-04-04T06:18:41.965 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.69218.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69218.log.gz 2024-04-04T06:18:41.966 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-04T06:18:41.967 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.75849.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75849.log.gz 2024-04-04T06:18:41.967 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71312.log 2024-04-04T06:18:41.968 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75695.log 2024-04-04T06:18:41.969 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.71312.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71312.log.gz 2024-04-04T06:18:41.969 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47091.log 2024-04-04T06:18:41.970 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.75695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69353.log 2024-04-04T06:18:41.971 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75695.log.gz 2024-04-04T06:18:41.972 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47091.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47091.log.gz 2024-04-04T06:18:41.972 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72728.log 2024-04-04T06:18:41.973 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.69353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45617.log 2024-04-04T06:18:41.973 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4%/var/log/ceph/ceph-client.admin.72728.log: -- replaced with /var/log/ceph/ceph-client.admin.69353.log.gz 2024-04-04T06:18:41.974 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 89.1% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.72728.log.gz /var/log/ceph/ceph-client.admin.48309.log 2024-04-04T06:18:41.974 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-04-04T06:18:41.975 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.2.45617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54680.log 2024-04-04T06:18:41.975 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45617.log.gz 2024-04-04T06:18:41.976 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48309.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60128.log 2024-04-04T06:18:41.976 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48309.log.gz 2024-04-04T06:18:41.977 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70358.log 2024-04-04T06:18:41.977 INFO:teuthology.orchestra.run.smithi159.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54680.log.gz 2024-04-04T06:18:41.978 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.60128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79559.log 2024-04-04T06:18:41.979 INFO:teuthology.orchestra.run.smithi159.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.60128.log.gz 2024-04-04T06:18:41.979 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.70358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47349.log 2024-04-04T06:18:41.980 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70358.log.gz 2024-04-04T06:18:41.981 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.79559.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79559.log.gz 2024-04-04T06:18:41.981 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70687.log 2024-04-04T06:18:41.983 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51840.log 2024-04-04T06:18:41.983 INFO:teuthology.orchestra.run.smithi159.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47349.log.gz 2024-04-04T06:18:41.983 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.70687.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70687.log.gz 2024-04-04T06:18:41.984 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55199.log 2024-04-04T06:18:41.985 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.51840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69088.log 2024-04-04T06:18:41.985 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51840.log.gz 2024-04-04T06:18:41.986 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.55199.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55199.log.gz 2024-04-04T06:18:41.986 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71483.log 2024-04-04T06:18:41.987 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.69088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63300.log 2024-04-04T06:18:41.987 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69088.log.gz 2024-04-04T06:18:41.988 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.71483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42344.log 2024-04-04T06:18:41.989 INFO:teuthology.orchestra.run.smithi159.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.71483.log.gz 2024-04-04T06:18:41.989 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42932.log 2024-04-04T06:18:41.989 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63300.log.gz 2024-04-04T06:18:41.991 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58809.log 2024-04-04T06:18:41.991 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42344.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42344.log.gz 2024-04-04T06:18:41.991 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68224.log 2024-04-04T06:18:41.991 INFO:teuthology.orchestra.run.smithi159.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42932.log.gz 2024-04-04T06:18:41.992 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49207.log 2024-04-04T06:18:41.993 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58809.log.gz 2024-04-04T06:18:41.993 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.68224.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68224.log.gz 2024-04-04T06:18:41.994 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67207.log 2024-04-04T06:18:41.995 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.49207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68014.log 2024-04-04T06:18:41.995 INFO:teuthology.orchestra.run.smithi159.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49207.log.gz 2024-04-04T06:18:41.996 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.67207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75388.log 2024-04-04T06:18:41.996 INFO:teuthology.orchestra.run.smithi159.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.67207.log.gz 2024-04-04T06:18:41.997 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.68014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75392.log 2024-04-04T06:18:41.998 INFO:teuthology.orchestra.run.smithi178.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-04T06:18:41.998 INFO:teuthology.orchestra.run.smithi159.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68014.log.gz 2024-04-04T06:18:41.998 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.75388.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75388.log.gz 2024-04-04T06:18:41.998 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51373.log 2024-04-04T06:18:41.999 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72236.log 2024-04-04T06:18:42.000 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.75392.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75392.log.gz 2024-04-04T06:18:42.000 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.51373.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51373.log.gz 2024-04-04T06:18:42.000 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45792.log 2024-04-04T06:18:42.001 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.72236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69130.log 2024-04-04T06:18:42.002 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72236.log.gz 2024-04-04T06:18:42.003 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.45792.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45792.log.gz 2024-04-04T06:18:42.003 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77940.log 2024-04-04T06:18:42.004 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.69130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47595.log 2024-04-04T06:18:42.005 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.77940.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69130.log.gz 2024-04-04T06:18:42.005 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.77940.log.gz 2024-04-04T06:18:42.005 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69269.log 2024-04-04T06:18:42.006 INFO:teuthology.orchestra.run.smithi159.stderr:gzip/var/log/ceph/ceph-client.admin.47595.log: -5 --verbose -- /var/log/ceph/ceph.log 2024-04-04T06:18:42.006 INFO:teuthology.orchestra.run.smithi159.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47595.log.gz 2024-04-04T06:18:42.007 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.69269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59707.log 2024-04-04T06:18:42.007 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69269.log.gz 2024-04-04T06:18:42.008 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53408.log 2024-04-04T06:18:42.009 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.59707.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59707.log.gz 2024-04-04T06:18:42.010 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73083.log 2024-04-04T06:18:42.011 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53408.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53408.log.gz 2024-04-04T06:18:42.011 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50157.log 2024-04-04T06:18:42.011 INFO:teuthology.orchestra.run.smithi159.stderr: 87.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-04T06:18:42.012 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.73083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62269.log 2024-04-04T06:18:42.013 INFO:teuthology.orchestra.run.smithi159.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73083.log.gz 2024-04-04T06:18:42.013 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.50157.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-04T06:18:42.014 INFO:teuthology.orchestra.run.smithi159.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50157.log.gz 2024-04-04T06:18:42.014 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65777.log 2024-04-04T06:18:42.015 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62269.log.gz 2024-04-04T06:18:42.015 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62479.log 2024-04-04T06:18:42.016 INFO:teuthology.orchestra.run.smithi178.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-04T06:18:42.017 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.65777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74324.log 2024-04-04T06:18:42.017 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65777.log.gz 2024-04-04T06:18:42.018 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47679.log 2024-04-04T06:18:42.018 INFO:teuthology.orchestra.run.smithi159.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62479.log.gz 2024-04-04T06:18:42.020 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.74324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61376.log 2024-04-04T06:18:42.020 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74324.log.gz 2024-04-04T06:18:42.022 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68368.log 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47679.log.gz 2024-04-04T06:18:42.022 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-04-04T06:18:42.022 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61376.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61376.log.gz 2024-04-04T06:18:42.023 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70280.log 2024-04-04T06:18:42.024 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.68368.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-04T06:18:42.024 INFO:teuthology.orchestra.run.smithi159.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68368.log.gz 2024-04-04T06:18:42.025 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.70280.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70280.log.gz 2024-04-04T06:18:42.026 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49759.log 2024-04-04T06:18:42.027 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47889.log 2024-04-04T06:18:42.028 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.49759.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49759.log.gz 2024-04-04T06:18:42.029 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79391.log 2024-04-04T06:18:42.029 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47889.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47889.log.gz 2024-04-04T06:18:42.030 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43492.log 2024-04-04T06:18:42.032 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.79391.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79391.log.gz 2024-04-04T06:18:42.033 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.43492.log: -- /var/log/ceph/ceph-client.admin.74156.log 2024-04-04T06:18:42.033 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43492.log.gz 2024-04-04T06:18:42.034 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58628.log 2024-04-04T06:18:42.036 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.74156.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74156.log.gz 2024-04-04T06:18:42.036 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45960.log 2024-04-04T06:18:42.037 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58628.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58628.log.gz 2024-04-04T06:18:42.038 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57574.log 2024-04-04T06:18:42.039 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.45960.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45960.log.gz 2024-04-04T06:18:42.040 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73508.log/var/log/ceph/ceph-client.admin.57574.log: 2024-04-04T06:18:42.041 INFO:teuthology.orchestra.run.smithi159.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57574.log.gz 2024-04-04T06:18:42.042 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70420.log 2024-04-04T06:18:42.043 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.73508.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73508.log.gz 2024-04-04T06:18:42.044 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42722.log 2024-04-04T06:18:42.044 INFO:teuthology.orchestra.run.smithi178.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-04T06:18:42.045 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.70420.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70420.log.gz 2024-04-04T06:18:42.046 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58721.log 2024-04-04T06:18:42.047 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42722.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42722.log.gz 2024-04-04T06:18:42.048 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65711.log 2024-04-04T06:18:42.049 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58721.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.58721.log.gz 2024-04-04T06:18:42.050 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55919.log 2024-04-04T06:18:42.052 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.65711.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.65711.log.gz 2024-04-04T06:18:42.052 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64708.log/var/log/ceph/ceph-client.0.55919.log: 2024-04-04T06:18:42.052 INFO:teuthology.orchestra.run.smithi159.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55919.log.gz 2024-04-04T06:18:42.053 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76002.log 2024-04-04T06:18:42.055 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.64708.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64708.log.gz 2024-04-04T06:18:42.056 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42214.log 2024-04-04T06:18:42.056 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.76002.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76002.log.gz 2024-04-04T06:18:42.057 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51145.log 2024-04-04T06:18:42.059 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42214.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42214.log.gz 2024-04-04T06:18:42.059 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.51145.log: /var/log/ceph/ceph-client.admin.52063.log 2024-04-04T06:18:42.060 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51145.log.gz 2024-04-04T06:18:42.061 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42300.log 2024-04-04T06:18:42.062 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52063.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52063.log.gz 2024-04-04T06:18:42.063 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71998.log 2024-04-04T06:18:42.064 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.42300.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42300.log.gz 2024-04-04T06:18:42.066 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.71998.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.71998.log.gz 2024-04-04T06:18:42.066 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69483.log 2024-04-04T06:18:42.068 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57018.log 2024-04-04T06:18:42.069 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.69483.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69483.log.gz 2024-04-04T06:18:42.070 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78803.log 2024-04-04T06:18:42.070 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57018.log.gz 2024-04-04T06:18:42.071 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46971.log 2024-04-04T06:18:42.073 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.78803.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78803.log.gz 2024-04-04T06:18:42.073 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79601.log 2024-04-04T06:18:42.074 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46971.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46971.log.gz 2024-04-04T06:18:42.075 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53940.log 2024-04-04T06:18:42.076 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.79601.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79601.log.gz 2024-04-04T06:18:42.077 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46338.log 2024-04-04T06:18:42.078 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53940.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53940.log.gz 2024-04-04T06:18:42.079 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43753.log 2024-04-04T06:18:42.081 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46338.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46338.log.gz 2024-04-04T06:18:42.081 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48639.log/var/log/ceph/ceph-client.admin.43753.log: 2024-04-04T06:18:42.081 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43753.log.gz 2024-04-04T06:18:42.082 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78373.log 2024-04-04T06:18:42.083 INFO:teuthology.orchestra.run.smithi178.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-04T06:18:42.084 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48639.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48639.log.gz 2024-04-04T06:18:42.084 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62227.log 2024-04-04T06:18:42.085 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.78373.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78373.log.gz 2024-04-04T06:18:42.086 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49459.log 2024-04-04T06:18:42.087 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-04-04T06:18:42.087 INFO:teuthology.orchestra.run.smithi178.stderr:real 0m0.782s 2024-04-04T06:18:42.087 INFO:teuthology.orchestra.run.smithi178.stderr:user 0m3.608s 2024-04-04T06:18:42.087 INFO:teuthology.orchestra.run.smithi178.stderr:sys 0m0.421s 2024-04-04T06:18:42.088 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62227.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62227.log.gz 2024-04-04T06:18:42.088 INFO:teuthology.orchestra.run.smithi159.stderr:gzip/var/log/ceph/ceph-client.admin.49459.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.78415.log 2024-04-04T06:18:42.091 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49459.log.gz 2024-04-04T06:18:42.091 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61226.log 2024-04-04T06:18:42.091 INFO:teuthology.orchestra.run.smithi159.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-04T06:18:42.091 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.78415.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.78415.log.gz 2024-04-04T06:18:42.092 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46506.log 2024-04-04T06:18:42.093 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50073.log 2024-04-04T06:18:42.093 INFO:teuthology.orchestra.run.smithi159.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.61226.log.gz 2024-04-04T06:18:42.098 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77503.log 2024-04-04T06:18:42.098 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46506.log.gz 2024-04-04T06:18:42.098 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.50073.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50073.log.gz 2024-04-04T06:18:42.098 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54463.log 2024-04-04T06:18:42.098 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.77503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41964.log 2024-04-04T06:18:42.098 INFO:teuthology.orchestra.run.smithi159.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77503.log.gz 2024-04-04T06:18:42.099 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54463.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54463.log.gz 2024-04-04T06:18:42.099 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.41964.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41964.log.gz 2024-04-04T06:18:42.100 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43151.log 2024-04-04T06:18:42.101 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71872.log 2024-04-04T06:18:42.103 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51415.log 2024-04-04T06:18:42.103 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43151.log.gz 2024-04-04T06:18:42.104 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.71872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80189.log 2024-04-04T06:18:42.104 INFO:teuthology.orchestra.run.smithi159.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.71872.log.gz 2024-04-04T06:18:42.105 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.51415.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51415.log.gz 2024-04-04T06:18:42.106 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46128.log 2024-04-04T06:18:42.106 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.80189.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80189.log.gz 2024-04-04T06:18:42.107 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48225.log 2024-04-04T06:18:42.108 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43786.log 2024-04-04T06:18:42.108 INFO:teuthology.orchestra.run.smithi159.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46128.log.gz 2024-04-04T06:18:42.110 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48225.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.48225.log.gz 2024-04-04T06:18:42.110 INFO:teuthology.orchestra.run.smithi159.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52361.log 2024-04-04T06:18:42.111 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43786.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43786.log.gz 2024-04-04T06:18:42.112 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55018.log 2024-04-04T06:18:42.113 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.52361.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52361.log.gz 2024-04-04T06:18:42.113 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57887.log 2024-04-04T06:18:42.115 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.55018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77301.log 2024-04-04T06:18:42.115 INFO:teuthology.orchestra.run.smithi159.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55018.log.gz 2024-04-04T06:18:42.116 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63515.log 2024-04-04T06:18:42.116 INFO:teuthology.orchestra.run.smithi159.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57887.log.gz 2024-04-04T06:18:42.117 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.77301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68621.log 2024-04-04T06:18:42.118 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63515.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.77301.log.gz 2024-04-04T06:18:42.119 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63515.log.gz 2024-04-04T06:18:42.119 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51756.log 2024-04-04T06:18:42.120 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.68621.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68621.log.gz 2024-04-04T06:18:42.120 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68140.log 2024-04-04T06:18:42.122 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.51756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48267.log 2024-04-04T06:18:42.122 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51756.log.gz 2024-04-04T06:18:42.123 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.68140.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68140.log.gz 2024-04-04T06:18:42.123 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61544.log 2024-04-04T06:18:42.124 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48267.log.gz 2024-04-04T06:18:42.125 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67465.log 2024-04-04T06:18:42.126 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68705.log 2024-04-04T06:18:42.126 INFO:teuthology.orchestra.run.smithi159.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61544.log.gz 2024-04-04T06:18:42.128 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.67465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46734.log 2024-04-04T06:18:42.128 INFO:teuthology.orchestra.run.smithi159.stderr: 91.1%/var/log/ceph/ceph-client.admin.68705.log: -- replaced with /var/log/ceph/ceph-client.admin.67465.log.gz 2024-04-04T06:18:42.129 INFO:teuthology.orchestra.run.smithi159.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68705.log.gz 2024-04-04T06:18:42.129 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48057.log 2024-04-04T06:18:42.131 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.46734.log: gzip -5 --verbose -- 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46734.log.gz 2024-04-04T06:18:42.131 INFO:teuthology.orchestra.run.smithi159.stderr: /var/log/ceph/ceph-client.admin.57280.log 2024-04-04T06:18:42.132 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.48057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45157.log 2024-04-04T06:18:42.132 INFO:teuthology.orchestra.run.smithi159.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48057.log.gz 2024-04-04T06:18:42.134 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.57280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75014.log 2024-04-04T06:18:42.134 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57280.log.gz 2024-04-04T06:18:42.135 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.45157.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45157.log.gz 2024-04-04T06:18:42.135 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61160.log 2024-04-04T06:18:42.136 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.75014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45309.log 2024-04-04T06:18:42.137 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75014.log.gz 2024-04-04T06:18:42.138 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.61160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47391.log 2024-04-04T06:18:42.138 INFO:teuthology.orchestra.run.smithi159.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.61160.log.gz 2024-04-04T06:18:42.139 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.45309.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45309.log.gz 2024-04-04T06:18:42.140 INFO:teuthology.orchestra.run.smithi159.stderr:gzip/var/log/ceph/ceph-client.admin.47391.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.73167.log 2024-04-04T06:18:42.140 INFO:teuthology.orchestra.run.smithi159.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47391.log.gz 2024-04-04T06:18:42.141 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58097.log 2024-04-04T06:18:42.143 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.73167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74870.log 2024-04-04T06:18:42.143 INFO:teuthology.orchestra.run.smithi159.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73167.log.gz 2024-04-04T06:18:42.144 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.58097.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58097.log.gz 2024-04-04T06:18:42.145 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55115.log 2024-04-04T06:18:42.145 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.74870.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74870.log.gz 2024-04-04T06:18:42.146 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53283.log 2024-04-04T06:18:42.147 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.55115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63738.log 2024-04-04T06:18:42.147 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55115.log.gz 2024-04-04T06:18:42.149 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.53283.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.53283.log.gz 2024-04-04T06:18:42.150 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.63738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43491.log 2024-04-04T06:18:42.150 INFO:teuthology.orchestra.run.smithi159.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63738.log.gz 2024-04-04T06:18:42.151 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73551.log 2024-04-04T06:18:42.152 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62521.log 2024-04-04T06:18:42.153 INFO:teuthology.orchestra.run.smithi159.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43491.log.gz 2024-04-04T06:18:42.154 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.73551.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73551.log.gz -5 2024-04-04T06:18:42.154 INFO:teuthology.orchestra.run.smithi159.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49291.log 2024-04-04T06:18:42.155 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.62521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45176.log 2024-04-04T06:18:42.155 INFO:teuthology.orchestra.run.smithi159.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62521.log.gz 2024-04-04T06:18:42.157 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.49291.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-04T06:18:42.157 INFO:teuthology.orchestra.run.smithi159.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49291.log.gz 2024-04-04T06:18:42.157 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.45176.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45176.log.gz 2024-04-04T06:18:42.158 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67794.log 2024-04-04T06:18:42.159 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73305.log 2024-04-04T06:18:42.161 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.67794.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67794.log.gz 2024-04-04T06:18:42.162 INFO:teuthology.orchestra.run.smithi159.stderr:gzip/var/log/ceph/ceph-client.admin.73305.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.55493.log 2024-04-04T06:18:42.162 INFO:teuthology.orchestra.run.smithi159.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73305.log.gz 2024-04-04T06:18:42.163 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74030.log 2024-04-04T06:18:42.165 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.55493.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55493.log.gz 2024-04-04T06:18:42.165 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47013.log 2024-04-04T06:18:42.166 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.74030.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74030.log.gz 2024-04-04T06:18:42.167 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.47013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43483.log 2024-04-04T06:18:42.168 INFO:teuthology.orchestra.run.smithi159.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47013.log.gz 2024-04-04T06:18:42.169 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77944.log 2024-04-04T06:18:42.170 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.43483.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43483.log.gz 2024-04-04T06:18:42.171 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54168.log 2024-04-04T06:18:42.171 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.77944.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.77944.log.gz 2024-04-04T06:18:42.172 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65357.log 2024-04-04T06:18:42.174 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.admin.54168.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54168.log.gz 2024-04-04T06:18:42.175 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/ceph-client.0.65357.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65357.log.gz 2024-04-04T06:18:42.234 INFO:teuthology.orchestra.run.smithi159.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-04T06:18:42.450 INFO:teuthology.orchestra.run.smithi159.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-04T06:18:42.482 INFO:teuthology.orchestra.run.smithi159.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-04T06:18:42.497 INFO:teuthology.orchestra.run.smithi159.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-04T06:18:42.585 INFO:teuthology.orchestra.run.smithi159.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-04T06:18:42.797 INFO:teuthology.orchestra.run.smithi159.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-04T06:18:42.802 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-04-04T06:18:42.802 INFO:teuthology.orchestra.run.smithi159.stderr:real 0m1.502s 2024-04-04T06:18:42.802 INFO:teuthology.orchestra.run.smithi159.stderr:user 0m4.060s 2024-04-04T06:18:42.802 INFO:teuthology.orchestra.run.smithi159.stderr:sys 0m2.108s 2024-04-04T06:18:42.803 INFO:tasks.ceph:Archiving logs... 2024-04-04T06:18:42.803 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-04-04_05:15:36-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi/7639210/remote/smithi159/log 2024-04-04T06:18:42.804 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-04T06:18:44.078 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-04-04_05:15:36-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi/7639210/remote/smithi178/log 2024-04-04T06:18:44.079 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-04T06:18:44.639 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-04T06:18:44.649 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-04T06:18:44.649 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-04T06:18:44.657 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-04T06:18:44.810 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-04T06:18:44.820 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-04T06:18:44.820 DEBUG:teuthology.orchestra.run.smithi159:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-04T06:18:44.828 DEBUG:teuthology.orchestra.run.smithi178:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-04T06:18:44.875 INFO:teuthology.orchestra.run.smithi159.stderr:bash: line 1: ntpq: command not found 2024-04-04T06:18:44.881 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-04-04T06:18:44.885 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-04T06:18:44.885 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2024-04-04T06:18:44.885 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 7 -357us[ -357us] +/- 84ms 2024-04-04T06:18:44.885 INFO:teuthology.orchestra.run.smithi159.stdout:^* hv02.front.sepia.ceph.com 3 6 377 8 -740us[ -724us] +/- 51ms 2024-04-04T06:18:44.886 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 +1679us[+1679us] +/- 91ms 2024-04-04T06:18:44.886 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-04T06:18:44.890 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-04T06:18:44.890 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-04-04T06:18:44.890 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 6 -373us[ -342us] +/- 84ms 2024-04-04T06:18:44.890 INFO:teuthology.orchestra.run.smithi178.stdout:^* hv02.front.sepia.ceph.com 3 6 377 4 -749us[ -717us] +/- 51ms 2024-04-04T06:18:44.890 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +1690us[+1721us] +/- 91ms 2024-04-04T06:18:44.890 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-04T06:18:44.892 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-04T06:18:44.901 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-04T06:18:44.902 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-04T06:18:44.911 DEBUG:teuthology.orchestra.run.smithi159:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-04T06:18:45.153 DEBUG:teuthology.orchestra.run.smithi178:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-04T06:18:45.339 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-04T06:18:45.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-04T06:18:45.446 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-04T06:18:45.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-04T06:18:45.548 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-04T06:18:45.558 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-04T06:18:45.578 INFO:teuthology.task.internal:Duration was 1348.608727 seconds 2024-04-04T06:18:45.578 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-04T06:18:45.588 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-04T06:18:45.588 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-04T06:18:45.596 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-04T06:18:45.755 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-04T06:18:45.760 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-04T06:18:46.210 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-04T06:18:46.210 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi159.front.sepia.ceph.com 2024-04-04T06:18:46.210 DEBUG:teuthology.orchestra.run.smithi159:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-04T06:18:46.275 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi178.front.sepia.ceph.com 2024-04-04T06:18:46.275 DEBUG:teuthology.orchestra.run.smithi178:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-04T06:18:46.354 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-04T06:18:46.355 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-04T06:18:46.362 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-04T06:18:46.454 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-04T06:18:46.454 DEBUG:teuthology.orchestra.run.smithi159:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-04T06:18:46.459 DEBUG:teuthology.orchestra.run.smithi178:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-04T06:18:46.662 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-04T06:18:46.672 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-04T06:18:46.672 DEBUG:teuthology.orchestra.run.smithi159:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-04T06:18:46.769 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-04T06:18:46.869 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-04T06:18:46.881 DEBUG:teuthology.orchestra.run.smithi159:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-04T06:18:46.889 DEBUG:teuthology.orchestra.run.smithi178:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-04T06:18:46.993 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = core 2024-04-04T06:18:46.995 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = core 2024-04-04T06:18:47.050 DEBUG:teuthology.orchestra.run.smithi159:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-04T06:18:47.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-04T06:18:47.094 DEBUG:teuthology.orchestra.run.smithi178:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-04T06:18:47.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-04T06:18:47.146 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-04T06:18:47.156 INFO:teuthology.task.internal:Transferring archived files... 2024-04-04T06:18:47.156 DEBUG:teuthology.misc:Transferring archived files from smithi159:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-04-04_05:15:36-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi/7639210/remote/smithi159 2024-04-04T06:18:47.157 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-04T06:18:47.264 DEBUG:teuthology.misc:Transferring archived files from smithi178:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-04-04_05:15:36-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi/7639210/remote/smithi178 2024-04-04T06:18:47.264 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-04T06:18:47.363 INFO:teuthology.task.internal:Removing archive directory... 2024-04-04T06:18:47.363 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-04T06:18:47.371 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-04T06:18:47.424 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-04T06:18:47.435 INFO:teuthology.task.internal:Not uploading archives. 2024-04-04T06:18:47.435 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-04T06:18:47.445 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-04T06:18:47.445 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-04T06:18:47.452 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-04T06:18:47.500 INFO:teuthology.orchestra.run.smithi159.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 4 06:18 /home/ubuntu/cephtest 2024-04-04T06:18:47.507 INFO:teuthology.orchestra.run.smithi178.stdout: 393945 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 4 06:18 /home/ubuntu/cephtest 2024-04-04T06:18:47.510 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-04T06:18:47.520 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-04T06:18:47.541 INFO:teuthology.nuke:Checking targets against current locks 2024-04-04T06:18:47.565 DEBUG:teuthology.nuke:shortname: smithi159 2024-04-04T06:18:47.566 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-04T06:18:47.592 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-04_05:15:36-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi/7639210', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-04 05:46:31.995958', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI9+sH1zuQF/EKfdRkBUOY6pUkyed9HhnOnkKrOygkBuTyZsrgsl2SJcEyeid/oNbaQwHgtR9HSF5IzgfoYOAgM='} 2024-04-04T06:18:47.595 DEBUG:teuthology.nuke:shortname: smithi178 2024-04-04T06:18:47.596 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-04T06:18:47.617 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-04-04_05:15:36-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi/7639210', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-04 05:46:31.994384', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI75ZPTr++WrvmTssgpUy8yMeTqSpAYeKCfe+sxpb8CHITh35dNVY8L6Ku/faFCk6orumF6zBTI9iJ1DATOBKxI='} 2024-04-04T06:18:47.651 INFO:teuthology.orchestra.console.smithi159:Power off 2024-04-04T06:18:47.652 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-04T06:18:47.681 DEBUG:teuthology.orchestra.console.smithi159:power off output: Chassis Power Control: Down/Off 2024-04-04T06:18:47.682 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-04T06:18:47.699 INFO:teuthology.orchestra.console.smithi178:Power off 2024-04-04T06:18:47.699 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-04T06:18:47.709 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2024-04-04T06:18:47.721 DEBUG:teuthology.orchestra.console.smithi178:power off output: Chassis Power Control: Down/Off 2024-04-04T06:18:47.721 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-04T06:18:47.743 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on 2024-04-04T06:18:51.710 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-04T06:18:51.744 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-04T06:18:51.821 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2024-04-04T06:18:51.854 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on 2024-04-04T06:18:55.822 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-04T06:18:55.855 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-04T06:18:55.933 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off 2024-04-04T06:18:55.965 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off 2024-04-04T06:18:56.035 INFO:teuthology.orchestra.console.smithi159:Power off completed 2024-04-04T06:18:56.066 INFO:teuthology.orchestra.console.smithi178:Power off completed 2024-04-04T06:18:56.171 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 client.1-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} duration: 1348.6087265014648 failure_reason: '"2024-04-04T06:12:57.115301+0000 mds.a (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir)" in cluster log' flavor: default owner: scheduled_vshankar@teuthology success: false 2024-04-04T06:18:56.171 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-04T06:18:56.256 INFO:teuthology.run:FAIL